2024-04-25T09:46:28.366 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T09:46:28.366 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_neesingh@teuthology --verbose --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033 --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033/orig.config.yaml 2024-04-25T09:46:28.415 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:46:28.503 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673033' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v2 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKaviYy5ug0s6Xg9RhNtgSu65eUWVy5bOV1NRmbSw/69Fn70W+VhtzEqIEnUu6woMugLhFX4JpgsdloSm/KJ71Y= smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAa/e7AreuiaIKjOsSkwgKV2Lo7i9LPQwl+/icoFA5SlnlNewaexHVsLLXQC4wozaL4KLVcHX/h/MrVg5sNJfvM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T09:46:28.503 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T09:46:28.504 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T09:46:28.504 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T09:46:28.506 INFO:teuthology.task.internal:Checking packages... 2024-04-25T09:46:28.531 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T09:46:28.531 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T09:46:28.532 INFO:teuthology.packaging:ref: None 2024-04-25T09:46:28.532 INFO:teuthology.packaging:tag: None 2024-04-25T09:46:28.532 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T09:46:28.532 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:46:28.532 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T09:46:28.675 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628.g97ae6cc9 2024-04-25T09:46:28.676 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T09:46:28.683 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T09:46:28.684 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T09:46:28.702 INFO:teuthology.task.internal:Saving configuration 2024-04-25T09:46:28.719 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T09:46:28.725 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:46:28.750 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:41:28.980315', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKaviYy5ug0s6Xg9RhNtgSu65eUWVy5bOV1NRmbSw/69Fn70W+VhtzEqIEnUu6woMugLhFX4JpgsdloSm/KJ71Y='} 2024-04-25T09:46:28.774 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:41:28.982220', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAa/e7AreuiaIKjOsSkwgKV2Lo7i9LPQwl+/icoFA5SlnlNewaexHVsLLXQC4wozaL4KLVcHX/h/MrVg5sNJfvM='} 2024-04-25T09:46:28.774 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T09:46:28.781 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T09:46:28.781 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T09:46:28.781 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T09:46:28.859 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f28654cb160>, signals=[15]) 2024-04-25T09:46:28.860 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T09:46:28.866 INFO:teuthology.task.internal:Opening connections... 2024-04-25T09:46:28.866 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:46:28.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:46:28.944 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T09:46:28.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:46:29.019 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T09:46:29.026 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-04-25T09:46:29.048 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-04-25T09:46:29.048 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-04-25T09:46:29.107 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-04-25T09:46:29.107 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-04-25T09:46:29.107 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-04-25T09:46:29.107 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:46:29.107 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-04-25T09:46:29.107 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:46:29.107 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:46:29.107 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-04-25T09:46:29.107 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:46:29.107 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:46:29.107 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:46:29.108 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:46:29.108 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:46:29.108 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:46:29.108 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-04-25T09:46:29.134 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-04-25T09:46:29.152 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-04-25T09:46:29.152 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-04-25T09:46:29.208 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:46:29.209 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:46:29.210 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:46:29.210 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-04-25T09:46:29.233 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T09:46:29.243 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T09:46:29.248 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T09:46:29.249 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:46:29.251 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:46:29.267 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T09:46:29.273 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T09:46:29.274 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:46:29.309 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:46:29.557 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T09:46:29.607 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T09:46:29.622 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T09:46:29.622 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-25T09:46:29.622 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-25T09:46:29.622 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:46:29.622 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:46:29.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:46:29.639 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-04-25T09:46:29.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:46:29.640 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-04-25T09:46:29.696 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:46:29.696 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-437.el9.x86_64 2024-04-25T09:46:29.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2024-04-25T09:46:29.698 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:46:29.698 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-437.el9.x86_64 2024-04-25T09:46:29.698 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-04-25T09:46:32.165 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-04-25T09:46:32.779 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.0 MB 00:01 2024-04-25T09:46:34.476 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-25T09:46:38.677 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T09:46:38.797 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 19 MB 00:04 2024-04-25T09:46:40.119 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-04-25T09:46:41.237 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-04-25T09:46:43.048 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T09:46:44.573 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-04-25T09:46:45.869 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T09:46:46.337 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-25T09:46:48.218 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T09:46:48.218 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T09:46:48.297 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T09:46:48.298 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-04-25T09:46:48.298 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T09:46:48.392 DEBUG:teuthology.orchestra.run.smithi112:> echo no | sudo yum reinstall kernel || true 2024-04-25T09:46:48.889 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 09:46:46 AM UTC. 2024-04-25T09:46:49.008 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T09:46:49.016 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T09:46:49.016 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-04-25T09:46:49.016 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T09:46:49.016 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalling: 2024-04-25T09:46:49.016 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:46:49.016 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:46:49.016 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T09:46:49.017 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T09:46:49.017 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:46:49.017 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 255 k 2024-04-25T09:46:49.017 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 0 2024-04-25T09:46:49.017 INFO:teuthology.orchestra.run.smithi112.stderr:Operation aborted. 2024-04-25T09:46:49.075 INFO:teuthology.orchestra.run.smithi112.stdout:Is this ok [y/N]: 2024-04-25T09:46:49.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum reinstall -y kernel || true 2024-04-25T09:46:49.569 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 09:46:46 AM UTC. 2024-04-25T09:46:49.685 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T09:46:49.686 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T09:46:49.686 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-04-25T09:46:49.686 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T09:46:49.686 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalling: 2024-04-25T09:46:49.687 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:46:49.687 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:46:49.687 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T09:46:49.687 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T09:46:49.687 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:46:49.687 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 255 k 2024-04-25T09:46:49.687 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 0 2024-04-25T09:46:49.687 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-25T09:46:50.523 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-437.el9.x86_64.rpm 454 kB/s | 255 kB 00:00 2024-04-25T09:46:50.523 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:46:50.523 INFO:teuthology.orchestra.run.smithi112.stdout:Total 305 kB/s | 255 kB 00:00 2024-04-25T09:46:50.539 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-25T09:46:50.732 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-25T09:46:50.732 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-25T09:46:50.737 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-25T09:46:50.737 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-25T09:46:50.881 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-25T09:46:50.958 INFO:teuthology.orchestra.run.smithi112.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:46:50.958 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:51.191 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-25T09:46:51.616 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:51.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:46:52.291 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:52.291 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:46:52.291 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalled: 2024-04-25T09:46:52.291 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T09:46:52.291 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:46:52.291 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T09:46:52.504 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T09:46:52.551 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T09:46:52.551 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T09:46:52.551 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T09:46:52.552 DEBUG:teuthology.parallel:result is None 2024-04-25T09:46:53.206 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T09:46:53.206 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T09:46:53.308 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T09:46:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-04-25T09:46:53.310 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T09:46:53.412 DEBUG:teuthology.orchestra.run.smithi019:> echo no | sudo yum reinstall kernel || true 2024-04-25T09:46:53.974 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 09:46:51 AM UTC. 2024-04-25T09:46:54.104 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T09:46:54.112 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T09:46:54.112 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-04-25T09:46:54.112 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T09:46:54.112 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalling: 2024-04-25T09:46:54.112 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:46:54.112 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:46:54.112 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-25T09:46:54.113 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T09:46:54.113 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:46:54.113 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 255 k 2024-04-25T09:46:54.113 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 0 2024-04-25T09:46:54.114 INFO:teuthology.orchestra.run.smithi019.stderr:Operation aborted. 2024-04-25T09:46:54.171 INFO:teuthology.orchestra.run.smithi019.stdout:Is this ok [y/N]: 2024-04-25T09:46:54.171 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum reinstall -y kernel || true 2024-04-25T09:46:54.691 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 09:46:51 AM UTC. 2024-04-25T09:46:54.812 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T09:46:54.813 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T09:46:54.813 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-04-25T09:46:54.813 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T09:46:54.813 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalling: 2024-04-25T09:46:54.813 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:46:54.813 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:46:54.813 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-25T09:46:54.813 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T09:46:54.813 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:46:54.813 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 255 k 2024-04-25T09:46:54.814 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 0 2024-04-25T09:46:54.814 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-04-25T09:46:55.027 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.7 MB/s | 255 kB 00:00 2024-04-25T09:46:55.027 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:46:55.027 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.2 MB/s | 255 kB 00:00 2024-04-25T09:46:55.047 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-04-25T09:46:55.231 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-04-25T09:46:55.231 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-04-25T09:46:55.236 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-04-25T09:46:55.236 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-04-25T09:46:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-04-25T09:46:55.452 INFO:teuthology.orchestra.run.smithi019.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:46:55.453 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:56.167 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:56.167 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:46:56.859 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:56.859 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:46:56.859 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalled: 2024-04-25T09:46:56.859 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T09:46:56.860 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:46:56.860 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T09:46:57.042 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T09:46:57.089 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T09:46:57.090 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T09:46:57.090 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T09:46:57.090 DEBUG:teuthology.parallel:result is None 2024-04-25T09:46:57.091 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T09:46:57.103 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T09:46:57.104 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:46:57.107 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:46:57.125 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T09:46:57.132 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T09:46:57.139 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T09:46:57.139 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:46:57.162 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:46:57.197 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T09:46:57.204 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T09:46:57.205 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:46:57.227 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:46:57.268 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:46:57.272 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:46:57.282 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:46:57.289 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:46:57.291 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T09:46:57.298 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T09:46:57.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:46:57.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:46:57.350 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T09:46:57.359 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T09:46:57.360 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:46:57.365 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:46:57.408 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:46:57.460 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:46:57.508 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:46:57.589 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:46:57.629 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:46:57.629 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:46:57.693 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:46:57.737 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:46:57.791 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:46:57.875 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:46:57.916 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:46:57.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:46:57.982 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-04-25T09:46:57.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-04-25T09:46:58.026 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:46:58.055 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:46:58.443 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T09:46:58.451 INFO:teuthology.task.internal:Starting timer... 2024-04-25T09:46:58.452 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T09:46:58.472 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T09:46:58.481 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-04-25T09:46:58.546 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-04-25T09:46:58.548 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-04-25T09:46:58.597 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-04-25T09:46:58.599 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T09:46:58.599 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-04-25T09:46:58.618 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-04-25T09:46:58.619 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-04-25T09:46:58.640 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-04-25T09:46:58.641 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi112.front.sepia.ceph.com': 'permissive'} 2024-04-25T09:46:58.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:46:58.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:46:58.684 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:46:58.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:46:58.710 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T09:46:58.710 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:46:58.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:46:58.774 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T09:46:58.783 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T09:46:58.800 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T09:46:58.802 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi112.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T09:50:30.531 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi112.front.sepia.ceph.com')] 2024-04-25T09:50:30.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T09:50:30.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:50:30.612 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-04-25T09:50:30.690 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T09:50:30.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T09:50:30.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:50:30.766 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-25T09:50:30.849 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T09:50:30.849 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T09:50:30.860 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T09:50:30.860 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:50:30.861 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:50:30.863 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:50:30.864 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:50:30.892 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:50:30.906 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:50:30.916 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:50:30.928 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:50:30.930 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-04-25T09:50:30.941 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-04-25T09:50:30.950 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-04-25T09:50:30.955 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:50:30.961 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-04-25T09:50:30.969 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:50:30.974 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:50:30.986 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:50:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:50:31.030 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:50:31.030 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-25T09:50:31.030 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:50:31.030 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:50:31.030 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:50:31.030 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:50:31.042 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:50:31.045 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:50:31.045 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T09:50:31.045 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:50:31.045 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:50:31.045 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:50:31.045 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:50:31.046 INFO:teuthology.run_tasks:Running task install... 2024-04-25T09:50:31.056 DEBUG:teuthology.task.install:project ceph 2024-04-25T09:50:31.056 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T09:50:31.056 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T09:50:31.056 INFO:teuthology.task.install:Using flavor: default 2024-04-25T09:50:31.062 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T09:50:31.062 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T09:50:31.062 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:50:31.063 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:50:31.065 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:50:31.065 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:50:31.207 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:50:31.207 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:50:31.227 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:50:31.227 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:50:31.330 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:50:31.331 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:50:31.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:50:31.355 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:50:31.355 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:50:31.355 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:50:31.357 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:50:31.357 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:50:31.382 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:50:31.382 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:50:31.423 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:50:31.447 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:50:31.497 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:50:31.520 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:50:31.561 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-04-25T09:50:31.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-04-25T09:50:31.584 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-04-25T09:50:31.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-04-25T09:50:31.797 INFO:teuthology.orchestra.run.smithi019.stdout:48 files removed 2024-04-25T09:50:31.811 INFO:teuthology.orchestra.run.smithi112.stdout:48 files removed 2024-04-25T09:50:31.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:50:31.838 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:50:32.438 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 240 kB/s | 82 kB 00:00 2024-04-25T09:50:32.456 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 227 kB/s | 82 kB 00:00 2024-04-25T09:50:32.764 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 36 kB/s | 10 kB 00:00 2024-04-25T09:50:32.815 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-04-25T09:50:33.098 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 6.3 kB/s | 1.9 kB 00:00 2024-04-25T09:50:33.132 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 6.7 kB/s | 1.9 kB 00:00 2024-04-25T09:50:33.598 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-04-25T09:50:34.224 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.0 MB 00:01 2024-04-25T09:50:35.491 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T09:50:37.075 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-25T09:50:39.734 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.2 MB 00:00 2024-04-25T09:50:41.226 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-25T09:50:41.360 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-25T09:50:42.369 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-25T09:50:42.702 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-25T09:50:43.920 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-25T09:50:47.403 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-04-25T09:50:49.137 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T09:50:49.562 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:50:49.563 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:50:49.574 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:50:49.575 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:50:49.576 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:50:49.576 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:50:49.578 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:50:49.580 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:50:49.581 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:50:49.641 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T09:50:49.650 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T09:50:49.650 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-04-25T09:50:49.650 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T09:50:49.650 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-04-25T09:50:49.650 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:50:49.650 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:50:49.650 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:50:49.650 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:50:49.650 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:50:49.650 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:50:49.651 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:50:49.652 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:50:49.653 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:50:49.654 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:50:49.655 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:50:49.656 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:50:49.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:50:49.658 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:50:49.659 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:50:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:50:49.661 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T09:50:49.662 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T09:50:49.662 INFO:teuthology.orchestra.run.smithi112.stdout:Install 148 Packages 2024-04-25T09:50:49.662 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 4 Packages 2024-04-25T09:50:49.662 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:50:49.662 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 215 M 2024-04-25T09:50:49.662 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-25T09:50:50.829 INFO:teuthology.orchestra.run.smithi112.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-25T09:50:51.271 INFO:teuthology.orchestra.run.smithi112.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 2.0 MB/s | 887 kB 00:00 2024-04-25T09:50:51.364 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:50:51.365 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:50:51.377 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:50:51.377 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:50:51.378 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:50:51.379 INFO:teuthology.orchestra.run.smithi112.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 1.3 MB/s | 145 kB 00:00 2024-04-25T09:50:51.380 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:50:51.381 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:50:51.383 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:50:51.384 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:50:51.449 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T09:50:51.459 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T09:50:51.459 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-04-25T09:50:51.459 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T09:50:51.459 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-04-25T09:50:51.459 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:50:51.459 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:50:51.459 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:50:51.459 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:50:51.459 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:50:51.459 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:50:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:50:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:50:51.462 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:50:51.463 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:50:51.464 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:50:51.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:50:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:50:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:50:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:50:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:50:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:50:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:50:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:50:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:50:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:50:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:50:51.466 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:50:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:50:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:50:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:50:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-04-25T09:50:51.471 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 4 Packages 2024-04-25T09:50:51.472 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:50:51.472 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 215 M 2024-04-25T09:50:51.472 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-04-25T09:50:51.946 INFO:teuthology.orchestra.run.smithi112.stdout:(4/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 3.8 MB/s | 2.1 MB 00:00 2024-04-25T09:50:52.130 INFO:teuthology.orchestra.run.smithi112.stdout:(5/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 3.5 MB/s | 5.3 MB 00:01 2024-04-25T09:50:52.197 INFO:teuthology.orchestra.run.smithi019.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-04-25T09:50:52.197 INFO:teuthology.orchestra.run.smithi112.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 6.0 MB/s | 1.5 MB 00:00 2024-04-25T09:50:52.814 INFO:teuthology.orchestra.run.smithi112.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 6.9 MB/s | 4.7 MB 00:00 2024-04-25T09:50:53.030 INFO:teuthology.orchestra.run.smithi019.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.0 MB/s | 887 kB 00:00 2024-04-25T09:50:53.381 INFO:teuthology.orchestra.run.smithi019.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 416 kB/s | 145 kB 00:00 2024-04-25T09:50:53.581 INFO:teuthology.orchestra.run.smithi112.stdout:(8/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 6.7 MB/s | 20 MB 00:02 2024-04-25T09:50:53.664 INFO:teuthology.orchestra.run.smithi112.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 303 kB/s | 25 kB 00:00 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi112.stdout:(10/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 9.7 MB/s | 9.8 MB 00:01 2024-04-25T09:50:53.906 INFO:teuthology.orchestra.run.smithi112.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 2.6 MB/s | 220 kB 00:00 2024-04-25T09:50:53.981 INFO:teuthology.orchestra.run.smithi112.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 360 kB/s | 27 kB 00:00 2024-04-25T09:50:54.057 INFO:teuthology.orchestra.run.smithi112.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 416 kB/s | 31 kB 00:00 2024-04-25T09:50:54.165 INFO:teuthology.orchestra.run.smithi112.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 6.7 MB/s | 744 kB 00:00 2024-04-25T09:50:54.249 INFO:teuthology.orchestra.run.smithi112.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 1.9 MB/s | 163 kB 00:00 2024-04-25T09:50:54.361 INFO:teuthology.orchestra.run.smithi112.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.1 MB/s | 127 kB 00:00 2024-04-25T09:50:54.458 INFO:teuthology.orchestra.run.smithi112.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 5.1 MB/s | 501 kB 00:00 2024-04-25T09:50:54.797 INFO:teuthology.orchestra.run.smithi019.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 1.9 MB/s | 5.3 MB 00:02 2024-04-25T09:50:54.848 INFO:teuthology.orchestra.run.smithi019.stdout:(5/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 1.5 MB/s | 2.1 MB 00:01 2024-04-25T09:50:55.058 INFO:teuthology.orchestra.run.smithi112.stdout:(18/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 8.5 MB/s | 5.1 MB 00:00 2024-04-25T09:50:55.150 INFO:teuthology.orchestra.run.smithi112.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 492 kB/s | 45 kB 00:00 2024-04-25T09:50:55.226 INFO:teuthology.orchestra.run.smithi112.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.7 MB/s | 131 kB 00:00 2024-04-25T09:50:55.318 INFO:teuthology.orchestra.run.smithi112.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 1.7 MB/s | 162 kB 00:00 2024-04-25T09:50:55.418 INFO:teuthology.orchestra.run.smithi112.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 3.1 MB/s | 321 kB 00:00 2024-04-25T09:50:55.531 INFO:teuthology.orchestra.run.smithi019.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 2.0 MB/s | 1.5 MB 00:00 2024-04-25T09:50:55.535 INFO:teuthology.orchestra.run.smithi112.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 2.5 MB/s | 298 kB 00:00 2024-04-25T09:50:55.627 INFO:teuthology.orchestra.run.smithi112.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.1 MB/s | 99 kB 00:00 2024-04-25T09:50:55.794 INFO:teuthology.orchestra.run.smithi112.stdout:(25/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 4.7 MB/s | 17 MB 00:03 2024-04-25T09:50:55.819 INFO:teuthology.orchestra.run.smithi112.stdout:(26/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 445 kB/s | 85 kB 00:00 2024-04-25T09:50:55.927 INFO:teuthology.orchestra.run.smithi112.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-25T09:50:56.020 INFO:teuthology.orchestra.run.smithi112.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 340 kB/s | 31 kB 00:00 2024-04-25T09:50:56.129 INFO:teuthology.orchestra.run.smithi112.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.2 MB/s | 138 kB 00:00 2024-04-25T09:50:56.562 INFO:teuthology.orchestra.run.smithi112.stdout:(30/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 4.1 MB/s | 1.8 MB 00:00 2024-04-25T09:50:56.756 INFO:teuthology.orchestra.run.smithi019.stdout:(7/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 4.2 MB/s | 20 MB 00:04 2024-04-25T09:50:56.762 INFO:teuthology.orchestra.run.smithi112.stdout:(31/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 3.2 MB/s | 3.1 MB 00:00 2024-04-25T09:50:57.365 INFO:teuthology.orchestra.run.smithi019.stdout:(8/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 1.9 MB/s | 4.7 MB 00:02 2024-04-25T09:50:57.457 INFO:teuthology.orchestra.run.smithi019.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 275 kB/s | 25 kB 00:00 2024-04-25T09:50:57.580 INFO:teuthology.orchestra.run.smithi112.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 7.3 MB/s | 7.4 MB 00:01 2024-04-25T09:50:57.605 INFO:teuthology.orchestra.run.smithi112.stdout:(33/152): ceph-mgr-modules-core-19.0.0-1628.g97 294 kB/s | 248 kB 00:00 2024-04-25T09:50:57.663 INFO:teuthology.orchestra.run.smithi112.stdout:(34/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 592 kB/s | 49 kB 00:00 2024-04-25T09:50:57.689 INFO:teuthology.orchestra.run.smithi112.stdout:(35/152): ceph-prometheus-alerts-19.0.0-1628.g9 183 kB/s | 15 kB 00:00 2024-04-25T09:50:57.830 INFO:teuthology.orchestra.run.smithi112.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 5.2 MB/s | 748 kB 00:00 2024-04-25T09:50:57.840 INFO:teuthology.orchestra.run.smithi019.stdout:(10/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 9.0 MB/s | 9.8 MB 00:01 2024-04-25T09:50:57.889 INFO:teuthology.orchestra.run.smithi112.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 312 kB/s | 18 kB 00:00 2024-04-25T09:50:57.907 INFO:teuthology.orchestra.run.smithi019.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 3.2 MB/s | 220 kB 00:00 2024-04-25T09:50:57.974 INFO:teuthology.orchestra.run.smithi019.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 405 kB/s | 27 kB 00:00 2024-04-25T09:50:58.006 INFO:teuthology.orchestra.run.smithi112.stdout:(38/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 770 kB/s | 264 kB 00:00 2024-04-25T09:50:58.033 INFO:teuthology.orchestra.run.smithi019.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 537 kB/s | 31 kB 00:00 2024-04-25T09:50:58.073 INFO:teuthology.orchestra.run.smithi112.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 437 kB/s | 80 kB 00:00 2024-04-25T09:50:58.125 INFO:teuthology.orchestra.run.smithi019.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 8.0 MB/s | 744 kB 00:00 2024-04-25T09:50:58.190 INFO:teuthology.orchestra.run.smithi112.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-04-25T09:50:58.192 INFO:teuthology.orchestra.run.smithi019.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 2.4 MB/s | 163 kB 00:00 2024-04-25T09:50:58.242 INFO:teuthology.orchestra.run.smithi019.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 2.5 MB/s | 127 kB 00:00 2024-04-25T09:50:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 7.4 MB/s | 501 kB 00:00 2024-04-25T09:50:58.607 INFO:teuthology.orchestra.run.smithi112.stdout:(41/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 9.6 MB/s | 47 MB 00:04 2024-04-25T09:50:58.632 INFO:teuthology.orchestra.run.smithi112.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 573 kB/s | 253 kB 00:00 2024-04-25T09:50:58.691 INFO:teuthology.orchestra.run.smithi112.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T09:50:58.701 INFO:teuthology.orchestra.run.smithi019.stdout:(18/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 13 MB/s | 5.1 MB 00:00 2024-04-25T09:50:58.724 INFO:teuthology.orchestra.run.smithi112.stdout:(44/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-25T09:50:58.750 INFO:teuthology.orchestra.run.smithi112.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T09:50:58.751 INFO:teuthology.orchestra.run.smithi019.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 905 kB/s | 45 kB 00:00 2024-04-25T09:50:58.784 INFO:teuthology.orchestra.run.smithi112.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T09:50:58.817 INFO:teuthology.orchestra.run.smithi112.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-04-25T09:50:58.818 INFO:teuthology.orchestra.run.smithi019.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.9 MB/s | 131 kB 00:00 2024-04-25T09:50:58.876 INFO:teuthology.orchestra.run.smithi112.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 7.3 MB/s | 689 kB 00:00 2024-04-25T09:50:58.876 INFO:teuthology.orchestra.run.smithi019.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 2.7 MB/s | 162 kB 00:00 2024-04-25T09:50:58.934 INFO:teuthology.orchestra.run.smithi112.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T09:50:58.943 INFO:teuthology.orchestra.run.smithi019.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 4.7 MB/s | 321 kB 00:00 2024-04-25T09:50:58.985 INFO:teuthology.orchestra.run.smithi112.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-04-25T09:50:59.002 INFO:teuthology.orchestra.run.smithi019.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 5.0 MB/s | 298 kB 00:00 2024-04-25T09:50:59.018 INFO:teuthology.orchestra.run.smithi112.stdout:(51/152): python3-cryptography-36.0.1-4.el9.x86 3.0 MB/s | 1.2 MB 00:00 2024-04-25T09:50:59.052 INFO:teuthology.orchestra.run.smithi112.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-04-25T09:50:59.053 INFO:teuthology.orchestra.run.smithi019.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.9 MB/s | 99 kB 00:00 2024-04-25T09:50:59.077 INFO:teuthology.orchestra.run.smithi112.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-04-25T09:50:59.102 INFO:teuthology.orchestra.run.smithi112.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-25T09:50:59.103 INFO:teuthology.orchestra.run.smithi019.stdout:(25/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-25T09:50:59.136 INFO:teuthology.orchestra.run.smithi112.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-25T09:50:59.161 INFO:teuthology.orchestra.run.smithi112.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-25T09:50:59.195 INFO:teuthology.orchestra.run.smithi112.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T09:50:59.220 INFO:teuthology.orchestra.run.smithi112.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-04-25T09:50:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-04-25T09:50:59.345 INFO:teuthology.orchestra.run.smithi019.stdout:(26/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 13 MB/s | 3.1 MB 00:00 2024-04-25T09:50:59.404 INFO:teuthology.orchestra.run.smithi019.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.9 MB/s | 170 kB 00:00 2024-04-25T09:50:59.445 INFO:teuthology.orchestra.run.smithi019.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 756 kB/s | 31 kB 00:00 2024-04-25T09:50:59.504 INFO:teuthology.orchestra.run.smithi019.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 2.3 MB/s | 138 kB 00:00 2024-04-25T09:50:59.512 INFO:teuthology.orchestra.run.smithi112.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 1.9 MB/s | 582 kB 00:00 2024-04-25T09:50:59.538 INFO:teuthology.orchestra.run.smithi112.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 102 kB/s | 25 kB 00:00 2024-04-25T09:50:59.588 INFO:teuthology.orchestra.run.smithi112.stdout:(62/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.6 MB/s | 3.0 MB 00:00 2024-04-25T09:50:59.613 INFO:teuthology.orchestra.run.smithi112.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-04-25T09:50:59.638 INFO:teuthology.orchestra.run.smithi112.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-04-25T09:50:59.665 INFO:teuthology.orchestra.run.smithi019.stdout:(30/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 11 MB/s | 1.8 MB 00:00 2024-04-25T09:50:59.666 INFO:teuthology.orchestra.run.smithi112.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-25T09:50:59.689 INFO:teuthology.orchestra.run.smithi112.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 467 kB/s | 35 kB 00:00 2024-04-25T09:50:59.939 INFO:teuthology.orchestra.run.smithi112.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 6.4 MB/s | 1.6 MB 00:00 2024-04-25T09:51:00.006 INFO:teuthology.orchestra.run.smithi112.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-04-25T09:51:00.065 INFO:teuthology.orchestra.run.smithi112.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T09:51:00.090 INFO:teuthology.orchestra.run.smithi112.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-25T09:51:00.115 INFO:teuthology.orchestra.run.smithi112.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-04-25T09:51:00.174 INFO:teuthology.orchestra.run.smithi112.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T09:51:00.205 INFO:teuthology.orchestra.run.smithi019.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 14 MB/s | 7.4 MB 00:00 2024-04-25T09:51:00.207 INFO:teuthology.orchestra.run.smithi112.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-04-25T09:51:00.275 INFO:teuthology.orchestra.run.smithi112.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-25T09:51:00.280 INFO:teuthology.orchestra.run.smithi019.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 3.2 MB/s | 248 kB 00:00 2024-04-25T09:51:00.342 INFO:teuthology.orchestra.run.smithi112.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-25T09:51:00.372 INFO:teuthology.orchestra.run.smithi019.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 540 kB/s | 49 kB 00:00 2024-04-25T09:51:00.400 INFO:teuthology.orchestra.run.smithi112.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-04-25T09:51:00.414 INFO:teuthology.orchestra.run.smithi019.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 369 kB/s | 15 kB 00:00 2024-04-25T09:51:00.473 INFO:teuthology.orchestra.run.smithi019.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 4.4 MB/s | 264 kB 00:00 2024-04-25T09:51:00.501 INFO:teuthology.orchestra.run.smithi112.stdout:(77/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-25T09:51:00.526 INFO:teuthology.orchestra.run.smithi112.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-04-25T09:51:00.564 INFO:teuthology.orchestra.run.smithi019.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 8.0 MB/s | 748 kB 00:00 2024-04-25T09:51:00.567 INFO:teuthology.orchestra.run.smithi112.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-25T09:51:00.593 INFO:teuthology.orchestra.run.smithi112.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-04-25T09:51:00.614 INFO:teuthology.orchestra.run.smithi019.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 367 kB/s | 18 kB 00:00 2024-04-25T09:51:00.652 INFO:teuthology.orchestra.run.smithi112.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-25T09:51:00.673 INFO:teuthology.orchestra.run.smithi019.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 80 kB 00:00 2024-04-25T09:51:00.719 INFO:teuthology.orchestra.run.smithi112.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-25T09:51:00.811 INFO:teuthology.orchestra.run.smithi112.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 9.4 MB/s | 6.0 MB 00:00 2024-04-25T09:51:00.836 INFO:teuthology.orchestra.run.smithi112.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 659 kB/s | 77 kB 00:00 2024-04-25T09:51:00.899 INFO:teuthology.orchestra.run.smithi019.stdout:(39/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 3.1 MB/s | 17 MB 00:05 2024-04-25T09:51:00.911 INFO:teuthology.orchestra.run.smithi112.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T09:51:00.928 INFO:teuthology.orchestra.run.smithi112.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 360 kB/s | 42 kB 00:00 2024-04-25T09:51:00.941 INFO:teuthology.orchestra.run.smithi019.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-25T09:51:00.953 INFO:teuthology.orchestra.run.smithi112.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-04-25T09:51:00.978 INFO:teuthology.orchestra.run.smithi112.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-25T09:51:01.004 INFO:teuthology.orchestra.run.smithi112.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-25T09:51:01.007 INFO:teuthology.orchestra.run.smithi019.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-25T09:51:01.029 INFO:teuthology.orchestra.run.smithi112.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 674 kB/s | 51 kB 00:00 2024-04-25T09:51:01.033 INFO:teuthology.orchestra.run.smithi019.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-25T09:51:01.054 INFO:teuthology.orchestra.run.smithi112.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-04-25T09:51:01.088 INFO:teuthology.orchestra.run.smithi112.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-04-25T09:51:01.095 INFO:teuthology.orchestra.run.smithi019.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T09:51:01.113 INFO:teuthology.orchestra.run.smithi112.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 754 kB/s | 44 kB 00:00 2024-04-25T09:51:01.142 INFO:teuthology.orchestra.run.smithi019.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-04-25T09:51:01.147 INFO:teuthology.orchestra.run.smithi112.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T09:51:01.172 INFO:teuthology.orchestra.run.smithi112.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-04-25T09:51:01.184 INFO:teuthology.orchestra.run.smithi019.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-25T09:51:01.206 INFO:teuthology.orchestra.run.smithi112.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 806 kB/s | 47 kB 00:00 2024-04-25T09:51:01.226 INFO:teuthology.orchestra.run.smithi019.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-25T09:51:01.231 INFO:teuthology.orchestra.run.smithi112.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T09:51:01.265 INFO:teuthology.orchestra.run.smithi112.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 989 kB/s | 58 kB 00:00 2024-04-25T09:51:01.276 INFO:teuthology.orchestra.run.smithi019.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-25T09:51:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T09:51:01.324 INFO:teuthology.orchestra.run.smithi112.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T09:51:01.349 INFO:teuthology.orchestra.run.smithi112.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-04-25T09:51:01.391 INFO:teuthology.orchestra.run.smithi112.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 333 kB/s | 22 kB 00:00 2024-04-25T09:51:01.418 INFO:teuthology.orchestra.run.smithi019.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-04-25T09:51:01.468 INFO:teuthology.orchestra.run.smithi019.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-04-25T09:51:01.476 INFO:teuthology.orchestra.run.smithi112.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.6 MB/s | 863 kB 00:00 2024-04-25T09:51:01.485 INFO:teuthology.orchestra.run.smithi019.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T09:51:01.519 INFO:teuthology.orchestra.run.smithi019.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-25T09:51:01.544 INFO:teuthology.orchestra.run.smithi019.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-25T09:51:01.568 INFO:teuthology.orchestra.run.smithi112.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 3.0 MB/s | 548 kB 00:00 2024-04-25T09:51:01.569 INFO:teuthology.orchestra.run.smithi019.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-04-25T09:51:01.610 INFO:teuthology.orchestra.run.smithi112.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-04-25T09:51:01.611 INFO:teuthology.orchestra.run.smithi019.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-04-25T09:51:01.636 INFO:teuthology.orchestra.run.smithi112.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 290 kB/s | 19 kB 00:00 2024-04-25T09:51:01.653 INFO:teuthology.orchestra.run.smithi019.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-04-25T09:51:01.686 INFO:teuthology.orchestra.run.smithi112.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-04-25T09:51:01.703 INFO:teuthology.orchestra.run.smithi019.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T09:51:01.736 INFO:teuthology.orchestra.run.smithi112.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 973 kB/s | 49 kB 00:00 2024-04-25T09:51:01.745 INFO:teuthology.orchestra.run.smithi019.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-25T09:51:01.771 INFO:teuthology.orchestra.run.smithi019.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-04-25T09:51:01.804 INFO:teuthology.orchestra.run.smithi019.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T09:51:01.846 INFO:teuthology.orchestra.run.smithi019.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-25T09:51:01.863 INFO:teuthology.orchestra.run.smithi019.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-04-25T09:51:01.880 INFO:teuthology.orchestra.run.smithi019.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T09:51:01.895 INFO:teuthology.orchestra.run.smithi112.stdout:(109/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-25T09:51:01.912 INFO:teuthology.orchestra.run.smithi112.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 384 kB/s | 67 kB 00:00 2024-04-25T09:51:01.913 INFO:teuthology.orchestra.run.smithi019.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T09:51:01.939 INFO:teuthology.orchestra.run.smithi019.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-25T09:51:01.956 INFO:teuthology.orchestra.run.smithi019.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 841 kB/s | 35 kB 00:00 2024-04-25T09:51:01.979 INFO:teuthology.orchestra.run.smithi112.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-25T09:51:02.029 INFO:teuthology.orchestra.run.smithi112.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-04-25T09:51:02.079 INFO:teuthology.orchestra.run.smithi112.stdout:(113/152): luarocks-3.9.2-1.el9.noarch.rpm 821 kB/s | 151 kB 00:00 2024-04-25T09:51:02.105 INFO:teuthology.orchestra.run.smithi112.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 576 kB/s | 43 kB 00:00 2024-04-25T09:51:02.130 INFO:teuthology.orchestra.run.smithi112.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-04-25T09:51:02.139 INFO:teuthology.orchestra.run.smithi019.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-04-25T09:51:02.147 INFO:teuthology.orchestra.run.smithi112.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 359 kB/s | 15 kB 00:00 2024-04-25T09:51:02.172 INFO:teuthology.orchestra.run.smithi112.stdout:(117/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-04-25T09:51:02.197 INFO:teuthology.orchestra.run.smithi112.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-25T09:51:02.206 INFO:teuthology.orchestra.run.smithi019.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T09:51:02.231 INFO:teuthology.orchestra.run.smithi112.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-25T09:51:02.256 INFO:teuthology.orchestra.run.smithi112.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 602 kB/s | 35 kB 00:00 2024-04-25T09:51:02.265 INFO:teuthology.orchestra.run.smithi019.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T09:51:02.307 INFO:teuthology.orchestra.run.smithi112.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-25T09:51:02.307 INFO:teuthology.orchestra.run.smithi019.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-25T09:51:02.332 INFO:teuthology.orchestra.run.smithi112.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T09:51:02.332 INFO:teuthology.orchestra.run.smithi019.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-25T09:51:02.349 INFO:teuthology.orchestra.run.smithi112.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-04-25T09:51:02.382 INFO:teuthology.orchestra.run.smithi112.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-04-25T09:51:02.399 INFO:teuthology.orchestra.run.smithi019.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T09:51:02.407 INFO:teuthology.orchestra.run.smithi112.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-04-25T09:51:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-04-25T09:51:02.449 INFO:teuthology.orchestra.run.smithi112.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-25T09:51:02.475 INFO:teuthology.orchestra.run.smithi112.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-04-25T09:51:02.500 INFO:teuthology.orchestra.run.smithi112.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-25T09:51:02.526 INFO:teuthology.orchestra.run.smithi112.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-04-25T09:51:02.567 INFO:teuthology.orchestra.run.smithi112.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-04-25T09:51:02.593 INFO:teuthology.orchestra.run.smithi112.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-25T09:51:02.608 INFO:teuthology.orchestra.run.smithi019.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-04-25T09:51:02.618 INFO:teuthology.orchestra.run.smithi112.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T09:51:02.652 INFO:teuthology.orchestra.run.smithi112.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-04-25T09:51:02.677 INFO:teuthology.orchestra.run.smithi112.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 279 kB/s | 16 kB 00:00 2024-04-25T09:51:02.702 INFO:teuthology.orchestra.run.smithi112.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-04-25T09:51:02.717 INFO:teuthology.orchestra.run.smithi019.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-04-25T09:51:02.728 INFO:teuthology.orchestra.run.smithi112.stdout:(136/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-25T09:51:02.742 INFO:teuthology.orchestra.run.smithi019.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-25T09:51:02.753 INFO:teuthology.orchestra.run.smithi112.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-25T09:51:02.767 INFO:teuthology.orchestra.run.smithi019.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T09:51:02.778 INFO:teuthology.orchestra.run.smithi112.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-25T09:51:02.793 INFO:teuthology.orchestra.run.smithi019.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-04-25T09:51:02.803 INFO:teuthology.orchestra.run.smithi112.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-25T09:51:02.818 INFO:teuthology.orchestra.run.smithi019.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-25T09:51:02.829 INFO:teuthology.orchestra.run.smithi112.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-04-25T09:51:02.843 INFO:teuthology.orchestra.run.smithi019.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T09:51:02.854 INFO:teuthology.orchestra.run.smithi112.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-25T09:51:02.868 INFO:teuthology.orchestra.run.smithi019.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-04-25T09:51:02.879 INFO:teuthology.orchestra.run.smithi112.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-04-25T09:51:02.904 INFO:teuthology.orchestra.run.smithi112.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T09:51:02.919 INFO:teuthology.orchestra.run.smithi019.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-25T09:51:02.930 INFO:teuthology.orchestra.run.smithi112.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-25T09:51:02.946 INFO:teuthology.orchestra.run.smithi112.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-04-25T09:51:02.977 INFO:teuthology.orchestra.run.smithi019.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T09:51:02.980 INFO:teuthology.orchestra.run.smithi112.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-04-25T09:51:03.014 INFO:teuthology.orchestra.run.smithi112.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-25T09:51:03.044 INFO:teuthology.orchestra.run.smithi019.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-04-25T09:51:03.070 INFO:teuthology.orchestra.run.smithi019.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 839 kB/s | 77 kB 00:00 2024-04-25T09:51:03.089 INFO:teuthology.orchestra.run.smithi112.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T09:51:03.095 INFO:teuthology.orchestra.run.smithi019.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-04-25T09:51:03.120 INFO:teuthology.orchestra.run.smithi019.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T09:51:03.145 INFO:teuthology.orchestra.run.smithi019.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T09:51:03.171 INFO:teuthology.orchestra.run.smithi019.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T09:51:03.196 INFO:teuthology.orchestra.run.smithi019.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T09:51:03.221 INFO:teuthology.orchestra.run.smithi019.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T09:51:03.271 INFO:teuthology.orchestra.run.smithi019.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 839 kB/s | 42 kB 00:00 2024-04-25T09:51:03.406 INFO:teuthology.orchestra.run.smithi112.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 1.3 MB 00:00 2024-04-25T09:51:03.463 INFO:teuthology.orchestra.run.smithi019.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 230 kB/s | 44 kB 00:00 2024-04-25T09:51:03.473 INFO:teuthology.orchestra.run.smithi112.stdout:(150/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 6.8 MB/s | 3.1 MB 00:00 2024-04-25T09:51:03.514 INFO:teuthology.orchestra.run.smithi019.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T09:51:03.564 INFO:teuthology.orchestra.run.smithi019.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-04-25T09:51:03.614 INFO:teuthology.orchestra.run.smithi019.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 952 kB/s | 47 kB 00:00 2024-04-25T09:51:03.615 INFO:teuthology.orchestra.run.smithi112.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-25T09:51:03.665 INFO:teuthology.orchestra.run.smithi019.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T09:51:03.715 INFO:teuthology.orchestra.run.smithi019.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T09:51:03.882 INFO:teuthology.orchestra.run.smithi019.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-25T09:51:03.907 INFO:teuthology.orchestra.run.smithi019.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 488 kB/s | 94 kB 00:00 2024-04-25T09:51:03.924 INFO:teuthology.orchestra.run.smithi019.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-04-25T09:51:03.941 INFO:teuthology.orchestra.run.smithi019.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T09:51:04.008 INFO:teuthology.orchestra.run.smithi019.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-04-25T09:51:04.150 INFO:teuthology.orchestra.run.smithi019.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 3.8 MB/s | 548 kB 00:00 2024-04-25T09:51:04.200 INFO:teuthology.orchestra.run.smithi019.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-04-25T09:51:04.225 INFO:teuthology.orchestra.run.smithi019.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-04-25T09:51:04.276 INFO:teuthology.orchestra.run.smithi019.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-04-25T09:51:04.309 INFO:teuthology.orchestra.run.smithi019.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 782 kB/s | 26 kB 00:00 2024-04-25T09:51:04.363 INFO:teuthology.orchestra.run.smithi019.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 976 kB/s | 49 kB 00:00 2024-04-25T09:51:04.401 INFO:teuthology.orchestra.run.smithi019.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-25T09:51:04.482 INFO:teuthology.orchestra.run.smithi112.stdout:(152/152): librados2-19.0.0-1628.g97ae6cc9.el9. 2.2 MB/s | 3.4 MB 00:01 2024-04-25T09:51:04.485 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:51:04.485 INFO:teuthology.orchestra.run.smithi112.stdout:Total 15 MB/s | 215 MB 00:14 2024-04-25T09:51:04.485 INFO:teuthology.orchestra.run.smithi019.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T09:51:04.510 INFO:teuthology.orchestra.run.smithi019.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-25T09:51:04.570 INFO:teuthology.orchestra.run.smithi019.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-25T09:51:04.611 INFO:teuthology.orchestra.run.smithi019.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-04-25T09:51:04.636 INFO:teuthology.orchestra.run.smithi019.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 651 kB/s | 43 kB 00:00 2024-04-25T09:51:04.662 INFO:teuthology.orchestra.run.smithi019.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-04-25T09:51:04.687 INFO:teuthology.orchestra.run.smithi019.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-25T09:51:04.712 INFO:teuthology.orchestra.run.smithi019.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-25T09:51:04.746 INFO:teuthology.orchestra.run.smithi019.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-25T09:51:04.796 INFO:teuthology.orchestra.run.smithi019.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-25T09:51:04.821 INFO:teuthology.orchestra.run.smithi019.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 470 kB/s | 35 kB 00:00 2024-04-25T09:51:04.855 INFO:teuthology.orchestra.run.smithi019.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-25T09:51:04.930 INFO:teuthology.orchestra.run.smithi019.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-25T09:51:04.972 INFO:teuthology.orchestra.run.smithi019.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-04-25T09:51:05.022 INFO:teuthology.orchestra.run.smithi019.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-04-25T09:51:05.047 INFO:teuthology.orchestra.run.smithi019.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-25T09:51:05.073 INFO:teuthology.orchestra.run.smithi019.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-25T09:51:05.098 INFO:teuthology.orchestra.run.smithi019.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-04-25T09:51:05.123 INFO:teuthology.orchestra.run.smithi019.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-04-25T09:51:05.165 INFO:teuthology.orchestra.run.smithi019.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-25T09:51:05.198 INFO:teuthology.orchestra.run.smithi019.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-25T09:51:05.224 INFO:teuthology.orchestra.run.smithi019.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-25T09:51:05.249 INFO:teuthology.orchestra.run.smithi019.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T09:51:05.275 INFO:teuthology.orchestra.run.smithi019.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-25T09:51:05.300 INFO:teuthology.orchestra.run.smithi019.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-25T09:51:05.317 INFO:teuthology.orchestra.run.smithi019.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-04-25T09:51:05.350 INFO:teuthology.orchestra.run.smithi019.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-04-25T09:51:05.367 INFO:teuthology.orchestra.run.smithi019.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-04-25T09:51:05.401 INFO:teuthology.orchestra.run.smithi019.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-25T09:51:05.476 INFO:teuthology.orchestra.run.smithi019.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-04-25T09:51:05.543 INFO:teuthology.orchestra.run.smithi019.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 252 kB/s | 36 kB 00:00 2024-04-25T09:51:05.556 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-25T09:51:05.568 INFO:teuthology.orchestra.run.smithi019.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-04-25T09:51:05.602 INFO:teuthology.orchestra.run.smithi019.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-25T09:51:05.619 INFO:teuthology.orchestra.run.smithi019.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-25T09:51:05.653 INFO:teuthology.orchestra.run.smithi019.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 8.4 MB/s | 427 kB 00:00 2024-04-25T09:51:05.678 INFO:teuthology.orchestra.run.smithi019.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 343 kB/s | 20 kB 00:00 2024-04-25T09:51:05.696 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-25T09:51:05.696 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-25T09:51:05.703 INFO:teuthology.orchestra.run.smithi019.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T09:51:05.778 INFO:teuthology.orchestra.run.smithi019.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T09:51:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:(148/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-25T09:51:06.995 INFO:teuthology.orchestra.run.smithi019.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 13 MB/s | 1.3 MB 00:00 2024-04-25T09:51:07.421 INFO:teuthology.orchestra.run.smithi019.stdout:(150/152): ceph-test-19.0.0-1628.g97ae6cc9.el9. 4.8 MB/s | 47 MB 00:09 2024-04-25T09:51:07.554 INFO:teuthology.orchestra.run.smithi019.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 2.4 MB 00:00 2024-04-25T09:51:07.604 INFO:teuthology.orchestra.run.smithi019.stdout:(152/152): librados2-19.0.0-1628.g97ae6cc9.el9. 1.8 MB/s | 3.4 MB 00:01 2024-04-25T09:51:07.608 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:51:07.608 INFO:teuthology.orchestra.run.smithi019.stdout:Total 13 MB/s | 215 MB 00:16 2024-04-25T09:51:07.718 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-25T09:51:07.718 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-25T09:51:08.738 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-04-25T09:51:08.898 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-04-25T09:51:08.898 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-04-25T09:51:10.766 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-04-25T09:51:10.766 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-04-25T09:51:13.584 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-25T09:51:13.669 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:51:13.716 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:51:13.777 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:51:13.811 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:51:13.837 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:51:13.879 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:51:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:51:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:51:14.426 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:51:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:51:14.546 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:51:14.577 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:51:14.724 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:51:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:51:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:51:14.888 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:51:14.911 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:51:14.957 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:51:14.979 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:51:15.004 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:51:15.178 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:51:15.211 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:51:15.317 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:51:15.422 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:51:15.461 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:51:15.496 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:51:15.578 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:51:15.612 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:51:15.656 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:51:15.688 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:51:15.716 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:51:15.786 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:51:15.839 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:51:15.869 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:51:15.912 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:51:15.987 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:51:16.024 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:51:16.058 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:51:16.087 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:51:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:51:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:51:16.276 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:51:16.410 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:51:16.463 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T09:51:16.774 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-04-25T09:51:16.867 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:51:16.906 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:51:16.960 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:51:16.993 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T09:51:17.001 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:51:17.026 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:51:17.070 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:51:17.075 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T09:51:17.146 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T09:51:17.185 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T09:51:17.226 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T09:51:17.382 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T09:51:17.437 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T09:51:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:51:17.493 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T09:51:17.519 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:51:17.565 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T09:51:17.596 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T09:51:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T09:51:17.643 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:51:17.670 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:51:17.680 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T09:51:17.716 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T09:51:17.759 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T09:51:17.762 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:51:17.791 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:51:17.801 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T09:51:17.937 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T09:51:17.947 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:51:17.981 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T09:51:17.997 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:51:18.012 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T09:51:18.024 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:51:18.053 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T09:51:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T09:51:18.171 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:51:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T09:51:18.200 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:51:18.226 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T09:51:18.252 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T09:51:18.263 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:51:18.277 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T09:51:18.284 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:51:18.307 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:51:18.350 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T09:51:18.479 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T09:51:18.502 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:51:18.518 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T09:51:18.534 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:51:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T09:51:18.580 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T09:51:18.619 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T09:51:18.644 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:51:18.650 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T09:51:18.680 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T09:51:18.715 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T09:51:18.741 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:51:18.748 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T09:51:18.774 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:51:18.810 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:51:18.857 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T09:51:18.886 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:51:18.934 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:51:18.978 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:51:18.991 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T09:51:19.011 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:51:19.038 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:51:19.072 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T09:51:19.110 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:51:19.121 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T09:51:19.153 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T09:51:19.162 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:51:19.191 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:51:19.201 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T09:51:19.235 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:51:19.306 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T09:51:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:51:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:51:19.353 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T09:51:19.389 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:51:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:51:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:51:19.557 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:51:19.635 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:51:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:51:19.853 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T09:51:20.075 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T09:51:20.126 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T09:51:20.160 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T09:51:20.257 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T09:51:20.309 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T09:51:20.418 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T09:51:20.506 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T09:51:20.508 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T09:51:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T09:51:20.585 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T09:51:20.604 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T09:51:20.623 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T09:51:20.665 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T09:51:20.781 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T09:51:20.810 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T09:51:20.821 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T09:51:20.876 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T09:51:20.895 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T09:51:20.933 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T09:51:21.008 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T09:51:21.052 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T09:51:21.088 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T09:51:21.136 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T09:51:21.171 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T09:51:21.215 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T09:51:21.257 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T09:51:21.399 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T09:51:21.445 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T09:51:21.475 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T09:51:21.517 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T09:51:21.536 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T09:51:21.614 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T09:51:21.658 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T09:51:21.667 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T09:51:21.698 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T09:51:21.723 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T09:51:21.748 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T09:51:21.826 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T09:51:21.956 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T09:51:21.998 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T09:51:22.025 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T09:51:22.059 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T09:51:22.099 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T09:51:22.131 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T09:51:22.160 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T09:51:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T09:51:22.228 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T09:51:22.270 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T09:51:22.304 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T09:51:22.581 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T09:51:22.644 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T09:51:22.682 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T09:51:22.730 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T09:51:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T09:51:22.884 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T09:51:22.893 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T09:51:22.956 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T09:51:22.984 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T09:51:23.017 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T09:51:23.328 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T09:51:23.372 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:51:23.462 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:51:23.493 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T09:51:23.529 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T09:51:23.659 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T09:51:23.706 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T09:51:23.740 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T09:51:23.843 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T09:51:23.915 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T09:51:23.981 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T09:51:24.008 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:51:24.008 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T09:51:24.058 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T09:51:24.096 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:51:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T09:51:24.242 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T09:51:24.272 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T09:51:24.353 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T09:51:25.033 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T09:51:25.149 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T09:51:26.269 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:51:26.298 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:51:26.446 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T09:51:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T09:51:26.838 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T09:51:26.840 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:51:26.871 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T09:51:26.879 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:51:26.969 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:51:26.969 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T09:51:26.969 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T09:51:26.969 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:26.996 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:51:27.198 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T09:51:27.241 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:51:27.341 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:51:27.371 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T09:51:27.407 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T09:51:27.886 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T09:51:27.910 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:51:27.982 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:51:28.012 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T09:51:30.304 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:51:30.333 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:51:30.902 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:51:30.937 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:51:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:51:31.011 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T09:51:31.011 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T09:51:31.011 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:51:31.038 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:51:38.373 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:51:38.373 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-04-25T09:51:38.373 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-04-25T09:51:38.373 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-04-25T09:51:38.373 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-04-25T09:51:38.373 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-04-25T09:51:38.373 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-04-25T09:51:38.374 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-04-25T09:51:38.374 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:38.534 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:51:38.549 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:51:38.549 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:38.549 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T09:51:38.549 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:40.039 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:51:40.063 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:51:40.063 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:40.064 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T09:51:40.064 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:51:40.064 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:51:40.064 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:40.264 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:51:40.286 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:51:40.287 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:40.287 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T09:51:40.287 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:51:40.287 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:51:40.287 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:40.731 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:51:40.754 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:51:40.754 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:40.754 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T09:51:40.754 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:51:40.754 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:51:40.754 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:40.861 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T09:51:40.909 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T09:51:40.965 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T09:51:41.147 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T09:51:41.207 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T09:51:41.284 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T09:51:41.372 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:51:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:51:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T09:51:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T09:51:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T09:51:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:51:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:51:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T09:51:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T09:51:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:51:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:51:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:51:42.898 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:51:42.899 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-04-25T09:51:42.899 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-04-25T09:51:42.899 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-04-25T09:51:42.899 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-04-25T09:51:42.899 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-04-25T09:51:42.899 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-04-25T09:51:42.899 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-04-25T09:51:42.899 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:51:43.066 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:51:43.083 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:51:43.083 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:43.083 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T09:51:43.083 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:51:43.430 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:51:43.456 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:51:43.557 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:51:43.711 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T09:51:43.733 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:51:43.756 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:51:43.756 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:43.756 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T09:51:43.756 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:51:43.756 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:51:43.757 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:51:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:51:43.909 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T09:51:43.965 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T09:51:44.611 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:51:44.638 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:51:44.638 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:44.639 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T09:51:44.639 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:51:44.639 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:51:44.639 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:51:44.828 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T09:51:44.844 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:51:44.862 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:51:44.868 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:51:44.868 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:44.868 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T09:51:44.868 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:51:44.868 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:51:44.868 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:51:44.884 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:51:44.884 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:44.884 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T09:51:44.884 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:51:44.884 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:51:44.884 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:44.929 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:51:44.951 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:51:44.951 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:44.951 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T09:51:44.951 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:44.996 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:51:45.018 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:51:45.018 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:45.018 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T09:51:45.019 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:51:45.019 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:51:45.019 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:45.322 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:51:45.337 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:51:45.344 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:51:45.344 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:45.344 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T09:51:45.344 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:51:45.344 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:51:45.344 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:45.360 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:51:45.361 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:45.361 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T09:51:45.361 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:51:45.361 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:51:45.361 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:51:45.476 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T09:51:45.534 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T09:51:45.598 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T09:51:45.794 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T09:51:45.847 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T09:51:45.928 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T09:51:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:51:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:51:46.213 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T09:51:46.245 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T09:51:46.272 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T09:51:46.750 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:51:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:51:46.811 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T09:51:46.874 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T09:51:46.902 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:51:47.142 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:51:47.177 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:51:48.191 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:51:48.219 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:51:48.328 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:51:48.499 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T09:51:48.530 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:51:48.554 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:51:48.554 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:48.554 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T09:51:48.555 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:51:48.555 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:51:48.555 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:51:48.605 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:51:48.617 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:51:48.718 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T09:51:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T09:51:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T09:51:49.700 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:51:49.724 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:51:49.724 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:49.724 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T09:51:49.724 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:51:49.724 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:51:49.724 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:51:49.734 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T09:51:49.775 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T09:51:49.775 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:51:49.798 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:51:49.799 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:49.799 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T09:51:49.799 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:51:49.816 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T09:51:49.843 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:51:49.848 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T09:51:49.866 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:51:49.866 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:49.867 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T09:51:49.867 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:51:49.867 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:51:49.867 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:51:49.877 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T09:51:50.056 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T09:51:50.096 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T09:51:50.127 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T09:51:50.193 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:51:50.217 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:51:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T09:51:50.217 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:51:50.218 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:51:50.218 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:51:50.758 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:51:50.791 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:51:50.810 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:51:51.629 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T09:51:51.713 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T09:51:51.754 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T09:51:51.834 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T09:51:51.875 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T09:51:51.875 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:51:51.959 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:51:51.992 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:51:51.992 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:51:52.092 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:51:52.157 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:51:54.922 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T09:51:54.963 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T09:51:55.004 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T09:51:55.038 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T09:51:55.065 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T09:51:55.257 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T09:51:55.303 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T09:51:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T09:51:55.969 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:51:56.014 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:51:56.035 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:51:56.944 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T09:51:57.044 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T09:51:57.086 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T09:51:57.171 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T09:51:57.207 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T09:51:57.207 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:51:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:51:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T09:51:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T09:51:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T09:51:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T09:51:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T09:51:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T09:51:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T09:51:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T09:51:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T09:51:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T09:51:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T09:51:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T09:51:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T09:51:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T09:51:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T09:51:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T09:51:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T09:51:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T09:51:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T09:51:57.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T09:51:57.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T09:51:57.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T09:51:57.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T09:51:57.258 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T09:51:57.259 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T09:51:57.260 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T09:51:57.261 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T09:51:57.261 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T09:51:57.261 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T09:51:57.261 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T09:51:57.261 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:51:57.261 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T09:51:57.291 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:51:57.332 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:51:57.332 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:51:57.461 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:51:57.536 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:51:59.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:51:59.347 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:59.347 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-04-25T09:51:59.347 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.348 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.349 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T09:51:59.350 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T09:51:59.351 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T09:51:59.352 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T09:51:59.353 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.354 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.354 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.354 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T09:51:59.354 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T09:51:59.354 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T09:51:59.354 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T09:51:59.354 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T09:51:59.354 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T09:51:59.354 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T09:51:59.355 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T09:51:59.356 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T09:51:59.356 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T09:51:59.356 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T09:51:59.356 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T09:51:59.356 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T09:51:59.356 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T09:51:59.356 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T09:51:59.356 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T09:51:59.356 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T09:51:59.356 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T09:51:59.357 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.358 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.358 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T09:51:59.358 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T09:51:59.358 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T09:51:59.358 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.358 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T09:51:59.358 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T09:51:59.358 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T09:51:59.358 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T09:51:59.358 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T09:51:59.359 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T09:51:59.359 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T09:51:59.359 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T09:51:59.359 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T09:51:59.359 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T09:51:59.359 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T09:51:59.359 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T09:51:59.359 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T09:51:59.359 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T09:51:59.359 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T09:51:59.359 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.360 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.360 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:59.360 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T09:51:59.360 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T09:51:59.360 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T09:51:59.360 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T09:51:59.360 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T09:51:59.360 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T09:51:59.361 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T09:51:59.361 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:51:59.361 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T09:51:59.781 DEBUG:teuthology.parallel:result is None 2024-04-25T09:52:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:52:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T09:52:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T09:52:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T09:52:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T09:52:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T09:52:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T09:52:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T09:52:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:52:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:52:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T09:52:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T09:52:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T09:52:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T09:52:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T09:52:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T09:52:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T09:52:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T09:52:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T09:52:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T09:52:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T09:52:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T09:52:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T09:52:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T09:52:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T09:52:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T09:52:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T09:52:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T09:52:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T09:52:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T09:52:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T09:52:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T09:52:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T09:52:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T09:52:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T09:52:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T09:52:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T09:52:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T09:52:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T09:52:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T09:52:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T09:52:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:52:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T09:52:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T09:52:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T09:52:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T09:52:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T09:52:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T09:52:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T09:52:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T09:52:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T09:52:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T09:52:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T09:52:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T09:52:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T09:52:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:04.657 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T09:52:05.062 DEBUG:teuthology.parallel:result is None 2024-04-25T09:52:05.063 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:52:05.209 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T09:52:05.231 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:52:05.232 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:52:05.232 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T09:52:05.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:52:05.371 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T09:52:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:52:05.397 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:52:05.397 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T09:52:05.399 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T09:52:05.399 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:05.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T09:52:05.426 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:05.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T09:52:05.467 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T09:52:05.468 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:05.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T09:52:05.493 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T09:52:05.558 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:05.558 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T09:52:05.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T09:52:05.653 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T09:52:05.653 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:05.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T09:52:05.678 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T09:52:05.742 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:05.742 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T09:52:05.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T09:52:05.835 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T09:52:05.835 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:05.835 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T09:52:05.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T09:52:05.924 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:05.924 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T09:52:05.950 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T09:52:06.015 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T09:52:06.099 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T09:52:06.099 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /var/log/ceph 2024-04-25T09:52:06.101 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /var/log/ceph 2024-04-25T09:52:06.126 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T09:52:06.126 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T09:52:06.167 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T09:52:06.192 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T09:52:06.192 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T09:52:06.232 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T09:52:06.263 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T09:52:06.266 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T09:52:06.266 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T09:52:06.267 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:06.267 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T09:52:06.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:06.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:06.445 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:06.445 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T09:52:06.460 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:06.569 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:06.613 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T09:52:06.613 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T09:52:06.613 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673033', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi019.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKaviYy5ug0s6Xg9RhNtgSu65eUWVy5bOV1NRmbSw/69Fn70W+VhtzEqIEnUu6woMugLhFX4JpgsdloSm/KJ71Y=', 'smithi112.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAa/e7AreuiaIKjOsSkwgKV2Lo7i9LPQwl+/icoFA5SlnlNewaexHVsLLXQC4wozaL4KLVcHX/h/MrVg5sNJfvM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T09:52:06.614 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T09:52:06.616 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T09:52:06.662 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T09:52:06.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T09:52:06.725 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:06.726 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T09:52:06.750 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T09:52:06.751 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-04-25T09:52:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T09:52:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T09:52:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 09:51:57.915496705 +0000 2024-04-25T09:52:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 09:50:12.167295050 +0000 2024-04-25T09:52:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 09:50:12.167295050 +0000 2024-04-25T09:52:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 09:50:12.167295050 +0000 2024-04-25T09:52:06.807 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T09:52:06.870 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:52:06.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:52:06.871 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000259993 s, 2.0 MB/s 2024-04-25T09:52:06.872 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T09:52:06.929 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-04-25T09:52:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T09:52:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:06.985 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T09:52:06.985 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:06.985 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:06.985 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 09:51:57.915496705 +0000 2024-04-25T09:52:06.985 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 09:50:12.510291526 +0000 2024-04-25T09:52:06.985 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 09:50:12.510291526 +0000 2024-04-25T09:52:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 09:50:12.510291526 +0000 2024-04-25T09:52:06.985 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T09:52:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:52:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:52:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000202256 s, 2.5 MB/s 2024-04-25T09:52:07.049 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T09:52:07.104 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-04-25T09:52:07.160 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T09:52:07.160 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:07.160 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T09:52:07.160 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:07.160 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:07.160 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 09:51:57.915496705 +0000 2024-04-25T09:52:07.160 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 09:50:12.845288084 +0000 2024-04-25T09:52:07.160 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 09:50:12.845288084 +0000 2024-04-25T09:52:07.161 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 09:50:12.845288084 +0000 2024-04-25T09:52:07.161 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T09:52:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:52:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:52:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000187905 s, 2.7 MB/s 2024-04-25T09:52:07.224 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T09:52:07.281 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-04-25T09:52:07.336 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T09:52:07.336 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:07.336 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T09:52:07.336 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:07.336 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:07.336 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 09:51:57.916496694 +0000 2024-04-25T09:52:07.337 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 09:50:13.206284375 +0000 2024-04-25T09:52:07.337 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 09:50:13.206284375 +0000 2024-04-25T09:52:07.337 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 09:50:13.206284375 +0000 2024-04-25T09:52:07.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T09:52:07.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:52:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:52:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000226044 s, 2.3 MB/s 2024-04-25T09:52:07.400 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T09:52:07.456 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:07.456 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:07.456 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T09:52:07.473 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T09:52:07.473 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-04-25T09:52:07.529 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T09:52:07.530 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:07.530 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T09:52:07.530 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:07.530 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:07.530 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 09:51:52.537935639 +0000 2024-04-25T09:52:07.530 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 09:50:06.566952855 +0000 2024-04-25T09:52:07.530 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 09:50:06.566952855 +0000 2024-04-25T09:52:07.530 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 09:50:06.566952855 +0000 2024-04-25T09:52:07.530 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T09:52:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T09:52:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T09:52:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000194166 s, 2.6 MB/s 2024-04-25T09:52:07.595 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T09:52:07.651 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-04-25T09:52:07.708 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T09:52:07.708 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:07.708 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T09:52:07.708 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:07.709 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:07.709 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 09:51:52.538935612 +0000 2024-04-25T09:52:07.709 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 09:50:06.896943585 +0000 2024-04-25T09:52:07.709 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 09:50:06.896943585 +0000 2024-04-25T09:52:07.709 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 09:50:06.896943585 +0000 2024-04-25T09:52:07.710 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T09:52:07.772 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T09:52:07.772 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T09:52:07.772 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000225827 s, 2.3 MB/s 2024-04-25T09:52:07.774 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T09:52:07.831 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-04-25T09:52:07.887 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T09:52:07.887 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:07.887 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T09:52:07.887 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:07.887 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:07.887 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 09:51:52.538935612 +0000 2024-04-25T09:52:07.888 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 09:50:07.225934348 +0000 2024-04-25T09:52:07.888 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 09:50:07.225934348 +0000 2024-04-25T09:52:07.888 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 09:50:07.225934348 +0000 2024-04-25T09:52:07.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T09:52:07.950 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T09:52:07.950 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T09:52:07.950 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000204554 s, 2.5 MB/s 2024-04-25T09:52:07.951 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T09:52:08.007 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-04-25T09:52:08.063 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T09:52:08.063 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:08.063 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T09:52:08.063 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:08.063 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:08.063 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 09:51:52.538935612 +0000 2024-04-25T09:52:08.063 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 09:50:07.559924971 +0000 2024-04-25T09:52:08.063 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 09:50:07.559924971 +0000 2024-04-25T09:52:08.063 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 09:50:07.559924971 +0000 2024-04-25T09:52:08.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T09:52:08.126 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T09:52:08.126 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T09:52:08.126 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000187348 s, 2.7 MB/s 2024-04-25T09:52:08.127 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T09:52:08.183 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:08.183 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T09:52:08.184 INFO:tasks.ceph:Generating config... 2024-04-25T09:52:08.185 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T09:52:08.185 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T09:52:08.185 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T09:52:08.185 INFO:tasks.ceph:[client] pid file = 2024-04-25T09:52:08.185 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T09:52:08.185 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T09:52:08.185 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T09:52:08.185 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T09:52:08.185 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T09:52:08.186 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T09:52:08.186 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-25T09:52:08.186 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T09:52:08.186 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T09:52:08.186 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-25T09:52:08.186 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T09:52:08.186 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T09:52:08.186 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T09:52:08.186 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T09:52:08.187 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T09:52:08.187 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T09:52:08.187 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T09:52:08.187 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T09:52:08.187 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T09:52:08.187 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T09:52:08.187 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T09:52:08.187 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T09:52:08.187 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T09:52:08.187 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T09:52:08.187 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T09:52:08.187 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T09:52:08.187 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T09:52:08.188 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T09:52:08.188 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T09:52:08.188 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T09:52:08.188 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T09:52:08.188 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T09:52:08.188 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T09:52:08.188 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T09:52:08.188 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T09:52:08.188 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T09:52:08.256 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T09:52:08.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T09:52:08.300 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:52:08.364 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.19'), ('mon.b', '172.21.15.112'), ('mon.c', '[v2:172.21.15.112:3301,v1:172.21.15.112:6790]')] 2024-04-25T09:52:08.365 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.19,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T09:52:08.366 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:08.366 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T09:52:08.419 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.19 --add b 172.21.15.112 --addv c '[v2:172.21.15.112:3301,v1:172.21.15.112:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:08.504 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: generated fsid 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T09:52:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-04-25T09:52:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 0 2024-04-25T09:52:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T09:52:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:last_changed 2024-04-25T09:52:08.504240+0000 2024-04-25T09:52:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-04-25T09:52:08.504240+0000 2024-04-25T09:52:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:min_mon_release 17 (quincy) 2024-04-25T09:52:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:election_strategy: 1 2024-04-25T09:52:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T09:52:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-25T09:52:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:2: [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] mon.c 2024-04-25T09:52:08.506 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T09:52:08.507 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T09:52:08.521 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 342f98bd-d5a6-4934-bb27-7f8bea67a3be... 2024-04-25T09:52:08.522 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:52:08.564 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:52:08.605 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-04-25T09:52:08.605 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-04-25T09:52:08.605 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:52:08.605 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-04-25T09:52:08.605 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.605 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-04-25T09:52:08.606 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:52:08.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on skipped message = False 2024-04-25T09:52:08.608 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-04-25T09:52:08.609 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.610 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 20 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 20 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 10 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-04-25T09:52:08.611 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.612 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:52:08.613 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.614 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:52:08.614 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:52:08.614 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:52:08.614 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.614 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-04-25T09:52:08.614 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:52:08.614 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.614 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:52:08.614 INFO:teuthology.orchestra.run.smithi019.stdout: mon op complaint time = 120 2024-04-25T09:52:08.614 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:08.615 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-04-25T09:52:08.615 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-04-25T09:52:08.615 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-04-25T09:52:08.615 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-04-25T09:52:08.615 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:52:08.615 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:52:08.615 INFO:teuthology.orchestra.run.smithi019.stdout: client mount timeout = 600 2024-04-25T09:52:08.615 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-04-25T09:52:08.615 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T09:52:08.615 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = "" 2024-04-25T09:52:08.615 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout:[mds] 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds = 20 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds balancer = 20 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout: mds cache max size = 1000 2024-04-25T09:52:08.616 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug frag = True 2024-04-25T09:52:08.617 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug scatterstat = True 2024-04-25T09:52:08.617 INFO:teuthology.orchestra.run.smithi019.stdout: mds log max segments = 1 2024-04-25T09:52:08.617 INFO:teuthology.orchestra.run.smithi019.stdout: mds op complaint time = 180 2024-04-25T09:52:08.617 INFO:teuthology.orchestra.run.smithi019.stdout: mds verify scatter = True 2024-04-25T09:52:08.617 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T09:52:08.617 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T09:52:08.617 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-04-25T09:52:08.618 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-04-25T09:52:08.619 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.620 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.19,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.621 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-04-25T09:52:08.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-04-25T09:52:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-04-25T09:52:08.624 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T09:52:08.625 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:52:08.626 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: mon op complaint time = 120 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:52:08.627 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout: client mount timeout = 600 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-04-25T09:52:08.628 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds = 20 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds balancer = 20 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: mds cache max size = 1000 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug frag = True 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug scatterstat = True 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: mds log max segments = 1 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: mds op complaint time = 180 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: mds verify scatter = True 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T09:52:08.629 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T09:52:08.631 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T09:52:08.631 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T09:52:08.724 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T09:52:08.724 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:08.724 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T09:52:08.739 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:08.739 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T09:52:08.794 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:52:08.794 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:08.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T09:52:08.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:52:08.868 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:08.868 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:08.923 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T09:52:08.923 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:08.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T09:52:08.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:52:08.962 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:08.963 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:09.020 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T09:52:09.020 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T09:52:09.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T09:52:09.070 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T09:52:09.072 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T09:52:09.153 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T09:52:09.156 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T09:52:09.156 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T09:52:09.205 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T09:52:09.208 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T09:52:09.232 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T09:52:09.322 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T09:52:09.324 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T09:52:09.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T09:52:09.401 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T09:52:09.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T09:52:09.428 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T09:52:09.517 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T09:52:09.520 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T09:52:09.543 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T09:52:09.543 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T09:52:09.583 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T09:52:09.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T09:52:09.676 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T09:52:09.688 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T09:52:09.727 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T09:52:09.739 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T09:52:09.818 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T09:52:09.830 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T09:52:09.830 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T09:52:09.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T09:52:09.855 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:09.855 INFO:tasks.ceph:role: osd.0 2024-04-25T09:52:09.855 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:52:09.855 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T09:52:09.927 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:09.927 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:09.927 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:09.927 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:09.927 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:09.927 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:09.927 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:09.928 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:09.928 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:09.928 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:10.430 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-04-25T09:52:10.432 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-04-25T09:52:10.432 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T09:52:10.564 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T09:52:10.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T09:52:10.662 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:10.662 INFO:tasks.ceph:role: osd.1 2024-04-25T09:52:10.662 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:52:10.662 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T09:52:10.734 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:10.734 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:10.734 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:10.735 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:10.735 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:10.735 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:10.735 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:10.735 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:10.735 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:10.735 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:11.192 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-04-25T09:52:11.194 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-04-25T09:52:11.194 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T09:52:11.291 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T09:52:11.323 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T09:52:11.390 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:11.390 INFO:tasks.ceph:role: osd.2 2024-04-25T09:52:11.391 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:52:11.391 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T09:52:11.464 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:11.465 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:11.465 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:11.465 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:11.465 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:11.465 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:11.465 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:11.465 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:11.465 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:11.465 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:11.970 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-04-25T09:52:11.972 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-04-25T09:52:11.972 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T09:52:12.129 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T09:52:12.162 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T09:52:12.229 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:12.229 INFO:tasks.ceph:role: osd.3 2024-04-25T09:52:12.229 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:52:12.230 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T09:52:12.303 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:12.303 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:12.303 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:12.304 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:12.304 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:12.304 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:12.304 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-04-25T09:52:12.817 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-04-25T09:52:12.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T09:52:12.964 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T09:52:12.997 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:13.083+0000 7f4908118640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T09:52:13.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:13.084+0000 7f4908118640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T09:52:13.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:13.084+0000 7f4908118640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:52:13.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:13.084+0000 7f4908118640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:52:13.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:13.084+0000 7f4908118640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:52:13.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:13.085+0000 7f4908118640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T09:52:15.943 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T09:52:15.972 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:16.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:16.061+0000 7f5e8000c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T09:52:16.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:16.061+0000 7f5e8000c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T09:52:16.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:16.061+0000 7f5e8000c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:52:16.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:16.061+0000 7f5e8000c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:52:16.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:16.061+0000 7f5e8000c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:52:16.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:16.062+0000 7f5e8000c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T09:52:18.916 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T09:52:18.945 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:19.033+0000 7f8235618640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T09:52:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:19.034+0000 7f8235618640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T09:52:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:19.034+0000 7f8235618640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:52:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:19.034+0000 7f8235618640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:52:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:19.034+0000 7f8235618640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:52:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:19.034+0000 7f8235618640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T09:52:21.894 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T09:52:21.923 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:22.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:22.010+0000 7fd7fa20f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T09:52:22.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:22.011+0000 7fd7fa20f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T09:52:22.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:22.011+0000 7fd7fa20f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:52:22.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:22.011+0000 7fd7fa20f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:52:22.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:22.011+0000 7fd7fa20f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:52:22.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:22.011+0000 7fd7fa20f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T09:52:24.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T09:52:24.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T09:52:24.917 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:24.918 INFO:tasks.ceph:role: osd.4 2024-04-25T09:52:24.918 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T09:52:24.918 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T09:52:24.993 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:24.993 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:24.993 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:24.994 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:24.994 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:25.520 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T09:52:25.522 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T09:52:25.522 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T09:52:25.673 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T09:52:25.701 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T09:52:25.765 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:25.765 INFO:tasks.ceph:role: osd.5 2024-04-25T09:52:25.765 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T09:52:25.765 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T09:52:25.837 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:25.837 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:25.837 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:25.837 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:25.838 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:25.838 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:26.281 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T09:52:26.283 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T09:52:26.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T09:52:26.367 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T09:52:26.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T09:52:26.502 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:26.502 INFO:tasks.ceph:role: osd.6 2024-04-25T09:52:26.502 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T09:52:26.502 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T09:52:26.572 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:26.573 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:26.573 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:26.573 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:26.573 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:26.573 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:26.573 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:26.573 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:26.573 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:26.573 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:27.074 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T09:52:27.077 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T09:52:27.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T09:52:27.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T09:52:27.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T09:52:27.306 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:27.306 INFO:tasks.ceph:role: osd.7 2024-04-25T09:52:27.306 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T09:52:27.306 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T09:52:27.377 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:27.377 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:27.377 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:27.377 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:27.377 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:27.377 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:27.377 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:27.378 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:27.378 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:27.378 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:27.883 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T09:52:27.884 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T09:52:27.885 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T09:52:28.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T09:52:28.031 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:28.120+0000 7f5d33814640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T09:52:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:28.120+0000 7f5d33814640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T09:52:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:28.120+0000 7f5d33814640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:52:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:28.120+0000 7f5d33814640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:52:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:28.120+0000 7f5d33814640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:52:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:28.120+0000 7f5d33814640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T09:52:30.973 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T09:52:30.999 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:31.084+0000 7f7a0a1fc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T09:52:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:31.084+0000 7f7a0a1fc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T09:52:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:31.084+0000 7f7a0a1fc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:52:31.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:31.084+0000 7f7a0a1fc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:52:31.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:31.084+0000 7f7a0a1fc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:52:31.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:31.085+0000 7f7a0a1fc640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T09:52:33.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T09:52:33.959 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:34.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:34.044+0000 7f4ba0a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T09:52:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:34.045+0000 7f4ba0a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T09:52:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:34.045+0000 7f4ba0a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:52:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:34.045+0000 7f4ba0a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:52:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:34.045+0000 7f4ba0a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:52:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:34.045+0000 7f4ba0a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T09:52:36.615 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:36.618 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:36.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T09:52:36.926 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:37.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:37.009+0000 7f385ba18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T09:52:37.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:37.009+0000 7f385ba18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T09:52:37.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:37.009+0000 7f385ba18640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:52:37.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:37.009+0000 7f385ba18640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:52:37.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:37.009+0000 7f385ba18640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:52:37.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:37.010+0000 7f385ba18640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T09:52:39.856 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T09:52:39.884 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T09:52:39.884 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:39.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T09:52:39.914 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:39.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T09:52:39.982 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:39.982 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T09:52:40.049 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:40.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T09:52:40.116 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:40.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T09:52:40.182 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:40.182 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T09:52:40.248 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:40.248 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T09:52:40.313 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:40.313 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T09:52:40.341 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:40.341 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T09:52:40.408 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:40.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T09:52:40.475 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:40.475 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T09:52:40.541 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:40.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T09:52:40.607 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:40.607 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T09:52:40.672 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:40.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T09:52:40.737 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:40.737 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T09:52:40.753 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:40.754 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T09:52:40.810 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:40.810 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T09:52:40.826 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:40.826 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T09:52:40.881 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T09:52:40.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T09:52:40.884 DEBUG:teuthology.orchestra.run.smithi112:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T09:52:40.925 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.y] 2024-04-25T09:52:40.925 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg== 2024-04-25T09:52:40.925 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-04-25T09:52:40.925 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g== 2024-04-25T09:52:40.925 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-04-25T09:52:40.925 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ== 2024-04-25T09:52:40.925 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.0] 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ== 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.1] 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ== 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.2] 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw== 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.3] 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA== 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.x] 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig== 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg== 2024-04-25T09:52:40.926 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw== 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.4] 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q== 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.5] 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q== 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.6] 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA== 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.7] 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w== 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout:[client.2] 2024-04-25T09:52:40.927 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA== 2024-04-25T09:52:40.928 INFO:teuthology.orchestra.run.smithi019.stdout:[client.1] 2024-04-25T09:52:40.928 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg== 2024-04-25T09:52:40.928 INFO:teuthology.orchestra.run.smithi019.stdout:[client.3] 2024-04-25T09:52:40.928 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw== 2024-04-25T09:52:40.944 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.y] 2024-04-25T09:52:40.944 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg== 2024-04-25T09:52:40.944 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-04-25T09:52:40.944 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g== 2024-04-25T09:52:40.944 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-04-25T09:52:40.944 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ== 2024-04-25T09:52:40.944 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.0] 2024-04-25T09:52:40.944 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ== 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.1] 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ== 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.2] 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw== 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.3] 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA== 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.x] 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig== 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg== 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-04-25T09:52:40.945 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw== 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.4] 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q== 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.5] 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q== 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.6] 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA== 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.7] 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w== 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout:[client.2] 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA== 2024-04-25T09:52:40.946 INFO:teuthology.orchestra.run.smithi112.stdout:[client.1] 2024-04-25T09:52:40.947 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg== 2024-04-25T09:52:40.947 INFO:teuthology.orchestra.run.smithi112.stdout:[client.3] 2024-04-25T09:52:40.947 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw== 2024-04-25T09:52:40.948 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:52:40.972 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:52:41.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:41.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:41.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:41.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:41.144 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.146 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.195 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.197 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.245 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.298 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:52:41.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:52:41.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:41.400 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:41.451 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:41.453 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:41.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.507 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.556 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.558 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.606 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.609 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.657 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.659 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:41.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:41.709 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:41.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:41.764 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:41.815 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:41.817 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:41.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:41.871 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:41.922 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T09:52:41.922 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T09:52:41.946 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:52:42.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T09:52:42.064 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T09:52:42.088 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:52:42.183 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T09:52:42.206 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T09:52:42.270 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:52:42.363 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T09:52:42.386 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:42.390 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:42.441 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T09:52:42.441 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T09:52:42.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T09:52:42.445 INFO:tasks.ceph.mon.a:Started 2024-04-25T09:52:42.445 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T09:52:42.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T09:52:42.484 INFO:tasks.ceph.mon.b:Started 2024-04-25T09:52:42.484 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T09:52:42.484 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T09:52:42.486 INFO:tasks.ceph.mon.c:Started 2024-04-25T09:52:42.486 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T09:52:42.487 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T09:52:42.487 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T09:52:42.489 INFO:tasks.ceph.mgr.y:Started 2024-04-25T09:52:42.489 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T09:52:42.489 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T09:52:42.491 INFO:tasks.ceph.mgr.x:Started 2024-04-25T09:52:42.491 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T09:52:42.492 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T09:52:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.593+0000 7f393b06b640 1 Processor -- start 2024-04-25T09:52:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.593+0000 7f393b06b640 1 -- start start 2024-04-25T09:52:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f393b06b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f393415dbb0 0x7f393406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f393b06b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cbf00 0x7f393406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f393b06b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f39340cc360 0x7f393406c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f393b06b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39340cae60 con 0x7f393415dbb0 2024-04-25T09:52:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f393b06b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39340cb140 con 0x7f39340cc360 2024-04-25T09:52:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f393b06b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39340cb420 con 0x7f39340cbf00 2024-04-25T09:52:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3938e07640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f393415dbb0 0x7f393406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3938e07640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f393415dbb0 0x7f393406fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41274/0 (socket says 172.21.15.19:41274) 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3938e07640 1 -- 172.21.15.19:0/3660766449 learned_addr learned my addr 172.21.15.19:0/3660766449 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3933fff640 1 --2- 172.21.15.19:0/3660766449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cbf00 0x7f393406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3938e07640 1 -- 172.21.15.19:0/3660766449 >> v2:172.21.15.19:3300/0 conn(0x7f393415dbb0 msgr2=0x7f393406fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3938e07640 1 -- 172.21.15.19:0/3660766449 >> v2:172.21.15.19:3300/0 conn(0x7f393415dbb0 msgr2=0x7f393406fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3938e07640 1 --2- 172.21.15.19:0/3660766449 >> v2:172.21.15.19:3300/0 conn(0x7f393415dbb0 0x7f393406fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3939608640 1 --2- 172.21.15.19:0/3660766449 >> v2:172.21.15.112:3300/0 conn(0x7f39340cc360 0x7f393406c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3938e07640 1 --2- 172.21.15.19:0/3660766449 >> v2:172.21.15.19:3300/0 conn(0x7f393415dbb0 0x7f393406fea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3933fff640 1 -- 172.21.15.19:0/3660766449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cbf00 msgr2=0x7f393406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3933fff640 1 -- 172.21.15.19:0/3660766449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cbf00 msgr2=0x7f393406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.594+0000 7f3933fff640 1 --2- 172.21.15.19:0/3660766449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cbf00 0x7f393406bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.595+0000 7f3933fff640 1 --2- 172.21.15.19:0/3660766449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cbf00 0x7f393406bbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T09:52:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.595+0000 7f3939608640 1 -- 172.21.15.19:0/3660766449 >> v2:172.21.15.112:3300/0 conn(0x7f39340cc360 msgr2=0x7f393406c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T09:52:42.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.595+0000 7f3939608640 1 -- 172.21.15.19:0/3660766449 >> v2:172.21.15.112:3300/0 conn(0x7f39340cc360 msgr2=0x7f393406c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T09:52:42.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.595+0000 7f3939608640 1 --2- 172.21.15.19:0/3660766449 >> v2:172.21.15.112:3300/0 conn(0x7f39340cc360 0x7f393406c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T09:52:42.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.595+0000 7f3939608640 1 --2- 172.21.15.19:0/3660766449 >> v2:172.21.15.112:3300/0 conn(0x7f39340cc360 0x7f393406c130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T09:52:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.795+0000 7f3938e07640 1 --2- 172.21.15.19:0/3660766449 >> v2:172.21.15.19:3300/0 conn(0x7f393415dbb0 0x7f393406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.795+0000 7f3938e07640 1 -- 172.21.15.19:0/3660766449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cbf00 msgr2=0x7f393406bbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:52:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.795+0000 7f3938e07640 1 --2- 172.21.15.19:0/3660766449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cbf00 0x7f393406bbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.795+0000 7f3938e07640 1 -- 172.21.15.19:0/3660766449 >> v2:172.21.15.112:3300/0 conn(0x7f39340cc360 msgr2=0x7f393406c130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:52:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.795+0000 7f3938e07640 1 --2- 172.21.15.19:0/3660766449 >> v2:172.21.15.112:3300/0 conn(0x7f39340cc360 0x7f393406c130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.795+0000 7f3938e07640 1 -- 172.21.15.19:0/3660766449 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39340cb700 con 0x7f393415dbb0 2024-04-25T09:52:42.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.796+0000 7f3938e07640 1 --2- 172.21.15.19:0/3660766449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393415dbb0 0x7f393406fea0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f39341627e0 tx=0x7f392802a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c094b1aa41e8af2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.796+0000 7f39337fe640 1 -- 172.21.15.19:0/3660766449 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392800f040 con 0x7f393415dbb0 2024-04-25T09:52:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.796+0000 7f39337fe640 1 -- 172.21.15.19:0/3660766449 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392802f710 con 0x7f393415dbb0 2024-04-25T09:52:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.796+0000 7f39337fe640 1 -- 172.21.15.19:0/3660766449 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392802fb00 con 0x7f393415dbb0 2024-04-25T09:52:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.797+0000 7f393b06b640 1 -- 172.21.15.19:0/3660766449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393415dbb0 msgr2=0x7f393406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.797+0000 7f393b06b640 1 --2- 172.21.15.19:0/3660766449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393415dbb0 0x7f393406fea0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f39341627e0 tx=0x7f392802a490 comp rx=0 tx=0).stop 2024-04-25T09:52:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.797+0000 7f393b06b640 1 -- 172.21.15.19:0/3660766449 shutdown_connections 2024-04-25T09:52:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.797+0000 7f393b06b640 1 --2- 172.21.15.19:0/3660766449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cbf00 0x7f393406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.797+0000 7f393b06b640 1 --2- 172.21.15.19:0/3660766449 >> v2:172.21.15.112:3300/0 conn(0x7f39340cc360 0x7f393406c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.797+0000 7f393b06b640 1 --2- 172.21.15.19:0/3660766449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393415dbb0 0x7f393406fea0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.797+0000 7f393b06b640 1 -- 172.21.15.19:0/3660766449 >> 172.21.15.19:0/3660766449 conn(0x7f3934071af0 msgr2=0x7f3934071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.797+0000 7f393b06b640 1 -- 172.21.15.19:0/3660766449 shutdown_connections 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.797+0000 7f393b06b640 1 -- 172.21.15.19:0/3660766449 wait complete. 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.798+0000 7f393b06b640 1 Processor -- start 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.798+0000 7f393b06b640 1 -- start start 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f393b06b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39340cbf00 0x7f393414ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f393b06b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cc360 0x7f39340c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f393b06b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393415dbb0 0x7f393406ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f393b06b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39340cab10 con 0x7f39340cbf00 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f393b06b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393415df90 con 0x7f393415dbb0 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f393b06b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39340703e0 con 0x7f39340cc360 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f3938e07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39340cbf00 0x7f393414ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f3938e07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39340cbf00 0x7f393414ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41318/0 (socket says 172.21.15.19:41318) 2024-04-25T09:52:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f3938e07640 1 -- 172.21.15.19:0/1811043485 learned_addr learned my addr 172.21.15.19:0/1811043485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f3933fff640 1 --2- 172.21.15.19:0/1811043485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cc360 0x7f39340c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f3939608640 1 --2- 172.21.15.19:0/1811043485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393415dbb0 0x7f393406ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f3938e07640 1 -- 172.21.15.19:0/1811043485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cc360 msgr2=0x7f39340c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f3938e07640 1 --2- 172.21.15.19:0/1811043485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cc360 0x7f39340c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f3938e07640 1 -- 172.21.15.19:0/1811043485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393415dbb0 msgr2=0x7f393406ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f3938e07640 1 --2- 172.21.15.19:0/1811043485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393415dbb0 0x7f393406ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f3938e07640 1 -- 172.21.15.19:0/1811043485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39340c8d60 con 0x7f39340cbf00 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.799+0000 7f3938e07640 1 --2- 172.21.15.19:0/1811043485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39340cbf00 0x7f393414ea50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3928009980 tx=0x7f3928004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.800+0000 7f39317fa640 1 -- 172.21.15.19:0/1811043485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392800f000 con 0x7f39340cbf00 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.800+0000 7f39317fa640 1 -- 172.21.15.19:0/1811043485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3928003820 con 0x7f39340cbf00 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.800+0000 7f393b06b640 1 -- 172.21.15.19:0/1811043485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39340c9040 con 0x7f39340cbf00 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.800+0000 7f39317fa640 1 -- 172.21.15.19:0/1811043485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928003bf0 con 0x7f39340cbf00 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.800+0000 7f393b06b640 1 -- 172.21.15.19:0/1811043485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39340c9600 con 0x7f39340cbf00 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.800+0000 7f39317fa640 1 -- 172.21.15.19:0/1811043485 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3928039040 con 0x7f39340cbf00 2024-04-25T09:52:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.800+0000 7f39317fa640 1 -- 172.21.15.19:0/1811043485 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f392803cdf0 con 0x7f39340cbf00 2024-04-25T09:52:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.801+0000 7f393b06b640 1 -- 172.21.15.19:0/1811043485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3934073a00 con 0x7f39340cbf00 2024-04-25T09:52:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.803+0000 7f39317fa640 1 -- 172.21.15.19:0/1811043485 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3928040030 con 0x7f39340cbf00 2024-04-25T09:52:42.871 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:42.869+0000 7f7531021040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T09:52:42.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.870+0000 7f393b06b640 1 -- 172.21.15.19:0/1811043485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3934061b50 con 0x7f39340cbf00 2024-04-25T09:52:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.872+0000 7f39317fa640 1 -- 172.21.15.19:0/1811043485 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3928003d90 con 0x7f39340cbf00 2024-04-25T09:52:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-04-25T09:52:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.873+0000 7f393b06b640 1 -- 172.21.15.19:0/1811043485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39340cbf00 msgr2=0x7f393414ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.873+0000 7f393b06b640 1 --2- 172.21.15.19:0/1811043485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39340cbf00 0x7f393414ea50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3928009980 tx=0x7f3928004730 comp rx=0 tx=0).stop 2024-04-25T09:52:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.873+0000 7f393b06b640 1 -- 172.21.15.19:0/1811043485 shutdown_connections 2024-04-25T09:52:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.873+0000 7f393b06b640 1 --2- 172.21.15.19:0/1811043485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393415dbb0 0x7f393406ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.873+0000 7f393b06b640 1 --2- 172.21.15.19:0/1811043485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39340cc360 0x7f39340c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.873+0000 7f393b06b640 1 --2- 172.21.15.19:0/1811043485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39340cbf00 0x7f393414ea50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.873+0000 7f393b06b640 1 -- 172.21.15.19:0/1811043485 >> 172.21.15.19:0/1811043485 conn(0x7f3934071af0 msgr2=0x7f393415c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.873+0000 7f393b06b640 1 -- 172.21.15.19:0/1811043485 shutdown_connections 2024-04-25T09:52:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:42.873+0000 7f393b06b640 1 -- 172.21.15.19:0/1811043485 wait complete. 2024-04-25T09:52:42.876 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:42.875+0000 7fbc99722040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T09:52:42.885 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T09:52:42.885 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T09:52:42.885 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:42.885 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T09:52:42.910 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:42.910 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T09:52:42.975 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:42.975 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T09:52:43.019 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:43.018+0000 7f7531021040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:52:43.040 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:52:43.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T09:52:43.041 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:43.040+0000 7fbc99722040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:52:43.065 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:43.065 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T09:52:43.073 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:43.071+0000 7f7531021040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T09:52:43.089 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:43.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T09:52:43.098 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:43.097+0000 7fbc99722040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T09:52:43.152 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:43.152 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T09:52:43.215 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:52:43.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T09:52:43.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 131c238a-d117-470a-a62f-2589738cf304 0 2024-04-25T09:52:43.317 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:43.315+0000 7f7531021040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T09:52:43.347 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:43.346+0000 7fbc99722040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T09:52:43.365 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:43.363+0000 7f7531021040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T09:52:43.394 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:43.393+0000 7fbc99722040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T09:52:43.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.412+0000 7f167acfc640 1 Processor -- start 2024-04-25T09:52:43.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.412+0000 7f167acfc640 1 -- start start 2024-04-25T09:52:43.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f167acfc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f167415a9f0 0x7f16740c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f167acfc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cdbf0 0x7f16740ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f167acfc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16740c6940 con 0x7f16740cd520 2024-04-25T09:52:43.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f167acfc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16740b0020 con 0x7f167415a9f0 2024-04-25T09:52:43.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f167acfc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16740cb5a0 con 0x7f16740cdbf0 2024-04-25T09:52:43.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f1678a98640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f167415a9f0 0x7f16740c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f1673fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cdbf0 0x7f16740ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f1678a98640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f167415a9f0 0x7f16740c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34548/0 (socket says 172.21.15.112:34548) 2024-04-25T09:52:43.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f1673fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cdbf0 0x7f16740ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:52448/0 (socket says 172.21.15.112:52448) 2024-04-25T09:52:43.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f1678a98640 1 -- 172.21.15.112:0/1988173830 learned_addr learned my addr 172.21.15.112:0/1988173830 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:43.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f1673fff640 1 -- 172.21.15.112:0/1988173830 >> v2:172.21.15.112:3300/0 conn(0x7f167415a9f0 msgr2=0x7f16740c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:43.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f1673fff640 1 --2- 172.21.15.112:0/1988173830 >> v2:172.21.15.112:3300/0 conn(0x7f167415a9f0 0x7f16740c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f1673fff640 1 -- 172.21.15.112:0/1988173830 >> v1:172.21.15.19:6789/0 conn(0x7f16740cd520 legacy=0x7f16740ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f1673fff640 1 -- 172.21.15.112:0/1988173830 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167415add0 con 0x7f16740cdbf0 2024-04-25T09:52:43.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.413+0000 7f1678a98640 1 --2- 172.21.15.112:0/1988173830 >> v2:172.21.15.112:3300/0 conn(0x7f167415a9f0 0x7f16740c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:43.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.414+0000 7f1673fff640 1 --2- 172.21.15.112:0/1988173830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cdbf0 0x7f16740ccfe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f16600097a0 tx=0x7f166002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c493c1e96ef3e420 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:43.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.414+0000 7f16737fe640 1 -- 172.21.15.112:0/1988173830 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660037070 con 0x7f16740cdbf0 2024-04-25T09:52:43.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.414+0000 7f16737fe640 1 -- 172.21.15.112:0/1988173830 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1660032440 con 0x7f16740cdbf0 2024-04-25T09:52:43.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.414+0000 7f16737fe640 1 -- 172.21.15.112:0/1988173830 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660032810 con 0x7f16740cdbf0 2024-04-25T09:52:43.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.415+0000 7f167acfc640 1 -- 172.21.15.112:0/1988173830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cdbf0 msgr2=0x7f16740ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.415+0000 7f167acfc640 1 --2- 172.21.15.112:0/1988173830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cdbf0 0x7f16740ccfe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f16600097a0 tx=0x7f166002a180 comp rx=0 tx=0).stop 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.415+0000 7f167acfc640 1 -- 172.21.15.112:0/1988173830 shutdown_connections 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.415+0000 7f167acfc640 1 --2- 172.21.15.112:0/1988173830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cdbf0 0x7f16740ccfe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.415+0000 7f167acfc640 1 --2- 172.21.15.112:0/1988173830 >> v2:172.21.15.112:3300/0 conn(0x7f167415a9f0 0x7f16740c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.415+0000 7f167acfc640 1 -- 172.21.15.112:0/1988173830 >> 172.21.15.112:0/1988173830 conn(0x7f16740b2390 msgr2=0x7f16740b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.415+0000 7f167acfc640 1 -- 172.21.15.112:0/1988173830 shutdown_connections 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.415+0000 7f167acfc640 1 -- 172.21.15.112:0/1988173830 wait complete. 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.416+0000 7f167acfc640 1 Processor -- start 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.416+0000 7f167acfc640 1 -- start start 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f167acfc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cd520 0x7f16740af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f167acfc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16740cdbf0 0x7f1674149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f167acfc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167415a9f0 0x7f16740c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f167acfc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16740cb230 con 0x7f16740cdbf0 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f167acfc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16740caf50 con 0x7f167415a9f0 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f167acfc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16740ce070 con 0x7f16740cd520 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f1678a98640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cd520 0x7f16740af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f1678a98640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cd520 0x7f16740af820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:52456/0 (socket says 172.21.15.112:52456) 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f1678a98640 1 -- 172.21.15.112:0/2330502247 learned_addr learned my addr 172.21.15.112:0/2330502247 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f1679299640 1 --2- 172.21.15.112:0/2330502247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167415a9f0 0x7f16740c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f1673fff640 1 --2- 172.21.15.112:0/2330502247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16740cdbf0 0x7f1674149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f1678a98640 1 -- 172.21.15.112:0/2330502247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167415a9f0 msgr2=0x7f16740c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f1678a98640 1 --2- 172.21.15.112:0/2330502247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167415a9f0 0x7f16740c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f1678a98640 1 -- 172.21.15.112:0/2330502247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16740cdbf0 msgr2=0x7f1674149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f1678a98640 1 --2- 172.21.15.112:0/2330502247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16740cdbf0 0x7f1674149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f1678a98640 1 -- 172.21.15.112:0/2330502247 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16740b3620 con 0x7f16740cd520 2024-04-25T09:52:43.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.417+0000 7f1678a98640 1 --2- 172.21.15.112:0/2330502247 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cd520 0x7f16740af820 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f166800e9b0 tx=0x7f166800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:43.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.418+0000 7f16717fa640 1 -- 172.21.15.112:0/2330502247 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668017070 con 0x7f16740cd520 2024-04-25T09:52:43.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.418+0000 7f16717fa640 1 -- 172.21.15.112:0/2330502247 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166800c9a0 con 0x7f16740cd520 2024-04-25T09:52:43.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.418+0000 7f167acfc640 1 -- 172.21.15.112:0/2330502247 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16740af030 con 0x7f16740cd520 2024-04-25T09:52:43.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.418+0000 7f16717fa640 1 -- 172.21.15.112:0/2330502247 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166800cd50 con 0x7f16740cd520 2024-04-25T09:52:43.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.418+0000 7f167acfc640 1 -- 172.21.15.112:0/2330502247 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16740af300 con 0x7f16740cd520 2024-04-25T09:52:43.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.418+0000 7f16717fa640 1 -- 172.21.15.112:0/2330502247 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f166801a070 con 0x7f16740cd520 2024-04-25T09:52:43.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.418+0000 7f16717fa640 1 -- 172.21.15.112:0/2330502247 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1668013470 con 0x7f16740cd520 2024-04-25T09:52:43.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.419+0000 7f167acfc640 1 -- 172.21.15.112:0/2330502247 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167406cc80 con 0x7f16740cd520 2024-04-25T09:52:43.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.421+0000 7f16717fa640 1 -- 172.21.15.112:0/2330502247 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1668005990 con 0x7f16740cd520 2024-04-25T09:52:43.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.493+0000 7f167acfc640 1 -- 172.21.15.112:0/2330502247 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "131c238a-d117-470a-a62f-2589738cf304", "id": 0} v 0) v1 -- 0x7f1674073a00 con 0x7f16740cd520 2024-04-25T09:52:43.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.499+0000 7f16717fa640 1 -- 172.21.15.112:0/2330502247 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "131c238a-d117-470a-a62f-2589738cf304", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1668005b70 con 0x7f16740cd520 2024-04-25T09:52:43.501 INFO:teuthology.orchestra.run.smithi112.stdout:0 2024-04-25T09:52:43.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.500+0000 7f167acfc640 1 -- 172.21.15.112:0/2330502247 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cd520 msgr2=0x7f16740af820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.500+0000 7f167acfc640 1 --2- 172.21.15.112:0/2330502247 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cd520 0x7f16740af820 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f166800e9b0 tx=0x7f166800ee70 comp rx=0 tx=0).stop 2024-04-25T09:52:43.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.500+0000 7f167acfc640 1 -- 172.21.15.112:0/2330502247 shutdown_connections 2024-04-25T09:52:43.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.501+0000 7f167acfc640 1 --2- 172.21.15.112:0/2330502247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f167415a9f0 0x7f16740c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.501+0000 7f167acfc640 1 --2- 172.21.15.112:0/2330502247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16740cdbf0 0x7f1674149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.501+0000 7f167acfc640 1 --2- 172.21.15.112:0/2330502247 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16740cd520 0x7f16740af820 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.501+0000 7f167acfc640 1 -- 172.21.15.112:0/2330502247 >> 172.21.15.112:0/2330502247 conn(0x7f16740b2390 msgr2=0x7f167406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:43.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.501+0000 7f167acfc640 1 -- 172.21.15.112:0/2330502247 shutdown_connections 2024-04-25T09:52:43.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.501+0000 7f167acfc640 1 -- 172.21.15.112:0/2330502247 wait complete. 2024-04-25T09:52:43.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 3a714f50-4bcb-4995-b6b4-788924493ae3 1 2024-04-25T09:52:43.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.609+0000 7f7f9144d640 1 Processor -- start 2024-04-25T09:52:43.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.609+0000 7f7f9144d640 1 -- start start 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.609+0000 7f7f9144d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 0x7f7f8c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.609+0000 7f7f9144d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f8c161710 0x7f7f8c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.609+0000 7f7f9144d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c15e000 con 0x7f7f8c0b9330 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.609+0000 7f7f9144d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8c0d1de0 con 0x7f7f8c161710 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.609+0000 7f7f9144d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c161c60 con 0x7f7f8c15dc20 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.610+0000 7f7f8a7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f8c161710 0x7f7f8c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.610+0000 7f7f8affd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 0x7f7f8c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.610+0000 7f7f8b7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7f8c0b9330 0x7f7f8c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57510/0 (socket says 172.21.15.112:57510) 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.610+0000 7f7f8b7fe640 1 -- 172.21.15.112:0/4155838852 learned_addr learned my addr 172.21.15.112:0/4155838852 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.610+0000 7f7f8affd640 1 -- 172.21.15.112:0/4155838852 >> v2:172.21.15.112:3300/0 conn(0x7f7f8c161710 msgr2=0x7f7f8c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.610+0000 7f7f8affd640 1 --2- 172.21.15.112:0/4155838852 >> v2:172.21.15.112:3300/0 conn(0x7f7f8c161710 0x7f7f8c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.610+0000 7f7f8affd640 1 -- 172.21.15.112:0/4155838852 >> v1:172.21.15.112:6789/0 conn(0x7f7f8c0b9330 legacy=0x7f7f8c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.610+0000 7f7f8affd640 1 -- 172.21.15.112:0/4155838852 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c161f40 con 0x7f7f8c15dc20 2024-04-25T09:52:43.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.610+0000 7f7f89ffb640 1 -- 172.21.15.112:0/4155838852 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195441759 0 0) 0x7f7f8c15e000 con 0x7f7f8c0b9330 2024-04-25T09:52:43.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.610+0000 7f7f8a7fc640 1 --2- 172.21.15.112:0/4155838852 >> v2:172.21.15.112:3300/0 conn(0x7f7f8c161710 0x7f7f8c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:43.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f8affd640 1 --2- 172.21.15.112:0/4155838852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 0x7f7f8c0ce670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0097a0 tx=0x7f7f7c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acf2e811cafac826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:43.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f89ffb640 1 -- 172.21.15.112:0/4155838852 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c037070 con 0x7f7f8c15dc20 2024-04-25T09:52:43.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f89ffb640 1 -- 172.21.15.112:0/4155838852 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7c032440 con 0x7f7f8c15dc20 2024-04-25T09:52:43.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f89ffb640 1 -- 172.21.15.112:0/4155838852 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c032810 con 0x7f7f8c15dc20 2024-04-25T09:52:43.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f9144d640 1 -- 172.21.15.112:0/4155838852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 msgr2=0x7f7f8c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f9144d640 1 --2- 172.21.15.112:0/4155838852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 0x7f7f8c0ce670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0097a0 tx=0x7f7f7c02a180 comp rx=0 tx=0).stop 2024-04-25T09:52:43.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f9144d640 1 -- 172.21.15.112:0/4155838852 shutdown_connections 2024-04-25T09:52:43.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f9144d640 1 --2- 172.21.15.112:0/4155838852 >> v2:172.21.15.112:3300/0 conn(0x7f7f8c161710 0x7f7f8c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f9144d640 1 --2- 172.21.15.112:0/4155838852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 0x7f7f8c0ce670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f9144d640 1 -- 172.21.15.112:0/4155838852 >> 172.21.15.112:0/4155838852 conn(0x7f7f8c0d0420 msgr2=0x7f7f8c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f9144d640 1 -- 172.21.15.112:0/4155838852 shutdown_connections 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.611+0000 7f7f9144d640 1 -- 172.21.15.112:0/4155838852 wait complete. 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.613+0000 7f7f9144d640 1 Processor -- start 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.613+0000 7f7f9144d640 1 -- start start 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.613+0000 7f7f9144d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f8c0b9330 0x7f7f8c15e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.613+0000 7f7f9144d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 0x7f7f8c0acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.613+0000 7f7f9144d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c161710 0x7f7f8c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.613+0000 7f7f9144d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0b5e50 con 0x7f7f8c161710 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.613+0000 7f7f9144d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c162500 con 0x7f7f8c0b9330 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.613+0000 7f7f9144d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0b5fd0 con 0x7f7f8c15dc20 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.613+0000 7f7f8affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f8c0b9330 0x7f7f8c15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.613+0000 7f7f8a7fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 0x7f7f8c0acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f8a7fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 0x7f7f8c0acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:52476/0 (socket says 172.21.15.112:52476) 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f8affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f8c0b9330 0x7f7f8c15e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34568/0 (socket says 172.21.15.112:34568) 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f8a7fc640 1 -- 172.21.15.112:0/3198206690 learned_addr learned my addr 172.21.15.112:0/3198206690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f8b7fe640 1 --2- 172.21.15.112:0/3198206690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c161710 0x7f7f8c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f8affd640 1 -- 172.21.15.112:0/3198206690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 msgr2=0x7f7f8c0acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f8affd640 1 --2- 172.21.15.112:0/3198206690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 0x7f7f8c0acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f8affd640 1 -- 172.21.15.112:0/3198206690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c161710 msgr2=0x7f7f8c15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f8affd640 1 --2- 172.21.15.112:0/3198206690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c161710 0x7f7f8c15c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f8affd640 1 -- 172.21.15.112:0/3198206690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c15e6c0 con 0x7f7f8c0b9330 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f8b7fe640 1 --2- 172.21.15.112:0/3198206690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c161710 0x7f7f8c15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f8affd640 1 --2- 172.21.15.112:0/3198206690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f8c0b9330 0x7f7f8c15e180 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c031000 tx=0x7f7f7c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f737fe640 1 -- 172.21.15.112:0/3198206690 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c041030 con 0x7f7f8c0b9330 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f737fe640 1 -- 172.21.15.112:0/3198206690 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7c030500 con 0x7f7f8c0b9330 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f9144d640 1 -- 172.21.15.112:0/3198206690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c15e990 con 0x7f7f8c0b9330 2024-04-25T09:52:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f737fe640 1 -- 172.21.15.112:0/3198206690 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c037040 con 0x7f7f8c0b9330 2024-04-25T09:52:43.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f9144d640 1 -- 172.21.15.112:0/3198206690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c15ef50 con 0x7f7f8c0b9330 2024-04-25T09:52:43.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f737fe640 1 -- 172.21.15.112:0/3198206690 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7f7c044070 con 0x7f7f8c0b9330 2024-04-25T09:52:43.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.614+0000 7f7f737fe640 1 -- 172.21.15.112:0/3198206690 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f7f7c03d6d0 con 0x7f7f8c0b9330 2024-04-25T09:52:43.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.615+0000 7f7f9144d640 1 -- 172.21.15.112:0/3198206690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8c073a00 con 0x7f7f8c0b9330 2024-04-25T09:52:43.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.619+0000 7f7f737fe640 1 -- 172.21.15.112:0/3198206690 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7f7c0308d0 con 0x7f7f8c0b9330 2024-04-25T09:52:43.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.691+0000 7f7f9144d640 1 -- 172.21.15.112:0/3198206690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3a714f50-4bcb-4995-b6b4-788924493ae3", "id": 1} v 0) v1 -- 0x7f7f8c0b8480 con 0x7f7f8c0b9330 2024-04-25T09:52:43.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.695+0000 7f7f737fe640 1 -- 172.21.15.112:0/3198206690 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3a714f50-4bcb-4995-b6b4-788924493ae3", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7f7c030ab0 con 0x7f7f8c0b9330 2024-04-25T09:52:43.697 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-04-25T09:52:43.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.696+0000 7f7f9144d640 1 -- 172.21.15.112:0/3198206690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f8c0b9330 msgr2=0x7f7f8c15e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.696+0000 7f7f9144d640 1 --2- 172.21.15.112:0/3198206690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f8c0b9330 0x7f7f8c15e180 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c031000 tx=0x7f7f7c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.696+0000 7f7f9144d640 1 -- 172.21.15.112:0/3198206690 shutdown_connections 2024-04-25T09:52:43.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.696+0000 7f7f9144d640 1 --2- 172.21.15.112:0/3198206690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c161710 0x7f7f8c15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.696+0000 7f7f9144d640 1 --2- 172.21.15.112:0/3198206690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f8c15dc20 0x7f7f8c0acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.696+0000 7f7f9144d640 1 --2- 172.21.15.112:0/3198206690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f8c0b9330 0x7f7f8c15e180 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.696+0000 7f7f9144d640 1 -- 172.21.15.112:0/3198206690 >> 172.21.15.112:0/3198206690 conn(0x7f7f8c0d0420 msgr2=0x7f7f8c0d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:43.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.696+0000 7f7f9144d640 1 -- 172.21.15.112:0/3198206690 shutdown_connections 2024-04-25T09:52:43.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.696+0000 7f7f9144d640 1 -- 172.21.15.112:0/3198206690 wait complete. 2024-04-25T09:52:43.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new d6c10f87-21de-44ad-bbf7-9a886f78550f 2 2024-04-25T09:52:43.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.804+0000 7fc49eb90640 1 Processor -- start 2024-04-25T09:52:43.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.804+0000 7fc49eb90640 1 -- start start 2024-04-25T09:52:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.805+0000 7fc49eb90640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49815de80 0x7fc4980cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.805+0000 7fc49eb90640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4980cefb0 0x7fc4980cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.805+0000 7fc49eb90640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4980afc80 con 0x7fc4980cf950 2024-04-25T09:52:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.805+0000 7fc49eb90640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4980ce4d0 con 0x7fc4980cefb0 2024-04-25T09:52:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.805+0000 7fc49eb90640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980ce7b0 con 0x7fc49815de80 2024-04-25T09:52:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.805+0000 7fc49c92c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49815de80 0x7fc4980cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.805+0000 7fc49c92c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49815de80 0x7fc4980cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:52486/0 (socket says 172.21.15.112:52486) 2024-04-25T09:52:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.805+0000 7fc49c92c640 1 -- 172.21.15.112:0/2229195460 learned_addr learned my addr 172.21.15.112:0/2229195460 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.805+0000 7fc497fff640 1 --2- 172.21.15.112:0/2229195460 >> v2:172.21.15.112:3300/0 conn(0x7fc4980cefb0 0x7fc4980cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.806+0000 7fc49c92c640 1 -- 172.21.15.112:0/2229195460 >> v2:172.21.15.112:3300/0 conn(0x7fc4980cefb0 msgr2=0x7fc4980cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.806+0000 7fc49c92c640 1 --2- 172.21.15.112:0/2229195460 >> v2:172.21.15.112:3300/0 conn(0x7fc4980cefb0 0x7fc4980cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.806+0000 7fc49c92c640 1 -- 172.21.15.112:0/2229195460 >> v1:172.21.15.112:6789/0 conn(0x7fc4980cf950 legacy=0x7fc49806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.806+0000 7fc4977fe640 1 -- 172.21.15.112:0/2229195460 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706013660 0 0) 0x7fc4980afc80 con 0x7fc4980cf950 2024-04-25T09:52:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.806+0000 7fc49c92c640 1 -- 172.21.15.112:0/2229195460 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4980cea90 con 0x7fc49815de80 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.806+0000 7fc497fff640 1 --2- 172.21.15.112:0/2229195460 >> v2:172.21.15.112:3300/0 conn(0x7fc4980cefb0 0x7fc4980cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.806+0000 7fc49c92c640 1 --2- 172.21.15.112:0/2229195460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49815de80 0x7fc4980cadb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc488009870 tx=0x7fc48802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc602978e940c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.806+0000 7fc4977fe640 1 -- 172.21.15.112:0/2229195460 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488037070 con 0x7fc49815de80 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.807+0000 7fc4977fe640 1 -- 172.21.15.112:0/2229195460 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc488032440 con 0x7fc49815de80 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.807+0000 7fc4977fe640 1 -- 172.21.15.112:0/2229195460 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488032810 con 0x7fc49815de80 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.807+0000 7fc49eb90640 1 -- 172.21.15.112:0/2229195460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49815de80 msgr2=0x7fc4980cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.807+0000 7fc49eb90640 1 --2- 172.21.15.112:0/2229195460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49815de80 0x7fc4980cadb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc488009870 tx=0x7fc48802a270 comp rx=0 tx=0).stop 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.807+0000 7fc49eb90640 1 -- 172.21.15.112:0/2229195460 shutdown_connections 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.807+0000 7fc49eb90640 1 --2- 172.21.15.112:0/2229195460 >> v2:172.21.15.112:3300/0 conn(0x7fc4980cefb0 0x7fc4980cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.807+0000 7fc49eb90640 1 --2- 172.21.15.112:0/2229195460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49815de80 0x7fc4980cadb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.807+0000 7fc49eb90640 1 -- 172.21.15.112:0/2229195460 >> 172.21.15.112:0/2229195460 conn(0x7fc4980b2390 msgr2=0x7fc4980b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.807+0000 7fc49eb90640 1 -- 172.21.15.112:0/2229195460 shutdown_connections 2024-04-25T09:52:43.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.807+0000 7fc49eb90640 1 -- 172.21.15.112:0/2229195460 wait complete. 2024-04-25T09:52:43.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.808+0000 7fc49eb90640 1 Processor -- start 2024-04-25T09:52:43.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.808+0000 7fc49eb90640 1 -- start start 2024-04-25T09:52:43.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.809+0000 7fc49eb90640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4980cefb0 0x7fc4981f0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.809+0000 7fc49eb90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4980cf950 0x7fc4980c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.809+0000 7fc49eb90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49815de80 0x7fc4980b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:43.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.809+0000 7fc49eb90640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980b3030 con 0x7fc4980cf950 2024-04-25T09:52:43.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.809+0000 7fc49eb90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980b2eb0 con 0x7fc49815de80 2024-04-25T09:52:43.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.809+0000 7fc49eb90640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980c5600 con 0x7fc4980cefb0 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.809+0000 7fc49c92c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4980cefb0 0x7fc4981f0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.809+0000 7fc49c92c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4980cefb0 0x7fc4981f0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:52490/0 (socket says 172.21.15.112:52490) 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.809+0000 7fc49d12d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49815de80 0x7fc4980b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.809+0000 7fc49c92c640 1 -- 172.21.15.112:0/596559499 learned_addr learned my addr 172.21.15.112:0/596559499 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.809+0000 7fc497fff640 1 --2- 172.21.15.112:0/596559499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4980cf950 0x7fc4980c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc49c92c640 1 -- 172.21.15.112:0/596559499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49815de80 msgr2=0x7fc4980b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc49c92c640 1 --2- 172.21.15.112:0/596559499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49815de80 0x7fc4980b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc49c92c640 1 -- 172.21.15.112:0/596559499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4980cf950 msgr2=0x7fc4980c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc49c92c640 1 --2- 172.21.15.112:0/596559499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4980cf950 0x7fc4980c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc49c92c640 1 -- 172.21.15.112:0/596559499 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4980ad600 con 0x7fc4980cefb0 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc49d12d640 1 --2- 172.21.15.112:0/596559499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49815de80 0x7fc4980b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc49c92c640 1 --2- 172.21.15.112:0/596559499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4980cefb0 0x7fc4981f0bc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc48802a2b0 tx=0x7fc4880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc4957fa640 1 -- 172.21.15.112:0/596559499 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488041030 con 0x7fc4980cefb0 2024-04-25T09:52:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc4957fa640 1 -- 172.21.15.112:0/596559499 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc488030440 con 0x7fc4980cefb0 2024-04-25T09:52:43.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc49eb90640 1 -- 172.21.15.112:0/596559499 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4980b4260 con 0x7fc4980cefb0 2024-04-25T09:52:43.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc4957fa640 1 -- 172.21.15.112:0/596559499 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488037040 con 0x7fc4980cefb0 2024-04-25T09:52:43.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc49eb90640 1 -- 172.21.15.112:0/596559499 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4981f7060 con 0x7fc4980cefb0 2024-04-25T09:52:43.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc4957fa640 1 -- 172.21.15.112:0/596559499 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc488044070 con 0x7fc4980cefb0 2024-04-25T09:52:43.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.810+0000 7fc4957fa640 1 -- 172.21.15.112:0/596559499 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fc48803dac0 con 0x7fc4980cefb0 2024-04-25T09:52:43.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.811+0000 7fc49eb90640 1 -- 172.21.15.112:0/596559499 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4980c6560 con 0x7fc4980cefb0 2024-04-25T09:52:43.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.813+0000 7fc4957fa640 1 -- 172.21.15.112:0/596559499 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc4980c6560 con 0x7fc4980cefb0 2024-04-25T09:52:43.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.886+0000 7fc49eb90640 1 -- 172.21.15.112:0/596559499 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d6c10f87-21de-44ad-bbf7-9a886f78550f", "id": 2} v 0) v1 -- 0x7fc498073a00 con 0x7fc4980cefb0 2024-04-25T09:52:43.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.889+0000 7fc4957fa640 1 -- 172.21.15.112:0/596559499 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d6c10f87-21de-44ad-bbf7-9a886f78550f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc48803dcc0 con 0x7fc4980cefb0 2024-04-25T09:52:43.890 INFO:teuthology.orchestra.run.smithi112.stdout:2 2024-04-25T09:52:43.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.890+0000 7fc49eb90640 1 -- 172.21.15.112:0/596559499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4980cefb0 msgr2=0x7fc4981f0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:43.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.890+0000 7fc49eb90640 1 --2- 172.21.15.112:0/596559499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4980cefb0 0x7fc4981f0bc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc48802a2b0 tx=0x7fc4880046c0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.890+0000 7fc49eb90640 1 -- 172.21.15.112:0/596559499 shutdown_connections 2024-04-25T09:52:43.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.890+0000 7fc49eb90640 1 --2- 172.21.15.112:0/596559499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49815de80 0x7fc4980b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.890+0000 7fc49eb90640 1 --2- 172.21.15.112:0/596559499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4980cf950 0x7fc4980c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.890+0000 7fc49eb90640 1 --2- 172.21.15.112:0/596559499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4980cefb0 0x7fc4981f0bc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:43.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.890+0000 7fc49eb90640 1 -- 172.21.15.112:0/596559499 >> 172.21.15.112:0/596559499 conn(0x7fc4980b2390 msgr2=0x7fc49815cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:43.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.890+0000 7fc49eb90640 1 -- 172.21.15.112:0/596559499 shutdown_connections 2024-04-25T09:52:43.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.890+0000 7fc49eb90640 1 -- 172.21.15.112:0/596559499 wait complete. 2024-04-25T09:52:43.904 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 0726e0ce-2ffe-41e8-a69f-7a8274edaf6b 3 2024-04-25T09:52:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.999+0000 7f7776e46640 1 Processor -- start 2024-04-25T09:52:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:43.999+0000 7f7776e46640 1 -- start start 2024-04-25T09:52:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f7776e46640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f777015de60 0x7f77700c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f7776e46640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f777006fc50 0x7f777015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f7776e46640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77700704f0 con 0x7f77700cbee0 2024-04-25T09:52:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f7776e46640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77700ca540 con 0x7f777015de60 2024-04-25T09:52:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f7776e46640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77700ca820 con 0x7f777006fc50 2024-04-25T09:52:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f776ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f777006fc50 0x7f777015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f776ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f777006fc50 0x7f777015e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34600/0 (socket says 172.21.15.112:34600) 2024-04-25T09:52:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f77753e3640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f77700cbee0 0x7f777006bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57524/0 (socket says 172.21.15.112:57524) 2024-04-25T09:52:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f776ffff640 1 -- 172.21.15.112:0/3346772594 learned_addr learned my addr 172.21.15.112:0/3346772594 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f7774be2640 1 --2- 172.21.15.112:0/3346772594 >> v2:172.21.15.19:3300/0 conn(0x7f777015de60 0x7f77700c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f776ffff640 1 -- 172.21.15.112:0/3346772594 >> v2:172.21.15.19:3300/0 conn(0x7f777015de60 msgr2=0x7f77700c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f776ffff640 1 --2- 172.21.15.112:0/3346772594 >> v2:172.21.15.19:3300/0 conn(0x7f777015de60 0x7f77700c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f776ffff640 1 -- 172.21.15.112:0/3346772594 >> v1:172.21.15.112:6789/0 conn(0x7f77700cbee0 legacy=0x7f777006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.001+0000 7f776ffff640 1 -- 172.21.15.112:0/3346772594 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77700ceef0 con 0x7f777006fc50 2024-04-25T09:52:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.002+0000 7f776ffff640 1 --2- 172.21.15.112:0/3346772594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777006fc50 0x7f777015e3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f777009e0d0 tx=0x7f775c02c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b8a12033d22746b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.002+0000 7f776f7fe640 1 -- 172.21.15.112:0/3346772594 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c036020 con 0x7f777006fc50 2024-04-25T09:52:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.002+0000 7f776f7fe640 1 -- 172.21.15.112:0/3346772594 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775c032640 con 0x7f777006fc50 2024-04-25T09:52:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.002+0000 7f776f7fe640 1 -- 172.21.15.112:0/3346772594 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c032a10 con 0x7f777006fc50 2024-04-25T09:52:44.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.003+0000 7f7776e46640 1 -- 172.21.15.112:0/3346772594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777006fc50 msgr2=0x7f777015e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.003+0000 7f7776e46640 1 --2- 172.21.15.112:0/3346772594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777006fc50 0x7f777015e3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f777009e0d0 tx=0x7f775c02c940 comp rx=0 tx=0).stop 2024-04-25T09:52:44.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.003+0000 7f7776e46640 1 -- 172.21.15.112:0/3346772594 shutdown_connections 2024-04-25T09:52:44.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.003+0000 7f7776e46640 1 --2- 172.21.15.112:0/3346772594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777006fc50 0x7f777015e3f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.003+0000 7f7776e46640 1 --2- 172.21.15.112:0/3346772594 >> v2:172.21.15.19:3300/0 conn(0x7f777015de60 0x7f77700c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.003+0000 7f7776e46640 1 -- 172.21.15.112:0/3346772594 >> 172.21.15.112:0/3346772594 conn(0x7f7770071af0 msgr2=0x7f7770071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:44.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.003+0000 7f7776e46640 1 -- 172.21.15.112:0/3346772594 shutdown_connections 2024-04-25T09:52:44.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.003+0000 7f7776e46640 1 -- 172.21.15.112:0/3346772594 wait complete. 2024-04-25T09:52:44.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.005+0000 7f7776e46640 1 Processor -- start 2024-04-25T09:52:44.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.005+0000 7f7776e46640 1 -- start start 2024-04-25T09:52:44.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.006+0000 7f7776e46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777006fc50 0x7f7770155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.006+0000 7f7776e46640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77700cbee0 0x7f77700addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.006+0000 7f7776e46640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f777015de60 0x7f77700ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.006+0000 7f7776e46640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77700cfa30 con 0x7f77700cbee0 2024-04-25T09:52:44.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.006+0000 7f7776e46640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777015e930 con 0x7f777006fc50 2024-04-25T09:52:44.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.006+0000 7f7776e46640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77700cfbb0 con 0x7f777015de60 2024-04-25T09:52:44.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.006+0000 7f7774be2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777006fc50 0x7f7770155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.006+0000 7f77753e3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f777015de60 0x7f77700ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.006+0000 7f7774be2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777006fc50 0x7f7770155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34608/0 (socket says 172.21.15.112:34608) 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.006+0000 7f7774be2640 1 -- 172.21.15.112:0/1913877340 learned_addr learned my addr 172.21.15.112:0/1913877340 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.006+0000 7f776ffff640 1 --2- 172.21.15.112:0/1913877340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77700cbee0 0x7f77700addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f77753e3640 1 -- 172.21.15.112:0/1913877340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777006fc50 msgr2=0x7f7770155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f77753e3640 1 --2- 172.21.15.112:0/1913877340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777006fc50 0x7f7770155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f77753e3640 1 -- 172.21.15.112:0/1913877340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77700cbee0 msgr2=0x7f77700addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f77753e3640 1 --2- 172.21.15.112:0/1913877340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77700cbee0 0x7f77700addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f77753e3640 1 -- 172.21.15.112:0/1913877340 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77700b1460 con 0x7f777015de60 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f7774be2640 1 --2- 172.21.15.112:0/1913877340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777006fc50 0x7f7770155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f77753e3640 1 --2- 172.21.15.112:0/1913877340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f777015de60 0x7f77700ae310 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7770072ab0 tx=0x7f776800cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f776d7fa640 1 -- 172.21.15.112:0/1913877340 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768018070 con 0x7f777015de60 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f776d7fa640 1 -- 172.21.15.112:0/1913877340 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7768004d80 con 0x7f777015de60 2024-04-25T09:52:44.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f7776e46640 1 -- 172.21.15.112:0/1913877340 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77700b1740 con 0x7f777015de60 2024-04-25T09:52:44.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f776d7fa640 1 -- 172.21.15.112:0/1913877340 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776800a820 con 0x7f777015de60 2024-04-25T09:52:44.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f7776e46640 1 -- 172.21.15.112:0/1913877340 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77700b58b0 con 0x7f777015de60 2024-04-25T09:52:44.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f776d7fa640 1 -- 172.21.15.112:0/1913877340 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f776801b070 con 0x7f777015de60 2024-04-25T09:52:44.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.007+0000 7f776d7fa640 1 -- 172.21.15.112:0/1913877340 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f77680079b0 con 0x7f777015de60 2024-04-25T09:52:44.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.008+0000 7f7776e46640 1 -- 172.21.15.112:0/1913877340 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7770073a00 con 0x7f777015de60 2024-04-25T09:52:44.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.010+0000 7f776d7fa640 1 -- 172.21.15.112:0/1913877340 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f77700704f0 con 0x7f777015de60 2024-04-25T09:52:44.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.083+0000 7f7776e46640 1 -- 172.21.15.112:0/1913877340 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0726e0ce-2ffe-41e8-a69f-7a8274edaf6b", "id": 3} v 0) v1 -- 0x7f77700b5dd0 con 0x7f777015de60 2024-04-25T09:52:44.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.086+0000 7f776d7fa640 1 -- 172.21.15.112:0/1913877340 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0726e0ce-2ffe-41e8-a69f-7a8274edaf6b", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f776800adf0 con 0x7f777015de60 2024-04-25T09:52:44.088 INFO:teuthology.orchestra.run.smithi112.stdout:3 2024-04-25T09:52:44.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.087+0000 7f7776e46640 1 -- 172.21.15.112:0/1913877340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f777015de60 msgr2=0x7f77700ae310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.087+0000 7f7776e46640 1 --2- 172.21.15.112:0/1913877340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f777015de60 0x7f77700ae310 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7770072ab0 tx=0x7f776800cb90 comp rx=0 tx=0).stop 2024-04-25T09:52:44.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.087+0000 7f7776e46640 1 -- 172.21.15.112:0/1913877340 shutdown_connections 2024-04-25T09:52:44.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.087+0000 7f7776e46640 1 --2- 172.21.15.112:0/1913877340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f777015de60 0x7f77700ae310 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.087+0000 7f7776e46640 1 --2- 172.21.15.112:0/1913877340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77700cbee0 0x7f77700addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.087+0000 7f7776e46640 1 --2- 172.21.15.112:0/1913877340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777006fc50 0x7f7770155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.087+0000 7f7776e46640 1 -- 172.21.15.112:0/1913877340 >> 172.21.15.112:0/1913877340 conn(0x7f7770071af0 msgr2=0x7f77700c61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:44.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.087+0000 7f7776e46640 1 -- 172.21.15.112:0/1913877340 shutdown_connections 2024-04-25T09:52:44.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.087+0000 7f7776e46640 1 -- 172.21.15.112:0/1913877340 wait complete. 2024-04-25T09:52:44.099 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 33e334e0-01c4-4451-a5ed-47ccec5978c0 4 2024-04-25T09:52:44.131 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:44.129+0000 7f7531021040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T09:52:44.134 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:44.133+0000 7fbc99722040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T09:52:44.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.194+0000 7fac6f1e7640 1 Processor -- start 2024-04-25T09:52:44.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.194+0000 7fac6f1e7640 1 -- start start 2024-04-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6f1e7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac6815dbb0 0x7fac6806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6f1e7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 0x7fac6806db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6f1e7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac680cae60 con 0x7fac680cbf00 2024-04-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6f1e7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac680cb140 con 0x7fac6815dbb0 2024-04-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6f1e7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac680cb420 con 0x7fac6806c300 2024-04-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6cf83640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac6815dbb0 0x7fac6806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6d784640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 0x7fac6806db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6cf83640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac6815dbb0 0x7fac6806fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34624/0 (socket says 172.21.15.112:34624) 2024-04-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6cf83640 1 -- 172.21.15.112:0/2918540066 learned_addr learned my addr 172.21.15.112:0/2918540066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6d784640 1 -- 172.21.15.112:0/2918540066 >> v2:172.21.15.112:3300/0 conn(0x7fac6815dbb0 msgr2=0x7fac6806fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6d784640 1 --2- 172.21.15.112:0/2918540066 >> v2:172.21.15.112:3300/0 conn(0x7fac6815dbb0 0x7fac6806fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6d784640 1 -- 172.21.15.112:0/2918540066 >> v1:172.21.15.112:6789/0 conn(0x7fac680cbf00 legacy=0x7fac6806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6d784640 1 -- 172.21.15.112:0/2918540066 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac680cb700 con 0x7fac6806c300 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.195+0000 7fac6cf83640 1 --2- 172.21.15.112:0/2918540066 >> v2:172.21.15.112:3300/0 conn(0x7fac6815dbb0 0x7fac6806fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.196+0000 7fac6d784640 1 --2- 172.21.15.112:0/2918540066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 0x7fac6806db50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac60009d90 tx=0x7fac6002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80bc2ef3026e3ad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.196+0000 7fac677fe640 1 -- 172.21.15.112:0/2918540066 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60007740 con 0x7fac6806c300 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.196+0000 7fac677fe640 1 -- 172.21.15.112:0/2918540066 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac600078e0 con 0x7fac6806c300 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.196+0000 7fac677fe640 1 -- 172.21.15.112:0/2918540066 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60007cb0 con 0x7fac6806c300 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.197+0000 7fac6f1e7640 1 -- 172.21.15.112:0/2918540066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 msgr2=0x7fac6806db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.197+0000 7fac6f1e7640 1 --2- 172.21.15.112:0/2918540066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 0x7fac6806db50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac60009d90 tx=0x7fac6002a660 comp rx=0 tx=0).stop 2024-04-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.197+0000 7fac6f1e7640 1 -- 172.21.15.112:0/2918540066 shutdown_connections 2024-04-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.197+0000 7fac6f1e7640 1 --2- 172.21.15.112:0/2918540066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 0x7fac6806db50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.197+0000 7fac6f1e7640 1 --2- 172.21.15.112:0/2918540066 >> v2:172.21.15.112:3300/0 conn(0x7fac6815dbb0 0x7fac6806fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.197+0000 7fac6f1e7640 1 -- 172.21.15.112:0/2918540066 >> 172.21.15.112:0/2918540066 conn(0x7fac68071af0 msgr2=0x7fac68071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.197+0000 7fac6f1e7640 1 -- 172.21.15.112:0/2918540066 shutdown_connections 2024-04-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.197+0000 7fac6f1e7640 1 -- 172.21.15.112:0/2918540066 wait complete. 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.198+0000 7fac6f1e7640 1 Processor -- start 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.198+0000 7fac6f1e7640 1 -- start start 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac6f1e7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 0x7fac680c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac6f1e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac680cbf00 0x7fac680c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac6f1e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6815dbb0 0x7fac6814b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac6f1e7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac680cab10 con 0x7fac6815dbb0 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac6f1e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6815df90 con 0x7fac680cbf00 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac6f1e7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac680703e0 con 0x7fac6806c300 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac67fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac680cbf00 0x7fac680c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac6cf83640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 0x7fac680c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac6cf83640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 0x7fac680c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:52516/0 (socket says 172.21.15.112:52516) 2024-04-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac67fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac680cbf00 0x7fac680c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34640/0 (socket says 172.21.15.112:34640) 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac6cf83640 1 -- 172.21.15.112:0/3382208807 learned_addr learned my addr 172.21.15.112:0/3382208807 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac6d784640 1 --2- 172.21.15.112:0/3382208807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6815dbb0 0x7fac6814b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac67fff640 1 -- 172.21.15.112:0/3382208807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 msgr2=0x7fac680c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac67fff640 1 --2- 172.21.15.112:0/3382208807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 0x7fac680c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac67fff640 1 -- 172.21.15.112:0/3382208807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6815dbb0 msgr2=0x7fac6814b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac67fff640 1 --2- 172.21.15.112:0/3382208807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6815dbb0 0x7fac6814b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac67fff640 1 -- 172.21.15.112:0/3382208807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac680c9660 con 0x7fac680cbf00 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac6cf83640 1 --2- 172.21.15.112:0/3382208807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 0x7fac680c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.199+0000 7fac67fff640 1 --2- 172.21.15.112:0/3382208807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac680cbf00 0x7fac680c7190 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c340 tx=0x7fac5400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.200+0000 7fac657fa640 1 -- 172.21.15.112:0/3382208807 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540052d0 con 0x7fac680cbf00 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.200+0000 7fac657fa640 1 -- 172.21.15.112:0/3382208807 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac54005490 con 0x7fac680cbf00 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.200+0000 7fac657fa640 1 -- 172.21.15.112:0/3382208807 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54007260 con 0x7fac680cbf00 2024-04-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.200+0000 7fac6f1e7640 1 -- 172.21.15.112:0/3382208807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac680c9940 con 0x7fac680cbf00 2024-04-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.200+0000 7fac6f1e7640 1 -- 172.21.15.112:0/3382208807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac680c9f00 con 0x7fac680cbf00 2024-04-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.200+0000 7fac657fa640 1 -- 172.21.15.112:0/3382208807 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fac54007400 con 0x7fac680cbf00 2024-04-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.200+0000 7fac657fa640 1 -- 172.21.15.112:0/3382208807 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fac54011590 con 0x7fac680cbf00 2024-04-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.201+0000 7fac6f1e7640 1 -- 172.21.15.112:0/3382208807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac68073a00 con 0x7fac680cbf00 2024-04-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.205+0000 7fac657fa640 1 -- 172.21.15.112:0/3382208807 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fac54003a00 con 0x7fac680cbf00 2024-04-25T09:52:44.225 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:44.223+0000 7f7531021040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T09:52:44.225 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:44.224+0000 7fbc99722040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T09:52:44.268 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:44.267+0000 7f7531021040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T09:52:44.269 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:44.268+0000 7fbc99722040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T09:52:44.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.277+0000 7fac6f1e7640 1 -- 172.21.15.112:0/3382208807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "33e334e0-01c4-4451-a5ed-47ccec5978c0", "id": 4} v 0) v1 -- 0x7fac681610e0 con 0x7fac680cbf00 2024-04-25T09:52:44.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.283+0000 7fac657fa640 1 -- 172.21.15.112:0/3382208807 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "33e334e0-01c4-4451-a5ed-47ccec5978c0", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fac54003be0 con 0x7fac680cbf00 2024-04-25T09:52:44.285 INFO:teuthology.orchestra.run.smithi112.stdout:4 2024-04-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.284+0000 7fac6f1e7640 1 -- 172.21.15.112:0/3382208807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac680cbf00 msgr2=0x7fac680c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.284+0000 7fac6f1e7640 1 --2- 172.21.15.112:0/3382208807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac680cbf00 0x7fac680c7190 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c340 tx=0x7fac5400c800 comp rx=0 tx=0).stop 2024-04-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.284+0000 7fac6f1e7640 1 -- 172.21.15.112:0/3382208807 shutdown_connections 2024-04-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.284+0000 7fac6f1e7640 1 --2- 172.21.15.112:0/3382208807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6815dbb0 0x7fac6814b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.284+0000 7fac6f1e7640 1 --2- 172.21.15.112:0/3382208807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac680cbf00 0x7fac680c7190 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.284+0000 7fac6f1e7640 1 --2- 172.21.15.112:0/3382208807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac6806c300 0x7fac680c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.284+0000 7fac6f1e7640 1 -- 172.21.15.112:0/3382208807 >> 172.21.15.112:0/3382208807 conn(0x7fac68071af0 msgr2=0x7fac680c50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:44.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.284+0000 7fac6f1e7640 1 -- 172.21.15.112:0/3382208807 shutdown_connections 2024-04-25T09:52:44.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.284+0000 7fac6f1e7640 1 -- 172.21.15.112:0/3382208807 wait complete. 2024-04-25T09:52:44.296 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new ed9503ba-06a9-4784-9879-6a20100a4959 5 2024-04-25T09:52:44.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.431+0000 7f7aeaab3640 1 Processor -- start 2024-04-25T09:52:44.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.431+0000 7f7aeaab3640 1 -- start start 2024-04-25T09:52:44.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7aeaab3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7ae40acd80 0x7f7ae40c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7aeaab3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ae40cb850 0x7f7ae40cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7aeaab3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae40c4eb0 con 0x7f7ae406bbf0 2024-04-25T09:52:44.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7aeaab3640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae40c5ac0 con 0x7f7ae40cb850 2024-04-25T09:52:44.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7aeaab3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae40c5190 con 0x7f7ae40acd80 2024-04-25T09:52:44.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7ae3fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7ae40acd80 0x7f7ae40c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7ae3fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7ae40acd80 0x7f7ae40c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34648/0 (socket says 172.21.15.112:34648) 2024-04-25T09:52:44.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7ae3fff640 1 -- 172.21.15.112:0/1889953145 learned_addr learned my addr 172.21.15.112:0/1889953145 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:44.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7ae9050640 1 --2- 172.21.15.112:0/1889953145 >> v2:172.21.15.19:3300/0 conn(0x7f7ae40cb850 0x7f7ae40cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7ae3fff640 1 -- 172.21.15.112:0/1889953145 >> v2:172.21.15.19:3300/0 conn(0x7f7ae40cb850 msgr2=0x7f7ae40cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:44.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7ae3fff640 1 --2- 172.21.15.112:0/1889953145 >> v2:172.21.15.19:3300/0 conn(0x7f7ae40cb850 0x7f7ae40cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7ae3fff640 1 -- 172.21.15.112:0/1889953145 >> v1:172.21.15.19:6789/0 conn(0x7f7ae406bbf0 legacy=0x7f7ae406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.433+0000 7f7ae3fff640 1 -- 172.21.15.112:0/1889953145 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae40c5470 con 0x7f7ae40acd80 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.434+0000 7f7ae3fff640 1 --2- 172.21.15.112:0/1889953145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ae40acd80 0x7f7ae40c5fc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ae406c780 tx=0x7f7ad002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ba0ab24d4071e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.434+0000 7f7ae37fe640 1 -- 172.21.15.112:0/1889953145 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0037070 con 0x7f7ae40acd80 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.434+0000 7f7ae37fe640 1 -- 172.21.15.112:0/1889953145 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad0032440 con 0x7f7ae40acd80 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.434+0000 7f7ae37fe640 1 -- 172.21.15.112:0/1889953145 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0032810 con 0x7f7ae40acd80 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.435+0000 7f7aeaab3640 1 -- 172.21.15.112:0/1889953145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ae40acd80 msgr2=0x7f7ae40c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.435+0000 7f7aeaab3640 1 --2- 172.21.15.112:0/1889953145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ae40acd80 0x7f7ae40c5fc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ae406c780 tx=0x7f7ad002a270 comp rx=0 tx=0).stop 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.435+0000 7f7aeaab3640 1 -- 172.21.15.112:0/1889953145 shutdown_connections 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.435+0000 7f7aeaab3640 1 --2- 172.21.15.112:0/1889953145 >> v2:172.21.15.19:3300/0 conn(0x7f7ae40cb850 0x7f7ae40cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.435+0000 7f7aeaab3640 1 --2- 172.21.15.112:0/1889953145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ae40acd80 0x7f7ae40c5fc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.435+0000 7f7aeaab3640 1 -- 172.21.15.112:0/1889953145 >> 172.21.15.112:0/1889953145 conn(0x7f7ae4071af0 msgr2=0x7f7ae4071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:44.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.435+0000 7f7aeaab3640 1 -- 172.21.15.112:0/1889953145 shutdown_connections 2024-04-25T09:52:44.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.435+0000 7f7aeaab3640 1 -- 172.21.15.112:0/1889953145 wait complete. 2024-04-25T09:52:44.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.436+0000 7f7aeaab3640 1 Processor -- start 2024-04-25T09:52:44.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.436+0000 7f7aeaab3640 1 -- start start 2024-04-25T09:52:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7aeaab3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae406bbf0 0x7f7ae414f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7aeaab3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ae40acd80 0x7f7ae415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7aeaab3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ae40cb850 0x7f7ae40ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7aeaab3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40c6960 con 0x7f7ae406bbf0 2024-04-25T09:52:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7aeaab3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40c6500 con 0x7f7ae40cb850 2024-04-25T09:52:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7aeaab3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40c6ae0 con 0x7f7ae40acd80 2024-04-25T09:52:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae9050640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ae40cb850 0x7f7ae40ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae9050640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ae40cb850 0x7f7ae40ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34650/0 (socket says 172.21.15.112:34650) 2024-04-25T09:52:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae9050640 1 -- 172.21.15.112:0/3658211460 learned_addr learned my addr 172.21.15.112:0/3658211460 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae3fff640 1 --2- 172.21.15.112:0/3658211460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ae40acd80 0x7f7ae415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae884f640 1 --2- 172.21.15.112:0/3658211460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae406bbf0 0x7f7ae414f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae3fff640 1 -- 172.21.15.112:0/3658211460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ae40cb850 msgr2=0x7f7ae40ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae3fff640 1 --2- 172.21.15.112:0/3658211460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ae40cb850 0x7f7ae40ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae3fff640 1 -- 172.21.15.112:0/3658211460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae406bbf0 msgr2=0x7f7ae414f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae3fff640 1 --2- 172.21.15.112:0/3658211460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae406bbf0 0x7f7ae414f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae3fff640 1 -- 172.21.15.112:0/3658211460 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae40c6f20 con 0x7f7ae40acd80 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae9050640 1 --2- 172.21.15.112:0/3658211460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ae40cb850 0x7f7ae40ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.437+0000 7f7ae884f640 1 --2- 172.21.15.112:0/3658211460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae406bbf0 0x7f7ae414f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.438+0000 7f7ae3fff640 1 --2- 172.21.15.112:0/3658211460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ae40acd80 0x7f7ae415ebe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0031000 tx=0x7f7ad00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.438+0000 7f7ae17fa640 1 -- 172.21.15.112:0/3658211460 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0041030 con 0x7f7ae40acd80 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.438+0000 7f7ae17fa640 1 -- 172.21.15.112:0/3658211460 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad0030440 con 0x7f7ae40acd80 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.438+0000 7f7aeaab3640 1 -- 172.21.15.112:0/3658211460 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae40c7200 con 0x7f7ae40acd80 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.438+0000 7f7ae17fa640 1 -- 172.21.15.112:0/3658211460 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0037040 con 0x7f7ae40acd80 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.438+0000 7f7aeaab3640 1 -- 172.21.15.112:0/3658211460 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae40c77c0 con 0x7f7ae40acd80 2024-04-25T09:52:44.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.438+0000 7f7ae17fa640 1 -- 172.21.15.112:0/3658211460 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7ad0044070 con 0x7f7ae40acd80 2024-04-25T09:52:44.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.438+0000 7f7ae17fa640 1 -- 172.21.15.112:0/3658211460 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f7ad003dae0 con 0x7f7ae40acd80 2024-04-25T09:52:44.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.439+0000 7f7aeaab3640 1 -- 172.21.15.112:0/3658211460 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae4158fe0 con 0x7f7ae40acd80 2024-04-25T09:52:44.444 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:44.440+0000 7fbc99722040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T09:52:44.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.441+0000 7f7ae17fa640 1 -- 172.21.15.112:0/3658211460 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7ad003ddc0 con 0x7f7ae40acd80 2024-04-25T09:52:44.445 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:44.443+0000 7f7531021040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T09:52:44.494 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:44.492+0000 7f7531021040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T09:52:44.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.513+0000 7f7aeaab3640 1 -- 172.21.15.112:0/3658211460 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ed9503ba-06a9-4784-9879-6a20100a4959", "id": 5} v 0) v1 -- 0x7f7ae4073a00 con 0x7f7ae40acd80 2024-04-25T09:52:44.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.519+0000 7f7ae17fa640 1 -- 172.21.15.112:0/3658211460 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ed9503ba-06a9-4784-9879-6a20100a4959", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7ad003d560 con 0x7f7ae40acd80 2024-04-25T09:52:44.521 INFO:teuthology.orchestra.run.smithi112.stdout:5 2024-04-25T09:52:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.520+0000 7f7aeaab3640 1 -- 172.21.15.112:0/3658211460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ae40acd80 msgr2=0x7f7ae415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.520+0000 7f7aeaab3640 1 --2- 172.21.15.112:0/3658211460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ae40acd80 0x7f7ae415ebe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0031000 tx=0x7f7ad00046c0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.520+0000 7f7aeaab3640 1 -- 172.21.15.112:0/3658211460 shutdown_connections 2024-04-25T09:52:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.520+0000 7f7aeaab3640 1 --2- 172.21.15.112:0/3658211460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ae40cb850 0x7f7ae40ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.520+0000 7f7aeaab3640 1 --2- 172.21.15.112:0/3658211460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ae40acd80 0x7f7ae415ebe0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.520+0000 7f7aeaab3640 1 --2- 172.21.15.112:0/3658211460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae406bbf0 0x7f7ae414f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.520+0000 7f7aeaab3640 1 -- 172.21.15.112:0/3658211460 >> 172.21.15.112:0/3658211460 conn(0x7f7ae4071af0 msgr2=0x7f7ae40ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.520+0000 7f7aeaab3640 1 -- 172.21.15.112:0/3658211460 shutdown_connections 2024-04-25T09:52:44.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.520+0000 7f7aeaab3640 1 -- 172.21.15.112:0/3658211460 wait complete. 2024-04-25T09:52:44.532 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 1a2be840-63b7-4280-bfe5-b908a9351c59 6 2024-04-25T09:52:44.550 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:44.548+0000 7f7531021040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T09:52:44.559 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:44.558+0000 7fbc99722040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T09:52:44.614 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:44.613+0000 7fbc99722040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T09:52:44.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.627+0000 7f0ee57fe640 1 Processor -- start 2024-04-25T09:52:44.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.627+0000 7f0ee57fe640 1 -- start start 2024-04-25T09:52:44.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ee57fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0ee015dc20 0x7f0ee00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ee57fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0ee0161710 0x7f0ee00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ee57fe640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee015e000 con 0x7f0ee00b9330 2024-04-25T09:52:44.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ee57fe640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee00d1de0 con 0x7f0ee015dc20 2024-04-25T09:52:44.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ee57fe640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee0161c60 con 0x7f0ee0161710 2024-04-25T09:52:44.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ede7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0ee0161710 0x7f0ee00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ede7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0ee0161710 0x7f0ee00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34662/0 (socket says 172.21.15.112:34662) 2024-04-25T09:52:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ede7fc640 1 -- 172.21.15.112:0/1993008358 learned_addr learned my addr 172.21.15.112:0/1993008358 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0edeffd640 1 --2- 172.21.15.112:0/1993008358 >> v2:172.21.15.19:3300/0 conn(0x7f0ee015dc20 0x7f0ee00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ede7fc640 1 -- 172.21.15.112:0/1993008358 >> v2:172.21.15.19:3300/0 conn(0x7f0ee015dc20 msgr2=0x7f0ee00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ede7fc640 1 --2- 172.21.15.112:0/1993008358 >> v2:172.21.15.19:3300/0 conn(0x7f0ee015dc20 0x7f0ee00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ede7fc640 1 -- 172.21.15.112:0/1993008358 >> v1:172.21.15.19:6789/0 conn(0x7f0ee00b9330 legacy=0x7f0ee006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0ede7fc640 1 -- 172.21.15.112:0/1993008358 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0161f40 con 0x7f0ee0161710 2024-04-25T09:52:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.628+0000 7f0edeffd640 1 --2- 172.21.15.112:0/1993008358 >> v2:172.21.15.19:3300/0 conn(0x7f0ee015dc20 0x7f0ee00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:52:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.629+0000 7f0ede7fc640 1 --2- 172.21.15.112:0/1993008358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee0161710 0x7f0ee00cebb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ee006da90 tx=0x7f0ed002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b32eb700b330dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.629+0000 7f0eddffb640 1 -- 172.21.15.112:0/1993008358 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0037070 con 0x7f0ee0161710 2024-04-25T09:52:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.629+0000 7f0eddffb640 1 -- 172.21.15.112:0/1993008358 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed0032440 con 0x7f0ee0161710 2024-04-25T09:52:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.629+0000 7f0eddffb640 1 -- 172.21.15.112:0/1993008358 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0032810 con 0x7f0ee0161710 2024-04-25T09:52:44.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.629+0000 7f0ee57fe640 1 -- 172.21.15.112:0/1993008358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee0161710 msgr2=0x7f0ee00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.629+0000 7f0ee57fe640 1 --2- 172.21.15.112:0/1993008358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee0161710 0x7f0ee00cebb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ee006da90 tx=0x7f0ed002a270 comp rx=0 tx=0).stop 2024-04-25T09:52:44.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.630+0000 7f0ee57fe640 1 -- 172.21.15.112:0/1993008358 shutdown_connections 2024-04-25T09:52:44.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.630+0000 7f0ee57fe640 1 --2- 172.21.15.112:0/1993008358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee0161710 0x7f0ee00cebb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.630+0000 7f0ee57fe640 1 --2- 172.21.15.112:0/1993008358 >> v2:172.21.15.19:3300/0 conn(0x7f0ee015dc20 0x7f0ee00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.630+0000 7f0ee57fe640 1 -- 172.21.15.112:0/1993008358 >> 172.21.15.112:0/1993008358 conn(0x7f0ee00d0420 msgr2=0x7f0ee00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:44.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.630+0000 7f0ee57fe640 1 -- 172.21.15.112:0/1993008358 shutdown_connections 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.630+0000 7f0ee57fe640 1 -- 172.21.15.112:0/1993008358 wait complete. 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.631+0000 7f0ee57fe640 1 Processor -- start 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.631+0000 7f0ee57fe640 1 -- start start 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ee57fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ee00b9330 0x7f0ee00cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ee57fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee015dc20 0x7f0ee0158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ee57fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee0161710 0x7f0ee015e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ee57fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00b5e50 con 0x7f0ee015dc20 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ee57fe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee0162500 con 0x7f0ee0161710 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ee57fe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00b5fd0 con 0x7f0ee00b9330 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0edf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee0161710 0x7f0ee015e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0edeffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ee00b9330 0x7f0ee00cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0edf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee0161710 0x7f0ee015e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34678/0 (socket says 172.21.15.112:34678) 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0edeffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ee00b9330 0x7f0ee00cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:52538/0 (socket says 172.21.15.112:52538) 2024-04-25T09:52:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0edf7fe640 1 -- 172.21.15.112:0/2232341437 learned_addr learned my addr 172.21.15.112:0/2232341437 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ede7fc640 1 --2- 172.21.15.112:0/2232341437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee015dc20 0x7f0ee0158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0edf7fe640 1 -- 172.21.15.112:0/2232341437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ee00b9330 msgr2=0x7f0ee00cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0edf7fe640 1 --2- 172.21.15.112:0/2232341437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ee00b9330 0x7f0ee00cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0edf7fe640 1 -- 172.21.15.112:0/2232341437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee015dc20 msgr2=0x7f0ee0158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0edf7fe640 1 --2- 172.21.15.112:0/2232341437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee015dc20 0x7f0ee0158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0edf7fe640 1 -- 172.21.15.112:0/2232341437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0158a80 con 0x7f0ee0161710 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0edf7fe640 1 --2- 172.21.15.112:0/2232341437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee0161710 0x7f0ee015e1d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800c730 tx=0x7f0ed800cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ec77fe640 1 -- 172.21.15.112:0/2232341437 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8003b70 con 0x7f0ee0161710 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ec77fe640 1 -- 172.21.15.112:0/2232341437 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed8003d30 con 0x7f0ee0161710 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ee57fe640 1 -- 172.21.15.112:0/2232341437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee01e7510 con 0x7f0ee0161710 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ec77fe640 1 -- 172.21.15.112:0/2232341437 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8005310 con 0x7f0ee0161710 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.632+0000 7f0ee57fe640 1 -- 172.21.15.112:0/2232341437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee01e7db0 con 0x7f0ee0161710 2024-04-25T09:52:44.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.633+0000 7f0ec77fe640 1 -- 172.21.15.112:0/2232341437 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0ed80054b0 con 0x7f0ee0161710 2024-04-25T09:52:44.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.633+0000 7f0ec77fe640 1 -- 172.21.15.112:0/2232341437 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f0ed8012b00 con 0x7f0ee0161710 2024-04-25T09:52:44.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.633+0000 7f0ee57fe640 1 -- 172.21.15.112:0/2232341437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee01e7900 con 0x7f0ee0161710 2024-04-25T09:52:44.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.635+0000 7f0ec77fe640 1 -- 172.21.15.112:0/2232341437 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0ee0158a80 con 0x7f0ee0161710 2024-04-25T09:52:44.669 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:44.667+0000 7f7531021040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:52:44.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.708+0000 7f0ee57fe640 1 -- 172.21.15.112:0/2232341437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1a2be840-63b7-4280-bfe5-b908a9351c59", "id": 6} v 0) v1 -- 0x7f0ee0073a00 con 0x7f0ee0161710 2024-04-25T09:52:44.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.714+0000 7f0ec77fe640 1 -- 172.21.15.112:0/2232341437 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1a2be840-63b7-4280-bfe5-b908a9351c59", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0ed8012de0 con 0x7f0ee0161710 2024-04-25T09:52:44.717 INFO:teuthology.orchestra.run.smithi112.stdout:6 2024-04-25T09:52:44.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.714+0000 7f0ee57fe640 1 -- 172.21.15.112:0/2232341437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee0161710 msgr2=0x7f0ee015e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.714+0000 7f0ee57fe640 1 --2- 172.21.15.112:0/2232341437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee0161710 0x7f0ee015e1d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800c730 tx=0x7f0ed800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.715+0000 7f0ee57fe640 1 -- 172.21.15.112:0/2232341437 shutdown_connections 2024-04-25T09:52:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.715+0000 7f0ee57fe640 1 --2- 172.21.15.112:0/2232341437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee0161710 0x7f0ee015e1d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.715+0000 7f0ee57fe640 1 --2- 172.21.15.112:0/2232341437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ee015dc20 0x7f0ee0158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.715+0000 7f0ee57fe640 1 --2- 172.21.15.112:0/2232341437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ee00b9330 0x7f0ee00cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.715+0000 7f0ee57fe640 1 -- 172.21.15.112:0/2232341437 >> 172.21.15.112:0/2232341437 conn(0x7f0ee00d0420 msgr2=0x7f0ee00b8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.715+0000 7f0ee57fe640 1 -- 172.21.15.112:0/2232341437 shutdown_connections 2024-04-25T09:52:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.715+0000 7f0ee57fe640 1 -- 172.21.15.112:0/2232341437 wait complete. 2024-04-25T09:52:44.727 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 6efd9150-bb02-4e46-9205-3a713f594e9a 7 2024-04-25T09:52:44.731 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:44.729+0000 7fbc99722040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:52:44.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.861+0000 7f5ed23a5640 1 Processor -- start 2024-04-25T09:52:44.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.861+0000 7f5ed23a5640 1 -- start start 2024-04-25T09:52:44.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ed23a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 0x7f5ecc0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ed23a5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ecc0cdcd0 0x7f5ecc0cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ed23a5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ecc0cb190 con 0x7f5ecc15bb20 2024-04-25T09:52:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ed23a5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ecc0d0450 con 0x7f5ecc0cdcd0 2024-04-25T09:52:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ed23a5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0cb470 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ed0942640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ecc0cdcd0 0x7f5ecc0cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ecb7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 0x7f5ecc0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ed0942640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ecc0cdcd0 0x7f5ecc0cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34682/0 (socket says 172.21.15.112:34682) 2024-04-25T09:52:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ed0942640 1 -- 172.21.15.112:0/3863620863 learned_addr learned my addr 172.21.15.112:0/3863620863 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ecb7fe640 1 -- 172.21.15.112:0/3863620863 >> v2:172.21.15.112:3300/0 conn(0x7f5ecc0cdcd0 msgr2=0x7f5ecc0cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ecb7fe640 1 --2- 172.21.15.112:0/3863620863 >> v2:172.21.15.112:3300/0 conn(0x7f5ecc0cdcd0 0x7f5ecc0cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ecb7fe640 1 -- 172.21.15.112:0/3863620863 >> v1:172.21.15.112:6789/0 conn(0x7f5ecc15bb20 legacy=0x7f5ecc0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ecb7fe640 1 -- 172.21.15.112:0/3863620863 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc0cb750 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.862+0000 7f5ed0942640 1 --2- 172.21.15.112:0/3863620863 >> v2:172.21.15.112:3300/0 conn(0x7f5ecc0cdcd0 0x7f5ecc0cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:44.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.863+0000 7f5ecb7fe640 1 --2- 172.21.15.112:0/3863620863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 0x7f5ecc0cf9d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0097a0 tx=0x7f5ebc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26cfa37978e22e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:44.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.863+0000 7f5ecaffd640 1 -- 172.21.15.112:0/3863620863 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc037070 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.863+0000 7f5ecaffd640 1 -- 172.21.15.112:0/3863620863 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc02ae50 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.863+0000 7f5ecaffd640 1 -- 172.21.15.112:0/3863620863 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc032660 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.864+0000 7f5ed23a5640 1 -- 172.21.15.112:0/3863620863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 msgr2=0x7f5ecc0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.864+0000 7f5ed23a5640 1 --2- 172.21.15.112:0/3863620863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 0x7f5ecc0cf9d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0097a0 tx=0x7f5ebc02a100 comp rx=0 tx=0).stop 2024-04-25T09:52:44.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.864+0000 7f5ed23a5640 1 -- 172.21.15.112:0/3863620863 shutdown_connections 2024-04-25T09:52:44.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.864+0000 7f5ed23a5640 1 --2- 172.21.15.112:0/3863620863 >> v2:172.21.15.112:3300/0 conn(0x7f5ecc0cdcd0 0x7f5ecc0cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.864+0000 7f5ed23a5640 1 --2- 172.21.15.112:0/3863620863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 0x7f5ecc0cf9d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.864+0000 7f5ed23a5640 1 -- 172.21.15.112:0/3863620863 >> 172.21.15.112:0/3863620863 conn(0x7f5ecc0b2390 msgr2=0x7f5ecc0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:44.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.864+0000 7f5ed23a5640 1 -- 172.21.15.112:0/3863620863 shutdown_connections 2024-04-25T09:52:44.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.864+0000 7f5ed23a5640 1 -- 172.21.15.112:0/3863620863 wait complete. 2024-04-25T09:52:44.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.865+0000 7f5ed23a5640 1 Processor -- start 2024-04-25T09:52:44.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.865+0000 7f5ed23a5640 1 -- start start 2024-04-25T09:52:44.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ed23a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 0x7f5ecc14e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ed23a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc0cdcd0 0x7f5ecc0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ed23a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ecc15bb20 0x7f5ecc0c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:44.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ed23a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc15bf00 con 0x7f5ecc0cdcd0 2024-04-25T09:52:44.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ed23a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0cbd10 con 0x7f5ecc15bb20 2024-04-25T09:52:44.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ed23a5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc15c080 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 0x7f5ecc14e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 0x7f5ecc14e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:52570/0 (socket says 172.21.15.112:52570) 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ed0942640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ecc15bb20 0x7f5ecc0c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ecbfff640 1 -- 172.21.15.112:0/1631553647 learned_addr learned my addr 172.21.15.112:0/1631553647 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ecb7fe640 1 --2- 172.21.15.112:0/1631553647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc0cdcd0 0x7f5ecc0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ecbfff640 1 -- 172.21.15.112:0/1631553647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ecc15bb20 msgr2=0x7f5ecc0c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ecbfff640 1 --2- 172.21.15.112:0/1631553647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ecc15bb20 0x7f5ecc0c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ecbfff640 1 -- 172.21.15.112:0/1631553647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc0cdcd0 msgr2=0x7f5ecc0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ecbfff640 1 --2- 172.21.15.112:0/1631553647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc0cdcd0 0x7f5ecc0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ecbfff640 1 -- 172.21.15.112:0/1631553647 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc0b0a60 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.866+0000 7f5ed0942640 1 --2- 172.21.15.112:0/1631553647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ecc15bb20 0x7f5ecc0c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.867+0000 7f5ecbfff640 1 --2- 172.21.15.112:0/1631553647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 0x7f5ecc14e600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c3c0 tx=0x7f5ec000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.867+0000 7f5ec8ff9640 1 -- 172.21.15.112:0/1631553647 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec00052d0 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.867+0000 7f5ec8ff9640 1 -- 172.21.15.112:0/1631553647 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec0005490 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.867+0000 7f5ec8ff9640 1 -- 172.21.15.112:0/1631553647 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0007260 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.867+0000 7f5ed23a5640 1 -- 172.21.15.112:0/1631553647 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc0b0d40 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.867+0000 7f5ed23a5640 1 -- 172.21.15.112:0/1631553647 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc0b1300 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.867+0000 7f5ec8ff9640 1 -- 172.21.15.112:0/1631553647 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5ec0007400 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.867+0000 7f5ec8ff9640 1 -- 172.21.15.112:0/1631553647 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f5ec0011bf0 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.868+0000 7f5ed23a5640 1 -- 172.21.15.112:0/1631553647 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc073a00 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.871+0000 7f5ec8ff9640 1 -- 172.21.15.112:0/1631553647 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5ec00035f0 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.943+0000 7f5ed23a5640 1 -- 172.21.15.112:0/1631553647 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6efd9150-bb02-4e46-9205-3a713f594e9a", "id": 7} v 0) v1 -- 0x7f5ecc0cd890 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.952+0000 7f5ec8ff9640 1 -- 172.21.15.112:0/1631553647 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6efd9150-bb02-4e46-9205-3a713f594e9a", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5ec0003a00 con 0x7f5ecc0ccdd0 2024-04-25T09:52:44.954 INFO:teuthology.orchestra.run.smithi112.stdout:7 2024-04-25T09:52:44.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.953+0000 7f5ed23a5640 1 -- 172.21.15.112:0/1631553647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 msgr2=0x7f5ecc14e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:44.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.953+0000 7f5ed23a5640 1 --2- 172.21.15.112:0/1631553647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 0x7f5ecc14e600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c3c0 tx=0x7f5ec000c880 comp rx=0 tx=0).stop 2024-04-25T09:52:44.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.953+0000 7f5ed23a5640 1 -- 172.21.15.112:0/1631553647 shutdown_connections 2024-04-25T09:52:44.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.953+0000 7f5ed23a5640 1 --2- 172.21.15.112:0/1631553647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ecc15bb20 0x7f5ecc0c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.953+0000 7f5ed23a5640 1 --2- 172.21.15.112:0/1631553647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc0cdcd0 0x7f5ecc0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.953+0000 7f5ed23a5640 1 --2- 172.21.15.112:0/1631553647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ecc0ccdd0 0x7f5ecc14e600 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:44.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.953+0000 7f5ed23a5640 1 -- 172.21.15.112:0/1631553647 >> 172.21.15.112:0/1631553647 conn(0x7f5ecc0b2390 msgr2=0x7f5ecc06e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:44.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.953+0000 7f5ed23a5640 1 -- 172.21.15.112:0/1631553647 shutdown_connections 2024-04-25T09:52:44.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T09:52:44.953+0000 7f5ed23a5640 1 -- 172.21.15.112:0/1631553647 wait complete. 2024-04-25T09:52:44.965 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T09:52:44.965 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T09:52:44.968 INFO:tasks.ceph.osd.0:Started 2024-04-25T09:52:44.968 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T09:52:44.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T09:52:44.970 INFO:tasks.ceph.osd.1:Started 2024-04-25T09:52:44.971 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T09:52:44.971 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T09:52:44.973 INFO:tasks.ceph.osd.2:Started 2024-04-25T09:52:44.973 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T09:52:44.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T09:52:44.975 INFO:tasks.ceph.osd.3:Started 2024-04-25T09:52:44.975 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T09:52:44.975 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T09:52:44.977 INFO:tasks.ceph.osd.4:Started 2024-04-25T09:52:44.977 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T09:52:44.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T09:52:44.979 INFO:tasks.ceph.osd.5:Started 2024-04-25T09:52:44.979 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T09:52:44.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T09:52:44.981 INFO:tasks.ceph.osd.6:Started 2024-04-25T09:52:44.981 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T09:52:44.982 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T09:52:44.984 INFO:tasks.ceph.osd.7:Started 2024-04-25T09:52:44.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:52:45.081 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:45.079+0000 7f7531021040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T09:52:45.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.116+0000 7f27ae8fc640 1 Processor -- start 2024-04-25T09:52:45.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.116+0000 7f27ae8fc640 1 -- start start 2024-04-25T09:52:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.116+0000 7f27ae8fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f27a815da40 0x7f27a806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.116+0000 7f27ae8fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27a806c2f0 0x7f27a80ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.116+0000 7f27ae8fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f27a806f700 0x7f27a80ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.116+0000 7f27ae8fc640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a806b920 con 0x7f27a806f700 2024-04-25T09:52:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.116+0000 7f27ae8fc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a806ad10 con 0x7f27a815da40 2024-04-25T09:52:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.116+0000 7f27ae8fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a806aff0 con 0x7f27a806c2f0 2024-04-25T09:52:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.117+0000 7f27ace99640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f27a806f700 0x7f27a80ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.117+0000 7f27ace99640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f27a806f700 0x7f27a80ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41352/0 (socket says 172.21.15.19:41352) 2024-04-25T09:52:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.117+0000 7f27ace99640 1 -- 172.21.15.19:0/3015576132 learned_addr learned my addr 172.21.15.19:0/3015576132 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:52:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.117+0000 7f27a77fe640 1 --2- 172.21.15.19:0/3015576132 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27a806c2f0 0x7f27a80ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.117+0000 7f27a7fff640 1 --2- 172.21.15.19:0/3015576132 >> v2:172.21.15.112:3300/0 conn(0x7f27a815da40 0x7f27a806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.117+0000 7f27ace99640 1 -- 172.21.15.19:0/3015576132 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27a806c2f0 msgr2=0x7f27a80ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.117+0000 7f27ace99640 1 --2- 172.21.15.19:0/3015576132 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27a806c2f0 0x7f27a80ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.117+0000 7f27ace99640 1 -- 172.21.15.19:0/3015576132 >> v2:172.21.15.112:3300/0 conn(0x7f27a815da40 msgr2=0x7f27a806bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.117+0000 7f27ace99640 1 --2- 172.21.15.19:0/3015576132 >> v2:172.21.15.112:3300/0 conn(0x7f27a815da40 0x7f27a806bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.117+0000 7f27ace99640 1 -- 172.21.15.19:0/3015576132 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a806b2d0 con 0x7f27a806f700 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.118+0000 7f27ace99640 1 --2- 172.21.15.19:0/3015576132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27a806f700 0x7f27a80ad3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27a8072c50 tx=0x7f27a002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8a04499e1a3f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.118+0000 7f27a6ffd640 1 -- 172.21.15.19:0/3015576132 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0036070 con 0x7f27a806f700 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.118+0000 7f27a6ffd640 1 -- 172.21.15.19:0/3015576132 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a0031600 con 0x7f27a806f700 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.118+0000 7f27a6ffd640 1 -- 172.21.15.19:0/3015576132 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a00319d0 con 0x7f27a806f700 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.118+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3015576132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27a806f700 msgr2=0x7f27a80ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.118+0000 7f27ae8fc640 1 --2- 172.21.15.19:0/3015576132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27a806f700 0x7f27a80ad3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27a8072c50 tx=0x7f27a002a570 comp rx=0 tx=0).stop 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.118+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3015576132 shutdown_connections 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.118+0000 7f27ae8fc640 1 --2- 172.21.15.19:0/3015576132 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27a806c2f0 0x7f27a80ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.118+0000 7f27ae8fc640 1 --2- 172.21.15.19:0/3015576132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27a806f700 0x7f27a80ad3a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.118+0000 7f27ae8fc640 1 --2- 172.21.15.19:0/3015576132 >> v2:172.21.15.112:3300/0 conn(0x7f27a815da40 0x7f27a806bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.118+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3015576132 >> 172.21.15.19:0/3015576132 conn(0x7f27a80cfe10 msgr2=0x7f27a80d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.119+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3015576132 shutdown_connections 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.119+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3015576132 wait complete. 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.120+0000 7f27ae8fc640 1 Processor -- start 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.120+0000 7f27ae8fc640 1 -- start start 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27ae8fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a806f700 0x7f27a8147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27ae8fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27a815da40 0x7f27a80cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27ae8fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27a8147bd0 0x7f27a80b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27ae8fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a806ec70 con 0x7f27a8147bd0 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27ae8fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a815de20 con 0x7f27a806f700 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27ae8fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a806dc00 con 0x7f27a815da40 2024-04-25T09:52:45.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a806f700 0x7f27a8147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27a77fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27a815da40 0x7f27a80cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a806f700 0x7f27a8147550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:40606/0 (socket says 172.21.15.19:40606) 2024-04-25T09:52:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27a7fff640 1 -- 172.21.15.19:0/3198531330 learned_addr learned my addr 172.21.15.19:0/3198531330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:52:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27ace99640 1 --2- 172.21.15.19:0/3198531330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27a8147bd0 0x7f27a80b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27a7fff640 1 -- 172.21.15.19:0/3198531330 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27a815da40 msgr2=0x7f27a80cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27a7fff640 1 --2- 172.21.15.19:0/3198531330 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27a815da40 0x7f27a80cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27a7fff640 1 -- 172.21.15.19:0/3198531330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27a8147bd0 msgr2=0x7f27a80b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27a7fff640 1 --2- 172.21.15.19:0/3198531330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27a8147bd0 0x7f27a80b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27a7fff640 1 -- 172.21.15.19:0/3198531330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a8160470 con 0x7f27a806f700 2024-04-25T09:52:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.121+0000 7f27a7fff640 1 --2- 172.21.15.19:0/3198531330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a806f700 0x7f27a8147550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f279c00e8a0 tx=0x7f279c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.122+0000 7f27a4ff9640 1 -- 172.21.15.19:0/3198531330 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c017070 con 0x7f27a806f700 2024-04-25T09:52:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.122+0000 7f27a4ff9640 1 -- 172.21.15.19:0/3198531330 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c00c940 con 0x7f27a806f700 2024-04-25T09:52:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.122+0000 7f27a4ff9640 1 -- 172.21.15.19:0/3198531330 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c00cd30 con 0x7f27a806f700 2024-04-25T09:52:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.122+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3198531330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a80ccf30 con 0x7f27a806f700 2024-04-25T09:52:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.122+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3198531330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a815e320 con 0x7f27a806f700 2024-04-25T09:52:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.122+0000 7f27a4ff9640 1 -- 172.21.15.19:0/3198531330 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f279c01a070 con 0x7f27a806f700 2024-04-25T09:52:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.122+0000 7f27a4ff9640 1 -- 172.21.15.19:0/3198531330 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f279c006020 con 0x7f27a806f700 2024-04-25T09:52:45.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.123+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3198531330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a8073990 con 0x7f27a806f700 2024-04-25T09:52:45.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.125+0000 7f27a4ff9640 1 -- 172.21.15.19:0/3198531330 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f279c006480 con 0x7f27a806f700 2024-04-25T09:52:45.170 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:45.168+0000 7fbc99722040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T09:52:45.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.187+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3198531330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27a80617b0 con 0x7f27a806f700 2024-04-25T09:52:45.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.187+0000 7f27a4ff9640 1 -- 172.21.15.19:0/3198531330 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f279c006660 con 0x7f27a806f700 2024-04-25T09:52:45.190 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:52:45.190 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:52:44.945978+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:52:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.188+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3198531330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a806f700 msgr2=0x7f27a8147550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.188+0000 7f27ae8fc640 1 --2- 172.21.15.19:0/3198531330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a806f700 0x7f27a8147550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f279c00e8a0 tx=0x7f279c00ed60 comp rx=0 tx=0).stop 2024-04-25T09:52:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.188+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3198531330 shutdown_connections 2024-04-25T09:52:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.188+0000 7f27ae8fc640 1 --2- 172.21.15.19:0/3198531330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27a8147bd0 0x7f27a80b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.188+0000 7f27ae8fc640 1 --2- 172.21.15.19:0/3198531330 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27a815da40 0x7f27a80cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.188+0000 7f27ae8fc640 1 --2- 172.21.15.19:0/3198531330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27a806f700 0x7f27a8147550 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.188+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3198531330 >> 172.21.15.19:0/3198531330 conn(0x7f27a80cfe10 msgr2=0x7f27a80afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.189+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3198531330 shutdown_connections 2024-04-25T09:52:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:52:45.189+0000 7f27ae8fc640 1 -- 172.21.15.19:0/3198531330 wait complete. 2024-04-25T09:52:45.201 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T09:52:45.201 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T09:52:45.338 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:45.336+0000 7f7531021040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T09:52:45.391 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:45.389+0000 7f7531021040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T09:52:45.440 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:45.438+0000 7fbc99722040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T09:52:45.494 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:45.493+0000 7fbc99722040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T09:52:45.507 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:45.506+0000 7f7531021040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T09:52:45.573 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-04-25T09:52:45.571+0000 7fc27b723640 -1 Falling back to public interface 2024-04-25T09:52:45.580 INFO:tasks.ceph.osd.4.smithi112.stderr:2024-04-25T09:52:45.578+0000 7ff56a827640 -1 Falling back to public interface 2024-04-25T09:52:45.584 INFO:tasks.ceph.osd.5.smithi112.stderr:2024-04-25T09:52:45.582+0000 7f597d215640 -1 Falling back to public interface 2024-04-25T09:52:45.590 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-04-25T09:52:45.588+0000 7f05b583d640 -1 Falling back to public interface 2024-04-25T09:52:45.592 INFO:tasks.ceph.osd.6.smithi112.stderr:2024-04-25T09:52:45.590+0000 7f37d1011640 -1 Falling back to public interface 2024-04-25T09:52:45.592 INFO:tasks.ceph.osd.7.smithi112.stderr:2024-04-25T09:52:45.590+0000 7f6311610640 -1 Falling back to public interface 2024-04-25T09:52:45.597 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-04-25T09:52:45.596+0000 7f62fa3a1640 -1 Falling back to public interface 2024-04-25T09:52:45.607 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-04-25T09:52:45.606+0000 7fb7daa0d640 -1 Falling back to public interface 2024-04-25T09:52:45.615 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:45.614+0000 7fbc99722040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T09:52:45.792 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:45.790+0000 7f7531021040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T09:52:45.840 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:45.839+0000 7f7531021040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T09:52:45.907 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:45.905+0000 7fbc99722040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T09:52:45.957 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:45.954+0000 7fbc99722040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T09:52:45.999 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:45.997+0000 7f7531021040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T09:52:46.007 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:52:46.099 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.096+0000 7f3ac3d42640 1 Processor -- start 2024-04-25T09:52:46.099 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.097+0000 7f3ac3d42640 1 -- start start 2024-04-25T09:52:46.101 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.097+0000 7f3ac3d42640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3abc15ba20 0x7f3abc0b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:46.101 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.097+0000 7f3ac3d42640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3abc15be00 0x7f3abc0cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:46.101 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.097+0000 7f3ac3d42640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3abc0cfe10 0x7f3abc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:46.101 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.097+0000 7f3ac3d42640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3abc0d05f0 con 0x7f3abc15be00 2024-04-25T09:52:46.101 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.097+0000 7f3ac3d42640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3abc0ccd10 con 0x7f3abc0cfe10 2024-04-25T09:52:46.101 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.097+0000 7f3ac3d42640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc0cd6e0 con 0x7f3abc15ba20 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac12dd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3abc15be00 0x7f3abc0cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac12dd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3abc15be00 0x7f3abc0cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41380/0 (socket says 172.21.15.19:41380) 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac12dd640 1 -- 172.21.15.19:0/2525943562 learned_addr learned my addr 172.21.15.19:0/2525943562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac1ade640 1 --2- 172.21.15.19:0/2525943562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3abc15ba20 0x7f3abc0b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac22df640 1 --2- 172.21.15.19:0/2525943562 >> v2:172.21.15.112:3300/0 conn(0x7f3abc0cfe10 0x7f3abc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac12dd640 1 -- 172.21.15.19:0/2525943562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3abc15ba20 msgr2=0x7f3abc0b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac12dd640 1 --2- 172.21.15.19:0/2525943562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3abc15ba20 0x7f3abc0b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac12dd640 1 -- 172.21.15.19:0/2525943562 >> v2:172.21.15.112:3300/0 conn(0x7f3abc0cfe10 msgr2=0x7f3abc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac12dd640 1 --2- 172.21.15.19:0/2525943562 >> v2:172.21.15.112:3300/0 conn(0x7f3abc0cfe10 0x7f3abc0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac12dd640 1 -- 172.21.15.19:0/2525943562 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc0cfb40 con 0x7f3abc15be00 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac1ade640 1 --2- 172.21.15.19:0/2525943562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3abc15ba20 0x7f3abc0b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:46.102 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.098+0000 7f3ac22df640 1 --2- 172.21.15.19:0/2525943562 >> v2:172.21.15.112:3300/0 conn(0x7f3abc0cfe10 0x7f3abc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:46.104 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.099+0000 7f3ac12dd640 1 --2- 172.21.15.19:0/2525943562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3abc15be00 0x7f3abc0cd1a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0b1320 tx=0x7f3ab802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c3520a9372c623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:46.104 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.099+0000 7f3ac0adc640 1 -- 172.21.15.19:0/2525943562 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8037070 con 0x7f3abc15be00 2024-04-25T09:52:46.104 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.099+0000 7f3ac0adc640 1 -- 172.21.15.19:0/2525943562 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab8032440 con 0x7f3abc15be00 2024-04-25T09:52:46.104 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.099+0000 7f3ac0adc640 1 -- 172.21.15.19:0/2525943562 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8032810 con 0x7f3abc15be00 2024-04-25T09:52:46.104 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.100+0000 7f3ac3d42640 1 -- 172.21.15.19:0/2525943562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3abc15be00 msgr2=0x7f3abc0cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:46.104 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.100+0000 7f3ac3d42640 1 --2- 172.21.15.19:0/2525943562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3abc15be00 0x7f3abc0cd1a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0b1320 tx=0x7f3ab802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:52:46.104 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.100+0000 7f3ac3d42640 1 -- 172.21.15.19:0/2525943562 shutdown_connections 2024-04-25T09:52:46.104 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.100+0000 7f3ac3d42640 1 --2- 172.21.15.19:0/2525943562 >> v2:172.21.15.112:3300/0 conn(0x7f3abc0cfe10 0x7f3abc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:46.105 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.100+0000 7f3ac3d42640 1 --2- 172.21.15.19:0/2525943562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3abc15be00 0x7f3abc0cd1a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:46.105 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.100+0000 7f3ac3d42640 1 --2- 172.21.15.19:0/2525943562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3abc15ba20 0x7f3abc0b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:46.105 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.100+0000 7f3ac3d42640 1 -- 172.21.15.19:0/2525943562 >> 172.21.15.19:0/2525943562 conn(0x7f3abc0b3460 msgr2=0x7f3abc0b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:46.105 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.100+0000 7f3ac3d42640 1 -- 172.21.15.19:0/2525943562 shutdown_connections 2024-04-25T09:52:46.105 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.100+0000 7f3ac3d42640 1 -- 172.21.15.19:0/2525943562 wait complete. 2024-04-25T09:52:46.105 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.101+0000 7f3ac3d42640 1 Processor -- start 2024-04-25T09:52:46.105 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.101+0000 7f3ac3d42640 1 -- start start 2024-04-25T09:52:46.107 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac3d42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3abc0cfe10 0x7f3abc0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:46.107 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac3d42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3abc15ba20 0x7f3abc0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:46.107 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac3d42640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3abc15be00 0x7f3abc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:46.107 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac3d42640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc0d0280 con 0x7f3abc0cfe10 2024-04-25T09:52:46.107 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac3d42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc0cf860 con 0x7f3abc15ba20 2024-04-25T09:52:46.107 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac3d42640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc0ce780 con 0x7f3abc15be00 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac1ade640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3abc0cfe10 0x7f3abc0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac1ade640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3abc0cfe10 0x7f3abc0c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41388/0 (socket says 172.21.15.19:41388) 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac1ade640 1 -- 172.21.15.19:0/3838611291 learned_addr learned my addr 172.21.15.19:0/3838611291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac12dd640 1 --2- 172.21.15.19:0/3838611291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3abc15ba20 0x7f3abc0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac22df640 1 --2- 172.21.15.19:0/3838611291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3abc15be00 0x7f3abc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac1ade640 1 -- 172.21.15.19:0/3838611291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3abc15be00 msgr2=0x7f3abc14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac1ade640 1 --2- 172.21.15.19:0/3838611291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3abc15be00 0x7f3abc14b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac1ade640 1 -- 172.21.15.19:0/3838611291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3abc15ba20 msgr2=0x7f3abc0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac1ade640 1 --2- 172.21.15.19:0/3838611291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3abc15ba20 0x7f3abc0cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.102+0000 7f3ac1ade640 1 -- 172.21.15.19:0/3838611291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc0c9fe0 con 0x7f3abc0cfe10 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.103+0000 7f3ac1ade640 1 --2- 172.21.15.19:0/3838611291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3abc0cfe10 0x7f3abc0c96e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00ebe0 tx=0x7f3aac009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.103+0000 7f3ab27fc640 1 -- 172.21.15.19:0/3838611291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac017070 con 0x7f3abc0cfe10 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.103+0000 7f3ab27fc640 1 -- 172.21.15.19:0/3838611291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac00ca00 con 0x7f3abc0cfe10 2024-04-25T09:52:46.108 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.103+0000 7f3ac3d42640 1 -- 172.21.15.19:0/3838611291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3abc0ca2c0 con 0x7f3abc0cfe10 2024-04-25T09:52:46.109 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.103+0000 7f3ab27fc640 1 -- 172.21.15.19:0/3838611291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00cdd0 con 0x7f3abc0cfe10 2024-04-25T09:52:46.109 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.103+0000 7f3ac3d42640 1 -- 172.21.15.19:0/3838611291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3abc0ca880 con 0x7f3abc0cfe10 2024-04-25T09:52:46.109 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.103+0000 7f3ab27fc640 1 -- 172.21.15.19:0/3838611291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3aac01a070 con 0x7f3abc0cfe10 2024-04-25T09:52:46.109 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.103+0000 7f3ab27fc640 1 -- 172.21.15.19:0/3838611291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f3aac01cb20 con 0x7f3abc0cfe10 2024-04-25T09:52:46.109 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.104+0000 7f3ac3d42640 1 -- 172.21.15.19:0/3838611291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3abc0cc090 con 0x7f3abc0cfe10 2024-04-25T09:52:46.109 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.106+0000 7f3ab27fc640 1 -- 172.21.15.19:0/3838611291 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3abc0cc090 con 0x7f3abc0cfe10 2024-04-25T09:52:46.124 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:46.122+0000 7fbc99722040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T09:52:46.136 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:46.134+0000 7f7531021040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T09:52:46.171 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.168+0000 7f3ac3d42640 1 -- 172.21.15.19:0/3838611291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3abc15f560 con 0x7f3abc0cfe10 2024-04-25T09:52:46.172 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.169+0000 7f3ab27fc640 1 -- 172.21.15.19:0/3838611291 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f3aac01c450 con 0x7f3abc0cfe10 2024-04-25T09:52:46.172 INFO:teuthology.misc.health.smithi019.stdout: 2024-04-25T09:52:46.172 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":10,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:52:44.945978+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:52:46.173 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.170+0000 7f3ac3d42640 1 -- 172.21.15.19:0/3838611291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3abc0cfe10 msgr2=0x7f3abc0c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:46.173 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.170+0000 7f3ac3d42640 1 --2- 172.21.15.19:0/3838611291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3abc0cfe10 0x7f3abc0c96e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00ebe0 tx=0x7f3aac009e20 comp rx=0 tx=0).stop 2024-04-25T09:52:46.173 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.170+0000 7f3ac3d42640 1 -- 172.21.15.19:0/3838611291 shutdown_connections 2024-04-25T09:52:46.173 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.170+0000 7f3ac3d42640 1 --2- 172.21.15.19:0/3838611291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3abc15be00 0x7f3abc14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:46.173 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.170+0000 7f3ac3d42640 1 --2- 172.21.15.19:0/3838611291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3abc15ba20 0x7f3abc0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:46.173 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.170+0000 7f3ac3d42640 1 --2- 172.21.15.19:0/3838611291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3abc0cfe10 0x7f3abc0c96e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:46.174 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.170+0000 7f3ac3d42640 1 -- 172.21.15.19:0/3838611291 >> 172.21.15.19:0/3838611291 conn(0x7f3abc0b3460 msgr2=0x7f3abc0b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:46.174 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.170+0000 7f3ac3d42640 1 -- 172.21.15.19:0/3838611291 shutdown_connections 2024-04-25T09:52:46.174 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:46.170+0000 7f3ac3d42640 1 -- 172.21.15.19:0/3838611291 wait complete. 2024-04-25T09:52:46.182 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T09:52:46.187 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:46.186+0000 7f7531021040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T09:52:46.257 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:46.255+0000 7f7531021040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T09:52:46.269 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:46.266+0000 7fbc99722040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T09:52:46.302 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:46.300+0000 7f7531021040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T09:52:46.321 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:46.318+0000 7fbc99722040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T09:52:46.394 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:46.391+0000 7fbc99722040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T09:52:46.442 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:46.439+0000 7fbc99722040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T09:52:46.782 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:46.780+0000 7f7531021040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T09:52:46.897 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:46.896+0000 7f7531021040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T09:52:46.941 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:46.939+0000 7fbc99722040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T09:52:47.061 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T09:52:47.059+0000 7fbc99722040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T09:52:47.915 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:47.913+0000 7f74d175c640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T09:52:47.916 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-04-25T09:52:47.914+0000 7f74d175c640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T09:52:49.041 INFO:tasks.ceph.osd.4.smithi112.stderr:2024-04-25T09:52:49.039+0000 7ff56a827640 -1 osd.4 0 log_to_monitors true 2024-04-25T09:52:49.056 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-04-25T09:52:49.054+0000 7fc27b723640 -1 osd.0 0 log_to_monitors true 2024-04-25T09:52:49.080 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-04-25T09:52:49.078+0000 7f05b583d640 -1 osd.1 0 log_to_monitors true 2024-04-25T09:52:49.118 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-04-25T09:52:49.115+0000 7f62fa3a1640 -1 osd.2 0 log_to_monitors true 2024-04-25T09:52:49.125 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-04-25T09:52:49.123+0000 7fb7daa0d640 -1 osd.3 0 log_to_monitors true 2024-04-25T09:52:49.140 INFO:tasks.ceph.osd.7.smithi112.stderr:2024-04-25T09:52:49.138+0000 7f6311610640 -1 osd.7 0 log_to_monitors true 2024-04-25T09:52:49.142 INFO:tasks.ceph.osd.5.smithi112.stderr:2024-04-25T09:52:49.141+0000 7f597d215640 -1 osd.5 0 log_to_monitors true 2024-04-25T09:52:49.154 INFO:tasks.ceph.osd.6.smithi112.stderr:2024-04-25T09:52:49.152+0000 7f37d1011640 -1 osd.6 0 log_to_monitors true 2024-04-25T09:52:52.897 INFO:tasks.ceph.osd.6.smithi112.stderr:2024-04-25T09:52:52.894+0000 7f37cc8f3640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T09:52:52.937 INFO:tasks.ceph.osd.6.smithi112.stderr:2024-04-25T09:52:52.935+0000 7f37c4713640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:52:52.988 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:52:53.033 INFO:tasks.ceph.osd.4.smithi112.stderr:2024-04-25T09:52:53.031+0000 7ff5667bd640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T09:52:53.046 INFO:tasks.ceph.osd.4.smithi112.stderr:2024-04-25T09:52:53.044+0000 7ff55e5dd640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:52:53.060 INFO:tasks.ceph.osd.7.smithi112.stderr:2024-04-25T09:52:53.058+0000 7f630cfdf640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T09:52:53.074 INFO:tasks.ceph.osd.7.smithi112.stderr:2024-04-25T09:52:53.072+0000 7f6304dff640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:52:53.105 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-04-25T09:52:53.095+0000 7f05b17d3640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T09:52:53.111 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-04-25T09:52:53.108+0000 7fb7d62f3640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T09:52:53.123 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-04-25T09:52:53.119+0000 7f05a95f3640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:52:53.123 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-04-25T09:52:53.120+0000 7f62f6337640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T09:52:53.128 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-04-25T09:52:53.126+0000 7fb7ce113640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:52:53.130 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-04-25T09:52:53.128+0000 7fc2776b9640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T09:52:53.131 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.128+0000 7f3f38212640 1 Processor -- start 2024-04-25T09:52:53.131 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.129+0000 7f3f38212640 1 -- start start 2024-04-25T09:52:53.132 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.130+0000 7f3f38212640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f300b1e10 0x7f3f300b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:53.132 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.130+0000 7f3f38212640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f300b5580 con 0x7f3f3015e760 2024-04-25T09:52:53.132 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.130+0000 7f3f38212640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f300b5860 con 0x7f3f3006ab90 2024-04-25T09:52:53.133 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.130+0000 7f3f38212640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f300b3c50 con 0x7f3f300b1e10 2024-04-25T09:52:53.133 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.130+0000 7f3f367af640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3f3015e760 0x7f3f3015ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60014/0 (socket says 172.21.15.19:60014) 2024-04-25T09:52:53.133 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.130+0000 7f3f367af640 1 -- 172.21.15.19:0/746221680 learned_addr learned my addr 172.21.15.19:0/746221680 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:52:53.133 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.130+0000 7f3f357ad640 1 --2- 172.21.15.19:0/746221680 >> v2:172.21.15.112:3300/0 conn(0x7f3f300b1e10 0x7f3f300b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:53.133 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.130+0000 7f3f357ad640 1 -- 172.21.15.19:0/746221680 >> v1:172.21.15.112:6789/0 conn(0x7f3f3006ab90 legacy=0x7f3f3006af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.133 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.130+0000 7f3f357ad640 1 -- 172.21.15.19:0/746221680 >> v1:172.21.15.19:6789/0 conn(0x7f3f3015e760 legacy=0x7f3f3015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.133 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.130+0000 7f3f357ad640 1 -- 172.21.15.19:0/746221680 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300b5b40 con 0x7f3f300b1e10 2024-04-25T09:52:53.134 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.131+0000 7f3f357ad640 1 --2- 172.21.15.19:0/746221680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f300b1e10 0x7f3f300b3710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3f200097d0 tx=0x7f3f2002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10361a55df1df124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:53.136 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.134+0000 7f3f34fac640 1 -- 172.21.15.19:0/746221680 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20037070 con 0x7f3f300b1e10 2024-04-25T09:52:53.136 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.134+0000 7f3f34fac640 1 -- 172.21.15.19:0/746221680 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f20032440 con 0x7f3f300b1e10 2024-04-25T09:52:53.136 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.134+0000 7f3f34fac640 1 -- 172.21.15.19:0/746221680 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20032810 con 0x7f3f300b1e10 2024-04-25T09:52:53.137 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.134+0000 7f3f38212640 1 -- 172.21.15.19:0/746221680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f300b1e10 msgr2=0x7f3f300b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.137 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.134+0000 7f3f38212640 1 --2- 172.21.15.19:0/746221680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f300b1e10 0x7f3f300b3710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3f200097d0 tx=0x7f3f2002a270 comp rx=0 tx=0).stop 2024-04-25T09:52:53.137 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.134+0000 7f3f38212640 1 -- 172.21.15.19:0/746221680 shutdown_connections 2024-04-25T09:52:53.137 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.134+0000 7f3f38212640 1 --2- 172.21.15.19:0/746221680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f300b1e10 0x7f3f300b3710 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.137 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.134+0000 7f3f38212640 1 -- 172.21.15.19:0/746221680 >> 172.21.15.19:0/746221680 conn(0x7f3f300b5e50 msgr2=0x7f3f30099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:53.137 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.134+0000 7f3f38212640 1 -- 172.21.15.19:0/746221680 shutdown_connections 2024-04-25T09:52:53.137 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.134+0000 7f3f38212640 1 -- 172.21.15.19:0/746221680 wait complete. 2024-04-25T09:52:53.138 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.136+0000 7f3f38212640 1 Processor -- start 2024-04-25T09:52:53.138 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.136+0000 7f3f38212640 1 -- start start 2024-04-25T09:52:53.138 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-04-25T09:52:53.136+0000 7f62ee157640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:52:53.141 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.136+0000 7f3f38212640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f3006ab90 0x7f3f30143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:53.141 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.136+0000 7f3f38212640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f300b1e10 0x7f3f301f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:53.141 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.136+0000 7f3f38212640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f3015e760 0x7f3f30139a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:53.141 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.136+0000 7f3f38212640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3015e170 con 0x7f3f3006ab90 2024-04-25T09:52:53.141 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.136+0000 7f3f38212640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f30146b60 con 0x7f3f300b1e10 2024-04-25T09:52:53.141 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.136+0000 7f3f38212640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3013e4c0 con 0x7f3f3015e760 2024-04-25T09:52:53.141 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f35fae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f3006ab90 0x7f3f30143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:53.141 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f367af640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f3015e760 0x7f3f30139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:53.141 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f357ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f300b1e10 0x7f3f301f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:53.141 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f367af640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f3015e760 0x7f3f30139a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:46686/0 (socket says 172.21.15.19:46686) 2024-04-25T09:52:53.141 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f367af640 1 -- 172.21.15.19:0/1457074144 learned_addr learned my addr 172.21.15.19:0/1457074144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f35fae640 1 -- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f3015e760 msgr2=0x7f3f30139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f35fae640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f3015e760 0x7f3f30139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f35fae640 1 -- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f300b1e10 msgr2=0x7f3f301f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f35fae640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f300b1e10 0x7f3f301f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f35fae640 1 -- 172.21.15.19:0/1457074144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3013e710 con 0x7f3f3006ab90 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f367af640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f3015e760 0x7f3f30139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f357ad640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f300b1e10 0x7f3f301f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f35fae640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f3006ab90 0x7f3f30143ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c340 tx=0x7f3f2c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f267fc640 1 -- 172.21.15.19:0/1457074144 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0052d0 con 0x7f3f3006ab90 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.137+0000 7f3f38212640 1 -- 172.21.15.19:0/1457074144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3013e9f0 con 0x7f3f3006ab90 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.138+0000 7f3f267fc640 1 -- 172.21.15.19:0/1457074144 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2c005490 con 0x7f3f3006ab90 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.138+0000 7f3f267fc640 1 -- 172.21.15.19:0/1457074144 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c007260 con 0x7f3f3006ab90 2024-04-25T09:52:53.142 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.138+0000 7f3f38212640 1 -- 172.21.15.19:0/1457074144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3015da20 con 0x7f3f3006ab90 2024-04-25T09:52:53.143 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.139+0000 7f3f38212640 1 -- 172.21.15.19:0/1457074144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3004ef20 con 0x7f3f3006ab90 2024-04-25T09:52:53.143 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.139+0000 7f3f267fc640 1 -- 172.21.15.19:0/1457074144 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f3f2c007400 con 0x7f3f3006ab90 2024-04-25T09:52:53.143 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.139+0000 7f3f267fc640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f1406b440 0x7f3f1406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:53.143 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.139+0000 7f3f267fc640 1 -- 172.21.15.19:0/1457074144 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f3f2c088790 con 0x7f3f3006ab90 2024-04-25T09:52:53.144 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.140+0000 7f3f357ad640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f1406b440 0x7f3f1406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:53.144 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.140+0000 7f3f357ad640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f1406b440 0x7f3f1406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f20034000 tx=0x7f3f2002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:53.144 INFO:tasks.ceph.osd.5.smithi112.stderr:2024-04-25T09:52:53.142+0000 7f5978b0c640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T09:52:53.146 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-04-25T09:52:53.144+0000 7fc26f4d9640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:52:53.146 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.144+0000 7f3f267fc640 1 -- 172.21.15.19:0/1457074144 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f2c057d90 con 0x7f3f3006ab90 2024-04-25T09:52:53.158 INFO:tasks.ceph.osd.5.smithi112.stderr:2024-04-25T09:52:53.153+0000 7f597092c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:52:53.332 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.330+0000 7f3f38212640 1 -- 172.21.15.19:0/1457074144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f30141be0 con 0x7f3f3006ab90 2024-04-25T09:52:53.333 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.330+0000 7f3f267fc640 1 -- 172.21.15.19:0/1457074144 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f3f2c05be30 con 0x7f3f3006ab90 2024-04-25T09:52:53.333 INFO:teuthology.misc.health.smithi019.stdout: 2024-04-25T09:52:53.333 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":12,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:52:50.916396+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:52:53.335 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 -- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f1406b440 msgr2=0x7f3f1406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.335 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f1406b440 0x7f3f1406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f20034000 tx=0x7f3f2002e000 comp rx=0 tx=0).stop 2024-04-25T09:52:53.335 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 -- 172.21.15.19:0/1457074144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f3006ab90 msgr2=0x7f3f30143ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.335 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f3006ab90 0x7f3f30143ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c340 tx=0x7f3f2c00c800 comp rx=0 tx=0).stop 2024-04-25T09:52:53.336 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 -- 172.21.15.19:0/1457074144 shutdown_connections 2024-04-25T09:52:53.336 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f1406b440 0x7f3f1406d900 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.336 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f3015e760 0x7f3f30139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.336 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f300b1e10 0x7f3f301f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.336 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 --2- 172.21.15.19:0/1457074144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f3006ab90 0x7f3f30143ea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.336 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 -- 172.21.15.19:0/1457074144 >> 172.21.15.19:0/1457074144 conn(0x7f3f300b5e50 msgr2=0x7f3f30145160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:53.336 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 -- 172.21.15.19:0/1457074144 shutdown_connections 2024-04-25T09:52:53.336 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:52:53.332+0000 7f3f38212640 1 -- 172.21.15.19:0/1457074144 wait complete. 2024-04-25T09:52:53.344 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T09:53:00.152 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:00.256 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.253+0000 7f832ef22640 1 Processor -- start 2024-04-25T09:53:00.256 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.253+0000 7f832ef22640 1 -- start start 2024-04-25T09:53:00.258 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.254+0000 7f832ef22640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f832815dbb0 0x7f832815df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.258 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.254+0000 7f832ef22640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83280cf4d0 con 0x7f832815a230 2024-04-25T09:53:00.258 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.254+0000 7f832ef22640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832806e6e0 con 0x7f83280aede0 2024-04-25T09:53:00.258 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.254+0000 7f832ef22640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832806e9c0 con 0x7f832815dbb0 2024-04-25T09:53:00.258 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.254+0000 7f832ccbe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f832815dbb0 0x7f832815df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.258 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.254+0000 7f8327fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f832815a230 0x7f832815e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50288/0 (socket says 172.21.15.19:50288) 2024-04-25T09:53:00.258 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.254+0000 7f832ccbe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f832815dbb0 0x7f832815df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41438/0 (socket says 172.21.15.19:41438) 2024-04-25T09:53:00.258 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.254+0000 7f8327fff640 1 -- 172.21.15.19:0/2438307275 learned_addr learned my addr 172.21.15.19:0/2438307275 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:00.259 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.255+0000 7f832ccbe640 1 -- 172.21.15.19:0/2438307275 >> v1:172.21.15.112:6789/0 conn(0x7f83280aede0 legacy=0x7f83280b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.259 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.255+0000 7f832ccbe640 1 -- 172.21.15.19:0/2438307275 >> v1:172.21.15.19:6789/0 conn(0x7f832815a230 legacy=0x7f832815e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.259 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.255+0000 7f83277fe640 1 -- 172.21.15.19:0/2438307275 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524264984 0 0) 0x7f83280cf4d0 con 0x7f832815a230 2024-04-25T09:53:00.259 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.255+0000 7f832ccbe640 1 -- 172.21.15.19:0/2438307275 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832806eca0 con 0x7f832815dbb0 2024-04-25T09:53:00.259 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.255+0000 7f832ccbe640 1 --2- 172.21.15.19:0/2438307275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832815dbb0 0x7f832815df90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8320009d90 tx=0x7f832002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4c21ab833ed987 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:00.259 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.255+0000 7f83277fe640 1 -- 172.21.15.19:0/2438307275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320007740 con 0x7f832815dbb0 2024-04-25T09:53:00.259 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.256+0000 7f83277fe640 1 -- 172.21.15.19:0/2438307275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83200078e0 con 0x7f832815dbb0 2024-04-25T09:53:00.259 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.256+0000 7f83277fe640 1 -- 172.21.15.19:0/2438307275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320007cb0 con 0x7f832815dbb0 2024-04-25T09:53:00.260 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.256+0000 7f832ef22640 1 -- 172.21.15.19:0/2438307275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832815dbb0 msgr2=0x7f832815df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.260 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.256+0000 7f832ef22640 1 --2- 172.21.15.19:0/2438307275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832815dbb0 0x7f832815df90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8320009d90 tx=0x7f832002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.260 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.256+0000 7f832ef22640 1 -- 172.21.15.19:0/2438307275 shutdown_connections 2024-04-25T09:53:00.260 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.256+0000 7f832ef22640 1 --2- 172.21.15.19:0/2438307275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832815dbb0 0x7f832815df90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.260 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.256+0000 7f832ef22640 1 -- 172.21.15.19:0/2438307275 >> 172.21.15.19:0/2438307275 conn(0x7f83280b1470 msgr2=0x7f83280b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:00.260 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.256+0000 7f832ef22640 1 -- 172.21.15.19:0/2438307275 shutdown_connections 2024-04-25T09:53:00.260 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.256+0000 7f832ef22640 1 -- 172.21.15.19:0/2438307275 wait complete. 2024-04-25T09:53:00.261 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.257+0000 7f832ef22640 1 Processor -- start 2024-04-25T09:53:00.261 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.257+0000 7f832ef22640 1 -- start start 2024-04-25T09:53:00.263 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.258+0000 7f832ef22640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83280aede0 0x7f832815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.263 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.258+0000 7f832ef22640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832815a230 0x7f8328150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.263 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.258+0000 7f832ef22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832815dbb0 0x7f8328141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.263 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.258+0000 7f832ef22640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832806d420 con 0x7f832815a230 2024-04-25T09:53:00.263 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.258+0000 7f832ef22640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832806d2a0 con 0x7f832815dbb0 2024-04-25T09:53:00.263 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.258+0000 7f832ef22640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832806d5a0 con 0x7f83280aede0 2024-04-25T09:53:00.263 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.258+0000 7f8327fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832815a230 0x7f8328150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.263 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f8327fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832815a230 0x7f8328150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41454/0 (socket says 172.21.15.19:41454) 2024-04-25T09:53:00.263 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f8327fff640 1 -- 172.21.15.19:0/3491073636 learned_addr learned my addr 172.21.15.19:0/3491073636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:00.263 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f832ccbe640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83280aede0 0x7f832815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.263 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f832d4bf640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832815dbb0 0x7f8328141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f8327fff640 1 -- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83280aede0 msgr2=0x7f832815ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f8327fff640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83280aede0 0x7f832815ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f8327fff640 1 -- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832815dbb0 msgr2=0x7f8328141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f8327fff640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832815dbb0 0x7f8328141170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f8327fff640 1 -- 172.21.15.19:0/3491073636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83281400d0 con 0x7f832815a230 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f8327fff640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832815a230 0x7f8328150580 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f831c00c4a0 tx=0x7f831c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f83257fa640 1 -- 172.21.15.19:0/3491073636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c018070 con 0x7f832815a230 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f83257fa640 1 -- 172.21.15.19:0/3491073636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c0074e0 con 0x7f832815a230 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f83257fa640 1 -- 172.21.15.19:0/3491073636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c004a20 con 0x7f832815a230 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f832ef22640 1 -- 172.21.15.19:0/3491073636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83281403b0 con 0x7f832815a230 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.259+0000 7f832ef22640 1 -- 172.21.15.19:0/3491073636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8328140c50 con 0x7f832815a230 2024-04-25T09:53:00.264 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.260+0000 7f832ef22640 1 -- 172.21.15.19:0/3491073636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83281407a0 con 0x7f832815a230 2024-04-25T09:53:00.265 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.261+0000 7f83257fa640 1 -- 172.21.15.19:0/3491073636 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f831c01b070 con 0x7f832815a230 2024-04-25T09:53:00.265 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.261+0000 7f83257fa640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831406b4d0 0x7f831406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.265 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.261+0000 7f83257fa640 1 -- 172.21.15.19:0/3491073636 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f831c0890e0 con 0x7f832815a230 2024-04-25T09:53:00.265 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.261+0000 7f832ccbe640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831406b4d0 0x7f831406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.265 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.262+0000 7f832ccbe640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831406b4d0 0x7f831406d990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8320034000 tx=0x7f832002f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:00.268 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.266+0000 7f83257fa640 1 -- 172.21.15.19:0/3491073636 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f831c057cf0 con 0x7f832815a230 2024-04-25T09:53:00.452 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.449+0000 7f832ef22640 1 -- 172.21.15.19:0/3491073636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83280680b0 con 0x7f832815a230 2024-04-25T09:53:00.452 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.450+0000 7f83257fa640 1 -- 172.21.15.19:0/3491073636 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f831c05bd90 con 0x7f832815a230 2024-04-25T09:53:00.453 INFO:teuthology.misc.health.smithi019.stdout: 2024-04-25T09:53:00.453 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":16,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:52:56.957996+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:00.455 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 -- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831406b4d0 msgr2=0x7f831406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.455 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831406b4d0 0x7f831406d990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8320034000 tx=0x7f832002f000 comp rx=0 tx=0).stop 2024-04-25T09:53:00.455 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 -- 172.21.15.19:0/3491073636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832815a230 msgr2=0x7f8328150580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.455 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832815a230 0x7f8328150580 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f831c00c4a0 tx=0x7f831c00c960 comp rx=0 tx=0).stop 2024-04-25T09:53:00.455 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 -- 172.21.15.19:0/3491073636 shutdown_connections 2024-04-25T09:53:00.455 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831406b4d0 0x7f831406d990 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.455 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f832815dbb0 0x7f8328141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.456 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f832815a230 0x7f8328150580 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.456 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 --2- 172.21.15.19:0/3491073636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83280aede0 0x7f832815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.456 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 -- 172.21.15.19:0/3491073636 >> 172.21.15.19:0/3491073636 conn(0x7f83280b1470 msgr2=0x7f83281481e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:00.456 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 -- 172.21.15.19:0/3491073636 shutdown_connections 2024-04-25T09:53:00.456 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T09:53:00.452+0000 7f832ef22640 1 -- 172.21.15.19:0/3491073636 wait complete. 2024-04-25T09:53:00.465 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T09:53:00.465 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T09:53:00.465 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:53:00.466 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:53:00.468 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:53:00.468 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:53:00.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:53:00.470 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:53:00.470 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:53:00.470 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:53:00.473 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:53:00.473 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:53:00.473 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:53:00.475 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:53:00.475 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T09:53:00.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:00.544 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T09:53:00.544 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T09:53:00.545 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T09:53:00.543+0000 7fcf99dc7640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2024-04-25T09:53:00.555 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T09:53:00.556 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T09:53:00.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.578+0000 7f93ae14c640 1 Processor -- start 2024-04-25T09:53:00.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.578+0000 7f93ae14c640 1 -- start start 2024-04-25T09:53:00.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.579+0000 7f93ae14c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f93a815db50 0x7f93a80c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.579+0000 7f93ae14c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a80c7050 0x7f93a80cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.579+0000 7f93ae14c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a81625d0 con 0x7f93a80cd5b0 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.579+0000 7f93ae14c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a815df30 con 0x7f93a815db50 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.579+0000 7f93ae14c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a80cc2b0 con 0x7f93a80c7050 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.579+0000 7f93a77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f93a815db50 0x7f93a80c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.579+0000 7f93a77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f93a815db50 0x7f93a80c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41502/0 (socket says 172.21.15.19:41502) 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.579+0000 7f93a77fe640 1 -- 172.21.15.19:0/3304171147 learned_addr learned my addr 172.21.15.19:0/3304171147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.580+0000 7f939effd640 1 --2- 172.21.15.19:0/3304171147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a80c7050 0x7f93a80cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.580+0000 7f93a77fe640 1 -- 172.21.15.19:0/3304171147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a80c7050 msgr2=0x7f93a80cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.580+0000 7f93a77fe640 1 --2- 172.21.15.19:0/3304171147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a80c7050 0x7f93a80cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.580+0000 7f93a77fe640 1 -- 172.21.15.19:0/3304171147 >> v1:172.21.15.112:6789/0 conn(0x7f93a80cd5b0 legacy=0x7f93a80cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.580+0000 7f93a77fe640 1 -- 172.21.15.19:0/3304171147 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a80cc590 con 0x7f93a815db50 2024-04-25T09:53:00.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93a77fe640 1 --2- 172.21.15.19:0/3304171147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a815db50 0x7f93a80c6b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9398009b00 tx=0x7f939802a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70fe64bed75351e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:00.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93a6ffd640 1 -- 172.21.15.19:0/3304171147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398036070 con 0x7f93a815db50 2024-04-25T09:53:00.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93a6ffd640 1 -- 172.21.15.19:0/3304171147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9398031600 con 0x7f93a815db50 2024-04-25T09:53:00.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93a6ffd640 1 -- 172.21.15.19:0/3304171147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93980319d0 con 0x7f93a815db50 2024-04-25T09:53:00.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93ae14c640 1 -- 172.21.15.19:0/3304171147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a815db50 msgr2=0x7f93a80c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93ae14c640 1 --2- 172.21.15.19:0/3304171147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a815db50 0x7f93a80c6b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9398009b00 tx=0x7f939802a450 comp rx=0 tx=0).stop 2024-04-25T09:53:00.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93ae14c640 1 -- 172.21.15.19:0/3304171147 shutdown_connections 2024-04-25T09:53:00.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93ae14c640 1 --2- 172.21.15.19:0/3304171147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a80c7050 0x7f93a80cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93ae14c640 1 --2- 172.21.15.19:0/3304171147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a815db50 0x7f93a80c6b10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93ae14c640 1 -- 172.21.15.19:0/3304171147 >> 172.21.15.19:0/3304171147 conn(0x7f93a80d1800 msgr2=0x7f93a80d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:00.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93ae14c640 1 -- 172.21.15.19:0/3304171147 shutdown_connections 2024-04-25T09:53:00.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.581+0000 7f93ae14c640 1 -- 172.21.15.19:0/3304171147 wait complete. 2024-04-25T09:53:00.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.582+0000 7f93ae14c640 1 Processor -- start 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.583+0000 7f93ae14c640 1 -- start start 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.583+0000 7f93ae14c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80c7050 0x7f93a815cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.583+0000 7f93ae14c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a80cd5b0 0x7f93a815f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.583+0000 7f93ae14c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a815db50 0x7f93a80c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.583+0000 7f93ae14c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a80d0240 con 0x7f93a80c7050 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.583+0000 7f93ae14c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a80ccb50 con 0x7f93a80cd5b0 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.583+0000 7f93ae14c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a80d03c0 con 0x7f93a815db50 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.583+0000 7f93a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80c7050 0x7f93a815cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.583+0000 7f93a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80c7050 0x7f93a815cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41508/0 (socket says 172.21.15.19:41508) 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.583+0000 7f93a77fe640 1 -- 172.21.15.19:0/153813347 learned_addr learned my addr 172.21.15.19:0/153813347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f939effd640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a80cd5b0 0x7f93a815f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93a7fff640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a815db50 0x7f93a80c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93a77fe640 1 -- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a815db50 msgr2=0x7f93a80c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93a77fe640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a815db50 0x7f93a80c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93a77fe640 1 -- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a80cd5b0 msgr2=0x7f93a815f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93a77fe640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a80cd5b0 0x7f93a815f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93a77fe640 1 -- 172.21.15.19:0/153813347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a815d250 con 0x7f93a80c7050 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93a7fff640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a815db50 0x7f93a80c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f939effd640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a80cd5b0 0x7f93a815f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93a77fe640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80c7050 0x7f93a815cd10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9398000c00 tx=0x7f93980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93a4ff9640 1 -- 172.21.15.19:0/153813347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398040070 con 0x7f93a80c7050 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93a4ff9640 1 -- 172.21.15.19:0/153813347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9398031df0 con 0x7f93a80c7050 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93a4ff9640 1 -- 172.21.15.19:0/153813347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398036040 con 0x7f93a80c7050 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93ae14c640 1 -- 172.21.15.19:0/153813347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a8160eb0 con 0x7f93a80c7050 2024-04-25T09:53:00.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.584+0000 7f93ae14c640 1 -- 172.21.15.19:0/153813347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a80b5af0 con 0x7f93a80c7050 2024-04-25T09:53:00.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.585+0000 7f93ae14c640 1 -- 172.21.15.19:0/153813347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a8073990 con 0x7f93a80c7050 2024-04-25T09:53:00.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.586+0000 7f93a4ff9640 1 -- 172.21.15.19:0/153813347 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9398043070 con 0x7f93a80c7050 2024-04-25T09:53:00.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.586+0000 7f93a4ff9640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b4d0 0x7f938806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.586+0000 7f93a4ff9640 1 -- 172.21.15.19:0/153813347 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f93980aa360 con 0x7f93a80c7050 2024-04-25T09:53:00.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.587+0000 7f939effd640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b4d0 0x7f938806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.587+0000 7f939effd640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b4d0 0x7f938806d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f93a80b3240 tx=0x7f9390008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:00.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.590+0000 7f93a4ff9640 1 -- 172.21.15.19:0/153813347 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93980795c0 con 0x7f93a80c7050 2024-04-25T09:53:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.773+0000 7f93ae14c640 1 -- 172.21.15.19:0/153813347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93a8161390 con 0x7f93a80c7050 2024-04-25T09:53:00.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.774+0000 7f93a4ff9640 1 -- 172.21.15.19:0/153813347 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f9398078ec0 con 0x7f93a80c7050 2024-04-25T09:53:00.777 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:00.777 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":16,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:52:56.957996+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:00.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 -- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b4d0 msgr2=0x7f938806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b4d0 0x7f938806d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f93a80b3240 tx=0x7f9390008040 comp rx=0 tx=0).stop 2024-04-25T09:53:00.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 -- 172.21.15.19:0/153813347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80c7050 msgr2=0x7f93a815cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80c7050 0x7f93a815cd10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9398000c00 tx=0x7f93980046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 -- 172.21.15.19:0/153813347 shutdown_connections 2024-04-25T09:53:00.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b4d0 0x7f938806d990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a815db50 0x7f93a80c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a80cd5b0 0x7f93a815f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 --2- 172.21.15.19:0/153813347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80c7050 0x7f93a815cd10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 -- 172.21.15.19:0/153813347 >> 172.21.15.19:0/153813347 conn(0x7f93a80d1800 msgr2=0x7f93a8158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:00.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 -- 172.21.15.19:0/153813347 shutdown_connections 2024-04-25T09:53:00.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.776+0000 7f93ae14c640 1 -- 172.21.15.19:0/153813347 wait complete. 2024-04-25T09:53:00.789 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:53:00.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:00.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.891+0000 7f805b8b3640 1 Processor -- start 2024-04-25T09:53:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.891+0000 7f805b8b3640 1 -- start start 2024-04-25T09:53:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f805b8b3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f80540ced80 0x7f80540d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f805b8b3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805415e000 con 0x7f805415dc20 2024-04-25T09:53:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f805b8b3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80540d1de0 con 0x7f8054161710 2024-04-25T09:53:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f805b8b3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8054161c60 con 0x7f80540ced80 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f805964f640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f805415dc20 0x7f80540ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50308/0 (socket says 172.21.15.19:50308) 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f805964f640 1 -- 172.21.15.19:0/1394543837 learned_addr learned my addr 172.21.15.19:0/1394543837 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149665461 0 0) 0x7f805415e000 con 0x7f805415dc20 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f8058e4e640 1 --2- 172.21.15.19:0/1394543837 >> v2:172.21.15.112:3300/0 conn(0x7f80540ced80 0x7f80540d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8040003520 con 0x7f805415dc20 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4241788431 0 0) 0x7f8040003520 con 0x7f805415dc20 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 >> v2:172.21.15.112:3300/0 conn(0x7f80540ced80 msgr2=0x7f80540d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f804bfff640 1 --2- 172.21.15.19:0/1394543837 >> v2:172.21.15.112:3300/0 conn(0x7f80540ced80 0x7f80540d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 >> v1:172.21.15.112:6789/0 conn(0x7f8054161710 legacy=0x7f80540d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8054161f40 con 0x7f805415dc20 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8050001670 con 0x7f805415dc20 2024-04-25T09:53:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 >> v1:172.21.15.19:6789/0 conn(0x7f805415dc20 legacy=0x7f80540ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.892+0000 7f804bfff640 1 --2- 172.21.15.19:0/1394543837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80400039b0 0x7f8040003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f804bfff640 1 --2- 172.21.15.19:0/1394543837 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8040004800 0x7f8040004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f804bfff640 1 --2- 172.21.15.19:0/1394543837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80400051c0 0x7f804000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8054161c60 con 0x7f80400039b0 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804000c060 con 0x7f80400051c0 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804000c260 con 0x7f8040004800 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f805964f640 1 --2- 172.21.15.19:0/1394543837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80400039b0 0x7f8040003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f8059e50640 1 --2- 172.21.15.19:0/1394543837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80400051c0 0x7f804000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f8058e4e640 1 --2- 172.21.15.19:0/1394543837 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8040004800 0x7f8040004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f805964f640 1 -- 172.21.15.19:0/1394543837 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8040004800 msgr2=0x7f8040004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f805964f640 1 --2- 172.21.15.19:0/1394543837 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8040004800 0x7f8040004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f805964f640 1 -- 172.21.15.19:0/1394543837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80400051c0 msgr2=0x7f804000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f805964f640 1 --2- 172.21.15.19:0/1394543837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80400051c0 0x7f804000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f805964f640 1 -- 172.21.15.19:0/1394543837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040003520 con 0x7f80400039b0 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.893+0000 7f8059e50640 1 -- 172.21.15.19:0/1394543837 reap_dead start 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.894+0000 7f805964f640 1 --2- 172.21.15.19:0/1394543837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80400039b0 0x7f8040003dd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f805000c240 tx=0x7f8040005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703b08c926912077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.894+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050007d10 con 0x7f80400039b0 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.894+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8050005ad0 con 0x7f80400039b0 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.894+0000 7f804bfff640 1 -- 172.21.15.19:0/1394543837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050007070 con 0x7f80400039b0 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.894+0000 7f805b8b3640 1 -- 172.21.15.19:0/1394543837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80400039b0 msgr2=0x7f8040003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.894+0000 7f805b8b3640 1 --2- 172.21.15.19:0/1394543837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80400039b0 0x7f8040003dd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f805000c240 tx=0x7f8040005830 comp rx=0 tx=0).stop 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.894+0000 7f805b8b3640 1 -- 172.21.15.19:0/1394543837 shutdown_connections 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.894+0000 7f805b8b3640 1 --2- 172.21.15.19:0/1394543837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80400039b0 0x7f8040003dd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.894+0000 7f805b8b3640 1 -- 172.21.15.19:0/1394543837 >> 172.21.15.19:0/1394543837 conn(0x7f80540d0420 msgr2=0x7f80540d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.894+0000 7f805b8b3640 1 -- 172.21.15.19:0/1394543837 shutdown_connections 2024-04-25T09:53:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.894+0000 7f805b8b3640 1 -- 172.21.15.19:0/1394543837 wait complete. 2024-04-25T09:53:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.895+0000 7f805b8b3640 1 Processor -- start 2024-04-25T09:53:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f805b8b3640 1 -- start start 2024-04-25T09:53:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f805b8b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f80400039b0 0x7f80541f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f805b8b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8054149950 0x7f80541f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f805b8b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80540aa380 0x7f80540aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f805b8b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80540b5e50 con 0x7f80540aa380 2024-04-25T09:53:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f805b8b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8054162500 con 0x7f8054149950 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f805b8b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80540b5fd0 con 0x7f80400039b0 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f8059e50640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80540aa380 0x7f80540aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f8059e50640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80540aa380 0x7f80540aa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41530/0 (socket says 172.21.15.19:41530) 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f8059e50640 1 -- 172.21.15.19:0/3780155888 learned_addr learned my addr 172.21.15.19:0/3780155888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f8058e4e640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8054149950 0x7f80541f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.896+0000 7f805964f640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f80400039b0 0x7f80541f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f8059e50640 1 -- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f80400039b0 msgr2=0x7f80541f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f8059e50640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f80400039b0 0x7f80541f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f8059e50640 1 -- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8054149950 msgr2=0x7f80541f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f8059e50640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8054149950 0x7f80541f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f8059e50640 1 -- 172.21.15.19:0/3780155888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80541f4200 con 0x7f80540aa380 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f8058e4e640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8054149950 0x7f80541f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f8059e50640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80540aa380 0x7f80540aa800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f804c007e20 tx=0x7f804c00e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f8049ffb640 1 -- 172.21.15.19:0/3780155888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c019070 con 0x7f80540aa380 2024-04-25T09:53:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f8049ffb640 1 -- 172.21.15.19:0/3780155888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c006d60 con 0x7f80540aa380 2024-04-25T09:53:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f805b8b3640 1 -- 172.21.15.19:0/3780155888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80541f44e0 con 0x7f80540aa380 2024-04-25T09:53:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f8049ffb640 1 -- 172.21.15.19:0/3780155888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c003b90 con 0x7f80540aa380 2024-04-25T09:53:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.897+0000 7f805b8b3640 1 -- 172.21.15.19:0/3780155888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80541f4aa0 con 0x7f80540aa380 2024-04-25T09:53:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.898+0000 7f805b8b3640 1 -- 172.21.15.19:0/3780155888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8054073a00 con 0x7f80540aa380 2024-04-25T09:53:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.898+0000 7f8049ffb640 1 -- 172.21.15.19:0/3780155888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f804c005110 con 0x7f80540aa380 2024-04-25T09:53:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.899+0000 7f8049ffb640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f803806b4d0 0x7f803806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.899+0000 7f8049ffb640 1 -- 172.21.15.19:0/3780155888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f804c089250 con 0x7f80540aa380 2024-04-25T09:53:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.899+0000 7f805964f640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f803806b4d0 0x7f803806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.899+0000 7f805964f640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f803806b4d0 0x7f803806d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8050033000 tx=0x7f805002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:00.903+0000 7f8049ffb640 1 -- 172.21.15.19:0/3780155888 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f804c057e60 con 0x7f80540aa380 2024-04-25T09:53:01.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.083+0000 7f805b8b3640 1 -- 172.21.15.19:0/3780155888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80540d16b0 con 0x7f80540aa380 2024-04-25T09:53:01.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.083+0000 7f8049ffb640 1 -- 172.21.15.19:0/3780155888 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f804c05ce20 con 0x7f80540aa380 2024-04-25T09:53:01.086 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 -- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f803806b4d0 msgr2=0x7f803806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f803806b4d0 0x7f803806d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8050033000 tx=0x7f805002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 -- 172.21.15.19:0/3780155888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80540aa380 msgr2=0x7f80540aa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80540aa380 0x7f80540aa800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f804c007e20 tx=0x7f804c00e910 comp rx=0 tx=0).stop 2024-04-25T09:53:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 -- 172.21.15.19:0/3780155888 shutdown_connections 2024-04-25T09:53:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f803806b4d0 0x7f803806d990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80540aa380 0x7f80540aa800 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8054149950 0x7f80541f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 --2- 172.21.15.19:0/3780155888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f80400039b0 0x7f80541f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 -- 172.21.15.19:0/3780155888 >> 172.21.15.19:0/3780155888 conn(0x7f80540d0420 msgr2=0x7f80540c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 -- 172.21.15.19:0/3780155888 shutdown_connections 2024-04-25T09:53:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.085+0000 7f805b8b3640 1 -- 172.21.15.19:0/3780155888 wait complete. 2024-04-25T09:53:01.099 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:01.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.197+0000 7ff599322640 1 Processor -- start 2024-04-25T09:53:01.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.197+0000 7ff599322640 1 -- start start 2024-04-25T09:53:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff599322640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff59415dbe0 0x7ff5940d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff599322640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5940b92a0 0x7ff5940d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff599322640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5940b97a0 con 0x7ff5940b2cb0 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff599322640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59415dfc0 con 0x7ff59415dbe0 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff599322640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940d1a00 con 0x7ff5940b92a0 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff58bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff5940b2cb0 0x7ff5940b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50322/0 (socket says 172.21.15.19:50322) 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff58bfff640 1 -- 172.21.15.19:0/3575539916 learned_addr learned my addr 172.21.15.19:0/3575539916 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff5937fe640 1 --2- 172.21.15.19:0/3575539916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5940b92a0 0x7ff5940d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff592ffd640 1 --2- 172.21.15.19:0/3575539916 >> v2:172.21.15.112:3300/0 conn(0x7ff59415dbe0 0x7ff5940d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff5927fc640 1 -- 172.21.15.19:0/3575539916 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824955058 0 0) 0x7ff5940b97a0 con 0x7ff5940b2cb0 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff5927fc640 1 -- 172.21.15.19:0/3575539916 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff580003520 con 0x7ff5940b2cb0 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff592ffd640 1 -- 172.21.15.19:0/3575539916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5940b92a0 msgr2=0x7ff5940d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff592ffd640 1 --2- 172.21.15.19:0/3575539916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5940b92a0 0x7ff5940d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff592ffd640 1 -- 172.21.15.19:0/3575539916 >> v1:172.21.15.19:6789/0 conn(0x7ff5940b2cb0 legacy=0x7ff5940b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.198+0000 7ff592ffd640 1 -- 172.21.15.19:0/3575539916 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940d1c50 con 0x7ff59415dbe0 2024-04-25T09:53:01.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.199+0000 7ff5937fe640 1 --2- 172.21.15.19:0/3575539916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5940b92a0 0x7ff5940d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.199+0000 7ff592ffd640 1 --2- 172.21.15.19:0/3575539916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff59415dbe0 0x7ff5940d0910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097a0 tx=0x7ff58402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c24e7df45532619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.199+0000 7ff5927fc640 1 -- 172.21.15.19:0/3575539916 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584037070 con 0x7ff59415dbe0 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.199+0000 7ff5927fc640 1 -- 172.21.15.19:0/3575539916 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff584032440 con 0x7ff59415dbe0 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.199+0000 7ff5927fc640 1 -- 172.21.15.19:0/3575539916 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584032810 con 0x7ff59415dbe0 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.200+0000 7ff599322640 1 -- 172.21.15.19:0/3575539916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff59415dbe0 msgr2=0x7ff5940d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.200+0000 7ff599322640 1 --2- 172.21.15.19:0/3575539916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff59415dbe0 0x7ff5940d0910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097a0 tx=0x7ff58402a180 comp rx=0 tx=0).stop 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.200+0000 7ff599322640 1 -- 172.21.15.19:0/3575539916 shutdown_connections 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.200+0000 7ff599322640 1 --2- 172.21.15.19:0/3575539916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5940b92a0 0x7ff5940d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.200+0000 7ff599322640 1 --2- 172.21.15.19:0/3575539916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff59415dbe0 0x7ff5940d0910 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.200+0000 7ff599322640 1 -- 172.21.15.19:0/3575539916 >> 172.21.15.19:0/3575539916 conn(0x7ff594061a20 msgr2=0x7ff594061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.200+0000 7ff599322640 1 -- 172.21.15.19:0/3575539916 shutdown_connections 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.200+0000 7ff599322640 1 -- 172.21.15.19:0/3575539916 wait complete. 2024-04-25T09:53:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.201+0000 7ff599322640 1 Processor -- start 2024-04-25T09:53:01.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.201+0000 7ff599322640 1 -- start start 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff599322640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5940b2cb0 0x7ff5941e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff599322640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940b92a0 0x7ff5941e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff599322640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff59415dbe0 0x7ff5940cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff599322640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940b6ab0 con 0x7ff5940b2cb0 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff599322640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940d2210 con 0x7ff5940b92a0 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff599322640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940b6c30 con 0x7ff59415dbe0 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff592ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5940b2cb0 0x7ff5941e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff592ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5940b2cb0 0x7ff5941e60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41532/0 (socket says 172.21.15.19:41532) 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff592ffd640 1 -- 172.21.15.19:0/2588876297 learned_addr learned my addr 172.21.15.19:0/2588876297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff5937fe640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff59415dbe0 0x7ff5940cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff58bfff640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940b92a0 0x7ff5941e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff592ffd640 1 -- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff59415dbe0 msgr2=0x7ff5940cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff592ffd640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff59415dbe0 0x7ff5940cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff592ffd640 1 -- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940b92a0 msgr2=0x7ff5941e71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff592ffd640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940b92a0 0x7ff5941e71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff592ffd640 1 -- 172.21.15.19:0/2588876297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940cc480 con 0x7ff5940b2cb0 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.202+0000 7ff58bfff640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940b92a0 0x7ff5941e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.203+0000 7ff592ffd640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5940b2cb0 0x7ff5941e60a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff584035040 tx=0x7ff5840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.203+0000 7ff58b7fe640 1 -- 172.21.15.19:0/2588876297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584041030 con 0x7ff5940b2cb0 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.203+0000 7ff58b7fe640 1 -- 172.21.15.19:0/2588876297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58403d4e0 con 0x7ff5940b2cb0 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.203+0000 7ff58b7fe640 1 -- 172.21.15.19:0/2588876297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584037040 con 0x7ff5940b2cb0 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.203+0000 7ff599322640 1 -- 172.21.15.19:0/2588876297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5940cc760 con 0x7ff5940b2cb0 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.203+0000 7ff599322640 1 -- 172.21.15.19:0/2588876297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5940ccd20 con 0x7ff5940b2cb0 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.204+0000 7ff599322640 1 -- 172.21.15.19:0/2588876297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff594073990 con 0x7ff5940b2cb0 2024-04-25T09:53:01.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.204+0000 7ff58b7fe640 1 -- 172.21.15.19:0/2588876297 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff584044070 con 0x7ff5940b2cb0 2024-04-25T09:53:01.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.205+0000 7ff58b7fe640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff57406b4d0 0x7ff57406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.205+0000 7ff58b7fe640 1 -- 172.21.15.19:0/2588876297 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff5840aa560 con 0x7ff5940b2cb0 2024-04-25T09:53:01.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.205+0000 7ff58bfff640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff57406b4d0 0x7ff57406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.205+0000 7ff58bfff640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff57406b4d0 0x7ff57406d990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff59406cbd0 tx=0x7ff57c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:01.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.209+0000 7ff58b7fe640 1 -- 172.21.15.19:0/2588876297 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff584044e40 con 0x7ff5940b2cb0 2024-04-25T09:53:01.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.426+0000 7ff599322640 1 -- 172.21.15.19:0/2588876297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff594154c60 con 0x7ff5940b2cb0 2024-04-25T09:53:01.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.427+0000 7ff58b7fe640 1 -- 172.21.15.19:0/2588876297 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7ff584079730 con 0x7ff5940b2cb0 2024-04-25T09:53:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:01.430 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T09:53:01.430 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 2 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 -- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff57406b4d0 msgr2=0x7ff57406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff57406b4d0 0x7ff57406d990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff59406cbd0 tx=0x7ff57c009040 comp rx=0 tx=0).stop 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 -- 172.21.15.19:0/2588876297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5940b2cb0 msgr2=0x7ff5941e60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5940b2cb0 0x7ff5941e60a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff584035040 tx=0x7ff5840046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 -- 172.21.15.19:0/2588876297 shutdown_connections 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff57406b4d0 0x7ff57406d990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff59415dbe0 0x7ff5940cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5940b92a0 0x7ff5941e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 --2- 172.21.15.19:0/2588876297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5940b2cb0 0x7ff5941e60a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 -- 172.21.15.19:0/2588876297 >> 172.21.15.19:0/2588876297 conn(0x7ff594061a20 msgr2=0x7ff59415c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 -- 172.21.15.19:0/2588876297 shutdown_connections 2024-04-25T09:53:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:01.429+0000 7ff599322640 1 -- 172.21.15.19:0/2588876297 wait complete. 2024-04-25T09:53:03.443 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:03.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.554+0000 7f0530dd5640 1 Processor -- start 2024-04-25T09:53:03.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.554+0000 7f0530dd5640 1 -- start start 2024-04-25T09:53:03.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f0530dd5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f052c15dbe0 0x7f052c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f0530dd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c0b92a0 0x7f052c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f0530dd5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052c0b97a0 con 0x7f052c0b2cb0 2024-04-25T09:53:03.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f0530dd5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c15dfc0 con 0x7f052c15dbe0 2024-04-25T09:53:03.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f0530dd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0d1a00 con 0x7f052c0b92a0 2024-04-25T09:53:03.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f052a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f052c15dbe0 0x7f052c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f052ad9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c0b92a0 0x7f052c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f0529d9b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f052c0b2cb0 0x7f052c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:55246/0 (socket says 172.21.15.19:55246) 2024-04-25T09:53:03.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f0529d9b640 1 -- 172.21.15.19:0/1951268242 learned_addr learned my addr 172.21.15.19:0/1951268242 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:03.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f052a59c640 1 -- 172.21.15.19:0/1951268242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c0b92a0 msgr2=0x7f052c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:03.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f052a59c640 1 --2- 172.21.15.19:0/1951268242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c0b92a0 0x7f052c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f052a59c640 1 -- 172.21.15.19:0/1951268242 >> v1:172.21.15.112:6789/0 conn(0x7f052c0b2cb0 legacy=0x7f052c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:03.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f052a59c640 1 -- 172.21.15.19:0/1951268242 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c0d1c50 con 0x7f052c15dbe0 2024-04-25T09:53:03.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f052ad9d640 1 --2- 172.21.15.19:0/1951268242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c0b92a0 0x7f052c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:03.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.556+0000 7f052a59c640 1 --2- 172.21.15.19:0/1951268242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f052c15dbe0 0x7f052c0d0910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f051c0097a0 tx=0x7f051c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff10c5b0d0068fef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:03.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.557+0000 7f052959a640 1 -- 172.21.15.19:0/1951268242 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c037070 con 0x7f052c15dbe0 2024-04-25T09:53:03.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.557+0000 7f052959a640 1 -- 172.21.15.19:0/1951268242 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c032440 con 0x7f052c15dbe0 2024-04-25T09:53:03.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.557+0000 7f052959a640 1 -- 172.21.15.19:0/1951268242 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c032810 con 0x7f052c15dbe0 2024-04-25T09:53:03.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.557+0000 7f0530dd5640 1 -- 172.21.15.19:0/1951268242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f052c15dbe0 msgr2=0x7f052c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:03.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.557+0000 7f0530dd5640 1 --2- 172.21.15.19:0/1951268242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f052c15dbe0 0x7f052c0d0910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f051c0097a0 tx=0x7f051c02a180 comp rx=0 tx=0).stop 2024-04-25T09:53:03.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.557+0000 7f0530dd5640 1 -- 172.21.15.19:0/1951268242 shutdown_connections 2024-04-25T09:53:03.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.557+0000 7f0530dd5640 1 --2- 172.21.15.19:0/1951268242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c0b92a0 0x7f052c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.557+0000 7f0530dd5640 1 --2- 172.21.15.19:0/1951268242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f052c15dbe0 0x7f052c0d0910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.557+0000 7f0530dd5640 1 -- 172.21.15.19:0/1951268242 >> 172.21.15.19:0/1951268242 conn(0x7f052c061a20 msgr2=0x7f052c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:03.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.557+0000 7f0530dd5640 1 -- 172.21.15.19:0/1951268242 shutdown_connections 2024-04-25T09:53:03.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.557+0000 7f0530dd5640 1 -- 172.21.15.19:0/1951268242 wait complete. 2024-04-25T09:53:03.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.559+0000 7f0530dd5640 1 Processor -- start 2024-04-25T09:53:03.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.559+0000 7f0530dd5640 1 -- start start 2024-04-25T09:53:03.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.559+0000 7f0530dd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0b2cb0 0x7f052c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.559+0000 7f0530dd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f052c0b92a0 0x7f052c0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.559+0000 7f0530dd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c15dbe0 0x7f052c0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.559+0000 7f0530dd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0b6ab0 con 0x7f052c0b2cb0 2024-04-25T09:53:03.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.559+0000 7f0530dd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0d2210 con 0x7f052c0b92a0 2024-04-25T09:53:03.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.559+0000 7f0530dd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0b6c30 con 0x7f052c15dbe0 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f052a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0b2cb0 0x7f052c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f052a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0b2cb0 0x7f052c0cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41540/0 (socket says 172.21.15.19:41540) 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f052a59c640 1 -- 172.21.15.19:0/2949971505 learned_addr learned my addr 172.21.15.19:0/2949971505 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f052ad9d640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c15dbe0 0x7f052c0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f0529d9b640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f052c0b92a0 0x7f052c0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f052a59c640 1 -- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c15dbe0 msgr2=0x7f052c0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f052a59c640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c15dbe0 0x7f052c0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f052a59c640 1 -- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f052c0b92a0 msgr2=0x7f052c0c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f052a59c640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f052c0b92a0 0x7f052c0c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f052a59c640 1 -- 172.21.15.19:0/2949971505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c0ca1a0 con 0x7f052c0b2cb0 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f052ad9d640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c15dbe0 0x7f052c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f052a59c640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0b2cb0 0x7f052c0cd740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f051c035040 tx=0x7f051c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:03.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f051affd640 1 -- 172.21.15.19:0/2949971505 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c037040 con 0x7f052c0b2cb0 2024-04-25T09:53:03.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f051affd640 1 -- 172.21.15.19:0/2949971505 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c030500 con 0x7f052c0b2cb0 2024-04-25T09:53:03.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f0530dd5640 1 -- 172.21.15.19:0/2949971505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c0cc6a0 con 0x7f052c0b2cb0 2024-04-25T09:53:03.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f051affd640 1 -- 172.21.15.19:0/2949971505 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c0308d0 con 0x7f052c0b2cb0 2024-04-25T09:53:03.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.560+0000 7f0530dd5640 1 -- 172.21.15.19:0/2949971505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c0ccc60 con 0x7f052c0b2cb0 2024-04-25T09:53:03.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.561+0000 7f0530dd5640 1 -- 172.21.15.19:0/2949971505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c073990 con 0x7f052c0b2cb0 2024-04-25T09:53:03.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.562+0000 7f051affd640 1 -- 172.21.15.19:0/2949971505 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f051c044020 con 0x7f052c0b2cb0 2024-04-25T09:53:03.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.562+0000 7f051affd640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f050c06b4d0 0x7f050c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.562+0000 7f051affd640 1 -- 172.21.15.19:0/2949971505 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f051c0aa130 con 0x7f052c0b2cb0 2024-04-25T09:53:03.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.562+0000 7f0529d9b640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f050c06b4d0 0x7f050c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.563+0000 7f0529d9b640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f050c06b4d0 0x7f050c06d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0520005880 tx=0x7f0520009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:03.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.567+0000 7f051affd640 1 -- 172.21.15.19:0/2949971505 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f051c079300 con 0x7f052c0b2cb0 2024-04-25T09:53:03.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.787+0000 7f0530dd5640 1 -- 172.21.15.19:0/2949971505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f052c1554a0 con 0x7f052c0b2cb0 2024-04-25T09:53:03.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.787+0000 7f051affd640 1 -- 172.21.15.19:0/2949971505 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7f051c078c00 con 0x7f052c0b2cb0 2024-04-25T09:53:03.790 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:03.790 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4000846111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":4000846111},{"type":"v1","addr":"172.21.15.112:6837","nonce":4000846111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1969525282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1969525282},{"type":"v1","addr":"172.21.15.19:6835","nonce":1969525282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/1080352556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1080352556},{"type":"v1","addr":"172.21.15.112:6836","nonce":1080352556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T09:53:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-04-25T09:53:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 -- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f050c06b4d0 msgr2=0x7f050c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f050c06b4d0 0x7f050c06d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0520005880 tx=0x7f0520009040 comp rx=0 tx=0).stop 2024-04-25T09:53:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 -- 172.21.15.19:0/2949971505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0b2cb0 msgr2=0x7f052c0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0b2cb0 0x7f052c0cd740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f051c035040 tx=0x7f051c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 -- 172.21.15.19:0/2949971505 shutdown_connections 2024-04-25T09:53:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f050c06b4d0 0x7f050c06d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f052c15dbe0 0x7f052c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f052c0b92a0 0x7f052c0c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 --2- 172.21.15.19:0/2949971505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0b2cb0 0x7f052c0cd740 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 -- 172.21.15.19:0/2949971505 >> 172.21.15.19:0/2949971505 conn(0x7f052c061a20 msgr2=0x7f052c15b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:03.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 -- 172.21.15.19:0/2949971505 shutdown_connections 2024-04-25T09:53:03.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.789+0000 7f0530dd5640 1 -- 172.21.15.19:0/2949971505 wait complete. 2024-04-25T09:53:03.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.901+0000 7ff4db342640 1 Processor -- start 2024-04-25T09:53:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.901+0000 7ff4db342640 1 -- start start 2024-04-25T09:53:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.902+0000 7ff4db342640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff4d40d1020 0x7ff4d40d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.902+0000 7ff4db342640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d40b97a0 con 0x7ff4d415dbe0 2024-04-25T09:53:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.902+0000 7ff4db342640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d40d1c50 con 0x7ff4d40b2cb0 2024-04-25T09:53:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.902+0000 7ff4db342640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d415dfc0 con 0x7ff4d40d1020 2024-04-25T09:53:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.902+0000 7ff4d90de640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff4d415dbe0 0x7ff4d40d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50338/0 (socket says 172.21.15.19:50338) 2024-04-25T09:53:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.902+0000 7ff4d90de640 1 -- 172.21.15.19:0/1856770959 learned_addr learned my addr 172.21.15.19:0/1856770959 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.902+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199792585 0 0) 0x7ff4d40b97a0 con 0x7ff4d415dbe0 2024-04-25T09:53:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.902+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4bc003520 con 0x7ff4d415dbe0 2024-04-25T09:53:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.902+0000 7ff4d98df640 1 --2- 172.21.15.19:0/1856770959 >> v2:172.21.15.112:3300/0 conn(0x7ff4d40d1020 0x7ff4d40d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3705953972 0 0) 0x7ff4bc003520 con 0x7ff4d415dbe0 2024-04-25T09:53:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 >> v2:172.21.15.112:3300/0 conn(0x7ff4d40d1020 msgr2=0x7ff4d40d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 --2- 172.21.15.19:0/1856770959 >> v2:172.21.15.112:3300/0 conn(0x7ff4d40d1020 0x7ff4d40d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 >> v1:172.21.15.112:6789/0 conn(0x7ff4d40b2cb0 legacy=0x7ff4d40d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d40d1f30 con 0x7ff4d415dbe0 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7ff4cc001670 con 0x7ff4d415dbe0 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d98df640 1 --2- 172.21.15.19:0/1856770959 >> v2:172.21.15.112:3300/0 conn(0x7ff4d40d1020 0x7ff4d40d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 >> v1:172.21.15.19:6789/0 conn(0x7ff4d415dbe0 legacy=0x7ff4d40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 --2- 172.21.15.19:0/1856770959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4bc0039b0 0x7ff4bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 --2- 172.21.15.19:0/1856770959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4bc004800 0x7ff4bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 --2- 172.21.15.19:0/1856770959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4bc0051c0 0x7ff4bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc00c0e0 con 0x7ff4bc0051c0 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc00c290 con 0x7ff4bc0039b0 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc00c490 con 0x7ff4bc004800 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d98df640 1 --2- 172.21.15.19:0/1856770959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4bc0051c0 0x7ff4bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d90de640 1 --2- 172.21.15.19:0/1856770959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4bc0039b0 0x7ff4bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d88dd640 1 --2- 172.21.15.19:0/1856770959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4bc004800 0x7ff4bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d98df640 1 -- 172.21.15.19:0/1856770959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4bc004800 msgr2=0x7ff4bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d98df640 1 --2- 172.21.15.19:0/1856770959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4bc004800 0x7ff4bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d98df640 1 -- 172.21.15.19:0/1856770959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4bc0039b0 msgr2=0x7ff4bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d98df640 1 --2- 172.21.15.19:0/1856770959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4bc0039b0 0x7ff4bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.903+0000 7ff4d98df640 1 -- 172.21.15.19:0/1856770959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc003520 con 0x7ff4bc0051c0 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.904+0000 7ff4d98df640 1 -- 172.21.15.19:0/1856770959 reap_dead start 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.904+0000 7ff4d98df640 1 --2- 172.21.15.19:0/1856770959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4bc0051c0 0x7ff4bc00b9c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40073e0 tx=0x7ff4bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655719325d138ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.904+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4009c80 con 0x7ff4bc0051c0 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.904+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c402c6c0 con 0x7ff4bc0051c0 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.904+0000 7ff4d3fff640 1 -- 172.21.15.19:0/1856770959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c402cab0 con 0x7ff4bc0051c0 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.905+0000 7ff4db342640 1 -- 172.21.15.19:0/1856770959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4bc0051c0 msgr2=0x7ff4bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.905+0000 7ff4db342640 1 --2- 172.21.15.19:0/1856770959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4bc0051c0 0x7ff4bc00b9c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40073e0 tx=0x7ff4bc005830 comp rx=0 tx=0).stop 2024-04-25T09:53:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.905+0000 7ff4db342640 1 -- 172.21.15.19:0/1856770959 shutdown_connections 2024-04-25T09:53:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.905+0000 7ff4db342640 1 --2- 172.21.15.19:0/1856770959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4bc0051c0 0x7ff4bc00b9c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.905+0000 7ff4db342640 1 -- 172.21.15.19:0/1856770959 >> 172.21.15.19:0/1856770959 conn(0x7ff4d4061a20 msgr2=0x7ff4d4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.905+0000 7ff4db342640 1 -- 172.21.15.19:0/1856770959 shutdown_connections 2024-04-25T09:53:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.905+0000 7ff4db342640 1 -- 172.21.15.19:0/1856770959 wait complete. 2024-04-25T09:53:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.906+0000 7ff4db342640 1 Processor -- start 2024-04-25T09:53:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.906+0000 7ff4db342640 1 -- start start 2024-04-25T09:53:03.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4db342640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4bc0051c0 0x7ff4d414af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4db342640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4d415f350 0x7ff4d40ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4db342640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4d415f7d0 0x7ff4d40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4db342640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d40b92a0 con 0x7ff4d415f7d0 2024-04-25T09:53:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4db342640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d40d1a00 con 0x7ff4d415f350 2024-04-25T09:53:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4db342640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d40b9420 con 0x7ff4bc0051c0 2024-04-25T09:53:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d98df640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4d415f7d0 0x7ff4d40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d98df640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4d415f7d0 0x7ff4d40cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41560/0 (socket says 172.21.15.19:41560) 2024-04-25T09:53:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d98df640 1 -- 172.21.15.19:0/346867165 learned_addr learned my addr 172.21.15.19:0/346867165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d98df640 1 -- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4bc0051c0 msgr2=0x7ff4d414af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d98df640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4bc0051c0 0x7ff4d414af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d98df640 1 -- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4d415f350 msgr2=0x7ff4d40ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d98df640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4d415f350 0x7ff4d40ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d98df640 1 -- 172.21.15.19:0/346867165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d415fe80 con 0x7ff4d415f7d0 2024-04-25T09:53:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d98df640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4d415f7d0 0x7ff4d40cd160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff4c402b160 tx=0x7ff4c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d1ffb640 1 -- 172.21.15.19:0/346867165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4009c80 con 0x7ff4d415f7d0 2024-04-25T09:53:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d1ffb640 1 -- 172.21.15.19:0/346867165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c402da20 con 0x7ff4d415f7d0 2024-04-25T09:53:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4d1ffb640 1 -- 172.21.15.19:0/346867165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c402de10 con 0x7ff4d415f7d0 2024-04-25T09:53:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4db342640 1 -- 172.21.15.19:0/346867165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d4160120 con 0x7ff4d415f7d0 2024-04-25T09:53:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.907+0000 7ff4db342640 1 -- 172.21.15.19:0/346867165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d41606e0 con 0x7ff4d415f7d0 2024-04-25T09:53:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.908+0000 7ff4db342640 1 -- 172.21.15.19:0/346867165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d415fbd0 con 0x7ff4d415f7d0 2024-04-25T09:53:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.909+0000 7ff4d1ffb640 1 -- 172.21.15.19:0/346867165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4c4036400 con 0x7ff4d415f7d0 2024-04-25T09:53:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.909+0000 7ff4d1ffb640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c006b4d0 0x7ff4c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.909+0000 7ff4d1ffb640 1 -- 172.21.15.19:0/346867165 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff4c40aa550 con 0x7ff4d415f7d0 2024-04-25T09:53:03.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.909+0000 7ff4d90de640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c006b4d0 0x7ff4c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:03.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.910+0000 7ff4d90de640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c006b4d0 0x7ff4c006d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc001b80 tx=0x7ff4cc001af0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:03.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:03.913+0000 7ff4d1ffb640 1 -- 172.21.15.19:0/346867165 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4c4032030 con 0x7ff4d415f7d0 2024-04-25T09:53:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.093+0000 7ff4db342640 1 -- 172.21.15.19:0/346867165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4d4073990 con 0x7ff4d415f7d0 2024-04-25T09:53:04.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.094+0000 7ff4d1ffb640 1 -- 172.21.15.19:0/346867165 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7ff4c40790d0 con 0x7ff4d415f7d0 2024-04-25T09:53:04.096 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:04.096 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":16,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:52:56.957996+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 -- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c006b4d0 msgr2=0x7ff4c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c006b4d0 0x7ff4c006d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc001b80 tx=0x7ff4cc001af0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 -- 172.21.15.19:0/346867165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4d415f7d0 msgr2=0x7ff4d40cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4d415f7d0 0x7ff4d40cd160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff4c402b160 tx=0x7ff4c4004ab0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 -- 172.21.15.19:0/346867165 shutdown_connections 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c006b4d0 0x7ff4c006d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4d415f7d0 0x7ff4d40cd160 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4d415f350 0x7ff4d40ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 --2- 172.21.15.19:0/346867165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4bc0051c0 0x7ff4d414af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 -- 172.21.15.19:0/346867165 >> 172.21.15.19:0/346867165 conn(0x7ff4d4061a20 msgr2=0x7ff4d4074060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 -- 172.21.15.19:0/346867165 shutdown_connections 2024-04-25T09:53:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.096+0000 7ff4db342640 1 -- 172.21.15.19:0/346867165 wait complete. 2024-04-25T09:53:04.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:53:04.109 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:04.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.211+0000 7fb5f7df9640 1 Processor -- start 2024-04-25T09:53:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.211+0000 7fb5f7df9640 1 -- start start 2024-04-25T09:53:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f7df9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f00ced80 0x7fb5f00d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f7df9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb5f0161710 0x7fb5f00d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f7df9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f015e000 con 0x7fb5f015dc20 2024-04-25T09:53:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f7df9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f00d1de0 con 0x7fb5f0161710 2024-04-25T09:53:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f7df9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f0161c60 con 0x7fb5f00ced80 2024-04-25T09:53:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f6396640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb5f0161710 0x7fb5f00d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f5b95640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb5f015dc20 0x7fb5f00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:55266/0 (socket says 172.21.15.19:55266) 2024-04-25T09:53:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f5394640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f00ced80 0x7fb5f00d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f5b95640 1 -- 172.21.15.19:0/432977607 learned_addr learned my addr 172.21.15.19:0/432977607 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f4b93640 1 -- 172.21.15.19:0/432977607 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660984278 0 0) 0x7fb5f015e000 con 0x7fb5f015dc20 2024-04-25T09:53:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f4b93640 1 -- 172.21.15.19:0/432977607 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5dc003520 con 0x7fb5f015dc20 2024-04-25T09:53:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f6396640 1 -- 172.21.15.19:0/432977607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f00ced80 msgr2=0x7fb5f00d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f6396640 1 --2- 172.21.15.19:0/432977607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f00ced80 0x7fb5f00d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f6396640 1 -- 172.21.15.19:0/432977607 >> v1:172.21.15.112:6789/0 conn(0x7fb5f015dc20 legacy=0x7fb5f00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f6396640 1 -- 172.21.15.19:0/432977607 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f0161f40 con 0x7fb5f0161710 2024-04-25T09:53:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.212+0000 7fb5f5394640 1 --2- 172.21.15.19:0/432977607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f00ced80 0x7fb5f00d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:04.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.213+0000 7fb5f6396640 1 --2- 172.21.15.19:0/432977607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f0161710 0x7fb5f00d2820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009d90 tx=0x7fb5e802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c54d46cb9393f432 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.213+0000 7fb5f4b93640 1 -- 172.21.15.19:0/432977607 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8007740 con 0x7fb5f0161710 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.213+0000 7fb5f4b93640 1 -- 172.21.15.19:0/432977607 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e80078e0 con 0x7fb5f0161710 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.213+0000 7fb5f4b93640 1 -- 172.21.15.19:0/432977607 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8007cb0 con 0x7fb5f0161710 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.214+0000 7fb5f7df9640 1 -- 172.21.15.19:0/432977607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f0161710 msgr2=0x7fb5f00d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.214+0000 7fb5f7df9640 1 --2- 172.21.15.19:0/432977607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f0161710 0x7fb5f00d2820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009d90 tx=0x7fb5e802a660 comp rx=0 tx=0).stop 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.214+0000 7fb5f7df9640 1 -- 172.21.15.19:0/432977607 shutdown_connections 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.214+0000 7fb5f7df9640 1 --2- 172.21.15.19:0/432977607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f0161710 0x7fb5f00d2820 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.214+0000 7fb5f7df9640 1 --2- 172.21.15.19:0/432977607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f00ced80 0x7fb5f00d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.214+0000 7fb5f7df9640 1 -- 172.21.15.19:0/432977607 >> 172.21.15.19:0/432977607 conn(0x7fb5f00d0420 msgr2=0x7fb5f00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.214+0000 7fb5f7df9640 1 -- 172.21.15.19:0/432977607 shutdown_connections 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.214+0000 7fb5f7df9640 1 -- 172.21.15.19:0/432977607 wait complete. 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.215+0000 7fb5f7df9640 1 Processor -- start 2024-04-25T09:53:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.215+0000 7fb5f7df9640 1 -- start start 2024-04-25T09:53:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f7df9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f00ced80 0x7fb5f00c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f7df9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f015dc20 0x7fb5f0146a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f7df9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5f0161710 0x7fb5f0157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f7df9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00b5e50 con 0x7fb5f0161710 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f7df9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f0162500 con 0x7fb5f00ced80 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f7df9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00b5fd0 con 0x7fb5f015dc20 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f6396640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5f0161710 0x7fb5f0157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f5394640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f015dc20 0x7fb5f0146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f6396640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5f0161710 0x7fb5f0157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41568/0 (socket says 172.21.15.19:41568) 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f6396640 1 -- 172.21.15.19:0/3172235846 learned_addr learned my addr 172.21.15.19:0/3172235846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f5b95640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f00ced80 0x7fb5f00c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f6396640 1 -- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f015dc20 msgr2=0x7fb5f0146a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f6396640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f015dc20 0x7fb5f0146a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f6396640 1 -- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f00ced80 msgr2=0x7fb5f00c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f6396640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f00ced80 0x7fb5f00c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f6396640 1 -- 172.21.15.19:0/3172235846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f015aa40 con 0x7fb5f0161710 2024-04-25T09:53:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.216+0000 7fb5f6396640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5f0161710 0x7fb5f0157c10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009eb0 tx=0x7fb5e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.217+0000 7fb5e67fc640 1 -- 172.21.15.19:0/3172235846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8040070 con 0x7fb5f0161710 2024-04-25T09:53:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.217+0000 7fb5e67fc640 1 -- 172.21.15.19:0/3172235846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e8031bc0 con 0x7fb5f0161710 2024-04-25T09:53:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.217+0000 7fb5f7df9640 1 -- 172.21.15.19:0/3172235846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f015a760 con 0x7fb5f0161710 2024-04-25T09:53:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.217+0000 7fb5e67fc640 1 -- 172.21.15.19:0/3172235846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8009450 con 0x7fb5f0161710 2024-04-25T09:53:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.217+0000 7fb5f7df9640 1 -- 172.21.15.19:0/3172235846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f015f5f0 con 0x7fb5f0161710 2024-04-25T09:53:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.218+0000 7fb5f7df9640 1 -- 172.21.15.19:0/3172235846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f0073a00 con 0x7fb5f0161710 2024-04-25T09:53:04.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.218+0000 7fb5e67fc640 1 -- 172.21.15.19:0/3172235846 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5e8039040 con 0x7fb5f0161710 2024-04-25T09:53:04.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.219+0000 7fb5e67fc640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb5d406b4d0 0x7fb5d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.219+0000 7fb5e67fc640 1 -- 172.21.15.19:0/3172235846 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb5e80ab460 con 0x7fb5f0161710 2024-04-25T09:53:04.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.219+0000 7fb5f5b95640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb5d406b4d0 0x7fb5d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.219+0000 7fb5f5b95640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb5d406b4d0 0x7fb5d406d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec005880 tx=0x7fb5ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:04.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.223+0000 7fb5e67fc640 1 -- 172.21.15.19:0/3172235846 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5e807a060 con 0x7fb5f0161710 2024-04-25T09:53:04.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.401+0000 7fb5f7df9640 1 -- 172.21.15.19:0/3172235846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5f00b7ff0 con 0x7fb5f0161710 2024-04-25T09:53:04.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.402+0000 7fb5e67fc640 1 -- 172.21.15.19:0/3172235846 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5e807e100 con 0x7fb5f0161710 2024-04-25T09:53:04.404 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 -- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb5d406b4d0 msgr2=0x7fb5d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb5d406b4d0 0x7fb5d406d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec005880 tx=0x7fb5ec009040 comp rx=0 tx=0).stop 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 -- 172.21.15.19:0/3172235846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5f0161710 msgr2=0x7fb5f0157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5f0161710 0x7fb5f0157c10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009eb0 tx=0x7fb5e8004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 -- 172.21.15.19:0/3172235846 shutdown_connections 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb5d406b4d0 0x7fb5d406d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5f0161710 0x7fb5f0157c10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f015dc20 0x7fb5f0146a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 --2- 172.21.15.19:0/3172235846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f00ced80 0x7fb5f00c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 -- 172.21.15.19:0/3172235846 >> 172.21.15.19:0/3172235846 conn(0x7fb5f00d0420 msgr2=0x7fb5f00d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 -- 172.21.15.19:0/3172235846 shutdown_connections 2024-04-25T09:53:04.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.404+0000 7fb5f7df9640 1 -- 172.21.15.19:0/3172235846 wait complete. 2024-04-25T09:53:04.416 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:53:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.515+0000 7fbfed561640 1 Processor -- start 2024-04-25T09:53:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.515+0000 7fbfed561640 1 -- start start 2024-04-25T09:53:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.515+0000 7fbfed561640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbfe80b2cb0 0x7fbfe80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.515+0000 7fbfed561640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfe80d1020 0x7fbfe80d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.515+0000 7fbfed561640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe80b97a0 con 0x7fbfe815dbe0 2024-04-25T09:53:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.515+0000 7fbfed561640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe815dfc0 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.515+0000 7fbfed561640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe80d1a00 con 0x7fbfe80d1020 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.516+0000 7fbfdffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbfe80b2cb0 0x7fbfe80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.516+0000 7fbfe6ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbfe815dbe0 0x7fbfe80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50344/0 (socket says 172.21.15.19:50344) 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.516+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/2548974496 learned_addr learned my addr 172.21.15.19:0/2548974496 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.516+0000 7fbfe77fe640 1 --2- 172.21.15.19:0/2548974496 >> v2:172.21.15.112:3300/0 conn(0x7fbfe80d1020 0x7fbfe80d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.516+0000 7fbfdffff640 1 -- 172.21.15.19:0/2548974496 >> v2:172.21.15.112:3300/0 conn(0x7fbfe80d1020 msgr2=0x7fbfe80d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.516+0000 7fbfdffff640 1 --2- 172.21.15.19:0/2548974496 >> v2:172.21.15.112:3300/0 conn(0x7fbfe80d1020 0x7fbfe80d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.516+0000 7fbfdffff640 1 -- 172.21.15.19:0/2548974496 >> v1:172.21.15.19:6789/0 conn(0x7fbfe815dbe0 legacy=0x7fbfe80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.516+0000 7fbfdffff640 1 -- 172.21.15.19:0/2548974496 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80d1c50 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.516+0000 7fbfe67fc640 1 -- 172.21.15.19:0/2548974496 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771532086 0 0) 0x7fbfe80b97a0 con 0x7fbfe815dbe0 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.516+0000 7fbfe77fe640 1 --2- 172.21.15.19:0/2548974496 >> v2:172.21.15.112:3300/0 conn(0x7fbfe80d1020 0x7fbfe80d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfdffff640 1 --2- 172.21.15.19:0/2548974496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe80b2cb0 0x7fbfe80d2730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80cf570 tx=0x7fbfd002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2fb5d84fdc2ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfe67fc640 1 -- 172.21.15.19:0/2548974496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0037070 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfe67fc640 1 -- 172.21.15.19:0/2548974496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd0032440 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfe67fc640 1 -- 172.21.15.19:0/2548974496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0032810 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfed561640 1 -- 172.21.15.19:0/2548974496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe80b2cb0 msgr2=0x7fbfe80d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfed561640 1 --2- 172.21.15.19:0/2548974496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe80b2cb0 0x7fbfe80d2730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80cf570 tx=0x7fbfd002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfed561640 1 -- 172.21.15.19:0/2548974496 shutdown_connections 2024-04-25T09:53:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfed561640 1 --2- 172.21.15.19:0/2548974496 >> v2:172.21.15.112:3300/0 conn(0x7fbfe80d1020 0x7fbfe80d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfed561640 1 --2- 172.21.15.19:0/2548974496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe80b2cb0 0x7fbfe80d2730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfed561640 1 -- 172.21.15.19:0/2548974496 >> 172.21.15.19:0/2548974496 conn(0x7fbfe8061a20 msgr2=0x7fbfe8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfed561640 1 -- 172.21.15.19:0/2548974496 shutdown_connections 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.517+0000 7fbfed561640 1 -- 172.21.15.19:0/2548974496 wait complete. 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.518+0000 7fbfed561640 1 Processor -- start 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.519+0000 7fbfed561640 1 -- start start 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.519+0000 7fbfed561640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe80b2cb0 0x7fbfe81f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.519+0000 7fbfed561640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe80d1020 0x7fbfe81f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.519+0000 7fbfed561640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfe815dbe0 0x7fbfe80b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.519+0000 7fbfed561640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80b92a0 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.519+0000 7fbfed561640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80d2210 con 0x7fbfe80d1020 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.519+0000 7fbfed561640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80b9420 con 0x7fbfe815dbe0 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.519+0000 7fbfe6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe80b2cb0 0x7fbfe81f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.519+0000 7fbfe6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe80b2cb0 0x7fbfe81f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41574/0 (socket says 172.21.15.19:41574) 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.519+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/3733460301 learned_addr learned my addr 172.21.15.19:0/3733460301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.519+0000 7fbfdffff640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe80d1020 0x7fbfe81f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfe77fe640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfe815dbe0 0x7fbfe80b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfe815dbe0 msgr2=0x7fbfe80b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfe6ffd640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfe815dbe0 0x7fbfe80b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe80d1020 msgr2=0x7fbfe81f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfe6ffd640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe80d1020 0x7fbfe81f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/3733460301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80b37b0 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfe6ffd640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe80b2cb0 0x7fbfe81f7170 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c430 tx=0x7fbfd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3733460301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80074e0 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3733460301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd80047e0 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfed561640 1 -- 172.21.15.19:0/3733460301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe81f3930 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3733460301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8004bd0 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.520+0000 7fbfed561640 1 -- 172.21.15.19:0/3733460301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe81f3ef0 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.521+0000 7fbfed561640 1 -- 172.21.15.19:0/3733460301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe8073990 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.521+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3733460301 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfd8011070 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.522+0000 7fbfdf7fe640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfc806b4d0 0x7fbfc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.522+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3733460301 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbfd8089a40 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.522+0000 7fbfdffff640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfc806b4d0 0x7fbfc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.523+0000 7fbfdffff640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfc806b4d0 0x7fbfc806d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0034000 tx=0x7fbfd002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.526+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3733460301 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfd8058ca0 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.712+0000 7fbfed561640 1 -- 172.21.15.19:0/3733460301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbfe80d27d0 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.712+0000 7fbfdf7fe640 1 -- 172.21.15.19:0/3733460301 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbfd80585a0 con 0x7fbfe80b2cb0 2024-04-25T09:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T09:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T09:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T09:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:53:04.714 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T09:53:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 -- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfc806b4d0 msgr2=0x7fbfc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfc806b4d0 0x7fbfc806d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0034000 tx=0x7fbfd002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 -- 172.21.15.19:0/3733460301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe80b2cb0 msgr2=0x7fbfe81f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe80b2cb0 0x7fbfe81f7170 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c430 tx=0x7fbfd800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 -- 172.21.15.19:0/3733460301 shutdown_connections 2024-04-25T09:53:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfc806b4d0 0x7fbfc806d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfe815dbe0 0x7fbfe80b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe80d1020 0x7fbfe81f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 --2- 172.21.15.19:0/3733460301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfe80b2cb0 0x7fbfe81f7170 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 -- 172.21.15.19:0/3733460301 >> 172.21.15.19:0/3733460301 conn(0x7fbfe8061a20 msgr2=0x7fbfe8061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 -- 172.21.15.19:0/3733460301 shutdown_connections 2024-04-25T09:53:04.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.714+0000 7fbfed561640 1 -- 172.21.15.19:0/3733460301 wait complete. 2024-04-25T09:53:04.726 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:53:04.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:53:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.830+0000 7f027e15c640 1 Processor -- start 2024-04-25T09:53:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.830+0000 7f027e15c640 1 -- start start 2024-04-25T09:53:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f027e15c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f02780ced80 0x7f02780d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f027e15c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0278161710 0x7f02780d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f027e15c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027815e000 con 0x7f027815dc20 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f027e15c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02780d1de0 con 0x7f02780ced80 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f027e15c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0278161c60 con 0x7f0278161710 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f0276ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f02780ced80 0x7f02780d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f02777fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f027815dc20 0x7f02780ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50354/0 (socket says 172.21.15.19:50354) 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f02777fe640 1 -- 172.21.15.19:0/2879529687 learned_addr learned my addr 172.21.15.19:0/2879529687 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f0276ffd640 1 --2- 172.21.15.19:0/2879529687 >> v2:172.21.15.19:3300/0 conn(0x7f02780ced80 0x7f02780d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41580/0 (socket says 172.21.15.19:41580) 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f0277fff640 1 --2- 172.21.15.19:0/2879529687 >> v2:172.21.15.112:3300/0 conn(0x7f0278161710 0x7f02780d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f0276ffd640 1 -- 172.21.15.19:0/2879529687 >> v2:172.21.15.112:3300/0 conn(0x7f0278161710 msgr2=0x7f02780d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f0276ffd640 1 --2- 172.21.15.19:0/2879529687 >> v2:172.21.15.112:3300/0 conn(0x7f0278161710 0x7f02780d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f0276ffd640 1 -- 172.21.15.19:0/2879529687 >> v1:172.21.15.19:6789/0 conn(0x7f027815dc20 legacy=0x7f02780ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f0276ffd640 1 -- 172.21.15.19:0/2879529687 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0278161f40 con 0x7f02780ced80 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.831+0000 7f02767fc640 1 -- 172.21.15.19:0/2879529687 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645186764 0 0) 0x7f027815e000 con 0x7f027815dc20 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.832+0000 7f0276ffd640 1 --2- 172.21.15.19:0/2879529687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02780ced80 0x7f02780d22e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f02780b6570 tx=0x7f026802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340371274be6a492 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.832+0000 7f02767fc640 1 -- 172.21.15.19:0/2879529687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268037070 con 0x7f02780ced80 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.832+0000 7f02767fc640 1 -- 172.21.15.19:0/2879529687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0268032440 con 0x7f02780ced80 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.832+0000 7f02767fc640 1 -- 172.21.15.19:0/2879529687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268032810 con 0x7f02780ced80 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.832+0000 7f027e15c640 1 -- 172.21.15.19:0/2879529687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02780ced80 msgr2=0x7f02780d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.832+0000 7f027e15c640 1 --2- 172.21.15.19:0/2879529687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02780ced80 0x7f02780d22e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f02780b6570 tx=0x7f026802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.832+0000 7f027e15c640 1 -- 172.21.15.19:0/2879529687 shutdown_connections 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.832+0000 7f027e15c640 1 --2- 172.21.15.19:0/2879529687 >> v2:172.21.15.112:3300/0 conn(0x7f0278161710 0x7f02780d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.832+0000 7f027e15c640 1 --2- 172.21.15.19:0/2879529687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02780ced80 0x7f02780d22e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.832+0000 7f027e15c640 1 -- 172.21.15.19:0/2879529687 >> 172.21.15.19:0/2879529687 conn(0x7f02780d0420 msgr2=0x7f02780d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.833+0000 7f027e15c640 1 -- 172.21.15.19:0/2879529687 shutdown_connections 2024-04-25T09:53:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.833+0000 7f027e15c640 1 -- 172.21.15.19:0/2879529687 wait complete. 2024-04-25T09:53:04.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.834+0000 7f027e15c640 1 Processor -- start 2024-04-25T09:53:04.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.834+0000 7f027e15c640 1 -- start start 2024-04-25T09:53:04.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.834+0000 7f027e15c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02780ced80 0x7f0278152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.834+0000 7f027e15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027815dc20 0x7f02781e7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f027e15c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0278161710 0x7f02781e7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f027e15c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02780b5e50 con 0x7f027815dc20 2024-04-25T09:53:04.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f027e15c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0278162500 con 0x7f02780ced80 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f027e15c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02780b5fd0 con 0x7f0278161710 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f0276ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027815dc20 0x7f02781e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f0276ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027815dc20 0x7f02781e7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41586/0 (socket says 172.21.15.19:41586) 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f0276ffd640 1 -- 172.21.15.19:0/854289970 learned_addr learned my addr 172.21.15.19:0/854289970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f0276ffd640 1 -- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0278161710 msgr2=0x7f02781e7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f0276ffd640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0278161710 0x7f02781e7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f0276ffd640 1 -- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02780ced80 msgr2=0x7f0278152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f0276ffd640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02780ced80 0x7f0278152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f0276ffd640 1 -- 172.21.15.19:0/854289970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027815f570 con 0x7f027815dc20 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f0276ffd640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027815dc20 0x7f02781e7510 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0268009890 tx=0x7f02680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f025ffff640 1 -- 172.21.15.19:0/854289970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268037040 con 0x7f027815dc20 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f025ffff640 1 -- 172.21.15.19:0/854289970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0268042070 con 0x7f027815dc20 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f027e15c640 1 -- 172.21.15.19:0/854289970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02780c7db0 con 0x7f027815dc20 2024-04-25T09:53:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f025ffff640 1 -- 172.21.15.19:0/854289970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268030bd0 con 0x7f027815dc20 2024-04-25T09:53:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.835+0000 7f027e15c640 1 -- 172.21.15.19:0/854289970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02780c8330 con 0x7f027815dc20 2024-04-25T09:53:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.836+0000 7f027e15c640 1 -- 172.21.15.19:0/854289970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0278073a00 con 0x7f027815dc20 2024-04-25T09:53:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.837+0000 7f025ffff640 1 -- 172.21.15.19:0/854289970 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0268047460 con 0x7f027815dc20 2024-04-25T09:53:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.837+0000 7f025ffff640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f026406b4d0 0x7f026406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.837+0000 7f025ffff640 1 -- 172.21.15.19:0/854289970 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f02680aaae0 con 0x7f027815dc20 2024-04-25T09:53:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.838+0000 7f02777fe640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f026406b4d0 0x7f026406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.838+0000 7f02777fe640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f026406b4d0 0x7f026406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f026c007fd0 tx=0x7f026c007960 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:04.841+0000 7f025ffff640 1 -- 172.21.15.19:0/854289970 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f027815f570 con 0x7f027815dc20 2024-04-25T09:53:05.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.022+0000 7f027e15c640 1 -- 172.21.15.19:0/854289970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f02780d16b0 con 0x7f027815dc20 2024-04-25T09:53:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.756+0000 7f025ffff640 1 -- 172.21.15.19:0/854289970 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0268032e20 con 0x7f027815dc20 2024-04-25T09:53:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T09:53:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.759+0000 7f027e15c640 1 -- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f026406b4d0 msgr2=0x7f026406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.759+0000 7f027e15c640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f026406b4d0 0x7f026406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f026c007fd0 tx=0x7f026c007960 comp rx=0 tx=0).stop 2024-04-25T09:53:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.759+0000 7f027e15c640 1 -- 172.21.15.19:0/854289970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027815dc20 msgr2=0x7f02781e7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.759+0000 7f027e15c640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027815dc20 0x7f02781e7510 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0268009890 tx=0x7f02680046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.760+0000 7f027e15c640 1 -- 172.21.15.19:0/854289970 shutdown_connections 2024-04-25T09:53:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.760+0000 7f027e15c640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f026406b4d0 0x7f026406d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.760+0000 7f027e15c640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0278161710 0x7f02781e7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.760+0000 7f027e15c640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f027815dc20 0x7f02781e7510 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.760+0000 7f027e15c640 1 --2- 172.21.15.19:0/854289970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02780ced80 0x7f0278152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.760+0000 7f027e15c640 1 -- 172.21.15.19:0/854289970 >> 172.21.15.19:0/854289970 conn(0x7f02780d0420 msgr2=0x7f02780b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:05.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.761+0000 7f027e15c640 1 -- 172.21.15.19:0/854289970 shutdown_connections 2024-04-25T09:53:05.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.761+0000 7f027e15c640 1 -- 172.21.15.19:0/854289970 wait complete. 2024-04-25T09:53:05.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T09:53:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.890+0000 7fad89a4c640 1 Processor -- start 2024-04-25T09:53:05.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.890+0000 7fad89a4c640 1 -- start start 2024-04-25T09:53:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.891+0000 7fad89a4c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8406c300 0x7fad840c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.891+0000 7fad89a4c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad840cc2e0 con 0x7fad840ca5a0 2024-04-25T09:53:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.891+0000 7fad89a4c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad840cc5c0 con 0x7fad8415aa10 2024-04-25T09:53:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.891+0000 7fad89a4c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840cfa70 con 0x7fad8406c300 2024-04-25T09:53:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.892+0000 7fad827fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fad840ca5a0 0x7fad840acf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50360/0 (socket says 172.21.15.19:50360) 2024-04-25T09:53:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.892+0000 7fad827fc640 1 -- 172.21.15.19:0/3729057037 learned_addr learned my addr 172.21.15.19:0/3729057037 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.892+0000 7fad837fe640 1 --2- 172.21.15.19:0/3729057037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8406c300 0x7fad840c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.892+0000 7fad81ffb640 1 -- 172.21.15.19:0/3729057037 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315070874 0 0) 0x7fad840cc2e0 con 0x7fad840ca5a0 2024-04-25T09:53:05.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.892+0000 7fad81ffb640 1 -- 172.21.15.19:0/3729057037 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad70003520 con 0x7fad840ca5a0 2024-04-25T09:53:05.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.892+0000 7fad81ffb640 1 -- 172.21.15.19:0/3729057037 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673508287 0 0) 0x7fad840cc5c0 con 0x7fad8415aa10 2024-04-25T09:53:05.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.892+0000 7fad81ffb640 1 -- 172.21.15.19:0/3729057037 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad840cc2e0 con 0x7fad8415aa10 2024-04-25T09:53:05.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.892+0000 7fad837fe640 1 -- 172.21.15.19:0/3729057037 >> v1:172.21.15.112:6789/0 conn(0x7fad8415aa10 legacy=0x7fad8406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:05.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.892+0000 7fad837fe640 1 -- 172.21.15.19:0/3729057037 >> v1:172.21.15.19:6789/0 conn(0x7fad840ca5a0 legacy=0x7fad840acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:05.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.892+0000 7fad837fe640 1 -- 172.21.15.19:0/3729057037 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840cf420 con 0x7fad8406c300 2024-04-25T09:53:05.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.893+0000 7fad837fe640 1 --2- 172.21.15.19:0/3729057037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8406c300 0x7fad840c6830 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fad74009dc0 tx=0x7fad7402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6daf81ac843ff80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:05.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.893+0000 7fad81ffb640 1 -- 172.21.15.19:0/3729057037 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7402ae60 con 0x7fad8406c300 2024-04-25T09:53:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.893+0000 7fad81ffb640 1 -- 172.21.15.19:0/3729057037 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad74007740 con 0x7fad8406c300 2024-04-25T09:53:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.893+0000 7fad81ffb640 1 -- 172.21.15.19:0/3729057037 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74007b10 con 0x7fad8406c300 2024-04-25T09:53:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.894+0000 7fad89a4c640 1 -- 172.21.15.19:0/3729057037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8406c300 msgr2=0x7fad840c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.894+0000 7fad89a4c640 1 --2- 172.21.15.19:0/3729057037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8406c300 0x7fad840c6830 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fad74009dc0 tx=0x7fad7402a640 comp rx=0 tx=0).stop 2024-04-25T09:53:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.894+0000 7fad89a4c640 1 -- 172.21.15.19:0/3729057037 shutdown_connections 2024-04-25T09:53:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.894+0000 7fad89a4c640 1 --2- 172.21.15.19:0/3729057037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8406c300 0x7fad840c6830 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.894+0000 7fad89a4c640 1 -- 172.21.15.19:0/3729057037 >> 172.21.15.19:0/3729057037 conn(0x7fad84071af0 msgr2=0x7fad84071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.894+0000 7fad89a4c640 1 -- 172.21.15.19:0/3729057037 shutdown_connections 2024-04-25T09:53:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.894+0000 7fad89a4c640 1 -- 172.21.15.19:0/3729057037 wait complete. 2024-04-25T09:53:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.895+0000 7fad89a4c640 1 Processor -- start 2024-04-25T09:53:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.895+0000 7fad89a4c640 1 -- start start 2024-04-25T09:53:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad89a4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad8406c300 0x7fad840c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad89a4c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad840ca5a0 0x7fad84148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad89a4c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8415aa10 0x7fad84154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad89a4c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad840cbff0 con 0x7fad840ca5a0 2024-04-25T09:53:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad89a4c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad840cbe70 con 0x7fad8406c300 2024-04-25T09:53:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad89a4c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840ad640 con 0x7fad8415aa10 2024-04-25T09:53:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad827fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad840ca5a0 0x7fad84148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad827fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad840ca5a0 0x7fad84148980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41600/0 (socket says 172.21.15.19:41600) 2024-04-25T09:53:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad827fc640 1 -- 172.21.15.19:0/1415674870 learned_addr learned my addr 172.21.15.19:0/1415674870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad82ffd640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad8406c300 0x7fad840c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad837fe640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8415aa10 0x7fad84154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad827fc640 1 -- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8415aa10 msgr2=0x7fad84154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad827fc640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8415aa10 0x7fad84154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad827fc640 1 -- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad8406c300 msgr2=0x7fad840c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad827fc640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad8406c300 0x7fad840c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad827fc640 1 -- 172.21.15.19:0/1415674870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840c7d70 con 0x7fad840ca5a0 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad837fe640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8415aa10 0x7fad84154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad82ffd640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad8406c300 0x7fad840c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad827fc640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad840ca5a0 0x7fad84148980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c500 tx=0x7fad6c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad7bfff640 1 -- 172.21.15.19:0/1415674870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c017070 con 0x7fad840ca5a0 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.896+0000 7fad7bfff640 1 -- 172.21.15.19:0/1415674870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c005ce0 con 0x7fad840ca5a0 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.897+0000 7fad89a4c640 1 -- 172.21.15.19:0/1415674870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad840c8a30 con 0x7fad840ca5a0 2024-04-25T09:53:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.897+0000 7fad7bfff640 1 -- 172.21.15.19:0/1415674870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0072f0 con 0x7fad840ca5a0 2024-04-25T09:53:05.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.897+0000 7fad89a4c640 1 -- 172.21.15.19:0/1415674870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad840c8ff0 con 0x7fad840ca5a0 2024-04-25T09:53:05.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.897+0000 7fad89a4c640 1 -- 172.21.15.19:0/1415674870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad840c91f0 con 0x7fad840ca5a0 2024-04-25T09:53:05.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.898+0000 7fad7bfff640 1 -- 172.21.15.19:0/1415674870 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad6c01a070 con 0x7fad840ca5a0 2024-04-25T09:53:05.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.898+0000 7fad7bfff640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad6406b4d0 0x7fad6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:05.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.899+0000 7fad7bfff640 1 -- 172.21.15.19:0/1415674870 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fad6c089320 con 0x7fad840ca5a0 2024-04-25T09:53:05.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.899+0000 7fad82ffd640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad6406b4d0 0x7fad6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:05.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.899+0000 7fad82ffd640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad6406b4d0 0x7fad6406d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0011e0 tx=0x7fad7c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:05.903+0000 7fad7bfff640 1 -- 172.21.15.19:0/1415674870 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad840c91f0 con 0x7fad840ca5a0 2024-04-25T09:53:06.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.084+0000 7fad89a4c640 1 -- 172.21.15.19:0/1415674870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fad84073a00 con 0x7fad840ca5a0 2024-04-25T09:53:06.681 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:06.684 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.758+0000 7fad7bfff640 1 -- 172.21.15.19:0/1415674870 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fad6c057dc0 con 0x7fad840ca5a0 2024-04-25T09:53:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T09:53:06.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.761+0000 7fad89a4c640 1 -- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad6406b4d0 msgr2=0x7fad6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:06.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.761+0000 7fad89a4c640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad6406b4d0 0x7fad6406d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0011e0 tx=0x7fad7c009040 comp rx=0 tx=0).stop 2024-04-25T09:53:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.761+0000 7fad89a4c640 1 -- 172.21.15.19:0/1415674870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad840ca5a0 msgr2=0x7fad84148980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.761+0000 7fad89a4c640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad840ca5a0 0x7fad84148980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c500 tx=0x7fad6c00c9c0 comp rx=0 tx=0).stop 2024-04-25T09:53:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.761+0000 7fad89a4c640 1 -- 172.21.15.19:0/1415674870 shutdown_connections 2024-04-25T09:53:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.762+0000 7fad89a4c640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad6406b4d0 0x7fad6406d990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.762+0000 7fad89a4c640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad8415aa10 0x7fad84154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.762+0000 7fad89a4c640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad840ca5a0 0x7fad84148980 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.762+0000 7fad89a4c640 1 --2- 172.21.15.19:0/1415674870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad8406c300 0x7fad840c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.762+0000 7fad89a4c640 1 -- 172.21.15.19:0/1415674870 >> 172.21.15.19:0/1415674870 conn(0x7fad84071af0 msgr2=0x7fad8406db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.762+0000 7fad89a4c640 1 -- 172.21.15.19:0/1415674870 shutdown_connections 2024-04-25T09:53:06.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.763+0000 7fad89a4c640 1 -- 172.21.15.19:0/1415674870 wait complete. 2024-04-25T09:53:06.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:53:06.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.941+0000 7f5dff15c640 1 Processor -- start 2024-04-25T09:53:06.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.941+0000 7f5dff15c640 1 -- start start 2024-04-25T09:53:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.942+0000 7f5dff15c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df815bb50 0x7f5df80c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.942+0000 7f5dff15c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5df80cfd10 0x7f5df80cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.942+0000 7f5dff15c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df80c75e0 con 0x7f5df80c70d0 2024-04-25T09:53:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.942+0000 7f5dff15c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df80c8680 con 0x7f5df80cfd10 2024-04-25T09:53:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.942+0000 7f5dff15c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80c78c0 con 0x7f5df815bb50 2024-04-25T09:53:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.942+0000 7f5df7fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5df80c70d0 0x7f5df80cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50370/0 (socket says 172.21.15.19:50370) 2024-04-25T09:53:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.942+0000 7f5df7fff640 1 -- 172.21.15.19:0/344390222 learned_addr learned my addr 172.21.15.19:0/344390222 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5dfcef8640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df815bb50 0x7f5df80c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5dfd6f9640 1 --2- 172.21.15.19:0/344390222 >> v2:172.21.15.112:3300/0 conn(0x7f5df80cfd10 0x7f5df80cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822760979 0 0) 0x7f5df80c75e0 con 0x7f5df80c70d0 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de0003520 con 0x7f5df80c70d0 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2793389791 0 0) 0x7f5de0003520 con 0x7f5df80c70d0 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df815bb50 msgr2=0x7f5df80c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df815bb50 0x7f5df80c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 >> v2:172.21.15.112:3300/0 conn(0x7f5df80cfd10 msgr2=0x7f5df80cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 --2- 172.21.15.19:0/344390222 >> v2:172.21.15.112:3300/0 conn(0x7f5df80cfd10 0x7f5df80cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80c7ba0 con 0x7f5df80c70d0 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f5dec001670 con 0x7f5df80c70d0 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 >> v1:172.21.15.19:6789/0 conn(0x7f5df80c70d0 legacy=0x7f5df80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df815bb50 0x7f5de00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de0003f80 0x7f5de0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de0004d40 0x7f5de00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80c8680 con 0x7f5de0004d40 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80c78c0 con 0x7f5de0003f80 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de000ba40 con 0x7f5df815bb50 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5dfd6f9640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de0004d40 0x7f5de00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5dfcef8640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df815bb50 0x7f5de00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5df7fff640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de0003f80 0x7f5de0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5dfd6f9640 1 -- 172.21.15.19:0/344390222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df815bb50 msgr2=0x7f5de00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5dfd6f9640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df815bb50 0x7f5de00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5dfd6f9640 1 -- 172.21.15.19:0/344390222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de0003f80 msgr2=0x7f5de0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5dfd6f9640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de0003f80 0x7f5de0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.943+0000 7f5dfd6f9640 1 -- 172.21.15.19:0/344390222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de0003520 con 0x7f5de0004d40 2024-04-25T09:53:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.944+0000 7f5dfd6f9640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de0004d40 0x7f5de00051c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009f90 tx=0x7f5de0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde2467a2d7ce578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:06.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.944+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0009c10 con 0x7f5de0004d40 2024-04-25T09:53:06.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.944+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df0036070 con 0x7f5de0004d40 2024-04-25T09:53:06.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.944+0000 7f5df77fe640 1 -- 172.21.15.19:0/344390222 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0031840 con 0x7f5de0004d40 2024-04-25T09:53:06.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.945+0000 7f5dff15c640 1 -- 172.21.15.19:0/344390222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de0004d40 msgr2=0x7f5de00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:06.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.945+0000 7f5dff15c640 1 --2- 172.21.15.19:0/344390222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de0004d40 0x7f5de00051c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009f90 tx=0x7f5de0005830 comp rx=0 tx=0).stop 2024-04-25T09:53:06.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.945+0000 7f5dfd6f9640 1 -- 172.21.15.19:0/344390222 reap_dead start 2024-04-25T09:53:06.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.945+0000 7f5dff15c640 1 -- 172.21.15.19:0/344390222 shutdown_connections 2024-04-25T09:53:06.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.945+0000 7f5dff15c640 1 -- 172.21.15.19:0/344390222 >> 172.21.15.19:0/344390222 conn(0x7f5df806f500 msgr2=0x7f5df806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:06.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.945+0000 7f5dff15c640 1 -- 172.21.15.19:0/344390222 shutdown_connections 2024-04-25T09:53:06.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.945+0000 7f5dff15c640 1 -- 172.21.15.19:0/344390222 wait complete. 2024-04-25T09:53:06.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.946+0000 7f5dff15c640 1 Processor -- start 2024-04-25T09:53:06.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.946+0000 7f5dff15c640 1 -- start start 2024-04-25T09:53:06.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dff15c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df814c940 0x7f5df80afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:06.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dff15c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df815aba0 0x7f5df8157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:06.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dff15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df80b0030 0x7f5df80c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:06.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dff15c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df815f6d0 con 0x7f5df80b0030 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dff15c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80c8160 con 0x7f5df815aba0 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dff15c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df815f850 con 0x7f5df814c940 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dfd6f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df80b0030 0x7f5df80c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dfd6f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df80b0030 0x7f5df80c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41618/0 (socket says 172.21.15.19:41618) 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dfd6f9640 1 -- 172.21.15.19:0/2360247315 learned_addr learned my addr 172.21.15.19:0/2360247315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dfcef8640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df814c940 0x7f5df80afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5df7fff640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df815aba0 0x7f5df8157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dfd6f9640 1 -- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df814c940 msgr2=0x7f5df80afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dfd6f9640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df814c940 0x7f5df80afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dfd6f9640 1 -- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df815aba0 msgr2=0x7f5df8157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dfd6f9640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df815aba0 0x7f5df8157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:06.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.947+0000 7f5dfd6f9640 1 -- 172.21.15.19:0/2360247315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df815b030 con 0x7f5df80b0030 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.948+0000 7f5dfd6f9640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df80b0030 0x7f5df80c9dd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009f90 tx=0x7f5df0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.948+0000 7f5df57fa640 1 -- 172.21.15.19:0/2360247315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0009c10 con 0x7f5df80b0030 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.948+0000 7f5df57fa640 1 -- 172.21.15.19:0/2360247315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df0031df0 con 0x7f5df80b0030 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.948+0000 7f5dff15c640 1 -- 172.21.15.19:0/2360247315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df80b5920 con 0x7f5df80b0030 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.948+0000 7f5df57fa640 1 -- 172.21.15.19:0/2360247315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0045670 con 0x7f5df80b0030 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.948+0000 7f5dff15c640 1 -- 172.21.15.19:0/2360247315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df80b5ee0 con 0x7f5df80b0030 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.949+0000 7f5dff15c640 1 -- 172.21.15.19:0/2360247315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df80b60e0 con 0x7f5df80b0030 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.949+0000 7f5df57fa640 1 -- 172.21.15.19:0/2360247315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5df0005d10 con 0x7f5df80b0030 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.950+0000 7f5df57fa640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de406b4d0 0x7f5de406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.950+0000 7f5df57fa640 1 -- 172.21.15.19:0/2360247315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5df00aae00 con 0x7f5df80b0030 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.950+0000 7f5dfcef8640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de406b4d0 0x7f5de406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:06.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.950+0000 7f5dfcef8640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de406b4d0 0x7f5de406d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5df80b7bc0 tx=0x7f5de8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:06.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:06.954+0000 7f5df57fa640 1 -- 172.21.15.19:0/2360247315 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5df80b60e0 con 0x7f5df80b0030 2024-04-25T09:53:07.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.173+0000 7f5dff15c640 1 -- 172.21.15.19:0/2360247315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5df8073a00 con 0x7f5df80b0030 2024-04-25T09:53:07.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.785+0000 7f5df57fa640 1 -- 172.21.15.19:0/2360247315 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5df0079e30 con 0x7f5df80b0030 2024-04-25T09:53:07.788 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T09:53:07.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.792+0000 7f5dff15c640 1 -- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de406b4d0 msgr2=0x7f5de406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:07.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.792+0000 7f5dff15c640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de406b4d0 0x7f5de406d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5df80b7bc0 tx=0x7f5de8009190 comp rx=0 tx=0).stop 2024-04-25T09:53:07.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.792+0000 7f5dff15c640 1 -- 172.21.15.19:0/2360247315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df80b0030 msgr2=0x7f5df80c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:07.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.792+0000 7f5dff15c640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df80b0030 0x7f5df80c9dd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009f90 tx=0x7f5df0009750 comp rx=0 tx=0).stop 2024-04-25T09:53:07.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.793+0000 7f5dff15c640 1 -- 172.21.15.19:0/2360247315 shutdown_connections 2024-04-25T09:53:07.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.793+0000 7f5dff15c640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de406b4d0 0x7f5de406d990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:07.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.793+0000 7f5dff15c640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df80b0030 0x7f5df80c9dd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:07.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.793+0000 7f5dff15c640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df815aba0 0x7f5df8157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:07.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.793+0000 7f5dff15c640 1 --2- 172.21.15.19:0/2360247315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df814c940 0x7f5df80afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:07.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.793+0000 7f5dff15c640 1 -- 172.21.15.19:0/2360247315 >> 172.21.15.19:0/2360247315 conn(0x7f5df806f500 msgr2=0x7f5df8156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:07.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.793+0000 7f5dff15c640 1 -- 172.21.15.19:0/2360247315 shutdown_connections 2024-04-25T09:53:07.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.793+0000 7f5dff15c640 1 -- 172.21.15.19:0/2360247315 wait complete. 2024-04-25T09:53:07.814 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T09:53:07.814 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T09:53:07.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.921+0000 7f8130bfe640 1 Processor -- start 2024-04-25T09:53:07.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.921+0000 7f8130bfe640 1 -- start start 2024-04-25T09:53:07.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.922+0000 7f8130bfe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f812c161710 0x7f812c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:07.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.922+0000 7f8130bfe640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812c15e000 con 0x7f812c0ced80 2024-04-25T09:53:07.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.922+0000 7f8130bfe640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812c0d1de0 con 0x7f812c15dc20 2024-04-25T09:53:07.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.922+0000 7f8130bfe640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812c161c60 con 0x7f812c161710 2024-04-25T09:53:07.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.922+0000 7f812ad9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f812c161710 0x7f812c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:07.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.922+0000 7f8129d9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f812c0ced80 0x7f812c0d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48134/0 (socket says 172.21.15.19:48134) 2024-04-25T09:53:07.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.922+0000 7f8129d9b640 1 -- 172.21.15.19:0/1758333404 learned_addr learned my addr 172.21.15.19:0/1758333404 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:07.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.922+0000 7f812ad9d640 1 -- 172.21.15.19:0/1758333404 >> v1:172.21.15.112:6789/0 conn(0x7f812c15dc20 legacy=0x7f812c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:07.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.922+0000 7f812ad9d640 1 -- 172.21.15.19:0/1758333404 >> v1:172.21.15.19:6789/0 conn(0x7f812c0ced80 legacy=0x7f812c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:07.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.922+0000 7f812959a640 1 -- 172.21.15.19:0/1758333404 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136700477 0 0) 0x7f812c15e000 con 0x7f812c0ced80 2024-04-25T09:53:07.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.922+0000 7f812ad9d640 1 -- 172.21.15.19:0/1758333404 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c161f40 con 0x7f812c161710 2024-04-25T09:53:07.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.923+0000 7f812ad9d640 1 --2- 172.21.15.19:0/1758333404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c161710 0x7f812c0d29f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81240097a0 tx=0x7f812402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a557f3e5cd3da7a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:07.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.923+0000 7f812959a640 1 -- 172.21.15.19:0/1758333404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124037070 con 0x7f812c161710 2024-04-25T09:53:07.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.923+0000 7f812959a640 1 -- 172.21.15.19:0/1758333404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8124032440 con 0x7f812c161710 2024-04-25T09:53:07.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.923+0000 7f812959a640 1 -- 172.21.15.19:0/1758333404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124032810 con 0x7f812c161710 2024-04-25T09:53:07.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.924+0000 7f8130bfe640 1 -- 172.21.15.19:0/1758333404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c161710 msgr2=0x7f812c0d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:07.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.924+0000 7f8130bfe640 1 --2- 172.21.15.19:0/1758333404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c161710 0x7f812c0d29f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81240097a0 tx=0x7f812402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:53:07.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.924+0000 7f8130bfe640 1 -- 172.21.15.19:0/1758333404 shutdown_connections 2024-04-25T09:53:07.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.924+0000 7f8130bfe640 1 --2- 172.21.15.19:0/1758333404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c161710 0x7f812c0d29f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:07.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.924+0000 7f8130bfe640 1 -- 172.21.15.19:0/1758333404 >> 172.21.15.19:0/1758333404 conn(0x7f812c0d0420 msgr2=0x7f812c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:07.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.924+0000 7f8130bfe640 1 -- 172.21.15.19:0/1758333404 shutdown_connections 2024-04-25T09:53:07.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.924+0000 7f8130bfe640 1 -- 172.21.15.19:0/1758333404 wait complete. 2024-04-25T09:53:07.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.925+0000 7f8130bfe640 1 Processor -- start 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.925+0000 7f8130bfe640 1 -- start start 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8130bfe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0ced80 0x7f812c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8130bfe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c15dc20 0x7f812c1603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8130bfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f812c161710 0x7f812c160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8130bfe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0b5e50 con 0x7f812c15dc20 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8130bfe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c162500 con 0x7f812c161710 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8130bfe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0b5fd0 con 0x7f812c0ced80 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8129d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c15dc20 0x7f812c1603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8129d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c15dc20 0x7f812c1603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43480/0 (socket says 172.21.15.19:43480) 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8129d9b640 1 -- 172.21.15.19:0/1598799189 learned_addr learned my addr 172.21.15.19:0/1598799189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f812ad9d640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f812c161710 0x7f812c160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f812a59c640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0ced80 0x7f812c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8129d9b640 1 -- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0ced80 msgr2=0x7f812c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8129d9b640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0ced80 0x7f812c15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8129d9b640 1 -- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f812c161710 msgr2=0x7f812c160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8129d9b640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f812c161710 0x7f812c160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8129d9b640 1 -- 172.21.15.19:0/1598799189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c15b520 con 0x7f812c15dc20 2024-04-25T09:53:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.926+0000 7f8129d9b640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c15dc20 0x7f812c1603e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f811400c460 tx=0x7f811400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.927+0000 7f8122ffd640 1 -- 172.21.15.19:0/1598799189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114018070 con 0x7f812c15dc20 2024-04-25T09:53:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.927+0000 7f8122ffd640 1 -- 172.21.15.19:0/1598799189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8114007500 con 0x7f812c15dc20 2024-04-25T09:53:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.927+0000 7f8130bfe640 1 -- 172.21.15.19:0/1598799189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c1f42f0 con 0x7f812c15dc20 2024-04-25T09:53:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.927+0000 7f8122ffd640 1 -- 172.21.15.19:0/1598799189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114004a20 con 0x7f812c15dc20 2024-04-25T09:53:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.927+0000 7f8130bfe640 1 -- 172.21.15.19:0/1598799189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c1f48b0 con 0x7f812c15dc20 2024-04-25T09:53:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.928+0000 7f8130bfe640 1 -- 172.21.15.19:0/1598799189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812c073a00 con 0x7f812c15dc20 2024-04-25T09:53:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.928+0000 7f8122ffd640 1 -- 172.21.15.19:0/1598799189 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f811401b070 con 0x7f812c15dc20 2024-04-25T09:53:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.929+0000 7f8122ffd640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f810c06b4d0 0x7f810c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.929+0000 7f8122ffd640 1 -- 172.21.15.19:0/1598799189 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8114089420 con 0x7f812c15dc20 2024-04-25T09:53:07.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.929+0000 7f812a59c640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f810c06b4d0 0x7f810c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:07.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.929+0000 7f812a59c640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f810c06b4d0 0x7f810c06d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f811c005660 tx=0x7f811c003d90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:07.933+0000 7f8122ffd640 1 -- 172.21.15.19:0/1598799189 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8114057c50 con 0x7f812c15dc20 2024-04-25T09:53:08.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.121+0000 7f8130bfe640 1 -- 172.21.15.19:0/1598799189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f812c0d16b0 con 0x7f812c15dc20 2024-04-25T09:53:08.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.768+0000 7f8122ffd640 1 -- 172.21.15.19:0/1598799189 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f811405bcf0 con 0x7f812c15dc20 2024-04-25T09:53:08.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.771+0000 7f8130bfe640 1 -- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f810c06b4d0 msgr2=0x7f810c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.771+0000 7f8130bfe640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f810c06b4d0 0x7f810c06d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f811c005660 tx=0x7f811c003d90 comp rx=0 tx=0).stop 2024-04-25T09:53:08.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.771+0000 7f8130bfe640 1 -- 172.21.15.19:0/1598799189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c15dc20 msgr2=0x7f812c1603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.771+0000 7f8130bfe640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c15dc20 0x7f812c1603e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f811400c460 tx=0x7f811400c920 comp rx=0 tx=0).stop 2024-04-25T09:53:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.771+0000 7f8130bfe640 1 -- 172.21.15.19:0/1598799189 shutdown_connections 2024-04-25T09:53:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.771+0000 7f8130bfe640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f810c06b4d0 0x7f810c06d990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.771+0000 7f8130bfe640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f812c161710 0x7f812c160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.771+0000 7f8130bfe640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c15dc20 0x7f812c1603e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.771+0000 7f8130bfe640 1 --2- 172.21.15.19:0/1598799189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0ced80 0x7f812c15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.771+0000 7f8130bfe640 1 -- 172.21.15.19:0/1598799189 >> 172.21.15.19:0/1598799189 conn(0x7f812c0d0420 msgr2=0x7f812c0d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.771+0000 7f8130bfe640 1 -- 172.21.15.19:0/1598799189 shutdown_connections 2024-04-25T09:53:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.772+0000 7f8130bfe640 1 -- 172.21.15.19:0/1598799189 wait complete. 2024-04-25T09:53:08.788 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:53:08.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.901+0000 7f174e24a640 1 Processor -- start 2024-04-25T09:53:08.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.901+0000 7f174e24a640 1 -- start start 2024-04-25T09:53:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.902+0000 7f174e24a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174815b990 0x7f1748072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.902+0000 7f174e24a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f174806c300 0x7f17480acd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.902+0000 7f174e24a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1748072ca0 con 0x7f174815bd70 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f1747fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f174806c300 0x7f17480acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f174e24a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174815c260 con 0x7f174806c300 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f174e24a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174815c540 con 0x7f174815b990 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f1746ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f174815bd70 0x7f174806bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48146/0 (socket says 172.21.15.19:48146) 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f1746ffd640 1 -- 172.21.15.19:0/1379384494 learned_addr learned my addr 172.21.15.19:0/1379384494 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f17477fe640 1 --2- 172.21.15.19:0/1379384494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174815b990 0x7f1748072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f1747fff640 1 -- 172.21.15.19:0/1379384494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174815b990 msgr2=0x7f1748072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f1747fff640 1 --2- 172.21.15.19:0/1379384494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174815b990 0x7f1748072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f1747fff640 1 -- 172.21.15.19:0/1379384494 >> v1:172.21.15.19:6789/0 conn(0x7f174815bd70 legacy=0x7f174806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f1747fff640 1 -- 172.21.15.19:0/1379384494 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17480cb9d0 con 0x7f174806c300 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f17477fe640 1 --2- 172.21.15.19:0/1379384494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174815b990 0x7f1748072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.903+0000 7f1747fff640 1 --2- 172.21.15.19:0/1379384494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f174806c300 0x7f17480acd80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1740009d90 tx=0x7f174002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880e89dcc4c20657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.904+0000 7f17467fc640 1 -- 172.21.15.19:0/1379384494 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740007740 con 0x7f174806c300 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.904+0000 7f17467fc640 1 -- 172.21.15.19:0/1379384494 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17400078e0 con 0x7f174806c300 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.904+0000 7f17467fc640 1 -- 172.21.15.19:0/1379384494 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740007cb0 con 0x7f174806c300 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.904+0000 7f174e24a640 1 -- 172.21.15.19:0/1379384494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f174806c300 msgr2=0x7f17480acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.904+0000 7f174e24a640 1 --2- 172.21.15.19:0/1379384494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f174806c300 0x7f17480acd80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1740009d90 tx=0x7f174002a660 comp rx=0 tx=0).stop 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.904+0000 7f174e24a640 1 -- 172.21.15.19:0/1379384494 shutdown_connections 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.904+0000 7f174e24a640 1 --2- 172.21.15.19:0/1379384494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f174806c300 0x7f17480acd80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.904+0000 7f174e24a640 1 --2- 172.21.15.19:0/1379384494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174815b990 0x7f1748072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.904+0000 7f174e24a640 1 -- 172.21.15.19:0/1379384494 >> 172.21.15.19:0/1379384494 conn(0x7f1748071af0 msgr2=0x7f1748071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.905+0000 7f174e24a640 1 -- 172.21.15.19:0/1379384494 shutdown_connections 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.905+0000 7f174e24a640 1 -- 172.21.15.19:0/1379384494 wait complete. 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.906+0000 7f174e24a640 1 Processor -- start 2024-04-25T09:53:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.906+0000 7f174e24a640 1 -- start start 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.906+0000 7f174e24a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174806c300 0x7f17480adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.906+0000 7f174e24a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f174815b990 0x7f17480ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.906+0000 7f174e24a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f174815bd70 0x7f17480c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.906+0000 7f174e24a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17480af250 con 0x7f174815b990 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.906+0000 7f174e24a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17480af0d0 con 0x7f174815bd70 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f174e24a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17480af3d0 con 0x7f174806c300 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f1746ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f174815b990 0x7f17480ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f1746ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f174815b990 0x7f17480ae440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43506/0 (socket says 172.21.15.19:43506) 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f1746ffd640 1 -- 172.21.15.19:0/2060489 learned_addr learned my addr 172.21.15.19:0/2060489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f17477fe640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174806c300 0x7f17480adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f1746ffd640 1 -- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174806c300 msgr2=0x7f17480adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f1746ffd640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174806c300 0x7f17480adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f1746ffd640 1 -- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f174815bd70 msgr2=0x7f17480c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f1747fff640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f174815bd70 0x7f17480c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f1746ffd640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f174815bd70 0x7f17480c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f1746ffd640 1 -- 172.21.15.19:0/2060489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748154180 con 0x7f174815b990 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f17477fe640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174806c300 0x7f17480adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f1746ffd640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f174815b990 0x7f17480ae440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f173c00c440 tx=0x7f173c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f172ffff640 1 -- 172.21.15.19:0/2060489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c017070 con 0x7f174815b990 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f172ffff640 1 -- 172.21.15.19:0/2060489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c005ce0 con 0x7f174815b990 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f174e24a640 1 -- 172.21.15.19:0/2060489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17481557e0 con 0x7f174815b990 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f172ffff640 1 -- 172.21.15.19:0/2060489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c0072f0 con 0x7f174815b990 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.907+0000 7f174e24a640 1 -- 172.21.15.19:0/2060489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1748155da0 con 0x7f174815b990 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.908+0000 7f174e24a640 1 -- 172.21.15.19:0/2060489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1748073a00 con 0x7f174815b990 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.909+0000 7f172ffff640 1 -- 172.21.15.19:0/2060489 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f173c01a070 con 0x7f174815b990 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.909+0000 7f172ffff640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f173406b4d0 0x7f173406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.909+0000 7f172ffff640 1 -- 172.21.15.19:0/2060489 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6007+0+0 (secure 0 0 0) 0x7f173c0895a0 con 0x7f174815b990 2024-04-25T09:53:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.909+0000 7f17477fe640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f173406b4d0 0x7f173406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.910+0000 7f17477fe640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f173406b4d0 0x7f173406d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1738006fd0 tx=0x7f1738008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:08.913+0000 7f172ffff640 1 -- 172.21.15.19:0/2060489 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f173c058380 con 0x7f174815b990 2024-04-25T09:53:09.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:09.093+0000 7f174e24a640 1 -- 172.21.15.19:0/2060489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f17481555c0 con 0x7f174815b990 2024-04-25T09:53:10.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.797+0000 7f172ffff640 1 -- 172.21.15.19:0/2060489 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f173c057c80 con 0x7f174815b990 2024-04-25T09:53:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data_ec' created 2024-04-25T09:53:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 -- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f173406b4d0 msgr2=0x7f173406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f173406b4d0 0x7f173406d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1738006fd0 tx=0x7f1738008040 comp rx=0 tx=0).stop 2024-04-25T09:53:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 -- 172.21.15.19:0/2060489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f174815b990 msgr2=0x7f17480ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f174815b990 0x7f17480ae440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f173c00c440 tx=0x7f173c00c900 comp rx=0 tx=0).stop 2024-04-25T09:53:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 -- 172.21.15.19:0/2060489 shutdown_connections 2024-04-25T09:53:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f173406b4d0 0x7f173406d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f174815bd70 0x7f17480c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f174815b990 0x7f17480ae440 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 --2- 172.21.15.19:0/2060489 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f174806c300 0x7f17480adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 -- 172.21.15.19:0/2060489 >> 172.21.15.19:0/2060489 conn(0x7f1748071af0 msgr2=0x7f174806fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 -- 172.21.15.19:0/2060489 shutdown_connections 2024-04-25T09:53:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.801+0000 7f174e24a640 1 -- 172.21.15.19:0/2060489 wait complete. 2024-04-25T09:53:10.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T09:53:10.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.930+0000 7f49feb5c640 1 Processor -- start 2024-04-25T09:53:10.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.930+0000 7f49feb5c640 1 -- start start 2024-04-25T09:53:10.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.931+0000 7f49feb5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f49f80b9750 0x7f49f80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.931+0000 7f49feb5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 0x7f49f80d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.931+0000 7f49feb5c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f80cf060 con 0x7f49f815da90 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.931+0000 7f49feb5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f8162600 con 0x7f49f80b9750 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.931+0000 7f49feb5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8161d70 con 0x7f49f80cebb0 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.931+0000 7f49fd0f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 0x7f49f80d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.931+0000 7f49f7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f49f80b9750 0x7f49f80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.931+0000 7f49fd0f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 0x7f49f80d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:48980/0 (socket says 172.21.15.19:48980) 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.931+0000 7f49fd0f9640 1 -- 172.21.15.19:0/2794431905 learned_addr learned my addr 172.21.15.19:0/2794431905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.932+0000 7f49fd0f9640 1 -- 172.21.15.19:0/2794431905 >> v2:172.21.15.112:3300/0 conn(0x7f49f80b9750 msgr2=0x7f49f80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.932+0000 7f49fd0f9640 1 --2- 172.21.15.19:0/2794431905 >> v2:172.21.15.112:3300/0 conn(0x7f49f80b9750 0x7f49f80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.932+0000 7f49fd0f9640 1 -- 172.21.15.19:0/2794431905 >> v1:172.21.15.112:6789/0 conn(0x7f49f815da90 legacy=0x7f49f8161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.932+0000 7f49fd0f9640 1 -- 172.21.15.19:0/2794431905 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f8162040 con 0x7f49f80cebb0 2024-04-25T09:53:10.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.932+0000 7f49f7fff640 1 --2- 172.21.15.19:0/2794431905 >> v2:172.21.15.112:3300/0 conn(0x7f49f80b9750 0x7f49f80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:10.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.932+0000 7f49fd0f9640 1 --2- 172.21.15.19:0/2794431905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 0x7f49f80d2310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009e50 tx=0x7f49f002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10bdb0dde857d20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:10.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.932+0000 7f49f77fe640 1 -- 172.21.15.19:0/2794431905 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0036070 con 0x7f49f80cebb0 2024-04-25T09:53:10.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.932+0000 7f49f77fe640 1 -- 172.21.15.19:0/2794431905 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f0031600 con 0x7f49f80cebb0 2024-04-25T09:53:10.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.933+0000 7f49f77fe640 1 -- 172.21.15.19:0/2794431905 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f00319d0 con 0x7f49f80cebb0 2024-04-25T09:53:10.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.933+0000 7f49feb5c640 1 -- 172.21.15.19:0/2794431905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 msgr2=0x7f49f80d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.933+0000 7f49feb5c640 1 --2- 172.21.15.19:0/2794431905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 0x7f49f80d2310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009e50 tx=0x7f49f002a680 comp rx=0 tx=0).stop 2024-04-25T09:53:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.933+0000 7f49feb5c640 1 -- 172.21.15.19:0/2794431905 shutdown_connections 2024-04-25T09:53:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.933+0000 7f49feb5c640 1 --2- 172.21.15.19:0/2794431905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 0x7f49f80d2310 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.933+0000 7f49feb5c640 1 --2- 172.21.15.19:0/2794431905 >> v2:172.21.15.112:3300/0 conn(0x7f49f80b9750 0x7f49f80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.933+0000 7f49feb5c640 1 -- 172.21.15.19:0/2794431905 >> 172.21.15.19:0/2794431905 conn(0x7f49f8061a20 msgr2=0x7f49f8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.933+0000 7f49feb5c640 1 -- 172.21.15.19:0/2794431905 shutdown_connections 2024-04-25T09:53:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.933+0000 7f49feb5c640 1 -- 172.21.15.19:0/2794431905 wait complete. 2024-04-25T09:53:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.934+0000 7f49feb5c640 1 Processor -- start 2024-04-25T09:53:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.934+0000 7f49feb5c640 1 -- start start 2024-04-25T09:53:10.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.935+0000 7f49feb5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 0x7f49f80c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:10.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.935+0000 7f49feb5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f815da90 0x7f49f815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:10.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.935+0000 7f49feb5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f80caa70 0x7f49f80d3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:10.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.935+0000 7f49feb5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f80d1df0 con 0x7f49f815da90 2024-04-25T09:53:10.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.935+0000 7f49feb5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f80d1b10 con 0x7f49f80caa70 2024-04-25T09:53:10.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.935+0000 7f49feb5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f815de70 con 0x7f49f80cebb0 2024-04-25T09:53:10.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.935+0000 7f49f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f815da90 0x7f49f815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:10.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.935+0000 7f49f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f815da90 0x7f49f815f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43520/0 (socket says 172.21.15.19:43520) 2024-04-25T09:53:10.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.935+0000 7f49f7fff640 1 -- 172.21.15.19:0/3951198822 learned_addr learned my addr 172.21.15.19:0/3951198822 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:10.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49fc8f8640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 0x7f49f80c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49fd0f9640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f80caa70 0x7f49f80d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49f7fff640 1 -- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 msgr2=0x7f49f80c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49f7fff640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 0x7f49f80c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49f7fff640 1 -- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f80caa70 msgr2=0x7f49f80d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49f7fff640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f80caa70 0x7f49f80d3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49f7fff640 1 -- 172.21.15.19:0/3951198822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f80c9a60 con 0x7f49f815da90 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49fc8f8640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 0x7f49f80c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49f7fff640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f815da90 0x7f49f815f5e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00ea10 tx=0x7f49ec00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49f57fa640 1 -- 172.21.15.19:0/3951198822 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec017070 con 0x7f49f815da90 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49f57fa640 1 -- 172.21.15.19:0/3951198822 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec00ca60 con 0x7f49f815da90 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49feb5c640 1 -- 172.21.15.19:0/3951198822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f80cbc40 con 0x7f49f815da90 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49f57fa640 1 -- 172.21.15.19:0/3951198822 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00ce50 con 0x7f49f815da90 2024-04-25T09:53:10.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.936+0000 7f49feb5c640 1 -- 172.21.15.19:0/3951198822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f80cb310 con 0x7f49f815da90 2024-04-25T09:53:10.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.937+0000 7f49feb5c640 1 -- 172.21.15.19:0/3951198822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f8073a00 con 0x7f49f815da90 2024-04-25T09:53:10.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.938+0000 7f49f57fa640 1 -- 172.21.15.19:0/3951198822 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49ec01a070 con 0x7f49f815da90 2024-04-25T09:53:10.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.938+0000 7f49f57fa640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f49e406b4d0 0x7f49e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:10.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.938+0000 7f49f57fa640 1 -- 172.21.15.19:0/3951198822 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f49ec08a0a0 con 0x7f49f815da90 2024-04-25T09:53:10.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.938+0000 7f49fc8f8640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f49e406b4d0 0x7f49e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:10.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.939+0000 7f49fc8f8640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f49e406b4d0 0x7f49e406d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49e8001660 tx=0x7f49e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:10.943+0000 7f49f57fa640 1 -- 172.21.15.19:0/3951198822 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49f80c9a60 con 0x7f49f815da90 2024-04-25T09:53:11.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.122+0000 7f49feb5c640 1 -- 172.21.15.19:0/3951198822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f49f80b4060 con 0x7f49f815da90 2024-04-25T09:53:11.559 INFO:tasks.ceph.mds.d.smithi112.stderr:2024-04-25T09:53:11.557+0000 7fa09dfca640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:53:11.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.804+0000 7f49f57fa640 1 -- 172.21.15.19:0/3951198822 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f49ec058c00 con 0x7f49f815da90 2024-04-25T09:53:11.807 INFO:teuthology.orchestra.run.smithi019.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T09:53:11.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.809+0000 7f49feb5c640 1 -- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f49e406b4d0 msgr2=0x7f49e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.809+0000 7f49feb5c640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f49e406b4d0 0x7f49e406d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49e8001660 tx=0x7f49e8009040 comp rx=0 tx=0).stop 2024-04-25T09:53:11.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.809+0000 7f49feb5c640 1 -- 172.21.15.19:0/3951198822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f815da90 msgr2=0x7f49f815f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.809+0000 7f49feb5c640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f815da90 0x7f49f815f5e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00ea10 tx=0x7f49ec00eed0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.810+0000 7f49feb5c640 1 -- 172.21.15.19:0/3951198822 shutdown_connections 2024-04-25T09:53:11.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.810+0000 7f49feb5c640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f49e406b4d0 0x7f49e406d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.810+0000 7f49feb5c640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f80caa70 0x7f49f80d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.810+0000 7f49feb5c640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f815da90 0x7f49f815f5e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.810+0000 7f49feb5c640 1 --2- 172.21.15.19:0/3951198822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49f80cebb0 0x7f49f80c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.810+0000 7f49feb5c640 1 -- 172.21.15.19:0/3951198822 >> 172.21.15.19:0/3951198822 conn(0x7f49f8061a20 msgr2=0x7f49f80b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:11.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.815+0000 7f49feb5c640 1 -- 172.21.15.19:0/3951198822 shutdown_connections 2024-04-25T09:53:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.816+0000 7f49feb5c640 1 -- 172.21.15.19:0/3951198822 wait complete. 2024-04-25T09:53:11.842 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T09:53:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.947+0000 7fc474aac640 1 Processor -- start 2024-04-25T09:53:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.947+0000 7fc474aac640 1 -- start start 2024-04-25T09:53:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc474aac640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc47015da90 0x7fc470161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc474aac640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc470161ba0 0x7fc4700d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc474aac640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc470162040 con 0x7fc4700b9750 2024-04-25T09:53:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc474aac640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc470162600 con 0x7fc47015da90 2024-04-25T09:53:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc474aac640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc470162320 con 0x7fc470161ba0 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc46e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc47015da90 0x7fc470161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc46e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc47015da90 0x7fc470161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43534/0 (socket says 172.21.15.19:43534) 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc46e59c640 1 -- 172.21.15.19:0/2620942200 learned_addr learned my addr 172.21.15.19:0/2620942200 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc46ed9d640 1 --2- 172.21.15.19:0/2620942200 >> v2:172.21.15.112:3300/0 conn(0x7fc470161ba0 0x7fc4700d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc46e59c640 1 -- 172.21.15.19:0/2620942200 >> v2:172.21.15.112:3300/0 conn(0x7fc470161ba0 msgr2=0x7fc4700d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc46e59c640 1 --2- 172.21.15.19:0/2620942200 >> v2:172.21.15.112:3300/0 conn(0x7fc470161ba0 0x7fc4700d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc46e59c640 1 -- 172.21.15.19:0/2620942200 >> v1:172.21.15.112:6789/0 conn(0x7fc4700b9750 legacy=0x7fc4700ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.948+0000 7fc46e59c640 1 -- 172.21.15.19:0/2620942200 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4700d1b10 con 0x7fc47015da90 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.949+0000 7fc46e59c640 1 --2- 172.21.15.19:0/2620942200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc47015da90 0x7fc470161660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc4700b6080 tx=0x7fc46802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99e71c69c0938a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.949+0000 7fc46d59a640 1 -- 172.21.15.19:0/2620942200 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468037070 con 0x7fc47015da90 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.949+0000 7fc46d59a640 1 -- 172.21.15.19:0/2620942200 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc468032440 con 0x7fc47015da90 2024-04-25T09:53:11.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.949+0000 7fc46d59a640 1 -- 172.21.15.19:0/2620942200 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468032810 con 0x7fc47015da90 2024-04-25T09:53:11.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.950+0000 7fc474aac640 1 -- 172.21.15.19:0/2620942200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc47015da90 msgr2=0x7fc470161660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.950+0000 7fc474aac640 1 --2- 172.21.15.19:0/2620942200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc47015da90 0x7fc470161660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc4700b6080 tx=0x7fc46802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.950+0000 7fc474aac640 1 -- 172.21.15.19:0/2620942200 shutdown_connections 2024-04-25T09:53:11.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.950+0000 7fc474aac640 1 --2- 172.21.15.19:0/2620942200 >> v2:172.21.15.112:3300/0 conn(0x7fc470161ba0 0x7fc4700d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.950+0000 7fc474aac640 1 --2- 172.21.15.19:0/2620942200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc47015da90 0x7fc470161660 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.950+0000 7fc474aac640 1 -- 172.21.15.19:0/2620942200 >> 172.21.15.19:0/2620942200 conn(0x7fc470061a20 msgr2=0x7fc470061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:11.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.950+0000 7fc474aac640 1 -- 172.21.15.19:0/2620942200 shutdown_connections 2024-04-25T09:53:11.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.950+0000 7fc474aac640 1 -- 172.21.15.19:0/2620942200 wait complete. 2024-04-25T09:53:11.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.951+0000 7fc474aac640 1 Processor -- start 2024-04-25T09:53:11.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.951+0000 7fc474aac640 1 -- start start 2024-04-25T09:53:11.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc474aac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc47015da90 0x7fc4700cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc474aac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc470161ba0 0x7fc4701f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc474aac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4701f1040 0x7fc4700b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc474aac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700ced80 con 0x7fc47015da90 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc474aac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47015de70 con 0x7fc470161ba0 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc474aac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700cef00 con 0x7fc4701f1040 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc46e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc47015da90 0x7fc4700cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc46e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc47015da90 0x7fc4700cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43536/0 (socket says 172.21.15.19:43536) 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc46e59c640 1 -- 172.21.15.19:0/994581134 learned_addr learned my addr 172.21.15.19:0/994581134 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc46dd9b640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc470161ba0 0x7fc4701f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc46ed9d640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4701f1040 0x7fc4700b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc46e59c640 1 -- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4701f1040 msgr2=0x7fc4700b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc46e59c640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4701f1040 0x7fc4700b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc46e59c640 1 -- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc470161ba0 msgr2=0x7fc4701f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.952+0000 7fc46e59c640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc470161ba0 0x7fc4701f0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.953+0000 7fc46e59c640 1 -- 172.21.15.19:0/994581134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4700cca90 con 0x7fc47015da90 2024-04-25T09:53:11.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.953+0000 7fc46e59c640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc47015da90 0x7fc4700cc550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc46802a220 tx=0x7fc4680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:11.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.953+0000 7fc45effd640 1 -- 172.21.15.19:0/994581134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468037040 con 0x7fc47015da90 2024-04-25T09:53:11.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.953+0000 7fc45effd640 1 -- 172.21.15.19:0/994581134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc468042070 con 0x7fc47015da90 2024-04-25T09:53:11.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.953+0000 7fc474aac640 1 -- 172.21.15.19:0/994581134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4700caa70 con 0x7fc47015da90 2024-04-25T09:53:11.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.953+0000 7fc45effd640 1 -- 172.21.15.19:0/994581134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468030880 con 0x7fc47015da90 2024-04-25T09:53:11.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.953+0000 7fc474aac640 1 -- 172.21.15.19:0/994581134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4700cb2d0 con 0x7fc47015da90 2024-04-25T09:53:11.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.954+0000 7fc474aac640 1 -- 172.21.15.19:0/994581134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc470073a00 con 0x7fc47015da90 2024-04-25T09:53:11.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.954+0000 7fc45effd640 1 -- 172.21.15.19:0/994581134 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc468030a20 con 0x7fc47015da90 2024-04-25T09:53:11.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.955+0000 7fc45effd640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc45006b4d0 0x7fc45006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.955+0000 7fc45effd640 1 -- 172.21.15.19:0/994581134 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc4680aac20 con 0x7fc47015da90 2024-04-25T09:53:11.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.955+0000 7fc46dd9b640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc45006b4d0 0x7fc45006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.955+0000 7fc46dd9b640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc45006b4d0 0x7fc45006d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4700cc230 tx=0x7fc464009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:11.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:11.959+0000 7fc45effd640 1 -- 172.21.15.19:0/994581134 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4700cca90 con 0x7fc47015da90 2024-04-25T09:53:12.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.180+0000 7fc474aac640 1 -- 172.21.15.19:0/994581134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fc4700d1590 con 0x7fc47015da90 2024-04-25T09:53:12.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.811+0000 7fc45effd640 1 -- 172.21.15.19:0/994581134 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fc4680791c0 con 0x7fc47015da90 2024-04-25T09:53:12.813 INFO:teuthology.orchestra.run.smithi019.stderr:added data pool 4 to fsmap 2024-04-25T09:53:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.813+0000 7fc474aac640 1 -- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc45006b4d0 msgr2=0x7fc45006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.813+0000 7fc474aac640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc45006b4d0 0x7fc45006d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4700cc230 tx=0x7fc464009040 comp rx=0 tx=0).stop 2024-04-25T09:53:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.813+0000 7fc474aac640 1 -- 172.21.15.19:0/994581134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc47015da90 msgr2=0x7fc4700cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.813+0000 7fc474aac640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc47015da90 0x7fc4700cc550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc46802a220 tx=0x7fc4680046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.814+0000 7fc474aac640 1 -- 172.21.15.19:0/994581134 shutdown_connections 2024-04-25T09:53:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.814+0000 7fc474aac640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc45006b4d0 0x7fc45006d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.814+0000 7fc474aac640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4701f1040 0x7fc4700b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.814+0000 7fc474aac640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc470161ba0 0x7fc4701f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.814+0000 7fc474aac640 1 --2- 172.21.15.19:0/994581134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc47015da90 0x7fc4700cc550 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.814+0000 7fc474aac640 1 -- 172.21.15.19:0/994581134 >> 172.21.15.19:0/994581134 conn(0x7fc470061a20 msgr2=0x7fc4700ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.814+0000 7fc474aac640 1 -- 172.21.15.19:0/994581134 shutdown_connections 2024-04-25T09:53:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.814+0000 7fc474aac640 1 -- 172.21.15.19:0/994581134 wait complete. 2024-04-25T09:53:12.829 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:12.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.934+0000 7f9668e6b640 1 Processor -- start 2024-04-25T09:53:12.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.934+0000 7f9668e6b640 1 -- start start 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.935+0000 7f9668e6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f966415db60 0x7f96640ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.935+0000 7f9668e6b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96640b5f00 0x7f96640d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.935+0000 7f9668e6b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96640cec00 con 0x7f966415df40 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.935+0000 7f9668e6b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96640ce930 con 0x7f96640b5f00 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.935+0000 7f9668e6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640b6450 con 0x7f966415db60 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.935+0000 7f9661d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96640b5f00 0x7f96640d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.935+0000 7f9662d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f966415df40 0x7f9664161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48148/0 (socket says 172.21.15.19:48148) 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.935+0000 7f9662d9d640 1 -- 172.21.15.19:0/3574273315 learned_addr learned my addr 172.21.15.19:0/3574273315 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.936+0000 7f966259c640 1 --2- 172.21.15.19:0/3574273315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f966415db60 0x7f96640ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.936+0000 7f9661d9b640 1 -- 172.21.15.19:0/3574273315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f966415db60 msgr2=0x7f96640ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.936+0000 7f9661d9b640 1 --2- 172.21.15.19:0/3574273315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f966415db60 0x7f96640ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.936+0000 7f9661d9b640 1 -- 172.21.15.19:0/3574273315 >> v1:172.21.15.19:6789/0 conn(0x7f966415df40 legacy=0x7f9664161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.936+0000 7f966159a640 1 -- 172.21.15.19:0/3574273315 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635373392 0 0) 0x7f96640cec00 con 0x7f966415df40 2024-04-25T09:53:12.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.936+0000 7f9661d9b640 1 -- 172.21.15.19:0/3574273315 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640ceee0 con 0x7f96640b5f00 2024-04-25T09:53:12.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.937+0000 7f9661d9b640 1 --2- 172.21.15.19:0/3574273315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96640b5f00 0x7f96640d0400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96580097d0 tx=0x7f965802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c1166c5a4b6cc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.937+0000 7f966159a640 1 -- 172.21.15.19:0/3574273315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658037070 con 0x7f96640b5f00 2024-04-25T09:53:12.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.937+0000 7f966159a640 1 -- 172.21.15.19:0/3574273315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965802ae40 con 0x7f96640b5f00 2024-04-25T09:53:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.937+0000 7f966159a640 1 -- 172.21.15.19:0/3574273315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658032660 con 0x7f96640b5f00 2024-04-25T09:53:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.937+0000 7f9668e6b640 1 -- 172.21.15.19:0/3574273315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96640b5f00 msgr2=0x7f96640d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.937+0000 7f9668e6b640 1 --2- 172.21.15.19:0/3574273315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96640b5f00 0x7f96640d0400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96580097d0 tx=0x7f965802a130 comp rx=0 tx=0).stop 2024-04-25T09:53:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.937+0000 7f9668e6b640 1 -- 172.21.15.19:0/3574273315 shutdown_connections 2024-04-25T09:53:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.937+0000 7f9668e6b640 1 --2- 172.21.15.19:0/3574273315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96640b5f00 0x7f96640d0400 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.937+0000 7f9668e6b640 1 --2- 172.21.15.19:0/3574273315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f966415db60 0x7f96640ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.937+0000 7f9668e6b640 1 -- 172.21.15.19:0/3574273315 >> 172.21.15.19:0/3574273315 conn(0x7f9664061600 msgr2=0x7f9664061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.937+0000 7f9668e6b640 1 -- 172.21.15.19:0/3574273315 shutdown_connections 2024-04-25T09:53:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.938+0000 7f9668e6b640 1 -- 172.21.15.19:0/3574273315 wait complete. 2024-04-25T09:53:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.939+0000 7f9668e6b640 1 Processor -- start 2024-04-25T09:53:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.939+0000 7f9668e6b640 1 -- start start 2024-04-25T09:53:12.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9668e6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96640b5f00 0x7f966414be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9668e6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f966415db60 0x7f96640c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9668e6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f966415df40 0x7f96640ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9668e6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96640d3280 con 0x7f96640b5f00 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9668e6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96640d1d50 con 0x7f966415db60 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9668e6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640d3400 con 0x7f966415df40 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f966259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96640b5f00 0x7f966414be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f966259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96640b5f00 0x7f966414be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43558/0 (socket says 172.21.15.19:43558) 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f966259c640 1 -- 172.21.15.19:0/3137288349 learned_addr learned my addr 172.21.15.19:0/3137288349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9662d9d640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f966415df40 0x7f96640ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9661d9b640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f966415db60 0x7f96640c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f966259c640 1 -- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f966415df40 msgr2=0x7f96640ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f966259c640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f966415df40 0x7f96640ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f966259c640 1 -- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f966415db60 msgr2=0x7f96640c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f966259c640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f966415db60 0x7f96640c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f966259c640 1 -- 172.21.15.19:0/3137288349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966414c3d0 con 0x7f96640b5f00 2024-04-25T09:53:12.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f966259c640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96640b5f00 0x7f966414be90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f965c00d830 tx=0x7f965c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9652ffd640 1 -- 172.21.15.19:0/3137288349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00b6c0 con 0x7f96640b5f00 2024-04-25T09:53:12.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9652ffd640 1 -- 172.21.15.19:0/3137288349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c019070 con 0x7f96640b5f00 2024-04-25T09:53:12.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9668e6b640 1 -- 172.21.15.19:0/3137288349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96640cd580 con 0x7f96640b5f00 2024-04-25T09:53:12.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9652ffd640 1 -- 172.21.15.19:0/3137288349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00ba80 con 0x7f96640b5f00 2024-04-25T09:53:12.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.940+0000 7f9668e6b640 1 -- 172.21.15.19:0/3137288349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96640c75a0 con 0x7f96640b5f00 2024-04-25T09:53:12.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.942+0000 7f9668e6b640 1 -- 172.21.15.19:0/3137288349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9664073270 con 0x7f96640b5f00 2024-04-25T09:53:12.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.942+0000 7f9652ffd640 1 -- 172.21.15.19:0/3137288349 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f965c01e460 con 0x7f96640b5f00 2024-04-25T09:53:12.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.942+0000 7f9652ffd640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f964406b4d0 0x7f964406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.943+0000 7f9652ffd640 1 -- 172.21.15.19:0/3137288349 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f965c08abf0 con 0x7f96640b5f00 2024-04-25T09:53:12.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.943+0000 7f9661d9b640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f964406b4d0 0x7f964406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.943+0000 7f9661d9b640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f964406b4d0 0x7f964406d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9658034000 tx=0x7f965802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:12.947+0000 7f9652ffd640 1 -- 172.21.15.19:0/3137288349 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f965c059730 con 0x7f96640b5f00 2024-04-25T09:53:13.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.170+0000 7f9668e6b640 1 -- 172.21.15.19:0/3137288349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9664073990 con 0x7f96640b5f00 2024-04-25T09:53:13.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.170+0000 7f9652ffd640 1 -- 172.21.15.19:0/3137288349 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f965c059030 con 0x7f96640b5f00 2024-04-25T09:53:13.172 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:13.172 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4000846111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":4000846111},{"type":"v1","addr":"172.21.15.112:6837","nonce":4000846111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1969525282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1969525282},{"type":"v1","addr":"172.21.15.19:6835","nonce":1969525282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:07.175350+0000","modified":"2024-04-25T09:53:12.181345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1080352556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1080352556},{"type":"v1","addr":"172.21.15.112:6836","nonce":1080352556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T09:53:13.173 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 7 2024-04-25T09:53:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 -- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f964406b4d0 msgr2=0x7f964406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f964406b4d0 0x7f964406d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9658034000 tx=0x7f965802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 -- 172.21.15.19:0/3137288349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96640b5f00 msgr2=0x7f966414be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96640b5f00 0x7f966414be90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f965c00d830 tx=0x7f965c00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 -- 172.21.15.19:0/3137288349 shutdown_connections 2024-04-25T09:53:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f964406b4d0 0x7f964406d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f966415df40 0x7f96640ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f966415db60 0x7f96640c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 --2- 172.21.15.19:0/3137288349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96640b5f00 0x7f966414be90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 -- 172.21.15.19:0/3137288349 >> 172.21.15.19:0/3137288349 conn(0x7f9664061600 msgr2=0x7f96640b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:13.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 -- 172.21.15.19:0/3137288349 shutdown_connections 2024-04-25T09:53:13.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.172+0000 7f9668e6b640 1 -- 172.21.15.19:0/3137288349 wait complete. 2024-04-25T09:53:13.185 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.283+0000 7f852622a640 1 Processor -- start 2024-04-25T09:53:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.283+0000 7f852622a640 1 -- start start 2024-04-25T09:53:13.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f852622a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200b2cb0 0x7f85200d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f852622a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f85200d1020 0x7f85200d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f852622a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85200b97a0 con 0x7f852015dbe0 2024-04-25T09:53:13.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f852622a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852015dfc0 con 0x7f85200d1020 2024-04-25T09:53:13.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f852622a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85200d1a00 con 0x7f85200b2cb0 2024-04-25T09:53:13.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f851ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f85200d1020 0x7f85200d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f851f7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f852015dbe0 0x7f85200d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48162/0 (socket says 172.21.15.19:48162) 2024-04-25T09:53:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f851f7fe640 1 -- 172.21.15.19:0/3971061038 learned_addr learned my addr 172.21.15.19:0/3971061038 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f851effd640 1 --2- 172.21.15.19:0/3971061038 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200b2cb0 0x7f85200d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f851ffff640 1 -- 172.21.15.19:0/3971061038 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200b2cb0 msgr2=0x7f85200d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f851ffff640 1 --2- 172.21.15.19:0/3971061038 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200b2cb0 0x7f85200d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f851e7fc640 1 -- 172.21.15.19:0/3971061038 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563214405 0 0) 0x7f85200b97a0 con 0x7f852015dbe0 2024-04-25T09:53:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.284+0000 7f851ffff640 1 -- 172.21.15.19:0/3971061038 >> v1:172.21.15.19:6789/0 conn(0x7f852015dbe0 legacy=0x7f85200d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.285+0000 7f851ffff640 1 -- 172.21.15.19:0/3971061038 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85200d1c50 con 0x7f85200d1020 2024-04-25T09:53:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.285+0000 7f851effd640 1 --2- 172.21.15.19:0/3971061038 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200b2cb0 0x7f85200d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.285+0000 7f851ffff640 1 --2- 172.21.15.19:0/3971061038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85200d1020 0x7f85200d2c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8518009d90 tx=0x7f851802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca8988bcecd8495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.285+0000 7f851e7fc640 1 -- 172.21.15.19:0/3971061038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518007740 con 0x7f85200d1020 2024-04-25T09:53:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.285+0000 7f851e7fc640 1 -- 172.21.15.19:0/3971061038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85180078e0 con 0x7f85200d1020 2024-04-25T09:53:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.285+0000 7f851e7fc640 1 -- 172.21.15.19:0/3971061038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518007cb0 con 0x7f85200d1020 2024-04-25T09:53:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.286+0000 7f852622a640 1 -- 172.21.15.19:0/3971061038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85200d1020 msgr2=0x7f85200d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.286+0000 7f852622a640 1 --2- 172.21.15.19:0/3971061038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85200d1020 0x7f85200d2c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8518009d90 tx=0x7f851802a660 comp rx=0 tx=0).stop 2024-04-25T09:53:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.286+0000 7f852622a640 1 -- 172.21.15.19:0/3971061038 shutdown_connections 2024-04-25T09:53:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.286+0000 7f852622a640 1 --2- 172.21.15.19:0/3971061038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85200d1020 0x7f85200d2c70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.286+0000 7f852622a640 1 --2- 172.21.15.19:0/3971061038 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200b2cb0 0x7f85200d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.286+0000 7f852622a640 1 -- 172.21.15.19:0/3971061038 >> 172.21.15.19:0/3971061038 conn(0x7f8520061a20 msgr2=0x7f8520061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.286+0000 7f852622a640 1 -- 172.21.15.19:0/3971061038 shutdown_connections 2024-04-25T09:53:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.286+0000 7f852622a640 1 -- 172.21.15.19:0/3971061038 wait complete. 2024-04-25T09:53:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.287+0000 7f852622a640 1 Processor -- start 2024-04-25T09:53:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.287+0000 7f852622a640 1 -- start start 2024-04-25T09:53:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.288+0000 7f852622a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85200b2cb0 0x7f85201f0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.288+0000 7f852622a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200d1020 0x7f8520156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.288+0000 7f852622a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f852015dbe0 0x7f85201f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.288+0000 7f852622a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85200b92a0 con 0x7f85200b2cb0 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.288+0000 7f852622a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85200d2210 con 0x7f852015dbe0 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.288+0000 7f852622a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85200b9420 con 0x7f85200d1020 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.288+0000 7f851f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85200b2cb0 0x7f85201f0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.288+0000 7f851f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85200b2cb0 0x7f85201f0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43566/0 (socket says 172.21.15.19:43566) 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.288+0000 7f851f7fe640 1 -- 172.21.15.19:0/62900892 learned_addr learned my addr 172.21.15.19:0/62900892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.288+0000 7f851effd640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200d1020 0x7f8520156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.288+0000 7f851ffff640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f852015dbe0 0x7f85201f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f851f7fe640 1 -- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200d1020 msgr2=0x7f8520156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f851f7fe640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200d1020 0x7f8520156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f851f7fe640 1 -- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f852015dbe0 msgr2=0x7f85201f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f851f7fe640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f852015dbe0 0x7f85201f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f851f7fe640 1 -- 172.21.15.19:0/62900892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85201f3eb0 con 0x7f85200b2cb0 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f851effd640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200d1020 0x7f8520156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f851ffff640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f852015dbe0 0x7f85201f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f851f7fe640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85200b2cb0 0x7f85201f0870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f851000c430 tx=0x7f851000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f8507fff640 1 -- 172.21.15.19:0/62900892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100074e0 con 0x7f85200b2cb0 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f8507fff640 1 -- 172.21.15.19:0/62900892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85100047e0 con 0x7f85200b2cb0 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f8507fff640 1 -- 172.21.15.19:0/62900892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510004bd0 con 0x7f85200b2cb0 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f852622a640 1 -- 172.21.15.19:0/62900892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85201f4190 con 0x7f85200b2cb0 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.289+0000 7f852622a640 1 -- 172.21.15.19:0/62900892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8520156030 con 0x7f85200b2cb0 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.290+0000 7f852622a640 1 -- 172.21.15.19:0/62900892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8520073990 con 0x7f85200b2cb0 2024-04-25T09:53:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.290+0000 7f8507fff640 1 -- 172.21.15.19:0/62900892 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8510011070 con 0x7f85200b2cb0 2024-04-25T09:53:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.291+0000 7f8507fff640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f850c06b4d0 0x7f850c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.291+0000 7f8507fff640 1 -- 172.21.15.19:0/62900892 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f851008a0e0 con 0x7f85200b2cb0 2024-04-25T09:53:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.291+0000 7f851effd640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f850c06b4d0 0x7f850c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.291+0000 7f851effd640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f850c06b4d0 0x7f850c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f852015c0b0 tx=0x7f8514008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.295+0000 7f8507fff640 1 -- 172.21.15.19:0/62900892 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f851000e280 con 0x7f85200b2cb0 2024-04-25T09:53:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.481+0000 7f852622a640 1 -- 172.21.15.19:0/62900892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85201f4360 con 0x7f85200b2cb0 2024-04-25T09:53:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.482+0000 7f8507fff640 1 -- 172.21.15.19:0/62900892 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f8510058cb0 con 0x7f85200b2cb0 2024-04-25T09:53:13.485 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:13.485 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.484+0000 7f852622a640 1 -- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f850c06b4d0 msgr2=0x7f850c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.484+0000 7f852622a640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f850c06b4d0 0x7f850c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f852015c0b0 tx=0x7f8514008040 comp rx=0 tx=0).stop 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.484+0000 7f852622a640 1 -- 172.21.15.19:0/62900892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85200b2cb0 msgr2=0x7f85201f0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.484+0000 7f852622a640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85200b2cb0 0x7f85201f0870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f851000c430 tx=0x7f851000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.484+0000 7f852622a640 1 -- 172.21.15.19:0/62900892 shutdown_connections 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.484+0000 7f852622a640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f850c06b4d0 0x7f850c06d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.484+0000 7f852622a640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f852015dbe0 0x7f85201f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.484+0000 7f852622a640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85200d1020 0x7f8520156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.484+0000 7f852622a640 1 --2- 172.21.15.19:0/62900892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85200b2cb0 0x7f85201f0870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.484+0000 7f852622a640 1 -- 172.21.15.19:0/62900892 >> 172.21.15.19:0/62900892 conn(0x7f8520061a20 msgr2=0x7f85200b6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.485+0000 7f852622a640 1 -- 172.21.15.19:0/62900892 shutdown_connections 2024-04-25T09:53:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.485+0000 7f852622a640 1 -- 172.21.15.19:0/62900892 wait complete. 2024-04-25T09:53:13.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:13.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.596+0000 7fb96684d640 1 Processor -- start 2024-04-25T09:53:13.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.596+0000 7fb96684d640 1 -- start start 2024-04-25T09:53:13.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb96684d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600b2cb0 0x7fb9600d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb96684d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9600b97a0 con 0x7fb9600d1020 2024-04-25T09:53:13.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb96684d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9600d1c50 con 0x7fb96015dbe0 2024-04-25T09:53:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb96684d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96015dfc0 con 0x7fb9600b2cb0 2024-04-25T09:53:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb964dea640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb9600d1020 0x7fb9600d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48178/0 (socket says 172.21.15.19:48178) 2024-04-25T09:53:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb964dea640 1 -- 172.21.15.19:0/2035811507 learned_addr learned my addr 172.21.15.19:0/2035811507 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb95f7fe640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600b2cb0 0x7fb9600d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980162037 0 0) 0x7fb9600b97a0 con 0x7fb9600d1020 2024-04-25T09:53:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb94c003520 con 0x7fb9600d1020 2024-04-25T09:53:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 469336642 0 0) 0x7fb94c003520 con 0x7fb9600d1020 2024-04-25T09:53:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600b2cb0 msgr2=0x7fb9600d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb95effd640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600b2cb0 0x7fb9600d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 >> v1:172.21.15.112:6789/0 conn(0x7fb96015dbe0 legacy=0x7fb9600d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9600d1f30 con 0x7fb9600d1020 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fb9580019f0 con 0x7fb9600d1020 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.597+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 >> v1:172.21.15.19:6789/0 conn(0x7fb9600d1020 legacy=0x7fb9600d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95effd640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 0x7fb94c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95effd640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb94c004390 0x7fb94c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95effd640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600b2cb0 0x7fb94c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c00b9c0 con 0x7fb94c0039e0 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c00bb40 con 0x7fb94c004390 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c00bd40 con 0x7fb9600b2cb0 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95ffff640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 0x7fb94c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95f7fe640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb94c004390 0x7fb94c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb964dea640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600b2cb0 0x7fb94c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95ffff640 1 -- 172.21.15.19:0/2035811507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600b2cb0 msgr2=0x7fb94c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95ffff640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600b2cb0 0x7fb94c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95ffff640 1 -- 172.21.15.19:0/2035811507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb94c004390 msgr2=0x7fb94c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95ffff640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb94c004390 0x7fb94c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.598+0000 7fb95ffff640 1 -- 172.21.15.19:0/2035811507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c003520 con 0x7fb94c0039e0 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.599+0000 7fb95ffff640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 0x7fb94c003dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb950001860 tx=0x7fb94c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46010565420f8c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.599+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950037070 con 0x7fb94c0039e0 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.599+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95002ac30 con 0x7fb94c0039e0 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.599+0000 7fb95effd640 1 -- 172.21.15.19:0/2035811507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950001430 con 0x7fb94c0039e0 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.599+0000 7fb96684d640 1 -- 172.21.15.19:0/2035811507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 msgr2=0x7fb94c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.599+0000 7fb96684d640 1 --2- 172.21.15.19:0/2035811507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 0x7fb94c003dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb950001860 tx=0x7fb94c005830 comp rx=0 tx=0).stop 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.599+0000 7fb964dea640 1 -- 172.21.15.19:0/2035811507 reap_dead start 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.599+0000 7fb96684d640 1 -- 172.21.15.19:0/2035811507 shutdown_connections 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.599+0000 7fb96684d640 1 -- 172.21.15.19:0/2035811507 >> 172.21.15.19:0/2035811507 conn(0x7fb960061a20 msgr2=0x7fb960061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:13.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.599+0000 7fb96684d640 1 -- 172.21.15.19:0/2035811507 shutdown_connections 2024-04-25T09:53:13.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.599+0000 7fb96684d640 1 -- 172.21.15.19:0/2035811507 wait complete. 2024-04-25T09:53:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.601+0000 7fb96684d640 1 Processor -- start 2024-04-25T09:53:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.601+0000 7fb96684d640 1 -- start start 2024-04-25T09:53:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.601+0000 7fb96684d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 0x7fb9600cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.601+0000 7fb96684d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9601556a0 0x7fb960148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.601+0000 7fb96684d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600c94a0 0x7fb9600cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.601+0000 7fb96684d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600b92a0 con 0x7fb94c0039e0 2024-04-25T09:53:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.601+0000 7fb96684d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600d1a00 con 0x7fb9601556a0 2024-04-25T09:53:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.601+0000 7fb96684d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600b9420 con 0x7fb9600c94a0 2024-04-25T09:53:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.601+0000 7fb95ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 0x7fb9600cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 0x7fb9600cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43594/0 (socket says 172.21.15.19:43594) 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95ffff640 1 -- 172.21.15.19:0/828614918 learned_addr learned my addr 172.21.15.19:0/828614918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95f7fe640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9601556a0 0x7fb960148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb964dea640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600c94a0 0x7fb9600cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95ffff640 1 -- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600c94a0 msgr2=0x7fb9600cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95ffff640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600c94a0 0x7fb9600cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95ffff640 1 -- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9601556a0 msgr2=0x7fb960148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95ffff640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9601556a0 0x7fb960148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95ffff640 1 -- 172.21.15.19:0/828614918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb960155ba0 con 0x7fb94c0039e0 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95f7fe640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9601556a0 0x7fb960148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95ffff640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 0x7fb9600cd260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb95002a6c0 tx=0x7fb950001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95cff9640 1 -- 172.21.15.19:0/828614918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950040020 con 0x7fb94c0039e0 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95cff9640 1 -- 172.21.15.19:0/828614918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb950005270 con 0x7fb94c0039e0 2024-04-25T09:53:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb96684d640 1 -- 172.21.15.19:0/828614918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb960155e80 con 0x7fb94c0039e0 2024-04-25T09:53:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb95cff9640 1 -- 172.21.15.19:0/828614918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950037040 con 0x7fb94c0039e0 2024-04-25T09:53:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.602+0000 7fb96684d640 1 -- 172.21.15.19:0/828614918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb960156440 con 0x7fb94c0039e0 2024-04-25T09:53:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.603+0000 7fb96684d640 1 -- 172.21.15.19:0/828614918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb960073990 con 0x7fb94c0039e0 2024-04-25T09:53:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.604+0000 7fb95cff9640 1 -- 172.21.15.19:0/828614918 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb950043020 con 0x7fb94c0039e0 2024-04-25T09:53:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.604+0000 7fb95cff9640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb94006b4d0 0x7fb94006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.604+0000 7fb95cff9640 1 -- 172.21.15.19:0/828614918 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb9500aa5c0 con 0x7fb94c0039e0 2024-04-25T09:53:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.604+0000 7fb95f7fe640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb94006b4d0 0x7fb94006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.605+0000 7fb95f7fe640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb94006b4d0 0x7fb94006d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9600b8f70 tx=0x7fb948008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.608+0000 7fb95cff9640 1 -- 172.21.15.19:0/828614918 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb95004a030 con 0x7fb94c0039e0 2024-04-25T09:53:13.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.788+0000 7fb96684d640 1 -- 172.21.15.19:0/828614918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9600b77c0 con 0x7fb94c0039e0 2024-04-25T09:53:13.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.789+0000 7fb95cff9640 1 -- 172.21.15.19:0/828614918 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fb950079190 con 0x7fb94c0039e0 2024-04-25T09:53:13.792 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:13.792 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:13.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 -- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb94006b4d0 msgr2=0x7fb94006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb94006b4d0 0x7fb94006d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9600b8f70 tx=0x7fb948008040 comp rx=0 tx=0).stop 2024-04-25T09:53:13.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 -- 172.21.15.19:0/828614918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 msgr2=0x7fb9600cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 0x7fb9600cd260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb95002a6c0 tx=0x7fb950001230 comp rx=0 tx=0).stop 2024-04-25T09:53:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 -- 172.21.15.19:0/828614918 shutdown_connections 2024-04-25T09:53:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb94006b4d0 0x7fb94006d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9600c94a0 0x7fb9600cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9601556a0 0x7fb960148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 --2- 172.21.15.19:0/828614918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c0039e0 0x7fb9600cd260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 -- 172.21.15.19:0/828614918 >> 172.21.15.19:0/828614918 conn(0x7fb960061a20 msgr2=0x7fb96015a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 -- 172.21.15.19:0/828614918 shutdown_connections 2024-04-25T09:53:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.791+0000 7fb96684d640 1 -- 172.21.15.19:0/828614918 wait complete. 2024-04-25T09:53:13.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:13.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.922+0000 7f41f208f640 1 Processor -- start 2024-04-25T09:53:13.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.922+0000 7f41f208f640 1 -- start start 2024-04-25T09:53:13.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.923+0000 7f41f208f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41ec15dc70 0x7f41ec0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.923+0000 7f41f208f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f41ec09a450 0x7f41ec0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.923+0000 7f41f208f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ec06c220 con 0x7f41ec06c700 2024-04-25T09:53:13.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.923+0000 7f41f208f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ec09a8b0 con 0x7f41ec09a450 2024-04-25T09:53:13.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.923+0000 7f41f208f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec0d0c90 con 0x7f41ec15dc70 2024-04-25T09:53:13.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.924+0000 7f41eaffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f41ec09a450 0x7f41ec0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.924+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41ec15dc70 0x7f41ec0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.924+0000 7f41ebfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f41ec06c700 0x7f41ec0cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:35872/0 (socket says 172.21.15.19:35872) 2024-04-25T09:53:13.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.924+0000 7f41ebfff640 1 -- 172.21.15.19:0/1466373899 learned_addr learned my addr 172.21.15.19:0/1466373899 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:13.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.924+0000 7f41ea7fc640 1 -- 172.21.15.19:0/1466373899 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416424638 0 0) 0x7f41ec06c220 con 0x7f41ec06c700 2024-04-25T09:53:13.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.924+0000 7f41ea7fc640 1 -- 172.21.15.19:0/1466373899 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41d4003520 con 0x7f41ec06c700 2024-04-25T09:53:13.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.924+0000 7f41eb7fe640 1 -- 172.21.15.19:0/1466373899 >> v2:172.21.15.112:3300/0 conn(0x7f41ec09a450 msgr2=0x7f41ec0cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:13.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.924+0000 7f41eb7fe640 1 --2- 172.21.15.19:0/1466373899 >> v2:172.21.15.112:3300/0 conn(0x7f41ec09a450 0x7f41ec0cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.924+0000 7f41eb7fe640 1 -- 172.21.15.19:0/1466373899 >> v1:172.21.15.112:6789/0 conn(0x7f41ec06c700 legacy=0x7f41ec0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.924+0000 7f41eb7fe640 1 -- 172.21.15.19:0/1466373899 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec0d0080 con 0x7f41ec15dc70 2024-04-25T09:53:13.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.924+0000 7f41eaffd640 1 --2- 172.21.15.19:0/1466373899 >> v2:172.21.15.112:3300/0 conn(0x7f41ec09a450 0x7f41ec0cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:13.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.925+0000 7f41eb7fe640 1 --2- 172.21.15.19:0/1466373899 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41ec15dc70 0x7f41ec0d1120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097a0 tx=0x7f41dc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1037af212eb6d4d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.925+0000 7f41ea7fc640 1 -- 172.21.15.19:0/1466373899 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc037070 con 0x7f41ec15dc70 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.925+0000 7f41ea7fc640 1 -- 172.21.15.19:0/1466373899 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc032440 con 0x7f41ec15dc70 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.925+0000 7f41ea7fc640 1 -- 172.21.15.19:0/1466373899 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc032810 con 0x7f41ec15dc70 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.925+0000 7f41f208f640 1 -- 172.21.15.19:0/1466373899 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41ec15dc70 msgr2=0x7f41ec0d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.925+0000 7f41f208f640 1 --2- 172.21.15.19:0/1466373899 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41ec15dc70 0x7f41ec0d1120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097a0 tx=0x7f41dc02a180 comp rx=0 tx=0).stop 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.925+0000 7f41f208f640 1 -- 172.21.15.19:0/1466373899 shutdown_connections 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.925+0000 7f41f208f640 1 --2- 172.21.15.19:0/1466373899 >> v2:172.21.15.112:3300/0 conn(0x7f41ec09a450 0x7f41ec0cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.925+0000 7f41f208f640 1 --2- 172.21.15.19:0/1466373899 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41ec15dc70 0x7f41ec0d1120 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.925+0000 7f41f208f640 1 -- 172.21.15.19:0/1466373899 >> 172.21.15.19:0/1466373899 conn(0x7f41ec0b3270 msgr2=0x7f41ec0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.926+0000 7f41f208f640 1 -- 172.21.15.19:0/1466373899 shutdown_connections 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.926+0000 7f41f208f640 1 -- 172.21.15.19:0/1466373899 wait complete. 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.927+0000 7f41f208f640 1 Processor -- start 2024-04-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.927+0000 7f41f208f640 1 -- start start 2024-04-25T09:53:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41f208f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41ec06c700 0x7f41ec0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41f208f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec09a450 0x7f41ec1f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41f208f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41ec15dc70 0x7f41ec1f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41f208f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec0d0920 con 0x7f41ec06c700 2024-04-25T09:53:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41f208f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec0d0640 con 0x7f41ec09a450 2024-04-25T09:53:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41f208f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec15e050 con 0x7f41ec15dc70 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41ec06c700 0x7f41ec0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41ec06c700 0x7f41ec0c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43606/0 (socket says 172.21.15.19:43606) 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41eb7fe640 1 -- 172.21.15.19:0/1837964418 learned_addr learned my addr 172.21.15.19:0/1837964418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41ebfff640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41ec15dc70 0x7f41ec1f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41eaffd640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec09a450 0x7f41ec1f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41eb7fe640 1 -- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41ec15dc70 msgr2=0x7f41ec1f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41eb7fe640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41ec15dc70 0x7f41ec1f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41eb7fe640 1 -- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec09a450 msgr2=0x7f41ec1f0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41eb7fe640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec09a450 0x7f41ec1f0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41eb7fe640 1 -- 172.21.15.19:0/1837964418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec0cad10 con 0x7f41ec06c700 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41eb7fe640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41ec06c700 0x7f41ec0c9260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41dc000c00 tx=0x7f41dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41d3fff640 1 -- 172.21.15.19:0/1837964418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc041020 con 0x7f41ec06c700 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41d3fff640 1 -- 172.21.15.19:0/1837964418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc03d640 con 0x7f41ec06c700 2024-04-25T09:53:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41f208f640 1 -- 172.21.15.19:0/1837964418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec0c9840 con 0x7f41ec06c700 2024-04-25T09:53:13.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41d3fff640 1 -- 172.21.15.19:0/1837964418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc037040 con 0x7f41ec06c700 2024-04-25T09:53:13.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.928+0000 7f41f208f640 1 -- 172.21.15.19:0/1837964418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec0c9e00 con 0x7f41ec06c700 2024-04-25T09:53:13.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.930+0000 7f41f208f640 1 -- 172.21.15.19:0/1837964418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ec073990 con 0x7f41ec06c700 2024-04-25T09:53:13.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.930+0000 7f41d3fff640 1 -- 172.21.15.19:0/1837964418 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41dc044020 con 0x7f41ec06c700 2024-04-25T09:53:13.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.931+0000 7f41d3fff640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41d806b500 0x7f41d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.931+0000 7f41d3fff640 1 -- 172.21.15.19:0/1837964418 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f41dc0aaab0 con 0x7f41ec06c700 2024-04-25T09:53:13.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.931+0000 7f41eaffd640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41d806b500 0x7f41d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.931+0000 7f41eaffd640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41d806b500 0x7f41d806d9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009560 tx=0x7f41e0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:13.935+0000 7f41d3fff640 1 -- 172.21.15.19:0/1837964418 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41dc04b030 con 0x7f41ec06c700 2024-04-25T09:53:14.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.158+0000 7f41f208f640 1 -- 172.21.15.19:0/1837964418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41ec0b40a0 con 0x7f41ec06c700 2024-04-25T09:53:14.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.158+0000 7f41d3fff640 1 -- 172.21.15.19:0/1837964418 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f41dc078fb0 con 0x7f41ec06c700 2024-04-25T09:53:14.161 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:14.161 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4000846111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":4000846111},{"type":"v1","addr":"172.21.15.112:6837","nonce":4000846111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1969525282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1969525282},{"type":"v1","addr":"172.21.15.19:6835","nonce":1969525282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:07.175350+0000","modified":"2024-04-25T09:53:12.181345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1080352556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1080352556},{"type":"v1","addr":"172.21.15.112:6836","nonce":1080352556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T09:53:14.161 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 7 2024-04-25T09:53:14.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 -- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41d806b500 msgr2=0x7f41d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41d806b500 0x7f41d806d9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009560 tx=0x7f41e0009190 comp rx=0 tx=0).stop 2024-04-25T09:53:14.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 -- 172.21.15.19:0/1837964418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41ec06c700 msgr2=0x7f41ec0c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41ec06c700 0x7f41ec0c9260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41dc000c00 tx=0x7f41dc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 -- 172.21.15.19:0/1837964418 shutdown_connections 2024-04-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41d806b500 0x7f41d806d9c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41ec15dc70 0x7f41ec1f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec09a450 0x7f41ec1f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 --2- 172.21.15.19:0/1837964418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41ec06c700 0x7f41ec0c9260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 -- 172.21.15.19:0/1837964418 >> 172.21.15.19:0/1837964418 conn(0x7f41ec0b3270 msgr2=0x7f41ec0b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 -- 172.21.15.19:0/1837964418 shutdown_connections 2024-04-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.160+0000 7f41f208f640 1 -- 172.21.15.19:0/1837964418 wait complete. 2024-04-25T09:53:14.173 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.271+0000 7fc36b589640 1 Processor -- start 2024-04-25T09:53:14.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.272+0000 7fc36b589640 1 -- start start 2024-04-25T09:53:14.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.272+0000 7fc36b589640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3640701e0 0x7fc3640cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.272+0000 7fc36b589640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3640cbbf0 0x7fc3640cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.272+0000 7fc36b589640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36406f6c0 con 0x7fc36415df00 2024-04-25T09:53:14.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.272+0000 7fc36b589640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36406f950 con 0x7fc3640cbbf0 2024-04-25T09:53:14.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.272+0000 7fc36b589640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640cdf00 con 0x7fc3640701e0 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.273+0000 7fc369b26640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3640cbbf0 0x7fc3640cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.273+0000 7fc369b26640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3640cbbf0 0x7fc3640cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43622/0 (socket says 172.21.15.19:43622) 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.273+0000 7fc369b26640 1 -- 172.21.15.19:0/1757840389 learned_addr learned my addr 172.21.15.19:0/1757840389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.273+0000 7fc368b24640 1 --2- 172.21.15.19:0/1757840389 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3640701e0 0x7fc3640cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.273+0000 7fc369b26640 1 -- 172.21.15.19:0/1757840389 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3640701e0 msgr2=0x7fc3640cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.273+0000 7fc369b26640 1 --2- 172.21.15.19:0/1757840389 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3640701e0 0x7fc3640cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.273+0000 7fc369b26640 1 -- 172.21.15.19:0/1757840389 >> v1:172.21.15.112:6789/0 conn(0x7fc36415df00 legacy=0x7fc3640cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.273+0000 7fc369b26640 1 -- 172.21.15.19:0/1757840389 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640cece0 con 0x7fc3640cbbf0 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.273+0000 7fc368b24640 1 --2- 172.21.15.19:0/1757840389 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3640701e0 0x7fc3640cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc369b26640 1 --2- 172.21.15.19:0/1757840389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640cbbf0 0x7fc3640cc080 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009e50 tx=0x7fc35c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b721f9db5421dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc35bfff640 1 -- 172.21.15.19:0/1757840389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c036070 con 0x7fc3640cbbf0 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc35bfff640 1 -- 172.21.15.19:0/1757840389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c031600 con 0x7fc3640cbbf0 2024-04-25T09:53:14.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc35bfff640 1 -- 172.21.15.19:0/1757840389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0319d0 con 0x7fc3640cbbf0 2024-04-25T09:53:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc36b589640 1 -- 172.21.15.19:0/1757840389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640cbbf0 msgr2=0x7fc3640cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc36b589640 1 --2- 172.21.15.19:0/1757840389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640cbbf0 0x7fc3640cc080 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009e50 tx=0x7fc35c02a6c0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc36b589640 1 -- 172.21.15.19:0/1757840389 shutdown_connections 2024-04-25T09:53:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc36b589640 1 --2- 172.21.15.19:0/1757840389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640cbbf0 0x7fc3640cc080 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc36b589640 1 --2- 172.21.15.19:0/1757840389 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3640701e0 0x7fc3640cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc36b589640 1 -- 172.21.15.19:0/1757840389 >> 172.21.15.19:0/1757840389 conn(0x7fc3640b2290 msgr2=0x7fc3640b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc36b589640 1 -- 172.21.15.19:0/1757840389 shutdown_connections 2024-04-25T09:53:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.274+0000 7fc36b589640 1 -- 172.21.15.19:0/1757840389 wait complete. 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.275+0000 7fc36b589640 1 Processor -- start 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.276+0000 7fc36b589640 1 -- start start 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.276+0000 7fc36b589640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640701e0 0x7fc3640c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.276+0000 7fc36b589640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3640cbbf0 0x7fc364151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.276+0000 7fc36b589640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc36415df00 0x7fc364157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.276+0000 7fc36b589640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36406fd00 con 0x7fc3640701e0 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.276+0000 7fc36b589640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640cf4e0 con 0x7fc3640cbbf0 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.276+0000 7fc36b589640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36406fe80 con 0x7fc36415df00 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.276+0000 7fc369325640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640701e0 0x7fc3640c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.276+0000 7fc369325640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640701e0 0x7fc3640c40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43634/0 (socket says 172.21.15.19:43634) 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.276+0000 7fc369325640 1 -- 172.21.15.19:0/3693158488 learned_addr learned my addr 172.21.15.19:0/3693158488 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:14.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.277+0000 7fc369325640 1 -- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc36415df00 msgr2=0x7fc364157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.277+0000 7fc369325640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc36415df00 0x7fc364157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.277+0000 7fc369325640 1 -- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3640cbbf0 msgr2=0x7fc364151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.277+0000 7fc369325640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3640cbbf0 0x7fc364151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.277+0000 7fc369325640 1 -- 172.21.15.19:0/3693158488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640c4760 con 0x7fc3640701e0 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.277+0000 7fc369325640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640701e0 0x7fc3640c40d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc36000b420 tx=0x7fc36000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.277+0000 7fc359ffb640 1 -- 172.21.15.19:0/3693158488 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360011830 con 0x7fc3640701e0 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.277+0000 7fc359ffb640 1 -- 172.21.15.19:0/3693158488 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc360016070 con 0x7fc3640701e0 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.277+0000 7fc359ffb640 1 -- 172.21.15.19:0/3693158488 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360011c10 con 0x7fc3640701e0 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.277+0000 7fc36b589640 1 -- 172.21.15.19:0/3693158488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3640c4a40 con 0x7fc3640701e0 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.277+0000 7fc36b589640 1 -- 172.21.15.19:0/3693158488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3640c5000 con 0x7fc3640701e0 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.278+0000 7fc36b589640 1 -- 172.21.15.19:0/3693158488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc364073990 con 0x7fc3640701e0 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.278+0000 7fc359ffb640 1 -- 172.21.15.19:0/3693158488 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc36001b460 con 0x7fc3640701e0 2024-04-25T09:53:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.279+0000 7fc359ffb640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc34806b4d0 0x7fc34806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.279+0000 7fc359ffb640 1 -- 172.21.15.19:0/3693158488 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc360089f90 con 0x7fc3640701e0 2024-04-25T09:53:14.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.279+0000 7fc368b24640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc34806b4d0 0x7fc34806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.280+0000 7fc368b24640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc34806b4d0 0x7fc34806d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc354006fd0 tx=0x7fc354008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:14.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.283+0000 7fc359ffb640 1 -- 172.21.15.19:0/3693158488 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc360020020 con 0x7fc3640701e0 2024-04-25T09:53:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.464+0000 7fc36b589640 1 -- 172.21.15.19:0/3693158488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3640b28a0 con 0x7fc3640701e0 2024-04-25T09:53:14.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.465+0000 7fc359ffb640 1 -- 172.21.15.19:0/3693158488 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fc360058b60 con 0x7fc3640701e0 2024-04-25T09:53:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:14.468 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:14.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 -- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc34806b4d0 msgr2=0x7fc34806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc34806b4d0 0x7fc34806d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc354006fd0 tx=0x7fc354008040 comp rx=0 tx=0).stop 2024-04-25T09:53:14.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 -- 172.21.15.19:0/3693158488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640701e0 msgr2=0x7fc3640c40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640701e0 0x7fc3640c40d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc36000b420 tx=0x7fc36000b8e0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 -- 172.21.15.19:0/3693158488 shutdown_connections 2024-04-25T09:53:14.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc34806b4d0 0x7fc34806d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc36415df00 0x7fc364157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3640cbbf0 0x7fc364151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 --2- 172.21.15.19:0/3693158488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640701e0 0x7fc3640c40d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 -- 172.21.15.19:0/3693158488 >> 172.21.15.19:0/3693158488 conn(0x7fc3640b2290 msgr2=0x7fc36415bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:14.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 -- 172.21.15.19:0/3693158488 shutdown_connections 2024-04-25T09:53:14.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.467+0000 7fc36b589640 1 -- 172.21.15.19:0/3693158488 wait complete. 2024-04-25T09:53:14.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:53:07.175350+0000', 'modified': '2024-04-25T09:53:12.181345+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4300': {'gid': 4300, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6836/1080352556', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 1080352556}, {'type': 'v1', 'addr': '172.21.15.112:6836', 'nonce': 1080352556}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T09:53:14.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:53:14.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:53:14.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.987+0000 7f298f1ce640 1 Processor -- start 2024-04-25T09:53:14.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.987+0000 7f298f1ce640 1 -- start start 2024-04-25T09:53:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.987+0000 7f298f1ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298815b7d0 0x7f29880cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.987+0000 7f298f1ce640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f29880b5620 0x7f29880b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.987+0000 7f298f1ce640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298806cf50 con 0x7f298806c2d0 2024-04-25T09:53:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.987+0000 7f298f1ce640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29880b5fc0 con 0x7f29880b5620 2024-04-25T09:53:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.987+0000 7f298f1ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29880ce530 con 0x7f298815b7d0 2024-04-25T09:53:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.988+0000 7f2987fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f29880b5620 0x7f29880b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.988+0000 7f298d76b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f298806c2d0 0x7f298806c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48182/0 (socket says 172.21.15.19:48182) 2024-04-25T09:53:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.988+0000 7f298d76b640 1 -- 172.21.15.19:0/1002318533 learned_addr learned my addr 172.21.15.19:0/1002318533 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.988+0000 7f298cf6a640 1 --2- 172.21.15.19:0/1002318533 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298815b7d0 0x7f29880cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.988+0000 7f2987fff640 1 -- 172.21.15.19:0/1002318533 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298815b7d0 msgr2=0x7f29880cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:14.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.988+0000 7f2987fff640 1 --2- 172.21.15.19:0/1002318533 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298815b7d0 0x7f29880cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.988+0000 7f2987fff640 1 -- 172.21.15.19:0/1002318533 >> v1:172.21.15.19:6789/0 conn(0x7f298806c2d0 legacy=0x7f298806c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.988+0000 7f2987fff640 1 -- 172.21.15.19:0/1002318533 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29880ce810 con 0x7f29880b5620 2024-04-25T09:53:14.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.988+0000 7f29877fe640 1 -- 172.21.15.19:0/1002318533 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176654971 0 0) 0x7f298806cf50 con 0x7f298806c2d0 2024-04-25T09:53:14.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.988+0000 7f298cf6a640 1 --2- 172.21.15.19:0/1002318533 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298815b7d0 0x7f29880cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:14.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f2987fff640 1 --2- 172.21.15.19:0/1002318533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29880b5620 0x7f29880b5a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f29740097a0 tx=0x7f297402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bf496bd9dee70e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:14.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f29877fe640 1 -- 172.21.15.19:0/1002318533 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974037070 con 0x7f29880b5620 2024-04-25T09:53:14.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f29877fe640 1 -- 172.21.15.19:0/1002318533 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2974032440 con 0x7f29880b5620 2024-04-25T09:53:14.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f29877fe640 1 -- 172.21.15.19:0/1002318533 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974032810 con 0x7f29880b5620 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f298f1ce640 1 -- 172.21.15.19:0/1002318533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29880b5620 msgr2=0x7f29880b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f298f1ce640 1 --2- 172.21.15.19:0/1002318533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29880b5620 0x7f29880b5a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f29740097a0 tx=0x7f297402a180 comp rx=0 tx=0).stop 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f298f1ce640 1 -- 172.21.15.19:0/1002318533 shutdown_connections 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f298f1ce640 1 --2- 172.21.15.19:0/1002318533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29880b5620 0x7f29880b5a80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f298f1ce640 1 --2- 172.21.15.19:0/1002318533 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298815b7d0 0x7f29880cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f298f1ce640 1 -- 172.21.15.19:0/1002318533 >> 172.21.15.19:0/1002318533 conn(0x7f29880d0800 msgr2=0x7f29880d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f298f1ce640 1 -- 172.21.15.19:0/1002318533 shutdown_connections 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.989+0000 7f298f1ce640 1 -- 172.21.15.19:0/1002318533 wait complete. 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.990+0000 7f298f1ce640 1 Processor -- start 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.991+0000 7f298f1ce640 1 -- start start 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.991+0000 7f298f1ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298806c2d0 0x7f2988159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.991+0000 7f298f1ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29880b5620 0x7f29880cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.991+0000 7f298f1ce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298815b7d0 0x7f29880b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.991+0000 7f298f1ce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29880cf550 con 0x7f298815b7d0 2024-04-25T09:53:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.991+0000 7f298f1ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298806d230 con 0x7f29880b5620 2024-04-25T09:53:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.991+0000 7f298f1ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29880cf6d0 con 0x7f298806c2d0 2024-04-25T09:53:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.991+0000 7f298d76b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298815b7d0 0x7f29880b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.991+0000 7f298d76b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298815b7d0 0x7f29880b8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43646/0 (socket says 172.21.15.19:43646) 2024-04-25T09:53:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.991+0000 7f298d76b640 1 -- 172.21.15.19:0/3726825186 learned_addr learned my addr 172.21.15.19:0/3726825186 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f298cf6a640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298806c2d0 0x7f2988159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f2987fff640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29880b5620 0x7f29880cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f298d76b640 1 -- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298806c2d0 msgr2=0x7f2988159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f298d76b640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298806c2d0 0x7f2988159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f298d76b640 1 -- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29880b5620 msgr2=0x7f29880cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f298d76b640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29880b5620 0x7f29880cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f298d76b640 1 -- 172.21.15.19:0/3726825186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29880cae10 con 0x7f298815b7d0 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f298d76b640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298815b7d0 0x7f29880b8120 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f298000c830 tx=0x7f298000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f29857fa640 1 -- 172.21.15.19:0/3726825186 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980018070 con 0x7f298815b7d0 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f29857fa640 1 -- 172.21.15.19:0/3726825186 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298000ce20 con 0x7f298815b7d0 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f298f1ce640 1 -- 172.21.15.19:0/3726825186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29880cb0b0 con 0x7f298815b7d0 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f29857fa640 1 -- 172.21.15.19:0/3726825186 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298000a880 con 0x7f298815b7d0 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.992+0000 7f298f1ce640 1 -- 172.21.15.19:0/3726825186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29880cb670 con 0x7f298815b7d0 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.994+0000 7f29857fa640 1 -- 172.21.15.19:0/3726825186 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f298001b070 con 0x7f298815b7d0 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.994+0000 7f29857fa640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f296c06b4d0 0x7f296c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.994+0000 7f29857fa640 1 -- 172.21.15.19:0/3726825186 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f29800899d0 con 0x7f298815b7d0 2024-04-25T09:53:15.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.994+0000 7f298cf6a640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f296c06b4d0 0x7f296c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.995+0000 7f298cf6a640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f296c06b4d0 0x7f296c06d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f297c00a950 tx=0x7f297c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.995+0000 7f2991fab740 1 Processor -- start 2024-04-25T09:53:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.995+0000 7f2991fab740 1 -- start start 2024-04-25T09:53:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f2991fab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9ae230 0x55806a9c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f2991fab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a9cc070 0x55806a9ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f2991fab740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f2991fab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806a8e6a10 con 0x55806a9ae230 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f2991fab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806a88fb30 con 0x55806a9cc070 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f2991fab740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55806a8e2c50 con 0x55806a9bf3f0 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f298cf6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9ae230 0x55806a9c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f298cf6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9ae230 0x55806a9c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43658/0 (socket says 172.21.15.19:43658) 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f298cf6a640 1 -- 172.21.15.19:0/3571969915 learned_addr learned my addr 172.21.15.19:0/3571969915 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f2987fff640 1 --2- 172.21.15.19:0/3571969915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a9cc070 0x55806a9ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f298d76b640 1 --2- 172.21.15.19:0/3571969915 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f298cf6a640 1 -- 172.21.15.19:0/3571969915 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 msgr2=0x55806a9bf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f298cf6a640 1 --2- 172.21.15.19:0/3571969915 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9bf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f298cf6a640 1 -- 172.21.15.19:0/3571969915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a9cc070 msgr2=0x55806a9ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f298cf6a640 1 --2- 172.21.15.19:0/3571969915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a9cc070 0x55806a9ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f298cf6a640 1 -- 172.21.15.19:0/3571969915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806a8a4e10 con 0x55806a9ae230 2024-04-25T09:53:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f2987fff640 1 --2- 172.21.15.19:0/3571969915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a9cc070 0x55806a9ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f298cf6a640 1 --2- 172.21.15.19:0/3571969915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9ae230 0x55806a9c9c10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f297c00ebd0 tx=0x7f297c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb31144219c90a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f29727fc640 1 -- 172.21.15.19:0/3571969915 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c01a050 con 0x55806a9ae230 2024-04-25T09:53:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f29727fc640 1 -- 172.21.15.19:0/3571969915 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c024070 con 0x55806a9ae230 2024-04-25T09:53:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.996+0000 7f29727fc640 1 -- 172.21.15.19:0/3571969915 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c01f440 con 0x55806a9ae230 2024-04-25T09:53:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.997+0000 7f2991fab740 1 -- 172.21.15.19:0/3571969915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9ae230 msgr2=0x55806a9c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.997+0000 7f2991fab740 1 --2- 172.21.15.19:0/3571969915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9ae230 0x55806a9c9c10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f297c00ebd0 tx=0x7f297c011000 comp rx=0 tx=0).stop 2024-04-25T09:53:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.997+0000 7f2991fab740 1 -- 172.21.15.19:0/3571969915 shutdown_connections 2024-04-25T09:53:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.997+0000 7f2991fab740 1 --2- 172.21.15.19:0/3571969915 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9bf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.997+0000 7f2991fab740 1 --2- 172.21.15.19:0/3571969915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a9cc070 0x55806a9ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.997+0000 7f2991fab740 1 --2- 172.21.15.19:0/3571969915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9ae230 0x55806a9c9c10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.997+0000 7f2991fab740 1 -- 172.21.15.19:0/3571969915 >> 172.21.15.19:0/3571969915 conn(0x55806a980840 msgr2=0x55806a9c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.997+0000 7f2991fab740 1 -- 172.21.15.19:0/3571969915 shutdown_connections 2024-04-25T09:53:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.997+0000 7f2991fab740 1 -- 172.21.15.19:0/3571969915 wait complete. 2024-04-25T09:53:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.998+0000 7f2991fab740 1 -- start start 2024-04-25T09:53:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.998+0000 7f2991fab740 1 Processor -- start 2024-04-25T09:53:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2991fab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9bf3f0 0x55806a9a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2991fab740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9cc070 0x55806a9bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2991fab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a981de0 0x55806a982260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2991fab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806a9abe60 con 0x55806a9bf3f0 2024-04-25T09:53:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2991fab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806a8a30c0 con 0x55806a981de0 2024-04-25T09:53:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2991fab740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55806a97e670 con 0x55806a9cc070 2024-04-25T09:53:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2991fab740 10 client.0 _ll_register_callbacks cb 0x55806a896ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:53:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f298cf6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9bf3f0 0x55806a9a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f298cf6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9bf3f0 0x55806a9a8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43662/0 (socket says 172.21.15.19:43662) 2024-04-25T09:53:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f298cf6a640 1 -- 172.21.15.19:0/440885267 learned_addr learned my addr 172.21.15.19:0/440885267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f298d76b640 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a981de0 0x55806a982260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2987fff640 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9cc070 0x55806a9bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f298cf6a640 1 -- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9cc070 msgr2=0x55806a9bf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f298cf6a640 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9cc070 0x55806a9bf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f298cf6a640 1 -- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a981de0 msgr2=0x55806a982260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f298cf6a640 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a981de0 0x55806a982260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f298cf6a640 1 -- 172.21.15.19:0/440885267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55806a5fb2d0 con 0x55806a9bf3f0 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f298d76b640 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a981de0 0x55806a982260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2987fff640 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9cc070 0x55806a9bf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f298cf6a640 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9bf3f0 0x55806a9a8b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f297c008010 tx=0x7f297c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b11ab077da1fcbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f29677fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f29677fe640 1 -- 172.21.15.19:0/440885267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c006200 con 0x55806a9bf3f0 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f29677fe640 1 -- 172.21.15.19:0/440885267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f297c0063a0 con 0x55806a9bf3f0 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2991fab740 1 -- 172.21.15.19:0/440885267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f297c024070 con 0x55806a9bf3f0 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f29677fe640 1 -- 172.21.15.19:0/440885267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f297c039050 con 0x55806a9bf3f0 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2991fab740 10 client.4410 fetch_fsmap learned FSMap version 7 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:14.999+0000 7f2991fab740 1 -- 172.21.15.19:0/440885267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55806a6c56b0 con 0x55806a9bf3f0 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.000+0000 7f29677fe640 1 -- 172.21.15.19:0/440885267 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c024070 con 0x55806a9bf3f0 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.000+0000 7f29677fe640 1 -- 172.21.15.19:0/440885267 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f297c03c020 con 0x55806a9bf3f0 2024-04-25T09:53:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.000+0000 7f2991fab740 10 client.4410 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.000+0000 7f2991fab740 10 client.4410 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.000+0000 7f2991fab740 20 client.4410 populate_metadata read hostname 'smithi019' 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.000+0000 7f2991fab740 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55806a9c1e70 0x55806a9c2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.000+0000 7f2991fab740 4 client.4410 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.000+0000 7f2991fab740 1 -- 172.21.15.19:0/440885267 --> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55806a945920 con 0x55806a9c1e70 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.000+0000 7f2987fff640 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55806a9c1e70 0x55806a9c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.000+0000 7f29677fe640 10 client.4410 ms_handle_connect on v2:172.21.15.112:6834/1080352556 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.000+0000 7f2987fff640 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55806a9c1e70 0x55806a9c2250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983a1ec7be8fa581 server_cookie=d14a4e37858469b6 in_seq=0 out_seq=0 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f29677fe640 1 -- 172.21.15.19:0/440885267 <== mds.0 v2:172.21.15.112:6834/1080352556 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55806a945920 con 0x55806a9c1e70 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f29677fe640 10 client.4410 handle_command_reply: tid=0 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f2991fab740 1 client.4410 shutdown 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f2991fab740 1 -- 172.21.15.19:0/440885267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9bf3f0 msgr2=0x55806a9a8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f2991fab740 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9bf3f0 0x55806a9a8b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f297c008010 tx=0x7f297c004730 comp rx=0 tx=0).stop 2024-04-25T09:53:15.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f2991fab740 1 -- 172.21.15.19:0/440885267 shutdown_connections 2024-04-25T09:53:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f2991fab740 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55806a9c1e70 0x55806a9c2250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f2991fab740 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a981de0 0x55806a982260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f29677fe640 0 client.4410 ms_handle_reset on v2:172.21.15.112:6834/1080352556 2024-04-25T09:53:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f2991fab740 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9cc070 0x55806a9bf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f2991fab740 1 --2- 172.21.15.19:0/440885267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9bf3f0 0x55806a9a8b90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f2991fab740 1 -- 172.21.15.19:0/440885267 >> 172.21.15.19:0/440885267 conn(0x55806a9ae230 msgr2=0x55806a97dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f2991fab740 1 -- 172.21.15.19:0/440885267 shutdown_connections 2024-04-25T09:53:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.001+0000 7f2991fab740 1 -- 172.21.15.19:0/440885267 wait complete. 2024-04-25T09:53:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.002+0000 7f2991fab740 20 client.4410 trim_cache size 0 max 16384 2024-04-25T09:53:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.017+0000 7f2991fab740 1 Processor -- start 2024-04-25T09:53:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.017+0000 7f2991fab740 1 -- start start 2024-04-25T09:53:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f2991fab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a981de0 0x55806a9ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f2991fab740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9c2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f2991fab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9cc070 0x55806a9b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f2991fab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2988071430 con 0x55806a9cc070 2024-04-25T09:53:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f2991fab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29880705d0 con 0x55806a981de0 2024-04-25T09:53:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f2991fab740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298806fa20 con 0x55806a9bf3f0 2024-04-25T09:53:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f298d76b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9cc070 0x55806a9b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f298d76b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9cc070 0x55806a9b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43666/0 (socket says 172.21.15.19:43666) 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f298d76b640 1 -- 172.21.15.19:0/1383202250 learned_addr learned my addr 172.21.15.19:0/1383202250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f298d76b640 1 -- 172.21.15.19:0/1383202250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 msgr2=0x55806a9c2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f2987fff640 1 --2- 172.21.15.19:0/1383202250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f298d76b640 1 --2- 172.21.15.19:0/1383202250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9c2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f298d76b640 1 -- 172.21.15.19:0/1383202250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a981de0 msgr2=0x55806a9ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f298d76b640 1 --2- 172.21.15.19:0/1383202250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a981de0 0x55806a9ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f298d76b640 1 -- 172.21.15.19:0/1383202250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806a9b24d0 con 0x55806a9cc070 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.018+0000 7f298d76b640 1 --2- 172.21.15.19:0/1383202250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9cc070 0x55806a9b0f90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2980057c60 tx=0x7f2980089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65172cd514a5a482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2971ffb640 1 -- 172.21.15.19:0/1383202250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980056260 con 0x55806a9cc070 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2971ffb640 1 -- 172.21.15.19:0/1383202250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29800545d0 con 0x55806a9cc070 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2971ffb640 1 -- 172.21.15.19:0/1383202250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298005e270 con 0x55806a9cc070 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2991fab740 1 -- 172.21.15.19:0/1383202250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9cc070 msgr2=0x55806a9b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2991fab740 1 --2- 172.21.15.19:0/1383202250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9cc070 0x55806a9b0f90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2980057c60 tx=0x7f2980089040 comp rx=0 tx=0).stop 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2991fab740 1 -- 172.21.15.19:0/1383202250 shutdown_connections 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2991fab740 1 --2- 172.21.15.19:0/1383202250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a9cc070 0x55806a9b0f90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2991fab740 1 --2- 172.21.15.19:0/1383202250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2991fab740 1 --2- 172.21.15.19:0/1383202250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a981de0 0x55806a9ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2991fab740 1 -- 172.21.15.19:0/1383202250 >> 172.21.15.19:0/1383202250 conn(0x55806a9ae230 msgr2=0x55806a9cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2991fab740 1 -- 172.21.15.19:0/1383202250 shutdown_connections 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.019+0000 7f2991fab740 1 -- 172.21.15.19:0/1383202250 wait complete. 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.020+0000 7f2991fab740 1 -- start start 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.020+0000 7f2991fab740 1 Processor -- start 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f2991fab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a981de0 0x55806aa3c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f2991fab740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f2991fab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a9cc070 0x55806a982a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f2991fab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806a9c0af0 con 0x55806a981de0 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f2991fab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298806ffd0 con 0x55806a9cc070 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f2991fab740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55806a9a9920 con 0x55806a9bf3f0 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f2991fab740 10 client.0 _ll_register_callbacks cb 0x7f297c00a1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f298cf6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a981de0 0x55806aa3c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f298cf6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a981de0 0x55806aa3c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43674/0 (socket says 172.21.15.19:43674) 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f298cf6a640 1 -- 172.21.15.19:0/560352053 learned_addr learned my addr 172.21.15.19:0/560352053 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f2987fff640 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f298d76b640 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a9cc070 0x55806a982a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f298cf6a640 1 -- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 msgr2=0x55806a9c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f298cf6a640 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f298cf6a640 1 -- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a9cc070 msgr2=0x55806a982a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f298cf6a640 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a9cc070 0x55806a982a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f298cf6a640 1 -- 172.21.15.19:0/560352053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55806a9ade70 con 0x55806a981de0 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f298cf6a640 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a981de0 0x55806aa3c2a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f297c00a340 tx=0x7f297c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177839414e0d6b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f29677fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f29677fe640 1 -- 172.21.15.19:0/560352053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c042060 con 0x55806a981de0 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f29677fe640 1 -- 172.21.15.19:0/560352053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f297c01fb60 con 0x55806a981de0 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.021+0000 7f2991fab740 1 -- 172.21.15.19:0/560352053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29800545d0 con 0x55806a981de0 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f29677fe640 1 -- 172.21.15.19:0/560352053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f297c048050 con 0x55806a981de0 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f2991fab740 10 client.4416 fetch_fsmap learned FSMap version 7 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f2991fab740 1 -- 172.21.15.19:0/560352053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55806a9cc7d0 con 0x55806a981de0 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f29677fe640 1 -- 172.21.15.19:0/560352053 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29800545d0 con 0x55806a981de0 2024-04-25T09:53:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f29677fe640 1 -- 172.21.15.19:0/560352053 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f297c04b020 con 0x55806a981de0 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f2991fab740 10 client.4416 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f2991fab740 10 client.4416 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f2991fab740 20 client.4416 populate_metadata read hostname 'smithi019' 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f2991fab740 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55806a9b11f0 0x55806a9df7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f2991fab740 4 client.4416 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f2991fab740 1 -- 172.21.15.19:0/560352053 --> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55806a8f3f50 con 0x55806a9b11f0 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f2987fff640 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55806a9b11f0 0x55806a9df7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f29677fe640 10 client.4416 ms_handle_connect on v2:172.21.15.112:6834/1080352556 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.022+0000 7f2987fff640 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55806a9b11f0 0x55806a9df7a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87bb379e89968616 server_cookie=cff084cb2f7da976 in_seq=0 out_seq=0 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f29677fe640 1 -- 172.21.15.19:0/560352053 <== mds.0 v2:172.21.15.112:6834/1080352556 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55806a8f3f50 con 0x55806a9b11f0 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f29677fe640 10 client.4416 handle_command_reply: tid=0 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 1 client.4416 shutdown 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 1 -- 172.21.15.19:0/560352053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a981de0 msgr2=0x55806aa3c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a981de0 0x55806aa3c2a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f297c00a340 tx=0x7f297c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 1 -- 172.21.15.19:0/560352053 shutdown_connections 2024-04-25T09:53:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55806a9b11f0 0x55806a9df7a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55806a9cc070 0x55806a982a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55806a9bf3f0 0x55806a9c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f29677fe640 0 client.4416 ms_handle_reset on v2:172.21.15.112:6834/1080352556 2024-04-25T09:53:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 1 --2- 172.21.15.19:0/560352053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55806a981de0 0x55806aa3c2a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 1 -- 172.21.15.19:0/560352053 >> 172.21.15.19:0/560352053 conn(0x55806a9ae230 msgr2=0x55806a9a7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 1 -- 172.21.15.19:0/560352053 shutdown_connections 2024-04-25T09:53:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 1 -- 172.21.15.19:0/560352053 wait complete. 2024-04-25T09:53:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f2991fab740 20 client.4416 trim_cache size 0 max 16384 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 4300, 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.2401619620000002, 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 7, 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 24, 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 14.467165612000001 2024-04-25T09:53:15.035 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:53:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f298f1ce640 1 -- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f296c06b4d0 msgr2=0x7f296c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f298f1ce640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f296c06b4d0 0x7f296c06d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f297c00a950 tx=0x7f297c008040 comp rx=0 tx=0).stop 2024-04-25T09:53:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f298f1ce640 1 -- 172.21.15.19:0/3726825186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298815b7d0 msgr2=0x7f29880b8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.023+0000 7f298f1ce640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298815b7d0 0x7f29880b8120 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f298000c830 tx=0x7f298000ccf0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.024+0000 7f298f1ce640 1 -- 172.21.15.19:0/3726825186 shutdown_connections 2024-04-25T09:53:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.024+0000 7f298f1ce640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f296c06b4d0 0x7f296c06d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.024+0000 7f298f1ce640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298815b7d0 0x7f29880b8120 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.024+0000 7f298f1ce640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29880b5620 0x7f29880cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.024+0000 7f298f1ce640 1 --2- 172.21.15.19:0/3726825186 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298806c2d0 0x7f2988159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.024+0000 7f298f1ce640 1 -- 172.21.15.19:0/3726825186 >> 172.21.15.19:0/3726825186 conn(0x7f29880d0800 msgr2=0x7f2988160d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.024+0000 7f298f1ce640 1 -- 172.21.15.19:0/3726825186 shutdown_connections 2024-04-25T09:53:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.024+0000 7f298f1ce640 1 -- 172.21.15.19:0/3726825186 wait complete. 2024-04-25T09:53:15.038 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:53:15.038 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:15.038 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:53:15.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.137+0000 7f85e8607640 1 Processor -- start 2024-04-25T09:53:15.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.137+0000 7f85e8607640 1 -- start start 2024-04-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e8607640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f85e015dbe0 0x7f85e00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e8607640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f85e00b92a0 0x7f85e00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e8607640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e00b97a0 con 0x7f85e00b2cb0 2024-04-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e8607640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e015dfc0 con 0x7f85e00b92a0 2024-04-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e8607640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e00d1a00 con 0x7f85e015dbe0 2024-04-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e6ba4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f85e00b92a0 0x7f85e00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e5ba2640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f85e00b2cb0 0x7f85e00b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48186/0 (socket says 172.21.15.19:48186) 2024-04-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e5ba2640 1 -- 172.21.15.19:0/3473484486 learned_addr learned my addr 172.21.15.19:0/3473484486 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e63a3640 1 --2- 172.21.15.19:0/3473484486 >> v2:172.21.15.112:3300/0 conn(0x7f85e015dbe0 0x7f85e00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e6ba4640 1 -- 172.21.15.19:0/3473484486 >> v2:172.21.15.112:3300/0 conn(0x7f85e015dbe0 msgr2=0x7f85e00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e6ba4640 1 --2- 172.21.15.19:0/3473484486 >> v2:172.21.15.112:3300/0 conn(0x7f85e015dbe0 0x7f85e00d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e6ba4640 1 -- 172.21.15.19:0/3473484486 >> v1:172.21.15.19:6789/0 conn(0x7f85e00b2cb0 legacy=0x7f85e00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e6ba4640 1 -- 172.21.15.19:0/3473484486 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e00d1c50 con 0x7f85e00b92a0 2024-04-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.138+0000 7f85e53a1640 1 -- 172.21.15.19:0/3473484486 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989084138 0 0) 0x7f85e00b97a0 con 0x7f85e00b2cb0 2024-04-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.139+0000 7f85e6ba4640 1 --2- 172.21.15.19:0/3473484486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e00b92a0 0x7f85e00d0e50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f85e006c140 tx=0x7f85d802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfda8d228b692b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.139+0000 7f85e53a1640 1 -- 172.21.15.19:0/3473484486 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8007740 con 0x7f85e00b92a0 2024-04-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.139+0000 7f85e53a1640 1 -- 172.21.15.19:0/3473484486 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d80078e0 con 0x7f85e00b92a0 2024-04-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.139+0000 7f85e53a1640 1 -- 172.21.15.19:0/3473484486 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8007cb0 con 0x7f85e00b92a0 2024-04-25T09:53:15.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.140+0000 7f85e8607640 1 -- 172.21.15.19:0/3473484486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e00b92a0 msgr2=0x7f85e00d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.140+0000 7f85e8607640 1 --2- 172.21.15.19:0/3473484486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e00b92a0 0x7f85e00d0e50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f85e006c140 tx=0x7f85d802a5f0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.140+0000 7f85e8607640 1 -- 172.21.15.19:0/3473484486 shutdown_connections 2024-04-25T09:53:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.140+0000 7f85e8607640 1 --2- 172.21.15.19:0/3473484486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e00b92a0 0x7f85e00d0e50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.140+0000 7f85e8607640 1 --2- 172.21.15.19:0/3473484486 >> v2:172.21.15.112:3300/0 conn(0x7f85e015dbe0 0x7f85e00d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.140+0000 7f85e8607640 1 -- 172.21.15.19:0/3473484486 >> 172.21.15.19:0/3473484486 conn(0x7f85e0061a20 msgr2=0x7f85e0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.140+0000 7f85e8607640 1 -- 172.21.15.19:0/3473484486 shutdown_connections 2024-04-25T09:53:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.140+0000 7f85e8607640 1 -- 172.21.15.19:0/3473484486 wait complete. 2024-04-25T09:53:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.141+0000 7f85e8607640 1 Processor -- start 2024-04-25T09:53:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.141+0000 7f85e8607640 1 -- start start 2024-04-25T09:53:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e8607640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e00b2cb0 0x7f85e00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e8607640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85e00b92a0 0x7f85e0159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e8607640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85e015dbe0 0x7f85e00b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e8607640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e00b6ab0 con 0x7f85e00b2cb0 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e8607640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e00d2210 con 0x7f85e015dbe0 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e8607640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e00b6c30 con 0x7f85e00b92a0 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e63a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e00b2cb0 0x7f85e00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e63a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e00b2cb0 0x7f85e00cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43702/0 (socket says 172.21.15.19:43702) 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e63a3640 1 -- 172.21.15.19:0/1968519702 learned_addr learned my addr 172.21.15.19:0/1968519702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e6ba4640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85e015dbe0 0x7f85e00b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e5ba2640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85e00b92a0 0x7f85e0159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e63a3640 1 -- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85e00b92a0 msgr2=0x7f85e0159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e63a3640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85e00b92a0 0x7f85e0159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e63a3640 1 -- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85e015dbe0 msgr2=0x7f85e00b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e63a3640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85e015dbe0 0x7f85e00b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e63a3640 1 -- 172.21.15.19:0/1968519702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e01f4ed0 con 0x7f85e00b2cb0 2024-04-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e6ba4640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85e015dbe0 0x7f85e00b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e63a3640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e00b2cb0 0x7f85e00cd3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00e8a0 tx=0x7f85dc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85d6ffd640 1 -- 172.21.15.19:0/1968519702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc017070 con 0x7f85e00b2cb0 2024-04-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85d6ffd640 1 -- 172.21.15.19:0/1968519702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc00c940 con 0x7f85e00b2cb0 2024-04-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85e8607640 1 -- 172.21.15.19:0/1968519702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e01f51b0 con 0x7f85e00b2cb0 2024-04-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.142+0000 7f85d6ffd640 1 -- 172.21.15.19:0/1968519702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00cd30 con 0x7f85e00b2cb0 2024-04-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.143+0000 7f85e8607640 1 -- 172.21.15.19:0/1968519702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e01f5770 con 0x7f85e00b2cb0 2024-04-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.144+0000 7f85e8607640 1 -- 172.21.15.19:0/1968519702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e0073990 con 0x7f85e00b2cb0 2024-04-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.144+0000 7f85d6ffd640 1 -- 172.21.15.19:0/1968519702 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85dc01a070 con 0x7f85e00b2cb0 2024-04-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.144+0000 7f85d6ffd640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c406b4d0 0x7f85c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.144+0000 7f85d6ffd640 1 -- 172.21.15.19:0/1968519702 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f85dc08a6c0 con 0x7f85e00b2cb0 2024-04-25T09:53:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.144+0000 7f85e5ba2640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c406b4d0 0x7f85c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.145+0000 7f85e5ba2640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c406b4d0 0x7f85c406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85e01610a0 tx=0x7f85d0007d50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.148+0000 7f85d6ffd640 1 -- 172.21.15.19:0/1968519702 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85dc058bb0 con 0x7f85e00b2cb0 2024-04-25T09:53:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.365+0000 7f85e8607640 1 -- 172.21.15.19:0/1968519702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f85e00b2990 con 0x7f85e00b2cb0 2024-04-25T09:53:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.366+0000 7f85d6ffd640 1 -- 172.21.15.19:0/1968519702 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f85dc05cc50 con 0x7f85e00b2cb0 2024-04-25T09:53:15.368 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:53:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 -- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c406b4d0 msgr2=0x7f85c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c406b4d0 0x7f85c406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85e01610a0 tx=0x7f85d0007d50 comp rx=0 tx=0).stop 2024-04-25T09:53:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 -- 172.21.15.19:0/1968519702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e00b2cb0 msgr2=0x7f85e00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e00b2cb0 0x7f85e00cd3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00e8a0 tx=0x7f85dc00ed60 comp rx=0 tx=0).stop 2024-04-25T09:53:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 -- 172.21.15.19:0/1968519702 shutdown_connections 2024-04-25T09:53:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c406b4d0 0x7f85c406d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85e015dbe0 0x7f85e00b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85e00b92a0 0x7f85e0159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 --2- 172.21.15.19:0/1968519702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e00b2cb0 0x7f85e00cd3f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 -- 172.21.15.19:0/1968519702 >> 172.21.15.19:0/1968519702 conn(0x7f85e0061a20 msgr2=0x7f85e0061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:15.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 -- 172.21.15.19:0/1968519702 shutdown_connections 2024-04-25T09:53:15.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:15.368+0000 7f85e8607640 1 -- 172.21.15.19:0/1968519702 wait complete. 2024-04-25T09:53:15.380 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:53:15.380 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:53:15.380 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T09:53:15.380 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T09:53:15.381 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:15.381 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:53:15.381 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:53:15.381 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:53:15.381 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T09:53:15.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:15.381 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T09:53:15.397 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:15.397 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:15.397 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T09:53:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T09:53:15.398 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:53:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T09:53:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:53:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42751sec preferred_lft 42751sec 2024-04-25T09:53:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T09:53:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:15.399 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T09:53:15.400 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T09:53:15.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:15.400 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:15.400 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:15.400 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-04-25T09:53:15.400 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-04-25T09:53:15.400 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-04-25T09:53:15.400 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T09:53:15.400 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:15.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:15.541 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:53:15.566 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T09:53:15.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:15.567 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T09:53:15.646 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T09:53:15.646 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:53:15.646 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:53:15.646 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:53:15.646 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:53:15.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:15.647 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:15.647 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:15.647 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:53:15.647 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:53:15.647 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:53:15.647 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:15.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:15.869 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:53:15.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:15.885 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-04-25T09:53:15.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:15.940 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:15.940 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:15.940 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:15.940 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T09:53:15.940 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:16.016 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T09:53:16.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:16.016 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:16.016 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:16.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T09:53:16.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T09:53:16.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T09:53:16.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T09:53:16.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T09:53:16.017 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:16.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:16.151 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:16.151 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:16.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T09:53:16.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T09:53:16.151 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:16.187 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T09:53:16.187 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:16.187 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:16.243 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T09:53:16.243 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:16.243 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:16.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-04-25T09:53:16.363 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-04-25T09:53:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-04-25T09:53:16.421 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-04-25T09:53:16.422 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:53:16.422 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:53:16.422 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:53:16.422 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:53:16.422 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-04-25T09:53:16.422 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:16.422 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:16.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:16.422 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-04-25T09:53:16.475 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:53:16.475 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T09:53:16.518 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-04-25T09:53:16.541 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:53:16.597 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-04-25T09:53:16.598 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-04-25T09:53:16.598 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-04-25T09:53:16.598 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-04-25T09:53:16.598 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-04-25T09:53:16.598 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-04-25T09:53:16.598 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-04-25T09:53:16.598 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-04-25T09:53:16.598 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-04-25T09:53:16.598 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-04-25T09:53:16.598 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-04-25T09:53:16.598 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-04-25T09:53:16.599 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:53:16.600 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:53:16.600 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:53:16.600 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:53:16.600 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-04-25T09:53:16.600 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:16.600 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:16.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:16.600 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-04-25T09:53:16.606 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:2024-04-25T09:53:16.604+0000 7fca9520c480 -1 init, newargv = 0x560b8a087650 newargc=15 2024-04-25T09:53:16.606 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[41990]: starting ceph client 2024-04-25T09:53:16.612 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[41990]: starting fuse 2024-04-25T09:53:16.620 INFO:teuthology.orchestra.run.smithi019.stdout:43 2024-04-25T09:53:16.621 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T09:53:16.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:53:16.621 DEBUG:teuthology.orchestra.run.smithi019:> import glob 2024-04-25T09:53:16.621 DEBUG:teuthology.orchestra.run.smithi019:> import re 2024-04-25T09:53:16.621 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T09:53:16.621 DEBUG:teuthology.orchestra.run.smithi019:> import subprocess 2024-04-25T09:53:16.621 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:53:16.621 DEBUG:teuthology.orchestra.run.smithi019:> def _find_admin_socket(client_name): 2024-04-25T09:53:16.621 DEBUG:teuthology.orchestra.run.smithi019:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T09:53:16.621 DEBUG:teuthology.orchestra.run.smithi019:> files = glob.glob(asok_path) 2024-04-25T09:53:16.621 DEBUG:teuthology.orchestra.run.smithi019:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T09:53:16.621 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> # Given a non-glob path, it better be there 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> if "*" not in asok_path: 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> assert(len(files) == 1) 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> return files[0] 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> for f in files: 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> contents = proc_f.read() 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> if mountpoint in contents: 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> return f 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:53:16.622 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:53:16.623 DEBUG:teuthology.orchestra.run.smithi019:> print(_find_admin_socket("client.admin")) 2024-04-25T09:53:16.623 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T09:53:16.739 INFO:teuthology.orchestra.run.smithi019.stdout:/var/run/ceph/ceph-client.admin.41990.asok 2024-04-25T09:53:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:53:16.750 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41990.asok 2024-04-25T09:53:16.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:16.750 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41990.asok status 2024-04-25T09:53:16.885 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "metadata": { 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "entity_id": "admin", 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": "smithi019", 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "pid": "41990", 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "root": "/" 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_count": 0, 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_pinned_count": 0, 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 4434, 2024-04-25T09:53:16.886 INFO:teuthology.orchestra.run.smithi019.stdout: "inst": { 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: "name": { 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "client", 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 4434 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3963910650 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:53:16.887 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3963910650 2024-04-25T09:53:16.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:53:16.888 INFO:teuthology.orchestra.run.smithi019.stdout: "inst_str": "client.4434 192.168.0.1:0/3963910650", 2024-04-25T09:53:16.888 INFO:teuthology.orchestra.run.smithi019.stdout: "addr_str": "192.168.0.1:0/3963910650", 2024-04-25T09:53:16.888 INFO:teuthology.orchestra.run.smithi019.stdout: "inode_count": 1, 2024-04-25T09:53:16.888 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_epoch": 7, 2024-04-25T09:53:16.888 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch": 24, 2024-04-25T09:53:16.888 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:53:16.888 INFO:teuthology.orchestra.run.smithi019.stdout: "blocklisted": false, 2024-04-25T09:53:16.888 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs" 2024-04-25T09:53:16.888 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:53:16.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:16.894 DEBUG:teuthology.orchestra.run.smithi019:> stat --file-system '--printf=%T 2024-04-25T09:53:16.894 DEBUG:teuthology.orchestra.run.smithi019:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:16.910 INFO:teuthology.orchestra.run.smithi019.stdout:fuseblk 2024-04-25T09:53:16.910 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:16.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:16.910 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:16.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:16.977 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T09:53:17.049 INFO:teuthology.orchestra.run.smithi019.stdout:# file: . 2024-04-25T09:53:17.049 INFO:teuthology.orchestra.run.smithi019.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T09:53:17.049 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:17.050 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:53:17.050 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:17.106 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:17.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.244+0000 7f84ea19e640 1 Processor -- start 2024-04-25T09:53:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.245+0000 7f84ea19e640 1 -- start start 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.245+0000 7f84ea19e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f84e415dc70 0x7f84e40d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.245+0000 7f84ea19e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e409a450 0x7f84e40cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.245+0000 7f84ea19e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e406c220 con 0x7f84e406c700 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.245+0000 7f84ea19e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e409a8b0 con 0x7f84e415dc70 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.245+0000 7f84ea19e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40d0c90 con 0x7f84e409a450 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.246+0000 7f84e37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f84e415dc70 0x7f84e40d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.246+0000 7f84e2ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e409a450 0x7f84e40cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.246+0000 7f84e3fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f84e406c700 0x7f84e40cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:35894/0 (socket says 172.21.15.19:35894) 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.246+0000 7f84e3fff640 1 -- 172.21.15.19:0/64825989 learned_addr learned my addr 172.21.15.19:0/64825989 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.246+0000 7f84e2ffd640 1 -- 172.21.15.19:0/64825989 >> v2:172.21.15.112:3300/0 conn(0x7f84e415dc70 msgr2=0x7f84e40d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.246+0000 7f84e2ffd640 1 --2- 172.21.15.19:0/64825989 >> v2:172.21.15.112:3300/0 conn(0x7f84e415dc70 0x7f84e40d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.246+0000 7f84e2ffd640 1 -- 172.21.15.19:0/64825989 >> v1:172.21.15.112:6789/0 conn(0x7f84e406c700 legacy=0x7f84e40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.246+0000 7f84e2ffd640 1 -- 172.21.15.19:0/64825989 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40d0080 con 0x7f84e409a450 2024-04-25T09:53:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.246+0000 7f84e37fe640 1 --2- 172.21.15.19:0/64825989 >> v2:172.21.15.112:3300/0 conn(0x7f84e415dc70 0x7f84e40d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.247+0000 7f84e2ffd640 1 --2- 172.21.15.19:0/64825989 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e409a450 0x7f84e40cf9c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f84d80097a0 tx=0x7f84d802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb8d5e402e23114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.247+0000 7f84e27fc640 1 -- 172.21.15.19:0/64825989 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8037070 con 0x7f84e409a450 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.247+0000 7f84e27fc640 1 -- 172.21.15.19:0/64825989 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d8032440 con 0x7f84e409a450 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.247+0000 7f84e27fc640 1 -- 172.21.15.19:0/64825989 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8032810 con 0x7f84e409a450 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.247+0000 7f84ea19e640 1 -- 172.21.15.19:0/64825989 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e409a450 msgr2=0x7f84e40cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.247+0000 7f84ea19e640 1 --2- 172.21.15.19:0/64825989 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e409a450 0x7f84e40cf9c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f84d80097a0 tx=0x7f84d802a180 comp rx=0 tx=0).stop 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.248+0000 7f84ea19e640 1 -- 172.21.15.19:0/64825989 shutdown_connections 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.248+0000 7f84ea19e640 1 --2- 172.21.15.19:0/64825989 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e409a450 0x7f84e40cf9c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.248+0000 7f84ea19e640 1 --2- 172.21.15.19:0/64825989 >> v2:172.21.15.112:3300/0 conn(0x7f84e415dc70 0x7f84e40d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.248+0000 7f84ea19e640 1 -- 172.21.15.19:0/64825989 >> 172.21.15.19:0/64825989 conn(0x7f84e40b3270 msgr2=0x7f84e40b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.248+0000 7f84ea19e640 1 -- 172.21.15.19:0/64825989 shutdown_connections 2024-04-25T09:53:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.248+0000 7f84ea19e640 1 -- 172.21.15.19:0/64825989 wait complete. 2024-04-25T09:53:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.249+0000 7f84ea19e640 1 Processor -- start 2024-04-25T09:53:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.249+0000 7f84ea19e640 1 -- start start 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84ea19e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e406c700 0x7f84e4151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84ea19e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e409a450 0x7f84e40cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84ea19e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e415dc70 0x7f84e40ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84ea19e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40d0920 con 0x7f84e406c700 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84ea19e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40d0640 con 0x7f84e409a450 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84ea19e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e415e050 con 0x7f84e415dc70 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e406c700 0x7f84e4151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e406c700 0x7f84e4151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43716/0 (socket says 172.21.15.19:43716) 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e37fe640 1 -- 172.21.15.19:0/3803890930 learned_addr learned my addr 172.21.15.19:0/3803890930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e2ffd640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e409a450 0x7f84e40cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e3fff640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e415dc70 0x7f84e40ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e37fe640 1 -- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e415dc70 msgr2=0x7f84e40ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e37fe640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e415dc70 0x7f84e40ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e37fe640 1 -- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e409a450 msgr2=0x7f84e40cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e37fe640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e409a450 0x7f84e40cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e37fe640 1 -- 172.21.15.19:0/3803890930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e415f1c0 con 0x7f84e406c700 2024-04-25T09:53:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e3fff640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e415dc70 0x7f84e40ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e2ffd640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e409a450 0x7f84e40cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84e37fe640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e406c700 0x7f84e4151b90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84d400ebe0 tx=0x7f84d400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84cbfff640 1 -- 172.21.15.19:0/3803890930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4017070 con 0x7f84e406c700 2024-04-25T09:53:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84cbfff640 1 -- 172.21.15.19:0/3803890930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d400cc30 con 0x7f84e406c700 2024-04-25T09:53:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84ea19e640 1 -- 172.21.15.19:0/3803890930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e415f4a0 con 0x7f84e406c700 2024-04-25T09:53:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.250+0000 7f84cbfff640 1 -- 172.21.15.19:0/3803890930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4013470 con 0x7f84e406c700 2024-04-25T09:53:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.251+0000 7f84ea19e640 1 -- 172.21.15.19:0/3803890930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e415fa60 con 0x7f84e406c700 2024-04-25T09:53:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.252+0000 7f84ea19e640 1 -- 172.21.15.19:0/3803890930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e4073990 con 0x7f84e406c700 2024-04-25T09:53:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.252+0000 7f84cbfff640 1 -- 172.21.15.19:0/3803890930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84d401a070 con 0x7f84e406c700 2024-04-25T09:53:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.252+0000 7f84cbfff640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84d006b4d0 0x7f84d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.252+0000 7f84cbfff640 1 -- 172.21.15.19:0/3803890930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f84d408a850 con 0x7f84e406c700 2024-04-25T09:53:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.253+0000 7f84e2ffd640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84d006b4d0 0x7f84d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.253+0000 7f84e2ffd640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84d006b4d0 0x7f84d006d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84d8034000 tx=0x7f84d802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.257+0000 7f84cbfff640 1 -- 172.21.15.19:0/3803890930 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84d40593a0 con 0x7f84e406c700 2024-04-25T09:53:17.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.439+0000 7f84ea19e640 1 -- 172.21.15.19:0/3803890930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84e40b40a0 con 0x7f84e406c700 2024-04-25T09:53:17.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.440+0000 7f84cbfff640 1 -- 172.21.15.19:0/3803890930 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f84d4058ca0 con 0x7f84e406c700 2024-04-25T09:53:17.442 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:17.443 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.442+0000 7f84ea19e640 1 -- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84d006b4d0 msgr2=0x7f84d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.442+0000 7f84ea19e640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84d006b4d0 0x7f84d006d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84d8034000 tx=0x7f84d802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.442+0000 7f84ea19e640 1 -- 172.21.15.19:0/3803890930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e406c700 msgr2=0x7f84e4151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.442+0000 7f84ea19e640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e406c700 0x7f84e4151b90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84d400ebe0 tx=0x7f84d400c520 comp rx=0 tx=0).stop 2024-04-25T09:53:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.442+0000 7f84ea19e640 1 -- 172.21.15.19:0/3803890930 shutdown_connections 2024-04-25T09:53:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.442+0000 7f84ea19e640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84d006b4d0 0x7f84d006d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.442+0000 7f84ea19e640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84e415dc70 0x7f84e40ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.442+0000 7f84ea19e640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e409a450 0x7f84e40cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.442+0000 7f84ea19e640 1 --2- 172.21.15.19:0/3803890930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e406c700 0x7f84e4151b90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.442+0000 7f84ea19e640 1 -- 172.21.15.19:0/3803890930 >> 172.21.15.19:0/3803890930 conn(0x7f84e40b3270 msgr2=0x7f84e4070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.443+0000 7f84ea19e640 1 -- 172.21.15.19:0/3803890930 shutdown_connections 2024-04-25T09:53:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.443+0000 7f84ea19e640 1 -- 172.21.15.19:0/3803890930 wait complete. 2024-04-25T09:53:17.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:17.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.557+0000 7fe85c6c4640 1 Processor -- start 2024-04-25T09:53:17.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.557+0000 7fe85c6c4640 1 -- start start 2024-04-25T09:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.558+0000 7fe85c6c4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe854161710 0x7fe8540d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.558+0000 7fe85c6c4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85415e000 con 0x7fe85415dc20 2024-04-25T09:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.558+0000 7fe85c6c4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8540d1de0 con 0x7fe8540ced80 2024-04-25T09:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.558+0000 7fe85c6c4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe854161c60 con 0x7fe854161710 2024-04-25T09:53:17.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.558+0000 7fe85ac61640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe854161710 0x7fe8540d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.558+0000 7fe85a460640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe85415dc20 0x7fe8540ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48200/0 (socket says 172.21.15.19:48200) 2024-04-25T09:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.558+0000 7fe85a460640 1 -- 172.21.15.19:0/3192234769 learned_addr learned my addr 172.21.15.19:0/3192234769 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.558+0000 7fe85ac61640 1 -- 172.21.15.19:0/3192234769 >> v1:172.21.15.112:6789/0 conn(0x7fe8540ced80 legacy=0x7fe8540d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.558+0000 7fe85ac61640 1 -- 172.21.15.19:0/3192234769 >> v1:172.21.15.19:6789/0 conn(0x7fe85415dc20 legacy=0x7fe8540ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.558+0000 7fe85945e640 1 -- 172.21.15.19:0/3192234769 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654649637 0 0) 0x7fe85415e000 con 0x7fe85415dc20 2024-04-25T09:53:17.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.558+0000 7fe85ac61640 1 -- 172.21.15.19:0/3192234769 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe854161f40 con 0x7fe854161710 2024-04-25T09:53:17.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.559+0000 7fe85ac61640 1 --2- 172.21.15.19:0/3192234769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe854161710 0x7fe8540d29f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009d90 tx=0x7fe84c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c6f4b01cba15f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:17.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.559+0000 7fe85945e640 1 -- 172.21.15.19:0/3192234769 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c007740 con 0x7fe854161710 2024-04-25T09:53:17.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.559+0000 7fe85945e640 1 -- 172.21.15.19:0/3192234769 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c0078e0 con 0x7fe854161710 2024-04-25T09:53:17.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.559+0000 7fe85945e640 1 -- 172.21.15.19:0/3192234769 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c007cb0 con 0x7fe854161710 2024-04-25T09:53:17.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.560+0000 7fe85c6c4640 1 -- 172.21.15.19:0/3192234769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe854161710 msgr2=0x7fe8540d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.560+0000 7fe85c6c4640 1 --2- 172.21.15.19:0/3192234769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe854161710 0x7fe8540d29f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009d90 tx=0x7fe84c02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.560+0000 7fe85c6c4640 1 -- 172.21.15.19:0/3192234769 shutdown_connections 2024-04-25T09:53:17.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.560+0000 7fe85c6c4640 1 --2- 172.21.15.19:0/3192234769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe854161710 0x7fe8540d29f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.560+0000 7fe85c6c4640 1 -- 172.21.15.19:0/3192234769 >> 172.21.15.19:0/3192234769 conn(0x7fe8540d0420 msgr2=0x7fe8540d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:17.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.560+0000 7fe85c6c4640 1 -- 172.21.15.19:0/3192234769 shutdown_connections 2024-04-25T09:53:17.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.560+0000 7fe85c6c4640 1 -- 172.21.15.19:0/3192234769 wait complete. 2024-04-25T09:53:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.561+0000 7fe85c6c4640 1 Processor -- start 2024-04-25T09:53:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.561+0000 7fe85c6c4640 1 -- start start 2024-04-25T09:53:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85c6c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8540ced80 0x7fe85415f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85c6c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85415dc20 0x7fe8540d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85c6c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe854161710 0x7fe854154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85c6c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540b5e50 con 0x7fe8540ced80 2024-04-25T09:53:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85c6c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe854162500 con 0x7fe85415dc20 2024-04-25T09:53:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85c6c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540b5fd0 con 0x7fe854161710 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85a460640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8540ced80 0x7fe85415f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85a460640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8540ced80 0x7fe85415f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43744/0 (socket says 172.21.15.19:43744) 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85a460640 1 -- 172.21.15.19:0/4049330610 learned_addr learned my addr 172.21.15.19:0/4049330610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe859c5f640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85415dc20 0x7fe8540d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85ac61640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe854161710 0x7fe854154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85a460640 1 -- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe854161710 msgr2=0x7fe854154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85a460640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe854161710 0x7fe854154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85a460640 1 -- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85415dc20 msgr2=0x7fe8540d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85a460640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85415dc20 0x7fe8540d2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85a460640 1 -- 172.21.15.19:0/4049330610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8541f3560 con 0x7fe8540ced80 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe859c5f640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85415dc20 0x7fe8540d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.562+0000 7fe85a460640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8540ced80 0x7fe85415f930 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe85000c4a0 tx=0x7fe85000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:17.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.563+0000 7fe84affd640 1 -- 172.21.15.19:0/4049330610 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850018070 con 0x7fe8540ced80 2024-04-25T09:53:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.563+0000 7fe84affd640 1 -- 172.21.15.19:0/4049330610 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8500074e0 con 0x7fe8540ced80 2024-04-25T09:53:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.563+0000 7fe85c6c4640 1 -- 172.21.15.19:0/4049330610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8541f3820 con 0x7fe8540ced80 2024-04-25T09:53:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.563+0000 7fe84affd640 1 -- 172.21.15.19:0/4049330610 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850004a20 con 0x7fe8540ced80 2024-04-25T09:53:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.563+0000 7fe85c6c4640 1 -- 172.21.15.19:0/4049330610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8541f3de0 con 0x7fe8540ced80 2024-04-25T09:53:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.564+0000 7fe85c6c4640 1 -- 172.21.15.19:0/4049330610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe854073a00 con 0x7fe8540ced80 2024-04-25T09:53:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.564+0000 7fe84affd640 1 -- 172.21.15.19:0/4049330610 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe85001b070 con 0x7fe8540ced80 2024-04-25T09:53:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.565+0000 7fe84affd640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe83806b500 0x7fe83806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.565+0000 7fe84affd640 1 -- 172.21.15.19:0/4049330610 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe850089820 con 0x7fe8540ced80 2024-04-25T09:53:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.565+0000 7fe859c5f640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe83806b500 0x7fe83806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.565+0000 7fe859c5f640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe83806b500 0x7fe83806d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe844007fd0 tx=0x7fe844009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:17.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.570+0000 7fe84affd640 1 -- 172.21.15.19:0/4049330610 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe850058360 con 0x7fe8540ced80 2024-04-25T09:53:17.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.748+0000 7fe85c6c4640 1 -- 172.21.15.19:0/4049330610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8540cdfd0 con 0x7fe8540ced80 2024-04-25T09:53:17.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.748+0000 7fe84affd640 1 -- 172.21.15.19:0/4049330610 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe850057c60 con 0x7fe8540ced80 2024-04-25T09:53:17.750 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 -- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe83806b500 msgr2=0x7fe83806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe83806b500 0x7fe83806d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe844007fd0 tx=0x7fe844009040 comp rx=0 tx=0).stop 2024-04-25T09:53:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 -- 172.21.15.19:0/4049330610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8540ced80 msgr2=0x7fe85415f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8540ced80 0x7fe85415f930 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe85000c4a0 tx=0x7fe85000c960 comp rx=0 tx=0).stop 2024-04-25T09:53:17.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 -- 172.21.15.19:0/4049330610 shutdown_connections 2024-04-25T09:53:17.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe83806b500 0x7fe83806d9c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe854161710 0x7fe854154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe85415dc20 0x7fe8540d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 --2- 172.21.15.19:0/4049330610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8540ced80 0x7fe85415f930 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 -- 172.21.15.19:0/4049330610 >> 172.21.15.19:0/4049330610 conn(0x7fe8540d0420 msgr2=0x7fe8540d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:17.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 -- 172.21.15.19:0/4049330610 shutdown_connections 2024-04-25T09:53:17.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.750+0000 7fe85c6c4640 1 -- 172.21.15.19:0/4049330610 wait complete. 2024-04-25T09:53:17.763 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:17.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.864+0000 7f37d8348640 1 Processor -- start 2024-04-25T09:53:17.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.864+0000 7f37d8348640 1 -- start start 2024-04-25T09:53:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d8348640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f37d00ced80 0x7f37d00d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d8348640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37d0161710 0x7f37d00d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d8348640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d015e000 con 0x7f37d015dc20 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d8348640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d00d1de0 con 0x7f37d00ced80 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d8348640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d0161c60 con 0x7f37d0161710 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d58e3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f37d00ced80 0x7f37d00d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d60e4640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f37d015dc20 0x7f37d00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46770/0 (socket says 172.21.15.19:46770) 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d60e4640 1 -- 172.21.15.19:0/1240115599 learned_addr learned my addr 172.21.15.19:0/1240115599 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d68e5640 1 --2- 172.21.15.19:0/1240115599 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37d0161710 0x7f37d00d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d58e3640 1 -- 172.21.15.19:0/1240115599 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37d0161710 msgr2=0x7f37d00d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d58e3640 1 --2- 172.21.15.19:0/1240115599 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37d0161710 0x7f37d00d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d58e3640 1 -- 172.21.15.19:0/1240115599 >> v1:172.21.15.19:6789/0 conn(0x7f37d015dc20 legacy=0x7f37d00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d58e3640 1 -- 172.21.15.19:0/1240115599 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d0161f40 con 0x7f37d00ced80 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d50e2640 1 -- 172.21.15.19:0/1240115599 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431642650 0 0) 0x7f37d015e000 con 0x7f37d015dc20 2024-04-25T09:53:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.865+0000 7f37d68e5640 1 --2- 172.21.15.19:0/1240115599 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37d0161710 0x7f37d00d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.866+0000 7f37d58e3640 1 --2- 172.21.15.19:0/1240115599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d00ced80 0x7f37d00d22e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f37c40097a0 tx=0x7f37c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567b2b04869f1c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.866+0000 7f37d50e2640 1 -- 172.21.15.19:0/1240115599 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4037070 con 0x7f37d00ced80 2024-04-25T09:53:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.866+0000 7f37d50e2640 1 -- 172.21.15.19:0/1240115599 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c4032440 con 0x7f37d00ced80 2024-04-25T09:53:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.866+0000 7f37d50e2640 1 -- 172.21.15.19:0/1240115599 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4032810 con 0x7f37d00ced80 2024-04-25T09:53:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.867+0000 7f37d8348640 1 -- 172.21.15.19:0/1240115599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d00ced80 msgr2=0x7f37d00d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.867+0000 7f37d8348640 1 --2- 172.21.15.19:0/1240115599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d00ced80 0x7f37d00d22e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f37c40097a0 tx=0x7f37c402a180 comp rx=0 tx=0).stop 2024-04-25T09:53:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.867+0000 7f37d8348640 1 -- 172.21.15.19:0/1240115599 shutdown_connections 2024-04-25T09:53:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.867+0000 7f37d8348640 1 --2- 172.21.15.19:0/1240115599 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37d0161710 0x7f37d00d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.867+0000 7f37d8348640 1 --2- 172.21.15.19:0/1240115599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d00ced80 0x7f37d00d22e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.867+0000 7f37d8348640 1 -- 172.21.15.19:0/1240115599 >> 172.21.15.19:0/1240115599 conn(0x7f37d00d0420 msgr2=0x7f37d00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.867+0000 7f37d8348640 1 -- 172.21.15.19:0/1240115599 shutdown_connections 2024-04-25T09:53:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.867+0000 7f37d8348640 1 -- 172.21.15.19:0/1240115599 wait complete. 2024-04-25T09:53:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.868+0000 7f37d8348640 1 Processor -- start 2024-04-25T09:53:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.868+0000 7f37d8348640 1 -- start start 2024-04-25T09:53:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d8348640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37d00ced80 0x7f37d014fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d8348640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d015dc20 0x7f37d00cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d8348640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37d0161710 0x7f37d00b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d8348640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d00b5e50 con 0x7f37d015dc20 2024-04-25T09:53:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d8348640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d0162500 con 0x7f37d00ced80 2024-04-25T09:53:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d8348640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d00b5fd0 con 0x7f37d0161710 2024-04-25T09:53:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d58e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d015dc20 0x7f37d00cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d58e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d015dc20 0x7f37d00cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35424/0 (socket says 172.21.15.19:35424) 2024-04-25T09:53:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d58e3640 1 -- 172.21.15.19:0/1019336755 learned_addr learned my addr 172.21.15.19:0/1019336755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d68e5640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37d0161710 0x7f37d00b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d60e4640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37d00ced80 0x7f37d014fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d58e3640 1 -- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37d0161710 msgr2=0x7f37d00b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d58e3640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37d0161710 0x7f37d00b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d58e3640 1 -- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37d00ced80 msgr2=0x7f37d014fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d58e3640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37d00ced80 0x7f37d014fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.869+0000 7f37d58e3640 1 -- 172.21.15.19:0/1019336755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d00ccea0 con 0x7f37d015dc20 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.870+0000 7f37d58e3640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d015dc20 0x7f37d00cc670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f37c4035040 tx=0x7f37c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.870+0000 7f37ceffd640 1 -- 172.21.15.19:0/1019336755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4037040 con 0x7f37d015dc20 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.870+0000 7f37ceffd640 1 -- 172.21.15.19:0/1019336755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c4030770 con 0x7f37d015dc20 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.870+0000 7f37ceffd640 1 -- 172.21.15.19:0/1019336755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4030b80 con 0x7f37d015dc20 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.870+0000 7f37d8348640 1 -- 172.21.15.19:0/1019336755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d00cd180 con 0x7f37d015dc20 2024-04-25T09:53:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.870+0000 7f37d8348640 1 -- 172.21.15.19:0/1019336755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d00cd740 con 0x7f37d015dc20 2024-04-25T09:53:17.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.871+0000 7f37d8348640 1 -- 172.21.15.19:0/1019336755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d0073a00 con 0x7f37d015dc20 2024-04-25T09:53:17.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.871+0000 7f37ceffd640 1 -- 172.21.15.19:0/1019336755 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37c4044020 con 0x7f37d015dc20 2024-04-25T09:53:17.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.872+0000 7f37ceffd640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37bc06b4d0 0x7f37bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:17.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.872+0000 7f37ceffd640 1 -- 172.21.15.19:0/1019336755 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f37c40aaac0 con 0x7f37d015dc20 2024-04-25T09:53:17.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.872+0000 7f37d60e4640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37bc06b4d0 0x7f37bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:17.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.872+0000 7f37d60e4640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37bc06b4d0 0x7f37bc06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f37c8000f60 tx=0x7f37c8007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:17.876+0000 7f37ceffd640 1 -- 172.21.15.19:0/1019336755 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37c4032e20 con 0x7f37d015dc20 2024-04-25T09:53:18.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.055+0000 7f37d8348640 1 -- 172.21.15.19:0/1019336755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37d00d16b0 con 0x7f37d015dc20 2024-04-25T09:53:18.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.055+0000 7f37ceffd640 1 -- 172.21.15.19:0/1019336755 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37c40795f0 con 0x7f37d015dc20 2024-04-25T09:53:18.057 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:18.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 -- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37bc06b4d0 msgr2=0x7f37bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37bc06b4d0 0x7f37bc06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f37c8000f60 tx=0x7f37c8007c00 comp rx=0 tx=0).stop 2024-04-25T09:53:18.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 -- 172.21.15.19:0/1019336755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d015dc20 msgr2=0x7f37d00cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d015dc20 0x7f37d00cc670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f37c4035040 tx=0x7f37c40046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 -- 172.21.15.19:0/1019336755 shutdown_connections 2024-04-25T09:53:18.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37bc06b4d0 0x7f37bc06d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37d0161710 0x7f37d00b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37d015dc20 0x7f37d00cc670 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 --2- 172.21.15.19:0/1019336755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37d00ced80 0x7f37d014fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 -- 172.21.15.19:0/1019336755 >> 172.21.15.19:0/1019336755 conn(0x7f37d00d0420 msgr2=0x7f37d00d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:18.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 -- 172.21.15.19:0/1019336755 shutdown_connections 2024-04-25T09:53:18.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.057+0000 7f37d8348640 1 -- 172.21.15.19:0/1019336755 wait complete. 2024-04-25T09:53:18.069 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:18.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.172+0000 7f3535be7640 1 Processor -- start 2024-04-25T09:53:18.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.172+0000 7f3535be7640 1 -- start start 2024-04-25T09:53:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.172+0000 7f3535be7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f353015a9f0 0x7f35300c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.172+0000 7f3535be7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35300cdbf0 0x7f35300ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.172+0000 7f3535be7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35300c6940 con 0x7f35300cd520 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.172+0000 7f3535be7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35300b0020 con 0x7f35300cdbf0 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.172+0000 7f3535be7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35300cb5a0 con 0x7f353015a9f0 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.173+0000 7f352effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35300cdbf0 0x7f35300ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.173+0000 7f352effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35300cdbf0 0x7f35300ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35432/0 (socket says 172.21.15.19:35432) 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.173+0000 7f352effd640 1 -- 172.21.15.19:0/3060934796 learned_addr learned my addr 172.21.15.19:0/3060934796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.173+0000 7f352f7fe640 1 --2- 172.21.15.19:0/3060934796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f353015a9f0 0x7f35300c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.173+0000 7f352effd640 1 -- 172.21.15.19:0/3060934796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f353015a9f0 msgr2=0x7f35300c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.173+0000 7f352effd640 1 --2- 172.21.15.19:0/3060934796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f353015a9f0 0x7f35300c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.173+0000 7f352effd640 1 -- 172.21.15.19:0/3060934796 >> v1:172.21.15.112:6789/0 conn(0x7f35300cd520 legacy=0x7f35300ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.173+0000 7f352effd640 1 -- 172.21.15.19:0/3060934796 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353015add0 con 0x7f35300cdbf0 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f352effd640 1 --2- 172.21.15.19:0/3060934796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35300cdbf0 0x7f35300ccfe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f352400bb10 tx=0x7f352402c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2e5a285d8927c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f352e7fc640 1 -- 172.21.15.19:0/3060934796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524032790 con 0x7f35300cdbf0 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f352e7fc640 1 -- 172.21.15.19:0/3060934796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3524038070 con 0x7f35300cdbf0 2024-04-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f352e7fc640 1 -- 172.21.15.19:0/3060934796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524032b50 con 0x7f35300cdbf0 2024-04-25T09:53:18.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f3535be7640 1 -- 172.21.15.19:0/3060934796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35300cdbf0 msgr2=0x7f35300ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f3535be7640 1 --2- 172.21.15.19:0/3060934796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35300cdbf0 0x7f35300ccfe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f352400bb10 tx=0x7f352402c8d0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f3535be7640 1 -- 172.21.15.19:0/3060934796 shutdown_connections 2024-04-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f3535be7640 1 --2- 172.21.15.19:0/3060934796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35300cdbf0 0x7f35300ccfe0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f3535be7640 1 --2- 172.21.15.19:0/3060934796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f353015a9f0 0x7f35300c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f3535be7640 1 -- 172.21.15.19:0/3060934796 >> 172.21.15.19:0/3060934796 conn(0x7f35300b2390 msgr2=0x7f35300b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f3535be7640 1 -- 172.21.15.19:0/3060934796 shutdown_connections 2024-04-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.174+0000 7f3535be7640 1 -- 172.21.15.19:0/3060934796 wait complete. 2024-04-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.175+0000 7f3535be7640 1 Processor -- start 2024-04-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.176+0000 7f3535be7640 1 -- start start 2024-04-25T09:53:18.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.176+0000 7f3535be7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35300cd520 0x7f353014c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.176+0000 7f3535be7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35300cdbf0 0x7f35300c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.176+0000 7f3535be7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f353015a9f0 0x7f3530157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.176+0000 7f3535be7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35300cb230 con 0x7f353015a9f0 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.176+0000 7f3535be7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35300caf50 con 0x7f35300cdbf0 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.176+0000 7f3535be7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35300ce070 con 0x7f35300cd520 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.176+0000 7f352ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f353015a9f0 0x7f3530157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f352ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f353015a9f0 0x7f3530157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35438/0 (socket says 172.21.15.19:35438) 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f352ffff640 1 -- 172.21.15.19:0/988553399 learned_addr learned my addr 172.21.15.19:0/988553399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f352effd640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35300cdbf0 0x7f35300c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f352f7fe640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35300cd520 0x7f353014c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f352ffff640 1 -- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35300cd520 msgr2=0x7f353014c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f352ffff640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35300cd520 0x7f353014c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f352ffff640 1 -- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35300cdbf0 msgr2=0x7f35300c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f352ffff640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35300cdbf0 0x7f35300c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f352ffff640 1 -- 172.21.15.19:0/988553399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35300b5de0 con 0x7f353015a9f0 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f352f7fe640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35300cd520 0x7f353014c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f352ffff640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f353015a9f0 0x7f3530157e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f352800c7b0 tx=0x7f352800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f3517fff640 1 -- 172.21.15.19:0/988553399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528004b60 con 0x7f353015a9f0 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f3517fff640 1 -- 172.21.15.19:0/988553399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3528004d00 con 0x7f353015a9f0 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f3535be7640 1 -- 172.21.15.19:0/988553399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35300b60c0 con 0x7f353015a9f0 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f3517fff640 1 -- 172.21.15.19:0/988553399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352800aa90 con 0x7f353015a9f0 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.177+0000 7f3535be7640 1 -- 172.21.15.19:0/988553399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35300c7830 con 0x7f353015a9f0 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.178+0000 7f3535be7640 1 -- 172.21.15.19:0/988553399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353006cc80 con 0x7f353015a9f0 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.179+0000 7f3517fff640 1 -- 172.21.15.19:0/988553399 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f352800ac30 con 0x7f353015a9f0 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.179+0000 7f3517fff640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f351c06b4d0 0x7f351c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.179+0000 7f3517fff640 1 -- 172.21.15.19:0/988553399 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f352808a730 con 0x7f353015a9f0 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.179+0000 7f352f7fe640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f351c06b4d0 0x7f351c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.180+0000 7f352f7fe640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f351c06b4d0 0x7f351c06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3520006fd0 tx=0x7f3520008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.183+0000 7f3517fff640 1 -- 172.21.15.19:0/988553399 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3528020070 con 0x7f353015a9f0 2024-04-25T09:53:18.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.362+0000 7f3535be7640 1 -- 172.21.15.19:0/988553399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f35300cc5e0 con 0x7f353015a9f0 2024-04-25T09:53:18.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.362+0000 7f3517fff640 1 -- 172.21.15.19:0/988553399 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3528059270 con 0x7f353015a9f0 2024-04-25T09:53:18.364 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 -- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f351c06b4d0 msgr2=0x7f351c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f351c06b4d0 0x7f351c06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3520006fd0 tx=0x7f3520008040 comp rx=0 tx=0).stop 2024-04-25T09:53:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 -- 172.21.15.19:0/988553399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f353015a9f0 msgr2=0x7f3530157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f353015a9f0 0x7f3530157e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f352800c7b0 tx=0x7f352800cc70 comp rx=0 tx=0).stop 2024-04-25T09:53:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 -- 172.21.15.19:0/988553399 shutdown_connections 2024-04-25T09:53:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f351c06b4d0 0x7f351c06d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f353015a9f0 0x7f3530157e30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35300cdbf0 0x7f35300c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 --2- 172.21.15.19:0/988553399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35300cd520 0x7f353014c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 -- 172.21.15.19:0/988553399 >> 172.21.15.19:0/988553399 conn(0x7f35300b2390 msgr2=0x7f35300c5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 -- 172.21.15.19:0/988553399 shutdown_connections 2024-04-25T09:53:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.364+0000 7f3535be7640 1 -- 172.21.15.19:0/988553399 wait complete. 2024-04-25T09:53:18.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.478+0000 7f1fabefc640 1 Processor -- start 2024-04-25T09:53:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.478+0000 7f1fabefc640 1 -- start start 2024-04-25T09:53:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fabefc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1fa415dc20 0x7f1fa40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fabefc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 0x7f1fa40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fabefc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa415e000 con 0x7f1fa40b9330 2024-04-25T09:53:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fabefc640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa40d1de0 con 0x7f1fa415dc20 2024-04-25T09:53:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fabefc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4161c60 con 0x7f1fa4161710 2024-04-25T09:53:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fa9c98640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1fa415dc20 0x7f1fa40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fa9c98640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1fa415dc20 0x7f1fa40ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35454/0 (socket says 172.21.15.19:35454) 2024-04-25T09:53:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fa9c98640 1 -- 172.21.15.19:0/3426001097 learned_addr learned my addr 172.21.15.19:0/3426001097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:18.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fa9497640 1 --2- 172.21.15.19:0/3426001097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 0x7f1fa40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fa9c98640 1 -- 172.21.15.19:0/3426001097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 msgr2=0x7f1fa40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:18.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fa9c98640 1 --2- 172.21.15.19:0/3426001097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 0x7f1fa40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fa9c98640 1 -- 172.21.15.19:0/3426001097 >> v1:172.21.15.112:6789/0 conn(0x7f1fa40b9330 legacy=0x7f1fa406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:18.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fa9c98640 1 -- 172.21.15.19:0/3426001097 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa4161f40 con 0x7f1fa415dc20 2024-04-25T09:53:18.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.479+0000 7f1fa9497640 1 --2- 172.21.15.19:0/3426001097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 0x7f1fa40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.480+0000 7f1fa9c98640 1 --2- 172.21.15.19:0/3426001097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fa415dc20 0x7f1fa40ce670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009870 tx=0x7f1fa002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3cadec568afc8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.480+0000 7f1fa8c96640 1 -- 172.21.15.19:0/3426001097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0037070 con 0x7f1fa415dc20 2024-04-25T09:53:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.480+0000 7f1fa8c96640 1 -- 172.21.15.19:0/3426001097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0032440 con 0x7f1fa415dc20 2024-04-25T09:53:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.480+0000 7f1fa8c96640 1 -- 172.21.15.19:0/3426001097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0032810 con 0x7f1fa415dc20 2024-04-25T09:53:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.480+0000 7f1fabefc640 1 -- 172.21.15.19:0/3426001097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fa415dc20 msgr2=0x7f1fa40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.480+0000 7f1fabefc640 1 --2- 172.21.15.19:0/3426001097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fa415dc20 0x7f1fa40ce670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009870 tx=0x7f1fa002a210 comp rx=0 tx=0).stop 2024-04-25T09:53:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.480+0000 7f1fabefc640 1 -- 172.21.15.19:0/3426001097 shutdown_connections 2024-04-25T09:53:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.480+0000 7f1fabefc640 1 --2- 172.21.15.19:0/3426001097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 0x7f1fa40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.480+0000 7f1fabefc640 1 --2- 172.21.15.19:0/3426001097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fa415dc20 0x7f1fa40ce670 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.480+0000 7f1fabefc640 1 -- 172.21.15.19:0/3426001097 >> 172.21.15.19:0/3426001097 conn(0x7f1fa40d0420 msgr2=0x7f1fa40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.481+0000 7f1fabefc640 1 -- 172.21.15.19:0/3426001097 shutdown_connections 2024-04-25T09:53:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.481+0000 7f1fabefc640 1 -- 172.21.15.19:0/3426001097 wait complete. 2024-04-25T09:53:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.482+0000 7f1fabefc640 1 Processor -- start 2024-04-25T09:53:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.482+0000 7f1fabefc640 1 -- start start 2024-04-25T09:53:18.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.482+0000 7f1fabefc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fa40b9330 0x7f1fa40cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.482+0000 7f1fabefc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa415dc20 0x7f1fa4158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fabefc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 0x7f1fa40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fabefc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa40b5e50 con 0x7f1fa40b9330 2024-04-25T09:53:18.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fabefc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4162500 con 0x7f1fa415dc20 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fabefc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa40b5fd0 con 0x7f1fa4161710 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fa9c98640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fa40b9330 0x7f1fa40cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fa9c98640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fa40b9330 0x7f1fa40cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35466/0 (socket says 172.21.15.19:35466) 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fa9c98640 1 -- 172.21.15.19:0/489651269 learned_addr learned my addr 172.21.15.19:0/489651269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fa9497640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa415dc20 0x7f1fa4158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1faa499640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 0x7f1fa40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fa9c98640 1 -- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 msgr2=0x7f1fa40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fa9c98640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 0x7f1fa40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fa9c98640 1 -- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa415dc20 msgr2=0x7f1fa4158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fa9c98640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa415dc20 0x7f1fa4158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fa9c98640 1 -- 172.21.15.19:0/489651269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa40cade0 con 0x7f1fa40b9330 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1faa499640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 0x7f1fa40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fa9c98640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fa40b9330 0x7f1fa40cbe60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0000c00 tx=0x7f1fa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1f9a7fc640 1 -- 172.21.15.19:0/489651269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0041020 con 0x7f1fa40b9330 2024-04-25T09:53:18.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1f9a7fc640 1 -- 172.21.15.19:0/489651269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa00306c0 con 0x7f1fa40b9330 2024-04-25T09:53:18.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1f9a7fc640 1 -- 172.21.15.19:0/489651269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0037040 con 0x7f1fa40b9330 2024-04-25T09:53:18.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fabefc640 1 -- 172.21.15.19:0/489651269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa40cb0c0 con 0x7f1fa40b9330 2024-04-25T09:53:18.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.483+0000 7f1fabefc640 1 -- 172.21.15.19:0/489651269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa40cb680 con 0x7f1fa40b9330 2024-04-25T09:53:18.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.484+0000 7f1fabefc640 1 -- 172.21.15.19:0/489651269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa40ce670 con 0x7f1fa40b9330 2024-04-25T09:53:18.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.485+0000 7f1f9a7fc640 1 -- 172.21.15.19:0/489651269 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1fa0044020 con 0x7f1fa40b9330 2024-04-25T09:53:18.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.485+0000 7f1f9a7fc640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f8806b4d0 0x7f1f8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.485+0000 7f1f9a7fc640 1 -- 172.21.15.19:0/489651269 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1fa00aada0 con 0x7f1fa40b9330 2024-04-25T09:53:18.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.485+0000 7f1fa9497640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f8806b4d0 0x7f1f8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.486+0000 7f1fa9497640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f8806b4d0 0x7f1f8806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40b4dd0 tx=0x7f1f94008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.490+0000 7f1f9a7fc640 1 -- 172.21.15.19:0/489651269 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1fa40ce670 con 0x7f1fa40b9330 2024-04-25T09:53:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.667+0000 7f1fabefc640 1 -- 172.21.15.19:0/489651269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1fa4073a00 con 0x7f1fa40b9330 2024-04-25T09:53:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.667+0000 7f1f9a7fc640 1 -- 172.21.15.19:0/489651269 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1fa004b030 con 0x7f1fa40b9330 2024-04-25T09:53:18.669 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 -- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f8806b4d0 msgr2=0x7f1f8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f8806b4d0 0x7f1f8806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40b4dd0 tx=0x7f1f94008040 comp rx=0 tx=0).stop 2024-04-25T09:53:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 -- 172.21.15.19:0/489651269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fa40b9330 msgr2=0x7f1fa40cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fa40b9330 0x7f1fa40cbe60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0000c00 tx=0x7f1fa00046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 -- 172.21.15.19:0/489651269 shutdown_connections 2024-04-25T09:53:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f8806b4d0 0x7f1f8806d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fa4161710 0x7f1fa40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa415dc20 0x7f1fa4158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 --2- 172.21.15.19:0/489651269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fa40b9330 0x7f1fa40cbe60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 -- 172.21.15.19:0/489651269 >> 172.21.15.19:0/489651269 conn(0x7f1fa40d0420 msgr2=0x7f1fa40b8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 -- 172.21.15.19:0/489651269 shutdown_connections 2024-04-25T09:53:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.669+0000 7f1fabefc640 1 -- 172.21.15.19:0/489651269 wait complete. 2024-04-25T09:53:18.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:53:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.781+0000 7f46d3902640 1 Processor -- start 2024-04-25T09:53:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.781+0000 7f46d3902640 1 -- start start 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.782+0000 7f46d3902640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46cc15dc20 0x7f46cc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.782+0000 7f46d3902640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc06cda0 con 0x7f46cc0d32f0 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.782+0000 7f46d3902640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc15e000 con 0x7f46cc161710 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.782+0000 7f46d3902640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0d1de0 con 0x7f46cc15dc20 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.782+0000 7f46d1e9f640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f46cc0d32f0 0x7f46cc06c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46782/0 (socket says 172.21.15.19:46782) 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.782+0000 7f46d1e9f640 1 -- 172.21.15.19:0/2074395341 learned_addr learned my addr 172.21.15.19:0/2074395341 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.783+0000 7f46d169e640 1 --2- 172.21.15.19:0/2074395341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46cc15dc20 0x7f46cc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.783+0000 7f46cbfff640 1 -- 172.21.15.19:0/2074395341 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290545576 0 0) 0x7f46cc06cda0 con 0x7f46cc0d32f0 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.783+0000 7f46cbfff640 1 -- 172.21.15.19:0/2074395341 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46b4003520 con 0x7f46cc0d32f0 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.783+0000 7f46cbfff640 1 -- 172.21.15.19:0/2074395341 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693574602 0 0) 0x7f46cc15e000 con 0x7f46cc161710 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.783+0000 7f46cbfff640 1 -- 172.21.15.19:0/2074395341 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46cc06cda0 con 0x7f46cc161710 2024-04-25T09:53:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.783+0000 7f46cbfff640 1 -- 172.21.15.19:0/2074395341 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1376753151 0 0) 0x7f46b4003520 con 0x7f46cc0d32f0 2024-04-25T09:53:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.783+0000 7f46d169e640 1 -- 172.21.15.19:0/2074395341 >> v1:172.21.15.112:6789/0 conn(0x7f46cc161710 legacy=0x7f46cc06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.783+0000 7f46d169e640 1 -- 172.21.15.19:0/2074395341 >> v1:172.21.15.19:6789/0 conn(0x7f46cc0d32f0 legacy=0x7f46cc06c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.783+0000 7f46d169e640 1 -- 172.21.15.19:0/2074395341 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc161c60 con 0x7f46cc15dc20 2024-04-25T09:53:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.784+0000 7f46d169e640 1 --2- 172.21.15.19:0/2074395341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46cc15dc20 0x7f46cc0ce670 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0097d0 tx=0x7f46bc02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fcee9d7b37a99b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.784+0000 7f46cbfff640 1 -- 172.21.15.19:0/2074395341 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc037070 con 0x7f46cc15dc20 2024-04-25T09:53:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.784+0000 7f46cbfff640 1 -- 172.21.15.19:0/2074395341 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc02ac30 con 0x7f46cc15dc20 2024-04-25T09:53:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.784+0000 7f46cbfff640 1 -- 172.21.15.19:0/2074395341 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40019f0 con 0x7f46cc15dc20 2024-04-25T09:53:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.784+0000 7f46d3902640 1 -- 172.21.15.19:0/2074395341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46cc15dc20 msgr2=0x7f46cc0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.784+0000 7f46d3902640 1 --2- 172.21.15.19:0/2074395341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46cc15dc20 0x7f46cc0ce670 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0097d0 tx=0x7f46bc02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.784+0000 7f46d3902640 1 -- 172.21.15.19:0/2074395341 shutdown_connections 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.784+0000 7f46d3902640 1 --2- 172.21.15.19:0/2074395341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46cc15dc20 0x7f46cc0ce670 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.784+0000 7f46d3902640 1 -- 172.21.15.19:0/2074395341 >> 172.21.15.19:0/2074395341 conn(0x7f46cc0d0420 msgr2=0x7f46cc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.785+0000 7f46d3902640 1 -- 172.21.15.19:0/2074395341 shutdown_connections 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.785+0000 7f46d3902640 1 -- 172.21.15.19:0/2074395341 wait complete. 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.786+0000 7f46d3902640 1 Processor -- start 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.786+0000 7f46d3902640 1 -- start start 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.786+0000 7f46d3902640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc0d32f0 0x7f46cc1f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.786+0000 7f46d3902640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46cc15dc20 0x7f46cc152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.786+0000 7f46d3902640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46cc161710 0x7f46cc160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.786+0000 7f46d3902640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc162500 con 0x7f46cc0d32f0 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.786+0000 7f46d3902640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc162220 con 0x7f46cc15dc20 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.786+0000 7f46d3902640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0b5e50 con 0x7f46cc161710 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d169e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc0d32f0 0x7f46cc1f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d169e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc0d32f0 0x7f46cc1f7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35470/0 (socket says 172.21.15.19:35470) 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d169e640 1 -- 172.21.15.19:0/1994242015 learned_addr learned my addr 172.21.15.19:0/1994242015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d0e9d640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46cc15dc20 0x7f46cc152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d1e9f640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46cc161710 0x7f46cc160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d169e640 1 -- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46cc161710 msgr2=0x7f46cc160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d169e640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46cc161710 0x7f46cc160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d169e640 1 -- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46cc15dc20 msgr2=0x7f46cc152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d169e640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46cc15dc20 0x7f46cc152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d169e640 1 -- 172.21.15.19:0/1994242015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc153370 con 0x7f46cc0d32f0 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d169e640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc0d32f0 0x7f46cc1f7af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009720 tx=0x7f46bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46c9ffb640 1 -- 172.21.15.19:0/1994242015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc041070 con 0x7f46cc0d32f0 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46c9ffb640 1 -- 172.21.15.19:0/1994242015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc03c660 con 0x7f46cc0d32f0 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46c9ffb640 1 -- 172.21.15.19:0/1994242015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc037040 con 0x7f46cc0d32f0 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d3902640 1 -- 172.21.15.19:0/1994242015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc15d420 con 0x7f46cc0d32f0 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.787+0000 7f46d3902640 1 -- 172.21.15.19:0/1994242015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc15f580 con 0x7f46cc0d32f0 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.788+0000 7f46d3902640 1 -- 172.21.15.19:0/1994242015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc073a00 con 0x7f46cc0d32f0 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.789+0000 7f46c9ffb640 1 -- 172.21.15.19:0/1994242015 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46bc044070 con 0x7f46cc0d32f0 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.789+0000 7f46c9ffb640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46b806b4d0 0x7f46b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.789+0000 7f46c9ffb640 1 -- 172.21.15.19:0/1994242015 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f46bc0aae30 con 0x7f46cc0d32f0 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.789+0000 7f46d0e9d640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46b806b4d0 0x7f46b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.790+0000 7f46d0e9d640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46b806b4d0 0x7f46b806d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f46cc15ccd0 tx=0x7f46c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.794+0000 7f46c9ffb640 1 -- 172.21.15.19:0/1994242015 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46cc153370 con 0x7f46cc0d32f0 2024-04-25T09:53:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.987+0000 7f46d3902640 1 -- 172.21.15.19:0/1994242015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f46cc0d16b0 con 0x7f46cc0d32f0 2024-04-25T09:53:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.988+0000 7f46c9ffb640 1 -- 172.21.15.19:0/1994242015 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f46bc079440 con 0x7f46cc0d32f0 2024-04-25T09:53:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T09:53:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 -- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46b806b4d0 msgr2=0x7f46b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46b806b4d0 0x7f46b806d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f46cc15ccd0 tx=0x7f46c0009040 comp rx=0 tx=0).stop 2024-04-25T09:53:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 -- 172.21.15.19:0/1994242015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc0d32f0 msgr2=0x7f46cc1f7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc0d32f0 0x7f46cc1f7af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009720 tx=0x7f46bc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 -- 172.21.15.19:0/1994242015 shutdown_connections 2024-04-25T09:53:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46b806b4d0 0x7f46b806d990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46cc161710 0x7f46cc160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46cc15dc20 0x7f46cc152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 --2- 172.21.15.19:0/1994242015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc0d32f0 0x7f46cc1f7af0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 -- 172.21.15.19:0/1994242015 >> 172.21.15.19:0/1994242015 conn(0x7f46cc0d0420 msgr2=0x7f46cc0c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:18.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 -- 172.21.15.19:0/1994242015 shutdown_connections 2024-04-25T09:53:18.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:18.990+0000 7f46d3902640 1 -- 172.21.15.19:0/1994242015 wait complete. 2024-04-25T09:53:19.002 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:53:19.002 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:19.018 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.157+0000 7fc4a144a640 1 Processor -- start 2024-04-25T09:53:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.157+0000 7fc4a144a640 1 -- start start 2024-04-25T09:53:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.158+0000 7fc4a144a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc49c0cd070 0x7fc49c0cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.158+0000 7fc4a144a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49c0cfb30 0x7fc49c0cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.158+0000 7fc4a144a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49c1625d0 con 0x7fc49c15db50 2024-04-25T09:53:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.158+0000 7fc49affd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc49c15db50 0x7fc49c0c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46790/0 (socket says 172.21.15.19:46790) 2024-04-25T09:53:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.158+0000 7fc49a7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc49c0cd070 0x7fc49c0cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.158+0000 7fc4a144a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c15df30 con 0x7fc49c0cd070 2024-04-25T09:53:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.158+0000 7fc49affd640 1 -- 172.21.15.19:0/2608205915 learned_addr learned my addr 172.21.15.19:0/2608205915 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.158+0000 7fc4a144a640 1 -- 172.21.15.19:0/2608205915 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0cc2b0 con 0x7fc49c0cfb30 2024-04-25T09:53:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.158+0000 7fc49b7fe640 1 --2- 172.21.15.19:0/2608205915 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49c0cfb30 0x7fc49c0cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.159+0000 7fc499ffb640 1 -- 172.21.15.19:0/2608205915 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958246508 0 0) 0x7fc49c1625d0 con 0x7fc49c15db50 2024-04-25T09:53:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.159+0000 7fc499ffb640 1 -- 172.21.15.19:0/2608205915 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc488003520 con 0x7fc49c15db50 2024-04-25T09:53:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.159+0000 7fc49a7fc640 1 -- 172.21.15.19:0/2608205915 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49c0cfb30 msgr2=0x7fc49c0cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.159+0000 7fc49a7fc640 1 --2- 172.21.15.19:0/2608205915 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49c0cfb30 0x7fc49c0cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.159+0000 7fc49a7fc640 1 -- 172.21.15.19:0/2608205915 >> v1:172.21.15.19:6789/0 conn(0x7fc49c15db50 legacy=0x7fc49c0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.159+0000 7fc49a7fc640 1 -- 172.21.15.19:0/2608205915 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0cc590 con 0x7fc49c0cd070 2024-04-25T09:53:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.159+0000 7fc49a7fc640 1 --2- 172.21.15.19:0/2608205915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc49c0cd070 0x7fc49c0cd4d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4840097a0 tx=0x7fc48402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcea1727a5879ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.160+0000 7fc499ffb640 1 -- 172.21.15.19:0/2608205915 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484037070 con 0x7fc49c0cd070 2024-04-25T09:53:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.160+0000 7fc499ffb640 1 -- 172.21.15.19:0/2608205915 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48402ae50 con 0x7fc49c0cd070 2024-04-25T09:53:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.160+0000 7fc499ffb640 1 -- 172.21.15.19:0/2608205915 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484032660 con 0x7fc49c0cd070 2024-04-25T09:53:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.160+0000 7fc4a144a640 1 -- 172.21.15.19:0/2608205915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc49c0cd070 msgr2=0x7fc49c0cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.160+0000 7fc4a144a640 1 --2- 172.21.15.19:0/2608205915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc49c0cd070 0x7fc49c0cd4d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4840097a0 tx=0x7fc48402a100 comp rx=0 tx=0).stop 2024-04-25T09:53:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.160+0000 7fc4a144a640 1 -- 172.21.15.19:0/2608205915 shutdown_connections 2024-04-25T09:53:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.160+0000 7fc4a144a640 1 --2- 172.21.15.19:0/2608205915 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49c0cfb30 0x7fc49c0cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.160+0000 7fc4a144a640 1 --2- 172.21.15.19:0/2608205915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc49c0cd070 0x7fc49c0cd4d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.160+0000 7fc4a144a640 1 -- 172.21.15.19:0/2608205915 >> 172.21.15.19:0/2608205915 conn(0x7fc49c0d1800 msgr2=0x7fc49c0d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.160+0000 7fc4a144a640 1 -- 172.21.15.19:0/2608205915 shutdown_connections 2024-04-25T09:53:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.160+0000 7fc4a144a640 1 -- 172.21.15.19:0/2608205915 wait complete. 2024-04-25T09:53:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.161+0000 7fc4a144a640 1 Processor -- start 2024-04-25T09:53:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.162+0000 7fc4a144a640 1 -- start start 2024-04-25T09:53:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.162+0000 7fc4a144a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49c0cd070 0x7fc49c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.162+0000 7fc4a144a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc49c0cfb30 0x7fc49c158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.162+0000 7fc4a144a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49c15db50 0x7fc49c14c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.162+0000 7fc4a144a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0c7220 con 0x7fc49c0cfb30 2024-04-25T09:53:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.162+0000 7fc4a144a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0ccb50 con 0x7fc49c15db50 2024-04-25T09:53:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.162+0000 7fc4a144a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0c73a0 con 0x7fc49c0cd070 2024-04-25T09:53:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.162+0000 7fc49a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc49c0cfb30 0x7fc49c158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.162+0000 7fc49a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc49c0cfb30 0x7fc49c158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35492/0 (socket says 172.21.15.19:35492) 2024-04-25T09:53:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.162+0000 7fc49a7fc640 1 -- 172.21.15.19:0/951137491 learned_addr learned my addr 172.21.15.19:0/951137491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc49b7fe640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49c15db50 0x7fc49c14c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc49affd640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49c0cd070 0x7fc49c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc49a7fc640 1 -- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49c0cd070 msgr2=0x7fc49c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc49a7fc640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49c0cd070 0x7fc49c06ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc49a7fc640 1 -- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49c15db50 msgr2=0x7fc49c14c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc49a7fc640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49c15db50 0x7fc49c14c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc49a7fc640 1 -- 172.21.15.19:0/951137491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c1f71c0 con 0x7fc49c0cfb30 2024-04-25T09:53:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc49b7fe640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49c15db50 0x7fc49c14c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc49affd640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49c0cd070 0x7fc49c06ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc49a7fc640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc49c0cfb30 0x7fc49c158830 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc484035040 tx=0x7fc4840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc4937fe640 1 -- 172.21.15.19:0/951137491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484041070 con 0x7fc49c0cfb30 2024-04-25T09:53:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc4937fe640 1 -- 172.21.15.19:0/951137491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48403c460 con 0x7fc49c0cfb30 2024-04-25T09:53:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc4a144a640 1 -- 172.21.15.19:0/951137491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c1f74a0 con 0x7fc49c0cfb30 2024-04-25T09:53:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc4937fe640 1 -- 172.21.15.19:0/951137491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484037040 con 0x7fc49c0cfb30 2024-04-25T09:53:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.163+0000 7fc4a144a640 1 -- 172.21.15.19:0/951137491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c1f6e50 con 0x7fc49c0cfb30 2024-04-25T09:53:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.164+0000 7fc4a144a640 1 -- 172.21.15.19:0/951137491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c073990 con 0x7fc49c0cfb30 2024-04-25T09:53:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.164+0000 7fc4937fe640 1 -- 172.21.15.19:0/951137491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc48403a070 con 0x7fc49c0cfb30 2024-04-25T09:53:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.165+0000 7fc4937fe640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 0x7fc47c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.165+0000 7fc4937fe640 1 -- 172.21.15.19:0/951137491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc4840aa6e0 con 0x7fc49c0cfb30 2024-04-25T09:53:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.165+0000 7fc49affd640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 0x7fc47c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.166+0000 7fc49affd640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 0x7fc47c06d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc494005880 tx=0x7fc494009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.169+0000 7fc4937fe640 1 -- 172.21.15.19:0/951137491 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc484079340 con 0x7fc49c0cfb30 2024-04-25T09:53:19.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.351+0000 7fc4a144a640 1 -- 172.21.15.19:0/951137491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc49c0c6e30 con 0x7fc49c0cfb30 2024-04-25T09:53:19.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.352+0000 7fc4937fe640 1 -- 172.21.15.19:0/951137491 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fc484078c40 con 0x7fc49c0cfb30 2024-04-25T09:53:19.354 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:19.354 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 -- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 msgr2=0x7fc47c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 0x7fc47c06d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc494005880 tx=0x7fc494009040 comp rx=0 tx=0).stop 2024-04-25T09:53:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 -- 172.21.15.19:0/951137491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc49c0cfb30 msgr2=0x7fc49c158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc49c0cfb30 0x7fc49c158830 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc484035040 tx=0x7fc4840046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 -- 172.21.15.19:0/951137491 shutdown_connections 2024-04-25T09:53:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 0x7fc47c06d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc49c15db50 0x7fc49c14c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc49c0cfb30 0x7fc49c158830 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 --2- 172.21.15.19:0/951137491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49c0cd070 0x7fc49c06ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 -- 172.21.15.19:0/951137491 >> 172.21.15.19:0/951137491 conn(0x7fc49c0d1800 msgr2=0x7fc49c155fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 -- 172.21.15.19:0/951137491 shutdown_connections 2024-04-25T09:53:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.354+0000 7fc4a144a640 1 -- 172.21.15.19:0/951137491 wait complete. 2024-04-25T09:53:19.367 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:19.367 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.468+0000 7fbc27a9e640 1 Processor -- start 2024-04-25T09:53:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.468+0000 7fbc27a9e640 1 -- start start 2024-04-25T09:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.469+0000 7fbc27a9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbc2015dc20 0x7fbc200ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.469+0000 7fbc27a9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbc20161710 0x7fbc200cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.469+0000 7fbc27a9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2015e000 con 0x7fbc200b9330 2024-04-25T09:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.469+0000 7fbc27a9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc200d1de0 con 0x7fbc20161710 2024-04-25T09:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.469+0000 7fbc27a9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc20161c60 con 0x7fbc2015dc20 2024-04-25T09:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.469+0000 7fbc25039640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbc20161710 0x7fbc200cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.469+0000 7fbc2603b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbc200b9330 0x7fbc2006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46792/0 (socket says 172.21.15.19:46792) 2024-04-25T09:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.469+0000 7fbc2603b640 1 -- 172.21.15.19:0/3417549396 learned_addr learned my addr 172.21.15.19:0/3417549396 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.469+0000 7fbc2583a640 1 --2- 172.21.15.19:0/3417549396 >> v2:172.21.15.112:3300/0 conn(0x7fbc2015dc20 0x7fbc200ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.470+0000 7fbc24838640 1 -- 172.21.15.19:0/3417549396 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680937246 0 0) 0x7fbc2015e000 con 0x7fbc200b9330 2024-04-25T09:53:19.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.470+0000 7fbc24838640 1 -- 172.21.15.19:0/3417549396 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc08003520 con 0x7fbc200b9330 2024-04-25T09:53:19.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.470+0000 7fbc25039640 1 -- 172.21.15.19:0/3417549396 >> v2:172.21.15.112:3300/0 conn(0x7fbc2015dc20 msgr2=0x7fbc200ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:19.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.470+0000 7fbc25039640 1 --2- 172.21.15.19:0/3417549396 >> v2:172.21.15.112:3300/0 conn(0x7fbc2015dc20 0x7fbc200ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.470+0000 7fbc25039640 1 -- 172.21.15.19:0/3417549396 >> v1:172.21.15.19:6789/0 conn(0x7fbc200b9330 legacy=0x7fbc2006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.470+0000 7fbc25039640 1 -- 172.21.15.19:0/3417549396 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc20161f40 con 0x7fbc20161710 2024-04-25T09:53:19.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.470+0000 7fbc25039640 1 --2- 172.21.15.19:0/3417549396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc20161710 0x7fbc200cebb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc200d1520 tx=0x7fbc1402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3ab487495a97c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.471+0000 7fbc24838640 1 -- 172.21.15.19:0/3417549396 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14037070 con 0x7fbc20161710 2024-04-25T09:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.471+0000 7fbc24838640 1 -- 172.21.15.19:0/3417549396 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc14032440 con 0x7fbc20161710 2024-04-25T09:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.471+0000 7fbc24838640 1 -- 172.21.15.19:0/3417549396 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14032810 con 0x7fbc20161710 2024-04-25T09:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.471+0000 7fbc27a9e640 1 -- 172.21.15.19:0/3417549396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc20161710 msgr2=0x7fbc200cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.471+0000 7fbc27a9e640 1 --2- 172.21.15.19:0/3417549396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc20161710 0x7fbc200cebb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc200d1520 tx=0x7fbc1402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.471+0000 7fbc27a9e640 1 -- 172.21.15.19:0/3417549396 shutdown_connections 2024-04-25T09:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.471+0000 7fbc27a9e640 1 --2- 172.21.15.19:0/3417549396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc20161710 0x7fbc200cebb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.471+0000 7fbc27a9e640 1 --2- 172.21.15.19:0/3417549396 >> v2:172.21.15.112:3300/0 conn(0x7fbc2015dc20 0x7fbc200ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.471+0000 7fbc27a9e640 1 -- 172.21.15.19:0/3417549396 >> 172.21.15.19:0/3417549396 conn(0x7fbc200d0420 msgr2=0x7fbc200d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.471+0000 7fbc27a9e640 1 -- 172.21.15.19:0/3417549396 shutdown_connections 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.471+0000 7fbc27a9e640 1 -- 172.21.15.19:0/3417549396 wait complete. 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.472+0000 7fbc27a9e640 1 Processor -- start 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.472+0000 7fbc27a9e640 1 -- start start 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc27a9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc200b9330 0x7fbc2015f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc27a9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc2015dc20 0x7fbc20154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc27a9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc20161710 0x7fbc20155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc27a9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc200b5e50 con 0x7fbc20161710 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc27a9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc20162500 con 0x7fbc2015dc20 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc27a9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc200b5fd0 con 0x7fbc200b9330 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc2603b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc20161710 0x7fbc20155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc2603b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc20161710 0x7fbc20155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35512/0 (socket says 172.21.15.19:35512) 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc2603b640 1 -- 172.21.15.19:0/414000383 learned_addr learned my addr 172.21.15.19:0/414000383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc25039640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc2015dc20 0x7fbc20154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc2583a640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc200b9330 0x7fbc2015f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc2603b640 1 -- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc200b9330 msgr2=0x7fbc2015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc2603b640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc200b9330 0x7fbc2015f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc2603b640 1 -- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc2015dc20 msgr2=0x7fbc20154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc2603b640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc2015dc20 0x7fbc20154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc2603b640 1 -- 172.21.15.19:0/414000383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc200cdc70 con 0x7fbc20161710 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.473+0000 7fbc2583a640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc200b9330 0x7fbc2015f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.474+0000 7fbc2603b640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc20161710 0x7fbc20155040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c9f0 tx=0x7fbc1000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.474+0000 7fbc1e7fc640 1 -- 172.21.15.19:0/414000383 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10017070 con 0x7fbc20161710 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.474+0000 7fbc1e7fc640 1 -- 172.21.15.19:0/414000383 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1000ab70 con 0x7fbc20161710 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.474+0000 7fbc1e7fc640 1 -- 172.21.15.19:0/414000383 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10012400 con 0x7fbc20161710 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.474+0000 7fbc27a9e640 1 -- 172.21.15.19:0/414000383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2015fd00 con 0x7fbc20161710 2024-04-25T09:53:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.474+0000 7fbc27a9e640 1 -- 172.21.15.19:0/414000383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc20160430 con 0x7fbc20161710 2024-04-25T09:53:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.475+0000 7fbc27a9e640 1 -- 172.21.15.19:0/414000383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc200cf940 con 0x7fbc20161710 2024-04-25T09:53:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.475+0000 7fbc1e7fc640 1 -- 172.21.15.19:0/414000383 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc1001a070 con 0x7fbc20161710 2024-04-25T09:53:19.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.476+0000 7fbc1e7fc640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0c06b4d0 0x7fbc0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.476+0000 7fbc1e7fc640 1 -- 172.21.15.19:0/414000383 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbc1008a000 con 0x7fbc20161710 2024-04-25T09:53:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.476+0000 7fbc2583a640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0c06b4d0 0x7fbc0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.476+0000 7fbc2583a640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0c06b4d0 0x7fbc0c06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009af0 tx=0x7fbc18008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.480+0000 7fbc1e7fc640 1 -- 172.21.15.19:0/414000383 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc200cf940 con 0x7fbc20161710 2024-04-25T09:53:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.662+0000 7fbc27a9e640 1 -- 172.21.15.19:0/414000383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc200d1150 con 0x7fbc20161710 2024-04-25T09:53:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.662+0000 7fbc1e7fc640 1 -- 172.21.15.19:0/414000383 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc10058580 con 0x7fbc20161710 2024-04-25T09:53:19.665 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.664+0000 7fbc27a9e640 1 -- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0c06b4d0 msgr2=0x7fbc0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.664+0000 7fbc27a9e640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0c06b4d0 0x7fbc0c06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009af0 tx=0x7fbc18008040 comp rx=0 tx=0).stop 2024-04-25T09:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.664+0000 7fbc27a9e640 1 -- 172.21.15.19:0/414000383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc20161710 msgr2=0x7fbc20155040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.664+0000 7fbc27a9e640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc20161710 0x7fbc20155040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c9f0 tx=0x7fbc1000ceb0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.665+0000 7fbc27a9e640 1 -- 172.21.15.19:0/414000383 shutdown_connections 2024-04-25T09:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.665+0000 7fbc27a9e640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0c06b4d0 0x7fbc0c06d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.665+0000 7fbc27a9e640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc20161710 0x7fbc20155040 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.665+0000 7fbc27a9e640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc2015dc20 0x7fbc20154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.665+0000 7fbc27a9e640 1 --2- 172.21.15.19:0/414000383 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc200b9330 0x7fbc2015f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.665+0000 7fbc27a9e640 1 -- 172.21.15.19:0/414000383 >> 172.21.15.19:0/414000383 conn(0x7fbc200d0420 msgr2=0x7fbc2015cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.665+0000 7fbc27a9e640 1 -- 172.21.15.19:0/414000383 shutdown_connections 2024-04-25T09:53:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.665+0000 7fbc27a9e640 1 -- 172.21.15.19:0/414000383 wait complete. 2024-04-25T09:53:19.677 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:19.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.778+0000 7f30dbb0a640 1 Processor -- start 2024-04-25T09:53:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.779+0000 7f30dbb0a640 1 -- start start 2024-04-25T09:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.779+0000 7f30dbb0a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f30d415dc20 0x7f30d40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.779+0000 7f30dbb0a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d4161710 0x7f30d40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.779+0000 7f30dbb0a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d415e000 con 0x7f30d40b9330 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.779+0000 7f30dbb0a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d40d1de0 con 0x7f30d415dc20 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.779+0000 7f30dbb0a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4161c60 con 0x7f30d4161710 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30da0a7640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f30d40b9330 0x7f30d406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46804/0 (socket says 172.21.15.19:46804) 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30da0a7640 1 -- 172.21.15.19:0/3036905418 learned_addr learned my addr 172.21.15.19:0/3036905418 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d98a6640 1 --2- 172.21.15.19:0/3036905418 >> v2:172.21.15.112:3300/0 conn(0x7f30d415dc20 0x7f30d40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510763431 0 0) 0x7f30d415e000 con 0x7f30d40b9330 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d90a5640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d4161710 0x7f30d40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30bc003520 con 0x7f30d40b9330 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2674139947 0 0) 0x7f30bc003520 con 0x7f30d40b9330 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d4161710 msgr2=0x7f30d40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d4161710 0x7f30d40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 >> v2:172.21.15.112:3300/0 conn(0x7f30d415dc20 msgr2=0x7f30d40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 --2- 172.21.15.19:0/3036905418 >> v2:172.21.15.112:3300/0 conn(0x7f30d415dc20 0x7f30d40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d4161f40 con 0x7f30d40b9330 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f30c40019f0 con 0x7f30d40b9330 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 >> v1:172.21.15.19:6789/0 conn(0x7f30d40b9330 legacy=0x7f30d406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 0x7f30bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d4161710 0x7f30bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30bc004d40 0x7f30bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40d1de0 con 0x7f30bc0039b0 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4161c60 con 0x7f30bc004d40 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc00b9c0 con 0x7f30d4161710 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.780+0000 7f30d98a6640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 0x7f30bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.781+0000 7f30da0a7640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30bc004d40 0x7f30bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.781+0000 7f30d90a5640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d4161710 0x7f30bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.781+0000 7f30d98a6640 1 -- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d4161710 msgr2=0x7f30bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.781+0000 7f30d98a6640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d4161710 0x7f30bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.781+0000 7f30d98a6640 1 -- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30bc004d40 msgr2=0x7f30bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.781+0000 7f30d98a6640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30bc004d40 0x7f30bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.781+0000 7f30d98a6640 1 -- 172.21.15.19:0/3036905418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc003520 con 0x7f30bc0039b0 2024-04-25T09:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.781+0000 7f30d90a5640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d4161710 0x7f30bc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.781+0000 7f30d98a6640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 0x7f30bc003dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009ac0 tx=0x7f30bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12ce53230459961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.782+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc036070 con 0x7f30bc0039b0 2024-04-25T09:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.782+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc031400 con 0x7f30bc0039b0 2024-04-25T09:53:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.782+0000 7f30d88a4640 1 -- 172.21.15.19:0/3036905418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc009690 con 0x7f30bc0039b0 2024-04-25T09:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.782+0000 7f30dbb0a640 1 -- 172.21.15.19:0/3036905418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 msgr2=0x7f30bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.782+0000 7f30dbb0a640 1 --2- 172.21.15.19:0/3036905418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 0x7f30bc003dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009ac0 tx=0x7f30bc005830 comp rx=0 tx=0).stop 2024-04-25T09:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.782+0000 7f30da0a7640 1 -- 172.21.15.19:0/3036905418 reap_dead start 2024-04-25T09:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.782+0000 7f30dbb0a640 1 -- 172.21.15.19:0/3036905418 shutdown_connections 2024-04-25T09:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.782+0000 7f30dbb0a640 1 -- 172.21.15.19:0/3036905418 >> 172.21.15.19:0/3036905418 conn(0x7f30d40d0420 msgr2=0x7f30d40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.782+0000 7f30dbb0a640 1 -- 172.21.15.19:0/3036905418 shutdown_connections 2024-04-25T09:53:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.782+0000 7f30dbb0a640 1 -- 172.21.15.19:0/3036905418 wait complete. 2024-04-25T09:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.783+0000 7f30dbb0a640 1 Processor -- start 2024-04-25T09:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.783+0000 7f30dbb0a640 1 -- start start 2024-04-25T09:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.784+0000 7f30dbb0a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 0x7f30d40b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.784+0000 7f30dbb0a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d415a250 0x7f30d415fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.784+0000 7f30dbb0a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40aaa70 0x7f30d40aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.784+0000 7f30dbb0a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40b5e50 con 0x7f30bc0039b0 2024-04-25T09:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.784+0000 7f30dbb0a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4162500 con 0x7f30d40aaa70 2024-04-25T09:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.784+0000 7f30dbb0a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40b5fd0 con 0x7f30d415a250 2024-04-25T09:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.784+0000 7f30d98a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 0x7f30d40b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.784+0000 7f30d98a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 0x7f30d40b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35534/0 (socket says 172.21.15.19:35534) 2024-04-25T09:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.784+0000 7f30d98a6640 1 -- 172.21.15.19:0/2861567960 learned_addr learned my addr 172.21.15.19:0/2861567960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.785+0000 7f30d98a6640 1 -- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d415a250 msgr2=0x7f30d415fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.785+0000 7f30d98a6640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d415a250 0x7f30d415fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.785+0000 7f30d98a6640 1 -- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40aaa70 msgr2=0x7f30d40aaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.785+0000 7f30d98a6640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40aaa70 0x7f30d40aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.785+0000 7f30d98a6640 1 -- 172.21.15.19:0/2861567960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d4160120 con 0x7f30bc0039b0 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.785+0000 7f30d98a6640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 0x7f30d40b2650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f30cc034040 tx=0x7f30cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.785+0000 7f30d27fc640 1 -- 172.21.15.19:0/2861567960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc040020 con 0x7f30bc0039b0 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.785+0000 7f30d27fc640 1 -- 172.21.15.19:0/2861567960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc0038d0 con 0x7f30bc0039b0 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.785+0000 7f30dbb0a640 1 -- 172.21.15.19:0/2861567960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d41603f0 con 0x7f30bc0039b0 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.785+0000 7f30d27fc640 1 -- 172.21.15.19:0/2861567960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc036040 con 0x7f30bc0039b0 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.785+0000 7f30dbb0a640 1 -- 172.21.15.19:0/2861567960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d41609b0 con 0x7f30bc0039b0 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.786+0000 7f30dbb0a640 1 -- 172.21.15.19:0/2861567960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d40cfc20 con 0x7f30bc0039b0 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.786+0000 7f30d27fc640 1 -- 172.21.15.19:0/2861567960 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30cc043020 con 0x7f30bc0039b0 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.787+0000 7f30d27fc640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30c006b4d0 0x7f30c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.787+0000 7f30d27fc640 1 -- 172.21.15.19:0/2861567960 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f30cc0aa4a0 con 0x7f30bc0039b0 2024-04-25T09:53:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.787+0000 7f30d90a5640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30c006b4d0 0x7f30c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.788+0000 7f30d90a5640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30c006b4d0 0x7f30c006d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f30d4161040 tx=0x7f30c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.791+0000 7f30d27fc640 1 -- 172.21.15.19:0/2861567960 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30d40cfc20 con 0x7f30bc0039b0 2024-04-25T09:53:19.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.975+0000 7f30dbb0a640 1 -- 172.21.15.19:0/2861567960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30d4073a00 con 0x7f30bc0039b0 2024-04-25T09:53:19.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.975+0000 7f30d27fc640 1 -- 172.21.15.19:0/2861567960 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f30cc04a070 con 0x7f30bc0039b0 2024-04-25T09:53:19.977 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:19.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.977+0000 7f30dbb0a640 1 -- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30c006b4d0 msgr2=0x7f30c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.977+0000 7f30dbb0a640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30c006b4d0 0x7f30c006d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f30d4161040 tx=0x7f30c8008040 comp rx=0 tx=0).stop 2024-04-25T09:53:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.977+0000 7f30dbb0a640 1 -- 172.21.15.19:0/2861567960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 msgr2=0x7f30d40b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.977+0000 7f30dbb0a640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 0x7f30d40b2650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f30cc034040 tx=0x7f30cc004730 comp rx=0 tx=0).stop 2024-04-25T09:53:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.977+0000 7f30dbb0a640 1 -- 172.21.15.19:0/2861567960 shutdown_connections 2024-04-25T09:53:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.977+0000 7f30dbb0a640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30c006b4d0 0x7f30c006d990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.977+0000 7f30dbb0a640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40aaa70 0x7f30d40aaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.977+0000 7f30dbb0a640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30d415a250 0x7f30d415fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.977+0000 7f30dbb0a640 1 --2- 172.21.15.19:0/2861567960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0039b0 0x7f30d40b2650 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.977+0000 7f30dbb0a640 1 -- 172.21.15.19:0/2861567960 >> 172.21.15.19:0/2861567960 conn(0x7f30d40d0420 msgr2=0x7f30d40b8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.977+0000 7f30dbb0a640 1 -- 172.21.15.19:0/2861567960 shutdown_connections 2024-04-25T09:53:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:19.978+0000 7f30dbb0a640 1 -- 172.21.15.19:0/2861567960 wait complete. 2024-04-25T09:53:19.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:20.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.091+0000 7fc6a53f4640 1 Processor -- start 2024-04-25T09:53:20.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.091+0000 7fc6a53f4640 1 -- start start 2024-04-25T09:53:20.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.092+0000 7fc6a53f4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc6a00caa60 0x7fc6a00cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.092+0000 7fc6a53f4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6a006bbf0 0x7fc6a006c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.092+0000 7fc6a53f4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a00cc6c0 con 0x7fc6a015dbd0 2024-04-25T09:53:20.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.092+0000 7fc6a53f4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a006c5a0 con 0x7fc6a006bbf0 2024-04-25T09:53:20.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.092+0000 7fc6a53f4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a00cae90 con 0x7fc6a00caa60 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.092+0000 7fc69f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6a006bbf0 0x7fc6a006c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.092+0000 7fc69effd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc6a015dbd0 0x7fc6a006ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46806/0 (socket says 172.21.15.19:46806) 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.092+0000 7fc69effd640 1 -- 172.21.15.19:0/2471622747 learned_addr learned my addr 172.21.15.19:0/2471622747 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.093+0000 7fc69e7fc640 1 --2- 172.21.15.19:0/2471622747 >> v2:172.21.15.112:3300/0 conn(0x7fc6a00caa60 0x7fc6a00cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.093+0000 7fc69dffb640 1 -- 172.21.15.19:0/2471622747 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433064378 0 0) 0x7fc6a00cc6c0 con 0x7fc6a015dbd0 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.093+0000 7fc69dffb640 1 -- 172.21.15.19:0/2471622747 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc68c003520 con 0x7fc6a015dbd0 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.093+0000 7fc69f7fe640 1 -- 172.21.15.19:0/2471622747 >> v2:172.21.15.112:3300/0 conn(0x7fc6a00caa60 msgr2=0x7fc6a00cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.093+0000 7fc69f7fe640 1 --2- 172.21.15.19:0/2471622747 >> v2:172.21.15.112:3300/0 conn(0x7fc6a00caa60 0x7fc6a00cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.093+0000 7fc69f7fe640 1 -- 172.21.15.19:0/2471622747 >> v1:172.21.15.19:6789/0 conn(0x7fc6a015dbd0 legacy=0x7fc6a006ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.093+0000 7fc69f7fe640 1 -- 172.21.15.19:0/2471622747 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a00cb0e0 con 0x7fc6a006bbf0 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.093+0000 7fc69f7fe640 1 --2- 172.21.15.19:0/2471622747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a006bbf0 0x7fc6a006c060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0161710 tx=0x7fc69002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ba184b1a3c2a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.094+0000 7fc69dffb640 1 -- 172.21.15.19:0/2471622747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690007740 con 0x7fc6a006bbf0 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.094+0000 7fc69dffb640 1 -- 172.21.15.19:0/2471622747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6900078e0 con 0x7fc6a006bbf0 2024-04-25T09:53:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.094+0000 7fc69dffb640 1 -- 172.21.15.19:0/2471622747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690007cb0 con 0x7fc6a006bbf0 2024-04-25T09:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.094+0000 7fc6a53f4640 1 -- 172.21.15.19:0/2471622747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a006bbf0 msgr2=0x7fc6a006c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.094+0000 7fc6a53f4640 1 --2- 172.21.15.19:0/2471622747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a006bbf0 0x7fc6a006c060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0161710 tx=0x7fc69002a680 comp rx=0 tx=0).stop 2024-04-25T09:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.094+0000 7fc6a53f4640 1 -- 172.21.15.19:0/2471622747 shutdown_connections 2024-04-25T09:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.094+0000 7fc6a53f4640 1 --2- 172.21.15.19:0/2471622747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a006bbf0 0x7fc6a006c060 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.094+0000 7fc6a53f4640 1 --2- 172.21.15.19:0/2471622747 >> v2:172.21.15.112:3300/0 conn(0x7fc6a00caa60 0x7fc6a00cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.094+0000 7fc6a53f4640 1 -- 172.21.15.19:0/2471622747 >> 172.21.15.19:0/2471622747 conn(0x7fc6a0071af0 msgr2=0x7fc6a0071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.094+0000 7fc6a53f4640 1 -- 172.21.15.19:0/2471622747 shutdown_connections 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.094+0000 7fc6a53f4640 1 -- 172.21.15.19:0/2471622747 wait complete. 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.095+0000 7fc6a53f4640 1 Processor -- start 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc6a53f4640 1 -- start start 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc6a53f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a006bbf0 0x7fc6a00b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc6a53f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6a00caa60 0x7fc6a00c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc6a53f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a015dbd0 0x7fc6a01557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc6a53f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00cb980 con 0x7fc6a015dbd0 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc6a53f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00cb6a0 con 0x7fc6a006bbf0 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc6a53f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00cbc60 con 0x7fc6a00caa60 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc69f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a015dbd0 0x7fc6a01557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc69f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a015dbd0 0x7fc6a01557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35544/0 (socket says 172.21.15.19:35544) 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc69f7fe640 1 -- 172.21.15.19:0/1674714163 learned_addr learned my addr 172.21.15.19:0/1674714163 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc69e7fc640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6a00caa60 0x7fc6a00c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.096+0000 7fc69effd640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a006bbf0 0x7fc6a00b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc69f7fe640 1 -- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6a00caa60 msgr2=0x7fc6a00c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc69f7fe640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6a00caa60 0x7fc6a00c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc69f7fe640 1 -- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a006bbf0 msgr2=0x7fc6a00b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc69f7fe640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a006bbf0 0x7fc6a00b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc69f7fe640 1 -- 172.21.15.19:0/1674714163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a01f4ac0 con 0x7fc6a015dbd0 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc69e7fc640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6a00caa60 0x7fc6a00c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc69effd640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a006bbf0 0x7fc6a00b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc69f7fe640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a015dbd0 0x7fc6a01557c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc69002ac70 tx=0x7fc690004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc6977fe640 1 -- 172.21.15.19:0/1674714163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690040070 con 0x7fc6a015dbd0 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc6977fe640 1 -- 172.21.15.19:0/1674714163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc690031bc0 con 0x7fc6a015dbd0 2024-04-25T09:53:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc6a53f4640 1 -- 172.21.15.19:0/1674714163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a01f4cd0 con 0x7fc6a015dbd0 2024-04-25T09:53:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc6977fe640 1 -- 172.21.15.19:0/1674714163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690009450 con 0x7fc6a015dbd0 2024-04-25T09:53:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.097+0000 7fc6a53f4640 1 -- 172.21.15.19:0/1674714163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a01f42c0 con 0x7fc6a015dbd0 2024-04-25T09:53:20.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.098+0000 7fc6a53f4640 1 -- 172.21.15.19:0/1674714163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a0073a00 con 0x7fc6a015dbd0 2024-04-25T09:53:20.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.098+0000 7fc6977fe640 1 -- 172.21.15.19:0/1674714163 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc690039040 con 0x7fc6a015dbd0 2024-04-25T09:53:20.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.099+0000 7fc6977fe640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc68006b4d0 0x7fc68006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.099+0000 7fc6977fe640 1 -- 172.21.15.19:0/1674714163 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc6900abdc0 con 0x7fc6a015dbd0 2024-04-25T09:53:20.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.099+0000 7fc69effd640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc68006b4d0 0x7fc68006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.100+0000 7fc69effd640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc68006b4d0 0x7fc68006d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc698007fd0 tx=0x7fc698009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.103+0000 7fc6977fe640 1 -- 172.21.15.19:0/1674714163 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc69007aa20 con 0x7fc6a015dbd0 2024-04-25T09:53:20.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.281+0000 7fc6a53f4640 1 -- 172.21.15.19:0/1674714163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6a006f830 con 0x7fc6a015dbd0 2024-04-25T09:53:20.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.282+0000 7fc6977fe640 1 -- 172.21.15.19:0/1674714163 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc69007a320 con 0x7fc6a015dbd0 2024-04-25T09:53:20.284 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:20.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 -- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc68006b4d0 msgr2=0x7fc68006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc68006b4d0 0x7fc68006d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc698007fd0 tx=0x7fc698009040 comp rx=0 tx=0).stop 2024-04-25T09:53:20.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 -- 172.21.15.19:0/1674714163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a015dbd0 msgr2=0x7fc6a01557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a015dbd0 0x7fc6a01557c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc69002ac70 tx=0x7fc690004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:20.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 -- 172.21.15.19:0/1674714163 shutdown_connections 2024-04-25T09:53:20.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc68006b4d0 0x7fc68006d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a015dbd0 0x7fc6a01557c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6a00caa60 0x7fc6a00c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 --2- 172.21.15.19:0/1674714163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a006bbf0 0x7fc6a00b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 -- 172.21.15.19:0/1674714163 >> 172.21.15.19:0/1674714163 conn(0x7fc6a0071af0 msgr2=0x7fc6a00afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:20.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 -- 172.21.15.19:0/1674714163 shutdown_connections 2024-04-25T09:53:20.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.284+0000 7fc6a53f4640 1 -- 172.21.15.19:0/1674714163 wait complete. 2024-04-25T09:53:20.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.399+0000 7fa0f82c4640 1 Processor -- start 2024-04-25T09:53:20.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.399+0000 7fa0f82c4640 1 -- start start 2024-04-25T09:53:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.400+0000 7fa0f82c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f015dc20 0x7fa0f00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.400+0000 7fa0f82c4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0f00d32f0 0x7fa0f00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.400+0000 7fa0f82c4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f015e000 con 0x7fa0f0161710 2024-04-25T09:53:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.400+0000 7fa0f82c4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f00d1de0 con 0x7fa0f00d32f0 2024-04-25T09:53:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.400+0000 7fa0f82c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f0161c60 con 0x7fa0f015dc20 2024-04-25T09:53:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.400+0000 7fa0f6861640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0f00d32f0 0x7fa0f00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.400+0000 7fa0f6861640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0f00d32f0 0x7fa0f00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35550/0 (socket says 172.21.15.19:35550) 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.400+0000 7fa0f6861640 1 -- 172.21.15.19:0/194956632 learned_addr learned my addr 172.21.15.19:0/194956632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.401+0000 7fa0f6060640 1 --2- 172.21.15.19:0/194956632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f015dc20 0x7fa0f00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.401+0000 7fa0f6861640 1 -- 172.21.15.19:0/194956632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f015dc20 msgr2=0x7fa0f00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.401+0000 7fa0f6861640 1 --2- 172.21.15.19:0/194956632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f015dc20 0x7fa0f00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.401+0000 7fa0f6861640 1 -- 172.21.15.19:0/194956632 >> v1:172.21.15.112:6789/0 conn(0x7fa0f0161710 legacy=0x7fa0f006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.401+0000 7fa0f6861640 1 -- 172.21.15.19:0/194956632 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f0161f40 con 0x7fa0f00d32f0 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.401+0000 7fa0f6861640 1 --2- 172.21.15.19:0/194956632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f00d32f0 0x7fa0f00cebb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009870 tx=0x7fa0e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6e94f1782ff855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.402+0000 7fa0f505e640 1 -- 172.21.15.19:0/194956632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8037070 con 0x7fa0f00d32f0 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.402+0000 7fa0f505e640 1 -- 172.21.15.19:0/194956632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e8032440 con 0x7fa0f00d32f0 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.402+0000 7fa0f505e640 1 -- 172.21.15.19:0/194956632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8032810 con 0x7fa0f00d32f0 2024-04-25T09:53:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.402+0000 7fa0f82c4640 1 -- 172.21.15.19:0/194956632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f00d32f0 msgr2=0x7fa0f00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.402+0000 7fa0f82c4640 1 --2- 172.21.15.19:0/194956632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f00d32f0 0x7fa0f00cebb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009870 tx=0x7fa0e802a270 comp rx=0 tx=0).stop 2024-04-25T09:53:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.402+0000 7fa0f82c4640 1 -- 172.21.15.19:0/194956632 shutdown_connections 2024-04-25T09:53:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.402+0000 7fa0f82c4640 1 --2- 172.21.15.19:0/194956632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f00d32f0 0x7fa0f00cebb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.402+0000 7fa0f82c4640 1 --2- 172.21.15.19:0/194956632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f015dc20 0x7fa0f00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.402+0000 7fa0f82c4640 1 -- 172.21.15.19:0/194956632 >> 172.21.15.19:0/194956632 conn(0x7fa0f00d0420 msgr2=0x7fa0f00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.402+0000 7fa0f82c4640 1 -- 172.21.15.19:0/194956632 shutdown_connections 2024-04-25T09:53:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.402+0000 7fa0f82c4640 1 -- 172.21.15.19:0/194956632 wait complete. 2024-04-25T09:53:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.403+0000 7fa0f82c4640 1 Processor -- start 2024-04-25T09:53:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.403+0000 7fa0f82c4640 1 -- start start 2024-04-25T09:53:20.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.404+0000 7fa0f82c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f00d32f0 0x7fa0f0156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.404+0000 7fa0f82c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f015dc20 0x7fa0f01603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.404+0000 7fa0f82c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f0161710 0x7fa0f01608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.404+0000 7fa0f82c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00b5e50 con 0x7fa0f015dc20 2024-04-25T09:53:20.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.404+0000 7fa0f82c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f0162500 con 0x7fa0f0161710 2024-04-25T09:53:20.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.404+0000 7fa0f82c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00b5fd0 con 0x7fa0f00d32f0 2024-04-25T09:53:20.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.404+0000 7fa0f585f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f015dc20 0x7fa0f01603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.404+0000 7fa0f585f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f015dc20 0x7fa0f01603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35566/0 (socket says 172.21.15.19:35566) 2024-04-25T09:53:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.404+0000 7fa0f585f640 1 -- 172.21.15.19:0/2106280321 learned_addr learned my addr 172.21.15.19:0/2106280321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.404+0000 7fa0f6060640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f00d32f0 0x7fa0f0156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.404+0000 7fa0f6861640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f0161710 0x7fa0f01608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.405+0000 7fa0f585f640 1 -- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f00d32f0 msgr2=0x7fa0f0156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.405+0000 7fa0f585f640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f00d32f0 0x7fa0f0156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.405+0000 7fa0f585f640 1 -- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f0161710 msgr2=0x7fa0f01608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.405+0000 7fa0f585f640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f0161710 0x7fa0f01608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.405+0000 7fa0f585f640 1 -- 172.21.15.19:0/2106280321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00aa380 con 0x7fa0f015dc20 2024-04-25T09:53:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.405+0000 7fa0f585f640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f015dc20 0x7fa0f01603a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c430 tx=0x7fa0e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.405+0000 7fa0eeffd640 1 -- 172.21.15.19:0/2106280321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4017070 con 0x7fa0f015dc20 2024-04-25T09:53:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.405+0000 7fa0eeffd640 1 -- 172.21.15.19:0/2106280321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e40052d0 con 0x7fa0f015dc20 2024-04-25T09:53:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.405+0000 7fa0f82c4640 1 -- 172.21.15.19:0/2106280321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f00aa660 con 0x7fa0f015dc20 2024-04-25T09:53:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.405+0000 7fa0eeffd640 1 -- 172.21.15.19:0/2106280321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4003830 con 0x7fa0f015dc20 2024-04-25T09:53:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.405+0000 7fa0f82c4640 1 -- 172.21.15.19:0/2106280321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f00aac20 con 0x7fa0f015dc20 2024-04-25T09:53:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.406+0000 7fa0f82c4640 1 -- 172.21.15.19:0/2106280321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f0073a00 con 0x7fa0f015dc20 2024-04-25T09:53:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.406+0000 7fa0eeffd640 1 -- 172.21.15.19:0/2106280321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0e401a070 con 0x7fa0f015dc20 2024-04-25T09:53:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.407+0000 7fa0eeffd640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0dc06b4d0 0x7fa0dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.407+0000 7fa0eeffd640 1 -- 172.21.15.19:0/2106280321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa0e4089810 con 0x7fa0f015dc20 2024-04-25T09:53:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.407+0000 7fa0f6060640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0dc06b4d0 0x7fa0dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.407+0000 7fa0f6060640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0dc06b4d0 0x7fa0dc06d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0003aa0 tx=0x7fa0e0009830 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.411+0000 7fa0eeffd640 1 -- 172.21.15.19:0/2106280321 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0e4057d00 con 0x7fa0f015dc20 2024-04-25T09:53:20.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.589+0000 7fa0f82c4640 1 -- 172.21.15.19:0/2106280321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa0f00b6d00 con 0x7fa0f015dc20 2024-04-25T09:53:20.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.589+0000 7fa0eeffd640 1 -- 172.21.15.19:0/2106280321 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa0e4087e20 con 0x7fa0f015dc20 2024-04-25T09:53:20.591 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:20.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 -- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0dc06b4d0 msgr2=0x7fa0dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0dc06b4d0 0x7fa0dc06d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0003aa0 tx=0x7fa0e0009830 comp rx=0 tx=0).stop 2024-04-25T09:53:20.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 -- 172.21.15.19:0/2106280321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f015dc20 msgr2=0x7fa0f01603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f015dc20 0x7fa0f01603a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c430 tx=0x7fa0e400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 -- 172.21.15.19:0/2106280321 shutdown_connections 2024-04-25T09:53:20.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0dc06b4d0 0x7fa0dc06d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f0161710 0x7fa0f01608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f015dc20 0x7fa0f01603a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 --2- 172.21.15.19:0/2106280321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f00d32f0 0x7fa0f0156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 -- 172.21.15.19:0/2106280321 >> 172.21.15.19:0/2106280321 conn(0x7fa0f00d0420 msgr2=0x7fa0f00b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 -- 172.21.15.19:0/2106280321 shutdown_connections 2024-04-25T09:53:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.591+0000 7fa0f82c4640 1 -- 172.21.15.19:0/2106280321 wait complete. 2024-04-25T09:53:20.604 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:53:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.703+0000 7efc3c45d640 1 Processor -- start 2024-04-25T09:53:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.703+0000 7efc3c45d640 1 -- start start 2024-04-25T09:53:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc3c45d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc3415b7d0 0x7efc340cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc3c45d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc340b5620 0x7efc340b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc3c45d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3406cf50 con 0x7efc3406c2d0 2024-04-25T09:53:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc3c45d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc340b5fc0 con 0x7efc3415b7d0 2024-04-25T09:53:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc3c45d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc340ce530 con 0x7efc340b5620 2024-04-25T09:53:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc3a9fa640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efc3406c2d0 0x7efc3406c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46818/0 (socket says 172.21.15.19:46818) 2024-04-25T09:53:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc3a9fa640 1 -- 172.21.15.19:0/483915419 learned_addr learned my addr 172.21.15.19:0/483915419 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc3a1f9640 1 --2- 172.21.15.19:0/483915419 >> v2:172.21.15.112:3300/0 conn(0x7efc3415b7d0 0x7efc340cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc399f8640 1 --2- 172.21.15.19:0/483915419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc340b5620 0x7efc340b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc391f7640 1 -- 172.21.15.19:0/483915419 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310620502 0 0) 0x7efc3406cf50 con 0x7efc3406c2d0 2024-04-25T09:53:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc391f7640 1 -- 172.21.15.19:0/483915419 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc20003520 con 0x7efc3406c2d0 2024-04-25T09:53:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.704+0000 7efc399f8640 1 -- 172.21.15.19:0/483915419 >> v2:172.21.15.112:3300/0 conn(0x7efc3415b7d0 msgr2=0x7efc340cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.705+0000 7efc399f8640 1 --2- 172.21.15.19:0/483915419 >> v2:172.21.15.112:3300/0 conn(0x7efc3415b7d0 0x7efc340cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.705+0000 7efc399f8640 1 -- 172.21.15.19:0/483915419 >> v1:172.21.15.19:6789/0 conn(0x7efc3406c2d0 legacy=0x7efc3406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.705+0000 7efc391f7640 1 -- 172.21.15.19:0/483915419 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1034156939 0 0) 0x7efc20003520 con 0x7efc3406c2d0 2024-04-25T09:53:20.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.705+0000 7efc399f8640 1 -- 172.21.15.19:0/483915419 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc340ce810 con 0x7efc340b5620 2024-04-25T09:53:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.705+0000 7efc399f8640 1 --2- 172.21.15.19:0/483915419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc340b5620 0x7efc340b5a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efc240097a0 tx=0x7efc2402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2afdd950567986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.705+0000 7efc391f7640 1 -- 172.21.15.19:0/483915419 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24037070 con 0x7efc340b5620 2024-04-25T09:53:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.706+0000 7efc391f7640 1 -- 172.21.15.19:0/483915419 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc24032440 con 0x7efc340b5620 2024-04-25T09:53:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.706+0000 7efc391f7640 1 -- 172.21.15.19:0/483915419 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc240327f0 con 0x7efc340b5620 2024-04-25T09:53:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.706+0000 7efc3c45d640 1 -- 172.21.15.19:0/483915419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc340b5620 msgr2=0x7efc340b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.706+0000 7efc3c45d640 1 --2- 172.21.15.19:0/483915419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc340b5620 0x7efc340b5a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efc240097a0 tx=0x7efc2402a180 comp rx=0 tx=0).stop 2024-04-25T09:53:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.706+0000 7efc3c45d640 1 -- 172.21.15.19:0/483915419 shutdown_connections 2024-04-25T09:53:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.706+0000 7efc3c45d640 1 --2- 172.21.15.19:0/483915419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc340b5620 0x7efc340b5a80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.706+0000 7efc3c45d640 1 --2- 172.21.15.19:0/483915419 >> v2:172.21.15.112:3300/0 conn(0x7efc3415b7d0 0x7efc340cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.706+0000 7efc3c45d640 1 -- 172.21.15.19:0/483915419 >> 172.21.15.19:0/483915419 conn(0x7efc340d0800 msgr2=0x7efc340d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.706+0000 7efc3c45d640 1 -- 172.21.15.19:0/483915419 shutdown_connections 2024-04-25T09:53:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.706+0000 7efc3c45d640 1 -- 172.21.15.19:0/483915419 wait complete. 2024-04-25T09:53:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.707+0000 7efc3c45d640 1 Processor -- start 2024-04-25T09:53:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.707+0000 7efc3c45d640 1 -- start start 2024-04-25T09:53:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc3c45d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3406c2d0 0x7efc340c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc3c45d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc340b5620 0x7efc340b8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc3c45d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc3415b7d0 0x7efc340c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc3c45d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc340cf550 con 0x7efc3406c2d0 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc3c45d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3406d230 con 0x7efc340b5620 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc3c45d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc340cf6d0 con 0x7efc3415b7d0 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc3a1f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3406c2d0 0x7efc340c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc3a1f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3406c2d0 0x7efc340c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35578/0 (socket says 172.21.15.19:35578) 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc3a1f9640 1 -- 172.21.15.19:0/2718221068 learned_addr learned my addr 172.21.15.19:0/2718221068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc3a9fa640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc3415b7d0 0x7efc340c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc3a1f9640 1 -- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc3415b7d0 msgr2=0x7efc340c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.708+0000 7efc399f8640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc340b5620 0x7efc340b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.709+0000 7efc3a1f9640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc3415b7d0 0x7efc340c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.709+0000 7efc3a1f9640 1 -- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc340b5620 msgr2=0x7efc340b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.709+0000 7efc3a1f9640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc340b5620 0x7efc340b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.709+0000 7efc3a1f9640 1 -- 172.21.15.19:0/2718221068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc341f3a20 con 0x7efc3406c2d0 2024-04-25T09:53:20.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.709+0000 7efc399f8640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc340b5620 0x7efc340b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:20.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.709+0000 7efc3a1f9640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3406c2d0 0x7efc340c91c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efc3000e870 tx=0x7efc3000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.709+0000 7efc2affd640 1 -- 172.21.15.19:0/2718221068 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30017070 con 0x7efc3406c2d0 2024-04-25T09:53:20.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.709+0000 7efc2affd640 1 -- 172.21.15.19:0/2718221068 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3000c940 con 0x7efc3406c2d0 2024-04-25T09:53:20.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.709+0000 7efc3c45d640 1 -- 172.21.15.19:0/2718221068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc341f3f80 con 0x7efc3406c2d0 2024-04-25T09:53:20.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.709+0000 7efc2affd640 1 -- 172.21.15.19:0/2718221068 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3000cd30 con 0x7efc3406c2d0 2024-04-25T09:53:20.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.709+0000 7efc3c45d640 1 -- 172.21.15.19:0/2718221068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc340b8940 con 0x7efc3406c2d0 2024-04-25T09:53:20.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.710+0000 7efc3c45d640 1 -- 172.21.15.19:0/2718221068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc34073a00 con 0x7efc3406c2d0 2024-04-25T09:53:20.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.710+0000 7efc2affd640 1 -- 172.21.15.19:0/2718221068 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc3001a070 con 0x7efc3406c2d0 2024-04-25T09:53:20.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.711+0000 7efc2affd640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc1806b4d0 0x7efc1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.711+0000 7efc2affd640 1 -- 172.21.15.19:0/2718221068 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efc3008a3c0 con 0x7efc3406c2d0 2024-04-25T09:53:20.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.711+0000 7efc399f8640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc1806b4d0 0x7efc1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.712+0000 7efc399f8640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc1806b4d0 0x7efc1806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efc24034000 tx=0x7efc2402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.715+0000 7efc2affd640 1 -- 172.21.15.19:0/2718221068 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc341f3a20 con 0x7efc3406c2d0 2024-04-25T09:53:20.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.906+0000 7efc3c45d640 1 -- 172.21.15.19:0/2718221068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efc3406b210 con 0x7efc3406c2d0 2024-04-25T09:53:20.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.907+0000 7efc2affd640 1 -- 172.21.15.19:0/2718221068 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7efc30058ef0 con 0x7efc3406c2d0 2024-04-25T09:53:20.909 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T09:53:20.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.908+0000 7efc3c45d640 1 -- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc1806b4d0 msgr2=0x7efc1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.908+0000 7efc3c45d640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc1806b4d0 0x7efc1806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efc24034000 tx=0x7efc2402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.908+0000 7efc3c45d640 1 -- 172.21.15.19:0/2718221068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3406c2d0 msgr2=0x7efc340c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.908+0000 7efc3c45d640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3406c2d0 0x7efc340c91c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efc3000e870 tx=0x7efc3000ed30 comp rx=0 tx=0).stop 2024-04-25T09:53:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.908+0000 7efc3c45d640 1 -- 172.21.15.19:0/2718221068 shutdown_connections 2024-04-25T09:53:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.908+0000 7efc3c45d640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc1806b4d0 0x7efc1806d990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.908+0000 7efc3c45d640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc3415b7d0 0x7efc340c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.908+0000 7efc3c45d640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc340b5620 0x7efc340b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.908+0000 7efc3c45d640 1 --2- 172.21.15.19:0/2718221068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3406c2d0 0x7efc340c91c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.908+0000 7efc3c45d640 1 -- 172.21.15.19:0/2718221068 >> 172.21.15.19:0/2718221068 conn(0x7efc340d0800 msgr2=0x7efc340d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:20.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.909+0000 7efc3c45d640 1 -- 172.21.15.19:0/2718221068 shutdown_connections 2024-04-25T09:53:20.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:20.909+0000 7efc3c45d640 1 -- 172.21.15.19:0/2718221068 wait complete. 2024-04-25T09:53:20.921 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi019.front.sepia.ceph.com... 2024-04-25T09:53:20.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:20.921 DEBUG:teuthology.orchestra.run.smithi019:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:20.944 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[41990]: fuse finished with error 0 and tester_r 0 2024-04-25T09:53:20.946 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:53:20.998 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:53:20.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:20.998 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T09:53:21.019 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:21.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.158+0000 7f9766ac4640 1 Processor -- start 2024-04-25T09:53:21.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.158+0000 7f9766ac4640 1 -- start start 2024-04-25T09:53:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.159+0000 7f9766ac4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f97601602e0 0x7f97600ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.159+0000 7f9766ac4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f976015f5f0 0x7f976015fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.159+0000 7f9766ac4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97600cd710 con 0x7f976015ba00 2024-04-25T09:53:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.159+0000 7f9766ac4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9760099b70 con 0x7f97601602e0 2024-04-25T09:53:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.159+0000 7f9766ac4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9760099e50 con 0x7f976015f5f0 2024-04-25T09:53:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.159+0000 7f9765061640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f976015f5f0 0x7f976015fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.159+0000 7f975ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f97601602e0 0x7f97600ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.159+0000 7f9764860640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f976015ba00 0x7f976009a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:43298/0 (socket says 172.21.15.19:43298) 2024-04-25T09:53:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.160+0000 7f9764860640 1 -- 172.21.15.19:0/1412648618 learned_addr learned my addr 172.21.15.19:0/1412648618 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.160+0000 7f975ffff640 1 -- 172.21.15.19:0/1412648618 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f976015f5f0 msgr2=0x7f976015fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.160+0000 7f975ffff640 1 --2- 172.21.15.19:0/1412648618 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f976015f5f0 0x7f976015fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.160+0000 7f975ffff640 1 -- 172.21.15.19:0/1412648618 >> v1:172.21.15.112:6789/0 conn(0x7f976015ba00 legacy=0x7f976009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.160+0000 7f975ffff640 1 -- 172.21.15.19:0/1412648618 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976009a130 con 0x7f97601602e0 2024-04-25T09:53:21.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f975ffff640 1 --2- 172.21.15.19:0/1412648618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97601602e0 0x7f97600ccda0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97540097a0 tx=0x7f975402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98085d273a188b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f975f7fe640 1 -- 172.21.15.19:0/1412648618 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754037070 con 0x7f97601602e0 2024-04-25T09:53:21.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f975f7fe640 1 -- 172.21.15.19:0/1412648618 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975402adb0 con 0x7f97601602e0 2024-04-25T09:53:21.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f975f7fe640 1 -- 172.21.15.19:0/1412648618 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754032660 con 0x7f97601602e0 2024-04-25T09:53:21.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f9766ac4640 1 -- 172.21.15.19:0/1412648618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97601602e0 msgr2=0x7f97600ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f9766ac4640 1 --2- 172.21.15.19:0/1412648618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97601602e0 0x7f97600ccda0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97540097a0 tx=0x7f975402a060 comp rx=0 tx=0).stop 2024-04-25T09:53:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f9766ac4640 1 -- 172.21.15.19:0/1412648618 shutdown_connections 2024-04-25T09:53:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f9766ac4640 1 --2- 172.21.15.19:0/1412648618 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f976015f5f0 0x7f976015fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f9766ac4640 1 --2- 172.21.15.19:0/1412648618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97601602e0 0x7f97600ccda0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f9766ac4640 1 -- 172.21.15.19:0/1412648618 >> 172.21.15.19:0/1412648618 conn(0x7f97600b3270 msgr2=0x7f97600b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f9766ac4640 1 -- 172.21.15.19:0/1412648618 shutdown_connections 2024-04-25T09:53:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.161+0000 7f9766ac4640 1 -- 172.21.15.19:0/1412648618 wait complete. 2024-04-25T09:53:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.162+0000 7f9766ac4640 1 Processor -- start 2024-04-25T09:53:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.162+0000 7f9766ac4640 1 -- start start 2024-04-25T09:53:21.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.163+0000 7f9766ac4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f976015ba00 0x7f97600b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.163+0000 7f9766ac4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f976015f5f0 0x7f9760154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.163+0000 7f9766ac4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97601602e0 0x7f976006cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.163+0000 7f9766ac4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97600cd460 con 0x7f976015f5f0 2024-04-25T09:53:21.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.163+0000 7f9766ac4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97600cd2e0 con 0x7f97601602e0 2024-04-25T09:53:21.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.163+0000 7f9766ac4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97600c52d0 con 0x7f976015ba00 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.163+0000 7f975ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f976015f5f0 0x7f9760154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.163+0000 7f975ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f976015f5f0 0x7f9760154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35584/0 (socket says 172.21.15.19:35584) 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.163+0000 7f975ffff640 1 -- 172.21.15.19:0/4000807477 learned_addr learned my addr 172.21.15.19:0/4000807477 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f9764860640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f976015ba00 0x7f97600b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f9765061640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97601602e0 0x7f976006cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f975ffff640 1 -- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f976015ba00 msgr2=0x7f97600b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f975ffff640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f976015ba00 0x7f97600b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f975ffff640 1 -- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97601602e0 msgr2=0x7f976006cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f975ffff640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97601602e0 0x7f976006cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f975ffff640 1 -- 172.21.15.19:0/4000807477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9760155520 con 0x7f976015f5f0 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f9765061640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97601602e0 0x7f976006cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f975ffff640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f976015f5f0 0x7f9760154fe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9754000c00 tx=0x7f97540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f975d7fa640 1 -- 172.21.15.19:0/4000807477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754041070 con 0x7f976015f5f0 2024-04-25T09:53:21.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f975d7fa640 1 -- 172.21.15.19:0/4000807477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975403c460 con 0x7f976015f5f0 2024-04-25T09:53:21.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f975d7fa640 1 -- 172.21.15.19:0/4000807477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754037040 con 0x7f976015f5f0 2024-04-25T09:53:21.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f9766ac4640 1 -- 172.21.15.19:0/4000807477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976015d3f0 con 0x7f976015f5f0 2024-04-25T09:53:21.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.164+0000 7f9766ac4640 1 -- 172.21.15.19:0/4000807477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976015d970 con 0x7f976015f5f0 2024-04-25T09:53:21.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.165+0000 7f9766ac4640 1 -- 172.21.15.19:0/4000807477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9760073990 con 0x7f976015f5f0 2024-04-25T09:53:21.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.166+0000 7f975d7fa640 1 -- 172.21.15.19:0/4000807477 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9754044070 con 0x7f976015f5f0 2024-04-25T09:53:21.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.166+0000 7f975d7fa640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f974c06b4d0 0x7f974c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.166+0000 7f975d7fa640 1 -- 172.21.15.19:0/4000807477 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f97540aad40 con 0x7f976015f5f0 2024-04-25T09:53:21.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.166+0000 7f9764860640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f974c06b4d0 0x7f974c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.167+0000 7f9764860640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f974c06b4d0 0x7f974c06d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97580012e0 tx=0x7f9758001a30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.171+0000 7f975d7fa640 1 -- 172.21.15.19:0/4000807477 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97540793e0 con 0x7f976015f5f0 2024-04-25T09:53:21.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.352+0000 7f9766ac4640 1 -- 172.21.15.19:0/4000807477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97600b40a0 con 0x7f976015f5f0 2024-04-25T09:53:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.353+0000 7f975d7fa640 1 -- 172.21.15.19:0/4000807477 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f975407d480 con 0x7f976015f5f0 2024-04-25T09:53:21.355 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:21.355 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 -- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f974c06b4d0 msgr2=0x7f974c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f974c06b4d0 0x7f974c06d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97580012e0 tx=0x7f9758001a30 comp rx=0 tx=0).stop 2024-04-25T09:53:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 -- 172.21.15.19:0/4000807477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f976015f5f0 msgr2=0x7f9760154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f976015f5f0 0x7f9760154fe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9754000c00 tx=0x7f97540046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 -- 172.21.15.19:0/4000807477 shutdown_connections 2024-04-25T09:53:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f974c06b4d0 0x7f974c06d990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97601602e0 0x7f976006cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f976015f5f0 0x7f9760154fe0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 --2- 172.21.15.19:0/4000807477 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f976015ba00 0x7f97600b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 -- 172.21.15.19:0/4000807477 >> 172.21.15.19:0/4000807477 conn(0x7f97600b3270 msgr2=0x7f97600c5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:21.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 -- 172.21.15.19:0/4000807477 shutdown_connections 2024-04-25T09:53:21.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.355+0000 7f9766ac4640 1 -- 172.21.15.19:0/4000807477 wait complete. 2024-04-25T09:53:21.368 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:21.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.467+0000 7fbc02c6f640 1 Processor -- start 2024-04-25T09:53:21.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.467+0000 7fbc02c6f640 1 -- start start 2024-04-25T09:53:21.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc02c6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dbe0 0x7fbbfc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc02c6f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbfc0b92a0 0x7fbbfc0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc02c6f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc0b97a0 con 0x7fbbfc0b2cb0 2024-04-25T09:53:21.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc02c6f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbfc15dfc0 con 0x7fbbfc0b92a0 2024-04-25T09:53:21.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc02c6f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0d1a00 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc0120c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbfc0b92a0 0x7fbbfc0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc0120c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbfc0b92a0 0x7fbbfc0d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35596/0 (socket says 172.21.15.19:35596) 2024-04-25T09:53:21.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc0120c640 1 -- 172.21.15.19:0/236927197 learned_addr learned my addr 172.21.15.19:0/236927197 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:21.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc00a0b640 1 --2- 172.21.15.19:0/236927197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dbe0 0x7fbbfc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc0120c640 1 -- 172.21.15.19:0/236927197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dbe0 msgr2=0x7fbbfc0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc0120c640 1 --2- 172.21.15.19:0/236927197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dbe0 0x7fbbfc0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc0120c640 1 -- 172.21.15.19:0/236927197 >> v1:172.21.15.112:6789/0 conn(0x7fbbfc0b2cb0 legacy=0x7fbbfc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.468+0000 7fbc0120c640 1 -- 172.21.15.19:0/236927197 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc0d1c50 con 0x7fbbfc0b92a0 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.469+0000 7fbc0120c640 1 --2- 172.21.15.19:0/236927197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0b92a0 0x7fbbfc0d0e50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009e50 tx=0x7fbbf402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1756eb15b96c8ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.469+0000 7fbbfb7fe640 1 -- 172.21.15.19:0/236927197 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4036070 con 0x7fbbfc0b92a0 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.469+0000 7fbbfb7fe640 1 -- 172.21.15.19:0/236927197 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4031600 con 0x7fbbfc0b92a0 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.469+0000 7fbbfb7fe640 1 -- 172.21.15.19:0/236927197 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40319d0 con 0x7fbbfc0b92a0 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.470+0000 7fbc02c6f640 1 -- 172.21.15.19:0/236927197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0b92a0 msgr2=0x7fbbfc0d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.470+0000 7fbc02c6f640 1 --2- 172.21.15.19:0/236927197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0b92a0 0x7fbbfc0d0e50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009e50 tx=0x7fbbf402a710 comp rx=0 tx=0).stop 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.470+0000 7fbc02c6f640 1 -- 172.21.15.19:0/236927197 shutdown_connections 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.470+0000 7fbc02c6f640 1 --2- 172.21.15.19:0/236927197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0b92a0 0x7fbbfc0d0e50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.470+0000 7fbc02c6f640 1 --2- 172.21.15.19:0/236927197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dbe0 0x7fbbfc0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.470+0000 7fbc02c6f640 1 -- 172.21.15.19:0/236927197 >> 172.21.15.19:0/236927197 conn(0x7fbbfc061a20 msgr2=0x7fbbfc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.470+0000 7fbc02c6f640 1 -- 172.21.15.19:0/236927197 shutdown_connections 2024-04-25T09:53:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.470+0000 7fbc02c6f640 1 -- 172.21.15.19:0/236927197 wait complete. 2024-04-25T09:53:21.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.471+0000 7fbc02c6f640 1 Processor -- start 2024-04-25T09:53:21.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.471+0000 7fbc02c6f640 1 -- start start 2024-04-25T09:53:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc02c6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc0b2cb0 0x7fbbfc0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc02c6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbfc0b92a0 0x7fbbfc149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc02c6f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc15dbe0 0x7fbbfc0cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc02c6f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0b6ab0 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc02c6f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0d2210 con 0x7fbbfc0b92a0 2024-04-25T09:53:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc02c6f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0b6c30 con 0x7fbbfc0b2cb0 2024-04-25T09:53:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc0120c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc15dbe0 0x7fbbfc0cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc0120c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc15dbe0 0x7fbbfc0cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35600/0 (socket says 172.21.15.19:35600) 2024-04-25T09:53:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc0120c640 1 -- 172.21.15.19:0/1057247391 learned_addr learned my addr 172.21.15.19:0/1057247391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc00a0b640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc0b2cb0 0x7fbbfc0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbbfbfff640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbfc0b92a0 0x7fbbfc149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc0120c640 1 -- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc0b2cb0 msgr2=0x7fbbfc0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc0120c640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc0b2cb0 0x7fbbfc0cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc0120c640 1 -- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbfc0b92a0 msgr2=0x7fbbfc149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc0120c640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbfc0b92a0 0x7fbbfc149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc0120c640 1 -- 172.21.15.19:0/1057247391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc0cd0b0 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbbfbfff640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbfc0b92a0 0x7fbbfc149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.472+0000 7fbc0120c640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc15dbe0 0x7fbbfc0cacc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4034040 tx=0x7fbbf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.473+0000 7fbbf97fa640 1 -- 172.21.15.19:0/1057247391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4036050 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.473+0000 7fbbf97fa640 1 -- 172.21.15.19:0/1057247391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4042070 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.473+0000 7fbc02c6f640 1 -- 172.21.15.19:0/1057247391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc0cd390 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.473+0000 7fbbf97fa640 1 -- 172.21.15.19:0/1057247391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4009510 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.473+0000 7fbc02c6f640 1 -- 172.21.15.19:0/1057247391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc0cd950 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.474+0000 7fbc02c6f640 1 -- 172.21.15.19:0/1057247391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc0d3640 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.474+0000 7fbbf97fa640 1 -- 172.21.15.19:0/1057247391 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbf4007bf0 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.475+0000 7fbbf97fa640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbe006b4d0 0x7fbbe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.475+0000 7fbbf97fa640 1 -- 172.21.15.19:0/1057247391 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbbf40ab410 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.475+0000 7fbc00a0b640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbe006b4d0 0x7fbbe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.475+0000 7fbc00a0b640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbe006b4d0 0x7fbbe006d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009a30 tx=0x7fbbf0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.479+0000 7fbbf97fa640 1 -- 172.21.15.19:0/1057247391 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbbfc0d3640 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.659+0000 7fbc02c6f640 1 -- 172.21.15.19:0/1057247391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbfc073990 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.660+0000 7fbbf97fa640 1 -- 172.21.15.19:0/1057247391 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fbbf4045360 con 0x7fbbfc15dbe0 2024-04-25T09:53:21.663 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:21.663 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:21.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 -- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbe006b4d0 msgr2=0x7fbbe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbe006b4d0 0x7fbbe006d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009a30 tx=0x7fbbf0008040 comp rx=0 tx=0).stop 2024-04-25T09:53:21.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 -- 172.21.15.19:0/1057247391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc15dbe0 msgr2=0x7fbbfc0cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc15dbe0 0x7fbbfc0cacc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4034040 tx=0x7fbbf4004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:21.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 -- 172.21.15.19:0/1057247391 shutdown_connections 2024-04-25T09:53:21.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbe006b4d0 0x7fbbe006d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc15dbe0 0x7fbbfc0cacc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbfc0b92a0 0x7fbbfc149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 --2- 172.21.15.19:0/1057247391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc0b2cb0 0x7fbbfc0cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 -- 172.21.15.19:0/1057247391 >> 172.21.15.19:0/1057247391 conn(0x7fbbfc061a20 msgr2=0x7fbbfc0ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:21.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 -- 172.21.15.19:0/1057247391 shutdown_connections 2024-04-25T09:53:21.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.662+0000 7fbc02c6f640 1 -- 172.21.15.19:0/1057247391 wait complete. 2024-04-25T09:53:21.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:53:21.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.777+0000 7f1a67088640 1 Processor -- start 2024-04-25T09:53:21.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.777+0000 7f1a67088640 1 -- start start 2024-04-25T09:53:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a67088640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a6015dc20 0x7f1a600ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a67088640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1a600d32f0 0x7f1a600cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a67088640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6015e000 con 0x7f1a60161710 2024-04-25T09:53:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a67088640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a600d1de0 con 0x7f1a600d32f0 2024-04-25T09:53:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a67088640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a60161c60 con 0x7f1a6015dc20 2024-04-25T09:53:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a65625640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1a600d32f0 0x7f1a600cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a65625640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1a600d32f0 0x7f1a600cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35602/0 (socket says 172.21.15.19:35602) 2024-04-25T09:53:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a65625640 1 -- 172.21.15.19:0/3046174409 learned_addr learned my addr 172.21.15.19:0/3046174409 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a64e24640 1 --2- 172.21.15.19:0/3046174409 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a6015dc20 0x7f1a600ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a65625640 1 -- 172.21.15.19:0/3046174409 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a6015dc20 msgr2=0x7f1a600ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a65625640 1 --2- 172.21.15.19:0/3046174409 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a6015dc20 0x7f1a600ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a65625640 1 -- 172.21.15.19:0/3046174409 >> v1:172.21.15.112:6789/0 conn(0x7f1a60161710 legacy=0x7f1a6006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.778+0000 7f1a65625640 1 -- 172.21.15.19:0/3046174409 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a60161f40 con 0x7f1a600d32f0 2024-04-25T09:53:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.779+0000 7f1a65625640 1 --2- 172.21.15.19:0/3046174409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a600d32f0 0x7f1a600cebb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009870 tx=0x7f1a4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4b1299963bb476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.779+0000 7f1a5f7fe640 1 -- 172.21.15.19:0/3046174409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48037070 con 0x7f1a600d32f0 2024-04-25T09:53:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.779+0000 7f1a5f7fe640 1 -- 172.21.15.19:0/3046174409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a48032440 con 0x7f1a600d32f0 2024-04-25T09:53:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.779+0000 7f1a5f7fe640 1 -- 172.21.15.19:0/3046174409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48032810 con 0x7f1a600d32f0 2024-04-25T09:53:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.780+0000 7f1a67088640 1 -- 172.21.15.19:0/3046174409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a600d32f0 msgr2=0x7f1a600cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.780+0000 7f1a67088640 1 --2- 172.21.15.19:0/3046174409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a600d32f0 0x7f1a600cebb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009870 tx=0x7f1a4802a270 comp rx=0 tx=0).stop 2024-04-25T09:53:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.780+0000 7f1a67088640 1 -- 172.21.15.19:0/3046174409 shutdown_connections 2024-04-25T09:53:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.780+0000 7f1a67088640 1 --2- 172.21.15.19:0/3046174409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a600d32f0 0x7f1a600cebb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.780+0000 7f1a67088640 1 --2- 172.21.15.19:0/3046174409 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a6015dc20 0x7f1a600ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.780+0000 7f1a67088640 1 -- 172.21.15.19:0/3046174409 >> 172.21.15.19:0/3046174409 conn(0x7f1a600d0420 msgr2=0x7f1a600d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.780+0000 7f1a67088640 1 -- 172.21.15.19:0/3046174409 shutdown_connections 2024-04-25T09:53:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.780+0000 7f1a67088640 1 -- 172.21.15.19:0/3046174409 wait complete. 2024-04-25T09:53:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.781+0000 7f1a67088640 1 Processor -- start 2024-04-25T09:53:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.781+0000 7f1a67088640 1 -- start start 2024-04-25T09:53:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a67088640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a600d32f0 0x7f1a6014c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a67088640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a6015dc20 0x7f1a600acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a67088640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a60161710 0x7f1a600ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a67088640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600b5e50 con 0x7f1a6015dc20 2024-04-25T09:53:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a67088640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a60162500 con 0x7f1a60161710 2024-04-25T09:53:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a67088640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600b5fd0 con 0x7f1a600d32f0 2024-04-25T09:53:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a6015dc20 0x7f1a600acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a6015dc20 0x7f1a600acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35606/0 (socket says 172.21.15.19:35606) 2024-04-25T09:53:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a5ffff640 1 -- 172.21.15.19:0/1042544630 learned_addr learned my addr 172.21.15.19:0/1042544630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a64e24640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a600d32f0 0x7f1a6014c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a65625640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a60161710 0x7f1a600ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a5ffff640 1 -- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a600d32f0 msgr2=0x7f1a6014c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a5ffff640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a600d32f0 0x7f1a6014c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a5ffff640 1 -- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a60161710 msgr2=0x7f1a600ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a5ffff640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a60161710 0x7f1a600ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.782+0000 7f1a5ffff640 1 -- 172.21.15.19:0/1042544630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600cd350 con 0x7f1a6015dc20 2024-04-25T09:53:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.783+0000 7f1a5ffff640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a6015dc20 0x7f1a600acf20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c3c0 tx=0x7f1a5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.783+0000 7f1a5d7fa640 1 -- 172.21.15.19:0/1042544630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500052d0 con 0x7f1a6015dc20 2024-04-25T09:53:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.783+0000 7f1a5d7fa640 1 -- 172.21.15.19:0/1042544630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a50005490 con 0x7f1a6015dc20 2024-04-25T09:53:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.783+0000 7f1a5d7fa640 1 -- 172.21.15.19:0/1042544630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50007260 con 0x7f1a6015dc20 2024-04-25T09:53:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.783+0000 7f1a67088640 1 -- 172.21.15.19:0/1042544630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6014cc30 con 0x7f1a6015dc20 2024-04-25T09:53:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.783+0000 7f1a67088640 1 -- 172.21.15.19:0/1042544630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a600cc5d0 con 0x7f1a6015dc20 2024-04-25T09:53:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.784+0000 7f1a67088640 1 -- 172.21.15.19:0/1042544630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6014ca50 con 0x7f1a6015dc20 2024-04-25T09:53:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.784+0000 7f1a5d7fa640 1 -- 172.21.15.19:0/1042544630 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a50007400 con 0x7f1a6015dc20 2024-04-25T09:53:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.785+0000 7f1a5d7fa640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a4006b500 0x7f1a4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.785+0000 7f1a5d7fa640 1 -- 172.21.15.19:0/1042544630 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1a50089870 con 0x7f1a6015dc20 2024-04-25T09:53:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.785+0000 7f1a64e24640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a4006b500 0x7f1a4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.785+0000 7f1a64e24640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a4006b500 0x7f1a4006d9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a600aa590 tx=0x7f1a5800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:21.789+0000 7f1a5d7fa640 1 -- 172.21.15.19:0/1042544630 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a6014ca50 con 0x7f1a6015dc20 2024-04-25T09:53:22.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.006+0000 7f1a67088640 1 -- 172.21.15.19:0/1042544630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1a60073a00 con 0x7f1a6015dc20 2024-04-25T09:53:22.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.983+0000 7f1a5d7fa640 1 -- 172.21.15.19:0/1042544630 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1a50057dd0 con 0x7f1a6015dc20 2024-04-25T09:53:22.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 -- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a4006b500 msgr2=0x7f1a4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a4006b500 0x7f1a4006d9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a600aa590 tx=0x7f1a5800b040 comp rx=0 tx=0).stop 2024-04-25T09:53:22.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 -- 172.21.15.19:0/1042544630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a6015dc20 msgr2=0x7f1a600acf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a6015dc20 0x7f1a600acf20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c3c0 tx=0x7f1a5000c880 comp rx=0 tx=0).stop 2024-04-25T09:53:22.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 -- 172.21.15.19:0/1042544630 shutdown_connections 2024-04-25T09:53:22.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a4006b500 0x7f1a4006d9c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a60161710 0x7f1a600ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a6015dc20 0x7f1a600acf20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 --2- 172.21.15.19:0/1042544630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a600d32f0 0x7f1a6014c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 -- 172.21.15.19:0/1042544630 >> 172.21.15.19:0/1042544630 conn(0x7f1a600d0420 msgr2=0x7f1a600b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 -- 172.21.15.19:0/1042544630 shutdown_connections 2024-04-25T09:53:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:22.986+0000 7f1a67088640 1 -- 172.21.15.19:0/1042544630 wait complete. 2024-04-25T09:53:23.001 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T09:53:23.001 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:53:23.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.109+0000 7f3ec221f640 1 Processor -- start 2024-04-25T09:53:23.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.109+0000 7f3ec221f640 1 -- start start 2024-04-25T09:53:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ec221f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc15dc20 0x7f3ebc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ec221f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ebc161710 0x7f3ebc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ec221f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ebc15e000 con 0x7f3ebc0b9330 2024-04-25T09:53:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ec221f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ebc0d1de0 con 0x7f3ebc161710 2024-04-25T09:53:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ec221f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc161c60 con 0x7f3ebc15dc20 2024-04-25T09:53:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ebaffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ebc161710 0x7f3ebc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ebbfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3ebc0b9330 0x7f3ebc06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46830/0 (socket says 172.21.15.19:46830) 2024-04-25T09:53:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ebbfff640 1 -- 172.21.15.19:0/2165077978 learned_addr learned my addr 172.21.15.19:0/2165077978 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ebb7fe640 1 --2- 172.21.15.19:0/2165077978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc15dc20 0x7f3ebc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ebaffd640 1 -- 172.21.15.19:0/2165077978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc15dc20 msgr2=0x7f3ebc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ebaffd640 1 --2- 172.21.15.19:0/2165077978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc15dc20 0x7f3ebc0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ebaffd640 1 -- 172.21.15.19:0/2165077978 >> v1:172.21.15.19:6789/0 conn(0x7f3ebc0b9330 legacy=0x7f3ebc06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3eba7fc640 1 -- 172.21.15.19:0/2165077978 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813662730 0 0) 0x7f3ebc15e000 con 0x7f3ebc0b9330 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.110+0000 7f3ebaffd640 1 -- 172.21.15.19:0/2165077978 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc161f40 con 0x7f3ebc161710 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.111+0000 7f3ebaffd640 1 --2- 172.21.15.19:0/2165077978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc161710 0x7f3ebc0cebb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00097a0 tx=0x7f3eb002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ef747b0bd668e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.111+0000 7f3eba7fc640 1 -- 172.21.15.19:0/2165077978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0037070 con 0x7f3ebc161710 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.111+0000 7f3eba7fc640 1 -- 172.21.15.19:0/2165077978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb0032440 con 0x7f3ebc161710 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.111+0000 7f3eba7fc640 1 -- 172.21.15.19:0/2165077978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0032810 con 0x7f3ebc161710 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.112+0000 7f3ec221f640 1 -- 172.21.15.19:0/2165077978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc161710 msgr2=0x7f3ebc0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.112+0000 7f3ec221f640 1 --2- 172.21.15.19:0/2165077978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc161710 0x7f3ebc0cebb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00097a0 tx=0x7f3eb002a180 comp rx=0 tx=0).stop 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.112+0000 7f3ec221f640 1 -- 172.21.15.19:0/2165077978 shutdown_connections 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.112+0000 7f3ec221f640 1 --2- 172.21.15.19:0/2165077978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc161710 0x7f3ebc0cebb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.112+0000 7f3ec221f640 1 --2- 172.21.15.19:0/2165077978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc15dc20 0x7f3ebc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.112+0000 7f3ec221f640 1 -- 172.21.15.19:0/2165077978 >> 172.21.15.19:0/2165077978 conn(0x7f3ebc0d0420 msgr2=0x7f3ebc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:23.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.112+0000 7f3ec221f640 1 -- 172.21.15.19:0/2165077978 shutdown_connections 2024-04-25T09:53:23.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.112+0000 7f3ec221f640 1 -- 172.21.15.19:0/2165077978 wait complete. 2024-04-25T09:53:23.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.113+0000 7f3ec221f640 1 Processor -- start 2024-04-25T09:53:23.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.113+0000 7f3ec221f640 1 -- start start 2024-04-25T09:53:23.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ec221f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ebc0b9330 0x7f3ebc1f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:23.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ec221f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc15dc20 0x7f3ebc0b8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:23.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ec221f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc161710 0x7f3ebc0b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:23.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ec221f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0b5e50 con 0x7f3ebc15dc20 2024-04-25T09:53:23.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ec221f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc162500 con 0x7f3ebc0b9330 2024-04-25T09:53:23.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ec221f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0b5fd0 con 0x7f3ebc161710 2024-04-25T09:53:23.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc15dc20 0x7f3ebc0b8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:23.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc15dc20 0x7f3ebc0b8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35622/0 (socket says 172.21.15.19:35622) 2024-04-25T09:53:23.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebaffd640 1 -- 172.21.15.19:0/3362071522 learned_addr learned my addr 172.21.15.19:0/3362071522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:23.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebbfff640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc161710 0x7f3ebc0b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:23.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebb7fe640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ebc0b9330 0x7f3ebc1f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:23.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebaffd640 1 -- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc161710 msgr2=0x7f3ebc0b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:23.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebaffd640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc161710 0x7f3ebc0b87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebaffd640 1 -- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ebc0b9330 msgr2=0x7f3ebc1f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebaffd640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ebc0b9330 0x7f3ebc1f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebaffd640 1 -- 172.21.15.19:0/3362071522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc157100 con 0x7f3ebc15dc20 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebb7fe640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ebc0b9330 0x7f3ebc1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.114+0000 7f3ebbfff640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc161710 0x7f3ebc0b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.115+0000 7f3ebaffd640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc15dc20 0x7f3ebc0b8290 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00097a0 tx=0x7f3eb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.115+0000 7f3ea3fff640 1 -- 172.21.15.19:0/3362071522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0037050 con 0x7f3ebc15dc20 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.115+0000 7f3ea3fff640 1 -- 172.21.15.19:0/3362071522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb0038040 con 0x7f3ebc15dc20 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.115+0000 7f3ec221f640 1 -- 172.21.15.19:0/3362071522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc1573e0 con 0x7f3ebc15dc20 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.115+0000 7f3ea3fff640 1 -- 172.21.15.19:0/3362071522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0004470 con 0x7f3ebc15dc20 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.115+0000 7f3ec221f640 1 -- 172.21.15.19:0/3362071522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc1579a0 con 0x7f3ebc15dc20 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.116+0000 7f3ec221f640 1 -- 172.21.15.19:0/3362071522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc073a00 con 0x7f3ebc15dc20 2024-04-25T09:53:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.116+0000 7f3ea3fff640 1 -- 172.21.15.19:0/3362071522 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3eb0030900 con 0x7f3ebc15dc20 2024-04-25T09:53:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.117+0000 7f3ea3fff640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ea806b4d0 0x7f3ea806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.117+0000 7f3ea3fff640 1 -- 172.21.15.19:0/3362071522 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3eb00ab020 con 0x7f3ebc15dc20 2024-04-25T09:53:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.117+0000 7f3ebb7fe640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ea806b4d0 0x7f3ea806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.118+0000 7f3ebb7fe640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ea806b4d0 0x7f3ea806d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006fd0 tx=0x7f3eac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:23.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.121+0000 7f3ea3fff640 1 -- 172.21.15.19:0/3362071522 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3eb0079b60 con 0x7f3ebc15dc20 2024-04-25T09:53:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.344+0000 7f3ec221f640 1 -- 172.21.15.19:0/3362071522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3ebc0d47d0 con 0x7f3ebc15dc20 2024-04-25T09:53:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.984+0000 7f3ea3fff640 1 -- 172.21.15.19:0/3362071522 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3eb0079460 con 0x7f3ebc15dc20 2024-04-25T09:53:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 -- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ea806b4d0 msgr2=0x7f3ea806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ea806b4d0 0x7f3ea806d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006fd0 tx=0x7f3eac008040 comp rx=0 tx=0).stop 2024-04-25T09:53:23.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 -- 172.21.15.19:0/3362071522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc15dc20 msgr2=0x7f3ebc0b8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:23.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc15dc20 0x7f3ebc0b8290 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00097a0 tx=0x7f3eb00046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 -- 172.21.15.19:0/3362071522 shutdown_connections 2024-04-25T09:53:23.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ea806b4d0 0x7f3ea806d990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc161710 0x7f3ebc0b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc15dc20 0x7f3ebc0b8290 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 --2- 172.21.15.19:0/3362071522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ebc0b9330 0x7f3ebc1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 -- 172.21.15.19:0/3362071522 >> 172.21.15.19:0/3362071522 conn(0x7f3ebc0d0420 msgr2=0x7f3ebc0b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 -- 172.21.15.19:0/3362071522 shutdown_connections 2024-04-25T09:53:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:23.987+0000 7f3ec221f640 1 -- 172.21.15.19:0/3362071522 wait complete. 2024-04-25T09:53:24.003 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:53:24.003 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:24.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.108+0000 7fd4b9207640 1 Processor -- start 2024-04-25T09:53:24.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.108+0000 7fd4b9207640 1 -- start start 2024-04-25T09:53:24.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.109+0000 7fd4b9207640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd4b415dbe0 0x7fd4b40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.109+0000 7fd4b9207640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b92a0 0x7fd4b40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.109+0000 7fd4b9207640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b40b97a0 con 0x7fd4b40b2cb0 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.109+0000 7fd4b9207640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b415dfc0 con 0x7fd4b415dbe0 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.109+0000 7fd4b9207640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40d1a00 con 0x7fd4b40b92a0 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.109+0000 7fd4b359e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b92a0 0x7fd4b40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.109+0000 7fd4b359e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b92a0 0x7fd4b40d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55498/0 (socket says 172.21.15.19:55498) 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.109+0000 7fd4b2d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd4b415dbe0 0x7fd4b40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.109+0000 7fd4b359e640 1 -- 172.21.15.19:0/1252691919 learned_addr learned my addr 172.21.15.19:0/1252691919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.110+0000 7fd4b359e640 1 -- 172.21.15.19:0/1252691919 >> v2:172.21.15.112:3300/0 conn(0x7fd4b415dbe0 msgr2=0x7fd4b40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.110+0000 7fd4b359e640 1 --2- 172.21.15.19:0/1252691919 >> v2:172.21.15.112:3300/0 conn(0x7fd4b415dbe0 0x7fd4b40d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.110+0000 7fd4b359e640 1 -- 172.21.15.19:0/1252691919 >> v1:172.21.15.112:6789/0 conn(0x7fd4b40b2cb0 legacy=0x7fd4b40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.110+0000 7fd4b359e640 1 -- 172.21.15.19:0/1252691919 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b40d1c50 con 0x7fd4b40b92a0 2024-04-25T09:53:24.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.110+0000 7fd4b2d9d640 1 --2- 172.21.15.19:0/1252691919 >> v2:172.21.15.112:3300/0 conn(0x7fd4b415dbe0 0x7fd4b40d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:24.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.110+0000 7fd4b359e640 1 --2- 172.21.15.19:0/1252691919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b92a0 0x7fd4b40d0e50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009e50 tx=0x7fd4a402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9a29bb39eb18dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:24.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.111+0000 7fd4b1d9b640 1 -- 172.21.15.19:0/1252691919 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4036070 con 0x7fd4b40b92a0 2024-04-25T09:53:24.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.111+0000 7fd4b1d9b640 1 -- 172.21.15.19:0/1252691919 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a4031600 con 0x7fd4b40b92a0 2024-04-25T09:53:24.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.111+0000 7fd4b1d9b640 1 -- 172.21.15.19:0/1252691919 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a40319d0 con 0x7fd4b40b92a0 2024-04-25T09:53:24.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.111+0000 7fd4b9207640 1 -- 172.21.15.19:0/1252691919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b92a0 msgr2=0x7fd4b40d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.111+0000 7fd4b9207640 1 --2- 172.21.15.19:0/1252691919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b92a0 0x7fd4b40d0e50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009e50 tx=0x7fd4a402a710 comp rx=0 tx=0).stop 2024-04-25T09:53:24.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.111+0000 7fd4b9207640 1 -- 172.21.15.19:0/1252691919 shutdown_connections 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.111+0000 7fd4b9207640 1 --2- 172.21.15.19:0/1252691919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b92a0 0x7fd4b40d0e50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.111+0000 7fd4b9207640 1 --2- 172.21.15.19:0/1252691919 >> v2:172.21.15.112:3300/0 conn(0x7fd4b415dbe0 0x7fd4b40d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.111+0000 7fd4b9207640 1 -- 172.21.15.19:0/1252691919 >> 172.21.15.19:0/1252691919 conn(0x7fd4b4061a20 msgr2=0x7fd4b4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.111+0000 7fd4b9207640 1 -- 172.21.15.19:0/1252691919 shutdown_connections 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.111+0000 7fd4b9207640 1 -- 172.21.15.19:0/1252691919 wait complete. 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.113+0000 7fd4b9207640 1 Processor -- start 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.113+0000 7fd4b9207640 1 -- start start 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b9207640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b2cb0 0x7fd4b4153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b9207640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40b92a0 0x7fd4b415f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b9207640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b415dbe0 0x7fd4b415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b9207640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40b6ab0 con 0x7fd4b415dbe0 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b9207640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40d2210 con 0x7fd4b40b92a0 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b9207640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40b6c30 con 0x7fd4b40b2cb0 2024-04-25T09:53:24.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b415dbe0 0x7fd4b415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b415dbe0 0x7fd4b415eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35630/0 (socket says 172.21.15.19:35630) 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b359e640 1 -- 172.21.15.19:0/2720135013 learned_addr learned my addr 172.21.15.19:0/2720135013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b2d9d640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b2cb0 0x7fd4b4153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b259c640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40b92a0 0x7fd4b415f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b359e640 1 -- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b2cb0 msgr2=0x7fd4b4153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b359e640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b2cb0 0x7fd4b4153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b359e640 1 -- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40b92a0 msgr2=0x7fd4b415f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b359e640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40b92a0 0x7fd4b415f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b359e640 1 -- 172.21.15.19:0/2720135013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b414f030 con 0x7fd4b415dbe0 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b2d9d640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b2cb0 0x7fd4b4153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b359e640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b415dbe0 0x7fd4b415eb60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009f70 tx=0x7fd4a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4ab7fe640 1 -- 172.21.15.19:0/2720135013 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a403f020 con 0x7fd4b415dbe0 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4ab7fe640 1 -- 172.21.15.19:0/2720135013 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a4037040 con 0x7fd4b415dbe0 2024-04-25T09:53:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4ab7fe640 1 -- 172.21.15.19:0/2720135013 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4036070 con 0x7fd4b415dbe0 2024-04-25T09:53:24.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.114+0000 7fd4b9207640 1 -- 172.21.15.19:0/2720135013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b4152b00 con 0x7fd4b415dbe0 2024-04-25T09:53:24.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.115+0000 7fd4b9207640 1 -- 172.21.15.19:0/2720135013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b4153360 con 0x7fd4b415dbe0 2024-04-25T09:53:24.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.116+0000 7fd4b9207640 1 -- 172.21.15.19:0/2720135013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b4073990 con 0x7fd4b415dbe0 2024-04-25T09:53:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.116+0000 7fd4ab7fe640 1 -- 172.21.15.19:0/2720135013 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4a4042020 con 0x7fd4b415dbe0 2024-04-25T09:53:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.116+0000 7fd4ab7fe640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd49406b4d0 0x7fd49406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.117+0000 7fd4ab7fe640 1 -- 172.21.15.19:0/2720135013 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd4a40aa9a0 con 0x7fd4b415dbe0 2024-04-25T09:53:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.117+0000 7fd4b2d9d640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd49406b4d0 0x7fd49406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.117+0000 7fd4b2d9d640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd49406b4d0 0x7fd49406d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009af0 tx=0x7fd4ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:24.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.121+0000 7fd4ab7fe640 1 -- 172.21.15.19:0/2720135013 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4a4049030 con 0x7fd4b415dbe0 2024-04-25T09:53:24.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.337+0000 7fd4b9207640 1 -- 172.21.15.19:0/2720135013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4b415ced0 con 0x7fd4b415dbe0 2024-04-25T09:53:24.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.337+0000 7fd4ab7fe640 1 -- 172.21.15.19:0/2720135013 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fd4a40794e0 con 0x7fd4b415dbe0 2024-04-25T09:53:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:24.341 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4000846111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":4000846111},{"type":"v1","addr":"172.21.15.112:6837","nonce":4000846111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1969525282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1969525282},{"type":"v1","addr":"172.21.15.19:6835","nonce":1969525282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:07.175350+0000","modified":"2024-04-25T09:53:23.345595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1080352556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1080352556},{"type":"v1","addr":"172.21.15.112:6836","nonce":1080352556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:53:24.342 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T09:53:24.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.339+0000 7fd4b9207640 1 -- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd49406b4d0 msgr2=0x7fd49406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.339+0000 7fd4b9207640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd49406b4d0 0x7fd49406d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009af0 tx=0x7fd4ac008040 comp rx=0 tx=0).stop 2024-04-25T09:53:24.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.339+0000 7fd4b9207640 1 -- 172.21.15.19:0/2720135013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b415dbe0 msgr2=0x7fd4b415eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.339+0000 7fd4b9207640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b415dbe0 0x7fd4b415eb60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009f70 tx=0x7fd4a4004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:24.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.340+0000 7fd4b9207640 1 -- 172.21.15.19:0/2720135013 shutdown_connections 2024-04-25T09:53:24.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.340+0000 7fd4b9207640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd49406b4d0 0x7fd49406d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.340+0000 7fd4b9207640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4b415dbe0 0x7fd4b415eb60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.340+0000 7fd4b9207640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4b40b92a0 0x7fd4b415f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.340+0000 7fd4b9207640 1 --2- 172.21.15.19:0/2720135013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4b40b2cb0 0x7fd4b4153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.340+0000 7fd4b9207640 1 -- 172.21.15.19:0/2720135013 >> 172.21.15.19:0/2720135013 conn(0x7fd4b4061a20 msgr2=0x7fd4b40d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:24.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.340+0000 7fd4b9207640 1 -- 172.21.15.19:0/2720135013 shutdown_connections 2024-04-25T09:53:24.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.340+0000 7fd4b9207640 1 -- 172.21.15.19:0/2720135013 wait complete. 2024-04-25T09:53:24.352 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:24.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.453+0000 7ff05813b640 1 Processor -- start 2024-04-25T09:53:24.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.453+0000 7ff05813b640 1 -- start start 2024-04-25T09:53:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff05813b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff05015dbe0 0x7ff0500d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff05813b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0500b2cb0 0x7ff0500d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff05813b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0500b97a0 con 0x7ff0500b92a0 2024-04-25T09:53:24.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff05813b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05015dfc0 con 0x7ff05015dbe0 2024-04-25T09:53:24.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff05813b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500d1a00 con 0x7ff0500b2cb0 2024-04-25T09:53:24.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff055ed7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff05015dbe0 0x7ff0500d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff0566d8640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff0500b92a0 0x7ff0500b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46846/0 (socket says 172.21.15.19:46846) 2024-04-25T09:53:24.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff0566d8640 1 -- 172.21.15.19:0/1753515762 learned_addr learned my addr 172.21.15.19:0/1753515762 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:24.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff0556d6640 1 --2- 172.21.15.19:0/1753515762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0500b2cb0 0x7ff0500d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff054ed5640 1 -- 172.21.15.19:0/1753515762 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902115128 0 0) 0x7ff0500b97a0 con 0x7ff0500b92a0 2024-04-25T09:53:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff054ed5640 1 -- 172.21.15.19:0/1753515762 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff03c003520 con 0x7ff0500b92a0 2024-04-25T09:53:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff055ed7640 1 -- 172.21.15.19:0/1753515762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0500b2cb0 msgr2=0x7ff0500d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff055ed7640 1 --2- 172.21.15.19:0/1753515762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0500b2cb0 0x7ff0500d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff055ed7640 1 -- 172.21.15.19:0/1753515762 >> v1:172.21.15.19:6789/0 conn(0x7ff0500b92a0 legacy=0x7ff0500b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.454+0000 7ff055ed7640 1 -- 172.21.15.19:0/1753515762 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0500d1c50 con 0x7ff05015dbe0 2024-04-25T09:53:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.455+0000 7ff055ed7640 1 --2- 172.21.15.19:0/1753515762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05015dbe0 0x7ff0500d0910 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0097a0 tx=0x7ff04c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c900c9db830972 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.455+0000 7ff054ed5640 1 -- 172.21.15.19:0/1753515762 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c037070 con 0x7ff05015dbe0 2024-04-25T09:53:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.455+0000 7ff054ed5640 1 -- 172.21.15.19:0/1753515762 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c032440 con 0x7ff05015dbe0 2024-04-25T09:53:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.455+0000 7ff054ed5640 1 -- 172.21.15.19:0/1753515762 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c0327f0 con 0x7ff05015dbe0 2024-04-25T09:53:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.455+0000 7ff05813b640 1 -- 172.21.15.19:0/1753515762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05015dbe0 msgr2=0x7ff0500d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.455+0000 7ff05813b640 1 --2- 172.21.15.19:0/1753515762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05015dbe0 0x7ff0500d0910 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0097a0 tx=0x7ff04c02a180 comp rx=0 tx=0).stop 2024-04-25T09:53:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.456+0000 7ff05813b640 1 -- 172.21.15.19:0/1753515762 shutdown_connections 2024-04-25T09:53:24.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.456+0000 7ff05813b640 1 --2- 172.21.15.19:0/1753515762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0500b2cb0 0x7ff0500d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.456+0000 7ff05813b640 1 --2- 172.21.15.19:0/1753515762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05015dbe0 0x7ff0500d0910 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.456+0000 7ff05813b640 1 -- 172.21.15.19:0/1753515762 >> 172.21.15.19:0/1753515762 conn(0x7ff050061a20 msgr2=0x7ff050061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:24.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.456+0000 7ff05813b640 1 -- 172.21.15.19:0/1753515762 shutdown_connections 2024-04-25T09:53:24.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.456+0000 7ff05813b640 1 -- 172.21.15.19:0/1753515762 wait complete. 2024-04-25T09:53:24.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.457+0000 7ff05813b640 1 Processor -- start 2024-04-25T09:53:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.457+0000 7ff05813b640 1 -- start start 2024-04-25T09:53:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.457+0000 7ff05813b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0500b2cb0 0x7ff0500cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.457+0000 7ff05813b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0500b92a0 0x7ff0501f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff05813b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05015dbe0 0x7ff0500cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff05813b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500b6ab0 con 0x7ff05015dbe0 2024-04-25T09:53:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff05813b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500d2210 con 0x7ff0500b2cb0 2024-04-25T09:53:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff05813b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500b6c30 con 0x7ff0500b92a0 2024-04-25T09:53:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0566d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05015dbe0 0x7ff0500cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0566d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05015dbe0 0x7ff0500cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35648/0 (socket says 172.21.15.19:35648) 2024-04-25T09:53:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0566d8640 1 -- 172.21.15.19:0/1688882651 learned_addr learned my addr 172.21.15.19:0/1688882651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0556d6640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0500b92a0 0x7ff0501f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff055ed7640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0500b2cb0 0x7ff0500cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0566d8640 1 -- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0500b92a0 msgr2=0x7ff0501f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0566d8640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0500b92a0 0x7ff0501f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0566d8640 1 -- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0500b2cb0 msgr2=0x7ff0500cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0566d8640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0500b2cb0 0x7ff0500cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0566d8640 1 -- 172.21.15.19:0/1688882651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0500cb770 con 0x7ff05015dbe0 2024-04-25T09:53:24.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff055ed7640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0500b2cb0 0x7ff0500cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:24.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0566d8640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05015dbe0 0x7ff0500cd260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff04800cae0 tx=0x7ff04800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:24.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0467fc640 1 -- 172.21.15.19:0/1688882651 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048017070 con 0x7ff05015dbe0 2024-04-25T09:53:24.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0467fc640 1 -- 172.21.15.19:0/1688882651 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04800ac40 con 0x7ff05015dbe0 2024-04-25T09:53:24.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff05813b640 1 -- 172.21.15.19:0/1688882651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0501f3860 con 0x7ff05015dbe0 2024-04-25T09:53:24.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff0467fc640 1 -- 172.21.15.19:0/1688882651 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048012640 con 0x7ff05015dbe0 2024-04-25T09:53:24.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.458+0000 7ff05813b640 1 -- 172.21.15.19:0/1688882651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0501f3e20 con 0x7ff05015dbe0 2024-04-25T09:53:24.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.459+0000 7ff05813b640 1 -- 172.21.15.19:0/1688882651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff050073990 con 0x7ff05015dbe0 2024-04-25T09:53:24.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.460+0000 7ff0467fc640 1 -- 172.21.15.19:0/1688882651 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff04801a070 con 0x7ff05015dbe0 2024-04-25T09:53:24.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.460+0000 7ff0467fc640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03406b4d0 0x7ff03406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.460+0000 7ff0467fc640 1 -- 172.21.15.19:0/1688882651 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff04808a210 con 0x7ff05015dbe0 2024-04-25T09:53:24.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.461+0000 7ff055ed7640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03406b4d0 0x7ff03406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.461+0000 7ff055ed7640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03406b4d0 0x7ff03406d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff04c034000 tx=0x7ff04c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:24.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.465+0000 7ff0467fc640 1 -- 172.21.15.19:0/1688882651 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff048058790 con 0x7ff05015dbe0 2024-04-25T09:53:24.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.645+0000 7ff05813b640 1 -- 172.21.15.19:0/1688882651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff050061fa0 con 0x7ff05015dbe0 2024-04-25T09:53:24.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.647+0000 7ff0467fc640 1 -- 172.21.15.19:0/1688882651 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7ff04805c830 con 0x7ff05015dbe0 2024-04-25T09:53:24.649 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:24.649 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:24.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 -- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03406b4d0 msgr2=0x7ff03406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03406b4d0 0x7ff03406d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff04c034000 tx=0x7ff04c02e000 comp rx=0 tx=0).stop 2024-04-25T09:53:24.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 -- 172.21.15.19:0/1688882651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05015dbe0 msgr2=0x7ff0500cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05015dbe0 0x7ff0500cd260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff04800cae0 tx=0x7ff04800a060 comp rx=0 tx=0).stop 2024-04-25T09:53:24.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 -- 172.21.15.19:0/1688882651 shutdown_connections 2024-04-25T09:53:24.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03406b4d0 0x7ff03406d990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05015dbe0 0x7ff0500cd260 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0500b92a0 0x7ff0501f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 --2- 172.21.15.19:0/1688882651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0500b2cb0 0x7ff0500cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 -- 172.21.15.19:0/1688882651 >> 172.21.15.19:0/1688882651 conn(0x7ff050061a20 msgr2=0x7ff05015b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:24.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 -- 172.21.15.19:0/1688882651 shutdown_connections 2024-04-25T09:53:24.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.649+0000 7ff05813b640 1 -- 172.21.15.19:0/1688882651 wait complete. 2024-04-25T09:53:24.663 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:53:24.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.762+0000 7f1c087c1640 1 Processor -- start 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.762+0000 7f1c087c1640 1 -- start start 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.763+0000 7f1c087c1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c0015dd20 0x7f1c000b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.763+0000 7f1c087c1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1c000d11b0 0x7f1c000d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.763+0000 7f1c087c1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c0015eac0 con 0x7f1c0015e100 2024-04-25T09:53:24.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.763+0000 7f1c087c1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c000d1c00 con 0x7f1c0015dd20 2024-04-25T09:53:24.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.763+0000 7f1c087c1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c0015eda0 con 0x7f1c000d11b0 2024-04-25T09:53:24.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.763+0000 7f1c05d5c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1c0015e100 0x7f1c000d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46854/0 (socket says 172.21.15.19:46854) 2024-04-25T09:53:24.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.763+0000 7f1c0655d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c0015dd20 0x7f1c000b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.763+0000 7f1c05d5c640 1 -- 172.21.15.19:0/3581953468 learned_addr learned my addr 172.21.15.19:0/3581953468 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:24.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.763+0000 7f1c06d5e640 1 --2- 172.21.15.19:0/3581953468 >> v2:172.21.15.112:3300/0 conn(0x7f1c000d11b0 0x7f1c000d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.764+0000 7f1c0655d640 1 -- 172.21.15.19:0/3581953468 >> v2:172.21.15.112:3300/0 conn(0x7f1c000d11b0 msgr2=0x7f1c000d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:24.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.764+0000 7f1c0655d640 1 --2- 172.21.15.19:0/3581953468 >> v2:172.21.15.112:3300/0 conn(0x7f1c000d11b0 0x7f1c000d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.764+0000 7f1c0655d640 1 -- 172.21.15.19:0/3581953468 >> v1:172.21.15.19:6789/0 conn(0x7f1c0015e100 legacy=0x7f1c000d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.764+0000 7f1c0655d640 1 -- 172.21.15.19:0/3581953468 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0015f080 con 0x7f1c0015dd20 2024-04-25T09:53:24.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.764+0000 7f1c0555b640 1 -- 172.21.15.19:0/3581953468 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787948825 0 0) 0x7f1c0015eac0 con 0x7f1c0015e100 2024-04-25T09:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.764+0000 7f1c0655d640 1 --2- 172.21.15.19:0/3581953468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015dd20 0x7f1c000b96d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c000d1990 tx=0x7f1bf802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63df5ecc2c1579a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.764+0000 7f1c0555b640 1 -- 172.21.15.19:0/3581953468 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8037070 con 0x7f1c0015dd20 2024-04-25T09:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.765+0000 7f1c0555b640 1 -- 172.21.15.19:0/3581953468 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf8032440 con 0x7f1c0015dd20 2024-04-25T09:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.765+0000 7f1c0555b640 1 -- 172.21.15.19:0/3581953468 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf802ae60 con 0x7f1c0015dd20 2024-04-25T09:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.765+0000 7f1c087c1640 1 -- 172.21.15.19:0/3581953468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015dd20 msgr2=0x7f1c000b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.765+0000 7f1c087c1640 1 --2- 172.21.15.19:0/3581953468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015dd20 0x7f1c000b96d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c000d1990 tx=0x7f1bf802a110 comp rx=0 tx=0).stop 2024-04-25T09:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.765+0000 7f1c087c1640 1 -- 172.21.15.19:0/3581953468 shutdown_connections 2024-04-25T09:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.765+0000 7f1c087c1640 1 --2- 172.21.15.19:0/3581953468 >> v2:172.21.15.112:3300/0 conn(0x7f1c000d11b0 0x7f1c000d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.765+0000 7f1c087c1640 1 --2- 172.21.15.19:0/3581953468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015dd20 0x7f1c000b96d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.765+0000 7f1c087c1640 1 -- 172.21.15.19:0/3581953468 >> 172.21.15.19:0/3581953468 conn(0x7f1c00061a20 msgr2=0x7f1c00061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.765+0000 7f1c087c1640 1 -- 172.21.15.19:0/3581953468 shutdown_connections 2024-04-25T09:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.765+0000 7f1c087c1640 1 -- 172.21.15.19:0/3581953468 wait complete. 2024-04-25T09:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.766+0000 7f1c087c1640 1 Processor -- start 2024-04-25T09:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.766+0000 7f1c087c1640 1 -- start start 2024-04-25T09:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.767+0000 7f1c087c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c000d11b0 0x7f1c001f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.767+0000 7f1c087c1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015dd20 0x7f1c001f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.767+0000 7f1c087c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0015e100 0x7f1c0015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.767+0000 7f1c087c1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0015e710 con 0x7f1c0015dd20 2024-04-25T09:53:24.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.767+0000 7f1c087c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0015e590 con 0x7f1c0015e100 2024-04-25T09:53:24.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.767+0000 7f1c087c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c000ab250 con 0x7f1c000d11b0 2024-04-25T09:53:24.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.767+0000 7f1c05d5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015dd20 0x7f1c001f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.767+0000 7f1c05d5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015dd20 0x7f1c001f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35664/0 (socket says 172.21.15.19:35664) 2024-04-25T09:53:24.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.767+0000 7f1c05d5c640 1 -- 172.21.15.19:0/1190199734 learned_addr learned my addr 172.21.15.19:0/1190199734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:24.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.767+0000 7f1c0655d640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c000d11b0 0x7f1c001f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.767+0000 7f1c06d5e640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0015e100 0x7f1c0015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1c05d5c640 1 -- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c000d11b0 msgr2=0x7f1c001f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1c05d5c640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c000d11b0 0x7f1c001f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1c05d5c640 1 -- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0015e100 msgr2=0x7f1c0015c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1c05d5c640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0015e100 0x7f1c0015c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1c05d5c640 1 -- 172.21.15.19:0/1190199734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c000ab530 con 0x7f1c0015dd20 2024-04-25T09:53:24.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1c06d5e640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0015e100 0x7f1c0015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1c05d5c640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015dd20 0x7f1c001f70c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000c430 tx=0x7f1bf000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1bf6ffd640 1 -- 172.21.15.19:0/1190199734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf00074e0 con 0x7f1c0015dd20 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1bf6ffd640 1 -- 172.21.15.19:0/1190199734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf00047e0 con 0x7f1c0015dd20 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1c087c1640 1 -- 172.21.15.19:0/1190199734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c000ab870 con 0x7f1c0015dd20 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1bf6ffd640 1 -- 172.21.15.19:0/1190199734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0004bd0 con 0x7f1c0015dd20 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.768+0000 7f1c087c1640 1 -- 172.21.15.19:0/1190199734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c000b2200 con 0x7f1c0015dd20 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.769+0000 7f1c087c1640 1 -- 172.21.15.19:0/1190199734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c00073990 con 0x7f1c0015dd20 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.769+0000 7f1bf6ffd640 1 -- 172.21.15.19:0/1190199734 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bf0011070 con 0x7f1c0015dd20 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.770+0000 7f1bf6ffd640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be406b4d0 0x7f1be406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.770+0000 7f1bf6ffd640 1 -- 172.21.15.19:0/1190199734 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1bf008a120 con 0x7f1c0015dd20 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.770+0000 7f1c0655d640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be406b4d0 0x7f1be406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:24.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.770+0000 7f1c0655d640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be406b4d0 0x7f1be406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8034000 tx=0x7f1bf802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.774+0000 7f1bf6ffd640 1 -- 172.21.15.19:0/1190199734 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c000ab870 con 0x7f1c0015dd20 2024-04-25T09:53:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.952+0000 7f1c087c1640 1 -- 172.21.15.19:0/1190199734 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c000b4b40 con 0x7f1be406b4d0 2024-04-25T09:53:24.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.961+0000 7f1bf6ffd640 1 -- 172.21.15.19:0/1190199734 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f1c000b4b40 con 0x7f1be406b4d0 2024-04-25T09:53:24.967 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:53:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 -- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be406b4d0 msgr2=0x7f1be406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be406b4d0 0x7f1be406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8034000 tx=0x7f1bf802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 -- 172.21.15.19:0/1190199734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015dd20 msgr2=0x7f1c001f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015dd20 0x7f1c001f70c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000c430 tx=0x7f1bf000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 -- 172.21.15.19:0/1190199734 shutdown_connections 2024-04-25T09:53:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be406b4d0 0x7f1be406d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0015e100 0x7f1c0015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015dd20 0x7f1c001f70c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 --2- 172.21.15.19:0/1190199734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c000d11b0 0x7f1c001f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:24.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 -- 172.21.15.19:0/1190199734 >> 172.21.15.19:0/1190199734 conn(0x7f1c00061a20 msgr2=0x7f1c000d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:24.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 -- 172.21.15.19:0/1190199734 shutdown_connections 2024-04-25T09:53:24.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:24.964+0000 7f1c087c1640 1 -- 172.21.15.19:0/1190199734 wait complete. 2024-04-25T09:53:24.977 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T09:53:24.914622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219368,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755352,"statfs":{"total":773094113280,"available":772869480448,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003374"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326249+0000","last_change":"2024-04-25T09:53:06.778392+0000","last_active":"2024-04-25T09:53:13.326249+0000","last_peered":"2024-04-25T09:53:13.326249+0000","last_clean":"2024-04-25T09:53:13.326249+0000","last_became_active":"2024-04-25T09:53:06.777894+0000","last_became_peered":"2024-04-25T09:53:06.777894+0000","last_unstale":"2024-04-25T09:53:13.326249+0000","last_undegraded":"2024-04-25T09:53:13.326249+0000","last_fullsized":"2024-04-25T09:53:13.326249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.898659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326801+0000","last_change":"2024-04-25T09:53:07.792733+0000","last_active":"2024-04-25T09:53:13.326801+0000","last_peered":"2024-04-25T09:53:13.326801+0000","last_clean":"2024-04-25T09:53:13.326801+0000","last_became_active":"2024-04-25T09:53:07.792294+0000","last_became_peered":"2024-04-25T09:53:07.792294+0000","last_unstale":"2024-04-25T09:53:13.326801+0000","last_undegraded":"2024-04-25T09:53:13.326801+0000","last_fullsized":"2024-04-25T09:53:13.326801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325683+0000","last_change":"2024-04-25T09:53:11.822234+0000","last_active":"2024-04-25T09:53:13.325683+0000","last_peered":"2024-04-25T09:53:13.325683+0000","last_clean":"2024-04-25T09:53:13.325683+0000","last_became_active":"2024-04-25T09:53:11.821231+0000","last_became_peered":"2024-04-25T09:53:11.821231+0000","last_unstale":"2024-04-25T09:53:13.325683+0000","last_undegraded":"2024-04-25T09:53:13.325683+0000","last_fullsized":"2024-04-25T09:53:13.325683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:19.880548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327066+0000","last_change":"2024-04-25T09:53:06.779890+0000","last_active":"2024-04-25T09:53:13.327066+0000","last_peered":"2024-04-25T09:53:13.327066+0000","last_clean":"2024-04-25T09:53:13.327066+0000","last_became_active":"2024-04-25T09:53:06.779482+0000","last_became_peered":"2024-04-25T09:53:06.779482+0000","last_unstale":"2024-04-25T09:53:13.327066+0000","last_undegraded":"2024-04-25T09:53:13.327066+0000","last_fullsized":"2024-04-25T09:53:13.327066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325889+0000","last_change":"2024-04-25T09:53:07.789471+0000","last_active":"2024-04-25T09:53:13.325889+0000","last_peered":"2024-04-25T09:53:13.325889+0000","last_clean":"2024-04-25T09:53:13.325889+0000","last_became_active":"2024-04-25T09:53:07.789190+0000","last_became_peered":"2024-04-25T09:53:07.789190+0000","last_unstale":"2024-04-25T09:53:13.325889+0000","last_undegraded":"2024-04-25T09:53:13.325889+0000","last_fullsized":"2024-04-25T09:53:13.325889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326006+0000","last_change":"2024-04-25T09:53:11.841059+0000","last_active":"2024-04-25T09:53:13.326006+0000","last_peered":"2024-04-25T09:53:13.326006+0000","last_clean":"2024-04-25T09:53:13.326006+0000","last_became_active":"2024-04-25T09:53:11.840640+0000","last_became_peered":"2024-04-25T09:53:11.840640+0000","last_unstale":"2024-04-25T09:53:13.326006+0000","last_undegraded":"2024-04-25T09:53:13.326006+0000","last_fullsized":"2024-04-25T09:53:13.326006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:54.172755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810382+0000","last_change":"2024-04-25T09:53:06.777792+0000","last_active":"2024-04-25T09:53:12.810382+0000","last_peered":"2024-04-25T09:53:12.810382+0000","last_clean":"2024-04-25T09:53:12.810382+0000","last_became_active":"2024-04-25T09:53:06.777459+0000","last_became_peered":"2024-04-25T09:53:06.777459+0000","last_unstale":"2024-04-25T09:53:12.810382+0000","last_undegraded":"2024-04-25T09:53:12.810382+0000","last_fullsized":"2024-04-25T09:53:12.810382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:44.406768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326553+0000","last_change":"2024-04-25T09:53:07.798127+0000","last_active":"2024-04-25T09:53:13.326553+0000","last_peered":"2024-04-25T09:53:13.326553+0000","last_clean":"2024-04-25T09:53:13.326553+0000","last_became_active":"2024-04-25T09:53:07.797863+0000","last_became_peered":"2024-04-25T09:53:07.797863+0000","last_unstale":"2024-04-25T09:53:13.326553+0000","last_undegraded":"2024-04-25T09:53:13.326553+0000","last_fullsized":"2024-04-25T09:53:13.326553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326109+0000","last_change":"2024-04-25T09:53:11.841255+0000","last_active":"2024-04-25T09:53:13.326109+0000","last_peered":"2024-04-25T09:53:13.326109+0000","last_clean":"2024-04-25T09:53:13.326109+0000","last_became_active":"2024-04-25T09:53:11.840795+0000","last_became_peered":"2024-04-25T09:53:11.840795+0000","last_unstale":"2024-04-25T09:53:13.326109+0000","last_undegraded":"2024-04-25T09:53:13.326109+0000","last_fullsized":"2024-04-25T09:53:13.326109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:28.118453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810018+0000","last_change":"2024-04-25T09:53:06.784013+0000","last_active":"2024-04-25T09:53:12.810018+0000","last_peered":"2024-04-25T09:53:12.810018+0000","last_clean":"2024-04-25T09:53:12.810018+0000","last_became_active":"2024-04-25T09:53:06.783726+0000","last_became_peered":"2024-04-25T09:53:06.783726+0000","last_unstale":"2024-04-25T09:53:12.810018+0000","last_undegraded":"2024-04-25T09:53:12.810018+0000","last_fullsized":"2024-04-25T09:53:12.810018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327442+0000","last_change":"2024-04-25T09:53:07.796042+0000","last_active":"2024-04-25T09:53:13.327442+0000","last_peered":"2024-04-25T09:53:13.327442+0000","last_clean":"2024-04-25T09:53:13.327442+0000","last_became_active":"2024-04-25T09:53:07.795676+0000","last_became_peered":"2024-04-25T09:53:07.795676+0000","last_unstale":"2024-04-25T09:53:13.327442+0000","last_undegraded":"2024-04-25T09:53:13.327442+0000","last_fullsized":"2024-04-25T09:53:13.327442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:12.845651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326221+0000","last_change":"2024-04-25T09:53:11.827699+0000","last_active":"2024-04-25T09:53:13.326221+0000","last_peered":"2024-04-25T09:53:13.326221+0000","last_clean":"2024-04-25T09:53:13.326221+0000","last_became_active":"2024-04-25T09:53:11.827325+0000","last_became_peered":"2024-04-25T09:53:11.827325+0000","last_unstale":"2024-04-25T09:53:13.326221+0000","last_undegraded":"2024-04-25T09:53:13.326221+0000","last_fullsized":"2024-04-25T09:53:13.326221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.837684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810063+0000","last_change":"2024-04-25T09:53:07.800199+0000","last_active":"2024-04-25T09:53:12.810063+0000","last_peered":"2024-04-25T09:53:12.810063+0000","last_clean":"2024-04-25T09:53:12.810063+0000","last_became_active":"2024-04-25T09:53:07.799912+0000","last_became_peered":"2024-04-25T09:53:07.799912+0000","last_unstale":"2024-04-25T09:53:12.810063+0000","last_undegraded":"2024-04-25T09:53:12.810063+0000","last_fullsized":"2024-04-25T09:53:12.810063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810410+0000","last_change":"2024-04-25T09:53:06.783380+0000","last_active":"2024-04-25T09:53:12.810410+0000","last_peered":"2024-04-25T09:53:12.810410+0000","last_clean":"2024-04-25T09:53:12.810410+0000","last_became_active":"2024-04-25T09:53:06.783049+0000","last_became_peered":"2024-04-25T09:53:06.783049+0000","last_unstale":"2024-04-25T09:53:12.810410+0000","last_undegraded":"2024-04-25T09:53:12.810410+0000","last_fullsized":"2024-04-25T09:53:12.810410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327316+0000","last_change":"2024-04-25T09:53:11.839467+0000","last_active":"2024-04-25T09:53:13.327316+0000","last_peered":"2024-04-25T09:53:13.327316+0000","last_clean":"2024-04-25T09:53:13.327316+0000","last_became_active":"2024-04-25T09:53:11.839079+0000","last_became_peered":"2024-04-25T09:53:11.839079+0000","last_unstale":"2024-04-25T09:53:13.327316+0000","last_undegraded":"2024-04-25T09:53:13.327316+0000","last_fullsized":"2024-04-25T09:53:13.327316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325291+0000","last_change":"2024-04-25T09:53:07.791763+0000","last_active":"2024-04-25T09:53:13.325291+0000","last_peered":"2024-04-25T09:53:13.325291+0000","last_clean":"2024-04-25T09:53:13.325291+0000","last_became_active":"2024-04-25T09:53:07.790924+0000","last_became_peered":"2024-04-25T09:53:07.790924+0000","last_unstale":"2024-04-25T09:53:13.325291+0000","last_undegraded":"2024-04-25T09:53:13.325291+0000","last_fullsized":"2024-04-25T09:53:13.325291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326553+0000","last_change":"2024-04-25T09:53:06.779703+0000","last_active":"2024-04-25T09:53:13.326553+0000","last_peered":"2024-04-25T09:53:13.326553+0000","last_clean":"2024-04-25T09:53:13.326553+0000","last_became_active":"2024-04-25T09:53:06.779413+0000","last_became_peered":"2024-04-25T09:53:06.779413+0000","last_unstale":"2024-04-25T09:53:13.326553+0000","last_undegraded":"2024-04-25T09:53:13.326553+0000","last_fullsized":"2024-04-25T09:53:13.326553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326368+0000","last_change":"2024-04-25T09:53:11.840454+0000","last_active":"2024-04-25T09:53:13.326368+0000","last_peered":"2024-04-25T09:53:13.326368+0000","last_clean":"2024-04-25T09:53:13.326368+0000","last_became_active":"2024-04-25T09:53:11.840034+0000","last_became_peered":"2024-04-25T09:53:11.840034+0000","last_unstale":"2024-04-25T09:53:13.326368+0000","last_undegraded":"2024-04-25T09:53:13.326368+0000","last_fullsized":"2024-04-25T09:53:13.326368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.438696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326270+0000","last_change":"2024-04-25T09:53:07.786389+0000","last_active":"2024-04-25T09:53:13.326270+0000","last_peered":"2024-04-25T09:53:13.326270+0000","last_clean":"2024-04-25T09:53:13.326270+0000","last_became_active":"2024-04-25T09:53:07.786116+0000","last_became_peered":"2024-04-25T09:53:07.786116+0000","last_unstale":"2024-04-25T09:53:13.326270+0000","last_undegraded":"2024-04-25T09:53:13.326270+0000","last_fullsized":"2024-04-25T09:53:13.326270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325212+0000","last_change":"2024-04-25T09:53:06.780897+0000","last_active":"2024-04-25T09:53:13.325212+0000","last_peered":"2024-04-25T09:53:13.325212+0000","last_clean":"2024-04-25T09:53:13.325212+0000","last_became_active":"2024-04-25T09:53:06.780616+0000","last_became_peered":"2024-04-25T09:53:06.780616+0000","last_unstale":"2024-04-25T09:53:13.325212+0000","last_undegraded":"2024-04-25T09:53:13.325212+0000","last_fullsized":"2024-04-25T09:53:13.325212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:24.763796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326272+0000","last_change":"2024-04-25T09:53:11.838039+0000","last_active":"2024-04-25T09:53:13.326272+0000","last_peered":"2024-04-25T09:53:13.326272+0000","last_clean":"2024-04-25T09:53:13.326272+0000","last_became_active":"2024-04-25T09:53:11.837388+0000","last_became_peered":"2024-04-25T09:53:11.837388+0000","last_unstale":"2024-04-25T09:53:13.326272+0000","last_undegraded":"2024-04-25T09:53:13.326272+0000","last_fullsized":"2024-04-25T09:53:13.326272+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:39.753991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326308+0000","last_change":"2024-04-25T09:53:07.784394+0000","last_active":"2024-04-25T09:53:13.326308+0000","last_peered":"2024-04-25T09:53:13.326308+0000","last_clean":"2024-04-25T09:53:13.326308+0000","last_became_active":"2024-04-25T09:53:07.784060+0000","last_became_peered":"2024-04-25T09:53:07.784060+0000","last_unstale":"2024-04-25T09:53:13.326308+0000","last_undegraded":"2024-04-25T09:53:13.326308+0000","last_fullsized":"2024-04-25T09:53:13.326308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:36.593849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809684+0000","last_change":"2024-04-25T09:53:06.779579+0000","last_active":"2024-04-25T09:53:12.809684+0000","last_peered":"2024-04-25T09:53:12.809684+0000","last_clean":"2024-04-25T09:53:12.809684+0000","last_became_active":"2024-04-25T09:53:06.779296+0000","last_became_peered":"2024-04-25T09:53:06.779296+0000","last_unstale":"2024-04-25T09:53:12.809684+0000","last_undegraded":"2024-04-25T09:53:12.809684+0000","last_fullsized":"2024-04-25T09:53:12.809684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327226+0000","last_change":"2024-04-25T09:53:11.839381+0000","last_active":"2024-04-25T09:53:13.327226+0000","last_peered":"2024-04-25T09:53:13.327226+0000","last_clean":"2024-04-25T09:53:13.327226+0000","last_became_active":"2024-04-25T09:53:11.838941+0000","last_became_peered":"2024-04-25T09:53:11.838941+0000","last_unstale":"2024-04-25T09:53:13.327226+0000","last_undegraded":"2024-04-25T09:53:13.327226+0000","last_fullsized":"2024-04-25T09:53:13.327226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:59.415156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326930+0000","last_change":"2024-04-25T09:53:07.785627+0000","last_active":"2024-04-25T09:53:13.326930+0000","last_peered":"2024-04-25T09:53:13.326930+0000","last_clean":"2024-04-25T09:53:13.326930+0000","last_became_active":"2024-04-25T09:53:07.782773+0000","last_became_peered":"2024-04-25T09:53:07.782773+0000","last_unstale":"2024-04-25T09:53:13.326930+0000","last_undegraded":"2024-04-25T09:53:13.326930+0000","last_fullsized":"2024-04-25T09:53:13.326930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811412+0000","last_change":"2024-04-25T09:53:06.778161+0000","last_active":"2024-04-25T09:53:12.811412+0000","last_peered":"2024-04-25T09:53:12.811412+0000","last_clean":"2024-04-25T09:53:12.811412+0000","last_became_active":"2024-04-25T09:53:06.777837+0000","last_became_peered":"2024-04-25T09:53:06.777837+0000","last_unstale":"2024-04-25T09:53:12.811412+0000","last_undegraded":"2024-04-25T09:53:12.811412+0000","last_fullsized":"2024-04-25T09:53:12.811412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325940+0000","last_change":"2024-04-25T09:53:11.823677+0000","last_active":"2024-04-25T09:53:13.325940+0000","last_peered":"2024-04-25T09:53:13.325940+0000","last_clean":"2024-04-25T09:53:13.325940+0000","last_became_active":"2024-04-25T09:53:11.823278+0000","last_became_peered":"2024-04-25T09:53:11.823278+0000","last_unstale":"2024-04-25T09:53:13.325940+0000","last_undegraded":"2024-04-25T09:53:13.325940+0000","last_fullsized":"2024-04-25T09:53:13.325940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:45.277051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327064+0000","last_change":"2024-04-25T09:53:07.792276+0000","last_active":"2024-04-25T09:53:13.327064+0000","last_peered":"2024-04-25T09:53:13.327064+0000","last_clean":"2024-04-25T09:53:13.327064+0000","last_became_active":"2024-04-25T09:53:07.790961+0000","last_became_peered":"2024-04-25T09:53:07.790961+0000","last_unstale":"2024-04-25T09:53:13.327064+0000","last_undegraded":"2024-04-25T09:53:13.327064+0000","last_fullsized":"2024-04-25T09:53:13.327064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325875+0000","last_change":"2024-04-25T09:53:06.779465+0000","last_active":"2024-04-25T09:53:13.325875+0000","last_peered":"2024-04-25T09:53:13.325875+0000","last_clean":"2024-04-25T09:53:13.325875+0000","last_became_active":"2024-04-25T09:53:06.779122+0000","last_became_peered":"2024-04-25T09:53:06.779122+0000","last_unstale":"2024-04-25T09:53:13.325875+0000","last_undegraded":"2024-04-25T09:53:13.325875+0000","last_fullsized":"2024-04-25T09:53:13.325875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325863+0000","last_change":"2024-04-25T09:53:11.832958+0000","last_active":"2024-04-25T09:53:13.325863+0000","last_peered":"2024-04-25T09:53:13.325863+0000","last_clean":"2024-04-25T09:53:13.325863+0000","last_became_active":"2024-04-25T09:53:11.832522+0000","last_became_peered":"2024-04-25T09:53:11.832522+0000","last_unstale":"2024-04-25T09:53:13.325863+0000","last_undegraded":"2024-04-25T09:53:13.325863+0000","last_fullsized":"2024-04-25T09:53:13.325863+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:58.718292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325598+0000","last_change":"2024-04-25T09:53:07.779236+0000","last_active":"2024-04-25T09:53:13.325598+0000","last_peered":"2024-04-25T09:53:13.325598+0000","last_clean":"2024-04-25T09:53:13.325598+0000","last_became_active":"2024-04-25T09:53:07.778938+0000","last_became_peered":"2024-04-25T09:53:07.778938+0000","last_unstale":"2024-04-25T09:53:13.325598+0000","last_undegraded":"2024-04-25T09:53:13.325598+0000","last_fullsized":"2024-04-25T09:53:13.325598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810604+0000","last_change":"2024-04-25T09:53:06.780019+0000","last_active":"2024-04-25T09:53:12.810604+0000","last_peered":"2024-04-25T09:53:12.810604+0000","last_clean":"2024-04-25T09:53:12.810604+0000","last_became_active":"2024-04-25T09:53:06.779429+0000","last_became_peered":"2024-04-25T09:53:06.779429+0000","last_unstale":"2024-04-25T09:53:12.810604+0000","last_undegraded":"2024-04-25T09:53:12.810604+0000","last_fullsized":"2024-04-25T09:53:12.810604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:24.763796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325466+0000","last_change":"2024-04-25T09:53:11.835047+0000","last_active":"2024-04-25T09:53:13.325466+0000","last_peered":"2024-04-25T09:53:13.325466+0000","last_clean":"2024-04-25T09:53:13.325466+0000","last_became_active":"2024-04-25T09:53:11.833932+0000","last_became_peered":"2024-04-25T09:53:11.833932+0000","last_unstale":"2024-04-25T09:53:13.325466+0000","last_undegraded":"2024-04-25T09:53:13.325466+0000","last_fullsized":"2024-04-25T09:53:13.325466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:39.753991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327090+0000","last_change":"2024-04-25T09:53:07.786462+0000","last_active":"2024-04-25T09:53:13.327090+0000","last_peered":"2024-04-25T09:53:13.327090+0000","last_clean":"2024-04-25T09:53:13.327090+0000","last_became_active":"2024-04-25T09:53:07.785793+0000","last_became_peered":"2024-04-25T09:53:07.785793+0000","last_unstale":"2024-04-25T09:53:13.327090+0000","last_undegraded":"2024-04-25T09:53:13.327090+0000","last_fullsized":"2024-04-25T09:53:13.327090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.221246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325690+0000","last_change":"2024-04-25T09:53:06.780745+0000","last_active":"2024-04-25T09:53:13.325690+0000","last_peered":"2024-04-25T09:53:13.325690+0000","last_clean":"2024-04-25T09:53:13.325690+0000","last_became_active":"2024-04-25T09:53:06.780468+0000","last_became_peered":"2024-04-25T09:53:06.780468+0000","last_unstale":"2024-04-25T09:53:13.325690+0000","last_undegraded":"2024-04-25T09:53:13.325690+0000","last_fullsized":"2024-04-25T09:53:13.325690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326300+0000","last_change":"2024-04-25T09:53:11.832263+0000","last_active":"2024-04-25T09:53:13.326300+0000","last_peered":"2024-04-25T09:53:13.326300+0000","last_clean":"2024-04-25T09:53:13.326300+0000","last_became_active":"2024-04-25T09:53:11.831846+0000","last_became_peered":"2024-04-25T09:53:11.831846+0000","last_unstale":"2024-04-25T09:53:13.326300+0000","last_undegraded":"2024-04-25T09:53:13.326300+0000","last_fullsized":"2024-04-25T09:53:13.326300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:16.884859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326368+0000","last_change":"2024-04-25T09:53:07.797016+0000","last_active":"2024-04-25T09:53:13.326368+0000","last_peered":"2024-04-25T09:53:13.326368+0000","last_clean":"2024-04-25T09:53:13.326368+0000","last_became_active":"2024-04-25T09:53:07.796749+0000","last_became_peered":"2024-04-25T09:53:07.796749+0000","last_unstale":"2024-04-25T09:53:13.326368+0000","last_undegraded":"2024-04-25T09:53:13.326368+0000","last_fullsized":"2024-04-25T09:53:13.326368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:21.755503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326938+0000","last_change":"2024-04-25T09:53:06.782783+0000","last_active":"2024-04-25T09:53:13.326938+0000","last_peered":"2024-04-25T09:53:13.326938+0000","last_clean":"2024-04-25T09:53:13.326938+0000","last_became_active":"2024-04-25T09:53:06.782388+0000","last_became_peered":"2024-04-25T09:53:06.782388+0000","last_unstale":"2024-04-25T09:53:13.326938+0000","last_undegraded":"2024-04-25T09:53:13.326938+0000","last_fullsized":"2024-04-25T09:53:13.326938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325544+0000","last_change":"2024-04-25T09:53:11.832074+0000","last_active":"2024-04-25T09:53:13.325544+0000","last_peered":"2024-04-25T09:53:13.325544+0000","last_clean":"2024-04-25T09:53:13.325544+0000","last_became_active":"2024-04-25T09:53:11.831621+0000","last_became_peered":"2024-04-25T09:53:11.831621+0000","last_unstale":"2024-04-25T09:53:13.325544+0000","last_undegraded":"2024-04-25T09:53:13.325544+0000","last_fullsized":"2024-04-25T09:53:13.325544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.324944+0000","last_change":"2024-04-25T09:53:07.783910+0000","last_active":"2024-04-25T09:53:13.324944+0000","last_peered":"2024-04-25T09:53:13.324944+0000","last_clean":"2024-04-25T09:53:13.324944+0000","last_became_active":"2024-04-25T09:53:07.783625+0000","last_became_peered":"2024-04-25T09:53:07.783625+0000","last_unstale":"2024-04-25T09:53:13.324944+0000","last_undegraded":"2024-04-25T09:53:13.324944+0000","last_fullsized":"2024-04-25T09:53:13.324944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809538+0000","last_change":"2024-04-25T09:53:06.776134+0000","last_active":"2024-04-25T09:53:12.809538+0000","last_peered":"2024-04-25T09:53:12.809538+0000","last_clean":"2024-04-25T09:53:12.809538+0000","last_became_active":"2024-04-25T09:53:06.775787+0000","last_became_peered":"2024-04-25T09:53:06.775787+0000","last_unstale":"2024-04-25T09:53:12.809538+0000","last_undegraded":"2024-04-25T09:53:12.809538+0000","last_fullsized":"2024-04-25T09:53:12.809538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326396+0000","last_change":"2024-04-25T09:53:11.837526+0000","last_active":"2024-04-25T09:53:13.326396+0000","last_peered":"2024-04-25T09:53:13.326396+0000","last_clean":"2024-04-25T09:53:13.326396+0000","last_became_active":"2024-04-25T09:53:11.832037+0000","last_became_peered":"2024-04-25T09:53:11.832037+0000","last_unstale":"2024-04-25T09:53:13.326396+0000","last_undegraded":"2024-04-25T09:53:13.326396+0000","last_fullsized":"2024-04-25T09:53:13.326396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.727046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325424+0000","last_change":"2024-04-25T09:53:07.787155+0000","last_active":"2024-04-25T09:53:13.325424+0000","last_peered":"2024-04-25T09:53:13.325424+0000","last_clean":"2024-04-25T09:53:13.325424+0000","last_became_active":"2024-04-25T09:53:07.783553+0000","last_became_peered":"2024-04-25T09:53:07.783553+0000","last_unstale":"2024-04-25T09:53:13.325424+0000","last_undegraded":"2024-04-25T09:53:13.325424+0000","last_fullsized":"2024-04-25T09:53:13.325424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:12.089487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810045+0000","last_change":"2024-04-25T09:53:06.780626+0000","last_active":"2024-04-25T09:53:12.810045+0000","last_peered":"2024-04-25T09:53:12.810045+0000","last_clean":"2024-04-25T09:53:12.810045+0000","last_became_active":"2024-04-25T09:53:06.780364+0000","last_became_peered":"2024-04-25T09:53:06.780364+0000","last_unstale":"2024-04-25T09:53:12.810045+0000","last_undegraded":"2024-04-25T09:53:12.810045+0000","last_fullsized":"2024-04-25T09:53:12.810045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326877+0000","last_change":"2024-04-25T09:53:11.832887+0000","last_active":"2024-04-25T09:53:13.326877+0000","last_peered":"2024-04-25T09:53:13.326877+0000","last_clean":"2024-04-25T09:53:13.326877+0000","last_became_active":"2024-04-25T09:53:11.832471+0000","last_became_peered":"2024-04-25T09:53:11.832471+0000","last_unstale":"2024-04-25T09:53:13.326877+0000","last_undegraded":"2024-04-25T09:53:13.326877+0000","last_fullsized":"2024-04-25T09:53:13.326877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:03.438164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.324983+0000","last_change":"2024-04-25T09:53:07.793466+0000","last_active":"2024-04-25T09:53:13.324983+0000","last_peered":"2024-04-25T09:53:13.324983+0000","last_clean":"2024-04-25T09:53:13.324983+0000","last_became_active":"2024-04-25T09:53:07.793130+0000","last_became_peered":"2024-04-25T09:53:07.793130+0000","last_unstale":"2024-04-25T09:53:13.324983+0000","last_undegraded":"2024-04-25T09:53:13.324983+0000","last_fullsized":"2024-04-25T09:53:13.324983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326577+0000","last_change":"2024-04-25T09:53:06.780762+0000","last_active":"2024-04-25T09:53:13.326577+0000","last_peered":"2024-04-25T09:53:13.326577+0000","last_clean":"2024-04-25T09:53:13.326577+0000","last_became_active":"2024-04-25T09:53:06.780498+0000","last_became_peered":"2024-04-25T09:53:06.780498+0000","last_unstale":"2024-04-25T09:53:13.326577+0000","last_undegraded":"2024-04-25T09:53:13.326577+0000","last_fullsized":"2024-04-25T09:53:13.326577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810151+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:53:12.810151+0000","last_peered":"2024-04-25T09:53:12.810151+0000","last_clean":"2024-04-25T09:53:12.810151+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:53:12.810151+0000","last_undegraded":"2024-04-25T09:53:12.810151+0000","last_fullsized":"2024-04-25T09:53:12.810151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810776+0000","last_change":"2024-04-25T09:53:11.825407+0000","last_active":"2024-04-25T09:53:12.810776+0000","last_peered":"2024-04-25T09:53:12.810776+0000","last_clean":"2024-04-25T09:53:12.810776+0000","last_became_active":"2024-04-25T09:53:11.824448+0000","last_became_peered":"2024-04-25T09:53:11.824448+0000","last_unstale":"2024-04-25T09:53:12.810776+0000","last_undegraded":"2024-04-25T09:53:12.810776+0000","last_fullsized":"2024-04-25T09:53:12.810776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:59.415156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810231+0000","last_change":"2024-04-25T09:53:07.803634+0000","last_active":"2024-04-25T09:53:12.810231+0000","last_peered":"2024-04-25T09:53:12.810231+0000","last_clean":"2024-04-25T09:53:12.810231+0000","last_became_active":"2024-04-25T09:53:07.803367+0000","last_became_peered":"2024-04-25T09:53:07.803367+0000","last_unstale":"2024-04-25T09:53:12.810231+0000","last_undegraded":"2024-04-25T09:53:12.810231+0000","last_fullsized":"2024-04-25T09:53:12.810231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:55.375947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809562+0000","last_change":"2024-04-25T09:53:06.783978+0000","last_active":"2024-04-25T09:53:12.809562+0000","last_peered":"2024-04-25T09:53:12.809562+0000","last_clean":"2024-04-25T09:53:12.809562+0000","last_became_active":"2024-04-25T09:53:06.783614+0000","last_became_peered":"2024-04-25T09:53:06.783614+0000","last_unstale":"2024-04-25T09:53:12.809562+0000","last_undegraded":"2024-04-25T09:53:12.809562+0000","last_fullsized":"2024-04-25T09:53:12.809562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:52.306054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810011+0000","last_change":"2024-04-25T09:53:11.835086+0000","last_active":"2024-04-25T09:53:12.810011+0000","last_peered":"2024-04-25T09:53:12.810011+0000","last_clean":"2024-04-25T09:53:12.810011+0000","last_became_active":"2024-04-25T09:53:11.834674+0000","last_became_peered":"2024-04-25T09:53:11.834674+0000","last_unstale":"2024-04-25T09:53:12.810011+0000","last_undegraded":"2024-04-25T09:53:12.810011+0000","last_fullsized":"2024-04-25T09:53:12.810011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:01.272017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326681+0000","last_change":"2024-04-25T09:53:07.786966+0000","last_active":"2024-04-25T09:53:13.326681+0000","last_peered":"2024-04-25T09:53:13.326681+0000","last_clean":"2024-04-25T09:53:13.326681+0000","last_became_active":"2024-04-25T09:53:07.786097+0000","last_became_peered":"2024-04-25T09:53:07.786097+0000","last_unstale":"2024-04-25T09:53:13.326681+0000","last_undegraded":"2024-04-25T09:53:13.326681+0000","last_fullsized":"2024-04-25T09:53:13.326681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326102+0000","last_change":"2024-04-25T09:53:06.781647+0000","last_active":"2024-04-25T09:53:13.326102+0000","last_peered":"2024-04-25T09:53:13.326102+0000","last_clean":"2024-04-25T09:53:13.326102+0000","last_became_active":"2024-04-25T09:53:06.781360+0000","last_became_peered":"2024-04-25T09:53:06.781360+0000","last_unstale":"2024-04-25T09:53:13.326102+0000","last_undegraded":"2024-04-25T09:53:13.326102+0000","last_fullsized":"2024-04-25T09:53:13.326102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:44.406768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326043+0000","last_change":"2024-04-25T09:53:11.835499+0000","last_active":"2024-04-25T09:53:13.326043+0000","last_peered":"2024-04-25T09:53:13.326043+0000","last_clean":"2024-04-25T09:53:13.326043+0000","last_became_active":"2024-04-25T09:53:11.834613+0000","last_became_peered":"2024-04-25T09:53:11.834613+0000","last_unstale":"2024-04-25T09:53:13.326043+0000","last_undegraded":"2024-04-25T09:53:13.326043+0000","last_fullsized":"2024-04-25T09:53:13.326043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:16.884859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811569+0000","last_change":"2024-04-25T09:53:07.799270+0000","last_active":"2024-04-25T09:53:12.811569+0000","last_peered":"2024-04-25T09:53:12.811569+0000","last_clean":"2024-04-25T09:53:12.811569+0000","last_became_active":"2024-04-25T09:53:07.799005+0000","last_became_peered":"2024-04-25T09:53:07.799005+0000","last_unstale":"2024-04-25T09:53:12.811569+0000","last_undegraded":"2024-04-25T09:53:12.811569+0000","last_fullsized":"2024-04-25T09:53:12.811569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:47.687838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809643+0000","last_change":"2024-04-25T09:53:06.776508+0000","last_active":"2024-04-25T09:53:12.809643+0000","last_peered":"2024-04-25T09:53:12.809643+0000","last_clean":"2024-04-25T09:53:12.809643+0000","last_became_active":"2024-04-25T09:53:06.775938+0000","last_became_peered":"2024-04-25T09:53:06.775938+0000","last_unstale":"2024-04-25T09:53:12.809643+0000","last_undegraded":"2024-04-25T09:53:12.809643+0000","last_fullsized":"2024-04-25T09:53:12.809643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325709+0000","last_change":"2024-04-25T09:53:11.823495+0000","last_active":"2024-04-25T09:53:13.325709+0000","last_peered":"2024-04-25T09:53:13.325709+0000","last_clean":"2024-04-25T09:53:13.325709+0000","last_became_active":"2024-04-25T09:53:11.823070+0000","last_became_peered":"2024-04-25T09:53:11.823070+0000","last_unstale":"2024-04-25T09:53:13.325709+0000","last_undegraded":"2024-04-25T09:53:13.325709+0000","last_fullsized":"2024-04-25T09:53:13.325709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:58.601489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810162+0000","last_change":"2024-04-25T09:53:07.796560+0000","last_active":"2024-04-25T09:53:12.810162+0000","last_peered":"2024-04-25T09:53:12.810162+0000","last_clean":"2024-04-25T09:53:12.810162+0000","last_became_active":"2024-04-25T09:53:07.796288+0000","last_became_peered":"2024-04-25T09:53:07.796288+0000","last_unstale":"2024-04-25T09:53:12.810162+0000","last_undegraded":"2024-04-25T09:53:12.810162+0000","last_fullsized":"2024-04-25T09:53:12.810162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327440+0000","last_change":"2024-04-25T09:53:06.776524+0000","last_active":"2024-04-25T09:53:13.327440+0000","last_peered":"2024-04-25T09:53:13.327440+0000","last_clean":"2024-04-25T09:53:13.327440+0000","last_became_active":"2024-04-25T09:53:06.776156+0000","last_became_peered":"2024-04-25T09:53:06.776156+0000","last_unstale":"2024-04-25T09:53:13.327440+0000","last_undegraded":"2024-04-25T09:53:13.327440+0000","last_fullsized":"2024-04-25T09:53:13.327440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:23.222453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325855+0000","last_change":"2024-04-25T09:53:11.840103+0000","last_active":"2024-04-25T09:53:13.325855+0000","last_peered":"2024-04-25T09:53:13.325855+0000","last_clean":"2024-04-25T09:53:13.325855+0000","last_became_active":"2024-04-25T09:53:11.839706+0000","last_became_peered":"2024-04-25T09:53:11.839706+0000","last_unstale":"2024-04-25T09:53:13.325855+0000","last_undegraded":"2024-04-25T09:53:13.325855+0000","last_fullsized":"2024-04-25T09:53:13.325855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:35.127642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809957+0000","last_change":"2024-04-25T09:53:07.796678+0000","last_active":"2024-04-25T09:53:12.809957+0000","last_peered":"2024-04-25T09:53:12.809957+0000","last_clean":"2024-04-25T09:53:12.809957+0000","last_became_active":"2024-04-25T09:53:07.796401+0000","last_became_peered":"2024-04-25T09:53:07.796401+0000","last_unstale":"2024-04-25T09:53:12.809957+0000","last_undegraded":"2024-04-25T09:53:12.809957+0000","last_fullsized":"2024-04-25T09:53:12.809957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809151+0000","last_change":"2024-04-25T09:53:06.780411+0000","last_active":"2024-04-25T09:53:12.809151+0000","last_peered":"2024-04-25T09:53:12.809151+0000","last_clean":"2024-04-25T09:53:12.809151+0000","last_became_active":"2024-04-25T09:53:06.780094+0000","last_became_peered":"2024-04-25T09:53:06.780094+0000","last_unstale":"2024-04-25T09:53:12.809151+0000","last_undegraded":"2024-04-25T09:53:12.809151+0000","last_fullsized":"2024-04-25T09:53:12.809151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326778+0000","last_change":"2024-04-25T09:53:11.823203+0000","last_active":"2024-04-25T09:53:13.326778+0000","last_peered":"2024-04-25T09:53:13.326778+0000","last_clean":"2024-04-25T09:53:13.326778+0000","last_became_active":"2024-04-25T09:53:11.822736+0000","last_became_peered":"2024-04-25T09:53:11.822736+0000","last_unstale":"2024-04-25T09:53:13.326778+0000","last_undegraded":"2024-04-25T09:53:13.326778+0000","last_fullsized":"2024-04-25T09:53:13.326778+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:05.899921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809909+0000","last_change":"2024-04-25T09:53:07.797880+0000","last_active":"2024-04-25T09:53:12.809909+0000","last_peered":"2024-04-25T09:53:12.809909+0000","last_clean":"2024-04-25T09:53:12.809909+0000","last_became_active":"2024-04-25T09:53:07.797626+0000","last_became_peered":"2024-04-25T09:53:07.797626+0000","last_unstale":"2024-04-25T09:53:12.809909+0000","last_undegraded":"2024-04-25T09:53:12.809909+0000","last_fullsized":"2024-04-25T09:53:12.809909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810332+0000","last_change":"2024-04-25T09:53:06.781752+0000","last_active":"2024-04-25T09:53:12.810332+0000","last_peered":"2024-04-25T09:53:12.810332+0000","last_clean":"2024-04-25T09:53:12.810332+0000","last_became_active":"2024-04-25T09:53:06.781191+0000","last_became_peered":"2024-04-25T09:53:06.781191+0000","last_unstale":"2024-04-25T09:53:12.810332+0000","last_undegraded":"2024-04-25T09:53:12.810332+0000","last_fullsized":"2024-04-25T09:53:12.810332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326993+0000","last_change":"2024-04-25T09:53:11.839382+0000","last_active":"2024-04-25T09:53:13.326993+0000","last_peered":"2024-04-25T09:53:13.326993+0000","last_clean":"2024-04-25T09:53:13.326993+0000","last_became_active":"2024-04-25T09:53:11.839017+0000","last_became_peered":"2024-04-25T09:53:11.839017+0000","last_unstale":"2024-04-25T09:53:13.326993+0000","last_undegraded":"2024-04-25T09:53:13.326993+0000","last_fullsized":"2024-04-25T09:53:13.326993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:08.728070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810403+0000","last_change":"2024-04-25T09:53:07.797176+0000","last_active":"2024-04-25T09:53:12.810403+0000","last_peered":"2024-04-25T09:53:12.810403+0000","last_clean":"2024-04-25T09:53:12.810403+0000","last_became_active":"2024-04-25T09:53:07.796900+0000","last_became_peered":"2024-04-25T09:53:07.796900+0000","last_unstale":"2024-04-25T09:53:12.810403+0000","last_undegraded":"2024-04-25T09:53:12.810403+0000","last_fullsized":"2024-04-25T09:53:12.810403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326836+0000","last_change":"2024-04-25T09:53:06.783568+0000","last_active":"2024-04-25T09:53:13.326836+0000","last_peered":"2024-04-25T09:53:13.326836+0000","last_clean":"2024-04-25T09:53:13.326836+0000","last_became_active":"2024-04-25T09:53:06.783237+0000","last_became_peered":"2024-04-25T09:53:06.783237+0000","last_unstale":"2024-04-25T09:53:13.326836+0000","last_undegraded":"2024-04-25T09:53:13.326836+0000","last_fullsized":"2024-04-25T09:53:13.326836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326468+0000","last_change":"2024-04-25T09:53:11.835512+0000","last_active":"2024-04-25T09:53:13.326468+0000","last_peered":"2024-04-25T09:53:13.326468+0000","last_clean":"2024-04-25T09:53:13.326468+0000","last_became_active":"2024-04-25T09:53:11.835133+0000","last_became_peered":"2024-04-25T09:53:11.835133+0000","last_unstale":"2024-04-25T09:53:13.326468+0000","last_undegraded":"2024-04-25T09:53:13.326468+0000","last_fullsized":"2024-04-25T09:53:13.326468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:13.599619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326511+0000","last_change":"2024-04-25T09:53:07.788286+0000","last_active":"2024-04-25T09:53:13.326511+0000","last_peered":"2024-04-25T09:53:13.326511+0000","last_clean":"2024-04-25T09:53:13.326511+0000","last_became_active":"2024-04-25T09:53:07.787920+0000","last_became_peered":"2024-04-25T09:53:07.787920+0000","last_unstale":"2024-04-25T09:53:13.326511+0000","last_undegraded":"2024-04-25T09:53:13.326511+0000","last_fullsized":"2024-04-25T09:53:13.326511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325923+0000","last_change":"2024-04-25T09:53:06.772006+0000","last_active":"2024-04-25T09:53:13.325923+0000","last_peered":"2024-04-25T09:53:13.325923+0000","last_clean":"2024-04-25T09:53:13.325923+0000","last_became_active":"2024-04-25T09:53:06.771702+0000","last_became_peered":"2024-04-25T09:53:06.771702+0000","last_unstale":"2024-04-25T09:53:13.325923+0000","last_undegraded":"2024-04-25T09:53:13.325923+0000","last_fullsized":"2024-04-25T09:53:13.325923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:59.219341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327029+0000","last_change":"2024-04-25T09:53:11.821559+0000","last_active":"2024-04-25T09:53:13.327029+0000","last_peered":"2024-04-25T09:53:13.327029+0000","last_clean":"2024-04-25T09:53:13.327029+0000","last_became_active":"2024-04-25T09:53:11.821133+0000","last_became_peered":"2024-04-25T09:53:11.821133+0000","last_unstale":"2024-04-25T09:53:13.327029+0000","last_undegraded":"2024-04-25T09:53:13.327029+0000","last_fullsized":"2024-04-25T09:53:13.327029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:58.718292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810108+0000","last_change":"2024-04-25T09:53:07.797403+0000","last_active":"2024-04-25T09:53:12.810108+0000","last_peered":"2024-04-25T09:53:12.810108+0000","last_clean":"2024-04-25T09:53:12.810108+0000","last_became_active":"2024-04-25T09:53:07.796257+0000","last_became_peered":"2024-04-25T09:53:07.796257+0000","last_unstale":"2024-04-25T09:53:12.810108+0000","last_undegraded":"2024-04-25T09:53:12.810108+0000","last_fullsized":"2024-04-25T09:53:12.810108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:17.046098+0000","last_change":"2024-04-25T09:53:06.773036+0000","last_active":"2024-04-25T09:53:17.046098+0000","last_peered":"2024-04-25T09:53:17.046098+0000","last_clean":"2024-04-25T09:53:17.046098+0000","last_became_active":"2024-04-25T09:53:06.772729+0000","last_became_peered":"2024-04-25T09:53:06.772729+0000","last_unstale":"2024-04-25T09:53:17.046098+0000","last_undegraded":"2024-04-25T09:53:17.046098+0000","last_fullsized":"2024-04-25T09:53:17.046098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326453+0000","last_change":"2024-04-25T09:53:11.833689+0000","last_active":"2024-04-25T09:53:13.326453+0000","last_peered":"2024-04-25T09:53:13.326453+0000","last_clean":"2024-04-25T09:53:13.326453+0000","last_became_active":"2024-04-25T09:53:11.832588+0000","last_became_peered":"2024-04-25T09:53:11.832588+0000","last_unstale":"2024-04-25T09:53:13.326453+0000","last_undegraded":"2024-04-25T09:53:13.326453+0000","last_fullsized":"2024-04-25T09:53:13.326453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:05.899921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327010+0000","last_change":"2024-04-25T09:53:07.791857+0000","last_active":"2024-04-25T09:53:13.327010+0000","last_peered":"2024-04-25T09:53:13.327010+0000","last_clean":"2024-04-25T09:53:13.327010+0000","last_became_active":"2024-04-25T09:53:07.791606+0000","last_became_peered":"2024-04-25T09:53:07.791606+0000","last_unstale":"2024-04-25T09:53:13.327010+0000","last_undegraded":"2024-04-25T09:53:13.327010+0000","last_fullsized":"2024-04-25T09:53:13.327010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.768421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325995+0000","last_change":"2024-04-25T09:53:06.783616+0000","last_active":"2024-04-25T09:53:13.325995+0000","last_peered":"2024-04-25T09:53:13.325995+0000","last_clean":"2024-04-25T09:53:13.325995+0000","last_became_active":"2024-04-25T09:53:06.783302+0000","last_became_peered":"2024-04-25T09:53:06.783302+0000","last_unstale":"2024-04-25T09:53:13.325995+0000","last_undegraded":"2024-04-25T09:53:13.325995+0000","last_fullsized":"2024-04-25T09:53:13.325995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326861+0000","last_change":"2024-04-25T09:53:11.840956+0000","last_active":"2024-04-25T09:53:13.326861+0000","last_peered":"2024-04-25T09:53:13.326861+0000","last_clean":"2024-04-25T09:53:13.326861+0000","last_became_active":"2024-04-25T09:53:11.840316+0000","last_became_peered":"2024-04-25T09:53:11.840316+0000","last_unstale":"2024-04-25T09:53:13.326861+0000","last_undegraded":"2024-04-25T09:53:13.326861+0000","last_fullsized":"2024-04-25T09:53:13.326861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:01.272017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326067+0000","last_change":"2024-04-25T09:53:07.787945+0000","last_active":"2024-04-25T09:53:13.326067+0000","last_peered":"2024-04-25T09:53:13.326067+0000","last_clean":"2024-04-25T09:53:13.326067+0000","last_became_active":"2024-04-25T09:53:07.786113+0000","last_became_peered":"2024-04-25T09:53:07.786113+0000","last_unstale":"2024-04-25T09:53:13.326067+0000","last_undegraded":"2024-04-25T09:53:13.326067+0000","last_fullsized":"2024-04-25T09:53:13.326067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327211+0000","last_change":"2024-04-25T09:53:06.783604+0000","last_active":"2024-04-25T09:53:13.327211+0000","last_peered":"2024-04-25T09:53:13.327211+0000","last_clean":"2024-04-25T09:53:13.327211+0000","last_became_active":"2024-04-25T09:53:06.783341+0000","last_became_peered":"2024-04-25T09:53:06.783341+0000","last_unstale":"2024-04-25T09:53:13.327211+0000","last_undegraded":"2024-04-25T09:53:13.327211+0000","last_fullsized":"2024-04-25T09:53:13.327211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326141+0000","last_change":"2024-04-25T09:53:11.836053+0000","last_active":"2024-04-25T09:53:13.326141+0000","last_peered":"2024-04-25T09:53:13.326141+0000","last_clean":"2024-04-25T09:53:13.326141+0000","last_became_active":"2024-04-25T09:53:11.835318+0000","last_became_peered":"2024-04-25T09:53:11.835318+0000","last_unstale":"2024-04-25T09:53:13.326141+0000","last_undegraded":"2024-04-25T09:53:13.326141+0000","last_fullsized":"2024-04-25T09:53:13.326141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:49.717043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327141+0000","last_change":"2024-04-25T09:53:07.793149+0000","last_active":"2024-04-25T09:53:13.327141+0000","last_peered":"2024-04-25T09:53:13.327141+0000","last_clean":"2024-04-25T09:53:13.327141+0000","last_became_active":"2024-04-25T09:53:07.792896+0000","last_became_peered":"2024-04-25T09:53:07.792896+0000","last_unstale":"2024-04-25T09:53:13.327141+0000","last_undegraded":"2024-04-25T09:53:13.327141+0000","last_fullsized":"2024-04-25T09:53:13.327141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:55.375947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811223+0000","last_change":"2024-04-25T09:53:06.780916+0000","last_active":"2024-04-25T09:53:12.811223+0000","last_peered":"2024-04-25T09:53:12.811223+0000","last_clean":"2024-04-25T09:53:12.811223+0000","last_became_active":"2024-04-25T09:53:06.780631+0000","last_became_peered":"2024-04-25T09:53:06.780631+0000","last_unstale":"2024-04-25T09:53:12.811223+0000","last_undegraded":"2024-04-25T09:53:12.811223+0000","last_fullsized":"2024-04-25T09:53:12.811223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325834+0000","last_change":"2024-04-25T09:53:11.821088+0000","last_active":"2024-04-25T09:53:13.325834+0000","last_peered":"2024-04-25T09:53:13.325834+0000","last_clean":"2024-04-25T09:53:13.325834+0000","last_became_active":"2024-04-25T09:53:11.820662+0000","last_became_peered":"2024-04-25T09:53:11.820662+0000","last_unstale":"2024-04-25T09:53:13.325834+0000","last_undegraded":"2024-04-25T09:53:13.325834+0000","last_fullsized":"2024-04-25T09:53:13.325834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:58.601489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326720+0000","last_change":"2024-04-25T09:53:11.838966+0000","last_active":"2024-04-25T09:53:13.326720+0000","last_peered":"2024-04-25T09:53:13.326720+0000","last_clean":"2024-04-25T09:53:13.326720+0000","last_became_active":"2024-04-25T09:53:11.838542+0000","last_became_peered":"2024-04-25T09:53:11.838542+0000","last_unstale":"2024-04-25T09:53:13.326720+0000","last_undegraded":"2024-04-25T09:53:13.326720+0000","last_fullsized":"2024-04-25T09:53:13.326720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.727046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325110+0000","last_change":"2024-04-25T09:53:07.784870+0000","last_active":"2024-04-25T09:53:13.325110+0000","last_peered":"2024-04-25T09:53:13.325110+0000","last_clean":"2024-04-25T09:53:13.325110+0000","last_became_active":"2024-04-25T09:53:07.784587+0000","last_became_peered":"2024-04-25T09:53:07.784587+0000","last_unstale":"2024-04-25T09:53:13.325110+0000","last_undegraded":"2024-04-25T09:53:13.325110+0000","last_fullsized":"2024-04-25T09:53:13.325110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325791+0000","last_change":"2024-04-25T09:53:06.782572+0000","last_active":"2024-04-25T09:53:13.325791+0000","last_peered":"2024-04-25T09:53:13.325791+0000","last_clean":"2024-04-25T09:53:13.325791+0000","last_became_active":"2024-04-25T09:53:06.781792+0000","last_became_peered":"2024-04-25T09:53:06.781792+0000","last_unstale":"2024-04-25T09:53:13.325791+0000","last_undegraded":"2024-04-25T09:53:13.325791+0000","last_fullsized":"2024-04-25T09:53:13.325791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811107+0000","last_change":"2024-04-25T09:53:11.838770+0000","last_active":"2024-04-25T09:53:12.811107+0000","last_peered":"2024-04-25T09:53:12.811107+0000","last_clean":"2024-04-25T09:53:12.811107+0000","last_became_active":"2024-04-25T09:53:11.838328+0000","last_became_peered":"2024-04-25T09:53:11.838328+0000","last_unstale":"2024-04-25T09:53:12.811107+0000","last_undegraded":"2024-04-25T09:53:12.811107+0000","last_fullsized":"2024-04-25T09:53:12.811107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326187+0000","last_change":"2024-04-25T09:53:07.791725+0000","last_active":"2024-04-25T09:53:13.326187+0000","last_peered":"2024-04-25T09:53:13.326187+0000","last_clean":"2024-04-25T09:53:13.326187+0000","last_became_active":"2024-04-25T09:53:07.791386+0000","last_became_peered":"2024-04-25T09:53:07.791386+0000","last_unstale":"2024-04-25T09:53:13.326187+0000","last_undegraded":"2024-04-25T09:53:13.326187+0000","last_fullsized":"2024-04-25T09:53:13.326187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326069+0000","last_change":"2024-04-25T09:53:06.775205+0000","last_active":"2024-04-25T09:53:13.326069+0000","last_peered":"2024-04-25T09:53:13.326069+0000","last_clean":"2024-04-25T09:53:13.326069+0000","last_became_active":"2024-04-25T09:53:06.774911+0000","last_became_peered":"2024-04-25T09:53:06.774911+0000","last_unstale":"2024-04-25T09:53:13.326069+0000","last_undegraded":"2024-04-25T09:53:13.326069+0000","last_fullsized":"2024-04-25T09:53:13.326069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.898659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810239+0000","last_change":"2024-04-25T09:53:06.781528+0000","last_active":"2024-04-25T09:53:12.810239+0000","last_peered":"2024-04-25T09:53:12.810239+0000","last_clean":"2024-04-25T09:53:12.810239+0000","last_became_active":"2024-04-25T09:53:06.781248+0000","last_became_peered":"2024-04-25T09:53:06.781248+0000","last_unstale":"2024-04-25T09:53:12.810239+0000","last_undegraded":"2024-04-25T09:53:12.810239+0000","last_fullsized":"2024-04-25T09:53:12.810239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325829+0000","last_change":"2024-04-25T09:53:07.783095+0000","last_active":"2024-04-25T09:53:13.325829+0000","last_peered":"2024-04-25T09:53:13.325829+0000","last_clean":"2024-04-25T09:53:13.325829+0000","last_became_active":"2024-04-25T09:53:07.782578+0000","last_became_peered":"2024-04-25T09:53:07.782578+0000","last_unstale":"2024-04-25T09:53:13.325829+0000","last_undegraded":"2024-04-25T09:53:13.325829+0000","last_fullsized":"2024-04-25T09:53:13.325829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809581+0000","last_change":"2024-04-25T09:53:11.834200+0000","last_active":"2024-04-25T09:53:12.809581+0000","last_peered":"2024-04-25T09:53:12.809581+0000","last_clean":"2024-04-25T09:53:12.809581+0000","last_became_active":"2024-04-25T09:53:11.833774+0000","last_became_peered":"2024-04-25T09:53:11.833774+0000","last_unstale":"2024-04-25T09:53:12.809581+0000","last_undegraded":"2024-04-25T09:53:12.809581+0000","last_fullsized":"2024-04-25T09:53:12.809581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:42.139275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325379+0000","last_change":"2024-04-25T09:53:06.777867+0000","last_active":"2024-04-25T09:53:13.325379+0000","last_peered":"2024-04-25T09:53:13.325379+0000","last_clean":"2024-04-25T09:53:13.325379+0000","last_became_active":"2024-04-25T09:53:06.777566+0000","last_became_peered":"2024-04-25T09:53:06.777566+0000","last_unstale":"2024-04-25T09:53:13.325379+0000","last_undegraded":"2024-04-25T09:53:13.325379+0000","last_fullsized":"2024-04-25T09:53:13.325379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:52.306054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326173+0000","last_change":"2024-04-25T09:53:07.781653+0000","last_active":"2024-04-25T09:53:13.326173+0000","last_peered":"2024-04-25T09:53:13.326173+0000","last_clean":"2024-04-25T09:53:13.326173+0000","last_became_active":"2024-04-25T09:53:07.781274+0000","last_became_peered":"2024-04-25T09:53:07.781274+0000","last_unstale":"2024-04-25T09:53:13.326173+0000","last_undegraded":"2024-04-25T09:53:13.326173+0000","last_fullsized":"2024-04-25T09:53:13.326173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:02.037522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326829+0000","last_change":"2024-04-25T09:53:11.834554+0000","last_active":"2024-04-25T09:53:13.326829+0000","last_peered":"2024-04-25T09:53:13.326829+0000","last_clean":"2024-04-25T09:53:13.326829+0000","last_became_active":"2024-04-25T09:53:11.834143+0000","last_became_peered":"2024-04-25T09:53:11.834143+0000","last_unstale":"2024-04-25T09:53:13.326829+0000","last_undegraded":"2024-04-25T09:53:13.326829+0000","last_fullsized":"2024-04-25T09:53:13.326829+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:37.372311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809684+0000","last_change":"2024-04-25T09:53:07.798578+0000","last_active":"2024-04-25T09:53:12.809684+0000","last_peered":"2024-04-25T09:53:12.809684+0000","last_clean":"2024-04-25T09:53:12.809684+0000","last_became_active":"2024-04-25T09:53:07.798310+0000","last_became_peered":"2024-04-25T09:53:07.798310+0000","last_unstale":"2024-04-25T09:53:12.809684+0000","last_undegraded":"2024-04-25T09:53:12.809684+0000","last_fullsized":"2024-04-25T09:53:12.809684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809471+0000","last_change":"2024-04-25T09:53:07.780646+0000","last_active":"2024-04-25T09:53:12.809471+0000","last_peered":"2024-04-25T09:53:12.809471+0000","last_clean":"2024-04-25T09:53:12.809471+0000","last_became_active":"2024-04-25T09:53:07.780321+0000","last_became_peered":"2024-04-25T09:53:07.780321+0000","last_unstale":"2024-04-25T09:53:12.809471+0000","last_undegraded":"2024-04-25T09:53:12.809471+0000","last_fullsized":"2024-04-25T09:53:12.809471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:44.576758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326872+0000","last_change":"2024-04-25T09:53:07.787491+0000","last_active":"2024-04-25T09:53:13.326872+0000","last_peered":"2024-04-25T09:53:13.326872+0000","last_clean":"2024-04-25T09:53:13.326872+0000","last_became_active":"2024-04-25T09:53:07.787214+0000","last_became_peered":"2024-04-25T09:53:07.787214+0000","last_unstale":"2024-04-25T09:53:13.326872+0000","last_undegraded":"2024-04-25T09:53:13.326872+0000","last_fullsized":"2024-04-25T09:53:13.326872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:35.714783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325728+0000","last_change":"2024-04-25T09:53:07.789805+0000","last_active":"2024-04-25T09:53:13.325728+0000","last_peered":"2024-04-25T09:53:13.325728+0000","last_clean":"2024-04-25T09:53:13.325728+0000","last_became_active":"2024-04-25T09:53:07.788987+0000","last_became_peered":"2024-04-25T09:53:07.788987+0000","last_unstale":"2024-04-25T09:53:13.325728+0000","last_undegraded":"2024-04-25T09:53:13.325728+0000","last_fullsized":"2024-04-25T09:53:13.325728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326515+0000","last_change":"2024-04-25T09:53:07.792754+0000","last_active":"2024-04-25T09:53:13.326515+0000","last_peered":"2024-04-25T09:53:13.326515+0000","last_clean":"2024-04-25T09:53:13.326515+0000","last_became_active":"2024-04-25T09:53:07.792223+0000","last_became_peered":"2024-04-25T09:53:07.792223+0000","last_unstale":"2024-04-25T09:53:13.326515+0000","last_undegraded":"2024-04-25T09:53:13.326515+0000","last_fullsized":"2024-04-25T09:53:13.326515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325889+0000","last_change":"2024-04-25T09:53:07.783936+0000","last_active":"2024-04-25T09:53:13.325889+0000","last_peered":"2024-04-25T09:53:13.325889+0000","last_clean":"2024-04-25T09:53:13.325889+0000","last_became_active":"2024-04-25T09:53:07.783646+0000","last_became_peered":"2024-04-25T09:53:07.783646+0000","last_unstale":"2024-04-25T09:53:13.325889+0000","last_undegraded":"2024-04-25T09:53:13.325889+0000","last_fullsized":"2024-04-25T09:53:13.325889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810830+0000","last_change":"2024-04-25T09:53:07.789428+0000","last_active":"2024-04-25T09:53:12.810830+0000","last_peered":"2024-04-25T09:53:12.810830+0000","last_clean":"2024-04-25T09:53:12.810830+0000","last_became_active":"2024-04-25T09:53:07.789086+0000","last_became_peered":"2024-04-25T09:53:07.789086+0000","last_unstale":"2024-04-25T09:53:12.810830+0000","last_undegraded":"2024-04-25T09:53:12.810830+0000","last_fullsized":"2024-04-25T09:53:12.810830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327059+0000","last_change":"2024-04-25T09:53:07.785822+0000","last_active":"2024-04-25T09:53:13.327059+0000","last_peered":"2024-04-25T09:53:13.327059+0000","last_clean":"2024-04-25T09:53:13.327059+0000","last_became_active":"2024-04-25T09:53:07.785535+0000","last_became_peered":"2024-04-25T09:53:07.785535+0000","last_unstale":"2024-04-25T09:53:13.327059+0000","last_undegraded":"2024-04-25T09:53:13.327059+0000","last_fullsized":"2024-04-25T09:53:13.327059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325833+0000","last_change":"2024-04-25T09:53:07.787296+0000","last_active":"2024-04-25T09:53:13.325833+0000","last_peered":"2024-04-25T09:53:13.325833+0000","last_clean":"2024-04-25T09:53:13.325833+0000","last_became_active":"2024-04-25T09:53:07.787011+0000","last_became_peered":"2024-04-25T09:53:07.787011+0000","last_unstale":"2024-04-25T09:53:13.325833+0000","last_undegraded":"2024-04-25T09:53:13.325833+0000","last_fullsized":"2024-04-25T09:53:13.325833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325938+0000","last_change":"2024-04-25T09:53:07.785405+0000","last_active":"2024-04-25T09:53:13.325938+0000","last_peered":"2024-04-25T09:53:13.325938+0000","last_clean":"2024-04-25T09:53:13.325938+0000","last_became_active":"2024-04-25T09:53:07.785085+0000","last_became_peered":"2024-04-25T09:53:07.785085+0000","last_unstale":"2024-04-25T09:53:13.325938+0000","last_undegraded":"2024-04-25T09:53:13.325938+0000","last_fullsized":"2024-04-25T09:53:13.325938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326705+0000","last_change":"2024-04-25T09:53:07.792913+0000","last_active":"2024-04-25T09:53:13.326705+0000","last_peered":"2024-04-25T09:53:13.326705+0000","last_clean":"2024-04-25T09:53:13.326705+0000","last_became_active":"2024-04-25T09:53:07.792666+0000","last_became_peered":"2024-04-25T09:53:07.792666+0000","last_unstale":"2024-04-25T09:53:13.326705+0000","last_undegraded":"2024-04-25T09:53:13.326705+0000","last_fullsized":"2024-04-25T09:53:13.326705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325521+0000","last_change":"2024-04-25T09:53:07.789785+0000","last_active":"2024-04-25T09:53:13.325521+0000","last_peered":"2024-04-25T09:53:13.325521+0000","last_clean":"2024-04-25T09:53:13.325521+0000","last_became_active":"2024-04-25T09:53:07.789109+0000","last_became_peered":"2024-04-25T09:53:07.789109+0000","last_unstale":"2024-04-25T09:53:13.325521+0000","last_undegraded":"2024-04-25T09:53:13.325521+0000","last_fullsized":"2024-04-25T09:53:13.325521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:28.708261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811599+0000","last_change":"2024-04-25T09:53:07.800839+0000","last_active":"2024-04-25T09:53:12.811599+0000","last_peered":"2024-04-25T09:53:12.811599+0000","last_clean":"2024-04-25T09:53:12.811599+0000","last_became_active":"2024-04-25T09:53:07.800570+0000","last_became_peered":"2024-04-25T09:53:07.800570+0000","last_unstale":"2024-04-25T09:53:12.811599+0000","last_undegraded":"2024-04-25T09:53:12.811599+0000","last_fullsized":"2024-04-25T09:53:12.811599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810497+0000","last_change":"2024-04-25T09:53:07.793203+0000","last_active":"2024-04-25T09:53:12.810497+0000","last_peered":"2024-04-25T09:53:12.810497+0000","last_clean":"2024-04-25T09:53:12.810497+0000","last_became_active":"2024-04-25T09:53:07.792940+0000","last_became_peered":"2024-04-25T09:53:07.792940+0000","last_unstale":"2024-04-25T09:53:12.810497+0000","last_undegraded":"2024-04-25T09:53:12.810497+0000","last_fullsized":"2024-04-25T09:53:12.810497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.221246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809630+0000","last_change":"2024-04-25T09:53:07.789420+0000","last_active":"2024-04-25T09:53:12.809630+0000","last_peered":"2024-04-25T09:53:12.809630+0000","last_clean":"2024-04-25T09:53:12.809630+0000","last_became_active":"2024-04-25T09:53:07.789112+0000","last_became_peered":"2024-04-25T09:53:07.789112+0000","last_unstale":"2024-04-25T09:53:12.809630+0000","last_undegraded":"2024-04-25T09:53:12.809630+0000","last_fullsized":"2024-04-25T09:53:12.809630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.768421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327332+0000","last_change":"2024-04-25T09:53:07.794475+0000","last_active":"2024-04-25T09:53:13.327332+0000","last_peered":"2024-04-25T09:53:13.327332+0000","last_clean":"2024-04-25T09:53:13.327332+0000","last_became_active":"2024-04-25T09:53:07.794225+0000","last_became_peered":"2024-04-25T09:53:07.794225+0000","last_unstale":"2024-04-25T09:53:13.327332+0000","last_undegraded":"2024-04-25T09:53:13.327332+0000","last_fullsized":"2024-04-25T09:53:13.327332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:47.687838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325668+0000","last_change":"2024-04-25T09:53:07.798926+0000","last_active":"2024-04-25T09:53:13.325668+0000","last_peered":"2024-04-25T09:53:13.325668+0000","last_clean":"2024-04-25T09:53:13.325668+0000","last_became_active":"2024-04-25T09:53:07.798446+0000","last_became_peered":"2024-04-25T09:53:07.798446+0000","last_unstale":"2024-04-25T09:53:13.325668+0000","last_undegraded":"2024-04-25T09:53:13.325668+0000","last_fullsized":"2024-04-25T09:53:13.325668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325115+0000","last_change":"2024-04-25T09:53:07.784610+0000","last_active":"2024-04-25T09:53:13.325115+0000","last_peered":"2024-04-25T09:53:13.325115+0000","last_clean":"2024-04-25T09:53:13.325115+0000","last_became_active":"2024-04-25T09:53:07.784323+0000","last_became_peered":"2024-04-25T09:53:07.784323+0000","last_unstale":"2024-04-25T09:53:13.325115+0000","last_undegraded":"2024-04-25T09:53:13.325115+0000","last_fullsized":"2024-04-25T09:53:13.325115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326504+0000","last_change":"2024-04-25T09:53:07.781472+0000","last_active":"2024-04-25T09:53:13.326504+0000","last_peered":"2024-04-25T09:53:13.326504+0000","last_clean":"2024-04-25T09:53:13.326504+0000","last_became_active":"2024-04-25T09:53:07.781156+0000","last_became_peered":"2024-04-25T09:53:07.781156+0000","last_unstale":"2024-04-25T09:53:13.326504+0000","last_undegraded":"2024-04-25T09:53:13.326504+0000","last_fullsized":"2024-04-25T09:53:13.326504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:49.469537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325209+0000","last_change":"2024-04-25T09:53:07.790342+0000","last_active":"2024-04-25T09:53:13.325209+0000","last_peered":"2024-04-25T09:53:13.325209+0000","last_clean":"2024-04-25T09:53:13.325209+0000","last_became_active":"2024-04-25T09:53:07.790003+0000","last_became_peered":"2024-04-25T09:53:07.790003+0000","last_unstale":"2024-04-25T09:53:13.325209+0000","last_undegraded":"2024-04-25T09:53:13.325209+0000","last_fullsized":"2024-04-25T09:53:13.325209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326892+0000","last_change":"2024-04-25T09:53:07.791198+0000","last_active":"2024-04-25T09:53:13.326892+0000","last_peered":"2024-04-25T09:53:13.326892+0000","last_clean":"2024-04-25T09:53:13.326892+0000","last_became_active":"2024-04-25T09:53:07.790896+0000","last_became_peered":"2024-04-25T09:53:07.790896+0000","last_unstale":"2024-04-25T09:53:13.326892+0000","last_undegraded":"2024-04-25T09:53:13.326892+0000","last_fullsized":"2024-04-25T09:53:13.326892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:45.411393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810894+0000","last_change":"2024-04-25T09:53:07.790314+0000","last_active":"2024-04-25T09:53:12.810894+0000","last_peered":"2024-04-25T09:53:12.810894+0000","last_clean":"2024-04-25T09:53:12.810894+0000","last_became_active":"2024-04-25T09:53:07.789999+0000","last_became_peered":"2024-04-25T09:53:07.789999+0000","last_unstale":"2024-04-25T09:53:12.810894+0000","last_undegraded":"2024-04-25T09:53:12.810894+0000","last_fullsized":"2024-04-25T09:53:12.810894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810741+0000","last_change":"2024-04-25T09:53:07.793396+0000","last_active":"2024-04-25T09:53:12.810741+0000","last_peered":"2024-04-25T09:53:12.810741+0000","last_clean":"2024-04-25T09:53:12.810741+0000","last_became_active":"2024-04-25T09:53:07.792132+0000","last_became_peered":"2024-04-25T09:53:07.792132+0000","last_unstale":"2024-04-25T09:53:12.810741+0000","last_undegraded":"2024-04-25T09:53:12.810741+0000","last_fullsized":"2024-04-25T09:53:12.810741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811509+0000","last_change":"2024-04-25T09:53:07.800101+0000","last_active":"2024-04-25T09:53:12.811509+0000","last_peered":"2024-04-25T09:53:12.811509+0000","last_clean":"2024-04-25T09:53:12.811509+0000","last_became_active":"2024-04-25T09:53:07.799794+0000","last_became_peered":"2024-04-25T09:53:07.799794+0000","last_unstale":"2024-04-25T09:53:12.811509+0000","last_undegraded":"2024-04-25T09:53:12.811509+0000","last_fullsized":"2024-04-25T09:53:12.811509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:12.845651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325964+0000","last_change":"2024-04-25T09:53:07.793404+0000","last_active":"2024-04-25T09:53:13.325964+0000","last_peered":"2024-04-25T09:53:13.325964+0000","last_clean":"2024-04-25T09:53:13.325964+0000","last_became_active":"2024-04-25T09:53:07.793140+0000","last_became_peered":"2024-04-25T09:53:07.793140+0000","last_unstale":"2024-04-25T09:53:13.325964+0000","last_undegraded":"2024-04-25T09:53:13.325964+0000","last_fullsized":"2024-04-25T09:53:13.325964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809861+0000","last_change":"2024-04-25T09:53:07.797288+0000","last_active":"2024-04-25T09:53:12.809861+0000","last_peered":"2024-04-25T09:53:12.809861+0000","last_clean":"2024-04-25T09:53:12.809861+0000","last_became_active":"2024-04-25T09:53:07.797017+0000","last_became_peered":"2024-04-25T09:53:07.797017+0000","last_unstale":"2024-04-25T09:53:12.809861+0000","last_undegraded":"2024-04-25T09:53:12.809861+0000","last_fullsized":"2024-04-25T09:53:12.809861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326593+0000","last_change":"2024-04-25T09:53:07.800134+0000","last_active":"2024-04-25T09:53:13.326593+0000","last_peered":"2024-04-25T09:53:13.326593+0000","last_clean":"2024-04-25T09:53:13.326593+0000","last_became_active":"2024-04-25T09:53:07.799817+0000","last_became_peered":"2024-04-25T09:53:07.799817+0000","last_unstale":"2024-04-25T09:53:13.326593+0000","last_undegraded":"2024-04-25T09:53:13.326593+0000","last_fullsized":"2024-04-25T09:53:13.326593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810285+0000","last_change":"2024-04-25T09:53:07.792436+0000","last_active":"2024-04-25T09:53:12.810285+0000","last_peered":"2024-04-25T09:53:12.810285+0000","last_clean":"2024-04-25T09:53:12.810285+0000","last_became_active":"2024-04-25T09:53:07.792150+0000","last_became_peered":"2024-04-25T09:53:07.792150+0000","last_unstale":"2024-04-25T09:53:12.810285+0000","last_undegraded":"2024-04-25T09:53:12.810285+0000","last_fullsized":"2024-04-25T09:53:12.810285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326560+0000","last_change":"2024-04-25T09:53:07.788302+0000","last_active":"2024-04-25T09:53:13.326560+0000","last_peered":"2024-04-25T09:53:13.326560+0000","last_clean":"2024-04-25T09:53:13.326560+0000","last_became_active":"2024-04-25T09:53:07.788033+0000","last_became_peered":"2024-04-25T09:53:07.788033+0000","last_unstale":"2024-04-25T09:53:13.326560+0000","last_undegraded":"2024-04-25T09:53:13.326560+0000","last_fullsized":"2024-04-25T09:53:13.326560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325682+0000","last_change":"2024-04-25T09:53:07.776111+0000","last_active":"2024-04-25T09:53:13.325682+0000","last_peered":"2024-04-25T09:53:13.325682+0000","last_clean":"2024-04-25T09:53:13.325682+0000","last_became_active":"2024-04-25T09:53:07.775779+0000","last_became_peered":"2024-04-25T09:53:07.775779+0000","last_unstale":"2024-04-25T09:53:13.325682+0000","last_undegraded":"2024-04-25T09:53:13.325682+0000","last_fullsized":"2024-04-25T09:53:13.325682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325773+0000","last_change":"2024-04-25T09:53:07.782293+0000","last_active":"2024-04-25T09:53:13.325773+0000","last_peered":"2024-04-25T09:53:13.325773+0000","last_clean":"2024-04-25T09:53:13.325773+0000","last_became_active":"2024-04-25T09:53:07.781957+0000","last_became_peered":"2024-04-25T09:53:07.781957+0000","last_unstale":"2024-04-25T09:53:13.325773+0000","last_undegraded":"2024-04-25T09:53:13.325773+0000","last_fullsized":"2024-04-25T09:53:13.325773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811414+0000","last_change":"2024-04-25T09:53:07.791822+0000","last_active":"2024-04-25T09:53:12.811414+0000","last_peered":"2024-04-25T09:53:12.811414+0000","last_clean":"2024-04-25T09:53:12.811414+0000","last_became_active":"2024-04-25T09:53:07.791566+0000","last_became_peered":"2024-04-25T09:53:07.791566+0000","last_unstale":"2024-04-25T09:53:12.811414+0000","last_undegraded":"2024-04-25T09:53:12.811414+0000","last_fullsized":"2024-04-25T09:53:12.811414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:35.714783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:53:24.983 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:53:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.117+0000 7fb0a754f640 1 Processor -- start 2024-04-25T09:53:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.117+0000 7fb0a754f640 1 -- start start 2024-04-25T09:53:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb0a754f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b2cb0 0x7fb0a00d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb0a754f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb0a00d1020 0x7fb0a00d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb0a754f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a00b97a0 con 0x7fb0a015dbe0 2024-04-25T09:53:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb0a754f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a015dfc0 con 0x7fb0a00d1020 2024-04-25T09:53:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb0a754f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00d1a00 con 0x7fb0a00b2cb0 2024-04-25T09:53:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb0a52eb640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb0a015dbe0 0x7fb0a00d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46860/0 (socket says 172.21.15.19:46860) 2024-04-25T09:53:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb0a52eb640 1 -- 172.21.15.19:0/3432819785 learned_addr learned my addr 172.21.15.19:0/3432819785 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb0a4aea640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b2cb0 0x7fb0a00d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb0a5aec640 1 --2- 172.21.15.19:0/3432819785 >> v2:172.21.15.112:3300/0 conn(0x7fb0a00d1020 0x7fb0a00d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815867850 0 0) 0x7fb0a00b97a0 con 0x7fb0a015dbe0 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb08c003520 con 0x7fb0a015dbe0 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2225969368 0 0) 0x7fb08c003520 con 0x7fb0a015dbe0 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b2cb0 msgr2=0x7fb0a00d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b2cb0 0x7fb0a00d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 >> v2:172.21.15.112:3300/0 conn(0x7fb0a00d1020 msgr2=0x7fb0a00d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 --2- 172.21.15.19:0/3432819785 >> v2:172.21.15.112:3300/0 conn(0x7fb0a00d1020 0x7fb0a00d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a00d1c50 con 0x7fb0a015dbe0 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fb09c001670 con 0x7fb0a015dbe0 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 >> v1:172.21.15.19:6789/0 conn(0x7fb0a015dbe0 legacy=0x7fb0a00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c0039b0 0x7fb08c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb08c004800 0x7fb08c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b2cb0 0x7fb08c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00d1a00 con 0x7fb08c0039b0 2024-04-25T09:53:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c00b9c0 con 0x7fb08c004800 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.118+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c00bb40 con 0x7fb0a00b2cb0 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.119+0000 7fb0a52eb640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c0039b0 0x7fb08c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.119+0000 7fb0a4aea640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb08c004800 0x7fb08c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.119+0000 7fb0a5aec640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b2cb0 0x7fb08c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.119+0000 7fb0a52eb640 1 -- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b2cb0 msgr2=0x7fb08c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.119+0000 7fb0a52eb640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b2cb0 0x7fb08c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.119+0000 7fb0a52eb640 1 -- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb08c004800 msgr2=0x7fb08c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.119+0000 7fb0a52eb640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb08c004800 0x7fb08c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.119+0000 7fb0a52eb640 1 -- 172.21.15.19:0/3432819785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c003520 con 0x7fb08c0039b0 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.119+0000 7fb0a4aea640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb08c004800 0x7fb08c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.120+0000 7fb0a52eb640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c0039b0 0x7fb08c003dd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb09c007fd0 tx=0x7fb08c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4582ca4a05db0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.120+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c007d50 con 0x7fb08c0039b0 2024-04-25T09:53:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.120+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c005ad0 con 0x7fb08c0039b0 2024-04-25T09:53:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.120+0000 7fb097fff640 1 -- 172.21.15.19:0/3432819785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c007070 con 0x7fb08c0039b0 2024-04-25T09:53:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.120+0000 7fb0a754f640 1 -- 172.21.15.19:0/3432819785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c0039b0 msgr2=0x7fb08c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.120+0000 7fb0a754f640 1 --2- 172.21.15.19:0/3432819785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb08c0039b0 0x7fb08c003dd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb09c007fd0 tx=0x7fb08c005830 comp rx=0 tx=0).stop 2024-04-25T09:53:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.120+0000 7fb0a5aec640 1 -- 172.21.15.19:0/3432819785 reap_dead start 2024-04-25T09:53:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.120+0000 7fb0a754f640 1 -- 172.21.15.19:0/3432819785 shutdown_connections 2024-04-25T09:53:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.120+0000 7fb0a754f640 1 -- 172.21.15.19:0/3432819785 >> 172.21.15.19:0/3432819785 conn(0x7fb0a0061a20 msgr2=0x7fb0a0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.120+0000 7fb0a754f640 1 -- 172.21.15.19:0/3432819785 shutdown_connections 2024-04-25T09:53:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.120+0000 7fb0a754f640 1 -- 172.21.15.19:0/3432819785 wait complete. 2024-04-25T09:53:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.121+0000 7fb0a754f640 1 Processor -- start 2024-04-25T09:53:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a754f640 1 -- start start 2024-04-25T09:53:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a754f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb08c0039b0 0x7fb0a0157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a754f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a015f650 0x7fb0a014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a754f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a014c070 0x7fb0a00cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a754f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00b92a0 con 0x7fb0a014c070 2024-04-25T09:53:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a754f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00d2210 con 0x7fb08c0039b0 2024-04-25T09:53:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a754f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00b9420 con 0x7fb0a015f650 2024-04-25T09:53:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a5aec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a014c070 0x7fb0a00cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a5aec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a014c070 0x7fb0a00cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35680/0 (socket says 172.21.15.19:35680) 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a5aec640 1 -- 172.21.15.19:0/796182027 learned_addr learned my addr 172.21.15.19:0/796182027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a4aea640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a015f650 0x7fb0a014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.122+0000 7fb0a52eb640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb08c0039b0 0x7fb0a0157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb0a5aec640 1 -- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a015f650 msgr2=0x7fb0a014bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb0a5aec640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a015f650 0x7fb0a014bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb0a5aec640 1 -- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb08c0039b0 msgr2=0x7fb0a0157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb0a5aec640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb08c0039b0 0x7fb0a0157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb0a5aec640 1 -- 172.21.15.19:0/796182027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a0160d60 con 0x7fb0a014c070 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb0a52eb640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb08c0039b0 0x7fb0a0157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb0a5aec640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a014c070 0x7fb0a00cb770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00cfe70 tx=0x7fb09800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb095ffb640 1 -- 172.21.15.19:0/796182027 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090009690 con 0x7fb0a014c070 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb095ffb640 1 -- 172.21.15.19:0/796182027 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09801d070 con 0x7fb0a014c070 2024-04-25T09:53:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb0a754f640 1 -- 172.21.15.19:0/796182027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a015eb70 con 0x7fb0a014c070 2024-04-25T09:53:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb095ffb640 1 -- 172.21.15.19:0/796182027 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098015430 con 0x7fb0a014c070 2024-04-25T09:53:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.123+0000 7fb0a754f640 1 -- 172.21.15.19:0/796182027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a0155240 con 0x7fb0a014c070 2024-04-25T09:53:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.124+0000 7fb0a754f640 1 -- 172.21.15.19:0/796182027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a0073990 con 0x7fb0a014c070 2024-04-25T09:53:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.125+0000 7fb095ffb640 1 -- 172.21.15.19:0/796182027 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0980155d0 con 0x7fb0a014c070 2024-04-25T09:53:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.125+0000 7fb095ffb640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 0x7fb08406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.125+0000 7fb095ffb640 1 -- 172.21.15.19:0/796182027 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb09808ac10 con 0x7fb0a014c070 2024-04-25T09:53:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.125+0000 7fb0a52eb640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 0x7fb08406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.126+0000 7fb0a52eb640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 0x7fb08406d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb09c033000 tx=0x7fb09c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.129+0000 7fb095ffb640 1 -- 172.21.15.19:0/796182027 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb098010040 con 0x7fb0a014c070 2024-04-25T09:53:25.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.307+0000 7fb0a754f640 1 -- 172.21.15.19:0/796182027 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0a00d31a0 con 0x7fb08406b4d0 2024-04-25T09:53:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.315+0000 7fb095ffb640 1 -- 172.21.15.19:0/796182027 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7fb0a00d31a0 con 0x7fb08406b4d0 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 -- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 msgr2=0x7fb08406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 0x7fb08406d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb09c033000 tx=0x7fb09c009030 comp rx=0 tx=0).stop 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 -- 172.21.15.19:0/796182027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a014c070 msgr2=0x7fb0a00cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a014c070 0x7fb0a00cb770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00cfe70 tx=0x7fb09800f1b0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 -- 172.21.15.19:0/796182027 shutdown_connections 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 0x7fb08406d990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a014c070 0x7fb0a00cb770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a015f650 0x7fb0a014bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 --2- 172.21.15.19:0/796182027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb08c0039b0 0x7fb0a0157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 -- 172.21.15.19:0/796182027 >> 172.21.15.19:0/796182027 conn(0x7fb0a0061a20 msgr2=0x7fb0a00b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 -- 172.21.15.19:0/796182027 shutdown_connections 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.319+0000 7fb0a754f640 1 -- 172.21.15.19:0/796182027 wait complete. 2024-04-25T09:53:25.331 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T09:53:24.914622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219368,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755352,"statfs":{"total":773094113280,"available":772869480448,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003374"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326249+0000","last_change":"2024-04-25T09:53:06.778392+0000","last_active":"2024-04-25T09:53:13.326249+0000","last_peered":"2024-04-25T09:53:13.326249+0000","last_clean":"2024-04-25T09:53:13.326249+0000","last_became_active":"2024-04-25T09:53:06.777894+0000","last_became_peered":"2024-04-25T09:53:06.777894+0000","last_unstale":"2024-04-25T09:53:13.326249+0000","last_undegraded":"2024-04-25T09:53:13.326249+0000","last_fullsized":"2024-04-25T09:53:13.326249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.898659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326801+0000","last_change":"2024-04-25T09:53:07.792733+0000","last_active":"2024-04-25T09:53:13.326801+0000","last_peered":"2024-04-25T09:53:13.326801+0000","last_clean":"2024-04-25T09:53:13.326801+0000","last_became_active":"2024-04-25T09:53:07.792294+0000","last_became_peered":"2024-04-25T09:53:07.792294+0000","last_unstale":"2024-04-25T09:53:13.326801+0000","last_undegraded":"2024-04-25T09:53:13.326801+0000","last_fullsized":"2024-04-25T09:53:13.326801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325683+0000","last_change":"2024-04-25T09:53:11.822234+0000","last_active":"2024-04-25T09:53:13.325683+0000","last_peered":"2024-04-25T09:53:13.325683+0000","last_clean":"2024-04-25T09:53:13.325683+0000","last_became_active":"2024-04-25T09:53:11.821231+0000","last_became_peered":"2024-04-25T09:53:11.821231+0000","last_unstale":"2024-04-25T09:53:13.325683+0000","last_undegraded":"2024-04-25T09:53:13.325683+0000","last_fullsized":"2024-04-25T09:53:13.325683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:19.880548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327066+0000","last_change":"2024-04-25T09:53:06.779890+0000","last_active":"2024-04-25T09:53:13.327066+0000","last_peered":"2024-04-25T09:53:13.327066+0000","last_clean":"2024-04-25T09:53:13.327066+0000","last_became_active":"2024-04-25T09:53:06.779482+0000","last_became_peered":"2024-04-25T09:53:06.779482+0000","last_unstale":"2024-04-25T09:53:13.327066+0000","last_undegraded":"2024-04-25T09:53:13.327066+0000","last_fullsized":"2024-04-25T09:53:13.327066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325889+0000","last_change":"2024-04-25T09:53:07.789471+0000","last_active":"2024-04-25T09:53:13.325889+0000","last_peered":"2024-04-25T09:53:13.325889+0000","last_clean":"2024-04-25T09:53:13.325889+0000","last_became_active":"2024-04-25T09:53:07.789190+0000","last_became_peered":"2024-04-25T09:53:07.789190+0000","last_unstale":"2024-04-25T09:53:13.325889+0000","last_undegraded":"2024-04-25T09:53:13.325889+0000","last_fullsized":"2024-04-25T09:53:13.325889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326006+0000","last_change":"2024-04-25T09:53:11.841059+0000","last_active":"2024-04-25T09:53:13.326006+0000","last_peered":"2024-04-25T09:53:13.326006+0000","last_clean":"2024-04-25T09:53:13.326006+0000","last_became_active":"2024-04-25T09:53:11.840640+0000","last_became_peered":"2024-04-25T09:53:11.840640+0000","last_unstale":"2024-04-25T09:53:13.326006+0000","last_undegraded":"2024-04-25T09:53:13.326006+0000","last_fullsized":"2024-04-25T09:53:13.326006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:54.172755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810382+0000","last_change":"2024-04-25T09:53:06.777792+0000","last_active":"2024-04-25T09:53:12.810382+0000","last_peered":"2024-04-25T09:53:12.810382+0000","last_clean":"2024-04-25T09:53:12.810382+0000","last_became_active":"2024-04-25T09:53:06.777459+0000","last_became_peered":"2024-04-25T09:53:06.777459+0000","last_unstale":"2024-04-25T09:53:12.810382+0000","last_undegraded":"2024-04-25T09:53:12.810382+0000","last_fullsized":"2024-04-25T09:53:12.810382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:44.406768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326553+0000","last_change":"2024-04-25T09:53:07.798127+0000","last_active":"2024-04-25T09:53:13.326553+0000","last_peered":"2024-04-25T09:53:13.326553+0000","last_clean":"2024-04-25T09:53:13.326553+0000","last_became_active":"2024-04-25T09:53:07.797863+0000","last_became_peered":"2024-04-25T09:53:07.797863+0000","last_unstale":"2024-04-25T09:53:13.326553+0000","last_undegraded":"2024-04-25T09:53:13.326553+0000","last_fullsized":"2024-04-25T09:53:13.326553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326109+0000","last_change":"2024-04-25T09:53:11.841255+0000","last_active":"2024-04-25T09:53:13.326109+0000","last_peered":"2024-04-25T09:53:13.326109+0000","last_clean":"2024-04-25T09:53:13.326109+0000","last_became_active":"2024-04-25T09:53:11.840795+0000","last_became_peered":"2024-04-25T09:53:11.840795+0000","last_unstale":"2024-04-25T09:53:13.326109+0000","last_undegraded":"2024-04-25T09:53:13.326109+0000","last_fullsized":"2024-04-25T09:53:13.326109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:28.118453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810018+0000","last_change":"2024-04-25T09:53:06.784013+0000","last_active":"2024-04-25T09:53:12.810018+0000","last_peered":"2024-04-25T09:53:12.810018+0000","last_clean":"2024-04-25T09:53:12.810018+0000","last_became_active":"2024-04-25T09:53:06.783726+0000","last_became_peered":"2024-04-25T09:53:06.783726+0000","last_unstale":"2024-04-25T09:53:12.810018+0000","last_undegraded":"2024-04-25T09:53:12.810018+0000","last_fullsized":"2024-04-25T09:53:12.810018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327442+0000","last_change":"2024-04-25T09:53:07.796042+0000","last_active":"2024-04-25T09:53:13.327442+0000","last_peered":"2024-04-25T09:53:13.327442+0000","last_clean":"2024-04-25T09:53:13.327442+0000","last_became_active":"2024-04-25T09:53:07.795676+0000","last_became_peered":"2024-04-25T09:53:07.795676+0000","last_unstale":"2024-04-25T09:53:13.327442+0000","last_undegraded":"2024-04-25T09:53:13.327442+0000","last_fullsized":"2024-04-25T09:53:13.327442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:12.845651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326221+0000","last_change":"2024-04-25T09:53:11.827699+0000","last_active":"2024-04-25T09:53:13.326221+0000","last_peered":"2024-04-25T09:53:13.326221+0000","last_clean":"2024-04-25T09:53:13.326221+0000","last_became_active":"2024-04-25T09:53:11.827325+0000","last_became_peered":"2024-04-25T09:53:11.827325+0000","last_unstale":"2024-04-25T09:53:13.326221+0000","last_undegraded":"2024-04-25T09:53:13.326221+0000","last_fullsized":"2024-04-25T09:53:13.326221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.837684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810063+0000","last_change":"2024-04-25T09:53:07.800199+0000","last_active":"2024-04-25T09:53:12.810063+0000","last_peered":"2024-04-25T09:53:12.810063+0000","last_clean":"2024-04-25T09:53:12.810063+0000","last_became_active":"2024-04-25T09:53:07.799912+0000","last_became_peered":"2024-04-25T09:53:07.799912+0000","last_unstale":"2024-04-25T09:53:12.810063+0000","last_undegraded":"2024-04-25T09:53:12.810063+0000","last_fullsized":"2024-04-25T09:53:12.810063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810410+0000","last_change":"2024-04-25T09:53:06.783380+0000","last_active":"2024-04-25T09:53:12.810410+0000","last_peered":"2024-04-25T09:53:12.810410+0000","last_clean":"2024-04-25T09:53:12.810410+0000","last_became_active":"2024-04-25T09:53:06.783049+0000","last_became_peered":"2024-04-25T09:53:06.783049+0000","last_unstale":"2024-04-25T09:53:12.810410+0000","last_undegraded":"2024-04-25T09:53:12.810410+0000","last_fullsized":"2024-04-25T09:53:12.810410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327316+0000","last_change":"2024-04-25T09:53:11.839467+0000","last_active":"2024-04-25T09:53:13.327316+0000","last_peered":"2024-04-25T09:53:13.327316+0000","last_clean":"2024-04-25T09:53:13.327316+0000","last_became_active":"2024-04-25T09:53:11.839079+0000","last_became_peered":"2024-04-25T09:53:11.839079+0000","last_unstale":"2024-04-25T09:53:13.327316+0000","last_undegraded":"2024-04-25T09:53:13.327316+0000","last_fullsized":"2024-04-25T09:53:13.327316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325291+0000","last_change":"2024-04-25T09:53:07.791763+0000","last_active":"2024-04-25T09:53:13.325291+0000","last_peered":"2024-04-25T09:53:13.325291+0000","last_clean":"2024-04-25T09:53:13.325291+0000","last_became_active":"2024-04-25T09:53:07.790924+0000","last_became_peered":"2024-04-25T09:53:07.790924+0000","last_unstale":"2024-04-25T09:53:13.325291+0000","last_undegraded":"2024-04-25T09:53:13.325291+0000","last_fullsized":"2024-04-25T09:53:13.325291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326553+0000","last_change":"2024-04-25T09:53:06.779703+0000","last_active":"2024-04-25T09:53:13.326553+0000","last_peered":"2024-04-25T09:53:13.326553+0000","last_clean":"2024-04-25T09:53:13.326553+0000","last_became_active":"2024-04-25T09:53:06.779413+0000","last_became_peered":"2024-04-25T09:53:06.779413+0000","last_unstale":"2024-04-25T09:53:13.326553+0000","last_undegraded":"2024-04-25T09:53:13.326553+0000","last_fullsized":"2024-04-25T09:53:13.326553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326368+0000","last_change":"2024-04-25T09:53:11.840454+0000","last_active":"2024-04-25T09:53:13.326368+0000","last_peered":"2024-04-25T09:53:13.326368+0000","last_clean":"2024-04-25T09:53:13.326368+0000","last_became_active":"2024-04-25T09:53:11.840034+0000","last_became_peered":"2024-04-25T09:53:11.840034+0000","last_unstale":"2024-04-25T09:53:13.326368+0000","last_undegraded":"2024-04-25T09:53:13.326368+0000","last_fullsized":"2024-04-25T09:53:13.326368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.438696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326270+0000","last_change":"2024-04-25T09:53:07.786389+0000","last_active":"2024-04-25T09:53:13.326270+0000","last_peered":"2024-04-25T09:53:13.326270+0000","last_clean":"2024-04-25T09:53:13.326270+0000","last_became_active":"2024-04-25T09:53:07.786116+0000","last_became_peered":"2024-04-25T09:53:07.786116+0000","last_unstale":"2024-04-25T09:53:13.326270+0000","last_undegraded":"2024-04-25T09:53:13.326270+0000","last_fullsized":"2024-04-25T09:53:13.326270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325212+0000","last_change":"2024-04-25T09:53:06.780897+0000","last_active":"2024-04-25T09:53:13.325212+0000","last_peered":"2024-04-25T09:53:13.325212+0000","last_clean":"2024-04-25T09:53:13.325212+0000","last_became_active":"2024-04-25T09:53:06.780616+0000","last_became_peered":"2024-04-25T09:53:06.780616+0000","last_unstale":"2024-04-25T09:53:13.325212+0000","last_undegraded":"2024-04-25T09:53:13.325212+0000","last_fullsized":"2024-04-25T09:53:13.325212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:24.763796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326272+0000","last_change":"2024-04-25T09:53:11.838039+0000","last_active":"2024-04-25T09:53:13.326272+0000","last_peered":"2024-04-25T09:53:13.326272+0000","last_clean":"2024-04-25T09:53:13.326272+0000","last_became_active":"2024-04-25T09:53:11.837388+0000","last_became_peered":"2024-04-25T09:53:11.837388+0000","last_unstale":"2024-04-25T09:53:13.326272+0000","last_undegraded":"2024-04-25T09:53:13.326272+0000","last_fullsized":"2024-04-25T09:53:13.326272+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:39.753991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326308+0000","last_change":"2024-04-25T09:53:07.784394+0000","last_active":"2024-04-25T09:53:13.326308+0000","last_peered":"2024-04-25T09:53:13.326308+0000","last_clean":"2024-04-25T09:53:13.326308+0000","last_became_active":"2024-04-25T09:53:07.784060+0000","last_became_peered":"2024-04-25T09:53:07.784060+0000","last_unstale":"2024-04-25T09:53:13.326308+0000","last_undegraded":"2024-04-25T09:53:13.326308+0000","last_fullsized":"2024-04-25T09:53:13.326308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:36.593849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809684+0000","last_change":"2024-04-25T09:53:06.779579+0000","last_active":"2024-04-25T09:53:12.809684+0000","last_peered":"2024-04-25T09:53:12.809684+0000","last_clean":"2024-04-25T09:53:12.809684+0000","last_became_active":"2024-04-25T09:53:06.779296+0000","last_became_peered":"2024-04-25T09:53:06.779296+0000","last_unstale":"2024-04-25T09:53:12.809684+0000","last_undegraded":"2024-04-25T09:53:12.809684+0000","last_fullsized":"2024-04-25T09:53:12.809684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327226+0000","last_change":"2024-04-25T09:53:11.839381+0000","last_active":"2024-04-25T09:53:13.327226+0000","last_peered":"2024-04-25T09:53:13.327226+0000","last_clean":"2024-04-25T09:53:13.327226+0000","last_became_active":"2024-04-25T09:53:11.838941+0000","last_became_peered":"2024-04-25T09:53:11.838941+0000","last_unstale":"2024-04-25T09:53:13.327226+0000","last_undegraded":"2024-04-25T09:53:13.327226+0000","last_fullsized":"2024-04-25T09:53:13.327226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:59.415156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326930+0000","last_change":"2024-04-25T09:53:07.785627+0000","last_active":"2024-04-25T09:53:13.326930+0000","last_peered":"2024-04-25T09:53:13.326930+0000","last_clean":"2024-04-25T09:53:13.326930+0000","last_became_active":"2024-04-25T09:53:07.782773+0000","last_became_peered":"2024-04-25T09:53:07.782773+0000","last_unstale":"2024-04-25T09:53:13.326930+0000","last_undegraded":"2024-04-25T09:53:13.326930+0000","last_fullsized":"2024-04-25T09:53:13.326930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811412+0000","last_change":"2024-04-25T09:53:06.778161+0000","last_active":"2024-04-25T09:53:12.811412+0000","last_peered":"2024-04-25T09:53:12.811412+0000","last_clean":"2024-04-25T09:53:12.811412+0000","last_became_active":"2024-04-25T09:53:06.777837+0000","last_became_peered":"2024-04-25T09:53:06.777837+0000","last_unstale":"2024-04-25T09:53:12.811412+0000","last_undegraded":"2024-04-25T09:53:12.811412+0000","last_fullsized":"2024-04-25T09:53:12.811412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325940+0000","last_change":"2024-04-25T09:53:11.823677+0000","last_active":"2024-04-25T09:53:13.325940+0000","last_peered":"2024-04-25T09:53:13.325940+0000","last_clean":"2024-04-25T09:53:13.325940+0000","last_became_active":"2024-04-25T09:53:11.823278+0000","last_became_peered":"2024-04-25T09:53:11.823278+0000","last_unstale":"2024-04-25T09:53:13.325940+0000","last_undegraded":"2024-04-25T09:53:13.325940+0000","last_fullsized":"2024-04-25T09:53:13.325940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:45.277051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327064+0000","last_change":"2024-04-25T09:53:07.792276+0000","last_active":"2024-04-25T09:53:13.327064+0000","last_peered":"2024-04-25T09:53:13.327064+0000","last_clean":"2024-04-25T09:53:13.327064+0000","last_became_active":"2024-04-25T09:53:07.790961+0000","last_became_peered":"2024-04-25T09:53:07.790961+0000","last_unstale":"2024-04-25T09:53:13.327064+0000","last_undegraded":"2024-04-25T09:53:13.327064+0000","last_fullsized":"2024-04-25T09:53:13.327064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325875+0000","last_change":"2024-04-25T09:53:06.779465+0000","last_active":"2024-04-25T09:53:13.325875+0000","last_peered":"2024-04-25T09:53:13.325875+0000","last_clean":"2024-04-25T09:53:13.325875+0000","last_became_active":"2024-04-25T09:53:06.779122+0000","last_became_peered":"2024-04-25T09:53:06.779122+0000","last_unstale":"2024-04-25T09:53:13.325875+0000","last_undegraded":"2024-04-25T09:53:13.325875+0000","last_fullsized":"2024-04-25T09:53:13.325875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325863+0000","last_change":"2024-04-25T09:53:11.832958+0000","last_active":"2024-04-25T09:53:13.325863+0000","last_peered":"2024-04-25T09:53:13.325863+0000","last_clean":"2024-04-25T09:53:13.325863+0000","last_became_active":"2024-04-25T09:53:11.832522+0000","last_became_peered":"2024-04-25T09:53:11.832522+0000","last_unstale":"2024-04-25T09:53:13.325863+0000","last_undegraded":"2024-04-25T09:53:13.325863+0000","last_fullsized":"2024-04-25T09:53:13.325863+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:58.718292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325598+0000","last_change":"2024-04-25T09:53:07.779236+0000","last_active":"2024-04-25T09:53:13.325598+0000","last_peered":"2024-04-25T09:53:13.325598+0000","last_clean":"2024-04-25T09:53:13.325598+0000","last_became_active":"2024-04-25T09:53:07.778938+0000","last_became_peered":"2024-04-25T09:53:07.778938+0000","last_unstale":"2024-04-25T09:53:13.325598+0000","last_undegraded":"2024-04-25T09:53:13.325598+0000","last_fullsized":"2024-04-25T09:53:13.325598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810604+0000","last_change":"2024-04-25T09:53:06.780019+0000","last_active":"2024-04-25T09:53:12.810604+0000","last_peered":"2024-04-25T09:53:12.810604+0000","last_clean":"2024-04-25T09:53:12.810604+0000","last_became_active":"2024-04-25T09:53:06.779429+0000","last_became_peered":"2024-04-25T09:53:06.779429+0000","last_unstale":"2024-04-25T09:53:12.810604+0000","last_undegraded":"2024-04-25T09:53:12.810604+0000","last_fullsized":"2024-04-25T09:53:12.810604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:24.763796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325466+0000","last_change":"2024-04-25T09:53:11.835047+0000","last_active":"2024-04-25T09:53:13.325466+0000","last_peered":"2024-04-25T09:53:13.325466+0000","last_clean":"2024-04-25T09:53:13.325466+0000","last_became_active":"2024-04-25T09:53:11.833932+0000","last_became_peered":"2024-04-25T09:53:11.833932+0000","last_unstale":"2024-04-25T09:53:13.325466+0000","last_undegraded":"2024-04-25T09:53:13.325466+0000","last_fullsized":"2024-04-25T09:53:13.325466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:39.753991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327090+0000","last_change":"2024-04-25T09:53:07.786462+0000","last_active":"2024-04-25T09:53:13.327090+0000","last_peered":"2024-04-25T09:53:13.327090+0000","last_clean":"2024-04-25T09:53:13.327090+0000","last_became_active":"2024-04-25T09:53:07.785793+0000","last_became_peered":"2024-04-25T09:53:07.785793+0000","last_unstale":"2024-04-25T09:53:13.327090+0000","last_undegraded":"2024-04-25T09:53:13.327090+0000","last_fullsized":"2024-04-25T09:53:13.327090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.221246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325690+0000","last_change":"2024-04-25T09:53:06.780745+0000","last_active":"2024-04-25T09:53:13.325690+0000","last_peered":"2024-04-25T09:53:13.325690+0000","last_clean":"2024-04-25T09:53:13.325690+0000","last_became_active":"2024-04-25T09:53:06.780468+0000","last_became_peered":"2024-04-25T09:53:06.780468+0000","last_unstale":"2024-04-25T09:53:13.325690+0000","last_undegraded":"2024-04-25T09:53:13.325690+0000","last_fullsized":"2024-04-25T09:53:13.325690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326300+0000","last_change":"2024-04-25T09:53:11.832263+0000","last_active":"2024-04-25T09:53:13.326300+0000","last_peered":"2024-04-25T09:53:13.326300+0000","last_clean":"2024-04-25T09:53:13.326300+0000","last_became_active":"2024-04-25T09:53:11.831846+0000","last_became_peered":"2024-04-25T09:53:11.831846+0000","last_unstale":"2024-04-25T09:53:13.326300+0000","last_undegraded":"2024-04-25T09:53:13.326300+0000","last_fullsized":"2024-04-25T09:53:13.326300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:16.884859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326368+0000","last_change":"2024-04-25T09:53:07.797016+0000","last_active":"2024-04-25T09:53:13.326368+0000","last_peered":"2024-04-25T09:53:13.326368+0000","last_clean":"2024-04-25T09:53:13.326368+0000","last_became_active":"2024-04-25T09:53:07.796749+0000","last_became_peered":"2024-04-25T09:53:07.796749+0000","last_unstale":"2024-04-25T09:53:13.326368+0000","last_undegraded":"2024-04-25T09:53:13.326368+0000","last_fullsized":"2024-04-25T09:53:13.326368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:21.755503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326938+0000","last_change":"2024-04-25T09:53:06.782783+0000","last_active":"2024-04-25T09:53:13.326938+0000","last_peered":"2024-04-25T09:53:13.326938+0000","last_clean":"2024-04-25T09:53:13.326938+0000","last_became_active":"2024-04-25T09:53:06.782388+0000","last_became_peered":"2024-04-25T09:53:06.782388+0000","last_unstale":"2024-04-25T09:53:13.326938+0000","last_undegraded":"2024-04-25T09:53:13.326938+0000","last_fullsized":"2024-04-25T09:53:13.326938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325544+0000","last_change":"2024-04-25T09:53:11.832074+0000","last_active":"2024-04-25T09:53:13.325544+0000","last_peered":"2024-04-25T09:53:13.325544+0000","last_clean":"2024-04-25T09:53:13.325544+0000","last_became_active":"2024-04-25T09:53:11.831621+0000","last_became_peered":"2024-04-25T09:53:11.831621+0000","last_unstale":"2024-04-25T09:53:13.325544+0000","last_undegraded":"2024-04-25T09:53:13.325544+0000","last_fullsized":"2024-04-25T09:53:13.325544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.324944+0000","last_change":"2024-04-25T09:53:07.783910+0000","last_active":"2024-04-25T09:53:13.324944+0000","last_peered":"2024-04-25T09:53:13.324944+0000","last_clean":"2024-04-25T09:53:13.324944+0000","last_became_active":"2024-04-25T09:53:07.783625+0000","last_became_peered":"2024-04-25T09:53:07.783625+0000","last_unstale":"2024-04-25T09:53:13.324944+0000","last_undegraded":"2024-04-25T09:53:13.324944+0000","last_fullsized":"2024-04-25T09:53:13.324944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809538+0000","last_change":"2024-04-25T09:53:06.776134+0000","last_active":"2024-04-25T09:53:12.809538+0000","last_peered":"2024-04-25T09:53:12.809538+0000","last_clean":"2024-04-25T09:53:12.809538+0000","last_became_active":"2024-04-25T09:53:06.775787+0000","last_became_peered":"2024-04-25T09:53:06.775787+0000","last_unstale":"2024-04-25T09:53:12.809538+0000","last_undegraded":"2024-04-25T09:53:12.809538+0000","last_fullsized":"2024-04-25T09:53:12.809538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326396+0000","last_change":"2024-04-25T09:53:11.837526+0000","last_active":"2024-04-25T09:53:13.326396+0000","last_peered":"2024-04-25T09:53:13.326396+0000","last_clean":"2024-04-25T09:53:13.326396+0000","last_became_active":"2024-04-25T09:53:11.832037+0000","last_became_peered":"2024-04-25T09:53:11.832037+0000","last_unstale":"2024-04-25T09:53:13.326396+0000","last_undegraded":"2024-04-25T09:53:13.326396+0000","last_fullsized":"2024-04-25T09:53:13.326396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.727046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325424+0000","last_change":"2024-04-25T09:53:07.787155+0000","last_active":"2024-04-25T09:53:13.325424+0000","last_peered":"2024-04-25T09:53:13.325424+0000","last_clean":"2024-04-25T09:53:13.325424+0000","last_became_active":"2024-04-25T09:53:07.783553+0000","last_became_peered":"2024-04-25T09:53:07.783553+0000","last_unstale":"2024-04-25T09:53:13.325424+0000","last_undegraded":"2024-04-25T09:53:13.325424+0000","last_fullsized":"2024-04-25T09:53:13.325424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:12.089487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810045+0000","last_change":"2024-04-25T09:53:06.780626+0000","last_active":"2024-04-25T09:53:12.810045+0000","last_peered":"2024-04-25T09:53:12.810045+0000","last_clean":"2024-04-25T09:53:12.810045+0000","last_became_active":"2024-04-25T09:53:06.780364+0000","last_became_peered":"2024-04-25T09:53:06.780364+0000","last_unstale":"2024-04-25T09:53:12.810045+0000","last_undegraded":"2024-04-25T09:53:12.810045+0000","last_fullsized":"2024-04-25T09:53:12.810045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326877+0000","last_change":"2024-04-25T09:53:11.832887+0000","last_active":"2024-04-25T09:53:13.326877+0000","last_peered":"2024-04-25T09:53:13.326877+0000","last_clean":"2024-04-25T09:53:13.326877+0000","last_became_active":"2024-04-25T09:53:11.832471+0000","last_became_peered":"2024-04-25T09:53:11.832471+0000","last_unstale":"2024-04-25T09:53:13.326877+0000","last_undegraded":"2024-04-25T09:53:13.326877+0000","last_fullsized":"2024-04-25T09:53:13.326877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:03.438164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.324983+0000","last_change":"2024-04-25T09:53:07.793466+0000","last_active":"2024-04-25T09:53:13.324983+0000","last_peered":"2024-04-25T09:53:13.324983+0000","last_clean":"2024-04-25T09:53:13.324983+0000","last_became_active":"2024-04-25T09:53:07.793130+0000","last_became_peered":"2024-04-25T09:53:07.793130+0000","last_unstale":"2024-04-25T09:53:13.324983+0000","last_undegraded":"2024-04-25T09:53:13.324983+0000","last_fullsized":"2024-04-25T09:53:13.324983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326577+0000","last_change":"2024-04-25T09:53:06.780762+0000","last_active":"2024-04-25T09:53:13.326577+0000","last_peered":"2024-04-25T09:53:13.326577+0000","last_clean":"2024-04-25T09:53:13.326577+0000","last_became_active":"2024-04-25T09:53:06.780498+0000","last_became_peered":"2024-04-25T09:53:06.780498+0000","last_unstale":"2024-04-25T09:53:13.326577+0000","last_undegraded":"2024-04-25T09:53:13.326577+0000","last_fullsized":"2024-04-25T09:53:13.326577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810151+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:53:12.810151+0000","last_peered":"2024-04-25T09:53:12.810151+0000","last_clean":"2024-04-25T09:53:12.810151+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:53:12.810151+0000","last_undegraded":"2024-04-25T09:53:12.810151+0000","last_fullsized":"2024-04-25T09:53:12.810151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810776+0000","last_change":"2024-04-25T09:53:11.825407+0000","last_active":"2024-04-25T09:53:12.810776+0000","last_peered":"2024-04-25T09:53:12.810776+0000","last_clean":"2024-04-25T09:53:12.810776+0000","last_became_active":"2024-04-25T09:53:11.824448+0000","last_became_peered":"2024-04-25T09:53:11.824448+0000","last_unstale":"2024-04-25T09:53:12.810776+0000","last_undegraded":"2024-04-25T09:53:12.810776+0000","last_fullsized":"2024-04-25T09:53:12.810776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:59.415156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810231+0000","last_change":"2024-04-25T09:53:07.803634+0000","last_active":"2024-04-25T09:53:12.810231+0000","last_peered":"2024-04-25T09:53:12.810231+0000","last_clean":"2024-04-25T09:53:12.810231+0000","last_became_active":"2024-04-25T09:53:07.803367+0000","last_became_peered":"2024-04-25T09:53:07.803367+0000","last_unstale":"2024-04-25T09:53:12.810231+0000","last_undegraded":"2024-04-25T09:53:12.810231+0000","last_fullsized":"2024-04-25T09:53:12.810231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:55.375947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809562+0000","last_change":"2024-04-25T09:53:06.783978+0000","last_active":"2024-04-25T09:53:12.809562+0000","last_peered":"2024-04-25T09:53:12.809562+0000","last_clean":"2024-04-25T09:53:12.809562+0000","last_became_active":"2024-04-25T09:53:06.783614+0000","last_became_peered":"2024-04-25T09:53:06.783614+0000","last_unstale":"2024-04-25T09:53:12.809562+0000","last_undegraded":"2024-04-25T09:53:12.809562+0000","last_fullsized":"2024-04-25T09:53:12.809562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:52.306054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810011+0000","last_change":"2024-04-25T09:53:11.835086+0000","last_active":"2024-04-25T09:53:12.810011+0000","last_peered":"2024-04-25T09:53:12.810011+0000","last_clean":"2024-04-25T09:53:12.810011+0000","last_became_active":"2024-04-25T09:53:11.834674+0000","last_became_peered":"2024-04-25T09:53:11.834674+0000","last_unstale":"2024-04-25T09:53:12.810011+0000","last_undegraded":"2024-04-25T09:53:12.810011+0000","last_fullsized":"2024-04-25T09:53:12.810011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:01.272017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326681+0000","last_change":"2024-04-25T09:53:07.786966+0000","last_active":"2024-04-25T09:53:13.326681+0000","last_peered":"2024-04-25T09:53:13.326681+0000","last_clean":"2024-04-25T09:53:13.326681+0000","last_became_active":"2024-04-25T09:53:07.786097+0000","last_became_peered":"2024-04-25T09:53:07.786097+0000","last_unstale":"2024-04-25T09:53:13.326681+0000","last_undegraded":"2024-04-25T09:53:13.326681+0000","last_fullsized":"2024-04-25T09:53:13.326681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326102+0000","last_change":"2024-04-25T09:53:06.781647+0000","last_active":"2024-04-25T09:53:13.326102+0000","last_peered":"2024-04-25T09:53:13.326102+0000","last_clean":"2024-04-25T09:53:13.326102+0000","last_became_active":"2024-04-25T09:53:06.781360+0000","last_became_peered":"2024-04-25T09:53:06.781360+0000","last_unstale":"2024-04-25T09:53:13.326102+0000","last_undegraded":"2024-04-25T09:53:13.326102+0000","last_fullsized":"2024-04-25T09:53:13.326102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:44.406768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326043+0000","last_change":"2024-04-25T09:53:11.835499+0000","last_active":"2024-04-25T09:53:13.326043+0000","last_peered":"2024-04-25T09:53:13.326043+0000","last_clean":"2024-04-25T09:53:13.326043+0000","last_became_active":"2024-04-25T09:53:11.834613+0000","last_became_peered":"2024-04-25T09:53:11.834613+0000","last_unstale":"2024-04-25T09:53:13.326043+0000","last_undegraded":"2024-04-25T09:53:13.326043+0000","last_fullsized":"2024-04-25T09:53:13.326043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:16.884859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811569+0000","last_change":"2024-04-25T09:53:07.799270+0000","last_active":"2024-04-25T09:53:12.811569+0000","last_peered":"2024-04-25T09:53:12.811569+0000","last_clean":"2024-04-25T09:53:12.811569+0000","last_became_active":"2024-04-25T09:53:07.799005+0000","last_became_peered":"2024-04-25T09:53:07.799005+0000","last_unstale":"2024-04-25T09:53:12.811569+0000","last_undegraded":"2024-04-25T09:53:12.811569+0000","last_fullsized":"2024-04-25T09:53:12.811569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:47.687838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809643+0000","last_change":"2024-04-25T09:53:06.776508+0000","last_active":"2024-04-25T09:53:12.809643+0000","last_peered":"2024-04-25T09:53:12.809643+0000","last_clean":"2024-04-25T09:53:12.809643+0000","last_became_active":"2024-04-25T09:53:06.775938+0000","last_became_peered":"2024-04-25T09:53:06.775938+0000","last_unstale":"2024-04-25T09:53:12.809643+0000","last_undegraded":"2024-04-25T09:53:12.809643+0000","last_fullsized":"2024-04-25T09:53:12.809643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325709+0000","last_change":"2024-04-25T09:53:11.823495+0000","last_active":"2024-04-25T09:53:13.325709+0000","last_peered":"2024-04-25T09:53:13.325709+0000","last_clean":"2024-04-25T09:53:13.325709+0000","last_became_active":"2024-04-25T09:53:11.823070+0000","last_became_peered":"2024-04-25T09:53:11.823070+0000","last_unstale":"2024-04-25T09:53:13.325709+0000","last_undegraded":"2024-04-25T09:53:13.325709+0000","last_fullsized":"2024-04-25T09:53:13.325709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:58.601489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810162+0000","last_change":"2024-04-25T09:53:07.796560+0000","last_active":"2024-04-25T09:53:12.810162+0000","last_peered":"2024-04-25T09:53:12.810162+0000","last_clean":"2024-04-25T09:53:12.810162+0000","last_became_active":"2024-04-25T09:53:07.796288+0000","last_became_peered":"2024-04-25T09:53:07.796288+0000","last_unstale":"2024-04-25T09:53:12.810162+0000","last_undegraded":"2024-04-25T09:53:12.810162+0000","last_fullsized":"2024-04-25T09:53:12.810162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327440+0000","last_change":"2024-04-25T09:53:06.776524+0000","last_active":"2024-04-25T09:53:13.327440+0000","last_peered":"2024-04-25T09:53:13.327440+0000","last_clean":"2024-04-25T09:53:13.327440+0000","last_became_active":"2024-04-25T09:53:06.776156+0000","last_became_peered":"2024-04-25T09:53:06.776156+0000","last_unstale":"2024-04-25T09:53:13.327440+0000","last_undegraded":"2024-04-25T09:53:13.327440+0000","last_fullsized":"2024-04-25T09:53:13.327440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:23.222453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325855+0000","last_change":"2024-04-25T09:53:11.840103+0000","last_active":"2024-04-25T09:53:13.325855+0000","last_peered":"2024-04-25T09:53:13.325855+0000","last_clean":"2024-04-25T09:53:13.325855+0000","last_became_active":"2024-04-25T09:53:11.839706+0000","last_became_peered":"2024-04-25T09:53:11.839706+0000","last_unstale":"2024-04-25T09:53:13.325855+0000","last_undegraded":"2024-04-25T09:53:13.325855+0000","last_fullsized":"2024-04-25T09:53:13.325855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:35.127642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809957+0000","last_change":"2024-04-25T09:53:07.796678+0000","last_active":"2024-04-25T09:53:12.809957+0000","last_peered":"2024-04-25T09:53:12.809957+0000","last_clean":"2024-04-25T09:53:12.809957+0000","last_became_active":"2024-04-25T09:53:07.796401+0000","last_became_peered":"2024-04-25T09:53:07.796401+0000","last_unstale":"2024-04-25T09:53:12.809957+0000","last_undegraded":"2024-04-25T09:53:12.809957+0000","last_fullsized":"2024-04-25T09:53:12.809957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809151+0000","last_change":"2024-04-25T09:53:06.780411+0000","last_active":"2024-04-25T09:53:12.809151+0000","last_peered":"2024-04-25T09:53:12.809151+0000","last_clean":"2024-04-25T09:53:12.809151+0000","last_became_active":"2024-04-25T09:53:06.780094+0000","last_became_peered":"2024-04-25T09:53:06.780094+0000","last_unstale":"2024-04-25T09:53:12.809151+0000","last_undegraded":"2024-04-25T09:53:12.809151+0000","last_fullsized":"2024-04-25T09:53:12.809151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326778+0000","last_change":"2024-04-25T09:53:11.823203+0000","last_active":"2024-04-25T09:53:13.326778+0000","last_peered":"2024-04-25T09:53:13.326778+0000","last_clean":"2024-04-25T09:53:13.326778+0000","last_became_active":"2024-04-25T09:53:11.822736+0000","last_became_peered":"2024-04-25T09:53:11.822736+0000","last_unstale":"2024-04-25T09:53:13.326778+0000","last_undegraded":"2024-04-25T09:53:13.326778+0000","last_fullsized":"2024-04-25T09:53:13.326778+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:05.899921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809909+0000","last_change":"2024-04-25T09:53:07.797880+0000","last_active":"2024-04-25T09:53:12.809909+0000","last_peered":"2024-04-25T09:53:12.809909+0000","last_clean":"2024-04-25T09:53:12.809909+0000","last_became_active":"2024-04-25T09:53:07.797626+0000","last_became_peered":"2024-04-25T09:53:07.797626+0000","last_unstale":"2024-04-25T09:53:12.809909+0000","last_undegraded":"2024-04-25T09:53:12.809909+0000","last_fullsized":"2024-04-25T09:53:12.809909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810332+0000","last_change":"2024-04-25T09:53:06.781752+0000","last_active":"2024-04-25T09:53:12.810332+0000","last_peered":"2024-04-25T09:53:12.810332+0000","last_clean":"2024-04-25T09:53:12.810332+0000","last_became_active":"2024-04-25T09:53:06.781191+0000","last_became_peered":"2024-04-25T09:53:06.781191+0000","last_unstale":"2024-04-25T09:53:12.810332+0000","last_undegraded":"2024-04-25T09:53:12.810332+0000","last_fullsized":"2024-04-25T09:53:12.810332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326993+0000","last_change":"2024-04-25T09:53:11.839382+0000","last_active":"2024-04-25T09:53:13.326993+0000","last_peered":"2024-04-25T09:53:13.326993+0000","last_clean":"2024-04-25T09:53:13.326993+0000","last_became_active":"2024-04-25T09:53:11.839017+0000","last_became_peered":"2024-04-25T09:53:11.839017+0000","last_unstale":"2024-04-25T09:53:13.326993+0000","last_undegraded":"2024-04-25T09:53:13.326993+0000","last_fullsized":"2024-04-25T09:53:13.326993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:08.728070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810403+0000","last_change":"2024-04-25T09:53:07.797176+0000","last_active":"2024-04-25T09:53:12.810403+0000","last_peered":"2024-04-25T09:53:12.810403+0000","last_clean":"2024-04-25T09:53:12.810403+0000","last_became_active":"2024-04-25T09:53:07.796900+0000","last_became_peered":"2024-04-25T09:53:07.796900+0000","last_unstale":"2024-04-25T09:53:12.810403+0000","last_undegraded":"2024-04-25T09:53:12.810403+0000","last_fullsized":"2024-04-25T09:53:12.810403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326836+0000","last_change":"2024-04-25T09:53:06.783568+0000","last_active":"2024-04-25T09:53:13.326836+0000","last_peered":"2024-04-25T09:53:13.326836+0000","last_clean":"2024-04-25T09:53:13.326836+0000","last_became_active":"2024-04-25T09:53:06.783237+0000","last_became_peered":"2024-04-25T09:53:06.783237+0000","last_unstale":"2024-04-25T09:53:13.326836+0000","last_undegraded":"2024-04-25T09:53:13.326836+0000","last_fullsized":"2024-04-25T09:53:13.326836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326468+0000","last_change":"2024-04-25T09:53:11.835512+0000","last_active":"2024-04-25T09:53:13.326468+0000","last_peered":"2024-04-25T09:53:13.326468+0000","last_clean":"2024-04-25T09:53:13.326468+0000","last_became_active":"2024-04-25T09:53:11.835133+0000","last_became_peered":"2024-04-25T09:53:11.835133+0000","last_unstale":"2024-04-25T09:53:13.326468+0000","last_undegraded":"2024-04-25T09:53:13.326468+0000","last_fullsized":"2024-04-25T09:53:13.326468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:13.599619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326511+0000","last_change":"2024-04-25T09:53:07.788286+0000","last_active":"2024-04-25T09:53:13.326511+0000","last_peered":"2024-04-25T09:53:13.326511+0000","last_clean":"2024-04-25T09:53:13.326511+0000","last_became_active":"2024-04-25T09:53:07.787920+0000","last_became_peered":"2024-04-25T09:53:07.787920+0000","last_unstale":"2024-04-25T09:53:13.326511+0000","last_undegraded":"2024-04-25T09:53:13.326511+0000","last_fullsized":"2024-04-25T09:53:13.326511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325923+0000","last_change":"2024-04-25T09:53:06.772006+0000","last_active":"2024-04-25T09:53:13.325923+0000","last_peered":"2024-04-25T09:53:13.325923+0000","last_clean":"2024-04-25T09:53:13.325923+0000","last_became_active":"2024-04-25T09:53:06.771702+0000","last_became_peered":"2024-04-25T09:53:06.771702+0000","last_unstale":"2024-04-25T09:53:13.325923+0000","last_undegraded":"2024-04-25T09:53:13.325923+0000","last_fullsized":"2024-04-25T09:53:13.325923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:59.219341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327029+0000","last_change":"2024-04-25T09:53:11.821559+0000","last_active":"2024-04-25T09:53:13.327029+0000","last_peered":"2024-04-25T09:53:13.327029+0000","last_clean":"2024-04-25T09:53:13.327029+0000","last_became_active":"2024-04-25T09:53:11.821133+0000","last_became_peered":"2024-04-25T09:53:11.821133+0000","last_unstale":"2024-04-25T09:53:13.327029+0000","last_undegraded":"2024-04-25T09:53:13.327029+0000","last_fullsized":"2024-04-25T09:53:13.327029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:58.718292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810108+0000","last_change":"2024-04-25T09:53:07.797403+0000","last_active":"2024-04-25T09:53:12.810108+0000","last_peered":"2024-04-25T09:53:12.810108+0000","last_clean":"2024-04-25T09:53:12.810108+0000","last_became_active":"2024-04-25T09:53:07.796257+0000","last_became_peered":"2024-04-25T09:53:07.796257+0000","last_unstale":"2024-04-25T09:53:12.810108+0000","last_undegraded":"2024-04-25T09:53:12.810108+0000","last_fullsized":"2024-04-25T09:53:12.810108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:17.046098+0000","last_change":"2024-04-25T09:53:06.773036+0000","last_active":"2024-04-25T09:53:17.046098+0000","last_peered":"2024-04-25T09:53:17.046098+0000","last_clean":"2024-04-25T09:53:17.046098+0000","last_became_active":"2024-04-25T09:53:06.772729+0000","last_became_peered":"2024-04-25T09:53:06.772729+0000","last_unstale":"2024-04-25T09:53:17.046098+0000","last_undegraded":"2024-04-25T09:53:17.046098+0000","last_fullsized":"2024-04-25T09:53:17.046098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326453+0000","last_change":"2024-04-25T09:53:11.833689+0000","last_active":"2024-04-25T09:53:13.326453+0000","last_peered":"2024-04-25T09:53:13.326453+0000","last_clean":"2024-04-25T09:53:13.326453+0000","last_became_active":"2024-04-25T09:53:11.832588+0000","last_became_peered":"2024-04-25T09:53:11.832588+0000","last_unstale":"2024-04-25T09:53:13.326453+0000","last_undegraded":"2024-04-25T09:53:13.326453+0000","last_fullsized":"2024-04-25T09:53:13.326453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:05.899921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327010+0000","last_change":"2024-04-25T09:53:07.791857+0000","last_active":"2024-04-25T09:53:13.327010+0000","last_peered":"2024-04-25T09:53:13.327010+0000","last_clean":"2024-04-25T09:53:13.327010+0000","last_became_active":"2024-04-25T09:53:07.791606+0000","last_became_peered":"2024-04-25T09:53:07.791606+0000","last_unstale":"2024-04-25T09:53:13.327010+0000","last_undegraded":"2024-04-25T09:53:13.327010+0000","last_fullsized":"2024-04-25T09:53:13.327010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.768421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325995+0000","last_change":"2024-04-25T09:53:06.783616+0000","last_active":"2024-04-25T09:53:13.325995+0000","last_peered":"2024-04-25T09:53:13.325995+0000","last_clean":"2024-04-25T09:53:13.325995+0000","last_became_active":"2024-04-25T09:53:06.783302+0000","last_became_peered":"2024-04-25T09:53:06.783302+0000","last_unstale":"2024-04-25T09:53:13.325995+0000","last_undegraded":"2024-04-25T09:53:13.325995+0000","last_fullsized":"2024-04-25T09:53:13.325995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326861+0000","last_change":"2024-04-25T09:53:11.840956+0000","last_active":"2024-04-25T09:53:13.326861+0000","last_peered":"2024-04-25T09:53:13.326861+0000","last_clean":"2024-04-25T09:53:13.326861+0000","last_became_active":"2024-04-25T09:53:11.840316+0000","last_became_peered":"2024-04-25T09:53:11.840316+0000","last_unstale":"2024-04-25T09:53:13.326861+0000","last_undegraded":"2024-04-25T09:53:13.326861+0000","last_fullsized":"2024-04-25T09:53:13.326861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:01.272017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326067+0000","last_change":"2024-04-25T09:53:07.787945+0000","last_active":"2024-04-25T09:53:13.326067+0000","last_peered":"2024-04-25T09:53:13.326067+0000","last_clean":"2024-04-25T09:53:13.326067+0000","last_became_active":"2024-04-25T09:53:07.786113+0000","last_became_peered":"2024-04-25T09:53:07.786113+0000","last_unstale":"2024-04-25T09:53:13.326067+0000","last_undegraded":"2024-04-25T09:53:13.326067+0000","last_fullsized":"2024-04-25T09:53:13.326067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327211+0000","last_change":"2024-04-25T09:53:06.783604+0000","last_active":"2024-04-25T09:53:13.327211+0000","last_peered":"2024-04-25T09:53:13.327211+0000","last_clean":"2024-04-25T09:53:13.327211+0000","last_became_active":"2024-04-25T09:53:06.783341+0000","last_became_peered":"2024-04-25T09:53:06.783341+0000","last_unstale":"2024-04-25T09:53:13.327211+0000","last_undegraded":"2024-04-25T09:53:13.327211+0000","last_fullsized":"2024-04-25T09:53:13.327211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326141+0000","last_change":"2024-04-25T09:53:11.836053+0000","last_active":"2024-04-25T09:53:13.326141+0000","last_peered":"2024-04-25T09:53:13.326141+0000","last_clean":"2024-04-25T09:53:13.326141+0000","last_became_active":"2024-04-25T09:53:11.835318+0000","last_became_peered":"2024-04-25T09:53:11.835318+0000","last_unstale":"2024-04-25T09:53:13.326141+0000","last_undegraded":"2024-04-25T09:53:13.326141+0000","last_fullsized":"2024-04-25T09:53:13.326141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:49.717043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327141+0000","last_change":"2024-04-25T09:53:07.793149+0000","last_active":"2024-04-25T09:53:13.327141+0000","last_peered":"2024-04-25T09:53:13.327141+0000","last_clean":"2024-04-25T09:53:13.327141+0000","last_became_active":"2024-04-25T09:53:07.792896+0000","last_became_peered":"2024-04-25T09:53:07.792896+0000","last_unstale":"2024-04-25T09:53:13.327141+0000","last_undegraded":"2024-04-25T09:53:13.327141+0000","last_fullsized":"2024-04-25T09:53:13.327141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:55.375947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811223+0000","last_change":"2024-04-25T09:53:06.780916+0000","last_active":"2024-04-25T09:53:12.811223+0000","last_peered":"2024-04-25T09:53:12.811223+0000","last_clean":"2024-04-25T09:53:12.811223+0000","last_became_active":"2024-04-25T09:53:06.780631+0000","last_became_peered":"2024-04-25T09:53:06.780631+0000","last_unstale":"2024-04-25T09:53:12.811223+0000","last_undegraded":"2024-04-25T09:53:12.811223+0000","last_fullsized":"2024-04-25T09:53:12.811223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325834+0000","last_change":"2024-04-25T09:53:11.821088+0000","last_active":"2024-04-25T09:53:13.325834+0000","last_peered":"2024-04-25T09:53:13.325834+0000","last_clean":"2024-04-25T09:53:13.325834+0000","last_became_active":"2024-04-25T09:53:11.820662+0000","last_became_peered":"2024-04-25T09:53:11.820662+0000","last_unstale":"2024-04-25T09:53:13.325834+0000","last_undegraded":"2024-04-25T09:53:13.325834+0000","last_fullsized":"2024-04-25T09:53:13.325834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:58.601489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326720+0000","last_change":"2024-04-25T09:53:11.838966+0000","last_active":"2024-04-25T09:53:13.326720+0000","last_peered":"2024-04-25T09:53:13.326720+0000","last_clean":"2024-04-25T09:53:13.326720+0000","last_became_active":"2024-04-25T09:53:11.838542+0000","last_became_peered":"2024-04-25T09:53:11.838542+0000","last_unstale":"2024-04-25T09:53:13.326720+0000","last_undegraded":"2024-04-25T09:53:13.326720+0000","last_fullsized":"2024-04-25T09:53:13.326720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.727046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325110+0000","last_change":"2024-04-25T09:53:07.784870+0000","last_active":"2024-04-25T09:53:13.325110+0000","last_peered":"2024-04-25T09:53:13.325110+0000","last_clean":"2024-04-25T09:53:13.325110+0000","last_became_active":"2024-04-25T09:53:07.784587+0000","last_became_peered":"2024-04-25T09:53:07.784587+0000","last_unstale":"2024-04-25T09:53:13.325110+0000","last_undegraded":"2024-04-25T09:53:13.325110+0000","last_fullsized":"2024-04-25T09:53:13.325110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325791+0000","last_change":"2024-04-25T09:53:06.782572+0000","last_active":"2024-04-25T09:53:13.325791+0000","last_peered":"2024-04-25T09:53:13.325791+0000","last_clean":"2024-04-25T09:53:13.325791+0000","last_became_active":"2024-04-25T09:53:06.781792+0000","last_became_peered":"2024-04-25T09:53:06.781792+0000","last_unstale":"2024-04-25T09:53:13.325791+0000","last_undegraded":"2024-04-25T09:53:13.325791+0000","last_fullsized":"2024-04-25T09:53:13.325791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811107+0000","last_change":"2024-04-25T09:53:11.838770+0000","last_active":"2024-04-25T09:53:12.811107+0000","last_peered":"2024-04-25T09:53:12.811107+0000","last_clean":"2024-04-25T09:53:12.811107+0000","last_became_active":"2024-04-25T09:53:11.838328+0000","last_became_peered":"2024-04-25T09:53:11.838328+0000","last_unstale":"2024-04-25T09:53:12.811107+0000","last_undegraded":"2024-04-25T09:53:12.811107+0000","last_fullsized":"2024-04-25T09:53:12.811107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326187+0000","last_change":"2024-04-25T09:53:07.791725+0000","last_active":"2024-04-25T09:53:13.326187+0000","last_peered":"2024-04-25T09:53:13.326187+0000","last_clean":"2024-04-25T09:53:13.326187+0000","last_became_active":"2024-04-25T09:53:07.791386+0000","last_became_peered":"2024-04-25T09:53:07.791386+0000","last_unstale":"2024-04-25T09:53:13.326187+0000","last_undegraded":"2024-04-25T09:53:13.326187+0000","last_fullsized":"2024-04-25T09:53:13.326187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326069+0000","last_change":"2024-04-25T09:53:06.775205+0000","last_active":"2024-04-25T09:53:13.326069+0000","last_peered":"2024-04-25T09:53:13.326069+0000","last_clean":"2024-04-25T09:53:13.326069+0000","last_became_active":"2024-04-25T09:53:06.774911+0000","last_became_peered":"2024-04-25T09:53:06.774911+0000","last_unstale":"2024-04-25T09:53:13.326069+0000","last_undegraded":"2024-04-25T09:53:13.326069+0000","last_fullsized":"2024-04-25T09:53:13.326069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.898659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810239+0000","last_change":"2024-04-25T09:53:06.781528+0000","last_active":"2024-04-25T09:53:12.810239+0000","last_peered":"2024-04-25T09:53:12.810239+0000","last_clean":"2024-04-25T09:53:12.810239+0000","last_became_active":"2024-04-25T09:53:06.781248+0000","last_became_peered":"2024-04-25T09:53:06.781248+0000","last_unstale":"2024-04-25T09:53:12.810239+0000","last_undegraded":"2024-04-25T09:53:12.810239+0000","last_fullsized":"2024-04-25T09:53:12.810239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325829+0000","last_change":"2024-04-25T09:53:07.783095+0000","last_active":"2024-04-25T09:53:13.325829+0000","last_peered":"2024-04-25T09:53:13.325829+0000","last_clean":"2024-04-25T09:53:13.325829+0000","last_became_active":"2024-04-25T09:53:07.782578+0000","last_became_peered":"2024-04-25T09:53:07.782578+0000","last_unstale":"2024-04-25T09:53:13.325829+0000","last_undegraded":"2024-04-25T09:53:13.325829+0000","last_fullsized":"2024-04-25T09:53:13.325829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809581+0000","last_change":"2024-04-25T09:53:11.834200+0000","last_active":"2024-04-25T09:53:12.809581+0000","last_peered":"2024-04-25T09:53:12.809581+0000","last_clean":"2024-04-25T09:53:12.809581+0000","last_became_active":"2024-04-25T09:53:11.833774+0000","last_became_peered":"2024-04-25T09:53:11.833774+0000","last_unstale":"2024-04-25T09:53:12.809581+0000","last_undegraded":"2024-04-25T09:53:12.809581+0000","last_fullsized":"2024-04-25T09:53:12.809581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:42.139275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325379+0000","last_change":"2024-04-25T09:53:06.777867+0000","last_active":"2024-04-25T09:53:13.325379+0000","last_peered":"2024-04-25T09:53:13.325379+0000","last_clean":"2024-04-25T09:53:13.325379+0000","last_became_active":"2024-04-25T09:53:06.777566+0000","last_became_peered":"2024-04-25T09:53:06.777566+0000","last_unstale":"2024-04-25T09:53:13.325379+0000","last_undegraded":"2024-04-25T09:53:13.325379+0000","last_fullsized":"2024-04-25T09:53:13.325379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:52.306054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326173+0000","last_change":"2024-04-25T09:53:07.781653+0000","last_active":"2024-04-25T09:53:13.326173+0000","last_peered":"2024-04-25T09:53:13.326173+0000","last_clean":"2024-04-25T09:53:13.326173+0000","last_became_active":"2024-04-25T09:53:07.781274+0000","last_became_peered":"2024-04-25T09:53:07.781274+0000","last_unstale":"2024-04-25T09:53:13.326173+0000","last_undegraded":"2024-04-25T09:53:13.326173+0000","last_fullsized":"2024-04-25T09:53:13.326173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:02.037522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326829+0000","last_change":"2024-04-25T09:53:11.834554+0000","last_active":"2024-04-25T09:53:13.326829+0000","last_peered":"2024-04-25T09:53:13.326829+0000","last_clean":"2024-04-25T09:53:13.326829+0000","last_became_active":"2024-04-25T09:53:11.834143+0000","last_became_peered":"2024-04-25T09:53:11.834143+0000","last_unstale":"2024-04-25T09:53:13.326829+0000","last_undegraded":"2024-04-25T09:53:13.326829+0000","last_fullsized":"2024-04-25T09:53:13.326829+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:37.372311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809684+0000","last_change":"2024-04-25T09:53:07.798578+0000","last_active":"2024-04-25T09:53:12.809684+0000","last_peered":"2024-04-25T09:53:12.809684+0000","last_clean":"2024-04-25T09:53:12.809684+0000","last_became_active":"2024-04-25T09:53:07.798310+0000","last_became_peered":"2024-04-25T09:53:07.798310+0000","last_unstale":"2024-04-25T09:53:12.809684+0000","last_undegraded":"2024-04-25T09:53:12.809684+0000","last_fullsized":"2024-04-25T09:53:12.809684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809471+0000","last_change":"2024-04-25T09:53:07.780646+0000","last_active":"2024-04-25T09:53:12.809471+0000","last_peered":"2024-04-25T09:53:12.809471+0000","last_clean":"2024-04-25T09:53:12.809471+0000","last_became_active":"2024-04-25T09:53:07.780321+0000","last_became_peered":"2024-04-25T09:53:07.780321+0000","last_unstale":"2024-04-25T09:53:12.809471+0000","last_undegraded":"2024-04-25T09:53:12.809471+0000","last_fullsized":"2024-04-25T09:53:12.809471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:44.576758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326872+0000","last_change":"2024-04-25T09:53:07.787491+0000","last_active":"2024-04-25T09:53:13.326872+0000","last_peered":"2024-04-25T09:53:13.326872+0000","last_clean":"2024-04-25T09:53:13.326872+0000","last_became_active":"2024-04-25T09:53:07.787214+0000","last_became_peered":"2024-04-25T09:53:07.787214+0000","last_unstale":"2024-04-25T09:53:13.326872+0000","last_undegraded":"2024-04-25T09:53:13.326872+0000","last_fullsized":"2024-04-25T09:53:13.326872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:35.714783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325728+0000","last_change":"2024-04-25T09:53:07.789805+0000","last_active":"2024-04-25T09:53:13.325728+0000","last_peered":"2024-04-25T09:53:13.325728+0000","last_clean":"2024-04-25T09:53:13.325728+0000","last_became_active":"2024-04-25T09:53:07.788987+0000","last_became_peered":"2024-04-25T09:53:07.788987+0000","last_unstale":"2024-04-25T09:53:13.325728+0000","last_undegraded":"2024-04-25T09:53:13.325728+0000","last_fullsized":"2024-04-25T09:53:13.325728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326515+0000","last_change":"2024-04-25T09:53:07.792754+0000","last_active":"2024-04-25T09:53:13.326515+0000","last_peered":"2024-04-25T09:53:13.326515+0000","last_clean":"2024-04-25T09:53:13.326515+0000","last_became_active":"2024-04-25T09:53:07.792223+0000","last_became_peered":"2024-04-25T09:53:07.792223+0000","last_unstale":"2024-04-25T09:53:13.326515+0000","last_undegraded":"2024-04-25T09:53:13.326515+0000","last_fullsized":"2024-04-25T09:53:13.326515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325889+0000","last_change":"2024-04-25T09:53:07.783936+0000","last_active":"2024-04-25T09:53:13.325889+0000","last_peered":"2024-04-25T09:53:13.325889+0000","last_clean":"2024-04-25T09:53:13.325889+0000","last_became_active":"2024-04-25T09:53:07.783646+0000","last_became_peered":"2024-04-25T09:53:07.783646+0000","last_unstale":"2024-04-25T09:53:13.325889+0000","last_undegraded":"2024-04-25T09:53:13.325889+0000","last_fullsized":"2024-04-25T09:53:13.325889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810830+0000","last_change":"2024-04-25T09:53:07.789428+0000","last_active":"2024-04-25T09:53:12.810830+0000","last_peered":"2024-04-25T09:53:12.810830+0000","last_clean":"2024-04-25T09:53:12.810830+0000","last_became_active":"2024-04-25T09:53:07.789086+0000","last_became_peered":"2024-04-25T09:53:07.789086+0000","last_unstale":"2024-04-25T09:53:12.810830+0000","last_undegraded":"2024-04-25T09:53:12.810830+0000","last_fullsized":"2024-04-25T09:53:12.810830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327059+0000","last_change":"2024-04-25T09:53:07.785822+0000","last_active":"2024-04-25T09:53:13.327059+0000","last_peered":"2024-04-25T09:53:13.327059+0000","last_clean":"2024-04-25T09:53:13.327059+0000","last_became_active":"2024-04-25T09:53:07.785535+0000","last_became_peered":"2024-04-25T09:53:07.785535+0000","last_unstale":"2024-04-25T09:53:13.327059+0000","last_undegraded":"2024-04-25T09:53:13.327059+0000","last_fullsized":"2024-04-25T09:53:13.327059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325833+0000","last_change":"2024-04-25T09:53:07.787296+0000","last_active":"2024-04-25T09:53:13.325833+0000","last_peered":"2024-04-25T09:53:13.325833+0000","last_clean":"2024-04-25T09:53:13.325833+0000","last_became_active":"2024-04-25T09:53:07.787011+0000","last_became_peered":"2024-04-25T09:53:07.787011+0000","last_unstale":"2024-04-25T09:53:13.325833+0000","last_undegraded":"2024-04-25T09:53:13.325833+0000","last_fullsized":"2024-04-25T09:53:13.325833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325938+0000","last_change":"2024-04-25T09:53:07.785405+0000","last_active":"2024-04-25T09:53:13.325938+0000","last_peered":"2024-04-25T09:53:13.325938+0000","last_clean":"2024-04-25T09:53:13.325938+0000","last_became_active":"2024-04-25T09:53:07.785085+0000","last_became_peered":"2024-04-25T09:53:07.785085+0000","last_unstale":"2024-04-25T09:53:13.325938+0000","last_undegraded":"2024-04-25T09:53:13.325938+0000","last_fullsized":"2024-04-25T09:53:13.325938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326705+0000","last_change":"2024-04-25T09:53:07.792913+0000","last_active":"2024-04-25T09:53:13.326705+0000","last_peered":"2024-04-25T09:53:13.326705+0000","last_clean":"2024-04-25T09:53:13.326705+0000","last_became_active":"2024-04-25T09:53:07.792666+0000","last_became_peered":"2024-04-25T09:53:07.792666+0000","last_unstale":"2024-04-25T09:53:13.326705+0000","last_undegraded":"2024-04-25T09:53:13.326705+0000","last_fullsized":"2024-04-25T09:53:13.326705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325521+0000","last_change":"2024-04-25T09:53:07.789785+0000","last_active":"2024-04-25T09:53:13.325521+0000","last_peered":"2024-04-25T09:53:13.325521+0000","last_clean":"2024-04-25T09:53:13.325521+0000","last_became_active":"2024-04-25T09:53:07.789109+0000","last_became_peered":"2024-04-25T09:53:07.789109+0000","last_unstale":"2024-04-25T09:53:13.325521+0000","last_undegraded":"2024-04-25T09:53:13.325521+0000","last_fullsized":"2024-04-25T09:53:13.325521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:28.708261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811599+0000","last_change":"2024-04-25T09:53:07.800839+0000","last_active":"2024-04-25T09:53:12.811599+0000","last_peered":"2024-04-25T09:53:12.811599+0000","last_clean":"2024-04-25T09:53:12.811599+0000","last_became_active":"2024-04-25T09:53:07.800570+0000","last_became_peered":"2024-04-25T09:53:07.800570+0000","last_unstale":"2024-04-25T09:53:12.811599+0000","last_undegraded":"2024-04-25T09:53:12.811599+0000","last_fullsized":"2024-04-25T09:53:12.811599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810497+0000","last_change":"2024-04-25T09:53:07.793203+0000","last_active":"2024-04-25T09:53:12.810497+0000","last_peered":"2024-04-25T09:53:12.810497+0000","last_clean":"2024-04-25T09:53:12.810497+0000","last_became_active":"2024-04-25T09:53:07.792940+0000","last_became_peered":"2024-04-25T09:53:07.792940+0000","last_unstale":"2024-04-25T09:53:12.810497+0000","last_undegraded":"2024-04-25T09:53:12.810497+0000","last_fullsized":"2024-04-25T09:53:12.810497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.221246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809630+0000","last_change":"2024-04-25T09:53:07.789420+0000","last_active":"2024-04-25T09:53:12.809630+0000","last_peered":"2024-04-25T09:53:12.809630+0000","last_clean":"2024-04-25T09:53:12.809630+0000","last_became_active":"2024-04-25T09:53:07.789112+0000","last_became_peered":"2024-04-25T09:53:07.789112+0000","last_unstale":"2024-04-25T09:53:12.809630+0000","last_undegraded":"2024-04-25T09:53:12.809630+0000","last_fullsized":"2024-04-25T09:53:12.809630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.768421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327332+0000","last_change":"2024-04-25T09:53:07.794475+0000","last_active":"2024-04-25T09:53:13.327332+0000","last_peered":"2024-04-25T09:53:13.327332+0000","last_clean":"2024-04-25T09:53:13.327332+0000","last_became_active":"2024-04-25T09:53:07.794225+0000","last_became_peered":"2024-04-25T09:53:07.794225+0000","last_unstale":"2024-04-25T09:53:13.327332+0000","last_undegraded":"2024-04-25T09:53:13.327332+0000","last_fullsized":"2024-04-25T09:53:13.327332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:47.687838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325668+0000","last_change":"2024-04-25T09:53:07.798926+0000","last_active":"2024-04-25T09:53:13.325668+0000","last_peered":"2024-04-25T09:53:13.325668+0000","last_clean":"2024-04-25T09:53:13.325668+0000","last_became_active":"2024-04-25T09:53:07.798446+0000","last_became_peered":"2024-04-25T09:53:07.798446+0000","last_unstale":"2024-04-25T09:53:13.325668+0000","last_undegraded":"2024-04-25T09:53:13.325668+0000","last_fullsized":"2024-04-25T09:53:13.325668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325115+0000","last_change":"2024-04-25T09:53:07.784610+0000","last_active":"2024-04-25T09:53:13.325115+0000","last_peered":"2024-04-25T09:53:13.325115+0000","last_clean":"2024-04-25T09:53:13.325115+0000","last_became_active":"2024-04-25T09:53:07.784323+0000","last_became_peered":"2024-04-25T09:53:07.784323+0000","last_unstale":"2024-04-25T09:53:13.325115+0000","last_undegraded":"2024-04-25T09:53:13.325115+0000","last_fullsized":"2024-04-25T09:53:13.325115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326504+0000","last_change":"2024-04-25T09:53:07.781472+0000","last_active":"2024-04-25T09:53:13.326504+0000","last_peered":"2024-04-25T09:53:13.326504+0000","last_clean":"2024-04-25T09:53:13.326504+0000","last_became_active":"2024-04-25T09:53:07.781156+0000","last_became_peered":"2024-04-25T09:53:07.781156+0000","last_unstale":"2024-04-25T09:53:13.326504+0000","last_undegraded":"2024-04-25T09:53:13.326504+0000","last_fullsized":"2024-04-25T09:53:13.326504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:49.469537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325209+0000","last_change":"2024-04-25T09:53:07.790342+0000","last_active":"2024-04-25T09:53:13.325209+0000","last_peered":"2024-04-25T09:53:13.325209+0000","last_clean":"2024-04-25T09:53:13.325209+0000","last_became_active":"2024-04-25T09:53:07.790003+0000","last_became_peered":"2024-04-25T09:53:07.790003+0000","last_unstale":"2024-04-25T09:53:13.325209+0000","last_undegraded":"2024-04-25T09:53:13.325209+0000","last_fullsized":"2024-04-25T09:53:13.325209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326892+0000","last_change":"2024-04-25T09:53:07.791198+0000","last_active":"2024-04-25T09:53:13.326892+0000","last_peered":"2024-04-25T09:53:13.326892+0000","last_clean":"2024-04-25T09:53:13.326892+0000","last_became_active":"2024-04-25T09:53:07.790896+0000","last_became_peered":"2024-04-25T09:53:07.790896+0000","last_unstale":"2024-04-25T09:53:13.326892+0000","last_undegraded":"2024-04-25T09:53:13.326892+0000","last_fullsized":"2024-04-25T09:53:13.326892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:45.411393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810894+0000","last_change":"2024-04-25T09:53:07.790314+0000","last_active":"2024-04-25T09:53:12.810894+0000","last_peered":"2024-04-25T09:53:12.810894+0000","last_clean":"2024-04-25T09:53:12.810894+0000","last_became_active":"2024-04-25T09:53:07.789999+0000","last_became_peered":"2024-04-25T09:53:07.789999+0000","last_unstale":"2024-04-25T09:53:12.810894+0000","last_undegraded":"2024-04-25T09:53:12.810894+0000","last_fullsized":"2024-04-25T09:53:12.810894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810741+0000","last_change":"2024-04-25T09:53:07.793396+0000","last_active":"2024-04-25T09:53:12.810741+0000","last_peered":"2024-04-25T09:53:12.810741+0000","last_clean":"2024-04-25T09:53:12.810741+0000","last_became_active":"2024-04-25T09:53:07.792132+0000","last_became_peered":"2024-04-25T09:53:07.792132+0000","last_unstale":"2024-04-25T09:53:12.810741+0000","last_undegraded":"2024-04-25T09:53:12.810741+0000","last_fullsized":"2024-04-25T09:53:12.810741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811509+0000","last_change":"2024-04-25T09:53:07.800101+0000","last_active":"2024-04-25T09:53:12.811509+0000","last_peered":"2024-04-25T09:53:12.811509+0000","last_clean":"2024-04-25T09:53:12.811509+0000","last_became_active":"2024-04-25T09:53:07.799794+0000","last_became_peered":"2024-04-25T09:53:07.799794+0000","last_unstale":"2024-04-25T09:53:12.811509+0000","last_undegraded":"2024-04-25T09:53:12.811509+0000","last_fullsized":"2024-04-25T09:53:12.811509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:12.845651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325964+0000","last_change":"2024-04-25T09:53:07.793404+0000","last_active":"2024-04-25T09:53:13.325964+0000","last_peered":"2024-04-25T09:53:13.325964+0000","last_clean":"2024-04-25T09:53:13.325964+0000","last_became_active":"2024-04-25T09:53:07.793140+0000","last_became_peered":"2024-04-25T09:53:07.793140+0000","last_unstale":"2024-04-25T09:53:13.325964+0000","last_undegraded":"2024-04-25T09:53:13.325964+0000","last_fullsized":"2024-04-25T09:53:13.325964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809861+0000","last_change":"2024-04-25T09:53:07.797288+0000","last_active":"2024-04-25T09:53:12.809861+0000","last_peered":"2024-04-25T09:53:12.809861+0000","last_clean":"2024-04-25T09:53:12.809861+0000","last_became_active":"2024-04-25T09:53:07.797017+0000","last_became_peered":"2024-04-25T09:53:07.797017+0000","last_unstale":"2024-04-25T09:53:12.809861+0000","last_undegraded":"2024-04-25T09:53:12.809861+0000","last_fullsized":"2024-04-25T09:53:12.809861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326593+0000","last_change":"2024-04-25T09:53:07.800134+0000","last_active":"2024-04-25T09:53:13.326593+0000","last_peered":"2024-04-25T09:53:13.326593+0000","last_clean":"2024-04-25T09:53:13.326593+0000","last_became_active":"2024-04-25T09:53:07.799817+0000","last_became_peered":"2024-04-25T09:53:07.799817+0000","last_unstale":"2024-04-25T09:53:13.326593+0000","last_undegraded":"2024-04-25T09:53:13.326593+0000","last_fullsized":"2024-04-25T09:53:13.326593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810285+0000","last_change":"2024-04-25T09:53:07.792436+0000","last_active":"2024-04-25T09:53:12.810285+0000","last_peered":"2024-04-25T09:53:12.810285+0000","last_clean":"2024-04-25T09:53:12.810285+0000","last_became_active":"2024-04-25T09:53:07.792150+0000","last_became_peered":"2024-04-25T09:53:07.792150+0000","last_unstale":"2024-04-25T09:53:12.810285+0000","last_undegraded":"2024-04-25T09:53:12.810285+0000","last_fullsized":"2024-04-25T09:53:12.810285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326560+0000","last_change":"2024-04-25T09:53:07.788302+0000","last_active":"2024-04-25T09:53:13.326560+0000","last_peered":"2024-04-25T09:53:13.326560+0000","last_clean":"2024-04-25T09:53:13.326560+0000","last_became_active":"2024-04-25T09:53:07.788033+0000","last_became_peered":"2024-04-25T09:53:07.788033+0000","last_unstale":"2024-04-25T09:53:13.326560+0000","last_undegraded":"2024-04-25T09:53:13.326560+0000","last_fullsized":"2024-04-25T09:53:13.326560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325682+0000","last_change":"2024-04-25T09:53:07.776111+0000","last_active":"2024-04-25T09:53:13.325682+0000","last_peered":"2024-04-25T09:53:13.325682+0000","last_clean":"2024-04-25T09:53:13.325682+0000","last_became_active":"2024-04-25T09:53:07.775779+0000","last_became_peered":"2024-04-25T09:53:07.775779+0000","last_unstale":"2024-04-25T09:53:13.325682+0000","last_undegraded":"2024-04-25T09:53:13.325682+0000","last_fullsized":"2024-04-25T09:53:13.325682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325773+0000","last_change":"2024-04-25T09:53:07.782293+0000","last_active":"2024-04-25T09:53:13.325773+0000","last_peered":"2024-04-25T09:53:13.325773+0000","last_clean":"2024-04-25T09:53:13.325773+0000","last_became_active":"2024-04-25T09:53:07.781957+0000","last_became_peered":"2024-04-25T09:53:07.781957+0000","last_unstale":"2024-04-25T09:53:13.325773+0000","last_undegraded":"2024-04-25T09:53:13.325773+0000","last_fullsized":"2024-04-25T09:53:13.325773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811414+0000","last_change":"2024-04-25T09:53:07.791822+0000","last_active":"2024-04-25T09:53:12.811414+0000","last_peered":"2024-04-25T09:53:12.811414+0000","last_clean":"2024-04-25T09:53:12.811414+0000","last_became_active":"2024-04-25T09:53:07.791566+0000","last_became_peered":"2024-04-25T09:53:07.791566+0000","last_unstale":"2024-04-25T09:53:12.811414+0000","last_undegraded":"2024-04-25T09:53:12.811414+0000","last_fullsized":"2024-04-25T09:53:12.811414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:35.714783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:53:25.336 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:53:25.337 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T09:53:25.337 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T09:53:25.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T09:53:25.337 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T09:53:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.470+0000 7f0aab86b640 1 Processor -- start 2024-04-25T09:53:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.470+0000 7f0aab86b640 1 -- start start 2024-04-25T09:53:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.471+0000 7f0aab86b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0aa40d1120 0x7f0aa409a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.471+0000 7f0aab86b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa406c220 con 0x7f0aa406ce10 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.471+0000 7f0aab86b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa40d0080 con 0x7f0aa415dc70 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.471+0000 7f0aab86b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa40d0c90 con 0x7f0aa40d1120 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.471+0000 7f0aa8e06640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0aa406ce10 0x7f0aa40cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46864/0 (socket says 172.21.15.19:46864) 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.471+0000 7f0aa8e06640 1 -- 172.21.15.19:0/182560811 learned_addr learned my addr 172.21.15.19:0/182560811 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0aa9e08640 1 --2- 172.21.15.19:0/182560811 >> v2:172.21.15.112:3300/0 conn(0x7f0aa40d1120 0x7f0aa409a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149475883 0 0) 0x7f0aa406c220 con 0x7f0aa406ce10 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a90003520 con 0x7f0aa406ce10 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 766537038 0 0) 0x7f0a90003520 con 0x7f0aa406ce10 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 >> v2:172.21.15.112:3300/0 conn(0x7f0aa40d1120 msgr2=0x7f0aa409a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 --2- 172.21.15.19:0/182560811 >> v2:172.21.15.112:3300/0 conn(0x7f0aa40d1120 0x7f0aa409a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 >> v1:172.21.15.112:6789/0 conn(0x7f0aa415dc70 legacy=0x7f0aa406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa40d0360 con 0x7f0aa406ce10 2024-04-25T09:53:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0a94001670 con 0x7f0aa406ce10 2024-04-25T09:53:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 >> v1:172.21.15.19:6789/0 conn(0x7f0aa406ce10 legacy=0x7f0aa40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 --2- 172.21.15.19:0/182560811 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a900039b0 0x7f0a90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 --2- 172.21.15.19:0/182560811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 0x7f0a90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 --2- 172.21.15.19:0/182560811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a900051c0 0x7f0a9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa40d0c90 con 0x7f0a90004800 2024-04-25T09:53:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9000c060 con 0x7f0a900051c0 2024-04-25T09:53:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9000c260 con 0x7f0a900039b0 2024-04-25T09:53:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0aa8e06640 1 --2- 172.21.15.19:0/182560811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 0x7f0a90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0aa9607640 1 --2- 172.21.15.19:0/182560811 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a900039b0 0x7f0a90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0aa9e08640 1 --2- 172.21.15.19:0/182560811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a900051c0 0x7f0a9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0aa8e06640 1 -- 172.21.15.19:0/182560811 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a900039b0 msgr2=0x7f0a90003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0aa8e06640 1 --2- 172.21.15.19:0/182560811 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a900039b0 0x7f0a90003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0aa8e06640 1 -- 172.21.15.19:0/182560811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a900051c0 msgr2=0x7f0a9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0aa8e06640 1 --2- 172.21.15.19:0/182560811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a900051c0 0x7f0a9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0aa9e08640 1 -- 172.21.15.19:0/182560811 reap_dead start 2024-04-25T09:53:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.472+0000 7f0aa8e06640 1 -- 172.21.15.19:0/182560811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a90003520 con 0x7f0a90004800 2024-04-25T09:53:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.473+0000 7f0aa8e06640 1 --2- 172.21.15.19:0/182560811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 0x7f0a90004c80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c240 tx=0x7f0a90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0e7b8171666818 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.473+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94007d10 con 0x7f0a90004800 2024-04-25T09:53:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.473+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a94005ad0 con 0x7f0a90004800 2024-04-25T09:53:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.473+0000 7f0a9bfff640 1 -- 172.21.15.19:0/182560811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94007070 con 0x7f0a90004800 2024-04-25T09:53:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.474+0000 7f0aab86b640 1 -- 172.21.15.19:0/182560811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 msgr2=0x7f0a90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.474+0000 7f0aab86b640 1 --2- 172.21.15.19:0/182560811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 0x7f0a90004c80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c240 tx=0x7f0a90005830 comp rx=0 tx=0).stop 2024-04-25T09:53:25.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.474+0000 7f0aab86b640 1 -- 172.21.15.19:0/182560811 shutdown_connections 2024-04-25T09:53:25.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.474+0000 7f0aab86b640 1 --2- 172.21.15.19:0/182560811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 0x7f0a90004c80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.474+0000 7f0aab86b640 1 -- 172.21.15.19:0/182560811 >> 172.21.15.19:0/182560811 conn(0x7f0aa40b3270 msgr2=0x7f0aa40b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.474+0000 7f0aab86b640 1 -- 172.21.15.19:0/182560811 shutdown_connections 2024-04-25T09:53:25.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.474+0000 7f0aab86b640 1 -- 172.21.15.19:0/182560811 wait complete. 2024-04-25T09:53:25.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.475+0000 7f0aab86b640 1 Processor -- start 2024-04-25T09:53:25.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.475+0000 7f0aab86b640 1 -- start start 2024-04-25T09:53:25.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aab86b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 0x7f0aa414ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aab86b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0aa415e1d0 0x7f0aa41f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aab86b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa41f42c0 0x7f0aa41f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aab86b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa415e050 con 0x7f0a90004800 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aab86b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa40d0920 con 0x7f0aa41f42c0 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aab86b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa409a450 con 0x7f0aa415e1d0 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aa9607640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 0x7f0aa414ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aa9607640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 0x7f0aa414ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35688/0 (socket says 172.21.15.19:35688) 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aa9607640 1 -- 172.21.15.19:0/1354365784 learned_addr learned my addr 172.21.15.19:0/1354365784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aa8e06640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0aa415e1d0 0x7f0aa41f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aa9e08640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa41f42c0 0x7f0aa41f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aa9607640 1 -- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0aa415e1d0 msgr2=0x7f0aa41f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aa9607640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0aa415e1d0 0x7f0aa41f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aa9607640 1 -- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa41f42c0 msgr2=0x7f0aa41f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aa9607640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa41f42c0 0x7f0aa41f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aa9607640 1 -- 172.21.15.19:0/1354365784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa41f4c40 con 0x7f0a90004800 2024-04-25T09:53:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0aa9607640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 0x7f0aa414ed80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00f5f0 tx=0x7f0a9c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.476+0000 7f0a99ffb640 1 -- 172.21.15.19:0/1354365784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0052e0 con 0x7f0a90004800 2024-04-25T09:53:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.477+0000 7f0a99ffb640 1 -- 172.21.15.19:0/1354365784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c018070 con 0x7f0a90004800 2024-04-25T09:53:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.477+0000 7f0aab86b640 1 -- 172.21.15.19:0/1354365784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa414f2c0 con 0x7f0a90004800 2024-04-25T09:53:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.477+0000 7f0a99ffb640 1 -- 172.21.15.19:0/1354365784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c003810 con 0x7f0a90004800 2024-04-25T09:53:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.477+0000 7f0aab86b640 1 -- 172.21.15.19:0/1354365784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa4160ec0 con 0x7f0a90004800 2024-04-25T09:53:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.478+0000 7f0aab86b640 1 -- 172.21.15.19:0/1354365784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa4073990 con 0x7f0a90004800 2024-04-25T09:53:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.478+0000 7f0a99ffb640 1 -- 172.21.15.19:0/1354365784 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a9c0039b0 con 0x7f0a90004800 2024-04-25T09:53:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.479+0000 7f0a99ffb640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a8806b4d0 0x7f0a8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.479+0000 7f0a99ffb640 1 -- 172.21.15.19:0/1354365784 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0a9c08a3e0 con 0x7f0a90004800 2024-04-25T09:53:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.479+0000 7f0aa8e06640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a8806b4d0 0x7f0a8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.479+0000 7f0aa8e06640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a8806b4d0 0x7f0a8806d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a94032000 tx=0x7f0a94034040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.483+0000 7f0a99ffb640 1 -- 172.21.15.19:0/1354365784 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0aa414f2c0 con 0x7f0a90004800 2024-04-25T09:53:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.711+0000 7f0aab86b640 1 -- 172.21.15.19:0/1354365784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0aa40cb380 con 0x7f0a90004800 2024-04-25T09:53:25.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.714+0000 7f0a99ffb640 1 -- 172.21.15.19:0/1354365784 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171792 (secure 0 0 0) 0x7f0aa41f4c40 con 0x7f0a90004800 2024-04-25T09:53:25.718 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 -- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a8806b4d0 msgr2=0x7f0a8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a8806b4d0 0x7f0a8806d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a94032000 tx=0x7f0a94034040 comp rx=0 tx=0).stop 2024-04-25T09:53:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 -- 172.21.15.19:0/1354365784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 msgr2=0x7f0aa414ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 0x7f0aa414ed80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00f5f0 tx=0x7f0a9c00fab0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 -- 172.21.15.19:0/1354365784 shutdown_connections 2024-04-25T09:53:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a8806b4d0 0x7f0a8806d990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa41f42c0 0x7f0aa41f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0aa415e1d0 0x7f0aa41f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 --2- 172.21.15.19:0/1354365784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a90004800 0x7f0aa414ed80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 -- 172.21.15.19:0/1354365784 >> 172.21.15.19:0/1354365784 conn(0x7f0aa40b3270 msgr2=0x7f0aa40caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 -- 172.21.15.19:0/1354365784 shutdown_connections 2024-04-25T09:53:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.716+0000 7f0aab86b640 1 -- 172.21.15.19:0/1354365784 wait complete. 2024-04-25T09:53:25.729 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2701671132},{"type":"v1","addr":"172.21.15.112:6833","nonce":2701671132}]},"active_addr":"172.21.15.112:6833/2701671132","active_change":"2024-04-25T09:52:46.898894+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":4183136298}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2808572723}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3518941397}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2924796684}]}]} 2024-04-25T09:53:25.732 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T09:53:25.732 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T09:53:25.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:25.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.831+0000 7fc20e9ce640 1 Processor -- start 2024-04-25T09:53:25.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.831+0000 7fc20e9ce640 1 -- start start 2024-04-25T09:53:25.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.832+0000 7fc20e9ce640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc2080b5c30 0x7fc208168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.832+0000 7fc20e9ce640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2080ceee0 con 0x7fc2080b6cb0 2024-04-25T09:53:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.832+0000 7fc20e9ce640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2080d1d10 con 0x7fc20815db70 2024-04-25T09:53:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.832+0000 7fc20e9ce640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2080d3800 con 0x7fc2080b5c30 2024-04-25T09:53:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.832+0000 7fc20cf6b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc2080b5c30 0x7fc208168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.832+0000 7fc2077fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc2080b6cb0 0x7fc2080b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46876/0 (socket says 172.21.15.19:46876) 2024-04-25T09:53:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.832+0000 7fc2077fe640 1 -- 172.21.15.19:0/3779983849 learned_addr learned my addr 172.21.15.19:0/3779983849 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.833+0000 7fc20cf6b640 1 -- 172.21.15.19:0/3779983849 >> v1:172.21.15.112:6789/0 conn(0x7fc20815db70 legacy=0x7fc208162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.833+0000 7fc20cf6b640 1 -- 172.21.15.19:0/3779983849 >> v1:172.21.15.19:6789/0 conn(0x7fc2080b6cb0 legacy=0x7fc2080b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.833+0000 7fc20cf6b640 1 -- 172.21.15.19:0/3779983849 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2080d1ff0 con 0x7fc2080b5c30 2024-04-25T09:53:25.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.833+0000 7fc206ffd640 1 -- 172.21.15.19:0/3779983849 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912038358 0 0) 0x7fc2080ceee0 con 0x7fc2080b6cb0 2024-04-25T09:53:25.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.833+0000 7fc20cf6b640 1 --2- 172.21.15.19:0/3779983849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2080b5c30 0x7fc208168ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc2080d3980 tx=0x7fc20002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdce1f55f7d6481 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.834+0000 7fc206ffd640 1 -- 172.21.15.19:0/3779983849 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200007740 con 0x7fc2080b5c30 2024-04-25T09:53:25.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.834+0000 7fc206ffd640 1 -- 172.21.15.19:0/3779983849 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2000078e0 con 0x7fc2080b5c30 2024-04-25T09:53:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.834+0000 7fc206ffd640 1 -- 172.21.15.19:0/3779983849 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200007cb0 con 0x7fc2080b5c30 2024-04-25T09:53:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.834+0000 7fc20e9ce640 1 -- 172.21.15.19:0/3779983849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2080b5c30 msgr2=0x7fc208168ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.834+0000 7fc20e9ce640 1 --2- 172.21.15.19:0/3779983849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2080b5c30 0x7fc208168ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc2080d3980 tx=0x7fc20002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.834+0000 7fc20e9ce640 1 -- 172.21.15.19:0/3779983849 shutdown_connections 2024-04-25T09:53:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.834+0000 7fc20e9ce640 1 --2- 172.21.15.19:0/3779983849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2080b5c30 0x7fc208168ad0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.834+0000 7fc20e9ce640 1 -- 172.21.15.19:0/3779983849 >> 172.21.15.19:0/3779983849 conn(0x7fc2080d0520 msgr2=0x7fc2080d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.834+0000 7fc20e9ce640 1 -- 172.21.15.19:0/3779983849 shutdown_connections 2024-04-25T09:53:25.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.834+0000 7fc20e9ce640 1 -- 172.21.15.19:0/3779983849 wait complete. 2024-04-25T09:53:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.835+0000 7fc20e9ce640 1 Processor -- start 2024-04-25T09:53:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.836+0000 7fc20e9ce640 1 -- start start 2024-04-25T09:53:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.836+0000 7fc20e9ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2080b5c30 0x7fc2080c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.836+0000 7fc20e9ce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2080b6cb0 0x7fc20815e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.836+0000 7fc20e9ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc20815db70 0x7fc208158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.836+0000 7fc20e9ce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080ce530 con 0x7fc2080b6cb0 2024-04-25T09:53:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.836+0000 7fc20e9ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20815df50 con 0x7fc2080b5c30 2024-04-25T09:53:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.836+0000 7fc20e9ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080cf1c0 con 0x7fc20815db70 2024-04-25T09:53:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.836+0000 7fc2077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2080b6cb0 0x7fc20815e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.836+0000 7fc2077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2080b6cb0 0x7fc20815e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35702/0 (socket says 172.21.15.19:35702) 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.836+0000 7fc2077fe640 1 -- 172.21.15.19:0/1249910417 learned_addr learned my addr 172.21.15.19:0/1249910417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc20cf6b640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc20815db70 0x7fc208158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc207fff640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2080b5c30 0x7fc2080c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc2077fe640 1 -- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc20815db70 msgr2=0x7fc208158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc2077fe640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc20815db70 0x7fc208158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc2077fe640 1 -- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2080b5c30 msgr2=0x7fc2080c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc2077fe640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2080b5c30 0x7fc2080c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc2077fe640 1 -- 172.21.15.19:0/1249910417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2081572d0 con 0x7fc2080b6cb0 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc207fff640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2080b5c30 0x7fc2080c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc2077fe640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2080b6cb0 0x7fc20815e160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c490 tx=0x7fc1fc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc204ff9640 1 -- 172.21.15.19:0/1249910417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc018070 con 0x7fc2080b6cb0 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc204ff9640 1 -- 172.21.15.19:0/1249910417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1fc0074e0 con 0x7fc2080b6cb0 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc20e9ce640 1 -- 172.21.15.19:0/1249910417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2080cd2c0 con 0x7fc2080b6cb0 2024-04-25T09:53:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc204ff9640 1 -- 172.21.15.19:0/1249910417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc004a20 con 0x7fc2080b6cb0 2024-04-25T09:53:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.837+0000 7fc20e9ce640 1 -- 172.21.15.19:0/1249910417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2080cd880 con 0x7fc2080b6cb0 2024-04-25T09:53:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.838+0000 7fc20e9ce640 1 -- 172.21.15.19:0/1249910417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20806d100 con 0x7fc2080b6cb0 2024-04-25T09:53:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.839+0000 7fc204ff9640 1 -- 172.21.15.19:0/1249910417 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1fc01b070 con 0x7fc2080b6cb0 2024-04-25T09:53:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.839+0000 7fc204ff9640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc1f406b4d0 0x7fc1f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.839+0000 7fc204ff9640 1 -- 172.21.15.19:0/1249910417 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc1fc0896e0 con 0x7fc2080b6cb0 2024-04-25T09:53:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.839+0000 7fc207fff640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc1f406b4d0 0x7fc1f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.840+0000 7fc207fff640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc1f406b4d0 0x7fc1f406d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8007fd0 tx=0x7fc1f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:25.843+0000 7fc204ff9640 1 -- 172.21.15.19:0/1249910417 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc20806d100 con 0x7fc2080b6cb0 2024-04-25T09:53:26.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.024+0000 7fc20e9ce640 1 -- 172.21.15.19:0/1249910417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2080d1820 con 0x7fc2080b6cb0 2024-04-25T09:53:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.025+0000 7fc204ff9640 1 -- 172.21.15.19:0/1249910417 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fc1fc057bd0 con 0x7fc2080b6cb0 2024-04-25T09:53:26.027 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:26.027 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:26.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 -- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc1f406b4d0 msgr2=0x7fc1f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc1f406b4d0 0x7fc1f406d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8007fd0 tx=0x7fc1f8009040 comp rx=0 tx=0).stop 2024-04-25T09:53:26.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 -- 172.21.15.19:0/1249910417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2080b6cb0 msgr2=0x7fc20815e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2080b6cb0 0x7fc20815e160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c490 tx=0x7fc1fc00c950 comp rx=0 tx=0).stop 2024-04-25T09:53:26.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 -- 172.21.15.19:0/1249910417 shutdown_connections 2024-04-25T09:53:26.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc1f406b4d0 0x7fc1f406d990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc20815db70 0x7fc208158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2080b6cb0 0x7fc20815e160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 --2- 172.21.15.19:0/1249910417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2080b5c30 0x7fc2080c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 -- 172.21.15.19:0/1249910417 >> 172.21.15.19:0/1249910417 conn(0x7fc2080d0520 msgr2=0x7fc2080b90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 -- 172.21.15.19:0/1249910417 shutdown_connections 2024-04-25T09:53:26.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.027+0000 7fc20e9ce640 1 -- 172.21.15.19:0/1249910417 wait complete. 2024-04-25T09:53:26.040 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T09:53:26.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.140+0000 7fadd1529640 1 Processor -- start 2024-04-25T09:53:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.140+0000 7fadd1529640 1 -- start start 2024-04-25T09:53:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.141+0000 7fadd1529640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fadcc0cfec0 0x7fadcc0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.141+0000 7fadd1529640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 0x7fadcc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.141+0000 7fadd1529640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc071480 con 0x7fadcc154ee0 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.141+0000 7fadd1529640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc0ce830 con 0x7fadcc0cfec0 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.141+0000 7fadd1529640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc072090 con 0x7fadcc0ce3c0 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.142+0000 7fadcb7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 0x7fadcc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.142+0000 7fadca7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fadcc0cfec0 0x7fadcc0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.142+0000 7fadcb7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 0x7fadcc06d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55578/0 (socket says 172.21.15.19:55578) 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.142+0000 7fadca7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fadcc0cfec0 0x7fadcc0cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:51726/0 (socket says 172.21.15.19:51726) 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.142+0000 7fadcaffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fadcc154ee0 0x7fadcc0cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:43360/0 (socket says 172.21.15.19:43360) 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.142+0000 7fadcb7fe640 1 -- 172.21.15.19:0/3858390873 learned_addr learned my addr 172.21.15.19:0/3858390873 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.142+0000 7fadcb7fe640 1 -- 172.21.15.19:0/3858390873 >> v2:172.21.15.112:3300/0 conn(0x7fadcc0cfec0 msgr2=0x7fadcc0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.142+0000 7fadcb7fe640 1 --2- 172.21.15.19:0/3858390873 >> v2:172.21.15.112:3300/0 conn(0x7fadcc0cfec0 0x7fadcc0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.142+0000 7fadcb7fe640 1 -- 172.21.15.19:0/3858390873 >> v1:172.21.15.112:6789/0 conn(0x7fadcc154ee0 legacy=0x7fadcc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.142+0000 7fadcb7fe640 1 -- 172.21.15.19:0/3858390873 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc071760 con 0x7fadcc0ce3c0 2024-04-25T09:53:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.143+0000 7fadcb7fe640 1 --2- 172.21.15.19:0/3858390873 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 0x7fadcc06d6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009e50 tx=0x7fadc402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc4ee2800cb86b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.143+0000 7fadc9ffb640 1 -- 172.21.15.19:0/3858390873 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc4036070 con 0x7fadcc0ce3c0 2024-04-25T09:53:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.143+0000 7fadc9ffb640 1 -- 172.21.15.19:0/3858390873 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc4031600 con 0x7fadcc0ce3c0 2024-04-25T09:53:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.143+0000 7fadc9ffb640 1 -- 172.21.15.19:0/3858390873 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc40319d0 con 0x7fadcc0ce3c0 2024-04-25T09:53:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.144+0000 7fadd1529640 1 -- 172.21.15.19:0/3858390873 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 msgr2=0x7fadcc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.144+0000 7fadd1529640 1 --2- 172.21.15.19:0/3858390873 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 0x7fadcc06d6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009e50 tx=0x7fadc402a710 comp rx=0 tx=0).stop 2024-04-25T09:53:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.144+0000 7fadd1529640 1 -- 172.21.15.19:0/3858390873 shutdown_connections 2024-04-25T09:53:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.144+0000 7fadd1529640 1 --2- 172.21.15.19:0/3858390873 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 0x7fadcc06d6f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.144+0000 7fadd1529640 1 --2- 172.21.15.19:0/3858390873 >> v2:172.21.15.112:3300/0 conn(0x7fadcc0cfec0 0x7fadcc0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.144+0000 7fadd1529640 1 -- 172.21.15.19:0/3858390873 >> 172.21.15.19:0/3858390873 conn(0x7fadcc06abe0 msgr2=0x7fadcc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.144+0000 7fadd1529640 1 -- 172.21.15.19:0/3858390873 shutdown_connections 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.144+0000 7fadd1529640 1 -- 172.21.15.19:0/3858390873 wait complete. 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.145+0000 7fadd1529640 1 Processor -- start 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.145+0000 7fadd1529640 1 -- start start 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadd1529640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 0x7fadcc14c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadd1529640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc0cfec0 0x7fadcc15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadd1529640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadcc154ee0 0x7fadcc15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadd1529640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc071d20 con 0x7fadcc0cfec0 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadd1529640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc071230 con 0x7fadcc154ee0 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadd1529640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc1552c0 con 0x7fadcc0ce3c0 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadca7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc0cfec0 0x7fadcc15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadca7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc0cfec0 0x7fadcc15b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35712/0 (socket says 172.21.15.19:35712) 2024-04-25T09:53:26.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadca7fc640 1 -- 172.21.15.19:0/3179831796 learned_addr learned my addr 172.21.15.19:0/3179831796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadcb7fe640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadcc154ee0 0x7fadcc15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadcaffd640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 0x7fadcc14c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadca7fc640 1 -- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 msgr2=0x7fadcc14c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadca7fc640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 0x7fadcc14c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadca7fc640 1 -- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadcc154ee0 msgr2=0x7fadcc15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadca7fc640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadcc154ee0 0x7fadcc15bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.146+0000 7fadca7fc640 1 -- 172.21.15.19:0/3179831796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc1581b0 con 0x7fadcc0cfec0 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.147+0000 7fadca7fc640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc0cfec0 0x7fadcc15b7b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00e9e0 tx=0x7fadbc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.147+0000 7fadb37fe640 1 -- 172.21.15.19:0/3179831796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc017070 con 0x7fadcc0cfec0 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.147+0000 7fadb37fe640 1 -- 172.21.15.19:0/3179831796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc00c9d0 con 0x7fadcc0cfec0 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.147+0000 7fadd1529640 1 -- 172.21.15.19:0/3179831796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc158490 con 0x7fadcc0cfec0 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.147+0000 7fadb37fe640 1 -- 172.21.15.19:0/3179831796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00cdc0 con 0x7fadcc0cfec0 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.147+0000 7fadd1529640 1 -- 172.21.15.19:0/3179831796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc158a50 con 0x7fadcc0cfec0 2024-04-25T09:53:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.148+0000 7fadd1529640 1 -- 172.21.15.19:0/3179831796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc073990 con 0x7fadcc0cfec0 2024-04-25T09:53:26.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.148+0000 7fadb37fe640 1 -- 172.21.15.19:0/3179831796 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadbc01a070 con 0x7fadcc0cfec0 2024-04-25T09:53:26.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.149+0000 7fadb37fe640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fadb806b4d0 0x7fadb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.149+0000 7fadb37fe640 1 -- 172.21.15.19:0/3179831796 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fadbc08a980 con 0x7fadcc0cfec0 2024-04-25T09:53:26.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.149+0000 7fadcaffd640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fadb806b4d0 0x7fadb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.149+0000 7fadcaffd640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fadb806b4d0 0x7fadb806d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fadc0000f60 tx=0x7fadc0007ce0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.153+0000 7fadb37fe640 1 -- 172.21.15.19:0/3179831796 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fadbc0594c0 con 0x7fadcc0cfec0 2024-04-25T09:53:26.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.361+0000 7fadd1529640 1 -- 172.21.15.19:0/3179831796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadcc15a830 con 0x7fadcc0cfec0 2024-04-25T09:53:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.362+0000 7fadb37fe640 1 -- 172.21.15.19:0/3179831796 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fadbc058dc0 con 0x7fadcc0cfec0 2024-04-25T09:53:26.364 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:26.364 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 -- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fadb806b4d0 msgr2=0x7fadb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fadb806b4d0 0x7fadb806d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fadc0000f60 tx=0x7fadc0007ce0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 -- 172.21.15.19:0/3179831796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc0cfec0 msgr2=0x7fadcc15b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc0cfec0 0x7fadcc15b7b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00e9e0 tx=0x7fadbc00eea0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 -- 172.21.15.19:0/3179831796 shutdown_connections 2024-04-25T09:53:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fadb806b4d0 0x7fadb806d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadcc154ee0 0x7fadcc15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc0cfec0 0x7fadcc15b7b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 --2- 172.21.15.19:0/3179831796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fadcc0ce3c0 0x7fadcc14c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 -- 172.21.15.19:0/3179831796 >> 172.21.15.19:0/3179831796 conn(0x7fadcc06abe0 msgr2=0x7fadcc152ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 -- 172.21.15.19:0/3179831796 shutdown_connections 2024-04-25T09:53:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.364+0000 7fadd1529640 1 -- 172.21.15.19:0/3179831796 wait complete. 2024-04-25T09:53:26.378 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T09:53:26.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T09:53:26.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T09:53:26.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T09:53:26.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T09:53:26.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T09:53:26.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T09:53:26.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T09:53:26.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.562+0000 7f987ed34640 1 Processor -- start 2024-04-25T09:53:26.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.561+0000 7f0b90702640 1 Processor -- start 2024-04-25T09:53:26.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.561+0000 7f0b90702640 1 -- start start 2024-04-25T09:53:26.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f0b90702640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b880cd2e0 0x7f0b8816e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f0b90702640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b880cec60 con 0x7f0b8816aab0 2024-04-25T09:53:26.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f0b90702640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8816e9e0 con 0x7f0b88072220 2024-04-25T09:53:26.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f0b90702640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8806e810 con 0x7f0b880cd2e0 2024-04-25T09:53:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f0b8e49e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0b8816aab0 0x7f0b88071b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46880/0 (socket says 172.21.15.19:46880) 2024-04-25T09:53:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f0b8e49e640 1 -- 172.21.15.19:0/4236205134 learned_addr learned my addr 172.21.15.19:0/4236205134 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f0b8ec9f640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b880cd2e0 0x7f0b8816e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442303199 0 0) 0x7f0b880cec60 con 0x7f0b8816aab0 2024-04-25T09:53:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f967725a640 1 Processor -- start 2024-04-25T09:53:26.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f967725a640 1 -- start start 2024-04-25T09:53:26.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b74003520 con 0x7f0b8816aab0 2024-04-25T09:53:26.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348145217 0 0) 0x7f0b8816e9e0 con 0x7f0b88072220 2024-04-25T09:53:26.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b880cec60 con 0x7f0b88072220 2024-04-25T09:53:26.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3923497338 0 0) 0x7f0b74003520 con 0x7f0b8816aab0 2024-04-25T09:53:26.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b880cd2e0 msgr2=0x7f0b8816e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b880cd2e0 0x7f0b8816e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 >> v1:172.21.15.112:6789/0 conn(0x7f0b88072220 legacy=0x7f0b88173780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8816ecc0 con 0x7f0b8816aab0 2024-04-25T09:53:26.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0b800019f0 con 0x7f0b8816aab0 2024-04-25T09:53:26.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 >> v1:172.21.15.19:6789/0 conn(0x7f0b8816aab0 legacy=0x7f0b88071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b74003b70 0x7f0b74003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.562+0000 7f987ed34640 1 -- start start 2024-04-25T09:53:26.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f987ed34640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f987815e290 0x7f98780ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f987ed34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780aed70 0x7f987815a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f987ed34640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987815e740 con 0x7f9878157700 2024-04-25T09:53:26.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f987ed34640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98780af1e0 con 0x7f987815e290 2024-04-25T09:53:26.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.563+0000 7f987ed34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987815d840 con 0x7f98780aed70 2024-04-25T09:53:26.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f987cad0640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9878157700 0x7f98780cd2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:43378/0 (socket says 172.21.15.19:43378) 2024-04-25T09:53:26.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f987cad0640 1 -- 172.21.15.19:0/3275343690 learned_addr learned my addr 172.21.15.19:0/3275343690 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:26.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f9877fff640 1 --2- 172.21.15.19:0/3275343690 >> v2:172.21.15.19:3300/0 conn(0x7f987815e290 0x7f98780ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f98777fe640 1 -- 172.21.15.19:0/3275343690 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828364341 0 0) 0x7f987815e740 con 0x7f9878157700 2024-04-25T09:53:26.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f98777fe640 1 -- 172.21.15.19:0/3275343690 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9860003520 con 0x7f9878157700 2024-04-25T09:53:26.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f9877fff640 1 -- 172.21.15.19:0/3275343690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780aed70 msgr2=0x7f987815a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b740051c0 0x7f0b740141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8806e810 con 0x7f0b740051c0 2024-04-25T09:53:26.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b74005990 con 0x7f0b74003b70 2024-04-25T09:53:26.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b74005b90 con 0x7f0b74004800 2024-04-25T09:53:26.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8e49e640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b74003b70 0x7f0b74003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8ec9f640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b880cd2e0 0x7f0b8816e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:26.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.564+0000 7f0b8dc9d640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f98777fe640 1 -- 172.21.15.19:0/3275343690 <== mon.1 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3166589411 0 0) 0x7f9860003520 con 0x7f9878157700 2024-04-25T09:53:26.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f987d2d1640 1 --2- 172.21.15.19:0/3275343690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780aed70 0x7f987815a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f0b8e49e640 1 -- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b74004800 msgr2=0x7f0b74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f0b8e49e640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f0b8e49e640 1 -- 172.21.15.19:0/4236205134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b740051c0 msgr2=0x7f0b740141d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:53:26.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f0b8ec9f640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b740051c0 0x7f0b740141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f0b8e49e640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b740051c0 0x7f0b740141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f0b8e49e640 1 -- 172.21.15.19:0/4236205134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8816ecc0 con 0x7f0b74003b70 2024-04-25T09:53:26.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.566+0000 7f0b8e49e640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b74003b70 0x7f0b74003f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000a8b0 tx=0x7f0b74005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bd0e72130a9b4d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.566+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000a480 con 0x7f0b74003b70 2024-04-25T09:53:26.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.566+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8002f070 con 0x7f0b74003b70 2024-04-25T09:53:26.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.566+0000 7f0b8d49c640 1 -- 172.21.15.19:0/4236205134 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8002bd00 con 0x7f0b74003b70 2024-04-25T09:53:26.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.567+0000 7f0b8ec9f640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b740051c0 0x7f0b740141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:26.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.567+0000 7f0b90702640 1 -- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b74003b70 msgr2=0x7f0b74003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.567+0000 7f0b90702640 1 --2- 172.21.15.19:0/4236205134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b74003b70 0x7f0b74003f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000a8b0 tx=0x7f0b74005830 comp rx=0 tx=0).stop 2024-04-25T09:53:26.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.567+0000 7f0b8ec9f640 1 -- 172.21.15.19:0/4236205134 reap_dead start 2024-04-25T09:53:26.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.567+0000 7f0b90702640 1 -- 172.21.15.19:0/4236205134 shutdown_connections 2024-04-25T09:53:26.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.567+0000 7f0b90702640 1 -- 172.21.15.19:0/4236205134 >> 172.21.15.19:0/4236205134 conn(0x7f0b8806abe0 msgr2=0x7f0b8806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.567+0000 7f0b90702640 1 -- 172.21.15.19:0/4236205134 shutdown_connections 2024-04-25T09:53:26.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.568+0000 7f0b90702640 1 -- 172.21.15.19:0/4236205134 wait complete. 2024-04-25T09:53:26.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.568+0000 7f9877fff640 1 --2- 172.21.15.19:0/3275343690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780aed70 0x7f987815a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.568+0000 7f9877fff640 1 -- 172.21.15.19:0/3275343690 >> v1:172.21.15.112:6789/0 conn(0x7f9878157700 legacy=0x7f98780cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f967725a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9670161240 0x7f96700b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f967725a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96700b54a0 0x7f96700b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f967725a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96700cf1d0 con 0x7f9670161890 2024-04-25T09:53:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f967725a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96700b7120 con 0x7f96700b54a0 2024-04-25T09:53:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f967725a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96700b5e30 con 0x7f9670161240 2024-04-25T09:53:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f96757f7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96700b54a0 0x7f96700b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f96757f7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96700b54a0 0x7f96700b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35732/0 (socket says 172.21.15.19:35732) 2024-04-25T09:53:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.565+0000 7f96757f7640 1 -- 172.21.15.19:0/4243619716 learned_addr learned my addr 172.21.15.19:0/4243619716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.566+0000 7f966f7fe640 1 -- 172.21.15.19:0/4243619716 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103565913 0 0) 0x7f96700cf1d0 con 0x7f9670161890 2024-04-25T09:53:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.566+0000 7f966f7fe640 1 -- 172.21.15.19:0/4243619716 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9658003520 con 0x7f9670161890 2024-04-25T09:53:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.566+0000 7f96757f7640 1 -- 172.21.15.19:0/4243619716 >> v2:172.21.15.112:3300/0 conn(0x7f9670161240 msgr2=0x7f96700b6be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:53:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.566+0000 7f96757f7640 1 --2- 172.21.15.19:0/4243619716 >> v2:172.21.15.112:3300/0 conn(0x7f9670161240 0x7f96700b6be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.566+0000 7f96757f7640 1 -- 172.21.15.19:0/4243619716 >> v1:172.21.15.112:6789/0 conn(0x7f9670161890 legacy=0x7f96700ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.566+0000 7f96757f7640 1 -- 172.21.15.19:0/4243619716 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9670161620 con 0x7f96700b54a0 2024-04-25T09:53:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.567+0000 7f96757f7640 1 --2- 172.21.15.19:0/4243619716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96700b54a0 0x7f96700b58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f96700d0400 tx=0x7f966402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1f89ead8b28613 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.567+0000 7f966f7fe640 1 -- 172.21.15.19:0/4243619716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664037070 con 0x7f96700b54a0 2024-04-25T09:53:26.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.567+0000 7f966f7fe640 1 -- 172.21.15.19:0/4243619716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9664032440 con 0x7f96700b54a0 2024-04-25T09:53:26.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.568+0000 7f966f7fe640 1 -- 172.21.15.19:0/4243619716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96640327f0 con 0x7f96700b54a0 2024-04-25T09:53:26.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.568+0000 7f967725a640 1 -- 172.21.15.19:0/4243619716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96700b54a0 msgr2=0x7f96700b58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.568+0000 7f967725a640 1 --2- 172.21.15.19:0/4243619716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96700b54a0 0x7f96700b58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f96700d0400 tx=0x7f966402a290 comp rx=0 tx=0).stop 2024-04-25T09:53:26.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.568+0000 7f9877fff640 1 -- 172.21.15.19:0/3275343690 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9878071c50 con 0x7f987815e290 2024-04-25T09:53:26.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.569+0000 7f987d2d1640 1 --2- 172.21.15.19:0/3275343690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780aed70 0x7f987815a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:26.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.569+0000 7f967725a640 1 -- 172.21.15.19:0/4243619716 shutdown_connections 2024-04-25T09:53:26.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.569+0000 7f967725a640 1 --2- 172.21.15.19:0/4243619716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96700b54a0 0x7f96700b58f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.569+0000 7f967725a640 1 --2- 172.21.15.19:0/4243619716 >> v2:172.21.15.112:3300/0 conn(0x7f9670161240 0x7f96700b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.569+0000 7f967725a640 1 -- 172.21.15.19:0/4243619716 >> 172.21.15.19:0/4243619716 conn(0x7f96700d04a0 msgr2=0x7f96700d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.569+0000 7f967725a640 1 -- 172.21.15.19:0/4243619716 shutdown_connections 2024-04-25T09:53:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.569+0000 7f9877fff640 1 --2- 172.21.15.19:0/3275343690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987815e290 0x7f98780ae830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98680097d0 tx=0x7f986802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827096e2d1c3d650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.569+0000 7f967725a640 1 -- 172.21.15.19:0/4243619716 wait complete. 2024-04-25T09:53:26.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.570+0000 7f0b90702640 1 Processor -- start 2024-04-25T09:53:26.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.570+0000 7f0b90702640 1 -- start start 2024-04-25T09:53:26.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7f0b90702640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b74003b70 0x7f0b8815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7f0b90702640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b880c9600 0x7f0b880c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7f0b90702640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b880cb660 0x7f0b880cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7f0b90702640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8816e400 con 0x7f0b880cb660 2024-04-25T09:53:26.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7f0b90702640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8816f280 con 0x7f0b880c9600 2024-04-25T09:53:26.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7f0b90702640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8816e580 con 0x7f0b74003b70 2024-04-25T09:53:26.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7f967725a640 1 Processor -- start 2024-04-25T09:53:26.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7f0b8e49e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b74003b70 0x7f0b8815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7f0b8dc9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b880c9600 0x7f0b880c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7f0b8e49e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b74003b70 0x7f0b8815bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55602/0 (socket says 172.21.15.19:55602) 2024-04-25T09:53:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7f0b8e49e640 1 -- 172.21.15.19:0/3921829290 learned_addr learned my addr 172.21.15.19:0/3921829290 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7f967725a640 1 -- start start 2024-04-25T09:53:26.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7f0b8ec9f640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b880cb660 0x7f0b880cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7f98777fe640 1 -- 172.21.15.19:0/3275343690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868037070 con 0x7f987815e290 2024-04-25T09:53:26.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7f98777fe640 1 -- 172.21.15.19:0/3275343690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986802abd0 con 0x7f987815e290 2024-04-25T09:53:26.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7f98777fe640 1 -- 172.21.15.19:0/3275343690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868037070 con 0x7f987815e290 2024-04-25T09:53:26.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f0b8ec9f640 1 -- 172.21.15.19:0/3921829290 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b74003b70 msgr2=0x7f0b8815bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f0b8ec9f640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b74003b70 0x7f0b8815bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f0b8ec9f640 1 -- 172.21.15.19:0/3921829290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b880c9600 msgr2=0x7f0b880c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f0b8ec9f640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b880c9600 0x7f0b880c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f0b8ec9f640 1 -- 172.21.15.19:0/3921829290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b880cc370 con 0x7f0b880cb660 2024-04-25T09:53:26.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f987ed34640 1 -- 172.21.15.19:0/3275343690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987815e290 msgr2=0x7f98780ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f987ed34640 1 --2- 172.21.15.19:0/3275343690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987815e290 0x7f98780ae830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98680097d0 tx=0x7f986802a140 comp rx=0 tx=0).stop 2024-04-25T09:53:26.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f987ed34640 1 -- 172.21.15.19:0/3275343690 shutdown_connections 2024-04-25T09:53:26.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f987ed34640 1 --2- 172.21.15.19:0/3275343690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780aed70 0x7f987815a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f987ed34640 1 --2- 172.21.15.19:0/3275343690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987815e290 0x7f98780ae830 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98680097d0 tx=0x7f986802a140 comp rx=0 tx=0).stop 2024-04-25T09:53:26.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f987ed34640 1 -- 172.21.15.19:0/3275343690 >> 172.21.15.19:0/3275343690 conn(0x7f987806abe0 msgr2=0x7f987806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.570+0000 7fb99815c640 1 Processor -- start 2024-04-25T09:53:26.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.570+0000 7fb99815c640 1 -- start start 2024-04-25T09:53:26.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7fb99815c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9880df210 0x7fb988038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7fb99815c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb988053c60 0x7fb98804fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7fb99815c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb988033ee0 0x7fb98804ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7fb99815c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9880544b0 con 0x7fb988053c60 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7fb99815c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9880541d0 con 0x7fb9880df210 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.571+0000 7fb99815c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb988005f20 con 0x7fb988033ee0 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7fb9956f7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb988053c60 0x7fb98804fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7fb9956f7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb988053c60 0x7fb98804fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35738/0 (socket says 172.21.15.19:35738) 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7fb9956f7640 1 -- 172.21.15.19:0/1334760739 learned_addr learned my addr 172.21.15.19:0/1334760739 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7fb9966f9640 1 --2- 172.21.15.19:0/1334760739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb988033ee0 0x7fb98804ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7fb995ef8640 1 --2- 172.21.15.19:0/1334760739 >> v2:172.21.15.112:3300/0 conn(0x7fb9880df210 0x7fb988038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7fb9956f7640 1 -- 172.21.15.19:0/1334760739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb988033ee0 msgr2=0x7fb98804ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7fb9956f7640 1 --2- 172.21.15.19:0/1334760739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb988033ee0 0x7fb98804ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.572+0000 7fb9956f7640 1 -- 172.21.15.19:0/1334760739 >> v2:172.21.15.112:3300/0 conn(0x7fb9880df210 msgr2=0x7fb988038070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7fb9956f7640 1 --2- 172.21.15.19:0/1334760739 >> v2:172.21.15.112:3300/0 conn(0x7fb9880df210 0x7fb988038070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7fb9956f7640 1 -- 172.21.15.19:0/1334760739 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb988006200 con 0x7fb988053c60 2024-04-25T09:53:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f967725a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96700b54a0 0x7f96700c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f967725a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9670161240 0x7f96700cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f967725a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9670161890 0x7f96701fb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f967725a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96701623c0 con 0x7f9670161890 2024-04-25T09:53:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f967725a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96701620e0 con 0x7f96700b54a0 2024-04-25T09:53:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f967725a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96701626a0 con 0x7f9670161240 2024-04-25T09:53:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f966ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9670161240 0x7f96700cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f966ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9670161240 0x7f96700cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55608/0 (socket says 172.21.15.19:55608) 2024-04-25T09:53:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f966ffff640 1 -- 172.21.15.19:0/3729175847 learned_addr learned my addr 172.21.15.19:0/3729175847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f9674ff6640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96700b54a0 0x7f96700c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f0b8ec9f640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b880cb660 0x7f0b880cbae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0b880d07b0 tx=0x7f0b7800fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.573+0000 7f96757f7640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9670161890 0x7f96701fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f966ffff640 1 -- 172.21.15.19:0/3729175847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96700b54a0 msgr2=0x7f96700c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f966ffff640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96700b54a0 0x7f96700c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f966ffff640 1 -- 172.21.15.19:0/3729175847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9670161890 msgr2=0x7f96701fb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f966ffff640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9670161890 0x7f96701fb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f966ffff640 1 -- 172.21.15.19:0/3729175847 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96700c9d20 con 0x7f9670161240 2024-04-25T09:53:26.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f0b7effd640 1 -- 172.21.15.19:0/3921829290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800d690 con 0x7f0b880cb660 2024-04-25T09:53:26.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f0b90702640 1 -- 172.21.15.19:0/3921829290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b880c6e40 con 0x7f0b880cb660 2024-04-25T09:53:26.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f0b90702640 1 -- 172.21.15.19:0/3921829290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b88200670 con 0x7f0b880cb660 2024-04-25T09:53:26.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f0b7effd640 1 -- 172.21.15.19:0/3921829290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7801b070 con 0x7f0b880cb660 2024-04-25T09:53:26.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f0b7effd640 1 -- 172.21.15.19:0/3921829290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800da90 con 0x7f0b880cb660 2024-04-25T09:53:26.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f987ed34640 1 -- 172.21.15.19:0/3275343690 shutdown_connections 2024-04-25T09:53:26.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7fb9956f7640 1 --2- 172.21.15.19:0/1334760739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb988053c60 0x7fb98804fa80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb98800b090 tx=0x7fb98c04c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd083f0edca9b66d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f96757f7640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9670161890 0x7f96701fb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f966ffff640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9670161240 0x7f96700cd1a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f966000c460 tx=0x7f966000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.575+0000 7f966d7fa640 1 -- 172.21.15.19:0/3729175847 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96600052d0 con 0x7f9670161240 2024-04-25T09:53:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.575+0000 7f967725a640 1 -- 172.21.15.19:0/3729175847 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96700ca000 con 0x7f9670161240 2024-04-25T09:53:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.575+0000 7f967725a640 1 -- 172.21.15.19:0/3729175847 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96700ca2e0 con 0x7f9670161240 2024-04-25T09:53:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.575+0000 7f966d7fa640 1 -- 172.21.15.19:0/3729175847 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9660005470 con 0x7f9670161240 2024-04-25T09:53:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.575+0000 7f966d7fa640 1 -- 172.21.15.19:0/3729175847 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660012930 con 0x7f9670161240 2024-04-25T09:53:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.575+0000 7f9836fe7640 1 Processor -- start 2024-04-25T09:53:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.576+0000 7f0b90702640 1 -- 172.21.15.19:0/3921829290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b70001030 con 0x7f0b880cb660 2024-04-25T09:53:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.577+0000 7f967725a640 1 -- 172.21.15.19:0/3729175847 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9650001030 con 0x7f9670161240 2024-04-25T09:53:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.576+0000 7f9836fe7640 1 -- start start 2024-04-25T09:53:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.577+0000 7f9836fe7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f983006bbf0 0x7f983006bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.577+0000 7f9836fe7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98300acd80 0x7f98300c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.577+0000 7f9836fe7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98300cda20 0x7f98300cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.577+0000 7f9836fe7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98300c5ac0 con 0x7f98300cda20 2024-04-25T09:53:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.577+0000 7f9836fe7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983006c510 con 0x7f983006bbf0 2024-04-25T09:53:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.577+0000 7f9836fe7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98300c4eb0 con 0x7f98300acd80 2024-04-25T09:53:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.577+0000 7fcf04cf5640 1 Processor -- start 2024-04-25T09:53:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.577+0000 7fcf04cf5640 1 -- start start 2024-04-25T09:53:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.577+0000 7f982ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98300acd80 0x7f98300c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f9835584640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98300cda20 0x7f98300cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f982ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98300acd80 0x7f98300c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55620/0 (socket says 172.21.15.19:55620) 2024-04-25T09:53:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f982ffff640 1 -- 172.21.15.19:0/3364320600 learned_addr learned my addr 172.21.15.19:0/3364320600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f9834d83640 1 --2- 172.21.15.19:0/3364320600 >> v2:172.21.15.112:3300/0 conn(0x7f983006bbf0 0x7f983006bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7fb994ef6640 1 -- 172.21.15.19:0/1334760739 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c059070 con 0x7fb988053c60 2024-04-25T09:53:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7fb994ef6640 1 -- 172.21.15.19:0/1334760739 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c054440 con 0x7fb988053c60 2024-04-25T09:53:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7fb994ef6640 1 -- 172.21.15.19:0/1334760739 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c054850 con 0x7fb988053c60 2024-04-25T09:53:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.574+0000 7f987ed34640 1 -- 172.21.15.19:0/3275343690 wait complete. 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.576+0000 7f987ed34640 1 Processor -- start 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.576+0000 7f987ed34640 1 -- start start 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f987ed34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98780aed70 0x7f98781493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f987ed34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9878157700 0x7f98780b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f987ed34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780b3270 0x7f987813f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f987ed34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987815dd70 con 0x7f9878157700 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f987ed34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987815da90 con 0x7f98780aed70 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f987ed34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98780718e0 con 0x7f98780b3270 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f987d2d1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780b3270 0x7f987813f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f987d2d1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780b3270 0x7f987813f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55634/0 (socket says 172.21.15.19:55634) 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f987d2d1640 1 -- 172.21.15.19:0/4016353824 learned_addr learned my addr 172.21.15.19:0/4016353824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f987cad0640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98780aed70 0x7f98781493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f987d2d1640 1 -- 172.21.15.19:0/4016353824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98780aed70 msgr2=0x7f98781493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f987d2d1640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98780aed70 0x7f98781493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f987d2d1640 1 -- 172.21.15.19:0/4016353824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9878157700 msgr2=0x7f98780b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f987d2d1640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9878157700 0x7f98780b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f987d2d1640 1 -- 172.21.15.19:0/4016353824 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9878154e00 con 0x7f98780b3270 2024-04-25T09:53:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7fcf04cf5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf000cc6e0 0x7fcf00159b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fcf04cf5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf000716b0 0x7fcf001646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fcf04cf5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf000b3b80 con 0x7fcf0006f090 2024-04-25T09:53:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fcf04cf5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0006d9d0 con 0x7fcf000716b0 2024-04-25T09:53:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fcf04cf5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0006d0a0 con 0x7fcf000cc6e0 2024-04-25T09:53:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fcefdd9b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcf0006f090 0x7fcf00070fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:43394/0 (socket says 172.21.15.19:43394) 2024-04-25T09:53:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fcefdd9b640 1 -- 172.21.15.19:0/815110133 learned_addr learned my addr 172.21.15.19:0/815110133 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fcefe59c640 1 --2- 172.21.15.19:0/815110133 >> v2:172.21.15.112:3300/0 conn(0x7fcf000cc6e0 0x7fcf00159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f987d2d1640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780b3270 0x7f987813f840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c820 tx=0x7f986c00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f982ffff640 1 -- 172.21.15.19:0/3364320600 >> v2:172.21.15.112:3300/0 conn(0x7f983006bbf0 msgr2=0x7f983006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fb99815c640 1 -- 172.21.15.19:0/1334760739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb988053c60 msgr2=0x7fb98804fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fb99815c640 1 --2- 172.21.15.19:0/1334760739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb988053c60 0x7fb98804fa80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb98800b090 tx=0x7fb98c04c190 comp rx=0 tx=0).stop 2024-04-25T09:53:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f98757fa640 1 -- 172.21.15.19:0/4016353824 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c015070 con 0x7f98780b3270 2024-04-25T09:53:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f98757fa640 1 -- 172.21.15.19:0/4016353824 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c007a30 con 0x7f98780b3270 2024-04-25T09:53:26.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f98757fa640 1 -- 172.21.15.19:0/4016353824 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c007e20 con 0x7f98780b3270 2024-04-25T09:53:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fb99815c640 1 -- 172.21.15.19:0/1334760739 shutdown_connections 2024-04-25T09:53:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fb99815c640 1 --2- 172.21.15.19:0/1334760739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb988033ee0 0x7fb98804ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fb99815c640 1 --2- 172.21.15.19:0/1334760739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb988053c60 0x7fb98804fa80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fb99815c640 1 --2- 172.21.15.19:0/1334760739 >> v2:172.21.15.112:3300/0 conn(0x7fb9880df210 0x7fb988038070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7fb99815c640 1 -- 172.21.15.19:0/1334760739 >> 172.21.15.19:0/1334760739 conn(0x7fb988009fa0 msgr2=0x7fb98800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f966d7fa640 1 -- 172.21.15.19:0/3729175847 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9660012ad0 con 0x7f9670161240 2024-04-25T09:53:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f966d7fa640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965c06b5a0 0x7f965c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.579+0000 7f966d7fa640 1 -- 172.21.15.19:0/3729175847 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f966008a070 con 0x7f9670161240 2024-04-25T09:53:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7f966d7fa640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] conn(0x7f965c074ca0 0x7f965c0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7f966d7fa640 1 -- 172.21.15.19:0/3729175847 --> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f965c0777b0 con 0x7f965c074ca0 2024-04-25T09:53:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7f966d7fa640 1 -- 172.21.15.19:0/3729175847 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9660091050 con 0x7f9670161240 2024-04-25T09:53:26.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f982ffff640 1 --2- 172.21.15.19:0/3364320600 >> v2:172.21.15.112:3300/0 conn(0x7f983006bbf0 0x7f983006bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f982ffff640 1 -- 172.21.15.19:0/3364320600 >> v2:172.21.15.19:3300/0 conn(0x7f98300cda20 msgr2=0x7f98300cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f982ffff640 1 --2- 172.21.15.19:0/3364320600 >> v2:172.21.15.19:3300/0 conn(0x7f98300cda20 0x7f98300cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.578+0000 7f982ffff640 1 -- 172.21.15.19:0/3364320600 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98300c5190 con 0x7f98300acd80 2024-04-25T09:53:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7f982ffff640 1 --2- 172.21.15.19:0/3364320600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98300acd80 0x7f98300c5fc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f983006c7d0 tx=0x7f982002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c705e12a89ef326c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7fcefe59c640 1 -- 172.21.15.19:0/815110133 >> v2:172.21.15.19:3300/0 conn(0x7fcf000716b0 msgr2=0x7fcf001646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7fcefe59c640 1 --2- 172.21.15.19:0/815110133 >> v2:172.21.15.19:3300/0 conn(0x7fcf000716b0 0x7fcf001646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7fcefe59c640 1 -- 172.21.15.19:0/815110133 >> v1:172.21.15.112:6789/0 conn(0x7fcf0006f090 legacy=0x7fcf00070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7fcefe59c640 1 -- 172.21.15.19:0/815110133 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0006d380 con 0x7fcf000cc6e0 2024-04-25T09:53:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7f9674ff6640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965c06b5a0 0x7f965c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7f0b7effd640 1 -- 172.21.15.19:0/3921829290 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b78012070 con 0x7f0b880cb660 2024-04-25T09:53:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7f0b7effd640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b6c06b5a0 0x7f0b6c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7f982f7fe640 1 -- 172.21.15.19:0/3364320600 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820037070 con 0x7f98300acd80 2024-04-25T09:53:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7fcefe59c640 1 --2- 172.21.15.19:0/815110133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf000cc6e0 0x7fcf00159b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf0006cdb0 tx=0x7fcef002c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bdfbadd2b4cd1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7f96757f7640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] conn(0x7f965c074ca0 0x7f965c0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7f982f7fe640 1 -- 172.21.15.19:0/3364320600 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9820032440 con 0x7f98300acd80 2024-04-25T09:53:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7f982f7fe640 1 -- 172.21.15.19:0/3364320600 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820032810 con 0x7f98300acd80 2024-04-25T09:53:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7f0b8e49e640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b6c06b5a0 0x7f0b6c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7fcefd59a640 1 -- 172.21.15.19:0/815110133 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0033640 con 0x7fcf000cc6e0 2024-04-25T09:53:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7fcefd59a640 1 -- 172.21.15.19:0/815110133 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0038070 con 0x7fcf000cc6e0 2024-04-25T09:53:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7fcefd59a640 1 -- 172.21.15.19:0/815110133 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0033a40 con 0x7fcf000cc6e0 2024-04-25T09:53:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7f0b7effd640 1 -- 172.21.15.19:0/3921829290 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0b7808a060 con 0x7f0b880cb660 2024-04-25T09:53:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.580+0000 7f987ed34640 1 -- 172.21.15.19:0/4016353824 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98781550e0 con 0x7f98780b3270 2024-04-25T09:53:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7f987ed34640 1 -- 172.21.15.19:0/4016353824 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98781556a0 con 0x7f98780b3270 2024-04-25T09:53:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7f9674ff6640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965c06b5a0 0x7f965c06da60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9668009f60 tx=0x7f96680094f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7f0b7effd640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] conn(0x7f0b6c074ca0 0x7f0b6c0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7f0b7effd640 1 -- 172.21.15.19:0/3921829290 --> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b78021e50 con 0x7f0b6c074ca0 2024-04-25T09:53:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7f0b7effd640 1 -- 172.21.15.19:0/3921829290 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b78091050 con 0x7f0b880cb660 2024-04-25T09:53:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.581+0000 7fb99815c640 1 -- 172.21.15.19:0/1334760739 shutdown_connections 2024-04-25T09:53:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.582+0000 7f96757f7640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] conn(0x7f965c074ca0 0x7f965c0770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.582+0000 7fb99815c640 1 -- 172.21.15.19:0/1334760739 wait complete. 2024-04-25T09:53:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.582+0000 7f0b8dc9d640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] conn(0x7f0b6c074ca0 0x7f0b6c0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.582+0000 7fcf04cf5640 1 -- 172.21.15.19:0/815110133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf000cc6e0 msgr2=0x7fcf00159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.582+0000 7fcf04cf5640 1 --2- 172.21.15.19:0/815110133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf000cc6e0 0x7fcf00159b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf0006cdb0 tx=0x7fcef002c8b0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.582+0000 7f0b8dc9d640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] conn(0x7f0b6c074ca0 0x7f0b6c0770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.582+0000 7fcf04cf5640 1 -- 172.21.15.19:0/815110133 shutdown_connections 2024-04-25T09:53:26.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.582+0000 7fcf04cf5640 1 --2- 172.21.15.19:0/815110133 >> v2:172.21.15.19:3300/0 conn(0x7fcf000716b0 0x7fcf001646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.582+0000 7fcf04cf5640 1 --2- 172.21.15.19:0/815110133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf000cc6e0 0x7fcf00159b20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.582+0000 7fcf04cf5640 1 -- 172.21.15.19:0/815110133 >> 172.21.15.19:0/815110133 conn(0x7fcf000b1170 msgr2=0x7fcf000b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.583+0000 7fcf04cf5640 1 -- 172.21.15.19:0/815110133 shutdown_connections 2024-04-25T09:53:26.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.583+0000 7f966d7fa640 1 -- 172.21.15.19:0/3729175847 <== osd.3 v2:172.21.15.19:6824/577725691 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f965c0777b0 con 0x7f965c074ca0 2024-04-25T09:53:26.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.583+0000 7f987ed34640 1 -- 172.21.15.19:0/4016353824 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9878073a00 con 0x7f98780b3270 2024-04-25T09:53:26.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.583+0000 7fb99815c640 1 Processor -- start 2024-04-25T09:53:26.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.584+0000 7fb99815c640 1 -- start start 2024-04-25T09:53:26.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.584+0000 7f0b7effd640 1 -- 172.21.15.19:0/3921829290 <== osd.0 v2:172.21.15.19:6800/3769136101 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0b78021e50 con 0x7f0b6c074ca0 2024-04-25T09:53:26.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.583+0000 7fcf04cf5640 1 -- 172.21.15.19:0/815110133 wait complete. 2024-04-25T09:53:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.584+0000 7f9836fe7640 1 -- 172.21.15.19:0/3364320600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98300acd80 msgr2=0x7f98300c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.584+0000 7f9836fe7640 1 --2- 172.21.15.19:0/3364320600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98300acd80 0x7f98300c5fc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f983006c7d0 tx=0x7f982002a210 comp rx=0 tx=0).stop 2024-04-25T09:53:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.585+0000 7fcf04cf5640 1 Processor -- start 2024-04-25T09:53:26.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.585+0000 7fb99815c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb988033ee0 0x7fb9880d7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.585+0000 7fcf04cf5640 1 -- start start 2024-04-25T09:53:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.585+0000 7fb99815c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb988053c60 0x7fb9880d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.585+0000 7fb995ef8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb988033ee0 0x7fb9880d7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.585+0000 7fb995ef8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb988033ee0 0x7fb9880d7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55638/0 (socket says 172.21.15.19:55638) 2024-04-25T09:53:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.585+0000 7fb9956f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb988053c60 0x7fb9880d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.585+0000 7fb9956f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb988053c60 0x7fb9880d3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35776/0 (socket says 172.21.15.19:35776) 2024-04-25T09:53:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.585+0000 7fb99815c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9880df210 0x7fb988180910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fb99815c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb988037a80 con 0x7fb988053c60 2024-04-25T09:53:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fb995ef8640 1 -- 172.21.15.19:0/1576607358 learned_addr learned my addr 172.21.15.19:0/1576607358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb988037900 con 0x7fb9880df210 2024-04-25T09:53:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fcf04cf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0006f090 0x7fcf0014da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fcf04cf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf000716b0 0x7fcf00157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880df5f0 con 0x7fb988033ee0 2024-04-25T09:53:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fcf04cf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf000cc6e0 0x7fcf000b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fb9966f9640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9880df210 0x7fb988180910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7f98757fa640 1 -- 172.21.15.19:0/4016353824 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f986c018070 con 0x7f98780b3270 2024-04-25T09:53:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fcf04cf5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000b3920 con 0x7fcf0006f090 2024-04-25T09:53:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fcf04cf5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000b37a0 con 0x7fcf000cc6e0 2024-04-25T09:53:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fcf04cf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000cef40 con 0x7fcf000716b0 2024-04-25T09:53:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fcefe59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0006f090 0x7fcf0014da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fcefe59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0006f090 0x7fcf0014da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35790/0 (socket says 172.21.15.19:35790) 2024-04-25T09:53:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fcefe59c640 1 -- 172.21.15.19:0/3013102996 learned_addr learned my addr 172.21.15.19:0/3013102996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fcefdd9b640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf000716b0 0x7fcf00157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fcefed9d640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf000cc6e0 0x7fcf000b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7f98757fa640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f986406b5d0 0x7f986406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.586+0000 7fb995ef8640 1 -- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9880df210 msgr2=0x7fb988180910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fb995ef8640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9880df210 0x7fb988180910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fb995ef8640 1 -- 172.21.15.19:0/1576607358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb988053c60 msgr2=0x7fb9880d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fb995ef8640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb988053c60 0x7fb9880d3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fcefe59c640 1 -- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf000716b0 msgr2=0x7fcf00157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fcefe59c640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf000716b0 0x7fcf00157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fcefe59c640 1 -- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf000cc6e0 msgr2=0x7fcf000b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fcefe59c640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf000cc6e0 0x7fcf000b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fcefe59c640 1 -- 172.21.15.19:0/3013102996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00156ab0 con 0x7fcf0006f090 2024-04-25T09:53:26.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fb995ef8640 1 -- 172.21.15.19:0/1576607358 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9880d70c0 con 0x7fb988033ee0 2024-04-25T09:53:26.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7f987cad0640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f986406b5d0 0x7f986406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7f98757fa640 1 -- 172.21.15.19:0/4016353824 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f986c08a470 con 0x7f98780b3270 2024-04-25T09:53:26.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fcefdd9b640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf000716b0 0x7fcf00157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7f98757fa640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] conn(0x7f9864074d50 0x7f9864077190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fb9956f7640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb988053c60 0x7fb9880d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fcefe59c640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0006f090 0x7fcf0014da30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcef0000f80 tx=0x7fcef0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7f98757fa640 1 -- 172.21.15.19:0/4016353824 --> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9864077860 con 0x7f9864074d50 2024-04-25T09:53:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7f98757fa640 1 -- 172.21.15.19:0/4016353824 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f986c091050 con 0x7f98780b3270 2024-04-25T09:53:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fb9966f9640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9880df210 0x7fb988180910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7f9877fff640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] conn(0x7f9864074d50 0x7f9864077190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7f9836fe7640 1 -- 172.21.15.19:0/3364320600 shutdown_connections 2024-04-25T09:53:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7f9836fe7640 1 --2- 172.21.15.19:0/3364320600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98300acd80 0x7f98300c5fc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7f9836fe7640 1 --2- 172.21.15.19:0/3364320600 >> v2:172.21.15.19:3300/0 conn(0x7f98300cda20 0x7f98300cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7f9836fe7640 1 --2- 172.21.15.19:0/3364320600 >> v2:172.21.15.112:3300/0 conn(0x7f983006bbf0 0x7f983006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.588+0000 7f9836fe7640 1 -- 172.21.15.19:0/3364320600 >> 172.21.15.19:0/3364320600 conn(0x7f9830071af0 msgr2=0x7f9830071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fceeeffd640 1 -- 172.21.15.19:0/3013102996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef003c800 con 0x7fcf0006f090 2024-04-25T09:53:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fceeeffd640 1 -- 172.21.15.19:0/3013102996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef003c9c0 con 0x7fcf0006f090 2024-04-25T09:53:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.587+0000 7fceeeffd640 1 -- 172.21.15.19:0/3013102996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef003cdb0 con 0x7fcf0006f090 2024-04-25T09:53:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.588+0000 7fcf04cf5640 1 -- 172.21.15.19:0/3013102996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf00156d90 con 0x7fcf0006f090 2024-04-25T09:53:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.588+0000 7fcf04cf5640 1 -- 172.21.15.19:0/3013102996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf00152310 con 0x7fcf0006f090 2024-04-25T09:53:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.588+0000 7f987cad0640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f986406b5d0 0x7f986406da90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98700012e0 tx=0x7f9870007cf0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.588+0000 7f9877fff640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] conn(0x7f9864074d50 0x7f9864077190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.588+0000 7f0b8e49e640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b6c06b5a0 0x7f0b6c06da60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b80000f80 tx=0x7f0b80009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.589+0000 7f98757fa640 1 -- 172.21.15.19:0/4016353824 <== osd.1 v2:172.21.15.19:6808/495652536 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9864077860 con 0x7f9864074d50 2024-04-25T09:53:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.589+0000 7f9836fe7640 1 -- 172.21.15.19:0/3364320600 shutdown_connections 2024-04-25T09:53:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.590+0000 7fcf04cf5640 1 -- 172.21.15.19:0/3013102996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf001526f0 con 0x7fcf0006f090 2024-04-25T09:53:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.590+0000 7fb995ef8640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb988033ee0 0x7fb9880d7e80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb98000d800 tx=0x7fb98000dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.590+0000 7fb9867fc640 1 -- 172.21.15.19:0/1576607358 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98000b6c0 con 0x7fb988033ee0 2024-04-25T09:53:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.590+0000 7fb9867fc640 1 -- 172.21.15.19:0/1576607358 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb980019070 con 0x7fb988033ee0 2024-04-25T09:53:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.590+0000 7fb9867fc640 1 -- 172.21.15.19:0/1576607358 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98000ba80 con 0x7fb988033ee0 2024-04-25T09:53:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.591+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9880d73a0 con 0x7fb988033ee0 2024-04-25T09:53:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.591+0000 7fceeeffd640 1 -- 172.21.15.19:0/3013102996 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcef004f440 con 0x7fcf0006f090 2024-04-25T09:53:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.591+0000 7fceeeffd640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcee006b5a0 0x7fcee006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.591+0000 7fceeeffd640 1 -- 172.21.15.19:0/3013102996 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcef00ab6f0 con 0x7fcf0006f090 2024-04-25T09:53:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.592+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9880d7960 con 0x7fb988033ee0 2024-04-25T09:53:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.591+0000 7fceeeffd640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] conn(0x7fcee0074ca0 0x7fcee00770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.591+0000 7fceeeffd640 1 -- 172.21.15.19:0/3013102996 --> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcef0047e70 con 0x7fcee0074ca0 2024-04-25T09:53:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.591+0000 7fceeeffd640 1 -- 172.21.15.19:0/3013102996 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcef00b2050 con 0x7fcf0006f090 2024-04-25T09:53:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.592+0000 7fcefed9d640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] conn(0x7fcee0074ca0 0x7fcee00770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.592+0000 7fcefdd9b640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcee006b5a0 0x7fcee006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.593+0000 7fcefed9d640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] conn(0x7fcee0074ca0 0x7fcee00770e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.593+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb988005380 con 0x7fb988033ee0 2024-04-25T09:53:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.594+0000 7fceeeffd640 1 -- 172.21.15.19:0/3013102996 <== osd.7 v2:172.21.15.112:6817/3405244921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcef0047e70 con 0x7fcee0074ca0 2024-04-25T09:53:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.595+0000 7fb9867fc640 1 -- 172.21.15.19:0/1576607358 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb98000bc20 con 0x7fb988033ee0 2024-04-25T09:53:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.595+0000 7fb9867fc640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb97406b520 0x7fb97406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.595+0000 7fb9867fc640 1 -- 172.21.15.19:0/1576607358 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb980057920 con 0x7fb988033ee0 2024-04-25T09:53:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.596+0000 7fb9867fc640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x7fb9740758f0 0x7fb974077d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.596+0000 7fb9867fc640 1 -- 172.21.15.19:0/1576607358 --> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb974078460 con 0x7fb9740758f0 2024-04-25T09:53:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.596+0000 7fb9867fc640 1 -- 172.21.15.19:0/1576607358 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9800562b0 con 0x7fb988033ee0 2024-04-25T09:53:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.596+0000 7fb9966f9640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x7fb9740758f0 0x7fb974077d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.593+0000 7f9836fe7640 1 -- 172.21.15.19:0/3364320600 wait complete. 2024-04-25T09:53:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.595+0000 7f9836fe7640 1 Processor -- start 2024-04-25T09:53:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.596+0000 7fb9956f7640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb97406b520 0x7fb97406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.596+0000 7fcefdd9b640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcee006b5a0 0x7fcee006da60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf00151bb0 tx=0x7fcef8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.597+0000 7fb9966f9640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x7fb9740758f0 0x7fb974077d90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.598+0000 7fb9956f7640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb97406b520 0x7fb97406d9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb98c056000 tx=0x7fb98c050000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.598+0000 7fb9867fc640 1 -- 172.21.15.19:0/1576607358 <== osd.2 v2:172.21.15.19:6816/3205329770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb974078460 con 0x7fb9740758f0 2024-04-25T09:53:26.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.595+0000 7f9836fe7640 1 -- start start 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.597+0000 7f9836fe7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f983006bbf0 0x7f98300b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.597+0000 7f9836fe7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98300acd80 0x7f98301665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.597+0000 7f9836fe7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98300cda20 0x7f9830150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.597+0000 7f9836fe7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98300c6500 con 0x7f98300acd80 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.597+0000 7f9836fe7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98300c5750 con 0x7f98300cda20 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.597+0000 7f9836fe7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98300c6960 con 0x7f983006bbf0 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.597+0000 7f982ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98300acd80 0x7f98301665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.598+0000 7f982ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98300acd80 0x7f98301665e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35794/0 (socket says 172.21.15.19:35794) 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.598+0000 7f982ffff640 1 -- 172.21.15.19:0/4097326261 learned_addr learned my addr 172.21.15.19:0/4097326261 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.598+0000 7f982ffff640 1 -- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f983006bbf0 msgr2=0x7f98300b2b90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.598+0000 7f982ffff640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f983006bbf0 0x7f98300b2b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.598+0000 7f982ffff640 1 -- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98300cda20 msgr2=0x7f9830150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.598+0000 7f982ffff640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98300cda20 0x7f9830150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.598+0000 7f982ffff640 1 -- 172.21.15.19:0/4097326261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983015d4d0 con 0x7f98300acd80 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.598+0000 7f982ffff640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98300acd80 0x7f98301665e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9820009960 tx=0x7f98200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.599+0000 7f982d7fa640 1 -- 172.21.15.19:0/4097326261 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820041020 con 0x7f98300acd80 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.599+0000 7f9836fe7640 1 -- 172.21.15.19:0/4097326261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98300b3200 con 0x7f98300acd80 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.604+0000 7f9836fe7640 1 -- 172.21.15.19:0/4097326261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98300b37c0 con 0x7f98300acd80 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.605+0000 7f982d7fa640 1 -- 172.21.15.19:0/4097326261 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9820030700 con 0x7f98300acd80 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.605+0000 7f982d7fa640 1 -- 172.21.15.19:0/4097326261 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820037040 con 0x7f98300acd80 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.605+0000 7f982d7fa640 1 -- 172.21.15.19:0/4097326261 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9820044020 con 0x7f98300acd80 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.606+0000 7f982d7fa640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f981006b5d0 0x7f981006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.606+0000 7f982d7fa640 1 -- 172.21.15.19:0/4097326261 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f98200af060 con 0x7f98300acd80 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.608+0000 7f9834d83640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f981006b5d0 0x7f981006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.610+0000 7f9836fe7640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x7f9814001650 0x7f9814003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.610+0000 7f9836fe7640 1 -- 172.21.15.19:0/4097326261 --> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9814006bb0 con 0x7f9814001650 2024-04-25T09:53:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.611+0000 7f9835584640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x7f9814001650 0x7f9814003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.612+0000 7f9834d83640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f981006b5d0 0x7f981006da90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f98301530d0 tx=0x7f98280097b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.617+0000 7f9835584640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x7f9814001650 0x7f9814003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.618+0000 7f982d7fa640 1 -- 172.21.15.19:0/4097326261 <== osd.6 v2:172.21.15.112:6816/3693987094 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9814006bb0 con 0x7f9814001650 2024-04-25T09:53:26.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.617+0000 7f0b90702640 1 -- 172.21.15.19:0/3921829290 --> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0b70002dd0 con 0x7f0b6c074ca0 2024-04-25T09:53:26.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.618+0000 7f0b7effd640 1 -- 172.21.15.19:0/3921829290 <== osd.0 v2:172.21.15.19:6800/3769136101 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0b70002dd0 con 0x7f0b6c074ca0 2024-04-25T09:53:26.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.620+0000 7f987ed34640 1 -- 172.21.15.19:0/4016353824 --> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f98780cf480 con 0x7f9864074d50 2024-04-25T09:53:26.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.619+0000 7f0b7cff9640 1 -- 172.21.15.19:0/3921829290 >> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] conn(0x7f0b6c074ca0 msgr2=0x7f0b6c0770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.619+0000 7f0b7cff9640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] conn(0x7f0b6c074ca0 0x7f0b6c0770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.620+0000 7f0b7cff9640 1 -- 172.21.15.19:0/3921829290 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b6c06b5a0 msgr2=0x7f0b6c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.621+0000 7f98757fa640 1 -- 172.21.15.19:0/4016353824 <== osd.1 v2:172.21.15.19:6808/495652536 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f98780cf480 con 0x7f9864074d50 2024-04-25T09:53:26.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.621+0000 7f0b7cff9640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b6c06b5a0 0x7f0b6c06da60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b80000f80 tx=0x7f0b80009030 comp rx=0 tx=0).stop 2024-04-25T09:53:26.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.621+0000 7f0b7cff9640 1 -- 172.21.15.19:0/3921829290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b880cb660 msgr2=0x7f0b880cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.621+0000 7f0b7cff9640 1 --2- 172.21.15.19:0/3921829290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b880cb660 0x7f0b880cbae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0b880d07b0 tx=0x7f0b7800fc10 comp rx=0 tx=0).stop 2024-04-25T09:53:26.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.621+0000 7f0b8ec9f640 1 -- 172.21.15.19:0/3921829290 reap_dead start 2024-04-25T09:53:26.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.621+0000 7f0b7cff9640 1 -- 172.21.15.19:0/3921829290 shutdown_connections 2024-04-25T09:53:26.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.621+0000 7f0b7cff9640 1 -- 172.21.15.19:0/3921829290 >> 172.21.15.19:0/3921829290 conn(0x7f0b8806abe0 msgr2=0x7f0b880b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.620+0000 7f543ef68640 1 Processor -- start 2024-04-25T09:53:26.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.620+0000 7f543ef68640 1 -- start start 2024-04-25T09:53:26.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f0b7cff9640 1 -- 172.21.15.19:0/3921829290 shutdown_connections 2024-04-25T09:53:26.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f0b7cff9640 1 -- 172.21.15.19:0/3921829290 wait complete. 2024-04-25T09:53:26.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f543ef68640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f543815a9f0 0x7f54380c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f543ef68640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f54380cdbf0 0x7f54380ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f543ef68640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54380c6940 con 0x7f54380cd520 2024-04-25T09:53:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f543ef68640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54380b0020 con 0x7f54380cdbf0 2024-04-25T09:53:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f543ef68640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54380cb5a0 con 0x7f543815a9f0 2024-04-25T09:53:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f987ed34640 1 -- 172.21.15.19:0/4016353824 >> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] conn(0x7f9864074d50 msgr2=0x7f9864077190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f987ed34640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] conn(0x7f9864074d50 0x7f9864077190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f5437fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f54380cdbf0 0x7f54380ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f5437fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f54380cdbf0 0x7f54380ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35802/0 (socket says 172.21.15.19:35802) 2024-04-25T09:53:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f5437fff640 1 -- 172.21.15.19:0/1736980655 learned_addr learned my addr 172.21.15.19:0/1736980655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.622+0000 7f543cd04640 1 --2- 172.21.15.19:0/1736980655 >> v2:172.21.15.112:3300/0 conn(0x7f543815a9f0 0x7f54380c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.623+0000 7f5437fff640 1 -- 172.21.15.19:0/1736980655 >> v2:172.21.15.112:3300/0 conn(0x7f543815a9f0 msgr2=0x7f54380c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.623+0000 7f5437fff640 1 --2- 172.21.15.19:0/1736980655 >> v2:172.21.15.112:3300/0 conn(0x7f543815a9f0 0x7f54380c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.623+0000 7f5437fff640 1 -- 172.21.15.19:0/1736980655 >> v1:172.21.15.112:6789/0 conn(0x7f54380cd520 legacy=0x7f54380ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.623+0000 7f5437fff640 1 -- 172.21.15.19:0/1736980655 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543815add0 con 0x7f54380cdbf0 2024-04-25T09:53:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.623+0000 7f543cd04640 1 --2- 172.21.15.19:0/1736980655 >> v2:172.21.15.112:3300/0 conn(0x7f543815a9f0 0x7f54380c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.624+0000 7f987ed34640 1 -- 172.21.15.19:0/4016353824 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f986406b5d0 msgr2=0x7f986406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.624+0000 7f967725a640 1 -- 172.21.15.19:0/3729175847 --> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9650002dd0 con 0x7f965c074ca0 2024-04-25T09:53:26.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.624+0000 7f5437fff640 1 --2- 172.21.15.19:0/1736980655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54380cdbf0 0x7f54380ccfe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f54380c77e0 tx=0x7f5428032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665a8b046b512e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.624+0000 7f987ed34640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f986406b5d0 0x7f986406da90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98700012e0 tx=0x7f9870007cf0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.624+0000 7f987ed34640 1 -- 172.21.15.19:0/4016353824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780b3270 msgr2=0x7f987813f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.624+0000 7f987ed34640 1 --2- 172.21.15.19:0/4016353824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98780b3270 0x7f987813f840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c820 tx=0x7f986c00cce0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.624+0000 7f987d2d1640 1 -- 172.21.15.19:0/4016353824 reap_dead start 2024-04-25T09:53:26.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.625+0000 7f987ed34640 1 -- 172.21.15.19:0/4016353824 shutdown_connections 2024-04-25T09:53:26.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.625+0000 7f987ed34640 1 -- 172.21.15.19:0/4016353824 >> 172.21.15.19:0/4016353824 conn(0x7f987806abe0 msgr2=0x7f98780cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.625+0000 7f966d7fa640 1 -- 172.21.15.19:0/3729175847 <== osd.3 v2:172.21.15.19:6824/577725691 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9650002dd0 con 0x7f965c074ca0 2024-04-25T09:53:26.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.625+0000 7f987ed34640 1 -- 172.21.15.19:0/4016353824 shutdown_connections 2024-04-25T09:53:26.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.625+0000 7f987ed34640 1 -- 172.21.15.19:0/4016353824 wait complete. 2024-04-25T09:53:26.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.624+0000 7f54377fe640 1 -- 172.21.15.19:0/1736980655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428040070 con 0x7f54380cdbf0 2024-04-25T09:53:26.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.624+0000 7f54377fe640 1 -- 172.21.15.19:0/1736980655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5428033c90 con 0x7f54380cdbf0 2024-04-25T09:53:26.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.625+0000 7f543ef68640 1 -- 172.21.15.19:0/1736980655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54380cdbf0 msgr2=0x7f54380ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.625+0000 7f543ef68640 1 --2- 172.21.15.19:0/1736980655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54380cdbf0 0x7f54380ccfe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f54380c77e0 tx=0x7f5428032f40 comp rx=0 tx=0).stop 2024-04-25T09:53:26.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.626+0000 7f543ef68640 1 -- 172.21.15.19:0/1736980655 shutdown_connections 2024-04-25T09:53:26.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.626+0000 7f543ef68640 1 --2- 172.21.15.19:0/1736980655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54380cdbf0 0x7f54380ccfe0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.626+0000 7f543ef68640 1 --2- 172.21.15.19:0/1736980655 >> v2:172.21.15.112:3300/0 conn(0x7f543815a9f0 0x7f54380c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.626+0000 7f543ef68640 1 -- 172.21.15.19:0/1736980655 >> 172.21.15.19:0/1736980655 conn(0x7f54380b2390 msgr2=0x7f54380b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.626+0000 7f967725a640 1 -- 172.21.15.19:0/3729175847 >> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] conn(0x7f965c074ca0 msgr2=0x7f965c0770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.626+0000 7f967725a640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] conn(0x7f965c074ca0 0x7f965c0770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.626+0000 7f543ef68640 1 -- 172.21.15.19:0/1736980655 shutdown_connections 2024-04-25T09:53:26.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.626+0000 7f543ef68640 1 -- 172.21.15.19:0/1736980655 wait complete. 2024-04-25T09:53:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.627+0000 7fabbc4fc640 1 Processor -- start 2024-04-25T09:53:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.627+0000 7fcf04cf5640 1 -- 172.21.15.19:0/3013102996 --> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf00073990 con 0x7fcee0074ca0 2024-04-25T09:53:26.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.627+0000 7fabbc4fc640 1 -- start start 2024-04-25T09:53:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.628+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 --> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb988005b80 con 0x7fb9740758f0 2024-04-25T09:53:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.628+0000 7f543ef68640 1 Processor -- start 2024-04-25T09:53:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.628+0000 7f543ef68640 1 -- start start 2024-04-25T09:53:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.628+0000 7f967725a640 1 -- 172.21.15.19:0/3729175847 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965c06b5a0 msgr2=0x7f965c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.628+0000 7f967725a640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965c06b5a0 0x7f965c06da60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9668009f60 tx=0x7f96680094f0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7f967725a640 1 -- 172.21.15.19:0/3729175847 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9670161240 msgr2=0x7f96700cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7f967725a640 1 --2- 172.21.15.19:0/3729175847 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9670161240 0x7f96700cd1a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f966000c460 tx=0x7f966000c920 comp rx=0 tx=0).stop 2024-04-25T09:53:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7f96757f7640 1 -- 172.21.15.19:0/3729175847 reap_dead start 2024-04-25T09:53:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7fb9867fc640 1 -- 172.21.15.19:0/1576607358 <== osd.2 v2:172.21.15.19:6816/3205329770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb988005b80 con 0x7fb9740758f0 2024-04-25T09:53:26.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7fabbc4fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fabb415e290 0x7fabb40cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7fabbc4fc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb415e740 con 0x7fabb406b6d0 2024-04-25T09:53:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7fabbc4fc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb4071c50 con 0x7fabb40cd820 2024-04-25T09:53:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7fabbc4fc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb415d840 con 0x7fabb415e290 2024-04-25T09:53:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7fabbaa99640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fabb40cd820 0x7fabb40ae830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:43416/0 (socket says 172.21.15.19:43416) 2024-04-25T09:53:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7fabb9a97640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fabb415e290 0x7fabb40cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7fabbaa99640 1 -- 172.21.15.19:0/83101179 learned_addr learned my addr 172.21.15.19:0/83101179 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:26.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x7fb9740758f0 msgr2=0x7fb974077d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7fb99815c640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x7fb9740758f0 0x7fb974077d90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb97406b520 msgr2=0x7fb97406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fb99815c640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb97406b520 0x7fb97406d9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb98c056000 tx=0x7fb98c050000 comp rx=0 tx=0).stop 2024-04-25T09:53:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fabb9a97640 1 -- 172.21.15.19:0/83101179 >> v1:172.21.15.112:6789/0 conn(0x7fabb40cd820 legacy=0x7fabb40ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fabb9a97640 1 -- 172.21.15.19:0/83101179 >> v1:172.21.15.19:6789/0 conn(0x7fabb406b6d0 legacy=0x7fabb4157700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:53:26.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb988033ee0 msgr2=0x7fb9880d7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7f967725a640 1 -- 172.21.15.19:0/3729175847 shutdown_connections 2024-04-25T09:53:26.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.629+0000 7f967725a640 1 -- 172.21.15.19:0/3729175847 >> 172.21.15.19:0/3729175847 conn(0x7f96700d04a0 msgr2=0x7f967015b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fb99815c640 1 --2- 172.21.15.19:0/1576607358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb988033ee0 0x7fb9880d7e80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb98000d800 tx=0x7fb98000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fb9966f9640 1 -- 172.21.15.19:0/1576607358 reap_dead start 2024-04-25T09:53:26.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 shutdown_connections 2024-04-25T09:53:26.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 >> 172.21.15.19:0/1576607358 conn(0x7fb988009fa0 msgr2=0x7fb9880df8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fabb9a97640 1 -- 172.21.15.19:0/83101179 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb4071f30 con 0x7fabb415e290 2024-04-25T09:53:26.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fceeeffd640 1 -- 172.21.15.19:0/3013102996 <== osd.7 v2:172.21.15.112:6817/3405244921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcf00073990 con 0x7fcee0074ca0 2024-04-25T09:53:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 shutdown_connections 2024-04-25T09:53:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7fb99815c640 1 -- 172.21.15.19:0/1576607358 wait complete. 2024-04-25T09:53:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7f543ef68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54380cd520 0x7f54381f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7f543ef68640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54380cdbf0 0x7f54380ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7f543ef68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543815a9f0 0x7f543806e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.630+0000 7f543d505640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543815a9f0 0x7f543806e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543cd04640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54380cd520 0x7f54381f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543d505640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543815a9f0 0x7f543806e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35806/0 (socket says 172.21.15.19:35806) 2024-04-25T09:53:26.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f5437fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54380cdbf0 0x7f54380ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543cd04640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54380cd520 0x7f54381f00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:51876/0 (socket says 172.21.15.19:51876) 2024-04-25T09:53:26.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f5437fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54380cdbf0 0x7f54380ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55660/0 (socket says 172.21.15.19:55660) 2024-04-25T09:53:26.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543d505640 1 -- 172.21.15.19:0/4105476855 learned_addr learned my addr 172.21.15.19:0/4105476855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7fcf04cf5640 1 -- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] conn(0x7fcee0074ca0 msgr2=0x7fcee00770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7fcf04cf5640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] conn(0x7fcee0074ca0 0x7fcee00770e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54380cb230 con 0x7f543815a9f0 2024-04-25T09:53:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54380caf50 con 0x7f54380cd520 2024-04-25T09:53:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54380ce070 con 0x7f54380cdbf0 2024-04-25T09:53:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7fcf04cf5640 1 -- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcee006b5a0 msgr2=0x7fcee006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7fcf04cf5640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcee006b5a0 0x7fcee006da60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf00151bb0 tx=0x7fcef8009040 comp rx=0 tx=0).stop 2024-04-25T09:53:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7fcf04cf5640 1 -- 172.21.15.19:0/3013102996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0006f090 msgr2=0x7fcf0014da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7fcf04cf5640 1 --2- 172.21.15.19:0/3013102996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0006f090 0x7fcf0014da30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcef0000f80 tx=0x7fcef0004ab0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7fcefed9d640 1 -- 172.21.15.19:0/3013102996 reap_dead start 2024-04-25T09:53:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543d505640 1 -- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54380cdbf0 msgr2=0x7f54380ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543d505640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54380cdbf0 0x7f54380ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543d505640 1 -- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54380cd520 msgr2=0x7f54381f00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543d505640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54380cd520 0x7f54381f00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7f543d505640 1 -- 172.21.15.19:0/4105476855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54381f2b90 con 0x7f543815a9f0 2024-04-25T09:53:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7fcf04cf5640 1 -- 172.21.15.19:0/3013102996 shutdown_connections 2024-04-25T09:53:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.631+0000 7fcf04cf5640 1 -- 172.21.15.19:0/3013102996 >> 172.21.15.19:0/3013102996 conn(0x7fcf000b1170 msgr2=0x7fcf000b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7fabb9a97640 1 --2- 172.21.15.19:0/83101179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb415e290 0x7fabb40cd2e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009750 tx=0x7fabb002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2784c42a5cc2ddf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7fcf04cf5640 1 -- 172.21.15.19:0/3013102996 shutdown_connections 2024-04-25T09:53:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7f5437fff640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54380cdbf0 0x7f54380ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7fcf04cf5640 1 -- 172.21.15.19:0/3013102996 wait complete. 2024-04-25T09:53:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7fabb9296640 1 -- 172.21.15.19:0/83101179 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0037070 con 0x7fabb415e290 2024-04-25T09:53:26.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7fabb9296640 1 -- 172.21.15.19:0/83101179 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb002ad90 con 0x7fabb415e290 2024-04-25T09:53:26.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7fabb9296640 1 -- 172.21.15.19:0/83101179 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0032660 con 0x7fabb415e290 2024-04-25T09:53:26.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7f543d505640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543815a9f0 0x7f543806e740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f543000cb20 tx=0x7f54300020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7f54357fa640 1 -- 172.21.15.19:0/4105476855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430016020 con 0x7f543815a9f0 2024-04-25T09:53:26.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7f54357fa640 1 -- 172.21.15.19:0/4105476855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5430006d50 con 0x7f543815a9f0 2024-04-25T09:53:26.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7f54357fa640 1 -- 172.21.15.19:0/4105476855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430003bb0 con 0x7f543815a9f0 2024-04-25T09:53:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.632+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54381f2e70 con 0x7f543815a9f0 2024-04-25T09:53:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.633+0000 7fabbc4fc640 1 -- 172.21.15.19:0/83101179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb415e290 msgr2=0x7fabb40cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.633+0000 7fabbc4fc640 1 --2- 172.21.15.19:0/83101179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb415e290 0x7fabb40cd2e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009750 tx=0x7fabb002a080 comp rx=0 tx=0).stop 2024-04-25T09:53:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.633+0000 7fabbc4fc640 1 -- 172.21.15.19:0/83101179 shutdown_connections 2024-04-25T09:53:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.633+0000 7fabbc4fc640 1 --2- 172.21.15.19:0/83101179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb415e290 0x7fabb40cd2e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.633+0000 7fabbc4fc640 1 -- 172.21.15.19:0/83101179 >> 172.21.15.19:0/83101179 conn(0x7fabb406abe0 msgr2=0x7fabb406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.633+0000 7fabbc4fc640 1 -- 172.21.15.19:0/83101179 shutdown_connections 2024-04-25T09:53:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.633+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54381f3430 con 0x7f543815a9f0 2024-04-25T09:53:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.633+0000 7fabbc4fc640 1 -- 172.21.15.19:0/83101179 wait complete. 2024-04-25T09:53:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.634+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5438073a00 con 0x7f543815a9f0 2024-04-25T09:53:26.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.635+0000 7f967725a640 1 -- 172.21.15.19:0/3729175847 shutdown_connections 2024-04-25T09:53:26.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.635+0000 7f967725a640 1 -- 172.21.15.19:0/3729175847 wait complete. 2024-04-25T09:53:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.635+0000 7fabbc4fc640 1 Processor -- start 2024-04-25T09:53:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.635+0000 7fabbc4fc640 1 -- start start 2024-04-25T09:53:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.636+0000 7fabbc4fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabb406b6d0 0x7fabb40b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.636+0000 7fabbc4fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb40cd820 0x7fabb4151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.636+0000 7fabbc4fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabb415e290 0x7fabb40a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.637+0000 7fabbc4fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40718e0 con 0x7fabb415e290 2024-04-25T09:53:26.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.637+0000 7fabbaa99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabb415e290 0x7fabb40a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.637+0000 7fabbaa99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabb415e290 0x7fabb40a8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35812/0 (socket says 172.21.15.19:35812) 2024-04-25T09:53:26.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.637+0000 7fabbaa99640 1 -- 172.21.15.19:0/3482942506 learned_addr learned my addr 172.21.15.19:0/3482942506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.637+0000 7fabbc4fc640 1 -- 172.21.15.19:0/3482942506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb415dd70 con 0x7fabb40cd820 2024-04-25T09:53:26.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.637+0000 7fabbc4fc640 1 -- 172.21.15.19:0/3482942506 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40aef40 con 0x7fabb406b6d0 2024-04-25T09:53:26.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.636+0000 7f54357fa640 1 -- 172.21.15.19:0/4105476855 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f543000a760 con 0x7f543815a9f0 2024-04-25T09:53:26.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.637+0000 7f54357fa640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f542406b4d0 0x7f542406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.638+0000 7f543cd04640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f542406b4d0 0x7f542406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.638+0000 7fabbaa99640 1 -- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabb406b6d0 msgr2=0x7fabb40b1260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T09:53:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.638+0000 7fabbaa99640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabb406b6d0 0x7fabb40b1260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.638+0000 7fabbaa99640 1 -- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb40cd820 msgr2=0x7fabb4151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.638+0000 7fabbaa99640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabb40cd820 0x7fabb4151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.638+0000 7fabbaa99640 1 -- 172.21.15.19:0/3482942506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb40a86b0 con 0x7fabb415e290 2024-04-25T09:53:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.638+0000 7fabbaa99640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabb415e290 0x7fabb40a8e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faba400c550 tx=0x7faba400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.639+0000 7f543cd04640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f542406b4d0 0x7f542406d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54381566e0 tx=0x7f542c00f040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.639+0000 7f54357fa640 1 -- 172.21.15.19:0/4105476855 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f543008a160 con 0x7f543815a9f0 2024-04-25T09:53:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.639+0000 7f54357fa640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x7f5424074c50 0x7f5424077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.639+0000 7f54357fa640 1 -- 172.21.15.19:0/4105476855 --> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f543001eec0 con 0x7f5424074c50 2024-04-25T09:53:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.639+0000 7f54357fa640 1 -- 172.21.15.19:0/4105476855 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5430091050 con 0x7f543815a9f0 2024-04-25T09:53:26.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.639+0000 7fabaaffd640 1 -- 172.21.15.19:0/3482942506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4018070 con 0x7fabb415e290 2024-04-25T09:53:26.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.639+0000 7fabaaffd640 1 -- 172.21.15.19:0/3482942506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba4007500 con 0x7fabb415e290 2024-04-25T09:53:26.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.639+0000 7fabaaffd640 1 -- 172.21.15.19:0/3482942506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4004a20 con 0x7fabb415e290 2024-04-25T09:53:26.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.640+0000 7f5437fff640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x7f5424074c50 0x7f5424077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.640+0000 7fabbc4fc640 1 -- 172.21.15.19:0/3482942506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb40b1990 con 0x7fabb415e290 2024-04-25T09:53:26.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.640+0000 7fabbc4fc640 1 -- 172.21.15.19:0/3482942506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb40b2640 con 0x7fabb415e290 2024-04-25T09:53:26.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.643+0000 7fabaaffd640 1 -- 172.21.15.19:0/3482942506 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faba401b070 con 0x7fabb415e290 2024-04-25T09:53:26.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.644+0000 7fabaaffd640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab9806b5a0 0x7fab9806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.644+0000 7fabaaffd640 1 -- 172.21.15.19:0/3482942506 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faba4089ad0 con 0x7fabb415e290 2024-04-25T09:53:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.644+0000 7faba8ff9640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x7fabb40b0370 0x7fabb40b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.644+0000 7faba8ff9640 1 -- 172.21.15.19:0/3482942506 --> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabb40cf480 con 0x7fabb40b0370 2024-04-25T09:53:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.645+0000 7fabba298640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab9806b5a0 0x7fab9806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.645+0000 7fabb9a97640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x7fabb40b0370 0x7fabb40b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.679 INFO:teuthology.orchestra.run.smithi019.stdout:55834574857 2024-04-25T09:53:26.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.645+0000 7fabba298640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab9806b5a0 0x7fab9806da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fabb40aef10 tx=0x7fabac007d10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.680 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T09:53:26.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.646+0000 7f5437fff640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x7f5424074c50 0x7f5424077090 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.646+0000 7fabb9a97640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x7fabb40b0370 0x7fabb40b2810 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.682 INFO:teuthology.orchestra.run.smithi019.stdout:55834574856 2024-04-25T09:53:26.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T09:53:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.647+0000 7f54357fa640 1 -- 172.21.15.19:0/4105476855 <== osd.5 v2:172.21.15.112:6808/2948328480 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f543001eec0 con 0x7f5424074c50 2024-04-25T09:53:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.647+0000 7fabaaffd640 1 -- 172.21.15.19:0/3482942506 <== osd.4 v2:172.21.15.112:6800/1718655451 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fabb40cf480 con 0x7fabb40b0370 2024-04-25T09:53:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.648+0000 7f9836fe7640 1 -- 172.21.15.19:0/4097326261 --> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9814005c90 con 0x7f9814001650 2024-04-25T09:53:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.649+0000 7f982d7fa640 1 -- 172.21.15.19:0/4097326261 <== osd.6 v2:172.21.15.112:6816/3693987094 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9814005c90 con 0x7f9814001650 2024-04-25T09:53:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.650+0000 7f9836fe7640 1 -- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x7f9814001650 msgr2=0x7f9814003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.650+0000 7f9836fe7640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x7f9814001650 0x7f9814003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.650+0000 7f9836fe7640 1 -- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f981006b5d0 msgr2=0x7f981006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.650+0000 7f9836fe7640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f981006b5d0 0x7f981006da90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f98301530d0 tx=0x7f98280097b0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.650+0000 7f9836fe7640 1 -- 172.21.15.19:0/4097326261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98300acd80 msgr2=0x7f98301665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.650+0000 7f9836fe7640 1 --2- 172.21.15.19:0/4097326261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98300acd80 0x7f98301665e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9820009960 tx=0x7f98200046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.650+0000 7f9835584640 1 -- 172.21.15.19:0/4097326261 reap_dead start 2024-04-25T09:53:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.650+0000 7f9836fe7640 1 -- 172.21.15.19:0/4097326261 shutdown_connections 2024-04-25T09:53:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.650+0000 7f9836fe7640 1 -- 172.21.15.19:0/4097326261 >> 172.21.15.19:0/4097326261 conn(0x7f9830071af0 msgr2=0x7f98300ad970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.650+0000 7f9836fe7640 1 -- 172.21.15.19:0/4097326261 shutdown_connections 2024-04-25T09:53:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.650+0000 7f9836fe7640 1 -- 172.21.15.19:0/4097326261 wait complete. 2024-04-25T09:53:26.687 INFO:teuthology.orchestra.run.smithi019.stdout:55834574858 2024-04-25T09:53:26.687 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T09:53:26.688 INFO:teuthology.orchestra.run.smithi019.stdout:55834574858 2024-04-25T09:53:26.688 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T09:53:26.689 INFO:teuthology.orchestra.run.smithi019.stdout:55834574858 2024-04-25T09:53:26.690 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T09:53:26.691 INFO:teuthology.orchestra.run.smithi019.stdout:55834574858 2024-04-25T09:53:26.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T09:53:26.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.670+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 --> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f54380c63a0 con 0x7f5424074c50 2024-04-25T09:53:26.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f54357fa640 1 -- 172.21.15.19:0/4105476855 <== osd.5 v2:172.21.15.112:6808/2948328480 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f54380c63a0 con 0x7f5424074c50 2024-04-25T09:53:26.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x7f5424074c50 msgr2=0x7f5424077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f543ef68640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x7f5424074c50 0x7f5424077090 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f542406b4d0 msgr2=0x7f542406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f543ef68640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f542406b4d0 0x7f542406d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54381566e0 tx=0x7f542c00f040 comp rx=0 tx=0).stop 2024-04-25T09:53:26.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543815a9f0 msgr2=0x7f543806e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f543ef68640 1 --2- 172.21.15.19:0/4105476855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543815a9f0 0x7f543806e740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f543000cb20 tx=0x7f54300020e0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f543d505640 1 -- 172.21.15.19:0/4105476855 reap_dead start 2024-04-25T09:53:26.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 shutdown_connections 2024-04-25T09:53:26.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 >> 172.21.15.19:0/4105476855 conn(0x7f54380b2390 msgr2=0x7f54380b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 shutdown_connections 2024-04-25T09:53:26.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.671+0000 7f543ef68640 1 -- 172.21.15.19:0/4105476855 wait complete. 2024-04-25T09:53:26.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.678+0000 7faba8ff9640 1 -- 172.21.15.19:0/3482942506 --> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fabb4154fe0 con 0x7fabb40b0370 2024-04-25T09:53:26.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.679+0000 7fabaaffd640 1 -- 172.21.15.19:0/3482942506 <== osd.4 v2:172.21.15.112:6800/1718655451 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fabb4154fe0 con 0x7fabb40b0370 2024-04-25T09:53:26.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.679+0000 7faba8ff9640 1 -- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x7fabb40b0370 msgr2=0x7fabb40b2810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.679+0000 7faba8ff9640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x7fabb40b0370 0x7fabb40b2810 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.680+0000 7faba8ff9640 1 -- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab9806b5a0 msgr2=0x7fab9806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.680+0000 7faba8ff9640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab9806b5a0 0x7fab9806da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fabb40aef10 tx=0x7fabac007d10 comp rx=0 tx=0).stop 2024-04-25T09:53:26.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.680+0000 7faba8ff9640 1 -- 172.21.15.19:0/3482942506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabb415e290 msgr2=0x7fabb40a8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.680+0000 7faba8ff9640 1 --2- 172.21.15.19:0/3482942506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabb415e290 0x7fabb40a8e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faba400c550 tx=0x7faba400ca10 comp rx=0 tx=0).stop 2024-04-25T09:53:26.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.680+0000 7fabbaa99640 1 -- 172.21.15.19:0/3482942506 reap_dead start 2024-04-25T09:53:26.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.680+0000 7faba8ff9640 1 -- 172.21.15.19:0/3482942506 shutdown_connections 2024-04-25T09:53:26.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.680+0000 7faba8ff9640 1 -- 172.21.15.19:0/3482942506 >> 172.21.15.19:0/3482942506 conn(0x7fabb406abe0 msgr2=0x7fabb406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.680+0000 7faba8ff9640 1 -- 172.21.15.19:0/3482942506 shutdown_connections 2024-04-25T09:53:26.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.680+0000 7faba8ff9640 1 -- 172.21.15.19:0/3482942506 wait complete. 2024-04-25T09:53:26.698 INFO:teuthology.orchestra.run.smithi019.stdout:55834574857 2024-04-25T09:53:26.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T09:53:26.701 INFO:teuthology.orchestra.run.smithi019.stdout:55834574857 2024-04-25T09:53:26.702 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T09:53:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.843+0000 7f3addad2640 1 Processor -- start 2024-04-25T09:53:26.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.843+0000 7f3addad2640 1 -- start start 2024-04-25T09:53:26.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.845+0000 7f3addad2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ad8071720 0x7f3ad80cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.845+0000 7f3addad2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad80b2400 con 0x7f3ad81599e0 2024-04-25T09:53:26.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.845+0000 7f3addad2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad80b26e0 con 0x7f3ad80cecf0 2024-04-25T09:53:26.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.845+0000 7f3addad2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80b29c0 con 0x7f3ad8071720 2024-04-25T09:53:26.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.845+0000 7f3adcad0640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3ad81599e0 0x7f3ad8071010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46910/0 (socket says 172.21.15.19:46910) 2024-04-25T09:53:26.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.845+0000 7f3adcad0640 1 -- 172.21.15.19:0/2119690202 learned_addr learned my addr 172.21.15.19:0/2119690202 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:26.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.845+0000 7f3ad7fff640 1 --2- 172.21.15.19:0/2119690202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ad8071720 0x7f3ad80cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.846+0000 7f3ad77fe640 1 -- 172.21.15.19:0/2119690202 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583745370 0 0) 0x7f3ad80b2400 con 0x7f3ad81599e0 2024-04-25T09:53:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.846+0000 7f3ad7fff640 1 -- 172.21.15.19:0/2119690202 >> v1:172.21.15.112:6789/0 conn(0x7f3ad80cecf0 legacy=0x7f3ad80cca40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:53:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.846+0000 7f3ad7fff640 1 -- 172.21.15.19:0/2119690202 >> v1:172.21.15.19:6789/0 conn(0x7f3ad81599e0 legacy=0x7f3ad8071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.846+0000 7f3ad7fff640 1 -- 172.21.15.19:0/2119690202 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad80b2ca0 con 0x7f3ad8071720 2024-04-25T09:53:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.847+0000 7f3ad7fff640 1 --2- 172.21.15.19:0/2119690202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ad8071720 0x7f3ad80cc500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000bec0 tx=0x7f3ad002c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a00fbd9652c792 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.849+0000 7f3ad77fe640 1 -- 172.21.15.19:0/2119690202 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad003a070 con 0x7f3ad8071720 2024-04-25T09:53:26.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.849+0000 7f3ad77fe640 1 -- 172.21.15.19:0/2119690202 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad0032480 con 0x7f3ad8071720 2024-04-25T09:53:26.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.850+0000 7f3addad2640 1 -- 172.21.15.19:0/2119690202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ad8071720 msgr2=0x7f3ad80cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.850+0000 7f3addad2640 1 --2- 172.21.15.19:0/2119690202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ad8071720 0x7f3ad80cc500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000bec0 tx=0x7f3ad002c710 comp rx=0 tx=0).stop 2024-04-25T09:53:26.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.857+0000 7f3addad2640 1 -- 172.21.15.19:0/2119690202 shutdown_connections 2024-04-25T09:53:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.857+0000 7f3addad2640 1 --2- 172.21.15.19:0/2119690202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ad8071720 0x7f3ad80cc500 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.857+0000 7f3addad2640 1 -- 172.21.15.19:0/2119690202 >> 172.21.15.19:0/2119690202 conn(0x7f3ad806adc0 msgr2=0x7f3ad806b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.868+0000 7f3addad2640 1 -- 172.21.15.19:0/2119690202 shutdown_connections 2024-04-25T09:53:26.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.881+0000 7f3addad2640 1 -- 172.21.15.19:0/2119690202 wait complete. 2024-04-25T09:53:26.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.883+0000 7f3addad2640 1 Processor -- start 2024-04-25T09:53:26.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.886+0000 7f3addad2640 1 -- start start 2024-04-25T09:53:26.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.887+0000 7f3addad2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ad8071720 0x7f3ad815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.888+0000 7f3addad2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ad80cecf0 0x7f3ad81440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.888+0000 7f3ad7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ad80cecf0 0x7f3ad81440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.888+0000 7f3ad7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ad80cecf0 0x7f3ad81440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35818/0 (socket says 172.21.15.19:35818) 2024-04-25T09:53:26.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.888+0000 7f3ad7fff640 1 -- 172.21.15.19:0/3501346707 learned_addr learned my addr 172.21.15.19:0/3501346707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.889+0000 7f3addad2640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ad81599e0 0x7f3ad8140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.889+0000 7f3addad2640 1 -- 172.21.15.19:0/3501346707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80b39b0 con 0x7f3ad80cecf0 2024-04-25T09:53:26.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.889+0000 7f3addad2640 1 -- 172.21.15.19:0/3501346707 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80b34a0 con 0x7f3ad8071720 2024-04-25T09:53:26.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.889+0000 7f3addad2640 1 -- 172.21.15.19:0/3501346707 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80b3b30 con 0x7f3ad81599e0 2024-04-25T09:53:26.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.889+0000 7f3ad7fff640 1 -- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ad81599e0 msgr2=0x7f3ad8140d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T09:53:26.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.889+0000 7f3ad7fff640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ad81599e0 0x7f3ad8140d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.890+0000 7f3ad7fff640 1 -- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ad8071720 msgr2=0x7f3ad815b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T09:53:26.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.890+0000 7f3ad7fff640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ad8071720 0x7f3ad815b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.890+0000 7f3ad7fff640 1 -- 172.21.15.19:0/3501346707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad815a350 con 0x7f3ad80cecf0 2024-04-25T09:53:26.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.890+0000 7f3ad7fff640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ad80cecf0 0x7f3ad81440b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000d050 tx=0x7f3ad0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.893+0000 7f3ad57fa640 1 -- 172.21.15.19:0/3501346707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0044070 con 0x7f3ad80cecf0 2024-04-25T09:53:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.893+0000 7f3addad2640 1 -- 172.21.15.19:0/3501346707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad815a630 con 0x7f3ad80cecf0 2024-04-25T09:53:26.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.893+0000 7f3addad2640 1 -- 172.21.15.19:0/3501346707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad815abf0 con 0x7f3ad80cecf0 2024-04-25T09:53:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.897+0000 7f3ad57fa640 1 -- 172.21.15.19:0/3501346707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad0035c40 con 0x7f3ad80cecf0 2024-04-25T09:53:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.897+0000 7f3ad57fa640 1 -- 172.21.15.19:0/3501346707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad003a040 con 0x7f3ad80cecf0 2024-04-25T09:53:26.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.897+0000 7f3ad57fa640 1 -- 172.21.15.19:0/3501346707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ad0047070 con 0x7f3ad80cecf0 2024-04-25T09:53:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.904+0000 7f3ad57fa640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ac406b5d0 0x7f3ac406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.904+0000 7f3ad57fa640 1 -- 172.21.15.19:0/3501346707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ad00adda0 con 0x7f3ad80cecf0 2024-04-25T09:53:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.908+0000 7f3adcad0640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ac406b5d0 0x7f3ac406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.909+0000 7f3abeffd640 1 -- 172.21.15.19:0/3501346707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab8005290 con 0x7f3ad80cecf0 2024-04-25T09:53:26.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.915+0000 7fc1451a5640 1 Processor -- start 2024-04-25T09:53:26.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.915+0000 7fc1451a5640 1 -- start start 2024-04-25T09:53:26.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.916+0000 7fc1451a5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc138009180 0x7fc138052ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.916+0000 7fc1451a5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc138049060 0x7fc1380eb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.916+0000 7fc1451a5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc138009560 con 0x7fc138048c30 2024-04-25T09:53:26.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.916+0000 7fc1451a5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc138035960 con 0x7fc138009180 2024-04-25T09:53:26.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.916+0000 7fc1451a5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc138035c20 con 0x7fc138049060 2024-04-25T09:53:26.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.917+0000 7fc13e59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc138048c30 0x7fc1380eb290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46914/0 (socket says 172.21.15.19:46914) 2024-04-25T09:53:26.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.917+0000 7fc13e59c640 1 -- 172.21.15.19:0/214311892 learned_addr learned my addr 172.21.15.19:0/214311892 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:26.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.917+0000 7fc13ed9d640 1 --2- 172.21.15.19:0/214311892 >> v2:172.21.15.19:3300/0 conn(0x7fc138009180 0x7fc138052ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.918+0000 7fc13f59e640 1 --2- 172.21.15.19:0/214311892 >> v2:172.21.15.112:3300/0 conn(0x7fc138049060 0x7fc1380eb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.918+0000 7f3adcad0640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ac406b5d0 0x7f3ac406da90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8000c00 tx=0x7f3ac80079c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.919+0000 7f3ad57fa640 1 -- 172.21.15.19:0/3501346707 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ad0006e20 con 0x7f3ad80cecf0 2024-04-25T09:53:26.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.919+0000 7fc13ed9d640 1 -- 172.21.15.19:0/214311892 >> v2:172.21.15.112:3300/0 conn(0x7fc138049060 msgr2=0x7fc1380eb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.919+0000 7fc13ed9d640 1 --2- 172.21.15.19:0/214311892 >> v2:172.21.15.112:3300/0 conn(0x7fc138049060 0x7fc1380eb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.919+0000 7fc13ed9d640 1 -- 172.21.15.19:0/214311892 >> v1:172.21.15.19:6789/0 conn(0x7fc138048c30 legacy=0x7fc1380eb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.919+0000 7fc13ed9d640 1 -- 172.21.15.19:0/214311892 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc138035f00 con 0x7fc138009180 2024-04-25T09:53:26.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.920+0000 7f063cd89640 1 Processor -- start 2024-04-25T09:53:26.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.920+0000 7fc13ed9d640 1 --2- 172.21.15.19:0/214311892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc138009180 0x7fc138052ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc13804ccc0 tx=0x7fc13002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a62362c821e657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.920+0000 7fc13dd9b640 1 -- 172.21.15.19:0/214311892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130037070 con 0x7fc138009180 2024-04-25T09:53:26.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.921+0000 7fc13dd9b640 1 -- 172.21.15.19:0/214311892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc130032440 con 0x7fc138009180 2024-04-25T09:53:26.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.922+0000 7fc1451a5640 1 -- 172.21.15.19:0/214311892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc138009180 msgr2=0x7fc138052ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.922+0000 7fc1451a5640 1 --2- 172.21.15.19:0/214311892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc138009180 0x7fc138052ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc13804ccc0 tx=0x7fc13002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.920+0000 7f063cd89640 1 -- start start 2024-04-25T09:53:26.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.922+0000 7f063cd89640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0630048100 0x7f0630006f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.922+0000 7f063cd89640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0630007460 0x7f0630053e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.922+0000 7f063cd89640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06300547a0 con 0x7f0630054390 2024-04-25T09:53:26.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.922+0000 7f063cd89640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06300e7b90 con 0x7f0630007460 2024-04-25T09:53:26.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.922+0000 7f063cd89640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0630006a90 con 0x7f0630048100 2024-04-25T09:53:26.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.922+0000 7f063659c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0630048100 0x7f0630006f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.922+0000 7f0636d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0630054390 0x7f063004f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46928/0 (socket says 172.21.15.19:46928) 2024-04-25T09:53:26.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.922+0000 7f0636d9d640 1 -- 172.21.15.19:0/1376436049 learned_addr learned my addr 172.21.15.19:0/1376436049 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:26.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.922+0000 7f0635d9b640 1 --2- 172.21.15.19:0/1376436049 >> v2:172.21.15.19:3300/0 conn(0x7f0630007460 0x7f0630053e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.923+0000 7f61224e4640 1 Processor -- start 2024-04-25T09:53:26.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.923+0000 7f61224e4640 1 -- start start 2024-04-25T09:53:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.924+0000 7f61224e4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f611c0b3120 0x7f611c16cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.924+0000 7f61224e4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611c166890 con 0x7f611c166410 2024-04-25T09:53:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.924+0000 7f61224e4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611c162bb0 con 0x7f611c0cfc30 2024-04-25T09:53:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.924+0000 7f61224e4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611c1592e0 con 0x7f611c0b3120 2024-04-25T09:53:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.925+0000 7fc1451a5640 1 -- 172.21.15.19:0/214311892 shutdown_connections 2024-04-25T09:53:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.925+0000 7fc1451a5640 1 --2- 172.21.15.19:0/214311892 >> v2:172.21.15.112:3300/0 conn(0x7fc138049060 0x7fc1380eb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.925+0000 7fc1451a5640 1 --2- 172.21.15.19:0/214311892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc138009180 0x7fc138052ef0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.925+0000 7fc1451a5640 1 -- 172.21.15.19:0/214311892 >> 172.21.15.19:0/214311892 conn(0x7fc13801ef30 msgr2=0x7fc138055c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.925+0000 7f6120a81640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f611c0b3120 0x7f611c16cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.925+0000 7f6120a81640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f611c0b3120 0x7f611c16cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35838/0 (socket says 172.21.15.19:35838) 2024-04-25T09:53:26.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.925+0000 7f6120a81640 1 -- 172.21.15.19:0/1926009077 learned_addr learned my addr 172.21.15.19:0/1926009077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.924+0000 7f0635d9b640 1 -- 172.21.15.19:0/1376436049 >> v2:172.21.15.112:3300/0 conn(0x7f0630048100 msgr2=0x7f0630006f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.924+0000 7f0635d9b640 1 --2- 172.21.15.19:0/1376436049 >> v2:172.21.15.112:3300/0 conn(0x7f0630048100 0x7f0630006f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.924+0000 7f0635d9b640 1 -- 172.21.15.19:0/1376436049 >> v1:172.21.15.19:6789/0 conn(0x7f0630054390 legacy=0x7f063004f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.924+0000 7f0635d9b640 1 -- 172.21.15.19:0/1376436049 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0630005ec0 con 0x7f0630007460 2024-04-25T09:53:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.925+0000 7f0635d9b640 1 --2- 172.21.15.19:0/1376436049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0630007460 0x7f0630053e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06300525e0 tx=0x7f062c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988dcc7b28a2b63a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.923+0000 7f063559a640 1 -- 172.21.15.19:0/1376436049 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912799020 0 0) 0x7f06300547a0 con 0x7f0630054390 2024-04-25T09:53:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.926+0000 7f063559a640 1 -- 172.21.15.19:0/1376436049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c037070 con 0x7f0630007460 2024-04-25T09:53:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.926+0000 7f063559a640 1 -- 172.21.15.19:0/1376436049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c032440 con 0x7f0630007460 2024-04-25T09:53:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.926+0000 7f063559a640 1 -- 172.21.15.19:0/1376436049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c032810 con 0x7f0630007460 2024-04-25T09:53:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.926+0000 7f6120a81640 1 -- 172.21.15.19:0/1926009077 >> v1:172.21.15.112:6789/0 conn(0x7f611c0cfc30 legacy=0x7f611c1623a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:53:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.926+0000 7f6120a81640 1 -- 172.21.15.19:0/1926009077 >> v1:172.21.15.19:6789/0 conn(0x7f611c166410 legacy=0x7f611c0b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.926+0000 7f6120a81640 1 -- 172.21.15.19:0/1926009077 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c162e90 con 0x7f611c0b3120 2024-04-25T09:53:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.926+0000 7fc1451a5640 1 -- 172.21.15.19:0/214311892 shutdown_connections 2024-04-25T09:53:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.926+0000 7fc1451a5640 1 -- 172.21.15.19:0/214311892 wait complete. 2024-04-25T09:53:26.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.927+0000 7f6120a81640 1 --2- 172.21.15.19:0/1926009077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c0b3120 0x7f611c16cb20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6114009e50 tx=0x7f611402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7e1605b7b936a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.927+0000 7f611affd640 1 -- 172.21.15.19:0/1926009077 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114036070 con 0x7f611c0b3120 2024-04-25T09:53:26.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.927+0000 7f611affd640 1 -- 172.21.15.19:0/1926009077 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6114031600 con 0x7f611c0b3120 2024-04-25T09:53:26.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.928+0000 7f61224e4640 1 -- 172.21.15.19:0/1926009077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c0b3120 msgr2=0x7f611c16cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.928+0000 7f61224e4640 1 --2- 172.21.15.19:0/1926009077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c0b3120 0x7f611c16cb20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6114009e50 tx=0x7f611402a680 comp rx=0 tx=0).stop 2024-04-25T09:53:26.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.928+0000 7fc1451a5640 1 Processor -- start 2024-04-25T09:53:26.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.929+0000 7fc1451a5640 1 -- start start 2024-04-25T09:53:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.929+0000 7f61224e4640 1 -- 172.21.15.19:0/1926009077 shutdown_connections 2024-04-25T09:53:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.929+0000 7f61224e4640 1 --2- 172.21.15.19:0/1926009077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c0b3120 0x7f611c16cb20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.929+0000 7f61224e4640 1 -- 172.21.15.19:0/1926009077 >> 172.21.15.19:0/1926009077 conn(0x7f611c06abe0 msgr2=0x7f611c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.929+0000 7f61224e4640 1 -- 172.21.15.19:0/1926009077 shutdown_connections 2024-04-25T09:53:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.929+0000 7f063cd89640 1 -- 172.21.15.19:0/1376436049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0630007460 msgr2=0x7f0630053e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.929+0000 7f063cd89640 1 --2- 172.21.15.19:0/1376436049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0630007460 0x7f0630053e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06300525e0 tx=0x7f062c02a270 comp rx=0 tx=0).stop 2024-04-25T09:53:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7f063cd89640 1 -- 172.21.15.19:0/1376436049 shutdown_connections 2024-04-25T09:53:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7f063cd89640 1 --2- 172.21.15.19:0/1376436049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0630007460 0x7f0630053e50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7f063cd89640 1 --2- 172.21.15.19:0/1376436049 >> v2:172.21.15.112:3300/0 conn(0x7f0630048100 0x7f0630006f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7f063cd89640 1 -- 172.21.15.19:0/1376436049 >> 172.21.15.19:0/1376436049 conn(0x7f0630009fe0 msgr2=0x7f063000a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.930+0000 7fc1451a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138009180 0x7fc138008430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.930+0000 7fc1451a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc138048c30 0x7fc1380d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.930+0000 7fc1451a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc138049060 0x7fc1380512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.930+0000 7fc13ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138009180 0x7fc138008430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.930+0000 7fc13ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138009180 0x7fc138008430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:51920/0 (socket says 172.21.15.19:51920) 2024-04-25T09:53:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.930+0000 7fc13ed9d640 1 -- 172.21.15.19:0/3588993294 learned_addr learned my addr 172.21.15.19:0/3588993294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.930+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc138048c30 0x7fc1380d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.930+0000 7fc13f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc138049060 0x7fc1380512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1380367a0 con 0x7fc138049060 2024-04-25T09:53:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1380364c0 con 0x7fc138009180 2024-04-25T09:53:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc138054090 con 0x7fc138048c30 2024-04-25T09:53:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7fc13ed9d640 1 -- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc138048c30 msgr2=0x7fc1380d8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7fc13ed9d640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc138048c30 0x7fc1380d8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7fc13ed9d640 1 -- 172.21.15.19:0/3588993294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc138049060 msgr2=0x7fc1380512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7fc13ed9d640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc138049060 0x7fc1380512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7fc13ed9d640 1 -- 172.21.15.19:0/3588993294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc138052a10 con 0x7fc138009180 2024-04-25T09:53:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7fc13f59e640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc138049060 0x7fc1380512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.932+0000 7fc13ed9d640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138009180 0x7fc138008430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc13002a7b0 tx=0x7fc1300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.929+0000 7f61224e4640 1 -- 172.21.15.19:0/1926009077 wait complete. 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7f61224e4640 1 Processor -- start 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.931+0000 7f61224e4640 1 -- start start 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.932+0000 7f61224e4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c0b3120 0x7f611c089920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.932+0000 7f61224e4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f611c0cfc30 0x7f611c0ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.932+0000 7f61224e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c166410 0x7f611c0ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.932+0000 7f61224e4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c06e740 con 0x7f611c0b3120 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.932+0000 7f61224e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c06d210 con 0x7f611c166410 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.932+0000 7f61224e4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611c06e8c0 con 0x7f611c0cfc30 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.933+0000 7f611bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c0b3120 0x7f611c089920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.933+0000 7f611bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c0b3120 0x7f611c089920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35858/0 (socket says 172.21.15.19:35858) 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.933+0000 7f611bfff640 1 -- 172.21.15.19:0/4054247919 learned_addr learned my addr 172.21.15.19:0/4054247919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.933+0000 7f611bfff640 1 -- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f611c0cfc30 msgr2=0x7f611c0ab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.933+0000 7f611bfff640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f611c0cfc30 0x7f611c0ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.933+0000 7f611bfff640 1 -- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c166410 msgr2=0x7f611c0ab910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T09:53:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.933+0000 7f611bfff640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c166410 0x7f611c0ab910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.933+0000 7f611bfff640 1 -- 172.21.15.19:0/4054247919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c0ad610 con 0x7f611c0b3120 2024-04-25T09:53:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.932+0000 7f063cd89640 1 -- 172.21.15.19:0/1376436049 shutdown_connections 2024-04-25T09:53:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.932+0000 7fc1277fe640 1 -- 172.21.15.19:0/3588993294 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130040020 con 0x7fc138009180 2024-04-25T09:53:26.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.932+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc138051e50 con 0x7fc138009180 2024-04-25T09:53:26.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.933+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1380d87b0 con 0x7fc138009180 2024-04-25T09:53:26.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.933+0000 7fc1277fe640 1 -- 172.21.15.19:0/3588993294 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc130038040 con 0x7fc138009180 2024-04-25T09:53:26.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.933+0000 7fc1277fe640 1 -- 172.21.15.19:0/3588993294 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130037070 con 0x7fc138009180 2024-04-25T09:53:26.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.934+0000 7f611bfff640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c0b3120 0x7f611c089920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f611000b450 tx=0x7f611000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.934+0000 7f6118ff9640 1 -- 172.21.15.19:0/4054247919 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110015020 con 0x7f611c0b3120 2024-04-25T09:53:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.934+0000 7f61224e4640 1 -- 172.21.15.19:0/4054247919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611c08a300 con 0x7f611c0b3120 2024-04-25T09:53:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.932+0000 7f063cd89640 1 -- 172.21.15.19:0/1376436049 wait complete. 2024-04-25T09:53:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.935+0000 7f063cd89640 1 Processor -- start 2024-04-25T09:53:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.935+0000 7f063cd89640 1 -- start start 2024-04-25T09:53:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7fc1277fe640 1 -- 172.21.15.19:0/3588993294 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc130043020 con 0x7fc138009180 2024-04-25T09:53:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.935+0000 7fe2e237e640 1 Processor -- start 2024-04-25T09:53:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.935+0000 7fe2e237e640 1 -- start start 2024-04-25T09:53:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7f063cd89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0630007460 0x7f06300d4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7f063cd89640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0630048100 0x7f06300e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7f063cd89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0630054390 0x7f06300e8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7f063cd89640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630006440 con 0x7f0630048100 2024-04-25T09:53:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7f063cd89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630006160 con 0x7f0630007460 2024-04-25T09:53:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7f063cd89640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0630006720 con 0x7f0630054390 2024-04-25T09:53:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7f0636d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0630054390 0x7f06300e8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7f0636d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0630054390 0x7f06300e8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55708/0 (socket says 172.21.15.19:55708) 2024-04-25T09:53:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7f0636d9d640 1 -- 172.21.15.19:0/2088597109 learned_addr learned my addr 172.21.15.19:0/2088597109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7fc1277fe640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc12c06b5a0 0x7fc12c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7f063659c640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0630007460 0x7f06300d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7fe2e237e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d40f01d0 0x7fe2d40f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7fe2e237e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d4052630 con 0x7fe2d4033f80 2024-04-25T09:53:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7fe2e237e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d4052910 con 0x7fe2d4037c20 2024-04-25T09:53:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7fe2e237e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d4053520 con 0x7fe2d40f01d0 2024-04-25T09:53:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7fe2e091b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d40f01d0 0x7fe2d40f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7fe2dbfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe2d4033f80 0x7fe2d40539b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46934/0 (socket says 172.21.15.19:46934) 2024-04-25T09:53:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7fe2dbfff640 1 -- 172.21.15.19:0/279519528 learned_addr learned my addr 172.21.15.19:0/279519528 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.935+0000 7f6118ff9640 1 -- 172.21.15.19:0/4054247919 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61100118b0 con 0x7f611c0b3120 2024-04-25T09:53:26.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.935+0000 7f61224e4640 1 -- 172.21.15.19:0/4054247919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c08a8c0 con 0x7f611c0b3120 2024-04-25T09:53:26.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7fc1277fe640 1 -- 172.21.15.19:0/3588993294 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc1300aab00 con 0x7fc138009180 2024-04-25T09:53:26.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.936+0000 7f6118ff9640 1 -- 172.21.15.19:0/4054247919 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110011ca0 con 0x7f611c0b3120 2024-04-25T09:53:26.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7f61027fc640 1 -- 172.21.15.19:0/4054247919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60fc005290 con 0x7f611c0b3120 2024-04-25T09:53:26.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7f0636d9d640 1 -- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0630007460 msgr2=0x7f06300d4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7f0636d9d640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0630007460 0x7f06300d4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7f0636d9d640 1 -- 172.21.15.19:0/2088597109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0630048100 msgr2=0x7f06300e7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7f0636d9d640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0630048100 0x7f06300e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7f0636d9d640 1 -- 172.21.15.19:0/2088597109 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0630036420 con 0x7f0630054390 2024-04-25T09:53:26.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.938+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc120005290 con 0x7fc138009180 2024-04-25T09:53:26.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.938+0000 7fc13e59c640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc12c06b5a0 0x7fc12c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.938+0000 7f0636d9d640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0630054390 0x7f06300e8250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0638062780 tx=0x7f06380686c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.938+0000 7f0626ffd640 1 -- 172.21.15.19:0/2088597109 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638073280 con 0x7f0630054390 2024-04-25T09:53:26.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.938+0000 7f0626ffd640 1 -- 172.21.15.19:0/2088597109 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063808f020 con 0x7f0630054390 2024-04-25T09:53:26.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.938+0000 7f0626ffd640 1 -- 172.21.15.19:0/2088597109 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063806abe0 con 0x7f0630054390 2024-04-25T09:53:26.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7fe2e091b640 1 -- 172.21.15.19:0/279519528 >> v1:172.21.15.112:6789/0 conn(0x7fe2d4037c20 legacy=0x7fe2d40380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7fe2e091b640 1 -- 172.21.15.19:0/279519528 >> v1:172.21.15.19:6789/0 conn(0x7fe2d4033f80 legacy=0x7fe2d40539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7fe2e091b640 1 -- 172.21.15.19:0/279519528 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d4052bf0 con 0x7fe2d40f01d0 2024-04-25T09:53:26.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7f2ebffff640 1 Processor -- start 2024-04-25T09:53:26.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.937+0000 7f2ebffff640 1 -- start start 2024-04-25T09:53:26.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.938+0000 7f2ebffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2eb80e12b0 0x7f2eb80534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.938+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb8054b50 0x7f2eb8053a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7f2ebffff640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb8054fe0 con 0x7f2eb80e4d30 2024-04-25T09:53:26.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7f2ebffff640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb80552c0 con 0x7f2eb80e12b0 2024-04-25T09:53:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7f2ebffff640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80555a0 con 0x7f2eb8054b50 2024-04-25T09:53:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7f2ebe7fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb8054b50 0x7f2eb8053a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7f2ebe7fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb8054b50 0x7f2eb8053a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55718/0 (socket says 172.21.15.19:55718) 2024-04-25T09:53:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7f2ebe7fc640 1 -- 172.21.15.19:0/2847287283 learned_addr learned my addr 172.21.15.19:0/2847287283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.940+0000 7f6118ff9640 1 -- 172.21.15.19:0/4054247919 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6110018020 con 0x7f611c0b3120 2024-04-25T09:53:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.940+0000 7fe2e091b640 1 --2- 172.21.15.19:0/279519528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d40f01d0 0x7fe2d40f25c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc000b60 tx=0x7fe2dc0a5140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=181090c3eede7efe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7f2ebe7fc640 1 -- 172.21.15.19:0/2847287283 >> v2:172.21.15.19:3300/0 conn(0x7f2eb80e12b0 msgr2=0x7f2eb80534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7f2ebe7fc640 1 --2- 172.21.15.19:0/2847287283 >> v2:172.21.15.19:3300/0 conn(0x7f2eb80e12b0 0x7f2eb80534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7f2ebe7fc640 1 -- 172.21.15.19:0/2847287283 >> v1:172.21.15.19:6789/0 conn(0x7f2eb80e4d30 legacy=0x7f2eb80e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.939+0000 7f2ebe7fc640 1 -- 172.21.15.19:0/2847287283 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb8055880 con 0x7f2eb8054b50 2024-04-25T09:53:26.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.940+0000 7fe2daffd640 1 -- 172.21.15.19:0/279519528 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0b2070 con 0x7fe2d40f01d0 2024-04-25T09:53:26.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.940+0000 7fe2daffd640 1 -- 172.21.15.19:0/279519528 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc0ad440 con 0x7fe2d40f01d0 2024-04-25T09:53:26.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7f2ebe7fc640 1 --2- 172.21.15.19:0/2847287283 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb8054b50 0x7f2eb8053a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009870 tx=0x7f2eb002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e1d20ae3c252ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7f2ebdffb640 1 -- 172.21.15.19:0/2847287283 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0037070 con 0x7f2eb8054b50 2024-04-25T09:53:26.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7f2ebdffb640 1 -- 172.21.15.19:0/2847287283 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb0032440 con 0x7f2eb8054b50 2024-04-25T09:53:26.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7f2ebdffb640 1 -- 172.21.15.19:0/2847287283 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0032810 con 0x7f2eb8054b50 2024-04-25T09:53:26.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7fe2e237e640 1 -- 172.21.15.19:0/279519528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d40f01d0 msgr2=0x7fe2d40f25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7fe2e237e640 1 --2- 172.21.15.19:0/279519528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d40f01d0 0x7fe2d40f25c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc000b60 tx=0x7fe2dc0a5140 comp rx=0 tx=0).stop 2024-04-25T09:53:26.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7fe2e237e640 1 -- 172.21.15.19:0/279519528 shutdown_connections 2024-04-25T09:53:26.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7fe2e237e640 1 --2- 172.21.15.19:0/279519528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d40f01d0 0x7fe2d40f25c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7fe2e237e640 1 -- 172.21.15.19:0/279519528 >> 172.21.15.19:0/279519528 conn(0x7fe2d4009fe0 msgr2=0x7fe2d400a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7f6118ff9640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f610806b5a0 0x7f610806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7fe2e237e640 1 -- 172.21.15.19:0/279519528 shutdown_connections 2024-04-25T09:53:26.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.941+0000 7fe2e237e640 1 -- 172.21.15.19:0/279519528 wait complete. 2024-04-25T09:53:26.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.942+0000 7f6118ff9640 1 -- 172.21.15.19:0/4054247919 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f611008ac20 con 0x7f611c0b3120 2024-04-25T09:53:26.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.942+0000 7f611b7fe640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f610806b5a0 0x7f610806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.942+0000 7f2ebffff640 1 -- 172.21.15.19:0/2847287283 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb8054b50 msgr2=0x7f2eb8053a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.942+0000 7f2ebffff640 1 --2- 172.21.15.19:0/2847287283 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb8054b50 0x7f2eb8053a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009870 tx=0x7f2eb002a210 comp rx=0 tx=0).stop 2024-04-25T09:53:26.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.942+0000 7f2ebffff640 1 -- 172.21.15.19:0/2847287283 shutdown_connections 2024-04-25T09:53:26.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.942+0000 7f2ebffff640 1 --2- 172.21.15.19:0/2847287283 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb8054b50 0x7f2eb8053a30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.942+0000 7f2ebffff640 1 --2- 172.21.15.19:0/2847287283 >> v2:172.21.15.19:3300/0 conn(0x7f2eb80e12b0 0x7f2eb80534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.942+0000 7f2ebffff640 1 -- 172.21.15.19:0/2847287283 >> 172.21.15.19:0/2847287283 conn(0x7f2eb8020040 msgr2=0x7f2eb8056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.943+0000 7f611b7fe640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f610806b5a0 0x7f610806da60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f611c0ad400 tx=0x7f610c006ba0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.943+0000 7fc13e59c640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc12c06b5a0 0x7fc12c06da60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc134000f60 tx=0x7fc134009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.943+0000 7f063cd89640 1 -- 172.21.15.19:0/2088597109 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0630036140 con 0x7f0630054390 2024-04-25T09:53:26.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.943+0000 7f063cd89640 1 -- 172.21.15.19:0/2088597109 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0630049100 con 0x7f0630054390 2024-04-25T09:53:26.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.943+0000 7f2ebffff640 1 -- 172.21.15.19:0/2847287283 shutdown_connections 2024-04-25T09:53:26.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.942+0000 7f1378ff1640 1 Processor -- start 2024-04-25T09:53:26.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.943+0000 7f1378ff1640 1 -- start start 2024-04-25T09:53:26.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.943+0000 7f2ebffff640 1 -- 172.21.15.19:0/2847287283 wait complete. 2024-04-25T09:53:26.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.943+0000 7fc1277fe640 1 -- 172.21.15.19:0/3588993294 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc13004a030 con 0x7fc138009180 2024-04-25T09:53:26.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.943+0000 7fe2e237e640 1 Processor -- start 2024-04-25T09:53:26.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7f063cd89640 1 -- 172.21.15.19:0/2088597109 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06300053c0 con 0x7f0630054390 2024-04-25T09:53:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.944+0000 7fe2e237e640 1 -- start start 2024-04-25T09:53:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7fe2e237e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2d4033f80 0x7fe2d40db710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7fe2e237e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d4037c20 0x7fe2d404e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7fe2e237e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2d40f01d0 0x7fe2d40e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7fe2e237e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d40540c0 con 0x7fe2d4033f80 2024-04-25T09:53:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7fe2e237e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d40531b0 con 0x7fe2d40f01d0 2024-04-25T09:53:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7fe2e237e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d40500c0 con 0x7fe2d4037c20 2024-04-25T09:53:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7fe2db7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d4037c20 0x7fe2d404e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7fe2db7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d4037c20 0x7fe2d404e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55726/0 (socket says 172.21.15.19:55726) 2024-04-25T09:53:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7fe2db7fe640 1 -- 172.21.15.19:0/2654755800 learned_addr learned my addr 172.21.15.19:0/2654755800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.946+0000 7fe2e091b640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2d40f01d0 0x7fe2d40e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.944+0000 7f1378ff1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f136c0ebb40 0x7f136c0552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.944+0000 7f1378ff1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c0387d0 con 0x7f136c0eb050 2024-04-25T09:53:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.944+0000 7f1378ff1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c056d80 con 0x7f136c006190 2024-04-25T09:53:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.944+0000 7f1378ff1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f136c057e20 con 0x7f136c0ebb40 2024-04-25T09:53:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.944+0000 7f13737fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f136c0eb050 0x7f136c0eb430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46958/0 (socket says 172.21.15.19:46958) 2024-04-25T09:53:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.944+0000 7f13737fe640 1 -- 172.21.15.19:0/25395159 learned_addr learned my addr 172.21.15.19:0/25395159 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7f1372ffd640 1 --2- 172.21.15.19:0/25395159 >> v2:172.21.15.19:3300/0 conn(0x7f136c0ebb40 0x7f136c0552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7f1372ffd640 1 -- 172.21.15.19:0/25395159 >> v1:172.21.15.112:6789/0 conn(0x7f136c006190 legacy=0x7f136c006600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7f1372ffd640 1 -- 172.21.15.19:0/25395159 >> v1:172.21.15.19:6789/0 conn(0x7f136c0eb050 legacy=0x7f136c0eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7f1372ffd640 1 -- 172.21.15.19:0/25395159 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c057060 con 0x7f136c0ebb40 2024-04-25T09:53:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.946+0000 7fe2db7fe640 1 -- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2d40f01d0 msgr2=0x7fe2d40e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.946+0000 7fe2db7fe640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2d40f01d0 0x7fe2d40e90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.946+0000 7fe2db7fe640 1 -- 172.21.15.19:0/2654755800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2d4033f80 msgr2=0x7fe2d40db710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.946+0000 7fe2db7fe640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2d4033f80 0x7fe2d40db710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.946+0000 7fe2db7fe640 1 -- 172.21.15.19:0/2654755800 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d404c8b0 con 0x7fe2d4037c20 2024-04-25T09:53:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.946+0000 7fe2db7fe640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d4037c20 0x7fe2d404e4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800c480 tx=0x7fe2c800c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.947+0000 7fe2d8ff9640 1 -- 172.21.15.19:0/2654755800 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80052d0 con 0x7fe2d4037c20 2024-04-25T09:53:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.945+0000 7f2ebffff640 1 Processor -- start 2024-04-25T09:53:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.946+0000 7f2ebffff640 1 -- start start 2024-04-25T09:53:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.947+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eb80e12b0 0x7f2eb80d9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.947+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb80e4d30 0x7f2eb80e3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.947+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb80e3e40 0x7f2eb80df2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.947+0000 7f2ebffff640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb8057820 con 0x7f2eb80e12b0 2024-04-25T09:53:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.947+0000 7f2ebffff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80576a0 con 0x7f2eb80e4d30 2024-04-25T09:53:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.947+0000 7f2ebffff640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80e1da0 con 0x7f2eb80e3e40 2024-04-25T09:53:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7f2ebf7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb80e3e40 0x7f2eb80df2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7f2ebf7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb80e3e40 0x7f2eb80df2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55736/0 (socket says 172.21.15.19:55736) 2024-04-25T09:53:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7f2ebf7fe640 1 -- 172.21.15.19:0/3261933003 learned_addr learned my addr 172.21.15.19:0/3261933003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7f2ebe7fc640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb80e4d30 0x7f2eb80e3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.947+0000 7fe2e237e640 1 -- 172.21.15.19:0/2654755800 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d404cb90 con 0x7fe2d4037c20 2024-04-25T09:53:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.947+0000 7fe2e237e640 1 -- 172.21.15.19:0/2654755800 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d4049770 con 0x7fe2d4037c20 2024-04-25T09:53:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.947+0000 7fe2d8ff9640 1 -- 172.21.15.19:0/2654755800 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c8005470 con 0x7fe2d4037c20 2024-04-25T09:53:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7fe2d8ff9640 1 -- 172.21.15.19:0/2654755800 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8012870 con 0x7fe2d4037c20 2024-04-25T09:53:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7f2ebf7fe640 1 -- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb80e4d30 msgr2=0x7f2eb80e3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7f2ebf7fe640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb80e4d30 0x7f2eb80e3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7f2ebf7fe640 1 -- 172.21.15.19:0/3261933003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eb80e12b0 msgr2=0x7f2eb80d9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7f2ebf7fe640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eb80e12b0 0x7f2eb80d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7f2ebf7fe640 1 -- 172.21.15.19:0/3261933003 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80e43c0 con 0x7f2eb80e3e40 2024-04-25T09:53:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7f0626ffd640 1 -- 172.21.15.19:0/2088597109 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06380b4070 con 0x7f0630054390 2024-04-25T09:53:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7f2ebf7fe640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb80e3e40 0x7f2eb80df2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00697f0 tx=0x7f2ec00686c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.947+0000 7f1372ffd640 1 --2- 172.21.15.19:0/25395159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f136c0ebb40 0x7f136c0552f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f13600097a0 tx=0x7f136002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247ea234599c27c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.948+0000 7fe2e237e640 1 -- 172.21.15.19:0/2654755800 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc005290 con 0x7fe2d4037c20 2024-04-25T09:53:26.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.949+0000 7f6118ff9640 1 -- 172.21.15.19:0/4054247919 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f611001f020 con 0x7f611c0b3120 2024-04-25T09:53:26.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.949+0000 7f0626ffd640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061806b5d0 0x7f061806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.949+0000 7f0626ffd640 1 -- 172.21.15.19:0/2088597109 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0638105200 con 0x7f0630054390 2024-04-25T09:53:26.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.949+0000 7f13727fc640 1 -- 172.21.15.19:0/25395159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360037070 con 0x7f136c0ebb40 2024-04-25T09:53:26.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.949+0000 7f13727fc640 1 -- 172.21.15.19:0/25395159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136002ae50 con 0x7f136c0ebb40 2024-04-25T09:53:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.949+0000 7f063659c640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061806b5d0 0x7f061806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.949+0000 7f13727fc640 1 -- 172.21.15.19:0/25395159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360032660 con 0x7f136c0ebb40 2024-04-25T09:53:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.950+0000 7f063659c640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061806b5d0 0x7f061806da90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0628009b40 tx=0x7f0628009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.950+0000 7f1378ff1640 1 -- 172.21.15.19:0/25395159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f136c0ebb40 msgr2=0x7f136c0552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.950+0000 7f1378ff1640 1 --2- 172.21.15.19:0/25395159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f136c0ebb40 0x7f136c0552f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f13600097a0 tx=0x7f136002a140 comp rx=0 tx=0).stop 2024-04-25T09:53:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.950+0000 7f1378ff1640 1 -- 172.21.15.19:0/25395159 shutdown_connections 2024-04-25T09:53:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.950+0000 7f1378ff1640 1 --2- 172.21.15.19:0/25395159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f136c0ebb40 0x7f136c0552f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.950+0000 7f1378ff1640 1 -- 172.21.15.19:0/25395159 >> 172.21.15.19:0/25395159 conn(0x7f136c0559d0 msgr2=0x7f136c055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.950+0000 7f1378ff1640 1 -- 172.21.15.19:0/25395159 shutdown_connections 2024-04-25T09:53:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.950+0000 7f1378ff1640 1 -- 172.21.15.19:0/25395159 wait complete. 2024-04-25T09:53:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.952+0000 7f1378ff1640 1 Processor -- start 2024-04-25T09:53:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.952+0000 7f0626ffd640 1 -- 172.21.15.19:0/2088597109 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0638099030 con 0x7f0630054390 2024-04-25T09:53:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.952+0000 7f1378ff1640 1 -- start start 2024-04-25T09:53:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.952+0000 7f2ea77fe640 1 -- 172.21.15.19:0/3261933003 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0074610 con 0x7f2eb80e3e40 2024-04-25T09:53:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.953+0000 7f2ebffff640 1 -- 172.21.15.19:0/3261933003 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb80e46a0 con 0x7f2eb80e3e40 2024-04-25T09:53:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.953+0000 7f2ebffff640 1 -- 172.21.15.19:0/3261933003 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb804e050 con 0x7f2eb80e3e40 2024-04-25T09:53:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.953+0000 7f2ea77fe640 1 -- 172.21.15.19:0/3261933003 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec0081400 con 0x7f2eb80e3e40 2024-04-25T09:53:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.953+0000 7f2ea77fe640 1 -- 172.21.15.19:0/3261933003 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec00817d0 con 0x7f2eb80e3e40 2024-04-25T09:53:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.954+0000 7f1378ff1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f136c006190 0x7f136c04f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.954+0000 7f13737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f136c006190 0x7f136c04f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.954+0000 7f13737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f136c006190 0x7f136c04f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35904/0 (socket says 172.21.15.19:35904) 2024-04-25T09:53:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.954+0000 7f13737fe640 1 -- 172.21.15.19:0/1690452859 learned_addr learned my addr 172.21.15.19:0/1690452859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.954+0000 7f2ebffff640 1 -- 172.21.15.19:0/3261933003 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea0005290 con 0x7f2eb80e3e40 2024-04-25T09:53:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.954+0000 7f1378ff1640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f136c0eb050 0x7f136c0519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.954+0000 7fe2d8ff9640 1 -- 172.21.15.19:0/2654755800 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2c8012a10 con 0x7fe2d4037c20 2024-04-25T09:53:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.955+0000 7f1378ff1640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f136c0ebb40 0x7f136c051f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.955+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c057620 con 0x7f136c006190 2024-04-25T09:53:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.955+0000 7f1373fff640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f136c0ebb40 0x7f136c051f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.955+0000 7fe2d8ff9640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2c006b520 0x7fe2c006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.955+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c057990 con 0x7f136c0ebb40 2024-04-25T09:53:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.955+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0e1e00 con 0x7f136c0eb050 2024-04-25T09:53:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.955+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2c006b520 0x7fe2c006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.955+0000 7fe2d8ff9640 1 -- 172.21.15.19:0/2654755800 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe2c808aa70 con 0x7fe2d4037c20 2024-04-25T09:53:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.956+0000 7fe2dbfff640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2c006b520 0x7fe2c006d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00015f0 tx=0x7fe2d0007a40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.956+0000 7f1372ffd640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f136c0eb050 0x7f136c0519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.956+0000 7f13737fe640 1 -- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f136c0eb050 msgr2=0x7f136c0519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.956+0000 7f13737fe640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f136c0eb050 0x7f136c0519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.956+0000 7f13737fe640 1 -- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f136c0ebb40 msgr2=0x7f136c051f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.956+0000 7f13737fe640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f136c0ebb40 0x7f136c051f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.956+0000 7f13737fe640 1 -- 172.21.15.19:0/1690452859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c04d820 con 0x7f136c006190 2024-04-25T09:53:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.957+0000 7fe2d8ff9640 1 -- 172.21.15.19:0/2654755800 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2c8059720 con 0x7fe2d4037c20 2024-04-25T09:53:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.957+0000 7f13737fe640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f136c006190 0x7f136c04f370 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f136800c730 tx=0x7f136800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.957+0000 7f135ffff640 1 -- 172.21.15.19:0/1690452859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368017070 con 0x7f136c006190 2024-04-25T09:53:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.957+0000 7f135ffff640 1 -- 172.21.15.19:0/1690452859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1368005ab0 con 0x7f136c006190 2024-04-25T09:53:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.957+0000 7f135ffff640 1 -- 172.21.15.19:0/1690452859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368005e60 con 0x7f136c006190 2024-04-25T09:53:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.958+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c04db00 con 0x7f136c006190 2024-04-25T09:53:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.958+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c04e0c0 con 0x7f136c006190 2024-04-25T09:53:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.960+0000 7f2ea77fe640 1 -- 172.21.15.19:0/3261933003 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ec008a3f0 con 0x7f2eb80e3e40 2024-04-25T09:53:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.960+0000 7f2ea77fe640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eac06b4d0 0x7f2eac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.960+0000 7f2ea77fe640 1 -- 172.21.15.19:0/3261933003 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2ec00f4090 con 0x7f2eb80e3e40 2024-04-25T09:53:26.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.960+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1354005290 con 0x7f136c006190 2024-04-25T09:53:26.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.963+0000 7f135ffff640 1 -- 172.21.15.19:0/1690452859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f136801a070 con 0x7f136c006190 2024-04-25T09:53:26.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.963+0000 7f135ffff640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f135806b5a0 0x7f135806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.964+0000 7f2ebeffd640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eac06b4d0 0x7f2eac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.964+0000 7f2ea77fe640 1 -- 172.21.15.19:0/3261933003 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ec0090080 con 0x7f2eb80e3e40 2024-04-25T09:53:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.968+0000 7f2ebeffd640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eac06b4d0 0x7f2eac06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009710 tx=0x7f2eb4006c90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.970+0000 7f0092815640 1 Processor -- start 2024-04-25T09:53:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.968+0000 7f135ffff640 1 -- 172.21.15.19:0/1690452859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1368089c00 con 0x7f136c006190 2024-04-25T09:53:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.969+0000 7f1372ffd640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f135806b5a0 0x7f135806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.970+0000 7f135ffff640 1 -- 172.21.15.19:0/1690452859 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1368022020 con 0x7f136c006190 2024-04-25T09:53:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.970+0000 7f0092815640 1 -- start start 2024-04-25T09:53:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.971+0000 7f0092815640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f008c161ef0 0x7f008c0716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.971+0000 7f0092815640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f008c0ce7b0 0x7f008c0b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.971+0000 7f0092815640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008c0c6910 con 0x7f008c0cf590 2024-04-25T09:53:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.971+0000 7f0092815640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008c0c76d0 con 0x7f008c0ce7b0 2024-04-25T09:53:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.971+0000 7f0092815640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0c66c0 con 0x7f008c161ef0 2024-04-25T09:53:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.972+0000 7f008b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f008c0cf590 0x7f008c0ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46970/0 (socket says 172.21.15.19:46970) 2024-04-25T09:53:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.972+0000 7f008b7fe640 1 -- 172.21.15.19:0/3326945521 learned_addr learned my addr 172.21.15.19:0/3326945521 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.972+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279475993 0 0) 0x7f008c0c6910 con 0x7f008c0cf590 2024-04-25T09:53:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.972+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0078003520 con 0x7f008c0cf590 2024-04-25T09:53:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.972+0000 7f1372ffd640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f135806b5a0 0x7f135806da60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1360034000 tx=0x7f136002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2460090483 0 0) 0x7f0078003520 con 0x7f008c0cf590 2024-04-25T09:53:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f008c161ef0 msgr2=0x7f008c0716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 --2- 172.21.15.19:0/3326945521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f008c161ef0 0x7f008c0716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 >> v2:172.21.15.112:3300/0 conn(0x7f008c0ce7b0 msgr2=0x7f008c0b0e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:53:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 --2- 172.21.15.19:0/3326945521 >> v2:172.21.15.112:3300/0 conn(0x7f008c0ce7b0 0x7f008c0b0e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c0c6bf0 con 0x7f008c0cf590 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f007c001670 con 0x7f008c0cf590 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 >> v1:172.21.15.19:6789/0 conn(0x7f008c0cf590 legacy=0x7f008c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 --2- 172.21.15.19:0/3326945521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00780039b0 0x7f0078003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 --2- 172.21.15.19:0/3326945521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0078004800 0x7f0078004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 --2- 172.21.15.19:0/3326945521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00780052e0 0x7f007800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0c76d0 con 0x7f00780052e0 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0c66c0 con 0x7f00780039b0 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007800c210 con 0x7f0078004800 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.973+0000 7f0090db2640 1 --2- 172.21.15.19:0/3326945521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00780052e0 0x7f007800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.974+0000 7f0090db2640 1 -- 172.21.15.19:0/3326945521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0078004800 msgr2=0x7f0078004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.974+0000 7f0090db2640 1 --2- 172.21.15.19:0/3326945521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0078004800 0x7f0078004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.974+0000 7f0090db2640 1 -- 172.21.15.19:0/3326945521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00780039b0 msgr2=0x7f0078003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.974+0000 7f0090db2640 1 --2- 172.21.15.19:0/3326945521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00780039b0 0x7f0078003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.974+0000 7f0090db2640 1 -- 172.21.15.19:0/3326945521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0078003520 con 0x7f00780052e0 2024-04-25T09:53:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.975+0000 7f0090db2640 1 --2- 172.21.15.19:0/3326945521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00780052e0 0x7f007800b9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f008c0c8820 tx=0x7f0078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116fbaa6b6cf7b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.979+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074037070 con 0x7f00780052e0 2024-04-25T09:53:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.979+0000 7f008affd640 1 -- 172.21.15.19:0/3326945521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007402ac10 con 0x7f00780052e0 2024-04-25T09:53:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.980+0000 7f0092815640 1 -- 172.21.15.19:0/3326945521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00780052e0 msgr2=0x7f007800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.980+0000 7f0092815640 1 --2- 172.21.15.19:0/3326945521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00780052e0 0x7f007800b9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f008c0c8820 tx=0x7f0078005830 comp rx=0 tx=0).stop 2024-04-25T09:53:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.980+0000 7f0090db2640 1 -- 172.21.15.19:0/3326945521 reap_dead start 2024-04-25T09:53:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.980+0000 7f0092815640 1 -- 172.21.15.19:0/3326945521 shutdown_connections 2024-04-25T09:53:26.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.980+0000 7f0092815640 1 -- 172.21.15.19:0/3326945521 >> 172.21.15.19:0/3326945521 conn(0x7f008c06f500 msgr2=0x7f008c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.986+0000 7f0092815640 1 -- 172.21.15.19:0/3326945521 shutdown_connections 2024-04-25T09:53:26.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.988+0000 7f0092815640 1 -- 172.21.15.19:0/3326945521 wait complete. 2024-04-25T09:53:26.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.990+0000 7f0092815640 1 Processor -- start 2024-04-25T09:53:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.994+0000 7f0092815640 1 -- start start 2024-04-25T09:53:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.995+0000 7f0092815640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f00780052e0 0x7f008c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.995+0000 7f0092815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f008c150240 0x7f008c150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.995+0000 7f0092815640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f008c06ff50 0x7f008c150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.995+0000 7f008bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f00780052e0 0x7f008c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.995+0000 7f008bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f00780052e0 0x7f008c14fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55770/0 (socket says 172.21.15.19:55770) 2024-04-25T09:53:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.996+0000 7f008b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f008c150240 0x7f008c150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.996+0000 7f008b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f008c150240 0x7f008c150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:51982/0 (socket says 172.21.15.19:51982) 2024-04-25T09:53:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.996+0000 7f008bfff640 1 -- 172.21.15.19:0/2262828692 learned_addr learned my addr 172.21.15.19:0/2262828692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.996+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0b0b50 con 0x7f008c06ff50 2024-04-25T09:53:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.996+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0c71b0 con 0x7f008c150240 2024-04-25T09:53:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.996+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c1622d0 con 0x7f00780052e0 2024-04-25T09:53:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.996+0000 7f0090db2640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f008c06ff50 0x7f008c150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.997+0000 7f0090db2640 1 -- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f00780052e0 msgr2=0x7f008c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.997+0000 7f0090db2640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f00780052e0 0x7f008c14fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.997+0000 7f0090db2640 1 -- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f008c150240 msgr2=0x7f008c150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.997+0000 7f0090db2640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f008c150240 0x7f008c150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.997+0000 7f0090db2640 1 -- 172.21.15.19:0/2262828692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c144020 con 0x7f008c06ff50 2024-04-25T09:53:27.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:26.997+0000 7f0090db2640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f008c06ff50 0x7f008c150cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0074031000 tx=0x7f0074004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.000+0000 7f0088ff9640 1 -- 172.21.15.19:0/2262828692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074041070 con 0x7f008c06ff50 2024-04-25T09:53:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.000+0000 7f0088ff9640 1 -- 172.21.15.19:0/2262828692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007403d5f0 con 0x7f008c06ff50 2024-04-25T09:53:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.000+0000 7f0088ff9640 1 -- 172.21.15.19:0/2262828692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074037040 con 0x7f008c06ff50 2024-04-25T09:53:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.004+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c0b7540 con 0x7f008c06ff50 2024-04-25T09:53:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.004+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c14f500 con 0x7f008c06ff50 2024-04-25T09:53:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.006+0000 7f0088ff9640 1 -- 172.21.15.19:0/2262828692 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0074044070 con 0x7f008c06ff50 2024-04-25T09:53:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.007+0000 7f0088ff9640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006c06b5a0 0x7f006c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.007+0000 7f0088ff9640 1 -- 172.21.15.19:0/2262828692 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f007402ae00 con 0x7f008c06ff50 2024-04-25T09:53:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.007+0000 7f008bfff640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006c06b5a0 0x7f006c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.008+0000 7f008bfff640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006c06b5a0 0x7f006c06da60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0084009e20 tx=0x7f0084009470 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.010+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c067be0 con 0x7f008c06ff50 2024-04-25T09:53:27.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.021+0000 7f0088ff9640 1 -- 172.21.15.19:0/2262828692 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f008c067be0 con 0x7f008c06ff50 2024-04-25T09:53:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.301+0000 7f3abeffd640 1 -- 172.21.15.19:0/3501346707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3ab8005580 con 0x7f3ad80cecf0 2024-04-25T09:53:27.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.302+0000 7f3ad57fa640 1 -- 172.21.15.19:0/3501346707 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ad007ca70 con 0x7f3ad80cecf0 2024-04-25T09:53:27.304 INFO:teuthology.orchestra.run.smithi019.stdout:55834574857 2024-04-25T09:53:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.311+0000 7f3addad2640 1 -- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ac406b5d0 msgr2=0x7f3ac406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.311+0000 7f3addad2640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ac406b5d0 0x7f3ac406da90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8000c00 tx=0x7f3ac80079c0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.311+0000 7f3addad2640 1 -- 172.21.15.19:0/3501346707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ad80cecf0 msgr2=0x7f3ad81440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.311+0000 7f3addad2640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ad80cecf0 0x7f3ad81440b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000d050 tx=0x7f3ad0004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.311+0000 7f3addad2640 1 -- 172.21.15.19:0/3501346707 shutdown_connections 2024-04-25T09:53:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.311+0000 7f3addad2640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ac406b5d0 0x7f3ac406da90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.311+0000 7f3addad2640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ad81599e0 0x7f3ad8140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.311+0000 7f3addad2640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ad80cecf0 0x7f3ad81440b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.311+0000 7f3addad2640 1 --2- 172.21.15.19:0/3501346707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ad8071720 0x7f3ad815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.311+0000 7f3addad2640 1 -- 172.21.15.19:0/3501346707 >> 172.21.15.19:0/3501346707 conn(0x7f3ad806adc0 msgr2=0x7f3ad806b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.327+0000 7f3addad2640 1 -- 172.21.15.19:0/3501346707 shutdown_connections 2024-04-25T09:53:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.327+0000 7f3addad2640 1 -- 172.21.15.19:0/3501346707 wait complete. 2024-04-25T09:53:27.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-04-25T09:53:27.345 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:27.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.372+0000 7f61027fc640 1 -- 172.21.15.19:0/4054247919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f60fc005850 con 0x7f611c0b3120 2024-04-25T09:53:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.373+0000 7f6118ff9640 1 -- 172.21.15.19:0/4054247919 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f611000d040 con 0x7f611c0b3120 2024-04-25T09:53:27.375 INFO:teuthology.orchestra.run.smithi019.stdout:55834574856 2024-04-25T09:53:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.377+0000 7f61224e4640 1 -- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f610806b5a0 msgr2=0x7f610806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.377+0000 7f61224e4640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f610806b5a0 0x7f610806da60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f611c0ad400 tx=0x7f610c006ba0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.377+0000 7f61224e4640 1 -- 172.21.15.19:0/4054247919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c0b3120 msgr2=0x7f611c089920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.377+0000 7f61224e4640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c0b3120 0x7f611c089920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f611000b450 tx=0x7f611000b910 comp rx=0 tx=0).stop 2024-04-25T09:53:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.378+0000 7f61224e4640 1 -- 172.21.15.19:0/4054247919 shutdown_connections 2024-04-25T09:53:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.378+0000 7f61224e4640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f610806b5a0 0x7f610806da60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.378+0000 7f61224e4640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f611c166410 0x7f611c0ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.378+0000 7f61224e4640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f611c0cfc30 0x7f611c0ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.378+0000 7f61224e4640 1 --2- 172.21.15.19:0/4054247919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c0b3120 0x7f611c089920 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.378+0000 7f61224e4640 1 -- 172.21.15.19:0/4054247919 >> 172.21.15.19:0/4054247919 conn(0x7f611c06abe0 msgr2=0x7f611c0b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.378+0000 7f61224e4640 1 -- 172.21.15.19:0/4054247919 shutdown_connections 2024-04-25T09:53:27.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.379+0000 7f61224e4640 1 -- 172.21.15.19:0/4054247919 wait complete. 2024-04-25T09:53:27.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.392+0000 7f2ebffff640 1 -- 172.21.15.19:0/3261933003 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2ea0005850 con 0x7f2eb80e3e40 2024-04-25T09:53:27.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2024-04-25T09:53:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.395+0000 7f2ea77fe640 1 -- 172.21.15.19:0/3261933003 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ec007d030 con 0x7f2eb80e3e40 2024-04-25T09:53:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.395+0000 7f063cd89640 1 -- 172.21.15.19:0/2088597109 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0630052850 con 0x7f0630054390 2024-04-25T09:53:27.398 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:27.398 INFO:teuthology.orchestra.run.smithi019.stdout:55834574857 2024-04-25T09:53:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.396+0000 7f0626ffd640 1 -- 172.21.15.19:0/2088597109 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f06380d3db0 con 0x7f0630054390 2024-04-25T09:53:27.399 INFO:teuthology.orchestra.run.smithi019.stdout:55834574858 2024-04-25T09:53:27.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.398+0000 7f2ebffff640 1 -- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eac06b4d0 msgr2=0x7f2eac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.398+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eac06b4d0 0x7f2eac06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009710 tx=0x7f2eb4006c90 comp rx=0 tx=0).stop 2024-04-25T09:53:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.398+0000 7f2ebffff640 1 -- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb80e3e40 msgr2=0x7f2eb80df2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.398+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb80e3e40 0x7f2eb80df2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00697f0 tx=0x7f2ec00686c0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f2ebffff640 1 -- 172.21.15.19:0/3261933003 shutdown_connections 2024-04-25T09:53:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eac06b4d0 0x7f2eac06d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2eb80e3e40 0x7f2eb80df2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb80e4d30 0x7f2eb80e3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f2ebffff640 1 --2- 172.21.15.19:0/3261933003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eb80e12b0 0x7f2eb80d9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f2ebffff640 1 -- 172.21.15.19:0/3261933003 >> 172.21.15.19:0/3261933003 conn(0x7f2eb8020040 msgr2=0x7f2eb8052d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f2ebffff640 1 -- 172.21.15.19:0/3261933003 shutdown_connections 2024-04-25T09:53:27.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f2ebffff640 1 -- 172.21.15.19:0/3261933003 wait complete. 2024-04-25T09:53:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f063cd89640 1 -- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061806b5d0 msgr2=0x7f061806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f063cd89640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061806b5d0 0x7f061806da90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0628009b40 tx=0x7f0628009190 comp rx=0 tx=0).stop 2024-04-25T09:53:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f063cd89640 1 -- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0630054390 msgr2=0x7f06300e8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f063cd89640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0630054390 0x7f06300e8250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0638062780 tx=0x7f06380686c0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f063cd89640 1 -- 172.21.15.19:0/2088597109 shutdown_connections 2024-04-25T09:53:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f063cd89640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061806b5d0 0x7f061806da90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f063cd89640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0630054390 0x7f06300e8250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f063cd89640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0630048100 0x7f06300e7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f063cd89640 1 --2- 172.21.15.19:0/2088597109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0630007460 0x7f06300d4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.399+0000 7f063cd89640 1 -- 172.21.15.19:0/2088597109 >> 172.21.15.19:0/2088597109 conn(0x7f0630009fe0 msgr2=0x7f063004f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.400+0000 7f063cd89640 1 -- 172.21.15.19:0/2088597109 shutdown_connections 2024-04-25T09:53:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.400+0000 7f063cd89640 1 -- 172.21.15.19:0/2088597109 wait complete. 2024-04-25T09:53:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.402+0000 7fe2e237e640 1 -- 172.21.15.19:0/2654755800 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe2bc005850 con 0x7fe2d4037c20 2024-04-25T09:53:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.403+0000 7fe2d8ff9640 1 -- 172.21.15.19:0/2654755800 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe2c8059020 con 0x7fe2d4037c20 2024-04-25T09:53:27.405 INFO:teuthology.orchestra.run.smithi019.stdout:55834574858 2024-04-25T09:53:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.405+0000 7fe2e237e640 1 -- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2c006b520 msgr2=0x7fe2c006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.405+0000 7fe2e237e640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2c006b520 0x7fe2c006d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00015f0 tx=0x7fe2d0007a40 comp rx=0 tx=0).stop 2024-04-25T09:53:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.405+0000 7fe2e237e640 1 -- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d4037c20 msgr2=0x7fe2d404e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.405+0000 7fe2e237e640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d4037c20 0x7fe2d404e4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800c480 tx=0x7fe2c800c940 comp rx=0 tx=0).stop 2024-04-25T09:53:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.405+0000 7fe2e237e640 1 -- 172.21.15.19:0/2654755800 shutdown_connections 2024-04-25T09:53:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.405+0000 7fe2e237e640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2c006b520 0x7fe2c006d9e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.405+0000 7fe2e237e640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2d40f01d0 0x7fe2d40e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.405+0000 7fe2e237e640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2d4037c20 0x7fe2d404e4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.406+0000 7fe2e237e640 1 --2- 172.21.15.19:0/2654755800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2d4033f80 0x7fe2d40db710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.406+0000 7fe2e237e640 1 -- 172.21.15.19:0/2654755800 >> 172.21.15.19:0/2654755800 conn(0x7fe2d4009fe0 msgr2=0x7fe2d4038640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.406+0000 7fe2e237e640 1 -- 172.21.15.19:0/2654755800 shutdown_connections 2024-04-25T09:53:27.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.406+0000 7fe2e237e640 1 -- 172.21.15.19:0/2654755800 wait complete. 2024-04-25T09:53:27.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-04-25T09:53:27.421 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:27.421 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-25T09:53:27.422 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:27.422 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-04-25T09:53:27.422 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:27.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.440+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc120005580 con 0x7fc138009180 2024-04-25T09:53:27.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.441+0000 7fc1277fe640 1 -- 172.21.15.19:0/3588993294 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc1300796b0 con 0x7fc138009180 2024-04-25T09:53:27.443 INFO:teuthology.orchestra.run.smithi019.stdout:55834574858 2024-04-25T09:53:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc12c06b5a0 msgr2=0x7fc12c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc12c06b5a0 0x7fc12c06da60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc134000f60 tx=0x7fc134009040 comp rx=0 tx=0).stop 2024-04-25T09:53:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138009180 msgr2=0x7fc138008430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138009180 0x7fc138008430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc13002a7b0 tx=0x7fc1300046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 shutdown_connections 2024-04-25T09:53:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc12c06b5a0 0x7fc12c06da60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc138049060 0x7fc1380512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc138048c30 0x7fc1380d8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 --2- 172.21.15.19:0/3588993294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc138009180 0x7fc138008430 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 >> 172.21.15.19:0/3588993294 conn(0x7fc13801ef30 msgr2=0x7fc1380e7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 shutdown_connections 2024-04-25T09:53:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.444+0000 7fc1451a5640 1 -- 172.21.15.19:0/3588993294 wait complete. 2024-04-25T09:53:27.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-04-25T09:53:27.457 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.474+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f008c073a00 con 0x7f008c06ff50 2024-04-25T09:53:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.475+0000 7f0088ff9640 1 -- 172.21.15.19:0/2262828692 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0074079e20 con 0x7f008c06ff50 2024-04-25T09:53:27.478 INFO:teuthology.orchestra.run.smithi019.stdout:55834574857 2024-04-25T09:53:27.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006c06b5a0 msgr2=0x7f006c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006c06b5a0 0x7f006c06da60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0084009e20 tx=0x7f0084009470 comp rx=0 tx=0).stop 2024-04-25T09:53:27.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f008c06ff50 msgr2=0x7f008c150cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f008c06ff50 0x7f008c150cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0074031000 tx=0x7f0074004700 comp rx=0 tx=0).stop 2024-04-25T09:53:27.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 shutdown_connections 2024-04-25T09:53:27.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006c06b5a0 0x7f006c06da60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f008c06ff50 0x7f008c150cc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f008c150240 0x7f008c150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 --2- 172.21.15.19:0/2262828692 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f00780052e0 0x7f008c14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 >> 172.21.15.19:0/2262828692 conn(0x7f008c06f500 msgr2=0x7f008c074060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 shutdown_connections 2024-04-25T09:53:27.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.477+0000 7f0092815640 1 -- 172.21.15.19:0/2262828692 wait complete. 2024-04-25T09:53:27.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2024-04-25T09:53:27.490 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.500+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1354005850 con 0x7f136c006190 2024-04-25T09:53:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.500+0000 7f135ffff640 1 -- 172.21.15.19:0/1690452859 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f136c0387d0 con 0x7f136c006190 2024-04-25T09:53:27.503 INFO:teuthology.orchestra.run.smithi019.stdout:55834574858 2024-04-25T09:53:27.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f135806b5a0 msgr2=0x7f135806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f135806b5a0 0x7f135806da60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1360034000 tx=0x7f136002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f136c006190 msgr2=0x7f136c04f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f136c006190 0x7f136c04f370 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f136800c730 tx=0x7f136800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 shutdown_connections 2024-04-25T09:53:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f135806b5a0 0x7f135806da60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f136c0ebb40 0x7f136c051f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f136c0eb050 0x7f136c0519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 --2- 172.21.15.19:0/1690452859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f136c006190 0x7f136c04f370 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 >> 172.21.15.19:0/1690452859 conn(0x7f136c0559d0 msgr2=0x7f136c0dd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 shutdown_connections 2024-04-25T09:53:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.502+0000 7f1378ff1640 1 -- 172.21.15.19:0/1690452859 wait complete. 2024-04-25T09:53:27.515 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-04-25T09:53:27.516 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:27.516 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T09:53:27.516 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:53:27.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.654+0000 7fefa16f5640 1 Processor -- start 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.654+0000 7fefa16f5640 1 -- start start 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.655+0000 7fefa16f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fef9c15bf50 0x7fef9c0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.655+0000 7fefa16f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fef9c15c400 0x7fef9c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.655+0000 7fefa16f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c15c930 con 0x7fef9c1594f0 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.655+0000 7fefa16f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c0d12d0 con 0x7fef9c15c400 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.655+0000 7fefa16f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c1598d0 con 0x7fef9c15bf50 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.655+0000 7fef9b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fef9c15c400 0x7fef9c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.655+0000 7fef9b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fef9c15c400 0x7fef9c0d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35930/0 (socket says 172.21.15.19:35930) 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.655+0000 7fef9b7fe640 1 -- 172.21.15.19:0/500473650 learned_addr learned my addr 172.21.15.19:0/500473650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.656+0000 7fef9a7fc640 1 --2- 172.21.15.19:0/500473650 >> v2:172.21.15.112:3300/0 conn(0x7fef9c15bf50 0x7fef9c0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.656+0000 7fef9b7fe640 1 -- 172.21.15.19:0/500473650 >> v2:172.21.15.112:3300/0 conn(0x7fef9c15bf50 msgr2=0x7fef9c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.656+0000 7fef9b7fe640 1 --2- 172.21.15.19:0/500473650 >> v2:172.21.15.112:3300/0 conn(0x7fef9c15bf50 0x7fef9c0d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.656+0000 7fef9b7fe640 1 -- 172.21.15.19:0/500473650 >> v1:172.21.15.112:6789/0 conn(0x7fef9c1594f0 legacy=0x7fef9c0b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.656+0000 7fef99ffb640 1 -- 172.21.15.19:0/500473650 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919776342 0 0) 0x7fef9c15c930 con 0x7fef9c1594f0 2024-04-25T09:53:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.656+0000 7fef9b7fe640 1 -- 172.21.15.19:0/500473650 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c15cc10 con 0x7fef9c15c400 2024-04-25T09:53:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.656+0000 7fef9b7fe640 1 --2- 172.21.15.19:0/500473650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c15c400 0x7fef9c0d0d90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fef94009e50 tx=0x7fef9402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa5d3d6af5bcf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.657+0000 7fef99ffb640 1 -- 172.21.15.19:0/500473650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94036070 con 0x7fef9c15c400 2024-04-25T09:53:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.657+0000 7fef99ffb640 1 -- 172.21.15.19:0/500473650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef94031600 con 0x7fef9c15c400 2024-04-25T09:53:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.657+0000 7fef99ffb640 1 -- 172.21.15.19:0/500473650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef940319d0 con 0x7fef9c15c400 2024-04-25T09:53:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.657+0000 7fefa16f5640 1 -- 172.21.15.19:0/500473650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c15c400 msgr2=0x7fef9c0d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.657+0000 7fefa16f5640 1 --2- 172.21.15.19:0/500473650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c15c400 0x7fef9c0d0d90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fef94009e50 tx=0x7fef9402a6d0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.657+0000 7fefa16f5640 1 -- 172.21.15.19:0/500473650 shutdown_connections 2024-04-25T09:53:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.657+0000 7fefa16f5640 1 --2- 172.21.15.19:0/500473650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c15c400 0x7fef9c0d0d90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.657+0000 7fefa16f5640 1 --2- 172.21.15.19:0/500473650 >> v2:172.21.15.112:3300/0 conn(0x7fef9c15bf50 0x7fef9c0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.657+0000 7fefa16f5640 1 -- 172.21.15.19:0/500473650 >> 172.21.15.19:0/500473650 conn(0x7fef9c06f520 msgr2=0x7fef9c06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.657+0000 7fefa16f5640 1 -- 172.21.15.19:0/500473650 shutdown_connections 2024-04-25T09:53:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.657+0000 7fefa16f5640 1 -- 172.21.15.19:0/500473650 wait complete. 2024-04-25T09:53:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.659+0000 7fefa16f5640 1 Processor -- start 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.659+0000 7fefa16f5640 1 -- start start 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.659+0000 7fefa16f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef9c1594f0 0x7fef9c150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.659+0000 7fefa16f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c15bf50 0x7fef9c14e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.659+0000 7fefa16f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef9c15c400 0x7fef9c15e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.659+0000 7fefa16f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c070e10 con 0x7fef9c15bf50 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.659+0000 7fefa16f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0b4c60 con 0x7fef9c1594f0 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.659+0000 7fefa16f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c070f90 con 0x7fef9c15c400 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.659+0000 7fef9a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c15bf50 0x7fef9c14e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef9a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c15bf50 0x7fef9c14e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35934/0 (socket says 172.21.15.19:35934) 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef9a7fc640 1 -- 172.21.15.19:0/2796749104 learned_addr learned my addr 172.21.15.19:0/2796749104 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef9b7fe640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef9c15c400 0x7fef9c15e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef9affd640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef9c1594f0 0x7fef9c150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef9a7fc640 1 -- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef9c15c400 msgr2=0x7fef9c15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef9a7fc640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef9c15c400 0x7fef9c15e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef9a7fc640 1 -- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef9c1594f0 msgr2=0x7fef9c150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef9a7fc640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef9c1594f0 0x7fef9c150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef9a7fc640 1 -- 172.21.15.19:0/2796749104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c1532f0 con 0x7fef9c15bf50 2024-04-25T09:53:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef9a7fc640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c15bf50 0x7fef9c14e440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef9000e9b0 tx=0x7fef9000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef837fe640 1 -- 172.21.15.19:0/2796749104 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90017070 con 0x7fef9c15bf50 2024-04-25T09:53:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef837fe640 1 -- 172.21.15.19:0/2796749104 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9000c9a0 con 0x7fef9c15bf50 2024-04-25T09:53:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fefa16f5640 1 -- 172.21.15.19:0/2796749104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c153590 con 0x7fef9c15bf50 2024-04-25T09:53:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fef837fe640 1 -- 172.21.15.19:0/2796749104 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9000cd90 con 0x7fef9c15bf50 2024-04-25T09:53:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.660+0000 7fefa16f5640 1 -- 172.21.15.19:0/2796749104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c153b50 con 0x7fef9c15bf50 2024-04-25T09:53:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.661+0000 7fefa16f5640 1 -- 172.21.15.19:0/2796749104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c073990 con 0x7fef9c15bf50 2024-04-25T09:53:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.662+0000 7fef837fe640 1 -- 172.21.15.19:0/2796749104 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef9001a070 con 0x7fef9c15bf50 2024-04-25T09:53:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.662+0000 7fef837fe640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fef8806b4d0 0x7fef8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.662+0000 7fef837fe640 1 -- 172.21.15.19:0/2796749104 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fef9008a4e0 con 0x7fef9c15bf50 2024-04-25T09:53:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.663+0000 7fef9affd640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fef8806b4d0 0x7fef8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.663+0000 7fef9affd640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fef8806b4d0 0x7fef8806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fef8c000f60 tx=0x7fef8c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.667+0000 7fef837fe640 1 -- 172.21.15.19:0/2796749104 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef900589c0 con 0x7fef9c15bf50 2024-04-25T09:53:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.848+0000 7fefa16f5640 1 -- 172.21.15.19:0/2796749104 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef9c0700e0 con 0x7fef8806b4d0 2024-04-25T09:53:27.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.856+0000 7fef837fe640 1 -- 172.21.15.19:0/2796749104 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7fef9c0700e0 con 0x7fef8806b4d0 2024-04-25T09:53:27.861 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:27.863 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:53:27.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 -- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fef8806b4d0 msgr2=0x7fef8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fef8806b4d0 0x7fef8806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fef8c000f60 tx=0x7fef8c009040 comp rx=0 tx=0).stop 2024-04-25T09:53:27.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 -- 172.21.15.19:0/2796749104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c15bf50 msgr2=0x7fef9c14e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c15bf50 0x7fef9c14e440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef9000e9b0 tx=0x7fef9000ee70 comp rx=0 tx=0).stop 2024-04-25T09:53:27.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 -- 172.21.15.19:0/2796749104 shutdown_connections 2024-04-25T09:53:27.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fef8806b4d0 0x7fef8806d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef9c15c400 0x7fef9c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9c15bf50 0x7fef9c14e440 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 --2- 172.21.15.19:0/2796749104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef9c1594f0 0x7fef9c150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 -- 172.21.15.19:0/2796749104 >> 172.21.15.19:0/2796749104 conn(0x7fef9c06f520 msgr2=0x7fef9c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 -- 172.21.15.19:0/2796749104 shutdown_connections 2024-04-25T09:53:27.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:27.859+0000 7fefa16f5640 1 -- 172.21.15.19:0/2796749104 wait complete. 2024-04-25T09:53:27.872 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-25T09:53:26.915412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219368,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755352,"statfs":{"total":773094113280,"available":772869480448,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003738"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326249+0000","last_change":"2024-04-25T09:53:06.778392+0000","last_active":"2024-04-25T09:53:13.326249+0000","last_peered":"2024-04-25T09:53:13.326249+0000","last_clean":"2024-04-25T09:53:13.326249+0000","last_became_active":"2024-04-25T09:53:06.777894+0000","last_became_peered":"2024-04-25T09:53:06.777894+0000","last_unstale":"2024-04-25T09:53:13.326249+0000","last_undegraded":"2024-04-25T09:53:13.326249+0000","last_fullsized":"2024-04-25T09:53:13.326249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.898659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326801+0000","last_change":"2024-04-25T09:53:07.792733+0000","last_active":"2024-04-25T09:53:13.326801+0000","last_peered":"2024-04-25T09:53:13.326801+0000","last_clean":"2024-04-25T09:53:13.326801+0000","last_became_active":"2024-04-25T09:53:07.792294+0000","last_became_peered":"2024-04-25T09:53:07.792294+0000","last_unstale":"2024-04-25T09:53:13.326801+0000","last_undegraded":"2024-04-25T09:53:13.326801+0000","last_fullsized":"2024-04-25T09:53:13.326801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325683+0000","last_change":"2024-04-25T09:53:11.822234+0000","last_active":"2024-04-25T09:53:13.325683+0000","last_peered":"2024-04-25T09:53:13.325683+0000","last_clean":"2024-04-25T09:53:13.325683+0000","last_became_active":"2024-04-25T09:53:11.821231+0000","last_became_peered":"2024-04-25T09:53:11.821231+0000","last_unstale":"2024-04-25T09:53:13.325683+0000","last_undegraded":"2024-04-25T09:53:13.325683+0000","last_fullsized":"2024-04-25T09:53:13.325683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:19.880548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327066+0000","last_change":"2024-04-25T09:53:06.779890+0000","last_active":"2024-04-25T09:53:13.327066+0000","last_peered":"2024-04-25T09:53:13.327066+0000","last_clean":"2024-04-25T09:53:13.327066+0000","last_became_active":"2024-04-25T09:53:06.779482+0000","last_became_peered":"2024-04-25T09:53:06.779482+0000","last_unstale":"2024-04-25T09:53:13.327066+0000","last_undegraded":"2024-04-25T09:53:13.327066+0000","last_fullsized":"2024-04-25T09:53:13.327066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325889+0000","last_change":"2024-04-25T09:53:07.789471+0000","last_active":"2024-04-25T09:53:13.325889+0000","last_peered":"2024-04-25T09:53:13.325889+0000","last_clean":"2024-04-25T09:53:13.325889+0000","last_became_active":"2024-04-25T09:53:07.789190+0000","last_became_peered":"2024-04-25T09:53:07.789190+0000","last_unstale":"2024-04-25T09:53:13.325889+0000","last_undegraded":"2024-04-25T09:53:13.325889+0000","last_fullsized":"2024-04-25T09:53:13.325889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326006+0000","last_change":"2024-04-25T09:53:11.841059+0000","last_active":"2024-04-25T09:53:13.326006+0000","last_peered":"2024-04-25T09:53:13.326006+0000","last_clean":"2024-04-25T09:53:13.326006+0000","last_became_active":"2024-04-25T09:53:11.840640+0000","last_became_peered":"2024-04-25T09:53:11.840640+0000","last_unstale":"2024-04-25T09:53:13.326006+0000","last_undegraded":"2024-04-25T09:53:13.326006+0000","last_fullsized":"2024-04-25T09:53:13.326006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:54.172755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810382+0000","last_change":"2024-04-25T09:53:06.777792+0000","last_active":"2024-04-25T09:53:12.810382+0000","last_peered":"2024-04-25T09:53:12.810382+0000","last_clean":"2024-04-25T09:53:12.810382+0000","last_became_active":"2024-04-25T09:53:06.777459+0000","last_became_peered":"2024-04-25T09:53:06.777459+0000","last_unstale":"2024-04-25T09:53:12.810382+0000","last_undegraded":"2024-04-25T09:53:12.810382+0000","last_fullsized":"2024-04-25T09:53:12.810382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:44.406768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326553+0000","last_change":"2024-04-25T09:53:07.798127+0000","last_active":"2024-04-25T09:53:13.326553+0000","last_peered":"2024-04-25T09:53:13.326553+0000","last_clean":"2024-04-25T09:53:13.326553+0000","last_became_active":"2024-04-25T09:53:07.797863+0000","last_became_peered":"2024-04-25T09:53:07.797863+0000","last_unstale":"2024-04-25T09:53:13.326553+0000","last_undegraded":"2024-04-25T09:53:13.326553+0000","last_fullsized":"2024-04-25T09:53:13.326553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326109+0000","last_change":"2024-04-25T09:53:11.841255+0000","last_active":"2024-04-25T09:53:13.326109+0000","last_peered":"2024-04-25T09:53:13.326109+0000","last_clean":"2024-04-25T09:53:13.326109+0000","last_became_active":"2024-04-25T09:53:11.840795+0000","last_became_peered":"2024-04-25T09:53:11.840795+0000","last_unstale":"2024-04-25T09:53:13.326109+0000","last_undegraded":"2024-04-25T09:53:13.326109+0000","last_fullsized":"2024-04-25T09:53:13.326109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:28.118453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810018+0000","last_change":"2024-04-25T09:53:06.784013+0000","last_active":"2024-04-25T09:53:12.810018+0000","last_peered":"2024-04-25T09:53:12.810018+0000","last_clean":"2024-04-25T09:53:12.810018+0000","last_became_active":"2024-04-25T09:53:06.783726+0000","last_became_peered":"2024-04-25T09:53:06.783726+0000","last_unstale":"2024-04-25T09:53:12.810018+0000","last_undegraded":"2024-04-25T09:53:12.810018+0000","last_fullsized":"2024-04-25T09:53:12.810018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327442+0000","last_change":"2024-04-25T09:53:07.796042+0000","last_active":"2024-04-25T09:53:13.327442+0000","last_peered":"2024-04-25T09:53:13.327442+0000","last_clean":"2024-04-25T09:53:13.327442+0000","last_became_active":"2024-04-25T09:53:07.795676+0000","last_became_peered":"2024-04-25T09:53:07.795676+0000","last_unstale":"2024-04-25T09:53:13.327442+0000","last_undegraded":"2024-04-25T09:53:13.327442+0000","last_fullsized":"2024-04-25T09:53:13.327442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:12.845651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326221+0000","last_change":"2024-04-25T09:53:11.827699+0000","last_active":"2024-04-25T09:53:13.326221+0000","last_peered":"2024-04-25T09:53:13.326221+0000","last_clean":"2024-04-25T09:53:13.326221+0000","last_became_active":"2024-04-25T09:53:11.827325+0000","last_became_peered":"2024-04-25T09:53:11.827325+0000","last_unstale":"2024-04-25T09:53:13.326221+0000","last_undegraded":"2024-04-25T09:53:13.326221+0000","last_fullsized":"2024-04-25T09:53:13.326221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.837684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810063+0000","last_change":"2024-04-25T09:53:07.800199+0000","last_active":"2024-04-25T09:53:12.810063+0000","last_peered":"2024-04-25T09:53:12.810063+0000","last_clean":"2024-04-25T09:53:12.810063+0000","last_became_active":"2024-04-25T09:53:07.799912+0000","last_became_peered":"2024-04-25T09:53:07.799912+0000","last_unstale":"2024-04-25T09:53:12.810063+0000","last_undegraded":"2024-04-25T09:53:12.810063+0000","last_fullsized":"2024-04-25T09:53:12.810063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810410+0000","last_change":"2024-04-25T09:53:06.783380+0000","last_active":"2024-04-25T09:53:12.810410+0000","last_peered":"2024-04-25T09:53:12.810410+0000","last_clean":"2024-04-25T09:53:12.810410+0000","last_became_active":"2024-04-25T09:53:06.783049+0000","last_became_peered":"2024-04-25T09:53:06.783049+0000","last_unstale":"2024-04-25T09:53:12.810410+0000","last_undegraded":"2024-04-25T09:53:12.810410+0000","last_fullsized":"2024-04-25T09:53:12.810410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327316+0000","last_change":"2024-04-25T09:53:11.839467+0000","last_active":"2024-04-25T09:53:13.327316+0000","last_peered":"2024-04-25T09:53:13.327316+0000","last_clean":"2024-04-25T09:53:13.327316+0000","last_became_active":"2024-04-25T09:53:11.839079+0000","last_became_peered":"2024-04-25T09:53:11.839079+0000","last_unstale":"2024-04-25T09:53:13.327316+0000","last_undegraded":"2024-04-25T09:53:13.327316+0000","last_fullsized":"2024-04-25T09:53:13.327316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325291+0000","last_change":"2024-04-25T09:53:07.791763+0000","last_active":"2024-04-25T09:53:13.325291+0000","last_peered":"2024-04-25T09:53:13.325291+0000","last_clean":"2024-04-25T09:53:13.325291+0000","last_became_active":"2024-04-25T09:53:07.790924+0000","last_became_peered":"2024-04-25T09:53:07.790924+0000","last_unstale":"2024-04-25T09:53:13.325291+0000","last_undegraded":"2024-04-25T09:53:13.325291+0000","last_fullsized":"2024-04-25T09:53:13.325291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326553+0000","last_change":"2024-04-25T09:53:06.779703+0000","last_active":"2024-04-25T09:53:13.326553+0000","last_peered":"2024-04-25T09:53:13.326553+0000","last_clean":"2024-04-25T09:53:13.326553+0000","last_became_active":"2024-04-25T09:53:06.779413+0000","last_became_peered":"2024-04-25T09:53:06.779413+0000","last_unstale":"2024-04-25T09:53:13.326553+0000","last_undegraded":"2024-04-25T09:53:13.326553+0000","last_fullsized":"2024-04-25T09:53:13.326553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326368+0000","last_change":"2024-04-25T09:53:11.840454+0000","last_active":"2024-04-25T09:53:13.326368+0000","last_peered":"2024-04-25T09:53:13.326368+0000","last_clean":"2024-04-25T09:53:13.326368+0000","last_became_active":"2024-04-25T09:53:11.840034+0000","last_became_peered":"2024-04-25T09:53:11.840034+0000","last_unstale":"2024-04-25T09:53:13.326368+0000","last_undegraded":"2024-04-25T09:53:13.326368+0000","last_fullsized":"2024-04-25T09:53:13.326368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.438696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326270+0000","last_change":"2024-04-25T09:53:07.786389+0000","last_active":"2024-04-25T09:53:13.326270+0000","last_peered":"2024-04-25T09:53:13.326270+0000","last_clean":"2024-04-25T09:53:13.326270+0000","last_became_active":"2024-04-25T09:53:07.786116+0000","last_became_peered":"2024-04-25T09:53:07.786116+0000","last_unstale":"2024-04-25T09:53:13.326270+0000","last_undegraded":"2024-04-25T09:53:13.326270+0000","last_fullsized":"2024-04-25T09:53:13.326270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325212+0000","last_change":"2024-04-25T09:53:06.780897+0000","last_active":"2024-04-25T09:53:13.325212+0000","last_peered":"2024-04-25T09:53:13.325212+0000","last_clean":"2024-04-25T09:53:13.325212+0000","last_became_active":"2024-04-25T09:53:06.780616+0000","last_became_peered":"2024-04-25T09:53:06.780616+0000","last_unstale":"2024-04-25T09:53:13.325212+0000","last_undegraded":"2024-04-25T09:53:13.325212+0000","last_fullsized":"2024-04-25T09:53:13.325212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:24.763796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326272+0000","last_change":"2024-04-25T09:53:11.838039+0000","last_active":"2024-04-25T09:53:13.326272+0000","last_peered":"2024-04-25T09:53:13.326272+0000","last_clean":"2024-04-25T09:53:13.326272+0000","last_became_active":"2024-04-25T09:53:11.837388+0000","last_became_peered":"2024-04-25T09:53:11.837388+0000","last_unstale":"2024-04-25T09:53:13.326272+0000","last_undegraded":"2024-04-25T09:53:13.326272+0000","last_fullsized":"2024-04-25T09:53:13.326272+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:39.753991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326308+0000","last_change":"2024-04-25T09:53:07.784394+0000","last_active":"2024-04-25T09:53:13.326308+0000","last_peered":"2024-04-25T09:53:13.326308+0000","last_clean":"2024-04-25T09:53:13.326308+0000","last_became_active":"2024-04-25T09:53:07.784060+0000","last_became_peered":"2024-04-25T09:53:07.784060+0000","last_unstale":"2024-04-25T09:53:13.326308+0000","last_undegraded":"2024-04-25T09:53:13.326308+0000","last_fullsized":"2024-04-25T09:53:13.326308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:36.593849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809684+0000","last_change":"2024-04-25T09:53:06.779579+0000","last_active":"2024-04-25T09:53:12.809684+0000","last_peered":"2024-04-25T09:53:12.809684+0000","last_clean":"2024-04-25T09:53:12.809684+0000","last_became_active":"2024-04-25T09:53:06.779296+0000","last_became_peered":"2024-04-25T09:53:06.779296+0000","last_unstale":"2024-04-25T09:53:12.809684+0000","last_undegraded":"2024-04-25T09:53:12.809684+0000","last_fullsized":"2024-04-25T09:53:12.809684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327226+0000","last_change":"2024-04-25T09:53:11.839381+0000","last_active":"2024-04-25T09:53:13.327226+0000","last_peered":"2024-04-25T09:53:13.327226+0000","last_clean":"2024-04-25T09:53:13.327226+0000","last_became_active":"2024-04-25T09:53:11.838941+0000","last_became_peered":"2024-04-25T09:53:11.838941+0000","last_unstale":"2024-04-25T09:53:13.327226+0000","last_undegraded":"2024-04-25T09:53:13.327226+0000","last_fullsized":"2024-04-25T09:53:13.327226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:59.415156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326930+0000","last_change":"2024-04-25T09:53:07.785627+0000","last_active":"2024-04-25T09:53:13.326930+0000","last_peered":"2024-04-25T09:53:13.326930+0000","last_clean":"2024-04-25T09:53:13.326930+0000","last_became_active":"2024-04-25T09:53:07.782773+0000","last_became_peered":"2024-04-25T09:53:07.782773+0000","last_unstale":"2024-04-25T09:53:13.326930+0000","last_undegraded":"2024-04-25T09:53:13.326930+0000","last_fullsized":"2024-04-25T09:53:13.326930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811412+0000","last_change":"2024-04-25T09:53:06.778161+0000","last_active":"2024-04-25T09:53:12.811412+0000","last_peered":"2024-04-25T09:53:12.811412+0000","last_clean":"2024-04-25T09:53:12.811412+0000","last_became_active":"2024-04-25T09:53:06.777837+0000","last_became_peered":"2024-04-25T09:53:06.777837+0000","last_unstale":"2024-04-25T09:53:12.811412+0000","last_undegraded":"2024-04-25T09:53:12.811412+0000","last_fullsized":"2024-04-25T09:53:12.811412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325940+0000","last_change":"2024-04-25T09:53:11.823677+0000","last_active":"2024-04-25T09:53:13.325940+0000","last_peered":"2024-04-25T09:53:13.325940+0000","last_clean":"2024-04-25T09:53:13.325940+0000","last_became_active":"2024-04-25T09:53:11.823278+0000","last_became_peered":"2024-04-25T09:53:11.823278+0000","last_unstale":"2024-04-25T09:53:13.325940+0000","last_undegraded":"2024-04-25T09:53:13.325940+0000","last_fullsized":"2024-04-25T09:53:13.325940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:45.277051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327064+0000","last_change":"2024-04-25T09:53:07.792276+0000","last_active":"2024-04-25T09:53:13.327064+0000","last_peered":"2024-04-25T09:53:13.327064+0000","last_clean":"2024-04-25T09:53:13.327064+0000","last_became_active":"2024-04-25T09:53:07.790961+0000","last_became_peered":"2024-04-25T09:53:07.790961+0000","last_unstale":"2024-04-25T09:53:13.327064+0000","last_undegraded":"2024-04-25T09:53:13.327064+0000","last_fullsized":"2024-04-25T09:53:13.327064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325875+0000","last_change":"2024-04-25T09:53:06.779465+0000","last_active":"2024-04-25T09:53:13.325875+0000","last_peered":"2024-04-25T09:53:13.325875+0000","last_clean":"2024-04-25T09:53:13.325875+0000","last_became_active":"2024-04-25T09:53:06.779122+0000","last_became_peered":"2024-04-25T09:53:06.779122+0000","last_unstale":"2024-04-25T09:53:13.325875+0000","last_undegraded":"2024-04-25T09:53:13.325875+0000","last_fullsized":"2024-04-25T09:53:13.325875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325863+0000","last_change":"2024-04-25T09:53:11.832958+0000","last_active":"2024-04-25T09:53:13.325863+0000","last_peered":"2024-04-25T09:53:13.325863+0000","last_clean":"2024-04-25T09:53:13.325863+0000","last_became_active":"2024-04-25T09:53:11.832522+0000","last_became_peered":"2024-04-25T09:53:11.832522+0000","last_unstale":"2024-04-25T09:53:13.325863+0000","last_undegraded":"2024-04-25T09:53:13.325863+0000","last_fullsized":"2024-04-25T09:53:13.325863+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:58.718292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325598+0000","last_change":"2024-04-25T09:53:07.779236+0000","last_active":"2024-04-25T09:53:13.325598+0000","last_peered":"2024-04-25T09:53:13.325598+0000","last_clean":"2024-04-25T09:53:13.325598+0000","last_became_active":"2024-04-25T09:53:07.778938+0000","last_became_peered":"2024-04-25T09:53:07.778938+0000","last_unstale":"2024-04-25T09:53:13.325598+0000","last_undegraded":"2024-04-25T09:53:13.325598+0000","last_fullsized":"2024-04-25T09:53:13.325598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810604+0000","last_change":"2024-04-25T09:53:06.780019+0000","last_active":"2024-04-25T09:53:12.810604+0000","last_peered":"2024-04-25T09:53:12.810604+0000","last_clean":"2024-04-25T09:53:12.810604+0000","last_became_active":"2024-04-25T09:53:06.779429+0000","last_became_peered":"2024-04-25T09:53:06.779429+0000","last_unstale":"2024-04-25T09:53:12.810604+0000","last_undegraded":"2024-04-25T09:53:12.810604+0000","last_fullsized":"2024-04-25T09:53:12.810604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:24.763796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325466+0000","last_change":"2024-04-25T09:53:11.835047+0000","last_active":"2024-04-25T09:53:13.325466+0000","last_peered":"2024-04-25T09:53:13.325466+0000","last_clean":"2024-04-25T09:53:13.325466+0000","last_became_active":"2024-04-25T09:53:11.833932+0000","last_became_peered":"2024-04-25T09:53:11.833932+0000","last_unstale":"2024-04-25T09:53:13.325466+0000","last_undegraded":"2024-04-25T09:53:13.325466+0000","last_fullsized":"2024-04-25T09:53:13.325466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:39.753991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327090+0000","last_change":"2024-04-25T09:53:07.786462+0000","last_active":"2024-04-25T09:53:13.327090+0000","last_peered":"2024-04-25T09:53:13.327090+0000","last_clean":"2024-04-25T09:53:13.327090+0000","last_became_active":"2024-04-25T09:53:07.785793+0000","last_became_peered":"2024-04-25T09:53:07.785793+0000","last_unstale":"2024-04-25T09:53:13.327090+0000","last_undegraded":"2024-04-25T09:53:13.327090+0000","last_fullsized":"2024-04-25T09:53:13.327090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.221246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325690+0000","last_change":"2024-04-25T09:53:06.780745+0000","last_active":"2024-04-25T09:53:13.325690+0000","last_peered":"2024-04-25T09:53:13.325690+0000","last_clean":"2024-04-25T09:53:13.325690+0000","last_became_active":"2024-04-25T09:53:06.780468+0000","last_became_peered":"2024-04-25T09:53:06.780468+0000","last_unstale":"2024-04-25T09:53:13.325690+0000","last_undegraded":"2024-04-25T09:53:13.325690+0000","last_fullsized":"2024-04-25T09:53:13.325690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326300+0000","last_change":"2024-04-25T09:53:11.832263+0000","last_active":"2024-04-25T09:53:13.326300+0000","last_peered":"2024-04-25T09:53:13.326300+0000","last_clean":"2024-04-25T09:53:13.326300+0000","last_became_active":"2024-04-25T09:53:11.831846+0000","last_became_peered":"2024-04-25T09:53:11.831846+0000","last_unstale":"2024-04-25T09:53:13.326300+0000","last_undegraded":"2024-04-25T09:53:13.326300+0000","last_fullsized":"2024-04-25T09:53:13.326300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:16.884859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326368+0000","last_change":"2024-04-25T09:53:07.797016+0000","last_active":"2024-04-25T09:53:13.326368+0000","last_peered":"2024-04-25T09:53:13.326368+0000","last_clean":"2024-04-25T09:53:13.326368+0000","last_became_active":"2024-04-25T09:53:07.796749+0000","last_became_peered":"2024-04-25T09:53:07.796749+0000","last_unstale":"2024-04-25T09:53:13.326368+0000","last_undegraded":"2024-04-25T09:53:13.326368+0000","last_fullsized":"2024-04-25T09:53:13.326368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:21.755503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326938+0000","last_change":"2024-04-25T09:53:06.782783+0000","last_active":"2024-04-25T09:53:13.326938+0000","last_peered":"2024-04-25T09:53:13.326938+0000","last_clean":"2024-04-25T09:53:13.326938+0000","last_became_active":"2024-04-25T09:53:06.782388+0000","last_became_peered":"2024-04-25T09:53:06.782388+0000","last_unstale":"2024-04-25T09:53:13.326938+0000","last_undegraded":"2024-04-25T09:53:13.326938+0000","last_fullsized":"2024-04-25T09:53:13.326938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325544+0000","last_change":"2024-04-25T09:53:11.832074+0000","last_active":"2024-04-25T09:53:13.325544+0000","last_peered":"2024-04-25T09:53:13.325544+0000","last_clean":"2024-04-25T09:53:13.325544+0000","last_became_active":"2024-04-25T09:53:11.831621+0000","last_became_peered":"2024-04-25T09:53:11.831621+0000","last_unstale":"2024-04-25T09:53:13.325544+0000","last_undegraded":"2024-04-25T09:53:13.325544+0000","last_fullsized":"2024-04-25T09:53:13.325544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.324944+0000","last_change":"2024-04-25T09:53:07.783910+0000","last_active":"2024-04-25T09:53:13.324944+0000","last_peered":"2024-04-25T09:53:13.324944+0000","last_clean":"2024-04-25T09:53:13.324944+0000","last_became_active":"2024-04-25T09:53:07.783625+0000","last_became_peered":"2024-04-25T09:53:07.783625+0000","last_unstale":"2024-04-25T09:53:13.324944+0000","last_undegraded":"2024-04-25T09:53:13.324944+0000","last_fullsized":"2024-04-25T09:53:13.324944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809538+0000","last_change":"2024-04-25T09:53:06.776134+0000","last_active":"2024-04-25T09:53:12.809538+0000","last_peered":"2024-04-25T09:53:12.809538+0000","last_clean":"2024-04-25T09:53:12.809538+0000","last_became_active":"2024-04-25T09:53:06.775787+0000","last_became_peered":"2024-04-25T09:53:06.775787+0000","last_unstale":"2024-04-25T09:53:12.809538+0000","last_undegraded":"2024-04-25T09:53:12.809538+0000","last_fullsized":"2024-04-25T09:53:12.809538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326396+0000","last_change":"2024-04-25T09:53:11.837526+0000","last_active":"2024-04-25T09:53:13.326396+0000","last_peered":"2024-04-25T09:53:13.326396+0000","last_clean":"2024-04-25T09:53:13.326396+0000","last_became_active":"2024-04-25T09:53:11.832037+0000","last_became_peered":"2024-04-25T09:53:11.832037+0000","last_unstale":"2024-04-25T09:53:13.326396+0000","last_undegraded":"2024-04-25T09:53:13.326396+0000","last_fullsized":"2024-04-25T09:53:13.326396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.727046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325424+0000","last_change":"2024-04-25T09:53:07.787155+0000","last_active":"2024-04-25T09:53:13.325424+0000","last_peered":"2024-04-25T09:53:13.325424+0000","last_clean":"2024-04-25T09:53:13.325424+0000","last_became_active":"2024-04-25T09:53:07.783553+0000","last_became_peered":"2024-04-25T09:53:07.783553+0000","last_unstale":"2024-04-25T09:53:13.325424+0000","last_undegraded":"2024-04-25T09:53:13.325424+0000","last_fullsized":"2024-04-25T09:53:13.325424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:12.089487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810045+0000","last_change":"2024-04-25T09:53:06.780626+0000","last_active":"2024-04-25T09:53:12.810045+0000","last_peered":"2024-04-25T09:53:12.810045+0000","last_clean":"2024-04-25T09:53:12.810045+0000","last_became_active":"2024-04-25T09:53:06.780364+0000","last_became_peered":"2024-04-25T09:53:06.780364+0000","last_unstale":"2024-04-25T09:53:12.810045+0000","last_undegraded":"2024-04-25T09:53:12.810045+0000","last_fullsized":"2024-04-25T09:53:12.810045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326877+0000","last_change":"2024-04-25T09:53:11.832887+0000","last_active":"2024-04-25T09:53:13.326877+0000","last_peered":"2024-04-25T09:53:13.326877+0000","last_clean":"2024-04-25T09:53:13.326877+0000","last_became_active":"2024-04-25T09:53:11.832471+0000","last_became_peered":"2024-04-25T09:53:11.832471+0000","last_unstale":"2024-04-25T09:53:13.326877+0000","last_undegraded":"2024-04-25T09:53:13.326877+0000","last_fullsized":"2024-04-25T09:53:13.326877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:03.438164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.324983+0000","last_change":"2024-04-25T09:53:07.793466+0000","last_active":"2024-04-25T09:53:13.324983+0000","last_peered":"2024-04-25T09:53:13.324983+0000","last_clean":"2024-04-25T09:53:13.324983+0000","last_became_active":"2024-04-25T09:53:07.793130+0000","last_became_peered":"2024-04-25T09:53:07.793130+0000","last_unstale":"2024-04-25T09:53:13.324983+0000","last_undegraded":"2024-04-25T09:53:13.324983+0000","last_fullsized":"2024-04-25T09:53:13.324983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326577+0000","last_change":"2024-04-25T09:53:06.780762+0000","last_active":"2024-04-25T09:53:13.326577+0000","last_peered":"2024-04-25T09:53:13.326577+0000","last_clean":"2024-04-25T09:53:13.326577+0000","last_became_active":"2024-04-25T09:53:06.780498+0000","last_became_peered":"2024-04-25T09:53:06.780498+0000","last_unstale":"2024-04-25T09:53:13.326577+0000","last_undegraded":"2024-04-25T09:53:13.326577+0000","last_fullsized":"2024-04-25T09:53:13.326577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810151+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:53:12.810151+0000","last_peered":"2024-04-25T09:53:12.810151+0000","last_clean":"2024-04-25T09:53:12.810151+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:53:12.810151+0000","last_undegraded":"2024-04-25T09:53:12.810151+0000","last_fullsized":"2024-04-25T09:53:12.810151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810776+0000","last_change":"2024-04-25T09:53:11.825407+0000","last_active":"2024-04-25T09:53:12.810776+0000","last_peered":"2024-04-25T09:53:12.810776+0000","last_clean":"2024-04-25T09:53:12.810776+0000","last_became_active":"2024-04-25T09:53:11.824448+0000","last_became_peered":"2024-04-25T09:53:11.824448+0000","last_unstale":"2024-04-25T09:53:12.810776+0000","last_undegraded":"2024-04-25T09:53:12.810776+0000","last_fullsized":"2024-04-25T09:53:12.810776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:59.415156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810231+0000","last_change":"2024-04-25T09:53:07.803634+0000","last_active":"2024-04-25T09:53:12.810231+0000","last_peered":"2024-04-25T09:53:12.810231+0000","last_clean":"2024-04-25T09:53:12.810231+0000","last_became_active":"2024-04-25T09:53:07.803367+0000","last_became_peered":"2024-04-25T09:53:07.803367+0000","last_unstale":"2024-04-25T09:53:12.810231+0000","last_undegraded":"2024-04-25T09:53:12.810231+0000","last_fullsized":"2024-04-25T09:53:12.810231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:55.375947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809562+0000","last_change":"2024-04-25T09:53:06.783978+0000","last_active":"2024-04-25T09:53:12.809562+0000","last_peered":"2024-04-25T09:53:12.809562+0000","last_clean":"2024-04-25T09:53:12.809562+0000","last_became_active":"2024-04-25T09:53:06.783614+0000","last_became_peered":"2024-04-25T09:53:06.783614+0000","last_unstale":"2024-04-25T09:53:12.809562+0000","last_undegraded":"2024-04-25T09:53:12.809562+0000","last_fullsized":"2024-04-25T09:53:12.809562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:52.306054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810011+0000","last_change":"2024-04-25T09:53:11.835086+0000","last_active":"2024-04-25T09:53:12.810011+0000","last_peered":"2024-04-25T09:53:12.810011+0000","last_clean":"2024-04-25T09:53:12.810011+0000","last_became_active":"2024-04-25T09:53:11.834674+0000","last_became_peered":"2024-04-25T09:53:11.834674+0000","last_unstale":"2024-04-25T09:53:12.810011+0000","last_undegraded":"2024-04-25T09:53:12.810011+0000","last_fullsized":"2024-04-25T09:53:12.810011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:01.272017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326681+0000","last_change":"2024-04-25T09:53:07.786966+0000","last_active":"2024-04-25T09:53:13.326681+0000","last_peered":"2024-04-25T09:53:13.326681+0000","last_clean":"2024-04-25T09:53:13.326681+0000","last_became_active":"2024-04-25T09:53:07.786097+0000","last_became_peered":"2024-04-25T09:53:07.786097+0000","last_unstale":"2024-04-25T09:53:13.326681+0000","last_undegraded":"2024-04-25T09:53:13.326681+0000","last_fullsized":"2024-04-25T09:53:13.326681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326102+0000","last_change":"2024-04-25T09:53:06.781647+0000","last_active":"2024-04-25T09:53:13.326102+0000","last_peered":"2024-04-25T09:53:13.326102+0000","last_clean":"2024-04-25T09:53:13.326102+0000","last_became_active":"2024-04-25T09:53:06.781360+0000","last_became_peered":"2024-04-25T09:53:06.781360+0000","last_unstale":"2024-04-25T09:53:13.326102+0000","last_undegraded":"2024-04-25T09:53:13.326102+0000","last_fullsized":"2024-04-25T09:53:13.326102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:44.406768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326043+0000","last_change":"2024-04-25T09:53:11.835499+0000","last_active":"2024-04-25T09:53:13.326043+0000","last_peered":"2024-04-25T09:53:13.326043+0000","last_clean":"2024-04-25T09:53:13.326043+0000","last_became_active":"2024-04-25T09:53:11.834613+0000","last_became_peered":"2024-04-25T09:53:11.834613+0000","last_unstale":"2024-04-25T09:53:13.326043+0000","last_undegraded":"2024-04-25T09:53:13.326043+0000","last_fullsized":"2024-04-25T09:53:13.326043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:16.884859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811569+0000","last_change":"2024-04-25T09:53:07.799270+0000","last_active":"2024-04-25T09:53:12.811569+0000","last_peered":"2024-04-25T09:53:12.811569+0000","last_clean":"2024-04-25T09:53:12.811569+0000","last_became_active":"2024-04-25T09:53:07.799005+0000","last_became_peered":"2024-04-25T09:53:07.799005+0000","last_unstale":"2024-04-25T09:53:12.811569+0000","last_undegraded":"2024-04-25T09:53:12.811569+0000","last_fullsized":"2024-04-25T09:53:12.811569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:47.687838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809643+0000","last_change":"2024-04-25T09:53:06.776508+0000","last_active":"2024-04-25T09:53:12.809643+0000","last_peered":"2024-04-25T09:53:12.809643+0000","last_clean":"2024-04-25T09:53:12.809643+0000","last_became_active":"2024-04-25T09:53:06.775938+0000","last_became_peered":"2024-04-25T09:53:06.775938+0000","last_unstale":"2024-04-25T09:53:12.809643+0000","last_undegraded":"2024-04-25T09:53:12.809643+0000","last_fullsized":"2024-04-25T09:53:12.809643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325709+0000","last_change":"2024-04-25T09:53:11.823495+0000","last_active":"2024-04-25T09:53:13.325709+0000","last_peered":"2024-04-25T09:53:13.325709+0000","last_clean":"2024-04-25T09:53:13.325709+0000","last_became_active":"2024-04-25T09:53:11.823070+0000","last_became_peered":"2024-04-25T09:53:11.823070+0000","last_unstale":"2024-04-25T09:53:13.325709+0000","last_undegraded":"2024-04-25T09:53:13.325709+0000","last_fullsized":"2024-04-25T09:53:13.325709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:58.601489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810162+0000","last_change":"2024-04-25T09:53:07.796560+0000","last_active":"2024-04-25T09:53:12.810162+0000","last_peered":"2024-04-25T09:53:12.810162+0000","last_clean":"2024-04-25T09:53:12.810162+0000","last_became_active":"2024-04-25T09:53:07.796288+0000","last_became_peered":"2024-04-25T09:53:07.796288+0000","last_unstale":"2024-04-25T09:53:12.810162+0000","last_undegraded":"2024-04-25T09:53:12.810162+0000","last_fullsized":"2024-04-25T09:53:12.810162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327440+0000","last_change":"2024-04-25T09:53:06.776524+0000","last_active":"2024-04-25T09:53:13.327440+0000","last_peered":"2024-04-25T09:53:13.327440+0000","last_clean":"2024-04-25T09:53:13.327440+0000","last_became_active":"2024-04-25T09:53:06.776156+0000","last_became_peered":"2024-04-25T09:53:06.776156+0000","last_unstale":"2024-04-25T09:53:13.327440+0000","last_undegraded":"2024-04-25T09:53:13.327440+0000","last_fullsized":"2024-04-25T09:53:13.327440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:23.222453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325855+0000","last_change":"2024-04-25T09:53:11.840103+0000","last_active":"2024-04-25T09:53:13.325855+0000","last_peered":"2024-04-25T09:53:13.325855+0000","last_clean":"2024-04-25T09:53:13.325855+0000","last_became_active":"2024-04-25T09:53:11.839706+0000","last_became_peered":"2024-04-25T09:53:11.839706+0000","last_unstale":"2024-04-25T09:53:13.325855+0000","last_undegraded":"2024-04-25T09:53:13.325855+0000","last_fullsized":"2024-04-25T09:53:13.325855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:35.127642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809957+0000","last_change":"2024-04-25T09:53:07.796678+0000","last_active":"2024-04-25T09:53:12.809957+0000","last_peered":"2024-04-25T09:53:12.809957+0000","last_clean":"2024-04-25T09:53:12.809957+0000","last_became_active":"2024-04-25T09:53:07.796401+0000","last_became_peered":"2024-04-25T09:53:07.796401+0000","last_unstale":"2024-04-25T09:53:12.809957+0000","last_undegraded":"2024-04-25T09:53:12.809957+0000","last_fullsized":"2024-04-25T09:53:12.809957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809151+0000","last_change":"2024-04-25T09:53:06.780411+0000","last_active":"2024-04-25T09:53:12.809151+0000","last_peered":"2024-04-25T09:53:12.809151+0000","last_clean":"2024-04-25T09:53:12.809151+0000","last_became_active":"2024-04-25T09:53:06.780094+0000","last_became_peered":"2024-04-25T09:53:06.780094+0000","last_unstale":"2024-04-25T09:53:12.809151+0000","last_undegraded":"2024-04-25T09:53:12.809151+0000","last_fullsized":"2024-04-25T09:53:12.809151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326778+0000","last_change":"2024-04-25T09:53:11.823203+0000","last_active":"2024-04-25T09:53:13.326778+0000","last_peered":"2024-04-25T09:53:13.326778+0000","last_clean":"2024-04-25T09:53:13.326778+0000","last_became_active":"2024-04-25T09:53:11.822736+0000","last_became_peered":"2024-04-25T09:53:11.822736+0000","last_unstale":"2024-04-25T09:53:13.326778+0000","last_undegraded":"2024-04-25T09:53:13.326778+0000","last_fullsized":"2024-04-25T09:53:13.326778+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:05.899921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809909+0000","last_change":"2024-04-25T09:53:07.797880+0000","last_active":"2024-04-25T09:53:12.809909+0000","last_peered":"2024-04-25T09:53:12.809909+0000","last_clean":"2024-04-25T09:53:12.809909+0000","last_became_active":"2024-04-25T09:53:07.797626+0000","last_became_peered":"2024-04-25T09:53:07.797626+0000","last_unstale":"2024-04-25T09:53:12.809909+0000","last_undegraded":"2024-04-25T09:53:12.809909+0000","last_fullsized":"2024-04-25T09:53:12.809909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810332+0000","last_change":"2024-04-25T09:53:06.781752+0000","last_active":"2024-04-25T09:53:12.810332+0000","last_peered":"2024-04-25T09:53:12.810332+0000","last_clean":"2024-04-25T09:53:12.810332+0000","last_became_active":"2024-04-25T09:53:06.781191+0000","last_became_peered":"2024-04-25T09:53:06.781191+0000","last_unstale":"2024-04-25T09:53:12.810332+0000","last_undegraded":"2024-04-25T09:53:12.810332+0000","last_fullsized":"2024-04-25T09:53:12.810332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326993+0000","last_change":"2024-04-25T09:53:11.839382+0000","last_active":"2024-04-25T09:53:13.326993+0000","last_peered":"2024-04-25T09:53:13.326993+0000","last_clean":"2024-04-25T09:53:13.326993+0000","last_became_active":"2024-04-25T09:53:11.839017+0000","last_became_peered":"2024-04-25T09:53:11.839017+0000","last_unstale":"2024-04-25T09:53:13.326993+0000","last_undegraded":"2024-04-25T09:53:13.326993+0000","last_fullsized":"2024-04-25T09:53:13.326993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:08.728070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810403+0000","last_change":"2024-04-25T09:53:07.797176+0000","last_active":"2024-04-25T09:53:12.810403+0000","last_peered":"2024-04-25T09:53:12.810403+0000","last_clean":"2024-04-25T09:53:12.810403+0000","last_became_active":"2024-04-25T09:53:07.796900+0000","last_became_peered":"2024-04-25T09:53:07.796900+0000","last_unstale":"2024-04-25T09:53:12.810403+0000","last_undegraded":"2024-04-25T09:53:12.810403+0000","last_fullsized":"2024-04-25T09:53:12.810403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326836+0000","last_change":"2024-04-25T09:53:06.783568+0000","last_active":"2024-04-25T09:53:13.326836+0000","last_peered":"2024-04-25T09:53:13.326836+0000","last_clean":"2024-04-25T09:53:13.326836+0000","last_became_active":"2024-04-25T09:53:06.783237+0000","last_became_peered":"2024-04-25T09:53:06.783237+0000","last_unstale":"2024-04-25T09:53:13.326836+0000","last_undegraded":"2024-04-25T09:53:13.326836+0000","last_fullsized":"2024-04-25T09:53:13.326836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326468+0000","last_change":"2024-04-25T09:53:11.835512+0000","last_active":"2024-04-25T09:53:13.326468+0000","last_peered":"2024-04-25T09:53:13.326468+0000","last_clean":"2024-04-25T09:53:13.326468+0000","last_became_active":"2024-04-25T09:53:11.835133+0000","last_became_peered":"2024-04-25T09:53:11.835133+0000","last_unstale":"2024-04-25T09:53:13.326468+0000","last_undegraded":"2024-04-25T09:53:13.326468+0000","last_fullsized":"2024-04-25T09:53:13.326468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:13.599619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326511+0000","last_change":"2024-04-25T09:53:07.788286+0000","last_active":"2024-04-25T09:53:13.326511+0000","last_peered":"2024-04-25T09:53:13.326511+0000","last_clean":"2024-04-25T09:53:13.326511+0000","last_became_active":"2024-04-25T09:53:07.787920+0000","last_became_peered":"2024-04-25T09:53:07.787920+0000","last_unstale":"2024-04-25T09:53:13.326511+0000","last_undegraded":"2024-04-25T09:53:13.326511+0000","last_fullsized":"2024-04-25T09:53:13.326511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325923+0000","last_change":"2024-04-25T09:53:06.772006+0000","last_active":"2024-04-25T09:53:13.325923+0000","last_peered":"2024-04-25T09:53:13.325923+0000","last_clean":"2024-04-25T09:53:13.325923+0000","last_became_active":"2024-04-25T09:53:06.771702+0000","last_became_peered":"2024-04-25T09:53:06.771702+0000","last_unstale":"2024-04-25T09:53:13.325923+0000","last_undegraded":"2024-04-25T09:53:13.325923+0000","last_fullsized":"2024-04-25T09:53:13.325923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:59.219341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327029+0000","last_change":"2024-04-25T09:53:11.821559+0000","last_active":"2024-04-25T09:53:13.327029+0000","last_peered":"2024-04-25T09:53:13.327029+0000","last_clean":"2024-04-25T09:53:13.327029+0000","last_became_active":"2024-04-25T09:53:11.821133+0000","last_became_peered":"2024-04-25T09:53:11.821133+0000","last_unstale":"2024-04-25T09:53:13.327029+0000","last_undegraded":"2024-04-25T09:53:13.327029+0000","last_fullsized":"2024-04-25T09:53:13.327029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:58.718292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810108+0000","last_change":"2024-04-25T09:53:07.797403+0000","last_active":"2024-04-25T09:53:12.810108+0000","last_peered":"2024-04-25T09:53:12.810108+0000","last_clean":"2024-04-25T09:53:12.810108+0000","last_became_active":"2024-04-25T09:53:07.796257+0000","last_became_peered":"2024-04-25T09:53:07.796257+0000","last_unstale":"2024-04-25T09:53:12.810108+0000","last_undegraded":"2024-04-25T09:53:12.810108+0000","last_fullsized":"2024-04-25T09:53:12.810108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:17.046098+0000","last_change":"2024-04-25T09:53:06.773036+0000","last_active":"2024-04-25T09:53:17.046098+0000","last_peered":"2024-04-25T09:53:17.046098+0000","last_clean":"2024-04-25T09:53:17.046098+0000","last_became_active":"2024-04-25T09:53:06.772729+0000","last_became_peered":"2024-04-25T09:53:06.772729+0000","last_unstale":"2024-04-25T09:53:17.046098+0000","last_undegraded":"2024-04-25T09:53:17.046098+0000","last_fullsized":"2024-04-25T09:53:17.046098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326453+0000","last_change":"2024-04-25T09:53:11.833689+0000","last_active":"2024-04-25T09:53:13.326453+0000","last_peered":"2024-04-25T09:53:13.326453+0000","last_clean":"2024-04-25T09:53:13.326453+0000","last_became_active":"2024-04-25T09:53:11.832588+0000","last_became_peered":"2024-04-25T09:53:11.832588+0000","last_unstale":"2024-04-25T09:53:13.326453+0000","last_undegraded":"2024-04-25T09:53:13.326453+0000","last_fullsized":"2024-04-25T09:53:13.326453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:05.899921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327010+0000","last_change":"2024-04-25T09:53:07.791857+0000","last_active":"2024-04-25T09:53:13.327010+0000","last_peered":"2024-04-25T09:53:13.327010+0000","last_clean":"2024-04-25T09:53:13.327010+0000","last_became_active":"2024-04-25T09:53:07.791606+0000","last_became_peered":"2024-04-25T09:53:07.791606+0000","last_unstale":"2024-04-25T09:53:13.327010+0000","last_undegraded":"2024-04-25T09:53:13.327010+0000","last_fullsized":"2024-04-25T09:53:13.327010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.768421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325995+0000","last_change":"2024-04-25T09:53:06.783616+0000","last_active":"2024-04-25T09:53:13.325995+0000","last_peered":"2024-04-25T09:53:13.325995+0000","last_clean":"2024-04-25T09:53:13.325995+0000","last_became_active":"2024-04-25T09:53:06.783302+0000","last_became_peered":"2024-04-25T09:53:06.783302+0000","last_unstale":"2024-04-25T09:53:13.325995+0000","last_undegraded":"2024-04-25T09:53:13.325995+0000","last_fullsized":"2024-04-25T09:53:13.325995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326861+0000","last_change":"2024-04-25T09:53:11.840956+0000","last_active":"2024-04-25T09:53:13.326861+0000","last_peered":"2024-04-25T09:53:13.326861+0000","last_clean":"2024-04-25T09:53:13.326861+0000","last_became_active":"2024-04-25T09:53:11.840316+0000","last_became_peered":"2024-04-25T09:53:11.840316+0000","last_unstale":"2024-04-25T09:53:13.326861+0000","last_undegraded":"2024-04-25T09:53:13.326861+0000","last_fullsized":"2024-04-25T09:53:13.326861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:01.272017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326067+0000","last_change":"2024-04-25T09:53:07.787945+0000","last_active":"2024-04-25T09:53:13.326067+0000","last_peered":"2024-04-25T09:53:13.326067+0000","last_clean":"2024-04-25T09:53:13.326067+0000","last_became_active":"2024-04-25T09:53:07.786113+0000","last_became_peered":"2024-04-25T09:53:07.786113+0000","last_unstale":"2024-04-25T09:53:13.326067+0000","last_undegraded":"2024-04-25T09:53:13.326067+0000","last_fullsized":"2024-04-25T09:53:13.326067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327211+0000","last_change":"2024-04-25T09:53:06.783604+0000","last_active":"2024-04-25T09:53:13.327211+0000","last_peered":"2024-04-25T09:53:13.327211+0000","last_clean":"2024-04-25T09:53:13.327211+0000","last_became_active":"2024-04-25T09:53:06.783341+0000","last_became_peered":"2024-04-25T09:53:06.783341+0000","last_unstale":"2024-04-25T09:53:13.327211+0000","last_undegraded":"2024-04-25T09:53:13.327211+0000","last_fullsized":"2024-04-25T09:53:13.327211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326141+0000","last_change":"2024-04-25T09:53:11.836053+0000","last_active":"2024-04-25T09:53:13.326141+0000","last_peered":"2024-04-25T09:53:13.326141+0000","last_clean":"2024-04-25T09:53:13.326141+0000","last_became_active":"2024-04-25T09:53:11.835318+0000","last_became_peered":"2024-04-25T09:53:11.835318+0000","last_unstale":"2024-04-25T09:53:13.326141+0000","last_undegraded":"2024-04-25T09:53:13.326141+0000","last_fullsized":"2024-04-25T09:53:13.326141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:49.717043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327141+0000","last_change":"2024-04-25T09:53:07.793149+0000","last_active":"2024-04-25T09:53:13.327141+0000","last_peered":"2024-04-25T09:53:13.327141+0000","last_clean":"2024-04-25T09:53:13.327141+0000","last_became_active":"2024-04-25T09:53:07.792896+0000","last_became_peered":"2024-04-25T09:53:07.792896+0000","last_unstale":"2024-04-25T09:53:13.327141+0000","last_undegraded":"2024-04-25T09:53:13.327141+0000","last_fullsized":"2024-04-25T09:53:13.327141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:55.375947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811223+0000","last_change":"2024-04-25T09:53:06.780916+0000","last_active":"2024-04-25T09:53:12.811223+0000","last_peered":"2024-04-25T09:53:12.811223+0000","last_clean":"2024-04-25T09:53:12.811223+0000","last_became_active":"2024-04-25T09:53:06.780631+0000","last_became_peered":"2024-04-25T09:53:06.780631+0000","last_unstale":"2024-04-25T09:53:12.811223+0000","last_undegraded":"2024-04-25T09:53:12.811223+0000","last_fullsized":"2024-04-25T09:53:12.811223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325834+0000","last_change":"2024-04-25T09:53:11.821088+0000","last_active":"2024-04-25T09:53:13.325834+0000","last_peered":"2024-04-25T09:53:13.325834+0000","last_clean":"2024-04-25T09:53:13.325834+0000","last_became_active":"2024-04-25T09:53:11.820662+0000","last_became_peered":"2024-04-25T09:53:11.820662+0000","last_unstale":"2024-04-25T09:53:13.325834+0000","last_undegraded":"2024-04-25T09:53:13.325834+0000","last_fullsized":"2024-04-25T09:53:13.325834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:58.601489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326720+0000","last_change":"2024-04-25T09:53:11.838966+0000","last_active":"2024-04-25T09:53:13.326720+0000","last_peered":"2024-04-25T09:53:13.326720+0000","last_clean":"2024-04-25T09:53:13.326720+0000","last_became_active":"2024-04-25T09:53:11.838542+0000","last_became_peered":"2024-04-25T09:53:11.838542+0000","last_unstale":"2024-04-25T09:53:13.326720+0000","last_undegraded":"2024-04-25T09:53:13.326720+0000","last_fullsized":"2024-04-25T09:53:13.326720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.727046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325110+0000","last_change":"2024-04-25T09:53:07.784870+0000","last_active":"2024-04-25T09:53:13.325110+0000","last_peered":"2024-04-25T09:53:13.325110+0000","last_clean":"2024-04-25T09:53:13.325110+0000","last_became_active":"2024-04-25T09:53:07.784587+0000","last_became_peered":"2024-04-25T09:53:07.784587+0000","last_unstale":"2024-04-25T09:53:13.325110+0000","last_undegraded":"2024-04-25T09:53:13.325110+0000","last_fullsized":"2024-04-25T09:53:13.325110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325791+0000","last_change":"2024-04-25T09:53:06.782572+0000","last_active":"2024-04-25T09:53:13.325791+0000","last_peered":"2024-04-25T09:53:13.325791+0000","last_clean":"2024-04-25T09:53:13.325791+0000","last_became_active":"2024-04-25T09:53:06.781792+0000","last_became_peered":"2024-04-25T09:53:06.781792+0000","last_unstale":"2024-04-25T09:53:13.325791+0000","last_undegraded":"2024-04-25T09:53:13.325791+0000","last_fullsized":"2024-04-25T09:53:13.325791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811107+0000","last_change":"2024-04-25T09:53:11.838770+0000","last_active":"2024-04-25T09:53:12.811107+0000","last_peered":"2024-04-25T09:53:12.811107+0000","last_clean":"2024-04-25T09:53:12.811107+0000","last_became_active":"2024-04-25T09:53:11.838328+0000","last_became_peered":"2024-04-25T09:53:11.838328+0000","last_unstale":"2024-04-25T09:53:12.811107+0000","last_undegraded":"2024-04-25T09:53:12.811107+0000","last_fullsized":"2024-04-25T09:53:12.811107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326187+0000","last_change":"2024-04-25T09:53:07.791725+0000","last_active":"2024-04-25T09:53:13.326187+0000","last_peered":"2024-04-25T09:53:13.326187+0000","last_clean":"2024-04-25T09:53:13.326187+0000","last_became_active":"2024-04-25T09:53:07.791386+0000","last_became_peered":"2024-04-25T09:53:07.791386+0000","last_unstale":"2024-04-25T09:53:13.326187+0000","last_undegraded":"2024-04-25T09:53:13.326187+0000","last_fullsized":"2024-04-25T09:53:13.326187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326069+0000","last_change":"2024-04-25T09:53:06.775205+0000","last_active":"2024-04-25T09:53:13.326069+0000","last_peered":"2024-04-25T09:53:13.326069+0000","last_clean":"2024-04-25T09:53:13.326069+0000","last_became_active":"2024-04-25T09:53:06.774911+0000","last_became_peered":"2024-04-25T09:53:06.774911+0000","last_unstale":"2024-04-25T09:53:13.326069+0000","last_undegraded":"2024-04-25T09:53:13.326069+0000","last_fullsized":"2024-04-25T09:53:13.326069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.898659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810239+0000","last_change":"2024-04-25T09:53:06.781528+0000","last_active":"2024-04-25T09:53:12.810239+0000","last_peered":"2024-04-25T09:53:12.810239+0000","last_clean":"2024-04-25T09:53:12.810239+0000","last_became_active":"2024-04-25T09:53:06.781248+0000","last_became_peered":"2024-04-25T09:53:06.781248+0000","last_unstale":"2024-04-25T09:53:12.810239+0000","last_undegraded":"2024-04-25T09:53:12.810239+0000","last_fullsized":"2024-04-25T09:53:12.810239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325829+0000","last_change":"2024-04-25T09:53:07.783095+0000","last_active":"2024-04-25T09:53:13.325829+0000","last_peered":"2024-04-25T09:53:13.325829+0000","last_clean":"2024-04-25T09:53:13.325829+0000","last_became_active":"2024-04-25T09:53:07.782578+0000","last_became_peered":"2024-04-25T09:53:07.782578+0000","last_unstale":"2024-04-25T09:53:13.325829+0000","last_undegraded":"2024-04-25T09:53:13.325829+0000","last_fullsized":"2024-04-25T09:53:13.325829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809581+0000","last_change":"2024-04-25T09:53:11.834200+0000","last_active":"2024-04-25T09:53:12.809581+0000","last_peered":"2024-04-25T09:53:12.809581+0000","last_clean":"2024-04-25T09:53:12.809581+0000","last_became_active":"2024-04-25T09:53:11.833774+0000","last_became_peered":"2024-04-25T09:53:11.833774+0000","last_unstale":"2024-04-25T09:53:12.809581+0000","last_undegraded":"2024-04-25T09:53:12.809581+0000","last_fullsized":"2024-04-25T09:53:12.809581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:42.139275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325379+0000","last_change":"2024-04-25T09:53:06.777867+0000","last_active":"2024-04-25T09:53:13.325379+0000","last_peered":"2024-04-25T09:53:13.325379+0000","last_clean":"2024-04-25T09:53:13.325379+0000","last_became_active":"2024-04-25T09:53:06.777566+0000","last_became_peered":"2024-04-25T09:53:06.777566+0000","last_unstale":"2024-04-25T09:53:13.325379+0000","last_undegraded":"2024-04-25T09:53:13.325379+0000","last_fullsized":"2024-04-25T09:53:13.325379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:52.306054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326173+0000","last_change":"2024-04-25T09:53:07.781653+0000","last_active":"2024-04-25T09:53:13.326173+0000","last_peered":"2024-04-25T09:53:13.326173+0000","last_clean":"2024-04-25T09:53:13.326173+0000","last_became_active":"2024-04-25T09:53:07.781274+0000","last_became_peered":"2024-04-25T09:53:07.781274+0000","last_unstale":"2024-04-25T09:53:13.326173+0000","last_undegraded":"2024-04-25T09:53:13.326173+0000","last_fullsized":"2024-04-25T09:53:13.326173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:02.037522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326829+0000","last_change":"2024-04-25T09:53:11.834554+0000","last_active":"2024-04-25T09:53:13.326829+0000","last_peered":"2024-04-25T09:53:13.326829+0000","last_clean":"2024-04-25T09:53:13.326829+0000","last_became_active":"2024-04-25T09:53:11.834143+0000","last_became_peered":"2024-04-25T09:53:11.834143+0000","last_unstale":"2024-04-25T09:53:13.326829+0000","last_undegraded":"2024-04-25T09:53:13.326829+0000","last_fullsized":"2024-04-25T09:53:13.326829+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:37.372311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809684+0000","last_change":"2024-04-25T09:53:07.798578+0000","last_active":"2024-04-25T09:53:12.809684+0000","last_peered":"2024-04-25T09:53:12.809684+0000","last_clean":"2024-04-25T09:53:12.809684+0000","last_became_active":"2024-04-25T09:53:07.798310+0000","last_became_peered":"2024-04-25T09:53:07.798310+0000","last_unstale":"2024-04-25T09:53:12.809684+0000","last_undegraded":"2024-04-25T09:53:12.809684+0000","last_fullsized":"2024-04-25T09:53:12.809684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809471+0000","last_change":"2024-04-25T09:53:07.780646+0000","last_active":"2024-04-25T09:53:12.809471+0000","last_peered":"2024-04-25T09:53:12.809471+0000","last_clean":"2024-04-25T09:53:12.809471+0000","last_became_active":"2024-04-25T09:53:07.780321+0000","last_became_peered":"2024-04-25T09:53:07.780321+0000","last_unstale":"2024-04-25T09:53:12.809471+0000","last_undegraded":"2024-04-25T09:53:12.809471+0000","last_fullsized":"2024-04-25T09:53:12.809471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:44.576758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326872+0000","last_change":"2024-04-25T09:53:07.787491+0000","last_active":"2024-04-25T09:53:13.326872+0000","last_peered":"2024-04-25T09:53:13.326872+0000","last_clean":"2024-04-25T09:53:13.326872+0000","last_became_active":"2024-04-25T09:53:07.787214+0000","last_became_peered":"2024-04-25T09:53:07.787214+0000","last_unstale":"2024-04-25T09:53:13.326872+0000","last_undegraded":"2024-04-25T09:53:13.326872+0000","last_fullsized":"2024-04-25T09:53:13.326872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:35.714783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325728+0000","last_change":"2024-04-25T09:53:07.789805+0000","last_active":"2024-04-25T09:53:13.325728+0000","last_peered":"2024-04-25T09:53:13.325728+0000","last_clean":"2024-04-25T09:53:13.325728+0000","last_became_active":"2024-04-25T09:53:07.788987+0000","last_became_peered":"2024-04-25T09:53:07.788987+0000","last_unstale":"2024-04-25T09:53:13.325728+0000","last_undegraded":"2024-04-25T09:53:13.325728+0000","last_fullsized":"2024-04-25T09:53:13.325728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326515+0000","last_change":"2024-04-25T09:53:07.792754+0000","last_active":"2024-04-25T09:53:13.326515+0000","last_peered":"2024-04-25T09:53:13.326515+0000","last_clean":"2024-04-25T09:53:13.326515+0000","last_became_active":"2024-04-25T09:53:07.792223+0000","last_became_peered":"2024-04-25T09:53:07.792223+0000","last_unstale":"2024-04-25T09:53:13.326515+0000","last_undegraded":"2024-04-25T09:53:13.326515+0000","last_fullsized":"2024-04-25T09:53:13.326515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325889+0000","last_change":"2024-04-25T09:53:07.783936+0000","last_active":"2024-04-25T09:53:13.325889+0000","last_peered":"2024-04-25T09:53:13.325889+0000","last_clean":"2024-04-25T09:53:13.325889+0000","last_became_active":"2024-04-25T09:53:07.783646+0000","last_became_peered":"2024-04-25T09:53:07.783646+0000","last_unstale":"2024-04-25T09:53:13.325889+0000","last_undegraded":"2024-04-25T09:53:13.325889+0000","last_fullsized":"2024-04-25T09:53:13.325889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810830+0000","last_change":"2024-04-25T09:53:07.789428+0000","last_active":"2024-04-25T09:53:12.810830+0000","last_peered":"2024-04-25T09:53:12.810830+0000","last_clean":"2024-04-25T09:53:12.810830+0000","last_became_active":"2024-04-25T09:53:07.789086+0000","last_became_peered":"2024-04-25T09:53:07.789086+0000","last_unstale":"2024-04-25T09:53:12.810830+0000","last_undegraded":"2024-04-25T09:53:12.810830+0000","last_fullsized":"2024-04-25T09:53:12.810830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327059+0000","last_change":"2024-04-25T09:53:07.785822+0000","last_active":"2024-04-25T09:53:13.327059+0000","last_peered":"2024-04-25T09:53:13.327059+0000","last_clean":"2024-04-25T09:53:13.327059+0000","last_became_active":"2024-04-25T09:53:07.785535+0000","last_became_peered":"2024-04-25T09:53:07.785535+0000","last_unstale":"2024-04-25T09:53:13.327059+0000","last_undegraded":"2024-04-25T09:53:13.327059+0000","last_fullsized":"2024-04-25T09:53:13.327059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325833+0000","last_change":"2024-04-25T09:53:07.787296+0000","last_active":"2024-04-25T09:53:13.325833+0000","last_peered":"2024-04-25T09:53:13.325833+0000","last_clean":"2024-04-25T09:53:13.325833+0000","last_became_active":"2024-04-25T09:53:07.787011+0000","last_became_peered":"2024-04-25T09:53:07.787011+0000","last_unstale":"2024-04-25T09:53:13.325833+0000","last_undegraded":"2024-04-25T09:53:13.325833+0000","last_fullsized":"2024-04-25T09:53:13.325833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325938+0000","last_change":"2024-04-25T09:53:07.785405+0000","last_active":"2024-04-25T09:53:13.325938+0000","last_peered":"2024-04-25T09:53:13.325938+0000","last_clean":"2024-04-25T09:53:13.325938+0000","last_became_active":"2024-04-25T09:53:07.785085+0000","last_became_peered":"2024-04-25T09:53:07.785085+0000","last_unstale":"2024-04-25T09:53:13.325938+0000","last_undegraded":"2024-04-25T09:53:13.325938+0000","last_fullsized":"2024-04-25T09:53:13.325938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326705+0000","last_change":"2024-04-25T09:53:07.792913+0000","last_active":"2024-04-25T09:53:13.326705+0000","last_peered":"2024-04-25T09:53:13.326705+0000","last_clean":"2024-04-25T09:53:13.326705+0000","last_became_active":"2024-04-25T09:53:07.792666+0000","last_became_peered":"2024-04-25T09:53:07.792666+0000","last_unstale":"2024-04-25T09:53:13.326705+0000","last_undegraded":"2024-04-25T09:53:13.326705+0000","last_fullsized":"2024-04-25T09:53:13.326705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325521+0000","last_change":"2024-04-25T09:53:07.789785+0000","last_active":"2024-04-25T09:53:13.325521+0000","last_peered":"2024-04-25T09:53:13.325521+0000","last_clean":"2024-04-25T09:53:13.325521+0000","last_became_active":"2024-04-25T09:53:07.789109+0000","last_became_peered":"2024-04-25T09:53:07.789109+0000","last_unstale":"2024-04-25T09:53:13.325521+0000","last_undegraded":"2024-04-25T09:53:13.325521+0000","last_fullsized":"2024-04-25T09:53:13.325521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:28.708261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811599+0000","last_change":"2024-04-25T09:53:07.800839+0000","last_active":"2024-04-25T09:53:12.811599+0000","last_peered":"2024-04-25T09:53:12.811599+0000","last_clean":"2024-04-25T09:53:12.811599+0000","last_became_active":"2024-04-25T09:53:07.800570+0000","last_became_peered":"2024-04-25T09:53:07.800570+0000","last_unstale":"2024-04-25T09:53:12.811599+0000","last_undegraded":"2024-04-25T09:53:12.811599+0000","last_fullsized":"2024-04-25T09:53:12.811599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810497+0000","last_change":"2024-04-25T09:53:07.793203+0000","last_active":"2024-04-25T09:53:12.810497+0000","last_peered":"2024-04-25T09:53:12.810497+0000","last_clean":"2024-04-25T09:53:12.810497+0000","last_became_active":"2024-04-25T09:53:07.792940+0000","last_became_peered":"2024-04-25T09:53:07.792940+0000","last_unstale":"2024-04-25T09:53:12.810497+0000","last_undegraded":"2024-04-25T09:53:12.810497+0000","last_fullsized":"2024-04-25T09:53:12.810497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.221246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809630+0000","last_change":"2024-04-25T09:53:07.789420+0000","last_active":"2024-04-25T09:53:12.809630+0000","last_peered":"2024-04-25T09:53:12.809630+0000","last_clean":"2024-04-25T09:53:12.809630+0000","last_became_active":"2024-04-25T09:53:07.789112+0000","last_became_peered":"2024-04-25T09:53:07.789112+0000","last_unstale":"2024-04-25T09:53:12.809630+0000","last_undegraded":"2024-04-25T09:53:12.809630+0000","last_fullsized":"2024-04-25T09:53:12.809630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.768421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327332+0000","last_change":"2024-04-25T09:53:07.794475+0000","last_active":"2024-04-25T09:53:13.327332+0000","last_peered":"2024-04-25T09:53:13.327332+0000","last_clean":"2024-04-25T09:53:13.327332+0000","last_became_active":"2024-04-25T09:53:07.794225+0000","last_became_peered":"2024-04-25T09:53:07.794225+0000","last_unstale":"2024-04-25T09:53:13.327332+0000","last_undegraded":"2024-04-25T09:53:13.327332+0000","last_fullsized":"2024-04-25T09:53:13.327332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:47.687838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325668+0000","last_change":"2024-04-25T09:53:07.798926+0000","last_active":"2024-04-25T09:53:13.325668+0000","last_peered":"2024-04-25T09:53:13.325668+0000","last_clean":"2024-04-25T09:53:13.325668+0000","last_became_active":"2024-04-25T09:53:07.798446+0000","last_became_peered":"2024-04-25T09:53:07.798446+0000","last_unstale":"2024-04-25T09:53:13.325668+0000","last_undegraded":"2024-04-25T09:53:13.325668+0000","last_fullsized":"2024-04-25T09:53:13.325668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325115+0000","last_change":"2024-04-25T09:53:07.784610+0000","last_active":"2024-04-25T09:53:13.325115+0000","last_peered":"2024-04-25T09:53:13.325115+0000","last_clean":"2024-04-25T09:53:13.325115+0000","last_became_active":"2024-04-25T09:53:07.784323+0000","last_became_peered":"2024-04-25T09:53:07.784323+0000","last_unstale":"2024-04-25T09:53:13.325115+0000","last_undegraded":"2024-04-25T09:53:13.325115+0000","last_fullsized":"2024-04-25T09:53:13.325115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326504+0000","last_change":"2024-04-25T09:53:07.781472+0000","last_active":"2024-04-25T09:53:13.326504+0000","last_peered":"2024-04-25T09:53:13.326504+0000","last_clean":"2024-04-25T09:53:13.326504+0000","last_became_active":"2024-04-25T09:53:07.781156+0000","last_became_peered":"2024-04-25T09:53:07.781156+0000","last_unstale":"2024-04-25T09:53:13.326504+0000","last_undegraded":"2024-04-25T09:53:13.326504+0000","last_fullsized":"2024-04-25T09:53:13.326504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:49.469537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325209+0000","last_change":"2024-04-25T09:53:07.790342+0000","last_active":"2024-04-25T09:53:13.325209+0000","last_peered":"2024-04-25T09:53:13.325209+0000","last_clean":"2024-04-25T09:53:13.325209+0000","last_became_active":"2024-04-25T09:53:07.790003+0000","last_became_peered":"2024-04-25T09:53:07.790003+0000","last_unstale":"2024-04-25T09:53:13.325209+0000","last_undegraded":"2024-04-25T09:53:13.325209+0000","last_fullsized":"2024-04-25T09:53:13.325209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326892+0000","last_change":"2024-04-25T09:53:07.791198+0000","last_active":"2024-04-25T09:53:13.326892+0000","last_peered":"2024-04-25T09:53:13.326892+0000","last_clean":"2024-04-25T09:53:13.326892+0000","last_became_active":"2024-04-25T09:53:07.790896+0000","last_became_peered":"2024-04-25T09:53:07.790896+0000","last_unstale":"2024-04-25T09:53:13.326892+0000","last_undegraded":"2024-04-25T09:53:13.326892+0000","last_fullsized":"2024-04-25T09:53:13.326892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:45.411393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810894+0000","last_change":"2024-04-25T09:53:07.790314+0000","last_active":"2024-04-25T09:53:12.810894+0000","last_peered":"2024-04-25T09:53:12.810894+0000","last_clean":"2024-04-25T09:53:12.810894+0000","last_became_active":"2024-04-25T09:53:07.789999+0000","last_became_peered":"2024-04-25T09:53:07.789999+0000","last_unstale":"2024-04-25T09:53:12.810894+0000","last_undegraded":"2024-04-25T09:53:12.810894+0000","last_fullsized":"2024-04-25T09:53:12.810894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810741+0000","last_change":"2024-04-25T09:53:07.793396+0000","last_active":"2024-04-25T09:53:12.810741+0000","last_peered":"2024-04-25T09:53:12.810741+0000","last_clean":"2024-04-25T09:53:12.810741+0000","last_became_active":"2024-04-25T09:53:07.792132+0000","last_became_peered":"2024-04-25T09:53:07.792132+0000","last_unstale":"2024-04-25T09:53:12.810741+0000","last_undegraded":"2024-04-25T09:53:12.810741+0000","last_fullsized":"2024-04-25T09:53:12.810741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811509+0000","last_change":"2024-04-25T09:53:07.800101+0000","last_active":"2024-04-25T09:53:12.811509+0000","last_peered":"2024-04-25T09:53:12.811509+0000","last_clean":"2024-04-25T09:53:12.811509+0000","last_became_active":"2024-04-25T09:53:07.799794+0000","last_became_peered":"2024-04-25T09:53:07.799794+0000","last_unstale":"2024-04-25T09:53:12.811509+0000","last_undegraded":"2024-04-25T09:53:12.811509+0000","last_fullsized":"2024-04-25T09:53:12.811509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:12.845651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325964+0000","last_change":"2024-04-25T09:53:07.793404+0000","last_active":"2024-04-25T09:53:13.325964+0000","last_peered":"2024-04-25T09:53:13.325964+0000","last_clean":"2024-04-25T09:53:13.325964+0000","last_became_active":"2024-04-25T09:53:07.793140+0000","last_became_peered":"2024-04-25T09:53:07.793140+0000","last_unstale":"2024-04-25T09:53:13.325964+0000","last_undegraded":"2024-04-25T09:53:13.325964+0000","last_fullsized":"2024-04-25T09:53:13.325964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809861+0000","last_change":"2024-04-25T09:53:07.797288+0000","last_active":"2024-04-25T09:53:12.809861+0000","last_peered":"2024-04-25T09:53:12.809861+0000","last_clean":"2024-04-25T09:53:12.809861+0000","last_became_active":"2024-04-25T09:53:07.797017+0000","last_became_peered":"2024-04-25T09:53:07.797017+0000","last_unstale":"2024-04-25T09:53:12.809861+0000","last_undegraded":"2024-04-25T09:53:12.809861+0000","last_fullsized":"2024-04-25T09:53:12.809861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326593+0000","last_change":"2024-04-25T09:53:07.800134+0000","last_active":"2024-04-25T09:53:13.326593+0000","last_peered":"2024-04-25T09:53:13.326593+0000","last_clean":"2024-04-25T09:53:13.326593+0000","last_became_active":"2024-04-25T09:53:07.799817+0000","last_became_peered":"2024-04-25T09:53:07.799817+0000","last_unstale":"2024-04-25T09:53:13.326593+0000","last_undegraded":"2024-04-25T09:53:13.326593+0000","last_fullsized":"2024-04-25T09:53:13.326593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810285+0000","last_change":"2024-04-25T09:53:07.792436+0000","last_active":"2024-04-25T09:53:12.810285+0000","last_peered":"2024-04-25T09:53:12.810285+0000","last_clean":"2024-04-25T09:53:12.810285+0000","last_became_active":"2024-04-25T09:53:07.792150+0000","last_became_peered":"2024-04-25T09:53:07.792150+0000","last_unstale":"2024-04-25T09:53:12.810285+0000","last_undegraded":"2024-04-25T09:53:12.810285+0000","last_fullsized":"2024-04-25T09:53:12.810285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326560+0000","last_change":"2024-04-25T09:53:07.788302+0000","last_active":"2024-04-25T09:53:13.326560+0000","last_peered":"2024-04-25T09:53:13.326560+0000","last_clean":"2024-04-25T09:53:13.326560+0000","last_became_active":"2024-04-25T09:53:07.788033+0000","last_became_peered":"2024-04-25T09:53:07.788033+0000","last_unstale":"2024-04-25T09:53:13.326560+0000","last_undegraded":"2024-04-25T09:53:13.326560+0000","last_fullsized":"2024-04-25T09:53:13.326560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325682+0000","last_change":"2024-04-25T09:53:07.776111+0000","last_active":"2024-04-25T09:53:13.325682+0000","last_peered":"2024-04-25T09:53:13.325682+0000","last_clean":"2024-04-25T09:53:13.325682+0000","last_became_active":"2024-04-25T09:53:07.775779+0000","last_became_peered":"2024-04-25T09:53:07.775779+0000","last_unstale":"2024-04-25T09:53:13.325682+0000","last_undegraded":"2024-04-25T09:53:13.325682+0000","last_fullsized":"2024-04-25T09:53:13.325682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325773+0000","last_change":"2024-04-25T09:53:07.782293+0000","last_active":"2024-04-25T09:53:13.325773+0000","last_peered":"2024-04-25T09:53:13.325773+0000","last_clean":"2024-04-25T09:53:13.325773+0000","last_became_active":"2024-04-25T09:53:07.781957+0000","last_became_peered":"2024-04-25T09:53:07.781957+0000","last_unstale":"2024-04-25T09:53:13.325773+0000","last_undegraded":"2024-04-25T09:53:13.325773+0000","last_fullsized":"2024-04-25T09:53:13.325773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811414+0000","last_change":"2024-04-25T09:53:07.791822+0000","last_active":"2024-04-25T09:53:12.811414+0000","last_peered":"2024-04-25T09:53:12.811414+0000","last_clean":"2024-04-25T09:53:12.811414+0000","last_became_active":"2024-04-25T09:53:07.791566+0000","last_became_peered":"2024-04-25T09:53:07.791566+0000","last_unstale":"2024-04-25T09:53:12.811414+0000","last_undegraded":"2024-04-25T09:53:12.811414+0000","last_fullsized":"2024-04-25T09:53:12.811414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:35.714783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:53:27.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:53:28.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.013+0000 7f61ed4ea640 1 Processor -- start 2024-04-25T09:53:28.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.013+0000 7f61ed4ea640 1 -- start start 2024-04-25T09:53:28.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61ed4ea640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f61e80b2cb0 0x7f61e80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61ed4ea640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61e80d1020 0x7f61e80d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61ed4ea640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e80b97a0 con 0x7f61e815dbe0 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61ed4ea640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e815dfc0 con 0x7f61e80d1020 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61ed4ea640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e80d1a00 con 0x7f61e80b2cb0 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61e6ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f61e815dbe0 0x7f61e80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47492/0 (socket says 172.21.15.19:47492) 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61e6ffd640 1 -- 172.21.15.19:0/996051635 learned_addr learned my addr 172.21.15.19:0/996051635 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61e77fe640 1 --2- 172.21.15.19:0/996051635 >> v2:172.21.15.19:3300/0 conn(0x7f61e80d1020 0x7f61e80d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61e67fc640 1 --2- 172.21.15.19:0/996051635 >> v2:172.21.15.112:3300/0 conn(0x7f61e80b2cb0 0x7f61e80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61e5ffb640 1 -- 172.21.15.19:0/996051635 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581860806 0 0) 0x7f61e80b97a0 con 0x7f61e815dbe0 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61e5ffb640 1 -- 172.21.15.19:0/996051635 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d4003520 con 0x7f61e815dbe0 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61e77fe640 1 -- 172.21.15.19:0/996051635 >> v2:172.21.15.112:3300/0 conn(0x7f61e80b2cb0 msgr2=0x7f61e80d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:28.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61e77fe640 1 --2- 172.21.15.19:0/996051635 >> v2:172.21.15.112:3300/0 conn(0x7f61e80b2cb0 0x7f61e80d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61e77fe640 1 -- 172.21.15.19:0/996051635 >> v1:172.21.15.19:6789/0 conn(0x7f61e815dbe0 legacy=0x7f61e80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.014+0000 7f61e77fe640 1 -- 172.21.15.19:0/996051635 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e80d1c50 con 0x7f61e80d1020 2024-04-25T09:53:28.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.015+0000 7f61e77fe640 1 --2- 172.21.15.19:0/996051635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e80d1020 0x7f61e80d2c70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f61e806cff0 tx=0x7f61d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabe596be2cbf08e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.015+0000 7f61e5ffb640 1 -- 172.21.15.19:0/996051635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8007740 con 0x7f61e80d1020 2024-04-25T09:53:28.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.015+0000 7f61e5ffb640 1 -- 172.21.15.19:0/996051635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d80078e0 con 0x7f61e80d1020 2024-04-25T09:53:28.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.015+0000 7f61e5ffb640 1 -- 172.21.15.19:0/996051635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8007cb0 con 0x7f61e80d1020 2024-04-25T09:53:28.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.016+0000 7f61ed4ea640 1 -- 172.21.15.19:0/996051635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e80d1020 msgr2=0x7f61e80d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.016+0000 7f61ed4ea640 1 --2- 172.21.15.19:0/996051635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e80d1020 0x7f61e80d2c70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f61e806cff0 tx=0x7f61d802a680 comp rx=0 tx=0).stop 2024-04-25T09:53:28.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.016+0000 7f61ed4ea640 1 -- 172.21.15.19:0/996051635 shutdown_connections 2024-04-25T09:53:28.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.016+0000 7f61ed4ea640 1 --2- 172.21.15.19:0/996051635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e80d1020 0x7f61e80d2c70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.016+0000 7f61ed4ea640 1 --2- 172.21.15.19:0/996051635 >> v2:172.21.15.112:3300/0 conn(0x7f61e80b2cb0 0x7f61e80d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.016+0000 7f61ed4ea640 1 -- 172.21.15.19:0/996051635 >> 172.21.15.19:0/996051635 conn(0x7f61e8061a20 msgr2=0x7f61e8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.016+0000 7f61ed4ea640 1 -- 172.21.15.19:0/996051635 shutdown_connections 2024-04-25T09:53:28.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.016+0000 7f61ed4ea640 1 -- 172.21.15.19:0/996051635 wait complete. 2024-04-25T09:53:28.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.017+0000 7f61ed4ea640 1 Processor -- start 2024-04-25T09:53:28.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.017+0000 7f61ed4ea640 1 -- start start 2024-04-25T09:53:28.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61ed4ea640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61e80b2cb0 0x7f61e80c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61ed4ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61e80d1020 0x7f61e80b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61ed4ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e815dbe0 0x7f61e8153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61ed4ea640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80b92a0 con 0x7f61e815dbe0 2024-04-25T09:53:28.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61ed4ea640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80d2210 con 0x7f61e80d1020 2024-04-25T09:53:28.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61ed4ea640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80b9420 con 0x7f61e80b2cb0 2024-04-25T09:53:28.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e815dbe0 0x7f61e8153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e815dbe0 0x7f61e8153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37220/0 (socket says 172.21.15.19:37220) 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e77fe640 1 -- 172.21.15.19:0/543280736 learned_addr learned my addr 172.21.15.19:0/543280736 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e6ffd640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61e80b2cb0 0x7f61e80c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e67fc640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61e80d1020 0x7f61e80b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e77fe640 1 -- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61e80b2cb0 msgr2=0x7f61e80c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e77fe640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61e80b2cb0 0x7f61e80c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e77fe640 1 -- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61e80d1020 msgr2=0x7f61e80b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e77fe640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61e80d1020 0x7f61e80b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e77fe640 1 -- 172.21.15.19:0/543280736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e80b3650 con 0x7f61e815dbe0 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e67fc640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61e80d1020 0x7f61e80b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.018+0000 7f61e6ffd640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61e80b2cb0 0x7f61e80c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.019+0000 7f61e77fe640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e815dbe0 0x7f61e8153820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009e50 tx=0x7f61d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.019+0000 7f61df7fe640 1 -- 172.21.15.19:0/543280736 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8040070 con 0x7f61e815dbe0 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.019+0000 7f61df7fe640 1 -- 172.21.15.19:0/543280736 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d8031bf0 con 0x7f61e815dbe0 2024-04-25T09:53:28.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.019+0000 7f61df7fe640 1 -- 172.21.15.19:0/543280736 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8009450 con 0x7f61e815dbe0 2024-04-25T09:53:28.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.019+0000 7f61ed4ea640 1 -- 172.21.15.19:0/543280736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e80b3930 con 0x7f61e815dbe0 2024-04-25T09:53:28.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.019+0000 7f61ed4ea640 1 -- 172.21.15.19:0/543280736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e80b3ef0 con 0x7f61e815dbe0 2024-04-25T09:53:28.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.020+0000 7f61ed4ea640 1 -- 172.21.15.19:0/543280736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e8073990 con 0x7f61e815dbe0 2024-04-25T09:53:28.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.020+0000 7f61df7fe640 1 -- 172.21.15.19:0/543280736 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61d8039040 con 0x7f61e815dbe0 2024-04-25T09:53:28.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.021+0000 7f61df7fe640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f61c806b4d0 0x7f61c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.021+0000 7f61df7fe640 1 -- 172.21.15.19:0/543280736 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f61d80abd50 con 0x7f61e815dbe0 2024-04-25T09:53:28.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.021+0000 7f61e6ffd640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f61c806b4d0 0x7f61c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.021+0000 7f61e6ffd640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f61c806b4d0 0x7f61c806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f61e0007fd0 tx=0x7f61e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.025+0000 7f61df7fe640 1 -- 172.21.15.19:0/543280736 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61d8043360 con 0x7f61e815dbe0 2024-04-25T09:53:28.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.202+0000 7f61ed4ea640 1 -- 172.21.15.19:0/543280736 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61e8155080 con 0x7f61c806b4d0 2024-04-25T09:53:28.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.211+0000 7f61df7fe640 1 -- 172.21.15.19:0/543280736 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f61e8155080 con 0x7f61c806b4d0 2024-04-25T09:53:28.216 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 -- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f61c806b4d0 msgr2=0x7f61c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f61c806b4d0 0x7f61c806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f61e0007fd0 tx=0x7f61e0009040 comp rx=0 tx=0).stop 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 -- 172.21.15.19:0/543280736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e815dbe0 msgr2=0x7f61e8153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e815dbe0 0x7f61e8153820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009e50 tx=0x7f61d8004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 -- 172.21.15.19:0/543280736 shutdown_connections 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f61c806b4d0 0x7f61c806d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61e815dbe0 0x7f61e8153820 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61e80d1020 0x7f61e80b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 --2- 172.21.15.19:0/543280736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61e80b2cb0 0x7f61e80c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 -- 172.21.15.19:0/543280736 >> 172.21.15.19:0/543280736 conn(0x7f61e8061a20 msgr2=0x7f61e8158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 -- 172.21.15.19:0/543280736 shutdown_connections 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.214+0000 7f61ed4ea640 1 -- 172.21.15.19:0/543280736 wait complete. 2024-04-25T09:53:28.227 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-25T09:53:26.915412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219368,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755352,"statfs":{"total":773094113280,"available":772869480448,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003738"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326249+0000","last_change":"2024-04-25T09:53:06.778392+0000","last_active":"2024-04-25T09:53:13.326249+0000","last_peered":"2024-04-25T09:53:13.326249+0000","last_clean":"2024-04-25T09:53:13.326249+0000","last_became_active":"2024-04-25T09:53:06.777894+0000","last_became_peered":"2024-04-25T09:53:06.777894+0000","last_unstale":"2024-04-25T09:53:13.326249+0000","last_undegraded":"2024-04-25T09:53:13.326249+0000","last_fullsized":"2024-04-25T09:53:13.326249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.898659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326801+0000","last_change":"2024-04-25T09:53:07.792733+0000","last_active":"2024-04-25T09:53:13.326801+0000","last_peered":"2024-04-25T09:53:13.326801+0000","last_clean":"2024-04-25T09:53:13.326801+0000","last_became_active":"2024-04-25T09:53:07.792294+0000","last_became_peered":"2024-04-25T09:53:07.792294+0000","last_unstale":"2024-04-25T09:53:13.326801+0000","last_undegraded":"2024-04-25T09:53:13.326801+0000","last_fullsized":"2024-04-25T09:53:13.326801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325683+0000","last_change":"2024-04-25T09:53:11.822234+0000","last_active":"2024-04-25T09:53:13.325683+0000","last_peered":"2024-04-25T09:53:13.325683+0000","last_clean":"2024-04-25T09:53:13.325683+0000","last_became_active":"2024-04-25T09:53:11.821231+0000","last_became_peered":"2024-04-25T09:53:11.821231+0000","last_unstale":"2024-04-25T09:53:13.325683+0000","last_undegraded":"2024-04-25T09:53:13.325683+0000","last_fullsized":"2024-04-25T09:53:13.325683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:19.880548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327066+0000","last_change":"2024-04-25T09:53:06.779890+0000","last_active":"2024-04-25T09:53:13.327066+0000","last_peered":"2024-04-25T09:53:13.327066+0000","last_clean":"2024-04-25T09:53:13.327066+0000","last_became_active":"2024-04-25T09:53:06.779482+0000","last_became_peered":"2024-04-25T09:53:06.779482+0000","last_unstale":"2024-04-25T09:53:13.327066+0000","last_undegraded":"2024-04-25T09:53:13.327066+0000","last_fullsized":"2024-04-25T09:53:13.327066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325889+0000","last_change":"2024-04-25T09:53:07.789471+0000","last_active":"2024-04-25T09:53:13.325889+0000","last_peered":"2024-04-25T09:53:13.325889+0000","last_clean":"2024-04-25T09:53:13.325889+0000","last_became_active":"2024-04-25T09:53:07.789190+0000","last_became_peered":"2024-04-25T09:53:07.789190+0000","last_unstale":"2024-04-25T09:53:13.325889+0000","last_undegraded":"2024-04-25T09:53:13.325889+0000","last_fullsized":"2024-04-25T09:53:13.325889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326006+0000","last_change":"2024-04-25T09:53:11.841059+0000","last_active":"2024-04-25T09:53:13.326006+0000","last_peered":"2024-04-25T09:53:13.326006+0000","last_clean":"2024-04-25T09:53:13.326006+0000","last_became_active":"2024-04-25T09:53:11.840640+0000","last_became_peered":"2024-04-25T09:53:11.840640+0000","last_unstale":"2024-04-25T09:53:13.326006+0000","last_undegraded":"2024-04-25T09:53:13.326006+0000","last_fullsized":"2024-04-25T09:53:13.326006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:54.172755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810382+0000","last_change":"2024-04-25T09:53:06.777792+0000","last_active":"2024-04-25T09:53:12.810382+0000","last_peered":"2024-04-25T09:53:12.810382+0000","last_clean":"2024-04-25T09:53:12.810382+0000","last_became_active":"2024-04-25T09:53:06.777459+0000","last_became_peered":"2024-04-25T09:53:06.777459+0000","last_unstale":"2024-04-25T09:53:12.810382+0000","last_undegraded":"2024-04-25T09:53:12.810382+0000","last_fullsized":"2024-04-25T09:53:12.810382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:44.406768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326553+0000","last_change":"2024-04-25T09:53:07.798127+0000","last_active":"2024-04-25T09:53:13.326553+0000","last_peered":"2024-04-25T09:53:13.326553+0000","last_clean":"2024-04-25T09:53:13.326553+0000","last_became_active":"2024-04-25T09:53:07.797863+0000","last_became_peered":"2024-04-25T09:53:07.797863+0000","last_unstale":"2024-04-25T09:53:13.326553+0000","last_undegraded":"2024-04-25T09:53:13.326553+0000","last_fullsized":"2024-04-25T09:53:13.326553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326109+0000","last_change":"2024-04-25T09:53:11.841255+0000","last_active":"2024-04-25T09:53:13.326109+0000","last_peered":"2024-04-25T09:53:13.326109+0000","last_clean":"2024-04-25T09:53:13.326109+0000","last_became_active":"2024-04-25T09:53:11.840795+0000","last_became_peered":"2024-04-25T09:53:11.840795+0000","last_unstale":"2024-04-25T09:53:13.326109+0000","last_undegraded":"2024-04-25T09:53:13.326109+0000","last_fullsized":"2024-04-25T09:53:13.326109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:28.118453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810018+0000","last_change":"2024-04-25T09:53:06.784013+0000","last_active":"2024-04-25T09:53:12.810018+0000","last_peered":"2024-04-25T09:53:12.810018+0000","last_clean":"2024-04-25T09:53:12.810018+0000","last_became_active":"2024-04-25T09:53:06.783726+0000","last_became_peered":"2024-04-25T09:53:06.783726+0000","last_unstale":"2024-04-25T09:53:12.810018+0000","last_undegraded":"2024-04-25T09:53:12.810018+0000","last_fullsized":"2024-04-25T09:53:12.810018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327442+0000","last_change":"2024-04-25T09:53:07.796042+0000","last_active":"2024-04-25T09:53:13.327442+0000","last_peered":"2024-04-25T09:53:13.327442+0000","last_clean":"2024-04-25T09:53:13.327442+0000","last_became_active":"2024-04-25T09:53:07.795676+0000","last_became_peered":"2024-04-25T09:53:07.795676+0000","last_unstale":"2024-04-25T09:53:13.327442+0000","last_undegraded":"2024-04-25T09:53:13.327442+0000","last_fullsized":"2024-04-25T09:53:13.327442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:12.845651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326221+0000","last_change":"2024-04-25T09:53:11.827699+0000","last_active":"2024-04-25T09:53:13.326221+0000","last_peered":"2024-04-25T09:53:13.326221+0000","last_clean":"2024-04-25T09:53:13.326221+0000","last_became_active":"2024-04-25T09:53:11.827325+0000","last_became_peered":"2024-04-25T09:53:11.827325+0000","last_unstale":"2024-04-25T09:53:13.326221+0000","last_undegraded":"2024-04-25T09:53:13.326221+0000","last_fullsized":"2024-04-25T09:53:13.326221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.837684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810063+0000","last_change":"2024-04-25T09:53:07.800199+0000","last_active":"2024-04-25T09:53:12.810063+0000","last_peered":"2024-04-25T09:53:12.810063+0000","last_clean":"2024-04-25T09:53:12.810063+0000","last_became_active":"2024-04-25T09:53:07.799912+0000","last_became_peered":"2024-04-25T09:53:07.799912+0000","last_unstale":"2024-04-25T09:53:12.810063+0000","last_undegraded":"2024-04-25T09:53:12.810063+0000","last_fullsized":"2024-04-25T09:53:12.810063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810410+0000","last_change":"2024-04-25T09:53:06.783380+0000","last_active":"2024-04-25T09:53:12.810410+0000","last_peered":"2024-04-25T09:53:12.810410+0000","last_clean":"2024-04-25T09:53:12.810410+0000","last_became_active":"2024-04-25T09:53:06.783049+0000","last_became_peered":"2024-04-25T09:53:06.783049+0000","last_unstale":"2024-04-25T09:53:12.810410+0000","last_undegraded":"2024-04-25T09:53:12.810410+0000","last_fullsized":"2024-04-25T09:53:12.810410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327316+0000","last_change":"2024-04-25T09:53:11.839467+0000","last_active":"2024-04-25T09:53:13.327316+0000","last_peered":"2024-04-25T09:53:13.327316+0000","last_clean":"2024-04-25T09:53:13.327316+0000","last_became_active":"2024-04-25T09:53:11.839079+0000","last_became_peered":"2024-04-25T09:53:11.839079+0000","last_unstale":"2024-04-25T09:53:13.327316+0000","last_undegraded":"2024-04-25T09:53:13.327316+0000","last_fullsized":"2024-04-25T09:53:13.327316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325291+0000","last_change":"2024-04-25T09:53:07.791763+0000","last_active":"2024-04-25T09:53:13.325291+0000","last_peered":"2024-04-25T09:53:13.325291+0000","last_clean":"2024-04-25T09:53:13.325291+0000","last_became_active":"2024-04-25T09:53:07.790924+0000","last_became_peered":"2024-04-25T09:53:07.790924+0000","last_unstale":"2024-04-25T09:53:13.325291+0000","last_undegraded":"2024-04-25T09:53:13.325291+0000","last_fullsized":"2024-04-25T09:53:13.325291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326553+0000","last_change":"2024-04-25T09:53:06.779703+0000","last_active":"2024-04-25T09:53:13.326553+0000","last_peered":"2024-04-25T09:53:13.326553+0000","last_clean":"2024-04-25T09:53:13.326553+0000","last_became_active":"2024-04-25T09:53:06.779413+0000","last_became_peered":"2024-04-25T09:53:06.779413+0000","last_unstale":"2024-04-25T09:53:13.326553+0000","last_undegraded":"2024-04-25T09:53:13.326553+0000","last_fullsized":"2024-04-25T09:53:13.326553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326368+0000","last_change":"2024-04-25T09:53:11.840454+0000","last_active":"2024-04-25T09:53:13.326368+0000","last_peered":"2024-04-25T09:53:13.326368+0000","last_clean":"2024-04-25T09:53:13.326368+0000","last_became_active":"2024-04-25T09:53:11.840034+0000","last_became_peered":"2024-04-25T09:53:11.840034+0000","last_unstale":"2024-04-25T09:53:13.326368+0000","last_undegraded":"2024-04-25T09:53:13.326368+0000","last_fullsized":"2024-04-25T09:53:13.326368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:59.438696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326270+0000","last_change":"2024-04-25T09:53:07.786389+0000","last_active":"2024-04-25T09:53:13.326270+0000","last_peered":"2024-04-25T09:53:13.326270+0000","last_clean":"2024-04-25T09:53:13.326270+0000","last_became_active":"2024-04-25T09:53:07.786116+0000","last_became_peered":"2024-04-25T09:53:07.786116+0000","last_unstale":"2024-04-25T09:53:13.326270+0000","last_undegraded":"2024-04-25T09:53:13.326270+0000","last_fullsized":"2024-04-25T09:53:13.326270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325212+0000","last_change":"2024-04-25T09:53:06.780897+0000","last_active":"2024-04-25T09:53:13.325212+0000","last_peered":"2024-04-25T09:53:13.325212+0000","last_clean":"2024-04-25T09:53:13.325212+0000","last_became_active":"2024-04-25T09:53:06.780616+0000","last_became_peered":"2024-04-25T09:53:06.780616+0000","last_unstale":"2024-04-25T09:53:13.325212+0000","last_undegraded":"2024-04-25T09:53:13.325212+0000","last_fullsized":"2024-04-25T09:53:13.325212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:24.763796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326272+0000","last_change":"2024-04-25T09:53:11.838039+0000","last_active":"2024-04-25T09:53:13.326272+0000","last_peered":"2024-04-25T09:53:13.326272+0000","last_clean":"2024-04-25T09:53:13.326272+0000","last_became_active":"2024-04-25T09:53:11.837388+0000","last_became_peered":"2024-04-25T09:53:11.837388+0000","last_unstale":"2024-04-25T09:53:13.326272+0000","last_undegraded":"2024-04-25T09:53:13.326272+0000","last_fullsized":"2024-04-25T09:53:13.326272+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:39.753991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326308+0000","last_change":"2024-04-25T09:53:07.784394+0000","last_active":"2024-04-25T09:53:13.326308+0000","last_peered":"2024-04-25T09:53:13.326308+0000","last_clean":"2024-04-25T09:53:13.326308+0000","last_became_active":"2024-04-25T09:53:07.784060+0000","last_became_peered":"2024-04-25T09:53:07.784060+0000","last_unstale":"2024-04-25T09:53:13.326308+0000","last_undegraded":"2024-04-25T09:53:13.326308+0000","last_fullsized":"2024-04-25T09:53:13.326308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:36.593849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809684+0000","last_change":"2024-04-25T09:53:06.779579+0000","last_active":"2024-04-25T09:53:12.809684+0000","last_peered":"2024-04-25T09:53:12.809684+0000","last_clean":"2024-04-25T09:53:12.809684+0000","last_became_active":"2024-04-25T09:53:06.779296+0000","last_became_peered":"2024-04-25T09:53:06.779296+0000","last_unstale":"2024-04-25T09:53:12.809684+0000","last_undegraded":"2024-04-25T09:53:12.809684+0000","last_fullsized":"2024-04-25T09:53:12.809684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327226+0000","last_change":"2024-04-25T09:53:11.839381+0000","last_active":"2024-04-25T09:53:13.327226+0000","last_peered":"2024-04-25T09:53:13.327226+0000","last_clean":"2024-04-25T09:53:13.327226+0000","last_became_active":"2024-04-25T09:53:11.838941+0000","last_became_peered":"2024-04-25T09:53:11.838941+0000","last_unstale":"2024-04-25T09:53:13.327226+0000","last_undegraded":"2024-04-25T09:53:13.327226+0000","last_fullsized":"2024-04-25T09:53:13.327226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:59.415156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326930+0000","last_change":"2024-04-25T09:53:07.785627+0000","last_active":"2024-04-25T09:53:13.326930+0000","last_peered":"2024-04-25T09:53:13.326930+0000","last_clean":"2024-04-25T09:53:13.326930+0000","last_became_active":"2024-04-25T09:53:07.782773+0000","last_became_peered":"2024-04-25T09:53:07.782773+0000","last_unstale":"2024-04-25T09:53:13.326930+0000","last_undegraded":"2024-04-25T09:53:13.326930+0000","last_fullsized":"2024-04-25T09:53:13.326930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811412+0000","last_change":"2024-04-25T09:53:06.778161+0000","last_active":"2024-04-25T09:53:12.811412+0000","last_peered":"2024-04-25T09:53:12.811412+0000","last_clean":"2024-04-25T09:53:12.811412+0000","last_became_active":"2024-04-25T09:53:06.777837+0000","last_became_peered":"2024-04-25T09:53:06.777837+0000","last_unstale":"2024-04-25T09:53:12.811412+0000","last_undegraded":"2024-04-25T09:53:12.811412+0000","last_fullsized":"2024-04-25T09:53:12.811412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325940+0000","last_change":"2024-04-25T09:53:11.823677+0000","last_active":"2024-04-25T09:53:13.325940+0000","last_peered":"2024-04-25T09:53:13.325940+0000","last_clean":"2024-04-25T09:53:13.325940+0000","last_became_active":"2024-04-25T09:53:11.823278+0000","last_became_peered":"2024-04-25T09:53:11.823278+0000","last_unstale":"2024-04-25T09:53:13.325940+0000","last_undegraded":"2024-04-25T09:53:13.325940+0000","last_fullsized":"2024-04-25T09:53:13.325940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:45.277051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327064+0000","last_change":"2024-04-25T09:53:07.792276+0000","last_active":"2024-04-25T09:53:13.327064+0000","last_peered":"2024-04-25T09:53:13.327064+0000","last_clean":"2024-04-25T09:53:13.327064+0000","last_became_active":"2024-04-25T09:53:07.790961+0000","last_became_peered":"2024-04-25T09:53:07.790961+0000","last_unstale":"2024-04-25T09:53:13.327064+0000","last_undegraded":"2024-04-25T09:53:13.327064+0000","last_fullsized":"2024-04-25T09:53:13.327064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325875+0000","last_change":"2024-04-25T09:53:06.779465+0000","last_active":"2024-04-25T09:53:13.325875+0000","last_peered":"2024-04-25T09:53:13.325875+0000","last_clean":"2024-04-25T09:53:13.325875+0000","last_became_active":"2024-04-25T09:53:06.779122+0000","last_became_peered":"2024-04-25T09:53:06.779122+0000","last_unstale":"2024-04-25T09:53:13.325875+0000","last_undegraded":"2024-04-25T09:53:13.325875+0000","last_fullsized":"2024-04-25T09:53:13.325875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325863+0000","last_change":"2024-04-25T09:53:11.832958+0000","last_active":"2024-04-25T09:53:13.325863+0000","last_peered":"2024-04-25T09:53:13.325863+0000","last_clean":"2024-04-25T09:53:13.325863+0000","last_became_active":"2024-04-25T09:53:11.832522+0000","last_became_peered":"2024-04-25T09:53:11.832522+0000","last_unstale":"2024-04-25T09:53:13.325863+0000","last_undegraded":"2024-04-25T09:53:13.325863+0000","last_fullsized":"2024-04-25T09:53:13.325863+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:58.718292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325598+0000","last_change":"2024-04-25T09:53:07.779236+0000","last_active":"2024-04-25T09:53:13.325598+0000","last_peered":"2024-04-25T09:53:13.325598+0000","last_clean":"2024-04-25T09:53:13.325598+0000","last_became_active":"2024-04-25T09:53:07.778938+0000","last_became_peered":"2024-04-25T09:53:07.778938+0000","last_unstale":"2024-04-25T09:53:13.325598+0000","last_undegraded":"2024-04-25T09:53:13.325598+0000","last_fullsized":"2024-04-25T09:53:13.325598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810604+0000","last_change":"2024-04-25T09:53:06.780019+0000","last_active":"2024-04-25T09:53:12.810604+0000","last_peered":"2024-04-25T09:53:12.810604+0000","last_clean":"2024-04-25T09:53:12.810604+0000","last_became_active":"2024-04-25T09:53:06.779429+0000","last_became_peered":"2024-04-25T09:53:06.779429+0000","last_unstale":"2024-04-25T09:53:12.810604+0000","last_undegraded":"2024-04-25T09:53:12.810604+0000","last_fullsized":"2024-04-25T09:53:12.810604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:24.763796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325466+0000","last_change":"2024-04-25T09:53:11.835047+0000","last_active":"2024-04-25T09:53:13.325466+0000","last_peered":"2024-04-25T09:53:13.325466+0000","last_clean":"2024-04-25T09:53:13.325466+0000","last_became_active":"2024-04-25T09:53:11.833932+0000","last_became_peered":"2024-04-25T09:53:11.833932+0000","last_unstale":"2024-04-25T09:53:13.325466+0000","last_undegraded":"2024-04-25T09:53:13.325466+0000","last_fullsized":"2024-04-25T09:53:13.325466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:39.753991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327090+0000","last_change":"2024-04-25T09:53:07.786462+0000","last_active":"2024-04-25T09:53:13.327090+0000","last_peered":"2024-04-25T09:53:13.327090+0000","last_clean":"2024-04-25T09:53:13.327090+0000","last_became_active":"2024-04-25T09:53:07.785793+0000","last_became_peered":"2024-04-25T09:53:07.785793+0000","last_unstale":"2024-04-25T09:53:13.327090+0000","last_undegraded":"2024-04-25T09:53:13.327090+0000","last_fullsized":"2024-04-25T09:53:13.327090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.221246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325690+0000","last_change":"2024-04-25T09:53:06.780745+0000","last_active":"2024-04-25T09:53:13.325690+0000","last_peered":"2024-04-25T09:53:13.325690+0000","last_clean":"2024-04-25T09:53:13.325690+0000","last_became_active":"2024-04-25T09:53:06.780468+0000","last_became_peered":"2024-04-25T09:53:06.780468+0000","last_unstale":"2024-04-25T09:53:13.325690+0000","last_undegraded":"2024-04-25T09:53:13.325690+0000","last_fullsized":"2024-04-25T09:53:13.325690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326300+0000","last_change":"2024-04-25T09:53:11.832263+0000","last_active":"2024-04-25T09:53:13.326300+0000","last_peered":"2024-04-25T09:53:13.326300+0000","last_clean":"2024-04-25T09:53:13.326300+0000","last_became_active":"2024-04-25T09:53:11.831846+0000","last_became_peered":"2024-04-25T09:53:11.831846+0000","last_unstale":"2024-04-25T09:53:13.326300+0000","last_undegraded":"2024-04-25T09:53:13.326300+0000","last_fullsized":"2024-04-25T09:53:13.326300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:16.884859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326368+0000","last_change":"2024-04-25T09:53:07.797016+0000","last_active":"2024-04-25T09:53:13.326368+0000","last_peered":"2024-04-25T09:53:13.326368+0000","last_clean":"2024-04-25T09:53:13.326368+0000","last_became_active":"2024-04-25T09:53:07.796749+0000","last_became_peered":"2024-04-25T09:53:07.796749+0000","last_unstale":"2024-04-25T09:53:13.326368+0000","last_undegraded":"2024-04-25T09:53:13.326368+0000","last_fullsized":"2024-04-25T09:53:13.326368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:21.755503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326938+0000","last_change":"2024-04-25T09:53:06.782783+0000","last_active":"2024-04-25T09:53:13.326938+0000","last_peered":"2024-04-25T09:53:13.326938+0000","last_clean":"2024-04-25T09:53:13.326938+0000","last_became_active":"2024-04-25T09:53:06.782388+0000","last_became_peered":"2024-04-25T09:53:06.782388+0000","last_unstale":"2024-04-25T09:53:13.326938+0000","last_undegraded":"2024-04-25T09:53:13.326938+0000","last_fullsized":"2024-04-25T09:53:13.326938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325544+0000","last_change":"2024-04-25T09:53:11.832074+0000","last_active":"2024-04-25T09:53:13.325544+0000","last_peered":"2024-04-25T09:53:13.325544+0000","last_clean":"2024-04-25T09:53:13.325544+0000","last_became_active":"2024-04-25T09:53:11.831621+0000","last_became_peered":"2024-04-25T09:53:11.831621+0000","last_unstale":"2024-04-25T09:53:13.325544+0000","last_undegraded":"2024-04-25T09:53:13.325544+0000","last_fullsized":"2024-04-25T09:53:13.325544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.324944+0000","last_change":"2024-04-25T09:53:07.783910+0000","last_active":"2024-04-25T09:53:13.324944+0000","last_peered":"2024-04-25T09:53:13.324944+0000","last_clean":"2024-04-25T09:53:13.324944+0000","last_became_active":"2024-04-25T09:53:07.783625+0000","last_became_peered":"2024-04-25T09:53:07.783625+0000","last_unstale":"2024-04-25T09:53:13.324944+0000","last_undegraded":"2024-04-25T09:53:13.324944+0000","last_fullsized":"2024-04-25T09:53:13.324944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809538+0000","last_change":"2024-04-25T09:53:06.776134+0000","last_active":"2024-04-25T09:53:12.809538+0000","last_peered":"2024-04-25T09:53:12.809538+0000","last_clean":"2024-04-25T09:53:12.809538+0000","last_became_active":"2024-04-25T09:53:06.775787+0000","last_became_peered":"2024-04-25T09:53:06.775787+0000","last_unstale":"2024-04-25T09:53:12.809538+0000","last_undegraded":"2024-04-25T09:53:12.809538+0000","last_fullsized":"2024-04-25T09:53:12.809538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326396+0000","last_change":"2024-04-25T09:53:11.837526+0000","last_active":"2024-04-25T09:53:13.326396+0000","last_peered":"2024-04-25T09:53:13.326396+0000","last_clean":"2024-04-25T09:53:13.326396+0000","last_became_active":"2024-04-25T09:53:11.832037+0000","last_became_peered":"2024-04-25T09:53:11.832037+0000","last_unstale":"2024-04-25T09:53:13.326396+0000","last_undegraded":"2024-04-25T09:53:13.326396+0000","last_fullsized":"2024-04-25T09:53:13.326396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.727046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325424+0000","last_change":"2024-04-25T09:53:07.787155+0000","last_active":"2024-04-25T09:53:13.325424+0000","last_peered":"2024-04-25T09:53:13.325424+0000","last_clean":"2024-04-25T09:53:13.325424+0000","last_became_active":"2024-04-25T09:53:07.783553+0000","last_became_peered":"2024-04-25T09:53:07.783553+0000","last_unstale":"2024-04-25T09:53:13.325424+0000","last_undegraded":"2024-04-25T09:53:13.325424+0000","last_fullsized":"2024-04-25T09:53:13.325424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:12.089487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810045+0000","last_change":"2024-04-25T09:53:06.780626+0000","last_active":"2024-04-25T09:53:12.810045+0000","last_peered":"2024-04-25T09:53:12.810045+0000","last_clean":"2024-04-25T09:53:12.810045+0000","last_became_active":"2024-04-25T09:53:06.780364+0000","last_became_peered":"2024-04-25T09:53:06.780364+0000","last_unstale":"2024-04-25T09:53:12.810045+0000","last_undegraded":"2024-04-25T09:53:12.810045+0000","last_fullsized":"2024-04-25T09:53:12.810045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326877+0000","last_change":"2024-04-25T09:53:11.832887+0000","last_active":"2024-04-25T09:53:13.326877+0000","last_peered":"2024-04-25T09:53:13.326877+0000","last_clean":"2024-04-25T09:53:13.326877+0000","last_became_active":"2024-04-25T09:53:11.832471+0000","last_became_peered":"2024-04-25T09:53:11.832471+0000","last_unstale":"2024-04-25T09:53:13.326877+0000","last_undegraded":"2024-04-25T09:53:13.326877+0000","last_fullsized":"2024-04-25T09:53:13.326877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:03.438164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.324983+0000","last_change":"2024-04-25T09:53:07.793466+0000","last_active":"2024-04-25T09:53:13.324983+0000","last_peered":"2024-04-25T09:53:13.324983+0000","last_clean":"2024-04-25T09:53:13.324983+0000","last_became_active":"2024-04-25T09:53:07.793130+0000","last_became_peered":"2024-04-25T09:53:07.793130+0000","last_unstale":"2024-04-25T09:53:13.324983+0000","last_undegraded":"2024-04-25T09:53:13.324983+0000","last_fullsized":"2024-04-25T09:53:13.324983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326577+0000","last_change":"2024-04-25T09:53:06.780762+0000","last_active":"2024-04-25T09:53:13.326577+0000","last_peered":"2024-04-25T09:53:13.326577+0000","last_clean":"2024-04-25T09:53:13.326577+0000","last_became_active":"2024-04-25T09:53:06.780498+0000","last_became_peered":"2024-04-25T09:53:06.780498+0000","last_unstale":"2024-04-25T09:53:13.326577+0000","last_undegraded":"2024-04-25T09:53:13.326577+0000","last_fullsized":"2024-04-25T09:53:13.326577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810151+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:53:12.810151+0000","last_peered":"2024-04-25T09:53:12.810151+0000","last_clean":"2024-04-25T09:53:12.810151+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:53:12.810151+0000","last_undegraded":"2024-04-25T09:53:12.810151+0000","last_fullsized":"2024-04-25T09:53:12.810151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810776+0000","last_change":"2024-04-25T09:53:11.825407+0000","last_active":"2024-04-25T09:53:12.810776+0000","last_peered":"2024-04-25T09:53:12.810776+0000","last_clean":"2024-04-25T09:53:12.810776+0000","last_became_active":"2024-04-25T09:53:11.824448+0000","last_became_peered":"2024-04-25T09:53:11.824448+0000","last_unstale":"2024-04-25T09:53:12.810776+0000","last_undegraded":"2024-04-25T09:53:12.810776+0000","last_fullsized":"2024-04-25T09:53:12.810776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:59.415156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810231+0000","last_change":"2024-04-25T09:53:07.803634+0000","last_active":"2024-04-25T09:53:12.810231+0000","last_peered":"2024-04-25T09:53:12.810231+0000","last_clean":"2024-04-25T09:53:12.810231+0000","last_became_active":"2024-04-25T09:53:07.803367+0000","last_became_peered":"2024-04-25T09:53:07.803367+0000","last_unstale":"2024-04-25T09:53:12.810231+0000","last_undegraded":"2024-04-25T09:53:12.810231+0000","last_fullsized":"2024-04-25T09:53:12.810231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:55.375947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809562+0000","last_change":"2024-04-25T09:53:06.783978+0000","last_active":"2024-04-25T09:53:12.809562+0000","last_peered":"2024-04-25T09:53:12.809562+0000","last_clean":"2024-04-25T09:53:12.809562+0000","last_became_active":"2024-04-25T09:53:06.783614+0000","last_became_peered":"2024-04-25T09:53:06.783614+0000","last_unstale":"2024-04-25T09:53:12.809562+0000","last_undegraded":"2024-04-25T09:53:12.809562+0000","last_fullsized":"2024-04-25T09:53:12.809562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:52.306054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810011+0000","last_change":"2024-04-25T09:53:11.835086+0000","last_active":"2024-04-25T09:53:12.810011+0000","last_peered":"2024-04-25T09:53:12.810011+0000","last_clean":"2024-04-25T09:53:12.810011+0000","last_became_active":"2024-04-25T09:53:11.834674+0000","last_became_peered":"2024-04-25T09:53:11.834674+0000","last_unstale":"2024-04-25T09:53:12.810011+0000","last_undegraded":"2024-04-25T09:53:12.810011+0000","last_fullsized":"2024-04-25T09:53:12.810011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:01.272017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326681+0000","last_change":"2024-04-25T09:53:07.786966+0000","last_active":"2024-04-25T09:53:13.326681+0000","last_peered":"2024-04-25T09:53:13.326681+0000","last_clean":"2024-04-25T09:53:13.326681+0000","last_became_active":"2024-04-25T09:53:07.786097+0000","last_became_peered":"2024-04-25T09:53:07.786097+0000","last_unstale":"2024-04-25T09:53:13.326681+0000","last_undegraded":"2024-04-25T09:53:13.326681+0000","last_fullsized":"2024-04-25T09:53:13.326681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326102+0000","last_change":"2024-04-25T09:53:06.781647+0000","last_active":"2024-04-25T09:53:13.326102+0000","last_peered":"2024-04-25T09:53:13.326102+0000","last_clean":"2024-04-25T09:53:13.326102+0000","last_became_active":"2024-04-25T09:53:06.781360+0000","last_became_peered":"2024-04-25T09:53:06.781360+0000","last_unstale":"2024-04-25T09:53:13.326102+0000","last_undegraded":"2024-04-25T09:53:13.326102+0000","last_fullsized":"2024-04-25T09:53:13.326102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:44.406768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326043+0000","last_change":"2024-04-25T09:53:11.835499+0000","last_active":"2024-04-25T09:53:13.326043+0000","last_peered":"2024-04-25T09:53:13.326043+0000","last_clean":"2024-04-25T09:53:13.326043+0000","last_became_active":"2024-04-25T09:53:11.834613+0000","last_became_peered":"2024-04-25T09:53:11.834613+0000","last_unstale":"2024-04-25T09:53:13.326043+0000","last_undegraded":"2024-04-25T09:53:13.326043+0000","last_fullsized":"2024-04-25T09:53:13.326043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:16.884859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811569+0000","last_change":"2024-04-25T09:53:07.799270+0000","last_active":"2024-04-25T09:53:12.811569+0000","last_peered":"2024-04-25T09:53:12.811569+0000","last_clean":"2024-04-25T09:53:12.811569+0000","last_became_active":"2024-04-25T09:53:07.799005+0000","last_became_peered":"2024-04-25T09:53:07.799005+0000","last_unstale":"2024-04-25T09:53:12.811569+0000","last_undegraded":"2024-04-25T09:53:12.811569+0000","last_fullsized":"2024-04-25T09:53:12.811569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:47.687838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809643+0000","last_change":"2024-04-25T09:53:06.776508+0000","last_active":"2024-04-25T09:53:12.809643+0000","last_peered":"2024-04-25T09:53:12.809643+0000","last_clean":"2024-04-25T09:53:12.809643+0000","last_became_active":"2024-04-25T09:53:06.775938+0000","last_became_peered":"2024-04-25T09:53:06.775938+0000","last_unstale":"2024-04-25T09:53:12.809643+0000","last_undegraded":"2024-04-25T09:53:12.809643+0000","last_fullsized":"2024-04-25T09:53:12.809643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325709+0000","last_change":"2024-04-25T09:53:11.823495+0000","last_active":"2024-04-25T09:53:13.325709+0000","last_peered":"2024-04-25T09:53:13.325709+0000","last_clean":"2024-04-25T09:53:13.325709+0000","last_became_active":"2024-04-25T09:53:11.823070+0000","last_became_peered":"2024-04-25T09:53:11.823070+0000","last_unstale":"2024-04-25T09:53:13.325709+0000","last_undegraded":"2024-04-25T09:53:13.325709+0000","last_fullsized":"2024-04-25T09:53:13.325709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:58.601489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810162+0000","last_change":"2024-04-25T09:53:07.796560+0000","last_active":"2024-04-25T09:53:12.810162+0000","last_peered":"2024-04-25T09:53:12.810162+0000","last_clean":"2024-04-25T09:53:12.810162+0000","last_became_active":"2024-04-25T09:53:07.796288+0000","last_became_peered":"2024-04-25T09:53:07.796288+0000","last_unstale":"2024-04-25T09:53:12.810162+0000","last_undegraded":"2024-04-25T09:53:12.810162+0000","last_fullsized":"2024-04-25T09:53:12.810162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327440+0000","last_change":"2024-04-25T09:53:06.776524+0000","last_active":"2024-04-25T09:53:13.327440+0000","last_peered":"2024-04-25T09:53:13.327440+0000","last_clean":"2024-04-25T09:53:13.327440+0000","last_became_active":"2024-04-25T09:53:06.776156+0000","last_became_peered":"2024-04-25T09:53:06.776156+0000","last_unstale":"2024-04-25T09:53:13.327440+0000","last_undegraded":"2024-04-25T09:53:13.327440+0000","last_fullsized":"2024-04-25T09:53:13.327440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:23.222453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325855+0000","last_change":"2024-04-25T09:53:11.840103+0000","last_active":"2024-04-25T09:53:13.325855+0000","last_peered":"2024-04-25T09:53:13.325855+0000","last_clean":"2024-04-25T09:53:13.325855+0000","last_became_active":"2024-04-25T09:53:11.839706+0000","last_became_peered":"2024-04-25T09:53:11.839706+0000","last_unstale":"2024-04-25T09:53:13.325855+0000","last_undegraded":"2024-04-25T09:53:13.325855+0000","last_fullsized":"2024-04-25T09:53:13.325855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:35.127642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809957+0000","last_change":"2024-04-25T09:53:07.796678+0000","last_active":"2024-04-25T09:53:12.809957+0000","last_peered":"2024-04-25T09:53:12.809957+0000","last_clean":"2024-04-25T09:53:12.809957+0000","last_became_active":"2024-04-25T09:53:07.796401+0000","last_became_peered":"2024-04-25T09:53:07.796401+0000","last_unstale":"2024-04-25T09:53:12.809957+0000","last_undegraded":"2024-04-25T09:53:12.809957+0000","last_fullsized":"2024-04-25T09:53:12.809957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809151+0000","last_change":"2024-04-25T09:53:06.780411+0000","last_active":"2024-04-25T09:53:12.809151+0000","last_peered":"2024-04-25T09:53:12.809151+0000","last_clean":"2024-04-25T09:53:12.809151+0000","last_became_active":"2024-04-25T09:53:06.780094+0000","last_became_peered":"2024-04-25T09:53:06.780094+0000","last_unstale":"2024-04-25T09:53:12.809151+0000","last_undegraded":"2024-04-25T09:53:12.809151+0000","last_fullsized":"2024-04-25T09:53:12.809151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326778+0000","last_change":"2024-04-25T09:53:11.823203+0000","last_active":"2024-04-25T09:53:13.326778+0000","last_peered":"2024-04-25T09:53:13.326778+0000","last_clean":"2024-04-25T09:53:13.326778+0000","last_became_active":"2024-04-25T09:53:11.822736+0000","last_became_peered":"2024-04-25T09:53:11.822736+0000","last_unstale":"2024-04-25T09:53:13.326778+0000","last_undegraded":"2024-04-25T09:53:13.326778+0000","last_fullsized":"2024-04-25T09:53:13.326778+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:05.899921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809909+0000","last_change":"2024-04-25T09:53:07.797880+0000","last_active":"2024-04-25T09:53:12.809909+0000","last_peered":"2024-04-25T09:53:12.809909+0000","last_clean":"2024-04-25T09:53:12.809909+0000","last_became_active":"2024-04-25T09:53:07.797626+0000","last_became_peered":"2024-04-25T09:53:07.797626+0000","last_unstale":"2024-04-25T09:53:12.809909+0000","last_undegraded":"2024-04-25T09:53:12.809909+0000","last_fullsized":"2024-04-25T09:53:12.809909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810332+0000","last_change":"2024-04-25T09:53:06.781752+0000","last_active":"2024-04-25T09:53:12.810332+0000","last_peered":"2024-04-25T09:53:12.810332+0000","last_clean":"2024-04-25T09:53:12.810332+0000","last_became_active":"2024-04-25T09:53:06.781191+0000","last_became_peered":"2024-04-25T09:53:06.781191+0000","last_unstale":"2024-04-25T09:53:12.810332+0000","last_undegraded":"2024-04-25T09:53:12.810332+0000","last_fullsized":"2024-04-25T09:53:12.810332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326993+0000","last_change":"2024-04-25T09:53:11.839382+0000","last_active":"2024-04-25T09:53:13.326993+0000","last_peered":"2024-04-25T09:53:13.326993+0000","last_clean":"2024-04-25T09:53:13.326993+0000","last_became_active":"2024-04-25T09:53:11.839017+0000","last_became_peered":"2024-04-25T09:53:11.839017+0000","last_unstale":"2024-04-25T09:53:13.326993+0000","last_undegraded":"2024-04-25T09:53:13.326993+0000","last_fullsized":"2024-04-25T09:53:13.326993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:08.728070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810403+0000","last_change":"2024-04-25T09:53:07.797176+0000","last_active":"2024-04-25T09:53:12.810403+0000","last_peered":"2024-04-25T09:53:12.810403+0000","last_clean":"2024-04-25T09:53:12.810403+0000","last_became_active":"2024-04-25T09:53:07.796900+0000","last_became_peered":"2024-04-25T09:53:07.796900+0000","last_unstale":"2024-04-25T09:53:12.810403+0000","last_undegraded":"2024-04-25T09:53:12.810403+0000","last_fullsized":"2024-04-25T09:53:12.810403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326836+0000","last_change":"2024-04-25T09:53:06.783568+0000","last_active":"2024-04-25T09:53:13.326836+0000","last_peered":"2024-04-25T09:53:13.326836+0000","last_clean":"2024-04-25T09:53:13.326836+0000","last_became_active":"2024-04-25T09:53:06.783237+0000","last_became_peered":"2024-04-25T09:53:06.783237+0000","last_unstale":"2024-04-25T09:53:13.326836+0000","last_undegraded":"2024-04-25T09:53:13.326836+0000","last_fullsized":"2024-04-25T09:53:13.326836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326468+0000","last_change":"2024-04-25T09:53:11.835512+0000","last_active":"2024-04-25T09:53:13.326468+0000","last_peered":"2024-04-25T09:53:13.326468+0000","last_clean":"2024-04-25T09:53:13.326468+0000","last_became_active":"2024-04-25T09:53:11.835133+0000","last_became_peered":"2024-04-25T09:53:11.835133+0000","last_unstale":"2024-04-25T09:53:13.326468+0000","last_undegraded":"2024-04-25T09:53:13.326468+0000","last_fullsized":"2024-04-25T09:53:13.326468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:13.599619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326511+0000","last_change":"2024-04-25T09:53:07.788286+0000","last_active":"2024-04-25T09:53:13.326511+0000","last_peered":"2024-04-25T09:53:13.326511+0000","last_clean":"2024-04-25T09:53:13.326511+0000","last_became_active":"2024-04-25T09:53:07.787920+0000","last_became_peered":"2024-04-25T09:53:07.787920+0000","last_unstale":"2024-04-25T09:53:13.326511+0000","last_undegraded":"2024-04-25T09:53:13.326511+0000","last_fullsized":"2024-04-25T09:53:13.326511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325923+0000","last_change":"2024-04-25T09:53:06.772006+0000","last_active":"2024-04-25T09:53:13.325923+0000","last_peered":"2024-04-25T09:53:13.325923+0000","last_clean":"2024-04-25T09:53:13.325923+0000","last_became_active":"2024-04-25T09:53:06.771702+0000","last_became_peered":"2024-04-25T09:53:06.771702+0000","last_unstale":"2024-04-25T09:53:13.325923+0000","last_undegraded":"2024-04-25T09:53:13.325923+0000","last_fullsized":"2024-04-25T09:53:13.325923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:59.219341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327029+0000","last_change":"2024-04-25T09:53:11.821559+0000","last_active":"2024-04-25T09:53:13.327029+0000","last_peered":"2024-04-25T09:53:13.327029+0000","last_clean":"2024-04-25T09:53:13.327029+0000","last_became_active":"2024-04-25T09:53:11.821133+0000","last_became_peered":"2024-04-25T09:53:11.821133+0000","last_unstale":"2024-04-25T09:53:13.327029+0000","last_undegraded":"2024-04-25T09:53:13.327029+0000","last_fullsized":"2024-04-25T09:53:13.327029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:58.718292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810108+0000","last_change":"2024-04-25T09:53:07.797403+0000","last_active":"2024-04-25T09:53:12.810108+0000","last_peered":"2024-04-25T09:53:12.810108+0000","last_clean":"2024-04-25T09:53:12.810108+0000","last_became_active":"2024-04-25T09:53:07.796257+0000","last_became_peered":"2024-04-25T09:53:07.796257+0000","last_unstale":"2024-04-25T09:53:12.810108+0000","last_undegraded":"2024-04-25T09:53:12.810108+0000","last_fullsized":"2024-04-25T09:53:12.810108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:57.232809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:17.046098+0000","last_change":"2024-04-25T09:53:06.773036+0000","last_active":"2024-04-25T09:53:17.046098+0000","last_peered":"2024-04-25T09:53:17.046098+0000","last_clean":"2024-04-25T09:53:17.046098+0000","last_became_active":"2024-04-25T09:53:06.772729+0000","last_became_peered":"2024-04-25T09:53:06.772729+0000","last_unstale":"2024-04-25T09:53:17.046098+0000","last_undegraded":"2024-04-25T09:53:17.046098+0000","last_fullsized":"2024-04-25T09:53:17.046098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:20.750879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326453+0000","last_change":"2024-04-25T09:53:11.833689+0000","last_active":"2024-04-25T09:53:13.326453+0000","last_peered":"2024-04-25T09:53:13.326453+0000","last_clean":"2024-04-25T09:53:13.326453+0000","last_became_active":"2024-04-25T09:53:11.832588+0000","last_became_peered":"2024-04-25T09:53:11.832588+0000","last_unstale":"2024-04-25T09:53:13.326453+0000","last_undegraded":"2024-04-25T09:53:13.326453+0000","last_fullsized":"2024-04-25T09:53:13.326453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:05.899921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327010+0000","last_change":"2024-04-25T09:53:07.791857+0000","last_active":"2024-04-25T09:53:13.327010+0000","last_peered":"2024-04-25T09:53:13.327010+0000","last_clean":"2024-04-25T09:53:13.327010+0000","last_became_active":"2024-04-25T09:53:07.791606+0000","last_became_peered":"2024-04-25T09:53:07.791606+0000","last_unstale":"2024-04-25T09:53:13.327010+0000","last_undegraded":"2024-04-25T09:53:13.327010+0000","last_fullsized":"2024-04-25T09:53:13.327010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.768421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325995+0000","last_change":"2024-04-25T09:53:06.783616+0000","last_active":"2024-04-25T09:53:13.325995+0000","last_peered":"2024-04-25T09:53:13.325995+0000","last_clean":"2024-04-25T09:53:13.325995+0000","last_became_active":"2024-04-25T09:53:06.783302+0000","last_became_peered":"2024-04-25T09:53:06.783302+0000","last_unstale":"2024-04-25T09:53:13.325995+0000","last_undegraded":"2024-04-25T09:53:13.325995+0000","last_fullsized":"2024-04-25T09:53:13.325995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326861+0000","last_change":"2024-04-25T09:53:11.840956+0000","last_active":"2024-04-25T09:53:13.326861+0000","last_peered":"2024-04-25T09:53:13.326861+0000","last_clean":"2024-04-25T09:53:13.326861+0000","last_became_active":"2024-04-25T09:53:11.840316+0000","last_became_peered":"2024-04-25T09:53:11.840316+0000","last_unstale":"2024-04-25T09:53:13.326861+0000","last_undegraded":"2024-04-25T09:53:13.326861+0000","last_fullsized":"2024-04-25T09:53:13.326861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:01.272017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326067+0000","last_change":"2024-04-25T09:53:07.787945+0000","last_active":"2024-04-25T09:53:13.326067+0000","last_peered":"2024-04-25T09:53:13.326067+0000","last_clean":"2024-04-25T09:53:13.326067+0000","last_became_active":"2024-04-25T09:53:07.786113+0000","last_became_peered":"2024-04-25T09:53:07.786113+0000","last_unstale":"2024-04-25T09:53:13.326067+0000","last_undegraded":"2024-04-25T09:53:13.326067+0000","last_fullsized":"2024-04-25T09:53:13.326067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327211+0000","last_change":"2024-04-25T09:53:06.783604+0000","last_active":"2024-04-25T09:53:13.327211+0000","last_peered":"2024-04-25T09:53:13.327211+0000","last_clean":"2024-04-25T09:53:13.327211+0000","last_became_active":"2024-04-25T09:53:06.783341+0000","last_became_peered":"2024-04-25T09:53:06.783341+0000","last_unstale":"2024-04-25T09:53:13.327211+0000","last_undegraded":"2024-04-25T09:53:13.327211+0000","last_fullsized":"2024-04-25T09:53:13.327211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.032898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326141+0000","last_change":"2024-04-25T09:53:11.836053+0000","last_active":"2024-04-25T09:53:13.326141+0000","last_peered":"2024-04-25T09:53:13.326141+0000","last_clean":"2024-04-25T09:53:13.326141+0000","last_became_active":"2024-04-25T09:53:11.835318+0000","last_became_peered":"2024-04-25T09:53:11.835318+0000","last_unstale":"2024-04-25T09:53:13.326141+0000","last_undegraded":"2024-04-25T09:53:13.326141+0000","last_fullsized":"2024-04-25T09:53:13.326141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:49.717043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327141+0000","last_change":"2024-04-25T09:53:07.793149+0000","last_active":"2024-04-25T09:53:13.327141+0000","last_peered":"2024-04-25T09:53:13.327141+0000","last_clean":"2024-04-25T09:53:13.327141+0000","last_became_active":"2024-04-25T09:53:07.792896+0000","last_became_peered":"2024-04-25T09:53:07.792896+0000","last_unstale":"2024-04-25T09:53:13.327141+0000","last_undegraded":"2024-04-25T09:53:13.327141+0000","last_fullsized":"2024-04-25T09:53:13.327141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:55.375947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811223+0000","last_change":"2024-04-25T09:53:06.780916+0000","last_active":"2024-04-25T09:53:12.811223+0000","last_peered":"2024-04-25T09:53:12.811223+0000","last_clean":"2024-04-25T09:53:12.811223+0000","last_became_active":"2024-04-25T09:53:06.780631+0000","last_became_peered":"2024-04-25T09:53:06.780631+0000","last_unstale":"2024-04-25T09:53:12.811223+0000","last_undegraded":"2024-04-25T09:53:12.811223+0000","last_fullsized":"2024-04-25T09:53:12.811223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.906271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325834+0000","last_change":"2024-04-25T09:53:11.821088+0000","last_active":"2024-04-25T09:53:13.325834+0000","last_peered":"2024-04-25T09:53:13.325834+0000","last_clean":"2024-04-25T09:53:13.325834+0000","last_became_active":"2024-04-25T09:53:11.820662+0000","last_became_peered":"2024-04-25T09:53:11.820662+0000","last_unstale":"2024-04-25T09:53:13.325834+0000","last_undegraded":"2024-04-25T09:53:13.325834+0000","last_fullsized":"2024-04-25T09:53:13.325834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:58.601489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326720+0000","last_change":"2024-04-25T09:53:11.838966+0000","last_active":"2024-04-25T09:53:13.326720+0000","last_peered":"2024-04-25T09:53:13.326720+0000","last_clean":"2024-04-25T09:53:13.326720+0000","last_became_active":"2024-04-25T09:53:11.838542+0000","last_became_peered":"2024-04-25T09:53:11.838542+0000","last_unstale":"2024-04-25T09:53:13.326720+0000","last_undegraded":"2024-04-25T09:53:13.326720+0000","last_fullsized":"2024-04-25T09:53:13.326720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:51.727046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325110+0000","last_change":"2024-04-25T09:53:07.784870+0000","last_active":"2024-04-25T09:53:13.325110+0000","last_peered":"2024-04-25T09:53:13.325110+0000","last_clean":"2024-04-25T09:53:13.325110+0000","last_became_active":"2024-04-25T09:53:07.784587+0000","last_became_peered":"2024-04-25T09:53:07.784587+0000","last_unstale":"2024-04-25T09:53:13.325110+0000","last_undegraded":"2024-04-25T09:53:13.325110+0000","last_fullsized":"2024-04-25T09:53:13.325110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325791+0000","last_change":"2024-04-25T09:53:06.782572+0000","last_active":"2024-04-25T09:53:13.325791+0000","last_peered":"2024-04-25T09:53:13.325791+0000","last_clean":"2024-04-25T09:53:13.325791+0000","last_became_active":"2024-04-25T09:53:06.781792+0000","last_became_peered":"2024-04-25T09:53:06.781792+0000","last_unstale":"2024-04-25T09:53:13.325791+0000","last_undegraded":"2024-04-25T09:53:13.325791+0000","last_fullsized":"2024-04-25T09:53:13.325791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:58.454337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811107+0000","last_change":"2024-04-25T09:53:11.838770+0000","last_active":"2024-04-25T09:53:12.811107+0000","last_peered":"2024-04-25T09:53:12.811107+0000","last_clean":"2024-04-25T09:53:12.811107+0000","last_became_active":"2024-04-25T09:53:11.838328+0000","last_became_peered":"2024-04-25T09:53:11.838328+0000","last_unstale":"2024-04-25T09:53:12.811107+0000","last_undegraded":"2024-04-25T09:53:12.811107+0000","last_fullsized":"2024-04-25T09:53:12.811107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.260455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326187+0000","last_change":"2024-04-25T09:53:07.791725+0000","last_active":"2024-04-25T09:53:13.326187+0000","last_peered":"2024-04-25T09:53:13.326187+0000","last_clean":"2024-04-25T09:53:13.326187+0000","last_became_active":"2024-04-25T09:53:07.791386+0000","last_became_peered":"2024-04-25T09:53:07.791386+0000","last_unstale":"2024-04-25T09:53:13.326187+0000","last_undegraded":"2024-04-25T09:53:13.326187+0000","last_fullsized":"2024-04-25T09:53:13.326187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326069+0000","last_change":"2024-04-25T09:53:06.775205+0000","last_active":"2024-04-25T09:53:13.326069+0000","last_peered":"2024-04-25T09:53:13.326069+0000","last_clean":"2024-04-25T09:53:13.326069+0000","last_became_active":"2024-04-25T09:53:06.774911+0000","last_became_peered":"2024-04-25T09:53:06.774911+0000","last_unstale":"2024-04-25T09:53:13.326069+0000","last_undegraded":"2024-04-25T09:53:13.326069+0000","last_fullsized":"2024-04-25T09:53:13.326069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.898659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810239+0000","last_change":"2024-04-25T09:53:06.781528+0000","last_active":"2024-04-25T09:53:12.810239+0000","last_peered":"2024-04-25T09:53:12.810239+0000","last_clean":"2024-04-25T09:53:12.810239+0000","last_became_active":"2024-04-25T09:53:06.781248+0000","last_became_peered":"2024-04-25T09:53:06.781248+0000","last_unstale":"2024-04-25T09:53:12.810239+0000","last_undegraded":"2024-04-25T09:53:12.810239+0000","last_fullsized":"2024-04-25T09:53:12.810239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:35.589225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325829+0000","last_change":"2024-04-25T09:53:07.783095+0000","last_active":"2024-04-25T09:53:13.325829+0000","last_peered":"2024-04-25T09:53:13.325829+0000","last_clean":"2024-04-25T09:53:13.325829+0000","last_became_active":"2024-04-25T09:53:07.782578+0000","last_became_peered":"2024-04-25T09:53:07.782578+0000","last_unstale":"2024-04-25T09:53:13.325829+0000","last_undegraded":"2024-04-25T09:53:13.325829+0000","last_fullsized":"2024-04-25T09:53:13.325829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809581+0000","last_change":"2024-04-25T09:53:11.834200+0000","last_active":"2024-04-25T09:53:12.809581+0000","last_peered":"2024-04-25T09:53:12.809581+0000","last_clean":"2024-04-25T09:53:12.809581+0000","last_became_active":"2024-04-25T09:53:11.833774+0000","last_became_peered":"2024-04-25T09:53:11.833774+0000","last_unstale":"2024-04-25T09:53:12.809581+0000","last_undegraded":"2024-04-25T09:53:12.809581+0000","last_fullsized":"2024-04-25T09:53:12.809581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:42.139275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325379+0000","last_change":"2024-04-25T09:53:06.777867+0000","last_active":"2024-04-25T09:53:13.325379+0000","last_peered":"2024-04-25T09:53:13.325379+0000","last_clean":"2024-04-25T09:53:13.325379+0000","last_became_active":"2024-04-25T09:53:06.777566+0000","last_became_peered":"2024-04-25T09:53:06.777566+0000","last_unstale":"2024-04-25T09:53:13.325379+0000","last_undegraded":"2024-04-25T09:53:13.325379+0000","last_fullsized":"2024-04-25T09:53:13.325379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:05.752749+0000","last_clean_scrub_stamp":"2024-04-25T09:53:05.752749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:52.306054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326173+0000","last_change":"2024-04-25T09:53:07.781653+0000","last_active":"2024-04-25T09:53:13.326173+0000","last_peered":"2024-04-25T09:53:13.326173+0000","last_clean":"2024-04-25T09:53:13.326173+0000","last_became_active":"2024-04-25T09:53:07.781274+0000","last_became_peered":"2024-04-25T09:53:07.781274+0000","last_unstale":"2024-04-25T09:53:13.326173+0000","last_undegraded":"2024-04-25T09:53:13.326173+0000","last_fullsized":"2024-04-25T09:53:13.326173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:02.037522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326829+0000","last_change":"2024-04-25T09:53:11.834554+0000","last_active":"2024-04-25T09:53:13.326829+0000","last_peered":"2024-04-25T09:53:13.326829+0000","last_clean":"2024-04-25T09:53:13.326829+0000","last_became_active":"2024-04-25T09:53:11.834143+0000","last_became_peered":"2024-04-25T09:53:11.834143+0000","last_unstale":"2024-04-25T09:53:13.326829+0000","last_undegraded":"2024-04-25T09:53:13.326829+0000","last_fullsized":"2024-04-25T09:53:13.326829+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:10.796582+0000","last_clean_scrub_stamp":"2024-04-25T09:53:10.796582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:37.372311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809684+0000","last_change":"2024-04-25T09:53:07.798578+0000","last_active":"2024-04-25T09:53:12.809684+0000","last_peered":"2024-04-25T09:53:12.809684+0000","last_clean":"2024-04-25T09:53:12.809684+0000","last_became_active":"2024-04-25T09:53:07.798310+0000","last_became_peered":"2024-04-25T09:53:07.798310+0000","last_unstale":"2024-04-25T09:53:12.809684+0000","last_undegraded":"2024-04-25T09:53:12.809684+0000","last_fullsized":"2024-04-25T09:53:12.809684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809471+0000","last_change":"2024-04-25T09:53:07.780646+0000","last_active":"2024-04-25T09:53:12.809471+0000","last_peered":"2024-04-25T09:53:12.809471+0000","last_clean":"2024-04-25T09:53:12.809471+0000","last_became_active":"2024-04-25T09:53:07.780321+0000","last_became_peered":"2024-04-25T09:53:07.780321+0000","last_unstale":"2024-04-25T09:53:12.809471+0000","last_undegraded":"2024-04-25T09:53:12.809471+0000","last_fullsized":"2024-04-25T09:53:12.809471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:44.576758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326872+0000","last_change":"2024-04-25T09:53:07.787491+0000","last_active":"2024-04-25T09:53:13.326872+0000","last_peered":"2024-04-25T09:53:13.326872+0000","last_clean":"2024-04-25T09:53:13.326872+0000","last_became_active":"2024-04-25T09:53:07.787214+0000","last_became_peered":"2024-04-25T09:53:07.787214+0000","last_unstale":"2024-04-25T09:53:13.326872+0000","last_undegraded":"2024-04-25T09:53:13.326872+0000","last_fullsized":"2024-04-25T09:53:13.326872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:35.714783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325728+0000","last_change":"2024-04-25T09:53:07.789805+0000","last_active":"2024-04-25T09:53:13.325728+0000","last_peered":"2024-04-25T09:53:13.325728+0000","last_clean":"2024-04-25T09:53:13.325728+0000","last_became_active":"2024-04-25T09:53:07.788987+0000","last_became_peered":"2024-04-25T09:53:07.788987+0000","last_unstale":"2024-04-25T09:53:13.325728+0000","last_undegraded":"2024-04-25T09:53:13.325728+0000","last_fullsized":"2024-04-25T09:53:13.325728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326515+0000","last_change":"2024-04-25T09:53:07.792754+0000","last_active":"2024-04-25T09:53:13.326515+0000","last_peered":"2024-04-25T09:53:13.326515+0000","last_clean":"2024-04-25T09:53:13.326515+0000","last_became_active":"2024-04-25T09:53:07.792223+0000","last_became_peered":"2024-04-25T09:53:07.792223+0000","last_unstale":"2024-04-25T09:53:13.326515+0000","last_undegraded":"2024-04-25T09:53:13.326515+0000","last_fullsized":"2024-04-25T09:53:13.326515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:24.079245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325889+0000","last_change":"2024-04-25T09:53:07.783936+0000","last_active":"2024-04-25T09:53:13.325889+0000","last_peered":"2024-04-25T09:53:13.325889+0000","last_clean":"2024-04-25T09:53:13.325889+0000","last_became_active":"2024-04-25T09:53:07.783646+0000","last_became_peered":"2024-04-25T09:53:07.783646+0000","last_unstale":"2024-04-25T09:53:13.325889+0000","last_undegraded":"2024-04-25T09:53:13.325889+0000","last_fullsized":"2024-04-25T09:53:13.325889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810830+0000","last_change":"2024-04-25T09:53:07.789428+0000","last_active":"2024-04-25T09:53:12.810830+0000","last_peered":"2024-04-25T09:53:12.810830+0000","last_clean":"2024-04-25T09:53:12.810830+0000","last_became_active":"2024-04-25T09:53:07.789086+0000","last_became_peered":"2024-04-25T09:53:07.789086+0000","last_unstale":"2024-04-25T09:53:12.810830+0000","last_undegraded":"2024-04-25T09:53:12.810830+0000","last_fullsized":"2024-04-25T09:53:12.810830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327059+0000","last_change":"2024-04-25T09:53:07.785822+0000","last_active":"2024-04-25T09:53:13.327059+0000","last_peered":"2024-04-25T09:53:13.327059+0000","last_clean":"2024-04-25T09:53:13.327059+0000","last_became_active":"2024-04-25T09:53:07.785535+0000","last_became_peered":"2024-04-25T09:53:07.785535+0000","last_unstale":"2024-04-25T09:53:13.327059+0000","last_undegraded":"2024-04-25T09:53:13.327059+0000","last_fullsized":"2024-04-25T09:53:13.327059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325833+0000","last_change":"2024-04-25T09:53:07.787296+0000","last_active":"2024-04-25T09:53:13.325833+0000","last_peered":"2024-04-25T09:53:13.325833+0000","last_clean":"2024-04-25T09:53:13.325833+0000","last_became_active":"2024-04-25T09:53:07.787011+0000","last_became_peered":"2024-04-25T09:53:07.787011+0000","last_unstale":"2024-04-25T09:53:13.325833+0000","last_undegraded":"2024-04-25T09:53:13.325833+0000","last_fullsized":"2024-04-25T09:53:13.325833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325938+0000","last_change":"2024-04-25T09:53:07.785405+0000","last_active":"2024-04-25T09:53:13.325938+0000","last_peered":"2024-04-25T09:53:13.325938+0000","last_clean":"2024-04-25T09:53:13.325938+0000","last_became_active":"2024-04-25T09:53:07.785085+0000","last_became_peered":"2024-04-25T09:53:07.785085+0000","last_unstale":"2024-04-25T09:53:13.325938+0000","last_undegraded":"2024-04-25T09:53:13.325938+0000","last_fullsized":"2024-04-25T09:53:13.325938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326705+0000","last_change":"2024-04-25T09:53:07.792913+0000","last_active":"2024-04-25T09:53:13.326705+0000","last_peered":"2024-04-25T09:53:13.326705+0000","last_clean":"2024-04-25T09:53:13.326705+0000","last_became_active":"2024-04-25T09:53:07.792666+0000","last_became_peered":"2024-04-25T09:53:07.792666+0000","last_unstale":"2024-04-25T09:53:13.326705+0000","last_undegraded":"2024-04-25T09:53:13.326705+0000","last_fullsized":"2024-04-25T09:53:13.326705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325521+0000","last_change":"2024-04-25T09:53:07.789785+0000","last_active":"2024-04-25T09:53:13.325521+0000","last_peered":"2024-04-25T09:53:13.325521+0000","last_clean":"2024-04-25T09:53:13.325521+0000","last_became_active":"2024-04-25T09:53:07.789109+0000","last_became_peered":"2024-04-25T09:53:07.789109+0000","last_unstale":"2024-04-25T09:53:13.325521+0000","last_undegraded":"2024-04-25T09:53:13.325521+0000","last_fullsized":"2024-04-25T09:53:13.325521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:28.708261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811599+0000","last_change":"2024-04-25T09:53:07.800839+0000","last_active":"2024-04-25T09:53:12.811599+0000","last_peered":"2024-04-25T09:53:12.811599+0000","last_clean":"2024-04-25T09:53:12.811599+0000","last_became_active":"2024-04-25T09:53:07.800570+0000","last_became_peered":"2024-04-25T09:53:07.800570+0000","last_unstale":"2024-04-25T09:53:12.811599+0000","last_undegraded":"2024-04-25T09:53:12.811599+0000","last_fullsized":"2024-04-25T09:53:12.811599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:59.977985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810497+0000","last_change":"2024-04-25T09:53:07.793203+0000","last_active":"2024-04-25T09:53:12.810497+0000","last_peered":"2024-04-25T09:53:12.810497+0000","last_clean":"2024-04-25T09:53:12.810497+0000","last_became_active":"2024-04-25T09:53:07.792940+0000","last_became_peered":"2024-04-25T09:53:07.792940+0000","last_unstale":"2024-04-25T09:53:12.810497+0000","last_undegraded":"2024-04-25T09:53:12.810497+0000","last_fullsized":"2024-04-25T09:53:12.810497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.221246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809630+0000","last_change":"2024-04-25T09:53:07.789420+0000","last_active":"2024-04-25T09:53:12.809630+0000","last_peered":"2024-04-25T09:53:12.809630+0000","last_clean":"2024-04-25T09:53:12.809630+0000","last_became_active":"2024-04-25T09:53:07.789112+0000","last_became_peered":"2024-04-25T09:53:07.789112+0000","last_unstale":"2024-04-25T09:53:12.809630+0000","last_undegraded":"2024-04-25T09:53:12.809630+0000","last_fullsized":"2024-04-25T09:53:12.809630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.768421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.327332+0000","last_change":"2024-04-25T09:53:07.794475+0000","last_active":"2024-04-25T09:53:13.327332+0000","last_peered":"2024-04-25T09:53:13.327332+0000","last_clean":"2024-04-25T09:53:13.327332+0000","last_became_active":"2024-04-25T09:53:07.794225+0000","last_became_peered":"2024-04-25T09:53:07.794225+0000","last_unstale":"2024-04-25T09:53:13.327332+0000","last_undegraded":"2024-04-25T09:53:13.327332+0000","last_fullsized":"2024-04-25T09:53:13.327332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:47.687838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325668+0000","last_change":"2024-04-25T09:53:07.798926+0000","last_active":"2024-04-25T09:53:13.325668+0000","last_peered":"2024-04-25T09:53:13.325668+0000","last_clean":"2024-04-25T09:53:13.325668+0000","last_became_active":"2024-04-25T09:53:07.798446+0000","last_became_peered":"2024-04-25T09:53:07.798446+0000","last_unstale":"2024-04-25T09:53:13.325668+0000","last_undegraded":"2024-04-25T09:53:13.325668+0000","last_fullsized":"2024-04-25T09:53:13.325668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325115+0000","last_change":"2024-04-25T09:53:07.784610+0000","last_active":"2024-04-25T09:53:13.325115+0000","last_peered":"2024-04-25T09:53:13.325115+0000","last_clean":"2024-04-25T09:53:13.325115+0000","last_became_active":"2024-04-25T09:53:07.784323+0000","last_became_peered":"2024-04-25T09:53:07.784323+0000","last_unstale":"2024-04-25T09:53:13.325115+0000","last_undegraded":"2024-04-25T09:53:13.325115+0000","last_fullsized":"2024-04-25T09:53:13.325115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:53.733676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326504+0000","last_change":"2024-04-25T09:53:07.781472+0000","last_active":"2024-04-25T09:53:13.326504+0000","last_peered":"2024-04-25T09:53:13.326504+0000","last_clean":"2024-04-25T09:53:13.326504+0000","last_became_active":"2024-04-25T09:53:07.781156+0000","last_became_peered":"2024-04-25T09:53:07.781156+0000","last_unstale":"2024-04-25T09:53:13.326504+0000","last_undegraded":"2024-04-25T09:53:13.326504+0000","last_fullsized":"2024-04-25T09:53:13.326504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:49.469537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325209+0000","last_change":"2024-04-25T09:53:07.790342+0000","last_active":"2024-04-25T09:53:13.325209+0000","last_peered":"2024-04-25T09:53:13.325209+0000","last_clean":"2024-04-25T09:53:13.325209+0000","last_became_active":"2024-04-25T09:53:07.790003+0000","last_became_peered":"2024-04-25T09:53:07.790003+0000","last_unstale":"2024-04-25T09:53:13.325209+0000","last_undegraded":"2024-04-25T09:53:13.325209+0000","last_fullsized":"2024-04-25T09:53:13.325209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326892+0000","last_change":"2024-04-25T09:53:07.791198+0000","last_active":"2024-04-25T09:53:13.326892+0000","last_peered":"2024-04-25T09:53:13.326892+0000","last_clean":"2024-04-25T09:53:13.326892+0000","last_became_active":"2024-04-25T09:53:07.790896+0000","last_became_peered":"2024-04-25T09:53:07.790896+0000","last_unstale":"2024-04-25T09:53:13.326892+0000","last_undegraded":"2024-04-25T09:53:13.326892+0000","last_fullsized":"2024-04-25T09:53:13.326892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:45.411393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810894+0000","last_change":"2024-04-25T09:53:07.790314+0000","last_active":"2024-04-25T09:53:12.810894+0000","last_peered":"2024-04-25T09:53:12.810894+0000","last_clean":"2024-04-25T09:53:12.810894+0000","last_became_active":"2024-04-25T09:53:07.789999+0000","last_became_peered":"2024-04-25T09:53:07.789999+0000","last_unstale":"2024-04-25T09:53:12.810894+0000","last_undegraded":"2024-04-25T09:53:12.810894+0000","last_fullsized":"2024-04-25T09:53:12.810894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.999929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810741+0000","last_change":"2024-04-25T09:53:07.793396+0000","last_active":"2024-04-25T09:53:12.810741+0000","last_peered":"2024-04-25T09:53:12.810741+0000","last_clean":"2024-04-25T09:53:12.810741+0000","last_became_active":"2024-04-25T09:53:07.792132+0000","last_became_peered":"2024-04-25T09:53:07.792132+0000","last_unstale":"2024-04-25T09:53:12.810741+0000","last_undegraded":"2024-04-25T09:53:12.810741+0000","last_fullsized":"2024-04-25T09:53:12.810741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811509+0000","last_change":"2024-04-25T09:53:07.800101+0000","last_active":"2024-04-25T09:53:12.811509+0000","last_peered":"2024-04-25T09:53:12.811509+0000","last_clean":"2024-04-25T09:53:12.811509+0000","last_became_active":"2024-04-25T09:53:07.799794+0000","last_became_peered":"2024-04-25T09:53:07.799794+0000","last_unstale":"2024-04-25T09:53:12.811509+0000","last_undegraded":"2024-04-25T09:53:12.811509+0000","last_fullsized":"2024-04-25T09:53:12.811509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:12.845651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325964+0000","last_change":"2024-04-25T09:53:07.793404+0000","last_active":"2024-04-25T09:53:13.325964+0000","last_peered":"2024-04-25T09:53:13.325964+0000","last_clean":"2024-04-25T09:53:13.325964+0000","last_became_active":"2024-04-25T09:53:07.793140+0000","last_became_peered":"2024-04-25T09:53:07.793140+0000","last_unstale":"2024-04-25T09:53:13.325964+0000","last_undegraded":"2024-04-25T09:53:13.325964+0000","last_fullsized":"2024-04-25T09:53:13.325964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:50.362943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.809861+0000","last_change":"2024-04-25T09:53:07.797288+0000","last_active":"2024-04-25T09:53:12.809861+0000","last_peered":"2024-04-25T09:53:12.809861+0000","last_clean":"2024-04-25T09:53:12.809861+0000","last_became_active":"2024-04-25T09:53:07.797017+0000","last_became_peered":"2024-04-25T09:53:07.797017+0000","last_unstale":"2024-04-25T09:53:12.809861+0000","last_undegraded":"2024-04-25T09:53:12.809861+0000","last_fullsized":"2024-04-25T09:53:12.809861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:21.798476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326593+0000","last_change":"2024-04-25T09:53:07.800134+0000","last_active":"2024-04-25T09:53:13.326593+0000","last_peered":"2024-04-25T09:53:13.326593+0000","last_clean":"2024-04-25T09:53:13.326593+0000","last_became_active":"2024-04-25T09:53:07.799817+0000","last_became_peered":"2024-04-25T09:53:07.799817+0000","last_unstale":"2024-04-25T09:53:13.326593+0000","last_undegraded":"2024-04-25T09:53:13.326593+0000","last_fullsized":"2024-04-25T09:53:13.326593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.133547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.810285+0000","last_change":"2024-04-25T09:53:07.792436+0000","last_active":"2024-04-25T09:53:12.810285+0000","last_peered":"2024-04-25T09:53:12.810285+0000","last_clean":"2024-04-25T09:53:12.810285+0000","last_became_active":"2024-04-25T09:53:07.792150+0000","last_became_peered":"2024-04-25T09:53:07.792150+0000","last_unstale":"2024-04-25T09:53:12.810285+0000","last_undegraded":"2024-04-25T09:53:12.810285+0000","last_fullsized":"2024-04-25T09:53:12.810285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.326560+0000","last_change":"2024-04-25T09:53:07.788302+0000","last_active":"2024-04-25T09:53:13.326560+0000","last_peered":"2024-04-25T09:53:13.326560+0000","last_clean":"2024-04-25T09:53:13.326560+0000","last_became_active":"2024-04-25T09:53:07.788033+0000","last_became_peered":"2024-04-25T09:53:07.788033+0000","last_unstale":"2024-04-25T09:53:13.326560+0000","last_undegraded":"2024-04-25T09:53:13.326560+0000","last_fullsized":"2024-04-25T09:53:13.326560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:55.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325682+0000","last_change":"2024-04-25T09:53:07.776111+0000","last_active":"2024-04-25T09:53:13.325682+0000","last_peered":"2024-04-25T09:53:13.325682+0000","last_clean":"2024-04-25T09:53:13.325682+0000","last_became_active":"2024-04-25T09:53:07.775779+0000","last_became_peered":"2024-04-25T09:53:07.775779+0000","last_unstale":"2024-04-25T09:53:13.325682+0000","last_undegraded":"2024-04-25T09:53:13.325682+0000","last_fullsized":"2024-04-25T09:53:13.325682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:31.088434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:13.325773+0000","last_change":"2024-04-25T09:53:07.782293+0000","last_active":"2024-04-25T09:53:13.325773+0000","last_peered":"2024-04-25T09:53:13.325773+0000","last_clean":"2024-04-25T09:53:13.325773+0000","last_became_active":"2024-04-25T09:53:07.781957+0000","last_became_peered":"2024-04-25T09:53:07.781957+0000","last_unstale":"2024-04-25T09:53:13.325773+0000","last_undegraded":"2024-04-25T09:53:13.325773+0000","last_fullsized":"2024-04-25T09:53:13.325773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.237842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:12.811414+0000","last_change":"2024-04-25T09:53:07.791822+0000","last_active":"2024-04-25T09:53:12.811414+0000","last_peered":"2024-04-25T09:53:12.811414+0000","last_clean":"2024-04-25T09:53:12.811414+0000","last_became_active":"2024-04-25T09:53:07.791566+0000","last_became_peered":"2024-04-25T09:53:07.791566+0000","last_unstale":"2024-04-25T09:53:12.811414+0000","last_undegraded":"2024-04-25T09:53:12.811414+0000","last_fullsized":"2024-04-25T09:53:12.811414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:06.757374+0000","last_clean_scrub_stamp":"2024-04-25T09:53:06.757374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:35.714783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:53:28.234 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T09:53:28.234 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T09:53:28.234 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T09:53:28.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T09:53:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.365+0000 7fa4578cb640 1 Processor -- start 2024-04-25T09:53:28.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.366+0000 7fa4578cb640 1 -- start start 2024-04-25T09:53:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.366+0000 7fa4578cb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa4500b4890 0x7fa4500aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.366+0000 7fa4578cb640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa45015ad90 con 0x7fa45006ee40 2024-04-25T09:53:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.366+0000 7fa4578cb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa45006f370 con 0x7fa45015a9b0 2024-04-25T09:53:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.366+0000 7fa4578cb640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45006f650 con 0x7fa4500b4890 2024-04-25T09:53:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.367+0000 7fa454e66640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa45006ee40 0x7fa4500b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47508/0 (socket says 172.21.15.19:47508) 2024-04-25T09:53:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.367+0000 7fa454e66640 1 -- 172.21.15.19:0/2949372689 learned_addr learned my addr 172.21.15.19:0/2949372689 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.367+0000 7fa455e68640 1 --2- 172.21.15.19:0/2949372689 >> v2:172.21.15.112:3300/0 conn(0x7fa4500b4890 0x7fa4500aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.367+0000 7fa447fff640 1 -- 172.21.15.19:0/2949372689 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775476250 0 0) 0x7fa45015ad90 con 0x7fa45006ee40 2024-04-25T09:53:28.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.367+0000 7fa447fff640 1 -- 172.21.15.19:0/2949372689 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa43c003520 con 0x7fa45006ee40 2024-04-25T09:53:28.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.367+0000 7fa455e68640 1 -- 172.21.15.19:0/2949372689 >> v1:172.21.15.112:6789/0 conn(0x7fa45015a9b0 legacy=0x7fa4500b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.367+0000 7fa455e68640 1 -- 172.21.15.19:0/2949372689 >> v1:172.21.15.19:6789/0 conn(0x7fa45006ee40 legacy=0x7fa4500b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.367+0000 7fa455e68640 1 -- 172.21.15.19:0/2949372689 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45006f930 con 0x7fa4500b4890 2024-04-25T09:53:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.368+0000 7fa455e68640 1 --2- 172.21.15.19:0/2949372689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4500b4890 0x7fa4500aedb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa448009d90 tx=0x7fa44802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51511c1482d1f56f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.368+0000 7fa447fff640 1 -- 172.21.15.19:0/2949372689 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448007740 con 0x7fa4500b4890 2024-04-25T09:53:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.368+0000 7fa447fff640 1 -- 172.21.15.19:0/2949372689 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4480078e0 con 0x7fa4500b4890 2024-04-25T09:53:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.368+0000 7fa447fff640 1 -- 172.21.15.19:0/2949372689 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448007c90 con 0x7fa4500b4890 2024-04-25T09:53:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.368+0000 7fa4578cb640 1 -- 172.21.15.19:0/2949372689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4500b4890 msgr2=0x7fa4500aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.368+0000 7fa4578cb640 1 --2- 172.21.15.19:0/2949372689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4500b4890 0x7fa4500aedb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa448009d90 tx=0x7fa44802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.368+0000 7fa4578cb640 1 -- 172.21.15.19:0/2949372689 shutdown_connections 2024-04-25T09:53:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.368+0000 7fa4578cb640 1 --2- 172.21.15.19:0/2949372689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4500b4890 0x7fa4500aedb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.368+0000 7fa4578cb640 1 -- 172.21.15.19:0/2949372689 >> 172.21.15.19:0/2949372689 conn(0x7fa4500b5af0 msgr2=0x7fa4500b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.369+0000 7fa4578cb640 1 -- 172.21.15.19:0/2949372689 shutdown_connections 2024-04-25T09:53:28.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.369+0000 7fa4578cb640 1 -- 172.21.15.19:0/2949372689 wait complete. 2024-04-25T09:53:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.370+0000 7fa4578cb640 1 Processor -- start 2024-04-25T09:53:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.370+0000 7fa4578cb640 1 -- start start 2024-04-25T09:53:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.370+0000 7fa4578cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa45006ee40 0x7fa45015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.370+0000 7fa4578cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4500b4890 0x7fa450155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.370+0000 7fa4578cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45015a9b0 0x7fa4500d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.370+0000 7fa4578cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4501654e0 con 0x7fa45006ee40 2024-04-25T09:53:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.370+0000 7fa4578cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45006fef0 con 0x7fa45015a9b0 2024-04-25T09:53:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.370+0000 7fa4578cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa450165660 con 0x7fa4500b4890 2024-04-25T09:53:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa455667640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa45006ee40 0x7fa45015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa455667640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa45006ee40 0x7fa45015d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37228/0 (socket says 172.21.15.19:37228) 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa455667640 1 -- 172.21.15.19:0/1360254775 learned_addr learned my addr 172.21.15.19:0/1360254775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa454e66640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4500b4890 0x7fa450155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa455e68640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45015a9b0 0x7fa4500d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa455667640 1 -- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4500b4890 msgr2=0x7fa450155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa455667640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4500b4890 0x7fa450155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa455667640 1 -- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45015a9b0 msgr2=0x7fa4500d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa455667640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45015a9b0 0x7fa4500d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa455667640 1 -- 172.21.15.19:0/1360254775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa450141c00 con 0x7fa45006ee40 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa455667640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa45006ee40 0x7fa45015d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c3c0 tx=0x7fa44c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa445ffb640 1 -- 172.21.15.19:0/1360254775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c0052d0 con 0x7fa45006ee40 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa445ffb640 1 -- 172.21.15.19:0/1360254775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c005490 con 0x7fa45006ee40 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa4578cb640 1 -- 172.21.15.19:0/1360254775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa450141ee0 con 0x7fa45006ee40 2024-04-25T09:53:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.371+0000 7fa445ffb640 1 -- 172.21.15.19:0/1360254775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c007260 con 0x7fa45006ee40 2024-04-25T09:53:28.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.372+0000 7fa4578cb640 1 -- 172.21.15.19:0/1360254775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4501424a0 con 0x7fa45006ee40 2024-04-25T09:53:28.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.372+0000 7fa4578cb640 1 -- 172.21.15.19:0/1360254775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45006cc80 con 0x7fa45006ee40 2024-04-25T09:53:28.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.373+0000 7fa445ffb640 1 -- 172.21.15.19:0/1360254775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa44c007400 con 0x7fa45006ee40 2024-04-25T09:53:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.373+0000 7fa445ffb640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa43406b4d0 0x7fa43406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.373+0000 7fa445ffb640 1 -- 172.21.15.19:0/1360254775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa44c0899f0 con 0x7fa45006ee40 2024-04-25T09:53:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.374+0000 7fa454e66640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa43406b4d0 0x7fa43406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.374+0000 7fa454e66640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa43406b4d0 0x7fa43406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa45015a040 tx=0x7fa440009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.378+0000 7fa445ffb640 1 -- 172.21.15.19:0/1360254775 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa44c058530 con 0x7fa45006ee40 2024-04-25T09:53:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.595+0000 7fa4578cb640 1 -- 172.21.15.19:0/1360254775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa4500b1e70 con 0x7fa45006ee40 2024-04-25T09:53:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.595+0000 7fa445ffb640 1 -- 172.21.15.19:0/1360254775 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa44c057e30 con 0x7fa45006ee40 2024-04-25T09:53:28.597 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:28.598 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T09:53:28.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 -- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa43406b4d0 msgr2=0x7fa43406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa43406b4d0 0x7fa43406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa45015a040 tx=0x7fa440009040 comp rx=0 tx=0).stop 2024-04-25T09:53:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 -- 172.21.15.19:0/1360254775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa45006ee40 msgr2=0x7fa45015d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa45006ee40 0x7fa45015d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c3c0 tx=0x7fa44c00c880 comp rx=0 tx=0).stop 2024-04-25T09:53:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 -- 172.21.15.19:0/1360254775 shutdown_connections 2024-04-25T09:53:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa43406b4d0 0x7fa43406d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45015a9b0 0x7fa4500d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4500b4890 0x7fa450155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 --2- 172.21.15.19:0/1360254775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa45006ee40 0x7fa45015d3c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 -- 172.21.15.19:0/1360254775 >> 172.21.15.19:0/1360254775 conn(0x7fa4500b5af0 msgr2=0x7fa4500add60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 -- 172.21.15.19:0/1360254775 shutdown_connections 2024-04-25T09:53:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.597+0000 7fa4578cb640 1 -- 172.21.15.19:0/1360254775 wait complete. 2024-04-25T09:53:28.610 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T09:53:28.610 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:28.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.708+0000 7f912dbb0640 1 Processor -- start 2024-04-25T09:53:28.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.708+0000 7f912dbb0640 1 -- start start 2024-04-25T09:53:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.709+0000 7f912dbb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f912815a790 0x7f91280d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.709+0000 7f912dbb0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280b3710 con 0x7f91280b4eb0 2024-04-25T09:53:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.709+0000 7f912dbb0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9128073120 con 0x7f912806fea0 2024-04-25T09:53:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.709+0000 7f912dbb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91280cf0e0 con 0x7f912815a790 2024-04-25T09:53:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.709+0000 7f9127fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f91280b4eb0 0x7f9128072a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47522/0 (socket says 172.21.15.19:47522) 2024-04-25T09:53:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.709+0000 7f9127fff640 1 -- 172.21.15.19:0/4264603209 learned_addr learned my addr 172.21.15.19:0/4264603209 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.710+0000 7f91277fe640 1 --2- 172.21.15.19:0/4264603209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f912815a790 0x7f91280d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.710+0000 7f91267fc640 1 -- 172.21.15.19:0/4264603209 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925654520 0 0) 0x7f91280b3710 con 0x7f91280b4eb0 2024-04-25T09:53:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.710+0000 7f91267fc640 1 -- 172.21.15.19:0/4264603209 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9110003520 con 0x7f91280b4eb0 2024-04-25T09:53:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.710+0000 7f91267fc640 1 -- 172.21.15.19:0/4264603209 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675269269 0 0) 0x7f9128073120 con 0x7f912806fea0 2024-04-25T09:53:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.710+0000 7f91267fc640 1 -- 172.21.15.19:0/4264603209 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91280b3710 con 0x7f912806fea0 2024-04-25T09:53:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.710+0000 7f91267fc640 1 -- 172.21.15.19:0/4264603209 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 571368288 0 0) 0x7f9110003520 con 0x7f91280b4eb0 2024-04-25T09:53:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.710+0000 7f91277fe640 1 -- 172.21.15.19:0/4264603209 >> v1:172.21.15.112:6789/0 conn(0x7f912806fea0 legacy=0x7f9128072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.710+0000 7f91277fe640 1 -- 172.21.15.19:0/4264603209 >> v1:172.21.15.19:6789/0 conn(0x7f91280b4eb0 legacy=0x7f9128072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.710+0000 7f91277fe640 1 -- 172.21.15.19:0/4264603209 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280cf3c0 con 0x7f912815a790 2024-04-25T09:53:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.711+0000 7f91277fe640 1 --2- 172.21.15.19:0/4264603209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f912815a790 0x7f91280d0390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f91180097d0 tx=0x7f911802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a94821ead837881d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.711+0000 7f91267fc640 1 -- 172.21.15.19:0/4264603209 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118037070 con 0x7f912815a790 2024-04-25T09:53:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.711+0000 7f91267fc640 1 -- 172.21.15.19:0/4264603209 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911802ac30 con 0x7f912815a790 2024-04-25T09:53:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.711+0000 7f91267fc640 1 -- 172.21.15.19:0/4264603209 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91200019f0 con 0x7f912815a790 2024-04-25T09:53:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.711+0000 7f912dbb0640 1 -- 172.21.15.19:0/4264603209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f912815a790 msgr2=0x7f91280d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.711+0000 7f912dbb0640 1 --2- 172.21.15.19:0/4264603209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f912815a790 0x7f91280d0390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f91180097d0 tx=0x7f911802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.712+0000 7f912dbb0640 1 -- 172.21.15.19:0/4264603209 shutdown_connections 2024-04-25T09:53:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.712+0000 7f912dbb0640 1 --2- 172.21.15.19:0/4264603209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f912815a790 0x7f91280d0390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.712+0000 7f912dbb0640 1 -- 172.21.15.19:0/4264603209 >> 172.21.15.19:0/4264603209 conn(0x7f91280b2370 msgr2=0x7f91280b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.712+0000 7f912dbb0640 1 -- 172.21.15.19:0/4264603209 shutdown_connections 2024-04-25T09:53:28.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.712+0000 7f912dbb0640 1 -- 172.21.15.19:0/4264603209 wait complete. 2024-04-25T09:53:28.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.713+0000 7f912dbb0640 1 Processor -- start 2024-04-25T09:53:28.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.713+0000 7f912dbb0640 1 -- start start 2024-04-25T09:53:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f912dbb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912806fea0 0x7f91281493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f912dbb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280b4eb0 0x7f912815c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f912dbb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f912815a790 0x7f9128072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f912dbb0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91280cf980 con 0x7f91280b4eb0 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f912dbb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91280b3980 con 0x7f912806fea0 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f912dbb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912815ab70 con 0x7f912815a790 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f9126ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280b4eb0 0x7f912815c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f9126ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280b4eb0 0x7f912815c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37232/0 (socket says 172.21.15.19:37232) 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f9126ffd640 1 -- 172.21.15.19:0/3657494851 learned_addr learned my addr 172.21.15.19:0/3657494851 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f9127fff640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f912815a790 0x7f9128072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f91277fe640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912806fea0 0x7f91281493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f9126ffd640 1 -- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f912815a790 msgr2=0x7f9128072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f9126ffd640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f912815a790 0x7f9128072560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f9126ffd640 1 -- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912806fea0 msgr2=0x7f91281493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f9126ffd640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912806fea0 0x7f91281493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f9126ffd640 1 -- 172.21.15.19:0/3657494851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912815cf50 con 0x7f91280b4eb0 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f9126ffd640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280b4eb0 0x7f912815c4b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f911c00c3c0 tx=0x7f911c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f910ffff640 1 -- 172.21.15.19:0/3657494851 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c017070 con 0x7f91280b4eb0 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f910ffff640 1 -- 172.21.15.19:0/3657494851 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c0052f0 con 0x7f91280b4eb0 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f912dbb0640 1 -- 172.21.15.19:0/3657494851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912815d230 con 0x7f91280b4eb0 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f910ffff640 1 -- 172.21.15.19:0/3657494851 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c003830 con 0x7f91280b4eb0 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.714+0000 7f912dbb0640 1 -- 172.21.15.19:0/3657494851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912815d7f0 con 0x7f91280b4eb0 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.715+0000 7f912dbb0640 1 -- 172.21.15.19:0/3657494851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9128073990 con 0x7f91280b4eb0 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.716+0000 7f910ffff640 1 -- 172.21.15.19:0/3657494851 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f911c01a070 con 0x7f91280b4eb0 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.716+0000 7f910ffff640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911406b4d0 0x7f911406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.716+0000 7f910ffff640 1 -- 172.21.15.19:0/3657494851 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f911c089950 con 0x7f91280b4eb0 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.716+0000 7f91277fe640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911406b4d0 0x7f911406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.717+0000 7f91277fe640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911406b4d0 0x7f911406d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9118034000 tx=0x7f911802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.720+0000 7f910ffff640 1 -- 172.21.15.19:0/3657494851 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f911c058550 con 0x7f91280b4eb0 2024-04-25T09:53:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.901+0000 7f912dbb0640 1 -- 172.21.15.19:0/3657494851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9128156db0 con 0x7f91280b4eb0 2024-04-25T09:53:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.902+0000 7f910ffff640 1 -- 172.21.15.19:0/3657494851 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f911c057e50 con 0x7f91280b4eb0 2024-04-25T09:53:28.904 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:28.904 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 -- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911406b4d0 msgr2=0x7f911406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911406b4d0 0x7f911406d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9118034000 tx=0x7f911802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 -- 172.21.15.19:0/3657494851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280b4eb0 msgr2=0x7f912815c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280b4eb0 0x7f912815c4b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f911c00c3c0 tx=0x7f911c00c880 comp rx=0 tx=0).stop 2024-04-25T09:53:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 -- 172.21.15.19:0/3657494851 shutdown_connections 2024-04-25T09:53:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911406b4d0 0x7f911406d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f912815a790 0x7f9128072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280b4eb0 0x7f912815c4b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 --2- 172.21.15.19:0/3657494851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912806fea0 0x7f91281493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 -- 172.21.15.19:0/3657494851 >> 172.21.15.19:0/3657494851 conn(0x7f91280b2370 msgr2=0x7f91280b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 -- 172.21.15.19:0/3657494851 shutdown_connections 2024-04-25T09:53:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:28.904+0000 7f912dbb0640 1 -- 172.21.15.19:0/3657494851 wait complete. 2024-04-25T09:53:28.917 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:28.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:29.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.020+0000 7f72b257e640 1 Processor -- start 2024-04-25T09:53:29.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.020+0000 7f72b257e640 1 -- start start 2024-04-25T09:53:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.020+0000 7f72b257e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f72ac070dc0 0x7f72ac06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.020+0000 7f72b257e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f72ac0711a0 0x7f72ac0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.020+0000 7f72b257e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72ac0b4830 0x7f72ac0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.020+0000 7f72b257e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ac1622b0 con 0x7f72ac070dc0 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.020+0000 7f72b257e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ac0b10f0 con 0x7f72ac0711a0 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.020+0000 7f72b257e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0d1770 con 0x7f72ac0b4830 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.021+0000 7f72abfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f72ac070dc0 0x7f72ac06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.021+0000 7f72abfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f72ac070dc0 0x7f72ac06ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37246/0 (socket says 172.21.15.19:37246) 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.021+0000 7f72abfff640 1 -- 172.21.15.19:0/1442849889 learned_addr learned my addr 172.21.15.19:0/1442849889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.021+0000 7f72ab7fe640 1 --2- 172.21.15.19:0/1442849889 >> v2:172.21.15.112:3300/0 conn(0x7f72ac0711a0 0x7f72ac0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.021+0000 7f72b0b1b640 1 --2- 172.21.15.19:0/1442849889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72ac0b4830 0x7f72ac0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.021+0000 7f72abfff640 1 -- 172.21.15.19:0/1442849889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72ac0b4830 msgr2=0x7f72ac0b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.021+0000 7f72abfff640 1 --2- 172.21.15.19:0/1442849889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72ac0b4830 0x7f72ac0b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.021+0000 7f72abfff640 1 -- 172.21.15.19:0/1442849889 >> v2:172.21.15.112:3300/0 conn(0x7f72ac0711a0 msgr2=0x7f72ac0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.021+0000 7f72abfff640 1 --2- 172.21.15.19:0/1442849889 >> v2:172.21.15.112:3300/0 conn(0x7f72ac0711a0 0x7f72ac0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.021+0000 7f72abfff640 1 -- 172.21.15.19:0/1442849889 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac162740 con 0x7f72ac070dc0 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72abfff640 1 --2- 172.21.15.19:0/1442849889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72ac070dc0 0x7f72ac06ccd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f72ac04efc0 tx=0x7f729c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f2f65280492ea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72aaffd640 1 -- 172.21.15.19:0/1442849889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c037070 con 0x7f72ac070dc0 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72aaffd640 1 -- 172.21.15.19:0/1442849889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c02ae10 con 0x7f72ac070dc0 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72aaffd640 1 -- 172.21.15.19:0/1442849889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c032660 con 0x7f72ac070dc0 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72b257e640 1 -- 172.21.15.19:0/1442849889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72ac070dc0 msgr2=0x7f72ac06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72b257e640 1 --2- 172.21.15.19:0/1442849889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72ac070dc0 0x7f72ac06ccd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f72ac04efc0 tx=0x7f729c02a100 comp rx=0 tx=0).stop 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72b257e640 1 -- 172.21.15.19:0/1442849889 shutdown_connections 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72b257e640 1 --2- 172.21.15.19:0/1442849889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72ac0b4830 0x7f72ac0b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72b257e640 1 --2- 172.21.15.19:0/1442849889 >> v2:172.21.15.112:3300/0 conn(0x7f72ac0711a0 0x7f72ac0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72b257e640 1 --2- 172.21.15.19:0/1442849889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72ac070dc0 0x7f72ac06ccd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72b257e640 1 -- 172.21.15.19:0/1442849889 >> 172.21.15.19:0/1442849889 conn(0x7f72ac0b3530 msgr2=0x7f72ac0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72b257e640 1 -- 172.21.15.19:0/1442849889 shutdown_connections 2024-04-25T09:53:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.022+0000 7f72b257e640 1 -- 172.21.15.19:0/1442849889 wait complete. 2024-04-25T09:53:29.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.024+0000 7f72b257e640 1 Processor -- start 2024-04-25T09:53:29.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.024+0000 7f72b257e640 1 -- start start 2024-04-25T09:53:29.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.024+0000 7f72b257e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72ac070dc0 0x7f72ac157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.024+0000 7f72b257e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72ac0711a0 0x7f72ac0cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.024+0000 7f72b257e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac0b4830 0x7f72ac0cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.024+0000 7f72b257e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac161f40 con 0x7f72ac070dc0 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.024+0000 7f72b257e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac161c60 con 0x7f72ac0b4830 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.024+0000 7f72b257e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac071630 con 0x7f72ac0711a0 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72ac070dc0 0x7f72ac157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72ac070dc0 0x7f72ac157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37254/0 (socket says 172.21.15.19:37254) 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72abfff640 1 -- 172.21.15.19:0/2126217045 learned_addr learned my addr 172.21.15.19:0/2126217045 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72b0b1b640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac0b4830 0x7f72ac0cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72ab7fe640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72ac0711a0 0x7f72ac0cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72abfff640 1 -- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72ac0711a0 msgr2=0x7f72ac0cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72abfff640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72ac0711a0 0x7f72ac0cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72abfff640 1 -- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac0b4830 msgr2=0x7f72ac0cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72abfff640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac0b4830 0x7f72ac0cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72abfff640 1 -- 172.21.15.19:0/2126217045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac158050 con 0x7f72ac070dc0 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72b0b1b640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac0b4830 0x7f72ac0cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72ab7fe640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72ac0711a0 0x7f72ac0cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72abfff640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72ac070dc0 0x7f72ac157b10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f729c02a140 tx=0x7f729c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72a8ff9640 1 -- 172.21.15.19:0/2126217045 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c041070 con 0x7f72ac070dc0 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72a8ff9640 1 -- 172.21.15.19:0/2126217045 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c03c670 con 0x7f72ac070dc0 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72b257e640 1 -- 172.21.15.19:0/2126217045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ac0cbd70 con 0x7f72ac070dc0 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72a8ff9640 1 -- 172.21.15.19:0/2126217045 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c037040 con 0x7f72ac070dc0 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.025+0000 7f72b257e640 1 -- 172.21.15.19:0/2126217045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ac0cad50 con 0x7f72ac070dc0 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.026+0000 7f72b257e640 1 -- 172.21.15.19:0/2126217045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ac073a00 con 0x7f72ac070dc0 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.027+0000 7f72a8ff9640 1 -- 172.21.15.19:0/2126217045 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f729c044070 con 0x7f72ac070dc0 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.027+0000 7f72a8ff9640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f729806b4d0 0x7f729806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.027+0000 7f72a8ff9640 1 -- 172.21.15.19:0/2126217045 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f729c0aaa70 con 0x7f72ac070dc0 2024-04-25T09:53:29.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.027+0000 7f72ab7fe640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f729806b4d0 0x7f729806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.028+0000 7f72ab7fe640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f729806b4d0 0x7f729806d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0b5460 tx=0x7f72a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.031+0000 7f72a8ff9640 1 -- 172.21.15.19:0/2126217045 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f729c0795b0 con 0x7f72ac070dc0 2024-04-25T09:53:29.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.212+0000 7f72b257e640 1 -- 172.21.15.19:0/2126217045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72ac0b1b00 con 0x7f72ac070dc0 2024-04-25T09:53:29.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.213+0000 7f72a8ff9640 1 -- 172.21.15.19:0/2126217045 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f729c078eb0 con 0x7f72ac070dc0 2024-04-25T09:53:29.215 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:29.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 -- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f729806b4d0 msgr2=0x7f729806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f729806b4d0 0x7f729806d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0b5460 tx=0x7f72a0008040 comp rx=0 tx=0).stop 2024-04-25T09:53:29.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 -- 172.21.15.19:0/2126217045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72ac070dc0 msgr2=0x7f72ac157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72ac070dc0 0x7f72ac157b10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f729c02a140 tx=0x7f729c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 -- 172.21.15.19:0/2126217045 shutdown_connections 2024-04-25T09:53:29.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f729806b4d0 0x7f729806d990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ac0b4830 0x7f72ac0cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72ac0711a0 0x7f72ac0cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 --2- 172.21.15.19:0/2126217045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72ac070dc0 0x7f72ac157b10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 -- 172.21.15.19:0/2126217045 >> 172.21.15.19:0/2126217045 conn(0x7f72ac0b3530 msgr2=0x7f72ac0b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 -- 172.21.15.19:0/2126217045 shutdown_connections 2024-04-25T09:53:29.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.215+0000 7f72b257e640 1 -- 172.21.15.19:0/2126217045 wait complete. 2024-04-25T09:53:29.228 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:29.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.330+0000 7f98a271b640 1 Processor -- start 2024-04-25T09:53:29.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.330+0000 7f98a271b640 1 -- start start 2024-04-25T09:53:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f98a271b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f989c15dc20 0x7f989c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f98a271b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989c0d32f0 0x7f989c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f98a271b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c15e000 con 0x7f989c161710 2024-04-25T09:53:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f98a271b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989c0d1de0 con 0x7f989c15dc20 2024-04-25T09:53:29.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f98a271b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c161c60 con 0x7f989c0d32f0 2024-04-25T09:53:29.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f989bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f989c15dc20 0x7f989c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f989bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f989c15dc20 0x7f989c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37262/0 (socket says 172.21.15.19:37262) 2024-04-25T09:53:29.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f989bfff640 1 -- 172.21.15.19:0/1840935668 learned_addr learned my addr 172.21.15.19:0/1840935668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:29.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f98a0cb8640 1 --2- 172.21.15.19:0/1840935668 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989c0d32f0 0x7f989c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f989bfff640 1 -- 172.21.15.19:0/1840935668 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989c0d32f0 msgr2=0x7f989c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:29.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f989bfff640 1 --2- 172.21.15.19:0/1840935668 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989c0d32f0 0x7f989c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f989bfff640 1 -- 172.21.15.19:0/1840935668 >> v1:172.21.15.112:6789/0 conn(0x7f989c161710 legacy=0x7f989c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.331+0000 7f989bfff640 1 -- 172.21.15.19:0/1840935668 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c161f40 con 0x7f989c15dc20 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.332+0000 7f989bfff640 1 --2- 172.21.15.19:0/1840935668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c15dc20 0x7f989c0ce670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9890009870 tx=0x7f989002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5702710066894b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.332+0000 7f989affd640 1 -- 172.21.15.19:0/1840935668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890037070 con 0x7f989c15dc20 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.332+0000 7f989affd640 1 -- 172.21.15.19:0/1840935668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9890032440 con 0x7f989c15dc20 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.332+0000 7f989affd640 1 -- 172.21.15.19:0/1840935668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890032810 con 0x7f989c15dc20 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.333+0000 7f98a271b640 1 -- 172.21.15.19:0/1840935668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c15dc20 msgr2=0x7f989c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.333+0000 7f98a271b640 1 --2- 172.21.15.19:0/1840935668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c15dc20 0x7f989c0ce670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9890009870 tx=0x7f989002a270 comp rx=0 tx=0).stop 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.333+0000 7f98a271b640 1 -- 172.21.15.19:0/1840935668 shutdown_connections 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.333+0000 7f98a271b640 1 --2- 172.21.15.19:0/1840935668 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989c0d32f0 0x7f989c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.333+0000 7f98a271b640 1 --2- 172.21.15.19:0/1840935668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c15dc20 0x7f989c0ce670 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.333+0000 7f98a271b640 1 -- 172.21.15.19:0/1840935668 >> 172.21.15.19:0/1840935668 conn(0x7f989c0d0420 msgr2=0x7f989c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.333+0000 7f98a271b640 1 -- 172.21.15.19:0/1840935668 shutdown_connections 2024-04-25T09:53:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.333+0000 7f98a271b640 1 -- 172.21.15.19:0/1840935668 wait complete. 2024-04-25T09:53:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.334+0000 7f98a271b640 1 Processor -- start 2024-04-25T09:53:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.334+0000 7f98a271b640 1 -- start start 2024-04-25T09:53:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a271b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989c0d32f0 0x7f989c15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a271b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989c15dc20 0x7f989c1e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a271b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c161710 0x7f989c1e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a271b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0b5e50 con 0x7f989c161710 2024-04-25T09:53:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a271b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c162500 con 0x7f989c0d32f0 2024-04-25T09:53:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a271b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0b5fd0 con 0x7f989c15dc20 2024-04-25T09:53:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a0cb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c161710 0x7f989c1e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a0cb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c161710 0x7f989c1e8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37274/0 (socket says 172.21.15.19:37274) 2024-04-25T09:53:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a0cb8640 1 -- 172.21.15.19:0/2202967735 learned_addr learned my addr 172.21.15.19:0/2202967735 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f989bfff640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989c0d32f0 0x7f989c15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f989b7fe640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989c15dc20 0x7f989c1e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a0cb8640 1 -- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989c15dc20 msgr2=0x7f989c1e82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a0cb8640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989c15dc20 0x7f989c1e82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a0cb8640 1 -- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989c0d32f0 msgr2=0x7f989c15c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a0cb8640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989c0d32f0 0x7f989c15c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a0cb8640 1 -- 172.21.15.19:0/2202967735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c1e90c0 con 0x7f989c161710 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f989b7fe640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989c15dc20 0x7f989c1e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f989bfff640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989c0d32f0 0x7f989c15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.335+0000 7f98a0cb8640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c161710 0x7f989c1e8810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f989400ec10 tx=0x7f989400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.336+0000 7f9898ff9640 1 -- 172.21.15.19:0/2202967735 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894018020 con 0x7f989c161710 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.336+0000 7f9898ff9640 1 -- 172.21.15.19:0/2202967735 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989400ccc0 con 0x7f989c161710 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.336+0000 7f98a271b640 1 -- 172.21.15.19:0/2202967735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989c1f4070 con 0x7f989c161710 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.336+0000 7f9898ff9640 1 -- 172.21.15.19:0/2202967735 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940146c0 con 0x7f989c161710 2024-04-25T09:53:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.336+0000 7f98a271b640 1 -- 172.21.15.19:0/2202967735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c15cd90 con 0x7f989c161710 2024-04-25T09:53:29.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.337+0000 7f98a271b640 1 -- 172.21.15.19:0/2202967735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989c073a00 con 0x7f989c161710 2024-04-25T09:53:29.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.337+0000 7f9898ff9640 1 -- 172.21.15.19:0/2202967735 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9894005d10 con 0x7f989c161710 2024-04-25T09:53:29.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.337+0000 7f9898ff9640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988806b4d0 0x7f988806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.338+0000 7f9898ff9640 1 -- 172.21.15.19:0/2202967735 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f989408b200 con 0x7f989c161710 2024-04-25T09:53:29.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.338+0000 7f989bfff640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988806b4d0 0x7f988806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.338+0000 7f989bfff640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988806b4d0 0x7f988806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9890034000 tx=0x7f989002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.342+0000 7f9898ff9640 1 -- 172.21.15.19:0/2202967735 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9894022030 con 0x7f989c161710 2024-04-25T09:53:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.519+0000 7f98a271b640 1 -- 172.21.15.19:0/2202967735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f989c0d0cc0 con 0x7f989c161710 2024-04-25T09:53:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.519+0000 7f9898ff9640 1 -- 172.21.15.19:0/2202967735 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9894010040 con 0x7f989c161710 2024-04-25T09:53:29.522 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:29.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 -- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988806b4d0 msgr2=0x7f988806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988806b4d0 0x7f988806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9890034000 tx=0x7f989002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 -- 172.21.15.19:0/2202967735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c161710 msgr2=0x7f989c1e8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c161710 0x7f989c1e8810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f989400ec10 tx=0x7f989400c490 comp rx=0 tx=0).stop 2024-04-25T09:53:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 -- 172.21.15.19:0/2202967735 shutdown_connections 2024-04-25T09:53:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988806b4d0 0x7f988806d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c161710 0x7f989c1e8810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989c15dc20 0x7f989c1e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 --2- 172.21.15.19:0/2202967735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989c0d32f0 0x7f989c15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 -- 172.21.15.19:0/2202967735 >> 172.21.15.19:0/2202967735 conn(0x7f989c0d0420 msgr2=0x7f989c0b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 -- 172.21.15.19:0/2202967735 shutdown_connections 2024-04-25T09:53:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.521+0000 7f98a271b640 1 -- 172.21.15.19:0/2202967735 wait complete. 2024-04-25T09:53:29.534 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.636+0000 7fd5bb207640 1 Processor -- start 2024-04-25T09:53:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.636+0000 7fd5bb207640 1 -- start start 2024-04-25T09:53:29.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5bb207640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5b40ced80 0x7fd5b40d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5bb207640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5b4161710 0x7fd5b40d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5bb207640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b415e000 con 0x7fd5b415dc20 2024-04-25T09:53:29.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5bb207640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b40d1de0 con 0x7fd5b4161710 2024-04-25T09:53:29.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5bb207640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b4161c60 con 0x7fd5b40ced80 2024-04-25T09:53:29.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5b97a4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5b4161710 0x7fd5b40d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5b97a4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5b4161710 0x7fd5b40d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37288/0 (socket says 172.21.15.19:37288) 2024-04-25T09:53:29.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5b97a4640 1 -- 172.21.15.19:0/1601146762 learned_addr learned my addr 172.21.15.19:0/1601146762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5b3fff640 1 --2- 172.21.15.19:0/1601146762 >> v2:172.21.15.112:3300/0 conn(0x7fd5b40ced80 0x7fd5b40d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5b97a4640 1 -- 172.21.15.19:0/1601146762 >> v2:172.21.15.112:3300/0 conn(0x7fd5b40ced80 msgr2=0x7fd5b40d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5b97a4640 1 --2- 172.21.15.19:0/1601146762 >> v2:172.21.15.112:3300/0 conn(0x7fd5b40ced80 0x7fd5b40d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5b97a4640 1 -- 172.21.15.19:0/1601146762 >> v1:172.21.15.112:6789/0 conn(0x7fd5b415dc20 legacy=0x7fd5b40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5b97a4640 1 -- 172.21.15.19:0/1601146762 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b4161f40 con 0x7fd5b4161710 2024-04-25T09:53:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.637+0000 7fd5b3fff640 1 --2- 172.21.15.19:0/1601146762 >> v2:172.21.15.112:3300/0 conn(0x7fd5b40ced80 0x7fd5b40d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.638+0000 7fd5b97a4640 1 --2- 172.21.15.19:0/1601146762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b4161710 0x7fd5b40d2820 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5b406da90 tx=0x7fd5ac02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472f8bda68508c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.638+0000 7fd5b37fe640 1 -- 172.21.15.19:0/1601146762 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac036070 con 0x7fd5b4161710 2024-04-25T09:53:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.638+0000 7fd5b37fe640 1 -- 172.21.15.19:0/1601146762 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ac031600 con 0x7fd5b4161710 2024-04-25T09:53:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.638+0000 7fd5b37fe640 1 -- 172.21.15.19:0/1601146762 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0319d0 con 0x7fd5b4161710 2024-04-25T09:53:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.638+0000 7fd5bb207640 1 -- 172.21.15.19:0/1601146762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b4161710 msgr2=0x7fd5b40d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.638+0000 7fd5bb207640 1 --2- 172.21.15.19:0/1601146762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b4161710 0x7fd5b40d2820 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5b406da90 tx=0x7fd5ac02a730 comp rx=0 tx=0).stop 2024-04-25T09:53:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.639+0000 7fd5bb207640 1 -- 172.21.15.19:0/1601146762 shutdown_connections 2024-04-25T09:53:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.639+0000 7fd5bb207640 1 --2- 172.21.15.19:0/1601146762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b4161710 0x7fd5b40d2820 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.639+0000 7fd5bb207640 1 --2- 172.21.15.19:0/1601146762 >> v2:172.21.15.112:3300/0 conn(0x7fd5b40ced80 0x7fd5b40d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.639+0000 7fd5bb207640 1 -- 172.21.15.19:0/1601146762 >> 172.21.15.19:0/1601146762 conn(0x7fd5b40d0420 msgr2=0x7fd5b40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.639+0000 7fd5bb207640 1 -- 172.21.15.19:0/1601146762 shutdown_connections 2024-04-25T09:53:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.639+0000 7fd5bb207640 1 -- 172.21.15.19:0/1601146762 wait complete. 2024-04-25T09:53:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.640+0000 7fd5bb207640 1 Processor -- start 2024-04-25T09:53:29.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.640+0000 7fd5bb207640 1 -- start start 2024-04-25T09:53:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5bb207640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b40ced80 0x7fd5b4159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5bb207640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b415dc20 0x7fd5b4160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5bb207640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5b4161710 0x7fd5b41e7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5bb207640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40b5e50 con 0x7fd5b415dc20 2024-04-25T09:53:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5bb207640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4162500 con 0x7fd5b40ced80 2024-04-25T09:53:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5bb207640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40b5fd0 con 0x7fd5b4161710 2024-04-25T09:53:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b415dc20 0x7fd5b4160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b415dc20 0x7fd5b4160a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37296/0 (socket says 172.21.15.19:37296) 2024-04-25T09:53:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b3fff640 1 -- 172.21.15.19:0/2490383103 learned_addr learned my addr 172.21.15.19:0/2490383103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b97a4640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5b4161710 0x7fd5b41e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b8fa3640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b40ced80 0x7fd5b4159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b3fff640 1 -- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5b4161710 msgr2=0x7fd5b41e7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b3fff640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5b4161710 0x7fd5b41e7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b3fff640 1 -- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b40ced80 msgr2=0x7fd5b4159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b3fff640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b40ced80 0x7fd5b4159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b3fff640 1 -- 172.21.15.19:0/2490383103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b41521d0 con 0x7fd5b415dc20 2024-04-25T09:53:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b8fa3640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b40ced80 0x7fd5b4159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b97a4640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5b4161710 0x7fd5b41e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b3fff640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b415dc20 0x7fd5b4160a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000d7d0 tx=0x7fd5a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b17fa640 1 -- 172.21.15.19:0/2490383103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0004030 con 0x7fd5b415dc20 2024-04-25T09:53:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b17fa640 1 -- 172.21.15.19:0/2490383103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a00041d0 con 0x7fd5b415dc20 2024-04-25T09:53:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5bb207640 1 -- 172.21.15.19:0/2490383103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b415a270 con 0x7fd5b415dc20 2024-04-25T09:53:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.641+0000 7fd5b17fa640 1 -- 172.21.15.19:0/2490383103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00096a0 con 0x7fd5b415dc20 2024-04-25T09:53:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.642+0000 7fd5bb207640 1 -- 172.21.15.19:0/2490383103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b415a830 con 0x7fd5b415dc20 2024-04-25T09:53:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.642+0000 7fd5bb207640 1 -- 172.21.15.19:0/2490383103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b40c7620 con 0x7fd5b415dc20 2024-04-25T09:53:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.643+0000 7fd5b17fa640 1 -- 172.21.15.19:0/2490383103 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5a0009840 con 0x7fd5b415dc20 2024-04-25T09:53:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.643+0000 7fd5b17fa640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd59806b4d0 0x7fd59806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.643+0000 7fd5b17fa640 1 -- 172.21.15.19:0/2490383103 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd5a008a760 con 0x7fd5b415dc20 2024-04-25T09:53:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.644+0000 7fd5b8fa3640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd59806b4d0 0x7fd59806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.644+0000 7fd5b8fa3640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd59806b4d0 0x7fd59806d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80015e0 tx=0x7fd5a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.647+0000 7fd5b17fa640 1 -- 172.21.15.19:0/2490383103 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5b41521d0 con 0x7fd5b415dc20 2024-04-25T09:53:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.827+0000 7fd5bb207640 1 -- 172.21.15.19:0/2490383103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5b4073a00 con 0x7fd5b415dc20 2024-04-25T09:53:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.827+0000 7fd5b17fa640 1 -- 172.21.15.19:0/2490383103 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5a0059330 con 0x7fd5b415dc20 2024-04-25T09:53:29.830 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:29.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.829+0000 7fd5bb207640 1 -- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd59806b4d0 msgr2=0x7fd59806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.829+0000 7fd5bb207640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd59806b4d0 0x7fd59806d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80015e0 tx=0x7fd5a8009040 comp rx=0 tx=0).stop 2024-04-25T09:53:29.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.829+0000 7fd5bb207640 1 -- 172.21.15.19:0/2490383103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b415dc20 msgr2=0x7fd5b4160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.829+0000 7fd5bb207640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b415dc20 0x7fd5b4160a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000d7d0 tx=0x7fd5a000dc90 comp rx=0 tx=0).stop 2024-04-25T09:53:29.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.830+0000 7fd5bb207640 1 -- 172.21.15.19:0/2490383103 shutdown_connections 2024-04-25T09:53:29.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.830+0000 7fd5bb207640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd59806b4d0 0x7fd59806d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.830+0000 7fd5bb207640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5b4161710 0x7fd5b41e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.830+0000 7fd5bb207640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b415dc20 0x7fd5b4160a50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.830+0000 7fd5bb207640 1 --2- 172.21.15.19:0/2490383103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b40ced80 0x7fd5b4159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.830+0000 7fd5bb207640 1 -- 172.21.15.19:0/2490383103 >> 172.21.15.19:0/2490383103 conn(0x7fd5b40d0420 msgr2=0x7fd5b40d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.830+0000 7fd5bb207640 1 -- 172.21.15.19:0/2490383103 shutdown_connections 2024-04-25T09:53:29.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.830+0000 7fd5bb207640 1 -- 172.21.15.19:0/2490383103 wait complete. 2024-04-25T09:53:29.842 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:29.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.944+0000 7fdbbfe9e640 1 Processor -- start 2024-04-25T09:53:29.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.944+0000 7fdbbfe9e640 1 -- start start 2024-04-25T09:53:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.945+0000 7fdbbfe9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdbb8161710 0x7fdbb80d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.945+0000 7fdbbfe9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb815e000 con 0x7fdbb815dc20 2024-04-25T09:53:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.945+0000 7fdbbfe9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb80d1de0 con 0x7fdbb80ced80 2024-04-25T09:53:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.945+0000 7fdbbfe9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb8161c60 con 0x7fdbb8161710 2024-04-25T09:53:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.945+0000 7fdbbe43b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdbb8161710 0x7fdbb80d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.945+0000 7fdbbdc3a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdbb815dc20 0x7fdbb80ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47526/0 (socket says 172.21.15.19:47526) 2024-04-25T09:53:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.945+0000 7fdbbdc3a640 1 -- 172.21.15.19:0/4121608096 learned_addr learned my addr 172.21.15.19:0/4121608096 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.945+0000 7fdbbe43b640 1 -- 172.21.15.19:0/4121608096 >> v1:172.21.15.112:6789/0 conn(0x7fdbb80ced80 legacy=0x7fdbb80d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:29.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.945+0000 7fdbbe43b640 1 -- 172.21.15.19:0/4121608096 >> v1:172.21.15.19:6789/0 conn(0x7fdbb815dc20 legacy=0x7fdbb80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.945+0000 7fdbbe43b640 1 -- 172.21.15.19:0/4121608096 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb8161f40 con 0x7fdbb8161710 2024-04-25T09:53:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.945+0000 7fdbbcc38640 1 -- 172.21.15.19:0/4121608096 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354245332 0 0) 0x7fdbb815e000 con 0x7fdbb815dc20 2024-04-25T09:53:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.946+0000 7fdbbe43b640 1 --2- 172.21.15.19:0/4121608096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb8161710 0x7fdbb80d29f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009d90 tx=0x7fdbb002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a1cce5749f6941 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.946+0000 7fdbbcc38640 1 -- 172.21.15.19:0/4121608096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0007740 con 0x7fdbb8161710 2024-04-25T09:53:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.946+0000 7fdbbcc38640 1 -- 172.21.15.19:0/4121608096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb00078e0 con 0x7fdbb8161710 2024-04-25T09:53:29.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.946+0000 7fdbbcc38640 1 -- 172.21.15.19:0/4121608096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0007cb0 con 0x7fdbb8161710 2024-04-25T09:53:29.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.946+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/4121608096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb8161710 msgr2=0x7fdbb80d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.946+0000 7fdbbfe9e640 1 --2- 172.21.15.19:0/4121608096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb8161710 0x7fdbb80d29f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009d90 tx=0x7fdbb002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.946+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/4121608096 shutdown_connections 2024-04-25T09:53:29.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.946+0000 7fdbbfe9e640 1 --2- 172.21.15.19:0/4121608096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb8161710 0x7fdbb80d29f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.946+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/4121608096 >> 172.21.15.19:0/4121608096 conn(0x7fdbb80d0420 msgr2=0x7fdbb80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.947+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/4121608096 shutdown_connections 2024-04-25T09:53:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.947+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/4121608096 wait complete. 2024-04-25T09:53:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.948+0000 7fdbbfe9e640 1 Processor -- start 2024-04-25T09:53:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.948+0000 7fdbbfe9e640 1 -- start start 2024-04-25T09:53:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.948+0000 7fdbbfe9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb80ced80 0x7fdbb81f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.948+0000 7fdbbfe9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdbb815dc20 0x7fdbb815e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.948+0000 7fdbbfe9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbb8161710 0x7fdbb815fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.948+0000 7fdbbfe9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80b5e50 con 0x7fdbb80ced80 2024-04-25T09:53:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.948+0000 7fdbbfe9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb8162500 con 0x7fdbb8161710 2024-04-25T09:53:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.948+0000 7fdbbfe9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80b5fd0 con 0x7fdbb815dc20 2024-04-25T09:53:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbdc3a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb80ced80 0x7fdbb81f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbdc3a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb80ced80 0x7fdbb81f7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37324/0 (socket says 172.21.15.19:37324) 2024-04-25T09:53:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbdc3a640 1 -- 172.21.15.19:0/2304081329 learned_addr learned my addr 172.21.15.19:0/2304081329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbd439640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdbb815dc20 0x7fdbb815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbe43b640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbb8161710 0x7fdbb815fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbdc3a640 1 -- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdbb815dc20 msgr2=0x7fdbb815e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbdc3a640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdbb815dc20 0x7fdbb815e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbdc3a640 1 -- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbb8161710 msgr2=0x7fdbb815fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbdc3a640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbb8161710 0x7fdbb815fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbdc3a640 1 -- 172.21.15.19:0/2304081329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb815e750 con 0x7fdbb80ced80 2024-04-25T09:53:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbd439640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdbb815dc20 0x7fdbb815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:29.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbe43b640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbb8161710 0x7fdbb815fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbdc3a640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb80ced80 0x7fdbb81f7a80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400c500 tx=0x7fdbb400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbae7fc640 1 -- 172.21.15.19:0/2304081329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4018070 con 0x7fdbb80ced80 2024-04-25T09:53:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbae7fc640 1 -- 172.21.15.19:0/2304081329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb4007580 con 0x7fdbb80ced80 2024-04-25T09:53:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/2304081329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb815ea20 con 0x7fdbb80ced80 2024-04-25T09:53:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbae7fc640 1 -- 172.21.15.19:0/2304081329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4004a20 con 0x7fdbb80ced80 2024-04-25T09:53:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.949+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/2304081329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb815efe0 con 0x7fdbb80ced80 2024-04-25T09:53:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.950+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/2304081329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb8073a00 con 0x7fdbb80ced80 2024-04-25T09:53:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.951+0000 7fdbae7fc640 1 -- 172.21.15.19:0/2304081329 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdbb401b070 con 0x7fdbb80ced80 2024-04-25T09:53:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.951+0000 7fdbae7fc640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb9c06b4d0 0x7fdb9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.951+0000 7fdbae7fc640 1 -- 172.21.15.19:0/2304081329 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdbb40897e0 con 0x7fdbb80ced80 2024-04-25T09:53:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.951+0000 7fdbbd439640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb9c06b4d0 0x7fdb9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.952+0000 7fdbbd439640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb9c06b4d0 0x7fdb9c06d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdbb81f7fc0 tx=0x7fdba8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:29.955+0000 7fdbae7fc640 1 -- 172.21.15.19:0/2304081329 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdbb40583b0 con 0x7fdbb80ced80 2024-04-25T09:53:30.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.134+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/2304081329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdbb80ca300 con 0x7fdbb80ced80 2024-04-25T09:53:30.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.134+0000 7fdbae7fc640 1 -- 172.21.15.19:0/2304081329 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdbb4057cb0 con 0x7fdbb80ced80 2024-04-25T09:53:30.136 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb9c06b4d0 msgr2=0x7fdb9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb9c06b4d0 0x7fdb9c06d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdbb81f7fc0 tx=0x7fdba8008040 comp rx=0 tx=0).stop 2024-04-25T09:53:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/2304081329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb80ced80 msgr2=0x7fdbb81f7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb80ced80 0x7fdbb81f7a80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400c500 tx=0x7fdbb400c9c0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/2304081329 shutdown_connections 2024-04-25T09:53:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb9c06b4d0 0x7fdb9c06d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbb8161710 0x7fdbb815fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdbb815dc20 0x7fdbb815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 --2- 172.21.15.19:0/2304081329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb80ced80 0x7fdbb81f7a80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/2304081329 >> 172.21.15.19:0/2304081329 conn(0x7fdbb80d0420 msgr2=0x7fdbb80d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/2304081329 shutdown_connections 2024-04-25T09:53:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.136+0000 7fdbbfe9e640 1 -- 172.21.15.19:0/2304081329 wait complete. 2024-04-25T09:53:30.149 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.248+0000 7f733886b640 1 Processor -- start 2024-04-25T09:53:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.248+0000 7f733886b640 1 -- start start 2024-04-25T09:53:30.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.248+0000 7f733886b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f733415a790 0x7f73340d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.248+0000 7f733886b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73340b4eb0 0x7f73340d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.248+0000 7f733886b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73340b3710 con 0x7f733406fea0 2024-04-25T09:53:30.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.248+0000 7f733886b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73340cf0e0 con 0x7f73340b4eb0 2024-04-25T09:53:30.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.248+0000 7f733886b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73340cf3c0 con 0x7f733415a790 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.249+0000 7f7332d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73340b4eb0 0x7f73340d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.249+0000 7f7332d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73340b4eb0 0x7f73340d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37334/0 (socket says 172.21.15.19:37334) 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.249+0000 7f7332d9d640 1 -- 172.21.15.19:0/2378834215 learned_addr learned my addr 172.21.15.19:0/2378834215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.249+0000 7f733259c640 1 --2- 172.21.15.19:0/2378834215 >> v2:172.21.15.112:3300/0 conn(0x7f733415a790 0x7f73340d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.249+0000 7f7332d9d640 1 -- 172.21.15.19:0/2378834215 >> v2:172.21.15.112:3300/0 conn(0x7f733415a790 msgr2=0x7f73340d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.249+0000 7f7332d9d640 1 --2- 172.21.15.19:0/2378834215 >> v2:172.21.15.112:3300/0 conn(0x7f733415a790 0x7f73340d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.249+0000 7f7332d9d640 1 -- 172.21.15.19:0/2378834215 >> v1:172.21.15.112:6789/0 conn(0x7f733406fea0 legacy=0x7f7334072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.249+0000 7f7332d9d640 1 -- 172.21.15.19:0/2378834215 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340cf6a0 con 0x7f73340b4eb0 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f7332d9d640 1 --2- 172.21.15.19:0/2378834215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73340b4eb0 0x7f73340d08d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f73340b2e90 tx=0x7f732c02c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a333b71123a35282 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f7331d9b640 1 -- 172.21.15.19:0/2378834215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c00b2e0 con 0x7f73340b4eb0 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f7331d9b640 1 -- 172.21.15.19:0/2378834215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c037070 con 0x7f73340b4eb0 2024-04-25T09:53:30.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f7331d9b640 1 -- 172.21.15.19:0/2378834215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c00b480 con 0x7f73340b4eb0 2024-04-25T09:53:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f733886b640 1 -- 172.21.15.19:0/2378834215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73340b4eb0 msgr2=0x7f73340d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f733886b640 1 --2- 172.21.15.19:0/2378834215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73340b4eb0 0x7f73340d08d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f73340b2e90 tx=0x7f732c02c940 comp rx=0 tx=0).stop 2024-04-25T09:53:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f733886b640 1 -- 172.21.15.19:0/2378834215 shutdown_connections 2024-04-25T09:53:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f733886b640 1 --2- 172.21.15.19:0/2378834215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73340b4eb0 0x7f73340d08d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f733886b640 1 --2- 172.21.15.19:0/2378834215 >> v2:172.21.15.112:3300/0 conn(0x7f733415a790 0x7f73340d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f733886b640 1 -- 172.21.15.19:0/2378834215 >> 172.21.15.19:0/2378834215 conn(0x7f73340b2370 msgr2=0x7f73340b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f733886b640 1 -- 172.21.15.19:0/2378834215 shutdown_connections 2024-04-25T09:53:30.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.250+0000 7f733886b640 1 -- 172.21.15.19:0/2378834215 wait complete. 2024-04-25T09:53:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.252+0000 7f733886b640 1 Processor -- start 2024-04-25T09:53:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.252+0000 7f733886b640 1 -- start start 2024-04-25T09:53:30.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.252+0000 7f733886b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733406fea0 0x7f733415d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.252+0000 7f733886b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73340b4eb0 0x7f733413ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.252+0000 7f733886b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f733415a790 0x7f733413f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.252+0000 7f733886b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733415ab70 con 0x7f733406fea0 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.252+0000 7f733886b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73340cf980 con 0x7f733415a790 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.252+0000 7f733886b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733415acf0 con 0x7f73340b4eb0 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f733259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733406fea0 0x7f733415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f733259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733406fea0 0x7f733415d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37342/0 (socket says 172.21.15.19:37342) 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f733259c640 1 -- 172.21.15.19:0/4117994728 learned_addr learned my addr 172.21.15.19:0/4117994728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f733259c640 1 -- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73340b4eb0 msgr2=0x7f733413ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f733259c640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73340b4eb0 0x7f733413ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f733259c640 1 -- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f733415a790 msgr2=0x7f733413f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f733259c640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f733415a790 0x7f733413f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f733259c640 1 -- 172.21.15.19:0/4117994728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733415dc90 con 0x7f733406fea0 2024-04-25T09:53:30.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f733259c640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733406fea0 0x7f733415d750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f732400e9b0 tx=0x7f732400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f732affd640 1 -- 172.21.15.19:0/4117994728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324017070 con 0x7f733406fea0 2024-04-25T09:53:30.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f732affd640 1 -- 172.21.15.19:0/4117994728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732400c9a0 con 0x7f733406fea0 2024-04-25T09:53:30.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f733886b640 1 -- 172.21.15.19:0/4117994728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7334151aa0 con 0x7f733406fea0 2024-04-25T09:53:30.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f732affd640 1 -- 172.21.15.19:0/4117994728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732400cd90 con 0x7f733406fea0 2024-04-25T09:53:30.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.253+0000 7f733886b640 1 -- 172.21.15.19:0/4117994728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73341523b0 con 0x7f733406fea0 2024-04-25T09:53:30.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.254+0000 7f733886b640 1 -- 172.21.15.19:0/4117994728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7334073990 con 0x7f733406fea0 2024-04-25T09:53:30.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.255+0000 7f732affd640 1 -- 172.21.15.19:0/4117994728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f732401a070 con 0x7f733406fea0 2024-04-25T09:53:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.255+0000 7f732affd640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f731406b4d0 0x7f731406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.255+0000 7f732affd640 1 -- 172.21.15.19:0/4117994728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f732408a4b0 con 0x7f733406fea0 2024-04-25T09:53:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.255+0000 7f732bfff640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f731406b4d0 0x7f731406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.256+0000 7f732bfff640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f731406b4d0 0x7f731406d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f731c000f60 tx=0x7f731c0079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.259+0000 7f732affd640 1 -- 172.21.15.19:0/4117994728 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7324059080 con 0x7f733406fea0 2024-04-25T09:53:30.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.477+0000 7f733886b640 1 -- 172.21.15.19:0/4117994728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f733406d330 con 0x7f733406fea0 2024-04-25T09:53:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.478+0000 7f732affd640 1 -- 172.21.15.19:0/4117994728 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f7324058980 con 0x7f733406fea0 2024-04-25T09:53:30.480 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:30.481 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4000846111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":4000846111},{"type":"v1","addr":"172.21.15.112:6837","nonce":4000846111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1969525282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1969525282},{"type":"v1","addr":"172.21.15.19:6835","nonce":1969525282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:07.175350+0000","modified":"2024-04-25T09:53:23.345595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1080352556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1080352556},{"type":"v1","addr":"172.21.15.112:6836","nonce":1080352556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:53:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T09:53:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 -- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f731406b4d0 msgr2=0x7f731406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f731406b4d0 0x7f731406d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f731c000f60 tx=0x7f731c0079d0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 -- 172.21.15.19:0/4117994728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733406fea0 msgr2=0x7f733415d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733406fea0 0x7f733415d750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f732400e9b0 tx=0x7f732400ee70 comp rx=0 tx=0).stop 2024-04-25T09:53:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 -- 172.21.15.19:0/4117994728 shutdown_connections 2024-04-25T09:53:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f731406b4d0 0x7f731406d990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f733415a790 0x7f733413f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73340b4eb0 0x7f733413ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 --2- 172.21.15.19:0/4117994728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733406fea0 0x7f733415d750 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 -- 172.21.15.19:0/4117994728 >> 172.21.15.19:0/4117994728 conn(0x7f73340b2370 msgr2=0x7f73341583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 -- 172.21.15.19:0/4117994728 shutdown_connections 2024-04-25T09:53:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.480+0000 7f733886b640 1 -- 172.21.15.19:0/4117994728 wait complete. 2024-04-25T09:53:30.493 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.592+0000 7f76dc05d640 1 Processor -- start 2024-04-25T09:53:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.592+0000 7f76dc05d640 1 -- start start 2024-04-25T09:53:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76dc05d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f76d415dbe0 0x7f76d40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76dc05d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f76d40b2cb0 0x7f76d40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76dc05d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d40b97a0 con 0x7f76d40b92a0 2024-04-25T09:53:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76dc05d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d415dfc0 con 0x7f76d415dbe0 2024-04-25T09:53:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76dc05d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d40d1a00 con 0x7f76d40b2cb0 2024-04-25T09:53:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76d9df9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f76d415dbe0 0x7f76d40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76da5fa640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f76d40b92a0 0x7f76d40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47538/0 (socket says 172.21.15.19:47538) 2024-04-25T09:53:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76da5fa640 1 -- 172.21.15.19:0/4012774730 learned_addr learned my addr 172.21.15.19:0/4012774730 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76d95f8640 1 --2- 172.21.15.19:0/4012774730 >> v2:172.21.15.112:3300/0 conn(0x7f76d40b2cb0 0x7f76d40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76d9df9640 1 -- 172.21.15.19:0/4012774730 >> v2:172.21.15.112:3300/0 conn(0x7f76d40b2cb0 msgr2=0x7f76d40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76d9df9640 1 --2- 172.21.15.19:0/4012774730 >> v2:172.21.15.112:3300/0 conn(0x7f76d40b2cb0 0x7f76d40d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76d9df9640 1 -- 172.21.15.19:0/4012774730 >> v1:172.21.15.19:6789/0 conn(0x7f76d40b92a0 legacy=0x7f76d40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.593+0000 7f76d9df9640 1 -- 172.21.15.19:0/4012774730 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d40d1c50 con 0x7f76d415dbe0 2024-04-25T09:53:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.594+0000 7f76d9df9640 1 --2- 172.21.15.19:0/4012774730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d415dbe0 0x7f76d40d0910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f76d406cff0 tx=0x7f76d002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbdadaffcb71026 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.594+0000 7f76d8df7640 1 -- 172.21.15.19:0/4012774730 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0037070 con 0x7f76d415dbe0 2024-04-25T09:53:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.594+0000 7f76d8df7640 1 -- 172.21.15.19:0/4012774730 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d0032440 con 0x7f76d415dbe0 2024-04-25T09:53:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.594+0000 7f76d8df7640 1 -- 172.21.15.19:0/4012774730 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d002ae60 con 0x7f76d415dbe0 2024-04-25T09:53:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.595+0000 7f76dc05d640 1 -- 172.21.15.19:0/4012774730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d415dbe0 msgr2=0x7f76d40d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.595+0000 7f76dc05d640 1 --2- 172.21.15.19:0/4012774730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d415dbe0 0x7f76d40d0910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f76d406cff0 tx=0x7f76d002a110 comp rx=0 tx=0).stop 2024-04-25T09:53:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.595+0000 7f76dc05d640 1 -- 172.21.15.19:0/4012774730 shutdown_connections 2024-04-25T09:53:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.595+0000 7f76dc05d640 1 --2- 172.21.15.19:0/4012774730 >> v2:172.21.15.112:3300/0 conn(0x7f76d40b2cb0 0x7f76d40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.595+0000 7f76dc05d640 1 --2- 172.21.15.19:0/4012774730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d415dbe0 0x7f76d40d0910 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.595+0000 7f76dc05d640 1 -- 172.21.15.19:0/4012774730 >> 172.21.15.19:0/4012774730 conn(0x7f76d4061a20 msgr2=0x7f76d4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.595+0000 7f76dc05d640 1 -- 172.21.15.19:0/4012774730 shutdown_connections 2024-04-25T09:53:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.595+0000 7f76dc05d640 1 -- 172.21.15.19:0/4012774730 wait complete. 2024-04-25T09:53:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.596+0000 7f76dc05d640 1 Processor -- start 2024-04-25T09:53:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.596+0000 7f76dc05d640 1 -- start start 2024-04-25T09:53:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76dc05d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76d40b2cb0 0x7f76d40c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76dc05d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76d40b92a0 0x7f76d40cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76dc05d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d415dbe0 0x7f76d40b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76dc05d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40b6ab0 con 0x7f76d415dbe0 2024-04-25T09:53:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76dc05d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40d2210 con 0x7f76d40b2cb0 2024-04-25T09:53:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76dc05d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40b6c30 con 0x7f76d40b92a0 2024-04-25T09:53:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76da5fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d415dbe0 0x7f76d40b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76da5fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d415dbe0 0x7f76d40b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37354/0 (socket says 172.21.15.19:37354) 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76da5fa640 1 -- 172.21.15.19:0/1579079478 learned_addr learned my addr 172.21.15.19:0/1579079478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76d9df9640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76d40b2cb0 0x7f76d40c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76d95f8640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76d40b92a0 0x7f76d40cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76da5fa640 1 -- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76d40b92a0 msgr2=0x7f76d40cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76da5fa640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76d40b92a0 0x7f76d40cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76da5fa640 1 -- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76d40b2cb0 msgr2=0x7f76d40c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76da5fa640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76d40b2cb0 0x7f76d40c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76da5fa640 1 -- 172.21.15.19:0/1579079478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d40cb040 con 0x7f76d415dbe0 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.597+0000 7f76da5fa640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d415dbe0 0x7f76d40b5c50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00caf0 tx=0x7f76cc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.598+0000 7f76c67fc640 1 -- 172.21.15.19:0/1579079478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc017070 con 0x7f76d415dbe0 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.598+0000 7f76c67fc640 1 -- 172.21.15.19:0/1579079478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc00aca0 con 0x7f76d415dbe0 2024-04-25T09:53:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.598+0000 7f76dc05d640 1 -- 172.21.15.19:0/1579079478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d40b4a90 con 0x7f76d415dbe0 2024-04-25T09:53:30.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.598+0000 7f76c67fc640 1 -- 172.21.15.19:0/1579079478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc012600 con 0x7f76d415dbe0 2024-04-25T09:53:30.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.598+0000 7f76dc05d640 1 -- 172.21.15.19:0/1579079478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d40b5450 con 0x7f76d415dbe0 2024-04-25T09:53:30.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.599+0000 7f76dc05d640 1 -- 172.21.15.19:0/1579079478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d4073990 con 0x7f76d415dbe0 2024-04-25T09:53:30.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.599+0000 7f76c67fc640 1 -- 172.21.15.19:0/1579079478 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76cc01a070 con 0x7f76d415dbe0 2024-04-25T09:53:30.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.599+0000 7f76c67fc640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f76b806b500 0x7f76b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.599+0000 7f76c67fc640 1 -- 172.21.15.19:0/1579079478 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f76cc08a110 con 0x7f76d415dbe0 2024-04-25T09:53:30.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.600+0000 7f76d9df9640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f76b806b500 0x7f76b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.600+0000 7f76d9df9640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f76b806b500 0x7f76b806d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f76d0034000 tx=0x7f76d002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.604+0000 7f76c67fc640 1 -- 172.21.15.19:0/1579079478 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76cc058600 con 0x7f76d415dbe0 2024-04-25T09:53:30.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.784+0000 7f76dc05d640 1 -- 172.21.15.19:0/1579079478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76d406c770 con 0x7f76d415dbe0 2024-04-25T09:53:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.785+0000 7f76c67fc640 1 -- 172.21.15.19:0/1579079478 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f76cc05c6a0 con 0x7f76d415dbe0 2024-04-25T09:53:30.788 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:30.788 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 -- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f76b806b500 msgr2=0x7f76b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f76b806b500 0x7f76b806d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f76d0034000 tx=0x7f76d002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 -- 172.21.15.19:0/1579079478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d415dbe0 msgr2=0x7f76d40b5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d415dbe0 0x7f76d40b5c50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00caf0 tx=0x7f76cc00a060 comp rx=0 tx=0).stop 2024-04-25T09:53:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 -- 172.21.15.19:0/1579079478 shutdown_connections 2024-04-25T09:53:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f76b806b500 0x7f76b806d9c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d415dbe0 0x7f76d40b5c50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76d40b92a0 0x7f76d40cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 --2- 172.21.15.19:0/1579079478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76d40b2cb0 0x7f76d40c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 -- 172.21.15.19:0/1579079478 >> 172.21.15.19:0/1579079478 conn(0x7f76d4061a20 msgr2=0x7f76d415c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:30.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 -- 172.21.15.19:0/1579079478 shutdown_connections 2024-04-25T09:53:30.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.788+0000 7f76dc05d640 1 -- 172.21.15.19:0/1579079478 wait complete. 2024-04-25T09:53:30.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:30.802 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:30.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.903+0000 7fdaf96f5640 1 Processor -- start 2024-04-25T09:53:30.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.903+0000 7fdaf96f5640 1 -- start start 2024-04-25T09:53:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.904+0000 7fdaf96f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdaf41616f0 0x7fdaf4161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.904+0000 7fdaf96f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdaf40ce0a0 0x7fdaf40ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.904+0000 7fdaf96f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf406c8d0 con 0x7fdaf41610f0 2024-04-25T09:53:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.904+0000 7fdaf96f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf41614d0 con 0x7fdaf41616f0 2024-04-25T09:53:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.904+0000 7fdaf96f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40c6960 con 0x7fdaf40ce0a0 2024-04-25T09:53:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.904+0000 7fdaf27fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdaf41616f0 0x7fdaf4161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.904+0000 7fdaf27fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdaf41616f0 0x7fdaf4161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37364/0 (socket says 172.21.15.19:37364) 2024-04-25T09:53:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.904+0000 7fdaf27fc640 1 -- 172.21.15.19:0/1345265040 learned_addr learned my addr 172.21.15.19:0/1345265040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.905+0000 7fdaf37fe640 1 --2- 172.21.15.19:0/1345265040 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdaf40ce0a0 0x7fdaf40ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.905+0000 7fdaf27fc640 1 -- 172.21.15.19:0/1345265040 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdaf40ce0a0 msgr2=0x7fdaf40ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:30.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.905+0000 7fdaf27fc640 1 --2- 172.21.15.19:0/1345265040 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdaf40ce0a0 0x7fdaf40ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.905+0000 7fdaf27fc640 1 -- 172.21.15.19:0/1345265040 >> v1:172.21.15.112:6789/0 conn(0x7fdaf41610f0 legacy=0x7fdaf406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:30.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.905+0000 7fdaf27fc640 1 -- 172.21.15.19:0/1345265040 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf4162260 con 0x7fdaf41616f0 2024-04-25T09:53:30.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.905+0000 7fdaf27fc640 1 --2- 172.21.15.19:0/1345265040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf41616f0 0x7fdaf4161b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009870 tx=0x7fdae802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489e11e060d85d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.905+0000 7fdaf1ffb640 1 -- 172.21.15.19:0/1345265040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8037070 con 0x7fdaf41616f0 2024-04-25T09:53:30.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.906+0000 7fdaf1ffb640 1 -- 172.21.15.19:0/1345265040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae8032440 con 0x7fdaf41616f0 2024-04-25T09:53:30.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.906+0000 7fdaf1ffb640 1 -- 172.21.15.19:0/1345265040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8032810 con 0x7fdaf41616f0 2024-04-25T09:53:30.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.906+0000 7fdaf96f5640 1 -- 172.21.15.19:0/1345265040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf41616f0 msgr2=0x7fdaf4161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.906+0000 7fdaf96f5640 1 --2- 172.21.15.19:0/1345265040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf41616f0 0x7fdaf4161b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009870 tx=0x7fdae802a210 comp rx=0 tx=0).stop 2024-04-25T09:53:30.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.906+0000 7fdaf96f5640 1 -- 172.21.15.19:0/1345265040 shutdown_connections 2024-04-25T09:53:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.906+0000 7fdaf96f5640 1 --2- 172.21.15.19:0/1345265040 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdaf40ce0a0 0x7fdaf40ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.906+0000 7fdaf96f5640 1 --2- 172.21.15.19:0/1345265040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf41616f0 0x7fdaf4161b50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.906+0000 7fdaf96f5640 1 -- 172.21.15.19:0/1345265040 >> 172.21.15.19:0/1345265040 conn(0x7fdaf406f500 msgr2=0x7fdaf406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.906+0000 7fdaf96f5640 1 -- 172.21.15.19:0/1345265040 shutdown_connections 2024-04-25T09:53:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.906+0000 7fdaf96f5640 1 -- 172.21.15.19:0/1345265040 wait complete. 2024-04-25T09:53:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.907+0000 7fdaf96f5640 1 Processor -- start 2024-04-25T09:53:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.907+0000 7fdaf96f5640 1 -- start start 2024-04-25T09:53:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.908+0000 7fdaf96f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdaf40ce0a0 0x7fdaf4149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.908+0000 7fdaf96f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf41610f0 0x7fdaf4147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.908+0000 7fdaf96f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf41616f0 0x7fdaf4159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.908+0000 7fdaf96f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40cfce0 con 0x7fdaf41610f0 2024-04-25T09:53:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.908+0000 7fdaf96f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40cfa00 con 0x7fdaf41616f0 2024-04-25T09:53:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.908+0000 7fdaf96f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40cf690 con 0x7fdaf40ce0a0 2024-04-25T09:53:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.908+0000 7fdaf27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf41610f0 0x7fdaf4147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.908+0000 7fdaf27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf41610f0 0x7fdaf4147ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37372/0 (socket says 172.21.15.19:37372) 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.908+0000 7fdaf27fc640 1 -- 172.21.15.19:0/877117696 learned_addr learned my addr 172.21.15.19:0/877117696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.908+0000 7fdaf2ffd640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdaf40ce0a0 0x7fdaf4149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdaf37fe640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf41616f0 0x7fdaf4159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdaf27fc640 1 -- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdaf40ce0a0 msgr2=0x7fdaf4149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdaf27fc640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdaf40ce0a0 0x7fdaf4149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdaf27fc640 1 -- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf41616f0 msgr2=0x7fdaf4159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdaf27fc640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf41616f0 0x7fdaf4159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdaf27fc640 1 -- 172.21.15.19:0/877117696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf41e75d0 con 0x7fdaf41610f0 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdaf37fe640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf41616f0 0x7fdaf4159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdaf27fc640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf41610f0 0x7fdaf4147ac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdae8000c00 tx=0x7fdae80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdadb7fe640 1 -- 172.21.15.19:0/877117696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8041020 con 0x7fdaf41610f0 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdadb7fe640 1 -- 172.21.15.19:0/877117696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae80306c0 con 0x7fdaf41610f0 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdaf96f5640 1 -- 172.21.15.19:0/877117696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf41e78b0 con 0x7fdaf41610f0 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdadb7fe640 1 -- 172.21.15.19:0/877117696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8037040 con 0x7fdaf41610f0 2024-04-25T09:53:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.909+0000 7fdaf96f5640 1 -- 172.21.15.19:0/877117696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf41e7e70 con 0x7fdaf41610f0 2024-04-25T09:53:30.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.910+0000 7fdaf96f5640 1 -- 172.21.15.19:0/877117696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf406afe0 con 0x7fdaf41610f0 2024-04-25T09:53:30.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.910+0000 7fdadb7fe640 1 -- 172.21.15.19:0/877117696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdae8044020 con 0x7fdaf41610f0 2024-04-25T09:53:30.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.911+0000 7fdadb7fe640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdae006b500 0x7fdae006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.911+0000 7fdadb7fe640 1 -- 172.21.15.19:0/877117696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdae80aad40 con 0x7fdaf41610f0 2024-04-25T09:53:30.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.911+0000 7fdaf2ffd640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdae006b500 0x7fdae006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.912+0000 7fdaf2ffd640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdae006b500 0x7fdae006d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdaec004ea0 tx=0x7fdaec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:30.915+0000 7fdadb7fe640 1 -- 172.21.15.19:0/877117696 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdaf406afe0 con 0x7fdaf41610f0 2024-04-25T09:53:31.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.099+0000 7fdaf96f5640 1 -- 172.21.15.19:0/877117696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdaf4073a00 con 0x7fdaf41610f0 2024-04-25T09:53:31.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.100+0000 7fdadb7fe640 1 -- 172.21.15.19:0/877117696 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdae804b030 con 0x7fdaf41610f0 2024-04-25T09:53:31.102 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:31.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 -- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdae006b500 msgr2=0x7fdae006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdae006b500 0x7fdae006d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdaec004ea0 tx=0x7fdaec009040 comp rx=0 tx=0).stop 2024-04-25T09:53:31.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 -- 172.21.15.19:0/877117696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf41610f0 msgr2=0x7fdaf4147ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf41610f0 0x7fdaf4147ac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdae8000c00 tx=0x7fdae80046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 -- 172.21.15.19:0/877117696 shutdown_connections 2024-04-25T09:53:31.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdae006b500 0x7fdae006d9c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf41616f0 0x7fdaf4159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf41610f0 0x7fdaf4147ac0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 --2- 172.21.15.19:0/877117696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdaf40ce0a0 0x7fdaf4149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 -- 172.21.15.19:0/877117696 >> 172.21.15.19:0/877117696 conn(0x7fdaf406f500 msgr2=0x7fdaf40d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 -- 172.21.15.19:0/877117696 shutdown_connections 2024-04-25T09:53:31.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.102+0000 7fdaf96f5640 1 -- 172.21.15.19:0/877117696 wait complete. 2024-04-25T09:53:31.115 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.218+0000 7f8f5dc1f640 1 Processor -- start 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.218+0000 7f8f5dc1f640 1 -- start start 2024-04-25T09:53:31.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.218+0000 7f8f5dc1f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8f580b5620 0x7f8f580b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.218+0000 7f8f5dc1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5806c2d0 0x7f8f5806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.218+0000 7f8f5dc1f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5806cc80 con 0x7f8f5815b7d0 2024-04-25T09:53:31.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.218+0000 7f8f5dc1f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5806cf50 con 0x7f8f580b5620 2024-04-25T09:53:31.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.218+0000 7f8f5dc1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f580ce530 con 0x7f8f5806c2d0 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f577fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8f5815b7d0 0x7f8f580cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47546/0 (socket says 172.21.15.19:47546) 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f577fe640 1 -- 172.21.15.19:0/386739600 learned_addr learned my addr 172.21.15.19:0/386739600 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f57fff640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5806c2d0 0x7f8f5806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f56ffd640 1 --2- 172.21.15.19:0/386739600 >> v2:172.21.15.112:3300/0 conn(0x7f8f580b5620 0x7f8f580b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713320111 0 0) 0x7f8f5806cc80 con 0x7f8f5815b7d0 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f40003520 con 0x7f8f5815b7d0 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3724290205 0 0) 0x7f8f40003520 con 0x7f8f5815b7d0 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5806c2d0 msgr2=0x7f8f5806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5806c2d0 0x7f8f5806c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 >> v2:172.21.15.112:3300/0 conn(0x7f8f580b5620 msgr2=0x7f8f580b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 --2- 172.21.15.19:0/386739600 >> v2:172.21.15.112:3300/0 conn(0x7f8f580b5620 0x7f8f580b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f580ce810 con 0x7f8f5815b7d0 2024-04-25T09:53:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8f48001670 con 0x7f8f5815b7d0 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 >> v1:172.21.15.19:6789/0 conn(0x7f8f5815b7d0 legacy=0x7f8f580cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f400039b0 0x7f8f40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 0x7f8f40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5806c2d0 0x7f8f400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5806cf50 con 0x7f8f40004800 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f580ce530 con 0x7f8f400039b0 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4000b9c0 con 0x7f8f5806c2d0 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f56ffd640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 0x7f8f40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f577fe640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f400039b0 0x7f8f40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.219+0000 7f8f57fff640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5806c2d0 0x7f8f400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.220+0000 7f8f56ffd640 1 -- 172.21.15.19:0/386739600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5806c2d0 msgr2=0x7f8f400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.220+0000 7f8f56ffd640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5806c2d0 0x7f8f400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.220+0000 7f8f56ffd640 1 -- 172.21.15.19:0/386739600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f400039b0 msgr2=0x7f8f40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.220+0000 7f8f56ffd640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f400039b0 0x7f8f40003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.220+0000 7f8f56ffd640 1 -- 172.21.15.19:0/386739600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f40003520 con 0x7f8f40004800 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.220+0000 7f8f56ffd640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 0x7f8f40004c80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009ac0 tx=0x7f8f40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0116d73d02f946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.220+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c036070 con 0x7f8f40004800 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.221+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4c031400 con 0x7f8f40004800 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.221+0000 7f8f567fc640 1 -- 172.21.15.19:0/386739600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c009690 con 0x7f8f40004800 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.221+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/386739600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 msgr2=0x7f8f40004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.221+0000 7f8f5dc1f640 1 --2- 172.21.15.19:0/386739600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 0x7f8f40004c80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009ac0 tx=0x7f8f40005830 comp rx=0 tx=0).stop 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.221+0000 7f8f57fff640 1 -- 172.21.15.19:0/386739600 reap_dead start 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.221+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/386739600 shutdown_connections 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.221+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/386739600 >> 172.21.15.19:0/386739600 conn(0x7f8f580d0800 msgr2=0x7f8f580d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.221+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/386739600 shutdown_connections 2024-04-25T09:53:31.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.221+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/386739600 wait complete. 2024-04-25T09:53:31.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.222+0000 7f8f5dc1f640 1 Processor -- start 2024-04-25T09:53:31.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.222+0000 7f8f5dc1f640 1 -- start start 2024-04-25T09:53:31.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.223+0000 7f8f5dc1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 0x7f8f580cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.223+0000 7f8f5dc1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f580cac70 0x7f8f5814edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.223+0000 7f8f5dc1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f581f6010 0x7f8f581f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.223+0000 7f8f5dc1f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5815bbb0 con 0x7f8f40004800 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.223+0000 7f8f5dc1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5806d230 con 0x7f8f581f6010 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.223+0000 7f8f5dc1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5815bd30 con 0x7f8f580cac70 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.223+0000 7f8f577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 0x7f8f580cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.223+0000 7f8f577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 0x7f8f580cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37380/0 (socket says 172.21.15.19:37380) 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.223+0000 7f8f577fe640 1 -- 172.21.15.19:0/3237385408 learned_addr learned my addr 172.21.15.19:0/3237385408 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.223+0000 7f8f56ffd640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f580cac70 0x7f8f5814edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.223+0000 7f8f57fff640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f581f6010 0x7f8f581f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f577fe640 1 -- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f580cac70 msgr2=0x7f8f5814edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f577fe640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f580cac70 0x7f8f5814edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f577fe640 1 -- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f581f6010 msgr2=0x7f8f581f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f577fe640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f581f6010 0x7f8f581f6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f577fe640 1 -- 172.21.15.19:0/3237385408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f580cc3c0 con 0x7f8f40004800 2024-04-25T09:53:31.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f57fff640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f581f6010 0x7f8f581f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f577fe640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 0x7f8f580cbb40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8f48007340 tx=0x7f8f48007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f3ffff640 1 -- 172.21.15.19:0/3237385408 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800f8c0 con 0x7f8f40004800 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f3ffff640 1 -- 172.21.15.19:0/3237385408 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f48018070 con 0x7f8f40004800 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f3ffff640 1 -- 172.21.15.19:0/3237385408 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800fcc0 con 0x7f8f40004800 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/3237385408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f580cc6a0 con 0x7f8f40004800 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.224+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/3237385408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f580ccc60 con 0x7f8f40004800 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.225+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/3237385408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f58073a00 con 0x7f8f40004800 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.226+0000 7f8f3ffff640 1 -- 172.21.15.19:0/3237385408 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f4801c410 con 0x7f8f40004800 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.226+0000 7f8f3ffff640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f4406b500 0x7f8f4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.226+0000 7f8f3ffff640 1 -- 172.21.15.19:0/3237385408 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8f4808a260 con 0x7f8f40004800 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.226+0000 7f8f56ffd640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f4406b500 0x7f8f4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.227+0000 7f8f56ffd640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f4406b500 0x7f8f4406d9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c033000 tx=0x7f8f4c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.230+0000 7f8f3ffff640 1 -- 172.21.15.19:0/3237385408 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f4800fe60 con 0x7f8f40004800 2024-04-25T09:53:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.408+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/3237385408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f5815b1d0 con 0x7f8f40004800 2024-04-25T09:53:31.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.408+0000 7f8f3ffff640 1 -- 172.21.15.19:0/3237385408 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f48058d10 con 0x7f8f40004800 2024-04-25T09:53:31.410 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f4406b500 msgr2=0x7f8f4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f4406b500 0x7f8f4406d9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c033000 tx=0x7f8f4c008030 comp rx=0 tx=0).stop 2024-04-25T09:53:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/3237385408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 msgr2=0x7f8f580cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 0x7f8f580cbb40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8f48007340 tx=0x7f8f48007800 comp rx=0 tx=0).stop 2024-04-25T09:53:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/3237385408 shutdown_connections 2024-04-25T09:53:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f4406b500 0x7f8f4406d9c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f581f6010 0x7f8f581f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f580cac70 0x7f8f5814edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 --2- 172.21.15.19:0/3237385408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f40004800 0x7f8f580cbb40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/3237385408 >> 172.21.15.19:0/3237385408 conn(0x7f8f580d0800 msgr2=0x7f8f5815d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/3237385408 shutdown_connections 2024-04-25T09:53:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.410+0000 7f8f5dc1f640 1 -- 172.21.15.19:0/3237385408 wait complete. 2024-04-25T09:53:31.423 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.524+0000 7ff0319f5640 1 Processor -- start 2024-04-25T09:53:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.524+0000 7ff0319f5640 1 -- start start 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.525+0000 7ff0319f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff02c0ccdd0 0x7ff02c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.525+0000 7ff0319f5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02c0cb190 con 0x7ff02c0cdcd0 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.525+0000 7ff0319f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02c0cb470 con 0x7ff02c15bb20 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.525+0000 7ff0319f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c0cb750 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.525+0000 7ff02b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff02c0cdcd0 0x7ff02c0ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47554/0 (socket says 172.21.15.19:47554) 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.525+0000 7ff02b7fe640 1 -- 172.21.15.19:0/3755544131 learned_addr learned my addr 172.21.15.19:0/3755544131 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.525+0000 7ff02a7fc640 1 --2- 172.21.15.19:0/3755544131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff02c0ccdd0 0x7ff02c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.526+0000 7ff029ffb640 1 -- 172.21.15.19:0/3755544131 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654720603 0 0) 0x7ff02c0cb190 con 0x7ff02c0cdcd0 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.526+0000 7ff029ffb640 1 -- 172.21.15.19:0/3755544131 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff014003520 con 0x7ff02c0cdcd0 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.526+0000 7ff029ffb640 1 -- 172.21.15.19:0/3755544131 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1588164450 0 0) 0x7ff014003520 con 0x7ff02c0cdcd0 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.526+0000 7ff02a7fc640 1 -- 172.21.15.19:0/3755544131 >> v1:172.21.15.112:6789/0 conn(0x7ff02c15bb20 legacy=0x7ff02c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.526+0000 7ff02a7fc640 1 -- 172.21.15.19:0/3755544131 >> v1:172.21.15.19:6789/0 conn(0x7ff02c0cdcd0 legacy=0x7ff02c0ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.526+0000 7ff02a7fc640 1 -- 172.21.15.19:0/3755544131 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c0cba30 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.527+0000 7ff02a7fc640 1 --2- 172.21.15.19:0/3755544131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff02c0ccdd0 0x7ff02c0cf9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff0200097d0 tx=0x7ff02002a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe34c6c47d26ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.527+0000 7ff029ffb640 1 -- 172.21.15.19:0/3755544131 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020037070 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.527+0000 7ff029ffb640 1 -- 172.21.15.19:0/3755544131 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02002aac0 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.527+0000 7ff029ffb640 1 -- 172.21.15.19:0/3755544131 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0019f0 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.527+0000 7ff0319f5640 1 -- 172.21.15.19:0/3755544131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff02c0ccdd0 msgr2=0x7ff02c0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.527+0000 7ff0319f5640 1 --2- 172.21.15.19:0/3755544131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff02c0ccdd0 0x7ff02c0cf9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff0200097d0 tx=0x7ff02002a030 comp rx=0 tx=0).stop 2024-04-25T09:53:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.527+0000 7ff0319f5640 1 -- 172.21.15.19:0/3755544131 shutdown_connections 2024-04-25T09:53:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.527+0000 7ff0319f5640 1 --2- 172.21.15.19:0/3755544131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff02c0ccdd0 0x7ff02c0cf9d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.527+0000 7ff0319f5640 1 -- 172.21.15.19:0/3755544131 >> 172.21.15.19:0/3755544131 conn(0x7ff02c0b2390 msgr2=0x7ff02c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.527+0000 7ff0319f5640 1 -- 172.21.15.19:0/3755544131 shutdown_connections 2024-04-25T09:53:31.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.527+0000 7ff0319f5640 1 -- 172.21.15.19:0/3755544131 wait complete. 2024-04-25T09:53:31.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.528+0000 7ff0319f5640 1 Processor -- start 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.529+0000 7ff0319f5640 1 -- start start 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.529+0000 7ff0319f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02c0ccdd0 0x7ff02c1f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.529+0000 7ff0319f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff02c0cdcd0 0x7ff02c0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.529+0000 7ff0319f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff02c15bb20 0x7ff02c153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.529+0000 7ff0319f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c15c080 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.529+0000 7ff0319f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c15bf00 con 0x7ff02c0cdcd0 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.529+0000 7ff0319f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c0cadb0 con 0x7ff02c15bb20 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.529+0000 7ff02affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02c0ccdd0 0x7ff02c1f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.529+0000 7ff02affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02c0ccdd0 0x7ff02c1f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37392/0 (socket says 172.21.15.19:37392) 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.529+0000 7ff02affd640 1 -- 172.21.15.19:0/117375469 learned_addr learned my addr 172.21.15.19:0/117375469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff02b7fe640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff02c15bb20 0x7ff02c153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff02a7fc640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff02c0cdcd0 0x7ff02c0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff02affd640 1 -- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff02c15bb20 msgr2=0x7ff02c153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff02affd640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff02c15bb20 0x7ff02c153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff02affd640 1 -- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff02c0cdcd0 msgr2=0x7ff02c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff02affd640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff02c0cdcd0 0x7ff02c0b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff02affd640 1 -- 172.21.15.19:0/117375469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c0b3bf0 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff02affd640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02c0ccdd0 0x7ff02c1f14a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff02400c570 tx=0x7ff02400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff013fff640 1 -- 172.21.15.19:0/117375469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024005480 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff013fff640 1 -- 172.21.15.19:0/117375469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0240035f0 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff0319f5640 1 -- 172.21.15.19:0/117375469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02c0b3ed0 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff013fff640 1 -- 172.21.15.19:0/117375469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0240039c0 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.530+0000 7ff0319f5640 1 -- 172.21.15.19:0/117375469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c153760 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.531+0000 7ff0319f5640 1 -- 172.21.15.19:0/117375469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02c073a00 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.532+0000 7ff013fff640 1 -- 172.21.15.19:0/117375469 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff024003b60 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.532+0000 7ff013fff640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff01806b670 0x7ff01806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.532+0000 7ff013fff640 1 -- 172.21.15.19:0/117375469 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff02408a1a0 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.533+0000 7ff02a7fc640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff01806b670 0x7ff01806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.533+0000 7ff02a7fc640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff01806b670 0x7ff01806db30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff020034000 tx=0x7ff02002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.536+0000 7ff013fff640 1 -- 172.21.15.19:0/117375469 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff024058720 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.717+0000 7ff0319f5640 1 -- 172.21.15.19:0/117375469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff02c0b4f60 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.717+0000 7ff013fff640 1 -- 172.21.15.19:0/117375469 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff02405c7c0 con 0x7ff02c0ccdd0 2024-04-25T09:53:31.719 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:31.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 -- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff01806b670 msgr2=0x7ff01806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff01806b670 0x7ff01806db30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff020034000 tx=0x7ff02002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:31.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 -- 172.21.15.19:0/117375469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02c0ccdd0 msgr2=0x7ff02c1f14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02c0ccdd0 0x7ff02c1f14a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff02400c570 tx=0x7ff02400ca30 comp rx=0 tx=0).stop 2024-04-25T09:53:31.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 -- 172.21.15.19:0/117375469 shutdown_connections 2024-04-25T09:53:31.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff01806b670 0x7ff01806db30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff02c15bb20 0x7ff02c153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff02c0cdcd0 0x7ff02c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 --2- 172.21.15.19:0/117375469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff02c0ccdd0 0x7ff02c1f14a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 -- 172.21.15.19:0/117375469 >> 172.21.15.19:0/117375469 conn(0x7ff02c0b2390 msgr2=0x7ff02c15ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 -- 172.21.15.19:0/117375469 shutdown_connections 2024-04-25T09:53:31.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.719+0000 7ff0319f5640 1 -- 172.21.15.19:0/117375469 wait complete. 2024-04-25T09:53:31.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.836+0000 7fb79ffd5640 1 Processor -- start 2024-04-25T09:53:31.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.836+0000 7fb79ffd5640 1 -- start start 2024-04-25T09:53:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79ffd5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb7980ced80 0x7fb7980d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79ffd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb798161710 0x7fb7980d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79ffd5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79815e000 con 0x7fb79815dc20 2024-04-25T09:53:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79ffd5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7980d1de0 con 0x7fb7980ced80 2024-04-25T09:53:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79ffd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb798161c60 con 0x7fb798161710 2024-04-25T09:53:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79d570640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb7980ced80 0x7fb7980d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79d570640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb7980ced80 0x7fb7980d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37400/0 (socket says 172.21.15.19:37400) 2024-04-25T09:53:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79d570640 1 -- 172.21.15.19:0/3222278662 learned_addr learned my addr 172.21.15.19:0/3222278662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79e572640 1 --2- 172.21.15.19:0/3222278662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb798161710 0x7fb7980d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79d570640 1 -- 172.21.15.19:0/3222278662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb798161710 msgr2=0x7fb7980d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79d570640 1 --2- 172.21.15.19:0/3222278662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb798161710 0x7fb7980d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79d570640 1 -- 172.21.15.19:0/3222278662 >> v1:172.21.15.112:6789/0 conn(0x7fb79815dc20 legacy=0x7fb7980ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.837+0000 7fb79d570640 1 -- 172.21.15.19:0/3222278662 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798161f40 con 0x7fb7980ced80 2024-04-25T09:53:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.838+0000 7fb79d570640 1 --2- 172.21.15.19:0/3222278662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7980ced80 0x7fb7980d22e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb788009870 tx=0x7fb78802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a111c822a890f8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.838+0000 7fb79cd6f640 1 -- 172.21.15.19:0/3222278662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788037070 con 0x7fb7980ced80 2024-04-25T09:53:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.838+0000 7fb79cd6f640 1 -- 172.21.15.19:0/3222278662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb788032440 con 0x7fb7980ced80 2024-04-25T09:53:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.838+0000 7fb79cd6f640 1 -- 172.21.15.19:0/3222278662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788032810 con 0x7fb7980ced80 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.838+0000 7fb79ffd5640 1 -- 172.21.15.19:0/3222278662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7980ced80 msgr2=0x7fb7980d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.839+0000 7fb79ffd5640 1 --2- 172.21.15.19:0/3222278662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7980ced80 0x7fb7980d22e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb788009870 tx=0x7fb78802a270 comp rx=0 tx=0).stop 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.839+0000 7fb79ffd5640 1 -- 172.21.15.19:0/3222278662 shutdown_connections 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.839+0000 7fb79ffd5640 1 --2- 172.21.15.19:0/3222278662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb798161710 0x7fb7980d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.839+0000 7fb79ffd5640 1 --2- 172.21.15.19:0/3222278662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7980ced80 0x7fb7980d22e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.839+0000 7fb79ffd5640 1 -- 172.21.15.19:0/3222278662 >> 172.21.15.19:0/3222278662 conn(0x7fb7980d0420 msgr2=0x7fb7980d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.839+0000 7fb79ffd5640 1 -- 172.21.15.19:0/3222278662 shutdown_connections 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.839+0000 7fb79ffd5640 1 -- 172.21.15.19:0/3222278662 wait complete. 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.840+0000 7fb79ffd5640 1 Processor -- start 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.840+0000 7fb79ffd5640 1 -- start start 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79ffd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7980ced80 0x7fb7980cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79ffd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79815dc20 0x7fb7980c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79ffd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb798161710 0x7fb7980b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79ffd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980b5e50 con 0x7fb7980ced80 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79ffd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798162500 con 0x7fb79815dc20 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79ffd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980b5fd0 con 0x7fb798161710 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79dd71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7980ced80 0x7fb7980cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79dd71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7980ced80 0x7fb7980cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37408/0 (socket says 172.21.15.19:37408) 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79dd71640 1 -- 172.21.15.19:0/2820619517 learned_addr learned my addr 172.21.15.19:0/2820619517 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79e572640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb798161710 0x7fb7980b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79d570640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79815dc20 0x7fb7980c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79dd71640 1 -- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb798161710 msgr2=0x7fb7980b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79dd71640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb798161710 0x7fb7980b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79dd71640 1 -- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79815dc20 msgr2=0x7fb7980c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79dd71640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79815dc20 0x7fb7980c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79dd71640 1 -- 172.21.15.19:0/2820619517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7980c8be0 con 0x7fb7980ced80 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79d570640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79815dc20 0x7fb7980c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79dd71640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7980ced80 0x7fb7980cd830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb79400c430 tx=0x7fb79400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb78e7fc640 1 -- 172.21.15.19:0/2820619517 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794017070 con 0x7fb7980ced80 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb78e7fc640 1 -- 172.21.15.19:0/2820619517 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7940052d0 con 0x7fb7980ced80 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb78e7fc640 1 -- 172.21.15.19:0/2820619517 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794003830 con 0x7fb7980ced80 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.841+0000 7fb79ffd5640 1 -- 172.21.15.19:0/2820619517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7980c8ec0 con 0x7fb7980ced80 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.842+0000 7fb79ffd5640 1 -- 172.21.15.19:0/2820619517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7980c9760 con 0x7fb7980ced80 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.842+0000 7fb79ffd5640 1 -- 172.21.15.19:0/2820619517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb798073a00 con 0x7fb7980ced80 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.843+0000 7fb78e7fc640 1 -- 172.21.15.19:0/2820619517 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb79401a070 con 0x7fb7980ced80 2024-04-25T09:53:31.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.843+0000 7fb78e7fc640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb77c06b4d0 0x7fb77c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.844+0000 7fb78e7fc640 1 -- 172.21.15.19:0/2820619517 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb794089810 con 0x7fb7980ced80 2024-04-25T09:53:31.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.844+0000 7fb79d570640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb77c06b4d0 0x7fb77c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.844+0000 7fb79d570640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb77c06b4d0 0x7fb77c06d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb788034000 tx=0x7fb78802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:31.848+0000 7fb78e7fc640 1 -- 172.21.15.19:0/2820619517 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb794057d00 con 0x7fb7980ced80 2024-04-25T09:53:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.027+0000 7fb79ffd5640 1 -- 172.21.15.19:0/2820619517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb7980d16b0 con 0x7fb7980ced80 2024-04-25T09:53:32.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.027+0000 7fb78e7fc640 1 -- 172.21.15.19:0/2820619517 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb794087e20 con 0x7fb7980ced80 2024-04-25T09:53:32.030 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:32.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 -- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb77c06b4d0 msgr2=0x7fb77c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb77c06b4d0 0x7fb77c06d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb788034000 tx=0x7fb78802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 -- 172.21.15.19:0/2820619517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7980ced80 msgr2=0x7fb7980cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7980ced80 0x7fb7980cd830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb79400c430 tx=0x7fb79400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 -- 172.21.15.19:0/2820619517 shutdown_connections 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb77c06b4d0 0x7fb77c06d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb798161710 0x7fb7980b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79815dc20 0x7fb7980c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 --2- 172.21.15.19:0/2820619517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7980ced80 0x7fb7980cd830 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 -- 172.21.15.19:0/2820619517 >> 172.21.15.19:0/2820619517 conn(0x7fb7980d0420 msgr2=0x7fb7980d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 -- 172.21.15.19:0/2820619517 shutdown_connections 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.029+0000 7fb79ffd5640 1 -- 172.21.15.19:0/2820619517 wait complete. 2024-04-25T09:53:32.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.141+0000 7feeb2f30640 1 Processor -- start 2024-04-25T09:53:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.141+0000 7feeb2f30640 1 -- start start 2024-04-25T09:53:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.141+0000 7feeb2f30640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feeac0ce920 0x7feeac0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.141+0000 7feeb2f30640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac144b30 con 0x7feeac144750 2024-04-25T09:53:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.141+0000 7feeb2f30640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac0712d0 con 0x7feeac06d290 2024-04-25T09:53:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.141+0000 7feeb2f30640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac0715b0 con 0x7feeac0ce920 2024-04-25T09:53:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.141+0000 7feeb0ccc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feeac144750 0x7feeac06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47568/0 (socket says 172.21.15.19:47568) 2024-04-25T09:53:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.142+0000 7feeb0ccc640 1 -- 172.21.15.19:0/312883740 learned_addr learned my addr 172.21.15.19:0/312883740 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.142+0000 7feeabfff640 1 --2- 172.21.15.19:0/312883740 >> v2:172.21.15.19:3300/0 conn(0x7feeac0ce920 0x7feeac0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.142+0000 7feeab7fe640 1 -- 172.21.15.19:0/312883740 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184505865 0 0) 0x7feeac144b30 con 0x7feeac144750 2024-04-25T09:53:32.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.142+0000 7feeab7fe640 1 -- 172.21.15.19:0/312883740 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee9c003520 con 0x7feeac144750 2024-04-25T09:53:32.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.142+0000 7feeabfff640 1 -- 172.21.15.19:0/312883740 >> v1:172.21.15.112:6789/0 conn(0x7feeac06d290 legacy=0x7feeac06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.142+0000 7feeabfff640 1 -- 172.21.15.19:0/312883740 >> v1:172.21.15.19:6789/0 conn(0x7feeac144750 legacy=0x7feeac06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.142+0000 7feeabfff640 1 -- 172.21.15.19:0/312883740 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac071890 con 0x7feeac0ce920 2024-04-25T09:53:32.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.143+0000 7feeabfff640 1 --2- 172.21.15.19:0/312883740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeac0ce920 0x7feeac0cedb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fee980097a0 tx=0x7fee9802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e4199d210e6df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.143+0000 7feeab7fe640 1 -- 172.21.15.19:0/312883740 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98037070 con 0x7feeac0ce920 2024-04-25T09:53:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.143+0000 7feeab7fe640 1 -- 172.21.15.19:0/312883740 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee98032440 con 0x7feeac0ce920 2024-04-25T09:53:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.143+0000 7feeab7fe640 1 -- 172.21.15.19:0/312883740 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee980327f0 con 0x7feeac0ce920 2024-04-25T09:53:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.143+0000 7feeb2f30640 1 -- 172.21.15.19:0/312883740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeac0ce920 msgr2=0x7feeac0cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.143+0000 7feeb2f30640 1 --2- 172.21.15.19:0/312883740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeac0ce920 0x7feeac0cedb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fee980097a0 tx=0x7fee9802a230 comp rx=0 tx=0).stop 2024-04-25T09:53:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.143+0000 7feeb2f30640 1 -- 172.21.15.19:0/312883740 shutdown_connections 2024-04-25T09:53:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.143+0000 7feeb2f30640 1 --2- 172.21.15.19:0/312883740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeac0ce920 0x7feeac0cedb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.143+0000 7feeb2f30640 1 -- 172.21.15.19:0/312883740 >> 172.21.15.19:0/312883740 conn(0x7feeac06abe0 msgr2=0x7feeac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.143+0000 7feeb2f30640 1 -- 172.21.15.19:0/312883740 shutdown_connections 2024-04-25T09:53:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.143+0000 7feeb2f30640 1 -- 172.21.15.19:0/312883740 wait complete. 2024-04-25T09:53:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.145+0000 7feeb2f30640 1 Processor -- start 2024-04-25T09:53:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.145+0000 7feeb2f30640 1 -- start start 2024-04-25T09:53:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.145+0000 7feeb2f30640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac06d290 0x7feeac130650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.145+0000 7feeb2f30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeac0ce920 0x7feeac141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.145+0000 7feeb2f30640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeac144750 0x7feeac142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.145+0000 7feeb2f30640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac06f020 con 0x7feeac0ce920 2024-04-25T09:53:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.145+0000 7feeb2f30640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac06ece0 con 0x7feeac06d290 2024-04-25T09:53:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.145+0000 7feeb2f30640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac145450 con 0x7feeac144750 2024-04-25T09:53:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.145+0000 7feeabfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeac0ce920 0x7feeac141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeabfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeac0ce920 0x7feeac141d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37432/0 (socket says 172.21.15.19:37432) 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeabfff640 1 -- 172.21.15.19:0/846667764 learned_addr learned my addr 172.21.15.19:0/846667764 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeb14cd640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeac144750 0x7feeac142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeb0ccc640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac06d290 0x7feeac130650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeabfff640 1 -- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeac144750 msgr2=0x7feeac142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeabfff640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeac144750 0x7feeac142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeabfff640 1 -- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac06d290 msgr2=0x7feeac130650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeabfff640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac06d290 0x7feeac130650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeabfff640 1 -- 172.21.15.19:0/846667764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0b0950 con 0x7feeac0ce920 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeb0ccc640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac06d290 0x7feeac130650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeabfff640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeac0ce920 0x7feeac141d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fee980097a0 tx=0x7fee980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feea97fa640 1 -- 172.21.15.19:0/846667764 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98037050 con 0x7feeac0ce920 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feea97fa640 1 -- 172.21.15.19:0/846667764 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee98038040 con 0x7feeac0ce920 2024-04-25T09:53:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeb2f30640 1 -- 172.21.15.19:0/846667764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac0b0c30 con 0x7feeac0ce920 2024-04-25T09:53:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feea97fa640 1 -- 172.21.15.19:0/846667764 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98004470 con 0x7feeac0ce920 2024-04-25T09:53:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.146+0000 7feeb2f30640 1 -- 172.21.15.19:0/846667764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac0b11f0 con 0x7feeac0ce920 2024-04-25T09:53:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.147+0000 7feeb2f30640 1 -- 172.21.15.19:0/846667764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeac073990 con 0x7feeac0ce920 2024-04-25T09:53:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.148+0000 7feea97fa640 1 -- 172.21.15.19:0/846667764 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee9803c6d0 con 0x7feeac0ce920 2024-04-25T09:53:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.148+0000 7feea97fa640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b4d0 0x7fee9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.148+0000 7feea97fa640 1 -- 172.21.15.19:0/846667764 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fee980aaf90 con 0x7feeac0ce920 2024-04-25T09:53:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.148+0000 7feeb0ccc640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b4d0 0x7fee9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.149+0000 7feeb0ccc640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b4d0 0x7fee9006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feea00011e0 tx=0x7feea0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.152+0000 7feea97fa640 1 -- 172.21.15.19:0/846667764 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee98079ad0 con 0x7feeac0ce920 2024-04-25T09:53:32.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.372+0000 7feeb2f30640 1 -- 172.21.15.19:0/846667764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feeac0d0510 con 0x7feeac0ce920 2024-04-25T09:53:32.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.373+0000 7feea97fa640 1 -- 172.21.15.19:0/846667764 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fee980793d0 con 0x7feeac0ce920 2024-04-25T09:53:32.375 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:32.375 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4000846111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":4000846111},{"type":"v1","addr":"172.21.15.112:6837","nonce":4000846111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1969525282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1969525282},{"type":"v1","addr":"172.21.15.19:6835","nonce":1969525282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:07.175350+0000","modified":"2024-04-25T09:53:23.345595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1080352556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1080352556},{"type":"v1","addr":"172.21.15.112:6836","nonce":1080352556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:53:32.375 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T09:53:32.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.374+0000 7feeb2f30640 1 -- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b4d0 msgr2=0x7fee9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.374+0000 7feeb2f30640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b4d0 0x7fee9006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feea00011e0 tx=0x7feea0009040 comp rx=0 tx=0).stop 2024-04-25T09:53:32.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.375+0000 7feeb2f30640 1 -- 172.21.15.19:0/846667764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeac0ce920 msgr2=0x7feeac141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.375+0000 7feeb2f30640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeac0ce920 0x7feeac141d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fee980097a0 tx=0x7fee980046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.375+0000 7feeb2f30640 1 -- 172.21.15.19:0/846667764 shutdown_connections 2024-04-25T09:53:32.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.375+0000 7feeb2f30640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b4d0 0x7fee9006d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.375+0000 7feeb2f30640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeac144750 0x7feeac142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.375+0000 7feeb2f30640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeac0ce920 0x7feeac141d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.375+0000 7feeb2f30640 1 --2- 172.21.15.19:0/846667764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeac06d290 0x7feeac130650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.375+0000 7feeb2f30640 1 -- 172.21.15.19:0/846667764 >> 172.21.15.19:0/846667764 conn(0x7feeac06abe0 msgr2=0x7feeac1439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.375+0000 7feeb2f30640 1 -- 172.21.15.19:0/846667764 shutdown_connections 2024-04-25T09:53:32.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.375+0000 7feeb2f30640 1 -- 172.21.15.19:0/846667764 wait complete. 2024-04-25T09:53:32.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.486+0000 7f4fe4cdd640 1 Processor -- start 2024-04-25T09:53:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.486+0000 7f4fe4cdd640 1 -- start start 2024-04-25T09:53:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fe4cdd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4fe00b5c30 0x7f4fe0168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fe4cdd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe00ceee0 con 0x7f4fe015db70 2024-04-25T09:53:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fe4cdd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe00d1d10 con 0x7f4fe00b6cb0 2024-04-25T09:53:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fe4cdd640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe00d3800 con 0x7f4fe00b5c30 2024-04-25T09:53:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fde59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4fe015db70 0x7f4fe0162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47578/0 (socket says 172.21.15.19:47578) 2024-04-25T09:53:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fde59c640 1 -- 172.21.15.19:0/689565684 learned_addr learned my addr 172.21.15.19:0/689565684 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fded9d640 1 --2- 172.21.15.19:0/689565684 >> v2:172.21.15.19:3300/0 conn(0x7f4fe00b5c30 0x7f4fe0168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fddd9b640 1 -- 172.21.15.19:0/689565684 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2387299296 0 0) 0x7f4fe00ceee0 con 0x7f4fe015db70 2024-04-25T09:53:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fddd9b640 1 -- 172.21.15.19:0/689565684 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fcc003520 con 0x7f4fe015db70 2024-04-25T09:53:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fded9d640 1 -- 172.21.15.19:0/689565684 >> v1:172.21.15.112:6789/0 conn(0x7f4fe00b6cb0 legacy=0x7f4fe00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fded9d640 1 -- 172.21.15.19:0/689565684 >> v1:172.21.15.19:6789/0 conn(0x7f4fe015db70 legacy=0x7f4fe0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.487+0000 7f4fded9d640 1 -- 172.21.15.19:0/689565684 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe00d1ff0 con 0x7f4fe00b5c30 2024-04-25T09:53:32.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.488+0000 7f4fded9d640 1 --2- 172.21.15.19:0/689565684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fe00b5c30 0x7f4fe0168ad0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00d3980 tx=0x7f4fd802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb605f065ab2d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.488+0000 7f4fddd9b640 1 -- 172.21.15.19:0/689565684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8007740 con 0x7f4fe00b5c30 2024-04-25T09:53:32.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.488+0000 7f4fddd9b640 1 -- 172.21.15.19:0/689565684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd80078e0 con 0x7f4fe00b5c30 2024-04-25T09:53:32.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.488+0000 7f4fddd9b640 1 -- 172.21.15.19:0/689565684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8007c90 con 0x7f4fe00b5c30 2024-04-25T09:53:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.489+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/689565684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fe00b5c30 msgr2=0x7f4fe0168ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.489+0000 7f4fe4cdd640 1 --2- 172.21.15.19:0/689565684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fe00b5c30 0x7f4fe0168ad0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00d3980 tx=0x7f4fd802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.489+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/689565684 shutdown_connections 2024-04-25T09:53:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.489+0000 7f4fe4cdd640 1 --2- 172.21.15.19:0/689565684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fe00b5c30 0x7f4fe0168ad0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.489+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/689565684 >> 172.21.15.19:0/689565684 conn(0x7f4fe00d0520 msgr2=0x7f4fe00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.489+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/689565684 shutdown_connections 2024-04-25T09:53:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.489+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/689565684 wait complete. 2024-04-25T09:53:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.490+0000 7f4fe4cdd640 1 Processor -- start 2024-04-25T09:53:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.490+0000 7f4fe4cdd640 1 -- start start 2024-04-25T09:53:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fe4cdd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe00b5c30 0x7f4fe00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fe4cdd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4fe00b6cb0 0x7f4fe0159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fe4cdd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fe015db70 0x7f4fe015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fe4cdd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe00ce530 con 0x7f4fe015db70 2024-04-25T09:53:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fe4cdd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe015df50 con 0x7f4fe00b5c30 2024-04-25T09:53:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fe4cdd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe00cf1c0 con 0x7f4fe00b6cb0 2024-04-25T09:53:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fded9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fe015db70 0x7f4fe015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fded9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fe015db70 0x7f4fe015a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37442/0 (socket says 172.21.15.19:37442) 2024-04-25T09:53:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fded9d640 1 -- 172.21.15.19:0/692021935 learned_addr learned my addr 172.21.15.19:0/692021935 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fde59c640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe00b5c30 0x7f4fe00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fd5d9b640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4fe00b6cb0 0x7f4fe0159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fded9d640 1 -- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4fe00b6cb0 msgr2=0x7f4fe0159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fded9d640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4fe00b6cb0 0x7f4fe0159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fded9d640 1 -- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe00b5c30 msgr2=0x7f4fe00b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fded9d640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe00b5c30 0x7f4fe00b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fded9d640 1 -- 172.21.15.19:0/692021935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe015fea0 con 0x7f4fe015db70 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.491+0000 7f4fded9d640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fe015db70 0x7f4fe015a2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4fd802ace0 tx=0x7f4fd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.492+0000 7f4fd77fe640 1 -- 172.21.15.19:0/692021935 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8009450 con 0x7f4fe015db70 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.492+0000 7f4fd77fe640 1 -- 172.21.15.19:0/692021935 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd8004240 con 0x7f4fe015db70 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.492+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/692021935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe0160180 con 0x7f4fe015db70 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.492+0000 7f4fd77fe640 1 -- 172.21.15.19:0/692021935 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8004630 con 0x7f4fe015db70 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.492+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/692021935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe0160740 con 0x7f4fe015db70 2024-04-25T09:53:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.493+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/692021935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe0073990 con 0x7f4fe015db70 2024-04-25T09:53:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.493+0000 7f4fd77fe640 1 -- 172.21.15.19:0/692021935 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fd8039040 con 0x7f4fe015db70 2024-04-25T09:53:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.494+0000 7f4fd77fe640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4fc006b4d0 0x7f4fc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.494+0000 7f4fd77fe640 1 -- 172.21.15.19:0/692021935 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4fd80ab500 con 0x7f4fe015db70 2024-04-25T09:53:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.494+0000 7f4fde59c640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4fc006b4d0 0x7f4fc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.494+0000 7f4fde59c640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4fc006b4d0 0x7f4fc006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00011e0 tx=0x7f4fd0007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.498+0000 7f4fd77fe640 1 -- 172.21.15.19:0/692021935 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fd8040030 con 0x7f4fe015db70 2024-04-25T09:53:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.678+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/692021935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fe00d1820 con 0x7f4fe015db70 2024-04-25T09:53:32.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.679+0000 7f4fd77fe640 1 -- 172.21.15.19:0/692021935 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f4fd8079ab0 con 0x7f4fe015db70 2024-04-25T09:53:32.681 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:32.681 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:32.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4fc006b4d0 msgr2=0x7f4fc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4fc006b4d0 0x7f4fc006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00011e0 tx=0x7f4fd0007d30 comp rx=0 tx=0).stop 2024-04-25T09:53:32.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/692021935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fe015db70 msgr2=0x7f4fe015a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fe015db70 0x7f4fe015a2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4fd802ace0 tx=0x7f4fd8004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:32.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/692021935 shutdown_connections 2024-04-25T09:53:32.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4fc006b4d0 0x7f4fc006d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fe015db70 0x7f4fe015a2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4fe00b6cb0 0x7f4fe0159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 --2- 172.21.15.19:0/692021935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe00b5c30 0x7f4fe00b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/692021935 >> 172.21.15.19:0/692021935 conn(0x7f4fe00d0520 msgr2=0x7f4fe00b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/692021935 shutdown_connections 2024-04-25T09:53:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.681+0000 7f4fe4cdd640 1 -- 172.21.15.19:0/692021935 wait complete. 2024-04-25T09:53:32.694 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:32.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.793+0000 7f15df77e640 1 Processor -- start 2024-04-25T09:53:32.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.793+0000 7f15df77e640 1 -- start start 2024-04-25T09:53:32.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15df77e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15d80b2cb0 0x7f15d80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15df77e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15d80d1020 0x7f15d80d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15df77e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d80b97a0 con 0x7f15d815dbe0 2024-04-25T09:53:32.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15df77e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d815dfc0 con 0x7f15d80b2cb0 2024-04-25T09:53:32.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15df77e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d80d1a00 con 0x7f15d80d1020 2024-04-25T09:53:32.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15dcd19640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15d80b2cb0 0x7f15d80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15dcd19640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15d80b2cb0 0x7f15d80d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37444/0 (socket says 172.21.15.19:37444) 2024-04-25T09:53:32.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15dcd19640 1 -- 172.21.15.19:0/1872217514 learned_addr learned my addr 172.21.15.19:0/1872217514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:32.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15ddd1b640 1 --2- 172.21.15.19:0/1872217514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15d80d1020 0x7f15d80d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15dcd19640 1 -- 172.21.15.19:0/1872217514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15d80d1020 msgr2=0x7f15d80d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:32.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15dcd19640 1 --2- 172.21.15.19:0/1872217514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15d80d1020 0x7f15d80d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15dcd19640 1 -- 172.21.15.19:0/1872217514 >> v1:172.21.15.112:6789/0 conn(0x7f15d815dbe0 legacy=0x7f15d80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.794+0000 7f15dcd19640 1 -- 172.21.15.19:0/1872217514 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d80d1c50 con 0x7f15d80b2cb0 2024-04-25T09:53:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.795+0000 7f15dcd19640 1 --2- 172.21.15.19:0/1872217514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d80b2cb0 0x7f15d80d2730 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009870 tx=0x7f15c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2a5f6afd15b246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.795+0000 7f15cffff640 1 -- 172.21.15.19:0/1872217514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8037070 con 0x7f15d80b2cb0 2024-04-25T09:53:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.795+0000 7f15cffff640 1 -- 172.21.15.19:0/1872217514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8032440 con 0x7f15d80b2cb0 2024-04-25T09:53:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.795+0000 7f15cffff640 1 -- 172.21.15.19:0/1872217514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8032810 con 0x7f15d80b2cb0 2024-04-25T09:53:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.796+0000 7f15df77e640 1 -- 172.21.15.19:0/1872217514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d80b2cb0 msgr2=0x7f15d80d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.796+0000 7f15df77e640 1 --2- 172.21.15.19:0/1872217514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d80b2cb0 0x7f15d80d2730 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009870 tx=0x7f15c802a270 comp rx=0 tx=0).stop 2024-04-25T09:53:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.796+0000 7f15df77e640 1 -- 172.21.15.19:0/1872217514 shutdown_connections 2024-04-25T09:53:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.796+0000 7f15df77e640 1 --2- 172.21.15.19:0/1872217514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15d80d1020 0x7f15d80d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.796+0000 7f15df77e640 1 --2- 172.21.15.19:0/1872217514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d80b2cb0 0x7f15d80d2730 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.796+0000 7f15df77e640 1 -- 172.21.15.19:0/1872217514 >> 172.21.15.19:0/1872217514 conn(0x7f15d8061a20 msgr2=0x7f15d8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.796+0000 7f15df77e640 1 -- 172.21.15.19:0/1872217514 shutdown_connections 2024-04-25T09:53:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.796+0000 7f15df77e640 1 -- 172.21.15.19:0/1872217514 wait complete. 2024-04-25T09:53:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.797+0000 7f15df77e640 1 Processor -- start 2024-04-25T09:53:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.797+0000 7f15df77e640 1 -- start start 2024-04-25T09:53:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15df77e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d80b2cb0 0x7f15d81f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15df77e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15d80d1020 0x7f15d815e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15df77e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d815dbe0 0x7f15d80b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15df77e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d80b92a0 con 0x7f15d815dbe0 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15df77e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d80d2210 con 0x7f15d80b2cb0 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15df77e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d80b9420 con 0x7f15d80d1020 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15ddd1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d815dbe0 0x7f15d80b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15ddd1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d815dbe0 0x7f15d80b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37448/0 (socket says 172.21.15.19:37448) 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15ddd1b640 1 -- 172.21.15.19:0/803037526 learned_addr learned my addr 172.21.15.19:0/803037526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15dcd19640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15d80d1020 0x7f15d815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15dd51a640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d80b2cb0 0x7f15d81f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15ddd1b640 1 -- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15d80d1020 msgr2=0x7f15d815e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15ddd1b640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15d80d1020 0x7f15d815e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15ddd1b640 1 -- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d80b2cb0 msgr2=0x7f15d81f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15ddd1b640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d80b2cb0 0x7f15d81f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15ddd1b640 1 -- 172.21.15.19:0/803037526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d815ec10 con 0x7f15d815dbe0 2024-04-25T09:53:32.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.798+0000 7f15ddd1b640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d815dbe0 0x7f15d80b5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15d000efc0 tx=0x7f15d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.799+0000 7f15cdffb640 1 -- 172.21.15.19:0/803037526 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d0018020 con 0x7f15d815dbe0 2024-04-25T09:53:32.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.799+0000 7f15cdffb640 1 -- 172.21.15.19:0/803037526 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d0014480 con 0x7f15d815dbe0 2024-04-25T09:53:32.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.799+0000 7f15df77e640 1 -- 172.21.15.19:0/803037526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d815eef0 con 0x7f15d815dbe0 2024-04-25T09:53:32.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.799+0000 7f15cdffb640 1 -- 172.21.15.19:0/803037526 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d0014870 con 0x7f15d815dbe0 2024-04-25T09:53:32.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.799+0000 7f15df77e640 1 -- 172.21.15.19:0/803037526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d815f4b0 con 0x7f15d815dbe0 2024-04-25T09:53:32.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.800+0000 7f15df77e640 1 -- 172.21.15.19:0/803037526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d8073990 con 0x7f15d815dbe0 2024-04-25T09:53:32.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.800+0000 7f15cdffb640 1 -- 172.21.15.19:0/803037526 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15d0005d10 con 0x7f15d815dbe0 2024-04-25T09:53:32.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.801+0000 7f15cdffb640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15bc06b4d0 0x7f15bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.801+0000 7f15cdffb640 1 -- 172.21.15.19:0/803037526 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f15d008b280 con 0x7f15d815dbe0 2024-04-25T09:53:32.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.801+0000 7f15dd51a640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15bc06b4d0 0x7f15bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.802+0000 7f15dd51a640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15bc06b4d0 0x7f15bc06d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f15d80b4b60 tx=0x7f15d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:32.805+0000 7f15cdffb640 1 -- 172.21.15.19:0/803037526 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15d0022030 con 0x7f15d815dbe0 2024-04-25T09:53:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.023+0000 7f15df77e640 1 -- 172.21.15.19:0/803037526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15d80d4790 con 0x7f15d815dbe0 2024-04-25T09:53:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.024+0000 7f15cdffb640 1 -- 172.21.15.19:0/803037526 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f15d0010040 con 0x7f15d815dbe0 2024-04-25T09:53:33.026 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:33.026 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4000846111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":4000846111},{"type":"v1","addr":"172.21.15.112:6837","nonce":4000846111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1969525282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1969525282},{"type":"v1","addr":"172.21.15.19:6835","nonce":1969525282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:07.175350+0000","modified":"2024-04-25T09:53:23.345595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1080352556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1080352556},{"type":"v1","addr":"172.21.15.112:6836","nonce":1080352556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:53:33.027 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T09:53:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 -- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15bc06b4d0 msgr2=0x7f15bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15bc06b4d0 0x7f15bc06d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f15d80b4b60 tx=0x7f15d4009040 comp rx=0 tx=0).stop 2024-04-25T09:53:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 -- 172.21.15.19:0/803037526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d815dbe0 msgr2=0x7f15d80b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d815dbe0 0x7f15d80b5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15d000efc0 tx=0x7f15d000c490 comp rx=0 tx=0).stop 2024-04-25T09:53:33.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 -- 172.21.15.19:0/803037526 shutdown_connections 2024-04-25T09:53:33.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15bc06b4d0 0x7f15bc06d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d815dbe0 0x7f15d80b5150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15d80d1020 0x7f15d815e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 --2- 172.21.15.19:0/803037526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d80b2cb0 0x7f15d81f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 -- 172.21.15.19:0/803037526 >> 172.21.15.19:0/803037526 conn(0x7f15d8061a20 msgr2=0x7f15d80d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 -- 172.21.15.19:0/803037526 shutdown_connections 2024-04-25T09:53:33.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.026+0000 7f15df77e640 1 -- 172.21.15.19:0/803037526 wait complete. 2024-04-25T09:53:33.442 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:53:07.175350+0000', 'modified': '2024-04-25T09:53:23.345595+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4300': {'gid': 4300, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6836/1080352556', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 1080352556}, {'type': 'v1', 'addr': '172.21.15.112:6836', 'nonce': 1080352556}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:53:33.443 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:53:33.443 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:53:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.547+0000 7f46df69a640 1 Processor -- start 2024-04-25T09:53:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.547+0000 7f46df69a640 1 -- start start 2024-04-25T09:53:33.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46df69a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815dc20 0x7f46d80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46df69a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f46d80d32f0 0x7f46d80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46df69a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d815e000 con 0x7f46d8161710 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46df69a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d80d1de0 con 0x7f46d80d32f0 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46df69a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d8161c60 con 0x7f46d815dc20 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46ddc37640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f46d80d32f0 0x7f46d80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46dcc35640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f46d8161710 0x7f46d806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47588/0 (socket says 172.21.15.19:47588) 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46dcc35640 1 -- 172.21.15.19:0/767748368 learned_addr learned my addr 172.21.15.19:0/767748368 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46dd436640 1 --2- 172.21.15.19:0/767748368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815dc20 0x7f46d80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46ddc37640 1 -- 172.21.15.19:0/767748368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815dc20 msgr2=0x7f46d80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46ddc37640 1 --2- 172.21.15.19:0/767748368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815dc20 0x7f46d80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46ddc37640 1 -- 172.21.15.19:0/767748368 >> v1:172.21.15.19:6789/0 conn(0x7f46d8161710 legacy=0x7f46d806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46cffff640 1 -- 172.21.15.19:0/767748368 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330464124 0 0) 0x7f46d815e000 con 0x7f46d8161710 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.548+0000 7f46ddc37640 1 -- 172.21.15.19:0/767748368 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d8161f40 con 0x7f46d80d32f0 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.549+0000 7f46ddc37640 1 --2- 172.21.15.19:0/767748368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d80d32f0 0x7f46d80cebb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009d90 tx=0x7f46d002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5484d0b4656a380 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.549+0000 7f46cffff640 1 -- 172.21.15.19:0/767748368 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d0007740 con 0x7f46d80d32f0 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.549+0000 7f46cffff640 1 -- 172.21.15.19:0/767748368 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d00078e0 con 0x7f46d80d32f0 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.549+0000 7f46cffff640 1 -- 172.21.15.19:0/767748368 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d0007cb0 con 0x7f46d80d32f0 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.550+0000 7f46df69a640 1 -- 172.21.15.19:0/767748368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d80d32f0 msgr2=0x7f46d80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.550+0000 7f46df69a640 1 --2- 172.21.15.19:0/767748368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d80d32f0 0x7f46d80cebb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009d90 tx=0x7f46d002a660 comp rx=0 tx=0).stop 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.550+0000 7f46df69a640 1 -- 172.21.15.19:0/767748368 shutdown_connections 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.550+0000 7f46df69a640 1 --2- 172.21.15.19:0/767748368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d80d32f0 0x7f46d80cebb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.550+0000 7f46df69a640 1 --2- 172.21.15.19:0/767748368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815dc20 0x7f46d80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.550+0000 7f46df69a640 1 -- 172.21.15.19:0/767748368 >> 172.21.15.19:0/767748368 conn(0x7f46d80d0420 msgr2=0x7f46d80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.550+0000 7f46df69a640 1 -- 172.21.15.19:0/767748368 shutdown_connections 2024-04-25T09:53:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.550+0000 7f46df69a640 1 -- 172.21.15.19:0/767748368 wait complete. 2024-04-25T09:53:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.551+0000 7f46df69a640 1 Processor -- start 2024-04-25T09:53:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.551+0000 7f46df69a640 1 -- start start 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46df69a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46d80d32f0 0x7f46d80d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46df69a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815dc20 0x7f46d80acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46df69a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8161710 0x7f46d80ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46df69a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80b5e50 con 0x7f46d8161710 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46df69a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d8162500 con 0x7f46d80d32f0 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46df69a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80b5fd0 con 0x7f46d815dc20 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46ddc37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8161710 0x7f46d80ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46ddc37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8161710 0x7f46d80ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37474/0 (socket says 172.21.15.19:37474) 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46ddc37640 1 -- 172.21.15.19:0/310272204 learned_addr learned my addr 172.21.15.19:0/310272204 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46dcc35640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815dc20 0x7f46d80acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46dd436640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46d80d32f0 0x7f46d80d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46ddc37640 1 -- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815dc20 msgr2=0x7f46d80acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46ddc37640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815dc20 0x7f46d80acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46ddc37640 1 -- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46d80d32f0 msgr2=0x7f46d80d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46ddc37640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46d80d32f0 0x7f46d80d3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46ddc37640 1 -- 172.21.15.19:0/310272204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d80ab0c0 con 0x7f46d8161710 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.552+0000 7f46dcc35640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815dc20 0x7f46d80acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.553+0000 7f46ddc37640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8161710 0x7f46d80ad1d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009d90 tx=0x7f46d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.553+0000 7f46cdffb640 1 -- 172.21.15.19:0/310272204 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d0040070 con 0x7f46d8161710 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.553+0000 7f46cdffb640 1 -- 172.21.15.19:0/310272204 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d0031c40 con 0x7f46d8161710 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.553+0000 7f46df69a640 1 -- 172.21.15.19:0/310272204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d80ab3a0 con 0x7f46d8161710 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.553+0000 7f46cdffb640 1 -- 172.21.15.19:0/310272204 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d0009450 con 0x7f46d8161710 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.553+0000 7f46df69a640 1 -- 172.21.15.19:0/310272204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d80ab960 con 0x7f46d8161710 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.554+0000 7f46cdffb640 1 -- 172.21.15.19:0/310272204 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46d0039040 con 0x7f46d8161710 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.555+0000 7f46cdffb640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b4d0 0x7f46bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.555+0000 7f46cdffb640 1 -- 172.21.15.19:0/310272204 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f46d00ab2e0 con 0x7f46d8161710 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.555+0000 7f46dd436640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b4d0 0x7f46bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.556+0000 7f46dd436640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b4d0 0x7f46bc06d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46d4006fd0 tx=0x7f46d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.556+0000 7f46e2397740 1 Processor -- start 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.556+0000 7f46e2397740 1 -- start start 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.556+0000 7f46e2397740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.556+0000 7f46e2397740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648e6ad0 0x55ed648e7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.556+0000 7f46e2397740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed6489ef10 0x55ed648e5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.556+0000 7f46e2397740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64801a10 con 0x55ed648c9230 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.556+0000 7f46e2397740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed647aab30 con 0x55ed6489ef10 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.556+0000 7f46e2397740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed647fdc50 con 0x55ed648e6ad0 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46dd436640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46dd436640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37476/0 (socket says 172.21.15.19:37476) 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46dd436640 1 -- 172.21.15.19:0/183264349 learned_addr learned my addr 172.21.15.19:0/183264349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46dcc35640 1 --2- 172.21.15.19:0/183264349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648e6ad0 0x55ed648e7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46ddc37640 1 --2- 172.21.15.19:0/183264349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed6489ef10 0x55ed648e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46dd436640 1 -- 172.21.15.19:0/183264349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648e6ad0 msgr2=0x55ed648e7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46dd436640 1 --2- 172.21.15.19:0/183264349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648e6ad0 0x55ed648e7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46dd436640 1 -- 172.21.15.19:0/183264349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed6489ef10 msgr2=0x55ed648e5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46dd436640 1 --2- 172.21.15.19:0/183264349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed6489ef10 0x55ed648e5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46dd436640 1 -- 172.21.15.19:0/183264349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed647bfe10 con 0x55ed648c9230 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46dd436640 1 --2- 172.21.15.19:0/183264349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648c9610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46d400fa20 tx=0x7f46d4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6983564bb52148cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46c2ffd640 1 -- 172.21.15.19:0/183264349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d401b050 con 0x55ed648c9230 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46c2ffd640 1 -- 172.21.15.19:0/183264349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d4025070 con 0x55ed648c9230 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.557+0000 7f46c2ffd640 1 -- 172.21.15.19:0/183264349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4014e00 con 0x55ed648c9230 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.558+0000 7f46e2397740 1 -- 172.21.15.19:0/183264349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 msgr2=0x55ed648c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.558+0000 7f46e2397740 1 --2- 172.21.15.19:0/183264349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648c9610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46d400fa20 tx=0x7f46d4012000 comp rx=0 tx=0).stop 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.558+0000 7f46e2397740 1 -- 172.21.15.19:0/183264349 shutdown_connections 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.558+0000 7f46e2397740 1 --2- 172.21.15.19:0/183264349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed6489ef10 0x55ed648e5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.558+0000 7f46e2397740 1 --2- 172.21.15.19:0/183264349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648e6ad0 0x55ed648e7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.558+0000 7f46e2397740 1 --2- 172.21.15.19:0/183264349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648c9610 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.558+0000 7f46e2397740 1 -- 172.21.15.19:0/183264349 >> 172.21.15.19:0/183264349 conn(0x55ed6489b840 msgr2=0x55ed648e4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.558+0000 7f46e2397740 1 -- 172.21.15.19:0/183264349 shutdown_connections 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.558+0000 7f46e2397740 1 -- 172.21.15.19:0/183264349 wait complete. 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.559+0000 7f46e2397740 1 -- start start 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.559+0000 7f46e2397740 1 Processor -- start 2024-04-25T09:53:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46e2397740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46e2397740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648e6ad0 0x55ed648b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46e2397740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed648b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46e2397740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6489fcb0 con 0x55ed648c9230 2024-04-25T09:53:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46e2397740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed647be0c0 con 0x55ed648b2600 2024-04-25T09:53:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46e2397740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed64899770 con 0x55ed648e6ad0 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46e2397740 10 client.0 _ll_register_callbacks cb 0x55ed647b1ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46dd436640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46dd436640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37490/0 (socket says 172.21.15.19:37490) 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46dd436640 1 -- 172.21.15.19:0/2017294782 learned_addr learned my addr 172.21.15.19:0/2017294782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46ddc37640 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed648b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46dcc35640 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648e6ad0 0x55ed648b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46dd436640 1 -- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648e6ad0 msgr2=0x55ed648b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46dd436640 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648e6ad0 0x55ed648b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46dd436640 1 -- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 msgr2=0x55ed648b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46dd436640 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed648b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46dd436640 1 -- 172.21.15.19:0/2017294782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed645162d0 con 0x55ed648c9230 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46dd436640 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648f3ce0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f46d4014830 tx=0x7f46d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb083a4aa638bc41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46b7fff640 1 -- 172.21.15.19:0/2017294782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4020850 con 0x55ed648c9230 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46b7fff640 1 -- 172.21.15.19:0/2017294782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f46d40209f0 con 0x55ed648c9230 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.560+0000 7f46e2397740 1 -- 172.21.15.19:0/2017294782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46d4025070 con 0x55ed648c9230 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46b7fff640 1 -- 172.21.15.19:0/2017294782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46d403c050 con 0x55ed648c9230 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46e2397740 10 client.4800 fetch_fsmap learned FSMap version 9 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46e2397740 1 -- 172.21.15.19:0/2017294782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55ed645e06b0 con 0x55ed648c9230 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46b7fff640 1 -- 172.21.15.19:0/2017294782 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d4025070 con 0x55ed648c9230 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46b7fff640 1 -- 172.21.15.19:0/2017294782 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f46d403f020 con 0x55ed648c9230 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46e2397740 10 client.4800 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46e2397740 10 client.4800 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46e2397740 20 client.4800 populate_metadata read hostname 'smithi019' 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46e2397740 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55ed648e2950 0x55ed648e2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46e2397740 4 client.4800 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46e2397740 1 -- 172.21.15.19:0/2017294782 --> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed64860920 con 0x55ed648e2950 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46dcc35640 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55ed648e2950 0x55ed648e2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46dcc35640 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55ed648e2950 0x55ed648e2d30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36fbc400ba3efbf server_cookie=8f68a239ff44f169 in_seq=0 out_seq=0 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.561+0000 7f46b7fff640 10 client.4800 ms_handle_connect on v2:172.21.15.112:6834/1080352556 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46b7fff640 1 -- 172.21.15.19:0/2017294782 <== mds.0 v2:172.21.15.112:6834/1080352556 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed64860920 con 0x55ed648e2950 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46b7fff640 10 client.4800 handle_command_reply: tid=0 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 1 client.4800 shutdown 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 1 -- 172.21.15.19:0/2017294782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 msgr2=0x55ed648f3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648f3ce0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f46d4014830 tx=0x7f46d4004730 comp rx=0 tx=0).stop 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 1 -- 172.21.15.19:0/2017294782 shutdown_connections 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55ed648e2950 0x55ed648e2d30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed648b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648e6ad0 0x55ed648b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 1 --2- 172.21.15.19:0/2017294782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648c9230 0x55ed648f3ce0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46b7fff640 0 client.4800 ms_handle_reset on v2:172.21.15.112:6834/1080352556 2024-04-25T09:53:33.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 1 -- 172.21.15.19:0/2017294782 >> 172.21.15.19:0/2017294782 conn(0x55ed6489ef10 msgr2=0x55ed648e75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 1 -- 172.21.15.19:0/2017294782 shutdown_connections 2024-04-25T09:53:33.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 1 -- 172.21.15.19:0/2017294782 wait complete. 2024-04-25T09:53:33.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.562+0000 7f46e2397740 20 client.4800 trim_cache size 0 max 16384 2024-04-25T09:53:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.578+0000 7f46e2397740 1 Processor -- start 2024-04-25T09:53:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.578+0000 7f46e2397740 1 -- start start 2024-04-25T09:53:33.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46e2397740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed6489ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46e2397740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648c9230 0x55ed648b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46e2397740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46e2397740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d82004a0 con 0x55ed648e6ad0 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46e2397740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d81ff900 con 0x55ed648b2600 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46e2397740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d81ffed0 con 0x55ed648c9230 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46ddc37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46ddc37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37498/0 (socket says 172.21.15.19:37498) 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46ddc37640 1 -- 172.21.15.19:0/3062810029 learned_addr learned my addr 172.21.15.19:0/3062810029 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46dcc35640 1 --2- 172.21.15.19:0/3062810029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648c9230 0x55ed648b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46dd436640 1 --2- 172.21.15.19:0/3062810029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed6489ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46ddc37640 1 -- 172.21.15.19:0/3062810029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648c9230 msgr2=0x55ed648b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46ddc37640 1 --2- 172.21.15.19:0/3062810029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648c9230 0x55ed648b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46ddc37640 1 -- 172.21.15.19:0/3062810029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 msgr2=0x55ed6489ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46ddc37640 1 --2- 172.21.15.19:0/3062810029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed6489ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46ddc37640 1 -- 172.21.15.19:0/3062810029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed64894140 con 0x55ed648e6ad0 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.579+0000 7f46ddc37640 1 --2- 172.21.15.19:0/3062810029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648b5170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f46d002a6a0 tx=0x7f46d0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78ffee7ff13d4aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46c27fc640 1 -- 172.21.15.19:0/3062810029 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d00a6030 con 0x55ed648e6ad0 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46c27fc640 1 -- 172.21.15.19:0/3062810029 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d007d900 con 0x55ed648e6ad0 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46c27fc640 1 -- 172.21.15.19:0/3062810029 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d0049040 con 0x55ed648e6ad0 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46e2397740 1 -- 172.21.15.19:0/3062810029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 msgr2=0x55ed648b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46e2397740 1 --2- 172.21.15.19:0/3062810029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648b5170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f46d002a6a0 tx=0x7f46d0009750 comp rx=0 tx=0).stop 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46e2397740 1 -- 172.21.15.19:0/3062810029 shutdown_connections 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46e2397740 1 --2- 172.21.15.19:0/3062810029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648b5170 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46e2397740 1 --2- 172.21.15.19:0/3062810029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648c9230 0x55ed648b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46e2397740 1 --2- 172.21.15.19:0/3062810029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed6489ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46e2397740 1 -- 172.21.15.19:0/3062810029 >> 172.21.15.19:0/3062810029 conn(0x55ed6489ef10 msgr2=0x55ed6489f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46e2397740 1 -- 172.21.15.19:0/3062810029 shutdown_connections 2024-04-25T09:53:33.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.580+0000 7f46e2397740 1 -- 172.21.15.19:0/3062810029 wait complete. 2024-04-25T09:53:33.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.581+0000 7f46e2397740 1 -- start start 2024-04-25T09:53:33.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.581+0000 7f46e2397740 1 Processor -- start 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46e2397740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed64938130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46e2397740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648c9230 0x55ed648b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46e2397740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46e2397740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed648b8a50 con 0x55ed648e6ad0 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46e2397740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed648b2b20 con 0x55ed648b2600 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46e2397740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed648c6d20 con 0x55ed648c9230 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46e2397740 10 client.0 _ll_register_callbacks cb 0x55ed64899c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46ddc37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46ddc37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648e3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37512/0 (socket says 172.21.15.19:37512) 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46ddc37640 1 -- 172.21.15.19:0/3259682828 learned_addr learned my addr 172.21.15.19:0/3259682828 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46dcc35640 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648c9230 0x55ed648b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46dd436640 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed64938130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46ddc37640 1 -- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648c9230 msgr2=0x55ed648b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46ddc37640 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648c9230 0x55ed648b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46ddc37640 1 -- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 msgr2=0x55ed64938130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46ddc37640 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed64938130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46ddc37640 1 -- 172.21.15.19:0/3259682828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed648feb60 con 0x55ed648e6ad0 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46ddc37640 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648e3e30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f46d002ac50 tx=0x7f46d0032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c56061b35ff3bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46b7fff640 1 -- 172.21.15.19:0/3259682828 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d00a6030 con 0x55ed648e6ad0 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46b7fff640 1 -- 172.21.15.19:0/3259682828 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f46d0045df0 con 0x55ed648e6ad0 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.582+0000 7f46e2397740 1 -- 172.21.15.19:0/3259682828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46d007d900 con 0x55ed648e6ad0 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46b7fff640 1 -- 172.21.15.19:0/3259682828 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46d0049040 con 0x55ed648e6ad0 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46e2397740 10 client.4806 fetch_fsmap learned FSMap version 9 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46e2397740 1 -- 172.21.15.19:0/3259682828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55ed648ff340 con 0x55ed648e6ad0 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46b7fff640 1 -- 172.21.15.19:0/3259682828 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d0098020 con 0x55ed648e6ad0 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46b7fff640 1 -- 172.21.15.19:0/3259682828 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f46d004c060 con 0x55ed648e6ad0 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46e2397740 10 client.4806 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46e2397740 10 client.4806 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46e2397740 20 client.4806 populate_metadata read hostname 'smithi019' 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46e2397740 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55ed64899030 0x55ed648c5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46e2397740 4 client.4806 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46e2397740 1 -- 172.21.15.19:0/3259682828 --> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ed648dc340 con 0x55ed64899030 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46dd436640 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55ed64899030 0x55ed648c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46dd436640 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55ed64899030 0x55ed648c5680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9519ab2d3c31c06 server_cookie=13059b0f5df81625 in_seq=0 out_seq=0 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.583+0000 7f46b7fff640 10 client.4806 ms_handle_connect on v2:172.21.15.112:6834/1080352556 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46b7fff640 1 -- 172.21.15.19:0/3259682828 <== mds.0 v2:172.21.15.112:6834/1080352556 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55ed648dc340 con 0x55ed64899030 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46b7fff640 10 client.4806 handle_command_reply: tid=0 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 1 client.4806 shutdown 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 1 -- 172.21.15.19:0/3259682828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 msgr2=0x55ed648e3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648e3e30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f46d002ac50 tx=0x7f46d0032f90 comp rx=0 tx=0).stop 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 1 -- 172.21.15.19:0/3259682828 shutdown_connections 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:6834/1080352556,v1:172.21.15.112:6836/1080352556] conn(0x55ed64899030 0x55ed648c5680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ed648e6ad0 0x55ed648e3e30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ed648c9230 0x55ed648b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46b7fff640 0 client.4806 ms_handle_reset on v2:172.21.15.112:6834/1080352556 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 1 --2- 172.21.15.19:0/3259682828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed648b2600 0x55ed64938130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 1 -- 172.21.15.19:0/3259682828 >> 172.21.15.19:0/3259682828 conn(0x55ed6489ef10 msgr2=0x55ed648c4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 1 -- 172.21.15.19:0/3259682828 shutdown_connections 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 1 -- 172.21.15.19:0/3259682828 wait complete. 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46e2397740 20 client.4806 trim_cache size 0 max 16384 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 4300, 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 25.801150513, 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 9, 2024-04-25T09:53:33.594 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 24, 2024-04-25T09:53:33.594 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T09:53:33.594 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 33.028155132000002 2024-04-25T09:53:33.594 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:53:33.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46df69a640 1 -- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b4d0 msgr2=0x7f46bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46df69a640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b4d0 0x7f46bc06d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46d4006fd0 tx=0x7f46d4008040 comp rx=0 tx=0).stop 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46df69a640 1 -- 172.21.15.19:0/310272204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8161710 msgr2=0x7f46d80ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.584+0000 7f46df69a640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8161710 0x7f46d80ad1d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009d90 tx=0x7f46d0004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.585+0000 7f46df69a640 1 -- 172.21.15.19:0/310272204 shutdown_connections 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.585+0000 7f46df69a640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b4d0 0x7f46bc06d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.585+0000 7f46df69a640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8161710 0x7f46d80ad1d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.585+0000 7f46df69a640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815dc20 0x7f46d80acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.585+0000 7f46df69a640 1 --2- 172.21.15.19:0/310272204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46d80d32f0 0x7f46d80d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.585+0000 7f46df69a640 1 -- 172.21.15.19:0/310272204 >> 172.21.15.19:0/310272204 conn(0x7f46d80d0420 msgr2=0x7f46d80b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.585+0000 7f46df69a640 1 -- 172.21.15.19:0/310272204 shutdown_connections 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:33.585+0000 7f46df69a640 1 -- 172.21.15.19:0/310272204 wait complete. 2024-04-25T09:53:33.597 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T09:53:33.610 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T09:53:33.611 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T09:53:33.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:33.611 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:53:33.626 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T09:53:33.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:33.627 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:33.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:33.691 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T09:53:33.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:33.800 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:53:33.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:33.815 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T09:53:33.881 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-04-25T09:53:33.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:53:33.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:33.883 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T09:53:33.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:33.902 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T09:53:33.972 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-04-25T09:53:33.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:53:33.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:33.974 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T09:53:34.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:34.032 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T09:53:34.100 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-04-25T09:53:34.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:53:34.102 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-25T09:53:34.102 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:53:34.102 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:53:34.102 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:53:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.202+0000 7f43efb13640 1 Processor -- start 2024-04-25T09:53:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.202+0000 7f43efb13640 1 -- start start 2024-04-25T09:53:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43efb13640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f43e8164970 0x7f43e8166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43efb13640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e8144b30 con 0x7f43e8144750 2024-04-25T09:53:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43efb13640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e80712d0 con 0x7f43e80ce920 2024-04-25T09:53:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43efb13640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e80715b0 con 0x7f43e8164970 2024-04-25T09:53:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43ed8af640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f43e8144750 0x7f43e806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47594/0 (socket says 172.21.15.19:47594) 2024-04-25T09:53:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43ed8af640 1 -- 172.21.15.19:0/321683671 learned_addr learned my addr 172.21.15.19:0/321683671 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43ee0b0640 1 --2- 172.21.15.19:0/321683671 >> v2:172.21.15.112:3300/0 conn(0x7f43e8164970 0x7f43e8166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43ec8ad640 1 -- 172.21.15.19:0/321683671 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765377767 0 0) 0x7f43e8144b30 con 0x7f43e8144750 2024-04-25T09:53:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43ec8ad640 1 -- 172.21.15.19:0/321683671 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d4003520 con 0x7f43e8144750 2024-04-25T09:53:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43ec8ad640 1 -- 172.21.15.19:0/321683671 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334678591 0 0) 0x7f43e80712d0 con 0x7f43e80ce920 2024-04-25T09:53:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43ec8ad640 1 -- 172.21.15.19:0/321683671 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43e8144b30 con 0x7f43e80ce920 2024-04-25T09:53:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43ee0b0640 1 -- 172.21.15.19:0/321683671 >> v1:172.21.15.112:6789/0 conn(0x7f43e80ce920 legacy=0x7f43e80cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43ee0b0640 1 -- 172.21.15.19:0/321683671 >> v1:172.21.15.19:6789/0 conn(0x7f43e8144750 legacy=0x7f43e806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.203+0000 7f43ee0b0640 1 -- 172.21.15.19:0/321683671 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e8071890 con 0x7f43e8164970 2024-04-25T09:53:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.204+0000 7f43ee0b0640 1 --2- 172.21.15.19:0/321683671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e8164970 0x7f43e8166d60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009d90 tx=0x7f43e002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5468ab2ff6077e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.204+0000 7f43ec8ad640 1 -- 172.21.15.19:0/321683671 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0007740 con 0x7f43e8164970 2024-04-25T09:53:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.204+0000 7f43ec8ad640 1 -- 172.21.15.19:0/321683671 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e00078e0 con 0x7f43e8164970 2024-04-25T09:53:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.204+0000 7f43ec8ad640 1 -- 172.21.15.19:0/321683671 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0007c90 con 0x7f43e8164970 2024-04-25T09:53:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.205+0000 7f43efb13640 1 -- 172.21.15.19:0/321683671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e8164970 msgr2=0x7f43e8166d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.205+0000 7f43efb13640 1 --2- 172.21.15.19:0/321683671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e8164970 0x7f43e8166d60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009d90 tx=0x7f43e002a660 comp rx=0 tx=0).stop 2024-04-25T09:53:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.205+0000 7f43efb13640 1 -- 172.21.15.19:0/321683671 shutdown_connections 2024-04-25T09:53:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.205+0000 7f43efb13640 1 --2- 172.21.15.19:0/321683671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e8164970 0x7f43e8166d60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:34.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.205+0000 7f43efb13640 1 -- 172.21.15.19:0/321683671 >> 172.21.15.19:0/321683671 conn(0x7f43e806abe0 msgr2=0x7f43e806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:34.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.205+0000 7f43efb13640 1 -- 172.21.15.19:0/321683671 shutdown_connections 2024-04-25T09:53:34.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.205+0000 7f43efb13640 1 -- 172.21.15.19:0/321683671 wait complete. 2024-04-25T09:53:34.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.206+0000 7f43efb13640 1 Processor -- start 2024-04-25T09:53:34.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.206+0000 7f43efb13640 1 -- start start 2024-04-25T09:53:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43efb13640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e80ce920 0x7f43e813d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43efb13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e8144750 0x7f43e813dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43efb13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e8164970 0x7f43e8163990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43efb13640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e8145450 con 0x7f43e8164970 2024-04-25T09:53:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43efb13640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e806f020 con 0x7f43e8144750 2024-04-25T09:53:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43efb13640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81455d0 con 0x7f43e80ce920 2024-04-25T09:53:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43ee0b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e8164970 0x7f43e8163990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43ee0b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e8164970 0x7f43e8163990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37516/0 (socket says 172.21.15.19:37516) 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43ee0b0640 1 -- 172.21.15.19:0/2775325313 learned_addr learned my addr 172.21.15.19:0/2775325313 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43ed8af640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e80ce920 0x7f43e813d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43ed0ae640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e8144750 0x7f43e813dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43ee0b0640 1 -- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e80ce920 msgr2=0x7f43e813d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43ee0b0640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e80ce920 0x7f43e813d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43ee0b0640 1 -- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e8144750 msgr2=0x7f43e813dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43ee0b0640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e8144750 0x7f43e813dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.207+0000 7f43ee0b0640 1 -- 172.21.15.19:0/2775325313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e813e220 con 0x7f43e8164970 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.208+0000 7f43ee0b0640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e8164970 0x7f43e8163990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009eb0 tx=0x7f43e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.208+0000 7f43de7fc640 1 -- 172.21.15.19:0/2775325313 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0009450 con 0x7f43e8164970 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.208+0000 7f43de7fc640 1 -- 172.21.15.19:0/2775325313 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e0004260 con 0x7f43e8164970 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.208+0000 7f43efb13640 1 -- 172.21.15.19:0/2775325313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e815c130 con 0x7f43e8164970 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.208+0000 7f43de7fc640 1 -- 172.21.15.19:0/2775325313 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0004650 con 0x7f43e8164970 2024-04-25T09:53:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.208+0000 7f43efb13640 1 -- 172.21.15.19:0/2775325313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e815c6f0 con 0x7f43e8164970 2024-04-25T09:53:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.209+0000 7f43efb13640 1 -- 172.21.15.19:0/2775325313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e8073990 con 0x7f43e8164970 2024-04-25T09:53:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.209+0000 7f43de7fc640 1 -- 172.21.15.19:0/2775325313 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43e0039040 con 0x7f43e8164970 2024-04-25T09:53:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.210+0000 7f43de7fc640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43cc06b500 0x7f43cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.210+0000 7f43de7fc640 1 -- 172.21.15.19:0/2775325313 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f43e00ab950 con 0x7f43e8164970 2024-04-25T09:53:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.210+0000 7f43ed8af640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43cc06b500 0x7f43cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:34.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.210+0000 7f43ed8af640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43cc06b500 0x7f43cc06d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000f60 tx=0x7f43e4007d70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:34.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.214+0000 7f43de7fc640 1 -- 172.21.15.19:0/2775325313 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43e0040030 con 0x7f43e8164970 2024-04-25T09:53:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.432+0000 7f43efb13640 1 -- 172.21.15.19:0/2775325313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f43e80d0510 con 0x7f43e8164970 2024-04-25T09:53:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.432+0000 7f43de7fc640 1 -- 172.21.15.19:0/2775325313 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f43e0079ff0 con 0x7f43e8164970 2024-04-25T09:53:34.434 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:53:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 -- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43cc06b500 msgr2=0x7f43cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43cc06b500 0x7f43cc06d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000f60 tx=0x7f43e4007d70 comp rx=0 tx=0).stop 2024-04-25T09:53:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 -- 172.21.15.19:0/2775325313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e8164970 msgr2=0x7f43e8163990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e8164970 0x7f43e8163990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009eb0 tx=0x7f43e0004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 -- 172.21.15.19:0/2775325313 shutdown_connections 2024-04-25T09:53:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43cc06b500 0x7f43cc06d9c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e8164970 0x7f43e8163990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e8144750 0x7f43e813dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 --2- 172.21.15.19:0/2775325313 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e80ce920 0x7f43e813d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 -- 172.21.15.19:0/2775325313 >> 172.21.15.19:0/2775325313 conn(0x7f43e806abe0 msgr2=0x7f43e81435d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 -- 172.21.15.19:0/2775325313 shutdown_connections 2024-04-25T09:53:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:34.434+0000 7f43efb13640 1 -- 172.21.15.19:0/2775325313 wait complete. 2024-04-25T09:53:34.447 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:53:34.447 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:53:34.447 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T09:53:34.447 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:53:34.447 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:53:34.447 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:53:34.447 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:53:34.447 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:53:34.447 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:53:34.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:34.448 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T09:53:34.463 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T09:53:34.464 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:53:34.465 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:34.465 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T09:53:34.465 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:53:34.465 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42732sec preferred_lft 42732sec 2024-04-25T09:53:34.465 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T09:53:34.465 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:34.465 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:34.465 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:34.465 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T09:53:34.465 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T09:53:34.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:34.466 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:34.466 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:34.466 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-04-25T09:53:34.466 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-04-25T09:53:34.466 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-04-25T09:53:34.467 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T09:53:34.467 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:34.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:34.561 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:53:34.584 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T09:53:34.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:34.585 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T09:53:34.642 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T09:53:34.642 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:53:34.642 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:53:34.642 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:53:34.642 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:53:34.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:34.643 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:34.643 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:34.643 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:53:34.643 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:53:34.643 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:53:34.643 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:34.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:34.729 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:53:34.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:34.745 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-04-25T09:53:34.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:34.800 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:34.800 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:34.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:53:34.801 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T09:53:34.801 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:34.877 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T09:53:34.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:34.877 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:34.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:34.989 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:34.989 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:34.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T09:53:34.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T09:53:34.989 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:35.025 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:35.025 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:53:35.080 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:53:35.080 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:35.081 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:53:35.139 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:53:35.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:35.140 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:35.221 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:35.221 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:35.336 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:35.336 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:35.336 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T09:53:35.336 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T09:53:35.336 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T09:53:35.337 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:53:35.337 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:53:35.337 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:53:35.337 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:53:35.337 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:53:35.339 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:53:35.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:35.339 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:53:35.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:35.404 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:35.461 INFO:teuthology.orchestra.run.smithi019.stdout:client4574 2024-04-25T09:53:35.461 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:53:35.461 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:35.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:35.518 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:35.575 INFO:teuthology.orchestra.run.smithi019.stdout:client4574 2024-04-25T09:53:35.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:35.575 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:35.633 INFO:teuthology.orchestra.run.smithi019.stdout:client4574 2024-04-25T09:53:35.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:35.633 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/342f98bd-d5a6-4934-bb27-7f8bea67a3be.client4574/status') 2024-04-25T09:53:35.698 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.4574 (3)172.21.15.19:0/836552754 2024-04-25T09:53:35.698 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-04-25T09:53:35.698 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-04-25T09:53:35.699 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-04-25T09:53:35.699 INFO:teuthology.orchestra.run.smithi019.stderr:69 bytes copied, 2.455e-05 s, 2.8 MB/s 2024-04-25T09:53:35.700 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-25T09:53:35.700 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:53:35.700 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T09:53:35.700 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:53:35.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.841+0000 7f7c714ac640 1 Processor -- start 2024-04-25T09:53:35.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.841+0000 7f7c714ac640 1 -- start start 2024-04-25T09:53:35.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c714ac640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7c6c15db60 0x7f7c6c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c714ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c0b5f00 0x7f7c6c0d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c714ac640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7c6c15df40 0x7f7c6c0d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c714ac640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c0ce930 con 0x7f7c6c15db60 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c714ac640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c0b6450 con 0x7f7c6c15df40 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c714ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c0cec00 con 0x7f7c6c0b5f00 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7c6c15db60 0x7f7c6c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7c6c15db60 0x7f7c6c0ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37526/0 (socket says 172.21.15.19:37526) 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6affd640 1 -- 172.21.15.19:0/4104512109 learned_addr learned my addr 172.21.15.19:0/4104512109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6a7fc640 1 --2- 172.21.15.19:0/4104512109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c0b5f00 0x7f7c6c0d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6b7fe640 1 --2- 172.21.15.19:0/4104512109 >> v2:172.21.15.112:3300/0 conn(0x7f7c6c15df40 0x7f7c6c0d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6affd640 1 -- 172.21.15.19:0/4104512109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c0b5f00 msgr2=0x7f7c6c0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6affd640 1 --2- 172.21.15.19:0/4104512109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c0b5f00 0x7f7c6c0d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6affd640 1 -- 172.21.15.19:0/4104512109 >> v2:172.21.15.112:3300/0 conn(0x7f7c6c15df40 msgr2=0x7f7c6c0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6affd640 1 --2- 172.21.15.19:0/4104512109 >> v2:172.21.15.112:3300/0 conn(0x7f7c6c15df40 0x7f7c6c0d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6affd640 1 -- 172.21.15.19:0/4104512109 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c0ceee0 con 0x7f7c6c15db60 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6a7fc640 1 --2- 172.21.15.19:0/4104512109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c0b5f00 0x7f7c6c0d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.842+0000 7f7c6b7fe640 1 --2- 172.21.15.19:0/4104512109 >> v2:172.21.15.112:3300/0 conn(0x7f7c6c15df40 0x7f7c6c0d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.843+0000 7f7c6affd640 1 --2- 172.21.15.19:0/4104512109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c6c15db60 0x7f7c6c0ce3f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c161890 tx=0x7f7c5c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83ed73fca52709f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.843+0000 7f7c69ffb640 1 -- 172.21.15.19:0/4104512109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c037070 con 0x7f7c6c15db60 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.843+0000 7f7c69ffb640 1 -- 172.21.15.19:0/4104512109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c032440 con 0x7f7c6c15db60 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.843+0000 7f7c69ffb640 1 -- 172.21.15.19:0/4104512109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c032810 con 0x7f7c6c15db60 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.844+0000 7f7c714ac640 1 -- 172.21.15.19:0/4104512109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c6c15db60 msgr2=0x7f7c6c0ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.844+0000 7f7c714ac640 1 --2- 172.21.15.19:0/4104512109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c6c15db60 0x7f7c6c0ce3f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c161890 tx=0x7f7c5c02a180 comp rx=0 tx=0).stop 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.844+0000 7f7c714ac640 1 -- 172.21.15.19:0/4104512109 shutdown_connections 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.844+0000 7f7c714ac640 1 --2- 172.21.15.19:0/4104512109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c0b5f00 0x7f7c6c0d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.844+0000 7f7c714ac640 1 --2- 172.21.15.19:0/4104512109 >> v2:172.21.15.112:3300/0 conn(0x7f7c6c15df40 0x7f7c6c0d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.844+0000 7f7c714ac640 1 --2- 172.21.15.19:0/4104512109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c6c15db60 0x7f7c6c0ce3f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.844+0000 7f7c714ac640 1 -- 172.21.15.19:0/4104512109 >> 172.21.15.19:0/4104512109 conn(0x7f7c6c061600 msgr2=0x7f7c6c061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.844+0000 7f7c714ac640 1 -- 172.21.15.19:0/4104512109 shutdown_connections 2024-04-25T09:53:35.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.844+0000 7f7c714ac640 1 -- 172.21.15.19:0/4104512109 wait complete. 2024-04-25T09:53:35.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.845+0000 7f7c714ac640 1 Processor -- start 2024-04-25T09:53:35.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.845+0000 7f7c714ac640 1 -- start start 2024-04-25T09:53:35.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c714ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c6c0b5f00 0x7f7c6c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c714ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c6c15db60 0x7f7c6c0ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c714ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c15df40 0x7f7c6c0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c714ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c161950 con 0x7f7c6c15db60 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c714ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c0d1d50 con 0x7f7c6c0b5f00 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c714ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c161ad0 con 0x7f7c6c15df40 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c6c15db60 0x7f7c6c0ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c6c15db60 0x7f7c6c0ab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37542/0 (socket says 172.21.15.19:37542) 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6a7fc640 1 -- 172.21.15.19:0/1040832049 learned_addr learned my addr 172.21.15.19:0/1040832049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6b7fe640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c15df40 0x7f7c6c0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6affd640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c6c0b5f00 0x7f7c6c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6a7fc640 1 -- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c15df40 msgr2=0x7f7c6c0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6a7fc640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c15df40 0x7f7c6c0cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6a7fc640 1 -- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c6c0b5f00 msgr2=0x7f7c6c15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6a7fc640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c6c0b5f00 0x7f7c6c15b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6a7fc640 1 -- 172.21.15.19:0/1040832049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c0cce30 con 0x7f7c6c15db60 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6b7fe640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c15df40 0x7f7c6c0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6affd640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c6c0b5f00 0x7f7c6c15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c6a7fc640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c6c15db60 0x7f7c6c0ab860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400e840 tx=0x7f7c5400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c637fe640 1 -- 172.21.15.19:0/1040832049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54017070 con 0x7f7c6c15db60 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.846+0000 7f7c637fe640 1 -- 172.21.15.19:0/1040832049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5400c8d0 con 0x7f7c6c15db60 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.847+0000 7f7c637fe640 1 -- 172.21.15.19:0/1040832049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5400f040 con 0x7f7c6c15db60 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.847+0000 7f7c714ac640 1 -- 172.21.15.19:0/1040832049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6c0cd110 con 0x7f7c6c15db60 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.847+0000 7f7c714ac640 1 -- 172.21.15.19:0/1040832049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6c0cd6d0 con 0x7f7c6c15db60 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.847+0000 7f7c714ac640 1 -- 172.21.15.19:0/1040832049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6c073270 con 0x7f7c6c15db60 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.848+0000 7f7c637fe640 1 -- 172.21.15.19:0/1040832049 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c5401a070 con 0x7f7c6c15db60 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.848+0000 7f7c637fe640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c4c06b500 0x7f7c4c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.849+0000 7f7c637fe640 1 -- 172.21.15.19:0/1040832049 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7c5408a050 con 0x7f7c6c15db60 2024-04-25T09:53:35.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.849+0000 7f7c6affd640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c4c06b500 0x7f7c4c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.849+0000 7f7c6affd640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c4c06b500 0x7f7c4c06d9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c034000 tx=0x7f7c5c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:35.853+0000 7f7c637fe640 1 -- 172.21.15.19:0/1040832049 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c54009e20 con 0x7f7c6c15db60 2024-04-25T09:53:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.071+0000 7f7c714ac640 1 -- 172.21.15.19:0/1040832049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7c6c073990 con 0x7f7c6c15db60 2024-04-25T09:53:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.072+0000 7f7c637fe640 1 -- 172.21.15.19:0/1040832049 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7c54058ba0 con 0x7f7c6c15db60 2024-04-25T09:53:36.074 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.073+0000 7f7c714ac640 1 -- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c4c06b500 msgr2=0x7f7c4c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.073+0000 7f7c714ac640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c4c06b500 0x7f7c4c06d9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c034000 tx=0x7f7c5c02e000 comp rx=0 tx=0).stop 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.073+0000 7f7c714ac640 1 -- 172.21.15.19:0/1040832049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c6c15db60 msgr2=0x7f7c6c0ab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.073+0000 7f7c714ac640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c6c15db60 0x7f7c6c0ab860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400e840 tx=0x7f7c5400ed00 comp rx=0 tx=0).stop 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.073+0000 7f7c714ac640 1 -- 172.21.15.19:0/1040832049 shutdown_connections 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.073+0000 7f7c714ac640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c4c06b500 0x7f7c4c06d9c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.073+0000 7f7c714ac640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c6c15df40 0x7f7c6c0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.073+0000 7f7c714ac640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c6c15db60 0x7f7c6c0ab860 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.073+0000 7f7c714ac640 1 --2- 172.21.15.19:0/1040832049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c6c0b5f00 0x7f7c6c15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.073+0000 7f7c714ac640 1 -- 172.21.15.19:0/1040832049 >> 172.21.15.19:0/1040832049 conn(0x7f7c6c061600 msgr2=0x7f7c6c0b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.074+0000 7f7c714ac640 1 -- 172.21.15.19:0/1040832049 shutdown_connections 2024-04-25T09:53:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:36.074+0000 7f7c714ac640 1 -- 172.21.15.19:0/1040832049 wait complete. 2024-04-25T09:53:36.086 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:53:36.086 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:53:36.086 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T09:53:36.086 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T09:53:36.086 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T09:53:36.087 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:53:36.087 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:53:36.087 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:53:36.087 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T09:53:36.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.087 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T09:53:36.103 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:36.103 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:53:36.104 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42730sec preferred_lft 42730sec 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link tentative 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:53:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:53:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link tentative 2024-04-25T09:53:36.106 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.106 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:53:36.167 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T09:53:36.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.169 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T09:53:36.225 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T09:53:36.225 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:53:36.225 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:53:36.225 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:53:36.225 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:53:36.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.226 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:36.226 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:36.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:53:36.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:53:36.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:53:36.226 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:36.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.310 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:53:36.326 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:53:36.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.327 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-04-25T09:53:36.382 INFO:teuthology.orchestra.run.smithi019.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T09:53:36.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.383 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:36.383 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:36.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:53:36.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T09:53:36.384 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:36.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.460 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T09:53:36.524 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:36.524 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:36.524 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:36.524 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.524 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:36.525 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.525 INFO:teuthology.orchestra.run.smithi019.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:36.525 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether f6:d7:57:5e:3d:42 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:53:36.525 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:53:36.525 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.525 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::f4d7:57ff:fe5e:3d42/64 scope link tentative 2024-04-25T09:53:36.525 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T09:53:36.590 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.591 INFO:teuthology.orchestra.run.smithi019.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether f6:d7:57:5e:3d:42 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:53:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:53:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::f4d7:57ff:fe5e:3d42/64 scope link tentative 2024-04-25T09:53:36.591 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:36.593 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T09:53:36.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.593 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:36.593 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:36.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T09:53:36.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T09:53:36.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T09:53:36.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T09:53:36.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T09:53:36.593 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:36.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.704 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:53:36.704 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:53:36.704 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.1 up 2024-04-25T09:53:36.704 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T09:53:36.704 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:36.740 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T09:53:36.743 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:36.783 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:36.795 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T09:53:36.795 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:36.795 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T09:53:36.890 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-25T09:53:36.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.891 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:36.973 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:36.973 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:36.980 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:36.980 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:36.981 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T09:53:36.981 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 2@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T09:53:36.981 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T09:53:36.982 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T09:53:36.982 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:53:36.982 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:53:36.982 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:53:36.982 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:53:36.984 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:53:36.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:36.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T09:53:37.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:37.051 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:37.108 INFO:teuthology.orchestra.run.smithi019.stdout:client4580 2024-04-25T09:53:37.108 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:53:37.109 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:37.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:37.165 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:37.221 INFO:teuthology.orchestra.run.smithi019.stdout:client4580 2024-04-25T09:53:37.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:37.222 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:37.279 INFO:teuthology.orchestra.run.smithi019.stdout:client4580 2024-04-25T09:53:37.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:37.279 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/342f98bd-d5a6-4934-bb27-7f8bea67a3be.client4580/status') 2024-04-25T09:53:37.343 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.4580 (3)172.21.15.19:0/1930534181 2024-04-25T09:53:37.344 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-04-25T09:53:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-04-25T09:53:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-04-25T09:53:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:70 bytes copied, 2.389e-05 s, 2.9 MB/s 2024-04-25T09:53:37.345 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-25T09:53:37.345 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:53:37.345 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T09:53:37.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:53:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.483+0000 7f25a42a3640 1 Processor -- start 2024-04-25T09:53:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.483+0000 7f25a42a3640 1 -- start start 2024-04-25T09:53:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a42a3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c154ee0 0x7f259c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a42a3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f259c0cfcf0 0x7f259c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a42a3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c071480 con 0x7f259c0ce3c0 2024-04-25T09:53:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a42a3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259c0ce830 con 0x7f259c0cfcf0 2024-04-25T09:53:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a42a3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c072090 con 0x7f259c154ee0 2024-04-25T09:53:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a203f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c154ee0 0x7f259c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a183e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f259c0cfcf0 0x7f259c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a203f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c154ee0 0x7f259c0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:36020/0 (socket says 172.21.15.19:36020) 2024-04-25T09:53:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a2840640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f259c0ce3c0 0x7f259c15e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:40738/0 (socket says 172.21.15.19:40738) 2024-04-25T09:53:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a203f640 1 -- 172.21.15.19:0/149211894 learned_addr learned my addr 172.21.15.19:0/149211894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a183e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f259c0cfcf0 0x7f259c0cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:59584/0 (socket says 172.21.15.19:59584) 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a203f640 1 -- 172.21.15.19:0/149211894 >> v2:172.21.15.112:3300/0 conn(0x7f259c0cfcf0 msgr2=0x7f259c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a203f640 1 --2- 172.21.15.19:0/149211894 >> v2:172.21.15.112:3300/0 conn(0x7f259c0cfcf0 0x7f259c0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a203f640 1 -- 172.21.15.19:0/149211894 >> v1:172.21.15.112:6789/0 conn(0x7f259c0ce3c0 legacy=0x7f259c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.484+0000 7f25a203f640 1 -- 172.21.15.19:0/149211894 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c071760 con 0x7f259c154ee0 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.485+0000 7f25a203f640 1 --2- 172.21.15.19:0/149211894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c154ee0 0x7f259c0cf7b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2598009870 tx=0x7f259802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989ca56845eeb1cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.485+0000 7f25a103d640 1 -- 172.21.15.19:0/149211894 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598037070 con 0x7f259c154ee0 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.485+0000 7f25a103d640 1 -- 172.21.15.19:0/149211894 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2598032440 con 0x7f259c154ee0 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.485+0000 7f25a103d640 1 -- 172.21.15.19:0/149211894 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598032810 con 0x7f259c154ee0 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.486+0000 7f25a42a3640 1 -- 172.21.15.19:0/149211894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c154ee0 msgr2=0x7f259c0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.486+0000 7f25a42a3640 1 --2- 172.21.15.19:0/149211894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c154ee0 0x7f259c0cf7b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2598009870 tx=0x7f259802a270 comp rx=0 tx=0).stop 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.486+0000 7f25a42a3640 1 -- 172.21.15.19:0/149211894 shutdown_connections 2024-04-25T09:53:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.486+0000 7f25a42a3640 1 --2- 172.21.15.19:0/149211894 >> v2:172.21.15.112:3300/0 conn(0x7f259c0cfcf0 0x7f259c0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.486+0000 7f25a42a3640 1 --2- 172.21.15.19:0/149211894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c154ee0 0x7f259c0cf7b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.486+0000 7f25a42a3640 1 -- 172.21.15.19:0/149211894 >> 172.21.15.19:0/149211894 conn(0x7f259c06abe0 msgr2=0x7f259c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.486+0000 7f25a42a3640 1 -- 172.21.15.19:0/149211894 shutdown_connections 2024-04-25T09:53:37.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.486+0000 7f25a42a3640 1 -- 172.21.15.19:0/149211894 wait complete. 2024-04-25T09:53:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.487+0000 7f25a42a3640 1 Processor -- start 2024-04-25T09:53:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.487+0000 7f25a42a3640 1 -- start start 2024-04-25T09:53:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a42a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0ce3c0 0x7f259c143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a42a3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c0cfcf0 0x7f259c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a42a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f259c154ee0 0x7f259c1e3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a42a3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c071230 con 0x7f259c0ce3c0 2024-04-25T09:53:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a42a3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c071d20 con 0x7f259c154ee0 2024-04-25T09:53:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a42a3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c1552c0 con 0x7f259c0cfcf0 2024-04-25T09:53:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a203f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0ce3c0 0x7f259c143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a203f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0ce3c0 0x7f259c143e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37546/0 (socket says 172.21.15.19:37546) 2024-04-25T09:53:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a203f640 1 -- 172.21.15.19:0/3570678268 learned_addr learned my addr 172.21.15.19:0/3570678268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a183e640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c0cfcf0 0x7f259c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a2840640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f259c154ee0 0x7f259c1e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a203f640 1 -- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c0cfcf0 msgr2=0x7f259c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a203f640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c0cfcf0 0x7f259c159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a203f640 1 -- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f259c154ee0 msgr2=0x7f259c1e3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a203f640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f259c154ee0 0x7f259c1e3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a203f640 1 -- 172.21.15.19:0/3570678268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c144380 con 0x7f259c0ce3c0 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.488+0000 7f25a203f640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0ce3c0 0x7f259c143e40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f259802a2b0 tx=0x7f25980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.489+0000 7f2592ffd640 1 -- 172.21.15.19:0/3570678268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598041030 con 0x7f259c0ce3c0 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.489+0000 7f2592ffd640 1 -- 172.21.15.19:0/3570678268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2598030440 con 0x7f259c0ce3c0 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.489+0000 7f25a42a3640 1 -- 172.21.15.19:0/3570678268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c141860 con 0x7f259c0ce3c0 2024-04-25T09:53:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.489+0000 7f2592ffd640 1 -- 172.21.15.19:0/3570678268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598037040 con 0x7f259c0ce3c0 2024-04-25T09:53:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.489+0000 7f25a42a3640 1 -- 172.21.15.19:0/3570678268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c141e20 con 0x7f259c0ce3c0 2024-04-25T09:53:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.490+0000 7f25a42a3640 1 -- 172.21.15.19:0/3570678268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c073990 con 0x7f259c0ce3c0 2024-04-25T09:53:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.490+0000 7f2592ffd640 1 -- 172.21.15.19:0/3570678268 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2598044070 con 0x7f259c0ce3c0 2024-04-25T09:53:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.490+0000 7f2592ffd640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f258006b4d0 0x7f258006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.490+0000 7f2592ffd640 1 -- 172.21.15.19:0/3570678268 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f25980aa7e0 con 0x7f259c0ce3c0 2024-04-25T09:53:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.491+0000 7f25a183e640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f258006b4d0 0x7f258006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.491+0000 7f25a183e640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f258006b4d0 0x7f258006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f258c00b240 tx=0x7f258c00b3a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:37.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.495+0000 7f2592ffd640 1 -- 172.21.15.19:0/3570678268 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f259804b070 con 0x7f259c0ce3c0 2024-04-25T09:53:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.715+0000 7f25a42a3640 1 -- 172.21.15.19:0/3570678268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f259c06d360 con 0x7f259c0ce3c0 2024-04-25T09:53:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.716+0000 7f2592ffd640 1 -- 172.21.15.19:0/3570678268 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f259c144380 con 0x7f259c0ce3c0 2024-04-25T09:53:37.718 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:53:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 -- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f258006b4d0 msgr2=0x7f258006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f258006b4d0 0x7f258006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f258c00b240 tx=0x7f258c00b3a0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 -- 172.21.15.19:0/3570678268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0ce3c0 msgr2=0x7f259c143e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0ce3c0 0x7f259c143e40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f259802a2b0 tx=0x7f25980046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 -- 172.21.15.19:0/3570678268 shutdown_connections 2024-04-25T09:53:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f258006b4d0 0x7f258006d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f259c154ee0 0x7f259c1e3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f259c0cfcf0 0x7f259c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 --2- 172.21.15.19:0/3570678268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0ce3c0 0x7f259c143e40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 -- 172.21.15.19:0/3570678268 >> 172.21.15.19:0/3570678268 conn(0x7f259c06abe0 msgr2=0x7f259c15aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 -- 172.21.15.19:0/3570678268 shutdown_connections 2024-04-25T09:53:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:37.718+0000 7f25a42a3640 1 -- 172.21.15.19:0/3570678268 wait complete. 2024-04-25T09:53:37.731 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:53:37.731 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:53:37.731 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-04-25T09:53:37.731 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T09:53:37.731 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T09:53:37.731 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:53:37.731 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:53:37.731 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:53:37.731 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T09:53:37.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:37.732 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-04-25T09:53:37.751 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:37.751 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:37.752 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:37.752 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:37.752 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:37.752 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:37.752 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:37.752 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:37.752 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-04-25T09:53:37.752 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:37.752 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:37.752 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-04-25T09:53:37.752 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:37.753 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:37.753 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:53:37.753 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:37.753 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:53:37.754 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42744sec preferred_lft 42744sec 2024-04-25T09:53:37.754 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::f6c9:7e02:e254:f621/64 scope link noprefixroute 2024-04-25T09:53:37.754 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:37.754 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T09:53:37.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:37.756 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2024-04-25T09:53:37.756 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T09:53:37.756 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-04-25T09:53:37.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-04-25T09:53:37.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-04-25T09:53:37.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T09:53:37.757 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T09:53:37.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:37.896 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:53:37.920 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-04-25T09:53:37.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:37.922 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-04-25T09:53:37.992 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-04-25T09:53:37.993 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:53:37.993 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:53:37.993 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:53:37.993 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:53:37.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:37.993 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2024-04-25T09:53:37.993 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T09:53:37.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:53:37.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:53:37.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:53:37.994 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T09:53:38.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:38.250 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T09:53:38.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:38.265 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-04-25T09:53:38.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:38.322 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2024-04-25T09:53:38.322 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T09:53:38.322 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:53:38.322 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T09:53:38.322 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T09:53:38.400 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T09:53:38.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:38.400 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2024-04-25T09:53:38.400 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T09:53:38.400 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T09:53:38.400 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T09:53:38.401 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T09:53:38.401 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T09:53:38.401 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T09:53:38.401 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T09:53:38.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:38.532 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2024-04-25T09:53:38.532 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T09:53:38.532 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-04-25T09:53:38.533 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T09:53:38.533 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T09:53:38.567 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:38.567 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T09:53:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T09:53:38.622 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:38.623 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T09:53:38.678 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T09:53:38.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:38.679 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:38.770 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:38.770 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:38.881 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:38.882 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:38.882 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T09:53:38.882 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with new device syntax: 1@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T09:53:38.882 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T09:53:38.882 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T09:53:38.883 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:53:38.883 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:53:38.883 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:53:38.883 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:53:38.884 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:53:38.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:38.884 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T09:53:38.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:38.907 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:38.964 INFO:teuthology.orchestra.run.smithi112.stdout:client4824 2024-04-25T09:53:38.965 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:53:38.965 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:39.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:39.025 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:39.081 INFO:teuthology.orchestra.run.smithi112.stdout:client4824 2024-04-25T09:53:39.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:39.081 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:39.137 INFO:teuthology.orchestra.run.smithi112.stdout:client4824 2024-04-25T09:53:39.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:39.138 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/342f98bd-d5a6-4934-bb27-7f8bea67a3be.client4824/status') 2024-04-25T09:53:39.201 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.4824 (3)172.21.15.112:0/3291549541 2024-04-25T09:53:39.201 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-04-25T09:53:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-04-25T09:53:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-04-25T09:53:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:71 bytes copied, 2.2381e-05 s, 3.2 MB/s 2024-04-25T09:53:39.203 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-25T09:53:39.203 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:53:39.203 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T09:53:39.203 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:53:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.315+0000 7f2140570640 1 Processor -- start 2024-04-25T09:53:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.315+0000 7f2140570640 1 -- start start 2024-04-25T09:53:39.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.316+0000 7f2140570640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f213815db70 0x7f2138162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.316+0000 7f2140570640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f21380b6cb0 0x7f21380b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.316+0000 7f2140570640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2138162690 con 0x7f21380b5a60 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.316+0000 7f2140570640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21380d3800 con 0x7f21380b6cb0 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.316+0000 7f2140570640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21380ceee0 con 0x7f213815db70 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.316+0000 7f213db0b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f21380b6cb0 0x7f21380b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.316+0000 7f213e30c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f213815db70 0x7f2138162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.316+0000 7f213eb0d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f21380b5a60 0x7f213806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:55938/0 (socket says 172.21.15.19:55938) 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.316+0000 7f213eb0d640 1 -- 172.21.15.19:0/2888547770 learned_addr learned my addr 172.21.15.19:0/2888547770 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.316+0000 7f213d30a640 1 -- 172.21.15.19:0/2888547770 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807150172 0 0) 0x7f2138162690 con 0x7f21380b5a60 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.316+0000 7f213d30a640 1 -- 172.21.15.19:0/2888547770 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2124003520 con 0x7f21380b5a60 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.317+0000 7f213e30c640 1 -- 172.21.15.19:0/2888547770 >> v2:172.21.15.112:3300/0 conn(0x7f21380b6cb0 msgr2=0x7f21380b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.317+0000 7f213e30c640 1 --2- 172.21.15.19:0/2888547770 >> v2:172.21.15.112:3300/0 conn(0x7f21380b6cb0 0x7f21380b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.317+0000 7f213e30c640 1 -- 172.21.15.19:0/2888547770 >> v1:172.21.15.112:6789/0 conn(0x7f21380b5a60 legacy=0x7f213806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.317+0000 7f213e30c640 1 -- 172.21.15.19:0/2888547770 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21380d1d10 con 0x7f213815db70 2024-04-25T09:53:39.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.317+0000 7f213db0b640 1 --2- 172.21.15.19:0/2888547770 >> v2:172.21.15.112:3300/0 conn(0x7f21380b6cb0 0x7f21380b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:39.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.317+0000 7f213e30c640 1 --2- 172.21.15.19:0/2888547770 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f213815db70 0x7f2138162150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f21340097a0 tx=0x7f213402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=859511ff654f0546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.317+0000 7f213d30a640 1 -- 172.21.15.19:0/2888547770 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134037070 con 0x7f213815db70 2024-04-25T09:53:39.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.318+0000 7f213d30a640 1 -- 172.21.15.19:0/2888547770 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2134032440 con 0x7f213815db70 2024-04-25T09:53:39.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.318+0000 7f213d30a640 1 -- 172.21.15.19:0/2888547770 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21340327f0 con 0x7f213815db70 2024-04-25T09:53:39.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.318+0000 7f2140570640 1 -- 172.21.15.19:0/2888547770 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f213815db70 msgr2=0x7f2138162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.318+0000 7f2140570640 1 --2- 172.21.15.19:0/2888547770 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f213815db70 0x7f2138162150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f21340097a0 tx=0x7f213402a180 comp rx=0 tx=0).stop 2024-04-25T09:53:39.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.318+0000 7f2140570640 1 -- 172.21.15.19:0/2888547770 shutdown_connections 2024-04-25T09:53:39.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.318+0000 7f2140570640 1 --2- 172.21.15.19:0/2888547770 >> v2:172.21.15.112:3300/0 conn(0x7f21380b6cb0 0x7f21380b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.318+0000 7f2140570640 1 --2- 172.21.15.19:0/2888547770 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f213815db70 0x7f2138162150 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.318+0000 7f2140570640 1 -- 172.21.15.19:0/2888547770 >> 172.21.15.19:0/2888547770 conn(0x7f21380d0520 msgr2=0x7f21380d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:39.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.318+0000 7f2140570640 1 -- 172.21.15.19:0/2888547770 shutdown_connections 2024-04-25T09:53:39.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.318+0000 7f2140570640 1 -- 172.21.15.19:0/2888547770 wait complete. 2024-04-25T09:53:39.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.319+0000 7f2140570640 1 Processor -- start 2024-04-25T09:53:39.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.319+0000 7f2140570640 1 -- start start 2024-04-25T09:53:39.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.320+0000 7f2140570640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21380b5a60 0x7f213815f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.320+0000 7f2140570640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21380b6cb0 0x7f2138160a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.320+0000 7f2140570640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213815db70 0x7f213815be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.320+0000 7f2140570640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213815df50 con 0x7f213815db70 2024-04-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.320+0000 7f2140570640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380d22d0 con 0x7f21380b6cb0 2024-04-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.320+0000 7f2140570640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21380ce530 con 0x7f21380b5a60 2024-04-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.320+0000 7f213eb0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213815db70 0x7f213815be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.320+0000 7f213eb0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213815db70 0x7f213815be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60656/0 (socket says 172.21.15.19:60656) 2024-04-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.320+0000 7f213eb0d640 1 -- 172.21.15.19:0/2394307236 learned_addr learned my addr 172.21.15.19:0/2394307236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.320+0000 7f213db0b640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21380b6cb0 0x7f2138160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.320+0000 7f213e30c640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21380b5a60 0x7f213815f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.321+0000 7f213eb0d640 1 -- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21380b5a60 msgr2=0x7f213815f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.321+0000 7f213eb0d640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21380b5a60 0x7f213815f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.321+0000 7f213eb0d640 1 -- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21380b6cb0 msgr2=0x7f2138160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.321+0000 7f213eb0d640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21380b6cb0 0x7f2138160a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.321+0000 7f213eb0d640 1 -- 172.21.15.19:0/2394307236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21381f39c0 con 0x7f213815db70 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.321+0000 7f213eb0d640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213815db70 0x7f213815be00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f213000caa0 tx=0x7f213000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.321+0000 7f212effd640 1 -- 172.21.15.19:0/2394307236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130017070 con 0x7f213815db70 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.321+0000 7f212effd640 1 -- 172.21.15.19:0/2394307236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213000ac00 con 0x7f213815db70 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.321+0000 7f2140570640 1 -- 172.21.15.19:0/2394307236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21381f3ca0 con 0x7f213815db70 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.321+0000 7f212effd640 1 -- 172.21.15.19:0/2394307236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130012400 con 0x7f213815db70 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.321+0000 7f2140570640 1 -- 172.21.15.19:0/2394307236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21381f4260 con 0x7f213815db70 2024-04-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.322+0000 7f2140570640 1 -- 172.21.15.19:0/2394307236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2138073990 con 0x7f213815db70 2024-04-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.323+0000 7f212effd640 1 -- 172.21.15.19:0/2394307236 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f213001a070 con 0x7f213815db70 2024-04-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.323+0000 7f212effd640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f211c06b4d0 0x7f211c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.323+0000 7f212effd640 1 -- 172.21.15.19:0/2394307236 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f213008a340 con 0x7f213815db70 2024-04-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.323+0000 7f213e30c640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f211c06b4d0 0x7f211c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.324+0000 7f213e30c640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f211c06b4d0 0x7f211c06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2134034000 tx=0x7f213402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.327+0000 7f212effd640 1 -- 172.21.15.19:0/2394307236 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21300587b0 con 0x7f213815db70 2024-04-25T09:53:39.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.552+0000 7f2140570640 1 -- 172.21.15.19:0/2394307236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2138158420 con 0x7f213815db70 2024-04-25T09:53:39.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.553+0000 7f212effd640 1 -- 172.21.15.19:0/2394307236 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f213005c850 con 0x7f213815db70 2024-04-25T09:53:39.555 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 -- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f211c06b4d0 msgr2=0x7f211c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f211c06b4d0 0x7f211c06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2134034000 tx=0x7f213402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 -- 172.21.15.19:0/2394307236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213815db70 msgr2=0x7f213815be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213815db70 0x7f213815be00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f213000caa0 tx=0x7f213000cf60 comp rx=0 tx=0).stop 2024-04-25T09:53:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 -- 172.21.15.19:0/2394307236 shutdown_connections 2024-04-25T09:53:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f211c06b4d0 0x7f211c06d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213815db70 0x7f213815be00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21380b6cb0 0x7f2138160a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 --2- 172.21.15.19:0/2394307236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21380b5a60 0x7f213815f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 -- 172.21.15.19:0/2394307236 >> 172.21.15.19:0/2394307236 conn(0x7f21380d0520 msgr2=0x7f213815a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 -- 172.21.15.19:0/2394307236 shutdown_connections 2024-04-25T09:53:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:39.555+0000 7f2140570640 1 -- 172.21.15.19:0/2394307236 wait complete. 2024-04-25T09:53:39.567 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:53:39.568 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:53:39.568 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-04-25T09:53:39.568 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T09:53:39.568 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T09:53:39.568 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:53:39.568 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:53:39.568 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:53:39.569 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T09:53:39.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:39.569 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-04-25T09:53:39.586 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:39.586 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:39.586 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:39.586 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:39.586 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:39.586 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:39.587 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:39.587 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:39.587 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-04-25T09:53:39.587 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:39.587 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:39.587 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-04-25T09:53:39.587 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:39.587 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:39.587 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:53:39.588 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:39.588 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:53:39.588 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42742sec preferred_lft 42742sec 2024-04-25T09:53:39.588 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::f6c9:7e02:e254:f621/64 scope link noprefixroute 2024-04-25T09:53:39.588 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:39.588 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:39.588 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether fa:ab:e7:93:20:29 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:39.588 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:53:39.588 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:39.589 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::7079:4dff:fe26:d18c/64 scope link 2024-04-25T09:53:39.589 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:39.589 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:53:39.589 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether fa:ab:e7:93:20:29 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:53:39.589 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::f8ab:e7ff:fe93:2029/64 scope link tentative 2024-04-25T09:53:39.589 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:39.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:39.590 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:53:39.649 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-04-25T09:53:39.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:39.650 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-04-25T09:53:39.707 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-04-25T09:53:39.707 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:53:39.707 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:53:39.707 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:53:39.707 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:53:39.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:39.708 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2024-04-25T09:53:39.708 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T09:53:39.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:53:39.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:53:39.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:53:39.708 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T09:53:39.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:39.791 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T09:53:39.846 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T09:53:39.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:39.850 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-04-25T09:53:39.901 INFO:teuthology.orchestra.run.smithi112.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T09:53:39.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:39.901 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2024-04-25T09:53:39.901 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T09:53:39.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T09:53:39.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T09:53:39.901 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T09:53:39.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:39.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T09:53:40.040 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:40.040 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:40.040 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:40.040 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:40.041 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:40.041 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:40.041 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:40.041 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:61:ed:e1:89:94 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:53:40.041 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:53:40.041 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:40.041 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8461:edff:fee1:8994/64 scope link tentative 2024-04-25T09:53:40.041 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:40.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:40.042 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T09:53:40.106 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:40.107 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:40.107 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:40.107 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:40.107 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:40.107 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:40.108 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:40.108 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:61:ed:e1:89:94 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:53:40.108 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:53:40.108 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:40.108 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::8461:edff:fee1:8994/64 scope link tentative 2024-04-25T09:53:40.108 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:40.109 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T09:53:40.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:40.109 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2024-04-25T09:53:40.109 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T09:53:40.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T09:53:40.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T09:53:40.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T09:53:40.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T09:53:40.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T09:53:40.110 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T09:53:40.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:40.217 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2024-04-25T09:53:40.217 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T09:53:40.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.1 up 2024-04-25T09:53:40.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T09:53:40.217 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T09:53:40.251 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:40.251 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T09:53:40.307 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T09:53:40.307 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:40.307 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T09:53:40.362 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-25T09:53:40.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:40.362 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:40.443 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:40.443 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:40.450 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:40.450 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:40.451 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T09:53:40.451 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with new device syntax: 3@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T09:53:40.451 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T09:53:40.453 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T09:53:40.453 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:53:40.453 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:53:40.453 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:53:40.453 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:53:40.453 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:53:40.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:40.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T09:53:40.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:40.518 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:40.574 INFO:teuthology.orchestra.run.smithi112.stdout:client4830 2024-04-25T09:53:40.574 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:53:40.574 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:40.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:40.629 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:40.688 INFO:teuthology.orchestra.run.smithi112.stdout:client4830 2024-04-25T09:53:40.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:40.689 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:40.745 INFO:teuthology.orchestra.run.smithi112.stdout:client4830 2024-04-25T09:53:40.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:40.745 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/342f98bd-d5a6-4934-bb27-7f8bea67a3be.client4830/status') 2024-04-25T09:53:40.808 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.4830 (3)172.21.15.112:0/3431519584 2024-04-25T09:53:40.808 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-04-25T09:53:40.808 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-04-25T09:53:40.808 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-04-25T09:53:40.808 INFO:teuthology.orchestra.run.smithi112.stderr:71 bytes copied, 2.1991e-05 s, 3.2 MB/s 2024-04-25T09:53:40.809 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T09:53:40.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:40.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.938+0000 7f4f85b09640 1 Processor -- start 2024-04-25T09:53:40.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.938+0000 7f4f85b09640 1 -- start start 2024-04-25T09:53:40.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.939+0000 7f4f85b09640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4f800ce330 0x7f4f8006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.939+0000 7f4f85b09640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f801626e0 con 0x7f4f8006abe0 2024-04-25T09:53:40.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.939+0000 7f4f85b09640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f800cd270 con 0x7f4f800700a0 2024-04-25T09:53:40.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.939+0000 7f4f85b09640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f800cd550 con 0x7f4f800ce330 2024-04-25T09:53:40.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.939+0000 7f4f7effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4f800ce330 0x7f4f8006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.939+0000 7f4f7f7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4f8006abe0 0x7f4f8006afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57318/0 (socket says 172.21.15.19:57318) 2024-04-25T09:53:40.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.939+0000 7f4f7f7fe640 1 -- 172.21.15.19:0/3232446925 learned_addr learned my addr 172.21.15.19:0/3232446925 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:40.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.939+0000 7f4f7effd640 1 -- 172.21.15.19:0/3232446925 >> v1:172.21.15.112:6789/0 conn(0x7f4f800700a0 legacy=0x7f4f8015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.939+0000 7f4f7effd640 1 -- 172.21.15.19:0/3232446925 >> v1:172.21.15.19:6789/0 conn(0x7f4f8006abe0 legacy=0x7f4f8006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.939+0000 7f4f7e7fc640 1 -- 172.21.15.19:0/3232446925 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522721761 0 0) 0x7f4f801626e0 con 0x7f4f8006abe0 2024-04-25T09:53:40.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.939+0000 7f4f7effd640 1 -- 172.21.15.19:0/3232446925 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800cd830 con 0x7f4f800ce330 2024-04-25T09:53:40.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.940+0000 7f4f7effd640 1 --2- 172.21.15.19:0/3232446925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f800ce330 0x7f4f8006fb60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4f740097a0 tx=0x7f4f7402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2fa93749611ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.940+0000 7f4f7e7fc640 1 -- 172.21.15.19:0/3232446925 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74037070 con 0x7f4f800ce330 2024-04-25T09:53:40.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.940+0000 7f4f7e7fc640 1 -- 172.21.15.19:0/3232446925 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f74032440 con 0x7f4f800ce330 2024-04-25T09:53:40.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.940+0000 7f4f7e7fc640 1 -- 172.21.15.19:0/3232446925 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74032810 con 0x7f4f800ce330 2024-04-25T09:53:40.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.941+0000 7f4f85b09640 1 -- 172.21.15.19:0/3232446925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f800ce330 msgr2=0x7f4f8006fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.941+0000 7f4f85b09640 1 --2- 172.21.15.19:0/3232446925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f800ce330 0x7f4f8006fb60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4f740097a0 tx=0x7f4f7402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.941+0000 7f4f85b09640 1 -- 172.21.15.19:0/3232446925 shutdown_connections 2024-04-25T09:53:40.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.941+0000 7f4f85b09640 1 --2- 172.21.15.19:0/3232446925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f800ce330 0x7f4f8006fb60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.941+0000 7f4f85b09640 1 -- 172.21.15.19:0/3232446925 >> 172.21.15.19:0/3232446925 conn(0x7f4f800b2220 msgr2=0x7f4f800b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.941+0000 7f4f85b09640 1 -- 172.21.15.19:0/3232446925 shutdown_connections 2024-04-25T09:53:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.941+0000 7f4f85b09640 1 -- 172.21.15.19:0/3232446925 wait complete. 2024-04-25T09:53:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.942+0000 7f4f85b09640 1 Processor -- start 2024-04-25T09:53:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.942+0000 7f4f85b09640 1 -- start start 2024-04-25T09:53:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f85b09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8006abe0 0x7f4f8014e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f85b09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f800700a0 0x7f4f800c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f85b09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f800ce330 0x7f4f80160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f85b09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8015ea90 con 0x7f4f800700a0 2024-04-25T09:53:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f85b09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80161780 con 0x7f4f8006abe0 2024-04-25T09:53:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f85b09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8015ec10 con 0x7f4f800ce330 2024-04-25T09:53:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f7effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f800700a0 0x7f4f800c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f7effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f800700a0 0x7f4f800c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60678/0 (socket says 172.21.15.19:60678) 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f7effd640 1 -- 172.21.15.19:0/2096273558 learned_addr learned my addr 172.21.15.19:0/2096273558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f7ffff640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f800ce330 0x7f4f80160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f7f7fe640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8006abe0 0x7f4f8014e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.943+0000 7f4f7effd640 1 -- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f800ce330 msgr2=0x7f4f80160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.944+0000 7f4f7effd640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f800ce330 0x7f4f80160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.944+0000 7f4f7effd640 1 -- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8006abe0 msgr2=0x7f4f8014e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.944+0000 7f4f7effd640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8006abe0 0x7f4f8014e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.944+0000 7f4f7effd640 1 -- 172.21.15.19:0/2096273558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8014e5d0 con 0x7f4f800700a0 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.944+0000 7f4f7f7fe640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8006abe0 0x7f4f8014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.944+0000 7f4f7effd640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f800700a0 0x7f4f800c9e80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4f74035040 tx=0x7f4f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.944+0000 7f4f67fff640 1 -- 172.21.15.19:0/2096273558 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74037050 con 0x7f4f800700a0 2024-04-25T09:53:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.944+0000 7f4f67fff640 1 -- 172.21.15.19:0/2096273558 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f74038040 con 0x7f4f800700a0 2024-04-25T09:53:40.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.944+0000 7f4f85b09640 1 -- 172.21.15.19:0/2096273558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f801f4ba0 con 0x7f4f800700a0 2024-04-25T09:53:40.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.944+0000 7f4f67fff640 1 -- 172.21.15.19:0/2096273558 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74004470 con 0x7f4f800700a0 2024-04-25T09:53:40.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.944+0000 7f4f85b09640 1 -- 172.21.15.19:0/2096273558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f801f5160 con 0x7f4f800700a0 2024-04-25T09:53:40.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.945+0000 7f4f85b09640 1 -- 172.21.15.19:0/2096273558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f800c99b0 con 0x7f4f800700a0 2024-04-25T09:53:40.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.946+0000 7f4f67fff640 1 -- 172.21.15.19:0/2096273558 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f74030880 con 0x7f4f800700a0 2024-04-25T09:53:40.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.946+0000 7f4f67fff640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f6c06b4d0 0x7f4f6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.946+0000 7f4f67fff640 1 -- 172.21.15.19:0/2096273558 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4f740ab020 con 0x7f4f800700a0 2024-04-25T09:53:40.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.946+0000 7f4f7f7fe640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f6c06b4d0 0x7f4f6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.947+0000 7f4f7f7fe640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f6c06b4d0 0x7f4f6c06d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4f70000f60 tx=0x7f4f70007c70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:40.950+0000 7f4f67fff640 1 -- 172.21.15.19:0/2096273558 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f74079c80 con 0x7f4f800700a0 2024-04-25T09:53:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.136+0000 7f4f85b09640 1 -- 172.21.15.19:0/2096273558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f8006fe80 con 0x7f4f800700a0 2024-04-25T09:53:41.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.137+0000 7f4f67fff640 1 -- 172.21.15.19:0/2096273558 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f4f74079580 con 0x7f4f800700a0 2024-04-25T09:53:41.139 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:41.139 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:41.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 -- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f6c06b4d0 msgr2=0x7f4f6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f6c06b4d0 0x7f4f6c06d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4f70000f60 tx=0x7f4f70007c70 comp rx=0 tx=0).stop 2024-04-25T09:53:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 -- 172.21.15.19:0/2096273558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f800700a0 msgr2=0x7f4f800c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f800700a0 0x7f4f800c9e80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4f74035040 tx=0x7f4f740046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 -- 172.21.15.19:0/2096273558 shutdown_connections 2024-04-25T09:53:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f6c06b4d0 0x7f4f6c06d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f800ce330 0x7f4f80160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f800700a0 0x7f4f800c9e80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 --2- 172.21.15.19:0/2096273558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8006abe0 0x7f4f8014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 -- 172.21.15.19:0/2096273558 >> 172.21.15.19:0/2096273558 conn(0x7f4f800b2220 msgr2=0x7f4f800ad500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 -- 172.21.15.19:0/2096273558 shutdown_connections 2024-04-25T09:53:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.139+0000 7f4f85b09640 1 -- 172.21.15.19:0/2096273558 wait complete. 2024-04-25T09:53:41.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:41.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:41.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.255+0000 7fab61c38640 1 Processor -- start 2024-04-25T09:53:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.255+0000 7fab61c38640 1 -- start start 2024-04-25T09:53:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab61c38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 0x7fab5c161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab61c38640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fab5c161ba0 0x7fab5c0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab61c38640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5c162040 con 0x7fab5c0b9750 2024-04-25T09:53:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab61c38640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5c162600 con 0x7fab5c161ba0 2024-04-25T09:53:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab61c38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c162320 con 0x7fab5c15da90 2024-04-25T09:53:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab5bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fab5c161ba0 0x7fab5c0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 0x7fab5c161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 0x7fab5c161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:35426/0 (socket says 172.21.15.19:35426) 2024-04-25T09:53:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab5bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fab5c161ba0 0x7fab5c0d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:59646/0 (socket says 172.21.15.19:59646) 2024-04-25T09:53:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2602264222 learned_addr learned my addr 172.21.15.19:0/2602264222 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab5affd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fab5c0b9750 0x7fab5c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:55946/0 (socket says 172.21.15.19:55946) 2024-04-25T09:53:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2602264222 >> v2:172.21.15.112:3300/0 conn(0x7fab5c161ba0 msgr2=0x7fab5c0d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/2602264222 >> v2:172.21.15.112:3300/0 conn(0x7fab5c161ba0 0x7fab5c0d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2602264222 >> v1:172.21.15.112:6789/0 conn(0x7fab5c0b9750 legacy=0x7fab5c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab5b7fe640 1 -- 172.21.15.19:0/2602264222 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0d1b10 con 0x7fab5c15da90 2024-04-25T09:53:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.256+0000 7fab5bfff640 1 --2- 172.21.15.19:0/2602264222 >> v2:172.21.15.112:3300/0 conn(0x7fab5c161ba0 0x7fab5c0d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.257+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/2602264222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 0x7fab5c161660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009870 tx=0x7fab4c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85960b9865507681 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.257+0000 7fab5a7fc640 1 -- 172.21.15.19:0/2602264222 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c037070 con 0x7fab5c15da90 2024-04-25T09:53:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.257+0000 7fab5a7fc640 1 -- 172.21.15.19:0/2602264222 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c032440 con 0x7fab5c15da90 2024-04-25T09:53:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.257+0000 7fab5a7fc640 1 -- 172.21.15.19:0/2602264222 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c032810 con 0x7fab5c15da90 2024-04-25T09:53:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.258+0000 7fab61c38640 1 -- 172.21.15.19:0/2602264222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 msgr2=0x7fab5c161660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.258+0000 7fab61c38640 1 --2- 172.21.15.19:0/2602264222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 0x7fab5c161660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009870 tx=0x7fab4c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.258+0000 7fab61c38640 1 -- 172.21.15.19:0/2602264222 shutdown_connections 2024-04-25T09:53:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.258+0000 7fab61c38640 1 --2- 172.21.15.19:0/2602264222 >> v2:172.21.15.112:3300/0 conn(0x7fab5c161ba0 0x7fab5c0d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.258+0000 7fab61c38640 1 --2- 172.21.15.19:0/2602264222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 0x7fab5c161660 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.258+0000 7fab61c38640 1 -- 172.21.15.19:0/2602264222 >> 172.21.15.19:0/2602264222 conn(0x7fab5c061a20 msgr2=0x7fab5c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.258+0000 7fab61c38640 1 -- 172.21.15.19:0/2602264222 shutdown_connections 2024-04-25T09:53:41.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.258+0000 7fab61c38640 1 -- 172.21.15.19:0/2602264222 wait complete. 2024-04-25T09:53:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.259+0000 7fab61c38640 1 Processor -- start 2024-04-25T09:53:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.259+0000 7fab61c38640 1 -- start start 2024-04-25T09:53:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab61c38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 0x7fab5c1608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab61c38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c161ba0 0x7fab5c15c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab61c38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5c15e030 0x7fab5c0c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab61c38640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0ced80 con 0x7fab5c161ba0 2024-04-25T09:53:41.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab61c38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c15de70 con 0x7fab5c15e030 2024-04-25T09:53:41.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab61c38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0cef00 con 0x7fab5c15da90 2024-04-25T09:53:41.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c161ba0 0x7fab5c15c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 0x7fab5c1608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c161ba0 0x7fab5c15c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60692/0 (socket says 172.21.15.19:60692) 2024-04-25T09:53:41.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab5bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5c15e030 0x7fab5c0c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab5affd640 1 -- 172.21.15.19:0/3889591796 learned_addr learned my addr 172.21.15.19:0/3889591796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:41.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab5affd640 1 -- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 msgr2=0x7fab5c1608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab5affd640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 0x7fab5c1608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab5affd640 1 -- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5c15e030 msgr2=0x7fab5c0c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab5affd640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5c15e030 0x7fab5c0c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab5affd640 1 -- 172.21.15.19:0/3889591796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c15c950 con 0x7fab5c161ba0 2024-04-25T09:53:41.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.260+0000 7fab5bfff640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5c15e030 0x7fab5c0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:41.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.261+0000 7fab5affd640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c161ba0 0x7fab5c15c410 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fab5000c3c0 tx=0x7fab5000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.261+0000 7fab43fff640 1 -- 172.21.15.19:0/3889591796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab500074e0 con 0x7fab5c161ba0 2024-04-25T09:53:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.261+0000 7fab43fff640 1 -- 172.21.15.19:0/3889591796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab50007680 con 0x7fab5c161ba0 2024-04-25T09:53:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.261+0000 7fab61c38640 1 -- 172.21.15.19:0/3889591796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c0cd570 con 0x7fab5c161ba0 2024-04-25T09:53:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.261+0000 7fab43fff640 1 -- 172.21.15.19:0/3889591796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50004a00 con 0x7fab5c161ba0 2024-04-25T09:53:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.261+0000 7fab61c38640 1 -- 172.21.15.19:0/3889591796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c0cdb30 con 0x7fab5c161ba0 2024-04-25T09:53:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.262+0000 7fab61c38640 1 -- 172.21.15.19:0/3889591796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c073a00 con 0x7fab5c161ba0 2024-04-25T09:53:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.262+0000 7fab43fff640 1 -- 172.21.15.19:0/3889591796 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab5001a020 con 0x7fab5c161ba0 2024-04-25T09:53:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.263+0000 7fab43fff640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab4806b4d0 0x7fab4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.263+0000 7fab43fff640 1 -- 172.21.15.19:0/3889591796 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fab50089b60 con 0x7fab5c161ba0 2024-04-25T09:53:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.263+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab4806b4d0 0x7fab4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.264+0000 7fab5b7fe640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab4806b4d0 0x7fab4806d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab4c034000 tx=0x7fab4c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.267+0000 7fab43fff640 1 -- 172.21.15.19:0/3889591796 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab500586a0 con 0x7fab5c161ba0 2024-04-25T09:53:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.446+0000 7fab61c38640 1 -- 172.21.15.19:0/3889591796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab5c061e00 con 0x7fab5c161ba0 2024-04-25T09:53:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.446+0000 7fab43fff640 1 -- 172.21.15.19:0/3889591796 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab50057fa0 con 0x7fab5c161ba0 2024-04-25T09:53:41.448 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 -- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab4806b4d0 msgr2=0x7fab4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab4806b4d0 0x7fab4806d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab4c034000 tx=0x7fab4c02e000 comp rx=0 tx=0).stop 2024-04-25T09:53:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 -- 172.21.15.19:0/3889591796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c161ba0 msgr2=0x7fab5c15c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c161ba0 0x7fab5c15c410 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fab5000c3c0 tx=0x7fab5000c7c0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 -- 172.21.15.19:0/3889591796 shutdown_connections 2024-04-25T09:53:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab4806b4d0 0x7fab4806d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5c15e030 0x7fab5c0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5c161ba0 0x7fab5c15c410 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 --2- 172.21.15.19:0/3889591796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab5c15da90 0x7fab5c1608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 -- 172.21.15.19:0/3889591796 >> 172.21.15.19:0/3889591796 conn(0x7fab5c061a20 msgr2=0x7fab5c0d2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 -- 172.21.15.19:0/3889591796 shutdown_connections 2024-04-25T09:53:41.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.448+0000 7fab61c38640 1 -- 172.21.15.19:0/3889591796 wait complete. 2024-04-25T09:53:41.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:41.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.563+0000 7f0d1d3a9640 1 Processor -- start 2024-04-25T09:53:41.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.563+0000 7f0d1d3a9640 1 -- start start 2024-04-25T09:53:41.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.563+0000 7f0d1d3a9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0d180b5620 0x7f0d180b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.563+0000 7f0d1d3a9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d1806c2d0 0x7f0d1806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.563+0000 7f0d1d3a9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1806cc80 con 0x7f0d1815b7d0 2024-04-25T09:53:41.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.563+0000 7f0d1d3a9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1806cf50 con 0x7f0d180b5620 2024-04-25T09:53:41.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.563+0000 7f0d1d3a9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d180ce530 con 0x7f0d1806c2d0 2024-04-25T09:53:41.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.564+0000 7f0d177fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d1806c2d0 0x7f0d1806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.564+0000 7f0d16ffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0d1815b7d0 0x7f0d180cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:55958/0 (socket says 172.21.15.19:55958) 2024-04-25T09:53:41.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.564+0000 7f0d16ffd640 1 -- 172.21.15.19:0/1079991980 learned_addr learned my addr 172.21.15.19:0/1079991980 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:41.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.564+0000 7f0d0e7fc640 1 --2- 172.21.15.19:0/1079991980 >> v2:172.21.15.112:3300/0 conn(0x7f0d180b5620 0x7f0d180b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.564+0000 7f0d167fc640 1 -- 172.21.15.19:0/1079991980 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874241401 0 0) 0x7f0d1806cc80 con 0x7f0d1815b7d0 2024-04-25T09:53:41.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.564+0000 7f0d0e7fc640 1 -- 172.21.15.19:0/1079991980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d1806c2d0 msgr2=0x7f0d1806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:41.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.564+0000 7f0d0e7fc640 1 --2- 172.21.15.19:0/1079991980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d1806c2d0 0x7f0d1806c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.564+0000 7f0d0e7fc640 1 -- 172.21.15.19:0/1079991980 >> v1:172.21.15.112:6789/0 conn(0x7f0d1815b7d0 legacy=0x7f0d180cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.564+0000 7f0d0e7fc640 1 -- 172.21.15.19:0/1079991980 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180ce810 con 0x7f0d180b5620 2024-04-25T09:53:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.565+0000 7f0d0e7fc640 1 --2- 172.21.15.19:0/1079991980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d180b5620 0x7f0d180b5ab0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d000097a0 tx=0x7f0d0002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27efac193059367d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.565+0000 7f0d167fc640 1 -- 172.21.15.19:0/1079991980 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00037070 con 0x7f0d180b5620 2024-04-25T09:53:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.565+0000 7f0d167fc640 1 -- 172.21.15.19:0/1079991980 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d00032440 con 0x7f0d180b5620 2024-04-25T09:53:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.565+0000 7f0d167fc640 1 -- 172.21.15.19:0/1079991980 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00032810 con 0x7f0d180b5620 2024-04-25T09:53:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.565+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/1079991980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d180b5620 msgr2=0x7f0d180b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.565+0000 7f0d1d3a9640 1 --2- 172.21.15.19:0/1079991980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d180b5620 0x7f0d180b5ab0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d000097a0 tx=0x7f0d0002a180 comp rx=0 tx=0).stop 2024-04-25T09:53:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.565+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/1079991980 shutdown_connections 2024-04-25T09:53:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.565+0000 7f0d1d3a9640 1 --2- 172.21.15.19:0/1079991980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d1806c2d0 0x7f0d1806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.566+0000 7f0d1d3a9640 1 --2- 172.21.15.19:0/1079991980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d180b5620 0x7f0d180b5ab0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.566+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/1079991980 >> 172.21.15.19:0/1079991980 conn(0x7f0d180d0800 msgr2=0x7f0d180d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.566+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/1079991980 shutdown_connections 2024-04-25T09:53:41.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.566+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/1079991980 wait complete. 2024-04-25T09:53:41.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.567+0000 7f0d1d3a9640 1 Processor -- start 2024-04-25T09:53:41.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.567+0000 7f0d1d3a9640 1 -- start start 2024-04-25T09:53:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d1d3a9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d1806c2d0 0x7f0d180ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d1d3a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d180b5620 0x7f0d180b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d1d3a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d1815b7d0 0x7f0d18147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d1d3a9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1815bbb0 con 0x7f0d180b5620 2024-04-25T09:53:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d1d3a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1806d230 con 0x7f0d1815b7d0 2024-04-25T09:53:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d1d3a9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1815bd30 con 0x7f0d1806c2d0 2024-04-25T09:53:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d180b5620 0x7f0d180b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d180b5620 0x7f0d180b4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60708/0 (socket says 172.21.15.19:60708) 2024-04-25T09:53:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0e7fc640 1 -- 172.21.15.19:0/3231407201 learned_addr learned my addr 172.21.15.19:0/3231407201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d177fe640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d1815b7d0 0x7f0d18147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d16ffd640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d1806c2d0 0x7f0d180ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0e7fc640 1 -- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d1806c2d0 msgr2=0x7f0d180ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0e7fc640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d1806c2d0 0x7f0d180ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0e7fc640 1 -- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d1815b7d0 msgr2=0x7f0d18147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0e7fc640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d1815b7d0 0x7f0d18147fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0e7fc640 1 -- 172.21.15.19:0/3231407201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d181f3620 con 0x7f0d180b5620 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d16ffd640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d1806c2d0 0x7f0d180ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0e7fc640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d180b5620 0x7f0d180b4a90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d000097a0 tx=0x7f0d000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0ffff640 1 -- 172.21.15.19:0/3231407201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00037050 con 0x7f0d180b5620 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0ffff640 1 -- 172.21.15.19:0/3231407201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d00038040 con 0x7f0d180b5620 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d0ffff640 1 -- 172.21.15.19:0/3231407201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00004490 con 0x7f0d180b5620 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.568+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/3231407201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d180cb050 con 0x7f0d180b5620 2024-04-25T09:53:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.569+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/3231407201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d180b3fb0 con 0x7f0d180b5620 2024-04-25T09:53:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.570+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/3231407201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d18073a00 con 0x7f0d180b5620 2024-04-25T09:53:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.570+0000 7f0d0ffff640 1 -- 172.21.15.19:0/3231407201 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d0003c6d0 con 0x7f0d180b5620 2024-04-25T09:53:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.570+0000 7f0d0ffff640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0cf806b500 0x7f0cf806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.571+0000 7f0d0ffff640 1 -- 172.21.15.19:0/3231407201 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0d000aaec0 con 0x7f0d180b5620 2024-04-25T09:53:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.571+0000 7f0d16ffd640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0cf806b500 0x7f0cf806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.571+0000 7f0d16ffd640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0cf806b500 0x7f0cf806d9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0d08000f60 tx=0x7f0d08007ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.575+0000 7f0d0ffff640 1 -- 172.21.15.19:0/3231407201 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d00079a00 con 0x7f0d180b5620 2024-04-25T09:53:41.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.753+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/3231407201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d1815c890 con 0x7f0d180b5620 2024-04-25T09:53:41.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.753+0000 7f0d0ffff640 1 -- 172.21.15.19:0/3231407201 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d00079300 con 0x7f0d180b5620 2024-04-25T09:53:41.755 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:41.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0cf806b500 msgr2=0x7f0cf806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0cf806b500 0x7f0cf806d9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0d08000f60 tx=0x7f0d08007ca0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/3231407201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d180b5620 msgr2=0x7f0d180b4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d180b5620 0x7f0d180b4a90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d000097a0 tx=0x7f0d000046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/3231407201 shutdown_connections 2024-04-25T09:53:41.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0cf806b500 0x7f0cf806d9c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d1815b7d0 0x7f0d18147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d180b5620 0x7f0d180b4a90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 --2- 172.21.15.19:0/3231407201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d1806c2d0 0x7f0d180ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/3231407201 >> 172.21.15.19:0/3231407201 conn(0x7f0d180d0800 msgr2=0x7f0d1815b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/3231407201 shutdown_connections 2024-04-25T09:53:41.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.755+0000 7f0d1d3a9640 1 -- 172.21.15.19:0/3231407201 wait complete. 2024-04-25T09:53:41.768 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:41.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.869+0000 7f6034cee640 1 Processor -- start 2024-04-25T09:53:41.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.869+0000 7f6034cee640 1 -- start start 2024-04-25T09:53:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.870+0000 7f6034cee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60300cadb0 0x7f60300cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.870+0000 7f6034cee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f60300ccfe0 0x7f60300cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.870+0000 7f6034cee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60300afc80 con 0x7f603015de80 2024-04-25T09:53:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.870+0000 7f6034cee640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60300cb210 con 0x7f60300ccfe0 2024-04-25T09:53:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.870+0000 7f6034cee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60300ce4d0 con 0x7f60300cadb0 2024-04-25T09:53:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.871+0000 7f602ed9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f60300ccfe0 0x7f60300cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.871+0000 7f602dd9b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60300cadb0 0x7f60300cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.871+0000 7f602e59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f603015de80 0x7f60300cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:55974/0 (socket says 172.21.15.19:55974) 2024-04-25T09:53:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.871+0000 7f602e59c640 1 -- 172.21.15.19:0/3268200622 learned_addr learned my addr 172.21.15.19:0/3268200622 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.871+0000 7f602ed9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f60300ccfe0 0x7f60300cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:59680/0 (socket says 172.21.15.19:59680) 2024-04-25T09:53:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.871+0000 7f602ed9d640 1 -- 172.21.15.19:0/3268200622 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60300cadb0 msgr2=0x7f60300cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.871+0000 7f602ed9d640 1 --2- 172.21.15.19:0/3268200622 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60300cadb0 0x7f60300cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.871+0000 7f602ed9d640 1 -- 172.21.15.19:0/3268200622 >> v1:172.21.15.112:6789/0 conn(0x7f603015de80 legacy=0x7f60300cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.871+0000 7f602ed9d640 1 -- 172.21.15.19:0/3268200622 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300ce7b0 con 0x7f60300ccfe0 2024-04-25T09:53:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.871+0000 7f602d59a640 1 -- 172.21.15.19:0/3268200622 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186464837 0 0) 0x7f60300afc80 con 0x7f603015de80 2024-04-25T09:53:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.871+0000 7f602dd9b640 1 --2- 172.21.15.19:0/3268200622 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60300cadb0 0x7f60300cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.872+0000 7f602ed9d640 1 --2- 172.21.15.19:0/3268200622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60300ccfe0 0x7f60300cd470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6020009d90 tx=0x7f602002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b51535f632c455e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.872+0000 7f602d59a640 1 -- 172.21.15.19:0/3268200622 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020007740 con 0x7f60300ccfe0 2024-04-25T09:53:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.872+0000 7f602d59a640 1 -- 172.21.15.19:0/3268200622 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60200078e0 con 0x7f60300ccfe0 2024-04-25T09:53:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.872+0000 7f602d59a640 1 -- 172.21.15.19:0/3268200622 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020007cb0 con 0x7f60300ccfe0 2024-04-25T09:53:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.872+0000 7f6034cee640 1 -- 172.21.15.19:0/3268200622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60300ccfe0 msgr2=0x7f60300cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.872+0000 7f6034cee640 1 --2- 172.21.15.19:0/3268200622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60300ccfe0 0x7f60300cd470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6020009d90 tx=0x7f602002a660 comp rx=0 tx=0).stop 2024-04-25T09:53:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.873+0000 7f6034cee640 1 -- 172.21.15.19:0/3268200622 shutdown_connections 2024-04-25T09:53:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.873+0000 7f6034cee640 1 --2- 172.21.15.19:0/3268200622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60300ccfe0 0x7f60300cd470 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.873+0000 7f6034cee640 1 --2- 172.21.15.19:0/3268200622 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60300cadb0 0x7f60300cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.873+0000 7f6034cee640 1 -- 172.21.15.19:0/3268200622 >> 172.21.15.19:0/3268200622 conn(0x7f60300b2390 msgr2=0x7f60300b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.873+0000 7f6034cee640 1 -- 172.21.15.19:0/3268200622 shutdown_connections 2024-04-25T09:53:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.873+0000 7f6034cee640 1 -- 172.21.15.19:0/3268200622 wait complete. 2024-04-25T09:53:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.874+0000 7f6034cee640 1 Processor -- start 2024-04-25T09:53:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.874+0000 7f6034cee640 1 -- start start 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f6034cee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60300cadb0 0x7f60300b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f6034cee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60300ccfe0 0x7f603006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f6034cee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603015de80 0x7f60301e3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f6034cee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603015e260 con 0x7f603015de80 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f6034cee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300cfc00 con 0x7f60300ccfe0 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f6034cee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60300b2eb0 con 0x7f60300cadb0 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f602ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603015de80 0x7f60301e3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f602ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603015de80 0x7f60301e3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60724/0 (socket says 172.21.15.19:60724) 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f602ed9d640 1 -- 172.21.15.19:0/1921209415 learned_addr learned my addr 172.21.15.19:0/1921209415 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f602e59c640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60300cadb0 0x7f60300b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f602dd9b640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60300ccfe0 0x7f603006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f602ed9d640 1 -- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60300cadb0 msgr2=0x7f60300b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f602ed9d640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60300cadb0 0x7f60300b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f602ed9d640 1 -- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60300ccfe0 msgr2=0x7f603006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f602ed9d640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60300ccfe0 0x7f603006f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f602ed9d640 1 -- 172.21.15.19:0/1921209415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300b13a0 con 0x7f603015de80 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f602ed9d640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603015de80 0x7f60301e3be0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6020009eb0 tx=0x7f6020004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f6026ffd640 1 -- 172.21.15.19:0/1921209415 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020040070 con 0x7f603015de80 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f6026ffd640 1 -- 172.21.15.19:0/1921209415 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6020031bc0 con 0x7f603015de80 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f6026ffd640 1 -- 172.21.15.19:0/1921209415 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020009450 con 0x7f603015de80 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.875+0000 7f6034cee640 1 -- 172.21.15.19:0/1921209415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60300b1680 con 0x7f603015de80 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.876+0000 7f6034cee640 1 -- 172.21.15.19:0/1921209415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60300b1c40 con 0x7f603015de80 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.876+0000 7f6034cee640 1 -- 172.21.15.19:0/1921209415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6030073a00 con 0x7f603015de80 2024-04-25T09:53:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.877+0000 7f6026ffd640 1 -- 172.21.15.19:0/1921209415 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6020039040 con 0x7f603015de80 2024-04-25T09:53:41.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.877+0000 7f6026ffd640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601006b500 0x7f601006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.878+0000 7f6026ffd640 1 -- 172.21.15.19:0/1921209415 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f60200abcd0 con 0x7f603015de80 2024-04-25T09:53:41.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.878+0000 7f602e59c640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601006b500 0x7f601006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.878+0000 7f602e59c640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601006b500 0x7f601006d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6028007fd0 tx=0x7f6028007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:41.882+0000 7f6026ffd640 1 -- 172.21.15.19:0/1921209415 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f602007a1c0 con 0x7f603015de80 2024-04-25T09:53:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.063+0000 7f6034cee640 1 -- 172.21.15.19:0/1921209415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f60300c5ce0 con 0x7f603015de80 2024-04-25T09:53:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.063+0000 7f6026ffd640 1 -- 172.21.15.19:0/1921209415 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f602007e260 con 0x7f603015de80 2024-04-25T09:53:42.065 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 -- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601006b500 msgr2=0x7f601006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601006b500 0x7f601006d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6028007fd0 tx=0x7f6028007de0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 -- 172.21.15.19:0/1921209415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603015de80 msgr2=0x7f60301e3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603015de80 0x7f60301e3be0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6020009eb0 tx=0x7f6020004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 -- 172.21.15.19:0/1921209415 shutdown_connections 2024-04-25T09:53:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601006b500 0x7f601006d9c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603015de80 0x7f60301e3be0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60300ccfe0 0x7f603006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 --2- 172.21.15.19:0/1921209415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60300cadb0 0x7f60300b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 -- 172.21.15.19:0/1921209415 >> 172.21.15.19:0/1921209415 conn(0x7f60300b2390 msgr2=0x7f6030158700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 -- 172.21.15.19:0/1921209415 shutdown_connections 2024-04-25T09:53:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.065+0000 7f6034cee640 1 -- 172.21.15.19:0/1921209415 wait complete. 2024-04-25T09:53:42.079 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:42.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.181+0000 7f773a89a640 1 Processor -- start 2024-04-25T09:53:42.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.181+0000 7f773a89a640 1 -- start start 2024-04-25T09:53:42.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f773a89a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f77340ced80 0x7f77340d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f773a89a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773415e000 con 0x7f7734161710 2024-04-25T09:53:42.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f773a89a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77340d1de0 con 0x7f773415dc20 2024-04-25T09:53:42.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f773a89a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7734161c60 con 0x7f77340ced80 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f77337fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f77340ced80 0x7f77340d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f77337fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f77340ced80 0x7f77340d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60732/0 (socket says 172.21.15.19:60732) 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f7738e37640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7734161710 0x7f77340d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57320/0 (socket says 172.21.15.19:57320) 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f77337fe640 1 -- 172.21.15.19:0/4276967132 learned_addr learned my addr 172.21.15.19:0/4276967132 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f77337fe640 1 -- 172.21.15.19:0/4276967132 >> v1:172.21.15.112:6789/0 conn(0x7f773415dc20 legacy=0x7f77340ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f77337fe640 1 -- 172.21.15.19:0/4276967132 >> v1:172.21.15.19:6789/0 conn(0x7f7734161710 legacy=0x7f77340d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f7732ffd640 1 -- 172.21.15.19:0/4276967132 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017558178 0 0) 0x7f773415e000 con 0x7f7734161710 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.182+0000 7f77337fe640 1 -- 172.21.15.19:0/4276967132 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7734161f40 con 0x7f77340ced80 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.183+0000 7f77337fe640 1 --2- 172.21.15.19:0/4276967132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77340ced80 0x7f77340d22e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7724009870 tx=0x7f772402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73d668bc33bbf8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.183+0000 7f7732ffd640 1 -- 172.21.15.19:0/4276967132 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724037070 con 0x7f77340ced80 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.183+0000 7f7732ffd640 1 -- 172.21.15.19:0/4276967132 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772402ae50 con 0x7f77340ced80 2024-04-25T09:53:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.183+0000 7f7732ffd640 1 -- 172.21.15.19:0/4276967132 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724032860 con 0x7f77340ced80 2024-04-25T09:53:42.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.184+0000 7f773a89a640 1 -- 172.21.15.19:0/4276967132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77340ced80 msgr2=0x7f77340d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.184+0000 7f773a89a640 1 --2- 172.21.15.19:0/4276967132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77340ced80 0x7f77340d22e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7724009870 tx=0x7f772402a300 comp rx=0 tx=0).stop 2024-04-25T09:53:42.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.184+0000 7f773a89a640 1 -- 172.21.15.19:0/4276967132 shutdown_connections 2024-04-25T09:53:42.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.184+0000 7f773a89a640 1 --2- 172.21.15.19:0/4276967132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77340ced80 0x7f77340d22e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.184+0000 7f773a89a640 1 -- 172.21.15.19:0/4276967132 >> 172.21.15.19:0/4276967132 conn(0x7f77340d0420 msgr2=0x7f77340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:42.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.184+0000 7f773a89a640 1 -- 172.21.15.19:0/4276967132 shutdown_connections 2024-04-25T09:53:42.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.184+0000 7f773a89a640 1 -- 172.21.15.19:0/4276967132 wait complete. 2024-04-25T09:53:42.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.185+0000 7f773a89a640 1 Processor -- start 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.185+0000 7f773a89a640 1 -- start start 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f773a89a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77340ced80 0x7f7734154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f773a89a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f773415dc20 0x7f77340c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f773a89a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7734161710 0x7f773415f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f773a89a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340b5e50 con 0x7f77340ced80 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f773a89a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7734162500 con 0x7f773415dc20 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f773a89a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77340b5fd0 con 0x7f7734161710 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f7733fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77340ced80 0x7f7734154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f7733fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77340ced80 0x7f7734154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60744/0 (socket says 172.21.15.19:60744) 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f7733fff640 1 -- 172.21.15.19:0/3801058778 learned_addr learned my addr 172.21.15.19:0/3801058778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f77337fe640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f773415dc20 0x7f77340c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f7738e37640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7734161710 0x7f773415f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f7733fff640 1 -- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7734161710 msgr2=0x7f773415f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f7733fff640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7734161710 0x7f773415f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f7733fff640 1 -- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f773415dc20 msgr2=0x7f77340c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f7733fff640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f773415dc20 0x7f77340c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.186+0000 7f7733fff640 1 -- 172.21.15.19:0/3801058778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77340c7c00 con 0x7f77340ced80 2024-04-25T09:53:42.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.187+0000 7f7733fff640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77340ced80 0x7f7734154400 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f772800c450 tx=0x7f772800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.187+0000 7f7730ff9640 1 -- 172.21.15.19:0/3801058778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77280052d0 con 0x7f77340ced80 2024-04-25T09:53:42.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.187+0000 7f7730ff9640 1 -- 172.21.15.19:0/3801058778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7728005470 con 0x7f77340ced80 2024-04-25T09:53:42.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.187+0000 7f773a89a640 1 -- 172.21.15.19:0/3801058778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77340c7ee0 con 0x7f77340ced80 2024-04-25T09:53:42.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.187+0000 7f7730ff9640 1 -- 172.21.15.19:0/3801058778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728012870 con 0x7f77340ced80 2024-04-25T09:53:42.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.187+0000 7f773a89a640 1 -- 172.21.15.19:0/3801058778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77340c8710 con 0x7f77340ced80 2024-04-25T09:53:42.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.188+0000 7f773a89a640 1 -- 172.21.15.19:0/3801058778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7734073a00 con 0x7f77340ced80 2024-04-25T09:53:42.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.188+0000 7f7730ff9640 1 -- 172.21.15.19:0/3801058778 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7728012a10 con 0x7f77340ced80 2024-04-25T09:53:42.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.189+0000 7f7730ff9640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f772006b500 0x7f772006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.189+0000 7f7730ff9640 1 -- 172.21.15.19:0/3801058778 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7728089f40 con 0x7f77340ced80 2024-04-25T09:53:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.189+0000 7f77337fe640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f772006b500 0x7f772006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.189+0000 7f77337fe640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f772006b500 0x7f772006d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7724034000 tx=0x7f772402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.193+0000 7f7730ff9640 1 -- 172.21.15.19:0/3801058778 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f77280584c0 con 0x7f77340ced80 2024-04-25T09:53:42.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.372+0000 7f773a89a640 1 -- 172.21.15.19:0/3801058778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f773415b590 con 0x7f77340ced80 2024-04-25T09:53:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.373+0000 7f7730ff9640 1 -- 172.21.15.19:0/3801058778 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f772805c560 con 0x7f77340ced80 2024-04-25T09:53:42.375 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 -- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f772006b500 msgr2=0x7f772006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f772006b500 0x7f772006d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7724034000 tx=0x7f772402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 -- 172.21.15.19:0/3801058778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77340ced80 msgr2=0x7f7734154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77340ced80 0x7f7734154400 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f772800c450 tx=0x7f772800c910 comp rx=0 tx=0).stop 2024-04-25T09:53:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 -- 172.21.15.19:0/3801058778 shutdown_connections 2024-04-25T09:53:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f772006b500 0x7f772006d9c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7734161710 0x7f773415f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f773415dc20 0x7f77340c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 --2- 172.21.15.19:0/3801058778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77340ced80 0x7f7734154400 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 -- 172.21.15.19:0/3801058778 >> 172.21.15.19:0/3801058778 conn(0x7f77340d0420 msgr2=0x7f77340d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:42.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 -- 172.21.15.19:0/3801058778 shutdown_connections 2024-04-25T09:53:42.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.375+0000 7f773a89a640 1 -- 172.21.15.19:0/3801058778 wait complete. 2024-04-25T09:53:42.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:42.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.487+0000 7f34ccf70640 1 Processor -- start 2024-04-25T09:53:42.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.487+0000 7f34ccf70640 1 -- start start 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.488+0000 7f34ccf70640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f34c80b6cb0 0x7f34c80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.488+0000 7f34ccf70640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34c80b5a60 0x7f34c80d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.488+0000 7f34ccf70640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c80ceee0 con 0x7f34c815db70 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.488+0000 7f34ccf70640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c80d1d10 con 0x7f34c80b6cb0 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.488+0000 7f34ccf70640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80d1ff0 con 0x7f34c80b5a60 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.488+0000 7f34c659c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f34c815db70 0x7f34c8162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57332/0 (socket says 172.21.15.19:57332) 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.488+0000 7f34c659c640 1 -- 172.21.15.19:0/4227440450 learned_addr learned my addr 172.21.15.19:0/4227440450 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.488+0000 7f34c6d9d640 1 --2- 172.21.15.19:0/4227440450 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34c80b5a60 0x7f34c80d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.488+0000 7f34c5d9b640 1 --2- 172.21.15.19:0/4227440450 >> v2:172.21.15.112:3300/0 conn(0x7f34c80b6cb0 0x7f34c80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.488+0000 7f34c559a640 1 -- 172.21.15.19:0/4227440450 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498755938 0 0) 0x7f34c80ceee0 con 0x7f34c815db70 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.488+0000 7f34c559a640 1 -- 172.21.15.19:0/4227440450 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34b0003520 con 0x7f34c815db70 2024-04-25T09:53:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.489+0000 7f34c559a640 1 -- 172.21.15.19:0/4227440450 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 366670039 0 0) 0x7f34b0003520 con 0x7f34c815db70 2024-04-25T09:53:42.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.489+0000 7f34c5d9b640 1 -- 172.21.15.19:0/4227440450 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34c80b5a60 msgr2=0x7f34c80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:42.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.489+0000 7f34c5d9b640 1 --2- 172.21.15.19:0/4227440450 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34c80b5a60 0x7f34c80d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.489+0000 7f34c5d9b640 1 -- 172.21.15.19:0/4227440450 >> v1:172.21.15.19:6789/0 conn(0x7f34c815db70 legacy=0x7f34c8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.489+0000 7f34c5d9b640 1 -- 172.21.15.19:0/4227440450 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c80d22d0 con 0x7f34c80b6cb0 2024-04-25T09:53:42.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.489+0000 7f34c6d9d640 1 --2- 172.21.15.19:0/4227440450 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34c80b5a60 0x7f34c80d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:42.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.489+0000 7f34c5d9b640 1 --2- 172.21.15.19:0/4227440450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80b6cb0 0x7f34c80b5520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0097a0 tx=0x7f34bc02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7817cb6a63fa0fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.490+0000 7f34c559a640 1 -- 172.21.15.19:0/4227440450 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc037070 con 0x7f34c80b6cb0 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.490+0000 7f34c559a640 1 -- 172.21.15.19:0/4227440450 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34bc02aba0 con 0x7f34c80b6cb0 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.490+0000 7f34c559a640 1 -- 172.21.15.19:0/4227440450 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8001670 con 0x7f34c80b6cb0 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.490+0000 7f34ccf70640 1 -- 172.21.15.19:0/4227440450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80b6cb0 msgr2=0x7f34c80b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.490+0000 7f34ccf70640 1 --2- 172.21.15.19:0/4227440450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80b6cb0 0x7f34c80b5520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0097a0 tx=0x7f34bc02a110 comp rx=0 tx=0).stop 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.490+0000 7f34ccf70640 1 -- 172.21.15.19:0/4227440450 shutdown_connections 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.490+0000 7f34ccf70640 1 --2- 172.21.15.19:0/4227440450 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34c80b5a60 0x7f34c80d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.490+0000 7f34ccf70640 1 --2- 172.21.15.19:0/4227440450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80b6cb0 0x7f34c80b5520 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.490+0000 7f34ccf70640 1 -- 172.21.15.19:0/4227440450 >> 172.21.15.19:0/4227440450 conn(0x7f34c80d0520 msgr2=0x7f34c80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.490+0000 7f34ccf70640 1 -- 172.21.15.19:0/4227440450 shutdown_connections 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.490+0000 7f34ccf70640 1 -- 172.21.15.19:0/4227440450 wait complete. 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.492+0000 7f34ccf70640 1 Processor -- start 2024-04-25T09:53:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.492+0000 7f34ccf70640 1 -- start start 2024-04-25T09:53:42.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.492+0000 7f34ccf70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c80b5a60 0x7f34c80c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.492+0000 7f34ccf70640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34c80b6cb0 0x7f34c815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.492+0000 7f34ccf70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c815db70 0x7f34c815f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.492+0000 7f34ccf70640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80cf1c0 con 0x7f34c80b5a60 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.492+0000 7f34ccf70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80ce530 con 0x7f34c815db70 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.492+0000 7f34ccf70640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80cf340 con 0x7f34c80b6cb0 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.492+0000 7f34c659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c80b5a60 0x7f34c80c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.492+0000 7f34c659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c80b5a60 0x7f34c80c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60752/0 (socket says 172.21.15.19:60752) 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.492+0000 7f34c659c640 1 -- 172.21.15.19:0/2085008432 learned_addr learned my addr 172.21.15.19:0/2085008432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34c5d9b640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34c80b6cb0 0x7f34c815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34c6d9d640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c815db70 0x7f34c815f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34c659c640 1 -- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34c80b6cb0 msgr2=0x7f34c815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34c659c640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34c80b6cb0 0x7f34c815f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34c659c640 1 -- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c815db70 msgr2=0x7f34c815f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34c659c640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c815db70 0x7f34c815f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34c659c640 1 -- 172.21.15.19:0/2085008432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c8157240 con 0x7f34c80b5a60 2024-04-25T09:53:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34c659c640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c80b5a60 0x7f34c80c9850 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f34b800c700 tx=0x7f34b800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34b6ffd640 1 -- 172.21.15.19:0/2085008432 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8017070 con 0x7f34c80b5a60 2024-04-25T09:53:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34b6ffd640 1 -- 172.21.15.19:0/2085008432 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b8005ce0 con 0x7f34c80b5a60 2024-04-25T09:53:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34ccf70640 1 -- 172.21.15.19:0/2085008432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c80cdce0 con 0x7f34c80b5a60 2024-04-25T09:53:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34b6ffd640 1 -- 172.21.15.19:0/2085008432 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8007270 con 0x7f34c80b5a60 2024-04-25T09:53:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.493+0000 7f34ccf70640 1 -- 172.21.15.19:0/2085008432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c81e7540 con 0x7f34c80b5a60 2024-04-25T09:53:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.494+0000 7f34ccf70640 1 -- 172.21.15.19:0/2085008432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c8073990 con 0x7f34c80b5a60 2024-04-25T09:53:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.495+0000 7f34b6ffd640 1 -- 172.21.15.19:0/2085008432 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34b801a070 con 0x7f34c80b5a60 2024-04-25T09:53:42.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.495+0000 7f34b6ffd640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f34a806b4d0 0x7f34a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.495+0000 7f34b6ffd640 1 -- 172.21.15.19:0/2085008432 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f34b80897c0 con 0x7f34c80b5a60 2024-04-25T09:53:42.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.495+0000 7f34c5d9b640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f34a806b4d0 0x7f34a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.496+0000 7f34c5d9b640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f34a806b4d0 0x7f34a806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34bc034000 tx=0x7f34bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.499+0000 7f34b6ffd640 1 -- 172.21.15.19:0/2085008432 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34c8157240 con 0x7f34c80b5a60 2024-04-25T09:53:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.679+0000 7f34ccf70640 1 -- 172.21.15.19:0/2085008432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34c8157d10 con 0x7f34c80b5a60 2024-04-25T09:53:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.680+0000 7f34b6ffd640 1 -- 172.21.15.19:0/2085008432 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f34b8057cc0 con 0x7f34c80b5a60 2024-04-25T09:53:42.683 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:42.683 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 -- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f34a806b4d0 msgr2=0x7f34a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f34a806b4d0 0x7f34a806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34bc034000 tx=0x7f34bc02e000 comp rx=0 tx=0).stop 2024-04-25T09:53:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 -- 172.21.15.19:0/2085008432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c80b5a60 msgr2=0x7f34c80c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c80b5a60 0x7f34c80c9850 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f34b800c700 tx=0x7f34b800cb00 comp rx=0 tx=0).stop 2024-04-25T09:53:42.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 -- 172.21.15.19:0/2085008432 shutdown_connections 2024-04-25T09:53:42.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f34a806b4d0 0x7f34a806d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c815db70 0x7f34c815f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34c80b6cb0 0x7f34c815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 --2- 172.21.15.19:0/2085008432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c80b5a60 0x7f34c80c9850 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 -- 172.21.15.19:0/2085008432 >> 172.21.15.19:0/2085008432 conn(0x7f34c80d0520 msgr2=0x7f34c806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 -- 172.21.15.19:0/2085008432 shutdown_connections 2024-04-25T09:53:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.682+0000 7f34ccf70640 1 -- 172.21.15.19:0/2085008432 wait complete. 2024-04-25T09:53:42.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:42.696 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.797+0000 7fa51179b640 1 Processor -- start 2024-04-25T09:53:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.797+0000 7fa51179b640 1 -- start start 2024-04-25T09:53:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.798+0000 7fa51179b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa50c161890 0x7fa50c0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.798+0000 7fa51179b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c0cf1d0 con 0x7fa50c0b54a0 2024-04-25T09:53:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.798+0000 7fa51179b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c161620 con 0x7fa50c161240 2024-04-25T09:53:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.798+0000 7fa51179b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50c0b7120 con 0x7fa50c161890 2024-04-25T09:53:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.798+0000 7fa50b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa50c0b54a0 0x7fa50c0d1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57346/0 (socket says 172.21.15.19:57346) 2024-04-25T09:53:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.798+0000 7fa50b7fe640 1 -- 172.21.15.19:0/612330311 learned_addr learned my addr 172.21.15.19:0/612330311 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.798+0000 7fa50a7fc640 1 --2- 172.21.15.19:0/612330311 >> v2:172.21.15.112:3300/0 conn(0x7fa50c161890 0x7fa50c0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.799+0000 7fa509ffb640 1 -- 172.21.15.19:0/612330311 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973561019 0 0) 0x7fa50c0cf1d0 con 0x7fa50c0b54a0 2024-04-25T09:53:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.799+0000 7fa509ffb640 1 -- 172.21.15.19:0/612330311 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4f8003520 con 0x7fa50c0b54a0 2024-04-25T09:53:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.799+0000 7fa509ffb640 1 -- 172.21.15.19:0/612330311 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386933815 0 0) 0x7fa50c161620 con 0x7fa50c161240 2024-04-25T09:53:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.799+0000 7fa509ffb640 1 -- 172.21.15.19:0/612330311 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa50c0cf1d0 con 0x7fa50c161240 2024-04-25T09:53:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.799+0000 7fa50a7fc640 1 -- 172.21.15.19:0/612330311 >> v1:172.21.15.112:6789/0 conn(0x7fa50c161240 legacy=0x7fa50c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.799+0000 7fa50a7fc640 1 -- 172.21.15.19:0/612330311 >> v1:172.21.15.19:6789/0 conn(0x7fa50c0b54a0 legacy=0x7fa50c0d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.799+0000 7fa50a7fc640 1 -- 172.21.15.19:0/612330311 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c161e00 con 0x7fa50c161890 2024-04-25T09:53:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.800+0000 7fa50a7fc640 1 --2- 172.21.15.19:0/612330311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa50c161890 0x7fa50c0b6be0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40097a0 tx=0x7fa4f402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f143a9735d485c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.800+0000 7fa509ffb640 1 -- 172.21.15.19:0/612330311 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4037070 con 0x7fa50c161890 2024-04-25T09:53:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.800+0000 7fa509ffb640 1 -- 172.21.15.19:0/612330311 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f4032440 con 0x7fa50c161890 2024-04-25T09:53:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.800+0000 7fa509ffb640 1 -- 172.21.15.19:0/612330311 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4032810 con 0x7fa50c161890 2024-04-25T09:53:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.800+0000 7fa51179b640 1 -- 172.21.15.19:0/612330311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa50c161890 msgr2=0x7fa50c0b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.800+0000 7fa51179b640 1 --2- 172.21.15.19:0/612330311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa50c161890 0x7fa50c0b6be0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40097a0 tx=0x7fa4f402a230 comp rx=0 tx=0).stop 2024-04-25T09:53:42.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.800+0000 7fa51179b640 1 -- 172.21.15.19:0/612330311 shutdown_connections 2024-04-25T09:53:42.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.800+0000 7fa51179b640 1 --2- 172.21.15.19:0/612330311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa50c161890 0x7fa50c0b6be0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.800+0000 7fa51179b640 1 -- 172.21.15.19:0/612330311 >> 172.21.15.19:0/612330311 conn(0x7fa50c0d04a0 msgr2=0x7fa50c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:42.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.800+0000 7fa51179b640 1 -- 172.21.15.19:0/612330311 shutdown_connections 2024-04-25T09:53:42.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.800+0000 7fa51179b640 1 -- 172.21.15.19:0/612330311 wait complete. 2024-04-25T09:53:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.802+0000 7fa51179b640 1 Processor -- start 2024-04-25T09:53:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.802+0000 7fa51179b640 1 -- start start 2024-04-25T09:53:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.802+0000 7fa51179b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa50c0b54a0 0x7fa50c151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.802+0000 7fa51179b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa50c161240 0x7fa50c15f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.802+0000 7fa51179b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa50c161890 0x7fa50c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.802+0000 7fa51179b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c1626a0 con 0x7fa50c161890 2024-04-25T09:53:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.802+0000 7fa51179b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c1623c0 con 0x7fa50c0b54a0 2024-04-25T09:53:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.802+0000 7fa51179b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0d15f0 con 0x7fa50c161240 2024-04-25T09:53:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa50b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa50c161890 0x7fa50c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa50b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa50c161890 0x7fa50c0b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60754/0 (socket says 172.21.15.19:60754) 2024-04-25T09:53:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa50b7fe640 1 -- 172.21.15.19:0/380823105 learned_addr learned my addr 172.21.15.19:0/380823105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa50a7fc640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa50c161240 0x7fa50c15f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa50affd640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa50c0b54a0 0x7fa50c151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa50b7fe640 1 -- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa50c161240 msgr2=0x7fa50c15f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa50b7fe640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa50c161240 0x7fa50c15f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa50b7fe640 1 -- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa50c0b54a0 msgr2=0x7fa50c151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa50b7fe640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa50c0b54a0 0x7fa50c151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa50b7fe640 1 -- 172.21.15.19:0/380823105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c15f960 con 0x7fa50c161890 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa50b7fe640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa50c161890 0x7fa50c0b46d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00cbd0 tx=0x7fa4fc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa5037fe640 1 -- 172.21.15.19:0/380823105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc017070 con 0x7fa50c161890 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa5037fe640 1 -- 172.21.15.19:0/380823105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4fc00ace0 con 0x7fa50c161890 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa51179b640 1 -- 172.21.15.19:0/380823105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50c0b3630 con 0x7fa50c161890 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa5037fe640 1 -- 172.21.15.19:0/380823105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc012620 con 0x7fa50c161890 2024-04-25T09:53:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.803+0000 7fa51179b640 1 -- 172.21.15.19:0/380823105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c0b3bf0 con 0x7fa50c161890 2024-04-25T09:53:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.804+0000 7fa51179b640 1 -- 172.21.15.19:0/380823105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c073a00 con 0x7fa50c161890 2024-04-25T09:53:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.805+0000 7fa5037fe640 1 -- 172.21.15.19:0/380823105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4fc01a070 con 0x7fa50c161890 2024-04-25T09:53:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.805+0000 7fa5037fe640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa4ec06b500 0x7fa4ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.805+0000 7fa5037fe640 1 -- 172.21.15.19:0/380823105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa4fc08a4f0 con 0x7fa50c161890 2024-04-25T09:53:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.805+0000 7fa50affd640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa4ec06b500 0x7fa4ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.806+0000 7fa50affd640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa4ec06b500 0x7fa4ec06d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa5040011e0 tx=0x7fa504009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.810+0000 7fa5037fe640 1 -- 172.21.15.19:0/380823105 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4fc0590c0 con 0x7fa50c161890 2024-04-25T09:53:42.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.991+0000 7fa51179b640 1 -- 172.21.15.19:0/380823105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa50c06d240 con 0x7fa50c161890 2024-04-25T09:53:42.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.991+0000 7fa5037fe640 1 -- 172.21.15.19:0/380823105 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa4fc0589c0 con 0x7fa50c161890 2024-04-25T09:53:42.993 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 -- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa4ec06b500 msgr2=0x7fa4ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa4ec06b500 0x7fa4ec06d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa5040011e0 tx=0x7fa504009040 comp rx=0 tx=0).stop 2024-04-25T09:53:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 -- 172.21.15.19:0/380823105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa50c161890 msgr2=0x7fa50c0b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa50c161890 0x7fa50c0b46d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00cbd0 tx=0x7fa4fc00a430 comp rx=0 tx=0).stop 2024-04-25T09:53:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 -- 172.21.15.19:0/380823105 shutdown_connections 2024-04-25T09:53:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa4ec06b500 0x7fa4ec06d9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa50c161890 0x7fa50c0b46d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa50c161240 0x7fa50c15f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 --2- 172.21.15.19:0/380823105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa50c0b54a0 0x7fa50c151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 -- 172.21.15.19:0/380823105 >> 172.21.15.19:0/380823105 conn(0x7fa50c0d04a0 msgr2=0x7fa50c15d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 -- 172.21.15.19:0/380823105 shutdown_connections 2024-04-25T09:53:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:42.993+0000 7fa51179b640 1 -- 172.21.15.19:0/380823105 wait complete. 2024-04-25T09:53:43.006 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.109+0000 7f9552bbd640 1 Processor -- start 2024-04-25T09:53:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.109+0000 7f9552bbd640 1 -- start start 2024-04-25T09:53:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.109+0000 7f9552bbd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f954c15dc20 0x7f954c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f9552bbd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f954c0d32f0 0x7f954c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f9552bbd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954c15e000 con 0x7f954c161710 2024-04-25T09:53:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f9552bbd640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954c0d1de0 con 0x7f954c0d32f0 2024-04-25T09:53:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f9552bbd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954c161c60 con 0x7f954c15dc20 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f955115a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f954c0d32f0 0x7f954c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f954bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f954c161710 0x7f954c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57354/0 (socket says 172.21.15.19:57354) 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f954bfff640 1 -- 172.21.15.19:0/1250561257 learned_addr learned my addr 172.21.15.19:0/1250561257 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f9550959640 1 --2- 172.21.15.19:0/1250561257 >> v2:172.21.15.112:3300/0 conn(0x7f954c15dc20 0x7f954c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f955115a640 1 -- 172.21.15.19:0/1250561257 >> v2:172.21.15.112:3300/0 conn(0x7f954c15dc20 msgr2=0x7f954c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f955115a640 1 --2- 172.21.15.19:0/1250561257 >> v2:172.21.15.112:3300/0 conn(0x7f954c15dc20 0x7f954c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f955115a640 1 -- 172.21.15.19:0/1250561257 >> v1:172.21.15.19:6789/0 conn(0x7f954c161710 legacy=0x7f954c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f954b7fe640 1 -- 172.21.15.19:0/1250561257 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297893085 0 0) 0x7f954c15e000 con 0x7f954c161710 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.110+0000 7f955115a640 1 -- 172.21.15.19:0/1250561257 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c161f40 con 0x7f954c0d32f0 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f955115a640 1 --2- 172.21.15.19:0/1250561257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954c0d32f0 0x7f954c0cebb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f954c0d1520 tx=0x7f954402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33696b934e864b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f954b7fe640 1 -- 172.21.15.19:0/1250561257 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544007740 con 0x7f954c0d32f0 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f954b7fe640 1 -- 172.21.15.19:0/1250561257 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95440078e0 con 0x7f954c0d32f0 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f954b7fe640 1 -- 172.21.15.19:0/1250561257 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544007cb0 con 0x7f954c0d32f0 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f9552bbd640 1 -- 172.21.15.19:0/1250561257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954c0d32f0 msgr2=0x7f954c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f9552bbd640 1 --2- 172.21.15.19:0/1250561257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954c0d32f0 0x7f954c0cebb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f954c0d1520 tx=0x7f954402a680 comp rx=0 tx=0).stop 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f9552bbd640 1 -- 172.21.15.19:0/1250561257 shutdown_connections 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f9552bbd640 1 --2- 172.21.15.19:0/1250561257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954c0d32f0 0x7f954c0cebb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f9552bbd640 1 --2- 172.21.15.19:0/1250561257 >> v2:172.21.15.112:3300/0 conn(0x7f954c15dc20 0x7f954c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f9552bbd640 1 -- 172.21.15.19:0/1250561257 >> 172.21.15.19:0/1250561257 conn(0x7f954c0d0420 msgr2=0x7f954c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f9552bbd640 1 -- 172.21.15.19:0/1250561257 shutdown_connections 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.111+0000 7f9552bbd640 1 -- 172.21.15.19:0/1250561257 wait complete. 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.113+0000 7f9552bbd640 1 Processor -- start 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.113+0000 7f9552bbd640 1 -- start start 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9552bbd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954c0d32f0 0x7f954c152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9552bbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c15dc20 0x7f954c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9552bbd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f954c161710 0x7f954c157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9552bbd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c0b5e50 con 0x7f954c0d32f0 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9552bbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c162500 con 0x7f954c15dc20 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9552bbd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c0b5fd0 con 0x7f954c161710 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9550959640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954c0d32f0 0x7f954c152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9550959640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954c0d32f0 0x7f954c152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60782/0 (socket says 172.21.15.19:60782) 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9550959640 1 -- 172.21.15.19:0/2235627756 learned_addr learned my addr 172.21.15.19:0/2235627756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9550959640 1 -- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f954c161710 msgr2=0x7f954c157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9550959640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f954c161710 0x7f954c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9550959640 1 -- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c15dc20 msgr2=0x7f954c156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9550959640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c15dc20 0x7f954c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9550959640 1 -- 172.21.15.19:0/2235627756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c157380 con 0x7f954c0d32f0 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9550959640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954c0d32f0 0x7f954c152c50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f953c00d620 tx=0x7f953c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f95497fa640 1 -- 172.21.15.19:0/2235627756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00b620 con 0x7f954c0d32f0 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f95497fa640 1 -- 172.21.15.19:0/2235627756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c019070 con 0x7f954c0d32f0 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9552bbd640 1 -- 172.21.15.19:0/2235627756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c15fec0 con 0x7f954c0d32f0 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f95497fa640 1 -- 172.21.15.19:0/2235627756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00b9e0 con 0x7f954c0d32f0 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.114+0000 7f9552bbd640 1 -- 172.21.15.19:0/2235627756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c160480 con 0x7f954c0d32f0 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.115+0000 7f9552bbd640 1 -- 172.21.15.19:0/2235627756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954c0cf940 con 0x7f954c0d32f0 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.116+0000 7f95497fa640 1 -- 172.21.15.19:0/2235627756 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f953c00bb80 con 0x7f954c0d32f0 2024-04-25T09:53:43.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.116+0000 7f95497fa640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952c06b4d0 0x7f952c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.116+0000 7f95497fa640 1 -- 172.21.15.19:0/2235627756 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f953c08a8d0 con 0x7f954c0d32f0 2024-04-25T09:53:43.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.116+0000 7f954bfff640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952c06b4d0 0x7f952c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.117+0000 7f954bfff640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952c06b4d0 0x7f952c06d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9534007fd0 tx=0x7f9534007960 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.121+0000 7f95497fa640 1 -- 172.21.15.19:0/2235627756 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f954c0cf940 con 0x7f954c0d32f0 2024-04-25T09:53:43.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.299+0000 7f9552bbd640 1 -- 172.21.15.19:0/2235627756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f954c0cf380 con 0x7f954c0d32f0 2024-04-25T09:53:43.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.300+0000 7f95497fa640 1 -- 172.21.15.19:0/2235627756 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f954c0cf380 con 0x7f954c0d32f0 2024-04-25T09:53:43.302 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.301+0000 7f9552bbd640 1 -- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952c06b4d0 msgr2=0x7f952c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.301+0000 7f9552bbd640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952c06b4d0 0x7f952c06d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9534007fd0 tx=0x7f9534007960 comp rx=0 tx=0).stop 2024-04-25T09:53:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.301+0000 7f9552bbd640 1 -- 172.21.15.19:0/2235627756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954c0d32f0 msgr2=0x7f954c152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.301+0000 7f9552bbd640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954c0d32f0 0x7f954c152c50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f953c00d620 tx=0x7f953c00dae0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.302+0000 7f9552bbd640 1 -- 172.21.15.19:0/2235627756 shutdown_connections 2024-04-25T09:53:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.302+0000 7f9552bbd640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952c06b4d0 0x7f952c06d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.302+0000 7f9552bbd640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f954c161710 0x7f954c157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.302+0000 7f9552bbd640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c15dc20 0x7f954c156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.302+0000 7f9552bbd640 1 --2- 172.21.15.19:0/2235627756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954c0d32f0 0x7f954c152c50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.302+0000 7f9552bbd640 1 -- 172.21.15.19:0/2235627756 >> 172.21.15.19:0/2235627756 conn(0x7f954c0d0420 msgr2=0x7f954c0d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.302+0000 7f9552bbd640 1 -- 172.21.15.19:0/2235627756 shutdown_connections 2024-04-25T09:53:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.302+0000 7f9552bbd640 1 -- 172.21.15.19:0/2235627756 wait complete. 2024-04-25T09:53:43.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:43.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.416+0000 7f0ab662c640 1 Processor -- start 2024-04-25T09:53:43.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.416+0000 7f0ab662c640 1 -- start start 2024-04-25T09:53:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.417+0000 7f0ab662c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0ab015dc20 0x7f0ab00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.417+0000 7f0ab662c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0ab00d32f0 0x7f0ab00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.417+0000 7f0ab662c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab015e000 con 0x7f0ab0161710 2024-04-25T09:53:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.417+0000 7f0ab662c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab00d1de0 con 0x7f0ab00d32f0 2024-04-25T09:53:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.417+0000 7f0ab662c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab0161c60 con 0x7f0ab015dc20 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.417+0000 7f0ab4bc9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0ab00d32f0 0x7f0ab00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.417+0000 7f0ab4bc9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0ab00d32f0 0x7f0ab00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60794/0 (socket says 172.21.15.19:60794) 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.417+0000 7f0ab4bc9640 1 -- 172.21.15.19:0/2835162569 learned_addr learned my addr 172.21.15.19:0/2835162569 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.417+0000 7f0aaffff640 1 --2- 172.21.15.19:0/2835162569 >> v2:172.21.15.112:3300/0 conn(0x7f0ab015dc20 0x7f0ab00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.418+0000 7f0ab4bc9640 1 -- 172.21.15.19:0/2835162569 >> v2:172.21.15.112:3300/0 conn(0x7f0ab015dc20 msgr2=0x7f0ab00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.418+0000 7f0ab4bc9640 1 --2- 172.21.15.19:0/2835162569 >> v2:172.21.15.112:3300/0 conn(0x7f0ab015dc20 0x7f0ab00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.418+0000 7f0ab4bc9640 1 -- 172.21.15.19:0/2835162569 >> v1:172.21.15.112:6789/0 conn(0x7f0ab0161710 legacy=0x7f0ab006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.418+0000 7f0ab4bc9640 1 -- 172.21.15.19:0/2835162569 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab0161f40 con 0x7f0ab00d32f0 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.418+0000 7f0ab4bc9640 1 --2- 172.21.15.19:0/2835162569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ab00d32f0 0x7f0ab00cebb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00d1520 tx=0x7f0aa802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d07ce60c302c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.418+0000 7f0aaeffd640 1 -- 172.21.15.19:0/2835162569 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8036070 con 0x7f0ab00d32f0 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.418+0000 7f0aaeffd640 1 -- 172.21.15.19:0/2835162569 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa8031600 con 0x7f0ab00d32f0 2024-04-25T09:53:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.418+0000 7f0aaeffd640 1 -- 172.21.15.19:0/2835162569 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80319d0 con 0x7f0ab00d32f0 2024-04-25T09:53:43.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.419+0000 7f0ab662c640 1 -- 172.21.15.19:0/2835162569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ab00d32f0 msgr2=0x7f0ab00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.419+0000 7f0ab662c640 1 --2- 172.21.15.19:0/2835162569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ab00d32f0 0x7f0ab00cebb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00d1520 tx=0x7f0aa802a730 comp rx=0 tx=0).stop 2024-04-25T09:53:43.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.419+0000 7f0ab662c640 1 -- 172.21.15.19:0/2835162569 shutdown_connections 2024-04-25T09:53:43.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.419+0000 7f0ab662c640 1 --2- 172.21.15.19:0/2835162569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ab00d32f0 0x7f0ab00cebb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.419+0000 7f0ab662c640 1 --2- 172.21.15.19:0/2835162569 >> v2:172.21.15.112:3300/0 conn(0x7f0ab015dc20 0x7f0ab00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.419+0000 7f0ab662c640 1 -- 172.21.15.19:0/2835162569 >> 172.21.15.19:0/2835162569 conn(0x7f0ab00d0420 msgr2=0x7f0ab00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.419+0000 7f0ab662c640 1 -- 172.21.15.19:0/2835162569 shutdown_connections 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.419+0000 7f0ab662c640 1 -- 172.21.15.19:0/2835162569 wait complete. 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.420+0000 7f0ab662c640 1 Processor -- start 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.420+0000 7f0ab662c640 1 -- start start 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0ab662c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ab00d32f0 0x7f0ab00cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0ab662c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ab015dc20 0x7f0ab01f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0ab662c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ab0161710 0x7f0ab01f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0ab662c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab00b5e50 con 0x7f0ab015dc20 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0ab662c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab0162500 con 0x7f0ab00d32f0 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0ab662c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab00b5fd0 con 0x7f0ab0161710 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0aaf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ab015dc20 0x7f0ab01f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0aaf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ab015dc20 0x7f0ab01f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60798/0 (socket says 172.21.15.19:60798) 2024-04-25T09:53:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0aaf7fe640 1 -- 172.21.15.19:0/1093090747 learned_addr learned my addr 172.21.15.19:0/1093090747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0aaffff640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ab00d32f0 0x7f0ab00cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0ab4bc9640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ab0161710 0x7f0ab01f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0aaf7fe640 1 -- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ab0161710 msgr2=0x7f0ab01f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.421+0000 7f0aaf7fe640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ab0161710 0x7f0ab01f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.422+0000 7f0aaf7fe640 1 -- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ab00d32f0 msgr2=0x7f0ab00cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.422+0000 7f0aaf7fe640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ab00d32f0 0x7f0ab00cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.422+0000 7f0aaf7fe640 1 -- 172.21.15.19:0/1093090747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab00cc590 con 0x7f0ab015dc20 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.422+0000 7f0ab4bc9640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ab0161710 0x7f0ab01f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.422+0000 7f0aaf7fe640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ab015dc20 0x7f0ab01f36d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800c3c0 tx=0x7f0a9800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.422+0000 7f0aacff9640 1 -- 172.21.15.19:0/1093090747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a980052d0 con 0x7f0ab015dc20 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.422+0000 7f0aacff9640 1 -- 172.21.15.19:0/1093090747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a98005490 con 0x7f0ab015dc20 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.422+0000 7f0ab662c640 1 -- 172.21.15.19:0/1093090747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab00cc870 con 0x7f0ab015dc20 2024-04-25T09:53:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.422+0000 7f0aacff9640 1 -- 172.21.15.19:0/1093090747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98007260 con 0x7f0ab015dc20 2024-04-25T09:53:43.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.422+0000 7f0ab662c640 1 -- 172.21.15.19:0/1093090747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab00cce30 con 0x7f0ab015dc20 2024-04-25T09:53:43.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.423+0000 7f0ab662c640 1 -- 172.21.15.19:0/1093090747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab00cd030 con 0x7f0ab015dc20 2024-04-25T09:53:43.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.423+0000 7f0aacff9640 1 -- 172.21.15.19:0/1093090747 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a98007400 con 0x7f0ab015dc20 2024-04-25T09:53:43.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.424+0000 7f0aacff9640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a9006b4d0 0x7f0a9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.424+0000 7f0aacff9640 1 -- 172.21.15.19:0/1093090747 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0a98089970 con 0x7f0ab015dc20 2024-04-25T09:53:43.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.424+0000 7f0aaffff640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a9006b4d0 0x7f0a9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.425+0000 7f0aaffff640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a9006b4d0 0x7f0a9006d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00079a0 tx=0x7f0aa0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.428+0000 7f0aacff9640 1 -- 172.21.15.19:0/1093090747 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a98057e60 con 0x7f0ab015dc20 2024-04-25T09:53:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.606+0000 7f0ab662c640 1 -- 172.21.15.19:0/1093090747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ab0073a00 con 0x7f0ab015dc20 2024-04-25T09:53:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.606+0000 7f0aacff9640 1 -- 172.21.15.19:0/1093090747 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a9805ce20 con 0x7f0ab015dc20 2024-04-25T09:53:43.608 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 -- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a9006b4d0 msgr2=0x7f0a9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a9006b4d0 0x7f0a9006d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00079a0 tx=0x7f0aa0008040 comp rx=0 tx=0).stop 2024-04-25T09:53:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 -- 172.21.15.19:0/1093090747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ab015dc20 msgr2=0x7f0ab01f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ab015dc20 0x7f0ab01f36d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800c3c0 tx=0x7f0a9800c880 comp rx=0 tx=0).stop 2024-04-25T09:53:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 -- 172.21.15.19:0/1093090747 shutdown_connections 2024-04-25T09:53:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a9006b4d0 0x7f0a9006d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ab0161710 0x7f0ab01f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ab015dc20 0x7f0ab01f36d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 --2- 172.21.15.19:0/1093090747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ab00d32f0 0x7f0ab00cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 -- 172.21.15.19:0/1093090747 >> 172.21.15.19:0/1093090747 conn(0x7f0ab00d0420 msgr2=0x7f0ab015c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 -- 172.21.15.19:0/1093090747 shutdown_connections 2024-04-25T09:53:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.608+0000 7f0ab662c640 1 -- 172.21.15.19:0/1093090747 wait complete. 2024-04-25T09:53:43.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.723+0000 7fc9586cb640 1 Processor -- start 2024-04-25T09:53:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.723+0000 7fc9586cb640 1 -- start start 2024-04-25T09:53:43.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc9586cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc95006abe0 0x7fc95006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc9586cb640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9500ad570 0x7fc95006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc9586cb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95006b500 con 0x7fc95015dd80 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc9586cb640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95006e410 con 0x7fc9500ad570 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc9586cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500b3930 con 0x7fc95006abe0 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc956c68640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9500ad570 0x7fc95006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc956c68640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9500ad570 0x7fc95006ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60812/0 (socket says 172.21.15.19:60812) 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc956c68640 1 -- 172.21.15.19:0/1206951875 learned_addr learned my addr 172.21.15.19:0/1206951875 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc956467640 1 --2- 172.21.15.19:0/1206951875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc95006abe0 0x7fc95006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc956c68640 1 -- 172.21.15.19:0/1206951875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc95006abe0 msgr2=0x7fc95006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc956c68640 1 --2- 172.21.15.19:0/1206951875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc95006abe0 0x7fc95006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc956c68640 1 -- 172.21.15.19:0/1206951875 >> v1:172.21.15.112:6789/0 conn(0x7fc95015dd80 legacy=0x7fc9500b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.724+0000 7fc956c68640 1 -- 172.21.15.19:0/1206951875 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9500b3c10 con 0x7fc9500ad570 2024-04-25T09:53:43.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.725+0000 7fc956c68640 1 --2- 172.21.15.19:0/1206951875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9500ad570 0x7fc95006ee60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc948009e50 tx=0x7fc94802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16dabb6a0921c3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.725+0000 7fc955465640 1 -- 172.21.15.19:0/1206951875 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948036070 con 0x7fc9500ad570 2024-04-25T09:53:43.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.725+0000 7fc955465640 1 -- 172.21.15.19:0/1206951875 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc948031600 con 0x7fc9500ad570 2024-04-25T09:53:43.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.725+0000 7fc955465640 1 -- 172.21.15.19:0/1206951875 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9480319d0 con 0x7fc9500ad570 2024-04-25T09:53:43.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.726+0000 7fc9586cb640 1 -- 172.21.15.19:0/1206951875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9500ad570 msgr2=0x7fc95006ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.726+0000 7fc9586cb640 1 --2- 172.21.15.19:0/1206951875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9500ad570 0x7fc95006ee60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc948009e50 tx=0x7fc94802a6c0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.726+0000 7fc9586cb640 1 -- 172.21.15.19:0/1206951875 shutdown_connections 2024-04-25T09:53:43.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.726+0000 7fc9586cb640 1 --2- 172.21.15.19:0/1206951875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9500ad570 0x7fc95006ee60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.726+0000 7fc9586cb640 1 --2- 172.21.15.19:0/1206951875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc95006abe0 0x7fc95006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.726+0000 7fc9586cb640 1 -- 172.21.15.19:0/1206951875 >> 172.21.15.19:0/1206951875 conn(0x7fc9500b5af0 msgr2=0x7fc9500b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.726+0000 7fc9586cb640 1 -- 172.21.15.19:0/1206951875 shutdown_connections 2024-04-25T09:53:43.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.726+0000 7fc9586cb640 1 -- 172.21.15.19:0/1206951875 wait complete. 2024-04-25T09:53:43.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.727+0000 7fc9586cb640 1 Processor -- start 2024-04-25T09:53:43.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.727+0000 7fc9586cb640 1 -- start start 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc9586cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc95006abe0 0x7fc950151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc9586cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9500ad570 0x7fc950144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc9586cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95015dd80 0x7fc95015f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc9586cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc950070280 con 0x7fc95015dd80 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc9586cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95006e940 con 0x7fc9500ad570 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc9586cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc950070400 con 0x7fc95006abe0 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc956c68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95015dd80 0x7fc95015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc956c68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95015dd80 0x7fc95015f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60814/0 (socket says 172.21.15.19:60814) 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc956c68640 1 -- 172.21.15.19:0/3399094188 learned_addr learned my addr 172.21.15.19:0/3399094188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc956467640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc95006abe0 0x7fc950151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc955c66640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9500ad570 0x7fc950144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc956c68640 1 -- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc95006abe0 msgr2=0x7fc950151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc956c68640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc95006abe0 0x7fc950151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc956c68640 1 -- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9500ad570 msgr2=0x7fc950144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc956c68640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9500ad570 0x7fc950144f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc956c68640 1 -- 172.21.15.19:0/3399094188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950154a60 con 0x7fc95015dd80 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.728+0000 7fc956c68640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95015dd80 0x7fc95015f200 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc948009e50 tx=0x7fc948004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.729+0000 7fc946ffd640 1 -- 172.21.15.19:0/3399094188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948036050 con 0x7fc95015dd80 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.729+0000 7fc946ffd640 1 -- 172.21.15.19:0/3399094188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc948042070 con 0x7fc95015dd80 2024-04-25T09:53:43.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.729+0000 7fc9586cb640 1 -- 172.21.15.19:0/3399094188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc950154d40 con 0x7fc95015dd80 2024-04-25T09:53:43.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.729+0000 7fc946ffd640 1 -- 172.21.15.19:0/3399094188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948009510 con 0x7fc95015dd80 2024-04-25T09:53:43.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.729+0000 7fc9586cb640 1 -- 172.21.15.19:0/3399094188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc950158320 con 0x7fc95015dd80 2024-04-25T09:53:43.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.730+0000 7fc9586cb640 1 -- 172.21.15.19:0/3399094188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9500d09c0 con 0x7fc95015dd80 2024-04-25T09:53:43.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.730+0000 7fc946ffd640 1 -- 172.21.15.19:0/3399094188 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc948007c50 con 0x7fc95015dd80 2024-04-25T09:53:43.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.730+0000 7fc946ffd640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc93406b4d0 0x7fc93406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.731+0000 7fc946ffd640 1 -- 172.21.15.19:0/3399094188 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc9480ab400 con 0x7fc95015dd80 2024-04-25T09:53:43.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.731+0000 7fc956467640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc93406b4d0 0x7fc93406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.731+0000 7fc956467640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc93406b4d0 0x7fc93406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc94c006fd0 tx=0x7fc94c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.735+0000 7fc946ffd640 1 -- 172.21.15.19:0/3399094188 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc948079fd0 con 0x7fc95015dd80 2024-04-25T09:53:43.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.917+0000 7fc9586cb640 1 -- 172.21.15.19:0/3399094188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc9500739e0 con 0x7fc95015dd80 2024-04-25T09:53:43.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.917+0000 7fc946ffd640 1 -- 172.21.15.19:0/3399094188 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc9480798d0 con 0x7fc95015dd80 2024-04-25T09:53:43.919 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:43.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 -- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc93406b4d0 msgr2=0x7fc93406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc93406b4d0 0x7fc93406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc94c006fd0 tx=0x7fc94c008040 comp rx=0 tx=0).stop 2024-04-25T09:53:43.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 -- 172.21.15.19:0/3399094188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95015dd80 msgr2=0x7fc95015f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95015dd80 0x7fc95015f200 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc948009e50 tx=0x7fc948004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:43.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 -- 172.21.15.19:0/3399094188 shutdown_connections 2024-04-25T09:53:43.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc93406b4d0 0x7fc93406d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc95015dd80 0x7fc95015f200 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9500ad570 0x7fc950144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 --2- 172.21.15.19:0/3399094188 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc95006abe0 0x7fc950151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 -- 172.21.15.19:0/3399094188 >> 172.21.15.19:0/3399094188 conn(0x7fc9500b5af0 msgr2=0x7fc9500ae6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 -- 172.21.15.19:0/3399094188 shutdown_connections 2024-04-25T09:53:43.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:43.919+0000 7fc9586cb640 1 -- 172.21.15.19:0/3399094188 wait complete. 2024-04-25T09:53:43.932 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:44.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.031+0000 7f7b4a7c8640 1 Processor -- start 2024-04-25T09:53:44.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.031+0000 7f7b4a7c8640 1 -- start start 2024-04-25T09:53:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.032+0000 7f7b4a7c8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 0x7f7b440b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.032+0000 7f7b4a7c8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7b440d0ad0 0x7f7b440b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.032+0000 7f7b4a7c8640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b440d1df0 con 0x7f7b440b9770 2024-04-25T09:53:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.032+0000 7f7b4a7c8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b440d20d0 con 0x7f7b440d0ad0 2024-04-25T09:53:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.032+0000 7f7b4a7c8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440d23b0 con 0x7f7b4415dd30 2024-04-25T09:53:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.032+0000 7f7b43fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 0x7f7b440b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.032+0000 7f7b43fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 0x7f7b440b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:35534/0 (socket says 172.21.15.19:35534) 2024-04-25T09:53:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.032+0000 7f7b43fff640 1 -- 172.21.15.19:0/1312854151 learned_addr learned my addr 172.21.15.19:0/1312854151 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.032+0000 7f7b48d65640 1 --2- 172.21.15.19:0/1312854151 >> v2:172.21.15.112:3300/0 conn(0x7f7b440d0ad0 0x7f7b440b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.033+0000 7f7b43fff640 1 -- 172.21.15.19:0/1312854151 >> v2:172.21.15.112:3300/0 conn(0x7f7b440d0ad0 msgr2=0x7f7b440b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.033+0000 7f7b43fff640 1 --2- 172.21.15.19:0/1312854151 >> v2:172.21.15.112:3300/0 conn(0x7f7b440d0ad0 0x7f7b440b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.033+0000 7f7b43fff640 1 -- 172.21.15.19:0/1312854151 >> v1:172.21.15.112:6789/0 conn(0x7f7b440b9770 legacy=0x7f7b440d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.033+0000 7f7b43fff640 1 -- 172.21.15.19:0/1312854151 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b440d2690 con 0x7f7b4415dd30 2024-04-25T09:53:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.033+0000 7f7b48d65640 1 --2- 172.21.15.19:0/1312854151 >> v2:172.21.15.112:3300/0 conn(0x7f7b440d0ad0 0x7f7b440b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.033+0000 7f7b43fff640 1 --2- 172.21.15.19:0/1312854151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 0x7f7b440b9230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009870 tx=0x7f7b2c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54db03dc1a4a0f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.033+0000 7f7b42ffd640 1 -- 172.21.15.19:0/1312854151 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c037070 con 0x7f7b4415dd30 2024-04-25T09:53:44.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.034+0000 7f7b42ffd640 1 -- 172.21.15.19:0/1312854151 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c032440 con 0x7f7b4415dd30 2024-04-25T09:53:44.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.034+0000 7f7b42ffd640 1 -- 172.21.15.19:0/1312854151 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c032810 con 0x7f7b4415dd30 2024-04-25T09:53:44.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.034+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/1312854151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 msgr2=0x7f7b440b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.034+0000 7f7b4a7c8640 1 --2- 172.21.15.19:0/1312854151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 0x7f7b440b9230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009870 tx=0x7f7b2c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.034+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/1312854151 shutdown_connections 2024-04-25T09:53:44.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.034+0000 7f7b4a7c8640 1 --2- 172.21.15.19:0/1312854151 >> v2:172.21.15.112:3300/0 conn(0x7f7b440d0ad0 0x7f7b440b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.034+0000 7f7b4a7c8640 1 --2- 172.21.15.19:0/1312854151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 0x7f7b440b9230 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.034+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/1312854151 >> 172.21.15.19:0/1312854151 conn(0x7f7b44061c30 msgr2=0x7f7b4406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:44.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.034+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/1312854151 shutdown_connections 2024-04-25T09:53:44.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.034+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/1312854151 wait complete. 2024-04-25T09:53:44.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.035+0000 7f7b4a7c8640 1 Processor -- start 2024-04-25T09:53:44.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.036+0000 7f7b4a7c8640 1 -- start start 2024-04-25T09:53:44.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.036+0000 7f7b4a7c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b440b9770 0x7f7b44160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.036+0000 7f7b4a7c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b440d0ad0 0x7f7b4414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.036+0000 7f7b4a7c8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 0x7f7b4415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.036+0000 7f7b4a7c8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440d1b90 con 0x7f7b440d0ad0 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.036+0000 7f7b4a7c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440d1a10 con 0x7f7b440b9770 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.036+0000 7f7b4a7c8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440d2e90 con 0x7f7b4415dd30 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.036+0000 7f7b437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b440d0ad0 0x7f7b4414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.036+0000 7f7b437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b440d0ad0 0x7f7b4414def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60830/0 (socket says 172.21.15.19:60830) 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.036+0000 7f7b437fe640 1 -- 172.21.15.19:0/2540402925 learned_addr learned my addr 172.21.15.19:0/2540402925 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b43fff640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b440b9770 0x7f7b44160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b48d65640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 0x7f7b4415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b437fe640 1 -- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 msgr2=0x7f7b4415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b437fe640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 0x7f7b4415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b437fe640 1 -- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b440b9770 msgr2=0x7f7b44160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b437fe640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b440b9770 0x7f7b44160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b437fe640 1 -- 172.21.15.19:0/2540402925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4415fbb0 con 0x7f7b440d0ad0 2024-04-25T09:53:44.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b437fe640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b440d0ad0 0x7f7b4414def0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400c400 tx=0x7f7b3400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b40ff9640 1 -- 172.21.15.19:0/2540402925 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b340052d0 con 0x7f7b440d0ad0 2024-04-25T09:53:44.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b40ff9640 1 -- 172.21.15.19:0/2540402925 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b340054b0 con 0x7f7b440d0ad0 2024-04-25T09:53:44.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b40ff9640 1 -- 172.21.15.19:0/2540402925 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b34006a40 con 0x7f7b440d0ad0 2024-04-25T09:53:44.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/2540402925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4415fe90 con 0x7f7b440d0ad0 2024-04-25T09:53:44.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.037+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/2540402925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b44160450 con 0x7f7b440d0ad0 2024-04-25T09:53:44.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.038+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/2540402925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b44073990 con 0x7f7b440d0ad0 2024-04-25T09:53:44.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.039+0000 7f7b40ff9640 1 -- 172.21.15.19:0/2540402925 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b34006be0 con 0x7f7b440d0ad0 2024-04-25T09:53:44.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.039+0000 7f7b40ff9640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b2406b4d0 0x7f7b2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.039+0000 7f7b40ff9640 1 -- 172.21.15.19:0/2540402925 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7b340898d0 con 0x7f7b440d0ad0 2024-04-25T09:53:44.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.039+0000 7f7b43fff640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b2406b4d0 0x7f7b2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.040+0000 7f7b43fff640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b2406b4d0 0x7f7b2406d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c034000 tx=0x7f7b2c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.043+0000 7f7b40ff9640 1 -- 172.21.15.19:0/2540402925 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b34057dc0 con 0x7f7b440d0ad0 2024-04-25T09:53:44.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.224+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/2540402925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b440cdd50 con 0x7f7b440d0ad0 2024-04-25T09:53:44.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.225+0000 7f7b40ff9640 1 -- 172.21.15.19:0/2540402925 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f7b3405be60 con 0x7f7b440d0ad0 2024-04-25T09:53:44.228 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:44.228 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:44.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.227+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b2406b4d0 msgr2=0x7f7b2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.227+0000 7f7b4a7c8640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b2406b4d0 0x7f7b2406d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c034000 tx=0x7f7b2c02e000 comp rx=0 tx=0).stop 2024-04-25T09:53:44.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.227+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/2540402925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b440d0ad0 msgr2=0x7f7b4414def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.227+0000 7f7b4a7c8640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b440d0ad0 0x7f7b4414def0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400c400 tx=0x7f7b3400c800 comp rx=0 tx=0).stop 2024-04-25T09:53:44.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.227+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/2540402925 shutdown_connections 2024-04-25T09:53:44.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.227+0000 7f7b4a7c8640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b2406b4d0 0x7f7b2406d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.227+0000 7f7b4a7c8640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b4415dd30 0x7f7b4415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.227+0000 7f7b4a7c8640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b440d0ad0 0x7f7b4414def0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.227+0000 7f7b4a7c8640 1 --2- 172.21.15.19:0/2540402925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b440b9770 0x7f7b44160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.227+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/2540402925 >> 172.21.15.19:0/2540402925 conn(0x7f7b44061c30 msgr2=0x7f7b440c80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:44.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.227+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/2540402925 shutdown_connections 2024-04-25T09:53:44.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.228+0000 7f7b4a7c8640 1 -- 172.21.15.19:0/2540402925 wait complete. 2024-04-25T09:53:44.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:44.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.342+0000 7f351a90a640 1 Processor -- start 2024-04-25T09:53:44.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.343+0000 7f351a90a640 1 -- start start 2024-04-25T09:53:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.343+0000 7f351a90a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35140ced80 0x7f35140d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.343+0000 7f351a90a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35140d1ce0 0x7f35140b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.343+0000 7f351a90a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35140d23a0 con 0x7f351415dc20 2024-04-25T09:53:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.343+0000 7f351a90a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35140d2170 con 0x7f35140ced80 2024-04-25T09:53:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.343+0000 7f351a90a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35140d2680 con 0x7f35140d1ce0 2024-04-25T09:53:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.343+0000 7f35137fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35140ced80 0x7f35140d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.344+0000 7f3513fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f351415dc20 0x7f35140ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57356/0 (socket says 172.21.15.19:57356) 2024-04-25T09:53:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.344+0000 7f3513fff640 1 -- 172.21.15.19:0/108033129 learned_addr learned my addr 172.21.15.19:0/108033129 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.344+0000 7f35137fe640 1 -- 172.21.15.19:0/108033129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35140d1ce0 msgr2=0x7f35140b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.344+0000 7f3518ea7640 1 --2- 172.21.15.19:0/108033129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35140d1ce0 0x7f35140b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.344+0000 7f35137fe640 1 --2- 172.21.15.19:0/108033129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35140d1ce0 0x7f35140b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.344+0000 7f3512ffd640 1 -- 172.21.15.19:0/108033129 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098322376 0 0) 0x7f35140d23a0 con 0x7f351415dc20 2024-04-25T09:53:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.344+0000 7f35137fe640 1 -- 172.21.15.19:0/108033129 >> v1:172.21.15.19:6789/0 conn(0x7f351415dc20 legacy=0x7f35140ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.344+0000 7f35137fe640 1 -- 172.21.15.19:0/108033129 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35140d2960 con 0x7f35140ced80 2024-04-25T09:53:44.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.344+0000 7f3518ea7640 1 --2- 172.21.15.19:0/108033129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35140d1ce0 0x7f35140b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f35137fe640 1 --2- 172.21.15.19:0/108033129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35140ced80 0x7f35140d3440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f35080097a0 tx=0x7f350802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e00029c2d57372a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f3512ffd640 1 -- 172.21.15.19:0/108033129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508037070 con 0x7f35140ced80 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f3512ffd640 1 -- 172.21.15.19:0/108033129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350802ae50 con 0x7f35140ced80 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f3512ffd640 1 -- 172.21.15.19:0/108033129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508032660 con 0x7f35140ced80 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f351a90a640 1 -- 172.21.15.19:0/108033129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35140ced80 msgr2=0x7f35140d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f351a90a640 1 --2- 172.21.15.19:0/108033129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35140ced80 0x7f35140d3440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f35080097a0 tx=0x7f350802a100 comp rx=0 tx=0).stop 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f351a90a640 1 -- 172.21.15.19:0/108033129 shutdown_connections 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f351a90a640 1 --2- 172.21.15.19:0/108033129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35140d1ce0 0x7f35140b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f351a90a640 1 --2- 172.21.15.19:0/108033129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35140ced80 0x7f35140d3440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f351a90a640 1 -- 172.21.15.19:0/108033129 >> 172.21.15.19:0/108033129 conn(0x7f3514061a20 msgr2=0x7f3514061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f351a90a640 1 -- 172.21.15.19:0/108033129 shutdown_connections 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.345+0000 7f351a90a640 1 -- 172.21.15.19:0/108033129 wait complete. 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.346+0000 7f351a90a640 1 Processor -- start 2024-04-25T09:53:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.347+0000 7f351a90a640 1 -- start start 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.347+0000 7f351a90a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35140ced80 0x7f351414ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.347+0000 7f351a90a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35140d1ce0 0x7f35140caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.347+0000 7f351a90a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f351415dc20 0x7f35140c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.347+0000 7f351a90a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35140b6a30 con 0x7f35140d1ce0 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.347+0000 7f351a90a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35140d2f20 con 0x7f351415dc20 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.347+0000 7f351a90a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35140b6bb0 con 0x7f35140ced80 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f35137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35140d1ce0 0x7f35140caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f35137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35140d1ce0 0x7f35140caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60844/0 (socket says 172.21.15.19:60844) 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f35137fe640 1 -- 172.21.15.19:0/1577481242 learned_addr learned my addr 172.21.15.19:0/1577481242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f3513fff640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35140ced80 0x7f351414ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f3518ea7640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f351415dc20 0x7f35140c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f35137fe640 1 -- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35140ced80 msgr2=0x7f351414ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f35137fe640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35140ced80 0x7f351414ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f35137fe640 1 -- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f351415dc20 msgr2=0x7f35140c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f35137fe640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f351415dc20 0x7f35140c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f35137fe640 1 -- 172.21.15.19:0/1577481242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35140cb2f0 con 0x7f35140d1ce0 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f3518ea7640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f351415dc20 0x7f35140c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f35137fe640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35140d1ce0 0x7f35140caa70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f35080098c0 tx=0x7f35080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f3510ff9640 1 -- 172.21.15.19:0/1577481242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508041030 con 0x7f35140d1ce0 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f3510ff9640 1 -- 172.21.15.19:0/1577481242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350802ac50 con 0x7f35140d1ce0 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f351a90a640 1 -- 172.21.15.19:0/1577481242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35140cb5d0 con 0x7f35140d1ce0 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f3510ff9640 1 -- 172.21.15.19:0/1577481242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508037040 con 0x7f35140d1ce0 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.348+0000 7f351a90a640 1 -- 172.21.15.19:0/1577481242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351414d820 con 0x7f35140d1ce0 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.349+0000 7f351a90a640 1 -- 172.21.15.19:0/1577481242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3514073a00 con 0x7f35140d1ce0 2024-04-25T09:53:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.350+0000 7f3510ff9640 1 -- 172.21.15.19:0/1577481242 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3508044070 con 0x7f35140d1ce0 2024-04-25T09:53:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.350+0000 7f3510ff9640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f350006b4d0 0x7f350006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.350+0000 7f3510ff9640 1 -- 172.21.15.19:0/1577481242 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f350802ae00 con 0x7f35140d1ce0 2024-04-25T09:53:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.350+0000 7f3513fff640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f350006b4d0 0x7f350006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.351+0000 7f3513fff640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f350006b4d0 0x7f350006d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3504000f60 tx=0x7f3504009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.354+0000 7f3510ff9640 1 -- 172.21.15.19:0/1577481242 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3508079600 con 0x7f35140d1ce0 2024-04-25T09:53:44.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.537+0000 7f351a90a640 1 -- 172.21.15.19:0/1577481242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35140cded0 con 0x7f35140d1ce0 2024-04-25T09:53:44.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.538+0000 7f3510ff9640 1 -- 172.21.15.19:0/1577481242 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3508078f00 con 0x7f35140d1ce0 2024-04-25T09:53:44.540 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:44.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 -- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f350006b4d0 msgr2=0x7f350006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f350006b4d0 0x7f350006d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3504000f60 tx=0x7f3504009040 comp rx=0 tx=0).stop 2024-04-25T09:53:44.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 -- 172.21.15.19:0/1577481242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35140d1ce0 msgr2=0x7f35140caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35140d1ce0 0x7f35140caa70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f35080098c0 tx=0x7f35080046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 -- 172.21.15.19:0/1577481242 shutdown_connections 2024-04-25T09:53:44.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f350006b4d0 0x7f350006d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f351415dc20 0x7f35140c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35140d1ce0 0x7f35140caa70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 --2- 172.21.15.19:0/1577481242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35140ced80 0x7f351414ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 -- 172.21.15.19:0/1577481242 >> 172.21.15.19:0/1577481242 conn(0x7f3514061a20 msgr2=0x7f351406c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 -- 172.21.15.19:0/1577481242 shutdown_connections 2024-04-25T09:53:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.540+0000 7f351a90a640 1 -- 172.21.15.19:0/1577481242 wait complete. 2024-04-25T09:53:44.553 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:44.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.655+0000 7fcccbc9e640 1 Processor -- start 2024-04-25T09:53:44.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.655+0000 7fcccbc9e640 1 -- start start 2024-04-25T09:53:44.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fcccbc9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc41446d0 0x7fccc40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fcccbc9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fccc406e8a0 0x7fccc40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fcccbc9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc40cccc0 con 0x7fccc4144d50 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fcccbc9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc40b2b20 con 0x7fccc406e8a0 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fcccbc9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc406d400 con 0x7fccc41446d0 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fccca23b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fccc406e8a0 0x7fccc40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fccc9239640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fccc4144d50 0x7fccc40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57372/0 (socket says 172.21.15.19:57372) 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fccc9239640 1 -- 172.21.15.19:0/1843898133 learned_addr learned my addr 172.21.15.19:0/1843898133 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fccc9a3a640 1 --2- 172.21.15.19:0/1843898133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc41446d0 0x7fccc40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fccca23b640 1 -- 172.21.15.19:0/1843898133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc41446d0 msgr2=0x7fccc40b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fccca23b640 1 --2- 172.21.15.19:0/1843898133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc41446d0 0x7fccc40b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fccca23b640 1 -- 172.21.15.19:0/1843898133 >> v1:172.21.15.19:6789/0 conn(0x7fccc4144d50 legacy=0x7fccc40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fccc8a38640 1 -- 172.21.15.19:0/1843898133 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392325762 0 0) 0x7fccc40cccc0 con 0x7fccc4144d50 2024-04-25T09:53:44.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.656+0000 7fccca23b640 1 -- 172.21.15.19:0/1843898133 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc40cc140 con 0x7fccc406e8a0 2024-04-25T09:53:44.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.657+0000 7fccca23b640 1 --2- 172.21.15.19:0/1843898133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc406e8a0 0x7fccc40b4460 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009d90 tx=0x7fccbc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403c9ba8152d3f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.657+0000 7fccc8a38640 1 -- 172.21.15.19:0/1843898133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc007740 con 0x7fccc406e8a0 2024-04-25T09:53:44.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.657+0000 7fccc8a38640 1 -- 172.21.15.19:0/1843898133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc0078e0 con 0x7fccc406e8a0 2024-04-25T09:53:44.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.657+0000 7fccc8a38640 1 -- 172.21.15.19:0/1843898133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc007cb0 con 0x7fccc406e8a0 2024-04-25T09:53:44.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.658+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1843898133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc406e8a0 msgr2=0x7fccc40b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.658+0000 7fcccbc9e640 1 --2- 172.21.15.19:0/1843898133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc406e8a0 0x7fccc40b4460 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009d90 tx=0x7fccbc02a660 comp rx=0 tx=0).stop 2024-04-25T09:53:44.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.658+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1843898133 shutdown_connections 2024-04-25T09:53:44.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.658+0000 7fcccbc9e640 1 --2- 172.21.15.19:0/1843898133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc406e8a0 0x7fccc40b4460 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.658+0000 7fcccbc9e640 1 --2- 172.21.15.19:0/1843898133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc41446d0 0x7fccc40b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.658+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1843898133 >> 172.21.15.19:0/1843898133 conn(0x7fccc406abe0 msgr2=0x7fccc406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:44.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.658+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1843898133 shutdown_connections 2024-04-25T09:53:44.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.658+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1843898133 wait complete. 2024-04-25T09:53:44.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.659+0000 7fcccbc9e640 1 Processor -- start 2024-04-25T09:53:44.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.659+0000 7fcccbc9e640 1 -- start start 2024-04-25T09:53:44.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fcccbc9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc406e8a0 0x7fccc41410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fcccbc9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc41446d0 0x7fccc41f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fcccbc9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc4144d50 0x7fccc41e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fcccbc9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40cc700 con 0x7fccc4144d50 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fcccbc9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40b49a0 con 0x7fccc406e8a0 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fcccbc9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40cc9e0 con 0x7fccc41446d0 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccca23b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc4144d50 0x7fccc41e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccca23b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc4144d50 0x7fccc41e5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60862/0 (socket says 172.21.15.19:60862) 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccca23b640 1 -- 172.21.15.19:0/1273375003 learned_addr learned my addr 172.21.15.19:0/1273375003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccc9a3a640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc406e8a0 0x7fccc41410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccc9239640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc41446d0 0x7fccc41f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccca23b640 1 -- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc41446d0 msgr2=0x7fccc41f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccca23b640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc41446d0 0x7fccc41f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccca23b640 1 -- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc406e8a0 msgr2=0x7fccc41410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccca23b640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc406e8a0 0x7fccc41410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccca23b640 1 -- 172.21.15.19:0/1273375003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc41f4310 con 0x7fccc4144d50 2024-04-25T09:53:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccc9a3a640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc406e8a0 0x7fccc41410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.660+0000 7fccca23b640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc4144d50 0x7fccc41e5c40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009eb0 tx=0x7fccbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.661+0000 7fccba7fc640 1 -- 172.21.15.19:0/1273375003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc040070 con 0x7fccc4144d50 2024-04-25T09:53:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.661+0000 7fccba7fc640 1 -- 172.21.15.19:0/1273375003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc031bc0 con 0x7fccc4144d50 2024-04-25T09:53:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.661+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1273375003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc41f45f0 con 0x7fccc4144d50 2024-04-25T09:53:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.661+0000 7fccba7fc640 1 -- 172.21.15.19:0/1273375003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc045460 con 0x7fccc4144d50 2024-04-25T09:53:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.661+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1273375003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc40cd6f0 con 0x7fccc4144d50 2024-04-25T09:53:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.662+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1273375003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc4073a00 con 0x7fccc4144d50 2024-04-25T09:53:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.662+0000 7fccba7fc640 1 -- 172.21.15.19:0/1273375003 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccbc039040 con 0x7fccc4144d50 2024-04-25T09:53:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.663+0000 7fccba7fc640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 0x7fcca806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.663+0000 7fccba7fc640 1 -- 172.21.15.19:0/1273375003 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fccbc0abe80 con 0x7fccc4144d50 2024-04-25T09:53:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.663+0000 7fccc9a3a640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 0x7fcca806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.664+0000 7fccc9a3a640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 0x7fcca806d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fccc0006fd0 tx=0x7fccc0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.667+0000 7fccba7fc640 1 -- 172.21.15.19:0/1273375003 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fccbc07aad0 con 0x7fccc4144d50 2024-04-25T09:53:44.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.848+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1273375003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fccc41f47c0 con 0x7fccc4144d50 2024-04-25T09:53:44.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.849+0000 7fccba7fc640 1 -- 172.21.15.19:0/1273375003 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fccbc07a3d0 con 0x7fccc4144d50 2024-04-25T09:53:44.851 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:44.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 msgr2=0x7fcca806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 0x7fcca806d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fccc0006fd0 tx=0x7fccc0008040 comp rx=0 tx=0).stop 2024-04-25T09:53:44.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1273375003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc4144d50 msgr2=0x7fccc41e5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc4144d50 0x7fccc41e5c40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009eb0 tx=0x7fccbc004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:44.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1273375003 shutdown_connections 2024-04-25T09:53:44.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 0x7fcca806d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc4144d50 0x7fccc41e5c40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc41446d0 0x7fccc41f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 --2- 172.21.15.19:0/1273375003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc406e8a0 0x7fccc41410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1273375003 >> 172.21.15.19:0/1273375003 conn(0x7fccc406abe0 msgr2=0x7fccc4143770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:44.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1273375003 shutdown_connections 2024-04-25T09:53:44.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.851+0000 7fcccbc9e640 1 -- 172.21.15.19:0/1273375003 wait complete. 2024-04-25T09:53:44.864 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.966+0000 7f7f954fc640 1 Processor -- start 2024-04-25T09:53:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.966+0000 7f7f954fc640 1 -- start start 2024-04-25T09:53:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f954fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f900ced80 0x7f7f900d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f954fc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9015e000 con 0x7f7f9015dc20 2024-04-25T09:53:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f954fc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f900d1de0 con 0x7f7f90161710 2024-04-25T09:53:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f954fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f90161c60 con 0x7f7f900ced80 2024-04-25T09:53:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f8effd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7f9015dc20 0x7f7f900ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57376/0 (socket says 172.21.15.19:57376) 2024-04-25T09:53:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f8effd640 1 -- 172.21.15.19:0/639972763 learned_addr learned my addr 172.21.15.19:0/639972763 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f8e7fc640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f900ced80 0x7f7f900d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215116707 0 0) 0x7f7f9015e000 con 0x7f7f9015dc20 2024-04-25T09:53:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f78003520 con 0x7f7f9015dc20 2024-04-25T09:53:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 997918753 0 0) 0x7f7f78003520 con 0x7f7f9015dc20 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f900ced80 msgr2=0x7f7f900d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f8dffb640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f900ced80 0x7f7f900d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 >> v1:172.21.15.112:6789/0 conn(0x7f7f90161710 legacy=0x7f7f900d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90161f40 con 0x7f7f9015dc20 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.967+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f7f80001670 con 0x7f7f9015dc20 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 >> v1:172.21.15.19:6789/0 conn(0x7f7f9015dc20 legacy=0x7f7f900ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8dffb640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f900ced80 0x7f7f780039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8dffb640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f78003f60 0x7f7f78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8dffb640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f78004d40 0x7f7f780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f90161c60 con 0x7f7f78003f60 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7800baa0 con 0x7f7f78004d40 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7800bca0 con 0x7f7f900ced80 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8e7fc640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f78003f60 0x7f7f78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8effd640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f900ced80 0x7f7f780039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8f7fe640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f78004d40 0x7f7f780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8e7fc640 1 -- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f900ced80 msgr2=0x7f7f780039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8e7fc640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f900ced80 0x7f7f780039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8e7fc640 1 -- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f78004d40 msgr2=0x7f7f780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8e7fc640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f78004d40 0x7f7f780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8e7fc640 1 -- 172.21.15.19:0/639972763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f78003520 con 0x7f7f78003f60 2024-04-25T09:53:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.968+0000 7f7f8f7fe640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f78004d40 0x7f7f780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.969+0000 7f7f8e7fc640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f78003f60 0x7f7f78004800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009ac0 tx=0x7f7f78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c469dfe1742e75a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.969+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84036070 con 0x7f7f78003f60 2024-04-25T09:53:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.969+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f84031400 con 0x7f7f78003f60 2024-04-25T09:53:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.969+0000 7f7f8dffb640 1 -- 172.21.15.19:0/639972763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84009690 con 0x7f7f78003f60 2024-04-25T09:53:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.969+0000 7f7f954fc640 1 -- 172.21.15.19:0/639972763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f78003f60 msgr2=0x7f7f78004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.969+0000 7f7f954fc640 1 --2- 172.21.15.19:0/639972763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f78003f60 0x7f7f78004800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009ac0 tx=0x7f7f78005830 comp rx=0 tx=0).stop 2024-04-25T09:53:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.969+0000 7f7f8f7fe640 1 -- 172.21.15.19:0/639972763 reap_dead start 2024-04-25T09:53:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.969+0000 7f7f954fc640 1 -- 172.21.15.19:0/639972763 shutdown_connections 2024-04-25T09:53:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.969+0000 7f7f954fc640 1 -- 172.21.15.19:0/639972763 >> 172.21.15.19:0/639972763 conn(0x7f7f900d0420 msgr2=0x7f7f900d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.969+0000 7f7f954fc640 1 -- 172.21.15.19:0/639972763 shutdown_connections 2024-04-25T09:53:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.969+0000 7f7f954fc640 1 -- 172.21.15.19:0/639972763 wait complete. 2024-04-25T09:53:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.971+0000 7f7f954fc640 1 Processor -- start 2024-04-25T09:53:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.971+0000 7f7f954fc640 1 -- start start 2024-04-25T09:53:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.971+0000 7f7f954fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f78003f60 0x7f7f900cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.971+0000 7f7f954fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f901f7740 0x7f7f900caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.971+0000 7f7f954fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f901603e0 0x7f7f90160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.971+0000 7f7f954fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f900b5e50 con 0x7f7f901603e0 2024-04-25T09:53:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.971+0000 7f7f954fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f90162500 con 0x7f7f78003f60 2024-04-25T09:53:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.971+0000 7f7f954fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f900b5fd0 con 0x7f7f901f7740 2024-04-25T09:53:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f901603e0 0x7f7f90160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f901603e0 0x7f7f90160860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60880/0 (socket says 172.21.15.19:60880) 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8f7fe640 1 -- 172.21.15.19:0/3229389318 learned_addr learned my addr 172.21.15.19:0/3229389318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8e7fc640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f901f7740 0x7f7f900caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8f7fe640 1 -- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f901f7740 msgr2=0x7f7f900caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8f7fe640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f901f7740 0x7f7f900caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8effd640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f78003f60 0x7f7f900cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8f7fe640 1 -- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f78003f60 msgr2=0x7f7f900cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8f7fe640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f78003f60 0x7f7f900cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8f7fe640 1 -- 172.21.15.19:0/3229389318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f900cafb0 con 0x7f7f901603e0 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8effd640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f78003f60 0x7f7f900cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f8f7fe640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f901603e0 0x7f7f90160860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7f90161af0 tx=0x7f7f8800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f777fe640 1 -- 172.21.15.19:0/3229389318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88006b50 con 0x7f7f901603e0 2024-04-25T09:53:44.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f777fe640 1 -- 172.21.15.19:0/3229389318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f88006cf0 con 0x7f7f901603e0 2024-04-25T09:53:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f954fc640 1 -- 172.21.15.19:0/3229389318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f900cb230 con 0x7f7f901603e0 2024-04-25T09:53:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f777fe640 1 -- 172.21.15.19:0/3229389318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88003970 con 0x7f7f901603e0 2024-04-25T09:53:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.972+0000 7f7f954fc640 1 -- 172.21.15.19:0/3229389318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f901f6f40 con 0x7f7f901603e0 2024-04-25T09:53:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.973+0000 7f7f954fc640 1 -- 172.21.15.19:0/3229389318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f90073a00 con 0x7f7f901603e0 2024-04-25T09:53:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.974+0000 7f7f777fe640 1 -- 172.21.15.19:0/3229389318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f8801a020 con 0x7f7f901603e0 2024-04-25T09:53:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.974+0000 7f7f777fe640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f7c06b4d0 0x7f7f7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.974+0000 7f7f777fe640 1 -- 172.21.15.19:0/3229389318 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7f88089af0 con 0x7f7f901603e0 2024-04-25T09:53:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.974+0000 7f7f8effd640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f7c06b4d0 0x7f7f7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.975+0000 7f7f8effd640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f7c06b4d0 0x7f7f7c06d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f80007d90 tx=0x7f7f80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:44.978+0000 7f7f777fe640 1 -- 172.21.15.19:0/3229389318 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f900cafb0 con 0x7f7f901603e0 2024-04-25T09:53:45.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.159+0000 7f7f954fc640 1 -- 172.21.15.19:0/3229389318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f900d16b0 con 0x7f7f901603e0 2024-04-25T09:53:45.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.159+0000 7f7f777fe640 1 -- 172.21.15.19:0/3229389318 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f88006ee0 con 0x7f7f901603e0 2024-04-25T09:53:45.162 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 -- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f7c06b4d0 msgr2=0x7f7f7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f7c06b4d0 0x7f7f7c06d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f80007d90 tx=0x7f7f80009040 comp rx=0 tx=0).stop 2024-04-25T09:53:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 -- 172.21.15.19:0/3229389318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f901603e0 msgr2=0x7f7f90160860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f901603e0 0x7f7f90160860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7f90161af0 tx=0x7f7f8800eb90 comp rx=0 tx=0).stop 2024-04-25T09:53:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 -- 172.21.15.19:0/3229389318 shutdown_connections 2024-04-25T09:53:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f7c06b4d0 0x7f7f7c06d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f901603e0 0x7f7f90160860 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f901f7740 0x7f7f900caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 --2- 172.21.15.19:0/3229389318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f78003f60 0x7f7f900cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 -- 172.21.15.19:0/3229389318 >> 172.21.15.19:0/3229389318 conn(0x7f7f900d0420 msgr2=0x7f7f900d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 -- 172.21.15.19:0/3229389318 shutdown_connections 2024-04-25T09:53:45.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.161+0000 7f7f954fc640 1 -- 172.21.15.19:0/3229389318 wait complete. 2024-04-25T09:53:45.174 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:45.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.275+0000 7f3f685e7640 1 Processor -- start 2024-04-25T09:53:45.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.276+0000 7f3f685e7640 1 -- start start 2024-04-25T09:53:45.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.276+0000 7f3f685e7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f601446d0 0x7f3f600b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.276+0000 7f3f685e7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f600cccc0 con 0x7f3f6006e8a0 2024-04-25T09:53:45.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.276+0000 7f3f685e7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f600d0460 con 0x7f3f60144d50 2024-04-25T09:53:45.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.276+0000 7f3f685e7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f600b2b20 con 0x7f3f601446d0 2024-04-25T09:53:45.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f66b84640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3f6006e8a0 0x7f3f600cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57378/0 (socket says 172.21.15.19:57378) 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f66b84640 1 -- 172.21.15.19:0/1287339337 learned_addr learned my addr 172.21.15.19:0/1287339337 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f66383640 1 --2- 172.21.15.19:0/1287339337 >> v2:172.21.15.112:3300/0 conn(0x7f3f601446d0 0x7f3f600b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464954936 0 0) 0x7f3f600cccc0 con 0x7f3f6006e8a0 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f4c003520 con 0x7f3f6006e8a0 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1203571339 0 0) 0x7f3f4c003520 con 0x7f3f6006e8a0 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 >> v2:172.21.15.112:3300/0 conn(0x7f3f601446d0 msgr2=0x7f3f600b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 --2- 172.21.15.19:0/1287339337 >> v2:172.21.15.112:3300/0 conn(0x7f3f601446d0 0x7f3f600b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 >> v1:172.21.15.112:6789/0 conn(0x7f3f60144d50 legacy=0x7f3f600cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f600cc140 con 0x7f3f6006e8a0 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f3f580019f0 con 0x7f3f6006e8a0 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 >> v1:172.21.15.19:6789/0 conn(0x7f3f6006e8a0 legacy=0x7f3f600cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 --2- 172.21.15.19:0/1287339337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f4c0039b0 0x7f3f4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 --2- 172.21.15.19:0/1287339337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f4c004800 0x7f3f4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 --2- 172.21.15.19:0/1287339337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c0051c0 0x7f3f4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f600b2b20 con 0x7f3f4c0039b0 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c00c060 con 0x7f3f4c0051c0 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c00c260 con 0x7f3f4c004800 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.277+0000 7f3f66383640 1 --2- 172.21.15.19:0/1287339337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f4c0039b0 0x7f3f4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.278+0000 7f3f65b82640 1 --2- 172.21.15.19:0/1287339337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f4c004800 0x7f3f4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.278+0000 7f3f66b84640 1 --2- 172.21.15.19:0/1287339337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c0051c0 0x7f3f4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.278+0000 7f3f66383640 1 -- 172.21.15.19:0/1287339337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f4c004800 msgr2=0x7f3f4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.278+0000 7f3f66383640 1 --2- 172.21.15.19:0/1287339337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f4c004800 0x7f3f4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.278+0000 7f3f66383640 1 -- 172.21.15.19:0/1287339337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c0051c0 msgr2=0x7f3f4c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.278+0000 7f3f66383640 1 --2- 172.21.15.19:0/1287339337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c0051c0 0x7f3f4c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.278+0000 7f3f66383640 1 -- 172.21.15.19:0/1287339337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c003520 con 0x7f3f4c0039b0 2024-04-25T09:53:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.278+0000 7f3f66b84640 1 -- 172.21.15.19:0/1287339337 reap_dead start 2024-04-25T09:53:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.278+0000 7f3f66383640 1 --2- 172.21.15.19:0/1287339337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f4c0039b0 0x7f3f4c003dd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009ac0 tx=0x7f3f4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89353b9ee6e7f13f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.278+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c036070 con 0x7f3f4c0039b0 2024-04-25T09:53:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.279+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c031400 con 0x7f3f4c0039b0 2024-04-25T09:53:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.279+0000 7f3f65381640 1 -- 172.21.15.19:0/1287339337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c009690 con 0x7f3f4c0039b0 2024-04-25T09:53:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.279+0000 7f3f685e7640 1 -- 172.21.15.19:0/1287339337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f4c0039b0 msgr2=0x7f3f4c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.279+0000 7f3f685e7640 1 --2- 172.21.15.19:0/1287339337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f4c0039b0 0x7f3f4c003dd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009ac0 tx=0x7f3f4c005830 comp rx=0 tx=0).stop 2024-04-25T09:53:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.279+0000 7f3f685e7640 1 -- 172.21.15.19:0/1287339337 shutdown_connections 2024-04-25T09:53:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.279+0000 7f3f685e7640 1 --2- 172.21.15.19:0/1287339337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f4c0039b0 0x7f3f4c003dd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.279+0000 7f3f685e7640 1 -- 172.21.15.19:0/1287339337 >> 172.21.15.19:0/1287339337 conn(0x7f3f6006abe0 msgr2=0x7f3f6006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.279+0000 7f3f685e7640 1 -- 172.21.15.19:0/1287339337 shutdown_connections 2024-04-25T09:53:45.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.279+0000 7f3f685e7640 1 -- 172.21.15.19:0/1287339337 wait complete. 2024-04-25T09:53:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.280+0000 7f3f685e7640 1 Processor -- start 2024-04-25T09:53:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.280+0000 7f3f685e7640 1 -- start start 2024-04-25T09:53:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.281+0000 7f3f685e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c0039b0 0x7f3f60153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.281+0000 7f3f685e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6013dd00 0x7f3f60151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.281+0000 7f3f685e7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f601f2e60 0x7f3f601f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.281+0000 7f3f685e7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f600cc700 con 0x7f3f6013dd00 2024-04-25T09:53:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.281+0000 7f3f685e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6006d400 con 0x7f3f4c0039b0 2024-04-25T09:53:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.281+0000 7f3f685e7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f600cc9e0 con 0x7f3f601f2e60 2024-04-25T09:53:45.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.281+0000 7f3f65b82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6013dd00 0x7f3f60151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.281+0000 7f3f65b82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6013dd00 0x7f3f60151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60894/0 (socket says 172.21.15.19:60894) 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.281+0000 7f3f65b82640 1 -- 172.21.15.19:0/416601005 learned_addr learned my addr 172.21.15.19:0/416601005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.281+0000 7f3f66383640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c0039b0 0x7f3f60153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.281+0000 7f3f66b84640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f601f2e60 0x7f3f601f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f65b82640 1 -- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f601f2e60 msgr2=0x7f3f601f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f65b82640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f601f2e60 0x7f3f601f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f65b82640 1 -- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c0039b0 msgr2=0x7f3f60153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f65b82640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c0039b0 0x7f3f60153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f65b82640 1 -- 172.21.15.19:0/416601005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f60153c50 con 0x7f3f6013dd00 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f66b84640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f601f2e60 0x7f3f601f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f65b82640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6013dd00 0x7f3f60151600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000e4d0 tx=0x7f3f5000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f56ffd640 1 -- 172.21.15.19:0/416601005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f500053a0 con 0x7f3f6013dd00 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f56ffd640 1 -- 172.21.15.19:0/416601005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f50018070 con 0x7f3f6013dd00 2024-04-25T09:53:45.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f685e7640 1 -- 172.21.15.19:0/416601005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f60153f30 con 0x7f3f6013dd00 2024-04-25T09:53:45.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f56ffd640 1 -- 172.21.15.19:0/416601005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50003810 con 0x7f3f6013dd00 2024-04-25T09:53:45.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.282+0000 7f3f685e7640 1 -- 172.21.15.19:0/416601005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6013c750 con 0x7f3f6013dd00 2024-04-25T09:53:45.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.283+0000 7f3f685e7640 1 -- 172.21.15.19:0/416601005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f60073a00 con 0x7f3f6013dd00 2024-04-25T09:53:45.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.283+0000 7f3f56ffd640 1 -- 172.21.15.19:0/416601005 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f500039b0 con 0x7f3f6013dd00 2024-04-25T09:53:45.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.284+0000 7f3f56ffd640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f4406b500 0x7f3f4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.284+0000 7f3f56ffd640 1 -- 172.21.15.19:0/416601005 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3f5008a230 con 0x7f3f6013dd00 2024-04-25T09:53:45.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.284+0000 7f3f66383640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f4406b500 0x7f3f4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.285+0000 7f3f66383640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f4406b500 0x7f3f4406d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c033000 tx=0x7f3f5c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.288+0000 7f3f56ffd640 1 -- 172.21.15.19:0/416601005 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f50058d70 con 0x7f3f6013dd00 2024-04-25T09:53:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.467+0000 7f3f685e7640 1 -- 172.21.15.19:0/416601005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3f600cccc0 con 0x7f3f6013dd00 2024-04-25T09:53:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.467+0000 7f3f56ffd640 1 -- 172.21.15.19:0/416601005 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3f50058670 con 0x7f3f6013dd00 2024-04-25T09:53:45.469 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:45.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 -- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f4406b500 msgr2=0x7f3f4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f4406b500 0x7f3f4406d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c033000 tx=0x7f3f5c008030 comp rx=0 tx=0).stop 2024-04-25T09:53:45.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 -- 172.21.15.19:0/416601005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6013dd00 msgr2=0x7f3f60151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6013dd00 0x7f3f60151600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000e4d0 tx=0x7f3f5000e990 comp rx=0 tx=0).stop 2024-04-25T09:53:45.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 -- 172.21.15.19:0/416601005 shutdown_connections 2024-04-25T09:53:45.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f4406b500 0x7f3f4406d9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f601f2e60 0x7f3f601f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6013dd00 0x7f3f60151600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 --2- 172.21.15.19:0/416601005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f4c0039b0 0x7f3f60153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 -- 172.21.15.19:0/416601005 >> 172.21.15.19:0/416601005 conn(0x7f3f6006abe0 msgr2=0x7f3f601434e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 -- 172.21.15.19:0/416601005 shutdown_connections 2024-04-25T09:53:45.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.469+0000 7f3f685e7640 1 -- 172.21.15.19:0/416601005 wait complete. 2024-04-25T09:53:45.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.581+0000 7f761205a640 1 Processor -- start 2024-04-25T09:53:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.581+0000 7f761205a640 1 -- start start 2024-04-25T09:53:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f761205a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f760c15dd20 0x7f760c0b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f761205a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f760c15e100 0x7f760c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f761205a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f760c15eac0 con 0x7f760c15e560 2024-04-25T09:53:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f761205a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760c0d1c00 con 0x7f760c15dd20 2024-04-25T09:53:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f761205a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760c15eda0 con 0x7f760c15e100 2024-04-25T09:53:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f760b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f760c15dd20 0x7f760c0b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f760bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f760c15e560 0x7f760c0d3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57384/0 (socket says 172.21.15.19:57384) 2024-04-25T09:53:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f760bfff640 1 -- 172.21.15.19:0/2324949274 learned_addr learned my addr 172.21.15.19:0/2324949274 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f760affd640 1 --2- 172.21.15.19:0/2324949274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f760c15e100 0x7f760c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f760b7fe640 1 -- 172.21.15.19:0/2324949274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f760c15e100 msgr2=0x7f760c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f760b7fe640 1 --2- 172.21.15.19:0/2324949274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f760c15e100 0x7f760c0d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f760b7fe640 1 -- 172.21.15.19:0/2324949274 >> v1:172.21.15.19:6789/0 conn(0x7f760c15e560 legacy=0x7f760c0d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f760b7fe640 1 -- 172.21.15.19:0/2324949274 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c15f080 con 0x7f760c15dd20 2024-04-25T09:53:45.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.582+0000 7f760a7fc640 1 -- 172.21.15.19:0/2324949274 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21965743 0 0) 0x7f760c15eac0 con 0x7f760c15e560 2024-04-25T09:53:45.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.583+0000 7f760b7fe640 1 --2- 172.21.15.19:0/2324949274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f760c15dd20 0x7f760c0b96d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f76000097a0 tx=0x7f760002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fdcc0cf8c9fad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.583+0000 7f760a7fc640 1 -- 172.21.15.19:0/2324949274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600037070 con 0x7f760c15dd20 2024-04-25T09:53:45.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.583+0000 7f760a7fc640 1 -- 172.21.15.19:0/2324949274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760002ae50 con 0x7f760c15dd20 2024-04-25T09:53:45.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.583+0000 7f760a7fc640 1 -- 172.21.15.19:0/2324949274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600032660 con 0x7f760c15dd20 2024-04-25T09:53:45.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.583+0000 7f761205a640 1 -- 172.21.15.19:0/2324949274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f760c15dd20 msgr2=0x7f760c0b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.583+0000 7f761205a640 1 --2- 172.21.15.19:0/2324949274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f760c15dd20 0x7f760c0b96d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f76000097a0 tx=0x7f760002a100 comp rx=0 tx=0).stop 2024-04-25T09:53:45.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.583+0000 7f761205a640 1 -- 172.21.15.19:0/2324949274 shutdown_connections 2024-04-25T09:53:45.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.583+0000 7f761205a640 1 --2- 172.21.15.19:0/2324949274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f760c15e100 0x7f760c0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.583+0000 7f761205a640 1 --2- 172.21.15.19:0/2324949274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f760c15dd20 0x7f760c0b96d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.584+0000 7f761205a640 1 -- 172.21.15.19:0/2324949274 >> 172.21.15.19:0/2324949274 conn(0x7f760c061a20 msgr2=0x7f760c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:45.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.584+0000 7f761205a640 1 -- 172.21.15.19:0/2324949274 shutdown_connections 2024-04-25T09:53:45.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.584+0000 7f761205a640 1 -- 172.21.15.19:0/2324949274 wait complete. 2024-04-25T09:53:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.585+0000 7f761205a640 1 Processor -- start 2024-04-25T09:53:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.585+0000 7f761205a640 1 -- start start 2024-04-25T09:53:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.585+0000 7f761205a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f760c15dd20 0x7f760c1f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f761205a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f760c15e100 0x7f760c14e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f761205a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f760c15e560 0x7f760c14e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f761205a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c0d3500 con 0x7f760c15e560 2024-04-25T09:53:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f761205a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c0d3380 con 0x7f760c15dd20 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f761205a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760c0d3680 con 0x7f760c15e100 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f760bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f760c15e560 0x7f760c14e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f760bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f760c15e560 0x7f760c14e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60910/0 (socket says 172.21.15.19:60910) 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f760bfff640 1 -- 172.21.15.19:0/2101616268 learned_addr learned my addr 172.21.15.19:0/2101616268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f760b7fe640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f760c15dd20 0x7f760c1f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f760affd640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f760c15e100 0x7f760c14e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f760bfff640 1 -- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f760c15e100 msgr2=0x7f760c14e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f760bfff640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f760c15e100 0x7f760c14e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f760bfff640 1 -- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f760c15dd20 msgr2=0x7f760c1f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f760bfff640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f760c15dd20 0x7f760c1f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f760bfff640 1 -- 172.21.15.19:0/2101616268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c1f4380 con 0x7f760c15e560 2024-04-25T09:53:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f760bfff640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f760c15e560 0x7f760c14e620 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f760400c6d0 tx=0x7f760400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f75f3fff640 1 -- 172.21.15.19:0/2101616268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604018070 con 0x7f760c15e560 2024-04-25T09:53:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f75f3fff640 1 -- 172.21.15.19:0/2101616268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7604004d60 con 0x7f760c15e560 2024-04-25T09:53:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f761205a640 1 -- 172.21.15.19:0/2101616268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760c14ec20 con 0x7f760c15e560 2024-04-25T09:53:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f75f3fff640 1 -- 172.21.15.19:0/2101616268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760400a880 con 0x7f760c15e560 2024-04-25T09:53:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.586+0000 7f761205a640 1 -- 172.21.15.19:0/2101616268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760c14f1e0 con 0x7f760c15e560 2024-04-25T09:53:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.588+0000 7f761205a640 1 -- 172.21.15.19:0/2101616268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760c073990 con 0x7f760c15e560 2024-04-25T09:53:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.588+0000 7f75f3fff640 1 -- 172.21.15.19:0/2101616268 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f760400ac30 con 0x7f760c15e560 2024-04-25T09:53:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.588+0000 7f75f3fff640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75f806b4d0 0x7f75f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.588+0000 7f75f3fff640 1 -- 172.21.15.19:0/2101616268 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7604089c70 con 0x7f760c15e560 2024-04-25T09:53:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.589+0000 7f760b7fe640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75f806b4d0 0x7f75f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.589+0000 7f760b7fe640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75f806b4d0 0x7f75f806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7600034000 tx=0x7f760002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.593+0000 7f75f3fff640 1 -- 172.21.15.19:0/2101616268 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f760c1f4380 con 0x7f760c15e560 2024-04-25T09:53:45.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.773+0000 7f761205a640 1 -- 172.21.15.19:0/2101616268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f760c0d27b0 con 0x7f760c15e560 2024-04-25T09:53:45.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.774+0000 7f75f3fff640 1 -- 172.21.15.19:0/2101616268 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f7604058170 con 0x7f760c15e560 2024-04-25T09:53:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:45.777 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:12.181194+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:45.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.776+0000 7f761205a640 1 -- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75f806b4d0 msgr2=0x7f75f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.776+0000 7f761205a640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75f806b4d0 0x7f75f806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7600034000 tx=0x7f760002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:45.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.776+0000 7f761205a640 1 -- 172.21.15.19:0/2101616268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f760c15e560 msgr2=0x7f760c14e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.776+0000 7f761205a640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f760c15e560 0x7f760c14e620 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f760400c6d0 tx=0x7f760400cb90 comp rx=0 tx=0).stop 2024-04-25T09:53:45.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.776+0000 7f761205a640 1 -- 172.21.15.19:0/2101616268 shutdown_connections 2024-04-25T09:53:45.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.776+0000 7f761205a640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75f806b4d0 0x7f75f806d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.776+0000 7f761205a640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f760c15e560 0x7f760c14e620 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.776+0000 7f761205a640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f760c15e100 0x7f760c14e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.776+0000 7f761205a640 1 --2- 172.21.15.19:0/2101616268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f760c15dd20 0x7f760c1f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.776+0000 7f761205a640 1 -- 172.21.15.19:0/2101616268 >> 172.21.15.19:0/2101616268 conn(0x7f760c061a20 msgr2=0x7f760c15d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.777+0000 7f761205a640 1 -- 172.21.15.19:0/2101616268 shutdown_connections 2024-04-25T09:53:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.777+0000 7f761205a640 1 -- 172.21.15.19:0/2101616268 wait complete. 2024-04-25T09:53:45.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:45.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.891+0000 7f0dc301f640 1 Processor -- start 2024-04-25T09:53:45.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.892+0000 7f0dc301f640 1 -- start start 2024-04-25T09:53:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.892+0000 7f0dc301f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1610f0 0x7f0dbc06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.892+0000 7f0dc301f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0dbc1616f0 0x7f0dbc161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.892+0000 7f0dc301f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc0c6960 con 0x7f0dbc0ce0a0 2024-04-25T09:53:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.892+0000 7f0dc301f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dbc1614d0 con 0x7f0dbc1616f0 2024-04-25T09:53:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.892+0000 7f0dc301f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc162260 con 0x7f0dbc1610f0 2024-04-25T09:53:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.893+0000 7f0dc0dbb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1610f0 0x7f0dbc06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.893+0000 7f0dc0dbb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1610f0 0x7f0dbc06c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:35670/0 (socket says 172.21.15.19:35670) 2024-04-25T09:53:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.893+0000 7f0dbbfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0dbc1616f0 0x7f0dbc161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.893+0000 7f0dc0dbb640 1 -- 172.21.15.19:0/1048699933 learned_addr learned my addr 172.21.15.19:0/1048699933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:45.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.893+0000 7f0dc0dbb640 1 -- 172.21.15.19:0/1048699933 >> v2:172.21.15.112:3300/0 conn(0x7f0dbc1616f0 msgr2=0x7f0dbc161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:45.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.893+0000 7f0dc0dbb640 1 --2- 172.21.15.19:0/1048699933 >> v2:172.21.15.112:3300/0 conn(0x7f0dbc1616f0 0x7f0dbc161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.893+0000 7f0dc0dbb640 1 -- 172.21.15.19:0/1048699933 >> v1:172.21.15.112:6789/0 conn(0x7f0dbc0ce0a0 legacy=0x7f0dbc0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.893+0000 7f0dc0dbb640 1 -- 172.21.15.19:0/1048699933 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc162540 con 0x7f0dbc1610f0 2024-04-25T09:53:45.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.893+0000 7f0dbbfff640 1 --2- 172.21.15.19:0/1048699933 >> v2:172.21.15.112:3300/0 conn(0x7f0dbc1616f0 0x7f0dbc161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:45.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.894+0000 7f0dc0dbb640 1 --2- 172.21.15.19:0/1048699933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1610f0 0x7f0dbc06c1c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009870 tx=0x7f0db002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=847c95578c71453c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.894+0000 7f0dbb7fe640 1 -- 172.21.15.19:0/1048699933 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0037070 con 0x7f0dbc1610f0 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.894+0000 7f0dbb7fe640 1 -- 172.21.15.19:0/1048699933 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db0032440 con 0x7f0dbc1610f0 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.894+0000 7f0dbb7fe640 1 -- 172.21.15.19:0/1048699933 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0032810 con 0x7f0dbc1610f0 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.894+0000 7f0dc301f640 1 -- 172.21.15.19:0/1048699933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1610f0 msgr2=0x7f0dbc06c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.894+0000 7f0dc301f640 1 --2- 172.21.15.19:0/1048699933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1610f0 0x7f0dbc06c1c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009870 tx=0x7f0db002a270 comp rx=0 tx=0).stop 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.894+0000 7f0dc301f640 1 -- 172.21.15.19:0/1048699933 shutdown_connections 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.894+0000 7f0dc301f640 1 --2- 172.21.15.19:0/1048699933 >> v2:172.21.15.112:3300/0 conn(0x7f0dbc1616f0 0x7f0dbc161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.894+0000 7f0dc301f640 1 --2- 172.21.15.19:0/1048699933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1610f0 0x7f0dbc06c1c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.894+0000 7f0dc301f640 1 -- 172.21.15.19:0/1048699933 >> 172.21.15.19:0/1048699933 conn(0x7f0dbc06f500 msgr2=0x7f0dbc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.895+0000 7f0dc301f640 1 -- 172.21.15.19:0/1048699933 shutdown_connections 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.895+0000 7f0dc301f640 1 -- 172.21.15.19:0/1048699933 wait complete. 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.896+0000 7f0dc301f640 1 Processor -- start 2024-04-25T09:53:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.896+0000 7f0dc301f640 1 -- start start 2024-04-25T09:53:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dc301f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dbc0ce0a0 0x7f0dbc15d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dc301f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dbc1610f0 0x7f0dbc0b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dc301f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1616f0 0x7f0dbc146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dc301f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0cf690 con 0x7f0dbc1610f0 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dc301f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0cfce0 con 0x7f0dbc0ce0a0 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dc301f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc06c700 con 0x7f0dbc1616f0 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dbc1610f0 0x7f0dbc0b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dbc1610f0 0x7f0dbc0b7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60916/0 (socket says 172.21.15.19:60916) 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dbbfff640 1 -- 172.21.15.19:0/4257060232 learned_addr learned my addr 172.21.15.19:0/4257060232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dc0dbb640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dbc0ce0a0 0x7f0dbc15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dc15bc640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1616f0 0x7f0dbc146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dbbfff640 1 -- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1616f0 msgr2=0x7f0dbc146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dbbfff640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1616f0 0x7f0dbc146060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dbbfff640 1 -- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dbc0ce0a0 msgr2=0x7f0dbc15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dbbfff640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dbc0ce0a0 0x7f0dbc15d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dbbfff640 1 -- 172.21.15.19:0/4257060232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc15f670 con 0x7f0dbc1610f0 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dbbfff640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dbc1610f0 0x7f0dbc0b7210 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0da800ebe0 tx=0x7f0da800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0db97fa640 1 -- 172.21.15.19:0/4257060232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8017070 con 0x7f0dbc1610f0 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0db97fa640 1 -- 172.21.15.19:0/4257060232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da800cc30 con 0x7f0dbc1610f0 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dc301f640 1 -- 172.21.15.19:0/4257060232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc15f950 con 0x7f0dbc1610f0 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0db97fa640 1 -- 172.21.15.19:0/4257060232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8013470 con 0x7f0dbc1610f0 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.897+0000 7f0dc301f640 1 -- 172.21.15.19:0/4257060232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc1601f0 con 0x7f0dbc1610f0 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.898+0000 7f0dc301f640 1 -- 172.21.15.19:0/4257060232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc073a00 con 0x7f0dbc1610f0 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.899+0000 7f0db97fa640 1 -- 172.21.15.19:0/4257060232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0da801a070 con 0x7f0dbc1610f0 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.899+0000 7f0db97fa640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0da006b4d0 0x7f0da006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.899+0000 7f0db97fa640 1 -- 172.21.15.19:0/4257060232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0da808a560 con 0x7f0dbc1610f0 2024-04-25T09:53:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.899+0000 7f0dc0dbb640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0da006b4d0 0x7f0da006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.900+0000 7f0dc0dbb640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0da006b4d0 0x7f0da006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0db0034000 tx=0x7f0db002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:45.903+0000 7f0db97fa640 1 -- 172.21.15.19:0/4257060232 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0da8059010 con 0x7f0dbc1610f0 2024-04-25T09:53:46.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.082+0000 7f0dc301f640 1 -- 172.21.15.19:0/4257060232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0dbc0b7a30 con 0x7f0dbc1610f0 2024-04-25T09:53:46.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.083+0000 7f0db97fa640 1 -- 172.21.15.19:0/4257060232 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0da8058910 con 0x7f0dbc1610f0 2024-04-25T09:53:46.085 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.084+0000 7f0dc301f640 1 -- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0da006b4d0 msgr2=0x7f0da006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.084+0000 7f0dc301f640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0da006b4d0 0x7f0da006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0db0034000 tx=0x7f0db002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.084+0000 7f0dc301f640 1 -- 172.21.15.19:0/4257060232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dbc1610f0 msgr2=0x7f0dbc0b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.084+0000 7f0dc301f640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dbc1610f0 0x7f0dbc0b7210 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0da800ebe0 tx=0x7f0da800c520 comp rx=0 tx=0).stop 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.084+0000 7f0dc301f640 1 -- 172.21.15.19:0/4257060232 shutdown_connections 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.084+0000 7f0dc301f640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0da006b4d0 0x7f0da006d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.084+0000 7f0dc301f640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dbc1616f0 0x7f0dbc146060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.085+0000 7f0dc301f640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dbc1610f0 0x7f0dbc0b7210 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.085+0000 7f0dc301f640 1 --2- 172.21.15.19:0/4257060232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dbc0ce0a0 0x7f0dbc15d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.085+0000 7f0dc301f640 1 -- 172.21.15.19:0/4257060232 >> 172.21.15.19:0/4257060232 conn(0x7f0dbc06f500 msgr2=0x7f0dbc0c6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.085+0000 7f0dc301f640 1 -- 172.21.15.19:0/4257060232 shutdown_connections 2024-04-25T09:53:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.085+0000 7f0dc301f640 1 -- 172.21.15.19:0/4257060232 wait complete. 2024-04-25T09:53:46.097 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.199+0000 7f463f77e640 1 Processor -- start 2024-04-25T09:53:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.199+0000 7f463f77e640 1 -- start start 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.200+0000 7f463f77e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4638161710 0x7f46380d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.200+0000 7f463f77e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463815e000 con 0x7f463815dc20 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.200+0000 7f463f77e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46380d1de0 con 0x7f46380ced80 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.200+0000 7f463f77e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4638161c60 con 0x7f4638161710 2024-04-25T09:53:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.200+0000 7f463dd1b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4638161710 0x7f46380d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.200+0000 7f463d51a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f463815dc20 0x7f46380ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57398/0 (socket says 172.21.15.19:57398) 2024-04-25T09:53:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.200+0000 7f463d51a640 1 -- 172.21.15.19:0/2606050411 learned_addr learned my addr 172.21.15.19:0/2606050411 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.200+0000 7f463dd1b640 1 -- 172.21.15.19:0/2606050411 >> v1:172.21.15.112:6789/0 conn(0x7f46380ced80 legacy=0x7f46380d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.200+0000 7f463dd1b640 1 -- 172.21.15.19:0/2606050411 >> v1:172.21.15.19:6789/0 conn(0x7f463815dc20 legacy=0x7f46380ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.200+0000 7f463dd1b640 1 -- 172.21.15.19:0/2606050411 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4638161f40 con 0x7f4638161710 2024-04-25T09:53:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.200+0000 7f462ffff640 1 -- 172.21.15.19:0/2606050411 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969005972 0 0) 0x7f463815e000 con 0x7f463815dc20 2024-04-25T09:53:46.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.201+0000 7f463dd1b640 1 --2- 172.21.15.19:0/2606050411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4638161710 0x7f46380d29f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4630009d90 tx=0x7f463002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32487cbfa0cac4bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:46.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.201+0000 7f462ffff640 1 -- 172.21.15.19:0/2606050411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630007740 con 0x7f4638161710 2024-04-25T09:53:46.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.201+0000 7f462ffff640 1 -- 172.21.15.19:0/2606050411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46300078e0 con 0x7f4638161710 2024-04-25T09:53:46.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.201+0000 7f462ffff640 1 -- 172.21.15.19:0/2606050411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630007cb0 con 0x7f4638161710 2024-04-25T09:53:46.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.202+0000 7f463f77e640 1 -- 172.21.15.19:0/2606050411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4638161710 msgr2=0x7f46380d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.202+0000 7f463f77e640 1 --2- 172.21.15.19:0/2606050411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4638161710 0x7f46380d29f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4630009d90 tx=0x7f463002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.202+0000 7f463f77e640 1 -- 172.21.15.19:0/2606050411 shutdown_connections 2024-04-25T09:53:46.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.202+0000 7f463f77e640 1 --2- 172.21.15.19:0/2606050411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4638161710 0x7f46380d29f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.202+0000 7f463f77e640 1 -- 172.21.15.19:0/2606050411 >> 172.21.15.19:0/2606050411 conn(0x7f46380d0420 msgr2=0x7f46380d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:46.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.202+0000 7f463f77e640 1 -- 172.21.15.19:0/2606050411 shutdown_connections 2024-04-25T09:53:46.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.202+0000 7f463f77e640 1 -- 172.21.15.19:0/2606050411 wait complete. 2024-04-25T09:53:46.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.203+0000 7f463f77e640 1 Processor -- start 2024-04-25T09:53:46.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.203+0000 7f463f77e640 1 -- start start 2024-04-25T09:53:46.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463f77e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46380ced80 0x7f463815f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463f77e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463815dc20 0x7f46380d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463f77e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4638161710 0x7f4638154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463f77e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380b5e50 con 0x7f4638161710 2024-04-25T09:53:46.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463f77e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4638162500 con 0x7f46380ced80 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463f77e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46380b5fd0 con 0x7f463815dc20 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463dd1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4638161710 0x7f4638154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463dd1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4638161710 0x7f4638154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60932/0 (socket says 172.21.15.19:60932) 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463dd1b640 1 -- 172.21.15.19:0/318891063 learned_addr learned my addr 172.21.15.19:0/318891063 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463d51a640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46380ced80 0x7f463815f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463cd19640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463815dc20 0x7f46380d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463dd1b640 1 -- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463815dc20 msgr2=0x7f46380d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463dd1b640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463815dc20 0x7f46380d2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463dd1b640 1 -- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46380ced80 msgr2=0x7f463815f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463dd1b640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46380ced80 0x7f463815f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463dd1b640 1 -- 172.21.15.19:0/318891063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46381f3560 con 0x7f4638161710 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463cd19640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463815dc20 0x7f46380d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.204+0000 7f463dd1b640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4638161710 0x7f4638154e00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4630009d90 tx=0x7f4630004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:46.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.205+0000 7f462dffb640 1 -- 172.21.15.19:0/318891063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630040070 con 0x7f4638161710 2024-04-25T09:53:46.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.205+0000 7f462dffb640 1 -- 172.21.15.19:0/318891063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4630031c40 con 0x7f4638161710 2024-04-25T09:53:46.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.205+0000 7f462dffb640 1 -- 172.21.15.19:0/318891063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630003990 con 0x7f4638161710 2024-04-25T09:53:46.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.205+0000 7f463f77e640 1 -- 172.21.15.19:0/318891063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46381f3820 con 0x7f4638161710 2024-04-25T09:53:46.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.205+0000 7f463f77e640 1 -- 172.21.15.19:0/318891063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46381f3de0 con 0x7f4638161710 2024-04-25T09:53:46.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.206+0000 7f463f77e640 1 -- 172.21.15.19:0/318891063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4638073a00 con 0x7f4638161710 2024-04-25T09:53:46.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.206+0000 7f462dffb640 1 -- 172.21.15.19:0/318891063 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4630039040 con 0x7f4638161710 2024-04-25T09:53:46.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.207+0000 7f462dffb640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461c06b500 0x7f461c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.207+0000 7f462dffb640 1 -- 172.21.15.19:0/318891063 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f46300ab820 con 0x7f4638161710 2024-04-25T09:53:46.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.207+0000 7f463d51a640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461c06b500 0x7f461c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.207+0000 7f463d51a640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461c06b500 0x7f461c06d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4634000f60 tx=0x7f4634007bb0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:46.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.211+0000 7f462dffb640 1 -- 172.21.15.19:0/318891063 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f463007a380 con 0x7f4638161710 2024-04-25T09:53:46.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.394+0000 7f463f77e640 1 -- 172.21.15.19:0/318891063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f46380cdfd0 con 0x7f4638161710 2024-04-25T09:53:46.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.395+0000 7f462dffb640 1 -- 172.21.15.19:0/318891063 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4630079c80 con 0x7f4638161710 2024-04-25T09:53:46.397 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 -- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461c06b500 msgr2=0x7f461c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461c06b500 0x7f461c06d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4634000f60 tx=0x7f4634007bb0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 -- 172.21.15.19:0/318891063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4638161710 msgr2=0x7f4638154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4638161710 0x7f4638154e00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4630009d90 tx=0x7f4630004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 -- 172.21.15.19:0/318891063 shutdown_connections 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461c06b500 0x7f461c06d9c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4638161710 0x7f4638154e00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463815dc20 0x7f46380d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 --2- 172.21.15.19:0/318891063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46380ced80 0x7f463815f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 -- 172.21.15.19:0/318891063 >> 172.21.15.19:0/318891063 conn(0x7f46380d0420 msgr2=0x7f46380d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 -- 172.21.15.19:0/318891063 shutdown_connections 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.397+0000 7f463f77e640 1 -- 172.21.15.19:0/318891063 wait complete. 2024-04-25T09:53:46.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:46.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.511+0000 7f2b3b838640 1 Processor -- start 2024-04-25T09:53:46.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.512+0000 7f2b3b838640 1 -- start start 2024-04-25T09:53:46.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.512+0000 7f2b3b838640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b3415dc20 0x7f2b340ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.512+0000 7f2b3b838640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2b34161710 0x7f2b340cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.512+0000 7f2b3b838640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2b340b9330 0x7f2b3406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.512+0000 7f2b3b838640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3415e000 con 0x7f2b340b9330 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.512+0000 7f2b3b838640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b340d1de0 con 0x7f2b34161710 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.512+0000 7f2b3b838640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b34161c60 con 0x7f2b3415dc20 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.512+0000 7f2b39dd5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2b340b9330 0x7f2b3406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.512+0000 7f2b39dd5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2b340b9330 0x7f2b3406bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60942/0 (socket says 172.21.15.19:60942) 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.512+0000 7f2b39dd5640 1 -- 172.21.15.19:0/3650809064 learned_addr learned my addr 172.21.15.19:0/3650809064 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.513+0000 7f2b395d4640 1 --2- 172.21.15.19:0/3650809064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b3415dc20 0x7f2b340ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.513+0000 7f2b38dd3640 1 --2- 172.21.15.19:0/3650809064 >> v2:172.21.15.112:3300/0 conn(0x7f2b34161710 0x7f2b340cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.513+0000 7f2b39dd5640 1 -- 172.21.15.19:0/3650809064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b3415dc20 msgr2=0x7f2b340ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.513+0000 7f2b39dd5640 1 --2- 172.21.15.19:0/3650809064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b3415dc20 0x7f2b340ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.513+0000 7f2b39dd5640 1 -- 172.21.15.19:0/3650809064 >> v2:172.21.15.112:3300/0 conn(0x7f2b34161710 msgr2=0x7f2b340cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:46.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.513+0000 7f2b39dd5640 1 --2- 172.21.15.19:0/3650809064 >> v2:172.21.15.112:3300/0 conn(0x7f2b34161710 0x7f2b340cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.513+0000 7f2b39dd5640 1 -- 172.21.15.19:0/3650809064 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b34161f40 con 0x7f2b340b9330 2024-04-25T09:53:46.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.513+0000 7f2b38dd3640 1 --2- 172.21.15.19:0/3650809064 >> v2:172.21.15.112:3300/0 conn(0x7f2b34161710 0x7f2b340cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:46.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.513+0000 7f2b395d4640 1 --2- 172.21.15.19:0/3650809064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b3415dc20 0x7f2b340ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:46.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.513+0000 7f2b39dd5640 1 --2- 172.21.15.19:0/3650809064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340b9330 0x7f2b3406bf80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2b3406d9a0 tx=0x7f2b2402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638e51969ae8de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:46.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b33fff640 1 -- 172.21.15.19:0/3650809064 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24036070 con 0x7f2b340b9330 2024-04-25T09:53:46.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b33fff640 1 -- 172.21.15.19:0/3650809064 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b24031600 con 0x7f2b340b9330 2024-04-25T09:53:46.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b33fff640 1 -- 172.21.15.19:0/3650809064 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b240319d0 con 0x7f2b340b9330 2024-04-25T09:53:46.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b3b838640 1 -- 172.21.15.19:0/3650809064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340b9330 msgr2=0x7f2b3406bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b3b838640 1 --2- 172.21.15.19:0/3650809064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340b9330 0x7f2b3406bf80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2b3406d9a0 tx=0x7f2b2402a6a0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b3b838640 1 -- 172.21.15.19:0/3650809064 shutdown_connections 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b3b838640 1 --2- 172.21.15.19:0/3650809064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340b9330 0x7f2b3406bf80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b3b838640 1 --2- 172.21.15.19:0/3650809064 >> v2:172.21.15.112:3300/0 conn(0x7f2b34161710 0x7f2b340cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b3b838640 1 --2- 172.21.15.19:0/3650809064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b3415dc20 0x7f2b340ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b3b838640 1 -- 172.21.15.19:0/3650809064 >> 172.21.15.19:0/3650809064 conn(0x7f2b340d0420 msgr2=0x7f2b340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b3b838640 1 -- 172.21.15.19:0/3650809064 shutdown_connections 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.514+0000 7f2b3b838640 1 -- 172.21.15.19:0/3650809064 wait complete. 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.515+0000 7f2b3b838640 1 Processor -- start 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.515+0000 7f2b3b838640 1 -- start start 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.516+0000 7f2b3b838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b340b9330 0x7f2b3414eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.516+0000 7f2b3b838640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b3415dc20 0x7f2b340cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.516+0000 7f2b3b838640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b34161710 0x7f2b340abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.516+0000 7f2b3b838640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340b5e50 con 0x7f2b3415dc20 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.516+0000 7f2b3b838640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b34162500 con 0x7f2b340b9330 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.516+0000 7f2b3b838640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340b5fd0 con 0x7f2b34161710 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.516+0000 7f2b38dd3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b3415dc20 0x7f2b340cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.516+0000 7f2b38dd3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b3415dc20 0x7f2b340cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60948/0 (socket says 172.21.15.19:60948) 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.516+0000 7f2b38dd3640 1 -- 172.21.15.19:0/2214968559 learned_addr learned my addr 172.21.15.19:0/2214968559 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b395d4640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b340b9330 0x7f2b3414eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b39dd5640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b34161710 0x7f2b340abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b38dd3640 1 -- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b34161710 msgr2=0x7f2b340abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b38dd3640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b34161710 0x7f2b340abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b38dd3640 1 -- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b340b9330 msgr2=0x7f2b3414eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b38dd3640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b340b9330 0x7f2b3414eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b38dd3640 1 -- 172.21.15.19:0/2214968559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340cdbd0 con 0x7f2b3415dc20 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b395d4640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b340b9330 0x7f2b3414eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b39dd5640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b34161710 0x7f2b340abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b38dd3640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b3415dc20 0x7f2b340cccc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c7f0 tx=0x7f2b2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b31ffb640 1 -- 172.21.15.19:0/2214968559 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28015070 con 0x7f2b3415dc20 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b31ffb640 1 -- 172.21.15.19:0/2214968559 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b28007820 con 0x7f2b3415dc20 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b3b838640 1 -- 172.21.15.19:0/2214968559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b340cd650 con 0x7f2b3415dc20 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b31ffb640 1 -- 172.21.15.19:0/2214968559 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28007bd0 con 0x7f2b3415dc20 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.517+0000 7f2b3b838640 1 -- 172.21.15.19:0/2214968559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b340cd200 con 0x7f2b3415dc20 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.518+0000 7f2b3b838640 1 -- 172.21.15.19:0/2214968559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b34073a00 con 0x7f2b3415dc20 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.518+0000 7f2b31ffb640 1 -- 172.21.15.19:0/2214968559 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b28018070 con 0x7f2b3415dc20 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.519+0000 7f2b31ffb640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2b2006b4d0 0x7f2b2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.519+0000 7f2b31ffb640 1 -- 172.21.15.19:0/2214968559 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2b2808a5b0 con 0x7f2b3415dc20 2024-04-25T09:53:46.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.519+0000 7f2b395d4640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2b2006b4d0 0x7f2b2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.520+0000 7f2b395d4640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2b2006b4d0 0x7f2b2006d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00b840 tx=0x7f2b2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:46.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.523+0000 7f2b31ffb640 1 -- 172.21.15.19:0/2214968559 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b28059180 con 0x7f2b3415dc20 2024-04-25T09:53:46.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.701+0000 7f2b3b838640 1 -- 172.21.15.19:0/2214968559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2b340cd430 con 0x7f2b3415dc20 2024-04-25T09:53:46.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.701+0000 7f2b31ffb640 1 -- 172.21.15.19:0/2214968559 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2b28058a80 con 0x7f2b3415dc20 2024-04-25T09:53:46.703 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:46.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 -- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2b2006b4d0 msgr2=0x7f2b2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2b2006b4d0 0x7f2b2006d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00b840 tx=0x7f2b2c008040 comp rx=0 tx=0).stop 2024-04-25T09:53:46.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 -- 172.21.15.19:0/2214968559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b3415dc20 msgr2=0x7f2b340cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b3415dc20 0x7f2b340cccc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c7f0 tx=0x7f2b2800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 -- 172.21.15.19:0/2214968559 shutdown_connections 2024-04-25T09:53:46.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2b2006b4d0 0x7f2b2006d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b34161710 0x7f2b340abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b3415dc20 0x7f2b340cccc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 --2- 172.21.15.19:0/2214968559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b340b9330 0x7f2b3414eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 -- 172.21.15.19:0/2214968559 >> 172.21.15.19:0/2214968559 conn(0x7f2b340d0420 msgr2=0x7f2b340b3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:46.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 -- 172.21.15.19:0/2214968559 shutdown_connections 2024-04-25T09:53:46.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.703+0000 7f2b3b838640 1 -- 172.21.15.19:0/2214968559 wait complete. 2024-04-25T09:53:46.716 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.819+0000 7f0254163640 1 Processor -- start 2024-04-25T09:53:46.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.819+0000 7f0254163640 1 -- start start 2024-04-25T09:53:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0254163640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f024c15b7d0 0x7f024c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0254163640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024c06cf50 con 0x7f024c06c9e0 2024-04-25T09:53:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0254163640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024c0ce530 con 0x7f024c0b5620 2024-04-25T09:53:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0254163640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f024c0ce810 con 0x7f024c15b7d0 2024-04-25T09:53:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0252700640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f024c06c9e0 0x7f024c0d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57406/0 (socket says 172.21.15.19:57406) 2024-04-25T09:53:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0252700640 1 -- 172.21.15.19:0/2500937057 learned_addr learned my addr 172.21.15.19:0/2500937057 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0251eff640 1 --2- 172.21.15.19:0/2500937057 >> v2:172.21.15.112:3300/0 conn(0x7f024c15b7d0 0x7f024c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0250efd640 1 -- 172.21.15.19:0/2500937057 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908466462 0 0) 0x7f024c06cf50 con 0x7f024c06c9e0 2024-04-25T09:53:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0250efd640 1 -- 172.21.15.19:0/2500937057 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0238003520 con 0x7f024c06c9e0 2024-04-25T09:53:46.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0250efd640 1 -- 172.21.15.19:0/2500937057 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450894198 0 0) 0x7f024c0ce530 con 0x7f024c0b5620 2024-04-25T09:53:46.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0250efd640 1 -- 172.21.15.19:0/2500937057 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f024c06cf50 con 0x7f024c0b5620 2024-04-25T09:53:46.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0251eff640 1 -- 172.21.15.19:0/2500937057 >> v1:172.21.15.112:6789/0 conn(0x7f024c0b5620 legacy=0x7f024c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0250efd640 1 -- 172.21.15.19:0/2500937057 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1158504743 0 0) 0x7f0238003520 con 0x7f024c06c9e0 2024-04-25T09:53:46.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0251eff640 1 -- 172.21.15.19:0/2500937057 >> v1:172.21.15.19:6789/0 conn(0x7f024c06c9e0 legacy=0x7f024c0d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.820+0000 7f0251eff640 1 -- 172.21.15.19:0/2500937057 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0ceaf0 con 0x7f024c15b7d0 2024-04-25T09:53:46.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.821+0000 7f0251eff640 1 --2- 172.21.15.19:0/2500937057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024c15b7d0 0x7f024c0cf010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02480097a0 tx=0x7f024802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77ac707755676c0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:46.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.821+0000 7f0250efd640 1 -- 172.21.15.19:0/2500937057 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248037070 con 0x7f024c15b7d0 2024-04-25T09:53:46.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.821+0000 7f0250efd640 1 -- 172.21.15.19:0/2500937057 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024802ac50 con 0x7f024c15b7d0 2024-04-25T09:53:46.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.821+0000 7f0250efd640 1 -- 172.21.15.19:0/2500937057 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02440019f0 con 0x7f024c15b7d0 2024-04-25T09:53:46.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.822+0000 7f0254163640 1 -- 172.21.15.19:0/2500937057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024c15b7d0 msgr2=0x7f024c0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.822+0000 7f0254163640 1 --2- 172.21.15.19:0/2500937057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024c15b7d0 0x7f024c0cf010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02480097a0 tx=0x7f024802a1c0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.822+0000 7f0254163640 1 -- 172.21.15.19:0/2500937057 shutdown_connections 2024-04-25T09:53:46.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.822+0000 7f0254163640 1 --2- 172.21.15.19:0/2500937057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024c15b7d0 0x7f024c0cf010 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.822+0000 7f0254163640 1 -- 172.21.15.19:0/2500937057 >> 172.21.15.19:0/2500937057 conn(0x7f024c0d0800 msgr2=0x7f024c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:46.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.822+0000 7f0254163640 1 -- 172.21.15.19:0/2500937057 shutdown_connections 2024-04-25T09:53:46.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.822+0000 7f0254163640 1 -- 172.21.15.19:0/2500937057 wait complete. 2024-04-25T09:53:46.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.823+0000 7f0254163640 1 Processor -- start 2024-04-25T09:53:46.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.823+0000 7f0254163640 1 -- start start 2024-04-25T09:53:46.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0254163640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024c06c9e0 0x7f024c150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0254163640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f024c0b5620 0x7f024c0c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0254163640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f024c15b7d0 0x7f024c0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0254163640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0cf6d0 con 0x7f024c15b7d0 2024-04-25T09:53:46.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0254163640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0cf550 con 0x7f024c06c9e0 2024-04-25T09:53:46.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0254163640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c15bbb0 con 0x7f024c0b5620 2024-04-25T09:53:46.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0252700640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f024c15b7d0 0x7f024c0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0252700640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f024c15b7d0 0x7f024c0b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60962/0 (socket says 172.21.15.19:60962) 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0252700640 1 -- 172.21.15.19:0/2829271416 learned_addr learned my addr 172.21.15.19:0/2829271416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0251eff640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024c06c9e0 0x7f024c150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f02516fe640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f024c0b5620 0x7f024c0c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0252700640 1 -- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f024c0b5620 msgr2=0x7f024c0c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0252700640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f024c0b5620 0x7f024c0c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0252700640 1 -- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024c06c9e0 msgr2=0x7f024c150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0252700640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024c06c9e0 0x7f024c150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.824+0000 7f0252700640 1 -- 172.21.15.19:0/2829271416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c150660 con 0x7f024c15b7d0 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.825+0000 7f0252700640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f024c15b7d0 0x7f024c0b8050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0244001350 tx=0x7f024400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.825+0000 7f02427fc640 1 -- 172.21.15.19:0/2829271416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244017070 con 0x7f024c15b7d0 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.825+0000 7f02427fc640 1 -- 172.21.15.19:0/2829271416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024400ad40 con 0x7f024c15b7d0 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.825+0000 7f0254163640 1 -- 172.21.15.19:0/2829271416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024c0c8030 con 0x7f024c15b7d0 2024-04-25T09:53:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.825+0000 7f02427fc640 1 -- 172.21.15.19:0/2829271416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244012600 con 0x7f024c15b7d0 2024-04-25T09:53:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.825+0000 7f0254163640 1 -- 172.21.15.19:0/2829271416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024c0c85f0 con 0x7f024c15b7d0 2024-04-25T09:53:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.826+0000 7f0254163640 1 -- 172.21.15.19:0/2829271416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024c073a00 con 0x7f024c15b7d0 2024-04-25T09:53:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.826+0000 7f02427fc640 1 -- 172.21.15.19:0/2829271416 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f024401a070 con 0x7f024c15b7d0 2024-04-25T09:53:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.827+0000 7f02427fc640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f023006b4d0 0x7f023006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.827+0000 7f02427fc640 1 -- 172.21.15.19:0/2829271416 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f024408a280 con 0x7f024c15b7d0 2024-04-25T09:53:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.827+0000 7f0251eff640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f023006b4d0 0x7f023006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.827+0000 7f0251eff640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f023006b4d0 0x7f023006d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0248034000 tx=0x7f024802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:46.831+0000 7f02427fc640 1 -- 172.21.15.19:0/2829271416 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0244058770 con 0x7f024c15b7d0 2024-04-25T09:53:47.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.008+0000 7f0254163640 1 -- 172.21.15.19:0/2829271416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f024c0c87c0 con 0x7f024c15b7d0 2024-04-25T09:53:47.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.009+0000 7f02427fc640 1 -- 172.21.15.19:0/2829271416 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f024405c810 con 0x7f024c15b7d0 2024-04-25T09:53:47.011 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:47.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.010+0000 7f0254163640 1 -- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f023006b4d0 msgr2=0x7f023006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.010+0000 7f0254163640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f023006b4d0 0x7f023006d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0248034000 tx=0x7f024802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:47.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.010+0000 7f0254163640 1 -- 172.21.15.19:0/2829271416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f024c15b7d0 msgr2=0x7f024c0b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.010+0000 7f0254163640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f024c15b7d0 0x7f024c0b8050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0244001350 tx=0x7f024400a430 comp rx=0 tx=0).stop 2024-04-25T09:53:47.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.011+0000 7f0254163640 1 -- 172.21.15.19:0/2829271416 shutdown_connections 2024-04-25T09:53:47.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.011+0000 7f0254163640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f023006b4d0 0x7f023006d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.011+0000 7f0254163640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f024c15b7d0 0x7f024c0b8050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.011+0000 7f0254163640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f024c0b5620 0x7f024c0c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.011+0000 7f0254163640 1 --2- 172.21.15.19:0/2829271416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f024c06c9e0 0x7f024c150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.011+0000 7f0254163640 1 -- 172.21.15.19:0/2829271416 >> 172.21.15.19:0/2829271416 conn(0x7f024c0d0800 msgr2=0x7f024c15afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:47.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.011+0000 7f0254163640 1 -- 172.21.15.19:0/2829271416 shutdown_connections 2024-04-25T09:53:47.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.011+0000 7f0254163640 1 -- 172.21.15.19:0/2829271416 wait complete. 2024-04-25T09:53:47.035 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-25T09:53:47.035 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:53:47.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:47.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.141+0000 7fa213509640 1 Processor -- start 2024-04-25T09:53:47.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.141+0000 7fa213509640 1 -- start start 2024-04-25T09:53:47.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa213509640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa20c15dbe0 0x7fa20c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa213509640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa20c0b2cb0 0x7fa20c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa213509640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20c0b97a0 con 0x7fa20c0b92a0 2024-04-25T09:53:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa213509640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20c15dfc0 con 0x7fa20c0b2cb0 2024-04-25T09:53:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa213509640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0d1a00 con 0x7fa20c15dbe0 2024-04-25T09:53:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa210aa4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa20c0b2cb0 0x7fa20c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa210aa4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa20c0b2cb0 0x7fa20c0d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60968/0 (socket says 172.21.15.19:60968) 2024-04-25T09:53:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa210aa4640 1 -- 172.21.15.19:0/3879753053 learned_addr learned my addr 172.21.15.19:0/3879753053 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa2112a5640 1 --2- 172.21.15.19:0/3879753053 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa20c15dbe0 0x7fa20c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa210aa4640 1 -- 172.21.15.19:0/3879753053 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa20c15dbe0 msgr2=0x7fa20c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa210aa4640 1 --2- 172.21.15.19:0/3879753053 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa20c15dbe0 0x7fa20c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa210aa4640 1 -- 172.21.15.19:0/3879753053 >> v1:172.21.15.112:6789/0 conn(0x7fa20c0b92a0 legacy=0x7fa20c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.142+0000 7fa210aa4640 1 -- 172.21.15.19:0/3879753053 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0d1c50 con 0x7fa20c0b2cb0 2024-04-25T09:53:47.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.143+0000 7fa210aa4640 1 --2- 172.21.15.19:0/3879753053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20c0b2cb0 0x7fa20c0d0e50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa200009870 tx=0x7fa20002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc789ad3321ad0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.143+0000 7fa1fffff640 1 -- 172.21.15.19:0/3879753053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200037070 con 0x7fa20c0b2cb0 2024-04-25T09:53:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.143+0000 7fa1fffff640 1 -- 172.21.15.19:0/3879753053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa200032440 con 0x7fa20c0b2cb0 2024-04-25T09:53:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.143+0000 7fa1fffff640 1 -- 172.21.15.19:0/3879753053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200032810 con 0x7fa20c0b2cb0 2024-04-25T09:53:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.144+0000 7fa213509640 1 -- 172.21.15.19:0/3879753053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20c0b2cb0 msgr2=0x7fa20c0d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.144+0000 7fa213509640 1 --2- 172.21.15.19:0/3879753053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20c0b2cb0 0x7fa20c0d0e50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa200009870 tx=0x7fa20002a270 comp rx=0 tx=0).stop 2024-04-25T09:53:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.144+0000 7fa213509640 1 -- 172.21.15.19:0/3879753053 shutdown_connections 2024-04-25T09:53:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.144+0000 7fa213509640 1 --2- 172.21.15.19:0/3879753053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20c0b2cb0 0x7fa20c0d0e50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.144+0000 7fa213509640 1 --2- 172.21.15.19:0/3879753053 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa20c15dbe0 0x7fa20c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.144+0000 7fa213509640 1 -- 172.21.15.19:0/3879753053 >> 172.21.15.19:0/3879753053 conn(0x7fa20c061a20 msgr2=0x7fa20c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.144+0000 7fa213509640 1 -- 172.21.15.19:0/3879753053 shutdown_connections 2024-04-25T09:53:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.144+0000 7fa213509640 1 -- 172.21.15.19:0/3879753053 wait complete. 2024-04-25T09:53:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.145+0000 7fa213509640 1 Processor -- start 2024-04-25T09:53:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.145+0000 7fa213509640 1 -- start start 2024-04-25T09:53:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa213509640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa20c0b2cb0 0x7fa20c15f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa213509640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20c0b92a0 0x7fa20c0c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa213509640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c15dbe0 0x7fa20c0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa213509640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0b6ab0 con 0x7fa20c0b92a0 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa213509640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0d2210 con 0x7fa20c15dbe0 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa213509640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0b6c30 con 0x7fa20c0b2cb0 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa210aa4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20c0b92a0 0x7fa20c0c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa210aa4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20c0b92a0 0x7fa20c0c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60976/0 (socket says 172.21.15.19:60976) 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa210aa4640 1 -- 172.21.15.19:0/821225364 learned_addr learned my addr 172.21.15.19:0/821225364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa2112a5640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa20c0b2cb0 0x7fa20c15f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa211aa6640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c15dbe0 0x7fa20c0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa210aa4640 1 -- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa20c0b2cb0 msgr2=0x7fa20c15f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa210aa4640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa20c0b2cb0 0x7fa20c15f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa210aa4640 1 -- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c15dbe0 msgr2=0x7fa20c0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa210aa4640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c15dbe0 0x7fa20c0b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa210aa4640 1 -- 172.21.15.19:0/821225364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c15e710 con 0x7fa20c0b92a0 2024-04-25T09:53:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa2112a5640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa20c0b2cb0 0x7fa20c15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa211aa6640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c15dbe0 0x7fa20c0b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.146+0000 7fa210aa4640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20c0b92a0 0x7fa20c0c9450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa200000c00 tx=0x7fa2000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.147+0000 7fa1fdffb640 1 -- 172.21.15.19:0/821225364 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200041020 con 0x7fa20c0b92a0 2024-04-25T09:53:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.147+0000 7fa1fdffb640 1 -- 172.21.15.19:0/821225364 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2000306c0 con 0x7fa20c0b92a0 2024-04-25T09:53:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.147+0000 7fa213509640 1 -- 172.21.15.19:0/821225364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20c15e9f0 con 0x7fa20c0b92a0 2024-04-25T09:53:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.147+0000 7fa1fdffb640 1 -- 172.21.15.19:0/821225364 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200037040 con 0x7fa20c0b92a0 2024-04-25T09:53:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.147+0000 7fa213509640 1 -- 172.21.15.19:0/821225364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c15efb0 con 0x7fa20c0b92a0 2024-04-25T09:53:47.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.148+0000 7fa213509640 1 -- 172.21.15.19:0/821225364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c073990 con 0x7fa20c0b92a0 2024-04-25T09:53:47.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.148+0000 7fa1fdffb640 1 -- 172.21.15.19:0/821225364 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa200044020 con 0x7fa20c0b92a0 2024-04-25T09:53:47.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.149+0000 7fa1fdffb640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1f006b4d0 0x7fa1f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.149+0000 7fa1fdffb640 1 -- 172.21.15.19:0/821225364 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa2000aad70 con 0x7fa20c0b92a0 2024-04-25T09:53:47.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.149+0000 7fa2112a5640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1f006b4d0 0x7fa1f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.149+0000 7fa2112a5640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1f006b4d0 0x7fa1f006d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa208006fd0 tx=0x7fa208008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.153+0000 7fa1fdffb640 1 -- 172.21.15.19:0/821225364 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa20004b030 con 0x7fa20c0b92a0 2024-04-25T09:53:47.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.332+0000 7fa213509640 1 -- 172.21.15.19:0/821225364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa20c0b48b0 con 0x7fa20c0b92a0 2024-04-25T09:53:47.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.333+0000 7fa1fdffb640 1 -- 172.21.15.19:0/821225364 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18892 (secure 0 0 0) 0x7fa2000797f0 con 0x7fa20c0b92a0 2024-04-25T09:53:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:47.335 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:47.066920+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 -- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1f006b4d0 msgr2=0x7fa1f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1f006b4d0 0x7fa1f006d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa208006fd0 tx=0x7fa208008040 comp rx=0 tx=0).stop 2024-04-25T09:53:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 -- 172.21.15.19:0/821225364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20c0b92a0 msgr2=0x7fa20c0c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20c0b92a0 0x7fa20c0c9450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa200000c00 tx=0x7fa2000046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 -- 172.21.15.19:0/821225364 shutdown_connections 2024-04-25T09:53:47.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1f006b4d0 0x7fa1f006d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa20c15dbe0 0x7fa20c0b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20c0b92a0 0x7fa20c0c9450 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 --2- 172.21.15.19:0/821225364 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa20c0b2cb0 0x7fa20c15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 -- 172.21.15.19:0/821225364 >> 172.21.15.19:0/821225364 conn(0x7fa20c061a20 msgr2=0x7fa20c0b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:47.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 -- 172.21.15.19:0/821225364 shutdown_connections 2024-04-25T09:53:47.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.335+0000 7fa213509640 1 -- 172.21.15.19:0/821225364 wait complete. 2024-04-25T09:53:47.348 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:47.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:47.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.450+0000 7f48d509e640 1 Processor -- start 2024-04-25T09:53:47.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.450+0000 7f48d509e640 1 -- start start 2024-04-25T09:53:47.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48d509e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48d0070dc0 0x7f48d006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48d509e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f48d00711a0 0x7f48d00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48d509e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d00d1770 con 0x7f48d00b4830 2024-04-25T09:53:47.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48d509e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d01622b0 con 0x7f48d00711a0 2024-04-25T09:53:47.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48d509e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d00b10f0 con 0x7f48d0070dc0 2024-04-25T09:53:47.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48c7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f48d00711a0 0x7f48d00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48c7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f48d00711a0 0x7f48d00b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60988/0 (socket says 172.21.15.19:60988) 2024-04-25T09:53:47.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48c7fff640 1 -- 172.21.15.19:0/3248807534 learned_addr learned my addr 172.21.15.19:0/3248807534 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48ced9d640 1 --2- 172.21.15.19:0/3248807534 >> v2:172.21.15.112:3300/0 conn(0x7f48d0070dc0 0x7f48d006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48c7fff640 1 -- 172.21.15.19:0/3248807534 >> v2:172.21.15.112:3300/0 conn(0x7f48d0070dc0 msgr2=0x7f48d006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48c7fff640 1 --2- 172.21.15.19:0/3248807534 >> v2:172.21.15.112:3300/0 conn(0x7f48d0070dc0 0x7f48d006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48c7fff640 1 -- 172.21.15.19:0/3248807534 >> v1:172.21.15.112:6789/0 conn(0x7f48d00b4830 legacy=0x7f48d00b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48c7fff640 1 -- 172.21.15.19:0/3248807534 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d0162740 con 0x7f48d00711a0 2024-04-25T09:53:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.451+0000 7f48ced9d640 1 --2- 172.21.15.19:0/3248807534 >> v2:172.21.15.112:3300/0 conn(0x7f48d0070dc0 0x7f48d006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.452+0000 7f48c7fff640 1 --2- 172.21.15.19:0/3248807534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d00711a0 0x7f48d00b07a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48d0070a00 tx=0x7f48b802a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff86eb91e7ab9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.452+0000 7f48ce59c640 1 -- 172.21.15.19:0/3248807534 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8037070 con 0x7f48d00711a0 2024-04-25T09:53:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.452+0000 7f48ce59c640 1 -- 172.21.15.19:0/3248807534 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b8032440 con 0x7f48d00711a0 2024-04-25T09:53:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.452+0000 7f48ce59c640 1 -- 172.21.15.19:0/3248807534 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8032810 con 0x7f48d00711a0 2024-04-25T09:53:47.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.453+0000 7f48d509e640 1 -- 172.21.15.19:0/3248807534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d00711a0 msgr2=0x7f48d00b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.453+0000 7f48d509e640 1 --2- 172.21.15.19:0/3248807534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d00711a0 0x7f48d00b07a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48d0070a00 tx=0x7f48b802a150 comp rx=0 tx=0).stop 2024-04-25T09:53:47.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.453+0000 7f48d509e640 1 -- 172.21.15.19:0/3248807534 shutdown_connections 2024-04-25T09:53:47.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.453+0000 7f48d509e640 1 --2- 172.21.15.19:0/3248807534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d00711a0 0x7f48d00b07a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.453+0000 7f48d509e640 1 --2- 172.21.15.19:0/3248807534 >> v2:172.21.15.112:3300/0 conn(0x7f48d0070dc0 0x7f48d006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.453+0000 7f48d509e640 1 -- 172.21.15.19:0/3248807534 >> 172.21.15.19:0/3248807534 conn(0x7f48d00b3530 msgr2=0x7f48d00b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.453+0000 7f48d509e640 1 -- 172.21.15.19:0/3248807534 shutdown_connections 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.453+0000 7f48d509e640 1 -- 172.21.15.19:0/3248807534 wait complete. 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.454+0000 7f48d509e640 1 Processor -- start 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.454+0000 7f48d509e640 1 -- start start 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48d509e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d0070dc0 0x7f48d0155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48d509e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d00711a0 0x7f48d0152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48d509e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48d00b4830 0x7f48d01525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48d509e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0161f40 con 0x7f48d00711a0 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48d509e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0161c60 con 0x7f48d0070dc0 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48d509e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0071630 con 0x7f48d00b4830 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d00711a0 0x7f48d0152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d00711a0 0x7f48d0152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60992/0 (socket says 172.21.15.19:60992) 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48c7fff640 1 -- 172.21.15.19:0/4126750926 learned_addr learned my addr 172.21.15.19:0/4126750926 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48ced9d640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d0070dc0 0x7f48d0155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48cf59e640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48d00b4830 0x7f48d01525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48c7fff640 1 -- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48d00b4830 msgr2=0x7f48d01525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48c7fff640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48d00b4830 0x7f48d01525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48c7fff640 1 -- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d0070dc0 msgr2=0x7f48d0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48c7fff640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d0070dc0 0x7f48d0155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48c7fff640 1 -- 172.21.15.19:0/4126750926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d01f6bf0 con 0x7f48d00711a0 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.455+0000 7f48c7fff640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d00711a0 0x7f48d0152060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f48b802a190 tx=0x7f48b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.456+0000 7f48c77fe640 1 -- 172.21.15.19:0/4126750926 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8041020 con 0x7f48d00711a0 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.456+0000 7f48c77fe640 1 -- 172.21.15.19:0/4126750926 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b8030760 con 0x7f48d00711a0 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.456+0000 7f48d509e640 1 -- 172.21.15.19:0/4126750926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d01f6e90 con 0x7f48d00711a0 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.456+0000 7f48c77fe640 1 -- 172.21.15.19:0/4126750926 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8037040 con 0x7f48d00711a0 2024-04-25T09:53:47.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.456+0000 7f48d509e640 1 -- 172.21.15.19:0/4126750926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d01f7450 con 0x7f48d00711a0 2024-04-25T09:53:47.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.457+0000 7f48d509e640 1 -- 172.21.15.19:0/4126750926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d0073a00 con 0x7f48d00711a0 2024-04-25T09:53:47.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.457+0000 7f48c77fe640 1 -- 172.21.15.19:0/4126750926 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48b8044020 con 0x7f48d00711a0 2024-04-25T09:53:47.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.457+0000 7f48c77fe640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48b006b4d0 0x7f48b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.457+0000 7f48c77fe640 1 -- 172.21.15.19:0/4126750926 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f48b80aae40 con 0x7f48d00711a0 2024-04-25T09:53:47.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.458+0000 7f48ced9d640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48b006b4d0 0x7f48b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.458+0000 7f48ced9d640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48b006b4d0 0x7f48b006d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f48d015f600 tx=0x7f48c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.462+0000 7f48c77fe640 1 -- 172.21.15.19:0/4126750926 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48b804b030 con 0x7f48d00711a0 2024-04-25T09:53:47.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.640+0000 7f48d509e640 1 -- 172.21.15.19:0/4126750926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48d00b5ef0 con 0x7f48d00711a0 2024-04-25T09:53:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.640+0000 7f48c77fe640 1 -- 172.21.15.19:0/4126750926 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48b8079950 con 0x7f48d00711a0 2024-04-25T09:53:47.643 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 -- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48b006b4d0 msgr2=0x7f48b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48b006b4d0 0x7f48b006d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f48d015f600 tx=0x7f48c0008040 comp rx=0 tx=0).stop 2024-04-25T09:53:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 -- 172.21.15.19:0/4126750926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d00711a0 msgr2=0x7f48d0152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d00711a0 0x7f48d0152060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f48b802a190 tx=0x7f48b80046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 -- 172.21.15.19:0/4126750926 shutdown_connections 2024-04-25T09:53:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48b006b4d0 0x7f48b006d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48d00b4830 0x7f48d01525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d00711a0 0x7f48d0152060 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 --2- 172.21.15.19:0/4126750926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48d0070dc0 0x7f48d0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 -- 172.21.15.19:0/4126750926 >> 172.21.15.19:0/4126750926 conn(0x7f48d00b3530 msgr2=0x7f48d006ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 -- 172.21.15.19:0/4126750926 shutdown_connections 2024-04-25T09:53:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.642+0000 7f48d509e640 1 -- 172.21.15.19:0/4126750926 wait complete. 2024-04-25T09:53:47.655 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:47.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.757+0000 7f067555c640 1 Processor -- start 2024-04-25T09:53:47.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.757+0000 7f067555c640 1 -- start start 2024-04-25T09:53:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f067555c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f067015dc20 0x7f06700ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f067555c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0670161710 0x7f06700cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f067555c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f067015e000 con 0x7f06700b9330 2024-04-25T09:53:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f067555c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06700d1de0 con 0x7f0670161710 2024-04-25T09:53:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f067555c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0670161c60 con 0x7f067015dc20 2024-04-25T09:53:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f066f7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f06700b9330 0x7f067006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35016/0 (socket says 172.21.15.19:35016) 2024-04-25T09:53:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f066f7fe640 1 -- 172.21.15.19:0/584313462 learned_addr learned my addr 172.21.15.19:0/584313462 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f066effd640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f067015dc20 0x7f06700ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480351899 0 0) 0x7f067015e000 con 0x7f06700b9330 2024-04-25T09:53:47.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f066e7fc640 1 --2- 172.21.15.19:0/584313462 >> v2:172.21.15.112:3300/0 conn(0x7f0670161710 0x7f06700cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0658003520 con 0x7f06700b9330 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1741989708 0 0) 0x7f0658003520 con 0x7f06700b9330 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.758+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f067015dc20 msgr2=0x7f06700ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f067015dc20 0x7f06700ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 >> v2:172.21.15.112:3300/0 conn(0x7f0670161710 msgr2=0x7f06700cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 --2- 172.21.15.19:0/584313462 >> v2:172.21.15.112:3300/0 conn(0x7f0670161710 0x7f06700cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0670161f40 con 0x7f06700b9330 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f06680019f0 con 0x7f06700b9330 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 >> v1:172.21.15.19:6789/0 conn(0x7f06700b9330 legacy=0x7f067006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06580039b0 0x7f0658003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0658004800 0x7f0658004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f067015dc20 0x7f06580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06700d1de0 con 0x7f06580039b0 2024-04-25T09:53:47.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0670161c60 con 0x7f0658004800 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065800b9c0 con 0x7f067015dc20 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066effd640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06580039b0 0x7f0658003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066e7fc640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0658004800 0x7f0658004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066f7fe640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f067015dc20 0x7f06580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066effd640 1 -- 172.21.15.19:0/584313462 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f067015dc20 msgr2=0x7f06580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066effd640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f067015dc20 0x7f06580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066effd640 1 -- 172.21.15.19:0/584313462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0658004800 msgr2=0x7f0658004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066effd640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0658004800 0x7f0658004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.759+0000 7f066effd640 1 -- 172.21.15.19:0/584313462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0658003520 con 0x7f06580039b0 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.760+0000 7f066effd640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06580039b0 0x7f0658003dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0660009ac0 tx=0x7f0658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72312589a86bf39f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.760+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660036070 con 0x7f06580039b0 2024-04-25T09:53:47.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.760+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0660031400 con 0x7f06580039b0 2024-04-25T09:53:47.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.760+0000 7f066dffb640 1 -- 172.21.15.19:0/584313462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660009690 con 0x7f06580039b0 2024-04-25T09:53:47.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.760+0000 7f067555c640 1 -- 172.21.15.19:0/584313462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06580039b0 msgr2=0x7f0658003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.760+0000 7f067555c640 1 --2- 172.21.15.19:0/584313462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06580039b0 0x7f0658003dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0660009ac0 tx=0x7f0658005830 comp rx=0 tx=0).stop 2024-04-25T09:53:47.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.760+0000 7f066f7fe640 1 -- 172.21.15.19:0/584313462 reap_dead start 2024-04-25T09:53:47.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.760+0000 7f067555c640 1 -- 172.21.15.19:0/584313462 shutdown_connections 2024-04-25T09:53:47.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.760+0000 7f067555c640 1 -- 172.21.15.19:0/584313462 >> 172.21.15.19:0/584313462 conn(0x7f06700d0420 msgr2=0x7f06700d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:47.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.760+0000 7f067555c640 1 -- 172.21.15.19:0/584313462 shutdown_connections 2024-04-25T09:53:47.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.761+0000 7f067555c640 1 -- 172.21.15.19:0/584313462 wait complete. 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.762+0000 7f067555c640 1 Processor -- start 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.762+0000 7f067555c640 1 -- start start 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f067555c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06580039b0 0x7f06700cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f067555c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06700aae60 0x7f067015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f067555c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06701f37b0 0x7f06700ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f067555c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06700b5e50 con 0x7f06700aae60 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f067555c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0670162500 con 0x7f06580039b0 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f067555c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06700b5fd0 con 0x7f06701f37b0 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06700aae60 0x7f067015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06700aae60 0x7f067015cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58874/0 (socket says 172.21.15.19:58874) 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066e7fc640 1 -- 172.21.15.19:0/1580284028 learned_addr learned my addr 172.21.15.19:0/1580284028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066f7fe640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06701f37b0 0x7f06700ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066effd640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06580039b0 0x7f06700cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066e7fc640 1 -- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06701f37b0 msgr2=0x7f06700ac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066e7fc640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06701f37b0 0x7f06700ac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066e7fc640 1 -- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06580039b0 msgr2=0x7f06700cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066e7fc640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06580039b0 0x7f06700cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066e7fc640 1 -- 172.21.15.19:0/1580284028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06700ab670 con 0x7f06700aae60 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066f7fe640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06701f37b0 0x7f06700ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f066e7fc640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06700aae60 0x7f067015cc10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0664011960 tx=0x7f0664011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f06577fe640 1 -- 172.21.15.19:0/1580284028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066400faf0 con 0x7f06700aae60 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f06577fe640 1 -- 172.21.15.19:0/1580284028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066400fc90 con 0x7f06700aae60 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f06577fe640 1 -- 172.21.15.19:0/1580284028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664014440 con 0x7f06700aae60 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f067555c640 1 -- 172.21.15.19:0/1580284028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06700cab10 con 0x7f06700aae60 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.763+0000 7f067555c640 1 -- 172.21.15.19:0/1580284028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06700cb0d0 con 0x7f06700aae60 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.764+0000 7f067555c640 1 -- 172.21.15.19:0/1580284028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06700cf690 con 0x7f06700aae60 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.765+0000 7f06577fe640 1 -- 172.21.15.19:0/1580284028 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0664012040 con 0x7f06700aae60 2024-04-25T09:53:47.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.765+0000 7f06577fe640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f065c06b4d0 0x7f065c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.765+0000 7f06577fe640 1 -- 172.21.15.19:0/1580284028 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f066408aaf0 con 0x7f06700aae60 2024-04-25T09:53:47.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.766+0000 7f066effd640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f065c06b4d0 0x7f065c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.766+0000 7f066effd640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f065c06b4d0 0x7f065c06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0660033000 tx=0x7f0660008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.770+0000 7f06577fe640 1 -- 172.21.15.19:0/1580284028 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06700cf690 con 0x7f06700aae60 2024-04-25T09:53:47.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.951+0000 7f067555c640 1 -- 172.21.15.19:0/1580284028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06700d16b0 con 0x7f06700aae60 2024-04-25T09:53:47.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.951+0000 7f06577fe640 1 -- 172.21.15.19:0/1580284028 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0664019030 con 0x7f06700aae60 2024-04-25T09:53:47.953 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:47.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 -- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f065c06b4d0 msgr2=0x7f065c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f065c06b4d0 0x7f065c06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0660033000 tx=0x7f0660008030 comp rx=0 tx=0).stop 2024-04-25T09:53:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 -- 172.21.15.19:0/1580284028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06700aae60 msgr2=0x7f067015cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06700aae60 0x7f067015cc10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0664011960 tx=0x7f0664011e20 comp rx=0 tx=0).stop 2024-04-25T09:53:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 -- 172.21.15.19:0/1580284028 shutdown_connections 2024-04-25T09:53:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f065c06b4d0 0x7f065c06d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06701f37b0 0x7f06700ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06700aae60 0x7f067015cc10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 --2- 172.21.15.19:0/1580284028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06580039b0 0x7f06700cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 -- 172.21.15.19:0/1580284028 >> 172.21.15.19:0/1580284028 conn(0x7f06700d0420 msgr2=0x7f06700d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 -- 172.21.15.19:0/1580284028 shutdown_connections 2024-04-25T09:53:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:47.953+0000 7f067555c640 1 -- 172.21.15.19:0/1580284028 wait complete. 2024-04-25T09:53:47.966 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:48.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.067+0000 7efcdce3f640 1 Processor -- start 2024-04-25T09:53:48.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.067+0000 7efcdce3f640 1 -- start start 2024-04-25T09:53:48.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.068+0000 7efcdce3f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcd815de80 0x7efcd80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.068+0000 7efcdce3f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efcd80ccfe0 0x7efcd80cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.068+0000 7efcdce3f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd80afc80 con 0x7efcd80cefb0 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.068+0000 7efcdce3f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd80ce4d0 con 0x7efcd80ccfe0 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.068+0000 7efcdce3f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80ce7b0 con 0x7efcd815de80 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.068+0000 7efcd6d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efcd80ccfe0 0x7efcd80cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.068+0000 7efcd6d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efcd80ccfe0 0x7efcd80cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58882/0 (socket says 172.21.15.19:58882) 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.068+0000 7efcd6d9d640 1 -- 172.21.15.19:0/2041253478 learned_addr learned my addr 172.21.15.19:0/2041253478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.069+0000 7efcd659c640 1 --2- 172.21.15.19:0/2041253478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcd815de80 0x7efcd80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.069+0000 7efcd6d9d640 1 -- 172.21.15.19:0/2041253478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcd815de80 msgr2=0x7efcd80cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.069+0000 7efcd6d9d640 1 --2- 172.21.15.19:0/2041253478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcd815de80 0x7efcd80cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.069+0000 7efcd6d9d640 1 -- 172.21.15.19:0/2041253478 >> v1:172.21.15.112:6789/0 conn(0x7efcd80cefb0 legacy=0x7efcd80cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.069+0000 7efcd6d9d640 1 -- 172.21.15.19:0/2041253478 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd80cea90 con 0x7efcd80ccfe0 2024-04-25T09:53:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.069+0000 7efcd6d9d640 1 --2- 172.21.15.19:0/2041253478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd80ccfe0 0x7efcd80cd470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009e50 tx=0x7efcc802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a93813949c53b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.070+0000 7efcd559a640 1 -- 172.21.15.19:0/2041253478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8036070 con 0x7efcd80ccfe0 2024-04-25T09:53:48.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.070+0000 7efcd559a640 1 -- 172.21.15.19:0/2041253478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc8031600 con 0x7efcd80ccfe0 2024-04-25T09:53:48.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.070+0000 7efcd559a640 1 -- 172.21.15.19:0/2041253478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc80319d0 con 0x7efcd80ccfe0 2024-04-25T09:53:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.070+0000 7efcdce3f640 1 -- 172.21.15.19:0/2041253478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd80ccfe0 msgr2=0x7efcd80cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.070+0000 7efcdce3f640 1 --2- 172.21.15.19:0/2041253478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd80ccfe0 0x7efcd80cd470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009e50 tx=0x7efcc802a6c0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.070+0000 7efcdce3f640 1 -- 172.21.15.19:0/2041253478 shutdown_connections 2024-04-25T09:53:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.070+0000 7efcdce3f640 1 --2- 172.21.15.19:0/2041253478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd80ccfe0 0x7efcd80cd470 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.070+0000 7efcdce3f640 1 --2- 172.21.15.19:0/2041253478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcd815de80 0x7efcd80cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.070+0000 7efcdce3f640 1 -- 172.21.15.19:0/2041253478 >> 172.21.15.19:0/2041253478 conn(0x7efcd80b2390 msgr2=0x7efcd80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.070+0000 7efcdce3f640 1 -- 172.21.15.19:0/2041253478 shutdown_connections 2024-04-25T09:53:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.070+0000 7efcdce3f640 1 -- 172.21.15.19:0/2041253478 wait complete. 2024-04-25T09:53:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.072+0000 7efcdce3f640 1 Processor -- start 2024-04-25T09:53:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.072+0000 7efcdce3f640 1 -- start start 2024-04-25T09:53:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.072+0000 7efcdce3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd80ccfe0 0x7efcd814d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.072+0000 7efcdce3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80cefb0 0x7efcd815fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.072+0000 7efcdce3f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcd815de80 0x7efcd8160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.072+0000 7efcdce3f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80cfca0 con 0x7efcd80ccfe0 2024-04-25T09:53:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.072+0000 7efcdce3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80cfb20 con 0x7efcd80cefb0 2024-04-25T09:53:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcdce3f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80b2eb0 con 0x7efcd815de80 2024-04-25T09:53:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcd659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd80ccfe0 0x7efcd814d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcd659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd80ccfe0 0x7efcd814d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58884/0 (socket says 172.21.15.19:58884) 2024-04-25T09:53:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcd659c640 1 -- 172.21.15.19:0/1415419794 learned_addr learned my addr 172.21.15.19:0/1415419794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcd6d9d640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcd815de80 0x7efcd8160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcd5d9b640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80cefb0 0x7efcd815fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcd659c640 1 -- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcd815de80 msgr2=0x7efcd8160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcd659c640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcd815de80 0x7efcd8160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcd659c640 1 -- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80cefb0 msgr2=0x7efcd815fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcd659c640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80cefb0 0x7efcd815fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcd659c640 1 -- 172.21.15.19:0/1415419794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd80aeea0 con 0x7efcd80ccfe0 2024-04-25T09:53:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.073+0000 7efcd659c640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd80ccfe0 0x7efcd814d5a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c7f0 tx=0x7efcd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.075+0000 7efcceffd640 1 -- 172.21.15.19:0/1415419794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0015070 con 0x7efcd80ccfe0 2024-04-25T09:53:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.075+0000 7efcceffd640 1 -- 172.21.15.19:0/1415419794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd0004220 con 0x7efcd80ccfe0 2024-04-25T09:53:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.075+0000 7efcdce3f640 1 -- 172.21.15.19:0/1415419794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd80af180 con 0x7efcd80ccfe0 2024-04-25T09:53:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.075+0000 7efcceffd640 1 -- 172.21.15.19:0/1415419794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0007620 con 0x7efcd80ccfe0 2024-04-25T09:53:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.075+0000 7efcdce3f640 1 -- 172.21.15.19:0/1415419794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd814dae0 con 0x7efcd80ccfe0 2024-04-25T09:53:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.076+0000 7efcdce3f640 1 -- 172.21.15.19:0/1415419794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd8073a00 con 0x7efcd80ccfe0 2024-04-25T09:53:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.076+0000 7efcceffd640 1 -- 172.21.15.19:0/1415419794 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efcd00079d0 con 0x7efcd80ccfe0 2024-04-25T09:53:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.077+0000 7efcceffd640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcb806b4d0 0x7efcb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.077+0000 7efcceffd640 1 -- 172.21.15.19:0/1415419794 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6735+0+0 (secure 0 0 0) 0x7efcd008a7f0 con 0x7efcd80ccfe0 2024-04-25T09:53:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.077+0000 7efcd5d9b640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcb806b4d0 0x7efcb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.078+0000 7efcd5d9b640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcb806b4d0 0x7efcb806d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efcd8161570 tx=0x7efcc0007e30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.081+0000 7efcceffd640 1 -- 172.21.15.19:0/1415419794 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efcd0058cb0 con 0x7efcd80ccfe0 2024-04-25T09:53:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.265+0000 7efcdce3f640 1 -- 172.21.15.19:0/1415419794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcd80b2100 con 0x7efcd80ccfe0 2024-04-25T09:53:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.265+0000 7efcceffd640 1 -- 172.21.15.19:0/1415419794 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7efcd005cd50 con 0x7efcd80ccfe0 2024-04-25T09:53:48.267 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 -- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcb806b4d0 msgr2=0x7efcb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcb806b4d0 0x7efcb806d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efcd8161570 tx=0x7efcc0007e30 comp rx=0 tx=0).stop 2024-04-25T09:53:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 -- 172.21.15.19:0/1415419794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd80ccfe0 msgr2=0x7efcd814d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd80ccfe0 0x7efcd814d5a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c7f0 tx=0x7efcd000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 -- 172.21.15.19:0/1415419794 shutdown_connections 2024-04-25T09:53:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcb806b4d0 0x7efcb806d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcd815de80 0x7efcd8160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80cefb0 0x7efcd815fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 --2- 172.21.15.19:0/1415419794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd80ccfe0 0x7efcd814d5a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 -- 172.21.15.19:0/1415419794 >> 172.21.15.19:0/1415419794 conn(0x7efcd80b2390 msgr2=0x7efcd80cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 -- 172.21.15.19:0/1415419794 shutdown_connections 2024-04-25T09:53:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.267+0000 7efcdce3f640 1 -- 172.21.15.19:0/1415419794 wait complete. 2024-04-25T09:53:48.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.382+0000 7ff09adc4640 1 Processor -- start 2024-04-25T09:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.382+0000 7ff09adc4640 1 -- start start 2024-04-25T09:53:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff09adc4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff09415dc20 0x7ff0940ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff09adc4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff0940d32f0 0x7ff0940cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff09adc4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09415e000 con 0x7ff094161710 2024-04-25T09:53:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff09adc4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0940d1de0 con 0x7ff09415dc20 2024-04-25T09:53:48.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff09adc4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff094161c60 con 0x7ff0940d32f0 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff098b60640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff09415dc20 0x7ff0940ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff098b60640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff09415dc20 0x7ff0940ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58892/0 (socket says 172.21.15.19:58892) 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff098b60640 1 -- 172.21.15.19:0/2249513731 learned_addr learned my addr 172.21.15.19:0/2249513731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff098b60640 1 -- 172.21.15.19:0/2249513731 >> v2:172.21.15.112:3300/0 conn(0x7ff0940d32f0 msgr2=0x7ff0940cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff099361640 1 --2- 172.21.15.19:0/2249513731 >> v2:172.21.15.112:3300/0 conn(0x7ff0940d32f0 0x7ff0940cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff098b60640 1 --2- 172.21.15.19:0/2249513731 >> v2:172.21.15.112:3300/0 conn(0x7ff0940d32f0 0x7ff0940cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff098b60640 1 -- 172.21.15.19:0/2249513731 >> v1:172.21.15.112:6789/0 conn(0x7ff094161710 legacy=0x7ff09406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff098b60640 1 -- 172.21.15.19:0/2249513731 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff094161f40 con 0x7ff09415dc20 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.383+0000 7ff099361640 1 --2- 172.21.15.19:0/2249513731 >> v2:172.21.15.112:3300/0 conn(0x7ff0940d32f0 0x7ff0940cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.384+0000 7ff098b60640 1 --2- 172.21.15.19:0/2249513731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09415dc20 0x7ff0940ce670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff0940b6910 tx=0x7ff08802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdf001115806fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.384+0000 7ff0937fe640 1 -- 172.21.15.19:0/2249513731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088037070 con 0x7ff09415dc20 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.384+0000 7ff0937fe640 1 -- 172.21.15.19:0/2249513731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff088032440 con 0x7ff09415dc20 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.384+0000 7ff0937fe640 1 -- 172.21.15.19:0/2249513731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088032810 con 0x7ff09415dc20 2024-04-25T09:53:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.385+0000 7ff09adc4640 1 -- 172.21.15.19:0/2249513731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09415dc20 msgr2=0x7ff0940ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.385+0000 7ff09adc4640 1 --2- 172.21.15.19:0/2249513731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09415dc20 0x7ff0940ce670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff0940b6910 tx=0x7ff08802a230 comp rx=0 tx=0).stop 2024-04-25T09:53:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.385+0000 7ff09adc4640 1 -- 172.21.15.19:0/2249513731 shutdown_connections 2024-04-25T09:53:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.385+0000 7ff09adc4640 1 --2- 172.21.15.19:0/2249513731 >> v2:172.21.15.112:3300/0 conn(0x7ff0940d32f0 0x7ff0940cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.385+0000 7ff09adc4640 1 --2- 172.21.15.19:0/2249513731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09415dc20 0x7ff0940ce670 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.385+0000 7ff09adc4640 1 -- 172.21.15.19:0/2249513731 >> 172.21.15.19:0/2249513731 conn(0x7ff0940d0420 msgr2=0x7ff0940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.385+0000 7ff09adc4640 1 -- 172.21.15.19:0/2249513731 shutdown_connections 2024-04-25T09:53:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.385+0000 7ff09adc4640 1 -- 172.21.15.19:0/2249513731 wait complete. 2024-04-25T09:53:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.386+0000 7ff09adc4640 1 Processor -- start 2024-04-25T09:53:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.386+0000 7ff09adc4640 1 -- start start 2024-04-25T09:53:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff09adc4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0940d32f0 0x7ff094159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff09adc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09415dc20 0x7ff0941f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff09adc4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff094161710 0x7ff0940cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff09adc4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0940b5e50 con 0x7ff09415dc20 2024-04-25T09:53:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff09adc4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff094162500 con 0x7ff0940d32f0 2024-04-25T09:53:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff09adc4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0940b5fd0 con 0x7ff094161710 2024-04-25T09:53:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff093fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09415dc20 0x7ff0941f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff093fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09415dc20 0x7ff0941f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58904/0 (socket says 172.21.15.19:58904) 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff093fff640 1 -- 172.21.15.19:0/1042575732 learned_addr learned my addr 172.21.15.19:0/1042575732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff098b60640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0940d32f0 0x7ff094159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff099361640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff094161710 0x7ff0940cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff093fff640 1 -- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff094161710 msgr2=0x7ff0940cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff093fff640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff094161710 0x7ff0940cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff093fff640 1 -- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0940d32f0 msgr2=0x7ff094159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff093fff640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0940d32f0 0x7ff094159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff093fff640 1 -- 172.21.15.19:0/1042575732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940cb580 con 0x7ff09415dc20 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff098b60640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0940d32f0 0x7ff094159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff093fff640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09415dc20 0x7ff0941f41e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff08000b3f0 tx=0x7ff08000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.387+0000 7ff0917fa640 1 -- 172.21.15.19:0/1042575732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080015020 con 0x7ff09415dc20 2024-04-25T09:53:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.388+0000 7ff0917fa640 1 -- 172.21.15.19:0/1042575732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff080011830 con 0x7ff09415dc20 2024-04-25T09:53:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.388+0000 7ff09adc4640 1 -- 172.21.15.19:0/1042575732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0940cb860 con 0x7ff09415dc20 2024-04-25T09:53:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.388+0000 7ff0917fa640 1 -- 172.21.15.19:0/1042575732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080011c20 con 0x7ff09415dc20 2024-04-25T09:53:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.388+0000 7ff09adc4640 1 -- 172.21.15.19:0/1042575732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0940cbe20 con 0x7ff09415dc20 2024-04-25T09:53:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.388+0000 7ff09adc4640 1 -- 172.21.15.19:0/1042575732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff094073a00 con 0x7ff09415dc20 2024-04-25T09:53:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.389+0000 7ff0917fa640 1 -- 172.21.15.19:0/1042575732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff080018020 con 0x7ff09415dc20 2024-04-25T09:53:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.389+0000 7ff0917fa640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff07806b4d0 0x7ff07806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.389+0000 7ff0917fa640 1 -- 172.21.15.19:0/1042575732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff08008a1a0 con 0x7ff09415dc20 2024-04-25T09:53:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.390+0000 7ff098b60640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff07806b4d0 0x7ff07806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.390+0000 7ff098b60640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff07806b4d0 0x7ff07806d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff088034000 tx=0x7ff08802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.393+0000 7ff0917fa640 1 -- 172.21.15.19:0/1042575732 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff08001f030 con 0x7ff09415dc20 2024-04-25T09:53:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.575+0000 7ff09adc4640 1 -- 172.21.15.19:0/1042575732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff0940d16b0 con 0x7ff09415dc20 2024-04-25T09:53:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.575+0000 7ff0917fa640 1 -- 172.21.15.19:0/1042575732 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff0800172c0 con 0x7ff09415dc20 2024-04-25T09:53:48.577 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 -- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff07806b4d0 msgr2=0x7ff07806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff07806b4d0 0x7ff07806d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff088034000 tx=0x7ff08802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 -- 172.21.15.19:0/1042575732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09415dc20 msgr2=0x7ff0941f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09415dc20 0x7ff0941f41e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff08000b3f0 tx=0x7ff08000b8b0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 -- 172.21.15.19:0/1042575732 shutdown_connections 2024-04-25T09:53:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff07806b4d0 0x7ff07806d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff094161710 0x7ff0940cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09415dc20 0x7ff0941f41e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 --2- 172.21.15.19:0/1042575732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0940d32f0 0x7ff094159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 -- 172.21.15.19:0/1042575732 >> 172.21.15.19:0/1042575732 conn(0x7ff0940d0420 msgr2=0x7ff0940b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:48.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 -- 172.21.15.19:0/1042575732 shutdown_connections 2024-04-25T09:53:48.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.577+0000 7ff09adc4640 1 -- 172.21.15.19:0/1042575732 wait complete. 2024-04-25T09:53:48.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T09:53:48.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.689+0000 7fe50d86b640 1 Processor -- start 2024-04-25T09:53:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.689+0000 7fe50d86b640 1 -- start start 2024-04-25T09:53:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe50d86b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe5080b6cb0 0x7fe5080b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe50d86b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe5080b5a60 0x7fe5080d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe50d86b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5080ceee0 con 0x7fe50815db70 2024-04-25T09:53:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe50d86b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5080d1d10 con 0x7fe5080b6cb0 2024-04-25T09:53:48.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe50d86b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5080d1ff0 con 0x7fe5080b5a60 2024-04-25T09:53:48.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe4fffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe5080b6cb0 0x7fe5080b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe506ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe50815db70 0x7fe508162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35026/0 (socket says 172.21.15.19:35026) 2024-04-25T09:53:48.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe506ffd640 1 -- 172.21.15.19:0/1026043780 learned_addr learned my addr 172.21.15.19:0/1026043780 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:48.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe5077fe640 1 --2- 172.21.15.19:0/1026043780 >> v2:172.21.15.112:3300/0 conn(0x7fe5080b5a60 0x7fe5080d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe4fffff640 1 -- 172.21.15.19:0/1026043780 >> v2:172.21.15.112:3300/0 conn(0x7fe5080b5a60 msgr2=0x7fe5080d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:48.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe4fffff640 1 --2- 172.21.15.19:0/1026043780 >> v2:172.21.15.112:3300/0 conn(0x7fe5080b5a60 0x7fe5080d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe4fffff640 1 -- 172.21.15.19:0/1026043780 >> v1:172.21.15.19:6789/0 conn(0x7fe50815db70 legacy=0x7fe508162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe4fffff640 1 -- 172.21.15.19:0/1026043780 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080d22d0 con 0x7fe5080b6cb0 2024-04-25T09:53:48.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe5067fc640 1 -- 172.21.15.19:0/1026043780 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868980801 0 0) 0x7fe5080ceee0 con 0x7fe50815db70 2024-04-25T09:53:48.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.690+0000 7fe5077fe640 1 --2- 172.21.15.19:0/1026043780 >> v2:172.21.15.112:3300/0 conn(0x7fe5080b5a60 0x7fe5080d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:48.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.691+0000 7fe4fffff640 1 --2- 172.21.15.19:0/1026043780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5080b6cb0 0x7fe5080b5520 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe5080cfa10 tx=0x7fe4f002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141bc70486df2b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.691+0000 7fe5067fc640 1 -- 172.21.15.19:0/1026043780 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0037070 con 0x7fe5080b6cb0 2024-04-25T09:53:48.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.691+0000 7fe5067fc640 1 -- 172.21.15.19:0/1026043780 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f0032440 con 0x7fe5080b6cb0 2024-04-25T09:53:48.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.691+0000 7fe5067fc640 1 -- 172.21.15.19:0/1026043780 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f002ae60 con 0x7fe5080b6cb0 2024-04-25T09:53:48.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.692+0000 7fe50d86b640 1 -- 172.21.15.19:0/1026043780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5080b6cb0 msgr2=0x7fe5080b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.692+0000 7fe50d86b640 1 --2- 172.21.15.19:0/1026043780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5080b6cb0 0x7fe5080b5520 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe5080cfa10 tx=0x7fe4f002a110 comp rx=0 tx=0).stop 2024-04-25T09:53:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.692+0000 7fe50d86b640 1 -- 172.21.15.19:0/1026043780 shutdown_connections 2024-04-25T09:53:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.692+0000 7fe50d86b640 1 --2- 172.21.15.19:0/1026043780 >> v2:172.21.15.112:3300/0 conn(0x7fe5080b5a60 0x7fe5080d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.692+0000 7fe50d86b640 1 --2- 172.21.15.19:0/1026043780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5080b6cb0 0x7fe5080b5520 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.692+0000 7fe50d86b640 1 -- 172.21.15.19:0/1026043780 >> 172.21.15.19:0/1026043780 conn(0x7fe5080d0520 msgr2=0x7fe5080d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.692+0000 7fe50d86b640 1 -- 172.21.15.19:0/1026043780 shutdown_connections 2024-04-25T09:53:48.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.692+0000 7fe50d86b640 1 -- 172.21.15.19:0/1026043780 wait complete. 2024-04-25T09:53:48.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.693+0000 7fe50d86b640 1 Processor -- start 2024-04-25T09:53:48.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.693+0000 7fe50d86b640 1 -- start start 2024-04-25T09:53:48.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe50d86b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5080b5a60 0x7fe50814bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe50d86b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5080b6cb0 0x7fe5080ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe50d86b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe50815db70 0x7fe50815f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe50d86b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080cf1c0 con 0x7fe50815db70 2024-04-25T09:53:48.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe50d86b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080ce530 con 0x7fe5080b5a60 2024-04-25T09:53:48.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe50d86b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080cf340 con 0x7fe5080b6cb0 2024-04-25T09:53:48.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe5077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe50815db70 0x7fe50815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe5077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe50815db70 0x7fe50815f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58932/0 (socket says 172.21.15.19:58932) 2024-04-25T09:53:48.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe5077fe640 1 -- 172.21.15.19:0/2614570936 learned_addr learned my addr 172.21.15.19:0/2614570936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:48.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe506ffd640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5080b5a60 0x7fe50814bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe4fffff640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5080b6cb0 0x7fe5080ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe5077fe640 1 -- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5080b6cb0 msgr2=0x7fe5080ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe5077fe640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5080b6cb0 0x7fe5080ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe5077fe640 1 -- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5080b5a60 msgr2=0x7fe50814bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe5077fe640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5080b5a60 0x7fe50814bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe5077fe640 1 -- 172.21.15.19:0/2614570936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50814c400 con 0x7fe50815db70 2024-04-25T09:53:48.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe506ffd640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5080b5a60 0x7fe50814bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:48.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.694+0000 7fe5077fe640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe50815db70 0x7fe50815f850 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe500007ef0 tx=0x7fe50000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.695+0000 7fe504ff9640 1 -- 172.21.15.19:0/2614570936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50001a070 con 0x7fe50815db70 2024-04-25T09:53:48.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.695+0000 7fe504ff9640 1 -- 172.21.15.19:0/2614570936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe500003b90 con 0x7fe50815db70 2024-04-25T09:53:48.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.695+0000 7fe50d86b640 1 -- 172.21.15.19:0/2614570936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50815ef70 con 0x7fe50815db70 2024-04-25T09:53:48.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.695+0000 7fe504ff9640 1 -- 172.21.15.19:0/2614570936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50000de20 con 0x7fe50815db70 2024-04-25T09:53:48.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.695+0000 7fe50d86b640 1 -- 172.21.15.19:0/2614570936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5080caef0 con 0x7fe50815db70 2024-04-25T09:53:48.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.696+0000 7fe50d86b640 1 -- 172.21.15.19:0/2614570936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe508073990 con 0x7fe50815db70 2024-04-25T09:53:48.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.696+0000 7fe504ff9640 1 -- 172.21.15.19:0/2614570936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe50001d070 con 0x7fe50815db70 2024-04-25T09:53:48.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.697+0000 7fe504ff9640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4e806b4d0 0x7fe4e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.697+0000 7fe504ff9640 1 -- 172.21.15.19:0/2614570936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe50008a6d0 con 0x7fe50815db70 2024-04-25T09:53:48.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.697+0000 7fe506ffd640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4e806b4d0 0x7fe4e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.698+0000 7fe506ffd640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4e806b4d0 0x7fe4e806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8000f60 tx=0x7fe4f8007ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.701+0000 7fe504ff9640 1 -- 172.21.15.19:0/2614570936 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe50814c400 con 0x7fe50815db70 2024-04-25T09:53:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:48.953+0000 7fe50d86b640 1 -- 172.21.15.19:0/2614570936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7fe5080d1820 con 0x7fe50815db70 2024-04-25T09:53:49.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.082+0000 7fe504ff9640 1 -- 172.21.15.19:0/2614570936 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v62) v1 ==== 151+0+0 (secure 0 0 0) 0x7fe500010070 con 0x7fe50815db70 2024-04-25T09:53:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 -- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4e806b4d0 msgr2=0x7fe4e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4e806b4d0 0x7fe4e806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8000f60 tx=0x7fe4f8007ca0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 -- 172.21.15.19:0/2614570936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe50815db70 msgr2=0x7fe50815f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe50815db70 0x7fe50815f850 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe500007ef0 tx=0x7fe50000d4a0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 -- 172.21.15.19:0/2614570936 shutdown_connections 2024-04-25T09:53:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4e806b4d0 0x7fe4e806d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe50815db70 0x7fe50815f850 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5080b6cb0 0x7fe5080ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 --2- 172.21.15.19:0/2614570936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5080b5a60 0x7fe50814bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 -- 172.21.15.19:0/2614570936 >> 172.21.15.19:0/2614570936 conn(0x7fe5080d0520 msgr2=0x7fe508162150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 -- 172.21.15.19:0/2614570936 shutdown_connections 2024-04-25T09:53:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.084+0000 7fe50d86b640 1 -- 172.21.15.19:0/2614570936 wait complete. 2024-04-25T09:53:49.098 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:53:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.199+0000 7feb78dc1640 1 Processor -- start 2024-04-25T09:53:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.199+0000 7feb78dc1640 1 -- start start 2024-04-25T09:53:49.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.200+0000 7feb78dc1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feb7415dac0 0x7feb740d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.200+0000 7feb78dc1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb740d1f30 0x7feb740d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.200+0000 7feb78dc1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb740d2420 con 0x7feb740d34c0 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.200+0000 7feb78dc1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb740d3030 con 0x7feb740d1f30 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.200+0000 7feb78dc1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb740d2700 con 0x7feb7415dac0 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.200+0000 7feb71d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb740d1f30 0x7feb740d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.200+0000 7feb71d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb740d1f30 0x7feb740d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58940/0 (socket says 172.21.15.19:58940) 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.200+0000 7feb71d9b640 1 -- 172.21.15.19:0/3289559 learned_addr learned my addr 172.21.15.19:0/3289559 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.201+0000 7feb71d9b640 1 -- 172.21.15.19:0/3289559 >> v2:172.21.15.112:3300/0 conn(0x7feb7415dac0 msgr2=0x7feb740d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.201+0000 7feb7259c640 1 --2- 172.21.15.19:0/3289559 >> v2:172.21.15.112:3300/0 conn(0x7feb7415dac0 0x7feb740d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.201+0000 7feb71d9b640 1 --2- 172.21.15.19:0/3289559 >> v2:172.21.15.112:3300/0 conn(0x7feb7415dac0 0x7feb740d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.201+0000 7feb71d9b640 1 -- 172.21.15.19:0/3289559 >> v1:172.21.15.112:6789/0 conn(0x7feb740d34c0 legacy=0x7feb740b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.201+0000 7feb71d9b640 1 -- 172.21.15.19:0/3289559 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb740d29e0 con 0x7feb740d1f30 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.201+0000 7feb7259c640 1 --2- 172.21.15.19:0/3289559 >> v2:172.21.15.112:3300/0 conn(0x7feb7415dac0 0x7feb740d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.201+0000 7feb71d9b640 1 --2- 172.21.15.19:0/3289559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb740d1f30 0x7feb740d03a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb740b6520 tx=0x7feb6802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c566ac3a9a494c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.201+0000 7feb7159a640 1 -- 172.21.15.19:0/3289559 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68004030 con 0x7feb740d1f30 2024-04-25T09:53:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.202+0000 7feb7159a640 1 -- 172.21.15.19:0/3289559 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb680041d0 con 0x7feb740d1f30 2024-04-25T09:53:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.202+0000 7feb7159a640 1 -- 172.21.15.19:0/3289559 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68003610 con 0x7feb740d1f30 2024-04-25T09:53:49.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.202+0000 7feb78dc1640 1 -- 172.21.15.19:0/3289559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb740d1f30 msgr2=0x7feb740d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.202+0000 7feb78dc1640 1 --2- 172.21.15.19:0/3289559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb740d1f30 0x7feb740d03a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb740b6520 tx=0x7feb6802a290 comp rx=0 tx=0).stop 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.202+0000 7feb78dc1640 1 -- 172.21.15.19:0/3289559 shutdown_connections 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.202+0000 7feb78dc1640 1 --2- 172.21.15.19:0/3289559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb740d1f30 0x7feb740d03a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.202+0000 7feb78dc1640 1 --2- 172.21.15.19:0/3289559 >> v2:172.21.15.112:3300/0 conn(0x7feb7415dac0 0x7feb740d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.202+0000 7feb78dc1640 1 -- 172.21.15.19:0/3289559 >> 172.21.15.19:0/3289559 conn(0x7feb74061c30 msgr2=0x7feb7406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.202+0000 7feb78dc1640 1 -- 172.21.15.19:0/3289559 shutdown_connections 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.202+0000 7feb78dc1640 1 -- 172.21.15.19:0/3289559 wait complete. 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.203+0000 7feb78dc1640 1 Processor -- start 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.204+0000 7feb78dc1640 1 -- start start 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.204+0000 7feb78dc1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb740d1f30 0x7feb7414a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.204+0000 7feb78dc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb740d34c0 0x7feb7415e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.204+0000 7feb78dc1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb7415dac0 0x7feb74160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.204+0000 7feb78dc1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb740b97d0 con 0x7feb7415dac0 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.204+0000 7feb78dc1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7415dea0 con 0x7feb740d34c0 2024-04-25T09:53:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.204+0000 7feb78dc1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb740b9950 con 0x7feb740d1f30 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.204+0000 7feb72d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb7415dac0 0x7feb74160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.204+0000 7feb72d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb7415dac0 0x7feb74160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58952/0 (socket says 172.21.15.19:58952) 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.204+0000 7feb72d9d640 1 -- 172.21.15.19:0/137720101 learned_addr learned my addr 172.21.15.19:0/137720101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb7259c640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb740d1f30 0x7feb7414a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb71d9b640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb740d34c0 0x7feb7415e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb72d9d640 1 -- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb740d1f30 msgr2=0x7feb7414a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb72d9d640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb740d1f30 0x7feb7414a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb72d9d640 1 -- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb740d34c0 msgr2=0x7feb7415e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb72d9d640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb740d34c0 0x7feb7415e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb72d9d640 1 -- 172.21.15.19:0/137720101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7414ab20 con 0x7feb7415dac0 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb71d9b640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb740d34c0 0x7feb7415e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb72d9d640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb7415dac0 0x7feb74160790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feb6400c410 tx=0x7feb6400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb62ffd640 1 -- 172.21.15.19:0/137720101 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64011790 con 0x7feb7415dac0 2024-04-25T09:53:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb62ffd640 1 -- 172.21.15.19:0/137720101 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb64017070 con 0x7feb7415dac0 2024-04-25T09:53:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb78dc1640 1 -- 172.21.15.19:0/137720101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7414ae00 con 0x7feb7415dac0 2024-04-25T09:53:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb62ffd640 1 -- 172.21.15.19:0/137720101 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64011bb0 con 0x7feb7415dac0 2024-04-25T09:53:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.205+0000 7feb78dc1640 1 -- 172.21.15.19:0/137720101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7414b3c0 con 0x7feb7415dac0 2024-04-25T09:53:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.206+0000 7feb78dc1640 1 -- 172.21.15.19:0/137720101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb740739e0 con 0x7feb7415dac0 2024-04-25T09:53:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.206+0000 7feb62ffd640 1 -- 172.21.15.19:0/137720101 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb6401c460 con 0x7feb7415dac0 2024-04-25T09:53:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.207+0000 7feb62ffd640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb5406b4d0 0x7feb5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.207+0000 7feb62ffd640 1 -- 172.21.15.19:0/137720101 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7feb6408a160 con 0x7feb7415dac0 2024-04-25T09:53:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.207+0000 7feb7259c640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb5406b4d0 0x7feb5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.208+0000 7feb7259c640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb5406b4d0 0x7feb5406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feb6c007510 tx=0x7feb6c0074a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.211+0000 7feb62ffd640 1 -- 172.21.15.19:0/137720101 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb64021020 con 0x7feb7415dac0 2024-04-25T09:53:49.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.392+0000 7feb78dc1640 1 -- 172.21.15.19:0/137720101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7feb740d0c30 con 0x7feb7415dac0 2024-04-25T09:53:49.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.396+0000 7feb62ffd640 1 -- 172.21.15.19:0/137720101 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7feb64058c70 con 0x7feb7415dac0 2024-04-25T09:53:49.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 -- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb5406b4d0 msgr2=0x7feb5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb5406b4d0 0x7feb5406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feb6c007510 tx=0x7feb6c0074a0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 -- 172.21.15.19:0/137720101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb7415dac0 msgr2=0x7feb74160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb7415dac0 0x7feb74160790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feb6400c410 tx=0x7feb6400c8d0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 -- 172.21.15.19:0/137720101 shutdown_connections 2024-04-25T09:53:49.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb5406b4d0 0x7feb5406d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb7415dac0 0x7feb74160790 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb740d34c0 0x7feb7415e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 --2- 172.21.15.19:0/137720101 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb740d1f30 0x7feb7414a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 -- 172.21.15.19:0/137720101 >> 172.21.15.19:0/137720101 conn(0x7feb74061c30 msgr2=0x7feb740c7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:49.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 -- 172.21.15.19:0/137720101 shutdown_connections 2024-04-25T09:53:49.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.398+0000 7feb78dc1640 1 -- 172.21.15.19:0/137720101 wait complete. 2024-04-25T09:53:49.411 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:53:49.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:53:49.451 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:53:49.499 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:53:49.499 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:53:49.499 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:49.515 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:53:49.515 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:49.572 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:53:49.573 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:49.629 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:49.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.767+0000 7f9794ee2640 1 Processor -- start 2024-04-25T09:53:49.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.767+0000 7f9794ee2640 1 -- start start 2024-04-25T09:53:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.768+0000 7f9794ee2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f979015ba00 0x7f979009a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.768+0000 7f9794ee2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97901602e0 0x7f97900ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.768+0000 7f9794ee2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97900cd710 con 0x7f979015f5f0 2024-04-25T09:53:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.768+0000 7f9794ee2640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9790099b70 con 0x7f979015ba00 2024-04-25T09:53:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.768+0000 7f9794ee2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9790099e50 con 0x7f97901602e0 2024-04-25T09:53:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.768+0000 7f978e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f979015ba00 0x7f979009a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.768+0000 7f978e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f979015ba00 0x7f979009a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58964/0 (socket says 172.21.15.19:58964) 2024-04-25T09:53:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.768+0000 7f978e59c640 1 -- 172.21.15.19:0/3611977745 learned_addr learned my addr 172.21.15.19:0/3611977745 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.769+0000 7f978dd9b640 1 --2- 172.21.15.19:0/3611977745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97901602e0 0x7f97900ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.769+0000 7f978e59c640 1 -- 172.21.15.19:0/3611977745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97901602e0 msgr2=0x7f97900ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.769+0000 7f978e59c640 1 --2- 172.21.15.19:0/3611977745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97901602e0 0x7f97900ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.769+0000 7f978e59c640 1 -- 172.21.15.19:0/3611977745 >> v1:172.21.15.112:6789/0 conn(0x7f979015f5f0 legacy=0x7f97900c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.769+0000 7f978e59c640 1 -- 172.21.15.19:0/3611977745 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979009a130 con 0x7f979015ba00 2024-04-25T09:53:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.769+0000 7f978dd9b640 1 --2- 172.21.15.19:0/3611977745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97901602e0 0x7f97900ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.769+0000 7f978e59c640 1 --2- 172.21.15.19:0/3611977745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f979015ba00 0x7f979009a930 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9788009870 tx=0x7f978802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73e95b47eb6736c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.770+0000 7f978d59a640 1 -- 172.21.15.19:0/3611977745 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788037070 con 0x7f979015ba00 2024-04-25T09:53:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.770+0000 7f978d59a640 1 -- 172.21.15.19:0/3611977745 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978802ae10 con 0x7f979015ba00 2024-04-25T09:53:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.770+0000 7f978d59a640 1 -- 172.21.15.19:0/3611977745 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788032660 con 0x7f979015ba00 2024-04-25T09:53:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.770+0000 7f9794ee2640 1 -- 172.21.15.19:0/3611977745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f979015ba00 msgr2=0x7f979009a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.770+0000 7f9794ee2640 1 --2- 172.21.15.19:0/3611977745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f979015ba00 0x7f979009a930 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9788009870 tx=0x7f978802a100 comp rx=0 tx=0).stop 2024-04-25T09:53:49.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.770+0000 7f9794ee2640 1 -- 172.21.15.19:0/3611977745 shutdown_connections 2024-04-25T09:53:49.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.770+0000 7f9794ee2640 1 --2- 172.21.15.19:0/3611977745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97901602e0 0x7f97900ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.770+0000 7f9794ee2640 1 --2- 172.21.15.19:0/3611977745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f979015ba00 0x7f979009a930 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.770+0000 7f9794ee2640 1 -- 172.21.15.19:0/3611977745 >> 172.21.15.19:0/3611977745 conn(0x7f97900b3270 msgr2=0x7f97900b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:49.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.770+0000 7f9794ee2640 1 -- 172.21.15.19:0/3611977745 shutdown_connections 2024-04-25T09:53:49.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.770+0000 7f9794ee2640 1 -- 172.21.15.19:0/3611977745 wait complete. 2024-04-25T09:53:49.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.772+0000 7f9794ee2640 1 Processor -- start 2024-04-25T09:53:49.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.772+0000 7f9794ee2640 1 -- start start 2024-04-25T09:53:49.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.772+0000 7f9794ee2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f979015ba00 0x7f9790150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.772+0000 7f9794ee2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f979015f5f0 0x7f97900cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.772+0000 7f9794ee2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97901602e0 0x7f9790158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.772+0000 7f9794ee2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979015bde0 con 0x7f97901602e0 2024-04-25T09:53:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.772+0000 7f9794ee2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97900cd2e0 con 0x7f979015f5f0 2024-04-25T09:53:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.772+0000 7f9794ee2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97900b46a0 con 0x7f979015ba00 2024-04-25T09:53:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.772+0000 7f978ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97901602e0 0x7f9790158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.772+0000 7f978ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97901602e0 0x7f9790158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58972/0 (socket says 172.21.15.19:58972) 2024-04-25T09:53:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.772+0000 7f978ed9d640 1 -- 172.21.15.19:0/2659406459 learned_addr learned my addr 172.21.15.19:0/2659406459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f978e59c640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f979015ba00 0x7f9790150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f978ed9d640 1 -- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f979015ba00 msgr2=0x7f9790150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f978ed9d640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f979015ba00 0x7f9790150d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f978ed9d640 1 -- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f979015f5f0 msgr2=0x7f97900cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f978dd9b640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f979015f5f0 0x7f97900cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f978ed9d640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f979015f5f0 0x7f97900cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f978ed9d640 1 -- 172.21.15.19:0/2659406459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97900b4980 con 0x7f97901602e0 2024-04-25T09:53:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f978ed9d640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97901602e0 0x7f9790158600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f978000b970 tx=0x7f978000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f9786ffd640 1 -- 172.21.15.19:0/2659406459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780019070 con 0x7f97901602e0 2024-04-25T09:53:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f9786ffd640 1 -- 172.21.15.19:0/2659406459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978000cb40 con 0x7f97901602e0 2024-04-25T09:53:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f9786ffd640 1 -- 172.21.15.19:0/2659406459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780014440 con 0x7f97901602e0 2024-04-25T09:53:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f9794ee2640 1 -- 172.21.15.19:0/2659406459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97900b4c60 con 0x7f97901602e0 2024-04-25T09:53:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.773+0000 7f9794ee2640 1 -- 172.21.15.19:0/2659406459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97900b5220 con 0x7f97901602e0 2024-04-25T09:53:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.774+0000 7f9794ee2640 1 -- 172.21.15.19:0/2659406459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9790073990 con 0x7f97901602e0 2024-04-25T09:53:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.775+0000 7f9786ffd640 1 -- 172.21.15.19:0/2659406459 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f978001c070 con 0x7f97901602e0 2024-04-25T09:53:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.775+0000 7f9786ffd640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977006b4d0 0x7f977006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.775+0000 7f9786ffd640 1 -- 172.21.15.19:0/2659406459 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f978008a1d0 con 0x7f97901602e0 2024-04-25T09:53:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.776+0000 7f978e59c640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977006b4d0 0x7f977006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.776+0000 7f978e59c640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977006b4d0 0x7f977006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9788034000 tx=0x7f978802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:49.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.780+0000 7f9786ffd640 1 -- 172.21.15.19:0/2659406459 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9780058c50 con 0x7f97901602e0 2024-04-25T09:53:49.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.960+0000 7f9794ee2640 1 -- 172.21.15.19:0/2659406459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97900b40a0 con 0x7f97901602e0 2024-04-25T09:53:49.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.961+0000 7f9786ffd640 1 -- 172.21.15.19:0/2659406459 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18892 (secure 0 0 0) 0x7f9780058550 con 0x7f97901602e0 2024-04-25T09:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:49.963 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:48.071585+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:49.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 -- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977006b4d0 msgr2=0x7f977006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977006b4d0 0x7f977006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9788034000 tx=0x7f978802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:49.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 -- 172.21.15.19:0/2659406459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97901602e0 msgr2=0x7f9790158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97901602e0 0x7f9790158600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f978000b970 tx=0x7f978000be30 comp rx=0 tx=0).stop 2024-04-25T09:53:49.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 -- 172.21.15.19:0/2659406459 shutdown_connections 2024-04-25T09:53:49.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977006b4d0 0x7f977006d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97901602e0 0x7f9790158600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f979015f5f0 0x7f97900cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 --2- 172.21.15.19:0/2659406459 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f979015ba00 0x7f9790150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 -- 172.21.15.19:0/2659406459 >> 172.21.15.19:0/2659406459 conn(0x7f97900b3270 msgr2=0x7f979015d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:49.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 -- 172.21.15.19:0/2659406459 shutdown_connections 2024-04-25T09:53:49.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:49.963+0000 7f9794ee2640 1 -- 172.21.15.19:0/2659406459 wait complete. 2024-04-25T09:53:49.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:49.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.078+0000 7f7b1b77e640 1 Processor -- start 2024-04-25T09:53:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.078+0000 7f7b1b77e640 1 -- start start 2024-04-25T09:53:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b1b77e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7b140cd8a0 0x7f7b1415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b1b77e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7b14150b90 0x7f7b14151020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b1b77e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1415e6e0 con 0x7f7b140cd4c0 2024-04-25T09:53:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b1b77e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b14151560 con 0x7f7b140cd8a0 2024-04-25T09:53:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b1b77e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1415e9c0 con 0x7f7b14150b90 2024-04-25T09:53:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b18d19640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7b140cd8a0 0x7f7b1415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b18d19640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7b140cd8a0 0x7f7b1415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58988/0 (socket says 172.21.15.19:58988) 2024-04-25T09:53:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b18d19640 1 -- 172.21.15.19:0/3006638286 learned_addr learned my addr 172.21.15.19:0/3006638286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b19d1b640 1 --2- 172.21.15.19:0/3006638286 >> v2:172.21.15.112:3300/0 conn(0x7f7b14150b90 0x7f7b14151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b18d19640 1 -- 172.21.15.19:0/3006638286 >> v2:172.21.15.112:3300/0 conn(0x7f7b14150b90 msgr2=0x7f7b14151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b18d19640 1 --2- 172.21.15.19:0/3006638286 >> v2:172.21.15.112:3300/0 conn(0x7f7b14150b90 0x7f7b14151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b18d19640 1 -- 172.21.15.19:0/3006638286 >> v1:172.21.15.112:6789/0 conn(0x7f7b140cd4c0 legacy=0x7f7b1415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b18d19640 1 -- 172.21.15.19:0/3006638286 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1415a9c0 con 0x7f7b140cd8a0 2024-04-25T09:53:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.079+0000 7f7b19d1b640 1 --2- 172.21.15.19:0/3006638286 >> v2:172.21.15.112:3300/0 conn(0x7f7b14150b90 0x7f7b14151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.080+0000 7f7b18d19640 1 --2- 172.21.15.19:0/3006638286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b140cd8a0 0x7f7b1415b1c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7b140cec10 tx=0x7f7b0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9804de1a72c4f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.080+0000 7f7b0bfff640 1 -- 172.21.15.19:0/3006638286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04037070 con 0x7f7b140cd8a0 2024-04-25T09:53:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.080+0000 7f7b0bfff640 1 -- 172.21.15.19:0/3006638286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b04032440 con 0x7f7b140cd8a0 2024-04-25T09:53:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.080+0000 7f7b0bfff640 1 -- 172.21.15.19:0/3006638286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04032810 con 0x7f7b140cd8a0 2024-04-25T09:53:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.080+0000 7f7b1b77e640 1 -- 172.21.15.19:0/3006638286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b140cd8a0 msgr2=0x7f7b1415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.080+0000 7f7b1b77e640 1 --2- 172.21.15.19:0/3006638286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b140cd8a0 0x7f7b1415b1c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7b140cec10 tx=0x7f7b0402a270 comp rx=0 tx=0).stop 2024-04-25T09:53:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.080+0000 7f7b1b77e640 1 -- 172.21.15.19:0/3006638286 shutdown_connections 2024-04-25T09:53:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.081+0000 7f7b1b77e640 1 --2- 172.21.15.19:0/3006638286 >> v2:172.21.15.112:3300/0 conn(0x7f7b14150b90 0x7f7b14151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.081+0000 7f7b1b77e640 1 --2- 172.21.15.19:0/3006638286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b140cd8a0 0x7f7b1415b1c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.081+0000 7f7b1b77e640 1 -- 172.21.15.19:0/3006638286 >> 172.21.15.19:0/3006638286 conn(0x7f7b1406abe0 msgr2=0x7f7b1406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.081+0000 7f7b1b77e640 1 -- 172.21.15.19:0/3006638286 shutdown_connections 2024-04-25T09:53:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.081+0000 7f7b1b77e640 1 -- 172.21.15.19:0/3006638286 wait complete. 2024-04-25T09:53:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.082+0000 7f7b1b77e640 1 Processor -- start 2024-04-25T09:53:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.082+0000 7f7b1b77e640 1 -- start start 2024-04-25T09:53:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.082+0000 7f7b1b77e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b140cd4c0 0x7f7b1413f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.082+0000 7f7b1b77e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b140cd8a0 0x7f7b1415c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b1b77e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b14150b90 0x7f7b1415ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b1b77e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1415e370 con 0x7f7b14150b90 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b1b77e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1415a670 con 0x7f7b140cd4c0 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b1b77e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1406eca0 con 0x7f7b140cd8a0 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b19d1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b14150b90 0x7f7b1415ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b19d1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b14150b90 0x7f7b1415ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59004/0 (socket says 172.21.15.19:59004) 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b19d1b640 1 -- 172.21.15.19:0/434236590 learned_addr learned my addr 172.21.15.19:0/434236590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b1951a640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b140cd4c0 0x7f7b1413f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b18d19640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b140cd8a0 0x7f7b1415c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b19d1b640 1 -- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b140cd8a0 msgr2=0x7f7b1415c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b19d1b640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b140cd8a0 0x7f7b1415c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b19d1b640 1 -- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b140cd4c0 msgr2=0x7f7b1413f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b19d1b640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b140cd4c0 0x7f7b1413f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b19d1b640 1 -- 172.21.15.19:0/434236590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b14142ee0 con 0x7f7b14150b90 2024-04-25T09:53:50.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.083+0000 7f7b19d1b640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b14150b90 0x7f7b1415ca60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c007df0 tx=0x7f7b0c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.087+0000 7f7b09ffb640 1 -- 172.21.15.19:0/434236590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c01a070 con 0x7f7b14150b90 2024-04-25T09:53:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.087+0000 7f7b09ffb640 1 -- 172.21.15.19:0/434236590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c003990 con 0x7f7b14150b90 2024-04-25T09:53:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.087+0000 7f7b1b77e640 1 -- 172.21.15.19:0/434236590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b14152c10 con 0x7f7b14150b90 2024-04-25T09:53:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.087+0000 7f7b09ffb640 1 -- 172.21.15.19:0/434236590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c003d80 con 0x7f7b14150b90 2024-04-25T09:53:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.087+0000 7f7b1b77e640 1 -- 172.21.15.19:0/434236590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b14143430 con 0x7f7b14150b90 2024-04-25T09:53:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.088+0000 7f7b1b77e640 1 -- 172.21.15.19:0/434236590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b14072190 con 0x7f7b14150b90 2024-04-25T09:53:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.088+0000 7f7b09ffb640 1 -- 172.21.15.19:0/434236590 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b0c01d070 con 0x7f7b14150b90 2024-04-25T09:53:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.089+0000 7f7b09ffb640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7af806b4d0 0x7f7af806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.089+0000 7f7b09ffb640 1 -- 172.21.15.19:0/434236590 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7b0c08a740 con 0x7f7b14150b90 2024-04-25T09:53:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.089+0000 7f7b1951a640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7af806b4d0 0x7f7af806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.090+0000 7f7b1951a640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7af806b4d0 0x7f7af806d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b10001780 tx=0x7f7b10009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.093+0000 7f7b09ffb640 1 -- 172.21.15.19:0/434236590 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b14072190 con 0x7f7b14150b90 2024-04-25T09:53:50.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.273+0000 7f7b1b77e640 1 -- 172.21.15.19:0/434236590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b140b4960 con 0x7f7b14150b90 2024-04-25T09:53:50.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.273+0000 7f7b09ffb640 1 -- 172.21.15.19:0/434236590 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b140b4960 con 0x7f7b14150b90 2024-04-25T09:53:50.276 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:50.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.275+0000 7f7b1b77e640 1 -- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7af806b4d0 msgr2=0x7f7af806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.275+0000 7f7b1b77e640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7af806b4d0 0x7f7af806d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b10001780 tx=0x7f7b10009040 comp rx=0 tx=0).stop 2024-04-25T09:53:50.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.275+0000 7f7b1b77e640 1 -- 172.21.15.19:0/434236590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b14150b90 msgr2=0x7f7b1415ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.275+0000 7f7b1b77e640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b14150b90 0x7f7b1415ca60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c007df0 tx=0x7f7b0c00d4a0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.275+0000 7f7b1b77e640 1 -- 172.21.15.19:0/434236590 shutdown_connections 2024-04-25T09:53:50.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.275+0000 7f7b1b77e640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7af806b4d0 0x7f7af806d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.275+0000 7f7b1b77e640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b14150b90 0x7f7b1415ca60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.275+0000 7f7b1b77e640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b140cd8a0 0x7f7b1415c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.275+0000 7f7b1b77e640 1 --2- 172.21.15.19:0/434236590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b140cd4c0 0x7f7b1413f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.275+0000 7f7b1b77e640 1 -- 172.21.15.19:0/434236590 >> 172.21.15.19:0/434236590 conn(0x7f7b1406abe0 msgr2=0x7f7b1406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:50.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.276+0000 7f7b1b77e640 1 -- 172.21.15.19:0/434236590 shutdown_connections 2024-04-25T09:53:50.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.276+0000 7f7b1b77e640 1 -- 172.21.15.19:0/434236590 wait complete. 2024-04-25T09:53:50.288 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.390+0000 7ff9538ac640 1 Processor -- start 2024-04-25T09:53:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.390+0000 7ff9538ac640 1 -- start start 2024-04-25T09:53:50.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff9538ac640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff94c0b5c40 0x7ff94c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff9538ac640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94c0cf320 con 0x7ff94c15b5a0 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff9538ac640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94c0cf040 con 0x7ff94c15b980 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff9538ac640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94c161970 con 0x7ff94c0b5c40 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff951648640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff94c15b5a0 0x7ff94c0b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35030/0 (socket says 172.21.15.19:35030) 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff951648640 1 -- 172.21.15.19:0/3073186631 learned_addr learned my addr 172.21.15.19:0/3073186631 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172014049 0 0) 0x7ff948001110 con 0x7ff94c15b5a0 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff950e47640 1 --2- 172.21.15.19:0/3073186631 >> v2:172.21.15.112:3300/0 conn(0x7ff94c0b5c40 0x7ff94c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff938003520 con 0x7ff94c15b5a0 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3927951981 0 0) 0x7ff938003520 con 0x7ff94c15b5a0 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 >> v2:172.21.15.112:3300/0 conn(0x7ff94c0b5c40 msgr2=0x7ff94c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 --2- 172.21.15.19:0/3073186631 >> v2:172.21.15.112:3300/0 conn(0x7ff94c0b5c40 0x7ff94c0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 >> v1:172.21.15.112:6789/0 conn(0x7ff94c15b980 legacy=0x7ff94c0ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c161c50 con 0x7ff94c15b5a0 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7ff948001840 con 0x7ff94c15b5a0 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 >> v1:172.21.15.19:6789/0 conn(0x7ff94c15b5a0 legacy=0x7ff94c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 0x7ff938003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff938004800 0x7ff938004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9380051c0 0x7ff93800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c161970 con 0x7ff9380039b0 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93800c060 con 0x7ff938004800 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93800c260 con 0x7ff9380051c0 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.391+0000 7ff951648640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 0x7ff938003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.392+0000 7ff950e47640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff938004800 0x7ff938004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.392+0000 7ff951e49640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9380051c0 0x7ff93800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.392+0000 7ff951648640 1 -- 172.21.15.19:0/3073186631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9380051c0 msgr2=0x7ff93800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.392+0000 7ff951648640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9380051c0 0x7ff93800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.392+0000 7ff951648640 1 -- 172.21.15.19:0/3073186631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff938004800 msgr2=0x7ff938004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.392+0000 7ff951648640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff938004800 0x7ff938004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.392+0000 7ff951648640 1 -- 172.21.15.19:0/3073186631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938003520 con 0x7ff9380039b0 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.392+0000 7ff951e49640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9380051c0 0x7ff93800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.392+0000 7ff951e49640 1 -- 172.21.15.19:0/3073186631 reap_dead start 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.392+0000 7ff951648640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 0x7ff938003dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff94800c4c0 tx=0x7ff938005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfdb9d80d71f0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.393+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94800c0e0 con 0x7ff9380039b0 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.393+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff948034070 con 0x7ff9380039b0 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.393+0000 7ff943fff640 1 -- 172.21.15.19:0/3073186631 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff948005d70 con 0x7ff9380039b0 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.393+0000 7ff9538ac640 1 -- 172.21.15.19:0/3073186631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 msgr2=0x7ff938003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.393+0000 7ff9538ac640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 0x7ff938003dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff94800c4c0 tx=0x7ff938005830 comp rx=0 tx=0).stop 2024-04-25T09:53:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.393+0000 7ff9538ac640 1 -- 172.21.15.19:0/3073186631 shutdown_connections 2024-04-25T09:53:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.393+0000 7ff9538ac640 1 --2- 172.21.15.19:0/3073186631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 0x7ff938003dd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.393+0000 7ff9538ac640 1 -- 172.21.15.19:0/3073186631 >> 172.21.15.19:0/3073186631 conn(0x7ff94c0d0530 msgr2=0x7ff94c0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.393+0000 7ff9538ac640 1 -- 172.21.15.19:0/3073186631 shutdown_connections 2024-04-25T09:53:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.393+0000 7ff9538ac640 1 -- 172.21.15.19:0/3073186631 wait complete. 2024-04-25T09:53:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.394+0000 7ff9538ac640 1 Processor -- start 2024-04-25T09:53:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.394+0000 7ff9538ac640 1 -- start start 2024-04-25T09:53:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.395+0000 7ff9538ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 0x7ff94c151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.395+0000 7ff9538ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c152f00 0x7ff94c0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.395+0000 7ff9538ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c160220 0x7ff94c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.395+0000 7ff9538ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c1625d0 con 0x7ff9380039b0 2024-04-25T09:53:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.395+0000 7ff9538ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c162450 con 0x7ff94c152f00 2024-04-25T09:53:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.395+0000 7ff9538ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c162750 con 0x7ff94c160220 2024-04-25T09:53:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.395+0000 7ff951648640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 0x7ff94c151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.395+0000 7ff951648640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 0x7ff94c151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59024/0 (socket says 172.21.15.19:59024) 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.395+0000 7ff951648640 1 -- 172.21.15.19:0/2037085297 learned_addr learned my addr 172.21.15.19:0/2037085297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.395+0000 7ff950e47640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c152f00 0x7ff94c0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.395+0000 7ff951648640 1 -- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c160220 msgr2=0x7ff94c1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.396+0000 7ff951648640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c160220 0x7ff94c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.396+0000 7ff951648640 1 -- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c152f00 msgr2=0x7ff94c0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.396+0000 7ff951648640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c152f00 0x7ff94c0cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.396+0000 7ff951648640 1 -- 172.21.15.19:0/2037085297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c1506a0 con 0x7ff9380039b0 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.396+0000 7ff951648640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 0x7ff94c151740 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff94800c4c0 tx=0x7ff94c161e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.396+0000 7ff941ffb640 1 -- 172.21.15.19:0/2037085297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff948007480 con 0x7ff9380039b0 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.396+0000 7ff941ffb640 1 -- 172.21.15.19:0/2037085297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94803f030 con 0x7ff9380039b0 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.396+0000 7ff9538ac640 1 -- 172.21.15.19:0/2037085297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c150980 con 0x7ff9380039b0 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.396+0000 7ff941ffb640 1 -- 172.21.15.19:0/2037085297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff948000ed0 con 0x7ff9380039b0 2024-04-25T09:53:50.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.396+0000 7ff9538ac640 1 -- 172.21.15.19:0/2037085297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c150f40 con 0x7ff9380039b0 2024-04-25T09:53:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.397+0000 7ff9538ac640 1 -- 172.21.15.19:0/2037085297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94c073a00 con 0x7ff9380039b0 2024-04-25T09:53:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.398+0000 7ff941ffb640 1 -- 172.21.15.19:0/2037085297 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff94802f460 con 0x7ff9380039b0 2024-04-25T09:53:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.398+0000 7ff941ffb640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff93006b670 0x7ff93006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.398+0000 7ff941ffb640 1 -- 172.21.15.19:0/2037085297 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff9480aaf30 con 0x7ff9380039b0 2024-04-25T09:53:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.398+0000 7ff950e47640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff93006b670 0x7ff93006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.399+0000 7ff950e47640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff93006b670 0x7ff93006db30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff94c1502e0 tx=0x7ff93c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.402+0000 7ff941ffb640 1 -- 172.21.15.19:0/2037085297 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff948049030 con 0x7ff9380039b0 2024-04-25T09:53:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.581+0000 7ff9538ac640 1 -- 172.21.15.19:0/2037085297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff94c0c9d20 con 0x7ff9380039b0 2024-04-25T09:53:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.581+0000 7ff941ffb640 1 -- 172.21.15.19:0/2037085297 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff94c0cf320 con 0x7ff9380039b0 2024-04-25T09:53:50.583 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 -- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff93006b670 msgr2=0x7ff93006db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff93006b670 0x7ff93006db30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff94c1502e0 tx=0x7ff93c008040 comp rx=0 tx=0).stop 2024-04-25T09:53:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 -- 172.21.15.19:0/2037085297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 msgr2=0x7ff94c151740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 0x7ff94c151740 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff94800c4c0 tx=0x7ff94c161e20 comp rx=0 tx=0).stop 2024-04-25T09:53:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 -- 172.21.15.19:0/2037085297 shutdown_connections 2024-04-25T09:53:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff93006b670 0x7ff93006db30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c160220 0x7ff94c1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c152f00 0x7ff94c0cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 --2- 172.21.15.19:0/2037085297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9380039b0 0x7ff94c151740 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 -- 172.21.15.19:0/2037085297 >> 172.21.15.19:0/2037085297 conn(0x7ff94c0d0530 msgr2=0x7ff94c15d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 -- 172.21.15.19:0/2037085297 shutdown_connections 2024-04-25T09:53:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.583+0000 7ff9538ac640 1 -- 172.21.15.19:0/2037085297 wait complete. 2024-04-25T09:53:50.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.697+0000 7f0f0d56a640 1 Processor -- start 2024-04-25T09:53:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.697+0000 7f0f0d56a640 1 -- start start 2024-04-25T09:53:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f0d56a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f0815b980 0x7f0f080cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f0d56a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f080cf320 con 0x7f0f0815b5a0 2024-04-25T09:53:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f0d56a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f080cf040 con 0x7f0f080b5c40 2024-04-25T09:53:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f0d56a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f08161970 con 0x7f0f0815b980 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f06ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0f0815b5a0 0x7f0f080b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35036/0 (socket says 172.21.15.19:35036) 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f06ffd640 1 -- 172.21.15.19:0/2257685076 learned_addr learned my addr 172.21.15.19:0/2257685076 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f077fe640 1 --2- 172.21.15.19:0/2257685076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f0815b980 0x7f0f080cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f05ffb640 1 -- 172.21.15.19:0/2257685076 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276729812 0 0) 0x7f0ef8001110 con 0x7f0f0815b5a0 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f05ffb640 1 -- 172.21.15.19:0/2257685076 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ef0003520 con 0x7f0f0815b5a0 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f05ffb640 1 -- 172.21.15.19:0/2257685076 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389775829 0 0) 0x7f0f080cf040 con 0x7f0f080b5c40 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f05ffb640 1 -- 172.21.15.19:0/2257685076 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ef8001110 con 0x7f0f080b5c40 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f05ffb640 1 -- 172.21.15.19:0/2257685076 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3697722602 0 0) 0x7f0ef0003520 con 0x7f0f0815b5a0 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f077fe640 1 -- 172.21.15.19:0/2257685076 >> v1:172.21.15.112:6789/0 conn(0x7f0f080b5c40 legacy=0x7f0f080ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f077fe640 1 -- 172.21.15.19:0/2257685076 >> v1:172.21.15.19:6789/0 conn(0x7f0f0815b5a0 legacy=0x7f0f080b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.698+0000 7f0f077fe640 1 -- 172.21.15.19:0/2257685076 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f08161c50 con 0x7f0f0815b980 2024-04-25T09:53:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.699+0000 7f0f077fe640 1 --2- 172.21.15.19:0/2257685076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f0815b980 0x7f0f080cea20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009dc0 tx=0x7f0f0002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ec6ac6dcfb1906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.699+0000 7f0f05ffb640 1 -- 172.21.15.19:0/2257685076 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8001840 con 0x7f0f0815b980 2024-04-25T09:53:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.699+0000 7f0f05ffb640 1 -- 172.21.15.19:0/2257685076 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f00007540 con 0x7f0f0815b980 2024-04-25T09:53:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.699+0000 7f0f05ffb640 1 -- 172.21.15.19:0/2257685076 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f000078f0 con 0x7f0f0815b980 2024-04-25T09:53:50.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.700+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2257685076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f0815b980 msgr2=0x7f0f080cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.700+0000 7f0f0d56a640 1 --2- 172.21.15.19:0/2257685076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f0815b980 0x7f0f080cea20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009dc0 tx=0x7f0f0002a660 comp rx=0 tx=0).stop 2024-04-25T09:53:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.700+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2257685076 shutdown_connections 2024-04-25T09:53:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.700+0000 7f0f0d56a640 1 --2- 172.21.15.19:0/2257685076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f0815b980 0x7f0f080cea20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.700+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2257685076 >> 172.21.15.19:0/2257685076 conn(0x7f0f080d0530 msgr2=0x7f0f080d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.700+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2257685076 shutdown_connections 2024-04-25T09:53:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.700+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2257685076 wait complete. 2024-04-25T09:53:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.701+0000 7f0f0d56a640 1 Processor -- start 2024-04-25T09:53:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.701+0000 7f0f0d56a640 1 -- start start 2024-04-25T09:53:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f0d56a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f080b5c40 0x7f0f080c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f0d56a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f0815b5a0 0x7f0f0815f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f0d56a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f0815b980 0x7f0f080cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f0d56a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f081625d0 con 0x7f0f0815b980 2024-04-25T09:53:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f0d56a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f08162450 con 0x7f0f080b5c40 2024-04-25T09:53:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f0d56a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f08162750 con 0x7f0f0815b5a0 2024-04-25T09:53:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f0815b980 0x7f0f080cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f0815b980 0x7f0f080cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59040/0 (socket says 172.21.15.19:59040) 2024-04-25T09:53:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f077fe640 1 -- 172.21.15.19:0/2001064446 learned_addr learned my addr 172.21.15.19:0/2001064446 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f06ffd640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f080b5c40 0x7f0f080c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f067fc640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f0815b5a0 0x7f0f0815f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f077fe640 1 -- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f0815b5a0 msgr2=0x7f0f0815f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f077fe640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f0815b5a0 0x7f0f0815f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f077fe640 1 -- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f080b5c40 msgr2=0x7f0f080c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f077fe640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f080b5c40 0x7f0f080c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f077fe640 1 -- 172.21.15.19:0/2001064446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f081f7540 con 0x7f0f0815b980 2024-04-25T09:53:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.702+0000 7f0f06ffd640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f080b5c40 0x7f0f080c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.703+0000 7f0f077fe640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f0815b980 0x7f0f080cd360 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009ee0 tx=0x7f0f00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.703+0000 7f0eef7fe640 1 -- 172.21.15.19:0/2001064446 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0003f070 con 0x7f0f0815b980 2024-04-25T09:53:50.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.703+0000 7f0eef7fe640 1 -- 172.21.15.19:0/2001064446 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f00031c40 con 0x7f0f0815b980 2024-04-25T09:53:50.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.703+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2001064446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f081f7820 con 0x7f0f0815b980 2024-04-25T09:53:50.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.703+0000 7f0eef7fe640 1 -- 172.21.15.19:0/2001064446 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00009450 con 0x7f0f0815b980 2024-04-25T09:53:50.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.703+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2001064446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f081f7de0 con 0x7f0f0815b980 2024-04-25T09:53:50.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.704+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2001064446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f080cf870 con 0x7f0f0815b980 2024-04-25T09:53:50.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.704+0000 7f0eef7fe640 1 -- 172.21.15.19:0/2001064446 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f00043410 con 0x7f0f0815b980 2024-04-25T09:53:50.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.705+0000 7f0eef7fe640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ef406b4d0 0x7f0ef406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.705+0000 7f0eef7fe640 1 -- 172.21.15.19:0/2001064446 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0f000ab880 con 0x7f0f0815b980 2024-04-25T09:53:50.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.705+0000 7f0f06ffd640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ef406b4d0 0x7f0ef406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.705+0000 7f0f06ffd640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ef406b4d0 0x7f0ef406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8001eb0 tx=0x7f0ef8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.709+0000 7f0eef7fe640 1 -- 172.21.15.19:0/2001064446 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f080cf870 con 0x7f0f0815b980 2024-04-25T09:53:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.887+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2001064446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f08073a00 con 0x7f0f0815b980 2024-04-25T09:53:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.887+0000 7f0eef7fe640 1 -- 172.21.15.19:0/2001064446 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f0007a2a0 con 0x7f0f0815b980 2024-04-25T09:53:51.019 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ef406b4d0 msgr2=0x7f0ef406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ef406b4d0 0x7f0ef406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8001eb0 tx=0x7f0ef8009040 comp rx=0 tx=0).stop 2024-04-25T09:53:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2001064446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f0815b980 msgr2=0x7f0f080cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f0815b980 0x7f0f080cd360 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009ee0 tx=0x7f0f00004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2001064446 shutdown_connections 2024-04-25T09:53:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ef406b4d0 0x7f0ef406d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f0815b980 0x7f0f080cd360 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f0815b5a0 0x7f0f0815f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 --2- 172.21.15.19:0/2001064446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f080b5c40 0x7f0f080c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2001064446 >> 172.21.15.19:0/2001064446 conn(0x7f0f080d0530 msgr2=0x7f0f080d25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2001064446 shutdown_connections 2024-04-25T09:53:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:50.889+0000 7f0f0d56a640 1 -- 172.21.15.19:0/2001064446 wait complete. 2024-04-25T09:53:51.021 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:51.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.124+0000 7fec7216a640 1 Processor -- start 2024-04-25T09:53:51.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.124+0000 7fec7216a640 1 -- start start 2024-04-25T09:53:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec7216a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec6c15de80 0x7fec6c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec7216a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fec6c0cefb0 0x7fec6c0cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec7216a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fec6c0cf950 0x7fec6c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec7216a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6c0afc80 con 0x7fec6c0cf950 2024-04-25T09:53:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec7216a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6c0ce4d0 con 0x7fec6c0cefb0 2024-04-25T09:53:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec7216a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0ce7b0 con 0x7fec6c15de80 2024-04-25T09:53:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec6bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fec6c0cf950 0x7fec6c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec6bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fec6c0cf950 0x7fec6c06abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59048/0 (socket says 172.21.15.19:59048) 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec6bfff640 1 -- 172.21.15.19:0/1149075468 learned_addr learned my addr 172.21.15.19:0/1149075468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec6b7fe640 1 --2- 172.21.15.19:0/1149075468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec6c15de80 0x7fec6c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec6bfff640 1 -- 172.21.15.19:0/1149075468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec6c15de80 msgr2=0x7fec6c0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec6bfff640 1 --2- 172.21.15.19:0/1149075468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec6c15de80 0x7fec6c0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec6affd640 1 --2- 172.21.15.19:0/1149075468 >> v2:172.21.15.112:3300/0 conn(0x7fec6c0cefb0 0x7fec6c0cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec6bfff640 1 -- 172.21.15.19:0/1149075468 >> v2:172.21.15.112:3300/0 conn(0x7fec6c0cefb0 msgr2=0x7fec6c0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec6bfff640 1 --2- 172.21.15.19:0/1149075468 >> v2:172.21.15.112:3300/0 conn(0x7fec6c0cefb0 0x7fec6c0cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec6bfff640 1 -- 172.21.15.19:0/1149075468 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0cea90 con 0x7fec6c0cf950 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.125+0000 7fec6affd640 1 --2- 172.21.15.19:0/1149075468 >> v2:172.21.15.112:3300/0 conn(0x7fec6c0cefb0 0x7fec6c0cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.126+0000 7fec6bfff640 1 --2- 172.21.15.19:0/1149075468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c0cf950 0x7fec6c06abe0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0d1860 tx=0x7fec6402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1c7ba5898531a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.126+0000 7fec6a7fc640 1 -- 172.21.15.19:0/1149075468 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64036070 con 0x7fec6c0cf950 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.126+0000 7fec6a7fc640 1 -- 172.21.15.19:0/1149075468 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec64031600 con 0x7fec6c0cf950 2024-04-25T09:53:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.126+0000 7fec6a7fc640 1 -- 172.21.15.19:0/1149075468 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec640319d0 con 0x7fec6c0cf950 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.127+0000 7fec7216a640 1 -- 172.21.15.19:0/1149075468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c0cf950 msgr2=0x7fec6c06abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.127+0000 7fec7216a640 1 --2- 172.21.15.19:0/1149075468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c0cf950 0x7fec6c06abe0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0d1860 tx=0x7fec6402a6a0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.127+0000 7fec7216a640 1 -- 172.21.15.19:0/1149075468 shutdown_connections 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.127+0000 7fec7216a640 1 --2- 172.21.15.19:0/1149075468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c0cf950 0x7fec6c06abe0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.127+0000 7fec7216a640 1 --2- 172.21.15.19:0/1149075468 >> v2:172.21.15.112:3300/0 conn(0x7fec6c0cefb0 0x7fec6c0cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.127+0000 7fec7216a640 1 --2- 172.21.15.19:0/1149075468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec6c15de80 0x7fec6c0cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.127+0000 7fec7216a640 1 -- 172.21.15.19:0/1149075468 >> 172.21.15.19:0/1149075468 conn(0x7fec6c0b2390 msgr2=0x7fec6c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.127+0000 7fec7216a640 1 -- 172.21.15.19:0/1149075468 shutdown_connections 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.127+0000 7fec7216a640 1 -- 172.21.15.19:0/1149075468 wait complete. 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.128+0000 7fec7216a640 1 Processor -- start 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.128+0000 7fec7216a640 1 -- start start 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec7216a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c0cefb0 0x7fec6c154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec7216a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c0cf950 0x7fec6c0c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec7216a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec6c15de80 0x7fec6c15c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec7216a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0b3030 con 0x7fec6c0cf950 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec7216a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0b2eb0 con 0x7fec6c0cefb0 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec7216a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0c5600 con 0x7fec6c15de80 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec6affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c0cf950 0x7fec6c0c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec6affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c0cf950 0x7fec6c0c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59050/0 (socket says 172.21.15.19:59050) 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec6affd640 1 -- 172.21.15.19:0/4204198808 learned_addr learned my addr 172.21.15.19:0/4204198808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec6b7fe640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c0cefb0 0x7fec6c154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec6bfff640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec6c15de80 0x7fec6c15c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec6affd640 1 -- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec6c15de80 msgr2=0x7fec6c15c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec6affd640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec6c15de80 0x7fec6c15c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec6affd640 1 -- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c0cefb0 msgr2=0x7fec6c154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec6affd640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c0cefb0 0x7fec6c154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec6affd640 1 -- 172.21.15.19:0/4204198808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0c90a0 con 0x7fec6c0cf950 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.129+0000 7fec6affd640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c0cf950 0x7fec6c0c8260 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c7c0 tx=0x7fec6000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.130+0000 7fec53fff640 1 -- 172.21.15.19:0/4204198808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60015070 con 0x7fec6c0cf950 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.130+0000 7fec53fff640 1 -- 172.21.15.19:0/4204198808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec600079d0 con 0x7fec6c0cf950 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.130+0000 7fec53fff640 1 -- 172.21.15.19:0/4204198808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60007da0 con 0x7fec6c0cf950 2024-04-25T09:53:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.130+0000 7fec7216a640 1 -- 172.21.15.19:0/4204198808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6c0c9380 con 0x7fec6c0cf950 2024-04-25T09:53:51.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.130+0000 7fec7216a640 1 -- 172.21.15.19:0/4204198808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c0c9940 con 0x7fec6c0cf950 2024-04-25T09:53:51.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.131+0000 7fec7216a640 1 -- 172.21.15.19:0/4204198808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6c073a00 con 0x7fec6c0cf950 2024-04-25T09:53:51.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.131+0000 7fec53fff640 1 -- 172.21.15.19:0/4204198808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec60018070 con 0x7fec6c0cf950 2024-04-25T09:53:51.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.132+0000 7fec53fff640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec5806b500 0x7fec5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.132+0000 7fec53fff640 1 -- 172.21.15.19:0/4204198808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fec6008a3a0 con 0x7fec6c0cf950 2024-04-25T09:53:51.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.132+0000 7fec6b7fe640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec5806b500 0x7fec5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.132+0000 7fec6b7fe640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec5806b500 0x7fec5806d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00aa20 tx=0x7fec5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.136+0000 7fec53fff640 1 -- 172.21.15.19:0/4204198808 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec60058860 con 0x7fec6c0cf950 2024-04-25T09:53:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.313+0000 7fec7216a640 1 -- 172.21.15.19:0/4204198808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fec6c06fa40 con 0x7fec6c0cf950 2024-04-25T09:53:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.314+0000 7fec53fff640 1 -- 172.21.15.19:0/4204198808 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fec6005c900 con 0x7fec6c0cf950 2024-04-25T09:53:51.316 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.315+0000 7fec7216a640 1 -- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec5806b500 msgr2=0x7fec5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.315+0000 7fec7216a640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec5806b500 0x7fec5806d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00aa20 tx=0x7fec5c008040 comp rx=0 tx=0).stop 2024-04-25T09:53:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.315+0000 7fec7216a640 1 -- 172.21.15.19:0/4204198808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c0cf950 msgr2=0x7fec6c0c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.315+0000 7fec7216a640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c0cf950 0x7fec6c0c8260 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c7c0 tx=0x7fec6000cc80 comp rx=0 tx=0).stop 2024-04-25T09:53:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.315+0000 7fec7216a640 1 -- 172.21.15.19:0/4204198808 shutdown_connections 2024-04-25T09:53:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.315+0000 7fec7216a640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec5806b500 0x7fec5806d9c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.315+0000 7fec7216a640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec6c15de80 0x7fec6c15c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.315+0000 7fec7216a640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec6c0cf950 0x7fec6c0c8260 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.315+0000 7fec7216a640 1 --2- 172.21.15.19:0/4204198808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c0cefb0 0x7fec6c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.315+0000 7fec7216a640 1 -- 172.21.15.19:0/4204198808 >> 172.21.15.19:0/4204198808 conn(0x7fec6c0b2390 msgr2=0x7fec6c0b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.316+0000 7fec7216a640 1 -- 172.21.15.19:0/4204198808 shutdown_connections 2024-04-25T09:53:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.316+0000 7fec7216a640 1 -- 172.21.15.19:0/4204198808 wait complete. 2024-04-25T09:53:51.329 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:53:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.428+0000 7f56723f5640 1 Processor -- start 2024-04-25T09:53:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.428+0000 7f56723f5640 1 -- start start 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f56723f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f566c15da90 0x7f566c161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f56723f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f566c0b9750 0x7f566c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f56723f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f566c161ba0 0x7f566c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f56723f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566c162600 con 0x7f566c0b9750 2024-04-25T09:53:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f56723f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566c162040 con 0x7f566c161ba0 2024-04-25T09:53:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f56723f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566c162320 con 0x7f566c15da90 2024-04-25T09:53:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f566b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f566c0b9750 0x7f566c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f566b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f566c0b9750 0x7f566c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59062/0 (socket says 172.21.15.19:59062) 2024-04-25T09:53:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f566b7fe640 1 -- 172.21.15.19:0/145580565 learned_addr learned my addr 172.21.15.19:0/145580565 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f5670992640 1 --2- 172.21.15.19:0/145580565 >> v2:172.21.15.112:3300/0 conn(0x7f566c161ba0 0x7f566c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f566bfff640 1 --2- 172.21.15.19:0/145580565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f566c15da90 0x7f566c161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f566b7fe640 1 -- 172.21.15.19:0/145580565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f566c15da90 msgr2=0x7f566c161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f566b7fe640 1 --2- 172.21.15.19:0/145580565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f566c15da90 0x7f566c161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f566b7fe640 1 -- 172.21.15.19:0/145580565 >> v2:172.21.15.112:3300/0 conn(0x7f566c161ba0 msgr2=0x7f566c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f566b7fe640 1 --2- 172.21.15.19:0/145580565 >> v2:172.21.15.112:3300/0 conn(0x7f566c161ba0 0x7f566c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f566b7fe640 1 -- 172.21.15.19:0/145580565 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c0d1b10 con 0x7f566c0b9750 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f5670992640 1 --2- 172.21.15.19:0/145580565 >> v2:172.21.15.112:3300/0 conn(0x7f566c161ba0 0x7f566c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.429+0000 7f566bfff640 1 --2- 172.21.15.19:0/145580565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f566c15da90 0x7f566c161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:51.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.430+0000 7f566b7fe640 1 --2- 172.21.15.19:0/145580565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c0b9750 0x7f566c0ce670 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f566c0d0b50 tx=0x7f565c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaf66c2b84c4a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.430+0000 7f566affd640 1 -- 172.21.15.19:0/145580565 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c037070 con 0x7f566c0b9750 2024-04-25T09:53:51.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.430+0000 7f566affd640 1 -- 172.21.15.19:0/145580565 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c032440 con 0x7f566c0b9750 2024-04-25T09:53:51.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.430+0000 7f566affd640 1 -- 172.21.15.19:0/145580565 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c032810 con 0x7f566c0b9750 2024-04-25T09:53:51.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.431+0000 7f56723f5640 1 -- 172.21.15.19:0/145580565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c0b9750 msgr2=0x7f566c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.431+0000 7f56723f5640 1 --2- 172.21.15.19:0/145580565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c0b9750 0x7f566c0ce670 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f566c0d0b50 tx=0x7f565c02a140 comp rx=0 tx=0).stop 2024-04-25T09:53:51.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.431+0000 7f56723f5640 1 -- 172.21.15.19:0/145580565 shutdown_connections 2024-04-25T09:53:51.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.431+0000 7f56723f5640 1 --2- 172.21.15.19:0/145580565 >> v2:172.21.15.112:3300/0 conn(0x7f566c161ba0 0x7f566c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.431+0000 7f56723f5640 1 --2- 172.21.15.19:0/145580565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c0b9750 0x7f566c0ce670 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.431+0000 7f56723f5640 1 --2- 172.21.15.19:0/145580565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f566c15da90 0x7f566c161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.431+0000 7f56723f5640 1 -- 172.21.15.19:0/145580565 >> 172.21.15.19:0/145580565 conn(0x7f566c061a20 msgr2=0x7f566c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.431+0000 7f56723f5640 1 -- 172.21.15.19:0/145580565 shutdown_connections 2024-04-25T09:53:51.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.431+0000 7f56723f5640 1 -- 172.21.15.19:0/145580565 wait complete. 2024-04-25T09:53:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.432+0000 7f56723f5640 1 Processor -- start 2024-04-25T09:53:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.432+0000 7f56723f5640 1 -- start start 2024-04-25T09:53:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f56723f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f566c15da90 0x7f566c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f56723f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c161ba0 0x7f566c15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f56723f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f566c1f3910 0x7f566c0caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f56723f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c06c120 con 0x7f566c161ba0 2024-04-25T09:53:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f56723f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c15de70 con 0x7f566c1f3910 2024-04-25T09:53:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f56723f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566c06c2a0 con 0x7f566c15da90 2024-04-25T09:53:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c161ba0 0x7f566c15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c161ba0 0x7f566c15f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59076/0 (socket says 172.21.15.19:59076) 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f566b7fe640 1 -- 172.21.15.19:0/4087059733 learned_addr learned my addr 172.21.15.19:0/4087059733 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f566bfff640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f566c15da90 0x7f566c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f5670992640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f566c1f3910 0x7f566c0caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f566b7fe640 1 -- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f566c15da90 msgr2=0x7f566c15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f566b7fe640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f566c15da90 0x7f566c15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f566b7fe640 1 -- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f566c1f3910 msgr2=0x7f566c0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f566b7fe640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f566c1f3910 0x7f566c0caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f566b7fe640 1 -- 172.21.15.19:0/4087059733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c15ace0 con 0x7f566c161ba0 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.433+0000 7f5670992640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f566c1f3910 0x7f566c0caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.434+0000 7f566b7fe640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c161ba0 0x7f566c15f4d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f565c000c00 tx=0x7f565c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.434+0000 7f5668ff9640 1 -- 172.21.15.19:0/4087059733 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c041020 con 0x7f566c161ba0 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.434+0000 7f5668ff9640 1 -- 172.21.15.19:0/4087059733 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c0306c0 con 0x7f566c161ba0 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.434+0000 7f56723f5640 1 -- 172.21.15.19:0/4087059733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566c15fe40 con 0x7f566c161ba0 2024-04-25T09:53:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.434+0000 7f5668ff9640 1 -- 172.21.15.19:0/4087059733 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c037040 con 0x7f566c161ba0 2024-04-25T09:53:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.434+0000 7f56723f5640 1 -- 172.21.15.19:0/4087059733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566c0c8710 con 0x7f566c161ba0 2024-04-25T09:53:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.435+0000 7f56723f5640 1 -- 172.21.15.19:0/4087059733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566c0c8520 con 0x7f566c161ba0 2024-04-25T09:53:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.435+0000 7f5668ff9640 1 -- 172.21.15.19:0/4087059733 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f565c044020 con 0x7f566c161ba0 2024-04-25T09:53:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.436+0000 7f5668ff9640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f565806b4d0 0x7f565806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.436+0000 7f5668ff9640 1 -- 172.21.15.19:0/4087059733 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f565c0aada0 con 0x7f566c161ba0 2024-04-25T09:53:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.436+0000 7f566bfff640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f565806b4d0 0x7f565806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.437+0000 7f566bfff640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f565806b4d0 0x7f565806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f566c15cfb0 tx=0x7f5660008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.440+0000 7f5668ff9640 1 -- 172.21.15.19:0/4087059733 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f565c04b030 con 0x7f566c161ba0 2024-04-25T09:53:51.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.632+0000 7f56723f5640 1 -- 172.21.15.19:0/4087059733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f566c0cfdf0 con 0x7f566c161ba0 2024-04-25T09:53:51.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.633+0000 7f5668ff9640 1 -- 172.21.15.19:0/4087059733 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f565c0798b0 con 0x7f566c161ba0 2024-04-25T09:53:51.635 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T09:53:51.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.634+0000 7f56723f5640 1 -- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f565806b4d0 msgr2=0x7f565806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.634+0000 7f56723f5640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f565806b4d0 0x7f565806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f566c15cfb0 tx=0x7f5660008040 comp rx=0 tx=0).stop 2024-04-25T09:53:51.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.634+0000 7f56723f5640 1 -- 172.21.15.19:0/4087059733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c161ba0 msgr2=0x7f566c15f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.634+0000 7f56723f5640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c161ba0 0x7f566c15f4d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f565c000c00 tx=0x7f565c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.634+0000 7f56723f5640 1 -- 172.21.15.19:0/4087059733 shutdown_connections 2024-04-25T09:53:51.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.634+0000 7f56723f5640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f565806b4d0 0x7f565806d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.634+0000 7f56723f5640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f566c1f3910 0x7f566c0caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.634+0000 7f56723f5640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f566c161ba0 0x7f566c15f4d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.634+0000 7f56723f5640 1 --2- 172.21.15.19:0/4087059733 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f566c15da90 0x7f566c15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.634+0000 7f56723f5640 1 -- 172.21.15.19:0/4087059733 >> 172.21.15.19:0/4087059733 conn(0x7f566c061a20 msgr2=0x7f566c15bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.635+0000 7f56723f5640 1 -- 172.21.15.19:0/4087059733 shutdown_connections 2024-04-25T09:53:51.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.635+0000 7f56723f5640 1 -- 172.21.15.19:0/4087059733 wait complete. 2024-04-25T09:53:51.647 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:53:51.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:51.648 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T09:53:51.674 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:53:51.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:51.675 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:53:51.730 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:53:51.730 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:51.749 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:53:51.749 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:51.808 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:53:51.809 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:51.866 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.965+0000 7fb69b02a640 1 Processor -- start 2024-04-25T09:53:51.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.966+0000 7fb69b02a640 1 -- start start 2024-04-25T09:53:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.966+0000 7fb69b02a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb69415dc70 0x7fb6940d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.966+0000 7fb69b02a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb69409a450 0x7fb6940cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.966+0000 7fb69b02a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb69406c700 0x7fb69406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.966+0000 7fb69b02a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6940d0c90 con 0x7fb69409a450 2024-04-25T09:53:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.966+0000 7fb69b02a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69406c220 con 0x7fb69415dc70 2024-04-25T09:53:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.966+0000 7fb69b02a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6940d0080 con 0x7fb69406c700 2024-04-25T09:53:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.966+0000 7fb693fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb69409a450 0x7fb6940cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.967+0000 7fb693fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb69409a450 0x7fb6940cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59080/0 (socket says 172.21.15.19:59080) 2024-04-25T09:53:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.967+0000 7fb693fff640 1 -- 172.21.15.19:0/3452822366 learned_addr learned my addr 172.21.15.19:0/3452822366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:51.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.967+0000 7fb6995c7640 1 --2- 172.21.15.19:0/3452822366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb69406c700 0x7fb69406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.967+0000 7fb698dc6640 1 --2- 172.21.15.19:0/3452822366 >> v2:172.21.15.112:3300/0 conn(0x7fb69415dc70 0x7fb6940d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.967+0000 7fb693fff640 1 -- 172.21.15.19:0/3452822366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb69406c700 msgr2=0x7fb69406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.967+0000 7fb693fff640 1 --2- 172.21.15.19:0/3452822366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb69406c700 0x7fb69406cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.967+0000 7fb693fff640 1 -- 172.21.15.19:0/3452822366 >> v2:172.21.15.112:3300/0 conn(0x7fb69415dc70 msgr2=0x7fb6940d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.967+0000 7fb693fff640 1 --2- 172.21.15.19:0/3452822366 >> v2:172.21.15.112:3300/0 conn(0x7fb69415dc70 0x7fb6940d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.967+0000 7fb693fff640 1 -- 172.21.15.19:0/3452822366 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6940d0360 con 0x7fb69409a450 2024-04-25T09:53:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.967+0000 7fb6995c7640 1 --2- 172.21.15.19:0/3452822366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb69406c700 0x7fb69406cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.967+0000 7fb693fff640 1 --2- 172.21.15.19:0/3452822366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb69409a450 0x7fb6940cf9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6940726a0 tx=0x7fb68802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdad6c6532d1be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb6937fe640 1 -- 172.21.15.19:0/3452822366 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688037070 con 0x7fb69409a450 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb6937fe640 1 -- 172.21.15.19:0/3452822366 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb688032440 con 0x7fb69409a450 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb6937fe640 1 -- 172.21.15.19:0/3452822366 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688032810 con 0x7fb69409a450 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb69b02a640 1 -- 172.21.15.19:0/3452822366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb69409a450 msgr2=0x7fb6940cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb69b02a640 1 --2- 172.21.15.19:0/3452822366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb69409a450 0x7fb6940cf9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6940726a0 tx=0x7fb68802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb69b02a640 1 -- 172.21.15.19:0/3452822366 shutdown_connections 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb69b02a640 1 --2- 172.21.15.19:0/3452822366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb69406c700 0x7fb69406cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb69b02a640 1 --2- 172.21.15.19:0/3452822366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb69409a450 0x7fb6940cf9c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb69b02a640 1 --2- 172.21.15.19:0/3452822366 >> v2:172.21.15.112:3300/0 conn(0x7fb69415dc70 0x7fb6940d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb69b02a640 1 -- 172.21.15.19:0/3452822366 >> 172.21.15.19:0/3452822366 conn(0x7fb6940b3270 msgr2=0x7fb6940b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb69b02a640 1 -- 172.21.15.19:0/3452822366 shutdown_connections 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.968+0000 7fb69b02a640 1 -- 172.21.15.19:0/3452822366 wait complete. 2024-04-25T09:53:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.969+0000 7fb69b02a640 1 Processor -- start 2024-04-25T09:53:51.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.969+0000 7fb69b02a640 1 -- start start 2024-04-25T09:53:51.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.970+0000 7fb69b02a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb69406c700 0x7fb6940c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.970+0000 7fb69b02a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb69409a450 0x7fb6940b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.970+0000 7fb69b02a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb69415dc70 0x7fb6940b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.970+0000 7fb69b02a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69415e050 con 0x7fb69409a450 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.970+0000 7fb69b02a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6940d0920 con 0x7fb69406c700 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.970+0000 7fb69b02a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6940cbdd0 con 0x7fb69415dc70 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.970+0000 7fb693fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb69409a450 0x7fb6940b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.970+0000 7fb693fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb69409a450 0x7fb6940b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59086/0 (socket says 172.21.15.19:59086) 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.970+0000 7fb693fff640 1 -- 172.21.15.19:0/2168028857 learned_addr learned my addr 172.21.15.19:0/2168028857 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.971+0000 7fb693fff640 1 -- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb69415dc70 msgr2=0x7fb6940b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.971+0000 7fb693fff640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb69415dc70 0x7fb6940b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.971+0000 7fb693fff640 1 -- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb69406c700 msgr2=0x7fb6940c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.971+0000 7fb693fff640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb69406c700 0x7fb6940c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.971+0000 7fb693fff640 1 -- 172.21.15.19:0/2168028857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6940b6ba0 con 0x7fb69409a450 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.971+0000 7fb693fff640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb69409a450 0x7fb6940b4370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb6880097c0 tx=0x7fb6880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.971+0000 7fb6917fa640 1 -- 172.21.15.19:0/2168028857 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688037040 con 0x7fb69409a450 2024-04-25T09:53:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.971+0000 7fb6917fa640 1 -- 172.21.15.19:0/2168028857 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb688042070 con 0x7fb69409a450 2024-04-25T09:53:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.971+0000 7fb69b02a640 1 -- 172.21.15.19:0/2168028857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6940b6e80 con 0x7fb69409a450 2024-04-25T09:53:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.971+0000 7fb6917fa640 1 -- 172.21.15.19:0/2168028857 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688030bb0 con 0x7fb69409a450 2024-04-25T09:53:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.971+0000 7fb69b02a640 1 -- 172.21.15.19:0/2168028857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6940b6540 con 0x7fb69409a450 2024-04-25T09:53:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.972+0000 7fb69b02a640 1 -- 172.21.15.19:0/2168028857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6940c80a0 con 0x7fb69409a450 2024-04-25T09:53:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.972+0000 7fb6917fa640 1 -- 172.21.15.19:0/2168028857 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb688047460 con 0x7fb69409a450 2024-04-25T09:53:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.973+0000 7fb6917fa640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb68006b4d0 0x7fb68006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.973+0000 7fb6917fa640 1 -- 172.21.15.19:0/2168028857 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb6880ab2c0 con 0x7fb69409a450 2024-04-25T09:53:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.973+0000 7fb698dc6640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb68006b4d0 0x7fb68006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.974+0000 7fb698dc6640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb68006b4d0 0x7fb68006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb68c007dc0 tx=0x7fb68c0072f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:51.977+0000 7fb6917fa640 1 -- 172.21.15.19:0/2168028857 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb688045360 con 0x7fb69409a450 2024-04-25T09:53:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.160+0000 7fb69b02a640 1 -- 172.21.15.19:0/2168028857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb694073990 con 0x7fb69409a450 2024-04-25T09:53:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.161+0000 7fb6917fa640 1 -- 172.21.15.19:0/2168028857 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18892 (secure 0 0 0) 0x7fb688079d40 con 0x7fb69409a450 2024-04-25T09:53:52.163 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:52.164 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:48.071585+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.163+0000 7fb69b02a640 1 -- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb68006b4d0 msgr2=0x7fb68006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.163+0000 7fb69b02a640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb68006b4d0 0x7fb68006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb68c007dc0 tx=0x7fb68c0072f0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.163+0000 7fb69b02a640 1 -- 172.21.15.19:0/2168028857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb69409a450 msgr2=0x7fb6940b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.163+0000 7fb69b02a640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb69409a450 0x7fb6940b4370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb6880097c0 tx=0x7fb6880046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.163+0000 7fb69b02a640 1 -- 172.21.15.19:0/2168028857 shutdown_connections 2024-04-25T09:53:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.163+0000 7fb69b02a640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb68006b4d0 0x7fb68006d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.163+0000 7fb69b02a640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb69415dc70 0x7fb6940b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.163+0000 7fb69b02a640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb69409a450 0x7fb6940b4370 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.163+0000 7fb69b02a640 1 --2- 172.21.15.19:0/2168028857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb69406c700 0x7fb6940c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.163+0000 7fb69b02a640 1 -- 172.21.15.19:0/2168028857 >> 172.21.15.19:0/2168028857 conn(0x7fb6940b3270 msgr2=0x7fb6940caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.164+0000 7fb69b02a640 1 -- 172.21.15.19:0/2168028857 shutdown_connections 2024-04-25T09:53:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.164+0000 7fb69b02a640 1 -- 172.21.15.19:0/2168028857 wait complete. 2024-04-25T09:53:52.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:52.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:52.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.279+0000 7fd537fbe640 1 Processor -- start 2024-04-25T09:53:52.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.279+0000 7fd537fbe640 1 -- start start 2024-04-25T09:53:52.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd537fbe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd53015dc20 0x7fd5300ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd537fbe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd530161710 0x7fd5300cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd537fbe640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53015e000 con 0x7fd5300b9330 2024-04-25T09:53:52.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd537fbe640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5300d1de0 con 0x7fd530161710 2024-04-25T09:53:52.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd537fbe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd530161c60 con 0x7fd53015dc20 2024-04-25T09:53:52.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd535559640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd530161710 0x7fd5300cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd53655b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd5300b9330 0x7fd53006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35040/0 (socket says 172.21.15.19:35040) 2024-04-25T09:53:52.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd53655b640 1 -- 172.21.15.19:0/3377975677 learned_addr learned my addr 172.21.15.19:0/3377975677 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:52.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd535d5a640 1 --2- 172.21.15.19:0/3377975677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd53015dc20 0x7fd5300ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd535559640 1 -- 172.21.15.19:0/3377975677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd53015dc20 msgr2=0x7fd5300ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:52.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd535559640 1 --2- 172.21.15.19:0/3377975677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd53015dc20 0x7fd5300ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd535559640 1 -- 172.21.15.19:0/3377975677 >> v1:172.21.15.19:6789/0 conn(0x7fd5300b9330 legacy=0x7fd53006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd535559640 1 -- 172.21.15.19:0/3377975677 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd530161f40 con 0x7fd530161710 2024-04-25T09:53:52.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.280+0000 7fd534d58640 1 -- 172.21.15.19:0/3377975677 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248842186 0 0) 0x7fd53015e000 con 0x7fd5300b9330 2024-04-25T09:53:52.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.281+0000 7fd535d5a640 1 --2- 172.21.15.19:0/3377975677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd53015dc20 0x7fd5300ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:52.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.281+0000 7fd535559640 1 --2- 172.21.15.19:0/3377975677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd530161710 0x7fd5300cebb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd5200097a0 tx=0x7fd52002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a54088cbcab0cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.281+0000 7fd534d58640 1 -- 172.21.15.19:0/3377975677 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520037070 con 0x7fd530161710 2024-04-25T09:53:52.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.281+0000 7fd534d58640 1 -- 172.21.15.19:0/3377975677 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd520032440 con 0x7fd530161710 2024-04-25T09:53:52.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.281+0000 7fd534d58640 1 -- 172.21.15.19:0/3377975677 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520032810 con 0x7fd530161710 2024-04-25T09:53:52.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.282+0000 7fd537fbe640 1 -- 172.21.15.19:0/3377975677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd530161710 msgr2=0x7fd5300cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.282+0000 7fd537fbe640 1 --2- 172.21.15.19:0/3377975677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd530161710 0x7fd5300cebb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd5200097a0 tx=0x7fd52002a180 comp rx=0 tx=0).stop 2024-04-25T09:53:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.282+0000 7fd537fbe640 1 -- 172.21.15.19:0/3377975677 shutdown_connections 2024-04-25T09:53:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.282+0000 7fd537fbe640 1 --2- 172.21.15.19:0/3377975677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd530161710 0x7fd5300cebb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.282+0000 7fd537fbe640 1 --2- 172.21.15.19:0/3377975677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd53015dc20 0x7fd5300ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.282+0000 7fd537fbe640 1 -- 172.21.15.19:0/3377975677 >> 172.21.15.19:0/3377975677 conn(0x7fd5300d0420 msgr2=0x7fd5300d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.282+0000 7fd537fbe640 1 -- 172.21.15.19:0/3377975677 shutdown_connections 2024-04-25T09:53:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.282+0000 7fd537fbe640 1 -- 172.21.15.19:0/3377975677 wait complete. 2024-04-25T09:53:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.283+0000 7fd537fbe640 1 Processor -- start 2024-04-25T09:53:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.283+0000 7fd537fbe640 1 -- start start 2024-04-25T09:53:52.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd537fbe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5300b9330 0x7fd5301551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd537fbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53015dc20 0x7fd530158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd537fbe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd530161710 0x7fd530156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd537fbe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300b5e50 con 0x7fd5300b9330 2024-04-25T09:53:52.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd537fbe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd530162500 con 0x7fd53015dc20 2024-04-25T09:53:52.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd537fbe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300b5fd0 con 0x7fd530161710 2024-04-25T09:53:52.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd535d5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5300b9330 0x7fd5301551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd535d5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5300b9330 0x7fd5301551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59098/0 (socket says 172.21.15.19:59098) 2024-04-25T09:53:52.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd535d5a640 1 -- 172.21.15.19:0/3167349294 learned_addr learned my addr 172.21.15.19:0/3167349294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:52.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd53655b640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd530161710 0x7fd530156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd535559640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53015dc20 0x7fd530158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd535d5a640 1 -- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd530161710 msgr2=0x7fd530156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd535d5a640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd530161710 0x7fd530156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd535d5a640 1 -- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53015dc20 msgr2=0x7fd530158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd535d5a640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53015dc20 0x7fd530158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd535d5a640 1 -- 172.21.15.19:0/3167349294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53015ec30 con 0x7fd5300b9330 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.284+0000 7fd535559640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53015dc20 0x7fd530158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.285+0000 7fd535d5a640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5300b9330 0x7fd5301551e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00d7d0 tx=0x7fd52c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.285+0000 7fd5267fc640 1 -- 172.21.15.19:0/3167349294 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c004030 con 0x7fd5300b9330 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.285+0000 7fd5267fc640 1 -- 172.21.15.19:0/3167349294 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c0041d0 con 0x7fd5300b9330 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.285+0000 7fd537fbe640 1 -- 172.21.15.19:0/3167349294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53015ef10 con 0x7fd5300b9330 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.285+0000 7fd5267fc640 1 -- 172.21.15.19:0/3167349294 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c009640 con 0x7fd5300b9330 2024-04-25T09:53:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.285+0000 7fd537fbe640 1 -- 172.21.15.19:0/3167349294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53015f4d0 con 0x7fd5300b9330 2024-04-25T09:53:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.286+0000 7fd537fbe640 1 -- 172.21.15.19:0/3167349294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd530073a00 con 0x7fd5300b9330 2024-04-25T09:53:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.286+0000 7fd5267fc640 1 -- 172.21.15.19:0/3167349294 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd52c0097e0 con 0x7fd5300b9330 2024-04-25T09:53:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.286+0000 7fd5267fc640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 0x7fd51406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.287+0000 7fd5267fc640 1 -- 172.21.15.19:0/3167349294 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd52c08a830 con 0x7fd5300b9330 2024-04-25T09:53:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.287+0000 7fd535559640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 0x7fd51406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.287+0000 7fd535559640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 0x7fd51406d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd520034000 tx=0x7fd52002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.291+0000 7fd5267fc640 1 -- 172.21.15.19:0/3167349294 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd52c0592b0 con 0x7fd5300b9330 2024-04-25T09:53:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.471+0000 7fd537fbe640 1 -- 172.21.15.19:0/3167349294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5300d16b0 con 0x7fd5300b9330 2024-04-25T09:53:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.471+0000 7fd5267fc640 1 -- 172.21.15.19:0/3167349294 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd52c058bb0 con 0x7fd5300b9330 2024-04-25T09:53:52.473 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.473+0000 7fd537fbe640 1 -- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 msgr2=0x7fd51406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.473+0000 7fd537fbe640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 0x7fd51406d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd520034000 tx=0x7fd52002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.474+0000 7fd537fbe640 1 -- 172.21.15.19:0/3167349294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5300b9330 msgr2=0x7fd5301551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.474+0000 7fd537fbe640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5300b9330 0x7fd5301551e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00d7d0 tx=0x7fd52c00dc90 comp rx=0 tx=0).stop 2024-04-25T09:53:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.474+0000 7fd537fbe640 1 -- 172.21.15.19:0/3167349294 shutdown_connections 2024-04-25T09:53:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.474+0000 7fd537fbe640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 0x7fd51406d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.474+0000 7fd537fbe640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd530161710 0x7fd530156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.474+0000 7fd537fbe640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd53015dc20 0x7fd530158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.474+0000 7fd537fbe640 1 --2- 172.21.15.19:0/3167349294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5300b9330 0x7fd5301551e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.474+0000 7fd537fbe640 1 -- 172.21.15.19:0/3167349294 >> 172.21.15.19:0/3167349294 conn(0x7fd5300d0420 msgr2=0x7fd5300d2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.474+0000 7fd537fbe640 1 -- 172.21.15.19:0/3167349294 shutdown_connections 2024-04-25T09:53:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.474+0000 7fd537fbe640 1 -- 172.21.15.19:0/3167349294 wait complete. 2024-04-25T09:53:52.487 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:52.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.588+0000 7f8c1f612640 1 Processor -- start 2024-04-25T09:53:52.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.588+0000 7f8c1f612640 1 -- start start 2024-04-25T09:53:52.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.589+0000 7f8c1f612640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8c1815dc20 0x7f8c180ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.589+0000 7f8c1f612640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8c180d32f0 0x7f8c180cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.589+0000 7f8c1f612640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1815e000 con 0x7f8c18161710 2024-04-25T09:53:52.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.589+0000 7f8c1f612640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c180d1de0 con 0x7f8c1815dc20 2024-04-25T09:53:52.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.589+0000 7f8c1f612640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c18161c60 con 0x7f8c180d32f0 2024-04-25T09:53:52.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.589+0000 7f8c1d3ae640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8c1815dc20 0x7f8c180ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.589+0000 7f8c1d3ae640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8c1815dc20 0x7f8c180ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59114/0 (socket says 172.21.15.19:59114) 2024-04-25T09:53:52.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.589+0000 7f8c1d3ae640 1 -- 172.21.15.19:0/3236723382 learned_addr learned my addr 172.21.15.19:0/3236723382 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:52.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.590+0000 7f8c1d3ae640 1 -- 172.21.15.19:0/3236723382 >> v2:172.21.15.112:3300/0 conn(0x7f8c180d32f0 msgr2=0x7f8c180cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:52.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.590+0000 7f8c1d3ae640 1 --2- 172.21.15.19:0/3236723382 >> v2:172.21.15.112:3300/0 conn(0x7f8c180d32f0 0x7f8c180cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.590+0000 7f8c1d3ae640 1 -- 172.21.15.19:0/3236723382 >> v1:172.21.15.112:6789/0 conn(0x7f8c18161710 legacy=0x7f8c1806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:52.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.590+0000 7f8c1d3ae640 1 -- 172.21.15.19:0/3236723382 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c18161f40 con 0x7f8c1815dc20 2024-04-25T09:53:52.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.590+0000 7f8c1d3ae640 1 --2- 172.21.15.19:0/3236723382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c1815dc20 0x7f8c180ce670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8c180b6910 tx=0x7f8c1002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7008b6f758274535 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.590+0000 7f8c17fff640 1 -- 172.21.15.19:0/3236723382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10037070 con 0x7f8c1815dc20 2024-04-25T09:53:52.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.591+0000 7f8c17fff640 1 -- 172.21.15.19:0/3236723382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c10032440 con 0x7f8c1815dc20 2024-04-25T09:53:52.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.591+0000 7f8c17fff640 1 -- 172.21.15.19:0/3236723382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10032810 con 0x7f8c1815dc20 2024-04-25T09:53:52.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.591+0000 7f8c1f612640 1 -- 172.21.15.19:0/3236723382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c1815dc20 msgr2=0x7f8c180ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.591+0000 7f8c1f612640 1 --2- 172.21.15.19:0/3236723382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c1815dc20 0x7f8c180ce670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8c180b6910 tx=0x7f8c1002a230 comp rx=0 tx=0).stop 2024-04-25T09:53:52.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.591+0000 7f8c1f612640 1 -- 172.21.15.19:0/3236723382 shutdown_connections 2024-04-25T09:53:52.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.591+0000 7f8c1f612640 1 --2- 172.21.15.19:0/3236723382 >> v2:172.21.15.112:3300/0 conn(0x7f8c180d32f0 0x7f8c180cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.591+0000 7f8c1f612640 1 --2- 172.21.15.19:0/3236723382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c1815dc20 0x7f8c180ce670 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.591+0000 7f8c1f612640 1 -- 172.21.15.19:0/3236723382 >> 172.21.15.19:0/3236723382 conn(0x7f8c180d0420 msgr2=0x7f8c180d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.591+0000 7f8c1f612640 1 -- 172.21.15.19:0/3236723382 shutdown_connections 2024-04-25T09:53:52.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.591+0000 7f8c1f612640 1 -- 172.21.15.19:0/3236723382 wait complete. 2024-04-25T09:53:52.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.592+0000 7f8c1f612640 1 Processor -- start 2024-04-25T09:53:52.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.592+0000 7f8c1f612640 1 -- start start 2024-04-25T09:53:52.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.593+0000 7f8c1f612640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c180d32f0 0x7f8c180c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.593+0000 7f8c1f612640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c1815dc20 0x7f8c18160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.593+0000 7f8c1f612640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c18161710 0x7f8c1815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.593+0000 7f8c1f612640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180b5e50 con 0x7f8c18161710 2024-04-25T09:53:52.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.593+0000 7f8c1f612640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18162500 con 0x7f8c180d32f0 2024-04-25T09:53:52.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.593+0000 7f8c1f612640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180b5fd0 con 0x7f8c1815dc20 2024-04-25T09:53:52.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.593+0000 7f8c1dbaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c18161710 0x7f8c1815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.593+0000 7f8c1dbaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c18161710 0x7f8c1815eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59116/0 (socket says 172.21.15.19:59116) 2024-04-25T09:53:52.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.593+0000 7f8c1dbaf640 1 -- 172.21.15.19:0/1862866983 learned_addr learned my addr 172.21.15.19:0/1862866983 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:52.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.593+0000 7f8c1cbad640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c1815dc20 0x7f8c18160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c1d3ae640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c180d32f0 0x7f8c180c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c1dbaf640 1 -- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c1815dc20 msgr2=0x7f8c18160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c1dbaf640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c1815dc20 0x7f8c18160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c1dbaf640 1 -- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c180d32f0 msgr2=0x7f8c180c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c1dbaf640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c180d32f0 0x7f8c180c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c1dbaf640 1 -- 172.21.15.19:0/1862866983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1815f970 con 0x7f8c18161710 2024-04-25T09:53:52.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c1cbad640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c1815dc20 0x7f8c18160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:52.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c1d3ae640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c180d32f0 0x7f8c180c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:52.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c1dbaf640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c18161710 0x7f8c1815eed0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800ba10 tx=0x7f8c0800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c15ffb640 1 -- 172.21.15.19:0/1862866983 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08019070 con 0x7f8c18161710 2024-04-25T09:53:52.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c15ffb640 1 -- 172.21.15.19:0/1862866983 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0800caa0 con 0x7f8c18161710 2024-04-25T09:53:52.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c1f612640 1 -- 172.21.15.19:0/1862866983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1815fc50 con 0x7f8c18161710 2024-04-25T09:53:52.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c15ffb640 1 -- 172.21.15.19:0/1862866983 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08014440 con 0x7f8c18161710 2024-04-25T09:53:52.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.594+0000 7f8c1f612640 1 -- 172.21.15.19:0/1862866983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c18160210 con 0x7f8c18161710 2024-04-25T09:53:52.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.595+0000 7f8c1f612640 1 -- 172.21.15.19:0/1862866983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c18073a00 con 0x7f8c18161710 2024-04-25T09:53:52.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.595+0000 7f8c15ffb640 1 -- 172.21.15.19:0/1862866983 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c0801c070 con 0x7f8c18161710 2024-04-25T09:53:52.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.596+0000 7f8c15ffb640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c0406b4d0 0x7f8c0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.596+0000 7f8c15ffb640 1 -- 172.21.15.19:0/1862866983 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8c0808a2c0 con 0x7f8c18161710 2024-04-25T09:53:52.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.596+0000 7f8c1d3ae640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c0406b4d0 0x7f8c0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.597+0000 7f8c1d3ae640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c0406b4d0 0x7f8c0406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c10034000 tx=0x7f8c1002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.600+0000 7f8c15ffb640 1 -- 172.21.15.19:0/1862866983 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c08058cb0 con 0x7f8c18161710 2024-04-25T09:53:52.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.779+0000 7f8c1f612640 1 -- 172.21.15.19:0/1862866983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c180d16b0 con 0x7f8c18161710 2024-04-25T09:53:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.779+0000 7f8c15ffb640 1 -- 172.21.15.19:0/1862866983 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c080585b0 con 0x7f8c18161710 2024-04-25T09:53:52.782 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:52.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 -- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c0406b4d0 msgr2=0x7f8c0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c0406b4d0 0x7f8c0406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c10034000 tx=0x7f8c1002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 -- 172.21.15.19:0/1862866983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c18161710 msgr2=0x7f8c1815eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c18161710 0x7f8c1815eed0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800ba10 tx=0x7f8c0800bed0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 -- 172.21.15.19:0/1862866983 shutdown_connections 2024-04-25T09:53:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c0406b4d0 0x7f8c0406d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c18161710 0x7f8c1815eed0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c1815dc20 0x7f8c18160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 --2- 172.21.15.19:0/1862866983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c180d32f0 0x7f8c180c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 -- 172.21.15.19:0/1862866983 >> 172.21.15.19:0/1862866983 conn(0x7f8c180d0420 msgr2=0x7f8c180b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 -- 172.21.15.19:0/1862866983 shutdown_connections 2024-04-25T09:53:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.781+0000 7f8c1f612640 1 -- 172.21.15.19:0/1862866983 wait complete. 2024-04-25T09:53:52.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:52.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.895+0000 7f1bdbf5c640 1 Processor -- start 2024-04-25T09:53:52.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.896+0000 7f1bdbf5c640 1 -- start start 2024-04-25T09:53:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.896+0000 7f1bdbf5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bd41616f0 0x7f1bd4161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.896+0000 7f1bdbf5c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1bd40ce0a0 0x7f1bd40ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.896+0000 7f1bdbf5c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd406c8d0 con 0x7f1bd41610f0 2024-04-25T09:53:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.896+0000 7f1bdbf5c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd41614d0 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.896+0000 7f1bdbf5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd40c6960 con 0x7f1bd41616f0 2024-04-25T09:53:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.896+0000 7f1bda4f9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1bd40ce0a0 0x7f1bd40ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.897+0000 7f1bda4f9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1bd40ce0a0 0x7f1bd40ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59126/0 (socket says 172.21.15.19:59126) 2024-04-25T09:53:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.897+0000 7f1bda4f9640 1 -- 172.21.15.19:0/311349142 learned_addr learned my addr 172.21.15.19:0/311349142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.897+0000 7f1bd94f7640 1 --2- 172.21.15.19:0/311349142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bd41616f0 0x7f1bd4161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.897+0000 7f1bda4f9640 1 -- 172.21.15.19:0/311349142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bd41616f0 msgr2=0x7f1bd4161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.897+0000 7f1bda4f9640 1 --2- 172.21.15.19:0/311349142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bd41616f0 0x7f1bd4161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.897+0000 7f1bda4f9640 1 -- 172.21.15.19:0/311349142 >> v1:172.21.15.112:6789/0 conn(0x7f1bd41610f0 legacy=0x7f1bd406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.897+0000 7f1bda4f9640 1 -- 172.21.15.19:0/311349142 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd4162260 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bda4f9640 1 --2- 172.21.15.19:0/311349142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd40ce0a0 0x7f1bd40ce530 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009e50 tx=0x7f1bc402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7926b0695176bf83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bd8cf6640 1 -- 172.21.15.19:0/311349142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4036070 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bd8cf6640 1 -- 172.21.15.19:0/311349142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc4031600 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bd8cf6640 1 -- 172.21.15.19:0/311349142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc40319d0 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/311349142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd40ce0a0 msgr2=0x7f1bd40ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bdbf5c640 1 --2- 172.21.15.19:0/311349142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd40ce0a0 0x7f1bd40ce530 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009e50 tx=0x7f1bc402a710 comp rx=0 tx=0).stop 2024-04-25T09:53:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/311349142 shutdown_connections 2024-04-25T09:53:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bdbf5c640 1 --2- 172.21.15.19:0/311349142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd40ce0a0 0x7f1bd40ce530 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bdbf5c640 1 --2- 172.21.15.19:0/311349142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bd41616f0 0x7f1bd4161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/311349142 >> 172.21.15.19:0/311349142 conn(0x7f1bd406f500 msgr2=0x7f1bd406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/311349142 shutdown_connections 2024-04-25T09:53:52.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.898+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/311349142 wait complete. 2024-04-25T09:53:52.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.900+0000 7f1bdbf5c640 1 Processor -- start 2024-04-25T09:53:52.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.900+0000 7f1bdbf5c640 1 -- start start 2024-04-25T09:53:52.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.900+0000 7f1bdbf5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd40ce0a0 0x7f1bd40b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.900+0000 7f1bdbf5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bd41610f0 0x7f1bd4145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.900+0000 7f1bdbf5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bd41616f0 0x7f1bd4158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.900+0000 7f1bdbf5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd40cfce0 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.900+0000 7f1bdbf5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd40cfa00 con 0x7f1bd41610f0 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.900+0000 7f1bdbf5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd40cf690 con 0x7f1bd41616f0 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd9cf8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd40ce0a0 0x7f1bd40b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd9cf8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd40ce0a0 0x7f1bd40b4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59134/0 (socket says 172.21.15.19:59134) 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd9cf8640 1 -- 172.21.15.19:0/2148453959 learned_addr learned my addr 172.21.15.19:0/2148453959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd94f7640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bd41610f0 0x7f1bd4145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bda4f9640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bd41616f0 0x7f1bd4158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd9cf8640 1 -- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bd41616f0 msgr2=0x7f1bd4158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd9cf8640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bd41616f0 0x7f1bd4158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd9cf8640 1 -- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bd41610f0 msgr2=0x7f1bd4145000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd9cf8640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bd41610f0 0x7f1bd4145000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd9cf8640 1 -- 172.21.15.19:0/2148453959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd4145540 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd94f7640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bd41610f0 0x7f1bd4145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bda4f9640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bd41616f0 0x7f1bd4158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd9cf8640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd40ce0a0 0x7f1bd40b4520 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00c400 tx=0x7f1bcc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd27fc640 1 -- 172.21.15.19:0/2148453959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0052d0 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd27fc640 1 -- 172.21.15.19:0/2148453959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc005490 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/2148453959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd40b4b40 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bd27fc640 1 -- 172.21.15.19:0/2148453959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc007260 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.901+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/2148453959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd40b7820 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.902+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/2148453959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd4073a00 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.903+0000 7f1bd27fc640 1 -- 172.21.15.19:0/2148453959 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bcc007400 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.903+0000 7f1bd27fc640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1bc006b4d0 0x7f1bc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.903+0000 7f1bd27fc640 1 -- 172.21.15.19:0/2148453959 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1bcc089ac0 con 0x7f1bd40ce0a0 2024-04-25T09:53:52.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.903+0000 7f1bd94f7640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1bc006b4d0 0x7f1bc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.904+0000 7f1bd94f7640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1bc006b4d0 0x7f1bc006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1bd41f6dd0 tx=0x7f1bc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:52.907+0000 7f1bd27fc640 1 -- 172.21.15.19:0/2148453959 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1bcc0585d0 con 0x7f1bd40ce0a0 2024-04-25T09:53:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.087+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/2148453959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1bd40cf150 con 0x7f1bd40ce0a0 2024-04-25T09:53:53.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.087+0000 7f1bd27fc640 1 -- 172.21.15.19:0/2148453959 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1bcc057ed0 con 0x7f1bd40ce0a0 2024-04-25T09:53:53.090 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:53.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1bc006b4d0 msgr2=0x7f1bc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1bc006b4d0 0x7f1bc006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1bd41f6dd0 tx=0x7f1bc8008040 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/2148453959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd40ce0a0 msgr2=0x7f1bd40b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd40ce0a0 0x7f1bd40b4520 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00c400 tx=0x7f1bcc00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/2148453959 shutdown_connections 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1bc006b4d0 0x7f1bc006d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bd41616f0 0x7f1bd4158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bd41610f0 0x7f1bd4145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 --2- 172.21.15.19:0/2148453959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bd40ce0a0 0x7f1bd40b4520 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/2148453959 >> 172.21.15.19:0/2148453959 conn(0x7f1bd406f500 msgr2=0x7f1bd41566e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/2148453959 shutdown_connections 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.089+0000 7f1bdbf5c640 1 -- 172.21.15.19:0/2148453959 wait complete. 2024-04-25T09:53:53.103 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.204+0000 7fa99b3b0640 1 Processor -- start 2024-04-25T09:53:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.204+0000 7fa99b3b0640 1 -- start start 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.205+0000 7fa99b3b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9940ced80 0x7fa9940d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.205+0000 7fa99b3b0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa994161710 0x7fa9940d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.205+0000 7fa99b3b0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99415e000 con 0x7fa99415dc20 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.205+0000 7fa99b3b0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9940d1de0 con 0x7fa994161710 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.205+0000 7fa99b3b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa994161c60 con 0x7fa9940ced80 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.205+0000 7fa99994d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa994161710 0x7fa9940d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.205+0000 7fa99994d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa994161710 0x7fa9940d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59138/0 (socket says 172.21.15.19:59138) 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.205+0000 7fa99994d640 1 -- 172.21.15.19:0/3973733969 learned_addr learned my addr 172.21.15.19:0/3973733969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.205+0000 7fa99894b640 1 --2- 172.21.15.19:0/3973733969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9940ced80 0x7fa9940d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.206+0000 7fa99994d640 1 -- 172.21.15.19:0/3973733969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9940ced80 msgr2=0x7fa9940d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.206+0000 7fa99994d640 1 --2- 172.21.15.19:0/3973733969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9940ced80 0x7fa9940d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.206+0000 7fa99994d640 1 -- 172.21.15.19:0/3973733969 >> v1:172.21.15.112:6789/0 conn(0x7fa99415dc20 legacy=0x7fa9940ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.206+0000 7fa99994d640 1 -- 172.21.15.19:0/3973733969 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa994161f40 con 0x7fa994161710 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.206+0000 7fa99894b640 1 --2- 172.21.15.19:0/3973733969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9940ced80 0x7fa9940d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.206+0000 7fa99994d640 1 --2- 172.21.15.19:0/3973733969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa994161710 0x7fa9940d2820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009870 tx=0x7fa98c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d153046afd456f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.206+0000 7fa993fff640 1 -- 172.21.15.19:0/3973733969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c037070 con 0x7fa994161710 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.206+0000 7fa993fff640 1 -- 172.21.15.19:0/3973733969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c032440 con 0x7fa994161710 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.206+0000 7fa993fff640 1 -- 172.21.15.19:0/3973733969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c032810 con 0x7fa994161710 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.207+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3973733969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa994161710 msgr2=0x7fa9940d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.207+0000 7fa99b3b0640 1 --2- 172.21.15.19:0/3973733969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa994161710 0x7fa9940d2820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009870 tx=0x7fa98c02a210 comp rx=0 tx=0).stop 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.207+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3973733969 shutdown_connections 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.207+0000 7fa99b3b0640 1 --2- 172.21.15.19:0/3973733969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa994161710 0x7fa9940d2820 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.207+0000 7fa99b3b0640 1 --2- 172.21.15.19:0/3973733969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9940ced80 0x7fa9940d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.207+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3973733969 >> 172.21.15.19:0/3973733969 conn(0x7fa9940d0420 msgr2=0x7fa9940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.207+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3973733969 shutdown_connections 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.207+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3973733969 wait complete. 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.208+0000 7fa99b3b0640 1 Processor -- start 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.208+0000 7fa99b3b0640 1 -- start start 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99b3b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9940ced80 0x7fa99414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99b3b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99415dc20 0x7fa9940aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99b3b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa994161710 0x7fa9941f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99b3b0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940b5e50 con 0x7fa9940ced80 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99b3b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa994162500 con 0x7fa99415dc20 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99b3b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940b5fd0 con 0x7fa994161710 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99914c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9940ced80 0x7fa99414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99914c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9940ced80 0x7fa99414fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59150/0 (socket says 172.21.15.19:59150) 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99914c640 1 -- 172.21.15.19:0/3595651978 learned_addr learned my addr 172.21.15.19:0/3595651978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99894b640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99415dc20 0x7fa9940aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99994d640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa994161710 0x7fa9941f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99914c640 1 -- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa994161710 msgr2=0x7fa9941f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99914c640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa994161710 0x7fa9941f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99914c640 1 -- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99415dc20 msgr2=0x7fa9940aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99914c640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99415dc20 0x7fa9940aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.209+0000 7fa99914c640 1 -- 172.21.15.19:0/3595651978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940cb450 con 0x7fa9940ced80 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.210+0000 7fa99914c640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9940ced80 0x7fa99414fe70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa98400ba50 tx=0x7fa98400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.210+0000 7fa991ffb640 1 -- 172.21.15.19:0/3595651978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984019070 con 0x7fa9940ced80 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.210+0000 7fa991ffb640 1 -- 172.21.15.19:0/3595651978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98400cc20 con 0x7fa9940ced80 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.210+0000 7fa991ffb640 1 -- 172.21.15.19:0/3595651978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984014680 con 0x7fa9940ced80 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.210+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3595651978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9940cb730 con 0x7fa9940ced80 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.210+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3595651978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9940aa8c0 con 0x7fa9940ced80 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.211+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3595651978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa994073a00 con 0x7fa9940ced80 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.211+0000 7fa991ffb640 1 -- 172.21.15.19:0/3595651978 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa98401c070 con 0x7fa9940ced80 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.212+0000 7fa991ffb640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa98006b4d0 0x7fa98006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.212+0000 7fa991ffb640 1 -- 172.21.15.19:0/3595651978 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa98408a2d0 con 0x7fa9940ced80 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.212+0000 7fa99894b640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa98006b4d0 0x7fa98006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.213+0000 7fa99894b640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa98006b4d0 0x7fa98006d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa9940d4510 tx=0x7fa988008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.216+0000 7fa991ffb640 1 -- 172.21.15.19:0/3595651978 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa984058700 con 0x7fa9940ced80 2024-04-25T09:53:53.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.396+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3595651978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa9940d16b0 con 0x7fa9940ced80 2024-04-25T09:53:53.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.397+0000 7fa991ffb640 1 -- 172.21.15.19:0/3595651978 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa98408a0f0 con 0x7fa9940ced80 2024-04-25T09:53:53.399 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:53.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa98006b4d0 msgr2=0x7fa98006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa98006b4d0 0x7fa98006d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa9940d4510 tx=0x7fa988008040 comp rx=0 tx=0).stop 2024-04-25T09:53:53.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3595651978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9940ced80 msgr2=0x7fa99414fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9940ced80 0x7fa99414fe70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa98400ba50 tx=0x7fa98400bf10 comp rx=0 tx=0).stop 2024-04-25T09:53:53.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3595651978 shutdown_connections 2024-04-25T09:53:53.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa98006b4d0 0x7fa98006d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa994161710 0x7fa9941f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa99415dc20 0x7fa9940aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 --2- 172.21.15.19:0/3595651978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9940ced80 0x7fa99414fe70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3595651978 >> 172.21.15.19:0/3595651978 conn(0x7fa9940d0420 msgr2=0x7fa9941601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3595651978 shutdown_connections 2024-04-25T09:53:53.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.399+0000 7fa99b3b0640 1 -- 172.21.15.19:0/3595651978 wait complete. 2024-04-25T09:53:53.412 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:53:53.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.512+0000 7f5225891640 1 Processor -- start 2024-04-25T09:53:53.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.512+0000 7f5225891640 1 -- start start 2024-04-25T09:53:53.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.512+0000 7f5225891640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f522015dc20 0x7f52200ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f5225891640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5220161710 0x7f52200cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f5225891640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522015e000 con 0x7f52200b9330 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f5225891640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52200d1de0 con 0x7f522015dc20 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f5225891640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5220161c60 con 0x7f5220161710 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f5217fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5220161710 0x7f52200cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f521effd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f522015dc20 0x7f52200ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f521f7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f52200b9330 0x7f522006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:38540/0 (socket says 172.21.15.19:38540) 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f5217fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5220161710 0x7f52200cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:44612/0 (socket says 172.21.15.19:44612) 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f521effd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f522015dc20 0x7f52200ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:43882/0 (socket says 172.21.15.19:43882) 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f521f7fe640 1 -- 172.21.15.19:0/127536413 learned_addr learned my addr 172.21.15.19:0/127536413 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f521effd640 1 -- 172.21.15.19:0/127536413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5220161710 msgr2=0x7f52200cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f521effd640 1 --2- 172.21.15.19:0/127536413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5220161710 0x7f52200cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f521effd640 1 -- 172.21.15.19:0/127536413 >> v1:172.21.15.112:6789/0 conn(0x7f52200b9330 legacy=0x7f522006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f521effd640 1 -- 172.21.15.19:0/127536413 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5220161f40 con 0x7f522015dc20 2024-04-25T09:53:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.513+0000 7f5217fff640 1 --2- 172.21.15.19:0/127536413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5220161710 0x7f52200cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:53.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.514+0000 7f521effd640 1 --2- 172.21.15.19:0/127536413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f522015dc20 0x7f52200ce670 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f52100097a0 tx=0x7f521002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a0c427dcdc9314 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.514+0000 7f521e7fc640 1 -- 172.21.15.19:0/127536413 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5210037070 con 0x7f522015dc20 2024-04-25T09:53:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.514+0000 7f521e7fc640 1 -- 172.21.15.19:0/127536413 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5210032440 con 0x7f522015dc20 2024-04-25T09:53:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.514+0000 7f521e7fc640 1 -- 172.21.15.19:0/127536413 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5210032810 con 0x7f522015dc20 2024-04-25T09:53:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.515+0000 7f5225891640 1 -- 172.21.15.19:0/127536413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f522015dc20 msgr2=0x7f52200ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.515+0000 7f5225891640 1 --2- 172.21.15.19:0/127536413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f522015dc20 0x7f52200ce670 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f52100097a0 tx=0x7f521002a180 comp rx=0 tx=0).stop 2024-04-25T09:53:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.515+0000 7f5225891640 1 -- 172.21.15.19:0/127536413 shutdown_connections 2024-04-25T09:53:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.515+0000 7f5225891640 1 --2- 172.21.15.19:0/127536413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5220161710 0x7f52200cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.515+0000 7f5225891640 1 --2- 172.21.15.19:0/127536413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f522015dc20 0x7f52200ce670 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.515+0000 7f5225891640 1 -- 172.21.15.19:0/127536413 >> 172.21.15.19:0/127536413 conn(0x7f52200d0420 msgr2=0x7f52200d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.515+0000 7f5225891640 1 -- 172.21.15.19:0/127536413 shutdown_connections 2024-04-25T09:53:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.515+0000 7f5225891640 1 -- 172.21.15.19:0/127536413 wait complete. 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.516+0000 7f5225891640 1 Processor -- start 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.516+0000 7f5225891640 1 -- start start 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5225891640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52200b9330 0x7f52201f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5225891640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f522015dc20 0x7f52200ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5225891640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5220161710 0x7f52200ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5225891640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52200b5e50 con 0x7f522015dc20 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5225891640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5220162500 con 0x7f5220161710 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5225891640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52200b5fd0 con 0x7f52200b9330 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5217fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f522015dc20 0x7f52200ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5217fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f522015dc20 0x7f52200ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59164/0 (socket says 172.21.15.19:59164) 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5217fff640 1 -- 172.21.15.19:0/3778779405 learned_addr learned my addr 172.21.15.19:0/3778779405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f521effd640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52200b9330 0x7f52201f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f521f7fe640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5220161710 0x7f52200ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5217fff640 1 -- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52200b9330 msgr2=0x7f52201f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5217fff640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52200b9330 0x7f52201f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5217fff640 1 -- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5220161710 msgr2=0x7f52200ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5217fff640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5220161710 0x7f52200ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5217fff640 1 -- 172.21.15.19:0/3778779405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52200cacc0 con 0x7f522015dc20 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.517+0000 7f5217fff640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f522015dc20 0x7f52200ac2d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f520800e960 tx=0x7f520800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.518+0000 7f521cff9640 1 -- 172.21.15.19:0/3778779405 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208017070 con 0x7f522015dc20 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.518+0000 7f521cff9640 1 -- 172.21.15.19:0/3778779405 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520800ca00 con 0x7f522015dc20 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.518+0000 7f5225891640 1 -- 172.21.15.19:0/3778779405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52200cafa0 con 0x7f522015dc20 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.518+0000 7f521cff9640 1 -- 172.21.15.19:0/3778779405 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520800cdd0 con 0x7f522015dc20 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.518+0000 7f5225891640 1 -- 172.21.15.19:0/3778779405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52200cb560 con 0x7f522015dc20 2024-04-25T09:53:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.519+0000 7f5225891640 1 -- 172.21.15.19:0/3778779405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52200caa50 con 0x7f522015dc20 2024-04-25T09:53:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.519+0000 7f521cff9640 1 -- 172.21.15.19:0/3778779405 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f520801a070 con 0x7f522015dc20 2024-04-25T09:53:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.520+0000 7f521cff9640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f520006b4d0 0x7f520006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.520+0000 7f521cff9640 1 -- 172.21.15.19:0/3778779405 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f520808a7c0 con 0x7f522015dc20 2024-04-25T09:53:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.520+0000 7f521effd640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f520006b4d0 0x7f520006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.520+0000 7f521effd640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f520006b4d0 0x7f520006d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5210034000 tx=0x7f521002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.524+0000 7f521cff9640 1 -- 172.21.15.19:0/3778779405 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5208058bf0 con 0x7f522015dc20 2024-04-25T09:53:53.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.715+0000 7f5225891640 1 -- 172.21.15.19:0/3778779405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5220073a00 con 0x7f522015dc20 2024-04-25T09:53:53.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.716+0000 7f521cff9640 1 -- 172.21.15.19:0/3778779405 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f520805cc90 con 0x7f522015dc20 2024-04-25T09:53:53.718 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T09:53:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 -- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f520006b4d0 msgr2=0x7f520006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f520006b4d0 0x7f520006d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5210034000 tx=0x7f521002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 -- 172.21.15.19:0/3778779405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f522015dc20 msgr2=0x7f52200ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f522015dc20 0x7f52200ac2d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f520800e960 tx=0x7f520800ee20 comp rx=0 tx=0).stop 2024-04-25T09:53:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 -- 172.21.15.19:0/3778779405 shutdown_connections 2024-04-25T09:53:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f520006b4d0 0x7f520006d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5220161710 0x7f52200ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f522015dc20 0x7f52200ac2d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 --2- 172.21.15.19:0/3778779405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52200b9330 0x7f52201f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 -- 172.21.15.19:0/3778779405 >> 172.21.15.19:0/3778779405 conn(0x7f52200d0420 msgr2=0x7f52200b8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 -- 172.21.15.19:0/3778779405 shutdown_connections 2024-04-25T09:53:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:53.718+0000 7f5225891640 1 -- 172.21.15.19:0/3778779405 wait complete. 2024-04-25T09:53:53.731 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T09:53:53.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:53.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T09:53:53.761 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T09:53:53.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:53.762 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:53:53.825 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:53:53.825 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:53.841 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:53:53.841 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:53.896 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:53:53.896 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:53.952 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.091+0000 7f4ce6b09640 1 Processor -- start 2024-04-25T09:53:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.091+0000 7f4ce6b09640 1 -- start start 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4ce6b09640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ce00d2e90 0x7f4ce00d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4ce6b09640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce00b9940 con 0x7f4ce015dd30 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4ce6b09640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce00d1df0 con 0x7f4ce00d0900 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4ce6b09640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce00d0d40 con 0x7f4ce00d2e90 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4ce48a5640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4ce015dd30 0x7f4ce00b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35050/0 (socket says 172.21.15.19:35050) 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4ce48a5640 1 -- 172.21.15.19:0/2388656710 learned_addr learned my addr 172.21.15.19:0/2388656710 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4cdffff640 1 --2- 172.21.15.19:0/2388656710 >> v2:172.21.15.112:3300/0 conn(0x7f4ce00d2e90 0x7f4ce00d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530319431 0 0) 0x7f4ce00b9940 con 0x7f4ce015dd30 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cd0003520 con 0x7f4ce015dd30 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4224317077 0 0) 0x7f4cd0003520 con 0x7f4ce015dd30 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 >> v2:172.21.15.112:3300/0 conn(0x7f4ce00d2e90 msgr2=0x7f4ce00d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4cdf7fe640 1 --2- 172.21.15.19:0/2388656710 >> v2:172.21.15.112:3300/0 conn(0x7f4ce00d2e90 0x7f4ce00d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 >> v1:172.21.15.112:6789/0 conn(0x7f4ce00d0900 legacy=0x7f4ce00b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce00d20d0 con 0x7f4ce015dd30 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f4cd4001670 con 0x7f4ce015dd30 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.092+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 >> v1:172.21.15.19:6789/0 conn(0x7f4ce015dd30 legacy=0x7f4ce00b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4cdf7fe640 1 --2- 172.21.15.19:0/2388656710 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cd00039b0 0x7f4cd0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4cdf7fe640 1 --2- 172.21.15.19:0/2388656710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd0004800 0x7f4cd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4cdf7fe640 1 --2- 172.21.15.19:0/2388656710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 0x7f4cd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00d0d40 con 0x7f4cd00051c0 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd000c110 con 0x7f4cd0004800 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd000c310 con 0x7f4cd00039b0 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4ce50a6640 1 --2- 172.21.15.19:0/2388656710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 0x7f4cd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4ce48a5640 1 --2- 172.21.15.19:0/2388656710 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cd00039b0 0x7f4cd0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4cdffff640 1 --2- 172.21.15.19:0/2388656710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd0004800 0x7f4cd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4ce50a6640 1 -- 172.21.15.19:0/2388656710 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cd00039b0 msgr2=0x7f4cd0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4ce50a6640 1 --2- 172.21.15.19:0/2388656710 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cd00039b0 0x7f4cd0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4ce50a6640 1 -- 172.21.15.19:0/2388656710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd0004800 msgr2=0x7f4cd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4ce50a6640 1 --2- 172.21.15.19:0/2388656710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd0004800 0x7f4cd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.093+0000 7f4ce50a6640 1 -- 172.21.15.19:0/2388656710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd0003520 con 0x7f4cd00051c0 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.094+0000 7f4ce50a6640 1 -- 172.21.15.19:0/2388656710 reap_dead start 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.094+0000 7f4ce50a6640 1 --2- 172.21.15.19:0/2388656710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 0x7f4cd000b9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8001f60 tx=0x7f4cd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7421527aa1a9e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.094+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8001b30 con 0x7f4cd00051c0 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.094+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd8003b90 con 0x7f4cd00051c0 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.094+0000 7f4cdf7fe640 1 -- 172.21.15.19:0/2388656710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8005110 con 0x7f4cd00051c0 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.094+0000 7f4ce6b09640 1 -- 172.21.15.19:0/2388656710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 msgr2=0x7f4cd000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.094+0000 7f4ce6b09640 1 --2- 172.21.15.19:0/2388656710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 0x7f4cd000b9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8001f60 tx=0x7f4cd0005830 comp rx=0 tx=0).stop 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.094+0000 7f4ce6b09640 1 -- 172.21.15.19:0/2388656710 shutdown_connections 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.094+0000 7f4ce6b09640 1 --2- 172.21.15.19:0/2388656710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 0x7f4cd000b9c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.094+0000 7f4ce6b09640 1 -- 172.21.15.19:0/2388656710 >> 172.21.15.19:0/2388656710 conn(0x7f4ce0061c30 msgr2=0x7f4ce006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.094+0000 7f4ce6b09640 1 -- 172.21.15.19:0/2388656710 shutdown_connections 2024-04-25T09:53:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.095+0000 7f4ce6b09640 1 -- 172.21.15.19:0/2388656710 wait complete. 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.096+0000 7f4ce6b09640 1 Processor -- start 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.096+0000 7f4ce6b09640 1 -- start start 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.096+0000 7f4ce6b09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 0x7f4ce0149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.096+0000 7f4ce6b09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce0153b80 0x7f4ce0154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.096+0000 7f4ce6b09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce0159090 0x7f4ce0159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.096+0000 7f4ce6b09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00d2970 con 0x7f4cd00051c0 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.096+0000 7f4ce6b09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00d2690 con 0x7f4ce0153b80 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.096+0000 7f4ce6b09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00d1a10 con 0x7f4ce0159090 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce48a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 0x7f4ce0149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce48a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 0x7f4ce0149b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59172/0 (socket says 172.21.15.19:59172) 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce48a5640 1 -- 172.21.15.19:0/3042608152 learned_addr learned my addr 172.21.15.19:0/3042608152 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce50a6640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce0159090 0x7f4ce0159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4cdffff640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce0153b80 0x7f4ce0154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce48a5640 1 -- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce0159090 msgr2=0x7f4ce0159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce48a5640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce0159090 0x7f4ce0159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce48a5640 1 -- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce0153b80 msgr2=0x7f4ce0154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce48a5640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce0153b80 0x7f4ce0154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce48a5640 1 -- 172.21.15.19:0/3042608152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce01546f0 con 0x7f4cd00051c0 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce50a6640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce0159090 0x7f4ce0159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce48a5640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 0x7f4ce0149b80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4004f20 tx=0x7f4cd40074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4cdd7fa640 1 -- 172.21.15.19:0/3042608152 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400f7b0 con 0x7f4cd00051c0 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4cdd7fa640 1 -- 172.21.15.19:0/3042608152 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd4018070 con 0x7f4cd00051c0 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce6b09640 1 -- 172.21.15.19:0/3042608152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce01549d0 con 0x7f4cd00051c0 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4cdd7fa640 1 -- 172.21.15.19:0/3042608152 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400fbd0 con 0x7f4cd00051c0 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.097+0000 7f4ce6b09640 1 -- 172.21.15.19:0/3042608152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce0154f90 con 0x7f4cd00051c0 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.099+0000 7f4ce6b09640 1 -- 172.21.15.19:0/3042608152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce0073990 con 0x7f4cd00051c0 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.099+0000 7f4cdd7fa640 1 -- 172.21.15.19:0/3042608152 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cd401d460 con 0x7f4cd00051c0 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.099+0000 7f4cdd7fa640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cc406b4d0 0x7f4cc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.099+0000 7f4cdd7fa640 1 -- 172.21.15.19:0/3042608152 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4cd408a4e0 con 0x7f4cd00051c0 2024-04-25T09:53:54.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.099+0000 7f4cdffff640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cc406b4d0 0x7f4cc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.100+0000 7f4cdffff640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cc406b4d0 0x7f4cc406d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ce015cce0 tx=0x7f4ccc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.105+0000 7f4cdd7fa640 1 -- 172.21.15.19:0/3042608152 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4cd40590b0 con 0x7f4cd00051c0 2024-04-25T09:53:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.287+0000 7f4ce6b09640 1 -- 172.21.15.19:0/3042608152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ce015c190 con 0x7f4cd00051c0 2024-04-25T09:53:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.288+0000 7f4cdd7fa640 1 -- 172.21.15.19:0/3042608152 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18892 (secure 0 0 0) 0x7f4cd40589b0 con 0x7f4cd00051c0 2024-04-25T09:53:54.290 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:54.290 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:48.071585+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 -- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cc406b4d0 msgr2=0x7f4cc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cc406b4d0 0x7f4cc406d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ce015cce0 tx=0x7f4ccc008040 comp rx=0 tx=0).stop 2024-04-25T09:53:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 -- 172.21.15.19:0/3042608152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 msgr2=0x7f4ce0149b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 0x7f4ce0149b80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4004f20 tx=0x7f4cd40074f0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 -- 172.21.15.19:0/3042608152 shutdown_connections 2024-04-25T09:53:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cc406b4d0 0x7f4cc406d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce0159090 0x7f4ce0159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce0153b80 0x7f4ce0154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 --2- 172.21.15.19:0/3042608152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd00051c0 0x7f4ce0149b80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 -- 172.21.15.19:0/3042608152 >> 172.21.15.19:0/3042608152 conn(0x7f4ce0061c30 msgr2=0x7f4ce015f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 -- 172.21.15.19:0/3042608152 shutdown_connections 2024-04-25T09:53:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.290+0000 7f4ce6b09640 1 -- 172.21.15.19:0/3042608152 wait complete. 2024-04-25T09:53:54.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:54.304 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:54.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.405+0000 7f095873b640 1 Processor -- start 2024-04-25T09:53:54.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.405+0000 7f095873b640 1 -- start start 2024-04-25T09:53:54.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f095873b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f095015b7d0 0x7f09500cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f095873b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09500b5620 0x7f09500b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f095873b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f095006c2d0 0x7f095006c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f095873b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09500b5fc0 con 0x7f095015b7d0 2024-04-25T09:53:54.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f095873b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095006cca0 con 0x7f095006c2d0 2024-04-25T09:53:54.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f095873b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095006cf50 con 0x7f09500b5620 2024-04-25T09:53:54.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f09564d7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f095015b7d0 0x7f09500cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f09564d7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f095015b7d0 0x7f09500cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59184/0 (socket says 172.21.15.19:59184) 2024-04-25T09:53:54.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f09564d7640 1 -- 172.21.15.19:0/2215135671 learned_addr learned my addr 172.21.15.19:0/2215135671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f0955cd6640 1 --2- 172.21.15.19:0/2215135671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09500b5620 0x7f09500b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f0956cd8640 1 --2- 172.21.15.19:0/2215135671 >> v2:172.21.15.112:3300/0 conn(0x7f095006c2d0 0x7f095006c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f09564d7640 1 -- 172.21.15.19:0/2215135671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09500b5620 msgr2=0x7f09500b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f09564d7640 1 --2- 172.21.15.19:0/2215135671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09500b5620 0x7f09500b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f09564d7640 1 -- 172.21.15.19:0/2215135671 >> v2:172.21.15.112:3300/0 conn(0x7f095006c2d0 msgr2=0x7f095006c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f09564d7640 1 --2- 172.21.15.19:0/2215135671 >> v2:172.21.15.112:3300/0 conn(0x7f095006c2d0 0x7f095006c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f09564d7640 1 -- 172.21.15.19:0/2215135671 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500ce530 con 0x7f095015b7d0 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.406+0000 7f0955cd6640 1 --2- 172.21.15.19:0/2215135671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09500b5620 0x7f09500b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.407+0000 7f09564d7640 1 --2- 172.21.15.19:0/2215135671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095015b7d0 0x7f09500cf010 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f095015e340 tx=0x7f094c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c412df3755b10482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.407+0000 7f09554d5640 1 -- 172.21.15.19:0/2215135671 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c037070 con 0x7f095015b7d0 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.407+0000 7f09554d5640 1 -- 172.21.15.19:0/2215135671 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c032440 con 0x7f095015b7d0 2024-04-25T09:53:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.407+0000 7f09554d5640 1 -- 172.21.15.19:0/2215135671 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c032810 con 0x7f095015b7d0 2024-04-25T09:53:54.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.408+0000 7f095873b640 1 -- 172.21.15.19:0/2215135671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095015b7d0 msgr2=0x7f09500cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.408+0000 7f095873b640 1 --2- 172.21.15.19:0/2215135671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095015b7d0 0x7f09500cf010 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f095015e340 tx=0x7f094c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.408+0000 7f095873b640 1 -- 172.21.15.19:0/2215135671 shutdown_connections 2024-04-25T09:53:54.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.408+0000 7f095873b640 1 --2- 172.21.15.19:0/2215135671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09500b5620 0x7f09500b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.408+0000 7f095873b640 1 --2- 172.21.15.19:0/2215135671 >> v2:172.21.15.112:3300/0 conn(0x7f095006c2d0 0x7f095006c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.408+0000 7f095873b640 1 --2- 172.21.15.19:0/2215135671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095015b7d0 0x7f09500cf010 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.408+0000 7f095873b640 1 -- 172.21.15.19:0/2215135671 >> 172.21.15.19:0/2215135671 conn(0x7f09500d0800 msgr2=0x7f09500d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.408+0000 7f095873b640 1 -- 172.21.15.19:0/2215135671 shutdown_connections 2024-04-25T09:53:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.408+0000 7f095873b640 1 -- 172.21.15.19:0/2215135671 wait complete. 2024-04-25T09:53:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.409+0000 7f095873b640 1 Processor -- start 2024-04-25T09:53:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.409+0000 7f095873b640 1 -- start start 2024-04-25T09:53:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f095873b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095006c2d0 0x7f09500b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f095873b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09500b5620 0x7f09500b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f095873b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095015b7d0 0x7f09500b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f095873b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095006d230 con 0x7f095006c2d0 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f095873b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500ceaf0 con 0x7f095015b7d0 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f095873b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09500cf550 con 0x7f09500b5620 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f09564d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095006c2d0 0x7f09500b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f09564d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095006c2d0 0x7f09500b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59188/0 (socket says 172.21.15.19:59188) 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f09564d7640 1 -- 172.21.15.19:0/2319298619 learned_addr learned my addr 172.21.15.19:0/2319298619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f0955cd6640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09500b5620 0x7f09500b79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f0956cd8640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095015b7d0 0x7f09500b7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f09564d7640 1 -- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09500b5620 msgr2=0x7f09500b79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f09564d7640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09500b5620 0x7f09500b79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f09564d7640 1 -- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095015b7d0 msgr2=0x7f09500b7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f09564d7640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095015b7d0 0x7f09500b7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f09564d7640 1 -- 172.21.15.19:0/2319298619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950158860 con 0x7f095006c2d0 2024-04-25T09:53:54.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f0956cd8640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095015b7d0 0x7f09500b7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:54.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f09564d7640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095006c2d0 0x7f09500b2f10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f094c02a220 tx=0x7f094c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f0946ffd640 1 -- 172.21.15.19:0/2319298619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c037040 con 0x7f095006c2d0 2024-04-25T09:53:54.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f0946ffd640 1 -- 172.21.15.19:0/2319298619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c042070 con 0x7f095006c2d0 2024-04-25T09:53:54.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f095873b640 1 -- 172.21.15.19:0/2319298619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09500b8540 con 0x7f095006c2d0 2024-04-25T09:53:54.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f0946ffd640 1 -- 172.21.15.19:0/2319298619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c030bb0 con 0x7f095006c2d0 2024-04-25T09:53:54.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.410+0000 7f095873b640 1 -- 172.21.15.19:0/2319298619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09501491c0 con 0x7f095006c2d0 2024-04-25T09:53:54.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.411+0000 7f095873b640 1 -- 172.21.15.19:0/2319298619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0950148fb0 con 0x7f095006c2d0 2024-04-25T09:53:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.412+0000 7f0946ffd640 1 -- 172.21.15.19:0/2319298619 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f094c047460 con 0x7f095006c2d0 2024-04-25T09:53:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.412+0000 7f0946ffd640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f093406b500 0x7f093406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.412+0000 7f0946ffd640 1 -- 172.21.15.19:0/2319298619 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f094c0ab1f0 con 0x7f095006c2d0 2024-04-25T09:53:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.412+0000 7f0955cd6640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f093406b500 0x7f093406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.413+0000 7f0955cd6640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f093406b500 0x7f093406d9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f095015df20 tx=0x7f0940008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.416+0000 7f0946ffd640 1 -- 172.21.15.19:0/2319298619 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f094c045360 con 0x7f095006c2d0 2024-04-25T09:53:54.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.598+0000 7f095873b640 1 -- 172.21.15.19:0/2319298619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f095015a630 con 0x7f095006c2d0 2024-04-25T09:53:54.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.599+0000 7f0946ffd640 1 -- 172.21.15.19:0/2319298619 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f094c079d00 con 0x7f095006c2d0 2024-04-25T09:53:54.603 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 -- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f093406b500 msgr2=0x7f093406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f093406b500 0x7f093406d9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f095015df20 tx=0x7f0940008040 comp rx=0 tx=0).stop 2024-04-25T09:53:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 -- 172.21.15.19:0/2319298619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095006c2d0 msgr2=0x7f09500b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095006c2d0 0x7f09500b2f10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f094c02a220 tx=0x7f094c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 -- 172.21.15.19:0/2319298619 shutdown_connections 2024-04-25T09:53:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f093406b500 0x7f093406d9c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f095015b7d0 0x7f09500b7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09500b5620 0x7f09500b79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 --2- 172.21.15.19:0/2319298619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f095006c2d0 0x7f09500b2f10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 -- 172.21.15.19:0/2319298619 >> 172.21.15.19:0/2319298619 conn(0x7f09500d0800 msgr2=0x7f0950160da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 -- 172.21.15.19:0/2319298619 shutdown_connections 2024-04-25T09:53:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.600+0000 7f095873b640 1 -- 172.21.15.19:0/2319298619 wait complete. 2024-04-25T09:53:54.613 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:54.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.715+0000 7f870542a640 1 Processor -- start 2024-04-25T09:53:54.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.715+0000 7f870542a640 1 -- start start 2024-04-25T09:53:54.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f870542a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870015dc20 0x7f87000ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f870542a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870006cda0 con 0x7f87000d32f0 2024-04-25T09:53:54.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f870542a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870015e000 con 0x7f8700161710 2024-04-25T09:53:54.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f870542a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87000d1de0 con 0x7f870015dc20 2024-04-25T09:53:54.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86ff7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f87000d32f0 0x7f870006c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35054/0 (socket says 172.21.15.19:35054) 2024-04-25T09:53:54.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86ff7fe640 1 -- 172.21.15.19:0/55259184 learned_addr learned my addr 172.21.15.19:0/55259184 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:54.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86feffd640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870015dc20 0x7f87000ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605298704 0 0) 0x7f870006cda0 con 0x7f87000d32f0 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e8003520 con 0x7f87000d32f0 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1632561400 0 0) 0x7f86e8003520 con 0x7f87000d32f0 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870015dc20 msgr2=0x7f87000ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870015dc20 0x7f87000ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 >> v1:172.21.15.112:6789/0 conn(0x7f8700161710 legacy=0x7f870006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8700161c60 con 0x7f87000d32f0 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f86f80019f0 con 0x7f87000d32f0 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 >> v1:172.21.15.19:6789/0 conn(0x7f87000d32f0 legacy=0x7f870006c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870015dc20 0x7f86e80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e8003f60 0x7f86e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e8004d40 0x7f86e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000d1de0 con 0x7f86e8003f60 2024-04-25T09:53:54.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e800baa0 con 0x7f86e8004d40 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.716+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e800bca0 con 0x7f870015dc20 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.717+0000 7f86fe7fc640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e8003f60 0x7f86e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.717+0000 7f86feffd640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870015dc20 0x7f86e80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.717+0000 7f86ff7fe640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e8004d40 0x7f86e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.717+0000 7f86fe7fc640 1 -- 172.21.15.19:0/55259184 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870015dc20 msgr2=0x7f86e80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.717+0000 7f86fe7fc640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870015dc20 0x7f86e80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.717+0000 7f86fe7fc640 1 -- 172.21.15.19:0/55259184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e8004d40 msgr2=0x7f86e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.717+0000 7f86fe7fc640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e8004d40 0x7f86e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.717+0000 7f86fe7fc640 1 -- 172.21.15.19:0/55259184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e8003520 con 0x7f86e8003f60 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.718+0000 7f86fe7fc640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e8003f60 0x7f86e8004800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f86f4001750 tx=0x7f86e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca0fded487b652d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.718+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4001370 con 0x7f86e8003f60 2024-04-25T09:53:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.718+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f40052d0 con 0x7f86e8003f60 2024-04-25T09:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.718+0000 7f86fdffb640 1 -- 172.21.15.19:0/55259184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f402abe0 con 0x7f86e8003f60 2024-04-25T09:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.718+0000 7f870542a640 1 -- 172.21.15.19:0/55259184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e8003f60 msgr2=0x7f86e8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.718+0000 7f870542a640 1 --2- 172.21.15.19:0/55259184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e8003f60 0x7f86e8004800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f86f4001750 tx=0x7f86e8005830 comp rx=0 tx=0).stop 2024-04-25T09:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.718+0000 7f86ff7fe640 1 -- 172.21.15.19:0/55259184 reap_dead start 2024-04-25T09:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.718+0000 7f870542a640 1 -- 172.21.15.19:0/55259184 shutdown_connections 2024-04-25T09:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.718+0000 7f870542a640 1 -- 172.21.15.19:0/55259184 >> 172.21.15.19:0/55259184 conn(0x7f87000d0420 msgr2=0x7f87000d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.718+0000 7f870542a640 1 -- 172.21.15.19:0/55259184 shutdown_connections 2024-04-25T09:53:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.718+0000 7f870542a640 1 -- 172.21.15.19:0/55259184 wait complete. 2024-04-25T09:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.720+0000 7f870542a640 1 Processor -- start 2024-04-25T09:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.720+0000 7f870542a640 1 -- start start 2024-04-25T09:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.720+0000 7f870542a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e8003f60 0x7f87000cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.720+0000 7f870542a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87001e7f00 0x7f87001e8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.720+0000 7f870542a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870014f8d0 0x7f870014fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.720+0000 7f870542a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8700162500 con 0x7f87001e7f00 2024-04-25T09:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.720+0000 7f870542a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8700162220 con 0x7f86e8003f60 2024-04-25T09:53:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.720+0000 7f870542a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87000b5e50 con 0x7f870014f8d0 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.720+0000 7f86fe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87001e7f00 0x7f87001e8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86fe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87001e7f00 0x7f87001e8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59198/0 (socket says 172.21.15.19:59198) 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86fe7fc640 1 -- 172.21.15.19:0/2044576487 learned_addr learned my addr 172.21.15.19:0/2044576487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86feffd640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e8003f60 0x7f87000cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86ff7fe640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870014f8d0 0x7f870014fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86fe7fc640 1 -- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870014f8d0 msgr2=0x7f870014fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86fe7fc640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870014f8d0 0x7f870014fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86fe7fc640 1 -- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e8003f60 msgr2=0x7f87000cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86fe7fc640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e8003f60 0x7f87000cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86fe7fc640 1 -- 172.21.15.19:0/2044576487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87001e89c0 con 0x7f87001e7f00 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86fe7fc640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87001e7f00 0x7f87001e8380 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f86f4005040 tx=0x7f86f4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86e77fe640 1 -- 172.21.15.19:0/2044576487 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4040020 con 0x7f87001e7f00 2024-04-25T09:53:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86e77fe640 1 -- 172.21.15.19:0/2044576487 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f4005480 con 0x7f87001e7f00 2024-04-25T09:53:54.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f870542a640 1 -- 172.21.15.19:0/2044576487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87001e8ca0 con 0x7f87001e7f00 2024-04-25T09:53:54.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f86e77fe640 1 -- 172.21.15.19:0/2044576487 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4003c30 con 0x7f87001e7f00 2024-04-25T09:53:54.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.721+0000 7f870542a640 1 -- 172.21.15.19:0/2044576487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87000caf60 con 0x7f87001e7f00 2024-04-25T09:53:54.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.722+0000 7f870542a640 1 -- 172.21.15.19:0/2044576487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8700073a00 con 0x7f87001e7f00 2024-04-25T09:53:54.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.723+0000 7f86e77fe640 1 -- 172.21.15.19:0/2044576487 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86f4043020 con 0x7f87001e7f00 2024-04-25T09:53:54.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.723+0000 7f86e77fe640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f86ec06b4d0 0x7f86ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.723+0000 7f86e77fe640 1 -- 172.21.15.19:0/2044576487 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f86f40050d0 con 0x7f87001e7f00 2024-04-25T09:53:54.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.723+0000 7f86feffd640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f86ec06b4d0 0x7f86ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.724+0000 7f86feffd640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f86ec06b4d0 0x7f86ec06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009b00 tx=0x7f86f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.727+0000 7f86e77fe640 1 -- 172.21.15.19:0/2044576487 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f86f404a030 con 0x7f87001e7f00 2024-04-25T09:53:54.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.907+0000 7f870542a640 1 -- 172.21.15.19:0/2044576487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87000d16b0 con 0x7f87001e7f00 2024-04-25T09:53:54.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.908+0000 7f86e77fe640 1 -- 172.21.15.19:0/2044576487 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f86f4079260 con 0x7f87001e7f00 2024-04-25T09:53:54.910 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:54.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 -- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f86ec06b4d0 msgr2=0x7f86ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f86ec06b4d0 0x7f86ec06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009b00 tx=0x7f86f0008040 comp rx=0 tx=0).stop 2024-04-25T09:53:54.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 -- 172.21.15.19:0/2044576487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87001e7f00 msgr2=0x7f87001e8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87001e7f00 0x7f87001e8380 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f86f4005040 tx=0x7f86f4001120 comp rx=0 tx=0).stop 2024-04-25T09:53:54.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 -- 172.21.15.19:0/2044576487 shutdown_connections 2024-04-25T09:53:54.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f86ec06b4d0 0x7f86ec06d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f870014f8d0 0x7f870014fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87001e7f00 0x7f87001e8380 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 --2- 172.21.15.19:0/2044576487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86e8003f60 0x7f87000cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 -- 172.21.15.19:0/2044576487 >> 172.21.15.19:0/2044576487 conn(0x7f87000d0420 msgr2=0x7f8700160220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 -- 172.21.15.19:0/2044576487 shutdown_connections 2024-04-25T09:53:54.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:54.910+0000 7f870542a640 1 -- 172.21.15.19:0/2044576487 wait complete. 2024-04-25T09:53:54.923 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.024+0000 7ff467edd640 1 Processor -- start 2024-04-25T09:53:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.024+0000 7ff467edd640 1 -- start start 2024-04-25T09:53:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.025+0000 7ff467edd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff46015dc20 0x7ff4600ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.025+0000 7ff467edd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4600d32f0 0x7ff4600cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.025+0000 7ff467edd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46015e000 con 0x7ff460161710 2024-04-25T09:53:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.025+0000 7ff467edd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4600d1de0 con 0x7ff46015dc20 2024-04-25T09:53:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.025+0000 7ff467edd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff460161c60 con 0x7ff4600d32f0 2024-04-25T09:53:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.025+0000 7ff46647a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4600d32f0 0x7ff4600cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.025+0000 7ff465c79640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff46015dc20 0x7ff4600ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.025+0000 7ff46647a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4600d32f0 0x7ff4600cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:44678/0 (socket says 172.21.15.19:44678) 2024-04-25T09:53:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.025+0000 7ff46647a640 1 -- 172.21.15.19:0/2844758843 learned_addr learned my addr 172.21.15.19:0/2844758843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.025+0000 7ff465478640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff460161710 0x7ff46006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:38562/0 (socket says 172.21.15.19:38562) 2024-04-25T09:53:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.026+0000 7ff46647a640 1 -- 172.21.15.19:0/2844758843 >> v2:172.21.15.112:3300/0 conn(0x7ff46015dc20 msgr2=0x7ff4600ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:55.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.026+0000 7ff46647a640 1 --2- 172.21.15.19:0/2844758843 >> v2:172.21.15.112:3300/0 conn(0x7ff46015dc20 0x7ff4600ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.026+0000 7ff46647a640 1 -- 172.21.15.19:0/2844758843 >> v1:172.21.15.112:6789/0 conn(0x7ff460161710 legacy=0x7ff46006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.026+0000 7ff46647a640 1 -- 172.21.15.19:0/2844758843 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460161f40 con 0x7ff4600d32f0 2024-04-25T09:53:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.026+0000 7ff46647a640 1 --2- 172.21.15.19:0/2844758843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4600d32f0 0x7ff4600cebb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff458009e50 tx=0x7ff45802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ffb6b8de6b930b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.026+0000 7ff464c77640 1 -- 172.21.15.19:0/2844758843 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458036070 con 0x7ff4600d32f0 2024-04-25T09:53:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.027+0000 7ff464c77640 1 -- 172.21.15.19:0/2844758843 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff458031600 con 0x7ff4600d32f0 2024-04-25T09:53:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.027+0000 7ff464c77640 1 -- 172.21.15.19:0/2844758843 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4580319d0 con 0x7ff4600d32f0 2024-04-25T09:53:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.027+0000 7ff467edd640 1 -- 172.21.15.19:0/2844758843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4600d32f0 msgr2=0x7ff4600cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.027+0000 7ff467edd640 1 --2- 172.21.15.19:0/2844758843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4600d32f0 0x7ff4600cebb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff458009e50 tx=0x7ff45802a710 comp rx=0 tx=0).stop 2024-04-25T09:53:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.027+0000 7ff467edd640 1 -- 172.21.15.19:0/2844758843 shutdown_connections 2024-04-25T09:53:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.027+0000 7ff467edd640 1 --2- 172.21.15.19:0/2844758843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4600d32f0 0x7ff4600cebb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.027+0000 7ff467edd640 1 --2- 172.21.15.19:0/2844758843 >> v2:172.21.15.112:3300/0 conn(0x7ff46015dc20 0x7ff4600ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.027+0000 7ff467edd640 1 -- 172.21.15.19:0/2844758843 >> 172.21.15.19:0/2844758843 conn(0x7ff4600d0420 msgr2=0x7ff4600d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.027+0000 7ff467edd640 1 -- 172.21.15.19:0/2844758843 shutdown_connections 2024-04-25T09:53:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.027+0000 7ff467edd640 1 -- 172.21.15.19:0/2844758843 wait complete. 2024-04-25T09:53:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.028+0000 7ff467edd640 1 Processor -- start 2024-04-25T09:53:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.028+0000 7ff467edd640 1 -- start start 2024-04-25T09:53:55.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.029+0000 7ff467edd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4600d32f0 0x7ff46015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.029+0000 7ff467edd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff46015dc20 0x7ff4600abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.029+0000 7ff467edd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff460161710 0x7ff4600b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.029+0000 7ff467edd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600b5e50 con 0x7ff4600d32f0 2024-04-25T09:53:55.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.029+0000 7ff467edd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff460162500 con 0x7ff46015dc20 2024-04-25T09:53:55.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.029+0000 7ff467edd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600b5fd0 con 0x7ff460161710 2024-04-25T09:53:55.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.029+0000 7ff465c79640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4600d32f0 0x7ff46015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.029+0000 7ff465c79640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4600d32f0 0x7ff46015ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59214/0 (socket says 172.21.15.19:59214) 2024-04-25T09:53:55.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.029+0000 7ff465c79640 1 -- 172.21.15.19:0/3861401323 learned_addr learned my addr 172.21.15.19:0/3861401323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:55.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.029+0000 7ff465478640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff46015dc20 0x7ff4600abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.029+0000 7ff46647a640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff460161710 0x7ff4600b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.030+0000 7ff465c79640 1 -- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff460161710 msgr2=0x7ff4600b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.030+0000 7ff465c79640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff460161710 0x7ff4600b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.030+0000 7ff465c79640 1 -- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff46015dc20 msgr2=0x7ff4600abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.030+0000 7ff465c79640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff46015dc20 0x7ff4600abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.030+0000 7ff465c79640 1 -- 172.21.15.19:0/3861401323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600ac800 con 0x7ff4600d32f0 2024-04-25T09:53:55.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.030+0000 7ff465c79640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4600d32f0 0x7ff46015ca80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00e870 tx=0x7ff45c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.030+0000 7ff4567fc640 1 -- 172.21.15.19:0/3861401323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c017070 con 0x7ff4600d32f0 2024-04-25T09:53:55.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.030+0000 7ff4567fc640 1 -- 172.21.15.19:0/3861401323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c00c940 con 0x7ff4600d32f0 2024-04-25T09:53:55.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.030+0000 7ff467edd640 1 -- 172.21.15.19:0/3861401323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4600acae0 con 0x7ff4600d32f0 2024-04-25T09:53:55.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.030+0000 7ff4567fc640 1 -- 172.21.15.19:0/3861401323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00cd30 con 0x7ff4600d32f0 2024-04-25T09:53:55.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.030+0000 7ff467edd640 1 -- 172.21.15.19:0/3861401323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4600ad0a0 con 0x7ff4600d32f0 2024-04-25T09:53:55.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.031+0000 7ff467edd640 1 -- 172.21.15.19:0/3861401323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4600cfc20 con 0x7ff4600d32f0 2024-04-25T09:53:55.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.032+0000 7ff4567fc640 1 -- 172.21.15.19:0/3861401323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff45c01a070 con 0x7ff4600d32f0 2024-04-25T09:53:55.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.032+0000 7ff4567fc640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff44406b4d0 0x7ff44406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.032+0000 7ff4567fc640 1 -- 172.21.15.19:0/3861401323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff45c08a770 con 0x7ff4600d32f0 2024-04-25T09:53:55.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.032+0000 7ff465478640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff44406b4d0 0x7ff44406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.033+0000 7ff465478640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff44406b4d0 0x7ff44406d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff4601f13d0 tx=0x7ff450007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.036+0000 7ff4567fc640 1 -- 172.21.15.19:0/3861401323 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4600cfc20 con 0x7ff4600d32f0 2024-04-25T09:53:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.219+0000 7ff467edd640 1 -- 172.21.15.19:0/3861401323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff460073a00 con 0x7ff4600d32f0 2024-04-25T09:53:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.219+0000 7ff4567fc640 1 -- 172.21.15.19:0/3861401323 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff45c058ba0 con 0x7ff4600d32f0 2024-04-25T09:53:55.221 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 -- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff44406b4d0 msgr2=0x7ff44406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff44406b4d0 0x7ff44406d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff4601f13d0 tx=0x7ff450007e80 comp rx=0 tx=0).stop 2024-04-25T09:53:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 -- 172.21.15.19:0/3861401323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4600d32f0 msgr2=0x7ff46015ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4600d32f0 0x7ff46015ca80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00e870 tx=0x7ff45c00ed30 comp rx=0 tx=0).stop 2024-04-25T09:53:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 -- 172.21.15.19:0/3861401323 shutdown_connections 2024-04-25T09:53:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff44406b4d0 0x7ff44406d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff460161710 0x7ff4600b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff46015dc20 0x7ff4600abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 --2- 172.21.15.19:0/3861401323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4600d32f0 0x7ff46015ca80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 -- 172.21.15.19:0/3861401323 >> 172.21.15.19:0/3861401323 conn(0x7ff4600d0420 msgr2=0x7ff4600d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 -- 172.21.15.19:0/3861401323 shutdown_connections 2024-04-25T09:53:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.221+0000 7ff467edd640 1 -- 172.21.15.19:0/3861401323 wait complete. 2024-04-25T09:53:55.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:55.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.338+0000 7feba60ac640 1 Processor -- start 2024-04-25T09:53:55.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.338+0000 7feba60ac640 1 -- start start 2024-04-25T09:53:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.339+0000 7feba60ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feba015dc20 0x7feba00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.339+0000 7feba60ac640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba006cda0 con 0x7feba0161710 2024-04-25T09:53:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.339+0000 7feba60ac640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba015e000 con 0x7feba00d32f0 2024-04-25T09:53:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.339+0000 7feba60ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba00d1de0 con 0x7feba015dc20 2024-04-25T09:53:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.339+0000 7feb9effd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feba0161710 0x7feba006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35066/0 (socket says 172.21.15.19:35066) 2024-04-25T09:53:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.339+0000 7feb9effd640 1 -- 172.21.15.19:0/3540849996 learned_addr learned my addr 172.21.15.19:0/3540849996 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.339+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218388433 0 0) 0x7feba006cda0 con 0x7feba0161710 2024-04-25T09:53:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.339+0000 7feb9f7fe640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feba015dc20 0x7feba00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb88003520 con 0x7feba0161710 2024-04-25T09:53:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4126215714 0 0) 0x7feb88003520 con 0x7feba0161710 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feba015dc20 msgr2=0x7feba00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feba015dc20 0x7feba00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 >> v1:172.21.15.112:6789/0 conn(0x7feba00d32f0 legacy=0x7feba006c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba0161c60 con 0x7feba0161710 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7feb94001670 con 0x7feba0161710 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 >> v1:172.21.15.19:6789/0 conn(0x7feba0161710 legacy=0x7feba006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb880039e0 0x7feb88003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb88004800 0x7feb88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feba015dc20 0x7feb880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba00d1de0 con 0x7feb88004800 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8800ba70 con 0x7feb880039e0 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8800bc70 con 0x7feba015dc20 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9effd640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb88004800 0x7feb88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9f7fe640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb880039e0 0x7feb88003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9ffff640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feba015dc20 0x7feb880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9effd640 1 -- 172.21.15.19:0/3540849996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feba015dc20 msgr2=0x7feb880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9effd640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feba015dc20 0x7feb880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9effd640 1 -- 172.21.15.19:0/3540849996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb880039e0 msgr2=0x7feb88003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9effd640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb880039e0 0x7feb88003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.340+0000 7feb9effd640 1 -- 172.21.15.19:0/3540849996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb88003520 con 0x7feb88004800 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.341+0000 7feb9effd640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb88004800 0x7feb88004c80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c210 tx=0x7feb88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ca4df6146118b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.341+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94007d10 con 0x7feb88004800 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.341+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb94005ad0 con 0x7feb88004800 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.341+0000 7feb9e7fc640 1 -- 172.21.15.19:0/3540849996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94007070 con 0x7feb88004800 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.342+0000 7feba60ac640 1 -- 172.21.15.19:0/3540849996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb88004800 msgr2=0x7feb88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.342+0000 7feba60ac640 1 --2- 172.21.15.19:0/3540849996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb88004800 0x7feb88004c80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c210 tx=0x7feb88005830 comp rx=0 tx=0).stop 2024-04-25T09:53:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.342+0000 7feb9ffff640 1 -- 172.21.15.19:0/3540849996 reap_dead start 2024-04-25T09:53:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.342+0000 7feba60ac640 1 -- 172.21.15.19:0/3540849996 shutdown_connections 2024-04-25T09:53:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.342+0000 7feba60ac640 1 -- 172.21.15.19:0/3540849996 >> 172.21.15.19:0/3540849996 conn(0x7feba00d0420 msgr2=0x7feba00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.342+0000 7feba60ac640 1 -- 172.21.15.19:0/3540849996 shutdown_connections 2024-04-25T09:53:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.342+0000 7feba60ac640 1 -- 172.21.15.19:0/3540849996 wait complete. 2024-04-25T09:53:55.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.343+0000 7feba60ac640 1 Processor -- start 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.343+0000 7feba60ac640 1 -- start start 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feba60ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb88004800 0x7feba00b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feba60ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba015c230 0x7feba00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feba60ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feba015c6b0 0x7feba00cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feba60ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba0162500 con 0x7feba015c230 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feba60ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba0162220 con 0x7feba015c6b0 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feba60ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba00b5e50 con 0x7feb88004800 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba015c230 0x7feba00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba015c230 0x7feba00caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59240/0 (socket says 172.21.15.19:59240) 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9effd640 1 -- 172.21.15.19:0/2253657012 learned_addr learned my addr 172.21.15.19:0/2253657012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9f7fe640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb88004800 0x7feba00b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9ffff640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feba015c6b0 0x7feba00cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9effd640 1 -- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb88004800 msgr2=0x7feba00b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9effd640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb88004800 0x7feba00b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9effd640 1 -- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feba015c6b0 msgr2=0x7feba00cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9effd640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feba015c6b0 0x7feba00cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9effd640 1 -- 172.21.15.19:0/2253657012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba00cb920 con 0x7feba015c230 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9ffff640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feba015c6b0 0x7feba00cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9f7fe640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb88004800 0x7feba00b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb9effd640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba015c230 0x7feba00caa70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c210 tx=0x7feba0161e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.344+0000 7feb87fff640 1 -- 172.21.15.19:0/2253657012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94041020 con 0x7feba015c230 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.345+0000 7feb87fff640 1 -- 172.21.15.19:0/2253657012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb94001430 con 0x7feba015c230 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.345+0000 7feba60ac640 1 -- 172.21.15.19:0/2253657012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba00cbc00 con 0x7feba015c230 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.345+0000 7feb87fff640 1 -- 172.21.15.19:0/2253657012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94007070 con 0x7feba015c230 2024-04-25T09:53:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.345+0000 7feba60ac640 1 -- 172.21.15.19:0/2253657012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba00cc1c0 con 0x7feba015c230 2024-04-25T09:53:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.346+0000 7feba60ac640 1 -- 172.21.15.19:0/2253657012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba0073a00 con 0x7feba015c230 2024-04-25T09:53:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.346+0000 7feb87fff640 1 -- 172.21.15.19:0/2253657012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb94044020 con 0x7feba015c230 2024-04-25T09:53:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.346+0000 7feb87fff640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb8c06b500 0x7feb8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.347+0000 7feb87fff640 1 -- 172.21.15.19:0/2253657012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7feb940aaa60 con 0x7feba015c230 2024-04-25T09:53:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.347+0000 7feb9f7fe640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb8c06b500 0x7feb8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.347+0000 7feb9f7fe640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb8c06b500 0x7feb8c06d9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feba00d2f90 tx=0x7feb90008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.351+0000 7feb87fff640 1 -- 172.21.15.19:0/2253657012 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb94005aa0 con 0x7feba015c230 2024-04-25T09:53:55.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.531+0000 7feba60ac640 1 -- 172.21.15.19:0/2253657012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feba00d1a80 con 0x7feba015c230 2024-04-25T09:53:55.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.532+0000 7feb87fff640 1 -- 172.21.15.19:0/2253657012 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7feb940794e0 con 0x7feba015c230 2024-04-25T09:53:55.534 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:55.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 -- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb8c06b500 msgr2=0x7feb8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb8c06b500 0x7feb8c06d9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feba00d2f90 tx=0x7feb90008040 comp rx=0 tx=0).stop 2024-04-25T09:53:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 -- 172.21.15.19:0/2253657012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba015c230 msgr2=0x7feba00caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba015c230 0x7feba00caa70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c210 tx=0x7feba0161e30 comp rx=0 tx=0).stop 2024-04-25T09:53:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 -- 172.21.15.19:0/2253657012 shutdown_connections 2024-04-25T09:53:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb8c06b500 0x7feb8c06d9c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feba015c6b0 0x7feba00cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba015c230 0x7feba00caa70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 --2- 172.21.15.19:0/2253657012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb88004800 0x7feba00b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 -- 172.21.15.19:0/2253657012 >> 172.21.15.19:0/2253657012 conn(0x7feba00d0420 msgr2=0x7feba00d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 -- 172.21.15.19:0/2253657012 shutdown_connections 2024-04-25T09:53:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.534+0000 7feba60ac640 1 -- 172.21.15.19:0/2253657012 wait complete. 2024-04-25T09:53:55.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:53:55.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.646+0000 7f8d4346b640 1 Processor -- start 2024-04-25T09:53:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.647+0000 7f8d4346b640 1 -- start start 2024-04-25T09:53:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.647+0000 7f8d4346b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8d3c1610f0 0x7f8d3c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.647+0000 7f8d4346b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8d3c0ce7b0 0x7f8d3c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.647+0000 7f8d4346b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3c0c6960 con 0x7f8d3c1616f0 2024-04-25T09:53:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.647+0000 7f8d4346b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3c1614d0 con 0x7f8d3c1610f0 2024-04-25T09:53:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.647+0000 7f8d4346b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3c162260 con 0x7f8d3c0ce7b0 2024-04-25T09:53:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.647+0000 7f8d41207640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8d3c1610f0 0x7f8d3c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.648+0000 7f8d41207640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8d3c1610f0 0x7f8d3c06c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59242/0 (socket says 172.21.15.19:59242) 2024-04-25T09:53:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.648+0000 7f8d41207640 1 -- 172.21.15.19:0/2284050949 learned_addr learned my addr 172.21.15.19:0/2284050949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.648+0000 7f8d41a08640 1 --2- 172.21.15.19:0/2284050949 >> v2:172.21.15.112:3300/0 conn(0x7f8d3c0ce7b0 0x7f8d3c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.648+0000 7f8d41207640 1 -- 172.21.15.19:0/2284050949 >> v2:172.21.15.112:3300/0 conn(0x7f8d3c0ce7b0 msgr2=0x7f8d3c161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.648+0000 7f8d41207640 1 --2- 172.21.15.19:0/2284050949 >> v2:172.21.15.112:3300/0 conn(0x7f8d3c0ce7b0 0x7f8d3c161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.648+0000 7f8d41207640 1 -- 172.21.15.19:0/2284050949 >> v1:172.21.15.112:6789/0 conn(0x7f8d3c1616f0 legacy=0x7f8d3c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.648+0000 7f8d41207640 1 -- 172.21.15.19:0/2284050949 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c162540 con 0x7f8d3c1610f0 2024-04-25T09:53:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d41207640 1 --2- 172.21.15.19:0/2284050949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c1610f0 0x7f8d3c06c1c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0b33e0 tx=0x7f8d3402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9141d3272c165ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d3bfff640 1 -- 172.21.15.19:0/2284050949 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34036070 con 0x7f8d3c1610f0 2024-04-25T09:53:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d3bfff640 1 -- 172.21.15.19:0/2284050949 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d34031600 con 0x7f8d3c1610f0 2024-04-25T09:53:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d3bfff640 1 -- 172.21.15.19:0/2284050949 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d340319d0 con 0x7f8d3c1610f0 2024-04-25T09:53:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d4346b640 1 -- 172.21.15.19:0/2284050949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c1610f0 msgr2=0x7f8d3c06c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d4346b640 1 --2- 172.21.15.19:0/2284050949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c1610f0 0x7f8d3c06c1c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0b33e0 tx=0x7f8d3402a730 comp rx=0 tx=0).stop 2024-04-25T09:53:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d4346b640 1 -- 172.21.15.19:0/2284050949 shutdown_connections 2024-04-25T09:53:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d4346b640 1 --2- 172.21.15.19:0/2284050949 >> v2:172.21.15.112:3300/0 conn(0x7f8d3c0ce7b0 0x7f8d3c161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d4346b640 1 --2- 172.21.15.19:0/2284050949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c1610f0 0x7f8d3c06c1c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d4346b640 1 -- 172.21.15.19:0/2284050949 >> 172.21.15.19:0/2284050949 conn(0x7f8d3c06f500 msgr2=0x7f8d3c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d4346b640 1 -- 172.21.15.19:0/2284050949 shutdown_connections 2024-04-25T09:53:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.649+0000 7f8d4346b640 1 -- 172.21.15.19:0/2284050949 wait complete. 2024-04-25T09:53:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.651+0000 7f8d4346b640 1 Processor -- start 2024-04-25T09:53:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.651+0000 7f8d4346b640 1 -- start start 2024-04-25T09:53:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.651+0000 7f8d4346b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d3c0ce7b0 0x7f8d3c152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.651+0000 7f8d4346b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d3c1610f0 0x7f8d3c155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.651+0000 7f8d4346b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c1616f0 0x7f8d3c155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.651+0000 7f8d4346b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0cf690 con 0x7f8d3c1616f0 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.651+0000 7f8d4346b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0cfce0 con 0x7f8d3c0ce7b0 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.651+0000 7f8d4346b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c06c700 con 0x7f8d3c1610f0 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.651+0000 7f8d41a08640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c1616f0 0x7f8d3c155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.651+0000 7f8d41a08640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c1616f0 0x7f8d3c155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59258/0 (socket says 172.21.15.19:59258) 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.651+0000 7f8d41a08640 1 -- 172.21.15.19:0/2018160367 learned_addr learned my addr 172.21.15.19:0/2018160367 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d41207640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d3c0ce7b0 0x7f8d3c152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d40a06640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d3c1610f0 0x7f8d3c155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d41a08640 1 -- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d3c1610f0 msgr2=0x7f8d3c155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d41a08640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d3c1610f0 0x7f8d3c155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d41a08640 1 -- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d3c0ce7b0 msgr2=0x7f8d3c152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d41a08640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d3c0ce7b0 0x7f8d3c152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d41a08640 1 -- 172.21.15.19:0/2018160367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c0ae0d0 con 0x7f8d3c1616f0 2024-04-25T09:53:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d41207640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d3c0ce7b0 0x7f8d3c152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d41a08640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c1616f0 0x7f8d3c155670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c004650 tx=0x7f8d2c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d39ffb640 1 -- 172.21.15.19:0/2018160367 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c017070 con 0x7f8d3c1616f0 2024-04-25T09:53:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d39ffb640 1 -- 172.21.15.19:0/2018160367 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c00c940 con 0x7f8d3c1616f0 2024-04-25T09:53:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d4346b640 1 -- 172.21.15.19:0/2018160367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3c1e76b0 con 0x7f8d3c1616f0 2024-04-25T09:53:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d39ffb640 1 -- 172.21.15.19:0/2018160367 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00cd30 con 0x7f8d3c1616f0 2024-04-25T09:53:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.652+0000 7f8d4346b640 1 -- 172.21.15.19:0/2018160367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3c1e7c70 con 0x7f8d3c1616f0 2024-04-25T09:53:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.653+0000 7f8d4346b640 1 -- 172.21.15.19:0/2018160367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3c073a00 con 0x7f8d3c1616f0 2024-04-25T09:53:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.654+0000 7f8d39ffb640 1 -- 172.21.15.19:0/2018160367 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d2c01a070 con 0x7f8d3c1616f0 2024-04-25T09:53:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.654+0000 7f8d39ffb640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d2806b4d0 0x7f8d2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.654+0000 7f8d39ffb640 1 -- 172.21.15.19:0/2018160367 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8d2c08a570 con 0x7f8d3c1616f0 2024-04-25T09:53:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.654+0000 7f8d41207640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d2806b4d0 0x7f8d2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.655+0000 7f8d41207640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d2806b4d0 0x7f8d2806d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8d34033000 tx=0x7f8d34009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.658+0000 7f8d39ffb640 1 -- 172.21.15.19:0/2018160367 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d2c059080 con 0x7f8d3c1616f0 2024-04-25T09:53:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.854+0000 7f8d4346b640 1 -- 172.21.15.19:0/2018160367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8d3c06f980 con 0x7f8d3c1616f0 2024-04-25T09:53:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.854+0000 7f8d39ffb640 1 -- 172.21.15.19:0/2018160367 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8d2c058980 con 0x7f8d3c1616f0 2024-04-25T09:53:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T09:53:55.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 -- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d2806b4d0 msgr2=0x7f8d2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d2806b4d0 0x7f8d2806d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8d34033000 tx=0x7f8d34009000 comp rx=0 tx=0).stop 2024-04-25T09:53:55.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 -- 172.21.15.19:0/2018160367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c1616f0 msgr2=0x7f8d3c155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c1616f0 0x7f8d3c155670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c004650 tx=0x7f8d2c00ece0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 -- 172.21.15.19:0/2018160367 shutdown_connections 2024-04-25T09:53:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d2806b4d0 0x7f8d2806d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c1616f0 0x7f8d3c155670 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d3c1610f0 0x7f8d3c155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 --2- 172.21.15.19:0/2018160367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d3c0ce7b0 0x7f8d3c152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 -- 172.21.15.19:0/2018160367 >> 172.21.15.19:0/2018160367 conn(0x7f8d3c06f500 msgr2=0x7f8d3c0c6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 -- 172.21.15.19:0/2018160367 shutdown_connections 2024-04-25T09:53:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:55.856+0000 7f8d4346b640 1 -- 172.21.15.19:0/2018160367 wait complete. 2024-04-25T09:53:55.869 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T09:53:55.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:55.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T09:53:55.894 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:53:55.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:55.895 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:53:55.951 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:53:55.951 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:55.969 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:53:55.970 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:56.031 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:53:56.032 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:56.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:56.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.202+0000 7f25422f5640 1 Processor -- start 2024-04-25T09:53:56.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.202+0000 7f25422f5640 1 -- start start 2024-04-25T09:53:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f25422f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f253c158220 0x7f253c15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f25422f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f253c0af720 0x7f253c0afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f25422f5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c15f180 con 0x7f253c1617b0 2024-04-25T09:53:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f25422f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c0b3670 con 0x7f253c0af720 2024-04-25T09:53:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f25422f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c070a10 con 0x7f253c158220 2024-04-25T09:53:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f2540892640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f253c1617b0 0x7f253c161c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35074/0 (socket says 172.21.15.19:35074) 2024-04-25T09:53:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f2540892640 1 -- 172.21.15.19:0/3332004360 learned_addr learned my addr 172.21.15.19:0/3332004360 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f253bfff640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f253c158220 0x7f253c15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f253b7fe640 1 --2- 172.21.15.19:0/3332004360 >> v2:172.21.15.112:3300/0 conn(0x7f253c0af720 0x7f253c0afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9216023 0 0) 0x7f2534001490 con 0x7f253c1617b0 2024-04-25T09:53:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2524003520 con 0x7f253c1617b0 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 882951786 0 0) 0x7f2524003520 con 0x7f253c1617b0 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f253c158220 msgr2=0x7f253c15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f253affd640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f253c158220 0x7f253c15ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 >> v2:172.21.15.112:3300/0 conn(0x7f253c0af720 msgr2=0x7f253c0afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f253affd640 1 --2- 172.21.15.19:0/3332004360 >> v2:172.21.15.112:3300/0 conn(0x7f253c0af720 0x7f253c0afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c06fe00 con 0x7f253c1617b0 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.203+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f2534001bc0 con 0x7f253c1617b0 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 >> v1:172.21.15.19:6789/0 conn(0x7f253c1617b0 legacy=0x7f253c161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253affd640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f253c158220 0x7f25240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253affd640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2524003f30 0x7f2524004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253affd640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2524004d40 0x7f25240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c070a10 con 0x7f2524003f30 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252400b9c0 con 0x7f2524004d40 2024-04-25T09:53:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252400bb40 con 0x7f253c158220 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253b7fe640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2524003f30 0x7f2524004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253bfff640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f253c158220 0x7f25240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f2540892640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2524004d40 0x7f25240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253b7fe640 1 -- 172.21.15.19:0/3332004360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f253c158220 msgr2=0x7f25240039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253b7fe640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f253c158220 0x7f25240039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253b7fe640 1 -- 172.21.15.19:0/3332004360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2524004d40 msgr2=0x7f25240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253b7fe640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2524004d40 0x7f25240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.204+0000 7f253b7fe640 1 -- 172.21.15.19:0/3332004360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524003520 con 0x7f2524003f30 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.205+0000 7f253b7fe640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2524003f30 0x7f2524004800 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f252c009ac0 tx=0x7f2524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14406d2ed2b698c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.205+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c036070 con 0x7f2524003f30 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.205+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c031400 con 0x7f2524003f30 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.205+0000 7f253affd640 1 -- 172.21.15.19:0/3332004360 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c009690 con 0x7f2524003f30 2024-04-25T09:53:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.206+0000 7f25422f5640 1 -- 172.21.15.19:0/3332004360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2524003f30 msgr2=0x7f2524004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.206+0000 7f25422f5640 1 --2- 172.21.15.19:0/3332004360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2524003f30 0x7f2524004800 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f252c009ac0 tx=0x7f2524005830 comp rx=0 tx=0).stop 2024-04-25T09:53:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.206+0000 7f2540892640 1 -- 172.21.15.19:0/3332004360 reap_dead start 2024-04-25T09:53:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.206+0000 7f25422f5640 1 -- 172.21.15.19:0/3332004360 shutdown_connections 2024-04-25T09:53:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.206+0000 7f25422f5640 1 -- 172.21.15.19:0/3332004360 >> 172.21.15.19:0/3332004360 conn(0x7f253c0b2370 msgr2=0x7f253c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.206+0000 7f25422f5640 1 -- 172.21.15.19:0/3332004360 shutdown_connections 2024-04-25T09:53:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.206+0000 7f25422f5640 1 -- 172.21.15.19:0/3332004360 wait complete. 2024-04-25T09:53:56.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.207+0000 7f25422f5640 1 Processor -- start 2024-04-25T09:53:56.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.207+0000 7f25422f5640 1 -- start start 2024-04-25T09:53:56.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f25422f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2524003f30 0x7f253c14c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f25422f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f253c15a610 0x7f253c154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f25422f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f253c154ba0 0x7f253c159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f25422f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0706a0 con 0x7f253c154ba0 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f25422f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0703c0 con 0x7f253c15a610 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f25422f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c158600 con 0x7f2524003f30 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f2540892640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f253c154ba0 0x7f253c159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f2540892640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f253c154ba0 0x7f253c159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59272/0 (socket says 172.21.15.19:59272) 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f2540892640 1 -- 172.21.15.19:0/3562623087 learned_addr learned my addr 172.21.15.19:0/3562623087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f253bfff640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2524003f30 0x7f253c14c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f253b7fe640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f253c15a610 0x7f253c154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f2540892640 1 -- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2524003f30 msgr2=0x7f253c14c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f2540892640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2524003f30 0x7f253c14c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f2540892640 1 -- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f253c15a610 msgr2=0x7f253c154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f2540892640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f253c15a610 0x7f253c154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f2540892640 1 -- 172.21.15.19:0/3562623087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c14c710 con 0x7f253c154ba0 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.208+0000 7f2540892640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f253c154ba0 0x7f253c159960 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f253400ab10 tx=0x7f253400af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.209+0000 7f2538ff9640 1 -- 172.21.15.19:0/3562623087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534007810 con 0x7f253c154ba0 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.209+0000 7f2538ff9640 1 -- 172.21.15.19:0/3562623087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2534018070 con 0x7f253c154ba0 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.209+0000 7f2538ff9640 1 -- 172.21.15.19:0/3562623087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340133f0 con 0x7f253c154ba0 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.209+0000 7f25422f5640 1 -- 172.21.15.19:0/3562623087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c0b1680 con 0x7f253c154ba0 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.209+0000 7f25422f5640 1 -- 172.21.15.19:0/3562623087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c159160 con 0x7f253c154ba0 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.210+0000 7f25422f5640 1 -- 172.21.15.19:0/3562623087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c073990 con 0x7f253c154ba0 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.211+0000 7f2538ff9640 1 -- 172.21.15.19:0/3562623087 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2534013590 con 0x7f253c154ba0 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.211+0000 7f2538ff9640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f252806b4d0 0x7f252806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.211+0000 7f2538ff9640 1 -- 172.21.15.19:0/3562623087 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f253408ab30 con 0x7f253c154ba0 2024-04-25T09:53:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.211+0000 7f253bfff640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f252806b4d0 0x7f252806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.212+0000 7f253bfff640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f252806b4d0 0x7f252806d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f253c161610 tx=0x7f2530009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.215+0000 7f2538ff9640 1 -- 172.21.15.19:0/3562623087 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f253c14c710 con 0x7f253c154ba0 2024-04-25T09:53:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.398+0000 7f25422f5640 1 -- 172.21.15.19:0/3562623087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f253c0b3bc0 con 0x7f253c154ba0 2024-04-25T09:53:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.400+0000 7f2538ff9640 1 -- 172.21.15.19:0/3562623087 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18892 (secure 0 0 0) 0x7f253c15f180 con 0x7f253c154ba0 2024-04-25T09:53:56.402 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:56.402 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:48.071585+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 -- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f252806b4d0 msgr2=0x7f252806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f252806b4d0 0x7f252806d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f253c161610 tx=0x7f2530009190 comp rx=0 tx=0).stop 2024-04-25T09:53:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 -- 172.21.15.19:0/3562623087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f253c154ba0 msgr2=0x7f253c159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f253c154ba0 0x7f253c159960 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f253400ab10 tx=0x7f253400af10 comp rx=0 tx=0).stop 2024-04-25T09:53:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 -- 172.21.15.19:0/3562623087 shutdown_connections 2024-04-25T09:53:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f252806b4d0 0x7f252806d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f253c154ba0 0x7f253c159960 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f253c15a610 0x7f253c154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 --2- 172.21.15.19:0/3562623087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2524003f30 0x7f253c14c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 -- 172.21.15.19:0/3562623087 >> 172.21.15.19:0/3562623087 conn(0x7f253c0b2370 msgr2=0x7f253c0b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 -- 172.21.15.19:0/3562623087 shutdown_connections 2024-04-25T09:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.402+0000 7f25422f5640 1 -- 172.21.15.19:0/3562623087 wait complete. 2024-04-25T09:53:56.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:56.416 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:56.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.520+0000 7fe54825d640 1 Processor -- start 2024-04-25T09:53:56.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.520+0000 7fe54825d640 1 -- start start 2024-04-25T09:53:56.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe54825d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe5400cd8a0 0x7fe54015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe54825d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54015e6e0 con 0x7fe540150b90 2024-04-25T09:53:56.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe54825d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54015e9c0 con 0x7fe5400cd4c0 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe54825d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54015a9c0 con 0x7fe5400cd8a0 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe5467fa640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe540150b90 0x7fe540071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35086/0 (socket says 172.21.15.19:35086) 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe5467fa640 1 -- 172.21.15.19:0/1732877550 learned_addr learned my addr 172.21.15.19:0/1732877550 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe5457f8640 1 --2- 172.21.15.19:0/1732877550 >> v2:172.21.15.112:3300/0 conn(0x7fe5400cd8a0 0x7fe54015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195964538 0 0) 0x7fe54015e6e0 con 0x7fe540150b90 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe52c003440 con 0x7fe540150b90 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3108580372 0 0) 0x7fe52c003440 con 0x7fe540150b90 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 >> v2:172.21.15.112:3300/0 conn(0x7fe5400cd8a0 msgr2=0x7fe54015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe544ff7640 1 --2- 172.21.15.19:0/1732877550 >> v2:172.21.15.112:3300/0 conn(0x7fe5400cd8a0 0x7fe54015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.521+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 >> v1:172.21.15.112:6789/0 conn(0x7fe5400cd4c0 legacy=0x7fe54015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54015aca0 con 0x7fe540150b90 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fe5380019f0 con 0x7fe540150b90 2024-04-25T09:53:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 >> v1:172.21.15.19:6789/0 conn(0x7fe540150b90 legacy=0x7fe540071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe544ff7640 1 --2- 172.21.15.19:0/1732877550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c003aa0 0x7fe52c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe544ff7640 1 --2- 172.21.15.19:0/1732877550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe52c008850 0x7fe52c00acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe544ff7640 1 --2- 172.21.15.19:0/1732877550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe52c00b4c0 0x7fe52c00d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54015a9c0 con 0x7fe52c008850 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c00e260 con 0x7fe52c003aa0 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c00e460 con 0x7fe52c00b4c0 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe5457f8640 1 --2- 172.21.15.19:0/1732877550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe52c008850 0x7fe52c00acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe545ff9640 1 --2- 172.21.15.19:0/1732877550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c003aa0 0x7fe52c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe5467fa640 1 --2- 172.21.15.19:0/1732877550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe52c00b4c0 0x7fe52c00d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe5457f8640 1 -- 172.21.15.19:0/1732877550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe52c00b4c0 msgr2=0x7fe52c00d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe5457f8640 1 --2- 172.21.15.19:0/1732877550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe52c00b4c0 0x7fe52c00d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe5457f8640 1 -- 172.21.15.19:0/1732877550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c003aa0 msgr2=0x7fe52c003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe5457f8640 1 --2- 172.21.15.19:0/1732877550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c003aa0 0x7fe52c003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe5467fa640 1 -- 172.21.15.19:0/1732877550 reap_dead start 2024-04-25T09:53:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.522+0000 7fe5457f8640 1 -- 172.21.15.19:0/1732877550 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c003440 con 0x7fe52c008850 2024-04-25T09:53:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.523+0000 7fe5457f8640 1 --2- 172.21.15.19:0/1732877550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe52c008850 0x7fe52c00acc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe530009ac0 tx=0x7fe52c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e2fd312331368 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.523+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530036070 con 0x7fe52c008850 2024-04-25T09:53:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.524+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe530031400 con 0x7fe52c008850 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.524+0000 7fe544ff7640 1 -- 172.21.15.19:0/1732877550 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530009690 con 0x7fe52c008850 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.524+0000 7fe54825d640 1 -- 172.21.15.19:0/1732877550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe52c008850 msgr2=0x7fe52c00acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.524+0000 7fe54825d640 1 --2- 172.21.15.19:0/1732877550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe52c008850 0x7fe52c00acc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe530009ac0 tx=0x7fe52c003730 comp rx=0 tx=0).stop 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.524+0000 7fe54825d640 1 -- 172.21.15.19:0/1732877550 shutdown_connections 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.524+0000 7fe54825d640 1 --2- 172.21.15.19:0/1732877550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe52c008850 0x7fe52c00acc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.524+0000 7fe54825d640 1 -- 172.21.15.19:0/1732877550 >> 172.21.15.19:0/1732877550 conn(0x7fe54006abe0 msgr2=0x7fe54006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.524+0000 7fe54825d640 1 -- 172.21.15.19:0/1732877550 shutdown_connections 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.524+0000 7fe54825d640 1 -- 172.21.15.19:0/1732877550 wait complete. 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.525+0000 7fe54825d640 1 Processor -- start 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe54825d640 1 -- start start 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe54825d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c008850 0x7fe54015c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe54825d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5400cbbc0 0x7fe540149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe54825d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe540146fb0 0x7fe540147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe54825d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54006ee20 con 0x7fe540146fb0 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe54825d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54006eca0 con 0x7fe52c008850 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe54825d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54015b1c0 con 0x7fe5400cbbc0 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe5467fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe540146fb0 0x7fe540147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe5467fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe540146fb0 0x7fe540147430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59298/0 (socket says 172.21.15.19:59298) 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe5467fa640 1 -- 172.21.15.19:0/8979264 learned_addr learned my addr 172.21.15.19:0/8979264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe5457f8640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5400cbbc0 0x7fe540149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.526+0000 7fe545ff9640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c008850 0x7fe54015c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe5467fa640 1 -- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5400cbbc0 msgr2=0x7fe540149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe5467fa640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5400cbbc0 0x7fe540149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe5467fa640 1 -- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c008850 msgr2=0x7fe54015c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe5467fa640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c008850 0x7fe54015c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe5467fa640 1 -- 172.21.15.19:0/8979264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54015cd70 con 0x7fe540146fb0 2024-04-25T09:53:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe5457f8640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5400cbbc0 0x7fe540149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe545ff9640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c008850 0x7fe54015c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe5467fa640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe540146fb0 0x7fe540147430 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe5400ce930 tx=0x7fe538007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe5367fc640 1 -- 172.21.15.19:0/8979264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538017020 con 0x7fe540146fb0 2024-04-25T09:53:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe5367fc640 1 -- 172.21.15.19:0/8979264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53800abe0 con 0x7fe540146fb0 2024-04-25T09:53:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe54825d640 1 -- 172.21.15.19:0/8979264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54015d050 con 0x7fe540146fb0 2024-04-25T09:53:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe5367fc640 1 -- 172.21.15.19:0/8979264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5380133f0 con 0x7fe540146fb0 2024-04-25T09:53:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.527+0000 7fe54825d640 1 -- 172.21.15.19:0/8979264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54015d610 con 0x7fe540146fb0 2024-04-25T09:53:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.529+0000 7fe54825d640 1 -- 172.21.15.19:0/8979264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe540073a00 con 0x7fe540146fb0 2024-04-25T09:53:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.529+0000 7fe5367fc640 1 -- 172.21.15.19:0/8979264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe53801a020 con 0x7fe540146fb0 2024-04-25T09:53:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.530+0000 7fe5367fc640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe52406b500 0x7fe52406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.530+0000 7fe5367fc640 1 -- 172.21.15.19:0/8979264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe53808ad40 con 0x7fe540146fb0 2024-04-25T09:53:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.530+0000 7fe545ff9640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe52406b500 0x7fe52406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.530+0000 7fe545ff9640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe52406b500 0x7fe52406d9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe53c000c00 tx=0x7fe53c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.534+0000 7fe5367fc640 1 -- 172.21.15.19:0/8979264 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe538021020 con 0x7fe540146fb0 2024-04-25T09:53:56.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.725+0000 7fe54825d640 1 -- 172.21.15.19:0/8979264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe540150640 con 0x7fe540146fb0 2024-04-25T09:53:56.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.725+0000 7fe5367fc640 1 -- 172.21.15.19:0/8979264 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe538059850 con 0x7fe540146fb0 2024-04-25T09:53:56.728 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 -- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe52406b500 msgr2=0x7fe52406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe52406b500 0x7fe52406d9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe53c000c00 tx=0x7fe53c009040 comp rx=0 tx=0).stop 2024-04-25T09:53:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 -- 172.21.15.19:0/8979264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe540146fb0 msgr2=0x7fe540147430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe540146fb0 0x7fe540147430 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe5400ce930 tx=0x7fe538007bd0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 -- 172.21.15.19:0/8979264 shutdown_connections 2024-04-25T09:53:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe52406b500 0x7fe52406d9c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe540146fb0 0x7fe540147430 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5400cbbc0 0x7fe540149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 --2- 172.21.15.19:0/8979264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c008850 0x7fe54015c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 -- 172.21.15.19:0/8979264 >> 172.21.15.19:0/8979264 conn(0x7fe54006abe0 msgr2=0x7fe540148400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 -- 172.21.15.19:0/8979264 shutdown_connections 2024-04-25T09:53:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.727+0000 7fe54825d640 1 -- 172.21.15.19:0/8979264 wait complete. 2024-04-25T09:53:56.740 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.845+0000 7fb27082a640 1 Processor -- start 2024-04-25T09:53:56.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.845+0000 7fb27082a640 1 -- start start 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.845+0000 7fb27082a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb26c0cd8a0 0x7fb26c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb27082a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c15e6e0 con 0x7fb26c150b90 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb27082a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c15e9c0 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb27082a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26c15a9c0 con 0x7fb26c0cd8a0 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb26ad9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb26c150b90 0x7fb26c071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35096/0 (socket says 172.21.15.19:35096) 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb26ad9d640 1 -- 172.21.15.19:0/3641584857 learned_addr learned my addr 172.21.15.19:0/3641584857 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb263fff640 1 --2- 172.21.15.19:0/3641584857 >> v2:172.21.15.112:3300/0 conn(0x7fb26c0cd8a0 0x7fb26c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb269d9b640 1 -- 172.21.15.19:0/3641584857 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436057908 0 0) 0x7fb26c15e6e0 con 0x7fb26c150b90 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb269d9b640 1 -- 172.21.15.19:0/3641584857 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb258003520 con 0x7fb26c150b90 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb269d9b640 1 -- 172.21.15.19:0/3641584857 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984664733 0 0) 0x7fb26c15e9c0 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb269d9b640 1 -- 172.21.15.19:0/3641584857 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb26c15e6e0 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb269d9b640 1 -- 172.21.15.19:0/3641584857 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1338169299 0 0) 0x7fb258003520 con 0x7fb26c150b90 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb263fff640 1 -- 172.21.15.19:0/3641584857 >> v1:172.21.15.112:6789/0 conn(0x7fb26c0cd4c0 legacy=0x7fb26c15dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb263fff640 1 -- 172.21.15.19:0/3641584857 >> v1:172.21.15.19:6789/0 conn(0x7fb26c150b90 legacy=0x7fb26c071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.846+0000 7fb263fff640 1 -- 172.21.15.19:0/3641584857 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c15aca0 con 0x7fb26c0cd8a0 2024-04-25T09:53:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.847+0000 7fb263fff640 1 --2- 172.21.15.19:0/3641584857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb26c0cd8a0 0x7fb26c15b1c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb25400a940 tx=0x7fb25402b580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8020a7253efa8eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.847+0000 7fb269d9b640 1 -- 172.21.15.19:0/3641584857 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2640019f0 con 0x7fb26c0cd8a0 2024-04-25T09:53:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.847+0000 7fb269d9b640 1 -- 172.21.15.19:0/3641584857 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb254030510 con 0x7fb26c0cd8a0 2024-04-25T09:53:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.847+0000 7fb269d9b640 1 -- 172.21.15.19:0/3641584857 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400b040 con 0x7fb26c0cd8a0 2024-04-25T09:53:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.848+0000 7fb27082a640 1 -- 172.21.15.19:0/3641584857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb26c0cd8a0 msgr2=0x7fb26c15b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.848+0000 7fb27082a640 1 --2- 172.21.15.19:0/3641584857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb26c0cd8a0 0x7fb26c15b1c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb25400a940 tx=0x7fb25402b580 comp rx=0 tx=0).stop 2024-04-25T09:53:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.848+0000 7fb27082a640 1 -- 172.21.15.19:0/3641584857 shutdown_connections 2024-04-25T09:53:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.848+0000 7fb27082a640 1 --2- 172.21.15.19:0/3641584857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb26c0cd8a0 0x7fb26c15b1c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.848+0000 7fb27082a640 1 -- 172.21.15.19:0/3641584857 >> 172.21.15.19:0/3641584857 conn(0x7fb26c06abe0 msgr2=0x7fb26c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.848+0000 7fb27082a640 1 -- 172.21.15.19:0/3641584857 shutdown_connections 2024-04-25T09:53:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.848+0000 7fb27082a640 1 -- 172.21.15.19:0/3641584857 wait complete. 2024-04-25T09:53:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.849+0000 7fb27082a640 1 Processor -- start 2024-04-25T09:53:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.849+0000 7fb27082a640 1 -- start start 2024-04-25T09:53:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb27082a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb26c0cd4c0 0x7fb26c148430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb27082a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb26c0cd8a0 0x7fb26c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb27082a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb26c150b90 0x7fb26c14d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb27082a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c06eca0 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb27082a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c15e370 con 0x7fb26c0cd8a0 2024-04-25T09:53:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb27082a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c06ee20 con 0x7fb26c150b90 2024-04-25T09:53:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb26a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb26c0cd4c0 0x7fb26c148430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb26a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb26c0cd4c0 0x7fb26c148430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59302/0 (socket says 172.21.15.19:59302) 2024-04-25T09:53:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb26a59c640 1 -- 172.21.15.19:0/2376781513 learned_addr learned my addr 172.21.15.19:0/2376781513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb26ad9d640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb26c150b90 0x7fb26c14d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb263fff640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb26c0cd8a0 0x7fb26c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb26a59c640 1 -- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb26c150b90 msgr2=0x7fb26c14d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb26a59c640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb26c150b90 0x7fb26c14d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb26a59c640 1 -- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb26c0cd8a0 msgr2=0x7fb26c0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb26a59c640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb26c0cd8a0 0x7fb26c0b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.850+0000 7fb26a59c640 1 -- 172.21.15.19:0/2376781513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c1f1ba0 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.851+0000 7fb26a59c640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb26c0cd4c0 0x7fb26c148430 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00c490 tx=0x7fb25c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.851+0000 7fb262ffd640 1 -- 172.21.15.19:0/2376781513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c017070 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.851+0000 7fb262ffd640 1 -- 172.21.15.19:0/2376781513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c0052d0 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.851+0000 7fb27082a640 1 -- 172.21.15.19:0/2376781513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c1f1e80 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.851+0000 7fb262ffd640 1 -- 172.21.15.19:0/2376781513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c003830 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.851+0000 7fb27082a640 1 -- 172.21.15.19:0/2376781513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c1f2440 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.852+0000 7fb27082a640 1 -- 172.21.15.19:0/2376781513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c072560 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.853+0000 7fb262ffd640 1 -- 172.21.15.19:0/2376781513 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb25c01a070 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.853+0000 7fb262ffd640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb24c06b4d0 0x7fb24c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.853+0000 7fb262ffd640 1 -- 172.21.15.19:0/2376781513 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb25c089a00 con 0x7fb26c0cd4c0 2024-04-25T09:53:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.853+0000 7fb263fff640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb24c06b4d0 0x7fb24c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.854+0000 7fb263fff640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb24c06b4d0 0x7fb24c06d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb254035000 tx=0x7fb25400c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:56.857+0000 7fb262ffd640 1 -- 172.21.15.19:0/2376781513 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb26c072560 con 0x7fb26c0cd4c0 2024-04-25T09:53:57.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.037+0000 7fb27082a640 1 -- 172.21.15.19:0/2376781513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb26c073a00 con 0x7fb26c0cd4c0 2024-04-25T09:53:57.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.037+0000 7fb262ffd640 1 -- 172.21.15.19:0/2376781513 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb25c058510 con 0x7fb26c0cd4c0 2024-04-25T09:53:57.039 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:57.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 -- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb24c06b4d0 msgr2=0x7fb24c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb24c06b4d0 0x7fb24c06d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb254035000 tx=0x7fb25400c040 comp rx=0 tx=0).stop 2024-04-25T09:53:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 -- 172.21.15.19:0/2376781513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb26c0cd4c0 msgr2=0x7fb26c148430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb26c0cd4c0 0x7fb26c148430 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00c490 tx=0x7fb25c00c950 comp rx=0 tx=0).stop 2024-04-25T09:53:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 -- 172.21.15.19:0/2376781513 shutdown_connections 2024-04-25T09:53:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb24c06b4d0 0x7fb24c06d990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb26c150b90 0x7fb26c14d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb26c0cd8a0 0x7fb26c0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 --2- 172.21.15.19:0/2376781513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb26c0cd4c0 0x7fb26c148430 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 -- 172.21.15.19:0/2376781513 >> 172.21.15.19:0/2376781513 conn(0x7fb26c06abe0 msgr2=0x7fb26c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 -- 172.21.15.19:0/2376781513 shutdown_connections 2024-04-25T09:53:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.039+0000 7fb27082a640 1 -- 172.21.15.19:0/2376781513 wait complete. 2024-04-25T09:53:57.052 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:57.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.154+0000 7f1cd1676640 1 Processor -- start 2024-04-25T09:53:57.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.154+0000 7f1cd1676640 1 -- start start 2024-04-25T09:53:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1cd1676640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15dc20 0x7f1ccc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1cd1676640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1ccc0d32f0 0x7f1ccc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1cd1676640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ccc15e000 con 0x7f1ccc161710 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1cd1676640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ccc0d1de0 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1cd1676640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc161c60 con 0x7f1ccc15dc20 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1cca7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1ccc161710 0x7f1ccc06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35100/0 (socket says 172.21.15.19:35100) 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1cca7fc640 1 -- 172.21.15.19:0/1509001226 learned_addr learned my addr 172.21.15.19:0/1509001226 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1ccaffd640 1 --2- 172.21.15.19:0/1509001226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15dc20 0x7f1ccc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1ccb7fe640 1 --2- 172.21.15.19:0/1509001226 >> v2:172.21.15.112:3300/0 conn(0x7f1ccc0d32f0 0x7f1ccc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1cc9ffb640 1 -- 172.21.15.19:0/1509001226 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691906612 0 0) 0x7f1ccc15e000 con 0x7f1ccc161710 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1cc9ffb640 1 -- 172.21.15.19:0/1509001226 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb8003520 con 0x7f1ccc161710 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1ccaffd640 1 -- 172.21.15.19:0/1509001226 >> v2:172.21.15.112:3300/0 conn(0x7f1ccc0d32f0 msgr2=0x7f1ccc0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1ccaffd640 1 --2- 172.21.15.19:0/1509001226 >> v2:172.21.15.112:3300/0 conn(0x7f1ccc0d32f0 0x7f1ccc0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1ccaffd640 1 -- 172.21.15.19:0/1509001226 >> v1:172.21.15.19:6789/0 conn(0x7f1ccc161710 legacy=0x7f1ccc06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1ccaffd640 1 -- 172.21.15.19:0/1509001226 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc161f40 con 0x7f1ccc15dc20 2024-04-25T09:53:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.155+0000 7f1ccb7fe640 1 --2- 172.21.15.19:0/1509001226 >> v2:172.21.15.112:3300/0 conn(0x7f1ccc0d32f0 0x7f1ccc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.156+0000 7f1ccaffd640 1 --2- 172.21.15.19:0/1509001226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15dc20 0x7f1ccc0ce670 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0097a0 tx=0x7f1cbc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8604534673df7ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.156+0000 7f1cc9ffb640 1 -- 172.21.15.19:0/1509001226 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc037070 con 0x7f1ccc15dc20 2024-04-25T09:53:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.156+0000 7f1cc9ffb640 1 -- 172.21.15.19:0/1509001226 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc032440 con 0x7f1ccc15dc20 2024-04-25T09:53:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.156+0000 7f1cc9ffb640 1 -- 172.21.15.19:0/1509001226 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0327f0 con 0x7f1ccc15dc20 2024-04-25T09:53:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.157+0000 7f1cd1676640 1 -- 172.21.15.19:0/1509001226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15dc20 msgr2=0x7f1ccc0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.157+0000 7f1cd1676640 1 --2- 172.21.15.19:0/1509001226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15dc20 0x7f1ccc0ce670 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0097a0 tx=0x7f1cbc02a180 comp rx=0 tx=0).stop 2024-04-25T09:53:57.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.157+0000 7f1cd1676640 1 -- 172.21.15.19:0/1509001226 shutdown_connections 2024-04-25T09:53:57.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.157+0000 7f1cd1676640 1 --2- 172.21.15.19:0/1509001226 >> v2:172.21.15.112:3300/0 conn(0x7f1ccc0d32f0 0x7f1ccc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.157+0000 7f1cd1676640 1 --2- 172.21.15.19:0/1509001226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15dc20 0x7f1ccc0ce670 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.157+0000 7f1cd1676640 1 -- 172.21.15.19:0/1509001226 >> 172.21.15.19:0/1509001226 conn(0x7f1ccc0d0420 msgr2=0x7f1ccc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.157+0000 7f1cd1676640 1 -- 172.21.15.19:0/1509001226 shutdown_connections 2024-04-25T09:53:57.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.157+0000 7f1cd1676640 1 -- 172.21.15.19:0/1509001226 wait complete. 2024-04-25T09:53:57.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.158+0000 7f1cd1676640 1 Processor -- start 2024-04-25T09:53:57.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.158+0000 7f1cd1676640 1 -- start start 2024-04-25T09:53:57.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.159+0000 7f1cd1676640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ccc0d32f0 0x7f1ccc15a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.159+0000 7f1cd1676640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc15dc20 0x7f1ccc0b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.159+0000 7f1cd1676640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc161710 0x7f1ccc160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.159+0000 7f1cd1676640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0b5e50 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.159+0000 7f1cd1676640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc162500 con 0x7f1ccc15dc20 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.159+0000 7f1cd1676640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0b5fd0 con 0x7f1ccc161710 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.159+0000 7f1ccaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ccc0d32f0 0x7f1ccc15a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.159+0000 7f1ccaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ccc0d32f0 0x7f1ccc15a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59308/0 (socket says 172.21.15.19:59308) 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.159+0000 7f1ccaffd640 1 -- 172.21.15.19:0/2690306549 learned_addr learned my addr 172.21.15.19:0/2690306549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.159+0000 7f1ccaffd640 1 -- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc161710 msgr2=0x7f1ccc160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.159+0000 7f1ccb7fe640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc161710 0x7f1ccc160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.160+0000 7f1ccaffd640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc161710 0x7f1ccc160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.160+0000 7f1cca7fc640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc15dc20 0x7f1ccc0b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.160+0000 7f1ccaffd640 1 -- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc15dc20 msgr2=0x7f1ccc0b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.160+0000 7f1ccaffd640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc15dc20 0x7f1ccc0b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.160+0000 7f1ccaffd640 1 -- 172.21.15.19:0/2690306549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc1f3bb0 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.160+0000 7f1ccaffd640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ccc0d32f0 0x7f1ccc15a510 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc02a1c0 tx=0x7f1cbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.160+0000 7f1cc37fe640 1 -- 172.21.15.19:0/2690306549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc037050 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.160+0000 7f1cd1676640 1 -- 172.21.15.19:0/2690306549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ccc1f3e90 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.160+0000 7f1cc37fe640 1 -- 172.21.15.19:0/2690306549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc043070 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.160+0000 7f1cc37fe640 1 -- 172.21.15.19:0/2690306549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc032a10 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.160+0000 7f1cd1676640 1 -- 172.21.15.19:0/2690306549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ccc1f4450 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.161+0000 7f1cd1676640 1 -- 172.21.15.19:0/2690306549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ccc073a00 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.161+0000 7f1cc37fe640 1 -- 172.21.15.19:0/2690306549 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1cbc032bb0 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.162+0000 7f1cc37fe640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1cac06b4d0 0x7f1cac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.162+0000 7f1cc37fe640 1 -- 172.21.15.19:0/2690306549 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1cbc0ab3f0 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.162+0000 7f1cca7fc640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1cac06b4d0 0x7f1cac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.163+0000 7f1cca7fc640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1cac06b4d0 0x7f1cac06d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc15aa50 tx=0x7f1cb4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.166+0000 7f1cc37fe640 1 -- 172.21.15.19:0/2690306549 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1cbc079fa0 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.351+0000 7f1cd1676640 1 -- 172.21.15.19:0/2690306549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ccc0d16b0 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.351+0000 7f1cc37fe640 1 -- 172.21.15.19:0/2690306549 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1cbc003e20 con 0x7f1ccc0d32f0 2024-04-25T09:53:57.354 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 -- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1cac06b4d0 msgr2=0x7f1cac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1cac06b4d0 0x7f1cac06d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc15aa50 tx=0x7f1cb4009040 comp rx=0 tx=0).stop 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 -- 172.21.15.19:0/2690306549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ccc0d32f0 msgr2=0x7f1ccc15a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ccc0d32f0 0x7f1ccc15a510 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc02a1c0 tx=0x7f1cbc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 -- 172.21.15.19:0/2690306549 shutdown_connections 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1cac06b4d0 0x7f1cac06d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc161710 0x7f1ccc160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc15dc20 0x7f1ccc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 --2- 172.21.15.19:0/2690306549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ccc0d32f0 0x7f1ccc15a510 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 -- 172.21.15.19:0/2690306549 >> 172.21.15.19:0/2690306549 conn(0x7f1ccc0d0420 msgr2=0x7f1ccc0b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 -- 172.21.15.19:0/2690306549 shutdown_connections 2024-04-25T09:53:57.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.353+0000 7f1cd1676640 1 -- 172.21.15.19:0/2690306549 wait complete. 2024-04-25T09:53:57.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:57.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.468+0000 7fe7188bd640 1 Processor -- start 2024-04-25T09:53:57.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.468+0000 7fe7188bd640 1 -- start start 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.469+0000 7fe7188bd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7140cc4f0 0x7fe7140cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.469+0000 7fe7188bd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe71415b980 0x7fe7140cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.469+0000 7fe7188bd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe714072280 con 0x7fe7140cdb20 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.469+0000 7fe7188bd640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe714071270 con 0x7fe7140cc4f0 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.469+0000 7fe7188bd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71415bd80 con 0x7fe71415b980 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.469+0000 7fe71259c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7140cc4f0 0x7fe7140cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.469+0000 7fe71259c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7140cc4f0 0x7fe7140cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59314/0 (socket says 172.21.15.19:59314) 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.469+0000 7fe71259c640 1 -- 172.21.15.19:0/456125424 learned_addr learned my addr 172.21.15.19:0/456125424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.470+0000 7fe712d9d640 1 --2- 172.21.15.19:0/456125424 >> v2:172.21.15.112:3300/0 conn(0x7fe71415b980 0x7fe7140cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.470+0000 7fe71259c640 1 -- 172.21.15.19:0/456125424 >> v2:172.21.15.112:3300/0 conn(0x7fe71415b980 msgr2=0x7fe7140cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.470+0000 7fe71259c640 1 --2- 172.21.15.19:0/456125424 >> v2:172.21.15.112:3300/0 conn(0x7fe71415b980 0x7fe7140cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.470+0000 7fe71259c640 1 -- 172.21.15.19:0/456125424 >> v1:172.21.15.112:6789/0 conn(0x7fe7140cdb20 legacy=0x7fe71415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.470+0000 7fe71259c640 1 -- 172.21.15.19:0/456125424 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7140714c0 con 0x7fe7140cc4f0 2024-04-25T09:53:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.470+0000 7fe71259c640 1 --2- 172.21.15.19:0/456125424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7140cc4f0 0x7fe7140cc8d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe7140b4c00 tx=0x7fe70c02c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128b41291e992586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.471+0000 7fe71159a640 1 -- 172.21.15.19:0/456125424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c036020 con 0x7fe7140cc4f0 2024-04-25T09:53:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.471+0000 7fe71159a640 1 -- 172.21.15.19:0/456125424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c032640 con 0x7fe7140cc4f0 2024-04-25T09:53:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.471+0000 7fe71159a640 1 -- 172.21.15.19:0/456125424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c032a10 con 0x7fe7140cc4f0 2024-04-25T09:53:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.471+0000 7fe7188bd640 1 -- 172.21.15.19:0/456125424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7140cc4f0 msgr2=0x7fe7140cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.471+0000 7fe7188bd640 1 --2- 172.21.15.19:0/456125424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7140cc4f0 0x7fe7140cc8d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe7140b4c00 tx=0x7fe70c02c940 comp rx=0 tx=0).stop 2024-04-25T09:53:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.471+0000 7fe7188bd640 1 -- 172.21.15.19:0/456125424 shutdown_connections 2024-04-25T09:53:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.471+0000 7fe7188bd640 1 --2- 172.21.15.19:0/456125424 >> v2:172.21.15.112:3300/0 conn(0x7fe71415b980 0x7fe7140cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.471+0000 7fe7188bd640 1 --2- 172.21.15.19:0/456125424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7140cc4f0 0x7fe7140cc8d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.471+0000 7fe7188bd640 1 -- 172.21.15.19:0/456125424 >> 172.21.15.19:0/456125424 conn(0x7fe71406abe0 msgr2=0x7fe71406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.471+0000 7fe7188bd640 1 -- 172.21.15.19:0/456125424 shutdown_connections 2024-04-25T09:53:57.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.471+0000 7fe7188bd640 1 -- 172.21.15.19:0/456125424 wait complete. 2024-04-25T09:53:57.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.473+0000 7fe7188bd640 1 Processor -- start 2024-04-25T09:53:57.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.473+0000 7fe7188bd640 1 -- start start 2024-04-25T09:53:57.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.473+0000 7fe7188bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7140cc4f0 0x7fe7141f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.473+0000 7fe7188bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7140cdb20 0x7fe714150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.473+0000 7fe7188bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe71415b980 0x7fe71415d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.473+0000 7fe7188bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe714071d60 con 0x7fe7140cc4f0 2024-04-25T09:53:57.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.473+0000 7fe7188bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe714071a80 con 0x7fe7140cdb20 2024-04-25T09:53:57.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.473+0000 7fe7188bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7140ae500 con 0x7fe71415b980 2024-04-25T09:53:57.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.473+0000 7fe71259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7140cc4f0 0x7fe7141f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.473+0000 7fe71259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7140cc4f0 0x7fe7141f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59328/0 (socket says 172.21.15.19:59328) 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe71259c640 1 -- 172.21.15.19:0/445089226 learned_addr learned my addr 172.21.15.19:0/445089226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe712d9d640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe71415b980 0x7fe71415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe711d9b640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7140cdb20 0x7fe714150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe71259c640 1 -- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe71415b980 msgr2=0x7fe71415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe71259c640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe71415b980 0x7fe71415d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe71259c640 1 -- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7140cdb20 msgr2=0x7fe714150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe71259c640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7140cdb20 0x7fe714150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe71259c640 1 -- 172.21.15.19:0/445089226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7141f38a0 con 0x7fe7140cc4f0 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe712d9d640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe71415b980 0x7fe71415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe711d9b640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7140cdb20 0x7fe714150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe71259c640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7140cc4f0 0x7fe7141f78c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe70c02c980 tx=0x7fe70c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe702ffd640 1 -- 172.21.15.19:0/445089226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c042070 con 0x7fe7140cc4f0 2024-04-25T09:53:57.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe702ffd640 1 -- 172.21.15.19:0/445089226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c031b50 con 0x7fe7140cc4f0 2024-04-25T09:53:57.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe7188bd640 1 -- 172.21.15.19:0/445089226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe714150ea0 con 0x7fe7140cc4f0 2024-04-25T09:53:57.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe702ffd640 1 -- 172.21.15.19:0/445089226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c036040 con 0x7fe7140cc4f0 2024-04-25T09:53:57.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.474+0000 7fe7188bd640 1 -- 172.21.15.19:0/445089226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7141f3e50 con 0x7fe7140cc4f0 2024-04-25T09:53:57.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.475+0000 7fe7188bd640 1 -- 172.21.15.19:0/445089226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe714073a00 con 0x7fe7140cc4f0 2024-04-25T09:53:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.476+0000 7fe702ffd640 1 -- 172.21.15.19:0/445089226 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe70c045070 con 0x7fe7140cc4f0 2024-04-25T09:53:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.476+0000 7fe702ffd640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6f406b4d0 0x7fe6f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.476+0000 7fe702ffd640 1 -- 172.21.15.19:0/445089226 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe70c0aaf60 con 0x7fe7140cc4f0 2024-04-25T09:53:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.476+0000 7fe711d9b640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6f406b4d0 0x7fe6f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.477+0000 7fe711d9b640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6f406b4d0 0x7fe6f406d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe71415dcd0 tx=0x7fe708009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.480+0000 7fe702ffd640 1 -- 172.21.15.19:0/445089226 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe70c079b00 con 0x7fe7140cc4f0 2024-04-25T09:53:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.659+0000 7fe7188bd640 1 -- 172.21.15.19:0/445089226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe71406d3a0 con 0x7fe7140cc4f0 2024-04-25T09:53:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.659+0000 7fe702ffd640 1 -- 172.21.15.19:0/445089226 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe70c079400 con 0x7fe7140cc4f0 2024-04-25T09:53:57.663 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 -- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6f406b4d0 msgr2=0x7fe6f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6f406b4d0 0x7fe6f406d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe71415dcd0 tx=0x7fe708009040 comp rx=0 tx=0).stop 2024-04-25T09:53:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 -- 172.21.15.19:0/445089226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7140cc4f0 msgr2=0x7fe7141f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7140cc4f0 0x7fe7141f78c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe70c02c980 tx=0x7fe70c004730 comp rx=0 tx=0).stop 2024-04-25T09:53:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 -- 172.21.15.19:0/445089226 shutdown_connections 2024-04-25T09:53:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6f406b4d0 0x7fe6f406d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe71415b980 0x7fe71415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7140cdb20 0x7fe714150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 --2- 172.21.15.19:0/445089226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7140cc4f0 0x7fe7141f78c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 -- 172.21.15.19:0/445089226 >> 172.21.15.19:0/445089226 conn(0x7fe71406abe0 msgr2=0x7fe7140cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 -- 172.21.15.19:0/445089226 shutdown_connections 2024-04-25T09:53:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.661+0000 7fe7188bd640 1 -- 172.21.15.19:0/445089226 wait complete. 2024-04-25T09:53:57.674 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:53:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.773+0000 7f2aaefcf640 1 Processor -- start 2024-04-25T09:53:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.773+0000 7f2aaefcf640 1 -- start start 2024-04-25T09:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.774+0000 7f2aaefcf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa815dd80 0x7f2aa80b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.774+0000 7f2aaefcf640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa80b3930 con 0x7f2aa80ad570 2024-04-25T09:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.774+0000 7f2aaefcf640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa80b3c10 con 0x7f2aa806abe0 2024-04-25T09:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.774+0000 7f2aaefcf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa806e410 con 0x7f2aa815dd80 2024-04-25T09:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.774+0000 7f2aad56c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2aa80ad570 0x7f2aa806ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48286/0 (socket says 172.21.15.19:48286) 2024-04-25T09:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.774+0000 7f2aad56c640 1 -- 172.21.15.19:0/2239369860 learned_addr learned my addr 172.21.15.19:0/2239369860 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.774+0000 7f2aa7fff640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa815dd80 0x7f2aa80b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.774+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618847437 0 0) 0x7f2aa80b3930 con 0x7f2aa80ad570 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a90003520 con 0x7f2aa80ad570 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1012810094 0 0) 0x7f2a90003520 con 0x7f2aa80ad570 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa815dd80 msgr2=0x7f2aa80b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa815dd80 0x7f2aa80b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 >> v1:172.21.15.112:6789/0 conn(0x7f2aa806abe0 legacy=0x7f2aa806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa806e660 con 0x7f2aa80ad570 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f2aa00019f0 con 0x7f2aa80ad570 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 >> v1:172.21.15.19:6789/0 conn(0x7f2aa80ad570 legacy=0x7f2aa806ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a900039e0 0x7f2a90003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa815dd80 0x7f2a90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a90004d40 0x7f2a900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa806e410 con 0x7f2a90004d40 2024-04-25T09:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9000b9c0 con 0x7f2a900039e0 2024-04-25T09:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9000bbc0 con 0x7f2aa815dd80 2024-04-25T09:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aad56c640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a90004d40 0x7f2a900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aacd6b640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a900039e0 0x7f2a90003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aa7fff640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa815dd80 0x7f2a90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aad56c640 1 -- 172.21.15.19:0/2239369860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa815dd80 msgr2=0x7f2a90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aad56c640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa815dd80 0x7f2a90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aad56c640 1 -- 172.21.15.19:0/2239369860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a900039e0 msgr2=0x7f2a90003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aad56c640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a900039e0 0x7f2a90003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.775+0000 7f2aad56c640 1 -- 172.21.15.19:0/2239369860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a90003520 con 0x7f2a90004d40 2024-04-25T09:53:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.776+0000 7f2aad56c640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a90004d40 0x7f2a900051c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000a920 tx=0x7f2a90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d29b1a2a427de5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.776+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000a4c0 con 0x7f2a90004d40 2024-04-25T09:53:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.776+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa002f070 con 0x7f2a90004d40 2024-04-25T09:53:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.776+0000 7f2aa77fe640 1 -- 172.21.15.19:0/2239369860 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa002bce0 con 0x7f2a90004d40 2024-04-25T09:53:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.777+0000 7f2aaefcf640 1 -- 172.21.15.19:0/2239369860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a90004d40 msgr2=0x7f2a900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.777+0000 7f2aaefcf640 1 --2- 172.21.15.19:0/2239369860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a90004d40 0x7f2a900051c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000a920 tx=0x7f2a90005830 comp rx=0 tx=0).stop 2024-04-25T09:53:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.777+0000 7f2aad56c640 1 -- 172.21.15.19:0/2239369860 reap_dead start 2024-04-25T09:53:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.777+0000 7f2aaefcf640 1 -- 172.21.15.19:0/2239369860 shutdown_connections 2024-04-25T09:53:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.777+0000 7f2aaefcf640 1 -- 172.21.15.19:0/2239369860 >> 172.21.15.19:0/2239369860 conn(0x7f2aa80b5af0 msgr2=0x7f2aa80b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.777+0000 7f2aaefcf640 1 -- 172.21.15.19:0/2239369860 shutdown_connections 2024-04-25T09:53:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.777+0000 7f2aaefcf640 1 -- 172.21.15.19:0/2239369860 wait complete. 2024-04-25T09:53:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.778+0000 7f2aaefcf640 1 Processor -- start 2024-04-25T09:53:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.778+0000 7f2aaefcf640 1 -- start start 2024-04-25T09:53:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aaefcf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa81589f0 0x7f2aa8147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aaefcf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa8147aa0 0x7f2aa80b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aaefcf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aa80b0670 0x7f2aa8148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aaefcf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa8070400 con 0x7f2aa80b0670 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aaefcf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa8070280 con 0x7f2aa81589f0 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aaefcf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80b3590 con 0x7f2aa8147aa0 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aad56c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aa80b0670 0x7f2aa8148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aad56c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aa80b0670 0x7f2aa8148be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44022/0 (socket says 172.21.15.19:44022) 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aad56c640 1 -- 172.21.15.19:0/3294892480 learned_addr learned my addr 172.21.15.19:0/3294892480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aa7fff640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa8147aa0 0x7f2aa80b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aacd6b640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa81589f0 0x7f2aa8147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aad56c640 1 -- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa8147aa0 msgr2=0x7f2aa80b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aad56c640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa8147aa0 0x7f2aa80b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aad56c640 1 -- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa81589f0 msgr2=0x7f2aa8147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aad56c640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa81589f0 0x7f2aa8147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aad56c640 1 -- 172.21.15.19:0/3294892480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa814a040 con 0x7f2aa80b0670 2024-04-25T09:53:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.779+0000 7f2aacd6b640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa81589f0 0x7f2aa8147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.780+0000 7f2aad56c640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aa80b0670 0x7f2aa8148be0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000aa40 tx=0x7f2aa002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.780+0000 7f2aa57fa640 1 -- 172.21.15.19:0/3294892480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000a4c0 con 0x7f2aa80b0670 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.780+0000 7f2aa57fa640 1 -- 172.21.15.19:0/3294892480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa00019e0 con 0x7f2aa80b0670 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.780+0000 7f2aaefcf640 1 -- 172.21.15.19:0/3294892480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa8147fb0 con 0x7f2aa80b0670 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.780+0000 7f2aa57fa640 1 -- 172.21.15.19:0/3294892480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0034690 con 0x7f2aa80b0670 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.780+0000 7f2aaefcf640 1 -- 172.21.15.19:0/3294892480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa8148570 con 0x7f2aa80b0670 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.781+0000 7f2aaefcf640 1 -- 172.21.15.19:0/3294892480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa8073a00 con 0x7f2aa80b0670 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.781+0000 7f2aa57fa640 1 -- 172.21.15.19:0/3294892480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2aa0034830 con 0x7f2aa80b0670 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.782+0000 7f2aa57fa640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a9406b4d0 0x7f2a9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.782+0000 7f2aa57fa640 1 -- 172.21.15.19:0/3294892480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2aa00aa270 con 0x7f2aa80b0670 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.782+0000 7f2aacd6b640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a9406b4d0 0x7f2a9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.783+0000 7f2aacd6b640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a9406b4d0 0x7f2a9406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2aa806cbb0 tx=0x7f2a98009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.786+0000 7f2aa57fa640 1 -- 172.21.15.19:0/3294892480 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2aa0046060 con 0x7f2aa80b0670 2024-04-25T09:53:57.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.977+0000 7f2aaefcf640 1 -- 172.21.15.19:0/3294892480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2aa806f790 con 0x7f2aa80b0670 2024-04-25T09:53:57.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.977+0000 7f2aa57fa640 1 -- 172.21.15.19:0/3294892480 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2aa0078db0 con 0x7f2aa80b0670 2024-04-25T09:53:57.980 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T09:53:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 -- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a9406b4d0 msgr2=0x7f2a9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a9406b4d0 0x7f2a9406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2aa806cbb0 tx=0x7f2a98009040 comp rx=0 tx=0).stop 2024-04-25T09:53:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 -- 172.21.15.19:0/3294892480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aa80b0670 msgr2=0x7f2aa8148be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aa80b0670 0x7f2aa8148be0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000aa40 tx=0x7f2aa002d070 comp rx=0 tx=0).stop 2024-04-25T09:53:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 -- 172.21.15.19:0/3294892480 shutdown_connections 2024-04-25T09:53:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a9406b4d0 0x7f2a9406d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aa80b0670 0x7f2aa8148be0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2aa8147aa0 0x7f2aa80b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 --2- 172.21.15.19:0/3294892480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa81589f0 0x7f2aa8147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 -- 172.21.15.19:0/3294892480 >> 172.21.15.19:0/3294892480 conn(0x7f2aa80b5af0 msgr2=0x7f2aa815c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 -- 172.21.15.19:0/3294892480 shutdown_connections 2024-04-25T09:53:57.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:57.979+0000 7f2aaefcf640 1 -- 172.21.15.19:0/3294892480 wait complete. 2024-04-25T09:53:57.992 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T09:53:57.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:57.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T09:53:58.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T09:53:58.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:58.024 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:53:58.082 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:53:58.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.182+0000 7f6ec9ff9640 1 Processor -- start 2024-04-25T09:53:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.182+0000 7f6ec9ff9640 1 -- start start 2024-04-25T09:53:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.183+0000 7f6ec9ff9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ec415a790 0x7f6ec40d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.183+0000 7f6ec9ff9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6ec40b4eb0 0x7f6ec40d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.183+0000 7f6ec9ff9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec40b3710 con 0x7f6ec406fea0 2024-04-25T09:53:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.183+0000 7f6ec9ff9640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec40cf0e0 con 0x7f6ec415a790 2024-04-25T09:53:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.183+0000 7f6ec9ff9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec40cf3c0 con 0x7f6ec40b4eb0 2024-04-25T09:53:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.183+0000 7f6ec37fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ec415a790 0x7f6ec40d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.183+0000 7f6ec37fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ec415a790 0x7f6ec40d0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44026/0 (socket says 172.21.15.19:44026) 2024-04-25T09:53:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.183+0000 7f6ec37fe640 1 -- 172.21.15.19:0/2208465306 learned_addr learned my addr 172.21.15.19:0/2208465306 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.184+0000 7f6ec3fff640 1 --2- 172.21.15.19:0/2208465306 >> v2:172.21.15.112:3300/0 conn(0x7f6ec40b4eb0 0x7f6ec40d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.184+0000 7f6ec37fe640 1 -- 172.21.15.19:0/2208465306 >> v2:172.21.15.112:3300/0 conn(0x7f6ec40b4eb0 msgr2=0x7f6ec40d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.184+0000 7f6ec37fe640 1 --2- 172.21.15.19:0/2208465306 >> v2:172.21.15.112:3300/0 conn(0x7f6ec40b4eb0 0x7f6ec40d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.184+0000 7f6ec37fe640 1 -- 172.21.15.19:0/2208465306 >> v1:172.21.15.112:6789/0 conn(0x7f6ec406fea0 legacy=0x7f6ec4072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.184+0000 7f6ec37fe640 1 -- 172.21.15.19:0/2208465306 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec40cf6a0 con 0x7f6ec415a790 2024-04-25T09:53:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec37fe640 1 --2- 172.21.15.19:0/2208465306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec415a790 0x7f6ec40d0390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40b3410 tx=0x7f6eb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7deecf9a1b45db95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec27fc640 1 -- 172.21.15.19:0/2208465306 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4037070 con 0x7f6ec415a790 2024-04-25T09:53:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec27fc640 1 -- 172.21.15.19:0/2208465306 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4032440 con 0x7f6ec415a790 2024-04-25T09:53:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec27fc640 1 -- 172.21.15.19:0/2208465306 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4032810 con 0x7f6ec415a790 2024-04-25T09:53:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/2208465306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec415a790 msgr2=0x7f6ec40d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec9ff9640 1 --2- 172.21.15.19:0/2208465306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec415a790 0x7f6ec40d0390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40b3410 tx=0x7f6eb402a270 comp rx=0 tx=0).stop 2024-04-25T09:53:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/2208465306 shutdown_connections 2024-04-25T09:53:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec9ff9640 1 --2- 172.21.15.19:0/2208465306 >> v2:172.21.15.112:3300/0 conn(0x7f6ec40b4eb0 0x7f6ec40d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec9ff9640 1 --2- 172.21.15.19:0/2208465306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec415a790 0x7f6ec40d0390 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/2208465306 >> 172.21.15.19:0/2208465306 conn(0x7f6ec40b2370 msgr2=0x7f6ec40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/2208465306 shutdown_connections 2024-04-25T09:53:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.185+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/2208465306 wait complete. 2024-04-25T09:53:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.186+0000 7f6ec9ff9640 1 Processor -- start 2024-04-25T09:53:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.187+0000 7f6ec9ff9640 1 -- start start 2024-04-25T09:53:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.187+0000 7f6ec9ff9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec406fea0 0x7f6ec4155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.187+0000 7f6ec9ff9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ec40b4eb0 0x7f6ec414b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.187+0000 7f6ec9ff9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ec415a790 0x7f6ec4151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.187+0000 7f6ec9ff9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec415ab70 con 0x7f6ec406fea0 2024-04-25T09:53:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.187+0000 7f6ec9ff9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40cf980 con 0x7f6ec40b4eb0 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.187+0000 7f6ec9ff9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec415acf0 con 0x7f6ec415a790 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.187+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec406fea0 0x7f6ec4155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.187+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec406fea0 0x7f6ec4155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44038/0 (socket says 172.21.15.19:44038) 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6ec37fe640 1 -- 172.21.15.19:0/4150455544 learned_addr learned my addr 172.21.15.19:0/4150455544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6ec3fff640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ec415a790 0x7f6ec4151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6ec2ffd640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ec40b4eb0 0x7f6ec414b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6ec37fe640 1 -- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ec415a790 msgr2=0x7f6ec4151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6ec37fe640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ec415a790 0x7f6ec4151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6ec37fe640 1 -- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ec40b4eb0 msgr2=0x7f6ec414b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6ec37fe640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ec40b4eb0 0x7f6ec414b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6ec37fe640 1 -- 172.21.15.19:0/4150455544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec4153f80 con 0x7f6ec406fea0 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6ec37fe640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec406fea0 0x7f6ec4155020 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402a2b0 tx=0x7f6eb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6eabfff640 1 -- 172.21.15.19:0/4150455544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4037040 con 0x7f6ec406fea0 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6eabfff640 1 -- 172.21.15.19:0/4150455544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4042070 con 0x7f6ec406fea0 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6eabfff640 1 -- 172.21.15.19:0/4150455544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4030880 con 0x7f6ec406fea0 2024-04-25T09:53:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/4150455544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec4154260 con 0x7f6ec406fea0 2024-04-25T09:53:58.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.188+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/4150455544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec4154820 con 0x7f6ec406fea0 2024-04-25T09:53:58.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.189+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/4150455544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec4073990 con 0x7f6ec406fea0 2024-04-25T09:53:58.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.190+0000 7f6eabfff640 1 -- 172.21.15.19:0/4150455544 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6eb4030a20 con 0x7f6ec406fea0 2024-04-25T09:53:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.190+0000 7f6eabfff640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eb006b4d0 0x7f6eb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.190+0000 7f6eabfff640 1 -- 172.21.15.19:0/4150455544 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6eb40aae60 con 0x7f6ec406fea0 2024-04-25T09:53:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.190+0000 7f6ec2ffd640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eb006b4d0 0x7f6eb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.191+0000 7f6ec2ffd640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eb006b4d0 0x7f6eb006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4157f60 tx=0x7f6eb8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.194+0000 7f6eabfff640 1 -- 172.21.15.19:0/4150455544 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6eb40798e0 con 0x7f6ec406fea0 2024-04-25T09:53:58.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.413+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/4150455544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ec40ccbf0 con 0x7f6ec406fea0 2024-04-25T09:53:58.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.413+0000 7f6eabfff640 1 -- 172.21.15.19:0/4150455544 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f6eb40791e0 con 0x7f6ec406fea0 2024-04-25T09:53:58.415 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:58.415 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4000846111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":4000846111},{"type":"v1","addr":"172.21.15.112:6837","nonce":4000846111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1969525282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1969525282},{"type":"v1","addr":"172.21.15.19:6835","nonce":1969525282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:07.175350+0000","modified":"2024-04-25T09:53:23.345595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1080352556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1080352556},{"type":"v1","addr":"172.21.15.112:6836","nonce":1080352556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:53:58.416 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T09:53:58.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eb006b4d0 msgr2=0x7f6eb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eb006b4d0 0x7f6eb006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4157f60 tx=0x7f6eb8009040 comp rx=0 tx=0).stop 2024-04-25T09:53:58.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/4150455544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec406fea0 msgr2=0x7f6ec4155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec406fea0 0x7f6ec4155020 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402a2b0 tx=0x7f6eb40046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/4150455544 shutdown_connections 2024-04-25T09:53:58.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eb006b4d0 0x7f6eb006d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ec415a790 0x7f6ec4151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ec40b4eb0 0x7f6ec414b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 --2- 172.21.15.19:0/4150455544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec406fea0 0x7f6ec4155020 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/4150455544 >> 172.21.15.19:0/4150455544 conn(0x7f6ec40b2370 msgr2=0x7f6ec406b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:58.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/4150455544 shutdown_connections 2024-04-25T09:53:58.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.415+0000 7f6ec9ff9640 1 -- 172.21.15.19:0/4150455544 wait complete. 2024-04-25T09:53:58.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:58.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.526+0000 7f792089e640 1 Processor -- start 2024-04-25T09:53:58.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.526+0000 7f792089e640 1 -- start start 2024-04-25T09:53:58.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.527+0000 7f792089e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f791c0d1120 0x7f791c09a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.527+0000 7f792089e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791c06c220 con 0x7f791c06ce10 2024-04-25T09:53:58.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.527+0000 7f792089e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791c0d0080 con 0x7f791c15dc70 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.527+0000 7f792089e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0d0c90 con 0x7f791c0d1120 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.527+0000 7f7919d9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f791c06ce10 0x7f791c0cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48302/0 (socket says 172.21.15.19:48302) 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.527+0000 7f7919d9b640 1 -- 172.21.15.19:0/2824792777 learned_addr learned my addr 172.21.15.19:0/2824792777 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.528+0000 7f791ad9d640 1 --2- 172.21.15.19:0/2824792777 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f791c0d1120 0x7f791c09a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.528+0000 7f791959a640 1 -- 172.21.15.19:0/2824792777 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013070260 0 0) 0x7f791c06c220 con 0x7f791c06ce10 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.528+0000 7f791959a640 1 -- 172.21.15.19:0/2824792777 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7908003520 con 0x7f791c06ce10 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.528+0000 7f791959a640 1 -- 172.21.15.19:0/2824792777 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943877642 0 0) 0x7f791c0d0080 con 0x7f791c15dc70 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.528+0000 7f791959a640 1 -- 172.21.15.19:0/2824792777 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f791c06c220 con 0x7f791c15dc70 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.528+0000 7f791ad9d640 1 -- 172.21.15.19:0/2824792777 >> v1:172.21.15.112:6789/0 conn(0x7f791c15dc70 legacy=0x7f791c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.528+0000 7f791ad9d640 1 -- 172.21.15.19:0/2824792777 >> v1:172.21.15.19:6789/0 conn(0x7f791c06ce10 legacy=0x7f791c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.528+0000 7f791959a640 1 -- 172.21.15.19:0/2824792777 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2021662121 0 0) 0x7f7908003520 con 0x7f791c06ce10 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.528+0000 7f791ad9d640 1 -- 172.21.15.19:0/2824792777 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c0d0360 con 0x7f791c0d1120 2024-04-25T09:53:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.529+0000 7f791ad9d640 1 --2- 172.21.15.19:0/2824792777 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f791c0d1120 0x7f791c09a450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f790c009dc0 tx=0x7f790c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d6129cb567aefc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.529+0000 7f791959a640 1 -- 172.21.15.19:0/2824792777 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904001670 con 0x7f791c0d1120 2024-04-25T09:53:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.529+0000 7f791959a640 1 -- 172.21.15.19:0/2824792777 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c007540 con 0x7f791c0d1120 2024-04-25T09:53:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.529+0000 7f791959a640 1 -- 172.21.15.19:0/2824792777 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c007910 con 0x7f791c0d1120 2024-04-25T09:53:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.529+0000 7f792089e640 1 -- 172.21.15.19:0/2824792777 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f791c0d1120 msgr2=0x7f791c09a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.529+0000 7f792089e640 1 --2- 172.21.15.19:0/2824792777 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f791c0d1120 0x7f791c09a450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f790c009dc0 tx=0x7f790c02a660 comp rx=0 tx=0).stop 2024-04-25T09:53:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.530+0000 7f792089e640 1 -- 172.21.15.19:0/2824792777 shutdown_connections 2024-04-25T09:53:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.530+0000 7f792089e640 1 --2- 172.21.15.19:0/2824792777 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f791c0d1120 0x7f791c09a450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.530+0000 7f792089e640 1 -- 172.21.15.19:0/2824792777 >> 172.21.15.19:0/2824792777 conn(0x7f791c0b3270 msgr2=0x7f791c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.530+0000 7f792089e640 1 -- 172.21.15.19:0/2824792777 shutdown_connections 2024-04-25T09:53:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.530+0000 7f792089e640 1 -- 172.21.15.19:0/2824792777 wait complete. 2024-04-25T09:53:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.531+0000 7f792089e640 1 Processor -- start 2024-04-25T09:53:58.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.531+0000 7f792089e640 1 -- start start 2024-04-25T09:53:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f792089e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f791c06ce10 0x7f791c158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f792089e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f791c0d1120 0x7f791c0c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f792089e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c15dc70 0x7f791c0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f792089e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c15e050 con 0x7f791c15dc70 2024-04-25T09:53:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f792089e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0d0920 con 0x7f791c0d1120 2024-04-25T09:53:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f792089e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0cf9c0 con 0x7f791c06ce10 2024-04-25T09:53:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f791ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c15dc70 0x7f791c0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f791ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c15dc70 0x7f791c0c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44048/0 (socket says 172.21.15.19:44048) 2024-04-25T09:53:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f791ad9d640 1 -- 172.21.15.19:0/2584926445 learned_addr learned my addr 172.21.15.19:0/2584926445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f791a59c640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f791c06ce10 0x7f791c158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f7919d9b640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f791c0d1120 0x7f791c0c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f791ad9d640 1 -- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f791c06ce10 msgr2=0x7f791c158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f791ad9d640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f791c06ce10 0x7f791c158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f791ad9d640 1 -- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f791c0d1120 msgr2=0x7f791c0c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f791ad9d640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f791c0d1120 0x7f791c0c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f791ad9d640 1 -- 172.21.15.19:0/2584926445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c0c62a0 con 0x7f791c15dc70 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f791ad9d640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c15dc70 0x7f791c0c6710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f790c009ee0 tx=0x7f790c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f7912ffd640 1 -- 172.21.15.19:0/2584926445 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c040070 con 0x7f791c15dc70 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f7912ffd640 1 -- 172.21.15.19:0/2584926445 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c03c830 con 0x7f791c15dc70 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.532+0000 7f792089e640 1 -- 172.21.15.19:0/2584926445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791c159160 con 0x7f791c15dc70 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.533+0000 7f7912ffd640 1 -- 172.21.15.19:0/2584926445 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c03cbe0 con 0x7f791c15dc70 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.533+0000 7f792089e640 1 -- 172.21.15.19:0/2584926445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791c0c5fc0 con 0x7f791c15dc70 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.533+0000 7f792089e640 1 -- 172.21.15.19:0/2584926445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791c073990 con 0x7f791c15dc70 2024-04-25T09:53:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.534+0000 7f7912ffd640 1 -- 172.21.15.19:0/2584926445 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f790c039040 con 0x7f791c15dc70 2024-04-25T09:53:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.534+0000 7f7912ffd640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78fc06b4d0 0x7f78fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.535+0000 7f7912ffd640 1 -- 172.21.15.19:0/2584926445 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f790c0abb40 con 0x7f791c15dc70 2024-04-25T09:53:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.535+0000 7f791a59c640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78fc06b4d0 0x7f78fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.535+0000 7f791a59c640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78fc06b4d0 0x7f78fc06d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f79140011e0 tx=0x7f7914009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.539+0000 7f7912ffd640 1 -- 172.21.15.19:0/2584926445 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f790c07a090 con 0x7f791c15dc70 2024-04-25T09:53:58.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.718+0000 7f792089e640 1 -- 172.21.15.19:0/2584926445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f791c06e3e0 con 0x7f791c15dc70 2024-04-25T09:53:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.719+0000 7f7912ffd640 1 -- 172.21.15.19:0/2584926445 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18892 (secure 0 0 0) 0x7f790c07e130 con 0x7f791c15dc70 2024-04-25T09:53:58.721 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:53:58.722 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:48.071585+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 -- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78fc06b4d0 msgr2=0x7f78fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78fc06b4d0 0x7f78fc06d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f79140011e0 tx=0x7f7914009040 comp rx=0 tx=0).stop 2024-04-25T09:53:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 -- 172.21.15.19:0/2584926445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c15dc70 msgr2=0x7f791c0c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c15dc70 0x7f791c0c6710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f790c009ee0 tx=0x7f790c004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 -- 172.21.15.19:0/2584926445 shutdown_connections 2024-04-25T09:53:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78fc06b4d0 0x7f78fc06d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c15dc70 0x7f791c0c6710 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f791c0d1120 0x7f791c0c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 --2- 172.21.15.19:0/2584926445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f791c06ce10 0x7f791c158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 -- 172.21.15.19:0/2584926445 >> 172.21.15.19:0/2584926445 conn(0x7f791c0b3270 msgr2=0x7f791c0cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 -- 172.21.15.19:0/2584926445 shutdown_connections 2024-04-25T09:53:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.721+0000 7f792089e640 1 -- 172.21.15.19:0/2584926445 wait complete. 2024-04-25T09:53:58.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:05.053469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:06.115537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:09.831285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:58.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.836+0000 7fc41bb70640 1 Processor -- start 2024-04-25T09:53:58.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.837+0000 7fc41bb70640 1 -- start start 2024-04-25T09:53:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41bb70640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4140ced80 0x7fc4140d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41bb70640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc414161710 0x7fc4140d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41bb70640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41415e000 con 0x7fc41415dc20 2024-04-25T09:53:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41bb70640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4140d1de0 con 0x7fc414161710 2024-04-25T09:53:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41bb70640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc414161c60 con 0x7fc4140ced80 2024-04-25T09:53:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41a10d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc414161710 0x7fc4140d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41a10d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc414161710 0x7fc4140d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44064/0 (socket says 172.21.15.19:44064) 2024-04-25T09:53:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41a10d640 1 -- 172.21.15.19:0/2574572263 learned_addr learned my addr 172.21.15.19:0/2574572263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41910b640 1 --2- 172.21.15.19:0/2574572263 >> v2:172.21.15.112:3300/0 conn(0x7fc4140ced80 0x7fc4140d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41a10d640 1 -- 172.21.15.19:0/2574572263 >> v2:172.21.15.112:3300/0 conn(0x7fc4140ced80 msgr2=0x7fc4140d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41a10d640 1 --2- 172.21.15.19:0/2574572263 >> v2:172.21.15.112:3300/0 conn(0x7fc4140ced80 0x7fc4140d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41a10d640 1 -- 172.21.15.19:0/2574572263 >> v1:172.21.15.112:6789/0 conn(0x7fc41415dc20 legacy=0x7fc4140ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.838+0000 7fc41a10d640 1 -- 172.21.15.19:0/2574572263 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414161f40 con 0x7fc414161710 2024-04-25T09:53:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.839+0000 7fc41a10d640 1 --2- 172.21.15.19:0/2574572263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc414161710 0x7fc4140d2820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc41406da90 tx=0x7fc40c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d9ec66f6f75fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.839+0000 7fc41890a640 1 -- 172.21.15.19:0/2574572263 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c036070 con 0x7fc414161710 2024-04-25T09:53:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.839+0000 7fc41890a640 1 -- 172.21.15.19:0/2574572263 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c031600 con 0x7fc414161710 2024-04-25T09:53:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.839+0000 7fc41890a640 1 -- 172.21.15.19:0/2574572263 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c0319d0 con 0x7fc414161710 2024-04-25T09:53:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.839+0000 7fc41bb70640 1 -- 172.21.15.19:0/2574572263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc414161710 msgr2=0x7fc4140d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.839+0000 7fc41bb70640 1 --2- 172.21.15.19:0/2574572263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc414161710 0x7fc4140d2820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc41406da90 tx=0x7fc40c02a730 comp rx=0 tx=0).stop 2024-04-25T09:53:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.840+0000 7fc41bb70640 1 -- 172.21.15.19:0/2574572263 shutdown_connections 2024-04-25T09:53:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.840+0000 7fc41bb70640 1 --2- 172.21.15.19:0/2574572263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc414161710 0x7fc4140d2820 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.840+0000 7fc41bb70640 1 --2- 172.21.15.19:0/2574572263 >> v2:172.21.15.112:3300/0 conn(0x7fc4140ced80 0x7fc4140d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.840+0000 7fc41bb70640 1 -- 172.21.15.19:0/2574572263 >> 172.21.15.19:0/2574572263 conn(0x7fc4140d0420 msgr2=0x7fc4140d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.840+0000 7fc41bb70640 1 -- 172.21.15.19:0/2574572263 shutdown_connections 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.840+0000 7fc41bb70640 1 -- 172.21.15.19:0/2574572263 wait complete. 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.841+0000 7fc41bb70640 1 Processor -- start 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.841+0000 7fc41bb70640 1 -- start start 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.841+0000 7fc41bb70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4140ced80 0x7fc4140c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.841+0000 7fc41bb70640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc41415dc20 0x7fc4140c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41bb70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc414161710 0x7fc414152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41bb70640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140b5e50 con 0x7fc4140ced80 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41bb70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc414162500 con 0x7fc414161710 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41bb70640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140b5fd0 con 0x7fc41415dc20 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41990c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4140ced80 0x7fc4140c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41990c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4140ced80 0x7fc4140c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44074/0 (socket says 172.21.15.19:44074) 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41990c640 1 -- 172.21.15.19:0/2595933366 learned_addr learned my addr 172.21.15.19:0/2595933366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41a10d640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc414161710 0x7fc414152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41910b640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc41415dc20 0x7fc4140c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41990c640 1 -- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc41415dc20 msgr2=0x7fc4140c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41990c640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc41415dc20 0x7fc4140c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41990c640 1 -- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc414161710 msgr2=0x7fc414152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41990c640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc414161710 0x7fc414152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41990c640 1 -- 172.21.15.19:0/2595933366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41415e1f0 con 0x7fc4140ced80 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41a10d640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc414161710 0x7fc414152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41910b640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc41415dc20 0x7fc4140c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41990c640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4140ced80 0x7fc4140c9f70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc41000c3c0 tx=0x7fc41000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc40a7fc640 1 -- 172.21.15.19:0/2595933366 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4100052d0 con 0x7fc4140ced80 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc40a7fc640 1 -- 172.21.15.19:0/2595933366 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc410005490 con 0x7fc4140ced80 2024-04-25T09:53:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc40a7fc640 1 -- 172.21.15.19:0/2595933366 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc410007260 con 0x7fc4140ced80 2024-04-25T09:53:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41bb70640 1 -- 172.21.15.19:0/2595933366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4140c8fe0 con 0x7fc4140ced80 2024-04-25T09:53:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.842+0000 7fc41bb70640 1 -- 172.21.15.19:0/2595933366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41415e4b0 con 0x7fc4140ced80 2024-04-25T09:53:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.843+0000 7fc41bb70640 1 -- 172.21.15.19:0/2595933366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4140ca730 con 0x7fc4140ced80 2024-04-25T09:53:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.844+0000 7fc40a7fc640 1 -- 172.21.15.19:0/2595933366 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc410007400 con 0x7fc4140ced80 2024-04-25T09:53:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.844+0000 7fc40a7fc640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3f806b500 0x7fc3f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.844+0000 7fc40a7fc640 1 -- 172.21.15.19:0/2595933366 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc410089ad0 con 0x7fc4140ced80 2024-04-25T09:53:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.844+0000 7fc41910b640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3f806b500 0x7fc3f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.845+0000 7fc41910b640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3f806b500 0x7fc3f806d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc404006fd0 tx=0x7fc404008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:58.848+0000 7fc40a7fc640 1 -- 172.21.15.19:0/2595933366 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4140ca730 con 0x7fc4140ced80 2024-04-25T09:53:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.027+0000 7fc41bb70640 1 -- 172.21.15.19:0/2595933366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc414073a00 con 0x7fc4140ced80 2024-04-25T09:53:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.027+0000 7fc40a7fc640 1 -- 172.21.15.19:0/2595933366 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc410058540 con 0x7fc4140ced80 2024-04-25T09:53:59.029 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:53:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 -- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3f806b500 msgr2=0x7fc3f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3f806b500 0x7fc3f806d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc404006fd0 tx=0x7fc404008040 comp rx=0 tx=0).stop 2024-04-25T09:53:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 -- 172.21.15.19:0/2595933366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4140ced80 msgr2=0x7fc4140c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4140ced80 0x7fc4140c9f70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc41000c3c0 tx=0x7fc41000c880 comp rx=0 tx=0).stop 2024-04-25T09:53:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 -- 172.21.15.19:0/2595933366 shutdown_connections 2024-04-25T09:53:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3f806b500 0x7fc3f806d9c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc414161710 0x7fc414152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc41415dc20 0x7fc4140c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 --2- 172.21.15.19:0/2595933366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4140ced80 0x7fc4140c9f70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 -- 172.21.15.19:0/2595933366 >> 172.21.15.19:0/2595933366 conn(0x7fc4140d0420 msgr2=0x7fc4140d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 -- 172.21.15.19:0/2595933366 shutdown_connections 2024-04-25T09:53:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.029+0000 7fc41bb70640 1 -- 172.21.15.19:0/2595933366 wait complete. 2024-04-25T09:53:59.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.144+0000 7f7db5753640 1 Processor -- start 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.144+0000 7f7db5753640 1 -- start start 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.145+0000 7f7db5753640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db00ced80 0x7f7db00d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.145+0000 7f7db5753640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7db0161710 0x7f7db00d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.145+0000 7f7db5753640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db015e000 con 0x7f7db015dc20 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.145+0000 7f7db5753640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00d1de0 con 0x7f7db0161710 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.145+0000 7f7db5753640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0161c60 con 0x7f7db00ced80 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.145+0000 7f7dae7fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db00ced80 0x7f7db00d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.145+0000 7f7daf7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7db0161710 0x7f7db00d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.145+0000 7f7dae7fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db00ced80 0x7f7db00d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:52170/0 (socket says 172.21.15.19:52170) 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.145+0000 7f7daf7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7db0161710 0x7f7db00d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:53496/0 (socket says 172.21.15.19:53496) 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.145+0000 7f7dae7fc640 1 -- 172.21.15.19:0/151923708 learned_addr learned my addr 172.21.15.19:0/151923708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.145+0000 7f7daeffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7db015dc20 0x7f7db00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:48618/0 (socket says 172.21.15.19:48618) 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.146+0000 7f7dae7fc640 1 -- 172.21.15.19:0/151923708 >> v2:172.21.15.112:3300/0 conn(0x7f7db0161710 msgr2=0x7f7db00d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.146+0000 7f7dae7fc640 1 --2- 172.21.15.19:0/151923708 >> v2:172.21.15.112:3300/0 conn(0x7f7db0161710 0x7f7db00d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.146+0000 7f7dae7fc640 1 -- 172.21.15.19:0/151923708 >> v1:172.21.15.112:6789/0 conn(0x7f7db015dc20 legacy=0x7f7db00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.146+0000 7f7dae7fc640 1 -- 172.21.15.19:0/151923708 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db0161f40 con 0x7f7db00ced80 2024-04-25T09:53:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.146+0000 7f7dae7fc640 1 --2- 172.21.15.19:0/151923708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db00ced80 0x7f7db00d22e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009870 tx=0x7f7da002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fba699ccfee03086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.146+0000 7f7dadffb640 1 -- 172.21.15.19:0/151923708 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0037070 con 0x7f7db00ced80 2024-04-25T09:53:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.147+0000 7f7dadffb640 1 -- 172.21.15.19:0/151923708 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da0032440 con 0x7f7db00ced80 2024-04-25T09:53:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.147+0000 7f7dadffb640 1 -- 172.21.15.19:0/151923708 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0032810 con 0x7f7db00ced80 2024-04-25T09:53:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.147+0000 7f7db5753640 1 -- 172.21.15.19:0/151923708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db00ced80 msgr2=0x7f7db00d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.147+0000 7f7db5753640 1 --2- 172.21.15.19:0/151923708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db00ced80 0x7f7db00d22e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009870 tx=0x7f7da002a270 comp rx=0 tx=0).stop 2024-04-25T09:53:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.147+0000 7f7db5753640 1 -- 172.21.15.19:0/151923708 shutdown_connections 2024-04-25T09:53:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.147+0000 7f7db5753640 1 --2- 172.21.15.19:0/151923708 >> v2:172.21.15.112:3300/0 conn(0x7f7db0161710 0x7f7db00d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.147+0000 7f7db5753640 1 --2- 172.21.15.19:0/151923708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db00ced80 0x7f7db00d22e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.147+0000 7f7db5753640 1 -- 172.21.15.19:0/151923708 >> 172.21.15.19:0/151923708 conn(0x7f7db00d0420 msgr2=0x7f7db00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.147+0000 7f7db5753640 1 -- 172.21.15.19:0/151923708 shutdown_connections 2024-04-25T09:53:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.147+0000 7f7db5753640 1 -- 172.21.15.19:0/151923708 wait complete. 2024-04-25T09:53:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.148+0000 7f7db5753640 1 Processor -- start 2024-04-25T09:53:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.148+0000 7f7db5753640 1 -- start start 2024-04-25T09:53:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.149+0000 7f7db5753640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7db00ced80 0x7f7db015c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.149+0000 7f7db5753640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db015dc20 0x7f7db015e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.149+0000 7f7db5753640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db0161710 0x7f7db015e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.149+0000 7f7db5753640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b5e50 con 0x7f7db00ced80 2024-04-25T09:53:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.149+0000 7f7db5753640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0162500 con 0x7f7db0161710 2024-04-25T09:53:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.149+0000 7f7db5753640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b5fd0 con 0x7f7db015dc20 2024-04-25T09:53:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.149+0000 7f7daeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7db00ced80 0x7f7db015c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.149+0000 7f7daeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7db00ced80 0x7f7db015c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44078/0 (socket says 172.21.15.19:44078) 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.149+0000 7f7daeffd640 1 -- 172.21.15.19:0/2754360352 learned_addr learned my addr 172.21.15.19:0/2754360352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7dae7fc640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db015dc20 0x7f7db015e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7daf7fe640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db0161710 0x7f7db015e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7daeffd640 1 -- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db015dc20 msgr2=0x7f7db015e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7daeffd640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db015dc20 0x7f7db015e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7daeffd640 1 -- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db0161710 msgr2=0x7f7db015e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7daeffd640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db0161710 0x7f7db015e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7daeffd640 1 -- 172.21.15.19:0/2754360352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db01f3cf0 con 0x7f7db00ced80 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7dae7fc640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db015dc20 0x7f7db015e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7daf7fe640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db0161710 0x7f7db015e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7daeffd640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7db00ced80 0x7f7db015c850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c3c0 tx=0x7f7d9800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7da77fe640 1 -- 172.21.15.19:0/2754360352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d980074e0 con 0x7f7db00ced80 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7da77fe640 1 -- 172.21.15.19:0/2754360352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d98007680 con 0x7f7db00ced80 2024-04-25T09:53:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7da77fe640 1 -- 172.21.15.19:0/2754360352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98004a00 con 0x7f7db00ced80 2024-04-25T09:53:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7db5753640 1 -- 172.21.15.19:0/2754360352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db01f3fd0 con 0x7f7db00ced80 2024-04-25T09:53:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.150+0000 7f7db5753640 1 -- 172.21.15.19:0/2754360352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db01f4590 con 0x7f7db00ced80 2024-04-25T09:53:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.151+0000 7f7db5753640 1 -- 172.21.15.19:0/2754360352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db01f3a80 con 0x7f7db00ced80 2024-04-25T09:53:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.151+0000 7f7da77fe640 1 -- 172.21.15.19:0/2754360352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d9801a020 con 0x7f7db00ced80 2024-04-25T09:53:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.152+0000 7f7da77fe640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d9006b500 0x7f7d9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.152+0000 7f7da77fe640 1 -- 172.21.15.19:0/2754360352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7d98089d40 con 0x7f7db00ced80 2024-04-25T09:53:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.152+0000 7f7dae7fc640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d9006b500 0x7f7d9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.153+0000 7f7dae7fc640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d9006b500 0x7f7d9006d9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7da0034000 tx=0x7f7da002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.156+0000 7f7da77fe640 1 -- 172.21.15.19:0/2754360352 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d980587a0 con 0x7f7db00ced80 2024-04-25T09:53:59.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.335+0000 7f7db5753640 1 -- 172.21.15.19:0/2754360352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7db0073a00 con 0x7f7db00ced80 2024-04-25T09:53:59.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.335+0000 7f7da77fe640 1 -- 172.21.15.19:0/2754360352 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d980580a0 con 0x7f7db00ced80 2024-04-25T09:53:59.337 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 -- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d9006b500 msgr2=0x7f7d9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d9006b500 0x7f7d9006d9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7da0034000 tx=0x7f7da002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 -- 172.21.15.19:0/2754360352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7db00ced80 msgr2=0x7f7db015c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7db00ced80 0x7f7db015c850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c3c0 tx=0x7f7d9800c880 comp rx=0 tx=0).stop 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 -- 172.21.15.19:0/2754360352 shutdown_connections 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d9006b500 0x7f7d9006d9c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db0161710 0x7f7db015e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7db015dc20 0x7f7db015e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 --2- 172.21.15.19:0/2754360352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7db00ced80 0x7f7db015c850 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 -- 172.21.15.19:0/2754360352 >> 172.21.15.19:0/2754360352 conn(0x7f7db00d0420 msgr2=0x7f7db00b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 -- 172.21.15.19:0/2754360352 shutdown_connections 2024-04-25T09:53:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.337+0000 7f7db5753640 1 -- 172.21.15.19:0/2754360352 wait complete. 2024-04-25T09:53:59.350 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.451+0000 7f1e61e58640 1 Processor -- start 2024-04-25T09:53:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.451+0000 7f1e61e58640 1 -- start start 2024-04-25T09:53:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.452+0000 7f1e61e58640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1e5c0b5620 0x7f1e5c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.452+0000 7f1e61e58640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e5c06c2d0 0x7f1e5c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.452+0000 7f1e61e58640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c06cc80 con 0x7f1e5c15b7d0 2024-04-25T09:53:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.452+0000 7f1e61e58640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c06cf50 con 0x7f1e5c0b5620 2024-04-25T09:53:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.452+0000 7f1e61e58640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0ce530 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.452+0000 7f1e5b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1e5c15b7d0 0x7f1e5c0cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48316/0 (socket says 172.21.15.19:48316) 2024-04-25T09:53:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.452+0000 7f1e5b7fe640 1 -- 172.21.15.19:0/3164665386 learned_addr learned my addr 172.21.15.19:0/3164665386 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.452+0000 7f1e5affd640 1 --2- 172.21.15.19:0/3164665386 >> v2:172.21.15.112:3300/0 conn(0x7f1e5c0b5620 0x7f1e5c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.452+0000 7f1e5bfff640 1 --2- 172.21.15.19:0/3164665386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e5c06c2d0 0x7f1e5c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.452+0000 7f1e5a7fc640 1 -- 172.21.15.19:0/3164665386 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777512773 0 0) 0x7f1e5c06cc80 con 0x7f1e5c15b7d0 2024-04-25T09:53:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.452+0000 7f1e5a7fc640 1 -- 172.21.15.19:0/3164665386 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e44003520 con 0x7f1e5c15b7d0 2024-04-25T09:53:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.453+0000 7f1e5a7fc640 1 -- 172.21.15.19:0/3164665386 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2508363442 0 0) 0x7f1e44003520 con 0x7f1e5c15b7d0 2024-04-25T09:53:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.453+0000 7f1e5affd640 1 -- 172.21.15.19:0/3164665386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e5c06c2d0 msgr2=0x7f1e5c06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.453+0000 7f1e5affd640 1 --2- 172.21.15.19:0/3164665386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e5c06c2d0 0x7f1e5c06c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.453+0000 7f1e5affd640 1 -- 172.21.15.19:0/3164665386 >> v1:172.21.15.19:6789/0 conn(0x7f1e5c15b7d0 legacy=0x7f1e5c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.453+0000 7f1e5affd640 1 -- 172.21.15.19:0/3164665386 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c0ce810 con 0x7f1e5c0b5620 2024-04-25T09:53:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.453+0000 7f1e5affd640 1 --2- 172.21.15.19:0/3164665386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0b5620 0x7f1e5c0b5ab0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1e500097a0 tx=0x7f1e5002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f235369f3e7b902 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.454+0000 7f1e5a7fc640 1 -- 172.21.15.19:0/3164665386 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50037070 con 0x7f1e5c0b5620 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.454+0000 7f1e5a7fc640 1 -- 172.21.15.19:0/3164665386 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5002aba0 con 0x7f1e5c0b5620 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.454+0000 7f1e5a7fc640 1 -- 172.21.15.19:0/3164665386 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c001670 con 0x7f1e5c0b5620 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.454+0000 7f1e61e58640 1 -- 172.21.15.19:0/3164665386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0b5620 msgr2=0x7f1e5c0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.454+0000 7f1e61e58640 1 --2- 172.21.15.19:0/3164665386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0b5620 0x7f1e5c0b5ab0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1e500097a0 tx=0x7f1e5002a110 comp rx=0 tx=0).stop 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.454+0000 7f1e61e58640 1 -- 172.21.15.19:0/3164665386 shutdown_connections 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.454+0000 7f1e61e58640 1 --2- 172.21.15.19:0/3164665386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e5c06c2d0 0x7f1e5c06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.454+0000 7f1e61e58640 1 --2- 172.21.15.19:0/3164665386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c0b5620 0x7f1e5c0b5ab0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.454+0000 7f1e61e58640 1 -- 172.21.15.19:0/3164665386 >> 172.21.15.19:0/3164665386 conn(0x7f1e5c0d0800 msgr2=0x7f1e5c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.454+0000 7f1e61e58640 1 -- 172.21.15.19:0/3164665386 shutdown_connections 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.454+0000 7f1e61e58640 1 -- 172.21.15.19:0/3164665386 wait complete. 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.455+0000 7f1e61e58640 1 Processor -- start 2024-04-25T09:53:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.456+0000 7f1e61e58640 1 -- start start 2024-04-25T09:53:59.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.456+0000 7f1e61e58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e5c06c2d0 0x7f1e5c157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.456+0000 7f1e61e58640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e5c0b5620 0x7f1e5c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.456+0000 7f1e61e58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c15b7d0 0x7f1e5c0aa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.456+0000 7f1e61e58640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c15bbb0 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.456+0000 7f1e61e58640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c06d230 con 0x7f1e5c15b7d0 2024-04-25T09:53:59.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.456+0000 7f1e61e58640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c15bd30 con 0x7f1e5c0b5620 2024-04-25T09:53:59.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.456+0000 7f1e5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e5c06c2d0 0x7f1e5c157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.456+0000 7f1e5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e5c06c2d0 0x7f1e5c157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44092/0 (socket says 172.21.15.19:44092) 2024-04-25T09:53:59.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.456+0000 7f1e5b7fe640 1 -- 172.21.15.19:0/2778670683 learned_addr learned my addr 172.21.15.19:0/2778670683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:59.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e5affd640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e5c0b5620 0x7f1e5c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e5bfff640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c15b7d0 0x7f1e5c0aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e5b7fe640 1 -- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e5c0b5620 msgr2=0x7f1e5c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e5b7fe640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e5c0b5620 0x7f1e5c0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e5b7fe640 1 -- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c15b7d0 msgr2=0x7f1e5c0aa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e5b7fe640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c15b7d0 0x7f1e5c0aa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e5b7fe640 1 -- 172.21.15.19:0/2778670683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c158190 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e5bfff640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c15b7d0 0x7f1e5c0aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e5affd640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e5c0b5620 0x7f1e5c0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e5b7fe640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e5c06c2d0 0x7f1e5c157c50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00c700 tx=0x7f1e4c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e43fff640 1 -- 172.21.15.19:0/2778670683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c017070 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e43fff640 1 -- 172.21.15.19:0/2778670683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c005ad0 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e43fff640 1 -- 172.21.15.19:0/2778670683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c007070 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e61e58640 1 -- 172.21.15.19:0/2778670683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c0c8660 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.457+0000 7f1e61e58640 1 -- 172.21.15.19:0/2778670683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c0c8c20 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.458+0000 7f1e61e58640 1 -- 172.21.15.19:0/2778670683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c073a00 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.459+0000 7f1e43fff640 1 -- 172.21.15.19:0/2778670683 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e4c01a070 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.459+0000 7f1e43fff640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e4806b4d0 0x7f1e4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.459+0000 7f1e43fff640 1 -- 172.21.15.19:0/2778670683 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1e4c0899a0 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.459+0000 7f1e5affd640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e4806b4d0 0x7f1e4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.460+0000 7f1e5affd640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e4806b4d0 0x7f1e4806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1e50034000 tx=0x7f1e5002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.463+0000 7f1e43fff640 1 -- 172.21.15.19:0/2778670683 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e4c057dd0 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.643+0000 7f1e61e58640 1 -- 172.21.15.19:0/2778670683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e5c15c4c0 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.643+0000 7f1e43fff640 1 -- 172.21.15.19:0/2778670683 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e4c05be70 con 0x7f1e5c06c2d0 2024-04-25T09:53:59.645 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:53:59.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 -- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e4806b4d0 msgr2=0x7f1e4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e4806b4d0 0x7f1e4806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1e50034000 tx=0x7f1e5002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 -- 172.21.15.19:0/2778670683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e5c06c2d0 msgr2=0x7f1e5c157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e5c06c2d0 0x7f1e5c157c50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00c700 tx=0x7f1e4c00cbc0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 -- 172.21.15.19:0/2778670683 shutdown_connections 2024-04-25T09:53:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e4806b4d0 0x7f1e4806d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c15b7d0 0x7f1e5c0aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e5c0b5620 0x7f1e5c0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 --2- 172.21.15.19:0/2778670683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e5c06c2d0 0x7f1e5c157c50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 -- 172.21.15.19:0/2778670683 >> 172.21.15.19:0/2778670683 conn(0x7f1e5c0d0800 msgr2=0x7f1e5c0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 -- 172.21.15.19:0/2778670683 shutdown_connections 2024-04-25T09:53:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.645+0000 7f1e61e58640 1 -- 172.21.15.19:0/2778670683 wait complete. 2024-04-25T09:53:59.658 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:59.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.759+0000 7f0db1a91640 1 Processor -- start 2024-04-25T09:53:59.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.759+0000 7f0db1a91640 1 -- start start 2024-04-25T09:53:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.760+0000 7f0db1a91640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0dac0d3440 0x7f0dac0d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.760+0000 7f0db1a91640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dac0d23a0 con 0x7f0dac15dc20 2024-04-25T09:53:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.760+0000 7f0db1a91640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dac0d2680 con 0x7f0dac0ced80 2024-04-25T09:53:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.760+0000 7f0db1a91640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dac0d2960 con 0x7f0dac0d3440 2024-04-25T09:53:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.760+0000 7f0dab7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0dac15dc20 0x7f0dac0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48328/0 (socket says 172.21.15.19:48328) 2024-04-25T09:53:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.760+0000 7f0dab7fe640 1 -- 172.21.15.19:0/3309691413 learned_addr learned my addr 172.21.15.19:0/3309691413 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:53:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.760+0000 7f0dabfff640 1 --2- 172.21.15.19:0/3309691413 >> v2:172.21.15.112:3300/0 conn(0x7f0dac0d3440 0x7f0dac0d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.760+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985263264 0 0) 0x7f0dac0d23a0 con 0x7f0dac15dc20 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.760+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d94003520 con 0x7f0dac15dc20 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 336181984 0 0) 0x7f0d94003520 con 0x7f0dac15dc20 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 >> v2:172.21.15.112:3300/0 conn(0x7f0dac0d3440 msgr2=0x7f0dac0d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 --2- 172.21.15.19:0/3309691413 >> v2:172.21.15.112:3300/0 conn(0x7f0dac0d3440 0x7f0dac0d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 >> v1:172.21.15.112:6789/0 conn(0x7f0dac0ced80 legacy=0x7f0dac0b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac0d2c40 con 0x7f0dac15dc20 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0d9c0019f0 con 0x7f0dac15dc20 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 >> v1:172.21.15.19:6789/0 conn(0x7f0dac15dc20 legacy=0x7f0dac0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 --2- 172.21.15.19:0/3309691413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d940039b0 0x7f0d94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 --2- 172.21.15.19:0/3309691413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d94004800 0x7f0d94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 --2- 172.21.15.19:0/3309691413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d940051c0 0x7f0d9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0d2960 con 0x7f0d940039b0 2024-04-25T09:53:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9400c160 con 0x7f0d94004800 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9400c360 con 0x7f0d940051c0 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0dab7fe640 1 --2- 172.21.15.19:0/3309691413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d940039b0 0x7f0d94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0daaffd640 1 --2- 172.21.15.19:0/3309691413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d94004800 0x7f0d94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0dabfff640 1 --2- 172.21.15.19:0/3309691413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d940051c0 0x7f0d9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0dab7fe640 1 -- 172.21.15.19:0/3309691413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d940051c0 msgr2=0x7f0d9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0dab7fe640 1 --2- 172.21.15.19:0/3309691413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d940051c0 0x7f0d9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0dab7fe640 1 -- 172.21.15.19:0/3309691413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d94004800 msgr2=0x7f0d94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0dab7fe640 1 --2- 172.21.15.19:0/3309691413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d94004800 0x7f0d94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0dab7fe640 1 -- 172.21.15.19:0/3309691413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d94003520 con 0x7f0d940039b0 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.761+0000 7f0dabfff640 1 -- 172.21.15.19:0/3309691413 reap_dead start 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.762+0000 7f0dab7fe640 1 --2- 172.21.15.19:0/3309691413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d940039b0 0x7f0d94003dd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00a860 tx=0x7f0d94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa514185f0430f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.762+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00a480 con 0x7f0d940039b0 2024-04-25T09:53:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.762+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c02f070 con 0x7f0d940039b0 2024-04-25T09:53:59.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.762+0000 7f0daa7fc640 1 -- 172.21.15.19:0/3309691413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c02bcb0 con 0x7f0d940039b0 2024-04-25T09:53:59.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.762+0000 7f0db1a91640 1 -- 172.21.15.19:0/3309691413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d940039b0 msgr2=0x7f0d94003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.762+0000 7f0db1a91640 1 --2- 172.21.15.19:0/3309691413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d940039b0 0x7f0d94003dd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00a860 tx=0x7f0d94005830 comp rx=0 tx=0).stop 2024-04-25T09:53:59.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.763+0000 7f0db1a91640 1 -- 172.21.15.19:0/3309691413 shutdown_connections 2024-04-25T09:53:59.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.763+0000 7f0db1a91640 1 --2- 172.21.15.19:0/3309691413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d940039b0 0x7f0d94003dd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.763+0000 7f0db1a91640 1 -- 172.21.15.19:0/3309691413 >> 172.21.15.19:0/3309691413 conn(0x7f0dac061a20 msgr2=0x7f0dac061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.763+0000 7f0db1a91640 1 -- 172.21.15.19:0/3309691413 shutdown_connections 2024-04-25T09:53:59.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.763+0000 7f0db1a91640 1 -- 172.21.15.19:0/3309691413 wait complete. 2024-04-25T09:53:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.764+0000 7f0db1a91640 1 Processor -- start 2024-04-25T09:53:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.764+0000 7f0db1a91640 1 -- start start 2024-04-25T09:53:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0db1a91640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d940039b0 0x7f0dac156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0db1a91640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dac159250 0x7f0dac1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0db1a91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dac0c8f70 0x7f0dac0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0db1a91640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0d1e60 con 0x7f0dac0c8f70 2024-04-25T09:53:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0db1a91640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0d1ce0 con 0x7f0d940039b0 2024-04-25T09:53:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0db1a91640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0d1fe0 con 0x7f0dac159250 2024-04-25T09:53:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0dabfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dac0c8f70 0x7f0dac0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0dabfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dac0c8f70 0x7f0dac0c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44102/0 (socket says 172.21.15.19:44102) 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0dabfff640 1 -- 172.21.15.19:0/2998562841 learned_addr learned my addr 172.21.15.19:0/2998562841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0daaffd640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dac159250 0x7f0dac1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0dab7fe640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d940039b0 0x7f0dac156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0dabfff640 1 -- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dac159250 msgr2=0x7f0dac1596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0dabfff640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dac159250 0x7f0dac1596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0dabfff640 1 -- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d940039b0 msgr2=0x7f0dac156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0dabfff640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d940039b0 0x7f0dac156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0dabfff640 1 -- 172.21.15.19:0/2998562841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac156f40 con 0x7f0dac0c8f70 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0daaffd640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dac159250 0x7f0dac1596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0dab7fe640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d940039b0 0x7f0dac156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0dabfff640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dac0c8f70 0x7f0dac0c9420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0b6500 tx=0x7f0da4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0d93fff640 1 -- 172.21.15.19:0/2998562841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da4003610 con 0x7f0dac0c8f70 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0d93fff640 1 -- 172.21.15.19:0/2998562841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da401b070 con 0x7f0dac0c8f70 2024-04-25T09:53:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.765+0000 7f0db1a91640 1 -- 172.21.15.19:0/2998562841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dac0ccbb0 con 0x7f0dac0c8f70 2024-04-25T09:53:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.766+0000 7f0d93fff640 1 -- 172.21.15.19:0/2998562841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da40039f0 con 0x7f0dac0c8f70 2024-04-25T09:53:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.766+0000 7f0db1a91640 1 -- 172.21.15.19:0/2998562841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac0cd170 con 0x7f0dac0c8f70 2024-04-25T09:53:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.766+0000 7f0db1a91640 1 -- 172.21.15.19:0/2998562841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dac0739e0 con 0x7f0dac0c8f70 2024-04-25T09:53:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.767+0000 7f0d93fff640 1 -- 172.21.15.19:0/2998562841 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0da4003b90 con 0x7f0dac0c8f70 2024-04-25T09:53:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.767+0000 7f0d93fff640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9806b500 0x7f0d9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.768+0000 7f0d93fff640 1 -- 172.21.15.19:0/2998562841 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0da408a8b0 con 0x7f0dac0c8f70 2024-04-25T09:53:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.768+0000 7f0dab7fe640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9806b500 0x7f0d9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.768+0000 7f0dab7fe640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9806b500 0x7f0d9806d9c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c033000 tx=0x7f0d9c038040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.772+0000 7f0d93fff640 1 -- 172.21.15.19:0/2998562841 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0da4059330 con 0x7f0dac0c8f70 2024-04-25T09:53:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.958+0000 7f0db1a91640 1 -- 172.21.15.19:0/2998562841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0dac062180 con 0x7f0dac0c8f70 2024-04-25T09:53:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.958+0000 7f0d93fff640 1 -- 172.21.15.19:0/2998562841 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0da4058c30 con 0x7f0dac0c8f70 2024-04-25T09:53:59.961 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:53:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.960+0000 7f0db1a91640 1 -- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9806b500 msgr2=0x7f0d9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.960+0000 7f0db1a91640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9806b500 0x7f0d9806d9c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c033000 tx=0x7f0d9c038040 comp rx=0 tx=0).stop 2024-04-25T09:53:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.960+0000 7f0db1a91640 1 -- 172.21.15.19:0/2998562841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dac0c8f70 msgr2=0x7f0dac0c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.960+0000 7f0db1a91640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dac0c8f70 0x7f0dac0c9420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0b6500 tx=0x7f0da4009f90 comp rx=0 tx=0).stop 2024-04-25T09:53:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.960+0000 7f0db1a91640 1 -- 172.21.15.19:0/2998562841 shutdown_connections 2024-04-25T09:53:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.960+0000 7f0db1a91640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9806b500 0x7f0d9806d9c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.960+0000 7f0db1a91640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dac0c8f70 0x7f0dac0c9420 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.960+0000 7f0db1a91640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dac159250 0x7f0dac1596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.960+0000 7f0db1a91640 1 --2- 172.21.15.19:0/2998562841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d940039b0 0x7f0dac156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.960+0000 7f0db1a91640 1 -- 172.21.15.19:0/2998562841 >> 172.21.15.19:0/2998562841 conn(0x7f0dac061a20 msgr2=0x7f0dac0b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.961+0000 7f0db1a91640 1 -- 172.21.15.19:0/2998562841 shutdown_connections 2024-04-25T09:53:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:53:59.961+0000 7f0db1a91640 1 -- 172.21.15.19:0/2998562841 wait complete. 2024-04-25T09:53:59.974 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.073+0000 7ff0e0d3b640 1 Processor -- start 2024-04-25T09:54:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.073+0000 7ff0e0d3b640 1 -- start start 2024-04-25T09:54:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.073+0000 7ff0e0d3b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff0dc15dbe0 0x7ff0dc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.073+0000 7ff0e0d3b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0dc0b6220 con 0x7ff0dc0b92a0 2024-04-25T09:54:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.073+0000 7ff0e0d3b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0dc0b97a0 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.073+0000 7ff0e0d3b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0dc15dfc0 con 0x7ff0dc15dbe0 2024-04-25T09:54:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.074+0000 7ff0da59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff0dc15dbe0 0x7ff0dc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.074+0000 7ff0dad9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff0dc0b92a0 0x7ff0dc0b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48338/0 (socket says 172.21.15.19:48338) 2024-04-25T09:54:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.074+0000 7ff0dad9d640 1 -- 172.21.15.19:0/3878726404 learned_addr learned my addr 172.21.15.19:0/3878726404 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.074+0000 7ff0da59c640 1 -- 172.21.15.19:0/3878726404 >> v1:172.21.15.112:6789/0 conn(0x7ff0dc0b2cb0 legacy=0x7ff0dc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.074+0000 7ff0da59c640 1 -- 172.21.15.19:0/3878726404 >> v1:172.21.15.19:6789/0 conn(0x7ff0dc0b92a0 legacy=0x7ff0dc0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.074+0000 7ff0da59c640 1 -- 172.21.15.19:0/3878726404 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc0d1c50 con 0x7ff0dc15dbe0 2024-04-25T09:54:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.075+0000 7ff0da59c640 1 --2- 172.21.15.19:0/3878726404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0dc15dbe0 0x7ff0dc0d0910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0097a0 tx=0x7ff0cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b273406a8f55f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.075+0000 7ff0d9d9b640 1 -- 172.21.15.19:0/3878726404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc037070 con 0x7ff0dc15dbe0 2024-04-25T09:54:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.075+0000 7ff0d9d9b640 1 -- 172.21.15.19:0/3878726404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc02ae50 con 0x7ff0dc15dbe0 2024-04-25T09:54:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.075+0000 7ff0d9d9b640 1 -- 172.21.15.19:0/3878726404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc032660 con 0x7ff0dc15dbe0 2024-04-25T09:54:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.075+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/3878726404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0dc15dbe0 msgr2=0x7ff0dc0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.075+0000 7ff0e0d3b640 1 --2- 172.21.15.19:0/3878726404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0dc15dbe0 0x7ff0dc0d0910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0097a0 tx=0x7ff0cc02a140 comp rx=0 tx=0).stop 2024-04-25T09:54:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.075+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/3878726404 shutdown_connections 2024-04-25T09:54:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.075+0000 7ff0e0d3b640 1 --2- 172.21.15.19:0/3878726404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0dc15dbe0 0x7ff0dc0d0910 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.075+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/3878726404 >> 172.21.15.19:0/3878726404 conn(0x7ff0dc061a20 msgr2=0x7ff0dc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.075+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/3878726404 shutdown_connections 2024-04-25T09:54:00.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.075+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/3878726404 wait complete. 2024-04-25T09:54:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.077+0000 7ff0e0d3b640 1 Processor -- start 2024-04-25T09:54:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.077+0000 7ff0e0d3b640 1 -- start start 2024-04-25T09:54:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.077+0000 7ff0e0d3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0dc0b2cb0 0x7ff0dc151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.077+0000 7ff0e0d3b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0dc0b92a0 0x7ff0dc15e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.077+0000 7ff0e0d3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0dc15dbe0 0x7ff0dc15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.077+0000 7ff0e0d3b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0d2210 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0e0d3b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0d1a00 con 0x7ff0dc15dbe0 2024-04-25T09:54:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0e0d3b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0b6ab0 con 0x7ff0dc0b92a0 2024-04-25T09:54:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0da59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0dc0b2cb0 0x7ff0dc151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0da59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0dc0b2cb0 0x7ff0dc151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44118/0 (socket says 172.21.15.19:44118) 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0da59c640 1 -- 172.21.15.19:0/1336439431 learned_addr learned my addr 172.21.15.19:0/1336439431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0dad9d640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0dc15dbe0 0x7ff0dc15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0d3fff640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0dc0b92a0 0x7ff0dc15e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0da59c640 1 -- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0dc0b92a0 msgr2=0x7ff0dc15e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0da59c640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0dc0b92a0 0x7ff0dc15e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0da59c640 1 -- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0dc15dbe0 msgr2=0x7ff0dc15eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0da59c640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0dc15dbe0 0x7ff0dc15eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0da59c640 1 -- 172.21.15.19:0/1336439431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc1531f0 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0da59c640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0dc0b2cb0 0x7ff0dc151b60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0097a0 tx=0x7ff0cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0d2ffd640 1 -- 172.21.15.19:0/1336439431 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc042070 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0d2ffd640 1 -- 172.21.15.19:0/1336439431 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc03e670 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0d2ffd640 1 -- 172.21.15.19:0/1336439431 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc037040 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/1336439431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0dc153490 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.078+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/1336439431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0dc153a50 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.079+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/1336439431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0dc073990 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.080+0000 7ff0d2ffd640 1 -- 172.21.15.19:0/1336439431 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0cc045070 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.080+0000 7ff0d2ffd640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0bc06b4d0 0x7ff0bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.080+0000 7ff0d2ffd640 1 -- 172.21.15.19:0/1336439431 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff0cc0aaf60 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.081+0000 7ff0d3fff640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0bc06b4d0 0x7ff0bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.081+0000 7ff0d3fff640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0bc06b4d0 0x7ff0bc06d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc153120 tx=0x7ff0c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.084+0000 7ff0d2ffd640 1 -- 172.21.15.19:0/1336439431 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0cc079af0 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.304+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/1336439431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0dc0b2610 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.305+0000 7ff0d2ffd640 1 -- 172.21.15.19:0/1336439431 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7ff0cc0793f0 con 0x7ff0dc0b2cb0 2024-04-25T09:54:00.307 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:00.307 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4000846111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":4000846111},{"type":"v1","addr":"172.21.15.112:6837","nonce":4000846111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1969525282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1969525282},{"type":"v1","addr":"172.21.15.19:6835","nonce":1969525282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:07.175350+0000","modified":"2024-04-25T09:53:23.345595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1080352556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1080352556},{"type":"v1","addr":"172.21.15.112:6836","nonce":1080352556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:54:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T09:54:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.306+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0bc06b4d0 msgr2=0x7ff0bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.306+0000 7ff0e0d3b640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0bc06b4d0 0x7ff0bc06d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc153120 tx=0x7ff0c4009040 comp rx=0 tx=0).stop 2024-04-25T09:54:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.306+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/1336439431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0dc0b2cb0 msgr2=0x7ff0dc151b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.306+0000 7ff0e0d3b640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0dc0b2cb0 0x7ff0dc151b60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0097a0 tx=0x7ff0cc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.307+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/1336439431 shutdown_connections 2024-04-25T09:54:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.307+0000 7ff0e0d3b640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0bc06b4d0 0x7ff0bc06d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.307+0000 7ff0e0d3b640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0dc15dbe0 0x7ff0dc15eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.307+0000 7ff0e0d3b640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0dc0b92a0 0x7ff0dc15e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.307+0000 7ff0e0d3b640 1 --2- 172.21.15.19:0/1336439431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0dc0b2cb0 0x7ff0dc151b60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.307+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/1336439431 >> 172.21.15.19:0/1336439431 conn(0x7ff0dc061a20 msgr2=0x7ff0dc0d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.307+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/1336439431 shutdown_connections 2024-04-25T09:54:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.307+0000 7ff0e0d3b640 1 -- 172.21.15.19:0/1336439431 wait complete. 2024-04-25T09:54:00.320 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:00.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.418+0000 7f9648058640 1 Processor -- start 2024-04-25T09:54:00.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.418+0000 7f9648058640 1 -- start start 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.419+0000 7f9648058640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f964015db60 0x7f96400ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.419+0000 7f9648058640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96400d0b10 0x7f96400d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.419+0000 7f9648058640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96400cec00 con 0x7f96400b5f00 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.419+0000 7f9648058640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96400ce930 con 0x7f96400d0b10 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.419+0000 7f9648058640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96400b6450 con 0x7f964015db60 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.419+0000 7f96465f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96400d0b10 0x7f96400d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.419+0000 7f96455f3640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f96400b5f00 0x7f96400d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48342/0 (socket says 172.21.15.19:48342) 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.419+0000 7f96455f3640 1 -- 172.21.15.19:0/1519201791 learned_addr learned my addr 172.21.15.19:0/1519201791 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.420+0000 7f9645df4640 1 --2- 172.21.15.19:0/1519201791 >> v2:172.21.15.112:3300/0 conn(0x7f964015db60 0x7f96400ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.420+0000 7f96465f5640 1 -- 172.21.15.19:0/1519201791 >> v2:172.21.15.112:3300/0 conn(0x7f964015db60 msgr2=0x7f96400ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.420+0000 7f96465f5640 1 --2- 172.21.15.19:0/1519201791 >> v2:172.21.15.112:3300/0 conn(0x7f964015db60 0x7f96400ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.420+0000 7f96465f5640 1 -- 172.21.15.19:0/1519201791 >> v1:172.21.15.19:6789/0 conn(0x7f96400b5f00 legacy=0x7f96400d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.420+0000 7f96465f5640 1 -- 172.21.15.19:0/1519201791 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400ceee0 con 0x7f96400d0b10 2024-04-25T09:54:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.420+0000 7f9645df4640 1 --2- 172.21.15.19:0/1519201791 >> v2:172.21.15.112:3300/0 conn(0x7f964015db60 0x7f96400ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.420+0000 7f96465f5640 1 --2- 172.21.15.19:0/1519201791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96400d0b10 0x7f96400d2270 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9638009d90 tx=0x7f963802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2549875967b9ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.421+0000 7f9644df2640 1 -- 172.21.15.19:0/1519201791 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638031600 con 0x7f96400d0b10 2024-04-25T09:54:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.421+0000 7f9644df2640 1 -- 172.21.15.19:0/1519201791 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96380317a0 con 0x7f96400d0b10 2024-04-25T09:54:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.421+0000 7f9644df2640 1 -- 172.21.15.19:0/1519201791 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638031b70 con 0x7f96400d0b10 2024-04-25T09:54:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.421+0000 7f9648058640 1 -- 172.21.15.19:0/1519201791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96400d0b10 msgr2=0x7f96400d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.421+0000 7f9648058640 1 --2- 172.21.15.19:0/1519201791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96400d0b10 0x7f96400d2270 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9638009d90 tx=0x7f963802a610 comp rx=0 tx=0).stop 2024-04-25T09:54:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.421+0000 7f9648058640 1 -- 172.21.15.19:0/1519201791 shutdown_connections 2024-04-25T09:54:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.421+0000 7f9648058640 1 --2- 172.21.15.19:0/1519201791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96400d0b10 0x7f96400d2270 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.421+0000 7f9648058640 1 --2- 172.21.15.19:0/1519201791 >> v2:172.21.15.112:3300/0 conn(0x7f964015db60 0x7f96400ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.421+0000 7f9648058640 1 -- 172.21.15.19:0/1519201791 >> 172.21.15.19:0/1519201791 conn(0x7f9640061600 msgr2=0x7f9640061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.421+0000 7f9648058640 1 -- 172.21.15.19:0/1519201791 shutdown_connections 2024-04-25T09:54:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.421+0000 7f9648058640 1 -- 172.21.15.19:0/1519201791 wait complete. 2024-04-25T09:54:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.422+0000 7f9648058640 1 Processor -- start 2024-04-25T09:54:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.422+0000 7f9648058640 1 -- start start 2024-04-25T09:54:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.423+0000 7f9648058640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96400b5f00 0x7f96401522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.423+0000 7f9648058640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96400d0b10 0x7f96401567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.423+0000 7f9648058640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f964015db60 0x7f9640154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.423+0000 7f9648058640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964015df40 con 0x7f96400b5f00 2024-04-25T09:54:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.423+0000 7f9648058640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400d1d50 con 0x7f96400d0b10 2024-04-25T09:54:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.423+0000 7f9648058640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964015e0c0 con 0x7f964015db60 2024-04-25T09:54:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.423+0000 7f9645df4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96400b5f00 0x7f96401522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.423+0000 7f9645df4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96400b5f00 0x7f96401522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44134/0 (socket says 172.21.15.19:44134) 2024-04-25T09:54:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.423+0000 7f9645df4640 1 -- 172.21.15.19:0/551078427 learned_addr learned my addr 172.21.15.19:0/551078427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.423+0000 7f96465f5640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f964015db60 0x7f9640154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f96455f3640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96400d0b10 0x7f96401567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f9645df4640 1 -- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f964015db60 msgr2=0x7f9640154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f9645df4640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f964015db60 0x7f9640154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f9645df4640 1 -- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96400d0b10 msgr2=0x7f96401567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f9645df4640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96400d0b10 0x7f96401567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f9645df4640 1 -- 172.21.15.19:0/551078427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9640153b90 con 0x7f96400b5f00 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f96455f3640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96400d0b10 0x7f96401567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f9645df4640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96400b5f00 0x7f96401522f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f963c00c7f0 tx=0x7f963c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f96367fc640 1 -- 172.21.15.19:0/551078427 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c015070 con 0x7f96400b5f00 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f96367fc640 1 -- 172.21.15.19:0/551078427 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c007820 con 0x7f96400b5f00 2024-04-25T09:54:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f96367fc640 1 -- 172.21.15.19:0/551078427 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c007bd0 con 0x7f96400b5f00 2024-04-25T09:54:00.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f9648058640 1 -- 172.21.15.19:0/551078427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9640155090 con 0x7f96400b5f00 2024-04-25T09:54:00.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.424+0000 7f9648058640 1 -- 172.21.15.19:0/551078427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964015e700 con 0x7f96400b5f00 2024-04-25T09:54:00.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.425+0000 7f9648058640 1 -- 172.21.15.19:0/551078427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9640073270 con 0x7f96400b5f00 2024-04-25T09:54:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.425+0000 7f96367fc640 1 -- 172.21.15.19:0/551078427 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f963c018070 con 0x7f96400b5f00 2024-04-25T09:54:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.426+0000 7f96367fc640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f962406b4d0 0x7f962406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.426+0000 7f96367fc640 1 -- 172.21.15.19:0/551078427 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f963c08a670 con 0x7f96400b5f00 2024-04-25T09:54:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.426+0000 7f96455f3640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f962406b4d0 0x7f962406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.427+0000 7f96455f3640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f962406b4d0 0x7f962406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96401606e0 tx=0x7f9630009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.430+0000 7f96367fc640 1 -- 172.21.15.19:0/551078427 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f963c059180 con 0x7f96400b5f00 2024-04-25T09:54:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.612+0000 7f9648058640 1 -- 172.21.15.19:0/551078427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9640073990 con 0x7f96400b5f00 2024-04-25T09:54:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.613+0000 7f96367fc640 1 -- 172.21.15.19:0/551078427 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18892 (secure 0 0 0) 0x7f963c058a80 con 0x7f96400b5f00 2024-04-25T09:54:00.616 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:00.616 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:48.071585+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 -- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f962406b4d0 msgr2=0x7f962406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f962406b4d0 0x7f962406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96401606e0 tx=0x7f9630009040 comp rx=0 tx=0).stop 2024-04-25T09:54:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 -- 172.21.15.19:0/551078427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96400b5f00 msgr2=0x7f96401522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96400b5f00 0x7f96401522f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f963c00c7f0 tx=0x7f963c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 -- 172.21.15.19:0/551078427 shutdown_connections 2024-04-25T09:54:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f962406b4d0 0x7f962406d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f964015db60 0x7f9640154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96400d0b10 0x7f96401567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 --2- 172.21.15.19:0/551078427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96400b5f00 0x7f96401522f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 -- 172.21.15.19:0/551078427 >> 172.21.15.19:0/551078427 conn(0x7f9640061600 msgr2=0x7f96400b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 -- 172.21.15.19:0/551078427 shutdown_connections 2024-04-25T09:54:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.615+0000 7f9648058640 1 -- 172.21.15.19:0/551078427 wait complete. 2024-04-25T09:54:00.629 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:54:00.629 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:00.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.727+0000 7fbf92142640 1 Processor -- start 2024-04-25T09:54:00.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.727+0000 7fbf92142640 1 -- start start 2024-04-25T09:54:00.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf92142640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbf8c15db70 0x7fbf8c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf92142640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbf8c0b6cb0 0x7fbf8c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf92142640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf8c0b5a60 0x7fbf8c06c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf92142640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf8c162690 con 0x7fbf8c15db70 2024-04-25T09:54:00.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf92142640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf8c0ceee0 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf92142640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0d1d10 con 0x7fbf8c0b5a60 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbf8c15db70 0x7fbf8c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbf8c15db70 0x7fbf8c162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44148/0 (socket says 172.21.15.19:44148) 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8b7fe640 1 -- 172.21.15.19:0/4162571370 learned_addr learned my addr 172.21.15.19:0/4162571370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8bfff640 1 --2- 172.21.15.19:0/4162571370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf8c0b5a60 0x7fbf8c06c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8affd640 1 --2- 172.21.15.19:0/4162571370 >> v2:172.21.15.112:3300/0 conn(0x7fbf8c0b6cb0 0x7fbf8c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8b7fe640 1 -- 172.21.15.19:0/4162571370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf8c0b5a60 msgr2=0x7fbf8c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8b7fe640 1 --2- 172.21.15.19:0/4162571370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf8c0b5a60 0x7fbf8c06c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8b7fe640 1 -- 172.21.15.19:0/4162571370 >> v2:172.21.15.112:3300/0 conn(0x7fbf8c0b6cb0 msgr2=0x7fbf8c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8b7fe640 1 --2- 172.21.15.19:0/4162571370 >> v2:172.21.15.112:3300/0 conn(0x7fbf8c0b6cb0 0x7fbf8c0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8b7fe640 1 -- 172.21.15.19:0/4162571370 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c0d1ff0 con 0x7fbf8c15db70 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8bfff640 1 --2- 172.21.15.19:0/4162571370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf8c0b5a60 0x7fbf8c06c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:00.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.728+0000 7fbf8affd640 1 --2- 172.21.15.19:0/4162571370 >> v2:172.21.15.112:3300/0 conn(0x7fbf8c0b6cb0 0x7fbf8c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:00.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.729+0000 7fbf8b7fe640 1 --2- 172.21.15.19:0/4162571370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf8c15db70 0x7fbf8c162150 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c08c600 tx=0x7fbf7c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d104814b397a2791 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.729+0000 7fbf8a7fc640 1 -- 172.21.15.19:0/4162571370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c037070 con 0x7fbf8c15db70 2024-04-25T09:54:00.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.729+0000 7fbf8a7fc640 1 -- 172.21.15.19:0/4162571370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c032440 con 0x7fbf8c15db70 2024-04-25T09:54:00.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.729+0000 7fbf8a7fc640 1 -- 172.21.15.19:0/4162571370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c032810 con 0x7fbf8c15db70 2024-04-25T09:54:00.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.730+0000 7fbf92142640 1 -- 172.21.15.19:0/4162571370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf8c15db70 msgr2=0x7fbf8c162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.730+0000 7fbf92142640 1 --2- 172.21.15.19:0/4162571370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf8c15db70 0x7fbf8c162150 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c08c600 tx=0x7fbf7c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.730+0000 7fbf92142640 1 -- 172.21.15.19:0/4162571370 shutdown_connections 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.730+0000 7fbf92142640 1 --2- 172.21.15.19:0/4162571370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf8c0b5a60 0x7fbf8c06c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.730+0000 7fbf92142640 1 --2- 172.21.15.19:0/4162571370 >> v2:172.21.15.112:3300/0 conn(0x7fbf8c0b6cb0 0x7fbf8c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.730+0000 7fbf92142640 1 --2- 172.21.15.19:0/4162571370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf8c15db70 0x7fbf8c162150 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.730+0000 7fbf92142640 1 -- 172.21.15.19:0/4162571370 >> 172.21.15.19:0/4162571370 conn(0x7fbf8c0d0520 msgr2=0x7fbf8c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.730+0000 7fbf92142640 1 -- 172.21.15.19:0/4162571370 shutdown_connections 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.730+0000 7fbf92142640 1 -- 172.21.15.19:0/4162571370 wait complete. 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.731+0000 7fbf92142640 1 Processor -- start 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.731+0000 7fbf92142640 1 -- start start 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf92142640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0b5a60 0x7fbf8c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf92142640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf8c0b6cb0 0x7fbf8c15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf92142640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf8c15db70 0x7fbf8c15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf92142640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0ce530 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf92142640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c15df50 con 0x7fbf8c0b5a60 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf92142640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0cf1c0 con 0x7fbf8c15db70 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf8affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf8c0b6cb0 0x7fbf8c15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf8affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf8c0b6cb0 0x7fbf8c15c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44164/0 (socket says 172.21.15.19:44164) 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf8affd640 1 -- 172.21.15.19:0/4079002624 learned_addr learned my addr 172.21.15.19:0/4079002624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf8b7fe640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0b5a60 0x7fbf8c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf8bfff640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf8c15db70 0x7fbf8c15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf8affd640 1 -- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf8c15db70 msgr2=0x7fbf8c15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf8affd640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf8c15db70 0x7fbf8c15cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf8affd640 1 -- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0b5a60 msgr2=0x7fbf8c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf8affd640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0b5a60 0x7fbf8c15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.732+0000 7fbf8affd640 1 -- 172.21.15.19:0/4079002624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c15f6a0 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.733+0000 7fbf8affd640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf8c0b6cb0 0x7fbf8c15c7c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000c7f0 tx=0x7fbf8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.733+0000 7fbf73fff640 1 -- 172.21.15.19:0/4079002624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80015070 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.733+0000 7fbf73fff640 1 -- 172.21.15.19:0/4079002624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf80007a30 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.733+0000 7fbf92142640 1 -- 172.21.15.19:0/4079002624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8c15f980 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.733+0000 7fbf73fff640 1 -- 172.21.15.19:0/4079002624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80007e00 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.733+0000 7fbf92142640 1 -- 172.21.15.19:0/4079002624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8c15ff40 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.734+0000 7fbf92142640 1 -- 172.21.15.19:0/4079002624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8c068410 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.734+0000 7fbf73fff640 1 -- 172.21.15.19:0/4079002624 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf80018070 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.735+0000 7fbf73fff640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf7806b4d0 0x7fbf7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.735+0000 7fbf73fff640 1 -- 172.21.15.19:0/4079002624 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbf8008a480 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.735+0000 7fbf8b7fe640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf7806b4d0 0x7fbf7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.735+0000 7fbf8b7fe640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf7806b4d0 0x7fbf7806d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c034000 tx=0x7fbf7c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.739+0000 7fbf73fff640 1 -- 172.21.15.19:0/4079002624 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf8c068410 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.955+0000 7fbf92142640 1 -- 172.21.15.19:0/4079002624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf8c0ca400 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.956+0000 7fbf73fff640 1 -- 172.21.15.19:0/4079002624 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fbf80058940 con 0x7fbf8c0b6cb0 2024-04-25T09:54:00.958 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:00.958 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4000846111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":4000846111},{"type":"v1","addr":"172.21.15.112:6837","nonce":4000846111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1969525282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1969525282},{"type":"v1","addr":"172.21.15.19:6835","nonce":1969525282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:07.175350+0000","modified":"2024-04-25T09:53:23.345595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1080352556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1080352556},{"type":"v1","addr":"172.21.15.112:6836","nonce":1080352556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:54:00.958 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T09:54:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.957+0000 7fbf92142640 1 -- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf7806b4d0 msgr2=0x7fbf7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.957+0000 7fbf92142640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf7806b4d0 0x7fbf7806d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c034000 tx=0x7fbf7c02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.957+0000 7fbf92142640 1 -- 172.21.15.19:0/4079002624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf8c0b6cb0 msgr2=0x7fbf8c15c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.957+0000 7fbf92142640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf8c0b6cb0 0x7fbf8c15c7c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000c7f0 tx=0x7fbf8000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.958+0000 7fbf92142640 1 -- 172.21.15.19:0/4079002624 shutdown_connections 2024-04-25T09:54:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.958+0000 7fbf92142640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf7806b4d0 0x7fbf7806d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.958+0000 7fbf92142640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf8c15db70 0x7fbf8c15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.958+0000 7fbf92142640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf8c0b6cb0 0x7fbf8c15c7c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.958+0000 7fbf92142640 1 --2- 172.21.15.19:0/4079002624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0b5a60 0x7fbf8c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.958+0000 7fbf92142640 1 -- 172.21.15.19:0/4079002624 >> 172.21.15.19:0/4079002624 conn(0x7fbf8c0d0520 msgr2=0x7fbf8c1581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.958+0000 7fbf92142640 1 -- 172.21.15.19:0/4079002624 shutdown_connections 2024-04-25T09:54:00.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:00.958+0000 7fbf92142640 1 -- 172.21.15.19:0/4079002624 wait complete. 2024-04-25T09:54:00.971 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:01.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.069+0000 7fc8501b7640 1 Processor -- start 2024-04-25T09:54:01.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.069+0000 7fc8501b7640 1 -- start start 2024-04-25T09:54:01.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.070+0000 7fc8501b7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc84815dbe0 0x7fc8480d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.070+0000 7fc8501b7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc8480b92a0 0x7fc8480d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.070+0000 7fc8501b7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8480b97a0 con 0x7fc8480b2cb0 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.070+0000 7fc8501b7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc84815dfc0 con 0x7fc8480b92a0 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.070+0000 7fc8501b7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480d1a00 con 0x7fc84815dbe0 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.070+0000 7fc84e754640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc8480b92a0 0x7fc8480d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.070+0000 7fc84e754640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc8480b92a0 0x7fc8480d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44172/0 (socket says 172.21.15.19:44172) 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.070+0000 7fc84e754640 1 -- 172.21.15.19:0/1454640547 learned_addr learned my addr 172.21.15.19:0/1454640547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.071+0000 7fc84df53640 1 --2- 172.21.15.19:0/1454640547 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc84815dbe0 0x7fc8480d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.071+0000 7fc84e754640 1 -- 172.21.15.19:0/1454640547 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc84815dbe0 msgr2=0x7fc8480d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.071+0000 7fc84e754640 1 --2- 172.21.15.19:0/1454640547 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc84815dbe0 0x7fc8480d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.071+0000 7fc84e754640 1 -- 172.21.15.19:0/1454640547 >> v1:172.21.15.112:6789/0 conn(0x7fc8480b2cb0 legacy=0x7fc8480b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.071+0000 7fc84e754640 1 -- 172.21.15.19:0/1454640547 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8480d1c50 con 0x7fc8480b92a0 2024-04-25T09:54:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.071+0000 7fc84df53640 1 --2- 172.21.15.19:0/1454640547 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc84815dbe0 0x7fc8480d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.071+0000 7fc84e754640 1 --2- 172.21.15.19:0/1454640547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8480b92a0 0x7fc8480d0e50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc838009870 tx=0x7fc83802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efef0cd12eac6fa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.072+0000 7fc84cf51640 1 -- 172.21.15.19:0/1454640547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838037070 con 0x7fc8480b92a0 2024-04-25T09:54:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.072+0000 7fc84cf51640 1 -- 172.21.15.19:0/1454640547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc838032440 con 0x7fc8480b92a0 2024-04-25T09:54:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.072+0000 7fc84cf51640 1 -- 172.21.15.19:0/1454640547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838032810 con 0x7fc8480b92a0 2024-04-25T09:54:01.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.072+0000 7fc8501b7640 1 -- 172.21.15.19:0/1454640547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8480b92a0 msgr2=0x7fc8480d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.072+0000 7fc8501b7640 1 --2- 172.21.15.19:0/1454640547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8480b92a0 0x7fc8480d0e50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc838009870 tx=0x7fc83802a270 comp rx=0 tx=0).stop 2024-04-25T09:54:01.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.072+0000 7fc8501b7640 1 -- 172.21.15.19:0/1454640547 shutdown_connections 2024-04-25T09:54:01.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.072+0000 7fc8501b7640 1 --2- 172.21.15.19:0/1454640547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8480b92a0 0x7fc8480d0e50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.072+0000 7fc8501b7640 1 --2- 172.21.15.19:0/1454640547 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc84815dbe0 0x7fc8480d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.072+0000 7fc8501b7640 1 -- 172.21.15.19:0/1454640547 >> 172.21.15.19:0/1454640547 conn(0x7fc848061a20 msgr2=0x7fc848061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.072+0000 7fc8501b7640 1 -- 172.21.15.19:0/1454640547 shutdown_connections 2024-04-25T09:54:01.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.072+0000 7fc8501b7640 1 -- 172.21.15.19:0/1454640547 wait complete. 2024-04-25T09:54:01.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.073+0000 7fc8501b7640 1 Processor -- start 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.073+0000 7fc8501b7640 1 -- start start 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.074+0000 7fc8501b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8480b2cb0 0x7fc8480cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.074+0000 7fc8501b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8480b92a0 0x7fc8480cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.074+0000 7fc8501b7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc84815dbe0 0x7fc8481518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.074+0000 7fc8501b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480b6ab0 con 0x7fc8480b92a0 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.074+0000 7fc8501b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480d2210 con 0x7fc8480b2cb0 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.074+0000 7fc8501b7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480b6c30 con 0x7fc84815dbe0 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.074+0000 7fc84d752640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8480b92a0 0x7fc8480cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.074+0000 7fc84d752640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8480b92a0 0x7fc8480cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44188/0 (socket says 172.21.15.19:44188) 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.074+0000 7fc84d752640 1 -- 172.21.15.19:0/3469087756 learned_addr learned my addr 172.21.15.19:0/3469087756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.074+0000 7fc84df53640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8480b2cb0 0x7fc8480cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.074+0000 7fc84e754640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc84815dbe0 0x7fc8481518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc84d752640 1 -- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc84815dbe0 msgr2=0x7fc8481518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc84d752640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc84815dbe0 0x7fc8481518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc84d752640 1 -- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8480b2cb0 msgr2=0x7fc8480cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc84d752640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8480b2cb0 0x7fc8480cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc84d752640 1 -- 172.21.15.19:0/3469087756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8480cc430 con 0x7fc8480b92a0 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc84df53640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8480b2cb0 0x7fc8480cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc84d752640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8480b92a0 0x7fc8480cba50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00c400 tx=0x7fc83c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc8467fc640 1 -- 172.21.15.19:0/3469087756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0052d0 con 0x7fc8480b92a0 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc8467fc640 1 -- 172.21.15.19:0/3469087756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c005490 con 0x7fc8480b92a0 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc8501b7640 1 -- 172.21.15.19:0/3469087756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8480cc710 con 0x7fc8480b92a0 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc8467fc640 1 -- 172.21.15.19:0/3469087756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c007260 con 0x7fc8480b92a0 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.075+0000 7fc8501b7640 1 -- 172.21.15.19:0/3469087756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8480cccd0 con 0x7fc8480b92a0 2024-04-25T09:54:01.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.076+0000 7fc8501b7640 1 -- 172.21.15.19:0/3469087756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc848073990 con 0x7fc8480b92a0 2024-04-25T09:54:01.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.076+0000 7fc8467fc640 1 -- 172.21.15.19:0/3469087756 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc83c007400 con 0x7fc8480b92a0 2024-04-25T09:54:01.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.077+0000 7fc8467fc640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc83406b4d0 0x7fc83406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.077+0000 7fc8467fc640 1 -- 172.21.15.19:0/3469087756 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc83c089b50 con 0x7fc8480b92a0 2024-04-25T09:54:01.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.077+0000 7fc84df53640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc83406b4d0 0x7fc83406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.078+0000 7fc84df53640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc83406b4d0 0x7fc83406d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc840003fd0 tx=0x7fc840003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.081+0000 7fc8467fc640 1 -- 172.21.15.19:0/3469087756 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc83c0585d0 con 0x7fc8480b92a0 2024-04-25T09:54:01.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.296+0000 7fc8501b7640 1 -- 172.21.15.19:0/3469087756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8480d33c0 con 0x7fc8480b92a0 2024-04-25T09:54:01.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.297+0000 7fc8467fc640 1 -- 172.21.15.19:0/3469087756 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fc83c057ed0 con 0x7fc8480b92a0 2024-04-25T09:54:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:01.299 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2974923402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2974923402},{"type":"v1","addr":"172.21.15.19:6833","nonce":2974923402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4000846111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":4000846111},{"type":"v1","addr":"172.21.15.112:6837","nonce":4000846111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1969525282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1969525282},{"type":"v1","addr":"172.21.15.19:6835","nonce":1969525282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:07.175350+0000","modified":"2024-04-25T09:53:23.345595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1080352556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1080352556},{"type":"v1","addr":"172.21.15.112:6836","nonce":1080352556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:54:01.300 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T09:54:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 -- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc83406b4d0 msgr2=0x7fc83406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc83406b4d0 0x7fc83406d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc840003fd0 tx=0x7fc840003990 comp rx=0 tx=0).stop 2024-04-25T09:54:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 -- 172.21.15.19:0/3469087756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8480b92a0 msgr2=0x7fc8480cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8480b92a0 0x7fc8480cba50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00c400 tx=0x7fc83c00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 -- 172.21.15.19:0/3469087756 shutdown_connections 2024-04-25T09:54:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc83406b4d0 0x7fc83406d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc84815dbe0 0x7fc8481518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8480b92a0 0x7fc8480cba50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 --2- 172.21.15.19:0/3469087756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8480b2cb0 0x7fc8480cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 -- 172.21.15.19:0/3469087756 >> 172.21.15.19:0/3469087756 conn(0x7fc848061a20 msgr2=0x7fc8480b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 -- 172.21.15.19:0/3469087756 shutdown_connections 2024-04-25T09:54:01.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.299+0000 7fc8501b7640 1 -- 172.21.15.19:0/3469087756 wait complete. 2024-04-25T09:54:01.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:01.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.410+0000 7f278e9c1640 1 Processor -- start 2024-04-25T09:54:01.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.410+0000 7f278e9c1640 1 -- start start 2024-04-25T09:54:01.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.410+0000 7f278e9c1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f278815dd30 0x7f27880b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.410+0000 7f278e9c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880d0ad0 0x7f27880b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.410+0000 7f278e9c1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27880d1df0 con 0x7f27880b9770 2024-04-25T09:54:01.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f278e9c1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27880d20d0 con 0x7f278815dd30 2024-04-25T09:54:01.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f278e9c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27880d23b0 con 0x7f27880d0ad0 2024-04-25T09:54:01.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f27877fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f27880b9770 0x7f27880d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48358/0 (socket says 172.21.15.19:48358) 2024-04-25T09:54:01.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f27877fe640 1 -- 172.21.15.19:0/2467431465 learned_addr learned my addr 172.21.15.19:0/2467431465 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181258014 0 0) 0x7f27880d1df0 con 0x7f27880b9770 2024-04-25T09:54:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f278cf5e640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880d0ad0 0x7f27880b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2787fff640 1 --2- 172.21.15.19:0/2467431465 >> v2:172.21.15.112:3300/0 conn(0x7f278815dd30 0x7f27880b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2770003520 con 0x7f27880b9770 2024-04-25T09:54:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3084302216 0 0) 0x7f2770003520 con 0x7f27880b9770 2024-04-25T09:54:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880d0ad0 msgr2=0x7f27880b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880d0ad0 0x7f27880b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 >> v2:172.21.15.112:3300/0 conn(0x7f278815dd30 msgr2=0x7f27880b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 --2- 172.21.15.19:0/2467431465 >> v2:172.21.15.112:3300/0 conn(0x7f278815dd30 0x7f27880b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27880d2690 con 0x7f27880b9770 2024-04-25T09:54:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f277c0017f0 con 0x7f27880b9770 2024-04-25T09:54:01.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 >> v1:172.21.15.19:6789/0 conn(0x7f27880b9770 legacy=0x7f27880d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880d0ad0 0x7f27700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770003f30 0x7f2770004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770004d40 0x7f27700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27880d20d0 con 0x7f2770003f30 2024-04-25T09:54:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27880d23b0 con 0x7f2770004d40 2024-04-25T09:54:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277000ba10 con 0x7f27880d0ad0 2024-04-25T09:54:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f27877fe640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770003f30 0x7f2770004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f2787fff640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880d0ad0 0x7f27700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.411+0000 7f278cf5e640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770004d40 0x7f27700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.412+0000 7f27877fe640 1 -- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880d0ad0 msgr2=0x7f27700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.412+0000 7f27877fe640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880d0ad0 0x7f27700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.412+0000 7f27877fe640 1 -- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770004d40 msgr2=0x7f27700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.412+0000 7f27877fe640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770004d40 0x7f27700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.412+0000 7f27877fe640 1 -- 172.21.15.19:0/2467431465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2770003520 con 0x7f2770003f30 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.412+0000 7f278cf5e640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770004d40 0x7f27700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.412+0000 7f27877fe640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770003f30 0x7f2770004800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f277c00c270 tx=0x7f2770005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c206d4b39560f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.412+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c007d30 con 0x7f2770003f30 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.413+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c007270 con 0x7f2770003f30 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.413+0000 7f2786ffd640 1 -- 172.21.15.19:0/2467431465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c0057a0 con 0x7f2770003f30 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.413+0000 7f278e9c1640 1 -- 172.21.15.19:0/2467431465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770003f30 msgr2=0x7f2770004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.413+0000 7f278e9c1640 1 --2- 172.21.15.19:0/2467431465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2770003f30 0x7f2770004800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f277c00c270 tx=0x7f2770005830 comp rx=0 tx=0).stop 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.413+0000 7f278cf5e640 1 -- 172.21.15.19:0/2467431465 reap_dead start 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.413+0000 7f278e9c1640 1 -- 172.21.15.19:0/2467431465 shutdown_connections 2024-04-25T09:54:01.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.413+0000 7f278e9c1640 1 -- 172.21.15.19:0/2467431465 >> 172.21.15.19:0/2467431465 conn(0x7f2788061c30 msgr2=0x7f278806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.413+0000 7f278e9c1640 1 -- 172.21.15.19:0/2467431465 shutdown_connections 2024-04-25T09:54:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.413+0000 7f278e9c1640 1 -- 172.21.15.19:0/2467431465 wait complete. 2024-04-25T09:54:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.414+0000 7f278e9c1640 1 Processor -- start 2024-04-25T09:54:01.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.414+0000 7f278e9c1640 1 -- start start 2024-04-25T09:54:01.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.415+0000 7f278e9c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770003f30 0x7f2788155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.415+0000 7f278e9c1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2788154e70 0x7f2788156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.415+0000 7f278e9c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880b7cd0 0x7f278814d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.415+0000 7f278e9c1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27880d1b90 con 0x7f2788154e70 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.415+0000 7f278e9c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27880d1a10 con 0x7f2770003f30 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.415+0000 7f278e9c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27880d2e90 con 0x7f27880b7cd0 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.415+0000 7f27877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2788154e70 0x7f2788156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.415+0000 7f27877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2788154e70 0x7f2788156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44210/0 (socket says 172.21.15.19:44210) 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.415+0000 7f27877fe640 1 -- 172.21.15.19:0/3948517808 learned_addr learned my addr 172.21.15.19:0/3948517808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.415+0000 7f278cf5e640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880b7cd0 0x7f278814d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f2787fff640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770003f30 0x7f2788155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f27877fe640 1 -- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880b7cd0 msgr2=0x7f278814d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f27877fe640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880b7cd0 0x7f278814d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f27877fe640 1 -- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770003f30 msgr2=0x7f2788155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f27877fe640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770003f30 0x7f2788155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f27877fe640 1 -- 172.21.15.19:0/3948517808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2788155e10 con 0x7f2788154e70 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f27877fe640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2788154e70 0x7f2788156eb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f277c00c390 tx=0x7f27880d2860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f2784ff9640 1 -- 172.21.15.19:0/3948517808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c042070 con 0x7f2788154e70 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f2784ff9640 1 -- 172.21.15.19:0/3948517808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c0059b0 con 0x7f2788154e70 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f278e9c1640 1 -- 172.21.15.19:0/3948517808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27881560f0 con 0x7f2788154e70 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f2784ff9640 1 -- 172.21.15.19:0/3948517808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c005da0 con 0x7f2788154e70 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.416+0000 7f278e9c1640 1 -- 172.21.15.19:0/3948517808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27881566b0 con 0x7f2788154e70 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.417+0000 7f278e9c1640 1 -- 172.21.15.19:0/3948517808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2788073990 con 0x7f2788154e70 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.418+0000 7f2784ff9640 1 -- 172.21.15.19:0/3948517808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f277c045070 con 0x7f2788154e70 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.418+0000 7f2784ff9640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f277406b500 0x7f277406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.418+0000 7f2784ff9640 1 -- 172.21.15.19:0/3948517808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f277c0ab1b0 con 0x7f2788154e70 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.418+0000 7f2787fff640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f277406b500 0x7f277406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.419+0000 7f2787fff640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f277406b500 0x7f277406d9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2778009c40 tx=0x7f2778009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.422+0000 7f2784ff9640 1 -- 172.21.15.19:0/3948517808 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f277c04c030 con 0x7f2788154e70 2024-04-25T09:54:01.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.604+0000 7f278e9c1640 1 -- 172.21.15.19:0/3948517808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f278815d350 con 0x7f2788154e70 2024-04-25T09:54:01.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.606+0000 7f2784ff9640 1 -- 172.21.15.19:0/3948517808 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18892 (secure 0 0 0) 0x7f277c079d80 con 0x7f2788154e70 2024-04-25T09:54:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:01.608 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:53:48.071585+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:05.053469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:06.115537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:09.831285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.607+0000 7f278e9c1640 1 -- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f277406b500 msgr2=0x7f277406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.607+0000 7f278e9c1640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f277406b500 0x7f277406d9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2778009c40 tx=0x7f2778009190 comp rx=0 tx=0).stop 2024-04-25T09:54:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.607+0000 7f278e9c1640 1 -- 172.21.15.19:0/3948517808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2788154e70 msgr2=0x7f2788156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.607+0000 7f278e9c1640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2788154e70 0x7f2788156eb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f277c00c390 tx=0x7f27880d2860 comp rx=0 tx=0).stop 2024-04-25T09:54:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.607+0000 7f278e9c1640 1 -- 172.21.15.19:0/3948517808 shutdown_connections 2024-04-25T09:54:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.607+0000 7f278e9c1640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f277406b500 0x7f277406d9c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.607+0000 7f278e9c1640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27880b7cd0 0x7f278814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.607+0000 7f278e9c1640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2788154e70 0x7f2788156eb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.607+0000 7f278e9c1640 1 --2- 172.21.15.19:0/3948517808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770003f30 0x7f2788155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.608+0000 7f278e9c1640 1 -- 172.21.15.19:0/3948517808 >> 172.21.15.19:0/3948517808 conn(0x7f2788061c30 msgr2=0x7f278815a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.608+0000 7f278e9c1640 1 -- 172.21.15.19:0/3948517808 shutdown_connections 2024-04-25T09:54:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.608+0000 7f278e9c1640 1 -- 172.21.15.19:0/3948517808 wait complete. 2024-04-25T09:54:01.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:54:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.722+0000 7f1ee6af5640 1 Processor -- start 2024-04-25T09:54:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.722+0000 7f1ee6af5640 1 -- start start 2024-04-25T09:54:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ee6af5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1ee00cc4f0 0x7f1ee00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ee6af5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee015b980 0x7f1ee00cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ee6af5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee0072280 con 0x7f1ee00cdb20 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ee6af5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee0071270 con 0x7f1ee00cc4f0 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ee6af5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00714c0 con 0x7f1ee015b980 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed7fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1ee00cdb20 0x7f1ee015b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48370/0 (socket says 172.21.15.19:48370) 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed7fff640 1 -- 172.21.15.19:0/2146864416 learned_addr learned my addr 172.21.15.19:0/2146864416 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ee5092640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee015b980 0x7f1ee00cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525498230 0 0) 0x7f1ee0072280 con 0x7f1ee00cdb20 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ee4891640 1 --2- 172.21.15.19:0/2146864416 >> v2:172.21.15.112:3300/0 conn(0x7f1ee00cc4f0 0x7f1ee00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec8003520 con 0x7f1ee00cdb20 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2648054026 0 0) 0x7f1ec8003520 con 0x7f1ee00cdb20 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee015b980 msgr2=0x7f1ee00cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed77fe640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee015b980 0x7f1ee00cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 >> v2:172.21.15.112:3300/0 conn(0x7f1ee00cc4f0 msgr2=0x7f1ee00cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed77fe640 1 --2- 172.21.15.19:0/2146864416 >> v2:172.21.15.112:3300/0 conn(0x7f1ee00cc4f0 0x7f1ee00cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00717a0 con 0x7f1ee00cdb20 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f1ed0001690 con 0x7f1ee00cdb20 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.723+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 >> v1:172.21.15.19:6789/0 conn(0x7f1ee00cdb20 legacy=0x7f1ee015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ed77fe640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec80039b0 0x7f1ec8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ed77fe640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee015b980 0x7f1ec8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ed77fe640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec8004d40 0x7f1ec80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0071270 con 0x7f1ec80039b0 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00714c0 con 0x7f1ec8004d40 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec800b9c0 con 0x7f1ee015b980 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ee4891640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec80039b0 0x7f1ec8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ed7fff640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee015b980 0x7f1ec8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ee5092640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec8004d40 0x7f1ec80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ee4891640 1 -- 172.21.15.19:0/2146864416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee015b980 msgr2=0x7f1ec8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ee4891640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee015b980 0x7f1ec8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ee4891640 1 -- 172.21.15.19:0/2146864416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec8004d40 msgr2=0x7f1ec80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ee4891640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec8004d40 0x7f1ec80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.724+0000 7f1ee4891640 1 -- 172.21.15.19:0/2146864416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec8003520 con 0x7f1ec80039b0 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.725+0000 7f1ee4891640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec80039b0 0x7f1ec8003dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009ac0 tx=0x7f1ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b7c0d6e6cf8d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.725+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc035070 con 0x7f1ec80039b0 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.725+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc031620 con 0x7f1ec80039b0 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.725+0000 7f1ed77fe640 1 -- 172.21.15.19:0/2146864416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc009690 con 0x7f1ec80039b0 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.725+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2146864416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec80039b0 msgr2=0x7f1ec8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.725+0000 7f1ee6af5640 1 --2- 172.21.15.19:0/2146864416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec80039b0 0x7f1ec8003dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009ac0 tx=0x7f1ec8005830 comp rx=0 tx=0).stop 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.725+0000 7f1ee5092640 1 -- 172.21.15.19:0/2146864416 reap_dead start 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.725+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2146864416 shutdown_connections 2024-04-25T09:54:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.725+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2146864416 >> 172.21.15.19:0/2146864416 conn(0x7f1ee006abe0 msgr2=0x7f1ee006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.725+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2146864416 shutdown_connections 2024-04-25T09:54:01.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.725+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2146864416 wait complete. 2024-04-25T09:54:01.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.727+0000 7f1ee6af5640 1 Processor -- start 2024-04-25T09:54:01.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.727+0000 7f1ee6af5640 1 -- start start 2024-04-25T09:54:01.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.727+0000 7f1ee6af5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec80039b0 0x7f1ee0159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.727+0000 7f1ee6af5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ee0144880 0x7f1ee0144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.727+0000 7f1ee6af5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee00c7ac0 0x7f1ee00c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.727+0000 7f1ee6af5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00cdfb0 con 0x7f1ee0144880 2024-04-25T09:54:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.727+0000 7f1ee6af5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee0071d60 con 0x7f1ec80039b0 2024-04-25T09:54:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.727+0000 7f1ee6af5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00ce130 con 0x7f1ee00c7ac0 2024-04-25T09:54:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ee0144880 0x7f1ee0144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ee0144880 0x7f1ee0144d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44234/0 (socket says 172.21.15.19:44234) 2024-04-25T09:54:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed7fff640 1 -- 172.21.15.19:0/2635177011 learned_addr learned my addr 172.21.15.19:0/2635177011 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ee5092640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee00c7ac0 0x7f1ee00c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ee4891640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec80039b0 0x7f1ee0159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed7fff640 1 -- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee00c7ac0 msgr2=0x7f1ee00c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed7fff640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee00c7ac0 0x7f1ee00c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed7fff640 1 -- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec80039b0 msgr2=0x7f1ee0159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed7fff640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec80039b0 0x7f1ee0159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed7fff640 1 -- 172.21.15.19:0/2635177011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee01486a0 con 0x7f1ee0144880 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ee5092640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee00c7ac0 0x7f1ee00c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed7fff640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ee0144880 0x7f1ee0144d00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0007340 tx=0x7f1ed0007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed57fa640 1 -- 172.21.15.19:0/2635177011 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000f940 con 0x7f1ee0144880 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed57fa640 1 -- 172.21.15.19:0/2635177011 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed0018070 con 0x7f1ee0144880 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2635177011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee0145320 con 0x7f1ee0144880 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ed57fa640 1 -- 172.21.15.19:0/2635177011 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000fd20 con 0x7f1ee0144880 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.728+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2635177011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee0142a00 con 0x7f1ee0144880 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.729+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2635177011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee01427f0 con 0x7f1ee0144880 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.730+0000 7f1ed57fa640 1 -- 172.21.15.19:0/2635177011 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ed001d460 con 0x7f1ee0144880 2024-04-25T09:54:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.730+0000 7f1ed57fa640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ecc06b4d0 0x7f1ecc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.730+0000 7f1ed57fa640 1 -- 172.21.15.19:0/2635177011 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1ed008a6a0 con 0x7f1ee0144880 2024-04-25T09:54:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.730+0000 7f1ee4891640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ecc06b4d0 0x7f1ecc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.731+0000 7f1ee4891640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ecc06b4d0 0x7f1ecc06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1edc037000 tx=0x7f1edc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.734+0000 7f1ed57fa640 1 -- 172.21.15.19:0/2635177011 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ed00591b0 con 0x7f1ee0144880 2024-04-25T09:54:01.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.952+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2635177011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1ee0073a00 con 0x7f1ee0144880 2024-04-25T09:54:01.954 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T09:54:01.952+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:54:01.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.953+0000 7f1ed57fa640 1 -- 172.21.15.19:0/2635177011 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1ed0058ab0 con 0x7f1ee0144880 2024-04-25T09:54:01.955 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.956+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ecc06b4d0 msgr2=0x7f1ecc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.956+0000 7f1ee6af5640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ecc06b4d0 0x7f1ecc06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1edc037000 tx=0x7f1edc008030 comp rx=0 tx=0).stop 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.956+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2635177011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ee0144880 msgr2=0x7f1ee0144d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.956+0000 7f1ee6af5640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ee0144880 0x7f1ee0144d00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0007340 tx=0x7f1ed0007800 comp rx=0 tx=0).stop 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.956+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2635177011 shutdown_connections 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.956+0000 7f1ee6af5640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ecc06b4d0 0x7f1ecc06d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.956+0000 7f1ee6af5640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ee00c7ac0 0x7f1ee00c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.956+0000 7f1ee6af5640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ee0144880 0x7f1ee0144d00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.956+0000 7f1ee6af5640 1 --2- 172.21.15.19:0/2635177011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec80039b0 0x7f1ee0159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.956+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2635177011 >> 172.21.15.19:0/2635177011 conn(0x7f1ee006abe0 msgr2=0x7f1ee00cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.957+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2635177011 shutdown_connections 2024-04-25T09:54:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:01.957+0000 7f1ee6af5640 1 -- 172.21.15.19:0/2635177011 wait complete. 2024-04-25T09:54:01.962 INFO:tasks.ceph.mds.d.smithi112.stderr: -4823> 2024-04-25T09:53:11.557+0000 7fa09dfca640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:54:01.969 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:54:01.994 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T09:54:02.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.070+0000 7ffbd9e7c640 1 Processor -- start 2024-04-25T09:54:02.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.070+0000 7ffbd9e7c640 1 -- start start 2024-04-25T09:54:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd9e7c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffbd40cc4f0 0x7ffbd40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd9e7c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffbd415b980 0x7ffbd40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd9e7c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd4072280 con 0x7ffbd40cdb20 2024-04-25T09:54:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd9e7c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd4071270 con 0x7ffbd40cc4f0 2024-04-25T09:54:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd9e7c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd415bd80 con 0x7ffbd415b980 2024-04-25T09:54:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd37fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffbd40cc4f0 0x7ffbd40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd2ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ffbd40cdb20 0x7ffbd415b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48374/0 (socket says 172.21.15.19:48374) 2024-04-25T09:54:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd2ffd640 1 -- 172.21.15.19:0/1085438137 learned_addr learned my addr 172.21.15.19:0/1085438137 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd3fff640 1 --2- 172.21.15.19:0/1085438137 >> v2:172.21.15.112:3300/0 conn(0x7ffbd415b980 0x7ffbd40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd27fc640 1 -- 172.21.15.19:0/1085438137 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617269639 0 0) 0x7ffbd4072280 con 0x7ffbd40cdb20 2024-04-25T09:54:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd27fc640 1 -- 172.21.15.19:0/1085438137 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbbc003520 con 0x7ffbd40cdb20 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd37fe640 1 -- 172.21.15.19:0/1085438137 >> v2:172.21.15.112:3300/0 conn(0x7ffbd415b980 msgr2=0x7ffbd40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd37fe640 1 --2- 172.21.15.19:0/1085438137 >> v2:172.21.15.112:3300/0 conn(0x7ffbd415b980 0x7ffbd40cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd37fe640 1 -- 172.21.15.19:0/1085438137 >> v1:172.21.15.19:6789/0 conn(0x7ffbd40cdb20 legacy=0x7ffbd415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.071+0000 7ffbd37fe640 1 -- 172.21.15.19:0/1085438137 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd40714c0 con 0x7ffbd40cc4f0 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.072+0000 7ffbd37fe640 1 --2- 172.21.15.19:0/1085438137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbd40cc4f0 0x7ffbd40cc8d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40b4c00 tx=0x7ffbc402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50be048f93441966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.072+0000 7ffbd27fc640 1 -- 172.21.15.19:0/1085438137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4037070 con 0x7ffbd40cc4f0 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.072+0000 7ffbd27fc640 1 -- 172.21.15.19:0/1085438137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc4032440 con 0x7ffbd40cc4f0 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.072+0000 7ffbd27fc640 1 -- 172.21.15.19:0/1085438137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4032810 con 0x7ffbd40cc4f0 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.072+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/1085438137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbd40cc4f0 msgr2=0x7ffbd40cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.073+0000 7ffbd9e7c640 1 --2- 172.21.15.19:0/1085438137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbd40cc4f0 0x7ffbd40cc8d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40b4c00 tx=0x7ffbc402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.073+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/1085438137 shutdown_connections 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.073+0000 7ffbd9e7c640 1 --2- 172.21.15.19:0/1085438137 >> v2:172.21.15.112:3300/0 conn(0x7ffbd415b980 0x7ffbd40cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.073+0000 7ffbd9e7c640 1 --2- 172.21.15.19:0/1085438137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbd40cc4f0 0x7ffbd40cc8d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.073+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/1085438137 >> 172.21.15.19:0/1085438137 conn(0x7ffbd406abe0 msgr2=0x7ffbd406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:02.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.073+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/1085438137 shutdown_connections 2024-04-25T09:54:02.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.073+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/1085438137 wait complete. 2024-04-25T09:54:02.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.074+0000 7ffbd9e7c640 1 Processor -- start 2024-04-25T09:54:02.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.074+0000 7ffbd9e7c640 1 -- start start 2024-04-25T09:54:02.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd9e7c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbd40cc4f0 0x7ffbd41584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd9e7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbd40cdb20 0x7ffbd4152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd9e7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd415b980 0x7ffbd4158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd9e7c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4071d60 con 0x7ffbd40cdb20 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd9e7c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4071a80 con 0x7ffbd415b980 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd9e7c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40ae500 con 0x7ffbd40cc4f0 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbd40cdb20 0x7ffbd4152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbd40cdb20 0x7ffbd4152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44254/0 (socket says 172.21.15.19:44254) 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd2ffd640 1 -- 172.21.15.19:0/211601070 learned_addr learned my addr 172.21.15.19:0/211601070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd3fff640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd415b980 0x7ffbd4158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd37fe640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbd40cc4f0 0x7ffbd41584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd2ffd640 1 -- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbd40cc4f0 msgr2=0x7ffbd41584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd2ffd640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbd40cc4f0 0x7ffbd41584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd2ffd640 1 -- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd415b980 msgr2=0x7ffbd4158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd2ffd640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd415b980 0x7ffbd4158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd2ffd640 1 -- 172.21.15.19:0/211601070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd4155730 con 0x7ffbd40cdb20 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd3fff640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd415b980 0x7ffbd4158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd37fe640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbd40cc4f0 0x7ffbd41584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbd2ffd640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbd40cdb20 0x7ffbd4152ff0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c430 tx=0x7ffbc800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.075+0000 7ffbbbfff640 1 -- 172.21.15.19:0/211601070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8017070 con 0x7ffbd40cdb20 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.076+0000 7ffbbbfff640 1 -- 172.21.15.19:0/211601070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc8005ce0 con 0x7ffbd40cdb20 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.076+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/211601070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd4155a10 con 0x7ffbd40cdb20 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.076+0000 7ffbbbfff640 1 -- 172.21.15.19:0/211601070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc80072f0 con 0x7ffbd40cdb20 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.076+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/211601070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd4156190 con 0x7ffbd40cdb20 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.076+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/211601070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd4073a00 con 0x7ffbd40cdb20 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.077+0000 7ffbbbfff640 1 -- 172.21.15.19:0/211601070 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbc801a070 con 0x7ffbd40cdb20 2024-04-25T09:54:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.077+0000 7ffbbbfff640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbc006b4d0 0x7ffbc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.077+0000 7ffbbbfff640 1 -- 172.21.15.19:0/211601070 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6821+0+0 (secure 0 0 0) 0x7ffbc8089b10 con 0x7ffbd40cdb20 2024-04-25T09:54:02.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.078+0000 7ffbd37fe640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbc006b4d0 0x7ffbc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.078+0000 7ffbd37fe640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbc006b4d0 0x7ffbc006d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4034000 tx=0x7ffbc402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.082+0000 7ffbbbfff640 1 -- 172.21.15.19:0/211601070 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbc8058530 con 0x7ffbd40cdb20 2024-04-25T09:54:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.300+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/211601070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffbd406d3a0 con 0x7ffbd40cdb20 2024-04-25T09:54:02.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.957+0000 7ffbbbfff640 1 -- 172.21.15.19:0/211601070 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffbc8057e30 con 0x7ffbd40cdb20 2024-04-25T09:54:02.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.959+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbc006b4d0 msgr2=0x7ffbc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.959+0000 7ffbd9e7c640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbc006b4d0 0x7ffbc006d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4034000 tx=0x7ffbc402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.959+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/211601070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbd40cdb20 msgr2=0x7ffbd4152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.959+0000 7ffbd9e7c640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbd40cdb20 0x7ffbd4152ff0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c430 tx=0x7ffbc800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.959+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/211601070 shutdown_connections 2024-04-25T09:54:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.959+0000 7ffbd9e7c640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbc006b4d0 0x7ffbc006d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.959+0000 7ffbd9e7c640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd415b980 0x7ffbd4158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.959+0000 7ffbd9e7c640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbd40cdb20 0x7ffbd4152ff0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.959+0000 7ffbd9e7c640 1 --2- 172.21.15.19:0/211601070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbd40cc4f0 0x7ffbd41584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.959+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/211601070 >> 172.21.15.19:0/211601070 conn(0x7ffbd406abe0 msgr2=0x7ffbd40cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:02.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.959+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/211601070 shutdown_connections 2024-04-25T09:54:02.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:02.960+0000 7ffbd9e7c640 1 -- 172.21.15.19:0/211601070 wait complete. 2024-04-25T09:54:02.975 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:54:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.083+0000 7f81a7130640 1 Processor -- start 2024-04-25T09:54:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.084+0000 7f81a7130640 1 -- start start 2024-04-25T09:54:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.084+0000 7f81a7130640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f81a00ce7b0 0x7f81a0161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.084+0000 7f81a7130640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a006c8d0 con 0x7f81a01610f0 2024-04-25T09:54:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.084+0000 7f81a7130640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a00c6960 con 0x7f81a01616f0 2024-04-25T09:54:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.084+0000 7f81a7130640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a01614d0 con 0x7f81a00ce7b0 2024-04-25T09:54:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.084+0000 7f81a4ecc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f81a01610f0 0x7f81a006c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48386/0 (socket says 172.21.15.19:48386) 2024-04-25T09:54:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.085+0000 7f81a4ecc640 1 -- 172.21.15.19:0/1303107967 learned_addr learned my addr 172.21.15.19:0/1303107967 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.085+0000 7f81a56cd640 1 --2- 172.21.15.19:0/1303107967 >> v2:172.21.15.19:3300/0 conn(0x7f81a00ce7b0 0x7f81a0161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.085+0000 7f819f7fe640 1 -- 172.21.15.19:0/1303107967 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331888996 0 0) 0x7f81a006c8d0 con 0x7f81a01610f0 2024-04-25T09:54:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.085+0000 7f819f7fe640 1 -- 172.21.15.19:0/1303107967 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8190003520 con 0x7f81a01610f0 2024-04-25T09:54:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.085+0000 7f81a56cd640 1 -- 172.21.15.19:0/1303107967 >> v1:172.21.15.112:6789/0 conn(0x7f81a01616f0 legacy=0x7f81a00ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.085+0000 7f81a56cd640 1 -- 172.21.15.19:0/1303107967 >> v1:172.21.15.19:6789/0 conn(0x7f81a01610f0 legacy=0x7f81a006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.085+0000 7f81a56cd640 1 -- 172.21.15.19:0/1303107967 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a0162260 con 0x7f81a00ce7b0 2024-04-25T09:54:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.086+0000 7f81a56cd640 1 --2- 172.21.15.19:0/1303107967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a00ce7b0 0x7f81a0161b50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8198009d90 tx=0x7f819802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db5a3c6c950ef1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.086+0000 7f819f7fe640 1 -- 172.21.15.19:0/1303107967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198007740 con 0x7f81a00ce7b0 2024-04-25T09:54:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.086+0000 7f819f7fe640 1 -- 172.21.15.19:0/1303107967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81980078e0 con 0x7f81a00ce7b0 2024-04-25T09:54:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.086+0000 7f819f7fe640 1 -- 172.21.15.19:0/1303107967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198007cb0 con 0x7f81a00ce7b0 2024-04-25T09:54:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.086+0000 7f81a7130640 1 -- 172.21.15.19:0/1303107967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a00ce7b0 msgr2=0x7f81a0161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.086+0000 7f81a7130640 1 --2- 172.21.15.19:0/1303107967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a00ce7b0 0x7f81a0161b50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8198009d90 tx=0x7f819802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.086+0000 7f81a7130640 1 -- 172.21.15.19:0/1303107967 shutdown_connections 2024-04-25T09:54:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.086+0000 7f81a7130640 1 --2- 172.21.15.19:0/1303107967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a00ce7b0 0x7f81a0161b50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.086+0000 7f81a7130640 1 -- 172.21.15.19:0/1303107967 >> 172.21.15.19:0/1303107967 conn(0x7f81a006f500 msgr2=0x7f81a006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.086+0000 7f81a7130640 1 -- 172.21.15.19:0/1303107967 shutdown_connections 2024-04-25T09:54:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.086+0000 7f81a7130640 1 -- 172.21.15.19:0/1303107967 wait complete. 2024-04-25T09:54:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.088+0000 7f81a7130640 1 Processor -- start 2024-04-25T09:54:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.088+0000 7f81a7130640 1 -- start start 2024-04-25T09:54:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.088+0000 7f81a7130640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a00ce7b0 0x7f81a0156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.088+0000 7f81a7130640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a01610f0 0x7f81a0157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.088+0000 7f81a7130640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a01616f0 0x7f81a015f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a7130640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00cfce0 con 0x7f81a01616f0 2024-04-25T09:54:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a7130640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00cfa00 con 0x7f81a00ce7b0 2024-04-25T09:54:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a7130640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00cf690 con 0x7f81a01610f0 2024-04-25T09:54:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a56cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a01616f0 0x7f81a015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a56cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a01616f0 0x7f81a015f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44274/0 (socket says 172.21.15.19:44274) 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a56cd640 1 -- 172.21.15.19:0/1789869732 learned_addr learned my addr 172.21.15.19:0/1789869732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a4ecc640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a00ce7b0 0x7f81a0156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a56cd640 1 -- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a01610f0 msgr2=0x7f81a0157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f819ffff640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a01610f0 0x7f81a0157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a56cd640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a01610f0 0x7f81a0157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a56cd640 1 -- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a00ce7b0 msgr2=0x7f81a0156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a56cd640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a00ce7b0 0x7f81a0156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a56cd640 1 -- 172.21.15.19:0/1789869732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a01584c0 con 0x7f81a01616f0 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f819ffff640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a01610f0 0x7f81a0157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a4ecc640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a00ce7b0 0x7f81a0156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a56cd640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a01616f0 0x7f81a015f730 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8198009eb0 tx=0x7f8198004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f819d7fa640 1 -- 172.21.15.19:0/1789869732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819803f070 con 0x7f81a01616f0 2024-04-25T09:54:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f819d7fa640 1 -- 172.21.15.19:0/1789869732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8198031bc0 con 0x7f81a01616f0 2024-04-25T09:54:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a7130640 1 -- 172.21.15.19:0/1789869732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a01587a0 con 0x7f81a01616f0 2024-04-25T09:54:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f819d7fa640 1 -- 172.21.15.19:0/1789869732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198009450 con 0x7f81a01616f0 2024-04-25T09:54:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.089+0000 7f81a7130640 1 -- 172.21.15.19:0/1789869732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a0158fe0 con 0x7f81a01616f0 2024-04-25T09:54:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.090+0000 7f81a7130640 1 -- 172.21.15.19:0/1789869732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a0073a00 con 0x7f81a01616f0 2024-04-25T09:54:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.091+0000 7f819d7fa640 1 -- 172.21.15.19:0/1789869732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f819803b410 con 0x7f81a01616f0 2024-04-25T09:54:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.091+0000 7f819d7fa640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 0x7f818406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.091+0000 7f819d7fa640 1 -- 172.21.15.19:0/1789869732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f81980abdb0 con 0x7f81a01616f0 2024-04-25T09:54:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.091+0000 7f81a4ecc640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 0x7f818406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.092+0000 7f81a4ecc640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 0x7f818406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f81940011e0 tx=0x7f8194009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:03.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.095+0000 7f819d7fa640 1 -- 172.21.15.19:0/1789869732 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f819807a8f0 con 0x7f81a01616f0 2024-04-25T09:54:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:03.280+0000 7f81a7130640 1 -- 172.21.15.19:0/1789869732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f81a00c7800 con 0x7f81a01616f0 2024-04-25T09:54:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.148+0000 7f819d7fa640 1 -- 172.21.15.19:0/1789869732 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f819807a1f0 con 0x7f81a01616f0 2024-04-25T09:54:04.151 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:54:04.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 -- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 msgr2=0x7f818406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 0x7f818406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f81940011e0 tx=0x7f8194009040 comp rx=0 tx=0).stop 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 -- 172.21.15.19:0/1789869732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a01616f0 msgr2=0x7f81a015f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a01616f0 0x7f81a015f730 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8198009eb0 tx=0x7f8198004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 -- 172.21.15.19:0/1789869732 shutdown_connections 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 0x7f818406d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a01616f0 0x7f81a015f730 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a01610f0 0x7f81a0157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 --2- 172.21.15.19:0/1789869732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a00ce7b0 0x7f81a0156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 -- 172.21.15.19:0/1789869732 >> 172.21.15.19:0/1789869732 conn(0x7f81a006f500 msgr2=0x7f81a00d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 -- 172.21.15.19:0/1789869732 shutdown_connections 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.152+0000 7f81a7130640 1 -- 172.21.15.19:0/1789869732 wait complete. 2024-04-25T09:54:04.169 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:54:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.278+0000 7f5a1f1cf640 1 Processor -- start 2024-04-25T09:54:04.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.278+0000 7f5a1f1cf640 1 -- start start 2024-04-25T09:54:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.279+0000 7f5a1f1cf640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5a1815dc20 0x7f5a180ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.279+0000 7f5a1f1cf640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1806cda0 con 0x7f5a18161710 2024-04-25T09:54:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.279+0000 7f5a1f1cf640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1815e000 con 0x7f5a180d32f0 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.279+0000 7f5a1f1cf640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a180d1de0 con 0x7f5a1815dc20 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.279+0000 7f5a17fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5a18161710 0x7f5a1806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48388/0 (socket says 172.21.15.19:48388) 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.279+0000 7f5a17fff640 1 -- 172.21.15.19:0/3062264443 learned_addr learned my addr 172.21.15.19:0/3062264443 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.279+0000 7f5a1cf6b640 1 --2- 172.21.15.19:0/3062264443 >> v2:172.21.15.112:3300/0 conn(0x7f5a1815dc20 0x7f5a180ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708889300 0 0) 0x7f5a1806cda0 con 0x7f5a18161710 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a00003520 con 0x7f5a18161710 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252196120 0 0) 0x7f5a1815e000 con 0x7f5a180d32f0 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a1806cda0 con 0x7f5a180d32f0 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023210084 0 0) 0x7f5a00003520 con 0x7f5a18161710 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 >> v2:172.21.15.112:3300/0 conn(0x7f5a1815dc20 msgr2=0x7f5a180ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 --2- 172.21.15.19:0/3062264443 >> v2:172.21.15.112:3300/0 conn(0x7f5a1815dc20 0x7f5a180ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 >> v1:172.21.15.112:6789/0 conn(0x7f5a180d32f0 legacy=0x7f5a1806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a18161c60 con 0x7f5a18161710 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f5a0c001670 con 0x7f5a18161710 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a1cf6b640 1 --2- 172.21.15.19:0/3062264443 >> v2:172.21.15.112:3300/0 conn(0x7f5a1815dc20 0x7f5a180ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 >> v1:172.21.15.19:6789/0 conn(0x7f5a18161710 legacy=0x7f5a1806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a00003b40 0x7f5a00003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a00004800 0x7f5a00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 0x7f5a0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180d1de0 con 0x7f5a000051c0 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0000c040 con 0x7f5a00004800 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0000c240 con 0x7f5a00003b40 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a1d76c640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 0x7f5a0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a1cf6b640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a00003b40 0x7f5a00003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a17fff640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a00004800 0x7f5a00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a1d76c640 1 -- 172.21.15.19:0/3062264443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a00003b40 msgr2=0x7f5a00003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a1d76c640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a00003b40 0x7f5a00003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a1d76c640 1 -- 172.21.15.19:0/3062264443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a00004800 msgr2=0x7f5a00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a1d76c640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a00004800 0x7f5a00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a1d76c640 1 -- 172.21.15.19:0/3062264443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a00003520 con 0x7f5a000051c0 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.280+0000 7f5a17fff640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a00004800 0x7f5a00004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.281+0000 7f5a1d76c640 1 -- 172.21.15.19:0/3062264443 reap_dead start 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.281+0000 7f5a1d76c640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 0x7f5a0000b9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5a080018a0 tx=0x7f5a00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd73177f285690f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.281+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a080014f0 con 0x7f5a000051c0 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.281+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a080052d0 con 0x7f5a000051c0 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.281+0000 7f5a177fe640 1 -- 172.21.15.19:0/3062264443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0802ac80 con 0x7f5a000051c0 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.282+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/3062264443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 msgr2=0x7f5a0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.282+0000 7f5a1f1cf640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 0x7f5a0000b9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5a080018a0 tx=0x7f5a00005830 comp rx=0 tx=0).stop 2024-04-25T09:54:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.282+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/3062264443 shutdown_connections 2024-04-25T09:54:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.282+0000 7f5a1f1cf640 1 --2- 172.21.15.19:0/3062264443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 0x7f5a0000b9c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.282+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/3062264443 >> 172.21.15.19:0/3062264443 conn(0x7f5a180d0420 msgr2=0x7f5a180d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.282+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/3062264443 shutdown_connections 2024-04-25T09:54:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.282+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/3062264443 wait complete. 2024-04-25T09:54:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.283+0000 7f5a1f1cf640 1 Processor -- start 2024-04-25T09:54:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.283+0000 7f5a1f1cf640 1 -- start start 2024-04-25T09:54:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1f1cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 0x7f5a180cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1f1cf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a180cb4d0 0x7f5a180c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1f1cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180c8c60 0x7f5a180b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1f1cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a18162500 con 0x7f5a000051c0 2024-04-25T09:54:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1f1cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a18162220 con 0x7f5a180c8c60 2024-04-25T09:54:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1f1cf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180b5e50 con 0x7f5a180cb4d0 2024-04-25T09:54:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1cf6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 0x7f5a180cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1cf6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 0x7f5a180cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44292/0 (socket says 172.21.15.19:44292) 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1cf6b640 1 -- 172.21.15.19:0/2264664083 learned_addr learned my addr 172.21.15.19:0/2264664083 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a17fff640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a180cb4d0 0x7f5a180c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1d76c640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180c8c60 0x7f5a180b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1cf6b640 1 -- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a180cb4d0 msgr2=0x7f5a180c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1cf6b640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a180cb4d0 0x7f5a180c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1cf6b640 1 -- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180c8c60 msgr2=0x7f5a180b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1cf6b640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180c8c60 0x7f5a180b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.284+0000 7f5a1cf6b640 1 -- 172.21.15.19:0/2264664083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a180c91e0 con 0x7f5a000051c0 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.285+0000 7f5a1cf6b640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 0x7f5a180cd490 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000f460 tx=0x7f5a1000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.285+0000 7f5a157fa640 1 -- 172.21.15.19:0/2264664083 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10003b90 con 0x7f5a000051c0 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.285+0000 7f5a157fa640 1 -- 172.21.15.19:0/2264664083 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a10003d50 con 0x7f5a000051c0 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.285+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/2264664083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a180cc6d0 con 0x7f5a000051c0 2024-04-25T09:54:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.285+0000 7f5a157fa640 1 -- 172.21.15.19:0/2264664083 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10004270 con 0x7f5a000051c0 2024-04-25T09:54:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.285+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/2264664083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a180ccc90 con 0x7f5a000051c0 2024-04-25T09:54:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.286+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/2264664083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a18073a00 con 0x7f5a000051c0 2024-04-25T09:54:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.286+0000 7f5a157fa640 1 -- 172.21.15.19:0/2264664083 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a10004410 con 0x7f5a000051c0 2024-04-25T09:54:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.287+0000 7f5a157fa640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5a0406b4d0 0x7f5a0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.287+0000 7f5a157fa640 1 -- 172.21.15.19:0/2264664083 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6383+0+0 (secure 0 0 0) 0x7f5a1008b4d0 con 0x7f5a000051c0 2024-04-25T09:54:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.287+0000 7f5a17fff640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5a0406b4d0 0x7f5a0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.287+0000 7f5a17fff640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5a0406b4d0 0x7f5a0406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a181521e0 tx=0x7f5a0c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.291+0000 7f5a157fa640 1 -- 172.21.15.19:0/2264664083 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a10019070 con 0x7f5a000051c0 2024-04-25T09:54:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:04.470+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/2264664083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a180c8500 con 0x7f5a000051c0 2024-04-25T09:54:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.154+0000 7f5a157fa640 1 -- 172.21.15.19:0/2264664083 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5a10010040 con 0x7f5a000051c0 2024-04-25T09:54:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T09:54:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.157+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5a0406b4d0 msgr2=0x7f5a0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.157+0000 7f5a1f1cf640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5a0406b4d0 0x7f5a0406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a181521e0 tx=0x7f5a0c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.157+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/2264664083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 msgr2=0x7f5a180cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.157+0000 7f5a1f1cf640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 0x7f5a180cd490 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000f460 tx=0x7f5a1000f920 comp rx=0 tx=0).stop 2024-04-25T09:54:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.158+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/2264664083 shutdown_connections 2024-04-25T09:54:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.158+0000 7f5a1f1cf640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5a0406b4d0 0x7f5a0406d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.158+0000 7f5a1f1cf640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a180c8c60 0x7f5a180b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.158+0000 7f5a1f1cf640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a180cb4d0 0x7f5a180c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.158+0000 7f5a1f1cf640 1 --2- 172.21.15.19:0/2264664083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a000051c0 0x7f5a180cd490 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.158+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/2264664083 >> 172.21.15.19:0/2264664083 conn(0x7f5a180d0420 msgr2=0x7f5a1815cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:05.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.158+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/2264664083 shutdown_connections 2024-04-25T09:54:05.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.158+0000 7f5a1f1cf640 1 -- 172.21.15.19:0/2264664083 wait complete. 2024-04-25T09:54:05.174 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T09:54:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.283+0000 7f8ef7153640 1 Processor -- start 2024-04-25T09:54:05.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.283+0000 7f8ef7153640 1 -- start start 2024-04-25T09:54:05.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.284+0000 7f8ef7153640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8ef01616f0 0x7f8ef0161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.284+0000 7f8ef7153640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef006c8d0 con 0x7f8ef00ce0a0 2024-04-25T09:54:05.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.284+0000 7f8ef7153640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef00c6960 con 0x7f8ef01610f0 2024-04-25T09:54:05.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.284+0000 7f8ef7153640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef01614d0 con 0x7f8ef01616f0 2024-04-25T09:54:05.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.284+0000 7f8ef56f0640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8ef00ce0a0 0x7f8ef00d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48392/0 (socket says 172.21.15.19:48392) 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.284+0000 7f8ef56f0640 1 -- 172.21.15.19:0/1482545357 learned_addr learned my addr 172.21.15.19:0/1482545357 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eeffff640 1 --2- 172.21.15.19:0/1482545357 >> v2:172.21.15.112:3300/0 conn(0x7f8ef01616f0 0x7f8ef0161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077607970 0 0) 0x7f8ef006c8d0 con 0x7f8ef00ce0a0 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ed8003520 con 0x7f8ef00ce0a0 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3693134566 0 0) 0x7f8ed8003520 con 0x7f8ef00ce0a0 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 >> v2:172.21.15.112:3300/0 conn(0x7f8ef01616f0 msgr2=0x7f8ef0161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 --2- 172.21.15.19:0/1482545357 >> v2:172.21.15.112:3300/0 conn(0x7f8ef01616f0 0x7f8ef0161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 >> v1:172.21.15.112:6789/0 conn(0x7f8ef01610f0 legacy=0x7f8ef006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef0162260 con 0x7f8ef00ce0a0 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8ee80019f0 con 0x7f8ef00ce0a0 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 >> v1:172.21.15.19:6789/0 conn(0x7f8ef00ce0a0 legacy=0x7f8ef00d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 0x7f8ed8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ed8004800 0x7f8ed8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed80051c0 0x7f8ed800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef01614d0 con 0x7f8ed80039b0 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed800c060 con 0x7f8ed80051c0 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed800c260 con 0x7f8ed8004800 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8ef4eef640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 0x7f8ed8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8ef56f0640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed80051c0 0x7f8ed800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8eeffff640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ed8004800 0x7f8ed8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8ef4eef640 1 -- 172.21.15.19:0/1482545357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ed8004800 msgr2=0x7f8ed8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8ef4eef640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ed8004800 0x7f8ed8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8ef4eef640 1 -- 172.21.15.19:0/1482545357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed80051c0 msgr2=0x7f8ed800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8ef4eef640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed80051c0 0x7f8ed800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8ef4eef640 1 -- 172.21.15.19:0/1482545357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed8003520 con 0x7f8ed80039b0 2024-04-25T09:54:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8ef56f0640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed80051c0 0x7f8ed800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.285+0000 7f8ef56f0640 1 -- 172.21.15.19:0/1482545357 reap_dead start 2024-04-25T09:54:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.286+0000 7f8ef4eef640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 0x7f8ed8003dd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0001750 tx=0x7f8ed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7819b4a7dee4c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.286+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0001370 con 0x7f8ed80039b0 2024-04-25T09:54:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.286+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee00052d0 con 0x7f8ed80039b0 2024-04-25T09:54:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.286+0000 7f8eef7fe640 1 -- 172.21.15.19:0/1482545357 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee002abc0 con 0x7f8ed80039b0 2024-04-25T09:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.287+0000 7f8ef7153640 1 -- 172.21.15.19:0/1482545357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 msgr2=0x7f8ed8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.287+0000 7f8ef7153640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 0x7f8ed8003dd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0001750 tx=0x7f8ed8005830 comp rx=0 tx=0).stop 2024-04-25T09:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.287+0000 7f8ef7153640 1 -- 172.21.15.19:0/1482545357 shutdown_connections 2024-04-25T09:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.287+0000 7f8ef7153640 1 --2- 172.21.15.19:0/1482545357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 0x7f8ed8003dd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.287+0000 7f8ef7153640 1 -- 172.21.15.19:0/1482545357 >> 172.21.15.19:0/1482545357 conn(0x7f8ef006f500 msgr2=0x7f8ef006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.287+0000 7f8ef7153640 1 -- 172.21.15.19:0/1482545357 shutdown_connections 2024-04-25T09:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.287+0000 7f8ef7153640 1 -- 172.21.15.19:0/1482545357 wait complete. 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.288+0000 7f8ef7153640 1 Processor -- start 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.288+0000 7f8ef7153640 1 -- start start 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef7153640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 0x7f8ef0156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef7153640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ef01554b0 0x7f8ef0147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef7153640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef015fb40 0x7f8ef015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef7153640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef00cfce0 con 0x7f8ed80039b0 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef7153640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef00cfa00 con 0x7f8ef01554b0 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef7153640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef00cf690 con 0x7f8ef015fb40 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef4eef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 0x7f8ef0156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef4eef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 0x7f8ef0156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44300/0 (socket says 172.21.15.19:44300) 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef4eef640 1 -- 172.21.15.19:0/1837168620 learned_addr learned my addr 172.21.15.19:0/1837168620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8eeffff640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ef01554b0 0x7f8ef0147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef56f0640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef015fb40 0x7f8ef015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef4eef640 1 -- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef015fb40 msgr2=0x7f8ef015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef4eef640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef015fb40 0x7f8ef015cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef4eef640 1 -- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ef01554b0 msgr2=0x7f8ef0147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef4eef640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ef01554b0 0x7f8ef0147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8ef4eef640 1 -- 172.21.15.19:0/1837168620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef0160500 con 0x7f8ed80039b0 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.289+0000 7f8eeffff640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ef01554b0 0x7f8ef0147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.290+0000 7f8ef4eef640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 0x7f8ef0156ba0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0001870 tx=0x7f8ee0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.290+0000 7f8eed7fa640 1 -- 172.21.15.19:0/1837168620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0040030 con 0x7f8ed80039b0 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.290+0000 7f8eed7fa640 1 -- 172.21.15.19:0/1837168620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee002be50 con 0x7f8ed80039b0 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.290+0000 7f8ef7153640 1 -- 172.21.15.19:0/1837168620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef00b5840 con 0x7f8ed80039b0 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.290+0000 7f8eed7fa640 1 -- 172.21.15.19:0/1837168620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0001370 con 0x7f8ed80039b0 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.290+0000 7f8ef7153640 1 -- 172.21.15.19:0/1837168620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef015ffc0 con 0x7f8ed80039b0 2024-04-25T09:54:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.291+0000 7f8ef7153640 1 -- 172.21.15.19:0/1837168620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef0160250 con 0x7f8ed80039b0 2024-04-25T09:54:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.291+0000 7f8eed7fa640 1 -- 172.21.15.19:0/1837168620 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ee0043070 con 0x7f8ed80039b0 2024-04-25T09:54:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.292+0000 7f8eed7fa640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8edc06b500 0x7f8edc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.292+0000 7f8eed7fa640 1 -- 172.21.15.19:0/1837168620 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8ee00a9ad0 con 0x7f8ed80039b0 2024-04-25T09:54:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.292+0000 7f8eeffff640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8edc06b500 0x7f8edc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.292+0000 7f8eeffff640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8edc06b500 0x7f8edc06d9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00713b0 tx=0x7f8ee4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.296+0000 7f8eed7fa640 1 -- 172.21.15.19:0/1837168620 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ee004b020 con 0x7f8ed80039b0 2024-04-25T09:54:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:05.481+0000 7f8ef7153640 1 -- 172.21.15.19:0/1837168620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ef0073a00 con 0x7f8ed80039b0 2024-04-25T09:54:06.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.155+0000 7f8eed7fa640 1 -- 172.21.15.19:0/1837168620 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7f8ee00787b0 con 0x7f8ed80039b0 2024-04-25T09:54:06.157 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data_ec' removed 2024-04-25T09:54:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 -- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8edc06b500 msgr2=0x7f8edc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8edc06b500 0x7f8edc06d9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00713b0 tx=0x7f8ee4008040 comp rx=0 tx=0).stop 2024-04-25T09:54:06.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 -- 172.21.15.19:0/1837168620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 msgr2=0x7f8ef0156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 0x7f8ef0156ba0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0001870 tx=0x7f8ee0001120 comp rx=0 tx=0).stop 2024-04-25T09:54:06.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 -- 172.21.15.19:0/1837168620 shutdown_connections 2024-04-25T09:54:06.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8edc06b500 0x7f8edc06d9c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef015fb40 0x7f8ef015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ef01554b0 0x7f8ef0147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 --2- 172.21.15.19:0/1837168620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ed80039b0 0x7f8ef0156ba0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 -- 172.21.15.19:0/1837168620 >> 172.21.15.19:0/1837168620 conn(0x7f8ef006f500 msgr2=0x7f8ef00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 -- 172.21.15.19:0/1837168620 shutdown_connections 2024-04-25T09:54:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.159+0000 7f8ef7153640 1 -- 172.21.15.19:0/1837168620 wait complete. 2024-04-25T09:54:06.176 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:54:06.177 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:54:06.177 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:54:06.177 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:06.177 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:54:06.177 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:54:06.177 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:54:06.177 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:06.178 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:54:06.178 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:54:06.178 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:54:06.178 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:06.178 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:54:06.178 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:54:06.178 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:54:06.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:06.189 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:54:06.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:54:06.190 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:54:06.190 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:54:06.191 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:54:06.191 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:54:06.192 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:54:06.192 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:54:06.231 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:54:06.232 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:06.233 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:54:06.233 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:06.234 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:54:06.234 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:06.234 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:54:06.235 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:06.235 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:54:06.311 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T09:54:06.311 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T09:54:06.314 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T09:54:06.314 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T09:54:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.345+0000 7ff188e07640 1 Processor -- start 2024-04-25T09:54:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.345+0000 7ff188e07640 1 -- start start 2024-04-25T09:54:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.346+0000 7ff188e07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 0x7ff18406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.346+0000 7ff188e07640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff1840cbf00 0x7ff18406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.346+0000 7ff188e07640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1840cae60 con 0x7ff1840cc360 2024-04-25T09:54:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.346+0000 7ff188e07640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1840cb140 con 0x7ff1840cbf00 2024-04-25T09:54:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.346+0000 7ff188e07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840cb420 con 0x7ff18415dbb0 2024-04-25T09:54:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.346+0000 7ff18259c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 0x7ff18406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.346+0000 7ff17bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff1840cbf00 0x7ff18406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.346+0000 7ff18259c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 0x7ff18406fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:52434/0 (socket says 172.21.15.19:52434) 2024-04-25T09:54:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.346+0000 7ff182d9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff1840cc360 0x7ff18406db50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:48690/0 (socket says 172.21.15.19:48690) 2024-04-25T09:54:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.346+0000 7ff18259c640 1 -- 172.21.15.19:0/3571666730 learned_addr learned my addr 172.21.15.19:0/3571666730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.346+0000 7ff17bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff1840cbf00 0x7ff18406bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:53726/0 (socket says 172.21.15.19:53726) 2024-04-25T09:54:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.347+0000 7ff181d9b640 1 -- 172.21.15.19:0/3571666730 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685433024 0 0) 0x7ff1840cae60 con 0x7ff1840cc360 2024-04-25T09:54:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.347+0000 7ff181d9b640 1 -- 172.21.15.19:0/3571666730 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff170003520 con 0x7ff1840cc360 2024-04-25T09:54:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.347+0000 7ff17bfff640 1 -- 172.21.15.19:0/3571666730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 msgr2=0x7ff18406fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.347+0000 7ff17bfff640 1 --2- 172.21.15.19:0/3571666730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 0x7ff18406fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.347+0000 7ff17bfff640 1 -- 172.21.15.19:0/3571666730 >> v1:172.21.15.112:6789/0 conn(0x7ff1840cc360 legacy=0x7ff18406db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.347+0000 7ff17bfff640 1 -- 172.21.15.19:0/3571666730 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840cb700 con 0x7ff1840cbf00 2024-04-25T09:54:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.347+0000 7ff18259c640 1 --2- 172.21.15.19:0/3571666730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 0x7ff18406fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff17bfff640 1 --2- 172.21.15.19:0/3571666730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1840cbf00 0x7ff18406bbf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0097a0 tx=0x7ff16c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6967f16b28973e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff181d9b640 1 -- 172.21.15.19:0/3571666730 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c037070 con 0x7ff1840cbf00 2024-04-25T09:54:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff181d9b640 1 -- 172.21.15.19:0/3571666730 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c032440 con 0x7ff1840cbf00 2024-04-25T09:54:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff181d9b640 1 -- 172.21.15.19:0/3571666730 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c032810 con 0x7ff1840cbf00 2024-04-25T09:54:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff188e07640 1 -- 172.21.15.19:0/3571666730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1840cbf00 msgr2=0x7ff18406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff188e07640 1 --2- 172.21.15.19:0/3571666730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1840cbf00 0x7ff18406bbf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0097a0 tx=0x7ff16c02a180 comp rx=0 tx=0).stop 2024-04-25T09:54:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff188e07640 1 -- 172.21.15.19:0/3571666730 shutdown_connections 2024-04-25T09:54:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff188e07640 1 --2- 172.21.15.19:0/3571666730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1840cbf00 0x7ff18406bbf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff188e07640 1 --2- 172.21.15.19:0/3571666730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 0x7ff18406fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff188e07640 1 -- 172.21.15.19:0/3571666730 >> 172.21.15.19:0/3571666730 conn(0x7ff184071af0 msgr2=0x7ff184071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff188e07640 1 -- 172.21.15.19:0/3571666730 shutdown_connections 2024-04-25T09:54:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.348+0000 7ff188e07640 1 -- 172.21.15.19:0/3571666730 wait complete. 2024-04-25T09:54:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.350+0000 7ff188e07640 1 Processor -- start 2024-04-25T09:54:06.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.350+0000 7ff188e07640 1 -- start start 2024-04-25T09:54:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.350+0000 7ff188e07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1840cbf00 0x7ff1840b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.350+0000 7ff188e07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1840cc360 0x7ff1840b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.350+0000 7ff188e07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 0x7ff184158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.350+0000 7ff188e07640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840cab10 con 0x7ff1840cc360 2024-04-25T09:54:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.350+0000 7ff188e07640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18415df90 con 0x7ff1840cbf00 2024-04-25T09:54:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.350+0000 7ff188e07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840703e0 con 0x7ff18415dbb0 2024-04-25T09:54:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1840cc360 0x7ff1840b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1840cc360 0x7ff1840b51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44340/0 (socket says 172.21.15.19:44340) 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17bfff640 1 -- 172.21.15.19:0/537060273 learned_addr learned my addr 172.21.15.19:0/537060273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff182d9d640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 0x7ff184158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff18259c640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1840cbf00 0x7ff1840b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17bfff640 1 -- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 msgr2=0x7ff184158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17bfff640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 0x7ff184158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17bfff640 1 -- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1840cbf00 msgr2=0x7ff1840b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17bfff640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1840cbf00 0x7ff1840b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17bfff640 1 -- 172.21.15.19:0/537060273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840b31f0 con 0x7ff1840cc360 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17bfff640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1840cc360 0x7ff1840b51f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0098c0 tx=0x7ff16c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17affd640 1 -- 172.21.15.19:0/537060273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c041030 con 0x7ff1840cc360 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17affd640 1 -- 172.21.15.19:0/537060273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c03d4e0 con 0x7ff1840cc360 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff188e07640 1 -- 172.21.15.19:0/537060273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1840b3490 con 0x7ff1840cc360 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff17affd640 1 -- 172.21.15.19:0/537060273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c037040 con 0x7ff1840cc360 2024-04-25T09:54:06.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.351+0000 7ff188e07640 1 -- 172.21.15.19:0/537060273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1840b3a50 con 0x7ff1840cc360 2024-04-25T09:54:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.352+0000 7ff188e07640 1 -- 172.21.15.19:0/537060273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18415d410 con 0x7ff1840cc360 2024-04-25T09:54:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.353+0000 7ff17affd640 1 -- 172.21.15.19:0/537060273 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff16c044070 con 0x7ff1840cc360 2024-04-25T09:54:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.353+0000 7ff17affd640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff16406b4d0 0x7ff16406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.353+0000 7ff17affd640 1 -- 172.21.15.19:0/537060273 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff16c0aa650 con 0x7ff1840cc360 2024-04-25T09:54:06.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.354+0000 7ff18259c640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff16406b4d0 0x7ff16406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.354+0000 7ff18259c640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff16406b4d0 0x7ff16406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff174009630 tx=0x7ff174009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.358+0000 7ff17affd640 1 -- 172.21.15.19:0/537060273 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff18415d410 con 0x7ff1840cc360 2024-04-25T09:54:06.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:06.549+0000 7ff188e07640 1 -- 172.21.15.19:0/537060273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff184073a00 con 0x7ff1840cc360 2024-04-25T09:54:06.840 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:06.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:07.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.171+0000 7ff17affd640 1 -- 172.21.15.19:0/537060273 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff16c079080 con 0x7ff1840cc360 2024-04-25T09:54:07.173 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T09:54:07.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 -- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff16406b4d0 msgr2=0x7ff16406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff16406b4d0 0x7ff16406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff174009630 tx=0x7ff174009240 comp rx=0 tx=0).stop 2024-04-25T09:54:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 -- 172.21.15.19:0/537060273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1840cc360 msgr2=0x7ff1840b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1840cc360 0x7ff1840b51f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0098c0 tx=0x7ff16c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 -- 172.21.15.19:0/537060273 shutdown_connections 2024-04-25T09:54:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff16406b4d0 0x7ff16406d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff18415dbb0 0x7ff184158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1840cc360 0x7ff1840b51f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 --2- 172.21.15.19:0/537060273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1840cbf00 0x7ff1840b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 -- 172.21.15.19:0/537060273 >> 172.21.15.19:0/537060273 conn(0x7ff184071af0 msgr2=0x7ff18406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 -- 172.21.15.19:0/537060273 shutdown_connections 2024-04-25T09:54:07.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.173+0000 7ff188e07640 1 -- 172.21.15.19:0/537060273 wait complete. 2024-04-25T09:54:07.193 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:54:07.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.293+0000 7fd52dbe7640 1 Processor -- start 2024-04-25T09:54:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.293+0000 7fd52dbe7640 1 -- start start 2024-04-25T09:54:07.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.294+0000 7fd52dbe7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd52815dd30 0x7fd5280b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.294+0000 7fd52dbe7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5280d1df0 con 0x7fd5280b9770 2024-04-25T09:54:07.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.294+0000 7fd52dbe7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5280d20d0 con 0x7fd5280d0ad0 2024-04-25T09:54:07.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.294+0000 7fd52dbe7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5280d23b0 con 0x7fd52815dd30 2024-04-25T09:54:07.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.294+0000 7fd5277fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd52815dd30 0x7fd5280b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.294+0000 7fd5277fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd52815dd30 0x7fd5280b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44344/0 (socket says 172.21.15.19:44344) 2024-04-25T09:54:07.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.294+0000 7fd5277fe640 1 -- 172.21.15.19:0/2619291621 learned_addr learned my addr 172.21.15.19:0/2619291621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:07.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.294+0000 7fd5277fe640 1 -- 172.21.15.19:0/2619291621 >> v1:172.21.15.112:6789/0 conn(0x7fd5280d0ad0 legacy=0x7fd5280b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.294+0000 7fd5277fe640 1 -- 172.21.15.19:0/2619291621 >> v1:172.21.15.19:6789/0 conn(0x7fd5280b9770 legacy=0x7fd5280d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.294+0000 7fd5277fe640 1 -- 172.21.15.19:0/2619291621 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5280d2690 con 0x7fd52815dd30 2024-04-25T09:54:07.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.295+0000 7fd5277fe640 1 --2- 172.21.15.19:0/2619291621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd52815dd30 0x7fd5280b9230 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd518009870 tx=0x7fd51802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78fab12f414e58f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.295+0000 7fd5267fc640 1 -- 172.21.15.19:0/2619291621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518037070 con 0x7fd52815dd30 2024-04-25T09:54:07.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.295+0000 7fd5267fc640 1 -- 172.21.15.19:0/2619291621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd518032440 con 0x7fd52815dd30 2024-04-25T09:54:07.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.295+0000 7fd5267fc640 1 -- 172.21.15.19:0/2619291621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518032810 con 0x7fd52815dd30 2024-04-25T09:54:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.296+0000 7fd52dbe7640 1 -- 172.21.15.19:0/2619291621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd52815dd30 msgr2=0x7fd5280b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.296+0000 7fd52dbe7640 1 --2- 172.21.15.19:0/2619291621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd52815dd30 0x7fd5280b9230 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd518009870 tx=0x7fd51802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.296+0000 7fd52dbe7640 1 -- 172.21.15.19:0/2619291621 shutdown_connections 2024-04-25T09:54:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.296+0000 7fd52dbe7640 1 --2- 172.21.15.19:0/2619291621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd52815dd30 0x7fd5280b9230 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.296+0000 7fd52dbe7640 1 -- 172.21.15.19:0/2619291621 >> 172.21.15.19:0/2619291621 conn(0x7fd528061c30 msgr2=0x7fd52806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.296+0000 7fd52dbe7640 1 -- 172.21.15.19:0/2619291621 shutdown_connections 2024-04-25T09:54:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.296+0000 7fd52dbe7640 1 -- 172.21.15.19:0/2619291621 wait complete. 2024-04-25T09:54:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.297+0000 7fd52dbe7640 1 Processor -- start 2024-04-25T09:54:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.297+0000 7fd52dbe7640 1 -- start start 2024-04-25T09:54:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd52dbe7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5280b9770 0x7fd528160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd52dbe7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5280d0ad0 0x7fd528159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd52dbe7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd52815dd30 0x7fd5280cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd52dbe7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280d1b90 con 0x7fd5280b9770 2024-04-25T09:54:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd52dbe7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280d1a10 con 0x7fd5280d0ad0 2024-04-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd52dbe7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280d2e90 con 0x7fd52815dd30 2024-04-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd5277fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5280b9770 0x7fd528160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd5277fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5280b9770 0x7fd528160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44350/0 (socket says 172.21.15.19:44350) 2024-04-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd5277fe640 1 -- 172.21.15.19:0/3548507658 learned_addr learned my addr 172.21.15.19:0/3548507658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd526ffd640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5280d0ad0 0x7fd528159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd527fff640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd52815dd30 0x7fd5280cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd5277fe640 1 -- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd52815dd30 msgr2=0x7fd5280cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd5277fe640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd52815dd30 0x7fd5280cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd5277fe640 1 -- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5280d0ad0 msgr2=0x7fd528159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd5277fe640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5280d0ad0 0x7fd528159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd5277fe640 1 -- 172.21.15.19:0/3548507658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52815a340 con 0x7fd5280b9770 2024-04-25T09:54:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd527fff640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd52815dd30 0x7fd5280cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.298+0000 7fd5277fe640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5280b9770 0x7fd528160bf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd518000c00 tx=0x7fd5180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.299+0000 7fd50ffff640 1 -- 172.21.15.19:0/3548507658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518037040 con 0x7fd5280b9770 2024-04-25T09:54:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.299+0000 7fd50ffff640 1 -- 172.21.15.19:0/3548507658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd518042070 con 0x7fd5280b9770 2024-04-25T09:54:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.299+0000 7fd52dbe7640 1 -- 172.21.15.19:0/3548507658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd528160150 con 0x7fd5280b9770 2024-04-25T09:54:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.299+0000 7fd50ffff640 1 -- 172.21.15.19:0/3548507658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518030bd0 con 0x7fd5280b9770 2024-04-25T09:54:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.299+0000 7fd52dbe7640 1 -- 172.21.15.19:0/3548507658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5280cb870 con 0x7fd5280b9770 2024-04-25T09:54:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.300+0000 7fd52dbe7640 1 -- 172.21.15.19:0/3548507658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd528073990 con 0x7fd5280b9770 2024-04-25T09:54:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.300+0000 7fd50ffff640 1 -- 172.21.15.19:0/3548507658 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd518047460 con 0x7fd5280b9770 2024-04-25T09:54:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.301+0000 7fd50ffff640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 0x7fd51406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.301+0000 7fd50ffff640 1 -- 172.21.15.19:0/3548507658 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd5180aab00 con 0x7fd5280b9770 2024-04-25T09:54:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.301+0000 7fd526ffd640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 0x7fd51406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.301+0000 7fd526ffd640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 0x7fd51406d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd528156f20 tx=0x7fd51c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.305+0000 7fd50ffff640 1 -- 172.21.15.19:0/3548507658 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd518079b40 con 0x7fd5280b9770 2024-04-25T09:54:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.561+0000 7fd52dbe7640 1 -- 172.21.15.19:0/3548507658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd52815c5d0 con 0x7fd5280b9770 2024-04-25T09:54:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.561+0000 7fd50ffff640 1 -- 172.21.15.19:0/3548507658 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fd518079440 con 0x7fd5280b9770 2024-04-25T09:54:07.563 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.564 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:54:07.566 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T09:54:07.567 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T09:54:07.569 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.570 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.571 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.571 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T09:54:07.571 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T09:54:07.571 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:07.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.572 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T09:54:07.573 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:54:07.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.575 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-mds", 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDqJypmEeoyLBAAXrLlhUB4LAIsDT4ksjZ0Jw==", 2024-04-25T09:54:07.576 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDqJypmUPIyLBAAb9UA4pcNUBLSUUiPkw2NSg==", 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-osd", 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDqJypmKfoyLBAAR9VeBU+SdF6ptfiThTBwOw==", 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T09:54:07.577 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDqJypmPAEzLBAAfK0dPM74+sbo0vEM/I+GwQ==", 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDqJypmoggzLBAAEF3upJAJK8aZSvcDXQWQyQ==", 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDqJypmORAzLBAABbyqvqH0WNYSGLUVtRws+w==", 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:54:07.579 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T09:54:07.580 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:54:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.563+0000 7fd52dbe7640 1 -- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 msgr2=0x7fd51406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.563+0000 7fd52dbe7640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 0x7fd51406d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd528156f20 tx=0x7fd51c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.563+0000 7fd52dbe7640 1 -- 172.21.15.19:0/3548507658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5280b9770 msgr2=0x7fd528160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.563+0000 7fd52dbe7640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5280b9770 0x7fd528160bf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd518000c00 tx=0x7fd5180046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.563+0000 7fd52dbe7640 1 -- 172.21.15.19:0/3548507658 shutdown_connections 2024-04-25T09:54:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.563+0000 7fd52dbe7640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd51406b4d0 0x7fd51406d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.563+0000 7fd52dbe7640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd52815dd30 0x7fd5280cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.563+0000 7fd52dbe7640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5280d0ad0 0x7fd528159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.563+0000 7fd52dbe7640 1 --2- 172.21.15.19:0/3548507658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5280b9770 0x7fd528160bf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.563+0000 7fd52dbe7640 1 -- 172.21.15.19:0/3548507658 >> 172.21.15.19:0/3548507658 conn(0x7fd528061c30 msgr2=0x7fd5280c7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.564+0000 7fd52dbe7640 1 -- 172.21.15.19:0/3548507658 shutdown_connections 2024-04-25T09:54:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.564+0000 7fd52dbe7640 1 -- 172.21.15.19:0/3548507658 wait complete. 2024-04-25T09:54:07.583 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T09:54:07.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.683+0000 7f94e44fc640 1 Processor -- start 2024-04-25T09:54:07.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.683+0000 7f94e44fc640 1 -- start start 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.684+0000 7f94e44fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f94dc15dac0 0x7f94dc0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.684+0000 7f94e44fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f94dc0d0ab0 0x7f94dc0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.684+0000 7f94e44fc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94dc0d2420 con 0x7f94dc0d1f30 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.684+0000 7f94e44fc640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94dc0d3030 con 0x7f94dc0d0ab0 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.684+0000 7f94e44fc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94dc0d2700 con 0x7f94dc15dac0 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.684+0000 7f94e2a99640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f94dc0d0ab0 0x7f94dc0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.684+0000 7f94e2a99640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f94dc0d0ab0 0x7f94dc0b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53164/0 (socket says 172.21.15.19:53164) 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.684+0000 7f94e2a99640 1 -- 172.21.15.19:0/2914266995 learned_addr learned my addr 172.21.15.19:0/2914266995 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.685+0000 7f94e2298640 1 --2- 172.21.15.19:0/2914266995 >> v2:172.21.15.112:3300/0 conn(0x7f94dc15dac0 0x7f94dc0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.685+0000 7f94e2a99640 1 -- 172.21.15.19:0/2914266995 >> v2:172.21.15.112:3300/0 conn(0x7f94dc15dac0 msgr2=0x7f94dc0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.685+0000 7f94e2a99640 1 --2- 172.21.15.19:0/2914266995 >> v2:172.21.15.112:3300/0 conn(0x7f94dc15dac0 0x7f94dc0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.685+0000 7f94e2a99640 1 -- 172.21.15.19:0/2914266995 >> v1:172.21.15.19:6789/0 conn(0x7f94dc0d1f30 legacy=0x7f94dc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.685+0000 7f94e2a99640 1 -- 172.21.15.19:0/2914266995 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0d29e0 con 0x7f94dc0d0ab0 2024-04-25T09:54:07.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.685+0000 7f94e2a99640 1 --2- 172.21.15.19:0/2914266995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc0d0ab0 0x7f94dc0b6990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0b6520 tx=0x7f94cc02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494ddf4609c4ae06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.686+0000 7f94e1296640 1 -- 172.21.15.19:0/2914266995 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc036070 con 0x7f94dc0d0ab0 2024-04-25T09:54:07.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.686+0000 7f94e1296640 1 -- 172.21.15.19:0/2914266995 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc031600 con 0x7f94dc0d0ab0 2024-04-25T09:54:07.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.686+0000 7f94e1296640 1 -- 172.21.15.19:0/2914266995 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0319f0 con 0x7f94dc0d0ab0 2024-04-25T09:54:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.686+0000 7f94e44fc640 1 -- 172.21.15.19:0/2914266995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc0d0ab0 msgr2=0x7f94dc0b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.686+0000 7f94e44fc640 1 --2- 172.21.15.19:0/2914266995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc0d0ab0 0x7f94dc0b6990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0b6520 tx=0x7f94cc02a7c0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.686+0000 7f94e44fc640 1 -- 172.21.15.19:0/2914266995 shutdown_connections 2024-04-25T09:54:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.686+0000 7f94e44fc640 1 --2- 172.21.15.19:0/2914266995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc0d0ab0 0x7f94dc0b6990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.686+0000 7f94e44fc640 1 --2- 172.21.15.19:0/2914266995 >> v2:172.21.15.112:3300/0 conn(0x7f94dc15dac0 0x7f94dc0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.686+0000 7f94e44fc640 1 -- 172.21.15.19:0/2914266995 >> 172.21.15.19:0/2914266995 conn(0x7f94dc061c30 msgr2=0x7f94dc06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.686+0000 7f94e44fc640 1 -- 172.21.15.19:0/2914266995 shutdown_connections 2024-04-25T09:54:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.686+0000 7f94e44fc640 1 -- 172.21.15.19:0/2914266995 wait complete. 2024-04-25T09:54:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.687+0000 7f94e44fc640 1 Processor -- start 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.687+0000 7f94e44fc640 1 -- start start 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.688+0000 7f94e44fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94dc0d0ab0 0x7f94dc0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.688+0000 7f94e44fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc0d1f30 0x7f94dc149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.688+0000 7f94e44fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94dc15dac0 0x7f94dc1605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.688+0000 7f94e44fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc0b97d0 con 0x7f94dc0d1f30 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.688+0000 7f94e44fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc15dea0 con 0x7f94dc15dac0 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.688+0000 7f94e44fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc0b9950 con 0x7f94dc0d0ab0 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.688+0000 7f94e1a97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc0d1f30 0x7f94dc149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.688+0000 7f94e1a97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc0d1f30 0x7f94dc149970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53172/0 (socket says 172.21.15.19:53172) 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.688+0000 7f94e1a97640 1 -- 172.21.15.19:0/1496592094 learned_addr learned my addr 172.21.15.19:0/1496592094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.688+0000 7f94e2298640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94dc0d0ab0 0x7f94dc0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.688+0000 7f94e2a99640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94dc15dac0 0x7f94dc1605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.689+0000 7f94e1a97640 1 -- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94dc0d0ab0 msgr2=0x7f94dc0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.689+0000 7f94e1a97640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94dc0d0ab0 0x7f94dc0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.689+0000 7f94e1a97640 1 -- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94dc15dac0 msgr2=0x7f94dc1605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.689+0000 7f94e1a97640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94dc15dac0 0x7f94dc1605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.689+0000 7f94e1a97640 1 -- 172.21.15.19:0/1496592094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0cabd0 con 0x7f94dc0d1f30 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.689+0000 7f94e1a97640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc0d1f30 0x7f94dc149970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f94d000c430 tx=0x7f94d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.689+0000 7f94daffd640 1 -- 172.21.15.19:0/1496592094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d0017070 con 0x7f94dc0d1f30 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.689+0000 7f94daffd640 1 -- 172.21.15.19:0/1496592094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d00035f0 con 0x7f94dc0d1f30 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.689+0000 7f94e44fc640 1 -- 172.21.15.19:0/1496592094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc0caeb0 con 0x7f94dc0d1f30 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.689+0000 7f94daffd640 1 -- 172.21.15.19:0/1496592094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d0003a20 con 0x7f94dc0d1f30 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.689+0000 7f94e44fc640 1 -- 172.21.15.19:0/1496592094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc0cd550 con 0x7f94dc0d1f30 2024-04-25T09:54:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.690+0000 7f94e44fc640 1 -- 172.21.15.19:0/1496592094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc0cd080 con 0x7f94dc0d1f30 2024-04-25T09:54:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.691+0000 7f94daffd640 1 -- 172.21.15.19:0/1496592094 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94d001a070 con 0x7f94dc0d1f30 2024-04-25T09:54:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.691+0000 7f94daffd640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f94c806b4d0 0x7f94c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.691+0000 7f94daffd640 1 -- 172.21.15.19:0/1496592094 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f94d00892a0 con 0x7f94dc0d1f30 2024-04-25T09:54:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.691+0000 7f94e2298640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f94c806b4d0 0x7f94c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.692+0000 7f94e2298640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f94c806b4d0 0x7f94c806d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009aa0 tx=0x7f94d4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.696+0000 7f94daffd640 1 -- 172.21.15.19:0/1496592094 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94d0057d20 con 0x7f94dc0d1f30 2024-04-25T09:54:07.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.951+0000 7f94e44fc640 1 -- 172.21.15.19:0/1496592094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f94dc073a00 con 0x7f94dc0d1f30 2024-04-25T09:54:07.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.953+0000 7f94daffd640 1 -- 172.21.15.19:0/1496592094 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f94d005bdc0 con 0x7f94dc0d1f30 2024-04-25T09:54:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 -- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f94c806b4d0 msgr2=0x7f94c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f94c806b4d0 0x7f94c806d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009aa0 tx=0x7f94d4009190 comp rx=0 tx=0).stop 2024-04-25T09:54:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 -- 172.21.15.19:0/1496592094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc0d1f30 msgr2=0x7f94dc149970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc0d1f30 0x7f94dc149970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f94d000c430 tx=0x7f94d000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 -- 172.21.15.19:0/1496592094 shutdown_connections 2024-04-25T09:54:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f94c806b4d0 0x7f94c806d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94dc15dac0 0x7f94dc1605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc0d1f30 0x7f94dc149970 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 --2- 172.21.15.19:0/1496592094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94dc0d0ab0 0x7f94dc0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 -- 172.21.15.19:0/1496592094 >> 172.21.15.19:0/1496592094 conn(0x7f94dc061c30 msgr2=0x7f94dc0b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 -- 172.21.15.19:0/1496592094 shutdown_connections 2024-04-25T09:54:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:07.956+0000 7f94e44fc640 1 -- 172.21.15.19:0/1496592094 wait complete. 2024-04-25T09:54:07.969 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T09:54:08.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.068+0000 7fcdcc0e1640 1 Processor -- start 2024-04-25T09:54:08.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.069+0000 7fcdcc0e1640 1 -- start start 2024-04-25T09:54:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.069+0000 7fcdcc0e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdc415bb50 0x7fcdc40c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.069+0000 7fcdcc0e1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdc40cfd10 0x7fcdc40cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.069+0000 7fcdcc0e1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc40c75e0 con 0x7fcdc40c70d0 2024-04-25T09:54:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.069+0000 7fcdcc0e1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc40c8680 con 0x7fcdc40cfd10 2024-04-25T09:54:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.069+0000 7fcdcc0e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40c78c0 con 0x7fcdc415bb50 2024-04-25T09:54:08.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.070+0000 7fcdca67e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdc40cfd10 0x7fcdc40cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.070+0000 7fcdca67e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdc40cfd10 0x7fcdc40cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53174/0 (socket says 172.21.15.19:53174) 2024-04-25T09:54:08.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.070+0000 7fcdca67e640 1 -- 172.21.15.19:0/3787532198 learned_addr learned my addr 172.21.15.19:0/3787532198 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:08.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.070+0000 7fcdc9e7d640 1 --2- 172.21.15.19:0/3787532198 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdc415bb50 0x7fcdc40c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.070+0000 7fcdca67e640 1 -- 172.21.15.19:0/3787532198 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdc415bb50 msgr2=0x7fcdc40c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.070+0000 7fcdca67e640 1 --2- 172.21.15.19:0/3787532198 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdc415bb50 0x7fcdc40c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.070+0000 7fcdca67e640 1 -- 172.21.15.19:0/3787532198 >> v1:172.21.15.112:6789/0 conn(0x7fcdc40c70d0 legacy=0x7fcdc40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.070+0000 7fcdca67e640 1 -- 172.21.15.19:0/3787532198 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc40c7ba0 con 0x7fcdc40cfd10 2024-04-25T09:54:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdca67e640 1 --2- 172.21.15.19:0/3787532198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc40cfd10 0x7fcdc40cbf20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009e50 tx=0x7fcdbc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991f93feebdead52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdc8e7b640 1 -- 172.21.15.19:0/3787532198 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc036070 con 0x7fcdc40cfd10 2024-04-25T09:54:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdc8e7b640 1 -- 172.21.15.19:0/3787532198 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc031600 con 0x7fcdc40cfd10 2024-04-25T09:54:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdc8e7b640 1 -- 172.21.15.19:0/3787532198 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0319d0 con 0x7fcdc40cfd10 2024-04-25T09:54:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/3787532198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc40cfd10 msgr2=0x7fcdc40cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdcc0e1640 1 --2- 172.21.15.19:0/3787532198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc40cfd10 0x7fcdc40cbf20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009e50 tx=0x7fcdbc02a7a0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/3787532198 shutdown_connections 2024-04-25T09:54:08.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdcc0e1640 1 --2- 172.21.15.19:0/3787532198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc40cfd10 0x7fcdc40cbf20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdcc0e1640 1 --2- 172.21.15.19:0/3787532198 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdc415bb50 0x7fcdc40c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/3787532198 >> 172.21.15.19:0/3787532198 conn(0x7fcdc406f500 msgr2=0x7fcdc406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/3787532198 shutdown_connections 2024-04-25T09:54:08.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.071+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/3787532198 wait complete. 2024-04-25T09:54:08.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.073+0000 7fcdcc0e1640 1 Processor -- start 2024-04-25T09:54:08.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.073+0000 7fcdcc0e1640 1 -- start start 2024-04-25T09:54:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.073+0000 7fcdcc0e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc40c70d0 0x7fcdc415cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.073+0000 7fcdcc0e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc40cfd10 0x7fcdc414ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.073+0000 7fcdcc0e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdc415bb50 0x7fcdc415e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.073+0000 7fcdcc0e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc415f6d0 con 0x7fcdc40c70d0 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.073+0000 7fcdcc0e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40c8160 con 0x7fcdc40cfd10 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.073+0000 7fcdcc0e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc415f850 con 0x7fcdc415bb50 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.073+0000 7fcdc9e7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc40c70d0 0x7fcdc415cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.073+0000 7fcdc9e7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc40c70d0 0x7fcdc415cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53182/0 (socket says 172.21.15.19:53182) 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.073+0000 7fcdc9e7d640 1 -- 172.21.15.19:0/2706302816 learned_addr learned my addr 172.21.15.19:0/2706302816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdc967c640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc40cfd10 0x7fcdc414ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdca67e640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdc415bb50 0x7fcdc415e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdc9e7d640 1 -- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdc415bb50 msgr2=0x7fcdc415e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdc9e7d640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdc415bb50 0x7fcdc415e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdc9e7d640 1 -- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc40cfd10 msgr2=0x7fcdc414ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdc9e7d640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc40cfd10 0x7fcdc414ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdc9e7d640 1 -- 172.21.15.19:0/2706302816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc40ca580 con 0x7fcdc40c70d0 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdc9e7d640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc40c70d0 0x7fcdc415cb70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000d5f0 tx=0x7fcdc000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdba7fc640 1 -- 172.21.15.19:0/2706302816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc00125f0 con 0x7fcdc40c70d0 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdba7fc640 1 -- 172.21.15.19:0/2706302816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc0019070 con 0x7fcdc40c70d0 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/2706302816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc40ca860 con 0x7fcdc40c70d0 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdba7fc640 1 -- 172.21.15.19:0/2706302816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc00129d0 con 0x7fcdc40c70d0 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.074+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/2706302816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc40cbe40 con 0x7fcdc40c70d0 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.075+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/2706302816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc4073a00 con 0x7fcdc40c70d0 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.076+0000 7fcdba7fc640 1 -- 172.21.15.19:0/2706302816 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdc0012b70 con 0x7fcdc40c70d0 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.076+0000 7fcdba7fc640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcda806b4d0 0x7fcda806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.076+0000 7fcdba7fc640 1 -- 172.21.15.19:0/2706302816 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcdc008a300 con 0x7fcdc40c70d0 2024-04-25T09:54:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.076+0000 7fcdc967c640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcda806b4d0 0x7fcda806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.077+0000 7fcdc967c640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcda806b4d0 0x7fcda806d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4153000 tx=0x7fcdb4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.080+0000 7fcdba7fc640 1 -- 172.21.15.19:0/2706302816 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdc0058cf0 con 0x7fcdc40c70d0 2024-04-25T09:54:08.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.334+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/2706302816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fcdc40cf150 con 0x7fcdc40c70d0 2024-04-25T09:54:08.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.336+0000 7fcdba7fc640 1 -- 172.21.15.19:0/2706302816 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcdc005cd90 con 0x7fcdc40c70d0 2024-04-25T09:54:08.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcda806b4d0 msgr2=0x7fcda806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcda806b4d0 0x7fcda806d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4153000 tx=0x7fcdb4009040 comp rx=0 tx=0).stop 2024-04-25T09:54:08.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/2706302816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc40c70d0 msgr2=0x7fcdc415cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc40c70d0 0x7fcdc415cb70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000d5f0 tx=0x7fcdc000dab0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/2706302816 shutdown_connections 2024-04-25T09:54:08.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcda806b4d0 0x7fcda806d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdc415bb50 0x7fcdc415e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc40cfd10 0x7fcdc414ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 --2- 172.21.15.19:0/2706302816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc40c70d0 0x7fcdc415cb70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/2706302816 >> 172.21.15.19:0/2706302816 conn(0x7fcdc406f500 msgr2=0x7fcdc40cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/2706302816 shutdown_connections 2024-04-25T09:54:08.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.339+0000 7fcdcc0e1640 1 -- 172.21.15.19:0/2706302816 wait complete. 2024-04-25T09:54:08.352 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T09:54:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.451+0000 7fe904589640 1 Processor -- start 2024-04-25T09:54:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.451+0000 7fe904589640 1 -- start start 2024-04-25T09:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe904589640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0b5620 0x7fe8fc0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe904589640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8fc06cf50 con 0x7fe8fc06c2d0 2024-04-25T09:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe904589640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8fc0ce530 con 0x7fe8fc15b7d0 2024-04-25T09:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe904589640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc0b5fc0 con 0x7fe8fc0b5620 2024-04-25T09:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe902b26640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe8fc06c2d0 0x7fe8fc06c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38742/0 (socket says 172.21.15.19:38742) 2024-04-25T09:54:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe902b26640 1 -- 172.21.15.19:0/413964755 learned_addr learned my addr 172.21.15.19:0/413964755 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901b24640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0b5620 0x7fe8fc0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926406386 0 0) 0x7fe8fc06cf50 con 0x7fe8fc06c2d0 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e8003520 con 0x7fe8fc06c2d0 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3501606517 0 0) 0x7fe8e8003520 con 0x7fe8fc06c2d0 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0b5620 msgr2=0x7fe8fc0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0b5620 0x7fe8fc0b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 >> v1:172.21.15.112:6789/0 conn(0x7fe8fc15b7d0 legacy=0x7fe8fc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc0ce810 con 0x7fe8fc06c2d0 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fe8f40019f0 con 0x7fe8fc06c2d0 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 >> v1:172.21.15.19:6789/0 conn(0x7fe8fc06c2d0 legacy=0x7fe8fc06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e80039e0 0x7fe8e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 0x7fe8e8004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0b5620 0x7fe8e80050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e800b9f0 con 0x7fe8e8004390 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e800bba0 con 0x7fe8e80039e0 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.452+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e800bda0 con 0x7fe8fc0b5620 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.453+0000 7fe901b24640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 0x7fe8e8004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.453+0000 7fe902325640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e80039e0 0x7fe8e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.453+0000 7fe902b26640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0b5620 0x7fe8e80050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.453+0000 7fe901b24640 1 -- 172.21.15.19:0/413964755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0b5620 msgr2=0x7fe8e80050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.453+0000 7fe901b24640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0b5620 0x7fe8e80050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.453+0000 7fe901b24640 1 -- 172.21.15.19:0/413964755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e80039e0 msgr2=0x7fe8e8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.453+0000 7fe901b24640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8e80039e0 0x7fe8e8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.453+0000 7fe901b24640 1 -- 172.21.15.19:0/413964755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e8003520 con 0x7fe8e8004390 2024-04-25T09:54:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.454+0000 7fe901b24640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 0x7fe8e8004860 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009a90 tx=0x7fe8e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82de9c093e340762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.454+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec036070 con 0x7fe8e8004390 2024-04-25T09:54:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.454+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec031400 con 0x7fe8e8004390 2024-04-25T09:54:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.454+0000 7fe901323640 1 -- 172.21.15.19:0/413964755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec009690 con 0x7fe8e8004390 2024-04-25T09:54:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.454+0000 7fe904589640 1 -- 172.21.15.19:0/413964755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 msgr2=0x7fe8e8004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.454+0000 7fe904589640 1 --2- 172.21.15.19:0/413964755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 0x7fe8e8004860 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009a90 tx=0x7fe8e8005830 comp rx=0 tx=0).stop 2024-04-25T09:54:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.454+0000 7fe902b26640 1 -- 172.21.15.19:0/413964755 reap_dead start 2024-04-25T09:54:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.454+0000 7fe904589640 1 -- 172.21.15.19:0/413964755 shutdown_connections 2024-04-25T09:54:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.454+0000 7fe904589640 1 -- 172.21.15.19:0/413964755 >> 172.21.15.19:0/413964755 conn(0x7fe8fc0d0800 msgr2=0x7fe8fc0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.454+0000 7fe904589640 1 -- 172.21.15.19:0/413964755 shutdown_connections 2024-04-25T09:54:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.454+0000 7fe904589640 1 -- 172.21.15.19:0/413964755 wait complete. 2024-04-25T09:54:08.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.456+0000 7fe904589640 1 Processor -- start 2024-04-25T09:54:08.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.456+0000 7fe904589640 1 -- start start 2024-04-25T09:54:08.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.456+0000 7fe904589640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 0x7fe8fc0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.456+0000 7fe904589640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0cc900 0x7fe8fc158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.456+0000 7fe904589640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8fc1f3710 0x7fe8fc0aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.456+0000 7fe904589640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc15bbb0 con 0x7fe8e8004390 2024-04-25T09:54:08.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.456+0000 7fe904589640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc06d230 con 0x7fe8fc1f3710 2024-04-25T09:54:08.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.456+0000 7fe904589640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc15bd30 con 0x7fe8fc0cc900 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.456+0000 7fe902325640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 0x7fe8fc0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.456+0000 7fe902325640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 0x7fe8fc0cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53186/0 (socket says 172.21.15.19:53186) 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.456+0000 7fe902325640 1 -- 172.21.15.19:0/550386757 learned_addr learned my addr 172.21.15.19:0/550386757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe902b26640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8fc1f3710 0x7fe8fc0aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe901b24640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0cc900 0x7fe8fc158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe902325640 1 -- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0cc900 msgr2=0x7fe8fc158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe902325640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0cc900 0x7fe8fc158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe902325640 1 -- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8fc1f3710 msgr2=0x7fe8fc0aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe902325640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8fc1f3710 0x7fe8fc0aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe902325640 1 -- 172.21.15.19:0/550386757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc0b3730 con 0x7fe8e8004390 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe902325640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 0x7fe8fc0cb3f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800f7c0 tx=0x7fe8f800fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe8f2ffd640 1 -- 172.21.15.19:0/550386757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f80052e0 con 0x7fe8e8004390 2024-04-25T09:54:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe8f2ffd640 1 -- 172.21.15.19:0/550386757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f8018070 con 0x7fe8e8004390 2024-04-25T09:54:08.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe8f2ffd640 1 -- 172.21.15.19:0/550386757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800d970 con 0x7fe8e8004390 2024-04-25T09:54:08.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe904589640 1 -- 172.21.15.19:0/550386757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8fc0b3a10 con 0x7fe8e8004390 2024-04-25T09:54:08.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.457+0000 7fe904589640 1 -- 172.21.15.19:0/550386757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8fc0b3fd0 con 0x7fe8e8004390 2024-04-25T09:54:08.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.458+0000 7fe904589640 1 -- 172.21.15.19:0/550386757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8fc0cfb00 con 0x7fe8e8004390 2024-04-25T09:54:08.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.458+0000 7fe8f2ffd640 1 -- 172.21.15.19:0/550386757 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8f800db10 con 0x7fe8e8004390 2024-04-25T09:54:08.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.459+0000 7fe8f2ffd640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8e006b500 0x7fe8e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.459+0000 7fe8f2ffd640 1 -- 172.21.15.19:0/550386757 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe8f8089d90 con 0x7fe8e8004390 2024-04-25T09:54:08.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.459+0000 7fe901b24640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8e006b500 0x7fe8e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.459+0000 7fe901b24640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8e006b500 0x7fe8e006d9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec033000 tx=0x7fe8ec008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.463+0000 7fe8f2ffd640 1 -- 172.21.15.19:0/550386757 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8fc0cfb00 con 0x7fe8e8004390 2024-04-25T09:54:08.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.711+0000 7fe904589640 1 -- 172.21.15.19:0/550386757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fe8fc073a00 con 0x7fe8e8004390 2024-04-25T09:54:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.714+0000 7fe8f2ffd640 1 -- 172.21.15.19:0/550386757 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe8f8058810 con 0x7fe8e8004390 2024-04-25T09:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.716+0000 7fe904589640 1 -- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8e006b500 msgr2=0x7fe8e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.716+0000 7fe904589640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8e006b500 0x7fe8e006d9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec033000 tx=0x7fe8ec008030 comp rx=0 tx=0).stop 2024-04-25T09:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.716+0000 7fe904589640 1 -- 172.21.15.19:0/550386757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 msgr2=0x7fe8fc0cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.716+0000 7fe904589640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 0x7fe8fc0cb3f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800f7c0 tx=0x7fe8f800fc80 comp rx=0 tx=0).stop 2024-04-25T09:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.716+0000 7fe904589640 1 -- 172.21.15.19:0/550386757 shutdown_connections 2024-04-25T09:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.716+0000 7fe904589640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8e006b500 0x7fe8e006d9c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.716+0000 7fe904589640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8fc1f3710 0x7fe8fc0aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.716+0000 7fe904589640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8fc0cc900 0x7fe8fc158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.716+0000 7fe904589640 1 --2- 172.21.15.19:0/550386757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8e8004390 0x7fe8fc0cb3f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.716+0000 7fe904589640 1 -- 172.21.15.19:0/550386757 >> 172.21.15.19:0/550386757 conn(0x7fe8fc0d0800 msgr2=0x7fe8fc15afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.717+0000 7fe904589640 1 -- 172.21.15.19:0/550386757 shutdown_connections 2024-04-25T09:54:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.717+0000 7fe904589640 1 -- 172.21.15.19:0/550386757 wait complete. 2024-04-25T09:54:08.730 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T09:54:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.830+0000 7fd729f9e640 1 Processor -- start 2024-04-25T09:54:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.830+0000 7fd729f9e640 1 -- start start 2024-04-25T09:54:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.831+0000 7fd729f9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd7240b5620 0x7fd7240b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.831+0000 7fd729f9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72406cf50 con 0x7fd72406c2d0 2024-04-25T09:54:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.831+0000 7fd729f9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7240ce530 con 0x7fd72415b7d0 2024-04-25T09:54:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.831+0000 7fd729f9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7240ce810 con 0x7fd7240b5620 2024-04-25T09:54:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.831+0000 7fd722ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd7240b5620 0x7fd7240b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.831+0000 7fd722ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd7240b5620 0x7fd7240b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53190/0 (socket says 172.21.15.19:53190) 2024-04-25T09:54:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.831+0000 7fd722ffd640 1 -- 172.21.15.19:0/3542296354 learned_addr learned my addr 172.21.15.19:0/3542296354 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.831+0000 7fd722ffd640 1 -- 172.21.15.19:0/3542296354 >> v1:172.21.15.112:6789/0 conn(0x7fd72415b7d0 legacy=0x7fd7240cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.831+0000 7fd722ffd640 1 -- 172.21.15.19:0/3542296354 >> v1:172.21.15.19:6789/0 conn(0x7fd72406c2d0 legacy=0x7fd72406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.831+0000 7fd722ffd640 1 -- 172.21.15.19:0/3542296354 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7240ceaf0 con 0x7fd7240b5620 2024-04-25T09:54:08.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.832+0000 7fd722ffd640 1 --2- 172.21.15.19:0/3542296354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7240b5620 0x7fd7240b5ab0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd718009870 tx=0x7fd71802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a13b1d81db38e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.832+0000 7fd7227fc640 1 -- 172.21.15.19:0/3542296354 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718037070 con 0x7fd7240b5620 2024-04-25T09:54:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.832+0000 7fd7227fc640 1 -- 172.21.15.19:0/3542296354 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71802ae50 con 0x7fd7240b5620 2024-04-25T09:54:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.832+0000 7fd7227fc640 1 -- 172.21.15.19:0/3542296354 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718032860 con 0x7fd7240b5620 2024-04-25T09:54:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.833+0000 7fd729f9e640 1 -- 172.21.15.19:0/3542296354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7240b5620 msgr2=0x7fd7240b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.833+0000 7fd729f9e640 1 --2- 172.21.15.19:0/3542296354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7240b5620 0x7fd7240b5ab0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd718009870 tx=0x7fd71802a300 comp rx=0 tx=0).stop 2024-04-25T09:54:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.833+0000 7fd729f9e640 1 -- 172.21.15.19:0/3542296354 shutdown_connections 2024-04-25T09:54:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.833+0000 7fd729f9e640 1 --2- 172.21.15.19:0/3542296354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7240b5620 0x7fd7240b5ab0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.833+0000 7fd729f9e640 1 -- 172.21.15.19:0/3542296354 >> 172.21.15.19:0/3542296354 conn(0x7fd7240d0800 msgr2=0x7fd7240d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.833+0000 7fd729f9e640 1 -- 172.21.15.19:0/3542296354 shutdown_connections 2024-04-25T09:54:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.833+0000 7fd729f9e640 1 -- 172.21.15.19:0/3542296354 wait complete. 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.834+0000 7fd729f9e640 1 Processor -- start 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.834+0000 7fd729f9e640 1 -- start start 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd729f9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72406c2d0 0x7fd7240c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd729f9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7240b5620 0x7fd7240c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd729f9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72415b7d0 0x7fd7241485a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd729f9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72415bd30 con 0x7fd72406c2d0 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd729f9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72415bbb0 con 0x7fd72415b7d0 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd729f9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240d1f40 con 0x7fd7240b5620 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd7237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72406c2d0 0x7fd7240c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd7237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72406c2d0 0x7fd7240c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53196/0 (socket says 172.21.15.19:53196) 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd7237fe640 1 -- 172.21.15.19:0/3880927329 learned_addr learned my addr 172.21.15.19:0/3880927329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd722ffd640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7240b5620 0x7fd7240c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd723fff640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72415b7d0 0x7fd7241485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd7237fe640 1 -- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7240b5620 msgr2=0x7fd7240c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd7237fe640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7240b5620 0x7fd7240c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd7237fe640 1 -- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72415b7d0 msgr2=0x7fd7241485a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd7237fe640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72415b7d0 0x7fd7241485a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd7237fe640 1 -- 172.21.15.19:0/3880927329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd724148ae0 con 0x7fd72406c2d0 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.835+0000 7fd723fff640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72415b7d0 0x7fd7241485a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.836+0000 7fd7237fe640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72406c2d0 0x7fd7240c76d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c3c0 tx=0x7fd71400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.836+0000 7fd70bfff640 1 -- 172.21.15.19:0/3880927329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7140052d0 con 0x7fd72406c2d0 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.836+0000 7fd70bfff640 1 -- 172.21.15.19:0/3880927329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd714005490 con 0x7fd72406c2d0 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.836+0000 7fd729f9e640 1 -- 172.21.15.19:0/3880927329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7240c7cb0 con 0x7fd72406c2d0 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.836+0000 7fd70bfff640 1 -- 172.21.15.19:0/3880927329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714007260 con 0x7fd72406c2d0 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.836+0000 7fd729f9e640 1 -- 172.21.15.19:0/3880927329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7241f72b0 con 0x7fd72406c2d0 2024-04-25T09:54:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.837+0000 7fd729f9e640 1 -- 172.21.15.19:0/3880927329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd724073a00 con 0x7fd72406c2d0 2024-04-25T09:54:08.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.837+0000 7fd70bfff640 1 -- 172.21.15.19:0/3880927329 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd714007400 con 0x7fd72406c2d0 2024-04-25T09:54:08.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.837+0000 7fd70bfff640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd71006b4d0 0x7fd71006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.838+0000 7fd70bfff640 1 -- 172.21.15.19:0/3880927329 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd714089470 con 0x7fd72406c2d0 2024-04-25T09:54:08.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.838+0000 7fd722ffd640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd71006b4d0 0x7fd71006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.838+0000 7fd722ffd640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd71006b4d0 0x7fd71006d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd718034000 tx=0x7fd71802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:08.842+0000 7fd70bfff640 1 -- 172.21.15.19:0/3880927329 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd714057e60 con 0x7fd72406c2d0 2024-04-25T09:54:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.094+0000 7fd729f9e640 1 -- 172.21.15.19:0/3880927329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fd72415b420 con 0x7fd72406c2d0 2024-04-25T09:54:09.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.097+0000 7fd70bfff640 1 -- 172.21.15.19:0/3880927329 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd71405ce20 con 0x7fd72406c2d0 2024-04-25T09:54:09.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.099+0000 7fd729f9e640 1 -- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd71006b4d0 msgr2=0x7fd71006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.099+0000 7fd729f9e640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd71006b4d0 0x7fd71006d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd718034000 tx=0x7fd71802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:09.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.099+0000 7fd729f9e640 1 -- 172.21.15.19:0/3880927329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72406c2d0 msgr2=0x7fd7240c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.099+0000 7fd729f9e640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72406c2d0 0x7fd7240c76d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c3c0 tx=0x7fd71400c880 comp rx=0 tx=0).stop 2024-04-25T09:54:09.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.099+0000 7fd729f9e640 1 -- 172.21.15.19:0/3880927329 shutdown_connections 2024-04-25T09:54:09.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.099+0000 7fd729f9e640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd71006b4d0 0x7fd71006d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.099+0000 7fd729f9e640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd72415b7d0 0x7fd7241485a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.099+0000 7fd729f9e640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7240b5620 0x7fd7240c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.099+0000 7fd729f9e640 1 --2- 172.21.15.19:0/3880927329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72406c2d0 0x7fd7240c76d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.100+0000 7fd729f9e640 1 -- 172.21.15.19:0/3880927329 >> 172.21.15.19:0/3880927329 conn(0x7fd7240d0800 msgr2=0x7fd72415fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.100+0000 7fd729f9e640 1 -- 172.21.15.19:0/3880927329 shutdown_connections 2024-04-25T09:54:09.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.100+0000 7fd729f9e640 1 -- 172.21.15.19:0/3880927329 wait complete. 2024-04-25T09:54:09.113 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T09:54:09.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.212+0000 7f81a8b28640 1 Processor -- start 2024-04-25T09:54:09.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.212+0000 7f81a8b28640 1 -- start start 2024-04-25T09:54:09.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.213+0000 7f81a8b28640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f81a415dc20 0x7f81a40d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.213+0000 7f81a8b28640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f81a40d1ce0 0x7f81a40b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.213+0000 7f81a8b28640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a40ce670 0x7f81a40ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.213+0000 7f81a8b28640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a40cf020 con 0x7f81a40d1ce0 2024-04-25T09:54:09.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.213+0000 7f81a8b28640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a40d2120 con 0x7f81a415dc20 2024-04-25T09:54:09.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.213+0000 7f81a8b28640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a40d23a0 con 0x7f81a40ce670 2024-04-25T09:54:09.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.213+0000 7f81a1d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f81a40d1ce0 0x7f81a40b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.213+0000 7f81a1d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f81a40d1ce0 0x7f81a40b9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53210/0 (socket says 172.21.15.19:53210) 2024-04-25T09:54:09.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.213+0000 7f81a1d9b640 1 -- 172.21.15.19:0/2467679156 learned_addr learned my addr 172.21.15.19:0/2467679156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:09.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.214+0000 7f81a259c640 1 --2- 172.21.15.19:0/2467679156 >> v2:172.21.15.112:3300/0 conn(0x7f81a415dc20 0x7f81a40d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.214+0000 7f81a2d9d640 1 --2- 172.21.15.19:0/2467679156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a40ce670 0x7f81a40ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.214+0000 7f81a1d9b640 1 -- 172.21.15.19:0/2467679156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a40ce670 msgr2=0x7f81a40ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.214+0000 7f81a1d9b640 1 --2- 172.21.15.19:0/2467679156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a40ce670 0x7f81a40ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.214+0000 7f81a1d9b640 1 -- 172.21.15.19:0/2467679156 >> v2:172.21.15.112:3300/0 conn(0x7f81a415dc20 msgr2=0x7f81a40d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.214+0000 7f81a1d9b640 1 --2- 172.21.15.19:0/2467679156 >> v2:172.21.15.112:3300/0 conn(0x7f81a415dc20 0x7f81a40d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.214+0000 7f81a1d9b640 1 -- 172.21.15.19:0/2467679156 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a40d2680 con 0x7f81a40d1ce0 2024-04-25T09:54:09.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.214+0000 7f81a1d9b640 1 --2- 172.21.15.19:0/2467679156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a40d1ce0 0x7f81a40b9490 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f81a40d1500 tx=0x7f819802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407b83c65823b1f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a159a640 1 -- 172.21.15.19:0/2467679156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198037070 con 0x7f81a40d1ce0 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a159a640 1 -- 172.21.15.19:0/2467679156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8198032440 con 0x7f81a40d1ce0 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a159a640 1 -- 172.21.15.19:0/2467679156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198032810 con 0x7f81a40d1ce0 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a8b28640 1 -- 172.21.15.19:0/2467679156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a40d1ce0 msgr2=0x7f81a40b9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a8b28640 1 --2- 172.21.15.19:0/2467679156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a40d1ce0 0x7f81a40b9490 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f81a40d1500 tx=0x7f819802a140 comp rx=0 tx=0).stop 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a8b28640 1 -- 172.21.15.19:0/2467679156 shutdown_connections 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a8b28640 1 --2- 172.21.15.19:0/2467679156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a40ce670 0x7f81a40ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a8b28640 1 --2- 172.21.15.19:0/2467679156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a40d1ce0 0x7f81a40b9490 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a8b28640 1 --2- 172.21.15.19:0/2467679156 >> v2:172.21.15.112:3300/0 conn(0x7f81a415dc20 0x7f81a40d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a8b28640 1 -- 172.21.15.19:0/2467679156 >> 172.21.15.19:0/2467679156 conn(0x7f81a4061a20 msgr2=0x7f81a4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a8b28640 1 -- 172.21.15.19:0/2467679156 shutdown_connections 2024-04-25T09:54:09.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.215+0000 7f81a8b28640 1 -- 172.21.15.19:0/2467679156 wait complete. 2024-04-25T09:54:09.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.217+0000 7f81a8b28640 1 Processor -- start 2024-04-25T09:54:09.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.217+0000 7f81a8b28640 1 -- start start 2024-04-25T09:54:09.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.217+0000 7f81a8b28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a40ce670 0x7f81a415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.217+0000 7f81a8b28640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a40d1ce0 0x7f81a40b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.217+0000 7f81a8b28640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a415dc20 0x7f81a40b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.217+0000 7f81a8b28640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a40d2f20 con 0x7f81a415dc20 2024-04-25T09:54:09.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.217+0000 7f81a8b28640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a40d2c40 con 0x7f81a40ce670 2024-04-25T09:54:09.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.217+0000 7f81a8b28640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a406c120 con 0x7f81a40d1ce0 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a415dc20 0x7f81a40b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a415dc20 0x7f81a40b6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53214/0 (socket says 172.21.15.19:53214) 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a2d9d640 1 -- 172.21.15.19:0/2645994101 learned_addr learned my addr 172.21.15.19:0/2645994101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a1d9b640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a40d1ce0 0x7f81a40b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a259c640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a40ce670 0x7f81a415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a2d9d640 1 -- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a40d1ce0 msgr2=0x7f81a40b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a2d9d640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a40d1ce0 0x7f81a40b7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a2d9d640 1 -- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a40ce670 msgr2=0x7f81a415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a2d9d640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a40ce670 0x7f81a415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a2d9d640 1 -- 172.21.15.19:0/2645994101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a41568f0 con 0x7f81a415dc20 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a1d9b640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a40d1ce0 0x7f81a40b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a2d9d640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a415dc20 0x7f81a40b6c60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8194007e40 tx=0x7f819400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f8192ffd640 1 -- 172.21.15.19:0/2645994101 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194014620 con 0x7f81a415dc20 2024-04-25T09:54:09.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f8192ffd640 1 -- 172.21.15.19:0/2645994101 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8194019070 con 0x7f81a415dc20 2024-04-25T09:54:09.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a8b28640 1 -- 172.21.15.19:0/2645994101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a4156bb0 con 0x7f81a415dc20 2024-04-25T09:54:09.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f8192ffd640 1 -- 172.21.15.19:0/2645994101 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194014a20 con 0x7f81a415dc20 2024-04-25T09:54:09.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.218+0000 7f81a8b28640 1 -- 172.21.15.19:0/2645994101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a415e0a0 con 0x7f81a415dc20 2024-04-25T09:54:09.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.219+0000 7f81a8b28640 1 -- 172.21.15.19:0/2645994101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a4073a00 con 0x7f81a415dc20 2024-04-25T09:54:09.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.220+0000 7f8192ffd640 1 -- 172.21.15.19:0/2645994101 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8194014bc0 con 0x7f81a415dc20 2024-04-25T09:54:09.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.220+0000 7f8192ffd640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 0x7f818406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.220+0000 7f8192ffd640 1 -- 172.21.15.19:0/2645994101 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f819408a9b0 con 0x7f81a415dc20 2024-04-25T09:54:09.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.220+0000 7f81a259c640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 0x7f818406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.221+0000 7f81a259c640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 0x7f818406d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f819c006fd0 tx=0x7f819c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.224+0000 7f8192ffd640 1 -- 172.21.15.19:0/2645994101 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8194059970 con 0x7f81a415dc20 2024-04-25T09:54:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.479+0000 7f81a8b28640 1 -- 172.21.15.19:0/2645994101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f81a40cdd60 con 0x7f81a415dc20 2024-04-25T09:54:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.481+0000 7f8192ffd640 1 -- 172.21.15.19:0/2645994101 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8194059270 con 0x7f81a415dc20 2024-04-25T09:54:09.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 -- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 msgr2=0x7f818406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 0x7f818406d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f819c006fd0 tx=0x7f819c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 -- 172.21.15.19:0/2645994101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a415dc20 msgr2=0x7f81a40b6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a415dc20 0x7f81a40b6c60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8194007e40 tx=0x7f819400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 -- 172.21.15.19:0/2645994101 shutdown_connections 2024-04-25T09:54:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f818406b4d0 0x7f818406d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a415dc20 0x7f81a40b6c60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81a40d1ce0 0x7f81a40b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 --2- 172.21.15.19:0/2645994101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a40ce670 0x7f81a415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 -- 172.21.15.19:0/2645994101 >> 172.21.15.19:0/2645994101 conn(0x7f81a4061a20 msgr2=0x7f81a40b8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 -- 172.21.15.19:0/2645994101 shutdown_connections 2024-04-25T09:54:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.483+0000 7f81a8b28640 1 -- 172.21.15.19:0/2645994101 wait complete. 2024-04-25T09:54:09.496 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T09:54:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.595+0000 7f3d2c31d640 1 Processor -- start 2024-04-25T09:54:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.595+0000 7f3d2c31d640 1 -- start start 2024-04-25T09:54:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.596+0000 7f3d2c31d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3d2415de80 0x7f3d240cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.596+0000 7f3d2c31d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3d240cefb0 0x7f3d240cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.596+0000 7f3d2c31d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d240afc80 con 0x7f3d240cf980 2024-04-25T09:54:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.596+0000 7f3d2c31d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d240ce4d0 con 0x7f3d240cefb0 2024-04-25T09:54:09.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.596+0000 7f3d2c31d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d240ce7b0 con 0x7f3d2415de80 2024-04-25T09:54:09.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.597+0000 7f3d298b8640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3d240cefb0 0x7f3d240cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.597+0000 7f3d298b8640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3d240cefb0 0x7f3d240cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53218/0 (socket says 172.21.15.19:53218) 2024-04-25T09:54:09.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.597+0000 7f3d298b8640 1 -- 172.21.15.19:0/773825819 learned_addr learned my addr 172.21.15.19:0/773825819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:09.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.597+0000 7f3d2a0b9640 1 --2- 172.21.15.19:0/773825819 >> v2:172.21.15.112:3300/0 conn(0x7f3d2415de80 0x7f3d240cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.597+0000 7f3d298b8640 1 -- 172.21.15.19:0/773825819 >> v2:172.21.15.112:3300/0 conn(0x7f3d2415de80 msgr2=0x7f3d240cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.597+0000 7f3d298b8640 1 --2- 172.21.15.19:0/773825819 >> v2:172.21.15.112:3300/0 conn(0x7f3d2415de80 0x7f3d240cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.597+0000 7f3d298b8640 1 -- 172.21.15.19:0/773825819 >> v1:172.21.15.19:6789/0 conn(0x7f3d240cf980 legacy=0x7f3d2406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.597+0000 7f3d298b8640 1 -- 172.21.15.19:0/773825819 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d240cea90 con 0x7f3d240cefb0 2024-04-25T09:54:09.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d298b8640 1 --2- 172.21.15.19:0/773825819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d240cefb0 0x7f3d240cf440 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3d240d1800 tx=0x7f3d1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb7d745fe50e4ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d290b7640 1 -- 172.21.15.19:0/773825819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14037070 con 0x7f3d240cefb0 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d290b7640 1 -- 172.21.15.19:0/773825819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d14032440 con 0x7f3d240cefb0 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d290b7640 1 -- 172.21.15.19:0/773825819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14032810 con 0x7f3d240cefb0 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d2c31d640 1 -- 172.21.15.19:0/773825819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d240cefb0 msgr2=0x7f3d240cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d2c31d640 1 --2- 172.21.15.19:0/773825819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d240cefb0 0x7f3d240cf440 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3d240d1800 tx=0x7f3d1402a270 comp rx=0 tx=0).stop 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d2c31d640 1 -- 172.21.15.19:0/773825819 shutdown_connections 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d2c31d640 1 --2- 172.21.15.19:0/773825819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d240cefb0 0x7f3d240cf440 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d2c31d640 1 --2- 172.21.15.19:0/773825819 >> v2:172.21.15.112:3300/0 conn(0x7f3d2415de80 0x7f3d240cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d2c31d640 1 -- 172.21.15.19:0/773825819 >> 172.21.15.19:0/773825819 conn(0x7f3d240b2390 msgr2=0x7f3d240b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d2c31d640 1 -- 172.21.15.19:0/773825819 shutdown_connections 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.598+0000 7f3d2c31d640 1 -- 172.21.15.19:0/773825819 wait complete. 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.599+0000 7f3d2c31d640 1 Processor -- start 2024-04-25T09:54:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.600+0000 7f3d2c31d640 1 -- start start 2024-04-25T09:54:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.600+0000 7f3d2c31d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d240cefb0 0x7f3d24158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.600+0000 7f3d2c31d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d240cf980 0x7f3d240c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.600+0000 7f3d2c31d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2415de80 0x7f3d24156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.600+0000 7f3d2c31d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d240b3030 con 0x7f3d240cf980 2024-04-25T09:54:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.600+0000 7f3d2c31d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d240b2eb0 con 0x7f3d240cefb0 2024-04-25T09:54:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.600+0000 7f3d2c31d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d240c5600 con 0x7f3d2415de80 2024-04-25T09:54:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.600+0000 7f3d298b8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d240cf980 0x7f3d240c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.600+0000 7f3d298b8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d240cf980 0x7f3d240c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53230/0 (socket says 172.21.15.19:53230) 2024-04-25T09:54:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.600+0000 7f3d298b8640 1 -- 172.21.15.19:0/2778998572 learned_addr learned my addr 172.21.15.19:0/2778998572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d2a0b9640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d240cefb0 0x7f3d24158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d2a8ba640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2415de80 0x7f3d24156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d298b8640 1 -- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2415de80 msgr2=0x7f3d24156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d298b8640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2415de80 0x7f3d24156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d298b8640 1 -- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d240cefb0 msgr2=0x7f3d24158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d298b8640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d240cefb0 0x7f3d24158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d298b8640 1 -- 172.21.15.19:0/2778998572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d241f71f0 con 0x7f3d240cf980 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d2a8ba640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2415de80 0x7f3d24156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d2a0b9640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d240cefb0 0x7f3d24158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d298b8640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d240cf980 0x7f3d240c8fc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d1402a2b0 tx=0x7f3d140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d1affd640 1 -- 172.21.15.19:0/2778998572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14041020 con 0x7f3d240cf980 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d1affd640 1 -- 172.21.15.19:0/2778998572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d140306c0 con 0x7f3d240cf980 2024-04-25T09:54:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d1affd640 1 -- 172.21.15.19:0/2778998572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14037040 con 0x7f3d240cf980 2024-04-25T09:54:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d2c31d640 1 -- 172.21.15.19:0/2778998572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d241f74d0 con 0x7f3d240cf980 2024-04-25T09:54:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.601+0000 7f3d2c31d640 1 -- 172.21.15.19:0/2778998572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d241f7a90 con 0x7f3d240cf980 2024-04-25T09:54:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.602+0000 7f3d2c31d640 1 -- 172.21.15.19:0/2778998572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d24073a00 con 0x7f3d240cf980 2024-04-25T09:54:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.602+0000 7f3d1affd640 1 -- 172.21.15.19:0/2778998572 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d14044020 con 0x7f3d240cf980 2024-04-25T09:54:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.603+0000 7f3d1affd640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d0806b4d0 0x7f3d0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.603+0000 7f3d1affd640 1 -- 172.21.15.19:0/2778998572 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3d140aa910 con 0x7f3d240cf980 2024-04-25T09:54:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.603+0000 7f3d2a0b9640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d0806b4d0 0x7f3d0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.604+0000 7f3d2a0b9640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d0806b4d0 0x7f3d0806d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3d20006fd0 tx=0x7f3d20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.607+0000 7f3d1affd640 1 -- 172.21.15.19:0/2778998572 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d1404b030 con 0x7f3d240cf980 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.856+0000 7f3d2c31d640 1 -- 172.21.15.19:0/2778998572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f3d240ca640 con 0x7f3d240cf980 2024-04-25T09:54:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.858+0000 7f3d1affd640 1 -- 172.21.15.19:0/2778998572 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3d140798b0 con 0x7f3d240cf980 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 -- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d0806b4d0 msgr2=0x7f3d0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d0806b4d0 0x7f3d0806d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3d20006fd0 tx=0x7f3d20008040 comp rx=0 tx=0).stop 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 -- 172.21.15.19:0/2778998572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d240cf980 msgr2=0x7f3d240c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d240cf980 0x7f3d240c8fc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d1402a2b0 tx=0x7f3d140046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 -- 172.21.15.19:0/2778998572 shutdown_connections 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d0806b4d0 0x7f3d0806d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2415de80 0x7f3d24156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d240cf980 0x7f3d240c8fc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 --2- 172.21.15.19:0/2778998572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d240cefb0 0x7f3d24158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 -- 172.21.15.19:0/2778998572 >> 172.21.15.19:0/2778998572 conn(0x7f3d240b2390 msgr2=0x7f3d240d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 -- 172.21.15.19:0/2778998572 shutdown_connections 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.860+0000 7f3d2c31d640 1 -- 172.21.15.19:0/2778998572 wait complete. 2024-04-25T09:54:09.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:09.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.972+0000 7f3bfbdd5640 1 Processor -- start 2024-04-25T09:54:09.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.972+0000 7f3bfbdd5640 1 -- start start 2024-04-25T09:54:09.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.973+0000 7f3bfbdd5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3bf40b6cb0 0x7f3bf40b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.973+0000 7f3bfbdd5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf40ceee0 con 0x7f3bf415db70 2024-04-25T09:54:09.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.973+0000 7f3bfbdd5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf40d1d10 con 0x7f3bf40b5a60 2024-04-25T09:54:09.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.973+0000 7f3bfbdd5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf40d1ff0 con 0x7f3bf40b6cb0 2024-04-25T09:54:09.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.973+0000 7f3bf9b71640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3bf415db70 0x7f3bf4162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38754/0 (socket says 172.21.15.19:38754) 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.973+0000 7f3bf9b71640 1 -- 172.21.15.19:0/1070217785 learned_addr learned my addr 172.21.15.19:0/1070217785 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.973+0000 7f3bf9370640 1 --2- 172.21.15.19:0/1070217785 >> v2:172.21.15.112:3300/0 conn(0x7f3bf40b6cb0 0x7f3bf40b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.973+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427961789 0 0) 0x7f3bf40ceee0 con 0x7f3bf415db70 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.973+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be0003520 con 0x7f3bf415db70 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2622016100 0 0) 0x7f3be0003520 con 0x7f3bf415db70 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 >> v2:172.21.15.112:3300/0 conn(0x7f3bf40b6cb0 msgr2=0x7f3bf40b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 --2- 172.21.15.19:0/1070217785 >> v2:172.21.15.112:3300/0 conn(0x7f3bf40b6cb0 0x7f3bf40b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 >> v1:172.21.15.112:6789/0 conn(0x7f3bf40b5a60 legacy=0x7f3bf40d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf40d22d0 con 0x7f3bf415db70 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f3bf0001670 con 0x7f3bf415db70 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 >> v1:172.21.15.19:6789/0 conn(0x7f3bf415db70 legacy=0x7f3bf4162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 --2- 172.21.15.19:0/1070217785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be00039b0 0x7f3be0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 --2- 172.21.15.19:0/1070217785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3be00043b0 0x7f3be0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 --2- 172.21.15.19:0/1070217785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 0x7f3be000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf40d1ff0 con 0x7f3be0005120 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be000c060 con 0x7f3be00039b0 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be000c260 con 0x7f3be00043b0 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bfa372640 1 --2- 172.21.15.19:0/1070217785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 0x7f3be000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf9b71640 1 --2- 172.21.15.19:0/1070217785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be00039b0 0x7f3be0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bf9370640 1 --2- 172.21.15.19:0/1070217785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3be00043b0 0x7f3be0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bfa372640 1 -- 172.21.15.19:0/1070217785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3be00043b0 msgr2=0x7f3be0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bfa372640 1 --2- 172.21.15.19:0/1070217785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3be00043b0 0x7f3be0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bfa372640 1 -- 172.21.15.19:0/1070217785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be00039b0 msgr2=0x7f3be0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bfa372640 1 --2- 172.21.15.19:0/1070217785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3be00039b0 0x7f3be0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.974+0000 7f3bfa372640 1 -- 172.21.15.19:0/1070217785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be0003520 con 0x7f3be0005120 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.975+0000 7f3bfa372640 1 -- 172.21.15.19:0/1070217785 reap_dead start 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.975+0000 7f3bfa372640 1 --2- 172.21.15.19:0/1070217785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 0x7f3be000b9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3bec001f60 tx=0x7f3be0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3d2292de5e9aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.975+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec001b30 con 0x7f3be0005120 2024-04-25T09:54:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.975+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec003b90 con 0x7f3be0005120 2024-04-25T09:54:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.975+0000 7f3bf8b6f640 1 -- 172.21.15.19:0/1070217785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec005110 con 0x7f3be0005120 2024-04-25T09:54:09.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.976+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/1070217785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 msgr2=0x7f3be000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.976+0000 7f3bfbdd5640 1 --2- 172.21.15.19:0/1070217785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 0x7f3be000b9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3bec001f60 tx=0x7f3be0005830 comp rx=0 tx=0).stop 2024-04-25T09:54:09.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.976+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/1070217785 shutdown_connections 2024-04-25T09:54:09.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.976+0000 7f3bfbdd5640 1 --2- 172.21.15.19:0/1070217785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 0x7f3be000b9c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.976+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/1070217785 >> 172.21.15.19:0/1070217785 conn(0x7f3bf40d0520 msgr2=0x7f3bf40d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.976+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/1070217785 shutdown_connections 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.976+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/1070217785 wait complete. 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.977+0000 7f3bfbdd5640 1 Processor -- start 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.977+0000 7f3bfbdd5640 1 -- start start 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bfbdd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 0x7f3bf4159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bfbdd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3bf40cd0e0 0x7f3bf40ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bfbdd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf40adcf0 0x7f3bf41f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bfbdd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf40cf1c0 con 0x7f3be0005120 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bfbdd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf40ce530 con 0x7f3bf40adcf0 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bfbdd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf40cf340 con 0x7f3bf40cd0e0 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bf9b71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 0x7f3bf4159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bf9b71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 0x7f3bf4159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53256/0 (socket says 172.21.15.19:53256) 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bf9b71640 1 -- 172.21.15.19:0/4122524291 learned_addr learned my addr 172.21.15.19:0/4122524291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bf9370640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3bf40cd0e0 0x7f3bf40ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bfa372640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf40adcf0 0x7f3bf41f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bf9b71640 1 -- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3bf40cd0e0 msgr2=0x7f3bf40ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bf9b71640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3bf40cd0e0 0x7f3bf40ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bf9b71640 1 -- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf40adcf0 msgr2=0x7f3bf41f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bf9b71640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf40adcf0 0x7f3bf41f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bf9b71640 1 -- 172.21.15.19:0/4122524291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf41f78a0 con 0x7f3be0005120 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bf9370640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3bf40cd0e0 0x7f3bf40ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bf9b71640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 0x7f3bf4159300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0007310 tx=0x7f3bf00077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bea7fc640 1 -- 172.21.15.19:0/4122524291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000f8e0 con 0x7f3be0005120 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bea7fc640 1 -- 172.21.15.19:0/4122524291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf0018070 con 0x7f3be0005120 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/4122524291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf40aa730 con 0x7f3be0005120 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bea7fc640 1 -- 172.21.15.19:0/4122524291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000fd00 con 0x7f3be0005120 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.978+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/4122524291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf40aacf0 con 0x7f3be0005120 2024-04-25T09:54:09.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.979+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/4122524291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf40ab410 con 0x7f3be0005120 2024-04-25T09:54:09.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.980+0000 7f3bea7fc640 1 -- 172.21.15.19:0/4122524291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bf001d460 con 0x7f3be0005120 2024-04-25T09:54:09.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.980+0000 7f3bea7fc640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3bd806b4d0 0x7f3bd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.981+0000 7f3bea7fc640 1 -- 172.21.15.19:0/4122524291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3bf008a010 con 0x7f3be0005120 2024-04-25T09:54:09.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.981+0000 7f3bf9370640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3bd806b4d0 0x7f3bd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.981+0000 7f3bf9370640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3bd806b4d0 0x7f3bd806d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bf415cf60 tx=0x7f3be4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:09.985+0000 7f3bea7fc640 1 -- 172.21.15.19:0/4122524291 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bf41f78a0 con 0x7f3be0005120 2024-04-25T09:54:10.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.167+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/4122524291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bf40d1820 con 0x7f3be0005120 2024-04-25T09:54:10.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.167+0000 7f3bea7fc640 1 -- 172.21.15.19:0/4122524291 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13152 (secure 0 0 0) 0x7f3bf0058a90 con 0x7f3be0005120 2024-04-25T09:54:10.170 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:10.170 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":34,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:54:07.180077+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3bd806b4d0 msgr2=0x7f3bd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3bd806b4d0 0x7f3bd806d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bf415cf60 tx=0x7f3be4008040 comp rx=0 tx=0).stop 2024-04-25T09:54:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/4122524291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 msgr2=0x7f3bf4159300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 0x7f3bf4159300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0007310 tx=0x7f3bf00077d0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/4122524291 shutdown_connections 2024-04-25T09:54:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3bd806b4d0 0x7f3bd806d990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf40adcf0 0x7f3bf41f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3bf40cd0e0 0x7f3bf40ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 --2- 172.21.15.19:0/4122524291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be0005120 0x7f3bf4159300 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/4122524291 >> 172.21.15.19:0/4122524291 conn(0x7f3bf40d0520 msgr2=0x7f3bf415bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/4122524291 shutdown_connections 2024-04-25T09:54:10.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.169+0000 7f3bfbdd5640 1 -- 172.21.15.19:0/4122524291 wait complete. 2024-04-25T09:54:10.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:54:10.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:10.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.287+0000 7f0a661c1640 1 Processor -- start 2024-04-25T09:54:10.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.287+0000 7f0a661c1640 1 -- start start 2024-04-25T09:54:10.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.287+0000 7f0a661c1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a601616f0 0x7f0a60161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.287+0000 7f0a661c1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6006c8d0 con 0x7f0a601610f0 2024-04-25T09:54:10.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.287+0000 7f0a661c1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a600c6960 con 0x7f0a600ce0a0 2024-04-25T09:54:10.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.287+0000 7f0a661c1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a601614d0 con 0x7f0a601616f0 2024-04-25T09:54:10.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.288+0000 7f0a5effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a601616f0 0x7f0a60161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.288+0000 7f0a5effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a601616f0 0x7f0a60161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53262/0 (socket says 172.21.15.19:53262) 2024-04-25T09:54:10.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.288+0000 7f0a5effd640 1 -- 172.21.15.19:0/3906423571 learned_addr learned my addr 172.21.15.19:0/3906423571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:10.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.288+0000 7f0a5effd640 1 -- 172.21.15.19:0/3906423571 >> v1:172.21.15.112:6789/0 conn(0x7f0a600ce0a0 legacy=0x7f0a600d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:10.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.288+0000 7f0a5effd640 1 -- 172.21.15.19:0/3906423571 >> v1:172.21.15.19:6789/0 conn(0x7f0a601610f0 legacy=0x7f0a6006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.288+0000 7f0a5effd640 1 -- 172.21.15.19:0/3906423571 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a60162260 con 0x7f0a601616f0 2024-04-25T09:54:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.289+0000 7f0a5effd640 1 --2- 172.21.15.19:0/3906423571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a601616f0 0x7f0a60161b50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009870 tx=0x7f0a5402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0a678563fd038b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.289+0000 7f0a5e7fc640 1 -- 172.21.15.19:0/3906423571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54037070 con 0x7f0a601616f0 2024-04-25T09:54:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.289+0000 7f0a5e7fc640 1 -- 172.21.15.19:0/3906423571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a54032440 con 0x7f0a601616f0 2024-04-25T09:54:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.289+0000 7f0a5e7fc640 1 -- 172.21.15.19:0/3906423571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54032810 con 0x7f0a601616f0 2024-04-25T09:54:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.289+0000 7f0a661c1640 1 -- 172.21.15.19:0/3906423571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a601616f0 msgr2=0x7f0a60161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.289+0000 7f0a661c1640 1 --2- 172.21.15.19:0/3906423571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a601616f0 0x7f0a60161b50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009870 tx=0x7f0a5402a2a0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.289+0000 7f0a661c1640 1 -- 172.21.15.19:0/3906423571 shutdown_connections 2024-04-25T09:54:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.289+0000 7f0a661c1640 1 --2- 172.21.15.19:0/3906423571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a601616f0 0x7f0a60161b50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.289+0000 7f0a661c1640 1 -- 172.21.15.19:0/3906423571 >> 172.21.15.19:0/3906423571 conn(0x7f0a6006f500 msgr2=0x7f0a6006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.289+0000 7f0a661c1640 1 -- 172.21.15.19:0/3906423571 shutdown_connections 2024-04-25T09:54:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.289+0000 7f0a661c1640 1 -- 172.21.15.19:0/3906423571 wait complete. 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.290+0000 7f0a661c1640 1 Processor -- start 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.291+0000 7f0a661c1640 1 -- start start 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.291+0000 7f0a661c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a600ce0a0 0x7f0a600b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.291+0000 7f0a661c1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a601610f0 0x7f0a60154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.291+0000 7f0a661c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a601616f0 0x7f0a601522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.291+0000 7f0a661c1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a600cfce0 con 0x7f0a601610f0 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.291+0000 7f0a661c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a600cfa00 con 0x7f0a601616f0 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.291+0000 7f0a661c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a600cf690 con 0x7f0a600ce0a0 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.291+0000 7f0a5effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a601610f0 0x7f0a60154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.291+0000 7f0a5effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a601610f0 0x7f0a60154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53274/0 (socket says 172.21.15.19:53274) 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.291+0000 7f0a5effd640 1 -- 172.21.15.19:0/4189492959 learned_addr learned my addr 172.21.15.19:0/4189492959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a5f7fe640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a600ce0a0 0x7f0a600b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a5ffff640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a601616f0 0x7f0a601522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a5effd640 1 -- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a600ce0a0 msgr2=0x7f0a600b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a5effd640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a600ce0a0 0x7f0a600b6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a5effd640 1 -- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a601616f0 msgr2=0x7f0a601522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a5effd640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a601616f0 0x7f0a601522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a5effd640 1 -- 172.21.15.19:0/4189492959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a600b5e90 con 0x7f0a601610f0 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a5f7fe640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a600ce0a0 0x7f0a600b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a5effd640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a601610f0 0x7f0a60154f60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009870 tx=0x7f0a540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a47fff640 1 -- 172.21.15.19:0/4189492959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54041020 con 0x7f0a601610f0 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a47fff640 1 -- 172.21.15.19:0/4189492959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a54030970 con 0x7f0a601610f0 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a47fff640 1 -- 172.21.15.19:0/4189492959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54037040 con 0x7f0a601610f0 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a661c1640 1 -- 172.21.15.19:0/4189492959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a600b6430 con 0x7f0a601610f0 2024-04-25T09:54:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.292+0000 7f0a661c1640 1 -- 172.21.15.19:0/4189492959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a600b69f0 con 0x7f0a601610f0 2024-04-25T09:54:10.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.293+0000 7f0a661c1640 1 -- 172.21.15.19:0/4189492959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a60073a00 con 0x7f0a601610f0 2024-04-25T09:54:10.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.294+0000 7f0a47fff640 1 -- 172.21.15.19:0/4189492959 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a54044020 con 0x7f0a601610f0 2024-04-25T09:54:10.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.294+0000 7f0a47fff640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.294+0000 7f0a47fff640 1 -- 172.21.15.19:0/4189492959 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0a540aa980 con 0x7f0a601610f0 2024-04-25T09:54:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.294+0000 7f0a5f7fe640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.295+0000 7f0a5f7fe640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a50001700 tx=0x7f0a50009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.298+0000 7f0a47fff640 1 -- 172.21.15.19:0/4189492959 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a5404b030 con 0x7f0a601610f0 2024-04-25T09:54:10.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.481+0000 7f0a661c1640 1 -- 172.21.15.19:0/4189492959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a600cf150 con 0x7f0a601610f0 2024-04-25T09:54:10.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.481+0000 7f0a47fff640 1 -- 172.21.15.19:0/4189492959 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a540799c0 con 0x7f0a601610f0 2024-04-25T09:54:10.483 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:54:10.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 -- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b4d0 msgr2=0x7f0a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a50001700 tx=0x7f0a50009040 comp rx=0 tx=0).stop 2024-04-25T09:54:10.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 -- 172.21.15.19:0/4189492959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a601610f0 msgr2=0x7f0a60154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a601610f0 0x7f0a60154f60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009870 tx=0x7f0a540046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 -- 172.21.15.19:0/4189492959 shutdown_connections 2024-04-25T09:54:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a601616f0 0x7f0a601522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a601610f0 0x7f0a60154f60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 --2- 172.21.15.19:0/4189492959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a600ce0a0 0x7f0a600b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 -- 172.21.15.19:0/4189492959 >> 172.21.15.19:0/4189492959 conn(0x7f0a6006f500 msgr2=0x7f0a600714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 -- 172.21.15.19:0/4189492959 shutdown_connections 2024-04-25T09:54:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.483+0000 7f0a661c1640 1 -- 172.21.15.19:0/4189492959 wait complete. 2024-04-25T09:54:10.496 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:54:10.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.594+0000 7f465a9cf640 1 Processor -- start 2024-04-25T09:54:10.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.594+0000 7f465a9cf640 1 -- start start 2024-04-25T09:54:10.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f465a9cf640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f465415db70 0x7f4654162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f465a9cf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46540b5c30 0x7f46540d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f465a9cf640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4654162690 con 0x7f46540b6cb0 2024-04-25T09:54:10.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f465a9cf640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46540ceee0 con 0x7f465415db70 2024-04-25T09:54:10.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f465a9cf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46540d1d10 con 0x7f46540b5c30 2024-04-25T09:54:10.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f46537fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f46540b6cb0 0x7f46540b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38770/0 (socket says 172.21.15.19:38770) 2024-04-25T09:54:10.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f46537fe640 1 -- 172.21.15.19:0/2468508801 learned_addr learned my addr 172.21.15.19:0/2468508801 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:10.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f4653fff640 1 --2- 172.21.15.19:0/2468508801 >> v2:172.21.15.112:3300/0 conn(0x7f465415db70 0x7f4654162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f4658f6c640 1 --2- 172.21.15.19:0/2468508801 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46540b5c30 0x7f46540d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f4652ffd640 1 -- 172.21.15.19:0/2468508801 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311179963 0 0) 0x7f4654162690 con 0x7f46540b6cb0 2024-04-25T09:54:10.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f4652ffd640 1 -- 172.21.15.19:0/2468508801 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4640003520 con 0x7f46540b6cb0 2024-04-25T09:54:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f4653fff640 1 -- 172.21.15.19:0/2468508801 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46540b5c30 msgr2=0x7f46540d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f4653fff640 1 --2- 172.21.15.19:0/2468508801 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46540b5c30 0x7f46540d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f4653fff640 1 -- 172.21.15.19:0/2468508801 >> v1:172.21.15.19:6789/0 conn(0x7f46540b6cb0 legacy=0x7f46540b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.595+0000 7f4653fff640 1 -- 172.21.15.19:0/2468508801 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46540d1ff0 con 0x7f465415db70 2024-04-25T09:54:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.596+0000 7f4653fff640 1 --2- 172.21.15.19:0/2468508801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f465415db70 0x7f4654162150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46440097a0 tx=0x7f464402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deebc9ba18663490 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.596+0000 7f4652ffd640 1 -- 172.21.15.19:0/2468508801 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644037070 con 0x7f465415db70 2024-04-25T09:54:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.597+0000 7f4652ffd640 1 -- 172.21.15.19:0/2468508801 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644032440 con 0x7f465415db70 2024-04-25T09:54:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.597+0000 7f4652ffd640 1 -- 172.21.15.19:0/2468508801 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46440327f0 con 0x7f465415db70 2024-04-25T09:54:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.597+0000 7f465a9cf640 1 -- 172.21.15.19:0/2468508801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f465415db70 msgr2=0x7f4654162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.597+0000 7f465a9cf640 1 --2- 172.21.15.19:0/2468508801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f465415db70 0x7f4654162150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46440097a0 tx=0x7f464402a180 comp rx=0 tx=0).stop 2024-04-25T09:54:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.597+0000 7f465a9cf640 1 -- 172.21.15.19:0/2468508801 shutdown_connections 2024-04-25T09:54:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.597+0000 7f465a9cf640 1 --2- 172.21.15.19:0/2468508801 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46540b5c30 0x7f46540d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.597+0000 7f465a9cf640 1 --2- 172.21.15.19:0/2468508801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f465415db70 0x7f4654162150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.597+0000 7f465a9cf640 1 -- 172.21.15.19:0/2468508801 >> 172.21.15.19:0/2468508801 conn(0x7f46540d0520 msgr2=0x7f46540d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.597+0000 7f465a9cf640 1 -- 172.21.15.19:0/2468508801 shutdown_connections 2024-04-25T09:54:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.597+0000 7f465a9cf640 1 -- 172.21.15.19:0/2468508801 wait complete. 2024-04-25T09:54:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.598+0000 7f465a9cf640 1 Processor -- start 2024-04-25T09:54:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.598+0000 7f465a9cf640 1 -- start start 2024-04-25T09:54:10.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.599+0000 7f465a9cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46540b5c30 0x7f46540cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.599+0000 7f465a9cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46540b6cb0 0x7f465415e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.599+0000 7f465a9cf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f465415db70 0x7f4654147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.599+0000 7f465a9cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46540ce530 con 0x7f46540b5c30 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.599+0000 7f465a9cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465415df50 con 0x7f46540b6cb0 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.599+0000 7f465a9cf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46540cf1c0 con 0x7f465415db70 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.599+0000 7f4653fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46540b5c30 0x7f46540cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.599+0000 7f4653fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46540b5c30 0x7f46540cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53282/0 (socket says 172.21.15.19:53282) 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.599+0000 7f4653fff640 1 -- 172.21.15.19:0/735310546 learned_addr learned my addr 172.21.15.19:0/735310546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.599+0000 7f4658f6c640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f465415db70 0x7f4654147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.599+0000 7f46537fe640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46540b6cb0 0x7f465415e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.600+0000 7f4653fff640 1 -- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f465415db70 msgr2=0x7f4654147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.600+0000 7f4653fff640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f465415db70 0x7f4654147bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.600+0000 7f4653fff640 1 -- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46540b6cb0 msgr2=0x7f465415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.600+0000 7f4653fff640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46540b6cb0 0x7f465415e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.600+0000 7f4653fff640 1 -- 172.21.15.19:0/735310546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46540cd7a0 con 0x7f46540b5c30 2024-04-25T09:54:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.600+0000 7f4653fff640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46540b5c30 0x7f46540cd260 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f464402a1c0 tx=0x7f46440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.600+0000 7f4650ff9640 1 -- 172.21.15.19:0/735310546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644037040 con 0x7f46540b5c30 2024-04-25T09:54:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.600+0000 7f4650ff9640 1 -- 172.21.15.19:0/735310546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644042070 con 0x7f46540b5c30 2024-04-25T09:54:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.600+0000 7f465a9cf640 1 -- 172.21.15.19:0/735310546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465415eb20 con 0x7f46540b5c30 2024-04-25T09:54:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.600+0000 7f4650ff9640 1 -- 172.21.15.19:0/735310546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464403b930 con 0x7f46540b5c30 2024-04-25T09:54:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.600+0000 7f465a9cf640 1 -- 172.21.15.19:0/735310546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465415f0e0 con 0x7f46540b5c30 2024-04-25T09:54:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.601+0000 7f465a9cf640 1 -- 172.21.15.19:0/735310546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4654073990 con 0x7f46540b5c30 2024-04-25T09:54:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.602+0000 7f4650ff9640 1 -- 172.21.15.19:0/735310546 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f464403bad0 con 0x7f46540b5c30 2024-04-25T09:54:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.602+0000 7f4650ff9640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f463406b4d0 0x7f463406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.602+0000 7f4650ff9640 1 -- 172.21.15.19:0/735310546 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f46440aa7e0 con 0x7f46540b5c30 2024-04-25T09:54:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.602+0000 7f46537fe640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f463406b4d0 0x7f463406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.603+0000 7f46537fe640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f463406b4d0 0x7f463406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f465414c240 tx=0x7f463c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.606+0000 7f4650ff9640 1 -- 172.21.15.19:0/735310546 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46440791d0 con 0x7f46540b5c30 2024-04-25T09:54:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.792+0000 7f465a9cf640 1 -- 172.21.15.19:0/735310546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f46540d1820 con 0x7f46540b5c30 2024-04-25T09:54:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.792+0000 7f4650ff9640 1 -- 172.21.15.19:0/735310546 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+65 (secure 0 0 0) 0x7f464407d270 con 0x7f46540b5c30 2024-04-25T09:54:10.795 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:10.795 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T09:54:10.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:54:10.795 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T09:54:10.795 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T09:54:10.795 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:54:10.795 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T09:54:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.794+0000 7f465a9cf640 1 -- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f463406b4d0 msgr2=0x7f463406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.794+0000 7f465a9cf640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f463406b4d0 0x7f463406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f465414c240 tx=0x7f463c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.794+0000 7f465a9cf640 1 -- 172.21.15.19:0/735310546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46540b5c30 msgr2=0x7f46540cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.794+0000 7f465a9cf640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46540b5c30 0x7f46540cd260 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f464402a1c0 tx=0x7f46440046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.795+0000 7f465a9cf640 1 -- 172.21.15.19:0/735310546 shutdown_connections 2024-04-25T09:54:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.795+0000 7f465a9cf640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f463406b4d0 0x7f463406d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.795+0000 7f465a9cf640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f465415db70 0x7f4654147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.795+0000 7f465a9cf640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46540b6cb0 0x7f465415e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.795+0000 7f465a9cf640 1 --2- 172.21.15.19:0/735310546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46540b5c30 0x7f46540cd260 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.795+0000 7f465a9cf640 1 -- 172.21.15.19:0/735310546 >> 172.21.15.19:0/735310546 conn(0x7f46540d0520 msgr2=0x7f465406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.795+0000 7f465a9cf640 1 -- 172.21.15.19:0/735310546 shutdown_connections 2024-04-25T09:54:10.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.795+0000 7f465a9cf640 1 -- 172.21.15.19:0/735310546 wait complete. 2024-04-25T09:54:10.808 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:54:10.808 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:54:10.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.910+0000 7fed611a9640 1 Processor -- start 2024-04-25T09:54:10.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.910+0000 7fed611a9640 1 -- start start 2024-04-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed611a9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fed5c0cc4f0 0x7fed5c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed611a9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed5c0cdb20 0x7fed5c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed611a9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5c072280 con 0x7fed5c15b270 2024-04-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed611a9640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5c071270 con 0x7fed5c0cdb20 2024-04-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed611a9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5c15bd80 con 0x7fed5c0cc4f0 2024-04-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed5a59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed5c0cdb20 0x7fed5c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed5a59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed5c0cdb20 0x7fed5c0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53288/0 (socket says 172.21.15.19:53288) 2024-04-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed5a59c640 1 -- 172.21.15.19:0/1705755656 learned_addr learned my addr 172.21.15.19:0/1705755656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed5ad9d640 1 --2- 172.21.15.19:0/1705755656 >> v2:172.21.15.112:3300/0 conn(0x7fed5c0cc4f0 0x7fed5c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed5a59c640 1 -- 172.21.15.19:0/1705755656 >> v2:172.21.15.112:3300/0 conn(0x7fed5c0cc4f0 msgr2=0x7fed5c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:10.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed5a59c640 1 --2- 172.21.15.19:0/1705755656 >> v2:172.21.15.112:3300/0 conn(0x7fed5c0cc4f0 0x7fed5c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed5a59c640 1 -- 172.21.15.19:0/1705755656 >> v1:172.21.15.112:6789/0 conn(0x7fed5c15b270 legacy=0x7fed5c15b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.911+0000 7fed5a59c640 1 -- 172.21.15.19:0/1705755656 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c0714c0 con 0x7fed5c0cdb20 2024-04-25T09:54:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.912+0000 7fed5a59c640 1 --2- 172.21.15.19:0/1705755656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed5c0cdb20 0x7fed5c0cdfb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0cd290 tx=0x7fed4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba70eb39ad2d7a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.912+0000 7fed59d9b640 1 -- 172.21.15.19:0/1705755656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44037070 con 0x7fed5c0cdb20 2024-04-25T09:54:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.912+0000 7fed59d9b640 1 -- 172.21.15.19:0/1705755656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed44032440 con 0x7fed5c0cdb20 2024-04-25T09:54:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.912+0000 7fed59d9b640 1 -- 172.21.15.19:0/1705755656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44032810 con 0x7fed5c0cdb20 2024-04-25T09:54:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.913+0000 7fed611a9640 1 -- 172.21.15.19:0/1705755656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed5c0cdb20 msgr2=0x7fed5c0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.913+0000 7fed611a9640 1 --2- 172.21.15.19:0/1705755656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed5c0cdb20 0x7fed5c0cdfb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0cd290 tx=0x7fed4402a270 comp rx=0 tx=0).stop 2024-04-25T09:54:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.913+0000 7fed611a9640 1 -- 172.21.15.19:0/1705755656 shutdown_connections 2024-04-25T09:54:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.913+0000 7fed611a9640 1 --2- 172.21.15.19:0/1705755656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed5c0cdb20 0x7fed5c0cdfb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.913+0000 7fed611a9640 1 --2- 172.21.15.19:0/1705755656 >> v2:172.21.15.112:3300/0 conn(0x7fed5c0cc4f0 0x7fed5c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.913+0000 7fed611a9640 1 -- 172.21.15.19:0/1705755656 >> 172.21.15.19:0/1705755656 conn(0x7fed5c06abe0 msgr2=0x7fed5c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.913+0000 7fed611a9640 1 -- 172.21.15.19:0/1705755656 shutdown_connections 2024-04-25T09:54:10.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.913+0000 7fed611a9640 1 -- 172.21.15.19:0/1705755656 wait complete. 2024-04-25T09:54:10.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.914+0000 7fed611a9640 1 Processor -- start 2024-04-25T09:54:10.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.915+0000 7fed611a9640 1 -- start start 2024-04-25T09:54:10.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.915+0000 7fed611a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c0cc4f0 0x7fed5c145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.915+0000 7fed611a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed5c0cdb20 0x7fed5c145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.915+0000 7fed611a9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5c15b270 0x7fed5c145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.915+0000 7fed611a9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c071d60 con 0x7fed5c0cdb20 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.915+0000 7fed611a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c071a80 con 0x7fed5c0cc4f0 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.915+0000 7fed611a9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c0ae500 con 0x7fed5c15b270 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.915+0000 7fed5a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed5c0cdb20 0x7fed5c145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.915+0000 7fed5a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed5c0cdb20 0x7fed5c145880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53292/0 (socket says 172.21.15.19:53292) 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.915+0000 7fed5a59c640 1 -- 172.21.15.19:0/1034715691 learned_addr learned my addr 172.21.15.19:0/1034715691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed5b59e640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5c15b270 0x7fed5c145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed5ad9d640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c0cc4f0 0x7fed5c145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed5a59c640 1 -- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5c15b270 msgr2=0x7fed5c145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed5a59c640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5c15b270 0x7fed5c145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed5a59c640 1 -- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c0cc4f0 msgr2=0x7fed5c145340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed5a59c640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c0cc4f0 0x7fed5c145340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed5a59c640 1 -- 172.21.15.19:0/1034715691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c153910 con 0x7fed5c0cdb20 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed5b59e640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5c15b270 0x7fed5c145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed5ad9d640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c0cc4f0 0x7fed5c145340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed5a59c640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed5c0cdb20 0x7fed5c145880 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fed44000c00 tx=0x7fed440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed537fe640 1 -- 172.21.15.19:0/1034715691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44041020 con 0x7fed5c0cdb20 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed537fe640 1 -- 172.21.15.19:0/1034715691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed440306c0 con 0x7fed5c0cdb20 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed611a9640 1 -- 172.21.15.19:0/1034715691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5c152d00 con 0x7fed5c0cdb20 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed537fe640 1 -- 172.21.15.19:0/1034715691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44037040 con 0x7fed5c0cdb20 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.916+0000 7fed611a9640 1 -- 172.21.15.19:0/1034715691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5c1532c0 con 0x7fed5c0cdb20 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.917+0000 7fed611a9640 1 -- 172.21.15.19:0/1034715691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5c073a00 con 0x7fed5c0cdb20 2024-04-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.918+0000 7fed537fe640 1 -- 172.21.15.19:0/1034715691 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed44044020 con 0x7fed5c0cdb20 2024-04-25T09:54:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.918+0000 7fed537fe640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3c06b500 0x7fed3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.918+0000 7fed537fe640 1 -- 172.21.15.19:0/1034715691 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fed440aa7b0 con 0x7fed5c0cdb20 2024-04-25T09:54:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.918+0000 7fed5ad9d640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3c06b500 0x7fed3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.919+0000 7fed5ad9d640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3c06b500 0x7fed3c06d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed5400b9f0 tx=0x7fed54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:10.922+0000 7fed537fe640 1 -- 172.21.15.19:0/1034715691 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed4404b030 con 0x7fed5c0cdb20 2024-04-25T09:54:11.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.101+0000 7fed611a9640 1 -- 172.21.15.19:0/1034715691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fed5c06d3a0 con 0x7fed5c0cdb20 2024-04-25T09:54:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.183+0000 7fed537fe640 1 -- 172.21.15.19:0/1034715691 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7fed44079880 con 0x7fed5c0cdb20 2024-04-25T09:54:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T09:54:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.185+0000 7fed611a9640 1 -- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3c06b500 msgr2=0x7fed3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.185+0000 7fed611a9640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3c06b500 0x7fed3c06d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed5400b9f0 tx=0x7fed54008040 comp rx=0 tx=0).stop 2024-04-25T09:54:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.185+0000 7fed611a9640 1 -- 172.21.15.19:0/1034715691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed5c0cdb20 msgr2=0x7fed5c145880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.185+0000 7fed611a9640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed5c0cdb20 0x7fed5c145880 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fed44000c00 tx=0x7fed440046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.186+0000 7fed611a9640 1 -- 172.21.15.19:0/1034715691 shutdown_connections 2024-04-25T09:54:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.186+0000 7fed611a9640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3c06b500 0x7fed3c06d9c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.186+0000 7fed611a9640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5c15b270 0x7fed5c145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.186+0000 7fed611a9640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed5c0cdb20 0x7fed5c145880 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.186+0000 7fed611a9640 1 --2- 172.21.15.19:0/1034715691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5c0cc4f0 0x7fed5c145340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.186+0000 7fed611a9640 1 -- 172.21.15.19:0/1034715691 >> 172.21.15.19:0/1034715691 conn(0x7fed5c06abe0 msgr2=0x7fed5c15b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.186+0000 7fed611a9640 1 -- 172.21.15.19:0/1034715691 shutdown_connections 2024-04-25T09:54:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.186+0000 7fed611a9640 1 -- 172.21.15.19:0/1034715691 wait complete. 2024-04-25T09:54:11.202 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:54:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.307+0000 7fce80038640 1 Processor -- start 2024-04-25T09:54:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.307+0000 7fce80038640 1 -- start start 2024-04-25T09:54:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.308+0000 7fce80038640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce7815b7d0 0x7fce780cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.308+0000 7fce80038640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7806cf50 con 0x7fce780b5620 2024-04-25T09:54:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.308+0000 7fce80038640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce780ce530 con 0x7fce7806c9e0 2024-04-25T09:54:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.308+0000 7fce80038640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce780ce810 con 0x7fce7815b7d0 2024-04-25T09:54:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.308+0000 7fce7d5d3640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fce780b5620 0x7fce7806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38786/0 (socket says 172.21.15.19:38786) 2024-04-25T09:54:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.308+0000 7fce7d5d3640 1 -- 172.21.15.19:0/3985408123 learned_addr learned my addr 172.21.15.19:0/3985408123 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.308+0000 7fce7ddd4640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce7815b7d0 0x7fce780cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.308+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626314560 0 0) 0x7fce7806cf50 con 0x7fce780b5620 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.308+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce64003520 con 0x7fce780b5620 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417429524 0 0) 0x7fce780ce530 con 0x7fce7806c9e0 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce7806cf50 con 0x7fce7806c9e0 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3213450322 0 0) 0x7fce64003520 con 0x7fce780b5620 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce7815b7d0 msgr2=0x7fce780cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce7815b7d0 0x7fce780cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 >> v1:172.21.15.112:6789/0 conn(0x7fce7806c9e0 legacy=0x7fce780d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce780ceaf0 con 0x7fce780b5620 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fce68001670 con 0x7fce780b5620 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 >> v1:172.21.15.19:6789/0 conn(0x7fce780b5620 legacy=0x7fce7806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce64003b70 0x7fce64003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce64004800 0x7fce64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce7815b7d0 0x7fce640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780ce810 con 0x7fce64004800 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6400ba70 con 0x7fce64003b70 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6400bc40 con 0x7fce7815b7d0 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7d5d3640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce64004800 0x7fce64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7ddd4640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce64003b70 0x7fce64003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7e5d5640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce7815b7d0 0x7fce640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7d5d3640 1 -- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce7815b7d0 msgr2=0x7fce640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7d5d3640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce7815b7d0 0x7fce640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7d5d3640 1 -- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce64003b70 msgr2=0x7fce64003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7d5d3640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce64003b70 0x7fce64003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7d5d3640 1 -- 172.21.15.19:0/3985408123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce64003520 con 0x7fce64004800 2024-04-25T09:54:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.309+0000 7fce7e5d5640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce7815b7d0 0x7fce640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.310+0000 7fce7d5d3640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce64004800 0x7fce64004c80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fce68007ca0 tx=0x7fce64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5f5527e82378dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.310+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68007ce0 con 0x7fce64004800 2024-04-25T09:54:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.310+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce68037070 con 0x7fce64004800 2024-04-25T09:54:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.310+0000 7fce7cdd2640 1 -- 172.21.15.19:0/3985408123 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68005d50 con 0x7fce64004800 2024-04-25T09:54:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.311+0000 7fce80038640 1 -- 172.21.15.19:0/3985408123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce64004800 msgr2=0x7fce64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.311+0000 7fce80038640 1 --2- 172.21.15.19:0/3985408123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce64004800 0x7fce64004c80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fce68007ca0 tx=0x7fce64005830 comp rx=0 tx=0).stop 2024-04-25T09:54:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.311+0000 7fce7e5d5640 1 -- 172.21.15.19:0/3985408123 reap_dead start 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.311+0000 7fce80038640 1 -- 172.21.15.19:0/3985408123 shutdown_connections 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.311+0000 7fce80038640 1 -- 172.21.15.19:0/3985408123 >> 172.21.15.19:0/3985408123 conn(0x7fce780d0800 msgr2=0x7fce780d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.311+0000 7fce80038640 1 -- 172.21.15.19:0/3985408123 shutdown_connections 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.311+0000 7fce80038640 1 -- 172.21.15.19:0/3985408123 wait complete. 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.312+0000 7fce80038640 1 Processor -- start 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.312+0000 7fce80038640 1 -- start start 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce80038640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce64004800 0x7fce78152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce80038640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce781534c0 0x7fce780c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce80038640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce78141db0 0x7fce78160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce80038640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7815bd30 con 0x7fce781534c0 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce80038640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7815bbb0 con 0x7fce64004800 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce80038640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce780d2650 con 0x7fce78141db0 2024-04-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7d5d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce781534c0 0x7fce780c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7d5d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce781534c0 0x7fce780c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53318/0 (socket says 172.21.15.19:53318) 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7d5d3640 1 -- 172.21.15.19:0/3845829144 learned_addr learned my addr 172.21.15.19:0/3845829144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7ddd4640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce64004800 0x7fce78152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7e5d5640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce78141db0 0x7fce78160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7d5d3640 1 -- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce78141db0 msgr2=0x7fce78160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7d5d3640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce78141db0 0x7fce78160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7d5d3640 1 -- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce64004800 msgr2=0x7fce78152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7d5d3640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce64004800 0x7fce78152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7d5d3640 1 -- 172.21.15.19:0/3845829144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce781422b0 con 0x7fce781534c0 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7ddd4640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce64004800 0x7fce78152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7e5d5640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce78141db0 0x7fce78160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce7d5d3640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce781534c0 0x7fce780c9fe0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fce6800c280 tx=0x7fce780cecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce6e7fc640 1 -- 172.21.15.19:0/3845829144 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6804d070 con 0x7fce781534c0 2024-04-25T09:54:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.313+0000 7fce6e7fc640 1 -- 172.21.15.19:0/3845829144 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce68004bf0 con 0x7fce781534c0 2024-04-25T09:54:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.314+0000 7fce80038640 1 -- 172.21.15.19:0/3845829144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce78142590 con 0x7fce781534c0 2024-04-25T09:54:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.314+0000 7fce6e7fc640 1 -- 172.21.15.19:0/3845829144 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68007ce0 con 0x7fce781534c0 2024-04-25T09:54:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.314+0000 7fce80038640 1 -- 172.21.15.19:0/3845829144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce78142b50 con 0x7fce781534c0 2024-04-25T09:54:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.315+0000 7fce80038640 1 -- 172.21.15.19:0/3845829144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce78073a00 con 0x7fce781534c0 2024-04-25T09:54:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.315+0000 7fce6e7fc640 1 -- 172.21.15.19:0/3845829144 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce68007070 con 0x7fce781534c0 2024-04-25T09:54:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.316+0000 7fce6e7fc640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce5c06b4d0 0x7fce5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.316+0000 7fce6e7fc640 1 -- 172.21.15.19:0/3845829144 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fce680b2310 con 0x7fce781534c0 2024-04-25T09:54:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.316+0000 7fce7ddd4640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce5c06b4d0 0x7fce5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.317+0000 7fce7ddd4640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce5c06b4d0 0x7fce5c06d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce781f2d40 tx=0x7fce74008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.320+0000 7fce6e7fc640 1 -- 172.21.15.19:0/3845829144 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce68080c70 con 0x7fce781534c0 2024-04-25T09:54:11.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:11.505+0000 7fce80038640 1 -- 172.21.15.19:0/3845829144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fce780b4770 con 0x7fce781534c0 2024-04-25T09:54:12.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.186+0000 7fce6e7fc640 1 -- 172.21.15.19:0/3845829144 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7fce68084d10 con 0x7fce781534c0 2024-04-25T09:54:12.188 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T09:54:12.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.189+0000 7fce80038640 1 -- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce5c06b4d0 msgr2=0x7fce5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:12.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.189+0000 7fce80038640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce5c06b4d0 0x7fce5c06d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce781f2d40 tx=0x7fce74008040 comp rx=0 tx=0).stop 2024-04-25T09:54:12.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.189+0000 7fce80038640 1 -- 172.21.15.19:0/3845829144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce781534c0 msgr2=0x7fce780c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:12.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.189+0000 7fce80038640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce781534c0 0x7fce780c9fe0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fce6800c280 tx=0x7fce780cecc0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.189+0000 7fce80038640 1 -- 172.21.15.19:0/3845829144 shutdown_connections 2024-04-25T09:54:12.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.189+0000 7fce80038640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce5c06b4d0 0x7fce5c06d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.189+0000 7fce80038640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce78141db0 0x7fce78160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.189+0000 7fce80038640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce781534c0 0x7fce780c9fe0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.189+0000 7fce80038640 1 --2- 172.21.15.19:0/3845829144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce64004800 0x7fce78152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.189+0000 7fce80038640 1 -- 172.21.15.19:0/3845829144 >> 172.21.15.19:0/3845829144 conn(0x7fce780d0800 msgr2=0x7fce7815ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:12.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.190+0000 7fce80038640 1 -- 172.21.15.19:0/3845829144 shutdown_connections 2024-04-25T09:54:12.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.190+0000 7fce80038640 1 -- 172.21.15.19:0/3845829144 wait complete. 2024-04-25T09:54:12.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:54:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.323+0000 7fdfbdaac640 1 Processor -- start 2024-04-25T09:54:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.324+0000 7fdfbdaac640 1 -- start start 2024-04-25T09:54:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.324+0000 7fdfbdaac640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdfb815dc20 0x7fdfb80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.324+0000 7fdfbdaac640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdfb8161710 0x7fdfb80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.324+0000 7fdfbdaac640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb815e000 con 0x7fdfb80b9330 2024-04-25T09:54:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.324+0000 7fdfbdaac640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb80d1de0 con 0x7fdfb815dc20 2024-04-25T09:54:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.324+0000 7fdfbdaac640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb8161c60 con 0x7fdfb8161710 2024-04-25T09:54:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.324+0000 7fdfb77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdfb815dc20 0x7fdfb80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.325+0000 7fdfb77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdfb815dc20 0x7fdfb80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53334/0 (socket says 172.21.15.19:53334) 2024-04-25T09:54:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.325+0000 7fdfb77fe640 1 -- 172.21.15.19:0/811705230 learned_addr learned my addr 172.21.15.19:0/811705230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.325+0000 7fdfb7fff640 1 --1- 172.21.15.19:0/811705230 >> v1:172.21.15.19:6789/0 conn(0x7fdfb80b9330 0x7fdfb806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38792/0 (socket says 172.21.15.19:38792) 2024-04-25T09:54:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.325+0000 7fdfb6ffd640 1 --2- 172.21.15.19:0/811705230 >> v2:172.21.15.112:3300/0 conn(0x7fdfb8161710 0x7fdfb80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.325+0000 7fdfb77fe640 1 -- 172.21.15.19:0/811705230 >> v2:172.21.15.112:3300/0 conn(0x7fdfb8161710 msgr2=0x7fdfb80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.325+0000 7fdfb77fe640 1 --2- 172.21.15.19:0/811705230 >> v2:172.21.15.112:3300/0 conn(0x7fdfb8161710 0x7fdfb80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.325+0000 7fdfb77fe640 1 -- 172.21.15.19:0/811705230 >> v1:172.21.15.19:6789/0 conn(0x7fdfb80b9330 legacy=0x7fdfb806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.325+0000 7fdfb77fe640 1 -- 172.21.15.19:0/811705230 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb8161f40 con 0x7fdfb815dc20 2024-04-25T09:54:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfb77fe640 1 --2- 172.21.15.19:0/811705230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb815dc20 0x7fdfb80ce670 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdfb806da90 tx=0x7fdfa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f04e5da792e0c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfb67fc640 1 -- 172.21.15.19:0/811705230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8037070 con 0x7fdfb815dc20 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfb67fc640 1 -- 172.21.15.19:0/811705230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa8032440 con 0x7fdfb815dc20 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfb67fc640 1 -- 172.21.15.19:0/811705230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8032810 con 0x7fdfb815dc20 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfbdaac640 1 -- 172.21.15.19:0/811705230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb815dc20 msgr2=0x7fdfb80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfbdaac640 1 --2- 172.21.15.19:0/811705230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb815dc20 0x7fdfb80ce670 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdfb806da90 tx=0x7fdfa802a270 comp rx=0 tx=0).stop 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfbdaac640 1 -- 172.21.15.19:0/811705230 shutdown_connections 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfbdaac640 1 --2- 172.21.15.19:0/811705230 >> v2:172.21.15.112:3300/0 conn(0x7fdfb8161710 0x7fdfb80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfbdaac640 1 --2- 172.21.15.19:0/811705230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb815dc20 0x7fdfb80ce670 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfbdaac640 1 -- 172.21.15.19:0/811705230 >> 172.21.15.19:0/811705230 conn(0x7fdfb80d0420 msgr2=0x7fdfb80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfbdaac640 1 -- 172.21.15.19:0/811705230 shutdown_connections 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.326+0000 7fdfbdaac640 1 -- 172.21.15.19:0/811705230 wait complete. 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.327+0000 7fdfbdaac640 1 Processor -- start 2024-04-25T09:54:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.328+0000 7fdfbdaac640 1 -- start start 2024-04-25T09:54:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.328+0000 7fdfbdaac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfb80b9330 0x7fdfb815d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.328+0000 7fdfbdaac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfb815dc20 0x7fdfb80b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.328+0000 7fdfbdaac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb8161710 0x7fdfb815c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.328+0000 7fdfbdaac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80b5e50 con 0x7fdfb8161710 2024-04-25T09:54:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.328+0000 7fdfbdaac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb8162500 con 0x7fdfb815dc20 2024-04-25T09:54:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.328+0000 7fdfbdaac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80b5fd0 con 0x7fdfb80b9330 2024-04-25T09:54:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb8161710 0x7fdfb815c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb8161710 0x7fdfb815c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53336/0 (socket says 172.21.15.19:53336) 2024-04-25T09:54:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb7fff640 1 -- 172.21.15.19:0/3381701745 learned_addr learned my addr 172.21.15.19:0/3381701745 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb77fe640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfb80b9330 0x7fdfb815d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb6ffd640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfb815dc20 0x7fdfb80b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb7fff640 1 -- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfb80b9330 msgr2=0x7fdfb815d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb7fff640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfb80b9330 0x7fdfb815d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb7fff640 1 -- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfb815dc20 msgr2=0x7fdfb80b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb7fff640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfb815dc20 0x7fdfb80b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb7fff640 1 -- 172.21.15.19:0/3381701745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb815ce80 con 0x7fdfb8161710 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb77fe640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfb80b9330 0x7fdfb815d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfb7fff640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb8161710 0x7fdfb815c460 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000c730 tx=0x7fdfb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdf9ffff640 1 -- 172.21.15.19:0/3381701745 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0018070 con 0x7fdfb8161710 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdf9ffff640 1 -- 172.21.15.19:0/3381701745 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb0004d60 con 0x7fdfb8161710 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfbdaac640 1 -- 172.21.15.19:0/3381701745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb815eec0 con 0x7fdfb8161710 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdf9ffff640 1 -- 172.21.15.19:0/3381701745 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb000a840 con 0x7fdfb8161710 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.329+0000 7fdfbdaac640 1 -- 172.21.15.19:0/3381701745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb81e86e0 con 0x7fdfb8161710 2024-04-25T09:54:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.330+0000 7fdfbdaac640 1 -- 172.21.15.19:0/3381701745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb81e88e0 con 0x7fdfb8161710 2024-04-25T09:54:12.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.331+0000 7fdf9ffff640 1 -- 172.21.15.19:0/3381701745 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfb000abf0 con 0x7fdfb8161710 2024-04-25T09:54:12.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.332+0000 7fdf9ffff640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfa406b4d0 0x7fdfa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.332+0000 7fdf9ffff640 1 -- 172.21.15.19:0/3381701745 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fdfb0089950 con 0x7fdfb8161710 2024-04-25T09:54:12.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.332+0000 7fdfb77fe640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfa406b4d0 0x7fdfa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:12.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.332+0000 7fdfb77fe640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfa406b4d0 0x7fdfa406d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8034000 tx=0x7fdfa802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:12.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.336+0000 7fdf9ffff640 1 -- 172.21.15.19:0/3381701745 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfb81e88e0 con 0x7fdfb8161710 2024-04-25T09:54:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:12.552+0000 7fdfbdaac640 1 -- 172.21.15.19:0/3381701745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdfb815f3e0 con 0x7fdfb8161710 2024-04-25T09:54:13.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.213+0000 7fdf9ffff640 1 -- 172.21.15.19:0/3381701745 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdfb0057f30 con 0x7fdfb8161710 2024-04-25T09:54:13.216 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T09:54:13.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.217+0000 7fdfbdaac640 1 -- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfa406b4d0 msgr2=0x7fdfa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.217+0000 7fdfbdaac640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfa406b4d0 0x7fdfa406d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8034000 tx=0x7fdfa802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:13.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.217+0000 7fdfbdaac640 1 -- 172.21.15.19:0/3381701745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb8161710 msgr2=0x7fdfb815c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.217+0000 7fdfbdaac640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb8161710 0x7fdfb815c460 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000c730 tx=0x7fdfb000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.223+0000 7fdfbdaac640 1 -- 172.21.15.19:0/3381701745 shutdown_connections 2024-04-25T09:54:13.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.223+0000 7fdfbdaac640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfa406b4d0 0x7fdfa406d990 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8034000 tx=0x7fdfa802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:13.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.223+0000 7fdfbdaac640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb8161710 0x7fdfb815c460 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.223+0000 7fdfbdaac640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfb815dc20 0x7fdfb80b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.223+0000 7fdfbdaac640 1 --2- 172.21.15.19:0/3381701745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfb80b9330 0x7fdfb815d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.223+0000 7fdfbdaac640 1 -- 172.21.15.19:0/3381701745 >> 172.21.15.19:0/3381701745 conn(0x7fdfb80d0420 msgr2=0x7fdfb80cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:13.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.225+0000 7fdfbdaac640 1 -- 172.21.15.19:0/3381701745 shutdown_connections 2024-04-25T09:54:13.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.225+0000 7fdfbdaac640 1 -- 172.21.15.19:0/3381701745 wait complete. 2024-04-25T09:54:13.247 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.350+0000 7f8f519c8640 1 Processor -- start 2024-04-25T09:54:13.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.351+0000 7f8f519c8640 1 -- start start 2024-04-25T09:54:13.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.351+0000 7f8f519c8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8f4c0d0b10 0x7f8f4c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.351+0000 7f8f519c8640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c0cec00 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.351+0000 7f8f519c8640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c0ceee0 con 0x7f8f4c15db60 2024-04-25T09:54:13.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.351+0000 7f8f519c8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4c0ce930 con 0x7f8f4c0d0b10 2024-04-25T09:54:13.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.352+0000 7f8f4b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8f4c0b5f00 0x7f8f4c0d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38800/0 (socket says 172.21.15.19:38800) 2024-04-25T09:54:13.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.352+0000 7f8f4b7fe640 1 -- 172.21.15.19:0/2286845884 learned_addr learned my addr 172.21.15.19:0/2286845884 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:13.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.352+0000 7f8f43fff640 1 --2- 172.21.15.19:0/2286845884 >> v2:172.21.15.112:3300/0 conn(0x7f8f4c0d0b10 0x7f8f4c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.352+0000 7f8f4a7fc640 1 -- 172.21.15.19:0/2286845884 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796435790 0 0) 0x7f8f4c0cec00 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.352+0000 7f8f4a7fc640 1 -- 172.21.15.19:0/2286845884 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f38003520 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.352+0000 7f8f43fff640 1 -- 172.21.15.19:0/2286845884 >> v1:172.21.15.112:6789/0 conn(0x7f8f4c15db60 legacy=0x7f8f4c0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.352+0000 7f8f4a7fc640 1 -- 172.21.15.19:0/2286845884 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122741783 0 0) 0x7f8f4c0ceee0 con 0x7f8f4c15db60 2024-04-25T09:54:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.352+0000 7f8f43fff640 1 -- 172.21.15.19:0/2286845884 >> v1:172.21.15.19:6789/0 conn(0x7f8f4c0b5f00 legacy=0x7f8f4c0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.352+0000 7f8f43fff640 1 -- 172.21.15.19:0/2286845884 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c0cf1c0 con 0x7f8f4c0d0b10 2024-04-25T09:54:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.353+0000 7f8f43fff640 1 --2- 172.21.15.19:0/2286845884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c0d0b10 0x7f8f4c0ce3f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8f340097d0 tx=0x7f8f3402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5d89751e60e8204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.353+0000 7f8f4a7fc640 1 -- 172.21.15.19:0/2286845884 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34037070 con 0x7f8f4c0d0b10 2024-04-25T09:54:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.353+0000 7f8f4a7fc640 1 -- 172.21.15.19:0/2286845884 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f34032440 con 0x7f8f4c0d0b10 2024-04-25T09:54:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.353+0000 7f8f4a7fc640 1 -- 172.21.15.19:0/2286845884 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34032810 con 0x7f8f4c0d0b10 2024-04-25T09:54:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.354+0000 7f8f519c8640 1 -- 172.21.15.19:0/2286845884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c0d0b10 msgr2=0x7f8f4c0ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.354+0000 7f8f519c8640 1 --2- 172.21.15.19:0/2286845884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c0d0b10 0x7f8f4c0ce3f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8f340097d0 tx=0x7f8f3402a1c0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.354+0000 7f8f519c8640 1 -- 172.21.15.19:0/2286845884 shutdown_connections 2024-04-25T09:54:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.354+0000 7f8f519c8640 1 --2- 172.21.15.19:0/2286845884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c0d0b10 0x7f8f4c0ce3f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.354+0000 7f8f519c8640 1 -- 172.21.15.19:0/2286845884 >> 172.21.15.19:0/2286845884 conn(0x7f8f4c061600 msgr2=0x7f8f4c061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.354+0000 7f8f519c8640 1 -- 172.21.15.19:0/2286845884 shutdown_connections 2024-04-25T09:54:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.354+0000 7f8f519c8640 1 -- 172.21.15.19:0/2286845884 wait complete. 2024-04-25T09:54:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.355+0000 7f8f519c8640 1 Processor -- start 2024-04-25T09:54:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.355+0000 7f8f519c8640 1 -- start start 2024-04-25T09:54:13.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f519c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f4c0b5f00 0x7f8f4c0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f519c8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f4c0d0b10 0x7f8f4c0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f519c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c15db60 0x7f8f4c15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f519c8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c15df40 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f519c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0b6450 con 0x7f8f4c15db60 2024-04-25T09:54:13.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f519c8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c15e0c0 con 0x7f8f4c0d0b10 2024-04-25T09:54:13.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f4affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f4c0b5f00 0x7f8f4c0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f4affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f4c0b5f00 0x7f8f4c0cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53350/0 (socket says 172.21.15.19:53350) 2024-04-25T09:54:13.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f4affd640 1 -- 172.21.15.19:0/1505318019 learned_addr learned my addr 172.21.15.19:0/1505318019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:13.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f4b7fe640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c15db60 0x7f8f4c15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f43fff640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f4c0d0b10 0x7f8f4c0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f4affd640 1 -- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f4c0d0b10 msgr2=0x7f8f4c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f4affd640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f4c0d0b10 0x7f8f4c0cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f4affd640 1 -- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c15db60 msgr2=0x7f8f4c15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f4affd640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c15db60 0x7f8f4c15a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f4affd640 1 -- 172.21.15.19:0/1505318019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c0cb6b0 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f4affd640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f4c0b5f00 0x7f8f4c0cab40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c3f0 tx=0x7f8f3c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f48ff9640 1 -- 172.21.15.19:0/1505318019 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c017070 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f48ff9640 1 -- 172.21.15.19:0/1505318019 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c0035f0 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f519c8640 1 -- 172.21.15.19:0/1505318019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c0cb990 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.356+0000 7f8f48ff9640 1 -- 172.21.15.19:0/1505318019 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0039e0 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.357+0000 7f8f519c8640 1 -- 172.21.15.19:0/1505318019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c0cbf50 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.357+0000 7f8f519c8640 1 -- 172.21.15.19:0/1505318019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4c073270 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.358+0000 7f8f48ff9640 1 -- 172.21.15.19:0/1505318019 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f3c01a070 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.358+0000 7f8f48ff9640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f2c06b500 0x7f8f2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.358+0000 7f8f48ff9640 1 -- 172.21.15.19:0/1505318019 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f3c0897a0 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.359+0000 7f8f43fff640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f2c06b500 0x7f8f2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.359+0000 7f8f43fff640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f2c06b500 0x7f8f2c06d9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8f34034000 tx=0x7f8f3402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.363+0000 7f8f48ff9640 1 -- 172.21.15.19:0/1505318019 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f3c057d20 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.547+0000 7f8f519c8640 1 -- 172.21.15.19:0/1505318019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f4c0d1860 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.548+0000 7f8f48ff9640 1 -- 172.21.15.19:0/1505318019 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f8f3c05bdc0 con 0x7f8f4c0b5f00 2024-04-25T09:54:13.551 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:13.551 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":37,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:54:12.553989+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:11.132892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:11.536478+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 -- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f2c06b500 msgr2=0x7f8f2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f2c06b500 0x7f8f2c06d9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8f34034000 tx=0x7f8f3402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 -- 172.21.15.19:0/1505318019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f4c0b5f00 msgr2=0x7f8f4c0cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f4c0b5f00 0x7f8f4c0cab40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c3f0 tx=0x7f8f3c00c8b0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 -- 172.21.15.19:0/1505318019 shutdown_connections 2024-04-25T09:54:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f2c06b500 0x7f8f2c06d9c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4c15db60 0x7f8f4c15a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f4c0d0b10 0x7f8f4c0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 --2- 172.21.15.19:0/1505318019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f4c0b5f00 0x7f8f4c0cab40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 -- 172.21.15.19:0/1505318019 >> 172.21.15.19:0/1505318019 conn(0x7f8f4c061600 msgr2=0x7f8f4c15b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 -- 172.21.15.19:0/1505318019 shutdown_connections 2024-04-25T09:54:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.550+0000 7f8f519c8640 1 -- 172.21.15.19:0/1505318019 wait complete. 2024-04-25T09:54:13.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.662+0000 7f32fdc5d640 1 Processor -- start 2024-04-25T09:54:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.662+0000 7f32fdc5d640 1 -- start start 2024-04-25T09:54:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.663+0000 7f32fdc5d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f32f815dc00 0x7f32f80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.663+0000 7f32fdc5d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 0x7f32f80d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.663+0000 7f32fdc5d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f815dfe0 con 0x7f32f80d1c00 2024-04-25T09:54:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.663+0000 7f32fdc5d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f80d2060 con 0x7f32f815dc00 2024-04-25T09:54:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.663+0000 7f32fdc5d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80d00c0 con 0x7f32f80ceb20 2024-04-25T09:54:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.663+0000 7f32f6ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 0x7f32f80d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.663+0000 7f32f77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f32f815dc00 0x7f32f80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.663+0000 7f32f6ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 0x7f32f80d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:48500/0 (socket says 172.21.15.19:48500) 2024-04-25T09:54:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.663+0000 7f32f6ffd640 1 -- 172.21.15.19:0/363278742 learned_addr learned my addr 172.21.15.19:0/363278742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.663+0000 7f32f7fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f32f80d1c00 0x7f32f80ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:51856/0 (socket says 172.21.15.19:51856) 2024-04-25T09:54:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.664+0000 7f32f67fc640 1 -- 172.21.15.19:0/363278742 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646829539 0 0) 0x7f32f0001490 con 0x7f32f80d1c00 2024-04-25T09:54:13.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.664+0000 7f32f6ffd640 1 -- 172.21.15.19:0/363278742 >> v2:172.21.15.112:3300/0 conn(0x7f32f815dc00 msgr2=0x7f32f80c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:13.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.664+0000 7f32f6ffd640 1 --2- 172.21.15.19:0/363278742 >> v2:172.21.15.112:3300/0 conn(0x7f32f815dc00 0x7f32f80c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.664+0000 7f32f6ffd640 1 -- 172.21.15.19:0/363278742 >> v1:172.21.15.112:6789/0 conn(0x7f32f80d1c00 legacy=0x7f32f80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.664+0000 7f32f6ffd640 1 -- 172.21.15.19:0/363278742 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f80d03a0 con 0x7f32f80ceb20 2024-04-25T09:54:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.664+0000 7f32f6ffd640 1 --2- 172.21.15.19:0/363278742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 0x7f32f80d16c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009870 tx=0x7f32e802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a4aeecfd1e90ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.665+0000 7f32f67fc640 1 -- 172.21.15.19:0/363278742 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8037070 con 0x7f32f80ceb20 2024-04-25T09:54:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.665+0000 7f32f67fc640 1 -- 172.21.15.19:0/363278742 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e8032440 con 0x7f32f80ceb20 2024-04-25T09:54:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.665+0000 7f32f67fc640 1 -- 172.21.15.19:0/363278742 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8032810 con 0x7f32f80ceb20 2024-04-25T09:54:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.665+0000 7f32fdc5d640 1 -- 172.21.15.19:0/363278742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 msgr2=0x7f32f80d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.665+0000 7f32fdc5d640 1 --2- 172.21.15.19:0/363278742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 0x7f32f80d16c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009870 tx=0x7f32e802a270 comp rx=0 tx=0).stop 2024-04-25T09:54:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.665+0000 7f32fdc5d640 1 -- 172.21.15.19:0/363278742 shutdown_connections 2024-04-25T09:54:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.665+0000 7f32fdc5d640 1 --2- 172.21.15.19:0/363278742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 0x7f32f80d16c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.665+0000 7f32fdc5d640 1 --2- 172.21.15.19:0/363278742 >> v2:172.21.15.112:3300/0 conn(0x7f32f815dc00 0x7f32f80c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.665+0000 7f32fdc5d640 1 -- 172.21.15.19:0/363278742 >> 172.21.15.19:0/363278742 conn(0x7f32f806f5d0 msgr2=0x7f32f806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.665+0000 7f32fdc5d640 1 -- 172.21.15.19:0/363278742 shutdown_connections 2024-04-25T09:54:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.665+0000 7f32fdc5d640 1 -- 172.21.15.19:0/363278742 wait complete. 2024-04-25T09:54:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.666+0000 7f32fdc5d640 1 Processor -- start 2024-04-25T09:54:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.667+0000 7f32fdc5d640 1 -- start start 2024-04-25T09:54:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.667+0000 7f32fdc5d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 0x7f32f80b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.667+0000 7f32fdc5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f80d1c00 0x7f32f80c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.667+0000 7f32fdc5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f815dc00 0x7f32f80c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.667+0000 7f32fdc5d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80cc130 con 0x7f32f815dc00 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.667+0000 7f32fdc5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80cc720 con 0x7f32f80d1c00 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.667+0000 7f32fdc5d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80cfd90 con 0x7f32f80ceb20 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f815dc00 0x7f32f80c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f815dc00 0x7f32f80c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53364/0 (socket says 172.21.15.19:53364) 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f7fff640 1 -- 172.21.15.19:0/2972773866 learned_addr learned my addr 172.21.15.19:0/2972773866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f77fe640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 0x7f32f80b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f6ffd640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f80d1c00 0x7f32f80c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f7fff640 1 -- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 msgr2=0x7f32f80b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f7fff640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 0x7f32f80b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f7fff640 1 -- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f80d1c00 msgr2=0x7f32f80c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f7fff640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f80d1c00 0x7f32f80c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f7fff640 1 -- 172.21.15.19:0/2972773866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f815d110 con 0x7f32f815dc00 2024-04-25T09:54:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f6ffd640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f80d1c00 0x7f32f80c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32f7fff640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f815dc00 0x7f32f80c9300 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c7f0 tx=0x7f32f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32effff640 1 -- 172.21.15.19:0/2972773866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0018070 con 0x7f32f815dc00 2024-04-25T09:54:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32effff640 1 -- 172.21.15.19:0/2972773866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f000a5e0 con 0x7f32f815dc00 2024-04-25T09:54:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32fdc5d640 1 -- 172.21.15.19:0/2972773866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f80c9930 con 0x7f32f815dc00 2024-04-25T09:54:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32effff640 1 -- 172.21.15.19:0/2972773866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000aa10 con 0x7f32f815dc00 2024-04-25T09:54:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.668+0000 7f32fdc5d640 1 -- 172.21.15.19:0/2972773866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f80c9ef0 con 0x7f32f815dc00 2024-04-25T09:54:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.669+0000 7f32fdc5d640 1 -- 172.21.15.19:0/2972773866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f80b0e90 con 0x7f32f815dc00 2024-04-25T09:54:13.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.670+0000 7f32effff640 1 -- 172.21.15.19:0/2972773866 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32f001b070 con 0x7f32f815dc00 2024-04-25T09:54:13.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.670+0000 7f32effff640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32d806b4d0 0x7f32d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.670+0000 7f32effff640 1 -- 172.21.15.19:0/2972773866 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32f0089960 con 0x7f32f815dc00 2024-04-25T09:54:13.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.670+0000 7f32f77fe640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32d806b4d0 0x7f32d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.671+0000 7f32f77fe640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32d806b4d0 0x7f32d806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009a30 tx=0x7f32e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.674+0000 7f32effff640 1 -- 172.21.15.19:0/2972773866 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32f80b0e90 con 0x7f32f815dc00 2024-04-25T09:54:13.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.854+0000 7f32fdc5d640 1 -- 172.21.15.19:0/2972773866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32f80c66c0 con 0x7f32f815dc00 2024-04-25T09:54:13.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.855+0000 7f32effff640 1 -- 172.21.15.19:0/2972773866 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f32f815dfe0 con 0x7f32f815dc00 2024-04-25T09:54:13.858 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:13.858 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":37,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:54:12.553989+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:11.132892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:11.536478+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:13.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.857+0000 7f32fdc5d640 1 -- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32d806b4d0 msgr2=0x7f32d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.857+0000 7f32fdc5d640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32d806b4d0 0x7f32d806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009a30 tx=0x7f32e0008040 comp rx=0 tx=0).stop 2024-04-25T09:54:13.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.857+0000 7f32fdc5d640 1 -- 172.21.15.19:0/2972773866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f815dc00 msgr2=0x7f32f80c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.857+0000 7f32fdc5d640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f815dc00 0x7f32f80c9300 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c7f0 tx=0x7f32f000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.857+0000 7f32fdc5d640 1 -- 172.21.15.19:0/2972773866 shutdown_connections 2024-04-25T09:54:13.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.857+0000 7f32fdc5d640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32d806b4d0 0x7f32d806d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.857+0000 7f32fdc5d640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f815dc00 0x7f32f80c9300 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.857+0000 7f32fdc5d640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f80d1c00 0x7f32f80c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.857+0000 7f32fdc5d640 1 --2- 172.21.15.19:0/2972773866 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f80ceb20 0x7f32f80b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.857+0000 7f32fdc5d640 1 -- 172.21.15.19:0/2972773866 >> 172.21.15.19:0/2972773866 conn(0x7f32f806f5d0 msgr2=0x7f32f8159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:13.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.858+0000 7f32fdc5d640 1 -- 172.21.15.19:0/2972773866 shutdown_connections 2024-04-25T09:54:13.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.858+0000 7f32fdc5d640 1 -- 172.21.15.19:0/2972773866 wait complete. 2024-04-25T09:54:13.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:54:13.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.973+0000 7f57d077e640 1 Processor -- start 2024-04-25T09:54:13.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.973+0000 7f57d077e640 1 -- start start 2024-04-25T09:54:13.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.974+0000 7f57d077e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f57c815dc20 0x7f57c80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.974+0000 7f57d077e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f57c8161710 0x7f57c80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.974+0000 7f57d077e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c815e000 con 0x7f57c80b9330 2024-04-25T09:54:13.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.974+0000 7f57d077e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c80d1de0 con 0x7f57c815dc20 2024-04-25T09:54:13.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.974+0000 7f57d077e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c8161c60 con 0x7f57c8161710 2024-04-25T09:54:13.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.974+0000 7f57ce51a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f57c815dc20 0x7f57c80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.974+0000 7f57ce51a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f57c815dc20 0x7f57c80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53366/0 (socket says 172.21.15.19:53366) 2024-04-25T09:54:13.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.974+0000 7f57ce51a640 1 -- 172.21.15.19:0/2134835983 learned_addr learned my addr 172.21.15.19:0/2134835983 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:13.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.974+0000 7f57cdd19640 1 --2- 172.21.15.19:0/2134835983 >> v2:172.21.15.112:3300/0 conn(0x7f57c8161710 0x7f57c80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.975+0000 7f57ce51a640 1 -- 172.21.15.19:0/2134835983 >> v2:172.21.15.112:3300/0 conn(0x7f57c8161710 msgr2=0x7f57c80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:13.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.975+0000 7f57ce51a640 1 --2- 172.21.15.19:0/2134835983 >> v2:172.21.15.112:3300/0 conn(0x7f57c8161710 0x7f57c80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.975+0000 7f57ce51a640 1 -- 172.21.15.19:0/2134835983 >> v1:172.21.15.112:6789/0 conn(0x7f57c80b9330 legacy=0x7f57c806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.975+0000 7f57ce51a640 1 -- 172.21.15.19:0/2134835983 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c8161f40 con 0x7f57c815dc20 2024-04-25T09:54:13.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.975+0000 7f57cdd19640 1 --2- 172.21.15.19:0/2134835983 >> v2:172.21.15.112:3300/0 conn(0x7f57c8161710 0x7f57c80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:13.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.975+0000 7f57ce51a640 1 --2- 172.21.15.19:0/2134835983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c815dc20 0x7f57c80ce670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f57c806da90 tx=0x7f57c002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3305851012b05e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.975+0000 7f57cd518640 1 -- 172.21.15.19:0/2134835983 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0036070 con 0x7f57c815dc20 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.976+0000 7f57cd518640 1 -- 172.21.15.19:0/2134835983 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c0031600 con 0x7f57c815dc20 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.976+0000 7f57cd518640 1 -- 172.21.15.19:0/2134835983 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00319d0 con 0x7f57c815dc20 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.976+0000 7f57d077e640 1 -- 172.21.15.19:0/2134835983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c815dc20 msgr2=0x7f57c80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.976+0000 7f57d077e640 1 --2- 172.21.15.19:0/2134835983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c815dc20 0x7f57c80ce670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f57c806da90 tx=0x7f57c002a730 comp rx=0 tx=0).stop 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.976+0000 7f57d077e640 1 -- 172.21.15.19:0/2134835983 shutdown_connections 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.976+0000 7f57d077e640 1 --2- 172.21.15.19:0/2134835983 >> v2:172.21.15.112:3300/0 conn(0x7f57c8161710 0x7f57c80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.976+0000 7f57d077e640 1 --2- 172.21.15.19:0/2134835983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c815dc20 0x7f57c80ce670 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.976+0000 7f57d077e640 1 -- 172.21.15.19:0/2134835983 >> 172.21.15.19:0/2134835983 conn(0x7f57c80d0420 msgr2=0x7f57c80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.976+0000 7f57d077e640 1 -- 172.21.15.19:0/2134835983 shutdown_connections 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.976+0000 7f57d077e640 1 -- 172.21.15.19:0/2134835983 wait complete. 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.977+0000 7f57d077e640 1 Processor -- start 2024-04-25T09:54:13.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.977+0000 7f57d077e640 1 -- start start 2024-04-25T09:54:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57d077e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c80b9330 0x7f57c81f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57d077e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57c815dc20 0x7f57c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57d077e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57c8161710 0x7f57c81f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57d077e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c80b5e50 con 0x7f57c80b9330 2024-04-25T09:54:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57d077e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c8162500 con 0x7f57c8161710 2024-04-25T09:54:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57d077e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c80b5fd0 con 0x7f57c815dc20 2024-04-25T09:54:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57ce51a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c80b9330 0x7f57c81f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57ce51a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c80b9330 0x7f57c81f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53382/0 (socket says 172.21.15.19:53382) 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57ce51a640 1 -- 172.21.15.19:0/1424490940 learned_addr learned my addr 172.21.15.19:0/1424490940 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57cdd19640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57c815dc20 0x7f57c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57ced1b640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57c8161710 0x7f57c81f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57ce51a640 1 -- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57c815dc20 msgr2=0x7f57c80b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57ce51a640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57c815dc20 0x7f57c80b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57ce51a640 1 -- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57c8161710 msgr2=0x7f57c81f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57ce51a640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57c8161710 0x7f57c81f0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57ce51a640 1 -- 172.21.15.19:0/1424490940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c81f4bb0 con 0x7f57c80b9330 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.978+0000 7f57cdd19640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57c815dc20 0x7f57c80b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.979+0000 7f57ce51a640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c80b9330 0x7f57c81f4670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f57c0034040 tx=0x7f57c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.979+0000 7f57c6ffd640 1 -- 172.21.15.19:0/1424490940 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0036050 con 0x7f57c80b9330 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.979+0000 7f57c6ffd640 1 -- 172.21.15.19:0/1424490940 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c0042070 con 0x7f57c80b9330 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.979+0000 7f57d077e640 1 -- 172.21.15.19:0/1424490940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c80aa720 con 0x7f57c80b9330 2024-04-25T09:54:13.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.979+0000 7f57c6ffd640 1 -- 172.21.15.19:0/1424490940 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0009510 con 0x7f57c80b9330 2024-04-25T09:54:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.979+0000 7f57d077e640 1 -- 172.21.15.19:0/1424490940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c81f5130 con 0x7f57c80b9330 2024-04-25T09:54:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.980+0000 7f57d077e640 1 -- 172.21.15.19:0/1424490940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c80ce340 con 0x7f57c80b9330 2024-04-25T09:54:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.980+0000 7f57c6ffd640 1 -- 172.21.15.19:0/1424490940 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57c0007c50 con 0x7f57c80b9330 2024-04-25T09:54:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.981+0000 7f57c6ffd640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f57b406b4d0 0x7f57b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.981+0000 7f57c6ffd640 1 -- 172.21.15.19:0/1424490940 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57c00ab1c0 con 0x7f57c80b9330 2024-04-25T09:54:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.981+0000 7f57cdd19640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f57b406b4d0 0x7f57b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.982+0000 7f57cdd19640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f57b406b4d0 0x7f57b406d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f57c80cf480 tx=0x7f57bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:13.985+0000 7f57c6ffd640 1 -- 172.21.15.19:0/1424490940 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57c80ce340 con 0x7f57c80b9330 2024-04-25T09:54:14.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.206+0000 7f57d077e640 1 -- 172.21.15.19:0/1424490940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f57c80cfc20 con 0x7f57c80b9330 2024-04-25T09:54:14.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.221+0000 7f57c6ffd640 1 -- 172.21.15.19:0/1424490940 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f57c80cfc20 con 0x7f57c80b9330 2024-04-25T09:54:14.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 -- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f57b406b4d0 msgr2=0x7f57b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:14.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f57b406b4d0 0x7f57b406d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f57c80cf480 tx=0x7f57bc008040 comp rx=0 tx=0).stop 2024-04-25T09:54:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 -- 172.21.15.19:0/1424490940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c80b9330 msgr2=0x7f57c81f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c80b9330 0x7f57c81f4670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f57c0034040 tx=0x7f57c0004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 -- 172.21.15.19:0/1424490940 shutdown_connections 2024-04-25T09:54:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f57b406b4d0 0x7f57b406d990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57c8161710 0x7f57c81f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57c815dc20 0x7f57c80b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 --2- 172.21.15.19:0/1424490940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c80b9330 0x7f57c81f4670 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 -- 172.21.15.19:0/1424490940 >> 172.21.15.19:0/1424490940 conn(0x7f57c80d0420 msgr2=0x7f57c80b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 -- 172.21.15.19:0/1424490940 shutdown_connections 2024-04-25T09:54:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.224+0000 7f57d077e640 1 -- 172.21.15.19:0/1424490940 wait complete. 2024-04-25T09:54:14.237 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:54:14.237 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:54:14.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.339+0000 7fb64c528640 1 Processor -- start 2024-04-25T09:54:14.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.339+0000 7fb64c528640 1 -- start start 2024-04-25T09:54:14.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.339+0000 7fb64c528640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb64415dc20 0x7fb6440d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb64c528640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb6440d1ce0 0x7fb6440b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb64c528640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6440d2120 con 0x7fb6440ce670 2024-04-25T09:54:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb64c528640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6440d23a0 con 0x7fb6440d1ce0 2024-04-25T09:54:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb64c528640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6440d2680 con 0x7fb64415dc20 2024-04-25T09:54:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb649ac3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb6440d1ce0 0x7fb6440b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb649ac3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb6440d1ce0 0x7fb6440b9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53394/0 (socket says 172.21.15.19:53394) 2024-04-25T09:54:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb649ac3640 1 -- 172.21.15.19:0/1485003555 learned_addr learned my addr 172.21.15.19:0/1485003555 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb64a2c4640 1 --2- 172.21.15.19:0/1485003555 >> v2:172.21.15.112:3300/0 conn(0x7fb64415dc20 0x7fb6440d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb649ac3640 1 -- 172.21.15.19:0/1485003555 >> v2:172.21.15.112:3300/0 conn(0x7fb64415dc20 msgr2=0x7fb6440d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb649ac3640 1 --2- 172.21.15.19:0/1485003555 >> v2:172.21.15.112:3300/0 conn(0x7fb64415dc20 0x7fb6440d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb649ac3640 1 -- 172.21.15.19:0/1485003555 >> v1:172.21.15.19:6789/0 conn(0x7fb6440ce670 legacy=0x7fb6440cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb649ac3640 1 -- 172.21.15.19:0/1485003555 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6440d2960 con 0x7fb6440d1ce0 2024-04-25T09:54:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.340+0000 7fb64a2c4640 1 --2- 172.21.15.19:0/1485003555 >> v2:172.21.15.112:3300/0 conn(0x7fb64415dc20 0x7fb6440d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.341+0000 7fb649ac3640 1 --2- 172.21.15.19:0/1485003555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6440d1ce0 0x7fb6440b9490 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb6440d1500 tx=0x7fb63402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f871f99226f927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.341+0000 7fb6492c2640 1 -- 172.21.15.19:0/1485003555 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634037070 con 0x7fb6440d1ce0 2024-04-25T09:54:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.341+0000 7fb6492c2640 1 -- 172.21.15.19:0/1485003555 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb634032440 con 0x7fb6440d1ce0 2024-04-25T09:54:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.341+0000 7fb6492c2640 1 -- 172.21.15.19:0/1485003555 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634032810 con 0x7fb6440d1ce0 2024-04-25T09:54:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.341+0000 7fb64c528640 1 -- 172.21.15.19:0/1485003555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6440d1ce0 msgr2=0x7fb6440b9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.341+0000 7fb64c528640 1 --2- 172.21.15.19:0/1485003555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6440d1ce0 0x7fb6440b9490 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb6440d1500 tx=0x7fb63402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.342+0000 7fb64c528640 1 -- 172.21.15.19:0/1485003555 shutdown_connections 2024-04-25T09:54:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.342+0000 7fb64c528640 1 --2- 172.21.15.19:0/1485003555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6440d1ce0 0x7fb6440b9490 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.342+0000 7fb64c528640 1 --2- 172.21.15.19:0/1485003555 >> v2:172.21.15.112:3300/0 conn(0x7fb64415dc20 0x7fb6440d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.342+0000 7fb64c528640 1 -- 172.21.15.19:0/1485003555 >> 172.21.15.19:0/1485003555 conn(0x7fb644061a20 msgr2=0x7fb644061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.342+0000 7fb64c528640 1 -- 172.21.15.19:0/1485003555 shutdown_connections 2024-04-25T09:54:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.342+0000 7fb64c528640 1 -- 172.21.15.19:0/1485003555 wait complete. 2024-04-25T09:54:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.343+0000 7fb64c528640 1 Processor -- start 2024-04-25T09:54:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.343+0000 7fb64c528640 1 -- start start 2024-04-25T09:54:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.343+0000 7fb64c528640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6440ce670 0x7fb64414c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.343+0000 7fb64c528640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6440d1ce0 0x7fb6441f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.343+0000 7fb64c528640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb64415dc20 0x7fb6440c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb64c528640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440b6a30 con 0x7fb6440d1ce0 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb64c528640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440d2f20 con 0x7fb64415dc20 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb64c528640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440b6bb0 con 0x7fb6440ce670 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb649ac3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6440d1ce0 0x7fb6441f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb649ac3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6440d1ce0 0x7fb6441f6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53400/0 (socket says 172.21.15.19:53400) 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb649ac3640 1 -- 172.21.15.19:0/184380738 learned_addr learned my addr 172.21.15.19:0/184380738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb64a2c4640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6440ce670 0x7fb64414c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb64aac5640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb64415dc20 0x7fb6440c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb649ac3640 1 -- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6440ce670 msgr2=0x7fb64414c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb649ac3640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6440ce670 0x7fb64414c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb649ac3640 1 -- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb64415dc20 msgr2=0x7fb6440c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb649ac3640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb64415dc20 0x7fb6440c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb649ac3640 1 -- 172.21.15.19:0/184380738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6441f74f0 con 0x7fb6440d1ce0 2024-04-25T09:54:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb649ac3640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6440d1ce0 0x7fb6441f6dc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb63402a220 tx=0x7fb6340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb63affd640 1 -- 172.21.15.19:0/184380738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634037040 con 0x7fb6440d1ce0 2024-04-25T09:54:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb63affd640 1 -- 172.21.15.19:0/184380738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb634042070 con 0x7fb6440d1ce0 2024-04-25T09:54:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb63affd640 1 -- 172.21.15.19:0/184380738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634030bb0 con 0x7fb6440d1ce0 2024-04-25T09:54:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb64c528640 1 -- 172.21.15.19:0/184380738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6441f77d0 con 0x7fb6440d1ce0 2024-04-25T09:54:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.344+0000 7fb64c528640 1 -- 172.21.15.19:0/184380738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6440c86c0 con 0x7fb6440d1ce0 2024-04-25T09:54:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.345+0000 7fb64c528640 1 -- 172.21.15.19:0/184380738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb644073a00 con 0x7fb6440d1ce0 2024-04-25T09:54:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.346+0000 7fb63affd640 1 -- 172.21.15.19:0/184380738 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb634047460 con 0x7fb6440d1ce0 2024-04-25T09:54:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.346+0000 7fb63affd640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62806b500 0x7fb62806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.346+0000 7fb63affd640 1 -- 172.21.15.19:0/184380738 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb6340aaf80 con 0x7fb6440d1ce0 2024-04-25T09:54:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.346+0000 7fb64a2c4640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62806b500 0x7fb62806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.347+0000 7fb64a2c4640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62806b500 0x7fb62806d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb640007510 tx=0x7fb6400074a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.351+0000 7fb63affd640 1 -- 172.21.15.19:0/184380738 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb634045360 con 0x7fb6440d1ce0 2024-04-25T09:54:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:14.569+0000 7fb64c528640 1 -- 172.21.15.19:0/184380738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb6440b7800 con 0x7fb6440d1ce0 2024-04-25T09:54:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.226+0000 7fb63affd640 1 -- 172.21.15.19:0/184380738 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb634079d00 con 0x7fb6440d1ce0 2024-04-25T09:54:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 -- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62806b500 msgr2=0x7fb62806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62806b500 0x7fb62806d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb640007510 tx=0x7fb6400074a0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 -- 172.21.15.19:0/184380738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6440d1ce0 msgr2=0x7fb6441f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6440d1ce0 0x7fb6441f6dc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb63402a220 tx=0x7fb6340046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 -- 172.21.15.19:0/184380738 shutdown_connections 2024-04-25T09:54:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62806b500 0x7fb62806d9c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb64415dc20 0x7fb6440c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6440d1ce0 0x7fb6441f6dc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 --2- 172.21.15.19:0/184380738 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6440ce670 0x7fb64414c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 -- 172.21.15.19:0/184380738 >> 172.21.15.19:0/184380738 conn(0x7fb644061a20 msgr2=0x7fb6440b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 -- 172.21.15.19:0/184380738 shutdown_connections 2024-04-25T09:54:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.229+0000 7fb64c528640 1 -- 172.21.15.19:0/184380738 wait complete. 2024-04-25T09:54:15.244 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:54:15.245 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.349+0000 7f8b78ca1640 1 Processor -- start 2024-04-25T09:54:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.349+0000 7f8b78ca1640 1 -- start start 2024-04-25T09:54:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b78ca1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8b7415db70 0x7f8b74162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b78ca1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b740b5c30 0x7f8b740d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b78ca1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b74162690 con 0x7f8b740b6cb0 2024-04-25T09:54:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b78ca1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b740ceee0 con 0x7f8b740b5c30 2024-04-25T09:54:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b78ca1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b740d1d10 con 0x7f8b7415db70 2024-04-25T09:54:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b72d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b740b5c30 0x7f8b740d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b72d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b740b5c30 0x7f8b740d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53414/0 (socket says 172.21.15.19:53414) 2024-04-25T09:54:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b72d9d640 1 -- 172.21.15.19:0/899330240 learned_addr learned my addr 172.21.15.19:0/899330240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b72d9d640 1 -- 172.21.15.19:0/899330240 >> v2:172.21.15.112:3300/0 conn(0x7f8b7415db70 msgr2=0x7f8b74162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b72d9d640 1 --2- 172.21.15.19:0/899330240 >> v2:172.21.15.112:3300/0 conn(0x7f8b7415db70 0x7f8b74162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b72d9d640 1 -- 172.21.15.19:0/899330240 >> v1:172.21.15.19:6789/0 conn(0x7f8b740b6cb0 legacy=0x7f8b740b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.350+0000 7f8b72d9d640 1 -- 172.21.15.19:0/899330240 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b740d1ff0 con 0x7f8b740b5c30 2024-04-25T09:54:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.351+0000 7f8b72d9d640 1 --2- 172.21.15.19:0/899330240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b740b5c30 0x7f8b740d27f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8b740b6570 tx=0x7f8b6c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a9384cd8bc25ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.351+0000 7f8b7159a640 1 -- 172.21.15.19:0/899330240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c036070 con 0x7f8b740b5c30 2024-04-25T09:54:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.351+0000 7f8b7159a640 1 -- 172.21.15.19:0/899330240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c031600 con 0x7f8b740b5c30 2024-04-25T09:54:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.351+0000 7f8b7159a640 1 -- 172.21.15.19:0/899330240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0319d0 con 0x7f8b740b5c30 2024-04-25T09:54:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.352+0000 7f8b78ca1640 1 -- 172.21.15.19:0/899330240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b740b5c30 msgr2=0x7f8b740d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.352+0000 7f8b78ca1640 1 --2- 172.21.15.19:0/899330240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b740b5c30 0x7f8b740d27f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8b740b6570 tx=0x7f8b6c02a730 comp rx=0 tx=0).stop 2024-04-25T09:54:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.352+0000 7f8b78ca1640 1 -- 172.21.15.19:0/899330240 shutdown_connections 2024-04-25T09:54:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.352+0000 7f8b78ca1640 1 --2- 172.21.15.19:0/899330240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b740b5c30 0x7f8b740d27f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.352+0000 7f8b78ca1640 1 --2- 172.21.15.19:0/899330240 >> v2:172.21.15.112:3300/0 conn(0x7f8b7415db70 0x7f8b74162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.352+0000 7f8b78ca1640 1 -- 172.21.15.19:0/899330240 >> 172.21.15.19:0/899330240 conn(0x7f8b740d0520 msgr2=0x7f8b740d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.352+0000 7f8b78ca1640 1 -- 172.21.15.19:0/899330240 shutdown_connections 2024-04-25T09:54:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.352+0000 7f8b78ca1640 1 -- 172.21.15.19:0/899330240 wait complete. 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.353+0000 7f8b78ca1640 1 Processor -- start 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.353+0000 7f8b78ca1640 1 -- start start 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.354+0000 7f8b78ca1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b740b5c30 0x7f8b74156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.354+0000 7f8b78ca1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b740b6cb0 0x7f8b741604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.354+0000 7f8b78ca1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b7415db70 0x7f8b74148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.354+0000 7f8b78ca1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b740ce530 con 0x7f8b740b6cb0 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.354+0000 7f8b78ca1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7415df50 con 0x7f8b7415db70 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.354+0000 7f8b78ca1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b740cf1c0 con 0x7f8b740b5c30 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.354+0000 7f8b71d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b740b6cb0 0x7f8b741604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.354+0000 7f8b71d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b740b6cb0 0x7f8b741604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53416/0 (socket says 172.21.15.19:53416) 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.354+0000 7f8b71d9b640 1 -- 172.21.15.19:0/1048230997 learned_addr learned my addr 172.21.15.19:0/1048230997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.354+0000 7f8b72d9d640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b7415db70 0x7f8b74148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.354+0000 7f8b7259c640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b740b5c30 0x7f8b74156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.355+0000 7f8b71d9b640 1 -- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b740b5c30 msgr2=0x7f8b74156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.355+0000 7f8b71d9b640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b740b5c30 0x7f8b74156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.355+0000 7f8b71d9b640 1 -- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b7415db70 msgr2=0x7f8b74148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.355+0000 7f8b71d9b640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b7415db70 0x7f8b74148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.355+0000 7f8b71d9b640 1 -- 172.21.15.19:0/1048230997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b74160a30 con 0x7f8b740b6cb0 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.355+0000 7f8b71d9b640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b740b6cb0 0x7f8b741604f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800c470 tx=0x7f8b68001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.355+0000 7f8b62ffd640 1 -- 172.21.15.19:0/1048230997 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680052d0 con 0x7f8b740b6cb0 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.355+0000 7f8b62ffd640 1 -- 172.21.15.19:0/1048230997 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b68005470 con 0x7f8b740b6cb0 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.355+0000 7f8b78ca1640 1 -- 172.21.15.19:0/1048230997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b741475e0 con 0x7f8b740b6cb0 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.355+0000 7f8b62ffd640 1 -- 172.21.15.19:0/1048230997 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68007260 con 0x7f8b740b6cb0 2024-04-25T09:54:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.355+0000 7f8b78ca1640 1 -- 172.21.15.19:0/1048230997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b74147ba0 con 0x7f8b740b6cb0 2024-04-25T09:54:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.356+0000 7f8b78ca1640 1 -- 172.21.15.19:0/1048230997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b74073990 con 0x7f8b740b6cb0 2024-04-25T09:54:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.357+0000 7f8b62ffd640 1 -- 172.21.15.19:0/1048230997 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b68007400 con 0x7f8b740b6cb0 2024-04-25T09:54:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.357+0000 7f8b62ffd640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b5406b4d0 0x7f8b5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.357+0000 7f8b62ffd640 1 -- 172.21.15.19:0/1048230997 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8b680897f0 con 0x7f8b740b6cb0 2024-04-25T09:54:15.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.357+0000 7f8b7259c640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b5406b4d0 0x7f8b5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.358+0000 7f8b7259c640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b5406b4d0 0x7f8b5406d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8b74160df0 tx=0x7f8b64009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.361+0000 7f8b62ffd640 1 -- 172.21.15.19:0/1048230997 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b68057e90 con 0x7f8b740b6cb0 2024-04-25T09:54:15.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.581+0000 7f8b78ca1640 1 -- 172.21.15.19:0/1048230997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b740d1820 con 0x7f8b740b6cb0 2024-04-25T09:54:15.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.582+0000 7f8b62ffd640 1 -- 172.21.15.19:0/1048230997 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f8b6805bf30 con 0x7f8b740b6cb0 2024-04-25T09:54:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:15.584 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3845932448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3845932448},{"type":"v1","addr":"172.21.15.112:6836","nonce":3845932448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5122,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1655535300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1655535300},{"type":"v1","addr":"172.21.15.19:6834","nonce":1655535300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/165157787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":165157787},{"type":"v1","addr":"172.21.15.112:6837","nonce":165157787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:12.554449+0000","modified":"2024-04-25T09:54:14.571034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3017773710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3017773710},{"type":"v1","addr":"172.21.15.19:6835","nonce":3017773710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:54:15.584 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 16 2024-04-25T09:54:15.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 -- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b5406b4d0 msgr2=0x7f8b5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b5406b4d0 0x7f8b5406d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8b74160df0 tx=0x7f8b64009060 comp rx=0 tx=0).stop 2024-04-25T09:54:15.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 -- 172.21.15.19:0/1048230997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b740b6cb0 msgr2=0x7f8b741604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b740b6cb0 0x7f8b741604f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800c470 tx=0x7f8b68001e70 comp rx=0 tx=0).stop 2024-04-25T09:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 -- 172.21.15.19:0/1048230997 shutdown_connections 2024-04-25T09:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b5406b4d0 0x7f8b5406d990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b7415db70 0x7f8b74148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b740b6cb0 0x7f8b741604f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 --2- 172.21.15.19:0/1048230997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b740b5c30 0x7f8b74156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 -- 172.21.15.19:0/1048230997 >> 172.21.15.19:0/1048230997 conn(0x7f8b740d0520 msgr2=0x7f8b7415c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 -- 172.21.15.19:0/1048230997 shutdown_connections 2024-04-25T09:54:15.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.584+0000 7f8b78ca1640 1 -- 172.21.15.19:0/1048230997 wait complete. 2024-04-25T09:54:15.597 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.695+0000 7fac54f09640 1 Processor -- start 2024-04-25T09:54:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.695+0000 7fac54f09640 1 -- start start 2024-04-25T09:54:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.696+0000 7fac54f09640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac5015ba00 0x7fac5009a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.696+0000 7fac54f09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac501602e0 0x7fac500ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.696+0000 7fac54f09640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac500cd710 con 0x7fac5015f5f0 2024-04-25T09:54:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.696+0000 7fac54f09640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac50099b70 con 0x7fac5015ba00 2024-04-25T09:54:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.696+0000 7fac54f09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac50099e50 con 0x7fac501602e0 2024-04-25T09:54:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.697+0000 7fac4e59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac5015ba00 0x7fac5009a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.697+0000 7fac4dd9b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac501602e0 0x7fac500ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.697+0000 7fac4ed9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fac5015f5f0 0x7fac500c52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:51874/0 (socket says 172.21.15.19:51874) 2024-04-25T09:54:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.697+0000 7fac4ed9d640 1 -- 172.21.15.19:0/3714422337 learned_addr learned my addr 172.21.15.19:0/3714422337 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.697+0000 7fac4e59c640 1 -- 172.21.15.19:0/3714422337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac501602e0 msgr2=0x7fac500ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.697+0000 7fac4e59c640 1 --2- 172.21.15.19:0/3714422337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac501602e0 0x7fac500ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.697+0000 7fac4e59c640 1 -- 172.21.15.19:0/3714422337 >> v1:172.21.15.112:6789/0 conn(0x7fac5015f5f0 legacy=0x7fac500c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.697+0000 7fac4e59c640 1 -- 172.21.15.19:0/3714422337 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5009a130 con 0x7fac5015ba00 2024-04-25T09:54:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.697+0000 7fac4d59a640 1 -- 172.21.15.19:0/3714422337 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720646457 0 0) 0x7fac40001490 con 0x7fac5015f5f0 2024-04-25T09:54:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.697+0000 7fac4dd9b640 1 --2- 172.21.15.19:0/3714422337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac501602e0 0x7fac500ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.698+0000 7fac4e59c640 1 --2- 172.21.15.19:0/3714422337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5015ba00 0x7fac5009a930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac480097a0 tx=0x7fac4802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e144ffdd6cedc66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.698+0000 7fac4d59a640 1 -- 172.21.15.19:0/3714422337 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48037070 con 0x7fac5015ba00 2024-04-25T09:54:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.698+0000 7fac4d59a640 1 -- 172.21.15.19:0/3714422337 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4802adb0 con 0x7fac5015ba00 2024-04-25T09:54:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.698+0000 7fac4d59a640 1 -- 172.21.15.19:0/3714422337 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48032660 con 0x7fac5015ba00 2024-04-25T09:54:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.699+0000 7fac54f09640 1 -- 172.21.15.19:0/3714422337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5015ba00 msgr2=0x7fac5009a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.699+0000 7fac54f09640 1 --2- 172.21.15.19:0/3714422337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5015ba00 0x7fac5009a930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac480097a0 tx=0x7fac4802a060 comp rx=0 tx=0).stop 2024-04-25T09:54:15.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.699+0000 7fac54f09640 1 -- 172.21.15.19:0/3714422337 shutdown_connections 2024-04-25T09:54:15.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.699+0000 7fac54f09640 1 --2- 172.21.15.19:0/3714422337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac501602e0 0x7fac500ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.699+0000 7fac54f09640 1 --2- 172.21.15.19:0/3714422337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5015ba00 0x7fac5009a930 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.699+0000 7fac54f09640 1 -- 172.21.15.19:0/3714422337 >> 172.21.15.19:0/3714422337 conn(0x7fac500b3270 msgr2=0x7fac500b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:15.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.699+0000 7fac54f09640 1 -- 172.21.15.19:0/3714422337 shutdown_connections 2024-04-25T09:54:15.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.699+0000 7fac54f09640 1 -- 172.21.15.19:0/3714422337 wait complete. 2024-04-25T09:54:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.700+0000 7fac54f09640 1 Processor -- start 2024-04-25T09:54:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.700+0000 7fac54f09640 1 -- start start 2024-04-25T09:54:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac54f09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5015ba00 0x7fac50154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac54f09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac5015f5f0 0x7fac500cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac54f09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac501602e0 0x7fac50155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac54f09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5015bde0 con 0x7fac5015ba00 2024-04-25T09:54:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac54f09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac500cd2e0 con 0x7fac501602e0 2024-04-25T09:54:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac54f09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac500b5450 con 0x7fac5015f5f0 2024-04-25T09:54:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac4e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5015ba00 0x7fac50154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac4e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5015ba00 0x7fac50154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53422/0 (socket says 172.21.15.19:53422) 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac4e59c640 1 -- 172.21.15.19:0/2541936360 learned_addr learned my addr 172.21.15.19:0/2541936360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac4ed9d640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac501602e0 0x7fac50155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac4dd9b640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac5015f5f0 0x7fac500cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac4e59c640 1 -- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac5015f5f0 msgr2=0x7fac500cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac4e59c640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac5015f5f0 0x7fac500cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac4e59c640 1 -- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac501602e0 msgr2=0x7fac50155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac4e59c640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac501602e0 0x7fac50155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac4e59c640 1 -- 172.21.15.19:0/2541936360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac500ca320 con 0x7fac5015ba00 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.701+0000 7fac4ed9d640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac501602e0 0x7fac50155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.702+0000 7fac4e59c640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5015ba00 0x7fac50154c50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fac48035040 tx=0x7fac480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.702+0000 7fac46ffd640 1 -- 172.21.15.19:0/2541936360 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48041070 con 0x7fac5015ba00 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.702+0000 7fac46ffd640 1 -- 172.21.15.19:0/2541936360 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4803c460 con 0x7fac5015ba00 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.702+0000 7fac54f09640 1 -- 172.21.15.19:0/2541936360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac500ca600 con 0x7fac5015ba00 2024-04-25T09:54:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.702+0000 7fac46ffd640 1 -- 172.21.15.19:0/2541936360 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48037040 con 0x7fac5015ba00 2024-04-25T09:54:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.702+0000 7fac54f09640 1 -- 172.21.15.19:0/2541936360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac500cabc0 con 0x7fac5015ba00 2024-04-25T09:54:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.703+0000 7fac54f09640 1 -- 172.21.15.19:0/2541936360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac50073990 con 0x7fac5015ba00 2024-04-25T09:54:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.703+0000 7fac46ffd640 1 -- 172.21.15.19:0/2541936360 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac4803a070 con 0x7fac5015ba00 2024-04-25T09:54:15.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.704+0000 7fac46ffd640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fac3006b500 0x7fac3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.704+0000 7fac46ffd640 1 -- 172.21.15.19:0/2541936360 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac480aa490 con 0x7fac5015ba00 2024-04-25T09:54:15.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.704+0000 7fac4dd9b640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fac3006b500 0x7fac3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.704+0000 7fac4dd9b640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fac3006b500 0x7fac3006d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fac38009560 tx=0x7fac38009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:15.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.708+0000 7fac46ffd640 1 -- 172.21.15.19:0/2541936360 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac48078bc0 con 0x7fac5015ba00 2024-04-25T09:54:15.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.888+0000 7fac54f09640 1 -- 172.21.15.19:0/2541936360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac5006f760 con 0x7fac5015ba00 2024-04-25T09:54:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.889+0000 7fac46ffd640 1 -- 172.21.15.19:0/2541936360 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16930 (secure 0 0 0) 0x7fac4807cc60 con 0x7fac5015ba00 2024-04-25T09:54:15.891 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:54:14.198029+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:11.132892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:11.536478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:15.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 -- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fac3006b500 msgr2=0x7fac3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fac3006b500 0x7fac3006d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fac38009560 tx=0x7fac38009190 comp rx=0 tx=0).stop 2024-04-25T09:54:15.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 -- 172.21.15.19:0/2541936360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5015ba00 msgr2=0x7fac50154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5015ba00 0x7fac50154c50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fac48035040 tx=0x7fac480046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 -- 172.21.15.19:0/2541936360 shutdown_connections 2024-04-25T09:54:15.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fac3006b500 0x7fac3006d9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac501602e0 0x7fac50155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac5015f5f0 0x7fac500cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 --2- 172.21.15.19:0/2541936360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac5015ba00 0x7fac50154c50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 -- 172.21.15.19:0/2541936360 >> 172.21.15.19:0/2541936360 conn(0x7fac500b3270 msgr2=0x7fac500cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:15.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 -- 172.21.15.19:0/2541936360 shutdown_connections 2024-04-25T09:54:15.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:15.891+0000 7fac54f09640 1 -- 172.21.15.19:0/2541936360 wait complete. 2024-04-25T09:54:15.905 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:54:15.905 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:54:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.003+0000 7f42213d5640 1 Processor -- start 2024-04-25T09:54:16.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.003+0000 7f42213d5640 1 -- start start 2024-04-25T09:54:16.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.004+0000 7f42213d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f421c0b6cb0 0x7f421c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.004+0000 7f42213d5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421c0ceee0 con 0x7f421c0b5a60 2024-04-25T09:54:16.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.004+0000 7f42213d5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421c0d1d10 con 0x7f421c15db70 2024-04-25T09:54:16.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.004+0000 7f42213d5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421c0d1ff0 con 0x7f421c0b6cb0 2024-04-25T09:54:16.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.004+0000 7f421a7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f421c0b6cb0 0x7f421c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.004+0000 7f421a7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f421c0b6cb0 0x7f421c0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53432/0 (socket says 172.21.15.19:53432) 2024-04-25T09:54:16.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.004+0000 7f421a7fc640 1 -- 172.21.15.19:0/1088163178 learned_addr learned my addr 172.21.15.19:0/1088163178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:16.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.004+0000 7f421a7fc640 1 -- 172.21.15.19:0/1088163178 >> v1:172.21.15.112:6789/0 conn(0x7f421c15db70 legacy=0x7f421c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.004+0000 7f421a7fc640 1 -- 172.21.15.19:0/1088163178 >> v1:172.21.15.19:6789/0 conn(0x7f421c0b5a60 legacy=0x7f421c0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.004+0000 7f421a7fc640 1 -- 172.21.15.19:0/1088163178 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c0d22d0 con 0x7f421c0b6cb0 2024-04-25T09:54:16.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.005+0000 7f421a7fc640 1 --2- 172.21.15.19:0/1088163178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421c0b6cb0 0x7f421c0b5520 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4204009870 tx=0x7f420402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e18b5b49859530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.005+0000 7f4219ffb640 1 -- 172.21.15.19:0/1088163178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204037070 con 0x7f421c0b6cb0 2024-04-25T09:54:16.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.005+0000 7f4219ffb640 1 -- 172.21.15.19:0/1088163178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4204032440 con 0x7f421c0b6cb0 2024-04-25T09:54:16.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.005+0000 7f4219ffb640 1 -- 172.21.15.19:0/1088163178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204032810 con 0x7f421c0b6cb0 2024-04-25T09:54:16.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.006+0000 7f42213d5640 1 -- 172.21.15.19:0/1088163178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421c0b6cb0 msgr2=0x7f421c0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.006+0000 7f42213d5640 1 --2- 172.21.15.19:0/1088163178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421c0b6cb0 0x7f421c0b5520 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4204009870 tx=0x7f420402a270 comp rx=0 tx=0).stop 2024-04-25T09:54:16.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.006+0000 7f42213d5640 1 -- 172.21.15.19:0/1088163178 shutdown_connections 2024-04-25T09:54:16.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.006+0000 7f42213d5640 1 --2- 172.21.15.19:0/1088163178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421c0b6cb0 0x7f421c0b5520 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.006+0000 7f42213d5640 1 -- 172.21.15.19:0/1088163178 >> 172.21.15.19:0/1088163178 conn(0x7f421c0d0520 msgr2=0x7f421c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.006+0000 7f42213d5640 1 -- 172.21.15.19:0/1088163178 shutdown_connections 2024-04-25T09:54:16.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.006+0000 7f42213d5640 1 -- 172.21.15.19:0/1088163178 wait complete. 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.007+0000 7f42213d5640 1 Processor -- start 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.007+0000 7f42213d5640 1 -- start start 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f42213d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f421c0b5a60 0x7f421c149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f42213d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0b6cb0 0x7f421c1e9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f42213d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421c15db70 0x7f421c1f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f42213d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0cf1c0 con 0x7f421c15db70 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f42213d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0ce530 con 0x7f421c0b5a60 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f42213d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0cf340 con 0x7f421c0b6cb0 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f421b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421c15db70 0x7f421c1f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f421b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421c15db70 0x7f421c1f7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53434/0 (socket says 172.21.15.19:53434) 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f421b7fe640 1 -- 172.21.15.19:0/434329973 learned_addr learned my addr 172.21.15.19:0/434329973 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:16.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f421affd640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f421c0b5a60 0x7f421c149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f421a7fc640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0b6cb0 0x7f421c1e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f421b7fe640 1 -- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0b6cb0 msgr2=0x7f421c1e9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f421b7fe640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0b6cb0 0x7f421c1e9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f421b7fe640 1 -- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f421c0b5a60 msgr2=0x7f421c149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f421b7fe640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f421c0b5a60 0x7f421c149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f421b7fe640 1 -- 172.21.15.19:0/434329973 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c1e90a0 con 0x7f421c15db70 2024-04-25T09:54:16.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.008+0000 7f421a7fc640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0b6cb0 0x7f421c1e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:16.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.009+0000 7f421b7fe640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421c15db70 0x7f421c1f7360 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c7a0 tx=0x7f420c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.009+0000 7f42137fe640 1 -- 172.21.15.19:0/434329973 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c003b70 con 0x7f421c15db70 2024-04-25T09:54:16.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.009+0000 7f42137fe640 1 -- 172.21.15.19:0/434329973 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c003d30 con 0x7f421c15db70 2024-04-25T09:54:16.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.009+0000 7f42137fe640 1 -- 172.21.15.19:0/434329973 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c00aa90 con 0x7f421c15db70 2024-04-25T09:54:16.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.009+0000 7f42213d5640 1 -- 172.21.15.19:0/434329973 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421c0c7230 con 0x7f421c15db70 2024-04-25T09:54:16.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.009+0000 7f42213d5640 1 -- 172.21.15.19:0/434329973 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421c14a370 con 0x7f421c15db70 2024-04-25T09:54:16.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.010+0000 7f42213d5640 1 -- 172.21.15.19:0/434329973 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421c073990 con 0x7f421c15db70 2024-04-25T09:54:16.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.010+0000 7f42137fe640 1 -- 172.21.15.19:0/434329973 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f420c00ac30 con 0x7f421c15db70 2024-04-25T09:54:16.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.011+0000 7f42137fe640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41fc06b4d0 0x7f41fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.011+0000 7f42137fe640 1 -- 172.21.15.19:0/434329973 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f420c08a2e0 con 0x7f421c15db70 2024-04-25T09:54:16.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.011+0000 7f421affd640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41fc06b4d0 0x7f41fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.011+0000 7f421affd640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41fc06b4d0 0x7f41fc06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4214007fd0 tx=0x7f4214009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.015+0000 7f42137fe640 1 -- 172.21.15.19:0/434329973 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f420c058fd0 con 0x7f421c15db70 2024-04-25T09:54:16.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.192+0000 7f42213d5640 1 -- 172.21.15.19:0/434329973 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f421c0d1820 con 0x7f41fc06b4d0 2024-04-25T09:54:16.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.199+0000 7f42137fe640 1 -- 172.21.15.19:0/434329973 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251890 (secure 0 0 0) 0x7f421c0d1820 con 0x7f41fc06b4d0 2024-04-25T09:54:16.204 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:16.206 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:54:16.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 -- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41fc06b4d0 msgr2=0x7f41fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41fc06b4d0 0x7f41fc06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4214007fd0 tx=0x7f4214009040 comp rx=0 tx=0).stop 2024-04-25T09:54:16.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 -- 172.21.15.19:0/434329973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421c15db70 msgr2=0x7f421c1f7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421c15db70 0x7f421c1f7360 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c7a0 tx=0x7f420c00cc60 comp rx=0 tx=0).stop 2024-04-25T09:54:16.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 -- 172.21.15.19:0/434329973 shutdown_connections 2024-04-25T09:54:16.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f41fc06b4d0 0x7f41fc06d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421c15db70 0x7f421c1f7360 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0b6cb0 0x7f421c1e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 --2- 172.21.15.19:0/434329973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f421c0b5a60 0x7f421c149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 -- 172.21.15.19:0/434329973 >> 172.21.15.19:0/434329973 conn(0x7f421c0d0520 msgr2=0x7f421c0b90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 -- 172.21.15.19:0/434329973 shutdown_connections 2024-04-25T09:54:16.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.202+0000 7f42213d5640 1 -- 172.21.15.19:0/434329973 wait complete. 2024-04-25T09:54:16.215 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-25T09:54:14.926947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220668,"kb_used_data":4460,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754052,"statfs":{"total":773094113280,"available":772868149248,"internally_reserved":0,"allocated":4567040,"data_stored":2620002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000703"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.234799+0000","last_change":"2024-04-25T09:54:13.229307+0000","last_active":"2024-04-25T09:54:13.234799+0000","last_peered":"2024-04-25T09:54:13.234799+0000","last_clean":"2024-04-25T09:54:13.234799+0000","last_became_active":"2024-04-25T09:54:13.229016+0000","last_became_peered":"2024-04-25T09:54:13.229016+0000","last_unstale":"2024-04-25T09:54:13.234799+0000","last_undegraded":"2024-04-25T09:54:13.234799+0000","last_fullsized":"2024-04-25T09:54:13.234799+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:39.803246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.227765+0000","last_change":"2024-04-25T09:54:13.221152+0000","last_active":"2024-04-25T09:54:13.227765+0000","last_peered":"2024-04-25T09:54:13.227765+0000","last_clean":"2024-04-25T09:54:13.227765+0000","last_became_active":"2024-04-25T09:54:13.220728+0000","last_became_peered":"2024-04-25T09:54:13.220728+0000","last_unstale":"2024-04-25T09:54:13.227765+0000","last_undegraded":"2024-04-25T09:54:13.227765+0000","last_fullsized":"2024-04-25T09:54:13.227765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:58.849128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223808+0000","last_change":"2024-04-25T09:54:13.216689+0000","last_active":"2024-04-25T09:54:13.223808+0000","last_peered":"2024-04-25T09:54:13.223808+0000","last_clean":"2024-04-25T09:54:13.223808+0000","last_became_active":"2024-04-25T09:54:13.216398+0000","last_became_peered":"2024-04-25T09:54:13.216398+0000","last_unstale":"2024-04-25T09:54:13.223808+0000","last_undegraded":"2024-04-25T09:54:13.223808+0000","last_fullsized":"2024-04-25T09:54:13.223808+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:55.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.225010+0000","last_change":"2024-04-25T09:54:13.220310+0000","last_active":"2024-04-25T09:54:13.225010+0000","last_peered":"2024-04-25T09:54:13.225010+0000","last_clean":"2024-04-25T09:54:13.225010+0000","last_became_active":"2024-04-25T09:54:13.219790+0000","last_became_peered":"2024-04-25T09:54:13.219790+0000","last_unstale":"2024-04-25T09:54:13.225010+0000","last_undegraded":"2024-04-25T09:54:13.225010+0000","last_fullsized":"2024-04-25T09:54:13.225010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:24.225676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226992+0000","last_change":"2024-04-25T09:54:13.219979+0000","last_active":"2024-04-25T09:54:13.226992+0000","last_peered":"2024-04-25T09:54:13.226992+0000","last_clean":"2024-04-25T09:54:13.226992+0000","last_became_active":"2024-04-25T09:54:13.219024+0000","last_became_peered":"2024-04-25T09:54:13.219024+0000","last_unstale":"2024-04-25T09:54:13.226992+0000","last_undegraded":"2024-04-25T09:54:13.226992+0000","last_fullsized":"2024-04-25T09:54:13.226992+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:02.274809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.235906+0000","last_change":"2024-04-25T09:54:13.232607+0000","last_active":"2024-04-25T09:54:13.235906+0000","last_peered":"2024-04-25T09:54:13.235906+0000","last_clean":"2024-04-25T09:54:13.235906+0000","last_became_active":"2024-04-25T09:54:13.232323+0000","last_became_peered":"2024-04-25T09:54:13.232323+0000","last_unstale":"2024-04-25T09:54:13.235906+0000","last_undegraded":"2024-04-25T09:54:13.235906+0000","last_fullsized":"2024-04-25T09:54:13.235906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:42.183276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223321+0000","last_change":"2024-04-25T09:54:13.220700+0000","last_active":"2024-04-25T09:54:13.223321+0000","last_peered":"2024-04-25T09:54:13.223321+0000","last_clean":"2024-04-25T09:54:13.223321+0000","last_became_active":"2024-04-25T09:54:13.220409+0000","last_became_peered":"2024-04-25T09:54:13.220409+0000","last_unstale":"2024-04-25T09:54:13.223321+0000","last_undegraded":"2024-04-25T09:54:13.223321+0000","last_fullsized":"2024-04-25T09:54:13.223321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:54.136097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226912+0000","last_change":"2024-04-25T09:54:13.222091+0000","last_active":"2024-04-25T09:54:13.226912+0000","last_peered":"2024-04-25T09:54:13.226912+0000","last_clean":"2024-04-25T09:54:13.226912+0000","last_became_active":"2024-04-25T09:54:13.221799+0000","last_became_peered":"2024-04-25T09:54:13.221799+0000","last_unstale":"2024-04-25T09:54:13.226912+0000","last_undegraded":"2024-04-25T09:54:13.226912+0000","last_fullsized":"2024-04-25T09:54:13.226912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:24.575994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223879+0000","last_change":"2024-04-25T09:54:13.216799+0000","last_active":"2024-04-25T09:54:13.223879+0000","last_peered":"2024-04-25T09:54:13.223879+0000","last_clean":"2024-04-25T09:54:13.223879+0000","last_became_active":"2024-04-25T09:54:13.216509+0000","last_became_peered":"2024-04-25T09:54:13.216509+0000","last_unstale":"2024-04-25T09:54:13.223879+0000","last_undegraded":"2024-04-25T09:54:13.223879+0000","last_fullsized":"2024-04-25T09:54:13.223879+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:42.183276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.210135+0000","last_change":"2024-04-25T09:54:13.208370+0000","last_active":"2024-04-25T09:54:13.210135+0000","last_peered":"2024-04-25T09:54:13.210135+0000","last_clean":"2024-04-25T09:54:13.210135+0000","last_became_active":"2024-04-25T09:54:13.207932+0000","last_became_peered":"2024-04-25T09:54:13.207932+0000","last_unstale":"2024-04-25T09:54:13.210135+0000","last_undegraded":"2024-04-25T09:54:13.210135+0000","last_fullsized":"2024-04-25T09:54:13.210135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:38.621459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223191+0000","last_change":"2024-04-25T09:54:13.218780+0000","last_active":"2024-04-25T09:54:13.223191+0000","last_peered":"2024-04-25T09:54:13.223191+0000","last_clean":"2024-04-25T09:54:13.223191+0000","last_became_active":"2024-04-25T09:54:13.218521+0000","last_became_peered":"2024-04-25T09:54:13.218521+0000","last_unstale":"2024-04-25T09:54:13.223191+0000","last_undegraded":"2024-04-25T09:54:13.223191+0000","last_fullsized":"2024-04-25T09:54:13.223191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:10.890138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223762+0000","last_change":"2024-04-25T09:54:13.216188+0000","last_active":"2024-04-25T09:54:13.223762+0000","last_peered":"2024-04-25T09:54:13.223762+0000","last_clean":"2024-04-25T09:54:13.223762+0000","last_became_active":"2024-04-25T09:54:13.215871+0000","last_became_peered":"2024-04-25T09:54:13.215871+0000","last_unstale":"2024-04-25T09:54:13.223762+0000","last_undegraded":"2024-04-25T09:54:13.223762+0000","last_fullsized":"2024-04-25T09:54:13.223762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.261566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.209604+0000","last_change":"2024-04-25T09:54:13.206445+0000","last_active":"2024-04-25T09:54:13.209604+0000","last_peered":"2024-04-25T09:54:13.209604+0000","last_clean":"2024-04-25T09:54:13.209604+0000","last_became_active":"2024-04-25T09:54:13.206159+0000","last_became_peered":"2024-04-25T09:54:13.206159+0000","last_unstale":"2024-04-25T09:54:13.209604+0000","last_undegraded":"2024-04-25T09:54:13.209604+0000","last_fullsized":"2024-04-25T09:54:13.209604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:53.335804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.220007+0000","last_change":"2024-04-25T09:54:13.217370+0000","last_active":"2024-04-25T09:54:13.220007+0000","last_peered":"2024-04-25T09:54:13.220007+0000","last_clean":"2024-04-25T09:54:13.220007+0000","last_became_active":"2024-04-25T09:54:13.217079+0000","last_became_peered":"2024-04-25T09:54:13.217079+0000","last_unstale":"2024-04-25T09:54:13.220007+0000","last_undegraded":"2024-04-25T09:54:13.220007+0000","last_fullsized":"2024-04-25T09:54:13.220007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:55.531727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.216798+0000","last_change":"2024-04-25T09:54:13.213430+0000","last_active":"2024-04-25T09:54:13.216798+0000","last_peered":"2024-04-25T09:54:13.216798+0000","last_clean":"2024-04-25T09:54:13.216798+0000","last_became_active":"2024-04-25T09:54:13.213160+0000","last_became_peered":"2024-04-25T09:54:13.213160+0000","last_unstale":"2024-04-25T09:54:13.216798+0000","last_undegraded":"2024-04-25T09:54:13.216798+0000","last_fullsized":"2024-04-25T09:54:13.216798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:58.849128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.207110+0000","last_change":"2024-04-25T09:54:13.206281+0000","last_active":"2024-04-25T09:54:13.207110+0000","last_peered":"2024-04-25T09:54:13.207110+0000","last_clean":"2024-04-25T09:54:13.207110+0000","last_became_active":"2024-04-25T09:54:13.205429+0000","last_became_peered":"2024-04-25T09:54:13.205429+0000","last_unstale":"2024-04-25T09:54:13.207110+0000","last_undegraded":"2024-04-25T09:54:13.207110+0000","last_fullsized":"2024-04-25T09:54:13.207110+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.391093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.215518+0000","last_change":"2024-04-25T09:54:13.206949+0000","last_active":"2024-04-25T09:54:13.215518+0000","last_peered":"2024-04-25T09:54:13.215518+0000","last_clean":"2024-04-25T09:54:13.215518+0000","last_became_active":"2024-04-25T09:54:13.206695+0000","last_became_peered":"2024-04-25T09:54:13.206695+0000","last_unstale":"2024-04-25T09:54:13.215518+0000","last_undegraded":"2024-04-25T09:54:13.215518+0000","last_fullsized":"2024-04-25T09:54:13.215518+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.261566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223155+0000","last_change":"2024-04-25T09:54:13.220536+0000","last_active":"2024-04-25T09:54:13.223155+0000","last_peered":"2024-04-25T09:54:13.223155+0000","last_clean":"2024-04-25T09:54:13.223155+0000","last_became_active":"2024-04-25T09:54:13.219875+0000","last_became_peered":"2024-04-25T09:54:13.219875+0000","last_unstale":"2024-04-25T09:54:13.223155+0000","last_undegraded":"2024-04-25T09:54:13.223155+0000","last_fullsized":"2024-04-25T09:54:13.223155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.330344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223811+0000","last_change":"2024-04-25T09:54:13.216202+0000","last_active":"2024-04-25T09:54:13.223811+0000","last_peered":"2024-04-25T09:54:13.223811+0000","last_clean":"2024-04-25T09:54:13.223811+0000","last_became_active":"2024-04-25T09:54:13.215944+0000","last_became_peered":"2024-04-25T09:54:13.215944+0000","last_unstale":"2024-04-25T09:54:13.223811+0000","last_undegraded":"2024-04-25T09:54:13.223811+0000","last_fullsized":"2024-04-25T09:54:13.223811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:52.807108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.220690+0000","last_change":"2024-04-25T09:54:13.218825+0000","last_active":"2024-04-25T09:54:13.220690+0000","last_peered":"2024-04-25T09:54:13.220690+0000","last_clean":"2024-04-25T09:54:13.220690+0000","last_became_active":"2024-04-25T09:54:13.218530+0000","last_became_peered":"2024-04-25T09:54:13.218530+0000","last_unstale":"2024-04-25T09:54:13.220690+0000","last_undegraded":"2024-04-25T09:54:13.220690+0000","last_fullsized":"2024-04-25T09:54:13.220690+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:39.005592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223162+0000","last_change":"2024-04-25T09:54:13.217966+0000","last_active":"2024-04-25T09:54:13.223162+0000","last_peered":"2024-04-25T09:54:13.223162+0000","last_clean":"2024-04-25T09:54:13.223162+0000","last_became_active":"2024-04-25T09:54:13.217668+0000","last_became_peered":"2024-04-25T09:54:13.217668+0000","last_unstale":"2024-04-25T09:54:13.223162+0000","last_undegraded":"2024-04-25T09:54:13.223162+0000","last_fullsized":"2024-04-25T09:54:13.223162+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:24.575994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.228599+0000","last_change":"2024-04-25T09:54:13.221967+0000","last_active":"2024-04-25T09:54:13.228599+0000","last_peered":"2024-04-25T09:54:13.228599+0000","last_clean":"2024-04-25T09:54:13.228599+0000","last_became_active":"2024-04-25T09:54:13.221640+0000","last_became_peered":"2024-04-25T09:54:13.221640+0000","last_unstale":"2024-04-25T09:54:13.228599+0000","last_undegraded":"2024-04-25T09:54:13.228599+0000","last_fullsized":"2024-04-25T09:54:13.228599+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:07.399889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.218082+0000","last_change":"2024-04-25T09:54:13.214764+0000","last_active":"2024-04-25T09:54:13.218082+0000","last_peered":"2024-04-25T09:54:13.218082+0000","last_clean":"2024-04-25T09:54:13.218082+0000","last_became_active":"2024-04-25T09:54:13.214493+0000","last_became_peered":"2024-04-25T09:54:13.214493+0000","last_unstale":"2024-04-25T09:54:13.218082+0000","last_undegraded":"2024-04-25T09:54:13.218082+0000","last_fullsized":"2024-04-25T09:54:13.218082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:57.999037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.219829+0000","last_change":"2024-04-25T09:54:13.211683+0000","last_active":"2024-04-25T09:54:13.219829+0000","last_peered":"2024-04-25T09:54:13.219829+0000","last_clean":"2024-04-25T09:54:13.219829+0000","last_became_active":"2024-04-25T09:54:13.211372+0000","last_became_peered":"2024-04-25T09:54:13.211372+0000","last_unstale":"2024-04-25T09:54:13.219829+0000","last_undegraded":"2024-04-25T09:54:13.219829+0000","last_fullsized":"2024-04-25T09:54:13.219829+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:10.116155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.232802+0000","last_change":"2024-04-25T09:54:13.229223+0000","last_active":"2024-04-25T09:54:13.232802+0000","last_peered":"2024-04-25T09:54:13.232802+0000","last_clean":"2024-04-25T09:54:13.232802+0000","last_became_active":"2024-04-25T09:54:13.228891+0000","last_became_peered":"2024-04-25T09:54:13.228891+0000","last_unstale":"2024-04-25T09:54:13.232802+0000","last_undegraded":"2024-04-25T09:54:13.232802+0000","last_fullsized":"2024-04-25T09:54:13.232802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:12.287753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.231170+0000","last_change":"2024-04-25T09:54:13.223366+0000","last_active":"2024-04-25T09:54:13.231170+0000","last_peered":"2024-04-25T09:54:13.231170+0000","last_clean":"2024-04-25T09:54:13.231170+0000","last_became_active":"2024-04-25T09:54:13.223060+0000","last_became_peered":"2024-04-25T09:54:13.223060+0000","last_unstale":"2024-04-25T09:54:13.231170+0000","last_undegraded":"2024-04-25T09:54:13.231170+0000","last_fullsized":"2024-04-25T09:54:13.231170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.261566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226584+0000","last_change":"2024-04-25T09:54:13.223911+0000","last_active":"2024-04-25T09:54:13.226584+0000","last_peered":"2024-04-25T09:54:13.226584+0000","last_clean":"2024-04-25T09:54:13.226584+0000","last_became_active":"2024-04-25T09:54:13.223675+0000","last_became_peered":"2024-04-25T09:54:13.223675+0000","last_unstale":"2024-04-25T09:54:13.226584+0000","last_undegraded":"2024-04-25T09:54:13.226584+0000","last_fullsized":"2024-04-25T09:54:13.226584+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:55.531727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.229609+0000","last_change":"2024-04-25T09:54:13.223024+0000","last_active":"2024-04-25T09:54:13.229609+0000","last_peered":"2024-04-25T09:54:13.229609+0000","last_clean":"2024-04-25T09:54:13.229609+0000","last_became_active":"2024-04-25T09:54:13.222553+0000","last_became_peered":"2024-04-25T09:54:13.222553+0000","last_unstale":"2024-04-25T09:54:13.229609+0000","last_undegraded":"2024-04-25T09:54:13.229609+0000","last_fullsized":"2024-04-25T09:54:13.229609+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:39.803246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.220879+0000","last_change":"2024-04-25T09:54:13.217777+0000","last_active":"2024-04-25T09:54:13.220879+0000","last_peered":"2024-04-25T09:54:13.220879+0000","last_clean":"2024-04-25T09:54:13.220879+0000","last_became_active":"2024-04-25T09:54:13.217510+0000","last_became_peered":"2024-04-25T09:54:13.217510+0000","last_unstale":"2024-04-25T09:54:13.220879+0000","last_undegraded":"2024-04-25T09:54:13.220879+0000","last_fullsized":"2024-04-25T09:54:13.220879+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:06.913173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.229116+0000","last_change":"2024-04-25T09:54:13.227226+0000","last_active":"2024-04-25T09:54:13.229116+0000","last_peered":"2024-04-25T09:54:13.229116+0000","last_clean":"2024-04-25T09:54:13.229116+0000","last_became_active":"2024-04-25T09:54:13.226947+0000","last_became_peered":"2024-04-25T09:54:13.226947+0000","last_unstale":"2024-04-25T09:54:13.229116+0000","last_undegraded":"2024-04-25T09:54:13.229116+0000","last_fullsized":"2024-04-25T09:54:13.229116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:26.469445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.218106+0000","last_change":"2024-04-25T09:54:13.215642+0000","last_active":"2024-04-25T09:54:13.218106+0000","last_peered":"2024-04-25T09:54:13.218106+0000","last_clean":"2024-04-25T09:54:13.218106+0000","last_became_active":"2024-04-25T09:54:13.215026+0000","last_became_peered":"2024-04-25T09:54:13.215026+0000","last_unstale":"2024-04-25T09:54:13.218106+0000","last_undegraded":"2024-04-25T09:54:13.218106+0000","last_fullsized":"2024-04-25T09:54:13.218106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:53.335804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226105+0000","last_change":"2024-04-25T09:54:13.221098+0000","last_active":"2024-04-25T09:54:13.226105+0000","last_peered":"2024-04-25T09:54:13.226105+0000","last_clean":"2024-04-25T09:54:13.226105+0000","last_became_active":"2024-04-25T09:54:13.220588+0000","last_became_peered":"2024-04-25T09:54:13.220588+0000","last_unstale":"2024-04-25T09:54:13.226105+0000","last_undegraded":"2024-04-25T09:54:13.226105+0000","last_fullsized":"2024-04-25T09:54:13.226105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.391093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.215860+0000","last_change":"2024-04-25T09:54:13.207629+0000","last_active":"2024-04-25T09:54:13.215860+0000","last_peered":"2024-04-25T09:54:13.215860+0000","last_clean":"2024-04-25T09:54:13.215860+0000","last_became_active":"2024-04-25T09:54:13.207360+0000","last_became_peered":"2024-04-25T09:54:13.207360+0000","last_unstale":"2024-04-25T09:54:13.215860+0000","last_undegraded":"2024-04-25T09:54:13.215860+0000","last_fullsized":"2024-04-25T09:54:13.215860+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:16.483875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.232933+0000","last_change":"2024-04-25T09:54:12.203395+0000","last_active":"2024-04-25T09:54:13.232933+0000","last_peered":"2024-04-25T09:54:13.232933+0000","last_clean":"2024-04-25T09:54:13.232933+0000","last_became_active":"2024-04-25T09:54:12.203097+0000","last_became_peered":"2024-04-25T09:54:12.203097+0000","last_unstale":"2024-04-25T09:54:13.232933+0000","last_undegraded":"2024-04-25T09:54:13.232933+0000","last_fullsized":"2024-04-25T09:54:13.232933+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:51.847718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.227833+0000","last_change":"2024-04-25T09:54:13.225510+0000","last_active":"2024-04-25T09:54:13.227833+0000","last_peered":"2024-04-25T09:54:13.227833+0000","last_clean":"2024-04-25T09:54:13.227833+0000","last_became_active":"2024-04-25T09:54:13.224996+0000","last_became_peered":"2024-04-25T09:54:13.224996+0000","last_unstale":"2024-04-25T09:54:13.227833+0000","last_undegraded":"2024-04-25T09:54:13.227833+0000","last_fullsized":"2024-04-25T09:54:13.227833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:55.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.235178+0000","last_change":"2024-04-25T09:54:12.208454+0000","last_active":"2024-04-25T09:54:13.235178+0000","last_peered":"2024-04-25T09:54:13.235178+0000","last_clean":"2024-04-25T09:54:13.235178+0000","last_became_active":"2024-04-25T09:54:12.208140+0000","last_became_peered":"2024-04-25T09:54:12.208140+0000","last_unstale":"2024-04-25T09:54:13.235178+0000","last_undegraded":"2024-04-25T09:54:13.235178+0000","last_fullsized":"2024-04-25T09:54:13.235178+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:06.383902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.219786+0000","last_change":"2024-04-25T09:54:13.217575+0000","last_active":"2024-04-25T09:54:13.219786+0000","last_peered":"2024-04-25T09:54:13.219786+0000","last_clean":"2024-04-25T09:54:13.219786+0000","last_became_active":"2024-04-25T09:54:13.217295+0000","last_became_peered":"2024-04-25T09:54:13.217295+0000","last_unstale":"2024-04-25T09:54:13.219786+0000","last_undegraded":"2024-04-25T09:54:13.219786+0000","last_fullsized":"2024-04-25T09:54:13.219786+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:26.469445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.235977+0000","last_change":"2024-04-25T09:54:12.208115+0000","last_active":"2024-04-25T09:54:13.235977+0000","last_peered":"2024-04-25T09:54:13.235977+0000","last_clean":"2024-04-25T09:54:13.235977+0000","last_became_active":"2024-04-25T09:54:12.207857+0000","last_became_peered":"2024-04-25T09:54:12.207857+0000","last_unstale":"2024-04-25T09:54:13.235977+0000","last_undegraded":"2024-04-25T09:54:13.235977+0000","last_fullsized":"2024-04-25T09:54:13.235977+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:11.280562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.230576+0000","last_change":"2024-04-25T09:54:13.227489+0000","last_active":"2024-04-25T09:54:13.230576+0000","last_peered":"2024-04-25T09:54:13.230576+0000","last_clean":"2024-04-25T09:54:13.230576+0000","last_became_active":"2024-04-25T09:54:13.227179+0000","last_became_peered":"2024-04-25T09:54:13.227179+0000","last_unstale":"2024-04-25T09:54:13.230576+0000","last_undegraded":"2024-04-25T09:54:13.230576+0000","last_fullsized":"2024-04-25T09:54:13.230576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:09.907450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.235279+0000","last_change":"2024-04-25T09:54:12.210031+0000","last_active":"2024-04-25T09:54:13.235279+0000","last_peered":"2024-04-25T09:54:13.235279+0000","last_clean":"2024-04-25T09:54:13.235279+0000","last_became_active":"2024-04-25T09:54:12.209741+0000","last_became_peered":"2024-04-25T09:54:12.209741+0000","last_unstale":"2024-04-25T09:54:13.235279+0000","last_undegraded":"2024-04-25T09:54:13.235279+0000","last_fullsized":"2024-04-25T09:54:13.235279+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:42.323153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226848+0000","last_change":"2024-04-25T09:54:13.218400+0000","last_active":"2024-04-25T09:54:13.226848+0000","last_peered":"2024-04-25T09:54:13.226848+0000","last_clean":"2024-04-25T09:54:13.226848+0000","last_became_active":"2024-04-25T09:54:13.218109+0000","last_became_peered":"2024-04-25T09:54:13.218109+0000","last_unstale":"2024-04-25T09:54:13.226848+0000","last_undegraded":"2024-04-25T09:54:13.226848+0000","last_fullsized":"2024-04-25T09:54:13.226848+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:55.404316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.234029+0000","last_change":"2024-04-25T09:54:12.209211+0000","last_active":"2024-04-25T09:54:13.234029+0000","last_peered":"2024-04-25T09:54:13.234029+0000","last_clean":"2024-04-25T09:54:13.234029+0000","last_became_active":"2024-04-25T09:54:12.208917+0000","last_became_peered":"2024-04-25T09:54:12.208917+0000","last_unstale":"2024-04-25T09:54:13.234029+0000","last_undegraded":"2024-04-25T09:54:13.234029+0000","last_fullsized":"2024-04-25T09:54:13.234029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:54.397125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223346+0000","last_change":"2024-04-25T09:54:13.221489+0000","last_active":"2024-04-25T09:54:13.223346+0000","last_peered":"2024-04-25T09:54:13.223346+0000","last_clean":"2024-04-25T09:54:13.223346+0000","last_became_active":"2024-04-25T09:54:13.221175+0000","last_became_peered":"2024-04-25T09:54:13.221175+0000","last_unstale":"2024-04-25T09:54:13.223346+0000","last_undegraded":"2024-04-25T09:54:13.223346+0000","last_fullsized":"2024-04-25T09:54:13.223346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:09.907450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.230716+0000","last_change":"2024-04-25T09:54:12.201478+0000","last_active":"2024-04-25T09:54:13.230716+0000","last_peered":"2024-04-25T09:54:13.230716+0000","last_clean":"2024-04-25T09:54:13.230716+0000","last_became_active":"2024-04-25T09:54:12.201198+0000","last_became_peered":"2024-04-25T09:54:12.201198+0000","last_unstale":"2024-04-25T09:54:13.230716+0000","last_undegraded":"2024-04-25T09:54:13.230716+0000","last_fullsized":"2024-04-25T09:54:13.230716+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:11.280562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.222983+0000","last_change":"2024-04-25T09:54:13.218920+0000","last_active":"2024-04-25T09:54:13.222983+0000","last_peered":"2024-04-25T09:54:13.222983+0000","last_clean":"2024-04-25T09:54:13.222983+0000","last_became_active":"2024-04-25T09:54:13.218632+0000","last_became_peered":"2024-04-25T09:54:13.218632+0000","last_unstale":"2024-04-25T09:54:13.222983+0000","last_undegraded":"2024-04-25T09:54:13.222983+0000","last_fullsized":"2024-04-25T09:54:13.222983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:12.287753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.210011+0000","last_change":"2024-04-25T09:54:12.213721+0000","last_active":"2024-04-25T09:54:13.210011+0000","last_peered":"2024-04-25T09:54:13.210011+0000","last_clean":"2024-04-25T09:54:13.210011+0000","last_became_active":"2024-04-25T09:54:12.213430+0000","last_became_peered":"2024-04-25T09:54:12.213430+0000","last_unstale":"2024-04-25T09:54:13.210011+0000","last_undegraded":"2024-04-25T09:54:13.210011+0000","last_fullsized":"2024-04-25T09:54:13.210011+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:54.524536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.227709+0000","last_change":"2024-04-25T09:54:13.225021+0000","last_active":"2024-04-25T09:54:13.227709+0000","last_peered":"2024-04-25T09:54:13.227709+0000","last_clean":"2024-04-25T09:54:13.227709+0000","last_became_active":"2024-04-25T09:54:13.223986+0000","last_became_peered":"2024-04-25T09:54:13.223986+0000","last_unstale":"2024-04-25T09:54:13.227709+0000","last_undegraded":"2024-04-25T09:54:13.227709+0000","last_fullsized":"2024-04-25T09:54:13.227709+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:34.596310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.208585+0000","last_change":"2024-04-25T09:54:12.205232+0000","last_active":"2024-04-25T09:54:13.208585+0000","last_peered":"2024-04-25T09:54:13.208585+0000","last_clean":"2024-04-25T09:54:13.208585+0000","last_became_active":"2024-04-25T09:54:12.204946+0000","last_became_peered":"2024-04-25T09:54:12.204946+0000","last_unstale":"2024-04-25T09:54:13.208585+0000","last_undegraded":"2024-04-25T09:54:13.208585+0000","last_fullsized":"2024-04-25T09:54:13.208585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:51.847718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.224835+0000","last_change":"2024-04-25T09:54:13.220082+0000","last_active":"2024-04-25T09:54:13.224835+0000","last_peered":"2024-04-25T09:54:13.224835+0000","last_clean":"2024-04-25T09:54:13.224835+0000","last_became_active":"2024-04-25T09:54:13.219433+0000","last_became_peered":"2024-04-25T09:54:13.219433+0000","last_unstale":"2024-04-25T09:54:13.224835+0000","last_undegraded":"2024-04-25T09:54:13.224835+0000","last_fullsized":"2024-04-25T09:54:13.224835+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:41.458847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.209197+0000","last_change":"2024-04-25T09:54:12.212615+0000","last_active":"2024-04-25T09:54:13.209197+0000","last_peered":"2024-04-25T09:54:13.209197+0000","last_clean":"2024-04-25T09:54:13.209197+0000","last_became_active":"2024-04-25T09:54:12.212246+0000","last_became_peered":"2024-04-25T09:54:12.212246+0000","last_unstale":"2024-04-25T09:54:13.209197+0000","last_undegraded":"2024-04-25T09:54:13.209197+0000","last_fullsized":"2024-04-25T09:54:13.209197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:56.991847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.218157+0000","last_change":"2024-04-25T09:54:13.215665+0000","last_active":"2024-04-25T09:54:13.218157+0000","last_peered":"2024-04-25T09:54:13.218157+0000","last_clean":"2024-04-25T09:54:13.218157+0000","last_became_active":"2024-04-25T09:54:13.215404+0000","last_became_peered":"2024-04-25T09:54:13.215404+0000","last_unstale":"2024-04-25T09:54:13.218157+0000","last_undegraded":"2024-04-25T09:54:13.218157+0000","last_fullsized":"2024-04-25T09:54:13.218157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:38.621459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.204111+0000","last_change":"2024-04-25T09:54:12.206943+0000","last_active":"2024-04-25T09:54:13.204111+0000","last_peered":"2024-04-25T09:54:13.204111+0000","last_clean":"2024-04-25T09:54:13.204111+0000","last_became_active":"2024-04-25T09:54:12.206649+0000","last_became_peered":"2024-04-25T09:54:12.206649+0000","last_unstale":"2024-04-25T09:54:13.204111+0000","last_undegraded":"2024-04-25T09:54:13.204111+0000","last_fullsized":"2024-04-25T09:54:13.204111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:07.751815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.224558+0000","last_change":"2024-04-25T09:54:13.219359+0000","last_active":"2024-04-25T09:54:13.224558+0000","last_peered":"2024-04-25T09:54:13.224558+0000","last_clean":"2024-04-25T09:54:13.224558+0000","last_became_active":"2024-04-25T09:54:13.219016+0000","last_became_peered":"2024-04-25T09:54:13.219016+0000","last_unstale":"2024-04-25T09:54:13.224558+0000","last_undegraded":"2024-04-25T09:54:13.224558+0000","last_fullsized":"2024-04-25T09:54:13.224558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:08.759006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.236171+0000","last_change":"2024-04-25T09:54:12.208283+0000","last_active":"2024-04-25T09:54:13.236171+0000","last_peered":"2024-04-25T09:54:13.236171+0000","last_clean":"2024-04-25T09:54:13.236171+0000","last_became_active":"2024-04-25T09:54:12.208046+0000","last_became_peered":"2024-04-25T09:54:12.208046+0000","last_unstale":"2024-04-25T09:54:13.236171+0000","last_undegraded":"2024-04-25T09:54:13.236171+0000","last_fullsized":"2024-04-25T09:54:13.236171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:54.188789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.229232+0000","last_change":"2024-04-25T09:54:13.222182+0000","last_active":"2024-04-25T09:54:13.229232+0000","last_peered":"2024-04-25T09:54:13.229232+0000","last_clean":"2024-04-25T09:54:13.229232+0000","last_became_active":"2024-04-25T09:54:13.221929+0000","last_became_peered":"2024-04-25T09:54:13.221929+0000","last_unstale":"2024-04-25T09:54:13.229232+0000","last_undegraded":"2024-04-25T09:54:13.229232+0000","last_fullsized":"2024-04-25T09:54:13.229232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.391093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.237096+0000","last_change":"2024-04-25T09:54:12.207303+0000","last_active":"2024-04-25T09:54:13.237096+0000","last_peered":"2024-04-25T09:54:13.237096+0000","last_clean":"2024-04-25T09:54:13.237096+0000","last_became_active":"2024-04-25T09:54:12.207000+0000","last_became_peered":"2024-04-25T09:54:12.207000+0000","last_unstale":"2024-04-25T09:54:13.237096+0000","last_undegraded":"2024-04-25T09:54:13.237096+0000","last_fullsized":"2024-04-25T09:54:13.237096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:07.751815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223804+0000","last_change":"2024-04-25T09:54:13.216226+0000","last_active":"2024-04-25T09:54:13.223804+0000","last_peered":"2024-04-25T09:54:13.223804+0000","last_clean":"2024-04-25T09:54:13.223804+0000","last_became_active":"2024-04-25T09:54:13.215925+0000","last_became_peered":"2024-04-25T09:54:13.215925+0000","last_unstale":"2024-04-25T09:54:13.223804+0000","last_undegraded":"2024-04-25T09:54:13.223804+0000","last_fullsized":"2024-04-25T09:54:13.223804+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:16.483875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.235538+0000","last_change":"2024-04-25T09:54:12.210453+0000","last_active":"2024-04-25T09:54:13.235538+0000","last_peered":"2024-04-25T09:54:13.235538+0000","last_clean":"2024-04-25T09:54:13.235538+0000","last_became_active":"2024-04-25T09:54:12.209865+0000","last_became_peered":"2024-04-25T09:54:12.209865+0000","last_unstale":"2024-04-25T09:54:13.235538+0000","last_undegraded":"2024-04-25T09:54:13.235538+0000","last_fullsized":"2024-04-25T09:54:13.235538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:41.176085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226950+0000","last_change":"2024-04-25T09:54:13.221263+0000","last_active":"2024-04-25T09:54:13.226950+0000","last_peered":"2024-04-25T09:54:13.226950+0000","last_clean":"2024-04-25T09:54:13.226950+0000","last_became_active":"2024-04-25T09:54:13.220038+0000","last_became_peered":"2024-04-25T09:54:13.220038+0000","last_unstale":"2024-04-25T09:54:13.226950+0000","last_undegraded":"2024-04-25T09:54:13.226950+0000","last_fullsized":"2024-04-25T09:54:13.226950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:22.609986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.210709+0000","last_change":"2024-04-25T09:54:12.213684+0000","last_active":"2024-04-25T09:54:13.210709+0000","last_peered":"2024-04-25T09:54:13.210709+0000","last_clean":"2024-04-25T09:54:13.210709+0000","last_became_active":"2024-04-25T09:54:12.213411+0000","last_became_peered":"2024-04-25T09:54:12.213411+0000","last_unstale":"2024-04-25T09:54:13.210709+0000","last_undegraded":"2024-04-25T09:54:13.210709+0000","last_fullsized":"2024-04-25T09:54:13.210709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.462254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.219092+0000","last_change":"2024-04-25T09:54:13.213035+0000","last_active":"2024-04-25T09:54:13.219092+0000","last_peered":"2024-04-25T09:54:13.219092+0000","last_clean":"2024-04-25T09:54:13.219092+0000","last_became_active":"2024-04-25T09:54:13.212759+0000","last_became_peered":"2024-04-25T09:54:13.212759+0000","last_unstale":"2024-04-25T09:54:13.219092+0000","last_undegraded":"2024-04-25T09:54:13.219092+0000","last_fullsized":"2024-04-25T09:54:13.219092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:55.404316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.206991+0000","last_change":"2024-04-25T09:54:12.203282+0000","last_active":"2024-04-25T09:54:13.206991+0000","last_peered":"2024-04-25T09:54:13.206991+0000","last_clean":"2024-04-25T09:54:13.206991+0000","last_became_active":"2024-04-25T09:54:12.202984+0000","last_became_peered":"2024-04-25T09:54:12.202984+0000","last_unstale":"2024-04-25T09:54:13.206991+0000","last_undegraded":"2024-04-25T09:54:13.206991+0000","last_fullsized":"2024-04-25T09:54:13.206991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:50.864860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226715+0000","last_change":"2024-04-25T09:54:13.219547+0000","last_active":"2024-04-25T09:54:13.226715+0000","last_peered":"2024-04-25T09:54:13.226715+0000","last_clean":"2024-04-25T09:54:13.226715+0000","last_became_active":"2024-04-25T09:54:13.219241+0000","last_became_peered":"2024-04-25T09:54:13.219241+0000","last_unstale":"2024-04-25T09:54:13.226715+0000","last_undegraded":"2024-04-25T09:54:13.226715+0000","last_fullsized":"2024-04-25T09:54:13.226715+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:53.335804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.232920+0000","last_change":"2024-04-25T09:54:12.200217+0000","last_active":"2024-04-25T09:54:13.232920+0000","last_peered":"2024-04-25T09:54:13.232920+0000","last_clean":"2024-04-25T09:54:13.232920+0000","last_became_active":"2024-04-25T09:54:12.199910+0000","last_became_peered":"2024-04-25T09:54:12.199910+0000","last_unstale":"2024-04-25T09:54:13.232920+0000","last_undegraded":"2024-04-25T09:54:13.232920+0000","last_fullsized":"2024-04-25T09:54:13.232920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.462254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.234110+0000","last_change":"2024-04-25T09:54:13.227463+0000","last_active":"2024-04-25T09:54:13.234110+0000","last_peered":"2024-04-25T09:54:13.234110+0000","last_clean":"2024-04-25T09:54:13.234110+0000","last_became_active":"2024-04-25T09:54:13.226259+0000","last_became_peered":"2024-04-25T09:54:13.226259+0000","last_unstale":"2024-04-25T09:54:13.234110+0000","last_undegraded":"2024-04-25T09:54:13.234110+0000","last_fullsized":"2024-04-25T09:54:13.234110+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:54.136097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.231798+0000","last_change":"2024-04-25T09:54:12.205548+0000","last_active":"2024-04-25T09:54:13.231798+0000","last_peered":"2024-04-25T09:54:13.231798+0000","last_clean":"2024-04-25T09:54:13.231798+0000","last_became_active":"2024-04-25T09:54:12.204843+0000","last_became_peered":"2024-04-25T09:54:12.204843+0000","last_unstale":"2024-04-25T09:54:13.231798+0000","last_undegraded":"2024-04-25T09:54:13.231798+0000","last_fullsized":"2024-04-25T09:54:13.231798+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:56.991847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223212+0000","last_change":"2024-04-25T09:54:13.219615+0000","last_active":"2024-04-25T09:54:13.223212+0000","last_peered":"2024-04-25T09:54:13.223212+0000","last_clean":"2024-04-25T09:54:13.223212+0000","last_became_active":"2024-04-25T09:54:13.218526+0000","last_became_peered":"2024-04-25T09:54:13.218526+0000","last_unstale":"2024-04-25T09:54:13.223212+0000","last_undegraded":"2024-04-25T09:54:13.223212+0000","last_fullsized":"2024-04-25T09:54:13.223212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:34.596310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.232608+0000","last_change":"2024-04-25T09:54:12.205130+0000","last_active":"2024-04-25T09:54:13.232608+0000","last_peered":"2024-04-25T09:54:13.232608+0000","last_clean":"2024-04-25T09:54:13.232608+0000","last_became_active":"2024-04-25T09:54:12.204593+0000","last_became_peered":"2024-04-25T09:54:12.204593+0000","last_unstale":"2024-04-25T09:54:13.232608+0000","last_undegraded":"2024-04-25T09:54:13.232608+0000","last_fullsized":"2024-04-25T09:54:13.232608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:37.998401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223050+0000","last_change":"2024-04-25T09:54:13.219148+0000","last_active":"2024-04-25T09:54:13.223050+0000","last_peered":"2024-04-25T09:54:13.223050+0000","last_clean":"2024-04-25T09:54:13.223050+0000","last_became_active":"2024-04-25T09:54:13.218823+0000","last_became_peered":"2024-04-25T09:54:13.218823+0000","last_unstale":"2024-04-25T09:54:13.223050+0000","last_undegraded":"2024-04-25T09:54:13.223050+0000","last_fullsized":"2024-04-25T09:54:13.223050+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:01.620004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.231565+0000","last_change":"2024-04-25T09:54:12.202150+0000","last_active":"2024-04-25T09:54:13.231565+0000","last_peered":"2024-04-25T09:54:13.231565+0000","last_clean":"2024-04-25T09:54:13.231565+0000","last_became_active":"2024-04-25T09:54:12.201835+0000","last_became_peered":"2024-04-25T09:54:12.201835+0000","last_unstale":"2024-04-25T09:54:13.231565+0000","last_undegraded":"2024-04-25T09:54:13.231565+0000","last_fullsized":"2024-04-25T09:54:13.231565+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:13.980512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226594+0000","last_change":"2024-04-25T09:54:13.220716+0000","last_active":"2024-04-25T09:54:13.226594+0000","last_peered":"2024-04-25T09:54:13.226594+0000","last_clean":"2024-04-25T09:54:13.226594+0000","last_became_active":"2024-04-25T09:54:13.220402+0000","last_became_peered":"2024-04-25T09:54:13.220402+0000","last_unstale":"2024-04-25T09:54:13.226594+0000","last_undegraded":"2024-04-25T09:54:13.226594+0000","last_fullsized":"2024-04-25T09:54:13.226594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:58.139201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.212311+0000","last_change":"2024-04-25T09:54:12.207715+0000","last_active":"2024-04-25T09:54:13.212311+0000","last_peered":"2024-04-25T09:54:13.212311+0000","last_clean":"2024-04-25T09:54:13.212311+0000","last_became_active":"2024-04-25T09:54:12.207411+0000","last_became_peered":"2024-04-25T09:54:12.207411+0000","last_unstale":"2024-04-25T09:54:13.212311+0000","last_undegraded":"2024-04-25T09:54:13.212311+0000","last_fullsized":"2024-04-25T09:54:13.212311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:00.612813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.218180+0000","last_change":"2024-04-25T09:54:13.215019+0000","last_active":"2024-04-25T09:54:13.218180+0000","last_peered":"2024-04-25T09:54:13.218180+0000","last_clean":"2024-04-25T09:54:13.218180+0000","last_became_active":"2024-04-25T09:54:13.212810+0000","last_became_peered":"2024-04-25T09:54:13.212810+0000","last_unstale":"2024-04-25T09:54:13.218180+0000","last_undegraded":"2024-04-25T09:54:13.218180+0000","last_fullsized":"2024-04-25T09:54:13.218180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:08.759006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.234470+0000","last_change":"2024-04-25T09:54:12.210184+0000","last_active":"2024-04-25T09:54:13.234470+0000","last_peered":"2024-04-25T09:54:13.234470+0000","last_clean":"2024-04-25T09:54:13.234470+0000","last_became_active":"2024-04-25T09:54:12.209821+0000","last_became_peered":"2024-04-25T09:54:12.209821+0000","last_unstale":"2024-04-25T09:54:13.234470+0000","last_undegraded":"2024-04-25T09:54:13.234470+0000","last_fullsized":"2024-04-25T09:54:13.234470+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:50.097937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.224608+0000","last_change":"2024-04-25T09:54:13.218521+0000","last_active":"2024-04-25T09:54:13.224608+0000","last_peered":"2024-04-25T09:54:13.224608+0000","last_clean":"2024-04-25T09:54:13.224608+0000","last_became_active":"2024-04-25T09:54:13.218153+0000","last_became_peered":"2024-04-25T09:54:13.218153+0000","last_unstale":"2024-04-25T09:54:13.224608+0000","last_undegraded":"2024-04-25T09:54:13.224608+0000","last_fullsized":"2024-04-25T09:54:13.224608+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:57.999037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.204608+0000","last_change":"2024-04-25T09:54:12.202685+0000","last_active":"2024-04-25T09:54:13.204608+0000","last_peered":"2024-04-25T09:54:13.204608+0000","last_clean":"2024-04-25T09:54:13.204608+0000","last_became_active":"2024-04-25T09:54:12.201183+0000","last_became_peered":"2024-04-25T09:54:12.201183+0000","last_unstale":"2024-04-25T09:54:13.204608+0000","last_undegraded":"2024-04-25T09:54:13.204608+0000","last_fullsized":"2024-04-25T09:54:13.204608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:58.982383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.216815+0000","last_change":"2024-04-25T09:54:13.213472+0000","last_active":"2024-04-25T09:54:13.216815+0000","last_peered":"2024-04-25T09:54:13.216815+0000","last_clean":"2024-04-25T09:54:13.216815+0000","last_became_active":"2024-04-25T09:54:13.212692+0000","last_became_peered":"2024-04-25T09:54:13.212692+0000","last_unstale":"2024-04-25T09:54:13.216815+0000","last_undegraded":"2024-04-25T09:54:13.216815+0000","last_fullsized":"2024-04-25T09:54:13.216815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:22.609986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.201627+0000","last_change":"2024-04-25T09:54:12.204289+0000","last_active":"2024-04-25T09:54:13.201627+0000","last_peered":"2024-04-25T09:54:13.201627+0000","last_clean":"2024-04-25T09:54:13.201627+0000","last_became_active":"2024-04-25T09:54:12.203980+0000","last_became_peered":"2024-04-25T09:54:12.203980+0000","last_unstale":"2024-04-25T09:54:13.201627+0000","last_undegraded":"2024-04-25T09:54:13.201627+0000","last_fullsized":"2024-04-25T09:54:13.201627+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:00.612813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.227687+0000","last_change":"2024-04-25T09:54:13.224318+0000","last_active":"2024-04-25T09:54:13.227687+0000","last_peered":"2024-04-25T09:54:13.227687+0000","last_clean":"2024-04-25T09:54:13.227687+0000","last_became_active":"2024-04-25T09:54:13.220621+0000","last_became_peered":"2024-04-25T09:54:13.220621+0000","last_unstale":"2024-04-25T09:54:13.227687+0000","last_undegraded":"2024-04-25T09:54:13.227687+0000","last_fullsized":"2024-04-25T09:54:13.227687+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:10.890138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.204775+0000","last_change":"2024-04-25T09:54:12.206903+0000","last_active":"2024-04-25T09:54:13.204775+0000","last_peered":"2024-04-25T09:54:13.204775+0000","last_clean":"2024-04-25T09:54:13.204775+0000","last_became_active":"2024-04-25T09:54:12.206621+0000","last_became_peered":"2024-04-25T09:54:12.206621+0000","last_unstale":"2024-04-25T09:54:13.204775+0000","last_undegraded":"2024-04-25T09:54:13.204775+0000","last_fullsized":"2024-04-25T09:54:13.204775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:59.099186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226965+0000","last_change":"2024-04-25T09:54:13.224904+0000","last_active":"2024-04-25T09:54:13.226965+0000","last_peered":"2024-04-25T09:54:13.226965+0000","last_clean":"2024-04-25T09:54:13.226965+0000","last_became_active":"2024-04-25T09:54:13.223777+0000","last_became_peered":"2024-04-25T09:54:13.223777+0000","last_unstale":"2024-04-25T09:54:13.226965+0000","last_undegraded":"2024-04-25T09:54:13.226965+0000","last_fullsized":"2024-04-25T09:54:13.226965+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:16.483875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.236024+0000","last_change":"2024-04-25T09:54:12.208301+0000","last_active":"2024-04-25T09:54:13.236024+0000","last_peered":"2024-04-25T09:54:13.236024+0000","last_clean":"2024-04-25T09:54:13.236024+0000","last_became_active":"2024-04-25T09:54:12.208016+0000","last_became_peered":"2024-04-25T09:54:12.208016+0000","last_unstale":"2024-04-25T09:54:13.236024+0000","last_undegraded":"2024-04-25T09:54:13.236024+0000","last_fullsized":"2024-04-25T09:54:13.236024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:41.176085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.215789+0000","last_change":"2024-04-25T09:54:13.212052+0000","last_active":"2024-04-25T09:54:13.215789+0000","last_peered":"2024-04-25T09:54:13.215789+0000","last_clean":"2024-04-25T09:54:13.215789+0000","last_became_active":"2024-04-25T09:54:13.211556+0000","last_became_peered":"2024-04-25T09:54:13.211556+0000","last_unstale":"2024-04-25T09:54:13.215789+0000","last_undegraded":"2024-04-25T09:54:13.215789+0000","last_fullsized":"2024-04-25T09:54:13.215789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:42.183276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.205389+0000","last_change":"2024-04-25T09:54:12.205196+0000","last_active":"2024-04-25T09:54:13.205389+0000","last_peered":"2024-04-25T09:54:13.205389+0000","last_clean":"2024-04-25T09:54:13.205389+0000","last_became_active":"2024-04-25T09:54:12.204844+0000","last_became_peered":"2024-04-25T09:54:12.204844+0000","last_unstale":"2024-04-25T09:54:13.205389+0000","last_undegraded":"2024-04-25T09:54:13.205389+0000","last_fullsized":"2024-04-25T09:54:13.205389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:37.753205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.227743+0000","last_change":"2024-04-25T09:54:13.221021+0000","last_active":"2024-04-25T09:54:13.227743+0000","last_peered":"2024-04-25T09:54:13.227743+0000","last_clean":"2024-04-25T09:54:13.227743+0000","last_became_active":"2024-04-25T09:54:13.220377+0000","last_became_peered":"2024-04-25T09:54:13.220377+0000","last_unstale":"2024-04-25T09:54:13.227743+0000","last_undegraded":"2024-04-25T09:54:13.227743+0000","last_fullsized":"2024-04-25T09:54:13.227743+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:38.621459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.211179+0000","last_change":"2024-04-25T09:54:12.208556+0000","last_active":"2024-04-25T09:54:13.211179+0000","last_peered":"2024-04-25T09:54:13.211179+0000","last_clean":"2024-04-25T09:54:13.211179+0000","last_became_active":"2024-04-25T09:54:12.208050+0000","last_became_peered":"2024-04-25T09:54:12.208050+0000","last_unstale":"2024-04-25T09:54:13.211179+0000","last_undegraded":"2024-04-25T09:54:13.211179+0000","last_fullsized":"2024-04-25T09:54:13.211179+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:00.612813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.231310+0000","last_change":"2024-04-25T09:54:13.228463+0000","last_active":"2024-04-25T09:54:13.231310+0000","last_peered":"2024-04-25T09:54:13.231310+0000","last_clean":"2024-04-25T09:54:13.231310+0000","last_became_active":"2024-04-25T09:54:13.228160+0000","last_became_peered":"2024-04-25T09:54:13.228160+0000","last_unstale":"2024-04-25T09:54:13.231310+0000","last_undegraded":"2024-04-25T09:54:13.231310+0000","last_fullsized":"2024-04-25T09:54:13.231310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:53.335804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.211403+0000","last_change":"2024-04-25T09:54:12.213001+0000","last_active":"2024-04-25T09:54:13.211403+0000","last_peered":"2024-04-25T09:54:13.211403+0000","last_clean":"2024-04-25T09:54:13.211403+0000","last_became_active":"2024-04-25T09:54:12.212769+0000","last_became_peered":"2024-04-25T09:54:12.212769+0000","last_unstale":"2024-04-25T09:54:13.211403+0000","last_undegraded":"2024-04-25T09:54:13.211403+0000","last_fullsized":"2024-04-25T09:54:13.211403+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:06.392698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226126+0000","last_change":"2024-04-25T09:54:13.220238+0000","last_active":"2024-04-25T09:54:13.226126+0000","last_peered":"2024-04-25T09:54:13.226126+0000","last_clean":"2024-04-25T09:54:13.226126+0000","last_became_active":"2024-04-25T09:54:13.219930+0000","last_became_peered":"2024-04-25T09:54:13.219930+0000","last_unstale":"2024-04-25T09:54:13.226126+0000","last_undegraded":"2024-04-25T09:54:13.226126+0000","last_fullsized":"2024-04-25T09:54:13.226126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:07.399889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.210995+0000","last_change":"2024-04-25T09:54:12.210026+0000","last_active":"2024-04-25T09:54:13.210995+0000","last_peered":"2024-04-25T09:54:13.210995+0000","last_clean":"2024-04-25T09:54:13.210995+0000","last_became_active":"2024-04-25T09:54:12.209775+0000","last_became_peered":"2024-04-25T09:54:12.209775+0000","last_unstale":"2024-04-25T09:54:13.210995+0000","last_undegraded":"2024-04-25T09:54:13.210995+0000","last_fullsized":"2024-04-25T09:54:13.210995+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:49.356287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.230178+0000","last_change":"2024-04-25T09:54:13.228303+0000","last_active":"2024-04-25T09:54:13.230178+0000","last_peered":"2024-04-25T09:54:13.230178+0000","last_clean":"2024-04-25T09:54:13.230178+0000","last_became_active":"2024-04-25T09:54:13.227969+0000","last_became_peered":"2024-04-25T09:54:13.227969+0000","last_unstale":"2024-04-25T09:54:13.230178+0000","last_undegraded":"2024-04-25T09:54:13.230178+0000","last_fullsized":"2024-04-25T09:54:13.230178+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.261566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.208824+0000","last_change":"2024-04-25T09:54:12.201019+0000","last_active":"2024-04-25T09:54:13.208824+0000","last_peered":"2024-04-25T09:54:13.208824+0000","last_clean":"2024-04-25T09:54:13.208824+0000","last_became_active":"2024-04-25T09:54:12.200636+0000","last_became_peered":"2024-04-25T09:54:12.200636+0000","last_unstale":"2024-04-25T09:54:13.208824+0000","last_undegraded":"2024-04-25T09:54:13.208824+0000","last_fullsized":"2024-04-25T09:54:13.208824+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:05.905982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.214901+0000","last_change":"2024-04-25T09:54:13.211800+0000","last_active":"2024-04-25T09:54:13.214901+0000","last_peered":"2024-04-25T09:54:13.214901+0000","last_clean":"2024-04-25T09:54:13.214901+0000","last_became_active":"2024-04-25T09:54:13.211537+0000","last_became_peered":"2024-04-25T09:54:13.211537+0000","last_unstale":"2024-04-25T09:54:13.214901+0000","last_undegraded":"2024-04-25T09:54:13.214901+0000","last_fullsized":"2024-04-25T09:54:13.214901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:55.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.207845+0000","last_change":"2024-04-25T09:54:12.206750+0000","last_active":"2024-04-25T09:54:13.207845+0000","last_peered":"2024-04-25T09:54:13.207845+0000","last_clean":"2024-04-25T09:54:13.207845+0000","last_became_active":"2024-04-25T09:54:12.206406+0000","last_became_peered":"2024-04-25T09:54:12.206406+0000","last_unstale":"2024-04-25T09:54:13.207845+0000","last_undegraded":"2024-04-25T09:54:13.207845+0000","last_fullsized":"2024-04-25T09:54:13.207845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:54.397125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.232424+0000","last_change":"2024-04-25T09:54:13.230179+0000","last_active":"2024-04-25T09:54:13.232424+0000","last_peered":"2024-04-25T09:54:13.232424+0000","last_clean":"2024-04-25T09:54:13.232424+0000","last_became_active":"2024-04-25T09:54:13.229903+0000","last_became_peered":"2024-04-25T09:54:13.229903+0000","last_unstale":"2024-04-25T09:54:13.232424+0000","last_undegraded":"2024-04-25T09:54:13.232424+0000","last_fullsized":"2024-04-25T09:54:13.232424+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:48.938736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.202818+0000","last_change":"2024-04-25T09:54:12.207714+0000","last_active":"2024-04-25T09:54:13.202818+0000","last_peered":"2024-04-25T09:54:13.202818+0000","last_clean":"2024-04-25T09:54:13.202818+0000","last_became_active":"2024-04-25T09:54:12.207420+0000","last_became_peered":"2024-04-25T09:54:12.207420+0000","last_unstale":"2024-04-25T09:54:13.202818+0000","last_undegraded":"2024-04-25T09:54:13.202818+0000","last_fullsized":"2024-04-25T09:54:13.202818+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:06.392698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.201210+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:54:13.201210+0000","last_peered":"2024-04-25T09:54:13.201210+0000","last_clean":"2024-04-25T09:54:13.201210+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:54:13.201210+0000","last_undegraded":"2024-04-25T09:54:13.201210+0000","last_fullsized":"2024-04-25T09:54:13.201210+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":183932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1024000,"data_stored":773730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:54:16.220 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:54:16.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.353+0000 7fbe7d161640 1 Processor -- start 2024-04-25T09:54:16.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.354+0000 7fbe7d161640 1 -- start start 2024-04-25T09:54:16.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.354+0000 7fbe7d161640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbe780b2cb0 0x7fbe780d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.354+0000 7fbe7d161640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbe780d1020 0x7fbe780d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.354+0000 7fbe7d161640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe780b97a0 con 0x7fbe7815dbe0 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.354+0000 7fbe7d161640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe7815dfc0 con 0x7fbe780d1020 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.354+0000 7fbe7d161640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe780d1a00 con 0x7fbe780b2cb0 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.354+0000 7fbe7759e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbe780d1020 0x7fbe780d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.355+0000 7fbe7759e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbe780d1020 0x7fbe780d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53440/0 (socket says 172.21.15.19:53440) 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.355+0000 7fbe7759e640 1 -- 172.21.15.19:0/2698531040 learned_addr learned my addr 172.21.15.19:0/2698531040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.355+0000 7fbe6ffff640 1 --2- 172.21.15.19:0/2698531040 >> v2:172.21.15.112:3300/0 conn(0x7fbe780b2cb0 0x7fbe780d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.355+0000 7fbe7759e640 1 -- 172.21.15.19:0/2698531040 >> v2:172.21.15.112:3300/0 conn(0x7fbe780b2cb0 msgr2=0x7fbe780d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.355+0000 7fbe7759e640 1 --2- 172.21.15.19:0/2698531040 >> v2:172.21.15.112:3300/0 conn(0x7fbe780b2cb0 0x7fbe780d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.355+0000 7fbe7759e640 1 -- 172.21.15.19:0/2698531040 >> v1:172.21.15.19:6789/0 conn(0x7fbe7815dbe0 legacy=0x7fbe780d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.355+0000 7fbe7759e640 1 -- 172.21.15.19:0/2698531040 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe780d1c50 con 0x7fbe780d1020 2024-04-25T09:54:16.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.355+0000 7fbe7759e640 1 --2- 172.21.15.19:0/2698531040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe780d1020 0x7fbe780d2c70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbe7806cff0 tx=0x7fbe7002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4006675348ba9f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.356+0000 7fbe7659c640 1 -- 172.21.15.19:0/2698531040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70036070 con 0x7fbe780d1020 2024-04-25T09:54:16.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.356+0000 7fbe7659c640 1 -- 172.21.15.19:0/2698531040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70031600 con 0x7fbe780d1020 2024-04-25T09:54:16.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.356+0000 7fbe7659c640 1 -- 172.21.15.19:0/2698531040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe700319d0 con 0x7fbe780d1020 2024-04-25T09:54:16.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.356+0000 7fbe7d161640 1 -- 172.21.15.19:0/2698531040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe780d1020 msgr2=0x7fbe780d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.356+0000 7fbe7d161640 1 --2- 172.21.15.19:0/2698531040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe780d1020 0x7fbe780d2c70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbe7806cff0 tx=0x7fbe7002a730 comp rx=0 tx=0).stop 2024-04-25T09:54:16.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.356+0000 7fbe7d161640 1 -- 172.21.15.19:0/2698531040 shutdown_connections 2024-04-25T09:54:16.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.356+0000 7fbe7d161640 1 --2- 172.21.15.19:0/2698531040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe780d1020 0x7fbe780d2c70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.356+0000 7fbe7d161640 1 --2- 172.21.15.19:0/2698531040 >> v2:172.21.15.112:3300/0 conn(0x7fbe780b2cb0 0x7fbe780d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.356+0000 7fbe7d161640 1 -- 172.21.15.19:0/2698531040 >> 172.21.15.19:0/2698531040 conn(0x7fbe78061a20 msgr2=0x7fbe78061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.356+0000 7fbe7d161640 1 -- 172.21.15.19:0/2698531040 shutdown_connections 2024-04-25T09:54:16.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.356+0000 7fbe7d161640 1 -- 172.21.15.19:0/2698531040 wait complete. 2024-04-25T09:54:16.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.358+0000 7fbe7d161640 1 Processor -- start 2024-04-25T09:54:16.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.358+0000 7fbe7d161640 1 -- start start 2024-04-25T09:54:16.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.358+0000 7fbe7d161640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe780b2cb0 0x7fbe780c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.358+0000 7fbe7d161640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe780d1020 0x7fbe780cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.358+0000 7fbe7d161640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe7815dbe0 0x7fbe780cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.358+0000 7fbe7d161640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780b92a0 con 0x7fbe780b2cb0 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.358+0000 7fbe7d161640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780d2210 con 0x7fbe7815dbe0 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.358+0000 7fbe7d161640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780b9420 con 0x7fbe780d1020 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe76d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe780b2cb0 0x7fbe780c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe76d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe780b2cb0 0x7fbe780c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53452/0 (socket says 172.21.15.19:53452) 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe76d9d640 1 -- 172.21.15.19:0/1403350937 learned_addr learned my addr 172.21.15.19:0/1403350937 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe7759e640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe7815dbe0 0x7fbe780cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe6ffff640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe780d1020 0x7fbe780cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe76d9d640 1 -- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe780d1020 msgr2=0x7fbe780cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe76d9d640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe780d1020 0x7fbe780cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe76d9d640 1 -- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe7815dbe0 msgr2=0x7fbe780cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe76d9d640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe7815dbe0 0x7fbe780cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe76d9d640 1 -- 172.21.15.19:0/1403350937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7815ee80 con 0x7fbe780b2cb0 2024-04-25T09:54:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe76d9d640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe780b2cb0 0x7fbe780c94d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800c3c0 tx=0x7fbe6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/1403350937 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe680052d0 con 0x7fbe780b2cb0 2024-04-25T09:54:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/1403350937 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe68005490 con 0x7fbe780b2cb0 2024-04-25T09:54:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/1403350937 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe68007260 con 0x7fbe780b2cb0 2024-04-25T09:54:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe7d161640 1 -- 172.21.15.19:0/1403350937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7815f160 con 0x7fbe780b2cb0 2024-04-25T09:54:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.359+0000 7fbe7d161640 1 -- 172.21.15.19:0/1403350937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7815f720 con 0x7fbe780b2cb0 2024-04-25T09:54:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.360+0000 7fbe7d161640 1 -- 172.21.15.19:0/1403350937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe78073990 con 0x7fbe780b2cb0 2024-04-25T09:54:16.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.361+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/1403350937 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe68007400 con 0x7fbe780b2cb0 2024-04-25T09:54:16.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.361+0000 7fbe6f7fe640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbe5806b4d0 0x7fbe5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.362+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/1403350937 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbe680897e0 con 0x7fbe780b2cb0 2024-04-25T09:54:16.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.362+0000 7fbe6ffff640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbe5806b4d0 0x7fbe5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.362+0000 7fbe6ffff640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbe5806b4d0 0x7fbe5806d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbe78159670 tx=0x7fbe60008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.366+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/1403350937 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe68058560 con 0x7fbe780b2cb0 2024-04-25T09:54:16.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.540+0000 7fbe7d161640 1 -- 172.21.15.19:0/1403350937 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe780d34e0 con 0x7fbe5806b4d0 2024-04-25T09:54:16.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.547+0000 7fbe6f7fe640 1 -- 172.21.15.19:0/1403350937 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251890 (secure 0 0 0) 0x7fbe780d34e0 con 0x7fbe5806b4d0 2024-04-25T09:54:16.551 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 -- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbe5806b4d0 msgr2=0x7fbe5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbe5806b4d0 0x7fbe5806d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbe78159670 tx=0x7fbe60008040 comp rx=0 tx=0).stop 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 -- 172.21.15.19:0/1403350937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe780b2cb0 msgr2=0x7fbe780c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe780b2cb0 0x7fbe780c94d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800c3c0 tx=0x7fbe6800c880 comp rx=0 tx=0).stop 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 -- 172.21.15.19:0/1403350937 shutdown_connections 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbe5806b4d0 0x7fbe5806d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe7815dbe0 0x7fbe780cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe780d1020 0x7fbe780cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 --2- 172.21.15.19:0/1403350937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe780b2cb0 0x7fbe780c94d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 -- 172.21.15.19:0/1403350937 >> 172.21.15.19:0/1403350937 conn(0x7fbe78061a20 msgr2=0x7fbe7806c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 -- 172.21.15.19:0/1403350937 shutdown_connections 2024-04-25T09:54:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.550+0000 7fbe7d161640 1 -- 172.21.15.19:0/1403350937 wait complete. 2024-04-25T09:54:16.563 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-25T09:54:14.926947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220668,"kb_used_data":4460,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754052,"statfs":{"total":773094113280,"available":772868149248,"internally_reserved":0,"allocated":4567040,"data_stored":2620002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000703"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.234799+0000","last_change":"2024-04-25T09:54:13.229307+0000","last_active":"2024-04-25T09:54:13.234799+0000","last_peered":"2024-04-25T09:54:13.234799+0000","last_clean":"2024-04-25T09:54:13.234799+0000","last_became_active":"2024-04-25T09:54:13.229016+0000","last_became_peered":"2024-04-25T09:54:13.229016+0000","last_unstale":"2024-04-25T09:54:13.234799+0000","last_undegraded":"2024-04-25T09:54:13.234799+0000","last_fullsized":"2024-04-25T09:54:13.234799+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:39.803246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.227765+0000","last_change":"2024-04-25T09:54:13.221152+0000","last_active":"2024-04-25T09:54:13.227765+0000","last_peered":"2024-04-25T09:54:13.227765+0000","last_clean":"2024-04-25T09:54:13.227765+0000","last_became_active":"2024-04-25T09:54:13.220728+0000","last_became_peered":"2024-04-25T09:54:13.220728+0000","last_unstale":"2024-04-25T09:54:13.227765+0000","last_undegraded":"2024-04-25T09:54:13.227765+0000","last_fullsized":"2024-04-25T09:54:13.227765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:58.849128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223808+0000","last_change":"2024-04-25T09:54:13.216689+0000","last_active":"2024-04-25T09:54:13.223808+0000","last_peered":"2024-04-25T09:54:13.223808+0000","last_clean":"2024-04-25T09:54:13.223808+0000","last_became_active":"2024-04-25T09:54:13.216398+0000","last_became_peered":"2024-04-25T09:54:13.216398+0000","last_unstale":"2024-04-25T09:54:13.223808+0000","last_undegraded":"2024-04-25T09:54:13.223808+0000","last_fullsized":"2024-04-25T09:54:13.223808+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:55.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.225010+0000","last_change":"2024-04-25T09:54:13.220310+0000","last_active":"2024-04-25T09:54:13.225010+0000","last_peered":"2024-04-25T09:54:13.225010+0000","last_clean":"2024-04-25T09:54:13.225010+0000","last_became_active":"2024-04-25T09:54:13.219790+0000","last_became_peered":"2024-04-25T09:54:13.219790+0000","last_unstale":"2024-04-25T09:54:13.225010+0000","last_undegraded":"2024-04-25T09:54:13.225010+0000","last_fullsized":"2024-04-25T09:54:13.225010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:24.225676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226992+0000","last_change":"2024-04-25T09:54:13.219979+0000","last_active":"2024-04-25T09:54:13.226992+0000","last_peered":"2024-04-25T09:54:13.226992+0000","last_clean":"2024-04-25T09:54:13.226992+0000","last_became_active":"2024-04-25T09:54:13.219024+0000","last_became_peered":"2024-04-25T09:54:13.219024+0000","last_unstale":"2024-04-25T09:54:13.226992+0000","last_undegraded":"2024-04-25T09:54:13.226992+0000","last_fullsized":"2024-04-25T09:54:13.226992+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:02.274809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.235906+0000","last_change":"2024-04-25T09:54:13.232607+0000","last_active":"2024-04-25T09:54:13.235906+0000","last_peered":"2024-04-25T09:54:13.235906+0000","last_clean":"2024-04-25T09:54:13.235906+0000","last_became_active":"2024-04-25T09:54:13.232323+0000","last_became_peered":"2024-04-25T09:54:13.232323+0000","last_unstale":"2024-04-25T09:54:13.235906+0000","last_undegraded":"2024-04-25T09:54:13.235906+0000","last_fullsized":"2024-04-25T09:54:13.235906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:42.183276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223321+0000","last_change":"2024-04-25T09:54:13.220700+0000","last_active":"2024-04-25T09:54:13.223321+0000","last_peered":"2024-04-25T09:54:13.223321+0000","last_clean":"2024-04-25T09:54:13.223321+0000","last_became_active":"2024-04-25T09:54:13.220409+0000","last_became_peered":"2024-04-25T09:54:13.220409+0000","last_unstale":"2024-04-25T09:54:13.223321+0000","last_undegraded":"2024-04-25T09:54:13.223321+0000","last_fullsized":"2024-04-25T09:54:13.223321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:54.136097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226912+0000","last_change":"2024-04-25T09:54:13.222091+0000","last_active":"2024-04-25T09:54:13.226912+0000","last_peered":"2024-04-25T09:54:13.226912+0000","last_clean":"2024-04-25T09:54:13.226912+0000","last_became_active":"2024-04-25T09:54:13.221799+0000","last_became_peered":"2024-04-25T09:54:13.221799+0000","last_unstale":"2024-04-25T09:54:13.226912+0000","last_undegraded":"2024-04-25T09:54:13.226912+0000","last_fullsized":"2024-04-25T09:54:13.226912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:24.575994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223879+0000","last_change":"2024-04-25T09:54:13.216799+0000","last_active":"2024-04-25T09:54:13.223879+0000","last_peered":"2024-04-25T09:54:13.223879+0000","last_clean":"2024-04-25T09:54:13.223879+0000","last_became_active":"2024-04-25T09:54:13.216509+0000","last_became_peered":"2024-04-25T09:54:13.216509+0000","last_unstale":"2024-04-25T09:54:13.223879+0000","last_undegraded":"2024-04-25T09:54:13.223879+0000","last_fullsized":"2024-04-25T09:54:13.223879+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:42.183276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.210135+0000","last_change":"2024-04-25T09:54:13.208370+0000","last_active":"2024-04-25T09:54:13.210135+0000","last_peered":"2024-04-25T09:54:13.210135+0000","last_clean":"2024-04-25T09:54:13.210135+0000","last_became_active":"2024-04-25T09:54:13.207932+0000","last_became_peered":"2024-04-25T09:54:13.207932+0000","last_unstale":"2024-04-25T09:54:13.210135+0000","last_undegraded":"2024-04-25T09:54:13.210135+0000","last_fullsized":"2024-04-25T09:54:13.210135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:38.621459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223191+0000","last_change":"2024-04-25T09:54:13.218780+0000","last_active":"2024-04-25T09:54:13.223191+0000","last_peered":"2024-04-25T09:54:13.223191+0000","last_clean":"2024-04-25T09:54:13.223191+0000","last_became_active":"2024-04-25T09:54:13.218521+0000","last_became_peered":"2024-04-25T09:54:13.218521+0000","last_unstale":"2024-04-25T09:54:13.223191+0000","last_undegraded":"2024-04-25T09:54:13.223191+0000","last_fullsized":"2024-04-25T09:54:13.223191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:10.890138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223762+0000","last_change":"2024-04-25T09:54:13.216188+0000","last_active":"2024-04-25T09:54:13.223762+0000","last_peered":"2024-04-25T09:54:13.223762+0000","last_clean":"2024-04-25T09:54:13.223762+0000","last_became_active":"2024-04-25T09:54:13.215871+0000","last_became_peered":"2024-04-25T09:54:13.215871+0000","last_unstale":"2024-04-25T09:54:13.223762+0000","last_undegraded":"2024-04-25T09:54:13.223762+0000","last_fullsized":"2024-04-25T09:54:13.223762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.261566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.209604+0000","last_change":"2024-04-25T09:54:13.206445+0000","last_active":"2024-04-25T09:54:13.209604+0000","last_peered":"2024-04-25T09:54:13.209604+0000","last_clean":"2024-04-25T09:54:13.209604+0000","last_became_active":"2024-04-25T09:54:13.206159+0000","last_became_peered":"2024-04-25T09:54:13.206159+0000","last_unstale":"2024-04-25T09:54:13.209604+0000","last_undegraded":"2024-04-25T09:54:13.209604+0000","last_fullsized":"2024-04-25T09:54:13.209604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:53.335804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.220007+0000","last_change":"2024-04-25T09:54:13.217370+0000","last_active":"2024-04-25T09:54:13.220007+0000","last_peered":"2024-04-25T09:54:13.220007+0000","last_clean":"2024-04-25T09:54:13.220007+0000","last_became_active":"2024-04-25T09:54:13.217079+0000","last_became_peered":"2024-04-25T09:54:13.217079+0000","last_unstale":"2024-04-25T09:54:13.220007+0000","last_undegraded":"2024-04-25T09:54:13.220007+0000","last_fullsized":"2024-04-25T09:54:13.220007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:55.531727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.216798+0000","last_change":"2024-04-25T09:54:13.213430+0000","last_active":"2024-04-25T09:54:13.216798+0000","last_peered":"2024-04-25T09:54:13.216798+0000","last_clean":"2024-04-25T09:54:13.216798+0000","last_became_active":"2024-04-25T09:54:13.213160+0000","last_became_peered":"2024-04-25T09:54:13.213160+0000","last_unstale":"2024-04-25T09:54:13.216798+0000","last_undegraded":"2024-04-25T09:54:13.216798+0000","last_fullsized":"2024-04-25T09:54:13.216798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:58.849128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.207110+0000","last_change":"2024-04-25T09:54:13.206281+0000","last_active":"2024-04-25T09:54:13.207110+0000","last_peered":"2024-04-25T09:54:13.207110+0000","last_clean":"2024-04-25T09:54:13.207110+0000","last_became_active":"2024-04-25T09:54:13.205429+0000","last_became_peered":"2024-04-25T09:54:13.205429+0000","last_unstale":"2024-04-25T09:54:13.207110+0000","last_undegraded":"2024-04-25T09:54:13.207110+0000","last_fullsized":"2024-04-25T09:54:13.207110+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.391093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.215518+0000","last_change":"2024-04-25T09:54:13.206949+0000","last_active":"2024-04-25T09:54:13.215518+0000","last_peered":"2024-04-25T09:54:13.215518+0000","last_clean":"2024-04-25T09:54:13.215518+0000","last_became_active":"2024-04-25T09:54:13.206695+0000","last_became_peered":"2024-04-25T09:54:13.206695+0000","last_unstale":"2024-04-25T09:54:13.215518+0000","last_undegraded":"2024-04-25T09:54:13.215518+0000","last_fullsized":"2024-04-25T09:54:13.215518+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.261566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223155+0000","last_change":"2024-04-25T09:54:13.220536+0000","last_active":"2024-04-25T09:54:13.223155+0000","last_peered":"2024-04-25T09:54:13.223155+0000","last_clean":"2024-04-25T09:54:13.223155+0000","last_became_active":"2024-04-25T09:54:13.219875+0000","last_became_peered":"2024-04-25T09:54:13.219875+0000","last_unstale":"2024-04-25T09:54:13.223155+0000","last_undegraded":"2024-04-25T09:54:13.223155+0000","last_fullsized":"2024-04-25T09:54:13.223155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.330344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223811+0000","last_change":"2024-04-25T09:54:13.216202+0000","last_active":"2024-04-25T09:54:13.223811+0000","last_peered":"2024-04-25T09:54:13.223811+0000","last_clean":"2024-04-25T09:54:13.223811+0000","last_became_active":"2024-04-25T09:54:13.215944+0000","last_became_peered":"2024-04-25T09:54:13.215944+0000","last_unstale":"2024-04-25T09:54:13.223811+0000","last_undegraded":"2024-04-25T09:54:13.223811+0000","last_fullsized":"2024-04-25T09:54:13.223811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:52.807108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.220690+0000","last_change":"2024-04-25T09:54:13.218825+0000","last_active":"2024-04-25T09:54:13.220690+0000","last_peered":"2024-04-25T09:54:13.220690+0000","last_clean":"2024-04-25T09:54:13.220690+0000","last_became_active":"2024-04-25T09:54:13.218530+0000","last_became_peered":"2024-04-25T09:54:13.218530+0000","last_unstale":"2024-04-25T09:54:13.220690+0000","last_undegraded":"2024-04-25T09:54:13.220690+0000","last_fullsized":"2024-04-25T09:54:13.220690+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:39.005592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223162+0000","last_change":"2024-04-25T09:54:13.217966+0000","last_active":"2024-04-25T09:54:13.223162+0000","last_peered":"2024-04-25T09:54:13.223162+0000","last_clean":"2024-04-25T09:54:13.223162+0000","last_became_active":"2024-04-25T09:54:13.217668+0000","last_became_peered":"2024-04-25T09:54:13.217668+0000","last_unstale":"2024-04-25T09:54:13.223162+0000","last_undegraded":"2024-04-25T09:54:13.223162+0000","last_fullsized":"2024-04-25T09:54:13.223162+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:24.575994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.228599+0000","last_change":"2024-04-25T09:54:13.221967+0000","last_active":"2024-04-25T09:54:13.228599+0000","last_peered":"2024-04-25T09:54:13.228599+0000","last_clean":"2024-04-25T09:54:13.228599+0000","last_became_active":"2024-04-25T09:54:13.221640+0000","last_became_peered":"2024-04-25T09:54:13.221640+0000","last_unstale":"2024-04-25T09:54:13.228599+0000","last_undegraded":"2024-04-25T09:54:13.228599+0000","last_fullsized":"2024-04-25T09:54:13.228599+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:07.399889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.218082+0000","last_change":"2024-04-25T09:54:13.214764+0000","last_active":"2024-04-25T09:54:13.218082+0000","last_peered":"2024-04-25T09:54:13.218082+0000","last_clean":"2024-04-25T09:54:13.218082+0000","last_became_active":"2024-04-25T09:54:13.214493+0000","last_became_peered":"2024-04-25T09:54:13.214493+0000","last_unstale":"2024-04-25T09:54:13.218082+0000","last_undegraded":"2024-04-25T09:54:13.218082+0000","last_fullsized":"2024-04-25T09:54:13.218082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:57.999037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.219829+0000","last_change":"2024-04-25T09:54:13.211683+0000","last_active":"2024-04-25T09:54:13.219829+0000","last_peered":"2024-04-25T09:54:13.219829+0000","last_clean":"2024-04-25T09:54:13.219829+0000","last_became_active":"2024-04-25T09:54:13.211372+0000","last_became_peered":"2024-04-25T09:54:13.211372+0000","last_unstale":"2024-04-25T09:54:13.219829+0000","last_undegraded":"2024-04-25T09:54:13.219829+0000","last_fullsized":"2024-04-25T09:54:13.219829+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:10.116155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.232802+0000","last_change":"2024-04-25T09:54:13.229223+0000","last_active":"2024-04-25T09:54:13.232802+0000","last_peered":"2024-04-25T09:54:13.232802+0000","last_clean":"2024-04-25T09:54:13.232802+0000","last_became_active":"2024-04-25T09:54:13.228891+0000","last_became_peered":"2024-04-25T09:54:13.228891+0000","last_unstale":"2024-04-25T09:54:13.232802+0000","last_undegraded":"2024-04-25T09:54:13.232802+0000","last_fullsized":"2024-04-25T09:54:13.232802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:12.287753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.231170+0000","last_change":"2024-04-25T09:54:13.223366+0000","last_active":"2024-04-25T09:54:13.231170+0000","last_peered":"2024-04-25T09:54:13.231170+0000","last_clean":"2024-04-25T09:54:13.231170+0000","last_became_active":"2024-04-25T09:54:13.223060+0000","last_became_peered":"2024-04-25T09:54:13.223060+0000","last_unstale":"2024-04-25T09:54:13.231170+0000","last_undegraded":"2024-04-25T09:54:13.231170+0000","last_fullsized":"2024-04-25T09:54:13.231170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.261566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226584+0000","last_change":"2024-04-25T09:54:13.223911+0000","last_active":"2024-04-25T09:54:13.226584+0000","last_peered":"2024-04-25T09:54:13.226584+0000","last_clean":"2024-04-25T09:54:13.226584+0000","last_became_active":"2024-04-25T09:54:13.223675+0000","last_became_peered":"2024-04-25T09:54:13.223675+0000","last_unstale":"2024-04-25T09:54:13.226584+0000","last_undegraded":"2024-04-25T09:54:13.226584+0000","last_fullsized":"2024-04-25T09:54:13.226584+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:55.531727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.229609+0000","last_change":"2024-04-25T09:54:13.223024+0000","last_active":"2024-04-25T09:54:13.229609+0000","last_peered":"2024-04-25T09:54:13.229609+0000","last_clean":"2024-04-25T09:54:13.229609+0000","last_became_active":"2024-04-25T09:54:13.222553+0000","last_became_peered":"2024-04-25T09:54:13.222553+0000","last_unstale":"2024-04-25T09:54:13.229609+0000","last_undegraded":"2024-04-25T09:54:13.229609+0000","last_fullsized":"2024-04-25T09:54:13.229609+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:39.803246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.220879+0000","last_change":"2024-04-25T09:54:13.217777+0000","last_active":"2024-04-25T09:54:13.220879+0000","last_peered":"2024-04-25T09:54:13.220879+0000","last_clean":"2024-04-25T09:54:13.220879+0000","last_became_active":"2024-04-25T09:54:13.217510+0000","last_became_peered":"2024-04-25T09:54:13.217510+0000","last_unstale":"2024-04-25T09:54:13.220879+0000","last_undegraded":"2024-04-25T09:54:13.220879+0000","last_fullsized":"2024-04-25T09:54:13.220879+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:06.913173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.229116+0000","last_change":"2024-04-25T09:54:13.227226+0000","last_active":"2024-04-25T09:54:13.229116+0000","last_peered":"2024-04-25T09:54:13.229116+0000","last_clean":"2024-04-25T09:54:13.229116+0000","last_became_active":"2024-04-25T09:54:13.226947+0000","last_became_peered":"2024-04-25T09:54:13.226947+0000","last_unstale":"2024-04-25T09:54:13.229116+0000","last_undegraded":"2024-04-25T09:54:13.229116+0000","last_fullsized":"2024-04-25T09:54:13.229116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:26.469445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.218106+0000","last_change":"2024-04-25T09:54:13.215642+0000","last_active":"2024-04-25T09:54:13.218106+0000","last_peered":"2024-04-25T09:54:13.218106+0000","last_clean":"2024-04-25T09:54:13.218106+0000","last_became_active":"2024-04-25T09:54:13.215026+0000","last_became_peered":"2024-04-25T09:54:13.215026+0000","last_unstale":"2024-04-25T09:54:13.218106+0000","last_undegraded":"2024-04-25T09:54:13.218106+0000","last_fullsized":"2024-04-25T09:54:13.218106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:53.335804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226105+0000","last_change":"2024-04-25T09:54:13.221098+0000","last_active":"2024-04-25T09:54:13.226105+0000","last_peered":"2024-04-25T09:54:13.226105+0000","last_clean":"2024-04-25T09:54:13.226105+0000","last_became_active":"2024-04-25T09:54:13.220588+0000","last_became_peered":"2024-04-25T09:54:13.220588+0000","last_unstale":"2024-04-25T09:54:13.226105+0000","last_undegraded":"2024-04-25T09:54:13.226105+0000","last_fullsized":"2024-04-25T09:54:13.226105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.391093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.215860+0000","last_change":"2024-04-25T09:54:13.207629+0000","last_active":"2024-04-25T09:54:13.215860+0000","last_peered":"2024-04-25T09:54:13.215860+0000","last_clean":"2024-04-25T09:54:13.215860+0000","last_became_active":"2024-04-25T09:54:13.207360+0000","last_became_peered":"2024-04-25T09:54:13.207360+0000","last_unstale":"2024-04-25T09:54:13.215860+0000","last_undegraded":"2024-04-25T09:54:13.215860+0000","last_fullsized":"2024-04-25T09:54:13.215860+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:16.483875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.232933+0000","last_change":"2024-04-25T09:54:12.203395+0000","last_active":"2024-04-25T09:54:13.232933+0000","last_peered":"2024-04-25T09:54:13.232933+0000","last_clean":"2024-04-25T09:54:13.232933+0000","last_became_active":"2024-04-25T09:54:12.203097+0000","last_became_peered":"2024-04-25T09:54:12.203097+0000","last_unstale":"2024-04-25T09:54:13.232933+0000","last_undegraded":"2024-04-25T09:54:13.232933+0000","last_fullsized":"2024-04-25T09:54:13.232933+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:51.847718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.227833+0000","last_change":"2024-04-25T09:54:13.225510+0000","last_active":"2024-04-25T09:54:13.227833+0000","last_peered":"2024-04-25T09:54:13.227833+0000","last_clean":"2024-04-25T09:54:13.227833+0000","last_became_active":"2024-04-25T09:54:13.224996+0000","last_became_peered":"2024-04-25T09:54:13.224996+0000","last_unstale":"2024-04-25T09:54:13.227833+0000","last_undegraded":"2024-04-25T09:54:13.227833+0000","last_fullsized":"2024-04-25T09:54:13.227833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:55.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.235178+0000","last_change":"2024-04-25T09:54:12.208454+0000","last_active":"2024-04-25T09:54:13.235178+0000","last_peered":"2024-04-25T09:54:13.235178+0000","last_clean":"2024-04-25T09:54:13.235178+0000","last_became_active":"2024-04-25T09:54:12.208140+0000","last_became_peered":"2024-04-25T09:54:12.208140+0000","last_unstale":"2024-04-25T09:54:13.235178+0000","last_undegraded":"2024-04-25T09:54:13.235178+0000","last_fullsized":"2024-04-25T09:54:13.235178+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:06.383902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.219786+0000","last_change":"2024-04-25T09:54:13.217575+0000","last_active":"2024-04-25T09:54:13.219786+0000","last_peered":"2024-04-25T09:54:13.219786+0000","last_clean":"2024-04-25T09:54:13.219786+0000","last_became_active":"2024-04-25T09:54:13.217295+0000","last_became_peered":"2024-04-25T09:54:13.217295+0000","last_unstale":"2024-04-25T09:54:13.219786+0000","last_undegraded":"2024-04-25T09:54:13.219786+0000","last_fullsized":"2024-04-25T09:54:13.219786+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:26.469445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.235977+0000","last_change":"2024-04-25T09:54:12.208115+0000","last_active":"2024-04-25T09:54:13.235977+0000","last_peered":"2024-04-25T09:54:13.235977+0000","last_clean":"2024-04-25T09:54:13.235977+0000","last_became_active":"2024-04-25T09:54:12.207857+0000","last_became_peered":"2024-04-25T09:54:12.207857+0000","last_unstale":"2024-04-25T09:54:13.235977+0000","last_undegraded":"2024-04-25T09:54:13.235977+0000","last_fullsized":"2024-04-25T09:54:13.235977+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:11.280562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.230576+0000","last_change":"2024-04-25T09:54:13.227489+0000","last_active":"2024-04-25T09:54:13.230576+0000","last_peered":"2024-04-25T09:54:13.230576+0000","last_clean":"2024-04-25T09:54:13.230576+0000","last_became_active":"2024-04-25T09:54:13.227179+0000","last_became_peered":"2024-04-25T09:54:13.227179+0000","last_unstale":"2024-04-25T09:54:13.230576+0000","last_undegraded":"2024-04-25T09:54:13.230576+0000","last_fullsized":"2024-04-25T09:54:13.230576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:09.907450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.235279+0000","last_change":"2024-04-25T09:54:12.210031+0000","last_active":"2024-04-25T09:54:13.235279+0000","last_peered":"2024-04-25T09:54:13.235279+0000","last_clean":"2024-04-25T09:54:13.235279+0000","last_became_active":"2024-04-25T09:54:12.209741+0000","last_became_peered":"2024-04-25T09:54:12.209741+0000","last_unstale":"2024-04-25T09:54:13.235279+0000","last_undegraded":"2024-04-25T09:54:13.235279+0000","last_fullsized":"2024-04-25T09:54:13.235279+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:42.323153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226848+0000","last_change":"2024-04-25T09:54:13.218400+0000","last_active":"2024-04-25T09:54:13.226848+0000","last_peered":"2024-04-25T09:54:13.226848+0000","last_clean":"2024-04-25T09:54:13.226848+0000","last_became_active":"2024-04-25T09:54:13.218109+0000","last_became_peered":"2024-04-25T09:54:13.218109+0000","last_unstale":"2024-04-25T09:54:13.226848+0000","last_undegraded":"2024-04-25T09:54:13.226848+0000","last_fullsized":"2024-04-25T09:54:13.226848+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:55.404316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.234029+0000","last_change":"2024-04-25T09:54:12.209211+0000","last_active":"2024-04-25T09:54:13.234029+0000","last_peered":"2024-04-25T09:54:13.234029+0000","last_clean":"2024-04-25T09:54:13.234029+0000","last_became_active":"2024-04-25T09:54:12.208917+0000","last_became_peered":"2024-04-25T09:54:12.208917+0000","last_unstale":"2024-04-25T09:54:13.234029+0000","last_undegraded":"2024-04-25T09:54:13.234029+0000","last_fullsized":"2024-04-25T09:54:13.234029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:54.397125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223346+0000","last_change":"2024-04-25T09:54:13.221489+0000","last_active":"2024-04-25T09:54:13.223346+0000","last_peered":"2024-04-25T09:54:13.223346+0000","last_clean":"2024-04-25T09:54:13.223346+0000","last_became_active":"2024-04-25T09:54:13.221175+0000","last_became_peered":"2024-04-25T09:54:13.221175+0000","last_unstale":"2024-04-25T09:54:13.223346+0000","last_undegraded":"2024-04-25T09:54:13.223346+0000","last_fullsized":"2024-04-25T09:54:13.223346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:09.907450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.230716+0000","last_change":"2024-04-25T09:54:12.201478+0000","last_active":"2024-04-25T09:54:13.230716+0000","last_peered":"2024-04-25T09:54:13.230716+0000","last_clean":"2024-04-25T09:54:13.230716+0000","last_became_active":"2024-04-25T09:54:12.201198+0000","last_became_peered":"2024-04-25T09:54:12.201198+0000","last_unstale":"2024-04-25T09:54:13.230716+0000","last_undegraded":"2024-04-25T09:54:13.230716+0000","last_fullsized":"2024-04-25T09:54:13.230716+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:11.280562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.222983+0000","last_change":"2024-04-25T09:54:13.218920+0000","last_active":"2024-04-25T09:54:13.222983+0000","last_peered":"2024-04-25T09:54:13.222983+0000","last_clean":"2024-04-25T09:54:13.222983+0000","last_became_active":"2024-04-25T09:54:13.218632+0000","last_became_peered":"2024-04-25T09:54:13.218632+0000","last_unstale":"2024-04-25T09:54:13.222983+0000","last_undegraded":"2024-04-25T09:54:13.222983+0000","last_fullsized":"2024-04-25T09:54:13.222983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:12.287753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.210011+0000","last_change":"2024-04-25T09:54:12.213721+0000","last_active":"2024-04-25T09:54:13.210011+0000","last_peered":"2024-04-25T09:54:13.210011+0000","last_clean":"2024-04-25T09:54:13.210011+0000","last_became_active":"2024-04-25T09:54:12.213430+0000","last_became_peered":"2024-04-25T09:54:12.213430+0000","last_unstale":"2024-04-25T09:54:13.210011+0000","last_undegraded":"2024-04-25T09:54:13.210011+0000","last_fullsized":"2024-04-25T09:54:13.210011+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:54.524536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.227709+0000","last_change":"2024-04-25T09:54:13.225021+0000","last_active":"2024-04-25T09:54:13.227709+0000","last_peered":"2024-04-25T09:54:13.227709+0000","last_clean":"2024-04-25T09:54:13.227709+0000","last_became_active":"2024-04-25T09:54:13.223986+0000","last_became_peered":"2024-04-25T09:54:13.223986+0000","last_unstale":"2024-04-25T09:54:13.227709+0000","last_undegraded":"2024-04-25T09:54:13.227709+0000","last_fullsized":"2024-04-25T09:54:13.227709+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:34.596310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.208585+0000","last_change":"2024-04-25T09:54:12.205232+0000","last_active":"2024-04-25T09:54:13.208585+0000","last_peered":"2024-04-25T09:54:13.208585+0000","last_clean":"2024-04-25T09:54:13.208585+0000","last_became_active":"2024-04-25T09:54:12.204946+0000","last_became_peered":"2024-04-25T09:54:12.204946+0000","last_unstale":"2024-04-25T09:54:13.208585+0000","last_undegraded":"2024-04-25T09:54:13.208585+0000","last_fullsized":"2024-04-25T09:54:13.208585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:51.847718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.224835+0000","last_change":"2024-04-25T09:54:13.220082+0000","last_active":"2024-04-25T09:54:13.224835+0000","last_peered":"2024-04-25T09:54:13.224835+0000","last_clean":"2024-04-25T09:54:13.224835+0000","last_became_active":"2024-04-25T09:54:13.219433+0000","last_became_peered":"2024-04-25T09:54:13.219433+0000","last_unstale":"2024-04-25T09:54:13.224835+0000","last_undegraded":"2024-04-25T09:54:13.224835+0000","last_fullsized":"2024-04-25T09:54:13.224835+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:41.458847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.209197+0000","last_change":"2024-04-25T09:54:12.212615+0000","last_active":"2024-04-25T09:54:13.209197+0000","last_peered":"2024-04-25T09:54:13.209197+0000","last_clean":"2024-04-25T09:54:13.209197+0000","last_became_active":"2024-04-25T09:54:12.212246+0000","last_became_peered":"2024-04-25T09:54:12.212246+0000","last_unstale":"2024-04-25T09:54:13.209197+0000","last_undegraded":"2024-04-25T09:54:13.209197+0000","last_fullsized":"2024-04-25T09:54:13.209197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:56.991847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.218157+0000","last_change":"2024-04-25T09:54:13.215665+0000","last_active":"2024-04-25T09:54:13.218157+0000","last_peered":"2024-04-25T09:54:13.218157+0000","last_clean":"2024-04-25T09:54:13.218157+0000","last_became_active":"2024-04-25T09:54:13.215404+0000","last_became_peered":"2024-04-25T09:54:13.215404+0000","last_unstale":"2024-04-25T09:54:13.218157+0000","last_undegraded":"2024-04-25T09:54:13.218157+0000","last_fullsized":"2024-04-25T09:54:13.218157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:38.621459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.204111+0000","last_change":"2024-04-25T09:54:12.206943+0000","last_active":"2024-04-25T09:54:13.204111+0000","last_peered":"2024-04-25T09:54:13.204111+0000","last_clean":"2024-04-25T09:54:13.204111+0000","last_became_active":"2024-04-25T09:54:12.206649+0000","last_became_peered":"2024-04-25T09:54:12.206649+0000","last_unstale":"2024-04-25T09:54:13.204111+0000","last_undegraded":"2024-04-25T09:54:13.204111+0000","last_fullsized":"2024-04-25T09:54:13.204111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:07.751815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.224558+0000","last_change":"2024-04-25T09:54:13.219359+0000","last_active":"2024-04-25T09:54:13.224558+0000","last_peered":"2024-04-25T09:54:13.224558+0000","last_clean":"2024-04-25T09:54:13.224558+0000","last_became_active":"2024-04-25T09:54:13.219016+0000","last_became_peered":"2024-04-25T09:54:13.219016+0000","last_unstale":"2024-04-25T09:54:13.224558+0000","last_undegraded":"2024-04-25T09:54:13.224558+0000","last_fullsized":"2024-04-25T09:54:13.224558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:08.759006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.236171+0000","last_change":"2024-04-25T09:54:12.208283+0000","last_active":"2024-04-25T09:54:13.236171+0000","last_peered":"2024-04-25T09:54:13.236171+0000","last_clean":"2024-04-25T09:54:13.236171+0000","last_became_active":"2024-04-25T09:54:12.208046+0000","last_became_peered":"2024-04-25T09:54:12.208046+0000","last_unstale":"2024-04-25T09:54:13.236171+0000","last_undegraded":"2024-04-25T09:54:13.236171+0000","last_fullsized":"2024-04-25T09:54:13.236171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:54.188789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.229232+0000","last_change":"2024-04-25T09:54:13.222182+0000","last_active":"2024-04-25T09:54:13.229232+0000","last_peered":"2024-04-25T09:54:13.229232+0000","last_clean":"2024-04-25T09:54:13.229232+0000","last_became_active":"2024-04-25T09:54:13.221929+0000","last_became_peered":"2024-04-25T09:54:13.221929+0000","last_unstale":"2024-04-25T09:54:13.229232+0000","last_undegraded":"2024-04-25T09:54:13.229232+0000","last_fullsized":"2024-04-25T09:54:13.229232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:07.391093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.237096+0000","last_change":"2024-04-25T09:54:12.207303+0000","last_active":"2024-04-25T09:54:13.237096+0000","last_peered":"2024-04-25T09:54:13.237096+0000","last_clean":"2024-04-25T09:54:13.237096+0000","last_became_active":"2024-04-25T09:54:12.207000+0000","last_became_peered":"2024-04-25T09:54:12.207000+0000","last_unstale":"2024-04-25T09:54:13.237096+0000","last_undegraded":"2024-04-25T09:54:13.237096+0000","last_fullsized":"2024-04-25T09:54:13.237096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:07.751815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223804+0000","last_change":"2024-04-25T09:54:13.216226+0000","last_active":"2024-04-25T09:54:13.223804+0000","last_peered":"2024-04-25T09:54:13.223804+0000","last_clean":"2024-04-25T09:54:13.223804+0000","last_became_active":"2024-04-25T09:54:13.215925+0000","last_became_peered":"2024-04-25T09:54:13.215925+0000","last_unstale":"2024-04-25T09:54:13.223804+0000","last_undegraded":"2024-04-25T09:54:13.223804+0000","last_fullsized":"2024-04-25T09:54:13.223804+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:16.483875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.235538+0000","last_change":"2024-04-25T09:54:12.210453+0000","last_active":"2024-04-25T09:54:13.235538+0000","last_peered":"2024-04-25T09:54:13.235538+0000","last_clean":"2024-04-25T09:54:13.235538+0000","last_became_active":"2024-04-25T09:54:12.209865+0000","last_became_peered":"2024-04-25T09:54:12.209865+0000","last_unstale":"2024-04-25T09:54:13.235538+0000","last_undegraded":"2024-04-25T09:54:13.235538+0000","last_fullsized":"2024-04-25T09:54:13.235538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:41.176085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226950+0000","last_change":"2024-04-25T09:54:13.221263+0000","last_active":"2024-04-25T09:54:13.226950+0000","last_peered":"2024-04-25T09:54:13.226950+0000","last_clean":"2024-04-25T09:54:13.226950+0000","last_became_active":"2024-04-25T09:54:13.220038+0000","last_became_peered":"2024-04-25T09:54:13.220038+0000","last_unstale":"2024-04-25T09:54:13.226950+0000","last_undegraded":"2024-04-25T09:54:13.226950+0000","last_fullsized":"2024-04-25T09:54:13.226950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:22.609986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.210709+0000","last_change":"2024-04-25T09:54:12.213684+0000","last_active":"2024-04-25T09:54:13.210709+0000","last_peered":"2024-04-25T09:54:13.210709+0000","last_clean":"2024-04-25T09:54:13.210709+0000","last_became_active":"2024-04-25T09:54:12.213411+0000","last_became_peered":"2024-04-25T09:54:12.213411+0000","last_unstale":"2024-04-25T09:54:13.210709+0000","last_undegraded":"2024-04-25T09:54:13.210709+0000","last_fullsized":"2024-04-25T09:54:13.210709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.462254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.219092+0000","last_change":"2024-04-25T09:54:13.213035+0000","last_active":"2024-04-25T09:54:13.219092+0000","last_peered":"2024-04-25T09:54:13.219092+0000","last_clean":"2024-04-25T09:54:13.219092+0000","last_became_active":"2024-04-25T09:54:13.212759+0000","last_became_peered":"2024-04-25T09:54:13.212759+0000","last_unstale":"2024-04-25T09:54:13.219092+0000","last_undegraded":"2024-04-25T09:54:13.219092+0000","last_fullsized":"2024-04-25T09:54:13.219092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:55.404316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.206991+0000","last_change":"2024-04-25T09:54:12.203282+0000","last_active":"2024-04-25T09:54:13.206991+0000","last_peered":"2024-04-25T09:54:13.206991+0000","last_clean":"2024-04-25T09:54:13.206991+0000","last_became_active":"2024-04-25T09:54:12.202984+0000","last_became_peered":"2024-04-25T09:54:12.202984+0000","last_unstale":"2024-04-25T09:54:13.206991+0000","last_undegraded":"2024-04-25T09:54:13.206991+0000","last_fullsized":"2024-04-25T09:54:13.206991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:50.864860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226715+0000","last_change":"2024-04-25T09:54:13.219547+0000","last_active":"2024-04-25T09:54:13.226715+0000","last_peered":"2024-04-25T09:54:13.226715+0000","last_clean":"2024-04-25T09:54:13.226715+0000","last_became_active":"2024-04-25T09:54:13.219241+0000","last_became_peered":"2024-04-25T09:54:13.219241+0000","last_unstale":"2024-04-25T09:54:13.226715+0000","last_undegraded":"2024-04-25T09:54:13.226715+0000","last_fullsized":"2024-04-25T09:54:13.226715+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:53.335804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.232920+0000","last_change":"2024-04-25T09:54:12.200217+0000","last_active":"2024-04-25T09:54:13.232920+0000","last_peered":"2024-04-25T09:54:13.232920+0000","last_clean":"2024-04-25T09:54:13.232920+0000","last_became_active":"2024-04-25T09:54:12.199910+0000","last_became_peered":"2024-04-25T09:54:12.199910+0000","last_unstale":"2024-04-25T09:54:13.232920+0000","last_undegraded":"2024-04-25T09:54:13.232920+0000","last_fullsized":"2024-04-25T09:54:13.232920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.462254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.234110+0000","last_change":"2024-04-25T09:54:13.227463+0000","last_active":"2024-04-25T09:54:13.234110+0000","last_peered":"2024-04-25T09:54:13.234110+0000","last_clean":"2024-04-25T09:54:13.234110+0000","last_became_active":"2024-04-25T09:54:13.226259+0000","last_became_peered":"2024-04-25T09:54:13.226259+0000","last_unstale":"2024-04-25T09:54:13.234110+0000","last_undegraded":"2024-04-25T09:54:13.234110+0000","last_fullsized":"2024-04-25T09:54:13.234110+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:54.136097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.231798+0000","last_change":"2024-04-25T09:54:12.205548+0000","last_active":"2024-04-25T09:54:13.231798+0000","last_peered":"2024-04-25T09:54:13.231798+0000","last_clean":"2024-04-25T09:54:13.231798+0000","last_became_active":"2024-04-25T09:54:12.204843+0000","last_became_peered":"2024-04-25T09:54:12.204843+0000","last_unstale":"2024-04-25T09:54:13.231798+0000","last_undegraded":"2024-04-25T09:54:13.231798+0000","last_fullsized":"2024-04-25T09:54:13.231798+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:56.991847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223212+0000","last_change":"2024-04-25T09:54:13.219615+0000","last_active":"2024-04-25T09:54:13.223212+0000","last_peered":"2024-04-25T09:54:13.223212+0000","last_clean":"2024-04-25T09:54:13.223212+0000","last_became_active":"2024-04-25T09:54:13.218526+0000","last_became_peered":"2024-04-25T09:54:13.218526+0000","last_unstale":"2024-04-25T09:54:13.223212+0000","last_undegraded":"2024-04-25T09:54:13.223212+0000","last_fullsized":"2024-04-25T09:54:13.223212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:34.596310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.232608+0000","last_change":"2024-04-25T09:54:12.205130+0000","last_active":"2024-04-25T09:54:13.232608+0000","last_peered":"2024-04-25T09:54:13.232608+0000","last_clean":"2024-04-25T09:54:13.232608+0000","last_became_active":"2024-04-25T09:54:12.204593+0000","last_became_peered":"2024-04-25T09:54:12.204593+0000","last_unstale":"2024-04-25T09:54:13.232608+0000","last_undegraded":"2024-04-25T09:54:13.232608+0000","last_fullsized":"2024-04-25T09:54:13.232608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:37.998401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.223050+0000","last_change":"2024-04-25T09:54:13.219148+0000","last_active":"2024-04-25T09:54:13.223050+0000","last_peered":"2024-04-25T09:54:13.223050+0000","last_clean":"2024-04-25T09:54:13.223050+0000","last_became_active":"2024-04-25T09:54:13.218823+0000","last_became_peered":"2024-04-25T09:54:13.218823+0000","last_unstale":"2024-04-25T09:54:13.223050+0000","last_undegraded":"2024-04-25T09:54:13.223050+0000","last_fullsized":"2024-04-25T09:54:13.223050+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:01.620004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.231565+0000","last_change":"2024-04-25T09:54:12.202150+0000","last_active":"2024-04-25T09:54:13.231565+0000","last_peered":"2024-04-25T09:54:13.231565+0000","last_clean":"2024-04-25T09:54:13.231565+0000","last_became_active":"2024-04-25T09:54:12.201835+0000","last_became_peered":"2024-04-25T09:54:12.201835+0000","last_unstale":"2024-04-25T09:54:13.231565+0000","last_undegraded":"2024-04-25T09:54:13.231565+0000","last_fullsized":"2024-04-25T09:54:13.231565+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:13.980512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226594+0000","last_change":"2024-04-25T09:54:13.220716+0000","last_active":"2024-04-25T09:54:13.226594+0000","last_peered":"2024-04-25T09:54:13.226594+0000","last_clean":"2024-04-25T09:54:13.226594+0000","last_became_active":"2024-04-25T09:54:13.220402+0000","last_became_peered":"2024-04-25T09:54:13.220402+0000","last_unstale":"2024-04-25T09:54:13.226594+0000","last_undegraded":"2024-04-25T09:54:13.226594+0000","last_fullsized":"2024-04-25T09:54:13.226594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:58.139201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.212311+0000","last_change":"2024-04-25T09:54:12.207715+0000","last_active":"2024-04-25T09:54:13.212311+0000","last_peered":"2024-04-25T09:54:13.212311+0000","last_clean":"2024-04-25T09:54:13.212311+0000","last_became_active":"2024-04-25T09:54:12.207411+0000","last_became_peered":"2024-04-25T09:54:12.207411+0000","last_unstale":"2024-04-25T09:54:13.212311+0000","last_undegraded":"2024-04-25T09:54:13.212311+0000","last_fullsized":"2024-04-25T09:54:13.212311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:00.612813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.218180+0000","last_change":"2024-04-25T09:54:13.215019+0000","last_active":"2024-04-25T09:54:13.218180+0000","last_peered":"2024-04-25T09:54:13.218180+0000","last_clean":"2024-04-25T09:54:13.218180+0000","last_became_active":"2024-04-25T09:54:13.212810+0000","last_became_peered":"2024-04-25T09:54:13.212810+0000","last_unstale":"2024-04-25T09:54:13.218180+0000","last_undegraded":"2024-04-25T09:54:13.218180+0000","last_fullsized":"2024-04-25T09:54:13.218180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:08.759006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.234470+0000","last_change":"2024-04-25T09:54:12.210184+0000","last_active":"2024-04-25T09:54:13.234470+0000","last_peered":"2024-04-25T09:54:13.234470+0000","last_clean":"2024-04-25T09:54:13.234470+0000","last_became_active":"2024-04-25T09:54:12.209821+0000","last_became_peered":"2024-04-25T09:54:12.209821+0000","last_unstale":"2024-04-25T09:54:13.234470+0000","last_undegraded":"2024-04-25T09:54:13.234470+0000","last_fullsized":"2024-04-25T09:54:13.234470+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:50.097937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.224608+0000","last_change":"2024-04-25T09:54:13.218521+0000","last_active":"2024-04-25T09:54:13.224608+0000","last_peered":"2024-04-25T09:54:13.224608+0000","last_clean":"2024-04-25T09:54:13.224608+0000","last_became_active":"2024-04-25T09:54:13.218153+0000","last_became_peered":"2024-04-25T09:54:13.218153+0000","last_unstale":"2024-04-25T09:54:13.224608+0000","last_undegraded":"2024-04-25T09:54:13.224608+0000","last_fullsized":"2024-04-25T09:54:13.224608+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:57.999037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.204608+0000","last_change":"2024-04-25T09:54:12.202685+0000","last_active":"2024-04-25T09:54:13.204608+0000","last_peered":"2024-04-25T09:54:13.204608+0000","last_clean":"2024-04-25T09:54:13.204608+0000","last_became_active":"2024-04-25T09:54:12.201183+0000","last_became_peered":"2024-04-25T09:54:12.201183+0000","last_unstale":"2024-04-25T09:54:13.204608+0000","last_undegraded":"2024-04-25T09:54:13.204608+0000","last_fullsized":"2024-04-25T09:54:13.204608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:58.982383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.216815+0000","last_change":"2024-04-25T09:54:13.213472+0000","last_active":"2024-04-25T09:54:13.216815+0000","last_peered":"2024-04-25T09:54:13.216815+0000","last_clean":"2024-04-25T09:54:13.216815+0000","last_became_active":"2024-04-25T09:54:13.212692+0000","last_became_peered":"2024-04-25T09:54:13.212692+0000","last_unstale":"2024-04-25T09:54:13.216815+0000","last_undegraded":"2024-04-25T09:54:13.216815+0000","last_fullsized":"2024-04-25T09:54:13.216815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:22.609986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.201627+0000","last_change":"2024-04-25T09:54:12.204289+0000","last_active":"2024-04-25T09:54:13.201627+0000","last_peered":"2024-04-25T09:54:13.201627+0000","last_clean":"2024-04-25T09:54:13.201627+0000","last_became_active":"2024-04-25T09:54:12.203980+0000","last_became_peered":"2024-04-25T09:54:12.203980+0000","last_unstale":"2024-04-25T09:54:13.201627+0000","last_undegraded":"2024-04-25T09:54:13.201627+0000","last_fullsized":"2024-04-25T09:54:13.201627+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:00.612813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.227687+0000","last_change":"2024-04-25T09:54:13.224318+0000","last_active":"2024-04-25T09:54:13.227687+0000","last_peered":"2024-04-25T09:54:13.227687+0000","last_clean":"2024-04-25T09:54:13.227687+0000","last_became_active":"2024-04-25T09:54:13.220621+0000","last_became_peered":"2024-04-25T09:54:13.220621+0000","last_unstale":"2024-04-25T09:54:13.227687+0000","last_undegraded":"2024-04-25T09:54:13.227687+0000","last_fullsized":"2024-04-25T09:54:13.227687+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:10.890138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.204775+0000","last_change":"2024-04-25T09:54:12.206903+0000","last_active":"2024-04-25T09:54:13.204775+0000","last_peered":"2024-04-25T09:54:13.204775+0000","last_clean":"2024-04-25T09:54:13.204775+0000","last_became_active":"2024-04-25T09:54:12.206621+0000","last_became_peered":"2024-04-25T09:54:12.206621+0000","last_unstale":"2024-04-25T09:54:13.204775+0000","last_undegraded":"2024-04-25T09:54:13.204775+0000","last_fullsized":"2024-04-25T09:54:13.204775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:59.099186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226965+0000","last_change":"2024-04-25T09:54:13.224904+0000","last_active":"2024-04-25T09:54:13.226965+0000","last_peered":"2024-04-25T09:54:13.226965+0000","last_clean":"2024-04-25T09:54:13.226965+0000","last_became_active":"2024-04-25T09:54:13.223777+0000","last_became_peered":"2024-04-25T09:54:13.223777+0000","last_unstale":"2024-04-25T09:54:13.226965+0000","last_undegraded":"2024-04-25T09:54:13.226965+0000","last_fullsized":"2024-04-25T09:54:13.226965+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:16.483875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.236024+0000","last_change":"2024-04-25T09:54:12.208301+0000","last_active":"2024-04-25T09:54:13.236024+0000","last_peered":"2024-04-25T09:54:13.236024+0000","last_clean":"2024-04-25T09:54:13.236024+0000","last_became_active":"2024-04-25T09:54:12.208016+0000","last_became_peered":"2024-04-25T09:54:12.208016+0000","last_unstale":"2024-04-25T09:54:13.236024+0000","last_undegraded":"2024-04-25T09:54:13.236024+0000","last_fullsized":"2024-04-25T09:54:13.236024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:41.176085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.215789+0000","last_change":"2024-04-25T09:54:13.212052+0000","last_active":"2024-04-25T09:54:13.215789+0000","last_peered":"2024-04-25T09:54:13.215789+0000","last_clean":"2024-04-25T09:54:13.215789+0000","last_became_active":"2024-04-25T09:54:13.211556+0000","last_became_peered":"2024-04-25T09:54:13.211556+0000","last_unstale":"2024-04-25T09:54:13.215789+0000","last_undegraded":"2024-04-25T09:54:13.215789+0000","last_fullsized":"2024-04-25T09:54:13.215789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:42.183276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.205389+0000","last_change":"2024-04-25T09:54:12.205196+0000","last_active":"2024-04-25T09:54:13.205389+0000","last_peered":"2024-04-25T09:54:13.205389+0000","last_clean":"2024-04-25T09:54:13.205389+0000","last_became_active":"2024-04-25T09:54:12.204844+0000","last_became_peered":"2024-04-25T09:54:12.204844+0000","last_unstale":"2024-04-25T09:54:13.205389+0000","last_undegraded":"2024-04-25T09:54:13.205389+0000","last_fullsized":"2024-04-25T09:54:13.205389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:37.753205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.227743+0000","last_change":"2024-04-25T09:54:13.221021+0000","last_active":"2024-04-25T09:54:13.227743+0000","last_peered":"2024-04-25T09:54:13.227743+0000","last_clean":"2024-04-25T09:54:13.227743+0000","last_became_active":"2024-04-25T09:54:13.220377+0000","last_became_peered":"2024-04-25T09:54:13.220377+0000","last_unstale":"2024-04-25T09:54:13.227743+0000","last_undegraded":"2024-04-25T09:54:13.227743+0000","last_fullsized":"2024-04-25T09:54:13.227743+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:38.621459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.211179+0000","last_change":"2024-04-25T09:54:12.208556+0000","last_active":"2024-04-25T09:54:13.211179+0000","last_peered":"2024-04-25T09:54:13.211179+0000","last_clean":"2024-04-25T09:54:13.211179+0000","last_became_active":"2024-04-25T09:54:12.208050+0000","last_became_peered":"2024-04-25T09:54:12.208050+0000","last_unstale":"2024-04-25T09:54:13.211179+0000","last_undegraded":"2024-04-25T09:54:13.211179+0000","last_fullsized":"2024-04-25T09:54:13.211179+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:00.612813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.231310+0000","last_change":"2024-04-25T09:54:13.228463+0000","last_active":"2024-04-25T09:54:13.231310+0000","last_peered":"2024-04-25T09:54:13.231310+0000","last_clean":"2024-04-25T09:54:13.231310+0000","last_became_active":"2024-04-25T09:54:13.228160+0000","last_became_peered":"2024-04-25T09:54:13.228160+0000","last_unstale":"2024-04-25T09:54:13.231310+0000","last_undegraded":"2024-04-25T09:54:13.231310+0000","last_fullsized":"2024-04-25T09:54:13.231310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:53.335804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.211403+0000","last_change":"2024-04-25T09:54:12.213001+0000","last_active":"2024-04-25T09:54:13.211403+0000","last_peered":"2024-04-25T09:54:13.211403+0000","last_clean":"2024-04-25T09:54:13.211403+0000","last_became_active":"2024-04-25T09:54:12.212769+0000","last_became_peered":"2024-04-25T09:54:12.212769+0000","last_unstale":"2024-04-25T09:54:13.211403+0000","last_undegraded":"2024-04-25T09:54:13.211403+0000","last_fullsized":"2024-04-25T09:54:13.211403+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:06.392698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.226126+0000","last_change":"2024-04-25T09:54:13.220238+0000","last_active":"2024-04-25T09:54:13.226126+0000","last_peered":"2024-04-25T09:54:13.226126+0000","last_clean":"2024-04-25T09:54:13.226126+0000","last_became_active":"2024-04-25T09:54:13.219930+0000","last_became_peered":"2024-04-25T09:54:13.219930+0000","last_unstale":"2024-04-25T09:54:13.226126+0000","last_undegraded":"2024-04-25T09:54:13.226126+0000","last_fullsized":"2024-04-25T09:54:13.226126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:07.399889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.210995+0000","last_change":"2024-04-25T09:54:12.210026+0000","last_active":"2024-04-25T09:54:13.210995+0000","last_peered":"2024-04-25T09:54:13.210995+0000","last_clean":"2024-04-25T09:54:13.210995+0000","last_became_active":"2024-04-25T09:54:12.209775+0000","last_became_peered":"2024-04-25T09:54:12.209775+0000","last_unstale":"2024-04-25T09:54:13.210995+0000","last_undegraded":"2024-04-25T09:54:13.210995+0000","last_fullsized":"2024-04-25T09:54:13.210995+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:49.356287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.230178+0000","last_change":"2024-04-25T09:54:13.228303+0000","last_active":"2024-04-25T09:54:13.230178+0000","last_peered":"2024-04-25T09:54:13.230178+0000","last_clean":"2024-04-25T09:54:13.230178+0000","last_became_active":"2024-04-25T09:54:13.227969+0000","last_became_peered":"2024-04-25T09:54:13.227969+0000","last_unstale":"2024-04-25T09:54:13.230178+0000","last_undegraded":"2024-04-25T09:54:13.230178+0000","last_fullsized":"2024-04-25T09:54:13.230178+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.261566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.208824+0000","last_change":"2024-04-25T09:54:12.201019+0000","last_active":"2024-04-25T09:54:13.208824+0000","last_peered":"2024-04-25T09:54:13.208824+0000","last_clean":"2024-04-25T09:54:13.208824+0000","last_became_active":"2024-04-25T09:54:12.200636+0000","last_became_peered":"2024-04-25T09:54:12.200636+0000","last_unstale":"2024-04-25T09:54:13.208824+0000","last_undegraded":"2024-04-25T09:54:13.208824+0000","last_fullsized":"2024-04-25T09:54:13.208824+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:05.905982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.214901+0000","last_change":"2024-04-25T09:54:13.211800+0000","last_active":"2024-04-25T09:54:13.214901+0000","last_peered":"2024-04-25T09:54:13.214901+0000","last_clean":"2024-04-25T09:54:13.214901+0000","last_became_active":"2024-04-25T09:54:13.211537+0000","last_became_peered":"2024-04-25T09:54:13.211537+0000","last_unstale":"2024-04-25T09:54:13.214901+0000","last_undegraded":"2024-04-25T09:54:13.214901+0000","last_fullsized":"2024-04-25T09:54:13.214901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:55.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.207845+0000","last_change":"2024-04-25T09:54:12.206750+0000","last_active":"2024-04-25T09:54:13.207845+0000","last_peered":"2024-04-25T09:54:13.207845+0000","last_clean":"2024-04-25T09:54:13.207845+0000","last_became_active":"2024-04-25T09:54:12.206406+0000","last_became_peered":"2024-04-25T09:54:12.206406+0000","last_unstale":"2024-04-25T09:54:13.207845+0000","last_undegraded":"2024-04-25T09:54:13.207845+0000","last_fullsized":"2024-04-25T09:54:13.207845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:54.397125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.232424+0000","last_change":"2024-04-25T09:54:13.230179+0000","last_active":"2024-04-25T09:54:13.232424+0000","last_peered":"2024-04-25T09:54:13.232424+0000","last_clean":"2024-04-25T09:54:13.232424+0000","last_became_active":"2024-04-25T09:54:13.229903+0000","last_became_peered":"2024-04-25T09:54:13.229903+0000","last_unstale":"2024-04-25T09:54:13.232424+0000","last_undegraded":"2024-04-25T09:54:13.232424+0000","last_fullsized":"2024-04-25T09:54:13.232424+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:12.184666+0000","last_clean_scrub_stamp":"2024-04-25T09:54:12.184666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:48.938736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.202818+0000","last_change":"2024-04-25T09:54:12.207714+0000","last_active":"2024-04-25T09:54:13.202818+0000","last_peered":"2024-04-25T09:54:13.202818+0000","last_clean":"2024-04-25T09:54:13.202818+0000","last_became_active":"2024-04-25T09:54:12.207420+0000","last_became_peered":"2024-04-25T09:54:12.207420+0000","last_unstale":"2024-04-25T09:54:13.202818+0000","last_undegraded":"2024-04-25T09:54:13.202818+0000","last_fullsized":"2024-04-25T09:54:13.202818+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.177475+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.177475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:06.392698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:13.201210+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:54:13.201210+0000","last_peered":"2024-04-25T09:54:13.201210+0000","last_clean":"2024-04-25T09:54:13.201210+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:54:13.201210+0000","last_undegraded":"2024-04-25T09:54:13.201210+0000","last_fullsized":"2024-04-25T09:54:13.201210+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":183932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1024000,"data_stored":773730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:54:16.567 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:54:16.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:54:16.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.709+0000 7f0012cc4640 1 Processor -- start 2024-04-25T09:54:16.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.709+0000 7f0012cc4640 1 -- start start 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.710+0000 7f0012cc4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f000c0c6010 0x7f000c0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.710+0000 7f0012cc4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f000c0cd520 0x7f000c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.710+0000 7f0012cc4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000c0c6940 con 0x7f000c15a9f0 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.710+0000 7f0012cc4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000c0b0020 con 0x7f000c0c6010 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.710+0000 7f0012cc4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0cb5a0 con 0x7f000c0cd520 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.710+0000 7f0010a60640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f000c15a9f0 0x7f000c0cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38844/0 (socket says 172.21.15.19:38844) 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.710+0000 7f0010a60640 1 -- 172.21.15.19:0/3966136747 learned_addr learned my addr 172.21.15.19:0/3966136747 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f0011261640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f000c0cd520 0x7f000c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000bfff640 1 --2- 172.21.15.19:0/3966136747 >> v2:172.21.15.112:3300/0 conn(0x7f000c0c6010 0x7f000c0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982772193 0 0) 0x7f0004001490 con 0x7f000c15a9f0 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff4003520 con 0x7f000c15a9f0 2024-04-25T09:54:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4210856164 0 0) 0x7efff4003520 con 0x7f000c15a9f0 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f000c0cd520 msgr2=0x7f000c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f000c0cd520 0x7f000c0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 >> v2:172.21.15.112:3300/0 conn(0x7f000c0c6010 msgr2=0x7f000c0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 --2- 172.21.15.19:0/3966136747 >> v2:172.21.15.112:3300/0 conn(0x7f000c0c6010 0x7f000c0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c15add0 con 0x7f000c15a9f0 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0004001bc0 con 0x7f000c15a9f0 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000bfff640 1 --2- 172.21.15.19:0/3966136747 >> v2:172.21.15.112:3300/0 conn(0x7f000c0c6010 0x7f000c0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 >> v1:172.21.15.19:6789/0 conn(0x7f000c15a9f0 legacy=0x7f000c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efff40039b0 0x7efff4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40043a0 0x7efff4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efff4004ef0 0x7efff400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0cb5a0 con 0x7efff40043a0 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff400bf00 con 0x7efff4004ef0 2024-04-25T09:54:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff400c080 con 0x7efff40039b0 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f0011261640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f000c0cd520 0x7f000c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f000bfff640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40043a0 0x7efff4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.711+0000 7f0010a60640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efff40039b0 0x7efff4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.712+0000 7f0011261640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efff4004ef0 0x7efff400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.712+0000 7f000bfff640 1 -- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efff40039b0 msgr2=0x7efff4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.712+0000 7f000bfff640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efff40039b0 0x7efff4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.712+0000 7f000bfff640 1 -- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efff4004ef0 msgr2=0x7efff400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.712+0000 7f000bfff640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efff4004ef0 0x7efff400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.712+0000 7f000bfff640 1 -- 172.21.15.19:0/3966136747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff4003520 con 0x7efff40043a0 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.712+0000 7f0011261640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efff4004ef0 0x7efff400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.712+0000 7f000bfff640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40043a0 0x7efff4004880 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0000009c90 tx=0x7efff4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a67e68c2d04a1fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.712+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000036070 con 0x7efff40043a0 2024-04-25T09:54:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.713+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0000031400 con 0x7efff40043a0 2024-04-25T09:54:16.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.713+0000 7f000b7fe640 1 -- 172.21.15.19:0/3966136747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000009860 con 0x7efff40043a0 2024-04-25T09:54:16.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.713+0000 7f0012cc4640 1 -- 172.21.15.19:0/3966136747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40043a0 msgr2=0x7efff4004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.713+0000 7f0012cc4640 1 --2- 172.21.15.19:0/3966136747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40043a0 0x7efff4004880 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0000009c90 tx=0x7efff4005830 comp rx=0 tx=0).stop 2024-04-25T09:54:16.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.713+0000 7f0011261640 1 -- 172.21.15.19:0/3966136747 reap_dead start 2024-04-25T09:54:16.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.713+0000 7f0012cc4640 1 -- 172.21.15.19:0/3966136747 shutdown_connections 2024-04-25T09:54:16.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.713+0000 7f0012cc4640 1 -- 172.21.15.19:0/3966136747 >> 172.21.15.19:0/3966136747 conn(0x7f000c0b2390 msgr2=0x7f000c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.713+0000 7f0012cc4640 1 -- 172.21.15.19:0/3966136747 shutdown_connections 2024-04-25T09:54:16.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.713+0000 7f0012cc4640 1 -- 172.21.15.19:0/3966136747 wait complete. 2024-04-25T09:54:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.714+0000 7f0012cc4640 1 Processor -- start 2024-04-25T09:54:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.714+0000 7f0012cc4640 1 -- start start 2024-04-25T09:54:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.715+0000 7f0012cc4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efff40043a0 0x7f000c1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.715+0000 7f0012cc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f000c0ae150 0x7f000c0ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.715+0000 7f0012cc4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000c0c82d0 0x7f000c0c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.715+0000 7f0012cc4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0cb230 con 0x7f000c0ae150 2024-04-25T09:54:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.715+0000 7f0012cc4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0caf50 con 0x7f000c0c82d0 2024-04-25T09:54:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.715+0000 7f0012cc4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c06abe0 con 0x7efff40043a0 2024-04-25T09:54:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.715+0000 7f000bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f000c0ae150 0x7f000c0ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.715+0000 7f000bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f000c0ae150 0x7f000c0ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53476/0 (socket says 172.21.15.19:53476) 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.715+0000 7f000bfff640 1 -- 172.21.15.19:0/3513153613 learned_addr learned my addr 172.21.15.19:0/3513153613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.715+0000 7f0011261640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000c0c82d0 0x7f000c0c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.715+0000 7f0010a60640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efff40043a0 0x7f000c1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.716+0000 7f000bfff640 1 -- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efff40043a0 msgr2=0x7f000c1533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.716+0000 7f000bfff640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efff40043a0 0x7f000c1533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.716+0000 7f000bfff640 1 -- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000c0c82d0 msgr2=0x7f000c0c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.716+0000 7f000bfff640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000c0c82d0 0x7f000c0c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.716+0000 7f000bfff640 1 -- 172.21.15.19:0/3513153613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c0c9060 con 0x7f000c0ae150 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.716+0000 7f000bfff640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f000c0ae150 0x7f000c0ae5d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0000009db0 tx=0x7f0000004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.716+0000 7f00097fa640 1 -- 172.21.15.19:0/3513153613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000040020 con 0x7f000c0ae150 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.716+0000 7f00097fa640 1 -- 172.21.15.19:0/3513153613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000003b680 con 0x7f000c0ae150 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.716+0000 7f0012cc4640 1 -- 172.21.15.19:0/3513153613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000c153a50 con 0x7f000c0ae150 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.716+0000 7f00097fa640 1 -- 172.21.15.19:0/3513153613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000036040 con 0x7f000c0ae150 2024-04-25T09:54:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.716+0000 7f0012cc4640 1 -- 172.21.15.19:0/3513153613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c154010 con 0x7f000c0ae150 2024-04-25T09:54:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.717+0000 7f0012cc4640 1 -- 172.21.15.19:0/3513153613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000c0aeb10 con 0x7f000c0ae150 2024-04-25T09:54:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.718+0000 7f00097fa640 1 -- 172.21.15.19:0/3513153613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0000043020 con 0x7f000c0ae150 2024-04-25T09:54:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.718+0000 7f00097fa640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efff806b4d0 0x7efff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.718+0000 7f00097fa640 1 -- 172.21.15.19:0/3513153613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f00000aa8e0 con 0x7f000c0ae150 2024-04-25T09:54:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.718+0000 7f0010a60640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efff806b4d0 0x7efff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.719+0000 7f0010a60640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efff806b4d0 0x7efff806d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0004004f20 tx=0x7f0004006040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.722+0000 7f00097fa640 1 -- 172.21.15.19:0/3513153613 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f000004a030 con 0x7f000c0ae150 2024-04-25T09:54:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.970+0000 7f0012cc4640 1 -- 172.21.15.19:0/3513153613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f000c06cc80 con 0x7f000c0ae150 2024-04-25T09:54:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.972+0000 7f00097fa640 1 -- 172.21.15.19:0/3513153613 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0000078ef0 con 0x7f000c0ae150 2024-04-25T09:54:16.975 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T09:54:16.975 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T09:54:16.975 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T09:54:16.975 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T09:54:16.975 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:54:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T09:54:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 -- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efff806b4d0 msgr2=0x7efff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efff806b4d0 0x7efff806d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0004004f20 tx=0x7f0004006040 comp rx=0 tx=0).stop 2024-04-25T09:54:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 -- 172.21.15.19:0/3513153613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f000c0ae150 msgr2=0x7f000c0ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f000c0ae150 0x7f000c0ae5d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0000009db0 tx=0x7f0000004730 comp rx=0 tx=0).stop 2024-04-25T09:54:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 -- 172.21.15.19:0/3513153613 shutdown_connections 2024-04-25T09:54:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efff806b4d0 0x7efff806d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000c0c82d0 0x7f000c0c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f000c0ae150 0x7f000c0ae5d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 --2- 172.21.15.19:0/3513153613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efff40043a0 0x7f000c1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 -- 172.21.15.19:0/3513153613 >> 172.21.15.19:0/3513153613 conn(0x7f000c0b2390 msgr2=0x7f000c0c5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 -- 172.21.15.19:0/3513153613 shutdown_connections 2024-04-25T09:54:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:16.975+0000 7f0012cc4640 1 -- 172.21.15.19:0/3513153613 wait complete. 2024-04-25T09:54:16.988 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.087+0000 7f81d755c640 1 Processor -- start 2024-04-25T09:54:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.087+0000 7f81d755c640 1 -- start start 2024-04-25T09:54:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d755c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f81d015dc20 0x7f81d01616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d755c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81d00b93a0 0x7f81d00ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d755c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d015e000 con 0x7f81d00cec20 2024-04-25T09:54:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d755c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d00b6bb0 con 0x7f81d015dc20 2024-04-25T09:54:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d755c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00b6030 con 0x7f81d00b93a0 2024-04-25T09:54:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d4af7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81d00b93a0 0x7f81d00ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d52f8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f81d015dc20 0x7f81d01616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d5af9640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f81d00cec20 0x7f81d006c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:51892/0 (socket says 172.21.15.19:51892) 2024-04-25T09:54:17.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d52f8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f81d015dc20 0x7f81d01616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:51280/0 (socket says 172.21.15.19:51280) 2024-04-25T09:54:17.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d5af9640 1 -- 172.21.15.19:0/216329056 learned_addr learned my addr 172.21.15.19:0/216329056 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:17.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d4af7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81d00b93a0 0x7f81d00ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:48616/0 (socket says 172.21.15.19:48616) 2024-04-25T09:54:17.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d52f8640 1 -- 172.21.15.19:0/216329056 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81d00b93a0 msgr2=0x7f81d00ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d52f8640 1 --2- 172.21.15.19:0/216329056 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81d00b93a0 0x7f81d00ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d52f8640 1 -- 172.21.15.19:0/216329056 >> v1:172.21.15.112:6789/0 conn(0x7f81d00cec20 legacy=0x7f81d006c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.088+0000 7f81d52f8640 1 -- 172.21.15.19:0/216329056 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d0161de0 con 0x7f81d015dc20 2024-04-25T09:54:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.089+0000 7f81d52f8640 1 --2- 172.21.15.19:0/216329056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d015dc20 0x7f81d01616b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0097a0 tx=0x7f81cc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d33de75abd4bda7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.089+0000 7f81c7fff640 1 -- 172.21.15.19:0/216329056 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc037070 con 0x7f81d015dc20 2024-04-25T09:54:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.089+0000 7f81c7fff640 1 -- 172.21.15.19:0/216329056 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc032440 con 0x7f81d015dc20 2024-04-25T09:54:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.089+0000 7f81c7fff640 1 -- 172.21.15.19:0/216329056 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc032810 con 0x7f81d015dc20 2024-04-25T09:54:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.090+0000 7f81d755c640 1 -- 172.21.15.19:0/216329056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d015dc20 msgr2=0x7f81d01616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.090+0000 7f81d755c640 1 --2- 172.21.15.19:0/216329056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d015dc20 0x7f81d01616b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0097a0 tx=0x7f81cc02a180 comp rx=0 tx=0).stop 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.090+0000 7f81d755c640 1 -- 172.21.15.19:0/216329056 shutdown_connections 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.090+0000 7f81d755c640 1 --2- 172.21.15.19:0/216329056 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81d00b93a0 0x7f81d00ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.090+0000 7f81d755c640 1 --2- 172.21.15.19:0/216329056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d015dc20 0x7f81d01616b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.090+0000 7f81d755c640 1 -- 172.21.15.19:0/216329056 >> 172.21.15.19:0/216329056 conn(0x7f81d00d0490 msgr2=0x7f81d00d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.090+0000 7f81d755c640 1 -- 172.21.15.19:0/216329056 shutdown_connections 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.090+0000 7f81d755c640 1 -- 172.21.15.19:0/216329056 wait complete. 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.091+0000 7f81d755c640 1 Processor -- start 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.091+0000 7f81d755c640 1 -- start start 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d755c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d00b93a0 0x7f81d015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d755c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d00cec20 0x7f81d01507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d755c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81d015dc20 0x7f81d00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d755c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00d3520 con 0x7f81d00b93a0 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d755c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d01623a0 con 0x7f81d00cec20 2024-04-25T09:54:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d755c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00d36a0 con 0x7f81d015dc20 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d52f8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d00b93a0 0x7f81d015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d52f8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d00b93a0 0x7f81d015eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53486/0 (socket says 172.21.15.19:53486) 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d52f8640 1 -- 172.21.15.19:0/3275761794 learned_addr learned my addr 172.21.15.19:0/3275761794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d5af9640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81d015dc20 0x7f81d00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d4af7640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d00cec20 0x7f81d01507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d52f8640 1 -- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81d015dc20 msgr2=0x7f81d00b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d52f8640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81d015dc20 0x7f81d00b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d52f8640 1 -- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d00cec20 msgr2=0x7f81d01507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d52f8640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d00cec20 0x7f81d01507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.092+0000 7f81d52f8640 1 -- 172.21.15.19:0/3275761794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d015f4b0 con 0x7f81d00b93a0 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.093+0000 7f81d52f8640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d00b93a0 0x7f81d015eb80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f81cc035040 tx=0x7f81cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.093+0000 7f81c5ffb640 1 -- 172.21.15.19:0/3275761794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc037040 con 0x7f81d00b93a0 2024-04-25T09:54:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.093+0000 7f81c5ffb640 1 -- 172.21.15.19:0/3275761794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc030500 con 0x7f81d00b93a0 2024-04-25T09:54:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.093+0000 7f81c5ffb640 1 -- 172.21.15.19:0/3275761794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc030910 con 0x7f81d00b93a0 2024-04-25T09:54:17.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.093+0000 7f81d755c640 1 -- 172.21.15.19:0/3275761794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d01f3ab0 con 0x7f81d00b93a0 2024-04-25T09:54:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.093+0000 7f81d755c640 1 -- 172.21.15.19:0/3275761794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d01f3790 con 0x7f81d00b93a0 2024-04-25T09:54:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.094+0000 7f81d755c640 1 -- 172.21.15.19:0/3275761794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d0073990 con 0x7f81d00b93a0 2024-04-25T09:54:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.094+0000 7f81c5ffb640 1 -- 172.21.15.19:0/3275761794 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81cc044020 con 0x7f81d00b93a0 2024-04-25T09:54:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.095+0000 7f81c5ffb640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81b406b4d0 0x7f81b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.095+0000 7f81c5ffb640 1 -- 172.21.15.19:0/3275761794 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81cc0aa4e0 con 0x7f81d00b93a0 2024-04-25T09:54:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.095+0000 7f81d4af7640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81b406b4d0 0x7f81b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.096+0000 7f81d4af7640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81b406b4d0 0x7f81b406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f81d01e84c0 tx=0x7f81c0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.099+0000 7f81c5ffb640 1 -- 172.21.15.19:0/3275761794 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81cc078b80 con 0x7f81d00b93a0 2024-04-25T09:54:17.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.320+0000 7f81d755c640 1 -- 172.21.15.19:0/3275761794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81d00d1720 con 0x7f81d00b93a0 2024-04-25T09:54:17.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.321+0000 7f81c5ffb640 1 -- 172.21.15.19:0/3275761794 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f81cc07cc20 con 0x7f81d00b93a0 2024-04-25T09:54:17.323 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:54:17.324 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3845932448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3845932448},{"type":"v1","addr":"172.21.15.112:6836","nonce":3845932448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5122,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1655535300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1655535300},{"type":"v1","addr":"172.21.15.19:6834","nonce":1655535300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/165157787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":165157787},{"type":"v1","addr":"172.21.15.112:6837","nonce":165157787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:12.554449+0000","modified":"2024-04-25T09:54:14.571034+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3017773710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3017773710},{"type":"v1","addr":"172.21.15.19:6835","nonce":3017773710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:54:17.324 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 16 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 -- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81b406b4d0 msgr2=0x7f81b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81b406b4d0 0x7f81b406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f81d01e84c0 tx=0x7f81c0009190 comp rx=0 tx=0).stop 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 -- 172.21.15.19:0/3275761794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d00b93a0 msgr2=0x7f81d015eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d00b93a0 0x7f81d015eb80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f81cc035040 tx=0x7f81cc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 -- 172.21.15.19:0/3275761794 shutdown_connections 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81b406b4d0 0x7f81b406d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81d015dc20 0x7f81d00b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81d00cec20 0x7f81d01507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 --2- 172.21.15.19:0/3275761794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d00b93a0 0x7f81d015eb80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 -- 172.21.15.19:0/3275761794 >> 172.21.15.19:0/3275761794 conn(0x7f81d00d0490 msgr2=0x7f81d006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 -- 172.21.15.19:0/3275761794 shutdown_connections 2024-04-25T09:54:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.323+0000 7f81d755c640 1 -- 172.21.15.19:0/3275761794 wait complete. 2024-04-25T09:54:17.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:54:12.554449+0000', 'modified': '2024-04-25T09:54:14.571034+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5286}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5286': {'gid': 5286, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/3017773710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 3017773710}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3017773710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:54:17.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:54:17.739 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:54:17.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.847+0000 7f8291f34640 1 Processor -- start 2024-04-25T09:54:17.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.847+0000 7f8291f34640 1 -- start start 2024-04-25T09:54:17.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f8291f34640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f828c15dac0 0x7f828c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f8291f34640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f828c0d1f30 0x7f828c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f8291f34640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c0d2420 con 0x7f828c0d34c0 2024-04-25T09:54:17.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f8291f34640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828c0d3030 con 0x7f828c0d1f30 2024-04-25T09:54:17.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f8291f34640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828c0d2700 con 0x7f828c15dac0 2024-04-25T09:54:17.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f828affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f828c0d1f30 0x7f828c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f828affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f828c0d1f30 0x7f828c0d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34026/0 (socket says 172.21.15.19:34026) 2024-04-25T09:54:17.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f828affd640 1 -- 172.21.15.19:0/2803050907 learned_addr learned my addr 172.21.15.19:0/2803050907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:17.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f828b7fe640 1 --2- 172.21.15.19:0/2803050907 >> v2:172.21.15.112:3300/0 conn(0x7f828c15dac0 0x7f828c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f828affd640 1 -- 172.21.15.19:0/2803050907 >> v2:172.21.15.112:3300/0 conn(0x7f828c15dac0 msgr2=0x7f828c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f828affd640 1 --2- 172.21.15.19:0/2803050907 >> v2:172.21.15.112:3300/0 conn(0x7f828c15dac0 0x7f828c0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f828affd640 1 -- 172.21.15.19:0/2803050907 >> v1:172.21.15.19:6789/0 conn(0x7f828c0d34c0 legacy=0x7f828c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f828affd640 1 -- 172.21.15.19:0/2803050907 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0d29e0 con 0x7f828c0d1f30 2024-04-25T09:54:17.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.848+0000 7f828b7fe640 1 --2- 172.21.15.19:0/2803050907 >> v2:172.21.15.112:3300/0 conn(0x7f828c15dac0 0x7f828c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:17.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.849+0000 7f828affd640 1 --2- 172.21.15.19:0/2803050907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c0d1f30 0x7f828c0d03a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f828c0b6520 tx=0x7f827c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65528e0f8890af28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.849+0000 7f828a7fc640 1 -- 172.21.15.19:0/2803050907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c004230 con 0x7f828c0d1f30 2024-04-25T09:54:17.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.849+0000 7f828a7fc640 1 -- 172.21.15.19:0/2803050907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c02ae50 con 0x7f828c0d1f30 2024-04-25T09:54:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.849+0000 7f828a7fc640 1 -- 172.21.15.19:0/2803050907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c003610 con 0x7f828c0d1f30 2024-04-25T09:54:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.849+0000 7f8291f34640 1 -- 172.21.15.19:0/2803050907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c0d1f30 msgr2=0x7f828c0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.849+0000 7f8291f34640 1 --2- 172.21.15.19:0/2803050907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c0d1f30 0x7f828c0d03a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f828c0b6520 tx=0x7f827c02a300 comp rx=0 tx=0).stop 2024-04-25T09:54:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.850+0000 7f8291f34640 1 -- 172.21.15.19:0/2803050907 shutdown_connections 2024-04-25T09:54:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.850+0000 7f8291f34640 1 --2- 172.21.15.19:0/2803050907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c0d1f30 0x7f828c0d03a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.850+0000 7f8291f34640 1 --2- 172.21.15.19:0/2803050907 >> v2:172.21.15.112:3300/0 conn(0x7f828c15dac0 0x7f828c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.850+0000 7f8291f34640 1 -- 172.21.15.19:0/2803050907 >> 172.21.15.19:0/2803050907 conn(0x7f828c061c30 msgr2=0x7f828c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.850+0000 7f8291f34640 1 -- 172.21.15.19:0/2803050907 shutdown_connections 2024-04-25T09:54:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.850+0000 7f8291f34640 1 -- 172.21.15.19:0/2803050907 wait complete. 2024-04-25T09:54:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.851+0000 7f8291f34640 1 Processor -- start 2024-04-25T09:54:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.851+0000 7f8291f34640 1 -- start start 2024-04-25T09:54:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f8291f34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c0d1f30 0x7f828c0c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f8291f34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f828c0d34c0 0x7f828c1f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f8291f34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f828c15dac0 0x7f828c160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f8291f34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0b97d0 con 0x7f828c0d1f30 2024-04-25T09:54:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f8291f34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c15dea0 con 0x7f828c15dac0 2024-04-25T09:54:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f8291f34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0b9950 con 0x7f828c0d34c0 2024-04-25T09:54:17.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f828b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c0d1f30 0x7f828c0c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f828b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c0d1f30 0x7f828c0c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34028/0 (socket says 172.21.15.19:34028) 2024-04-25T09:54:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f828b7fe640 1 -- 172.21.15.19:0/336643618 learned_addr learned my addr 172.21.15.19:0/336643618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f828affd640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f828c0d34c0 0x7f828c1f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f828bfff640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f828c15dac0 0x7f828c160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f828b7fe640 1 -- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f828c0d34c0 msgr2=0x7f828c1f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f828b7fe640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f828c0d34c0 0x7f828c1f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f828b7fe640 1 -- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f828c15dac0 msgr2=0x7f828c160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f828b7fe640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f828c15dac0 0x7f828c160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f828b7fe640 1 -- 172.21.15.19:0/336643618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0b4410 con 0x7f828c0d1f30 2024-04-25T09:54:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.852+0000 7f828b7fe640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c0d1f30 0x7f828c0c95d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f827400d7d0 tx=0x7f827400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.853+0000 7f8283fff640 1 -- 172.21.15.19:0/336643618 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274004030 con 0x7f828c0d1f30 2024-04-25T09:54:17.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.853+0000 7f8283fff640 1 -- 172.21.15.19:0/336643618 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82740041d0 con 0x7f828c0d1f30 2024-04-25T09:54:17.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.853+0000 7f8283fff640 1 -- 172.21.15.19:0/336643618 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827400b6c0 con 0x7f828c0d1f30 2024-04-25T09:54:17.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.853+0000 7f8291f34640 1 -- 172.21.15.19:0/336643618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c0b4750 con 0x7f828c0d1f30 2024-04-25T09:54:17.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.853+0000 7f8291f34640 1 -- 172.21.15.19:0/336643618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c0b4f90 con 0x7f828c0d1f30 2024-04-25T09:54:17.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.854+0000 7f8283fff640 1 -- 172.21.15.19:0/336643618 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f827400b860 con 0x7f828c0d1f30 2024-04-25T09:54:17.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.855+0000 7f8283fff640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f826c06b4d0 0x7f826c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.855+0000 7f8283fff640 1 -- 172.21.15.19:0/336643618 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f827408a650 con 0x7f828c0d1f30 2024-04-25T09:54:17.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.855+0000 7f828affd640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f826c06b4d0 0x7f826c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.855+0000 7f828affd640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f826c06b4d0 0x7f826c06d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f827c033000 tx=0x7f827c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.856+0000 7f8294c39740 1 Processor -- start 2024-04-25T09:54:17.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.856+0000 7f8294c39740 1 -- start start 2024-04-25T09:54:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.856+0000 7f8294c39740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af89230 0x55db5af89610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.856+0000 7f8294c39740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afa7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.856+0000 7f8294c39740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5af5ef10 0x55db5afa5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.856+0000 7f8294c39740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5aec1a10 con 0x55db5afa6ad0 2024-04-25T09:54:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.856+0000 7f8294c39740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5ae6ab30 con 0x55db5af89230 2024-04-25T09:54:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.856+0000 7f8294c39740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db5aebdc50 con 0x55db5af5ef10 2024-04-25T09:54:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f828affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afa7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f828affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afa7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34034/0 (socket says 172.21.15.19:34034) 2024-04-25T09:54:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f828bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5af5ef10 0x55db5afa5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f828affd640 1 -- 172.21.15.19:0/2817433135 learned_addr learned my addr 172.21.15.19:0/2817433135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f828b7fe640 1 --2- 172.21.15.19:0/2817433135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af89230 0x55db5af89610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f828affd640 1 -- 172.21.15.19:0/2817433135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5af5ef10 msgr2=0x55db5afa5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f828affd640 1 --2- 172.21.15.19:0/2817433135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5af5ef10 0x55db5afa5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f828affd640 1 -- 172.21.15.19:0/2817433135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af89230 msgr2=0x55db5af89610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f828affd640 1 --2- 172.21.15.19:0/2817433135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af89230 0x55db5af89610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f828affd640 1 -- 172.21.15.19:0/2817433135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db5ae7fe10 con 0x55db5afa6ad0 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f828affd640 1 --2- 172.21.15.19:0/2817433135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afa7020 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f827c009940 tx=0x7f827c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d815ce2b1ed10791 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f82817fa640 1 -- 172.21.15.19:0/2817433135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c04e070 con 0x55db5afa6ad0 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f82817fa640 1 -- 172.21.15.19:0/2817433135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c02ae50 con 0x55db5afa6ad0 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.857+0000 7f82817fa640 1 -- 172.21.15.19:0/2817433135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c003ae0 con 0x55db5afa6ad0 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.858+0000 7f8294c39740 1 -- 172.21.15.19:0/2817433135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 msgr2=0x55db5afa7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.858+0000 7f8294c39740 1 --2- 172.21.15.19:0/2817433135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afa7020 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f827c009940 tx=0x7f827c02c000 comp rx=0 tx=0).stop 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.858+0000 7f8294c39740 1 -- 172.21.15.19:0/2817433135 shutdown_connections 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.858+0000 7f8294c39740 1 --2- 172.21.15.19:0/2817433135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5af5ef10 0x55db5afa5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.858+0000 7f8294c39740 1 --2- 172.21.15.19:0/2817433135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afa7020 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.858+0000 7f8294c39740 1 --2- 172.21.15.19:0/2817433135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af89230 0x55db5af89610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.858+0000 7f8294c39740 1 -- 172.21.15.19:0/2817433135 >> 172.21.15.19:0/2817433135 conn(0x55db5af5b840 msgr2=0x55db5afa4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.858+0000 7f8294c39740 1 -- 172.21.15.19:0/2817433135 shutdown_connections 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.858+0000 7f8294c39740 1 -- 172.21.15.19:0/2817433135 wait complete. 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.859+0000 7f8294c39740 1 -- start start 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.859+0000 7f8294c39740 1 Processor -- start 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.859+0000 7f8294c39740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af9dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.859+0000 7f8294c39740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5afa6ad0 0x55db5af8af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.859+0000 7f8294c39740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af8b480 0x55db5afb6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.859+0000 7f8294c39740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5af5fcb0 con 0x55db5af89230 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.859+0000 7f8294c39740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5ae7e0c0 con 0x55db5af8b480 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.859+0000 7f8294c39740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db5af59770 con 0x55db5afa6ad0 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f8294c39740 10 client.0 _ll_register_callbacks cb 0x55db5ae71ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f828b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af9dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f828b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af9dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34042/0 (socket says 172.21.15.19:34042) 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f828b7fe640 1 -- 172.21.15.19:0/2530990579 learned_addr learned my addr 172.21.15.19:0/2530990579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f828b7fe640 1 -- 172.21.15.19:0/2530990579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5afa6ad0 msgr2=0x55db5af8af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f828affd640 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5afa6ad0 0x55db5af8af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f828b7fe640 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5afa6ad0 0x55db5af8af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f828bfff640 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af8b480 0x55db5afb6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f828b7fe640 1 -- 172.21.15.19:0/2530990579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af8b480 msgr2=0x55db5afb6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f828b7fe640 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af8b480 0x55db5afb6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f828b7fe640 1 -- 172.21.15.19:0/2530990579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db5abd62d0 con 0x55db5af89230 2024-04-25T09:54:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f828b7fe640 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af9dc20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8274058bd0 tx=0x7f8274059340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bd7ec3cea85743 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f826a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:54:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f826a7fc640 1 -- 172.21.15.19:0/2530990579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274088060 con 0x55db5af89230 2024-04-25T09:54:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f826a7fc640 1 -- 172.21.15.19:0/2530990579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f827400bb50 con 0x55db5af89230 2024-04-25T09:54:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f8294c39740 1 -- 172.21.15.19:0/2530990579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f827c02ae50 con 0x55db5af89230 2024-04-25T09:54:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f826a7fc640 1 -- 172.21.15.19:0/2530990579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8274070030 con 0x55db5af89230 2024-04-25T09:54:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f8294c39740 10 client.5442 fetch_fsmap learned FSMap version 16 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f8294c39740 1 -- 172.21.15.19:0/2530990579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55db5aca06b0 con 0x55db5af89230 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.860+0000 7f826a7fc640 1 -- 172.21.15.19:0/2530990579 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c02ae50 con 0x55db5af89230 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.861+0000 7f826a7fc640 1 -- 172.21.15.19:0/2530990579 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8274013070 con 0x55db5af89230 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.861+0000 7f8294c39740 10 client.5442 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.861+0000 7f8294c39740 10 client.5442 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.861+0000 7f8294c39740 20 client.5442 populate_metadata read hostname 'smithi019' 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.861+0000 7f8294c39740 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55db5afb7e80 0x55db5afb5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.861+0000 7f8294c39740 4 client.5442 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.861+0000 7f8294c39740 1 -- 172.21.15.19:0/2530990579 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db5af20920 con 0x55db5afb7e80 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.861+0000 7f828affd640 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55db5afb7e80 0x55db5afb5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.861+0000 7f826a7fc640 10 client.5442 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:54:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.861+0000 7f828affd640 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55db5afb7e80 0x55db5afb5ab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589a744aa629602a server_cookie=9b5137a074f3a0c1 in_seq=0 out_seq=0 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f826a7fc640 1 -- 172.21.15.19:0/2530990579 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db5af20920 con 0x55db5afb7e80 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f826a7fc640 10 client.5442 handle_command_reply: tid=0 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 1 client.5442 shutdown 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 1 -- 172.21.15.19:0/2530990579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 msgr2=0x55db5af9dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af9dc20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8274058bd0 tx=0x7f8274059340 comp rx=0 tx=0).stop 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 1 -- 172.21.15.19:0/2530990579 shutdown_connections 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af8b480 0x55db5afb6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5afa6ad0 0x55db5af8af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55db5afb7e80 0x55db5afb5ab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 1 --2- 172.21.15.19:0/2530990579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af9dc20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f826a7fc640 0 client.5442 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 1 -- 172.21.15.19:0/2530990579 >> 172.21.15.19:0/2530990579 conn(0x55db5af5ef10 msgr2=0x55db5af882b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 1 -- 172.21.15.19:0/2530990579 shutdown_connections 2024-04-25T09:54:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 1 -- 172.21.15.19:0/2530990579 wait complete. 2024-04-25T09:54:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.862+0000 7f8294c39740 20 client.5442 trim_cache size 0 max 16384 2024-04-25T09:54:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.878+0000 7f8294c39740 1 Processor -- start 2024-04-25T09:54:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.878+0000 7f8294c39740 1 -- start start 2024-04-25T09:54:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f8294c39740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f8294c39740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af8b480 0x55db5af9e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f8294c39740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5afa6ad0 0x55db5af87490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f8294c39740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c1ff650 con 0x55db5af89230 2024-04-25T09:54:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f8294c39740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0ccd00 con 0x55db5af8b480 2024-04-25T09:54:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f8294c39740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c1ff050 con 0x55db5afa6ad0 2024-04-25T09:54:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af886b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34054/0 (socket says 172.21.15.19:34054) 2024-04-25T09:54:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828b7fe640 1 -- 172.21.15.19:0/3401203653 learned_addr learned my addr 172.21.15.19:0/3401203653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828bfff640 1 --2- 172.21.15.19:0/3401203653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5afa6ad0 0x55db5af87490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828affd640 1 --2- 172.21.15.19:0/3401203653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af8b480 0x55db5af9e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828b7fe640 1 -- 172.21.15.19:0/3401203653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5afa6ad0 msgr2=0x55db5af87490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828b7fe640 1 --2- 172.21.15.19:0/3401203653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5afa6ad0 0x55db5af87490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828b7fe640 1 -- 172.21.15.19:0/3401203653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af8b480 msgr2=0x55db5af9e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828b7fe640 1 --2- 172.21.15.19:0/3401203653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af8b480 0x55db5af9e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828b7fe640 1 -- 172.21.15.19:0/3401203653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db5af8c060 con 0x55db5af89230 2024-04-25T09:54:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828affd640 1 --2- 172.21.15.19:0/3401203653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af8b480 0x55db5af9e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f828b7fe640 1 --2- 172.21.15.19:0/3401203653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af886b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f827400b4b0 tx=0x7f827408a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546cd86c9a37720f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f8280ff9640 1 -- 172.21.15.19:0/3401203653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827405cde0 con 0x55db5af89230 2024-04-25T09:54:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f8280ff9640 1 -- 172.21.15.19:0/3401203653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827400bb50 con 0x55db5af89230 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.879+0000 7f8280ff9640 1 -- 172.21.15.19:0/3401203653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274058090 con 0x55db5af89230 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.880+0000 7f8294c39740 1 -- 172.21.15.19:0/3401203653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 msgr2=0x55db5af886b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.880+0000 7f8294c39740 1 --2- 172.21.15.19:0/3401203653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af886b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f827400b4b0 tx=0x7f827408a050 comp rx=0 tx=0).stop 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.880+0000 7f8294c39740 1 -- 172.21.15.19:0/3401203653 shutdown_connections 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.880+0000 7f8294c39740 1 --2- 172.21.15.19:0/3401203653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5afa6ad0 0x55db5af87490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.880+0000 7f8294c39740 1 --2- 172.21.15.19:0/3401203653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af8b480 0x55db5af9e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.880+0000 7f8294c39740 1 --2- 172.21.15.19:0/3401203653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5af89230 0x55db5af886b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.880+0000 7f8294c39740 1 -- 172.21.15.19:0/3401203653 >> 172.21.15.19:0/3401203653 conn(0x55db5af5ef10 msgr2=0x55db5af5f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.880+0000 7f8294c39740 1 -- 172.21.15.19:0/3401203653 shutdown_connections 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.880+0000 7f8294c39740 1 -- 172.21.15.19:0/3401203653 wait complete. 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f8294c39740 1 -- start start 2024-04-25T09:54:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f8294c39740 1 Processor -- start 2024-04-25T09:54:17.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f8294c39740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af89230 0x55db5b00ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f8294c39740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5af8b480 0x55db5afb5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f8294c39740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afb5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f8294c39740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5afb7d00 con 0x55db5afa6ad0 2024-04-25T09:54:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f8294c39740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db5af8c4d0 con 0x55db5af89230 2024-04-25T09:54:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f8294c39740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db5afb82a0 con 0x55db5af8b480 2024-04-25T09:54:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f8294c39740 10 client.0 _ll_register_callbacks cb 0x55db5af59c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:54:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f828bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afb5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f828bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afb5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34060/0 (socket says 172.21.15.19:34060) 2024-04-25T09:54:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f828b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af89230 0x55db5b00ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f828bfff640 1 -- 172.21.15.19:0/1068404225 learned_addr learned my addr 172.21.15.19:0/1068404225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.881+0000 7f828affd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5af8b480 0x55db5afb5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f828bfff640 1 -- 172.21.15.19:0/1068404225 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5af8b480 msgr2=0x55db5afb5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f828bfff640 1 --2- 172.21.15.19:0/1068404225 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5af8b480 0x55db5afb5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f828bfff640 1 -- 172.21.15.19:0/1068404225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af89230 msgr2=0x55db5b00ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f828bfff640 1 --2- 172.21.15.19:0/1068404225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af89230 0x55db5b00ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f828bfff640 1 -- 172.21.15.19:0/1068404225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db5afb3f50 con 0x55db5afa6ad0 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f828bfff640 1 --2- 172.21.15.19:0/1068404225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afb5d60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f82840012e0 tx=0x7f8284001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c309b2425832a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f826a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f826a7fc640 1 -- 172.21.15.19:0/1068404225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284006d60 con 0x55db5afa6ad0 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f826a7fc640 1 -- 172.21.15.19:0/1068404225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8284019910 con 0x55db5afa6ad0 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f8294c39740 1 -- 172.21.15.19:0/1068404225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f827400bb50 con 0x55db5afa6ad0 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f826a7fc640 1 -- 172.21.15.19:0/1068404225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8284019b10 con 0x55db5afa6ad0 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f8294c39740 10 client.5448 fetch_fsmap learned FSMap version 16 2024-04-25T09:54:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f8294c39740 1 -- 172.21.15.19:0/1068404225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55db5afa6790 con 0x55db5afa6ad0 2024-04-25T09:54:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f826a7fc640 1 -- 172.21.15.19:0/1068404225 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827400bb50 con 0x55db5afa6ad0 2024-04-25T09:54:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f826a7fc640 1 -- 172.21.15.19:0/1068404225 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8284019cb0 con 0x55db5afa6ad0 2024-04-25T09:54:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f8294c39740 10 client.5448 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T09:54:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f8294c39740 10 client.5448 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:54:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.882+0000 7f8294c39740 20 client.5448 populate_metadata read hostname 'smithi019' 2024-04-25T09:54:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 --2- 172.21.15.19:0/1068404225 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55db5af5cde0 0x55db5af5d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 4 client.5448 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:54:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 -- 172.21.15.19:0/1068404225 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55db5aecef50 con 0x55db5af5cde0 2024-04-25T09:54:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f828b7fe640 1 --2- 172.21.15.19:0/1068404225 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55db5af5cde0 0x55db5af5d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f826a7fc640 10 client.5448 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:54:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f828b7fe640 1 --2- 172.21.15.19:0/1068404225 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55db5af5cde0 0x55db5af5d1c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06a840ee060496d server_cookie=7727f69764be90c3 in_seq=0 out_seq=0 2024-04-25T09:54:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f826a7fc640 1 -- 172.21.15.19:0/1068404225 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55db5aecef50 con 0x55db5af5cde0 2024-04-25T09:54:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f826a7fc640 10 client.5448 handle_command_reply: tid=0 2024-04-25T09:54:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 client.5448 shutdown 2024-04-25T09:54:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 -- 172.21.15.19:0/1068404225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 msgr2=0x55db5afb5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 --2- 172.21.15.19:0/1068404225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afb5d60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f82840012e0 tx=0x7f8284001c70 comp rx=0 tx=0).stop 2024-04-25T09:54:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 -- 172.21.15.19:0/1068404225 shutdown_connections 2024-04-25T09:54:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 --2- 172.21.15.19:0/1068404225 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55db5af5cde0 0x55db5af5d1c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 --2- 172.21.15.19:0/1068404225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5afa6ad0 0x55db5afb5d60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 --2- 172.21.15.19:0/1068404225 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55db5af8b480 0x55db5afb5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f826a7fc640 0 client.5448 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:54:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 --2- 172.21.15.19:0/1068404225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db5af89230 0x55db5b00ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 -- 172.21.15.19:0/1068404225 >> 172.21.15.19:0/1068404225 conn(0x55db5af5ef10 msgr2=0x55db5afb4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 -- 172.21.15.19:0/1068404225 shutdown_connections 2024-04-25T09:54:17.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.883+0000 7f8294c39740 1 -- 172.21.15.19:0/1068404225 wait complete. 2024-04-25T09:54:17.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8294c39740 20 client.5448 trim_cache size 0 max 16384 2024-04-25T09:54:17.894 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:54:17.894 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T09:54:17.895 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T09:54:17.895 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 5286, 2024-04-25T09:54:17.895 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T09:54:17.895 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T09:54:17.895 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T09:54:17.895 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 4.670232554, 2024-04-25T09:54:17.895 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 16, 2024-04-25T09:54:17.895 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 38, 2024-04-25T09:54:17.895 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 38, 2024-04-25T09:54:17.895 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 11.568791892 2024-04-25T09:54:17.895 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:54:17.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 -- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f826c06b4d0 msgr2=0x7f826c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f826c06b4d0 0x7f826c06d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f827c033000 tx=0x7f827c02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:17.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 -- 172.21.15.19:0/336643618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c0d1f30 msgr2=0x7f828c0c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c0d1f30 0x7f828c0c95d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f827400d7d0 tx=0x7f827400dc90 comp rx=0 tx=0).stop 2024-04-25T09:54:17.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 -- 172.21.15.19:0/336643618 shutdown_connections 2024-04-25T09:54:17.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f826c06b4d0 0x7f826c06d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f828c15dac0 0x7f828c160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f828c0d34c0 0x7f828c1f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 --2- 172.21.15.19:0/336643618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c0d1f30 0x7f828c0c95d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 -- 172.21.15.19:0/336643618 >> 172.21.15.19:0/336643618 conn(0x7f828c061c30 msgr2=0x7f828c0b8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 -- 172.21.15.19:0/336643618 shutdown_connections 2024-04-25T09:54:17.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.884+0000 7f8291f34640 1 -- 172.21.15.19:0/336643618 wait complete. 2024-04-25T09:54:17.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:54:18.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.997+0000 7fb218190640 1 Processor -- start 2024-04-25T09:54:18.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.998+0000 7fb218190640 1 -- start start 2024-04-25T09:54:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.998+0000 7fb218190640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb21015dc70 0x7fb2100d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.998+0000 7fb218190640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb21006ce10 0x7fb2100cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.998+0000 7fb218190640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21006c220 con 0x7fb21009a450 2024-04-25T09:54:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.998+0000 7fb218190640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2100d0c90 con 0x7fb21015dc70 2024-04-25T09:54:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.998+0000 7fb218190640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2100d0080 con 0x7fb21006ce10 2024-04-25T09:54:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.998+0000 7fb215f2c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb21015dc70 0x7fb2100d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.998+0000 7fb215f2c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb21015dc70 0x7fb2100d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34064/0 (socket says 172.21.15.19:34064) 2024-04-25T09:54:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.998+0000 7fb215f2c640 1 -- 172.21.15.19:0/1707528241 learned_addr learned my addr 172.21.15.19:0/1707528241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.999+0000 7fb21672d640 1 --2- 172.21.15.19:0/1707528241 >> v2:172.21.15.112:3300/0 conn(0x7fb21006ce10 0x7fb2100cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.999+0000 7fb215f2c640 1 -- 172.21.15.19:0/1707528241 >> v2:172.21.15.112:3300/0 conn(0x7fb21006ce10 msgr2=0x7fb2100cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.999+0000 7fb215f2c640 1 --2- 172.21.15.19:0/1707528241 >> v2:172.21.15.112:3300/0 conn(0x7fb21006ce10 0x7fb2100cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.999+0000 7fb215f2c640 1 -- 172.21.15.19:0/1707528241 >> v1:172.21.15.112:6789/0 conn(0x7fb21009a450 legacy=0x7fb21006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.999+0000 7fb215f2c640 1 -- 172.21.15.19:0/1707528241 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100d0360 con 0x7fb21015dc70 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:17.999+0000 7fb215f2c640 1 --2- 172.21.15.19:0/1707528241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb21015dc70 0x7fb2100d1120 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb2100b7720 tx=0x7fb20c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae854f8aaa3f9dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.000+0000 7fb214f2a640 1 -- 172.21.15.19:0/1707528241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c037070 con 0x7fb21015dc70 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.000+0000 7fb214f2a640 1 -- 172.21.15.19:0/1707528241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c032440 con 0x7fb21015dc70 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.000+0000 7fb214f2a640 1 -- 172.21.15.19:0/1707528241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c032810 con 0x7fb21015dc70 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.000+0000 7fb218190640 1 -- 172.21.15.19:0/1707528241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb21015dc70 msgr2=0x7fb2100d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.000+0000 7fb218190640 1 --2- 172.21.15.19:0/1707528241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb21015dc70 0x7fb2100d1120 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb2100b7720 tx=0x7fb20c02a270 comp rx=0 tx=0).stop 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.000+0000 7fb218190640 1 -- 172.21.15.19:0/1707528241 shutdown_connections 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.000+0000 7fb218190640 1 --2- 172.21.15.19:0/1707528241 >> v2:172.21.15.112:3300/0 conn(0x7fb21006ce10 0x7fb2100cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.000+0000 7fb218190640 1 --2- 172.21.15.19:0/1707528241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb21015dc70 0x7fb2100d1120 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.000+0000 7fb218190640 1 -- 172.21.15.19:0/1707528241 >> 172.21.15.19:0/1707528241 conn(0x7fb2100b3270 msgr2=0x7fb2100b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:18.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.000+0000 7fb218190640 1 -- 172.21.15.19:0/1707528241 shutdown_connections 2024-04-25T09:54:18.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.000+0000 7fb218190640 1 -- 172.21.15.19:0/1707528241 wait complete. 2024-04-25T09:54:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.001+0000 7fb218190640 1 Processor -- start 2024-04-25T09:54:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.002+0000 7fb218190640 1 -- start start 2024-04-25T09:54:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.002+0000 7fb218190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb21006ce10 0x7fb21014cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.002+0000 7fb218190640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb21009a450 0x7fb2100b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.002+0000 7fb218190640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb21015dc70 0x7fb2100b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.002+0000 7fb218190640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21015e050 con 0x7fb21006ce10 2024-04-25T09:54:18.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.002+0000 7fb218190640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100d0920 con 0x7fb21009a450 2024-04-25T09:54:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.002+0000 7fb218190640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100cbdd0 con 0x7fb21015dc70 2024-04-25T09:54:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.002+0000 7fb215f2c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb21006ce10 0x7fb21014cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.002+0000 7fb215f2c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb21006ce10 0x7fb21014cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34072/0 (socket says 172.21.15.19:34072) 2024-04-25T09:54:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.002+0000 7fb215f2c640 1 -- 172.21.15.19:0/781347999 learned_addr learned my addr 172.21.15.19:0/781347999 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb21672d640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb21015dc70 0x7fb2100b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb21572b640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb21009a450 0x7fb2100b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb215f2c640 1 -- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb21015dc70 msgr2=0x7fb2100b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb215f2c640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb21015dc70 0x7fb2100b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb215f2c640 1 -- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb21009a450 msgr2=0x7fb2100b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb215f2c640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb21009a450 0x7fb2100b6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb215f2c640 1 -- 172.21.15.19:0/781347999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2101f6d70 con 0x7fb21006ce10 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb215f2c640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb21006ce10 0x7fb21014cd60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb20c000c00 tx=0x7fb20c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb2067fc640 1 -- 172.21.15.19:0/781347999 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c041070 con 0x7fb21006ce10 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb2067fc640 1 -- 172.21.15.19:0/781347999 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c03c870 con 0x7fb21006ce10 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb218190640 1 -- 172.21.15.19:0/781347999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2101f7050 con 0x7fb21006ce10 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb2067fc640 1 -- 172.21.15.19:0/781347999 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c037040 con 0x7fb21006ce10 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.003+0000 7fb218190640 1 -- 172.21.15.19:0/781347999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2101f7610 con 0x7fb21006ce10 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.004+0000 7fb218190640 1 -- 172.21.15.19:0/781347999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb210073990 con 0x7fb21006ce10 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.004+0000 7fb2067fc640 1 -- 172.21.15.19:0/781347999 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb20c044070 con 0x7fb21006ce10 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.005+0000 7fb2067fc640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1f406b4d0 0x7fb1f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.005+0000 7fb2067fc640 1 -- 172.21.15.19:0/781347999 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb20c0aafe0 con 0x7fb21006ce10 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.005+0000 7fb21572b640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1f406b4d0 0x7fb1f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.005+0000 7fb21572b640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1f406b4d0 0x7fb1f406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb2100c6520 tx=0x7fb200009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:18.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.009+0000 7fb2067fc640 1 -- 172.21.15.19:0/781347999 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb20c079830 con 0x7fb21006ce10 2024-04-25T09:54:18.181 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T09:54:18.179+0000 7f8de9928640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:54:18.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.227+0000 7fb218190640 1 -- 172.21.15.19:0/781347999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb2100cd620 con 0x7fb21006ce10 2024-04-25T09:54:18.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.227+0000 7fb2067fc640 1 -- 172.21.15.19:0/781347999 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb20c07d8d0 con 0x7fb21006ce10 2024-04-25T09:54:18.230 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:54:18.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.229+0000 7fb218190640 1 -- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1f406b4d0 msgr2=0x7fb1f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.229+0000 7fb218190640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1f406b4d0 0x7fb1f406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb2100c6520 tx=0x7fb200009040 comp rx=0 tx=0).stop 2024-04-25T09:54:18.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.229+0000 7fb218190640 1 -- 172.21.15.19:0/781347999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb21006ce10 msgr2=0x7fb21014cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.229+0000 7fb218190640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb21006ce10 0x7fb21014cd60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb20c000c00 tx=0x7fb20c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.229+0000 7fb218190640 1 -- 172.21.15.19:0/781347999 shutdown_connections 2024-04-25T09:54:18.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.229+0000 7fb218190640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1f406b4d0 0x7fb1f406d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.229+0000 7fb218190640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb21015dc70 0x7fb2100b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.229+0000 7fb218190640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb21009a450 0x7fb2100b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.229+0000 7fb218190640 1 --2- 172.21.15.19:0/781347999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb21006ce10 0x7fb21014cd60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.229+0000 7fb218190640 1 -- 172.21.15.19:0/781347999 >> 172.21.15.19:0/781347999 conn(0x7fb2100b3270 msgr2=0x7fb2100b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.230+0000 7fb218190640 1 -- 172.21.15.19:0/781347999 shutdown_connections 2024-04-25T09:54:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:18.230+0000 7fb218190640 1 -- 172.21.15.19:0/781347999 wait complete. 2024-04-25T09:54:18.243 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:54:18.243 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:54:18.243 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T09:54:18.243 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:54:18.243 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:18.243 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:54:18.243 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:54:18.243 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:54:18.243 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:54:18.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:18.244 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T09:54:18.259 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:54:18.259 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:18.260 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42687sec preferred_lft 42687sec 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T09:54:18.261 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:18.262 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:54:18.262 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:54:18.262 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T09:54:18.262 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:18.262 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:54:18.262 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:54:18.262 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T09:54:18.262 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:18.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:18.263 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:54:18.323 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T09:54:18.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:18.325 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T09:54:18.381 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T09:54:18.382 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:54:18.382 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:54:18.382 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:54:18.382 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:54:18.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:18.382 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:54:18.382 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:54:18.382 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:54:18.382 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:54:18.382 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:54:18.383 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:54:18.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:18.468 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:54:18.483 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:54:18.483 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:54:18.483 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:54:18.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:18.484 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T09:54:18.548 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:54:18.548 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:18.603 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:54:18.604 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:54:18.604 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:18.659 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:54:18.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:18.659 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:54:18.741 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:54:18.742 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:54:18.753 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:54:18.753 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:54:18.753 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T09:54:18.753 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T09:54:18.753 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T09:54:18.754 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:54:18.754 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:54:18.754 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:54:18.754 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:54:18.754 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:54:18.756 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:54:18.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:18.756 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:18.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:18.818 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:54:18.880 INFO:teuthology.orchestra.run.smithi019.stdout:client5457 2024-04-25T09:54:18.881 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:54:18.881 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:18.937 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T09:54:19.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.080+0000 7f2df3113640 1 Processor -- start 2024-04-25T09:54:19.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.080+0000 7f2df3113640 1 -- start start 2024-04-25T09:54:19.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.081+0000 7f2df3113640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2dec1446d0 0x7f2dec0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.081+0000 7f2df3113640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dec0cccc0 con 0x7f2dec06e8a0 2024-04-25T09:54:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.081+0000 7f2df3113640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dec0d0460 con 0x7f2dec144d50 2024-04-25T09:54:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.081+0000 7f2df3113640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dec0b2b20 con 0x7f2dec1446d0 2024-04-25T09:54:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.081+0000 7f2df0eaf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2dec1446d0 0x7f2dec0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.081+0000 7f2df16b0640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2dec06e8a0 0x7f2dec0cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58380/0 (socket says 172.21.15.19:58380) 2024-04-25T09:54:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.081+0000 7f2df16b0640 1 -- 172.21.15.19:0/3302942031 learned_addr learned my addr 172.21.15.19:0/3302942031 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:54:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.081+0000 7f2df0eaf640 1 -- 172.21.15.19:0/3302942031 >> v1:172.21.15.112:6789/0 conn(0x7f2dec144d50 legacy=0x7f2dec0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.081+0000 7f2df0eaf640 1 -- 172.21.15.19:0/3302942031 >> v1:172.21.15.19:6789/0 conn(0x7f2dec06e8a0 legacy=0x7f2dec0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.081+0000 7f2de37fe640 1 -- 172.21.15.19:0/3302942031 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302889659 0 0) 0x7f2dec0cccc0 con 0x7f2dec06e8a0 2024-04-25T09:54:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.081+0000 7f2df0eaf640 1 -- 172.21.15.19:0/3302942031 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec0cc140 con 0x7f2dec1446d0 2024-04-25T09:54:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.082+0000 7f2df0eaf640 1 --2- 172.21.15.19:0/3302942031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec1446d0 0x7f2dec0b3f20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2de80097a0 tx=0x7f2de802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64d8522fd8cc104 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.082+0000 7f2de37fe640 1 -- 172.21.15.19:0/3302942031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8037070 con 0x7f2dec1446d0 2024-04-25T09:54:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.082+0000 7f2de37fe640 1 -- 172.21.15.19:0/3302942031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de8032440 con 0x7f2dec1446d0 2024-04-25T09:54:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.082+0000 7f2de37fe640 1 -- 172.21.15.19:0/3302942031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8032810 con 0x7f2dec1446d0 2024-04-25T09:54:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.083+0000 7f2df3113640 1 -- 172.21.15.19:0/3302942031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec1446d0 msgr2=0x7f2dec0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.083+0000 7f2df3113640 1 --2- 172.21.15.19:0/3302942031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec1446d0 0x7f2dec0b3f20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2de80097a0 tx=0x7f2de802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.083+0000 7f2df3113640 1 -- 172.21.15.19:0/3302942031 shutdown_connections 2024-04-25T09:54:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.083+0000 7f2df3113640 1 --2- 172.21.15.19:0/3302942031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec1446d0 0x7f2dec0b3f20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.083+0000 7f2df3113640 1 -- 172.21.15.19:0/3302942031 >> 172.21.15.19:0/3302942031 conn(0x7f2dec06abe0 msgr2=0x7f2dec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.083+0000 7f2df3113640 1 -- 172.21.15.19:0/3302942031 shutdown_connections 2024-04-25T09:54:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.083+0000 7f2df3113640 1 -- 172.21.15.19:0/3302942031 wait complete. 2024-04-25T09:54:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.084+0000 7f2df3113640 1 Processor -- start 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.084+0000 7f2df3113640 1 -- start start 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df3113640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec06e8a0 0x7f2dec0cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df3113640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dec1446d0 0x7f2dec13c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df3113640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dec144d50 0x7f2dec13ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df3113640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec0cc700 con 0x7f2dec06e8a0 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df3113640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec06d400 con 0x7f2dec144d50 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df3113640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec0cc9e0 con 0x7f2dec1446d0 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df0eaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec06e8a0 0x7f2dec0cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df0eaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec06e8a0 0x7f2dec0cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34096/0 (socket says 172.21.15.19:34096) 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df0eaf640 1 -- 172.21.15.19:0/2129887094 learned_addr learned my addr 172.21.15.19:0/2129887094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df16b0640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dec144d50 0x7f2dec13ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2de3fff640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dec1446d0 0x7f2dec13c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df0eaf640 1 -- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dec1446d0 msgr2=0x7f2dec13c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df0eaf640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dec1446d0 0x7f2dec13c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df0eaf640 1 -- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dec144d50 msgr2=0x7f2dec13ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df0eaf640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dec144d50 0x7f2dec13ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df0eaf640 1 -- 172.21.15.19:0/2129887094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec13f070 con 0x7f2dec06e8a0 2024-04-25T09:54:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df16b0640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dec144d50 0x7f2dec13ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.085+0000 7f2df0eaf640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec06e8a0 0x7f2dec0cef80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2de8035040 tx=0x7f2de80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.086+0000 7f2de17fa640 1 -- 172.21.15.19:0/2129887094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8037040 con 0x7f2dec06e8a0 2024-04-25T09:54:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.086+0000 7f2de17fa640 1 -- 172.21.15.19:0/2129887094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de8004230 con 0x7f2dec06e8a0 2024-04-25T09:54:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.086+0000 7f2de17fa640 1 -- 172.21.15.19:0/2129887094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8032e40 con 0x7f2dec06e8a0 2024-04-25T09:54:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.086+0000 7f2df3113640 1 -- 172.21.15.19:0/2129887094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dec13f350 con 0x7f2dec06e8a0 2024-04-25T09:54:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.086+0000 7f2df3113640 1 -- 172.21.15.19:0/2129887094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dec13f910 con 0x7f2dec06e8a0 2024-04-25T09:54:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.087+0000 7f2df3113640 1 -- 172.21.15.19:0/2129887094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dec073a00 con 0x7f2dec06e8a0 2024-04-25T09:54:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.087+0000 7f2de17fa640 1 -- 172.21.15.19:0/2129887094 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2de8030770 con 0x7f2dec06e8a0 2024-04-25T09:54:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.088+0000 7f2de17fa640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2dd806b4d0 0x7f2dd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.088+0000 7f2de17fa640 1 -- 172.21.15.19:0/2129887094 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2de80ab670 con 0x7f2dec06e8a0 2024-04-25T09:54:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.088+0000 7f2de3fff640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2dd806b4d0 0x7f2dd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.088+0000 7f2de3fff640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2dd806b4d0 0x7f2dd806d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2dec144000 tx=0x7f2ddc009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.092+0000 7f2de17fa640 1 -- 172.21.15.19:0/2129887094 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2de807a350 con 0x7f2dec06e8a0 2024-04-25T09:54:19.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:19.313+0000 7f2df3113640 1 -- 172.21.15.19:0/2129887094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2dec0ae370 con 0x7f2dec06e8a0 2024-04-25T09:54:20.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.239+0000 7f2de17fa640 1 -- 172.21.15.19:0/2129887094 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v17) v1 ==== 167+0+0 (secure 0 0 0) 0x7f2de8079c50 con 0x7f2dec06e8a0 2024-04-25T09:54:20.242 INFO:teuthology.orchestra.run.smithi019.stderr:enabled new snapshots 2024-04-25T09:54:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 -- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2dd806b4d0 msgr2=0x7f2dd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2dd806b4d0 0x7f2dd806d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2dec144000 tx=0x7f2ddc009060 comp rx=0 tx=0).stop 2024-04-25T09:54:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 -- 172.21.15.19:0/2129887094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec06e8a0 msgr2=0x7f2dec0cef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec06e8a0 0x7f2dec0cef80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2de8035040 tx=0x7f2de80046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 -- 172.21.15.19:0/2129887094 shutdown_connections 2024-04-25T09:54:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2dd806b4d0 0x7f2dd806d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dec144d50 0x7f2dec13ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dec1446d0 0x7f2dec13c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 --2- 172.21.15.19:0/2129887094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dec06e8a0 0x7f2dec0cef80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 -- 172.21.15.19:0/2129887094 >> 172.21.15.19:0/2129887094 conn(0x7f2dec06abe0 msgr2=0x7f2dec0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 -- 172.21.15.19:0/2129887094 shutdown_connections 2024-04-25T09:54:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:20.243+0000 7f2df3113640 1 -- 172.21.15.19:0/2129887094 wait complete. 2024-04-25T09:54:20.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:20.259 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T09:54:20.283 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:54:20.283 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:20.341 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T09:54:20.341 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T09:54:20.341 DEBUG:teuthology.orchestra.run.smithi019:> import uuid 2024-04-25T09:54:20.341 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:54:20.341 DEBUG:teuthology.orchestra.run.smithi019:> n = 2000 2024-04-25T09:54:20.341 DEBUG:teuthology.orchestra.run.smithi019:> create_hard_links = False 2024-04-25T09:54:20.341 DEBUG:teuthology.orchestra.run.smithi019:> if 3 > 0: 2024-04-25T09:54:20.341 DEBUG:teuthology.orchestra.run.smithi019:> create_hard_links = True 2024-04-25T09:54:20.341 DEBUG:teuthology.orchestra.run.smithi019:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> dpath = os.path.dirname(path) 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> fnameprefix = os.path.basename(path) 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> os.makedirs(dpath, exist_ok=True) 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> for i in range(n): 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> f.write(f"{i}") 2024-04-25T09:54:20.342 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> f.flush() 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> os.fsync(f.fileno()) 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> os.unlink(fpath) 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> os.fsync(dirfd) 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> if create_hard_links: 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> for j in range(3): 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> os.fsync(dirfd) 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> finally: 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> os.close(dirfd) 2024-04-25T09:54:20.343 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T09:54:36.901 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:36.904 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:37.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:54:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:54:37.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:37.067 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:54:37.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:37.120 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T09:55:04.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:04.395 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:55:04.460 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:55:04.460 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T09:55:04.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.573+0000 7f4b296e1640 1 Processor -- start 2024-04-25T09:55:04.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.573+0000 7f4b296e1640 1 -- start start 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b296e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b2415dc20 0x7f4b240ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b296e1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4b24161710 0x7f4b240cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b296e1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b240b9330 0x7f4b2406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b296e1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2415e000 con 0x7f4b24161710 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b296e1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b240d1de0 con 0x7f4b240b9330 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b296e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b24161c60 con 0x7f4b2415dc20 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b1bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4b24161710 0x7f4b240cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b1bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4b24161710 0x7f4b240cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37162/0 (socket says 172.21.15.19:37162) 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b1bfff640 1 -- 172.21.15.19:0/4290521315 learned_addr learned my addr 172.21.15.19:0/4290521315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/4290521315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b2415dc20 0x7f4b240ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b237fe640 1 --2- 172.21.15.19:0/4290521315 >> v2:172.21.15.112:3300/0 conn(0x7f4b240b9330 0x7f4b2406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b1bfff640 1 -- 172.21.15.19:0/4290521315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b2415dc20 msgr2=0x7f4b240ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/4290521315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b2415dc20 0x7f4b240ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b1bfff640 1 -- 172.21.15.19:0/4290521315 >> v2:172.21.15.112:3300/0 conn(0x7f4b240b9330 msgr2=0x7f4b2406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/4290521315 >> v2:172.21.15.112:3300/0 conn(0x7f4b240b9330 0x7f4b2406bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b1bfff640 1 -- 172.21.15.19:0/4290521315 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b24161f40 con 0x7f4b24161710 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.574+0000 7f4b237fe640 1 --2- 172.21.15.19:0/4290521315 >> v2:172.21.15.112:3300/0 conn(0x7f4b240b9330 0x7f4b2406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.575+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/4290521315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b24161710 0x7f4b240cebb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4b240b6570 tx=0x7f4b0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e117fb6a56bfd95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.575+0000 7f4b227fc640 1 -- 172.21.15.19:0/4290521315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c037070 con 0x7f4b24161710 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.575+0000 7f4b227fc640 1 -- 172.21.15.19:0/4290521315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c032440 con 0x7f4b24161710 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.575+0000 7f4b227fc640 1 -- 172.21.15.19:0/4290521315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c032810 con 0x7f4b24161710 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.576+0000 7f4b296e1640 1 -- 172.21.15.19:0/4290521315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b24161710 msgr2=0x7f4b240cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.576+0000 7f4b296e1640 1 --2- 172.21.15.19:0/4290521315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b24161710 0x7f4b240cebb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4b240b6570 tx=0x7f4b0c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.576+0000 7f4b296e1640 1 -- 172.21.15.19:0/4290521315 shutdown_connections 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.576+0000 7f4b296e1640 1 --2- 172.21.15.19:0/4290521315 >> v2:172.21.15.112:3300/0 conn(0x7f4b240b9330 0x7f4b2406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.576+0000 7f4b296e1640 1 --2- 172.21.15.19:0/4290521315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b24161710 0x7f4b240cebb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.576+0000 7f4b296e1640 1 --2- 172.21.15.19:0/4290521315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b2415dc20 0x7f4b240ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.576+0000 7f4b296e1640 1 -- 172.21.15.19:0/4290521315 >> 172.21.15.19:0/4290521315 conn(0x7f4b240d0420 msgr2=0x7f4b240d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.576+0000 7f4b296e1640 1 -- 172.21.15.19:0/4290521315 shutdown_connections 2024-04-25T09:55:04.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.576+0000 7f4b296e1640 1 -- 172.21.15.19:0/4290521315 wait complete. 2024-04-25T09:55:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.577+0000 7f4b296e1640 1 Processor -- start 2024-04-25T09:55:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.577+0000 7f4b296e1640 1 -- start start 2024-04-25T09:55:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b296e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b240b9330 0x7f4b240aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b296e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b2415dc20 0x7f4b241f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b296e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b24161710 0x7f4b2415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b296e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240b5e50 con 0x7f4b240b9330 2024-04-25T09:55:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b296e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b24162500 con 0x7f4b2415dc20 2024-04-25T09:55:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b296e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240b5fd0 con 0x7f4b24161710 2024-04-25T09:55:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b22ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b240b9330 0x7f4b240aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b22ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b240b9330 0x7f4b240aad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37170/0 (socket says 172.21.15.19:37170) 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b22ffd640 1 -- 172.21.15.19:0/126849549 learned_addr learned my addr 172.21.15.19:0/126849549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b2415dc20 0x7f4b241f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b237fe640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b24161710 0x7f4b2415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b22ffd640 1 -- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b24161710 msgr2=0x7f4b2415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b24161710 0x7f4b2415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b22ffd640 1 -- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b2415dc20 msgr2=0x7f4b241f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b2415dc20 0x7f4b241f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b22ffd640 1 -- 172.21.15.19:0/126849549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b240ab3d0 con 0x7f4b240b9330 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b2415dc20 0x7f4b241f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b237fe640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b24161710 0x7f4b2415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b240b9330 0x7f4b240aad50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400d5f0 tx=0x7f4b1400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.578+0000 7f4b1b7fe640 1 -- 172.21.15.19:0/126849549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b140125f0 con 0x7f4b240b9330 2024-04-25T09:55:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.579+0000 7f4b1b7fe640 1 -- 172.21.15.19:0/126849549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14019070 con 0x7f4b240b9330 2024-04-25T09:55:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.579+0000 7f4b296e1640 1 -- 172.21.15.19:0/126849549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b240ab6b0 con 0x7f4b240b9330 2024-04-25T09:55:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.579+0000 7f4b1b7fe640 1 -- 172.21.15.19:0/126849549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b140129d0 con 0x7f4b240b9330 2024-04-25T09:55:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.579+0000 7f4b296e1640 1 -- 172.21.15.19:0/126849549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b240abc70 con 0x7f4b240b9330 2024-04-25T09:55:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.580+0000 7f4b1b7fe640 1 -- 172.21.15.19:0/126849549 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b14012b70 con 0x7f4b240b9330 2024-04-25T09:55:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.580+0000 7f4b1b7fe640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b0406b4d0 0x7f4b0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.581+0000 7f4b1b7fe640 1 -- 172.21.15.19:0/126849549 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4b1408a860 con 0x7f4b240b9330 2024-04-25T09:55:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.581+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b0406b4d0 0x7f4b0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.581+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b0406b4d0 0x7f4b0406d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c034000 tx=0x7f4b0c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.582+0000 7f4b2c3e3740 1 Processor -- start 2024-04-25T09:55:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.582+0000 7f4b2c3e3740 1 -- start start 2024-04-25T09:55:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.582+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54120e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.582+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5415b370 0x562e5411f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.582+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e54149be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.582+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e54081b90 con 0x562e5411eca0 2024-04-25T09:55:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.582+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5402ae40 con 0x562e5415b370 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.582+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e5407dc30 con 0x562e54149760 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b22ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54120e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b22ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54120e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37180/0 (socket says 172.21.15.19:37180) 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b22ffd640 1 -- 172.21.15.19:0/1480878637 learned_addr learned my addr 172.21.15.19:0/1480878637 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/1480878637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5415b370 0x562e5411f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b237fe640 1 --2- 172.21.15.19:0/1480878637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e54149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b22ffd640 1 -- 172.21.15.19:0/1480878637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 msgr2=0x562e54149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/1480878637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e54149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b22ffd640 1 -- 172.21.15.19:0/1480878637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5415b370 msgr2=0x562e5411f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/1480878637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5415b370 0x562e5411f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b22ffd640 1 -- 172.21.15.19:0/1480878637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e53e607d0 con 0x562e5411eca0 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b237fe640 1 --2- 172.21.15.19:0/1480878637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e54149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:04.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/1480878637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54120e00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4b140551f0 tx=0x7f4b14059490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed1c3179f78c2bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b18ff9640 1 -- 172.21.15.19:0/1480878637 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b140614a0 con 0x562e5411eca0 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b18ff9640 1 -- 172.21.15.19:0/1480878637 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14056e30 con 0x562e5411eca0 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.583+0000 7f4b18ff9640 1 -- 172.21.15.19:0/1480878637 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1405f070 con 0x562e5411eca0 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.584+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/1480878637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 msgr2=0x562e54120e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.584+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/1480878637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54120e00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4b140551f0 tx=0x7f4b14059490 comp rx=0 tx=0).stop 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.584+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/1480878637 shutdown_connections 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.584+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/1480878637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e54149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.584+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/1480878637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5415b370 0x562e5411f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.584+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/1480878637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54120e00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.584+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/1480878637 >> 172.21.15.19:0/1480878637 conn(0x562e540e6870 msgr2=0x562e54164760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.584+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/1480878637 shutdown_connections 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.584+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/1480878637 wait complete. 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.584+0000 7f4b2c3e3740 1 -- start start 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b2c3e3740 1 Processor -- start 2024-04-25T09:55:04.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5411eca0 0x562e5414caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e541786f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5415b370 0x562e5415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5403e3e0 con 0x562e5415b370 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e541472b0 con 0x562e5411eca0 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e5415d1e0 con 0x562e54149760 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b2c3e3740 10 client.0 _ll_register_callbacks cb 0x562e5403ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5415b370 0x562e5415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5415b370 0x562e5415f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37182/0 (socket says 172.21.15.19:37182) 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b237fe640 1 -- 172.21.15.19:0/232990653 learned_addr learned my addr 172.21.15.19:0/232990653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5411eca0 0x562e5414caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.585+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e541786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b237fe640 1 -- 172.21.15.19:0/232990653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 msgr2=0x562e541786f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b237fe640 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e541786f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b237fe640 1 -- 172.21.15.19:0/232990653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5411eca0 msgr2=0x562e5414caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b237fe640 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5411eca0 0x562e5414caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b237fe640 1 -- 172.21.15.19:0/232990653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e53d560f0 con 0x562e5415b370 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b237fe640 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5415b370 0x562e5415f0d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0079d0 tx=0x7f4b1c007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9489bfcbecf938e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b01ffb640 1 -- 172.21.15.19:0/232990653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c01a030 con 0x562e5415b370 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b01ffb640 1 -- 172.21.15.19:0/232990653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4b1c010a70 con 0x562e5415b370 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/232990653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b14056e30 con 0x562e5415b370 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b01ffb640 1 -- 172.21.15.19:0/232990653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b1c023050 con 0x562e5415b370 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b2c3e3740 10 client.5478 fetch_fsmap learned FSMap version 17 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/232990653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x562e53ec0f60 con 0x562e5415b370 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b01ffb640 1 -- 172.21.15.19:0/232990653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14056e30 con 0x562e5415b370 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.586+0000 7f4b01ffb640 1 -- 172.21.15.19:0/232990653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4b1c026020 con 0x562e5415b370 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b2c3e3740 10 client.5478 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:55:04.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b2c3e3740 10 client.5478 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b2c3e3740 20 client.5478 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x562e5415c960 0x562e5415d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b2c3e3740 4 client.5478 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/232990653 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e5408f0d0 con 0x562e5415c960 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x562e5415c960 0x562e5415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x562e5415c960 0x562e5415d730 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204cf14fb5ba13e9 server_cookie=6f0669e5e00d089d in_seq=0 out_seq=0 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b01ffb640 10 client.5478 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b01ffb640 1 -- 172.21.15.19:0/232990653 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e5408f0d0 con 0x562e5415c960 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b01ffb640 10 client.5478 handle_command_reply: tid=0 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.587+0000 7f4b2c3e3740 1 client.5478 shutdown 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/232990653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5415b370 msgr2=0x562e5415f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5415b370 0x562e5415f0d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0079d0 tx=0x7f4b1c007e90 comp rx=0 tx=0).stop 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/232990653 shutdown_connections 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x562e5415c960 0x562e5415d730 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5415b370 0x562e5415f0d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e541786f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b01ffb640 0 client.5478 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:04.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/232990653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5411eca0 0x562e5414caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/232990653 >> 172.21.15.19:0/232990653 conn(0x562e540e6870 msgr2=0x562e54147490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/232990653 shutdown_connections 2024-04-25T09:55:04.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/232990653 wait complete. 2024-04-25T09:55:04.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.588+0000 7f4b2c3e3740 20 client.5478 trim_cache size 0 max 16384 2024-04-25T09:55:04.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.602+0000 7f4b2c3e3740 1 Processor -- start 2024-04-25T09:55:04.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.602+0000 7f4b2c3e3740 1 -- start start 2024-04-25T09:55:04.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e54149760 0x562e5415bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e5415b370 0x562e541871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b24200550 con 0x562e5411eca0 2024-04-25T09:55:04.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241ff700 con 0x562e54149760 2024-04-25T09:55:04.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240cbe90 con 0x562e5415b370 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b22ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b22ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37196/0 (socket says 172.21.15.19:37196) 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b22ffd640 1 -- 172.21.15.19:0/2350241229 learned_addr learned my addr 172.21.15.19:0/2350241229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b22ffd640 1 -- 172.21.15.19:0/2350241229 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e5415b370 msgr2=0x562e541871b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/2350241229 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e5415b370 0x562e541871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b22ffd640 1 -- 172.21.15.19:0/2350241229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e54149760 msgr2=0x562e5415bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/2350241229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e54149760 0x562e5415bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b22ffd640 1 -- 172.21.15.19:0/2350241229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e53d2f1b0 con 0x562e5411eca0 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.603+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/2350241229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177880 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4b14059640 tx=0x7f4b14020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e90a37c9978fc9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b03fff640 1 -- 172.21.15.19:0/2350241229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14075070 con 0x562e5411eca0 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b03fff640 1 -- 172.21.15.19:0/2350241229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14056e30 con 0x562e5411eca0 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b03fff640 1 -- 172.21.15.19:0/2350241229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14088040 con 0x562e5411eca0 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/2350241229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 msgr2=0x562e54177880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/2350241229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177880 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4b14059640 tx=0x7f4b14020000 comp rx=0 tx=0).stop 2024-04-25T09:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/2350241229 shutdown_connections 2024-04-25T09:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/2350241229 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e5415b370 0x562e541871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/2350241229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e54149760 0x562e5415bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/2350241229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177880 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/2350241229 >> 172.21.15.19:0/2350241229 conn(0x562e540e6870 msgr2=0x562e5415eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/2350241229 shutdown_connections 2024-04-25T09:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.604+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/2350241229 wait complete. 2024-04-25T09:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.605+0000 7f4b2c3e3740 1 -- start start 2024-04-25T09:55:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.605+0000 7f4b2c3e3740 1 Processor -- start 2024-04-25T09:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.605+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.605+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e5415ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.605+0000 7f4b2c3e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5415b370 0x562e54160470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.605+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5408f0d0 con 0x562e5411eca0 2024-04-25T09:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.605+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240cc490 con 0x562e5415b370 2024-04-25T09:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.605+0000 7f4b2c3e3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e5415d020 con 0x562e54149760 2024-04-25T09:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b2c3e3740 10 client.0 _ll_register_callbacks cb 0x7f4b24062560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b22ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b22ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37212/0 (socket says 172.21.15.19:37212) 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b22ffd640 1 -- 172.21.15.19:0/3261045514 learned_addr learned my addr 172.21.15.19:0/3261045514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b237fe640 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5415b370 0x562e54160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e5415ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b22ffd640 1 -- 172.21.15.19:0/3261045514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 msgr2=0x562e5415ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e5415ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b22ffd640 1 -- 172.21.15.19:0/3261045514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5415b370 msgr2=0x562e54160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5415b370 0x562e54160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b22ffd640 1 -- 172.21.15.19:0/3261045514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e541641c0 con 0x562e5411eca0 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e5415ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b22ffd640 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177810 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4b140614c0 tx=0x7f4b14010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e770c67c9c2ed506 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:55:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b01ffb640 1 -- 172.21.15.19:0/3261045514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14075070 con 0x562e5411eca0 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b01ffb640 1 -- 172.21.15.19:0/3261045514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4b14057110 con 0x562e5411eca0 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/3261045514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b14056e30 con 0x562e5411eca0 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b01ffb640 1 -- 172.21.15.19:0/3261045514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b14080050 con 0x562e5411eca0 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b2c3e3740 10 client.5484 fetch_fsmap learned FSMap version 17 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.606+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/3261045514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x562e54163c60 con 0x562e5411eca0 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b01ffb640 1 -- 172.21.15.19:0/3261045514 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14056e30 con 0x562e5411eca0 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b01ffb640 1 -- 172.21.15.19:0/3261045514 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4b14013070 con 0x562e5411eca0 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b2c3e3740 10 client.5484 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b2c3e3740 10 client.5484 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b2c3e3740 20 client.5484 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x562e541ce690 0x562e54167e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b2c3e3740 4 client.5484 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/3261045514 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x562e54089420 con 0x562e541ce690 2024-04-25T09:55:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x562e541ce690 0x562e54167e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b1bfff640 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x562e541ce690 0x562e54167e30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=256f3cb63e39584e server_cookie=64af9c9877e6fd1b in_seq=0 out_seq=0 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b01ffb640 10 client.5484 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b01ffb640 1 -- 172.21.15.19:0/3261045514 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17935 (crc 0 0 0) 0x562e54089420 con 0x562e541ce690 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.607+0000 7f4b01ffb640 10 client.5484 handle_command_reply: tid=0 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 1 client.5484 shutdown 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/3261045514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 msgr2=0x562e54177810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177810 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4b140614c0 tx=0x7f4b14010000 comp rx=0 tx=0).stop 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/3261045514 shutdown_connections 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e5415b370 0x562e54160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562e54149760 0x562e5415ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x562e541ce690 0x562e54167e30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 1 --2- 172.21.15.19:0/3261045514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562e5411eca0 0x562e54177810 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b01ffb640 0 client.5484 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/3261045514 >> 172.21.15.19:0/3261045514 conn(0x562e540e6870 msgr2=0x562e5415eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/3261045514 shutdown_connections 2024-04-25T09:55:04.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 1 -- 172.21.15.19:0/3261045514 wait complete. 2024-04-25T09:55:04.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b2c3e3740 20 client.5484 trim_cache size 0 max 16384 2024-04-25T09:55:04.617 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:55:04.617 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:55:04.617 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2084, 2024-04-25T09:55:04.617 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2083, 2024-04-25T09:55:04.617 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 522202, 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 42280779, 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 9, 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.296131332, 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.167541322, 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.046650793, 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.069541486, 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2082, 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.288099900, 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000138376 2024-04-25T09:55:04.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2, 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000218, 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000109 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1066, 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1080, 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 268330, 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 21523137, 2024-04-25T09:55:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.192970226, 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.120326320, 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.026225703, 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.038456195, 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1079, 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.208108739, 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000192871 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.620 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 104976 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 28787, 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 51306, 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 9687663, 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 54327903, 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 3.524365033, 2024-04-25T09:55:04.621 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 2.424921617, 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 1.674499222, 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.049747994, 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 51306, 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.180070062, 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000023000 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 74138, 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.009327068, 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000125 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 116256, 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 17584 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1231, 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 17.628179311, 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.014320210 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.623 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1956, 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.008165394, 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000515421 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 25266, 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 25266, 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 25266, 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 430.304080840, 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.017030953 2024-04-25T09:55:04.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 12, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 19831, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 19831, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 19831, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-04-25T09:55:04.625 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 173, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 173, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 2, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 45863, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 45467, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 107, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-04-25T09:55:04.626 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T09:55:04.627 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 1343, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 4052, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 4095, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 3287, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 23, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 13896 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-04-25T09:55:04.628 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 1841, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 4116, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 1829, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 2211, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-04-25T09:55:04.629 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 29386, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 29386, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-04-25T09:55:04.630 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 50, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 50, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 4, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 104234783, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 0, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2602, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 127.924858494, 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.049164050 2024-04-25T09:55:04.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 0 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 2014, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 2014, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 13, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 13, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 19831, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 19832, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 1, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 173, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 2946, 2024-04-25T09:55:04.632 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 2773, 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 120200, 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215396 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 28146, 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 25266, 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 3, 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2000, 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.661998200, 2024-04-25T09:55:04.633 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000830999 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 29, 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.004818557, 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000166157 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.634 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 6000, 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 6.876810329, 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001146135 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 7013, 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 6.773031530, 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000965782 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.635 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-04-25T09:55:04.636 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000045499, 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000045499 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-04-25T09:55:04.637 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.003302398, 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.003302398 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.023184236, 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.023184236 2024-04-25T09:55:04.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8, 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.531910094, 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.191488761 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2211, 2024-04-25T09:55:04.639 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 396.532234525, 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.179345198 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-04-25T09:55:04.640 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.048588885, 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.048588885 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-04-25T09:55:04.641 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000257739, 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000257739 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.642 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.643 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8000, 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 16.847898848, 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.002105987 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-04-25T09:55:04.644 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 1, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 18402, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 18402, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 42, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:55:04.645 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:55:04.646 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:55:04.647 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 16536822, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 6154, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 352, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 4, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T09:55:04.648 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 5520, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 78, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 20922970, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 870263, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T09:55:04.649 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 4628, 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 100414868, 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 37, 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 4628, 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4628, 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 23.307759533, 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.005036248 2024-04-25T09:55:04.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4628, 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 4714 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 4628, 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 35, 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 4593, 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 0, 2024-04-25T09:55:04.651 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 11, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 35, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 2637, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 24, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 1670, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 194, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 0, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 22, 2024-04-25T09:55:04.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-04-25T09:55:04.653 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 40, 2024-04-25T09:55:04.654 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 3, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 8, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 8, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 93, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 0, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 28 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 1, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-04-25T09:55:04.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 32, 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 1829, 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 1829, 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:55:04.656 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 31933, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 7599383, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 31933, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 31933, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 7599383, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.657 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 4628, 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 100612957, 2024-04-25T09:55:04.658 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 4628, 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 100612957, 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T09:55:04.659 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 4628, 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 4628, 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 4628, 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 4628, 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:55:04.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 1829, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 198089, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 1829, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 35, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 198089, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.661 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55f1b38246e0": { 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 95373, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 29386, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 100040479, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 29386, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2602, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 99945106, 2024-04-25T09:55:04.662 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:55:04.663 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:04.663 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:04.663 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:04.663 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:04.663 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:04.663 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:55:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 -- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b0406b4d0 msgr2=0x7f4b0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b0406b4d0 0x7f4b0406d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c034000 tx=0x7f4b0c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 -- 172.21.15.19:0/126849549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b240b9330 msgr2=0x7f4b240aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b240b9330 0x7f4b240aad50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400d5f0 tx=0x7f4b1400dab0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 -- 172.21.15.19:0/126849549 shutdown_connections 2024-04-25T09:55:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b0406b4d0 0x7f4b0406d990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b24161710 0x7f4b2415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b2415dc20 0x7f4b241f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 --2- 172.21.15.19:0/126849549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b240b9330 0x7f4b240aad50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 -- 172.21.15.19:0/126849549 >> 172.21.15.19:0/126849549 conn(0x7f4b240d0420 msgr2=0x7f4b240b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 -- 172.21.15.19:0/126849549 shutdown_connections 2024-04-25T09:55:04.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.608+0000 7f4b296e1640 1 -- 172.21.15.19:0/126849549 wait complete. 2024-04-25T09:55:04.666 INFO:tasks.cephfs.test_scrub_checks:num of strays: 1841 2024-04-25T09:55:04.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:04.666 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T09:55:04.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.768+0000 7f967903a640 1 Processor -- start 2024-04-25T09:55:04.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.768+0000 7f967903a640 1 -- start start 2024-04-25T09:55:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.769+0000 7f967903a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96741610f0 0x7f967406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.769+0000 7f967903a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f96741616f0 0x7f9674161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.769+0000 7f967903a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740ce0a0 0x7f96740ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.769+0000 7f967903a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96741614d0 con 0x7f96741610f0 2024-04-25T09:55:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.769+0000 7f967903a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96740c6960 con 0x7f96741616f0 2024-04-25T09:55:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.769+0000 7f967903a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9674162260 con 0x7f96740ce0a0 2024-04-25T09:55:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.769+0000 7f9672d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96741610f0 0x7f967406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.769+0000 7f9672d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96741610f0 0x7f967406c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37214/0 (socket says 172.21.15.19:37214) 2024-04-25T09:55:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.769+0000 7f9672d9d640 1 -- 172.21.15.19:0/1369446110 learned_addr learned my addr 172.21.15.19:0/1369446110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.769+0000 7f967259c640 1 --2- 172.21.15.19:0/1369446110 >> v2:172.21.15.112:3300/0 conn(0x7f96741616f0 0x7f9674161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.770+0000 7f967359e640 1 --2- 172.21.15.19:0/1369446110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740ce0a0 0x7f96740ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.770+0000 7f9672d9d640 1 -- 172.21.15.19:0/1369446110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740ce0a0 msgr2=0x7f96740ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.770+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1369446110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740ce0a0 0x7f96740ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.770+0000 7f9672d9d640 1 -- 172.21.15.19:0/1369446110 >> v2:172.21.15.112:3300/0 conn(0x7f96741616f0 msgr2=0x7f9674161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.770+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1369446110 >> v2:172.21.15.112:3300/0 conn(0x7f96741616f0 0x7f9674161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.770+0000 7f9672d9d640 1 -- 172.21.15.19:0/1369446110 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674162540 con 0x7f96741610f0 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.770+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1369446110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96741610f0 0x7f967406c1c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f967406f220 tx=0x7f966c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea00564dd05f7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.770+0000 7f9671d9b640 1 -- 172.21.15.19:0/1369446110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c037070 con 0x7f96741610f0 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.770+0000 7f9671d9b640 1 -- 172.21.15.19:0/1369446110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966c032440 con 0x7f96741610f0 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.770+0000 7f9671d9b640 1 -- 172.21.15.19:0/1369446110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c032810 con 0x7f96741610f0 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.771+0000 7f967903a640 1 -- 172.21.15.19:0/1369446110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96741610f0 msgr2=0x7f967406c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.771+0000 7f967903a640 1 --2- 172.21.15.19:0/1369446110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96741610f0 0x7f967406c1c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f967406f220 tx=0x7f966c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.771+0000 7f967903a640 1 -- 172.21.15.19:0/1369446110 shutdown_connections 2024-04-25T09:55:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.771+0000 7f967903a640 1 --2- 172.21.15.19:0/1369446110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740ce0a0 0x7f96740ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.771+0000 7f967903a640 1 --2- 172.21.15.19:0/1369446110 >> v2:172.21.15.112:3300/0 conn(0x7f96741616f0 0x7f9674161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.771+0000 7f967903a640 1 --2- 172.21.15.19:0/1369446110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96741610f0 0x7f967406c1c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.771+0000 7f967903a640 1 -- 172.21.15.19:0/1369446110 >> 172.21.15.19:0/1369446110 conn(0x7f967406f500 msgr2=0x7f967406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.771+0000 7f967903a640 1 -- 172.21.15.19:0/1369446110 shutdown_connections 2024-04-25T09:55:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.771+0000 7f967903a640 1 -- 172.21.15.19:0/1369446110 wait complete. 2024-04-25T09:55:04.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.773+0000 7f967903a640 1 Processor -- start 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.773+0000 7f967903a640 1 -- start start 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.773+0000 7f967903a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740ce0a0 0x7f96741f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.773+0000 7f967903a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96741610f0 0x7f9674155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.773+0000 7f967903a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96741616f0 0x7f96740ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.773+0000 7f967903a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96740cf690 con 0x7f96741616f0 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.773+0000 7f967903a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96740cfce0 con 0x7f96741610f0 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.773+0000 7f967903a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967406c700 con 0x7f96740ce0a0 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96741616f0 0x7f96740ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96741616f0 0x7f96740ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37230/0 (socket says 172.21.15.19:37230) 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967359e640 1 -- 172.21.15.19:0/1329789947 learned_addr learned my addr 172.21.15.19:0/1329789947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740ce0a0 0x7f96741f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967259c640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96741610f0 0x7f9674155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967359e640 1 -- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740ce0a0 msgr2=0x7f96741f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967359e640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740ce0a0 0x7f96741f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967359e640 1 -- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96741610f0 msgr2=0x7f9674155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967359e640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96741610f0 0x7f9674155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967359e640 1 -- 172.21.15.19:0/1329789947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674160030 con 0x7f96741616f0 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967259c640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96741610f0 0x7f9674155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967359e640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96741616f0 0x7f96740ae810 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9664007e60 tx=0x7f966400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f966b7fe640 1 -- 172.21.15.19:0/1329789947 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664014620 con 0x7f96741616f0 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f966b7fe640 1 -- 172.21.15.19:0/1329789947 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9664019070 con 0x7f96741616f0 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f966b7fe640 1 -- 172.21.15.19:0/1329789947 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664014a20 con 0x7f96741616f0 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967903a640 1 -- 172.21.15.19:0/1329789947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9674160310 con 0x7f96741616f0 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.774+0000 7f967903a640 1 -- 172.21.15.19:0/1329789947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96741608d0 con 0x7f96741616f0 2024-04-25T09:55:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.776+0000 7f966b7fe640 1 -- 172.21.15.19:0/1329789947 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9664014bc0 con 0x7f96741616f0 2024-04-25T09:55:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.776+0000 7f966b7fe640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.776+0000 7f966b7fe640 1 -- 172.21.15.19:0/1329789947 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f966408ac70 con 0x7f96741616f0 2024-04-25T09:55:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.776+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.777+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f966c034000 tx=0x7f966c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.777+0000 7f967be0f740 1 Processor -- start 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.777+0000 7f967be0f740 1 -- start start 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967be0f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8710 0x55e0e07eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967be0f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8af0 0x55e0e07eb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967be0f740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07bb370 0x55e0e07beb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967be0f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e0723bc0 con 0x55e0e07e8af0 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967be0f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e06cce70 con 0x55e0e07e8710 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967be0f740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0e071fc60 con 0x55e0e07bb370 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8af0 0x55e0e07eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f9672d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8710 0x55e0e07eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8af0 0x55e0e07eb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37244/0 (socket says 172.21.15.19:37244) 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f9672d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8710 0x55e0e07eaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:45664/0 (socket says 172.21.15.19:45664) 2024-04-25T09:55:04.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967259c640 1 -- 172.21.15.19:0/3851171452 learned_addr learned my addr 172.21.15.19:0/3851171452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967359e640 1 --2- 172.21.15.19:0/3851171452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07bb370 0x55e0e07beb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967259c640 1 -- 172.21.15.19:0/3851171452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07bb370 msgr2=0x55e0e07beb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967259c640 1 --2- 172.21.15.19:0/3851171452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07bb370 0x55e0e07beb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967259c640 1 -- 172.21.15.19:0/3851171452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8710 msgr2=0x55e0e07eaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967259c640 1 --2- 172.21.15.19:0/3851171452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8710 0x55e0e07eaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f967259c640 1 -- 172.21.15.19:0/3851171452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0e06edad0 con 0x55e0e07e8af0 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.778+0000 7f9672d9d640 1 --2- 172.21.15.19:0/3851171452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8710 0x55e0e07eaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f967259c640 1 --2- 172.21.15.19:0/3851171452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8af0 0x55e0e07eb460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f965c009910 tx=0x7f965c009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb98cc6d8aca817b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f9668ff9640 1 -- 172.21.15.19:0/3851171452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c017020 con 0x55e0e07e8af0 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f9668ff9640 1 -- 172.21.15.19:0/3851171452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c00e9b0 con 0x55e0e07e8af0 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f9668ff9640 1 -- 172.21.15.19:0/3851171452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00f040 con 0x55e0e07e8af0 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f967be0f740 1 -- 172.21.15.19:0/3851171452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8af0 msgr2=0x55e0e07eb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f967be0f740 1 --2- 172.21.15.19:0/3851171452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8af0 0x55e0e07eb460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f965c009910 tx=0x7f965c009dd0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f967be0f740 1 -- 172.21.15.19:0/3851171452 shutdown_connections 2024-04-25T09:55:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f967be0f740 1 --2- 172.21.15.19:0/3851171452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07bb370 0x55e0e07beb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f967be0f740 1 --2- 172.21.15.19:0/3851171452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8af0 0x55e0e07eb460 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f967be0f740 1 --2- 172.21.15.19:0/3851171452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8710 0x55e0e07eaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f967be0f740 1 -- 172.21.15.19:0/3851171452 >> 172.21.15.19:0/3851171452 conn(0x55e0e07bd5e0 msgr2=0x55e0e0805f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.779+0000 7f967be0f740 1 -- 172.21.15.19:0/3851171452 shutdown_connections 2024-04-25T09:55:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.780+0000 7f967be0f740 1 -- 172.21.15.19:0/3851171452 wait complete. 2024-04-25T09:55:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.780+0000 7f967be0f740 1 -- start start 2024-04-25T09:55:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.780+0000 7f967be0f740 1 Processor -- start 2024-04-25T09:55:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f967be0f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e07de4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f967be0f740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07e8af0 0x55e0e07df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f967be0f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e0809360 0x55e0e07bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f967be0f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e07c0790 con 0x55e0e07e8710 2024-04-25T09:55:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f967be0f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e06c24e0 con 0x55e0e0809360 2024-04-25T09:55:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f967be0f740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0e07c0a70 con 0x55e0e07e8af0 2024-04-25T09:55:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f967be0f740 10 client.0 _ll_register_callbacks cb 0x55e0e06dfde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f9672d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e07de4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f9672d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e07de4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37256/0 (socket says 172.21.15.19:37256) 2024-04-25T09:55:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f9672d9d640 1 -- 172.21.15.19:0/1561411292 learned_addr learned my addr 172.21.15.19:0/1561411292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f967259c640 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07e8af0 0x55e0e07df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f967359e640 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e0809360 0x55e0e07bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f9672d9d640 1 -- 172.21.15.19:0/1561411292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07e8af0 msgr2=0x55e0e07df3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07e8af0 0x55e0e07df3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f9672d9d640 1 -- 172.21.15.19:0/1561411292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e0809360 msgr2=0x55e0e07bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e0809360 0x55e0e07bfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f9672d9d640 1 -- 172.21.15.19:0/1561411292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0e0504940 con 0x55e0e07e8710 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e07de4d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f966c02a220 tx=0x7f966c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6439adae7c91c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.781+0000 7f9651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f9651ffb640 1 -- 172.21.15.19:0/1561411292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c037040 con 0x55e0e07e8710 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f9651ffb640 1 -- 172.21.15.19:0/1561411292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f966c03eb30 con 0x55e0e07e8710 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f967be0f740 1 -- 172.21.15.19:0/1561411292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f965c00e9b0 con 0x55e0e07e8710 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f9651ffb640 1 -- 172.21.15.19:0/1561411292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f966c057050 con 0x55e0e07e8710 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f967be0f740 10 client.5496 fetch_fsmap learned FSMap version 17 2024-04-25T09:55:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f967be0f740 1 -- 172.21.15.19:0/1561411292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55e0e05650d0 con 0x55e0e07e8710 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f9651ffb640 1 -- 172.21.15.19:0/1561411292 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c00e9b0 con 0x55e0e07e8710 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f9651ffb640 1 -- 172.21.15.19:0/1561411292 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f966c05a020 con 0x55e0e07e8710 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f967be0f740 10 client.5496 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f967be0f740 10 client.5496 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f967be0f740 20 client.5496 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f967be0f740 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55e0e0807050 0x55e0e0807450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f967be0f740 4 client.5496 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f967be0f740 1 -- 172.21.15.19:0/1561411292 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0e0731100 con 0x55e0e0807050 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f967259c640 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55e0e0807050 0x55e0e0807450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f9651ffb640 10 client.5496 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.782+0000 7f967259c640 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55e0e0807050 0x55e0e0807450 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6206c9d7cf76a28 server_cookie=56fba7bbeb9616d in_seq=0 out_seq=0 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f9651ffb640 1 -- 172.21.15.19:0/1561411292 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0e0731100 con 0x55e0e0807050 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f9651ffb640 10 client.5496 handle_command_reply: tid=0 2024-04-25T09:55:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 1 client.5496 shutdown 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 1 -- 172.21.15.19:0/1561411292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 msgr2=0x55e0e07de4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e07de4d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f966c02a220 tx=0x7f966c02c070 comp rx=0 tx=0).stop 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 1 -- 172.21.15.19:0/1561411292 shutdown_connections 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e0809360 0x55e0e07bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07e8af0 0x55e0e07df3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55e0e0807050 0x55e0e0807450 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 1 --2- 172.21.15.19:0/1561411292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e07de4d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f9651ffb640 0 client.5496 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 1 -- 172.21.15.19:0/1561411292 >> 172.21.15.19:0/1561411292 conn(0x55e0e07bb370 msgr2=0x55e0e07bc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 1 -- 172.21.15.19:0/1561411292 shutdown_connections 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 1 -- 172.21.15.19:0/1561411292 wait complete. 2024-04-25T09:55:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.783+0000 7f967be0f740 20 client.5496 trim_cache size 0 max 16384 2024-04-25T09:55:04.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.798+0000 7f967be0f740 1 Processor -- start 2024-04-25T09:55:04.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.798+0000 7f967be0f740 1 -- start start 2024-04-25T09:55:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967be0f740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07e8710 0x55e0e0828eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967be0f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8af0 0x55e0e0805f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967be0f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e0809360 0x55e0e0818260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967be0f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9674200470 con 0x55e0e0809360 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967be0f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96741ff6c0 con 0x55e0e07e8af0 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967be0f740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96741f7ec0 con 0x55e0e07e8710 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e0809360 0x55e0e0818260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e0809360 0x55e0e0818260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37258/0 (socket says 172.21.15.19:37258) 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967359e640 1 -- 172.21.15.19:0/2225959742 learned_addr learned my addr 172.21.15.19:0/2225959742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f9672d9d640 1 --2- 172.21.15.19:0/2225959742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07e8710 0x55e0e0828eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967359e640 1 -- 172.21.15.19:0/2225959742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07e8710 msgr2=0x55e0e0828eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967259c640 1 --2- 172.21.15.19:0/2225959742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8af0 0x55e0e0805f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967359e640 1 --2- 172.21.15.19:0/2225959742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07e8710 0x55e0e0828eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967359e640 1 -- 172.21.15.19:0/2225959742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8af0 msgr2=0x55e0e0805f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967359e640 1 --2- 172.21.15.19:0/2225959742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8af0 0x55e0e0805f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967359e640 1 -- 172.21.15.19:0/2225959742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0e03f8230 con 0x55e0e0809360 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f9672d9d640 1 --2- 172.21.15.19:0/2225959742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07e8710 0x55e0e0828eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.799+0000 7f967259c640 1 --2- 172.21.15.19:0/2225959742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8af0 0x55e0e0805f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f967359e640 1 --2- 172.21.15.19:0/2225959742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e0809360 0x55e0e0818260 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f96640591d0 tx=0x7f96640554e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113debc3ab8b8459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f9653fff640 1 -- 172.21.15.19:0/2225959742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664087070 con 0x55e0e0809360 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f9653fff640 1 -- 172.21.15.19:0/2225959742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966405d390 con 0x55e0e0809360 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f9653fff640 1 -- 172.21.15.19:0/2225959742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664061950 con 0x55e0e0809360 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f967be0f740 1 -- 172.21.15.19:0/2225959742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e0809360 msgr2=0x55e0e0818260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f967be0f740 1 --2- 172.21.15.19:0/2225959742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e0809360 0x55e0e0818260 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f96640591d0 tx=0x7f96640554e0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f967be0f740 1 -- 172.21.15.19:0/2225959742 shutdown_connections 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f967be0f740 1 --2- 172.21.15.19:0/2225959742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e0809360 0x55e0e0818260 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f967be0f740 1 --2- 172.21.15.19:0/2225959742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8af0 0x55e0e0805f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f967be0f740 1 --2- 172.21.15.19:0/2225959742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e07e8710 0x55e0e0828eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f967be0f740 1 -- 172.21.15.19:0/2225959742 >> 172.21.15.19:0/2225959742 conn(0x55e0e07bb370 msgr2=0x55e0e07e28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f967be0f740 1 -- 172.21.15.19:0/2225959742 shutdown_connections 2024-04-25T09:55:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.800+0000 7f967be0f740 1 -- 172.21.15.19:0/2225959742 wait complete. 2024-04-25T09:55:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.801+0000 7f967be0f740 1 -- start start 2024-04-25T09:55:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.801+0000 7f967be0f740 1 Processor -- start 2024-04-25T09:55:04.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f967be0f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e0864e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f967be0f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8af0 0x55e0e07e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f967be0f740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e0809360 0x55e0e07febc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f967be0f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967415ef90 con 0x55e0e07e8710 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f967be0f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e0808ec0 con 0x55e0e07e8af0 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f967be0f740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0e07dddf0 con 0x55e0e0809360 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f967be0f740 10 client.0 _ll_register_callbacks cb 0x7f966c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9672d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e0864e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9672d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e0864e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37262/0 (socket says 172.21.15.19:37262) 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9672d9d640 1 -- 172.21.15.19:0/1021354680 learned_addr learned my addr 172.21.15.19:0/1021354680 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f967259c640 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8af0 0x55e0e07e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f967359e640 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e0809360 0x55e0e07febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9672d9d640 1 -- 172.21.15.19:0/1021354680 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e0809360 msgr2=0x55e0e07febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e0809360 0x55e0e07febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9672d9d640 1 -- 172.21.15.19:0/1021354680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8af0 msgr2=0x55e0e07e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8af0 0x55e0e07e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9672d9d640 1 -- 172.21.15.19:0/1021354680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0e0818be0 con 0x55e0e07e8710 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9672d9d640 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e0864e50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f966c03c800 tx=0x7f966c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b9e2d092ccfa20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9651ffb640 1 -- 172.21.15.19:0/1021354680 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c037070 con 0x55e0e07e8710 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f9651ffb640 1 -- 172.21.15.19:0/1021354680 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f966c05cc50 con 0x55e0e07e8710 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.802+0000 7f967be0f740 1 -- 172.21.15.19:0/1021354680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f966405d390 con 0x55e0e07e8710 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f9651ffb640 1 -- 172.21.15.19:0/1021354680 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f966c05ce50 con 0x55e0e07e8710 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f967be0f740 10 client.5502 fetch_fsmap learned FSMap version 17 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f967be0f740 1 -- 172.21.15.19:0/1021354680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55e0e07fc940 con 0x55e0e07e8710 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f9651ffb640 1 -- 172.21.15.19:0/1021354680 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966405d390 con 0x55e0e07e8710 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f9651ffb640 1 -- 172.21.15.19:0/1021354680 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f966c063050 con 0x55e0e07e8710 2024-04-25T09:55:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f967be0f740 10 client.5502 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f967be0f740 10 client.5502 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f967be0f740 20 client.5502 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f967be0f740 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55e0e07ff100 0x55e0e07dcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f967be0f740 4 client.5502 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f967be0f740 1 -- 172.21.15.19:0/1021354680 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x55e0e072b450 con 0x55e0e07ff100 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f967259c640 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55e0e07ff100 0x55e0e07dcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f967259c640 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55e0e07ff100 0x55e0e07dcdc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6688cf37a596fdf6 server_cookie=7fbffc092b798288 in_seq=0 out_seq=0 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.803+0000 7f9651ffb640 10 client.5502 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f9651ffb640 1 -- 172.21.15.19:0/1021354680 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e0e072b450 con 0x55e0e07ff100 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f9651ffb640 10 client.5502 handle_command_reply: tid=0 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 1 client.5502 shutdown 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 1 -- 172.21.15.19:0/1021354680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 msgr2=0x55e0e0864e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e0864e50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f966c03c800 tx=0x7f966c004730 comp rx=0 tx=0).stop 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 1 -- 172.21.15.19:0/1021354680 shutdown_connections 2024-04-25T09:55:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e0e0809360 0x55e0e07febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e07e8af0 0x55e0e07e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55e0e07ff100 0x55e0e07dcdc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 1 --2- 172.21.15.19:0/1021354680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0e07e8710 0x55e0e0864e50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f9651ffb640 0 client.5502 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 1 -- 172.21.15.19:0/1021354680 >> 172.21.15.19:0/1021354680 conn(0x55e0e07bb370 msgr2=0x55e0e07e28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 1 -- 172.21.15.19:0/1021354680 shutdown_connections 2024-04-25T09:55:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 1 -- 172.21.15.19:0/1021354680 wait complete. 2024-04-25T09:55:04.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.811+0000 7f967be0f740 20 client.5502 trim_cache size 0 max 16384 2024-04-25T09:55:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:55:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:55:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "de52356d-8777-47b5-b284-856444e2d4c2", 2024-04-25T09:55:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:55:04.816 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:55:04.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 -- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 msgr2=0x7f965406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f966c034000 tx=0x7f966c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:04.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 -- 172.21.15.19:0/1329789947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96741616f0 msgr2=0x7f96740ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96741616f0 0x7f96740ae810 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9664007e60 tx=0x7f966400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 -- 172.21.15.19:0/1329789947 shutdown_connections 2024-04-25T09:55:04.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96741616f0 0x7f96740ae810 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96741610f0 0x7f9674155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 --2- 172.21.15.19:0/1329789947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740ce0a0 0x7f96741f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 -- 172.21.15.19:0/1329789947 >> 172.21.15.19:0/1329789947 conn(0x7f967406f500 msgr2=0x7f96740d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 -- 172.21.15.19:0/1329789947 shutdown_connections 2024-04-25T09:55:04.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.812+0000 7f967903a640 1 -- 172.21.15.19:0/1329789947 wait complete. 2024-04-25T09:55:04.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:04.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T09:55:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.950+0000 7f1807d09640 1 Processor -- start 2024-04-25T09:55:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.950+0000 7f1807d09640 1 -- start start 2024-04-25T09:55:04.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.951+0000 7f1807d09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015de80 0x7f18000cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.951+0000 7f1807d09640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18000cefb0 0x7f18000cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.951+0000 7f1807d09640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18000afc80 con 0x7f18000cf950 2024-04-25T09:55:04.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.951+0000 7f1807d09640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18000ce4d0 con 0x7f18000cefb0 2024-04-25T09:55:04.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.951+0000 7f1807d09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18000ce7b0 con 0x7f180015de80 2024-04-25T09:55:04.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.951+0000 7f1805aa5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015de80 0x7f18000cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.951+0000 7f18062a6640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f18000cf950 0x7f180006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:51438/0 (socket says 172.21.15.19:51438) 2024-04-25T09:55:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.951+0000 7f18062a6640 1 -- 172.21.15.19:0/3111898948 learned_addr learned my addr 172.21.15.19:0/3111898948 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.951+0000 7f18052a4640 1 --2- 172.21.15.19:0/3111898948 >> v2:172.21.15.112:3300/0 conn(0x7f18000cefb0 0x7f18000cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.952+0000 7f1804aa3640 1 -- 172.21.15.19:0/3111898948 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617269937 0 0) 0x7f18000afc80 con 0x7f18000cf950 2024-04-25T09:55:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.952+0000 7f1804aa3640 1 -- 172.21.15.19:0/3111898948 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17ec003520 con 0x7f18000cf950 2024-04-25T09:55:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.952+0000 7f1805aa5640 1 -- 172.21.15.19:0/3111898948 >> v2:172.21.15.112:3300/0 conn(0x7f18000cefb0 msgr2=0x7f18000cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.952+0000 7f1805aa5640 1 --2- 172.21.15.19:0/3111898948 >> v2:172.21.15.112:3300/0 conn(0x7f18000cefb0 0x7f18000cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.952+0000 7f1805aa5640 1 -- 172.21.15.19:0/3111898948 >> v1:172.21.15.112:6789/0 conn(0x7f18000cf950 legacy=0x7f180006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.952+0000 7f1805aa5640 1 -- 172.21.15.19:0/3111898948 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18000cea90 con 0x7f180015de80 2024-04-25T09:55:04.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.952+0000 7f18052a4640 1 --2- 172.21.15.19:0/3111898948 >> v2:172.21.15.112:3300/0 conn(0x7f18000cefb0 0x7f18000cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:04.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.953+0000 7f1805aa5640 1 --2- 172.21.15.19:0/3111898948 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015de80 0x7f18000cadb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0097a0 tx=0x7f17fc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777369f896988e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.953+0000 7f1804aa3640 1 -- 172.21.15.19:0/3111898948 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc037070 con 0x7f180015de80 2024-04-25T09:55:04.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.953+0000 7f1804aa3640 1 -- 172.21.15.19:0/3111898948 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc032440 con 0x7f180015de80 2024-04-25T09:55:04.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.953+0000 7f1804aa3640 1 -- 172.21.15.19:0/3111898948 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc032810 con 0x7f180015de80 2024-04-25T09:55:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.954+0000 7f1807d09640 1 -- 172.21.15.19:0/3111898948 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015de80 msgr2=0x7f18000cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.954+0000 7f1807d09640 1 --2- 172.21.15.19:0/3111898948 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015de80 0x7f18000cadb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0097a0 tx=0x7f17fc02a180 comp rx=0 tx=0).stop 2024-04-25T09:55:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.954+0000 7f1807d09640 1 -- 172.21.15.19:0/3111898948 shutdown_connections 2024-04-25T09:55:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.954+0000 7f1807d09640 1 --2- 172.21.15.19:0/3111898948 >> v2:172.21.15.112:3300/0 conn(0x7f18000cefb0 0x7f18000cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.954+0000 7f1807d09640 1 --2- 172.21.15.19:0/3111898948 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015de80 0x7f18000cadb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.954+0000 7f1807d09640 1 -- 172.21.15.19:0/3111898948 >> 172.21.15.19:0/3111898948 conn(0x7f18000b2390 msgr2=0x7f18000b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.954+0000 7f1807d09640 1 -- 172.21.15.19:0/3111898948 shutdown_connections 2024-04-25T09:55:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.954+0000 7f1807d09640 1 -- 172.21.15.19:0/3111898948 wait complete. 2024-04-25T09:55:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.956+0000 7f1807d09640 1 Processor -- start 2024-04-25T09:55:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.956+0000 7f1807d09640 1 -- start start 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1807d09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000cefb0 0x7f18000cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1807d09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18000cf950 0x7f18000cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1807d09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f180015de80 0x7f18000b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1807d09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18000b3030 con 0x7f18000cf950 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1807d09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18000b2eb0 con 0x7f180015de80 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1807d09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18000c5600 con 0x7f18000cefb0 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1805aa5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000cefb0 0x7f18000cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1805aa5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000cefb0 0x7f18000cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:54546/0 (socket says 172.21.15.19:54546) 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f18062a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f180015de80 0x7f18000b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1805aa5640 1 -- 172.21.15.19:0/1949582547 learned_addr learned my addr 172.21.15.19:0/1949582547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f18052a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18000cf950 0x7f18000cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1805aa5640 1 -- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f180015de80 msgr2=0x7f18000b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1805aa5640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f180015de80 0x7f18000b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1805aa5640 1 -- 172.21.15.19:0/1949582547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18000cf950 msgr2=0x7f18000cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1805aa5640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18000cf950 0x7f18000cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.957+0000 7f1805aa5640 1 -- 172.21.15.19:0/1949582547 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18000c9090 con 0x7f18000cefb0 2024-04-25T09:55:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.958+0000 7f1805aa5640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000cefb0 0x7f18000cb540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17fc000c00 tx=0x7f17fc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.958+0000 7f17f67fc640 1 -- 172.21.15.19:0/1949582547 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc041020 con 0x7f18000cefb0 2024-04-25T09:55:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.958+0000 7f17f67fc640 1 -- 172.21.15.19:0/1949582547 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc03d640 con 0x7f18000cefb0 2024-04-25T09:55:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.958+0000 7f17f67fc640 1 -- 172.21.15.19:0/1949582547 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc037040 con 0x7f18000cefb0 2024-04-25T09:55:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.958+0000 7f1807d09640 1 -- 172.21.15.19:0/1949582547 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18000c9370 con 0x7f18000cefb0 2024-04-25T09:55:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.958+0000 7f1807d09640 1 -- 172.21.15.19:0/1949582547 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18000c9930 con 0x7f18000cefb0 2024-04-25T09:55:04.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.961+0000 7f17f67fc640 1 -- 172.21.15.19:0/1949582547 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17fc044020 con 0x7f18000cefb0 2024-04-25T09:55:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.962+0000 7f17f67fc640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.962+0000 7f17f67fc640 1 -- 172.21.15.19:0/1949582547 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f17fc0aaa80 con 0x7f18000cefb0 2024-04-25T09:55:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.962+0000 7f18052a4640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.962+0000 7f180aa12740 1 Processor -- start 2024-04-25T09:55:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.962+0000 7f180aa12740 1 -- start start 2024-04-25T09:55:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f18052a4640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 0x7f17e406d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009560 tx=0x7f17f0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f180aa12740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 0x55c729ade320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f180aa12740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 0x55c729ade860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f180aa12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729aae770 0x55c729ab1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f180aa12740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c729a17da0 con 0x55c729adbb10 2024-04-25T09:55:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f180aa12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7299c0b80 con 0x55c729aae770 2024-04-25T09:55:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f180aa12740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c729a13e40 con 0x55c729adbef0 2024-04-25T09:55:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f1805aa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 0x55c729ade320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f18052a4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 0x55c729ade860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f1805aa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 0x55c729ade320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37286/0 (socket says 172.21.15.19:37286) 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f1805aa5640 1 -- 172.21.15.19:0/1136871994 learned_addr learned my addr 172.21.15.19:0/1136871994 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.963+0000 7f18062a6640 1 --2- 172.21.15.19:0/1136871994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729aae770 0x55c729ab1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.964+0000 7f1805aa5640 1 -- 172.21.15.19:0/1136871994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 msgr2=0x55c729ade860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.964+0000 7f1805aa5640 1 --2- 172.21.15.19:0/1136871994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 0x55c729ade860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.964+0000 7f1805aa5640 1 -- 172.21.15.19:0/1136871994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729aae770 msgr2=0x55c729ab1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.964+0000 7f1805aa5640 1 --2- 172.21.15.19:0/1136871994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729aae770 0x55c729ab1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.964+0000 7f1805aa5640 1 -- 172.21.15.19:0/1136871994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7299caf10 con 0x55c729adbb10 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.964+0000 7f18062a6640 1 --2- 172.21.15.19:0/1136871994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729aae770 0x55c729ab1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.964+0000 7f1805aa5640 1 --2- 172.21.15.19:0/1136871994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 0x55c729ade320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0a8d30 tx=0x7f17fc0789d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300f4e1b39651966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.964+0000 7f17e37fe640 1 -- 172.21.15.19:0/1136871994 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc095070 con 0x55c729adbb10 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.965+0000 7f17e37fe640 1 -- 172.21.15.19:0/1136871994 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc0aa040 con 0x55c729adbb10 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.965+0000 7f17e37fe640 1 -- 172.21.15.19:0/1136871994 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc090070 con 0x55c729adbb10 2024-04-25T09:55:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.965+0000 7f180aa12740 1 -- 172.21.15.19:0/1136871994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 msgr2=0x55c729ade320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.965+0000 7f180aa12740 1 --2- 172.21.15.19:0/1136871994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 0x55c729ade320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0a8d30 tx=0x7f17fc0789d0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.965+0000 7f180aa12740 1 -- 172.21.15.19:0/1136871994 shutdown_connections 2024-04-25T09:55:04.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.965+0000 7f180aa12740 1 --2- 172.21.15.19:0/1136871994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729aae770 0x55c729ab1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.965+0000 7f180aa12740 1 --2- 172.21.15.19:0/1136871994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 0x55c729ade860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.965+0000 7f180aa12740 1 --2- 172.21.15.19:0/1136871994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 0x55c729ade320 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.965+0000 7f180aa12740 1 -- 172.21.15.19:0/1136871994 >> 172.21.15.19:0/1136871994 conn(0x55c729ab09e0 msgr2=0x55c729af9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.965+0000 7f180aa12740 1 -- 172.21.15.19:0/1136871994 shutdown_connections 2024-04-25T09:55:04.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.966+0000 7f180aa12740 1 -- 172.21.15.19:0/1136871994 wait complete. 2024-04-25T09:55:04.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.966+0000 7f180aa12740 1 -- start start 2024-04-25T09:55:04.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.967+0000 7f180aa12740 1 Processor -- start 2024-04-25T09:55:04.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.968+0000 7f180aa12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 0x55c729ae0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.968+0000 7f180aa12740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 0x55c729afb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.968+0000 7f180aa12740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729ad2d10 0x55c729afbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.968+0000 7f1805aa5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 0x55c729ae0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.968+0000 7f1805aa5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 0x55c729ae0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:45700/0 (socket says 172.21.15.19:45700) 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.968+0000 7f18052a4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 0x55c729afb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.968+0000 7f18052a4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 0x55c729afb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:54562/0 (socket says 172.21.15.19:54562) 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.968+0000 7f1805aa5640 1 -- 172.21.15.19:0/1642452760 learned_addr learned my addr 172.21.15.19:0/1642452760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.968+0000 7f18062a6640 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729ad2d10 0x55c729afbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.968+0000 7f180aa12740 1 -- 172.21.15.19:0/1642452760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c729a17da0 con 0x55c729ad2d10 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.968+0000 7f180aa12740 1 -- 172.21.15.19:0/1642452760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7299b64b0 con 0x55c729adbb10 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f180aa12740 1 -- 172.21.15.19:0/1642452760 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c729ab3b90 con 0x55c729adbef0 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f180aa12740 10 client.0 _ll_register_callbacks cb 0x55c7299f6180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f1805aa5640 1 -- 172.21.15.19:0/1642452760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 msgr2=0x55c729afb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f1805aa5640 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 0x55c729afb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f1805aa5640 1 -- 172.21.15.19:0/1642452760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729ad2d10 msgr2=0x55c729afbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f1805aa5640 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729ad2d10 0x55c729afbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f1805aa5640 1 -- 172.21.15.19:0/1642452760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7297f67d0 con 0x55c729adbb10 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f18052a4640 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 0x55c729afb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f180aa12740 1 -- 172.21.15.19:0/1642452760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17fc0aa040 con 0x55c729adbb10 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f1805aa5640 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 0x55c729ae0110 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0754d0 tx=0x7f17fc047000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a6c9d0e77d53c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f17e0ff9640 10 client.5329 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.969+0000 7f17e0ff9640 1 -- 172.21.15.19:0/1642452760 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc09c060 con 0x55c729adbb10 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.970+0000 7f17e0ff9640 1 -- 172.21.15.19:0/1642452760 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f17fc081240 con 0x55c729adbb10 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.970+0000 7f17e0ff9640 1 -- 172.21.15.19:0/1642452760 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17fc095040 con 0x55c729adbb10 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.970+0000 7f180aa12740 10 client.5329 fetch_fsmap learned FSMap version 17 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.970+0000 7f180aa12740 1 -- 172.21.15.19:0/1642452760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=17}) v3 -- 0x55c72972c3f0 con 0x55c729adbb10 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.970+0000 7f17e0ff9640 1 -- 172.21.15.19:0/1642452760 <== mon.1 v2:172.21.15.112:3300/0 4 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f17fc07b030 con 0x55c729adbb10 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.970+0000 7f180aa12740 10 client.5329 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.970+0000 7f180aa12740 10 client.5329 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:55:04.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.970+0000 7f180aa12740 20 client.5329 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.970+0000 7f180aa12740 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55c729ad2380 0x55c729b16700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.970+0000 7f180aa12740 4 client.5329 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.971+0000 7f180aa12740 1 -- 172.21.15.19:0/1642452760 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c729a252e0 con 0x55c729ad2380 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.971+0000 7f18052a4640 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55c729ad2380 0x55c729b16700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.971+0000 7f18052a4640 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55c729ad2380 0x55c729b16700 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d87d89b7d1c665be server_cookie=14300431e80722d0 in_seq=0 out_seq=0 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.971+0000 7f17e0ff9640 10 client.5329 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f17e0ff9640 1 -- 172.21.15.19:0/1642452760 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c729a252e0 con 0x55c729ad2380 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f17e0ff9640 10 client.5329 handle_command_reply: tid=0 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f180aa12740 1 client.5329 shutdown 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f180aa12740 1 -- 172.21.15.19:0/1642452760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 msgr2=0x55c729ae0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f180aa12740 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 0x55c729ae0110 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0754d0 tx=0x7f17fc047000 comp rx=0 tx=0).stop 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f180aa12740 1 -- 172.21.15.19:0/1642452760 shutdown_connections 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f180aa12740 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55c729ad2380 0x55c729b16700 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f17e0ff9640 0 client.5329 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f180aa12740 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729ad2d10 0x55c729afbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f180aa12740 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729adbef0 0x55c729afb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f180aa12740 1 --2- 172.21.15.19:0/1642452760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 0x55c729ae0110 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f180aa12740 1 -- 172.21.15.19:0/1642452760 >> 172.21.15.19:0/1642452760 conn(0x55c729aae770 msgr2=0x55c729ab3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f180aa12740 1 -- 172.21.15.19:0/1642452760 shutdown_connections 2024-04-25T09:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.972+0000 7f180aa12740 1 -- 172.21.15.19:0/1642452760 wait complete. 2024-04-25T09:55:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:04.973+0000 7f180aa12740 20 client.5329 trim_cache size 0 max 16384 2024-04-25T09:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.005+0000 7f180aa12740 1 Processor -- start 2024-04-25T09:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.005+0000 7f180aa12740 1 -- start start 2024-04-25T09:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.006+0000 7f180aa12740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 0x55c729ab3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.006+0000 7f180aa12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 0x55c729aaeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.006+0000 7f180aa12740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbef0 0x55c729aaf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.006+0000 7f180aa12740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18002047c0 con 0x55c729adbef0 2024-04-25T09:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.006+0000 7f180aa12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1800203980 con 0x55c729adbb10 2024-04-25T09:55:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.006+0000 7f180aa12740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18000ae970 con 0x55c729ad2d10 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.006+0000 7f18062a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbef0 0x55c729aaf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.006+0000 7f18062a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbef0 0x55c729aaf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37312/0 (socket says 172.21.15.19:37312) 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.006+0000 7f18062a6640 1 -- 172.21.15.19:0/3384094402 learned_addr learned my addr 172.21.15.19:0/3384094402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.006+0000 7f18052a4640 1 --2- 172.21.15.19:0/3384094402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 0x55c729aaeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.006+0000 7f1805aa5640 1 --2- 172.21.15.19:0/3384094402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 0x55c729ab3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.007+0000 7f18052a4640 1 -- 172.21.15.19:0/3384094402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 msgr2=0x55c729ab3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.007+0000 7f18052a4640 1 --2- 172.21.15.19:0/3384094402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 0x55c729ab3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.007+0000 7f18052a4640 1 -- 172.21.15.19:0/3384094402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbef0 msgr2=0x55c729aaf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.007+0000 7f18052a4640 1 --2- 172.21.15.19:0/3384094402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbef0 0x55c729aaf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.007+0000 7f18052a4640 1 -- 172.21.15.19:0/3384094402 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7296ec0f0 con 0x55c729adbb10 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.007+0000 7f18062a6640 1 --2- 172.21.15.19:0/3384094402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbef0 0x55c729aaf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.007+0000 7f18052a4640 1 --2- 172.21.15.19:0/3384094402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 0x55c729aaeb50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55c729adf1c0 tx=0x7f17f0019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df339bad6d983e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.007+0000 7f17e2ffd640 1 -- 172.21.15.19:0/3384094402 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0003a60 con 0x55c729adbb10 2024-04-25T09:55:05.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.007+0000 7f17e2ffd640 1 -- 172.21.15.19:0/3384094402 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f0031070 con 0x55c729adbb10 2024-04-25T09:55:05.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.007+0000 7f17e2ffd640 1 -- 172.21.15.19:0/3384094402 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0003e00 con 0x55c729adbb10 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.008+0000 7f180aa12740 1 -- 172.21.15.19:0/3384094402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 msgr2=0x55c729aaeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.008+0000 7f180aa12740 1 --2- 172.21.15.19:0/3384094402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 0x55c729aaeb50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55c729adf1c0 tx=0x7f17f0019000 comp rx=0 tx=0).stop 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.008+0000 7f180aa12740 1 -- 172.21.15.19:0/3384094402 shutdown_connections 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.008+0000 7f180aa12740 1 --2- 172.21.15.19:0/3384094402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbef0 0x55c729aaf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.008+0000 7f180aa12740 1 --2- 172.21.15.19:0/3384094402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbb10 0x55c729aaeb50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.008+0000 7f180aa12740 1 --2- 172.21.15.19:0/3384094402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 0x55c729ab3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.008+0000 7f180aa12740 1 -- 172.21.15.19:0/3384094402 >> 172.21.15.19:0/3384094402 conn(0x55c729aae770 msgr2=0x55c729aa9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.008+0000 7f180aa12740 1 -- 172.21.15.19:0/3384094402 shutdown_connections 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.008+0000 7f180aa12740 1 -- 172.21.15.19:0/3384094402 wait complete. 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.009+0000 7f180aa12740 1 -- start start 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.009+0000 7f180aa12740 1 Processor -- start 2024-04-25T09:55:05.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f180aa12740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 0x55c729ae0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f180aa12740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 0x55c729aaef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f180aa12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbef0 0x55c729b625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f180aa12740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c729af62b0 con 0x55c729adbb10 2024-04-25T09:55:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f180aa12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18000aef70 con 0x55c729adbef0 2024-04-25T09:55:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f180aa12740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c729ae00c0 con 0x55c729ad2d10 2024-04-25T09:55:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f180aa12740 10 client.0 _ll_register_callbacks cb 0x7f17fc009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f1805aa5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 0x55c729ae0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f1805aa5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 0x55c729ae0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:54580/0 (socket says 172.21.15.19:54580) 2024-04-25T09:55:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f1805aa5640 1 -- 172.21.15.19:0/3495112238 learned_addr learned my addr 172.21.15.19:0/3495112238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f18062a6640 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbef0 0x55c729b625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f18052a4640 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 0x55c729aaef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f1805aa5640 1 -- 172.21.15.19:0/3495112238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbef0 msgr2=0x55c729b625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f1805aa5640 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbef0 0x55c729b625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f1805aa5640 1 -- 172.21.15.19:0/3495112238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 msgr2=0x55c729aaef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f1805aa5640 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 0x55c729aaef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f1805aa5640 1 -- 172.21.15.19:0/3495112238 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c729aaf490 con 0x55c729ad2d10 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.011+0000 7f18052a4640 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 0x55c729aaef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.012+0000 7f1805aa5640 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 0x55c729ae0d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f17fc078d70 tx=0x7f17fc009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb788ec881ed40a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.012+0000 7f17e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3301/0 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.012+0000 7f17e0ff9640 1 -- 172.21.15.19:0/3495112238 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc09c040 con 0x55c729ad2d10 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.012+0000 7f180aa12740 1 -- 172.21.15.19:0/3495112238 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17f0031070 con 0x55c729ad2d10 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.012+0000 7f17e0ff9640 1 -- 172.21.15.19:0/3495112238 <== mon.2 v2:172.21.15.112:3301/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f17fc0784c0 con 0x55c729ad2d10 2024-04-25T09:55:05.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.012+0000 7f17e0ff9640 1 -- 172.21.15.19:0/3495112238 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17fc081720 con 0x55c729ad2d10 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.012+0000 7f180aa12740 10 client.4985 fetch_fsmap learned FSMap version 17 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.012+0000 7f180aa12740 1 -- 172.21.15.19:0/3495112238 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55c729aaf490 con 0x55c729ad2d10 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.013+0000 7f17e0ff9640 1 -- 172.21.15.19:0/3495112238 <== mon.2 v2:172.21.15.112:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f0031070 con 0x55c729ad2d10 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.013+0000 7f17e0ff9640 1 -- 172.21.15.19:0/3495112238 <== mon.2 v2:172.21.15.112:3301/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f17fc07b040 con 0x55c729ad2d10 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.013+0000 7f180aa12740 10 client.4985 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.013+0000 7f180aa12740 10 client.4985 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.013+0000 7f180aa12740 20 client.4985 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.013+0000 7f180aa12740 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55c729b62b20 0x55c729af9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.013+0000 7f180aa12740 4 client.4985 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.013+0000 7f180aa12740 1 -- 172.21.15.19:0/3495112238 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c729af67f0 con 0x55c729b62b20 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.013+0000 7f18052a4640 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55c729b62b20 0x55c729af9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.014+0000 7f17e0ff9640 10 client.4985 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.014+0000 7f18052a4640 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55c729b62b20 0x55c729af9600 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e4fdfad62cabf8 server_cookie=5edff7da0893dc94 in_seq=0 out_seq=0 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.014+0000 7f17e0ff9640 1 -- 172.21.15.19:0/3495112238 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+274 (crc 0 0 0) 0x55c729af67f0 con 0x55c729b62b20 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.014+0000 7f17e0ff9640 10 client.4985 handle_command_reply: tid=0 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.014+0000 7f180aa12740 1 client.4985 shutdown 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f180aa12740 1 -- 172.21.15.19:0/3495112238 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 msgr2=0x55c729ae0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f180aa12740 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 0x55c729ae0d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f17fc078d70 tx=0x7f17fc009190 comp rx=0 tx=0).stop 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f180aa12740 1 -- 172.21.15.19:0/3495112238 shutdown_connections 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f180aa12740 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c729adbef0 0x55c729b625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f180aa12740 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55c729b62b20 0x55c729af9600 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f180aa12740 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c729adbb10 0x55c729aaef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f180aa12740 1 --2- 172.21.15.19:0/3495112238 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c729ad2d10 0x55c729ae0d60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f180aa12740 1 -- 172.21.15.19:0/3495112238 >> 172.21.15.19:0/3495112238 conn(0x55c729aae770 msgr2=0x55c729adf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:05.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f17e0ff9640 0 client.4985 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:05.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f180aa12740 1 -- 172.21.15.19:0/3495112238 shutdown_connections 2024-04-25T09:55:05.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f180aa12740 1 -- 172.21.15.19:0/3495112238 wait complete. 2024-04-25T09:55:05.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.015+0000 7f180aa12740 20 client.4985 trim_cache size 0 max 16384 2024-04-25T09:55:05.023 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:55:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (680 inodes in the stack)", 2024-04-25T09:55:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T09:55:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "de52356d-8777-47b5-b284-856444e2d4c2": { 2024-04-25T09:55:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T09:55:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "de52356d-8777-47b5-b284-856444e2d4c2", 2024-04-25T09:55:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T09:55:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:05.024 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:55:05.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 -- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 msgr2=0x7f17e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 0x7f17e406d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009560 tx=0x7f17f0009190 comp rx=0 tx=0).stop 2024-04-25T09:55:05.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 -- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000cefb0 msgr2=0x7f18000cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000cefb0 0x7f18000cb540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17fc000c00 tx=0x7f17fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 -- 172.21.15.19:0/1949582547 shutdown_connections 2024-04-25T09:55:05.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f180015de80 0x7f18000b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18000cf950 0x7f18000cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 --2- 172.21.15.19:0/1949582547 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000cefb0 0x7f18000cb540 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 -- 172.21.15.19:0/1949582547 >> 172.21.15.19:0/1949582547 conn(0x7f18000b2390 msgr2=0x7f18000b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:05.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 -- 172.21.15.19:0/1949582547 shutdown_connections 2024-04-25T09:55:05.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:05.016+0000 7f1807d09640 1 -- 172.21.15.19:0/1949582547 wait complete. 2024-04-25T09:55:05.033 INFO:tasks.cephfs.filesystem:scrub status for tag:de52356d-8777-47b5-b284-856444e2d4c2 - {'path': '/', 'tag': 'de52356d-8777-47b5-b284-856444e2d4c2', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T09:55:07.016 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:07.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:35.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:35.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T09:55:35.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.148+0000 7f9b1484d640 1 Processor -- start 2024-04-25T09:55:35.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.148+0000 7f9b1484d640 1 -- start start 2024-04-25T09:55:35.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.149+0000 7f9b1484d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b1015dac0 0x7f9b100d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.149+0000 7f9b1484d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b100d1f30 0x7f9b100d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.149+0000 7f9b1484d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b100d2420 con 0x7f9b100d34c0 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.149+0000 7f9b1484d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b100d3030 con 0x7f9b1015dac0 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.149+0000 7f9b1484d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b100d2700 con 0x7f9b100d1f30 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.149+0000 7f9b0e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b1015dac0 0x7f9b100d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.149+0000 7f9b0ed9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9b100d34c0 0x7f9b100b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54392/0 (socket says 172.21.15.19:54392) 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.149+0000 7f9b0ed9d640 1 -- 172.21.15.19:0/262614179 learned_addr learned my addr 172.21.15.19:0/262614179 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.149+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/262614179 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b100d1f30 0x7f9b100d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.150+0000 7f9b0e59c640 1 -- 172.21.15.19:0/262614179 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b100d1f30 msgr2=0x7f9b100d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.150+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/262614179 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b100d1f30 0x7f9b100d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.150+0000 7f9b0e59c640 1 -- 172.21.15.19:0/262614179 >> v1:172.21.15.19:6789/0 conn(0x7f9b100d34c0 legacy=0x7f9b100b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.150+0000 7f9b0e59c640 1 -- 172.21.15.19:0/262614179 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b100d29e0 con 0x7f9b1015dac0 2024-04-25T09:55:35.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.150+0000 7f9b0d59a640 1 -- 172.21.15.19:0/262614179 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196384364 0 0) 0x7f9b100d2420 con 0x7f9b100d34c0 2024-04-25T09:55:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.150+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/262614179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b1015dac0 0x7f9b100d19f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9b080097a0 tx=0x7f9b0802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e287bedd71ebfd61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.151+0000 7f9b0d59a640 1 -- 172.21.15.19:0/262614179 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08037070 con 0x7f9b1015dac0 2024-04-25T09:55:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.151+0000 7f9b0d59a640 1 -- 172.21.15.19:0/262614179 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b08032440 con 0x7f9b1015dac0 2024-04-25T09:55:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.151+0000 7f9b0d59a640 1 -- 172.21.15.19:0/262614179 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08032810 con 0x7f9b1015dac0 2024-04-25T09:55:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.151+0000 7f9b1484d640 1 -- 172.21.15.19:0/262614179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b1015dac0 msgr2=0x7f9b100d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.151+0000 7f9b1484d640 1 --2- 172.21.15.19:0/262614179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b1015dac0 0x7f9b100d19f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9b080097a0 tx=0x7f9b0802a210 comp rx=0 tx=0).stop 2024-04-25T09:55:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.151+0000 7f9b1484d640 1 -- 172.21.15.19:0/262614179 shutdown_connections 2024-04-25T09:55:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.151+0000 7f9b1484d640 1 --2- 172.21.15.19:0/262614179 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b100d1f30 0x7f9b100d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.151+0000 7f9b1484d640 1 --2- 172.21.15.19:0/262614179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b1015dac0 0x7f9b100d19f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.151+0000 7f9b1484d640 1 -- 172.21.15.19:0/262614179 >> 172.21.15.19:0/262614179 conn(0x7f9b10061c30 msgr2=0x7f9b1006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.151+0000 7f9b1484d640 1 -- 172.21.15.19:0/262614179 shutdown_connections 2024-04-25T09:55:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.151+0000 7f9b1484d640 1 -- 172.21.15.19:0/262614179 wait complete. 2024-04-25T09:55:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.153+0000 7f9b1484d640 1 Processor -- start 2024-04-25T09:55:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.153+0000 7f9b1484d640 1 -- start start 2024-04-25T09:55:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b1484d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b100d1f30 0x7f9b100b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b1484d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b100d34c0 0x7f9b1015f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b1484d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b1015dac0 0x7f9b1015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b1484d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b100b97d0 con 0x7f9b100d34c0 2024-04-25T09:55:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b1484d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1015dea0 con 0x7f9b100d1f30 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b1484d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b100b9950 con 0x7f9b1015dac0 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b100d34c0 0x7f9b1015f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b100d34c0 0x7f9b1015f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48136/0 (socket says 172.21.15.19:48136) 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0dd9b640 1 -- 172.21.15.19:0/1198838159 learned_addr learned my addr 172.21.15.19:0/1198838159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0ed9d640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b1015dac0 0x7f9b1015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b100d1f30 0x7f9b100b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0dd9b640 1 -- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b1015dac0 msgr2=0x7f9b1015fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b1015dac0 0x7f9b1015fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0dd9b640 1 -- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b100d1f30 msgr2=0x7f9b100b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b100d1f30 0x7f9b100b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0dd9b640 1 -- 172.21.15.19:0/1198838159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1015e270 con 0x7f9b100d34c0 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b100d1f30 0x7f9b100b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0ed9d640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b1015dac0 0x7f9b1015fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b100d34c0 0x7f9b1015f030 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e8d0 tx=0x7f9af800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b06ffd640 1 -- 172.21.15.19:0/1198838159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8017070 con 0x7f9b100d34c0 2024-04-25T09:55:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.154+0000 7f9b06ffd640 1 -- 172.21.15.19:0/1198838159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af800c940 con 0x7f9b100d34c0 2024-04-25T09:55:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.155+0000 7f9b1484d640 1 -- 172.21.15.19:0/1198838159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1015e550 con 0x7f9b100d34c0 2024-04-25T09:55:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.155+0000 7f9b06ffd640 1 -- 172.21.15.19:0/1198838159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af800cd30 con 0x7f9b100d34c0 2024-04-25T09:55:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.155+0000 7f9b1484d640 1 -- 172.21.15.19:0/1198838159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1015eb10 con 0x7f9b100d34c0 2024-04-25T09:55:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.156+0000 7f9b06ffd640 1 -- 172.21.15.19:0/1198838159 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9af801a070 con 0x7f9b100d34c0 2024-04-25T09:55:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.157+0000 7f9b06ffd640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9af006b4d0 0x7f9af006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.157+0000 7f9b06ffd640 1 -- 172.21.15.19:0/1198838159 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f9af808a7f0 con 0x7f9b100d34c0 2024-04-25T09:55:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.157+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9af006b4d0 0x7f9af006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.157+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9af006b4d0 0x7f9af006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9b08034000 tx=0x7f9b0802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.157+0000 7f9b1762c740 1 Processor -- start 2024-04-25T09:55:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.158+0000 7f9b1762c740 1 -- start start 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.158+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f72b10 0x55a7d1f75320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.158+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72ef0 0x55a7d1f75860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.158+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f45770 0x55a7d1f48f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.158+0000 7f9b1762c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1eaeda0 con 0x55a7d1f45770 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.158+0000 7f9b1762c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1e57b80 con 0x55a7d1f72b10 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.158+0000 7f9b1762c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1eaae40 con 0x55a7d1f72ef0 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9b0ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f45770 0x55a7d1f48f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9b0ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f45770 0x55a7d1f48f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48152/0 (socket says 172.21.15.19:48152) 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9b0ed9d640 1 -- 172.21.15.19:0/1895161155 learned_addr learned my addr 172.21.15.19:0/1895161155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/1895161155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f72b10 0x55a7d1f75320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/1895161155 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72ef0 0x55a7d1f75860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9b0ed9d640 1 -- 172.21.15.19:0/1895161155 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72ef0 msgr2=0x55a7d1f75860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9b0ed9d640 1 --2- 172.21.15.19:0/1895161155 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72ef0 0x55a7d1f75860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9b0ed9d640 1 -- 172.21.15.19:0/1895161155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f72b10 msgr2=0x55a7d1f75320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9b0ed9d640 1 --2- 172.21.15.19:0/1895161155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f72b10 0x55a7d1f75320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9b0ed9d640 1 -- 172.21.15.19:0/1895161155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7d1e61f10 con 0x55a7d1f45770 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9b0ed9d640 1 --2- 172.21.15.19:0/1895161155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f45770 0x55a7d1f48f80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9b00007350 tx=0x7f9b00007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead21f2d66b50ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9aeffff640 1 -- 172.21.15.19:0/1895161155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00019070 con 0x55a7d1f45770 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9aeffff640 1 -- 172.21.15.19:0/1895161155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b00001dc0 con 0x55a7d1f45770 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.159+0000 7f9aeffff640 1 -- 172.21.15.19:0/1895161155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b000112e0 con 0x55a7d1f45770 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.160+0000 7f9b1762c740 1 -- 172.21.15.19:0/1895161155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f45770 msgr2=0x55a7d1f48f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.160+0000 7f9b1762c740 1 --2- 172.21.15.19:0/1895161155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f45770 0x55a7d1f48f80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9b00007350 tx=0x7f9b00007810 comp rx=0 tx=0).stop 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.160+0000 7f9b1762c740 1 -- 172.21.15.19:0/1895161155 shutdown_connections 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.160+0000 7f9b1762c740 1 --2- 172.21.15.19:0/1895161155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f45770 0x55a7d1f48f80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.160+0000 7f9b1762c740 1 --2- 172.21.15.19:0/1895161155 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72ef0 0x55a7d1f75860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.160+0000 7f9b1762c740 1 --2- 172.21.15.19:0/1895161155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f72b10 0x55a7d1f75320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.160+0000 7f9b1762c740 1 -- 172.21.15.19:0/1895161155 >> 172.21.15.19:0/1895161155 conn(0x55a7d1f479e0 msgr2=0x55a7d1f90320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.160+0000 7f9b1762c740 1 -- 172.21.15.19:0/1895161155 shutdown_connections 2024-04-25T09:55:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.160+0000 7f9b1762c740 1 -- 172.21.15.19:0/1895161155 wait complete. 2024-04-25T09:55:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.161+0000 7f9b1762c740 1 -- start start 2024-04-25T09:55:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.161+0000 7f9b1762c740 1 Processor -- start 2024-04-25T09:55:35.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f70b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72ef0 0x55a7d1f92ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f6f2e0 0x55a7d1f93df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b1762c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1f4ab90 con 0x55a7d1f72b10 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b1762c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1e4d4b0 con 0x55a7d1f6f2e0 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b1762c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1f4ae70 con 0x55a7d1f72ef0 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b1762c740 10 client.0 _ll_register_callbacks cb 0x55a7d1e8d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b0e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f70b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b0e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f70b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48166/0 (socket says 172.21.15.19:48166) 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b0e59c640 1 -- 172.21.15.19:0/2729097755 learned_addr learned my addr 172.21.15.19:0/2729097755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72ef0 0x55a7d1f92ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b0ed9d640 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f6f2e0 0x55a7d1f93df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b0e59c640 1 -- 172.21.15.19:0/2729097755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72ef0 msgr2=0x55a7d1f92ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72ef0 0x55a7d1f92ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b0e59c640 1 -- 172.21.15.19:0/2729097755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f6f2e0 msgr2=0x55a7d1f93df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f6f2e0 0x55a7d1f93df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b0e59c640 1 -- 172.21.15.19:0/2729097755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7d1c8d7d0 con 0x55a7d1f72b10 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f70b60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9b0802a800 tx=0x7f9b08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ca680547bda0ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.162+0000 7f9aed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9aed7fa640 1 -- 172.21.15.19:0/2729097755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0804e060 con 0x55a7d1f72b10 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9aed7fa640 1 -- 172.21.15.19:0/2729097755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f9b0803eb30 con 0x55a7d1f72b10 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9b1762c740 1 -- 172.21.15.19:0/2729097755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b00001dc0 con 0x55a7d1f72b10 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9aed7fa640 1 -- 172.21.15.19:0/2729097755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b08054050 con 0x55a7d1f72b10 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9b1762c740 10 client.5532 fetch_fsmap learned FSMap version 17 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9b1762c740 1 -- 172.21.15.19:0/2729097755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a7d1bc33f0 con 0x55a7d1f72b10 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9aed7fa640 1 -- 172.21.15.19:0/2729097755 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b00001dc0 con 0x55a7d1f72b10 2024-04-25T09:55:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9aed7fa640 1 -- 172.21.15.19:0/2729097755 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9b08057020 con 0x55a7d1f72b10 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9b1762c740 10 client.5532 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9b1762c740 10 client.5532 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9b1762c740 20 client.5532 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9b1762c740 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55a7d1f69f90 0x55a7d1f91450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9b1762c740 4 client.5532 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9b1762c740 1 -- 172.21.15.19:0/2729097755 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7d1ebc2e0 con 0x55a7d1f69f90 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55a7d1f69f90 0x55a7d1f91450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9aed7fa640 10 client.5532 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.163+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55a7d1f69f90 0x55a7d1f91450 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e9a007c86c530b6 server_cookie=4cb0de1e2ccae3ef in_seq=0 out_seq=0 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9aed7fa640 1 -- 172.21.15.19:0/2729097755 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a7d1ebc2e0 con 0x55a7d1f69f90 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9aed7fa640 10 client.5532 handle_command_reply: tid=0 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9b1762c740 1 client.5532 shutdown 2024-04-25T09:55:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9b1762c740 1 -- 172.21.15.19:0/2729097755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 msgr2=0x55a7d1f70b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9b1762c740 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f70b60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9b0802a800 tx=0x7f9b08004730 comp rx=0 tx=0).stop 2024-04-25T09:55:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9b1762c740 1 -- 172.21.15.19:0/2729097755 shutdown_connections 2024-04-25T09:55:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9b1762c740 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f6f2e0 0x55a7d1f93df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9b1762c740 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72ef0 0x55a7d1f92ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9b1762c740 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55a7d1f69f90 0x55a7d1f91450 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9b1762c740 1 --2- 172.21.15.19:0/2729097755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f70b60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9aed7fa640 0 client.5532 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9b1762c740 1 -- 172.21.15.19:0/2729097755 >> 172.21.15.19:0/2729097755 conn(0x55a7d1f45770 msgr2=0x55a7d1f71190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9b1762c740 1 -- 172.21.15.19:0/2729097755 shutdown_connections 2024-04-25T09:55:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.164+0000 7f9b1762c740 1 -- 172.21.15.19:0/2729097755 wait complete. 2024-04-25T09:55:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.165+0000 7f9b1762c740 20 client.5532 trim_cache size 0 max 16384 2024-04-25T09:55:35.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.180+0000 7f9b1762c740 1 Processor -- start 2024-04-25T09:55:35.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.180+0000 7f9b1762c740 1 -- start start 2024-04-25T09:55:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.180+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f6f2e0 0x55a7d1f8d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72b10 0x55a7d1f41020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72ef0 0x55a7d1f72380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b1762c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b10200420 con 0x55a7d1f72ef0 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b1762c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b101ff5e0 con 0x55a7d1f6f2e0 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b1762c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b101572e0 con 0x55a7d1f72b10 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72ef0 0x55a7d1f72380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72ef0 0x55a7d1f72380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48180/0 (socket says 172.21.15.19:48180) 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0ed9d640 1 -- 172.21.15.19:0/3647851927 learned_addr learned my addr 172.21.15.19:0/3647851927 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/3647851927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f6f2e0 0x55a7d1f8d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/3647851927 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72b10 0x55a7d1f41020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0ed9d640 1 -- 172.21.15.19:0/3647851927 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72b10 msgr2=0x55a7d1f41020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0ed9d640 1 --2- 172.21.15.19:0/3647851927 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72b10 0x55a7d1f41020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0ed9d640 1 -- 172.21.15.19:0/3647851927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f6f2e0 msgr2=0x55a7d1f8d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0ed9d640 1 --2- 172.21.15.19:0/3647851927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f6f2e0 0x55a7d1f8d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0ed9d640 1 -- 172.21.15.19:0/3647851927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7d1b830f0 con 0x55a7d1f72ef0 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/3647851927 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72b10 0x55a7d1f41020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/3647851927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f6f2e0 0x55a7d1f8d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9b0ed9d640 1 --2- 172.21.15.19:0/3647851927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72ef0 0x55a7d1f72380 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9b000012e0 tx=0x7f9b00011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2524d94317d4bfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9aef7fe640 1 -- 172.21.15.19:0/3647851927 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00024070 con 0x55a7d1f72ef0 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9aef7fe640 1 -- 172.21.15.19:0/3647851927 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0000b070 con 0x55a7d1f72ef0 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.181+0000 7f9aef7fe640 1 -- 172.21.15.19:0/3647851927 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00019040 con 0x55a7d1f72ef0 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.182+0000 7f9b1762c740 1 -- 172.21.15.19:0/3647851927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72ef0 msgr2=0x55a7d1f72380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.182+0000 7f9b1762c740 1 --2- 172.21.15.19:0/3647851927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72ef0 0x55a7d1f72380 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9b000012e0 tx=0x7f9b00011000 comp rx=0 tx=0).stop 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.182+0000 7f9b1762c740 1 -- 172.21.15.19:0/3647851927 shutdown_connections 2024-04-25T09:55:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.182+0000 7f9b1762c740 1 --2- 172.21.15.19:0/3647851927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72ef0 0x55a7d1f72380 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.182+0000 7f9b1762c740 1 --2- 172.21.15.19:0/3647851927 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f72b10 0x55a7d1f41020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.182+0000 7f9b1762c740 1 --2- 172.21.15.19:0/3647851927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f6f2e0 0x55a7d1f8d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.182+0000 7f9b1762c740 1 -- 172.21.15.19:0/3647851927 >> 172.21.15.19:0/3647851927 conn(0x55a7d1f45770 msgr2=0x55a7d1f90b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.182+0000 7f9b1762c740 1 -- 172.21.15.19:0/3647851927 shutdown_connections 2024-04-25T09:55:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.182+0000 7f9b1762c740 1 -- 172.21.15.19:0/3647851927 wait complete. 2024-04-25T09:55:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.182+0000 7f9b1762c740 1 -- start start 2024-04-25T09:55:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.183+0000 7f9b1762c740 1 Processor -- start 2024-04-25T09:55:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.183+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f6f2e0 0x55a7d1fe09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.183+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f6c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.183+0000 7f9b1762c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f72ef0 0x55a7d1f91fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.183+0000 7f9b1762c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b101ff010 con 0x55a7d1f72b10 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.183+0000 7f9b1762c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1f8fbc0 con 0x55a7d1f72ef0 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.183+0000 7f9b1762c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7d1f78b60 con 0x55a7d1f6f2e0 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b1762c740 10 client.0 _ll_register_callbacks cb 0x7f9b00005cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f6c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f6c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48184/0 (socket says 172.21.15.19:48184) 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0dd9b640 1 -- 172.21.15.19:0/615314189 learned_addr learned my addr 172.21.15.19:0/615314189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f6f2e0 0x55a7d1fe09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0ed9d640 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f72ef0 0x55a7d1f91fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0dd9b640 1 -- 172.21.15.19:0/615314189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f6f2e0 msgr2=0x55a7d1fe09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f6f2e0 0x55a7d1fe09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0dd9b640 1 -- 172.21.15.19:0/615314189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f72ef0 msgr2=0x55a7d1f91fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f72ef0 0x55a7d1f91fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0dd9b640 1 -- 172.21.15.19:0/615314189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7d1fb2cd0 con 0x55a7d1f72b10 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f6f2e0 0x55a7d1fe09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b0dd9b640 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f6c7c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9af8055210 tx=0x7f9af800e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3456047625266517 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9aed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9aed7fa640 1 -- 172.21.15.19:0/615314189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8021040 con 0x55a7d1f72b10 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9aed7fa640 1 -- 172.21.15.19:0/615314189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f9af80570b0 con 0x55a7d1f72b10 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b1762c740 1 -- 172.21.15.19:0/615314189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b0000b070 con 0x55a7d1f72b10 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9aed7fa640 1 -- 172.21.15.19:0/615314189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9af807c050 con 0x55a7d1f72b10 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b1762c740 10 client.5538 fetch_fsmap learned FSMap version 17 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9b1762c740 1 -- 172.21.15.19:0/615314189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a7d1f78900 con 0x55a7d1f72b10 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.184+0000 7f9aed7fa640 1 -- 172.21.15.19:0/615314189 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0000b070 con 0x55a7d1f72b10 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9aed7fa640 1 -- 172.21.15.19:0/615314189 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9af807f020 con 0x55a7d1f72b10 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 10 client.5538 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:55:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 10 client.5538 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 20 client.5538 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55a7d1f77e40 0x55a7d1f78250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 4 client.5538 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 1 -- 172.21.15.19:0/615314189 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7d1eb6630 con 0x55a7d1f77e40 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55a7d1f77e40 0x55a7d1f78250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9aed7fa640 10 client.5538 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b0e59c640 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55a7d1f77e40 0x55a7d1f78250 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c6c6eae5438c18 server_cookie=4d956be663697a4f in_seq=0 out_seq=0 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9aed7fa640 1 -- 172.21.15.19:0/615314189 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a7d1eb6630 con 0x55a7d1f77e40 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9aed7fa640 10 client.5538 handle_command_reply: tid=0 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 1 client.5538 shutdown 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 1 -- 172.21.15.19:0/615314189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 msgr2=0x55a7d1f6c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f6c7c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9af8055210 tx=0x7f9af800e850 comp rx=0 tx=0).stop 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 1 -- 172.21.15.19:0/615314189 shutdown_connections 2024-04-25T09:55:35.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7d1f72ef0 0x55a7d1f91fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x55a7d1f77e40 0x55a7d1f78250 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7d1f72b10 0x55a7d1f6c7c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 1 --2- 172.21.15.19:0/615314189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a7d1f6f2e0 0x55a7d1fe09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9b1762c740 1 -- 172.21.15.19:0/615314189 >> 172.21.15.19:0/615314189 conn(0x55a7d1f45770 msgr2=0x55a7d1f692a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.185+0000 7f9aed7fa640 0 client.5538 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:35.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1762c740 1 -- 172.21.15.19:0/615314189 shutdown_connections 2024-04-25T09:55:35.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1762c740 1 -- 172.21.15.19:0/615314189 wait complete. 2024-04-25T09:55:35.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1762c740 20 client.5538 trim_cache size 0 max 16384 2024-04-25T09:55:35.195 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:55:35.196 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T09:55:35.196 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T09:55:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:55:35.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 -- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9af006b4d0 msgr2=0x7f9af006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9af006b4d0 0x7f9af006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9b08034000 tx=0x7f9b0802e000 comp rx=0 tx=0).stop 2024-04-25T09:55:35.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 -- 172.21.15.19:0/1198838159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b100d34c0 msgr2=0x7f9b1015f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b100d34c0 0x7f9b1015f030 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e8d0 tx=0x7f9af800ed90 comp rx=0 tx=0).stop 2024-04-25T09:55:35.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 -- 172.21.15.19:0/1198838159 shutdown_connections 2024-04-25T09:55:35.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9af006b4d0 0x7f9af006d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b1015dac0 0x7f9b1015fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b100d34c0 0x7f9b1015f030 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 --2- 172.21.15.19:0/1198838159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b100d1f30 0x7f9b100b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 -- 172.21.15.19:0/1198838159 >> 172.21.15.19:0/1198838159 conn(0x7f9b10061c30 msgr2=0x7f9b100ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 -- 172.21.15.19:0/1198838159 shutdown_connections 2024-04-25T09:55:35.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.186+0000 7f9b1484d640 1 -- 172.21.15.19:0/1198838159 wait complete. 2024-04-25T09:55:35.199 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:55:35.200 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:55:35.200 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T09:55:35.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.300+0000 7f28f701f640 1 Processor -- start 2024-04-25T09:55:35.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.300+0000 7f28f701f640 1 -- start start 2024-04-25T09:55:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28f701f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f28f015dac0 0x7f28f00d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28f701f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f00d2420 con 0x7f28f00d1f30 2024-04-25T09:55:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28f701f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f00d2700 con 0x7f28f00d0ab0 2024-04-25T09:55:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28f701f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f00d3030 con 0x7f28f015dac0 2024-04-25T09:55:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28effff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f28f00d1f30 0x7f28f00d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54402/0 (socket says 172.21.15.19:54402) 2024-04-25T09:55:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28effff640 1 -- 172.21.15.19:0/1701878687 learned_addr learned my addr 172.21.15.19:0/1701878687 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/1701878687 >> v2:172.21.15.112:3300/0 conn(0x7f28f015dac0 0x7f28f00d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578545731 0 0) 0x7f28f00d2420 con 0x7f28f00d1f30 2024-04-25T09:55:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28e0003520 con 0x7f28f00d1f30 2024-04-25T09:55:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3024521226 0 0) 0x7f28e0003520 con 0x7f28f00d1f30 2024-04-25T09:55:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 >> v2:172.21.15.112:3300/0 conn(0x7f28f015dac0 msgr2=0x7f28f00d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 --2- 172.21.15.19:0/1701878687 >> v2:172.21.15.112:3300/0 conn(0x7f28f015dac0 0x7f28f00d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 >> v1:172.21.15.112:6789/0 conn(0x7f28f00d0ab0 legacy=0x7f28f00b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f00d29e0 con 0x7f28f00d1f30 2024-04-25T09:55:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f28dc0017f0 con 0x7f28f00d1f30 2024-04-25T09:55:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 >> v1:172.21.15.19:6789/0 conn(0x7f28f00d1f30 legacy=0x7f28f00d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28e00039b0 0x7f28e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28e0004800 0x7f28e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28e00051c0 0x7f28e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00d3030 con 0x7f28e0004800 2024-04-25T09:55:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e000bf80 con 0x7f28e00039b0 2024-04-25T09:55:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.301+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e000c180 con 0x7f28e00051c0 2024-04-25T09:55:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.302+0000 7f28effff640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28e0004800 0x7f28e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.302+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28e00039b0 0x7f28e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.302+0000 7f28f55bc640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28e00051c0 0x7f28e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.302+0000 7f28effff640 1 -- 172.21.15.19:0/1701878687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28e00051c0 msgr2=0x7f28e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.302+0000 7f28effff640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28e00051c0 0x7f28e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.302+0000 7f28effff640 1 -- 172.21.15.19:0/1701878687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28e00039b0 msgr2=0x7f28e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.302+0000 7f28effff640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28e00039b0 0x7f28e0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.302+0000 7f28effff640 1 -- 172.21.15.19:0/1701878687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0003520 con 0x7f28e0004800 2024-04-25T09:55:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.302+0000 7f28f55bc640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28e00051c0 0x7f28e000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.302+0000 7f28f55bc640 1 -- 172.21.15.19:0/1701878687 reap_dead start 2024-04-25T09:55:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.302+0000 7f28effff640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28e0004800 0x7f28e0004c80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00c110 tx=0x7f28e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6bbc7ab89a8d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.303+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc007c30 con 0x7f28e0004800 2024-04-25T09:55:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.303+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc007290 con 0x7f28e0004800 2024-04-25T09:55:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.303+0000 7f28ef7fe640 1 -- 172.21.15.19:0/1701878687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0057a0 con 0x7f28e0004800 2024-04-25T09:55:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.303+0000 7f28f701f640 1 -- 172.21.15.19:0/1701878687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28e0004800 msgr2=0x7f28e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.303+0000 7f28f701f640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28e0004800 0x7f28e0004c80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00c110 tx=0x7f28e0005830 comp rx=0 tx=0).stop 2024-04-25T09:55:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.303+0000 7f28f701f640 1 -- 172.21.15.19:0/1701878687 shutdown_connections 2024-04-25T09:55:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.303+0000 7f28f701f640 1 --2- 172.21.15.19:0/1701878687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28e0004800 0x7f28e0004c80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.303+0000 7f28f701f640 1 -- 172.21.15.19:0/1701878687 >> 172.21.15.19:0/1701878687 conn(0x7f28f0061c30 msgr2=0x7f28f006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.303+0000 7f28f701f640 1 -- 172.21.15.19:0/1701878687 shutdown_connections 2024-04-25T09:55:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.303+0000 7f28f701f640 1 -- 172.21.15.19:0/1701878687 wait complete. 2024-04-25T09:55:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.304+0000 7f28f701f640 1 Processor -- start 2024-04-25T09:55:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.304+0000 7f28f701f640 1 -- start start 2024-04-25T09:55:35.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.305+0000 7f28f701f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28e0004800 0x7f28f0159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.305+0000 7f28f701f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28f01532c0 0x7f28f0152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.305+0000 7f28f701f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f0154c50 0x7f28f015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.305+0000 7f28f701f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00b97d0 con 0x7f28f0154c50 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.305+0000 7f28f701f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f015dea0 con 0x7f28e0004800 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.305+0000 7f28f701f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00b9950 con 0x7f28f01532c0 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.305+0000 7f28f55bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f0154c50 0x7f28f015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.305+0000 7f28f55bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f0154c50 0x7f28f015f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48212/0 (socket says 172.21.15.19:48212) 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.305+0000 7f28f55bc640 1 -- 172.21.15.19:0/2349053292 learned_addr learned my addr 172.21.15.19:0/2349053292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28effff640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28f01532c0 0x7f28f0152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28e0004800 0x7f28f0159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28f55bc640 1 -- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28f01532c0 msgr2=0x7f28f0152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28f55bc640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28f01532c0 0x7f28f0152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28f55bc640 1 -- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28e0004800 msgr2=0x7f28f0159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28f55bc640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28e0004800 0x7f28f0159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28f55bc640 1 -- 172.21.15.19:0/2349053292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f015a690 con 0x7f28f0154c50 2024-04-25T09:55:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28e0004800 0x7f28f0159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28f55bc640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f0154c50 0x7f28f015f6a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f28f00b64d0 tx=0x7f28e800ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28ed7fa640 1 -- 172.21.15.19:0/2349053292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8019070 con 0x7f28f0154c50 2024-04-25T09:55:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28ed7fa640 1 -- 172.21.15.19:0/2349053292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e8006b50 con 0x7f28f0154c50 2024-04-25T09:55:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28f701f640 1 -- 172.21.15.19:0/2349053292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f0153e90 con 0x7f28f0154c50 2024-04-25T09:55:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28ed7fa640 1 -- 172.21.15.19:0/2349053292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8003970 con 0x7f28f0154c50 2024-04-25T09:55:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.306+0000 7f28f701f640 1 -- 172.21.15.19:0/2349053292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f0154450 con 0x7f28f0154c50 2024-04-25T09:55:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.308+0000 7f28ed7fa640 1 -- 172.21.15.19:0/2349053292 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28e801c070 con 0x7f28f0154c50 2024-04-25T09:55:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.308+0000 7f28ed7fa640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d406b4d0 0x7f28d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.308+0000 7f28ed7fa640 1 -- 172.21.15.19:0/2349053292 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f28e8089600 con 0x7f28f0154c50 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.308+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d406b4d0 0x7f28d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.309+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d406b4d0 0x7f28d406d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009c40 tx=0x7f28e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.309+0000 7f28f9e04740 1 Processor -- start 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.309+0000 7f28f9e04740 1 -- start start 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10b1ca0 0x5634a10b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10ee370 0x5634a10b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10dc760 0x5634a10dcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28f9e04740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a1014b90 con 0x5634a10ee370 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28f9e04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a0fbde40 con 0x5634a10b1ca0 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28f9e04740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634a1010c30 con 0x5634a10dc760 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28effff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10ee370 0x5634a10b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28effff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10ee370 0x5634a10b2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48224/0 (socket says 172.21.15.19:48224) 2024-04-25T09:55:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28effff640 1 -- 172.21.15.19:0/2579213866 learned_addr learned my addr 172.21.15.19:0/2579213866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/2579213866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10b1ca0 0x5634a10b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28f55bc640 1 --2- 172.21.15.19:0/2579213866 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10dc760 0x5634a10dcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28effff640 1 -- 172.21.15.19:0/2579213866 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10dc760 msgr2=0x5634a10dcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28effff640 1 --2- 172.21.15.19:0/2579213866 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10dc760 0x5634a10dcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28effff640 1 -- 172.21.15.19:0/2579213866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10b1ca0 msgr2=0x5634a10b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28effff640 1 --2- 172.21.15.19:0/2579213866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10b1ca0 0x5634a10b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28effff640 1 -- 172.21.15.19:0/2579213866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634a0df37d0 con 0x5634a10ee370 2024-04-25T09:55:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.310+0000 7f28effff640 1 --2- 172.21.15.19:0/2579213866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10ee370 0x5634a10b2080 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f28dc02ac60 tx=0x7f28dc007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bdf40e6acf0c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28da7fc640 1 -- 172.21.15.19:0/2579213866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc043070 con 0x5634a10ee370 2024-04-25T09:55:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28da7fc640 1 -- 172.21.15.19:0/2579213866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc038040 con 0x5634a10ee370 2024-04-25T09:55:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28da7fc640 1 -- 172.21.15.19:0/2579213866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0072b0 con 0x5634a10ee370 2024-04-25T09:55:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28f9e04740 1 -- 172.21.15.19:0/2579213866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10ee370 msgr2=0x5634a10b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28f9e04740 1 --2- 172.21.15.19:0/2579213866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10ee370 0x5634a10b2080 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f28dc02ac60 tx=0x7f28dc007000 comp rx=0 tx=0).stop 2024-04-25T09:55:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28f9e04740 1 -- 172.21.15.19:0/2579213866 shutdown_connections 2024-04-25T09:55:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28f9e04740 1 --2- 172.21.15.19:0/2579213866 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10dc760 0x5634a10dcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28f9e04740 1 --2- 172.21.15.19:0/2579213866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10ee370 0x5634a10b2080 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28f9e04740 1 --2- 172.21.15.19:0/2579213866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10b1ca0 0x5634a10b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28f9e04740 1 -- 172.21.15.19:0/2579213866 >> 172.21.15.19:0/2579213866 conn(0x5634a1079870 msgr2=0x5634a10f7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28f9e04740 1 -- 172.21.15.19:0/2579213866 shutdown_connections 2024-04-25T09:55:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.311+0000 7f28f9e04740 1 -- 172.21.15.19:0/2579213866 wait complete. 2024-04-25T09:55:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.312+0000 7f28f9e04740 1 -- start start 2024-04-25T09:55:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.312+0000 7f28f9e04740 1 Processor -- start 2024-04-25T09:55:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10dc760 0x5634a10efee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10ee370 0x5634a10daf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f9e04740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a0fd13e0 con 0x5634a10b1ca0 2024-04-25T09:55:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f9e04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a10da2b0 con 0x5634a10ee370 2024-04-25T09:55:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f9e04740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634a10dd030 con 0x5634a10dc760 2024-04-25T09:55:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f9e04740 10 client.0 _ll_register_callbacks cb 0x5634a0fd0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f4dbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f4dbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48236/0 (socket says 172.21.15.19:48236) 2024-04-25T09:55:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f4dbb640 1 -- 172.21.15.19:0/3544105535 learned_addr learned my addr 172.21.15.19:0/3544105535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28effff640 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10dc760 0x5634a10efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f55bc640 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10ee370 0x5634a10daf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f4dbb640 1 -- 172.21.15.19:0/3544105535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10dc760 msgr2=0x5634a10efee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10dc760 0x5634a10efee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f4dbb640 1 -- 172.21.15.19:0/3544105535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10ee370 msgr2=0x5634a10daf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10ee370 0x5634a10daf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f4dbb640 1 -- 172.21.15.19:0/3544105535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634a0ce90f0 con 0x5634a10b1ca0 2024-04-25T09:55:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.313+0000 7f28f55bc640 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10ee370 0x5634a10daf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10cc120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f28e4004650 tx=0x7f28e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec30f147d1a15b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:55:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28cf7fe640 1 -- 172.21.15.19:0/3544105535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4023c10 con 0x5634a10b1ca0 2024-04-25T09:55:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28cf7fe640 1 -- 172.21.15.19:0/3544105535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f28e4023db0 con 0x5634a10b1ca0 2024-04-25T09:55:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28f9e04740 1 -- 172.21.15.19:0/3544105535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28dc038040 con 0x5634a10b1ca0 2024-04-25T09:55:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28cf7fe640 1 -- 172.21.15.19:0/3544105535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e4036050 con 0x5634a10b1ca0 2024-04-25T09:55:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28f9e04740 10 client.5550 fetch_fsmap learned FSMap version 17 2024-04-25T09:55:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28f9e04740 1 -- 172.21.15.19:0/3544105535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5634a0e53f60 con 0x5634a10b1ca0 2024-04-25T09:55:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28cf7fe640 1 -- 172.21.15.19:0/3544105535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc038040 con 0x5634a10b1ca0 2024-04-25T09:55:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28cf7fe640 1 -- 172.21.15.19:0/3544105535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f28e4039020 con 0x5634a10b1ca0 2024-04-25T09:55:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28f9e04740 10 client.5550 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:55:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28f9e04740 10 client.5550 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:55:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28f9e04740 20 client.5550 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28f9e04740 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x5634a10dcbe0 0x5634a110aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28f9e04740 4 client.5550 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:55:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28f9e04740 1 -- 172.21.15.19:0/3544105535 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634a10220d0 con 0x5634a10dcbe0 2024-04-25T09:55:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.314+0000 7f28effff640 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x5634a10dcbe0 0x5634a110aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28effff640 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x5634a10dcbe0 0x5634a110aab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce4bc92469e8dac server_cookie=f5fb93605726c9d9 in_seq=0 out_seq=0 2024-04-25T09:55:35.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28cf7fe640 10 client.5550 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28cf7fe640 1 -- 172.21.15.19:0/3544105535 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634a10220d0 con 0x5634a10dcbe0 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28cf7fe640 10 client.5550 handle_command_reply: tid=0 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28f9e04740 1 client.5550 shutdown 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28f9e04740 1 -- 172.21.15.19:0/3544105535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 msgr2=0x5634a10cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28f9e04740 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10cc120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f28e4004650 tx=0x7f28e4004730 comp rx=0 tx=0).stop 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28f9e04740 1 -- 172.21.15.19:0/3544105535 shutdown_connections 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28f9e04740 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10ee370 0x5634a10daf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28f9e04740 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10dc760 0x5634a10efee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28f9e04740 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x5634a10dcbe0 0x5634a110aab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28f9e04740 1 --2- 172.21.15.19:0/3544105535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10cc120 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28f9e04740 1 -- 172.21.15.19:0/3544105535 >> 172.21.15.19:0/3544105535 conn(0x5634a1079870 msgr2=0x5634a10d77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28cf7fe640 0 client.5550 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28f9e04740 1 -- 172.21.15.19:0/3544105535 shutdown_connections 2024-04-25T09:55:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.315+0000 7f28f9e04740 1 -- 172.21.15.19:0/3544105535 wait complete. 2024-04-25T09:55:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.316+0000 7f28f9e04740 20 client.5550 trim_cache size 0 max 16384 2024-04-25T09:55:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.330+0000 7f28f9e04740 1 Processor -- start 2024-04-25T09:55:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.330+0000 7f28f9e04740 1 -- start start 2024-04-25T09:55:35.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10dc760 0x5634a10fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10ee370 0x5634a1158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f9e04740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f0200a80 con 0x5634a10b1ca0 2024-04-25T09:55:35.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f9e04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f01ffc20 con 0x5634a10dc760 2024-04-25T09:55:35.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f9e04740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f0158130 con 0x5634a10ee370 2024-04-25T09:55:35.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f4dbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f4dbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10df0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48242/0 (socket says 172.21.15.19:48242) 2024-04-25T09:55:35.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f4dbb640 1 -- 172.21.15.19:0/3006605200 learned_addr learned my addr 172.21.15.19:0/3006605200 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f55bc640 1 --2- 172.21.15.19:0/3006605200 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10ee370 0x5634a1158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28effff640 1 --2- 172.21.15.19:0/3006605200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10dc760 0x5634a10fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f4dbb640 1 -- 172.21.15.19:0/3006605200 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10ee370 msgr2=0x5634a1158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/3006605200 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10ee370 0x5634a1158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f4dbb640 1 -- 172.21.15.19:0/3006605200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10dc760 msgr2=0x5634a10fb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/3006605200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10dc760 0x5634a10fb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f4dbb640 1 -- 172.21.15.19:0/3006605200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634a10dac80 con 0x5634a10b1ca0 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28effff640 1 --2- 172.21.15.19:0/3006605200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10dc760 0x5634a10fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f55bc640 1 --2- 172.21.15.19:0/3006605200 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10ee370 0x5634a1158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/3006605200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10df0d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f28e4016540 tx=0x7f28e4026070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c940de4e5e85cd1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28d9ffb640 1 -- 172.21.15.19:0/3006605200 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4023e10 con 0x5634a10b1ca0 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28d9ffb640 1 -- 172.21.15.19:0/3006605200 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e4023790 con 0x5634a10b1ca0 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.331+0000 7f28d9ffb640 1 -- 172.21.15.19:0/3006605200 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4024920 con 0x5634a10b1ca0 2024-04-25T09:55:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.332+0000 7f28f9e04740 1 -- 172.21.15.19:0/3006605200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 msgr2=0x5634a10df0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.332+0000 7f28f9e04740 1 --2- 172.21.15.19:0/3006605200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10df0d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f28e4016540 tx=0x7f28e4026070 comp rx=0 tx=0).stop 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.332+0000 7f28f9e04740 1 -- 172.21.15.19:0/3006605200 shutdown_connections 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.332+0000 7f28f9e04740 1 --2- 172.21.15.19:0/3006605200 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10ee370 0x5634a1158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.332+0000 7f28f9e04740 1 --2- 172.21.15.19:0/3006605200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10dc760 0x5634a10fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.332+0000 7f28f9e04740 1 --2- 172.21.15.19:0/3006605200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a10df0d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.332+0000 7f28f9e04740 1 -- 172.21.15.19:0/3006605200 >> 172.21.15.19:0/3006605200 conn(0x5634a1079870 msgr2=0x5634a10dfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.332+0000 7f28f9e04740 1 -- 172.21.15.19:0/3006605200 shutdown_connections 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.332+0000 7f28f9e04740 1 -- 172.21.15.19:0/3006605200 wait complete. 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.332+0000 7f28f9e04740 1 -- start start 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.333+0000 7f28f9e04740 1 Processor -- start 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.333+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a1143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.333+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10dc760 0x5634a10ed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.333+0000 7f28f9e04740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10ee370 0x5634a10ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.333+0000 7f28f9e04740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a10d6790 con 0x5634a10b1ca0 2024-04-25T09:55:35.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.333+0000 7f28f9e04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f01ff620 con 0x5634a10dc760 2024-04-25T09:55:35.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.333+0000 7f28f9e04740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634a10adf70 con 0x5634a10ee370 2024-04-25T09:55:35.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f9e04740 10 client.0 _ll_register_callbacks cb 0x7f28e4009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:35.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f4dbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a1143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f4dbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a1143fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48250/0 (socket says 172.21.15.19:48250) 2024-04-25T09:55:35.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f4dbb640 1 -- 172.21.15.19:0/1175970150 learned_addr learned my addr 172.21.15.19:0/1175970150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f4dbb640 1 -- 172.21.15.19:0/1175970150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10ee370 msgr2=0x5634a10ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28effff640 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10dc760 0x5634a10ed0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f55bc640 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10ee370 0x5634a10ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10ee370 0x5634a10ed630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f4dbb640 1 -- 172.21.15.19:0/1175970150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10dc760 msgr2=0x5634a10ed0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10dc760 0x5634a10ed0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f4dbb640 1 -- 172.21.15.19:0/1175970150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634a0cc21b0 con 0x5634a10b1ca0 2024-04-25T09:55:35.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f4dbb640 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a1143fe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009660 tx=0x7f28e4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7eb978b58d0475 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28cf7fe640 1 -- 172.21.15.19:0/1175970150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e404b070 con 0x5634a10b1ca0 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28cf7fe640 1 -- 172.21.15.19:0/1175970150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f28e4050c70 con 0x5634a10b1ca0 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28f9e04740 1 -- 172.21.15.19:0/1175970150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28e4023790 con 0x5634a10b1ca0 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.334+0000 7f28cf7fe640 1 -- 172.21.15.19:0/1175970150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e4050e90 con 0x5634a10b1ca0 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28f9e04740 10 client.5556 fetch_fsmap learned FSMap version 17 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28f9e04740 1 -- 172.21.15.19:0/1175970150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5634a10b1230 con 0x5634a10b1ca0 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28cf7fe640 1 -- 172.21.15.19:0/1175970150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e4023790 con 0x5634a10b1ca0 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28cf7fe640 1 -- 172.21.15.19:0/1175970150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f28e4043020 con 0x5634a10b1ca0 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28f9e04740 10 client.5556 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28f9e04740 10 client.5556 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:55:35.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28f9e04740 20 client.5556 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:35.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28f9e04740 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x5634a110d200 0x5634a110eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28f9e04740 4 client.5556 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:55:35.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28f9e04740 1 -- 172.21.15.19:0/1175970150 --> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5634a101c420 con 0x5634a110d200 2024-04-25T09:55:35.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28effff640 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x5634a110d200 0x5634a110eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28cf7fe640 10 client.5556 ms_handle_connect on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:35.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.335+0000 7f28effff640 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x5634a110d200 0x5634a110eae0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a47d2db99a97da09 server_cookie=211021dacffd3ddf in_seq=0 out_seq=0 2024-04-25T09:55:35.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28cf7fe640 1 -- 172.21.15.19:0/1175970150 <== mds.0 v2:172.21.15.19:6833/3017773710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17943 (crc 0 0 0) 0x5634a101c420 con 0x5634a110d200 2024-04-25T09:55:35.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28cf7fe640 10 client.5556 handle_command_reply: tid=0 2024-04-25T09:55:35.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 1 client.5556 shutdown 2024-04-25T09:55:35.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 1 -- 172.21.15.19:0/1175970150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 msgr2=0x5634a1143fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a1143fe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009660 tx=0x7f28e4019000 comp rx=0 tx=0).stop 2024-04-25T09:55:35.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 1 -- 172.21.15.19:0/1175970150 shutdown_connections 2024-04-25T09:55:35.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5634a10ee370 0x5634a10ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634a10dc760 0x5634a10ed0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.19:6833/3017773710,v1:172.21.15.19:6835/3017773710] conn(0x5634a110d200 0x5634a110eae0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 1 --2- 172.21.15.19:0/1175970150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634a10b1ca0 0x5634a1143fe0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28cf7fe640 0 client.5556 ms_handle_reset on v2:172.21.15.19:6833/3017773710 2024-04-25T09:55:35.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 1 -- 172.21.15.19:0/1175970150 >> 172.21.15.19:0/1175970150 conn(0x5634a1079870 msgr2=0x5634a10dfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 1 -- 172.21.15.19:0/1175970150 shutdown_connections 2024-04-25T09:55:35.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 1 -- 172.21.15.19:0/1175970150 wait complete. 2024-04-25T09:55:35.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.336+0000 7f28f9e04740 20 client.5556 trim_cache size 0 max 16384 2024-04-25T09:55:35.348 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:55:35.348 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:55:35.348 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 3600, 2024-04-25T09:55:35.348 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 3599, 2024-04-25T09:55:35.348 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 907354, 2024-04-25T09:55:35.348 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 44114902, 2024-04-25T09:55:35.348 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 17, 2024-04-25T09:55:35.348 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 6, 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.400854118, 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.209841893, 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.070549225, 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.101085496, 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3598, 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.376480365, 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000104636 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 10, 2024-04-25T09:55:35.349 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000989, 2024-04-25T09:55:35.350 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000098 2024-04-25T09:55:35.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.350 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-25T09:55:35.350 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T09:55:35.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.350 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:55:35.350 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2543, 2024-04-25T09:55:35.350 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2563, 2024-04-25T09:55:35.350 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 643618, 2024-04-25T09:55:35.350 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 22020123, 2024-04-25T09:55:35.350 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.294227896, 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.162252767, 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.047837373, 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.068411078, 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2562, 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.302916263, 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000118234 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:55:35.351 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T09:55:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 115728 2024-04-25T09:55:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:55:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 29599, 2024-04-25T09:55:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 52078, 2024-04-25T09:55:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 10090170, 2024-04-25T09:55:35.352 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 54769465, 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 3.579727164, 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 2.441974221, 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 1.689012969, 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.069760956, 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 52078, 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.231446717, 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000023646 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.353 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:55:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 74188, 2024-04-25T09:55:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.009336705, 2024-04-25T09:55:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000125 2024-04-25T09:55:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 120400, 2024-04-25T09:55:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 24432 2024-04-25T09:55:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T09:55:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-04-25T09:55:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T09:55:35.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2515, 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 18.049907846, 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.007176901 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T09:55:35.355 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3887, 2024-04-25T09:55:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.233157045, 2024-04-25T09:55:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000317251 2024-04-25T09:55:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-04-25T09:55:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 25266, 2024-04-25T09:55:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 25266, 2024-04-25T09:55:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-04-25T09:55:35.356 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 25266, 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 430.304080840, 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.017030953 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 12, 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 17831, 2024-04-25T09:55:35.357 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 17831, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 17831, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 2, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 2, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 2, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 45864, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 45468, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-04-25T09:55:35.358 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 107, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 1854, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-04-25T09:55:35.359 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 1854, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 13, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 1841, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:55:35.360 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 1343, 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 4054, 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 4104, 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 3288, 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 26, 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 13896 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-04-25T09:55:35.361 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-04-25T09:55:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-04-25T09:55:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-04-25T09:55:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-04-25T09:55:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-04-25T09:55:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-04-25T09:55:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:55:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 4116, 2024-04-25T09:55:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 3670, 2024-04-25T09:55:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 2211, 2024-04-25T09:55:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-04-25T09:55:35.362 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.363 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 31244, 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 31244, 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 52, 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 52, 2024-04-25T09:55:35.364 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 4, 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 105381408, 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 0, 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2610, 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 127.964844102, 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.049028675 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 0 2024-04-25T09:55:35.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-04-25T09:55:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 14, 2024-04-25T09:55:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 2016, 2024-04-25T09:55:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 2002, 2024-04-25T09:55:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 13, 2024-04-25T09:55:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 13, 2024-04-25T09:55:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-04-25T09:55:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 17831, 2024-04-25T09:55:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 19832, 2024-04-25T09:55:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 2001, 2024-04-25T09:55:35.366 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 2, 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 2946, 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 2944, 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 111608, 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215396 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 28146, 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 25266, 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 5, 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-04-25T09:55:35.367 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-04-25T09:55:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2000, 2024-04-25T09:55:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.661998200, 2024-04-25T09:55:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000830999 2024-04-25T09:55:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-04-25T09:55:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 29, 2024-04-25T09:55:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.004818557, 2024-04-25T09:55:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000166157 2024-04-25T09:55:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-04-25T09:55:35.368 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 6000, 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 6.876810329, 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001146135 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 7013, 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 6.773031530, 2024-04-25T09:55:35.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000965782 2024-04-25T09:55:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-04-25T09:55:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-04-25T09:55:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-04-25T09:55:35.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:55:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000045499, 2024-04-25T09:55:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000045499 2024-04-25T09:55:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-04-25T09:55:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-04-25T09:55:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:55:35.372 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.003302398, 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.003302398 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.023184236, 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.023184236 2024-04-25T09:55:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8, 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.531910094, 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.191488761 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-04-25T09:55:35.374 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2211, 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 396.532234525, 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.179345198 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.375 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.048588885, 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.048588885 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.376 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000257739, 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000257739 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-04-25T09:55:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-04-25T09:55:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.378 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.379 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8000, 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 16.847898848, 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.002105987 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:35.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 1, 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 18402, 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 18402, 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 72, 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T09:55:35.381 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:55:35.382 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:55:35.383 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:55:35.384 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 250116, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 91, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 352, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 4, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 5624, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 80, 2024-04-25T09:55:35.385 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 13432125, 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 660887, 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-04-25T09:55:35.386 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 8366, 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 101854869, 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 37, 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 8366, 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8366, 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 33.935650120, 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.004056377 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-04-25T09:55:35.387 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8366, 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 8456 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 8366, 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 1903, 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 6463, 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 0, 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 11, 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 47, 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 2655, 2024-04-25T09:55:35.388 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 29, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 1670, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 2036, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1856, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 22, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:55:35.389 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-04-25T09:55:35.390 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-04-25T09:55:35.391 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 42, 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 5, 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 8, 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 8, 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 99, 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 0, 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 31 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-04-25T09:55:35.392 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 1, 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-04-25T09:55:35.393 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 32, 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 3670, 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 3670, 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 35738, 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 8419207, 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:55:35.394 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 35738, 2024-04-25T09:55:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:55:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:55:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 35738, 2024-04-25T09:55:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 8419207, 2024-04-25T09:55:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:55:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.395 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-04-25T09:55:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:55:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T09:55:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:55:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T09:55:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T09:55:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:55:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T09:55:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 8366, 2024-04-25T09:55:35.396 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 102238899, 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 8366, 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 102238899, 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-04-25T09:55:35.397 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:55:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T09:55:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T09:55:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:55:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T09:55:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 8366, 2024-04-25T09:55:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 8366, 2024-04-25T09:55:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 8366, 2024-04-25T09:55:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 8366, 2024-04-25T09:55:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:55:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.398 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 3670, 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 384030, 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 3670, 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:55:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:55:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 45, 2024-04-25T09:55:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 384030, 2024-04-25T09:55:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:55:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55f1b38246e0": { 2024-04-25T09:55:35.400 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 31244, 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 101187104, 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 31244, 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2610, 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 101187104, 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:55:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 -- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d406b4d0 msgr2=0x7f28d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d406b4d0 0x7f28d406d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009c40 tx=0x7f28e4008040 comp rx=0 tx=0).stop 2024-04-25T09:55:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 -- 172.21.15.19:0/2349053292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f0154c50 msgr2=0x7f28f015f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f0154c50 0x7f28f015f6a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f28f00b64d0 tx=0x7f28e800ebe0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 -- 172.21.15.19:0/2349053292 shutdown_connections 2024-04-25T09:55:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d406b4d0 0x7f28d406d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f0154c50 0x7f28f015f6a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28f01532c0 0x7f28f0152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 --2- 172.21.15.19:0/2349053292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28e0004800 0x7f28f0159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 -- 172.21.15.19:0/2349053292 >> 172.21.15.19:0/2349053292 conn(0x7f28f0061c30 msgr2=0x7f28f00ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 -- 172.21.15.19:0/2349053292 shutdown_connections 2024-04-25T09:55:35.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.337+0000 7f28f701f640 1 -- 172.21.15.19:0/2349053292 wait complete. 2024-04-25T09:55:35.406 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:55:35.406 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:55:35.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:55:35.492 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:55:35.493 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:55:35.493 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:35.508 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:55:35.508 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:35.564 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.702+0000 7fc638c93640 1 Processor -- start 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.702+0000 7fc638c93640 1 -- start start 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.703+0000 7fc638c93640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc63415dc00 0x7fc6340c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.703+0000 7fc638c93640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6340d1dd0 0x7fc6340ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.703+0000 7fc638c93640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63415dfe0 con 0x7fc6340ceb20 2024-04-25T09:55:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.703+0000 7fc638c93640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6340d00c0 con 0x7fc6340d1dd0 2024-04-25T09:55:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.703+0000 7fc638c93640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6340d03a0 con 0x7fc63415dc00 2024-04-25T09:55:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.703+0000 7fc632d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6340d1dd0 0x7fc6340ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.703+0000 7fc632d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6340d1dd0 0x7fc6340ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48254/0 (socket says 172.21.15.19:48254) 2024-04-25T09:55:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.703+0000 7fc632d9d640 1 -- 172.21.15.19:0/2844342244 learned_addr learned my addr 172.21.15.19:0/2844342244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.703+0000 7fc63259c640 1 --2- 172.21.15.19:0/2844342244 >> v2:172.21.15.112:3300/0 conn(0x7fc63415dc00 0x7fc6340c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.704+0000 7fc632d9d640 1 -- 172.21.15.19:0/2844342244 >> v2:172.21.15.112:3300/0 conn(0x7fc63415dc00 msgr2=0x7fc6340c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.704+0000 7fc632d9d640 1 --2- 172.21.15.19:0/2844342244 >> v2:172.21.15.112:3300/0 conn(0x7fc63415dc00 0x7fc6340c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.704+0000 7fc632d9d640 1 -- 172.21.15.19:0/2844342244 >> v1:172.21.15.112:6789/0 conn(0x7fc6340ceb20 legacy=0x7fc6340d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.704+0000 7fc632d9d640 1 -- 172.21.15.19:0/2844342244 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6340cc440 con 0x7fc6340d1dd0 2024-04-25T09:55:35.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.704+0000 7fc632d9d640 1 --2- 172.21.15.19:0/2844342244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6340d1dd0 0x7fc6340ccc40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc63406efe0 tx=0x7fc62402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d902563aed4ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.704+0000 7fc63159a640 1 -- 172.21.15.19:0/2844342244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc624036070 con 0x7fc6340d1dd0 2024-04-25T09:55:35.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.704+0000 7fc63159a640 1 -- 172.21.15.19:0/2844342244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc624031600 con 0x7fc6340d1dd0 2024-04-25T09:55:35.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.704+0000 7fc63159a640 1 -- 172.21.15.19:0/2844342244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6240319d0 con 0x7fc6340d1dd0 2024-04-25T09:55:35.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.705+0000 7fc638c93640 1 -- 172.21.15.19:0/2844342244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6340d1dd0 msgr2=0x7fc6340ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.705+0000 7fc638c93640 1 --2- 172.21.15.19:0/2844342244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6340d1dd0 0x7fc6340ccc40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc63406efe0 tx=0x7fc62402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.705+0000 7fc638c93640 1 -- 172.21.15.19:0/2844342244 shutdown_connections 2024-04-25T09:55:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.705+0000 7fc638c93640 1 --2- 172.21.15.19:0/2844342244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6340d1dd0 0x7fc6340ccc40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.705+0000 7fc638c93640 1 --2- 172.21.15.19:0/2844342244 >> v2:172.21.15.112:3300/0 conn(0x7fc63415dc00 0x7fc6340c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.705+0000 7fc638c93640 1 -- 172.21.15.19:0/2844342244 >> 172.21.15.19:0/2844342244 conn(0x7fc63406f5d0 msgr2=0x7fc63406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.705+0000 7fc638c93640 1 -- 172.21.15.19:0/2844342244 shutdown_connections 2024-04-25T09:55:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.705+0000 7fc638c93640 1 -- 172.21.15.19:0/2844342244 wait complete. 2024-04-25T09:55:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.706+0000 7fc638c93640 1 Processor -- start 2024-04-25T09:55:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.706+0000 7fc638c93640 1 -- start start 2024-04-25T09:55:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.707+0000 7fc638c93640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6340ceb20 0x7fc634153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.707+0000 7fc638c93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6340d1dd0 0x7fc63415ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.707+0000 7fc638c93640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc63415dc00 0x7fc634157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.707+0000 7fc638c93640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6340cfd90 con 0x7fc6340d1dd0 2024-04-25T09:55:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.707+0000 7fc638c93640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6340cc130 con 0x7fc6340ceb20 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.707+0000 7fc638c93640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6340cdd70 con 0x7fc63415dc00 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.707+0000 7fc631d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6340d1dd0 0x7fc63415ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.707+0000 7fc631d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6340d1dd0 0x7fc63415ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48260/0 (socket says 172.21.15.19:48260) 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.707+0000 7fc631d9b640 1 -- 172.21.15.19:0/428107688 learned_addr learned my addr 172.21.15.19:0/428107688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.707+0000 7fc632d9d640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc63415dc00 0x7fc634157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.707+0000 7fc63259c640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6340ceb20 0x7fc634153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.708+0000 7fc631d9b640 1 -- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc63415dc00 msgr2=0x7fc634157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.708+0000 7fc631d9b640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc63415dc00 0x7fc634157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.708+0000 7fc631d9b640 1 -- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6340ceb20 msgr2=0x7fc634153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.708+0000 7fc631d9b640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6340ceb20 0x7fc634153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.708+0000 7fc631d9b640 1 -- 172.21.15.19:0/428107688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc634154430 con 0x7fc6340d1dd0 2024-04-25T09:55:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.708+0000 7fc631d9b640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6340d1dd0 0x7fc63415ab30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00c430 tx=0x7fc61c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.708+0000 7fc62affd640 1 -- 172.21.15.19:0/428107688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c011790 con 0x7fc6340d1dd0 2024-04-25T09:55:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.708+0000 7fc62affd640 1 -- 172.21.15.19:0/428107688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c017070 con 0x7fc6340d1dd0 2024-04-25T09:55:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.708+0000 7fc638c93640 1 -- 172.21.15.19:0/428107688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc634154710 con 0x7fc6340d1dd0 2024-04-25T09:55:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.708+0000 7fc62affd640 1 -- 172.21.15.19:0/428107688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c011bb0 con 0x7fc6340d1dd0 2024-04-25T09:55:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.708+0000 7fc638c93640 1 -- 172.21.15.19:0/428107688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc634154cd0 con 0x7fc6340d1dd0 2024-04-25T09:55:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.709+0000 7fc638c93640 1 -- 172.21.15.19:0/428107688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6340cd9e0 con 0x7fc6340d1dd0 2024-04-25T09:55:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.709+0000 7fc62affd640 1 -- 172.21.15.19:0/428107688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc61c01c460 con 0x7fc6340d1dd0 2024-04-25T09:55:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.710+0000 7fc62affd640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 0x7fc61406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.710+0000 7fc62affd640 1 -- 172.21.15.19:0/428107688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fc61c08a240 con 0x7fc6340d1dd0 2024-04-25T09:55:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.710+0000 7fc63259c640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 0x7fc61406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.711+0000 7fc63259c640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 0x7fc61406d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc62c006fd0 tx=0x7fc62c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.714+0000 7fc62affd640 1 -- 172.21.15.19:0/428107688 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6340cd9e0 con 0x7fc6340d1dd0 2024-04-25T09:55:35.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.894+0000 7fc638c93640 1 -- 172.21.15.19:0/428107688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6340c66c0 con 0x7fc6340d1dd0 2024-04-25T09:55:35.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.895+0000 7fc62affd640 1 -- 172.21.15.19:0/428107688 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16988 (secure 0 0 0) 0x7fc61c058f00 con 0x7fc6340d1dd0 2024-04-25T09:55:35.898 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:35.898 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:12.545580+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:11.132892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:11.536478+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:35.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 -- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 msgr2=0x7fc61406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 0x7fc61406d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc62c006fd0 tx=0x7fc62c008040 comp rx=0 tx=0).stop 2024-04-25T09:55:35.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 -- 172.21.15.19:0/428107688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6340d1dd0 msgr2=0x7fc63415ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6340d1dd0 0x7fc63415ab30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00c430 tx=0x7fc61c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 -- 172.21.15.19:0/428107688 shutdown_connections 2024-04-25T09:55:35.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 0x7fc61406d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc63415dc00 0x7fc634157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6340d1dd0 0x7fc63415ab30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 --2- 172.21.15.19:0/428107688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6340ceb20 0x7fc634153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 -- 172.21.15.19:0/428107688 >> 172.21.15.19:0/428107688 conn(0x7fc63406f5d0 msgr2=0x7fc6340b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 -- 172.21.15.19:0/428107688 shutdown_connections 2024-04-25T09:55:35.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:35.897+0000 7fc638c93640 1 -- 172.21.15.19:0/428107688 wait complete. 2024-04-25T09:55:35.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:11.132892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:11.536478+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 41, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:55:35.911 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:36.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.012+0000 7fc3f819e640 1 Processor -- start 2024-04-25T09:55:36.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.012+0000 7fc3f819e640 1 -- start start 2024-04-25T09:55:36.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.013+0000 7fc3f819e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc3f0161710 0x7fc3f00d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.013+0000 7fc3f819e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f015e000 con 0x7fc3f00ced80 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.013+0000 7fc3f819e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f00d1de0 con 0x7fc3f015dc20 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.013+0000 7fc3f819e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f0161c60 con 0x7fc3f0161710 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.013+0000 7fc3f5739640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc3f00ced80 0x7fc3f00d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54404/0 (socket says 172.21.15.19:54404) 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.013+0000 7fc3f5739640 1 -- 172.21.15.19:0/3090681155 learned_addr learned my addr 172.21.15.19:0/3090681155 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.013+0000 7fc3f673b640 1 --2- 172.21.15.19:0/3090681155 >> v2:172.21.15.112:3300/0 conn(0x7fc3f0161710 0x7fc3f00d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.013+0000 7fc3f4f38640 1 -- 172.21.15.19:0/3090681155 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349068749 0 0) 0x7fc3f015e000 con 0x7fc3f00ced80 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.013+0000 7fc3f4f38640 1 -- 172.21.15.19:0/3090681155 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3dc003520 con 0x7fc3f00ced80 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.013+0000 7fc3f4f38640 1 -- 172.21.15.19:0/3090681155 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395520374 0 0) 0x7fc3f00d1de0 con 0x7fc3f015dc20 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.013+0000 7fc3f4f38640 1 -- 172.21.15.19:0/3090681155 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3f015e000 con 0x7fc3f015dc20 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.014+0000 7fc3f673b640 1 -- 172.21.15.19:0/3090681155 >> v1:172.21.15.112:6789/0 conn(0x7fc3f015dc20 legacy=0x7fc3f00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.014+0000 7fc3f673b640 1 -- 172.21.15.19:0/3090681155 >> v1:172.21.15.19:6789/0 conn(0x7fc3f00ced80 legacy=0x7fc3f00d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.014+0000 7fc3f673b640 1 -- 172.21.15.19:0/3090681155 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f0161f40 con 0x7fc3f0161710 2024-04-25T09:55:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.014+0000 7fc3f673b640 1 --2- 172.21.15.19:0/3090681155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3f0161710 0x7fc3f00d29f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009d90 tx=0x7fc3e802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b59d222667e85a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.014+0000 7fc3f4f38640 1 -- 172.21.15.19:0/3090681155 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8007740 con 0x7fc3f0161710 2024-04-25T09:55:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.015+0000 7fc3f4f38640 1 -- 172.21.15.19:0/3090681155 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e80078e0 con 0x7fc3f0161710 2024-04-25T09:55:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.015+0000 7fc3f4f38640 1 -- 172.21.15.19:0/3090681155 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8007c90 con 0x7fc3f0161710 2024-04-25T09:55:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.015+0000 7fc3f819e640 1 -- 172.21.15.19:0/3090681155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3f0161710 msgr2=0x7fc3f00d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.015+0000 7fc3f819e640 1 --2- 172.21.15.19:0/3090681155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3f0161710 0x7fc3f00d29f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009d90 tx=0x7fc3e802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.015+0000 7fc3f819e640 1 -- 172.21.15.19:0/3090681155 shutdown_connections 2024-04-25T09:55:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.015+0000 7fc3f819e640 1 --2- 172.21.15.19:0/3090681155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3f0161710 0x7fc3f00d29f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.015+0000 7fc3f819e640 1 -- 172.21.15.19:0/3090681155 >> 172.21.15.19:0/3090681155 conn(0x7fc3f00d0420 msgr2=0x7fc3f00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.015+0000 7fc3f819e640 1 -- 172.21.15.19:0/3090681155 shutdown_connections 2024-04-25T09:55:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.015+0000 7fc3f819e640 1 -- 172.21.15.19:0/3090681155 wait complete. 2024-04-25T09:55:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.016+0000 7fc3f819e640 1 Processor -- start 2024-04-25T09:55:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.016+0000 7fc3f819e640 1 -- start start 2024-04-25T09:55:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.017+0000 7fc3f819e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3f00ced80 0x7fc3f014f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.017+0000 7fc3f819e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3f015dc20 0x7fc3f0160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.017+0000 7fc3f819e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f0161710 0x7fc3f01f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.017+0000 7fc3f819e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f00b5e50 con 0x7fc3f0161710 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.017+0000 7fc3f819e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f0162500 con 0x7fc3f00ced80 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.017+0000 7fc3f819e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f00b5fd0 con 0x7fc3f015dc20 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.017+0000 7fc3f673b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f0161710 0x7fc3f01f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.017+0000 7fc3f673b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f0161710 0x7fc3f01f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48262/0 (socket says 172.21.15.19:48262) 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.017+0000 7fc3f673b640 1 -- 172.21.15.19:0/794223656 learned_addr learned my addr 172.21.15.19:0/794223656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.017+0000 7fc3f5f3a640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3f00ced80 0x7fc3f014f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.017+0000 7fc3f5739640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3f015dc20 0x7fc3f0160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.018+0000 7fc3f673b640 1 -- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3f015dc20 msgr2=0x7fc3f0160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.018+0000 7fc3f673b640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3f015dc20 0x7fc3f0160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.018+0000 7fc3f673b640 1 -- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3f00ced80 msgr2=0x7fc3f014f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.018+0000 7fc3f673b640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3f00ced80 0x7fc3f014f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.018+0000 7fc3f673b640 1 -- 172.21.15.19:0/794223656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f015fd50 con 0x7fc3f0161710 2024-04-25T09:55:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.018+0000 7fc3f673b640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f0161710 0x7fc3f01f3660 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009eb0 tx=0x7fc3e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.018+0000 7fc3e67fc640 1 -- 172.21.15.19:0/794223656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8040070 con 0x7fc3f0161710 2024-04-25T09:55:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.018+0000 7fc3e67fc640 1 -- 172.21.15.19:0/794223656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e8031c60 con 0x7fc3f0161710 2024-04-25T09:55:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.018+0000 7fc3f819e640 1 -- 172.21.15.19:0/794223656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f0156c70 con 0x7fc3f0161710 2024-04-25T09:55:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.018+0000 7fc3e67fc640 1 -- 172.21.15.19:0/794223656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8009450 con 0x7fc3f0161710 2024-04-25T09:55:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.018+0000 7fc3f819e640 1 -- 172.21.15.19:0/794223656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f00c9c50 con 0x7fc3f0161710 2024-04-25T09:55:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.019+0000 7fc3f819e640 1 -- 172.21.15.19:0/794223656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f0073a00 con 0x7fc3f0161710 2024-04-25T09:55:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.019+0000 7fc3e67fc640 1 -- 172.21.15.19:0/794223656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3e8039040 con 0x7fc3f0161710 2024-04-25T09:55:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.020+0000 7fc3e67fc640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3d406b4d0 0x7fc3d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.020+0000 7fc3e67fc640 1 -- 172.21.15.19:0/794223656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fc3e80ab670 con 0x7fc3f0161710 2024-04-25T09:55:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.020+0000 7fc3f5f3a640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3d406b4d0 0x7fc3d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.021+0000 7fc3f5f3a640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3d406b4d0 0x7fc3d406d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0158af0 tx=0x7fc3ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.024+0000 7fc3e67fc640 1 -- 172.21.15.19:0/794223656 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3e807a3c0 con 0x7fc3f0161710 2024-04-25T09:55:36.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.202+0000 7fc3f819e640 1 -- 172.21.15.19:0/794223656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3f00d16b0 con 0x7fc3f0161710 2024-04-25T09:55:36.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.202+0000 7fc3e67fc640 1 -- 172.21.15.19:0/794223656 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3e8079cc0 con 0x7fc3f0161710 2024-04-25T09:55:36.205 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:55:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 -- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3d406b4d0 msgr2=0x7fc3d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3d406b4d0 0x7fc3d406d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0158af0 tx=0x7fc3ec009040 comp rx=0 tx=0).stop 2024-04-25T09:55:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 -- 172.21.15.19:0/794223656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f0161710 msgr2=0x7fc3f01f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f0161710 0x7fc3f01f3660 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009eb0 tx=0x7fc3e8004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 -- 172.21.15.19:0/794223656 shutdown_connections 2024-04-25T09:55:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc3d406b4d0 0x7fc3d406d990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f0161710 0x7fc3f01f3660 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3f015dc20 0x7fc3f0160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 --2- 172.21.15.19:0/794223656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3f00ced80 0x7fc3f014f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 -- 172.21.15.19:0/794223656 >> 172.21.15.19:0/794223656 conn(0x7fc3f00d0420 msgr2=0x7fc3f00b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 -- 172.21.15.19:0/794223656 shutdown_connections 2024-04-25T09:55:36.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.204+0000 7fc3f819e640 1 -- 172.21.15.19:0/794223656 wait complete. 2024-04-25T09:55:36.217 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.318+0000 7eff5cf22640 1 Processor -- start 2024-04-25T09:55:36.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.318+0000 7eff5cf22640 1 -- start start 2024-04-25T09:55:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.319+0000 7eff5cf22640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7eff580ced80 0x7eff580d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.319+0000 7eff5cf22640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7eff58161710 0x7eff580d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.319+0000 7eff5cf22640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5815e000 con 0x7eff5815dc20 2024-04-25T09:55:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.319+0000 7eff5cf22640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff580d1de0 con 0x7eff58161710 2024-04-25T09:55:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.319+0000 7eff5cf22640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff58161c60 con 0x7eff580ced80 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.319+0000 7eff56d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7eff58161710 0x7eff580d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.319+0000 7eff5659c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7eff5815dc20 0x7eff580ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54418/0 (socket says 172.21.15.19:54418) 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.319+0000 7eff5659c640 1 -- 172.21.15.19:0/2497301179 learned_addr learned my addr 172.21.15.19:0/2497301179 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.319+0000 7eff55d9b640 1 --2- 172.21.15.19:0/2497301179 >> v2:172.21.15.112:3300/0 conn(0x7eff580ced80 0x7eff580d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.320+0000 7eff56d9d640 1 -- 172.21.15.19:0/2497301179 >> v2:172.21.15.112:3300/0 conn(0x7eff580ced80 msgr2=0x7eff580d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.320+0000 7eff56d9d640 1 --2- 172.21.15.19:0/2497301179 >> v2:172.21.15.112:3300/0 conn(0x7eff580ced80 0x7eff580d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.320+0000 7eff56d9d640 1 -- 172.21.15.19:0/2497301179 >> v1:172.21.15.19:6789/0 conn(0x7eff5815dc20 legacy=0x7eff580ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.320+0000 7eff5559a640 1 -- 172.21.15.19:0/2497301179 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1291467036 0 0) 0x7eff5815e000 con 0x7eff5815dc20 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.320+0000 7eff56d9d640 1 -- 172.21.15.19:0/2497301179 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff58161f40 con 0x7eff58161710 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.320+0000 7eff56d9d640 1 --2- 172.21.15.19:0/2497301179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff58161710 0x7eff580d2820 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7eff5806da90 tx=0x7eff4802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dd10a2856c5dca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.321+0000 7eff5559a640 1 -- 172.21.15.19:0/2497301179 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48007740 con 0x7eff58161710 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.321+0000 7eff5559a640 1 -- 172.21.15.19:0/2497301179 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff480078e0 con 0x7eff58161710 2024-04-25T09:55:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.321+0000 7eff5559a640 1 -- 172.21.15.19:0/2497301179 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48007cb0 con 0x7eff58161710 2024-04-25T09:55:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.321+0000 7eff5cf22640 1 -- 172.21.15.19:0/2497301179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff58161710 msgr2=0x7eff580d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.321+0000 7eff5cf22640 1 --2- 172.21.15.19:0/2497301179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff58161710 0x7eff580d2820 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7eff5806da90 tx=0x7eff4802a680 comp rx=0 tx=0).stop 2024-04-25T09:55:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.321+0000 7eff5cf22640 1 -- 172.21.15.19:0/2497301179 shutdown_connections 2024-04-25T09:55:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.321+0000 7eff5cf22640 1 --2- 172.21.15.19:0/2497301179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff58161710 0x7eff580d2820 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.321+0000 7eff5cf22640 1 --2- 172.21.15.19:0/2497301179 >> v2:172.21.15.112:3300/0 conn(0x7eff580ced80 0x7eff580d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.321+0000 7eff5cf22640 1 -- 172.21.15.19:0/2497301179 >> 172.21.15.19:0/2497301179 conn(0x7eff580d0420 msgr2=0x7eff580d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.321+0000 7eff5cf22640 1 -- 172.21.15.19:0/2497301179 shutdown_connections 2024-04-25T09:55:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.321+0000 7eff5cf22640 1 -- 172.21.15.19:0/2497301179 wait complete. 2024-04-25T09:55:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.323+0000 7eff5cf22640 1 Processor -- start 2024-04-25T09:55:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.323+0000 7eff5cf22640 1 -- start start 2024-04-25T09:55:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.323+0000 7eff5cf22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff580ced80 0x7eff5815e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.323+0000 7eff5cf22640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5815dc20 0x7eff5815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.323+0000 7eff5cf22640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff58161710 0x7eff580c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.323+0000 7eff5cf22640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff580b5e50 con 0x7eff5815dc20 2024-04-25T09:55:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.323+0000 7eff5cf22640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff58162500 con 0x7eff580ced80 2024-04-25T09:55:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.323+0000 7eff5cf22640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff580b5fd0 con 0x7eff58161710 2024-04-25T09:55:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.323+0000 7eff55d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5815dc20 0x7eff5815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff55d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5815dc20 0x7eff5815ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48284/0 (socket says 172.21.15.19:48284) 2024-04-25T09:55:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff55d9b640 1 -- 172.21.15.19:0/2709144021 learned_addr learned my addr 172.21.15.19:0/2709144021 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff56d9d640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff58161710 0x7eff580c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff5659c640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff580ced80 0x7eff5815e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff55d9b640 1 -- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff58161710 msgr2=0x7eff580c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff55d9b640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff58161710 0x7eff580c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff55d9b640 1 -- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff580ced80 msgr2=0x7eff5815e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff55d9b640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff580ced80 0x7eff5815e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff55d9b640 1 -- 172.21.15.19:0/2709144021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5815f500 con 0x7eff5815dc20 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff5659c640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff580ced80 0x7eff5815e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff56d9d640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff58161710 0x7eff580c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff55d9b640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5815dc20 0x7eff5815ebe0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00e870 tx=0x7eff4c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff46ffd640 1 -- 172.21.15.19:0/2709144021 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c017070 con 0x7eff5815dc20 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff46ffd640 1 -- 172.21.15.19:0/2709144021 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4c00c940 con 0x7eff5815dc20 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff46ffd640 1 -- 172.21.15.19:0/2709144021 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00cd30 con 0x7eff5815dc20 2024-04-25T09:55:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff5cf22640 1 -- 172.21.15.19:0/2709144021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff581e81e0 con 0x7eff5815dc20 2024-04-25T09:55:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.324+0000 7eff5cf22640 1 -- 172.21.15.19:0/2709144021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff581e87a0 con 0x7eff5815dc20 2024-04-25T09:55:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.325+0000 7eff5cf22640 1 -- 172.21.15.19:0/2709144021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff580cf380 con 0x7eff5815dc20 2024-04-25T09:55:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.326+0000 7eff46ffd640 1 -- 172.21.15.19:0/2709144021 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff4c01a070 con 0x7eff5815dc20 2024-04-25T09:55:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.326+0000 7eff46ffd640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff3806b4d0 0x7eff3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.326+0000 7eff46ffd640 1 -- 172.21.15.19:0/2709144021 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7eff4c08a280 con 0x7eff5815dc20 2024-04-25T09:55:36.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.326+0000 7eff5659c640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff3806b4d0 0x7eff3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.327+0000 7eff5659c640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff3806b4d0 0x7eff3806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff50000f60 tx=0x7eff50009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.330+0000 7eff46ffd640 1 -- 172.21.15.19:0/2709144021 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff580cf380 con 0x7eff5815dc20 2024-04-25T09:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.509+0000 7eff5cf22640 1 -- 172.21.15.19:0/2709144021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff580cf940 con 0x7eff5815dc20 2024-04-25T09:55:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.509+0000 7eff46ffd640 1 -- 172.21.15.19:0/2709144021 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff580cf940 con 0x7eff5815dc20 2024-04-25T09:55:36.512 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:55:36.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.511+0000 7eff5cf22640 1 -- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff3806b4d0 msgr2=0x7eff3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.511+0000 7eff5cf22640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff3806b4d0 0x7eff3806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff50000f60 tx=0x7eff50009040 comp rx=0 tx=0).stop 2024-04-25T09:55:36.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.511+0000 7eff5cf22640 1 -- 172.21.15.19:0/2709144021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5815dc20 msgr2=0x7eff5815ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.512+0000 7eff5cf22640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5815dc20 0x7eff5815ebe0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00e870 tx=0x7eff4c00ed30 comp rx=0 tx=0).stop 2024-04-25T09:55:36.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.512+0000 7eff5cf22640 1 -- 172.21.15.19:0/2709144021 shutdown_connections 2024-04-25T09:55:36.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.512+0000 7eff5cf22640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff3806b4d0 0x7eff3806d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.512+0000 7eff5cf22640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff58161710 0x7eff580c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.512+0000 7eff5cf22640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5815dc20 0x7eff5815ebe0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.512+0000 7eff5cf22640 1 --2- 172.21.15.19:0/2709144021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff580ced80 0x7eff5815e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.512+0000 7eff5cf22640 1 -- 172.21.15.19:0/2709144021 >> 172.21.15.19:0/2709144021 conn(0x7eff580d0420 msgr2=0x7eff580d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.512+0000 7eff5cf22640 1 -- 172.21.15.19:0/2709144021 shutdown_connections 2024-04-25T09:55:36.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.512+0000 7eff5cf22640 1 -- 172.21.15.19:0/2709144021 wait complete. 2024-04-25T09:55:36.525 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:36.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.626+0000 7f961586f640 1 Processor -- start 2024-04-25T09:55:36.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.626+0000 7f961586f640 1 -- start start 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f961586f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9610161710 0x7f96100d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f961586f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961015e000 con 0x7f961015dc20 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f961586f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96100d1de0 con 0x7f96100ced80 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f961586f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9610161c60 con 0x7f9610161710 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960effd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f961015dc20 0x7f96100ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54424/0 (socket says 172.21.15.19:54424) 2024-04-25T09:55:36.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960effd640 1 -- 172.21.15.19:0/220817411 learned_addr learned my addr 172.21.15.19:0/220817411 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:36.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960f7fe640 1 --2- 172.21.15.19:0/220817411 >> v2:172.21.15.112:3300/0 conn(0x7f9610161710 0x7f96100d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358512332 0 0) 0x7f961015e000 con 0x7f961015dc20 2024-04-25T09:55:36.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95fc003520 con 0x7f961015dc20 2024-04-25T09:55:36.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1686313617 0 0) 0x7f95fc003520 con 0x7f961015dc20 2024-04-25T09:55:36.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 >> v2:172.21.15.112:3300/0 conn(0x7f9610161710 msgr2=0x7f96100d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 --2- 172.21.15.19:0/220817411 >> v2:172.21.15.112:3300/0 conn(0x7f9610161710 0x7f96100d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 >> v1:172.21.15.112:6789/0 conn(0x7f96100ced80 legacy=0x7f96100d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9610161f40 con 0x7f961015dc20 2024-04-25T09:55:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f9600001670 con 0x7f961015dc20 2024-04-25T09:55:36.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 >> v1:172.21.15.19:6789/0 conn(0x7f961015dc20 legacy=0x7f96100ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95fc0039b0 0x7f95fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc0043b0 0x7f95fc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc005080 0x7f95fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9610161c60 con 0x7f95fc0043b0 2024-04-25T09:55:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc00bfe0 con 0x7f95fc005080 2024-04-25T09:55:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc00c1e0 con 0x7f95fc0039b0 2024-04-25T09:55:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.627+0000 7f960f7fe640 1 --2- 172.21.15.19:0/220817411 >> v2:172.21.15.112:3300/0 conn(0x7f9610161710 0x7f96100d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.628+0000 7f9607fff640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc0043b0 0x7f95fc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.628+0000 7f960effd640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95fc0039b0 0x7f95fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.628+0000 7f960f7fe640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc005080 0x7f95fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.628+0000 7f9607fff640 1 -- 172.21.15.19:0/220817411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95fc0039b0 msgr2=0x7f95fc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.628+0000 7f9607fff640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95fc0039b0 0x7f95fc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.628+0000 7f9607fff640 1 -- 172.21.15.19:0/220817411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc005080 msgr2=0x7f95fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.628+0000 7f9607fff640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc005080 0x7f95fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.628+0000 7f9607fff640 1 -- 172.21.15.19:0/220817411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc003520 con 0x7f95fc0043b0 2024-04-25T09:55:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.628+0000 7f960f7fe640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc005080 0x7f95fc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:36.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.628+0000 7f960f7fe640 1 -- 172.21.15.19:0/220817411 reap_dead start 2024-04-25T09:55:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.629+0000 7f9607fff640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc0043b0 0x7f95fc004880 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f95f8001860 tx=0x7f95fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16acd188fd2e0583 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.629+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8037070 con 0x7f95fc0043b0 2024-04-25T09:55:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.629+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f802abe0 con 0x7f95fc0043b0 2024-04-25T09:55:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.629+0000 7f960e7fc640 1 -- 172.21.15.19:0/220817411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8001430 con 0x7f95fc0043b0 2024-04-25T09:55:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.629+0000 7f961586f640 1 -- 172.21.15.19:0/220817411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc0043b0 msgr2=0x7f95fc004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.629+0000 7f961586f640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc0043b0 0x7f95fc004880 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f95f8001860 tx=0x7f95fc005830 comp rx=0 tx=0).stop 2024-04-25T09:55:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.629+0000 7f961586f640 1 -- 172.21.15.19:0/220817411 shutdown_connections 2024-04-25T09:55:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.629+0000 7f961586f640 1 --2- 172.21.15.19:0/220817411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc0043b0 0x7f95fc004880 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.629+0000 7f961586f640 1 -- 172.21.15.19:0/220817411 >> 172.21.15.19:0/220817411 conn(0x7f96100d0420 msgr2=0x7f96100d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.629+0000 7f961586f640 1 -- 172.21.15.19:0/220817411 shutdown_connections 2024-04-25T09:55:36.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.629+0000 7f961586f640 1 -- 172.21.15.19:0/220817411 wait complete. 2024-04-25T09:55:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.630+0000 7f961586f640 1 Processor -- start 2024-04-25T09:55:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.631+0000 7f961586f640 1 -- start start 2024-04-25T09:55:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.631+0000 7f961586f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc0043b0 0x7f961015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.631+0000 7f961586f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96101499b0 0x7f96101e95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.631+0000 7f961586f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9610149e30 0x7f96101e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.631+0000 7f961586f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96100b5e50 con 0x7f9610149e30 2024-04-25T09:55:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.631+0000 7f961586f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9610162500 con 0x7f95fc0043b0 2024-04-25T09:55:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.631+0000 7f961586f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96100b5fd0 con 0x7f96101499b0 2024-04-25T09:55:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.631+0000 7f960f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9610149e30 0x7f96101e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.631+0000 7f960f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9610149e30 0x7f96101e9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48298/0 (socket says 172.21.15.19:48298) 2024-04-25T09:55:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.631+0000 7f960f7fe640 1 -- 172.21.15.19:0/3116634742 learned_addr learned my addr 172.21.15.19:0/3116634742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f9607fff640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96101499b0 0x7f96101e95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f960effd640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc0043b0 0x7f961015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f960f7fe640 1 -- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96101499b0 msgr2=0x7f96101e95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f960f7fe640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96101499b0 0x7f96101e95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f960f7fe640 1 -- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc0043b0 msgr2=0x7f961015c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f960f7fe640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc0043b0 0x7f961015c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f960f7fe640 1 -- 172.21.15.19:0/3116634742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96101e8540 con 0x7f9610149e30 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f9607fff640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96101499b0 0x7f96101e95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f960f7fe640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9610149e30 0x7f96101e9ae0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9608009ed0 tx=0x7f96080115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f960cff9640 1 -- 172.21.15.19:0/3116634742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608011e40 con 0x7f9610149e30 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f960cff9640 1 -- 172.21.15.19:0/3116634742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960801c070 con 0x7f9610149e30 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f961586f640 1 -- 172.21.15.19:0/3116634742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96101e87e0 con 0x7f9610149e30 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f960cff9640 1 -- 172.21.15.19:0/3116634742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96080166b0 con 0x7f9610149e30 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.632+0000 7f961586f640 1 -- 172.21.15.19:0/3116634742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96101e8da0 con 0x7f9610149e30 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.633+0000 7f961586f640 1 -- 172.21.15.19:0/3116634742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9610073a00 con 0x7f9610149e30 2024-04-25T09:55:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.633+0000 7f960cff9640 1 -- 172.21.15.19:0/3116634742 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9608005d10 con 0x7f9610149e30 2024-04-25T09:55:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.634+0000 7f960cff9640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95f006b500 0x7f95f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.634+0000 7f960cff9640 1 -- 172.21.15.19:0/3116634742 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f960808aad0 con 0x7f9610149e30 2024-04-25T09:55:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.634+0000 7f960effd640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95f006b500 0x7f95f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.634+0000 7f960effd640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95f006b500 0x7f95f006d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9600001050 tx=0x7f9600009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.638+0000 7f960cff9640 1 -- 172.21.15.19:0/3116634742 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96080596f0 con 0x7f9610149e30 2024-04-25T09:55:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.816+0000 7f961586f640 1 -- 172.21.15.19:0/3116634742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96100d2f20 con 0x7f9610149e30 2024-04-25T09:55:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.816+0000 7f960cff9640 1 -- 172.21.15.19:0/3116634742 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9608058ff0 con 0x7f9610149e30 2024-04-25T09:55:36.819 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 -- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95f006b500 msgr2=0x7f95f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95f006b500 0x7f95f006d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9600001050 tx=0x7f9600009040 comp rx=0 tx=0).stop 2024-04-25T09:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 -- 172.21.15.19:0/3116634742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9610149e30 msgr2=0x7f96101e9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9610149e30 0x7f96101e9ae0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9608009ed0 tx=0x7f96080115a0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 -- 172.21.15.19:0/3116634742 shutdown_connections 2024-04-25T09:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95f006b500 0x7f95f006d9c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9610149e30 0x7f96101e9ae0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96101499b0 0x7f96101e95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 --2- 172.21.15.19:0/3116634742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc0043b0 0x7f961015c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 -- 172.21.15.19:0/3116634742 >> 172.21.15.19:0/3116634742 conn(0x7f96100d0420 msgr2=0x7f96100b87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 -- 172.21.15.19:0/3116634742 shutdown_connections 2024-04-25T09:55:36.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.818+0000 7f961586f640 1 -- 172.21.15.19:0/3116634742 wait complete. 2024-04-25T09:55:36.832 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.930+0000 7f6dace4d640 1 Processor -- start 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.930+0000 7f6dace4d640 1 -- start start 2024-04-25T09:55:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.931+0000 7f6dace4d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6da8166a10 0x7f6da8168e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.931+0000 7f6dace4d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da80afc80 con 0x7f6da815de80 2024-04-25T09:55:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.931+0000 7f6dace4d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da80ce4d0 con 0x7f6da80cadb0 2024-04-25T09:55:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.931+0000 7f6dace4d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da80cb240 con 0x7f6da8166a10 2024-04-25T09:55:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.931+0000 7f6da659c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6da815de80 0x7f6da80cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54436/0 (socket says 172.21.15.19:54436) 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.931+0000 7f6da659c640 1 -- 172.21.15.19:0/3929491419 learned_addr learned my addr 172.21.15.19:0/3929491419 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.932+0000 7f6da6d9d640 1 --2- 172.21.15.19:0/3929491419 >> v2:172.21.15.112:3300/0 conn(0x7f6da8166a10 0x7f6da8168e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.932+0000 7f6da559a640 1 -- 172.21.15.19:0/3929491419 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893577636 0 0) 0x7f6da80afc80 con 0x7f6da815de80 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.932+0000 7f6da559a640 1 -- 172.21.15.19:0/3929491419 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d90003520 con 0x7f6da815de80 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.932+0000 7f6da559a640 1 -- 172.21.15.19:0/3929491419 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084633585 0 0) 0x7f6da80ce4d0 con 0x7f6da80cadb0 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.932+0000 7f6da559a640 1 -- 172.21.15.19:0/3929491419 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da80afc80 con 0x7f6da80cadb0 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.932+0000 7f6da6d9d640 1 -- 172.21.15.19:0/3929491419 >> v1:172.21.15.112:6789/0 conn(0x7f6da80cadb0 legacy=0x7f6da80cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.932+0000 7f6da6d9d640 1 -- 172.21.15.19:0/3929491419 >> v1:172.21.15.19:6789/0 conn(0x7f6da815de80 legacy=0x7f6da80cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.932+0000 7f6da559a640 1 -- 172.21.15.19:0/3929491419 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 904466774 0 0) 0x7f6d90003520 con 0x7f6da815de80 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.932+0000 7f6da6d9d640 1 -- 172.21.15.19:0/3929491419 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da80ce7b0 con 0x7f6da8166a10 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.933+0000 7f6da6d9d640 1 --2- 172.21.15.19:0/3929491419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da8166a10 0x7f6da8168e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009d90 tx=0x7f6d9802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3eeb248883e5e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.933+0000 7f6da559a640 1 -- 172.21.15.19:0/3929491419 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0001670 con 0x7f6da8166a10 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.933+0000 7f6da559a640 1 -- 172.21.15.19:0/3929491419 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d98007540 con 0x7f6da8166a10 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.933+0000 7f6da559a640 1 -- 172.21.15.19:0/3929491419 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d980078f0 con 0x7f6da8166a10 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.933+0000 7f6dace4d640 1 -- 172.21.15.19:0/3929491419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da8166a10 msgr2=0x7f6da8168e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.933+0000 7f6dace4d640 1 --2- 172.21.15.19:0/3929491419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da8166a10 0x7f6da8168e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009d90 tx=0x7f6d9802a5f0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.934+0000 7f6dace4d640 1 -- 172.21.15.19:0/3929491419 shutdown_connections 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.934+0000 7f6dace4d640 1 --2- 172.21.15.19:0/3929491419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da8166a10 0x7f6da8168e00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.934+0000 7f6dace4d640 1 -- 172.21.15.19:0/3929491419 >> 172.21.15.19:0/3929491419 conn(0x7f6da80b2390 msgr2=0x7f6da80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.934+0000 7f6dace4d640 1 -- 172.21.15.19:0/3929491419 shutdown_connections 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.934+0000 7f6dace4d640 1 -- 172.21.15.19:0/3929491419 wait complete. 2024-04-25T09:55:36.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.935+0000 7f6dace4d640 1 Processor -- start 2024-04-25T09:55:36.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.935+0000 7f6dace4d640 1 -- start start 2024-04-25T09:55:36.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6dace4d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6da80cadb0 0x7f6da806f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6dace4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da815de80 0x7f6da80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6dace4d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6da8166a10 0x7f6da80cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6dace4d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da80b2eb0 con 0x7f6da80cadb0 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6dace4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da815e260 con 0x7f6da815de80 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6dace4d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da80b3030 con 0x7f6da8166a10 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6da80cadb0 0x7f6da806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6da80cadb0 0x7f6da806f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48308/0 (socket says 172.21.15.19:48308) 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da659c640 1 -- 172.21.15.19:0/868560002 learned_addr learned my addr 172.21.15.19:0/868560002 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da6d9d640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6da8166a10 0x7f6da80cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da5d9b640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da815de80 0x7f6da80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da659c640 1 -- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6da8166a10 msgr2=0x7f6da80cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da659c640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6da8166a10 0x7f6da80cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da659c640 1 -- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da815de80 msgr2=0x7f6da80c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da659c640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da815de80 0x7f6da80c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da659c640 1 -- 172.21.15.19:0/868560002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da80cb620 con 0x7f6da80cadb0 2024-04-25T09:55:36.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da5d9b640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da815de80 0x7f6da80c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:36.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da6d9d640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6da8166a10 0x7f6da80cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:36.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6da659c640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6da80cadb0 0x7f6da806f970 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6da000c5f0 tx=0x7f6da000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6d96ffd640 1 -- 172.21.15.19:0/868560002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0017070 con 0x7f6da80cadb0 2024-04-25T09:55:36.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6d96ffd640 1 -- 172.21.15.19:0/868560002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da0005d00 con 0x7f6da80cadb0 2024-04-25T09:55:36.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6dace4d640 1 -- 172.21.15.19:0/868560002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da80cb900 con 0x7f6da80cadb0 2024-04-25T09:55:36.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6d96ffd640 1 -- 172.21.15.19:0/868560002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da00072b0 con 0x7f6da80cadb0 2024-04-25T09:55:36.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.936+0000 7f6dace4d640 1 -- 172.21.15.19:0/868560002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da806feb0 con 0x7f6da80cadb0 2024-04-25T09:55:36.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.937+0000 7f6dace4d640 1 -- 172.21.15.19:0/868560002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da80cbf10 con 0x7f6da80cadb0 2024-04-25T09:55:36.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.938+0000 7f6d96ffd640 1 -- 172.21.15.19:0/868560002 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6da001a070 con 0x7f6da80cadb0 2024-04-25T09:55:36.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.938+0000 7f6d96ffd640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6d8806b4d0 0x7f6d8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.938+0000 7f6d96ffd640 1 -- 172.21.15.19:0/868560002 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f6da0089800 con 0x7f6da80cadb0 2024-04-25T09:55:36.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.939+0000 7f6da5d9b640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6d8806b4d0 0x7f6d8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.939+0000 7f6da5d9b640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6d8806b4d0 0x7f6d8806d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6da80c6300 tx=0x7f6d9c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:36.943+0000 7f6d96ffd640 1 -- 172.21.15.19:0/868560002 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6da0058550 con 0x7f6da80cadb0 2024-04-25T09:55:37.081 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:37.083 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.135+0000 7f6dace4d640 1 -- 172.21.15.19:0/868560002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6da8073a00 con 0x7f6da80cadb0 2024-04-25T09:55:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.136+0000 7f6d96ffd640 1 -- 172.21.15.19:0/868560002 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6da0057e50 con 0x7f6da80cadb0 2024-04-25T09:55:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T09:55:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 -- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6d8806b4d0 msgr2=0x7f6d8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6d8806b4d0 0x7f6d8806d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6da80c6300 tx=0x7f6d9c009040 comp rx=0 tx=0).stop 2024-04-25T09:55:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 -- 172.21.15.19:0/868560002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6da80cadb0 msgr2=0x7f6da806f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6da80cadb0 0x7f6da806f970 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6da000c5f0 tx=0x7f6da000cab0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 -- 172.21.15.19:0/868560002 shutdown_connections 2024-04-25T09:55:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6d8806b4d0 0x7f6d8806d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6da8166a10 0x7f6da80cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da815de80 0x7f6da80c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 --2- 172.21.15.19:0/868560002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6da80cadb0 0x7f6da806f970 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 -- 172.21.15.19:0/868560002 >> 172.21.15.19:0/868560002 conn(0x7f6da80b2390 msgr2=0x7f6da80b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 -- 172.21.15.19:0/868560002 shutdown_connections 2024-04-25T09:55:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.138+0000 7f6dace4d640 1 -- 172.21.15.19:0/868560002 wait complete. 2024-04-25T09:55:37.151 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:55:37.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:37.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:37.225 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:55:37.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:37.226 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:55:37.282 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:55:37.282 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:37.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:55:37.300 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:55:37.300 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:37.337 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:55:37.337 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:55:37.337 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:37.357 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:55:37.357 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.475+0000 7f48aee3a640 1 Processor -- start 2024-04-25T09:55:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.476+0000 7f48aee3a640 1 -- start start 2024-04-25T09:55:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.476+0000 7f48aee3a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f48a815dbe0 0x7f48a80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.476+0000 7f48aee3a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48a80b92a0 0x7f48a80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.476+0000 7f48aee3a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a80b97a0 con 0x7f48a80b2cb0 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.476+0000 7f48aee3a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a815dfc0 con 0x7f48a815dbe0 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.476+0000 7f48aee3a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a80d1a00 con 0x7f48a80b92a0 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.476+0000 7f48acbd6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f48a815dbe0 0x7f48a80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.476+0000 7f48acbd6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f48a815dbe0 0x7f48a80d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48312/0 (socket says 172.21.15.19:48312) 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.477+0000 7f48acbd6640 1 -- 172.21.15.19:0/3981789930 learned_addr learned my addr 172.21.15.19:0/3981789930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.477+0000 7f48ad3d7640 1 --2- 172.21.15.19:0/3981789930 >> v2:172.21.15.112:3300/0 conn(0x7f48a80b92a0 0x7f48a80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.477+0000 7f48acbd6640 1 -- 172.21.15.19:0/3981789930 >> v2:172.21.15.112:3300/0 conn(0x7f48a80b92a0 msgr2=0x7f48a80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.477+0000 7f48acbd6640 1 --2- 172.21.15.19:0/3981789930 >> v2:172.21.15.112:3300/0 conn(0x7f48a80b92a0 0x7f48a80d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.477+0000 7f48acbd6640 1 -- 172.21.15.19:0/3981789930 >> v1:172.21.15.19:6789/0 conn(0x7f48a80b2cb0 legacy=0x7f48a80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.477+0000 7f48acbd6640 1 -- 172.21.15.19:0/3981789930 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a80d1c50 con 0x7f48a815dbe0 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.477+0000 7f48ad3d7640 1 --2- 172.21.15.19:0/3981789930 >> v2:172.21.15.112:3300/0 conn(0x7f48a80b92a0 0x7f48a80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.478+0000 7f48acbd6640 1 --2- 172.21.15.19:0/3981789930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48a815dbe0 0x7f48a80d0910 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f48a80cfb90 tx=0x7f48a002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f118caf964850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.478+0000 7f48a77fe640 1 -- 172.21.15.19:0/3981789930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0036070 con 0x7f48a815dbe0 2024-04-25T09:55:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.478+0000 7f48a77fe640 1 -- 172.21.15.19:0/3981789930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a0031600 con 0x7f48a815dbe0 2024-04-25T09:55:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.478+0000 7f48a77fe640 1 -- 172.21.15.19:0/3981789930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a00319d0 con 0x7f48a815dbe0 2024-04-25T09:55:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.478+0000 7f48aee3a640 1 -- 172.21.15.19:0/3981789930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48a815dbe0 msgr2=0x7f48a80d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.478+0000 7f48aee3a640 1 --2- 172.21.15.19:0/3981789930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48a815dbe0 0x7f48a80d0910 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f48a80cfb90 tx=0x7f48a002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.478+0000 7f48aee3a640 1 -- 172.21.15.19:0/3981789930 shutdown_connections 2024-04-25T09:55:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.478+0000 7f48aee3a640 1 --2- 172.21.15.19:0/3981789930 >> v2:172.21.15.112:3300/0 conn(0x7f48a80b92a0 0x7f48a80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.478+0000 7f48aee3a640 1 --2- 172.21.15.19:0/3981789930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48a815dbe0 0x7f48a80d0910 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.478+0000 7f48aee3a640 1 -- 172.21.15.19:0/3981789930 >> 172.21.15.19:0/3981789930 conn(0x7f48a8061a20 msgr2=0x7f48a8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.479+0000 7f48aee3a640 1 -- 172.21.15.19:0/3981789930 shutdown_connections 2024-04-25T09:55:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.479+0000 7f48aee3a640 1 -- 172.21.15.19:0/3981789930 wait complete. 2024-04-25T09:55:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.480+0000 7f48aee3a640 1 Processor -- start 2024-04-25T09:55:37.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.480+0000 7f48aee3a640 1 -- start start 2024-04-25T09:55:37.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.480+0000 7f48aee3a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48a80b2cb0 0x7f48a81f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48aee3a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48a80b92a0 0x7f48a81f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48aee3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a815dbe0 0x7f48a80c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48aee3a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a80b6ab0 con 0x7f48a80b2cb0 2024-04-25T09:55:37.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48aee3a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a80d2210 con 0x7f48a815dbe0 2024-04-25T09:55:37.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48aee3a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a80b6c30 con 0x7f48a80b92a0 2024-04-25T09:55:37.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48acbd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48a80b2cb0 0x7f48a81f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48acbd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48a80b2cb0 0x7f48a81f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48324/0 (socket says 172.21.15.19:48324) 2024-04-25T09:55:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48acbd6640 1 -- 172.21.15.19:0/1415041951 learned_addr learned my addr 172.21.15.19:0/1415041951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48ad3d7640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a815dbe0 0x7f48a80c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48a7fff640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48a80b92a0 0x7f48a81f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48acbd6640 1 -- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48a80b92a0 msgr2=0x7f48a81f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48acbd6640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48a80b92a0 0x7f48a81f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48acbd6640 1 -- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a815dbe0 msgr2=0x7f48a80c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48acbd6640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a815dbe0 0x7f48a80c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48acbd6640 1 -- 172.21.15.19:0/1415041951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a81f7730 con 0x7f48a80b2cb0 2024-04-25T09:55:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48ad3d7640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a815dbe0 0x7f48a80c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48acbd6640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48a80b2cb0 0x7f48a81f44b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f48a0034040 tx=0x7f48a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48a57fa640 1 -- 172.21.15.19:0/1415041951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0036050 con 0x7f48a80b2cb0 2024-04-25T09:55:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48a57fa640 1 -- 172.21.15.19:0/1415041951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a0042070 con 0x7f48a80b2cb0 2024-04-25T09:55:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48a57fa640 1 -- 172.21.15.19:0/1415041951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0009510 con 0x7f48a80b2cb0 2024-04-25T09:55:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48aee3a640 1 -- 172.21.15.19:0/1415041951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a81f7a10 con 0x7f48a80b2cb0 2024-04-25T09:55:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.481+0000 7f48aee3a640 1 -- 172.21.15.19:0/1415041951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a8144650 con 0x7f48a80b2cb0 2024-04-25T09:55:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.482+0000 7f48aee3a640 1 -- 172.21.15.19:0/1415041951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a8073990 con 0x7f48a80b2cb0 2024-04-25T09:55:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.483+0000 7f48a57fa640 1 -- 172.21.15.19:0/1415041951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48a0007c30 con 0x7f48a80b2cb0 2024-04-25T09:55:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.483+0000 7f48a57fa640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f488806b4d0 0x7f488806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.483+0000 7f48a57fa640 1 -- 172.21.15.19:0/1415041951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f48a00ab300 con 0x7f48a80b2cb0 2024-04-25T09:55:37.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.483+0000 7f48a7fff640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f488806b4d0 0x7f488806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.484+0000 7f48a7fff640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f488806b4d0 0x7f488806d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f48a81f49f0 tx=0x7f4898009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.487+0000 7f48a57fa640 1 -- 172.21.15.19:0/1415041951 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48a0045360 con 0x7f48a80b2cb0 2024-04-25T09:55:37.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.705+0000 7f48aee3a640 1 -- 172.21.15.19:0/1415041951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48a80b2610 con 0x7f48a80b2cb0 2024-04-25T09:55:37.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.705+0000 7f48a57fa640 1 -- 172.21.15.19:0/1415041951 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f48a0079fc0 con 0x7f48a80b2cb0 2024-04-25T09:55:37.709 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:37.709 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3845932448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3845932448},{"type":"v1","addr":"172.21.15.112:6836","nonce":3845932448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5122,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1655535300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1655535300},{"type":"v1","addr":"172.21.15.19:6834","nonce":1655535300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/165157787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":165157787},{"type":"v1","addr":"172.21.15.112:6837","nonce":165157787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:54:12.554449+0000","modified":"2024-04-25T09:54:19.314742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3017773710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3017773710},{"type":"v1","addr":"172.21.15.19:6835","nonce":3017773710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:55:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-04-25T09:55:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 -- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f488806b4d0 msgr2=0x7f488806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f488806b4d0 0x7f488806d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f48a81f49f0 tx=0x7f4898009040 comp rx=0 tx=0).stop 2024-04-25T09:55:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 -- 172.21.15.19:0/1415041951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48a80b2cb0 msgr2=0x7f48a81f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48a80b2cb0 0x7f48a81f44b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f48a0034040 tx=0x7f48a0004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 -- 172.21.15.19:0/1415041951 shutdown_connections 2024-04-25T09:55:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f488806b4d0 0x7f488806d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48a815dbe0 0x7f48a80c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48a80b92a0 0x7f48a81f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 --2- 172.21.15.19:0/1415041951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48a80b2cb0 0x7f48a81f44b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 -- 172.21.15.19:0/1415041951 >> 172.21.15.19:0/1415041951 conn(0x7f48a8061a20 msgr2=0x7f48a815b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 -- 172.21.15.19:0/1415041951 shutdown_connections 2024-04-25T09:55:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.708+0000 7f48aee3a640 1 -- 172.21.15.19:0/1415041951 wait complete. 2024-04-25T09:55:37.722 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.820+0000 7f227d2e1640 1 Processor -- start 2024-04-25T09:55:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.820+0000 7f227d2e1640 1 -- start start 2024-04-25T09:55:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f227d2e1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22780b2cb0 0x7f22780d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f227d2e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780d1020 0x7f22780d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f227d2e1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22780b97a0 con 0x7f227815dbe0 2024-04-25T09:55:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f227d2e1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227815dfc0 con 0x7f22780b2cb0 2024-04-25T09:55:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f227d2e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22780d1a00 con 0x7f22780d1020 2024-04-25T09:55:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f22767fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22780b2cb0 0x7f22780d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f22767fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22780b2cb0 0x7f22780d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39312/0 (socket says 172.21.15.19:39312) 2024-04-25T09:55:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f22767fc640 1 -- 172.21.15.19:0/3423755598 learned_addr learned my addr 172.21.15.19:0/3423755598 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f22777fe640 1 --2- 172.21.15.19:0/3423755598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780d1020 0x7f22780d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f22767fc640 1 -- 172.21.15.19:0/3423755598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780d1020 msgr2=0x7f22780d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f22767fc640 1 --2- 172.21.15.19:0/3423755598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780d1020 0x7f22780d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f22767fc640 1 -- 172.21.15.19:0/3423755598 >> v1:172.21.15.19:6789/0 conn(0x7f227815dbe0 legacy=0x7f22780d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.821+0000 7f22767fc640 1 -- 172.21.15.19:0/3423755598 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22780d1c50 con 0x7f22780b2cb0 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.822+0000 7f22767fc640 1 --2- 172.21.15.19:0/3423755598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22780b2cb0 0x7f22780d2730 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f226c009870 tx=0x7f226c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147cdc35d0c7a047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.822+0000 7f2275ffb640 1 -- 172.21.15.19:0/3423755598 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c037070 con 0x7f22780b2cb0 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.822+0000 7f2275ffb640 1 -- 172.21.15.19:0/3423755598 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c032440 con 0x7f22780b2cb0 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.822+0000 7f2275ffb640 1 -- 172.21.15.19:0/3423755598 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c032810 con 0x7f22780b2cb0 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.823+0000 7f227d2e1640 1 -- 172.21.15.19:0/3423755598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22780b2cb0 msgr2=0x7f22780d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.823+0000 7f227d2e1640 1 --2- 172.21.15.19:0/3423755598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22780b2cb0 0x7f22780d2730 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f226c009870 tx=0x7f226c02a270 comp rx=0 tx=0).stop 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.823+0000 7f227d2e1640 1 -- 172.21.15.19:0/3423755598 shutdown_connections 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.823+0000 7f227d2e1640 1 --2- 172.21.15.19:0/3423755598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780d1020 0x7f22780d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.823+0000 7f227d2e1640 1 --2- 172.21.15.19:0/3423755598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22780b2cb0 0x7f22780d2730 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.823+0000 7f227d2e1640 1 -- 172.21.15.19:0/3423755598 >> 172.21.15.19:0/3423755598 conn(0x7f2278061a20 msgr2=0x7f2278061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.823+0000 7f227d2e1640 1 -- 172.21.15.19:0/3423755598 shutdown_connections 2024-04-25T09:55:37.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.823+0000 7f227d2e1640 1 -- 172.21.15.19:0/3423755598 wait complete. 2024-04-25T09:55:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.824+0000 7f227d2e1640 1 Processor -- start 2024-04-25T09:55:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.824+0000 7f227d2e1640 1 -- start start 2024-04-25T09:55:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f227d2e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22780b2cb0 0x7f22781f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f227d2e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780d1020 0x7f227815e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f227d2e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815dbe0 0x7f22780b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f227d2e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22780b92a0 con 0x7f227815dbe0 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f227d2e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22780d2210 con 0x7f22780b2cb0 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f227d2e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22780b9420 con 0x7f22780d1020 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f22777fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815dbe0 0x7f22780b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f22777fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815dbe0 0x7f22780b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39324/0 (socket says 172.21.15.19:39324) 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f22777fe640 1 -- 172.21.15.19:0/499564860 learned_addr learned my addr 172.21.15.19:0/499564860 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f22767fc640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780d1020 0x7f227815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f2276ffd640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22780b2cb0 0x7f22781f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f22777fe640 1 -- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780d1020 msgr2=0x7f227815e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f22777fe640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780d1020 0x7f227815e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f22777fe640 1 -- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22780b2cb0 msgr2=0x7f22781f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f22777fe640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22780b2cb0 0x7f22781f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f22777fe640 1 -- 172.21.15.19:0/499564860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227815ec10 con 0x7f227815dbe0 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.825+0000 7f22777fe640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815dbe0 0x7f22780b5150 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f227000d5d0 tx=0x7f227000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.826+0000 7f225f7fe640 1 -- 172.21.15.19:0/499564860 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22700125f0 con 0x7f227815dbe0 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.826+0000 7f225f7fe640 1 -- 172.21.15.19:0/499564860 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2270019070 con 0x7f227815dbe0 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.826+0000 7f227d2e1640 1 -- 172.21.15.19:0/499564860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227815eef0 con 0x7f227815dbe0 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.826+0000 7f225f7fe640 1 -- 172.21.15.19:0/499564860 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22700129d0 con 0x7f227815dbe0 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.826+0000 7f227d2e1640 1 -- 172.21.15.19:0/499564860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227815f4b0 con 0x7f227815dbe0 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.827+0000 7f227d2e1640 1 -- 172.21.15.19:0/499564860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2278073990 con 0x7f227815dbe0 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.827+0000 7f225f7fe640 1 -- 172.21.15.19:0/499564860 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2270012b70 con 0x7f227815dbe0 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.827+0000 7f225f7fe640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f226406b4d0 0x7f226406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.827+0000 7f225f7fe640 1 -- 172.21.15.19:0/499564860 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f227008a650 con 0x7f227815dbe0 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.828+0000 7f2276ffd640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f226406b4d0 0x7f226406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.828+0000 7f2276ffd640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f226406b4d0 0x7f226406d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f22780b4b60 tx=0x7f2268009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:37.832+0000 7f225f7fe640 1 -- 172.21.15.19:0/499564860 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2270058d50 con 0x7f227815dbe0 2024-04-25T09:55:38.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.011+0000 7f227d2e1640 1 -- 172.21.15.19:0/499564860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22780d4790 con 0x7f227815dbe0 2024-04-25T09:55:38.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.012+0000 7f225f7fe640 1 -- 172.21.15.19:0/499564860 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f227005cdf0 con 0x7f227815dbe0 2024-04-25T09:55:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:38.015 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:12.545580+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:11.132892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:11.536478+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.014+0000 7f227d2e1640 1 -- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f226406b4d0 msgr2=0x7f226406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.015+0000 7f227d2e1640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f226406b4d0 0x7f226406d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f22780b4b60 tx=0x7f2268009040 comp rx=0 tx=0).stop 2024-04-25T09:55:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.015+0000 7f227d2e1640 1 -- 172.21.15.19:0/499564860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815dbe0 msgr2=0x7f22780b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.015+0000 7f227d2e1640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815dbe0 0x7f22780b5150 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f227000d5d0 tx=0x7f227000da90 comp rx=0 tx=0).stop 2024-04-25T09:55:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.015+0000 7f227d2e1640 1 -- 172.21.15.19:0/499564860 shutdown_connections 2024-04-25T09:55:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.015+0000 7f227d2e1640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f226406b4d0 0x7f226406d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.015+0000 7f227d2e1640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815dbe0 0x7f22780b5150 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.015+0000 7f227d2e1640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780d1020 0x7f227815e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.015+0000 7f227d2e1640 1 --2- 172.21.15.19:0/499564860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22780b2cb0 0x7f22781f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.015+0000 7f227d2e1640 1 -- 172.21.15.19:0/499564860 >> 172.21.15.19:0/499564860 conn(0x7f2278061a20 msgr2=0x7f22780d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.015+0000 7f227d2e1640 1 -- 172.21.15.19:0/499564860 shutdown_connections 2024-04-25T09:55:38.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.015+0000 7f227d2e1640 1 -- 172.21.15.19:0/499564860 wait complete. 2024-04-25T09:55:38.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:11.132892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:11.536478+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 41, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:55:38.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:38.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.129+0000 7f0a70e76640 1 Processor -- start 2024-04-25T09:55:38.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.129+0000 7f0a70e76640 1 -- start start 2024-04-25T09:55:38.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a70e76640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b5c40 0x7f0a6c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a70e76640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c0cf320 con 0x7f0a6c15b5a0 2024-04-25T09:55:38.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a70e76640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c0cf040 con 0x7f0a6c15b980 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a70e76640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c161970 con 0x7f0a6c0b5c40 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a6a59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0a6c15b5a0 0x7f0a6c0b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42468/0 (socket says 172.21.15.19:42468) 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a6a59c640 1 -- 172.21.15.19:0/951199819 learned_addr learned my addr 172.21.15.19:0/951199819 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a63fff640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b5c40 0x7f0a6c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984924418 0 0) 0x7f0a5c001110 con 0x7f0a6c15b5a0 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a58003520 con 0x7f0a6c15b5a0 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1731946162 0 0) 0x7f0a58003520 con 0x7f0a6c15b5a0 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b5c40 msgr2=0x7f0a6c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a69d9b640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b5c40 0x7f0a6c0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 >> v1:172.21.15.112:6789/0 conn(0x7f0a6c15b980 legacy=0x7f0a6c0ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c161c50 con 0x7f0a6c15b5a0 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0a5c001840 con 0x7f0a6c15b5a0 2024-04-25T09:55:38.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 >> v1:172.21.15.19:6789/0 conn(0x7f0a6c15b5a0 legacy=0x7f0a6c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.130+0000 7f0a69d9b640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b5c40 0x7f0a580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a69d9b640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 0x7f0a58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a69d9b640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a58004d40 0x7f0a580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c161970 con 0x7f0a58003f60 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5800baa0 con 0x7f0a58004d40 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5800bca0 con 0x7f0a6c0b5c40 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a63fff640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 0x7f0a58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a6a59c640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b5c40 0x7f0a580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a6ad9d640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a58004d40 0x7f0a580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a63fff640 1 -- 172.21.15.19:0/951199819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b5c40 msgr2=0x7f0a580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a63fff640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b5c40 0x7f0a580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a63fff640 1 -- 172.21.15.19:0/951199819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a58004d40 msgr2=0x7f0a580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a63fff640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a58004d40 0x7f0a580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.131+0000 7f0a63fff640 1 -- 172.21.15.19:0/951199819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a58003520 con 0x7f0a58003f60 2024-04-25T09:55:38.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.132+0000 7f0a63fff640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 0x7f0a58004800 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009ab0 tx=0x7f0a58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c9ab98a91bf2ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.132+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54036070 con 0x7f0a58003f60 2024-04-25T09:55:38.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.132+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a54005990 con 0x7f0a58003f60 2024-04-25T09:55:38.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.132+0000 7f0a69d9b640 1 -- 172.21.15.19:0/951199819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54009680 con 0x7f0a58003f60 2024-04-25T09:55:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.132+0000 7f0a70e76640 1 -- 172.21.15.19:0/951199819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 msgr2=0x7f0a58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.132+0000 7f0a70e76640 1 --2- 172.21.15.19:0/951199819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 0x7f0a58004800 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009ab0 tx=0x7f0a58005830 comp rx=0 tx=0).stop 2024-04-25T09:55:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.132+0000 7f0a6ad9d640 1 -- 172.21.15.19:0/951199819 reap_dead start 2024-04-25T09:55:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.132+0000 7f0a70e76640 1 -- 172.21.15.19:0/951199819 shutdown_connections 2024-04-25T09:55:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.132+0000 7f0a70e76640 1 -- 172.21.15.19:0/951199819 >> 172.21.15.19:0/951199819 conn(0x7f0a6c0d0530 msgr2=0x7f0a6c0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.132+0000 7f0a70e76640 1 -- 172.21.15.19:0/951199819 shutdown_connections 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.132+0000 7f0a70e76640 1 -- 172.21.15.19:0/951199819 wait complete. 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.134+0000 7f0a70e76640 1 Processor -- start 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.134+0000 7f0a70e76640 1 -- start start 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.134+0000 7f0a70e76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 0x7f0a6c15e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.134+0000 7f0a70e76640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b8d60 0x7f0a6c153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.134+0000 7f0a70e76640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a6c1561d0 0x7f0a6c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.134+0000 7f0a70e76640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1625d0 con 0x7f0a58003f60 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.134+0000 7f0a70e76640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c162450 con 0x7f0a6c1561d0 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.134+0000 7f0a70e76640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c162750 con 0x7f0a6c0b8d60 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 0x7f0a6c15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 0x7f0a6c15e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39340/0 (socket says 172.21.15.19:39340) 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a6a59c640 1 -- 172.21.15.19:0/324912281 learned_addr learned my addr 172.21.15.19:0/324912281 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a63fff640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b8d60 0x7f0a6c153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a6ad9d640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a6c1561d0 0x7f0a6c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a6a59c640 1 -- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b8d60 msgr2=0x7f0a6c153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a6a59c640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b8d60 0x7f0a6c153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a6a59c640 1 -- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a6c1561d0 msgr2=0x7f0a6c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a6a59c640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a6c1561d0 0x7f0a6c156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a6a59c640 1 -- 172.21.15.19:0/324912281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c15f560 con 0x7f0a58003f60 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a6ad9d640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a6c1561d0 0x7f0a6c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a6a59c640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 0x7f0a6c15e250 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c007290 tx=0x7f0a5c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a62ffd640 1 -- 172.21.15.19:0/324912281 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c001180 con 0x7f0a58003f60 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a62ffd640 1 -- 172.21.15.19:0/324912281 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5c018070 con 0x7f0a58003f60 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a70e76640 1 -- 172.21.15.19:0/324912281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c15e950 con 0x7f0a58003f60 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a62ffd640 1 -- 172.21.15.19:0/324912281 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c00fba0 con 0x7f0a58003f60 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.135+0000 7f0a70e76640 1 -- 172.21.15.19:0/324912281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c15ef10 con 0x7f0a58003f60 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.136+0000 7f0a70e76640 1 -- 172.21.15.19:0/324912281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c0d1260 con 0x7f0a58003f60 2024-04-25T09:55:38.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.137+0000 7f0a62ffd640 1 -- 172.21.15.19:0/324912281 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a5c00fd40 con 0x7f0a58003f60 2024-04-25T09:55:38.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.137+0000 7f0a62ffd640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b500 0x7f0a4c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.137+0000 7f0a62ffd640 1 -- 172.21.15.19:0/324912281 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f0a5c08a330 con 0x7f0a58003f60 2024-04-25T09:55:38.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.137+0000 7f0a63fff640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b500 0x7f0a4c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.138+0000 7f0a63fff640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b500 0x7f0a4c06d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0a54033000 tx=0x7f0a54008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.141+0000 7f0a62ffd640 1 -- 172.21.15.19:0/324912281 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a6c0d1260 con 0x7f0a58003f60 2024-04-25T09:55:38.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.320+0000 7f0a70e76640 1 -- 172.21.15.19:0/324912281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a6c073a00 con 0x7f0a58003f60 2024-04-25T09:55:38.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.321+0000 7f0a62ffd640 1 -- 172.21.15.19:0/324912281 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a6c0cf320 con 0x7f0a58003f60 2024-04-25T09:55:38.324 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:55:38.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.322+0000 7f0a70e76640 1 -- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b500 msgr2=0x7f0a4c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.322+0000 7f0a70e76640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b500 0x7f0a4c06d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0a54033000 tx=0x7f0a54008030 comp rx=0 tx=0).stop 2024-04-25T09:55:38.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.322+0000 7f0a70e76640 1 -- 172.21.15.19:0/324912281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 msgr2=0x7f0a6c15e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.322+0000 7f0a70e76640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 0x7f0a6c15e250 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c007290 tx=0x7f0a5c007750 comp rx=0 tx=0).stop 2024-04-25T09:55:38.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.322+0000 7f0a70e76640 1 -- 172.21.15.19:0/324912281 shutdown_connections 2024-04-25T09:55:38.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.322+0000 7f0a70e76640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a4c06b500 0x7f0a4c06d9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.322+0000 7f0a70e76640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a6c1561d0 0x7f0a6c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.322+0000 7f0a70e76640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a6c0b8d60 0x7f0a6c153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.322+0000 7f0a70e76640 1 --2- 172.21.15.19:0/324912281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a58003f60 0x7f0a6c15e250 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.322+0000 7f0a70e76640 1 -- 172.21.15.19:0/324912281 >> 172.21.15.19:0/324912281 conn(0x7f0a6c0d0530 msgr2=0x7f0a6c0d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.323+0000 7f0a70e76640 1 -- 172.21.15.19:0/324912281 shutdown_connections 2024-04-25T09:55:38.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.323+0000 7f0a70e76640 1 -- 172.21.15.19:0/324912281 wait complete. 2024-04-25T09:55:38.336 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:38.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.436+0000 7efe76da9640 1 Processor -- start 2024-04-25T09:55:38.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.437+0000 7efe76da9640 1 -- start start 2024-04-25T09:55:38.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.437+0000 7efe76da9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe7015bb20 0x7efe700cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.437+0000 7efe76da9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efe700ccdd0 0x7efe700cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.437+0000 7efe76da9640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe700cb190 con 0x7efe700cdcd0 2024-04-25T09:55:38.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.437+0000 7efe76da9640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe700cb470 con 0x7efe7015bb20 2024-04-25T09:55:38.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.437+0000 7efe76da9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe700cb750 con 0x7efe700ccdd0 2024-04-25T09:55:38.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.437+0000 7efe74b45640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe7015bb20 0x7efe700cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.438+0000 7efe74b45640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe7015bb20 0x7efe700cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39350/0 (socket says 172.21.15.19:39350) 2024-04-25T09:55:38.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.438+0000 7efe74b45640 1 -- 172.21.15.19:0/2897891129 learned_addr learned my addr 172.21.15.19:0/2897891129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:38.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.438+0000 7efe6ffff640 1 --2- 172.21.15.19:0/2897891129 >> v2:172.21.15.112:3300/0 conn(0x7efe700ccdd0 0x7efe700cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.438+0000 7efe74b45640 1 -- 172.21.15.19:0/2897891129 >> v2:172.21.15.112:3300/0 conn(0x7efe700ccdd0 msgr2=0x7efe700cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.438+0000 7efe74b45640 1 --2- 172.21.15.19:0/2897891129 >> v2:172.21.15.112:3300/0 conn(0x7efe700ccdd0 0x7efe700cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.438+0000 7efe74b45640 1 -- 172.21.15.19:0/2897891129 >> v1:172.21.15.19:6789/0 conn(0x7efe700cdcd0 legacy=0x7efe700ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.438+0000 7efe74b45640 1 -- 172.21.15.19:0/2897891129 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe700cba30 con 0x7efe7015bb20 2024-04-25T09:55:38.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe74b45640 1 --2- 172.21.15.19:0/2897891129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe7015bb20 0x7efe700cc230 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efe70071680 tx=0x7efe6802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1401c1f8af00934 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe6f7fe640 1 -- 172.21.15.19:0/2897891129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68037070 con 0x7efe7015bb20 2024-04-25T09:55:38.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe6f7fe640 1 -- 172.21.15.19:0/2897891129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe68032440 con 0x7efe7015bb20 2024-04-25T09:55:38.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe6f7fe640 1 -- 172.21.15.19:0/2897891129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68032810 con 0x7efe7015bb20 2024-04-25T09:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe76da9640 1 -- 172.21.15.19:0/2897891129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe7015bb20 msgr2=0x7efe700cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe76da9640 1 --2- 172.21.15.19:0/2897891129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe7015bb20 0x7efe700cc230 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efe70071680 tx=0x7efe6802a200 comp rx=0 tx=0).stop 2024-04-25T09:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe76da9640 1 -- 172.21.15.19:0/2897891129 shutdown_connections 2024-04-25T09:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe76da9640 1 --2- 172.21.15.19:0/2897891129 >> v2:172.21.15.112:3300/0 conn(0x7efe700ccdd0 0x7efe700cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe76da9640 1 --2- 172.21.15.19:0/2897891129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe7015bb20 0x7efe700cc230 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe76da9640 1 -- 172.21.15.19:0/2897891129 >> 172.21.15.19:0/2897891129 conn(0x7efe700b2390 msgr2=0x7efe700b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe76da9640 1 -- 172.21.15.19:0/2897891129 shutdown_connections 2024-04-25T09:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.439+0000 7efe76da9640 1 -- 172.21.15.19:0/2897891129 wait complete. 2024-04-25T09:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.440+0000 7efe76da9640 1 Processor -- start 2024-04-25T09:55:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.440+0000 7efe76da9640 1 -- start start 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.441+0000 7efe76da9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe700ccdd0 0x7efe7014aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.441+0000 7efe76da9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe700cdcd0 0x7efe70151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.441+0000 7efe76da9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7015bb20 0x7efe70152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.441+0000 7efe76da9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7015bf00 con 0x7efe700ccdd0 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.441+0000 7efe76da9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe700cc770 con 0x7efe7015bb20 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.441+0000 7efe76da9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7015c080 con 0x7efe700cdcd0 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.441+0000 7efe74b45640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe700ccdd0 0x7efe7014aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.441+0000 7efe74b45640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe700ccdd0 0x7efe7014aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39360/0 (socket says 172.21.15.19:39360) 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.441+0000 7efe74b45640 1 -- 172.21.15.19:0/1889986972 learned_addr learned my addr 172.21.15.19:0/1889986972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe6ffff640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe700cdcd0 0x7efe70151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe75346640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7015bb20 0x7efe70152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe74b45640 1 -- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe700cdcd0 msgr2=0x7efe70151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe74b45640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe700cdcd0 0x7efe70151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe74b45640 1 -- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7015bb20 msgr2=0x7efe70152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe74b45640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7015bb20 0x7efe70152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe74b45640 1 -- 172.21.15.19:0/1889986972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe700b3620 con 0x7efe700ccdd0 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe75346640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7015bb20 0x7efe70152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe74b45640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe700ccdd0 0x7efe7014aff0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efe6802a240 tx=0x7efe680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe6d7fa640 1 -- 172.21.15.19:0/1889986972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68041020 con 0x7efe700ccdd0 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe6d7fa640 1 -- 172.21.15.19:0/1889986972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe680306c0 con 0x7efe700ccdd0 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe6d7fa640 1 -- 172.21.15.19:0/1889986972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68037040 con 0x7efe700ccdd0 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe76da9640 1 -- 172.21.15.19:0/1889986972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe700b3900 con 0x7efe700ccdd0 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.442+0000 7efe76da9640 1 -- 172.21.15.19:0/1889986972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe700b3ec0 con 0x7efe700ccdd0 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.443+0000 7efe76da9640 1 -- 172.21.15.19:0/1889986972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe70073a00 con 0x7efe700ccdd0 2024-04-25T09:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.444+0000 7efe6d7fa640 1 -- 172.21.15.19:0/1889986972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe68044020 con 0x7efe700ccdd0 2024-04-25T09:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.444+0000 7efe6d7fa640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe5006b4d0 0x7efe5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.444+0000 7efe6d7fa640 1 -- 172.21.15.19:0/1889986972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7efe680aabf0 con 0x7efe700ccdd0 2024-04-25T09:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.444+0000 7efe6ffff640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe5006b4d0 0x7efe5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.445+0000 7efe6ffff640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe5006b4d0 0x7efe5006d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efe700b1180 tx=0x7efe600098d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.449+0000 7efe6d7fa640 1 -- 172.21.15.19:0/1889986972 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe6804b030 con 0x7efe700ccdd0 2024-04-25T09:55:38.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.628+0000 7efe76da9640 1 -- 172.21.15.19:0/1889986972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe700d0290 con 0x7efe700ccdd0 2024-04-25T09:55:38.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.628+0000 7efe6d7fa640 1 -- 172.21.15.19:0/1889986972 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe68079820 con 0x7efe700ccdd0 2024-04-25T09:55:38.631 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:55:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 -- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe5006b4d0 msgr2=0x7efe5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe5006b4d0 0x7efe5006d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efe700b1180 tx=0x7efe600098d0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 -- 172.21.15.19:0/1889986972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe700ccdd0 msgr2=0x7efe7014aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe700ccdd0 0x7efe7014aff0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efe6802a240 tx=0x7efe680046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 -- 172.21.15.19:0/1889986972 shutdown_connections 2024-04-25T09:55:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe5006b4d0 0x7efe5006d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7015bb20 0x7efe70152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe700cdcd0 0x7efe70151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 --2- 172.21.15.19:0/1889986972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe700ccdd0 0x7efe7014aff0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 -- 172.21.15.19:0/1889986972 >> 172.21.15.19:0/1889986972 conn(0x7efe700b2390 msgr2=0x7efe7006e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 -- 172.21.15.19:0/1889986972 shutdown_connections 2024-04-25T09:55:38.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.630+0000 7efe76da9640 1 -- 172.21.15.19:0/1889986972 wait complete. 2024-04-25T09:55:38.644 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:38.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.745+0000 7f8a19c76640 1 Processor -- start 2024-04-25T09:55:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.745+0000 7f8a19c76640 1 -- start start 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.745+0000 7f8a19c76640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8a140d34c0 0x7f8a140d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.745+0000 7f8a19c76640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8a140d08e0 0x7f8a140b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.745+0000 7f8a19c76640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a140d2420 con 0x7f8a1415dac0 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.745+0000 7f8a19c76640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a140d0d50 con 0x7f8a140d34c0 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.745+0000 7f8a19c76640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a140d3030 con 0x7f8a140d08e0 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.746+0000 7f8a12ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8a140d34c0 0x7f8a140d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.746+0000 7f8a12ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8a140d34c0 0x7f8a140d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39374/0 (socket says 172.21.15.19:39374) 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.746+0000 7f8a12ffd640 1 -- 172.21.15.19:0/218585874 learned_addr learned my addr 172.21.15.19:0/218585874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.746+0000 7f8a12ffd640 1 -- 172.21.15.19:0/218585874 >> v2:172.21.15.112:3300/0 conn(0x7f8a140d08e0 msgr2=0x7f8a140b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.746+0000 7f8a12ffd640 1 --2- 172.21.15.19:0/218585874 >> v2:172.21.15.112:3300/0 conn(0x7f8a140d08e0 0x7f8a140b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.746+0000 7f8a12ffd640 1 -- 172.21.15.19:0/218585874 >> v1:172.21.15.112:6789/0 conn(0x7f8a1415dac0 legacy=0x7f8a140d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.746+0000 7f8a12ffd640 1 -- 172.21.15.19:0/218585874 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140d2700 con 0x7f8a140d34c0 2024-04-25T09:55:38.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a12ffd640 1 --2- 172.21.15.19:0/218585874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a140d34c0 0x7f8a140d03a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8a140d2fa0 tx=0x7f8a0802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b99bdf0c6d9834 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a127fc640 1 -- 172.21.15.19:0/218585874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08004030 con 0x7f8a140d34c0 2024-04-25T09:55:38.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a127fc640 1 -- 172.21.15.19:0/218585874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a080041d0 con 0x7f8a140d34c0 2024-04-25T09:55:38.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a127fc640 1 -- 172.21.15.19:0/218585874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08003610 con 0x7f8a140d34c0 2024-04-25T09:55:38.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a19c76640 1 -- 172.21.15.19:0/218585874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a140d34c0 msgr2=0x7f8a140d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a19c76640 1 --2- 172.21.15.19:0/218585874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a140d34c0 0x7f8a140d03a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8a140d2fa0 tx=0x7f8a0802a290 comp rx=0 tx=0).stop 2024-04-25T09:55:38.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a19c76640 1 -- 172.21.15.19:0/218585874 shutdown_connections 2024-04-25T09:55:38.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a19c76640 1 --2- 172.21.15.19:0/218585874 >> v2:172.21.15.112:3300/0 conn(0x7f8a140d08e0 0x7f8a140b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a19c76640 1 --2- 172.21.15.19:0/218585874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a140d34c0 0x7f8a140d03a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a19c76640 1 -- 172.21.15.19:0/218585874 >> 172.21.15.19:0/218585874 conn(0x7f8a14061c30 msgr2=0x7f8a1406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a19c76640 1 -- 172.21.15.19:0/218585874 shutdown_connections 2024-04-25T09:55:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.747+0000 7f8a19c76640 1 -- 172.21.15.19:0/218585874 wait complete. 2024-04-25T09:55:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.748+0000 7f8a19c76640 1 Processor -- start 2024-04-25T09:55:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.749+0000 7f8a19c76640 1 -- start start 2024-04-25T09:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.749+0000 7f8a19c76640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a140d08e0 0x7f8a140cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.749+0000 7f8a19c76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a140d34c0 0x7f8a14159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.749+0000 7f8a19c76640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8a1415dac0 0x7f8a141f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.749+0000 7f8a19c76640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140d2100 con 0x7f8a140d34c0 2024-04-25T09:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.749+0000 7f8a19c76640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140d2cc0 con 0x7f8a140d08e0 2024-04-25T09:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.749+0000 7f8a19c76640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1415dea0 con 0x7f8a1415dac0 2024-04-25T09:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.749+0000 7f8a12ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a140d34c0 0x7f8a14159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.749+0000 7f8a12ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a140d34c0 0x7f8a14159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39382/0 (socket says 172.21.15.19:39382) 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a12ffd640 1 -- 172.21.15.19:0/2137707483 learned_addr learned my addr 172.21.15.19:0/2137707483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a13fff640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8a1415dac0 0x7f8a141f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a137fe640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a140d08e0 0x7f8a140cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a12ffd640 1 -- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8a1415dac0 msgr2=0x7f8a141f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a12ffd640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8a1415dac0 0x7f8a141f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a12ffd640 1 -- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a140d08e0 msgr2=0x7f8a140cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a12ffd640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a140d08e0 0x7f8a140cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a12ffd640 1 -- 172.21.15.19:0/2137707483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a141f6c50 con 0x7f8a140d34c0 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a13fff640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8a1415dac0 0x7f8a141f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a137fe640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a140d08e0 0x7f8a140cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a12ffd640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a140d34c0 0x7f8a14159830 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8a08034040 tx=0x7f8a080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f89fbfff640 1 -- 172.21.15.19:0/2137707483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08041020 con 0x7f8a140d34c0 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f89fbfff640 1 -- 172.21.15.19:0/2137707483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a08003c50 con 0x7f8a140d34c0 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a19c76640 1 -- 172.21.15.19:0/2137707483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a140cac40 con 0x7f8a140d34c0 2024-04-25T09:55:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f89fbfff640 1 -- 172.21.15.19:0/2137707483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08004030 con 0x7f8a140d34c0 2024-04-25T09:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.750+0000 7f8a19c76640 1 -- 172.21.15.19:0/2137707483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1415c810 con 0x7f8a140d34c0 2024-04-25T09:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.751+0000 7f8a19c76640 1 -- 172.21.15.19:0/2137707483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a14073a00 con 0x7f8a140d34c0 2024-04-25T09:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.752+0000 7f89fbfff640 1 -- 172.21.15.19:0/2137707483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a08044020 con 0x7f8a140d34c0 2024-04-25T09:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.752+0000 7f89fbfff640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8a0006b500 0x7f8a0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.752+0000 7f89fbfff640 1 -- 172.21.15.19:0/2137707483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f8a080ab100 con 0x7f8a140d34c0 2024-04-25T09:55:38.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.752+0000 7f8a137fe640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8a0006b500 0x7f8a0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.753+0000 7f8a137fe640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8a0006b500 0x7f8a0006d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006fd0 tx=0x7f8a04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.756+0000 7f89fbfff640 1 -- 172.21.15.19:0/2137707483 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a0804b030 con 0x7f8a140d34c0 2024-04-25T09:55:38.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.936+0000 7f8a19c76640 1 -- 172.21.15.19:0/2137707483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a1415b630 con 0x7f8a140d34c0 2024-04-25T09:55:38.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.937+0000 7f89fbfff640 1 -- 172.21.15.19:0/2137707483 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a08079dc0 con 0x7f8a140d34c0 2024-04-25T09:55:38.940 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:55:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.938+0000 7f8a19c76640 1 -- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8a0006b500 msgr2=0x7f8a0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.938+0000 7f8a19c76640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8a0006b500 0x7f8a0006d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006fd0 tx=0x7f8a04008040 comp rx=0 tx=0).stop 2024-04-25T09:55:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.938+0000 7f8a19c76640 1 -- 172.21.15.19:0/2137707483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a140d34c0 msgr2=0x7f8a14159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.938+0000 7f8a19c76640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a140d34c0 0x7f8a14159830 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8a08034040 tx=0x7f8a080046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.938+0000 7f8a19c76640 1 -- 172.21.15.19:0/2137707483 shutdown_connections 2024-04-25T09:55:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.938+0000 7f8a19c76640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8a0006b500 0x7f8a0006d9c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.938+0000 7f8a19c76640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8a1415dac0 0x7f8a141f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.938+0000 7f8a19c76640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a140d34c0 0x7f8a14159830 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.938+0000 7f8a19c76640 1 --2- 172.21.15.19:0/2137707483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a140d08e0 0x7f8a140cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.939+0000 7f8a19c76640 1 -- 172.21.15.19:0/2137707483 >> 172.21.15.19:0/2137707483 conn(0x7f8a14061c30 msgr2=0x7f8a140b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.939+0000 7f8a19c76640 1 -- 172.21.15.19:0/2137707483 shutdown_connections 2024-04-25T09:55:38.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:38.939+0000 7f8a19c76640 1 -- 172.21.15.19:0/2137707483 wait complete. 2024-04-25T09:55:38.952 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.050+0000 7fb10eb13640 1 Processor -- start 2024-04-25T09:55:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.050+0000 7fb10eb13640 1 -- start start 2024-04-25T09:55:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.051+0000 7fb10eb13640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb10815dc90 0x7fb1081625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.051+0000 7fb10eb13640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb1080d1810 0x7fb1080d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.051+0000 7fb10eb13640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1080cce50 con 0x7fb1080cc8e0 2024-04-25T09:55:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.051+0000 7fb10eb13640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1080d21c0 con 0x7fb1080d1810 2024-04-25T09:55:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.051+0000 7fb10eb13640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb108161af0 con 0x7fb10815dc90 2024-04-25T09:55:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.051+0000 7fb10d0b0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb1080d1810 0x7fb1080d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.051+0000 7fb10d0b0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb1080d1810 0x7fb1080d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39398/0 (socket says 172.21.15.19:39398) 2024-04-25T09:55:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.051+0000 7fb10d0b0640 1 -- 172.21.15.19:0/2566083090 learned_addr learned my addr 172.21.15.19:0/2566083090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.052+0000 7fb10d0b0640 1 -- 172.21.15.19:0/2566083090 >> v2:172.21.15.112:3300/0 conn(0x7fb10815dc90 msgr2=0x7fb1081625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.052+0000 7fb10d0b0640 1 --2- 172.21.15.19:0/2566083090 >> v2:172.21.15.112:3300/0 conn(0x7fb10815dc90 0x7fb1081625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.052+0000 7fb10d0b0640 1 -- 172.21.15.19:0/2566083090 >> v1:172.21.15.112:6789/0 conn(0x7fb1080cc8e0 legacy=0x7fb10815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.052+0000 7fb10d0b0640 1 -- 172.21.15.19:0/2566083090 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb108161dd0 con 0x7fb1080d1810 2024-04-25T09:55:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.052+0000 7fb10d0b0640 1 --2- 172.21.15.19:0/2566083090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1080d1810 0x7fb1080d1c80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb10806b820 tx=0x7fb10002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcc016b95b51cad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.052+0000 7fb1077fe640 1 -- 172.21.15.19:0/2566083090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100036070 con 0x7fb1080d1810 2024-04-25T09:55:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.052+0000 7fb1077fe640 1 -- 172.21.15.19:0/2566083090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb100031600 con 0x7fb1080d1810 2024-04-25T09:55:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.052+0000 7fb1077fe640 1 -- 172.21.15.19:0/2566083090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1000319d0 con 0x7fb1080d1810 2024-04-25T09:55:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.053+0000 7fb10eb13640 1 -- 172.21.15.19:0/2566083090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1080d1810 msgr2=0x7fb1080d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.053+0000 7fb10eb13640 1 --2- 172.21.15.19:0/2566083090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1080d1810 0x7fb1080d1c80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb10806b820 tx=0x7fb10002a660 comp rx=0 tx=0).stop 2024-04-25T09:55:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.053+0000 7fb10eb13640 1 -- 172.21.15.19:0/2566083090 shutdown_connections 2024-04-25T09:55:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.053+0000 7fb10eb13640 1 --2- 172.21.15.19:0/2566083090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1080d1810 0x7fb1080d1c80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.053+0000 7fb10eb13640 1 --2- 172.21.15.19:0/2566083090 >> v2:172.21.15.112:3300/0 conn(0x7fb10815dc90 0x7fb1081625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.053+0000 7fb10eb13640 1 -- 172.21.15.19:0/2566083090 >> 172.21.15.19:0/2566083090 conn(0x7fb10806f590 msgr2=0x7fb10806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.053+0000 7fb10eb13640 1 -- 172.21.15.19:0/2566083090 shutdown_connections 2024-04-25T09:55:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.053+0000 7fb10eb13640 1 -- 172.21.15.19:0/2566083090 wait complete. 2024-04-25T09:55:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.054+0000 7fb10eb13640 1 Processor -- start 2024-04-25T09:55:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.054+0000 7fb10eb13640 1 -- start start 2024-04-25T09:55:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.055+0000 7fb10eb13640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1080cc8e0 0x7fb1080c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.055+0000 7fb10eb13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1080d1810 0x7fb1081f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.055+0000 7fb10eb13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb10815dc90 0x7fb10815fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.055+0000 7fb10eb13640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10815e070 con 0x7fb1080d1810 2024-04-25T09:55:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.055+0000 7fb10eb13640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080cd130 con 0x7fb10815dc90 2024-04-25T09:55:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.055+0000 7fb10eb13640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10815e1f0 con 0x7fb1080cc8e0 2024-04-25T09:55:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.055+0000 7fb107fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1080d1810 0x7fb1081f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.055+0000 7fb107fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1080d1810 0x7fb1081f4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39410/0 (socket says 172.21.15.19:39410) 2024-04-25T09:55:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.055+0000 7fb107fff640 1 -- 172.21.15.19:0/1046824851 learned_addr learned my addr 172.21.15.19:0/1046824851 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.055+0000 7fb10c8af640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1080cc8e0 0x7fb1080c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.055+0000 7fb10d0b0640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb10815dc90 0x7fb10815fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb107fff640 1 -- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1080cc8e0 msgr2=0x7fb1080c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb107fff640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1080cc8e0 0x7fb1080c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb107fff640 1 -- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb10815dc90 msgr2=0x7fb10815fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb107fff640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb10815dc90 0x7fb10815fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb107fff640 1 -- 172.21.15.19:0/1046824851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1081f7a50 con 0x7fb1080d1810 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb10d0b0640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb10815dc90 0x7fb10815fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb107fff640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1080d1810 0x7fb1081f4460 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00c440 tx=0x7fb0fc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb1057fa640 1 -- 172.21.15.19:0/1046824851 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc017070 con 0x7fb1080d1810 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb1057fa640 1 -- 172.21.15.19:0/1046824851 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc0035f0 con 0x7fb1080d1810 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb10eb13640 1 -- 172.21.15.19:0/1046824851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1081f4cb0 con 0x7fb1080d1810 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb1057fa640 1 -- 172.21.15.19:0/1046824851 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0039e0 con 0x7fb1080d1810 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.056+0000 7fb10eb13640 1 -- 172.21.15.19:0/1046824851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1081f5270 con 0x7fb1080d1810 2024-04-25T09:55:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.057+0000 7fb10eb13640 1 -- 172.21.15.19:0/1046824851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb108073990 con 0x7fb1080d1810 2024-04-25T09:55:39.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.057+0000 7fb1057fa640 1 -- 172.21.15.19:0/1046824851 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0fc01a070 con 0x7fb1080d1810 2024-04-25T09:55:39.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.058+0000 7fb1057fa640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0f406b4d0 0x7fb0f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.058+0000 7fb1057fa640 1 -- 172.21.15.19:0/1046824851 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fb0fc089670 con 0x7fb1080d1810 2024-04-25T09:55:39.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.058+0000 7fb10c8af640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0f406b4d0 0x7fb0f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.059+0000 7fb10c8af640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0f406b4d0 0x7fb0f406d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1081f79b0 tx=0x7fb0f8009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.062+0000 7fb1057fa640 1 -- 172.21.15.19:0/1046824851 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0fc0010c0 con 0x7fb1080d1810 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.278+0000 7fb10eb13640 1 -- 172.21.15.19:0/1046824851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1080c85d0 con 0x7fb1080d1810 2024-04-25T09:55:39.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.279+0000 7fb1057fa640 1 -- 172.21.15.19:0/1046824851 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fb0fc057ce0 con 0x7fb1080d1810 2024-04-25T09:55:39.282 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3845932448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3845932448},{"type":"v1","addr":"172.21.15.112:6836","nonce":3845932448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5122,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1655535300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1655535300},{"type":"v1","addr":"172.21.15.19:6834","nonce":1655535300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/165157787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":165157787},{"type":"v1","addr":"172.21.15.112:6837","nonce":165157787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:54:12.554449+0000","modified":"2024-04-25T09:54:19.314742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3017773710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3017773710},{"type":"v1","addr":"172.21.15.19:6835","nonce":3017773710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:55:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-04-25T09:55:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 -- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0f406b4d0 msgr2=0x7fb0f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0f406b4d0 0x7fb0f406d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1081f79b0 tx=0x7fb0f8009060 comp rx=0 tx=0).stop 2024-04-25T09:55:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 -- 172.21.15.19:0/1046824851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1080d1810 msgr2=0x7fb1081f4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1080d1810 0x7fb1081f4460 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00c440 tx=0x7fb0fc00c900 comp rx=0 tx=0).stop 2024-04-25T09:55:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 -- 172.21.15.19:0/1046824851 shutdown_connections 2024-04-25T09:55:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0f406b4d0 0x7fb0f406d990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb10815dc90 0x7fb10815fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1080d1810 0x7fb1081f4460 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 --2- 172.21.15.19:0/1046824851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1080cc8e0 0x7fb1080c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 -- 172.21.15.19:0/1046824851 >> 172.21.15.19:0/1046824851 conn(0x7fb10806f590 msgr2=0x7fb1080b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 -- 172.21.15.19:0/1046824851 shutdown_connections 2024-04-25T09:55:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.281+0000 7fb10eb13640 1 -- 172.21.15.19:0/1046824851 wait complete. 2024-04-25T09:55:39.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.392+0000 7f677d0b3640 1 Processor -- start 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.392+0000 7f677d0b3640 1 -- start start 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.393+0000 7f677d0b3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f67780b2cb0 0x7f67780d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.393+0000 7f677d0b3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f67780d1020 0x7f67780d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.393+0000 7f677d0b3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67780b97a0 con 0x7f677815dbe0 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.393+0000 7f677d0b3640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f677815dfc0 con 0x7f67780b2cb0 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.393+0000 7f677d0b3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67780d1a00 con 0x7f67780d1020 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.393+0000 7f677659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f67780b2cb0 0x7f67780d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.393+0000 7f677659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f67780b2cb0 0x7f67780d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39416/0 (socket says 172.21.15.19:39416) 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.393+0000 7f677659c640 1 -- 172.21.15.19:0/1985733317 learned_addr learned my addr 172.21.15.19:0/1985733317 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.393+0000 7f677759e640 1 --2- 172.21.15.19:0/1985733317 >> v2:172.21.15.112:3300/0 conn(0x7f67780d1020 0x7f67780d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.394+0000 7f677659c640 1 -- 172.21.15.19:0/1985733317 >> v2:172.21.15.112:3300/0 conn(0x7f67780d1020 msgr2=0x7f67780d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.394+0000 7f677659c640 1 --2- 172.21.15.19:0/1985733317 >> v2:172.21.15.112:3300/0 conn(0x7f67780d1020 0x7f67780d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.394+0000 7f677659c640 1 -- 172.21.15.19:0/1985733317 >> v1:172.21.15.112:6789/0 conn(0x7f677815dbe0 legacy=0x7f67780d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.394+0000 7f677659c640 1 -- 172.21.15.19:0/1985733317 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67780d1c50 con 0x7f67780b2cb0 2024-04-25T09:55:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.394+0000 7f677659c640 1 --2- 172.21.15.19:0/1985733317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67780b2cb0 0x7f67780d2730 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f67780cf570 tx=0x7f676002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f574501f7947b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.394+0000 7f6775d9b640 1 -- 172.21.15.19:0/1985733317 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760037070 con 0x7f67780b2cb0 2024-04-25T09:55:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.394+0000 7f6775d9b640 1 -- 172.21.15.19:0/1985733317 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6760032440 con 0x7f67780b2cb0 2024-04-25T09:55:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.394+0000 7f6775d9b640 1 -- 172.21.15.19:0/1985733317 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760032810 con 0x7f67780b2cb0 2024-04-25T09:55:39.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.395+0000 7f677d0b3640 1 -- 172.21.15.19:0/1985733317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67780b2cb0 msgr2=0x7f67780d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.395+0000 7f677d0b3640 1 --2- 172.21.15.19:0/1985733317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67780b2cb0 0x7f67780d2730 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f67780cf570 tx=0x7f676002a230 comp rx=0 tx=0).stop 2024-04-25T09:55:39.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.395+0000 7f677d0b3640 1 -- 172.21.15.19:0/1985733317 shutdown_connections 2024-04-25T09:55:39.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.395+0000 7f677d0b3640 1 --2- 172.21.15.19:0/1985733317 >> v2:172.21.15.112:3300/0 conn(0x7f67780d1020 0x7f67780d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.395+0000 7f677d0b3640 1 --2- 172.21.15.19:0/1985733317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67780b2cb0 0x7f67780d2730 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.395+0000 7f677d0b3640 1 -- 172.21.15.19:0/1985733317 >> 172.21.15.19:0/1985733317 conn(0x7f6778061a20 msgr2=0x7f6778061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.395+0000 7f677d0b3640 1 -- 172.21.15.19:0/1985733317 shutdown_connections 2024-04-25T09:55:39.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.395+0000 7f677d0b3640 1 -- 172.21.15.19:0/1985733317 wait complete. 2024-04-25T09:55:39.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.397+0000 7f677d0b3640 1 Processor -- start 2024-04-25T09:55:39.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.397+0000 7f677d0b3640 1 -- start start 2024-04-25T09:55:39.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.397+0000 7f677d0b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67780b2cb0 0x7f6778160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.397+0000 7f677d0b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67780d1020 0x7f677815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.397+0000 7f677d0b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677815dbe0 0x7f6778158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.397+0000 7f677d0b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67780b92a0 con 0x7f677815dbe0 2024-04-25T09:55:39.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.397+0000 7f677d0b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67780d2210 con 0x7f67780b2cb0 2024-04-25T09:55:39.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.397+0000 7f677d0b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67780b9420 con 0x7f67780d1020 2024-04-25T09:55:39.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.397+0000 7f677759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677815dbe0 0x7f6778158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.397+0000 7f677759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677815dbe0 0x7f6778158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39428/0 (socket says 172.21.15.19:39428) 2024-04-25T09:55:39.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.397+0000 7f677759e640 1 -- 172.21.15.19:0/2639566288 learned_addr learned my addr 172.21.15.19:0/2639566288 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:39.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f677659c640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67780d1020 0x7f677815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f6776d9d640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67780b2cb0 0x7f6778160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f677759e640 1 -- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67780d1020 msgr2=0x7f677815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f677759e640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67780d1020 0x7f677815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f677759e640 1 -- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67780b2cb0 msgr2=0x7f6778160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f677759e640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67780b2cb0 0x7f6778160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f677759e640 1 -- 172.21.15.19:0/2639566288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677815f390 con 0x7f677815dbe0 2024-04-25T09:55:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f677659c640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67780d1020 0x7f677815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f677759e640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677815dbe0 0x7f6778158480 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6768007e70 tx=0x7f676800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f676f7fe640 1 -- 172.21.15.19:0/2639566288 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768014620 con 0x7f677815dbe0 2024-04-25T09:55:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f676f7fe640 1 -- 172.21.15.19:0/2639566288 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6768019070 con 0x7f677815dbe0 2024-04-25T09:55:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f677d0b3640 1 -- 172.21.15.19:0/2639566288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677815f670 con 0x7f677815dbe0 2024-04-25T09:55:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f676f7fe640 1 -- 172.21.15.19:0/2639566288 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768014a20 con 0x7f677815dbe0 2024-04-25T09:55:39.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.398+0000 7f677d0b3640 1 -- 172.21.15.19:0/2639566288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677815fc30 con 0x7f677815dbe0 2024-04-25T09:55:39.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.399+0000 7f677d0b3640 1 -- 172.21.15.19:0/2639566288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677815a970 con 0x7f677815dbe0 2024-04-25T09:55:39.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.400+0000 7f676f7fe640 1 -- 172.21.15.19:0/2639566288 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6768014bc0 con 0x7f677815dbe0 2024-04-25T09:55:39.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.400+0000 7f676f7fe640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f675806b4d0 0x7f675806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.400+0000 7f676f7fe640 1 -- 172.21.15.19:0/2639566288 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f676808abe0 con 0x7f677815dbe0 2024-04-25T09:55:39.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.400+0000 7f6776d9d640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f675806b4d0 0x7f675806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.401+0000 7f6776d9d640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f675806b4d0 0x7f675806d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6770006fd0 tx=0x7f6770008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.404+0000 7f676f7fe640 1 -- 172.21.15.19:0/2639566288 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f677815a970 con 0x7f677815dbe0 2024-04-25T09:55:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.584+0000 7f677d0b3640 1 -- 172.21.15.19:0/2639566288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6778073990 con 0x7f677815dbe0 2024-04-25T09:55:39.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.586+0000 7f676f7fe640 1 -- 172.21.15.19:0/2639566288 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f6768059930 con 0x7f677815dbe0 2024-04-25T09:55:39.588 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:39.589 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:12.545580+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:11.132892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:11.536478+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.587+0000 7f677d0b3640 1 -- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f675806b4d0 msgr2=0x7f675806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.587+0000 7f677d0b3640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f675806b4d0 0x7f675806d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6770006fd0 tx=0x7f6770008040 comp rx=0 tx=0).stop 2024-04-25T09:55:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.587+0000 7f677d0b3640 1 -- 172.21.15.19:0/2639566288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677815dbe0 msgr2=0x7f6778158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.587+0000 7f677d0b3640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677815dbe0 0x7f6778158480 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6768007e70 tx=0x7f676800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.587+0000 7f677d0b3640 1 -- 172.21.15.19:0/2639566288 shutdown_connections 2024-04-25T09:55:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.587+0000 7f677d0b3640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f675806b4d0 0x7f675806d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.587+0000 7f677d0b3640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f677815dbe0 0x7f6778158480 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.587+0000 7f677d0b3640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67780d1020 0x7f677815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.587+0000 7f677d0b3640 1 --2- 172.21.15.19:0/2639566288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67780b2cb0 0x7f6778160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.587+0000 7f677d0b3640 1 -- 172.21.15.19:0/2639566288 >> 172.21.15.19:0/2639566288 conn(0x7f6778061a20 msgr2=0x7f677815a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.588+0000 7f677d0b3640 1 -- 172.21.15.19:0/2639566288 shutdown_connections 2024-04-25T09:55:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.588+0000 7f677d0b3640 1 -- 172.21.15.19:0/2639566288 wait complete. 2024-04-25T09:55:39.601 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:55:39.602 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:39.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.699+0000 7fed69ff4640 1 Processor -- start 2024-04-25T09:55:39.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.700+0000 7fed69ff4640 1 -- start start 2024-04-25T09:55:39.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.700+0000 7fed69ff4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed6415dc00 0x7fed640c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.700+0000 7fed69ff4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6415dfe0 con 0x7fed640d1dd0 2024-04-25T09:55:39.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.700+0000 7fed69ff4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed640d00c0 con 0x7fed640ceb20 2024-04-25T09:55:39.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.700+0000 7fed69ff4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed640d03a0 con 0x7fed6415dc00 2024-04-25T09:55:39.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.701+0000 7fed637fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed6415dc00 0x7fed640c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.701+0000 7fed637fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed6415dc00 0x7fed640c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39432/0 (socket says 172.21.15.19:39432) 2024-04-25T09:55:39.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.701+0000 7fed637fe640 1 -- 172.21.15.19:0/3683363941 learned_addr learned my addr 172.21.15.19:0/3683363941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:39.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.701+0000 7fed637fe640 1 -- 172.21.15.19:0/3683363941 >> v1:172.21.15.112:6789/0 conn(0x7fed640ceb20 legacy=0x7fed640d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.701+0000 7fed637fe640 1 -- 172.21.15.19:0/3683363941 >> v1:172.21.15.19:6789/0 conn(0x7fed640d1dd0 legacy=0x7fed640ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.701+0000 7fed637fe640 1 -- 172.21.15.19:0/3683363941 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640cc440 con 0x7fed6415dc00 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.702+0000 7fed637fe640 1 --2- 172.21.15.19:0/3683363941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6415dc00 0x7fed640c6df0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed5402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c51e163366128 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.702+0000 7fed627fc640 1 -- 172.21.15.19:0/3683363941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54037070 con 0x7fed6415dc00 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.702+0000 7fed627fc640 1 -- 172.21.15.19:0/3683363941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5402ae50 con 0x7fed6415dc00 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.702+0000 7fed627fc640 1 -- 172.21.15.19:0/3683363941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54032860 con 0x7fed6415dc00 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.702+0000 7fed69ff4640 1 -- 172.21.15.19:0/3683363941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6415dc00 msgr2=0x7fed640c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.702+0000 7fed69ff4640 1 --2- 172.21.15.19:0/3683363941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6415dc00 0x7fed640c6df0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed5402a300 comp rx=0 tx=0).stop 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.702+0000 7fed69ff4640 1 -- 172.21.15.19:0/3683363941 shutdown_connections 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.702+0000 7fed69ff4640 1 --2- 172.21.15.19:0/3683363941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6415dc00 0x7fed640c6df0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.702+0000 7fed69ff4640 1 -- 172.21.15.19:0/3683363941 >> 172.21.15.19:0/3683363941 conn(0x7fed6406f5d0 msgr2=0x7fed6406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.702+0000 7fed69ff4640 1 -- 172.21.15.19:0/3683363941 shutdown_connections 2024-04-25T09:55:39.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.702+0000 7fed69ff4640 1 -- 172.21.15.19:0/3683363941 wait complete. 2024-04-25T09:55:39.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.703+0000 7fed69ff4640 1 Processor -- start 2024-04-25T09:55:39.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.704+0000 7fed69ff4640 1 -- start start 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.704+0000 7fed69ff4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed640ceb20 0x7fed640b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.704+0000 7fed69ff4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed640d1dd0 0x7fed640b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.704+0000 7fed69ff4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed6415dc00 0x7fed6414ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.704+0000 7fed69ff4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed640cfd90 con 0x7fed640d1dd0 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.704+0000 7fed69ff4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed640cc130 con 0x7fed640ceb20 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.704+0000 7fed69ff4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed640cdd70 con 0x7fed6415dc00 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.704+0000 7fed62ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed640d1dd0 0x7fed640b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.704+0000 7fed62ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed640d1dd0 0x7fed640b70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39436/0 (socket says 172.21.15.19:39436) 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed62ffd640 1 -- 172.21.15.19:0/764568142 learned_addr learned my addr 172.21.15.19:0/764568142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed637fe640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed640ceb20 0x7fed640b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed63fff640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed6415dc00 0x7fed6414ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed62ffd640 1 -- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed6415dc00 msgr2=0x7fed6414ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed62ffd640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed6415dc00 0x7fed6414ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed62ffd640 1 -- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed640ceb20 msgr2=0x7fed640b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed62ffd640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed640ceb20 0x7fed640b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed62ffd640 1 -- 172.21.15.19:0/764568142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640b4f70 con 0x7fed640d1dd0 2024-04-25T09:55:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed637fe640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed640ceb20 0x7fed640b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed62ffd640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed640d1dd0 0x7fed640b70d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c430 tx=0x7fed5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed4bfff640 1 -- 172.21.15.19:0/764568142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed580052d0 con 0x7fed640d1dd0 2024-04-25T09:55:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed4bfff640 1 -- 172.21.15.19:0/764568142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed58005490 con 0x7fed640d1dd0 2024-04-25T09:55:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed69ff4640 1 -- 172.21.15.19:0/764568142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed640b5250 con 0x7fed640d1dd0 2024-04-25T09:55:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed4bfff640 1 -- 172.21.15.19:0/764568142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58007260 con 0x7fed640d1dd0 2024-04-25T09:55:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.705+0000 7fed69ff4640 1 -- 172.21.15.19:0/764568142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed640b5af0 con 0x7fed640d1dd0 2024-04-25T09:55:39.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.706+0000 7fed69ff4640 1 -- 172.21.15.19:0/764568142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed64073990 con 0x7fed640d1dd0 2024-04-25T09:55:39.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.707+0000 7fed4bfff640 1 -- 172.21.15.19:0/764568142 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed58007400 con 0x7fed640d1dd0 2024-04-25T09:55:39.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.707+0000 7fed4bfff640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed5006b4d0 0x7fed5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.707+0000 7fed4bfff640 1 -- 172.21.15.19:0/764568142 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fed58089940 con 0x7fed640d1dd0 2024-04-25T09:55:39.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.708+0000 7fed637fe640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed5006b4d0 0x7fed5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.708+0000 7fed637fe640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed5006b4d0 0x7fed5006d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fed54034000 tx=0x7fed5402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.712+0000 7fed4bfff640 1 -- 172.21.15.19:0/764568142 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed58058570 con 0x7fed640d1dd0 2024-04-25T09:55:39.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.925+0000 7fed69ff4640 1 -- 172.21.15.19:0/764568142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed640b5420 con 0x7fed640d1dd0 2024-04-25T09:55:39.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.926+0000 7fed4bfff640 1 -- 172.21.15.19:0/764568142 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fed58057e70 con 0x7fed640d1dd0 2024-04-25T09:55:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:39.929 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3845932448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3845932448},{"type":"v1","addr":"172.21.15.112:6836","nonce":3845932448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5122,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1655535300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1655535300},{"type":"v1","addr":"172.21.15.19:6834","nonce":1655535300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/165157787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":165157787},{"type":"v1","addr":"172.21.15.112:6837","nonce":165157787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:54:12.554449+0000","modified":"2024-04-25T09:54:19.314742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3017773710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3017773710},{"type":"v1","addr":"172.21.15.19:6835","nonce":3017773710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:55:39.929 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-04-25T09:55:39.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 -- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed5006b4d0 msgr2=0x7fed5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed5006b4d0 0x7fed5006d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fed54034000 tx=0x7fed5402e000 comp rx=0 tx=0).stop 2024-04-25T09:55:39.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 -- 172.21.15.19:0/764568142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed640d1dd0 msgr2=0x7fed640b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed640d1dd0 0x7fed640b70d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c430 tx=0x7fed5800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 -- 172.21.15.19:0/764568142 shutdown_connections 2024-04-25T09:55:39.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed5006b4d0 0x7fed5006d990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed6415dc00 0x7fed6414ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed640d1dd0 0x7fed640b70d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 --2- 172.21.15.19:0/764568142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed640ceb20 0x7fed640b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 -- 172.21.15.19:0/764568142 >> 172.21.15.19:0/764568142 conn(0x7fed6406f5d0 msgr2=0x7fed640ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 -- 172.21.15.19:0/764568142 shutdown_connections 2024-04-25T09:55:39.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:39.928+0000 7fed69ff4640 1 -- 172.21.15.19:0/764568142 wait complete. 2024-04-25T09:55:39.942 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.039+0000 7f813139e640 1 Processor -- start 2024-04-25T09:55:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.039+0000 7f813139e640 1 -- start start 2024-04-25T09:55:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.040+0000 7f813139e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0afa00 0x7f812c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.040+0000 7f813139e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812c0aef60 con 0x7f812c145310 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.040+0000 7f813139e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812c0b5590 con 0x7f812c06b120 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.040+0000 7f813139e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0af200 con 0x7f812c0afa00 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.040+0000 7f812affd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f812c145310 0x7f812c15ff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42506/0 (socket says 172.21.15.19:42506) 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.040+0000 7f812affd640 1 -- 172.21.15.19:0/3361647963 learned_addr learned my addr 172.21.15.19:0/3361647963 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704076414 0 0) 0x7f812c0aef60 con 0x7f812c145310 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8114003520 con 0x7f812c145310 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f812a7fc640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0afa00 0x7f812c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3616091819 0 0) 0x7f8114003520 con 0x7f812c145310 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0afa00 msgr2=0x7f812c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0afa00 0x7f812c06abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 >> v1:172.21.15.112:6789/0 conn(0x7f812c06b120 legacy=0x7f812c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c0af4e0 con 0x7f812c145310 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8124001670 con 0x7f812c145310 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 >> v1:172.21.15.19:6789/0 conn(0x7f812c145310 legacy=0x7f812c15ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81140039e0 0x7f8114003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8114004800 0x7f8114004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0afa00 0x7f81140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0af200 con 0x7f8114004800 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811400ba70 con 0x7f81140039e0 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811400bc70 con 0x7f812c0afa00 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f812a7fc640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8114004800 0x7f8114004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f812b7fe640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0afa00 0x7f81140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f812affd640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81140039e0 0x7f8114003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f812a7fc640 1 -- 172.21.15.19:0/3361647963 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0afa00 msgr2=0x7f81140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f812a7fc640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c0afa00 0x7f81140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f812a7fc640 1 -- 172.21.15.19:0/3361647963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81140039e0 msgr2=0x7f8114003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f812a7fc640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81140039e0 0x7f8114003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.041+0000 7f812a7fc640 1 -- 172.21.15.19:0/3361647963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8114003520 con 0x7f8114004800 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.042+0000 7f812a7fc640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8114004800 0x7f8114004c80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8120009a90 tx=0x7f8114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6151db6107577d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.042+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120036070 con 0x7f8114004800 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.042+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8120031400 con 0x7f8114004800 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.042+0000 7f8129ffb640 1 -- 172.21.15.19:0/3361647963 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120009690 con 0x7f8114004800 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.043+0000 7f813139e640 1 -- 172.21.15.19:0/3361647963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8114004800 msgr2=0x7f8114004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.043+0000 7f813139e640 1 --2- 172.21.15.19:0/3361647963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8114004800 0x7f8114004c80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8120009a90 tx=0x7f8114005830 comp rx=0 tx=0).stop 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.043+0000 7f812b7fe640 1 -- 172.21.15.19:0/3361647963 reap_dead start 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.043+0000 7f813139e640 1 -- 172.21.15.19:0/3361647963 shutdown_connections 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.043+0000 7f813139e640 1 -- 172.21.15.19:0/3361647963 >> 172.21.15.19:0/3361647963 conn(0x7f812c0b5af0 msgr2=0x7f812c0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:40.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.043+0000 7f813139e640 1 -- 172.21.15.19:0/3361647963 shutdown_connections 2024-04-25T09:55:40.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.043+0000 7f813139e640 1 -- 172.21.15.19:0/3361647963 wait complete. 2024-04-25T09:55:40.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.044+0000 7f813139e640 1 Processor -- start 2024-04-25T09:55:40.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.044+0000 7f813139e640 1 -- start start 2024-04-25T09:55:40.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f813139e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8114004800 0x7f812c09e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f813139e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c13eca0 0x7f812c15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f813139e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c1402a0 0x7f812c0ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f813139e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0b51f0 con 0x7f812c13eca0 2024-04-25T09:55:40.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f813139e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0ad7e0 con 0x7f8114004800 2024-04-25T09:55:40.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f813139e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0b5370 con 0x7f812c1402a0 2024-04-25T09:55:40.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f812a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c13eca0 0x7f812c15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f812a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c13eca0 0x7f812c15b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39458/0 (socket says 172.21.15.19:39458) 2024-04-25T09:55:40.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f812a7fc640 1 -- 172.21.15.19:0/213045376 learned_addr learned my addr 172.21.15.19:0/213045376 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:40.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f812affd640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8114004800 0x7f812c09e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f812b7fe640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c1402a0 0x7f812c0ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f812a7fc640 1 -- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c1402a0 msgr2=0x7f812c0ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f812a7fc640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c1402a0 0x7f812c0ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f812a7fc640 1 -- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8114004800 msgr2=0x7f812c09e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f812a7fc640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8114004800 0x7f812c09e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.045+0000 7f812a7fc640 1 -- 172.21.15.19:0/213045376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c0cdd30 con 0x7f812c13eca0 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.046+0000 7f812a7fc640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c13eca0 0x7f812c15b430 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8120034040 tx=0x7f8120004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.046+0000 7f81137fe640 1 -- 172.21.15.19:0/213045376 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120040020 con 0x7f812c13eca0 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.046+0000 7f81137fe640 1 -- 172.21.15.19:0/213045376 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8120003940 con 0x7f812c13eca0 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.046+0000 7f81137fe640 1 -- 172.21.15.19:0/213045376 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120036040 con 0x7f812c13eca0 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.046+0000 7f813139e640 1 -- 172.21.15.19:0/213045376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c0cdfa0 con 0x7f812c13eca0 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.046+0000 7f813139e640 1 -- 172.21.15.19:0/213045376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c0ce560 con 0x7f812c13eca0 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.047+0000 7f813139e640 1 -- 172.21.15.19:0/213045376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812c073990 con 0x7f812c13eca0 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.047+0000 7f81137fe640 1 -- 172.21.15.19:0/213045376 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8120043020 con 0x7f812c13eca0 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.048+0000 7f81137fe640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f811806b4d0 0x7f811806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.048+0000 7f81137fe640 1 -- 172.21.15.19:0/213045376 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f81200aa7b0 con 0x7f812c13eca0 2024-04-25T09:55:40.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.048+0000 7f812affd640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f811806b4d0 0x7f811806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.048+0000 7f812affd640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f811806b4d0 0x7f811806d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8124001050 tx=0x7f8124000f80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.052+0000 7f81137fe640 1 -- 172.21.15.19:0/213045376 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f812004a030 con 0x7f812c13eca0 2024-04-25T09:55:40.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.270+0000 7f813139e640 1 -- 172.21.15.19:0/213045376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f812c0d0490 con 0x7f812c13eca0 2024-04-25T09:55:40.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.270+0000 7f81137fe640 1 -- 172.21.15.19:0/213045376 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f8120078eb0 con 0x7f812c13eca0 2024-04-25T09:55:40.273 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:40.273 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3845932448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3845932448},{"type":"v1","addr":"172.21.15.112:6836","nonce":3845932448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5122,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1655535300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1655535300},{"type":"v1","addr":"172.21.15.19:6834","nonce":1655535300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/165157787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":165157787},{"type":"v1","addr":"172.21.15.112:6837","nonce":165157787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:54:12.554449+0000","modified":"2024-04-25T09:54:19.314742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3017773710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3017773710},{"type":"v1","addr":"172.21.15.19:6835","nonce":3017773710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:55:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-04-25T09:55:40.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 -- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f811806b4d0 msgr2=0x7f811806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f811806b4d0 0x7f811806d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8124001050 tx=0x7f8124000f80 comp rx=0 tx=0).stop 2024-04-25T09:55:40.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 -- 172.21.15.19:0/213045376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c13eca0 msgr2=0x7f812c15b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c13eca0 0x7f812c15b430 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8120034040 tx=0x7f8120004730 comp rx=0 tx=0).stop 2024-04-25T09:55:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 -- 172.21.15.19:0/213045376 shutdown_connections 2024-04-25T09:55:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f811806b4d0 0x7f811806d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f812c1402a0 0x7f812c0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f812c13eca0 0x7f812c15b430 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 --2- 172.21.15.19:0/213045376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8114004800 0x7f812c09e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 -- 172.21.15.19:0/213045376 >> 172.21.15.19:0/213045376 conn(0x7f812c0b5af0 msgr2=0x7f812c0b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 -- 172.21.15.19:0/213045376 shutdown_connections 2024-04-25T09:55:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.272+0000 7f813139e640 1 -- 172.21.15.19:0/213045376 wait complete. 2024-04-25T09:55:40.286 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:40.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.383+0000 7f05705b6640 1 Processor -- start 2024-04-25T09:55:40.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.383+0000 7f05705b6640 1 -- start start 2024-04-25T09:55:40.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.384+0000 7f05705b6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f056815dd30 0x7f05680b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.384+0000 7f05705b6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f05680b9770 0x7f05680d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.384+0000 7f05705b6640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05680d1df0 con 0x7f05680d2e90 2024-04-25T09:55:40.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.384+0000 7f05705b6640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05680b7a50 con 0x7f05680b9770 2024-04-25T09:55:40.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.384+0000 7f05705b6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05680d20d0 con 0x7f056815dd30 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.384+0000 7f056db51640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f05680b9770 0x7f05680d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.384+0000 7f056db51640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f05680b9770 0x7f05680d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39460/0 (socket says 172.21.15.19:39460) 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.384+0000 7f056db51640 1 -- 172.21.15.19:0/3860038973 learned_addr learned my addr 172.21.15.19:0/3860038973 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.385+0000 7f056e352640 1 --2- 172.21.15.19:0/3860038973 >> v2:172.21.15.112:3300/0 conn(0x7f056815dd30 0x7f05680b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.385+0000 7f056db51640 1 -- 172.21.15.19:0/3860038973 >> v2:172.21.15.112:3300/0 conn(0x7f056815dd30 msgr2=0x7f05680b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.385+0000 7f056db51640 1 --2- 172.21.15.19:0/3860038973 >> v2:172.21.15.112:3300/0 conn(0x7f056815dd30 0x7f05680b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.385+0000 7f056db51640 1 -- 172.21.15.19:0/3860038973 >> v1:172.21.15.19:6789/0 conn(0x7f05680d2e90 legacy=0x7f05680b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.385+0000 7f056db51640 1 -- 172.21.15.19:0/3860038973 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05680d23b0 con 0x7f05680b9770 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.385+0000 7f056db51640 1 --2- 172.21.15.19:0/3860038973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05680b9770 0x7f05680d03c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f056806d000 tx=0x7f055c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128c7bdabffd32fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.386+0000 7f056d350640 1 -- 172.21.15.19:0/3860038973 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c004030 con 0x7f05680b9770 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.386+0000 7f056d350640 1 -- 172.21.15.19:0/3860038973 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c0041d0 con 0x7f05680b9770 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.386+0000 7f056d350640 1 -- 172.21.15.19:0/3860038973 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c003610 con 0x7f05680b9770 2024-04-25T09:55:40.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.386+0000 7f05705b6640 1 -- 172.21.15.19:0/3860038973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05680b9770 msgr2=0x7f05680d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.386+0000 7f05705b6640 1 --2- 172.21.15.19:0/3860038973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05680b9770 0x7f05680d03c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f056806d000 tx=0x7f055c02a130 comp rx=0 tx=0).stop 2024-04-25T09:55:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.386+0000 7f05705b6640 1 -- 172.21.15.19:0/3860038973 shutdown_connections 2024-04-25T09:55:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.386+0000 7f05705b6640 1 --2- 172.21.15.19:0/3860038973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05680b9770 0x7f05680d03c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.386+0000 7f05705b6640 1 --2- 172.21.15.19:0/3860038973 >> v2:172.21.15.112:3300/0 conn(0x7f056815dd30 0x7f05680b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.386+0000 7f05705b6640 1 -- 172.21.15.19:0/3860038973 >> 172.21.15.19:0/3860038973 conn(0x7f0568061c30 msgr2=0x7f056806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.386+0000 7f05705b6640 1 -- 172.21.15.19:0/3860038973 shutdown_connections 2024-04-25T09:55:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.386+0000 7f05705b6640 1 -- 172.21.15.19:0/3860038973 wait complete. 2024-04-25T09:55:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.387+0000 7f05705b6640 1 Processor -- start 2024-04-25T09:55:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.387+0000 7f05705b6640 1 -- start start 2024-04-25T09:55:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f05705b6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05680b9770 0x7f05681f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f05705b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05680d2e90 0x7f0568158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f05705b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f056815dd30 0x7f0568159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f05705b6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05680d1a10 con 0x7f05680b9770 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f05705b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05680d2970 con 0x7f05680d2e90 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f05705b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05680d1b90 con 0x7f056815dd30 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f056e352640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05680b9770 0x7f05681f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f056e352640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05680b9770 0x7f05681f3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39462/0 (socket says 172.21.15.19:39462) 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f056e352640 1 -- 172.21.15.19:0/3317798532 learned_addr learned my addr 172.21.15.19:0/3317798532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f056db51640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05680d2e90 0x7f0568158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f056eb53640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f056815dd30 0x7f0568159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f056e352640 1 -- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f056815dd30 msgr2=0x7f0568159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.388+0000 7f056e352640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f056815dd30 0x7f0568159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.389+0000 7f056e352640 1 -- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05680d2e90 msgr2=0x7f0568158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.389+0000 7f056e352640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05680d2e90 0x7f0568158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.389+0000 7f056e352640 1 -- 172.21.15.19:0/3317798532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05681f36c0 con 0x7f05680b9770 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.389+0000 7f056eb53640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f056815dd30 0x7f0568159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.389+0000 7f056e352640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05680b9770 0x7f05681f3180 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f056000d5f0 tx=0x7f056000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.389+0000 7f0566ffd640 1 -- 172.21.15.19:0/3317798532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05600125f0 con 0x7f05680b9770 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.389+0000 7f0566ffd640 1 -- 172.21.15.19:0/3317798532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0560019070 con 0x7f05680b9770 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.389+0000 7f0566ffd640 1 -- 172.21.15.19:0/3317798532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05600129d0 con 0x7f05680b9770 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.389+0000 7f05705b6640 1 -- 172.21.15.19:0/3317798532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056815ed50 con 0x7f05680b9770 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.389+0000 7f05705b6640 1 -- 172.21.15.19:0/3317798532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056815f310 con 0x7f05680b9770 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.390+0000 7f05705b6640 1 -- 172.21.15.19:0/3317798532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0568073990 con 0x7f05680b9770 2024-04-25T09:55:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.390+0000 7f0566ffd640 1 -- 172.21.15.19:0/3317798532 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0560012b70 con 0x7f05680b9770 2024-04-25T09:55:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.391+0000 7f0566ffd640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f055406b4d0 0x7f055406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.391+0000 7f0566ffd640 1 -- 172.21.15.19:0/3317798532 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f056008a780 con 0x7f05680b9770 2024-04-25T09:55:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.391+0000 7f056db51640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f055406b4d0 0x7f055406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.391+0000 7f056db51640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f055406b4d0 0x7f055406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f055c033000 tx=0x7f055c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.395+0000 7f0566ffd640 1 -- 172.21.15.19:0/3317798532 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0560059440 con 0x7f05680b9770 2024-04-25T09:55:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.579+0000 7f05705b6640 1 -- 172.21.15.19:0/3317798532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05680d4240 con 0x7f05680b9770 2024-04-25T09:55:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.580+0000 7f0566ffd640 1 -- 172.21.15.19:0/3317798532 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f0560058d40 con 0x7f05680b9770 2024-04-25T09:55:40.583 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:40.583 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:12.545580+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:11.132892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:11.536478+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 -- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f055406b4d0 msgr2=0x7f055406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f055406b4d0 0x7f055406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f055c033000 tx=0x7f055c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 -- 172.21.15.19:0/3317798532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05680b9770 msgr2=0x7f05681f3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05680b9770 0x7f05681f3180 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f056000d5f0 tx=0x7f056000dab0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 -- 172.21.15.19:0/3317798532 shutdown_connections 2024-04-25T09:55:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f055406b4d0 0x7f055406d990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f056815dd30 0x7f0568159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05680d2e90 0x7f0568158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 --2- 172.21.15.19:0/3317798532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05680b9770 0x7f05681f3180 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 -- 172.21.15.19:0/3317798532 >> 172.21.15.19:0/3317798532 conn(0x7f0568061c30 msgr2=0x7f05680b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 -- 172.21.15.19:0/3317798532 shutdown_connections 2024-04-25T09:55:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.583+0000 7f05705b6640 1 -- 172.21.15.19:0/3317798532 wait complete. 2024-04-25T09:55:40.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:55:40.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.695+0000 7fdefb693640 1 Processor -- start 2024-04-25T09:55:40.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.695+0000 7fdefb693640 1 -- start start 2024-04-25T09:55:40.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdefb693640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdef40ced80 0x7fdef40d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdefb693640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdef4161710 0x7fdef40d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdefb693640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef415e000 con 0x7fdef415dc20 2024-04-25T09:55:40.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdefb693640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef40d1de0 con 0x7fdef4161710 2024-04-25T09:55:40.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdefb693640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef4161c60 con 0x7fdef40ced80 2024-04-25T09:55:40.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdef9c30640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdef4161710 0x7fdef40d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdef9c30640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdef4161710 0x7fdef40d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39476/0 (socket says 172.21.15.19:39476) 2024-04-25T09:55:40.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdef9c30640 1 -- 172.21.15.19:0/462456661 learned_addr learned my addr 172.21.15.19:0/462456661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:40.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdef8c2e640 1 --2- 172.21.15.19:0/462456661 >> v2:172.21.15.112:3300/0 conn(0x7fdef40ced80 0x7fdef40d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdef9c30640 1 -- 172.21.15.19:0/462456661 >> v2:172.21.15.112:3300/0 conn(0x7fdef40ced80 msgr2=0x7fdef40d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:40.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdef9c30640 1 --2- 172.21.15.19:0/462456661 >> v2:172.21.15.112:3300/0 conn(0x7fdef40ced80 0x7fdef40d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdef9c30640 1 -- 172.21.15.19:0/462456661 >> v1:172.21.15.19:6789/0 conn(0x7fdef415dc20 legacy=0x7fdef40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.696+0000 7fdef9c30640 1 -- 172.21.15.19:0/462456661 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef4161f40 con 0x7fdef4161710 2024-04-25T09:55:40.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.697+0000 7fdef9c30640 1 --2- 172.21.15.19:0/462456661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdef4161710 0x7fdef40d2820 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdef406da90 tx=0x7fdeec02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4902bee984e3a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.697+0000 7fdeebfff640 1 -- 172.21.15.19:0/462456661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec036070 con 0x7fdef4161710 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.697+0000 7fdeebfff640 1 -- 172.21.15.19:0/462456661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec031600 con 0x7fdef4161710 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.697+0000 7fdeebfff640 1 -- 172.21.15.19:0/462456661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec0319d0 con 0x7fdef4161710 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.697+0000 7fdefb693640 1 -- 172.21.15.19:0/462456661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdef4161710 msgr2=0x7fdef40d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.697+0000 7fdefb693640 1 --2- 172.21.15.19:0/462456661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdef4161710 0x7fdef40d2820 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdef406da90 tx=0x7fdeec02a730 comp rx=0 tx=0).stop 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.698+0000 7fdefb693640 1 -- 172.21.15.19:0/462456661 shutdown_connections 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.698+0000 7fdefb693640 1 --2- 172.21.15.19:0/462456661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdef4161710 0x7fdef40d2820 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.698+0000 7fdefb693640 1 --2- 172.21.15.19:0/462456661 >> v2:172.21.15.112:3300/0 conn(0x7fdef40ced80 0x7fdef40d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.698+0000 7fdefb693640 1 -- 172.21.15.19:0/462456661 >> 172.21.15.19:0/462456661 conn(0x7fdef40d0420 msgr2=0x7fdef40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.698+0000 7fdefb693640 1 -- 172.21.15.19:0/462456661 shutdown_connections 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.698+0000 7fdefb693640 1 -- 172.21.15.19:0/462456661 wait complete. 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.699+0000 7fdefb693640 1 Processor -- start 2024-04-25T09:55:40.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.699+0000 7fdefb693640 1 -- start start 2024-04-25T09:55:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.699+0000 7fdefb693640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdef40ced80 0x7fdef40c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.699+0000 7fdefb693640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdef415dc20 0x7fdef40c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdefb693640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdef4161710 0x7fdef4152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdefb693640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40b5e50 con 0x7fdef415dc20 2024-04-25T09:55:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdefb693640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef4162500 con 0x7fdef40ced80 2024-04-25T09:55:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdefb693640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40b5fd0 con 0x7fdef4161710 2024-04-25T09:55:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef8c2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdef415dc20 0x7fdef40c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef8c2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdef415dc20 0x7fdef40c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39482/0 (socket says 172.21.15.19:39482) 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef8c2e640 1 -- 172.21.15.19:0/1771531457 learned_addr learned my addr 172.21.15.19:0/1771531457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef9c30640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdef4161710 0x7fdef4152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef942f640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdef40ced80 0x7fdef40c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef8c2e640 1 -- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdef4161710 msgr2=0x7fdef4152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef8c2e640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdef4161710 0x7fdef4152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef8c2e640 1 -- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdef40ced80 msgr2=0x7fdef40c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef8c2e640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdef40ced80 0x7fdef40c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef8c2e640 1 -- 172.21.15.19:0/1771531457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef415e1f0 con 0x7fdef415dc20 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef9c30640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdef4161710 0x7fdef4152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdef8c2e640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdef415dc20 0x7fdef40c8aa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdee400d5f0 tx=0x7fdee400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdee9ffb640 1 -- 172.21.15.19:0/1771531457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40125f0 con 0x7fdef415dc20 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdee9ffb640 1 -- 172.21.15.19:0/1771531457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee4019070 con 0x7fdef415dc20 2024-04-25T09:55:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdefb693640 1 -- 172.21.15.19:0/1771531457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef40c8fe0 con 0x7fdef415dc20 2024-04-25T09:55:40.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.700+0000 7fdee9ffb640 1 -- 172.21.15.19:0/1771531457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40129d0 con 0x7fdef415dc20 2024-04-25T09:55:40.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.701+0000 7fdefb693640 1 -- 172.21.15.19:0/1771531457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef415e4b0 con 0x7fdef415dc20 2024-04-25T09:55:40.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.702+0000 7fdefb693640 1 -- 172.21.15.19:0/1771531457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef40ca730 con 0x7fdef415dc20 2024-04-25T09:55:40.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.702+0000 7fdee9ffb640 1 -- 172.21.15.19:0/1771531457 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdee4012b70 con 0x7fdef415dc20 2024-04-25T09:55:40.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.702+0000 7fdee9ffb640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded806b500 0x7fded806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.702+0000 7fdee9ffb640 1 -- 172.21.15.19:0/1771531457 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdee408a6a0 con 0x7fdef415dc20 2024-04-25T09:55:40.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.703+0000 7fdef942f640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded806b500 0x7fded806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.703+0000 7fdef942f640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded806b500 0x7fded806d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdef0007fd0 tx=0x7fdef0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.707+0000 7fdee9ffb640 1 -- 172.21.15.19:0/1771531457 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdef40ca730 con 0x7fdef415dc20 2024-04-25T09:55:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.923+0000 7fdefb693640 1 -- 172.21.15.19:0/1771531457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdef4073a00 con 0x7fdef415dc20 2024-04-25T09:55:40.926 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T09:55:40.923+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:55:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.925+0000 7fdee9ffb640 1 -- 172.21.15.19:0/1771531457 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdee40593f0 con 0x7fdef415dc20 2024-04-25T09:55:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:55:40.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 -- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded806b500 msgr2=0x7fded806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded806b500 0x7fded806d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdef0007fd0 tx=0x7fdef0009040 comp rx=0 tx=0).stop 2024-04-25T09:55:40.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 -- 172.21.15.19:0/1771531457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdef415dc20 msgr2=0x7fdef40c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdef415dc20 0x7fdef40c8aa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdee400d5f0 tx=0x7fdee400dab0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 -- 172.21.15.19:0/1771531457 shutdown_connections 2024-04-25T09:55:40.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded806b500 0x7fded806d9c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdef4161710 0x7fdef4152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdef415dc20 0x7fdef40c8aa0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 --2- 172.21.15.19:0/1771531457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdef40ced80 0x7fdef40c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 -- 172.21.15.19:0/1771531457 >> 172.21.15.19:0/1771531457 conn(0x7fdef40d0420 msgr2=0x7fdef40d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:40.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 -- 172.21.15.19:0/1771531457 shutdown_connections 2024-04-25T09:55:40.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:40.928+0000 7fdefb693640 1 -- 172.21.15.19:0/1771531457 wait complete. 2024-04-25T09:55:40.942 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:55:40.970 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T09:55:41.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.043+0000 7fbcf3c1f640 1 Processor -- start 2024-04-25T09:55:41.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.043+0000 7fbcf3c1f640 1 -- start start 2024-04-25T09:55:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf3c1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcec15dac0 0x7fbcec0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf3c1f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec0d2420 con 0x7fbcec0d0ab0 2024-04-25T09:55:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf3c1f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec0d2700 con 0x7fbcec0d1f30 2024-04-25T09:55:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf3c1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0d3030 con 0x7fbcec15dac0 2024-04-25T09:55:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf21bc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbcec0d0ab0 0x7fbcec0b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42518/0 (socket says 172.21.15.19:42518) 2024-04-25T09:55:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf21bc640 1 -- 172.21.15.19:0/4056989879 learned_addr learned my addr 172.21.15.19:0/4056989879 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf19bb640 1 --2- 172.21.15.19:0/4056989879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcec15dac0 0x7fbcec0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf09b9640 1 -- 172.21.15.19:0/4056989879 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799787168 0 0) 0x7fbcec0d2420 con 0x7fbcec0d0ab0 2024-04-25T09:55:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf09b9640 1 -- 172.21.15.19:0/4056989879 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcd8003520 con 0x7fbcec0d0ab0 2024-04-25T09:55:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf19bb640 1 -- 172.21.15.19:0/4056989879 >> v1:172.21.15.112:6789/0 conn(0x7fbcec0d1f30 legacy=0x7fbcec0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf19bb640 1 -- 172.21.15.19:0/4056989879 >> v1:172.21.15.19:6789/0 conn(0x7fbcec0d0ab0 legacy=0x7fbcec0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.044+0000 7fbcf19bb640 1 -- 172.21.15.19:0/4056989879 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0d29e0 con 0x7fbcec15dac0 2024-04-25T09:55:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.045+0000 7fbcf19bb640 1 --2- 172.21.15.19:0/4056989879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcec15dac0 0x7fbcec0d19f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0d31b0 tx=0x7fbce802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a408665db6ed121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.045+0000 7fbcf09b9640 1 -- 172.21.15.19:0/4056989879 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8037070 con 0x7fbcec15dac0 2024-04-25T09:55:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.045+0000 7fbcf09b9640 1 -- 172.21.15.19:0/4056989879 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce8032440 con 0x7fbcec15dac0 2024-04-25T09:55:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.045+0000 7fbcf09b9640 1 -- 172.21.15.19:0/4056989879 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce80327f0 con 0x7fbcec15dac0 2024-04-25T09:55:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.046+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/4056989879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcec15dac0 msgr2=0x7fbcec0d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.046+0000 7fbcf3c1f640 1 --2- 172.21.15.19:0/4056989879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcec15dac0 0x7fbcec0d19f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0d31b0 tx=0x7fbce802a290 comp rx=0 tx=0).stop 2024-04-25T09:55:41.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.046+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/4056989879 shutdown_connections 2024-04-25T09:55:41.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.046+0000 7fbcf3c1f640 1 --2- 172.21.15.19:0/4056989879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcec15dac0 0x7fbcec0d19f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.046+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/4056989879 >> 172.21.15.19:0/4056989879 conn(0x7fbcec061c30 msgr2=0x7fbcec06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:41.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.046+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/4056989879 shutdown_connections 2024-04-25T09:55:41.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.046+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/4056989879 wait complete. 2024-04-25T09:55:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.047+0000 7fbcf3c1f640 1 Processor -- start 2024-04-25T09:55:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.047+0000 7fbcf3c1f640 1 -- start start 2024-04-25T09:55:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf3c1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcec0d0ab0 0x7fbcec153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf3c1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcec0d1f30 0x7fbcec160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf3c1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcec15dac0 0x7fbcec1f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf3c1f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0b97d0 con 0x7fbcec15dac0 2024-04-25T09:55:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf3c1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec15dea0 con 0x7fbcec0d0ab0 2024-04-25T09:55:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf3c1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0b9950 con 0x7fbcec0d1f30 2024-04-25T09:55:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf21bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcec15dac0 0x7fbcec1f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf21bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcec15dac0 0x7fbcec1f3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39500/0 (socket says 172.21.15.19:39500) 2024-04-25T09:55:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf21bc640 1 -- 172.21.15.19:0/3544966871 learned_addr learned my addr 172.21.15.19:0/3544966871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf19bb640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcec0d0ab0 0x7fbcec153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf11ba640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcec0d1f30 0x7fbcec160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf21bc640 1 -- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcec0d1f30 msgr2=0x7fbcec160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf21bc640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcec0d1f30 0x7fbcec160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf21bc640 1 -- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcec0d0ab0 msgr2=0x7fbcec153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf21bc640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcec0d0ab0 0x7fbcec153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf21bc640 1 -- 172.21.15.19:0/3544966871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec15f5e0 con 0x7fbcec15dac0 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbcf21bc640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcec15dac0 0x7fbcec1f3270 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbce400cbb0 tx=0x7fbce400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.048+0000 7fbce27fc640 1 -- 172.21.15.19:0/3544966871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4017070 con 0x7fbcec15dac0 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.049+0000 7fbce27fc640 1 -- 172.21.15.19:0/3544966871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce400ad40 con 0x7fbcec15dac0 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.049+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/3544966871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec15f8c0 con 0x7fbcec15dac0 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.049+0000 7fbce27fc640 1 -- 172.21.15.19:0/3544966871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4012600 con 0x7fbcec15dac0 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.049+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/3544966871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec15fe80 con 0x7fbcec15dac0 2024-04-25T09:55:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.049+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/3544966871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec073a00 con 0x7fbcec15dac0 2024-04-25T09:55:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.050+0000 7fbce27fc640 1 -- 172.21.15.19:0/3544966871 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbce401a070 con 0x7fbcec15dac0 2024-04-25T09:55:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.050+0000 7fbce27fc640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbcd006b4d0 0x7fbcd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.051+0000 7fbce27fc640 1 -- 172.21.15.19:0/3544966871 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fbce408a0a0 con 0x7fbcec15dac0 2024-04-25T09:55:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.051+0000 7fbcf19bb640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbcd006b4d0 0x7fbcd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.051+0000 7fbcf19bb640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbcd006b4d0 0x7fbcd006d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbce8034000 tx=0x7fbce802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.055+0000 7fbce27fc640 1 -- 172.21.15.19:0/3544966871 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbce4058760 con 0x7fbcec15dac0 2024-04-25T09:55:41.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.280+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/3544966871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbcec0ca010 con 0x7fbcec15dac0 2024-04-25T09:55:41.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.926+0000 7fbce27fc640 1 -- 172.21.15.19:0/3544966871 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbce405c800 con 0x7fbcec15dac0 2024-04-25T09:55:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbcd006b4d0 msgr2=0x7fbcd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbcd006b4d0 0x7fbcd006d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbce8034000 tx=0x7fbce802e000 comp rx=0 tx=0).stop 2024-04-25T09:55:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/3544966871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcec15dac0 msgr2=0x7fbcec1f3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcec15dac0 0x7fbcec1f3270 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbce400cbb0 tx=0x7fbce400a430 comp rx=0 tx=0).stop 2024-04-25T09:55:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/3544966871 shutdown_connections 2024-04-25T09:55:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbcd006b4d0 0x7fbcd006d990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcec15dac0 0x7fbcec1f3270 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcec0d1f30 0x7fbcec160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 --2- 172.21.15.19:0/3544966871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcec0d0ab0 0x7fbcec153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/3544966871 >> 172.21.15.19:0/3544966871 conn(0x7fbcec061c30 msgr2=0x7fbcec0b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/3544966871 shutdown_connections 2024-04-25T09:55:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:41.929+0000 7fbcf3c1f640 1 -- 172.21.15.19:0/3544966871 wait complete. 2024-04-25T09:55:41.947 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:55:42.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.096+0000 7f99917cf640 1 Processor -- start 2024-04-25T09:55:42.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.096+0000 7f99917cf640 1 -- start start 2024-04-25T09:55:42.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.097+0000 7f99917cf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 0x7f998c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:42.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.097+0000 7f99917cf640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998c15e000 con 0x7f998c0ced80 2024-04-25T09:55:42.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.097+0000 7f99917cf640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998c0d1de0 con 0x7f998c15dc20 2024-04-25T09:55:42.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.097+0000 7f99917cf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c161c60 con 0x7f998c161710 2024-04-25T09:55:42.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.097+0000 7f998a7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f998c0ced80 0x7f998c0d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42530/0 (socket says 172.21.15.19:42530) 2024-04-25T09:55:42.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.097+0000 7f998a7fc640 1 -- 172.21.15.19:0/2854869968 learned_addr learned my addr 172.21.15.19:0/2854869968 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:42.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.097+0000 7f998b7fe640 1 --2- 172.21.15.19:0/2854869968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 0x7f998c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:42.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.097+0000 7f9989ffb640 1 -- 172.21.15.19:0/2854869968 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084746590 0 0) 0x7f998c15e000 con 0x7f998c0ced80 2024-04-25T09:55:42.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.097+0000 7f9989ffb640 1 -- 172.21.15.19:0/2854869968 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9974003520 con 0x7f998c0ced80 2024-04-25T09:55:42.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.097+0000 7f9989ffb640 1 -- 172.21.15.19:0/2854869968 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637849839 0 0) 0x7f998c0d1de0 con 0x7f998c15dc20 2024-04-25T09:55:42.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.097+0000 7f9989ffb640 1 -- 172.21.15.19:0/2854869968 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f998c15e000 con 0x7f998c15dc20 2024-04-25T09:55:42.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.098+0000 7f998b7fe640 1 -- 172.21.15.19:0/2854869968 >> v1:172.21.15.112:6789/0 conn(0x7f998c15dc20 legacy=0x7f998c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.098+0000 7f998b7fe640 1 -- 172.21.15.19:0/2854869968 >> v1:172.21.15.19:6789/0 conn(0x7f998c0ced80 legacy=0x7f998c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.098+0000 7f998b7fe640 1 -- 172.21.15.19:0/2854869968 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c161f40 con 0x7f998c161710 2024-04-25T09:55:42.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.098+0000 7f998b7fe640 1 --2- 172.21.15.19:0/2854869968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 0x7f998c0d29f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9984009dc0 tx=0x7f998402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a1970de9023832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.099+0000 7f9989ffb640 1 -- 172.21.15.19:0/2854869968 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984007740 con 0x7f998c161710 2024-04-25T09:55:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.099+0000 7f9989ffb640 1 -- 172.21.15.19:0/2854869968 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99840078e0 con 0x7f998c161710 2024-04-25T09:55:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.099+0000 7f9989ffb640 1 -- 172.21.15.19:0/2854869968 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984007cb0 con 0x7f998c161710 2024-04-25T09:55:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.099+0000 7f99917cf640 1 -- 172.21.15.19:0/2854869968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 msgr2=0x7f998c0d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.099+0000 7f99917cf640 1 --2- 172.21.15.19:0/2854869968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 0x7f998c0d29f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9984009dc0 tx=0x7f998402a6d0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.099+0000 7f99917cf640 1 -- 172.21.15.19:0/2854869968 shutdown_connections 2024-04-25T09:55:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.099+0000 7f99917cf640 1 --2- 172.21.15.19:0/2854869968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 0x7f998c0d29f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.099+0000 7f99917cf640 1 -- 172.21.15.19:0/2854869968 >> 172.21.15.19:0/2854869968 conn(0x7f998c0d0420 msgr2=0x7f998c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.099+0000 7f99917cf640 1 -- 172.21.15.19:0/2854869968 shutdown_connections 2024-04-25T09:55:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.099+0000 7f99917cf640 1 -- 172.21.15.19:0/2854869968 wait complete. 2024-04-25T09:55:42.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.101+0000 7f99917cf640 1 Processor -- start 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.101+0000 7f99917cf640 1 -- start start 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.101+0000 7f99917cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998c0ced80 0x7f998c15fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.101+0000 7f99917cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f998c15dc20 0x7f998c0b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.101+0000 7f99917cf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 0x7f998c0b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.101+0000 7f99917cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c0b5e50 con 0x7f998c0ced80 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.101+0000 7f99917cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c162500 con 0x7f998c15dc20 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.101+0000 7f99917cf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c0b5fd0 con 0x7f998c161710 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.101+0000 7f998affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998c0ced80 0x7f998c15fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998c0ced80 0x7f998c15fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39506/0 (socket says 172.21.15.19:39506) 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998affd640 1 -- 172.21.15.19:0/3712458571 learned_addr learned my addr 172.21.15.19:0/3712458571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998a7fc640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f998c15dc20 0x7f998c0b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998b7fe640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 0x7f998c0b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998affd640 1 -- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 msgr2=0x7f998c0b80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998affd640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 0x7f998c0b80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998affd640 1 -- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f998c15dc20 msgr2=0x7f998c0b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998affd640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f998c15dc20 0x7f998c0b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998affd640 1 -- 172.21.15.19:0/3712458571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c0cc1e0 con 0x7f998c0ced80 2024-04-25T09:55:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998b7fe640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 0x7f998c0b80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998a7fc640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f998c15dc20 0x7f998c0b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f998affd640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998c0ced80 0x7f998c15fea0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f997c00c500 tx=0x7f997c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f99737fe640 1 -- 172.21.15.19:0/3712458571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c0052d0 con 0x7f998c0ced80 2024-04-25T09:55:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f99737fe640 1 -- 172.21.15.19:0/3712458571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997c005490 con 0x7f998c0ced80 2024-04-25T09:55:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f99737fe640 1 -- 172.21.15.19:0/3712458571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c012870 con 0x7f998c0ced80 2024-04-25T09:55:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f99917cf640 1 -- 172.21.15.19:0/3712458571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998c1605a0 con 0x7f998c0ced80 2024-04-25T09:55:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.102+0000 7f99917cf640 1 -- 172.21.15.19:0/3712458571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c160b60 con 0x7f998c0ced80 2024-04-25T09:55:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.103+0000 7f99917cf640 1 -- 172.21.15.19:0/3712458571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998c0ca730 con 0x7f998c0ced80 2024-04-25T09:55:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.104+0000 7f99737fe640 1 -- 172.21.15.19:0/3712458571 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f997c012a10 con 0x7f998c0ced80 2024-04-25T09:55:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.104+0000 7f99737fe640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997806b500 0x7f997806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.104+0000 7f99737fe640 1 -- 172.21.15.19:0/3712458571 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f997c08a1e0 con 0x7f998c0ced80 2024-04-25T09:55:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.104+0000 7f998a7fc640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997806b500 0x7f997806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.105+0000 7f998a7fc640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997806b500 0x7f997806d9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f998c0d49c0 tx=0x7f9980009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.108+0000 7f99737fe640 1 -- 172.21.15.19:0/3712458571 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f998c0ca730 con 0x7f998c0ced80 2024-04-25T09:55:42.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.289+0000 7f99917cf640 1 -- 172.21.15.19:0/3712458571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f998c0c70d0 con 0x7f998c0ced80 2024-04-25T09:55:42.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.932+0000 7f99737fe640 1 -- 172.21.15.19:0/3712458571 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7f998c0c70d0 con 0x7f998c0ced80 2024-04-25T09:55:42.935 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:55:42.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 -- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997806b500 msgr2=0x7f997806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997806b500 0x7f997806d9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f998c0d49c0 tx=0x7f9980009040 comp rx=0 tx=0).stop 2024-04-25T09:55:42.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 -- 172.21.15.19:0/3712458571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998c0ced80 msgr2=0x7f998c15fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998c0ced80 0x7f998c15fea0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f997c00c500 tx=0x7f997c00c9c0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 -- 172.21.15.19:0/3712458571 shutdown_connections 2024-04-25T09:55:42.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997806b500 0x7f997806d9c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998c161710 0x7f998c0b80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f998c15dc20 0x7f998c0b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 --2- 172.21.15.19:0/3712458571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f998c0ced80 0x7f998c15fea0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 -- 172.21.15.19:0/3712458571 >> 172.21.15.19:0/3712458571 conn(0x7f998c0d0420 msgr2=0x7f998c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:42.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 -- 172.21.15.19:0/3712458571 shutdown_connections 2024-04-25T09:55:42.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:42.935+0000 7f99917cf640 1 -- 172.21.15.19:0/3712458571 wait complete. 2024-04-25T09:55:42.954 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:55:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.063+0000 7f3425691640 1 Processor -- start 2024-04-25T09:55:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.063+0000 7f3425691640 1 -- start start 2024-04-25T09:55:43.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f3425691640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f342015b7d0 0x7f34200cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f3425691640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f34200b5620 0x7f34200b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f3425691640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342006c2d0 0x7f342006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f3425691640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342006cc80 con 0x7f342015b7d0 2024-04-25T09:55:43.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f3425691640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342006cf50 con 0x7f34200b5620 2024-04-25T09:55:43.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f3425691640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34200ce530 con 0x7f342006c2d0 2024-04-25T09:55:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f341effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f342015b7d0 0x7f34200cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f341effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f342015b7d0 0x7f34200cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39522/0 (socket says 172.21.15.19:39522) 2024-04-25T09:55:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f341effd640 1 -- 172.21.15.19:0/2927884904 learned_addr learned my addr 172.21.15.19:0/2927884904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f341f7fe640 1 --2- 172.21.15.19:0/2927884904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342006c2d0 0x7f342006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f3417fff640 1 --2- 172.21.15.19:0/2927884904 >> v2:172.21.15.112:3300/0 conn(0x7f34200b5620 0x7f34200b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f341effd640 1 -- 172.21.15.19:0/2927884904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342006c2d0 msgr2=0x7f342006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f341effd640 1 --2- 172.21.15.19:0/2927884904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342006c2d0 0x7f342006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f341effd640 1 -- 172.21.15.19:0/2927884904 >> v2:172.21.15.112:3300/0 conn(0x7f34200b5620 msgr2=0x7f34200b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f341effd640 1 --2- 172.21.15.19:0/2927884904 >> v2:172.21.15.112:3300/0 conn(0x7f34200b5620 0x7f34200b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f341effd640 1 -- 172.21.15.19:0/2927884904 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34200ce810 con 0x7f342015b7d0 2024-04-25T09:55:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.064+0000 7f3417fff640 1 --2- 172.21.15.19:0/2927884904 >> v2:172.21.15.112:3300/0 conn(0x7f34200b5620 0x7f34200b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.065+0000 7f341effd640 1 --2- 172.21.15.19:0/2927884904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342015b7d0 0x7f34200cf010 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f34200ce180 tx=0x7f341002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76a6360159e36ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.065+0000 7f341e7fc640 1 -- 172.21.15.19:0/2927884904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410037070 con 0x7f342015b7d0 2024-04-25T09:55:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.065+0000 7f341e7fc640 1 -- 172.21.15.19:0/2927884904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3410032440 con 0x7f342015b7d0 2024-04-25T09:55:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.065+0000 7f341e7fc640 1 -- 172.21.15.19:0/2927884904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410032810 con 0x7f342015b7d0 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.066+0000 7f3425691640 1 -- 172.21.15.19:0/2927884904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342015b7d0 msgr2=0x7f34200cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.066+0000 7f3425691640 1 --2- 172.21.15.19:0/2927884904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342015b7d0 0x7f34200cf010 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f34200ce180 tx=0x7f341002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.066+0000 7f3425691640 1 -- 172.21.15.19:0/2927884904 shutdown_connections 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.066+0000 7f3425691640 1 --2- 172.21.15.19:0/2927884904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342006c2d0 0x7f342006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.066+0000 7f3425691640 1 --2- 172.21.15.19:0/2927884904 >> v2:172.21.15.112:3300/0 conn(0x7f34200b5620 0x7f34200b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.066+0000 7f3425691640 1 --2- 172.21.15.19:0/2927884904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342015b7d0 0x7f34200cf010 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.066+0000 7f3425691640 1 -- 172.21.15.19:0/2927884904 >> 172.21.15.19:0/2927884904 conn(0x7f34200d0800 msgr2=0x7f34200d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.066+0000 7f3425691640 1 -- 172.21.15.19:0/2927884904 shutdown_connections 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.066+0000 7f3425691640 1 -- 172.21.15.19:0/2927884904 wait complete. 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.067+0000 7f3425691640 1 Processor -- start 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.067+0000 7f3425691640 1 -- start start 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f3425691640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342006c2d0 0x7f34200b9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f3425691640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34200b5620 0x7f342014cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f3425691640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342015b7d0 0x7f34200c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f3425691640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34200cf550 con 0x7f342006c2d0 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f3425691640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342006d230 con 0x7f34200b5620 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f3425691640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34200cf6d0 con 0x7f342015b7d0 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f341effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342006c2d0 0x7f34200b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f341effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342006c2d0 0x7f34200b9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39526/0 (socket says 172.21.15.19:39526) 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f341effd640 1 -- 172.21.15.19:0/3609577940 learned_addr learned my addr 172.21.15.19:0/3609577940 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f341f7fe640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342015b7d0 0x7f34200c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f3417fff640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34200b5620 0x7f342014cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f341effd640 1 -- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342015b7d0 msgr2=0x7f34200c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f341effd640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342015b7d0 0x7f34200c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f341effd640 1 -- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34200b5620 msgr2=0x7f342014cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f341effd640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34200b5620 0x7f342014cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f341effd640 1 -- 172.21.15.19:0/3609577940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34200b7f70 con 0x7f342006c2d0 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f3417fff640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34200b5620 0x7f342014cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f341f7fe640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342015b7d0 0x7f34200c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f341effd640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342006c2d0 0x7f34200b9010 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f341002a220 tx=0x7f34100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f34177fe640 1 -- 172.21.15.19:0/3609577940 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410041020 con 0x7f342006c2d0 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f34177fe640 1 -- 172.21.15.19:0/3609577940 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34100306c0 con 0x7f342006c2d0 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.068+0000 7f3425691640 1 -- 172.21.15.19:0/3609577940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34200b8250 con 0x7f342006c2d0 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.069+0000 7f34177fe640 1 -- 172.21.15.19:0/3609577940 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410037040 con 0x7f342006c2d0 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.069+0000 7f3425691640 1 -- 172.21.15.19:0/3609577940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34200b8810 con 0x7f342006c2d0 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.070+0000 7f3425691640 1 -- 172.21.15.19:0/3609577940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3420073a00 con 0x7f342006c2d0 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.070+0000 7f34177fe640 1 -- 172.21.15.19:0/3609577940 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3410044020 con 0x7f342006c2d0 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.071+0000 7f34177fe640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 0x7f340006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.071+0000 7f34177fe640 1 -- 172.21.15.19:0/3609577940 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f34100aa9c0 con 0x7f342006c2d0 2024-04-25T09:55:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.071+0000 7f3417fff640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 0x7f340006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.071+0000 7f3417fff640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 0x7f340006d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3408006fd0 tx=0x7f3408008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:43.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.075+0000 7f34177fe640 1 -- 172.21.15.19:0/3609577940 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34100ace20 con 0x7f342006c2d0 2024-04-25T09:55:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.259+0000 7f3425691640 1 -- 172.21.15.19:0/3609577940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34200c7300 con 0x7f342006c2d0 2024-04-25T09:55:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.938+0000 7f34177fe640 1 -- 172.21.15.19:0/3609577940 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7f341004b030 con 0x7f342006c2d0 2024-04-25T09:55:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T09:55:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.942+0000 7f3425691640 1 -- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 msgr2=0x7f340006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.942+0000 7f3425691640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 0x7f340006d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3408006fd0 tx=0x7f3408008040 comp rx=0 tx=0).stop 2024-04-25T09:55:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.942+0000 7f3425691640 1 -- 172.21.15.19:0/3609577940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342006c2d0 msgr2=0x7f34200b9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.942+0000 7f3425691640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342006c2d0 0x7f34200b9010 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f341002a220 tx=0x7f34100046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.943+0000 7f3425691640 1 -- 172.21.15.19:0/3609577940 shutdown_connections 2024-04-25T09:55:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.943+0000 7f3425691640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 0x7f340006d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.943+0000 7f3425691640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342015b7d0 0x7f34200c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.943+0000 7f3425691640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34200b5620 0x7f342014cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.943+0000 7f3425691640 1 --2- 172.21.15.19:0/3609577940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342006c2d0 0x7f34200b9010 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.943+0000 7f3425691640 1 -- 172.21.15.19:0/3609577940 >> 172.21.15.19:0/3609577940 conn(0x7f34200d0800 msgr2=0x7f34200b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.943+0000 7f3425691640 1 -- 172.21.15.19:0/3609577940 shutdown_connections 2024-04-25T09:55:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:43.944+0000 7f3425691640 1 -- 172.21.15.19:0/3609577940 wait complete. 2024-04-25T09:55:43.962 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:55:44.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.068+0000 7fab6d3e2640 1 Processor -- start 2024-04-25T09:55:44.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.069+0000 7fab6d3e2640 1 -- start start 2024-04-25T09:55:44.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.069+0000 7fab6d3e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab680b5620 0x7fab680b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.069+0000 7fab6d3e2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fab6806c2d0 0x7fab6806c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.069+0000 7fab6d3e2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6806cca0 con 0x7fab6815b7d0 2024-04-25T09:55:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.069+0000 7fab6d3e2640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab680b5fc0 con 0x7fab6806c2d0 2024-04-25T09:55:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.069+0000 7fab6d3e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6806cf50 con 0x7fab680b5620 2024-04-25T09:55:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.069+0000 7fab677fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fab6806c2d0 0x7fab6806c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.070+0000 7fab677fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fab6806c2d0 0x7fab6806c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39536/0 (socket says 172.21.15.19:39536) 2024-04-25T09:55:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.070+0000 7fab677fe640 1 -- 172.21.15.19:0/2755405767 learned_addr learned my addr 172.21.15.19:0/2755405767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.070+0000 7fab667fc640 1 --2- 172.21.15.19:0/2755405767 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab680b5620 0x7fab680b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.070+0000 7fab677fe640 1 -- 172.21.15.19:0/2755405767 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab680b5620 msgr2=0x7fab680b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.070+0000 7fab677fe640 1 --2- 172.21.15.19:0/2755405767 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab680b5620 0x7fab680b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.070+0000 7fab677fe640 1 -- 172.21.15.19:0/2755405767 >> v1:172.21.15.112:6789/0 conn(0x7fab6815b7d0 legacy=0x7fab680cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:44.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.070+0000 7fab677fe640 1 -- 172.21.15.19:0/2755405767 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab680ce530 con 0x7fab6806c2d0 2024-04-25T09:55:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab677fe640 1 --2- 172.21.15.19:0/2755405767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab6806c2d0 0x7fab6806c760 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fab58009e50 tx=0x7fab5802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b92e0f001e547 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab65ffb640 1 -- 172.21.15.19:0/2755405767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58036070 con 0x7fab6806c2d0 2024-04-25T09:55:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab65ffb640 1 -- 172.21.15.19:0/2755405767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab58031600 con 0x7fab6806c2d0 2024-04-25T09:55:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab65ffb640 1 -- 172.21.15.19:0/2755405767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab580319d0 con 0x7fab6806c2d0 2024-04-25T09:55:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2755405767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab6806c2d0 msgr2=0x7fab6806c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab6d3e2640 1 --2- 172.21.15.19:0/2755405767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab6806c2d0 0x7fab6806c760 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fab58009e50 tx=0x7fab5802a6c0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2755405767 shutdown_connections 2024-04-25T09:55:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab6d3e2640 1 --2- 172.21.15.19:0/2755405767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab6806c2d0 0x7fab6806c760 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab6d3e2640 1 --2- 172.21.15.19:0/2755405767 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab680b5620 0x7fab680b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2755405767 >> 172.21.15.19:0/2755405767 conn(0x7fab680d0800 msgr2=0x7fab680d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2755405767 shutdown_connections 2024-04-25T09:55:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.071+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2755405767 wait complete. 2024-04-25T09:55:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.073+0000 7fab6d3e2640 1 Processor -- start 2024-04-25T09:55:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.073+0000 7fab6d3e2640 1 -- start start 2024-04-25T09:55:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.073+0000 7fab6d3e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab6806c2d0 0x7fab68149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.073+0000 7fab6d3e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab680b5620 0x7fab680cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.073+0000 7fab6d3e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab6815b7d0 0x7fab680cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.073+0000 7fab6d3e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6806d230 con 0x7fab680b5620 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.073+0000 7fab6d3e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab680ceaf0 con 0x7fab6806c2d0 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.073+0000 7fab6d3e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6815bbb0 con 0x7fab6815b7d0 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.073+0000 7fab667fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab680b5620 0x7fab680cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.073+0000 7fab667fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab680b5620 0x7fab680cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39550/0 (socket says 172.21.15.19:39550) 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.073+0000 7fab667fc640 1 -- 172.21.15.19:0/2282453855 learned_addr learned my addr 172.21.15.19:0/2282453855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab66ffd640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab6806c2d0 0x7fab68149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab677fe640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab6815b7d0 0x7fab680cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab667fc640 1 -- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab6815b7d0 msgr2=0x7fab680cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab667fc640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab6815b7d0 0x7fab680cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab667fc640 1 -- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab6806c2d0 msgr2=0x7fab68149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab667fc640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab6806c2d0 0x7fab68149600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab667fc640 1 -- 172.21.15.19:0/2282453855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab68149d50 con 0x7fab680b5620 2024-04-25T09:55:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab667fc640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab680b5620 0x7fab680cd5d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c7f0 tx=0x7fab5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab4f7fe640 1 -- 172.21.15.19:0/2282453855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c015070 con 0x7fab680b5620 2024-04-25T09:55:44.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab4f7fe640 1 -- 172.21.15.19:0/2282453855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c004240 con 0x7fab680b5620 2024-04-25T09:55:44.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2282453855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6814a030 con 0x7fab680b5620 2024-04-25T09:55:44.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab4f7fe640 1 -- 172.21.15.19:0/2282453855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c007620 con 0x7fab680b5620 2024-04-25T09:55:44.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.074+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2282453855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6814a5f0 con 0x7fab680b5620 2024-04-25T09:55:44.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.075+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2282453855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab68073a00 con 0x7fab680b5620 2024-04-25T09:55:44.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.076+0000 7fab4f7fe640 1 -- 172.21.15.19:0/2282453855 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab5c0079d0 con 0x7fab680b5620 2024-04-25T09:55:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.076+0000 7fab4f7fe640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab5406b4d0 0x7fab5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.076+0000 7fab4f7fe640 1 -- 172.21.15.19:0/2282453855 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fab5c08a0d0 con 0x7fab680b5620 2024-04-25T09:55:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.076+0000 7fab66ffd640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab5406b4d0 0x7fab5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.077+0000 7fab66ffd640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab5406b4d0 0x7fab5406d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fab60004f60 tx=0x7fab60007e30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.081+0000 7fab4f7fe640 1 -- 172.21.15.19:0/2282453855 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab5c0589f0 con 0x7fab680b5620 2024-04-25T09:55:44.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.259+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2282453855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fab6806b210 con 0x7fab680b5620 2024-04-25T09:55:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.262+0000 7fab4f7fe640 1 -- 172.21.15.19:0/2282453855 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fab5c05ca90 con 0x7fab680b5620 2024-04-25T09:55:44.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.264+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab5406b4d0 msgr2=0x7fab5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.264+0000 7fab6d3e2640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab5406b4d0 0x7fab5406d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fab60004f60 tx=0x7fab60007e30 comp rx=0 tx=0).stop 2024-04-25T09:55:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.264+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2282453855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab680b5620 msgr2=0x7fab680cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.264+0000 7fab6d3e2640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab680b5620 0x7fab680cd5d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c7f0 tx=0x7fab5c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.264+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2282453855 shutdown_connections 2024-04-25T09:55:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.264+0000 7fab6d3e2640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab5406b4d0 0x7fab5406d990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.264+0000 7fab6d3e2640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab6815b7d0 0x7fab680cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.264+0000 7fab6d3e2640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab680b5620 0x7fab680cd5d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.265+0000 7fab6d3e2640 1 --2- 172.21.15.19:0/2282453855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab6806c2d0 0x7fab68149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.265+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2282453855 >> 172.21.15.19:0/2282453855 conn(0x7fab680d0800 msgr2=0x7fab6815b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.265+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2282453855 shutdown_connections 2024-04-25T09:55:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.265+0000 7fab6d3e2640 1 -- 172.21.15.19:0/2282453855 wait complete. 2024-04-25T09:55:44.279 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T09:55:44.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.382+0000 7f89b57e0640 1 Processor -- start 2024-04-25T09:55:44.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.382+0000 7f89b57e0640 1 -- start start 2024-04-25T09:55:44.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.383+0000 7f89b57e0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f89b015dbe0 0x7f89b00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.383+0000 7f89b57e0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f89b00b92a0 0x7f89b00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.383+0000 7f89b57e0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b00b97a0 con 0x7f89b00b2cb0 2024-04-25T09:55:44.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.383+0000 7f89b57e0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b015dfc0 con 0x7f89b00b92a0 2024-04-25T09:55:44.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.383+0000 7f89b57e0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b00d1a00 con 0x7f89b015dbe0 2024-04-25T09:55:44.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.383+0000 7f89af7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f89b00b92a0 0x7f89b00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.383+0000 7f89af7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f89b00b92a0 0x7f89b00d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39564/0 (socket says 172.21.15.19:39564) 2024-04-25T09:55:44.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.383+0000 7f89af7fe640 1 -- 172.21.15.19:0/2269461267 learned_addr learned my addr 172.21.15.19:0/2269461267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:44.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.384+0000 7f89aeffd640 1 --2- 172.21.15.19:0/2269461267 >> v2:172.21.15.112:3300/0 conn(0x7f89b015dbe0 0x7f89b00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.384+0000 7f89af7fe640 1 -- 172.21.15.19:0/2269461267 >> v2:172.21.15.112:3300/0 conn(0x7f89b015dbe0 msgr2=0x7f89b00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.384+0000 7f89af7fe640 1 --2- 172.21.15.19:0/2269461267 >> v2:172.21.15.112:3300/0 conn(0x7f89b015dbe0 0x7f89b00d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.384+0000 7f89af7fe640 1 -- 172.21.15.19:0/2269461267 >> v1:172.21.15.112:6789/0 conn(0x7f89b00b2cb0 legacy=0x7f89b00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.384+0000 7f89af7fe640 1 -- 172.21.15.19:0/2269461267 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b00d1c50 con 0x7f89b00b92a0 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.384+0000 7f89af7fe640 1 --2- 172.21.15.19:0/2269461267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b00b92a0 0x7f89b00d0e50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f89b006c140 tx=0x7f89a002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ee172c36af3402 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.385+0000 7f89adffb640 1 -- 172.21.15.19:0/2269461267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0036070 con 0x7f89b00b92a0 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.385+0000 7f89adffb640 1 -- 172.21.15.19:0/2269461267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a0031600 con 0x7f89b00b92a0 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.385+0000 7f89adffb640 1 -- 172.21.15.19:0/2269461267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a00319d0 con 0x7f89b00b92a0 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.385+0000 7f89b57e0640 1 -- 172.21.15.19:0/2269461267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b00b92a0 msgr2=0x7f89b00d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.385+0000 7f89b57e0640 1 --2- 172.21.15.19:0/2269461267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b00b92a0 0x7f89b00d0e50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f89b006c140 tx=0x7f89a002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.385+0000 7f89b57e0640 1 -- 172.21.15.19:0/2269461267 shutdown_connections 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.385+0000 7f89b57e0640 1 --2- 172.21.15.19:0/2269461267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b00b92a0 0x7f89b00d0e50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.385+0000 7f89b57e0640 1 --2- 172.21.15.19:0/2269461267 >> v2:172.21.15.112:3300/0 conn(0x7f89b015dbe0 0x7f89b00d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.385+0000 7f89b57e0640 1 -- 172.21.15.19:0/2269461267 >> 172.21.15.19:0/2269461267 conn(0x7f89b0061a20 msgr2=0x7f89b0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:44.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.385+0000 7f89b57e0640 1 -- 172.21.15.19:0/2269461267 shutdown_connections 2024-04-25T09:55:44.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.385+0000 7f89b57e0640 1 -- 172.21.15.19:0/2269461267 wait complete. 2024-04-25T09:55:44.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.387+0000 7f89b57e0640 1 Processor -- start 2024-04-25T09:55:44.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.387+0000 7f89b57e0640 1 -- start start 2024-04-25T09:55:44.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.387+0000 7f89b57e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89b00b2cb0 0x7f89b00cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.387+0000 7f89b57e0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89b00b92a0 0x7f89b00cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.387+0000 7f89b57e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b015dbe0 0x7f89b00ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.387+0000 7f89b57e0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00b6ab0 con 0x7f89b015dbe0 2024-04-25T09:55:44.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.387+0000 7f89b57e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00d2210 con 0x7f89b00b2cb0 2024-04-25T09:55:44.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.387+0000 7f89b57e0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00b6c30 con 0x7f89b00b92a0 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.387+0000 7f89af7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b015dbe0 0x7f89b00ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.387+0000 7f89af7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b015dbe0 0x7f89b00ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39568/0 (socket says 172.21.15.19:39568) 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.387+0000 7f89af7fe640 1 -- 172.21.15.19:0/75612150 learned_addr learned my addr 172.21.15.19:0/75612150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89aeffd640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89b00b2cb0 0x7f89b00cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89ae7fc640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89b00b92a0 0x7f89b00cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89af7fe640 1 -- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89b00b92a0 msgr2=0x7f89b00cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89af7fe640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89b00b92a0 0x7f89b00cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89af7fe640 1 -- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89b00b2cb0 msgr2=0x7f89b00cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89af7fe640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89b00b2cb0 0x7f89b00cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89af7fe640 1 -- 172.21.15.19:0/75612150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b00cd5d0 con 0x7f89b015dbe0 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89ae7fc640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89b00b92a0 0x7f89b00cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:44.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89af7fe640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b015dbe0 0x7f89b00ccfc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009f40 tx=0x7f89a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89a77fe640 1 -- 172.21.15.19:0/75612150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0036050 con 0x7f89b015dbe0 2024-04-25T09:55:44.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89a77fe640 1 -- 172.21.15.19:0/75612150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a0042070 con 0x7f89b015dbe0 2024-04-25T09:55:44.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89b57e0640 1 -- 172.21.15.19:0/75612150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b00cd8b0 con 0x7f89b015dbe0 2024-04-25T09:55:44.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89a77fe640 1 -- 172.21.15.19:0/75612150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0009510 con 0x7f89b015dbe0 2024-04-25T09:55:44.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.388+0000 7f89b57e0640 1 -- 172.21.15.19:0/75612150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b0156580 con 0x7f89b015dbe0 2024-04-25T09:55:44.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.389+0000 7f89b57e0640 1 -- 172.21.15.19:0/75612150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b0073990 con 0x7f89b015dbe0 2024-04-25T09:55:44.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.390+0000 7f89a77fe640 1 -- 172.21.15.19:0/75612150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89a0007c70 con 0x7f89b015dbe0 2024-04-25T09:55:44.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.390+0000 7f89a77fe640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f899006b4d0 0x7f899006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.390+0000 7f89a77fe640 1 -- 172.21.15.19:0/75612150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f89a00aadf0 con 0x7f89b015dbe0 2024-04-25T09:55:44.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.390+0000 7f89aeffd640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f899006b4d0 0x7f899006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.391+0000 7f89aeffd640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f899006b4d0 0x7f899006d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009aa0 tx=0x7f89a8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.394+0000 7f89a77fe640 1 -- 172.21.15.19:0/75612150 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89a0079e70 con 0x7f89b015dbe0 2024-04-25T09:55:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.644+0000 7f89b57e0640 1 -- 172.21.15.19:0/75612150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f89b00b7540 con 0x7f89b015dbe0 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.939+0000 7f89a77fe640 1 -- 172.21.15.19:0/75612150 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v142) v1 ==== 148+0+0 (secure 0 0 0) 0x7f89a0079770 con 0x7f89b015dbe0 2024-04-25T09:55:44.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 -- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f899006b4d0 msgr2=0x7f899006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f899006b4d0 0x7f899006d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009aa0 tx=0x7f89a8009190 comp rx=0 tx=0).stop 2024-04-25T09:55:44.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 -- 172.21.15.19:0/75612150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b015dbe0 msgr2=0x7f89b00ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b015dbe0 0x7f89b00ccfc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009f40 tx=0x7f89a0004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:44.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 -- 172.21.15.19:0/75612150 shutdown_connections 2024-04-25T09:55:44.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f899006b4d0 0x7f899006d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b015dbe0 0x7f89b00ccfc0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89b00b92a0 0x7f89b00cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 --2- 172.21.15.19:0/75612150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89b00b2cb0 0x7f89b00cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 -- 172.21.15.19:0/75612150 >> 172.21.15.19:0/75612150 conn(0x7f89b0061a20 msgr2=0x7f89b0160160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:44.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 -- 172.21.15.19:0/75612150 shutdown_connections 2024-04-25T09:55:44.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:44.941+0000 7f89b57e0640 1 -- 172.21.15.19:0/75612150 wait complete. 2024-04-25T09:55:44.956 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T09:55:44.956 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:55:44.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:45.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.054+0000 7f463b676640 1 Processor -- start 2024-04-25T09:55:45.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.054+0000 7f463b676640 1 -- start start 2024-04-25T09:55:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.055+0000 7f463b676640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f463415ba00 0x7f463409a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.055+0000 7f463b676640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46341602e0 0x7f46340ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.055+0000 7f463b676640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463415f5f0 0x7f463415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.055+0000 7f463b676640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46340cd710 con 0x7f463415ba00 2024-04-25T09:55:45.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.055+0000 7f463b676640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4634099b70 con 0x7f46341602e0 2024-04-25T09:55:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.055+0000 7f463b676640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4634099e50 con 0x7f463415f5f0 2024-04-25T09:55:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.055+0000 7f4639412640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f463415ba00 0x7f463409a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.055+0000 7f4639412640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f463415ba00 0x7f463409a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39580/0 (socket says 172.21.15.19:39580) 2024-04-25T09:55:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.055+0000 7f4639412640 1 -- 172.21.15.19:0/1677470386 learned_addr learned my addr 172.21.15.19:0/1677470386 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.056+0000 7f4639412640 1 -- 172.21.15.19:0/1677470386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463415f5f0 msgr2=0x7f463415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.056+0000 7f4639c13640 1 --2- 172.21.15.19:0/1677470386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463415f5f0 0x7f463415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.056+0000 7f4638c11640 1 --2- 172.21.15.19:0/1677470386 >> v2:172.21.15.112:3300/0 conn(0x7f46341602e0 0x7f46340ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.056+0000 7f4639412640 1 --2- 172.21.15.19:0/1677470386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463415f5f0 0x7f463415fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.056+0000 7f4639412640 1 -- 172.21.15.19:0/1677470386 >> v2:172.21.15.112:3300/0 conn(0x7f46341602e0 msgr2=0x7f46340ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.056+0000 7f4639412640 1 --2- 172.21.15.19:0/1677470386 >> v2:172.21.15.112:3300/0 conn(0x7f46341602e0 0x7f46340ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.056+0000 7f4639412640 1 -- 172.21.15.19:0/1677470386 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463409a130 con 0x7f463415ba00 2024-04-25T09:55:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.056+0000 7f4639412640 1 --2- 172.21.15.19:0/1677470386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f463415ba00 0x7f463409a930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f46340b7830 tx=0x7f463002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1baf0b9e938c1703 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f462bfff640 1 -- 172.21.15.19:0/1677470386 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630037070 con 0x7f463415ba00 2024-04-25T09:55:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f462bfff640 1 -- 172.21.15.19:0/1677470386 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463002ae00 con 0x7f463415ba00 2024-04-25T09:55:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f462bfff640 1 -- 172.21.15.19:0/1677470386 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630032660 con 0x7f463415ba00 2024-04-25T09:55:45.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f463b676640 1 -- 172.21.15.19:0/1677470386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f463415ba00 msgr2=0x7f463409a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f463b676640 1 --2- 172.21.15.19:0/1677470386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f463415ba00 0x7f463409a930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f46340b7830 tx=0x7f463002a0b0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f463b676640 1 -- 172.21.15.19:0/1677470386 shutdown_connections 2024-04-25T09:55:45.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f463b676640 1 --2- 172.21.15.19:0/1677470386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463415f5f0 0x7f463415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f463b676640 1 --2- 172.21.15.19:0/1677470386 >> v2:172.21.15.112:3300/0 conn(0x7f46341602e0 0x7f46340ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f463b676640 1 --2- 172.21.15.19:0/1677470386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f463415ba00 0x7f463409a930 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f463b676640 1 -- 172.21.15.19:0/1677470386 >> 172.21.15.19:0/1677470386 conn(0x7f46340b3270 msgr2=0x7f46340b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:45.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f463b676640 1 -- 172.21.15.19:0/1677470386 shutdown_connections 2024-04-25T09:55:45.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.057+0000 7f463b676640 1 -- 172.21.15.19:0/1677470386 wait complete. 2024-04-25T09:55:45.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.058+0000 7f463b676640 1 Processor -- start 2024-04-25T09:55:45.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.058+0000 7f463b676640 1 -- start start 2024-04-25T09:55:45.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.059+0000 7f463b676640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463415ba00 0x7f46340ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.059+0000 7f463b676640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f463415f5f0 0x7f463414ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.059+0000 7f463b676640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46341602e0 0x7f463406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.059+0000 7f463b676640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463415bde0 con 0x7f46341602e0 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.059+0000 7f463b676640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340cd2e0 con 0x7f463415f5f0 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.059+0000 7f463b676640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463406dcd0 con 0x7f463415ba00 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.059+0000 7f4639c13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46341602e0 0x7f463406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.059+0000 7f4639c13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46341602e0 0x7f463406d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39586/0 (socket says 172.21.15.19:39586) 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.059+0000 7f4639c13640 1 -- 172.21.15.19:0/3566056548 learned_addr learned my addr 172.21.15.19:0/3566056548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.059+0000 7f4639412640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463415ba00 0x7f46340ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4638c11640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f463415f5f0 0x7f463414ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4639c13640 1 -- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463415ba00 msgr2=0x7f46340ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4639c13640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463415ba00 0x7f46340ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4639c13640 1 -- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f463415f5f0 msgr2=0x7f463414ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4639c13640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f463415f5f0 0x7f463414ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4639c13640 1 -- 172.21.15.19:0/3566056548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340cae90 con 0x7f46341602e0 2024-04-25T09:55:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4639412640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463415ba00 0x7f46340ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4638c11640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f463415f5f0 0x7f463414ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4639c13640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46341602e0 0x7f463406d790 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f462c00cb80 tx=0x7f462c00a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4629ffb640 1 -- 172.21.15.19:0/3566056548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c015070 con 0x7f46341602e0 2024-04-25T09:55:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4629ffb640 1 -- 172.21.15.19:0/3566056548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c00abd0 con 0x7f46341602e0 2024-04-25T09:55:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f463b676640 1 -- 172.21.15.19:0/3566056548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46340cb170 con 0x7f46341602e0 2024-04-25T09:55:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f4629ffb640 1 -- 172.21.15.19:0/3566056548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00d040 con 0x7f46341602e0 2024-04-25T09:55:45.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.060+0000 7f463b676640 1 -- 172.21.15.19:0/3566056548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46340cb730 con 0x7f46341602e0 2024-04-25T09:55:45.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.061+0000 7f463b676640 1 -- 172.21.15.19:0/3566056548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4634073990 con 0x7f46341602e0 2024-04-25T09:55:45.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.062+0000 7f4629ffb640 1 -- 172.21.15.19:0/3566056548 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f462c018070 con 0x7f46341602e0 2024-04-25T09:55:45.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.062+0000 7f4629ffb640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461806b4d0 0x7f461806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.062+0000 7f4629ffb640 1 -- 172.21.15.19:0/3566056548 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f462c089cc0 con 0x7f46341602e0 2024-04-25T09:55:45.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.062+0000 7f4639412640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461806b4d0 0x7f461806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.063+0000 7f4639412640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461806b4d0 0x7f461806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4630034000 tx=0x7f463002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.066+0000 7f4629ffb640 1 -- 172.21.15.19:0/3566056548 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f462c058cb0 con 0x7f46341602e0 2024-04-25T09:55:45.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.246+0000 7f463b676640 1 -- 172.21.15.19:0/3566056548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46340b40a0 con 0x7f46341602e0 2024-04-25T09:55:45.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.247+0000 7f4629ffb640 1 -- 172.21.15.19:0/3566056548 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+13281 (secure 0 0 0) 0x7f462c0585b0 con 0x7f46341602e0 2024-04-25T09:55:45.249 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:45.250 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":45,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:43.933392+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3017773710":"2024-04-26T09:55:40.924418+0000","172.21.15.19:6833/3017773710":"2024-04-26T09:55:40.924418+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:45.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 -- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461806b4d0 msgr2=0x7f461806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461806b4d0 0x7f461806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4630034000 tx=0x7f463002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:45.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 -- 172.21.15.19:0/3566056548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46341602e0 msgr2=0x7f463406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46341602e0 0x7f463406d790 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f462c00cb80 tx=0x7f462c00a4c0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 -- 172.21.15.19:0/3566056548 shutdown_connections 2024-04-25T09:55:45.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f461806b4d0 0x7f461806d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46341602e0 0x7f463406d790 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f463415f5f0 0x7f463414ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 --2- 172.21.15.19:0/3566056548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f463415ba00 0x7f46340ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 -- 172.21.15.19:0/3566056548 >> 172.21.15.19:0/3566056548 conn(0x7f46340b3270 msgr2=0x7f46340c5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:45.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 -- 172.21.15.19:0/3566056548 shutdown_connections 2024-04-25T09:55:45.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.249+0000 7f463b676640 1 -- 172.21.15.19:0/3566056548 wait complete. 2024-04-25T09:55:45.263 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:55:45.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:45.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.364+0000 7f443881b640 1 Processor -- start 2024-04-25T09:55:45.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.364+0000 7f443881b640 1 -- start start 2024-04-25T09:55:45.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.364+0000 7f443881b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4434161890 0x7f44340b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.364+0000 7f443881b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f44340b54a0 0x7f44340b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.364+0000 7f443881b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44340cf1d0 con 0x7f4434161240 2024-04-25T09:55:45.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.364+0000 7f443881b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44340b7120 con 0x7f4434161890 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.364+0000 7f443881b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44340b5e30 con 0x7f44340b54a0 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.365+0000 7f442bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4434161890 0x7f44340b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.365+0000 7f443259c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4434161240 0x7f44340ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42546/0 (socket says 172.21.15.19:42546) 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.365+0000 7f443259c640 1 -- 172.21.15.19:0/484520877 learned_addr learned my addr 172.21.15.19:0/484520877 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.365+0000 7f442bfff640 1 -- 172.21.15.19:0/484520877 >> v2:172.21.15.112:3300/0 conn(0x7f44340b54a0 msgr2=0x7f44340b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.365+0000 7f4432d9d640 1 --2- 172.21.15.19:0/484520877 >> v2:172.21.15.112:3300/0 conn(0x7f44340b54a0 0x7f44340b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.365+0000 7f4431d9b640 1 -- 172.21.15.19:0/484520877 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912993101 0 0) 0x7f44340cf1d0 con 0x7f4434161240 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.365+0000 7f442bfff640 1 --2- 172.21.15.19:0/484520877 >> v2:172.21.15.112:3300/0 conn(0x7f44340b54a0 0x7f44340b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.365+0000 7f442bfff640 1 -- 172.21.15.19:0/484520877 >> v1:172.21.15.19:6789/0 conn(0x7f4434161240 legacy=0x7f44340ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.365+0000 7f442bfff640 1 -- 172.21.15.19:0/484520877 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4434161620 con 0x7f4434161890 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.365+0000 7f4432d9d640 1 --2- 172.21.15.19:0/484520877 >> v2:172.21.15.112:3300/0 conn(0x7f44340b54a0 0x7f44340b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:45.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f442bfff640 1 --2- 172.21.15.19:0/484520877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4434161890 0x7f44340b6be0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f443408ce10 tx=0x7f441c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2706d785484406c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f4431d9b640 1 -- 172.21.15.19:0/484520877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c037070 con 0x7f4434161890 2024-04-25T09:55:45.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f4431d9b640 1 -- 172.21.15.19:0/484520877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c032440 con 0x7f4434161890 2024-04-25T09:55:45.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f4431d9b640 1 -- 172.21.15.19:0/484520877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c032810 con 0x7f4434161890 2024-04-25T09:55:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f443881b640 1 -- 172.21.15.19:0/484520877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4434161890 msgr2=0x7f44340b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f443881b640 1 --2- 172.21.15.19:0/484520877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4434161890 0x7f44340b6be0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f443408ce10 tx=0x7f441c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f443881b640 1 -- 172.21.15.19:0/484520877 shutdown_connections 2024-04-25T09:55:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f443881b640 1 --2- 172.21.15.19:0/484520877 >> v2:172.21.15.112:3300/0 conn(0x7f44340b54a0 0x7f44340b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f443881b640 1 --2- 172.21.15.19:0/484520877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4434161890 0x7f44340b6be0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f443881b640 1 -- 172.21.15.19:0/484520877 >> 172.21.15.19:0/484520877 conn(0x7f44340d04a0 msgr2=0x7f44340d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f443881b640 1 -- 172.21.15.19:0/484520877 shutdown_connections 2024-04-25T09:55:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.366+0000 7f443881b640 1 -- 172.21.15.19:0/484520877 wait complete. 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.368+0000 7f443881b640 1 Processor -- start 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.368+0000 7f443881b640 1 -- start start 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.368+0000 7f443881b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44340b54a0 0x7f4434150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.368+0000 7f443881b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4434161240 0x7f44340c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.368+0000 7f443881b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4434161890 0x7f44340c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.368+0000 7f443881b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341623c0 con 0x7f4434161890 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.368+0000 7f443881b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341620e0 con 0x7f44340b54a0 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.368+0000 7f443881b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44341626a0 con 0x7f4434161240 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f4432d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4434161890 0x7f44340c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f4432d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4434161890 0x7f44340c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39608/0 (socket says 172.21.15.19:39608) 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f4432d9d640 1 -- 172.21.15.19:0/3859737987 learned_addr learned my addr 172.21.15.19:0/3859737987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:45.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f442bfff640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4434161240 0x7f44340c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f443259c640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44340b54a0 0x7f4434150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f4432d9d640 1 -- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4434161240 msgr2=0x7f44340c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f4432d9d640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4434161240 0x7f44340c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f4432d9d640 1 -- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44340b54a0 msgr2=0x7f4434150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f4432d9d640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44340b54a0 0x7f4434150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f4432d9d640 1 -- 172.21.15.19:0/3859737987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443415ce00 con 0x7f4434161890 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f443259c640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44340b54a0 0x7f4434150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f4432d9d640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4434161890 0x7f44340c99a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f442c00cef0 tx=0x7f442c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f442affd640 1 -- 172.21.15.19:0/3859737987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c01a070 con 0x7f4434161890 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f442affd640 1 -- 172.21.15.19:0/3859737987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442c00d860 con 0x7f4434161890 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f442affd640 1 -- 172.21.15.19:0/3859737987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c00dc50 con 0x7f4434161890 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f443881b640 1 -- 172.21.15.19:0/3859737987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443415d0e0 con 0x7f4434161890 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.369+0000 7f443881b640 1 -- 172.21.15.19:0/3859737987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443415d6a0 con 0x7f4434161890 2024-04-25T09:55:45.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.370+0000 7f443881b640 1 -- 172.21.15.19:0/3859737987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44340b52c0 con 0x7f4434161890 2024-04-25T09:55:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.371+0000 7f442affd640 1 -- 172.21.15.19:0/3859737987 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f442c01d070 con 0x7f4434161890 2024-04-25T09:55:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.371+0000 7f442affd640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f441406b4d0 0x7f441406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.371+0000 7f442affd640 1 -- 172.21.15.19:0/3859737987 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f442c08a8d0 con 0x7f4434161890 2024-04-25T09:55:45.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.371+0000 7f443259c640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f441406b4d0 0x7f441406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.372+0000 7f443259c640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f441406b4d0 0x7f441406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4424000f60 tx=0x7f4424009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.375+0000 7f442affd640 1 -- 172.21.15.19:0/3859737987 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44340b52c0 con 0x7f4434161890 2024-04-25T09:55:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.555+0000 7f443881b640 1 -- 172.21.15.19:0/3859737987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f44340b6f00 con 0x7f4434161890 2024-04-25T09:55:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.555+0000 7f442affd640 1 -- 172.21.15.19:0/3859737987 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f442c059a10 con 0x7f4434161890 2024-04-25T09:55:45.558 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:55:45.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 -- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f441406b4d0 msgr2=0x7f441406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f441406b4d0 0x7f441406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4424000f60 tx=0x7f4424009040 comp rx=0 tx=0).stop 2024-04-25T09:55:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 -- 172.21.15.19:0/3859737987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4434161890 msgr2=0x7f44340c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4434161890 0x7f44340c99a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f442c00cef0 tx=0x7f442c00a710 comp rx=0 tx=0).stop 2024-04-25T09:55:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 -- 172.21.15.19:0/3859737987 shutdown_connections 2024-04-25T09:55:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f441406b4d0 0x7f441406d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4434161890 0x7f44340c99a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4434161240 0x7f44340c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 --2- 172.21.15.19:0/3859737987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44340b54a0 0x7f4434150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 -- 172.21.15.19:0/3859737987 >> 172.21.15.19:0/3859737987 conn(0x7f44340d04a0 msgr2=0x7f44340d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 -- 172.21.15.19:0/3859737987 shutdown_connections 2024-04-25T09:55:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.557+0000 7f443881b640 1 -- 172.21.15.19:0/3859737987 wait complete. 2024-04-25T09:55:45.573 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T09:55:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.671+0000 7f3d86134640 1 Processor -- start 2024-04-25T09:55:45.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.671+0000 7f3d86134640 1 -- start start 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.671+0000 7f3d86134640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d8015dbe0 0x7f3d800d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.671+0000 7f3d86134640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3d800b92a0 0x7f3d800d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.671+0000 7f3d86134640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d800b97a0 con 0x7f3d800b2cb0 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.671+0000 7f3d86134640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8015dfc0 con 0x7f3d800b92a0 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.671+0000 7f3d86134640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800d1a00 con 0x7f3d8015dbe0 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.672+0000 7f3d7f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d8015dbe0 0x7f3d800d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.672+0000 7f3d7ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3d800b92a0 0x7f3d800d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.672+0000 7f3d7effd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3d800b2cb0 0x7f3d800b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:46468/0 (socket says 172.21.15.19:46468) 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.672+0000 7f3d7effd640 1 -- 172.21.15.19:0/562739867 learned_addr learned my addr 172.21.15.19:0/562739867 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.672+0000 7f3d7e7fc640 1 -- 172.21.15.19:0/562739867 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86878316 0 0) 0x7f3d800b97a0 con 0x7f3d800b2cb0 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.672+0000 7f3d7e7fc640 1 -- 172.21.15.19:0/562739867 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d68003520 con 0x7f3d800b2cb0 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.672+0000 7f3d7ffff640 1 -- 172.21.15.19:0/562739867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d8015dbe0 msgr2=0x7f3d800d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:45.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.672+0000 7f3d7ffff640 1 --2- 172.21.15.19:0/562739867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d8015dbe0 0x7f3d800d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.672+0000 7f3d7ffff640 1 -- 172.21.15.19:0/562739867 >> v1:172.21.15.112:6789/0 conn(0x7f3d800b2cb0 legacy=0x7f3d800b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.672+0000 7f3d7ffff640 1 -- 172.21.15.19:0/562739867 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d800d1c50 con 0x7f3d800b92a0 2024-04-25T09:55:45.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.672+0000 7f3d7f7fe640 1 --2- 172.21.15.19:0/562739867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d8015dbe0 0x7f3d800d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:45.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.673+0000 7f3d7ffff640 1 --2- 172.21.15.19:0/562739867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d800b92a0 0x7f3d800d0e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009d90 tx=0x7f3d7802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d63bf0e499f05b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.673+0000 7f3d7e7fc640 1 -- 172.21.15.19:0/562739867 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78007740 con 0x7f3d800b92a0 2024-04-25T09:55:45.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.673+0000 7f3d7e7fc640 1 -- 172.21.15.19:0/562739867 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d780078e0 con 0x7f3d800b92a0 2024-04-25T09:55:45.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.673+0000 7f3d7e7fc640 1 -- 172.21.15.19:0/562739867 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78007c90 con 0x7f3d800b92a0 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.674+0000 7f3d86134640 1 -- 172.21.15.19:0/562739867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d800b92a0 msgr2=0x7f3d800d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.674+0000 7f3d86134640 1 --2- 172.21.15.19:0/562739867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d800b92a0 0x7f3d800d0e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009d90 tx=0x7f3d7802a660 comp rx=0 tx=0).stop 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.674+0000 7f3d86134640 1 -- 172.21.15.19:0/562739867 shutdown_connections 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.674+0000 7f3d86134640 1 --2- 172.21.15.19:0/562739867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d800b92a0 0x7f3d800d0e50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.674+0000 7f3d86134640 1 --2- 172.21.15.19:0/562739867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d8015dbe0 0x7f3d800d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.674+0000 7f3d86134640 1 -- 172.21.15.19:0/562739867 >> 172.21.15.19:0/562739867 conn(0x7f3d80061a20 msgr2=0x7f3d80061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.674+0000 7f3d86134640 1 -- 172.21.15.19:0/562739867 shutdown_connections 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.674+0000 7f3d86134640 1 -- 172.21.15.19:0/562739867 wait complete. 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.675+0000 7f3d86134640 1 Processor -- start 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.675+0000 7f3d86134640 1 -- start start 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d86134640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d800b2cb0 0x7f3d801f0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d86134640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d800b92a0 0x7f3d800b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d86134640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d8015dbe0 0x7f3d80160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d86134640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800b6ab0 con 0x7f3d800b92a0 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d86134640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800d2210 con 0x7f3d8015dbe0 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d86134640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800b6c30 con 0x7f3d800b2cb0 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d7effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d800b92a0 0x7f3d800b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d7effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d800b92a0 0x7f3d800b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39618/0 (socket says 172.21.15.19:39618) 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d7effd640 1 -- 172.21.15.19:0/2062597765 learned_addr learned my addr 172.21.15.19:0/2062597765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d7f7fe640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d800b2cb0 0x7f3d801f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d7ffff640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d8015dbe0 0x7f3d80160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d7effd640 1 -- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d800b2cb0 msgr2=0x7f3d801f0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d7effd640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d800b2cb0 0x7f3d801f0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d7effd640 1 -- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d8015dbe0 msgr2=0x7f3d80160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d7effd640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d8015dbe0 0x7f3d80160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d7effd640 1 -- 172.21.15.19:0/2062597765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d800b39c0 con 0x7f3d800b92a0 2024-04-25T09:55:45.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.676+0000 7f3d7effd640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d800b92a0 0x7f3d800b32e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c390 tx=0x7f3d7400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.677+0000 7f3d67fff640 1 -- 172.21.15.19:0/2062597765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74017070 con 0x7f3d800b92a0 2024-04-25T09:55:45.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.677+0000 7f3d67fff640 1 -- 172.21.15.19:0/2062597765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74005ce0 con 0x7f3d800b92a0 2024-04-25T09:55:45.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.677+0000 7f3d86134640 1 -- 172.21.15.19:0/2062597765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d800b3ca0 con 0x7f3d800b92a0 2024-04-25T09:55:45.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.677+0000 7f3d67fff640 1 -- 172.21.15.19:0/2062597765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d740072f0 con 0x7f3d800b92a0 2024-04-25T09:55:45.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.677+0000 7f3d86134640 1 -- 172.21.15.19:0/2062597765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d801f3ac0 con 0x7f3d800b92a0 2024-04-25T09:55:45.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.677+0000 7f3d86134640 1 -- 172.21.15.19:0/2062597765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d801f33b0 con 0x7f3d800b92a0 2024-04-25T09:55:45.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.678+0000 7f3d67fff640 1 -- 172.21.15.19:0/2062597765 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d7401a070 con 0x7f3d800b92a0 2024-04-25T09:55:45.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.679+0000 7f3d67fff640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d6c06b4d0 0x7f3d6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.679+0000 7f3d67fff640 1 -- 172.21.15.19:0/2062597765 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3d74089340 con 0x7f3d800b92a0 2024-04-25T09:55:45.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.679+0000 7f3d7f7fe640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d6c06b4d0 0x7f3d6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.679+0000 7f3d7f7fe640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d6c06b4d0 0x7f3d6c06d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009560 tx=0x7f3d70009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.683+0000 7f3d67fff640 1 -- 172.21.15.19:0/2062597765 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d74057d70 con 0x7f3d800b92a0 2024-04-25T09:55:45.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:45.940+0000 7f3d86134640 1 -- 172.21.15.19:0/2062597765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f3d80073990 con 0x7f3d800b92a0 2024-04-25T09:55:46.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.948+0000 7f3d67fff640 1 -- 172.21.15.19:0/2062597765 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v144) v1 ==== 146+0+0 (secure 0 0 0) 0x7f3d7405be10 con 0x7f3d800b92a0 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.950+0000 7f3d86134640 1 -- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d6c06b4d0 msgr2=0x7f3d6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.950+0000 7f3d86134640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d6c06b4d0 0x7f3d6c06d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009560 tx=0x7f3d70009190 comp rx=0 tx=0).stop 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.950+0000 7f3d86134640 1 -- 172.21.15.19:0/2062597765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d800b92a0 msgr2=0x7f3d800b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.951+0000 7f3d86134640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d800b92a0 0x7f3d800b32e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c390 tx=0x7f3d7400c850 comp rx=0 tx=0).stop 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.951+0000 7f3d86134640 1 -- 172.21.15.19:0/2062597765 shutdown_connections 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.951+0000 7f3d86134640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d6c06b4d0 0x7f3d6c06d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.951+0000 7f3d86134640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d8015dbe0 0x7f3d80160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.951+0000 7f3d86134640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d800b92a0 0x7f3d800b32e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.951+0000 7f3d86134640 1 --2- 172.21.15.19:0/2062597765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d800b2cb0 0x7f3d801f0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.951+0000 7f3d86134640 1 -- 172.21.15.19:0/2062597765 >> 172.21.15.19:0/2062597765 conn(0x7f3d80061a20 msgr2=0x7f3d800b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.951+0000 7f3d86134640 1 -- 172.21.15.19:0/2062597765 shutdown_connections 2024-04-25T09:55:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:46.951+0000 7f3d86134640 1 -- 172.21.15.19:0/2062597765 wait complete. 2024-04-25T09:55:46.967 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:55:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.075+0000 7f114ae4d640 1 Processor -- start 2024-04-25T09:55:47.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.075+0000 7f114ae4d640 1 -- start start 2024-04-25T09:55:47.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.075+0000 7f114ae4d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f114415b7d0 0x7f11440cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.075+0000 7f114ae4d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11440b5620 0x7f11440b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.075+0000 7f114ae4d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114406cf50 con 0x7f114406c2d0 2024-04-25T09:55:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.075+0000 7f114ae4d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440ce530 con 0x7f11440b5620 2024-04-25T09:55:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.075+0000 7f114ae4d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440ce810 con 0x7f114415b7d0 2024-04-25T09:55:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.076+0000 7f1143fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11440b5620 0x7f11440b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.076+0000 7f1143fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11440b5620 0x7f11440b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39622/0 (socket says 172.21.15.19:39622) 2024-04-25T09:55:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.076+0000 7f1143fff640 1 -- 172.21.15.19:0/1988954118 learned_addr learned my addr 172.21.15.19:0/1988954118 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.076+0000 7f1148be9640 1 --2- 172.21.15.19:0/1988954118 >> v2:172.21.15.112:3300/0 conn(0x7f114415b7d0 0x7f11440cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.076+0000 7f1143fff640 1 -- 172.21.15.19:0/1988954118 >> v2:172.21.15.112:3300/0 conn(0x7f114415b7d0 msgr2=0x7f11440cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.076+0000 7f1143fff640 1 --2- 172.21.15.19:0/1988954118 >> v2:172.21.15.112:3300/0 conn(0x7f114415b7d0 0x7f11440cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.076+0000 7f1143fff640 1 -- 172.21.15.19:0/1988954118 >> v1:172.21.15.112:6789/0 conn(0x7f114406c2d0 legacy=0x7f114406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.076+0000 7f1143fff640 1 -- 172.21.15.19:0/1988954118 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440ceaf0 con 0x7f11440b5620 2024-04-25T09:55:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.076+0000 7f1148be9640 1 --2- 172.21.15.19:0/1988954118 >> v2:172.21.15.112:3300/0 conn(0x7f114415b7d0 0x7f11440cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.077+0000 7f1143fff640 1 --2- 172.21.15.19:0/1988954118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11440b5620 0x7f11440b5ab0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f114408c0d0 tx=0x7f113402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07b4aedfafde2f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.077+0000 7f11437fe640 1 -- 172.21.15.19:0/1988954118 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134037070 con 0x7f11440b5620 2024-04-25T09:55:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.077+0000 7f11437fe640 1 -- 172.21.15.19:0/1988954118 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1134032440 con 0x7f11440b5620 2024-04-25T09:55:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.077+0000 7f11437fe640 1 -- 172.21.15.19:0/1988954118 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134032810 con 0x7f11440b5620 2024-04-25T09:55:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.077+0000 7f114ae4d640 1 -- 172.21.15.19:0/1988954118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11440b5620 msgr2=0x7f11440b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.077+0000 7f114ae4d640 1 --2- 172.21.15.19:0/1988954118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11440b5620 0x7f11440b5ab0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f114408c0d0 tx=0x7f113402a230 comp rx=0 tx=0).stop 2024-04-25T09:55:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.077+0000 7f114ae4d640 1 -- 172.21.15.19:0/1988954118 shutdown_connections 2024-04-25T09:55:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.077+0000 7f114ae4d640 1 --2- 172.21.15.19:0/1988954118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11440b5620 0x7f11440b5ab0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.077+0000 7f114ae4d640 1 --2- 172.21.15.19:0/1988954118 >> v2:172.21.15.112:3300/0 conn(0x7f114415b7d0 0x7f11440cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.077+0000 7f114ae4d640 1 -- 172.21.15.19:0/1988954118 >> 172.21.15.19:0/1988954118 conn(0x7f11440d0800 msgr2=0x7f11440d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.077+0000 7f114ae4d640 1 -- 172.21.15.19:0/1988954118 shutdown_connections 2024-04-25T09:55:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.078+0000 7f114ae4d640 1 -- 172.21.15.19:0/1988954118 wait complete. 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.079+0000 7f114ae4d640 1 Processor -- start 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.079+0000 7f114ae4d640 1 -- start start 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f114ae4d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f114406c2d0 0x7f11440cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f114ae4d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f11440b5620 0x7f11440c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f114ae4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f114415b7d0 0x7f11440b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f114ae4d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440cf6d0 con 0x7f114406c2d0 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f114ae4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440cf550 con 0x7f114415b7d0 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f114ae4d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114415bbb0 con 0x7f11440b5620 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f1148be9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f114406c2d0 0x7f11440cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f1148be9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f114406c2d0 0x7f11440cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39628/0 (socket says 172.21.15.19:39628) 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f1148be9640 1 -- 172.21.15.19:0/1350544327 learned_addr learned my addr 172.21.15.19:0/1350544327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f1143fff640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f11440b5620 0x7f11440c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f11493ea640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f114415b7d0 0x7f11440b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f1148be9640 1 -- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f11440b5620 msgr2=0x7f11440c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f1148be9640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f11440b5620 0x7f11440c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f1148be9640 1 -- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f114415b7d0 msgr2=0x7f11440b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f1148be9640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f114415b7d0 0x7f11440b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f1148be9640 1 -- 172.21.15.19:0/1350544327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440cbe20 con 0x7f114406c2d0 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.080+0000 7f1148be9640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f114406c2d0 0x7f11440cd980 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f112c00c820 tx=0x7f112c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.081+0000 7f11417fa640 1 -- 172.21.15.19:0/1350544327 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c015070 con 0x7f114406c2d0 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.081+0000 7f11417fa640 1 -- 172.21.15.19:0/1350544327 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c007a30 con 0x7f114406c2d0 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.081+0000 7f114ae4d640 1 -- 172.21.15.19:0/1350544327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11440cc100 con 0x7f114406c2d0 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.081+0000 7f11417fa640 1 -- 172.21.15.19:0/1350544327 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c007de0 con 0x7f114406c2d0 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.081+0000 7f114ae4d640 1 -- 172.21.15.19:0/1350544327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11440c9580 con 0x7f114406c2d0 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.082+0000 7f114ae4d640 1 -- 172.21.15.19:0/1350544327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11440cf010 con 0x7f114406c2d0 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.082+0000 7f11417fa640 1 -- 172.21.15.19:0/1350544327 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f112c018070 con 0x7f114406c2d0 2024-04-25T09:55:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.082+0000 7f11417fa640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 0x7f112406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.083+0000 7f11417fa640 1 -- 172.21.15.19:0/1350544327 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f112c089f10 con 0x7f114406c2d0 2024-04-25T09:55:47.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.083+0000 7f1143fff640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 0x7f112406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.083+0000 7f1143fff640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 0x7f112406d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1134034000 tx=0x7f113402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.087+0000 7f11417fa640 1 -- 172.21.15.19:0/1350544327 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11440cf010 con 0x7f114406c2d0 2024-04-25T09:55:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.266+0000 7f114ae4d640 1 -- 172.21.15.19:0/1350544327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1144073a00 con 0x7f114406c2d0 2024-04-25T09:55:47.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.269+0000 7f11417fa640 1 -- 172.21.15.19:0/1350544327 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f112c0588b0 con 0x7f114406c2d0 2024-04-25T09:55:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.271+0000 7f114ae4d640 1 -- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 msgr2=0x7f112406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.271+0000 7f114ae4d640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 0x7f112406d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1134034000 tx=0x7f113402e000 comp rx=0 tx=0).stop 2024-04-25T09:55:47.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.271+0000 7f114ae4d640 1 -- 172.21.15.19:0/1350544327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f114406c2d0 msgr2=0x7f11440cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.271+0000 7f114ae4d640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f114406c2d0 0x7f11440cd980 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f112c00c820 tx=0x7f112c00cce0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.271+0000 7f114ae4d640 1 -- 172.21.15.19:0/1350544327 shutdown_connections 2024-04-25T09:55:47.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.271+0000 7f114ae4d640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 0x7f112406d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.271+0000 7f114ae4d640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f114415b7d0 0x7f11440b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.271+0000 7f114ae4d640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f11440b5620 0x7f11440c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.271+0000 7f114ae4d640 1 --2- 172.21.15.19:0/1350544327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f114406c2d0 0x7f11440cd980 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.271+0000 7f114ae4d640 1 -- 172.21.15.19:0/1350544327 >> 172.21.15.19:0/1350544327 conn(0x7f11440d0800 msgr2=0x7f11440b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:47.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.271+0000 7f114ae4d640 1 -- 172.21.15.19:0/1350544327 shutdown_connections 2024-04-25T09:55:47.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.272+0000 7f114ae4d640 1 -- 172.21.15.19:0/1350544327 wait complete. 2024-04-25T09:55:47.285 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:55:47.286 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:55:47.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:55:47.364 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:55:47.365 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:55:47.365 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:47.382 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:55:47.382 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:55:47.382 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:47.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:55:47.422 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:55:47.422 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:47.437 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:55:47.438 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:55:47.438 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:47.478 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:55:47.478 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:55:47.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.577+0000 7fb2517a5640 1 Processor -- start 2024-04-25T09:55:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.577+0000 7fb2517a5640 1 -- start start 2024-04-25T09:55:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb2517a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb24c0b2cb0 0x7fb24c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb2517a5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb24c0d1020 0x7fb24c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb2517a5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24c0b97a0 con 0x7fb24c15dbe0 2024-04-25T09:55:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb2517a5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb24c15dfc0 con 0x7fb24c0d1020 2024-04-25T09:55:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb2517a5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0d1a00 con 0x7fb24c0b2cb0 2024-04-25T09:55:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb24b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb24c0d1020 0x7fb24c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb24affd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb24c15dbe0 0x7fb24c0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42560/0 (socket says 172.21.15.19:42560) 2024-04-25T09:55:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb24affd640 1 -- 172.21.15.19:0/1100712718 learned_addr learned my addr 172.21.15.19:0/1100712718 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb24a7fc640 1 --2- 172.21.15.19:0/1100712718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb24c0b2cb0 0x7fb24c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb24b7fe640 1 -- 172.21.15.19:0/1100712718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb24c0b2cb0 msgr2=0x7fb24c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb24b7fe640 1 --2- 172.21.15.19:0/1100712718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb24c0b2cb0 0x7fb24c0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb24b7fe640 1 -- 172.21.15.19:0/1100712718 >> v1:172.21.15.19:6789/0 conn(0x7fb24c15dbe0 legacy=0x7fb24c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb24b7fe640 1 -- 172.21.15.19:0/1100712718 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c0d1c50 con 0x7fb24c0d1020 2024-04-25T09:55:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.578+0000 7fb249ffb640 1 -- 172.21.15.19:0/1100712718 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207884516 0 0) 0x7fb24c0b97a0 con 0x7fb24c15dbe0 2024-04-25T09:55:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.579+0000 7fb24b7fe640 1 --2- 172.21.15.19:0/1100712718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c0d1020 0x7fb24c0d2c70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb2440097a0 tx=0x7fb24402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7566d989910dfeea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.579+0000 7fb249ffb640 1 -- 172.21.15.19:0/1100712718 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244037070 con 0x7fb24c0d1020 2024-04-25T09:55:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.579+0000 7fb249ffb640 1 -- 172.21.15.19:0/1100712718 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb244032440 con 0x7fb24c0d1020 2024-04-25T09:55:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.579+0000 7fb249ffb640 1 -- 172.21.15.19:0/1100712718 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244032810 con 0x7fb24c0d1020 2024-04-25T09:55:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.580+0000 7fb2517a5640 1 -- 172.21.15.19:0/1100712718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c0d1020 msgr2=0x7fb24c0d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.580+0000 7fb2517a5640 1 --2- 172.21.15.19:0/1100712718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c0d1020 0x7fb24c0d2c70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb2440097a0 tx=0x7fb24402a180 comp rx=0 tx=0).stop 2024-04-25T09:55:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.580+0000 7fb2517a5640 1 -- 172.21.15.19:0/1100712718 shutdown_connections 2024-04-25T09:55:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.580+0000 7fb2517a5640 1 --2- 172.21.15.19:0/1100712718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c0d1020 0x7fb24c0d2c70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.580+0000 7fb2517a5640 1 --2- 172.21.15.19:0/1100712718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb24c0b2cb0 0x7fb24c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.580+0000 7fb2517a5640 1 -- 172.21.15.19:0/1100712718 >> 172.21.15.19:0/1100712718 conn(0x7fb24c061a20 msgr2=0x7fb24c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.580+0000 7fb2517a5640 1 -- 172.21.15.19:0/1100712718 shutdown_connections 2024-04-25T09:55:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.580+0000 7fb2517a5640 1 -- 172.21.15.19:0/1100712718 wait complete. 2024-04-25T09:55:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.581+0000 7fb2517a5640 1 Processor -- start 2024-04-25T09:55:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.581+0000 7fb2517a5640 1 -- start start 2024-04-25T09:55:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb2517a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb24c0b2cb0 0x7fb24c151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb2517a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb24c0d1020 0x7fb24c0b8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb2517a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c15dbe0 0x7fb24c0b88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb2517a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0b92a0 con 0x7fb24c15dbe0 2024-04-25T09:55:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb2517a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0d2210 con 0x7fb24c0d1020 2024-04-25T09:55:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb2517a5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0b9420 con 0x7fb24c0b2cb0 2024-04-25T09:55:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb24b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c15dbe0 0x7fb24c0b88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb24b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c15dbe0 0x7fb24c0b88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39634/0 (socket says 172.21.15.19:39634) 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb24b7fe640 1 -- 172.21.15.19:0/3520420404 learned_addr learned my addr 172.21.15.19:0/3520420404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb24affd640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb24c0b2cb0 0x7fb24c151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb24a7fc640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb24c0d1020 0x7fb24c0b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb24b7fe640 1 -- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb24c0b2cb0 msgr2=0x7fb24c151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb24b7fe640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb24c0b2cb0 0x7fb24c151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb24b7fe640 1 -- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb24c0d1020 msgr2=0x7fb24c0b8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb24b7fe640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb24c0d1020 0x7fb24c0b8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb24b7fe640 1 -- 172.21.15.19:0/3520420404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c15e2f0 con 0x7fb24c15dbe0 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.582+0000 7fb24a7fc640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb24c0d1020 0x7fb24c0b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.583+0000 7fb24b7fe640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c15dbe0 0x7fb24c0b88d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb244035040 tx=0x7fb2440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.583+0000 7fb2437fe640 1 -- 172.21.15.19:0/3520420404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244037040 con 0x7fb24c15dbe0 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.583+0000 7fb2437fe640 1 -- 172.21.15.19:0/3520420404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb244042070 con 0x7fb24c15dbe0 2024-04-25T09:55:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.583+0000 7fb2517a5640 1 -- 172.21.15.19:0/3520420404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24c15e5d0 con 0x7fb24c15dbe0 2024-04-25T09:55:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.583+0000 7fb2437fe640 1 -- 172.21.15.19:0/3520420404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2440308a0 con 0x7fb24c15dbe0 2024-04-25T09:55:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.583+0000 7fb2517a5640 1 -- 172.21.15.19:0/3520420404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24c15eb90 con 0x7fb24c15dbe0 2024-04-25T09:55:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.584+0000 7fb2517a5640 1 -- 172.21.15.19:0/3520420404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24c073990 con 0x7fb24c15dbe0 2024-04-25T09:55:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.584+0000 7fb2437fe640 1 -- 172.21.15.19:0/3520420404 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb244030a40 con 0x7fb24c15dbe0 2024-04-25T09:55:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.585+0000 7fb2437fe640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb22c06b4d0 0x7fb22c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.585+0000 7fb2437fe640 1 -- 172.21.15.19:0/3520420404 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb2440aa870 con 0x7fb24c15dbe0 2024-04-25T09:55:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.585+0000 7fb24affd640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb22c06b4d0 0x7fb22c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.585+0000 7fb24affd640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb22c06b4d0 0x7fb22c06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb23c007ed0 tx=0x7fb23c007e60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.589+0000 7fb2437fe640 1 -- 172.21.15.19:0/3520420404 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2440798f0 con 0x7fb24c15dbe0 2024-04-25T09:55:47.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.809+0000 7fb2517a5640 1 -- 172.21.15.19:0/3520420404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb24c061ea0 con 0x7fb24c15dbe0 2024-04-25T09:55:47.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.809+0000 7fb2437fe640 1 -- 172.21.15.19:0/3520420404 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fb2440791f0 con 0x7fb24c15dbe0 2024-04-25T09:55:47.813 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:47.813 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3845932448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3845932448},{"type":"v1","addr":"172.21.15.112:6836","nonce":3845932448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5122,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1655535300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1655535300},{"type":"v1","addr":"172.21.15.19:6834","nonce":1655535300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/165157787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":165157787},{"type":"v1","addr":"172.21.15.112:6837","nonce":165157787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2790415518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2790415518},{"type":"v1","addr":"172.21.15.19:6835","nonce":2790415518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2024-04-25T09:55:47.813 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 19 2024-04-25T09:55:47.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 -- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb22c06b4d0 msgr2=0x7fb22c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb22c06b4d0 0x7fb22c06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb23c007ed0 tx=0x7fb23c007e60 comp rx=0 tx=0).stop 2024-04-25T09:55:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 -- 172.21.15.19:0/3520420404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c15dbe0 msgr2=0x7fb24c0b88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c15dbe0 0x7fb24c0b88d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb244035040 tx=0x7fb2440046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 -- 172.21.15.19:0/3520420404 shutdown_connections 2024-04-25T09:55:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb22c06b4d0 0x7fb22c06d990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c15dbe0 0x7fb24c0b88d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb24c0d1020 0x7fb24c0b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 --2- 172.21.15.19:0/3520420404 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb24c0b2cb0 0x7fb24c151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 -- 172.21.15.19:0/3520420404 >> 172.21.15.19:0/3520420404 conn(0x7fb24c061a20 msgr2=0x7fb24c159780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 -- 172.21.15.19:0/3520420404 shutdown_connections 2024-04-25T09:55:47.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.811+0000 7fb2517a5640 1 -- 172.21.15.19:0/3520420404 wait complete. 2024-04-25T09:55:47.826 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:55:47.826 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:55:47.826 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:55:47.826 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:47.826 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:55:47.826 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:55:47.827 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:55:47.827 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:47.827 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:55:47.827 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:55:47.827 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:55:47.827 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:47.827 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:55:47.828 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:55:47.828 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:55:47.828 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:47.837 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:55:47.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:55:47.837 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:55:47.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:55:47.840 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:55:47.840 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:55:47.841 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:55:47.841 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:55:47.879 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:55:47.879 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:47.879 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:55:47.880 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:47.882 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:55:47.882 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:47.883 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:55:47.883 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:47.883 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:55:47.955 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T09:55:47.958 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T09:55:47.965 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T09:55:47.966 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T09:55:47.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.987+0000 7f983deee640 1 Processor -- start 2024-04-25T09:55:47.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.987+0000 7f983deee640 1 -- start start 2024-04-25T09:55:47.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.988+0000 7f983deee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f983815dc20 0x7f98380ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.988+0000 7f983deee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98380d32f0 0x7f98380cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.988+0000 7f983deee640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983815e000 con 0x7f9838161710 2024-04-25T09:55:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.988+0000 7f983deee640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98380d1de0 con 0x7f983815dc20 2024-04-25T09:55:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.988+0000 7f983deee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9838161c60 con 0x7f98380d32f0 2024-04-25T09:55:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.988+0000 7f9836ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9838161710 0x7f983806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35548/0 (socket says 172.21.15.19:35548) 2024-04-25T09:55:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.988+0000 7f9836ffd640 1 -- 172.21.15.19:0/2069643411 learned_addr learned my addr 172.21.15.19:0/2069643411 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.988+0000 7f9837fff640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98380d32f0 0x7f98380cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.988+0000 7f98377fe640 1 --2- 172.21.15.19:0/2069643411 >> v2:172.21.15.112:3300/0 conn(0x7f983815dc20 0x7f98380ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.988+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241323022 0 0) 0x7f983815e000 con 0x7f9838161710 2024-04-25T09:55:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.988+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9820003520 con 0x7f9838161710 2024-04-25T09:55:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2133190166 0 0) 0x7f9820003520 con 0x7f9838161710 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98380d32f0 msgr2=0x7f98380cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98380d32f0 0x7f98380cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 >> v2:172.21.15.112:3300/0 conn(0x7f983815dc20 msgr2=0x7f98380ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 --2- 172.21.15.19:0/2069643411 >> v2:172.21.15.112:3300/0 conn(0x7f983815dc20 0x7f98380ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9838161f40 con 0x7f9838161710 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f982c001670 con 0x7f9838161710 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98377fe640 1 --2- 172.21.15.19:0/2069643411 >> v2:172.21.15.112:3300/0 conn(0x7f983815dc20 0x7f98380ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 >> v1:172.21.15.19:6789/0 conn(0x7f9838161710 legacy=0x7f983806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98200039b0 0x7f9820003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98380d32f0 0x7f9820004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9820004d40 0x7f98200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98380d1de0 con 0x7f9820004d40 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9838161c60 con 0x7f98200039b0 2024-04-25T09:55:47.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982000b9c0 con 0x7f98380d32f0 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98377fe640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98200039b0 0x7f9820003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f9837fff640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9820004d40 0x7f98200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f9836ffd640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98380d32f0 0x7f9820004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98377fe640 1 -- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98380d32f0 msgr2=0x7f9820004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98377fe640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98380d32f0 0x7f9820004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98377fe640 1 -- 172.21.15.19:0/2069643411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9820004d40 msgr2=0x7f98200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98377fe640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9820004d40 0x7f98200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.989+0000 7f98377fe640 1 -- 172.21.15.19:0/2069643411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9820003520 con 0x7f98200039b0 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.990+0000 7f98377fe640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98200039b0 0x7f9820003dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9828009c90 tx=0x7f9820005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28024c5bbdbd9149 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.990+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828036070 con 0x7f98200039b0 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.990+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9828031400 con 0x7f98200039b0 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.990+0000 7f98367fc640 1 -- 172.21.15.19:0/2069643411 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828009860 con 0x7f98200039b0 2024-04-25T09:55:47.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.991+0000 7f983deee640 1 -- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98200039b0 msgr2=0x7f9820003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.991+0000 7f983deee640 1 --2- 172.21.15.19:0/2069643411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98200039b0 0x7f9820003dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9828009c90 tx=0x7f9820005830 comp rx=0 tx=0).stop 2024-04-25T09:55:47.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.991+0000 7f9837fff640 1 -- 172.21.15.19:0/2069643411 reap_dead start 2024-04-25T09:55:47.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.991+0000 7f983deee640 1 -- 172.21.15.19:0/2069643411 shutdown_connections 2024-04-25T09:55:47.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.991+0000 7f983deee640 1 -- 172.21.15.19:0/2069643411 >> 172.21.15.19:0/2069643411 conn(0x7f98380d0420 msgr2=0x7f98380d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:47.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.991+0000 7f983deee640 1 -- 172.21.15.19:0/2069643411 shutdown_connections 2024-04-25T09:55:47.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.991+0000 7f983deee640 1 -- 172.21.15.19:0/2069643411 wait complete. 2024-04-25T09:55:47.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.992+0000 7f983deee640 1 Processor -- start 2024-04-25T09:55:48.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.992+0000 7f983deee640 1 -- start start 2024-04-25T09:55:48.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.993+0000 7f983deee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98200039b0 0x7f9838156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.993+0000 7f983deee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9838157320 0x7f98380cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.993+0000 7f983deee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98380cd7f0 0x7f98380cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.993+0000 7f983deee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98380b5e50 con 0x7f98380cd7f0 2024-04-25T09:55:48.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.993+0000 7f983deee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9838162500 con 0x7f9838157320 2024-04-25T09:55:48.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.993+0000 7f983deee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98380b5fd0 con 0x7f98200039b0 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.993+0000 7f9837fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98380cd7f0 0x7f98380cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.993+0000 7f9837fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98380cd7f0 0x7f98380cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54114/0 (socket says 172.21.15.19:54114) 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.993+0000 7f9837fff640 1 -- 172.21.15.19:0/1806955779 learned_addr learned my addr 172.21.15.19:0/1806955779 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.993+0000 7f98377fe640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98200039b0 0x7f9838156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.993+0000 7f9836ffd640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9838157320 0x7f98380cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.994+0000 7f9837fff640 1 -- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98200039b0 msgr2=0x7f9838156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.994+0000 7f9837fff640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98200039b0 0x7f9838156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.994+0000 7f9837fff640 1 -- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9838157320 msgr2=0x7f98380cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.994+0000 7f9837fff640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9838157320 0x7f98380cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.994+0000 7f9837fff640 1 -- 172.21.15.19:0/1806955779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983815ed30 con 0x7f98380cd7f0 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.994+0000 7f9837fff640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98380cd7f0 0x7f98380cb1d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f98380b90f0 tx=0x7f9830011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.994+0000 7f981ffff640 1 -- 172.21.15.19:0/1806955779 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983000fbf0 con 0x7f98380cd7f0 2024-04-25T09:55:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.994+0000 7f981ffff640 1 -- 172.21.15.19:0/1806955779 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983000fd90 con 0x7f98380cd7f0 2024-04-25T09:55:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.994+0000 7f981ffff640 1 -- 172.21.15.19:0/1806955779 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830015670 con 0x7f98380cd7f0 2024-04-25T09:55:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.994+0000 7f983deee640 1 -- 172.21.15.19:0/1806955779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983815f010 con 0x7f98380cd7f0 2024-04-25T09:55:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.994+0000 7f983deee640 1 -- 172.21.15.19:0/1806955779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98381567d0 con 0x7f98380cd7f0 2024-04-25T09:55:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.995+0000 7f983deee640 1 -- 172.21.15.19:0/1806955779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9838156a90 con 0x7f98380cd7f0 2024-04-25T09:55:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.995+0000 7f981ffff640 1 -- 172.21.15.19:0/1806955779 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9830015810 con 0x7f98380cd7f0 2024-04-25T09:55:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.996+0000 7f981ffff640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f982406b4d0 0x7f982406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.996+0000 7f981ffff640 1 -- 172.21.15.19:0/1806955779 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f983008ad80 con 0x7f98380cd7f0 2024-04-25T09:55:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.996+0000 7f98377fe640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f982406b4d0 0x7f982406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:47.996+0000 7f98377fe640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f982406b4d0 0x7f982406d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9828033000 tx=0x7f9828008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:48.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.000+0000 7f981ffff640 1 -- 172.21.15.19:0/1806955779 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9830019030 con 0x7f98380cd7f0 2024-04-25T09:55:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.193+0000 7f983deee640 1 -- 172.21.15.19:0/1806955779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f98380b2d80 con 0x7f98380cd7f0 2024-04-25T09:55:48.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.963+0000 7f981ffff640 1 -- 172.21.15.19:0/1806955779 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) v1 ==== 97+0+0 (secure 0 0 0) 0x7f98380b2d80 con 0x7f98380cd7f0 2024-04-25T09:55:48.967 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T09:55:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 -- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f982406b4d0 msgr2=0x7f982406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f982406b4d0 0x7f982406d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9828033000 tx=0x7f9828008030 comp rx=0 tx=0).stop 2024-04-25T09:55:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 -- 172.21.15.19:0/1806955779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98380cd7f0 msgr2=0x7f98380cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98380cd7f0 0x7f98380cb1d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f98380b90f0 tx=0x7f9830011f10 comp rx=0 tx=0).stop 2024-04-25T09:55:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 -- 172.21.15.19:0/1806955779 shutdown_connections 2024-04-25T09:55:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f982406b4d0 0x7f982406d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98380cd7f0 0x7f98380cb1d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9838157320 0x7f98380cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 --2- 172.21.15.19:0/1806955779 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98200039b0 0x7f9838156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 -- 172.21.15.19:0/1806955779 >> 172.21.15.19:0/1806955779 conn(0x7f98380d0420 msgr2=0x7f98380d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 -- 172.21.15.19:0/1806955779 shutdown_connections 2024-04-25T09:55:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:48.967+0000 7f983deee640 1 -- 172.21.15.19:0/1806955779 wait complete. 2024-04-25T09:55:48.983 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:55:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.084+0000 7f5ba584d640 1 Processor -- start 2024-04-25T09:55:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.084+0000 7f5ba584d640 1 -- start start 2024-04-25T09:55:49.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.085+0000 7f5ba584d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ba0144750 0x7f5ba006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.085+0000 7f5ba584d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ba00cf030 0x7f5ba006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.085+0000 7f5ba584d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba0144b30 con 0x7f5ba006eb10 2024-04-25T09:55:49.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.085+0000 7f5ba584d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba00712d0 con 0x7f5ba00cf030 2024-04-25T09:55:49.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.085+0000 7f5ba584d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba00715b0 con 0x7f5ba0144750 2024-04-25T09:55:49.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.085+0000 7f5b9f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ba00cf030 0x7f5ba006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.085+0000 7f5b9f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ba00cf030 0x7f5ba006d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54124/0 (socket says 172.21.15.19:54124) 2024-04-25T09:55:49.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.085+0000 7f5b9f7fe640 1 -- 172.21.15.19:0/2996561183 learned_addr learned my addr 172.21.15.19:0/2996561183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:49.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.086+0000 7f5b9f7fe640 1 -- 172.21.15.19:0/2996561183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ba0144750 msgr2=0x7f5ba006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:49.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.086+0000 7f5b9f7fe640 1 --2- 172.21.15.19:0/2996561183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ba0144750 0x7f5ba006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.086+0000 7f5b9f7fe640 1 -- 172.21.15.19:0/2996561183 >> v1:172.21.15.112:6789/0 conn(0x7f5ba006eb10 legacy=0x7f5ba00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:49.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.086+0000 7f5b9f7fe640 1 -- 172.21.15.19:0/2996561183 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba0071890 con 0x7f5ba00cf030 2024-04-25T09:55:49.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.086+0000 7f5b9f7fe640 1 --2- 172.21.15.19:0/2996561183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ba00cf030 0x7f5ba006d290 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009e50 tx=0x7f5b9802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac38b72fce250013 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.086+0000 7f5b9e7fc640 1 -- 172.21.15.19:0/2996561183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98036070 con 0x7f5ba00cf030 2024-04-25T09:55:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.086+0000 7f5b9e7fc640 1 -- 172.21.15.19:0/2996561183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b98031600 con 0x7f5ba00cf030 2024-04-25T09:55:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.086+0000 7f5b9e7fc640 1 -- 172.21.15.19:0/2996561183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b980319d0 con 0x7f5ba00cf030 2024-04-25T09:55:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.087+0000 7f5ba584d640 1 -- 172.21.15.19:0/2996561183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ba00cf030 msgr2=0x7f5ba006d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.087+0000 7f5ba584d640 1 --2- 172.21.15.19:0/2996561183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ba00cf030 0x7f5ba006d290 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009e50 tx=0x7f5b9802a630 comp rx=0 tx=0).stop 2024-04-25T09:55:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.087+0000 7f5ba584d640 1 -- 172.21.15.19:0/2996561183 shutdown_connections 2024-04-25T09:55:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.087+0000 7f5ba584d640 1 --2- 172.21.15.19:0/2996561183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ba00cf030 0x7f5ba006d290 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.087+0000 7f5ba584d640 1 --2- 172.21.15.19:0/2996561183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ba0144750 0x7f5ba006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.087+0000 7f5ba584d640 1 -- 172.21.15.19:0/2996561183 >> 172.21.15.19:0/2996561183 conn(0x7f5ba006abe0 msgr2=0x7f5ba006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.087+0000 7f5ba584d640 1 -- 172.21.15.19:0/2996561183 shutdown_connections 2024-04-25T09:55:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.087+0000 7f5ba584d640 1 -- 172.21.15.19:0/2996561183 wait complete. 2024-04-25T09:55:49.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.088+0000 7f5ba584d640 1 Processor -- start 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.088+0000 7f5ba584d640 1 -- start start 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.089+0000 7f5ba584d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ba006eb10 0x7f5ba013f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.089+0000 7f5ba584d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ba00cf030 0x7f5ba00b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.089+0000 7f5ba584d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba0144750 0x7f5ba00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.089+0000 7f5ba584d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba01455d0 con 0x7f5ba00cf030 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.089+0000 7f5ba584d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba0145450 con 0x7f5ba0144750 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.089+0000 7f5ba584d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba015c000 con 0x7f5ba006eb10 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.089+0000 7f5b97fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ba00cf030 0x7f5ba00b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.089+0000 7f5b97fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ba00cf030 0x7f5ba00b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54136/0 (socket says 172.21.15.19:54136) 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.089+0000 7f5b97fff640 1 -- 172.21.15.19:0/2602627992 learned_addr learned my addr 172.21.15.19:0/2602627992 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.089+0000 7f5b9f7fe640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba0144750 0x7f5ba00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.089+0000 7f5b9effd640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ba006eb10 0x7f5ba013f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.090+0000 7f5b97fff640 1 -- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ba006eb10 msgr2=0x7f5ba013f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.090+0000 7f5b97fff640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ba006eb10 0x7f5ba013f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.090+0000 7f5b97fff640 1 -- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba0144750 msgr2=0x7f5ba00ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.090+0000 7f5b97fff640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba0144750 0x7f5ba00ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.090+0000 7f5b97fff640 1 -- 172.21.15.19:0/2602627992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba00b3140 con 0x7f5ba00cf030 2024-04-25T09:55:49.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.090+0000 7f5b97fff640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ba00cf030 0x7f5ba00b1b30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800b3f0 tx=0x7f5b8800b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.090+0000 7f5b9cff9640 1 -- 172.21.15.19:0/2602627992 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88015020 con 0x7f5ba00cf030 2024-04-25T09:55:49.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.090+0000 7f5b9cff9640 1 -- 172.21.15.19:0/2602627992 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b88011830 con 0x7f5ba00cf030 2024-04-25T09:55:49.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.090+0000 7f5ba584d640 1 -- 172.21.15.19:0/2602627992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba00b3420 con 0x7f5ba00cf030 2024-04-25T09:55:49.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.090+0000 7f5b9cff9640 1 -- 172.21.15.19:0/2602627992 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88011c00 con 0x7f5ba00cf030 2024-04-25T09:55:49.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.090+0000 7f5ba584d640 1 -- 172.21.15.19:0/2602627992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba00b39e0 con 0x7f5ba00cf030 2024-04-25T09:55:49.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.091+0000 7f5ba584d640 1 -- 172.21.15.19:0/2602627992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba0073990 con 0x7f5ba00cf030 2024-04-25T09:55:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.091+0000 7f5b9cff9640 1 -- 172.21.15.19:0/2602627992 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b88018020 con 0x7f5ba00cf030 2024-04-25T09:55:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.092+0000 7f5b9cff9640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b8006b4d0 0x7f5b8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.092+0000 7f5b9cff9640 1 -- 172.21.15.19:0/2602627992 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5b88089d80 con 0x7f5ba00cf030 2024-04-25T09:55:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.092+0000 7f5b9effd640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b8006b4d0 0x7f5b8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.092+0000 7f5b9effd640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b8006b4d0 0x7f5b8006d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009990 tx=0x7f5b90009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.096+0000 7f5b9cff9640 1 -- 172.21.15.19:0/2602627992 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b8801f020 con 0x7f5ba00cf030 2024-04-25T09:55:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.349+0000 7f5ba584d640 1 -- 172.21.15.19:0/2602627992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5ba01f37b0 con 0x7f5ba00cf030 2024-04-25T09:55:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.350+0000 7f5b9cff9640 1 -- 172.21.15.19:0/2602627992 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5b88058e50 con 0x7f5ba00cf030 2024-04-25T09:55:49.353 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:49.353 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:55:49.353 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T09:55:49.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.353 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T09:55:49.353 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T09:55:49.353 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.353 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:55:49.353 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:55:49.354 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.355 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.356 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.357 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:49.358 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.359 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.360 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T09:55:49.361 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T09:55:49.362 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.362 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:55:49.362 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T09:55:49.362 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:55:49.362 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.362 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T09:55:49.362 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T09:55:49.362 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:55:49.363 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T09:55:49.365 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:55:49.366 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.367 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:49.367 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T09:55:49.367 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:55:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 -- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b8006b4d0 msgr2=0x7f5b8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b8006b4d0 0x7f5b8006d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009990 tx=0x7f5b90009110 comp rx=0 tx=0).stop 2024-04-25T09:55:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 -- 172.21.15.19:0/2602627992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ba00cf030 msgr2=0x7f5ba00b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ba00cf030 0x7f5ba00b1b30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800b3f0 tx=0x7f5b8800b7f0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 -- 172.21.15.19:0/2602627992 shutdown_connections 2024-04-25T09:55:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b8006b4d0 0x7f5b8006d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba0144750 0x7f5ba00ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ba00cf030 0x7f5ba00b1b30 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 --2- 172.21.15.19:0/2602627992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ba006eb10 0x7f5ba013f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 -- 172.21.15.19:0/2602627992 >> 172.21.15.19:0/2602627992 conn(0x7f5ba006abe0 msgr2=0x7f5ba0143750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:49.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 -- 172.21.15.19:0/2602627992 shutdown_connections 2024-04-25T09:55:49.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.352+0000 7f5ba584d640 1 -- 172.21.15.19:0/2602627992 wait complete. 2024-04-25T09:55:49.369 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:49.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.467+0000 7f0133c1f640 1 Processor -- start 2024-04-25T09:55:49.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.467+0000 7f0133c1f640 1 -- start start 2024-04-25T09:55:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f0133c1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c06abe0 0x7f012c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f0133c1f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f012c15d370 0x7f012c15d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f0133c1f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f012c1626e0 con 0x7f012c0ce330 2024-04-25T09:55:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f0133c1f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012c06b500 con 0x7f012c15d370 2024-04-25T09:55:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f0133c1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012c070270 con 0x7f012c06abe0 2024-04-25T09:55:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f01321bc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f012c15d370 0x7f012c15d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f01319bb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c06abe0 0x7f012c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f01311ba640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f012c0ce330 0x7f012c06fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:58882/0 (socket says 172.21.15.19:58882) 2024-04-25T09:55:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f01321bc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f012c15d370 0x7f012c15d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:47506/0 (socket says 172.21.15.19:47506) 2024-04-25T09:55:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f01319bb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c06abe0 0x7f012c06afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:44708/0 (socket says 172.21.15.19:44708) 2024-04-25T09:55:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f01311ba640 1 -- 172.21.15.19:0/3413957582 learned_addr learned my addr 172.21.15.19:0/3413957582 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f01321bc640 1 -- 172.21.15.19:0/3413957582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c06abe0 msgr2=0x7f012c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f01321bc640 1 --2- 172.21.15.19:0/3413957582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c06abe0 0x7f012c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f01321bc640 1 -- 172.21.15.19:0/3413957582 >> v1:172.21.15.112:6789/0 conn(0x7f012c0ce330 legacy=0x7f012c06fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f01321bc640 1 -- 172.21.15.19:0/3413957582 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c0cd270 con 0x7f012c15d370 2024-04-25T09:55:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.468+0000 7f01319bb640 1 --2- 172.21.15.19:0/3413957582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c06abe0 0x7f012c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.469+0000 7f01321bc640 1 --2- 172.21.15.19:0/3413957582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012c15d370 0x7f012c15d800 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0124009d90 tx=0x7f012402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b15ab786b0297c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.469+0000 7f01309b9640 1 -- 172.21.15.19:0/3413957582 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124031600 con 0x7f012c15d370 2024-04-25T09:55:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.469+0000 7f01309b9640 1 -- 172.21.15.19:0/3413957582 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01240317a0 con 0x7f012c15d370 2024-04-25T09:55:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.469+0000 7f01309b9640 1 -- 172.21.15.19:0/3413957582 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124031b70 con 0x7f012c15d370 2024-04-25T09:55:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.470+0000 7f0133c1f640 1 -- 172.21.15.19:0/3413957582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012c15d370 msgr2=0x7f012c15d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.470+0000 7f0133c1f640 1 --2- 172.21.15.19:0/3413957582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012c15d370 0x7f012c15d800 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0124009d90 tx=0x7f012402a5d0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.470+0000 7f0133c1f640 1 -- 172.21.15.19:0/3413957582 shutdown_connections 2024-04-25T09:55:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.470+0000 7f0133c1f640 1 --2- 172.21.15.19:0/3413957582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012c15d370 0x7f012c15d800 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.470+0000 7f0133c1f640 1 --2- 172.21.15.19:0/3413957582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c06abe0 0x7f012c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.470+0000 7f0133c1f640 1 -- 172.21.15.19:0/3413957582 >> 172.21.15.19:0/3413957582 conn(0x7f012c0b2220 msgr2=0x7f012c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.470+0000 7f0133c1f640 1 -- 172.21.15.19:0/3413957582 shutdown_connections 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.470+0000 7f0133c1f640 1 -- 172.21.15.19:0/3413957582 wait complete. 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.471+0000 7f0133c1f640 1 Processor -- start 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.471+0000 7f0133c1f640 1 -- start start 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f0133c1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012c06abe0 0x7f012c0b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f0133c1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f012c0ce330 0x7f012c155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f0133c1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c15d370 0x7f012c1e8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f0133c1f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0cdb10 con 0x7f012c0ce330 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f0133c1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0cd830 con 0x7f012c06abe0 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f0133c1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012c161780 con 0x7f012c15d370 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01311ba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f012c0ce330 0x7f012c155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01311ba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f012c0ce330 0x7f012c155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54138/0 (socket says 172.21.15.19:54138) 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01311ba640 1 -- 172.21.15.19:0/1016619244 learned_addr learned my addr 172.21.15.19:0/1016619244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01319bb640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012c06abe0 0x7f012c0b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01321bc640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c15d370 0x7f012c1e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01311ba640 1 -- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c15d370 msgr2=0x7f012c1e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01311ba640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c15d370 0x7f012c1e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01311ba640 1 -- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012c06abe0 msgr2=0x7f012c0b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01311ba640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012c06abe0 0x7f012c0b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01311ba640 1 -- 172.21.15.19:0/1016619244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c1e8c40 con 0x7f012c0ce330 2024-04-25T09:55:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01319bb640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012c06abe0 0x7f012c0b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.472+0000 7f01311ba640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f012c0ce330 0x7f012c155790 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f011c00c400 tx=0x7f011c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.473+0000 7f01227fc640 1 -- 172.21.15.19:0/1016619244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c017070 con 0x7f012c0ce330 2024-04-25T09:55:49.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.473+0000 7f01227fc640 1 -- 172.21.15.19:0/1016619244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011c005ce0 con 0x7f012c0ce330 2024-04-25T09:55:49.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.473+0000 7f0133c1f640 1 -- 172.21.15.19:0/1016619244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012c157ca0 con 0x7f012c0ce330 2024-04-25T09:55:49.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.473+0000 7f01227fc640 1 -- 172.21.15.19:0/1016619244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c0072f0 con 0x7f012c0ce330 2024-04-25T09:55:49.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.473+0000 7f0133c1f640 1 -- 172.21.15.19:0/1016619244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012c158260 con 0x7f012c0ce330 2024-04-25T09:55:49.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.474+0000 7f0133c1f640 1 -- 172.21.15.19:0/1016619244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012c073990 con 0x7f012c0ce330 2024-04-25T09:55:49.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.474+0000 7f01227fc640 1 -- 172.21.15.19:0/1016619244 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f011c01a070 con 0x7f012c0ce330 2024-04-25T09:55:49.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.475+0000 7f01227fc640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f011006b4d0 0x7f011006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.475+0000 7f01227fc640 1 -- 172.21.15.19:0/1016619244 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f011c089340 con 0x7f012c0ce330 2024-04-25T09:55:49.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.475+0000 7f01319bb640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f011006b4d0 0x7f011006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.475+0000 7f01319bb640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f011006b4d0 0x7f011006d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0128009560 tx=0x7f0128009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.479+0000 7f01227fc640 1 -- 172.21.15.19:0/1016619244 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f011c057dc0 con 0x7f012c0ce330 2024-04-25T09:55:49.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.658+0000 7f0133c1f640 1 -- 172.21.15.19:0/1016619244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f012c0b2ff0 con 0x7f012c0ce330 2024-04-25T09:55:49.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.658+0000 7f01227fc640 1 -- 172.21.15.19:0/1016619244 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+13152 (secure 0 0 0) 0x7f011c05be60 con 0x7f012c0ce330 2024-04-25T09:55:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:49.661 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":49,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:48.971147+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:49.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 -- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f011006b4d0 msgr2=0x7f011006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f011006b4d0 0x7f011006d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0128009560 tx=0x7f0128009190 comp rx=0 tx=0).stop 2024-04-25T09:55:49.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 -- 172.21.15.19:0/1016619244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f012c0ce330 msgr2=0x7f012c155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f012c0ce330 0x7f012c155790 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f011c00c400 tx=0x7f011c00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 -- 172.21.15.19:0/1016619244 shutdown_connections 2024-04-25T09:55:49.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f011006b4d0 0x7f011006d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f012c15d370 0x7f012c1e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f012c0ce330 0x7f012c155790 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 --2- 172.21.15.19:0/1016619244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f012c06abe0 0x7f012c0b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 -- 172.21.15.19:0/1016619244 >> 172.21.15.19:0/1016619244 conn(0x7f012c0b2220 msgr2=0x7f012c0ad6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:49.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 -- 172.21.15.19:0/1016619244 shutdown_connections 2024-04-25T09:55:49.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.660+0000 7f0133c1f640 1 -- 172.21.15.19:0/1016619244 wait complete. 2024-04-25T09:55:49.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:55:49.675 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.775+0000 7f13837fe640 1 Processor -- start 2024-04-25T09:55:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.776+0000 7f13837fe640 1 -- start start 2024-04-25T09:55:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.776+0000 7f13837fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f137c0ced80 0x7f137c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.776+0000 7f13837fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f137c161710 0x7f137c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.776+0000 7f13837fe640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137c15e000 con 0x7f137c15dc20 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.776+0000 7f13837fe640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137c0d1de0 con 0x7f137c0ced80 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.776+0000 7f13837fe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c161c60 con 0x7f137c161710 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.776+0000 7f1380d99640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f137c0ced80 0x7f137c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.777+0000 7f138159a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f137c15dc20 0x7f137c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35554/0 (socket says 172.21.15.19:35554) 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.777+0000 7f138159a640 1 -- 172.21.15.19:0/2204086287 learned_addr learned my addr 172.21.15.19:0/2204086287 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.777+0000 7f1381d9b640 1 --2- 172.21.15.19:0/2204086287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f137c161710 0x7f137c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.777+0000 7f1380d99640 1 -- 172.21.15.19:0/2204086287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f137c161710 msgr2=0x7f137c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.777+0000 7f1380d99640 1 --2- 172.21.15.19:0/2204086287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f137c161710 0x7f137c0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.777+0000 7f1380d99640 1 -- 172.21.15.19:0/2204086287 >> v1:172.21.15.19:6789/0 conn(0x7f137c15dc20 legacy=0x7f137c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.777+0000 7f1373fff640 1 -- 172.21.15.19:0/2204086287 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076704102 0 0) 0x7f137c15e000 con 0x7f137c15dc20 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.777+0000 7f1380d99640 1 -- 172.21.15.19:0/2204086287 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c161f40 con 0x7f137c0ced80 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f1380d99640 1 --2- 172.21.15.19:0/2204086287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f137c0ced80 0x7f137c0d22e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f136c0097a0 tx=0x7f136c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd015b42d41a796e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f1373fff640 1 -- 172.21.15.19:0/2204086287 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c037070 con 0x7f137c0ced80 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f1373fff640 1 -- 172.21.15.19:0/2204086287 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c032440 con 0x7f137c0ced80 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f1373fff640 1 -- 172.21.15.19:0/2204086287 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c032810 con 0x7f137c0ced80 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f13837fe640 1 -- 172.21.15.19:0/2204086287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f137c0ced80 msgr2=0x7f137c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f13837fe640 1 --2- 172.21.15.19:0/2204086287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f137c0ced80 0x7f137c0d22e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f136c0097a0 tx=0x7f136c02a180 comp rx=0 tx=0).stop 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f13837fe640 1 -- 172.21.15.19:0/2204086287 shutdown_connections 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f13837fe640 1 --2- 172.21.15.19:0/2204086287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f137c161710 0x7f137c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f13837fe640 1 --2- 172.21.15.19:0/2204086287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f137c0ced80 0x7f137c0d22e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f13837fe640 1 -- 172.21.15.19:0/2204086287 >> 172.21.15.19:0/2204086287 conn(0x7f137c0d0420 msgr2=0x7f137c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f13837fe640 1 -- 172.21.15.19:0/2204086287 shutdown_connections 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.778+0000 7f13837fe640 1 -- 172.21.15.19:0/2204086287 wait complete. 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.780+0000 7f13837fe640 1 Processor -- start 2024-04-25T09:55:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.780+0000 7f13837fe640 1 -- start start 2024-04-25T09:55:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.780+0000 7f13837fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c0ced80 0x7f137c1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.780+0000 7f13837fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f137c15dc20 0x7f137c0b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.780+0000 7f13837fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f137c161710 0x7f137c0b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.780+0000 7f13837fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0b5e50 con 0x7f137c161710 2024-04-25T09:55:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.780+0000 7f13837fe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c162500 con 0x7f137c0ced80 2024-04-25T09:55:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.780+0000 7f13837fe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0b5fd0 con 0x7f137c15dc20 2024-04-25T09:55:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.780+0000 7f1381d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f137c161710 0x7f137c0b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.780+0000 7f1381d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f137c161710 0x7f137c0b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54162/0 (socket says 172.21.15.19:54162) 2024-04-25T09:55:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.780+0000 7f1381d9b640 1 -- 172.21.15.19:0/1123303249 learned_addr learned my addr 172.21.15.19:0/1123303249 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f138159a640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c0ced80 0x7f137c1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f1380d99640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f137c15dc20 0x7f137c0b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f1381d9b640 1 -- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f137c15dc20 msgr2=0x7f137c0b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f1381d9b640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f137c15dc20 0x7f137c0b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f1381d9b640 1 -- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c0ced80 msgr2=0x7f137c1553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f1381d9b640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c0ced80 0x7f137c1553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f1381d9b640 1 -- 172.21.15.19:0/1123303249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c0c7b00 con 0x7f137c161710 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f138159a640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c0ced80 0x7f137c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f1381d9b640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f137c161710 0x7f137c0b4c90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1374007dc0 tx=0x7f137400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f1371ffb640 1 -- 172.21.15.19:0/1123303249 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374014420 con 0x7f137c161710 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f1371ffb640 1 -- 172.21.15.19:0/1123303249 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1374019070 con 0x7f137c161710 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f1371ffb640 1 -- 172.21.15.19:0/1123303249 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374014820 con 0x7f137c161710 2024-04-25T09:55:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f13837fe640 1 -- 172.21.15.19:0/1123303249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137c0c7de0 con 0x7f137c161710 2024-04-25T09:55:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.781+0000 7f13837fe640 1 -- 172.21.15.19:0/1123303249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c0c83a0 con 0x7f137c161710 2024-04-25T09:55:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.782+0000 7f13837fe640 1 -- 172.21.15.19:0/1123303249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137c073a00 con 0x7f137c161710 2024-04-25T09:55:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.783+0000 7f1371ffb640 1 -- 172.21.15.19:0/1123303249 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13740149c0 con 0x7f137c161710 2024-04-25T09:55:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.783+0000 7f1371ffb640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136006b4d0 0x7f136006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.783+0000 7f1371ffb640 1 -- 172.21.15.19:0/1123303249 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f137408a840 con 0x7f137c161710 2024-04-25T09:55:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.783+0000 7f138159a640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136006b4d0 0x7f136006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.784+0000 7f138159a640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136006b4d0 0x7f136006d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1378000f60 tx=0x7f1378009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.787+0000 7f1371ffb640 1 -- 172.21.15.19:0/1123303249 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1374059890 con 0x7f137c161710 2024-04-25T09:55:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.968+0000 7f13837fe640 1 -- 172.21.15.19:0/1123303249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f137c0d16b0 con 0x7f137c161710 2024-04-25T09:55:49.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.969+0000 7f1371ffb640 1 -- 172.21.15.19:0/1123303249 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1374059190 con 0x7f137c161710 2024-04-25T09:55:49.972 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:55:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 -- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136006b4d0 msgr2=0x7f136006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136006b4d0 0x7f136006d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1378000f60 tx=0x7f1378009040 comp rx=0 tx=0).stop 2024-04-25T09:55:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 -- 172.21.15.19:0/1123303249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f137c161710 msgr2=0x7f137c0b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f137c161710 0x7f137c0b4c90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1374007dc0 tx=0x7f137400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 -- 172.21.15.19:0/1123303249 shutdown_connections 2024-04-25T09:55:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136006b4d0 0x7f136006d990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f137c161710 0x7f137c0b4c90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f137c15dc20 0x7f137c0b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 --2- 172.21.15.19:0/1123303249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c0ced80 0x7f137c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 -- 172.21.15.19:0/1123303249 >> 172.21.15.19:0/1123303249 conn(0x7f137c0d0420 msgr2=0x7f137c15cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 -- 172.21.15.19:0/1123303249 shutdown_connections 2024-04-25T09:55:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:49.971+0000 7f13837fe640 1 -- 172.21.15.19:0/1123303249 wait complete. 2024-04-25T09:55:49.985 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:55:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.083+0000 7febf2c6f640 1 Processor -- start 2024-04-25T09:55:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.083+0000 7febf2c6f640 1 -- start start 2024-04-25T09:55:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.084+0000 7febf2c6f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7febec15dd30 0x7febec0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.084+0000 7febf2c6f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febec0d1df0 con 0x7febec0b9770 2024-04-25T09:55:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.084+0000 7febf2c6f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febec0d20d0 con 0x7febec0d0ad0 2024-04-25T09:55:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.084+0000 7febf2c6f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febec0d23b0 con 0x7febec15dd30 2024-04-25T09:55:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.084+0000 7febebfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7febec0b9770 0x7febec0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35558/0 (socket says 172.21.15.19:35558) 2024-04-25T09:55:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.084+0000 7febebfff640 1 -- 172.21.15.19:0/3443950870 learned_addr learned my addr 172.21.15.19:0/3443950870 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.084+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000524194 0 0) 0x7febec0d1df0 con 0x7febec0b9770 2024-04-25T09:55:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febf0a0b640 1 --2- 172.21.15.19:0/3443950870 >> v2:172.21.15.112:3300/0 conn(0x7febec15dd30 0x7febec0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febd4003520 con 0x7febec0b9770 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2504466328 0 0) 0x7febd4003520 con 0x7febec0b9770 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 >> v2:172.21.15.112:3300/0 conn(0x7febec15dd30 msgr2=0x7febec0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 --2- 172.21.15.19:0/3443950870 >> v2:172.21.15.112:3300/0 conn(0x7febec15dd30 0x7febec0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 >> v1:172.21.15.112:6789/0 conn(0x7febec0d0ad0 legacy=0x7febec0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec0d2690 con 0x7febec0b9770 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7febe00017f0 con 0x7febec0b9770 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febf0a0b640 1 --2- 172.21.15.19:0/3443950870 >> v2:172.21.15.112:3300/0 conn(0x7febec15dd30 0x7febec0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 >> v1:172.21.15.19:6789/0 conn(0x7febec0b9770 legacy=0x7febec0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 --2- 172.21.15.19:0/3443950870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febd40039b0 0x7febd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 --2- 172.21.15.19:0/3443950870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febd4004800 0x7febd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 --2- 172.21.15.19:0/3443950870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febd40053c0 0x7febd400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec0d23b0 con 0x7febd40053c0 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd400c250 con 0x7febd40039b0 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd400c450 con 0x7febd4004800 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febf120c640 1 --2- 172.21.15.19:0/3443950870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febd40053c0 0x7febd400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febf0a0b640 1 --2- 172.21.15.19:0/3443950870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febd40039b0 0x7febd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febebfff640 1 --2- 172.21.15.19:0/3443950870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febd4004800 0x7febd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febf120c640 1 -- 172.21.15.19:0/3443950870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febd4004800 msgr2=0x7febd4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febf120c640 1 --2- 172.21.15.19:0/3443950870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febd4004800 0x7febd4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febf120c640 1 -- 172.21.15.19:0/3443950870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febd40039b0 msgr2=0x7febd4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febf120c640 1 --2- 172.21.15.19:0/3443950870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febd40039b0 0x7febd4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.085+0000 7febf120c640 1 -- 172.21.15.19:0/3443950870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd4003520 con 0x7febd40053c0 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.086+0000 7febf120c640 1 -- 172.21.15.19:0/3443950870 reap_dead start 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.086+0000 7febf120c640 1 --2- 172.21.15.19:0/3443950870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febd40053c0 0x7febd400b9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7febdc001750 tx=0x7febd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f75725991793911 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.086+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc001370 con 0x7febd40053c0 2024-04-25T09:55:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.086+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc005310 con 0x7febd40053c0 2024-04-25T09:55:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.086+0000 7febeb7fe640 1 -- 172.21.15.19:0/3443950870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc02abe0 con 0x7febd40053c0 2024-04-25T09:55:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.087+0000 7febf2c6f640 1 -- 172.21.15.19:0/3443950870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febd40053c0 msgr2=0x7febd400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.087+0000 7febf2c6f640 1 --2- 172.21.15.19:0/3443950870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febd40053c0 0x7febd400b9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7febdc001750 tx=0x7febd4005830 comp rx=0 tx=0).stop 2024-04-25T09:55:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.087+0000 7febf2c6f640 1 -- 172.21.15.19:0/3443950870 shutdown_connections 2024-04-25T09:55:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.087+0000 7febf2c6f640 1 --2- 172.21.15.19:0/3443950870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febd40053c0 0x7febd400b9c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.087+0000 7febf2c6f640 1 -- 172.21.15.19:0/3443950870 >> 172.21.15.19:0/3443950870 conn(0x7febec061c30 msgr2=0x7febec06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.087+0000 7febf2c6f640 1 -- 172.21.15.19:0/3443950870 shutdown_connections 2024-04-25T09:55:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.087+0000 7febf2c6f640 1 -- 172.21.15.19:0/3443950870 wait complete. 2024-04-25T09:55:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.088+0000 7febf2c6f640 1 Processor -- start 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.088+0000 7febf2c6f640 1 -- start start 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febf2c6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febd40053c0 0x7febec0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febf2c6f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febec1529f0 0x7febec1f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febf2c6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febec159a40 0x7febec159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febf2c6f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec0d1b90 con 0x7febec1529f0 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febf2c6f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec0d1a10 con 0x7febd40053c0 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febf2c6f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febec0d2e90 con 0x7febec159a40 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febec1529f0 0x7febec1f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febec1529f0 0x7febec1f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54180/0 (socket says 172.21.15.19:54180) 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febebfff640 1 -- 172.21.15.19:0/3089276451 learned_addr learned my addr 172.21.15.19:0/3089276451 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febf120c640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febec159a40 0x7febec159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febf0a0b640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febd40053c0 0x7febec0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febebfff640 1 -- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febec159a40 msgr2=0x7febec159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febebfff640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febec159a40 0x7febec159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febebfff640 1 -- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febd40053c0 msgr2=0x7febec0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febebfff640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febd40053c0 0x7febec0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febebfff640 1 -- 172.21.15.19:0/3089276451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec152e70 con 0x7febec1529f0 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.089+0000 7febf0a0b640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febd40053c0 0x7febec0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.090+0000 7febebfff640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febec1529f0 0x7febec1f45a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7febe000f800 tx=0x7febe000fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.090+0000 7febe97fa640 1 -- 172.21.15.19:0/3089276451 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe00072e0 con 0x7febec1529f0 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.090+0000 7febe97fa640 1 -- 172.21.15.19:0/3089276451 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe0018070 con 0x7febec1529f0 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.090+0000 7febf2c6f640 1 -- 172.21.15.19:0/3089276451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febec159260 con 0x7febec1529f0 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.090+0000 7febe97fa640 1 -- 172.21.15.19:0/3089276451 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe00057a0 con 0x7febec1529f0 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.090+0000 7febf2c6f640 1 -- 172.21.15.19:0/3089276451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febec157660 con 0x7febec1529f0 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.091+0000 7febf2c6f640 1 -- 172.21.15.19:0/3089276451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febec0ce000 con 0x7febec1529f0 2024-04-25T09:55:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.091+0000 7febe97fa640 1 -- 172.21.15.19:0/3089276451 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7febe0005940 con 0x7febec1529f0 2024-04-25T09:55:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.092+0000 7febe97fa640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7febd806b4d0 0x7febd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.092+0000 7febe97fa640 1 -- 172.21.15.19:0/3089276451 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5249+0+0 (secure 0 0 0) 0x7febe0089ff0 con 0x7febec1529f0 2024-04-25T09:55:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.092+0000 7febf0a0b640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7febd806b4d0 0x7febd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.092+0000 7febf0a0b640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7febd806b4d0 0x7febd806d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7febe4007330 tx=0x7febe400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.096+0000 7febe97fa640 1 -- 172.21.15.19:0/3089276451 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7febe001a310 con 0x7febec1529f0 2024-04-25T09:55:50.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.277+0000 7febf2c6f640 1 -- 172.21.15.19:0/3089276451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7febec073990 con 0x7febec1529f0 2024-04-25T09:55:50.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.277+0000 7febe97fa640 1 -- 172.21.15.19:0/3089276451 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v49) v1 ==== 84+0+65 (secure 0 0 0) 0x7febe0059030 con 0x7febec1529f0 2024-04-25T09:55:50.280 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:50.280 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T09:55:50.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:55:50.280 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T09:55:50.280 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T09:55:50.281 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:55:50.281 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T09:55:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 -- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7febd806b4d0 msgr2=0x7febd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7febd806b4d0 0x7febd806d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7febe4007330 tx=0x7febe400b040 comp rx=0 tx=0).stop 2024-04-25T09:55:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 -- 172.21.15.19:0/3089276451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febec1529f0 msgr2=0x7febec1f45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febec1529f0 0x7febec1f45a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7febe000f800 tx=0x7febe000fcc0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 -- 172.21.15.19:0/3089276451 shutdown_connections 2024-04-25T09:55:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7febd806b4d0 0x7febd806d990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febec159a40 0x7febec159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febec1529f0 0x7febec1f45a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 --2- 172.21.15.19:0/3089276451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febd40053c0 0x7febec0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 -- 172.21.15.19:0/3089276451 >> 172.21.15.19:0/3089276451 conn(0x7febec061c30 msgr2=0x7febec0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 -- 172.21.15.19:0/3089276451 shutdown_connections 2024-04-25T09:55:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.279+0000 7febf2c6f640 1 -- 172.21.15.19:0/3089276451 wait complete. 2024-04-25T09:55:50.293 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:55:50.293 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:55:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.395+0000 7fed580ee640 1 Processor -- start 2024-04-25T09:55:50.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.395+0000 7fed580ee640 1 -- start start 2024-04-25T09:55:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.396+0000 7fed580ee640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed500b5620 0x7fed500b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.396+0000 7fed580ee640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5006cf50 con 0x7fed5015b7d0 2024-04-25T09:55:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.396+0000 7fed580ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed500ce530 con 0x7fed5006c2d0 2024-04-25T09:55:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.396+0000 7fed580ee640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed500ce810 con 0x7fed500b5620 2024-04-25T09:55:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.396+0000 7fed55689640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed500b5620 0x7fed500b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.396+0000 7fed55e8a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fed5015b7d0 0x7fed500cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35560/0 (socket says 172.21.15.19:35560) 2024-04-25T09:55:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.396+0000 7fed55689640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed500b5620 0x7fed500b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54190/0 (socket says 172.21.15.19:54190) 2024-04-25T09:55:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.396+0000 7fed55e8a640 1 -- 172.21.15.19:0/2537208439 learned_addr learned my addr 172.21.15.19:0/2537208439 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.396+0000 7fed55689640 1 -- 172.21.15.19:0/2537208439 >> v1:172.21.15.112:6789/0 conn(0x7fed5006c2d0 legacy=0x7fed5006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:50.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.396+0000 7fed55689640 1 -- 172.21.15.19:0/2537208439 >> v1:172.21.15.19:6789/0 conn(0x7fed5015b7d0 legacy=0x7fed500cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.396+0000 7fed55689640 1 -- 172.21.15.19:0/2537208439 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed500ceaf0 con 0x7fed500b5620 2024-04-25T09:55:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.397+0000 7fed55689640 1 --2- 172.21.15.19:0/2537208439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed500b5620 0x7fed500b5ab0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fed400097a0 tx=0x7fed4002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1454d632ca225a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.397+0000 7fed54e88640 1 -- 172.21.15.19:0/2537208439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40037070 con 0x7fed500b5620 2024-04-25T09:55:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.397+0000 7fed54e88640 1 -- 172.21.15.19:0/2537208439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed40032440 con 0x7fed500b5620 2024-04-25T09:55:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.397+0000 7fed54e88640 1 -- 172.21.15.19:0/2537208439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40032810 con 0x7fed500b5620 2024-04-25T09:55:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.398+0000 7fed580ee640 1 -- 172.21.15.19:0/2537208439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed500b5620 msgr2=0x7fed500b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.398+0000 7fed580ee640 1 --2- 172.21.15.19:0/2537208439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed500b5620 0x7fed500b5ab0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fed400097a0 tx=0x7fed4002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.398+0000 7fed580ee640 1 -- 172.21.15.19:0/2537208439 shutdown_connections 2024-04-25T09:55:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.398+0000 7fed580ee640 1 --2- 172.21.15.19:0/2537208439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed500b5620 0x7fed500b5ab0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.398+0000 7fed580ee640 1 -- 172.21.15.19:0/2537208439 >> 172.21.15.19:0/2537208439 conn(0x7fed500d0800 msgr2=0x7fed500d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.398+0000 7fed580ee640 1 -- 172.21.15.19:0/2537208439 shutdown_connections 2024-04-25T09:55:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.398+0000 7fed580ee640 1 -- 172.21.15.19:0/2537208439 wait complete. 2024-04-25T09:55:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.399+0000 7fed580ee640 1 Processor -- start 2024-04-25T09:55:50.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.399+0000 7fed580ee640 1 -- start start 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed580ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5006c2d0 0x7fed50154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed580ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed500b5620 0x7fed500cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed580ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5015b7d0 0x7fed5015cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed580ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5015bd30 con 0x7fed500b5620 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed580ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5015bbb0 con 0x7fed5015b7d0 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed580ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed500d1f40 con 0x7fed5006c2d0 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed55689640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed500b5620 0x7fed500cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed55689640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed500b5620 0x7fed500cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54198/0 (socket says 172.21.15.19:54198) 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed55689640 1 -- 172.21.15.19:0/1010389096 learned_addr learned my addr 172.21.15.19:0/1010389096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed5668b640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5015b7d0 0x7fed5015cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed55e8a640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5006c2d0 0x7fed50154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed55689640 1 -- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5006c2d0 msgr2=0x7fed50154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed55689640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5006c2d0 0x7fed50154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed55689640 1 -- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5015b7d0 msgr2=0x7fed5015cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed55689640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5015b7d0 0x7fed5015cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed55689640 1 -- 172.21.15.19:0/1010389096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed500cd2a0 con 0x7fed500b5620 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed55e8a640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5006c2d0 0x7fed50154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.400+0000 7fed55689640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed500b5620 0x7fed500cb4f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fed400098c0 tx=0x7fed400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.401+0000 7fed467fc640 1 -- 172.21.15.19:0/1010389096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40037040 con 0x7fed500b5620 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.401+0000 7fed467fc640 1 -- 172.21.15.19:0/1010389096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed40042070 con 0x7fed500b5620 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.401+0000 7fed580ee640 1 -- 172.21.15.19:0/1010389096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed500cd540 con 0x7fed500b5620 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.401+0000 7fed467fc640 1 -- 172.21.15.19:0/1010389096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40030a70 con 0x7fed500b5620 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.401+0000 7fed580ee640 1 -- 172.21.15.19:0/1010389096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed500cdb00 con 0x7fed500b5620 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.402+0000 7fed580ee640 1 -- 172.21.15.19:0/1010389096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed50073a00 con 0x7fed500b5620 2024-04-25T09:55:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.402+0000 7fed467fc640 1 -- 172.21.15.19:0/1010389096 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed40030c10 con 0x7fed500b5620 2024-04-25T09:55:50.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.403+0000 7fed467fc640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3406b4d0 0x7fed3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.403+0000 7fed467fc640 1 -- 172.21.15.19:0/1010389096 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fed400aaa20 con 0x7fed500b5620 2024-04-25T09:55:50.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.403+0000 7fed55e8a640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3406b4d0 0x7fed3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.403+0000 7fed55e8a640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3406b4d0 0x7fed3406d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fed4c000f60 tx=0x7fed4c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.407+0000 7fed467fc640 1 -- 172.21.15.19:0/1010389096 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed40079a50 con 0x7fed500b5620 2024-04-25T09:55:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.587+0000 7fed580ee640 1 -- 172.21.15.19:0/1010389096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fed500d1960 con 0x7fed500b5620 2024-04-25T09:55:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.969+0000 7fed467fc640 1 -- 172.21.15.19:0/1010389096 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v50) v1 ==== 138+0+0 (secure 0 0 0) 0x7fed40079350 con 0x7fed500b5620 2024-04-25T09:55:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T09:55:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 -- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3406b4d0 msgr2=0x7fed3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3406b4d0 0x7fed3406d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fed4c000f60 tx=0x7fed4c009040 comp rx=0 tx=0).stop 2024-04-25T09:55:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 -- 172.21.15.19:0/1010389096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed500b5620 msgr2=0x7fed500cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed500b5620 0x7fed500cb4f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fed400098c0 tx=0x7fed400046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 -- 172.21.15.19:0/1010389096 shutdown_connections 2024-04-25T09:55:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed3406b4d0 0x7fed3406d990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed5015b7d0 0x7fed5015cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed500b5620 0x7fed500cb4f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 --2- 172.21.15.19:0/1010389096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed5006c2d0 0x7fed50154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 -- 172.21.15.19:0/1010389096 >> 172.21.15.19:0/1010389096 conn(0x7fed500d0800 msgr2=0x7fed50160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 -- 172.21.15.19:0/1010389096 shutdown_connections 2024-04-25T09:55:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:50.971+0000 7fed580ee640 1 -- 172.21.15.19:0/1010389096 wait complete. 2024-04-25T09:55:50.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:55:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.097+0000 7fe121815640 1 Processor -- start 2024-04-25T09:55:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.097+0000 7fe121815640 1 -- start start 2024-04-25T09:55:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe121815640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe11c0ced80 0x7fe11c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe121815640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe11c161710 0x7fe11c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe121815640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11c15e000 con 0x7fe11c15dc20 2024-04-25T09:55:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe121815640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11c0d1de0 con 0x7fe11c161710 2024-04-25T09:55:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe121815640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c161c60 con 0x7fe11c0ced80 2024-04-25T09:55:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe11b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe11c161710 0x7fe11c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe11b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe11c161710 0x7fe11c0d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54210/0 (socket says 172.21.15.19:54210) 2024-04-25T09:55:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe11b7fe640 1 -- 172.21.15.19:0/3909445527 learned_addr learned my addr 172.21.15.19:0/3909445527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe11affd640 1 --1- 172.21.15.19:0/3909445527 >> v1:172.21.15.19:6789/0 conn(0x7fe11c15dc20 0x7fe11c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35576/0 (socket says 172.21.15.19:35576) 2024-04-25T09:55:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe11a7fc640 1 --2- 172.21.15.19:0/3909445527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe11c0ced80 0x7fe11c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe11b7fe640 1 -- 172.21.15.19:0/3909445527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe11c0ced80 msgr2=0x7fe11c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe11b7fe640 1 --2- 172.21.15.19:0/3909445527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe11c0ced80 0x7fe11c0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe11b7fe640 1 -- 172.21.15.19:0/3909445527 >> v1:172.21.15.19:6789/0 conn(0x7fe11c15dc20 legacy=0x7fe11c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.098+0000 7fe11b7fe640 1 -- 172.21.15.19:0/3909445527 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c161f40 con 0x7fe11c161710 2024-04-25T09:55:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.099+0000 7fe11b7fe640 1 --2- 172.21.15.19:0/3909445527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe11c161710 0x7fe11c0d2820 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe114009e50 tx=0x7fe11402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378edfbeabcc2680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.099+0000 7fe119ffb640 1 -- 172.21.15.19:0/3909445527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114036070 con 0x7fe11c161710 2024-04-25T09:55:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.099+0000 7fe119ffb640 1 -- 172.21.15.19:0/3909445527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe114031600 con 0x7fe11c161710 2024-04-25T09:55:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.099+0000 7fe119ffb640 1 -- 172.21.15.19:0/3909445527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140319d0 con 0x7fe11c161710 2024-04-25T09:55:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.100+0000 7fe121815640 1 -- 172.21.15.19:0/3909445527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe11c161710 msgr2=0x7fe11c0d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.100+0000 7fe121815640 1 --2- 172.21.15.19:0/3909445527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe11c161710 0x7fe11c0d2820 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe114009e50 tx=0x7fe11402a710 comp rx=0 tx=0).stop 2024-04-25T09:55:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.100+0000 7fe121815640 1 -- 172.21.15.19:0/3909445527 shutdown_connections 2024-04-25T09:55:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.100+0000 7fe121815640 1 --2- 172.21.15.19:0/3909445527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe11c161710 0x7fe11c0d2820 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.100+0000 7fe121815640 1 --2- 172.21.15.19:0/3909445527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe11c0ced80 0x7fe11c0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.100+0000 7fe121815640 1 -- 172.21.15.19:0/3909445527 >> 172.21.15.19:0/3909445527 conn(0x7fe11c0d0420 msgr2=0x7fe11c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:51.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.100+0000 7fe121815640 1 -- 172.21.15.19:0/3909445527 shutdown_connections 2024-04-25T09:55:51.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.100+0000 7fe121815640 1 -- 172.21.15.19:0/3909445527 wait complete. 2024-04-25T09:55:51.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.101+0000 7fe121815640 1 Processor -- start 2024-04-25T09:55:51.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.101+0000 7fe121815640 1 -- start start 2024-04-25T09:55:51.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe121815640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe11c0ced80 0x7fe11c15ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe121815640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe11c15dc20 0x7fe11c15c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe121815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe11c161710 0x7fe11c0d3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe121815640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0b5e50 con 0x7fe11c0ced80 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe121815640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c162500 con 0x7fe11c161710 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe121815640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0b5fd0 con 0x7fe11c15dc20 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe11affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe11c0ced80 0x7fe11c15ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe11affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe11c0ced80 0x7fe11c15ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54216/0 (socket says 172.21.15.19:54216) 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe11affd640 1 -- 172.21.15.19:0/4172735359 learned_addr learned my addr 172.21.15.19:0/4172735359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe11b7fe640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe11c161710 0x7fe11c0d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe11a7fc640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe11c15dc20 0x7fe11c15c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe11affd640 1 -- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe11c15dc20 msgr2=0x7fe11c15c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe11affd640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe11c15dc20 0x7fe11c15c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe11affd640 1 -- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe11c161710 msgr2=0x7fe11c0d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe11affd640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe11c161710 0x7fe11c0d3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe11affd640 1 -- 172.21.15.19:0/4172735359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c1e9270 con 0x7fe11c0ced80 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.102+0000 7fe11b7fe640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe11c161710 0x7fe11c0d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.103+0000 7fe11affd640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe11c0ced80 0x7fe11c15ef00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00c3f0 tx=0x7fe10c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.103+0000 7fe103fff640 1 -- 172.21.15.19:0/4172735359 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0074e0 con 0x7fe11c0ced80 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.103+0000 7fe103fff640 1 -- 172.21.15.19:0/4172735359 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c007680 con 0x7fe11c0ced80 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.103+0000 7fe121815640 1 -- 172.21.15.19:0/4172735359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11c15e420 con 0x7fe11c0ced80 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.103+0000 7fe103fff640 1 -- 172.21.15.19:0/4172735359 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c004a00 con 0x7fe11c0ced80 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.103+0000 7fe121815640 1 -- 172.21.15.19:0/4172735359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11c15e9e0 con 0x7fe11c0ced80 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.104+0000 7fe121815640 1 -- 172.21.15.19:0/4172735359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11c073a00 con 0x7fe11c0ced80 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.104+0000 7fe103fff640 1 -- 172.21.15.19:0/4172735359 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe10c01a020 con 0x7fe11c0ced80 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.105+0000 7fe103fff640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10806b4d0 0x7fe10806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.105+0000 7fe103fff640 1 -- 172.21.15.19:0/4172735359 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fe10c089880 con 0x7fe11c0ced80 2024-04-25T09:55:51.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.105+0000 7fe11a7fc640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10806b4d0 0x7fe10806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.106+0000 7fe11a7fc640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10806b4d0 0x7fe10806d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe110006fd0 tx=0x7fe110008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:51.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.109+0000 7fe103fff640 1 -- 172.21.15.19:0/4172735359 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe10c010300 con 0x7fe11c0ced80 2024-04-25T09:55:51.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.291+0000 7fe121815640 1 -- 172.21.15.19:0/4172735359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe11c0d3490 con 0x7fe11c0ced80 2024-04-25T09:55:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.971+0000 7fe103fff640 1 -- 172.21.15.19:0/4172735359 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v51) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe10c0587e0 con 0x7fe11c0ced80 2024-04-25T09:55:51.974 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T09:55:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.974+0000 7fe121815640 1 -- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10806b4d0 msgr2=0x7fe10806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.974+0000 7fe121815640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10806b4d0 0x7fe10806d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe110006fd0 tx=0x7fe110008040 comp rx=0 tx=0).stop 2024-04-25T09:55:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.974+0000 7fe121815640 1 -- 172.21.15.19:0/4172735359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe11c0ced80 msgr2=0x7fe11c15ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.974+0000 7fe121815640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe11c0ced80 0x7fe11c15ef00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00c3f0 tx=0x7fe10c00c8b0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.975+0000 7fe121815640 1 -- 172.21.15.19:0/4172735359 shutdown_connections 2024-04-25T09:55:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.975+0000 7fe121815640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10806b4d0 0x7fe10806d990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.975+0000 7fe121815640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe11c161710 0x7fe11c0d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.975+0000 7fe121815640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe11c15dc20 0x7fe11c15c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.975+0000 7fe121815640 1 --2- 172.21.15.19:0/4172735359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe11c0ced80 0x7fe11c15ef00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.975+0000 7fe121815640 1 -- 172.21.15.19:0/4172735359 >> 172.21.15.19:0/4172735359 conn(0x7fe11c0d0420 msgr2=0x7fe11c0cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:51.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.975+0000 7fe121815640 1 -- 172.21.15.19:0/4172735359 shutdown_connections 2024-04-25T09:55:51.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:51.977+0000 7fe121815640 1 -- 172.21.15.19:0/4172735359 wait complete. 2024-04-25T09:55:52.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:55:52.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.113+0000 7fda5f81f640 1 Processor -- start 2024-04-25T09:55:52.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.113+0000 7fda5f81f640 1 -- start start 2024-04-25T09:55:52.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.114+0000 7fda5f81f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda5815bb50 0x7fda580c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.114+0000 7fda5f81f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda580cfd10 0x7fda580cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.114+0000 7fda5f81f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda580c75e0 con 0x7fda580c70d0 2024-04-25T09:55:52.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.114+0000 7fda5f81f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda580c8680 con 0x7fda5815bb50 2024-04-25T09:55:52.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.114+0000 7fda5f81f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda580c78c0 con 0x7fda580cfd10 2024-04-25T09:55:52.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.115+0000 7fda5ddbc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda580cfd10 0x7fda580cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.115+0000 7fda5d5bb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda5815bb50 0x7fda580c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.115+0000 7fda5cdba640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fda580c70d0 0x7fda580cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:58912/0 (socket says 172.21.15.19:58912) 2024-04-25T09:55:52.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.115+0000 7fda5cdba640 1 -- 172.21.15.19:0/3665317611 learned_addr learned my addr 172.21.15.19:0/3665317611 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:52.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.115+0000 7fda5ddbc640 1 -- 172.21.15.19:0/3665317611 >> v2:172.21.15.112:3300/0 conn(0x7fda5815bb50 msgr2=0x7fda580c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:52.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.115+0000 7fda5ddbc640 1 --2- 172.21.15.19:0/3665317611 >> v2:172.21.15.112:3300/0 conn(0x7fda5815bb50 0x7fda580c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.115+0000 7fda5ddbc640 1 -- 172.21.15.19:0/3665317611 >> v1:172.21.15.112:6789/0 conn(0x7fda580c70d0 legacy=0x7fda580cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.115+0000 7fda5ddbc640 1 -- 172.21.15.19:0/3665317611 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda580c7ba0 con 0x7fda580cfd10 2024-04-25T09:55:52.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.115+0000 7fda5d5bb640 1 --2- 172.21.15.19:0/3665317611 >> v2:172.21.15.112:3300/0 conn(0x7fda5815bb50 0x7fda580c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:52.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.116+0000 7fda5ddbc640 1 --2- 172.21.15.19:0/3665317611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda580cfd10 0x7fda580cbf20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fda480097a0 tx=0x7fda4802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eeac62353423499 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:52.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.116+0000 7fda57fff640 1 -- 172.21.15.19:0/3665317611 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48037070 con 0x7fda580cfd10 2024-04-25T09:55:52.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.116+0000 7fda57fff640 1 -- 172.21.15.19:0/3665317611 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda48032440 con 0x7fda580cfd10 2024-04-25T09:55:52.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.116+0000 7fda57fff640 1 -- 172.21.15.19:0/3665317611 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48032810 con 0x7fda580cfd10 2024-04-25T09:55:52.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.117+0000 7fda5f81f640 1 -- 172.21.15.19:0/3665317611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda580cfd10 msgr2=0x7fda580cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.117+0000 7fda5f81f640 1 --2- 172.21.15.19:0/3665317611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda580cfd10 0x7fda580cbf20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fda480097a0 tx=0x7fda4802a210 comp rx=0 tx=0).stop 2024-04-25T09:55:52.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.117+0000 7fda5f81f640 1 -- 172.21.15.19:0/3665317611 shutdown_connections 2024-04-25T09:55:52.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.117+0000 7fda5f81f640 1 --2- 172.21.15.19:0/3665317611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda580cfd10 0x7fda580cbf20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.117+0000 7fda5f81f640 1 --2- 172.21.15.19:0/3665317611 >> v2:172.21.15.112:3300/0 conn(0x7fda5815bb50 0x7fda580c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.117+0000 7fda5f81f640 1 -- 172.21.15.19:0/3665317611 >> 172.21.15.19:0/3665317611 conn(0x7fda5806f500 msgr2=0x7fda5806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:52.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.117+0000 7fda5f81f640 1 -- 172.21.15.19:0/3665317611 shutdown_connections 2024-04-25T09:55:52.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.117+0000 7fda5f81f640 1 -- 172.21.15.19:0/3665317611 wait complete. 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.118+0000 7fda5f81f640 1 Processor -- start 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.118+0000 7fda5f81f640 1 -- start start 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5f81f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda580c70d0 0x7fda580ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5f81f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda580cfd10 0x7fda580cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5f81f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda5815bb50 0x7fda58151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5f81f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5815f6d0 con 0x7fda580cfd10 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5f81f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda580c8160 con 0x7fda580c70d0 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5f81f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5815f850 con 0x7fda5815bb50 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5cdba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda580cfd10 0x7fda580cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5cdba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda580cfd10 0x7fda580cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54232/0 (socket says 172.21.15.19:54232) 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5cdba640 1 -- 172.21.15.19:0/3926265731 learned_addr learned my addr 172.21.15.19:0/3926265731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:52.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5d5bb640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda580c70d0 0x7fda580ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5ddbc640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda5815bb50 0x7fda58151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5cdba640 1 -- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda5815bb50 msgr2=0x7fda58151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5cdba640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda5815bb50 0x7fda58151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5cdba640 1 -- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda580c70d0 msgr2=0x7fda580ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5cdba640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda580c70d0 0x7fda580ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5cdba640 1 -- 172.21.15.19:0/3926265731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda580cc740 con 0x7fda580cfd10 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda5cdba640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda580cfd10 0x7fda580cbd10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00c500 tx=0x7fda4c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.119+0000 7fda55ffb640 1 -- 172.21.15.19:0/3926265731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c017070 con 0x7fda580cfd10 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.120+0000 7fda55ffb640 1 -- 172.21.15.19:0/3926265731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c005ce0 con 0x7fda580cfd10 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.120+0000 7fda55ffb640 1 -- 172.21.15.19:0/3926265731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c0072f0 con 0x7fda580cfd10 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.120+0000 7fda5f81f640 1 -- 172.21.15.19:0/3926265731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda580cca20 con 0x7fda580cfd10 2024-04-25T09:55:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.120+0000 7fda5f81f640 1 -- 172.21.15.19:0/3926265731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda580cd2c0 con 0x7fda580cfd10 2024-04-25T09:55:52.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.121+0000 7fda5f81f640 1 -- 172.21.15.19:0/3926265731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda58073a00 con 0x7fda580cfd10 2024-04-25T09:55:52.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.121+0000 7fda55ffb640 1 -- 172.21.15.19:0/3926265731 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda4c01a070 con 0x7fda580cfd10 2024-04-25T09:55:52.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.121+0000 7fda55ffb640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4406b4d0 0x7fda4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.122+0000 7fda55ffb640 1 -- 172.21.15.19:0/3926265731 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fda4c0895f0 con 0x7fda580cfd10 2024-04-25T09:55:52.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.122+0000 7fda5d5bb640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4406b4d0 0x7fda4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.122+0000 7fda5d5bb640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4406b4d0 0x7fda4406d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fda580cbad0 tx=0x7fda5000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:52.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.126+0000 7fda55ffb640 1 -- 172.21.15.19:0/3926265731 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda4c057ce0 con 0x7fda580cfd10 2024-04-25T09:55:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.343+0000 7fda5f81f640 1 -- 172.21.15.19:0/3926265731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fda580cf150 con 0x7fda580cfd10 2024-04-25T09:55:52.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:52.990+0000 7fda55ffb640 1 -- 172.21.15.19:0/3926265731 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7fda4c05bd80 con 0x7fda580cfd10 2024-04-25T09:55:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T09:55:53.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.001+0000 7fda5f81f640 1 -- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4406b4d0 msgr2=0x7fda4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.001+0000 7fda5f81f640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4406b4d0 0x7fda4406d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fda580cbad0 tx=0x7fda5000b040 comp rx=0 tx=0).stop 2024-04-25T09:55:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.001+0000 7fda5f81f640 1 -- 172.21.15.19:0/3926265731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda580cfd10 msgr2=0x7fda580cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.001+0000 7fda5f81f640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda580cfd10 0x7fda580cbd10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00c500 tx=0x7fda4c00c9c0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.003+0000 7fda5f81f640 1 -- 172.21.15.19:0/3926265731 shutdown_connections 2024-04-25T09:55:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.003+0000 7fda5f81f640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4406b4d0 0x7fda4406d990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.003+0000 7fda5f81f640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda5815bb50 0x7fda58151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.003+0000 7fda5f81f640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda580cfd10 0x7fda580cbd10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.003+0000 7fda5f81f640 1 --2- 172.21.15.19:0/3926265731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda580c70d0 0x7fda580ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.003+0000 7fda5f81f640 1 -- 172.21.15.19:0/3926265731 >> 172.21.15.19:0/3926265731 conn(0x7fda5806f500 msgr2=0x7fda580cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.004+0000 7fda5f81f640 1 -- 172.21.15.19:0/3926265731 shutdown_connections 2024-04-25T09:55:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.007+0000 7fda5f81f640 1 -- 172.21.15.19:0/3926265731 wait complete. 2024-04-25T09:55:53.032 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:53.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.135+0000 7f9c92370640 1 Processor -- start 2024-04-25T09:55:53.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.135+0000 7f9c92370640 1 -- start start 2024-04-25T09:55:53.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.136+0000 7f9c92370640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c8c15dbe0 0x7f9c8c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.136+0000 7f9c92370640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9c8c0b2cb0 0x7f9c8c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.136+0000 7f9c92370640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8c0b92a0 0x7f9c8c0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.136+0000 7f9c92370640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8c15dfc0 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.136+0000 7f9c92370640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8c0d1a00 con 0x7f9c8c0b2cb0 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.136+0000 7f9c92370640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c0b97a0 con 0x7f9c8c0b92a0 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.136+0000 7f9c8bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c8c15dbe0 0x7f9c8c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.136+0000 7f9c8bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c8c15dbe0 0x7f9c8c0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54242/0 (socket says 172.21.15.19:54242) 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.136+0000 7f9c8bfff640 1 -- 172.21.15.19:0/1785298069 learned_addr learned my addr 172.21.15.19:0/1785298069 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.136+0000 7f9c9090d640 1 --2- 172.21.15.19:0/1785298069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8c0b92a0 0x7f9c8c0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.136+0000 7f9c8b7fe640 1 --2- 172.21.15.19:0/1785298069 >> v2:172.21.15.112:3300/0 conn(0x7f9c8c0b2cb0 0x7f9c8c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.137+0000 7f9c8bfff640 1 -- 172.21.15.19:0/1785298069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8c0b92a0 msgr2=0x7f9c8c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.137+0000 7f9c8bfff640 1 --2- 172.21.15.19:0/1785298069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8c0b92a0 0x7f9c8c0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.137+0000 7f9c8bfff640 1 -- 172.21.15.19:0/1785298069 >> v2:172.21.15.112:3300/0 conn(0x7f9c8c0b2cb0 msgr2=0x7f9c8c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.137+0000 7f9c8bfff640 1 --2- 172.21.15.19:0/1785298069 >> v2:172.21.15.112:3300/0 conn(0x7f9c8c0b2cb0 0x7f9c8c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.137+0000 7f9c8bfff640 1 -- 172.21.15.19:0/1785298069 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c0d1c50 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.137+0000 7f9c9090d640 1 --2- 172.21.15.19:0/1785298069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8c0b92a0 0x7f9c8c0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.137+0000 7f9c8b7fe640 1 --2- 172.21.15.19:0/1785298069 >> v2:172.21.15.112:3300/0 conn(0x7f9c8c0b2cb0 0x7f9c8c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.137+0000 7f9c8bfff640 1 --2- 172.21.15.19:0/1785298069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8c15dbe0 0x7f9c8c0d0910 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c06cff0 tx=0x7f9c8002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824b7951c8e81740 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.137+0000 7f9c8affd640 1 -- 172.21.15.19:0/1785298069 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80037070 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.138+0000 7f9c8affd640 1 -- 172.21.15.19:0/1785298069 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c80032440 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.138+0000 7f9c8affd640 1 -- 172.21.15.19:0/1785298069 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80032810 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.138+0000 7f9c92370640 1 -- 172.21.15.19:0/1785298069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8c15dbe0 msgr2=0x7f9c8c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.138+0000 7f9c92370640 1 --2- 172.21.15.19:0/1785298069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8c15dbe0 0x7f9c8c0d0910 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c06cff0 tx=0x7f9c8002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.138+0000 7f9c92370640 1 -- 172.21.15.19:0/1785298069 shutdown_connections 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.138+0000 7f9c92370640 1 --2- 172.21.15.19:0/1785298069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8c0b92a0 0x7f9c8c0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.138+0000 7f9c92370640 1 --2- 172.21.15.19:0/1785298069 >> v2:172.21.15.112:3300/0 conn(0x7f9c8c0b2cb0 0x7f9c8c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.138+0000 7f9c92370640 1 --2- 172.21.15.19:0/1785298069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8c15dbe0 0x7f9c8c0d0910 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.138+0000 7f9c92370640 1 -- 172.21.15.19:0/1785298069 >> 172.21.15.19:0/1785298069 conn(0x7f9c8c061a20 msgr2=0x7f9c8c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.138+0000 7f9c92370640 1 -- 172.21.15.19:0/1785298069 shutdown_connections 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.138+0000 7f9c92370640 1 -- 172.21.15.19:0/1785298069 wait complete. 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.139+0000 7f9c92370640 1 Processor -- start 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.139+0000 7f9c92370640 1 -- start start 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.140+0000 7f9c92370640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c8c0b2cb0 0x7f9c8c1f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.140+0000 7f9c92370640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8c0b92a0 0x7f9c8c0b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.140+0000 7f9c92370640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8c15dbe0 0x7f9c8c0c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.140+0000 7f9c92370640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c0b6ab0 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.140+0000 7f9c92370640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c0d2210 con 0x7f9c8c0b2cb0 2024-04-25T09:55:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.140+0000 7f9c92370640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c0b6c30 con 0x7f9c8c0b92a0 2024-04-25T09:55:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.140+0000 7f9c9090d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8c15dbe0 0x7f9c8c0c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.140+0000 7f9c9090d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8c15dbe0 0x7f9c8c0c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54258/0 (socket says 172.21.15.19:54258) 2024-04-25T09:55:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.140+0000 7f9c9090d640 1 -- 172.21.15.19:0/182240704 learned_addr learned my addr 172.21.15.19:0/182240704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.140+0000 7f9c8bfff640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c8c0b2cb0 0x7f9c8c1f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.140+0000 7f9c8b7fe640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8c0b92a0 0x7f9c8c0b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c9090d640 1 -- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8c0b92a0 msgr2=0x7f9c8c0b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c9090d640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8c0b92a0 0x7f9c8c0b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c9090d640 1 -- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c8c0b2cb0 msgr2=0x7f9c8c1f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c9090d640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c8c0b2cb0 0x7f9c8c1f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c9090d640 1 -- 172.21.15.19:0/182240704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c0b3fd0 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c8bfff640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c8c0b2cb0 0x7f9c8c1f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c9090d640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8c15dbe0 0x7f9c8c0c9bc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9c8400ce80 tx=0x7f9c8400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c88ff9640 1 -- 172.21.15.19:0/182240704 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8401a070 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c88ff9640 1 -- 172.21.15.19:0/182240704 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8400d880 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c92370640 1 -- 172.21.15.19:0/182240704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8c0b42b0 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c88ff9640 1 -- 172.21.15.19:0/182240704 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8400dc70 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.141+0000 7f9c92370640 1 -- 172.21.15.19:0/182240704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c0b3ca0 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.142+0000 7f9c92370640 1 -- 172.21.15.19:0/182240704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8c0b4480 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.142+0000 7f9c88ff9640 1 -- 172.21.15.19:0/182240704 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c8401d070 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.143+0000 7f9c88ff9640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c7806b4d0 0x7f9c7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.143+0000 7f9c88ff9640 1 -- 172.21.15.19:0/182240704 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c8408aa80 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.143+0000 7f9c8bfff640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c7806b4d0 0x7f9c7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.144+0000 7f9c8bfff640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c7806b4d0 0x7f9c7806d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9c80034000 tx=0x7f9c8002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.147+0000 7f9c88ff9640 1 -- 172.21.15.19:0/182240704 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c84059120 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.329+0000 7f9c92370640 1 -- 172.21.15.19:0/182240704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c8c073990 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.331+0000 7f9c88ff9640 1 -- 172.21.15.19:0/182240704 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f9c8405d1c0 con 0x7f9c8c15dbe0 2024-04-25T09:55:53.334 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:53.334 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":52,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:52.344810+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:50.618180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:51.322489+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.332+0000 7f9c92370640 1 -- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c7806b4d0 msgr2=0x7f9c7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.332+0000 7f9c92370640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c7806b4d0 0x7f9c7806d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9c80034000 tx=0x7f9c8002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.332+0000 7f9c92370640 1 -- 172.21.15.19:0/182240704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8c15dbe0 msgr2=0x7f9c8c0c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.332+0000 7f9c92370640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8c15dbe0 0x7f9c8c0c9bc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9c8400ce80 tx=0x7f9c8400a710 comp rx=0 tx=0).stop 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.333+0000 7f9c92370640 1 -- 172.21.15.19:0/182240704 shutdown_connections 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.333+0000 7f9c92370640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c7806b4d0 0x7f9c7806d990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.333+0000 7f9c92370640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8c15dbe0 0x7f9c8c0c9bc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.333+0000 7f9c92370640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8c0b92a0 0x7f9c8c0b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.333+0000 7f9c92370640 1 --2- 172.21.15.19:0/182240704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c8c0b2cb0 0x7f9c8c1f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.333+0000 7f9c92370640 1 -- 172.21.15.19:0/182240704 >> 172.21.15.19:0/182240704 conn(0x7f9c8c061a20 msgr2=0x7f9c8c0b8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.333+0000 7f9c92370640 1 -- 172.21.15.19:0/182240704 shutdown_connections 2024-04-25T09:55:53.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.333+0000 7f9c92370640 1 -- 172.21.15.19:0/182240704 wait complete. 2024-04-25T09:55:53.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.444+0000 7fee80bfe640 1 Processor -- start 2024-04-25T09:55:53.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.444+0000 7fee80bfe640 1 -- start start 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.445+0000 7fee80bfe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee7c15dc70 0x7fee7c0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.445+0000 7fee80bfe640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7c06c220 con 0x7fee7c09a450 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.445+0000 7fee80bfe640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7c0d0080 con 0x7fee7c06ce10 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.445+0000 7fee80bfe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c09a8b0 con 0x7fee7c15dc70 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.445+0000 7fee79d9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fee7c09a450 0x7fee7c06c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35586/0 (socket says 172.21.15.19:35586) 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.445+0000 7fee79d9b640 1 -- 172.21.15.19:0/3643413203 learned_addr learned my addr 172.21.15.19:0/3643413203 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7a59c640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee7c15dc70 0x7fee7c0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605761903 0 0) 0x7fee7c06c220 con 0x7fee7c09a450 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee68003520 con 0x7fee7c09a450 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 420385303 0 0) 0x7fee68003520 con 0x7fee7c09a450 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee7c15dc70 msgr2=0x7fee7c0d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:53.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee7c15dc70 0x7fee7c0d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 >> v1:172.21.15.112:6789/0 conn(0x7fee7c06ce10 legacy=0x7fee7c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c0d0360 con 0x7fee7c09a450 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fee64001670 con 0x7fee7c09a450 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 >> v1:172.21.15.19:6789/0 conn(0x7fee7c09a450 legacy=0x7fee7c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee680039e0 0x7fee68003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee68004390 0x7fee68004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee7c15dc70 0x7fee680050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c09a8b0 con 0x7fee68004390 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6800b9c0 con 0x7fee680039e0 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6800bbc0 con 0x7fee7c15dc70 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee79d9b640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee68004390 0x7fee68004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7a59c640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee680039e0 0x7fee68003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.446+0000 7fee7ad9d640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee7c15dc70 0x7fee680050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.447+0000 7fee79d9b640 1 -- 172.21.15.19:0/3643413203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee7c15dc70 msgr2=0x7fee680050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.447+0000 7fee79d9b640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee7c15dc70 0x7fee680050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.447+0000 7fee79d9b640 1 -- 172.21.15.19:0/3643413203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee680039e0 msgr2=0x7fee68003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.447+0000 7fee79d9b640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee680039e0 0x7fee68003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.447+0000 7fee79d9b640 1 -- 172.21.15.19:0/3643413203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68003520 con 0x7fee68004390 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.447+0000 7fee79d9b640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee68004390 0x7fee68004860 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fee6400c210 tx=0x7fee68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc41337cc835ae05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.447+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64007ce0 con 0x7fee68004390 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.447+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee64005af0 con 0x7fee68004390 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.447+0000 7fee7959a640 1 -- 172.21.15.19:0/3643413203 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64007070 con 0x7fee68004390 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.448+0000 7fee80bfe640 1 -- 172.21.15.19:0/3643413203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee68004390 msgr2=0x7fee68004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.448+0000 7fee80bfe640 1 --2- 172.21.15.19:0/3643413203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee68004390 0x7fee68004860 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fee6400c210 tx=0x7fee68005830 comp rx=0 tx=0).stop 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.448+0000 7fee7ad9d640 1 -- 172.21.15.19:0/3643413203 reap_dead start 2024-04-25T09:55:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.448+0000 7fee80bfe640 1 -- 172.21.15.19:0/3643413203 shutdown_connections 2024-04-25T09:55:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.448+0000 7fee80bfe640 1 -- 172.21.15.19:0/3643413203 >> 172.21.15.19:0/3643413203 conn(0x7fee7c0b3270 msgr2=0x7fee7c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.448+0000 7fee80bfe640 1 -- 172.21.15.19:0/3643413203 shutdown_connections 2024-04-25T09:55:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.448+0000 7fee80bfe640 1 -- 172.21.15.19:0/3643413203 wait complete. 2024-04-25T09:55:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.449+0000 7fee80bfe640 1 Processor -- start 2024-04-25T09:55:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.449+0000 7fee80bfe640 1 -- start start 2024-04-25T09:55:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.450+0000 7fee80bfe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee68004390 0x7fee7c1f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.450+0000 7fee80bfe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee7c1f7580 0x7fee7c15a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.450+0000 7fee80bfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee7c15aca0 0x7fee7c0b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.450+0000 7fee80bfe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0d0920 con 0x7fee7c1f7580 2024-04-25T09:55:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.450+0000 7fee80bfe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0d0c90 con 0x7fee7c15aca0 2024-04-25T09:55:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.450+0000 7fee80bfe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c15e050 con 0x7fee68004390 2024-04-25T09:55:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.450+0000 7fee79d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee7c1f7580 0x7fee7c15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.450+0000 7fee79d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee7c1f7580 0x7fee7c15a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54276/0 (socket says 172.21.15.19:54276) 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.450+0000 7fee79d9b640 1 -- 172.21.15.19:0/1259283942 learned_addr learned my addr 172.21.15.19:0/1259283942 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.450+0000 7fee7ad9d640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee7c15aca0 0x7fee7c0b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.450+0000 7fee7a59c640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee68004390 0x7fee7c1f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee79d9b640 1 -- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee68004390 msgr2=0x7fee7c1f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee79d9b640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee68004390 0x7fee7c1f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee79d9b640 1 -- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee7c15aca0 msgr2=0x7fee7c0b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee79d9b640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee7c15aca0 0x7fee7c0b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee79d9b640 1 -- 172.21.15.19:0/1259283942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c1f7a00 con 0x7fee7c1f7580 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee7a59c640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee68004390 0x7fee7c1f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee79d9b640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee7c1f7580 0x7fee7c15a760 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fee6400c210 tx=0x7fee7c0d0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee72ffd640 1 -- 172.21.15.19:0/1259283942 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee640015c0 con 0x7fee7c1f7580 2024-04-25T09:55:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee72ffd640 1 -- 172.21.15.19:0/1259283942 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee64042070 con 0x7fee7c1f7580 2024-04-25T09:55:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee80bfe640 1 -- 172.21.15.19:0/1259283942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7c147050 con 0x7fee7c1f7580 2024-04-25T09:55:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee72ffd640 1 -- 172.21.15.19:0/1259283942 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64007270 con 0x7fee7c1f7580 2024-04-25T09:55:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.451+0000 7fee80bfe640 1 -- 172.21.15.19:0/1259283942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c15b7b0 con 0x7fee7c1f7580 2024-04-25T09:55:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.452+0000 7fee80bfe640 1 -- 172.21.15.19:0/1259283942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7c073990 con 0x7fee7c1f7580 2024-04-25T09:55:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.452+0000 7fee72ffd640 1 -- 172.21.15.19:0/1259283942 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee64007410 con 0x7fee7c1f7580 2024-04-25T09:55:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.453+0000 7fee72ffd640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee5c06b500 0x7fee5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.453+0000 7fee72ffd640 1 -- 172.21.15.19:0/1259283942 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fee640aa530 con 0x7fee7c1f7580 2024-04-25T09:55:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.453+0000 7fee7a59c640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee5c06b500 0x7fee5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.454+0000 7fee7a59c640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee5c06b500 0x7fee5c06d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee74009b40 tx=0x7fee74008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.457+0000 7fee72ffd640 1 -- 172.21.15.19:0/1259283942 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee7c1f7a00 con 0x7fee7c1f7580 2024-04-25T09:55:53.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.637+0000 7fee80bfe640 1 -- 172.21.15.19:0/1259283942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee7c15b9b0 con 0x7fee7c1f7580 2024-04-25T09:55:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.639+0000 7fee72ffd640 1 -- 172.21.15.19:0/1259283942 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16943 (secure 0 0 0) 0x7fee64079340 con 0x7fee7c1f7580 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":52,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:52.344810+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:50.618180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:51.322489+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:53.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 -- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee5c06b500 msgr2=0x7fee5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee5c06b500 0x7fee5c06d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee74009b40 tx=0x7fee74008040 comp rx=0 tx=0).stop 2024-04-25T09:55:53.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 -- 172.21.15.19:0/1259283942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee7c1f7580 msgr2=0x7fee7c15a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee7c1f7580 0x7fee7c15a760 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fee6400c210 tx=0x7fee7c0d0c00 comp rx=0 tx=0).stop 2024-04-25T09:55:53.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 -- 172.21.15.19:0/1259283942 shutdown_connections 2024-04-25T09:55:53.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee5c06b500 0x7fee5c06d9c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee7c15aca0 0x7fee7c0b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee7c1f7580 0x7fee7c15a760 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 --2- 172.21.15.19:0/1259283942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee68004390 0x7fee7c1f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 -- 172.21.15.19:0/1259283942 >> 172.21.15.19:0/1259283942 conn(0x7fee7c0b3270 msgr2=0x7fee7c0ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:53.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 -- 172.21.15.19:0/1259283942 shutdown_connections 2024-04-25T09:55:53.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.641+0000 7fee80bfe640 1 -- 172.21.15.19:0/1259283942 wait complete. 2024-04-25T09:55:53.655 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:55:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.755+0000 7fba9e076640 1 Processor -- start 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.755+0000 7fba9e076640 1 -- start start 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.756+0000 7fba9e076640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba980caa60 0x7fba980cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.756+0000 7fba9e076640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fba9806bbf0 0x7fba9806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.756+0000 7fba9e076640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba980cae60 con 0x7fba9815dbd0 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.756+0000 7fba9e076640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba980cc6c0 con 0x7fba9806bbf0 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.756+0000 7fba9e076640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba980cb0e0 con 0x7fba980caa60 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.756+0000 7fba97fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fba9806bbf0 0x7fba9806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.757+0000 7fba97fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fba9806bbf0 0x7fba9806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54278/0 (socket says 172.21.15.19:54278) 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.757+0000 7fba97fff640 1 -- 172.21.15.19:0/956486147 learned_addr learned my addr 172.21.15.19:0/956486147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.757+0000 7fba96ffd640 1 --2- 172.21.15.19:0/956486147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba980caa60 0x7fba980cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.757+0000 7fba97fff640 1 -- 172.21.15.19:0/956486147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba980caa60 msgr2=0x7fba980cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.757+0000 7fba97fff640 1 --2- 172.21.15.19:0/956486147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba980caa60 0x7fba980cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.757+0000 7fba97fff640 1 -- 172.21.15.19:0/956486147 >> v1:172.21.15.112:6789/0 conn(0x7fba9815dbd0 legacy=0x7fba9806ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.757+0000 7fba97fff640 1 -- 172.21.15.19:0/956486147 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba980cb3c0 con 0x7fba9806bbf0 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.757+0000 7fba97fff640 1 --2- 172.21.15.19:0/956486147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba9806bbf0 0x7fba9806c080 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fba90009e80 tx=0x7fba9002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac6cd8e58a8a9fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.758+0000 7fba967fc640 1 -- 172.21.15.19:0/956486147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90036070 con 0x7fba9806bbf0 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.758+0000 7fba967fc640 1 -- 172.21.15.19:0/956486147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba90031600 con 0x7fba9806bbf0 2024-04-25T09:55:53.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.758+0000 7fba967fc640 1 -- 172.21.15.19:0/956486147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba900319d0 con 0x7fba9806bbf0 2024-04-25T09:55:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.758+0000 7fba9e076640 1 -- 172.21.15.19:0/956486147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba9806bbf0 msgr2=0x7fba9806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.758+0000 7fba9e076640 1 --2- 172.21.15.19:0/956486147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba9806bbf0 0x7fba9806c080 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fba90009e80 tx=0x7fba9002a750 comp rx=0 tx=0).stop 2024-04-25T09:55:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.758+0000 7fba9e076640 1 -- 172.21.15.19:0/956486147 shutdown_connections 2024-04-25T09:55:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.758+0000 7fba9e076640 1 --2- 172.21.15.19:0/956486147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba9806bbf0 0x7fba9806c080 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.758+0000 7fba9e076640 1 --2- 172.21.15.19:0/956486147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba980caa60 0x7fba980cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.758+0000 7fba9e076640 1 -- 172.21.15.19:0/956486147 >> 172.21.15.19:0/956486147 conn(0x7fba98071af0 msgr2=0x7fba98071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.758+0000 7fba9e076640 1 -- 172.21.15.19:0/956486147 shutdown_connections 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.758+0000 7fba9e076640 1 -- 172.21.15.19:0/956486147 wait complete. 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.759+0000 7fba9e076640 1 Processor -- start 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.760+0000 7fba9e076640 1 -- start start 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.760+0000 7fba9e076640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba9806bbf0 0x7fba98152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.760+0000 7fba9e076640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba980caa60 0x7fba980c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.760+0000 7fba9e076640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba9815dbd0 0x7fba9815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.760+0000 7fba9e076640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba980cbc60 con 0x7fba980caa60 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.760+0000 7fba9e076640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba980cb980 con 0x7fba9815dbd0 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.760+0000 7fba9e076640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9815dfb0 con 0x7fba9806bbf0 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.760+0000 7fba96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba980caa60 0x7fba980c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.760+0000 7fba96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba980caa60 0x7fba980c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54294/0 (socket says 172.21.15.19:54294) 2024-04-25T09:55:53.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.760+0000 7fba96ffd640 1 -- 172.21.15.19:0/1648860009 learned_addr learned my addr 172.21.15.19:0/1648860009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba977fe640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba9806bbf0 0x7fba98152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba97fff640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba9815dbd0 0x7fba9815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba96ffd640 1 -- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba9806bbf0 msgr2=0x7fba98152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba96ffd640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba9806bbf0 0x7fba98152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba96ffd640 1 -- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba9815dbd0 msgr2=0x7fba9815c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba96ffd640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba9815dbd0 0x7fba9815c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba96ffd640 1 -- 172.21.15.19:0/1648860009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba980c7200 con 0x7fba980caa60 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba96ffd640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba980caa60 0x7fba980c7870 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00d800 tx=0x7fba8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba7ffff640 1 -- 172.21.15.19:0/1648860009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00b6c0 con 0x7fba980caa60 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba7ffff640 1 -- 172.21.15.19:0/1648860009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c019070 con 0x7fba980caa60 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba7ffff640 1 -- 172.21.15.19:0/1648860009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00bac0 con 0x7fba980caa60 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba9e076640 1 -- 172.21.15.19:0/1648860009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba98159a20 con 0x7fba980caa60 2024-04-25T09:55:53.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.761+0000 7fba9e076640 1 -- 172.21.15.19:0/1648860009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba981f37b0 con 0x7fba980caa60 2024-04-25T09:55:53.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.762+0000 7fba9e076640 1 -- 172.21.15.19:0/1648860009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba98073a00 con 0x7fba980caa60 2024-04-25T09:55:53.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.763+0000 7fba7ffff640 1 -- 172.21.15.19:0/1648860009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba8c00bc60 con 0x7fba980caa60 2024-04-25T09:55:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.763+0000 7fba7ffff640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fba8406b4d0 0x7fba8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.763+0000 7fba7ffff640 1 -- 172.21.15.19:0/1648860009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fba8c08a690 con 0x7fba980caa60 2024-04-25T09:55:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.763+0000 7fba977fe640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fba8406b4d0 0x7fba8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.764+0000 7fba977fe640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fba8406b4d0 0x7fba8406d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fba88001780 tx=0x7fba88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.767+0000 7fba7ffff640 1 -- 172.21.15.19:0/1648860009 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba8c058dc0 con 0x7fba980caa60 2024-04-25T09:55:53.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:53.983+0000 7fba9e076640 1 -- 172.21.15.19:0/1648860009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fba98061b30 con 0x7fba980caa60 2024-04-25T09:55:54.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.000+0000 7fba7ffff640 1 -- 172.21.15.19:0/1648860009 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fba8c05ce60 con 0x7fba980caa60 2024-04-25T09:55:54.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 -- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fba8406b4d0 msgr2=0x7fba8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fba8406b4d0 0x7fba8406d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fba88001780 tx=0x7fba88009040 comp rx=0 tx=0).stop 2024-04-25T09:55:54.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 -- 172.21.15.19:0/1648860009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba980caa60 msgr2=0x7fba980c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba980caa60 0x7fba980c7870 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00d800 tx=0x7fba8c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 -- 172.21.15.19:0/1648860009 shutdown_connections 2024-04-25T09:55:54.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fba8406b4d0 0x7fba8406d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba9815dbd0 0x7fba9815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba980caa60 0x7fba980c7870 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 --2- 172.21.15.19:0/1648860009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba9806bbf0 0x7fba98152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 -- 172.21.15.19:0/1648860009 >> 172.21.15.19:0/1648860009 conn(0x7fba98071af0 msgr2=0x7fba980b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:54.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 -- 172.21.15.19:0/1648860009 shutdown_connections 2024-04-25T09:55:54.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.002+0000 7fba9e076640 1 -- 172.21.15.19:0/1648860009 wait complete. 2024-04-25T09:55:54.016 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:55:54.016 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:55:54.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.117+0000 7fa1f435c640 1 Processor -- start 2024-04-25T09:55:54.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.117+0000 7fa1f435c640 1 -- start start 2024-04-25T09:55:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.118+0000 7fa1f435c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec15b980 0x7fa1ec0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.118+0000 7fa1f435c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec072280 con 0x7fa1ec0cc4f0 2024-04-25T09:55:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.118+0000 7fa1f435c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec0714c0 con 0x7fa1ec0ce230 2024-04-25T09:55:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.118+0000 7fa1f435c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec15bdc0 con 0x7fa1ec15b980 2024-04-25T09:55:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.118+0000 7fa1f20f8640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa1ec0cc4f0 0x7fa1ec0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35592/0 (socket says 172.21.15.19:35592) 2024-04-25T09:55:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.118+0000 7fa1f20f8640 1 -- 172.21.15.19:0/1081120134 learned_addr learned my addr 172.21.15.19:0/1081120134 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.118+0000 7fa1f28f9640 1 --2- 172.21.15.19:0/1081120134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec15b980 0x7fa1ec0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.118+0000 7fa1f10f6640 1 -- 172.21.15.19:0/1081120134 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026666259 0 0) 0x7fa1ec072280 con 0x7fa1ec0cc4f0 2024-04-25T09:55:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.118+0000 7fa1f10f6640 1 -- 172.21.15.19:0/1081120134 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d8005730 con 0x7fa1ec0cc4f0 2024-04-25T09:55:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.118+0000 7fa1f10f6640 1 -- 172.21.15.19:0/1081120134 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498749551 0 0) 0x7fa1ec0714c0 con 0x7fa1ec0ce230 2024-04-25T09:55:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.118+0000 7fa1f10f6640 1 -- 172.21.15.19:0/1081120134 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1ec072280 con 0x7fa1ec0ce230 2024-04-25T09:55:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.119+0000 7fa1f10f6640 1 -- 172.21.15.19:0/1081120134 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4259575079 0 0) 0x7fa1d8005730 con 0x7fa1ec0cc4f0 2024-04-25T09:55:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.119+0000 7fa1f28f9640 1 -- 172.21.15.19:0/1081120134 >> v1:172.21.15.112:6789/0 conn(0x7fa1ec0ce230 legacy=0x7fa1ec15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.119+0000 7fa1f28f9640 1 -- 172.21.15.19:0/1081120134 >> v1:172.21.15.19:6789/0 conn(0x7fa1ec0cc4f0 legacy=0x7fa1ec0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.119+0000 7fa1f28f9640 1 -- 172.21.15.19:0/1081120134 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec0717a0 con 0x7fa1ec15b980 2024-04-25T09:55:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.119+0000 7fa1f28f9640 1 --2- 172.21.15.19:0/1081120134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec15b980 0x7fa1ec0cc8d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009dc0 tx=0x7fa1e402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32ceb97f78ae94c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.119+0000 7fa1f10f6640 1 -- 172.21.15.19:0/1081120134 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8001670 con 0x7fa1ec15b980 2024-04-25T09:55:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.119+0000 7fa1f10f6640 1 -- 172.21.15.19:0/1081120134 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e4007540 con 0x7fa1ec15b980 2024-04-25T09:55:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.120+0000 7fa1f10f6640 1 -- 172.21.15.19:0/1081120134 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40078f0 con 0x7fa1ec15b980 2024-04-25T09:55:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.120+0000 7fa1f435c640 1 -- 172.21.15.19:0/1081120134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec15b980 msgr2=0x7fa1ec0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.120+0000 7fa1f435c640 1 --2- 172.21.15.19:0/1081120134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec15b980 0x7fa1ec0cc8d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009dc0 tx=0x7fa1e402a660 comp rx=0 tx=0).stop 2024-04-25T09:55:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.120+0000 7fa1f435c640 1 -- 172.21.15.19:0/1081120134 shutdown_connections 2024-04-25T09:55:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.120+0000 7fa1f435c640 1 --2- 172.21.15.19:0/1081120134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec15b980 0x7fa1ec0cc8d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.120+0000 7fa1f435c640 1 -- 172.21.15.19:0/1081120134 >> 172.21.15.19:0/1081120134 conn(0x7fa1ec06abe0 msgr2=0x7fa1ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.120+0000 7fa1f435c640 1 -- 172.21.15.19:0/1081120134 shutdown_connections 2024-04-25T09:55:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.120+0000 7fa1f435c640 1 -- 172.21.15.19:0/1081120134 wait complete. 2024-04-25T09:55:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.121+0000 7fa1f435c640 1 Processor -- start 2024-04-25T09:55:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.121+0000 7fa1f435c640 1 -- start start 2024-04-25T09:55:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f435c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec0cc4f0 0x7fa1ec142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f435c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec0ce230 0x7fa1ec0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f435c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec15b980 0x7fa1ec0cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f435c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec071d60 con 0x7fa1ec0ce230 2024-04-25T09:55:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f435c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec071270 con 0x7fa1ec15b980 2024-04-25T09:55:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f435c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec06e8a0 con 0x7fa1ec0cc4f0 2024-04-25T09:55:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f18f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec0ce230 0x7fa1ec0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f18f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec0ce230 0x7fa1ec0cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54300/0 (socket says 172.21.15.19:54300) 2024-04-25T09:55:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f18f7640 1 -- 172.21.15.19:0/3589297412 learned_addr learned my addr 172.21.15.19:0/3589297412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f20f8640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec0cc4f0 0x7fa1ec142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f28f9640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec15b980 0x7fa1ec0cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f18f7640 1 -- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec0cc4f0 msgr2=0x7fa1ec142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f18f7640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec0cc4f0 0x7fa1ec142010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f18f7640 1 -- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec15b980 msgr2=0x7fa1ec0cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.122+0000 7fa1f18f7640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec15b980 0x7fa1ec0cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.123+0000 7fa1f18f7640 1 -- 172.21.15.19:0/3589297412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec14f220 con 0x7fa1ec0ce230 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.123+0000 7fa1f18f7640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec0ce230 0x7fa1ec0cf6a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00c430 tx=0x7fa1dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.123+0000 7fa1e2ffd640 1 -- 172.21.15.19:0/3589297412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc017070 con 0x7fa1ec0ce230 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.123+0000 7fa1e2ffd640 1 -- 172.21.15.19:0/3589297412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc0052d0 con 0x7fa1ec0ce230 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.123+0000 7fa1f435c640 1 -- 172.21.15.19:0/3589297412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ec159de0 con 0x7fa1ec0ce230 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.123+0000 7fa1e2ffd640 1 -- 172.21.15.19:0/3589297412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc003830 con 0x7fa1ec0ce230 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.123+0000 7fa1f435c640 1 -- 172.21.15.19:0/3589297412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec15a3a0 con 0x7fa1ec0ce230 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.124+0000 7fa1f435c640 1 -- 172.21.15.19:0/3589297412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ec073a00 con 0x7fa1ec0ce230 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.124+0000 7fa1e2ffd640 1 -- 172.21.15.19:0/3589297412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1dc01a070 con 0x7fa1ec0ce230 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.125+0000 7fa1e2ffd640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1d006b4d0 0x7fa1d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.125+0000 7fa1e2ffd640 1 -- 172.21.15.19:0/3589297412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1dc089810 con 0x7fa1ec0ce230 2024-04-25T09:55:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.125+0000 7fa1f20f8640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1d006b4d0 0x7fa1d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.126+0000 7fa1f20f8640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1d006b4d0 0x7fa1d006d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80011e0 tx=0x7fa1e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.129+0000 7fa1e2ffd640 1 -- 172.21.15.19:0/3589297412 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1dc0585d0 con 0x7fa1ec0ce230 2024-04-25T09:55:54.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:54.351+0000 7fa1f435c640 1 -- 172.21.15.19:0/3589297412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa1ec06d3a0 con 0x7fa1ec0ce230 2024-04-25T09:55:54.971 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T09:55:54.968+0000 7f6cd601e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:55:55.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.007+0000 7fa1e2ffd640 1 -- 172.21.15.19:0/3589297412 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa1dc057ed0 con 0x7fa1ec0ce230 2024-04-25T09:55:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 -- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1d006b4d0 msgr2=0x7fa1d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1d006b4d0 0x7fa1d006d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80011e0 tx=0x7fa1e8009040 comp rx=0 tx=0).stop 2024-04-25T09:55:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 -- 172.21.15.19:0/3589297412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec0ce230 msgr2=0x7fa1ec0cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec0ce230 0x7fa1ec0cf6a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00c430 tx=0x7fa1dc00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 -- 172.21.15.19:0/3589297412 shutdown_connections 2024-04-25T09:55:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1d006b4d0 0x7fa1d006d990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec15b980 0x7fa1ec0cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec0ce230 0x7fa1ec0cf6a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 --2- 172.21.15.19:0/3589297412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec0cc4f0 0x7fa1ec142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 -- 172.21.15.19:0/3589297412 >> 172.21.15.19:0/3589297412 conn(0x7fa1ec06abe0 msgr2=0x7fa1ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:55.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 -- 172.21.15.19:0/3589297412 shutdown_connections 2024-04-25T09:55:55.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.010+0000 7fa1f435c640 1 -- 172.21.15.19:0/3589297412 wait complete. 2024-04-25T09:55:55.026 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:55:55.026 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:55.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.131+0000 7feec1da5640 1 Processor -- start 2024-04-25T09:55:55.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.131+0000 7feec1da5640 1 -- start start 2024-04-25T09:55:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.132+0000 7feec1da5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feebc0ce590 0x7feebc165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.132+0000 7feec1da5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feebc071480 con 0x7feebc0cfec0 2024-04-25T09:55:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.132+0000 7feec1da5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feebc071760 con 0x7feebc154ee0 2024-04-25T09:55:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.132+0000 7feec1da5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc072090 con 0x7feebc0ce590 2024-04-25T09:55:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.132+0000 7feebbfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feebc0ce590 0x7feebc165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.132+0000 7feebaffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feebc0cfec0 0x7feebc0cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35606/0 (socket says 172.21.15.19:35606) 2024-04-25T09:55:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.132+0000 7feebaffd640 1 -- 172.21.15.19:0/920643531 learned_addr learned my addr 172.21.15.19:0/920643531 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.132+0000 7feebbfff640 1 -- 172.21.15.19:0/920643531 >> v1:172.21.15.112:6789/0 conn(0x7feebc154ee0 legacy=0x7feebc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.132+0000 7feebbfff640 1 -- 172.21.15.19:0/920643531 >> v1:172.21.15.19:6789/0 conn(0x7feebc0cfec0 legacy=0x7feebc0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.132+0000 7feebbfff640 1 -- 172.21.15.19:0/920643531 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc071a40 con 0x7feebc0ce590 2024-04-25T09:55:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.133+0000 7feebbfff640 1 --2- 172.21.15.19:0/920643531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feebc0ce590 0x7feebc165900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009d90 tx=0x7feeb402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7374fb5a3c71b656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.133+0000 7feeba7fc640 1 -- 172.21.15.19:0/920643531 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4007740 con 0x7feebc0ce590 2024-04-25T09:55:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.133+0000 7feeba7fc640 1 -- 172.21.15.19:0/920643531 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb40078e0 con 0x7feebc0ce590 2024-04-25T09:55:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.133+0000 7feeba7fc640 1 -- 172.21.15.19:0/920643531 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4007cb0 con 0x7feebc0ce590 2024-04-25T09:55:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.134+0000 7feec1da5640 1 -- 172.21.15.19:0/920643531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feebc0ce590 msgr2=0x7feebc165900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.134+0000 7feec1da5640 1 --2- 172.21.15.19:0/920643531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feebc0ce590 0x7feebc165900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009d90 tx=0x7feeb402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.134+0000 7feec1da5640 1 -- 172.21.15.19:0/920643531 shutdown_connections 2024-04-25T09:55:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.134+0000 7feec1da5640 1 --2- 172.21.15.19:0/920643531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feebc0ce590 0x7feebc165900 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.134+0000 7feec1da5640 1 -- 172.21.15.19:0/920643531 >> 172.21.15.19:0/920643531 conn(0x7feebc06abe0 msgr2=0x7feebc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:55.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.134+0000 7feec1da5640 1 -- 172.21.15.19:0/920643531 shutdown_connections 2024-04-25T09:55:55.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.134+0000 7feec1da5640 1 -- 172.21.15.19:0/920643531 wait complete. 2024-04-25T09:55:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.135+0000 7feec1da5640 1 Processor -- start 2024-04-25T09:55:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.135+0000 7feec1da5640 1 -- start start 2024-04-25T09:55:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feec1da5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feebc0ce590 0x7feebc14c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feec1da5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feebc0cfec0 0x7feebc149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feec1da5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feebc154ee0 0x7feebc0b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feec1da5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc1552c0 con 0x7feebc154ee0 2024-04-25T09:55:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feec1da5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc071230 con 0x7feebc0cfec0 2024-04-25T09:55:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feec1da5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc155440 con 0x7feebc0ce590 2024-04-25T09:55:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feebbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feebc154ee0 0x7feebc0b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feebbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feebc154ee0 0x7feebc0b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54304/0 (socket says 172.21.15.19:54304) 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feebbfff640 1 -- 172.21.15.19:0/2769295168 learned_addr learned my addr 172.21.15.19:0/2769295168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feebaffd640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feebc0cfec0 0x7feebc149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feebb7fe640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feebc0ce590 0x7feebc14c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feebbfff640 1 -- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feebc0ce590 msgr2=0x7feebc14c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feebbfff640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feebc0ce590 0x7feebc14c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feebbfff640 1 -- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feebc0cfec0 msgr2=0x7feebc149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feebbfff640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feebc0cfec0 0x7feebc149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feebbfff640 1 -- 172.21.15.19:0/2769295168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc0b0950 con 0x7feebc154ee0 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.136+0000 7feebbfff640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feebc154ee0 0x7feebc0b1130 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009d90 tx=0x7feeb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.137+0000 7feea3fff640 1 -- 172.21.15.19:0/2769295168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4040070 con 0x7feebc154ee0 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.137+0000 7feea3fff640 1 -- 172.21.15.19:0/2769295168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb4031c90 con 0x7feebc154ee0 2024-04-25T09:55:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.137+0000 7feec1da5640 1 -- 172.21.15.19:0/2769295168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc149560 con 0x7feebc154ee0 2024-04-25T09:55:55.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.137+0000 7feea3fff640 1 -- 172.21.15.19:0/2769295168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb403c640 con 0x7feebc154ee0 2024-04-25T09:55:55.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.137+0000 7feec1da5640 1 -- 172.21.15.19:0/2769295168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc149b10 con 0x7feebc154ee0 2024-04-25T09:55:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.138+0000 7feec1da5640 1 -- 172.21.15.19:0/2769295168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc073990 con 0x7feebc154ee0 2024-04-25T09:55:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.138+0000 7feea3fff640 1 -- 172.21.15.19:0/2769295168 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feeb4039040 con 0x7feebc154ee0 2024-04-25T09:55:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.139+0000 7feea3fff640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feea806b4d0 0x7feea806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.139+0000 7feea3fff640 1 -- 172.21.15.19:0/2769295168 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feeb40ab8f0 con 0x7feebc154ee0 2024-04-25T09:55:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.139+0000 7feebb7fe640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feea806b4d0 0x7feea806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.140+0000 7feebb7fe640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feea806b4d0 0x7feea806d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feeac0016e0 tx=0x7feeac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.143+0000 7feea3fff640 1 -- 172.21.15.19:0/2769295168 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feeb4079fb0 con 0x7feebc154ee0 2024-04-25T09:55:55.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.359+0000 7feec1da5640 1 -- 172.21.15.19:0/2769295168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feebc06d360 con 0x7feebc154ee0 2024-04-25T09:55:55.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.360+0000 7feea3fff640 1 -- 172.21.15.19:0/2769295168 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7feeb407e050 con 0x7feebc154ee0 2024-04-25T09:55:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:55.363 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:55:54.352541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:55:55.363 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-04-25T09:55:55.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 -- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feea806b4d0 msgr2=0x7feea806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feea806b4d0 0x7feea806d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feeac0016e0 tx=0x7feeac009040 comp rx=0 tx=0).stop 2024-04-25T09:55:55.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 -- 172.21.15.19:0/2769295168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feebc154ee0 msgr2=0x7feebc0b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feebc154ee0 0x7feebc0b1130 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009d90 tx=0x7feeb4004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:55.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 -- 172.21.15.19:0/2769295168 shutdown_connections 2024-04-25T09:55:55.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feea806b4d0 0x7feea806d990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feebc154ee0 0x7feebc0b1130 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feebc0cfec0 0x7feebc149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 --2- 172.21.15.19:0/2769295168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feebc0ce590 0x7feebc14c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 -- 172.21.15.19:0/2769295168 >> 172.21.15.19:0/2769295168 conn(0x7feebc06abe0 msgr2=0x7feebc15ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 -- 172.21.15.19:0/2769295168 shutdown_connections 2024-04-25T09:55:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.362+0000 7feec1da5640 1 -- 172.21.15.19:0/2769295168 wait complete. 2024-04-25T09:55:55.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.473+0000 7fcc13488640 1 Processor -- start 2024-04-25T09:55:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.473+0000 7fcc13488640 1 -- start start 2024-04-25T09:55:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.474+0000 7fcc13488640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15ba70 0x7fcc0c0725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.474+0000 7fcc13488640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcc0c0aefa0 0x7fcc0c06d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.474+0000 7fcc13488640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0c15be50 con 0x7fcc0c06f360 2024-04-25T09:55:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.474+0000 7fcc13488640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0717f0 con 0x7fcc0c0aefa0 2024-04-25T09:55:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.474+0000 7fcc13488640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c071ad0 con 0x7fcc0c15ba70 2024-04-25T09:55:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.474+0000 7fcc10a23640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcc0c06f360 0x7fcc0c0cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35608/0 (socket says 172.21.15.19:35608) 2024-04-25T09:55:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.474+0000 7fcc10a23640 1 -- 172.21.15.19:0/296626943 learned_addr learned my addr 172.21.15.19:0/296626943 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc11a25640 1 --2- 172.21.15.19:0/296626943 >> v2:172.21.15.112:3300/0 conn(0x7fcc0c0aefa0 0x7fcc0c06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751638315 0 0) 0x7fcbfc001110 con 0x7fcc0c06f360 2024-04-25T09:55:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbf8003520 con 0x7fcc0c06f360 2024-04-25T09:55:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc11224640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15ba70 0x7fcc0c0725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4008971249 0 0) 0x7fcbf8003520 con 0x7fcc0c06f360 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15ba70 msgr2=0x7fcc0c0725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15ba70 0x7fcc0c0725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 >> v2:172.21.15.112:3300/0 conn(0x7fcc0c0aefa0 msgr2=0x7fcc0c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 --2- 172.21.15.19:0/296626943 >> v2:172.21.15.112:3300/0 conn(0x7fcc0c0aefa0 0x7fcc0c06d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c071db0 con 0x7fcc0c06f360 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fcbfc001840 con 0x7fcc0c06f360 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 >> v1:172.21.15.19:6789/0 conn(0x7fcc0c06f360 legacy=0x7fcc0c0cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf80039b0 0x7fcbf8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbf8004800 0x7fcbf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15ba70 0x7fcbf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0717f0 con 0x7fcbf80039b0 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c071ad0 con 0x7fcbf8004800 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf800b9c0 con 0x7fcc0c15ba70 2024-04-25T09:55:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc11224640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf80039b0 0x7fcbf8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc10a23640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbf8004800 0x7fcbf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc11a25640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15ba70 0x7fcbf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc11224640 1 -- 172.21.15.19:0/296626943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15ba70 msgr2=0x7fcbf80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc11224640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15ba70 0x7fcbf80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc11224640 1 -- 172.21.15.19:0/296626943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbf8004800 msgr2=0x7fcbf8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc11224640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbf8004800 0x7fcbf8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.475+0000 7fcc11224640 1 -- 172.21.15.19:0/296626943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf8003520 con 0x7fcbf80039b0 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.476+0000 7fcc11224640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf80039b0 0x7fcbf8003dd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009ac0 tx=0x7fcbf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb94edcf6eea038 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.476+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04036070 con 0x7fcbf80039b0 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.476+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc04031400 con 0x7fcbf80039b0 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.476+0000 7fcc03fff640 1 -- 172.21.15.19:0/296626943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04009690 con 0x7fcbf80039b0 2024-04-25T09:55:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.477+0000 7fcc13488640 1 -- 172.21.15.19:0/296626943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf80039b0 msgr2=0x7fcbf8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.477+0000 7fcc13488640 1 --2- 172.21.15.19:0/296626943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf80039b0 0x7fcbf8003dd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009ac0 tx=0x7fcbf8005830 comp rx=0 tx=0).stop 2024-04-25T09:55:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.477+0000 7fcc11a25640 1 -- 172.21.15.19:0/296626943 reap_dead start 2024-04-25T09:55:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.477+0000 7fcc13488640 1 -- 172.21.15.19:0/296626943 shutdown_connections 2024-04-25T09:55:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.477+0000 7fcc13488640 1 -- 172.21.15.19:0/296626943 >> 172.21.15.19:0/296626943 conn(0x7fcc0c0cfae0 msgr2=0x7fcc0c0cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.477+0000 7fcc13488640 1 -- 172.21.15.19:0/296626943 shutdown_connections 2024-04-25T09:55:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.477+0000 7fcc13488640 1 -- 172.21.15.19:0/296626943 wait complete. 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.478+0000 7fcc13488640 1 Processor -- start 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.478+0000 7fcc13488640 1 -- start start 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc13488640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbf80039b0 0x7fcc0c13f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc13488640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15a510 0x7fcc0c14e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc13488640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc0c14e8d0 0x7fcc0c14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc13488640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0cd420 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc13488640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0cd7e0 con 0x7fcbf80039b0 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc13488640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0cd5a0 con 0x7fcc0c15a510 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc11a25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc0c14e8d0 0x7fcc0c14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc11a25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc0c14e8d0 0x7fcc0c14b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54312/0 (socket says 172.21.15.19:54312) 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc11a25640 1 -- 172.21.15.19:0/2792353240 learned_addr learned my addr 172.21.15.19:0/2792353240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc11224640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbf80039b0 0x7fcc0c13f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc10a23640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15a510 0x7fcc0c14e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc11a25640 1 -- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15a510 msgr2=0x7fcc0c14e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc11a25640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15a510 0x7fcc0c14e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc11a25640 1 -- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbf80039b0 msgr2=0x7fcc0c13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc11a25640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbf80039b0 0x7fcc0c13f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc11a25640 1 -- 172.21.15.19:0/2792353240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c149a90 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc11224640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbf80039b0 0x7fcc0c13f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc11a25640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc0c14e8d0 0x7fcc0c14b790 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c070f80 tx=0x7fcc080115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.479+0000 7fcc01ffb640 1 -- 172.21.15.19:0/2792353240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08003990 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.480+0000 7fcc01ffb640 1 -- 172.21.15.19:0/2792353240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0801b070 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.480+0000 7fcc13488640 1 -- 172.21.15.19:0/2792353240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c14d010 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.480+0000 7fcc01ffb640 1 -- 172.21.15.19:0/2792353240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08003b70 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.480+0000 7fcc13488640 1 -- 172.21.15.19:0/2792353240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c14d5d0 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.481+0000 7fcc13488640 1 -- 172.21.15.19:0/2792353240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c073990 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.481+0000 7fcc01ffb640 1 -- 172.21.15.19:0/2792353240 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc08005d10 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.482+0000 7fcc01ffb640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcbf006b500 0x7fcbf006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.482+0000 7fcc01ffb640 1 -- 172.21.15.19:0/2792353240 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc0808a820 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.482+0000 7fcc11224640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcbf006b500 0x7fcbf006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.483+0000 7fcc11224640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcbf006b500 0x7fcbf006d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc04033000 tx=0x7fcc04008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.486+0000 7fcc01ffb640 1 -- 172.21.15.19:0/2792353240 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc0c149a90 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.667+0000 7fcc13488640 1 -- 172.21.15.19:0/2792353240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc0c156850 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.668+0000 7fcc01ffb640 1 -- 172.21.15.19:0/2792353240 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16930 (secure 0 0 0) 0x7fcc08059400 con 0x7fcc0c14e8d0 2024-04-25T09:55:55.671 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:55.671 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":53,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:53.990243+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:50.618180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:51.322489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 -- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcbf006b500 msgr2=0x7fcbf006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcbf006b500 0x7fcbf006d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc04033000 tx=0x7fcc04008030 comp rx=0 tx=0).stop 2024-04-25T09:55:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 -- 172.21.15.19:0/2792353240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc0c14e8d0 msgr2=0x7fcc0c14b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc0c14e8d0 0x7fcc0c14b790 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c070f80 tx=0x7fcc080115a0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 -- 172.21.15.19:0/2792353240 shutdown_connections 2024-04-25T09:55:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcbf006b500 0x7fcbf006d9c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc0c14e8d0 0x7fcc0c14b790 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc0c15a510 0x7fcc0c14e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 --2- 172.21.15.19:0/2792353240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbf80039b0 0x7fcc0c13f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 -- 172.21.15.19:0/2792353240 >> 172.21.15.19:0/2792353240 conn(0x7fcc0c0cfae0 msgr2=0x7fcc0c0afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 -- 172.21.15.19:0/2792353240 shutdown_connections 2024-04-25T09:55:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.670+0000 7fcc13488640 1 -- 172.21.15.19:0/2792353240 wait complete. 2024-04-25T09:55:55.684 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:55:55.685 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:55:55.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.782+0000 7f2f725ed640 1 Processor -- start 2024-04-25T09:55:55.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.783+0000 7f2f725ed640 1 -- start start 2024-04-25T09:55:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.783+0000 7f2f725ed640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f6c15ba00 0x7f2f6c09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.783+0000 7f2f725ed640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f6c15f5f0 0x7f2f6c15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.783+0000 7f2f725ed640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6c0cd710 con 0x7f2f6c1602e0 2024-04-25T09:55:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.783+0000 7f2f725ed640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c0cd4b0 con 0x7f2f6c15f5f0 2024-04-25T09:55:55.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.783+0000 7f2f725ed640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c099b70 con 0x7f2f6c15ba00 2024-04-25T09:55:55.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.783+0000 7f2f70b8a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f6c15f5f0 0x7f2f6c15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.784+0000 7f2f70b8a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f6c15f5f0 0x7f2f6c15faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54326/0 (socket says 172.21.15.19:54326) 2024-04-25T09:55:55.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.784+0000 7f2f70b8a640 1 -- 172.21.15.19:0/953988111 learned_addr learned my addr 172.21.15.19:0/953988111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:55.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.784+0000 7f2f6bfff640 1 --2- 172.21.15.19:0/953988111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f6c15ba00 0x7f2f6c09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.784+0000 7f2f70b8a640 1 -- 172.21.15.19:0/953988111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f6c15ba00 msgr2=0x7f2f6c09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:55.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.784+0000 7f2f70b8a640 1 --2- 172.21.15.19:0/953988111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f6c15ba00 0x7f2f6c09a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.784+0000 7f2f70b8a640 1 -- 172.21.15.19:0/953988111 >> v1:172.21.15.112:6789/0 conn(0x7f2f6c1602e0 legacy=0x7f2f6c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:55.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.784+0000 7f2f70b8a640 1 -- 172.21.15.19:0/953988111 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c099e50 con 0x7f2f6c15f5f0 2024-04-25T09:55:55.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.784+0000 7f2f70b8a640 1 --2- 172.21.15.19:0/953988111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f6c15f5f0 0x7f2f6c15faa0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009e50 tx=0x7f2f6402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5915b4025b64b233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.785+0000 7f2f6affd640 1 -- 172.21.15.19:0/953988111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64036070 con 0x7f2f6c15f5f0 2024-04-25T09:55:55.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.785+0000 7f2f6affd640 1 -- 172.21.15.19:0/953988111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f64031600 con 0x7f2f6c15f5f0 2024-04-25T09:55:55.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.785+0000 7f2f6affd640 1 -- 172.21.15.19:0/953988111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f640319d0 con 0x7f2f6c15f5f0 2024-04-25T09:55:55.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.785+0000 7f2f725ed640 1 -- 172.21.15.19:0/953988111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f6c15f5f0 msgr2=0x7f2f6c15faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.785+0000 7f2f725ed640 1 --2- 172.21.15.19:0/953988111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f6c15f5f0 0x7f2f6c15faa0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009e50 tx=0x7f2f6402a5b0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.785+0000 7f2f725ed640 1 -- 172.21.15.19:0/953988111 shutdown_connections 2024-04-25T09:55:55.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.785+0000 7f2f725ed640 1 --2- 172.21.15.19:0/953988111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f6c15f5f0 0x7f2f6c15faa0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.785+0000 7f2f725ed640 1 --2- 172.21.15.19:0/953988111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f6c15ba00 0x7f2f6c09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.785+0000 7f2f725ed640 1 -- 172.21.15.19:0/953988111 >> 172.21.15.19:0/953988111 conn(0x7f2f6c0b3270 msgr2=0x7f2f6c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:55.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.785+0000 7f2f725ed640 1 -- 172.21.15.19:0/953988111 shutdown_connections 2024-04-25T09:55:55.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.785+0000 7f2f725ed640 1 -- 172.21.15.19:0/953988111 wait complete. 2024-04-25T09:55:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.786+0000 7f2f725ed640 1 Processor -- start 2024-04-25T09:55:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.787+0000 7f2f725ed640 1 -- start start 2024-04-25T09:55:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.787+0000 7f2f725ed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c15ba00 0x7f2f6c153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.787+0000 7f2f725ed640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f6c15f5f0 0x7f2f6c1e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.787+0000 7f2f725ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f6c1602e0 0x7f2f6c0c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.787+0000 7f2f725ed640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c15bde0 con 0x7f2f6c1602e0 2024-04-25T09:55:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.787+0000 7f2f725ed640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c09ae70 con 0x7f2f6c15ba00 2024-04-25T09:55:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.787+0000 7f2f725ed640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c1e6340 con 0x7f2f6c15f5f0 2024-04-25T09:55:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f70b8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f6c1602e0 0x7f2f6c0c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f70b8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f6c1602e0 0x7f2f6c0c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54332/0 (socket says 172.21.15.19:54332) 2024-04-25T09:55:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f70b8a640 1 -- 172.21.15.19:0/2911965009 learned_addr learned my addr 172.21.15.19:0/2911965009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f6bfff640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c15ba00 0x7f2f6c153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f6b7fe640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f6c15f5f0 0x7f2f6c1e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f70b8a640 1 -- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f6c15f5f0 msgr2=0x7f2f6c1e5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f70b8a640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f6c15f5f0 0x7f2f6c1e5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f70b8a640 1 -- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c15ba00 msgr2=0x7f2f6c153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f70b8a640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c15ba00 0x7f2f6c153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f70b8a640 1 -- 172.21.15.19:0/2911965009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c1e65c0 con 0x7f2f6c1602e0 2024-04-25T09:55:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f6bfff640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c15ba00 0x7f2f6c153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f70b8a640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f6c1602e0 0x7f2f6c0c8850 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009f70 tx=0x7f2f64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f68ff9640 1 -- 172.21.15.19:0/2911965009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6403f020 con 0x7f2f6c1602e0 2024-04-25T09:55:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f68ff9640 1 -- 172.21.15.19:0/2911965009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f64037040 con 0x7f2f6c1602e0 2024-04-25T09:55:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f725ed640 1 -- 172.21.15.19:0/2911965009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6c0b5980 con 0x7f2f6c1602e0 2024-04-25T09:55:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f68ff9640 1 -- 172.21.15.19:0/2911965009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64036070 con 0x7f2f6c1602e0 2024-04-25T09:55:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.788+0000 7f2f725ed640 1 -- 172.21.15.19:0/2911965009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c0b5f40 con 0x7f2f6c1602e0 2024-04-25T09:55:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.789+0000 7f2f725ed640 1 -- 172.21.15.19:0/2911965009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6c0d06d0 con 0x7f2f6c1602e0 2024-04-25T09:55:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.790+0000 7f2f68ff9640 1 -- 172.21.15.19:0/2911965009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f64042020 con 0x7f2f6c1602e0 2024-04-25T09:55:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.790+0000 7f2f68ff9640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f5806b4d0 0x7f2f5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.791+0000 7f2f68ff9640 1 -- 172.21.15.19:0/2911965009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f640aa960 con 0x7f2f6c1602e0 2024-04-25T09:55:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.791+0000 7f2f6bfff640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f5806b4d0 0x7f2f5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.791+0000 7f2f6bfff640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f5806b4d0 0x7f2f5806d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00a9d0 tx=0x7f2f5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.795+0000 7f2f68ff9640 1 -- 172.21.15.19:0/2911965009 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f6c0d06d0 con 0x7f2f6c1602e0 2024-04-25T09:55:55.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.971+0000 7f2f725ed640 1 -- 172.21.15.19:0/2911965009 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f6c073990 con 0x7f2f5806b4d0 2024-04-25T09:55:55.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.978+0000 7f2f68ff9640 1 -- 172.21.15.19:0/2911965009 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251886 (secure 0 0 0) 0x7f2f6c073990 con 0x7f2f5806b4d0 2024-04-25T09:55:55.984 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:55.985 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:55:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 -- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f5806b4d0 msgr2=0x7f2f5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f5806b4d0 0x7f2f5806d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00a9d0 tx=0x7f2f5c008040 comp rx=0 tx=0).stop 2024-04-25T09:55:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 -- 172.21.15.19:0/2911965009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f6c1602e0 msgr2=0x7f2f6c0c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f6c1602e0 0x7f2f6c0c8850 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009f70 tx=0x7f2f64004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 -- 172.21.15.19:0/2911965009 shutdown_connections 2024-04-25T09:55:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f5806b4d0 0x7f2f5806d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f6c1602e0 0x7f2f6c0c8850 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f6c15f5f0 0x7f2f6c1e5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 --2- 172.21.15.19:0/2911965009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c15ba00 0x7f2f6c153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 -- 172.21.15.19:0/2911965009 >> 172.21.15.19:0/2911965009 conn(0x7f2f6c0b3270 msgr2=0x7f2f6c0b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 -- 172.21.15.19:0/2911965009 shutdown_connections 2024-04-25T09:55:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:55.981+0000 7f2f725ed640 1 -- 172.21.15.19:0/2911965009 wait complete. 2024-04-25T09:55:55.995 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-04-25T09:55:54.949633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":288804,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685916,"statfs":{"total":773094113280,"available":772798377984,"internally_reserved":0,"allocated":6201344,"data_stored":3399980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000846"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.017227+0000","last_change":"2024-04-25T09:55:53.014274+0000","last_active":"2024-04-25T09:55:53.017227+0000","last_peered":"2024-04-25T09:55:53.017227+0000","last_clean":"2024-04-25T09:55:53.017227+0000","last_became_active":"2024-04-25T09:55:53.013927+0000","last_became_peered":"2024-04-25T09:55:53.013927+0000","last_unstale":"2024-04-25T09:55:53.017227+0000","last_undegraded":"2024-04-25T09:55:53.017227+0000","last_fullsized":"2024-04-25T09:55:53.017227+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:15.415341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.008567+0000","last_change":"2024-04-25T09:55:53.004346+0000","last_active":"2024-04-25T09:55:53.008567+0000","last_peered":"2024-04-25T09:55:53.008567+0000","last_clean":"2024-04-25T09:55:53.008567+0000","last_became_active":"2024-04-25T09:55:53.004077+0000","last_became_peered":"2024-04-25T09:55:53.004077+0000","last_unstale":"2024-04-25T09:55:53.008567+0000","last_undegraded":"2024-04-25T09:55:53.008567+0000","last_fullsized":"2024-04-25T09:55:53.008567+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:28.108495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019899+0000","last_change":"2024-04-25T09:55:53.016677+0000","last_active":"2024-04-25T09:55:53.019899+0000","last_peered":"2024-04-25T09:55:53.019899+0000","last_clean":"2024-04-25T09:55:53.019899+0000","last_became_active":"2024-04-25T09:55:53.016169+0000","last_became_peered":"2024-04-25T09:55:53.016169+0000","last_unstale":"2024-04-25T09:55:53.019899+0000","last_undegraded":"2024-04-25T09:55:53.019899+0000","last_fullsized":"2024-04-25T09:55:53.019899+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:35.928243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007666+0000","last_change":"2024-04-25T09:55:53.002918+0000","last_active":"2024-04-25T09:55:53.007666+0000","last_peered":"2024-04-25T09:55:53.007666+0000","last_clean":"2024-04-25T09:55:53.007666+0000","last_became_active":"2024-04-25T09:55:53.002658+0000","last_became_peered":"2024-04-25T09:55:53.002658+0000","last_unstale":"2024-04-25T09:55:53.007666+0000","last_undegraded":"2024-04-25T09:55:53.007666+0000","last_fullsized":"2024-04-25T09:55:53.007666+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:22.954684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.998779+0000","last_change":"2024-04-25T09:55:52.995438+0000","last_active":"2024-04-25T09:55:52.998779+0000","last_peered":"2024-04-25T09:55:52.998779+0000","last_clean":"2024-04-25T09:55:52.998779+0000","last_became_active":"2024-04-25T09:55:52.995164+0000","last_became_peered":"2024-04-25T09:55:52.995164+0000","last_unstale":"2024-04-25T09:55:52.998779+0000","last_undegraded":"2024-04-25T09:55:52.998779+0000","last_fullsized":"2024-04-25T09:55:52.998779+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:46.525553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.993052+0000","last_change":"2024-04-25T09:55:52.991008+0000","last_active":"2024-04-25T09:55:52.993052+0000","last_peered":"2024-04-25T09:55:52.993052+0000","last_clean":"2024-04-25T09:55:52.993052+0000","last_became_active":"2024-04-25T09:55:52.990688+0000","last_became_peered":"2024-04-25T09:55:52.990688+0000","last_unstale":"2024-04-25T09:55:52.993052+0000","last_undegraded":"2024-04-25T09:55:52.993052+0000","last_fullsized":"2024-04-25T09:55:52.993052+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:43.212203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.022419+0000","last_change":"2024-04-25T09:55:53.016587+0000","last_active":"2024-04-25T09:55:53.022419+0000","last_peered":"2024-04-25T09:55:53.022419+0000","last_clean":"2024-04-25T09:55:53.022419+0000","last_became_active":"2024-04-25T09:55:53.015594+0000","last_became_peered":"2024-04-25T09:55:53.015594+0000","last_unstale":"2024-04-25T09:55:53.022419+0000","last_undegraded":"2024-04-25T09:55:53.022419+0000","last_fullsized":"2024-04-25T09:55:53.022419+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:05.647949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.015122+0000","last_change":"2024-04-25T09:55:53.006222+0000","last_active":"2024-04-25T09:55:53.015122+0000","last_peered":"2024-04-25T09:55:53.015122+0000","last_clean":"2024-04-25T09:55:53.015122+0000","last_became_active":"2024-04-25T09:55:53.005884+0000","last_became_peered":"2024-04-25T09:55:53.005884+0000","last_unstale":"2024-04-25T09:55:53.015122+0000","last_undegraded":"2024-04-25T09:55:53.015122+0000","last_fullsized":"2024-04-25T09:55:53.015122+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:44.800887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.022840+0000","last_change":"2024-04-25T09:55:53.022045+0000","last_active":"2024-04-25T09:55:53.022840+0000","last_peered":"2024-04-25T09:55:53.022840+0000","last_clean":"2024-04-25T09:55:53.022840+0000","last_became_active":"2024-04-25T09:55:53.021385+0000","last_became_peered":"2024-04-25T09:55:53.021385+0000","last_unstale":"2024-04-25T09:55:53.022840+0000","last_undegraded":"2024-04-25T09:55:53.022840+0000","last_fullsized":"2024-04-25T09:55:53.022840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:05.647949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.994236+0000","last_change":"2024-04-25T09:55:52.991915+0000","last_active":"2024-04-25T09:55:52.994236+0000","last_peered":"2024-04-25T09:55:52.994236+0000","last_clean":"2024-04-25T09:55:52.994236+0000","last_became_active":"2024-04-25T09:55:52.991584+0000","last_became_peered":"2024-04-25T09:55:52.991584+0000","last_unstale":"2024-04-25T09:55:52.994236+0000","last_undegraded":"2024-04-25T09:55:52.994236+0000","last_fullsized":"2024-04-25T09:55:52.994236+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:14.228686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.996850+0000","last_change":"2024-04-25T09:55:52.994177+0000","last_active":"2024-04-25T09:55:52.996850+0000","last_peered":"2024-04-25T09:55:52.996850+0000","last_clean":"2024-04-25T09:55:52.996850+0000","last_became_active":"2024-04-25T09:55:52.993875+0000","last_became_peered":"2024-04-25T09:55:52.993875+0000","last_unstale":"2024-04-25T09:55:52.996850+0000","last_undegraded":"2024-04-25T09:55:52.996850+0000","last_fullsized":"2024-04-25T09:55:52.996850+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:43.223545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007046+0000","last_change":"2024-04-25T09:55:53.003462+0000","last_active":"2024-04-25T09:55:53.007046+0000","last_peered":"2024-04-25T09:55:53.007046+0000","last_clean":"2024-04-25T09:55:53.007046+0000","last_became_active":"2024-04-25T09:55:53.002101+0000","last_became_peered":"2024-04-25T09:55:53.002101+0000","last_unstale":"2024-04-25T09:55:53.007046+0000","last_undegraded":"2024-04-25T09:55:53.007046+0000","last_fullsized":"2024-04-25T09:55:53.007046+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:35.928243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.990220+0000","last_change":"2024-04-25T09:55:52.988523+0000","last_active":"2024-04-25T09:55:52.990220+0000","last_peered":"2024-04-25T09:55:52.990220+0000","last_clean":"2024-04-25T09:55:52.990220+0000","last_became_active":"2024-04-25T09:55:52.988046+0000","last_became_peered":"2024-04-25T09:55:52.988046+0000","last_unstale":"2024-04-25T09:55:52.990220+0000","last_undegraded":"2024-04-25T09:55:52.990220+0000","last_fullsized":"2024-04-25T09:55:52.990220+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:41.341650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.020049+0000","last_change":"2024-04-25T09:55:53.015754+0000","last_active":"2024-04-25T09:55:53.020049+0000","last_peered":"2024-04-25T09:55:53.020049+0000","last_clean":"2024-04-25T09:55:53.020049+0000","last_became_active":"2024-04-25T09:55:53.015449+0000","last_became_peered":"2024-04-25T09:55:53.015449+0000","last_unstale":"2024-04-25T09:55:53.020049+0000","last_undegraded":"2024-04-25T09:55:53.020049+0000","last_fullsized":"2024-04-25T09:55:53.020049+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:35.312208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006483+0000","last_change":"2024-04-25T09:55:52.999887+0000","last_active":"2024-04-25T09:55:53.006483+0000","last_peered":"2024-04-25T09:55:53.006483+0000","last_clean":"2024-04-25T09:55:53.006483+0000","last_became_active":"2024-04-25T09:55:52.999576+0000","last_became_peered":"2024-04-25T09:55:52.999576+0000","last_unstale":"2024-04-25T09:55:53.006483+0000","last_undegraded":"2024-04-25T09:55:53.006483+0000","last_fullsized":"2024-04-25T09:55:53.006483+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:47.104369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.991681+0000","last_change":"2024-04-25T09:55:52.990597+0000","last_active":"2024-04-25T09:55:52.991681+0000","last_peered":"2024-04-25T09:55:52.991681+0000","last_clean":"2024-04-25T09:55:52.991681+0000","last_became_active":"2024-04-25T09:55:52.990169+0000","last_became_peered":"2024-04-25T09:55:52.990169+0000","last_unstale":"2024-04-25T09:55:52.991681+0000","last_undegraded":"2024-04-25T09:55:52.991681+0000","last_fullsized":"2024-04-25T09:55:52.991681+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:21.689482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.018397+0000","last_change":"2024-04-25T09:55:53.007248+0000","last_active":"2024-04-25T09:55:53.018397+0000","last_peered":"2024-04-25T09:55:53.018397+0000","last_clean":"2024-04-25T09:55:53.018397+0000","last_became_active":"2024-04-25T09:55:53.005196+0000","last_became_peered":"2024-04-25T09:55:53.005196+0000","last_unstale":"2024-04-25T09:55:53.018397+0000","last_undegraded":"2024-04-25T09:55:53.018397+0000","last_fullsized":"2024-04-25T09:55:53.018397+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:55.103013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006580+0000","last_change":"2024-04-25T09:55:53.000020+0000","last_active":"2024-04-25T09:55:53.006580+0000","last_peered":"2024-04-25T09:55:53.006580+0000","last_clean":"2024-04-25T09:55:53.006580+0000","last_became_active":"2024-04-25T09:55:52.999473+0000","last_became_peered":"2024-04-25T09:55:52.999473+0000","last_unstale":"2024-04-25T09:55:53.006580+0000","last_undegraded":"2024-04-25T09:55:53.006580+0000","last_fullsized":"2024-04-25T09:55:53.006580+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:59.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.018292+0000","last_change":"2024-04-25T09:55:53.008716+0000","last_active":"2024-04-25T09:55:53.018292+0000","last_peered":"2024-04-25T09:55:53.018292+0000","last_clean":"2024-04-25T09:55:53.018292+0000","last_became_active":"2024-04-25T09:55:53.008439+0000","last_became_peered":"2024-04-25T09:55:53.008439+0000","last_unstale":"2024-04-25T09:55:53.018292+0000","last_undegraded":"2024-04-25T09:55:53.018292+0000","last_fullsized":"2024-04-25T09:55:53.018292+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:22.954684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.001679+0000","last_change":"2024-04-25T09:55:53.000083+0000","last_active":"2024-04-25T09:55:53.001679+0000","last_peered":"2024-04-25T09:55:53.001679+0000","last_clean":"2024-04-25T09:55:53.001679+0000","last_became_active":"2024-04-25T09:55:52.999446+0000","last_became_peered":"2024-04-25T09:55:52.999446+0000","last_unstale":"2024-04-25T09:55:53.001679+0000","last_undegraded":"2024-04-25T09:55:53.001679+0000","last_fullsized":"2024-04-25T09:55:53.001679+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:41.341650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007937+0000","last_change":"2024-04-25T09:55:53.004097+0000","last_active":"2024-04-25T09:55:53.007937+0000","last_peered":"2024-04-25T09:55:53.007937+0000","last_clean":"2024-04-25T09:55:53.007937+0000","last_became_active":"2024-04-25T09:55:53.003769+0000","last_became_peered":"2024-04-25T09:55:53.003769+0000","last_unstale":"2024-04-25T09:55:53.007937+0000","last_undegraded":"2024-04-25T09:55:53.007937+0000","last_fullsized":"2024-04-25T09:55:53.007937+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:14.228686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.017083+0000","last_change":"2024-04-25T09:55:53.012842+0000","last_active":"2024-04-25T09:55:53.017083+0000","last_peered":"2024-04-25T09:55:53.017083+0000","last_clean":"2024-04-25T09:55:53.017083+0000","last_became_active":"2024-04-25T09:55:53.012559+0000","last_became_peered":"2024-04-25T09:55:53.012559+0000","last_unstale":"2024-04-25T09:55:53.017083+0000","last_undegraded":"2024-04-25T09:55:53.017083+0000","last_fullsized":"2024-04-25T09:55:53.017083+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:31.670322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.000414+0000","last_change":"2024-04-25T09:55:52.998605+0000","last_active":"2024-04-25T09:55:53.000414+0000","last_peered":"2024-04-25T09:55:53.000414+0000","last_clean":"2024-04-25T09:55:53.000414+0000","last_became_active":"2024-04-25T09:55:52.998321+0000","last_became_peered":"2024-04-25T09:55:52.998321+0000","last_unstale":"2024-04-25T09:55:53.000414+0000","last_undegraded":"2024-04-25T09:55:53.000414+0000","last_fullsized":"2024-04-25T09:55:53.000414+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.166286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019759+0000","last_change":"2024-04-25T09:55:53.015799+0000","last_active":"2024-04-25T09:55:53.019759+0000","last_peered":"2024-04-25T09:55:53.019759+0000","last_clean":"2024-04-25T09:55:53.019759+0000","last_became_active":"2024-04-25T09:55:53.015519+0000","last_became_peered":"2024-04-25T09:55:53.015519+0000","last_unstale":"2024-04-25T09:55:53.019759+0000","last_undegraded":"2024-04-25T09:55:53.019759+0000","last_fullsized":"2024-04-25T09:55:53.019759+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:42.298428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019556+0000","last_change":"2024-04-25T09:55:53.010889+0000","last_active":"2024-04-25T09:55:53.019556+0000","last_peered":"2024-04-25T09:55:53.019556+0000","last_clean":"2024-04-25T09:55:53.019556+0000","last_became_active":"2024-04-25T09:55:53.010545+0000","last_became_peered":"2024-04-25T09:55:53.010545+0000","last_unstale":"2024-04-25T09:55:53.019556+0000","last_undegraded":"2024-04-25T09:55:53.019556+0000","last_fullsized":"2024-04-25T09:55:53.019556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.863701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006496+0000","last_change":"2024-04-25T09:55:53.000727+0000","last_active":"2024-04-25T09:55:53.006496+0000","last_peered":"2024-04-25T09:55:53.006496+0000","last_clean":"2024-04-25T09:55:53.006496+0000","last_became_active":"2024-04-25T09:55:53.000416+0000","last_became_peered":"2024-04-25T09:55:53.000416+0000","last_unstale":"2024-04-25T09:55:53.006496+0000","last_undegraded":"2024-04-25T09:55:53.006496+0000","last_fullsized":"2024-04-25T09:55:53.006496+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:08.242990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.014520+0000","last_change":"2024-04-25T09:55:53.012736+0000","last_active":"2024-04-25T09:55:53.014520+0000","last_peered":"2024-04-25T09:55:53.014520+0000","last_clean":"2024-04-25T09:55:53.014520+0000","last_became_active":"2024-04-25T09:55:53.012357+0000","last_became_peered":"2024-04-25T09:55:53.012357+0000","last_unstale":"2024-04-25T09:55:53.014520+0000","last_undegraded":"2024-04-25T09:55:53.014520+0000","last_fullsized":"2024-04-25T09:55:53.014520+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:30.504579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.001658+0000","last_change":"2024-04-25T09:55:52.999689+0000","last_active":"2024-04-25T09:55:53.001658+0000","last_peered":"2024-04-25T09:55:53.001658+0000","last_clean":"2024-04-25T09:55:53.001658+0000","last_became_active":"2024-04-25T09:55:52.998960+0000","last_became_peered":"2024-04-25T09:55:52.998960+0000","last_unstale":"2024-04-25T09:55:53.001658+0000","last_undegraded":"2024-04-25T09:55:53.001658+0000","last_fullsized":"2024-04-25T09:55:53.001658+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:01.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019956+0000","last_change":"2024-04-25T09:55:53.017527+0000","last_active":"2024-04-25T09:55:53.019956+0000","last_peered":"2024-04-25T09:55:53.019956+0000","last_clean":"2024-04-25T09:55:53.019956+0000","last_became_active":"2024-04-25T09:55:53.017199+0000","last_became_peered":"2024-04-25T09:55:53.017199+0000","last_unstale":"2024-04-25T09:55:53.019956+0000","last_undegraded":"2024-04-25T09:55:53.019956+0000","last_fullsized":"2024-04-25T09:55:53.019956+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:31.670322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.997203+0000","last_change":"2024-04-25T09:55:52.992072+0000","last_active":"2024-04-25T09:55:52.997203+0000","last_peered":"2024-04-25T09:55:52.997203+0000","last_clean":"2024-04-25T09:55:52.997203+0000","last_became_active":"2024-04-25T09:55:52.991713+0000","last_became_peered":"2024-04-25T09:55:52.991713+0000","last_unstale":"2024-04-25T09:55:52.997203+0000","last_undegraded":"2024-04-25T09:55:52.997203+0000","last_fullsized":"2024-04-25T09:55:52.997203+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:29.140698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.018923+0000","last_change":"2024-04-25T09:55:53.010554+0000","last_active":"2024-04-25T09:55:53.018923+0000","last_peered":"2024-04-25T09:55:53.018923+0000","last_clean":"2024-04-25T09:55:53.018923+0000","last_became_active":"2024-04-25T09:55:53.010222+0000","last_became_peered":"2024-04-25T09:55:53.010222+0000","last_unstale":"2024-04-25T09:55:53.018923+0000","last_undegraded":"2024-04-25T09:55:53.018923+0000","last_fullsized":"2024-04-25T09:55:53.018923+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:28.108495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.010137+0000","last_change":"2024-04-25T09:55:53.005582+0000","last_active":"2024-04-25T09:55:53.010137+0000","last_peered":"2024-04-25T09:55:53.010137+0000","last_clean":"2024-04-25T09:55:53.010137+0000","last_became_active":"2024-04-25T09:55:53.005335+0000","last_became_peered":"2024-04-25T09:55:53.005335+0000","last_unstale":"2024-04-25T09:55:53.010137+0000","last_undegraded":"2024-04-25T09:55:53.010137+0000","last_fullsized":"2024-04-25T09:55:53.010137+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.863701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006549+0000","last_change":"2024-04-25T09:55:53.000903+0000","last_active":"2024-04-25T09:55:53.006549+0000","last_peered":"2024-04-25T09:55:53.006549+0000","last_clean":"2024-04-25T09:55:53.006549+0000","last_became_active":"2024-04-25T09:55:53.000580+0000","last_became_peered":"2024-04-25T09:55:53.000580+0000","last_unstale":"2024-04-25T09:55:53.006549+0000","last_undegraded":"2024-04-25T09:55:53.006549+0000","last_fullsized":"2024-04-25T09:55:53.006549+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:55.103013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.011132+0000","last_change":"2024-04-25T09:55:51.986147+0000","last_active":"2024-04-25T09:55:53.011132+0000","last_peered":"2024-04-25T09:55:53.011132+0000","last_clean":"2024-04-25T09:55:53.011132+0000","last_became_active":"2024-04-25T09:55:51.985860+0000","last_became_peered":"2024-04-25T09:55:51.985860+0000","last_unstale":"2024-04-25T09:55:53.011132+0000","last_undegraded":"2024-04-25T09:55:53.011132+0000","last_fullsized":"2024-04-25T09:55:53.011132+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:08.766351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.010008+0000","last_change":"2024-04-25T09:55:53.005345+0000","last_active":"2024-04-25T09:55:53.010008+0000","last_peered":"2024-04-25T09:55:53.010008+0000","last_clean":"2024-04-25T09:55:53.010008+0000","last_became_active":"2024-04-25T09:55:53.002058+0000","last_became_peered":"2024-04-25T09:55:53.002058+0000","last_unstale":"2024-04-25T09:55:53.010008+0000","last_undegraded":"2024-04-25T09:55:53.010008+0000","last_fullsized":"2024-04-25T09:55:53.010008+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:28.108495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.027577+0000","last_change":"2024-04-25T09:55:51.992222+0000","last_active":"2024-04-25T09:55:53.027577+0000","last_peered":"2024-04-25T09:55:53.027577+0000","last_clean":"2024-04-25T09:55:53.027577+0000","last_became_active":"2024-04-25T09:55:51.991957+0000","last_became_peered":"2024-04-25T09:55:51.991957+0000","last_unstale":"2024-04-25T09:55:53.027577+0000","last_undegraded":"2024-04-25T09:55:53.027577+0000","last_fullsized":"2024-04-25T09:55:53.027577+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:38.595139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.005471+0000","last_change":"2024-04-25T09:55:53.002778+0000","last_active":"2024-04-25T09:55:53.005471+0000","last_peered":"2024-04-25T09:55:53.005471+0000","last_clean":"2024-04-25T09:55:53.005471+0000","last_became_active":"2024-04-25T09:55:53.002504+0000","last_became_peered":"2024-04-25T09:55:53.002504+0000","last_unstale":"2024-04-25T09:55:53.005471+0000","last_undegraded":"2024-04-25T09:55:53.005471+0000","last_fullsized":"2024-04-25T09:55:53.005471+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:21.689482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.027751+0000","last_change":"2024-04-25T09:55:51.985673+0000","last_active":"2024-04-25T09:55:53.027751+0000","last_peered":"2024-04-25T09:55:53.027751+0000","last_clean":"2024-04-25T09:55:53.027751+0000","last_became_active":"2024-04-25T09:55:51.985384+0000","last_became_peered":"2024-04-25T09:55:51.985384+0000","last_unstale":"2024-04-25T09:55:53.027751+0000","last_undegraded":"2024-04-25T09:55:53.027751+0000","last_fullsized":"2024-04-25T09:55:53.027751+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:00.453283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007226+0000","last_change":"2024-04-25T09:55:53.000212+0000","last_active":"2024-04-25T09:55:53.007226+0000","last_peered":"2024-04-25T09:55:53.007226+0000","last_clean":"2024-04-25T09:55:53.007226+0000","last_became_active":"2024-04-25T09:55:52.999912+0000","last_became_peered":"2024-04-25T09:55:52.999912+0000","last_unstale":"2024-04-25T09:55:53.007226+0000","last_undegraded":"2024-04-25T09:55:53.007226+0000","last_fullsized":"2024-04-25T09:55:53.007226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:42.298428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.016288+0000","last_change":"2024-04-25T09:55:51.995907+0000","last_active":"2024-04-25T09:55:53.016288+0000","last_peered":"2024-04-25T09:55:53.016288+0000","last_clean":"2024-04-25T09:55:53.016288+0000","last_became_active":"2024-04-25T09:55:51.995600+0000","last_became_peered":"2024-04-25T09:55:51.995600+0000","last_unstale":"2024-04-25T09:55:53.016288+0000","last_undegraded":"2024-04-25T09:55:53.016288+0000","last_fullsized":"2024-04-25T09:55:53.016288+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:22.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.010164+0000","last_change":"2024-04-25T09:55:53.005994+0000","last_active":"2024-04-25T09:55:53.010164+0000","last_peered":"2024-04-25T09:55:53.010164+0000","last_clean":"2024-04-25T09:55:53.010164+0000","last_became_active":"2024-04-25T09:55:53.005751+0000","last_became_peered":"2024-04-25T09:55:53.005751+0000","last_unstale":"2024-04-25T09:55:53.010164+0000","last_undegraded":"2024-04-25T09:55:53.010164+0000","last_fullsized":"2024-04-25T09:55:53.010164+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:01.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.023071+0000","last_change":"2024-04-25T09:55:51.998290+0000","last_active":"2024-04-25T09:55:53.023071+0000","last_peered":"2024-04-25T09:55:53.023071+0000","last_clean":"2024-04-25T09:55:53.023071+0000","last_became_active":"2024-04-25T09:55:51.997971+0000","last_became_peered":"2024-04-25T09:55:51.997971+0000","last_unstale":"2024-04-25T09:55:53.023071+0000","last_undegraded":"2024-04-25T09:55:53.023071+0000","last_fullsized":"2024-04-25T09:55:53.023071+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:03.825032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.998754+0000","last_change":"2024-04-25T09:55:52.994628+0000","last_active":"2024-04-25T09:55:52.998754+0000","last_peered":"2024-04-25T09:55:52.998754+0000","last_clean":"2024-04-25T09:55:52.998754+0000","last_became_active":"2024-04-25T09:55:52.993364+0000","last_became_peered":"2024-04-25T09:55:52.993364+0000","last_unstale":"2024-04-25T09:55:52.998754+0000","last_undegraded":"2024-04-25T09:55:52.998754+0000","last_fullsized":"2024-04-25T09:55:52.998754+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:24.753929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.017272+0000","last_change":"2024-04-25T09:55:51.987886+0000","last_active":"2024-04-25T09:55:53.017272+0000","last_peered":"2024-04-25T09:55:53.017272+0000","last_clean":"2024-04-25T09:55:53.017272+0000","last_became_active":"2024-04-25T09:55:51.987593+0000","last_became_peered":"2024-04-25T09:55:51.987593+0000","last_unstale":"2024-04-25T09:55:53.017272+0000","last_undegraded":"2024-04-25T09:55:53.017272+0000","last_fullsized":"2024-04-25T09:55:53.017272+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:11.343893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.004798+0000","last_change":"2024-04-25T09:55:52.999900+0000","last_active":"2024-04-25T09:55:53.004798+0000","last_peered":"2024-04-25T09:55:53.004798+0000","last_clean":"2024-04-25T09:55:53.004798+0000","last_became_active":"2024-04-25T09:55:52.999439+0000","last_became_peered":"2024-04-25T09:55:52.999439+0000","last_unstale":"2024-04-25T09:55:53.004798+0000","last_undegraded":"2024-04-25T09:55:53.004798+0000","last_fullsized":"2024-04-25T09:55:53.004798+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:40.756411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.997501+0000","last_change":"2024-04-25T09:55:51.986260+0000","last_active":"2024-04-25T09:55:52.997501+0000","last_peered":"2024-04-25T09:55:52.997501+0000","last_clean":"2024-04-25T09:55:52.997501+0000","last_became_active":"2024-04-25T09:55:51.985977+0000","last_became_peered":"2024-04-25T09:55:51.985977+0000","last_unstale":"2024-04-25T09:55:52.997501+0000","last_undegraded":"2024-04-25T09:55:52.997501+0000","last_fullsized":"2024-04-25T09:55:52.997501+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:38.595139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.010206+0000","last_change":"2024-04-25T09:55:53.007480+0000","last_active":"2024-04-25T09:55:53.010206+0000","last_peered":"2024-04-25T09:55:53.010206+0000","last_clean":"2024-04-25T09:55:53.010206+0000","last_became_active":"2024-04-25T09:55:53.006929+0000","last_became_peered":"2024-04-25T09:55:53.006929+0000","last_unstale":"2024-04-25T09:55:53.010206+0000","last_undegraded":"2024-04-25T09:55:53.010206+0000","last_fullsized":"2024-04-25T09:55:53.010206+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.863701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.003075+0000","last_change":"2024-04-25T09:55:51.994361+0000","last_active":"2024-04-25T09:55:53.003075+0000","last_peered":"2024-04-25T09:55:53.003075+0000","last_clean":"2024-04-25T09:55:53.003075+0000","last_became_active":"2024-04-25T09:55:51.994081+0000","last_became_peered":"2024-04-25T09:55:51.994081+0000","last_unstale":"2024-04-25T09:55:53.003075+0000","last_undegraded":"2024-04-25T09:55:53.003075+0000","last_fullsized":"2024-04-25T09:55:53.003075+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:25.618033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006169+0000","last_change":"2024-04-25T09:55:53.003219+0000","last_active":"2024-04-25T09:55:53.006169+0000","last_peered":"2024-04-25T09:55:53.006169+0000","last_clean":"2024-04-25T09:55:53.006169+0000","last_became_active":"2024-04-25T09:55:53.002797+0000","last_became_peered":"2024-04-25T09:55:53.002797+0000","last_unstale":"2024-04-25T09:55:53.006169+0000","last_undegraded":"2024-04-25T09:55:53.006169+0000","last_fullsized":"2024-04-25T09:55:53.006169+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:19.655009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.986517+0000","last_change":"2024-04-25T09:55:51.988879+0000","last_active":"2024-04-25T09:55:52.986517+0000","last_peered":"2024-04-25T09:55:52.986517+0000","last_clean":"2024-04-25T09:55:52.986517+0000","last_became_active":"2024-04-25T09:55:51.988607+0000","last_became_peered":"2024-04-25T09:55:51.988607+0000","last_unstale":"2024-04-25T09:55:52.986517+0000","last_undegraded":"2024-04-25T09:55:52.986517+0000","last_fullsized":"2024-04-25T09:55:52.986517+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:44.668846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.009982+0000","last_change":"2024-04-25T09:55:53.005046+0000","last_active":"2024-04-25T09:55:53.009982+0000","last_peered":"2024-04-25T09:55:53.009982+0000","last_clean":"2024-04-25T09:55:53.009982+0000","last_became_active":"2024-04-25T09:55:53.004541+0000","last_became_peered":"2024-04-25T09:55:53.004541+0000","last_unstale":"2024-04-25T09:55:53.009982+0000","last_undegraded":"2024-04-25T09:55:53.009982+0000","last_fullsized":"2024-04-25T09:55:53.009982+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:22.954684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.987376+0000","last_change":"2024-04-25T09:55:51.991456+0000","last_active":"2024-04-25T09:55:52.987376+0000","last_peered":"2024-04-25T09:55:52.987376+0000","last_clean":"2024-04-25T09:55:52.987376+0000","last_became_active":"2024-04-25T09:55:51.991199+0000","last_became_peered":"2024-04-25T09:55:51.991199+0000","last_unstale":"2024-04-25T09:55:52.987376+0000","last_undegraded":"2024-04-25T09:55:52.987376+0000","last_fullsized":"2024-04-25T09:55:52.987376+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:17.524023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.020025+0000","last_change":"2024-04-25T09:55:53.012908+0000","last_active":"2024-04-25T09:55:53.020025+0000","last_peered":"2024-04-25T09:55:53.020025+0000","last_clean":"2024-04-25T09:55:53.020025+0000","last_became_active":"2024-04-25T09:55:53.011955+0000","last_became_peered":"2024-04-25T09:55:53.011955+0000","last_unstale":"2024-04-25T09:55:53.020025+0000","last_undegraded":"2024-04-25T09:55:53.020025+0000","last_fullsized":"2024-04-25T09:55:53.020025+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:08.242990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.027699+0000","last_change":"2024-04-25T09:55:51.985353+0000","last_active":"2024-04-25T09:55:53.027699+0000","last_peered":"2024-04-25T09:55:53.027699+0000","last_clean":"2024-04-25T09:55:53.027699+0000","last_became_active":"2024-04-25T09:55:51.985078+0000","last_became_peered":"2024-04-25T09:55:51.985078+0000","last_unstale":"2024-04-25T09:55:53.027699+0000","last_undegraded":"2024-04-25T09:55:53.027699+0000","last_fullsized":"2024-04-25T09:55:53.027699+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:03.825032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.998802+0000","last_change":"2024-04-25T09:55:52.995491+0000","last_active":"2024-04-25T09:55:52.998802+0000","last_peered":"2024-04-25T09:55:52.998802+0000","last_clean":"2024-04-25T09:55:52.998802+0000","last_became_active":"2024-04-25T09:55:52.995175+0000","last_became_peered":"2024-04-25T09:55:52.995175+0000","last_unstale":"2024-04-25T09:55:52.998802+0000","last_undegraded":"2024-04-25T09:55:52.998802+0000","last_fullsized":"2024-04-25T09:55:52.998802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:39.600291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.025017+0000","last_change":"2024-04-25T09:55:51.990270+0000","last_active":"2024-04-25T09:55:53.025017+0000","last_peered":"2024-04-25T09:55:53.025017+0000","last_clean":"2024-04-25T09:55:53.025017+0000","last_became_active":"2024-04-25T09:55:51.989742+0000","last_became_peered":"2024-04-25T09:55:51.989742+0000","last_unstale":"2024-04-25T09:55:53.025017+0000","last_undegraded":"2024-04-25T09:55:53.025017+0000","last_fullsized":"2024-04-25T09:55:53.025017+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:46.099217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.005494+0000","last_change":"2024-04-25T09:55:53.003775+0000","last_active":"2024-04-25T09:55:53.005494+0000","last_peered":"2024-04-25T09:55:53.005494+0000","last_clean":"2024-04-25T09:55:53.005494+0000","last_became_active":"2024-04-25T09:55:53.002086+0000","last_became_peered":"2024-04-25T09:55:53.002086+0000","last_unstale":"2024-04-25T09:55:53.005494+0000","last_undegraded":"2024-04-25T09:55:53.005494+0000","last_fullsized":"2024-04-25T09:55:53.005494+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:55.103013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.022567+0000","last_change":"2024-04-25T09:55:51.990030+0000","last_active":"2024-04-25T09:55:53.022567+0000","last_peered":"2024-04-25T09:55:53.022567+0000","last_clean":"2024-04-25T09:55:53.022567+0000","last_became_active":"2024-04-25T09:55:51.989732+0000","last_became_peered":"2024-04-25T09:55:51.989732+0000","last_unstale":"2024-04-25T09:55:53.022567+0000","last_undegraded":"2024-04-25T09:55:53.022567+0000","last_fullsized":"2024-04-25T09:55:53.022567+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:22.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019020+0000","last_change":"2024-04-25T09:55:53.010898+0000","last_active":"2024-04-25T09:55:53.019020+0000","last_peered":"2024-04-25T09:55:53.019020+0000","last_clean":"2024-04-25T09:55:53.019020+0000","last_became_active":"2024-04-25T09:55:53.010581+0000","last_became_peered":"2024-04-25T09:55:53.010581+0000","last_unstale":"2024-04-25T09:55:53.019020+0000","last_undegraded":"2024-04-25T09:55:53.019020+0000","last_fullsized":"2024-04-25T09:55:53.019020+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:40.756411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.998033+0000","last_change":"2024-04-25T09:55:51.998227+0000","last_active":"2024-04-25T09:55:52.998033+0000","last_peered":"2024-04-25T09:55:52.998033+0000","last_clean":"2024-04-25T09:55:52.998033+0000","last_became_active":"2024-04-25T09:55:51.997985+0000","last_became_peered":"2024-04-25T09:55:51.997985+0000","last_unstale":"2024-04-25T09:55:52.998033+0000","last_undegraded":"2024-04-25T09:55:52.998033+0000","last_fullsized":"2024-04-25T09:55:52.998033+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.649858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006467+0000","last_change":"2024-04-25T09:55:53.000147+0000","last_active":"2024-04-25T09:55:53.006467+0000","last_peered":"2024-04-25T09:55:53.006467+0000","last_clean":"2024-04-25T09:55:53.006467+0000","last_became_active":"2024-04-25T09:55:52.999599+0000","last_became_peered":"2024-04-25T09:55:52.999599+0000","last_unstale":"2024-04-25T09:55:53.006467+0000","last_undegraded":"2024-04-25T09:55:53.006467+0000","last_fullsized":"2024-04-25T09:55:53.006467+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:54.667894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.986279+0000","last_change":"2024-04-25T09:55:51.988216+0000","last_active":"2024-04-25T09:55:52.986279+0000","last_peered":"2024-04-25T09:55:52.986279+0000","last_clean":"2024-04-25T09:55:52.986279+0000","last_became_active":"2024-04-25T09:55:51.987893+0000","last_became_peered":"2024-04-25T09:55:51.987893+0000","last_unstale":"2024-04-25T09:55:52.986279+0000","last_undegraded":"2024-04-25T09:55:52.986279+0000","last_fullsized":"2024-04-25T09:55:52.986279+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:25.618033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.018372+0000","last_change":"2024-04-25T09:55:53.007097+0000","last_active":"2024-04-25T09:55:53.018372+0000","last_peered":"2024-04-25T09:55:53.018372+0000","last_clean":"2024-04-25T09:55:53.018372+0000","last_became_active":"2024-04-25T09:55:53.005896+0000","last_became_peered":"2024-04-25T09:55:53.005896+0000","last_unstale":"2024-04-25T09:55:53.018372+0000","last_undegraded":"2024-04-25T09:55:53.018372+0000","last_fullsized":"2024-04-25T09:55:53.018372+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:24.753929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":20,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.022092+0000","last_change":"2024-04-25T09:55:51.991337+0000","last_active":"2024-04-25T09:55:53.022092+0000","last_peered":"2024-04-25T09:55:53.022092+0000","last_clean":"2024-04-25T09:55:53.022092+0000","last_became_active":"2024-04-25T09:55:51.991034+0000","last_became_peered":"2024-04-25T09:55:51.991034+0000","last_unstale":"2024-04-25T09:55:53.022092+0000","last_undegraded":"2024-04-25T09:55:53.022092+0000","last_fullsized":"2024-04-25T09:55:53.022092+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:23.748777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007195+0000","last_change":"2024-04-25T09:55:53.001466+0000","last_active":"2024-04-25T09:55:53.007195+0000","last_peered":"2024-04-25T09:55:53.007195+0000","last_clean":"2024-04-25T09:55:53.007195+0000","last_became_active":"2024-04-25T09:55:53.001198+0000","last_became_peered":"2024-04-25T09:55:53.001198+0000","last_unstale":"2024-04-25T09:55:53.007195+0000","last_undegraded":"2024-04-25T09:55:53.007195+0000","last_fullsized":"2024-04-25T09:55:53.007195+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:35.312208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.022949+0000","last_change":"2024-04-25T09:55:51.999230+0000","last_active":"2024-04-25T09:55:53.022949+0000","last_peered":"2024-04-25T09:55:53.022949+0000","last_clean":"2024-04-25T09:55:53.022949+0000","last_became_active":"2024-04-25T09:55:51.998837+0000","last_became_peered":"2024-04-25T09:55:51.998837+0000","last_unstale":"2024-04-25T09:55:53.022949+0000","last_undegraded":"2024-04-25T09:55:53.022949+0000","last_fullsized":"2024-04-25T09:55:53.022949+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:28.135546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.001871+0000","last_change":"2024-04-25T09:55:52.999910+0000","last_active":"2024-04-25T09:55:53.001871+0000","last_peered":"2024-04-25T09:55:53.001871+0000","last_clean":"2024-04-25T09:55:53.001871+0000","last_became_active":"2024-04-25T09:55:52.999573+0000","last_became_peered":"2024-04-25T09:55:52.999573+0000","last_unstale":"2024-04-25T09:55:53.001871+0000","last_undegraded":"2024-04-25T09:55:53.001871+0000","last_fullsized":"2024-04-25T09:55:53.001871+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:39.250770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.014602+0000","last_change":"2024-04-25T09:55:51.983945+0000","last_active":"2024-04-25T09:55:53.014602+0000","last_peered":"2024-04-25T09:55:53.014602+0000","last_clean":"2024-04-25T09:55:53.014602+0000","last_became_active":"2024-04-25T09:55:51.983661+0000","last_became_peered":"2024-04-25T09:55:51.983661+0000","last_unstale":"2024-04-25T09:55:53.014602+0000","last_undegraded":"2024-04-25T09:55:53.014602+0000","last_fullsized":"2024-04-25T09:55:53.014602+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:25.781475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.010196+0000","last_change":"2024-04-25T09:55:53.006213+0000","last_active":"2024-04-25T09:55:53.010196+0000","last_peered":"2024-04-25T09:55:53.010196+0000","last_clean":"2024-04-25T09:55:53.010196+0000","last_became_active":"2024-04-25T09:55:53.005830+0000","last_became_peered":"2024-04-25T09:55:53.005830+0000","last_unstale":"2024-04-25T09:55:53.010196+0000","last_undegraded":"2024-04-25T09:55:53.010196+0000","last_fullsized":"2024-04-25T09:55:53.010196+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:41.341650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.027916+0000","last_change":"2024-04-25T09:55:51.988662+0000","last_active":"2024-04-25T09:55:53.027916+0000","last_peered":"2024-04-25T09:55:53.027916+0000","last_clean":"2024-04-25T09:55:53.027916+0000","last_became_active":"2024-04-25T09:55:51.988240+0000","last_became_peered":"2024-04-25T09:55:51.988240+0000","last_unstale":"2024-04-25T09:55:53.027916+0000","last_undegraded":"2024-04-25T09:55:53.027916+0000","last_fullsized":"2024-04-25T09:55:53.027916+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:45.520401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.014484+0000","last_change":"2024-04-25T09:55:53.003334+0000","last_active":"2024-04-25T09:55:53.014484+0000","last_peered":"2024-04-25T09:55:53.014484+0000","last_clean":"2024-04-25T09:55:53.014484+0000","last_became_active":"2024-04-25T09:55:53.003016+0000","last_became_peered":"2024-04-25T09:55:53.003016+0000","last_unstale":"2024-04-25T09:55:53.014484+0000","last_undegraded":"2024-04-25T09:55:53.014484+0000","last_fullsized":"2024-04-25T09:55:53.014484+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:41.887886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.995644+0000","last_change":"2024-04-25T09:55:51.989935+0000","last_active":"2024-04-25T09:55:52.995644+0000","last_peered":"2024-04-25T09:55:52.995644+0000","last_clean":"2024-04-25T09:55:52.995644+0000","last_became_active":"2024-04-25T09:55:51.989742+0000","last_became_peered":"2024-04-25T09:55:51.989742+0000","last_unstale":"2024-04-25T09:55:52.995644+0000","last_undegraded":"2024-04-25T09:55:52.995644+0000","last_fullsized":"2024-04-25T09:55:52.995644+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:11.343893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.005471+0000","last_change":"2024-04-25T09:55:53.003026+0000","last_active":"2024-04-25T09:55:53.005471+0000","last_peered":"2024-04-25T09:55:53.005471+0000","last_clean":"2024-04-25T09:55:53.005471+0000","last_became_active":"2024-04-25T09:55:53.002568+0000","last_became_peered":"2024-04-25T09:55:53.002568+0000","last_unstale":"2024-04-25T09:55:53.005471+0000","last_undegraded":"2024-04-25T09:55:53.005471+0000","last_fullsized":"2024-04-25T09:55:53.005471+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:59.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.027685+0000","last_change":"2024-04-25T09:55:51.989032+0000","last_active":"2024-04-25T09:55:53.027685+0000","last_peered":"2024-04-25T09:55:53.027685+0000","last_clean":"2024-04-25T09:55:53.027685+0000","last_became_active":"2024-04-25T09:55:51.988718+0000","last_became_peered":"2024-04-25T09:55:51.988718+0000","last_unstale":"2024-04-25T09:55:53.027685+0000","last_undegraded":"2024-04-25T09:55:53.027685+0000","last_fullsized":"2024-04-25T09:55:53.027685+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.649858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006491+0000","last_change":"2024-04-25T09:55:53.003389+0000","last_active":"2024-04-25T09:55:53.006491+0000","last_peered":"2024-04-25T09:55:53.006491+0000","last_clean":"2024-04-25T09:55:53.006491+0000","last_became_active":"2024-04-25T09:55:53.003125+0000","last_became_peered":"2024-04-25T09:55:53.003125+0000","last_unstale":"2024-04-25T09:55:53.006491+0000","last_undegraded":"2024-04-25T09:55:53.006491+0000","last_fullsized":"2024-04-25T09:55:53.006491+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:43.212203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.990323+0000","last_change":"2024-04-25T09:55:51.993882+0000","last_active":"2024-04-25T09:55:52.990323+0000","last_peered":"2024-04-25T09:55:52.990323+0000","last_clean":"2024-04-25T09:55:52.990323+0000","last_became_active":"2024-04-25T09:55:51.993593+0000","last_became_peered":"2024-04-25T09:55:51.993593+0000","last_unstale":"2024-04-25T09:55:52.990323+0000","last_undegraded":"2024-04-25T09:55:52.990323+0000","last_fullsized":"2024-04-25T09:55:52.990323+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:58.266708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019092+0000","last_change":"2024-04-25T09:55:53.011398+0000","last_active":"2024-04-25T09:55:53.019092+0000","last_peered":"2024-04-25T09:55:53.019092+0000","last_clean":"2024-04-25T09:55:53.019092+0000","last_became_active":"2024-04-25T09:55:53.011112+0000","last_became_peered":"2024-04-25T09:55:53.011112+0000","last_unstale":"2024-04-25T09:55:53.019092+0000","last_undegraded":"2024-04-25T09:55:53.019092+0000","last_fullsized":"2024-04-25T09:55:53.019092+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:54.667894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.003521+0000","last_change":"2024-04-25T09:55:51.990925+0000","last_active":"2024-04-25T09:55:53.003521+0000","last_peered":"2024-04-25T09:55:53.003521+0000","last_clean":"2024-04-25T09:55:53.003521+0000","last_became_active":"2024-04-25T09:55:51.990609+0000","last_became_peered":"2024-04-25T09:55:51.990609+0000","last_unstale":"2024-04-25T09:55:53.003521+0000","last_undegraded":"2024-04-25T09:55:53.003521+0000","last_fullsized":"2024-04-25T09:55:53.003521+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:44.668846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.014147+0000","last_change":"2024-04-25T09:55:53.002514+0000","last_active":"2024-04-25T09:55:53.014147+0000","last_peered":"2024-04-25T09:55:53.014147+0000","last_clean":"2024-04-25T09:55:53.014147+0000","last_became_active":"2024-04-25T09:55:53.002202+0000","last_became_peered":"2024-04-25T09:55:53.002202+0000","last_unstale":"2024-04-25T09:55:53.014147+0000","last_undegraded":"2024-04-25T09:55:53.014147+0000","last_fullsized":"2024-04-25T09:55:53.014147+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:47.218810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.989215+0000","last_change":"2024-04-25T09:55:51.979294+0000","last_active":"2024-04-25T09:55:52.989215+0000","last_peered":"2024-04-25T09:55:52.989215+0000","last_clean":"2024-04-25T09:55:52.989215+0000","last_became_active":"2024-04-25T09:55:51.978984+0000","last_became_peered":"2024-04-25T09:55:51.978984+0000","last_unstale":"2024-04-25T09:55:52.989215+0000","last_undegraded":"2024-04-25T09:55:52.989215+0000","last_fullsized":"2024-04-25T09:55:52.989215+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:23.954728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.004253+0000","last_change":"2024-04-25T09:55:53.001722+0000","last_active":"2024-04-25T09:55:53.004253+0000","last_peered":"2024-04-25T09:55:53.004253+0000","last_clean":"2024-04-25T09:55:53.004253+0000","last_became_active":"2024-04-25T09:55:53.001344+0000","last_became_peered":"2024-04-25T09:55:53.001344+0000","last_unstale":"2024-04-25T09:55:53.004253+0000","last_undegraded":"2024-04-25T09:55:53.004253+0000","last_fullsized":"2024-04-25T09:55:53.004253+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:24.753929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.020695+0000","last_change":"2024-04-25T09:55:51.996429+0000","last_active":"2024-04-25T09:55:53.020695+0000","last_peered":"2024-04-25T09:55:53.020695+0000","last_clean":"2024-04-25T09:55:53.020695+0000","last_became_active":"2024-04-25T09:55:51.996137+0000","last_became_peered":"2024-04-25T09:55:51.996137+0000","last_unstale":"2024-04-25T09:55:53.020695+0000","last_undegraded":"2024-04-25T09:55:53.020695+0000","last_fullsized":"2024-04-25T09:55:53.020695+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:17.524023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.994738+0000","last_change":"2024-04-25T09:55:52.992828+0000","last_active":"2024-04-25T09:55:52.994738+0000","last_peered":"2024-04-25T09:55:52.994738+0000","last_clean":"2024-04-25T09:55:52.994738+0000","last_became_active":"2024-04-25T09:55:52.991659+0000","last_became_peered":"2024-04-25T09:55:52.991659+0000","last_unstale":"2024-04-25T09:55:52.994738+0000","last_undegraded":"2024-04-25T09:55:52.994738+0000","last_fullsized":"2024-04-25T09:55:52.994738+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:07.070995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.988584+0000","last_change":"2024-04-25T09:55:51.984365+0000","last_active":"2024-04-25T09:55:52.988584+0000","last_peered":"2024-04-25T09:55:52.988584+0000","last_clean":"2024-04-25T09:55:52.988584+0000","last_became_active":"2024-04-25T09:55:51.984020+0000","last_became_peered":"2024-04-25T09:55:51.984020+0000","last_unstale":"2024-04-25T09:55:52.988584+0000","last_undegraded":"2024-04-25T09:55:52.988584+0000","last_fullsized":"2024-04-25T09:55:52.988584+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:22.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.004763+0000","last_change":"2024-04-25T09:55:52.999660+0000","last_active":"2024-04-25T09:55:53.004763+0000","last_peered":"2024-04-25T09:55:53.004763+0000","last_clean":"2024-04-25T09:55:53.004763+0000","last_became_active":"2024-04-25T09:55:52.999221+0000","last_became_peered":"2024-04-25T09:55:52.999221+0000","last_unstale":"2024-04-25T09:55:53.004763+0000","last_undegraded":"2024-04-25T09:55:53.004763+0000","last_fullsized":"2024-04-25T09:55:53.004763+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.863701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007276+0000","last_change":"2024-04-25T09:55:51.990060+0000","last_active":"2024-04-25T09:55:53.007276+0000","last_peered":"2024-04-25T09:55:53.007276+0000","last_clean":"2024-04-25T09:55:53.007276+0000","last_became_active":"2024-04-25T09:55:51.989550+0000","last_became_peered":"2024-04-25T09:55:51.989550+0000","last_unstale":"2024-04-25T09:55:53.007276+0000","last_undegraded":"2024-04-25T09:55:53.007276+0000","last_fullsized":"2024-04-25T09:55:53.007276+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:46.099217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.023115+0000","last_change":"2024-04-25T09:55:53.020590+0000","last_active":"2024-04-25T09:55:53.023115+0000","last_peered":"2024-04-25T09:55:53.023115+0000","last_clean":"2024-04-25T09:55:53.023115+0000","last_became_active":"2024-04-25T09:55:53.020321+0000","last_became_peered":"2024-04-25T09:55:53.020321+0000","last_unstale":"2024-04-25T09:55:53.023115+0000","last_undegraded":"2024-04-25T09:55:53.023115+0000","last_fullsized":"2024-04-25T09:55:53.023115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:36.043335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.990689+0000","last_change":"2024-04-25T09:55:51.993835+0000","last_active":"2024-04-25T09:55:52.990689+0000","last_peered":"2024-04-25T09:55:52.990689+0000","last_clean":"2024-04-25T09:55:52.990689+0000","last_became_active":"2024-04-25T09:55:51.993512+0000","last_became_peered":"2024-04-25T09:55:51.993512+0000","last_unstale":"2024-04-25T09:55:52.990689+0000","last_undegraded":"2024-04-25T09:55:52.990689+0000","last_fullsized":"2024-04-25T09:55:52.990689+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:54.097861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.005475+0000","last_change":"2024-04-25T09:55:53.002879+0000","last_active":"2024-04-25T09:55:53.005475+0000","last_peered":"2024-04-25T09:55:53.005475+0000","last_clean":"2024-04-25T09:55:53.005475+0000","last_became_active":"2024-04-25T09:55:53.002623+0000","last_became_peered":"2024-04-25T09:55:53.002623+0000","last_unstale":"2024-04-25T09:55:53.005475+0000","last_undegraded":"2024-04-25T09:55:53.005475+0000","last_fullsized":"2024-04-25T09:55:53.005475+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:47.104369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.991169+0000","last_change":"2024-04-25T09:55:51.994640+0000","last_active":"2024-04-25T09:55:52.991169+0000","last_peered":"2024-04-25T09:55:52.991169+0000","last_clean":"2024-04-25T09:55:52.991169+0000","last_became_active":"2024-04-25T09:55:51.994372+0000","last_became_peered":"2024-04-25T09:55:51.994372+0000","last_unstale":"2024-04-25T09:55:52.991169+0000","last_undegraded":"2024-04-25T09:55:52.991169+0000","last_fullsized":"2024-04-25T09:55:52.991169+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:21.949533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.020091+0000","last_change":"2024-04-25T09:55:53.016992+0000","last_active":"2024-04-25T09:55:53.020091+0000","last_peered":"2024-04-25T09:55:53.020091+0000","last_clean":"2024-04-25T09:55:53.020091+0000","last_became_active":"2024-04-25T09:55:53.016729+0000","last_became_peered":"2024-04-25T09:55:53.016729+0000","last_unstale":"2024-04-25T09:55:53.020091+0000","last_undegraded":"2024-04-25T09:55:53.020091+0000","last_fullsized":"2024-04-25T09:55:53.020091+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:44.800887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.986223+0000","last_change":"2024-04-25T09:55:51.987260+0000","last_active":"2024-04-25T09:55:52.986223+0000","last_peered":"2024-04-25T09:55:52.986223+0000","last_clean":"2024-04-25T09:55:52.986223+0000","last_became_active":"2024-04-25T09:55:51.986970+0000","last_became_peered":"2024-04-25T09:55:51.986970+0000","last_unstale":"2024-04-25T09:55:52.986223+0000","last_undegraded":"2024-04-25T09:55:52.986223+0000","last_fullsized":"2024-04-25T09:55:52.986223+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:08.766351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.999475+0000","last_change":"2024-04-25T09:55:52.998285+0000","last_active":"2024-04-25T09:55:52.999475+0000","last_peered":"2024-04-25T09:55:52.999475+0000","last_clean":"2024-04-25T09:55:52.999475+0000","last_became_active":"2024-04-25T09:55:52.997883+0000","last_became_peered":"2024-04-25T09:55:52.997883+0000","last_unstale":"2024-04-25T09:55:52.999475+0000","last_undegraded":"2024-04-25T09:55:52.999475+0000","last_fullsized":"2024-04-25T09:55:52.999475+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:28.108495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.995360+0000","last_change":"2024-04-25T09:55:51.988688+0000","last_active":"2024-04-25T09:55:52.995360+0000","last_peered":"2024-04-25T09:55:52.995360+0000","last_clean":"2024-04-25T09:55:52.995360+0000","last_became_active":"2024-04-25T09:55:51.988369+0000","last_became_peered":"2024-04-25T09:55:51.988369+0000","last_unstale":"2024-04-25T09:55:52.995360+0000","last_undegraded":"2024-04-25T09:55:52.995360+0000","last_fullsized":"2024-04-25T09:55:52.995360+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:28.135546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007695+0000","last_change":"2024-04-25T09:55:53.004171+0000","last_active":"2024-04-25T09:55:53.007695+0000","last_peered":"2024-04-25T09:55:53.007695+0000","last_clean":"2024-04-25T09:55:53.007695+0000","last_became_active":"2024-04-25T09:55:53.003882+0000","last_became_peered":"2024-04-25T09:55:53.003882+0000","last_unstale":"2024-04-25T09:55:53.007695+0000","last_undegraded":"2024-04-25T09:55:53.007695+0000","last_fullsized":"2024-04-25T09:55:53.007695+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:07.070995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007279+0000","last_change":"2024-04-25T09:55:51.990622+0000","last_active":"2024-04-25T09:55:53.007279+0000","last_peered":"2024-04-25T09:55:53.007279+0000","last_clean":"2024-04-25T09:55:53.007279+0000","last_became_active":"2024-04-25T09:55:51.990354+0000","last_became_peered":"2024-04-25T09:55:51.990354+0000","last_unstale":"2024-04-25T09:55:53.007279+0000","last_undegraded":"2024-04-25T09:55:53.007279+0000","last_fullsized":"2024-04-25T09:55:53.007279+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:58.266708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":235,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.985783+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:55:52.985783+0000","last_peered":"2024-04-25T09:55:52.985783+0000","last_clean":"2024-04-25T09:55:52.985783+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:55:52.985783+0000","last_undegraded":"2024-04-25T09:55:52.985783+0000","last_fullsized":"2024-04-25T09:55:52.985783+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36532,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335308,"statfs":{"total":96636764160,"available":96599355392,"internally_reserved":0,"allocated":1220608,"data_stored":867295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35952,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335888,"statfs":{"total":96636764160,"available":96599949312,"internally_reserved":0,"allocated":626688,"data_stored":277455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35948,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335892,"statfs":{"total":96636764160,"available":96599953408,"internally_reserved":0,"allocated":618496,"data_stored":276803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35952,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335888,"statfs":{"total":96636764160,"available":96599949312,"internally_reserved":0,"allocated":622592,"data_stored":277385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35968,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335872,"statfs":{"total":96636764160,"available":96599932928,"internally_reserved":0,"allocated":634880,"data_stored":278437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35960,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335880,"statfs":{"total":96636764160,"available":96599941120,"internally_reserved":0,"allocated":630784,"data_stored":278415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35956,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335884,"statfs":{"total":96636764160,"available":96599945216,"internally_reserved":0,"allocated":626688,"data_stored":276963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36536,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335304,"statfs":{"total":96636764160,"available":96599351296,"internally_reserved":0,"allocated":1220608,"data_stored":867227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:55:55.999 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:55:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.133+0000 7f46aaff5640 1 Processor -- start 2024-04-25T09:55:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.133+0000 7f46aaff5640 1 -- start start 2024-04-25T09:55:56.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.134+0000 7f46aaff5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46a4154ee0 0x7f46a40cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.134+0000 7f46aaff5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46a40ce590 0x7f46a406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.134+0000 7f46aaff5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a4071480 con 0x7f46a40cfcf0 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.134+0000 7f46aaff5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a4072090 con 0x7f46a4154ee0 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.134+0000 7f46aaff5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4071230 con 0x7f46a40ce590 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.134+0000 7f46a9592640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46a40ce590 0x7f46a406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.134+0000 7f46a8d91640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46a4154ee0 0x7f46a40cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.134+0000 7f46a8d91640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46a4154ee0 0x7f46a40cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:47716/0 (socket says 172.21.15.19:47716) 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.134+0000 7f46a8d91640 1 -- 172.21.15.19:0/4267573146 learned_addr learned my addr 172.21.15.19:0/4267573146 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.134+0000 7f46a3fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f46a40cfcf0 0x7f46a40cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:58984/0 (socket says 172.21.15.19:58984) 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.135+0000 7f46a8d91640 1 -- 172.21.15.19:0/4267573146 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46a40ce590 msgr2=0x7f46a406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.135+0000 7f46a8d91640 1 --2- 172.21.15.19:0/4267573146 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46a40ce590 0x7f46a406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.135+0000 7f46a8d91640 1 -- 172.21.15.19:0/4267573146 >> v1:172.21.15.112:6789/0 conn(0x7f46a40cfcf0 legacy=0x7f46a40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.135+0000 7f46a8d91640 1 -- 172.21.15.19:0/4267573146 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a4071760 con 0x7f46a4154ee0 2024-04-25T09:55:56.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.135+0000 7f46a37fe640 1 -- 172.21.15.19:0/4267573146 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216323469 0 0) 0x7f46a4071480 con 0x7f46a40cfcf0 2024-04-25T09:55:56.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.135+0000 7f46a8d91640 1 --2- 172.21.15.19:0/4267573146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a4154ee0 0x7f46a40cf7b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f469800bb10 tx=0x7f469802c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe477fed97259bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:56.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.135+0000 7f46a37fe640 1 -- 172.21.15.19:0/4267573146 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698036020 con 0x7f46a4154ee0 2024-04-25T09:55:56.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.135+0000 7f46a37fe640 1 -- 172.21.15.19:0/4267573146 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4698032820 con 0x7f46a4154ee0 2024-04-25T09:55:56.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.135+0000 7f46a37fe640 1 -- 172.21.15.19:0/4267573146 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698032bf0 con 0x7f46a4154ee0 2024-04-25T09:55:56.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.136+0000 7f46aaff5640 1 -- 172.21.15.19:0/4267573146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a4154ee0 msgr2=0x7f46a40cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.136+0000 7f46aaff5640 1 --2- 172.21.15.19:0/4267573146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a4154ee0 0x7f46a40cf7b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f469800bb10 tx=0x7f469802c920 comp rx=0 tx=0).stop 2024-04-25T09:55:56.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.136+0000 7f46aaff5640 1 -- 172.21.15.19:0/4267573146 shutdown_connections 2024-04-25T09:55:56.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.136+0000 7f46aaff5640 1 --2- 172.21.15.19:0/4267573146 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46a40ce590 0x7f46a406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.136+0000 7f46aaff5640 1 --2- 172.21.15.19:0/4267573146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a4154ee0 0x7f46a40cf7b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.136+0000 7f46aaff5640 1 -- 172.21.15.19:0/4267573146 >> 172.21.15.19:0/4267573146 conn(0x7f46a406abe0 msgr2=0x7f46a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:56.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.136+0000 7f46aaff5640 1 -- 172.21.15.19:0/4267573146 shutdown_connections 2024-04-25T09:55:56.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.136+0000 7f46aaff5640 1 -- 172.21.15.19:0/4267573146 wait complete. 2024-04-25T09:55:56.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.137+0000 7f46aaff5640 1 Processor -- start 2024-04-25T09:55:56.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.137+0000 7f46aaff5640 1 -- start start 2024-04-25T09:55:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.138+0000 7f46aaff5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a40ce590 0x7f46a4156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.138+0000 7f46aaff5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a40cfcf0 0x7f46a4159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.138+0000 7f46aaff5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46a4154ee0 0x7f46a4145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.138+0000 7f46aaff5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a41552c0 con 0x7f46a40ce590 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.138+0000 7f46aaff5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4071d20 con 0x7f46a40cfcf0 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.138+0000 7f46aaff5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4155440 con 0x7f46a4154ee0 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.138+0000 7f46a8d91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a40ce590 0x7f46a4156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.138+0000 7f46a8d91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a40ce590 0x7f46a4156030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54336/0 (socket says 172.21.15.19:54336) 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.138+0000 7f46a8d91640 1 -- 172.21.15.19:0/1470949690 learned_addr learned my addr 172.21.15.19:0/1470949690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.138+0000 7f46a3fff640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a40cfcf0 0x7f46a4159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.138+0000 7f46a9592640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46a4154ee0 0x7f46a4145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.139+0000 7f46a8d91640 1 -- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46a4154ee0 msgr2=0x7f46a4145fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.139+0000 7f46a8d91640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46a4154ee0 0x7f46a4145fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.139+0000 7f46a8d91640 1 -- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a40cfcf0 msgr2=0x7f46a4159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.139+0000 7f46a8d91640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a40cfcf0 0x7f46a4159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.139+0000 7f46a8d91640 1 -- 172.21.15.19:0/1470949690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a41567a0 con 0x7f46a40ce590 2024-04-25T09:55:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.139+0000 7f46a8d91640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a40ce590 0x7f46a4156030 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4698035040 tx=0x7f46980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.139+0000 7f46a17fa640 1 -- 172.21.15.19:0/1470949690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698036040 con 0x7f46a40ce590 2024-04-25T09:55:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.139+0000 7f46a17fa640 1 -- 172.21.15.19:0/1470949690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469803bb50 con 0x7f46a40ce590 2024-04-25T09:55:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.139+0000 7f46aaff5640 1 -- 172.21.15.19:0/1470949690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a4156a80 con 0x7f46a40ce590 2024-04-25T09:55:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.139+0000 7f46a17fa640 1 -- 172.21.15.19:0/1470949690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698004030 con 0x7f46a40ce590 2024-04-25T09:55:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.139+0000 7f46aaff5640 1 -- 172.21.15.19:0/1470949690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a4157510 con 0x7f46a40ce590 2024-04-25T09:55:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.140+0000 7f46aaff5640 1 -- 172.21.15.19:0/1470949690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a4156fa0 con 0x7f46a40ce590 2024-04-25T09:55:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.140+0000 7f46a17fa640 1 -- 172.21.15.19:0/1470949690 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f469804c060 con 0x7f46a40ce590 2024-04-25T09:55:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.141+0000 7f46a17fa640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f468806b4d0 0x7f468806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.141+0000 7f46a17fa640 1 -- 172.21.15.19:0/1470949690 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f46980ad0a0 con 0x7f46a40ce590 2024-04-25T09:55:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.141+0000 7f46a3fff640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f468806b4d0 0x7f468806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.142+0000 7f46a3fff640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f468806b4d0 0x7f468806d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f46a40b4110 tx=0x7f4690007d50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.145+0000 7f46a17fa640 1 -- 172.21.15.19:0/1470949690 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4698040070 con 0x7f46a40ce590 2024-04-25T09:55:56.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.324+0000 7f46aaff5640 1 -- 172.21.15.19:0/1470949690 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46a4157750 con 0x7f468806b4d0 2024-04-25T09:55:56.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.331+0000 7f46a17fa640 1 -- 172.21.15.19:0/1470949690 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251886 (secure 0 0 0) 0x7f46a4157750 con 0x7f468806b4d0 2024-04-25T09:55:56.336 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:56.338 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:55:56.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 -- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f468806b4d0 msgr2=0x7f468806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f468806b4d0 0x7f468806d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f46a40b4110 tx=0x7f4690007d50 comp rx=0 tx=0).stop 2024-04-25T09:55:56.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 -- 172.21.15.19:0/1470949690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a40ce590 msgr2=0x7f46a4156030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a40ce590 0x7f46a4156030 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4698035040 tx=0x7f46980046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 -- 172.21.15.19:0/1470949690 shutdown_connections 2024-04-25T09:55:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f468806b4d0 0x7f468806d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46a4154ee0 0x7f46a4145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a40cfcf0 0x7f46a4159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 --2- 172.21.15.19:0/1470949690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a40ce590 0x7f46a4156030 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 -- 172.21.15.19:0/1470949690 >> 172.21.15.19:0/1470949690 conn(0x7f46a406abe0 msgr2=0x7f46a415d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 -- 172.21.15.19:0/1470949690 shutdown_connections 2024-04-25T09:55:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.334+0000 7f46aaff5640 1 -- 172.21.15.19:0/1470949690 wait complete. 2024-04-25T09:55:56.347 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-04-25T09:55:54.949633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":288804,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685916,"statfs":{"total":773094113280,"available":772798377984,"internally_reserved":0,"allocated":6201344,"data_stored":3399980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000846"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.017227+0000","last_change":"2024-04-25T09:55:53.014274+0000","last_active":"2024-04-25T09:55:53.017227+0000","last_peered":"2024-04-25T09:55:53.017227+0000","last_clean":"2024-04-25T09:55:53.017227+0000","last_became_active":"2024-04-25T09:55:53.013927+0000","last_became_peered":"2024-04-25T09:55:53.013927+0000","last_unstale":"2024-04-25T09:55:53.017227+0000","last_undegraded":"2024-04-25T09:55:53.017227+0000","last_fullsized":"2024-04-25T09:55:53.017227+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:15.415341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.008567+0000","last_change":"2024-04-25T09:55:53.004346+0000","last_active":"2024-04-25T09:55:53.008567+0000","last_peered":"2024-04-25T09:55:53.008567+0000","last_clean":"2024-04-25T09:55:53.008567+0000","last_became_active":"2024-04-25T09:55:53.004077+0000","last_became_peered":"2024-04-25T09:55:53.004077+0000","last_unstale":"2024-04-25T09:55:53.008567+0000","last_undegraded":"2024-04-25T09:55:53.008567+0000","last_fullsized":"2024-04-25T09:55:53.008567+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:28.108495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019899+0000","last_change":"2024-04-25T09:55:53.016677+0000","last_active":"2024-04-25T09:55:53.019899+0000","last_peered":"2024-04-25T09:55:53.019899+0000","last_clean":"2024-04-25T09:55:53.019899+0000","last_became_active":"2024-04-25T09:55:53.016169+0000","last_became_peered":"2024-04-25T09:55:53.016169+0000","last_unstale":"2024-04-25T09:55:53.019899+0000","last_undegraded":"2024-04-25T09:55:53.019899+0000","last_fullsized":"2024-04-25T09:55:53.019899+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:35.928243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007666+0000","last_change":"2024-04-25T09:55:53.002918+0000","last_active":"2024-04-25T09:55:53.007666+0000","last_peered":"2024-04-25T09:55:53.007666+0000","last_clean":"2024-04-25T09:55:53.007666+0000","last_became_active":"2024-04-25T09:55:53.002658+0000","last_became_peered":"2024-04-25T09:55:53.002658+0000","last_unstale":"2024-04-25T09:55:53.007666+0000","last_undegraded":"2024-04-25T09:55:53.007666+0000","last_fullsized":"2024-04-25T09:55:53.007666+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:22.954684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.998779+0000","last_change":"2024-04-25T09:55:52.995438+0000","last_active":"2024-04-25T09:55:52.998779+0000","last_peered":"2024-04-25T09:55:52.998779+0000","last_clean":"2024-04-25T09:55:52.998779+0000","last_became_active":"2024-04-25T09:55:52.995164+0000","last_became_peered":"2024-04-25T09:55:52.995164+0000","last_unstale":"2024-04-25T09:55:52.998779+0000","last_undegraded":"2024-04-25T09:55:52.998779+0000","last_fullsized":"2024-04-25T09:55:52.998779+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:46.525553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.993052+0000","last_change":"2024-04-25T09:55:52.991008+0000","last_active":"2024-04-25T09:55:52.993052+0000","last_peered":"2024-04-25T09:55:52.993052+0000","last_clean":"2024-04-25T09:55:52.993052+0000","last_became_active":"2024-04-25T09:55:52.990688+0000","last_became_peered":"2024-04-25T09:55:52.990688+0000","last_unstale":"2024-04-25T09:55:52.993052+0000","last_undegraded":"2024-04-25T09:55:52.993052+0000","last_fullsized":"2024-04-25T09:55:52.993052+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:43.212203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.022419+0000","last_change":"2024-04-25T09:55:53.016587+0000","last_active":"2024-04-25T09:55:53.022419+0000","last_peered":"2024-04-25T09:55:53.022419+0000","last_clean":"2024-04-25T09:55:53.022419+0000","last_became_active":"2024-04-25T09:55:53.015594+0000","last_became_peered":"2024-04-25T09:55:53.015594+0000","last_unstale":"2024-04-25T09:55:53.022419+0000","last_undegraded":"2024-04-25T09:55:53.022419+0000","last_fullsized":"2024-04-25T09:55:53.022419+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:05.647949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.015122+0000","last_change":"2024-04-25T09:55:53.006222+0000","last_active":"2024-04-25T09:55:53.015122+0000","last_peered":"2024-04-25T09:55:53.015122+0000","last_clean":"2024-04-25T09:55:53.015122+0000","last_became_active":"2024-04-25T09:55:53.005884+0000","last_became_peered":"2024-04-25T09:55:53.005884+0000","last_unstale":"2024-04-25T09:55:53.015122+0000","last_undegraded":"2024-04-25T09:55:53.015122+0000","last_fullsized":"2024-04-25T09:55:53.015122+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:44.800887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.022840+0000","last_change":"2024-04-25T09:55:53.022045+0000","last_active":"2024-04-25T09:55:53.022840+0000","last_peered":"2024-04-25T09:55:53.022840+0000","last_clean":"2024-04-25T09:55:53.022840+0000","last_became_active":"2024-04-25T09:55:53.021385+0000","last_became_peered":"2024-04-25T09:55:53.021385+0000","last_unstale":"2024-04-25T09:55:53.022840+0000","last_undegraded":"2024-04-25T09:55:53.022840+0000","last_fullsized":"2024-04-25T09:55:53.022840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:05.647949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.994236+0000","last_change":"2024-04-25T09:55:52.991915+0000","last_active":"2024-04-25T09:55:52.994236+0000","last_peered":"2024-04-25T09:55:52.994236+0000","last_clean":"2024-04-25T09:55:52.994236+0000","last_became_active":"2024-04-25T09:55:52.991584+0000","last_became_peered":"2024-04-25T09:55:52.991584+0000","last_unstale":"2024-04-25T09:55:52.994236+0000","last_undegraded":"2024-04-25T09:55:52.994236+0000","last_fullsized":"2024-04-25T09:55:52.994236+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:14.228686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.996850+0000","last_change":"2024-04-25T09:55:52.994177+0000","last_active":"2024-04-25T09:55:52.996850+0000","last_peered":"2024-04-25T09:55:52.996850+0000","last_clean":"2024-04-25T09:55:52.996850+0000","last_became_active":"2024-04-25T09:55:52.993875+0000","last_became_peered":"2024-04-25T09:55:52.993875+0000","last_unstale":"2024-04-25T09:55:52.996850+0000","last_undegraded":"2024-04-25T09:55:52.996850+0000","last_fullsized":"2024-04-25T09:55:52.996850+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:43.223545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007046+0000","last_change":"2024-04-25T09:55:53.003462+0000","last_active":"2024-04-25T09:55:53.007046+0000","last_peered":"2024-04-25T09:55:53.007046+0000","last_clean":"2024-04-25T09:55:53.007046+0000","last_became_active":"2024-04-25T09:55:53.002101+0000","last_became_peered":"2024-04-25T09:55:53.002101+0000","last_unstale":"2024-04-25T09:55:53.007046+0000","last_undegraded":"2024-04-25T09:55:53.007046+0000","last_fullsized":"2024-04-25T09:55:53.007046+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:35.928243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.990220+0000","last_change":"2024-04-25T09:55:52.988523+0000","last_active":"2024-04-25T09:55:52.990220+0000","last_peered":"2024-04-25T09:55:52.990220+0000","last_clean":"2024-04-25T09:55:52.990220+0000","last_became_active":"2024-04-25T09:55:52.988046+0000","last_became_peered":"2024-04-25T09:55:52.988046+0000","last_unstale":"2024-04-25T09:55:52.990220+0000","last_undegraded":"2024-04-25T09:55:52.990220+0000","last_fullsized":"2024-04-25T09:55:52.990220+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:41.341650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.020049+0000","last_change":"2024-04-25T09:55:53.015754+0000","last_active":"2024-04-25T09:55:53.020049+0000","last_peered":"2024-04-25T09:55:53.020049+0000","last_clean":"2024-04-25T09:55:53.020049+0000","last_became_active":"2024-04-25T09:55:53.015449+0000","last_became_peered":"2024-04-25T09:55:53.015449+0000","last_unstale":"2024-04-25T09:55:53.020049+0000","last_undegraded":"2024-04-25T09:55:53.020049+0000","last_fullsized":"2024-04-25T09:55:53.020049+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:35.312208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006483+0000","last_change":"2024-04-25T09:55:52.999887+0000","last_active":"2024-04-25T09:55:53.006483+0000","last_peered":"2024-04-25T09:55:53.006483+0000","last_clean":"2024-04-25T09:55:53.006483+0000","last_became_active":"2024-04-25T09:55:52.999576+0000","last_became_peered":"2024-04-25T09:55:52.999576+0000","last_unstale":"2024-04-25T09:55:53.006483+0000","last_undegraded":"2024-04-25T09:55:53.006483+0000","last_fullsized":"2024-04-25T09:55:53.006483+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:47.104369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.991681+0000","last_change":"2024-04-25T09:55:52.990597+0000","last_active":"2024-04-25T09:55:52.991681+0000","last_peered":"2024-04-25T09:55:52.991681+0000","last_clean":"2024-04-25T09:55:52.991681+0000","last_became_active":"2024-04-25T09:55:52.990169+0000","last_became_peered":"2024-04-25T09:55:52.990169+0000","last_unstale":"2024-04-25T09:55:52.991681+0000","last_undegraded":"2024-04-25T09:55:52.991681+0000","last_fullsized":"2024-04-25T09:55:52.991681+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:21.689482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.018397+0000","last_change":"2024-04-25T09:55:53.007248+0000","last_active":"2024-04-25T09:55:53.018397+0000","last_peered":"2024-04-25T09:55:53.018397+0000","last_clean":"2024-04-25T09:55:53.018397+0000","last_became_active":"2024-04-25T09:55:53.005196+0000","last_became_peered":"2024-04-25T09:55:53.005196+0000","last_unstale":"2024-04-25T09:55:53.018397+0000","last_undegraded":"2024-04-25T09:55:53.018397+0000","last_fullsized":"2024-04-25T09:55:53.018397+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:55.103013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006580+0000","last_change":"2024-04-25T09:55:53.000020+0000","last_active":"2024-04-25T09:55:53.006580+0000","last_peered":"2024-04-25T09:55:53.006580+0000","last_clean":"2024-04-25T09:55:53.006580+0000","last_became_active":"2024-04-25T09:55:52.999473+0000","last_became_peered":"2024-04-25T09:55:52.999473+0000","last_unstale":"2024-04-25T09:55:53.006580+0000","last_undegraded":"2024-04-25T09:55:53.006580+0000","last_fullsized":"2024-04-25T09:55:53.006580+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:59.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.018292+0000","last_change":"2024-04-25T09:55:53.008716+0000","last_active":"2024-04-25T09:55:53.018292+0000","last_peered":"2024-04-25T09:55:53.018292+0000","last_clean":"2024-04-25T09:55:53.018292+0000","last_became_active":"2024-04-25T09:55:53.008439+0000","last_became_peered":"2024-04-25T09:55:53.008439+0000","last_unstale":"2024-04-25T09:55:53.018292+0000","last_undegraded":"2024-04-25T09:55:53.018292+0000","last_fullsized":"2024-04-25T09:55:53.018292+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:22.954684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.001679+0000","last_change":"2024-04-25T09:55:53.000083+0000","last_active":"2024-04-25T09:55:53.001679+0000","last_peered":"2024-04-25T09:55:53.001679+0000","last_clean":"2024-04-25T09:55:53.001679+0000","last_became_active":"2024-04-25T09:55:52.999446+0000","last_became_peered":"2024-04-25T09:55:52.999446+0000","last_unstale":"2024-04-25T09:55:53.001679+0000","last_undegraded":"2024-04-25T09:55:53.001679+0000","last_fullsized":"2024-04-25T09:55:53.001679+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:41.341650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007937+0000","last_change":"2024-04-25T09:55:53.004097+0000","last_active":"2024-04-25T09:55:53.007937+0000","last_peered":"2024-04-25T09:55:53.007937+0000","last_clean":"2024-04-25T09:55:53.007937+0000","last_became_active":"2024-04-25T09:55:53.003769+0000","last_became_peered":"2024-04-25T09:55:53.003769+0000","last_unstale":"2024-04-25T09:55:53.007937+0000","last_undegraded":"2024-04-25T09:55:53.007937+0000","last_fullsized":"2024-04-25T09:55:53.007937+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:14.228686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.017083+0000","last_change":"2024-04-25T09:55:53.012842+0000","last_active":"2024-04-25T09:55:53.017083+0000","last_peered":"2024-04-25T09:55:53.017083+0000","last_clean":"2024-04-25T09:55:53.017083+0000","last_became_active":"2024-04-25T09:55:53.012559+0000","last_became_peered":"2024-04-25T09:55:53.012559+0000","last_unstale":"2024-04-25T09:55:53.017083+0000","last_undegraded":"2024-04-25T09:55:53.017083+0000","last_fullsized":"2024-04-25T09:55:53.017083+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:31.670322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.000414+0000","last_change":"2024-04-25T09:55:52.998605+0000","last_active":"2024-04-25T09:55:53.000414+0000","last_peered":"2024-04-25T09:55:53.000414+0000","last_clean":"2024-04-25T09:55:53.000414+0000","last_became_active":"2024-04-25T09:55:52.998321+0000","last_became_peered":"2024-04-25T09:55:52.998321+0000","last_unstale":"2024-04-25T09:55:53.000414+0000","last_undegraded":"2024-04-25T09:55:53.000414+0000","last_fullsized":"2024-04-25T09:55:53.000414+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.166286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019759+0000","last_change":"2024-04-25T09:55:53.015799+0000","last_active":"2024-04-25T09:55:53.019759+0000","last_peered":"2024-04-25T09:55:53.019759+0000","last_clean":"2024-04-25T09:55:53.019759+0000","last_became_active":"2024-04-25T09:55:53.015519+0000","last_became_peered":"2024-04-25T09:55:53.015519+0000","last_unstale":"2024-04-25T09:55:53.019759+0000","last_undegraded":"2024-04-25T09:55:53.019759+0000","last_fullsized":"2024-04-25T09:55:53.019759+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:42.298428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019556+0000","last_change":"2024-04-25T09:55:53.010889+0000","last_active":"2024-04-25T09:55:53.019556+0000","last_peered":"2024-04-25T09:55:53.019556+0000","last_clean":"2024-04-25T09:55:53.019556+0000","last_became_active":"2024-04-25T09:55:53.010545+0000","last_became_peered":"2024-04-25T09:55:53.010545+0000","last_unstale":"2024-04-25T09:55:53.019556+0000","last_undegraded":"2024-04-25T09:55:53.019556+0000","last_fullsized":"2024-04-25T09:55:53.019556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.863701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006496+0000","last_change":"2024-04-25T09:55:53.000727+0000","last_active":"2024-04-25T09:55:53.006496+0000","last_peered":"2024-04-25T09:55:53.006496+0000","last_clean":"2024-04-25T09:55:53.006496+0000","last_became_active":"2024-04-25T09:55:53.000416+0000","last_became_peered":"2024-04-25T09:55:53.000416+0000","last_unstale":"2024-04-25T09:55:53.006496+0000","last_undegraded":"2024-04-25T09:55:53.006496+0000","last_fullsized":"2024-04-25T09:55:53.006496+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:08.242990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.014520+0000","last_change":"2024-04-25T09:55:53.012736+0000","last_active":"2024-04-25T09:55:53.014520+0000","last_peered":"2024-04-25T09:55:53.014520+0000","last_clean":"2024-04-25T09:55:53.014520+0000","last_became_active":"2024-04-25T09:55:53.012357+0000","last_became_peered":"2024-04-25T09:55:53.012357+0000","last_unstale":"2024-04-25T09:55:53.014520+0000","last_undegraded":"2024-04-25T09:55:53.014520+0000","last_fullsized":"2024-04-25T09:55:53.014520+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:30.504579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.001658+0000","last_change":"2024-04-25T09:55:52.999689+0000","last_active":"2024-04-25T09:55:53.001658+0000","last_peered":"2024-04-25T09:55:53.001658+0000","last_clean":"2024-04-25T09:55:53.001658+0000","last_became_active":"2024-04-25T09:55:52.998960+0000","last_became_peered":"2024-04-25T09:55:52.998960+0000","last_unstale":"2024-04-25T09:55:53.001658+0000","last_undegraded":"2024-04-25T09:55:53.001658+0000","last_fullsized":"2024-04-25T09:55:53.001658+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:01.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019956+0000","last_change":"2024-04-25T09:55:53.017527+0000","last_active":"2024-04-25T09:55:53.019956+0000","last_peered":"2024-04-25T09:55:53.019956+0000","last_clean":"2024-04-25T09:55:53.019956+0000","last_became_active":"2024-04-25T09:55:53.017199+0000","last_became_peered":"2024-04-25T09:55:53.017199+0000","last_unstale":"2024-04-25T09:55:53.019956+0000","last_undegraded":"2024-04-25T09:55:53.019956+0000","last_fullsized":"2024-04-25T09:55:53.019956+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:31.670322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.997203+0000","last_change":"2024-04-25T09:55:52.992072+0000","last_active":"2024-04-25T09:55:52.997203+0000","last_peered":"2024-04-25T09:55:52.997203+0000","last_clean":"2024-04-25T09:55:52.997203+0000","last_became_active":"2024-04-25T09:55:52.991713+0000","last_became_peered":"2024-04-25T09:55:52.991713+0000","last_unstale":"2024-04-25T09:55:52.997203+0000","last_undegraded":"2024-04-25T09:55:52.997203+0000","last_fullsized":"2024-04-25T09:55:52.997203+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:29.140698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.018923+0000","last_change":"2024-04-25T09:55:53.010554+0000","last_active":"2024-04-25T09:55:53.018923+0000","last_peered":"2024-04-25T09:55:53.018923+0000","last_clean":"2024-04-25T09:55:53.018923+0000","last_became_active":"2024-04-25T09:55:53.010222+0000","last_became_peered":"2024-04-25T09:55:53.010222+0000","last_unstale":"2024-04-25T09:55:53.018923+0000","last_undegraded":"2024-04-25T09:55:53.018923+0000","last_fullsized":"2024-04-25T09:55:53.018923+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:28.108495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.010137+0000","last_change":"2024-04-25T09:55:53.005582+0000","last_active":"2024-04-25T09:55:53.010137+0000","last_peered":"2024-04-25T09:55:53.010137+0000","last_clean":"2024-04-25T09:55:53.010137+0000","last_became_active":"2024-04-25T09:55:53.005335+0000","last_became_peered":"2024-04-25T09:55:53.005335+0000","last_unstale":"2024-04-25T09:55:53.010137+0000","last_undegraded":"2024-04-25T09:55:53.010137+0000","last_fullsized":"2024-04-25T09:55:53.010137+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.863701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006549+0000","last_change":"2024-04-25T09:55:53.000903+0000","last_active":"2024-04-25T09:55:53.006549+0000","last_peered":"2024-04-25T09:55:53.006549+0000","last_clean":"2024-04-25T09:55:53.006549+0000","last_became_active":"2024-04-25T09:55:53.000580+0000","last_became_peered":"2024-04-25T09:55:53.000580+0000","last_unstale":"2024-04-25T09:55:53.006549+0000","last_undegraded":"2024-04-25T09:55:53.006549+0000","last_fullsized":"2024-04-25T09:55:53.006549+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:55.103013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.011132+0000","last_change":"2024-04-25T09:55:51.986147+0000","last_active":"2024-04-25T09:55:53.011132+0000","last_peered":"2024-04-25T09:55:53.011132+0000","last_clean":"2024-04-25T09:55:53.011132+0000","last_became_active":"2024-04-25T09:55:51.985860+0000","last_became_peered":"2024-04-25T09:55:51.985860+0000","last_unstale":"2024-04-25T09:55:53.011132+0000","last_undegraded":"2024-04-25T09:55:53.011132+0000","last_fullsized":"2024-04-25T09:55:53.011132+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:08.766351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.010008+0000","last_change":"2024-04-25T09:55:53.005345+0000","last_active":"2024-04-25T09:55:53.010008+0000","last_peered":"2024-04-25T09:55:53.010008+0000","last_clean":"2024-04-25T09:55:53.010008+0000","last_became_active":"2024-04-25T09:55:53.002058+0000","last_became_peered":"2024-04-25T09:55:53.002058+0000","last_unstale":"2024-04-25T09:55:53.010008+0000","last_undegraded":"2024-04-25T09:55:53.010008+0000","last_fullsized":"2024-04-25T09:55:53.010008+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:28.108495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.027577+0000","last_change":"2024-04-25T09:55:51.992222+0000","last_active":"2024-04-25T09:55:53.027577+0000","last_peered":"2024-04-25T09:55:53.027577+0000","last_clean":"2024-04-25T09:55:53.027577+0000","last_became_active":"2024-04-25T09:55:51.991957+0000","last_became_peered":"2024-04-25T09:55:51.991957+0000","last_unstale":"2024-04-25T09:55:53.027577+0000","last_undegraded":"2024-04-25T09:55:53.027577+0000","last_fullsized":"2024-04-25T09:55:53.027577+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:38.595139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.005471+0000","last_change":"2024-04-25T09:55:53.002778+0000","last_active":"2024-04-25T09:55:53.005471+0000","last_peered":"2024-04-25T09:55:53.005471+0000","last_clean":"2024-04-25T09:55:53.005471+0000","last_became_active":"2024-04-25T09:55:53.002504+0000","last_became_peered":"2024-04-25T09:55:53.002504+0000","last_unstale":"2024-04-25T09:55:53.005471+0000","last_undegraded":"2024-04-25T09:55:53.005471+0000","last_fullsized":"2024-04-25T09:55:53.005471+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:21.689482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.027751+0000","last_change":"2024-04-25T09:55:51.985673+0000","last_active":"2024-04-25T09:55:53.027751+0000","last_peered":"2024-04-25T09:55:53.027751+0000","last_clean":"2024-04-25T09:55:53.027751+0000","last_became_active":"2024-04-25T09:55:51.985384+0000","last_became_peered":"2024-04-25T09:55:51.985384+0000","last_unstale":"2024-04-25T09:55:53.027751+0000","last_undegraded":"2024-04-25T09:55:53.027751+0000","last_fullsized":"2024-04-25T09:55:53.027751+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:00.453283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007226+0000","last_change":"2024-04-25T09:55:53.000212+0000","last_active":"2024-04-25T09:55:53.007226+0000","last_peered":"2024-04-25T09:55:53.007226+0000","last_clean":"2024-04-25T09:55:53.007226+0000","last_became_active":"2024-04-25T09:55:52.999912+0000","last_became_peered":"2024-04-25T09:55:52.999912+0000","last_unstale":"2024-04-25T09:55:53.007226+0000","last_undegraded":"2024-04-25T09:55:53.007226+0000","last_fullsized":"2024-04-25T09:55:53.007226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:42.298428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.016288+0000","last_change":"2024-04-25T09:55:51.995907+0000","last_active":"2024-04-25T09:55:53.016288+0000","last_peered":"2024-04-25T09:55:53.016288+0000","last_clean":"2024-04-25T09:55:53.016288+0000","last_became_active":"2024-04-25T09:55:51.995600+0000","last_became_peered":"2024-04-25T09:55:51.995600+0000","last_unstale":"2024-04-25T09:55:53.016288+0000","last_undegraded":"2024-04-25T09:55:53.016288+0000","last_fullsized":"2024-04-25T09:55:53.016288+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:22.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.010164+0000","last_change":"2024-04-25T09:55:53.005994+0000","last_active":"2024-04-25T09:55:53.010164+0000","last_peered":"2024-04-25T09:55:53.010164+0000","last_clean":"2024-04-25T09:55:53.010164+0000","last_became_active":"2024-04-25T09:55:53.005751+0000","last_became_peered":"2024-04-25T09:55:53.005751+0000","last_unstale":"2024-04-25T09:55:53.010164+0000","last_undegraded":"2024-04-25T09:55:53.010164+0000","last_fullsized":"2024-04-25T09:55:53.010164+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:01.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.023071+0000","last_change":"2024-04-25T09:55:51.998290+0000","last_active":"2024-04-25T09:55:53.023071+0000","last_peered":"2024-04-25T09:55:53.023071+0000","last_clean":"2024-04-25T09:55:53.023071+0000","last_became_active":"2024-04-25T09:55:51.997971+0000","last_became_peered":"2024-04-25T09:55:51.997971+0000","last_unstale":"2024-04-25T09:55:53.023071+0000","last_undegraded":"2024-04-25T09:55:53.023071+0000","last_fullsized":"2024-04-25T09:55:53.023071+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:03.825032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.998754+0000","last_change":"2024-04-25T09:55:52.994628+0000","last_active":"2024-04-25T09:55:52.998754+0000","last_peered":"2024-04-25T09:55:52.998754+0000","last_clean":"2024-04-25T09:55:52.998754+0000","last_became_active":"2024-04-25T09:55:52.993364+0000","last_became_peered":"2024-04-25T09:55:52.993364+0000","last_unstale":"2024-04-25T09:55:52.998754+0000","last_undegraded":"2024-04-25T09:55:52.998754+0000","last_fullsized":"2024-04-25T09:55:52.998754+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:24.753929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.017272+0000","last_change":"2024-04-25T09:55:51.987886+0000","last_active":"2024-04-25T09:55:53.017272+0000","last_peered":"2024-04-25T09:55:53.017272+0000","last_clean":"2024-04-25T09:55:53.017272+0000","last_became_active":"2024-04-25T09:55:51.987593+0000","last_became_peered":"2024-04-25T09:55:51.987593+0000","last_unstale":"2024-04-25T09:55:53.017272+0000","last_undegraded":"2024-04-25T09:55:53.017272+0000","last_fullsized":"2024-04-25T09:55:53.017272+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:11.343893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.004798+0000","last_change":"2024-04-25T09:55:52.999900+0000","last_active":"2024-04-25T09:55:53.004798+0000","last_peered":"2024-04-25T09:55:53.004798+0000","last_clean":"2024-04-25T09:55:53.004798+0000","last_became_active":"2024-04-25T09:55:52.999439+0000","last_became_peered":"2024-04-25T09:55:52.999439+0000","last_unstale":"2024-04-25T09:55:53.004798+0000","last_undegraded":"2024-04-25T09:55:53.004798+0000","last_fullsized":"2024-04-25T09:55:53.004798+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:40.756411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.997501+0000","last_change":"2024-04-25T09:55:51.986260+0000","last_active":"2024-04-25T09:55:52.997501+0000","last_peered":"2024-04-25T09:55:52.997501+0000","last_clean":"2024-04-25T09:55:52.997501+0000","last_became_active":"2024-04-25T09:55:51.985977+0000","last_became_peered":"2024-04-25T09:55:51.985977+0000","last_unstale":"2024-04-25T09:55:52.997501+0000","last_undegraded":"2024-04-25T09:55:52.997501+0000","last_fullsized":"2024-04-25T09:55:52.997501+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:38.595139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.010206+0000","last_change":"2024-04-25T09:55:53.007480+0000","last_active":"2024-04-25T09:55:53.010206+0000","last_peered":"2024-04-25T09:55:53.010206+0000","last_clean":"2024-04-25T09:55:53.010206+0000","last_became_active":"2024-04-25T09:55:53.006929+0000","last_became_peered":"2024-04-25T09:55:53.006929+0000","last_unstale":"2024-04-25T09:55:53.010206+0000","last_undegraded":"2024-04-25T09:55:53.010206+0000","last_fullsized":"2024-04-25T09:55:53.010206+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.863701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.003075+0000","last_change":"2024-04-25T09:55:51.994361+0000","last_active":"2024-04-25T09:55:53.003075+0000","last_peered":"2024-04-25T09:55:53.003075+0000","last_clean":"2024-04-25T09:55:53.003075+0000","last_became_active":"2024-04-25T09:55:51.994081+0000","last_became_peered":"2024-04-25T09:55:51.994081+0000","last_unstale":"2024-04-25T09:55:53.003075+0000","last_undegraded":"2024-04-25T09:55:53.003075+0000","last_fullsized":"2024-04-25T09:55:53.003075+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:25.618033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006169+0000","last_change":"2024-04-25T09:55:53.003219+0000","last_active":"2024-04-25T09:55:53.006169+0000","last_peered":"2024-04-25T09:55:53.006169+0000","last_clean":"2024-04-25T09:55:53.006169+0000","last_became_active":"2024-04-25T09:55:53.002797+0000","last_became_peered":"2024-04-25T09:55:53.002797+0000","last_unstale":"2024-04-25T09:55:53.006169+0000","last_undegraded":"2024-04-25T09:55:53.006169+0000","last_fullsized":"2024-04-25T09:55:53.006169+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:19.655009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.986517+0000","last_change":"2024-04-25T09:55:51.988879+0000","last_active":"2024-04-25T09:55:52.986517+0000","last_peered":"2024-04-25T09:55:52.986517+0000","last_clean":"2024-04-25T09:55:52.986517+0000","last_became_active":"2024-04-25T09:55:51.988607+0000","last_became_peered":"2024-04-25T09:55:51.988607+0000","last_unstale":"2024-04-25T09:55:52.986517+0000","last_undegraded":"2024-04-25T09:55:52.986517+0000","last_fullsized":"2024-04-25T09:55:52.986517+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:44.668846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.009982+0000","last_change":"2024-04-25T09:55:53.005046+0000","last_active":"2024-04-25T09:55:53.009982+0000","last_peered":"2024-04-25T09:55:53.009982+0000","last_clean":"2024-04-25T09:55:53.009982+0000","last_became_active":"2024-04-25T09:55:53.004541+0000","last_became_peered":"2024-04-25T09:55:53.004541+0000","last_unstale":"2024-04-25T09:55:53.009982+0000","last_undegraded":"2024-04-25T09:55:53.009982+0000","last_fullsized":"2024-04-25T09:55:53.009982+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:22.954684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.987376+0000","last_change":"2024-04-25T09:55:51.991456+0000","last_active":"2024-04-25T09:55:52.987376+0000","last_peered":"2024-04-25T09:55:52.987376+0000","last_clean":"2024-04-25T09:55:52.987376+0000","last_became_active":"2024-04-25T09:55:51.991199+0000","last_became_peered":"2024-04-25T09:55:51.991199+0000","last_unstale":"2024-04-25T09:55:52.987376+0000","last_undegraded":"2024-04-25T09:55:52.987376+0000","last_fullsized":"2024-04-25T09:55:52.987376+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:17.524023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.020025+0000","last_change":"2024-04-25T09:55:53.012908+0000","last_active":"2024-04-25T09:55:53.020025+0000","last_peered":"2024-04-25T09:55:53.020025+0000","last_clean":"2024-04-25T09:55:53.020025+0000","last_became_active":"2024-04-25T09:55:53.011955+0000","last_became_peered":"2024-04-25T09:55:53.011955+0000","last_unstale":"2024-04-25T09:55:53.020025+0000","last_undegraded":"2024-04-25T09:55:53.020025+0000","last_fullsized":"2024-04-25T09:55:53.020025+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:08.242990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.027699+0000","last_change":"2024-04-25T09:55:51.985353+0000","last_active":"2024-04-25T09:55:53.027699+0000","last_peered":"2024-04-25T09:55:53.027699+0000","last_clean":"2024-04-25T09:55:53.027699+0000","last_became_active":"2024-04-25T09:55:51.985078+0000","last_became_peered":"2024-04-25T09:55:51.985078+0000","last_unstale":"2024-04-25T09:55:53.027699+0000","last_undegraded":"2024-04-25T09:55:53.027699+0000","last_fullsized":"2024-04-25T09:55:53.027699+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:03.825032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.998802+0000","last_change":"2024-04-25T09:55:52.995491+0000","last_active":"2024-04-25T09:55:52.998802+0000","last_peered":"2024-04-25T09:55:52.998802+0000","last_clean":"2024-04-25T09:55:52.998802+0000","last_became_active":"2024-04-25T09:55:52.995175+0000","last_became_peered":"2024-04-25T09:55:52.995175+0000","last_unstale":"2024-04-25T09:55:52.998802+0000","last_undegraded":"2024-04-25T09:55:52.998802+0000","last_fullsized":"2024-04-25T09:55:52.998802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:39.600291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.025017+0000","last_change":"2024-04-25T09:55:51.990270+0000","last_active":"2024-04-25T09:55:53.025017+0000","last_peered":"2024-04-25T09:55:53.025017+0000","last_clean":"2024-04-25T09:55:53.025017+0000","last_became_active":"2024-04-25T09:55:51.989742+0000","last_became_peered":"2024-04-25T09:55:51.989742+0000","last_unstale":"2024-04-25T09:55:53.025017+0000","last_undegraded":"2024-04-25T09:55:53.025017+0000","last_fullsized":"2024-04-25T09:55:53.025017+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:46.099217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.005494+0000","last_change":"2024-04-25T09:55:53.003775+0000","last_active":"2024-04-25T09:55:53.005494+0000","last_peered":"2024-04-25T09:55:53.005494+0000","last_clean":"2024-04-25T09:55:53.005494+0000","last_became_active":"2024-04-25T09:55:53.002086+0000","last_became_peered":"2024-04-25T09:55:53.002086+0000","last_unstale":"2024-04-25T09:55:53.005494+0000","last_undegraded":"2024-04-25T09:55:53.005494+0000","last_fullsized":"2024-04-25T09:55:53.005494+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:55.103013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.022567+0000","last_change":"2024-04-25T09:55:51.990030+0000","last_active":"2024-04-25T09:55:53.022567+0000","last_peered":"2024-04-25T09:55:53.022567+0000","last_clean":"2024-04-25T09:55:53.022567+0000","last_became_active":"2024-04-25T09:55:51.989732+0000","last_became_peered":"2024-04-25T09:55:51.989732+0000","last_unstale":"2024-04-25T09:55:53.022567+0000","last_undegraded":"2024-04-25T09:55:53.022567+0000","last_fullsized":"2024-04-25T09:55:53.022567+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:22.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019020+0000","last_change":"2024-04-25T09:55:53.010898+0000","last_active":"2024-04-25T09:55:53.019020+0000","last_peered":"2024-04-25T09:55:53.019020+0000","last_clean":"2024-04-25T09:55:53.019020+0000","last_became_active":"2024-04-25T09:55:53.010581+0000","last_became_peered":"2024-04-25T09:55:53.010581+0000","last_unstale":"2024-04-25T09:55:53.019020+0000","last_undegraded":"2024-04-25T09:55:53.019020+0000","last_fullsized":"2024-04-25T09:55:53.019020+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:40.756411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.998033+0000","last_change":"2024-04-25T09:55:51.998227+0000","last_active":"2024-04-25T09:55:52.998033+0000","last_peered":"2024-04-25T09:55:52.998033+0000","last_clean":"2024-04-25T09:55:52.998033+0000","last_became_active":"2024-04-25T09:55:51.997985+0000","last_became_peered":"2024-04-25T09:55:51.997985+0000","last_unstale":"2024-04-25T09:55:52.998033+0000","last_undegraded":"2024-04-25T09:55:52.998033+0000","last_fullsized":"2024-04-25T09:55:52.998033+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.649858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006467+0000","last_change":"2024-04-25T09:55:53.000147+0000","last_active":"2024-04-25T09:55:53.006467+0000","last_peered":"2024-04-25T09:55:53.006467+0000","last_clean":"2024-04-25T09:55:53.006467+0000","last_became_active":"2024-04-25T09:55:52.999599+0000","last_became_peered":"2024-04-25T09:55:52.999599+0000","last_unstale":"2024-04-25T09:55:53.006467+0000","last_undegraded":"2024-04-25T09:55:53.006467+0000","last_fullsized":"2024-04-25T09:55:53.006467+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:54.667894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.986279+0000","last_change":"2024-04-25T09:55:51.988216+0000","last_active":"2024-04-25T09:55:52.986279+0000","last_peered":"2024-04-25T09:55:52.986279+0000","last_clean":"2024-04-25T09:55:52.986279+0000","last_became_active":"2024-04-25T09:55:51.987893+0000","last_became_peered":"2024-04-25T09:55:51.987893+0000","last_unstale":"2024-04-25T09:55:52.986279+0000","last_undegraded":"2024-04-25T09:55:52.986279+0000","last_fullsized":"2024-04-25T09:55:52.986279+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:25.618033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.018372+0000","last_change":"2024-04-25T09:55:53.007097+0000","last_active":"2024-04-25T09:55:53.018372+0000","last_peered":"2024-04-25T09:55:53.018372+0000","last_clean":"2024-04-25T09:55:53.018372+0000","last_became_active":"2024-04-25T09:55:53.005896+0000","last_became_peered":"2024-04-25T09:55:53.005896+0000","last_unstale":"2024-04-25T09:55:53.018372+0000","last_undegraded":"2024-04-25T09:55:53.018372+0000","last_fullsized":"2024-04-25T09:55:53.018372+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:24.753929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":20,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.022092+0000","last_change":"2024-04-25T09:55:51.991337+0000","last_active":"2024-04-25T09:55:53.022092+0000","last_peered":"2024-04-25T09:55:53.022092+0000","last_clean":"2024-04-25T09:55:53.022092+0000","last_became_active":"2024-04-25T09:55:51.991034+0000","last_became_peered":"2024-04-25T09:55:51.991034+0000","last_unstale":"2024-04-25T09:55:53.022092+0000","last_undegraded":"2024-04-25T09:55:53.022092+0000","last_fullsized":"2024-04-25T09:55:53.022092+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:23.748777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007195+0000","last_change":"2024-04-25T09:55:53.001466+0000","last_active":"2024-04-25T09:55:53.007195+0000","last_peered":"2024-04-25T09:55:53.007195+0000","last_clean":"2024-04-25T09:55:53.007195+0000","last_became_active":"2024-04-25T09:55:53.001198+0000","last_became_peered":"2024-04-25T09:55:53.001198+0000","last_unstale":"2024-04-25T09:55:53.007195+0000","last_undegraded":"2024-04-25T09:55:53.007195+0000","last_fullsized":"2024-04-25T09:55:53.007195+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:35.312208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.022949+0000","last_change":"2024-04-25T09:55:51.999230+0000","last_active":"2024-04-25T09:55:53.022949+0000","last_peered":"2024-04-25T09:55:53.022949+0000","last_clean":"2024-04-25T09:55:53.022949+0000","last_became_active":"2024-04-25T09:55:51.998837+0000","last_became_peered":"2024-04-25T09:55:51.998837+0000","last_unstale":"2024-04-25T09:55:53.022949+0000","last_undegraded":"2024-04-25T09:55:53.022949+0000","last_fullsized":"2024-04-25T09:55:53.022949+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:28.135546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.001871+0000","last_change":"2024-04-25T09:55:52.999910+0000","last_active":"2024-04-25T09:55:53.001871+0000","last_peered":"2024-04-25T09:55:53.001871+0000","last_clean":"2024-04-25T09:55:53.001871+0000","last_became_active":"2024-04-25T09:55:52.999573+0000","last_became_peered":"2024-04-25T09:55:52.999573+0000","last_unstale":"2024-04-25T09:55:53.001871+0000","last_undegraded":"2024-04-25T09:55:53.001871+0000","last_fullsized":"2024-04-25T09:55:53.001871+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:39.250770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.014602+0000","last_change":"2024-04-25T09:55:51.983945+0000","last_active":"2024-04-25T09:55:53.014602+0000","last_peered":"2024-04-25T09:55:53.014602+0000","last_clean":"2024-04-25T09:55:53.014602+0000","last_became_active":"2024-04-25T09:55:51.983661+0000","last_became_peered":"2024-04-25T09:55:51.983661+0000","last_unstale":"2024-04-25T09:55:53.014602+0000","last_undegraded":"2024-04-25T09:55:53.014602+0000","last_fullsized":"2024-04-25T09:55:53.014602+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:25.781475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.010196+0000","last_change":"2024-04-25T09:55:53.006213+0000","last_active":"2024-04-25T09:55:53.010196+0000","last_peered":"2024-04-25T09:55:53.010196+0000","last_clean":"2024-04-25T09:55:53.010196+0000","last_became_active":"2024-04-25T09:55:53.005830+0000","last_became_peered":"2024-04-25T09:55:53.005830+0000","last_unstale":"2024-04-25T09:55:53.010196+0000","last_undegraded":"2024-04-25T09:55:53.010196+0000","last_fullsized":"2024-04-25T09:55:53.010196+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:41.341650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.027916+0000","last_change":"2024-04-25T09:55:51.988662+0000","last_active":"2024-04-25T09:55:53.027916+0000","last_peered":"2024-04-25T09:55:53.027916+0000","last_clean":"2024-04-25T09:55:53.027916+0000","last_became_active":"2024-04-25T09:55:51.988240+0000","last_became_peered":"2024-04-25T09:55:51.988240+0000","last_unstale":"2024-04-25T09:55:53.027916+0000","last_undegraded":"2024-04-25T09:55:53.027916+0000","last_fullsized":"2024-04-25T09:55:53.027916+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:45.520401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.014484+0000","last_change":"2024-04-25T09:55:53.003334+0000","last_active":"2024-04-25T09:55:53.014484+0000","last_peered":"2024-04-25T09:55:53.014484+0000","last_clean":"2024-04-25T09:55:53.014484+0000","last_became_active":"2024-04-25T09:55:53.003016+0000","last_became_peered":"2024-04-25T09:55:53.003016+0000","last_unstale":"2024-04-25T09:55:53.014484+0000","last_undegraded":"2024-04-25T09:55:53.014484+0000","last_fullsized":"2024-04-25T09:55:53.014484+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:41.887886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.995644+0000","last_change":"2024-04-25T09:55:51.989935+0000","last_active":"2024-04-25T09:55:52.995644+0000","last_peered":"2024-04-25T09:55:52.995644+0000","last_clean":"2024-04-25T09:55:52.995644+0000","last_became_active":"2024-04-25T09:55:51.989742+0000","last_became_peered":"2024-04-25T09:55:51.989742+0000","last_unstale":"2024-04-25T09:55:52.995644+0000","last_undegraded":"2024-04-25T09:55:52.995644+0000","last_fullsized":"2024-04-25T09:55:52.995644+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:11.343893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.005471+0000","last_change":"2024-04-25T09:55:53.003026+0000","last_active":"2024-04-25T09:55:53.005471+0000","last_peered":"2024-04-25T09:55:53.005471+0000","last_clean":"2024-04-25T09:55:53.005471+0000","last_became_active":"2024-04-25T09:55:53.002568+0000","last_became_peered":"2024-04-25T09:55:53.002568+0000","last_unstale":"2024-04-25T09:55:53.005471+0000","last_undegraded":"2024-04-25T09:55:53.005471+0000","last_fullsized":"2024-04-25T09:55:53.005471+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:59.271860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.027685+0000","last_change":"2024-04-25T09:55:51.989032+0000","last_active":"2024-04-25T09:55:53.027685+0000","last_peered":"2024-04-25T09:55:53.027685+0000","last_clean":"2024-04-25T09:55:53.027685+0000","last_became_active":"2024-04-25T09:55:51.988718+0000","last_became_peered":"2024-04-25T09:55:51.988718+0000","last_unstale":"2024-04-25T09:55:53.027685+0000","last_undegraded":"2024-04-25T09:55:53.027685+0000","last_fullsized":"2024-04-25T09:55:53.027685+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:18.649858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.006491+0000","last_change":"2024-04-25T09:55:53.003389+0000","last_active":"2024-04-25T09:55:53.006491+0000","last_peered":"2024-04-25T09:55:53.006491+0000","last_clean":"2024-04-25T09:55:53.006491+0000","last_became_active":"2024-04-25T09:55:53.003125+0000","last_became_peered":"2024-04-25T09:55:53.003125+0000","last_unstale":"2024-04-25T09:55:53.006491+0000","last_undegraded":"2024-04-25T09:55:53.006491+0000","last_fullsized":"2024-04-25T09:55:53.006491+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:43.212203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.990323+0000","last_change":"2024-04-25T09:55:51.993882+0000","last_active":"2024-04-25T09:55:52.990323+0000","last_peered":"2024-04-25T09:55:52.990323+0000","last_clean":"2024-04-25T09:55:52.990323+0000","last_became_active":"2024-04-25T09:55:51.993593+0000","last_became_peered":"2024-04-25T09:55:51.993593+0000","last_unstale":"2024-04-25T09:55:52.990323+0000","last_undegraded":"2024-04-25T09:55:52.990323+0000","last_fullsized":"2024-04-25T09:55:52.990323+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:58.266708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.019092+0000","last_change":"2024-04-25T09:55:53.011398+0000","last_active":"2024-04-25T09:55:53.019092+0000","last_peered":"2024-04-25T09:55:53.019092+0000","last_clean":"2024-04-25T09:55:53.019092+0000","last_became_active":"2024-04-25T09:55:53.011112+0000","last_became_peered":"2024-04-25T09:55:53.011112+0000","last_unstale":"2024-04-25T09:55:53.019092+0000","last_undegraded":"2024-04-25T09:55:53.019092+0000","last_fullsized":"2024-04-25T09:55:53.019092+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:54.667894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.003521+0000","last_change":"2024-04-25T09:55:51.990925+0000","last_active":"2024-04-25T09:55:53.003521+0000","last_peered":"2024-04-25T09:55:53.003521+0000","last_clean":"2024-04-25T09:55:53.003521+0000","last_became_active":"2024-04-25T09:55:51.990609+0000","last_became_peered":"2024-04-25T09:55:51.990609+0000","last_unstale":"2024-04-25T09:55:53.003521+0000","last_undegraded":"2024-04-25T09:55:53.003521+0000","last_fullsized":"2024-04-25T09:55:53.003521+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:44.668846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.014147+0000","last_change":"2024-04-25T09:55:53.002514+0000","last_active":"2024-04-25T09:55:53.014147+0000","last_peered":"2024-04-25T09:55:53.014147+0000","last_clean":"2024-04-25T09:55:53.014147+0000","last_became_active":"2024-04-25T09:55:53.002202+0000","last_became_peered":"2024-04-25T09:55:53.002202+0000","last_unstale":"2024-04-25T09:55:53.014147+0000","last_undegraded":"2024-04-25T09:55:53.014147+0000","last_fullsized":"2024-04-25T09:55:53.014147+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:47.218810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.989215+0000","last_change":"2024-04-25T09:55:51.979294+0000","last_active":"2024-04-25T09:55:52.989215+0000","last_peered":"2024-04-25T09:55:52.989215+0000","last_clean":"2024-04-25T09:55:52.989215+0000","last_became_active":"2024-04-25T09:55:51.978984+0000","last_became_peered":"2024-04-25T09:55:51.978984+0000","last_unstale":"2024-04-25T09:55:52.989215+0000","last_undegraded":"2024-04-25T09:55:52.989215+0000","last_fullsized":"2024-04-25T09:55:52.989215+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:23.954728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.004253+0000","last_change":"2024-04-25T09:55:53.001722+0000","last_active":"2024-04-25T09:55:53.004253+0000","last_peered":"2024-04-25T09:55:53.004253+0000","last_clean":"2024-04-25T09:55:53.004253+0000","last_became_active":"2024-04-25T09:55:53.001344+0000","last_became_peered":"2024-04-25T09:55:53.001344+0000","last_unstale":"2024-04-25T09:55:53.004253+0000","last_undegraded":"2024-04-25T09:55:53.004253+0000","last_fullsized":"2024-04-25T09:55:53.004253+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:24.753929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.020695+0000","last_change":"2024-04-25T09:55:51.996429+0000","last_active":"2024-04-25T09:55:53.020695+0000","last_peered":"2024-04-25T09:55:53.020695+0000","last_clean":"2024-04-25T09:55:53.020695+0000","last_became_active":"2024-04-25T09:55:51.996137+0000","last_became_peered":"2024-04-25T09:55:51.996137+0000","last_unstale":"2024-04-25T09:55:53.020695+0000","last_undegraded":"2024-04-25T09:55:53.020695+0000","last_fullsized":"2024-04-25T09:55:53.020695+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:17.524023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.994738+0000","last_change":"2024-04-25T09:55:52.992828+0000","last_active":"2024-04-25T09:55:52.994738+0000","last_peered":"2024-04-25T09:55:52.994738+0000","last_clean":"2024-04-25T09:55:52.994738+0000","last_became_active":"2024-04-25T09:55:52.991659+0000","last_became_peered":"2024-04-25T09:55:52.991659+0000","last_unstale":"2024-04-25T09:55:52.994738+0000","last_undegraded":"2024-04-25T09:55:52.994738+0000","last_fullsized":"2024-04-25T09:55:52.994738+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:07.070995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.988584+0000","last_change":"2024-04-25T09:55:51.984365+0000","last_active":"2024-04-25T09:55:52.988584+0000","last_peered":"2024-04-25T09:55:52.988584+0000","last_clean":"2024-04-25T09:55:52.988584+0000","last_became_active":"2024-04-25T09:55:51.984020+0000","last_became_peered":"2024-04-25T09:55:51.984020+0000","last_unstale":"2024-04-25T09:55:52.988584+0000","last_undegraded":"2024-04-25T09:55:52.988584+0000","last_fullsized":"2024-04-25T09:55:52.988584+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:22.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.004763+0000","last_change":"2024-04-25T09:55:52.999660+0000","last_active":"2024-04-25T09:55:53.004763+0000","last_peered":"2024-04-25T09:55:53.004763+0000","last_clean":"2024-04-25T09:55:53.004763+0000","last_became_active":"2024-04-25T09:55:52.999221+0000","last_became_peered":"2024-04-25T09:55:52.999221+0000","last_unstale":"2024-04-25T09:55:53.004763+0000","last_undegraded":"2024-04-25T09:55:53.004763+0000","last_fullsized":"2024-04-25T09:55:53.004763+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.863701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007276+0000","last_change":"2024-04-25T09:55:51.990060+0000","last_active":"2024-04-25T09:55:53.007276+0000","last_peered":"2024-04-25T09:55:53.007276+0000","last_clean":"2024-04-25T09:55:53.007276+0000","last_became_active":"2024-04-25T09:55:51.989550+0000","last_became_peered":"2024-04-25T09:55:51.989550+0000","last_unstale":"2024-04-25T09:55:53.007276+0000","last_undegraded":"2024-04-25T09:55:53.007276+0000","last_fullsized":"2024-04-25T09:55:53.007276+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:46.099217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.023115+0000","last_change":"2024-04-25T09:55:53.020590+0000","last_active":"2024-04-25T09:55:53.023115+0000","last_peered":"2024-04-25T09:55:53.023115+0000","last_clean":"2024-04-25T09:55:53.023115+0000","last_became_active":"2024-04-25T09:55:53.020321+0000","last_became_peered":"2024-04-25T09:55:53.020321+0000","last_unstale":"2024-04-25T09:55:53.023115+0000","last_undegraded":"2024-04-25T09:55:53.023115+0000","last_fullsized":"2024-04-25T09:55:53.023115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:36.043335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.990689+0000","last_change":"2024-04-25T09:55:51.993835+0000","last_active":"2024-04-25T09:55:52.990689+0000","last_peered":"2024-04-25T09:55:52.990689+0000","last_clean":"2024-04-25T09:55:52.990689+0000","last_became_active":"2024-04-25T09:55:51.993512+0000","last_became_peered":"2024-04-25T09:55:51.993512+0000","last_unstale":"2024-04-25T09:55:52.990689+0000","last_undegraded":"2024-04-25T09:55:52.990689+0000","last_fullsized":"2024-04-25T09:55:52.990689+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:54.097861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.005475+0000","last_change":"2024-04-25T09:55:53.002879+0000","last_active":"2024-04-25T09:55:53.005475+0000","last_peered":"2024-04-25T09:55:53.005475+0000","last_clean":"2024-04-25T09:55:53.005475+0000","last_became_active":"2024-04-25T09:55:53.002623+0000","last_became_peered":"2024-04-25T09:55:53.002623+0000","last_unstale":"2024-04-25T09:55:53.005475+0000","last_undegraded":"2024-04-25T09:55:53.005475+0000","last_fullsized":"2024-04-25T09:55:53.005475+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:47.104369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.991169+0000","last_change":"2024-04-25T09:55:51.994640+0000","last_active":"2024-04-25T09:55:52.991169+0000","last_peered":"2024-04-25T09:55:52.991169+0000","last_clean":"2024-04-25T09:55:52.991169+0000","last_became_active":"2024-04-25T09:55:51.994372+0000","last_became_peered":"2024-04-25T09:55:51.994372+0000","last_unstale":"2024-04-25T09:55:52.991169+0000","last_undegraded":"2024-04-25T09:55:52.991169+0000","last_fullsized":"2024-04-25T09:55:52.991169+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:21.949533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.020091+0000","last_change":"2024-04-25T09:55:53.016992+0000","last_active":"2024-04-25T09:55:53.020091+0000","last_peered":"2024-04-25T09:55:53.020091+0000","last_clean":"2024-04-25T09:55:53.020091+0000","last_became_active":"2024-04-25T09:55:53.016729+0000","last_became_peered":"2024-04-25T09:55:53.016729+0000","last_unstale":"2024-04-25T09:55:53.020091+0000","last_undegraded":"2024-04-25T09:55:53.020091+0000","last_fullsized":"2024-04-25T09:55:53.020091+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:44.800887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.986223+0000","last_change":"2024-04-25T09:55:51.987260+0000","last_active":"2024-04-25T09:55:52.986223+0000","last_peered":"2024-04-25T09:55:52.986223+0000","last_clean":"2024-04-25T09:55:52.986223+0000","last_became_active":"2024-04-25T09:55:51.986970+0000","last_became_peered":"2024-04-25T09:55:51.986970+0000","last_unstale":"2024-04-25T09:55:52.986223+0000","last_undegraded":"2024-04-25T09:55:52.986223+0000","last_fullsized":"2024-04-25T09:55:52.986223+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:08.766351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.999475+0000","last_change":"2024-04-25T09:55:52.998285+0000","last_active":"2024-04-25T09:55:52.999475+0000","last_peered":"2024-04-25T09:55:52.999475+0000","last_clean":"2024-04-25T09:55:52.999475+0000","last_became_active":"2024-04-25T09:55:52.997883+0000","last_became_peered":"2024-04-25T09:55:52.997883+0000","last_unstale":"2024-04-25T09:55:52.999475+0000","last_undegraded":"2024-04-25T09:55:52.999475+0000","last_fullsized":"2024-04-25T09:55:52.999475+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:28.108495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.995360+0000","last_change":"2024-04-25T09:55:51.988688+0000","last_active":"2024-04-25T09:55:52.995360+0000","last_peered":"2024-04-25T09:55:52.995360+0000","last_clean":"2024-04-25T09:55:52.995360+0000","last_became_active":"2024-04-25T09:55:51.988369+0000","last_became_peered":"2024-04-25T09:55:51.988369+0000","last_unstale":"2024-04-25T09:55:52.995360+0000","last_undegraded":"2024-04-25T09:55:52.995360+0000","last_fullsized":"2024-04-25T09:55:52.995360+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:28.135546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007695+0000","last_change":"2024-04-25T09:55:53.004171+0000","last_active":"2024-04-25T09:55:53.007695+0000","last_peered":"2024-04-25T09:55:53.007695+0000","last_clean":"2024-04-25T09:55:53.007695+0000","last_became_active":"2024-04-25T09:55:53.003882+0000","last_became_peered":"2024-04-25T09:55:53.003882+0000","last_unstale":"2024-04-25T09:55:53.007695+0000","last_undegraded":"2024-04-25T09:55:53.007695+0000","last_fullsized":"2024-04-25T09:55:53.007695+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:51.969756+0000","last_clean_scrub_stamp":"2024-04-25T09:55:51.969756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:07.070995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:53.007279+0000","last_change":"2024-04-25T09:55:51.990622+0000","last_active":"2024-04-25T09:55:53.007279+0000","last_peered":"2024-04-25T09:55:53.007279+0000","last_clean":"2024-04-25T09:55:53.007279+0000","last_became_active":"2024-04-25T09:55:51.990354+0000","last_became_peered":"2024-04-25T09:55:51.990354+0000","last_unstale":"2024-04-25T09:55:53.007279+0000","last_undegraded":"2024-04-25T09:55:53.007279+0000","last_fullsized":"2024-04-25T09:55:53.007279+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.964604+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.964604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:58.266708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":235,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.985783+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:55:52.985783+0000","last_peered":"2024-04-25T09:55:52.985783+0000","last_clean":"2024-04-25T09:55:52.985783+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:55:52.985783+0000","last_undegraded":"2024-04-25T09:55:52.985783+0000","last_fullsized":"2024-04-25T09:55:52.985783+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36532,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335308,"statfs":{"total":96636764160,"available":96599355392,"internally_reserved":0,"allocated":1220608,"data_stored":867295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35952,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335888,"statfs":{"total":96636764160,"available":96599949312,"internally_reserved":0,"allocated":626688,"data_stored":277455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35948,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335892,"statfs":{"total":96636764160,"available":96599953408,"internally_reserved":0,"allocated":618496,"data_stored":276803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35952,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335888,"statfs":{"total":96636764160,"available":96599949312,"internally_reserved":0,"allocated":622592,"data_stored":277385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35968,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335872,"statfs":{"total":96636764160,"available":96599932928,"internally_reserved":0,"allocated":634880,"data_stored":278437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35960,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335880,"statfs":{"total":96636764160,"available":96599941120,"internally_reserved":0,"allocated":630784,"data_stored":278415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35956,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335884,"statfs":{"total":96636764160,"available":96599945216,"internally_reserved":0,"allocated":626688,"data_stored":276963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36536,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335304,"statfs":{"total":96636764160,"available":96599351296,"internally_reserved":0,"allocated":1220608,"data_stored":867227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:55:56.351 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:55:56.351 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:55:56.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.493+0000 7f71f399e640 1 Processor -- start 2024-04-25T09:55:56.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.493+0000 7f71f399e640 1 -- start start 2024-04-25T09:55:56.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f399e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71ec15dc20 0x7f71ec0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f399e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71ec161710 0x7f71ec0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f399e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ec15e000 con 0x7f71ec0b9330 2024-04-25T09:55:56.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f399e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ec0d1de0 con 0x7f71ec15dc20 2024-04-25T09:55:56.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f399e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec161c60 con 0x7f71ec161710 2024-04-25T09:55:56.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f0f39640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71ec161710 0x7f71ec0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f173a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71ec15dc20 0x7f71ec0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f1f3b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f71ec0b9330 0x7f71ec06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:58990/0 (socket says 172.21.15.19:58990) 2024-04-25T09:55:56.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f1f3b640 1 -- 172.21.15.19:0/1823694041 learned_addr learned my addr 172.21.15.19:0/1823694041 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:56.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f173a640 1 -- 172.21.15.19:0/1823694041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71ec161710 msgr2=0x7f71ec0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:56.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f173a640 1 --2- 172.21.15.19:0/1823694041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71ec161710 0x7f71ec0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f173a640 1 -- 172.21.15.19:0/1823694041 >> v1:172.21.15.112:6789/0 conn(0x7f71ec0b9330 legacy=0x7f71ec06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71f173a640 1 -- 172.21.15.19:0/1823694041 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec161f40 con 0x7f71ec15dc20 2024-04-25T09:55:56.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.494+0000 7f71e3fff640 1 -- 172.21.15.19:0/1823694041 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044826966 0 0) 0x7f71ec15e000 con 0x7f71ec0b9330 2024-04-25T09:55:56.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.495+0000 7f71f173a640 1 --2- 172.21.15.19:0/1823694041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ec15dc20 0x7f71ec0ce670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f71e80097a0 tx=0x7f71e802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daa12f052daad8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:56.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.495+0000 7f71e3fff640 1 -- 172.21.15.19:0/1823694041 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8037070 con 0x7f71ec15dc20 2024-04-25T09:55:56.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.495+0000 7f71e3fff640 1 -- 172.21.15.19:0/1823694041 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e8032440 con 0x7f71ec15dc20 2024-04-25T09:55:56.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.495+0000 7f71e3fff640 1 -- 172.21.15.19:0/1823694041 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8032810 con 0x7f71ec15dc20 2024-04-25T09:55:56.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.496+0000 7f71f399e640 1 -- 172.21.15.19:0/1823694041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ec15dc20 msgr2=0x7f71ec0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.496+0000 7f71f399e640 1 --2- 172.21.15.19:0/1823694041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ec15dc20 0x7f71ec0ce670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f71e80097a0 tx=0x7f71e802a180 comp rx=0 tx=0).stop 2024-04-25T09:55:56.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.496+0000 7f71f399e640 1 -- 172.21.15.19:0/1823694041 shutdown_connections 2024-04-25T09:55:56.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.496+0000 7f71f399e640 1 --2- 172.21.15.19:0/1823694041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71ec161710 0x7f71ec0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.496+0000 7f71f399e640 1 --2- 172.21.15.19:0/1823694041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ec15dc20 0x7f71ec0ce670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.496+0000 7f71f399e640 1 -- 172.21.15.19:0/1823694041 >> 172.21.15.19:0/1823694041 conn(0x7f71ec0d0420 msgr2=0x7f71ec0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:56.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.496+0000 7f71f399e640 1 -- 172.21.15.19:0/1823694041 shutdown_connections 2024-04-25T09:55:56.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.496+0000 7f71f399e640 1 -- 172.21.15.19:0/1823694041 wait complete. 2024-04-25T09:55:56.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.497+0000 7f71f399e640 1 Processor -- start 2024-04-25T09:55:56.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.497+0000 7f71f399e640 1 -- start start 2024-04-25T09:55:56.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f399e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71ec0b9330 0x7f71ec0c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f399e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ec15dc20 0x7f71ec0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f399e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec161710 0x7f71ec0b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f399e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec0b5e50 con 0x7f71ec161710 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f399e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec162500 con 0x7f71ec15dc20 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f399e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec0b5fd0 con 0x7f71ec0b9330 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f1f3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec161710 0x7f71ec0b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f0f39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ec15dc20 0x7f71ec0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f1f3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec161710 0x7f71ec0b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54350/0 (socket says 172.21.15.19:54350) 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f1f3b640 1 -- 172.21.15.19:0/1935602620 learned_addr learned my addr 172.21.15.19:0/1935602620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f173a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71ec0b9330 0x7f71ec0c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f1f3b640 1 -- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71ec0b9330 msgr2=0x7f71ec0c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f1f3b640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71ec0b9330 0x7f71ec0c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f1f3b640 1 -- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ec15dc20 msgr2=0x7f71ec0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f1f3b640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ec15dc20 0x7f71ec0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f1f3b640 1 -- 172.21.15.19:0/1935602620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec0ca2f0 con 0x7f71ec161710 2024-04-25T09:55:56.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.498+0000 7f71f0f39640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ec15dc20 0x7f71ec0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:56.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.499+0000 7f71f1f3b640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec161710 0x7f71ec0b49a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f71e400c7b0 tx=0x7f71e400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:56.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.499+0000 7f71e1ffb640 1 -- 172.21.15.19:0/1935602620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4018070 con 0x7f71ec161710 2024-04-25T09:55:56.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.499+0000 7f71e1ffb640 1 -- 172.21.15.19:0/1935602620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e400a640 con 0x7f71ec161710 2024-04-25T09:55:56.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.499+0000 7f71e1ffb640 1 -- 172.21.15.19:0/1935602620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e400ce60 con 0x7f71ec161710 2024-04-25T09:55:56.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.499+0000 7f71f399e640 1 -- 172.21.15.19:0/1935602620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ec1f7ba0 con 0x7f71ec161710 2024-04-25T09:55:56.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.499+0000 7f71f399e640 1 -- 172.21.15.19:0/1935602620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec1f3d90 con 0x7f71ec161710 2024-04-25T09:55:56.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.500+0000 7f71f399e640 1 -- 172.21.15.19:0/1935602620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ec073a00 con 0x7f71ec161710 2024-04-25T09:55:56.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.500+0000 7f71e1ffb640 1 -- 172.21.15.19:0/1935602620 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71e401b070 con 0x7f71ec161710 2024-04-25T09:55:56.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.501+0000 7f71e1ffb640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71d006b4d0 0x7f71d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.501+0000 7f71e1ffb640 1 -- 172.21.15.19:0/1935602620 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71e40897d0 con 0x7f71ec161710 2024-04-25T09:55:56.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.501+0000 7f71f173a640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71d006b4d0 0x7f71d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.501+0000 7f71f173a640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71d006b4d0 0x7f71d006d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f71e8034000 tx=0x7f71e802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:56.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.505+0000 7f71e1ffb640 1 -- 172.21.15.19:0/1935602620 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71e4057f00 con 0x7f71ec161710 2024-04-25T09:55:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.758+0000 7f71f399e640 1 -- 172.21.15.19:0/1935602620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f71ec0d16b0 con 0x7f71ec161710 2024-04-25T09:55:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.760+0000 7f71e1ffb640 1 -- 172.21.15.19:0/1935602620 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f71e405bfa0 con 0x7f71ec161710 2024-04-25T09:55:56.763 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T09:55:56.763 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T09:55:56.763 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T09:55:56.763 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T09:55:56.763 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:55:56.763 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T09:55:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 -- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71d006b4d0 msgr2=0x7f71d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71d006b4d0 0x7f71d006d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f71e8034000 tx=0x7f71e802e000 comp rx=0 tx=0).stop 2024-04-25T09:55:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 -- 172.21.15.19:0/1935602620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec161710 msgr2=0x7f71ec0b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec161710 0x7f71ec0b49a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f71e400c7b0 tx=0x7f71e400cc70 comp rx=0 tx=0).stop 2024-04-25T09:55:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 -- 172.21.15.19:0/1935602620 shutdown_connections 2024-04-25T09:55:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71d006b4d0 0x7f71d006d990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec161710 0x7f71ec0b49a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71ec15dc20 0x7f71ec0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 --2- 172.21.15.19:0/1935602620 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71ec0b9330 0x7f71ec0c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 -- 172.21.15.19:0/1935602620 >> 172.21.15.19:0/1935602620 conn(0x7f71ec0d0420 msgr2=0x7f71ec0d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 -- 172.21.15.19:0/1935602620 shutdown_connections 2024-04-25T09:55:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.762+0000 7f71f399e640 1 -- 172.21.15.19:0/1935602620 wait complete. 2024-04-25T09:55:56.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.874+0000 7f464f66b640 1 Processor -- start 2024-04-25T09:55:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.874+0000 7f464f66b640 1 -- start start 2024-04-25T09:55:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.875+0000 7f464f66b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f464815dd30 0x7f46480b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.875+0000 7f464f66b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46480d1df0 con 0x7f46480b9770 2024-04-25T09:55:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.875+0000 7f464f66b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46480d20d0 con 0x7f46480d0ad0 2024-04-25T09:55:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.875+0000 7f464f66b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46480d23b0 con 0x7f464815dd30 2024-04-25T09:55:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.875+0000 7f464cc06640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f46480b9770 0x7f46480d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35622/0 (socket says 172.21.15.19:35622) 2024-04-25T09:55:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.875+0000 7f464cc06640 1 -- 172.21.15.19:0/663788430 learned_addr learned my addr 172.21.15.19:0/663788430 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.875+0000 7f464d407640 1 --2- 172.21.15.19:0/663788430 >> v2:172.21.15.112:3300/0 conn(0x7f464815dd30 0x7f46480b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389186715 0 0) 0x7f46480d1df0 con 0x7f46480b9770 2024-04-25T09:55:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4634003520 con 0x7f46480b9770 2024-04-25T09:55:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 17566147 0 0) 0x7f4634003520 con 0x7f46480b9770 2024-04-25T09:55:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 >> v2:172.21.15.112:3300/0 conn(0x7f464815dd30 msgr2=0x7f46480b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 --2- 172.21.15.19:0/663788430 >> v2:172.21.15.112:3300/0 conn(0x7f464815dd30 0x7f46480b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 >> v1:172.21.15.112:6789/0 conn(0x7f46480d0ad0 legacy=0x7f46480b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46480d2690 con 0x7f46480b9770 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f46380017f0 con 0x7f46480b9770 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 >> v1:172.21.15.19:6789/0 conn(0x7f46480b9770 legacy=0x7f46480d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46340039b0 0x7f4634003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4634004800 0x7f4634004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46340052e0 0x7f463400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46480d23b0 con 0x7f4634004800 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463400c1b0 con 0x7f46340039b0 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463400c3b0 con 0x7f46340052e0 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f464cc06640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4634004800 0x7f4634004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f464d407640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46340039b0 0x7f4634003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f464dc08640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46340052e0 0x7f463400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f464cc06640 1 -- 172.21.15.19:0/663788430 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46340052e0 msgr2=0x7f463400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f464cc06640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46340052e0 0x7f463400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f464cc06640 1 -- 172.21.15.19:0/663788430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46340039b0 msgr2=0x7f4634003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f464cc06640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46340039b0 0x7f4634003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f464cc06640 1 -- 172.21.15.19:0/663788430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634003520 con 0x7f4634004800 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f464dc08640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46340052e0 0x7f463400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.876+0000 7f464dc08640 1 -- 172.21.15.19:0/663788430 reap_dead start 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.877+0000 7f464cc06640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4634004800 0x7f4634004c80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f463800c190 tx=0x7f4634005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d2bc8b8d67a807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.877+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638007d00 con 0x7f4634004800 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.877+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4638007270 con 0x7f4634004800 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.877+0000 7f463ffff640 1 -- 172.21.15.19:0/663788430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46380057a0 con 0x7f4634004800 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.878+0000 7f464f66b640 1 -- 172.21.15.19:0/663788430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4634004800 msgr2=0x7f4634004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.878+0000 7f464f66b640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4634004800 0x7f4634004c80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f463800c190 tx=0x7f4634005830 comp rx=0 tx=0).stop 2024-04-25T09:55:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.878+0000 7f464f66b640 1 -- 172.21.15.19:0/663788430 shutdown_connections 2024-04-25T09:55:56.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.878+0000 7f464f66b640 1 --2- 172.21.15.19:0/663788430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4634004800 0x7f4634004c80 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.878+0000 7f464f66b640 1 -- 172.21.15.19:0/663788430 >> 172.21.15.19:0/663788430 conn(0x7f4648061c30 msgr2=0x7f464806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:56.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.878+0000 7f464f66b640 1 -- 172.21.15.19:0/663788430 shutdown_connections 2024-04-25T09:55:56.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.878+0000 7f464f66b640 1 -- 172.21.15.19:0/663788430 wait complete. 2024-04-25T09:55:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.879+0000 7f464f66b640 1 Processor -- start 2024-04-25T09:55:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.879+0000 7f464f66b640 1 -- start start 2024-04-25T09:55:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464f66b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4634004800 0x7f46480caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464f66b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46481567e0 0x7f4648149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464f66b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46480ce410 0x7f46480ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464f66b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46480d1b90 con 0x7f46481567e0 2024-04-25T09:55:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464f66b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46480d1a10 con 0x7f46480ce410 2024-04-25T09:55:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464f66b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46480d2e90 con 0x7f4634004800 2024-04-25T09:55:56.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464cc06640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46481567e0 0x7f4648149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464cc06640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46481567e0 0x7f4648149eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54364/0 (socket says 172.21.15.19:54364) 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464cc06640 1 -- 172.21.15.19:0/2117103908 learned_addr learned my addr 172.21.15.19:0/2117103908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464dc08640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46480ce410 0x7f46480ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464d407640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4634004800 0x7f46480caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464cc06640 1 -- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4634004800 msgr2=0x7f46480caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464cc06640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4634004800 0x7f46480caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464cc06640 1 -- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46480ce410 msgr2=0x7f46480ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464cc06640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46480ce410 0x7f46480ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.880+0000 7f464cc06640 1 -- 172.21.15.19:0/2117103908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464814a3f0 con 0x7f46481567e0 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.881+0000 7f464cc06640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46481567e0 0x7f4648149eb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4638001af0 tx=0x7f46480d1fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.881+0000 7f463dffb640 1 -- 172.21.15.19:0/2117103908 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638040020 con 0x7f46481567e0 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.881+0000 7f463dffb640 1 -- 172.21.15.19:0/2117103908 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4638000f70 con 0x7f46481567e0 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.881+0000 7f464f66b640 1 -- 172.21.15.19:0/2117103908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4648156c60 con 0x7f46481567e0 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.881+0000 7f463dffb640 1 -- 172.21.15.19:0/2117103908 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638037040 con 0x7f46481567e0 2024-04-25T09:55:56.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.881+0000 7f464f66b640 1 -- 172.21.15.19:0/2117103908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46480cb420 con 0x7f46481567e0 2024-04-25T09:55:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.882+0000 7f464f66b640 1 -- 172.21.15.19:0/2117103908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46480cedd0 con 0x7f46481567e0 2024-04-25T09:55:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.882+0000 7f463dffb640 1 -- 172.21.15.19:0/2117103908 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4638043020 con 0x7f46481567e0 2024-04-25T09:55:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.883+0000 7f463dffb640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f462c06b500 0x7f462c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.883+0000 7f463dffb640 1 -- 172.21.15.19:0/2117103908 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f46380a9bb0 con 0x7f46481567e0 2024-04-25T09:55:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.883+0000 7f464d407640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f462c06b500 0x7f462c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:56.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.884+0000 7f464d407640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f462c06b500 0x7f462c06d9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4644009c40 tx=0x7f4644009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:56.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:56.887+0000 7f463dffb640 1 -- 172.21.15.19:0/2117103908 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f463804a070 con 0x7f46481567e0 2024-04-25T09:55:57.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.106+0000 7f464f66b640 1 -- 172.21.15.19:0/2117103908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4648073990 con 0x7f46481567e0 2024-04-25T09:55:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.107+0000 7f463dffb640 1 -- 172.21.15.19:0/2117103908 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f4638078950 con 0x7f46481567e0 2024-04-25T09:55:57.110 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:55:57.110 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:55:54.352541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:55:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-04-25T09:55:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 -- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f462c06b500 msgr2=0x7f462c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f462c06b500 0x7f462c06d9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4644009c40 tx=0x7f4644009190 comp rx=0 tx=0).stop 2024-04-25T09:55:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 -- 172.21.15.19:0/2117103908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46481567e0 msgr2=0x7f4648149eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46481567e0 0x7f4648149eb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4638001af0 tx=0x7f46480d1fc0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 -- 172.21.15.19:0/2117103908 shutdown_connections 2024-04-25T09:55:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f462c06b500 0x7f462c06d9c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46480ce410 0x7f46480ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46481567e0 0x7f4648149eb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 --2- 172.21.15.19:0/2117103908 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4634004800 0x7f46480caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 -- 172.21.15.19:0/2117103908 >> 172.21.15.19:0/2117103908 conn(0x7f4648061c30 msgr2=0x7f46480b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 -- 172.21.15.19:0/2117103908 shutdown_connections 2024-04-25T09:55:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.109+0000 7f464f66b640 1 -- 172.21.15.19:0/2117103908 wait complete. 2024-04-25T09:55:57.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:55:52.345127+0000', 'modified': '2024-04-25T09:55:54.352541+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5739': {'gid': 5739, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/2436445353', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2436445353}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2436445353}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:55:57.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:55:57.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:55:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.629+0000 7fcd7ed09640 1 Processor -- start 2024-04-25T09:55:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.629+0000 7fcd7ed09640 1 -- start start 2024-04-25T09:55:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.630+0000 7fcd7ed09640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcd7815de80 0x7fcd780cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.630+0000 7fcd7ed09640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd780ccfe0 0x7fcd780cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.630+0000 7fcd7ed09640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd780afc80 con 0x7fcd780cefb0 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.630+0000 7fcd7ed09640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd780ce4d0 con 0x7fcd780ccfe0 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.630+0000 7fcd7ed09640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd780ce7b0 con 0x7fcd7815de80 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.630+0000 7fcd7d2a6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd780ccfe0 0x7fcd780cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.630+0000 7fcd77fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcd780cefb0 0x7fcd780cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35628/0 (socket says 172.21.15.19:35628) 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.631+0000 7fcd77fff640 1 -- 172.21.15.19:0/530130919 learned_addr learned my addr 172.21.15.19:0/530130919 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.631+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/530130919 >> v2:172.21.15.112:3300/0 conn(0x7fcd7815de80 0x7fcd780cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.631+0000 7fcd7d2a6640 1 -- 172.21.15.19:0/530130919 >> v2:172.21.15.112:3300/0 conn(0x7fcd7815de80 msgr2=0x7fcd780cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.631+0000 7fcd7d2a6640 1 --2- 172.21.15.19:0/530130919 >> v2:172.21.15.112:3300/0 conn(0x7fcd7815de80 0x7fcd780cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.631+0000 7fcd7d2a6640 1 -- 172.21.15.19:0/530130919 >> v1:172.21.15.19:6789/0 conn(0x7fcd780cefb0 legacy=0x7fcd780cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.631+0000 7fcd7d2a6640 1 -- 172.21.15.19:0/530130919 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd780cea90 con 0x7fcd780ccfe0 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.631+0000 7fcd777fe640 1 -- 172.21.15.19:0/530130919 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751504629 0 0) 0x7fcd780afc80 con 0x7fcd780cefb0 2024-04-25T09:55:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd7d2a6640 1 --2- 172.21.15.19:0/530130919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd780ccfe0 0x7fcd780cd410 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcd780d1800 tx=0x7fcd6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7da2621c6a6a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:57.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd777fe640 1 -- 172.21.15.19:0/530130919 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68037070 con 0x7fcd780ccfe0 2024-04-25T09:55:57.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd777fe640 1 -- 172.21.15.19:0/530130919 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd68032440 con 0x7fcd780ccfe0 2024-04-25T09:55:57.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd777fe640 1 -- 172.21.15.19:0/530130919 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68032810 con 0x7fcd780ccfe0 2024-04-25T09:55:57.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd7ed09640 1 -- 172.21.15.19:0/530130919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd780ccfe0 msgr2=0x7fcd780cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd7ed09640 1 --2- 172.21.15.19:0/530130919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd780ccfe0 0x7fcd780cd410 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcd780d1800 tx=0x7fcd6802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd7ed09640 1 -- 172.21.15.19:0/530130919 shutdown_connections 2024-04-25T09:55:57.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd7ed09640 1 --2- 172.21.15.19:0/530130919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd780ccfe0 0x7fcd780cd410 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd7ed09640 1 --2- 172.21.15.19:0/530130919 >> v2:172.21.15.112:3300/0 conn(0x7fcd7815de80 0x7fcd780cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd7ed09640 1 -- 172.21.15.19:0/530130919 >> 172.21.15.19:0/530130919 conn(0x7fcd780b2390 msgr2=0x7fcd780b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd7ed09640 1 -- 172.21.15.19:0/530130919 shutdown_connections 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.632+0000 7fcd7ed09640 1 -- 172.21.15.19:0/530130919 wait complete. 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.634+0000 7fcd7ed09640 1 Processor -- start 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.634+0000 7fcd7ed09640 1 -- start start 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.634+0000 7fcd7ed09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd780ccfe0 0x7fcd780cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.634+0000 7fcd7ed09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd780cefb0 0x7fcd78151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.634+0000 7fcd7ed09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd7815de80 0x7fcd780ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.634+0000 7fcd7ed09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780cfb50 con 0x7fcd780ccfe0 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.634+0000 7fcd7ed09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780b2eb0 con 0x7fcd780cefb0 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.634+0000 7fcd7ed09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd781f7460 con 0x7fcd7815de80 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd7caa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd780ccfe0 0x7fcd780cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd7caa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd780ccfe0 0x7fcd780cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54378/0 (socket says 172.21.15.19:54378) 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3983006810 learned_addr learned my addr 172.21.15.19:0/3983006810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd77fff640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd780cefb0 0x7fcd78151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd7815de80 msgr2=0x7fcd780ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd7815de80 0x7fcd780ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd780cefb0 msgr2=0x7fcd78151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd780cefb0 0x7fcd78151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3983006810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd781f7740 con 0x7fcd780ccfe0 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd780ccfe0 0x7fcd780cc940 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000efc0 tx=0x7fcd7000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd757fa640 1 -- 172.21.15.19:0/3983006810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70018020 con 0x7fcd780ccfe0 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd757fa640 1 -- 172.21.15.19:0/3983006810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd70014480 con 0x7fcd780ccfe0 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd7ed09640 1 -- 172.21.15.19:0/3983006810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd781f7a20 con 0x7fcd780ccfe0 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd757fa640 1 -- 172.21.15.19:0/3983006810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70014850 con 0x7fcd780ccfe0 2024-04-25T09:55:57.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.635+0000 7fcd7ed09640 1 -- 172.21.15.19:0/3983006810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd780c96f0 con 0x7fcd780ccfe0 2024-04-25T09:55:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.637+0000 7fcd757fa640 1 -- 172.21.15.19:0/3983006810 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd70005d10 con 0x7fcd780ccfe0 2024-04-25T09:55:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.638+0000 7fcd757fa640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6406b4d0 0x7fcd6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.638+0000 7fcd757fa640 1 -- 172.21.15.19:0/3983006810 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd7008ae90 con 0x7fcd780ccfe0 2024-04-25T09:55:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.638+0000 7fcd77fff640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6406b4d0 0x7fcd6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.638+0000 7fcd81a15740 1 Processor -- start 2024-04-25T09:55:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.638+0000 7fcd81a15740 1 -- start start 2024-04-25T09:55:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.638+0000 7fcd77fff640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6406b4d0 0x7fcd6406d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c007fd0 tx=0x7fcd6c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.639+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bd0230 0x55fe55bd0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.639+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bedad0 0x55fe55bee020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.639+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55ba5f10 0x55fe55bec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.639+0000 7fcd81a15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55b08a10 con 0x55fe55ba5f10 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.639+0000 7fcd81a15740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55ab1b30 con 0x55fe55bd0230 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.639+0000 7fcd81a15740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe55b04c50 con 0x55fe55bedad0 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.639+0000 7fcd7d2a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55ba5f10 0x55fe55bec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.639+0000 7fcd7d2a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55ba5f10 0x55fe55bec190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54382/0 (socket says 172.21.15.19:54382) 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.639+0000 7fcd7d2a6640 1 -- 172.21.15.19:0/1054495254 learned_addr learned my addr 172.21.15.19:0/1054495254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.639+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/1054495254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bd0230 0x55fe55bd0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.639+0000 7fcd77fff640 1 --2- 172.21.15.19:0/1054495254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bedad0 0x55fe55bee020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd7d2a6640 1 -- 172.21.15.19:0/1054495254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bedad0 msgr2=0x55fe55bee020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd7d2a6640 1 --2- 172.21.15.19:0/1054495254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bedad0 0x55fe55bee020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd7d2a6640 1 -- 172.21.15.19:0/1054495254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bd0230 msgr2=0x55fe55bd0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd7d2a6640 1 --2- 172.21.15.19:0/1054495254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bd0230 0x55fe55bd0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd7d2a6640 1 -- 172.21.15.19:0/1054495254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe55ac6e10 con 0x55fe55ba5f10 2024-04-25T09:55:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd7d2a6640 1 --2- 172.21.15.19:0/1054495254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55ba5f10 0x55fe55bec190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcd6802a1e0 tx=0x7fcd6802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54f8c8b3ded1c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd5e7fc640 1 -- 172.21.15.19:0/1054495254 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68037040 con 0x55fe55ba5f10 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd5e7fc640 1 -- 172.21.15.19:0/1054495254 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd68042070 con 0x55fe55ba5f10 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd5e7fc640 1 -- 172.21.15.19:0/1054495254 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68006e10 con 0x55fe55ba5f10 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd81a15740 1 -- 172.21.15.19:0/1054495254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55ba5f10 msgr2=0x55fe55bec190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd81a15740 1 --2- 172.21.15.19:0/1054495254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55ba5f10 0x55fe55bec190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcd6802a1e0 tx=0x7fcd6802c070 comp rx=0 tx=0).stop 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd81a15740 1 -- 172.21.15.19:0/1054495254 shutdown_connections 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd81a15740 1 --2- 172.21.15.19:0/1054495254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55ba5f10 0x55fe55bec190 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd81a15740 1 --2- 172.21.15.19:0/1054495254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bedad0 0x55fe55bee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd81a15740 1 --2- 172.21.15.19:0/1054495254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bd0230 0x55fe55bd0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.640+0000 7fcd81a15740 1 -- 172.21.15.19:0/1054495254 >> 172.21.15.19:0/1054495254 conn(0x55fe55ba2840 msgr2=0x55fe55beb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.641+0000 7fcd81a15740 1 -- 172.21.15.19:0/1054495254 shutdown_connections 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.641+0000 7fcd81a15740 1 -- 172.21.15.19:0/1054495254 wait complete. 2024-04-25T09:55:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.641+0000 7fcd81a15740 1 -- start start 2024-04-25T09:55:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.641+0000 7fcd81a15740 1 Processor -- start 2024-04-25T09:55:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bc1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bedad0 0x55fe55bd2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bd3780 0x55fe55bfaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd81a15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55ba6cb0 con 0x55fe55bd0230 2024-04-25T09:55:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd81a15740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55ac50c0 con 0x55fe55bedad0 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd81a15740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe55ba0770 con 0x55fe55bd3780 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd81a15740 10 client.0 _ll_register_callbacks cb 0x55fe55ab8ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd7caa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bc1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd7caa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bc1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54388/0 (socket says 172.21.15.19:54388) 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3905752698 learned_addr learned my addr 172.21.15.19:0/3905752698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd77fff640 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bedad0 0x55fe55bd2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd7d2a6640 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bd3780 0x55fe55bfaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3905752698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bd3780 msgr2=0x55fe55bfaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bd3780 0x55fe55bfaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3905752698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bedad0 msgr2=0x55fe55bd2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.642+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bedad0 0x55fe55bd2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3905752698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe5581d2d0 con 0x55fe55bd0230 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd7d2a6640 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bd3780 0x55fe55bfaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bc1a40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcd700558d0 tx=0x7fcd7005d700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be4196b88f322e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd577fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd577fe640 1 -- 172.21.15.19:0/3905752698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70087060 con 0x55fe55bd0230 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd577fe640 1 -- 172.21.15.19:0/3905752698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd70082b80 con 0x55fe55bd0230 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd81a15740 1 -- 172.21.15.19:0/3905752698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd68042070 con 0x55fe55bd0230 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd577fe640 1 -- 172.21.15.19:0/3905752698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd70082d80 con 0x55fe55bd0230 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd81a15740 10 client.5874 fetch_fsmap learned FSMap version 24 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd81a15740 1 -- 172.21.15.19:0/3905752698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55fe558e76b0 con 0x55fe55bd0230 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd577fe640 1 -- 172.21.15.19:0/3905752698 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd68042070 con 0x55fe55bd0230 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd577fe640 1 -- 172.21.15.19:0/3905752698 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd700588c0 con 0x55fe55bd0230 2024-04-25T09:55:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd81a15740 10 client.5874 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:55:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd81a15740 10 client.5874 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:55:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd81a15740 20 client.5874 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd81a15740 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55fe55be24d0 0x55fe55be28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd81a15740 4 client.5874 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:55:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd81a15740 1 -- 172.21.15.19:0/3905752698 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe55b67920 con 0x55fe55be24d0 2024-04-25T09:55:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.643+0000 7fcd77fff640 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55fe55be24d0 0x55fe55be28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd77fff640 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55fe55be24d0 0x55fe55be28d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d3084c6a41918 server_cookie=cf614178b6a0fb8a in_seq=0 out_seq=0 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd577fe640 10 client.5874 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd577fe640 1 -- 172.21.15.19:0/3905752698 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe55b67920 con 0x55fe55be24d0 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd577fe640 10 client.5874 handle_command_reply: tid=0 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd81a15740 1 client.5874 shutdown 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd81a15740 1 -- 172.21.15.19:0/3905752698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 msgr2=0x55fe55bc1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd81a15740 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bc1a40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcd700558d0 tx=0x7fcd7005d700 comp rx=0 tx=0).stop 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd81a15740 1 -- 172.21.15.19:0/3905752698 shutdown_connections 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd81a15740 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55fe55be24d0 0x55fe55be28d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd81a15740 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bd3780 0x55fe55bfaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd81a15740 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bedad0 0x55fe55bd2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd577fe640 0 client.5874 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.644+0000 7fcd81a15740 1 --2- 172.21.15.19:0/3905752698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bc1a40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.645+0000 7fcd81a15740 1 -- 172.21.15.19:0/3905752698 >> 172.21.15.19:0/3905752698 conn(0x55fe55ba5f10 msgr2=0x55fe55bee5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.645+0000 7fcd81a15740 1 -- 172.21.15.19:0/3905752698 shutdown_connections 2024-04-25T09:55:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.645+0000 7fcd81a15740 1 -- 172.21.15.19:0/3905752698 wait complete. 2024-04-25T09:55:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.645+0000 7fcd81a15740 20 client.5874 trim_cache size 0 max 16384 2024-04-25T09:55:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.660+0000 7fcd81a15740 1 Processor -- start 2024-04-25T09:55:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.660+0000 7fcd81a15740 1 -- start start 2024-04-25T09:55:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bfad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bd3780 0x55fe55ba6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bedad0 0x55fe55ba6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd81a15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd781ff590 con 0x55fe55bd0230 2024-04-25T09:55:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd81a15740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780b5af0 con 0x55fe55bedad0 2024-04-25T09:55:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd81a15740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780cb970 con 0x55fe55bd3780 2024-04-25T09:55:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd7caa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bfad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd7caa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bfad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54394/0 (socket says 172.21.15.19:54394) 2024-04-25T09:55:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3015734209 learned_addr learned my addr 172.21.15.19:0/3015734209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3015734209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bd3780 msgr2=0x55fe55ba6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/3015734209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bd3780 0x55fe55ba6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3015734209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bedad0 msgr2=0x55fe55ba6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/3015734209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bedad0 0x55fe55ba6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.661+0000 7fcd7caa5640 1 -- 172.21.15.19:0/3015734209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe55ba1d00 con 0x55fe55bd0230 2024-04-25T09:55:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/3015734209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bfad90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000ef90 tx=0x7fcd7006e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439011f140d4ce90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd5dffb640 1 -- 172.21.15.19:0/3015734209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7007f070 con 0x55fe55bd0230 2024-04-25T09:55:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd5dffb640 1 -- 172.21.15.19:0/3015734209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7005d2c0 con 0x55fe55bd0230 2024-04-25T09:55:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd5dffb640 1 -- 172.21.15.19:0/3015734209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70087050 con 0x55fe55bd0230 2024-04-25T09:55:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd81a15740 1 -- 172.21.15.19:0/3015734209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 msgr2=0x55fe55bfad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd81a15740 1 --2- 172.21.15.19:0/3015734209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bfad90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000ef90 tx=0x7fcd7006e000 comp rx=0 tx=0).stop 2024-04-25T09:55:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd81a15740 1 -- 172.21.15.19:0/3015734209 shutdown_connections 2024-04-25T09:55:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd81a15740 1 --2- 172.21.15.19:0/3015734209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bedad0 0x55fe55ba6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd81a15740 1 --2- 172.21.15.19:0/3015734209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bd3780 0x55fe55ba6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd81a15740 1 --2- 172.21.15.19:0/3015734209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd0230 0x55fe55bfad90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd81a15740 1 -- 172.21.15.19:0/3015734209 >> 172.21.15.19:0/3015734209 conn(0x55fe55ba5f10 msgr2=0x55fe55bd2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd81a15740 1 -- 172.21.15.19:0/3015734209 shutdown_connections 2024-04-25T09:55:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.662+0000 7fcd81a15740 1 -- 172.21.15.19:0/3015734209 wait complete. 2024-04-25T09:55:57.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.663+0000 7fcd81a15740 1 -- start start 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.663+0000 7fcd81a15740 1 Processor -- start 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bd0230 0x55fe55bedf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd3780 0x55fe55bee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd81a15740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bedad0 0x55fe55bf98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd81a15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55bfb8d0 con 0x55fe55bd3780 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd81a15740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55bd2970 con 0x55fe55bd0230 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd81a15740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe55b9f800 con 0x55fe55bedad0 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd81a15740 10 client.0 _ll_register_callbacks cb 0x55fe55ba0c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd77fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd3780 0x55fe55bee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd77fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd3780 0x55fe55bee4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46768/0 (socket says 172.21.15.19:46768) 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd77fff640 1 -- 172.21.15.19:0/2506616800 learned_addr learned my addr 172.21.15.19:0/2506616800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:57.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd7d2a6640 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bedad0 0x55fe55bf98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bd0230 0x55fe55bedf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd77fff640 1 -- 172.21.15.19:0/2506616800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bedad0 msgr2=0x55fe55bf98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd77fff640 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bedad0 0x55fe55bf98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd77fff640 1 -- 172.21.15.19:0/2506616800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bd0230 msgr2=0x55fe55bedf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd77fff640 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bd0230 0x55fe55bedf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd77fff640 1 -- 172.21.15.19:0/2506616800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe55c58fb0 con 0x55fe55bd3780 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd77fff640 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd3780 0x55fe55bee4c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcd780c9330 tx=0x7fcd6c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942d086a605551d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.664+0000 7fcd577fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd577fe640 1 -- 172.21.15.19:0/2506616800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c02b060 con 0x55fe55bd3780 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd577fe640 1 -- 172.21.15.19:0/2506616800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd6c0049f0 con 0x55fe55bd3780 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd81a15740 1 -- 172.21.15.19:0/2506616800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd7005d2c0 con 0x55fe55bd3780 2024-04-25T09:55:57.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd577fe640 1 -- 172.21.15.19:0/2506616800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd6c026e60 con 0x55fe55bd3780 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd81a15740 10 client.5880 fetch_fsmap learned FSMap version 24 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd81a15740 1 -- 172.21.15.19:0/2506616800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55fe55c591c0 con 0x55fe55bd3780 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd577fe640 1 -- 172.21.15.19:0/2506616800 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7005d2c0 con 0x55fe55bd3780 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd577fe640 1 -- 172.21.15.19:0/2506616800 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd6c004bf0 con 0x55fe55bd3780 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd81a15740 10 client.5880 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd81a15740 10 client.5880 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd81a15740 20 client.5880 populate_metadata read hostname 'smithi019' 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd81a15740 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55fe55bd1020 0x55fe55bd1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd81a15740 4 client.5880 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd81a15740 1 -- 172.21.15.19:0/2506616800 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fe55b15f50 con 0x55fe55bd1020 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55fe55bd1020 0x55fe55bd1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.665+0000 7fcd577fe640 10 client.5880 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:55:57.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd7caa5640 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55fe55bd1020 0x55fe55bd1400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd6c6572795930f server_cookie=e2b5af8a015c1693 in_seq=0 out_seq=0 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd577fe640 1 -- 172.21.15.19:0/2506616800 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55fe55b15f50 con 0x55fe55bd1020 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd577fe640 10 client.5880 handle_command_reply: tid=0 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 1 client.5880 shutdown 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 1 -- 172.21.15.19:0/2506616800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd3780 msgr2=0x55fe55bee4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd3780 0x55fe55bee4c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcd780c9330 tx=0x7fcd6c007780 comp rx=0 tx=0).stop 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 1 -- 172.21.15.19:0/2506616800 shutdown_connections 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55fe55bd1020 0x55fe55bd1400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fe55bedad0 0x55fe55bf98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe55bd3780 0x55fe55bee4c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd577fe640 0 client.5880 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 1 --2- 172.21.15.19:0/2506616800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe55bd0230 0x55fe55bedf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 1 -- 172.21.15.19:0/2506616800 >> 172.21.15.19:0/2506616800 conn(0x55fe55ba5f10 msgr2=0x55fe55ba6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:57.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 1 -- 172.21.15.19:0/2506616800 shutdown_connections 2024-04-25T09:55:57.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 1 -- 172.21.15.19:0/2506616800 wait complete. 2024-04-25T09:55:57.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.666+0000 7fcd81a15740 20 client.5880 trim_cache size 0 max 16384 2024-04-25T09:55:57.676 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:55:57.676 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T09:55:57.676 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T09:55:57.676 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 5739, 2024-04-25T09:55:57.677 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T09:55:57.677 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T09:55:57.677 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T09:55:57.677 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 4.6739577350000001, 2024-04-25T09:55:57.677 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 24, 2024-04-25T09:55:57.677 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 53, 2024-04-25T09:55:57.677 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 53, 2024-04-25T09:55:57.677 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 9.7084597220000006 2024-04-25T09:55:57.677 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:55:57.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 -- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6406b4d0 msgr2=0x7fcd6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6406b4d0 0x7fcd6406d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c007fd0 tx=0x7fcd6c009040 comp rx=0 tx=0).stop 2024-04-25T09:55:57.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 -- 172.21.15.19:0/3983006810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd780ccfe0 msgr2=0x7fcd780cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd780ccfe0 0x7fcd780cc940 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000efc0 tx=0x7fcd7000c490 comp rx=0 tx=0).stop 2024-04-25T09:55:57.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 -- 172.21.15.19:0/3983006810 shutdown_connections 2024-04-25T09:55:57.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6406b4d0 0x7fcd6406d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd7815de80 0x7fcd780ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd780cefb0 0x7fcd78151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 --2- 172.21.15.19:0/3983006810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd780ccfe0 0x7fcd780cc940 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 -- 172.21.15.19:0/3983006810 >> 172.21.15.19:0/3983006810 conn(0x7fcd780b2390 msgr2=0x7fcd780cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:57.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 -- 172.21.15.19:0/3983006810 shutdown_connections 2024-04-25T09:55:57.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.667+0000 7fcd7ed09640 1 -- 172.21.15.19:0/3983006810 wait complete. 2024-04-25T09:55:57.680 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:55:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.779+0000 7f7a6e876640 1 Processor -- start 2024-04-25T09:55:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.779+0000 7f7a6e876640 1 -- start start 2024-04-25T09:55:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a6e876640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7a680c36d0 0x7f7a680afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a6e876640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a6806f3c0 0x7f7a680c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a6e876640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a680cef80 con 0x7f7a680cfa20 2024-04-25T09:55:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a6e876640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6806f880 con 0x7f7a6806f3c0 2024-04-25T09:55:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a6e876640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a680cf220 con 0x7f7a680c36d0 2024-04-25T09:55:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a677fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a6806f3c0 0x7f7a680c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a677fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a6806f3c0 0x7f7a680c3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46770/0 (socket says 172.21.15.19:46770) 2024-04-25T09:55:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a677fe640 1 -- 172.21.15.19:0/3957385753 learned_addr learned my addr 172.21.15.19:0/3957385753 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a6ce13640 1 --1- 172.21.15.19:0/3957385753 >> v1:172.21.15.19:6789/0 conn(0x7f7a680cfa20 0x7f7a680ad570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40538/0 (socket says 172.21.15.19:40538) 2024-04-25T09:55:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a67fff640 1 --2- 172.21.15.19:0/3957385753 >> v2:172.21.15.112:3300/0 conn(0x7f7a680c36d0 0x7f7a680afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a677fe640 1 -- 172.21.15.19:0/3957385753 >> v2:172.21.15.112:3300/0 conn(0x7f7a680c36d0 msgr2=0x7f7a680afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a66ffd640 1 -- 172.21.15.19:0/3957385753 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104813659 0 0) 0x7f7a680cef80 con 0x7f7a680cfa20 2024-04-25T09:55:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a677fe640 1 --2- 172.21.15.19:0/3957385753 >> v2:172.21.15.112:3300/0 conn(0x7f7a680c36d0 0x7f7a680afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a677fe640 1 -- 172.21.15.19:0/3957385753 >> v1:172.21.15.19:6789/0 conn(0x7f7a680cfa20 legacy=0x7f7a680ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a677fe640 1 -- 172.21.15.19:0/3957385753 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680cf500 con 0x7f7a6806f3c0 2024-04-25T09:55:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.780+0000 7f7a67fff640 1 --2- 172.21.15.19:0/3957385753 >> v2:172.21.15.112:3300/0 conn(0x7f7a680c36d0 0x7f7a680afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.781+0000 7f7a677fe640 1 --2- 172.21.15.19:0/3957385753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6806f3c0 0x7f7a680c3ab0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7a6806cf70 tx=0x7f7a5c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7219c6422c23089 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.781+0000 7f7a66ffd640 1 -- 172.21.15.19:0/3957385753 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c037070 con 0x7f7a6806f3c0 2024-04-25T09:55:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.781+0000 7f7a66ffd640 1 -- 172.21.15.19:0/3957385753 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c032440 con 0x7f7a6806f3c0 2024-04-25T09:55:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.781+0000 7f7a66ffd640 1 -- 172.21.15.19:0/3957385753 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c032810 con 0x7f7a6806f3c0 2024-04-25T09:55:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.781+0000 7f7a6e876640 1 -- 172.21.15.19:0/3957385753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6806f3c0 msgr2=0x7f7a680c3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.781+0000 7f7a6e876640 1 --2- 172.21.15.19:0/3957385753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6806f3c0 0x7f7a680c3ab0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7a6806cf70 tx=0x7f7a5c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.782+0000 7f7a6e876640 1 -- 172.21.15.19:0/3957385753 shutdown_connections 2024-04-25T09:55:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.782+0000 7f7a6e876640 1 --2- 172.21.15.19:0/3957385753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a6806f3c0 0x7f7a680c3ab0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.782+0000 7f7a6e876640 1 --2- 172.21.15.19:0/3957385753 >> v2:172.21.15.112:3300/0 conn(0x7f7a680c36d0 0x7f7a680afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.782+0000 7f7a6e876640 1 -- 172.21.15.19:0/3957385753 >> 172.21.15.19:0/3957385753 conn(0x7f7a680b2290 msgr2=0x7f7a680b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.782+0000 7f7a6e876640 1 -- 172.21.15.19:0/3957385753 shutdown_connections 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.782+0000 7f7a6e876640 1 -- 172.21.15.19:0/3957385753 wait complete. 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.783+0000 7f7a6e876640 1 Processor -- start 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.783+0000 7f7a6e876640 1 -- start start 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a6e876640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6806f3c0 0x7f7a681632b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a6e876640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a680c36d0 0x7f7a680b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a6e876640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a680cfa20 0x7f7a680b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a6e876640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6806fce0 con 0x7f7a680c36d0 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a6e876640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6806fb60 con 0x7f7a6806f3c0 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a6e876640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a681657f0 con 0x7f7a680cfa20 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a680c36d0 0x7f7a680b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a680c36d0 0x7f7a680b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46778/0 (socket says 172.21.15.19:46778) 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a677fe640 1 -- 172.21.15.19:0/3605159507 learned_addr learned my addr 172.21.15.19:0/3605159507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:55:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a67fff640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6806f3c0 0x7f7a681632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a6ce13640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a680cfa20 0x7f7a680b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a677fe640 1 -- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a680cfa20 msgr2=0x7f7a680b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a677fe640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a680cfa20 0x7f7a680b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a677fe640 1 -- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6806f3c0 msgr2=0x7f7a681632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a677fe640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6806f3c0 0x7f7a681632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a677fe640 1 -- 172.21.15.19:0/3605159507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a681637f0 con 0x7f7a680c36d0 2024-04-25T09:55:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a67fff640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6806f3c0 0x7f7a681632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a677fe640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a680c36d0 0x7f7a680b4dc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c02a220 tx=0x7f7a5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a64ff9640 1 -- 172.21.15.19:0/3605159507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c037040 con 0x7f7a680c36d0 2024-04-25T09:55:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a64ff9640 1 -- 172.21.15.19:0/3605159507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c042070 con 0x7f7a680c36d0 2024-04-25T09:55:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a64ff9640 1 -- 172.21.15.19:0/3605159507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c030bb0 con 0x7f7a680c36d0 2024-04-25T09:55:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.784+0000 7f7a6e876640 1 -- 172.21.15.19:0/3605159507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a680af770 con 0x7f7a680c36d0 2024-04-25T09:55:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.785+0000 7f7a6e876640 1 -- 172.21.15.19:0/3605159507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a680afd30 con 0x7f7a680c36d0 2024-04-25T09:55:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.786+0000 7f7a6e876640 1 -- 172.21.15.19:0/3605159507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a68073990 con 0x7f7a680c36d0 2024-04-25T09:55:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.786+0000 7f7a64ff9640 1 -- 172.21.15.19:0/3605159507 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a5c047460 con 0x7f7a680c36d0 2024-04-25T09:55:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.786+0000 7f7a64ff9640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a5406b4d0 0x7f7a5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.787+0000 7f7a64ff9640 1 -- 172.21.15.19:0/3605159507 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7a5c0ab060 con 0x7f7a680c36d0 2024-04-25T09:55:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.787+0000 7f7a67fff640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a5406b4d0 0x7f7a5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.787+0000 7f7a67fff640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a5406b4d0 0x7f7a5406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7a58006fd0 tx=0x7f7a58008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:57.791+0000 7f7a64ff9640 1 -- 172.21.15.19:0/3605159507 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a5c045360 con 0x7f7a680c36d0 2024-04-25T09:55:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.007+0000 7f7a6e876640 1 -- 172.21.15.19:0/3605159507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7a680b4ba0 con 0x7f7a680c36d0 2024-04-25T09:55:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.008+0000 7f7a64ff9640 1 -- 172.21.15.19:0/3605159507 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7a5c079cb0 con 0x7f7a680c36d0 2024-04-25T09:55:58.013 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:55:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 -- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a5406b4d0 msgr2=0x7f7a5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a5406b4d0 0x7f7a5406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7a58006fd0 tx=0x7f7a58008040 comp rx=0 tx=0).stop 2024-04-25T09:55:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 -- 172.21.15.19:0/3605159507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a680c36d0 msgr2=0x7f7a680b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a680c36d0 0x7f7a680b4dc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c02a220 tx=0x7f7a5c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 -- 172.21.15.19:0/3605159507 shutdown_connections 2024-04-25T09:55:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a5406b4d0 0x7f7a5406d990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a680cfa20 0x7f7a680b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a680c36d0 0x7f7a680b4dc0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 --2- 172.21.15.19:0/3605159507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6806f3c0 0x7f7a681632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 -- 172.21.15.19:0/3605159507 >> 172.21.15.19:0/3605159507 conn(0x7f7a680b2290 msgr2=0x7f7a680adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 -- 172.21.15.19:0/3605159507 shutdown_connections 2024-04-25T09:55:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58.010+0000 7f7a6e876640 1 -- 172.21.15.19:0/3605159507 wait complete. 2024-04-25T09:55:58.024 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:55:58.024 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:55:58.024 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T09:55:58.025 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:55:58.025 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:58.025 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:55:58.025 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:55:58.025 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:55:58.025 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:55:58.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:58.025 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T09:55:58.041 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:55:58.041 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:55:58.041 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:55:58.041 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T09:55:58.042 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:55:58.043 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42588sec preferred_lft 42588sec 2024-04-25T09:55:58.043 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T09:55:58.043 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:58.043 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:58.043 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:58.043 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T09:55:58.043 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:55:58.043 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:58.043 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:55:58.043 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:58.043 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T09:55:58.044 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:58.044 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:55:58.044 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:55:58.044 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T09:55:58.044 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:58.044 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:55:58.044 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:55:58.044 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T09:55:58.044 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:58.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:58.045 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:55:58.104 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T09:55:58.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:58.106 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T09:55:58.164 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T09:55:58.164 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:55:58.164 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:55:58.164 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:55:58.164 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:55:58.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:58.165 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:55:58.165 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:55:58.165 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:55:58.165 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:55:58.165 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:55:58.165 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:55:58.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:58.250 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:55:58.265 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:55:58.266 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:55:58.266 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:55:58.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:58.266 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T09:55:58.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:55:58.330 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:58.386 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:55:58.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:55:58.386 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:58.442 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:55:58.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:58.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:55:58.525 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:55:58.525 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:55:58.535 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:55:58.535 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:55:58.536 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T09:55:58.536 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T09:55:58.536 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T09:55:58.536 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:55:58.536 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:55:58.536 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:55:58.536 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:55:58.536 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:55:58.538 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:55:58.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:58.538 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:58.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:58.603 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:55:58.663 INFO:teuthology.orchestra.run.smithi019.stdout:client5892 2024-04-25T09:55:58.664 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:55:58.664 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:58.720 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:58.720 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T09:55:58.720 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T09:55:58.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:58.720 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T09:55:58.779 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T09:55:58.779 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> import json 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> except OSError as e: 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(e.errno) 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> print(json.dumps( 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> indent=2)) 2024-04-25T09:55:58.780 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T09:55:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:55:58 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T09:55:58.896 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T09:55:58.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:58.897 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T09:55:58.954 INFO:teuthology.orchestra.run.smithi019.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T09:55:59.110 INFO:teuthology.orchestra.run.smithi019.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T09:56:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:Updating files: 54% (760/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T09:56:01.757 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T09:56:01.757 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T09:56:01.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:01.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.894+0000 7f56bf2b3640 1 Processor -- start 2024-04-25T09:56:01.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.894+0000 7f56bf2b3640 1 -- start start 2024-04-25T09:56:01.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.895+0000 7f56bf2b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b815dd30 0x7f56b80b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.895+0000 7f56bf2b3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f56b80d0ad0 0x7f56b80b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.895+0000 7f56bf2b3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b80d1df0 con 0x7f56b80b9770 2024-04-25T09:56:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.895+0000 7f56bf2b3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b80d20d0 con 0x7f56b80d0ad0 2024-04-25T09:56:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.895+0000 7f56bf2b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b80d23b0 con 0x7f56b815dd30 2024-04-25T09:56:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.895+0000 7f56bc84e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f56b80b9770 0x7f56b80d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40542/0 (socket says 172.21.15.19:40542) 2024-04-25T09:56:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.895+0000 7f56bc84e640 1 -- 172.21.15.19:0/694705444 learned_addr learned my addr 172.21.15.19:0/694705444 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.895+0000 7f56bd850640 1 --2- 172.21.15.19:0/694705444 >> v2:172.21.15.112:3300/0 conn(0x7f56b80d0ad0 0x7f56b80b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.895+0000 7f56bd04f640 1 --2- 172.21.15.19:0/694705444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b815dd30 0x7f56b80b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.896+0000 7f56b7fff640 1 -- 172.21.15.19:0/694705444 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486056346 0 0) 0x7f56b80d1df0 con 0x7f56b80b9770 2024-04-25T09:56:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.896+0000 7f56b7fff640 1 -- 172.21.15.19:0/694705444 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a0003520 con 0x7f56b80b9770 2024-04-25T09:56:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.896+0000 7f56bd850640 1 -- 172.21.15.19:0/694705444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b815dd30 msgr2=0x7f56b80b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:01.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.896+0000 7f56bd850640 1 --2- 172.21.15.19:0/694705444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b815dd30 0x7f56b80b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.896+0000 7f56bd850640 1 -- 172.21.15.19:0/694705444 >> v1:172.21.15.19:6789/0 conn(0x7f56b80b9770 legacy=0x7f56b80d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:01.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.896+0000 7f56bd850640 1 -- 172.21.15.19:0/694705444 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80d2690 con 0x7f56b80d0ad0 2024-04-25T09:56:01.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.896+0000 7f56bd04f640 1 --2- 172.21.15.19:0/694705444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b815dd30 0x7f56b80b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:01.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.896+0000 7f56bd850640 1 --2- 172.21.15.19:0/694705444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b80d0ad0 0x7f56b80b6a40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009d90 tx=0x7f56a802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8c84254c3d7243e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:01.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.897+0000 7f56b7fff640 1 -- 172.21.15.19:0/694705444 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8031600 con 0x7f56b80d0ad0 2024-04-25T09:56:01.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.897+0000 7f56b7fff640 1 -- 172.21.15.19:0/694705444 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a80317a0 con 0x7f56b80d0ad0 2024-04-25T09:56:01.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.897+0000 7f56b7fff640 1 -- 172.21.15.19:0/694705444 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8031b70 con 0x7f56b80d0ad0 2024-04-25T09:56:01.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.897+0000 7f56bf2b3640 1 -- 172.21.15.19:0/694705444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b80d0ad0 msgr2=0x7f56b80b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:01.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.897+0000 7f56bf2b3640 1 --2- 172.21.15.19:0/694705444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b80d0ad0 0x7f56b80b6a40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009d90 tx=0x7f56a802a5d0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.897+0000 7f56bf2b3640 1 -- 172.21.15.19:0/694705444 shutdown_connections 2024-04-25T09:56:01.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.897+0000 7f56bf2b3640 1 --2- 172.21.15.19:0/694705444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b80d0ad0 0x7f56b80b6a40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.897+0000 7f56bf2b3640 1 --2- 172.21.15.19:0/694705444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b815dd30 0x7f56b80b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.897+0000 7f56bf2b3640 1 -- 172.21.15.19:0/694705444 >> 172.21.15.19:0/694705444 conn(0x7f56b8061c30 msgr2=0x7f56b806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.897+0000 7f56bf2b3640 1 -- 172.21.15.19:0/694705444 shutdown_connections 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.897+0000 7f56bf2b3640 1 -- 172.21.15.19:0/694705444 wait complete. 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.898+0000 7f56bf2b3640 1 Processor -- start 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.899+0000 7f56bf2b3640 1 -- start start 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.899+0000 7f56bf2b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b80b9770 0x7f56b80b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.899+0000 7f56bf2b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b80d0ad0 0x7f56b80cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.899+0000 7f56bf2b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b815dd30 0x7f56b81f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.899+0000 7f56bf2b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b80d1b90 con 0x7f56b80b9770 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.899+0000 7f56bf2b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b80d1a10 con 0x7f56b815dd30 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.899+0000 7f56bf2b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b80d2e90 con 0x7f56b80d0ad0 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.899+0000 7f56bd04f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b80b9770 0x7f56b80b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.899+0000 7f56bd04f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b80b9770 0x7f56b80b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46782/0 (socket says 172.21.15.19:46782) 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.899+0000 7f56bd04f640 1 -- 172.21.15.19:0/715225817 learned_addr learned my addr 172.21.15.19:0/715225817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bc84e640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b80d0ad0 0x7f56b80cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bd850640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b815dd30 0x7f56b81f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bd04f640 1 -- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b80d0ad0 msgr2=0x7f56b80cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bd04f640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b80d0ad0 0x7f56b80cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bd04f640 1 -- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b815dd30 msgr2=0x7f56b81f6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bd04f640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b815dd30 0x7f56b81f6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bd04f640 1 -- 172.21.15.19:0/715225817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80cbd90 con 0x7f56b80b9770 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bd850640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b815dd30 0x7f56b81f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bc84e640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b80d0ad0 0x7f56b80cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bd04f640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b80b9770 0x7f56b80b5d60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f56b000e960 tx=0x7f56b000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56b5ffb640 1 -- 172.21.15.19:0/715225817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0017070 con 0x7f56b80b9770 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bf2b3640 1 -- 172.21.15.19:0/715225817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b80cc070 con 0x7f56b80b9770 2024-04-25T09:56:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56b5ffb640 1 -- 172.21.15.19:0/715225817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b000ca00 con 0x7f56b80b9770 2024-04-25T09:56:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56b5ffb640 1 -- 172.21.15.19:0/715225817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b000cdd0 con 0x7f56b80b9770 2024-04-25T09:56:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.900+0000 7f56bf2b3640 1 -- 172.21.15.19:0/715225817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b80cb9d0 con 0x7f56b80b9770 2024-04-25T09:56:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.901+0000 7f56bf2b3640 1 -- 172.21.15.19:0/715225817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b8073990 con 0x7f56b80b9770 2024-04-25T09:56:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.902+0000 7f56b5ffb640 1 -- 172.21.15.19:0/715225817 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56b001a070 con 0x7f56b80b9770 2024-04-25T09:56:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.902+0000 7f56b5ffb640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a406b4d0 0x7f56a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.902+0000 7f56b5ffb640 1 -- 172.21.15.19:0/715225817 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56b008a370 con 0x7f56b80b9770 2024-04-25T09:56:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.903+0000 7f56bc84e640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a406b4d0 0x7f56a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.903+0000 7f56bc84e640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a406b4d0 0x7f56a406d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f56b80cd5f0 tx=0x7f56ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:01.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:01.907+0000 7f56b5ffb640 1 -- 172.21.15.19:0/715225817 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56b0058f40 con 0x7f56b80b9770 2024-04-25T09:56:02.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.126+0000 7f56bf2b3640 1 -- 172.21.15.19:0/715225817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56b815d2c0 con 0x7f56b80b9770 2024-04-25T09:56:02.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.127+0000 7f56b5ffb640 1 -- 172.21.15.19:0/715225817 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f56b0058840 con 0x7f56b80b9770 2024-04-25T09:56:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:02.131 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:55:54.352541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:02.131 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-04-25T09:56:02.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 -- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a406b4d0 msgr2=0x7f56a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a406b4d0 0x7f56a406d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f56b80cd5f0 tx=0x7f56ac009040 comp rx=0 tx=0).stop 2024-04-25T09:56:02.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 -- 172.21.15.19:0/715225817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b80b9770 msgr2=0x7f56b80b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b80b9770 0x7f56b80b5d60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f56b000e960 tx=0x7f56b000ee20 comp rx=0 tx=0).stop 2024-04-25T09:56:02.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 -- 172.21.15.19:0/715225817 shutdown_connections 2024-04-25T09:56:02.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a406b4d0 0x7f56a406d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b815dd30 0x7f56b81f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56b80d0ad0 0x7f56b80cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 --2- 172.21.15.19:0/715225817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56b80b9770 0x7f56b80b5d60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 -- 172.21.15.19:0/715225817 >> 172.21.15.19:0/715225817 conn(0x7f56b8061c30 msgr2=0x7f56b80ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 -- 172.21.15.19:0/715225817 shutdown_connections 2024-04-25T09:56:02.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.129+0000 7f56bf2b3640 1 -- 172.21.15.19:0/715225817 wait complete. 2024-04-25T09:56:02.144 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T09:56:02.250 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": -2 2024-04-25T09:56:02.250 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:02.258 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T09:56:02.259 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T09:56:02.259 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:02.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T09:56:02.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.359+0000 7fc7b2c4c640 1 Processor -- start 2024-04-25T09:56:02.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.359+0000 7fc7b2c4c640 1 -- start start 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.360+0000 7fc7b2c4c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc7ac15dc20 0x7fc7ac0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.360+0000 7fc7b2c4c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7ac0d32f0 0x7fc7ac0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.360+0000 7fc7b2c4c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7ac15e000 con 0x7fc7ac161710 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.360+0000 7fc7b2c4c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ac0d1de0 con 0x7fc7ac15dc20 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.360+0000 7fc7b2c4c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac161c60 con 0x7fc7ac0d32f0 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.361+0000 7fc7b09e8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc7ac15dc20 0x7fc7ac0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.361+0000 7fc7b11e9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7ac0d32f0 0x7fc7ac0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.361+0000 7fc7b09e8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc7ac15dc20 0x7fc7ac0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:58036/0 (socket says 172.21.15.19:58036) 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.361+0000 7fc7abfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc7ac161710 0x7fc7ac06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:56700/0 (socket says 172.21.15.19:56700) 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.361+0000 7fc7b09e8640 1 -- 172.21.15.19:0/788297279 learned_addr learned my addr 172.21.15.19:0/788297279 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.361+0000 7fc7b11e9640 1 -- 172.21.15.19:0/788297279 >> v2:172.21.15.112:3300/0 conn(0x7fc7ac15dc20 msgr2=0x7fc7ac0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.361+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/788297279 >> v2:172.21.15.112:3300/0 conn(0x7fc7ac15dc20 0x7fc7ac0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.361+0000 7fc7b11e9640 1 -- 172.21.15.19:0/788297279 >> v1:172.21.15.112:6789/0 conn(0x7fc7ac161710 legacy=0x7fc7ac06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.361+0000 7fc7b11e9640 1 -- 172.21.15.19:0/788297279 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac161f40 con 0x7fc7ac0d32f0 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.361+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/788297279 >> v2:172.21.15.112:3300/0 conn(0x7fc7ac15dc20 0x7fc7ac0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.362+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/788297279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7ac0d32f0 0x7fc7ac0cebb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0097a0 tx=0x7fc79c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=121a1d6a989a5481 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.362+0000 7fc7ab7fe640 1 -- 172.21.15.19:0/788297279 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c037070 con 0x7fc7ac0d32f0 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.362+0000 7fc7ab7fe640 1 -- 172.21.15.19:0/788297279 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c032440 con 0x7fc7ac0d32f0 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.362+0000 7fc7ab7fe640 1 -- 172.21.15.19:0/788297279 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c032810 con 0x7fc7ac0d32f0 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.362+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/788297279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7ac0d32f0 msgr2=0x7fc7ac0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.362+0000 7fc7b2c4c640 1 --2- 172.21.15.19:0/788297279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7ac0d32f0 0x7fc7ac0cebb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0097a0 tx=0x7fc79c02a180 comp rx=0 tx=0).stop 2024-04-25T09:56:02.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.362+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/788297279 shutdown_connections 2024-04-25T09:56:02.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.362+0000 7fc7b2c4c640 1 --2- 172.21.15.19:0/788297279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7ac0d32f0 0x7fc7ac0cebb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.362+0000 7fc7b2c4c640 1 --2- 172.21.15.19:0/788297279 >> v2:172.21.15.112:3300/0 conn(0x7fc7ac15dc20 0x7fc7ac0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.362+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/788297279 >> 172.21.15.19:0/788297279 conn(0x7fc7ac0d0420 msgr2=0x7fc7ac0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.363+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/788297279 shutdown_connections 2024-04-25T09:56:02.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.363+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/788297279 wait complete. 2024-04-25T09:56:02.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.364+0000 7fc7b2c4c640 1 Processor -- start 2024-04-25T09:56:02.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.364+0000 7fc7b2c4c640 1 -- start start 2024-04-25T09:56:02.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.364+0000 7fc7b2c4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7ac0d32f0 0x7fc7ac0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.364+0000 7fc7b2c4c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7ac15dc20 0x7fc7ac0ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.364+0000 7fc7b2c4c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac161710 0x7fc7ac0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.364+0000 7fc7b2c4c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0b5e50 con 0x7fc7ac161710 2024-04-25T09:56:02.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.364+0000 7fc7b2c4c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac162500 con 0x7fc7ac0d32f0 2024-04-25T09:56:02.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.364+0000 7fc7b2c4c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0b5fd0 con 0x7fc7ac15dc20 2024-04-25T09:56:02.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b11e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac161710 0x7fc7ac0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b11e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac161710 0x7fc7ac0cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46798/0 (socket says 172.21.15.19:46798) 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b11e9640 1 -- 172.21.15.19:0/2566298248 learned_addr learned my addr 172.21.15.19:0/2566298248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7ac0d32f0 0x7fc7ac0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7abfff640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7ac15dc20 0x7fc7ac0ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b11e9640 1 -- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7ac15dc20 msgr2=0x7fc7ac0ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7ac15dc20 0x7fc7ac0ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b11e9640 1 -- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7ac0d32f0 msgr2=0x7fc7ac0acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7ac0d32f0 0x7fc7ac0acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b11e9640 1 -- 172.21.15.19:0/2566298248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac0ccf90 con 0x7fc7ac161710 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7ac0d32f0 0x7fc7ac0acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac161710 0x7fc7ac0cbf60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc79c035040 tx=0x7fc79c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7a97fa640 1 -- 172.21.15.19:0/2566298248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c041020 con 0x7fc7ac161710 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7a97fa640 1 -- 172.21.15.19:0/2566298248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c03d640 con 0x7fc7ac161710 2024-04-25T09:56:02.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/2566298248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ac0cd270 con 0x7fc7ac161710 2024-04-25T09:56:02.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.365+0000 7fc7a97fa640 1 -- 172.21.15.19:0/2566298248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c037040 con 0x7fc7ac161710 2024-04-25T09:56:02.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.366+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/2566298248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac15ce90 con 0x7fc7ac161710 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.367+0000 7fc7a97fa640 1 -- 172.21.15.19:0/2566298248 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc79c044020 con 0x7fc7ac161710 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.368+0000 7fc7a97fa640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc78c06b4d0 0x7fc78c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.368+0000 7fc7a97fa640 1 -- 172.21.15.19:0/2566298248 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc79c0aa9a0 con 0x7fc7ac161710 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.368+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc78c06b4d0 0x7fc78c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.368+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc78c06b4d0 0x7fc78c06d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009d50 tx=0x7fc7a400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.369+0000 7fc7b5a20740 1 Processor -- start 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.369+0000 7fc7b5a20740 1 -- start start 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.369+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd102556b0 0x55bd10255a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.369+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10272f50 0x55bd102734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.369+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd1022b390 0x55bd10271610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.369+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1018dc60 con 0x55bd10272f50 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.369+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd10136ee0 con 0x55bd102556b0 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.369+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd10189d00 con 0x55bd1022b390 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.369+0000 7fc7abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10272f50 0x55bd102734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.369+0000 7fc7abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10272f50 0x55bd102734a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46804/0 (socket says 172.21.15.19:46804) 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.369+0000 7fc7abfff640 1 -- 172.21.15.19:0/3967709362 learned_addr learned my addr 172.21.15.19:0/3967709362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/3967709362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd102556b0 0x55bd10255a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/3967709362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd1022b390 0x55bd10271610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7abfff640 1 -- 172.21.15.19:0/3967709362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd1022b390 msgr2=0x55bd10271610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7abfff640 1 --2- 172.21.15.19:0/3967709362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd1022b390 0x55bd10271610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7abfff640 1 -- 172.21.15.19:0/3967709362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd102556b0 msgr2=0x55bd10255a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7abfff640 1 --2- 172.21.15.19:0/3967709362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd102556b0 0x55bd10255a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7abfff640 1 -- 172.21.15.19:0/3967709362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd0ff6e9f0 con 0x55bd10272f50 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/3967709362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd1022b390 0x55bd10271610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7abfff640 1 --2- 172.21.15.19:0/3967709362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10272f50 0x55bd102734a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc794004e10 tx=0x7fc794007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb18951b896befbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7a27fc640 1 -- 172.21.15.19:0/3967709362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7940017a0 con 0x55bd10272f50 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7a27fc640 1 -- 172.21.15.19:0/3967709362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc794018070 con 0x55bd10272f50 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.370+0000 7fc7a27fc640 1 -- 172.21.15.19:0/3967709362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794001b40 con 0x55bd10272f50 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.371+0000 7fc7b5a20740 1 -- 172.21.15.19:0/3967709362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10272f50 msgr2=0x55bd102734a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.371+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/3967709362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10272f50 0x55bd102734a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc794004e10 tx=0x7fc794007430 comp rx=0 tx=0).stop 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.371+0000 7fc7b5a20740 1 -- 172.21.15.19:0/3967709362 shutdown_connections 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.371+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/3967709362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd1022b390 0x55bd10271610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.371+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/3967709362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10272f50 0x55bd102734a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.371+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/3967709362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd102556b0 0x55bd10255a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.371+0000 7fc7b5a20740 1 -- 172.21.15.19:0/3967709362 >> 172.21.15.19:0/3967709362 conn(0x55bd10227cc0 msgr2=0x55bd102704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.371+0000 7fc7b5a20740 1 -- 172.21.15.19:0/3967709362 shutdown_connections 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.371+0000 7fc7b5a20740 1 -- 172.21.15.19:0/3967709362 wait complete. 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.371+0000 7fc7b5a20740 1 -- start start 2024-04-25T09:56:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.371+0000 7fc7b5a20740 1 Processor -- start 2024-04-25T09:56:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd1026f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10272f50 0x55bd10281060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd10269370 0x55bd10283bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1022c130 con 0x55bd102556b0 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1014a470 con 0x55bd10272f50 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd10225bf0 con 0x55bd10269370 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7b5a20740 10 client.0 _ll_register_callbacks cb 0x55bd101167d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7b09e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd1026f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7b09e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd1026f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46808/0 (socket says 172.21.15.19:46808) 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7b09e8640 1 -- 172.21.15.19:0/1108487880 learned_addr learned my addr 172.21.15.19:0/1108487880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7abfff640 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10272f50 0x55bd10281060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.372+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd10269370 0x55bd10283bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b09e8640 1 -- 172.21.15.19:0/1108487880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd10269370 msgr2=0x55bd10283bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd10269370 0x55bd10283bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b09e8640 1 -- 172.21.15.19:0/1108487880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10272f50 msgr2=0x55bd10281060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10272f50 0x55bd10281060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b09e8640 1 -- 172.21.15.19:0/1108487880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd0fe622e0 con 0x55bd102556b0 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd1026f300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4017f60 tx=0x7fc7a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd33bcc0aa9b362 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc78b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc78b7fe640 1 -- 172.21.15.19:0/1108487880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4026030 con 0x55bd102556b0 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc78b7fe640 1 -- 172.21.15.19:0/1108487880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc7a4022620 con 0x55bd102556b0 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b5a20740 1 -- 172.21.15.19:0/1108487880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc794018070 con 0x55bd102556b0 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc78b7fe640 1 -- 172.21.15.19:0/1108487880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7a4022820 con 0x55bd102556b0 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b5a20740 10 client.5907 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b5a20740 1 -- 172.21.15.19:0/1108487880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55bd0ffcf180 con 0x55bd102556b0 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc78b7fe640 1 -- 172.21.15.19:0/1108487880 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc794018070 con 0x55bd102556b0 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc78b7fe640 1 -- 172.21.15.19:0/1108487880 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc7a4035080 con 0x55bd102556b0 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b5a20740 10 client.5907 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b5a20740 10 client.5907 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b5a20740 20 client.5907 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55bd102840f0 0x55bd10224920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b5a20740 4 client.5907 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.373+0000 7fc7b5a20740 1 -- 172.21.15.19:0/1108487880 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd1019b1a0 con 0x55bd102840f0 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.374+0000 7fc7abfff640 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55bd102840f0 0x55bd10224920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.374+0000 7fc78b7fe640 10 client.5907 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.374+0000 7fc7abfff640 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55bd102840f0 0x55bd10224920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=549b78f603d8367e server_cookie=8e61bfdbad71258a in_seq=0 out_seq=0 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.374+0000 7fc78b7fe640 1 -- 172.21.15.19:0/1108487880 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd1019b1a0 con 0x55bd102840f0 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.374+0000 7fc78b7fe640 10 client.5907 handle_command_reply: tid=0 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.374+0000 7fc7b5a20740 1 client.5907 shutdown 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc7b5a20740 1 -- 172.21.15.19:0/1108487880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 msgr2=0x55bd1026f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd1026f300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4017f60 tx=0x7fc7a4004a40 comp rx=0 tx=0).stop 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc7b5a20740 1 -- 172.21.15.19:0/1108487880 shutdown_connections 2024-04-25T09:56:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55bd102840f0 0x55bd10224920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd10269370 0x55bd10283bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc78b7fe640 0 client.5907 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10272f50 0x55bd10281060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/1108487880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd1026f300 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc7b5a20740 1 -- 172.21.15.19:0/1108487880 >> 172.21.15.19:0/1108487880 conn(0x55bd1022b390 msgr2=0x55bd102742d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc7b5a20740 1 -- 172.21.15.19:0/1108487880 shutdown_connections 2024-04-25T09:56:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc7b5a20740 1 -- 172.21.15.19:0/1108487880 wait complete. 2024-04-25T09:56:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.375+0000 7fc7b5a20740 20 client.5907 trim_cache size 0 max 16384 2024-04-25T09:56:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.390+0000 7fc7b5a20740 1 Processor -- start 2024-04-25T09:56:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.390+0000 7fc7b5a20740 1 -- start start 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd102556b0 0x55bd1026f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10269370 0x55bd10252050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10272f50 0x55bd10224920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac2005b0 con 0x55bd10269370 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac1ff740 con 0x55bd10272f50 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac14e470 con 0x55bd102556b0 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10269370 0x55bd10252050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10269370 0x55bd10252050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46816/0 (socket says 172.21.15.19:46816) 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7abfff640 1 -- 172.21.15.19:0/441262117 learned_addr learned my addr 172.21.15.19:0/441262117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7abfff640 1 -- 172.21.15.19:0/441262117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd102556b0 msgr2=0x55bd1026f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7abfff640 1 --2- 172.21.15.19:0/441262117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd102556b0 0x55bd1026f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7abfff640 1 -- 172.21.15.19:0/441262117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10272f50 msgr2=0x55bd10224920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/441262117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10272f50 0x55bd10224920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7abfff640 1 --2- 172.21.15.19:0/441262117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10272f50 0x55bd10224920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7abfff640 1 -- 172.21.15.19:0/441262117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd0fe3b3a0 con 0x55bd10269370 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/441262117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10272f50 0x55bd10224920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.391+0000 7fc7abfff640 1 --2- 172.21.15.19:0/441262117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10269370 0x55bd10252050 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc794007470 tx=0x7fc79400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d03c1a86f43e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7a1ffb640 1 -- 172.21.15.19:0/441262117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7940283f0 con 0x55bd10269370 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7a1ffb640 1 -- 172.21.15.19:0/441262117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc794018070 con 0x55bd10269370 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7a1ffb640 1 -- 172.21.15.19:0/441262117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7940287f0 con 0x55bd10269370 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7b5a20740 1 -- 172.21.15.19:0/441262117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10269370 msgr2=0x55bd10252050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/441262117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10269370 0x55bd10252050 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc794007470 tx=0x7fc79400c000 comp rx=0 tx=0).stop 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7b5a20740 1 -- 172.21.15.19:0/441262117 shutdown_connections 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/441262117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10272f50 0x55bd10224920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/441262117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd10269370 0x55bd10252050 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/441262117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd102556b0 0x55bd1026f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7b5a20740 1 -- 172.21.15.19:0/441262117 >> 172.21.15.19:0/441262117 conn(0x55bd1022b390 msgr2=0x55bd10225050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7b5a20740 1 -- 172.21.15.19:0/441262117 shutdown_connections 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.392+0000 7fc7b5a20740 1 -- 172.21.15.19:0/441262117 wait complete. 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.393+0000 7fc7b5a20740 1 -- start start 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.393+0000 7fc7b5a20740 1 Processor -- start 2024-04-25T09:56:02.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd102dbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10269370 0x55bd10251e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b5a20740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd10272f50 0x55bd102da670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd10227980 con 0x55bd102556b0 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac1ff1d0 con 0x55bd10269370 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b5a20740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd10254d70 con 0x55bd10272f50 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b5a20740 10 client.0 _ll_register_callbacks cb 0x55bd102260d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b09e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd102dbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b09e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd102dbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46826/0 (socket says 172.21.15.19:46826) 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b09e8640 1 -- 172.21.15.19:0/967298966 learned_addr learned my addr 172.21.15.19:0/967298966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7abfff640 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10269370 0x55bd10251e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd10272f50 0x55bd102da670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b09e8640 1 -- 172.21.15.19:0/967298966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd10272f50 msgr2=0x55bd102da670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd10272f50 0x55bd102da670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b09e8640 1 -- 172.21.15.19:0/967298966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10269370 msgr2=0x55bd10251e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10269370 0x55bd10251e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b09e8640 1 -- 172.21.15.19:0/967298966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd102523b0 con 0x55bd102556b0 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b11e9640 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd10272f50 0x55bd102da670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b09e8640 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd102dbe70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4015c70 tx=0x7fc7a4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbe579bd5dbdee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc78b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc78b7fe640 1 -- 172.21.15.19:0/967298966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4026070 con 0x55bd102556b0 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc78b7fe640 1 -- 172.21.15.19:0/967298966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc7a403c650 con 0x55bd102556b0 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.394+0000 7fc7b5a20740 1 -- 172.21.15.19:0/967298966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc794018070 con 0x55bd102556b0 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc78b7fe640 1 -- 172.21.15.19:0/967298966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7a4044050 con 0x55bd102556b0 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc7b5a20740 10 client.5913 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc7b5a20740 1 -- 172.21.15.19:0/967298966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55bd102dabb0 con 0x55bd102556b0 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc78b7fe640 1 -- 172.21.15.19:0/967298966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc794018070 con 0x55bd102556b0 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc78b7fe640 1 -- 172.21.15.19:0/967298966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc7a4047020 con 0x55bd102556b0 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc7b5a20740 10 client.5913 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc7b5a20740 10 client.5913 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc7b5a20740 20 client.5913 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55bd1022a570 0x55bd1022a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc7b5a20740 4 client.5913 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc7b5a20740 1 -- 172.21.15.19:0/967298966 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55bd101954f0 con 0x55bd1022a570 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc7abfff640 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55bd1022a570 0x55bd1022a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc78b7fe640 10 client.5913 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.395+0000 7fc7abfff640 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55bd1022a570 0x55bd1022a980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc7648dba1adbf1 server_cookie=51c417f17af0dac7 in_seq=0 out_seq=0 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc78b7fe640 1 -- 172.21.15.19:0/967298966 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55bd101954f0 con 0x55bd1022a570 2024-04-25T09:56:02.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc78b7fe640 10 client.5913 handle_command_reply: tid=0 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 1 client.5913 shutdown 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 1 -- 172.21.15.19:0/967298966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 msgr2=0x55bd102dbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd102dbe70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4015c70 tx=0x7fc7a4009810 comp rx=0 tx=0).stop 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 1 -- 172.21.15.19:0/967298966 shutdown_connections 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55bd1022a570 0x55bd1022a980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55bd10272f50 0x55bd102da670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd10269370 0x55bd10251e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 1 --2- 172.21.15.19:0/967298966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd102556b0 0x55bd102dbe70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc78b7fe640 0 client.5913 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 1 -- 172.21.15.19:0/967298966 >> 172.21.15.19:0/967298966 conn(0x55bd1022b390 msgr2=0x55bd10271320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 1 -- 172.21.15.19:0/967298966 shutdown_connections 2024-04-25T09:56:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 1 -- 172.21.15.19:0/967298966 wait complete. 2024-04-25T09:56:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b5a20740 20 client.5913 trim_cache size 0 max 16384 2024-04-25T09:56:02.407 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": -2 2024-04-25T09:56:02.407 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc78c06b4d0 msgr2=0x7fc78c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b2c4c640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc78c06b4d0 0x7fc78c06d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009d50 tx=0x7fc7a400b040 comp rx=0 tx=0).stop 2024-04-25T09:56:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/2566298248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac161710 msgr2=0x7fc7ac0cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.396+0000 7fc7b2c4c640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac161710 0x7fc7ac0cbf60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc79c035040 tx=0x7fc79c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.397+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/2566298248 shutdown_connections 2024-04-25T09:56:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.397+0000 7fc7b2c4c640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc78c06b4d0 0x7fc78c06d990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.397+0000 7fc7b2c4c640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac161710 0x7fc7ac0cbf60 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.397+0000 7fc7b2c4c640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7ac15dc20 0x7fc7ac0ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.397+0000 7fc7b2c4c640 1 --2- 172.21.15.19:0/2566298248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7ac0d32f0 0x7fc7ac0acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.397+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/2566298248 >> 172.21.15.19:0/2566298248 conn(0x7fc7ac0d0420 msgr2=0x7fc7ac0d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.397+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/2566298248 shutdown_connections 2024-04-25T09:56:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.397+0000 7fc7b2c4c640 1 -- 172.21.15.19:0/2566298248 wait complete. 2024-04-25T09:56:02.411 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T09:56:02.411 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:56:02.411 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:56:02.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.507+0000 7fdef049e640 1 Processor -- start 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.507+0000 7fdef049e640 1 -- start start 2024-04-25T09:56:02.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.508+0000 7fdef049e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee80d1020 0x7fdee80d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.508+0000 7fdef049e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee80b97a0 con 0x7fdee80b2cb0 2024-04-25T09:56:02.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.508+0000 7fdef049e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee80d1c50 con 0x7fdee815dbe0 2024-04-25T09:56:02.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.508+0000 7fdef049e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee815dfc0 con 0x7fdee80d1020 2024-04-25T09:56:02.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.508+0000 7fdeeda39640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdee80b2cb0 0x7fdee80d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40544/0 (socket says 172.21.15.19:40544) 2024-04-25T09:56:02.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.508+0000 7fdeeda39640 1 -- 172.21.15.19:0/4021175763 learned_addr learned my addr 172.21.15.19:0/4021175763 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:02.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.509+0000 7fdeeea3b640 1 --2- 172.21.15.19:0/4021175763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee80d1020 0x7fdee80d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.509+0000 7fdeed238640 1 -- 172.21.15.19:0/4021175763 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908220073 0 0) 0x7fdee80b97a0 con 0x7fdee80b2cb0 2024-04-25T09:56:02.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.509+0000 7fdeed238640 1 -- 172.21.15.19:0/4021175763 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded0003520 con 0x7fdee80b2cb0 2024-04-25T09:56:02.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.509+0000 7fdeed238640 1 -- 172.21.15.19:0/4021175763 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471114931 0 0) 0x7fdee80d1c50 con 0x7fdee815dbe0 2024-04-25T09:56:02.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.509+0000 7fdeed238640 1 -- 172.21.15.19:0/4021175763 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee80b97a0 con 0x7fdee815dbe0 2024-04-25T09:56:02.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.509+0000 7fdeeea3b640 1 -- 172.21.15.19:0/4021175763 >> v1:172.21.15.112:6789/0 conn(0x7fdee815dbe0 legacy=0x7fdee80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.509+0000 7fdeeea3b640 1 -- 172.21.15.19:0/4021175763 >> v1:172.21.15.19:6789/0 conn(0x7fdee80b2cb0 legacy=0x7fdee80d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.509+0000 7fdeed238640 1 -- 172.21.15.19:0/4021175763 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 317780826 0 0) 0x7fded0003520 con 0x7fdee80b2cb0 2024-04-25T09:56:02.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.509+0000 7fdeeea3b640 1 -- 172.21.15.19:0/4021175763 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80d1f30 con 0x7fdee80d1020 2024-04-25T09:56:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.510+0000 7fdeeea3b640 1 --2- 172.21.15.19:0/4021175763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee80d1020 0x7fdee80d2e40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fded8009dc0 tx=0x7fded802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f758ac29dfcfbd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.510+0000 7fdeed238640 1 -- 172.21.15.19:0/4021175763 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc001670 con 0x7fdee80d1020 2024-04-25T09:56:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.510+0000 7fdeed238640 1 -- 172.21.15.19:0/4021175763 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded8007540 con 0x7fdee80d1020 2024-04-25T09:56:02.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.510+0000 7fdeed238640 1 -- 172.21.15.19:0/4021175763 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8007910 con 0x7fdee80d1020 2024-04-25T09:56:02.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.510+0000 7fdef049e640 1 -- 172.21.15.19:0/4021175763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee80d1020 msgr2=0x7fdee80d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.510+0000 7fdef049e640 1 --2- 172.21.15.19:0/4021175763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee80d1020 0x7fdee80d2e40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fded8009dc0 tx=0x7fded802a660 comp rx=0 tx=0).stop 2024-04-25T09:56:02.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.511+0000 7fdef049e640 1 -- 172.21.15.19:0/4021175763 shutdown_connections 2024-04-25T09:56:02.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.511+0000 7fdef049e640 1 --2- 172.21.15.19:0/4021175763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee80d1020 0x7fdee80d2e40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.511+0000 7fdef049e640 1 -- 172.21.15.19:0/4021175763 >> 172.21.15.19:0/4021175763 conn(0x7fdee8061a20 msgr2=0x7fdee8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.511+0000 7fdef049e640 1 -- 172.21.15.19:0/4021175763 shutdown_connections 2024-04-25T09:56:02.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.511+0000 7fdef049e640 1 -- 172.21.15.19:0/4021175763 wait complete. 2024-04-25T09:56:02.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.512+0000 7fdef049e640 1 Processor -- start 2024-04-25T09:56:02.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.512+0000 7fdef049e640 1 -- start start 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.512+0000 7fdef049e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdee80b2cb0 0x7fdee80b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.512+0000 7fdef049e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee80d1020 0x7fdee8154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.512+0000 7fdef049e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee815dbe0 0x7fdee8160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdef049e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80b9420 con 0x7fdee80b2cb0 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdef049e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80b92a0 con 0x7fdee80d1020 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdef049e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80b6ab0 con 0x7fdee815dbe0 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeee23a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdee80b2cb0 0x7fdee80b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeee23a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdee80b2cb0 0x7fdee80b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46834/0 (socket says 172.21.15.19:46834) 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeee23a640 1 -- 172.21.15.19:0/3587143242 learned_addr learned my addr 172.21.15.19:0/3587143242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeeea3b640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee815dbe0 0x7fdee8160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeeda39640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee80d1020 0x7fdee8154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeee23a640 1 -- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee815dbe0 msgr2=0x7fdee8160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeee23a640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee815dbe0 0x7fdee8160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeee23a640 1 -- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee80d1020 msgr2=0x7fdee8154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeee23a640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee80d1020 0x7fdee8154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeee23a640 1 -- 172.21.15.19:0/3587143242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80b41f0 con 0x7fdee80b2cb0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeeea3b640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee815dbe0 0x7fdee8160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdeee23a640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdee80b2cb0 0x7fdee80b3aa0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdee000c430 tx=0x7fdee000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdee6ffd640 1 -- 172.21.15.19:0/3587143242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0017070 con 0x7fdee80b2cb0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdee6ffd640 1 -- 172.21.15.19:0/3587143242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee00052f0 con 0x7fdee80b2cb0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdef049e640 1 -- 172.21.15.19:0/3587143242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee80b44d0 con 0x7fdee80b2cb0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdee6ffd640 1 -- 172.21.15.19:0/3587143242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0003830 con 0x7fdee80b2cb0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.513+0000 7fdef049e640 1 -- 172.21.15.19:0/3587143242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee80b4a90 con 0x7fdee80b2cb0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.514+0000 7fdef049e640 1 -- 172.21.15.19:0/3587143242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee8073990 con 0x7fdee80b2cb0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.515+0000 7fdee6ffd640 1 -- 172.21.15.19:0/3587143242 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdee001a070 con 0x7fdee80b2cb0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.515+0000 7fdee6ffd640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded406b4d0 0x7fded406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.515+0000 7fdee6ffd640 1 -- 172.21.15.19:0/3587143242 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdee0089740 con 0x7fdee80b2cb0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.515+0000 7fdeeda39640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded406b4d0 0x7fded406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.516+0000 7fdeeda39640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded406b4d0 0x7fded406d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdedc001340 tx=0x7fdedc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.519+0000 7fdee6ffd640 1 -- 172.21.15.19:0/3587143242 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdee00584c0 con 0x7fdee80b2cb0 2024-04-25T09:56:02.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.737+0000 7fdef049e640 1 -- 172.21.15.19:0/3587143242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdee80b26e0 con 0x7fdee80b2cb0 2024-04-25T09:56:02.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.738+0000 7fdee6ffd640 1 -- 172.21.15.19:0/3587143242 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fdee0057dc0 con 0x7fdee80b2cb0 2024-04-25T09:56:02.742 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:02.742 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:55:54.352541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:02.742 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-04-25T09:56:02.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 -- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded406b4d0 msgr2=0x7fded406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded406b4d0 0x7fded406d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdedc001340 tx=0x7fdedc009040 comp rx=0 tx=0).stop 2024-04-25T09:56:02.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 -- 172.21.15.19:0/3587143242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdee80b2cb0 msgr2=0x7fdee80b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdee80b2cb0 0x7fdee80b3aa0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdee000c430 tx=0x7fdee000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 -- 172.21.15.19:0/3587143242 shutdown_connections 2024-04-25T09:56:02.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fded406b4d0 0x7fded406d990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdee815dbe0 0x7fdee8160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee80d1020 0x7fdee8154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 --2- 172.21.15.19:0/3587143242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdee80b2cb0 0x7fdee80b3aa0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 -- 172.21.15.19:0/3587143242 >> 172.21.15.19:0/3587143242 conn(0x7fdee8061a20 msgr2=0x7fdee8061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 -- 172.21.15.19:0/3587143242 shutdown_connections 2024-04-25T09:56:02.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:02.740+0000 7fdef049e640 1 -- 172.21.15.19:0/3587143242 wait complete. 2024-04-25T09:56:02.755 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:56:02.939 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:02.939 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T09:56:02.939 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:02.948 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:56:02.948 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:56:02.948 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:02.948 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:56:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.049+0000 7ff86373b640 1 Processor -- start 2024-04-25T09:56:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.049+0000 7ff86373b640 1 -- start start 2024-04-25T09:56:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.050+0000 7ff86373b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff85c15bb20 0x7ff85c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.050+0000 7ff86373b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85c0cb190 con 0x7ff85c0d00e0 2024-04-25T09:56:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.050+0000 7ff86373b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85c0cb470 con 0x7ff85c0ccdd0 2024-04-25T09:56:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.050+0000 7ff86373b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0cb750 con 0x7ff85c15bb20 2024-04-25T09:56:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.050+0000 7ff861cd8640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff85c0d00e0 0x7ff85c0ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40554/0 (socket says 172.21.15.19:40554) 2024-04-25T09:56:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.050+0000 7ff861cd8640 1 -- 172.21.15.19:0/1166591227 learned_addr learned my addr 172.21.15.19:0/1166591227 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.051+0000 7ff8614d7640 1 --2- 172.21.15.19:0/1166591227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff85c15bb20 0x7ff85c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.051+0000 7ff85bfff640 1 -- 172.21.15.19:0/1166591227 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809715460 0 0) 0x7ff85c0cb190 con 0x7ff85c0d00e0 2024-04-25T09:56:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.051+0000 7ff85bfff640 1 -- 172.21.15.19:0/1166591227 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff844003520 con 0x7ff85c0d00e0 2024-04-25T09:56:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.051+0000 7ff85bfff640 1 -- 172.21.15.19:0/1166591227 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060556106 0 0) 0x7ff85c0cb470 con 0x7ff85c0ccdd0 2024-04-25T09:56:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.051+0000 7ff85bfff640 1 -- 172.21.15.19:0/1166591227 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff85c0cb190 con 0x7ff85c0ccdd0 2024-04-25T09:56:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.051+0000 7ff8614d7640 1 -- 172.21.15.19:0/1166591227 >> v1:172.21.15.112:6789/0 conn(0x7ff85c0ccdd0 legacy=0x7ff85c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.051+0000 7ff8614d7640 1 -- 172.21.15.19:0/1166591227 >> v1:172.21.15.19:6789/0 conn(0x7ff85c0d00e0 legacy=0x7ff85c0ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.051+0000 7ff8614d7640 1 -- 172.21.15.19:0/1166591227 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c0cba30 con 0x7ff85c15bb20 2024-04-25T09:56:03.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.052+0000 7ff8614d7640 1 --2- 172.21.15.19:0/1166591227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff85c15bb20 0x7ff85c0cc230 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8540097d0 tx=0x7ff85402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56156301d3da037a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.052+0000 7ff85bfff640 1 -- 172.21.15.19:0/1166591227 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854037070 con 0x7ff85c15bb20 2024-04-25T09:56:03.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.052+0000 7ff85bfff640 1 -- 172.21.15.19:0/1166591227 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff854032440 con 0x7ff85c15bb20 2024-04-25T09:56:03.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.052+0000 7ff85bfff640 1 -- 172.21.15.19:0/1166591227 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854032810 con 0x7ff85c15bb20 2024-04-25T09:56:03.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.052+0000 7ff86373b640 1 -- 172.21.15.19:0/1166591227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff85c15bb20 msgr2=0x7ff85c0cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.052+0000 7ff86373b640 1 --2- 172.21.15.19:0/1166591227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff85c15bb20 0x7ff85c0cc230 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8540097d0 tx=0x7ff85402a180 comp rx=0 tx=0).stop 2024-04-25T09:56:03.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.052+0000 7ff86373b640 1 -- 172.21.15.19:0/1166591227 shutdown_connections 2024-04-25T09:56:03.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.052+0000 7ff86373b640 1 --2- 172.21.15.19:0/1166591227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff85c15bb20 0x7ff85c0cc230 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.052+0000 7ff86373b640 1 -- 172.21.15.19:0/1166591227 >> 172.21.15.19:0/1166591227 conn(0x7ff85c0b2390 msgr2=0x7ff85c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.053+0000 7ff86373b640 1 -- 172.21.15.19:0/1166591227 shutdown_connections 2024-04-25T09:56:03.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.053+0000 7ff86373b640 1 -- 172.21.15.19:0/1166591227 wait complete. 2024-04-25T09:56:03.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.054+0000 7ff86373b640 1 Processor -- start 2024-04-25T09:56:03.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.054+0000 7ff86373b640 1 -- start start 2024-04-25T09:56:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.054+0000 7ff86373b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff85c0ccdd0 0x7ff85c158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.054+0000 7ff86373b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff85c0d00e0 0x7ff85c151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.054+0000 7ff86373b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff85c15bb20 0x7ff85c0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.054+0000 7ff86373b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c15bf00 con 0x7ff85c0d00e0 2024-04-25T09:56:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.054+0000 7ff86373b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0cc770 con 0x7ff85c15bb20 2024-04-25T09:56:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.054+0000 7ff86373b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c15c080 con 0x7ff85c0ccdd0 2024-04-25T09:56:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.054+0000 7ff860cd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff85c0d00e0 0x7ff85c151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.054+0000 7ff860cd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff85c0d00e0 0x7ff85c151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46844/0 (socket says 172.21.15.19:46844) 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff860cd6640 1 -- 172.21.15.19:0/3035687647 learned_addr learned my addr 172.21.15.19:0/3035687647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff861cd8640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff85c15bb20 0x7ff85c0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff8614d7640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff85c0ccdd0 0x7ff85c158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff860cd6640 1 -- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff85c0ccdd0 msgr2=0x7ff85c158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff860cd6640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff85c0ccdd0 0x7ff85c158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff860cd6640 1 -- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff85c15bb20 msgr2=0x7ff85c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff860cd6640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff85c15bb20 0x7ff85c0b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff860cd6640 1 -- 172.21.15.19:0/3035687647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c1593b0 con 0x7ff85c0d00e0 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff861cd8640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff85c15bb20 0x7ff85c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff860cd6640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff85c0d00e0 0x7ff85c151650 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff85000c500 tx=0x7ff85000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff859ffb640 1 -- 172.21.15.19:0/3035687647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8500052d0 con 0x7ff85c0d00e0 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff859ffb640 1 -- 172.21.15.19:0/3035687647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff850005490 con 0x7ff85c0d00e0 2024-04-25T09:56:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff86373b640 1 -- 172.21.15.19:0/3035687647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85c159690 con 0x7ff85c0d00e0 2024-04-25T09:56:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff859ffb640 1 -- 172.21.15.19:0/3035687647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850012870 con 0x7ff85c0d00e0 2024-04-25T09:56:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.055+0000 7ff86373b640 1 -- 172.21.15.19:0/3035687647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85c159c50 con 0x7ff85c0d00e0 2024-04-25T09:56:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.057+0000 7ff859ffb640 1 -- 172.21.15.19:0/3035687647 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff850012a10 con 0x7ff85c0d00e0 2024-04-25T09:56:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.057+0000 7ff859ffb640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff84806b4d0 0x7ff84806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.057+0000 7ff859ffb640 1 -- 172.21.15.19:0/3035687647 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff85008a070 con 0x7ff85c0d00e0 2024-04-25T09:56:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.057+0000 7ff8614d7640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff84806b4d0 0x7ff84806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.058+0000 7ff8614d7640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff84806b4d0 0x7ff84806d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff854034000 tx=0x7ff85402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.058+0000 7ff866442740 1 Processor -- start 2024-04-25T09:56:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.058+0000 7ff866442740 1 -- start start 2024-04-25T09:56:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff866442740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bca4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff866442740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bca22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff866442740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bccce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff866442740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562bbd3fe0 con 0x55562bcde5e0 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff866442740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562bc05d20 con 0x55562bccc9d0 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff866442740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55562bbaefa0 con 0x55562bca1f10 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff860cd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bca22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff860cd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bca22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46846/0 (socket says 172.21.15.19:46846) 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff860cd6640 1 -- 172.21.15.19:0/1578336360 learned_addr learned my addr 172.21.15.19:0/1578336360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff861cd8640 1 --2- 172.21.15.19:0/1578336360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bccce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff8614d7640 1 --2- 172.21.15.19:0/1578336360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bca4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff860cd6640 1 -- 172.21.15.19:0/1578336360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 msgr2=0x55562bca4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff860cd6640 1 --2- 172.21.15.19:0/1578336360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bca4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff860cd6640 1 -- 172.21.15.19:0/1578336360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 msgr2=0x55562bccce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff860cd6640 1 --2- 172.21.15.19:0/1578336360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bccce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.059+0000 7ff860cd6640 1 -- 172.21.15.19:0/1578336360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55562b9e6b00 con 0x55562bcde5e0 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff860cd6640 1 --2- 172.21.15.19:0/1578336360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bca22f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff850088410 tx=0x7ff850058e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03a7bd61e49b3f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff842ffd640 1 -- 172.21.15.19:0/1578336360 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850060d80 con 0x55562bcde5e0 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff842ffd640 1 -- 172.21.15.19:0/1578336360 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff850056710 con 0x55562bcde5e0 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff842ffd640 1 -- 172.21.15.19:0/1578336360 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850012d00 con 0x55562bcde5e0 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff866442740 1 -- 172.21.15.19:0/1578336360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 msgr2=0x55562bca22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff866442740 1 --2- 172.21.15.19:0/1578336360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bca22f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff850088410 tx=0x7ff850058e30 comp rx=0 tx=0).stop 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff866442740 1 -- 172.21.15.19:0/1578336360 shutdown_connections 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff866442740 1 --2- 172.21.15.19:0/1578336360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bccce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff866442740 1 --2- 172.21.15.19:0/1578336360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bca22f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff866442740 1 --2- 172.21.15.19:0/1578336360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bca4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff866442740 1 -- 172.21.15.19:0/1578336360 >> 172.21.15.19:0/1578336360 conn(0x55562bc6a100 msgr2=0x55562bce79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff866442740 1 -- 172.21.15.19:0/1578336360 shutdown_connections 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.060+0000 7ff866442740 1 -- 172.21.15.19:0/1578336360 wait complete. 2024-04-25T09:56:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.061+0000 7ff866442740 1 -- start start 2024-04-25T09:56:03.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.061+0000 7ff866442740 1 Processor -- start 2024-04-25T09:56:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff866442740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bcc86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff866442740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bce1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff866442740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bccf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff866442740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562bba4750 con 0x55562bcde5e0 2024-04-25T09:56:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff866442740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562bc01dc0 con 0x55562bccc9d0 2024-04-25T09:56:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff866442740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55562bcf7e60 con 0x55562bca1f10 2024-04-25T09:56:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff866442740 10 client.0 _ll_register_callbacks cb 0x55562bbbcee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff861cd8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bccf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff861cd8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bccf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46858/0 (socket says 172.21.15.19:46858) 2024-04-25T09:56:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff861cd8640 1 -- 172.21.15.19:0/1112992269 learned_addr learned my addr 172.21.15.19:0/1112992269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff8614d7640 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bcc86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff860cd6640 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bce1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff861cd8640 1 -- 172.21.15.19:0/1112992269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 msgr2=0x55562bcc86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff861cd8640 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bcc86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff861cd8640 1 -- 172.21.15.19:0/1112992269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 msgr2=0x55562bce1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff861cd8640 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bce1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff861cd8640 1 -- 172.21.15.19:0/1112992269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55562b8da3f0 con 0x55562bcde5e0 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff861cd8640 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bccf250 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00f890 tx=0x7ff84c00fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae368804438717a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.062+0000 7ff83bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff83bfff640 1 -- 172.21.15.19:0/1112992269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c017020 con 0x55562bcde5e0 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff83bfff640 1 -- 172.21.15.19:0/1112992269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff84c005de0 con 0x55562bcde5e0 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff866442740 1 -- 172.21.15.19:0/1112992269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff850056710 con 0x55562bcde5e0 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff83bfff640 1 -- 172.21.15.19:0/1112992269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff84c021050 con 0x55562bcde5e0 2024-04-25T09:56:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff866442740 10 client.5931 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff866442740 1 -- 172.21.15.19:0/1112992269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55562ba47290 con 0x55562bcde5e0 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff83bfff640 1 -- 172.21.15.19:0/1112992269 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff850056710 con 0x55562bcde5e0 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff83bfff640 1 -- 172.21.15.19:0/1112992269 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff84c024020 con 0x55562bcde5e0 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff866442740 10 client.5931 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff866442740 10 client.5931 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff866442740 20 client.5931 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff866442740 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55562bce1390 0x55562bd04990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff866442740 4 client.5931 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff866442740 1 -- 172.21.15.19:0/1112992269 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55562bc13260 con 0x55562bce1390 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.063+0000 7ff8614d7640 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55562bce1390 0x55562bd04990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff83bfff640 10 client.5931 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff8614d7640 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55562bce1390 0x55562bd04990 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb7c979cd39fc04 server_cookie=fc28ce1b45477b3d in_seq=0 out_seq=0 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff83bfff640 1 -- 172.21.15.19:0/1112992269 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55562bc13260 con 0x55562bce1390 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff83bfff640 10 client.5931 handle_command_reply: tid=0 2024-04-25T09:56:03.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff866442740 1 client.5931 shutdown 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff866442740 1 -- 172.21.15.19:0/1112992269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 msgr2=0x55562bccf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff866442740 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bccf250 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00f890 tx=0x7ff84c00fd50 comp rx=0 tx=0).stop 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff866442740 1 -- 172.21.15.19:0/1112992269 shutdown_connections 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff866442740 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55562bce1390 0x55562bd04990 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff866442740 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bccf250 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff83bfff640 0 client.5931 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff866442740 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bce1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff866442740 1 --2- 172.21.15.19:0/1112992269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bcc86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff866442740 1 -- 172.21.15.19:0/1112992269 >> 172.21.15.19:0/1112992269 conn(0x55562bc6a100 msgr2=0x55562bce8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff866442740 1 -- 172.21.15.19:0/1112992269 shutdown_connections 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.064+0000 7ff866442740 1 -- 172.21.15.19:0/1112992269 wait complete. 2024-04-25T09:56:03.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.065+0000 7ff866442740 20 client.5931 trim_cache size 0 max 16384 2024-04-25T09:56:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.080+0000 7ff866442740 1 Processor -- start 2024-04-25T09:56:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.080+0000 7ff866442740 1 -- start start 2024-04-25T09:56:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.080+0000 7ff866442740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bccd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.080+0000 7ff866442740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bccdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.080+0000 7ff866442740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bcce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.080+0000 7ff866442740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c2004f0 con 0x55562bcde5e0 2024-04-25T09:56:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.080+0000 7ff866442740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c1ff6d0 con 0x55562bccc9d0 2024-04-25T09:56:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.080+0000 7ff866442740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c1fead0 con 0x55562bca1f10 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.080+0000 7ff861cd8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bcce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff861cd8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bcce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46872/0 (socket says 172.21.15.19:46872) 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff861cd8640 1 -- 172.21.15.19:0/3713190012 learned_addr learned my addr 172.21.15.19:0/3713190012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff8614d7640 1 --2- 172.21.15.19:0/3713190012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bccd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff860cd6640 1 --2- 172.21.15.19:0/3713190012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bccdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff861cd8640 1 -- 172.21.15.19:0/3713190012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 msgr2=0x55562bccd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff861cd8640 1 --2- 172.21.15.19:0/3713190012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bccd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff861cd8640 1 -- 172.21.15.19:0/3713190012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 msgr2=0x55562bccdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff861cd8640 1 --2- 172.21.15.19:0/3713190012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bccdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff861cd8640 1 -- 172.21.15.19:0/3713190012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55562bca2660 con 0x55562bcde5e0 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff860cd6640 1 --2- 172.21.15.19:0/3713190012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bccdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff861cd8640 1 --2- 172.21.15.19:0/3713190012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bcce090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff84c001830 tx=0x7ff84c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312cf4bb383b24cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff8427fc640 1 -- 172.21.15.19:0/3713190012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c02c020 con 0x55562bcde5e0 2024-04-25T09:56:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff8427fc640 1 -- 172.21.15.19:0/3713190012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c00d7a0 con 0x55562bcde5e0 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.081+0000 7ff8427fc640 1 -- 172.21.15.19:0/3713190012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c017030 con 0x55562bcde5e0 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.082+0000 7ff866442740 1 -- 172.21.15.19:0/3713190012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 msgr2=0x55562bcce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.082+0000 7ff866442740 1 --2- 172.21.15.19:0/3713190012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bcce090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff84c001830 tx=0x7ff84c013000 comp rx=0 tx=0).stop 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.082+0000 7ff866442740 1 -- 172.21.15.19:0/3713190012 shutdown_connections 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.082+0000 7ff866442740 1 --2- 172.21.15.19:0/3713190012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bcde5e0 0x55562bcce090 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.082+0000 7ff866442740 1 --2- 172.21.15.19:0/3713190012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bccdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.082+0000 7ff866442740 1 --2- 172.21.15.19:0/3713190012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bca1f10 0x55562bccd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.082+0000 7ff866442740 1 -- 172.21.15.19:0/3713190012 >> 172.21.15.19:0/3713190012 conn(0x55562bc6a100 msgr2=0x55562bca2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.082+0000 7ff866442740 1 -- 172.21.15.19:0/3713190012 shutdown_connections 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.082+0000 7ff866442740 1 -- 172.21.15.19:0/3713190012 wait complete. 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.082+0000 7ff866442740 1 -- start start 2024-04-25T09:56:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.082+0000 7ff866442740 1 Processor -- start 2024-04-25T09:56:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.083+0000 7ff866442740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bca1f10 0x55562bccd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.083+0000 7ff866442740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bd4ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.083+0000 7ff866442740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bcde5e0 0x55562bccdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.083+0000 7ff866442740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562bcf7710 con 0x55562bca1f10 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.083+0000 7ff866442740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c1ff0d0 con 0x55562bccc9d0 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.083+0000 7ff866442740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55562bccc800 con 0x55562bcde5e0 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.083+0000 7ff866442740 10 client.0 _ll_register_callbacks cb 0x7ff850058440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.083+0000 7ff8614d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bca1f10 0x55562bccd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.083+0000 7ff8614d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bca1f10 0x55562bccd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46882/0 (socket says 172.21.15.19:46882) 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.083+0000 7ff8614d7640 1 -- 172.21.15.19:0/2312798562 learned_addr learned my addr 172.21.15.19:0/2312798562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff860cd6640 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bd4ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff861cd8640 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bcde5e0 0x55562bccdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff8614d7640 1 -- 172.21.15.19:0/2312798562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bcde5e0 msgr2=0x55562bccdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff8614d7640 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bcde5e0 0x55562bccdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff8614d7640 1 -- 172.21.15.19:0/2312798562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 msgr2=0x55562bd4ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff8614d7640 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bd4ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff8614d7640 1 -- 172.21.15.19:0/2312798562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55562b8b34b0 con 0x55562bca1f10 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff860cd6640 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bd4ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff861cd8640 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bcde5e0 0x55562bccdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff8614d7640 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bca1f10 0x55562bccd490 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff854000c00 tx=0x7ff854004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc01875620b63723 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff83bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff83bfff640 1 -- 172.21.15.19:0/2312798562 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854037040 con 0x55562bca1f10 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff83bfff640 1 -- 172.21.15.19:0/2312798562 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff85404d2e0 con 0x55562bca1f10 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff866442740 1 -- 172.21.15.19:0/2312798562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff84c00d7a0 con 0x55562bca1f10 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff83bfff640 1 -- 172.21.15.19:0/2312798562 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff854032e30 con 0x55562bca1f10 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff866442740 10 client.5937 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff866442740 1 -- 172.21.15.19:0/2312798562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55562bccfd30 con 0x55562bca1f10 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff83bfff640 1 -- 172.21.15.19:0/2312798562 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c00d7a0 con 0x55562bca1f10 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff83bfff640 1 -- 172.21.15.19:0/2312798562 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff85404d4e0 con 0x55562bca1f10 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff866442740 10 client.5937 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:03.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff866442740 10 client.5937 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff866442740 20 client.5937 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff866442740 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55562bd55790 0x55562bd51790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff866442740 4 client.5937 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.084+0000 7ff866442740 1 -- 172.21.15.19:0/2312798562 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55562bc0d5b0 con 0x55562bd55790 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.085+0000 7ff860cd6640 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55562bd55790 0x55562bd51790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.085+0000 7ff860cd6640 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55562bd55790 0x55562bd51790 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e332d2e2b0bb2c server_cookie=a134576761ddab51 in_seq=0 out_seq=0 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.085+0000 7ff83bfff640 10 client.5937 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.085+0000 7ff83bfff640 1 -- 172.21.15.19:0/2312798562 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55562bc0d5b0 con 0x55562bd55790 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.085+0000 7ff83bfff640 10 client.5937 handle_command_reply: tid=0 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.085+0000 7ff866442740 1 client.5937 shutdown 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff866442740 1 -- 172.21.15.19:0/2312798562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bca1f10 msgr2=0x55562bccd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff866442740 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bca1f10 0x55562bccd490 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff854000c00 tx=0x7ff854004730 comp rx=0 tx=0).stop 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff866442740 1 -- 172.21.15.19:0/2312798562 shutdown_connections 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff866442740 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55562bd55790 0x55562bd51790 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff866442740 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55562bcde5e0 0x55562bccdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff866442740 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55562bccc9d0 0x55562bd4ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff866442740 1 --2- 172.21.15.19:0/2312798562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55562bca1f10 0x55562bccd490 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff866442740 1 -- 172.21.15.19:0/2312798562 >> 172.21.15.19:0/2312798562 conn(0x55562bc6a100 msgr2=0x55562bcc84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff83bfff640 0 client.5937 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff866442740 1 -- 172.21.15.19:0/2312798562 shutdown_connections 2024-04-25T09:56:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff866442740 1 -- 172.21.15.19:0/2312798562 wait complete. 2024-04-25T09:56:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff866442740 20 client.5937 trim_cache size 0 max 16384 2024-04-25T09:56:03.096 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:03.096 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:56:03.096 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "4241dd9e-9e1c-4105-98e7-a68e4f24045c", 2024-04-25T09:56:03.096 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:56:03.097 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff86373b640 1 -- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff84806b4d0 msgr2=0x7ff84806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff86373b640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff84806b4d0 0x7ff84806d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff854034000 tx=0x7ff85402e000 comp rx=0 tx=0).stop 2024-04-25T09:56:03.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff86373b640 1 -- 172.21.15.19:0/3035687647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff85c0d00e0 msgr2=0x7ff85c151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff86373b640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff85c0d00e0 0x7ff85c151650 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff85000c500 tx=0x7ff85000c9c0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff86373b640 1 -- 172.21.15.19:0/3035687647 shutdown_connections 2024-04-25T09:56:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff86373b640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff84806b4d0 0x7ff84806d990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff86373b640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff85c15bb20 0x7ff85c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff86373b640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff85c0d00e0 0x7ff85c151650 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.086+0000 7ff86373b640 1 --2- 172.21.15.19:0/3035687647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff85c0ccdd0 0x7ff85c158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.087+0000 7ff86373b640 1 -- 172.21.15.19:0/3035687647 >> 172.21.15.19:0/3035687647 conn(0x7ff85c0b2390 msgr2=0x7ff85c06fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.087+0000 7ff86373b640 1 -- 172.21.15.19:0/3035687647 shutdown_connections 2024-04-25T09:56:03.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.087+0000 7ff86373b640 1 -- 172.21.15.19:0/3035687647 wait complete. 2024-04-25T09:56:03.101 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '4241dd9e-9e1c-4105-98e7-a68e4f24045c', 'mode': 'asynchronous'}' 2024-04-25T09:56:03.101 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:56:03.101 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:56:03.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:03.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.197+0000 7f13d31a5640 1 Processor -- start 2024-04-25T09:56:03.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.198+0000 7f13d31a5640 1 -- start start 2024-04-25T09:56:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.198+0000 7f13d31a5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f13cc15dbe0 0x7f13cc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.198+0000 7f13d31a5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13cc0b2cb0 0x7f13cc0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.198+0000 7f13d31a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13cc0b92a0 0x7f13cc0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.198+0000 7f13d31a5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13cc15dfc0 con 0x7f13cc15dbe0 2024-04-25T09:56:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.198+0000 7f13d31a5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13cc0d1a00 con 0x7f13cc0b2cb0 2024-04-25T09:56:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.198+0000 7f13d31a5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc0b97a0 con 0x7f13cc0b92a0 2024-04-25T09:56:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.198+0000 7f13d0f41640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f13cc15dbe0 0x7f13cc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.199+0000 7f13d0f41640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f13cc15dbe0 0x7f13cc0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46886/0 (socket says 172.21.15.19:46886) 2024-04-25T09:56:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.199+0000 7f13d0f41640 1 -- 172.21.15.19:0/2067272966 learned_addr learned my addr 172.21.15.19:0/2067272966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.199+0000 7f13cbfff640 1 --2- 172.21.15.19:0/2067272966 >> v2:172.21.15.112:3300/0 conn(0x7f13cc0b2cb0 0x7f13cc0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.199+0000 7f13d1742640 1 --2- 172.21.15.19:0/2067272966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13cc0b92a0 0x7f13cc0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.199+0000 7f13d0f41640 1 -- 172.21.15.19:0/2067272966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13cc0b92a0 msgr2=0x7f13cc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.199+0000 7f13d0f41640 1 --2- 172.21.15.19:0/2067272966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13cc0b92a0 0x7f13cc0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.199+0000 7f13d0f41640 1 -- 172.21.15.19:0/2067272966 >> v2:172.21.15.112:3300/0 conn(0x7f13cc0b2cb0 msgr2=0x7f13cc0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.199+0000 7f13d0f41640 1 --2- 172.21.15.19:0/2067272966 >> v2:172.21.15.112:3300/0 conn(0x7f13cc0b2cb0 0x7f13cc0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.199+0000 7f13d0f41640 1 -- 172.21.15.19:0/2067272966 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc0d1c50 con 0x7f13cc15dbe0 2024-04-25T09:56:03.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.199+0000 7f13d1742640 1 --2- 172.21.15.19:0/2067272966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13cc0b92a0 0x7f13cc0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:03.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13d0f41640 1 --2- 172.21.15.19:0/2067272966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13cc15dbe0 0x7f13cc0d0910 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f13cc06cff0 tx=0x7f13c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a502eab630fff30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13cb7fe640 1 -- 172.21.15.19:0/2067272966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0037070 con 0x7f13cc15dbe0 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13cb7fe640 1 -- 172.21.15.19:0/2067272966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c0032440 con 0x7f13cc15dbe0 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13cb7fe640 1 -- 172.21.15.19:0/2067272966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0032810 con 0x7f13cc15dbe0 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13d31a5640 1 -- 172.21.15.19:0/2067272966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13cc15dbe0 msgr2=0x7f13cc0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13d31a5640 1 --2- 172.21.15.19:0/2067272966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13cc15dbe0 0x7f13cc0d0910 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f13cc06cff0 tx=0x7f13c002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13d31a5640 1 -- 172.21.15.19:0/2067272966 shutdown_connections 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13d31a5640 1 --2- 172.21.15.19:0/2067272966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13cc0b92a0 0x7f13cc0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13d31a5640 1 --2- 172.21.15.19:0/2067272966 >> v2:172.21.15.112:3300/0 conn(0x7f13cc0b2cb0 0x7f13cc0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13d31a5640 1 --2- 172.21.15.19:0/2067272966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13cc15dbe0 0x7f13cc0d0910 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13d31a5640 1 -- 172.21.15.19:0/2067272966 >> 172.21.15.19:0/2067272966 conn(0x7f13cc061a20 msgr2=0x7f13cc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13d31a5640 1 -- 172.21.15.19:0/2067272966 shutdown_connections 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.200+0000 7f13d31a5640 1 -- 172.21.15.19:0/2067272966 wait complete. 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.201+0000 7f13d31a5640 1 Processor -- start 2024-04-25T09:56:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.202+0000 7f13d31a5640 1 -- start start 2024-04-25T09:56:03.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.202+0000 7f13d31a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13cc0b2cb0 0x7f13cc0b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.202+0000 7f13d31a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13cc0b92a0 0x7f13cc0b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.202+0000 7f13d31a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13cc15dbe0 0x7f13cc0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.202+0000 7f13d31a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc0b6ab0 con 0x7f13cc0b92a0 2024-04-25T09:56:03.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.202+0000 7f13d31a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc0d2210 con 0x7f13cc0b2cb0 2024-04-25T09:56:03.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.202+0000 7f13d31a5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc0b6c30 con 0x7f13cc15dbe0 2024-04-25T09:56:03.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.202+0000 7f13cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13cc0b92a0 0x7f13cc0b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13cc0b92a0 0x7f13cc0b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46894/0 (socket says 172.21.15.19:46894) 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13cbfff640 1 -- 172.21.15.19:0/1018722262 learned_addr learned my addr 172.21.15.19:0/1018722262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13d0f41640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13cc0b2cb0 0x7f13cc0b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13d1742640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13cc15dbe0 0x7f13cc0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13cbfff640 1 -- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13cc15dbe0 msgr2=0x7f13cc0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13cbfff640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13cc15dbe0 0x7f13cc0caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13cbfff640 1 -- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13cc0b2cb0 msgr2=0x7f13cc0b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13cbfff640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13cc0b2cb0 0x7f13cc0b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13cbfff640 1 -- 172.21.15.19:0/1018722262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc0b3820 con 0x7f13cc0b92a0 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13d0f41640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13cc0b2cb0 0x7f13cc0b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13cbfff640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13cc0b92a0 0x7f13cc0b32e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f13b800d800 tx=0x7f13b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13c97fa640 1 -- 172.21.15.19:0/1018722262 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b800b6c0 con 0x7f13cc0b92a0 2024-04-25T09:56:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13c97fa640 1 -- 172.21.15.19:0/1018722262 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b8019070 con 0x7f13cc0b92a0 2024-04-25T09:56:03.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13d31a5640 1 -- 172.21.15.19:0/1018722262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13cc0b61c0 con 0x7f13cc0b92a0 2024-04-25T09:56:03.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13c97fa640 1 -- 172.21.15.19:0/1018722262 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b800ba80 con 0x7f13cc0b92a0 2024-04-25T09:56:03.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.203+0000 7f13d31a5640 1 -- 172.21.15.19:0/1018722262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc148040 con 0x7f13cc0b92a0 2024-04-25T09:56:03.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.204+0000 7f13d31a5640 1 -- 172.21.15.19:0/1018722262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13cc147b90 con 0x7f13cc0b92a0 2024-04-25T09:56:03.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.205+0000 7f13c97fa640 1 -- 172.21.15.19:0/1018722262 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13b800bc20 con 0x7f13cc0b92a0 2024-04-25T09:56:03.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.205+0000 7f13c97fa640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f13b006b4d0 0x7f13b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.205+0000 7f13c97fa640 1 -- 172.21.15.19:0/1018722262 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13b808a7f0 con 0x7f13cc0b92a0 2024-04-25T09:56:03.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.206+0000 7f13d0f41640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f13b006b4d0 0x7f13b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.206+0000 7f13d0f41640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f13b006b4d0 0x7f13b006d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f13c0034000 tx=0x7f13c002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.210+0000 7f13c97fa640 1 -- 172.21.15.19:0/1018722262 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13b801c360 con 0x7f13cc0b92a0 2024-04-25T09:56:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.427+0000 7f13d31a5640 1 -- 172.21.15.19:0/1018722262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13cc073990 con 0x7f13cc0b92a0 2024-04-25T09:56:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.428+0000 7f13c97fa640 1 -- 172.21.15.19:0/1018722262 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f13b80594e0 con 0x7f13cc0b92a0 2024-04-25T09:56:03.432 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:03.432 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:55:54.352541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-04-25T09:56:03.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 -- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f13b006b4d0 msgr2=0x7f13b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f13b006b4d0 0x7f13b006d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f13c0034000 tx=0x7f13c002e000 comp rx=0 tx=0).stop 2024-04-25T09:56:03.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 -- 172.21.15.19:0/1018722262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13cc0b92a0 msgr2=0x7f13cc0b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13cc0b92a0 0x7f13cc0b32e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f13b800d800 tx=0x7f13b800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 -- 172.21.15.19:0/1018722262 shutdown_connections 2024-04-25T09:56:03.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f13b006b4d0 0x7f13b006d990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13cc15dbe0 0x7f13cc0caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13cc0b92a0 0x7f13cc0b32e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 --2- 172.21.15.19:0/1018722262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13cc0b2cb0 0x7f13cc0b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 -- 172.21.15.19:0/1018722262 >> 172.21.15.19:0/1018722262 conn(0x7f13cc061a20 msgr2=0x7f13cc15c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 -- 172.21.15.19:0/1018722262 shutdown_connections 2024-04-25T09:56:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.430+0000 7f13d31a5640 1 -- 172.21.15.19:0/1018722262 wait complete. 2024-04-25T09:56:03.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:56:03.554 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T09:56:03.554 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:03.563 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:56:03.563 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:56:03.563 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:03.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:56:03.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.664+0000 7f113d2cb640 1 Processor -- start 2024-04-25T09:56:03.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.664+0000 7f113d2cb640 1 -- start start 2024-04-25T09:56:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.664+0000 7f113d2cb640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f113815ba80 0x7f11380cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.664+0000 7f113d2cb640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113815ce20 con 0x7f11380cf250 2024-04-25T09:56:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.664+0000 7f113d2cb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113806fe50 con 0x7f113815c990 2024-04-25T09:56:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.664+0000 7f113d2cb640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1138070130 con 0x7f113815ba80 2024-04-25T09:56:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.665+0000 7f1136ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f113815ba80 0x7f11380cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.665+0000 7f1136ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f113815ba80 0x7f11380cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46904/0 (socket says 172.21.15.19:46904) 2024-04-25T09:56:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.665+0000 7f1136ffd640 1 -- 172.21.15.19:0/2282025087 learned_addr learned my addr 172.21.15.19:0/2282025087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.665+0000 7f11377fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f11380cf250 0x7f11380b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40570/0 (socket says 172.21.15.19:40570) 2024-04-25T09:56:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.665+0000 7f1136ffd640 1 -- 172.21.15.19:0/2282025087 >> v1:172.21.15.112:6789/0 conn(0x7f113815c990 legacy=0x7f11380ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.665+0000 7f1136ffd640 1 -- 172.21.15.19:0/2282025087 >> v1:172.21.15.19:6789/0 conn(0x7f11380cf250 legacy=0x7f11380b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.665+0000 7f1135ffb640 1 -- 172.21.15.19:0/2282025087 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515534578 0 0) 0x7f113815ce20 con 0x7f11380cf250 2024-04-25T09:56:03.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.665+0000 7f1136ffd640 1 -- 172.21.15.19:0/2282025087 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11380cc8b0 con 0x7f113815ba80 2024-04-25T09:56:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.666+0000 7f1136ffd640 1 --2- 172.21.15.19:0/2282025087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113815ba80 0x7f11380cd390 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1130009870 tx=0x7f113002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7d3f21dd836833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.666+0000 7f1135ffb640 1 -- 172.21.15.19:0/2282025087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130037070 con 0x7f113815ba80 2024-04-25T09:56:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.666+0000 7f1135ffb640 1 -- 172.21.15.19:0/2282025087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1130032440 con 0x7f113815ba80 2024-04-25T09:56:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.666+0000 7f1135ffb640 1 -- 172.21.15.19:0/2282025087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130032810 con 0x7f113815ba80 2024-04-25T09:56:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.666+0000 7f113d2cb640 1 -- 172.21.15.19:0/2282025087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113815ba80 msgr2=0x7f11380cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.666+0000 7f113d2cb640 1 --2- 172.21.15.19:0/2282025087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113815ba80 0x7f11380cd390 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1130009870 tx=0x7f113002a210 comp rx=0 tx=0).stop 2024-04-25T09:56:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.666+0000 7f113d2cb640 1 -- 172.21.15.19:0/2282025087 shutdown_connections 2024-04-25T09:56:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.666+0000 7f113d2cb640 1 --2- 172.21.15.19:0/2282025087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113815ba80 0x7f11380cd390 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.666+0000 7f113d2cb640 1 -- 172.21.15.19:0/2282025087 >> 172.21.15.19:0/2282025087 conn(0x7f113806abe0 msgr2=0x7f113806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.666+0000 7f113d2cb640 1 -- 172.21.15.19:0/2282025087 shutdown_connections 2024-04-25T09:56:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.666+0000 7f113d2cb640 1 -- 172.21.15.19:0/2282025087 wait complete. 2024-04-25T09:56:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.668+0000 7f113d2cb640 1 Processor -- start 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.668+0000 7f113d2cb640 1 -- start start 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.668+0000 7f113d2cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11380cf250 0x7f113814c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.668+0000 7f113d2cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f113815ba80 0x7f113806d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.668+0000 7f113d2cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113815c990 0x7f113806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.668+0000 7f113d2cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113806fb00 con 0x7f113815c990 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.668+0000 7f113d2cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11380cce70 con 0x7f11380cf250 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.668+0000 7f113d2cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113806b850 con 0x7f113815ba80 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f11377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113815c990 0x7f113806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f11377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113815c990 0x7f113806dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46920/0 (socket says 172.21.15.19:46920) 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f11377fe640 1 -- 172.21.15.19:0/1162452709 learned_addr learned my addr 172.21.15.19:0/1162452709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f1136ffd640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11380cf250 0x7f113814c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f11367fc640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f113815ba80 0x7f113806d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f11377fe640 1 -- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f113815ba80 msgr2=0x7f113806d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f11377fe640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f113815ba80 0x7f113806d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f11377fe640 1 -- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11380cf250 msgr2=0x7f113814c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f11377fe640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11380cf250 0x7f113814c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f11377fe640 1 -- 172.21.15.19:0/1162452709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113806e250 con 0x7f113815c990 2024-04-25T09:56:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f11377fe640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113815c990 0x7f113806dc20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f112800c700 tx=0x7f112800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f111f7fe640 1 -- 172.21.15.19:0/1162452709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128018070 con 0x7f113815c990 2024-04-25T09:56:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f111f7fe640 1 -- 172.21.15.19:0/1162452709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112800ce10 con 0x7f113815c990 2024-04-25T09:56:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f113d2cb640 1 -- 172.21.15.19:0/1162452709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113806e530 con 0x7f113815c990 2024-04-25T09:56:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f111f7fe640 1 -- 172.21.15.19:0/1162452709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112800a860 con 0x7f113815c990 2024-04-25T09:56:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.669+0000 7f113d2cb640 1 -- 172.21.15.19:0/1162452709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113806eaf0 con 0x7f113815c990 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.671+0000 7f111f7fe640 1 -- 172.21.15.19:0/1162452709 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f112801b070 con 0x7f113815c990 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.671+0000 7f111f7fe640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 0x7f112406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.671+0000 7f111f7fe640 1 -- 172.21.15.19:0/1162452709 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1128089990 con 0x7f113815c990 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.671+0000 7f1136ffd640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 0x7f112406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.672+0000 7f1136ffd640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 0x7f112406d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1130034000 tx=0x7f113002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.672+0000 7f113ffcf740 1 Processor -- start 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.672+0000 7f113ffcf740 1 -- start start 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecd21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beecd4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecb4af0 0x560beecd1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f113ffcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beebf0e60 con 0x560beecd3f60 2024-04-25T09:56:03.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f113ffcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beeb9a120 con 0x560beecb4af0 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f113ffcf740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560beebecf00 con 0x560beec8e130 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f1136ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecd21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f1136ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecd21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46936/0 (socket says 172.21.15.19:46936) 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f1136ffd640 1 -- 172.21.15.19:0/1871958522 learned_addr learned my addr 172.21.15.19:0/1871958522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f11377fe640 1 --2- 172.21.15.19:0/1871958522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecb4af0 0x560beecd1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f11367fc640 1 --2- 172.21.15.19:0/1871958522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beecd4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f1136ffd640 1 -- 172.21.15.19:0/1871958522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 msgr2=0x560beecd4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f1136ffd640 1 --2- 172.21.15.19:0/1871958522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beecd4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f1136ffd640 1 -- 172.21.15.19:0/1871958522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecb4af0 msgr2=0x560beecd1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f1136ffd640 1 --2- 172.21.15.19:0/1871958522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecb4af0 0x560beecd1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.673+0000 7f1136ffd640 1 -- 172.21.15.19:0/1871958522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bee9d1f00 con 0x560beecd3f60 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.674+0000 7f1136ffd640 1 --2- 172.21.15.19:0/1871958522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecd21b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1130000c00 tx=0x7f113002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5d21f9ef52b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.674+0000 7f111cff9640 1 -- 172.21.15.19:0/1871958522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130037040 con 0x560beecd3f60 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.674+0000 7f111cff9640 1 -- 172.21.15.19:0/1871958522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113004d070 con 0x560beecd3f60 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.674+0000 7f111cff9640 1 -- 172.21.15.19:0/1871958522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113003d640 con 0x560beecd3f60 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.674+0000 7f113ffcf740 1 -- 172.21.15.19:0/1871958522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 msgr2=0x560beecd21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.674+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1871958522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecd21b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1130000c00 tx=0x7f113002c000 comp rx=0 tx=0).stop 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.674+0000 7f113ffcf740 1 -- 172.21.15.19:0/1871958522 shutdown_connections 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.674+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1871958522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecb4af0 0x560beecd1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.674+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1871958522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beecd4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.674+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1871958522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecd21b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.674+0000 7f113ffcf740 1 -- 172.21.15.19:0/1871958522 >> 172.21.15.19:0/1871958522 conn(0x560beec54f70 msgr2=0x560beec8b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.675+0000 7f113ffcf740 1 -- 172.21.15.19:0/1871958522 shutdown_connections 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.675+0000 7f113ffcf740 1 -- 172.21.15.19:0/1871958522 wait complete. 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.675+0000 7f113ffcf740 1 -- start start 2024-04-25T09:56:03.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.675+0000 7f113ffcf740 1 Processor -- start 2024-04-25T09:56:03.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beecb68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecb4af0 0x560beeccf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beeccf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f113ffcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beecd3760 con 0x560beecd3f60 2024-04-25T09:56:03.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f113ffcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beeb8fcd0 con 0x560beecb4af0 2024-04-25T09:56:03.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f113ffcf740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560beecd3480 con 0x560beec8e130 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f113ffcf740 10 client.0 _ll_register_callbacks cb 0x560bee9052e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f11377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beeccf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f11377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beeccf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46938/0 (socket says 172.21.15.19:46938) 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f11377fe640 1 -- 172.21.15.19:0/1230666373 learned_addr learned my addr 172.21.15.19:0/1230666373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f1136ffd640 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beecb68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f11367fc640 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecb4af0 0x560beeccf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f11377fe640 1 -- 172.21.15.19:0/1230666373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 msgr2=0x560beecb68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f11377fe640 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beecb68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f11377fe640 1 -- 172.21.15.19:0/1230666373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecb4af0 msgr2=0x560beeccf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f11377fe640 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecb4af0 0x560beeccf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f11377fe640 1 -- 172.21.15.19:0/1230666373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bee8c57f0 con 0x560beecd3f60 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f11367fc640 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecb4af0 0x560beeccf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:03.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f1136ffd640 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beecb68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f11377fe640 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beeccf990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f112800c6d0 tx=0x7f11280579f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98be016761db9a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.676+0000 7f1115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f1115ffb640 1 -- 172.21.15.19:0/1230666373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11280575d0 con 0x560beecd3f60 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f1115ffb640 1 -- 172.21.15.19:0/1230666373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f112805e070 con 0x560beecd3f60 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f113ffcf740 1 -- 172.21.15.19:0/1230666373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f113004d070 con 0x560beecd3f60 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f1115ffb640 1 -- 172.21.15.19:0/1230666373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f112805e270 con 0x560beecd3f60 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f113ffcf740 10 client.5958 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f113ffcf740 1 -- 172.21.15.19:0/1230666373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x560beea32690 con 0x560beecd3f60 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f1115ffb640 1 -- 172.21.15.19:0/1230666373 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113004d070 con 0x560beecd3f60 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f1115ffb640 1 -- 172.21.15.19:0/1230666373 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1128053cd0 con 0x560beecd3f60 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f113ffcf740 10 client.5958 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f113ffcf740 10 client.5958 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f113ffcf740 20 client.5958 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560beec8cc10 0x560beec8d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f113ffcf740 4 client.5958 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f113ffcf740 1 -- 172.21.15.19:0/1230666373 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560beebfe3a0 con 0x560beec8cc10 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f1136ffd640 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560beec8cc10 0x560beec8d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f1115ffb640 10 client.5958 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.677+0000 7f1136ffd640 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560beec8cc10 0x560beec8d020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=705cec0b87a7d953 server_cookie=cb53904c0901fb05 in_seq=0 out_seq=0 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f1115ffb640 1 -- 172.21.15.19:0/1230666373 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560beebfe3a0 con 0x560beec8cc10 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f1115ffb640 10 client.5958 handle_command_reply: tid=0 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 1 client.5958 shutdown 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 1 -- 172.21.15.19:0/1230666373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 msgr2=0x560beeccf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beeccf990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f112800c6d0 tx=0x7f11280579f0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 1 -- 172.21.15.19:0/1230666373 shutdown_connections 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560beec8cc10 0x560beec8d020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beeccf990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecb4af0 0x560beeccf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1230666373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beecb68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f1115ffb640 0 client.5958 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 1 -- 172.21.15.19:0/1230666373 >> 172.21.15.19:0/1230666373 conn(0x560beec54f70 msgr2=0x560beecb3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 1 -- 172.21.15.19:0/1230666373 shutdown_connections 2024-04-25T09:56:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 1 -- 172.21.15.19:0/1230666373 wait complete. 2024-04-25T09:56:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.678+0000 7f113ffcf740 20 client.5958 trim_cache size 0 max 16384 2024-04-25T09:56:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.693+0000 7f113ffcf740 1 Processor -- start 2024-04-25T09:56:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.693+0000 7f113ffcf740 1 -- start start 2024-04-25T09:56:03.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beec8d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecb4af0 0x560beeccf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecd3f60 0x560beec89190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f113ffcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1138200220 con 0x560beecb4af0 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f113ffcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11381ff3c0 con 0x560beecd3f60 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f113ffcf740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113814dd30 con 0x560beec8e130 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f11367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecb4af0 0x560beeccf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f11367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecb4af0 0x560beeccf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46952/0 (socket says 172.21.15.19:46952) 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f11367fc640 1 -- 172.21.15.19:0/1614104570 learned_addr learned my addr 172.21.15.19:0/1614104570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f11367fc640 1 -- 172.21.15.19:0/1614104570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 msgr2=0x560beec8d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f11367fc640 1 --2- 172.21.15.19:0/1614104570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beec8d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f11367fc640 1 -- 172.21.15.19:0/1614104570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecd3f60 msgr2=0x560beec89190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f11367fc640 1 --2- 172.21.15.19:0/1614104570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecd3f60 0x560beec89190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.694+0000 7f11367fc640 1 -- 172.21.15.19:0/1614104570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bee89e8b0 con 0x560beecb4af0 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f11367fc640 1 --2- 172.21.15.19:0/1614104570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecb4af0 0x560beeccf570 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f112c004ce0 tx=0x7f112c0051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ec1163b72493b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f1117fff640 1 -- 172.21.15.19:0/1614104570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c01a070 con 0x560beecb4af0 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f1117fff640 1 -- 172.21.15.19:0/1614104570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c0138b0 con 0x560beecb4af0 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f1117fff640 1 -- 172.21.15.19:0/1614104570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c013ca0 con 0x560beecb4af0 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f113ffcf740 1 -- 172.21.15.19:0/1614104570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecb4af0 msgr2=0x560beeccf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1614104570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecb4af0 0x560beeccf570 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f112c004ce0 tx=0x7f112c0051a0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f113ffcf740 1 -- 172.21.15.19:0/1614104570 shutdown_connections 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1614104570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beecd3f60 0x560beec89190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1614104570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecb4af0 0x560beeccf570 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f113ffcf740 1 --2- 172.21.15.19:0/1614104570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beec8e130 0x560beec8d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f113ffcf740 1 -- 172.21.15.19:0/1614104570 >> 172.21.15.19:0/1614104570 conn(0x560beec54f70 msgr2=0x560beecf5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f113ffcf740 1 -- 172.21.15.19:0/1614104570 shutdown_connections 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.695+0000 7f113ffcf740 1 -- 172.21.15.19:0/1614104570 wait complete. 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.696+0000 7f113ffcf740 1 -- start start 2024-04-25T09:56:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.696+0000 7f113ffcf740 1 Processor -- start 2024-04-25T09:56:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beec8e130 0x560beed3cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beecb4af0 0x560beecc8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f113ffcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecc87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f113ffcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560beecb81f0 con 0x560beecd3f60 2024-04-25T09:56:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f113ffcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11380ad480 con 0x560beec8e130 2024-04-25T09:56:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f113ffcf740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560beec8bed0 con 0x560beecb4af0 2024-04-25T09:56:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f113ffcf740 10 client.0 _ll_register_callbacks cb 0x7f1130035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f11377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecc87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f11377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecc87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46966/0 (socket says 172.21.15.19:46966) 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f11377fe640 1 -- 172.21.15.19:0/772379057 learned_addr learned my addr 172.21.15.19:0/772379057 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f11377fe640 1 -- 172.21.15.19:0/772379057 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beecb4af0 msgr2=0x560beecc8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f11377fe640 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beecb4af0 0x560beecc8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f11377fe640 1 -- 172.21.15.19:0/772379057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beec8e130 msgr2=0x560beed3cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f1136ffd640 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beec8e130 0x560beed3cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f11377fe640 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beec8e130 0x560beed3cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f11377fe640 1 -- 172.21.15.19:0/772379057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560beecb5dd0 con 0x560beecd3f60 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f11377fe640 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecc87c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f11280543e0 tx=0x7f1128087c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8465bfe2d69cb252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f1115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f1115ffb640 1 -- 172.21.15.19:0/772379057 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128057f80 con 0x560beecd3f60 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f1115ffb640 1 -- 172.21.15.19:0/772379057 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f11280575e0 con 0x560beecd3f60 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f113ffcf740 1 -- 172.21.15.19:0/772379057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f112c0138b0 con 0x560beecd3f60 2024-04-25T09:56:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f1115ffb640 1 -- 172.21.15.19:0/772379057 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1128056e40 con 0x560beecd3f60 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f113ffcf740 10 client.5964 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.697+0000 7f113ffcf740 1 -- 172.21.15.19:0/772379057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x560beec8ddb0 con 0x560beecd3f60 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.698+0000 7f1115ffb640 1 -- 172.21.15.19:0/772379057 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c0138b0 con 0x560beecd3f60 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.698+0000 7f1115ffb640 1 -- 172.21.15.19:0/772379057 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1128056fe0 con 0x560beecd3f60 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.698+0000 7f113ffcf740 10 client.5964 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.698+0000 7f113ffcf740 10 client.5964 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.698+0000 7f113ffcf740 20 client.5964 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.698+0000 7f113ffcf740 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560beec8ec60 0x560beec89100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.698+0000 7f113ffcf740 4 client.5964 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.698+0000 7f113ffcf740 1 -- 172.21.15.19:0/772379057 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x560beebf86f0 con 0x560beec8ec60 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.698+0000 7f1136ffd640 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560beec8ec60 0x560beec89100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.698+0000 7f1115ffb640 10 client.5964 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.698+0000 7f1136ffd640 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560beec8ec60 0x560beec89100 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40466d7b9e6320ba server_cookie=e25530e904ab18a6 in_seq=0 out_seq=0 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f1115ffb640 1 -- 172.21.15.19:0/772379057 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560beebf86f0 con 0x560beec8ec60 2024-04-25T09:56:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f1115ffb640 10 client.5964 handle_command_reply: tid=0 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 1 client.5964 shutdown 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 1 -- 172.21.15.19:0/772379057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 msgr2=0x560beecc87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecc87c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f11280543e0 tx=0x7f1128087c00 comp rx=0 tx=0).stop 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 1 -- 172.21.15.19:0/772379057 shutdown_connections 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560beec8ec60 0x560beec89100 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560beecd3f60 0x560beecc87c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560beecb4af0 0x560beecc8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 1 --2- 172.21.15.19:0/772379057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560beec8e130 0x560beed3cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 1 -- 172.21.15.19:0/772379057 >> 172.21.15.19:0/772379057 conn(0x560beec54f70 msgr2=0x560beec8d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f1115ffb640 0 client.5964 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 1 -- 172.21.15.19:0/772379057 shutdown_connections 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 1 -- 172.21.15.19:0/772379057 wait complete. 2024-04-25T09:56:03.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113ffcf740 20 client.5964 trim_cache size 0 max 16384 2024-04-25T09:56:03.709 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:03.709 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:56:03.709 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "3f670385-c06d-4cd6-a126-aebacfa310ad", 2024-04-25T09:56:03.709 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:56:03.709 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:03.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113d2cb640 1 -- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 msgr2=0x7f112406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113d2cb640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 0x7f112406d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1130034000 tx=0x7f113002e000 comp rx=0 tx=0).stop 2024-04-25T09:56:03.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113d2cb640 1 -- 172.21.15.19:0/1162452709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113815c990 msgr2=0x7f113806dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.699+0000 7f113d2cb640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113815c990 0x7f113806dc20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f112800c700 tx=0x7f112800cbc0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.700+0000 7f113d2cb640 1 -- 172.21.15.19:0/1162452709 shutdown_connections 2024-04-25T09:56:03.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.700+0000 7f113d2cb640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f112406b4d0 0x7f112406d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.700+0000 7f113d2cb640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113815c990 0x7f113806dc20 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.700+0000 7f113d2cb640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f113815ba80 0x7f113806d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.700+0000 7f113d2cb640 1 --2- 172.21.15.19:0/1162452709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11380cf250 0x7f113814c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.700+0000 7f113d2cb640 1 -- 172.21.15.19:0/1162452709 >> 172.21.15.19:0/1162452709 conn(0x7f113806abe0 msgr2=0x7f11380b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.700+0000 7f113d2cb640 1 -- 172.21.15.19:0/1162452709 shutdown_connections 2024-04-25T09:56:03.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.700+0000 7f113d2cb640 1 -- 172.21.15.19:0/1162452709 wait complete. 2024-04-25T09:56:03.714 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '3f670385-c06d-4cd6-a126-aebacfa310ad', 'mode': 'asynchronous'}' 2024-04-25T09:56:03.714 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T09:56:03.714 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:56:03.714 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.811+0000 7f90ccc37640 1 Processor -- start 2024-04-25T09:56:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.811+0000 7f90ccc37640 1 -- start start 2024-04-25T09:56:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.812+0000 7f90ccc37640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f90c815db70 0x7f90c8162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.812+0000 7f90ccc37640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90c80b5c30 0x7f90c80d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.812+0000 7f90ccc37640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c8162690 con 0x7f90c80b6cb0 2024-04-25T09:56:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.812+0000 7f90ccc37640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c80ceee0 con 0x7f90c815db70 2024-04-25T09:56:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.812+0000 7f90ccc37640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80d1d10 con 0x7f90c80b5c30 2024-04-25T09:56:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.812+0000 7f90c5d9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f90c80b6cb0 0x7f90c80b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40582/0 (socket says 172.21.15.19:40582) 2024-04-25T09:56:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.812+0000 7f90c5d9b640 1 -- 172.21.15.19:0/2405305233 learned_addr learned my addr 172.21.15.19:0/2405305233 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.812+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186278945 0 0) 0x7f90c8162690 con 0x7f90c80b6cb0 2024-04-25T09:56:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.812+0000 7f90c659c640 1 --2- 172.21.15.19:0/2405305233 >> v2:172.21.15.112:3300/0 conn(0x7f90c815db70 0x7f90c8162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90b0003520 con 0x7f90c80b6cb0 2024-04-25T09:56:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c6d9d640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90c80b5c30 0x7f90c80d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1026072773 0 0) 0x7f90b0003520 con 0x7f90c80b6cb0 2024-04-25T09:56:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90c80b5c30 msgr2=0x7f90c80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90c80b5c30 0x7f90c80d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 >> v2:172.21.15.112:3300/0 conn(0x7f90c815db70 msgr2=0x7f90c8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 --2- 172.21.15.19:0/2405305233 >> v2:172.21.15.112:3300/0 conn(0x7f90c815db70 0x7f90c8162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c80d1ff0 con 0x7f90c80b6cb0 2024-04-25T09:56:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f90bc001670 con 0x7f90c80b6cb0 2024-04-25T09:56:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 >> v1:172.21.15.19:6789/0 conn(0x7f90c80b6cb0 legacy=0x7f90c80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b00039b0 0x7f90b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b0004800 0x7f90b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90c80b5c30 0x7f90b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80ceee0 con 0x7f90b00039b0 2024-04-25T09:56:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80d1d10 con 0x7f90b0004800 2024-04-25T09:56:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b000b9c0 con 0x7f90c80b5c30 2024-04-25T09:56:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c659c640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b00039b0 0x7f90b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c5d9b640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b0004800 0x7f90b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c6d9d640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90c80b5c30 0x7f90b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c659c640 1 -- 172.21.15.19:0/2405305233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90c80b5c30 msgr2=0x7f90b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c659c640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90c80b5c30 0x7f90b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c659c640 1 -- 172.21.15.19:0/2405305233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b0004800 msgr2=0x7f90b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c659c640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b0004800 0x7f90b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.813+0000 7f90c659c640 1 -- 172.21.15.19:0/2405305233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b0003520 con 0x7f90b00039b0 2024-04-25T09:56:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.814+0000 7f90c659c640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b00039b0 0x7f90b0003dd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009ac0 tx=0x7f90b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfe33a435e8dcea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.814+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c0036070 con 0x7f90b00039b0 2024-04-25T09:56:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.814+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c0031400 con 0x7f90b00039b0 2024-04-25T09:56:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.814+0000 7f90c559a640 1 -- 172.21.15.19:0/2405305233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c0009690 con 0x7f90b00039b0 2024-04-25T09:56:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.815+0000 7f90ccc37640 1 -- 172.21.15.19:0/2405305233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b00039b0 msgr2=0x7f90b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.815+0000 7f90ccc37640 1 --2- 172.21.15.19:0/2405305233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b00039b0 0x7f90b0003dd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009ac0 tx=0x7f90b0005830 comp rx=0 tx=0).stop 2024-04-25T09:56:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.815+0000 7f90c6d9d640 1 -- 172.21.15.19:0/2405305233 reap_dead start 2024-04-25T09:56:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.815+0000 7f90ccc37640 1 -- 172.21.15.19:0/2405305233 shutdown_connections 2024-04-25T09:56:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.815+0000 7f90ccc37640 1 -- 172.21.15.19:0/2405305233 >> 172.21.15.19:0/2405305233 conn(0x7f90c80d0520 msgr2=0x7f90c80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.815+0000 7f90ccc37640 1 -- 172.21.15.19:0/2405305233 shutdown_connections 2024-04-25T09:56:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.815+0000 7f90ccc37640 1 -- 172.21.15.19:0/2405305233 wait complete. 2024-04-25T09:56:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.816+0000 7f90ccc37640 1 Processor -- start 2024-04-25T09:56:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.816+0000 7f90ccc37640 1 -- start start 2024-04-25T09:56:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90ccc37640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90b00039b0 0x7f90c80cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90ccc37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90c8157500 0x7f90c8155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90ccc37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c80c9900 0x7f90c815bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90ccc37640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80ce530 con 0x7f90c80c9900 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90ccc37640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c815df50 con 0x7f90c8157500 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90ccc37640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80cf1c0 con 0x7f90b00039b0 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90c6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c80c9900 0x7f90c815bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90c6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c80c9900 0x7f90c815bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46974/0 (socket says 172.21.15.19:46974) 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90c6d9d640 1 -- 172.21.15.19:0/2249895763 learned_addr learned my addr 172.21.15.19:0/2249895763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90c5d9b640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90c8157500 0x7f90c8155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90c659c640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90b00039b0 0x7f90c80cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90c6d9d640 1 -- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90b00039b0 msgr2=0x7f90c80cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90c6d9d640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90b00039b0 0x7f90c80cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90c6d9d640 1 -- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90c8157500 msgr2=0x7f90c8155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90c6d9d640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90c8157500 0x7f90c8155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.817+0000 7f90c6d9d640 1 -- 172.21.15.19:0/2249895763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c81547c0 con 0x7f90c80c9900 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.818+0000 7f90c6d9d640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c80c9900 0x7f90c815bf70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f90c80d0490 tx=0x7f90b800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.818+0000 7f90b6ffd640 1 -- 172.21.15.19:0/2249895763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8019070 con 0x7f90c80c9900 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.818+0000 7f90b6ffd640 1 -- 172.21.15.19:0/2249895763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b800f930 con 0x7f90c80c9900 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.818+0000 7f90ccc37640 1 -- 172.21.15.19:0/2249895763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c8154aa0 con 0x7f90c80c9900 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.818+0000 7f90b6ffd640 1 -- 172.21.15.19:0/2249895763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8003e60 con 0x7f90c80c9900 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.818+0000 7f90ccc37640 1 -- 172.21.15.19:0/2249895763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c8155ac0 con 0x7f90c80c9900 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.819+0000 7f90ccc37640 1 -- 172.21.15.19:0/2249895763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c8073990 con 0x7f90c80c9900 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.820+0000 7f90b6ffd640 1 -- 172.21.15.19:0/2249895763 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90b801c070 con 0x7f90c80c9900 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.820+0000 7f90b6ffd640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90a806b4d0 0x7f90a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.820+0000 7f90b6ffd640 1 -- 172.21.15.19:0/2249895763 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90b808ab80 con 0x7f90c80c9900 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.820+0000 7f90c659c640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90a806b4d0 0x7f90a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.821+0000 7f90c659c640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90a806b4d0 0x7f90a806d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f90c0033000 tx=0x7f90c0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:03.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:03.824+0000 7f90b6ffd640 1 -- 172.21.15.19:0/2249895763 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90b8010040 con 0x7f90c80c9900 2024-04-25T09:56:04.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.046+0000 7f90ccc37640 1 -- 172.21.15.19:0/2249895763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90c80ce7d0 con 0x7f90c80c9900 2024-04-25T09:56:04.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.047+0000 7f90b6ffd640 1 -- 172.21.15.19:0/2249895763 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f90b80592b0 con 0x7f90c80c9900 2024-04-25T09:56:04.051 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:04.051 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:55:54.352541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:04.051 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-04-25T09:56:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 -- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90a806b4d0 msgr2=0x7f90a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90a806b4d0 0x7f90a806d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f90c0033000 tx=0x7f90c0008030 comp rx=0 tx=0).stop 2024-04-25T09:56:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 -- 172.21.15.19:0/2249895763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c80c9900 msgr2=0x7f90c815bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c80c9900 0x7f90c815bf70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f90c80d0490 tx=0x7f90b800f1b0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 -- 172.21.15.19:0/2249895763 shutdown_connections 2024-04-25T09:56:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90a806b4d0 0x7f90a806d990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c80c9900 0x7f90c815bf70 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90c8157500 0x7f90c8155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 --2- 172.21.15.19:0/2249895763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90b00039b0 0x7f90c80cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 -- 172.21.15.19:0/2249895763 >> 172.21.15.19:0/2249895763 conn(0x7f90c80d0520 msgr2=0x7f90c80b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 -- 172.21.15.19:0/2249895763 shutdown_connections 2024-04-25T09:56:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.049+0000 7f90ccc37640 1 -- 172.21.15.19:0/2249895763 wait complete. 2024-04-25T09:56:04.064 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-25T09:56:04.172 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:04.172 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T09:56:04.172 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:04.181 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:56:04.181 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T09:56:04.181 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:04.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T09:56:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.280+0000 7f2cd0a7d640 1 Processor -- start 2024-04-25T09:56:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.280+0000 7f2cd0a7d640 1 -- start start 2024-04-25T09:56:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.281+0000 7f2cd0a7d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ccc0b9750 0x7f2ccc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.281+0000 7f2cd0a7d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ccc0cf060 con 0x7f2ccc0cebb0 2024-04-25T09:56:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.281+0000 7f2cd0a7d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ccc161d70 con 0x7f2ccc15da90 2024-04-25T09:56:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.281+0000 7f2cd0a7d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ccc162600 con 0x7f2ccc0b9750 2024-04-25T09:56:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.282+0000 7f2cc9d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ccc0b9750 0x7f2ccc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.282+0000 7f2ccad9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2ccc0cebb0 0x7f2ccc0d2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40598/0 (socket says 172.21.15.19:40598) 2024-04-25T09:56:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.282+0000 7f2ccad9d640 1 -- 172.21.15.19:0/450880036 learned_addr learned my addr 172.21.15.19:0/450880036 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.282+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/450880036 >> v1:172.21.15.112:6789/0 conn(0x7f2ccc15da90 legacy=0x7f2ccc161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.282+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/450880036 >> v1:172.21.15.19:6789/0 conn(0x7f2ccc0cebb0 legacy=0x7f2ccc0d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.282+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/450880036 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc162040 con 0x7f2ccc0b9750 2024-04-25T09:56:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.282+0000 7f2cc959a640 1 -- 172.21.15.19:0/450880036 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043059469 0 0) 0x7f2ccc0cf060 con 0x7f2ccc0cebb0 2024-04-25T09:56:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.283+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/450880036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc0b9750 0x7f2ccc0ce670 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00097a0 tx=0x7f2cc002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca30cd846b4995b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.283+0000 7f2cc959a640 1 -- 172.21.15.19:0/450880036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0037070 con 0x7f2ccc0b9750 2024-04-25T09:56:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.283+0000 7f2cc959a640 1 -- 172.21.15.19:0/450880036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc002ae50 con 0x7f2ccc0b9750 2024-04-25T09:56:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.283+0000 7f2cc959a640 1 -- 172.21.15.19:0/450880036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0032660 con 0x7f2ccc0b9750 2024-04-25T09:56:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.283+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/450880036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc0b9750 msgr2=0x7f2ccc0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.283+0000 7f2cd0a7d640 1 --2- 172.21.15.19:0/450880036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc0b9750 0x7f2ccc0ce670 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00097a0 tx=0x7f2cc002a140 comp rx=0 tx=0).stop 2024-04-25T09:56:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.283+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/450880036 shutdown_connections 2024-04-25T09:56:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.283+0000 7f2cd0a7d640 1 --2- 172.21.15.19:0/450880036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc0b9750 0x7f2ccc0ce670 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.283+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/450880036 >> 172.21.15.19:0/450880036 conn(0x7f2ccc061a20 msgr2=0x7f2ccc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.283+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/450880036 shutdown_connections 2024-04-25T09:56:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.283+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/450880036 wait complete. 2024-04-25T09:56:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.285+0000 7f2cd0a7d640 1 Processor -- start 2024-04-25T09:56:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.285+0000 7f2cd0a7d640 1 -- start start 2024-04-25T09:56:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cd0a7d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ccc0cebb0 0x7f2ccc151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cd0a7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc15da90 0x7f2ccc1e8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cd0a7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ccc1e8d80 0x7f2ccc0c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cd0a7d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc0d1df0 con 0x7f2ccc15da90 2024-04-25T09:56:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cd0a7d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc0d1b10 con 0x7f2ccc1e8d80 2024-04-25T09:56:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cd0a7d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc15de70 con 0x7f2ccc0cebb0 2024-04-25T09:56:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc15da90 0x7f2ccc1e8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc15da90 0x7f2ccc1e8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46990/0 (socket says 172.21.15.19:46990) 2024-04-25T09:56:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/2924609749 learned_addr learned my addr 172.21.15.19:0/2924609749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2ccad9d640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ccc1e8d80 0x7f2ccc0c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ccc0cebb0 0x7f2ccc151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ccc0cebb0 msgr2=0x7f2ccc151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ccc0cebb0 0x7f2ccc151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ccc1e8d80 msgr2=0x7f2ccc0c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ccc1e8d80 0x7f2ccc0c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/2924609749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc1e74c0 con 0x7f2ccc15da90 2024-04-25T09:56:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc15da90 0x7f2ccc1e8840 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00097a0 tx=0x7f2cc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cbaffd640 1 -- 172.21.15.19:0/2924609749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0042070 con 0x7f2ccc15da90 2024-04-25T09:56:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cbaffd640 1 -- 172.21.15.19:0/2924609749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc003e670 con 0x7f2ccc15da90 2024-04-25T09:56:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/2924609749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ccc1e77a0 con 0x7f2ccc15da90 2024-04-25T09:56:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.286+0000 7f2cbaffd640 1 -- 172.21.15.19:0/2924609749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0037040 con 0x7f2ccc15da90 2024-04-25T09:56:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.287+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/2924609749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ccc1e7d60 con 0x7f2ccc15da90 2024-04-25T09:56:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.288+0000 7f2cbaffd640 1 -- 172.21.15.19:0/2924609749 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2cc0045070 con 0x7f2ccc15da90 2024-04-25T09:56:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.288+0000 7f2cbaffd640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2cac06b4d0 0x7f2cac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.288+0000 7f2cbaffd640 1 -- 172.21.15.19:0/2924609749 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2cc00aabf0 con 0x7f2ccc15da90 2024-04-25T09:56:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.289+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2cac06b4d0 0x7f2cac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.289+0000 7f2cd3783740 1 Processor -- start 2024-04-25T09:56:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.289+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2cac06b4d0 0x7f2cac06d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc003d90 tx=0x7f2cbc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.289+0000 7f2cd3783740 1 -- start start 2024-04-25T09:56:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282237f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3282213c0 0x55d328223d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d3281f3c40 0x55d3281f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cd3783740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32815ca50 con 0x55d328220fe0 2024-04-25T09:56:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cd3783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d328105d40 con 0x55d3282213c0 2024-04-25T09:56:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cd3783740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d328158af0 con 0x55d3281f3c40 2024-04-25T09:56:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cca59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282237f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cca59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282237f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46994/0 (socket says 172.21.15.19:46994) 2024-04-25T09:56:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cca59c640 1 -- 172.21.15.19:0/2266027343 learned_addr learned my addr 172.21.15.19:0/2266027343 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/2266027343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3282213c0 0x55d328223d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2ccad9d640 1 --2- 172.21.15.19:0/2266027343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d3281f3c40 0x55d3281f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cca59c640 1 -- 172.21.15.19:0/2266027343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d3281f3c40 msgr2=0x55d3281f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2266027343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d3281f3c40 0x55d3281f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cca59c640 1 -- 172.21.15.19:0/2266027343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3282213c0 msgr2=0x55d328223d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2266027343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3282213c0 0x55d328223d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.290+0000 7f2cca59c640 1 -- 172.21.15.19:0/2266027343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d327f3d690 con 0x55d328220fe0 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.291+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2266027343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282237f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00d880 tx=0x7f2cbc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213ffdb45b53176f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.291+0000 7f2cabfff640 1 -- 172.21.15.19:0/2266027343 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc024070 con 0x55d328220fe0 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.291+0000 7f2cabfff640 1 -- 172.21.15.19:0/2266027343 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc014ba0 con 0x55d328220fe0 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.291+0000 7f2cabfff640 1 -- 172.21.15.19:0/2266027343 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc01f400 con 0x55d328220fe0 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.291+0000 7f2cd3783740 1 -- 172.21.15.19:0/2266027343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 msgr2=0x55d3282237f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.291+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2266027343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282237f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00d880 tx=0x7f2cbc010000 comp rx=0 tx=0).stop 2024-04-25T09:56:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.291+0000 7f2cd3783740 1 -- 172.21.15.19:0/2266027343 shutdown_connections 2024-04-25T09:56:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.291+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2266027343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d3281f3c40 0x55d3281f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.291+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2266027343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3282213c0 0x55d328223d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.291+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2266027343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282237f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.291+0000 7f2cd3783740 1 -- 172.21.15.19:0/2266027343 >> 172.21.15.19:0/2266027343 conn(0x55d3281f5eb0 msgr2=0x55d32823e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.292+0000 7f2cd3783740 1 -- 172.21.15.19:0/2266027343 shutdown_connections 2024-04-25T09:56:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.292+0000 7f2cd3783740 1 -- 172.21.15.19:0/2266027343 wait complete. 2024-04-25T09:56:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.292+0000 7f2cd3783740 1 -- start start 2024-04-25T09:56:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.292+0000 7f2cd3783740 1 Processor -- start 2024-04-25T09:56:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328220fe0 0x55d328217e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d32824edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d328242760 0x55d32823b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cd3783740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3281f9060 con 0x55d3282213c0 2024-04-25T09:56:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cd3783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3280fb340 con 0x55d328220fe0 2024-04-25T09:56:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cd3783740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3281f9340 con 0x55d328242760 2024-04-25T09:56:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cd3783740 10 client.0 _ll_register_callbacks cb 0x55d328118cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d32824edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d32824edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47004/0 (socket says 172.21.15.19:47004) 2024-04-25T09:56:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/2788747022 learned_addr learned my addr 172.21.15.19:0/2788747022 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328220fe0 0x55d328217e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2ccad9d640 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d328242760 0x55d32823b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/2788747022 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d328242760 msgr2=0x55d32823b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.293+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d328242760 0x55d32823b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/2788747022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328220fe0 msgr2=0x55d328217e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328220fe0 0x55d328217e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/2788747022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d327f9de20 con 0x55d3282213c0 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d32824edd0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0032fd0 tx=0x7f2cc00793c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cffe13392367d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2ca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2788747022 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc004c040 con 0x55d3282213c0 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2788747022 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2cc0081580 con 0x55d3282213c0 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cd3783740 1 -- 172.21.15.19:0/2788747022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cbc014ba0 con 0x55d3282213c0 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2788747022 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cc0081780 con 0x55d3282213c0 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cd3783740 10 client.5985 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cd3783740 1 -- 172.21.15.19:0/2788747022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55d327e30f80 con 0x55d3282213c0 2024-04-25T09:56:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2788747022 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc014ba0 con 0x55d3282213c0 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2788747022 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2cc0078750 con 0x55d3282213c0 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cd3783740 10 client.5985 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cd3783740 10 client.5985 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cd3783740 20 client.5985 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d3282392f0 0x55d328240930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cd3783740 4 client.5985 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cd3783740 1 -- 172.21.15.19:0/2788747022 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d328169f90 con 0x55d3282392f0 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.294+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d3282392f0 0x55d328240930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2ca97fa640 10 client.5985 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d3282392f0 0x55d328240930 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63721d62243b614 server_cookie=2c98e44ba105515e in_seq=0 out_seq=0 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2788747022 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d328169f90 con 0x55d3282392f0 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2ca97fa640 10 client.5985 handle_command_reply: tid=0 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2cd3783740 1 client.5985 shutdown 2024-04-25T09:56:04.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2cd3783740 1 -- 172.21.15.19:0/2788747022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 msgr2=0x55d32824edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d32824edd0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0032fd0 tx=0x7f2cc00793c0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2cd3783740 1 -- 172.21.15.19:0/2788747022 shutdown_connections 2024-04-25T09:56:04.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d3282392f0 0x55d328240930 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d328242760 0x55d32823b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d32824edd0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2788747022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328220fe0 0x55d328217e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2ca97fa640 0 client.5985 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:04.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.295+0000 7f2cd3783740 1 -- 172.21.15.19:0/2788747022 >> 172.21.15.19:0/2788747022 conn(0x55d3281f3c40 msgr2=0x55d3281f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:04.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.296+0000 7f2cd3783740 1 -- 172.21.15.19:0/2788747022 shutdown_connections 2024-04-25T09:56:04.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.296+0000 7f2cd3783740 1 -- 172.21.15.19:0/2788747022 wait complete. 2024-04-25T09:56:04.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.296+0000 7f2cd3783740 20 client.5985 trim_cache size 0 max 16384 2024-04-25T09:56:04.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.311+0000 7f2cd3783740 1 Processor -- start 2024-04-25T09:56:04.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.311+0000 7f2cd3783740 1 -- start start 2024-04-25T09:56:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.311+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328220fe0 0x55d3281f8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.311+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d328240ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.311+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d328242760 0x55d3281f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.311+0000 7f2cd3783740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc2004c0 con 0x55d3282213c0 2024-04-25T09:56:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.311+0000 7f2cd3783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc1ff670 con 0x55d328220fe0 2024-04-25T09:56:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.311+0000 7f2cd3783740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc1f7f20 con 0x55d328242760 2024-04-25T09:56:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d328240ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d328240ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47014/0 (socket says 172.21.15.19:47014) 2024-04-25T09:56:04.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/1743613807 learned_addr learned my addr 172.21.15.19:0/1743613807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2ccad9d640 1 --2- 172.21.15.19:0/1743613807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d328242760 0x55d3281f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cca59c640 1 --2- 172.21.15.19:0/1743613807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328220fe0 0x55d3281f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/1743613807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d328242760 msgr2=0x55d3281f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/1743613807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d328242760 0x55d3281f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/1743613807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328220fe0 msgr2=0x55d3281f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/1743613807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328220fe0 0x55d3281f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cc9d9b640 1 -- 172.21.15.19:0/1743613807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d32811cbc0 con 0x55d3282213c0 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2ccad9d640 1 --2- 172.21.15.19:0/1743613807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d328242760 0x55d3281f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cca59c640 1 --2- 172.21.15.19:0/1743613807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328220fe0 0x55d3281f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/1743613807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d328240ee0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0079290 tx=0x7f2cc0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e68756a3e49de51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cab7fe640 1 -- 172.21.15.19:0/1743613807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc004c070 con 0x55d3282213c0 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cab7fe640 1 -- 172.21.15.19:0/1743613807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc002ae50 con 0x55d3282213c0 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.312+0000 7f2cab7fe640 1 -- 172.21.15.19:0/1743613807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0032e10 con 0x55d3282213c0 2024-04-25T09:56:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.313+0000 7f2cd3783740 1 -- 172.21.15.19:0/1743613807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 msgr2=0x55d328240ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.313+0000 7f2cd3783740 1 --2- 172.21.15.19:0/1743613807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d328240ee0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0079290 tx=0x7f2cc0040000 comp rx=0 tx=0).stop 2024-04-25T09:56:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.313+0000 7f2cd3783740 1 -- 172.21.15.19:0/1743613807 shutdown_connections 2024-04-25T09:56:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.313+0000 7f2cd3783740 1 --2- 172.21.15.19:0/1743613807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d328242760 0x55d3281f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.313+0000 7f2cd3783740 1 --2- 172.21.15.19:0/1743613807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3282213c0 0x55d328240ee0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.313+0000 7f2cd3783740 1 --2- 172.21.15.19:0/1743613807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328220fe0 0x55d3281f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.313+0000 7f2cd3783740 1 -- 172.21.15.19:0/1743613807 >> 172.21.15.19:0/1743613807 conn(0x55d3281f3c40 msgr2=0x55d3281f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.313+0000 7f2cd3783740 1 -- 172.21.15.19:0/1743613807 shutdown_connections 2024-04-25T09:56:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.313+0000 7f2cd3783740 1 -- 172.21.15.19:0/1743613807 wait complete. 2024-04-25T09:56:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.313+0000 7f2cd3783740 1 -- start start 2024-04-25T09:56:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.313+0000 7f2cd3783740 1 Processor -- start 2024-04-25T09:56:04.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.314+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282383f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.314+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d3282213c0 0x55d3281f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.314+0000 7f2cd3783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328242760 0x55d3281f7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.314+0000 7f2cd3783740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc15f580 con 0x55d328220fe0 2024-04-25T09:56:04.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.314+0000 7f2cd3783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3281f9270 con 0x55d328242760 2024-04-25T09:56:04.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.314+0000 7f2cd3783740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3281f5100 con 0x55d3282213c0 2024-04-25T09:56:04.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.314+0000 7f2cd3783740 10 client.0 _ll_register_callbacks cb 0x7f2cc0079130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:04.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.314+0000 7f2cca59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.314+0000 7f2cca59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282383f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47020/0 (socket says 172.21.15.19:47020) 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.314+0000 7f2cca59c640 1 -- 172.21.15.19:0/2480617456 learned_addr learned my addr 172.21.15.19:0/2480617456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d3282213c0 0x55d3281f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2ccad9d640 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328242760 0x55d3281f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2cca59c640 1 -- 172.21.15.19:0/2480617456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d3282213c0 msgr2=0x55d3281f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d3282213c0 0x55d3281f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2cca59c640 1 -- 172.21.15.19:0/2480617456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328242760 msgr2=0x55d3281f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328242760 0x55d3281f7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2cca59c640 1 -- 172.21.15.19:0/2480617456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3281f7d90 con 0x55d328220fe0 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d3282213c0 0x55d3281f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2cca59c640 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282383f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00d150 tx=0x7f2cbc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf07938e18bf642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2ca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2480617456 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc024040 con 0x55d328220fe0 2024-04-25T09:56:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2480617456 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2cbc038c30 con 0x55d328220fe0 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2cd3783740 1 -- 172.21.15.19:0/2480617456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cc002ae50 con 0x55d328220fe0 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2480617456 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cbc03f050 con 0x55d328220fe0 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2cd3783740 10 client.5991 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2cd3783740 1 -- 172.21.15.19:0/2480617456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55d3282907b0 con 0x55d328220fe0 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.315+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2480617456 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc002ae50 con 0x55d328220fe0 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2480617456 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2cbc042020 con 0x55d328220fe0 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2cd3783740 10 client.5991 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2cd3783740 10 client.5991 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2cd3783740 20 client.5991 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d32828fd60 0x55d328290170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2cd3783740 4 client.5991 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2cd3783740 1 -- 172.21.15.19:0/2480617456 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55d3281642e0 con 0x55d32828fd60 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d32828fd60 0x55d328290170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2cc9d9b640 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d32828fd60 0x55d328290170 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec5198b410f39d2 server_cookie=b5f14e0f12b795b in_seq=0 out_seq=0 2024-04-25T09:56:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2ca97fa640 10 client.5991 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2ca97fa640 1 -- 172.21.15.19:0/2480617456 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d3281642e0 con 0x55d32828fd60 2024-04-25T09:56:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.316+0000 7f2ca97fa640 10 client.5991 handle_command_reply: tid=0 2024-04-25T09:56:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 1 client.5991 shutdown 2024-04-25T09:56:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 1 -- 172.21.15.19:0/2480617456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 msgr2=0x55d3282383f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282383f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00d150 tx=0x7f2cbc0036f0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 1 -- 172.21.15.19:0/2480617456 shutdown_connections 2024-04-25T09:56:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d32828fd60 0x55d328290170 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d328242760 0x55d3281f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d3282213c0 0x55d3281f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 1 --2- 172.21.15.19:0/2480617456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d328220fe0 0x55d3282383f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2ca97fa640 0 client.5991 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 1 -- 172.21.15.19:0/2480617456 >> 172.21.15.19:0/2480617456 conn(0x55d3281f3c40 msgr2=0x55d3282237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 1 -- 172.21.15.19:0/2480617456 shutdown_connections 2024-04-25T09:56:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 1 -- 172.21.15.19:0/2480617456 wait complete. 2024-04-25T09:56:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd3783740 20 client.5991 trim_cache size 0 max 16384 2024-04-25T09:56:04.327 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:04.327 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:56:04.327 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "4195ee7d-b509-4ae5-ad3e-f37876a7321f", 2024-04-25T09:56:04.328 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:56:04.328 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:04.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2cac06b4d0 msgr2=0x7f2cac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd0a7d640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2cac06b4d0 0x7f2cac06d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc003d90 tx=0x7f2cbc009040 comp rx=0 tx=0).stop 2024-04-25T09:56:04.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/2924609749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc15da90 msgr2=0x7f2ccc1e8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.317+0000 7f2cd0a7d640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc15da90 0x7f2ccc1e8840 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00097a0 tx=0x7f2cc00046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.318+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/2924609749 shutdown_connections 2024-04-25T09:56:04.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.318+0000 7f2cd0a7d640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2cac06b4d0 0x7f2cac06d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.318+0000 7f2cd0a7d640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ccc1e8d80 0x7f2ccc0c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.318+0000 7f2cd0a7d640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ccc15da90 0x7f2ccc1e8840 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.318+0000 7f2cd0a7d640 1 --2- 172.21.15.19:0/2924609749 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ccc0cebb0 0x7f2ccc151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.318+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/2924609749 >> 172.21.15.19:0/2924609749 conn(0x7f2ccc061a20 msgr2=0x7f2ccc0b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:04.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.318+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/2924609749 shutdown_connections 2024-04-25T09:56:04.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.318+0000 7f2cd0a7d640 1 -- 172.21.15.19:0/2924609749 wait complete. 2024-04-25T09:56:04.333 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '4195ee7d-b509-4ae5-ad3e-f37876a7321f', 'mode': 'asynchronous'}' 2024-04-25T09:56:04.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:04.333 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T09:56:04.351 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T09:56:04.351 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.487+0000 7f163c69e640 1 Processor -- start 2024-04-25T09:56:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.487+0000 7f163c69e640 1 -- start start 2024-04-25T09:56:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f163c69e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f163415dd30 0x7f16340b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f163c69e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16340d0ad0 0x7f16340b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f163c69e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340d1df0 con 0x7f16340b9770 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f163c69e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16340d20d0 con 0x7f163415dd30 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f163c69e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340d23b0 con 0x7f16340d0ad0 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f163a43a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f163415dd30 0x7f16340b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f1639c39640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f16340b9770 0x7f16340d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40602/0 (socket says 172.21.15.19:40602) 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f1639c39640 1 -- 172.21.15.19:0/1521973545 learned_addr learned my addr 172.21.15.19:0/1521973545 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f163ac3b640 1 --2- 172.21.15.19:0/1521973545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16340d0ad0 0x7f16340b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f163a43a640 1 -- 172.21.15.19:0/1521973545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16340d0ad0 msgr2=0x7f16340b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f163a43a640 1 --2- 172.21.15.19:0/1521973545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16340d0ad0 0x7f16340b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f163a43a640 1 -- 172.21.15.19:0/1521973545 >> v1:172.21.15.19:6789/0 conn(0x7f16340b9770 legacy=0x7f16340d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f163a43a640 1 -- 172.21.15.19:0/1521973545 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340d2690 con 0x7f163415dd30 2024-04-25T09:56:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.488+0000 7f1639438640 1 -- 172.21.15.19:0/1521973545 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165289641 0 0) 0x7f16340d1df0 con 0x7f16340b9770 2024-04-25T09:56:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.489+0000 7f163a43a640 1 --2- 172.21.15.19:0/1521973545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f163415dd30 0x7f16340b9230 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f16300097a0 tx=0x7f163002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d703f761329b60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.489+0000 7f1639438640 1 -- 172.21.15.19:0/1521973545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630037070 con 0x7f163415dd30 2024-04-25T09:56:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.489+0000 7f1639438640 1 -- 172.21.15.19:0/1521973545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163002ae50 con 0x7f163415dd30 2024-04-25T09:56:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.489+0000 7f1639438640 1 -- 172.21.15.19:0/1521973545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630032660 con 0x7f163415dd30 2024-04-25T09:56:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.490+0000 7f163c69e640 1 -- 172.21.15.19:0/1521973545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f163415dd30 msgr2=0x7f16340b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.490+0000 7f163c69e640 1 --2- 172.21.15.19:0/1521973545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f163415dd30 0x7f16340b9230 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f16300097a0 tx=0x7f163002a100 comp rx=0 tx=0).stop 2024-04-25T09:56:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.490+0000 7f163c69e640 1 -- 172.21.15.19:0/1521973545 shutdown_connections 2024-04-25T09:56:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.490+0000 7f163c69e640 1 --2- 172.21.15.19:0/1521973545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16340d0ad0 0x7f16340b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.490+0000 7f163c69e640 1 --2- 172.21.15.19:0/1521973545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f163415dd30 0x7f16340b9230 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.490+0000 7f163c69e640 1 -- 172.21.15.19:0/1521973545 >> 172.21.15.19:0/1521973545 conn(0x7f1634061c30 msgr2=0x7f163406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:04.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.490+0000 7f163c69e640 1 -- 172.21.15.19:0/1521973545 shutdown_connections 2024-04-25T09:56:04.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.490+0000 7f163c69e640 1 -- 172.21.15.19:0/1521973545 wait complete. 2024-04-25T09:56:04.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.491+0000 7f163c69e640 1 Processor -- start 2024-04-25T09:56:04.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.491+0000 7f163c69e640 1 -- start start 2024-04-25T09:56:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163c69e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16340b9770 0x7f16340caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163c69e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16340d0ad0 0x7f163415b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163c69e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f163415dd30 0x7f16341f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163c69e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16340d1b90 con 0x7f16340b9770 2024-04-25T09:56:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163c69e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16340d1a10 con 0x7f16340d0ad0 2024-04-25T09:56:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163c69e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340d2e90 con 0x7f163415dd30 2024-04-25T09:56:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163a43a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16340b9770 0x7f16340caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163a43a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16340b9770 0x7f16340caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47046/0 (socket says 172.21.15.19:47046) 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163a43a640 1 -- 172.21.15.19:0/1187138953 learned_addr learned my addr 172.21.15.19:0/1187138953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f1639c39640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16340d0ad0 0x7f163415b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163ac3b640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f163415dd30 0x7f16341f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163a43a640 1 -- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f163415dd30 msgr2=0x7f16341f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163a43a640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f163415dd30 0x7f16341f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163a43a640 1 -- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16340d0ad0 msgr2=0x7f163415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163a43a640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16340d0ad0 0x7f163415b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.492+0000 7f163a43a640 1 -- 172.21.15.19:0/1187138953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163415e110 con 0x7f16340b9770 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.493+0000 7f163a43a640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16340b9770 0x7f16340caa80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1630035040 tx=0x7f16300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.493+0000 7f162affd640 1 -- 172.21.15.19:0/1187138953 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630041070 con 0x7f16340b9770 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.493+0000 7f162affd640 1 -- 172.21.15.19:0/1187138953 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163003c460 con 0x7f16340b9770 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.493+0000 7f163c69e640 1 -- 172.21.15.19:0/1187138953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163415e3d0 con 0x7f16340b9770 2024-04-25T09:56:04.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.493+0000 7f162affd640 1 -- 172.21.15.19:0/1187138953 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630037040 con 0x7f16340b9770 2024-04-25T09:56:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.493+0000 7f163c69e640 1 -- 172.21.15.19:0/1187138953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163415e990 con 0x7f16340b9770 2024-04-25T09:56:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.494+0000 7f163c69e640 1 -- 172.21.15.19:0/1187138953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1634073990 con 0x7f16340b9770 2024-04-25T09:56:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.494+0000 7f162affd640 1 -- 172.21.15.19:0/1187138953 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f163003a070 con 0x7f16340b9770 2024-04-25T09:56:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.495+0000 7f162affd640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f161806b4d0 0x7f161806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.495+0000 7f162affd640 1 -- 172.21.15.19:0/1187138953 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16300aa3c0 con 0x7f16340b9770 2024-04-25T09:56:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.495+0000 7f1639c39640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f161806b4d0 0x7f161806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.496+0000 7f1639c39640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f161806b4d0 0x7f161806d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16340cafc0 tx=0x7f1624009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.499+0000 7f162affd640 1 -- 172.21.15.19:0/1187138953 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1630078b80 con 0x7f16340b9770 2024-04-25T09:56:04.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.714+0000 7f163c69e640 1 -- 172.21.15.19:0/1187138953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16340c6fd0 con 0x7f16340b9770 2024-04-25T09:56:04.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.715+0000 7f162affd640 1 -- 172.21.15.19:0/1187138953 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f163007cc20 con 0x7f16340b9770 2024-04-25T09:56:04.719 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:04.719 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:55:54.352541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:04.719 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-04-25T09:56:04.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 -- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f161806b4d0 msgr2=0x7f161806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f161806b4d0 0x7f161806d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16340cafc0 tx=0x7f1624009040 comp rx=0 tx=0).stop 2024-04-25T09:56:04.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 -- 172.21.15.19:0/1187138953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16340b9770 msgr2=0x7f16340caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:04.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16340b9770 0x7f16340caa80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1630035040 tx=0x7f16300046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 -- 172.21.15.19:0/1187138953 shutdown_connections 2024-04-25T09:56:04.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f161806b4d0 0x7f161806d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f163415dd30 0x7f16341f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16340d0ad0 0x7f163415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 --2- 172.21.15.19:0/1187138953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16340b9770 0x7f16340caa80 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:04.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 -- 172.21.15.19:0/1187138953 >> 172.21.15.19:0/1187138953 conn(0x7f1634061c30 msgr2=0x7f163415cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:04.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 -- 172.21.15.19:0/1187138953 shutdown_connections 2024-04-25T09:56:04.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:04.717+0000 7f163c69e640 1 -- 172.21.15.19:0/1187138953 wait complete. 2024-04-25T09:56:04.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T09:56:04.844 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:04.844 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T09:56:04.844 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:04.853 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:56:04.853 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:56:04.854 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:04.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:04.869 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T09:56:04.934 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:56:04.934 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:56:04.934 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814932 s, 129 MB/s 2024-04-25T09:56:04.934 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T09:56:04.934 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:05.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.072+0000 7f66ec2b3640 1 Processor -- start 2024-04-25T09:56:05.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.072+0000 7f66ec2b3640 1 -- start start 2024-04-25T09:56:05.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ec2b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 0x7f66e40d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ec2b3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66e40d0900 0x7f66e40b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ec2b3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e40b9940 con 0x7f66e415dd30 2024-04-25T09:56:05.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ec2b3640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e40d0d70 con 0x7f66e40d0900 2024-04-25T09:56:05.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ec2b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40d1df0 con 0x7f66e40d2e90 2024-04-25T09:56:05.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ea850640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66e40d0900 0x7f66e40b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ea850640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66e40d0900 0x7f66e40b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47054/0 (socket says 172.21.15.19:47054) 2024-04-25T09:56:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ea850640 1 -- 172.21.15.19:0/3225631686 learned_addr learned my addr 172.21.15.19:0/3225631686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66e984e640 1 --2- 172.21.15.19:0/3225631686 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 0x7f66e40d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ea850640 1 -- 172.21.15.19:0/3225631686 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 msgr2=0x7f66e40d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ea850640 1 --2- 172.21.15.19:0/3225631686 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 0x7f66e40d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ea850640 1 -- 172.21.15.19:0/3225631686 >> v1:172.21.15.112:6789/0 conn(0x7f66e415dd30 legacy=0x7f66e40b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66ea850640 1 -- 172.21.15.19:0/3225631686 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e40d20d0 con 0x7f66e40d0900 2024-04-25T09:56:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.073+0000 7f66e984e640 1 --2- 172.21.15.19:0/3225631686 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 0x7f66e40d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.074+0000 7f66ea850640 1 --2- 172.21.15.19:0/3225631686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66e40d0900 0x7f66e40b6a40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009e50 tx=0x7f66dc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec06a31c9f667ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.074+0000 7f66e904d640 1 -- 172.21.15.19:0/3225631686 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc036070 con 0x7f66e40d0900 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.074+0000 7f66e904d640 1 -- 172.21.15.19:0/3225631686 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66dc031600 con 0x7f66e40d0900 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.074+0000 7f66e904d640 1 -- 172.21.15.19:0/3225631686 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc0319d0 con 0x7f66e40d0900 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.074+0000 7f66ec2b3640 1 -- 172.21.15.19:0/3225631686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66e40d0900 msgr2=0x7f66e40b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.074+0000 7f66ec2b3640 1 --2- 172.21.15.19:0/3225631686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66e40d0900 0x7f66e40b6a40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009e50 tx=0x7f66dc02a680 comp rx=0 tx=0).stop 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.075+0000 7f66ec2b3640 1 -- 172.21.15.19:0/3225631686 shutdown_connections 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.075+0000 7f66ec2b3640 1 --2- 172.21.15.19:0/3225631686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66e40d0900 0x7f66e40b6a40 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.075+0000 7f66ec2b3640 1 --2- 172.21.15.19:0/3225631686 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 0x7f66e40d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.075+0000 7f66ec2b3640 1 -- 172.21.15.19:0/3225631686 >> 172.21.15.19:0/3225631686 conn(0x7f66e4061c30 msgr2=0x7f66e406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.075+0000 7f66ec2b3640 1 -- 172.21.15.19:0/3225631686 shutdown_connections 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.075+0000 7f66ec2b3640 1 -- 172.21.15.19:0/3225631686 wait complete. 2024-04-25T09:56:05.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.076+0000 7f66ec2b3640 1 Processor -- start 2024-04-25T09:56:05.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.076+0000 7f66ec2b3640 1 -- start start 2024-04-25T09:56:05.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ec2b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0900 0x7f66e41571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ec2b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 0x7f66e4152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ec2b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66e415dd30 0x7f66e4153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ec2b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40d2970 con 0x7f66e415dd30 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ec2b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40d2690 con 0x7f66e40d0900 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ec2b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40d1a10 con 0x7f66e40d2e90 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ea850640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66e415dd30 0x7f66e4153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ea850640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66e415dd30 0x7f66e4153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47058/0 (socket says 172.21.15.19:47058) 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ea850640 1 -- 172.21.15.19:0/2347846012 learned_addr learned my addr 172.21.15.19:0/2347846012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66e984e640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 0x7f66e4152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ea04f640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0900 0x7f66e41571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ea850640 1 -- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 msgr2=0x7f66e4152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ea850640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 0x7f66e4152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ea850640 1 -- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0900 msgr2=0x7f66e41571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ea850640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0900 0x7f66e41571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ea850640 1 -- 172.21.15.19:0/2347846012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e40cb4a0 con 0x7f66e415dd30 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ea04f640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0900 0x7f66e41571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66e984e640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 0x7f66e4152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ea850640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66e415dd30 0x7f66e4153c60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009f70 tx=0x7f66dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66daffd640 1 -- 172.21.15.19:0/2347846012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc03f020 con 0x7f66e415dd30 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66daffd640 1 -- 172.21.15.19:0/2347846012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66dc037040 con 0x7f66e415dd30 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ec2b3640 1 -- 172.21.15.19:0/2347846012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e40cb740 con 0x7f66e415dd30 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66daffd640 1 -- 172.21.15.19:0/2347846012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc036070 con 0x7f66e415dd30 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.077+0000 7f66ec2b3640 1 -- 172.21.15.19:0/2347846012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e40cbd00 con 0x7f66e415dd30 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.078+0000 7f66ec2b3640 1 -- 172.21.15.19:0/2347846012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e4073990 con 0x7f66e415dd30 2024-04-25T09:56:05.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.079+0000 7f66daffd640 1 -- 172.21.15.19:0/2347846012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66dc042020 con 0x7f66e415dd30 2024-04-25T09:56:05.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.079+0000 7f66daffd640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66c806b4d0 0x7f66c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.079+0000 7f66daffd640 1 -- 172.21.15.19:0/2347846012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f66dc0aa7d0 con 0x7f66e415dd30 2024-04-25T09:56:05.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.080+0000 7f66ea04f640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66c806b4d0 0x7f66c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.080+0000 7f66ea04f640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66c806b4d0 0x7f66c806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f66e40c6eb0 tx=0x7f66e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.083+0000 7f66daffd640 1 -- 172.21.15.19:0/2347846012 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66dc049030 con 0x7f66e415dd30 2024-04-25T09:56:05.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.306+0000 7f66ec2b3640 1 -- 172.21.15.19:0/2347846012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66e4157720 con 0x7f66e415dd30 2024-04-25T09:56:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.307+0000 7f66daffd640 1 -- 172.21.15.19:0/2347846012 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f66dc079550 con 0x7f66e415dd30 2024-04-25T09:56:05.311 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:05.311 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:55:54.352541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:05.311 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-04-25T09:56:05.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 -- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66c806b4d0 msgr2=0x7f66c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66c806b4d0 0x7f66c806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f66e40c6eb0 tx=0x7f66e0009040 comp rx=0 tx=0).stop 2024-04-25T09:56:05.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 -- 172.21.15.19:0/2347846012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66e415dd30 msgr2=0x7f66e4153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66e415dd30 0x7f66e4153c60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009f70 tx=0x7f66dc004a40 comp rx=0 tx=0).stop 2024-04-25T09:56:05.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 -- 172.21.15.19:0/2347846012 shutdown_connections 2024-04-25T09:56:05.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66c806b4d0 0x7f66c806d990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66e415dd30 0x7f66e4153c60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66e40d2e90 0x7f66e4152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 --2- 172.21.15.19:0/2347846012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e40d0900 0x7f66e41571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 -- 172.21.15.19:0/2347846012 >> 172.21.15.19:0/2347846012 conn(0x7f66e4061c30 msgr2=0x7f66e40c7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 -- 172.21.15.19:0/2347846012 shutdown_connections 2024-04-25T09:56:05.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.309+0000 7f66ec2b3640 1 -- 172.21.15.19:0/2347846012 wait complete. 2024-04-25T09:56:05.324 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T09:56:05.434 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T09:56:05.434 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:05.443 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:56:05.443 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T09:56:05.443 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T09:56:05.443 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T09:56:05.444 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:56:05.444 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T09:56:05.444 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T09:56:05.510 INFO:teuthology.orchestra.run.smithi019.stdout:1099511630186 2024-04-25T09:56:05.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:56:05.519 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:05.519 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:56:05.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.657+0000 7fbbbdce1640 1 Processor -- start 2024-04-25T09:56:05.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.657+0000 7fbbbdce1640 1 -- start start 2024-04-25T09:56:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbbdce1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbb815b7d0 0x7fbbb80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbbdce1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbbb806c9e0 0x7fbbb80b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbbdce1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb806cf50 con 0x7fbbb80b5620 2024-04-25T09:56:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbbdce1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb80ce530 con 0x7fbbb815b7d0 2024-04-25T09:56:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbbdce1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb80ce810 con 0x7fbbb806c9e0 2024-04-25T09:56:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbb77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbb815b7d0 0x7fbbb80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbb6ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbbb80b5620 0x7fbbb806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40612/0 (socket says 172.21.15.19:40612) 2024-04-25T09:56:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/210779188 learned_addr learned my addr 172.21.15.19:0/210779188 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbb7fff640 1 --2- 172.21.15.19:0/210779188 >> v2:172.21.15.112:3300/0 conn(0x7fbbb806c9e0 0x7fbbb80b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbb77fe640 1 -- 172.21.15.19:0/210779188 >> v2:172.21.15.112:3300/0 conn(0x7fbbb806c9e0 msgr2=0x7fbbb80b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/210779188 >> v2:172.21.15.112:3300/0 conn(0x7fbbb806c9e0 0x7fbbb80b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbb77fe640 1 -- 172.21.15.19:0/210779188 >> v1:172.21.15.19:6789/0 conn(0x7fbbb80b5620 legacy=0x7fbbb806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbb67fc640 1 -- 172.21.15.19:0/210779188 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200293819 0 0) 0x7fbbb806cf50 con 0x7fbbb80b5620 2024-04-25T09:56:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbb77fe640 1 -- 172.21.15.19:0/210779188 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb80ceaf0 con 0x7fbbb815b7d0 2024-04-25T09:56:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.658+0000 7fbbb7fff640 1 --2- 172.21.15.19:0/210779188 >> v2:172.21.15.112:3300/0 conn(0x7fbbb806c9e0 0x7fbbb80b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.659+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/210779188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbb815b7d0 0x7fbbb80cf010 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8161890 tx=0x7fbba802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f31a5034c09a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.659+0000 7fbbb67fc640 1 -- 172.21.15.19:0/210779188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8037070 con 0x7fbbb815b7d0 2024-04-25T09:56:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.659+0000 7fbbb67fc640 1 -- 172.21.15.19:0/210779188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba8032440 con 0x7fbbb815b7d0 2024-04-25T09:56:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.659+0000 7fbbb67fc640 1 -- 172.21.15.19:0/210779188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8032810 con 0x7fbbb815b7d0 2024-04-25T09:56:05.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.660+0000 7fbbbdce1640 1 -- 172.21.15.19:0/210779188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbb815b7d0 msgr2=0x7fbbb80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.660+0000 7fbbbdce1640 1 --2- 172.21.15.19:0/210779188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbb815b7d0 0x7fbbb80cf010 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8161890 tx=0x7fbba802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.660+0000 7fbbbdce1640 1 -- 172.21.15.19:0/210779188 shutdown_connections 2024-04-25T09:56:05.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.660+0000 7fbbbdce1640 1 --2- 172.21.15.19:0/210779188 >> v2:172.21.15.112:3300/0 conn(0x7fbbb806c9e0 0x7fbbb80b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.660+0000 7fbbbdce1640 1 --2- 172.21.15.19:0/210779188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbb815b7d0 0x7fbbb80cf010 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.660+0000 7fbbbdce1640 1 -- 172.21.15.19:0/210779188 >> 172.21.15.19:0/210779188 conn(0x7fbbb80d0800 msgr2=0x7fbbb80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.660+0000 7fbbbdce1640 1 -- 172.21.15.19:0/210779188 shutdown_connections 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.660+0000 7fbbbdce1640 1 -- 172.21.15.19:0/210779188 wait complete. 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.661+0000 7fbbbdce1640 1 Processor -- start 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.661+0000 7fbbbdce1640 1 -- start start 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbbdce1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbb806c9e0 0x7fbbb81f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbbdce1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb80b5620 0x7fbbb80ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbbdce1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbb815b7d0 0x7fbbb80d2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbbdce1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80cf6d0 con 0x7fbbb815b7d0 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbbdce1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80cf550 con 0x7fbbb80b5620 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbbdce1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb815bbb0 con 0x7fbbb806c9e0 2024-04-25T09:56:05.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbb815b7d0 0x7fbbb80d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbb815b7d0 0x7fbbb80d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47090/0 (socket says 172.21.15.19:47090) 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbb7fff640 1 -- 172.21.15.19:0/3660803049 learned_addr learned my addr 172.21.15.19:0/3660803049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb80b5620 0x7fbbb80ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbb806c9e0 0x7fbbb81f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbb7fff640 1 -- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbb806c9e0 msgr2=0x7fbbb81f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbb7fff640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbb806c9e0 0x7fbbb81f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbb7fff640 1 -- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb80b5620 msgr2=0x7fbbb80ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbb7fff640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb80b5620 0x7fbbb80ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbb7fff640 1 -- 172.21.15.19:0/3660803049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb80cdf70 con 0x7fbbb815b7d0 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.662+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbb806c9e0 0x7fbbb81f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.663+0000 7fbbb7fff640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbb815b7d0 0x7fbbb80d2290 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ce70 tx=0x7fbbb000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.663+0000 7fbb9ffff640 1 -- 172.21.15.19:0/3660803049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb001a070 con 0x7fbbb815b7d0 2024-04-25T09:56:05.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.663+0000 7fbb9ffff640 1 -- 172.21.15.19:0/3660803049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb000d840 con 0x7fbbb815b7d0 2024-04-25T09:56:05.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.663+0000 7fbbbdce1640 1 -- 172.21.15.19:0/3660803049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb80cd9b0 con 0x7fbbb815b7d0 2024-04-25T09:56:05.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.663+0000 7fbb9ffff640 1 -- 172.21.15.19:0/3660803049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000dc30 con 0x7fbbb815b7d0 2024-04-25T09:56:05.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.663+0000 7fbbbdce1640 1 -- 172.21.15.19:0/3660803049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb80c7770 con 0x7fbbb815b7d0 2024-04-25T09:56:05.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.665+0000 7fbb9ffff640 1 -- 172.21.15.19:0/3660803049 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbb001d070 con 0x7fbbb815b7d0 2024-04-25T09:56:05.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.665+0000 7fbb9ffff640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbba406b4d0 0x7fbba406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.665+0000 7fbb9ffff640 1 -- 172.21.15.19:0/3660803049 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbbb008aab0 con 0x7fbbb815b7d0 2024-04-25T09:56:05.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.665+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbba406b4d0 0x7fbba406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.666+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbba406b4d0 0x7fbba406d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbba8034000 tx=0x7fbba802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.666+0000 7fbbc09e4740 1 Processor -- start 2024-04-25T09:56:05.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.666+0000 7fbbc09e4740 1 -- start start 2024-04-25T09:56:05.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d73aaa0 0x559a6d73d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d73d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d70d700 0x559a6d710f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6d676e00 con 0x559a6d73ae80 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6d61fd20 con 0x559a6d73aaa0 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a6d672ea0 con 0x559a6d70d700 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d73d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d73d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47106/0 (socket says 172.21.15.19:47106) 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/2122711020 learned_addr learned my addr 172.21.15.19:0/2122711020 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/2122711020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d73aaa0 0x559a6d73d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbb7fff640 1 --2- 172.21.15.19:0/2122711020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d70d700 0x559a6d710f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/2122711020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d70d700 msgr2=0x559a6d710f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/2122711020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d70d700 0x559a6d710f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/2122711020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d73aaa0 msgr2=0x559a6d73d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/2122711020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d73aaa0 0x559a6d73d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/2122711020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a6d641df0 con 0x559a6d73ae80 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/2122711020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d73d7f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0076a0 tx=0x7fbbac004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2e4c2d565dac36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbb9d7fa640 1 -- 172.21.15.19:0/2122711020 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac01a070 con 0x559a6d73ae80 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.667+0000 7fbb9d7fa640 1 -- 172.21.15.19:0/2122711020 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac0017b0 con 0x559a6d73ae80 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.668+0000 7fbb9d7fa640 1 -- 172.21.15.19:0/2122711020 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac001ba0 con 0x559a6d73ae80 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.668+0000 7fbbc09e4740 1 -- 172.21.15.19:0/2122711020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 msgr2=0x559a6d73d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.668+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/2122711020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d73d7f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0076a0 tx=0x7fbbac004bb0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.668+0000 7fbbc09e4740 1 -- 172.21.15.19:0/2122711020 shutdown_connections 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.668+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/2122711020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d70d700 0x559a6d710f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.668+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/2122711020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d73d7f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.668+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/2122711020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d73aaa0 0x559a6d73d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.668+0000 7fbbc09e4740 1 -- 172.21.15.19:0/2122711020 >> 172.21.15.19:0/2122711020 conn(0x559a6d70f970 msgr2=0x559a6d7582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.668+0000 7fbbc09e4740 1 -- 172.21.15.19:0/2122711020 shutdown_connections 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.668+0000 7fbbc09e4740 1 -- 172.21.15.19:0/2122711020 wait complete. 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.669+0000 7fbbc09e4740 1 -- start start 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.669+0000 7fbbc09e4740 1 Processor -- start 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d73aaa0 0x559a6d731d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d734170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d7346b0 0x559a6d75ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6d712b20 con 0x559a6d73ae80 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6d615590 con 0x559a6d7346b0 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a6d712e00 con 0x559a6d73aaa0 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbc09e4740 10 client.0 _ll_register_callbacks cb 0x559a6d621e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:05.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d734170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d734170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47114/0 (socket says 172.21.15.19:47114) 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/3469416838 learned_addr learned my addr 172.21.15.19:0/3469416838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d73aaa0 0x559a6d731d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb7fff640 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d7346b0 0x559a6d75ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/3469416838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d73aaa0 msgr2=0x559a6d731d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d73aaa0 0x559a6d731d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/3469416838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d7346b0 msgr2=0x559a6d75ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d7346b0 0x559a6d75ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/3469416838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a6d38b3f0 con 0x559a6d73ae80 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb7fff640 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d7346b0 0x559a6d75ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d734170 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x559a6d715000 tx=0x7fbbac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a52efeded1388 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbb967fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbb967fc640 1 -- 172.21.15.19:0/3469416838 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac01a030 con 0x559a6d73ae80 2024-04-25T09:56:05.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbb967fc640 1 -- 172.21.15.19:0/3469416838 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbbac0159b0 con 0x559a6d73ae80 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3469416838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbac0017b0 con 0x559a6d73ae80 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbb967fc640 1 -- 172.21.15.19:0/3469416838 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbac02e050 con 0x559a6d73ae80 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbc09e4740 10 client.6021 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.670+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3469416838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x559a6d4557d0 con 0x559a6d73ae80 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.671+0000 7fbb967fc640 1 -- 172.21.15.19:0/3469416838 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac0017b0 con 0x559a6d73ae80 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.671+0000 7fbb967fc640 1 -- 172.21.15.19:0/3469416838 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbbac031020 con 0x559a6d73ae80 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.671+0000 7fbbc09e4740 10 client.6021 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.671+0000 7fbbc09e4740 10 client.6021 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.671+0000 7fbbc09e4740 20 client.6021 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.671+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x559a6d753ed0 0x559a6d7542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.671+0000 7fbbc09e4740 4 client.6021 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.671+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3469416838 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a6d684340 con 0x559a6d753ed0 2024-04-25T09:56:05.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.671+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x559a6d753ed0 0x559a6d7542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.671+0000 7fbb967fc640 10 client.6021 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.671+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x559a6d753ed0 0x559a6d7542b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e001b0239db7d3e server_cookie=4b6a561736cbfb9a in_seq=0 out_seq=0 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbb967fc640 1 -- 172.21.15.19:0/3469416838 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559a6d684340 con 0x559a6d753ed0 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbb967fc640 10 client.6021 handle_command_reply: tid=0 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 1 client.6021 shutdown 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3469416838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 msgr2=0x559a6d734170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d734170 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x559a6d715000 tx=0x7fbbac012000 comp rx=0 tx=0).stop 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3469416838 shutdown_connections 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x559a6d753ed0 0x559a6d7542b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d7346b0 0x559a6d75ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73ae80 0x559a6d734170 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/3469416838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d73aaa0 0x559a6d731d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbb967fc640 0 client.6021 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3469416838 >> 172.21.15.19:0/3469416838 conn(0x559a6d70d700 msgr2=0x559a6d739120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3469416838 shutdown_connections 2024-04-25T09:56:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3469416838 wait complete. 2024-04-25T09:56:05.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.672+0000 7fbbc09e4740 20 client.6021 trim_cache size 0 max 16384 2024-04-25T09:56:05.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.684+0000 7fbbc09e4740 1 Processor -- start 2024-04-25T09:56:05.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.684+0000 7fbbc09e4740 1 -- start start 2024-04-25T09:56:05.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d7346b0 0x559a6d7519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d70caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d73ae80 0x559a6d741280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8200200 con 0x559a6d73aaa0 2024-04-25T09:56:05.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81ff3c0 con 0x559a6d7346b0 2024-04-25T09:56:05.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80c9250 con 0x559a6d73ae80 2024-04-25T09:56:05.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d70caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d70caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47120/0 (socket says 172.21.15.19:47120) 2024-04-25T09:56:05.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/3615164405 learned_addr learned my addr 172.21.15.19:0/3615164405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbb7fff640 1 --2- 172.21.15.19:0/3615164405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d73ae80 0x559a6d741280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/3615164405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d7346b0 0x559a6d7519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/3615164405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d73ae80 msgr2=0x559a6d741280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/3615164405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d73ae80 0x559a6d741280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/3615164405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d7346b0 msgr2=0x559a6d7519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/3615164405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d7346b0 0x559a6d7519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/3615164405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a6d7326c0 con 0x559a6d73aaa0 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.685+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/3615164405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d70caf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0011e0 tx=0x7fbbac009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86254cabdd064137 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbb9cff9640 1 -- 172.21.15.19:0/3615164405 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac036020 con 0x559a6d73aaa0 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbb9cff9640 1 -- 172.21.15.19:0/3615164405 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac01b040 con 0x559a6d73aaa0 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbb9cff9640 1 -- 172.21.15.19:0/3615164405 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac01a070 con 0x559a6d73aaa0 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3615164405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 msgr2=0x559a6d70caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/3615164405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d70caf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0011e0 tx=0x7fbbac009000 comp rx=0 tx=0).stop 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3615164405 shutdown_connections 2024-04-25T09:56:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/3615164405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d73ae80 0x559a6d741280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/3615164405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d70caf0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/3615164405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d7346b0 0x559a6d7519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3615164405 >> 172.21.15.19:0/3615164405 conn(0x559a6d70d700 msgr2=0x559a6d75b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3615164405 shutdown_connections 2024-04-25T09:56:05.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.686+0000 7fbbc09e4740 1 -- 172.21.15.19:0/3615164405 wait complete. 2024-04-25T09:56:05.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.687+0000 7fbbc09e4740 1 -- start start 2024-04-25T09:56:05.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.687+0000 7fbbc09e4740 1 Processor -- start 2024-04-25T09:56:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d7346b0 0x559a6d771e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d7723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbc09e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d73ae80 0x559a6d7c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6d732350 con 0x559a6d73aaa0 2024-04-25T09:56:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81fedc0 con 0x559a6d73ae80 2024-04-25T09:56:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbc09e4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a6d731b60 con 0x559a6d7346b0 2024-04-25T09:56:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbc09e4740 10 client.0 _ll_register_callbacks cb 0x7fbbac0070c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d7723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d7723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47128/0 (socket says 172.21.15.19:47128) 2024-04-25T09:56:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/1712226563 learned_addr learned my addr 172.21.15.19:0/1712226563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d7346b0 0x559a6d771e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbb7fff640 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d73ae80 0x559a6d7c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/1712226563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d7346b0 msgr2=0x559a6d771e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d7346b0 0x559a6d771e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/1712226563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d73ae80 msgr2=0x559a6d7c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d73ae80 0x559a6d7c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbb6ffd640 1 -- 172.21.15.19:0/1712226563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a6d7c3890 con 0x559a6d73aaa0 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbb6ffd640 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d7723c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0076a0 tx=0x7fbbac000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e95e3596ad5250 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbb967fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbb967fc640 1 -- 172.21.15.19:0/1712226563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac01a070 con 0x559a6d73aaa0 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbb967fc640 1 -- 172.21.15.19:0/1712226563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbbac021c10 con 0x559a6d73aaa0 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.688+0000 7fbbc09e4740 1 -- 172.21.15.19:0/1712226563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbac01b040 con 0x559a6d73aaa0 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbb967fc640 1 -- 172.21.15.19:0/1712226563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbac049050 con 0x559a6d73aaa0 2024-04-25T09:56:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbbc09e4740 10 client.6027 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbbc09e4740 1 -- 172.21.15.19:0/1712226563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x559a6d767010 con 0x559a6d73aaa0 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbb967fc640 1 -- 172.21.15.19:0/1712226563 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac01b040 con 0x559a6d73aaa0 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbb967fc640 1 -- 172.21.15.19:0/1712226563 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbbac031070 con 0x559a6d73aaa0 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbbc09e4740 10 client.6027 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbbc09e4740 10 client.6027 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbbc09e4740 20 client.6027 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x559a6d7c0130 0x559a6d7c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbbc09e4740 4 client.6027 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbbc09e4740 1 -- 172.21.15.19:0/1712226563 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a6d7332d0 con 0x559a6d7c0130 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x559a6d7c0130 0x559a6d7c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbb967fc640 10 client.6027 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbbb77fe640 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x559a6d7c0130 0x559a6d7c64c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc6ff139c0fc596 server_cookie=c3d6bf9b80234ed2 in_seq=0 out_seq=0 2024-04-25T09:56:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbb967fc640 1 -- 172.21.15.19:0/1712226563 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a6d7332d0 con 0x559a6d7c0130 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbb967fc640 10 client.6027 handle_command_reply: tid=0 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.689+0000 7fbbc09e4740 1 client.6027 shutdown 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbc09e4740 1 -- 172.21.15.19:0/1712226563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 msgr2=0x559a6d7723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d7723c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0076a0 tx=0x7fbbac000ed0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbc09e4740 1 -- 172.21.15.19:0/1712226563 shutdown_connections 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x559a6d7c0130 0x559a6d7c64c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a6d73ae80 0x559a6d7c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a6d73aaa0 0x559a6d7723c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbc09e4740 1 --2- 172.21.15.19:0/1712226563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559a6d7346b0 0x559a6d771e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbb967fc640 0 client.6027 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbc09e4740 1 -- 172.21.15.19:0/1712226563 >> 172.21.15.19:0/1712226563 conn(0x559a6d70d700 msgr2=0x559a6d750f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbc09e4740 1 -- 172.21.15.19:0/1712226563 shutdown_connections 2024-04-25T09:56:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbc09e4740 1 -- 172.21.15.19:0/1712226563 wait complete. 2024-04-25T09:56:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbc09e4740 20 client.6027 trim_cache size 0 max 16384 2024-04-25T09:56:05.701 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T09:56:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 -- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbba406b4d0 msgr2=0x7fbba406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbba406b4d0 0x7fbba406d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbba8034000 tx=0x7fbba802e000 comp rx=0 tx=0).stop 2024-04-25T09:56:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 -- 172.21.15.19:0/3660803049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbb815b7d0 msgr2=0x7fbbb80d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbb815b7d0 0x7fbbb80d2290 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ce70 tx=0x7fbbb000a710 comp rx=0 tx=0).stop 2024-04-25T09:56:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 -- 172.21.15.19:0/3660803049 shutdown_connections 2024-04-25T09:56:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbba406b4d0 0x7fbba406d990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbb815b7d0 0x7fbbb80d2290 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb80b5620 0x7fbbb80ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 --2- 172.21.15.19:0/3660803049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbb806c9e0 0x7fbbb81f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 -- 172.21.15.19:0/3660803049 >> 172.21.15.19:0/3660803049 conn(0x7fbbb80d0800 msgr2=0x7fbbb815d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 -- 172.21.15.19:0/3660803049 shutdown_connections 2024-04-25T09:56:05.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.690+0000 7fbbbdce1640 1 -- 172.21.15.19:0/3660803049 wait complete. 2024-04-25T09:56:05.705 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096a.00000000 parent 2024-04-25T09:56:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.743+0000 7fcbbd766f40 1 Processor -- start 2024-04-25T09:56:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.743+0000 7fcbbd766f40 1 -- start start 2024-04-25T09:56:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbd766f40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f82ea6dde0 0x55f82eab8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbd766f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb470a0 0x55f82eb49b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbd766f40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f82eb464b0 0x55f82eb4a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbd766f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82ea709a0 con 0x55f82eb470a0 2024-04-25T09:56:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbd766f40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82ea16df0 con 0x55f82eb464b0 2024-04-25T09:56:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbd766f40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f82ea6e1f0 con 0x55f82ea6dde0 2024-04-25T09:56:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbaa68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb470a0 0x55f82eb49b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbaa68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb470a0 0x55f82eb49b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47144/0 (socket says 172.21.15.19:47144) 2024-04-25T09:56:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbaa68640 1 -- 172.21.15.19:0/4261890529 learned_addr learned my addr 172.21.15.19:0/4261890529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbaa68640 1 -- 172.21.15.19:0/4261890529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f82ea6dde0 msgr2=0x55f82eab8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbaa68640 1 --2- 172.21.15.19:0/4261890529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f82ea6dde0 0x55f82eab8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbaa68640 1 -- 172.21.15.19:0/4261890529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f82eb464b0 msgr2=0x55f82eb4a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbaa68640 1 --2- 172.21.15.19:0/4261890529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f82eb464b0 0x55f82eb4a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.744+0000 7fcbbaa68640 1 -- 172.21.15.19:0/4261890529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f82e99cc50 con 0x55f82eb470a0 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbbaa68640 1 --2- 172.21.15.19:0/4261890529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb470a0 0x55f82eb49b30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcba400a770 tx=0x7fcba400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d1a623f49fd41c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbb9a66640 1 -- 172.21.15.19:0/4261890529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4018070 con 0x55f82eb470a0 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbb9a66640 1 -- 172.21.15.19:0/4261890529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba4007600 con 0x55f82eb470a0 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbb9a66640 1 -- 172.21.15.19:0/4261890529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba40079b0 con 0x55f82eb470a0 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbbd766f40 1 -- 172.21.15.19:0/4261890529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb470a0 msgr2=0x55f82eb49b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbbd766f40 1 --2- 172.21.15.19:0/4261890529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb470a0 0x55f82eb49b30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcba400a770 tx=0x7fcba400a860 comp rx=0 tx=0).stop 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbbd766f40 1 -- 172.21.15.19:0/4261890529 shutdown_connections 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbbd766f40 1 --2- 172.21.15.19:0/4261890529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f82eb464b0 0x55f82eb4a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbbd766f40 1 --2- 172.21.15.19:0/4261890529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb470a0 0x55f82eb49b30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbbd766f40 1 --2- 172.21.15.19:0/4261890529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f82ea6dde0 0x55f82eab8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbbd766f40 1 -- 172.21.15.19:0/4261890529 >> 172.21.15.19:0/4261890529 conn(0x55f82ea6e9b0 msgr2=0x55f82eab3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbbd766f40 1 -- 172.21.15.19:0/4261890529 shutdown_connections 2024-04-25T09:56:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.745+0000 7fcbbd766f40 1 -- 172.21.15.19:0/4261890529 wait complete. 2024-04-25T09:56:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.746+0000 7fcbbd766f40 1 Processor -- start 2024-04-25T09:56:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.746+0000 7fcbbd766f40 1 -- start start 2024-04-25T09:56:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbd766f40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f82ea6dde0 0x55f82eab4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbd766f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb464b0 0x55f82eb49ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbd766f40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f82eb470a0 0x55f82eb43430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbd766f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82eb42c40 con 0x55f82eb464b0 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbd766f40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82eb429f0 con 0x55f82eb470a0 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbd766f40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f82eb3b790 con 0x55f82ea6dde0 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbaa68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb464b0 0x55f82eb49ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbaa68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb464b0 0x55f82eb49ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47152/0 (socket says 172.21.15.19:47152) 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbaa68640 1 -- 172.21.15.19:0/2700979823 learned_addr learned my addr 172.21.15.19:0/2700979823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbaa68640 1 -- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f82ea6dde0 msgr2=0x55f82eab4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbba6a640 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f82eb470a0 0x55f82eb43430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbb269640 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f82ea6dde0 0x55f82eab4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbaa68640 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f82ea6dde0 0x55f82eab4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbaa68640 1 -- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f82eb470a0 msgr2=0x55f82eb43430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbaa68640 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f82eb470a0 0x55f82eb43430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbaa68640 1 -- 172.21.15.19:0/2700979823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f82eb3b910 con 0x55f82eb464b0 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbaa68640 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb464b0 0x55f82eb49ce0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcba4016040 tx=0x7fcba4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbb8a64640 1 -- 172.21.15.19:0/2700979823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4022020 con 0x55f82eb464b0 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbb8a64640 1 -- 172.21.15.19:0/2700979823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba401e690 con 0x55f82eb464b0 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbbd766f40 1 -- 172.21.15.19:0/2700979823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f82eb3e740 con 0x55f82eb464b0 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.747+0000 7fcbb8a64640 1 -- 172.21.15.19:0/2700979823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4018040 con 0x55f82eb464b0 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.748+0000 7fcbbd766f40 1 -- 172.21.15.19:0/2700979823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f82eb42490 con 0x55f82eb464b0 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.749+0000 7fcbb8a64640 1 -- 172.21.15.19:0/2700979823 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcba4025020 con 0x55f82eb464b0 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.749+0000 7fcbb8a64640 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb9c069ee0 0x7fcb9c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.749+0000 7fcbb8a64640 1 -- 172.21.15.19:0/2700979823 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcba408a1d0 con 0x55f82eb464b0 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.749+0000 7fcbbd766f40 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x55f82eb4a220 0x55f82eb43e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.749+0000 7fcbbd766f40 1 -- 172.21.15.19:0/2700979823 --> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] -- osd_op(unknown.0.0:1 8.11 8:88e5eea5:::1000000096a.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53) v9 -- 0x55f82eb44910 con 0x55f82eb4a220 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.749+0000 7fcbbb269640 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb9c069ee0 0x7fcb9c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.750+0000 7fcbbba6a640 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x55f82eb4a220 0x55f82eb43e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.750+0000 7fcbbb269640 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb9c069ee0 0x7fcb9c06c3a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4056000 tx=0x7fcbb404e070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.750+0000 7fcbbba6a640 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x55f82eb4a220 0x55f82eb43e80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbba6a640 1 -- 172.21.15.19:0/2700979823 <== osd.5 v2:172.21.15.112:6808/2948328480 1 ==== osd_op_reply(1 1000000096a.00000000 [rmxattr] v53'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcbac0016b0 con 0x55f82eb4a220 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbd766f40 1 -- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x55f82eb4a220 msgr2=0x55f82eb43e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbd766f40 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x55f82eb4a220 0x55f82eb43e80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbd766f40 1 -- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb9c069ee0 msgr2=0x7fcb9c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbd766f40 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb9c069ee0 0x7fcb9c06c3a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4056000 tx=0x7fcbb404e070 comp rx=0 tx=0).stop 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbd766f40 1 -- 172.21.15.19:0/2700979823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb464b0 msgr2=0x55f82eb49ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbd766f40 1 --2- 172.21.15.19:0/2700979823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f82eb464b0 0x55f82eb49ce0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcba4016040 tx=0x7fcba4003020 comp rx=0 tx=0).stop 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbba6a640 1 -- 172.21.15.19:0/2700979823 reap_dead start 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbd766f40 1 -- 172.21.15.19:0/2700979823 shutdown_connections 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbd766f40 1 -- 172.21.15.19:0/2700979823 >> 172.21.15.19:0/2700979823 conn(0x55f82ea6e9b0 msgr2=0x55f82eab3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbd766f40 1 -- 172.21.15.19:0/2700979823 shutdown_connections 2024-04-25T09:56:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.751+0000 7fcbbd766f40 1 -- 172.21.15.19:0/2700979823 wait complete. 2024-04-25T09:56:05.759 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T09:56:05.760 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:05.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T09:56:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.899+0000 7f78bdd5a640 1 Processor -- start 2024-04-25T09:56:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.899+0000 7f78bdd5a640 1 -- start start 2024-04-25T09:56:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78bdd5a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f78b815b7d0 0x7f78b80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78bdd5a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f78b806c9e0 0x7f78b80b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78bdd5a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b806cf50 con 0x7f78b80b5620 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78bdd5a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b80ce530 con 0x7f78b806c9e0 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78bdd5a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b80ce810 con 0x7f78b815b7d0 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78b7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f78b806c9e0 0x7f78b80b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78b6ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f78b80b5620 0x7f78b806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40624/0 (socket says 172.21.15.19:40624) 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78b6ffd640 1 -- 172.21.15.19:0/3232880194 learned_addr learned my addr 172.21.15.19:0/3232880194 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78b77fe640 1 --2- 172.21.15.19:0/3232880194 >> v2:172.21.15.112:3300/0 conn(0x7f78b815b7d0 0x7f78b80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78b7fff640 1 -- 172.21.15.19:0/3232880194 >> v2:172.21.15.112:3300/0 conn(0x7f78b815b7d0 msgr2=0x7f78b80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78b7fff640 1 --2- 172.21.15.19:0/3232880194 >> v2:172.21.15.112:3300/0 conn(0x7f78b815b7d0 0x7f78b80cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.900+0000 7f78b7fff640 1 -- 172.21.15.19:0/3232880194 >> v1:172.21.15.19:6789/0 conn(0x7f78b80b5620 legacy=0x7f78b806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.901+0000 7f78b7fff640 1 -- 172.21.15.19:0/3232880194 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b80ceaf0 con 0x7f78b806c9e0 2024-04-25T09:56:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.901+0000 7f78b67fc640 1 -- 172.21.15.19:0/3232880194 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465964918 0 0) 0x7f78b806cf50 con 0x7f78b80b5620 2024-04-25T09:56:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.901+0000 7f78b7fff640 1 --2- 172.21.15.19:0/3232880194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b806c9e0 0x7f78b80b5ab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f78b808c0d0 tx=0x7f78b002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed88eea5bbb77220 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.901+0000 7f78b67fc640 1 -- 172.21.15.19:0/3232880194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0007740 con 0x7f78b806c9e0 2024-04-25T09:56:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.901+0000 7f78b67fc640 1 -- 172.21.15.19:0/3232880194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b00078e0 con 0x7f78b806c9e0 2024-04-25T09:56:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.901+0000 7f78b67fc640 1 -- 172.21.15.19:0/3232880194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0007cb0 con 0x7f78b806c9e0 2024-04-25T09:56:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.902+0000 7f78bdd5a640 1 -- 172.21.15.19:0/3232880194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b806c9e0 msgr2=0x7f78b80b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.902+0000 7f78bdd5a640 1 --2- 172.21.15.19:0/3232880194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b806c9e0 0x7f78b80b5ab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f78b808c0d0 tx=0x7f78b002a680 comp rx=0 tx=0).stop 2024-04-25T09:56:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.902+0000 7f78bdd5a640 1 -- 172.21.15.19:0/3232880194 shutdown_connections 2024-04-25T09:56:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.902+0000 7f78bdd5a640 1 --2- 172.21.15.19:0/3232880194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b806c9e0 0x7f78b80b5ab0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.902+0000 7f78bdd5a640 1 --2- 172.21.15.19:0/3232880194 >> v2:172.21.15.112:3300/0 conn(0x7f78b815b7d0 0x7f78b80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.902+0000 7f78bdd5a640 1 -- 172.21.15.19:0/3232880194 >> 172.21.15.19:0/3232880194 conn(0x7f78b80d0800 msgr2=0x7f78b80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.902+0000 7f78bdd5a640 1 -- 172.21.15.19:0/3232880194 shutdown_connections 2024-04-25T09:56:05.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.902+0000 7f78bdd5a640 1 -- 172.21.15.19:0/3232880194 wait complete. 2024-04-25T09:56:05.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.903+0000 7f78bdd5a640 1 Processor -- start 2024-04-25T09:56:05.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.903+0000 7f78bdd5a640 1 -- start start 2024-04-25T09:56:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.904+0000 7f78bdd5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b806c9e0 0x7f78b8159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.904+0000 7f78bdd5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b80b5620 0x7f78b80c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.904+0000 7f78bdd5a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78b815b7d0 0x7f78b80b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.904+0000 7f78bdd5a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80cf6d0 con 0x7f78b806c9e0 2024-04-25T09:56:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.904+0000 7f78bdd5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80cf550 con 0x7f78b80b5620 2024-04-25T09:56:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.904+0000 7f78bdd5a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b815bbb0 con 0x7f78b815b7d0 2024-04-25T09:56:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.904+0000 7f78b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b806c9e0 0x7f78b8159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.904+0000 7f78b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b806c9e0 0x7f78b8159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47168/0 (socket says 172.21.15.19:47168) 2024-04-25T09:56:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.904+0000 7f78b77fe640 1 -- 172.21.15.19:0/726913678 learned_addr learned my addr 172.21.15.19:0/726913678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b80b5620 0x7f78b80c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f78b77fe640 1 -- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78b815b7d0 msgr2=0x7f78b80b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f78b7fff640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78b815b7d0 0x7f78b80b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f78b77fe640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78b815b7d0 0x7f78b80b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f78b77fe640 1 -- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b80b5620 msgr2=0x7f78b80c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f78b77fe640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b80b5620 0x7f78b80c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f78b77fe640 1 -- 172.21.15.19:0/726913678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b8159860 con 0x7f78b806c9e0 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b80b5620 0x7f78b80c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f78b77fe640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b806c9e0 0x7f78b8159320 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f78a800d5f0 tx=0x7f78a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f789ffff640 1 -- 172.21.15.19:0/726913678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a80125f0 con 0x7f78b806c9e0 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f789ffff640 1 -- 172.21.15.19:0/726913678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a8019070 con 0x7f78b806c9e0 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f789ffff640 1 -- 172.21.15.19:0/726913678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a80129d0 con 0x7f78b806c9e0 2024-04-25T09:56:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f78bdd5a640 1 -- 172.21.15.19:0/726913678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b80c8010 con 0x7f78b806c9e0 2024-04-25T09:56:05.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.905+0000 7f78bdd5a640 1 -- 172.21.15.19:0/726913678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b80c8af0 con 0x7f78b806c9e0 2024-04-25T09:56:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.906+0000 7f789ffff640 1 -- 172.21.15.19:0/726913678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78a8012b70 con 0x7f78b806c9e0 2024-04-25T09:56:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.907+0000 7f789ffff640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78a406b4d0 0x7f78a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.907+0000 7f789ffff640 1 -- 172.21.15.19:0/726913678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78a808a750 con 0x7f78b806c9e0 2024-04-25T09:56:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.907+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78a406b4d0 0x7f78a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.908+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78a406b4d0 0x7f78a406d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f78b81f6bb0 tx=0x7f78ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.908+0000 7f78c0a58740 1 Processor -- start 2024-04-25T09:56:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.908+0000 7f78c0a58740 1 -- start start 2024-04-25T09:56:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95bd360 0x5596b95e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95e6140 0x5596b95bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78c0a58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b951eee0 con 0x5596b95de360 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78c0a58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b94c7c50 con 0x5596b95bd360 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78c0a58740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596b951af80 con 0x5596b95e6140 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95fe7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47178/0 (socket says 172.21.15.19:47178) 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78b77fe640 1 -- 172.21.15.19:0/3916922602 learned_addr learned my addr 172.21.15.19:0/3916922602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78b7fff640 1 --2- 172.21.15.19:0/3916922602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95e6140 0x5596b95bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/3916922602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95bd360 0x5596b95e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78b77fe640 1 -- 172.21.15.19:0/3916922602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95e6140 msgr2=0x5596b95bb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78b77fe640 1 --2- 172.21.15.19:0/3916922602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95e6140 0x5596b95bb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78b77fe640 1 -- 172.21.15.19:0/3916922602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95bd360 msgr2=0x5596b95e5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78b77fe640 1 --2- 172.21.15.19:0/3916922602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95bd360 0x5596b95e5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78b77fe640 1 -- 172.21.15.19:0/3916922602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596b92ffc00 con 0x5596b95de360 2024-04-25T09:56:05.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f78b77fe640 1 --2- 172.21.15.19:0/3916922602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95fe7d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f78a8060f20 tx=0x7f78a8059580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da945b13199607b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f789d7fa640 1 -- 172.21.15.19:0/3916922602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8070020 con 0x5596b95de360 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f789d7fa640 1 -- 172.21.15.19:0/3916922602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a8061450 con 0x5596b95de360 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.909+0000 7f789d7fa640 1 -- 172.21.15.19:0/3916922602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a805f070 con 0x5596b95de360 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.910+0000 7f78c0a58740 1 -- 172.21.15.19:0/3916922602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 msgr2=0x5596b95fe7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.910+0000 7f78c0a58740 1 --2- 172.21.15.19:0/3916922602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95fe7d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f78a8060f20 tx=0x7f78a8059580 comp rx=0 tx=0).stop 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.910+0000 7f78c0a58740 1 -- 172.21.15.19:0/3916922602 shutdown_connections 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.910+0000 7f78c0a58740 1 --2- 172.21.15.19:0/3916922602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95e6140 0x5596b95bb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.910+0000 7f78c0a58740 1 --2- 172.21.15.19:0/3916922602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95bd360 0x5596b95e5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.910+0000 7f78c0a58740 1 --2- 172.21.15.19:0/3916922602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95fe7d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.910+0000 7f78c0a58740 1 -- 172.21.15.19:0/3916922602 >> 172.21.15.19:0/3916922602 conn(0x5596b9582810 msgr2=0x5596b9600c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.910+0000 7f78c0a58740 1 -- 172.21.15.19:0/3916922602 shutdown_connections 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.910+0000 7f78c0a58740 1 -- 172.21.15.19:0/3916922602 wait complete. 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.911+0000 7f78c0a58740 1 -- start start 2024-04-25T09:56:05.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.911+0000 7f78c0a58740 1 Processor -- start 2024-04-25T09:56:05.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 0x5596b95dd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95e6140 0x5596b9610c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78c0a58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b95bb330 con 0x5596b95de360 2024-04-25T09:56:05.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78c0a58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b94bd340 con 0x5596b95e6140 2024-04-25T09:56:05.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78c0a58740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596b95b61a0 con 0x5596b95bd360 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78c0a58740 10 client.0 _ll_register_callbacks cb 0x5596b9232fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95f9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47180/0 (socket says 172.21.15.19:47180) 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b6ffd640 1 -- 172.21.15.19:0/2927316722 learned_addr learned my addr 172.21.15.19:0/2927316722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b77fe640 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 0x5596b95dd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b7fff640 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95e6140 0x5596b9610c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b6ffd640 1 -- 172.21.15.19:0/2927316722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 msgr2=0x5596b95dd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 0x5596b95dd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b6ffd640 1 -- 172.21.15.19:0/2927316722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95e6140 msgr2=0x5596b9610c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95e6140 0x5596b9610c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b6ffd640 1 -- 172.21.15.19:0/2927316722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596b91f34f0 con 0x5596b95de360 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b77fe640 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 0x5596b95dd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:05.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b7fff640 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95e6140 0x5596b9610c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95f9de0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000f60 tx=0x7f78ac001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdcf01626bab254 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78967fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78967fc640 1 -- 172.21.15.19:0/2927316722 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac026060 con 0x5596b95de360 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78967fc640 1 -- 172.21.15.19:0/2927316722 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78ac021c30 con 0x5596b95de360 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78c0a58740 1 -- 172.21.15.19:0/2927316722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78a8061450 con 0x5596b95de360 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78967fc640 1 -- 172.21.15.19:0/2927316722 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78ac021e30 con 0x5596b95de360 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78c0a58740 10 client.6054 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.912+0000 7f78c0a58740 1 -- 172.21.15.19:0/2927316722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5596b9360390 con 0x5596b95de360 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.913+0000 7f78967fc640 1 -- 172.21.15.19:0/2927316722 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a8061450 con 0x5596b95de360 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.913+0000 7f78967fc640 1 -- 172.21.15.19:0/2927316722 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f78ac012410 con 0x5596b95de360 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.913+0000 7f78c0a58740 10 client.6054 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.913+0000 7f78c0a58740 10 client.6054 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.913+0000 7f78c0a58740 20 client.6054 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:05.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.913+0000 7f78c0a58740 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5596b95f8260 0x5596b9601050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.913+0000 7f78c0a58740 4 client.6054 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.913+0000 7f78c0a58740 1 -- 172.21.15.19:0/2927316722 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596b952c420 con 0x5596b95f8260 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.913+0000 7f78b77fe640 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5596b95f8260 0x5596b9601050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.913+0000 7f78967fc640 10 client.6054 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.913+0000 7f78b77fe640 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5596b95f8260 0x5596b9601050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=496689e83036e6f3 server_cookie=497dda3e251021e7 in_seq=0 out_seq=0 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78967fc640 1 -- 172.21.15.19:0/2927316722 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5596b952c420 con 0x5596b95f8260 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78967fc640 10 client.6054 handle_command_reply: tid=0 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 1 client.6054 shutdown 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 1 -- 172.21.15.19:0/2927316722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 msgr2=0x5596b95f9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95f9de0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000f60 tx=0x7f78ac001080 comp rx=0 tx=0).stop 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 1 -- 172.21.15.19:0/2927316722 shutdown_connections 2024-04-25T09:56:05.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5596b95f8260 0x5596b9601050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95e6140 0x5596b9610c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95de360 0x5596b95f9de0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78967fc640 0 client.6054 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:05.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 1 --2- 172.21.15.19:0/2927316722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 0x5596b95dd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 1 -- 172.21.15.19:0/2927316722 >> 172.21.15.19:0/2927316722 conn(0x5596b9582810 msgr2=0x5596b95bbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 1 -- 172.21.15.19:0/2927316722 shutdown_connections 2024-04-25T09:56:05.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 1 -- 172.21.15.19:0/2927316722 wait complete. 2024-04-25T09:56:05.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.914+0000 7f78c0a58740 20 client.6054 trim_cache size 0 max 16384 2024-04-25T09:56:05.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.930+0000 7f78c0a58740 1 Processor -- start 2024-04-25T09:56:05.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.930+0000 7f78c0a58740 1 -- start start 2024-04-25T09:56:05.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 0x5596b9600f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95de360 0x5596b9601480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95e6140 0x5596b960f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78c0a58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b8200330 con 0x5596b95e6140 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78c0a58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b81ff510 con 0x5596b95de360 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78c0a58740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b81fe940 con 0x5596b95bd360 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95e6140 0x5596b960f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95e6140 0x5596b960f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47192/0 (socket says 172.21.15.19:47192) 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b7fff640 1 -- 172.21.15.19:0/2188024715 learned_addr learned my addr 172.21.15.19:0/2188024715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/2188024715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95de360 0x5596b9601480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b77fe640 1 --2- 172.21.15.19:0/2188024715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 0x5596b9600f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b7fff640 1 -- 172.21.15.19:0/2188024715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 msgr2=0x5596b9600f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b7fff640 1 --2- 172.21.15.19:0/2188024715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 0x5596b9600f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b7fff640 1 -- 172.21.15.19:0/2188024715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95de360 msgr2=0x5596b9601480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b7fff640 1 --2- 172.21.15.19:0/2188024715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95de360 0x5596b9601480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b7fff640 1 -- 172.21.15.19:0/2188024715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596b91cc670 con 0x5596b95e6140 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/2188024715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95de360 0x5596b9601480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b77fe640 1 --2- 172.21.15.19:0/2188024715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 0x5596b9600f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f78b7fff640 1 --2- 172.21.15.19:0/2188024715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95e6140 0x5596b960f300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f78b002ac70 tx=0x7f78b002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcd979886029682 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f789cff9640 1 -- 172.21.15.19:0/2188024715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0003d50 con 0x5596b95e6140 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.931+0000 7f789cff9640 1 -- 172.21.15.19:0/2188024715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b0004040 con 0x5596b95e6140 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.932+0000 7f789cff9640 1 -- 172.21.15.19:0/2188024715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0004410 con 0x5596b95e6140 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.932+0000 7f78c0a58740 1 -- 172.21.15.19:0/2188024715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95e6140 msgr2=0x5596b960f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.932+0000 7f78c0a58740 1 --2- 172.21.15.19:0/2188024715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95e6140 0x5596b960f300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f78b002ac70 tx=0x7f78b002e000 comp rx=0 tx=0).stop 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.932+0000 7f78c0a58740 1 -- 172.21.15.19:0/2188024715 shutdown_connections 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.932+0000 7f78c0a58740 1 --2- 172.21.15.19:0/2188024715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95e6140 0x5596b960f300 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.932+0000 7f78c0a58740 1 --2- 172.21.15.19:0/2188024715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95de360 0x5596b9601480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.932+0000 7f78c0a58740 1 --2- 172.21.15.19:0/2188024715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95bd360 0x5596b9600f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.932+0000 7f78c0a58740 1 -- 172.21.15.19:0/2188024715 >> 172.21.15.19:0/2188024715 conn(0x5596b9582810 msgr2=0x5596b95dd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.932+0000 7f78c0a58740 1 -- 172.21.15.19:0/2188024715 shutdown_connections 2024-04-25T09:56:05.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.932+0000 7f78c0a58740 1 -- 172.21.15.19:0/2188024715 wait complete. 2024-04-25T09:56:05.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.933+0000 7f78c0a58740 1 -- start start 2024-04-25T09:56:05.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.933+0000 7f78c0a58740 1 Processor -- start 2024-04-25T09:56:05.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.933+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95bd360 0x5596b9668310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.933+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95de360 0x5596b95b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.933+0000 7f78c0a58740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95e6140 0x5596b95e8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.933+0000 7f78c0a58740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b95b1090 con 0x5596b95bd360 2024-04-25T09:56:05.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.933+0000 7f78c0a58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b81fef40 con 0x5596b95de360 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.933+0000 7f78c0a58740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596b95f9d60 con 0x5596b95e6140 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78c0a58740 10 client.0 _ll_register_callbacks cb 0x7f78a8055060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95bd360 0x5596b9668310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95bd360 0x5596b9668310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47194/0 (socket says 172.21.15.19:47194) 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78b77fe640 1 -- 172.21.15.19:0/1033916278 learned_addr learned my addr 172.21.15.19:0/1033916278 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95de360 0x5596b95b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78b7fff640 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95e6140 0x5596b95e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78b77fe640 1 -- 172.21.15.19:0/1033916278 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95e6140 msgr2=0x5596b95e8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78b77fe640 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95e6140 0x5596b95e8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78b77fe640 1 -- 172.21.15.19:0/1033916278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95de360 msgr2=0x5596b95b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78b77fe640 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95de360 0x5596b95b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78b77fe640 1 -- 172.21.15.19:0/1033916278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596b9610020 con 0x5596b95bd360 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78b77fe640 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95bd360 0x5596b9668310 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f78a80551a0 tx=0x7f78a805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b5f3e4ba6d2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78967fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78967fc640 1 -- 172.21.15.19:0/1033916278 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8070040 con 0x5596b95bd360 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78967fc640 1 -- 172.21.15.19:0/1033916278 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78a805f070 con 0x5596b95bd360 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78c0a58740 1 -- 172.21.15.19:0/1033916278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78b0004040 con 0x5596b95bd360 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78967fc640 1 -- 172.21.15.19:0/1033916278 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78a807e050 con 0x5596b95bd360 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78c0a58740 10 client.6060 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78c0a58740 1 -- 172.21.15.19:0/1033916278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5596b95e3bb0 con 0x5596b95bd360 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.934+0000 7f78967fc640 1 -- 172.21.15.19:0/1033916278 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b0004040 con 0x5596b95bd360 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78967fc640 1 -- 172.21.15.19:0/1033916278 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f78a8081020 con 0x5596b95bd360 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78c0a58740 10 client.6060 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78c0a58740 10 client.6060 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78c0a58740 20 client.6060 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78c0a58740 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5596b95e8cb0 0x5596b95b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:05.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78c0a58740 4 client.6060 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78c0a58740 1 -- 172.21.15.19:0/1033916278 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x5596b9526770 con 0x5596b95e8cb0 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5596b95e8cb0 0x5596b95b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78967fc640 10 client.6060 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78b6ffd640 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5596b95e8cb0 0x5596b95b9b20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d031b955a3cb6b server_cookie=d598d91575482c4d in_seq=0 out_seq=0 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78967fc640 1 -- 172.21.15.19:0/1033916278 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5596b9526770 con 0x5596b95e8cb0 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78967fc640 10 client.6060 handle_command_reply: tid=0 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78c0a58740 1 client.6060 shutdown 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78c0a58740 1 -- 172.21.15.19:0/1033916278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95bd360 msgr2=0x5596b9668310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.935+0000 7f78c0a58740 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95bd360 0x5596b9668310 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f78a80551a0 tx=0x7f78a805b000 comp rx=0 tx=0).stop 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78c0a58740 1 -- 172.21.15.19:0/1033916278 shutdown_connections 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78c0a58740 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5596b95e8cb0 0x5596b95b9b20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78c0a58740 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5596b95e6140 0x5596b95e8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78c0a58740 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5596b95de360 0x5596b95b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78967fc640 0 client.6060 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78c0a58740 1 --2- 172.21.15.19:0/1033916278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596b95bd360 0x5596b9668310 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78c0a58740 1 -- 172.21.15.19:0/1033916278 >> 172.21.15.19:0/1033916278 conn(0x5596b9582810 msgr2=0x5596b9604180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78c0a58740 1 -- 172.21.15.19:0/1033916278 shutdown_connections 2024-04-25T09:56:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78c0a58740 1 -- 172.21.15.19:0/1033916278 wait complete. 2024-04-25T09:56:05.946 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f6cd2016640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096a [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v450 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5892=0-4194304@1} caps={5892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5892 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x561ba4f4cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1000000096a:[<0x10000000001/new_file_0 v450>,<0x10000000000/ceph-qa-suite v40>,<0x1/scrub_test_path v11>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T09:56:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78c0a58740 20 client.6060 trim_cache size 0 max 16384 2024-04-25T09:56:05.947 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:56:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "c25d4699-c428-47da-bbba-88bee86fc907", 2024-04-25T09:56:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:56:05.947 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:05.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 -- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78a406b4d0 msgr2=0x7f78a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78a406b4d0 0x7f78a406d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f78b81f6bb0 tx=0x7f78ac009040 comp rx=0 tx=0).stop 2024-04-25T09:56:05.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 -- 172.21.15.19:0/726913678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b806c9e0 msgr2=0x7f78b8159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:05.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b806c9e0 0x7f78b8159320 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f78a800d5f0 tx=0x7f78a800dab0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 -- 172.21.15.19:0/726913678 shutdown_connections 2024-04-25T09:56:05.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78a406b4d0 0x7f78a406d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78b815b7d0 0x7f78b80b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b80b5620 0x7f78b80c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 --2- 172.21.15.19:0/726913678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b806c9e0 0x7f78b8159320 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:05.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 -- 172.21.15.19:0/726913678 >> 172.21.15.19:0/726913678 conn(0x7f78b80d0800 msgr2=0x7f78b8160660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:05.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 -- 172.21.15.19:0/726913678 shutdown_connections 2024-04-25T09:56:05.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:05.936+0000 7f78bdd5a640 1 -- 172.21.15.19:0/726913678 wait complete. 2024-04-25T09:56:05.951 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': 'c25d4699-c428-47da-bbba-88bee86fc907', 'mode': 'asynchronous'}' 2024-04-25T09:56:05.951 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:56:05.951 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:56:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.050+0000 7f8b9dff4640 1 Processor -- start 2024-04-25T09:56:06.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.050+0000 7f8b9dff4640 1 -- start start 2024-04-25T09:56:06.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b9dff4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b9815dc20 0x7f8b980ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b9dff4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b98161710 0x7f8b980cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b9dff4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b9815e000 con 0x7f8b980b9330 2024-04-25T09:56:06.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b9dff4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b980d1de0 con 0x7f8b9815dc20 2024-04-25T09:56:06.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b9dff4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b98161c60 con 0x7f8b98161710 2024-04-25T09:56:06.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b977fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b9815dc20 0x7f8b980ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b977fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b9815dc20 0x7f8b980ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47204/0 (socket says 172.21.15.19:47204) 2024-04-25T09:56:06.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b977fe640 1 -- 172.21.15.19:0/970448726 learned_addr learned my addr 172.21.15.19:0/970448726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:06.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/970448726 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b98161710 0x7f8b980cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b977fe640 1 -- 172.21.15.19:0/970448726 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b98161710 msgr2=0x7f8b980cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:06.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b977fe640 1 --2- 172.21.15.19:0/970448726 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b98161710 0x7f8b980cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b977fe640 1 -- 172.21.15.19:0/970448726 >> v1:172.21.15.112:6789/0 conn(0x7f8b980b9330 legacy=0x7f8b9806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:06.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.051+0000 7f8b977fe640 1 -- 172.21.15.19:0/970448726 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b98161f40 con 0x7f8b9815dc20 2024-04-25T09:56:06.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.052+0000 7f8b977fe640 1 --2- 172.21.15.19:0/970448726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b9815dc20 0x7f8b980ce670 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009870 tx=0x7f8b8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffc1912ffb70c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:06.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.052+0000 7f8b967fc640 1 -- 172.21.15.19:0/970448726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88037070 con 0x7f8b9815dc20 2024-04-25T09:56:06.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.052+0000 7f8b967fc640 1 -- 172.21.15.19:0/970448726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b88032440 con 0x7f8b9815dc20 2024-04-25T09:56:06.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.052+0000 7f8b967fc640 1 -- 172.21.15.19:0/970448726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88032810 con 0x7f8b9815dc20 2024-04-25T09:56:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.053+0000 7f8b9dff4640 1 -- 172.21.15.19:0/970448726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b9815dc20 msgr2=0x7f8b980ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.053+0000 7f8b9dff4640 1 --2- 172.21.15.19:0/970448726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b9815dc20 0x7f8b980ce670 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009870 tx=0x7f8b8802a270 comp rx=0 tx=0).stop 2024-04-25T09:56:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.053+0000 7f8b9dff4640 1 -- 172.21.15.19:0/970448726 shutdown_connections 2024-04-25T09:56:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.053+0000 7f8b9dff4640 1 --2- 172.21.15.19:0/970448726 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b98161710 0x7f8b980cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.053+0000 7f8b9dff4640 1 --2- 172.21.15.19:0/970448726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b9815dc20 0x7f8b980ce670 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.053+0000 7f8b9dff4640 1 -- 172.21.15.19:0/970448726 >> 172.21.15.19:0/970448726 conn(0x7f8b980d0420 msgr2=0x7f8b980d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.053+0000 7f8b9dff4640 1 -- 172.21.15.19:0/970448726 shutdown_connections 2024-04-25T09:56:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.053+0000 7f8b9dff4640 1 -- 172.21.15.19:0/970448726 wait complete. 2024-04-25T09:56:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.054+0000 7f8b9dff4640 1 Processor -- start 2024-04-25T09:56:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.054+0000 7f8b9dff4640 1 -- start start 2024-04-25T09:56:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b9dff4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b980b9330 0x7f8b98160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b9dff4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b9815dc20 0x7f8b9815a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b9dff4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b98161710 0x7f8b981576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b9dff4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980b5e50 con 0x7f8b980b9330 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b9dff4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b98162500 con 0x7f8b9815dc20 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b9dff4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980b5fd0 con 0x7f8b98161710 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b977fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b980b9330 0x7f8b98160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b977fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b980b9330 0x7f8b98160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47216/0 (socket says 172.21.15.19:47216) 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b977fe640 1 -- 172.21.15.19:0/4233401039 learned_addr learned my addr 172.21.15.19:0/4233401039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b9815dc20 0x7f8b9815a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b97fff640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b98161710 0x7f8b981576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b977fe640 1 -- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b98161710 msgr2=0x7f8b981576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b977fe640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b98161710 0x7f8b981576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b977fe640 1 -- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b9815dc20 msgr2=0x7f8b9815a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:06.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b977fe640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b9815dc20 0x7f8b9815a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b977fe640 1 -- 172.21.15.19:0/4233401039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9815f6b0 con 0x7f8b980b9330 2024-04-25T09:56:06.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b97fff640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b98161710 0x7f8b981576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:06.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b9815dc20 0x7f8b9815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:06.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.055+0000 7f8b977fe640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b980b9330 0x7f8b98160750 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8b8802a2b0 tx=0x7f8b880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:06.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.056+0000 7f8b7ffff640 1 -- 172.21.15.19:0/4233401039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88041020 con 0x7f8b980b9330 2024-04-25T09:56:06.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.056+0000 7f8b7ffff640 1 -- 172.21.15.19:0/4233401039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b880306c0 con 0x7f8b980b9330 2024-04-25T09:56:06.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.056+0000 7f8b9dff4640 1 -- 172.21.15.19:0/4233401039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9815f990 con 0x7f8b980b9330 2024-04-25T09:56:06.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.056+0000 7f8b7ffff640 1 -- 172.21.15.19:0/4233401039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88037040 con 0x7f8b980b9330 2024-04-25T09:56:06.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.056+0000 7f8b9dff4640 1 -- 172.21.15.19:0/4233401039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9815ff50 con 0x7f8b980b9330 2024-04-25T09:56:06.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.057+0000 7f8b7ffff640 1 -- 172.21.15.19:0/4233401039 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b88044020 con 0x7f8b980b9330 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.057+0000 7f8b7ffff640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b8406b4d0 0x7f8b8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.058+0000 7f8b7ffff640 1 -- 172.21.15.19:0/4233401039 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8b880aac30 con 0x7f8b980b9330 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.058+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b8406b4d0 0x7f8b8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.058+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b8406b4d0 0x7f8b8406d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c006fd0 tx=0x7f8b8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.058+0000 7f8ba0dcf740 1 Processor -- start 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.058+0000 7f8ba0dcf740 1 -- start start 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.059+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689e86b10 0x55d689e89320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.059+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689e89860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.059+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e59770 0x55d689e5cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.059+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d689dc2da0 con 0x55d689e86ef0 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.059+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d689d6bb80 con 0x55d689e59770 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.059+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d689dbee40 con 0x55d689e86b10 2024-04-25T09:56:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.059+0000 7f8b96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689e89860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.059+0000 7f8b96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689e89860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47228/0 (socket says 172.21.15.19:47228) 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.059+0000 7f8b96ffd640 1 -- 172.21.15.19:0/1415861063 learned_addr learned my addr 172.21.15.19:0/1415861063 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b977fe640 1 --2- 172.21.15.19:0/1415861063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689e86b10 0x55d689e89320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b97fff640 1 --2- 172.21.15.19:0/1415861063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e59770 0x55d689e5cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b96ffd640 1 -- 172.21.15.19:0/1415861063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689e86b10 msgr2=0x55d689e89320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/1415861063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689e86b10 0x55d689e89320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b96ffd640 1 -- 172.21.15.19:0/1415861063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e59770 msgr2=0x55d689e5cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/1415861063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e59770 0x55d689e5cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b96ffd640 1 -- 172.21.15.19:0/1415861063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d689d75f10 con 0x55d689e86ef0 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b977fe640 1 --2- 172.21.15.19:0/1415861063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689e86b10 0x55d689e89320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b97fff640 1 --2- 172.21.15.19:0/1415861063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e59770 0x55d689e5cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/1415861063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689e89860 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00fa00 tx=0x7f8b8c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b92935b39e4a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b7d7fa640 1 -- 172.21.15.19:0/1415861063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c01b050 con 0x55d689e86ef0 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b7d7fa640 1 -- 172.21.15.19:0/1415861063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c025070 con 0x55d689e86ef0 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.060+0000 7f8b7d7fa640 1 -- 172.21.15.19:0/1415861063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c014de0 con 0x55d689e86ef0 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.061+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/1415861063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 msgr2=0x55d689e89860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.061+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/1415861063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689e89860 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00fa00 tx=0x7f8b8c012000 comp rx=0 tx=0).stop 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.061+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/1415861063 shutdown_connections 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.061+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/1415861063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e59770 0x55d689e5cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.061+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/1415861063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689e89860 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.061+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/1415861063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689e86b10 0x55d689e89320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.061+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/1415861063 >> 172.21.15.19:0/1415861063 conn(0x55d689e5b9e0 msgr2=0x55d689ea4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.061+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/1415861063 shutdown_connections 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.061+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/1415861063 wait complete. 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.061+0000 7f8ba0dcf740 1 -- start start 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.061+0000 7f8ba0dcf740 1 Processor -- start 2024-04-25T09:56:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.062+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e86b10 0x55d689eab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.062+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689eab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.062+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689eabd30 0x55d689e7f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.062+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d689e5eb90 con 0x55d689e86ef0 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.062+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d689d614b0 con 0x55d689e86b10 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.062+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d689e5ee70 con 0x55d689eabd30 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.062+0000 7f8b96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689eab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.062+0000 7f8ba0dcf740 10 client.0 _ll_register_callbacks cb 0x55d689da1180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.062+0000 7f8b96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689eab7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47240/0 (socket says 172.21.15.19:47240) 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.062+0000 7f8b96ffd640 1 -- 172.21.15.19:0/2486539769 learned_addr learned my addr 172.21.15.19:0/2486539769 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.062+0000 7f8b977fe640 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e86b10 0x55d689eab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b97fff640 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689eabd30 0x55d689e7f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b96ffd640 1 -- 172.21.15.19:0/2486539769 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689eabd30 msgr2=0x55d689e7f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689eabd30 0x55d689e7f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b96ffd640 1 -- 172.21.15.19:0/2486539769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e86b10 msgr2=0x55d689eab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e86b10 0x55d689eab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b96ffd640 1 -- 172.21.15.19:0/2486539769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d689ba17d0 con 0x55d689e86ef0 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b97fff640 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689eabd30 0x55d689e7f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689eab7f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c014830 tx=0x7f8b8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146c04a22d98936d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b767fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b767fc640 1 -- 172.21.15.19:0/2486539769 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c020cb0 con 0x55d689e86ef0 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b767fc640 1 -- 172.21.15.19:0/2486539769 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8b8c034440 con 0x55d689e86ef0 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2486539769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b8c025070 con 0x55d689e86ef0 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b767fc640 1 -- 172.21.15.19:0/2486539769 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b8c03c050 con 0x55d689e86ef0 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8ba0dcf740 10 client.6072 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2486539769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55d689ad73f0 con 0x55d689e86ef0 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b767fc640 1 -- 172.21.15.19:0/2486539769 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c025070 con 0x55d689e86ef0 2024-04-25T09:56:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.063+0000 7f8b767fc640 1 -- 172.21.15.19:0/2486539769 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b8c03f020 con 0x55d689e86ef0 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8ba0dcf740 10 client.6072 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8ba0dcf740 10 client.6072 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8ba0dcf740 20 client.6072 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d689ea7af0 0x55d689ec1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8ba0dcf740 4 client.6072 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2486539769 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d689dd02e0 con 0x55d689ea7af0 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8b977fe640 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d689ea7af0 0x55d689ec1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8b767fc640 10 client.6072 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8b977fe640 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d689ea7af0 0x55d689ec1770 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59dba3f95e677564 server_cookie=44aedb0654a0aec8 in_seq=0 out_seq=0 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8b767fc640 1 -- 172.21.15.19:0/2486539769 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d689dd02e0 con 0x55d689ea7af0 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8b767fc640 10 client.6072 handle_command_reply: tid=0 2024-04-25T09:56:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.064+0000 7f8ba0dcf740 1 client.6072 shutdown 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2486539769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 msgr2=0x55d689eab7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689eab7f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c014830 tx=0x7f8b8c004730 comp rx=0 tx=0).stop 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2486539769 shutdown_connections 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d689ea7af0 0x55d689ec1770 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689eabd30 0x55d689e7f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689eab7f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8b767fc640 0 client.6072 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/2486539769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e86b10 0x55d689eab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2486539769 >> 172.21.15.19:0/2486539769 conn(0x55d689e59770 msgr2=0x55d689e81380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2486539769 shutdown_connections 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2486539769 wait complete. 2024-04-25T09:56:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.065+0000 7f8ba0dcf740 20 client.6072 trim_cache size 0 max 16384 2024-04-25T09:56:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.080+0000 7f8ba0dcf740 1 Processor -- start 2024-04-25T09:56:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.080+0000 7f8ba0dcf740 1 -- start start 2024-04-25T09:56:06.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.080+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e86b10 0x55d689e80110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.080+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.080+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689eabd30 0x55d689e5a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.080+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b98200490 con 0x55d689e86ef0 2024-04-25T09:56:06.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b981ff6c0 con 0x55d689e86b10 2024-04-25T09:56:06.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b98149560 con 0x55d689eabd30 2024-04-25T09:56:06.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47256/0 (socket says 172.21.15.19:47256) 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b96ffd640 1 -- 172.21.15.19:0/2135350262 learned_addr learned my addr 172.21.15.19:0/2135350262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b96ffd640 1 -- 172.21.15.19:0/2135350262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689eabd30 msgr2=0x55d689e5a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/2135350262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689eabd30 0x55d689e5a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b96ffd640 1 -- 172.21.15.19:0/2135350262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e86b10 msgr2=0x55d689e80110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/2135350262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e86b10 0x55d689e80110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b96ffd640 1 -- 172.21.15.19:0/2135350262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d689a970f0 con 0x55d689e86ef0 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/2135350262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea0fe0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00b390 tx=0x7f8b8c02d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598fedae0a3c20c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b7cff9640 1 -- 172.21.15.19:0/2135350262 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c045070 con 0x55d689e86ef0 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b7cff9640 1 -- 172.21.15.19:0/2135350262 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c025040 con 0x55d689e86ef0 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.081+0000 7f8b7cff9640 1 -- 172.21.15.19:0/2135350262 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c01b070 con 0x55d689e86ef0 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.082+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2135350262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 msgr2=0x55d689ea0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.082+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/2135350262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea0fe0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00b390 tx=0x7f8b8c02d040 comp rx=0 tx=0).stop 2024-04-25T09:56:06.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.082+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2135350262 shutdown_connections 2024-04-25T09:56:06.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.082+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/2135350262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689eabd30 0x55d689e5a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.082+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/2135350262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea0fe0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.082+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/2135350262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689e86b10 0x55d689e80110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.082+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2135350262 >> 172.21.15.19:0/2135350262 conn(0x55d689e59770 msgr2=0x55d689e54e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:06.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.082+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2135350262 shutdown_connections 2024-04-25T09:56:06.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.082+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/2135350262 wait complete. 2024-04-25T09:56:06.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.082+0000 7f8ba0dcf740 1 -- start start 2024-04-25T09:56:06.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.082+0000 7f8ba0dcf740 1 Processor -- start 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689e86b10 0x55d689ea1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8ba0dcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689eabd30 0x55d689e58600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d689d79d90 con 0x55d689e86ef0 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b981ff0f0 con 0x55d689eabd30 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8ba0dcf740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d689ea8890 con 0x55d689e86b10 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8ba0dcf740 10 client.0 _ll_register_callbacks cb 0x7f8b8c00b200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8b96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8b96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47260/0 (socket says 172.21.15.19:47260) 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8b96ffd640 1 -- 172.21.15.19:0/594906028 learned_addr learned my addr 172.21.15.19:0/594906028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8b97fff640 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689eabd30 0x55d689e58600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8b977fe640 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689e86b10 0x55d689ea1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8b96ffd640 1 -- 172.21.15.19:0/594906028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689e86b10 msgr2=0x55d689ea1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689e86b10 0x55d689ea1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8b96ffd640 1 -- 172.21.15.19:0/594906028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689eabd30 msgr2=0x55d689e58600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689eabd30 0x55d689e58600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.083+0000 7f8b96ffd640 1 -- 172.21.15.19:0/594906028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d689ec5a70 con 0x55d689e86ef0 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8b96ffd640 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea4320 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c006de0 tx=0x7f8b8c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e95d951a5863924 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8b767fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8b767fc640 1 -- 172.21.15.19:0/594906028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c045070 con 0x55d689e86ef0 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8b767fc640 1 -- 172.21.15.19:0/594906028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8b8c04ac40 con 0x55d689e86ef0 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/594906028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b8c025040 con 0x55d689e86ef0 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8b767fc640 1 -- 172.21.15.19:0/594906028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b8c01b040 con 0x55d689e86ef0 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8ba0dcf740 10 client.6078 fetch_fsmap learned FSMap version 24 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/594906028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55d689eb6520 con 0x55d689e86ef0 2024-04-25T09:56:06.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8b767fc640 1 -- 172.21.15.19:0/594906028 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c025040 con 0x55d689e86ef0 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8b767fc640 1 -- 172.21.15.19:0/594906028 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b8c03f050 con 0x55d689e86ef0 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8ba0dcf740 10 client.6078 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8ba0dcf740 10 client.6078 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8ba0dcf740 20 client.6078 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d689eb6080 0x55d689e58b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8ba0dcf740 4 client.6078 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/594906028 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d689eaae20 con 0x55d689eb6080 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.084+0000 7f8b977fe640 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d689eb6080 0x55d689e58b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8b977fe640 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d689eb6080 0x55d689e58b40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca092682155d074e server_cookie=f9febcc31e3e6d95 in_seq=0 out_seq=0 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8b767fc640 10 client.6078 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8b767fc640 1 -- 172.21.15.19:0/594906028 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+903 (crc 0 0 0) 0x55d689eaae20 con 0x55d689eb6080 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8b767fc640 10 client.6078 handle_command_reply: tid=0 2024-04-25T09:56:06.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 1 client.6078 shutdown 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/594906028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 msgr2=0x55d689ea4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea4320 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c006de0 tx=0x7f8b8c015000 comp rx=0 tx=0).stop 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/594906028 shutdown_connections 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d689eb6080 0x55d689e58b40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d689eabd30 0x55d689e58600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d689e86ef0 0x55d689ea4320 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8b767fc640 0 client.6078 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 1 --2- 172.21.15.19:0/594906028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d689e86b10 0x55d689ea1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/594906028 >> 172.21.15.19:0/594906028 conn(0x55d689e59770 msgr2=0x55d689eac270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/594906028 shutdown_connections 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 1 -- 172.21.15.19:0/594906028 wait complete. 2024-04-25T09:56:06.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8ba0dcf740 20 client.6078 trim_cache size 0 max 16384 2024-04-25T09:56:06.096 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:56:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T09:56:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "3f670385-c06d-4cd6-a126-aebacfa310ad": { 2024-04-25T09:56:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T09:56:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "3f670385-c06d-4cd6-a126-aebacfa310ad", 2024-04-25T09:56:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "" 2024-04-25T09:56:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "4195ee7d-b509-4ae5-ad3e-f37876a7321f": { 2024-04-25T09:56:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T09:56:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "4195ee7d-b509-4ae5-ad3e-f37876a7321f", 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "" 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "4241dd9e-9e1c-4105-98e7-a68e4f24045c": { 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "4241dd9e-9e1c-4105-98e7-a68e4f24045c", 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "" 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "c25d4699-c428-47da-bbba-88bee86fc907": { 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "c25d4699-c428-47da-bbba-88bee86fc907", 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "" 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:06.097 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:06.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8b9dff4640 1 -- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b8406b4d0 msgr2=0x7f8b8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:06.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8b9dff4640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b8406b4d0 0x7f8b8406d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c006fd0 tx=0x7f8b8c008040 comp rx=0 tx=0).stop 2024-04-25T09:56:06.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8b9dff4640 1 -- 172.21.15.19:0/4233401039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b980b9330 msgr2=0x7f8b98160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:06.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.085+0000 7f8b9dff4640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b980b9330 0x7f8b98160750 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8b8802a2b0 tx=0x7f8b880046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.086+0000 7f8b9dff4640 1 -- 172.21.15.19:0/4233401039 shutdown_connections 2024-04-25T09:56:06.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.086+0000 7f8b9dff4640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b8406b4d0 0x7f8b8406d990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.086+0000 7f8b9dff4640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b98161710 0x7f8b981576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.086+0000 7f8b9dff4640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b9815dc20 0x7f8b9815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.086+0000 7f8b9dff4640 1 --2- 172.21.15.19:0/4233401039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b980b9330 0x7f8b98160750 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:06.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.086+0000 7f8b9dff4640 1 -- 172.21.15.19:0/4233401039 >> 172.21.15.19:0/4233401039 conn(0x7f8b980d0420 msgr2=0x7f8b980b7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:06.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.086+0000 7f8b9dff4640 1 -- 172.21.15.19:0/4233401039 shutdown_connections 2024-04-25T09:56:06.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:06.086+0000 7f8b9dff4640 1 -- 172.21.15.19:0/4233401039 wait complete. 2024-04-25T09:56:07.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:56:07.156 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:56:09.036 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T09:56:09.032+0000 7fc4f0674640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T09:56:11.100 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:56:11.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:56:11.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.212+0000 7feeb56c4640 1 Processor -- start 2024-04-25T09:56:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.213+0000 7feeb56c4640 1 -- start start 2024-04-25T09:56:11.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.213+0000 7feeb56c4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feeb00d00e0 0x7feeb00ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.213+0000 7feeb56c4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb00cb190 con 0x7feeb00ccdd0 2024-04-25T09:56:11.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.213+0000 7feeb56c4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb00cb470 con 0x7feeb015bb20 2024-04-25T09:56:11.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.213+0000 7feeb56c4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb00cb750 con 0x7feeb00d00e0 2024-04-25T09:56:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.213+0000 7feeae7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feeb00ccdd0 0x7feeb00cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41410/0 (socket says 172.21.15.19:41410) 2024-04-25T09:56:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeae7fc640 1 -- 172.21.15.19:0/3297526189 learned_addr learned my addr 172.21.15.19:0/3297526189 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeaf7fe640 1 --2- 172.21.15.19:0/3297526189 >> v2:172.21.15.112:3300/0 conn(0x7feeb00d00e0 0x7feeb00ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840495505 0 0) 0x7feeb00cb190 con 0x7feeb00ccdd0 2024-04-25T09:56:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee9c003520 con 0x7feeb00ccdd0 2024-04-25T09:56:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234717204 0 0) 0x7fee9c003520 con 0x7feeb00ccdd0 2024-04-25T09:56:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 >> v2:172.21.15.112:3300/0 conn(0x7feeb00d00e0 msgr2=0x7feeb00ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 --2- 172.21.15.19:0/3297526189 >> v2:172.21.15.112:3300/0 conn(0x7feeb00d00e0 0x7feeb00ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 >> v1:172.21.15.112:6789/0 conn(0x7feeb015bb20 legacy=0x7feeb00cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb00cba30 con 0x7feeb00ccdd0 2024-04-25T09:56:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fee98001670 con 0x7feeb00ccdd0 2024-04-25T09:56:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 >> v1:172.21.15.19:6789/0 conn(0x7feeb00ccdd0 legacy=0x7feeb00cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 --2- 172.21.15.19:0/3297526189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 0x7fee9c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 --2- 172.21.15.19:0/3297526189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee9c0043b0 0x7fee9c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 --2- 172.21.15.19:0/3297526189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee9c005120 0x7fee9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb00cb750 con 0x7fee9c0039b0 2024-04-25T09:56:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c00c060 con 0x7fee9c0043b0 2024-04-25T09:56:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c00c260 con 0x7fee9c005120 2024-04-25T09:56:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeaeffd640 1 --2- 172.21.15.19:0/3297526189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 0x7fee9c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeae7fc640 1 --2- 172.21.15.19:0/3297526189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee9c0043b0 0x7fee9c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.214+0000 7feeaf7fe640 1 --2- 172.21.15.19:0/3297526189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee9c005120 0x7fee9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.215+0000 7feeaeffd640 1 -- 172.21.15.19:0/3297526189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee9c005120 msgr2=0x7fee9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.215+0000 7feeaeffd640 1 --2- 172.21.15.19:0/3297526189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee9c005120 0x7fee9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.215+0000 7feeaeffd640 1 -- 172.21.15.19:0/3297526189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee9c0043b0 msgr2=0x7fee9c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.215+0000 7feeaeffd640 1 --2- 172.21.15.19:0/3297526189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee9c0043b0 0x7fee9c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.215+0000 7feeaeffd640 1 -- 172.21.15.19:0/3297526189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c003520 con 0x7fee9c0039b0 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.215+0000 7feeaf7fe640 1 -- 172.21.15.19:0/3297526189 reap_dead start 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.215+0000 7feeaeffd640 1 --2- 172.21.15.19:0/3297526189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 0x7fee9c003d90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feea8001860 tx=0x7fee9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f582ab48d591441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.216+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8037070 con 0x7fee9c0039b0 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.216+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea802ac30 con 0x7fee9c0039b0 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.216+0000 7feeadffb640 1 -- 172.21.15.19:0/3297526189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8001430 con 0x7fee9c0039b0 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.216+0000 7feeb56c4640 1 -- 172.21.15.19:0/3297526189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 msgr2=0x7fee9c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.216+0000 7feeb56c4640 1 --2- 172.21.15.19:0/3297526189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 0x7fee9c003d90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feea8001860 tx=0x7fee9c005830 comp rx=0 tx=0).stop 2024-04-25T09:56:11.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.216+0000 7feeb56c4640 1 -- 172.21.15.19:0/3297526189 shutdown_connections 2024-04-25T09:56:11.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.216+0000 7feeb56c4640 1 --2- 172.21.15.19:0/3297526189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 0x7fee9c003d90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.216+0000 7feeb56c4640 1 -- 172.21.15.19:0/3297526189 >> 172.21.15.19:0/3297526189 conn(0x7feeb00b2390 msgr2=0x7feeb00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.216+0000 7feeb56c4640 1 -- 172.21.15.19:0/3297526189 shutdown_connections 2024-04-25T09:56:11.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.216+0000 7feeb56c4640 1 -- 172.21.15.19:0/3297526189 wait complete. 2024-04-25T09:56:11.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.218+0000 7feeb56c4640 1 Processor -- start 2024-04-25T09:56:11.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.218+0000 7feeb56c4640 1 -- start start 2024-04-25T09:56:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.218+0000 7feeb56c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 0x7feeb01f0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.218+0000 7feeb56c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb015a210 0x7feeb0159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.218+0000 7feeb56c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb0156be0 0x7feeb0157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.218+0000 7feeb56c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb015c080 con 0x7fee9c0039b0 2024-04-25T09:56:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.218+0000 7feeb56c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb015bf00 con 0x7feeb015a210 2024-04-25T09:56:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.218+0000 7feeb56c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb00cadb0 con 0x7feeb0156be0 2024-04-25T09:56:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.218+0000 7feeaeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 0x7feeb01f0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeaeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 0x7feeb01f0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33720/0 (socket says 172.21.15.19:33720) 2024-04-25T09:56:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeaeffd640 1 -- 172.21.15.19:0/2023518269 learned_addr learned my addr 172.21.15.19:0/2023518269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeae7fc640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb015a210 0x7feeb0159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeaf7fe640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb0156be0 0x7feeb0157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeaeffd640 1 -- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb0156be0 msgr2=0x7feeb0157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeaeffd640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb0156be0 0x7feeb0157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeaeffd640 1 -- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb015a210 msgr2=0x7feeb0159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeaeffd640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb015a210 0x7feeb0159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeaeffd640 1 -- 172.21.15.19:0/2023518269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb01595a0 con 0x7fee9c0039b0 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeaeffd640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 0x7feeb01f0d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7feea802a6c0 tx=0x7feea8001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feea77fe640 1 -- 172.21.15.19:0/2023518269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8040030 con 0x7fee9c0039b0 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feea77fe640 1 -- 172.21.15.19:0/2023518269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8005070 con 0x7fee9c0039b0 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeb56c4640 1 -- 172.21.15.19:0/2023518269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb015de50 con 0x7fee9c0039b0 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feea77fe640 1 -- 172.21.15.19:0/2023518269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8037040 con 0x7fee9c0039b0 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.219+0000 7feeb56c4640 1 -- 172.21.15.19:0/2023518269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb015e410 con 0x7fee9c0039b0 2024-04-25T09:56:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.221+0000 7feea77fe640 1 -- 172.21.15.19:0/2023518269 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feea8043070 con 0x7fee9c0039b0 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.221+0000 7feea77fe640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b500 0x7fee9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.221+0000 7feea77fe640 1 -- 172.21.15.19:0/2023518269 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feea80aa0f0 con 0x7fee9c0039b0 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.222+0000 7feeae7fc640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b500 0x7fee9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.222+0000 7feeb83ca740 1 Processor -- start 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.222+0000 7feeae7fc640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b500 0x7fee9006d9c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feeb01f4790 tx=0x7fee980078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.222+0000 7feeb83ca740 1 -- start start 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.223+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.223+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec49ef0 0x55681ec4c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.223+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec1c770 0x55681ec1ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.223+0000 7feeb83ca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55681eb85da0 con 0x55681ec49b10 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.223+0000 7feeb83ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55681eb2eb80 con 0x55681ec1c770 2024-04-25T09:56:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.223+0000 7feeb83ca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55681eb81e40 con 0x55681ec49ef0 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.223+0000 7feeaeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.223+0000 7feeaeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33728/0 (socket says 172.21.15.19:33728) 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.223+0000 7feeaeffd640 1 -- 172.21.15.19:0/66216111 learned_addr learned my addr 172.21.15.19:0/66216111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.223+0000 7feeae7fc640 1 --2- 172.21.15.19:0/66216111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec49ef0 0x55681ec4c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.223+0000 7feeaf7fe640 1 --2- 172.21.15.19:0/66216111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec1c770 0x55681ec1ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.224+0000 7feeaeffd640 1 -- 172.21.15.19:0/66216111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec49ef0 msgr2=0x55681ec4c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.224+0000 7feeaeffd640 1 --2- 172.21.15.19:0/66216111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec49ef0 0x55681ec4c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.224+0000 7feeaeffd640 1 -- 172.21.15.19:0/66216111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec1c770 msgr2=0x55681ec1ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.224+0000 7feeaeffd640 1 --2- 172.21.15.19:0/66216111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec1c770 0x55681ec1ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.224+0000 7feeaeffd640 1 -- 172.21.15.19:0/66216111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55681eb38f10 con 0x55681ec49b10 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.224+0000 7feeaeffd640 1 --2- 172.21.15.19:0/66216111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4c320 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7feea8005040 tx=0x7feea80780f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07fab9e99bd1dac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.224+0000 7feea4ff9640 1 -- 172.21.15.19:0/66216111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8080df0 con 0x55681ec49b10 2024-04-25T09:56:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.224+0000 7feea4ff9640 1 -- 172.21.15.19:0/66216111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8076a90 con 0x55681ec49b10 2024-04-25T09:56:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.224+0000 7feea4ff9640 1 -- 172.21.15.19:0/66216111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8090120 con 0x55681ec49b10 2024-04-25T09:56:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.224+0000 7feeb83ca740 1 -- 172.21.15.19:0/66216111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 msgr2=0x55681ec4c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.224+0000 7feeb83ca740 1 --2- 172.21.15.19:0/66216111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4c320 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7feea8005040 tx=0x7feea80780f0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.225+0000 7feeb83ca740 1 -- 172.21.15.19:0/66216111 shutdown_connections 2024-04-25T09:56:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.225+0000 7feeb83ca740 1 --2- 172.21.15.19:0/66216111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec1c770 0x55681ec1ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.225+0000 7feeb83ca740 1 --2- 172.21.15.19:0/66216111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec49ef0 0x55681ec4c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.225+0000 7feeb83ca740 1 --2- 172.21.15.19:0/66216111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4c320 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.225+0000 7feeb83ca740 1 -- 172.21.15.19:0/66216111 >> 172.21.15.19:0/66216111 conn(0x55681ec1e9e0 msgr2=0x55681ec67320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.225+0000 7feeb83ca740 1 -- 172.21.15.19:0/66216111 shutdown_connections 2024-04-25T09:56:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.225+0000 7feeb83ca740 1 -- 172.21.15.19:0/66216111 wait complete. 2024-04-25T09:56:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.225+0000 7feeb83ca740 1 -- start start 2024-04-25T09:56:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.225+0000 7feeb83ca740 1 Processor -- start 2024-04-25T09:56:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49ef0 0x55681ec40ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec69630 0x55681ec69ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeb83ca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55681eb81e40 con 0x55681ec49b10 2024-04-25T09:56:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeb83ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55681eb244b0 con 0x55681ec49ef0 2024-04-25T09:56:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeb83ca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55681eb85da0 con 0x55681ec69630 2024-04-25T09:56:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeb83ca740 10 client.0 _ll_register_callbacks cb 0x55681eb64180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeaeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeaeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33732/0 (socket says 172.21.15.19:33732) 2024-04-25T09:56:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeaeffd640 1 -- 172.21.15.19:0/136983476 learned_addr learned my addr 172.21.15.19:0/136983476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeaf7fe640 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec69630 0x55681ec69ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeae7fc640 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49ef0 0x55681ec40ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeaeffd640 1 -- 172.21.15.19:0/136983476 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec69630 msgr2=0x55681ec69ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeaeffd640 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec69630 0x55681ec69ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeaeffd640 1 -- 172.21.15.19:0/136983476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49ef0 msgr2=0x55681ec40ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeaeffd640 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49ef0 0x55681ec40ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeaeffd640 1 -- 172.21.15.19:0/136983476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55681e9647d0 con 0x55681ec49b10 2024-04-25T09:56:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.226+0000 7feeae7fc640 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49ef0 0x55681ec40ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7feeaeffd640 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f300 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feea80a84e0 tx=0x7feea807b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1636a9e264e6ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7fee8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7fee8dffb640 1 -- 172.21.15.19:0/136983476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea80a81c0 con 0x55681ec49b10 2024-04-25T09:56:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7fee8dffb640 1 -- 172.21.15.19:0/136983476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feea8001430 con 0x55681ec49b10 2024-04-25T09:56:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7feeb83ca740 1 -- 172.21.15.19:0/136983476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feea8076a90 con 0x55681ec49b10 2024-04-25T09:56:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7fee8dffb640 1 -- 172.21.15.19:0/136983476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feea8080df0 con 0x55681ec49b10 2024-04-25T09:56:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7feeb83ca740 10 client.6090 fetch_fsmap learned FSMap version 25 2024-04-25T09:56:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7feeb83ca740 1 -- 172.21.15.19:0/136983476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55681e89a3f0 con 0x55681ec49b10 2024-04-25T09:56:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7fee8dffb640 1 -- 172.21.15.19:0/136983476 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8076a90 con 0x55681ec49b10 2024-04-25T09:56:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7fee8dffb640 1 -- 172.21.15.19:0/136983476 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feea8097170 con 0x55681ec49b10 2024-04-25T09:56:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7feeb83ca740 10 client.6090 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:56:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7feeb83ca740 10 client.6090 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7feeb83ca740 20 client.6090 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7feeb83ca740 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55681ec1bad0 0x55681ec4d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7feeb83ca740 4 client.6090 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7feeb83ca740 1 -- 172.21.15.19:0/136983476 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55681eb932e0 con 0x55681ec1bad0 2024-04-25T09:56:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.227+0000 7feeae7fc640 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55681ec1bad0 0x55681ec4d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7feeae7fc640 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55681ec1bad0 0x55681ec4d1f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f287f5122db8d81d server_cookie=6059dbbe556d69ff in_seq=0 out_seq=0 2024-04-25T09:56:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7fee8dffb640 10 client.6090 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7fee8dffb640 1 -- 172.21.15.19:0/136983476 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55681eb932e0 con 0x55681ec1bad0 2024-04-25T09:56:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7fee8dffb640 10 client.6090 handle_command_reply: tid=0 2024-04-25T09:56:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7feeb83ca740 1 client.6090 shutdown 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7feeb83ca740 1 -- 172.21.15.19:0/136983476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 msgr2=0x55681ec4f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7feeb83ca740 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f300 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feea80a84e0 tx=0x7feea807b070 comp rx=0 tx=0).stop 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7feeb83ca740 1 -- 172.21.15.19:0/136983476 shutdown_connections 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7feeb83ca740 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55681ec1bad0 0x55681ec4d1f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7feeb83ca740 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec69630 0x55681ec69ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7feeb83ca740 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49ef0 0x55681ec40ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7fee8dffb640 0 client.6090 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7feeb83ca740 1 --2- 172.21.15.19:0/136983476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f300 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7feeb83ca740 1 -- 172.21.15.19:0/136983476 >> 172.21.15.19:0/136983476 conn(0x55681ec1c770 msgr2=0x55681ec66b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.228+0000 7feeb83ca740 1 -- 172.21.15.19:0/136983476 shutdown_connections 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.229+0000 7feeb83ca740 1 -- 172.21.15.19:0/136983476 wait complete. 2024-04-25T09:56:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.229+0000 7feeb83ca740 20 client.6090 trim_cache size 0 max 16384 2024-04-25T09:56:11.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.244+0000 7feeb83ca740 1 Processor -- start 2024-04-25T09:56:11.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.244+0000 7feeb83ca740 1 -- start start 2024-04-25T09:56:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49b10 0x55681ec40ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49ef0 0x55681ec206c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec69630 0x55681ec4f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeb83ca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01fe9d0 con 0x55681ec49ef0 2024-04-25T09:56:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeb83ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01fdbd0 con 0x55681ec49b10 2024-04-25T09:56:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeb83ca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb00b56a0 con 0x55681ec69630 2024-04-25T09:56:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeae7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49ef0 0x55681ec206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeae7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49ef0 0x55681ec206c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33734/0 (socket says 172.21.15.19:33734) 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeae7fc640 1 -- 172.21.15.19:0/4165905885 learned_addr learned my addr 172.21.15.19:0/4165905885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeaf7fe640 1 --2- 172.21.15.19:0/4165905885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec69630 0x55681ec4f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeae7fc640 1 -- 172.21.15.19:0/4165905885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec69630 msgr2=0x55681ec4f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeaeffd640 1 --2- 172.21.15.19:0/4165905885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49b10 0x55681ec40ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeae7fc640 1 --2- 172.21.15.19:0/4165905885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec69630 0x55681ec4f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeae7fc640 1 -- 172.21.15.19:0/4165905885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49b10 msgr2=0x55681ec40ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeae7fc640 1 --2- 172.21.15.19:0/4165905885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49b10 0x55681ec40ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeae7fc640 1 -- 172.21.15.19:0/4165905885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55681e85a0f0 con 0x55681ec49ef0 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeaeffd640 1 --2- 172.21.15.19:0/4165905885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49b10 0x55681ec40ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7feeae7fc640 1 --2- 172.21.15.19:0/4165905885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49ef0 0x55681ec206c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7feeb015d430 tx=0x7fee9800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69537e342bb9905 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7fee8ffff640 1 -- 172.21.15.19:0/4165905885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9802d070 con 0x55681ec49ef0 2024-04-25T09:56:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.245+0000 7fee8ffff640 1 -- 172.21.15.19:0/4165905885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee98027150 con 0x55681ec49ef0 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.246+0000 7fee8ffff640 1 -- 172.21.15.19:0/4165905885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98027e60 con 0x55681ec49ef0 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.246+0000 7feeb83ca740 1 -- 172.21.15.19:0/4165905885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49ef0 msgr2=0x55681ec206c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.246+0000 7feeb83ca740 1 --2- 172.21.15.19:0/4165905885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49ef0 0x55681ec206c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7feeb015d430 tx=0x7fee9800c000 comp rx=0 tx=0).stop 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.246+0000 7feeb83ca740 1 -- 172.21.15.19:0/4165905885 shutdown_connections 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.246+0000 7feeb83ca740 1 --2- 172.21.15.19:0/4165905885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec69630 0x55681ec4f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.246+0000 7feeb83ca740 1 --2- 172.21.15.19:0/4165905885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49ef0 0x55681ec206c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.246+0000 7feeb83ca740 1 --2- 172.21.15.19:0/4165905885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec49b10 0x55681ec40ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.246+0000 7feeb83ca740 1 -- 172.21.15.19:0/4165905885 >> 172.21.15.19:0/4165905885 conn(0x55681ec1c770 msgr2=0x55681ec47920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.246+0000 7feeb83ca740 1 -- 172.21.15.19:0/4165905885 shutdown_connections 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.246+0000 7feeb83ca740 1 -- 172.21.15.19:0/4165905885 wait complete. 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.247+0000 7feeb83ca740 1 -- start start 2024-04-25T09:56:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.247+0000 7feeb83ca740 1 Processor -- start 2024-04-25T09:56:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.247+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.247+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec49ef0 0x55681ec68450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.247+0000 7feeb83ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec69630 0x55681ec68990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.247+0000 7feeb83ca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55681eb2eb80 con 0x55681ec49b10 2024-04-25T09:56:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.247+0000 7feeb83ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01fd5d0 con 0x55681ec69630 2024-04-25T09:56:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.247+0000 7feeb83ca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55681ec22cc0 con 0x55681ec49ef0 2024-04-25T09:56:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.247+0000 7feeb83ca740 10 client.0 _ll_register_callbacks cb 0x7feea8007720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.247+0000 7feeaeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeaeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33736/0 (socket says 172.21.15.19:33736) 2024-04-25T09:56:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeaeffd640 1 -- 172.21.15.19:0/2963366133 learned_addr learned my addr 172.21.15.19:0/2963366133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeae7fc640 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec49ef0 0x55681ec68450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeaf7fe640 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec69630 0x55681ec68990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeaeffd640 1 -- 172.21.15.19:0/2963366133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec49ef0 msgr2=0x55681ec68450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeaeffd640 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec49ef0 0x55681ec68450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeaeffd640 1 -- 172.21.15.19:0/2963366133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec69630 msgr2=0x55681ec68990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeaeffd640 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec69630 0x55681ec68990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeaeffd640 1 -- 172.21.15.19:0/2963366133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55681ec47470 con 0x55681ec49b10 2024-04-25T09:56:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeae7fc640 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec49ef0 0x55681ec68450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeaeffd640 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7feea8035040 tx=0x7feea8007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c275532e3f86a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7fee8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7fee8dffb640 1 -- 172.21.15.19:0/2963366133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea80a81c0 con 0x55681ec49b10 2024-04-25T09:56:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7fee8dffb640 1 -- 172.21.15.19:0/2963366133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feea8078b60 con 0x55681ec49b10 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeb83ca740 1 -- 172.21.15.19:0/2963366133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee98027150 con 0x55681ec49b10 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7fee8dffb640 1 -- 172.21.15.19:0/2963366133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feea8090230 con 0x55681ec49b10 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeb83ca740 10 client.6096 fetch_fsmap learned FSMap version 25 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeb83ca740 1 -- 172.21.15.19:0/2963366133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55681ec47470 con 0x55681ec49b10 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7fee8dffb640 1 -- 172.21.15.19:0/2963366133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee98027150 con 0x55681ec49b10 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7fee8dffb640 1 -- 172.21.15.19:0/2963366133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feea8080960 con 0x55681ec49b10 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeb83ca740 10 client.6096 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.248+0000 7feeb83ca740 10 client.6096 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 20 client.6096 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55681ec48190 0x55681ec68ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 4 client.6096 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:56:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 -- 172.21.15.19:0/2963366133 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55681ec4ee70 con 0x55681ec48190 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeae7fc640 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55681ec48190 0x55681ec68ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeae7fc640 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55681ec48190 0x55681ec68ed0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea315f4cf642cae server_cookie=5b39c38eedc61882 in_seq=0 out_seq=0 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7fee8dffb640 10 client.6096 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7fee8dffb640 1 -- 172.21.15.19:0/2963366133 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55681ec4ee70 con 0x55681ec48190 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7fee8dffb640 10 client.6096 handle_command_reply: tid=0 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 client.6096 shutdown 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 -- 172.21.15.19:0/2963366133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 msgr2=0x55681ec4f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7feea8035040 tx=0x7feea8007ca0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 -- 172.21.15.19:0/2963366133 shutdown_connections 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55681ec48190 0x55681ec68ed0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55681ec69630 0x55681ec68990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55681ec49ef0 0x55681ec68450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7fee8dffb640 0 client.6096 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 --2- 172.21.15.19:0/2963366133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55681ec49b10 0x55681ec4f420 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 -- 172.21.15.19:0/2963366133 >> 172.21.15.19:0/2963366133 conn(0x55681ec1c770 msgr2=0x55681ec4c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 -- 172.21.15.19:0/2963366133 shutdown_connections 2024-04-25T09:56:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 1 -- 172.21.15.19:0/2963366133 wait complete. 2024-04-25T09:56:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.249+0000 7feeb83ca740 20 client.6096 trim_cache size 0 max 16384 2024-04-25T09:56:11.262 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T09:56:11.263 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T09:56:11.263 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 -- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b500 msgr2=0x7fee9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b500 0x7fee9006d9c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feeb01f4790 tx=0x7fee980078f0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 -- 172.21.15.19:0/2023518269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 msgr2=0x7feeb01f0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 0x7feeb01f0d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7feea802a6c0 tx=0x7feea8001230 comp rx=0 tx=0).stop 2024-04-25T09:56:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 -- 172.21.15.19:0/2023518269 shutdown_connections 2024-04-25T09:56:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee9006b500 0x7fee9006d9c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb0156be0 0x7feeb0157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb015a210 0x7feeb0159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 --2- 172.21.15.19:0/2023518269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0039b0 0x7feeb01f0d90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 -- 172.21.15.19:0/2023518269 >> 172.21.15.19:0/2023518269 conn(0x7feeb00b2390 msgr2=0x7feeb00ad600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 -- 172.21.15.19:0/2023518269 shutdown_connections 2024-04-25T09:56:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.250+0000 7feeb56c4640 1 -- 172.21.15.19:0/2023518269 wait complete. 2024-04-25T09:56:11.265 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:56:11.266 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:11.266 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:56:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.364+0000 7f5e0b438640 1 Processor -- start 2024-04-25T09:56:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.365+0000 7f5e0b438640 1 -- start start 2024-04-25T09:56:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.365+0000 7f5e0b438640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e040ced80 0x7f5e040d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.365+0000 7f5e0b438640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5e04161710 0x7f5e040d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.365+0000 7f5e0b438640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0415e000 con 0x7f5e0415dc20 2024-04-25T09:56:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.365+0000 7f5e0b438640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e040d1de0 con 0x7f5e04161710 2024-04-25T09:56:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.365+0000 7f5e0b438640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e04161c60 con 0x7f5e040ced80 2024-04-25T09:56:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.366+0000 7f5e099d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5e04161710 0x7f5e040d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.366+0000 7f5e099d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5e04161710 0x7f5e040d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33738/0 (socket says 172.21.15.19:33738) 2024-04-25T09:56:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.366+0000 7f5e099d5640 1 -- 172.21.15.19:0/257003951 learned_addr learned my addr 172.21.15.19:0/257003951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.366+0000 7f5e089d3640 1 --2- 172.21.15.19:0/257003951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e040ced80 0x7f5e040d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.366+0000 7f5e099d5640 1 -- 172.21.15.19:0/257003951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e040ced80 msgr2=0x7f5e040d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.366+0000 7f5e099d5640 1 --2- 172.21.15.19:0/257003951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e040ced80 0x7f5e040d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.366+0000 7f5e099d5640 1 -- 172.21.15.19:0/257003951 >> v1:172.21.15.112:6789/0 conn(0x7f5e0415dc20 legacy=0x7f5e040ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.366+0000 7f5e099d5640 1 -- 172.21.15.19:0/257003951 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e04161f40 con 0x7f5e04161710 2024-04-25T09:56:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.366+0000 7f5e089d3640 1 --2- 172.21.15.19:0/257003951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e040ced80 0x7f5e040d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.367+0000 7f5e099d5640 1 --2- 172.21.15.19:0/257003951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e04161710 0x7f5e040d2820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009e50 tx=0x7f5dfc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3133b2e1adb70c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.367+0000 7f5dfbfff640 1 -- 172.21.15.19:0/257003951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc036070 con 0x7f5e04161710 2024-04-25T09:56:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.367+0000 7f5dfbfff640 1 -- 172.21.15.19:0/257003951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc031600 con 0x7f5e04161710 2024-04-25T09:56:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.367+0000 7f5dfbfff640 1 -- 172.21.15.19:0/257003951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0319d0 con 0x7f5e04161710 2024-04-25T09:56:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.367+0000 7f5e0b438640 1 -- 172.21.15.19:0/257003951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e04161710 msgr2=0x7f5e040d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.367+0000 7f5e0b438640 1 --2- 172.21.15.19:0/257003951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e04161710 0x7f5e040d2820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009e50 tx=0x7f5dfc02a710 comp rx=0 tx=0).stop 2024-04-25T09:56:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.367+0000 7f5e0b438640 1 -- 172.21.15.19:0/257003951 shutdown_connections 2024-04-25T09:56:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.367+0000 7f5e0b438640 1 --2- 172.21.15.19:0/257003951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e04161710 0x7f5e040d2820 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.367+0000 7f5e0b438640 1 --2- 172.21.15.19:0/257003951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e040ced80 0x7f5e040d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.367+0000 7f5e0b438640 1 -- 172.21.15.19:0/257003951 >> 172.21.15.19:0/257003951 conn(0x7f5e040d0420 msgr2=0x7f5e040d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.368+0000 7f5e0b438640 1 -- 172.21.15.19:0/257003951 shutdown_connections 2024-04-25T09:56:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.368+0000 7f5e0b438640 1 -- 172.21.15.19:0/257003951 wait complete. 2024-04-25T09:56:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.369+0000 7f5e0b438640 1 Processor -- start 2024-04-25T09:56:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.369+0000 7f5e0b438640 1 -- start start 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.369+0000 7f5e0b438640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e040ced80 0x7f5e040d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.369+0000 7f5e0b438640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e0415dc20 0x7f5e040c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.369+0000 7f5e0b438640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e04161710 0x7f5e040c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.369+0000 7f5e0b438640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040b5e50 con 0x7f5e04161710 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.369+0000 7f5e0b438640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e04162500 con 0x7f5e040ced80 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.369+0000 7f5e0b438640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040b5fd0 con 0x7f5e0415dc20 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e099d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e04161710 0x7f5e040c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e099d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e04161710 0x7f5e040c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33744/0 (socket says 172.21.15.19:33744) 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e099d5640 1 -- 172.21.15.19:0/3007129166 learned_addr learned my addr 172.21.15.19:0/3007129166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e089d3640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e0415dc20 0x7f5e040c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e099d5640 1 -- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e0415dc20 msgr2=0x7f5e040c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e091d4640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e040ced80 0x7f5e040d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e099d5640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e0415dc20 0x7f5e040c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e099d5640 1 -- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e040ced80 msgr2=0x7f5e040d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e099d5640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e040ced80 0x7f5e040d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e099d5640 1 -- 172.21.15.19:0/3007129166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e041e9c40 con 0x7f5e04161710 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e089d3640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e0415dc20 0x7f5e040c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e091d4640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e040ced80 0x7f5e040d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e099d5640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e04161710 0x7f5e040c9530 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc034040 tx=0x7f5dfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5df9ffb640 1 -- 172.21.15.19:0/3007129166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc041020 con 0x7f5e04161710 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5df9ffb640 1 -- 172.21.15.19:0/3007129166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc0077a0 con 0x7f5e04161710 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5e0b438640 1 -- 172.21.15.19:0/3007129166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e041497c0 con 0x7f5e04161710 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.370+0000 7f5df9ffb640 1 -- 172.21.15.19:0/3007129166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc036040 con 0x7f5e04161710 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.371+0000 7f5e0b438640 1 -- 172.21.15.19:0/3007129166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e04149d80 con 0x7f5e04161710 2024-04-25T09:56:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.372+0000 7f5df9ffb640 1 -- 172.21.15.19:0/3007129166 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5dfc044020 con 0x7f5e04161710 2024-04-25T09:56:11.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.373+0000 7f5df9ffb640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de806b4d0 0x7f5de806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.373+0000 7f5df9ffb640 1 -- 172.21.15.19:0/3007129166 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5dfc0ab0f0 con 0x7f5e04161710 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.373+0000 7f5e091d4640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de806b4d0 0x7f5de806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.373+0000 7f5e091d4640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de806b4d0 0x7f5de806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5e0415c870 tx=0x7f5e00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.374+0000 7f5e0e21a740 1 Processor -- start 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.374+0000 7f5e0e21a740 1 -- start start 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.374+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c35aa0 0x557fd3c382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.374+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c387f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.374+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c08700 0x557fd3c0bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.374+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fd3b71e00 con 0x557fd3c35e80 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.374+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fd3b1ad20 con 0x557fd3c08700 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.374+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fd3b6dea0 con 0x557fd3c35aa0 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.374+0000 7f5e089d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.374+0000 7f5e089d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c387f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33760/0 (socket says 172.21.15.19:33760) 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.374+0000 7f5e089d3640 1 -- 172.21.15.19:0/2393879527 learned_addr learned my addr 172.21.15.19:0/2393879527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5e099d5640 1 --2- 172.21.15.19:0/2393879527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c08700 0x557fd3c0bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5e091d4640 1 --2- 172.21.15.19:0/2393879527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c35aa0 0x557fd3c382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5e089d3640 1 -- 172.21.15.19:0/2393879527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c35aa0 msgr2=0x557fd3c382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5e089d3640 1 --2- 172.21.15.19:0/2393879527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c35aa0 0x557fd3c382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5e089d3640 1 -- 172.21.15.19:0/2393879527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c08700 msgr2=0x557fd3c0bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5e089d3640 1 --2- 172.21.15.19:0/2393879527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c08700 0x557fd3c0bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5e089d3640 1 -- 172.21.15.19:0/2393879527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fd3b3cdf0 con 0x557fd3c35e80 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5e091d4640 1 --2- 172.21.15.19:0/2393879527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c35aa0 0x557fd3c382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5e089d3640 1 --2- 172.21.15.19:0/2393879527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c387f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5df400b9d0 tx=0x7f5df400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9347183bf42dbdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5deeffd640 1 -- 172.21.15.19:0/2393879527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4016070 con 0x557fd3c35e80 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5deeffd640 1 -- 172.21.15.19:0/2393879527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df400dc70 con 0x557fd3c35e80 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.375+0000 7f5deeffd640 1 -- 172.21.15.19:0/2393879527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4012660 con 0x557fd3c35e80 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.376+0000 7f5e0e21a740 1 -- 172.21.15.19:0/2393879527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 msgr2=0x557fd3c387f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.376+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/2393879527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c387f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5df400b9d0 tx=0x7f5df400be90 comp rx=0 tx=0).stop 2024-04-25T09:56:11.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.376+0000 7f5e0e21a740 1 -- 172.21.15.19:0/2393879527 shutdown_connections 2024-04-25T09:56:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.376+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/2393879527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c08700 0x557fd3c0bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.376+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/2393879527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c387f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.376+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/2393879527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c35aa0 0x557fd3c382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.376+0000 7f5e0e21a740 1 -- 172.21.15.19:0/2393879527 >> 172.21.15.19:0/2393879527 conn(0x557fd3c0a970 msgr2=0x557fd3c532b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.376+0000 7f5e0e21a740 1 -- 172.21.15.19:0/2393879527 shutdown_connections 2024-04-25T09:56:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.376+0000 7f5e0e21a740 1 -- 172.21.15.19:0/2393879527 wait complete. 2024-04-25T09:56:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.376+0000 7f5e0e21a740 1 -- start start 2024-04-25T09:56:11.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.376+0000 7f5e0e21a740 1 Processor -- start 2024-04-25T09:56:11.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.377+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c2f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.377+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c3c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.377+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3c2cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.377+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fd3c0db20 con 0x557fd3c35e80 2024-04-25T09:56:11.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.377+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fd3b10590 con 0x557fd3c35aa0 2024-04-25T09:56:11.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.377+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fd3c0de00 con 0x557fd3c2c790 2024-04-25T09:56:11.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.377+0000 7f5e0e21a740 10 client.0 _ll_register_callbacks cb 0x557fd3b1ce30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:11.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.377+0000 7f5e089d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c3c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.377+0000 7f5e089d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c3c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33770/0 (socket says 172.21.15.19:33770) 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.377+0000 7f5e089d3640 1 -- 172.21.15.19:0/3686057412 learned_addr learned my addr 172.21.15.19:0/3686057412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e091d4640 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c2f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e099d5640 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3c2cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e089d3640 1 -- 172.21.15.19:0/3686057412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 msgr2=0x557fd3c2cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e089d3640 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3c2cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e089d3640 1 -- 172.21.15.19:0/3686057412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 msgr2=0x557fd3c2f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e089d3640 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c2f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e089d3640 1 -- 172.21.15.19:0/3686057412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fd38863f0 con 0x557fd3c35e80 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e099d5640 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3c2cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e091d4640 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c2f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e089d3640 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c3c020 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x557fd3c10000 tx=0x7f5df4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ed52ebe00972bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5de3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5de3fff640 1 -- 172.21.15.19:0/3686057412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df402b060 con 0x557fd3c35e80 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5de3fff640 1 -- 172.21.15.19:0/3686057412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5df400de20 con 0x557fd3c35e80 2024-04-25T09:56:11.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e0e21a740 1 -- 172.21.15.19:0/3686057412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5df400dc70 con 0x557fd3c35e80 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5de3fff640 1 -- 172.21.15.19:0/3686057412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5df4031050 con 0x557fd3c35e80 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e0e21a740 10 client.6108 fetch_fsmap learned FSMap version 25 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e0e21a740 1 -- 172.21.15.19:0/3686057412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x557fd39507d0 con 0x557fd3c35e80 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5de3fff640 1 -- 172.21.15.19:0/3686057412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df400dc70 con 0x557fd3c35e80 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5de3fff640 1 -- 172.21.15.19:0/3686057412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5df4034020 con 0x557fd3c35e80 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e0e21a740 10 client.6108 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e0e21a740 10 client.6108 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e0e21a740 20 client.6108 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557fd3c63b20 0x557fd3c63f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e0e21a740 4 client.6108 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.378+0000 7f5e0e21a740 1 -- 172.21.15.19:0/3686057412 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fd3b7f340 con 0x557fd3c63b20 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.379+0000 7f5e091d4640 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557fd3c63b20 0x557fd3c63f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.379+0000 7f5de3fff640 10 client.6108 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.379+0000 7f5e091d4640 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557fd3c63b20 0x557fd3c63f40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ece0f96d9c1022 server_cookie=54f16c59983c7c19 in_seq=0 out_seq=0 2024-04-25T09:56:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.379+0000 7f5de3fff640 1 -- 172.21.15.19:0/3686057412 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557fd3b7f340 con 0x557fd3c63b20 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.379+0000 7f5de3fff640 10 client.6108 handle_command_reply: tid=0 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.379+0000 7f5e0e21a740 1 client.6108 shutdown 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5e0e21a740 1 -- 172.21.15.19:0/3686057412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 msgr2=0x557fd3c3c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c3c020 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x557fd3c10000 tx=0x7f5df4004730 comp rx=0 tx=0).stop 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5e0e21a740 1 -- 172.21.15.19:0/3686057412 shutdown_connections 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557fd3c63b20 0x557fd3c63f40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5de3fff640 0 client.6108 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3c2cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c3c020 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/3686057412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c2f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5e0e21a740 1 -- 172.21.15.19:0/3686057412 >> 172.21.15.19:0/3686057412 conn(0x557fd3c08700 msgr2=0x557fd3c38620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5e0e21a740 1 -- 172.21.15.19:0/3686057412 shutdown_connections 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5e0e21a740 1 -- 172.21.15.19:0/3686057412 wait complete. 2024-04-25T09:56:11.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.380+0000 7f5e0e21a740 20 client.6108 trim_cache size 0 max 16384 2024-04-25T09:56:11.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.392+0000 7f5e0e21a740 1 Processor -- start 2024-04-25T09:56:11.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.392+0000 7f5e0e21a740 1 -- start start 2024-04-25T09:56:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.392+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3c38270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.392+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.392+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.392+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e042006a0 con 0x557fd3c35e80 2024-04-25T09:56:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.392+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041ff860 con 0x557fd3c35aa0 2024-04-25T09:56:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.392+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0414b570 con 0x557fd3c2c790 2024-04-25T09:56:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.392+0000 7f5e099d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.392+0000 7f5e099d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33782/0 (socket says 172.21.15.19:33782) 2024-04-25T09:56:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.393+0000 7f5e099d5640 1 -- 172.21.15.19:0/2864668427 learned_addr learned my addr 172.21.15.19:0/2864668427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.393+0000 7f5e099d5640 1 -- 172.21.15.19:0/2864668427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 msgr2=0x557fd3c38270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.393+0000 7f5e099d5640 1 --2- 172.21.15.19:0/2864668427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3c38270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.393+0000 7f5e099d5640 1 -- 172.21.15.19:0/2864668427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 msgr2=0x557fd3c387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.393+0000 7f5e089d3640 1 --2- 172.21.15.19:0/2864668427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.393+0000 7f5e099d5640 1 --2- 172.21.15.19:0/2864668427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c387b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.393+0000 7f5e099d5640 1 -- 172.21.15.19:0/2864668427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fd3c04690 con 0x557fd3c35e80 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.393+0000 7f5e099d5640 1 --2- 172.21.15.19:0/2864668427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0a9430 tx=0x7f5dfc0ab040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac0a593d06e09a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.393+0000 7f5dee7fc640 1 -- 172.21.15.19:0/2864668427 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0797a0 con 0x557fd3c35e80 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.393+0000 7f5dee7fc640 1 -- 172.21.15.19:0/2864668427 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc0a7070 con 0x557fd3c35e80 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.393+0000 7f5dee7fc640 1 -- 172.21.15.19:0/2864668427 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc07d8f0 con 0x557fd3c35e80 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.394+0000 7f5e0e21a740 1 -- 172.21.15.19:0/2864668427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 msgr2=0x557fd3c08dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.394+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/2864668427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0a9430 tx=0x7f5dfc0ab040 comp rx=0 tx=0).stop 2024-04-25T09:56:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.394+0000 7f5e0e21a740 1 -- 172.21.15.19:0/2864668427 shutdown_connections 2024-04-25T09:56:11.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.394+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/2864668427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.394+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/2864668427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.394+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/2864668427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3c38270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.394+0000 7f5e0e21a740 1 -- 172.21.15.19:0/2864668427 >> 172.21.15.19:0/2864668427 conn(0x557fd3c08700 msgr2=0x557fd3c03e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.394+0000 7f5e0e21a740 1 -- 172.21.15.19:0/2864668427 shutdown_connections 2024-04-25T09:56:11.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.394+0000 7f5e0e21a740 1 -- 172.21.15.19:0/2864668427 wait complete. 2024-04-25T09:56:11.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.394+0000 7f5e0e21a740 1 -- start start 2024-04-25T09:56:11.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.394+0000 7f5e0e21a740 1 Processor -- start 2024-04-25T09:56:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3cbce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c6d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e0e21a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041ff260 con 0x557fd3c35e80 2024-04-25T09:56:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fd3c6c6e0 con 0x557fd3c35aa0 2024-04-25T09:56:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e0e21a740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fd3c0bfa0 con 0x557fd3c2c790 2024-04-25T09:56:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e0e21a740 10 client.0 _ll_register_callbacks cb 0x7f5df400b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e099d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e099d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33794/0 (socket says 172.21.15.19:33794) 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e099d5640 1 -- 172.21.15.19:0/401148906 learned_addr learned my addr 172.21.15.19:0/401148906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e091d4640 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3cbce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.395+0000 7f5e089d3640 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c6d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e099d5640 1 -- 172.21.15.19:0/401148906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 msgr2=0x557fd3cbce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e099d5640 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3cbce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e099d5640 1 -- 172.21.15.19:0/401148906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 msgr2=0x557fd3c6d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e099d5640 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c6d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e099d5640 1 -- 172.21.15.19:0/401148906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fd3c30830 con 0x557fd3c35e80 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e091d4640 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3cbce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e089d3640 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c6d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e099d5640 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0093e0 tx=0x7f5dfc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410184ad11562a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5de3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5de3fff640 1 -- 172.21.15.19:0/401148906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc098070 con 0x557fd3c35e80 2024-04-25T09:56:11.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5de3fff640 1 -- 172.21.15.19:0/401148906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5dfc098210 con 0x557fd3c35e80 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e0e21a740 1 -- 172.21.15.19:0/401148906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5dfc0a7070 con 0x557fd3c35e80 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5de3fff640 1 -- 172.21.15.19:0/401148906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dfc0797a0 con 0x557fd3c35e80 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e0e21a740 10 client.6114 fetch_fsmap learned FSMap version 25 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e0e21a740 1 -- 172.21.15.19:0/401148906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x557fd3c50480 con 0x557fd3c35e80 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5de3fff640 1 -- 172.21.15.19:0/401148906 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc0a7070 con 0x557fd3c35e80 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5de3fff640 1 -- 172.21.15.19:0/401148906 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5dfc007bb0 con 0x557fd3c35e80 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e0e21a740 10 client.6114 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e0e21a740 10 client.6114 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e0e21a740 20 client.6114 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557fd3c63b20 0x557fd3c09310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e0e21a740 4 client.6114 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e0e21a740 1 -- 172.21.15.19:0/401148906 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557fd3b79690 con 0x557fd3c63b20 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.396+0000 7f5e091d4640 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557fd3c63b20 0x557fd3c09310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e091d4640 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557fd3c63b20 0x557fd3c09310 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4870e267b47db952 server_cookie=3b09c8b092871460 in_seq=0 out_seq=0 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5de3fff640 10 client.6114 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5de3fff640 1 -- 172.21.15.19:0/401148906 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x557fd3b79690 con 0x557fd3c63b20 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5de3fff640 10 client.6114 handle_command_reply: tid=0 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 1 client.6114 shutdown 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 1 -- 172.21.15.19:0/401148906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 msgr2=0x557fd3c08dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0093e0 tx=0x7f5dfc02a710 comp rx=0 tx=0).stop 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 1 -- 172.21.15.19:0/401148906 shutdown_connections 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557fd3c63b20 0x557fd3c09310 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fd3c35e80 0x557fd3c08dd0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557fd3c35aa0 0x557fd3c6d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5de3fff640 0 client.6114 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 1 --2- 172.21.15.19:0/401148906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557fd3c2c790 0x557fd3cbce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 1 -- 172.21.15.19:0/401148906 >> 172.21.15.19:0/401148906 conn(0x557fd3c08700 msgr2=0x557fd3c53160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 1 -- 172.21.15.19:0/401148906 shutdown_connections 2024-04-25T09:56:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 1 -- 172.21.15.19:0/401148906 wait complete. 2024-04-25T09:56:11.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.397+0000 7f5e0e21a740 20 client.6114 trim_cache size 0 max 16384 2024-04-25T09:56:11.408 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T09:56:11.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:11.408 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "backtrace", 2024-04-25T09:56:11.408 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 2006957387, 2024-04-25T09:56:11.408 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511630186, 2024-04-25T09:56:11.408 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T09:56:11.408 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:11.408 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T09:56:11.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 -- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de806b4d0 msgr2=0x7f5de806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de806b4d0 0x7f5de806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5e0415c870 tx=0x7f5e00009040 comp rx=0 tx=0).stop 2024-04-25T09:56:11.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 -- 172.21.15.19:0/3007129166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e04161710 msgr2=0x7f5e040c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e04161710 0x7f5e040c9530 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc034040 tx=0x7f5dfc004a40 comp rx=0 tx=0).stop 2024-04-25T09:56:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 -- 172.21.15.19:0/3007129166 shutdown_connections 2024-04-25T09:56:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5de806b4d0 0x7f5de806d990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e04161710 0x7f5e040c9530 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e0415dc20 0x7f5e040c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 --2- 172.21.15.19:0/3007129166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e040ced80 0x7f5e040d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 -- 172.21.15.19:0/3007129166 >> 172.21.15.19:0/3007129166 conn(0x7f5e040d0420 msgr2=0x7f5e040b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 -- 172.21.15.19:0/3007129166 shutdown_connections 2024-04-25T09:56:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.398+0000 7f5e0b438640 1 -- 172.21.15.19:0/3007129166 wait complete. 2024-04-25T09:56:11.412 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:11.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.508+0000 7fd70af09640 1 Processor -- start 2024-04-25T09:56:11.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.508+0000 7fd70af09640 1 -- start start 2024-04-25T09:56:11.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.509+0000 7fd70af09640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd70415dc70 0x7fd7040d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.509+0000 7fd70af09640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd70406ce10 0x7fd7040cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.509+0000 7fd70af09640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70406c220 con 0x7fd70409a450 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.509+0000 7fd70af09640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7040d0c90 con 0x7fd70415dc70 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.509+0000 7fd70af09640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7040d0080 con 0x7fd70406ce10 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.509+0000 7fd708ca5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd70415dc70 0x7fd7040d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.509+0000 7fd708ca5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd70415dc70 0x7fd7040d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33802/0 (socket says 172.21.15.19:33802) 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.509+0000 7fd708ca5640 1 -- 172.21.15.19:0/3354967445 learned_addr learned my addr 172.21.15.19:0/3354967445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.510+0000 7fd7094a6640 1 --2- 172.21.15.19:0/3354967445 >> v2:172.21.15.112:3300/0 conn(0x7fd70406ce10 0x7fd7040cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.510+0000 7fd708ca5640 1 -- 172.21.15.19:0/3354967445 >> v2:172.21.15.112:3300/0 conn(0x7fd70406ce10 msgr2=0x7fd7040cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.510+0000 7fd708ca5640 1 --2- 172.21.15.19:0/3354967445 >> v2:172.21.15.112:3300/0 conn(0x7fd70406ce10 0x7fd7040cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.510+0000 7fd708ca5640 1 -- 172.21.15.19:0/3354967445 >> v1:172.21.15.112:6789/0 conn(0x7fd70409a450 legacy=0x7fd70406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.510+0000 7fd708ca5640 1 -- 172.21.15.19:0/3354967445 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7040d0360 con 0x7fd70415dc70 2024-04-25T09:56:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.510+0000 7fd708ca5640 1 --2- 172.21.15.19:0/3354967445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd70415dc70 0x7fd7040d1120 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd7040b7720 tx=0x7fd6fc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd23a0d3879c1a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.511+0000 7fd7037fe640 1 -- 172.21.15.19:0/3354967445 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc037070 con 0x7fd70415dc70 2024-04-25T09:56:11.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.511+0000 7fd7037fe640 1 -- 172.21.15.19:0/3354967445 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc032440 con 0x7fd70415dc70 2024-04-25T09:56:11.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.511+0000 7fd7037fe640 1 -- 172.21.15.19:0/3354967445 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc032810 con 0x7fd70415dc70 2024-04-25T09:56:11.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.511+0000 7fd70af09640 1 -- 172.21.15.19:0/3354967445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd70415dc70 msgr2=0x7fd7040d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.511+0000 7fd70af09640 1 --2- 172.21.15.19:0/3354967445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd70415dc70 0x7fd7040d1120 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd7040b7720 tx=0x7fd6fc02a230 comp rx=0 tx=0).stop 2024-04-25T09:56:11.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.511+0000 7fd70af09640 1 -- 172.21.15.19:0/3354967445 shutdown_connections 2024-04-25T09:56:11.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.511+0000 7fd70af09640 1 --2- 172.21.15.19:0/3354967445 >> v2:172.21.15.112:3300/0 conn(0x7fd70406ce10 0x7fd7040cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.511+0000 7fd70af09640 1 --2- 172.21.15.19:0/3354967445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd70415dc70 0x7fd7040d1120 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.511+0000 7fd70af09640 1 -- 172.21.15.19:0/3354967445 >> 172.21.15.19:0/3354967445 conn(0x7fd7040b3270 msgr2=0x7fd7040b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.511+0000 7fd70af09640 1 -- 172.21.15.19:0/3354967445 shutdown_connections 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.511+0000 7fd70af09640 1 -- 172.21.15.19:0/3354967445 wait complete. 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.512+0000 7fd70af09640 1 Processor -- start 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.512+0000 7fd70af09640 1 -- start start 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.513+0000 7fd70af09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd70406ce10 0x7fd7040c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.513+0000 7fd70af09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd70409a450 0x7fd70414ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.513+0000 7fd70af09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd70415dc70 0x7fd7041f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.513+0000 7fd70af09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70415e050 con 0x7fd70406ce10 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.513+0000 7fd70af09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040d0920 con 0x7fd70415dc70 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.513+0000 7fd70af09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040cbdd0 con 0x7fd70409a450 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.513+0000 7fd708ca5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd70406ce10 0x7fd7040c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.513+0000 7fd708ca5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd70406ce10 0x7fd7040c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33804/0 (socket says 172.21.15.19:33804) 2024-04-25T09:56:11.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.513+0000 7fd708ca5640 1 -- 172.21.15.19:0/1453887653 learned_addr learned my addr 172.21.15.19:0/1453887653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd703fff640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd70409a450 0x7fd70414ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd7094a6640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd70415dc70 0x7fd7041f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd708ca5640 1 -- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd70409a450 msgr2=0x7fd70414ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd708ca5640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd70409a450 0x7fd70414ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd708ca5640 1 -- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd70415dc70 msgr2=0x7fd7041f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd708ca5640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd70415dc70 0x7fd7041f3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd708ca5640 1 -- 172.21.15.19:0/1453887653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70415e690 con 0x7fd70406ce10 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd708ca5640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd70406ce10 0x7fd7040c7ca0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc000c00 tx=0x7fd6fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd7017fa640 1 -- 172.21.15.19:0/1453887653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc041070 con 0x7fd70406ce10 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd7017fa640 1 -- 172.21.15.19:0/1453887653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc03c870 con 0x7fd70406ce10 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd70af09640 1 -- 172.21.15.19:0/1453887653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd704160a90 con 0x7fd70406ce10 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd7017fa640 1 -- 172.21.15.19:0/1453887653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc037040 con 0x7fd70406ce10 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.514+0000 7fd70af09640 1 -- 172.21.15.19:0/1453887653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd704161050 con 0x7fd70406ce10 2024-04-25T09:56:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.515+0000 7fd70af09640 1 -- 172.21.15.19:0/1453887653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd704073990 con 0x7fd70406ce10 2024-04-25T09:56:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.515+0000 7fd7017fa640 1 -- 172.21.15.19:0/1453887653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6fc044070 con 0x7fd70406ce10 2024-04-25T09:56:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.516+0000 7fd7017fa640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd6e406b4d0 0x7fd6e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.516+0000 7fd7017fa640 1 -- 172.21.15.19:0/1453887653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6fc0ab220 con 0x7fd70406ce10 2024-04-25T09:56:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.516+0000 7fd703fff640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd6e406b4d0 0x7fd6e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.516+0000 7fd703fff640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd6e406b4d0 0x7fd6e406d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4007fd0 tx=0x7fd6f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.520+0000 7fd7017fa640 1 -- 172.21.15.19:0/1453887653 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6fc07a040 con 0x7fd70406ce10 2024-04-25T09:56:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.736+0000 7fd70af09640 1 -- 172.21.15.19:0/1453887653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7040b40a0 con 0x7fd70406ce10 2024-04-25T09:56:11.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.736+0000 7fd7017fa640 1 -- 172.21.15.19:0/1453887653 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fd6fc079940 con 0x7fd70406ce10 2024-04-25T09:56:11.740 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:11.741 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:56:09.029100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:11.741 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 25 2024-04-25T09:56:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 -- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd6e406b4d0 msgr2=0x7fd6e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd6e406b4d0 0x7fd6e406d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4007fd0 tx=0x7fd6f4009040 comp rx=0 tx=0).stop 2024-04-25T09:56:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 -- 172.21.15.19:0/1453887653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd70406ce10 msgr2=0x7fd7040c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd70406ce10 0x7fd7040c7ca0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc000c00 tx=0x7fd6fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 -- 172.21.15.19:0/1453887653 shutdown_connections 2024-04-25T09:56:11.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd6e406b4d0 0x7fd6e406d990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd70415dc70 0x7fd7041f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd70409a450 0x7fd70414ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 --2- 172.21.15.19:0/1453887653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd70406ce10 0x7fd7040c7ca0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 -- 172.21.15.19:0/1453887653 >> 172.21.15.19:0/1453887653 conn(0x7fd7040b3270 msgr2=0x7fd7040b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 -- 172.21.15.19:0/1453887653 shutdown_connections 2024-04-25T09:56:11.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.738+0000 7fd70af09640 1 -- 172.21.15.19:0/1453887653 wait complete. 2024-04-25T09:56:11.753 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 2006957387 2024-04-25T09:56:11.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.853+0000 7fd8aef0a640 1 Processor -- start 2024-04-25T09:56:11.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.853+0000 7fd8aef0a640 1 -- start start 2024-04-25T09:56:11.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8aef0a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd8a80cc4f0 0x7fd8a80cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8aef0a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd8a80cdb20 0x7fd8a80cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8aef0a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a815b270 0x7fd8a815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8aef0a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a8071270 con 0x7fd8a80cdb20 2024-04-25T09:56:11.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8aef0a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a8072280 con 0x7fd8a80cc4f0 2024-04-25T09:56:11.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8aef0a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80714c0 con 0x7fd8a815b270 2024-04-25T09:56:11.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8a7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd8a80cdb20 0x7fd8a80cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8a7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd8a80cdb20 0x7fd8a80cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33820/0 (socket says 172.21.15.19:33820) 2024-04-25T09:56:11.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8a7fff640 1 -- 172.21.15.19:0/3869358974 learned_addr learned my addr 172.21.15.19:0/3869358974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8ad4a7640 1 --2- 172.21.15.19:0/3869358974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a815b270 0x7fd8a815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8acca6640 1 --2- 172.21.15.19:0/3869358974 >> v2:172.21.15.112:3300/0 conn(0x7fd8a80cc4f0 0x7fd8a80cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8a7fff640 1 -- 172.21.15.19:0/3869358974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a815b270 msgr2=0x7fd8a815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/3869358974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a815b270 0x7fd8a815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8a7fff640 1 -- 172.21.15.19:0/3869358974 >> v2:172.21.15.112:3300/0 conn(0x7fd8a80cc4f0 msgr2=0x7fd8a80cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/3869358974 >> v2:172.21.15.112:3300/0 conn(0x7fd8a80cc4f0 0x7fd8a80cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.854+0000 7fd8a7fff640 1 -- 172.21.15.19:0/3869358974 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80717a0 con 0x7fd8a80cdb20 2024-04-25T09:56:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.855+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/3869358974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8a80cdb20 0x7fd8a80cdfb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80cd290 tx=0x7fd89402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1a8f1974e8c19c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.855+0000 7fd8a77fe640 1 -- 172.21.15.19:0/3869358974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894037070 con 0x7fd8a80cdb20 2024-04-25T09:56:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.855+0000 7fd8a77fe640 1 -- 172.21.15.19:0/3869358974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd894032440 con 0x7fd8a80cdb20 2024-04-25T09:56:11.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.855+0000 7fd8a77fe640 1 -- 172.21.15.19:0/3869358974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894032810 con 0x7fd8a80cdb20 2024-04-25T09:56:11.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.856+0000 7fd8aef0a640 1 -- 172.21.15.19:0/3869358974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8a80cdb20 msgr2=0x7fd8a80cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.856+0000 7fd8aef0a640 1 --2- 172.21.15.19:0/3869358974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8a80cdb20 0x7fd8a80cdfb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80cd290 tx=0x7fd89402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.856+0000 7fd8aef0a640 1 -- 172.21.15.19:0/3869358974 shutdown_connections 2024-04-25T09:56:11.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.856+0000 7fd8aef0a640 1 --2- 172.21.15.19:0/3869358974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a815b270 0x7fd8a815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.856+0000 7fd8aef0a640 1 --2- 172.21.15.19:0/3869358974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8a80cdb20 0x7fd8a80cdfb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.856+0000 7fd8aef0a640 1 --2- 172.21.15.19:0/3869358974 >> v2:172.21.15.112:3300/0 conn(0x7fd8a80cc4f0 0x7fd8a80cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.856+0000 7fd8aef0a640 1 -- 172.21.15.19:0/3869358974 >> 172.21.15.19:0/3869358974 conn(0x7fd8a806abe0 msgr2=0x7fd8a806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.856+0000 7fd8aef0a640 1 -- 172.21.15.19:0/3869358974 shutdown_connections 2024-04-25T09:56:11.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.856+0000 7fd8aef0a640 1 -- 172.21.15.19:0/3869358974 wait complete. 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.857+0000 7fd8aef0a640 1 Processor -- start 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.857+0000 7fd8aef0a640 1 -- start start 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8aef0a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80cc4f0 0x7fd8a814a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8aef0a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a80cdb20 0x7fd8a813a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8aef0a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8a815b270 0x7fd8a813ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8aef0a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a815bc00 con 0x7fd8a815b270 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8aef0a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8071d60 con 0x7fd8a80cc4f0 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8aef0a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a815bd80 con 0x7fd8a80cdb20 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8ad4a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8a815b270 0x7fd8a813ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8ad4a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8a815b270 0x7fd8a813ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33832/0 (socket says 172.21.15.19:33832) 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8ad4a7640 1 -- 172.21.15.19:0/461906359 learned_addr learned my addr 172.21.15.19:0/461906359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8acca6640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80cc4f0 0x7fd8a814a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a80cdb20 0x7fd8a813a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8ad4a7640 1 -- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a80cdb20 msgr2=0x7fd8a813a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8ad4a7640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a80cdb20 0x7fd8a813a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8ad4a7640 1 -- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80cc4f0 msgr2=0x7fd8a814a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8ad4a7640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80cc4f0 0x7fd8a814a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8ad4a7640 1 -- 172.21.15.19:0/461906359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a8148de0 con 0x7fd8a815b270 2024-04-25T09:56:11.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a80cdb20 0x7fd8a813a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:11.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8acca6640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80cc4f0 0x7fd8a814a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:11.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.858+0000 7fd8ad4a7640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8a815b270 0x7fd8a813ad40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0007e40 tx=0x7fd8a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.859+0000 7fd8a57fa640 1 -- 172.21.15.19:0/461906359 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0014620 con 0x7fd8a815b270 2024-04-25T09:56:11.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.859+0000 7fd8a57fa640 1 -- 172.21.15.19:0/461906359 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a0019070 con 0x7fd8a815b270 2024-04-25T09:56:11.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.859+0000 7fd8aef0a640 1 -- 172.21.15.19:0/461906359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a814aa70 con 0x7fd8a815b270 2024-04-25T09:56:11.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.859+0000 7fd8a57fa640 1 -- 172.21.15.19:0/461906359 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0014a20 con 0x7fd8a815b270 2024-04-25T09:56:11.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.859+0000 7fd8aef0a640 1 -- 172.21.15.19:0/461906359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a814b030 con 0x7fd8a815b270 2024-04-25T09:56:11.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.860+0000 7fd8a57fa640 1 -- 172.21.15.19:0/461906359 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8a0014bc0 con 0x7fd8a815b270 2024-04-25T09:56:11.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.861+0000 7fd8a57fa640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd88c06b4d0 0x7fd88c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.861+0000 7fd8a57fa640 1 -- 172.21.15.19:0/461906359 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8a008acc0 con 0x7fd8a815b270 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.861+0000 7fd8acca6640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd88c06b4d0 0x7fd88c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.861+0000 7fd8acca6640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd88c06b4d0 0x7fd88c06d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00ba50 tx=0x7fd89c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.862+0000 7fd8b1c09740 1 Processor -- start 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.862+0000 7fd8b1c09740 1 -- start start 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.862+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e2ef80 0x562d71e31790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.862+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2f360 0x562d71e31cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.862+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e01be0 0x562d71e053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.862+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d71d6aac0 con 0x562d71e2f360 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.862+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d71d13db0 con 0x562d71e2ef80 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.862+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d71d66b60 con 0x562d71e01be0 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8a7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2f360 0x562d71e31cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8a7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2f360 0x562d71e31cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33844/0 (socket says 172.21.15.19:33844) 2024-04-25T09:56:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8acca6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e2ef80 0x562d71e31790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8a7fff640 1 -- 172.21.15.19:0/4144252894 learned_addr learned my addr 172.21.15.19:0/4144252894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8ad4a7640 1 --2- 172.21.15.19:0/4144252894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e01be0 0x562d71e053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8a7fff640 1 -- 172.21.15.19:0/4144252894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e01be0 msgr2=0x562d71e053f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/4144252894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e01be0 0x562d71e053f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8a7fff640 1 -- 172.21.15.19:0/4144252894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e2ef80 msgr2=0x562d71e31790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/4144252894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e2ef80 0x562d71e31790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8a7fff640 1 -- 172.21.15.19:0/4144252894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d71b4b710 con 0x562d71e2f360 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8acca6640 1 --2- 172.21.15.19:0/4144252894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e2ef80 0x562d71e31790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/4144252894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2f360 0x562d71e31cd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd89402a220 tx=0x7fd89402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1894443cf9c74b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8927fc640 1 -- 172.21.15.19:0/4144252894 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894037040 con 0x562d71e2f360 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8927fc640 1 -- 172.21.15.19:0/4144252894 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89403e720 con 0x562d71e2f360 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.863+0000 7fd8927fc640 1 -- 172.21.15.19:0/4144252894 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89403eb10 con 0x562d71e2f360 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.864+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4144252894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2f360 msgr2=0x562d71e31cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.864+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/4144252894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2f360 0x562d71e31cd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd89402a220 tx=0x7fd89402c000 comp rx=0 tx=0).stop 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.864+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4144252894 shutdown_connections 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.864+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/4144252894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e01be0 0x562d71e053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.864+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/4144252894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2f360 0x562d71e31cd0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.864+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/4144252894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e2ef80 0x562d71e31790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.864+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4144252894 >> 172.21.15.19:0/4144252894 conn(0x562d71e03e50 msgr2=0x562d71e4c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.864+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4144252894 shutdown_connections 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.864+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4144252894 wait complete. 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.864+0000 7fd8b1c09740 1 -- start start 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.865+0000 7fd8b1c09740 1 Processor -- start 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.865+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e26110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.865+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e2f360 0x562d71e50340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.865+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 0x562d71e48620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.865+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d71e07000 con 0x562d71e2ef80 2024-04-25T09:56:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.865+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d71d094e0 con 0x562d71e481a0 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.865+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d71e072e0 con 0x562d71e2f360 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8b1c09740 10 client.0 _ll_register_callbacks cb 0x562d71a7eaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8acca6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e26110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8acca6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e26110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33846/0 (socket says 172.21.15.19:33846) 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8acca6640 1 -- 172.21.15.19:0/4068384272 learned_addr learned my addr 172.21.15.19:0/4068384272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8ad4a7640 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 0x562d71e48620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e2f360 0x562d71e50340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8acca6640 1 -- 172.21.15.19:0/4068384272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e2f360 msgr2=0x562d71e50340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8acca6640 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e2f360 0x562d71e50340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8acca6640 1 -- 172.21.15.19:0/4068384272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 msgr2=0x562d71e48620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8acca6640 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 0x562d71e48620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8acca6640 1 -- 172.21.15.19:0/4068384272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d71babea0 con 0x562d71e2ef80 2024-04-25T09:56:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8ad4a7640 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 0x562d71e48620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8acca6640 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e26110 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd89c014760 tx=0x7fd89c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed5d633766f05ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8877fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8877fe640 1 -- 172.21.15.19:0/4068384272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c025070 con 0x562d71e2ef80 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8877fe640 1 -- 172.21.15.19:0/4068384272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd89c015a90 con 0x562d71e2ef80 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4068384272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd89403e720 con 0x562d71e2ef80 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8877fe640 1 -- 172.21.15.19:0/4068384272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd89c01b040 con 0x562d71e2ef80 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8b1c09740 10 client.6132 fetch_fsmap learned FSMap version 25 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.866+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4068384272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562d71a3f000 con 0x562d71e2ef80 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.867+0000 7fd8877fe640 1 -- 172.21.15.19:0/4068384272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89403e720 con 0x562d71e2ef80 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.867+0000 7fd8877fe640 1 -- 172.21.15.19:0/4068384272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd89c033080 con 0x562d71e2ef80 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.867+0000 7fd8b1c09740 10 client.6132 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.867+0000 7fd8b1c09740 10 client.6132 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:56:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.867+0000 7fd8b1c09740 20 client.6132 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.867+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d71e46fd0 0x562d71e473d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.867+0000 7fd8b1c09740 4 client.6132 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.867+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4068384272 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d71d78000 con 0x562d71e46fd0 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.867+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d71e46fd0 0x562d71e473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.867+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d71e46fd0 0x562d71e473d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e102275724e8424 server_cookie=d1a0eeaa72a9a44d in_seq=0 out_seq=0 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.867+0000 7fd8877fe640 10 client.6132 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8877fe640 1 -- 172.21.15.19:0/4068384272 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d71d78000 con 0x562d71e46fd0 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8877fe640 10 client.6132 handle_command_reply: tid=0 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 1 client.6132 shutdown 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4068384272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 msgr2=0x562d71e26110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e26110 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd89c014760 tx=0x7fd89c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4068384272 shutdown_connections 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d71e46fd0 0x562d71e473d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 0x562d71e48620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8877fe640 0 client.6132 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e2f360 0x562d71e50340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/4068384272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e26110 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4068384272 >> 172.21.15.19:0/4068384272 conn(0x562d71e01be0 msgr2=0x562d71e2e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4068384272 shutdown_connections 2024-04-25T09:56:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 1 -- 172.21.15.19:0/4068384272 wait complete. 2024-04-25T09:56:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.868+0000 7fd8b1c09740 20 client.6132 trim_cache size 0 max 16384 2024-04-25T09:56:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.880+0000 7fd8b1c09740 1 Processor -- start 2024-04-25T09:56:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.880+0000 7fd8b1c09740 1 -- start start 2024-04-25T09:56:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e2f360 0x562d71e2e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e481a0 0x562d71e022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a81fd3e0 con 0x562d71e2ef80 2024-04-25T09:56:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8147bb0 con 0x562d71e2f360 2024-04-25T09:56:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a81481b0 con 0x562d71e481a0 2024-04-25T09:56:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8acca6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8acca6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33848/0 (socket says 172.21.15.19:33848) 2024-04-25T09:56:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8acca6640 1 -- 172.21.15.19:0/695110112 learned_addr learned my addr 172.21.15.19:0/695110112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/695110112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e2f360 0x562d71e2e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8ad4a7640 1 --2- 172.21.15.19:0/695110112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e481a0 0x562d71e022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8acca6640 1 -- 172.21.15.19:0/695110112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e481a0 msgr2=0x562d71e022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8acca6640 1 --2- 172.21.15.19:0/695110112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e481a0 0x562d71e022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8acca6640 1 -- 172.21.15.19:0/695110112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e2f360 msgr2=0x562d71e2e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8acca6640 1 --2- 172.21.15.19:0/695110112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e2f360 0x562d71e2e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8acca6640 1 -- 172.21.15.19:0/695110112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d71e06db0 con 0x562d71e2ef80 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8ad4a7640 1 --2- 172.21.15.19:0/695110112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e481a0 0x562d71e022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.881+0000 7fd8acca6640 1 --2- 172.21.15.19:0/695110112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2e260 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00f9c0 tx=0x7fd89c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0fc21296de425e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd891ffb640 1 -- 172.21.15.19:0/695110112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c025040 con 0x562d71e2ef80 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd891ffb640 1 -- 172.21.15.19:0/695110112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c01c040 con 0x562d71e2ef80 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd891ffb640 1 -- 172.21.15.19:0/695110112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c02a860 con 0x562d71e2ef80 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd8b1c09740 1 -- 172.21.15.19:0/695110112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 msgr2=0x562d71e2e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/695110112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2e260 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00f9c0 tx=0x7fd89c022000 comp rx=0 tx=0).stop 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd8b1c09740 1 -- 172.21.15.19:0/695110112 shutdown_connections 2024-04-25T09:56:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/695110112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e481a0 0x562d71e022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/695110112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e2f360 0x562d71e2e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/695110112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2e260 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd8b1c09740 1 -- 172.21.15.19:0/695110112 >> 172.21.15.19:0/695110112 conn(0x562d71e01be0 msgr2=0x562d71e25f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd8b1c09740 1 -- 172.21.15.19:0/695110112 shutdown_connections 2024-04-25T09:56:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.882+0000 7fd8b1c09740 1 -- 172.21.15.19:0/695110112 wait complete. 2024-04-25T09:56:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.883+0000 7fd8b1c09740 1 -- start start 2024-04-25T09:56:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.883+0000 7fd8b1c09740 1 Processor -- start 2024-04-25T09:56:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e2f360 0x562d71e42950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8b1c09740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 0x562d71e4e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d71e06a90 con 0x562d71e2ef80 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d71e50880 con 0x562d71e481a0 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8b1c09740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d71e32a20 con 0x562d71e2f360 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8b1c09740 10 client.0 _ll_register_callbacks cb 0x7fd894035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8acca6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8acca6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33860/0 (socket says 172.21.15.19:33860) 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8acca6640 1 -- 172.21.15.19:0/1251408332 learned_addr learned my addr 172.21.15.19:0/1251408332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e2f360 0x562d71e42950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8ad4a7640 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 0x562d71e4e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8acca6640 1 -- 172.21.15.19:0/1251408332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e2f360 msgr2=0x562d71e42950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8acca6640 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e2f360 0x562d71e42950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8acca6640 1 -- 172.21.15.19:0/1251408332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 msgr2=0x562d71e4e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8acca6640 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 0x562d71e4e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8acca6640 1 -- 172.21.15.19:0/1251408332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d71e421c0 con 0x562d71e2ef80 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e2f360 0x562d71e42950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8ad4a7640 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 0x562d71e4e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8acca6640 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2bf00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00b210 tx=0x7fd89c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7df6f37dc8494d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8877fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8877fe640 1 -- 172.21.15.19:0/1251408332 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c025040 con 0x562d71e2ef80 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.884+0000 7fd8877fe640 1 -- 172.21.15.19:0/1251408332 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd89c02ac80 con 0x562d71e2ef80 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8b1c09740 1 -- 172.21.15.19:0/1251408332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd89c01c040 con 0x562d71e2ef80 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8877fe640 1 -- 172.21.15.19:0/1251408332 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd89c04a050 con 0x562d71e2ef80 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8b1c09740 10 client.6138 fetch_fsmap learned FSMap version 25 2024-04-25T09:56:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8b1c09740 1 -- 172.21.15.19:0/1251408332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562d71e297e0 con 0x562d71e2ef80 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8877fe640 1 -- 172.21.15.19:0/1251408332 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c01c040 con 0x562d71e2ef80 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8877fe640 1 -- 172.21.15.19:0/1251408332 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd89c04d020 con 0x562d71e2ef80 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8b1c09740 10 client.6138 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8b1c09740 10 client.6138 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8b1c09740 20 client.6138 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d71ebbac0 0x562d71e4e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8b1c09740 4 client.6138 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2006957387}] 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8b1c09740 1 -- 172.21.15.19:0/1251408332 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2006957387}) v1 -- 0x562d71d72350 con 0x562d71ebbac0 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d71ebbac0 0x562d71e4e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8877fe640 10 client.6138 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.885+0000 7fd8a7fff640 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d71ebbac0 0x562d71e4e7a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a767f6f99771c987 server_cookie=dfa23eea2b1ba52b in_seq=0 out_seq=0 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8877fe640 1 -- 172.21.15.19:0/1251408332 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x562d71d72350 con 0x562d71ebbac0 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8877fe640 10 client.6138 handle_command_reply: tid=0 2024-04-25T09:56:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 1 client.6138 shutdown 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 1 -- 172.21.15.19:0/1251408332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 msgr2=0x562d71e2bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2bf00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00b210 tx=0x7fd89c016000 comp rx=0 tx=0).stop 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 1 -- 172.21.15.19:0/1251408332 shutdown_connections 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d71ebbac0 0x562d71e4e7a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d71e481a0 0x562d71e4e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d71e2f360 0x562d71e42950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8877fe640 0 client.6138 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 1 --2- 172.21.15.19:0/1251408332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d71e2ef80 0x562d71e2bf00 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 1 -- 172.21.15.19:0/1251408332 >> 172.21.15.19:0/1251408332 conn(0x562d71e01be0 msgr2=0x562d71e2e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 1 -- 172.21.15.19:0/1251408332 shutdown_connections 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 1 -- 172.21.15.19:0/1251408332 wait complete. 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8b1c09740 20 client.6138 trim_cache size 0 max 16384 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8aef0a640 1 -- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd88c06b4d0 msgr2=0x7fd88c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8aef0a640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd88c06b4d0 0x7fd88c06d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00ba50 tx=0x7fd89c008040 comp rx=0 tx=0).stop 2024-04-25T09:56:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8aef0a640 1 -- 172.21.15.19:0/461906359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8a815b270 msgr2=0x7fd8a813ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.886+0000 7fd8aef0a640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8a815b270 0x7fd8a813ad40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0007e40 tx=0x7fd8a000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.887+0000 7fd8aef0a640 1 -- 172.21.15.19:0/461906359 shutdown_connections 2024-04-25T09:56:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.887+0000 7fd8aef0a640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd88c06b4d0 0x7fd88c06d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.887+0000 7fd8aef0a640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8a815b270 0x7fd8a813ad40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.887+0000 7fd8aef0a640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a80cdb20 0x7fd8a813a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.887+0000 7fd8aef0a640 1 --2- 172.21.15.19:0/461906359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a80cc4f0 0x7fd8a814a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.887+0000 7fd8aef0a640 1 -- 172.21.15.19:0/461906359 >> 172.21.15.19:0/461906359 conn(0x7fd8a806abe0 msgr2=0x7fd8a806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.887+0000 7fd8aef0a640 1 -- 172.21.15.19:0/461906359 shutdown_connections 2024-04-25T09:56:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.887+0000 7fd8aef0a640 1 -- 172.21.15.19:0/461906359 wait complete. 2024-04-25T09:56:11.901 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:56:11.901 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:12.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.997+0000 7ffa7776a640 1 Processor -- start 2024-04-25T09:56:12.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.998+0000 7ffa7776a640 1 -- start start 2024-04-25T09:56:12.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.998+0000 7ffa7776a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffa7015dbe0 0x7ffa700d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.998+0000 7ffa7776a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffa700b2cb0 0x7ffa700d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.998+0000 7ffa7776a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa700b97a0 con 0x7ffa700b92a0 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.998+0000 7ffa7776a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7015dfc0 con 0x7ffa7015dbe0 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.999+0000 7ffa7776a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa700d1a00 con 0x7ffa700b2cb0 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.999+0000 7ffa75506640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffa7015dbe0 0x7ffa700d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.999+0000 7ffa75d07640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ffa700b92a0 0x7ffa700b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41422/0 (socket says 172.21.15.19:41422) 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.999+0000 7ffa75d07640 1 -- 172.21.15.19:0/1823668909 learned_addr learned my addr 172.21.15.19:0/1823668909 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.999+0000 7ffa74d05640 1 --2- 172.21.15.19:0/1823668909 >> v2:172.21.15.112:3300/0 conn(0x7ffa700b2cb0 0x7ffa700d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.999+0000 7ffa75506640 1 -- 172.21.15.19:0/1823668909 >> v2:172.21.15.112:3300/0 conn(0x7ffa700b2cb0 msgr2=0x7ffa700d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.999+0000 7ffa75506640 1 --2- 172.21.15.19:0/1823668909 >> v2:172.21.15.112:3300/0 conn(0x7ffa700b2cb0 0x7ffa700d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.999+0000 7ffa75506640 1 -- 172.21.15.19:0/1823668909 >> v1:172.21.15.19:6789/0 conn(0x7ffa700b92a0 legacy=0x7ffa700b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.999+0000 7ffa75506640 1 -- 172.21.15.19:0/1823668909 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700d1c50 con 0x7ffa7015dbe0 2024-04-25T09:56:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:11.999+0000 7ffa6ffff640 1 -- 172.21.15.19:0/1823668909 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012174422 0 0) 0x7ffa700b97a0 con 0x7ffa700b92a0 2024-04-25T09:56:12.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.000+0000 7ffa75506640 1 --2- 172.21.15.19:0/1823668909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7015dbe0 0x7ffa700d0910 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ffa7006cff0 tx=0x7ffa6802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72beb3e0c12ddffc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:12.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.000+0000 7ffa6ffff640 1 -- 172.21.15.19:0/1823668909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68037070 con 0x7ffa7015dbe0 2024-04-25T09:56:12.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.000+0000 7ffa6ffff640 1 -- 172.21.15.19:0/1823668909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa68032440 con 0x7ffa7015dbe0 2024-04-25T09:56:12.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.000+0000 7ffa6ffff640 1 -- 172.21.15.19:0/1823668909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6802ae60 con 0x7ffa7015dbe0 2024-04-25T09:56:12.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.000+0000 7ffa7776a640 1 -- 172.21.15.19:0/1823668909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7015dbe0 msgr2=0x7ffa700d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.000+0000 7ffa7776a640 1 --2- 172.21.15.19:0/1823668909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7015dbe0 0x7ffa700d0910 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ffa7006cff0 tx=0x7ffa6802a110 comp rx=0 tx=0).stop 2024-04-25T09:56:12.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.000+0000 7ffa7776a640 1 -- 172.21.15.19:0/1823668909 shutdown_connections 2024-04-25T09:56:12.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.000+0000 7ffa7776a640 1 --2- 172.21.15.19:0/1823668909 >> v2:172.21.15.112:3300/0 conn(0x7ffa700b2cb0 0x7ffa700d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.000+0000 7ffa7776a640 1 --2- 172.21.15.19:0/1823668909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7015dbe0 0x7ffa700d0910 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.000+0000 7ffa7776a640 1 -- 172.21.15.19:0/1823668909 >> 172.21.15.19:0/1823668909 conn(0x7ffa70061a20 msgr2=0x7ffa70061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.001+0000 7ffa7776a640 1 -- 172.21.15.19:0/1823668909 shutdown_connections 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.001+0000 7ffa7776a640 1 -- 172.21.15.19:0/1823668909 wait complete. 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.002+0000 7ffa7776a640 1 Processor -- start 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.002+0000 7ffa7776a640 1 -- start start 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.002+0000 7ffa7776a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa700b2cb0 0x7ffa700b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.002+0000 7ffa7776a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa700b92a0 0x7ffa700cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.002+0000 7ffa7776a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7015dbe0 0x7ffa70151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.002+0000 7ffa7776a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700b6ab0 con 0x7ffa7015dbe0 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.002+0000 7ffa7776a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700d2210 con 0x7ffa700b92a0 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.002+0000 7ffa7776a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700b6c30 con 0x7ffa700b2cb0 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.002+0000 7ffa75d07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7015dbe0 0x7ffa70151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa75d07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7015dbe0 0x7ffa70151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33876/0 (socket says 172.21.15.19:33876) 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa75d07640 1 -- 172.21.15.19:0/2208424517 learned_addr learned my addr 172.21.15.19:0/2208424517 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa75506640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa700b2cb0 0x7ffa700b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa75d07640 1 -- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa700b2cb0 msgr2=0x7ffa700b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa74d05640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa700b92a0 0x7ffa700cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa75d07640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa700b2cb0 0x7ffa700b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa75d07640 1 -- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa700b92a0 msgr2=0x7ffa700cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa75d07640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa700b92a0 0x7ffa700cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa75d07640 1 -- 172.21.15.19:0/2208424517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700b5810 con 0x7ffa7015dbe0 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa75d07640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7015dbe0 0x7ffa70151e90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000c7e0 tx=0x7ffa6000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa6dffb640 1 -- 172.21.15.19:0/2208424517 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60018070 con 0x7ffa7015dbe0 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa6dffb640 1 -- 172.21.15.19:0/2208424517 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6000a640 con 0x7ffa7015dbe0 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa6dffb640 1 -- 172.21.15.19:0/2208424517 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000aa10 con 0x7ffa7015dbe0 2024-04-25T09:56:12.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa7776a640 1 -- 172.21.15.19:0/2208424517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa700cb920 con 0x7ffa7015dbe0 2024-04-25T09:56:12.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.003+0000 7ffa7776a640 1 -- 172.21.15.19:0/2208424517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa700b4510 con 0x7ffa7015dbe0 2024-04-25T09:56:12.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.004+0000 7ffa7776a640 1 -- 172.21.15.19:0/2208424517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa70073990 con 0x7ffa7015dbe0 2024-04-25T09:56:12.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.005+0000 7ffa6dffb640 1 -- 172.21.15.19:0/2208424517 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa6001b070 con 0x7ffa7015dbe0 2024-04-25T09:56:12.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.005+0000 7ffa6dffb640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:12.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.005+0000 7ffa6dffb640 1 -- 172.21.15.19:0/2208424517 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffa600898d0 con 0x7ffa7015dbe0 2024-04-25T09:56:12.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.005+0000 7ffa75506640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:12.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.006+0000 7ffa75506640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffa68034000 tx=0x7ffa6802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.009+0000 7ffa6dffb640 1 -- 172.21.15.19:0/2208424517 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa60058650 con 0x7ffa7015dbe0 2024-04-25T09:56:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.227+0000 7ffa7776a640 1 -- 172.21.15.19:0/2208424517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa701523d0 con 0x7ffa7015dbe0 2024-04-25T09:56:12.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.227+0000 7ffa6dffb640 1 -- 172.21.15.19:0/2208424517 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7ffa60057f50 con 0x7ffa7015dbe0 2024-04-25T09:56:12.232 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:12.233 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:56:09.029100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 25 2024-04-25T09:56:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 -- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa5c06b4d0 msgr2=0x7ffa5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffa68034000 tx=0x7ffa6802e000 comp rx=0 tx=0).stop 2024-04-25T09:56:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 -- 172.21.15.19:0/2208424517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7015dbe0 msgr2=0x7ffa70151e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7015dbe0 0x7ffa70151e90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000c7e0 tx=0x7ffa6000cca0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 -- 172.21.15.19:0/2208424517 shutdown_connections 2024-04-25T09:56:12.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7015dbe0 0x7ffa70151e90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa700b92a0 0x7ffa700cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 --2- 172.21.15.19:0/2208424517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa700b2cb0 0x7ffa700b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 -- 172.21.15.19:0/2208424517 >> 172.21.15.19:0/2208424517 conn(0x7ffa70061a20 msgr2=0x7ffa700d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:12.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 -- 172.21.15.19:0/2208424517 shutdown_connections 2024-04-25T09:56:12.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.229+0000 7ffa7776a640 1 -- 172.21.15.19:0/2208424517 wait complete. 2024-04-25T09:56:12.244 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-25T09:56:12.351 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:12.351 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T09:56:12.351 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:12.360 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:56:12.360 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:12.360 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T09:56:12.360 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T09:56:12.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:12.360 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T09:56:12.376 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T09:56:12.376 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T09:56:12.376 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T09:56:12.376 DEBUG:teuthology.orchestra.run.smithi019:> import json 2024-04-25T09:56:12.376 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-04-25T09:56:12.376 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:56:12.377 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T09:56:12.377 DEBUG:teuthology.orchestra.run.smithi019:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T09:56:12.377 DEBUG:teuthology.orchestra.run.smithi019:> except OSError as e: 2024-04-25T09:56:12.377 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(e.errno) 2024-04-25T09:56:12.377 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:56:12.377 DEBUG:teuthology.orchestra.run.smithi019:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T09:56:12.377 DEBUG:teuthology.orchestra.run.smithi019:> print(json.dumps( 2024-04-25T09:56:12.377 DEBUG:teuthology.orchestra.run.smithi019:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T09:56:12.377 DEBUG:teuthology.orchestra.run.smithi019:> indent=2)) 2024-04-25T09:56:12.377 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T09:56:12.485 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "st_mode": 16877, 2024-04-25T09:56:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "st_ino": 1099511627777, 2024-04-25T09:56:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "st_dev": 43, 2024-04-25T09:56:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "st_nlink": 16, 2024-04-25T09:56:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "st_uid": 1000, 2024-04-25T09:56:12.485 INFO:teuthology.orchestra.run.smithi019.stdout: "st_gid": 1316, 2024-04-25T09:56:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "st_size": 18, 2024-04-25T09:56:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "st_atime": 1714038958.950001, 2024-04-25T09:56:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "st_mtime": 1714038964.9209397, 2024-04-25T09:56:12.486 INFO:teuthology.orchestra.run.smithi019.stdout: "st_ctime": 1714038964.9209397 2024-04-25T09:56:12.486 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:12.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:56:12.494 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T09:56:12.494 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T09:56:12.495 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:12.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.632+0000 7fdd857f5640 1 Processor -- start 2024-04-25T09:56:12.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.632+0000 7fdd857f5640 1 -- start start 2024-04-25T09:56:12.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd857f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdd8015dbe0 0x7fdd800d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:12.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd857f5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd800b6220 con 0x7fdd800b2cb0 2024-04-25T09:56:12.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd857f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd800b97a0 con 0x7fdd800b92a0 2024-04-25T09:56:12.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd857f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd8015dfc0 con 0x7fdd8015dbe0 2024-04-25T09:56:12.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd7effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdd8015dbe0 0x7fdd800d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:12.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd7effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdd8015dbe0 0x7fdd800d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33886/0 (socket says 172.21.15.19:33886) 2024-04-25T09:56:12.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd7e7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdd800b2cb0 0x7fdd800b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41430/0 (socket says 172.21.15.19:41430) 2024-04-25T09:56:12.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd7effd640 1 -- 172.21.15.19:0/4126961635 learned_addr learned my addr 172.21.15.19:0/4126961635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:12.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd7effd640 1 -- 172.21.15.19:0/4126961635 >> v1:172.21.15.112:6789/0 conn(0x7fdd800b92a0 legacy=0x7fdd800b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:12.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd7effd640 1 -- 172.21.15.19:0/4126961635 >> v1:172.21.15.19:6789/0 conn(0x7fdd800b2cb0 legacy=0x7fdd800b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd7dffb640 1 -- 172.21.15.19:0/4126961635 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759297847 0 0) 0x7fdd800b6220 con 0x7fdd800b2cb0 2024-04-25T09:56:12.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.633+0000 7fdd7effd640 1 -- 172.21.15.19:0/4126961635 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd800d1c50 con 0x7fdd8015dbe0 2024-04-25T09:56:12.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.634+0000 7fdd7effd640 1 --2- 172.21.15.19:0/4126961635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd8015dbe0 0x7fdd800d0910 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009870 tx=0x7fdd7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c56e56ba885fa89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:12.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.634+0000 7fdd7dffb640 1 -- 172.21.15.19:0/4126961635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70037070 con 0x7fdd8015dbe0 2024-04-25T09:56:12.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.634+0000 7fdd7dffb640 1 -- 172.21.15.19:0/4126961635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70032440 con 0x7fdd8015dbe0 2024-04-25T09:56:12.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.634+0000 7fdd7dffb640 1 -- 172.21.15.19:0/4126961635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70032810 con 0x7fdd8015dbe0 2024-04-25T09:56:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.635+0000 7fdd857f5640 1 -- 172.21.15.19:0/4126961635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd8015dbe0 msgr2=0x7fdd800d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.635+0000 7fdd857f5640 1 --2- 172.21.15.19:0/4126961635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd8015dbe0 0x7fdd800d0910 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009870 tx=0x7fdd7002a270 comp rx=0 tx=0).stop 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.635+0000 7fdd857f5640 1 -- 172.21.15.19:0/4126961635 shutdown_connections 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.635+0000 7fdd857f5640 1 --2- 172.21.15.19:0/4126961635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd8015dbe0 0x7fdd800d0910 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.635+0000 7fdd857f5640 1 -- 172.21.15.19:0/4126961635 >> 172.21.15.19:0/4126961635 conn(0x7fdd80061a20 msgr2=0x7fdd80061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.635+0000 7fdd857f5640 1 -- 172.21.15.19:0/4126961635 shutdown_connections 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.635+0000 7fdd857f5640 1 -- 172.21.15.19:0/4126961635 wait complete. 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.636+0000 7fdd857f5640 1 Processor -- start 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.636+0000 7fdd857f5640 1 -- start start 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd857f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd800b2cb0 0x7fdd800caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd857f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd800b92a0 0x7fdd80151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd857f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8015dbe0 0x7fdd8015e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd857f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800d2210 con 0x7fdd800b2cb0 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd857f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800d1a00 con 0x7fdd800b92a0 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd857f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800b6ab0 con 0x7fdd8015dbe0 2024-04-25T09:56:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd7effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd800b2cb0 0x7fdd800caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd7effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd800b2cb0 0x7fdd800caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33892/0 (socket says 172.21.15.19:33892) 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd7effd640 1 -- 172.21.15.19:0/3364325802 learned_addr learned my addr 172.21.15.19:0/3364325802 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd7e7fc640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd800b92a0 0x7fdd80151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd7f7fe640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8015dbe0 0x7fdd8015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd7effd640 1 -- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8015dbe0 msgr2=0x7fdd8015e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd7effd640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8015dbe0 0x7fdd8015e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd7effd640 1 -- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd800b92a0 msgr2=0x7fdd80151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd7effd640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd800b92a0 0x7fdd80151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd7effd640 1 -- 172.21.15.19:0/3364325802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8015f0f0 con 0x7fdd800b2cb0 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.637+0000 7fdd7effd640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd800b2cb0 0x7fdd800caa50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009870 tx=0x7fdd700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.638+0000 7fdd677fe640 1 -- 172.21.15.19:0/3364325802 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70041070 con 0x7fdd800b2cb0 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.638+0000 7fdd677fe640 1 -- 172.21.15.19:0/3364325802 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7003c870 con 0x7fdd800b2cb0 2024-04-25T09:56:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.638+0000 7fdd857f5640 1 -- 172.21.15.19:0/3364325802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8015eb30 con 0x7fdd800b2cb0 2024-04-25T09:56:12.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.638+0000 7fdd677fe640 1 -- 172.21.15.19:0/3364325802 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70037040 con 0x7fdd800b2cb0 2024-04-25T09:56:12.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.638+0000 7fdd857f5640 1 -- 172.21.15.19:0/3364325802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8015fad0 con 0x7fdd800b2cb0 2024-04-25T09:56:12.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.639+0000 7fdd857f5640 1 -- 172.21.15.19:0/3364325802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd80073990 con 0x7fdd800b2cb0 2024-04-25T09:56:12.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.639+0000 7fdd677fe640 1 -- 172.21.15.19:0/3364325802 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd70044070 con 0x7fdd800b2cb0 2024-04-25T09:56:12.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.640+0000 7fdd677fe640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6c06b4d0 0x7fdd6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:12.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.640+0000 7fdd677fe640 1 -- 172.21.15.19:0/3364325802 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd700aae90 con 0x7fdd800b2cb0 2024-04-25T09:56:12.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.640+0000 7fdd7e7fc640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6c06b4d0 0x7fdd6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:12.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.640+0000 7fdd7e7fc640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6c06b4d0 0x7fdd6c06d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdd800cdb70 tx=0x7fdd74007b90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:12.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.644+0000 7fdd677fe640 1 -- 172.21.15.19:0/3364325802 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd70079c10 con 0x7fdd800b2cb0 2024-04-25T09:56:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.862+0000 7fdd857f5640 1 -- 172.21.15.19:0/3364325802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd80061e00 con 0x7fdd800b2cb0 2024-04-25T09:56:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.863+0000 7fdd677fe640 1 -- 172.21.15.19:0/3364325802 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fdd70079510 con 0x7fdd800b2cb0 2024-04-25T09:56:12.867 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:56:09.029100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:12.867 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 25 2024-04-25T09:56:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 -- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6c06b4d0 msgr2=0x7fdd6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6c06b4d0 0x7fdd6c06d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdd800cdb70 tx=0x7fdd74007b90 comp rx=0 tx=0).stop 2024-04-25T09:56:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 -- 172.21.15.19:0/3364325802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd800b2cb0 msgr2=0x7fdd800caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd800b2cb0 0x7fdd800caa50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009870 tx=0x7fdd700046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 -- 172.21.15.19:0/3364325802 shutdown_connections 2024-04-25T09:56:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6c06b4d0 0x7fdd6c06d990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8015dbe0 0x7fdd8015e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd800b92a0 0x7fdd80151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 --2- 172.21.15.19:0/3364325802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd800b2cb0 0x7fdd800caa50 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:12.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 -- 172.21.15.19:0/3364325802 >> 172.21.15.19:0/3364325802 conn(0x7fdd80061a20 msgr2=0x7fdd8015b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:12.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 -- 172.21.15.19:0/3364325802 shutdown_connections 2024-04-25T09:56:12.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:12.865+0000 7fdd857f5640 1 -- 172.21.15.19:0/3364325802 wait complete. 2024-04-25T09:56:12.879 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T09:56:12.986 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:12.986 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": -2 2024-04-25T09:56:12.986 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:12.994 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T09:56:12.995 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T09:56:12.995 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:12.995 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T09:56:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.095+0000 7fe222cb3640 1 Processor -- start 2024-04-25T09:56:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.095+0000 7fe222cb3640 1 -- start start 2024-04-25T09:56:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe222cb3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 0x7fe21c0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe222cb3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe21c0d1b80 0x7fe21c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe222cb3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe21c06c870 0x7fe21c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe222cb3640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe21c158610 con 0x7fe21c0d1b80 2024-04-25T09:56:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe222cb3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe21c06bea0 con 0x7fe21c06c870 2024-04-25T09:56:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe222cb3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c0d2020 con 0x7fe21c158230 2024-04-25T09:56:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe213fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe21c0d1b80 0x7fe21c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe213fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe21c0d1b80 0x7fe21c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33900/0 (socket says 172.21.15.19:33900) 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe213fff640 1 -- 172.21.15.19:0/943136050 learned_addr learned my addr 172.21.15.19:0/943136050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe221250640 1 --2- 172.21.15.19:0/943136050 >> v2:172.21.15.112:3300/0 conn(0x7fe21c06c870 0x7fe21c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe213fff640 1 -- 172.21.15.19:0/943136050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 msgr2=0x7fe21c0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe220a4f640 1 --2- 172.21.15.19:0/943136050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 0x7fe21c0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe213fff640 1 --2- 172.21.15.19:0/943136050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 0x7fe21c0d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe213fff640 1 -- 172.21.15.19:0/943136050 >> v2:172.21.15.112:3300/0 conn(0x7fe21c06c870 msgr2=0x7fe21c0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe213fff640 1 --2- 172.21.15.19:0/943136050 >> v2:172.21.15.112:3300/0 conn(0x7fe21c06c870 0x7fe21c0b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe213fff640 1 -- 172.21.15.19:0/943136050 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c0d2300 con 0x7fe21c0d1b80 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.096+0000 7fe220a4f640 1 --2- 172.21.15.19:0/943136050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 0x7fe21c0d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.097+0000 7fe213fff640 1 --2- 172.21.15.19:0/943136050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe21c0d1b80 0x7fe21c06c330 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe21c0d13e0 tx=0x7fe20c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42c05a73ba170fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.097+0000 7fe2137fe640 1 -- 172.21.15.19:0/943136050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c004030 con 0x7fe21c0d1b80 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.097+0000 7fe2137fe640 1 -- 172.21.15.19:0/943136050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20c0041d0 con 0x7fe21c0d1b80 2024-04-25T09:56:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.097+0000 7fe2137fe640 1 -- 172.21.15.19:0/943136050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c003610 con 0x7fe21c0d1b80 2024-04-25T09:56:13.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.098+0000 7fe222cb3640 1 -- 172.21.15.19:0/943136050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe21c0d1b80 msgr2=0x7fe21c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.098+0000 7fe222cb3640 1 --2- 172.21.15.19:0/943136050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe21c0d1b80 0x7fe21c06c330 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe21c0d13e0 tx=0x7fe20c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.098+0000 7fe222cb3640 1 -- 172.21.15.19:0/943136050 shutdown_connections 2024-04-25T09:56:13.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.098+0000 7fe222cb3640 1 --2- 172.21.15.19:0/943136050 >> v2:172.21.15.112:3300/0 conn(0x7fe21c06c870 0x7fe21c0b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.098+0000 7fe222cb3640 1 --2- 172.21.15.19:0/943136050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe21c0d1b80 0x7fe21c06c330 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.098+0000 7fe222cb3640 1 --2- 172.21.15.19:0/943136050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 0x7fe21c0d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.098+0000 7fe222cb3640 1 -- 172.21.15.19:0/943136050 >> 172.21.15.19:0/943136050 conn(0x7fe21c061600 msgr2=0x7fe21c0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.098+0000 7fe222cb3640 1 -- 172.21.15.19:0/943136050 shutdown_connections 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.098+0000 7fe222cb3640 1 -- 172.21.15.19:0/943136050 wait complete. 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.099+0000 7fe222cb3640 1 Processor -- start 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.099+0000 7fe222cb3640 1 -- start start 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe222cb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe21c06c870 0x7fe21c1e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe222cb3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe21c0d1b80 0x7fe21c0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe222cb3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 0x7fe21c14c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe222cb3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c15e320 con 0x7fe21c0d1b80 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe222cb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c0d25e0 con 0x7fe21c06c870 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe222cb3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c15e4a0 con 0x7fe21c158230 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe213fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe21c0d1b80 0x7fe21c0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe213fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe21c0d1b80 0x7fe21c0cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33906/0 (socket says 172.21.15.19:33906) 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe213fff640 1 -- 172.21.15.19:0/3483953840 learned_addr learned my addr 172.21.15.19:0/3483953840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe220a4f640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe21c06c870 0x7fe21c1e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe221250640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 0x7fe21c14c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe213fff640 1 -- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 msgr2=0x7fe21c14c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe213fff640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 0x7fe21c14c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe213fff640 1 -- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe21c06c870 msgr2=0x7fe21c1e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe213fff640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe21c06c870 0x7fe21c1e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe213fff640 1 -- 172.21.15.19:0/3483953840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c0cc850 con 0x7fe21c0d1b80 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe220a4f640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe21c06c870 0x7fe21c1e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe221250640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 0x7fe21c14c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.100+0000 7fe213fff640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe21c0d1b80 0x7fe21c0cc020 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe20c02a200 tx=0x7fe20c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.101+0000 7fe2117fa640 1 -- 172.21.15.19:0/3483953840 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c041020 con 0x7fe21c0d1b80 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.101+0000 7fe2117fa640 1 -- 172.21.15.19:0/3483953840 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20c003c50 con 0x7fe21c0d1b80 2024-04-25T09:56:13.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.101+0000 7fe222cb3640 1 -- 172.21.15.19:0/3483953840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c0ccb30 con 0x7fe21c0d1b80 2024-04-25T09:56:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.101+0000 7fe2117fa640 1 -- 172.21.15.19:0/3483953840 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c004030 con 0x7fe21c0d1b80 2024-04-25T09:56:13.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.101+0000 7fe222cb3640 1 -- 172.21.15.19:0/3483953840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c0cd0f0 con 0x7fe21c0d1b80 2024-04-25T09:56:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.102+0000 7fe2117fa640 1 -- 172.21.15.19:0/3483953840 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe20c044020 con 0x7fe21c0d1b80 2024-04-25T09:56:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.103+0000 7fe2117fa640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe20806b4d0 0x7fe20806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.103+0000 7fe2117fa640 1 -- 172.21.15.19:0/3483953840 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe20c0aae70 con 0x7fe21c0d1b80 2024-04-25T09:56:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.103+0000 7fe220a4f640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe20806b4d0 0x7fe20806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.104+0000 7fe220a4f640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe20806b4d0 0x7fe20806d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe2180088b0 tx=0x7fe218008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.104+0000 7fe2259bc740 1 Processor -- start 2024-04-25T09:56:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.104+0000 7fe2259bc740 1 -- start start 2024-04-25T09:56:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcb69a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb86f50 0x5622dcb874a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb3f390 0x5622dcb85610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe2259bc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dcaa1c60 con 0x5622dcb3f390 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe2259bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dca4aee0 con 0x5622dcb696b0 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe2259bc740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622dca9dd00 con 0x5622dcb86f50 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe221250640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb3f390 0x5622dcb85610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe221250640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb3f390 0x5622dcb85610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33920/0 (socket says 172.21.15.19:33920) 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe221250640 1 -- 172.21.15.19:0/1816327632 learned_addr learned my addr 172.21.15.19:0/1816327632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe213fff640 1 --2- 172.21.15.19:0/1816327632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb86f50 0x5622dcb874a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe220a4f640 1 --2- 172.21.15.19:0/1816327632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcb69a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe221250640 1 -- 172.21.15.19:0/1816327632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb86f50 msgr2=0x5622dcb874a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe221250640 1 --2- 172.21.15.19:0/1816327632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb86f50 0x5622dcb874a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe221250640 1 -- 172.21.15.19:0/1816327632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 msgr2=0x5622dcb69a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe221250640 1 --2- 172.21.15.19:0/1816327632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcb69a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe221250640 1 -- 172.21.15.19:0/1816327632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622dc8829f0 con 0x5622dcb3f390 2024-04-25T09:56:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe220a4f640 1 --2- 172.21.15.19:0/1816327632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcb69a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe213fff640 1 --2- 172.21.15.19:0/1816327632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb86f50 0x5622dcb874a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe221250640 1 --2- 172.21.15.19:0/1816327632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb3f390 0x5622dcb85610 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe214009fb0 tx=0x7fe2140077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d5df72cb2b2bca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.105+0000 7fe2027fc640 1 -- 172.21.15.19:0/1816327632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21401a070 con 0x5622dcb3f390 2024-04-25T09:56:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.106+0000 7fe2027fc640 1 -- 172.21.15.19:0/1816327632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe214003da0 con 0x5622dcb3f390 2024-04-25T09:56:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.106+0000 7fe2027fc640 1 -- 172.21.15.19:0/1816327632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214004230 con 0x5622dcb3f390 2024-04-25T09:56:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.106+0000 7fe2259bc740 1 -- 172.21.15.19:0/1816327632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb3f390 msgr2=0x5622dcb85610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.106+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1816327632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb3f390 0x5622dcb85610 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe214009fb0 tx=0x7fe2140077c0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.106+0000 7fe2259bc740 1 -- 172.21.15.19:0/1816327632 shutdown_connections 2024-04-25T09:56:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.106+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1816327632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb3f390 0x5622dcb85610 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.106+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1816327632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb86f50 0x5622dcb874a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.106+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1816327632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcb69a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.106+0000 7fe2259bc740 1 -- 172.21.15.19:0/1816327632 >> 172.21.15.19:0/1816327632 conn(0x5622dcb3bcc0 msgr2=0x5622dcb844c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.106+0000 7fe2259bc740 1 -- 172.21.15.19:0/1816327632 shutdown_connections 2024-04-25T09:56:13.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.106+0000 7fe2259bc740 1 -- 172.21.15.19:0/1816327632 wait complete. 2024-04-25T09:56:13.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.107+0000 7fe2259bc740 1 -- start start 2024-04-25T09:56:13.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.107+0000 7fe2259bc740 1 Processor -- start 2024-04-25T09:56:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcb7b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb7bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb948f0 0x5622dcb94d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe2259bc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dcb40130 con 0x5622dcb86f50 2024-04-25T09:56:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe2259bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dca5e470 con 0x5622dcb696b0 2024-04-25T09:56:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe2259bc740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622dcb39bf0 con 0x5622dcb948f0 2024-04-25T09:56:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe2259bc740 10 client.0 _ll_register_callbacks cb 0x5622dca2a7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe213fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb7bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe213fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb7bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33922/0 (socket says 172.21.15.19:33922) 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe213fff640 1 -- 172.21.15.19:0/643954099 learned_addr learned my addr 172.21.15.19:0/643954099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe220a4f640 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcb7b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe221250640 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb948f0 0x5622dcb94d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe213fff640 1 -- 172.21.15.19:0/643954099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb948f0 msgr2=0x5622dcb94d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe213fff640 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb948f0 0x5622dcb94d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe213fff640 1 -- 172.21.15.19:0/643954099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 msgr2=0x5622dcb7b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe213fff640 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcb7b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe213fff640 1 -- 172.21.15.19:0/643954099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622dc7762e0 con 0x5622dcb86f50 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe213fff640 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb7bb60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe20c075950 tx=0x7fe20c0795b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf84b263a3d8bab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe1fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/643954099 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c081970 con 0x5622dcb86f50 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/643954099 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe20c07d170 con 0x5622dcb86f50 2024-04-25T09:56:13.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.108+0000 7fe2259bc740 1 -- 172.21.15.19:0/643954099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe214003da0 con 0x5622dcb86f50 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/643954099 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe20c07d370 con 0x5622dcb86f50 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe2259bc740 10 client.6168 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe2259bc740 1 -- 172.21.15.19:0/643954099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5622dc8e3180 con 0x5622dcb86f50 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/643954099 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe214003da0 con 0x5622dcb86f50 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/643954099 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe20c078940 con 0x5622dcb86f50 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe2259bc740 10 client.6168 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe2259bc740 10 client.6168 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe2259bc740 20 client.6168 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe2259bc740 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5622dcb9f090 0x5622dcba1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe2259bc740 4 client.6168 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe2259bc740 1 -- 172.21.15.19:0/643954099 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622dcaaf1a0 con 0x5622dcb9f090 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe220a4f640 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5622dcb9f090 0x5622dcba1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe220a4f640 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5622dcb9f090 0x5622dcba1480 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f5f05ff31a5cfe server_cookie=db44d4079a68d3b8 in_seq=0 out_seq=0 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.109+0000 7fe1fb7fe640 10 client.6168 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/643954099 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5622dcaaf1a0 con 0x5622dcb9f090 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe1fb7fe640 10 client.6168 handle_command_reply: tid=0 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 1 client.6168 shutdown 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 1 -- 172.21.15.19:0/643954099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 msgr2=0x5622dcb7bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb7bb60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe20c075950 tx=0x7fe20c0795b0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 1 -- 172.21.15.19:0/643954099 shutdown_connections 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5622dcb9f090 0x5622dcba1480 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb948f0 0x5622dcb94d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb7bb60 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 1 --2- 172.21.15.19:0/643954099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcb7b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe1fb7fe640 0 client.6168 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 1 -- 172.21.15.19:0/643954099 >> 172.21.15.19:0/643954099 conn(0x5622dcb3f390 msgr2=0x5622dcb882d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 1 -- 172.21.15.19:0/643954099 shutdown_connections 2024-04-25T09:56:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 1 -- 172.21.15.19:0/643954099 wait complete. 2024-04-25T09:56:13.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.110+0000 7fe2259bc740 20 client.6168 trim_cache size 0 max 16384 2024-04-25T09:56:13.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.126+0000 7fe2259bc740 1 Processor -- start 2024-04-25T09:56:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.126+0000 7fe2259bc740 1 -- start start 2024-04-25T09:56:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.126+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb696b0 0x5622dcb3e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.126+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb86f50 0x5622dcb3eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.126+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb948f0 0x5622dcb3d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.126+0000 7fe2259bc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c2003d0 con 0x5622dcb948f0 2024-04-25T09:56:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.126+0000 7fe2259bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1ff5e0 con 0x5622dcb86f50 2024-04-25T09:56:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.126+0000 7fe2259bc740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c15a5b0 con 0x5622dcb696b0 2024-04-25T09:56:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.126+0000 7fe221250640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb948f0 0x5622dcb3d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.126+0000 7fe221250640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb948f0 0x5622dcb3d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33926/0 (socket says 172.21.15.19:33926) 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.126+0000 7fe221250640 1 -- 172.21.15.19:0/1879548168 learned_addr learned my addr 172.21.15.19:0/1879548168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.127+0000 7fe221250640 1 -- 172.21.15.19:0/1879548168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb696b0 msgr2=0x5622dcb3e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.127+0000 7fe221250640 1 --2- 172.21.15.19:0/1879548168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb696b0 0x5622dcb3e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.127+0000 7fe221250640 1 -- 172.21.15.19:0/1879548168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb86f50 msgr2=0x5622dcb3eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.127+0000 7fe221250640 1 --2- 172.21.15.19:0/1879548168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb86f50 0x5622dcb3eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.127+0000 7fe221250640 1 -- 172.21.15.19:0/1879548168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622dcb7f9d0 con 0x5622dcb948f0 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.127+0000 7fe221250640 1 --2- 172.21.15.19:0/1879548168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb948f0 0x5622dcb3d260 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe214014000 tx=0x7fe214013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65ba7bcdd57a000 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.127+0000 7fe201ffb640 1 -- 172.21.15.19:0/1879548168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214026020 con 0x5622dcb948f0 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.127+0000 7fe201ffb640 1 -- 172.21.15.19:0/1879548168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21401b040 con 0x5622dcb948f0 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.127+0000 7fe201ffb640 1 -- 172.21.15.19:0/1879548168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21401a070 con 0x5622dcb948f0 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.127+0000 7fe2259bc740 1 -- 172.21.15.19:0/1879548168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb948f0 msgr2=0x5622dcb3d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.127+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1879548168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb948f0 0x5622dcb3d260 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe214014000 tx=0x7fe214013000 comp rx=0 tx=0).stop 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.128+0000 7fe2259bc740 1 -- 172.21.15.19:0/1879548168 shutdown_connections 2024-04-25T09:56:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.128+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1879548168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb948f0 0x5622dcb3d260 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.128+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1879548168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb86f50 0x5622dcb3eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.128+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1879548168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb696b0 0x5622dcb3e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.128+0000 7fe2259bc740 1 -- 172.21.15.19:0/1879548168 >> 172.21.15.19:0/1879548168 conn(0x5622dcb3f390 msgr2=0x5622dcb7cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.128+0000 7fe2259bc740 1 -- 172.21.15.19:0/1879548168 shutdown_connections 2024-04-25T09:56:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.128+0000 7fe2259bc740 1 -- 172.21.15.19:0/1879548168 wait complete. 2024-04-25T09:56:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.128+0000 7fe2259bc740 1 -- start start 2024-04-25T09:56:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.128+0000 7fe2259bc740 1 Processor -- start 2024-04-25T09:56:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcbe91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb62900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe2259bc740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb948f0 0x5622dcbf2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe2259bc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dcb7d830 con 0x5622dcb86f50 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe2259bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c15abb0 con 0x5622dcb696b0 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe2259bc740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622dcb3f1e0 con 0x5622dcb948f0 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe2259bc740 10 client.0 _ll_register_callbacks cb 0x5622dcb3a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe213fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb62900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe213fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb62900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33930/0 (socket says 172.21.15.19:33930) 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe213fff640 1 -- 172.21.15.19:0/1039918817 learned_addr learned my addr 172.21.15.19:0/1039918817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe220a4f640 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcbe91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe221250640 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb948f0 0x5622dcbf2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.129+0000 7fe213fff640 1 -- 172.21.15.19:0/1039918817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb948f0 msgr2=0x5622dcbf2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe213fff640 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb948f0 0x5622dcbf2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe213fff640 1 -- 172.21.15.19:0/1039918817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 msgr2=0x5622dcbe91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe213fff640 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcbe91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe213fff640 1 -- 172.21.15.19:0/1039918817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622dc74f3a0 con 0x5622dcb86f50 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe221250640 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb948f0 0x5622dcbf2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe213fff640 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb62900 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe20c02a7b0 tx=0x7fe20c009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101e32c41cb54ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe1fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/1039918817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c078e50 con 0x5622dcb86f50 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/1039918817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe20c0819a0 con 0x5622dcb86f50 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe2259bc740 1 -- 172.21.15.19:0/1039918817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe21401b040 con 0x5622dcb86f50 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/1039918817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe20c075790 con 0x5622dcb86f50 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe2259bc740 10 client.6174 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe2259bc740 1 -- 172.21.15.19:0/1039918817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5622dcba5f50 con 0x5622dcb86f50 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/1039918817 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21401b040 con 0x5622dcb86f50 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/1039918817 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe20c0783b0 con 0x5622dcb86f50 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe2259bc740 10 client.6174 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe2259bc740 10 client.6174 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe2259bc740 20 client.6174 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5622dcb7f5d0 0x5622dcb7b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe2259bc740 4 client.6174 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe2259bc740 1 -- 172.21.15.19:0/1039918817 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x5622dcaa94f0 con 0x5622dcb7f5d0 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.130+0000 7fe220a4f640 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5622dcb7f5d0 0x5622dcb7b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe1fb7fe640 10 client.6174 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe220a4f640 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5622dcb7f5d0 0x5622dcb7b410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98facbab8eba0a89 server_cookie=813bb67d69a7073c in_seq=0 out_seq=0 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe1fb7fe640 1 -- 172.21.15.19:0/1039918817 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x5622dcaa94f0 con 0x5622dcb7f5d0 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe1fb7fe640 10 client.6174 handle_command_reply: tid=0 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 1 client.6174 shutdown 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 1 -- 172.21.15.19:0/1039918817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 msgr2=0x5622dcb62900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb62900 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe20c02a7b0 tx=0x7fe20c009210 comp rx=0 tx=0).stop 2024-04-25T09:56:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 1 -- 172.21.15.19:0/1039918817 shutdown_connections 2024-04-25T09:56:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5622dcb7f5d0 0x5622dcb7b410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5622dcb948f0 0x5622dcbf2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622dcb86f50 0x5622dcb62900 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe1fb7fe640 0 client.6174 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 1 --2- 172.21.15.19:0/1039918817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622dcb696b0 0x5622dcbe91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 1 -- 172.21.15.19:0/1039918817 >> 172.21.15.19:0/1039918817 conn(0x5622dcb3f390 msgr2=0x5622dcb3e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 1 -- 172.21.15.19:0/1039918817 shutdown_connections 2024-04-25T09:56:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 1 -- 172.21.15.19:0/1039918817 wait complete. 2024-04-25T09:56:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.131+0000 7fe2259bc740 20 client.6174 trim_cache size 0 max 16384 2024-04-25T09:56:13.143 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:13.144 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": -2 2024-04-25T09:56:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 -- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe20806b4d0 msgr2=0x7fe20806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe20806b4d0 0x7fe20806d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe2180088b0 tx=0x7fe218008040 comp rx=0 tx=0).stop 2024-04-25T09:56:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 -- 172.21.15.19:0/3483953840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe21c0d1b80 msgr2=0x7fe21c0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe21c0d1b80 0x7fe21c0cc020 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe20c02a200 tx=0x7fe20c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 -- 172.21.15.19:0/3483953840 shutdown_connections 2024-04-25T09:56:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe20806b4d0 0x7fe20806d990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe21c158230 0x7fe21c14c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe21c0d1b80 0x7fe21c0cc020 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 --2- 172.21.15.19:0/3483953840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe21c06c870 0x7fe21c1e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 -- 172.21.15.19:0/3483953840 >> 172.21.15.19:0/3483953840 conn(0x7fe21c061600 msgr2=0x7fe21c15d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 -- 172.21.15.19:0/3483953840 shutdown_connections 2024-04-25T09:56:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.132+0000 7fe222cb3640 1 -- 172.21.15.19:0/3483953840 wait complete. 2024-04-25T09:56:13.146 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T09:56:13.146 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:56:13.147 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:13.147 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:56:13.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.246+0000 7f6a46e4a640 1 Processor -- start 2024-04-25T09:56:13.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.246+0000 7f6a46e4a640 1 -- start start 2024-04-25T09:56:13.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a46e4a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6a400b5620 0x7f6a400b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a46e4a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6a4006c2d0 0x7f6a4006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a46e4a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a4006cc80 con 0x7f6a4015b7d0 2024-04-25T09:56:13.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a46e4a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4006cf50 con 0x7f6a400b5620 2024-04-25T09:56:13.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a46e4a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a400ce530 con 0x7f6a4006c2d0 2024-04-25T09:56:13.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a3ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6a400b5620 0x7f6a400b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a44be6640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6a4015b7d0 0x7f6a400cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41438/0 (socket says 172.21.15.19:41438) 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a44be6640 1 -- 172.21.15.19:0/454612066 learned_addr learned my addr 172.21.15.19:0/454612066 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a453e7640 1 --2- 172.21.15.19:0/454612066 >> v2:172.21.15.112:3300/0 conn(0x7f6a4006c2d0 0x7f6a4006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a3ffff640 1 -- 172.21.15.19:0/454612066 >> v2:172.21.15.112:3300/0 conn(0x7f6a4006c2d0 msgr2=0x7f6a4006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/454612066 >> v2:172.21.15.112:3300/0 conn(0x7f6a4006c2d0 0x7f6a4006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a3ffff640 1 -- 172.21.15.19:0/454612066 >> v1:172.21.15.19:6789/0 conn(0x7f6a4015b7d0 legacy=0x7f6a400cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.248+0000 7f6a3ffff640 1 -- 172.21.15.19:0/454612066 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a400ce810 con 0x7f6a400b5620 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.247+0000 7f6a3f7fe640 1 -- 172.21.15.19:0/454612066 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354529136 0 0) 0x7f6a4006cc80 con 0x7f6a4015b7d0 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.248+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/454612066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a400b5620 0x7f6a400b5ab0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6a40161890 tx=0x7f6a3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1ca2be53920732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.248+0000 7f6a3f7fe640 1 -- 172.21.15.19:0/454612066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34037070 con 0x7f6a400b5620 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.248+0000 7f6a3f7fe640 1 -- 172.21.15.19:0/454612066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a34032440 con 0x7f6a400b5620 2024-04-25T09:56:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.248+0000 7f6a3f7fe640 1 -- 172.21.15.19:0/454612066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34032810 con 0x7f6a400b5620 2024-04-25T09:56:13.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.249+0000 7f6a46e4a640 1 -- 172.21.15.19:0/454612066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a400b5620 msgr2=0x7f6a400b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.249+0000 7f6a46e4a640 1 --2- 172.21.15.19:0/454612066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a400b5620 0x7f6a400b5ab0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6a40161890 tx=0x7f6a3402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.249+0000 7f6a46e4a640 1 -- 172.21.15.19:0/454612066 shutdown_connections 2024-04-25T09:56:13.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.249+0000 7f6a46e4a640 1 --2- 172.21.15.19:0/454612066 >> v2:172.21.15.112:3300/0 conn(0x7f6a4006c2d0 0x7f6a4006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.249+0000 7f6a46e4a640 1 --2- 172.21.15.19:0/454612066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a400b5620 0x7f6a400b5ab0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.249+0000 7f6a46e4a640 1 -- 172.21.15.19:0/454612066 >> 172.21.15.19:0/454612066 conn(0x7f6a400d0800 msgr2=0x7f6a400d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.249+0000 7f6a46e4a640 1 -- 172.21.15.19:0/454612066 shutdown_connections 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.249+0000 7f6a46e4a640 1 -- 172.21.15.19:0/454612066 wait complete. 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.250+0000 7f6a46e4a640 1 Processor -- start 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.250+0000 7f6a46e4a640 1 -- start start 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a46e4a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4006c2d0 0x7f6a40157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a46e4a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6a400b5620 0x7f6a400c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a46e4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a4015b7d0 0x7f6a40160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a46e4a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4015bbb0 con 0x7f6a4015b7d0 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a46e4a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4006d230 con 0x7f6a4006c2d0 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a46e4a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4015bd30 con 0x7f6a400b5620 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a453e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a4015b7d0 0x7f6a40160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a453e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a4015b7d0 0x7f6a40160050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33946/0 (socket says 172.21.15.19:33946) 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a453e7640 1 -- 172.21.15.19:0/3824261478 learned_addr learned my addr 172.21.15.19:0/3824261478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6a400b5620 0x7f6a400c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a44be6640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4006c2d0 0x7f6a40157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a453e7640 1 -- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6a400b5620 msgr2=0x7f6a400c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a453e7640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6a400b5620 0x7f6a400c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a453e7640 1 -- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4006c2d0 msgr2=0x7f6a40157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a453e7640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4006c2d0 0x7f6a40157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a453e7640 1 -- 172.21.15.19:0/3824261478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a400cd050 con 0x7f6a4015b7d0 2024-04-25T09:56:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.251+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6a400b5620 0x7f6a400c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.252+0000 7f6a453e7640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a4015b7d0 0x7f6a40160050 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000e860 tx=0x7f6a3000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.252+0000 7f6a3d7fa640 1 -- 172.21.15.19:0/3824261478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30017070 con 0x7f6a4015b7d0 2024-04-25T09:56:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.252+0000 7f6a3d7fa640 1 -- 172.21.15.19:0/3824261478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3000c6d0 con 0x7f6a4015b7d0 2024-04-25T09:56:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.252+0000 7f6a46e4a640 1 -- 172.21.15.19:0/3824261478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a400cd330 con 0x7f6a4015b7d0 2024-04-25T09:56:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.252+0000 7f6a3d7fa640 1 -- 172.21.15.19:0/3824261478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3000cac0 con 0x7f6a4015b7d0 2024-04-25T09:56:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.252+0000 7f6a46e4a640 1 -- 172.21.15.19:0/3824261478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a400cdbd0 con 0x7f6a4015b7d0 2024-04-25T09:56:13.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.253+0000 7f6a3d7fa640 1 -- 172.21.15.19:0/3824261478 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a3001a070 con 0x7f6a4015b7d0 2024-04-25T09:56:13.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.254+0000 7f6a3d7fa640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6a2c06b4d0 0x7f6a2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.254+0000 7f6a3d7fa640 1 -- 172.21.15.19:0/3824261478 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a3008a9b0 con 0x7f6a4015b7d0 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.254+0000 7f6a44be6640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6a2c06b4d0 0x7f6a2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.254+0000 7f6a44be6640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6a2c06b4d0 0x7f6a2c06d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6a38007ed0 tx=0x7f6a38007e60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.255+0000 7f6a49c27740 1 Processor -- start 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.255+0000 7f6a49c27740 1 -- start start 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.255+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84ced5f10 0x55c84ced8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.255+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf125e0 0x55c84ced62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.255+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84cf00e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.255+0000 7f6a49c27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84ce07fe0 con 0x55c84cf125e0 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.255+0000 7f6a49c27740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84ce39d20 con 0x55c84cf009d0 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.255+0000 7f6a49c27740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c84cde2fa0 con 0x55c84ced5f10 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf125e0 0x55c84ced62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf125e0 0x55c84ced62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33954/0 (socket says 172.21.15.19:33954) 2024-04-25T09:56:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a3ffff640 1 -- 172.21.15.19:0/1752844297 learned_addr learned my addr 172.21.15.19:0/1752844297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a44be6640 1 --2- 172.21.15.19:0/1752844297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84ced5f10 0x55c84ced8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a453e7640 1 --2- 172.21.15.19:0/1752844297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84cf00e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a3ffff640 1 -- 172.21.15.19:0/1752844297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84ced5f10 msgr2=0x55c84ced8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/1752844297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84ced5f10 0x55c84ced8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a3ffff640 1 -- 172.21.15.19:0/1752844297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 msgr2=0x55c84cf00e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/1752844297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84cf00e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a3ffff640 1 -- 172.21.15.19:0/1752844297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c84cc1ab00 con 0x55c84cf125e0 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/1752844297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf125e0 0x55c84ced62f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6a3402a790 tx=0x7f6a3402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed170ce3547cca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a267fc640 1 -- 172.21.15.19:0/1752844297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34037040 con 0x55c84cf125e0 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a267fc640 1 -- 172.21.15.19:0/1752844297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a340326c0 con 0x55c84cf125e0 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.256+0000 7f6a267fc640 1 -- 172.21.15.19:0/1752844297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34032a70 con 0x55c84cf125e0 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.257+0000 7f6a49c27740 1 -- 172.21.15.19:0/1752844297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf125e0 msgr2=0x55c84ced62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.257+0000 7f6a49c27740 1 --2- 172.21.15.19:0/1752844297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf125e0 0x55c84ced62f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6a3402a790 tx=0x7f6a3402c000 comp rx=0 tx=0).stop 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.257+0000 7f6a49c27740 1 -- 172.21.15.19:0/1752844297 shutdown_connections 2024-04-25T09:56:13.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.257+0000 7f6a49c27740 1 --2- 172.21.15.19:0/1752844297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84cf00e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.257+0000 7f6a49c27740 1 --2- 172.21.15.19:0/1752844297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf125e0 0x55c84ced62f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.257+0000 7f6a49c27740 1 --2- 172.21.15.19:0/1752844297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84ced5f10 0x55c84ced8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.257+0000 7f6a49c27740 1 -- 172.21.15.19:0/1752844297 >> 172.21.15.19:0/1752844297 conn(0x55c84ce9e100 msgr2=0x55c84cf1b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.257+0000 7f6a49c27740 1 -- 172.21.15.19:0/1752844297 shutdown_connections 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.257+0000 7f6a49c27740 1 -- 172.21.15.19:0/1752844297 wait complete. 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.257+0000 7f6a49c27740 1 -- start start 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.258+0000 7f6a49c27740 1 Processor -- start 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.258+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84ced5f10 0x55c84cf03be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.258+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf009d0 0x55c84cef1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.258+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 0x55c84cf1b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.258+0000 7f6a49c27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84cdd8750 con 0x55c84cf009d0 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.258+0000 7f6a49c27740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84ce35dc0 con 0x55c84ced5f10 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.258+0000 7f6a49c27740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c84cf1c580 con 0x55c84cf125e0 2024-04-25T09:56:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.258+0000 7f6a49c27740 10 client.0 _ll_register_callbacks cb 0x55c84cdf0ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.258+0000 7f6a3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf009d0 0x55c84cef1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.258+0000 7f6a3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf009d0 0x55c84cef1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33968/0 (socket says 172.21.15.19:33968) 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.258+0000 7f6a3ffff640 1 -- 172.21.15.19:0/3725577688 learned_addr learned my addr 172.21.15.19:0/3725577688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a44be6640 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84ced5f10 0x55c84cf03be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a453e7640 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 0x55c84cf1b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a3ffff640 1 -- 172.21.15.19:0/3725577688 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 msgr2=0x55c84cf1b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 0x55c84cf1b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a3ffff640 1 -- 172.21.15.19:0/3725577688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84ced5f10 msgr2=0x55c84cf03be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84ced5f10 0x55c84cf03be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a3ffff640 1 -- 172.21.15.19:0/3725577688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c84cb0e3f0 con 0x55c84cf009d0 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf009d0 0x55c84cef1a10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6a34000c00 tx=0x7f6a3402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f3daec9f84fb2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3725577688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34037070 con 0x55c84cf009d0 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3725577688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a34047c80 con 0x55c84cf009d0 2024-04-25T09:56:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a49c27740 1 -- 172.21.15.19:0/3725577688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a340326c0 con 0x55c84cf009d0 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3725577688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a3404e050 con 0x55c84cf009d0 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a49c27740 10 client.6189 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a49c27740 1 -- 172.21.15.19:0/3725577688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55c84cc7b290 con 0x55c84cf009d0 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3725577688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a340326c0 con 0x55c84cf009d0 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.259+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3725577688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6a34051020 con 0x55c84cf009d0 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.260+0000 7f6a49c27740 10 client.6189 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.260+0000 7f6a49c27740 10 client.6189 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.260+0000 7f6a49c27740 20 client.6189 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.260+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55c84cf1bde0 0x55c84cf19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.260+0000 7f6a49c27740 4 client.6189 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.260+0000 7f6a49c27740 1 -- 172.21.15.19:0/3725577688 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c84ce47260 con 0x55c84cf1bde0 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.260+0000 7f6a44be6640 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55c84cf1bde0 0x55c84cf19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.260+0000 7f6a44be6640 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55c84cf1bde0 0x55c84cf19b80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631b899a3d61426b server_cookie=71bdc3ac7bfd9247 in_seq=0 out_seq=0 2024-04-25T09:56:13.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.260+0000 7f6a1f7fe640 10 client.6189 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.260+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3725577688 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c84ce47260 con 0x55c84cf1bde0 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.260+0000 7f6a1f7fe640 10 client.6189 handle_command_reply: tid=0 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 1 client.6189 shutdown 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 1 -- 172.21.15.19:0/3725577688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf009d0 msgr2=0x55c84cef1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf009d0 0x55c84cef1a10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6a34000c00 tx=0x7f6a3402c070 comp rx=0 tx=0).stop 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 1 -- 172.21.15.19:0/3725577688 shutdown_connections 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55c84cf1bde0 0x55c84cf19b80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 0x55c84cf1b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84cf009d0 0x55c84cef1a10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3725577688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84ced5f10 0x55c84cf03be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a1f7fe640 0 client.6189 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 1 -- 172.21.15.19:0/3725577688 >> 172.21.15.19:0/3725577688 conn(0x55c84ce9e100 msgr2=0x55c84cf1eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 1 -- 172.21.15.19:0/3725577688 shutdown_connections 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 1 -- 172.21.15.19:0/3725577688 wait complete. 2024-04-25T09:56:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.261+0000 7f6a49c27740 20 client.6189 trim_cache size 0 max 16384 2024-04-25T09:56:13.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.276+0000 7f6a49c27740 1 Processor -- start 2024-04-25T09:56:13.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.276+0000 7f6a49c27740 1 -- start start 2024-04-25T09:56:13.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84ced4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 0x55c84ced5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a49c27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40200200 con 0x55c84ced5f10 2024-04-25T09:56:13.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a49c27740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a401ff460 con 0x55c84cf009d0 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a49c27740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400c8a10 con 0x55c84cf125e0 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a44be6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a44be6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33970/0 (socket says 172.21.15.19:33970) 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a44be6640 1 -- 172.21.15.19:0/459951962 learned_addr learned my addr 172.21.15.19:0/459951962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a44be6640 1 -- 172.21.15.19:0/459951962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 msgr2=0x55c84ced5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a44be6640 1 --2- 172.21.15.19:0/459951962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 0x55c84ced5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a44be6640 1 -- 172.21.15.19:0/459951962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 msgr2=0x55c84ced4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a44be6640 1 --2- 172.21.15.19:0/459951962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84ced4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a44be6640 1 -- 172.21.15.19:0/459951962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c84cae74b0 con 0x55c84ced5f10 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a44be6640 1 --2- 172.21.15.19:0/459951962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6a380019f0 tx=0x7f6a38012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e111c09d24d9629 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a25ffb640 1 -- 172.21.15.19:0/459951962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3802c070 con 0x55c84ced5f10 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a25ffb640 1 -- 172.21.15.19:0/459951962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a38017030 con 0x55c84ced5f10 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.277+0000 7f6a25ffb640 1 -- 172.21.15.19:0/459951962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38027e60 con 0x55c84ced5f10 2024-04-25T09:56:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.278+0000 7f6a49c27740 1 -- 172.21.15.19:0/459951962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 msgr2=0x55c84cf18b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.278+0000 7f6a49c27740 1 --2- 172.21.15.19:0/459951962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6a380019f0 tx=0x7f6a38012000 comp rx=0 tx=0).stop 2024-04-25T09:56:13.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.278+0000 7f6a49c27740 1 -- 172.21.15.19:0/459951962 shutdown_connections 2024-04-25T09:56:13.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.278+0000 7f6a49c27740 1 --2- 172.21.15.19:0/459951962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 0x55c84ced5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.278+0000 7f6a49c27740 1 --2- 172.21.15.19:0/459951962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84ced4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.278+0000 7f6a49c27740 1 --2- 172.21.15.19:0/459951962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.278+0000 7f6a49c27740 1 -- 172.21.15.19:0/459951962 >> 172.21.15.19:0/459951962 conn(0x55c84ce9e100 msgr2=0x55c84cecc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.278+0000 7f6a49c27740 1 -- 172.21.15.19:0/459951962 shutdown_connections 2024-04-25T09:56:13.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.278+0000 7f6a49c27740 1 -- 172.21.15.19:0/459951962 wait complete. 2024-04-25T09:56:13.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.279+0000 7f6a49c27740 1 -- start start 2024-04-25T09:56:13.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.279+0000 7f6a49c27740 1 Processor -- start 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.279+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.279+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84cf19080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.279+0000 7f6a49c27740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 0x55c84cf858f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.279+0000 7f6a49c27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400c9010 con 0x55c84ced5f10 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.279+0000 7f6a49c27740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84cf03f00 con 0x55c84cf009d0 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.279+0000 7f6a49c27740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c84cf86c60 con 0x55c84cf125e0 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a49c27740 10 client.0 _ll_register_callbacks cb 0x7f6a34035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a44be6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a44be6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33980/0 (socket says 172.21.15.19:33980) 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a44be6640 1 -- 172.21.15.19:0/3571740058 learned_addr learned my addr 172.21.15.19:0/3571740058 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a453e7640 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 0x55c84cf858f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84cf19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a44be6640 1 -- 172.21.15.19:0/3571740058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 msgr2=0x55c84cf858f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a44be6640 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 0x55c84cf858f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a44be6640 1 -- 172.21.15.19:0/3571740058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 msgr2=0x55c84cf19080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a44be6640 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84cf19080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a44be6640 1 -- 172.21.15.19:0/3571740058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c84cf85440 con 0x55c84ced5f10 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84cf19080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a44be6640 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6a38001a20 tx=0x7f6a38001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838598d2e1d436cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3571740058 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3802c040 con 0x55c84ced5f10 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3571740058 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a38027150 con 0x55c84ced5f10 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a49c27740 1 -- 172.21.15.19:0/3571740058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a38017030 con 0x55c84ced5f10 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3571740058 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a38039050 con 0x55c84ced5f10 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a49c27740 10 client.6195 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.280+0000 7f6a49c27740 1 -- 172.21.15.19:0/3571740058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55c84cf7f6a0 con 0x55c84ced5f10 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.281+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3571740058 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a38017030 con 0x55c84ced5f10 2024-04-25T09:56:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.281+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3571740058 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6a3803c020 con 0x55c84ced5f10 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.281+0000 7f6a49c27740 10 client.6195 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.281+0000 7f6a49c27740 10 client.6195 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.281+0000 7f6a49c27740 20 client.6195 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.281+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55c84cf864f0 0x55c84cf85e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.281+0000 7f6a49c27740 4 client.6195 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.281+0000 7f6a49c27740 1 -- 172.21.15.19:0/3571740058 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55c84ce415b0 con 0x55c84cf864f0 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.281+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55c84cf864f0 0x55c84cf85e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.281+0000 7f6a1f7fe640 10 client.6195 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.281+0000 7f6a3ffff640 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55c84cf864f0 0x55c84cf85e30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc844d6c31e0bc05 server_cookie=815efcfc00ec6517 in_seq=0 out_seq=0 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a1f7fe640 1 -- 172.21.15.19:0/3571740058 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c84ce415b0 con 0x55c84cf864f0 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a1f7fe640 10 client.6195 handle_command_reply: tid=0 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 1 client.6195 shutdown 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 1 -- 172.21.15.19:0/3571740058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 msgr2=0x55c84cf18b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6a38001a20 tx=0x7f6a38001250 comp rx=0 tx=0).stop 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 1 -- 172.21.15.19:0/3571740058 shutdown_connections 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55c84cf864f0 0x55c84cf85e30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c84cf125e0 0x55c84cf858f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c84cf009d0 0x55c84cf19080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 1 --2- 172.21.15.19:0/3571740058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c84ced5f10 0x55c84cf18b40 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a1f7fe640 0 client.6195 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 1 -- 172.21.15.19:0/3571740058 >> 172.21.15.19:0/3571740058 conn(0x55c84ce9e100 msgr2=0x55c84cf03610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 1 -- 172.21.15.19:0/3571740058 shutdown_connections 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 1 -- 172.21.15.19:0/3571740058 wait complete. 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a49c27740 20 client.6195 trim_cache size 0 max 16384 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:13.291 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:56:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "f76ee0e9-fc22-4442-ba02-f1a36e468b9c", 2024-04-25T09:56:13.292 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:56:13.292 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a46e4a640 1 -- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6a2c06b4d0 msgr2=0x7f6a2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a46e4a640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6a2c06b4d0 0x7f6a2c06d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6a38007ed0 tx=0x7f6a38007e60 comp rx=0 tx=0).stop 2024-04-25T09:56:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a46e4a640 1 -- 172.21.15.19:0/3824261478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a4015b7d0 msgr2=0x7f6a40160050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.282+0000 7f6a46e4a640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a4015b7d0 0x7f6a40160050 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000e860 tx=0x7f6a3000ed20 comp rx=0 tx=0).stop 2024-04-25T09:56:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.283+0000 7f6a46e4a640 1 -- 172.21.15.19:0/3824261478 shutdown_connections 2024-04-25T09:56:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.283+0000 7f6a46e4a640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6a2c06b4d0 0x7f6a2c06d990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.283+0000 7f6a46e4a640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a4015b7d0 0x7f6a40160050 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.283+0000 7f6a46e4a640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6a400b5620 0x7f6a400c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.283+0000 7f6a46e4a640 1 --2- 172.21.15.19:0/3824261478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a4006c2d0 0x7f6a40157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.283+0000 7f6a46e4a640 1 -- 172.21.15.19:0/3824261478 >> 172.21.15.19:0/3824261478 conn(0x7f6a400d0800 msgr2=0x7f6a4006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.283+0000 7f6a46e4a640 1 -- 172.21.15.19:0/3824261478 shutdown_connections 2024-04-25T09:56:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.283+0000 7f6a46e4a640 1 -- 172.21.15.19:0/3824261478 wait complete. 2024-04-25T09:56:13.297 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'f76ee0e9-fc22-4442-ba02-f1a36e468b9c', 'mode': 'asynchronous'}' 2024-04-25T09:56:13.297 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:56:13.297 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:13.297 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:56:13.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.397+0000 7f9a48ff5640 1 Processor -- start 2024-04-25T09:56:13.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.397+0000 7f9a48ff5640 1 -- start start 2024-04-25T09:56:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.398+0000 7f9a48ff5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9a4415b7d0 0x7f9a440cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.398+0000 7f9a48ff5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a440b5620 0x7f9a440b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.398+0000 7f9a48ff5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4406cf50 con 0x7f9a4406c2d0 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.398+0000 7f9a48ff5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a440b5fc0 con 0x7f9a4415b7d0 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.398+0000 7f9a48ff5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a440ce530 con 0x7f9a440b5620 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.398+0000 7f9a4259c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9a4415b7d0 0x7f9a440cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.398+0000 7f9a42d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9a4406c2d0 0x7f9a4406c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41446/0 (socket says 172.21.15.19:41446) 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.398+0000 7f9a42d9d640 1 -- 172.21.15.19:0/81858633 learned_addr learned my addr 172.21.15.19:0/81858633 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.398+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/81858633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a440b5620 0x7f9a440b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.399+0000 7f9a4259c640 1 -- 172.21.15.19:0/81858633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a440b5620 msgr2=0x7f9a440b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.399+0000 7f9a4259c640 1 --2- 172.21.15.19:0/81858633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a440b5620 0x7f9a440b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.399+0000 7f9a4259c640 1 -- 172.21.15.19:0/81858633 >> v1:172.21.15.19:6789/0 conn(0x7f9a4406c2d0 legacy=0x7f9a4406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.399+0000 7f9a4259c640 1 -- 172.21.15.19:0/81858633 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a440ce810 con 0x7f9a4415b7d0 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.399+0000 7f9a4159a640 1 -- 172.21.15.19:0/81858633 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814441196 0 0) 0x7f9a4406cf50 con 0x7f9a4406c2d0 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.399+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/81858633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a440b5620 0x7f9a440b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.399+0000 7f9a4259c640 1 --2- 172.21.15.19:0/81858633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4415b7d0 0x7f9a440cf010 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0097a0 tx=0x7f9a3c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beab0efe262161d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.400+0000 7f9a4159a640 1 -- 172.21.15.19:0/81858633 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c037070 con 0x7f9a4415b7d0 2024-04-25T09:56:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.400+0000 7f9a4159a640 1 -- 172.21.15.19:0/81858633 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c032440 con 0x7f9a4415b7d0 2024-04-25T09:56:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.400+0000 7f9a4159a640 1 -- 172.21.15.19:0/81858633 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c032810 con 0x7f9a4415b7d0 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.400+0000 7f9a48ff5640 1 -- 172.21.15.19:0/81858633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4415b7d0 msgr2=0x7f9a440cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.400+0000 7f9a48ff5640 1 --2- 172.21.15.19:0/81858633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4415b7d0 0x7f9a440cf010 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0097a0 tx=0x7f9a3c02a180 comp rx=0 tx=0).stop 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.400+0000 7f9a48ff5640 1 -- 172.21.15.19:0/81858633 shutdown_connections 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.400+0000 7f9a48ff5640 1 --2- 172.21.15.19:0/81858633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a440b5620 0x7f9a440b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.400+0000 7f9a48ff5640 1 --2- 172.21.15.19:0/81858633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a4415b7d0 0x7f9a440cf010 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.400+0000 7f9a48ff5640 1 -- 172.21.15.19:0/81858633 >> 172.21.15.19:0/81858633 conn(0x7f9a440d0800 msgr2=0x7f9a440d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.400+0000 7f9a48ff5640 1 -- 172.21.15.19:0/81858633 shutdown_connections 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.400+0000 7f9a48ff5640 1 -- 172.21.15.19:0/81858633 wait complete. 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.402+0000 7f9a48ff5640 1 Processor -- start 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.402+0000 7f9a48ff5640 1 -- start start 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.402+0000 7f9a48ff5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a4406c2d0 0x7f9a44150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.402+0000 7f9a48ff5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a440b5620 0x7f9a440cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.402+0000 7f9a48ff5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a4415b7d0 0x7f9a440cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.402+0000 7f9a48ff5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a440cf550 con 0x7f9a440b5620 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.402+0000 7f9a48ff5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4406d230 con 0x7f9a4406c2d0 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.402+0000 7f9a48ff5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a440cf6d0 con 0x7f9a4415b7d0 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.402+0000 7f9a41d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a440b5620 0x7f9a440cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a41d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a440b5620 0x7f9a440cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33998/0 (socket says 172.21.15.19:33998) 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a41d9b640 1 -- 172.21.15.19:0/210261084 learned_addr learned my addr 172.21.15.19:0/210261084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a4259c640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a4406c2d0 0x7f9a44150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a42d9d640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a4415b7d0 0x7f9a440cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a41d9b640 1 -- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a4415b7d0 msgr2=0x7f9a440cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a4415b7d0 0x7f9a440cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a41d9b640 1 -- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a4406c2d0 msgr2=0x7f9a44150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a4406c2d0 0x7f9a44150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a41d9b640 1 -- 172.21.15.19:0/210261084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a440c7770 con 0x7f9a440b5620 2024-04-25T09:56:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a4259c640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a4406c2d0 0x7f9a44150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a440b5620 0x7f9a440cd6e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00e7e0 tx=0x7f9a2c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a3affd640 1 -- 172.21.15.19:0/210261084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00f040 con 0x7f9a440b5620 2024-04-25T09:56:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a3affd640 1 -- 172.21.15.19:0/210261084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c004030 con 0x7f9a440b5620 2024-04-25T09:56:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a3affd640 1 -- 172.21.15.19:0/210261084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c004400 con 0x7f9a440b5620 2024-04-25T09:56:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a48ff5640 1 -- 172.21.15.19:0/210261084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a440c7a10 con 0x7f9a440b5620 2024-04-25T09:56:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.403+0000 7f9a48ff5640 1 -- 172.21.15.19:0/210261084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a440c7fd0 con 0x7f9a440b5620 2024-04-25T09:56:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.405+0000 7f9a3affd640 1 -- 172.21.15.19:0/210261084 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a2c00c6d0 con 0x7f9a440b5620 2024-04-25T09:56:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.405+0000 7f9a3affd640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a2406b4d0 0x7f9a2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.405+0000 7f9a3affd640 1 -- 172.21.15.19:0/210261084 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9a2c08a150 con 0x7f9a440b5620 2024-04-25T09:56:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.405+0000 7f9a4259c640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a2406b4d0 0x7f9a2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.406+0000 7f9a4259c640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a2406b4d0 0x7f9a2406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c034000 tx=0x7f9a3c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.406+0000 7f9a4bdd1740 1 Processor -- start 2024-04-25T09:56:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.406+0000 7f9a4bdd1740 1 -- start start 2024-04-25T09:56:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.407+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd892f60 0x557bcd8911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.407+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd84d130 0x557bcd893340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.407+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd890590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.407+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bcd7afe60 con 0x557bcd873af0 2024-04-25T09:56:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.407+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bcd759120 con 0x557bcd84d130 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.407+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bcd7abf00 con 0x557bcd892f60 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.407+0000 7f9a42d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd890590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.407+0000 7f9a42d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd890590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34000/0 (socket says 172.21.15.19:34000) 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.407+0000 7f9a42d9d640 1 -- 172.21.15.19:0/1164203965 learned_addr learned my addr 172.21.15.19:0/1164203965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.407+0000 7f9a4259c640 1 --2- 172.21.15.19:0/1164203965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd892f60 0x557bcd8911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.407+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/1164203965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd84d130 0x557bcd893340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.408+0000 7f9a42d9d640 1 -- 172.21.15.19:0/1164203965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd892f60 msgr2=0x557bcd8911b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.408+0000 7f9a42d9d640 1 --2- 172.21.15.19:0/1164203965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd892f60 0x557bcd8911b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.408+0000 7f9a42d9d640 1 -- 172.21.15.19:0/1164203965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd84d130 msgr2=0x557bcd893340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.408+0000 7f9a42d9d640 1 --2- 172.21.15.19:0/1164203965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd84d130 0x557bcd893340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.408+0000 7f9a42d9d640 1 -- 172.21.15.19:0/1164203965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bcd590f00 con 0x557bcd873af0 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.408+0000 7f9a42d9d640 1 --2- 172.21.15.19:0/1164203965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd890590 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9a34007300 tx=0x7f9a340077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cb728fd2714442 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.408+0000 7f9a23fff640 1 -- 172.21.15.19:0/1164203965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34019070 con 0x557bcd873af0 2024-04-25T09:56:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.408+0000 7f9a23fff640 1 -- 172.21.15.19:0/1164203965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a34001e20 con 0x557bcd873af0 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.408+0000 7f9a23fff640 1 -- 172.21.15.19:0/1164203965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34014480 con 0x557bcd873af0 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.409+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/1164203965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 msgr2=0x557bcd890590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.409+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/1164203965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd890590 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9a34007300 tx=0x7f9a340077c0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.409+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/1164203965 shutdown_connections 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.409+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/1164203965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd890590 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.409+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/1164203965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd84d130 0x557bcd893340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.409+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/1164203965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd892f60 0x557bcd8911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.409+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/1164203965 >> 172.21.15.19:0/1164203965 conn(0x557bcd813f70 msgr2=0x557bcd84a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.409+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/1164203965 shutdown_connections 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.409+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/1164203965 wait complete. 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.409+0000 7f9a4bdd1740 1 -- start start 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.409+0000 7f9a4bdd1740 1 Processor -- start 2024-04-25T09:56:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.410+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd84c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.410+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd873af0 0x557bcd876500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.410+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd892f60 0x557bcd876a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.410+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bcd892760 con 0x557bcd84d130 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.410+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bcd74ecd0 con 0x557bcd873af0 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.410+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bcd892480 con 0x557bcd892f60 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.410+0000 7f9a4bdd1740 10 client.0 _ll_register_callbacks cb 0x557bcd4c42e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.410+0000 7f9a4259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd84c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.410+0000 7f9a4259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd84c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34004/0 (socket says 172.21.15.19:34004) 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.410+0000 7f9a4259c640 1 -- 172.21.15.19:0/3882101158 learned_addr learned my addr 172.21.15.19:0/3882101158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a42d9d640 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd892f60 0x557bcd876a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd873af0 0x557bcd876500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a4259c640 1 -- 172.21.15.19:0/3882101158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd892f60 msgr2=0x557bcd876a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a4259c640 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd892f60 0x557bcd876a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a4259c640 1 -- 172.21.15.19:0/3882101158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd873af0 msgr2=0x557bcd876500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a4259c640 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd873af0 0x557bcd876500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a4259c640 1 -- 172.21.15.19:0/3882101158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bcd4847f0 con 0x557bcd84d130 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a42d9d640 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd892f60 0x557bcd876a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd873af0 0x557bcd876500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a4259c640 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd84c3b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0098c0 tx=0x7f9a3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e549599ad028449 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a217fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a217fa640 1 -- 172.21.15.19:0/3882101158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c037040 con 0x557bcd84d130 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a217fa640 1 -- 172.21.15.19:0/3882101158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9a3c03ee00 con 0x557bcd84d130 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/3882101158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a34001e20 con 0x557bcd84d130 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a217fa640 1 -- 172.21.15.19:0/3882101158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a3c056050 con 0x557bcd84d130 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a4bdd1740 10 client.6210 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/3882101158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x557bcd5f1690 con 0x557bcd84d130 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.411+0000 7f9a217fa640 1 -- 172.21.15.19:0/3882101158 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a34001e20 con 0x557bcd84d130 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a217fa640 1 -- 172.21.15.19:0/3882101158 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9a3c059020 con 0x557bcd84d130 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a4bdd1740 10 client.6210 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a4bdd1740 10 client.6210 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a4bdd1740 20 client.6210 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557bcd8757c0 0x557bcd875be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a4bdd1740 4 client.6210 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/3882101158 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bcd7bd3a0 con 0x557bcd8757c0 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557bcd8757c0 0x557bcd875be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557bcd8757c0 0x557bcd875be0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b7bed835c9c65e server_cookie=47e580d23944030c in_seq=0 out_seq=0 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a217fa640 10 client.6210 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a217fa640 1 -- 172.21.15.19:0/3882101158 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f9a2c081e50 con 0x557bcd8757c0 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.412+0000 7f9a217fa640 10 client.6210 handle_command_reply: tid=0 2024-04-25T09:56:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 1 client.6210 shutdown 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/3882101158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 msgr2=0x557bcd84c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd84c3b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0098c0 tx=0x7f9a3c004730 comp rx=0 tx=0).stop 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/3882101158 shutdown_connections 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557bcd8757c0 0x557bcd875be0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd892f60 0x557bcd876a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd873af0 0x557bcd876500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/3882101158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd84c3b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a217fa640 0 client.6210 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/3882101158 >> 172.21.15.19:0/3882101158 conn(0x557bcd813f70 msgr2=0x557bcd894db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/3882101158 shutdown_connections 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/3882101158 wait complete. 2024-04-25T09:56:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.413+0000 7f9a4bdd1740 20 client.6210 trim_cache size 0 max 16384 2024-04-25T09:56:13.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.428+0000 7f9a4bdd1740 1 Processor -- start 2024-04-25T09:56:13.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.428+0000 7f9a4bdd1740 1 -- start start 2024-04-25T09:56:13.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd848750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd873af0 0x557bcd84b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd892f60 0x557bcd88b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44200340 con 0x557bcd84d130 2024-04-25T09:56:13.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a441ff520 con 0x557bcd892f60 2024-04-25T09:56:13.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a440c9b10 con 0x557bcd873af0 2024-04-25T09:56:13.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd848750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd848750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34010/0 (socket says 172.21.15.19:34010) 2024-04-25T09:56:13.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4259c640 1 -- 172.21.15.19:0/86125167 learned_addr learned my addr 172.21.15.19:0/86125167 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4259c640 1 -- 172.21.15.19:0/86125167 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd873af0 msgr2=0x557bcd84b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4259c640 1 --2- 172.21.15.19:0/86125167 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd873af0 0x557bcd84b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4259c640 1 -- 172.21.15.19:0/86125167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd892f60 msgr2=0x557bcd88b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4259c640 1 --2- 172.21.15.19:0/86125167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd892f60 0x557bcd88b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.429+0000 7f9a4259c640 1 -- 172.21.15.19:0/86125167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bcd45d8b0 con 0x557bcd84d130 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a4259c640 1 --2- 172.21.15.19:0/86125167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd848750 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009680 tx=0x7f9a3c047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234c2324a9fe7d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a237fe640 1 -- 172.21.15.19:0/86125167 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c037070 con 0x557bcd84d130 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a237fe640 1 -- 172.21.15.19:0/86125167 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c04e070 con 0x557bcd84d130 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a237fe640 1 -- 172.21.15.19:0/86125167 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c03c910 con 0x557bcd84d130 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/86125167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 msgr2=0x557bcd848750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/86125167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd848750 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009680 tx=0x7f9a3c047000 comp rx=0 tx=0).stop 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/86125167 shutdown_connections 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/86125167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd892f60 0x557bcd88b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/86125167 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd873af0 0x557bcd84b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/86125167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd84d130 0x557bcd848750 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/86125167 >> 172.21.15.19:0/86125167 conn(0x557bcd813f70 msgr2=0x557bcd877c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/86125167 shutdown_connections 2024-04-25T09:56:13.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.430+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/86125167 wait complete. 2024-04-25T09:56:13.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.431+0000 7f9a4bdd1740 1 -- start start 2024-04-25T09:56:13.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.431+0000 7f9a4bdd1740 1 Processor -- start 2024-04-25T09:56:13.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd84d130 0x557bcd8fee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd84ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a4bdd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd892f60 0x557bcd86eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bcd87a910 con 0x557bcd873af0 2024-04-25T09:56:13.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a441fef50 con 0x557bcd892f60 2024-04-25T09:56:13.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a4bdd1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bcd86f860 con 0x557bcd84d130 2024-04-25T09:56:13.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a4bdd1740 10 client.0 _ll_register_callbacks cb 0x7f9a44062560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:13.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a41d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd84ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a41d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd84ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34018/0 (socket says 172.21.15.19:34018) 2024-04-25T09:56:13.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a41d9b640 1 -- 172.21.15.19:0/491166554 learned_addr learned my addr 172.21.15.19:0/491166554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a4259c640 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd84d130 0x557bcd8fee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a42d9d640 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd892f60 0x557bcd86eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a41d9b640 1 -- 172.21.15.19:0/491166554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd84d130 msgr2=0x557bcd8fee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd84d130 0x557bcd8fee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a41d9b640 1 -- 172.21.15.19:0/491166554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd892f60 msgr2=0x557bcd86eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd892f60 0x557bcd86eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a41d9b640 1 -- 172.21.15.19:0/491166554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bcd84bcf0 con 0x557bcd873af0 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.432+0000 7f9a42d9d640 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd892f60 0x557bcd86eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a41d9b640 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd84ab80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c058440 tx=0x7f9a2c054b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f7ba9a921c5dc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a217fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a217fa640 1 -- 172.21.15.19:0/491166554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c020040 con 0x557bcd873af0 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a217fa640 1 -- 172.21.15.19:0/491166554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9a2c070150 con 0x557bcd873af0 2024-04-25T09:56:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/491166554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a3c04e070 con 0x557bcd873af0 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a217fa640 1 -- 172.21.15.19:0/491166554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a2c054db0 con 0x557bcd873af0 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a4bdd1740 10 client.6216 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/491166554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x557bcd86f660 con 0x557bcd873af0 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a217fa640 1 -- 172.21.15.19:0/491166554 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c04e070 con 0x557bcd873af0 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a217fa640 1 -- 172.21.15.19:0/491166554 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9a2c088020 con 0x557bcd873af0 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a4bdd1740 10 client.6216 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a4bdd1740 10 client.6216 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a4bdd1740 20 client.6216 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557bcd86f080 0x557bcd88b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a4bdd1740 4 client.6216 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/491166554 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x557bcd8947f0 con 0x557bcd86f080 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.433+0000 7f9a4259c640 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557bcd86f080 0x557bcd88b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4259c640 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557bcd86f080 0x557bcd88b2a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54490147b8c393de server_cookie=f1f7ef80513dc6fb in_seq=0 out_seq=0 2024-04-25T09:56:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a217fa640 10 client.6216 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a217fa640 1 -- 172.21.15.19:0/491166554 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557bcd8947f0 con 0x557bcd86f080 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a217fa640 10 client.6216 handle_command_reply: tid=0 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4bdd1740 1 client.6216 shutdown 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/491166554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 msgr2=0x557bcd84ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd84ab80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c058440 tx=0x7f9a2c054b10 comp rx=0 tx=0).stop 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/491166554 shutdown_connections 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x557bcd86f080 0x557bcd88b2a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bcd892f60 0x557bcd86eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557bcd873af0 0x557bcd84ab80 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4bdd1740 1 --2- 172.21.15.19:0/491166554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557bcd84d130 0x557bcd8fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/491166554 >> 172.21.15.19:0/491166554 conn(0x557bcd813f70 msgr2=0x557bcd876c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a217fa640 0 client.6216 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/491166554 shutdown_connections 2024-04-25T09:56:13.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.434+0000 7f9a4bdd1740 1 -- 172.21.15.19:0/491166554 wait complete. 2024-04-25T09:56:13.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a4bdd1740 20 client.6216 trim_cache size 0 max 16384 2024-04-25T09:56:13.445 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:13.445 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:56:13.445 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "115486fb-60a0-4ed0-bedf-d0f5e261b812", 2024-04-25T09:56:13.445 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:56:13.445 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 -- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a2406b4d0 msgr2=0x7f9a2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a2406b4d0 0x7f9a2406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c034000 tx=0x7f9a3c02e000 comp rx=0 tx=0).stop 2024-04-25T09:56:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 -- 172.21.15.19:0/210261084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a440b5620 msgr2=0x7f9a440cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a440b5620 0x7f9a440cd6e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00e7e0 tx=0x7f9a2c00eca0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 -- 172.21.15.19:0/210261084 shutdown_connections 2024-04-25T09:56:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a2406b4d0 0x7f9a2406d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a4415b7d0 0x7f9a440cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a440b5620 0x7f9a440cd6e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 --2- 172.21.15.19:0/210261084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a4406c2d0 0x7f9a44150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 -- 172.21.15.19:0/210261084 >> 172.21.15.19:0/210261084 conn(0x7f9a440d0800 msgr2=0x7f9a440b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 -- 172.21.15.19:0/210261084 shutdown_connections 2024-04-25T09:56:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.435+0000 7f9a48ff5640 1 -- 172.21.15.19:0/210261084 wait complete. 2024-04-25T09:56:13.450 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '115486fb-60a0-4ed0-bedf-d0f5e261b812', 'mode': 'asynchronous'}' 2024-04-25T09:56:13.450 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T09:56:13.450 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:13.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T09:56:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.549+0000 7fe9c1b7e640 1 Processor -- start 2024-04-25T09:56:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.549+0000 7fe9c1b7e640 1 -- start start 2024-04-25T09:56:13.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.550+0000 7fe9c1b7e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9bc0d0ab0 0x7fe9bc168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.550+0000 7fe9c1b7e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc0d2420 con 0x7fe9bc15dac0 2024-04-25T09:56:13.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.550+0000 7fe9c1b7e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc0d2700 con 0x7fe9bc0d34c0 2024-04-25T09:56:13.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.550+0000 7fe9c1b7e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc0d3030 con 0x7fe9bc0d0ab0 2024-04-25T09:56:13.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.550+0000 7fe9bbfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9bc0d0ab0 0x7fe9bc168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.550+0000 7fe9bb7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe9bc15dac0 0x7fe9bc0d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41462/0 (socket says 172.21.15.19:41462) 2024-04-25T09:56:13.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.550+0000 7fe9bb7fe640 1 -- 172.21.15.19:0/983619742 learned_addr learned my addr 172.21.15.19:0/983619742 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:13.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.550+0000 7fe9bbfff640 1 -- 172.21.15.19:0/983619742 >> v1:172.21.15.112:6789/0 conn(0x7fe9bc0d34c0 legacy=0x7fe9bc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.550+0000 7fe9bbfff640 1 -- 172.21.15.19:0/983619742 >> v1:172.21.15.19:6789/0 conn(0x7fe9bc15dac0 legacy=0x7fe9bc0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.550+0000 7fe9bbfff640 1 -- 172.21.15.19:0/983619742 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc0d29e0 con 0x7fe9bc0d0ab0 2024-04-25T09:56:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.550+0000 7fe9ba7fc640 1 -- 172.21.15.19:0/983619742 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935408595 0 0) 0x7fe9bc0d2420 con 0x7fe9bc15dac0 2024-04-25T09:56:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.551+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/983619742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9bc0d0ab0 0x7fe9bc168670 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0d31b0 tx=0x7fe9b402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95acf3bac4ba4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.551+0000 7fe9ba7fc640 1 -- 172.21.15.19:0/983619742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4037070 con 0x7fe9bc0d0ab0 2024-04-25T09:56:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.551+0000 7fe9ba7fc640 1 -- 172.21.15.19:0/983619742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b4007740 con 0x7fe9bc0d0ab0 2024-04-25T09:56:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.551+0000 7fe9ba7fc640 1 -- 172.21.15.19:0/983619742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4007b10 con 0x7fe9bc0d0ab0 2024-04-25T09:56:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.552+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/983619742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9bc0d0ab0 msgr2=0x7fe9bc168670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.552+0000 7fe9c1b7e640 1 --2- 172.21.15.19:0/983619742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9bc0d0ab0 0x7fe9bc168670 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0d31b0 tx=0x7fe9b402a780 comp rx=0 tx=0).stop 2024-04-25T09:56:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.552+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/983619742 shutdown_connections 2024-04-25T09:56:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.552+0000 7fe9c1b7e640 1 --2- 172.21.15.19:0/983619742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9bc0d0ab0 0x7fe9bc168670 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.552+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/983619742 >> 172.21.15.19:0/983619742 conn(0x7fe9bc061c30 msgr2=0x7fe9bc06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.552+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/983619742 shutdown_connections 2024-04-25T09:56:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.552+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/983619742 wait complete. 2024-04-25T09:56:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.553+0000 7fe9c1b7e640 1 Processor -- start 2024-04-25T09:56:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.553+0000 7fe9c1b7e640 1 -- start start 2024-04-25T09:56:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9c1b7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9bc0d0ab0 0x7fe9bc0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9c1b7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9bc0d34c0 0x7fe9bc152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9c1b7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9bc15dac0 0x7fe9bc0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9c1b7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc15dea0 con 0x7fe9bc15dac0 2024-04-25T09:56:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9c1b7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0d2100 con 0x7fe9bc0d34c0 2024-04-25T09:56:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9c1b7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0b97d0 con 0x7fe9bc0d0ab0 2024-04-25T09:56:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9bc15dac0 0x7fe9bc0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9bc15dac0 0x7fe9bc0b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34046/0 (socket says 172.21.15.19:34046) 2024-04-25T09:56:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9bbfff640 1 -- 172.21.15.19:0/1154645114 learned_addr learned my addr 172.21.15.19:0/1154645114 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9baffd640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9bc0d34c0 0x7fe9bc152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9bc0d0ab0 0x7fe9bc0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9bbfff640 1 -- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9bc0d0ab0 msgr2=0x7fe9bc0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9bc0d0ab0 0x7fe9bc0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9bbfff640 1 -- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9bc0d34c0 msgr2=0x7fe9bc152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9bc0d34c0 0x7fe9bc152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9bbfff640 1 -- 172.21.15.19:0/1154645114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc0b4350 con 0x7fe9bc15dac0 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.554+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9bc0d0ab0 0x7fe9bc0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.555+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9bc15dac0 0x7fe9bc0b4830 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009e80 tx=0x7fe9b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.555+0000 7fe9a3fff640 1 -- 172.21.15.19:0/1154645114 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4042020 con 0x7fe9bc15dac0 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.555+0000 7fe9a3fff640 1 -- 172.21.15.19:0/1154645114 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b4032970 con 0x7fe9bc15dac0 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.555+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/1154645114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc0cd670 con 0x7fe9bc15dac0 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.555+0000 7fe9a3fff640 1 -- 172.21.15.19:0/1154645114 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4037040 con 0x7fe9bc15dac0 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.555+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/1154645114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc157090 con 0x7fe9bc15dac0 2024-04-25T09:56:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.556+0000 7fe9a3fff640 1 -- 172.21.15.19:0/1154645114 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9b4045020 con 0x7fe9bc15dac0 2024-04-25T09:56:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.557+0000 7fe9a3fff640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe9a806b4d0 0x7fe9a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.557+0000 7fe9a3fff640 1 -- 172.21.15.19:0/1154645114 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9b40ab580 con 0x7fe9bc15dac0 2024-04-25T09:56:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.557+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe9a806b4d0 0x7fe9a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.557+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe9a806b4d0 0x7fe9a806d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac007fd0 tx=0x7fe9ac007dc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.558+0000 7fe9c4966740 1 Processor -- start 2024-04-25T09:56:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.558+0000 7fe9c4966740 1 -- start start 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.558+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d582157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.558+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d582133c0 0x562d58215d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.558+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d581e5c40 0x562d581e9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.558+0000 7fe9c4966740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5814ea50 con 0x562d581e5c40 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.558+0000 7fe9c4966740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d580f7d40 con 0x562d58212fe0 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.558+0000 7fe9c4966740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d5814aaf0 con 0x562d582133c0 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d581e5c40 0x562d581e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d581e5c40 0x562d581e9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34062/0 (socket says 172.21.15.19:34062) 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9bbfff640 1 -- 172.21.15.19:0/2484262602 learned_addr learned my addr 172.21.15.19:0/2484262602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9baffd640 1 --2- 172.21.15.19:0/2484262602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d582133c0 0x562d58215d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/2484262602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d582157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9bbfff640 1 -- 172.21.15.19:0/2484262602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d582133c0 msgr2=0x562d58215d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/2484262602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d582133c0 0x562d58215d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9bbfff640 1 -- 172.21.15.19:0/2484262602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 msgr2=0x562d582157f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/2484262602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d582157f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9bbfff640 1 -- 172.21.15.19:0/2484262602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d57f2f690 con 0x562d581e5c40 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/2484262602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d582157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/2484262602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d581e5c40 0x562d581e9450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4079c20 tx=0x7fe9b4032d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f09c9c8fb3599 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9a17fa640 1 -- 172.21.15.19:0/2484262602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40800c0 con 0x562d581e5c40 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9a17fa640 1 -- 172.21.15.19:0/2484262602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b407dde0 con 0x562d581e5c40 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.559+0000 7fe9a17fa640 1 -- 172.21.15.19:0/2484262602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40823a0 con 0x562d581e5c40 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.560+0000 7fe9c4966740 1 -- 172.21.15.19:0/2484262602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d581e5c40 msgr2=0x562d581e9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.560+0000 7fe9c4966740 1 --2- 172.21.15.19:0/2484262602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d581e5c40 0x562d581e9450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4079c20 tx=0x7fe9b4032d40 comp rx=0 tx=0).stop 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.560+0000 7fe9c4966740 1 -- 172.21.15.19:0/2484262602 shutdown_connections 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.560+0000 7fe9c4966740 1 --2- 172.21.15.19:0/2484262602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d581e5c40 0x562d581e9450 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.560+0000 7fe9c4966740 1 --2- 172.21.15.19:0/2484262602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d582133c0 0x562d58215d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.560+0000 7fe9c4966740 1 --2- 172.21.15.19:0/2484262602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d582157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.560+0000 7fe9c4966740 1 -- 172.21.15.19:0/2484262602 >> 172.21.15.19:0/2484262602 conn(0x562d581e7eb0 msgr2=0x562d582307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.560+0000 7fe9c4966740 1 -- 172.21.15.19:0/2484262602 shutdown_connections 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.560+0000 7fe9c4966740 1 -- 172.21.15.19:0/2484262602 wait complete. 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.560+0000 7fe9c4966740 1 -- start start 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9c4966740 1 Processor -- start 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d5820a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d582133c0 0x562d5820cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d5820d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9c4966740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d581eb060 con 0x562d58210c50 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9c4966740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d580ed340 con 0x562d58212fe0 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9c4966740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d581eb340 con 0x562d582133c0 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9c4966740 10 client.0 _ll_register_callbacks cb 0x562d5810acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d5820d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d5820d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34078/0 (socket says 172.21.15.19:34078) 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9bbfff640 1 -- 172.21.15.19:0/876973880 learned_addr learned my addr 172.21.15.19:0/876973880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.561+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d5820a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9baffd640 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d582133c0 0x562d5820cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9bbfff640 1 -- 172.21.15.19:0/876973880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d582133c0 msgr2=0x562d5820cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d582133c0 0x562d5820cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9bbfff640 1 -- 172.21.15.19:0/876973880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 msgr2=0x562d5820a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d5820a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9bbfff640 1 -- 172.21.15.19:0/876973880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d57f8fe20 con 0x562d58210c50 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d5820d2f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4079990 tx=0x7fe9b4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72009ebac569503d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe99a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe99a7fc640 1 -- 172.21.15.19:0/876973880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40823a0 con 0x562d58210c50 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe99a7fc640 1 -- 172.21.15.19:0/876973880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9b4081f10 con 0x562d58210c50 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9c4966740 1 -- 172.21.15.19:0/876973880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9b407dde0 con 0x562d58210c50 2024-04-25T09:56:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe99a7fc640 1 -- 172.21.15.19:0/876973880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9b4082110 con 0x562d58210c50 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9c4966740 10 client.6231 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9c4966740 1 -- 172.21.15.19:0/876973880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x562d57e22f80 con 0x562d58210c50 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe99a7fc640 1 -- 172.21.15.19:0/876973880 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b407dde0 con 0x562d58210c50 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe99a7fc640 1 -- 172.21.15.19:0/876973880 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe9b40ab0b0 con 0x562d58210c50 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9c4966740 10 client.6231 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.562+0000 7fe9c4966740 10 client.6231 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.563+0000 7fe9c4966740 20 client.6231 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.563+0000 7fe9c4966740 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d5820f220 0x562d5820f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.563+0000 7fe9c4966740 4 client.6231 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.563+0000 7fe9c4966740 1 -- 172.21.15.19:0/876973880 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d5815bf90 con 0x562d5820f220 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.563+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d5820f220 0x562d5820f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.563+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d5820f220 0x562d5820f630 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d282cd55cbba78 server_cookie=dfb73b6cee636a79 in_seq=0 out_seq=0 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.563+0000 7fe99a7fc640 10 client.6231 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.563+0000 7fe99a7fc640 1 -- 172.21.15.19:0/876973880 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d5815bf90 con 0x562d5820f220 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.563+0000 7fe99a7fc640 10 client.6231 handle_command_reply: tid=0 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.563+0000 7fe9c4966740 1 client.6231 shutdown 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe9c4966740 1 -- 172.21.15.19:0/876973880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 msgr2=0x562d5820d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe9c4966740 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d5820d2f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4079990 tx=0x7fe9b4009750 comp rx=0 tx=0).stop 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe9c4966740 1 -- 172.21.15.19:0/876973880 shutdown_connections 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe9c4966740 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d5820f220 0x562d5820f630 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe9c4966740 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d5820d2f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe9c4966740 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d582133c0 0x562d5820cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe99a7fc640 0 client.6231 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe9c4966740 1 --2- 172.21.15.19:0/876973880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d5820a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe9c4966740 1 -- 172.21.15.19:0/876973880 >> 172.21.15.19:0/876973880 conn(0x562d581e5c40 msgr2=0x562d58212220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe9c4966740 1 -- 172.21.15.19:0/876973880 shutdown_connections 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe9c4966740 1 -- 172.21.15.19:0/876973880 wait complete. 2024-04-25T09:56:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.564+0000 7fe9c4966740 20 client.6231 trim_cache size 0 max 16384 2024-04-25T09:56:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.579+0000 7fe9c4966740 1 Processor -- start 2024-04-25T09:56:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.579+0000 7fe9c4966740 1 -- start start 2024-04-25T09:56:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d58210c50 0x562d581ea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d581ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d582133c0 0x562d5820cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9c4966740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc2004a0 con 0x562d582133c0 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9c4966740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc1ff630 con 0x562d58212fe0 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9c4966740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc160650 con 0x562d58210c50 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d582133c0 0x562d5820cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d582133c0 0x562d5820cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34092/0 (socket says 172.21.15.19:34092) 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9bbfff640 1 -- 172.21.15.19:0/1235992583 learned_addr learned my addr 172.21.15.19:0/1235992583 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/1235992583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d58210c50 0x562d581ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9baffd640 1 --2- 172.21.15.19:0/1235992583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d581ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9bbfff640 1 -- 172.21.15.19:0/1235992583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d58210c50 msgr2=0x562d581ea460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/1235992583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d58210c50 0x562d581ea460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9bbfff640 1 -- 172.21.15.19:0/1235992583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 msgr2=0x562d581ea9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/1235992583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d581ea9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9bbfff640 1 -- 172.21.15.19:0/1235992583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d58243c00 con 0x562d582133c0 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/1235992583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d582133c0 0x562d5820cdb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40a9920 tx=0x7fe9b4091000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2310a0a725048c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9a0ff9640 1 -- 172.21.15.19:0/1235992583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b407dde0 con 0x562d582133c0 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9a0ff9640 1 -- 172.21.15.19:0/1235992583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b40090a0 con 0x562d582133c0 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.580+0000 7fe9a0ff9640 1 -- 172.21.15.19:0/1235992583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40ab0b0 con 0x562d582133c0 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.581+0000 7fe9c4966740 1 -- 172.21.15.19:0/1235992583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d582133c0 msgr2=0x562d5820cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.581+0000 7fe9c4966740 1 --2- 172.21.15.19:0/1235992583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d582133c0 0x562d5820cdb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40a9920 tx=0x7fe9b4091000 comp rx=0 tx=0).stop 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.581+0000 7fe9c4966740 1 -- 172.21.15.19:0/1235992583 shutdown_connections 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.581+0000 7fe9c4966740 1 --2- 172.21.15.19:0/1235992583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d582133c0 0x562d5820cdb0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.581+0000 7fe9c4966740 1 --2- 172.21.15.19:0/1235992583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d58212fe0 0x562d581ea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.581+0000 7fe9c4966740 1 --2- 172.21.15.19:0/1235992583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d58210c50 0x562d581ea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.581+0000 7fe9c4966740 1 -- 172.21.15.19:0/1235992583 >> 172.21.15.19:0/1235992583 conn(0x562d581e5c40 msgr2=0x562d582388f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.581+0000 7fe9c4966740 1 -- 172.21.15.19:0/1235992583 shutdown_connections 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.581+0000 7fe9c4966740 1 -- 172.21.15.19:0/1235992583 wait complete. 2024-04-25T09:56:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.582+0000 7fe9c4966740 1 -- start start 2024-04-25T09:56:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.582+0000 7fe9c4966740 1 Processor -- start 2024-04-25T09:56:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.582+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d58211640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.582+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d58212fe0 0x562d5820cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.582+0000 7fe9c4966740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d582133c0 0x562d5820d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.582+0000 7fe9c4966740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc160c50 con 0x562d58210c50 2024-04-25T09:56:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.582+0000 7fe9c4966740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5820e210 con 0x562d582133c0 2024-04-25T09:56:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.582+0000 7fe9c4966740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d5823f840 con 0x562d58212fe0 2024-04-25T09:56:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.582+0000 7fe9c4966740 10 client.0 _ll_register_callbacks cb 0x562d58211590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.582+0000 7fe9bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d58211640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d58211640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34094/0 (socket says 172.21.15.19:34094) 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9bb7fe640 1 -- 172.21.15.19:0/279945417 learned_addr learned my addr 172.21.15.19:0/279945417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9baffd640 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d58212fe0 0x562d5820cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9bbfff640 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d582133c0 0x562d5820d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9bb7fe640 1 -- 172.21.15.19:0/279945417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d58212fe0 msgr2=0x562d5820cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d58212fe0 0x562d5820cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9bb7fe640 1 -- 172.21.15.19:0/279945417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d582133c0 msgr2=0x562d5820d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d582133c0 0x562d5820d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9bb7fe640 1 -- 172.21.15.19:0/279945417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d5820f910 con 0x562d58210c50 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9bb7fe640 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d58211640 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac007bb0 tx=0x7fe9ac007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497d1f60c53ac55f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe99a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe99a7fc640 1 -- 172.21.15.19:0/279945417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0268d0 con 0x562d58210c50 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe99a7fc640 1 -- 172.21.15.19:0/279945417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9ac01e040 con 0x562d58210c50 2024-04-25T09:56:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9c4966740 1 -- 172.21.15.19:0/279945417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9b40090a0 con 0x562d58210c50 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe99a7fc640 1 -- 172.21.15.19:0/279945417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9ac014880 con 0x562d58210c50 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9c4966740 10 client.6237 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe9c4966740 1 -- 172.21.15.19:0/279945417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x562d5829a250 con 0x562d58210c50 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.583+0000 7fe99a7fc640 1 -- 172.21.15.19:0/279945417 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b40090a0 con 0x562d58210c50 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe99a7fc640 1 -- 172.21.15.19:0/279945417 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe9ac038050 con 0x562d58210c50 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe9c4966740 10 client.6237 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe9c4966740 10 client.6237 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe9c4966740 20 client.6237 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe9c4966740 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d5823f140 0x562d58226740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe9c4966740 4 client.6237 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe9c4966740 1 -- 172.21.15.19:0/279945417 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x562d581562e0 con 0x562d5823f140 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe9baffd640 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d5823f140 0x562d58226740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe99a7fc640 10 client.6237 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe9baffd640 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d5823f140 0x562d58226740 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4692e5a091f8d0e2 server_cookie=65ca6e4602f362e4 in_seq=0 out_seq=0 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe99a7fc640 1 -- 172.21.15.19:0/279945417 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562d581562e0 con 0x562d5823f140 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe99a7fc640 10 client.6237 handle_command_reply: tid=0 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.584+0000 7fe9c4966740 1 client.6237 shutdown 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c4966740 1 -- 172.21.15.19:0/279945417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 msgr2=0x562d58211640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c4966740 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d58211640 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac007bb0 tx=0x7fe9ac007b40 comp rx=0 tx=0).stop 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c4966740 1 -- 172.21.15.19:0/279945417 shutdown_connections 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c4966740 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x562d5823f140 0x562d58226740 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c4966740 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d582133c0 0x562d5820d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c4966740 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562d58212fe0 0x562d5820cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe99a7fc640 0 client.6237 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c4966740 1 --2- 172.21.15.19:0/279945417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d58210c50 0x562d58211640 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c4966740 1 -- 172.21.15.19:0/279945417 >> 172.21.15.19:0/279945417 conn(0x562d581e5c40 msgr2=0x562d582388f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c4966740 1 -- 172.21.15.19:0/279945417 shutdown_connections 2024-04-25T09:56:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c4966740 1 -- 172.21.15.19:0/279945417 wait complete. 2024-04-25T09:56:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c4966740 20 client.6237 trim_cache size 0 max 16384 2024-04-25T09:56:13.595 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:13.595 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:56:13.595 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "ac61cca1-8b6f-41a7-847d-6a78f340c5e7", 2024-04-25T09:56:13.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:56:13.595 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe9a806b4d0 msgr2=0x7fe9a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c1b7e640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe9a806b4d0 0x7fe9a806d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac007fd0 tx=0x7fe9ac007dc0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/1154645114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9bc15dac0 msgr2=0x7fe9bc0b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c1b7e640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9bc15dac0 0x7fe9bc0b4830 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009e80 tx=0x7fe9b4004a40 comp rx=0 tx=0).stop 2024-04-25T09:56:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/1154645114 shutdown_connections 2024-04-25T09:56:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c1b7e640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe9a806b4d0 0x7fe9a806d990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c1b7e640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9bc15dac0 0x7fe9bc0b4830 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c1b7e640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9bc0d34c0 0x7fe9bc152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c1b7e640 1 --2- 172.21.15.19:0/1154645114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9bc0d0ab0 0x7fe9bc0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.585+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/1154645114 >> 172.21.15.19:0/1154645114 conn(0x7fe9bc061c30 msgr2=0x7fe9bc0c7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.586+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/1154645114 shutdown_connections 2024-04-25T09:56:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.586+0000 7fe9c1b7e640 1 -- 172.21.15.19:0/1154645114 wait complete. 2024-04-25T09:56:13.600 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'ac61cca1-8b6f-41a7-847d-6a78f340c5e7', 'mode': 'asynchronous'}' 2024-04-25T09:56:13.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:13.601 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T09:56:13.617 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T09:56:13.617 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.755+0000 7f112511d640 1 Processor -- start 2024-04-25T09:56:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.755+0000 7f112511d640 1 -- start start 2024-04-25T09:56:13.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.755+0000 7f112511d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b580 0x7f11200cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.755+0000 7f112511d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11200cffa0 0x7f112015d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.755+0000 7f112511d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1120161960 con 0x7f112006b1a0 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.755+0000 7f112511d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11200d0430 con 0x7f11200cffa0 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.755+0000 7f112511d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11200cbd50 con 0x7f112006b580 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.756+0000 7f111f59e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11200cffa0 0x7f112015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.756+0000 7f111f59e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11200cffa0 0x7f112015d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34098/0 (socket says 172.21.15.19:34098) 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.756+0000 7f111f59e640 1 -- 172.21.15.19:0/1936747150 learned_addr learned my addr 172.21.15.19:0/1936747150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.756+0000 7f111ed9d640 1 --1- 172.21.15.19:0/1936747150 >> v1:172.21.15.19:6789/0 conn(0x7f112006b1a0 0x7f112006f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41472/0 (socket says 172.21.15.19:41472) 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.756+0000 7f111e59c640 1 --2- 172.21.15.19:0/1936747150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b580 0x7f11200cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.756+0000 7f111f59e640 1 -- 172.21.15.19:0/1936747150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b580 msgr2=0x7f11200cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.756+0000 7f111f59e640 1 --2- 172.21.15.19:0/1936747150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b580 0x7f11200cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.756+0000 7f111f59e640 1 -- 172.21.15.19:0/1936747150 >> v1:172.21.15.19:6789/0 conn(0x7f112006b1a0 legacy=0x7f112006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.756+0000 7f111f59e640 1 -- 172.21.15.19:0/1936747150 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1120161c40 con 0x7f11200cffa0 2024-04-25T09:56:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.756+0000 7f111dd9b640 1 -- 172.21.15.19:0/1936747150 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099037572 0 0) 0x7f1120161960 con 0x7f112006b1a0 2024-04-25T09:56:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.756+0000 7f111e59c640 1 --2- 172.21.15.19:0/1936747150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b580 0x7f11200cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f111f59e640 1 --2- 172.21.15.19:0/1936747150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11200cffa0 0x7f112015d370 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1118009e50 tx=0x7f111802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8382f4d60750ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f111dd9b640 1 -- 172.21.15.19:0/1936747150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118036070 con 0x7f11200cffa0 2024-04-25T09:56:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f111dd9b640 1 -- 172.21.15.19:0/1936747150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1118031600 con 0x7f11200cffa0 2024-04-25T09:56:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f111dd9b640 1 -- 172.21.15.19:0/1936747150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11180319d0 con 0x7f11200cffa0 2024-04-25T09:56:13.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f112511d640 1 -- 172.21.15.19:0/1936747150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11200cffa0 msgr2=0x7f112015d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f112511d640 1 --2- 172.21.15.19:0/1936747150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11200cffa0 0x7f112015d370 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1118009e50 tx=0x7f111802a680 comp rx=0 tx=0).stop 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f112511d640 1 -- 172.21.15.19:0/1936747150 shutdown_connections 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f112511d640 1 --2- 172.21.15.19:0/1936747150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11200cffa0 0x7f112015d370 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f112511d640 1 --2- 172.21.15.19:0/1936747150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b580 0x7f11200cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f112511d640 1 -- 172.21.15.19:0/1936747150 >> 172.21.15.19:0/1936747150 conn(0x7f11200b2220 msgr2=0x7f11200b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f112511d640 1 -- 172.21.15.19:0/1936747150 shutdown_connections 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.757+0000 7f112511d640 1 -- 172.21.15.19:0/1936747150 wait complete. 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.759+0000 7f112511d640 1 Processor -- start 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.759+0000 7f112511d640 1 -- start start 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.759+0000 7f112511d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b1a0 0x7f1120153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f112511d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f112006b580 0x7f11200b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f112511d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11200cffa0 0x7f11200c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f112511d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1120162200 con 0x7f11200cffa0 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f112511d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11200700d0 con 0x7f112006b580 2024-04-25T09:56:13.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f112511d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1120162720 con 0x7f112006b1a0 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11200cffa0 0x7f11200c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11200cffa0 0x7f11200c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34110/0 (socket says 172.21.15.19:34110) 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111f59e640 1 -- 172.21.15.19:0/4239999894 learned_addr learned my addr 172.21.15.19:0/4239999894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111ed9d640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b1a0 0x7f1120153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111e59c640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f112006b580 0x7f11200b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111f59e640 1 -- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b1a0 msgr2=0x7f1120153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111f59e640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b1a0 0x7f1120153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111f59e640 1 -- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f112006b580 msgr2=0x7f11200b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111f59e640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f112006b580 0x7f11200b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111f59e640 1 -- 172.21.15.19:0/4239999894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112015e0c0 con 0x7f11200cffa0 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111ed9d640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b1a0 0x7f1120153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111e59c640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f112006b580 0x7f11200b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f111f59e640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11200cffa0 0x7f11200c9680 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1118009e50 tx=0x7f1118004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f11077fe640 1 -- 172.21.15.19:0/4239999894 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118041020 con 0x7f11200cffa0 2024-04-25T09:56:13.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f11077fe640 1 -- 172.21.15.19:0/4239999894 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11180077a0 con 0x7f11200cffa0 2024-04-25T09:56:13.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f11077fe640 1 -- 172.21.15.19:0/4239999894 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118036040 con 0x7f11200cffa0 2024-04-25T09:56:13.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f112511d640 1 -- 172.21.15.19:0/4239999894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11200b5890 con 0x7f11200cffa0 2024-04-25T09:56:13.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.760+0000 7f112511d640 1 -- 172.21.15.19:0/4239999894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11200b5e50 con 0x7f11200cffa0 2024-04-25T09:56:13.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.762+0000 7f112511d640 1 -- 172.21.15.19:0/4239999894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1120073990 con 0x7f11200cffa0 2024-04-25T09:56:13.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.762+0000 7f11077fe640 1 -- 172.21.15.19:0/4239999894 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1118044020 con 0x7f11200cffa0 2024-04-25T09:56:13.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.762+0000 7f11077fe640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f110c06b500 0x7f110c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:13.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.762+0000 7f11077fe640 1 -- 172.21.15.19:0/4239999894 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f11180ab000 con 0x7f11200cffa0 2024-04-25T09:56:13.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.763+0000 7f111ed9d640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f110c06b500 0x7f110c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:13.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.763+0000 7f111ed9d640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f110c06b500 0x7f110c06d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1110000f60 tx=0x7f1110009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:13.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.766+0000 7f11077fe640 1 -- 172.21.15.19:0/4239999894 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f111804b030 con 0x7f11200cffa0 2024-04-25T09:56:13.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.982+0000 7f112511d640 1 -- 172.21.15.19:0/4239999894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11200b2810 con 0x7f11200cffa0 2024-04-25T09:56:13.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.983+0000 7f11077fe640 1 -- 172.21.15.19:0/4239999894 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f112015e0c0 con 0x7f11200cffa0 2024-04-25T09:56:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:13.987 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:56:13.029515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:13.987 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-25T09:56:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 -- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f110c06b500 msgr2=0x7f110c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f110c06b500 0x7f110c06d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1110000f60 tx=0x7f1110009040 comp rx=0 tx=0).stop 2024-04-25T09:56:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 -- 172.21.15.19:0/4239999894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11200cffa0 msgr2=0x7f11200c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11200cffa0 0x7f11200c9680 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1118009e50 tx=0x7f1118004a40 comp rx=0 tx=0).stop 2024-04-25T09:56:13.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 -- 172.21.15.19:0/4239999894 shutdown_connections 2024-04-25T09:56:13.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f110c06b500 0x7f110c06d9c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11200cffa0 0x7f11200c9680 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f112006b580 0x7f11200b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 --2- 172.21.15.19:0/4239999894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f112006b1a0 0x7f1120153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:13.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 -- 172.21.15.19:0/4239999894 >> 172.21.15.19:0/4239999894 conn(0x7f11200b2220 msgr2=0x7f11200d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:13.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 -- 172.21.15.19:0/4239999894 shutdown_connections 2024-04-25T09:56:13.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:13.985+0000 7f112511d640 1 -- 172.21.15.19:0/4239999894 wait complete. 2024-04-25T09:56:14.000 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T09:56:14.111 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:14.111 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T09:56:14.111 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:14.120 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:56:14.120 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:56:14.120 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:14.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:14.135 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T09:56:14.199 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:56:14.200 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:56:14.200 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769778 s, 136 MB/s 2024-04-25T09:56:14.200 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T09:56:14.200 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:14.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.337+0000 7f63d184a640 1 Processor -- start 2024-04-25T09:56:14.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.337+0000 7f63d184a640 1 -- start start 2024-04-25T09:56:14.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.337+0000 7f63d184a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc15dd30 0x7f63cc0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.337+0000 7f63d184a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f63cc0d0ad0 0x7f63cc0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63d184a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63cc0d1df0 con 0x7f63cc0b9770 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63d184a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc0d20d0 con 0x7f63cc0d0ad0 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63d184a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0d23b0 con 0x7f63cc15dd30 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63ca7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f63cc0b9770 0x7f63cc0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41488/0 (socket says 172.21.15.19:41488) 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63ca7fc640 1 -- 172.21.15.19:0/734770314 learned_addr learned my addr 172.21.15.19:0/734770314 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63caffd640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc15dd30 0x7f63cc0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141366 0 0) 0x7f63cc0d1df0 con 0x7f63cc0b9770 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63cb7fe640 1 --2- 172.21.15.19:0/734770314 >> v2:172.21.15.112:3300/0 conn(0x7f63cc0d0ad0 0x7f63cc0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63b4003520 con 0x7f63cc0b9770 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 270977082 0 0) 0x7f63b4003520 con 0x7f63cc0b9770 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc15dd30 msgr2=0x7f63cc0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc15dd30 0x7f63cc0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 >> v2:172.21.15.112:3300/0 conn(0x7f63cc0d0ad0 msgr2=0x7f63cc0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 --2- 172.21.15.19:0/734770314 >> v2:172.21.15.112:3300/0 conn(0x7f63cc0d0ad0 0x7f63cc0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc0d2690 con 0x7f63cc0b9770 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f63c0005090 con 0x7f63cc0b9770 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 >> v1:172.21.15.19:6789/0 conn(0x7f63cc0b9770 legacy=0x7f63cc0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 0x7f63b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc15dd30 0x7f63b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b4004d40 0x7f63b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0d20d0 con 0x7f63b40039b0 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0d23b0 con 0x7f63b4004d40 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b400b9c0 con 0x7f63cc15dd30 2024-04-25T09:56:14.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.338+0000 7f63caffd640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 0x7f63b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.339+0000 7f63ca7fc640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc15dd30 0x7f63b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.339+0000 7f63cb7fe640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b4004d40 0x7f63b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.339+0000 7f63caffd640 1 -- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc15dd30 msgr2=0x7f63b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.339+0000 7f63caffd640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc15dd30 0x7f63b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.339+0000 7f63caffd640 1 -- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b4004d40 msgr2=0x7f63b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.339+0000 7f63caffd640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b4004d40 0x7f63b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.339+0000 7f63caffd640 1 -- 172.21.15.19:0/734770314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4003520 con 0x7f63b40039b0 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.339+0000 7f63cb7fe640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b4004d40 0x7f63b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.339+0000 7f63ca7fc640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc15dd30 0x7f63b4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.339+0000 7f63caffd640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 0x7f63b4003dd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009ac0 tx=0x7f63b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f62376bb4f3a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.340+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc036070 con 0x7f63b40039b0 2024-04-25T09:56:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.340+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc031400 con 0x7f63b40039b0 2024-04-25T09:56:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.340+0000 7f63c9ffb640 1 -- 172.21.15.19:0/734770314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc009690 con 0x7f63b40039b0 2024-04-25T09:56:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.340+0000 7f63d184a640 1 -- 172.21.15.19:0/734770314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 msgr2=0x7f63b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.340+0000 7f63d184a640 1 --2- 172.21.15.19:0/734770314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 0x7f63b4003dd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009ac0 tx=0x7f63b4005830 comp rx=0 tx=0).stop 2024-04-25T09:56:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.340+0000 7f63cb7fe640 1 -- 172.21.15.19:0/734770314 reap_dead start 2024-04-25T09:56:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.340+0000 7f63d184a640 1 -- 172.21.15.19:0/734770314 shutdown_connections 2024-04-25T09:56:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.340+0000 7f63d184a640 1 -- 172.21.15.19:0/734770314 >> 172.21.15.19:0/734770314 conn(0x7f63cc061c30 msgr2=0x7f63cc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.340+0000 7f63d184a640 1 -- 172.21.15.19:0/734770314 shutdown_connections 2024-04-25T09:56:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.340+0000 7f63d184a640 1 -- 172.21.15.19:0/734770314 wait complete. 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.341+0000 7f63d184a640 1 Processor -- start 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.342+0000 7f63d184a640 1 -- start start 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.342+0000 7f63d184a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 0x7f63cc0ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.342+0000 7f63d184a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63cc151d00 0x7f63cc1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.342+0000 7f63d184a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc160be0 0x7f63cc0aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.342+0000 7f63d184a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0d1b90 con 0x7f63b40039b0 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.342+0000 7f63d184a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0d1a10 con 0x7f63cc151d00 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.342+0000 7f63d184a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0d2e90 con 0x7f63cc160be0 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.342+0000 7f63caffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 0x7f63cc0ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.342+0000 7f63caffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 0x7f63cc0ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34122/0 (socket says 172.21.15.19:34122) 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.342+0000 7f63caffd640 1 -- 172.21.15.19:0/1250908739 learned_addr learned my addr 172.21.15.19:0/1250908739 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.343+0000 7f63caffd640 1 -- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc160be0 msgr2=0x7f63cc0aa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.343+0000 7f63caffd640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc160be0 0x7f63cc0aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.343+0000 7f63caffd640 1 -- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63cc151d00 msgr2=0x7f63cc1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.343+0000 7f63caffd640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63cc151d00 0x7f63cc1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.343+0000 7f63caffd640 1 -- 172.21.15.19:0/1250908739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc0ac590 con 0x7f63b40039b0 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.343+0000 7f63caffd640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 0x7f63cc0ac050 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009ac0 tx=0x7f63bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.343+0000 7f63b3fff640 1 -- 172.21.15.19:0/1250908739 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc040020 con 0x7f63b40039b0 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.343+0000 7f63b3fff640 1 -- 172.21.15.19:0/1250908739 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc003b40 con 0x7f63b40039b0 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.343+0000 7f63d184a640 1 -- 172.21.15.19:0/1250908739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc0aafb0 con 0x7f63b40039b0 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.343+0000 7f63b3fff640 1 -- 172.21.15.19:0/1250908739 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc036040 con 0x7f63b40039b0 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.343+0000 7f63d184a640 1 -- 172.21.15.19:0/1250908739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc0ab570 con 0x7f63b40039b0 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.344+0000 7f63d184a640 1 -- 172.21.15.19:0/1250908739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63cc161000 con 0x7f63b40039b0 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.344+0000 7f63b3fff640 1 -- 172.21.15.19:0/1250908739 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63bc043020 con 0x7f63b40039b0 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.345+0000 7f63b3fff640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63b806b4d0 0x7f63b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.345+0000 7f63b3fff640 1 -- 172.21.15.19:0/1250908739 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63bc0aa3e0 con 0x7f63b40039b0 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.345+0000 7f63ca7fc640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63b806b4d0 0x7f63b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.346+0000 7f63ca7fc640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63b806b4d0 0x7f63b806d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0cbb00 tx=0x7f63c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:14.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.349+0000 7f63b3fff640 1 -- 172.21.15.19:0/1250908739 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63bc04a070 con 0x7f63b40039b0 2024-04-25T09:56:14.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.565+0000 7f63d184a640 1 -- 172.21.15.19:0/1250908739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63cc15b2a0 con 0x7f63b40039b0 2024-04-25T09:56:14.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.566+0000 7f63b3fff640 1 -- 172.21.15.19:0/1250908739 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f63bc0044e0 con 0x7f63b40039b0 2024-04-25T09:56:14.570 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:14.570 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:56:13.029515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:14.570 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-25T09:56:14.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 -- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63b806b4d0 msgr2=0x7f63b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63b806b4d0 0x7f63b806d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0cbb00 tx=0x7f63c0009040 comp rx=0 tx=0).stop 2024-04-25T09:56:14.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 -- 172.21.15.19:0/1250908739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 msgr2=0x7f63cc0ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 0x7f63cc0ac050 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009ac0 tx=0x7f63bc004730 comp rx=0 tx=0).stop 2024-04-25T09:56:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 -- 172.21.15.19:0/1250908739 shutdown_connections 2024-04-25T09:56:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63b806b4d0 0x7f63b806d990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63cc160be0 0x7f63cc0aa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63cc151d00 0x7f63cc1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 --2- 172.21.15.19:0/1250908739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40039b0 0x7f63cc0ac050 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 -- 172.21.15.19:0/1250908739 >> 172.21.15.19:0/1250908739 conn(0x7f63cc061c30 msgr2=0x7f63cc0b8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 -- 172.21.15.19:0/1250908739 shutdown_connections 2024-04-25T09:56:14.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.568+0000 7f63d184a640 1 -- 172.21.15.19:0/1250908739 wait complete. 2024-04-25T09:56:14.583 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T09:56:14.694 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:14.694 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T09:56:14.694 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:14.703 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:56:14.703 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T09:56:14.703 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T09:56:14.703 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T09:56:14.703 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:56:14.703 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T09:56:14.703 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T09:56:14.770 INFO:teuthology.orchestra.run.smithi019.stdout:1099511630188 2024-04-25T09:56:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:56:14.779 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:14.781 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:56:14.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.919+0000 7f1fd71e7640 1 Processor -- start 2024-04-25T09:56:14.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.919+0000 7f1fd71e7640 1 -- start start 2024-04-25T09:56:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.919+0000 7f1fd71e7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd015dcd0 0x7f1fd006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.919+0000 7f1fd71e7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1fd006b8c0 0x7f1fd00b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.919+0000 7f1fd71e7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd00b3450 con 0x7f1fd00b49f0 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.919+0000 7f1fd71e7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd00b36f0 con 0x7f1fd006b8c0 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.919+0000 7f1fd71e7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd00b39d0 con 0x7f1fd015dcd0 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.920+0000 7f1fcffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1fd006b8c0 0x7f1fd00b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.920+0000 7f1fcffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1fd006b8c0 0x7f1fd00b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34128/0 (socket says 172.21.15.19:34128) 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.920+0000 7f1fcffff640 1 -- 172.21.15.19:0/1034079411 learned_addr learned my addr 172.21.15.19:0/1034079411 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.920+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/1034079411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd015dcd0 0x7f1fd006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.920+0000 7f1fcffff640 1 -- 172.21.15.19:0/1034079411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd015dcd0 msgr2=0x7f1fd006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.920+0000 7f1fcffff640 1 --2- 172.21.15.19:0/1034079411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd015dcd0 0x7f1fd006d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.920+0000 7f1fcffff640 1 -- 172.21.15.19:0/1034079411 >> v1:172.21.15.112:6789/0 conn(0x7f1fd00b49f0 legacy=0x7f1fd006e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.920+0000 7f1fcffff640 1 -- 172.21.15.19:0/1034079411 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd00b3cb0 con 0x7f1fd006b8c0 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fcffff640 1 --2- 172.21.15.19:0/1034079411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd006b8c0 0x7f1fd00b44b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009870 tx=0x7f1fc402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87a1912ee23569f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fcf7fe640 1 -- 172.21.15.19:0/1034079411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4037070 con 0x7f1fd006b8c0 2024-04-25T09:56:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fcf7fe640 1 -- 172.21.15.19:0/1034079411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc4032440 con 0x7f1fd006b8c0 2024-04-25T09:56:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fcf7fe640 1 -- 172.21.15.19:0/1034079411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4032810 con 0x7f1fd006b8c0 2024-04-25T09:56:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fd71e7640 1 -- 172.21.15.19:0/1034079411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd006b8c0 msgr2=0x7f1fd00b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fd71e7640 1 --2- 172.21.15.19:0/1034079411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd006b8c0 0x7f1fd00b44b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009870 tx=0x7f1fc402a210 comp rx=0 tx=0).stop 2024-04-25T09:56:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fd71e7640 1 -- 172.21.15.19:0/1034079411 shutdown_connections 2024-04-25T09:56:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fd71e7640 1 --2- 172.21.15.19:0/1034079411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd006b8c0 0x7f1fd00b44b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fd71e7640 1 --2- 172.21.15.19:0/1034079411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd015dcd0 0x7f1fd006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fd71e7640 1 -- 172.21.15.19:0/1034079411 >> 172.21.15.19:0/1034079411 conn(0x7f1fd006abe0 msgr2=0x7f1fd006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fd71e7640 1 -- 172.21.15.19:0/1034079411 shutdown_connections 2024-04-25T09:56:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.921+0000 7f1fd71e7640 1 -- 172.21.15.19:0/1034079411 wait complete. 2024-04-25T09:56:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.922+0000 7f1fd71e7640 1 Processor -- start 2024-04-25T09:56:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.923+0000 7f1fd71e7640 1 -- start start 2024-04-25T09:56:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.923+0000 7f1fd71e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd006b8c0 0x7f1fd01f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.923+0000 7f1fd71e7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd00b49f0 0x7f1fd014e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.923+0000 7f1fd71e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd015dcd0 0x7f1fd0153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.923+0000 7f1fd71e7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd006dda0 con 0x7f1fd006b8c0 2024-04-25T09:56:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.923+0000 7f1fd71e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd006dc20 con 0x7f1fd015dcd0 2024-04-25T09:56:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.923+0000 7f1fd71e7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd006ea00 con 0x7f1fd00b49f0 2024-04-25T09:56:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.923+0000 7f1fd4f83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd006b8c0 0x7f1fd01f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd4f83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd006b8c0 0x7f1fd01f7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34130/0 (socket says 172.21.15.19:34130) 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd4f83640 1 -- 172.21.15.19:0/552094902 learned_addr learned my addr 172.21.15.19:0/552094902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fcffff640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd00b49f0 0x7f1fd014e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd5784640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd015dcd0 0x7f1fd0153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd4f83640 1 -- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd00b49f0 msgr2=0x7f1fd014e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd00b49f0 0x7f1fd014e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd4f83640 1 -- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd015dcd0 msgr2=0x7f1fd0153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd015dcd0 0x7f1fd0153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd4f83640 1 -- 172.21.15.19:0/552094902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd0153fe0 con 0x7f1fd006b8c0 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd5784640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd015dcd0 0x7f1fd0153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fcffff640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd00b49f0 0x7f1fd014e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd006b8c0 0x7f1fd01f7e60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8007e40 tx=0x7f1fc800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fcd7fa640 1 -- 172.21.15.19:0/552094902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8014620 con 0x7f1fd006b8c0 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fcd7fa640 1 -- 172.21.15.19:0/552094902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc8019070 con 0x7f1fd006b8c0 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd71e7640 1 -- 172.21.15.19:0/552094902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd01f4520 con 0x7f1fd006b8c0 2024-04-25T09:56:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fcd7fa640 1 -- 172.21.15.19:0/552094902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8014a00 con 0x7f1fd006b8c0 2024-04-25T09:56:14.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.924+0000 7f1fd71e7640 1 -- 172.21.15.19:0/552094902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd01f4ae0 con 0x7f1fd006b8c0 2024-04-25T09:56:14.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.926+0000 7f1fcd7fa640 1 -- 172.21.15.19:0/552094902 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1fc8014ba0 con 0x7f1fd006b8c0 2024-04-25T09:56:14.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.926+0000 7f1fcd7fa640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.926+0000 7f1fcd7fa640 1 -- 172.21.15.19:0/552094902 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1fc808ac40 con 0x7f1fd006b8c0 2024-04-25T09:56:14.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.926+0000 7f1fcffff640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.927+0000 7f1fcffff640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4034000 tx=0x7f1fc402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:14.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.927+0000 7f1fd9fc8740 1 Processor -- start 2024-04-25T09:56:14.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.927+0000 7f1fd9fc8740 1 -- start start 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1aa0 0x5627999c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1e80 0x5627999c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562799994700 0x562799997f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627998fde00 con 0x5627999c1aa0 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627998a6d20 con 0x5627999c1e80 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627998f9ea0 con 0x562799994700 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd4f83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1aa0 0x5627999c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd4f83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1aa0 0x5627999c42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34138/0 (socket says 172.21.15.19:34138) 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd4f83640 1 -- 172.21.15.19:0/1276767464 learned_addr learned my addr 172.21.15.19:0/1276767464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fcffff640 1 --2- 172.21.15.19:0/1276767464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1e80 0x5627999c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd5784640 1 --2- 172.21.15.19:0/1276767464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562799994700 0x562799997f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd4f83640 1 -- 172.21.15.19:0/1276767464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562799994700 msgr2=0x562799997f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/1276767464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562799994700 0x562799997f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd4f83640 1 -- 172.21.15.19:0/1276767464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1e80 msgr2=0x5627999c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/1276767464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1e80 0x5627999c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd4f83640 1 -- 172.21.15.19:0/1276767464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627998c8df0 con 0x5627999c1aa0 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd5784640 1 --2- 172.21.15.19:0/1276767464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562799994700 0x562799997f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.928+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/1276767464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1aa0 0x5627999c42b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8055680 tx=0x7f1fc8059920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fae5bb32177014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fb67fc640 1 -- 172.21.15.19:0/1276767464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc805d370 con 0x5627999c1aa0 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fb67fc640 1 -- 172.21.15.19:0/1276767464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc8061930 con 0x5627999c1aa0 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fb67fc640 1 -- 172.21.15.19:0/1276767464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8055890 con 0x5627999c1aa0 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1276767464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1aa0 msgr2=0x5627999c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/1276767464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1aa0 0x5627999c42b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8055680 tx=0x7f1fc8059920 comp rx=0 tx=0).stop 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1276767464 shutdown_connections 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/1276767464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562799994700 0x562799997f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/1276767464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1e80 0x5627999c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/1276767464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1aa0 0x5627999c42b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1276767464 >> 172.21.15.19:0/1276767464 conn(0x562799996970 msgr2=0x5627999df2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1276767464 shutdown_connections 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.929+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1276767464 wait complete. 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.930+0000 7f1fd9fc8740 1 -- start start 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.930+0000 7f1fd9fc8740 1 Processor -- start 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x5627999d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999c1e80 0x5627999d9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562799999b20 con 0x5627999e2840 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279989c590 con 0x5627999c1aa0 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562799999e00 con 0x5627999c1e80 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd9fc8740 10 client.0 _ll_register_callbacks cb 0x5627998a8e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd5784640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd5784640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34146/0 (socket says 172.21.15.19:34146) 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd5784640 1 -- 172.21.15.19:0/331313561 learned_addr learned my addr 172.21.15.19:0/331313561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fcffff640 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999c1e80 0x5627999d9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x5627999d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd5784640 1 -- 172.21.15.19:0/331313561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999c1e80 msgr2=0x5627999d9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd5784640 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999c1e80 0x5627999d9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd5784640 1 -- 172.21.15.19:0/331313561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 msgr2=0x5627999d98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd5784640 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x5627999d98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd5784640 1 -- 172.21.15.19:0/331313561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627996123f0 con 0x5627999e2840 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1fd5784640 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e03e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000f170 tx=0x7f1fc000f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d25a60150d67923 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1faf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.931+0000 7f1faf7fe640 1 -- 172.21.15.19:0/331313561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0018020 con 0x5627999e2840 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1faf7fe640 1 -- 172.21.15.19:0/331313561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1fc0014410 con 0x5627999e2840 2024-04-25T09:56:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/331313561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fc8061930 con 0x5627999e2840 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1faf7fe640 1 -- 172.21.15.19:0/331313561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fc0022050 con 0x5627999e2840 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1fd9fc8740 10 client.6264 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/331313561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5627996dc7d0 con 0x5627999e2840 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1faf7fe640 1 -- 172.21.15.19:0/331313561 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc8061930 con 0x5627999e2840 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1faf7fe640 1 -- 172.21.15.19:0/331313561 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1fc0025020 con 0x5627999e2840 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1fd9fc8740 10 client.6264 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1fd9fc8740 10 client.6264 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1fd9fc8740 20 client.6264 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5627999e0920 0x5627999e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1fd9fc8740 4 client.6264 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/331313561 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56279990b340 con 0x5627999e0920 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5627999e0920 0x5627999e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1faf7fe640 10 client.6264 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.932+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5627999e0920 0x5627999e0d30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e4e042daebdb0d server_cookie=39bd68531d23a12a in_seq=0 out_seq=0 2024-04-25T09:56:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1faf7fe640 1 -- 172.21.15.19:0/331313561 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56279990b340 con 0x5627999e0920 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1faf7fe640 10 client.6264 handle_command_reply: tid=0 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 1 client.6264 shutdown 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/331313561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 msgr2=0x5627999e03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e03e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000f170 tx=0x7f1fc000f630 comp rx=0 tx=0).stop 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/331313561 shutdown_connections 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5627999e0920 0x5627999e0d30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e03e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999c1e80 0x5627999d9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/331313561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x5627999d98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1faf7fe640 0 client.6264 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/331313561 >> 172.21.15.19:0/331313561 conn(0x562799994700 msgr2=0x5627999953b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/331313561 shutdown_connections 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/331313561 wait complete. 2024-04-25T09:56:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.933+0000 7f1fd9fc8740 20 client.6264 trim_cache size 0 max 16384 2024-04-25T09:56:14.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.945+0000 7f1fd9fc8740 1 Processor -- start 2024-04-25T09:56:14.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.945+0000 7f1fd9fc8740 1 -- start start 2024-04-25T09:56:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x5627999900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1e80 0x5627999e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999e2840 0x5627999f8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd0200410 con 0x5627999c1e80 2024-04-25T09:56:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd01ff6c0 con 0x5627999c1aa0 2024-04-25T09:56:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd00cd8c0 con 0x5627999e2840 2024-04-25T09:56:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fcffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1e80 0x5627999e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fcffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1e80 0x5627999e0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34162/0 (socket says 172.21.15.19:34162) 2024-04-25T09:56:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fcffff640 1 -- 172.21.15.19:0/968376771 learned_addr learned my addr 172.21.15.19:0/968376771 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/968376771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x5627999900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fd5784640 1 --2- 172.21.15.19:0/968376771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999e2840 0x5627999f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fcffff640 1 -- 172.21.15.19:0/968376771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999e2840 msgr2=0x5627999f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fcffff640 1 --2- 172.21.15.19:0/968376771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999e2840 0x5627999f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fcffff640 1 -- 172.21.15.19:0/968376771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 msgr2=0x5627999900b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fcffff640 1 --2- 172.21.15.19:0/968376771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x5627999900b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fcffff640 1 -- 172.21.15.19:0/968376771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627999f04e0 con 0x5627999c1e80 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fd5784640 1 --2- 172.21.15.19:0/968376771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999e2840 0x5627999f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.946+0000 7f1fcffff640 1 --2- 172.21.15.19:0/968376771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1e80 0x5627999e0ef0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4000c00 tx=0x7f1fc402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd84ba4a53ed021 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fb5ffb640 1 -- 172.21.15.19:0/968376771 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4037040 con 0x5627999c1e80 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fb5ffb640 1 -- 172.21.15.19:0/968376771 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc4048470 con 0x5627999c1e80 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fb5ffb640 1 -- 172.21.15.19:0/968376771 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4048820 con 0x5627999c1e80 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/968376771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1e80 msgr2=0x5627999e0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/968376771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1e80 0x5627999e0ef0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4000c00 tx=0x7f1fc402c000 comp rx=0 tx=0).stop 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/968376771 shutdown_connections 2024-04-25T09:56:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/968376771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999e2840 0x5627999f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/968376771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999c1e80 0x5627999e0ef0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/968376771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x5627999900b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/968376771 >> 172.21.15.19:0/968376771 conn(0x562799994700 msgr2=0x5627999da880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/968376771 shutdown_connections 2024-04-25T09:56:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.947+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/968376771 wait complete. 2024-04-25T09:56:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.948+0000 7f1fd9fc8740 1 -- start start 2024-04-25T09:56:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.948+0000 7f1fd9fc8740 1 Processor -- start 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x562799a30240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999c1e80 0x5627999e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd9fc8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd00cdec0 con 0x5627999e2840 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279999acf0 con 0x5627999c1aa0 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd9fc8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627999f8c60 con 0x5627999c1e80 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd9fc8740 10 client.0 _ll_register_callbacks cb 0x7f1fc800ef10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd5784640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd5784640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34166/0 (socket says 172.21.15.19:34166) 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd5784640 1 -- 172.21.15.19:0/1173450067 learned_addr learned my addr 172.21.15.19:0/1173450067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x562799a30240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fcffff640 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999c1e80 0x5627999e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd5784640 1 -- 172.21.15.19:0/1173450067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999c1e80 msgr2=0x5627999e0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd5784640 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999c1e80 0x5627999e0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd5784640 1 -- 172.21.15.19:0/1173450067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 msgr2=0x562799a30240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd5784640 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x562799a30240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd5784640 1 -- 172.21.15.19:0/1173450067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627999da190 con 0x5627999e2840 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1fd5784640 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e1430 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0005550 tx=0x7f1fc0003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c72fbd2704d9dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.949+0000 7f1faf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1faf7fe640 1 -- 172.21.15.19:0/1173450067 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0018040 con 0x5627999e2840 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1faf7fe640 1 -- 172.21.15.19:0/1173450067 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1fc0014410 con 0x5627999e2840 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1173450067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fc4048470 con 0x5627999e2840 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1faf7fe640 1 -- 172.21.15.19:0/1173450067 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fc0010040 con 0x5627999e2840 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1fd9fc8740 10 client.6270 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1173450067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5627999e97d0 con 0x5627999e2840 2024-04-25T09:56:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1faf7fe640 1 -- 172.21.15.19:0/1173450067 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc4048470 con 0x5627999e2840 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1faf7fe640 1 -- 172.21.15.19:0/1173450067 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1fc0037050 con 0x5627999e2840 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1fd9fc8740 10 client.6270 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1fd9fc8740 10 client.6270 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1fd9fc8740 20 client.6270 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5627999dbd20 0x5627999e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1fd9fc8740 4 client.6270 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1173450067 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5627999c5c50 con 0x5627999dbd20 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5627999dbd20 0x5627999e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1faf7fe640 10 client.6270 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.950+0000 7f1fd4f83640 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5627999dbd20 0x5627999e7f40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00e47d892e73f73 server_cookie=23ae9ea2aba7aa5f in_seq=0 out_seq=0 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1faf7fe640 1 -- 172.21.15.19:0/1173450067 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5627999c5c50 con 0x5627999dbd20 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1faf7fe640 10 client.6270 handle_command_reply: tid=0 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 1 client.6270 shutdown 2024-04-25T09:56:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1173450067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 msgr2=0x5627999e1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e1430 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0005550 tx=0x7f1fc0003760 comp rx=0 tx=0).stop 2024-04-25T09:56:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1173450067 shutdown_connections 2024-04-25T09:56:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5627999dbd20 0x5627999e7f40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627999e2840 0x5627999e1430 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627999c1e80 0x5627999e0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 1 --2- 172.21.15.19:0/1173450067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627999c1aa0 0x562799a30240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1faf7fe640 0 client.6270 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1173450067 >> 172.21.15.19:0/1173450067 conn(0x562799994700 msgr2=0x56279998fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1173450067 shutdown_connections 2024-04-25T09:56:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 1 -- 172.21.15.19:0/1173450067 wait complete. 2024-04-25T09:56:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.951+0000 7f1fd9fc8740 20 client.6270 trim_cache size 0 max 16384 2024-04-25T09:56:14.962 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T09:56:14.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 -- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fbc06b4d0 msgr2=0x7f1fbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4034000 tx=0x7f1fc402e000 comp rx=0 tx=0).stop 2024-04-25T09:56:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 -- 172.21.15.19:0/552094902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd006b8c0 msgr2=0x7f1fd01f7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd006b8c0 0x7f1fd01f7e60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8007e40 tx=0x7f1fc800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 -- 172.21.15.19:0/552094902 shutdown_connections 2024-04-25T09:56:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd015dcd0 0x7f1fd0153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd00b49f0 0x7f1fd014e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 --2- 172.21.15.19:0/552094902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd006b8c0 0x7f1fd01f7e60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 -- 172.21.15.19:0/552094902 >> 172.21.15.19:0/552094902 conn(0x7f1fd006abe0 msgr2=0x7f1fd00b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 -- 172.21.15.19:0/552094902 shutdown_connections 2024-04-25T09:56:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:14.952+0000 7f1fd71e7640 1 -- 172.21.15.19:0/552094902 wait complete. 2024-04-25T09:56:14.967 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096c.00000000 parent 2024-04-25T09:56:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.005+0000 7f20d6c16f40 1 Processor -- start 2024-04-25T09:56:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.005+0000 7f20d6c16f40 1 -- start start 2024-04-25T09:56:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d6c16f40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994553ede0 0x559945614270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d6c16f40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559945589a80 0x5599456115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d6c16f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559945616060 0x559945616c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d6c16f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559945589f30 con 0x559945616060 2024-04-25T09:56:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d6c16f40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599454e7df0 con 0x55994553ede0 2024-04-25T09:56:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d6c16f40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55994558a1d0 con 0x559945589a80 2024-04-25T09:56:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d42a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559945616060 0x559945616c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d42a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559945616060 0x559945616c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34194/0 (socket says 172.21.15.19:34194) 2024-04-25T09:56:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d42a7640 1 -- 172.21.15.19:0/4070715960 learned_addr learned my addr 172.21.15.19:0/4070715960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d3aa6640 1 --2- 172.21.15.19:0/4070715960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994553ede0 0x559945614270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d32a5640 1 --2- 172.21.15.19:0/4070715960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559945589a80 0x5599456115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d42a7640 1 -- 172.21.15.19:0/4070715960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559945589a80 msgr2=0x5599456115d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d42a7640 1 --2- 172.21.15.19:0/4070715960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559945589a80 0x5599456115d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d42a7640 1 -- 172.21.15.19:0/4070715960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994553ede0 msgr2=0x559945614270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d42a7640 1 --2- 172.21.15.19:0/4070715960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994553ede0 0x559945614270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d42a7640 1 -- 172.21.15.19:0/4070715960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55994546dc50 con 0x559945616060 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d3aa6640 1 --2- 172.21.15.19:0/4070715960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994553ede0 0x559945614270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d32a5640 1 --2- 172.21.15.19:0/4070715960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559945589a80 0x5599456115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.006+0000 7f20d42a7640 1 --2- 172.21.15.19:0/4070715960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559945616060 0x559945616c70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f20c4001e50 tx=0x7f20c400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6a1e00840a6039 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d22a3640 1 -- 172.21.15.19:0/4070715960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4008070 con 0x559945616060 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d22a3640 1 -- 172.21.15.19:0/4070715960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c400d500 con 0x559945616060 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d22a3640 1 -- 172.21.15.19:0/4070715960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c400d950 con 0x559945616060 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d6c16f40 1 -- 172.21.15.19:0/4070715960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559945616060 msgr2=0x559945616c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d6c16f40 1 --2- 172.21.15.19:0/4070715960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559945616060 0x559945616c70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f20c4001e50 tx=0x7f20c400b6a0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d6c16f40 1 -- 172.21.15.19:0/4070715960 shutdown_connections 2024-04-25T09:56:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d6c16f40 1 --2- 172.21.15.19:0/4070715960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559945616060 0x559945616c70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d6c16f40 1 --2- 172.21.15.19:0/4070715960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559945589a80 0x5599456115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d6c16f40 1 --2- 172.21.15.19:0/4070715960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994553ede0 0x559945614270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d6c16f40 1 -- 172.21.15.19:0/4070715960 >> 172.21.15.19:0/4070715960 conn(0x55994553f9b0 msgr2=0x55994556a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d6c16f40 1 -- 172.21.15.19:0/4070715960 shutdown_connections 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.007+0000 7f20d6c16f40 1 -- 172.21.15.19:0/4070715960 wait complete. 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.008+0000 7f20d6c16f40 1 Processor -- start 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.008+0000 7f20d6c16f40 1 -- start start 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.008+0000 7f20d6c16f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55994553ede0 0x5599456ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.008+0000 7f20d6c16f40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559945589a80 0x559945543370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d6c16f40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559945616060 0x559945604770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d6c16f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559945611dc0 con 0x55994553ede0 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d6c16f40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559945618de0 con 0x559945616060 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d6c16f40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559945618b00 con 0x559945589a80 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d3aa6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55994553ede0 0x5599456ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d3aa6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55994553ede0 0x5599456ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34208/0 (socket says 172.21.15.19:34208) 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d3aa6640 1 -- 172.21.15.19:0/3216592718 learned_addr learned my addr 172.21.15.19:0/3216592718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d32a5640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559945589a80 0x559945543370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d42a7640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559945616060 0x559945604770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d3aa6640 1 -- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559945589a80 msgr2=0x559945543370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d3aa6640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559945589a80 0x559945543370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d3aa6640 1 -- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559945616060 msgr2=0x559945604770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d3aa6640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559945616060 0x559945604770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d3aa6640 1 -- 172.21.15.19:0/3216592718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559945586ce0 con 0x55994553ede0 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d32a5640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559945589a80 0x559945543370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d42a7640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559945616060 0x559945604770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d3aa6640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55994553ede0 0x5599456ae2e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00b480 tx=0x7f20cc00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d12a1640 1 -- 172.21.15.19:0/3216592718 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc016020 con 0x55994553ede0 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d12a1640 1 -- 172.21.15.19:0/3216592718 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc012660 con 0x55994553ede0 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d6c16f40 1 -- 172.21.15.19:0/3216592718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55994560e250 con 0x55994553ede0 2024-04-25T09:56:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d12a1640 1 -- 172.21.15.19:0/3216592718 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc012a10 con 0x55994553ede0 2024-04-25T09:56:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.009+0000 7f20d6c16f40 1 -- 172.21.15.19:0/3216592718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559945570a30 con 0x55994553ede0 2024-04-25T09:56:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.011+0000 7f20d12a1640 1 -- 172.21.15.19:0/3216592718 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20cc019020 con 0x55994553ede0 2024-04-25T09:56:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.011+0000 7f20d12a1640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20b4069eb0 0x7f20b406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.011+0000 7f20d12a1640 1 -- 172.21.15.19:0/3216592718 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20cc0895a0 con 0x55994553ede0 2024-04-25T09:56:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.011+0000 7f20d32a5640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20b4069eb0 0x7f20b406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.011+0000 7f20d6c16f40 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x55994560ce00 0x55994560d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.011+0000 7f20d6c16f40 1 -- 172.21.15.19:0/3216592718 --> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] -- osd_op(unknown.0.0:1 8.2b 8:d716f75e:::1000000096c.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53) v9 -- 0x55994560d760 con 0x55994560ce00 2024-04-25T09:56:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.012+0000 7f20d42a7640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x55994560ce00 0x55994560d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.012+0000 7f20d32a5640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20b4069eb0 0x7f20b406c370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f20bc057000 tx=0x7f20bc052000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.012+0000 7f20d42a7640 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x55994560ce00 0x55994560d220 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.013+0000 7f20d42a7640 1 -- 172.21.15.19:0/3216592718 <== osd.6 v2:172.21.15.112:6816/3693987094 1 ==== osd_op_reply(1 1000000096c.00000000 [rmxattr] v53'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f20c400d730 con 0x55994560ce00 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.013+0000 7f20d6c16f40 1 -- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x55994560ce00 msgr2=0x55994560d220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.013+0000 7f20d6c16f40 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x55994560ce00 0x55994560d220 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.013+0000 7f20d6c16f40 1 -- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20b4069eb0 msgr2=0x7f20b406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.013+0000 7f20d6c16f40 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20b4069eb0 0x7f20b406c370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f20bc057000 tx=0x7f20bc052000 comp rx=0 tx=0).stop 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.013+0000 7f20d6c16f40 1 -- 172.21.15.19:0/3216592718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55994553ede0 msgr2=0x5599456ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.013+0000 7f20d6c16f40 1 --2- 172.21.15.19:0/3216592718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55994553ede0 0x5599456ae2e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00b480 tx=0x7f20cc00b940 comp rx=0 tx=0).stop 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.013+0000 7f20d42a7640 1 -- 172.21.15.19:0/3216592718 reap_dead start 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.013+0000 7f20d6c16f40 1 -- 172.21.15.19:0/3216592718 shutdown_connections 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.013+0000 7f20d6c16f40 1 -- 172.21.15.19:0/3216592718 >> 172.21.15.19:0/3216592718 conn(0x55994553f9b0 msgr2=0x5599456115d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.014+0000 7f20d6c16f40 1 -- 172.21.15.19:0/3216592718 shutdown_connections 2024-04-25T09:56:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.014+0000 7f20d6c16f40 1 -- 172.21.15.19:0/3216592718 wait complete. 2024-04-25T09:56:15.021 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T09:56:15.022 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:15.022 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T09:56:15.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.160+0000 7f7c83fc1640 1 Processor -- start 2024-04-25T09:56:15.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.160+0000 7f7c83fc1640 1 -- start start 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.161+0000 7f7c83fc1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7c7c15b7d0 0x7f7c7c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.161+0000 7f7c83fc1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c7c06cf50 con 0x7f7c7c06c9e0 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.161+0000 7f7c83fc1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c7c0ce530 con 0x7f7c7c0b5620 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.161+0000 7f7c83fc1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c7c0ce810 con 0x7f7c7c15b7d0 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.161+0000 7f7c8255e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7c7c06c9e0 0x7f7c7c0d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41508/0 (socket says 172.21.15.19:41508) 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.161+0000 7f7c8255e640 1 -- 172.21.15.19:0/1149979752 learned_addr learned my addr 172.21.15.19:0/1149979752 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.161+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696282101 0 0) 0x7f7c7c06cf50 con 0x7f7c7c06c9e0 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.161+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c68003520 con 0x7f7c7c06c9e0 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.161+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/1149979752 >> v2:172.21.15.112:3300/0 conn(0x7f7c7c15b7d0 0x7f7c7c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3980974412 0 0) 0x7f7c68003520 con 0x7f7c7c06c9e0 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 >> v2:172.21.15.112:3300/0 conn(0x7f7c7c15b7d0 msgr2=0x7f7c7c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 --2- 172.21.15.19:0/1149979752 >> v2:172.21.15.112:3300/0 conn(0x7f7c7c15b7d0 0x7f7c7c0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 >> v1:172.21.15.112:6789/0 conn(0x7f7c7c0b5620 legacy=0x7f7c7c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c0ceaf0 con 0x7f7c7c06c9e0 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f7c740019f0 con 0x7f7c7c06c9e0 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 >> v1:172.21.15.19:6789/0 conn(0x7f7c7c06c9e0 legacy=0x7f7c7c0d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c680039b0 0x7f7c68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c68004800 0x7f7c68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c680051c0 0x7f7c6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0ce810 con 0x7f7c680051c0 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6800c060 con 0x7f7c680039b0 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6800c260 con 0x7f7c68004800 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c8255e640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c680051c0 0x7f7c6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c8155c640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c68004800 0x7f7c68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c680039b0 0x7f7c68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c8255e640 1 -- 172.21.15.19:0/1149979752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c68004800 msgr2=0x7f7c68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c8255e640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c68004800 0x7f7c68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c8255e640 1 -- 172.21.15.19:0/1149979752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c680039b0 msgr2=0x7f7c68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c8255e640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c680039b0 0x7f7c68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c8255e640 1 -- 172.21.15.19:0/1149979752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c68003520 con 0x7f7c680051c0 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.162+0000 7f7c8155c640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c68004800 0x7f7c68004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.163+0000 7f7c8255e640 1 -- 172.21.15.19:0/1149979752 reap_dead start 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.163+0000 7f7c8255e640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c680051c0 0x7f7c6800b9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400a840 tx=0x7f7c68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2791afaffd0484c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.163+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7400a460 con 0x7f7c680051c0 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.163+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7402f070 con 0x7f7c680051c0 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.163+0000 7f7c80d5b640 1 -- 172.21.15.19:0/1149979752 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7402bcb0 con 0x7f7c680051c0 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.164+0000 7f7c83fc1640 1 -- 172.21.15.19:0/1149979752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c680051c0 msgr2=0x7f7c6800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.164+0000 7f7c83fc1640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c680051c0 0x7f7c6800b9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400a840 tx=0x7f7c68005830 comp rx=0 tx=0).stop 2024-04-25T09:56:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.164+0000 7f7c83fc1640 1 -- 172.21.15.19:0/1149979752 shutdown_connections 2024-04-25T09:56:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.164+0000 7f7c83fc1640 1 --2- 172.21.15.19:0/1149979752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c680051c0 0x7f7c6800b9c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.164+0000 7f7c83fc1640 1 -- 172.21.15.19:0/1149979752 >> 172.21.15.19:0/1149979752 conn(0x7f7c7c0d0800 msgr2=0x7f7c7c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.164+0000 7f7c83fc1640 1 -- 172.21.15.19:0/1149979752 shutdown_connections 2024-04-25T09:56:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.164+0000 7f7c83fc1640 1 -- 172.21.15.19:0/1149979752 wait complete. 2024-04-25T09:56:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.165+0000 7f7c83fc1640 1 Processor -- start 2024-04-25T09:56:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.165+0000 7f7c83fc1640 1 -- start start 2024-04-25T09:56:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c83fc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c680051c0 0x7f7c7c1f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c83fc1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c7c0c8e90 0x7f7c7c14bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c83fc1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c7c159280 0x7f7c7c159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c83fc1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c15bd30 con 0x7f7c7c159280 2024-04-25T09:56:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c83fc1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c15bbb0 con 0x7f7c680051c0 2024-04-25T09:56:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c83fc1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0b5ab0 con 0x7f7c7c0c8e90 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c8255e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c7c159280 0x7f7c7c159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c8255e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c7c159280 0x7f7c7c159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34228/0 (socket says 172.21.15.19:34228) 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c8255e640 1 -- 172.21.15.19:0/964756818 learned_addr learned my addr 172.21.15.19:0/964756818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c680051c0 0x7f7c7c1f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c8155c640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c7c0c8e90 0x7f7c7c14bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c8255e640 1 -- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c7c0c8e90 msgr2=0x7f7c7c14bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c8255e640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c7c0c8e90 0x7f7c7c14bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c8255e640 1 -- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c680051c0 msgr2=0x7f7c7c1f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c8255e640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c680051c0 0x7f7c7c1f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c8255e640 1 -- 172.21.15.19:0/964756818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c1f4ba0 con 0x7f7c7c159280 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c680051c0 0x7f7c7c1f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c8255e640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c7c159280 0x7f7c7c159700 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7c7402b500 tx=0x7f7c7c0ce700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c727fc640 1 -- 172.21.15.19:0/964756818 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74001fb0 con 0x7f7c7c159280 2024-04-25T09:56:15.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c727fc640 1 -- 172.21.15.19:0/964756818 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c74001250 con 0x7f7c7c159280 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c83fc1640 1 -- 172.21.15.19:0/964756818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7c0c7870 con 0x7f7c7c159280 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c727fc640 1 -- 172.21.15.19:0/964756818 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74034870 con 0x7f7c7c159280 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.166+0000 7f7c83fc1640 1 -- 172.21.15.19:0/964756818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7c0c96c0 con 0x7f7c7c159280 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.168+0000 7f7c727fc640 1 -- 172.21.15.19:0/964756818 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c74034a10 con 0x7f7c7c159280 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.168+0000 7f7c727fc640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c6006b4d0 0x7f7c6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.168+0000 7f7c727fc640 1 -- 172.21.15.19:0/964756818 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7c740aa430 con 0x7f7c7c159280 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.169+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c6006b4d0 0x7f7c6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.169+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c6006b4d0 0x7f7c6006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009b40 tx=0x7f7c78008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.170+0000 7f7c86da1740 1 Processor -- start 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.170+0000 7f7c86da1740 1 -- start start 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.170+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639feccf360 0x5639fecef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.170+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639fecae360 0x5639fecd6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.170+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecd7140 0x5639fecac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.170+0000 7f7c86da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fec0fee0 con 0x5639feccf360 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.170+0000 7f7c86da1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639febb8c50 con 0x5639fecae360 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.170+0000 7f7c86da1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639fec0bf80 con 0x5639fecd7140 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.170+0000 7f7c81d5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639feccf360 0x5639fecef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.170+0000 7f7c81d5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639feccf360 0x5639fecef7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34240/0 (socket says 172.21.15.19:34240) 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.170+0000 7f7c81d5d640 1 -- 172.21.15.19:0/2145704485 learned_addr learned my addr 172.21.15.19:0/2145704485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.171+0000 7f7c8155c640 1 --2- 172.21.15.19:0/2145704485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639fecae360 0x5639fecd6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.171+0000 7f7c8255e640 1 --2- 172.21.15.19:0/2145704485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecd7140 0x5639fecac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.171+0000 7f7c81d5d640 1 -- 172.21.15.19:0/2145704485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecd7140 msgr2=0x5639fecac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.171+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/2145704485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecd7140 0x5639fecac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.171+0000 7f7c81d5d640 1 -- 172.21.15.19:0/2145704485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639fecae360 msgr2=0x5639fecd6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.171+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/2145704485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639fecae360 0x5639fecd6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.171+0000 7f7c81d5d640 1 -- 172.21.15.19:0/2145704485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639fe9f0c00 con 0x5639feccf360 2024-04-25T09:56:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.171+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/2145704485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639feccf360 0x5639fecef7d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7c78014e40 tx=0x7f7c78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a223094ecf15c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.171+0000 7f7c5f7fe640 1 -- 172.21.15.19:0/2145704485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78027070 con 0x5639feccf360 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.171+0000 7f7c5f7fe640 1 -- 172.21.15.19:0/2145704485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c78009790 con 0x5639feccf360 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.171+0000 7f7c5f7fe640 1 -- 172.21.15.19:0/2145704485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78023860 con 0x5639feccf360 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.172+0000 7f7c86da1740 1 -- 172.21.15.19:0/2145704485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639feccf360 msgr2=0x5639fecef7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.172+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2145704485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639feccf360 0x5639fecef7d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7c78014e40 tx=0x7f7c78017000 comp rx=0 tx=0).stop 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.172+0000 7f7c86da1740 1 -- 172.21.15.19:0/2145704485 shutdown_connections 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.172+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2145704485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecd7140 0x5639fecac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.172+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2145704485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639fecae360 0x5639fecd6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.172+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2145704485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639feccf360 0x5639fecef7d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.172+0000 7f7c86da1740 1 -- 172.21.15.19:0/2145704485 >> 172.21.15.19:0/2145704485 conn(0x5639fec73810 msgr2=0x5639fecf1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.172+0000 7f7c86da1740 1 -- 172.21.15.19:0/2145704485 shutdown_connections 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.172+0000 7f7c86da1740 1 -- 172.21.15.19:0/2145704485 wait complete. 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.172+0000 7f7c86da1740 1 -- start start 2024-04-25T09:56:15.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.172+0000 7f7c86da1740 1 Processor -- start 2024-04-25T09:56:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecd9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639feccf360 0x5639fece99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecd7140 0x5639fecea920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c86da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fecac330 con 0x5639fecae360 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c86da1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639febae340 con 0x5639feccf360 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c86da1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639feca71a0 con 0x5639fecd7140 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c86da1740 10 client.0 _ll_register_callbacks cb 0x5639fe923fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c81d5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecd9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c81d5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecd9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34244/0 (socket says 172.21.15.19:34244) 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c81d5d640 1 -- 172.21.15.19:0/2459466491 learned_addr learned my addr 172.21.15.19:0/2459466491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c8155c640 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639feccf360 0x5639fece99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.173+0000 7f7c8255e640 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecd7140 0x5639fecea920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c81d5d640 1 -- 172.21.15.19:0/2459466491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecd7140 msgr2=0x5639fecea920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecd7140 0x5639fecea920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c81d5d640 1 -- 172.21.15.19:0/2459466491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639feccf360 msgr2=0x5639fece99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639feccf360 0x5639fece99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c81d5d640 1 -- 172.21.15.19:0/2459466491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639fe8e44f0 con 0x5639fecae360 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c8155c640 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639feccf360 0x5639fece99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecd9b80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7c78014720 tx=0x7f7c78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e74e48f1edbd49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c5cff9640 1 -- 172.21.15.19:0/2459466491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78032070 con 0x5639fecae360 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c5cff9640 1 -- 172.21.15.19:0/2459466491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7c7802c860 con 0x5639fecae360 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c86da1740 1 -- 172.21.15.19:0/2459466491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639fec1d420 con 0x5639fecae360 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c5cff9640 1 -- 172.21.15.19:0/2459466491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c78040050 con 0x5639fecae360 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c86da1740 10 client.6294 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c86da1740 1 -- 172.21.15.19:0/2459466491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5639fea51390 con 0x5639fecae360 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c5cff9640 1 -- 172.21.15.19:0/2459466491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639fec1d420 con 0x5639fecae360 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c5cff9640 1 -- 172.21.15.19:0/2459466491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7c78043020 con 0x5639fecae360 2024-04-25T09:56:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c86da1740 10 client.6294 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c86da1740 10 client.6294 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c86da1740 20 client.6294 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5639fed020d0 0x5639fecf5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c86da1740 4 client.6294 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.174+0000 7f7c86da1740 1 -- 172.21.15.19:0/2459466491 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639fec17770 con 0x5639fed020d0 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.175+0000 7f7c8155c640 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5639fed020d0 0x5639fecf5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.175+0000 7f7c5cff9640 10 client.6294 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.175+0000 7f7c8155c640 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5639fed020d0 0x5639fecf5130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f9188bf3911195 server_cookie=2370fd6710559c6d in_seq=0 out_seq=0 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.175+0000 7f7c5cff9640 1 -- 172.21.15.19:0/2459466491 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639fec17770 con 0x5639fed020d0 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.175+0000 7f7c5cff9640 10 client.6294 handle_command_reply: tid=0 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.175+0000 7f7c86da1740 1 client.6294 shutdown 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.175+0000 7f7c86da1740 1 -- 172.21.15.19:0/2459466491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 msgr2=0x5639fecd9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.175+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecd9b80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7c78014720 tx=0x7f7c78004730 comp rx=0 tx=0).stop 2024-04-25T09:56:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.176+0000 7f7c86da1740 1 -- 172.21.15.19:0/2459466491 shutdown_connections 2024-04-25T09:56:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.176+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5639fed020d0 0x5639fecf5130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.176+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecd7140 0x5639fecea920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.176+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639feccf360 0x5639fece99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.176+0000 7f7c5cff9640 0 client.6294 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.176+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2459466491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecd9b80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.176+0000 7f7c86da1740 1 -- 172.21.15.19:0/2459466491 >> 172.21.15.19:0/2459466491 conn(0x5639fec73810 msgr2=0x5639fecaced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.176+0000 7f7c86da1740 1 -- 172.21.15.19:0/2459466491 shutdown_connections 2024-04-25T09:56:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.176+0000 7f7c86da1740 1 -- 172.21.15.19:0/2459466491 wait complete. 2024-04-25T09:56:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.176+0000 7f7c86da1740 20 client.6294 trim_cache size 0 max 16384 2024-04-25T09:56:15.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.191+0000 7f7c86da1740 1 Processor -- start 2024-04-25T09:56:15.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.191+0000 7f7c86da1740 1 -- start start 2024-04-25T09:56:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecae360 0x5639fed0b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639feccf360 0x5639fecd8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecd7140 0x5639fecd8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c86da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c200d10 con 0x5639fecd7140 2024-04-25T09:56:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c86da1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c1fff20 con 0x5639feccf360 2024-04-25T09:56:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c86da1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c1ff420 con 0x5639fecae360 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c8255e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecd7140 0x5639fecd8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c8255e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecd7140 0x5639fecd8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34258/0 (socket says 172.21.15.19:34258) 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c8255e640 1 -- 172.21.15.19:0/2299322627 learned_addr learned my addr 172.21.15.19:0/2299322627 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c8255e640 1 -- 172.21.15.19:0/2299322627 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecae360 msgr2=0x5639fed0b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c8255e640 1 --2- 172.21.15.19:0/2299322627 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecae360 0x5639fed0b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c8255e640 1 -- 172.21.15.19:0/2299322627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639feccf360 msgr2=0x5639fecd8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c8255e640 1 --2- 172.21.15.19:0/2299322627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639feccf360 0x5639fecd8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c8255e640 1 -- 172.21.15.19:0/2299322627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639feceac90 con 0x5639fecd7140 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c8255e640 1 --2- 172.21.15.19:0/2299322627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecd7140 0x5639fecd8d60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7c740787d0 tx=0x7f7c74078430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962bba4248f6444e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c5effd640 1 -- 172.21.15.19:0/2299322627 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7407cb70 con 0x5639fecd7140 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c5effd640 1 -- 172.21.15.19:0/2299322627 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7402f040 con 0x5639fecd7140 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.192+0000 7f7c5effd640 1 -- 172.21.15.19:0/2299322627 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74081130 con 0x5639fecd7140 2024-04-25T09:56:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.193+0000 7f7c86da1740 1 -- 172.21.15.19:0/2299322627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecd7140 msgr2=0x5639fecd8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.193+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2299322627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecd7140 0x5639fecd8d60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7c740787d0 tx=0x7f7c74078430 comp rx=0 tx=0).stop 2024-04-25T09:56:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.193+0000 7f7c86da1740 1 -- 172.21.15.19:0/2299322627 shutdown_connections 2024-04-25T09:56:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.193+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2299322627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecd7140 0x5639fecd8d60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.193+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2299322627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639feccf360 0x5639fecd8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.193+0000 7f7c86da1740 1 --2- 172.21.15.19:0/2299322627 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639fecae360 0x5639fed0b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.193+0000 7f7c86da1740 1 -- 172.21.15.19:0/2299322627 >> 172.21.15.19:0/2299322627 conn(0x5639fec73810 msgr2=0x5639fecd39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.193+0000 7f7c86da1740 1 -- 172.21.15.19:0/2299322627 shutdown_connections 2024-04-25T09:56:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.193+0000 7f7c86da1740 1 -- 172.21.15.19:0/2299322627 wait complete. 2024-04-25T09:56:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.194+0000 7f7c86da1740 1 -- start start 2024-04-25T09:56:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.194+0000 7f7c86da1740 1 Processor -- start 2024-04-25T09:56:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.194+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecf24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.194+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639feccf360 0x5639fecf52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.194+0000 7f7c86da1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639fecd7140 0x5639fecf57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.194+0000 7f7c86da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fecd92a0 con 0x5639fecae360 2024-04-25T09:56:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.194+0000 7f7c86da1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c1ff920 con 0x5639fecd7140 2024-04-25T09:56:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.194+0000 7f7c86da1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639fecaca40 con 0x5639feccf360 2024-04-25T09:56:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c86da1740 10 client.0 _ll_register_callbacks cb 0x7f7c7801a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c81d5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecf24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c81d5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecf24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34274/0 (socket says 172.21.15.19:34274) 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c81d5d640 1 -- 172.21.15.19:0/296028994 learned_addr learned my addr 172.21.15.19:0/296028994 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c8155c640 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639feccf360 0x5639fecf52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c8255e640 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639fecd7140 0x5639fecf57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c81d5d640 1 -- 172.21.15.19:0/296028994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639feccf360 msgr2=0x5639fecf52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639feccf360 0x5639fecf52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c81d5d640 1 -- 172.21.15.19:0/296028994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639fecd7140 msgr2=0x5639fecf57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639fecd7140 0x5639fecf57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c81d5d640 1 -- 172.21.15.19:0/296028994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639fe8bd670 con 0x5639fecae360 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c8155c640 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639feccf360 0x5639fecf52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c81d5d640 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecf24e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7c78017fd0 tx=0x7f7c78009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f75f7a195d5010 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c5cff9640 1 -- 172.21.15.19:0/296028994 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78048030 con 0x5639fecae360 2024-04-25T09:56:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c5cff9640 1 -- 172.21.15.19:0/296028994 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7c78014d10 con 0x5639fecae360 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c86da1740 1 -- 172.21.15.19:0/296028994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c7402f040 con 0x5639fecae360 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c5cff9640 1 -- 172.21.15.19:0/296028994 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c78027040 con 0x5639fecae360 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c86da1740 10 client.6300 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.195+0000 7f7c86da1740 1 -- 172.21.15.19:0/296028994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5639fecac780 con 0x5639fecae360 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c5cff9640 1 -- 172.21.15.19:0/296028994 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7402f040 con 0x5639fecae360 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c5cff9640 1 -- 172.21.15.19:0/296028994 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7c78043040 con 0x5639fecae360 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c86da1740 10 client.6300 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c86da1740 10 client.6300 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c86da1740 20 client.6300 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c86da1740 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5639fecac310 0x5639fecd20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c86da1740 4 client.6300 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c86da1740 1 -- 172.21.15.19:0/296028994 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x5639fecd6f30 con 0x5639fecac310 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c8155c640 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5639fecac310 0x5639fecd20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c5cff9640 10 client.6300 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c8155c640 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5639fecac310 0x5639fecd20a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc69c13c14769f72 server_cookie=75284698f9c673a5 in_seq=0 out_seq=0 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c5cff9640 1 -- 172.21.15.19:0/296028994 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5639fecd6f30 con 0x5639fecac310 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c5cff9640 10 client.6300 handle_command_reply: tid=0 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.196+0000 7f7c86da1740 1 client.6300 shutdown 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c86da1740 1 -- 172.21.15.19:0/296028994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 msgr2=0x5639fecf24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c86da1740 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecf24e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7c78017fd0 tx=0x7f7c78009190 comp rx=0 tx=0).stop 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c86da1740 1 -- 172.21.15.19:0/296028994 shutdown_connections 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c86da1740 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5639fecac310 0x5639fecd20a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c86da1740 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5639fecd7140 0x5639fecf57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c86da1740 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5639feccf360 0x5639fecf52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c86da1740 1 --2- 172.21.15.19:0/296028994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fecae360 0x5639fecf24e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c5cff9640 0 client.6300 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c86da1740 1 -- 172.21.15.19:0/296028994 >> 172.21.15.19:0/296028994 conn(0x5639fec73810 msgr2=0x5639fecd39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c86da1740 1 -- 172.21.15.19:0/296028994 shutdown_connections 2024-04-25T09:56:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c86da1740 1 -- 172.21.15.19:0/296028994 wait complete. 2024-04-25T09:56:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c86da1740 20 client.6300 trim_cache size 0 max 16384 2024-04-25T09:56:15.207 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f6cd2016640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096c [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v473 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5892=0-4194304@1} caps={5892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5892 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x561ba4f4d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1000000096c:[<0x10000000001/new_file_1 v473>,<0x10000000000/ceph-qa-suite v55>,<0x1/scrub_test_path v20>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T09:56:15.207 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:15.207 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:56:15.207 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "27c9e491-fa30-4a63-9b45-90142ba16b02", 2024-04-25T09:56:15.207 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:56:15.208 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 -- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c6006b4d0 msgr2=0x7f7c6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c6006b4d0 0x7f7c6006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009b40 tx=0x7f7c78008040 comp rx=0 tx=0).stop 2024-04-25T09:56:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 -- 172.21.15.19:0/964756818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c7c159280 msgr2=0x7f7c7c159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c7c159280 0x7f7c7c159700 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7c7402b500 tx=0x7f7c7c0ce700 comp rx=0 tx=0).stop 2024-04-25T09:56:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 -- 172.21.15.19:0/964756818 shutdown_connections 2024-04-25T09:56:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7c6006b4d0 0x7f7c6006d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c7c159280 0x7f7c7c159700 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c7c0c8e90 0x7f7c7c14bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 --2- 172.21.15.19:0/964756818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c680051c0 0x7f7c7c1f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 -- 172.21.15.19:0/964756818 >> 172.21.15.19:0/964756818 conn(0x7f7c7c0d0800 msgr2=0x7f7c7c15e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 -- 172.21.15.19:0/964756818 shutdown_connections 2024-04-25T09:56:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.197+0000 7f7c83fc1640 1 -- 172.21.15.19:0/964756818 wait complete. 2024-04-25T09:56:15.212 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '27c9e491-fa30-4a63-9b45-90142ba16b02', 'mode': 'asynchronous'}' 2024-04-25T09:56:15.212 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:56:15.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:56:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.311+0000 7fb12928f640 1 Processor -- start 2024-04-25T09:56:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.311+0000 7fb12928f640 1 -- start start 2024-04-25T09:56:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.311+0000 7fb12928f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb1240ced80 0x7fb1240d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.311+0000 7fb12928f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12415e000 con 0x7fb12415dc20 2024-04-25T09:56:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.311+0000 7fb12928f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1240d1de0 con 0x7fb124161710 2024-04-25T09:56:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.311+0000 7fb12928f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb124161c60 con 0x7fb1240ced80 2024-04-25T09:56:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.312+0000 7fb122ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb12415dc20 0x7fb1240ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41516/0 (socket says 172.21.15.19:41516) 2024-04-25T09:56:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.312+0000 7fb122ffd640 1 -- 172.21.15.19:0/1204723600 learned_addr learned my addr 172.21.15.19:0/1204723600 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.312+0000 7fb1227fc640 1 --2- 172.21.15.19:0/1204723600 >> v2:172.21.15.112:3300/0 conn(0x7fb1240ced80 0x7fb1240d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.312+0000 7fb121ffb640 1 -- 172.21.15.19:0/1204723600 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149485950 0 0) 0x7fb12415e000 con 0x7fb12415dc20 2024-04-25T09:56:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.312+0000 7fb121ffb640 1 -- 172.21.15.19:0/1204723600 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb10c003520 con 0x7fb12415dc20 2024-04-25T09:56:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.312+0000 7fb121ffb640 1 -- 172.21.15.19:0/1204723600 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970781644 0 0) 0x7fb1240d1de0 con 0x7fb124161710 2024-04-25T09:56:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.312+0000 7fb121ffb640 1 -- 172.21.15.19:0/1204723600 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb12415e000 con 0x7fb124161710 2024-04-25T09:56:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.312+0000 7fb1227fc640 1 -- 172.21.15.19:0/1204723600 >> v1:172.21.15.112:6789/0 conn(0x7fb124161710 legacy=0x7fb1240d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.312+0000 7fb1227fc640 1 -- 172.21.15.19:0/1204723600 >> v1:172.21.15.19:6789/0 conn(0x7fb12415dc20 legacy=0x7fb1240ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.312+0000 7fb1227fc640 1 -- 172.21.15.19:0/1204723600 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124161f40 con 0x7fb1240ced80 2024-04-25T09:56:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.313+0000 7fb1227fc640 1 --2- 172.21.15.19:0/1204723600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1240ced80 0x7fb1240d22e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb1180097a0 tx=0x7fb11802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79725bf2201d706b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.313+0000 7fb121ffb640 1 -- 172.21.15.19:0/1204723600 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118037070 con 0x7fb1240ced80 2024-04-25T09:56:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.313+0000 7fb121ffb640 1 -- 172.21.15.19:0/1204723600 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb118032440 con 0x7fb1240ced80 2024-04-25T09:56:15.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.313+0000 7fb121ffb640 1 -- 172.21.15.19:0/1204723600 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118032810 con 0x7fb1240ced80 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.314+0000 7fb12928f640 1 -- 172.21.15.19:0/1204723600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1240ced80 msgr2=0x7fb1240d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.314+0000 7fb12928f640 1 --2- 172.21.15.19:0/1204723600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1240ced80 0x7fb1240d22e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb1180097a0 tx=0x7fb11802a230 comp rx=0 tx=0).stop 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.314+0000 7fb12928f640 1 -- 172.21.15.19:0/1204723600 shutdown_connections 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.314+0000 7fb12928f640 1 --2- 172.21.15.19:0/1204723600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1240ced80 0x7fb1240d22e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.314+0000 7fb12928f640 1 -- 172.21.15.19:0/1204723600 >> 172.21.15.19:0/1204723600 conn(0x7fb1240d0420 msgr2=0x7fb1240d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.314+0000 7fb12928f640 1 -- 172.21.15.19:0/1204723600 shutdown_connections 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.314+0000 7fb12928f640 1 -- 172.21.15.19:0/1204723600 wait complete. 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.315+0000 7fb12928f640 1 Processor -- start 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.315+0000 7fb12928f640 1 -- start start 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb12928f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1240ced80 0x7fb1241f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb12928f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb12415dc20 0x7fb124156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb12928f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb124161710 0x7fb124156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb12928f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240b5e50 con 0x7fb1240ced80 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb12928f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb124162500 con 0x7fb12415dc20 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb12928f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240b5fd0 con 0x7fb124161710 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb122ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1240ced80 0x7fb1241f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb122ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1240ced80 0x7fb1241f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34276/0 (socket says 172.21.15.19:34276) 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb122ffd640 1 -- 172.21.15.19:0/811549353 learned_addr learned my addr 172.21.15.19:0/811549353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb1227fc640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb12415dc20 0x7fb124156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb1237fe640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb124161710 0x7fb124156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb122ffd640 1 -- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb124161710 msgr2=0x7fb124156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb122ffd640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb124161710 0x7fb124156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb122ffd640 1 -- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb12415dc20 msgr2=0x7fb124156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb122ffd640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb12415dc20 0x7fb124156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb122ffd640 1 -- 172.21.15.19:0/811549353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12415ff50 con 0x7fb1240ced80 2024-04-25T09:56:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb1227fc640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb12415dc20 0x7fb124156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.316+0000 7fb122ffd640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1240ced80 0x7fb1241f6aa0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c500 tx=0x7fb11400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.317+0000 7fb10b7fe640 1 -- 172.21.15.19:0/811549353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114017070 con 0x7fb1240ced80 2024-04-25T09:56:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.317+0000 7fb10b7fe640 1 -- 172.21.15.19:0/811549353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb114005ce0 con 0x7fb1240ced80 2024-04-25T09:56:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.317+0000 7fb12928f640 1 -- 172.21.15.19:0/811549353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb124160230 con 0x7fb1240ced80 2024-04-25T09:56:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.317+0000 7fb10b7fe640 1 -- 172.21.15.19:0/811549353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1140072f0 con 0x7fb1240ced80 2024-04-25T09:56:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.317+0000 7fb12928f640 1 -- 172.21.15.19:0/811549353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1241607f0 con 0x7fb1240ced80 2024-04-25T09:56:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.318+0000 7fb10b7fe640 1 -- 172.21.15.19:0/811549353 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb11401a070 con 0x7fb1240ced80 2024-04-25T09:56:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.318+0000 7fb10b7fe640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb11006b4d0 0x7fb11006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.319+0000 7fb10b7fe640 1 -- 172.21.15.19:0/811549353 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb114089840 con 0x7fb1240ced80 2024-04-25T09:56:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.319+0000 7fb1227fc640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb11006b4d0 0x7fb11006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.319+0000 7fb1227fc640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb11006b4d0 0x7fb11006d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb118034000 tx=0x7fb11802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.320+0000 7fb12bf8d740 1 Processor -- start 2024-04-25T09:56:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.320+0000 7fb12bf8d740 1 -- start start 2024-04-25T09:56:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.320+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649b10 0x5599c864c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.320+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649ef0 0x5599c864c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.320+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c861c770 0x5599c861ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c8585da0 con 0x5599c861c770 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c852eb80 con 0x5599c8649b10 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599c8581e40 con 0x5599c8649ef0 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb1237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c861c770 0x5599c861ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb1237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c861c770 0x5599c861ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34288/0 (socket says 172.21.15.19:34288) 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb1237fe640 1 -- 172.21.15.19:0/2501336226 learned_addr learned my addr 172.21.15.19:0/2501336226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb122ffd640 1 --2- 172.21.15.19:0/2501336226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649b10 0x5599c864c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb1227fc640 1 --2- 172.21.15.19:0/2501336226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649ef0 0x5599c864c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb1237fe640 1 -- 172.21.15.19:0/2501336226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649ef0 msgr2=0x5599c864c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb1237fe640 1 --2- 172.21.15.19:0/2501336226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649ef0 0x5599c864c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb1237fe640 1 -- 172.21.15.19:0/2501336226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649b10 msgr2=0x5599c864c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb1237fe640 1 --2- 172.21.15.19:0/2501336226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649b10 0x5599c864c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb1237fe640 1 -- 172.21.15.19:0/2501336226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c8538f10 con 0x5599c861c770 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb1237fe640 1 --2- 172.21.15.19:0/2501336226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c861c770 0x5599c861ff80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb11c001c80 tx=0x7fb11c002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fc22400c695f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb108ff9640 1 -- 172.21.15.19:0/2501336226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c019070 con 0x5599c861c770 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb108ff9640 1 -- 172.21.15.19:0/2501336226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c003b80 con 0x5599c861c770 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.321+0000 7fb108ff9640 1 -- 172.21.15.19:0/2501336226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c00e860 con 0x5599c861c770 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.322+0000 7fb12bf8d740 1 -- 172.21.15.19:0/2501336226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c861c770 msgr2=0x5599c861ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.322+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/2501336226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c861c770 0x5599c861ff80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb11c001c80 tx=0x7fb11c002140 comp rx=0 tx=0).stop 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.322+0000 7fb12bf8d740 1 -- 172.21.15.19:0/2501336226 shutdown_connections 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.322+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/2501336226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c861c770 0x5599c861ff80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.322+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/2501336226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649ef0 0x5599c864c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.322+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/2501336226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649b10 0x5599c864c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.322+0000 7fb12bf8d740 1 -- 172.21.15.19:0/2501336226 >> 172.21.15.19:0/2501336226 conn(0x5599c861e9e0 msgr2=0x5599c8667320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.322+0000 7fb12bf8d740 1 -- 172.21.15.19:0/2501336226 shutdown_connections 2024-04-25T09:56:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.322+0000 7fb12bf8d740 1 -- 172.21.15.19:0/2501336226 wait complete. 2024-04-25T09:56:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.323+0000 7fb12bf8d740 1 -- start start 2024-04-25T09:56:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.323+0000 7fb12bf8d740 1 Processor -- start 2024-04-25T09:56:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.323+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649b10 0x5599c8645660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.323+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649ef0 0x5599c866a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.323+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8641050 0x5599c8643520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.323+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c8621b90 con 0x5599c8641050 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.323+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c85244b0 con 0x5599c8649b10 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.323+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599c8621e70 con 0x5599c8649ef0 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.323+0000 7fb12bf8d740 10 client.0 _ll_register_callbacks cb 0x5599c8564180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb1237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8641050 0x5599c8643520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb1237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8641050 0x5599c8643520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34290/0 (socket says 172.21.15.19:34290) 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb1237fe640 1 -- 172.21.15.19:0/926008943 learned_addr learned my addr 172.21.15.19:0/926008943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb122ffd640 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649b10 0x5599c8645660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb1227fc640 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649ef0 0x5599c866a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb1237fe640 1 -- 172.21.15.19:0/926008943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649ef0 msgr2=0x5599c866a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb1237fe640 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649ef0 0x5599c866a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb1237fe640 1 -- 172.21.15.19:0/926008943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649b10 msgr2=0x5599c8645660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb1237fe640 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649b10 0x5599c8645660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb1237fe640 1 -- 172.21.15.19:0/926008943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599c83647d0 con 0x5599c8641050 2024-04-25T09:56:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb1227fc640 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649ef0 0x5599c866a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb1237fe640 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8641050 0x5599c8643520 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb11c001580 tx=0x7fb11c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470e0cf03aafdcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb101ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb101ffb640 1 -- 172.21.15.19:0/926008943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c019040 con 0x5599c8641050 2024-04-25T09:56:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb101ffb640 1 -- 172.21.15.19:0/926008943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb11c0050d0 con 0x5599c8641050 2024-04-25T09:56:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb12bf8d740 1 -- 172.21.15.19:0/926008943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb11c003b80 con 0x5599c8641050 2024-04-25T09:56:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb101ffb640 1 -- 172.21.15.19:0/926008943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb11c02a050 con 0x5599c8641050 2024-04-25T09:56:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb12bf8d740 10 client.6312 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.324+0000 7fb12bf8d740 1 -- 172.21.15.19:0/926008943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5599c829a3f0 con 0x5599c8641050 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.325+0000 7fb101ffb640 1 -- 172.21.15.19:0/926008943 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c003b80 con 0x5599c8641050 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.325+0000 7fb101ffb640 1 -- 172.21.15.19:0/926008943 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb11c02d020 con 0x5599c8641050 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.325+0000 7fb12bf8d740 10 client.6312 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.325+0000 7fb12bf8d740 10 client.6312 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.325+0000 7fb12bf8d740 20 client.6312 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.325+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5599c8643a60 0x5599c8684560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.325+0000 7fb12bf8d740 4 client.6312 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.325+0000 7fb12bf8d740 1 -- 172.21.15.19:0/926008943 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599c85932e0 con 0x5599c8643a60 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.325+0000 7fb122ffd640 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5599c8643a60 0x5599c8684560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.325+0000 7fb122ffd640 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5599c8643a60 0x5599c8684560 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c4b890f54f8eba8 server_cookie=7cb24dd0c95e3991 in_seq=0 out_seq=0 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.325+0000 7fb101ffb640 10 client.6312 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb101ffb640 1 -- 172.21.15.19:0/926008943 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599c85932e0 con 0x5599c8643a60 2024-04-25T09:56:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb101ffb640 10 client.6312 handle_command_reply: tid=0 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 1 client.6312 shutdown 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 1 -- 172.21.15.19:0/926008943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8641050 msgr2=0x5599c8643520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8641050 0x5599c8643520 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb11c001580 tx=0x7fb11c001250 comp rx=0 tx=0).stop 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 1 -- 172.21.15.19:0/926008943 shutdown_connections 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5599c8643a60 0x5599c8684560 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8641050 0x5599c8643520 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649ef0 0x5599c866a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/926008943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649b10 0x5599c8645660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb101ffb640 0 client.6312 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 1 -- 172.21.15.19:0/926008943 >> 172.21.15.19:0/926008943 conn(0x5599c861c770 msgr2=0x5599c8648f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 1 -- 172.21.15.19:0/926008943 shutdown_connections 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 1 -- 172.21.15.19:0/926008943 wait complete. 2024-04-25T09:56:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.326+0000 7fb12bf8d740 20 client.6312 trim_cache size 0 max 16384 2024-04-25T09:56:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.341+0000 7fb12bf8d740 1 Processor -- start 2024-04-25T09:56:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.341+0000 7fb12bf8d740 1 -- start start 2024-04-25T09:56:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8641050 0x5599c861b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649b10 0x5599c861bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649ef0 0x5599c8688780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb124200280 con 0x5599c8649b10 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1241ff570 con 0x5599c8649ef0 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb124146d70 con 0x5599c8641050 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb1227fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649b10 0x5599c861bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb1227fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649b10 0x5599c861bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34300/0 (socket says 172.21.15.19:34300) 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb1227fc640 1 -- 172.21.15.19:0/1680988277 learned_addr learned my addr 172.21.15.19:0/1680988277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb1237fe640 1 --2- 172.21.15.19:0/1680988277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649ef0 0x5599c8688780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb122ffd640 1 --2- 172.21.15.19:0/1680988277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8641050 0x5599c861b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb1227fc640 1 -- 172.21.15.19:0/1680988277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8641050 msgr2=0x5599c861b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb1227fc640 1 --2- 172.21.15.19:0/1680988277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8641050 0x5599c861b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb1227fc640 1 -- 172.21.15.19:0/1680988277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649ef0 msgr2=0x5599c8688780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb1227fc640 1 --2- 172.21.15.19:0/1680988277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649ef0 0x5599c8688780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb1227fc640 1 -- 172.21.15.19:0/1680988277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c825a0f0 con 0x5599c8649b10 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb1227fc640 1 --2- 172.21.15.19:0/1680988277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649b10 0x5599c861bb40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb1180098c0 tx=0x7fb118008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4d6cc3c10cdf1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb103fff640 1 -- 172.21.15.19:0/1680988277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118037040 con 0x5599c8649b10 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb103fff640 1 -- 172.21.15.19:0/1680988277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11804e070 con 0x5599c8649b10 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.342+0000 7fb103fff640 1 -- 172.21.15.19:0/1680988277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118049860 con 0x5599c8649b10 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.343+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1680988277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649b10 msgr2=0x5599c861bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.343+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/1680988277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649b10 0x5599c861bb40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb1180098c0 tx=0x7fb118008030 comp rx=0 tx=0).stop 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.343+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1680988277 shutdown_connections 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.343+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/1680988277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8649ef0 0x5599c8688780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.343+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/1680988277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649b10 0x5599c861bb40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.343+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/1680988277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8641050 0x5599c861b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.343+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1680988277 >> 172.21.15.19:0/1680988277 conn(0x5599c861c770 msgr2=0x5599c8648f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.343+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1680988277 shutdown_connections 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.343+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1680988277 wait complete. 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb12bf8d740 1 -- start start 2024-04-25T09:56:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb12bf8d740 1 Processor -- start 2024-04-25T09:56:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8641050 0x5599c8688580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649b10 0x5599c86d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb12bf8d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649ef0 0x5599c8688d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c8640ed0 con 0x5599c8649ef0 2024-04-25T09:56:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb124147370 con 0x5599c8641050 2024-04-25T09:56:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb12bf8d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599c8661ef0 con 0x5599c8649b10 2024-04-25T09:56:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb12bf8d740 10 client.0 _ll_register_callbacks cb 0x7fb124062560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb1237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649ef0 0x5599c8688d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb1237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649ef0 0x5599c8688d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34314/0 (socket says 172.21.15.19:34314) 2024-04-25T09:56:15.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.344+0000 7fb1237fe640 1 -- 172.21.15.19:0/1869668322 learned_addr learned my addr 172.21.15.19:0/1869668322 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb1227fc640 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649b10 0x5599c86d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb122ffd640 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8641050 0x5599c8688580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb1237fe640 1 -- 172.21.15.19:0/1869668322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649b10 msgr2=0x5599c86d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb1237fe640 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649b10 0x5599c86d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb1237fe640 1 -- 172.21.15.19:0/1869668322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8641050 msgr2=0x5599c8688580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb1237fe640 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8641050 0x5599c8688580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb1237fe640 1 -- 172.21.15.19:0/1869668322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599c864f200 con 0x5599c8649ef0 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb1227fc640 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649b10 0x5599c86d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb122ffd640 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8641050 0x5599c8688580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb1237fe640 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649ef0 0x5599c8688d80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5599c8650700 tx=0x7fb11c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b120021498546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb101ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb101ffb640 1 -- 172.21.15.19:0/1869668322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c019040 con 0x5599c8649ef0 2024-04-25T09:56:15.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb101ffb640 1 -- 172.21.15.19:0/1869668322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb11c005430 con 0x5599c8649ef0 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1869668322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb11804e070 con 0x5599c8649ef0 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb101ffb640 1 -- 172.21.15.19:0/1869668322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb11c0012f0 con 0x5599c8649ef0 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb12bf8d740 10 client.6318 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1869668322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5599c864fb00 con 0x5599c8649ef0 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.345+0000 7fb101ffb640 1 -- 172.21.15.19:0/1869668322 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11804e070 con 0x5599c8649ef0 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb101ffb640 1 -- 172.21.15.19:0/1869668322 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb11c02d040 con 0x5599c8649ef0 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 10 client.6318 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 10 client.6318 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 20 client.6318 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5599c86210c0 0x5599c8642e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 4 client.6318 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1869668322 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599c858d630 con 0x5599c86210c0 2024-04-25T09:56:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb122ffd640 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5599c86210c0 0x5599c8642e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb122ffd640 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5599c86210c0 0x5599c8642e80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bef2f8cd87a3a3f server_cookie=8fd73bf0b7e5c41b in_seq=0 out_seq=0 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb101ffb640 10 client.6318 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb101ffb640 1 -- 172.21.15.19:0/1869668322 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+903 (crc 0 0 0) 0x5599c858d630 con 0x5599c86210c0 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb101ffb640 10 client.6318 handle_command_reply: tid=0 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 client.6318 shutdown 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1869668322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649ef0 msgr2=0x5599c8688d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649ef0 0x5599c8688d80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5599c8650700 tx=0x7fb11c009040 comp rx=0 tx=0).stop 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1869668322 shutdown_connections 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x5599c86210c0 0x5599c8642e80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599c8649ef0 0x5599c8688d80 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5599c8649b10 0x5599c86d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb101ffb640 0 client.6318 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 --2- 172.21.15.19:0/1869668322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5599c8641050 0x5599c8688580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1869668322 >> 172.21.15.19:0/1869668322 conn(0x5599c861c770 msgr2=0x5599c861b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1869668322 shutdown_connections 2024-04-25T09:56:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 1 -- 172.21.15.19:0/1869668322 wait complete. 2024-04-25T09:56:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.346+0000 7fb12bf8d740 20 client.6318 trim_cache size 0 max 16384 2024-04-25T09:56:15.357 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:56:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T09:56:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "115486fb-60a0-4ed0-bedf-d0f5e261b812": { 2024-04-25T09:56:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T09:56:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "115486fb-60a0-4ed0-bedf-d0f5e261b812", 2024-04-25T09:56:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "" 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "27c9e491-fa30-4a63-9b45-90142ba16b02": { 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "27c9e491-fa30-4a63-9b45-90142ba16b02", 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "" 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "ac61cca1-8b6f-41a7-847d-6a78f340c5e7": { 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "ac61cca1-8b6f-41a7-847d-6a78f340c5e7", 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "" 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "f76ee0e9-fc22-4442-ba02-f1a36e468b9c": { 2024-04-25T09:56:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-25T09:56:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "f76ee0e9-fc22-4442-ba02-f1a36e468b9c", 2024-04-25T09:56:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "" 2024-04-25T09:56:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:15.359 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:15.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 -- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb11006b4d0 msgr2=0x7fb11006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb11006b4d0 0x7fb11006d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb118034000 tx=0x7fb11802e000 comp rx=0 tx=0).stop 2024-04-25T09:56:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 -- 172.21.15.19:0/811549353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1240ced80 msgr2=0x7fb1241f6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1240ced80 0x7fb1241f6aa0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c500 tx=0x7fb11400c9c0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 -- 172.21.15.19:0/811549353 shutdown_connections 2024-04-25T09:56:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb11006b4d0 0x7fb11006d990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb124161710 0x7fb124156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb12415dc20 0x7fb124156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 --2- 172.21.15.19:0/811549353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1240ced80 0x7fb1241f6aa0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 -- 172.21.15.19:0/811549353 >> 172.21.15.19:0/811549353 conn(0x7fb1240d0420 msgr2=0x7fb1240b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 -- 172.21.15.19:0/811549353 shutdown_connections 2024-04-25T09:56:15.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:15.347+0000 7fb12928f640 1 -- 172.21.15.19:0/811549353 wait complete. 2024-04-25T09:56:20.362 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:56:20.362 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:56:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.475+0000 7f1086c6f640 1 Processor -- start 2024-04-25T09:56:20.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.475+0000 7f1086c6f640 1 -- start start 2024-04-25T09:56:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f1086c6f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f10801454a0 0x7f10800aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f1086c6f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f108006b2f0 0x7f108015d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f1086c6f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10800af540 con 0x7f108015c970 2024-04-25T09:56:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f1086c6f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10800af820 con 0x7f108006b2f0 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f1086c6f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10800afb00 con 0x7f10801454a0 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f108520c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f108006b2f0 0x7f108015d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f108520c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f108006b2f0 0x7f108015d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43602/0 (socket says 172.21.15.19:43602) 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f108520c640 1 -- 172.21.15.19:0/48585628 learned_addr learned my addr 172.21.15.19:0/48585628 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f1084a0b640 1 --2- 172.21.15.19:0/48585628 >> v2:172.21.15.112:3300/0 conn(0x7f10801454a0 0x7f10800aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f108520c640 1 -- 172.21.15.19:0/48585628 >> v2:172.21.15.112:3300/0 conn(0x7f10801454a0 msgr2=0x7f10800aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f108520c640 1 --2- 172.21.15.19:0/48585628 >> v2:172.21.15.112:3300/0 conn(0x7f10801454a0 0x7f10800aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f108520c640 1 -- 172.21.15.19:0/48585628 >> v1:172.21.15.112:6789/0 conn(0x7f108015c970 legacy=0x7f108006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f108520c640 1 -- 172.21.15.19:0/48585628 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10800afde0 con 0x7f108006b2f0 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.476+0000 7f1084a0b640 1 --2- 172.21.15.19:0/48585628 >> v2:172.21.15.112:3300/0 conn(0x7f10801454a0 0x7f10800aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.477+0000 7f108520c640 1 --2- 172.21.15.19:0/48585628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f108006b2f0 0x7f108015d490 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f10800b3320 tx=0x7f107802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76a9b7dd265d9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.477+0000 7f107f7fe640 1 -- 172.21.15.19:0/48585628 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078036070 con 0x7f108006b2f0 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.477+0000 7f107f7fe640 1 -- 172.21.15.19:0/48585628 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1078031600 con 0x7f108006b2f0 2024-04-25T09:56:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.477+0000 7f107f7fe640 1 -- 172.21.15.19:0/48585628 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10780319d0 con 0x7f108006b2f0 2024-04-25T09:56:20.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.478+0000 7f1086c6f640 1 -- 172.21.15.19:0/48585628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f108006b2f0 msgr2=0x7f108015d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.478+0000 7f1086c6f640 1 --2- 172.21.15.19:0/48585628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f108006b2f0 0x7f108015d490 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f10800b3320 tx=0x7f107802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.478+0000 7f1086c6f640 1 -- 172.21.15.19:0/48585628 shutdown_connections 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.478+0000 7f1086c6f640 1 --2- 172.21.15.19:0/48585628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f108006b2f0 0x7f108015d490 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.478+0000 7f1086c6f640 1 --2- 172.21.15.19:0/48585628 >> v2:172.21.15.112:3300/0 conn(0x7f10801454a0 0x7f10800aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.478+0000 7f1086c6f640 1 -- 172.21.15.19:0/48585628 >> 172.21.15.19:0/48585628 conn(0x7f10800b5af0 msgr2=0x7f10800b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.478+0000 7f1086c6f640 1 -- 172.21.15.19:0/48585628 shutdown_connections 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.478+0000 7f1086c6f640 1 -- 172.21.15.19:0/48585628 wait complete. 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.479+0000 7f1086c6f640 1 Processor -- start 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.479+0000 7f1086c6f640 1 -- start start 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.480+0000 7f1086c6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f108006b2f0 0x7f10801e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.480+0000 7f1086c6f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10801454a0 0x7f1080147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.480+0000 7f1086c6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f108015c970 0x7f10801483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.480+0000 7f1086c6f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108015db50 con 0x7f10801454a0 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.480+0000 7f1086c6f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108015d9d0 con 0x7f108015c970 2024-04-25T09:56:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.480+0000 7f1086c6f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108015dcd0 con 0x7f108006b2f0 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.480+0000 7f107ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10801454a0 0x7f1080147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.480+0000 7f107ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10801454a0 0x7f1080147ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43610/0 (socket says 172.21.15.19:43610) 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.480+0000 7f107ffff640 1 -- 172.21.15.19:0/1690777197 learned_addr learned my addr 172.21.15.19:0/1690777197 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.480+0000 7f1084a0b640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f108006b2f0 0x7f10801e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.480+0000 7f108520c640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f108015c970 0x7f10801483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.481+0000 7f107ffff640 1 -- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f108006b2f0 msgr2=0x7f10801e74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.481+0000 7f107ffff640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f108006b2f0 0x7f10801e74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.481+0000 7f107ffff640 1 -- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f108015c970 msgr2=0x7f10801483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.481+0000 7f107ffff640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f108015c970 0x7f10801483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.481+0000 7f107ffff640 1 -- 172.21.15.19:0/1690777197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10801e6410 con 0x7f10801454a0 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.481+0000 7f107ffff640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10801454a0 0x7f1080147ea0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f106c00c390 tx=0x7f106c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.481+0000 7f107d7fa640 1 -- 172.21.15.19:0/1690777197 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c011790 con 0x7f10801454a0 2024-04-25T09:56:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.481+0000 7f107d7fa640 1 -- 172.21.15.19:0/1690777197 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c017070 con 0x7f10801454a0 2024-04-25T09:56:20.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.481+0000 7f1086c6f640 1 -- 172.21.15.19:0/1690777197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10801e66f0 con 0x7f10801454a0 2024-04-25T09:56:20.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.481+0000 7f107d7fa640 1 -- 172.21.15.19:0/1690777197 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c011b70 con 0x7f10801454a0 2024-04-25T09:56:20.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.481+0000 7f1086c6f640 1 -- 172.21.15.19:0/1690777197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10801e6cb0 con 0x7f10801454a0 2024-04-25T09:56:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.483+0000 7f107d7fa640 1 -- 172.21.15.19:0/1690777197 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f106c011d10 con 0x7f10801454a0 2024-04-25T09:56:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.483+0000 7f107d7fa640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.483+0000 7f107d7fa640 1 -- 172.21.15.19:0/1690777197 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f106c08a040 con 0x7f10801454a0 2024-04-25T09:56:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.484+0000 7f1084a0b640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.484+0000 7f1089973740 1 Processor -- start 2024-04-25T09:56:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.484+0000 7f1084a0b640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f106406b4d0 0x7f106406d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1074006fd0 tx=0x7f1074008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.484+0000 7f1089973740 1 -- start start 2024-04-25T09:56:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f1089973740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f795510 0x55d03f7958f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f1089973740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f765770 0x55d03f76b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f1089973740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f1089973740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03f6ceda0 con 0x55d03f7b1470 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f1089973740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03f677b80 con 0x55d03f795510 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f1089973740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d03f6cae40 con 0x55d03f765770 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f108520c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f108520c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43616/0 (socket says 172.21.15.19:43616) 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f108520c640 1 -- 172.21.15.19:0/274547744 learned_addr learned my addr 172.21.15.19:0/274547744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f1084a0b640 1 --2- 172.21.15.19:0/274547744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f795510 0x55d03f7958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f107ffff640 1 --2- 172.21.15.19:0/274547744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f765770 0x55d03f76b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f108520c640 1 -- 172.21.15.19:0/274547744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f765770 msgr2=0x55d03f76b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f108520c640 1 --2- 172.21.15.19:0/274547744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f765770 0x55d03f76b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f108520c640 1 -- 172.21.15.19:0/274547744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f795510 msgr2=0x55d03f7958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f108520c640 1 --2- 172.21.15.19:0/274547744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f795510 0x55d03f7958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f108520c640 1 -- 172.21.15.19:0/274547744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d03f681f10 con 0x55d03f7b1470 2024-04-25T09:56:20.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.485+0000 7f108520c640 1 --2- 172.21.15.19:0/274547744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7b18f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f107802ace0 tx=0x7f107802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3680f4703e5cf53a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f106a7fc640 1 -- 172.21.15.19:0/274547744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078042070 con 0x55d03f7b1470 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f106a7fc640 1 -- 172.21.15.19:0/274547744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1078037040 con 0x55d03f7b1470 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f106a7fc640 1 -- 172.21.15.19:0/274547744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078036040 con 0x55d03f7b1470 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f1089973740 1 -- 172.21.15.19:0/274547744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 msgr2=0x55d03f7b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f1089973740 1 --2- 172.21.15.19:0/274547744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7b18f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f107802ace0 tx=0x7f107802e000 comp rx=0 tx=0).stop 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f1089973740 1 -- 172.21.15.19:0/274547744 shutdown_connections 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f1089973740 1 --2- 172.21.15.19:0/274547744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7b18f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f1089973740 1 --2- 172.21.15.19:0/274547744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f765770 0x55d03f76b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f1089973740 1 --2- 172.21.15.19:0/274547744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f795510 0x55d03f7958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f1089973740 1 -- 172.21.15.19:0/274547744 >> 172.21.15.19:0/274547744 conn(0x55d03f7679e0 msgr2=0x55d03f7b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f1089973740 1 -- 172.21.15.19:0/274547744 shutdown_connections 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.486+0000 7f1089973740 1 -- 172.21.15.19:0/274547744 wait complete. 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.487+0000 7f1089973740 1 -- start start 2024-04-25T09:56:20.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.487+0000 7f1089973740 1 Processor -- start 2024-04-25T09:56:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f1089973740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f795510 0x55d03f7b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f1089973740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f7b1470 0x55d03f7b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f1089973740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f781e00 0x55d03f7b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f1089973740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03f76bce0 con 0x55d03f781e00 2024-04-25T09:56:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f1089973740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03f66d4b0 con 0x55d03f7b1470 2024-04-25T09:56:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f1089973740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d03f76b310 con 0x55d03f795510 2024-04-25T09:56:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f1089973740 10 client.0 _ll_register_callbacks cb 0x55d03f6ad180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f108520c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f781e00 0x55d03f7b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f108520c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f781e00 0x55d03f7b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43622/0 (socket says 172.21.15.19:43622) 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f108520c640 1 -- 172.21.15.19:0/1637652993 learned_addr learned my addr 172.21.15.19:0/1637652993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f107ffff640 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f7b1470 0x55d03f7b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f1084a0b640 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f795510 0x55d03f7b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f108520c640 1 -- 172.21.15.19:0/1637652993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f795510 msgr2=0x55d03f7b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f108520c640 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f795510 0x55d03f7b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f108520c640 1 -- 172.21.15.19:0/1637652993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f7b1470 msgr2=0x55d03f7b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f108520c640 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f7b1470 0x55d03f7b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f108520c640 1 -- 172.21.15.19:0/1637652993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d03f4ad7d0 con 0x55d03f781e00 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f108520c640 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f781e00 0x55d03f7b1870 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1078009f10 tx=0x7f1078004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b946c2e0c59cec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.488+0000 7f105f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f105f7fe640 1 -- 172.21.15.19:0/1637652993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078036050 con 0x55d03f781e00 2024-04-25T09:56:20.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f105f7fe640 1 -- 172.21.15.19:0/1637652993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1078004040 con 0x55d03f781e00 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f1089973740 1 -- 172.21.15.19:0/1637652993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1078037040 con 0x55d03f781e00 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f105f7fe640 1 -- 172.21.15.19:0/1637652993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1078052050 con 0x55d03f781e00 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f1089973740 10 client.6330 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f1089973740 1 -- 172.21.15.19:0/1637652993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55d03f3e33f0 con 0x55d03f781e00 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f105f7fe640 1 -- 172.21.15.19:0/1637652993 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1078037040 con 0x55d03f781e00 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f105f7fe640 1 -- 172.21.15.19:0/1637652993 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1078055020 con 0x55d03f781e00 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f1089973740 10 client.6330 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f1089973740 10 client.6330 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f1089973740 20 client.6330 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f1089973740 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d03f7b1db0 0x55d03f7b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f1089973740 4 client.6330 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f1089973740 1 -- 172.21.15.19:0/1637652993 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d03f6dc2e0 con 0x55d03f7b1db0 2024-04-25T09:56:20.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.489+0000 7f1084a0b640 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d03f7b1db0 0x55d03f7b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f105f7fe640 10 client.6330 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f1084a0b640 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d03f7b1db0 0x55d03f7b2190 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=171351e60e7eb7ce server_cookie=91d596bbf3e2277a in_seq=0 out_seq=0 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f105f7fe640 1 -- 172.21.15.19:0/1637652993 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d03f6dc2e0 con 0x55d03f7b1db0 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f105f7fe640 10 client.6330 handle_command_reply: tid=0 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f1089973740 1 client.6330 shutdown 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f1089973740 1 -- 172.21.15.19:0/1637652993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f781e00 msgr2=0x55d03f7b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f1089973740 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f781e00 0x55d03f7b1870 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1078009f10 tx=0x7f1078004ab0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f1089973740 1 -- 172.21.15.19:0/1637652993 shutdown_connections 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f1089973740 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d03f7b1db0 0x55d03f7b2190 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f1089973740 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f781e00 0x55d03f7b1870 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f1089973740 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f7b1470 0x55d03f7b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f1089973740 1 --2- 172.21.15.19:0/1637652993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f795510 0x55d03f7b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f105f7fe640 0 client.6330 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:20.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.490+0000 7f1089973740 1 -- 172.21.15.19:0/1637652993 >> 172.21.15.19:0/1637652993 conn(0x55d03f765770 msgr2=0x55d03f791b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.491+0000 7f1089973740 1 -- 172.21.15.19:0/1637652993 shutdown_connections 2024-04-25T09:56:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.491+0000 7f1089973740 1 -- 172.21.15.19:0/1637652993 wait complete. 2024-04-25T09:56:20.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.491+0000 7f1089973740 20 client.6330 trim_cache size 0 max 16384 2024-04-25T09:56:20.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.506+0000 7f1089973740 1 Processor -- start 2024-04-25T09:56:20.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.506+0000 7f1089973740 1 -- start start 2024-04-25T09:56:20.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.506+0000 7f1089973740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f781e00 0x55d03f798010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.506+0000 7f1089973740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f795510 0x55d03f7940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.506+0000 7f1089973740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f7b1470 0x55d03f7945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.506+0000 7f1089973740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10801fc050 con 0x55d03f795510 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.506+0000 7f1089973740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108006fc20 con 0x55d03f7b1470 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.506+0000 7f1089973740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108013e880 con 0x55d03f781e00 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f107ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f795510 0x55d03f7940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f107ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f795510 0x55d03f7940b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43628/0 (socket says 172.21.15.19:43628) 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f107ffff640 1 -- 172.21.15.19:0/3573026626 learned_addr learned my addr 172.21.15.19:0/3573026626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f108520c640 1 --2- 172.21.15.19:0/3573026626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f7b1470 0x55d03f7945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f1084a0b640 1 --2- 172.21.15.19:0/3573026626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f781e00 0x55d03f798010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f107ffff640 1 -- 172.21.15.19:0/3573026626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f781e00 msgr2=0x55d03f798010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f107ffff640 1 --2- 172.21.15.19:0/3573026626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f781e00 0x55d03f798010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f107ffff640 1 -- 172.21.15.19:0/3573026626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f7b1470 msgr2=0x55d03f7945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f107ffff640 1 --2- 172.21.15.19:0/3573026626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f7b1470 0x55d03f7945f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f107ffff640 1 -- 172.21.15.19:0/3573026626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d03f7b1be0 con 0x55d03f795510 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f108520c640 1 --2- 172.21.15.19:0/3573026626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f7b1470 0x55d03f7945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f107ffff640 1 --2- 172.21.15.19:0/3573026626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f795510 0x55d03f7940b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f106c058670 tx=0x7f106c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c24e030c16d25c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f1069ffb640 1 -- 172.21.15.19:0/3573026626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c0569e0 con 0x55d03f795510 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f1069ffb640 1 -- 172.21.15.19:0/3573026626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c054d50 con 0x55d03f795510 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.507+0000 7f1069ffb640 1 -- 172.21.15.19:0/3573026626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c057d70 con 0x55d03f795510 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.508+0000 7f1089973740 1 -- 172.21.15.19:0/3573026626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f795510 msgr2=0x55d03f7940b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.508+0000 7f1089973740 1 --2- 172.21.15.19:0/3573026626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f795510 0x55d03f7940b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f106c058670 tx=0x7f106c014040 comp rx=0 tx=0).stop 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.508+0000 7f1089973740 1 -- 172.21.15.19:0/3573026626 shutdown_connections 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.508+0000 7f1089973740 1 --2- 172.21.15.19:0/3573026626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f7b1470 0x55d03f7945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.508+0000 7f1089973740 1 --2- 172.21.15.19:0/3573026626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f795510 0x55d03f7940b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.508+0000 7f1089973740 1 --2- 172.21.15.19:0/3573026626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f781e00 0x55d03f798010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.508+0000 7f1089973740 1 -- 172.21.15.19:0/3573026626 >> 172.21.15.19:0/3573026626 conn(0x55d03f765770 msgr2=0x55d03f7989a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.508+0000 7f1089973740 1 -- 172.21.15.19:0/3573026626 shutdown_connections 2024-04-25T09:56:20.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.508+0000 7f1089973740 1 -- 172.21.15.19:0/3573026626 wait complete. 2024-04-25T09:56:20.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.508+0000 7f1089973740 1 -- start start 2024-04-25T09:56:20.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.509+0000 7f1089973740 1 Processor -- start 2024-04-25T09:56:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.509+0000 7f1089973740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f781e00 0x55d03f798960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.509+0000 7f1089973740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f795510 0x55d03f7c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.509+0000 7f1089973740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.509+0000 7f1089973740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03f781c10 con 0x55d03f7b1470 2024-04-25T09:56:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.509+0000 7f1089973740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108006f620 con 0x55d03f795510 2024-04-25T09:56:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.509+0000 7f1089973740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d03f7601a0 con 0x55d03f781e00 2024-04-25T09:56:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.509+0000 7f1089973740 10 client.0 _ll_register_callbacks cb 0x7f1078009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.509+0000 7f108520c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.509+0000 7f108520c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43642/0 (socket says 172.21.15.19:43642) 2024-04-25T09:56:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.509+0000 7f108520c640 1 -- 172.21.15.19:0/1323027427 learned_addr learned my addr 172.21.15.19:0/1323027427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f1084a0b640 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f781e00 0x55d03f798960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f107ffff640 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f795510 0x55d03f7c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f108520c640 1 -- 172.21.15.19:0/1323027427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f781e00 msgr2=0x55d03f798960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f108520c640 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f781e00 0x55d03f798960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f108520c640 1 -- 172.21.15.19:0/1323027427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f795510 msgr2=0x55d03f7c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f108520c640 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f795510 0x55d03f7c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f108520c640 1 -- 172.21.15.19:0/1323027427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d03f3a30f0 con 0x55d03f7b1470 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f108520c640 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7afc30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1078009bd0 tx=0x7f1078009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e384d308bfde7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f105f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f105f7fe640 1 -- 172.21.15.19:0/1323027427 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107805a070 con 0x55d03f7b1470 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f105f7fe640 1 -- 172.21.15.19:0/1323027427 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f107804b070 con 0x55d03f7b1470 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f1089973740 1 -- 172.21.15.19:0/1323027427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f106c054d50 con 0x55d03f7b1470 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f105f7fe640 1 -- 172.21.15.19:0/1323027427 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1078063050 con 0x55d03f7b1470 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f1089973740 10 client.6336 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f1089973740 1 -- 172.21.15.19:0/1323027427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55d03f81f2d0 con 0x55d03f7b1470 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f105f7fe640 1 -- 172.21.15.19:0/1323027427 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c054d50 con 0x55d03f7b1470 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f105f7fe640 1 -- 172.21.15.19:0/1323027427 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1078066020 con 0x55d03f7b1470 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f1089973740 10 client.6336 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f1089973740 10 client.6336 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f1089973740 20 client.6336 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f1089973740 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d03f7b0170 0x55d03f81e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f1089973740 4 client.6336 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.510+0000 7f1089973740 1 -- 172.21.15.19:0/1323027427 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d03f6d6630 con 0x55d03f7b0170 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f1084a0b640 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d03f7b0170 0x55d03f81e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f105f7fe640 10 client.6336 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f1084a0b640 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d03f7b0170 0x55d03f81e4e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e72290d1839bca server_cookie=c3930ea649048715 in_seq=0 out_seq=0 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f105f7fe640 1 -- 172.21.15.19:0/1323027427 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d03f6d6630 con 0x55d03f7b0170 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f105f7fe640 10 client.6336 handle_command_reply: tid=0 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f1089973740 1 client.6336 shutdown 2024-04-25T09:56:20.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f1089973740 1 -- 172.21.15.19:0/1323027427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 msgr2=0x55d03f7afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f1089973740 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7afc30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1078009bd0 tx=0x7f1078009000 comp rx=0 tx=0).stop 2024-04-25T09:56:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f1089973740 1 -- 172.21.15.19:0/1323027427 shutdown_connections 2024-04-25T09:56:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f1089973740 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55d03f7b0170 0x55d03f81e4e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f1089973740 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03f7b1470 0x55d03f7afc30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f1089973740 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d03f795510 0x55d03f7c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f1089973740 1 --2- 172.21.15.19:0/1323027427 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d03f781e00 0x55d03f798960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f105f7fe640 0 client.6336 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.511+0000 7f1089973740 1 -- 172.21.15.19:0/1323027427 >> 172.21.15.19:0/1323027427 conn(0x55d03f765770 msgr2=0x55d03f7b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1089973740 1 -- 172.21.15.19:0/1323027427 shutdown_connections 2024-04-25T09:56:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1089973740 1 -- 172.21.15.19:0/1323027427 wait complete. 2024-04-25T09:56:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1089973740 20 client.6336 trim_cache size 0 max 16384 2024-04-25T09:56:20.522 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:20.522 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T09:56:20.522 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T09:56:20.522 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 -- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f106406b4d0 msgr2=0x7f106406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f106406b4d0 0x7f106406d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1074006fd0 tx=0x7f1074008040 comp rx=0 tx=0).stop 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 -- 172.21.15.19:0/1690777197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10801454a0 msgr2=0x7f1080147ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10801454a0 0x7f1080147ea0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f106c00c390 tx=0x7f106c00c850 comp rx=0 tx=0).stop 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 -- 172.21.15.19:0/1690777197 shutdown_connections 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f108015c970 0x7f10801483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10801454a0 0x7f1080147ea0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 --2- 172.21.15.19:0/1690777197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f108006b2f0 0x7f10801e74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 -- 172.21.15.19:0/1690777197 >> 172.21.15.19:0/1690777197 conn(0x7f10800b5af0 msgr2=0x7f10800b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 -- 172.21.15.19:0/1690777197 shutdown_connections 2024-04-25T09:56:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.512+0000 7f1086c6f640 1 -- 172.21.15.19:0/1690777197 wait complete. 2024-04-25T09:56:20.526 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:56:20.526 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:20.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:56:20.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.626+0000 7f10bdb61640 1 Processor -- start 2024-04-25T09:56:20.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.626+0000 7f10bdb61640 1 -- start start 2024-04-25T09:56:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.627+0000 7f10bdb61640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b8161890 0x7f10b80b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.627+0000 7f10bdb61640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f10b80b54a0 0x7f10b80b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.627+0000 7f10bdb61640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b80cf1d0 con 0x7f10b8161240 2024-04-25T09:56:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.627+0000 7f10bdb61640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b80b7120 con 0x7f10b80b54a0 2024-04-25T09:56:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.627+0000 7f10bdb61640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8161620 con 0x7f10b8161890 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.627+0000 7f10b77fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f10b8161240 0x7f10b80ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33488/0 (socket says 172.21.15.19:33488) 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.627+0000 7f10b77fe640 1 -- 172.21.15.19:0/4050637553 learned_addr learned my addr 172.21.15.19:0/4050637553 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b7fff640 1 --2- 172.21.15.19:0/4050637553 >> v2:172.21.15.112:3300/0 conn(0x7f10b80b54a0 0x7f10b80b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b8161890 0x7f10b80b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892846912 0 0) 0x7f10b80cf1d0 con 0x7f10b8161240 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10a0003520 con 0x7f10b8161240 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2540547814 0 0) 0x7f10a0003520 con 0x7f10b8161240 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b8161890 msgr2=0x7f10b80b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b8161890 0x7f10b80b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 >> v2:172.21.15.112:3300/0 conn(0x7f10b80b54a0 msgr2=0x7f10b80b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 --2- 172.21.15.19:0/4050637553 >> v2:172.21.15.112:3300/0 conn(0x7f10b80b54a0 0x7f10b80b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b8161e00 con 0x7f10b8161240 2024-04-25T09:56:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f10a8001670 con 0x7f10b8161240 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 >> v1:172.21.15.19:6789/0 conn(0x7f10b8161240 legacy=0x7f10b80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b8161890 0x7f10a00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 0x7f10a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10a0004d40 0x7f10a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8161620 con 0x7f10a0003f30 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a000b9c0 con 0x7f10a0004d40 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a000bb40 con 0x7f10b8161890 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 0x7f10a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b77fe640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b8161890 0x7f10a00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b7fff640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10a0004d40 0x7f10a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b6ffd640 1 -- 172.21.15.19:0/4050637553 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b8161890 msgr2=0x7f10a00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b8161890 0x7f10a00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b6ffd640 1 -- 172.21.15.19:0/4050637553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10a0004d40 msgr2=0x7f10a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10a0004d40 0x7f10a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.628+0000 7f10b6ffd640 1 -- 172.21.15.19:0/4050637553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a0003520 con 0x7f10a0003f30 2024-04-25T09:56:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.629+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 0x7f10a0004800 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009ac0 tx=0x7f10a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488f7fa667ce6aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.629+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac036070 con 0x7f10a0003f30 2024-04-25T09:56:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.629+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac031400 con 0x7f10a0003f30 2024-04-25T09:56:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.629+0000 7f10b67fc640 1 -- 172.21.15.19:0/4050637553 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac009690 con 0x7f10a0003f30 2024-04-25T09:56:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.630+0000 7f10bdb61640 1 -- 172.21.15.19:0/4050637553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 msgr2=0x7f10a0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.630+0000 7f10bdb61640 1 --2- 172.21.15.19:0/4050637553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 0x7f10a0004800 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009ac0 tx=0x7f10a0005830 comp rx=0 tx=0).stop 2024-04-25T09:56:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.630+0000 7f10b7fff640 1 -- 172.21.15.19:0/4050637553 reap_dead start 2024-04-25T09:56:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.630+0000 7f10bdb61640 1 -- 172.21.15.19:0/4050637553 shutdown_connections 2024-04-25T09:56:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.630+0000 7f10bdb61640 1 -- 172.21.15.19:0/4050637553 >> 172.21.15.19:0/4050637553 conn(0x7f10b80d04a0 msgr2=0x7f10b80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.630+0000 7f10bdb61640 1 -- 172.21.15.19:0/4050637553 shutdown_connections 2024-04-25T09:56:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.630+0000 7f10bdb61640 1 -- 172.21.15.19:0/4050637553 wait complete. 2024-04-25T09:56:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.631+0000 7f10bdb61640 1 Processor -- start 2024-04-25T09:56:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.631+0000 7f10bdb61640 1 -- start start 2024-04-25T09:56:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10bdb61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 0x7f10b815e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10bdb61640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b80b8260 0x7f10b80ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10bdb61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b8151cb0 0x7f10b80c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10bdb61640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81626a0 con 0x7f10a0003f30 2024-04-25T09:56:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10bdb61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81623c0 con 0x7f10b8151cb0 2024-04-25T09:56:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10bdb61640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80d15f0 con 0x7f10b80b8260 2024-04-25T09:56:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 0x7f10b815e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 0x7f10b815e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43656/0 (socket says 172.21.15.19:43656) 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b77fe640 1 -- 172.21.15.19:0/563933755 learned_addr learned my addr 172.21.15.19:0/563933755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b7fff640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b8151cb0 0x7f10b80c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b80b8260 0x7f10b80ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b77fe640 1 -- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b80b8260 msgr2=0x7f10b80ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b77fe640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b80b8260 0x7f10b80ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b77fe640 1 -- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b8151cb0 msgr2=0x7f10b80c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b77fe640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b8151cb0 0x7f10b80c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b77fe640 1 -- 172.21.15.19:0/563933755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b815d6c0 con 0x7f10a0003f30 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b7fff640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b8151cb0 0x7f10b80c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.632+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b80b8260 0x7f10b80ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.633+0000 7f10b77fe640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 0x7f10b815e760 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f10a8007340 tx=0x7f10a8007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.633+0000 7f109ffff640 1 -- 172.21.15.19:0/563933755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a800f8e0 con 0x7f10a0003f30 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.633+0000 7f109ffff640 1 -- 172.21.15.19:0/563933755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a8018070 con 0x7f10a0003f30 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.633+0000 7f10bdb61640 1 -- 172.21.15.19:0/563933755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b815d9a0 con 0x7f10a0003f30 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.633+0000 7f109ffff640 1 -- 172.21.15.19:0/563933755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a800fcc0 con 0x7f10a0003f30 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.633+0000 7f10bdb61640 1 -- 172.21.15.19:0/563933755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b815df60 con 0x7f10a0003f30 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.634+0000 7f109ffff640 1 -- 172.21.15.19:0/563933755 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f10a801d460 con 0x7f10a0003f30 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.635+0000 7f109ffff640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f10a406b500 0x7f10a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.635+0000 7f109ffff640 1 -- 172.21.15.19:0/563933755 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10a808a540 con 0x7f10a0003f30 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.635+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f10a406b500 0x7f10a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.635+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f10a406b500 0x7f10a406d9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f10ac033000 tx=0x7f10ac008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.636+0000 7f10c085d740 1 Processor -- start 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.636+0000 7f10c085d740 1 -- start start 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.636+0000 7f10c085d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587097162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.636+0000 7f10c085d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558709713e80 0x5587097167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.636+0000 7f10c085d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587096e6700 0x5587096e9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.636+0000 7f10c085d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870964fe00 con 0x558709713aa0 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.636+0000 7f10c085d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587095f8d20 con 0x5587096e6700 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.636+0000 7f10c085d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55870964bea0 con 0x558709713e80 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587097162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587097162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43666/0 (socket says 172.21.15.19:43666) 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b77fe640 1 -- 172.21.15.19:0/2652237795 learned_addr learned my addr 172.21.15.19:0/2652237795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b7fff640 1 --2- 172.21.15.19:0/2652237795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587096e6700 0x5587096e9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/2652237795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558709713e80 0x5587097167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b77fe640 1 -- 172.21.15.19:0/2652237795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558709713e80 msgr2=0x5587097167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b77fe640 1 --2- 172.21.15.19:0/2652237795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558709713e80 0x5587097167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b77fe640 1 -- 172.21.15.19:0/2652237795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587096e6700 msgr2=0x5587096e9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b77fe640 1 --2- 172.21.15.19:0/2652237795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587096e6700 0x5587096e9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b77fe640 1 -- 172.21.15.19:0/2652237795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55870961adf0 con 0x558709713aa0 2024-04-25T09:56:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/2652237795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558709713e80 0x5587097167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b7fff640 1 --2- 172.21.15.19:0/2652237795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587096e6700 0x5587096e9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f10b77fe640 1 --2- 172.21.15.19:0/2652237795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587097162b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f10a8060c70 tx=0x7f10a801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181d893895c1ee38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f109d7fa640 1 -- 172.21.15.19:0/2652237795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a80611a0 con 0x558709713aa0 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f109d7fa640 1 -- 172.21.15.19:0/2652237795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a8088950 con 0x558709713aa0 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.637+0000 7f109d7fa640 1 -- 172.21.15.19:0/2652237795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a808a0c0 con 0x558709713aa0 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.638+0000 7f10c085d740 1 -- 172.21.15.19:0/2652237795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 msgr2=0x5587097162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.638+0000 7f10c085d740 1 --2- 172.21.15.19:0/2652237795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587097162b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f10a8060c70 tx=0x7f10a801d000 comp rx=0 tx=0).stop 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.638+0000 7f10c085d740 1 -- 172.21.15.19:0/2652237795 shutdown_connections 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.638+0000 7f10c085d740 1 --2- 172.21.15.19:0/2652237795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587096e6700 0x5587096e9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.638+0000 7f10c085d740 1 --2- 172.21.15.19:0/2652237795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558709713e80 0x5587097167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.638+0000 7f10c085d740 1 --2- 172.21.15.19:0/2652237795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587097162b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.638+0000 7f10c085d740 1 -- 172.21.15.19:0/2652237795 >> 172.21.15.19:0/2652237795 conn(0x5587096e8970 msgr2=0x5587097312b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.638+0000 7f10c085d740 1 -- 172.21.15.19:0/2652237795 shutdown_connections 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.638+0000 7f10c085d740 1 -- 172.21.15.19:0/2652237795 wait complete. 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10c085d740 1 -- start start 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10c085d740 1 Processor -- start 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10c085d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x55870970b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10c085d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713e80 0x558709734140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10c085d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5587097383a0 0x558709734680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10c085d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587096ebb20 con 0x558709713aa0 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10c085d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587095ee590 con 0x558709713e80 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10c085d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587096ebe00 con 0x5587097383a0 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10c085d740 10 client.0 _ll_register_callbacks cb 0x5587095fae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x55870970b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x55870970b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43670/0 (socket says 172.21.15.19:43670) 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.639+0000 7f10b77fe640 1 -- 172.21.15.19:0/777396644 learned_addr learned my addr 172.21.15.19:0/777396644 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713e80 0x558709734140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10b77fe640 1 -- 172.21.15.19:0/777396644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5587097383a0 msgr2=0x558709734680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10b77fe640 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5587097383a0 0x558709734680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10b77fe640 1 -- 172.21.15.19:0/777396644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713e80 msgr2=0x558709734140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10b77fe640 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713e80 0x558709734140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10b77fe640 1 -- 172.21.15.19:0/777396644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587093643f0 con 0x558709713aa0 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10b77fe640 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x55870970b050 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f10a8058d00 tx=0x7f10a8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768b7f232663ac9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10967fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10967fc640 1 -- 172.21.15.19:0/777396644 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8057f90 con 0x558709713aa0 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10967fc640 1 -- 172.21.15.19:0/777396644 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10a8058130 con 0x558709713aa0 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10c085d740 1 -- 172.21.15.19:0/777396644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10a8088950 con 0x558709713aa0 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10967fc640 1 -- 172.21.15.19:0/777396644 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10a805db00 con 0x558709713aa0 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10c085d740 10 client.6348 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10c085d740 1 -- 172.21.15.19:0/777396644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55870942e7d0 con 0x558709713aa0 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10967fc640 1 -- 172.21.15.19:0/777396644 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a8088950 con 0x558709713aa0 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10967fc640 1 -- 172.21.15.19:0/777396644 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f10a805dca0 con 0x558709713aa0 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10c085d740 10 client.6348 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10c085d740 10 client.6348 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10c085d740 20 client.6348 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10c085d740 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x558709717650 0x558709717a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10c085d740 4 client.6348 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.640+0000 7f10c085d740 1 -- 172.21.15.19:0/777396644 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55870965d340 con 0x558709717650 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.641+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x558709717650 0x558709717a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.641+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x558709717650 0x558709717a90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93afaab3152353c6 server_cookie=d316c649ac1056c2 in_seq=0 out_seq=0 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.641+0000 7f10967fc640 10 client.6348 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.641+0000 7f10967fc640 1 -- 172.21.15.19:0/777396644 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55870965d340 con 0x558709717650 2024-04-25T09:56:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.641+0000 7f10967fc640 10 client.6348 handle_command_reply: tid=0 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.641+0000 7f10c085d740 1 client.6348 shutdown 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10c085d740 1 -- 172.21.15.19:0/777396644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 msgr2=0x55870970b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10c085d740 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x55870970b050 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f10a8058d00 tx=0x7f10a8001a40 comp rx=0 tx=0).stop 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10c085d740 1 -- 172.21.15.19:0/777396644 shutdown_connections 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10c085d740 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x558709717650 0x558709717a90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10c085d740 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5587097383a0 0x558709734680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10967fc640 0 client.6348 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10c085d740 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713e80 0x558709734140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10c085d740 1 --2- 172.21.15.19:0/777396644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x55870970b050 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10c085d740 1 -- 172.21.15.19:0/777396644 >> 172.21.15.19:0/777396644 conn(0x5587096e6700 msgr2=0x5587096e73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10c085d740 1 -- 172.21.15.19:0/777396644 shutdown_connections 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10c085d740 1 -- 172.21.15.19:0/777396644 wait complete. 2024-04-25T09:56:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.642+0000 7f10c085d740 20 client.6348 trim_cache size 0 max 16384 2024-04-25T09:56:20.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.653+0000 7f10c085d740 1 Processor -- start 2024-04-25T09:56:20.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.654+0000 7f10c085d740 1 -- start start 2024-04-25T09:56:20.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.654+0000 7f10c085d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587096e7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.654+0000 7f10c085d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713e80 0x5587097304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.654+0000 7f10c085d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5587097383a0 0x5587097309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.654+0000 7f10c085d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8200a30 con 0x558709713aa0 2024-04-25T09:56:20.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.654+0000 7f10c085d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81ffbd0 con 0x558709713e80 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.654+0000 7f10c085d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8156690 con 0x5587097383a0 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.654+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587096e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587096e7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43676/0 (socket says 172.21.15.19:43676) 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f10b77fe640 1 -- 172.21.15.19:0/4047858358 learned_addr learned my addr 172.21.15.19:0/4047858358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/4047858358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713e80 0x5587097304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f10b7fff640 1 --2- 172.21.15.19:0/4047858358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5587097383a0 0x5587097309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f10b77fe640 1 -- 172.21.15.19:0/4047858358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5587097383a0 msgr2=0x5587097309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f10b77fe640 1 --2- 172.21.15.19:0/4047858358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5587097383a0 0x5587097309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f10b77fe640 1 -- 172.21.15.19:0/4047858358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713e80 msgr2=0x5587097304a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f10b77fe640 1 --2- 172.21.15.19:0/4047858358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713e80 0x5587097304a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f10b77fe640 1 -- 172.21.15.19:0/4047858358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587096e9b80 con 0x558709713aa0 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/4047858358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713e80 0x5587097304a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:20.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f10b77fe640 1 --2- 172.21.15.19:0/4047858358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587096e7280 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f10a8054fa0 tx=0x7f10a805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641a8ea606addce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f109cff9640 1 -- 172.21.15.19:0/4047858358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8088910 con 0x558709713aa0 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f109cff9640 1 -- 172.21.15.19:0/4047858358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a80549a0 con 0x558709713aa0 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.655+0000 7f109cff9640 1 -- 172.21.15.19:0/4047858358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8057f90 con 0x558709713aa0 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.656+0000 7f10c085d740 1 -- 172.21.15.19:0/4047858358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 msgr2=0x5587096e7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.656+0000 7f10c085d740 1 --2- 172.21.15.19:0/4047858358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587096e7280 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f10a8054fa0 tx=0x7f10a805b000 comp rx=0 tx=0).stop 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.656+0000 7f10c085d740 1 -- 172.21.15.19:0/4047858358 shutdown_connections 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.656+0000 7f10c085d740 1 --2- 172.21.15.19:0/4047858358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5587097383a0 0x5587097309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.656+0000 7f10c085d740 1 --2- 172.21.15.19:0/4047858358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713e80 0x5587097304a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.656+0000 7f10c085d740 1 --2- 172.21.15.19:0/4047858358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558709713aa0 0x5587096e7280 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.656+0000 7f10c085d740 1 -- 172.21.15.19:0/4047858358 >> 172.21.15.19:0/4047858358 conn(0x5587096e6700 msgr2=0x5587096ed590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.656+0000 7f10c085d740 1 -- 172.21.15.19:0/4047858358 shutdown_connections 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.656+0000 7f10c085d740 1 -- 172.21.15.19:0/4047858358 wait complete. 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.656+0000 7f10c085d740 1 -- start start 2024-04-25T09:56:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.656+0000 7f10c085d740 1 Processor -- start 2024-04-25T09:56:20.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10c085d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713aa0 0x55870979b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10c085d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558709713e80 0x5587097357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10c085d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587097383a0 0x558709735d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10c085d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81ff6d0 con 0x5587097383a0 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10c085d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587097182e0 con 0x558709713aa0 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10c085d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x558709734490 con 0x558709713e80 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10c085d740 10 client.0 _ll_register_callbacks cb 0x7f10a8054e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587097383a0 0x558709735d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587097383a0 0x558709735d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43678/0 (socket says 172.21.15.19:43678) 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10b7fff640 1 -- 172.21.15.19:0/3284860122 learned_addr learned my addr 172.21.15.19:0/3284860122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10b77fe640 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713aa0 0x55870979b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10b6ffd640 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558709713e80 0x5587097357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10b7fff640 1 -- 172.21.15.19:0/3284860122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558709713e80 msgr2=0x5587097357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10b7fff640 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558709713e80 0x5587097357f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10b7fff640 1 -- 172.21.15.19:0/3284860122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713aa0 msgr2=0x55870979b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10b7fff640 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713aa0 0x55870979b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.657+0000 7f10b7fff640 1 -- 172.21.15.19:0/3284860122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558709734770 con 0x5587097383a0 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10b7fff640 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587097383a0 0x558709735d30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f10b81617a0 tx=0x7f10b0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfda8f064c9592a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10967fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10967fc640 1 -- 172.21.15.19:0/3284860122 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0017ae0 con 0x5587097383a0 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10967fc640 1 -- 172.21.15.19:0/3284860122 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10b0022c70 con 0x5587097383a0 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10c085d740 1 -- 172.21.15.19:0/3284860122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10a80549a0 con 0x5587097383a0 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10967fc640 1 -- 172.21.15.19:0/3284860122 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10b0017c80 con 0x5587097383a0 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10c085d740 10 client.6354 fetch_fsmap learned FSMap version 26 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10c085d740 1 -- 172.21.15.19:0/3284860122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x558709730dd0 con 0x5587097383a0 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10967fc640 1 -- 172.21.15.19:0/3284860122 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a80549a0 con 0x5587097383a0 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10967fc640 1 -- 172.21.15.19:0/3284860122 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f10b0029050 con 0x5587097383a0 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10c085d740 10 client.6354 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:56:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10c085d740 10 client.6354 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10c085d740 20 client.6354 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10c085d740 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55870971a160 0x558709711650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10c085d740 4 client.6354 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10c085d740 1 -- 172.21.15.19:0/3284860122 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558709739790 con 0x55870971a160 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.658+0000 7f10b77fe640 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55870971a160 0x558709711650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10b77fe640 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55870971a160 0x558709711650 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e191c2e2caf6879 server_cookie=cd2852dbf011ea86 in_seq=0 out_seq=0 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10967fc640 10 client.6354 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10967fc640 1 -- 172.21.15.19:0/3284860122 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x558709739790 con 0x55870971a160 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10967fc640 10 client.6354 handle_command_reply: tid=0 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 1 client.6354 shutdown 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 1 -- 172.21.15.19:0/3284860122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587097383a0 msgr2=0x558709735d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587097383a0 0x558709735d30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f10b81617a0 tx=0x7f10b0003f90 comp rx=0 tx=0).stop 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 1 -- 172.21.15.19:0/3284860122 shutdown_connections 2024-04-25T09:56:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x55870971a160 0x558709711650 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587097383a0 0x558709735d30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558709713e80 0x5587097357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10967fc640 0 client.6354 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 1 --2- 172.21.15.19:0/3284860122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558709713aa0 0x55870979b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 1 -- 172.21.15.19:0/3284860122 >> 172.21.15.19:0/3284860122 conn(0x5587096e6700 msgr2=0x5587096e7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 1 -- 172.21.15.19:0/3284860122 shutdown_connections 2024-04-25T09:56:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 1 -- 172.21.15.19:0/3284860122 wait complete. 2024-04-25T09:56:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.659+0000 7f10c085d740 20 client.6354 trim_cache size 0 max 16384 2024-04-25T09:56:20.669 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T09:56:20.669 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:20.670 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "backtrace", 2024-04-25T09:56:20.670 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 1535679066, 2024-04-25T09:56:20.670 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511630188, 2024-04-25T09:56:20.670 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T09:56:20.670 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:20.670 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T09:56:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 -- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f10a406b500 msgr2=0x7f10a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f10a406b500 0x7f10a406d9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f10ac033000 tx=0x7f10ac008030 comp rx=0 tx=0).stop 2024-04-25T09:56:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 -- 172.21.15.19:0/563933755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 msgr2=0x7f10b815e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 0x7f10b815e760 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f10a8007340 tx=0x7f10a8007800 comp rx=0 tx=0).stop 2024-04-25T09:56:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 -- 172.21.15.19:0/563933755 shutdown_connections 2024-04-25T09:56:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f10a406b500 0x7f10a406d9c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b8151cb0 0x7f10b80c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10b80b8260 0x7f10b80ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 --2- 172.21.15.19:0/563933755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0003f30 0x7f10b815e760 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 -- 172.21.15.19:0/563933755 >> 172.21.15.19:0/563933755 conn(0x7f10b80d04a0 msgr2=0x7f10b80d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 -- 172.21.15.19:0/563933755 shutdown_connections 2024-04-25T09:56:20.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.660+0000 7f10bdb61640 1 -- 172.21.15.19:0/563933755 wait complete. 2024-04-25T09:56:20.673 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:20.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.771+0000 7f03404e2640 1 Processor -- start 2024-04-25T09:56:20.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.771+0000 7f03404e2640 1 -- start start 2024-04-25T09:56:20.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f03404e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f033815ba00 0x7f033809a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f03404e2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03381602e0 0x7f03380ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f03404e2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f033815f5f0 0x7f033815faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f03404e2640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03380cd710 con 0x7f03381602e0 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f03404e2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0338099b70 con 0x7f033815f5f0 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f03404e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0338099e50 con 0x7f033815ba00 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f033da7d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03381602e0 0x7f03380ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f033da7d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03381602e0 0x7f03380ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43690/0 (socket says 172.21.15.19:43690) 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f033da7d640 1 -- 172.21.15.19:0/2086814417 learned_addr learned my addr 172.21.15.19:0/2086814417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f033ea7f640 1 --2- 172.21.15.19:0/2086814417 >> v2:172.21.15.112:3300/0 conn(0x7f033815f5f0 0x7f033815faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f033e27e640 1 --2- 172.21.15.19:0/2086814417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f033815ba00 0x7f033809a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f033da7d640 1 -- 172.21.15.19:0/2086814417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f033815ba00 msgr2=0x7f033809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f033da7d640 1 --2- 172.21.15.19:0/2086814417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f033815ba00 0x7f033809a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f033da7d640 1 -- 172.21.15.19:0/2086814417 >> v2:172.21.15.112:3300/0 conn(0x7f033815f5f0 msgr2=0x7f033815faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f033da7d640 1 --2- 172.21.15.19:0/2086814417 >> v2:172.21.15.112:3300/0 conn(0x7f033815f5f0 0x7f033815faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.772+0000 7f033da7d640 1 -- 172.21.15.19:0/2086814417 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033809a130 con 0x7f03381602e0 2024-04-25T09:56:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.773+0000 7f033da7d640 1 --2- 172.21.15.19:0/2086814417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03381602e0 0x7f03380ccda0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f03380cbd10 tx=0x7f032802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6154d3818ac21404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.773+0000 7f033d27c640 1 -- 172.21.15.19:0/2086814417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328037070 con 0x7f03381602e0 2024-04-25T09:56:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.773+0000 7f033d27c640 1 -- 172.21.15.19:0/2086814417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032802ae00 con 0x7f03381602e0 2024-04-25T09:56:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.773+0000 7f033d27c640 1 -- 172.21.15.19:0/2086814417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328032660 con 0x7f03381602e0 2024-04-25T09:56:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.774+0000 7f03404e2640 1 -- 172.21.15.19:0/2086814417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03381602e0 msgr2=0x7f03380ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.774+0000 7f03404e2640 1 --2- 172.21.15.19:0/2086814417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03381602e0 0x7f03380ccda0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f03380cbd10 tx=0x7f032802a0b0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.774+0000 7f03404e2640 1 -- 172.21.15.19:0/2086814417 shutdown_connections 2024-04-25T09:56:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.774+0000 7f03404e2640 1 --2- 172.21.15.19:0/2086814417 >> v2:172.21.15.112:3300/0 conn(0x7f033815f5f0 0x7f033815faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.774+0000 7f03404e2640 1 --2- 172.21.15.19:0/2086814417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03381602e0 0x7f03380ccda0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.774+0000 7f03404e2640 1 --2- 172.21.15.19:0/2086814417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f033815ba00 0x7f033809a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.774+0000 7f03404e2640 1 -- 172.21.15.19:0/2086814417 >> 172.21.15.19:0/2086814417 conn(0x7f03380b3270 msgr2=0x7f03380b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:20.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.774+0000 7f03404e2640 1 -- 172.21.15.19:0/2086814417 shutdown_connections 2024-04-25T09:56:20.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.774+0000 7f03404e2640 1 -- 172.21.15.19:0/2086814417 wait complete. 2024-04-25T09:56:20.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.775+0000 7f03404e2640 1 Processor -- start 2024-04-25T09:56:20.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.775+0000 7f03404e2640 1 -- start start 2024-04-25T09:56:20.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f03404e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f033815ba00 0x7f03380cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f03404e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f033815f5f0 0x7f03380b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f03404e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03381602e0 0x7f033814feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f03404e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033815bde0 con 0x7f033815f5f0 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f03404e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03380cd2e0 con 0x7f033815ba00 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f03404e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03380b6770 con 0x7f03381602e0 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f033da7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f033815f5f0 0x7f03380b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f033da7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f033815f5f0 0x7f03380b6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43698/0 (socket says 172.21.15.19:43698) 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f033da7d640 1 -- 172.21.15.19:0/3402571001 learned_addr learned my addr 172.21.15.19:0/3402571001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f033e27e640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f033815ba00 0x7f03380cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f033ea7f640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03381602e0 0x7f033814feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f033da7d640 1 -- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03381602e0 msgr2=0x7f033814feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f033da7d640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03381602e0 0x7f033814feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f033da7d640 1 -- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f033815ba00 msgr2=0x7f03380cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f033da7d640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f033815ba00 0x7f03380cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f033da7d640 1 -- 172.21.15.19:0/3402571001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380c8aa0 con 0x7f033815f5f0 2024-04-25T09:56:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.776+0000 7f033da7d640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f033815f5f0 0x7f03380b6230 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0328000c00 tx=0x7f03280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.777+0000 7f032effd640 1 -- 172.21.15.19:0/3402571001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328041030 con 0x7f033815f5f0 2024-04-25T09:56:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.777+0000 7f032effd640 1 -- 172.21.15.19:0/3402571001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032802ac00 con 0x7f033815f5f0 2024-04-25T09:56:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.777+0000 7f03404e2640 1 -- 172.21.15.19:0/3402571001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03380c8d80 con 0x7f033815f5f0 2024-04-25T09:56:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.777+0000 7f032effd640 1 -- 172.21.15.19:0/3402571001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328037040 con 0x7f033815f5f0 2024-04-25T09:56:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.777+0000 7f03404e2640 1 -- 172.21.15.19:0/3402571001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03381f30e0 con 0x7f033815f5f0 2024-04-25T09:56:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.778+0000 7f03404e2640 1 -- 172.21.15.19:0/3402571001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03380c8830 con 0x7f033815f5f0 2024-04-25T09:56:20.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.778+0000 7f032effd640 1 -- 172.21.15.19:0/3402571001 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0328044070 con 0x7f033815f5f0 2024-04-25T09:56:20.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.779+0000 7f032effd640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f031c06b500 0x7f031c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:20.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.779+0000 7f032effd640 1 -- 172.21.15.19:0/3402571001 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03280aa320 con 0x7f033815f5f0 2024-04-25T09:56:20.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.779+0000 7f033e27e640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f031c06b500 0x7f031c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:20.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.779+0000 7f033e27e640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f031c06b500 0x7f031c06d9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0334003610 tx=0x7f0334008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:20.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.783+0000 7f032effd640 1 -- 172.21.15.19:0/3402571001 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f032804b070 con 0x7f033815f5f0 2024-04-25T09:56:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.999+0000 7f03404e2640 1 -- 172.21.15.19:0/3402571001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0338073990 con 0x7f033815f5f0 2024-04-25T09:56:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:20.999+0000 7f032effd640 1 -- 172.21.15.19:0/3402571001 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f03280789c0 con 0x7f033815f5f0 2024-04-25T09:56:21.003 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:21.003 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:56:13.029515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:21.003 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-25T09:56:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 -- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f031c06b500 msgr2=0x7f031c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f031c06b500 0x7f031c06d9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0334003610 tx=0x7f0334008040 comp rx=0 tx=0).stop 2024-04-25T09:56:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 -- 172.21.15.19:0/3402571001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f033815f5f0 msgr2=0x7f03380b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f033815f5f0 0x7f03380b6230 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0328000c00 tx=0x7f03280046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 -- 172.21.15.19:0/3402571001 shutdown_connections 2024-04-25T09:56:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f031c06b500 0x7f031c06d9c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03381602e0 0x7f033814feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f033815f5f0 0x7f03380b6230 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 --2- 172.21.15.19:0/3402571001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f033815ba00 0x7f03380cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 -- 172.21.15.19:0/3402571001 >> 172.21.15.19:0/3402571001 conn(0x7f03380b3270 msgr2=0x7f0338070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:21.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 -- 172.21.15.19:0/3402571001 shutdown_connections 2024-04-25T09:56:21.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.002+0000 7f03404e2640 1 -- 172.21.15.19:0/3402571001 wait complete. 2024-04-25T09:56:21.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 1535679066 2024-04-25T09:56:21.073 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T09:56:21.070+0000 7fc4f0674640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T09:56:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.117+0000 7f93ad9f5640 1 Processor -- start 2024-04-25T09:56:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.117+0000 7f93ad9f5640 1 -- start start 2024-04-25T09:56:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f93ad9f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a815dc20 0x7f93a80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f93ad9f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f93a8161710 0x7f93a80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f93ad9f5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a815e000 con 0x7f93a80b9330 2024-04-25T09:56:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f93ad9f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a80d1de0 con 0x7f93a8161710 2024-04-25T09:56:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f93ad9f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8161c60 con 0x7f93a815dc20 2024-04-25T09:56:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f939ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f93a8161710 0x7f93a80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f93a77fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f93a80b9330 0x7f93a806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33498/0 (socket says 172.21.15.19:33498) 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f93a77fe640 1 -- 172.21.15.19:0/1114823157 learned_addr learned my addr 172.21.15.19:0/1114823157 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/1114823157 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a815dc20 0x7f93a80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f93a67fc640 1 -- 172.21.15.19:0/1114823157 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810303348 0 0) 0x7f93a815e000 con 0x7f93a80b9330 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f93a67fc640 1 -- 172.21.15.19:0/1114823157 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9394003520 con 0x7f93a80b9330 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f939ffff640 1 -- 172.21.15.19:0/1114823157 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a815dc20 msgr2=0x7f93a80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f939ffff640 1 --2- 172.21.15.19:0/1114823157 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a815dc20 0x7f93a80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f939ffff640 1 -- 172.21.15.19:0/1114823157 >> v1:172.21.15.19:6789/0 conn(0x7f93a80b9330 legacy=0x7f93a806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.118+0000 7f939ffff640 1 -- 172.21.15.19:0/1114823157 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a8161f40 con 0x7f93a8161710 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.119+0000 7f939ffff640 1 --2- 172.21.15.19:0/1114823157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a8161710 0x7f93a80cebb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f93900097a0 tx=0x7f939002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccced9f10d55bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.119+0000 7f93a67fc640 1 -- 172.21.15.19:0/1114823157 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390037070 con 0x7f93a8161710 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.119+0000 7f93a67fc640 1 -- 172.21.15.19:0/1114823157 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9390032440 con 0x7f93a8161710 2024-04-25T09:56:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.119+0000 7f93a67fc640 1 -- 172.21.15.19:0/1114823157 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900327f0 con 0x7f93a8161710 2024-04-25T09:56:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.120+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1114823157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a8161710 msgr2=0x7f93a80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.120+0000 7f93ad9f5640 1 --2- 172.21.15.19:0/1114823157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a8161710 0x7f93a80cebb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f93900097a0 tx=0x7f939002a180 comp rx=0 tx=0).stop 2024-04-25T09:56:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.120+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1114823157 shutdown_connections 2024-04-25T09:56:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.120+0000 7f93ad9f5640 1 --2- 172.21.15.19:0/1114823157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a8161710 0x7f93a80cebb0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.120+0000 7f93ad9f5640 1 --2- 172.21.15.19:0/1114823157 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a815dc20 0x7f93a80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.120+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1114823157 >> 172.21.15.19:0/1114823157 conn(0x7f93a80d0420 msgr2=0x7f93a80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.120+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1114823157 shutdown_connections 2024-04-25T09:56:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.120+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1114823157 wait complete. 2024-04-25T09:56:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.121+0000 7f93ad9f5640 1 Processor -- start 2024-04-25T09:56:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.121+0000 7f93ad9f5640 1 -- start start 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93ad9f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80b9330 0x7f93a815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93ad9f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a815dc20 0x7f93a81600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93ad9f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a8161710 0x7f93a8158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93ad9f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a80b5e50 con 0x7f93a80b9330 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93ad9f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8162500 con 0x7f93a815dc20 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93ad9f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a80b5fd0 con 0x7f93a8161710 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80b9330 0x7f93a815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80b9330 0x7f93a815ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43712/0 (socket says 172.21.15.19:43712) 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a6ffd640 1 -- 172.21.15.19:0/1498005978 learned_addr learned my addr 172.21.15.19:0/1498005978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f939ffff640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a815dc20 0x7f93a81600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a77fe640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a8161710 0x7f93a8158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a6ffd640 1 -- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a8161710 msgr2=0x7f93a8158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a8161710 0x7f93a8158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a6ffd640 1 -- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a815dc20 msgr2=0x7f93a81600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a815dc20 0x7f93a81600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a6ffd640 1 -- 172.21.15.19:0/1498005978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a8158750 con 0x7f93a80b9330 2024-04-25T09:56:21.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80b9330 0x7f93a815ebc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f939800e8a0 tx=0x7f939800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a4ff9640 1 -- 172.21.15.19:0/1498005978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398017070 con 0x7f93a80b9330 2024-04-25T09:56:21.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a4ff9640 1 -- 172.21.15.19:0/1498005978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939800c940 con 0x7f93a80b9330 2024-04-25T09:56:21.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1498005978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a80caba0 con 0x7f93a80b9330 2024-04-25T09:56:21.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93a4ff9640 1 -- 172.21.15.19:0/1498005978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939800cd30 con 0x7f93a80b9330 2024-04-25T09:56:21.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.122+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1498005978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a80cb160 con 0x7f93a80b9330 2024-04-25T09:56:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.124+0000 7f93a4ff9640 1 -- 172.21.15.19:0/1498005978 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f939801a070 con 0x7f93a80b9330 2024-04-25T09:56:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.124+0000 7f93a4ff9640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b500 0x7f938806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.124+0000 7f93a4ff9640 1 -- 172.21.15.19:0/1498005978 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f939808a5b0 con 0x7f93a80b9330 2024-04-25T09:56:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.125+0000 7f939ffff640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b500 0x7f938806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.125+0000 7f939ffff640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b500 0x7f938806d9c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9390034000 tx=0x7f939002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.125+0000 7f93b07d5740 1 Processor -- start 2024-04-25T09:56:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.125+0000 7f93b07d5740 1 -- start start 2024-04-25T09:56:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.126+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430650f80 0x560430653790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.126+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x560430653cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.126+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430623be0 0x5604306273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.126+0000 7f93b07d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56043058cac0 con 0x560430623be0 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.126+0000 7f93b07d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x560430535db0 con 0x560430651360 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.126+0000 7f93b07d5740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560430588b60 con 0x560430650f80 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.126+0000 7f93a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430623be0 0x5604306273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.126+0000 7f93a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430623be0 0x5604306273f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43722/0 (socket says 172.21.15.19:43722) 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.126+0000 7f93a77fe640 1 -- 172.21.15.19:0/186462341 learned_addr learned my addr 172.21.15.19:0/186462341 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.126+0000 7f939ffff640 1 --2- 172.21.15.19:0/186462341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x560430653cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.126+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/186462341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430650f80 0x560430653790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.127+0000 7f93a77fe640 1 -- 172.21.15.19:0/186462341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430650f80 msgr2=0x560430653790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.127+0000 7f93a77fe640 1 --2- 172.21.15.19:0/186462341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430650f80 0x560430653790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.127+0000 7f93a77fe640 1 -- 172.21.15.19:0/186462341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 msgr2=0x560430653cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.127+0000 7f93a77fe640 1 --2- 172.21.15.19:0/186462341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x560430653cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.127+0000 7f93a77fe640 1 -- 172.21.15.19:0/186462341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56043036d710 con 0x560430623be0 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.127+0000 7f93a77fe640 1 --2- 172.21.15.19:0/186462341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430623be0 0x5604306273f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f93a0002070 tx=0x7f93a000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6c8a2996153811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.127+0000 7f939d7fa640 1 -- 172.21.15.19:0/186462341 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0018070 con 0x560430623be0 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.127+0000 7f939d7fa640 1 -- 172.21.15.19:0/186462341 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a0013670 con 0x560430623be0 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.127+0000 7f939d7fa640 1 -- 172.21.15.19:0/186462341 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0013a40 con 0x560430623be0 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.127+0000 7f93b07d5740 1 -- 172.21.15.19:0/186462341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430623be0 msgr2=0x5604306273f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.127+0000 7f93b07d5740 1 --2- 172.21.15.19:0/186462341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430623be0 0x5604306273f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f93a0002070 tx=0x7f93a000d520 comp rx=0 tx=0).stop 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.128+0000 7f93b07d5740 1 -- 172.21.15.19:0/186462341 shutdown_connections 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.128+0000 7f93b07d5740 1 --2- 172.21.15.19:0/186462341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430623be0 0x5604306273f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.128+0000 7f93b07d5740 1 --2- 172.21.15.19:0/186462341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x560430653cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.128+0000 7f93b07d5740 1 --2- 172.21.15.19:0/186462341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430650f80 0x560430653790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.128+0000 7f93b07d5740 1 -- 172.21.15.19:0/186462341 >> 172.21.15.19:0/186462341 conn(0x560430625e50 msgr2=0x56043066e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.128+0000 7f93b07d5740 1 -- 172.21.15.19:0/186462341 shutdown_connections 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.128+0000 7f93b07d5740 1 -- 172.21.15.19:0/186462341 wait complete. 2024-04-25T09:56:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.128+0000 7f93b07d5740 1 -- start start 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.128+0000 7f93b07d5740 1 Processor -- start 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430650f80 0x5604306715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430651360 0x56043064f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93b07d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560430629000 con 0x56043063c270 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93b07d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56043052b4e0 con 0x560430650f80 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93b07d5740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604306292e0 con 0x560430651360 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93b07d5740 10 client.0 _ll_register_callbacks cb 0x5604302a0af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43736/0 (socket says 172.21.15.19:43736) 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93a77fe640 1 -- 172.21.15.19:0/3109765177 learned_addr learned my addr 172.21.15.19:0/3109765177 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430650f80 0x5604306715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f939ffff640 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430651360 0x56043064f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93a77fe640 1 -- 172.21.15.19:0/3109765177 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430651360 msgr2=0x56043064f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93a77fe640 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430651360 0x56043064f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93a77fe640 1 -- 172.21.15.19:0/3109765177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430650f80 msgr2=0x5604306715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93a77fe640 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430650f80 0x5604306715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.129+0000 7f93a77fe640 1 -- 172.21.15.19:0/3109765177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604303cdea0 con 0x56043063c270 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93a77fe640 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064fb40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009010 tx=0x7f93a0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32e4d865d10af4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93867fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93867fc640 1 -- 172.21.15.19:0/3109765177 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0023020 con 0x56043063c270 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93867fc640 1 -- 172.21.15.19:0/3109765177 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f93a0001c00 con 0x56043063c270 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93b07d5740 1 -- 172.21.15.19:0/3109765177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93a0013670 con 0x56043063c270 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93867fc640 1 -- 172.21.15.19:0/3109765177 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a002d050 con 0x56043063c270 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93b07d5740 10 client.6375 fetch_fsmap learned FSMap version 27 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93b07d5740 1 -- 172.21.15.19:0/3109765177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x560430261000 con 0x56043063c270 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93867fc640 1 -- 172.21.15.19:0/3109765177 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a0013670 con 0x56043063c270 2024-04-25T09:56:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93867fc640 1 -- 172.21.15.19:0/3109765177 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f93a0030020 con 0x56043063c270 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93b07d5740 10 client.6375 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93b07d5740 10 client.6375 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93b07d5740 20 client.6375 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93b07d5740 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560430671b10 0x56043066f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93b07d5740 4 client.6375 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93b07d5740 1 -- 172.21.15.19:0/3109765177 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56043059a000 con 0x560430671b10 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.130+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560430671b10 0x56043066f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93867fc640 10 client.6375 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560430671b10 0x56043066f940 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c272ded8fd40cdac server_cookie=660a883de2c48b30 in_seq=0 out_seq=0 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93867fc640 1 -- 172.21.15.19:0/3109765177 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56043059a000 con 0x560430671b10 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93867fc640 10 client.6375 handle_command_reply: tid=0 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93b07d5740 1 client.6375 shutdown 2024-04-25T09:56:21.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93b07d5740 1 -- 172.21.15.19:0/3109765177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 msgr2=0x56043064fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93b07d5740 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064fb40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009010 tx=0x7f93a0001250 comp rx=0 tx=0).stop 2024-04-25T09:56:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93b07d5740 1 -- 172.21.15.19:0/3109765177 shutdown_connections 2024-04-25T09:56:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93b07d5740 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560430671b10 0x56043066f940 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93b07d5740 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064fb40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93867fc640 0 client.6375 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93b07d5740 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430651360 0x56043064f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93b07d5740 1 --2- 172.21.15.19:0/3109765177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430650f80 0x5604306715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.131+0000 7f93b07d5740 1 -- 172.21.15.19:0/3109765177 >> 172.21.15.19:0/3109765177 conn(0x560430623be0 msgr2=0x560430624c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.132+0000 7f93b07d5740 1 -- 172.21.15.19:0/3109765177 shutdown_connections 2024-04-25T09:56:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.132+0000 7f93b07d5740 1 -- 172.21.15.19:0/3109765177 wait complete. 2024-04-25T09:56:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.132+0000 7f93b07d5740 20 client.6375 trim_cache size 0 max 16384 2024-04-25T09:56:21.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.144+0000 7f93b07d5740 1 Processor -- start 2024-04-25T09:56:21.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.144+0000 7f93b07d5740 1 -- start start 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.144+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56043063c270 0x560430670fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.144+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430650f80 0x5604306714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.144+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x56043064f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.144+0000 7f93b07d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8200340 con 0x560430650f80 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.144+0000 7f93b07d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81ff4e0 con 0x560430651360 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.144+0000 7f93b07d5740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81fe910 con 0x56043063c270 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.144+0000 7f939ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430650f80 0x5604306714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.144+0000 7f939ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430650f80 0x5604306714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43738/0 (socket says 172.21.15.19:43738) 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f939ffff640 1 -- 172.21.15.19:0/3211686877 learned_addr learned my addr 172.21.15.19:0/3211686877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f93a77fe640 1 --2- 172.21.15.19:0/3211686877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x56043064f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/3211686877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56043063c270 0x560430670fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f939ffff640 1 -- 172.21.15.19:0/3211686877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56043063c270 msgr2=0x560430670fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f939ffff640 1 --2- 172.21.15.19:0/3211686877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56043063c270 0x560430670fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f939ffff640 1 -- 172.21.15.19:0/3211686877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 msgr2=0x56043064f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f939ffff640 1 --2- 172.21.15.19:0/3211686877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x56043064f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f939ffff640 1 -- 172.21.15.19:0/3211686877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560430672580 con 0x560430650f80 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f93a77fe640 1 --2- 172.21.15.19:0/3211686877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x56043064f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/3211686877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56043063c270 0x560430670fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f939ffff640 1 --2- 172.21.15.19:0/3211686877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430650f80 0x5604306714e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9390009890 tx=0x7f939002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ec839f05b9cbcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f939cff9640 1 -- 172.21.15.19:0/3211686877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939004f070 con 0x560430650f80 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f939cff9640 1 -- 172.21.15.19:0/3211686877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939004a480 con 0x560430650f80 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.145+0000 7f939cff9640 1 -- 172.21.15.19:0/3211686877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390037040 con 0x560430650f80 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.146+0000 7f93b07d5740 1 -- 172.21.15.19:0/3211686877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430650f80 msgr2=0x5604306714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.146+0000 7f93b07d5740 1 --2- 172.21.15.19:0/3211686877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430650f80 0x5604306714e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9390009890 tx=0x7f939002c000 comp rx=0 tx=0).stop 2024-04-25T09:56:21.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.146+0000 7f93b07d5740 1 -- 172.21.15.19:0/3211686877 shutdown_connections 2024-04-25T09:56:21.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.146+0000 7f93b07d5740 1 --2- 172.21.15.19:0/3211686877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x56043064f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.146+0000 7f93b07d5740 1 --2- 172.21.15.19:0/3211686877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560430650f80 0x5604306714e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.146+0000 7f93b07d5740 1 --2- 172.21.15.19:0/3211686877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56043063c270 0x560430670fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.146+0000 7f93b07d5740 1 -- 172.21.15.19:0/3211686877 >> 172.21.15.19:0/3211686877 conn(0x560430623be0 msgr2=0x56043061f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:21.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.146+0000 7f93b07d5740 1 -- 172.21.15.19:0/3211686877 shutdown_connections 2024-04-25T09:56:21.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.146+0000 7f93b07d5740 1 -- 172.21.15.19:0/3211686877 wait complete. 2024-04-25T09:56:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.146+0000 7f93b07d5740 1 -- start start 2024-04-25T09:56:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.146+0000 7f93b07d5740 1 Processor -- start 2024-04-25T09:56:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430650f80 0x56043064ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f93b07d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x560430664120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f93b07d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56043067f520 con 0x56043063c270 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f93b07d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81fef10 con 0x560430651360 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f93b07d5740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56043064df60 con 0x560430650f80 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f93b07d5740 10 client.0 _ll_register_callbacks cb 0x7f93a8062560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f93a6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f93a6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43746/0 (socket says 172.21.15.19:43746) 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f93a6ffd640 1 -- 172.21.15.19:0/4190842510 learned_addr learned my addr 172.21.15.19:0/4190842510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f939ffff640 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430650f80 0x56043064ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.147+0000 7f93a77fe640 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x560430664120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93a6ffd640 1 -- 172.21.15.19:0/4190842510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430650f80 msgr2=0x56043064ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430650f80 0x56043064ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93a6ffd640 1 -- 172.21.15.19:0/4190842510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 msgr2=0x560430664120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x560430664120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93a6ffd640 1 -- 172.21.15.19:0/4190842510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56043064c850 con 0x56043063c270 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93a6ffd640 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064a6c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f939800c520 tx=0x7f939808a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74eb3e9a2d336cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93867fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93867fc640 1 -- 172.21.15.19:0/4190842510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398021040 con 0x56043063c270 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93867fc640 1 -- 172.21.15.19:0/4190842510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9398056e90 con 0x56043063c270 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93b07d5740 1 -- 172.21.15.19:0/4190842510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f939004a480 con 0x56043063c270 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93867fc640 1 -- 172.21.15.19:0/4190842510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f939807c050 con 0x56043063c270 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93b07d5740 10 client.6381 fetch_fsmap learned FSMap version 27 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93b07d5740 1 -- 172.21.15.19:0/4190842510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x56043067dce0 con 0x56043063c270 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93867fc640 1 -- 172.21.15.19:0/4190842510 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939004a480 con 0x56043063c270 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93867fc640 1 -- 172.21.15.19:0/4190842510 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9398055200 con 0x56043063c270 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93b07d5740 10 client.6381 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:56:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93b07d5740 10 client.6381 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93b07d5740 20 client.6381 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93b07d5740 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560430664cd0 0x56043064b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93b07d5740 4 client.6381 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 1535679066}] 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f93b07d5740 1 -- 172.21.15.19:0/4190842510 --> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1535679066}) v1 -- 0x560430594350 con 0x560430664cd0 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.148+0000 7f939ffff640 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560430664cd0 0x56043064b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f939ffff640 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560430664cd0 0x56043064b7d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae547d57a092c1cd server_cookie=8552d145eb09a489 in_seq=0 out_seq=0 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93867fc640 10 client.6381 ms_handle_connect on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93867fc640 1 -- 172.21.15.19:0/4190842510 <== mds.0 v2:172.21.15.19:6834/2436445353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x560430594350 con 0x560430664cd0 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93867fc640 10 client.6381 handle_command_reply: tid=0 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 1 client.6381 shutdown 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 1 -- 172.21.15.19:0/4190842510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 msgr2=0x56043064a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064a6c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f939800c520 tx=0x7f939808a040 comp rx=0 tx=0).stop 2024-04-25T09:56:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 1 -- 172.21.15.19:0/4190842510 shutdown_connections 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.19:6834/2436445353,v1:172.21.15.19:6835/2436445353] conn(0x560430664cd0 0x56043064b7d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560430651360 0x560430664120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560430650f80 0x56043064ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 1 --2- 172.21.15.19:0/4190842510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56043063c270 0x56043064a6c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93867fc640 0 client.6381 ms_handle_reset on v2:172.21.15.19:6834/2436445353 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 1 -- 172.21.15.19:0/4190842510 >> 172.21.15.19:0/4190842510 conn(0x560430623be0 msgr2=0x560430670fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 1 -- 172.21.15.19:0/4190842510 shutdown_connections 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 1 -- 172.21.15.19:0/4190842510 wait complete. 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.149+0000 7f93b07d5740 20 client.6381 trim_cache size 0 max 16384 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b500 msgr2=0x7f938806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b500 0x7f938806d9c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9390034000 tx=0x7f939002e000 comp rx=0 tx=0).stop 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1498005978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80b9330 msgr2=0x7f93a815ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80b9330 0x7f93a815ebc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f939800e8a0 tx=0x7f939800ed60 comp rx=0 tx=0).stop 2024-04-25T09:56:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1498005978 shutdown_connections 2024-04-25T09:56:21.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f938806b500 0x7f938806d9c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a8161710 0x7f93a8158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a815dc20 0x7f93a81600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 --2- 172.21.15.19:0/1498005978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93a80b9330 0x7f93a815ebc0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1498005978 >> 172.21.15.19:0/1498005978 conn(0x7f93a80d0420 msgr2=0x7f93a80d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:21.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1498005978 shutdown_connections 2024-04-25T09:56:21.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.150+0000 7f93ad9f5640 1 -- 172.21.15.19:0/1498005978 wait complete. 2024-04-25T09:56:21.163 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:56:21.163 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:21.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.260+0000 7fcd0877e640 1 Processor -- start 2024-04-25T09:56:21.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.260+0000 7fcd0877e640 1 -- start start 2024-04-25T09:56:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.261+0000 7fcd0877e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcd000b6cb0 0x7fcd000b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.261+0000 7fcd0877e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd000ceee0 con 0x7fcd0015db70 2024-04-25T09:56:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.261+0000 7fcd0877e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd000d1d10 con 0x7fcd000b5a60 2024-04-25T09:56:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.261+0000 7fcd0877e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd000d1ff0 con 0x7fcd000b6cb0 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd0651a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcd0015db70 0x7fcd00162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33514/0 (socket says 172.21.15.19:33514) 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd0651a640 1 -- 172.21.15.19:0/2781351419 learned_addr learned my addr 172.21.15.19:0/2781351419 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05d19640 1 --2- 172.21.15.19:0/2781351419 >> v2:172.21.15.112:3300/0 conn(0x7fcd000b6cb0 0x7fcd000b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275653495 0 0) 0x7fcd000ceee0 con 0x7fcd0015db70 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccec003520 con 0x7fcd0015db70 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2465571502 0 0) 0x7fccec003520 con 0x7fcd0015db70 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 >> v2:172.21.15.112:3300/0 conn(0x7fcd000b6cb0 msgr2=0x7fcd000b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 --2- 172.21.15.19:0/2781351419 >> v2:172.21.15.112:3300/0 conn(0x7fcd000b6cb0 0x7fcd000b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 >> v1:172.21.15.112:6789/0 conn(0x7fcd000b5a60 legacy=0x7fcd000d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd000d22d0 con 0x7fcd0015db70 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fccfc001670 con 0x7fcd0015db70 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 >> v1:172.21.15.19:6789/0 conn(0x7fcd0015db70 legacy=0x7fcd00162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 --2- 172.21.15.19:0/2781351419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccec0039b0 0x7fccec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 --2- 172.21.15.19:0/2781351419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccec004800 0x7fccec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 --2- 172.21.15.19:0/2781351419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccec0051c0 0x7fccec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000d1ff0 con 0x7fccec0051c0 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec00c160 con 0x7fccec004800 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec00c360 con 0x7fccec0039b0 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd06d1b640 1 --2- 172.21.15.19:0/2781351419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccec0051c0 0x7fccec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd05d19640 1 --2- 172.21.15.19:0/2781351419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccec004800 0x7fccec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.262+0000 7fcd0651a640 1 --2- 172.21.15.19:0/2781351419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccec0039b0 0x7fccec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.263+0000 7fcd06d1b640 1 -- 172.21.15.19:0/2781351419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccec0039b0 msgr2=0x7fccec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.263+0000 7fcd06d1b640 1 --2- 172.21.15.19:0/2781351419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccec0039b0 0x7fccec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.263+0000 7fcd06d1b640 1 -- 172.21.15.19:0/2781351419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccec004800 msgr2=0x7fccec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.263+0000 7fcd06d1b640 1 --2- 172.21.15.19:0/2781351419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccec004800 0x7fccec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.263+0000 7fcd06d1b640 1 -- 172.21.15.19:0/2781351419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec003520 con 0x7fccec0051c0 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.263+0000 7fcd06d1b640 1 -- 172.21.15.19:0/2781351419 reap_dead start 2024-04-25T09:56:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.263+0000 7fcd06d1b640 1 --2- 172.21.15.19:0/2781351419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccec0051c0 0x7fccec00b9c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fccf8001e50 tx=0x7fccec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce84caf82642f803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.263+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8001a70 con 0x7fccec0051c0 2024-04-25T09:56:21.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.263+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf8003970 con 0x7fccec0051c0 2024-04-25T09:56:21.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.263+0000 7fcd05518640 1 -- 172.21.15.19:0/2781351419 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8003d60 con 0x7fccec0051c0 2024-04-25T09:56:21.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.264+0000 7fcd0877e640 1 -- 172.21.15.19:0/2781351419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccec0051c0 msgr2=0x7fccec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.264+0000 7fcd0877e640 1 --2- 172.21.15.19:0/2781351419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccec0051c0 0x7fccec00b9c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fccf8001e50 tx=0x7fccec005830 comp rx=0 tx=0).stop 2024-04-25T09:56:21.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.264+0000 7fcd0877e640 1 -- 172.21.15.19:0/2781351419 shutdown_connections 2024-04-25T09:56:21.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.264+0000 7fcd0877e640 1 --2- 172.21.15.19:0/2781351419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccec0051c0 0x7fccec00b9c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.264+0000 7fcd0877e640 1 -- 172.21.15.19:0/2781351419 >> 172.21.15.19:0/2781351419 conn(0x7fcd000d0520 msgr2=0x7fcd000d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:21.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.264+0000 7fcd0877e640 1 -- 172.21.15.19:0/2781351419 shutdown_connections 2024-04-25T09:56:21.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.264+0000 7fcd0877e640 1 -- 172.21.15.19:0/2781351419 wait complete. 2024-04-25T09:56:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.265+0000 7fcd0877e640 1 Processor -- start 2024-04-25T09:56:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.265+0000 7fcd0877e640 1 -- start start 2024-04-25T09:56:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.266+0000 7fcd0877e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccec0051c0 0x7fcd00152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.266+0000 7fcd0877e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd000cd710 0x7fcd000cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.266+0000 7fcd0877e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd00153ae0 0x7fcd00160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.266+0000 7fcd0877e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000cf1c0 con 0x7fcd000cd710 2024-04-25T09:56:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.266+0000 7fcd0877e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000ce530 con 0x7fcd00153ae0 2024-04-25T09:56:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.266+0000 7fcd0877e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000cf340 con 0x7fccec0051c0 2024-04-25T09:56:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.266+0000 7fcd05d19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd000cd710 0x7fcd000cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.266+0000 7fcd05d19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd000cd710 0x7fcd000cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43764/0 (socket says 172.21.15.19:43764) 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.266+0000 7fcd05d19640 1 -- 172.21.15.19:0/1625705317 learned_addr learned my addr 172.21.15.19:0/1625705317 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.266+0000 7fcd0651a640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccec0051c0 0x7fcd00152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.266+0000 7fcd06d1b640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd00153ae0 0x7fcd00160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fcd05d19640 1 -- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccec0051c0 msgr2=0x7fcd00152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fcd05d19640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccec0051c0 0x7fcd00152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fcd05d19640 1 -- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd00153ae0 msgr2=0x7fcd00160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fcd05d19640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd00153ae0 0x7fcd00160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fcd05d19640 1 -- 172.21.15.19:0/1625705317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd000cc670 con 0x7fcd000cd710 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fcd0651a640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccec0051c0 0x7fcd00152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fcd06d1b640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd00153ae0 0x7fcd00160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fcd05d19640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd000cd710 0x7fcd000cbf70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fccf0011990 tx=0x7fccf0011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fccf6ffd640 1 -- 172.21.15.19:0/1625705317 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0019070 con 0x7fcd000cd710 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fccf6ffd640 1 -- 172.21.15.19:0/1625705317 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf0004240 con 0x7fcd000cd710 2024-04-25T09:56:21.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fcd0877e640 1 -- 172.21.15.19:0/1625705317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd000cc950 con 0x7fcd000cd710 2024-04-25T09:56:21.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fccf6ffd640 1 -- 172.21.15.19:0/1625705317 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf000f860 con 0x7fcd000cd710 2024-04-25T09:56:21.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.267+0000 7fcd0877e640 1 -- 172.21.15.19:0/1625705317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd000ccf10 con 0x7fcd000cd710 2024-04-25T09:56:21.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.268+0000 7fcd0877e640 1 -- 172.21.15.19:0/1625705317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd00073990 con 0x7fcd000cd710 2024-04-25T09:56:21.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.268+0000 7fccf6ffd640 1 -- 172.21.15.19:0/1625705317 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccf001c070 con 0x7fcd000cd710 2024-04-25T09:56:21.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.269+0000 7fccf6ffd640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcce406b4d0 0x7fcce406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.269+0000 7fccf6ffd640 1 -- 172.21.15.19:0/1625705317 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fccf008a8a0 con 0x7fcd000cd710 2024-04-25T09:56:21.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.269+0000 7fcd0651a640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcce406b4d0 0x7fcce406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.270+0000 7fcd0651a640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcce406b4d0 0x7fcce406d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0015d0 tx=0x7fccfc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.273+0000 7fccf6ffd640 1 -- 172.21.15.19:0/1625705317 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fccf0059500 con 0x7fcd000cd710 2024-04-25T09:56:21.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.491+0000 7fcd0877e640 1 -- 172.21.15.19:0/1625705317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd00158f90 con 0x7fcd000cd710 2024-04-25T09:56:21.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.491+0000 7fccf6ffd640 1 -- 172.21.15.19:0/1625705317 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fccf0058e00 con 0x7fcd000cd710 2024-04-25T09:56:21.495 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:21.495 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:56:21.030062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-25T09:56:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 -- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcce406b4d0 msgr2=0x7fcce406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcce406b4d0 0x7fcce406d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0015d0 tx=0x7fccfc009040 comp rx=0 tx=0).stop 2024-04-25T09:56:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 -- 172.21.15.19:0/1625705317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd000cd710 msgr2=0x7fcd000cbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd000cd710 0x7fcd000cbf70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fccf0011990 tx=0x7fccf0011e50 comp rx=0 tx=0).stop 2024-04-25T09:56:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 -- 172.21.15.19:0/1625705317 shutdown_connections 2024-04-25T09:56:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcce406b4d0 0x7fcce406d990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd00153ae0 0x7fcd00160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd000cd710 0x7fcd000cbf70 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 --2- 172.21.15.19:0/1625705317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccec0051c0 0x7fcd00152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 -- 172.21.15.19:0/1625705317 >> 172.21.15.19:0/1625705317 conn(0x7fcd000d0520 msgr2=0x7fcd0015c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 -- 172.21.15.19:0/1625705317 shutdown_connections 2024-04-25T09:56:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.493+0000 7fcd0877e640 1 -- 172.21.15.19:0/1625705317 wait complete. 2024-04-25T09:56:21.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-25T09:56:21.615 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:21.615 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T09:56:21.615 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:21.624 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:56:21.624 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:56:21.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:56:21.668 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:56:21.709 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:56:21.709 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:56:21.709 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:21.725 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:56:21.725 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:21.780 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:21.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.918+0000 7f4a52516640 1 Processor -- start 2024-04-25T09:56:21.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.918+0000 7f4a52516640 1 -- start start 2024-04-25T09:56:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.919+0000 7f4a52516640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c144750 0x7f4a4c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.919+0000 7f4a52516640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4a4c06eb10 0x7f4a4c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.919+0000 7f4a52516640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4c144b30 con 0x7f4a4c0cee60 2024-04-25T09:56:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.919+0000 7f4a52516640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4c0712d0 con 0x7f4a4c06eb10 2024-04-25T09:56:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.919+0000 7f4a52516640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0715b0 con 0x7f4a4c144750 2024-04-25T09:56:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.919+0000 7f4a4b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4a4c06eb10 0x7f4a4c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.919+0000 7f4a50ab3640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4a4c0cee60 0x7f4a4c15f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33522/0 (socket says 172.21.15.19:33522) 2024-04-25T09:56:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.919+0000 7f4a50ab3640 1 -- 172.21.15.19:0/1922321322 learned_addr learned my addr 172.21.15.19:0/1922321322 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.920+0000 7f4a4bfff640 1 --2- 172.21.15.19:0/1922321322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c144750 0x7f4a4c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.920+0000 7f4a4b7fe640 1 -- 172.21.15.19:0/1922321322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c144750 msgr2=0x7f4a4c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.920+0000 7f4a4b7fe640 1 --2- 172.21.15.19:0/1922321322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c144750 0x7f4a4c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.920+0000 7f4a4b7fe640 1 -- 172.21.15.19:0/1922321322 >> v1:172.21.15.19:6789/0 conn(0x7f4a4c0cee60 legacy=0x7f4a4c15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.920+0000 7f4a4b7fe640 1 -- 172.21.15.19:0/1922321322 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c071890 con 0x7f4a4c06eb10 2024-04-25T09:56:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.920+0000 7f4a4affd640 1 -- 172.21.15.19:0/1922321322 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560028501 0 0) 0x7f4a4c144b30 con 0x7f4a4c0cee60 2024-04-25T09:56:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.920+0000 7f4a4bfff640 1 --2- 172.21.15.19:0/1922321322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c144750 0x7f4a4c06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.920+0000 7f4a4b7fe640 1 --2- 172.21.15.19:0/1922321322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a4c06eb10 0x7f4a4c0ce920 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0097a0 tx=0x7f4a3c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3270f29ce0cd2f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.921+0000 7f4a4affd640 1 -- 172.21.15.19:0/1922321322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c037070 con 0x7f4a4c06eb10 2024-04-25T09:56:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.921+0000 7f4a4affd640 1 -- 172.21.15.19:0/1922321322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c032440 con 0x7f4a4c06eb10 2024-04-25T09:56:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.921+0000 7f4a4affd640 1 -- 172.21.15.19:0/1922321322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c032810 con 0x7f4a4c06eb10 2024-04-25T09:56:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.921+0000 7f4a52516640 1 -- 172.21.15.19:0/1922321322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a4c06eb10 msgr2=0x7f4a4c0ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.921+0000 7f4a52516640 1 --2- 172.21.15.19:0/1922321322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a4c06eb10 0x7f4a4c0ce920 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0097a0 tx=0x7f4a3c02a180 comp rx=0 tx=0).stop 2024-04-25T09:56:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.921+0000 7f4a52516640 1 -- 172.21.15.19:0/1922321322 shutdown_connections 2024-04-25T09:56:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.921+0000 7f4a52516640 1 --2- 172.21.15.19:0/1922321322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a4c06eb10 0x7f4a4c0ce920 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.921+0000 7f4a52516640 1 --2- 172.21.15.19:0/1922321322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c144750 0x7f4a4c06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.921+0000 7f4a52516640 1 -- 172.21.15.19:0/1922321322 >> 172.21.15.19:0/1922321322 conn(0x7f4a4c06abe0 msgr2=0x7f4a4c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.921+0000 7f4a52516640 1 -- 172.21.15.19:0/1922321322 shutdown_connections 2024-04-25T09:56:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.921+0000 7f4a52516640 1 -- 172.21.15.19:0/1922321322 wait complete. 2024-04-25T09:56:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.922+0000 7f4a52516640 1 Processor -- start 2024-04-25T09:56:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.922+0000 7f4a52516640 1 -- start start 2024-04-25T09:56:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.923+0000 7f4a52516640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a4c06eb10 0x7f4a4c1f1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.923+0000 7f4a52516640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c0cee60 0x7f4a4c0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.923+0000 7f4a52516640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a4c144750 0x7f4a4c0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.923+0000 7f4a52516640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c145450 con 0x7f4a4c06eb10 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.923+0000 7f4a52516640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c06f020 con 0x7f4a4c144750 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.923+0000 7f4a52516640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c1455d0 con 0x7f4a4c0cee60 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.923+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a4c06eb10 0x7f4a4c1f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.923+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a4c06eb10 0x7f4a4c1f1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43788/0 (socket says 172.21.15.19:43788) 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.923+0000 7f4a4bfff640 1 -- 172.21.15.19:0/727052675 learned_addr learned my addr 172.21.15.19:0/727052675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a4b7fe640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c0cee60 0x7f4a4c0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a50ab3640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a4c144750 0x7f4a4c0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a4bfff640 1 -- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c0cee60 msgr2=0x7f4a4c0b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a4bfff640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c0cee60 0x7f4a4c0b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a4bfff640 1 -- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a4c144750 msgr2=0x7f4a4c0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a4bfff640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a4c144750 0x7f4a4c0b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a4bfff640 1 -- 172.21.15.19:0/727052675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0b0560 con 0x7f4a4c06eb10 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a4b7fe640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c0cee60 0x7f4a4c0b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a50ab3640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a4c144750 0x7f4a4c0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a4bfff640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a4c06eb10 0x7f4a4c1f1510 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400c7f0 tx=0x7f4a3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a48ff9640 1 -- 172.21.15.19:0/727052675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a34015070 con 0x7f4a4c06eb10 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a48ff9640 1 -- 172.21.15.19:0/727052675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a340077c0 con 0x7f4a4c06eb10 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a52516640 1 -- 172.21.15.19:0/727052675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c0b0840 con 0x7f4a4c06eb10 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a48ff9640 1 -- 172.21.15.19:0/727052675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a34007b70 con 0x7f4a4c06eb10 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.924+0000 7f4a52516640 1 -- 172.21.15.19:0/727052675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c0b0e00 con 0x7f4a4c06eb10 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.925+0000 7f4a52516640 1 -- 172.21.15.19:0/727052675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c144fa0 con 0x7f4a4c06eb10 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.925+0000 7f4a48ff9640 1 -- 172.21.15.19:0/727052675 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a34018070 con 0x7f4a4c06eb10 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.926+0000 7f4a48ff9640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a2c06b500 0x7f4a2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.926+0000 7f4a48ff9640 1 -- 172.21.15.19:0/727052675 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a3408a3b0 con 0x7f4a4c06eb10 2024-04-25T09:56:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.926+0000 7f4a4b7fe640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a2c06b500 0x7f4a2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.927+0000 7f4a4b7fe640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a2c06b500 0x7f4a2c06d9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c034000 tx=0x7f4a3c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:21.930+0000 7f4a48ff9640 1 -- 172.21.15.19:0/727052675 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a4c144fa0 con 0x7f4a4c06eb10 2024-04-25T09:56:22.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.111+0000 7f4a52516640 1 -- 172.21.15.19:0/727052675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a4c073990 con 0x7f4a4c06eb10 2024-04-25T09:56:22.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.112+0000 7f4a48ff9640 1 -- 172.21.15.19:0/727052675 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16930 (secure 0 0 0) 0x7f4a34059130 con 0x7f4a4c06eb10 2024-04-25T09:56:22.115 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:22.115 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":53,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:53.990243+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:50.618180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:51.322489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:22.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 -- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a2c06b500 msgr2=0x7f4a2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a2c06b500 0x7f4a2c06d9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c034000 tx=0x7f4a3c02e000 comp rx=0 tx=0).stop 2024-04-25T09:56:22.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 -- 172.21.15.19:0/727052675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a4c06eb10 msgr2=0x7f4a4c1f1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a4c06eb10 0x7f4a4c1f1510 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400c7f0 tx=0x7f4a3400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 -- 172.21.15.19:0/727052675 shutdown_connections 2024-04-25T09:56:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a2c06b500 0x7f4a2c06d9c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a4c144750 0x7f4a4c0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a4c0cee60 0x7f4a4c0b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 --2- 172.21.15.19:0/727052675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a4c06eb10 0x7f4a4c1f1510 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 -- 172.21.15.19:0/727052675 >> 172.21.15.19:0/727052675 conn(0x7f4a4c06abe0 msgr2=0x7f4a4c142850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 -- 172.21.15.19:0/727052675 shutdown_connections 2024-04-25T09:56:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.114+0000 7f4a52516640 1 -- 172.21.15.19:0/727052675 wait complete. 2024-04-25T09:56:22.128 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:55:50.618180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:55:51.322489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:56:22.128 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:56:22.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.228+0000 7f3f84c88640 1 Processor -- start 2024-04-25T09:56:22.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.228+0000 7f3f84c88640 1 -- start start 2024-04-25T09:56:22.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.229+0000 7f3f84c88640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f801610f0 0x7f3f8006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.229+0000 7f3f84c88640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f801616f0 0x7f3f80161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.229+0000 7f3f84c88640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f800ce0a0 0x7f3f800ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.229+0000 7f3f84c88640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f801614d0 con 0x7f3f800ce0a0 2024-04-25T09:56:22.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.229+0000 7f3f84c88640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f800c6960 con 0x7f3f801616f0 2024-04-25T09:56:22.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.229+0000 7f3f84c88640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f80162260 con 0x7f3f801610f0 2024-04-25T09:56:22.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.229+0000 7f3f7ed9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f800ce0a0 0x7f3f800ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.230+0000 7f3f7ed9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f800ce0a0 0x7f3f800ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43794/0 (socket says 172.21.15.19:43794) 2024-04-25T09:56:22.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.230+0000 7f3f7ed9d640 1 -- 172.21.15.19:0/1124994911 learned_addr learned my addr 172.21.15.19:0/1124994911 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:22.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.230+0000 7f3f7dd9b640 1 --2- 172.21.15.19:0/1124994911 >> v2:172.21.15.112:3300/0 conn(0x7f3f801616f0 0x7f3f80161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.230+0000 7f3f7e59c640 1 --2- 172.21.15.19:0/1124994911 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f801610f0 0x7f3f8006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.230+0000 7f3f7ed9d640 1 -- 172.21.15.19:0/1124994911 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f801610f0 msgr2=0x7f3f8006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:22.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.230+0000 7f3f7ed9d640 1 --2- 172.21.15.19:0/1124994911 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f801610f0 0x7f3f8006c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.230+0000 7f3f7ed9d640 1 -- 172.21.15.19:0/1124994911 >> v2:172.21.15.112:3300/0 conn(0x7f3f801616f0 msgr2=0x7f3f80161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:22.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.230+0000 7f3f7ed9d640 1 --2- 172.21.15.19:0/1124994911 >> v2:172.21.15.112:3300/0 conn(0x7f3f801616f0 0x7f3f80161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.230+0000 7f3f7ed9d640 1 -- 172.21.15.19:0/1124994911 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f80162540 con 0x7f3f800ce0a0 2024-04-25T09:56:22.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f7ed9d640 1 --2- 172.21.15.19:0/1124994911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f800ce0a0 0x7f3f800ce530 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3f80072020 tx=0x7f3f7002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a4cc808e4d9ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f7d59a640 1 -- 172.21.15.19:0/1124994911 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70036070 con 0x7f3f800ce0a0 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f7d59a640 1 -- 172.21.15.19:0/1124994911 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f70031600 con 0x7f3f800ce0a0 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f7d59a640 1 -- 172.21.15.19:0/1124994911 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f700319d0 con 0x7f3f800ce0a0 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f84c88640 1 -- 172.21.15.19:0/1124994911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f800ce0a0 msgr2=0x7f3f800ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f84c88640 1 --2- 172.21.15.19:0/1124994911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f800ce0a0 0x7f3f800ce530 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3f80072020 tx=0x7f3f7002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f84c88640 1 -- 172.21.15.19:0/1124994911 shutdown_connections 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f84c88640 1 --2- 172.21.15.19:0/1124994911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f800ce0a0 0x7f3f800ce530 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f84c88640 1 --2- 172.21.15.19:0/1124994911 >> v2:172.21.15.112:3300/0 conn(0x7f3f801616f0 0x7f3f80161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f84c88640 1 --2- 172.21.15.19:0/1124994911 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f801610f0 0x7f3f8006c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f84c88640 1 -- 172.21.15.19:0/1124994911 >> 172.21.15.19:0/1124994911 conn(0x7f3f8006f500 msgr2=0x7f3f8006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f84c88640 1 -- 172.21.15.19:0/1124994911 shutdown_connections 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.231+0000 7f3f84c88640 1 -- 172.21.15.19:0/1124994911 wait complete. 2024-04-25T09:56:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.233+0000 7f3f84c88640 1 Processor -- start 2024-04-25T09:56:22.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.233+0000 7f3f84c88640 1 -- start start 2024-04-25T09:56:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.233+0000 7f3f84c88640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f800ce0a0 0x7f3f8014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.233+0000 7f3f84c88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f801610f0 0x7f3f8015f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.233+0000 7f3f84c88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f801616f0 0x7f3f800adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.233+0000 7f3f84c88640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800cf690 con 0x7f3f801610f0 2024-04-25T09:56:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.233+0000 7f3f84c88640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800cfce0 con 0x7f3f801616f0 2024-04-25T09:56:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.233+0000 7f3f84c88640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8006c700 con 0x7f3f800ce0a0 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f801610f0 0x7f3f8015f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f801610f0 0x7f3f8015f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43804/0 (socket says 172.21.15.19:43804) 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f7dd9b640 1 -- 172.21.15.19:0/1373564384 learned_addr learned my addr 172.21.15.19:0/1373564384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f7e59c640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f800ce0a0 0x7f3f8014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f7ed9d640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f801616f0 0x7f3f800adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f7dd9b640 1 -- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f800ce0a0 msgr2=0x7f3f8014d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f7dd9b640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f800ce0a0 0x7f3f8014d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f7dd9b640 1 -- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f801616f0 msgr2=0x7f3f800adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f7dd9b640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f801616f0 0x7f3f800adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f7dd9b640 1 -- 172.21.15.19:0/1373564384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8014d8b0 con 0x7f3f801610f0 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f7dd9b640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f801610f0 0x7f3f8015f5f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800e870 tx=0x7f3f6800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f76ffd640 1 -- 172.21.15.19:0/1373564384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68017070 con 0x7f3f801610f0 2024-04-25T09:56:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f76ffd640 1 -- 172.21.15.19:0/1373564384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6800c940 con 0x7f3f801610f0 2024-04-25T09:56:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f84c88640 1 -- 172.21.15.19:0/1373564384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8015e550 con 0x7f3f801610f0 2024-04-25T09:56:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f76ffd640 1 -- 172.21.15.19:0/1373564384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6800cd30 con 0x7f3f801610f0 2024-04-25T09:56:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.234+0000 7f3f84c88640 1 -- 172.21.15.19:0/1373564384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8015eb10 con 0x7f3f801610f0 2024-04-25T09:56:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.235+0000 7f3f84c88640 1 -- 172.21.15.19:0/1373564384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f80073a00 con 0x7f3f801610f0 2024-04-25T09:56:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.236+0000 7f3f76ffd640 1 -- 172.21.15.19:0/1373564384 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f6801a070 con 0x7f3f801610f0 2024-04-25T09:56:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.236+0000 7f3f76ffd640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f6006b4d0 0x7f3f6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.236+0000 7f3f76ffd640 1 -- 172.21.15.19:0/1373564384 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3f6808a250 con 0x7f3f801610f0 2024-04-25T09:56:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.236+0000 7f3f7e59c640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f6006b4d0 0x7f3f6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.237+0000 7f3f7e59c640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f6006b4d0 0x7f3f6006d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009a50 tx=0x7f3f78008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.240+0000 7f3f76ffd640 1 -- 172.21.15.19:0/1373564384 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f68058f40 con 0x7f3f801610f0 2024-04-25T09:56:22.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.421+0000 7f3f84c88640 1 -- 172.21.15.19:0/1373564384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3f800b2470 con 0x7f3f801610f0 2024-04-25T09:56:22.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.421+0000 7f3f76ffd640 1 -- 172.21.15.19:0/1373564384 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3f68058840 con 0x7f3f801610f0 2024-04-25T09:56:22.425 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:56:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 -- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f6006b4d0 msgr2=0x7f3f6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f6006b4d0 0x7f3f6006d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009a50 tx=0x7f3f78008040 comp rx=0 tx=0).stop 2024-04-25T09:56:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 -- 172.21.15.19:0/1373564384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f801610f0 msgr2=0x7f3f8015f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f801610f0 0x7f3f8015f5f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800e870 tx=0x7f3f6800ed30 comp rx=0 tx=0).stop 2024-04-25T09:56:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 -- 172.21.15.19:0/1373564384 shutdown_connections 2024-04-25T09:56:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f6006b4d0 0x7f3f6006d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f801616f0 0x7f3f800adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f801610f0 0x7f3f8015f5f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 --2- 172.21.15.19:0/1373564384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f800ce0a0 0x7f3f8014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 -- 172.21.15.19:0/1373564384 >> 172.21.15.19:0/1373564384 conn(0x7f3f8006f500 msgr2=0x7f3f800d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 -- 172.21.15.19:0/1373564384 shutdown_connections 2024-04-25T09:56:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.423+0000 7f3f84c88640 1 -- 172.21.15.19:0/1373564384 wait complete. 2024-04-25T09:56:22.437 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:56:22.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.537+0000 7f1c28589640 1 Processor -- start 2024-04-25T09:56:22.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.537+0000 7f1c28589640 1 -- start start 2024-04-25T09:56:22.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.538+0000 7f1c28589640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1c2015de80 0x7f1c200cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.538+0000 7f1c28589640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c200cefb0 0x7f1c200cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.538+0000 7f1c28589640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c200afc80 con 0x7f1c200cf980 2024-04-25T09:56:22.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.538+0000 7f1c28589640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c200ce4d0 con 0x7f1c200cefb0 2024-04-25T09:56:22.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.538+0000 7f1c28589640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c200ce7b0 con 0x7f1c2015de80 2024-04-25T09:56:22.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.539+0000 7f1c25b24640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c200cefb0 0x7f1c200cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.539+0000 7f1c26b26640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1c200cf980 0x7f1c2006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33528/0 (socket says 172.21.15.19:33528) 2024-04-25T09:56:22.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.539+0000 7f1c26b26640 1 -- 172.21.15.19:0/1444198421 learned_addr learned my addr 172.21.15.19:0/1444198421 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:22.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.539+0000 7f1c26325640 1 --2- 172.21.15.19:0/1444198421 >> v2:172.21.15.112:3300/0 conn(0x7f1c2015de80 0x7f1c200cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.539+0000 7f1c25b24640 1 -- 172.21.15.19:0/1444198421 >> v2:172.21.15.112:3300/0 conn(0x7f1c2015de80 msgr2=0x7f1c200cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:22.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.539+0000 7f1c25b24640 1 --2- 172.21.15.19:0/1444198421 >> v2:172.21.15.112:3300/0 conn(0x7f1c2015de80 0x7f1c200cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.539+0000 7f1c25b24640 1 -- 172.21.15.19:0/1444198421 >> v1:172.21.15.19:6789/0 conn(0x7f1c200cf980 legacy=0x7f1c2006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.539+0000 7f1c25b24640 1 -- 172.21.15.19:0/1444198421 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200cea90 con 0x7f1c200cefb0 2024-04-25T09:56:22.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.539+0000 7f1c25323640 1 -- 172.21.15.19:0/1444198421 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968759282 0 0) 0x7f1c200afc80 con 0x7f1c200cf980 2024-04-25T09:56:22.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c25b24640 1 --2- 172.21.15.19:0/1444198421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c200cefb0 0x7f1c200cf440 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1c200d1800 tx=0x7f1c1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93bd00506c3440 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:22.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c25323640 1 -- 172.21.15.19:0/1444198421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10037070 con 0x7f1c200cefb0 2024-04-25T09:56:22.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c25323640 1 -- 172.21.15.19:0/1444198421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c10032440 con 0x7f1c200cefb0 2024-04-25T09:56:22.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c25323640 1 -- 172.21.15.19:0/1444198421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10032810 con 0x7f1c200cefb0 2024-04-25T09:56:22.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c28589640 1 -- 172.21.15.19:0/1444198421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c200cefb0 msgr2=0x7f1c200cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c28589640 1 --2- 172.21.15.19:0/1444198421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c200cefb0 0x7f1c200cf440 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1c200d1800 tx=0x7f1c1002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c28589640 1 -- 172.21.15.19:0/1444198421 shutdown_connections 2024-04-25T09:56:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c28589640 1 --2- 172.21.15.19:0/1444198421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c200cefb0 0x7f1c200cf440 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c28589640 1 --2- 172.21.15.19:0/1444198421 >> v2:172.21.15.112:3300/0 conn(0x7f1c2015de80 0x7f1c200cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c28589640 1 -- 172.21.15.19:0/1444198421 >> 172.21.15.19:0/1444198421 conn(0x7f1c200b2390 msgr2=0x7f1c200b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c28589640 1 -- 172.21.15.19:0/1444198421 shutdown_connections 2024-04-25T09:56:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.540+0000 7f1c28589640 1 -- 172.21.15.19:0/1444198421 wait complete. 2024-04-25T09:56:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.542+0000 7f1c28589640 1 Processor -- start 2024-04-25T09:56:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.542+0000 7f1c28589640 1 -- start start 2024-04-25T09:56:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.542+0000 7f1c28589640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c200cefb0 0x7f1c200b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.542+0000 7f1c28589640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c200cf980 0x7f1c200b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.542+0000 7f1c28589640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c2015de80 0x7f1c2015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.542+0000 7f1c28589640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200b3030 con 0x7f1c2015de80 2024-04-25T09:56:22.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.542+0000 7f1c28589640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200b2eb0 con 0x7f1c200cefb0 2024-04-25T09:56:22.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.542+0000 7f1c28589640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200c5600 con 0x7f1c200cf980 2024-04-25T09:56:22.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.542+0000 7f1c26b26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c2015de80 0x7f1c2015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c26b26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c2015de80 0x7f1c2015b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43824/0 (socket says 172.21.15.19:43824) 2024-04-25T09:56:22.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c26b26640 1 -- 172.21.15.19:0/2919337870 learned_addr learned my addr 172.21.15.19:0/2919337870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:22.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c26325640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c200cefb0 0x7f1c200b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c25b24640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c200cf980 0x7f1c200b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c26b26640 1 -- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c200cf980 msgr2=0x7f1c200b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c26b26640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c200cf980 0x7f1c200b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c26b26640 1 -- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c200cefb0 msgr2=0x7f1c200b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c26b26640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c200cefb0 0x7f1c200b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c26b26640 1 -- 172.21.15.19:0/2919337870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200b5ce0 con 0x7f1c2015de80 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c26325640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c200cefb0 0x7f1c200b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c26b26640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c2015de80 0x7f1c2015b710 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c7a0 tx=0x7f1c1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c16ffd640 1 -- 172.21.15.19:0/2919337870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18018070 con 0x7f1c2015de80 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c16ffd640 1 -- 172.21.15.19:0/2919337870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1800a640 con 0x7f1c2015de80 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c28589640 1 -- 172.21.15.19:0/2919337870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c201f7470 con 0x7f1c2015de80 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c16ffd640 1 -- 172.21.15.19:0/2919337870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800aab0 con 0x7f1c2015de80 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.543+0000 7f1c28589640 1 -- 172.21.15.19:0/2919337870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c201f7a30 con 0x7f1c2015de80 2024-04-25T09:56:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.544+0000 7f1c28589640 1 -- 172.21.15.19:0/2919337870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c20073a00 con 0x7f1c2015de80 2024-04-25T09:56:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.545+0000 7f1c16ffd640 1 -- 172.21.15.19:0/2919337870 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c1801b070 con 0x7f1c2015de80 2024-04-25T09:56:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.545+0000 7f1c16ffd640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0406b4d0 0x7f1c0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.545+0000 7f1c16ffd640 1 -- 172.21.15.19:0/2919337870 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c180899b0 con 0x7f1c2015de80 2024-04-25T09:56:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.545+0000 7f1c26325640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0406b4d0 0x7f1c0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.546+0000 7f1c26325640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0406b4d0 0x7f1c0406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c006fd0 tx=0x7f1c1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.549+0000 7f1c16ffd640 1 -- 172.21.15.19:0/2919337870 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c180586a0 con 0x7f1c2015de80 2024-04-25T09:56:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.727+0000 7f1c28589640 1 -- 172.21.15.19:0/2919337870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c200cf440 con 0x7f1c2015de80 2024-04-25T09:56:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.727+0000 7f1c16ffd640 1 -- 172.21.15.19:0/2919337870 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c18057fa0 con 0x7f1c2015de80 2024-04-25T09:56:22.730 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:56:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 -- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0406b4d0 msgr2=0x7f1c0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0406b4d0 0x7f1c0406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c006fd0 tx=0x7f1c1c008040 comp rx=0 tx=0).stop 2024-04-25T09:56:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 -- 172.21.15.19:0/2919337870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c2015de80 msgr2=0x7f1c2015b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c2015de80 0x7f1c2015b710 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c7a0 tx=0x7f1c1800cc60 comp rx=0 tx=0).stop 2024-04-25T09:56:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 -- 172.21.15.19:0/2919337870 shutdown_connections 2024-04-25T09:56:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0406b4d0 0x7f1c0406d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c2015de80 0x7f1c2015b710 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c200cf980 0x7f1c200b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 --2- 172.21.15.19:0/2919337870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c200cefb0 0x7f1c200b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 -- 172.21.15.19:0/2919337870 >> 172.21.15.19:0/2919337870 conn(0x7f1c200b2390 msgr2=0x7f1c2006f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 -- 172.21.15.19:0/2919337870 shutdown_connections 2024-04-25T09:56:22.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.729+0000 7f1c28589640 1 -- 172.21.15.19:0/2919337870 wait complete. 2024-04-25T09:56:22.743 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:56:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.844+0000 7fdd67b4f640 1 Processor -- start 2024-04-25T09:56:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.844+0000 7fdd67b4f640 1 -- start start 2024-04-25T09:56:22.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd67b4f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdd6015da90 0x7fdd60161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd67b4f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdd60161ba0 0x7fdd600d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd67b4f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd60162040 con 0x7fdd600b9750 2024-04-25T09:56:22.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd67b4f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd60162600 con 0x7fdd60161ba0 2024-04-25T09:56:22.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd67b4f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd60162320 con 0x7fdd6015da90 2024-04-25T09:56:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd660ec640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdd60161ba0 0x7fdd600d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd660ec640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdd60161ba0 0x7fdd600d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43826/0 (socket says 172.21.15.19:43826) 2024-04-25T09:56:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd660ec640 1 -- 172.21.15.19:0/1307369720 learned_addr learned my addr 172.21.15.19:0/1307369720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd658eb640 1 --2- 172.21.15.19:0/1307369720 >> v2:172.21.15.112:3300/0 conn(0x7fdd6015da90 0x7fdd60161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd660ec640 1 -- 172.21.15.19:0/1307369720 >> v2:172.21.15.112:3300/0 conn(0x7fdd6015da90 msgr2=0x7fdd60161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd660ec640 1 --2- 172.21.15.19:0/1307369720 >> v2:172.21.15.112:3300/0 conn(0x7fdd6015da90 0x7fdd60161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd660ec640 1 -- 172.21.15.19:0/1307369720 >> v1:172.21.15.112:6789/0 conn(0x7fdd600b9750 legacy=0x7fdd600ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:22.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.845+0000 7fdd660ec640 1 -- 172.21.15.19:0/1307369720 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600d1b10 con 0x7fdd60161ba0 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.846+0000 7fdd660ec640 1 --2- 172.21.15.19:0/1307369720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd60161ba0 0x7fdd600d2310 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdd600b61c0 tx=0x7fdd5802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec9dff893f83ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.846+0000 7fdd648e9640 1 -- 172.21.15.19:0/1307369720 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58036070 con 0x7fdd60161ba0 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.846+0000 7fdd648e9640 1 -- 172.21.15.19:0/1307369720 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58031600 con 0x7fdd60161ba0 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.846+0000 7fdd648e9640 1 -- 172.21.15.19:0/1307369720 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580319d0 con 0x7fdd60161ba0 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.847+0000 7fdd67b4f640 1 -- 172.21.15.19:0/1307369720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd60161ba0 msgr2=0x7fdd600d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.847+0000 7fdd67b4f640 1 --2- 172.21.15.19:0/1307369720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd60161ba0 0x7fdd600d2310 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdd600b61c0 tx=0x7fdd5802a6a0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.847+0000 7fdd67b4f640 1 -- 172.21.15.19:0/1307369720 shutdown_connections 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.847+0000 7fdd67b4f640 1 --2- 172.21.15.19:0/1307369720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd60161ba0 0x7fdd600d2310 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.847+0000 7fdd67b4f640 1 --2- 172.21.15.19:0/1307369720 >> v2:172.21.15.112:3300/0 conn(0x7fdd6015da90 0x7fdd60161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.847+0000 7fdd67b4f640 1 -- 172.21.15.19:0/1307369720 >> 172.21.15.19:0/1307369720 conn(0x7fdd60061a20 msgr2=0x7fdd60061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.847+0000 7fdd67b4f640 1 -- 172.21.15.19:0/1307369720 shutdown_connections 2024-04-25T09:56:22.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.847+0000 7fdd67b4f640 1 -- 172.21.15.19:0/1307369720 wait complete. 2024-04-25T09:56:22.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.848+0000 7fdd67b4f640 1 Processor -- start 2024-04-25T09:56:22.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.848+0000 7fdd67b4f640 1 -- start start 2024-04-25T09:56:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd67b4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6015da90 0x7fdd6015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd67b4f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd60161ba0 0x7fdd600ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd67b4f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd600cd050 0x7fdd6014e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd67b4f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600ced80 con 0x7fdd600cd050 2024-04-25T09:56:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd67b4f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6015de70 con 0x7fdd6015da90 2024-04-25T09:56:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd67b4f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600cef00 con 0x7fdd60161ba0 2024-04-25T09:56:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd660ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd600cd050 0x7fdd6014e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd660ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd600cd050 0x7fdd6014e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43830/0 (socket says 172.21.15.19:43830) 2024-04-25T09:56:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd660ec640 1 -- 172.21.15.19:0/3668293936 learned_addr learned my addr 172.21.15.19:0/3668293936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd658eb640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6015da90 0x7fdd6015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd650ea640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd60161ba0 0x7fdd600ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd660ec640 1 -- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd60161ba0 msgr2=0x7fdd600ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd660ec640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd60161ba0 0x7fdd600ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd660ec640 1 -- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6015da90 msgr2=0x7fdd6015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd660ec640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6015da90 0x7fdd6015bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd660ec640 1 -- 172.21.15.19:0/3668293936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600cd6f0 con 0x7fdd600cd050 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.849+0000 7fdd660ec640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd600cd050 0x7fdd6014e930 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdd5802ac90 tx=0x7fdd58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.850+0000 7fdd567fc640 1 -- 172.21.15.19:0/3668293936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5803f020 con 0x7fdd600cd050 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.850+0000 7fdd567fc640 1 -- 172.21.15.19:0/3668293936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58037040 con 0x7fdd600cd050 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.850+0000 7fdd67b4f640 1 -- 172.21.15.19:0/3668293936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd601f7170 con 0x7fdd600cd050 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.850+0000 7fdd567fc640 1 -- 172.21.15.19:0/3668293936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58036070 con 0x7fdd600cd050 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.850+0000 7fdd67b4f640 1 -- 172.21.15.19:0/3668293936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd601f7730 con 0x7fdd600cd050 2024-04-25T09:56:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.851+0000 7fdd67b4f640 1 -- 172.21.15.19:0/3668293936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd60073a00 con 0x7fdd600cd050 2024-04-25T09:56:22.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.851+0000 7fdd567fc640 1 -- 172.21.15.19:0/3668293936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd58042020 con 0x7fdd600cd050 2024-04-25T09:56:22.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.852+0000 7fdd567fc640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd4406b4d0 0x7fdd4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:22.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.852+0000 7fdd567fc640 1 -- 172.21.15.19:0/3668293936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd580aa680 con 0x7fdd600cd050 2024-04-25T09:56:22.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.852+0000 7fdd658eb640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd4406b4d0 0x7fdd4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:22.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.852+0000 7fdd658eb640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd4406b4d0 0x7fdd4406d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c006fd0 tx=0x7fdd5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:22.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:22.856+0000 7fdd567fc640 1 -- 172.21.15.19:0/3668293936 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd58049030 con 0x7fdd600cd050 2024-04-25T09:56:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.038+0000 7fdd67b4f640 1 -- 172.21.15.19:0/3668293936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd600d1590 con 0x7fdd600cd050 2024-04-25T09:56:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.038+0000 7fdd567fc640 1 -- 172.21.15.19:0/3668293936 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd58078db0 con 0x7fdd600cd050 2024-04-25T09:56:23.042 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:56:23.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 -- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd4406b4d0 msgr2=0x7fdd4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd4406b4d0 0x7fdd4406d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c006fd0 tx=0x7fdd5c008040 comp rx=0 tx=0).stop 2024-04-25T09:56:23.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 -- 172.21.15.19:0/3668293936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd600cd050 msgr2=0x7fdd6014e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd600cd050 0x7fdd6014e930 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdd5802ac90 tx=0x7fdd58004a40 comp rx=0 tx=0).stop 2024-04-25T09:56:23.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 -- 172.21.15.19:0/3668293936 shutdown_connections 2024-04-25T09:56:23.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd4406b4d0 0x7fdd4406d990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd600cd050 0x7fdd6014e930 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd60161ba0 0x7fdd600ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 --2- 172.21.15.19:0/3668293936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6015da90 0x7fdd6015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 -- 172.21.15.19:0/3668293936 >> 172.21.15.19:0/3668293936 conn(0x7fdd60061a20 msgr2=0x7fdd600b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 -- 172.21.15.19:0/3668293936 shutdown_connections 2024-04-25T09:56:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.040+0000 7fdd67b4f640 1 -- 172.21.15.19:0/3668293936 wait complete. 2024-04-25T09:56:23.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:56:23.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.153+0000 7f25d8161640 1 Processor -- start 2024-04-25T09:56:23.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.154+0000 7f25d8161640 1 -- start start 2024-04-25T09:56:23.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.154+0000 7f25d8161640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25d0070dc0 0x7f25d006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.154+0000 7f25d8161640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d015deb0 0x7f25d015e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.154+0000 7f25d8161640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d00d1770 con 0x7f25d00711a0 2024-04-25T09:56:23.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.154+0000 7f25d8161640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d00b0eb0 con 0x7f25d0070dc0 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.154+0000 7f25d8161640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d01622b0 con 0x7f25d015deb0 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.154+0000 7f25d5efd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25d0070dc0 0x7f25d006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.154+0000 7f25d56fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f25d00711a0 0x7f25d00b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33538/0 (socket says 172.21.15.19:33538) 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.154+0000 7f25d56fc640 1 -- 172.21.15.19:0/976230569 learned_addr learned my addr 172.21.15.19:0/976230569 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.155+0000 7f25d66fe640 1 --2- 172.21.15.19:0/976230569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d015deb0 0x7f25d015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.155+0000 7f25d5efd640 1 -- 172.21.15.19:0/976230569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d015deb0 msgr2=0x7f25d015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.155+0000 7f25d5efd640 1 --2- 172.21.15.19:0/976230569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d015deb0 0x7f25d015e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.155+0000 7f25d5efd640 1 -- 172.21.15.19:0/976230569 >> v1:172.21.15.19:6789/0 conn(0x7f25d00711a0 legacy=0x7f25d00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.155+0000 7f25d5efd640 1 -- 172.21.15.19:0/976230569 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d0162740 con 0x7f25d0070dc0 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.155+0000 7f25d4efb640 1 -- 172.21.15.19:0/976230569 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654523944 0 0) 0x7f25d00d1770 con 0x7f25d00711a0 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.155+0000 7f25d66fe640 1 --2- 172.21.15.19:0/976230569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d015deb0 0x7f25d015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.155+0000 7f25d5efd640 1 --2- 172.21.15.19:0/976230569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25d0070dc0 0x7f25d006ccd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f25c80097a0 tx=0x7f25c802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a78e3aa837aa06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.156+0000 7f25d4efb640 1 -- 172.21.15.19:0/976230569 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8037070 con 0x7f25d0070dc0 2024-04-25T09:56:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.156+0000 7f25d4efb640 1 -- 172.21.15.19:0/976230569 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c802adb0 con 0x7f25d0070dc0 2024-04-25T09:56:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.156+0000 7f25d4efb640 1 -- 172.21.15.19:0/976230569 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8032660 con 0x7f25d0070dc0 2024-04-25T09:56:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.156+0000 7f25d8161640 1 -- 172.21.15.19:0/976230569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25d0070dc0 msgr2=0x7f25d006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.156+0000 7f25d8161640 1 --2- 172.21.15.19:0/976230569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25d0070dc0 0x7f25d006ccd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f25c80097a0 tx=0x7f25c802a060 comp rx=0 tx=0).stop 2024-04-25T09:56:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.156+0000 7f25d8161640 1 -- 172.21.15.19:0/976230569 shutdown_connections 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.156+0000 7f25d8161640 1 --2- 172.21.15.19:0/976230569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d015deb0 0x7f25d015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.156+0000 7f25d8161640 1 --2- 172.21.15.19:0/976230569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25d0070dc0 0x7f25d006ccd0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.156+0000 7f25d8161640 1 -- 172.21.15.19:0/976230569 >> 172.21.15.19:0/976230569 conn(0x7f25d00b3530 msgr2=0x7f25d00b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.156+0000 7f25d8161640 1 -- 172.21.15.19:0/976230569 shutdown_connections 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.156+0000 7f25d8161640 1 -- 172.21.15.19:0/976230569 wait complete. 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.157+0000 7f25d8161640 1 Processor -- start 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.157+0000 7f25d8161640 1 -- start start 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.158+0000 7f25d8161640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0070dc0 0x7f25d015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.158+0000 7f25d8161640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00711a0 0x7f25d0152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.158+0000 7f25d8161640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25d015deb0 0x7f25d0155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.158+0000 7f25d8161640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0161c60 con 0x7f25d015deb0 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.158+0000 7f25d8161640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00b10f0 con 0x7f25d0070dc0 2024-04-25T09:56:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.158+0000 7f25d8161640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0161f40 con 0x7f25d00711a0 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.158+0000 7f25d66fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25d015deb0 0x7f25d0155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.158+0000 7f25d66fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25d015deb0 0x7f25d0155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43848/0 (socket says 172.21.15.19:43848) 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.158+0000 7f25d66fe640 1 -- 172.21.15.19:0/722946861 learned_addr learned my addr 172.21.15.19:0/722946861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d56fc640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00711a0 0x7f25d0152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d5efd640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0070dc0 0x7f25d015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d66fe640 1 -- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00711a0 msgr2=0x7f25d0152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d66fe640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00711a0 0x7f25d0152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d66fe640 1 -- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0070dc0 msgr2=0x7f25d015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d66fe640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0070dc0 0x7f25d015bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d66fe640 1 -- 172.21.15.19:0/722946861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d01f8a10 con 0x7f25d015deb0 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d5efd640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0070dc0 0x7f25d015bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d56fc640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00711a0 0x7f25d0152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:23.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d66fe640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25d015deb0 0x7f25d0155500 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f25c000cd50 tx=0x7f25c000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:23.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25ce7fc640 1 -- 172.21.15.19:0/722946861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c000f680 con 0x7f25d015deb0 2024-04-25T09:56:23.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25ce7fc640 1 -- 172.21.15.19:0/722946861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c000f840 con 0x7f25d015deb0 2024-04-25T09:56:23.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d8161640 1 -- 172.21.15.19:0/722946861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d015ad70 con 0x7f25d015deb0 2024-04-25T09:56:23.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25ce7fc640 1 -- 172.21.15.19:0/722946861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c000fc30 con 0x7f25d015deb0 2024-04-25T09:56:23.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.159+0000 7f25d8161640 1 -- 172.21.15.19:0/722946861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d015b330 con 0x7f25d015deb0 2024-04-25T09:56:23.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.160+0000 7f25d8161640 1 -- 172.21.15.19:0/722946861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d0073a00 con 0x7f25d015deb0 2024-04-25T09:56:23.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.160+0000 7f25ce7fc640 1 -- 172.21.15.19:0/722946861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25c00213f0 con 0x7f25d015deb0 2024-04-25T09:56:23.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.161+0000 7f25ce7fc640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f25bc06b4d0 0x7f25bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.161+0000 7f25ce7fc640 1 -- 172.21.15.19:0/722946861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f25c008b0d0 con 0x7f25d015deb0 2024-04-25T09:56:23.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.161+0000 7f25d5efd640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f25bc06b4d0 0x7f25bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.162+0000 7f25d5efd640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f25bc06b4d0 0x7f25bc06d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f25c8034000 tx=0x7f25c802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:23.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.165+0000 7f25ce7fc640 1 -- 172.21.15.19:0/722946861 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25c0014030 con 0x7f25d015deb0 2024-04-25T09:56:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.357+0000 7f25d8161640 1 -- 172.21.15.19:0/722946861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f25d00b1cb0 con 0x7f25d015deb0 2024-04-25T09:56:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.357+0000 7f25ce7fc640 1 -- 172.21.15.19:0/722946861 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7f25c0059e30 con 0x7f25d015deb0 2024-04-25T09:56:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T09:56:23.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 -- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f25bc06b4d0 msgr2=0x7f25bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f25bc06b4d0 0x7f25bc06d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f25c8034000 tx=0x7f25c802e000 comp rx=0 tx=0).stop 2024-04-25T09:56:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 -- 172.21.15.19:0/722946861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25d015deb0 msgr2=0x7f25d0155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25d015deb0 0x7f25d0155500 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f25c000cd50 tx=0x7f25c000a650 comp rx=0 tx=0).stop 2024-04-25T09:56:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 -- 172.21.15.19:0/722946861 shutdown_connections 2024-04-25T09:56:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f25bc06b4d0 0x7f25bc06d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25d015deb0 0x7f25d0155500 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25d00711a0 0x7f25d0152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 --2- 172.21.15.19:0/722946861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d0070dc0 0x7f25d015bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 -- 172.21.15.19:0/722946861 >> 172.21.15.19:0/722946861 conn(0x7f25d00b3530 msgr2=0x7f25d006cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 -- 172.21.15.19:0/722946861 shutdown_connections 2024-04-25T09:56:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:23.359+0000 7f25d8161640 1 -- 172.21.15.19:0/722946861 wait complete. 2024-04-25T09:56:23.373 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:56:23.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:23.373 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:24.709 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:56:24.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:24.710 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:56:24.725 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:56:24.725 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:24.744 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:56:24.744 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:56:24.744 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:24.781 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:56:24.781 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:56:24.781 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:24.802 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:56:24.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:24.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.918+0000 7f7587ad2640 1 Processor -- start 2024-04-25T09:56:24.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.918+0000 7f7587ad2640 1 -- start start 2024-04-25T09:56:24.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.919+0000 7f7587ad2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f758015dbe0 0x7f75800d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.919+0000 7f7587ad2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75800b6220 con 0x7f75800b92a0 2024-04-25T09:56:24.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.919+0000 7f7587ad2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75800b97a0 con 0x7f75800b2cb0 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.919+0000 7f7587ad2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758015dfc0 con 0x7f758015dbe0 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.919+0000 7f758606f640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f75800b92a0 0x7f75800b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33548/0 (socket says 172.21.15.19:33548) 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.919+0000 7f758606f640 1 -- 172.21.15.19:0/1825798137 learned_addr learned my addr 172.21.15.19:0/1825798137 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492859724 0 0) 0x7f75800b6220 con 0x7f75800b92a0 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7568003520 con 0x7f75800b92a0 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758586e640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f758015dbe0 0x7f75800d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3967964495 0 0) 0x7f7568003520 con 0x7f75800b92a0 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f758015dbe0 msgr2=0x7f75800d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f758015dbe0 0x7f75800d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 >> v1:172.21.15.112:6789/0 conn(0x7f75800b2cb0 legacy=0x7f75800b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75800d1c50 con 0x7f75800b92a0 2024-04-25T09:56:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f75700019f0 con 0x7f75800b92a0 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758586e640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f758015dbe0 0x7f75800d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 >> v1:172.21.15.19:6789/0 conn(0x7f75800b92a0 legacy=0x7f75800b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75680039e0 0x7f7568003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 0x7f7568004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f758015dbe0 0x7f75680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756800ba90 con 0x7f7568004800 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756800bc40 con 0x7f75680039e0 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756800be40 con 0x7f758015dbe0 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758506d640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 0x7f7568004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758586e640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75680039e0 0x7f7568003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758606f640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f758015dbe0 0x7f75680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758506d640 1 -- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f758015dbe0 msgr2=0x7f75680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758506d640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f758015dbe0 0x7f75680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758506d640 1 -- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75680039e0 msgr2=0x7f7568003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758506d640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75680039e0 0x7f7568003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.920+0000 7f758506d640 1 -- 172.21.15.19:0/1825798137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7568003520 con 0x7f7568004800 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.921+0000 7f758506d640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 0x7f7568004c80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7574001750 tx=0x7f7568005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f042b16197e61d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.921+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574001370 con 0x7f7568004800 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.921+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75740052d0 con 0x7f7568004800 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.921+0000 7f758486c640 1 -- 172.21.15.19:0/1825798137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757402abe0 con 0x7f7568004800 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.922+0000 7f7587ad2640 1 -- 172.21.15.19:0/1825798137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 msgr2=0x7f7568004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.922+0000 7f7587ad2640 1 --2- 172.21.15.19:0/1825798137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 0x7f7568004c80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7574001750 tx=0x7f7568005830 comp rx=0 tx=0).stop 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.922+0000 7f758606f640 1 -- 172.21.15.19:0/1825798137 reap_dead start 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.922+0000 7f7587ad2640 1 -- 172.21.15.19:0/1825798137 shutdown_connections 2024-04-25T09:56:24.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.922+0000 7f7587ad2640 1 -- 172.21.15.19:0/1825798137 >> 172.21.15.19:0/1825798137 conn(0x7f7580061a20 msgr2=0x7f7580061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.922+0000 7f7587ad2640 1 -- 172.21.15.19:0/1825798137 shutdown_connections 2024-04-25T09:56:24.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.922+0000 7f7587ad2640 1 -- 172.21.15.19:0/1825798137 wait complete. 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.923+0000 7f7587ad2640 1 Processor -- start 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.923+0000 7f7587ad2640 1 -- start start 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f7587ad2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 0x7f75801540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f7587ad2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75800b5ae0 0x7f7580156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f7587ad2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75800b5f60 0x7f75801571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f7587ad2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75800d2210 con 0x7f7568004800 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f7587ad2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75800d1a00 con 0x7f75800b5f60 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f7587ad2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75800b6ab0 con 0x7f75800b5ae0 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758586e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 0x7f75801540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758586e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 0x7f75801540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43862/0 (socket says 172.21.15.19:43862) 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758586e640 1 -- 172.21.15.19:0/32047326 learned_addr learned my addr 172.21.15.19:0/32047326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758606f640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75800b5f60 0x7f75801571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758506d640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75800b5ae0 0x7f7580156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758586e640 1 -- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75800b5ae0 msgr2=0x7f7580156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758586e640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75800b5ae0 0x7f7580156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758586e640 1 -- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75800b5f60 msgr2=0x7f75801571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758586e640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75800b5f60 0x7f75801571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758586e640 1 -- 172.21.15.19:0/32047326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7580154610 con 0x7f7568004800 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758506d640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75800b5ae0 0x7f7580156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.924+0000 7f758606f640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75800b5f60 0x7f75801571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.925+0000 7f758586e640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 0x7f75801540d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f75780115c0 tx=0x7f7578011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.925+0000 7f757e7fc640 1 -- 172.21.15.19:0/32047326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578019070 con 0x7f7568004800 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.925+0000 7f757e7fc640 1 -- 172.21.15.19:0/32047326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7578004220 con 0x7f7568004800 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.925+0000 7f7587ad2640 1 -- 172.21.15.19:0/32047326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758015ee40 con 0x7f7568004800 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.925+0000 7f757e7fc640 1 -- 172.21.15.19:0/32047326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757800f040 con 0x7f7568004800 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.925+0000 7f7587ad2640 1 -- 172.21.15.19:0/32047326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758015f400 con 0x7f7568004800 2024-04-25T09:56:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.926+0000 7f7587ad2640 1 -- 172.21.15.19:0/32047326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7580073990 con 0x7f7568004800 2024-04-25T09:56:24.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.926+0000 7f757e7fc640 1 -- 172.21.15.19:0/32047326 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f757800f6f0 con 0x7f7568004800 2024-04-25T09:56:24.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.927+0000 7f757e7fc640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756c06b4d0 0x7f756c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.927+0000 7f757e7fc640 1 -- 172.21.15.19:0/32047326 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f757808a690 con 0x7f7568004800 2024-04-25T09:56:24.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.927+0000 7f758506d640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756c06b4d0 0x7f756c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.927+0000 7f758506d640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756c06b4d0 0x7f756c06d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7574032000 tx=0x7f757402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:24.931+0000 7f757e7fc640 1 -- 172.21.15.19:0/32047326 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75780592f0 con 0x7f7568004800 2024-04-25T09:56:25.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.152+0000 7f7587ad2640 1 -- 172.21.15.19:0/32047326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75800d3340 con 0x7f7568004800 2024-04-25T09:56:25.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.153+0000 7f757e7fc640 1 -- 172.21.15.19:0/32047326 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f7578058bf0 con 0x7f7568004800 2024-04-25T09:56:25.156 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:25.156 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:56:25.030125+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:25.156 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-04-25T09:56:25.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 -- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756c06b4d0 msgr2=0x7f756c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756c06b4d0 0x7f756c06d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7574032000 tx=0x7f757402e000 comp rx=0 tx=0).stop 2024-04-25T09:56:25.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 -- 172.21.15.19:0/32047326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 msgr2=0x7f75801540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 0x7f75801540d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f75780115c0 tx=0x7f7578011a80 comp rx=0 tx=0).stop 2024-04-25T09:56:25.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 -- 172.21.15.19:0/32047326 shutdown_connections 2024-04-25T09:56:25.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756c06b4d0 0x7f756c06d990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75800b5f60 0x7f75801571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75800b5ae0 0x7f7580156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 --2- 172.21.15.19:0/32047326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7568004800 0x7f75801540d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 -- 172.21.15.19:0/32047326 >> 172.21.15.19:0/32047326 conn(0x7f7580061a20 msgr2=0x7f758015aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:25.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 -- 172.21.15.19:0/32047326 shutdown_connections 2024-04-25T09:56:25.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.155+0000 7f7587ad2640 1 -- 172.21.15.19:0/32047326 wait complete. 2024-04-25T09:56:25.168 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:25.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.266+0000 7fb733ac4640 1 Processor -- start 2024-04-25T09:56:25.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.266+0000 7fb733ac4640 1 -- start start 2024-04-25T09:56:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb733ac4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb72c0cc6c0 0x7fb72c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb733ac4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb72c15b320 0x7fb72c071770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb733ac4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72c071ed0 con 0x7fb72c13d190 2024-04-25T09:56:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb733ac4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72c071cb0 con 0x7fb72c15b320 2024-04-25T09:56:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb733ac4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c13d570 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb732061640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb72c15b320 0x7fb72c071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb732061640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb72c15b320 0x7fb72c071770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43866/0 (socket says 172.21.15.19:43866) 2024-04-25T09:56:25.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb732061640 1 -- 172.21.15.19:0/2208586189 learned_addr learned my addr 172.21.15.19:0/2208586189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:25.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb732061640 1 -- 172.21.15.19:0/2208586189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb72c0cc6c0 msgr2=0x7fb72c071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:25.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb73105f640 1 --2- 172.21.15.19:0/2208586189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb72c0cc6c0 0x7fb72c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb732061640 1 --2- 172.21.15.19:0/2208586189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb72c0cc6c0 0x7fb72c071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb732061640 1 -- 172.21.15.19:0/2208586189 >> v1:172.21.15.112:6789/0 conn(0x7fb72c13d190 legacy=0x7fb72c0cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:25.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.267+0000 7fb732061640 1 -- 172.21.15.19:0/2208586189 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c15a840 con 0x7fb72c15b320 2024-04-25T09:56:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.268+0000 7fb732061640 1 --2- 172.21.15.19:0/2208586189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb72c15b320 0x7fb72c071770 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb724009e50 tx=0x7fb72402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d445ae1f0455198 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.268+0000 7fb73085e640 1 -- 172.21.15.19:0/2208586189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724036070 con 0x7fb72c15b320 2024-04-25T09:56:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.268+0000 7fb73085e640 1 -- 172.21.15.19:0/2208586189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb724031600 con 0x7fb72c15b320 2024-04-25T09:56:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.268+0000 7fb73085e640 1 -- 172.21.15.19:0/2208586189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240319d0 con 0x7fb72c15b320 2024-04-25T09:56:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.269+0000 7fb733ac4640 1 -- 172.21.15.19:0/2208586189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb72c15b320 msgr2=0x7fb72c071770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.269+0000 7fb733ac4640 1 --2- 172.21.15.19:0/2208586189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb72c15b320 0x7fb72c071770 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb724009e50 tx=0x7fb72402a630 comp rx=0 tx=0).stop 2024-04-25T09:56:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.269+0000 7fb733ac4640 1 -- 172.21.15.19:0/2208586189 shutdown_connections 2024-04-25T09:56:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.269+0000 7fb733ac4640 1 --2- 172.21.15.19:0/2208586189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb72c15b320 0x7fb72c071770 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.269+0000 7fb733ac4640 1 --2- 172.21.15.19:0/2208586189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb72c0cc6c0 0x7fb72c071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.269+0000 7fb733ac4640 1 -- 172.21.15.19:0/2208586189 >> 172.21.15.19:0/2208586189 conn(0x7fb72c06abe0 msgr2=0x7fb72c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.269+0000 7fb733ac4640 1 -- 172.21.15.19:0/2208586189 shutdown_connections 2024-04-25T09:56:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.269+0000 7fb733ac4640 1 -- 172.21.15.19:0/2208586189 wait complete. 2024-04-25T09:56:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.270+0000 7fb733ac4640 1 Processor -- start 2024-04-25T09:56:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.270+0000 7fb733ac4640 1 -- start start 2024-04-25T09:56:25.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb733ac4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb72c0cc6c0 0x7fb72c0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb733ac4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb72c13d190 0x7fb72c1398e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb733ac4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb72c15b320 0x7fb72c14f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb733ac4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0cf620 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb733ac4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c15ae00 con 0x7fb72c15b320 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb733ac4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0cf7a0 con 0x7fb72c13d190 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb731860640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb72c0cc6c0 0x7fb72c0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb731860640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb72c0cc6c0 0x7fb72c0af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43874/0 (socket says 172.21.15.19:43874) 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb731860640 1 -- 172.21.15.19:0/2912784629 learned_addr learned my addr 172.21.15.19:0/2912784629 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb732061640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb72c15b320 0x7fb72c14f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb73105f640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb72c13d190 0x7fb72c1398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb731860640 1 -- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb72c13d190 msgr2=0x7fb72c1398e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb731860640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb72c13d190 0x7fb72c1398e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb731860640 1 -- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb72c15b320 msgr2=0x7fb72c14f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb731860640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb72c15b320 0x7fb72c14f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb731860640 1 -- 172.21.15.19:0/2912784629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c139e20 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb732061640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb72c15b320 0x7fb72c14f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb73105f640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb72c13d190 0x7fb72c1398e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb731860640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb72c0cc6c0 0x7fb72c0af2c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb72800c460 tx=0x7fb72800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb7227fc640 1 -- 172.21.15.19:0/2912784629 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728016020 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb7227fc640 1 -- 172.21.15.19:0/2912784629 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb728012870 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb733ac4640 1 -- 172.21.15.19:0/2912784629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72c158430 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.271+0000 7fb7227fc640 1 -- 172.21.15.19:0/2912784629 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728012c60 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.272+0000 7fb733ac4640 1 -- 172.21.15.19:0/2912784629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72c1589f0 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.272+0000 7fb733ac4640 1 -- 172.21.15.19:0/2912784629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c0b0ff0 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.273+0000 7fb7227fc640 1 -- 172.21.15.19:0/2912784629 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb728019020 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.273+0000 7fb7227fc640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb71006b4d0 0x7fb71006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.273+0000 7fb7227fc640 1 -- 172.21.15.19:0/2912784629 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb72808a040 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.273+0000 7fb73105f640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb71006b4d0 0x7fb71006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.274+0000 7fb73105f640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb71006b4d0 0x7fb71006d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb72c14eeb0 tx=0x7fb71c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:25.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.277+0000 7fb7227fc640 1 -- 172.21.15.19:0/2912784629 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb728020030 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.460+0000 7fb733ac4640 1 -- 172.21.15.19:0/2912784629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb72c073990 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.461+0000 7fb7227fc640 1 -- 172.21.15.19:0/2912784629 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16930 (secure 0 0 0) 0x7fb7280182c0 con 0x7fb72c0cc6c0 2024-04-25T09:56:25.465 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:25.465 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":53,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:53.990243+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:50.618180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:51.322489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 -- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb71006b4d0 msgr2=0x7fb71006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb71006b4d0 0x7fb71006d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb72c14eeb0 tx=0x7fb71c008040 comp rx=0 tx=0).stop 2024-04-25T09:56:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 -- 172.21.15.19:0/2912784629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb72c0cc6c0 msgr2=0x7fb72c0af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb72c0cc6c0 0x7fb72c0af2c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb72800c460 tx=0x7fb72800c920 comp rx=0 tx=0).stop 2024-04-25T09:56:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 -- 172.21.15.19:0/2912784629 shutdown_connections 2024-04-25T09:56:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb71006b4d0 0x7fb71006d990 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb72c15b320 0x7fb72c14f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb72c13d190 0x7fb72c1398e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 --2- 172.21.15.19:0/2912784629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb72c0cc6c0 0x7fb72c0af2c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 -- 172.21.15.19:0/2912784629 >> 172.21.15.19:0/2912784629 conn(0x7fb72c06abe0 msgr2=0x7fb72c1579b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:25.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 -- 172.21.15.19:0/2912784629 shutdown_connections 2024-04-25T09:56:25.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.464+0000 7fb733ac4640 1 -- 172.21.15.19:0/2912784629 wait complete. 2024-04-25T09:56:25.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:55:50.618180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:55:51.322489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:56:25.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:56:25.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.578+0000 7f6f64f30640 1 Processor -- start 2024-04-25T09:56:25.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.578+0000 7f6f64f30640 1 -- start start 2024-04-25T09:56:25.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.579+0000 7f6f64f30640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015b7d0 0x7f6f600cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.579+0000 7f6f64f30640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6006cf50 con 0x7f6f600b5620 2024-04-25T09:56:25.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.579+0000 7f6f64f30640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f600ce530 con 0x7f6f6006c9e0 2024-04-25T09:56:25.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.579+0000 7f6f64f30640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600ce810 con 0x7f6f6015b7d0 2024-04-25T09:56:25.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.579+0000 7f6f5dd9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6f600b5620 0x7f6f6006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33562/0 (socket says 172.21.15.19:33562) 2024-04-25T09:56:25.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.579+0000 7f6f5dd9b640 1 -- 172.21.15.19:0/2444318275 learned_addr learned my addr 172.21.15.19:0/2444318275 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:25.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.579+0000 7f6f5e59c640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015b7d0 0x7f6f600cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.579+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044199789 0 0) 0x7f6f6006cf50 con 0x7f6f600b5620 2024-04-25T09:56:25.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.579+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f48003520 con 0x7f6f600b5620 2024-04-25T09:56:25.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517026239 0 0) 0x7f6f600ce530 con 0x7f6f6006c9e0 2024-04-25T09:56:25.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f6006cf50 con 0x7f6f6006c9e0 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 310593740 0 0) 0x7f6f48003520 con 0x7f6f600b5620 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015b7d0 msgr2=0x7f6f600cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015b7d0 0x7f6f600cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 >> v1:172.21.15.112:6789/0 conn(0x7f6f6006c9e0 legacy=0x7f6f600d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f600ceaf0 con 0x7f6f600b5620 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f6f54001670 con 0x7f6f600b5620 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 >> v1:172.21.15.19:6789/0 conn(0x7f6f600b5620 legacy=0x7f6f6006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015b7d0 0x7f6f48003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f480040d0 0x7f6f48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48004d40 0x7f6f480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600ce810 con 0x7f6f480040d0 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4800baa0 con 0x7f6f48004d40 2024-04-25T09:56:25.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4800bca0 con 0x7f6f6015b7d0 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5dd9b640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f480040d0 0x7f6f48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5e59c640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015b7d0 0x7f6f48003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5ed9d640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48004d40 0x7f6f480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5dd9b640 1 -- 172.21.15.19:0/2444318275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015b7d0 msgr2=0x7f6f48003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5dd9b640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015b7d0 0x7f6f48003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5dd9b640 1 -- 172.21.15.19:0/2444318275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48004d40 msgr2=0x7f6f480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5dd9b640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48004d40 0x7f6f480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.580+0000 7f6f5dd9b640 1 -- 172.21.15.19:0/2444318275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f48003520 con 0x7f6f480040d0 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.581+0000 7f6f5dd9b640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f480040d0 0x7f6f48004800 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400c210 tx=0x7f6f48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51b837ace3f6e0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.581+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54007ce0 con 0x7f6f480040d0 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.581+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f54005ad0 con 0x7f6f480040d0 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.581+0000 7f6f5d59a640 1 -- 172.21.15.19:0/2444318275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54007070 con 0x7f6f480040d0 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.582+0000 7f6f64f30640 1 -- 172.21.15.19:0/2444318275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f480040d0 msgr2=0x7f6f48004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.582+0000 7f6f64f30640 1 --2- 172.21.15.19:0/2444318275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f480040d0 0x7f6f48004800 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400c210 tx=0x7f6f48005830 comp rx=0 tx=0).stop 2024-04-25T09:56:25.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.582+0000 7f6f5ed9d640 1 -- 172.21.15.19:0/2444318275 reap_dead start 2024-04-25T09:56:25.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.582+0000 7f6f64f30640 1 -- 172.21.15.19:0/2444318275 shutdown_connections 2024-04-25T09:56:25.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.582+0000 7f6f64f30640 1 -- 172.21.15.19:0/2444318275 >> 172.21.15.19:0/2444318275 conn(0x7f6f600d0800 msgr2=0x7f6f600d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:25.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.582+0000 7f6f64f30640 1 -- 172.21.15.19:0/2444318275 shutdown_connections 2024-04-25T09:56:25.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.582+0000 7f6f64f30640 1 -- 172.21.15.19:0/2444318275 wait complete. 2024-04-25T09:56:25.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.583+0000 7f6f64f30640 1 Processor -- start 2024-04-25T09:56:25.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.583+0000 7f6f64f30640 1 -- start start 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f64f30640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f480040d0 0x7f6f601e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f64f30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f60157fc0 0x7f6f6015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f64f30640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015c3f0 0x7f6f600cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f64f30640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6015bd30 con 0x7f6f60157fc0 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f64f30640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6015bbb0 con 0x7f6f480040d0 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f64f30640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600d2650 con 0x7f6f6015c3f0 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f5dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f60157fc0 0x7f6f6015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f5dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f60157fc0 0x7f6f6015beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43886/0 (socket says 172.21.15.19:43886) 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f5dd9b640 1 -- 172.21.15.19:0/2517988176 learned_addr learned my addr 172.21.15.19:0/2517988176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f5e59c640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f480040d0 0x7f6f601e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f5ed9d640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015c3f0 0x7f6f600cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f5dd9b640 1 -- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015c3f0 msgr2=0x7f6f600cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f5dd9b640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015c3f0 0x7f6f600cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f5dd9b640 1 -- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f480040d0 msgr2=0x7f6f601e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f5dd9b640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f480040d0 0x7f6f601e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f5dd9b640 1 -- 172.21.15.19:0/2517988176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f600cc330 con 0x7f6f60157fc0 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.584+0000 7f6f5dd9b640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f60157fc0 0x7f6f6015beb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400c210 tx=0x7f6f600cecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.585+0000 7f6f4effd640 1 -- 172.21.15.19:0/2517988176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54001670 con 0x7f6f60157fc0 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.585+0000 7f6f4effd640 1 -- 172.21.15.19:0/2517988176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f54042070 con 0x7f6f60157fc0 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.585+0000 7f6f64f30640 1 -- 172.21.15.19:0/2517988176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f600cc5d0 con 0x7f6f60157fc0 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.585+0000 7f6f4effd640 1 -- 172.21.15.19:0/2517988176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54007290 con 0x7f6f60157fc0 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.585+0000 7f6f64f30640 1 -- 172.21.15.19:0/2517988176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f600ccb90 con 0x7f6f60157fc0 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.586+0000 7f6f64f30640 1 -- 172.21.15.19:0/2517988176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f60073a00 con 0x7f6f60157fc0 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.586+0000 7f6f4effd640 1 -- 172.21.15.19:0/2517988176 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f54007430 con 0x7f6f60157fc0 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.586+0000 7f6f4effd640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f4006b4d0 0x7f6f4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.587+0000 7f6f4effd640 1 -- 172.21.15.19:0/2517988176 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6f540aa320 con 0x7f6f60157fc0 2024-04-25T09:56:25.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.587+0000 7f6f5e59c640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f4006b4d0 0x7f6f4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.587+0000 7f6f5e59c640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f4006b4d0 0x7f6f4006d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6f600b5020 tx=0x7f6f50009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:25.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.591+0000 7f6f4effd640 1 -- 172.21.15.19:0/2517988176 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f54005aa0 con 0x7f6f60157fc0 2024-04-25T09:56:25.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.770+0000 7f6f64f30640 1 -- 172.21.15.19:0/2517988176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f6015f1d0 con 0x7f6f60157fc0 2024-04-25T09:56:25.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.770+0000 7f6f4effd640 1 -- 172.21.15.19:0/2517988176 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f540790a0 con 0x7f6f60157fc0 2024-04-25T09:56:25.773 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:56:25.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 -- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f4006b4d0 msgr2=0x7f6f4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f4006b4d0 0x7f6f4006d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6f600b5020 tx=0x7f6f50009190 comp rx=0 tx=0).stop 2024-04-25T09:56:25.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 -- 172.21.15.19:0/2517988176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f60157fc0 msgr2=0x7f6f6015beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f60157fc0 0x7f6f6015beb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400c210 tx=0x7f6f600cecc0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 -- 172.21.15.19:0/2517988176 shutdown_connections 2024-04-25T09:56:25.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f4006b4d0 0x7f6f4006d990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f6015c3f0 0x7f6f600cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f60157fc0 0x7f6f6015beb0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 --2- 172.21.15.19:0/2517988176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f480040d0 0x7f6f601e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 -- 172.21.15.19:0/2517988176 >> 172.21.15.19:0/2517988176 conn(0x7f6f600d0800 msgr2=0x7f6f600d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:25.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 -- 172.21.15.19:0/2517988176 shutdown_connections 2024-04-25T09:56:25.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.772+0000 7f6f64f30640 1 -- 172.21.15.19:0/2517988176 wait complete. 2024-04-25T09:56:25.786 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:56:25.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.886+0000 7fdd8bd5c640 1 Processor -- start 2024-04-25T09:56:25.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.886+0000 7fdd8bd5c640 1 -- start start 2024-04-25T09:56:25.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd8bd5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8415dc20 0x7fdd840ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd8bd5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdd840d32f0 0x7fdd840cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd8bd5c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8415e000 con 0x7fdd84161710 2024-04-25T09:56:25.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd8bd5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd840d1de0 con 0x7fdd840d32f0 2024-04-25T09:56:25.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd8bd5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84161c60 con 0x7fdd8415dc20 2024-04-25T09:56:25.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd892f7640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdd84161710 0x7fdd8406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33570/0 (socket says 172.21.15.19:33570) 2024-04-25T09:56:25.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd892f7640 1 -- 172.21.15.19:0/2408423978 learned_addr learned my addr 172.21.15.19:0/2408423978 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:25.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd8a2f9640 1 --2- 172.21.15.19:0/2408423978 >> v2:172.21.15.112:3300/0 conn(0x7fdd840d32f0 0x7fdd840cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040329177 0 0) 0x7fdd8415e000 con 0x7fdd84161710 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd89af8640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8415dc20 0x7fdd840ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd70003520 con 0x7fdd84161710 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 141599606 0 0) 0x7fdd70003520 con 0x7fdd84161710 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8415dc20 msgr2=0x7fdd840ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd88af6640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8415dc20 0x7fdd840ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 >> v2:172.21.15.112:3300/0 conn(0x7fdd840d32f0 msgr2=0x7fdd840cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd88af6640 1 --2- 172.21.15.19:0/2408423978 >> v2:172.21.15.112:3300/0 conn(0x7fdd840d32f0 0x7fdd840cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84161f40 con 0x7fdd84161710 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fdd74001670 con 0x7fdd84161710 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.887+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 >> v1:172.21.15.19:6789/0 conn(0x7fdd84161710 legacy=0x7fdd8406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd88af6640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd700039b0 0x7fdd70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd88af6640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8415dc20 0x7fdd70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd88af6640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd70004d40 0x7fdd700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840d1de0 con 0x7fdd70004d40 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84161c60 con 0x7fdd700039b0 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7000b9c0 con 0x7fdd8415dc20 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd8a2f9640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd70004d40 0x7fdd700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd89af8640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd700039b0 0x7fdd70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd892f7640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8415dc20 0x7fdd70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd8a2f9640 1 -- 172.21.15.19:0/2408423978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8415dc20 msgr2=0x7fdd70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd8a2f9640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd8415dc20 0x7fdd70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd8a2f9640 1 -- 172.21.15.19:0/2408423978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd700039b0 msgr2=0x7fdd70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd8a2f9640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd700039b0 0x7fdd70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.888+0000 7fdd8a2f9640 1 -- 172.21.15.19:0/2408423978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd70003520 con 0x7fdd70004d40 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.889+0000 7fdd8a2f9640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd70004d40 0x7fdd700051c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009f90 tx=0x7fdd70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fed62c7b43f8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.889+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c009c10 con 0x7fdd70004d40 2024-04-25T09:56:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.889+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c036070 con 0x7fdd70004d40 2024-04-25T09:56:25.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.889+0000 7fdd88af6640 1 -- 172.21.15.19:0/2408423978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c031840 con 0x7fdd70004d40 2024-04-25T09:56:25.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.889+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/2408423978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd70004d40 msgr2=0x7fdd700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.889+0000 7fdd8bd5c640 1 --2- 172.21.15.19:0/2408423978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd70004d40 0x7fdd700051c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009f90 tx=0x7fdd70005830 comp rx=0 tx=0).stop 2024-04-25T09:56:25.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.889+0000 7fdd8a2f9640 1 -- 172.21.15.19:0/2408423978 reap_dead start 2024-04-25T09:56:25.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.889+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/2408423978 shutdown_connections 2024-04-25T09:56:25.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.889+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/2408423978 >> 172.21.15.19:0/2408423978 conn(0x7fdd840d0420 msgr2=0x7fdd840d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:25.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.890+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/2408423978 shutdown_connections 2024-04-25T09:56:25.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.890+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/2408423978 wait complete. 2024-04-25T09:56:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.891+0000 7fdd8bd5c640 1 Processor -- start 2024-04-25T09:56:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.891+0000 7fdd8bd5c640 1 -- start start 2024-04-25T09:56:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd8bd5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd84157ce0 0x7fdd840c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd8bd5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd841580c0 0x7fdd8415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd8bd5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd840c9f80 0x7fdd8415cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd8bd5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840b5e50 con 0x7fdd841580c0 2024-04-25T09:56:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd8bd5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84162500 con 0x7fdd840c9f80 2024-04-25T09:56:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd8bd5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840b5fd0 con 0x7fdd84157ce0 2024-04-25T09:56:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd892f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd841580c0 0x7fdd8415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd892f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd841580c0 0x7fdd8415c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43912/0 (socket says 172.21.15.19:43912) 2024-04-25T09:56:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd892f7640 1 -- 172.21.15.19:0/3603446010 learned_addr learned my addr 172.21.15.19:0/3603446010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd892f7640 1 -- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd84157ce0 msgr2=0x7fdd840c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd892f7640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd84157ce0 0x7fdd840c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd892f7640 1 -- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd840c9f80 msgr2=0x7fdd8415cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd892f7640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd840c9f80 0x7fdd8415cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd892f7640 1 -- 172.21.15.19:0/3603446010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd840c7570 con 0x7fdd841580c0 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd892f7640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd841580c0 0x7fdd8415c9c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fdd740072c0 tx=0x7fdd74007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd7a7fc640 1 -- 172.21.15.19:0/3603446010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7400fa00 con 0x7fdd841580c0 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd7a7fc640 1 -- 172.21.15.19:0/3603446010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd74018070 con 0x7fdd841580c0 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/3603446010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd840c7850 con 0x7fdd841580c0 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd7a7fc640 1 -- 172.21.15.19:0/3603446010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7400fe20 con 0x7fdd841580c0 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.892+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/3603446010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd840c7e10 con 0x7fdd841580c0 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.893+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/3603446010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd84073a00 con 0x7fdd841580c0 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.894+0000 7fdd7a7fc640 1 -- 172.21.15.19:0/3603446010 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd7401d460 con 0x7fdd841580c0 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.894+0000 7fdd7a7fc640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6806b4d0 0x7fdd6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.894+0000 7fdd7a7fc640 1 -- 172.21.15.19:0/3603446010 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd7408a510 con 0x7fdd841580c0 2024-04-25T09:56:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.895+0000 7fdd89af8640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6806b4d0 0x7fdd6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.895+0000 7fdd89af8640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6806b4d0 0x7fdd6806d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009a40 tx=0x7fdd80009320 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:25.899+0000 7fdd7a7fc640 1 -- 172.21.15.19:0/3603446010 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd74059200 con 0x7fdd841580c0 2024-04-25T09:56:26.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.076+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/3603446010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd840d16b0 con 0x7fdd841580c0 2024-04-25T09:56:26.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.076+0000 7fdd7a7fc640 1 -- 172.21.15.19:0/3603446010 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd74058b00 con 0x7fdd841580c0 2024-04-25T09:56:26.079 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:56:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6806b4d0 msgr2=0x7fdd6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6806b4d0 0x7fdd6806d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009a40 tx=0x7fdd80009320 comp rx=0 tx=0).stop 2024-04-25T09:56:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/3603446010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd841580c0 msgr2=0x7fdd8415c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd841580c0 0x7fdd8415c9c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fdd740072c0 tx=0x7fdd74007780 comp rx=0 tx=0).stop 2024-04-25T09:56:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/3603446010 shutdown_connections 2024-04-25T09:56:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd6806b4d0 0x7fdd6806d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd840c9f80 0x7fdd8415cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd841580c0 0x7fdd8415c9c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 --2- 172.21.15.19:0/3603446010 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd84157ce0 0x7fdd840c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/3603446010 >> 172.21.15.19:0/3603446010 conn(0x7fdd840d0420 msgr2=0x7fdd840b8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/3603446010 shutdown_connections 2024-04-25T09:56:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.078+0000 7fdd8bd5c640 1 -- 172.21.15.19:0/3603446010 wait complete. 2024-04-25T09:56:26.092 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:56:26.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.192+0000 7f16931e2640 1 Processor -- start 2024-04-25T09:56:26.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.192+0000 7f16931e2640 1 -- start start 2024-04-25T09:56:26.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f16931e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f168c06abe0 0x7f168c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f16931e2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f168c0ad570 0x7f168c06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f16931e2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168c06b500 con 0x7f168c15dd80 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f16931e2640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168c06e410 con 0x7f168c0ad570 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f16931e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c0b3930 con 0x7f168c06abe0 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f169177f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f168c0ad570 0x7f168c06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f168bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f168c15dd80 0x7f168c0b5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33580/0 (socket says 172.21.15.19:33580) 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f168bfff640 1 -- 172.21.15.19:0/1730387448 learned_addr learned my addr 172.21.15.19:0/1730387448 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f1690f7e640 1 --2- 172.21.15.19:0/1730387448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f168c06abe0 0x7f168c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f169177f640 1 -- 172.21.15.19:0/1730387448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f168c06abe0 msgr2=0x7f168c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f169177f640 1 --2- 172.21.15.19:0/1730387448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f168c06abe0 0x7f168c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f169177f640 1 -- 172.21.15.19:0/1730387448 >> v1:172.21.15.19:6789/0 conn(0x7f168c15dd80 legacy=0x7f168c0b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f168b7fe640 1 -- 172.21.15.19:0/1730387448 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855955642 0 0) 0x7f168c06b500 con 0x7f168c15dd80 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f169177f640 1 -- 172.21.15.19:0/1730387448 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c0b3c10 con 0x7f168c0ad570 2024-04-25T09:56:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.193+0000 7f1690f7e640 1 --2- 172.21.15.19:0/1730387448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f168c06abe0 0x7f168c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.194+0000 7f169177f640 1 --2- 172.21.15.19:0/1730387448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f168c0ad570 0x7f168c06ee60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1684009d90 tx=0x7f168402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fff4bb3c38b5d48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.194+0000 7f168b7fe640 1 -- 172.21.15.19:0/1730387448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1684007740 con 0x7f168c0ad570 2024-04-25T09:56:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.194+0000 7f168b7fe640 1 -- 172.21.15.19:0/1730387448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16840078e0 con 0x7f168c0ad570 2024-04-25T09:56:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.194+0000 7f168b7fe640 1 -- 172.21.15.19:0/1730387448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1684007cb0 con 0x7f168c0ad570 2024-04-25T09:56:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.195+0000 7f16931e2640 1 -- 172.21.15.19:0/1730387448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f168c0ad570 msgr2=0x7f168c06ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.195+0000 7f16931e2640 1 --2- 172.21.15.19:0/1730387448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f168c0ad570 0x7f168c06ee60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1684009d90 tx=0x7f168402a660 comp rx=0 tx=0).stop 2024-04-25T09:56:26.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.195+0000 7f16931e2640 1 -- 172.21.15.19:0/1730387448 shutdown_connections 2024-04-25T09:56:26.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.195+0000 7f16931e2640 1 --2- 172.21.15.19:0/1730387448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f168c0ad570 0x7f168c06ee60 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.195+0000 7f16931e2640 1 --2- 172.21.15.19:0/1730387448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f168c06abe0 0x7f168c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.195+0000 7f16931e2640 1 -- 172.21.15.19:0/1730387448 >> 172.21.15.19:0/1730387448 conn(0x7f168c0b5af0 msgr2=0x7f168c0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:26.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.195+0000 7f16931e2640 1 -- 172.21.15.19:0/1730387448 shutdown_connections 2024-04-25T09:56:26.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.195+0000 7f16931e2640 1 -- 172.21.15.19:0/1730387448 wait complete. 2024-04-25T09:56:26.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.196+0000 7f16931e2640 1 Processor -- start 2024-04-25T09:56:26.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.196+0000 7f16931e2640 1 -- start start 2024-04-25T09:56:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f16931e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f168c06abe0 0x7f168c13c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f16931e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c0ad570 0x7f168c15c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f16931e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f168c15dd80 0x7f168c15c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f16931e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c070280 con 0x7f168c15dd80 2024-04-25T09:56:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f16931e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c06e940 con 0x7f168c0ad570 2024-04-25T09:56:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f16931e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c070400 con 0x7f168c06abe0 2024-04-25T09:56:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f169177f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f168c15dd80 0x7f168c15c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f169177f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f168c15dd80 0x7f168c15c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43922/0 (socket says 172.21.15.19:43922) 2024-04-25T09:56:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f169177f640 1 -- 172.21.15.19:0/1266600447 learned_addr learned my addr 172.21.15.19:0/1266600447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f1690f7e640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f168c06abe0 0x7f168c13c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f168bfff640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c0ad570 0x7f168c15c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f169177f640 1 -- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f168c06abe0 msgr2=0x7f168c13c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f169177f640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f168c06abe0 0x7f168c13c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f169177f640 1 -- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c0ad570 msgr2=0x7f168c15c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f169177f640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c0ad570 0x7f168c15c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f169177f640 1 -- 172.21.15.19:0/1266600447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c160630 con 0x7f168c15dd80 2024-04-25T09:56:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.197+0000 7f168bfff640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c0ad570 0x7f168c15c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.198+0000 7f169177f640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f168c15dd80 0x7f168c15c900 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1684009eb0 tx=0x7f1684004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.198+0000 7f16897fa640 1 -- 172.21.15.19:0/1266600447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1684040070 con 0x7f168c15dd80 2024-04-25T09:56:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.198+0000 7f16897fa640 1 -- 172.21.15.19:0/1266600447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1684031bc0 con 0x7f168c15dd80 2024-04-25T09:56:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.198+0000 7f16931e2640 1 -- 172.21.15.19:0/1266600447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168c160910 con 0x7f168c15dd80 2024-04-25T09:56:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.198+0000 7f16897fa640 1 -- 172.21.15.19:0/1266600447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168403c420 con 0x7f168c15dd80 2024-04-25T09:56:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.198+0000 7f16931e2640 1 -- 172.21.15.19:0/1266600447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168c160ed0 con 0x7f168c15dd80 2024-04-25T09:56:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.199+0000 7f16931e2640 1 -- 172.21.15.19:0/1266600447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168c15d930 con 0x7f168c15dd80 2024-04-25T09:56:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.200+0000 7f16897fa640 1 -- 172.21.15.19:0/1266600447 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1684039040 con 0x7f168c15dd80 2024-04-25T09:56:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.200+0000 7f16897fa640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b500 0x7f167006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.200+0000 7f16897fa640 1 -- 172.21.15.19:0/1266600447 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16840ab7e0 con 0x7f168c15dd80 2024-04-25T09:56:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.200+0000 7f1690f7e640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b500 0x7f167006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.201+0000 7f1690f7e640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b500 0x7f167006d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1680006fd0 tx=0x7f1680008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:26.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.204+0000 7f16897fa640 1 -- 172.21.15.19:0/1266600447 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f168c15d930 con 0x7f168c15dd80 2024-04-25T09:56:26.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.388+0000 7f16931e2640 1 -- 172.21.15.19:0/1266600447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f168c073a00 con 0x7f168c15dd80 2024-04-25T09:56:26.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.388+0000 7f16897fa640 1 -- 172.21.15.19:0/1266600447 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f168407a4d0 con 0x7f168c15dd80 2024-04-25T09:56:26.392 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:56:26.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 -- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b500 msgr2=0x7f167006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b500 0x7f167006d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1680006fd0 tx=0x7f1680008040 comp rx=0 tx=0).stop 2024-04-25T09:56:26.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 -- 172.21.15.19:0/1266600447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f168c15dd80 msgr2=0x7f168c15c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f168c15dd80 0x7f168c15c900 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1684009eb0 tx=0x7f1684004a40 comp rx=0 tx=0).stop 2024-04-25T09:56:26.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 -- 172.21.15.19:0/1266600447 shutdown_connections 2024-04-25T09:56:26.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b500 0x7f167006d9c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f168c15dd80 0x7f168c15c900 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168c0ad570 0x7f168c15c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 --2- 172.21.15.19:0/1266600447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f168c06abe0 0x7f168c13c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 -- 172.21.15.19:0/1266600447 >> 172.21.15.19:0/1266600447 conn(0x7f168c0b5af0 msgr2=0x7f168c0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:26.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 -- 172.21.15.19:0/1266600447 shutdown_connections 2024-04-25T09:56:26.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.391+0000 7f16931e2640 1 -- 172.21.15.19:0/1266600447 wait complete. 2024-04-25T09:56:26.404 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:26.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.501+0000 7f2a8f14c640 1 Processor -- start 2024-04-25T09:56:26.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.502+0000 7f2a8f14c640 1 -- start start 2024-04-25T09:56:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.502+0000 7f2a8f14c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8815dc70 0x7f2a880d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.502+0000 7f2a8f14c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2a8809a450 0x7f2a880cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.502+0000 7f2a8f14c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8806c220 con 0x7f2a8806c700 2024-04-25T09:56:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.502+0000 7f2a8f14c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8809a8b0 con 0x7f2a8809a450 2024-04-25T09:56:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.502+0000 7f2a8f14c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880d0c90 con 0x7f2a8815dc70 2024-04-25T09:56:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.503+0000 7f2a8cee8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8815dc70 0x7f2a880d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.503+0000 7f2a8cee8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8815dc70 0x7f2a880d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:52486/0 (socket says 172.21.15.19:52486) 2024-04-25T09:56:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.503+0000 7f2a8cee8640 1 -- 172.21.15.19:0/1683146997 learned_addr learned my addr 172.21.15.19:0/1683146997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.503+0000 7f2a87fff640 1 --2- 172.21.15.19:0/1683146997 >> v2:172.21.15.112:3300/0 conn(0x7f2a8809a450 0x7f2a880cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.503+0000 7f2a8cee8640 1 -- 172.21.15.19:0/1683146997 >> v2:172.21.15.112:3300/0 conn(0x7f2a8809a450 msgr2=0x7f2a880cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.503+0000 7f2a8cee8640 1 --2- 172.21.15.19:0/1683146997 >> v2:172.21.15.112:3300/0 conn(0x7f2a8809a450 0x7f2a880cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.503+0000 7f2a8cee8640 1 -- 172.21.15.19:0/1683146997 >> v1:172.21.15.112:6789/0 conn(0x7f2a8806c700 legacy=0x7f2a880cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.503+0000 7f2a8cee8640 1 -- 172.21.15.19:0/1683146997 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a880d0080 con 0x7f2a8815dc70 2024-04-25T09:56:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.504+0000 7f2a8cee8640 1 --2- 172.21.15.19:0/1683146997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8815dc70 0x7f2a880d1120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009870 tx=0x7f2a7c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa5e871f9e7d387 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.504+0000 7f2a877fe640 1 -- 172.21.15.19:0/1683146997 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c037070 con 0x7f2a8815dc70 2024-04-25T09:56:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.504+0000 7f2a877fe640 1 -- 172.21.15.19:0/1683146997 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c032440 con 0x7f2a8815dc70 2024-04-25T09:56:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.504+0000 7f2a877fe640 1 -- 172.21.15.19:0/1683146997 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c032810 con 0x7f2a8815dc70 2024-04-25T09:56:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.505+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1683146997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8815dc70 msgr2=0x7f2a880d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.505+0000 7f2a8f14c640 1 --2- 172.21.15.19:0/1683146997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8815dc70 0x7f2a880d1120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009870 tx=0x7f2a7c02a270 comp rx=0 tx=0).stop 2024-04-25T09:56:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.505+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1683146997 shutdown_connections 2024-04-25T09:56:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.505+0000 7f2a8f14c640 1 --2- 172.21.15.19:0/1683146997 >> v2:172.21.15.112:3300/0 conn(0x7f2a8809a450 0x7f2a880cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.505+0000 7f2a8f14c640 1 --2- 172.21.15.19:0/1683146997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8815dc70 0x7f2a880d1120 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.505+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1683146997 >> 172.21.15.19:0/1683146997 conn(0x7f2a880b3270 msgr2=0x7f2a880b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.505+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1683146997 shutdown_connections 2024-04-25T09:56:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.505+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1683146997 wait complete. 2024-04-25T09:56:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.506+0000 7f2a8f14c640 1 Processor -- start 2024-04-25T09:56:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.506+0000 7f2a8f14c640 1 -- start start 2024-04-25T09:56:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8f14c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8806c700 0x7f2a880c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8f14c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8809a450 0x7f2a881f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8f14c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a8815dc70 0x7f2a88160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8f14c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880d0920 con 0x7f2a8815dc70 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8f14c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880d0640 con 0x7f2a8809a450 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8f14c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815e050 con 0x7f2a8806c700 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8d6e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a8815dc70 0x7f2a88160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8d6e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a8815dc70 0x7f2a88160d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43934/0 (socket says 172.21.15.19:43934) 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8cee8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8806c700 0x7f2a880c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a87fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8809a450 0x7f2a881f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8d6e9640 1 -- 172.21.15.19:0/1910914847 learned_addr learned my addr 172.21.15.19:0/1910914847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8d6e9640 1 -- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8806c700 msgr2=0x7f2a880c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8d6e9640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8806c700 0x7f2a880c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8d6e9640 1 -- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8809a450 msgr2=0x7f2a881f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8d6e9640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8809a450 0x7f2a881f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.507+0000 7f2a8d6e9640 1 -- 172.21.15.19:0/1910914847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a881f6d70 con 0x7f2a8815dc70 2024-04-25T09:56:26.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.508+0000 7f2a8d6e9640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a8815dc70 0x7f2a88160d10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c730 tx=0x7f2a8000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:26.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.508+0000 7f2a857fa640 1 -- 172.21.15.19:0/1910914847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80016020 con 0x7f2a8815dc70 2024-04-25T09:56:26.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.508+0000 7f2a857fa640 1 -- 172.21.15.19:0/1910914847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a80006d50 con 0x7f2a8815dc70 2024-04-25T09:56:26.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.508+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1910914847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a880c6680 con 0x7f2a8815dc70 2024-04-25T09:56:26.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.508+0000 7f2a857fa640 1 -- 172.21.15.19:0/1910914847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80003bb0 con 0x7f2a8815dc70 2024-04-25T09:56:26.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.508+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1910914847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a880c6c40 con 0x7f2a8815dc70 2024-04-25T09:56:26.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.509+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1910914847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a88073990 con 0x7f2a8815dc70 2024-04-25T09:56:26.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.509+0000 7f2a857fa640 1 -- 172.21.15.19:0/1910914847 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a800050d0 con 0x7f2a8815dc70 2024-04-25T09:56:26.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.510+0000 7f2a857fa640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a6c06b4d0 0x7f2a6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.510+0000 7f2a857fa640 1 -- 172.21.15.19:0/1910914847 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2a80089da0 con 0x7f2a8815dc70 2024-04-25T09:56:26.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.510+0000 7f2a8cee8640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a6c06b4d0 0x7f2a6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.510+0000 7f2a8cee8640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a6c06b4d0 0x7f2a6c06d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c034000 tx=0x7f2a7c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:26.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.514+0000 7f2a857fa640 1 -- 172.21.15.19:0/1910914847 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a80020020 con 0x7f2a8815dc70 2024-04-25T09:56:26.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.730+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1910914847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a880b40a0 con 0x7f2a8815dc70 2024-04-25T09:56:26.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.730+0000 7f2a857fa640 1 -- 172.21.15.19:0/1910914847 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f2a800584d0 con 0x7f2a8815dc70 2024-04-25T09:56:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:26.734 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:56:25.030125+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:26.734 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-04-25T09:56:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a6c06b4d0 msgr2=0x7f2a6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a6c06b4d0 0x7f2a6c06d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c034000 tx=0x7f2a7c02e000 comp rx=0 tx=0).stop 2024-04-25T09:56:26.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1910914847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a8815dc70 msgr2=0x7f2a88160d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a8815dc70 0x7f2a88160d10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c730 tx=0x7f2a8000cb30 comp rx=0 tx=0).stop 2024-04-25T09:56:26.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1910914847 shutdown_connections 2024-04-25T09:56:26.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a6c06b4d0 0x7f2a6c06d990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a8815dc70 0x7f2a88160d10 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a8809a450 0x7f2a881f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 --2- 172.21.15.19:0/1910914847 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a8806c700 0x7f2a880c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1910914847 >> 172.21.15.19:0/1910914847 conn(0x7f2a880b3270 msgr2=0x7f2a880b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:26.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1910914847 shutdown_connections 2024-04-25T09:56:26.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.732+0000 7f2a8f14c640 1 -- 172.21.15.19:0/1910914847 wait complete. 2024-04-25T09:56:26.746 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:26.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.844+0000 7fbba2dd5640 1 Processor -- start 2024-04-25T09:56:26.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.844+0000 7fbba2dd5640 1 -- start start 2024-04-25T09:56:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.845+0000 7fbba2dd5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbb9c15dbe0 0x7fbb9c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.845+0000 7fbba2dd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c0b2cb0 0x7fbb9c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.845+0000 7fbba2dd5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9c0b97a0 con 0x7fbb9c0b92a0 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.845+0000 7fbba2dd5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb9c15dfc0 con 0x7fbb9c15dbe0 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.845+0000 7fbba2dd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c0d1a00 con 0x7fbb9c0b2cb0 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.845+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c0b2cb0 0x7fbb9c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.845+0000 7fbba1372640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbb9c0b92a0 0x7fbb9c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:44768/0 (socket says 172.21.15.19:44768) 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.845+0000 7fbba0b71640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbb9c15dbe0 0x7fbb9c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.845+0000 7fbba1372640 1 -- 172.21.15.19:0/1900964236 learned_addr learned my addr 172.21.15.19:0/1900964236 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.845+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c0b2cb0 0x7fbb9c0d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:52504/0 (socket says 172.21.15.19:52504) 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.846+0000 7fbb9bfff640 1 -- 172.21.15.19:0/1900964236 >> v2:172.21.15.112:3300/0 conn(0x7fbb9c15dbe0 msgr2=0x7fbb9c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.846+0000 7fbb9bfff640 1 --2- 172.21.15.19:0/1900964236 >> v2:172.21.15.112:3300/0 conn(0x7fbb9c15dbe0 0x7fbb9c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.846+0000 7fbb9bfff640 1 -- 172.21.15.19:0/1900964236 >> v1:172.21.15.112:6789/0 conn(0x7fbb9c0b92a0 legacy=0x7fbb9c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.846+0000 7fbb9bfff640 1 -- 172.21.15.19:0/1900964236 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c0d1c50 con 0x7fbb9c0b2cb0 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.846+0000 7fbba0b71640 1 --2- 172.21.15.19:0/1900964236 >> v2:172.21.15.112:3300/0 conn(0x7fbb9c15dbe0 0x7fbb9c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:26.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.846+0000 7fbb9bfff640 1 --2- 172.21.15.19:0/1900964236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c0b2cb0 0x7fbb9c0d0e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb880097a0 tx=0x7fbb8802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=385421b85593e5e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:26.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.847+0000 7fbb9b7fe640 1 -- 172.21.15.19:0/1900964236 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88037070 con 0x7fbb9c0b2cb0 2024-04-25T09:56:26.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.847+0000 7fbb9b7fe640 1 -- 172.21.15.19:0/1900964236 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb88032440 con 0x7fbb9c0b2cb0 2024-04-25T09:56:26.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.847+0000 7fbb9b7fe640 1 -- 172.21.15.19:0/1900964236 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88032810 con 0x7fbb9c0b2cb0 2024-04-25T09:56:26.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.847+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1900964236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c0b2cb0 msgr2=0x7fbb9c0d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.847+0000 7fbba2dd5640 1 --2- 172.21.15.19:0/1900964236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c0b2cb0 0x7fbb9c0d0e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb880097a0 tx=0x7fbb8802a180 comp rx=0 tx=0).stop 2024-04-25T09:56:26.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.847+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1900964236 shutdown_connections 2024-04-25T09:56:26.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.847+0000 7fbba2dd5640 1 --2- 172.21.15.19:0/1900964236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c0b2cb0 0x7fbb9c0d0e50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.847+0000 7fbba2dd5640 1 --2- 172.21.15.19:0/1900964236 >> v2:172.21.15.112:3300/0 conn(0x7fbb9c15dbe0 0x7fbb9c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.847+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1900964236 >> 172.21.15.19:0/1900964236 conn(0x7fbb9c061a20 msgr2=0x7fbb9c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.847+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1900964236 shutdown_connections 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.847+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1900964236 wait complete. 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.848+0000 7fbba2dd5640 1 Processor -- start 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.848+0000 7fbba2dd5640 1 -- start start 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.849+0000 7fbba2dd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb9c0b2cb0 0x7fbb9c159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.849+0000 7fbba2dd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c0b92a0 0x7fbb9c0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.849+0000 7fbba2dd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c15dbe0 0x7fbb9c0c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.849+0000 7fbba2dd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c0b6ab0 con 0x7fbb9c0b92a0 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.849+0000 7fbba2dd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c0d2210 con 0x7fbb9c0b2cb0 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.849+0000 7fbba2dd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c0b6c30 con 0x7fbb9c15dbe0 2024-04-25T09:56:26.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.849+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c0b92a0 0x7fbb9c0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.849+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c0b92a0 0x7fbb9c0c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43938/0 (socket says 172.21.15.19:43938) 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.849+0000 7fbb9bfff640 1 -- 172.21.15.19:0/1626072938 learned_addr learned my addr 172.21.15.19:0/1626072938 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbba0b71640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb9c0b2cb0 0x7fbb9c159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbba1372640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c15dbe0 0x7fbb9c0c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbb9bfff640 1 -- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c15dbe0 msgr2=0x7fbb9c0c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbb9bfff640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c15dbe0 0x7fbb9c0c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbb9bfff640 1 -- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb9c0b2cb0 msgr2=0x7fbb9c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbb9bfff640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb9c0b2cb0 0x7fbb9c159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbb9bfff640 1 -- 172.21.15.19:0/1626072938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c155780 con 0x7fbb9c0b92a0 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbba1372640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c15dbe0 0x7fbb9c0c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbba0b71640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb9c0b2cb0 0x7fbb9c159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbb9bfff640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c0b92a0 0x7fbb9c0c8fb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbb88035040 tx=0x7fbb880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbb997fa640 1 -- 172.21.15.19:0/1626072938 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88037040 con 0x7fbb9c0b92a0 2024-04-25T09:56:26.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbb997fa640 1 -- 172.21.15.19:0/1626072938 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb88030500 con 0x7fbb9c0b92a0 2024-04-25T09:56:26.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1626072938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9c155a60 con 0x7fbb9c0b92a0 2024-04-25T09:56:26.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbb997fa640 1 -- 172.21.15.19:0/1626072938 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88030910 con 0x7fbb9c0b92a0 2024-04-25T09:56:26.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.850+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1626072938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9c156020 con 0x7fbb9c0b92a0 2024-04-25T09:56:26.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.851+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1626072938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9c073990 con 0x7fbb9c0b92a0 2024-04-25T09:56:26.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.851+0000 7fbb997fa640 1 -- 172.21.15.19:0/1626072938 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb88044020 con 0x7fbb9c0b92a0 2024-04-25T09:56:26.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.852+0000 7fbb997fa640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb8006b4d0 0x7fbb8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:26.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.852+0000 7fbb997fa640 1 -- 172.21.15.19:0/1626072938 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb880aa670 con 0x7fbb9c0b92a0 2024-04-25T09:56:26.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.852+0000 7fbba0b71640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb8006b4d0 0x7fbb8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:26.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.853+0000 7fbba0b71640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb8006b4d0 0x7fbb8006d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009af0 tx=0x7fbb90008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:26.856+0000 7fbb997fa640 1 -- 172.21.15.19:0/1626072938 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb880792d0 con 0x7fbb9c0b92a0 2024-04-25T09:56:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.036+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1626072938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb9c06b080 con 0x7fbb9c0b92a0 2024-04-25T09:56:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.038+0000 7fbb997fa640 1 -- 172.21.15.19:0/1626072938 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16930 (secure 0 0 0) 0x7fbb88078bd0 con 0x7fbb9c0b92a0 2024-04-25T09:56:27.041 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:27.041 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":53,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:53.990243+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:50.618180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:51.322489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb8006b4d0 msgr2=0x7fbb8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb8006b4d0 0x7fbb8006d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009af0 tx=0x7fbb90008040 comp rx=0 tx=0).stop 2024-04-25T09:56:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1626072938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c0b92a0 msgr2=0x7fbb9c0c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c0b92a0 0x7fbb9c0c8fb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbb88035040 tx=0x7fbb880046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1626072938 shutdown_connections 2024-04-25T09:56:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb8006b4d0 0x7fbb8006d990 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9c15dbe0 0x7fbb9c0c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c0b92a0 0x7fbb9c0c8fb0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 --2- 172.21.15.19:0/1626072938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb9c0b2cb0 0x7fbb9c159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1626072938 >> 172.21.15.19:0/1626072938 conn(0x7fbb9c061a20 msgr2=0x7fbb9c0b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1626072938 shutdown_connections 2024-04-25T09:56:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.040+0000 7fbba2dd5640 1 -- 172.21.15.19:0/1626072938 wait complete. 2024-04-25T09:56:27.054 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:56:27.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.153+0000 7f223bbe7640 1 Processor -- start 2024-04-25T09:56:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.153+0000 7f223bbe7640 1 -- start start 2024-04-25T09:56:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223bbe7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f223415db70 0x7f2234162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223bbe7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22340b6cb0 0x7f22340b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223bbe7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22340b5a60 0x7f223406c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223bbe7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2234162690 con 0x7f22340b5a60 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223bbe7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22340ceee0 con 0x7f223415db70 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223bbe7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22340d1d10 con 0x7f22340b6cb0 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223a184640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22340b5a60 0x7f223406c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223a184640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22340b5a60 0x7f223406c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43952/0 (socket says 172.21.15.19:43952) 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223a184640 1 -- 172.21.15.19:0/354833235 learned_addr learned my addr 172.21.15.19:0/354833235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f2239182640 1 --2- 172.21.15.19:0/354833235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22340b6cb0 0x7f22340b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f2239983640 1 --2- 172.21.15.19:0/354833235 >> v2:172.21.15.112:3300/0 conn(0x7f223415db70 0x7f2234162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223a184640 1 -- 172.21.15.19:0/354833235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22340b6cb0 msgr2=0x7f22340b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223a184640 1 --2- 172.21.15.19:0/354833235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22340b6cb0 0x7f22340b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223a184640 1 -- 172.21.15.19:0/354833235 >> v2:172.21.15.112:3300/0 conn(0x7f223415db70 msgr2=0x7f2234162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223a184640 1 --2- 172.21.15.19:0/354833235 >> v2:172.21.15.112:3300/0 conn(0x7f223415db70 0x7f2234162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.154+0000 7f223a184640 1 -- 172.21.15.19:0/354833235 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22340d1ff0 con 0x7f22340b5a60 2024-04-25T09:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.155+0000 7f223a184640 1 --2- 172.21.15.19:0/354833235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22340b5a60 0x7f223406c2d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f22340b6570 tx=0x7f222c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a963c8f086ade1f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.155+0000 7f2238981640 1 -- 172.21.15.19:0/354833235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c036070 con 0x7f22340b5a60 2024-04-25T09:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.155+0000 7f2238981640 1 -- 172.21.15.19:0/354833235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c031600 con 0x7f22340b5a60 2024-04-25T09:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.155+0000 7f2238981640 1 -- 172.21.15.19:0/354833235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c0319d0 con 0x7f22340b5a60 2024-04-25T09:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.156+0000 7f223bbe7640 1 -- 172.21.15.19:0/354833235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22340b5a60 msgr2=0x7f223406c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.156+0000 7f223bbe7640 1 --2- 172.21.15.19:0/354833235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22340b5a60 0x7f223406c2d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f22340b6570 tx=0x7f222c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.156+0000 7f223bbe7640 1 -- 172.21.15.19:0/354833235 shutdown_connections 2024-04-25T09:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.156+0000 7f223bbe7640 1 --2- 172.21.15.19:0/354833235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22340b6cb0 0x7f22340b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.156+0000 7f223bbe7640 1 --2- 172.21.15.19:0/354833235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22340b5a60 0x7f223406c2d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.156+0000 7f223bbe7640 1 --2- 172.21.15.19:0/354833235 >> v2:172.21.15.112:3300/0 conn(0x7f223415db70 0x7f2234162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.156+0000 7f223bbe7640 1 -- 172.21.15.19:0/354833235 >> 172.21.15.19:0/354833235 conn(0x7f22340d0520 msgr2=0x7f22340d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.156+0000 7f223bbe7640 1 -- 172.21.15.19:0/354833235 shutdown_connections 2024-04-25T09:56:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.156+0000 7f223bbe7640 1 -- 172.21.15.19:0/354833235 wait complete. 2024-04-25T09:56:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.157+0000 7f223bbe7640 1 Processor -- start 2024-04-25T09:56:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.157+0000 7f223bbe7640 1 -- start start 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223bbe7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22340b5a60 0x7f22341f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223bbe7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22340b6cb0 0x7f22341609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223bbe7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f223415db70 0x7f22340cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223bbe7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340ce530 con 0x7f223415db70 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223bbe7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223415df50 con 0x7f22340b6cb0 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223bbe7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22340cf1c0 con 0x7f22340b5a60 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223a184640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f223415db70 0x7f22340cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223a184640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f223415db70 0x7f22340cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43968/0 (socket says 172.21.15.19:43968) 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223a184640 1 -- 172.21.15.19:0/599591256 learned_addr learned my addr 172.21.15.19:0/599591256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f2239983640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22340b5a60 0x7f22341f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f2239182640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22340b6cb0 0x7f22341609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223a184640 1 -- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22340b5a60 msgr2=0x7f22341f0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223a184640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22340b5a60 0x7f22341f0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223a184640 1 -- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22340b6cb0 msgr2=0x7f22341609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223a184640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22340b6cb0 0x7f22341609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223a184640 1 -- 172.21.15.19:0/599591256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22340cac10 con 0x7f223415db70 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.158+0000 7f223a184640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f223415db70 0x7f22340cc980 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f222c009f40 tx=0x7f222c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.159+0000 7f222a7fc640 1 -- 172.21.15.19:0/599591256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c041020 con 0x7f223415db70 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.159+0000 7f222a7fc640 1 -- 172.21.15.19:0/599591256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c0077a0 con 0x7f223415db70 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.159+0000 7f223bbe7640 1 -- 172.21.15.19:0/599591256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22340caef0 con 0x7f223415db70 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.159+0000 7f222a7fc640 1 -- 172.21.15.19:0/599591256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c036040 con 0x7f223415db70 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.159+0000 7f223bbe7640 1 -- 172.21.15.19:0/599591256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22340cb4b0 con 0x7f223415db70 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.160+0000 7f223bbe7640 1 -- 172.21.15.19:0/599591256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2234073990 con 0x7f223415db70 2024-04-25T09:56:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.160+0000 7f222a7fc640 1 -- 172.21.15.19:0/599591256 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f222c044020 con 0x7f223415db70 2024-04-25T09:56:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.160+0000 7f222a7fc640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f221806b4d0 0x7f221806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.160+0000 7f222a7fc640 1 -- 172.21.15.19:0/599591256 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f222c0aaf90 con 0x7f223415db70 2024-04-25T09:56:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.161+0000 7f2239983640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f221806b4d0 0x7f221806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.161+0000 7f2239983640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f221806b4d0 0x7f221806d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2230009a30 tx=0x7f2230008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.164+0000 7f222a7fc640 1 -- 172.21.15.19:0/599591256 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f222c04b030 con 0x7f223415db70 2024-04-25T09:56:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.380+0000 7f223bbe7640 1 -- 172.21.15.19:0/599591256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2234161fc0 con 0x7f223415db70 2024-04-25T09:56:27.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.380+0000 7f222a7fc640 1 -- 172.21.15.19:0/599591256 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f222c079d40 con 0x7f223415db70 2024-04-25T09:56:27.383 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:27.384 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:56:25.030125+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:27.384 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-04-25T09:56:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 -- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f221806b4d0 msgr2=0x7f221806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f221806b4d0 0x7f221806d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2230009a30 tx=0x7f2230008040 comp rx=0 tx=0).stop 2024-04-25T09:56:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 -- 172.21.15.19:0/599591256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f223415db70 msgr2=0x7f22340cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f223415db70 0x7f22340cc980 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f222c009f40 tx=0x7f222c004a40 comp rx=0 tx=0).stop 2024-04-25T09:56:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 -- 172.21.15.19:0/599591256 shutdown_connections 2024-04-25T09:56:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f221806b4d0 0x7f221806d990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f223415db70 0x7f22340cc980 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22340b6cb0 0x7f22341609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 --2- 172.21.15.19:0/599591256 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22340b5a60 0x7f22341f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 -- 172.21.15.19:0/599591256 >> 172.21.15.19:0/599591256 conn(0x7f22340d0520 msgr2=0x7f223406c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:27.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 -- 172.21.15.19:0/599591256 shutdown_connections 2024-04-25T09:56:27.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.382+0000 7f223bbe7640 1 -- 172.21.15.19:0/599591256 wait complete. 2024-04-25T09:56:27.396 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.494+0000 7f20d83a5640 1 Processor -- start 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.494+0000 7f20d83a5640 1 -- start start 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.495+0000 7f20d83a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d0158220 0x7f20d015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.495+0000 7f20d83a5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20d00afba0 0x7f20d0161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.495+0000 7f20d83a5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d015f180 con 0x7f20d00af720 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.495+0000 7f20d83a5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d00b3670 con 0x7f20d00afba0 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.495+0000 7f20d83a5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0070a10 con 0x7f20d0158220 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.495+0000 7f20d6942640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20d00afba0 0x7f20d0161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.495+0000 7f20d5940640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f20d00af720 0x7f20d01617b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33586/0 (socket says 172.21.15.19:33586) 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.495+0000 7f20d5940640 1 -- 172.21.15.19:0/1180168434 learned_addr learned my addr 172.21.15.19:0/1180168434 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.496+0000 7f20d6141640 1 --2- 172.21.15.19:0/1180168434 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d0158220 0x7f20d015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.496+0000 7f20d6942640 1 -- 172.21.15.19:0/1180168434 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d0158220 msgr2=0x7f20d015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.496+0000 7f20d6942640 1 --2- 172.21.15.19:0/1180168434 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d0158220 0x7f20d015ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.496+0000 7f20d6942640 1 -- 172.21.15.19:0/1180168434 >> v1:172.21.15.19:6789/0 conn(0x7f20d00af720 legacy=0x7f20d01617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.496+0000 7f20d513f640 1 -- 172.21.15.19:0/1180168434 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373652921 0 0) 0x7f20c4001110 con 0x7f20d00af720 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.496+0000 7f20d6942640 1 -- 172.21.15.19:0/1180168434 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d00b0030 con 0x7f20d00afba0 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.496+0000 7f20d6942640 1 --2- 172.21.15.19:0/1180168434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d00afba0 0x7f20d0161ec0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f20c00097a0 tx=0x7f20c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47a3439b40f8d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.497+0000 7f20d513f640 1 -- 172.21.15.19:0/1180168434 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0037070 con 0x7f20d00afba0 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.497+0000 7f20d513f640 1 -- 172.21.15.19:0/1180168434 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c0032440 con 0x7f20d00afba0 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.497+0000 7f20d513f640 1 -- 172.21.15.19:0/1180168434 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0032810 con 0x7f20d00afba0 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.497+0000 7f20d83a5640 1 -- 172.21.15.19:0/1180168434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d00afba0 msgr2=0x7f20d0161ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.497+0000 7f20d83a5640 1 --2- 172.21.15.19:0/1180168434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d00afba0 0x7f20d0161ec0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f20c00097a0 tx=0x7f20c002a180 comp rx=0 tx=0).stop 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.497+0000 7f20d83a5640 1 -- 172.21.15.19:0/1180168434 shutdown_connections 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.497+0000 7f20d83a5640 1 --2- 172.21.15.19:0/1180168434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d00afba0 0x7f20d0161ec0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.497+0000 7f20d83a5640 1 --2- 172.21.15.19:0/1180168434 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d0158220 0x7f20d015ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.497+0000 7f20d83a5640 1 -- 172.21.15.19:0/1180168434 >> 172.21.15.19:0/1180168434 conn(0x7f20d00b2370 msgr2=0x7f20d00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.497+0000 7f20d83a5640 1 -- 172.21.15.19:0/1180168434 shutdown_connections 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.497+0000 7f20d83a5640 1 -- 172.21.15.19:0/1180168434 wait complete. 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.498+0000 7f20d83a5640 1 Processor -- start 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.498+0000 7f20d83a5640 1 -- start start 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.499+0000 7f20d83a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d00af720 0x7f20d015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.499+0000 7f20d83a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d00afba0 0x7f20d015c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.499+0000 7f20d83a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20d0158220 0x7f20d015caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.499+0000 7f20d83a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00703c0 con 0x7f20d00afba0 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.499+0000 7f20d83a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00700e0 con 0x7f20d0158220 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.499+0000 7f20d83a5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00706a0 con 0x7f20d00af720 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.499+0000 7f20d5940640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d00afba0 0x7f20d015c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.499+0000 7f20d5940640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d00afba0 0x7f20d015c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43976/0 (socket says 172.21.15.19:43976) 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.499+0000 7f20d5940640 1 -- 172.21.15.19:0/2231130708 learned_addr learned my addr 172.21.15.19:0/2231130708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.499+0000 7f20d6942640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20d0158220 0x7f20d015caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.499+0000 7f20d6141640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d00af720 0x7f20d015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.500+0000 7f20d5940640 1 -- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d00af720 msgr2=0x7f20d015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.500+0000 7f20d5940640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d00af720 0x7f20d015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.500+0000 7f20d5940640 1 -- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20d0158220 msgr2=0x7f20d015caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.500+0000 7f20d5940640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20d0158220 0x7f20d015caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.500+0000 7f20d5940640 1 -- 172.21.15.19:0/2231130708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d015cfe0 con 0x7f20d00afba0 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.500+0000 7f20d5940640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d00afba0 0x7f20d015c560 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c7b0 tx=0x7f20c400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.500+0000 7f20ceffd640 1 -- 172.21.15.19:0/2231130708 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4018070 con 0x7f20d00afba0 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.500+0000 7f20ceffd640 1 -- 172.21.15.19:0/2231130708 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c40072d0 con 0x7f20d00afba0 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.500+0000 7f20ceffd640 1 -- 172.21.15.19:0/2231130708 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40047e0 con 0x7f20d00afba0 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.500+0000 7f20d83a5640 1 -- 172.21.15.19:0/2231130708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d015d2b0 con 0x7f20d00afba0 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.500+0000 7f20d83a5640 1 -- 172.21.15.19:0/2231130708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d00cd710 con 0x7f20d00afba0 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.501+0000 7f20d83a5640 1 -- 172.21.15.19:0/2231130708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d0073990 con 0x7f20d00afba0 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.502+0000 7f20ceffd640 1 -- 172.21.15.19:0/2231130708 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20c401b070 con 0x7f20d00afba0 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.502+0000 7f20ceffd640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20bc06b4d0 0x7f20bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.502+0000 7f20ceffd640 1 -- 172.21.15.19:0/2231130708 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20c4089570 con 0x7f20d00afba0 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.502+0000 7f20d6141640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20bc06b4d0 0x7f20bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.503+0000 7f20d6141640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20bc06b4d0 0x7f20bc06d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f20c8003aa0 tx=0x7f20c8009830 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.506+0000 7f20ceffd640 1 -- 172.21.15.19:0/2231130708 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20d015f180 con 0x7f20d00afba0 2024-04-25T09:56:27.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.729+0000 7f20d83a5640 1 -- 172.21.15.19:0/2231130708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20d015f400 con 0x7f20d00afba0 2024-04-25T09:56:27.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.729+0000 7f20ceffd640 1 -- 172.21.15.19:0/2231130708 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f20c4057ca0 con 0x7f20d00afba0 2024-04-25T09:56:27.732 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:27.732 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:52.345127+0000","modified":"2024-04-25T09:56:25.030125+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6835/2436445353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2436445353},{"type":"v1","addr":"172.21.15.19:6835","nonce":2436445353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:56:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 -- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20bc06b4d0 msgr2=0x7f20bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20bc06b4d0 0x7f20bc06d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f20c8003aa0 tx=0x7f20c8009830 comp rx=0 tx=0).stop 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 -- 172.21.15.19:0/2231130708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d00afba0 msgr2=0x7f20d015c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d00afba0 0x7f20d015c560 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c7b0 tx=0x7f20c400cc70 comp rx=0 tx=0).stop 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 -- 172.21.15.19:0/2231130708 shutdown_connections 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20bc06b4d0 0x7f20bc06d990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20d0158220 0x7f20d015caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d00afba0 0x7f20d015c560 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 --2- 172.21.15.19:0/2231130708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d00af720 0x7f20d015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 -- 172.21.15.19:0/2231130708 >> 172.21.15.19:0/2231130708 conn(0x7f20d00b2370 msgr2=0x7f20d00b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 -- 172.21.15.19:0/2231130708 shutdown_connections 2024-04-25T09:56:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.731+0000 7f20d83a5640 1 -- 172.21.15.19:0/2231130708 wait complete. 2024-04-25T09:56:27.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:27.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.842+0000 7f0600d7e640 1 Processor -- start 2024-04-25T09:56:27.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.842+0000 7f0600d7e640 1 -- start start 2024-04-25T09:56:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.843+0000 7f0600d7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05fc0b2cb0 0x7f05fc0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.843+0000 7f0600d7e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc0b97a0 con 0x7f05fc0d1020 2024-04-25T09:56:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.843+0000 7f0600d7e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc0d1c50 con 0x7f05fc15dbe0 2024-04-25T09:56:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.843+0000 7f0600d7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc15dfc0 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.843+0000 7f05fad9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f05fc0d1020 0x7f05fc0d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51134/0 (socket says 172.21.15.19:51134) 2024-04-25T09:56:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.843+0000 7f05fad9d640 1 -- 172.21.15.19:0/270063603 learned_addr learned my addr 172.21.15.19:0/270063603 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.843+0000 7f05f9d9b640 1 --2- 172.21.15.19:0/270063603 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05fc0b2cb0 0x7f05fc0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.843+0000 7f05f959a640 1 -- 172.21.15.19:0/270063603 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15890345 0 0) 0x7f05fc0b97a0 con 0x7f05fc0d1020 2024-04-25T09:56:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.844+0000 7f05f959a640 1 -- 172.21.15.19:0/270063603 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e8003520 con 0x7f05fc0d1020 2024-04-25T09:56:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.844+0000 7f05f959a640 1 -- 172.21.15.19:0/270063603 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266164883 0 0) 0x7f05fc0d1c50 con 0x7f05fc15dbe0 2024-04-25T09:56:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.844+0000 7f05f959a640 1 -- 172.21.15.19:0/270063603 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05fc0b97a0 con 0x7f05fc15dbe0 2024-04-25T09:56:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.844+0000 7f05f9d9b640 1 -- 172.21.15.19:0/270063603 >> v1:172.21.15.112:6789/0 conn(0x7f05fc15dbe0 legacy=0x7f05fc0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.844+0000 7f05f9d9b640 1 -- 172.21.15.19:0/270063603 >> v1:172.21.15.19:6789/0 conn(0x7f05fc0d1020 legacy=0x7f05fc0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.844+0000 7f05f9d9b640 1 -- 172.21.15.19:0/270063603 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0d1f30 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.845+0000 7f05f9d9b640 1 --2- 172.21.15.19:0/270063603 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05fc0b2cb0 0x7f05fc0d2730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f05e40097d0 tx=0x7f05e402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7409ed9c9f778402 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:27.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.845+0000 7f05f959a640 1 -- 172.21.15.19:0/270063603 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4037070 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.845+0000 7f05f959a640 1 -- 172.21.15.19:0/270063603 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e4032440 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.845+0000 7f05f959a640 1 -- 172.21.15.19:0/270063603 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4032810 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.845+0000 7f0600d7e640 1 -- 172.21.15.19:0/270063603 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05fc0b2cb0 msgr2=0x7f05fc0d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.845+0000 7f0600d7e640 1 --2- 172.21.15.19:0/270063603 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05fc0b2cb0 0x7f05fc0d2730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f05e40097d0 tx=0x7f05e402a210 comp rx=0 tx=0).stop 2024-04-25T09:56:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.845+0000 7f0600d7e640 1 -- 172.21.15.19:0/270063603 shutdown_connections 2024-04-25T09:56:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.845+0000 7f0600d7e640 1 --2- 172.21.15.19:0/270063603 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05fc0b2cb0 0x7f05fc0d2730 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.845+0000 7f0600d7e640 1 -- 172.21.15.19:0/270063603 >> 172.21.15.19:0/270063603 conn(0x7f05fc061a20 msgr2=0x7f05fc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.845+0000 7f0600d7e640 1 -- 172.21.15.19:0/270063603 shutdown_connections 2024-04-25T09:56:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.845+0000 7f0600d7e640 1 -- 172.21.15.19:0/270063603 wait complete. 2024-04-25T09:56:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.847+0000 7f0600d7e640 1 Processor -- start 2024-04-25T09:56:27.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.847+0000 7f0600d7e640 1 -- start start 2024-04-25T09:56:27.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.847+0000 7f0600d7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05fc0b2cb0 0x7f05fc1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.847+0000 7f0600d7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05fc0d1020 0x7f05fc14ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.847+0000 7f0600d7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05fc15dbe0 0x7f05fc0b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.847+0000 7f0600d7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0b9420 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.847+0000 7f0600d7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0b92a0 con 0x7f05fc15dbe0 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.847+0000 7f0600d7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0b6ab0 con 0x7f05fc0d1020 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05fc0b2cb0 0x7f05fc1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05fc0b2cb0 0x7f05fc1556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49298/0 (socket says 172.21.15.19:49298) 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05fa59c640 1 -- 172.21.15.19:0/2008106281 learned_addr learned my addr 172.21.15.19:0/2008106281 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05f9d9b640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05fc0d1020 0x7f05fc14ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05fa59c640 1 -- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05fc0d1020 msgr2=0x7f05fc14ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05fa59c640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05fc0d1020 0x7f05fc14ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05fad9d640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05fc15dbe0 0x7f05fc0b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05fa59c640 1 -- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05fc15dbe0 msgr2=0x7f05fc0b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05fa59c640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05fc15dbe0 0x7f05fc0b5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:27.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05fa59c640 1 -- 172.21.15.19:0/2008106281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc1504b0 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05fa59c640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05fc0b2cb0 0x7f05fc1556e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c460 tx=0x7f05f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05f2ffd640 1 -- 172.21.15.19:0/2008106281 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4017070 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05f2ffd640 1 -- 172.21.15.19:0/2008106281 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f40052f0 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f0600d7e640 1 -- 172.21.15.19:0/2008106281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc0cb9d0 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f05f2ffd640 1 -- 172.21.15.19:0/2008106281 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4003830 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.848+0000 7f0600d7e640 1 -- 172.21.15.19:0/2008106281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc0cbf90 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.849+0000 7f0600d7e640 1 -- 172.21.15.19:0/2008106281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc073990 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.850+0000 7f05f2ffd640 1 -- 172.21.15.19:0/2008106281 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05f401a070 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.850+0000 7f05f2ffd640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f05dc06b4d0 0x7f05dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.850+0000 7f05f2ffd640 1 -- 172.21.15.19:0/2008106281 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f05f4089780 con 0x7f05fc0b2cb0 2024-04-25T09:56:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.850+0000 7f05f9d9b640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f05dc06b4d0 0x7f05dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:27.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.851+0000 7f05f9d9b640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f05dc06b4d0 0x7f05dc06d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f05e4034000 tx=0x7f05e402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:27.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:27.855+0000 7f05f2ffd640 1 -- 172.21.15.19:0/2008106281 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05f4058470 con 0x7f05fc0b2cb0 2024-04-25T09:56:28.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.037+0000 7f0600d7e640 1 -- 172.21.15.19:0/2008106281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05fc06cae0 con 0x7f05fc0b2cb0 2024-04-25T09:56:28.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.038+0000 7f05f2ffd640 1 -- 172.21.15.19:0/2008106281 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16930 (secure 0 0 0) 0x7f05f4057d70 con 0x7f05fc0b2cb0 2024-04-25T09:56:28.042 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:28.042 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":53,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:55:53.990243+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:50.618180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:51.322489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 -- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f05dc06b4d0 msgr2=0x7f05dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f05dc06b4d0 0x7f05dc06d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f05e4034000 tx=0x7f05e402e000 comp rx=0 tx=0).stop 2024-04-25T09:56:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 -- 172.21.15.19:0/2008106281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05fc0b2cb0 msgr2=0x7f05fc1556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05fc0b2cb0 0x7f05fc1556e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c460 tx=0x7f05f400c920 comp rx=0 tx=0).stop 2024-04-25T09:56:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 -- 172.21.15.19:0/2008106281 shutdown_connections 2024-04-25T09:56:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f05dc06b4d0 0x7f05dc06d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05fc15dbe0 0x7f05fc0b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05fc0d1020 0x7f05fc14ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 --2- 172.21.15.19:0/2008106281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05fc0b2cb0 0x7f05fc1556e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 -- 172.21.15.19:0/2008106281 >> 172.21.15.19:0/2008106281 conn(0x7f05fc061a20 msgr2=0x7f05fc15aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 -- 172.21.15.19:0/2008106281 shutdown_connections 2024-04-25T09:56:28.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.041+0000 7f0600d7e640 1 -- 172.21.15.19:0/2008106281 wait complete. 2024-04-25T09:56:28.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:56:28.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.153+0000 7fa034f5c640 1 Processor -- start 2024-04-25T09:56:28.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.154+0000 7fa034f5c640 1 -- start start 2024-04-25T09:56:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.154+0000 7fa034f5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0301446d0 0x7fa0300b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.154+0000 7fa034f5c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0300cccc0 con 0x7fa030144d50 2024-04-25T09:56:28.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.154+0000 7fa034f5c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0300d0460 con 0x7fa03006e8a0 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.154+0000 7fa034f5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0300b2b20 con 0x7fa0301446d0 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02dd9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa030144d50 0x7fa0300cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51138/0 (socket says 172.21.15.19:51138) 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02dd9b640 1 -- 172.21.15.19:0/85519923 learned_addr learned my addr 172.21.15.19:0/85519923 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02e59c640 1 --2- 172.21.15.19:0/85519923 >> v2:172.21.15.112:3300/0 conn(0x7fa0301446d0 0x7fa0300b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607795594 0 0) 0x7fa0300cccc0 con 0x7fa030144d50 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa018003520 con 0x7fa030144d50 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518251706 0 0) 0x7fa018003520 con 0x7fa030144d50 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 >> v2:172.21.15.112:3300/0 conn(0x7fa0301446d0 msgr2=0x7fa0300b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 --2- 172.21.15.19:0/85519923 >> v2:172.21.15.112:3300/0 conn(0x7fa0301446d0 0x7fa0300b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 >> v1:172.21.15.112:6789/0 conn(0x7fa03006e8a0 legacy=0x7fa0300cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0300cc140 con 0x7fa030144d50 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fa024001670 con 0x7fa030144d50 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 >> v1:172.21.15.19:6789/0 conn(0x7fa030144d50 legacy=0x7fa0300cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 --2- 172.21.15.19:0/85519923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0180039b0 0x7fa018003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 --2- 172.21.15.19:0/85519923 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0180043b0 0x7fa018004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 --2- 172.21.15.19:0/85519923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa018005080 0x7fa01800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300b2b20 con 0x7fa018005080 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01800bfe0 con 0x7fa0180039b0 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01800c1e0 con 0x7fa0180043b0 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02ed9d640 1 --2- 172.21.15.19:0/85519923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa018005080 0x7fa01800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.155+0000 7fa02e59c640 1 --2- 172.21.15.19:0/85519923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0180039b0 0x7fa018003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.156+0000 7fa02dd9b640 1 --2- 172.21.15.19:0/85519923 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0180043b0 0x7fa018004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.156+0000 7fa02ed9d640 1 -- 172.21.15.19:0/85519923 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0180043b0 msgr2=0x7fa018004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.156+0000 7fa02ed9d640 1 --2- 172.21.15.19:0/85519923 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0180043b0 0x7fa018004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.156+0000 7fa02ed9d640 1 -- 172.21.15.19:0/85519923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0180039b0 msgr2=0x7fa018003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.156+0000 7fa02ed9d640 1 --2- 172.21.15.19:0/85519923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0180039b0 0x7fa018003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.156+0000 7fa02ed9d640 1 -- 172.21.15.19:0/85519923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa018003520 con 0x7fa018005080 2024-04-25T09:56:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.156+0000 7fa02ed9d640 1 -- 172.21.15.19:0/85519923 reap_dead start 2024-04-25T09:56:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.156+0000 7fa02ed9d640 1 --2- 172.21.15.19:0/85519923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa018005080 0x7fa01800b9c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa028001f60 tx=0x7fa018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fa841162019654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.157+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028001b30 con 0x7fa018005080 2024-04-25T09:56:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.157+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa028003b90 con 0x7fa018005080 2024-04-25T09:56:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.157+0000 7fa02d59a640 1 -- 172.21.15.19:0/85519923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028005110 con 0x7fa018005080 2024-04-25T09:56:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.157+0000 7fa034f5c640 1 -- 172.21.15.19:0/85519923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa018005080 msgr2=0x7fa01800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.157+0000 7fa034f5c640 1 --2- 172.21.15.19:0/85519923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa018005080 0x7fa01800b9c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa028001f60 tx=0x7fa018005830 comp rx=0 tx=0).stop 2024-04-25T09:56:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.157+0000 7fa034f5c640 1 -- 172.21.15.19:0/85519923 shutdown_connections 2024-04-25T09:56:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.157+0000 7fa034f5c640 1 --2- 172.21.15.19:0/85519923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa018005080 0x7fa01800b9c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.157+0000 7fa034f5c640 1 -- 172.21.15.19:0/85519923 >> 172.21.15.19:0/85519923 conn(0x7fa03006abe0 msgr2=0x7fa03006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:28.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.157+0000 7fa034f5c640 1 -- 172.21.15.19:0/85519923 shutdown_connections 2024-04-25T09:56:28.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.157+0000 7fa034f5c640 1 -- 172.21.15.19:0/85519923 wait complete. 2024-04-25T09:56:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.159+0000 7fa034f5c640 1 Processor -- start 2024-04-25T09:56:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.159+0000 7fa034f5c640 1 -- start start 2024-04-25T09:56:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.159+0000 7fa034f5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa018005080 0x7fa030136fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.159+0000 7fa034f5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0300ab0c0 0x7fa0300a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.159+0000 7fa034f5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0300acf60 0x7fa0300ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.159+0000 7fa034f5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300cc700 con 0x7fa0300acf60 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.159+0000 7fa034f5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03006d400 con 0x7fa018005080 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.159+0000 7fa034f5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300cc9e0 con 0x7fa0300ab0c0 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.159+0000 7fa02ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0300acf60 0x7fa0300ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.159+0000 7fa02ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0300acf60 0x7fa0300ad410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49312/0 (socket says 172.21.15.19:49312) 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.159+0000 7fa02ed9d640 1 -- 172.21.15.19:0/2613281680 learned_addr learned my addr 172.21.15.19:0/2613281680 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa02e59c640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa018005080 0x7fa030136fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa02dd9b640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0300ab0c0 0x7fa0300a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa02ed9d640 1 -- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0300ab0c0 msgr2=0x7fa0300a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa02ed9d640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0300ab0c0 0x7fa0300a97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa02ed9d640 1 -- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa018005080 msgr2=0x7fa030136fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa02ed9d640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa018005080 0x7fa030136fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa02ed9d640 1 -- 172.21.15.19:0/2613281680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0300ac720 con 0x7fa0300acf60 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa02dd9b640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0300ab0c0 0x7fa0300a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa02ed9d640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0300acf60 0x7fa0300ad410 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa02802aa00 tx=0x7fa0280015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:28.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa01effd640 1 -- 172.21.15.19:0/2613281680 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028006d50 con 0x7fa0300acf60 2024-04-25T09:56:28.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa01effd640 1 -- 172.21.15.19:0/2613281680 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0280057e0 con 0x7fa0300acf60 2024-04-25T09:56:28.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa034f5c640 1 -- 172.21.15.19:0/2613281680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0300ab680 con 0x7fa0300acf60 2024-04-25T09:56:28.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa01effd640 1 -- 172.21.15.19:0/2613281680 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028007250 con 0x7fa0300acf60 2024-04-25T09:56:28.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.160+0000 7fa034f5c640 1 -- 172.21.15.19:0/2613281680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0300abc40 con 0x7fa0300acf60 2024-04-25T09:56:28.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.161+0000 7fa034f5c640 1 -- 172.21.15.19:0/2613281680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa030073a00 con 0x7fa0300acf60 2024-04-25T09:56:28.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.162+0000 7fa01effd640 1 -- 172.21.15.19:0/2613281680 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa02803c070 con 0x7fa0300acf60 2024-04-25T09:56:28.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.162+0000 7fa01effd640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa01006b4d0 0x7fa01006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.162+0000 7fa01effd640 1 -- 172.21.15.19:0/2613281680 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0280aa390 con 0x7fa0300acf60 2024-04-25T09:56:28.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.162+0000 7fa02e59c640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa01006b4d0 0x7fa01006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.163+0000 7fa02e59c640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa01006b4d0 0x7fa01006d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa020009a80 tx=0x7fa020008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:28.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.166+0000 7fa01effd640 1 -- 172.21.15.19:0/2613281680 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa028044060 con 0x7fa0300acf60 2024-04-25T09:56:28.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.383+0000 7fa034f5c640 1 -- 172.21.15.19:0/2613281680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa0300ce670 con 0x7fa0300acf60 2024-04-25T09:56:28.386 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T09:56:28.383+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:56:28.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.385+0000 7fa01effd640 1 -- 172.21.15.19:0/2613281680 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa028078ff0 con 0x7fa0300acf60 2024-04-25T09:56:28.388 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:56:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 -- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa01006b4d0 msgr2=0x7fa01006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa01006b4d0 0x7fa01006d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa020009a80 tx=0x7fa020008040 comp rx=0 tx=0).stop 2024-04-25T09:56:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 -- 172.21.15.19:0/2613281680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0300acf60 msgr2=0x7fa0300ad410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0300acf60 0x7fa0300ad410 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa02802aa00 tx=0x7fa0280015b0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 -- 172.21.15.19:0/2613281680 shutdown_connections 2024-04-25T09:56:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa01006b4d0 0x7fa01006d990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0300acf60 0x7fa0300ad410 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0300ab0c0 0x7fa0300a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 --2- 172.21.15.19:0/2613281680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa018005080 0x7fa030136fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 -- 172.21.15.19:0/2613281680 >> 172.21.15.19:0/2613281680 conn(0x7fa03006abe0 msgr2=0x7fa0300b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:28.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 -- 172.21.15.19:0/2613281680 shutdown_connections 2024-04-25T09:56:28.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.388+0000 7fa034f5c640 1 -- 172.21.15.19:0/2613281680 wait complete. 2024-04-25T09:56:28.402 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:56:28.431 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T09:56:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.504+0000 7f0b77e9e640 1 Processor -- start 2024-04-25T09:56:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.504+0000 7f0b77e9e640 1 -- start start 2024-04-25T09:56:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.505+0000 7f0b77e9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b70072220 0x7f0b7006e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.505+0000 7f0b77e9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b7006ec30 0x7f0b700cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.505+0000 7f0b77e9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b7016e5a0 con 0x7f0b7016a720 2024-04-25T09:56:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.505+0000 7f0b77e9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7016f170 con 0x7f0b7006ec30 2024-04-25T09:56:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.505+0000 7f0b77e9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7016e840 con 0x7f0b70072220 2024-04-25T09:56:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.505+0000 7f0b7643b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b7006ec30 0x7f0b700cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.505+0000 7f0b75c3a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0b7016a720 0x7f0b70071b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51142/0 (socket says 172.21.15.19:51142) 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.505+0000 7f0b75c3a640 1 -- 172.21.15.19:0/4110325344 learned_addr learned my addr 172.21.15.19:0/4110325344 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.505+0000 7f0b75439640 1 --2- 172.21.15.19:0/4110325344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b70072220 0x7f0b7006e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.505+0000 7f0b74c38640 1 -- 172.21.15.19:0/4110325344 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460165066 0 0) 0x7f0b7016e5a0 con 0x7f0b7016a720 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.510+0000 7f0b74c38640 1 -- 172.21.15.19:0/4110325344 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b5c003500 con 0x7f0b7016a720 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.510+0000 7f0b7643b640 1 -- 172.21.15.19:0/4110325344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b70072220 msgr2=0x7f0b7006e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.510+0000 7f0b7643b640 1 --2- 172.21.15.19:0/4110325344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b70072220 0x7f0b7006e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.510+0000 7f0b7643b640 1 -- 172.21.15.19:0/4110325344 >> v1:172.21.15.19:6789/0 conn(0x7f0b7016a720 legacy=0x7f0b70071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.510+0000 7f0b74c38640 1 -- 172.21.15.19:0/4110325344 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3959939949 0 0) 0x7f0b5c003500 con 0x7f0b7016a720 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.510+0000 7f0b7643b640 1 -- 172.21.15.19:0/4110325344 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7016eb20 con 0x7f0b7006ec30 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.511+0000 7f0b7643b640 1 --2- 172.21.15.19:0/4110325344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7006ec30 0x7f0b700cecc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800be30 tx=0x7f0b680092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a98272a8c087be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.511+0000 7f0b74c38640 1 -- 172.21.15.19:0/4110325344 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68036070 con 0x7f0b7006ec30 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.511+0000 7f0b74c38640 1 -- 172.21.15.19:0/4110325344 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6800b2e0 con 0x7f0b7006ec30 2024-04-25T09:56:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.511+0000 7f0b74c38640 1 -- 172.21.15.19:0/4110325344 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c001670 con 0x7f0b7006ec30 2024-04-25T09:56:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.511+0000 7f0b77e9e640 1 -- 172.21.15.19:0/4110325344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7006ec30 msgr2=0x7f0b700cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.511+0000 7f0b77e9e640 1 --2- 172.21.15.19:0/4110325344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7006ec30 0x7f0b700cecc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800be30 tx=0x7f0b680092f0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.511+0000 7f0b77e9e640 1 -- 172.21.15.19:0/4110325344 shutdown_connections 2024-04-25T09:56:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.511+0000 7f0b77e9e640 1 --2- 172.21.15.19:0/4110325344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7006ec30 0x7f0b700cecc0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.511+0000 7f0b77e9e640 1 --2- 172.21.15.19:0/4110325344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b70072220 0x7f0b7006e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.512+0000 7f0b77e9e640 1 -- 172.21.15.19:0/4110325344 >> 172.21.15.19:0/4110325344 conn(0x7f0b7006abe0 msgr2=0x7f0b7006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.512+0000 7f0b77e9e640 1 -- 172.21.15.19:0/4110325344 shutdown_connections 2024-04-25T09:56:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.512+0000 7f0b77e9e640 1 -- 172.21.15.19:0/4110325344 wait complete. 2024-04-25T09:56:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.513+0000 7f0b77e9e640 1 Processor -- start 2024-04-25T09:56:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.513+0000 7f0b77e9e640 1 -- start start 2024-04-25T09:56:28.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.513+0000 7f0b77e9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7006ec30 0x7f0b701640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b77e9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b70072220 0x7f0b700a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b77e9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b7016a720 0x7f0b700ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b77e9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7016ab00 con 0x7f0b7006ec30 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b77e9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7006f080 con 0x7f0b70072220 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b77e9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7016ac80 con 0x7f0b7016a720 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b75c3a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7006ec30 0x7f0b701640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b75c3a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7006ec30 0x7f0b701640e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49354/0 (socket says 172.21.15.19:49354) 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b75c3a640 1 -- 172.21.15.19:0/1792479282 learned_addr learned my addr 172.21.15.19:0/1792479282 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b75439640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b70072220 0x7f0b700a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b7643b640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b7016a720 0x7f0b700ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b75c3a640 1 -- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b7016a720 msgr2=0x7f0b700ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b75c3a640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b7016a720 0x7f0b700ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b75c3a640 1 -- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b70072220 msgr2=0x7f0b700a92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b75c3a640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b70072220 0x7f0b700a92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:28.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b75c3a640 1 -- 172.21.15.19:0/1792479282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b700c58c0 con 0x7f0b7006ec30 2024-04-25T09:56:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b7643b640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b7016a720 0x7f0b700ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b75c3a640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7006ec30 0x7f0b701640e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c700 tx=0x7f0b6c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b667fc640 1 -- 172.21.15.19:0/1792479282 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c017070 con 0x7f0b7006ec30 2024-04-25T09:56:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b667fc640 1 -- 172.21.15.19:0/1792479282 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c005ad0 con 0x7f0b7006ec30 2024-04-25T09:56:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b77e9e640 1 -- 172.21.15.19:0/1792479282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b700c5ba0 con 0x7f0b7006ec30 2024-04-25T09:56:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b667fc640 1 -- 172.21.15.19:0/1792479282 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c007070 con 0x7f0b7006ec30 2024-04-25T09:56:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.514+0000 7f0b77e9e640 1 -- 172.21.15.19:0/1792479282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b700c6160 con 0x7f0b7006ec30 2024-04-25T09:56:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.515+0000 7f0b77e9e640 1 -- 172.21.15.19:0/1792479282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b70073a00 con 0x7f0b7006ec30 2024-04-25T09:56:28.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.516+0000 7f0b667fc640 1 -- 172.21.15.19:0/1792479282 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b6c01a070 con 0x7f0b7006ec30 2024-04-25T09:56:28.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.516+0000 7f0b667fc640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b5406b4d0 0x7f0b5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:28.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.516+0000 7f0b667fc640 1 -- 172.21.15.19:0/1792479282 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0b6c0896c0 con 0x7f0b7006ec30 2024-04-25T09:56:28.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.517+0000 7f0b75439640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b5406b4d0 0x7f0b5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:28.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.517+0000 7f0b75439640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b5406b4d0 0x7f0b5406d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0b700cc340 tx=0x7f0b60005fe0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:28.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.521+0000 7f0b667fc640 1 -- 172.21.15.19:0/1792479282 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b6c057da0 con 0x7f0b7006ec30 2024-04-25T09:56:28.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:28.743+0000 7f0b77e9e640 1 -- 172.21.15.19:0/1792479282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0b700716c0 con 0x7f0b7006ec30 2024-04-25T09:56:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.389+0000 7f0b667fc640 1 -- 172.21.15.19:0/1792479282 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0b6c05be40 con 0x7f0b7006ec30 2024-04-25T09:56:29.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 -- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b5406b4d0 msgr2=0x7f0b5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:29.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b5406b4d0 0x7f0b5406d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0b700cc340 tx=0x7f0b60005fe0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 -- 172.21.15.19:0/1792479282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7006ec30 msgr2=0x7f0b701640e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:29.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7006ec30 0x7f0b701640e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c700 tx=0x7f0b6c00cbc0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 -- 172.21.15.19:0/1792479282 shutdown_connections 2024-04-25T09:56:29.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b5406b4d0 0x7f0b5406d990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b7016a720 0x7f0b700ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b70072220 0x7f0b700a92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 --2- 172.21.15.19:0/1792479282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7006ec30 0x7f0b701640e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 -- 172.21.15.19:0/1792479282 >> 172.21.15.19:0/1792479282 conn(0x7f0b7006abe0 msgr2=0x7f0b700cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:29.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 -- 172.21.15.19:0/1792479282 shutdown_connections 2024-04-25T09:56:29.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.392+0000 7f0b77e9e640 1 -- 172.21.15.19:0/1792479282 wait complete. 2024-04-25T09:56:29.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:56:29.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.560+0000 7f4666789640 1 Processor -- start 2024-04-25T09:56:29.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.560+0000 7f4666789640 1 -- start start 2024-04-25T09:56:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.561+0000 7f4666789640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46601610f0 0x7f466006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.561+0000 7f4666789640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f46601616f0 0x7f4660161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.561+0000 7f4666789640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46600ce0a0 0x7f46600ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.561+0000 7f4666789640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46601614d0 con 0x7f46601616f0 2024-04-25T09:56:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.561+0000 7f4666789640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46600c6960 con 0x7f46600ce0a0 2024-04-25T09:56:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.561+0000 7f4666789640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4660162260 con 0x7f46601610f0 2024-04-25T09:56:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.561+0000 7f465f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f46601616f0 0x7f4660161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.561+0000 7f465f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f46601616f0 0x7f4660161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49360/0 (socket says 172.21.15.19:49360) 2024-04-25T09:56:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.561+0000 7f465f7fe640 1 -- 172.21.15.19:0/2725694927 learned_addr learned my addr 172.21.15.19:0/2725694927 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.562+0000 7f4664d26640 1 --2- 172.21.15.19:0/2725694927 >> v2:172.21.15.112:3300/0 conn(0x7f46600ce0a0 0x7f46600ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.562+0000 7f465ffff640 1 --2- 172.21.15.19:0/2725694927 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46601610f0 0x7f466006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.562+0000 7f465f7fe640 1 -- 172.21.15.19:0/2725694927 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46601610f0 msgr2=0x7f466006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.562+0000 7f465f7fe640 1 --2- 172.21.15.19:0/2725694927 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46601610f0 0x7f466006c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.562+0000 7f465f7fe640 1 -- 172.21.15.19:0/2725694927 >> v2:172.21.15.112:3300/0 conn(0x7f46600ce0a0 msgr2=0x7f46600ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.562+0000 7f465f7fe640 1 --2- 172.21.15.19:0/2725694927 >> v2:172.21.15.112:3300/0 conn(0x7f46600ce0a0 0x7f46600ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.562+0000 7f465f7fe640 1 -- 172.21.15.19:0/2725694927 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4660162540 con 0x7f46601616f0 2024-04-25T09:56:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.562+0000 7f4664d26640 1 --2- 172.21.15.19:0/2725694927 >> v2:172.21.15.112:3300/0 conn(0x7f46600ce0a0 0x7f46600ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.562+0000 7f465f7fe640 1 --2- 172.21.15.19:0/2725694927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601616f0 0x7f4660161b50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f46600cee80 tx=0x7f465002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc674f13535d0dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f465effd640 1 -- 172.21.15.19:0/2725694927 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650037070 con 0x7f46601616f0 2024-04-25T09:56:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f465effd640 1 -- 172.21.15.19:0/2725694927 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4650032440 con 0x7f46601616f0 2024-04-25T09:56:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f465effd640 1 -- 172.21.15.19:0/2725694927 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650032810 con 0x7f46601616f0 2024-04-25T09:56:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f4666789640 1 -- 172.21.15.19:0/2725694927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601616f0 msgr2=0x7f4660161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f4666789640 1 --2- 172.21.15.19:0/2725694927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601616f0 0x7f4660161b50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f46600cee80 tx=0x7f465002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f4666789640 1 -- 172.21.15.19:0/2725694927 shutdown_connections 2024-04-25T09:56:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f4666789640 1 --2- 172.21.15.19:0/2725694927 >> v2:172.21.15.112:3300/0 conn(0x7f46600ce0a0 0x7f46600ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f4666789640 1 --2- 172.21.15.19:0/2725694927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601616f0 0x7f4660161b50 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f4666789640 1 --2- 172.21.15.19:0/2725694927 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46601610f0 0x7f466006c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f4666789640 1 -- 172.21.15.19:0/2725694927 >> 172.21.15.19:0/2725694927 conn(0x7f466006f500 msgr2=0x7f466006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f4666789640 1 -- 172.21.15.19:0/2725694927 shutdown_connections 2024-04-25T09:56:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.563+0000 7f4666789640 1 -- 172.21.15.19:0/2725694927 wait complete. 2024-04-25T09:56:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.565+0000 7f4666789640 1 Processor -- start 2024-04-25T09:56:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.565+0000 7f4666789640 1 -- start start 2024-04-25T09:56:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4666789640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46600ce0a0 0x7f4660150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4666789640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46601610f0 0x7f466015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4666789640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601616f0 0x7f4660160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4666789640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46600cf690 con 0x7f46601616f0 2024-04-25T09:56:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4666789640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46600cfce0 con 0x7f46600ce0a0 2024-04-25T09:56:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4666789640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466006c700 con 0x7f46601610f0 2024-04-25T09:56:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4664d26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601616f0 0x7f4660160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4664d26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601616f0 0x7f4660160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49362/0 (socket says 172.21.15.19:49362) 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4664d26640 1 -- 172.21.15.19:0/4220126105 learned_addr learned my addr 172.21.15.19:0/4220126105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f465f7fe640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46601610f0 0x7f466015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f465ffff640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46600ce0a0 0x7f4660150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4664d26640 1 -- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46601610f0 msgr2=0x7f466015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4664d26640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46601610f0 0x7f466015ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4664d26640 1 -- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46600ce0a0 msgr2=0x7f4660150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4664d26640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46600ce0a0 0x7f4660150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4664d26640 1 -- 172.21.15.19:0/4220126105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600acac0 con 0x7f46601616f0 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4664d26640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601616f0 0x7f4660160490 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4658007e40 tx=0x7f465800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f465cff9640 1 -- 172.21.15.19:0/4220126105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465801a070 con 0x7f46601616f0 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f465cff9640 1 -- 172.21.15.19:0/4220126105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4658003b90 con 0x7f46601616f0 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.566+0000 7f4666789640 1 -- 172.21.15.19:0/4220126105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46600acda0 con 0x7f46601616f0 2024-04-25T09:56:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.567+0000 7f465cff9640 1 -- 172.21.15.19:0/4220126105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465800de80 con 0x7f46601616f0 2024-04-25T09:56:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.567+0000 7f4666789640 1 -- 172.21.15.19:0/4220126105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46600ad360 con 0x7f46601616f0 2024-04-25T09:56:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.567+0000 7f4666789640 1 -- 172.21.15.19:0/4220126105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4660073a00 con 0x7f46601616f0 2024-04-25T09:56:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.568+0000 7f465cff9640 1 -- 172.21.15.19:0/4220126105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f465801d070 con 0x7f46601616f0 2024-04-25T09:56:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.568+0000 7f465cff9640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f464c06b4d0 0x7f464c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.568+0000 7f465cff9640 1 -- 172.21.15.19:0/4220126105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f465808a540 con 0x7f46601616f0 2024-04-25T09:56:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.569+0000 7f465ffff640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f464c06b4d0 0x7f464c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.569+0000 7f465ffff640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f464c06b4d0 0x7f464c06d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4654006fd0 tx=0x7f4654008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:29.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.573+0000 7f465cff9640 1 -- 172.21.15.19:0/4220126105 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4658010070 con 0x7f46601616f0 2024-04-25T09:56:29.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:29.755+0000 7f4666789640 1 -- 172.21.15.19:0/4220126105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f46600741d0 con 0x7f46601616f0 2024-04-25T09:56:30.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.390+0000 7f465cff9640 1 -- 172.21.15.19:0/4220126105 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4658058c20 con 0x7f46601616f0 2024-04-25T09:56:30.394 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:56:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.394+0000 7f4666789640 1 -- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f464c06b4d0 msgr2=0x7f464c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.394+0000 7f4666789640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f464c06b4d0 0x7f464c06d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4654006fd0 tx=0x7f4654008040 comp rx=0 tx=0).stop 2024-04-25T09:56:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.394+0000 7f4666789640 1 -- 172.21.15.19:0/4220126105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601616f0 msgr2=0x7f4660160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:30.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.394+0000 7f4666789640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601616f0 0x7f4660160490 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4658007e40 tx=0x7f465800d4a0 comp rx=0 tx=0).stop 2024-04-25T09:56:30.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.395+0000 7f4666789640 1 -- 172.21.15.19:0/4220126105 shutdown_connections 2024-04-25T09:56:30.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.395+0000 7f4666789640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f464c06b4d0 0x7f464c06d990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:30.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.395+0000 7f4666789640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46601616f0 0x7f4660160490 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:30.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.395+0000 7f4666789640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46601610f0 0x7f466015ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:30.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.395+0000 7f4666789640 1 --2- 172.21.15.19:0/4220126105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46600ce0a0 0x7f4660150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:30.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.395+0000 7f4666789640 1 -- 172.21.15.19:0/4220126105 >> 172.21.15.19:0/4220126105 conn(0x7f466006f500 msgr2=0x7f4660071800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:30.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.395+0000 7f4666789640 1 -- 172.21.15.19:0/4220126105 shutdown_connections 2024-04-25T09:56:30.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.395+0000 7f4666789640 1 -- 172.21.15.19:0/4220126105 wait complete. 2024-04-25T09:56:30.413 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:56:30.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.520+0000 7f7b5e6fc640 1 Processor -- start 2024-04-25T09:56:30.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.521+0000 7f7b5e6fc640 1 -- start start 2024-04-25T09:56:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.521+0000 7f7b5e6fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7b580cfa40 0x7f7b580cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.521+0000 7f7b5e6fc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b580c75e0 con 0x7f7b5815bb50 2024-04-25T09:56:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.521+0000 7f7b5e6fc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b580c78c0 con 0x7f7b580cf600 2024-04-25T09:56:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.521+0000 7f7b5e6fc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b580c8680 con 0x7f7b580cfa40 2024-04-25T09:56:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.521+0000 7f7b57fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7b5815bb50 0x7f7b580c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51146/0 (socket says 172.21.15.19:51146) 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b57fff640 1 -- 172.21.15.19:0/2824679043 learned_addr learned my addr 172.21.15.19:0/2824679043 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430352462 0 0) 0x7f7b580c75e0 con 0x7f7b5815bb50 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b5cc99640 1 --2- 172.21.15.19:0/2824679043 >> v2:172.21.15.112:3300/0 conn(0x7f7b580cfa40 0x7f7b580cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b40003520 con 0x7f7b5815bb50 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2604339030 0 0) 0x7f7b40003520 con 0x7f7b5815bb50 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 >> v2:172.21.15.112:3300/0 conn(0x7f7b580cfa40 msgr2=0x7f7b580cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 --2- 172.21.15.19:0/2824679043 >> v2:172.21.15.112:3300/0 conn(0x7f7b580cfa40 0x7f7b580cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 >> v1:172.21.15.112:6789/0 conn(0x7f7b580cf600 legacy=0x7f7b580cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b580c7ba0 con 0x7f7b5815bb50 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f7b48001670 con 0x7f7b5815bb50 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 >> v1:172.21.15.19:6789/0 conn(0x7f7b5815bb50 legacy=0x7f7b580c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 --2- 172.21.15.19:0/2824679043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b40003a00 0x7f7b40003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 --2- 172.21.15.19:0/2824679043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b40004800 0x7f7b40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 --2- 172.21.15.19:0/2824679043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b400051c0 0x7f7b4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580c8680 con 0x7f7b400051c0 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4000c0a0 con 0x7f7b40003a00 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4000c2a0 con 0x7f7b40004800 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b5cc99640 1 --2- 172.21.15.19:0/2824679043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b400051c0 0x7f7b4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b57fff640 1 --2- 172.21.15.19:0/2824679043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b40003a00 0x7f7b40003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.522+0000 7f7b577fe640 1 --2- 172.21.15.19:0/2824679043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b40004800 0x7f7b40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.523+0000 7f7b5cc99640 1 -- 172.21.15.19:0/2824679043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b40004800 msgr2=0x7f7b40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.523+0000 7f7b5cc99640 1 --2- 172.21.15.19:0/2824679043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b40004800 0x7f7b40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.523+0000 7f7b5cc99640 1 -- 172.21.15.19:0/2824679043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b40003a00 msgr2=0x7f7b40003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.523+0000 7f7b5cc99640 1 --2- 172.21.15.19:0/2824679043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b40003a00 0x7f7b40003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.523+0000 7f7b5cc99640 1 -- 172.21.15.19:0/2824679043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b40003520 con 0x7f7b400051c0 2024-04-25T09:56:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.523+0000 7f7b5cc99640 1 -- 172.21.15.19:0/2824679043 reap_dead start 2024-04-25T09:56:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.523+0000 7f7b5cc99640 1 --2- 172.21.15.19:0/2824679043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b400051c0 0x7f7b4000b9c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009fc0 tx=0x7f7b40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70380dcf1f7c3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.524+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50035020 con 0x7f7b400051c0 2024-04-25T09:56:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.524+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b50003bd0 con 0x7f7b400051c0 2024-04-25T09:56:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.524+0000 7f7b56ffd640 1 -- 172.21.15.19:0/2824679043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50009c10 con 0x7f7b400051c0 2024-04-25T09:56:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.524+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/2824679043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b400051c0 msgr2=0x7f7b4000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.524+0000 7f7b5e6fc640 1 --2- 172.21.15.19:0/2824679043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b400051c0 0x7f7b4000b9c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009fc0 tx=0x7f7b40005830 comp rx=0 tx=0).stop 2024-04-25T09:56:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.524+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/2824679043 shutdown_connections 2024-04-25T09:56:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.524+0000 7f7b5e6fc640 1 --2- 172.21.15.19:0/2824679043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b400051c0 0x7f7b4000b9c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.524+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/2824679043 >> 172.21.15.19:0/2824679043 conn(0x7f7b5806f500 msgr2=0x7f7b5806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.524+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/2824679043 shutdown_connections 2024-04-25T09:56:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.524+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/2824679043 wait complete. 2024-04-25T09:56:30.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.525+0000 7f7b5e6fc640 1 Processor -- start 2024-04-25T09:56:30.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.526+0000 7f7b5e6fc640 1 -- start start 2024-04-25T09:56:30.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.526+0000 7f7b5e6fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b400051c0 0x7f7b580c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.526+0000 7f7b5e6fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b581531b0 0x7f7b580b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.526+0000 7f7b5e6fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b580b5060 0x7f7b581e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.526+0000 7f7b5e6fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580cfe90 con 0x7f7b580b5060 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.526+0000 7f7b5e6fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580c8160 con 0x7f7b581531b0 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.526+0000 7f7b5e6fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5815f6d0 con 0x7f7b400051c0 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.526+0000 7f7b5cc99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b580b5060 0x7f7b581e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.526+0000 7f7b5cc99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b580b5060 0x7f7b581e71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49388/0 (socket says 172.21.15.19:49388) 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.526+0000 7f7b5cc99640 1 -- 172.21.15.19:0/1873384146 learned_addr learned my addr 172.21.15.19:0/1873384146 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.526+0000 7f7b577fe640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b581531b0 0x7f7b580b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b57fff640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b400051c0 0x7f7b580c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b5cc99640 1 -- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b400051c0 msgr2=0x7f7b580c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b5cc99640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b400051c0 0x7f7b580c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b5cc99640 1 -- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b581531b0 msgr2=0x7f7b580b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b5cc99640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b581531b0 0x7f7b580b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b5cc99640 1 -- 172.21.15.19:0/1873384146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b581e6130 con 0x7f7b580b5060 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b577fe640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b581531b0 0x7f7b580b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b5cc99640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b580b5060 0x7f7b581e71d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e230 tx=0x7f7b50009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b54ff9640 1 -- 172.21.15.19:0/1873384146 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50035030 con 0x7f7b580b5060 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b54ff9640 1 -- 172.21.15.19:0/1873384146 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b50043070 con 0x7f7b580b5060 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/1873384146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b581e6410 con 0x7f7b580b5060 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b54ff9640 1 -- 172.21.15.19:0/1873384146 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5002d070 con 0x7f7b580b5060 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.527+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/1873384146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b581e69d0 con 0x7f7b580b5060 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.528+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/1873384146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b58073a00 con 0x7f7b580b5060 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.529+0000 7f7b54ff9640 1 -- 172.21.15.19:0/1873384146 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b50005d10 con 0x7f7b580b5060 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.529+0000 7f7b54ff9640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b4406b500 0x7f7b4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.529+0000 7f7b54ff9640 1 -- 172.21.15.19:0/1873384146 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f7b500aada0 con 0x7f7b580b5060 2024-04-25T09:56:30.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.529+0000 7f7b57fff640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b4406b500 0x7f7b4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:30.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.530+0000 7f7b57fff640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b4406b500 0x7f7b4406d9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7b48001c20 tx=0x7f7b48001e00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:30.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.534+0000 7f7b54ff9640 1 -- 172.21.15.19:0/1873384146 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b50079630 con 0x7f7b580b5060 2024-04-25T09:56:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:30.712+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/1873384146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7b580b0b40 con 0x7f7b580b5060 2024-04-25T09:56:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.395+0000 7f7b54ff9640 1 -- 172.21.15.19:0/1873384146 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7b5007d6d0 con 0x7f7b580b5060 2024-04-25T09:56:31.398 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T09:56:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b4406b500 msgr2=0x7f7b4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b4406b500 0x7f7b4406d9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7b48001c20 tx=0x7f7b48001e00 comp rx=0 tx=0).stop 2024-04-25T09:56:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/1873384146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b580b5060 msgr2=0x7f7b581e71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b580b5060 0x7f7b581e71d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e230 tx=0x7f7b50009750 comp rx=0 tx=0).stop 2024-04-25T09:56:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/1873384146 shutdown_connections 2024-04-25T09:56:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7b4406b500 0x7f7b4406d9c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b580b5060 0x7f7b581e71d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b581531b0 0x7f7b580b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 --2- 172.21.15.19:0/1873384146 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b400051c0 0x7f7b580c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/1873384146 >> 172.21.15.19:0/1873384146 conn(0x7f7b5806f500 msgr2=0x7f7b58070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/1873384146 shutdown_connections 2024-04-25T09:56:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.400+0000 7f7b5e6fc640 1 -- 172.21.15.19:0/1873384146 wait complete. 2024-04-25T09:56:31.421 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:56:31.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.532+0000 7f7473415640 1 Processor -- start 2024-04-25T09:56:31.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.532+0000 7f7473415640 1 -- start start 2024-04-25T09:56:31.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7473415640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c06c9e0 0x7f746c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7473415640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746c06cf50 con 0x7f746c15b7d0 2024-04-25T09:56:31.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7473415640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746c0ce530 con 0x7f746c0b5620 2024-04-25T09:56:31.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7473415640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0ce810 con 0x7f746c06c9e0 2024-04-25T09:56:31.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f74711b1640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f746c15b7d0 0x7f746c0cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51154/0 (socket says 172.21.15.19:51154) 2024-04-25T09:56:31.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f74711b1640 1 -- 172.21.15.19:0/58245388 learned_addr learned my addr 172.21.15.19:0/58245388 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:31.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f74719b2640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c06c9e0 0x7f746c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093450426 0 0) 0x7f746c06cf50 con 0x7f746c15b7d0 2024-04-25T09:56:31.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7458003520 con 0x7f746c15b7d0 2024-04-25T09:56:31.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3917221216 0 0) 0x7f7458003520 con 0x7f746c15b7d0 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c06c9e0 msgr2=0x7f746c0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7463fff640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c06c9e0 0x7f746c0b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 >> v1:172.21.15.112:6789/0 conn(0x7f746c0b5620 legacy=0x7f746c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c0ceaf0 con 0x7f746c15b7d0 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f7468001690 con 0x7f746c15b7d0 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 >> v1:172.21.15.19:6789/0 conn(0x7f746c15b7d0 legacy=0x7f746c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.533+0000 7f7463fff640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c06c9e0 0x7f74580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f7463fff640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7458003f20 0x7f7458004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f7463fff640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7458004d40 0x7f74580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0ce810 con 0x7f7458003f20 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745800ba20 con 0x7f7458004d40 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745800bc20 con 0x7f746c06c9e0 2024-04-25T09:56:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f74709b0640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7458003f20 0x7f7458004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f74711b1640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c06c9e0 0x7f74580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f74719b2640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7458004d40 0x7f74580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f74709b0640 1 -- 172.21.15.19:0/58245388 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c06c9e0 msgr2=0x7f74580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f74709b0640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c06c9e0 0x7f74580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f74709b0640 1 -- 172.21.15.19:0/58245388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7458004d40 msgr2=0x7f74580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f74709b0640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7458004d40 0x7f74580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.534+0000 7f74709b0640 1 -- 172.21.15.19:0/58245388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7458003520 con 0x7f7458003f20 2024-04-25T09:56:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.535+0000 7f74709b0640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7458003f20 0x7f7458004800 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f745c001910 tx=0x7f7458005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84bafa460c4acdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.535+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c037070 con 0x7f7458003f20 2024-04-25T09:56:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.535+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c02abd0 con 0x7f7458003f20 2024-04-25T09:56:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.535+0000 7f7463fff640 1 -- 172.21.15.19:0/58245388 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c0014e0 con 0x7f7458003f20 2024-04-25T09:56:31.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.535+0000 7f7473415640 1 -- 172.21.15.19:0/58245388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7458003f20 msgr2=0x7f7458004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.535+0000 7f7473415640 1 --2- 172.21.15.19:0/58245388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7458003f20 0x7f7458004800 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f745c001910 tx=0x7f7458005830 comp rx=0 tx=0).stop 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.535+0000 7f74719b2640 1 -- 172.21.15.19:0/58245388 reap_dead start 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.535+0000 7f7473415640 1 -- 172.21.15.19:0/58245388 shutdown_connections 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.535+0000 7f7473415640 1 -- 172.21.15.19:0/58245388 >> 172.21.15.19:0/58245388 conn(0x7f746c0d0800 msgr2=0x7f746c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.535+0000 7f7473415640 1 -- 172.21.15.19:0/58245388 shutdown_connections 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.535+0000 7f7473415640 1 -- 172.21.15.19:0/58245388 wait complete. 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.537+0000 7f7473415640 1 Processor -- start 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.537+0000 7f7473415640 1 -- start start 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.537+0000 7f7473415640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7458003f20 0x7f746c1f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.537+0000 7f7473415640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0ca250 0x7f746c0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.537+0000 7f7473415640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c158180 0x7f746c15fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.537+0000 7f7473415640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c15bd30 con 0x7f746c0ca250 2024-04-25T09:56:31.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.537+0000 7f7473415640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c15bbb0 con 0x7f746c158180 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.537+0000 7f7473415640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0b5ab0 con 0x7f7458003f20 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.537+0000 7f74709b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0ca250 0x7f746c0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f74709b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0ca250 0x7f746c0cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49404/0 (socket says 172.21.15.19:49404) 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f74709b0640 1 -- 172.21.15.19:0/1406395807 learned_addr learned my addr 172.21.15.19:0/1406395807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f74719b2640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c158180 0x7f746c15fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f74711b1640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7458003f20 0x7f746c1f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f74709b0640 1 -- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7458003f20 msgr2=0x7f746c1f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f74709b0640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7458003f20 0x7f746c1f6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f74709b0640 1 -- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c158180 msgr2=0x7f746c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f74709b0640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c158180 0x7f746c15fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f74709b0640 1 -- 172.21.15.19:0/1406395807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c1f7100 con 0x7f746c0ca250 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f74719b2640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c158180 0x7f746c15fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:31.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f74709b0640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0ca250 0x7f746c0cac70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f745c001970 tx=0x7f745c0012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f7461ffb640 1 -- 172.21.15.19:0/1406395807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c037040 con 0x7f746c0ca250 2024-04-25T09:56:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f7461ffb640 1 -- 172.21.15.19:0/1406395807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c0052e0 con 0x7f746c0ca250 2024-04-25T09:56:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f7473415640 1 -- 172.21.15.19:0/1406395807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c1f73e0 con 0x7f746c0ca250 2024-04-25T09:56:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f7461ffb640 1 -- 172.21.15.19:0/1406395807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c007240 con 0x7f746c0ca250 2024-04-25T09:56:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.538+0000 7f7473415640 1 -- 172.21.15.19:0/1406395807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c158600 con 0x7f746c0ca250 2024-04-25T09:56:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.539+0000 7f7473415640 1 -- 172.21.15.19:0/1406395807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746c073a00 con 0x7f746c0ca250 2024-04-25T09:56:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.540+0000 7f7461ffb640 1 -- 172.21.15.19:0/1406395807 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f745c03a070 con 0x7f746c0ca250 2024-04-25T09:56:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.540+0000 7f7461ffb640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745006b500 0x7f745006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.540+0000 7f7461ffb640 1 -- 172.21.15.19:0/1406395807 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f745c0aa080 con 0x7f746c0ca250 2024-04-25T09:56:31.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.541+0000 7f74711b1640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745006b500 0x7f745006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.541+0000 7f74711b1640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745006b500 0x7f745006d9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f746c0b91c0 tx=0x7f7468009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.544+0000 7f7461ffb640 1 -- 172.21.15.19:0/1406395807 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f745c079060 con 0x7f746c0ca250 2024-04-25T09:56:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.726+0000 7f7473415640 1 -- 172.21.15.19:0/1406395807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f746c1f7900 con 0x7f746c0ca250 2024-04-25T09:56:31.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.729+0000 7f7461ffb640 1 -- 172.21.15.19:0/1406395807 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f745c078960 con 0x7f746c0ca250 2024-04-25T09:56:31.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 -- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745006b500 msgr2=0x7f745006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745006b500 0x7f745006d9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f746c0b91c0 tx=0x7f7468009040 comp rx=0 tx=0).stop 2024-04-25T09:56:31.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 -- 172.21.15.19:0/1406395807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0ca250 msgr2=0x7f746c0cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0ca250 0x7f746c0cac70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f745c001970 tx=0x7f745c0012b0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 -- 172.21.15.19:0/1406395807 shutdown_connections 2024-04-25T09:56:31.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745006b500 0x7f745006d9c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c158180 0x7f746c15fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0ca250 0x7f746c0cac70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 --2- 172.21.15.19:0/1406395807 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7458003f20 0x7f746c1f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 -- 172.21.15.19:0/1406395807 >> 172.21.15.19:0/1406395807 conn(0x7f746c0d0800 msgr2=0x7f746c0cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:31.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 -- 172.21.15.19:0/1406395807 shutdown_connections 2024-04-25T09:56:31.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.731+0000 7f7473415640 1 -- 172.21.15.19:0/1406395807 wait complete. 2024-04-25T09:56:31.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T09:56:31.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.848+0000 7f52b4a22640 1 Processor -- start 2024-04-25T09:56:31.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.848+0000 7f52b4a22640 1 -- start start 2024-04-25T09:56:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52b4a22640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52b015dd30 0x7f52b00b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52b4a22640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52b00d0ad0 0x7f52b00b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52b4a22640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b00d1df0 con 0x7f52b00b9770 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52b4a22640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b00d20d0 con 0x7f52b015dd30 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52b4a22640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00d23b0 con 0x7f52b00d0ad0 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52ae59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52b015dd30 0x7f52b00b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52add9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f52b00b9770 0x7f52b00d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51166/0 (socket says 172.21.15.19:51166) 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52add9b640 1 -- 172.21.15.19:0/3554155346 learned_addr learned my addr 172.21.15.19:0/3554155346 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52aed9d640 1 --2- 172.21.15.19:0/3554155346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52b00d0ad0 0x7f52b00b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52ad59a640 1 -- 172.21.15.19:0/3554155346 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876444759 0 0) 0x7f52b00d1df0 con 0x7f52b00b9770 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52ad59a640 1 -- 172.21.15.19:0/3554155346 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5298003520 con 0x7f52b00b9770 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52ae59c640 1 -- 172.21.15.19:0/3554155346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52b00d0ad0 msgr2=0x7f52b00b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52ae59c640 1 --2- 172.21.15.19:0/3554155346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52b00d0ad0 0x7f52b00b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52ae59c640 1 -- 172.21.15.19:0/3554155346 >> v1:172.21.15.19:6789/0 conn(0x7f52b00b9770 legacy=0x7f52b00d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.849+0000 7f52ae59c640 1 -- 172.21.15.19:0/3554155346 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b00d2690 con 0x7f52b015dd30 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.850+0000 7f52ae59c640 1 --2- 172.21.15.19:0/3554155346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b015dd30 0x7f52b00b9230 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f52a00097a0 tx=0x7f52a002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa010c33adb5d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.850+0000 7f52ad59a640 1 -- 172.21.15.19:0/3554155346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0037070 con 0x7f52b015dd30 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.850+0000 7f52ad59a640 1 -- 172.21.15.19:0/3554155346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a002ae50 con 0x7f52b015dd30 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.850+0000 7f52ad59a640 1 -- 172.21.15.19:0/3554155346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0032660 con 0x7f52b015dd30 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.851+0000 7f52b4a22640 1 -- 172.21.15.19:0/3554155346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b015dd30 msgr2=0x7f52b00b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.851+0000 7f52b4a22640 1 --2- 172.21.15.19:0/3554155346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b015dd30 0x7f52b00b9230 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f52a00097a0 tx=0x7f52a002a100 comp rx=0 tx=0).stop 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.851+0000 7f52b4a22640 1 -- 172.21.15.19:0/3554155346 shutdown_connections 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.851+0000 7f52b4a22640 1 --2- 172.21.15.19:0/3554155346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52b00d0ad0 0x7f52b00b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.851+0000 7f52b4a22640 1 --2- 172.21.15.19:0/3554155346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b015dd30 0x7f52b00b9230 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.851+0000 7f52b4a22640 1 -- 172.21.15.19:0/3554155346 >> 172.21.15.19:0/3554155346 conn(0x7f52b0061c30 msgr2=0x7f52b006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.851+0000 7f52b4a22640 1 -- 172.21.15.19:0/3554155346 shutdown_connections 2024-04-25T09:56:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.851+0000 7f52b4a22640 1 -- 172.21.15.19:0/3554155346 wait complete. 2024-04-25T09:56:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.852+0000 7f52b4a22640 1 Processor -- start 2024-04-25T09:56:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.852+0000 7f52b4a22640 1 -- start start 2024-04-25T09:56:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52b4a22640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52b00b9770 0x7f52b0160720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52b4a22640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b00d0ad0 0x7f52b0152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52b4a22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52b015dd30 0x7f52b0156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52b4a22640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00d1b90 con 0x7f52b00d0ad0 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52b4a22640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00d1a10 con 0x7f52b015dd30 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52b4a22640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00d2e90 con 0x7f52b00b9770 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52add9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b00d0ad0 0x7f52b0152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52add9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b00d0ad0 0x7f52b0152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49426/0 (socket says 172.21.15.19:49426) 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52add9b640 1 -- 172.21.15.19:0/3057610266 learned_addr learned my addr 172.21.15.19:0/3057610266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52ae59c640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52b00b9770 0x7f52b0160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52aed9d640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52b015dd30 0x7f52b0156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52add9b640 1 -- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52b00b9770 msgr2=0x7f52b0160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52add9b640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52b00b9770 0x7f52b0160720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52add9b640 1 -- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52b015dd30 msgr2=0x7f52b0156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52add9b640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52b015dd30 0x7f52b0156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52add9b640 1 -- 172.21.15.19:0/3057610266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b00b40b0 con 0x7f52b00d0ad0 2024-04-25T09:56:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52ae59c640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52b00b9770 0x7f52b0160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.853+0000 7f52add9b640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b00d0ad0 0x7f52b0152ed0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c700 tx=0x7f52a400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.854+0000 7f529effd640 1 -- 172.21.15.19:0/3057610266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4017070 con 0x7f52b00d0ad0 2024-04-25T09:56:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.854+0000 7f529effd640 1 -- 172.21.15.19:0/3057610266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4007290 con 0x7f52b00d0ad0 2024-04-25T09:56:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.854+0000 7f529effd640 1 -- 172.21.15.19:0/3057610266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4001d40 con 0x7f52b00d0ad0 2024-04-25T09:56:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.854+0000 7f52b4a22640 1 -- 172.21.15.19:0/3057610266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b0160d90 con 0x7f52b00d0ad0 2024-04-25T09:56:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.854+0000 7f52b4a22640 1 -- 172.21.15.19:0/3057610266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b00b4350 con 0x7f52b00d0ad0 2024-04-25T09:56:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.855+0000 7f52b4a22640 1 -- 172.21.15.19:0/3057610266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b0073990 con 0x7f52b00d0ad0 2024-04-25T09:56:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.855+0000 7f529effd640 1 -- 172.21.15.19:0/3057610266 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52a401a070 con 0x7f52b00d0ad0 2024-04-25T09:56:31.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.856+0000 7f529effd640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f529006b500 0x7f529006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:31.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.856+0000 7f529effd640 1 -- 172.21.15.19:0/3057610266 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f52a4089360 con 0x7f52b00d0ad0 2024-04-25T09:56:31.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.856+0000 7f52ae59c640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f529006b500 0x7f529006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:31.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.856+0000 7f52ae59c640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f529006b500 0x7f529006d9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f52a0034000 tx=0x7f52a002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:31.860+0000 7f529effd640 1 -- 172.21.15.19:0/3057610266 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f52a40583e0 con 0x7f52b00d0ad0 2024-04-25T09:56:32.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.116+0000 7f52b4a22640 1 -- 172.21.15.19:0/3057610266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f52b00b6d80 con 0x7f52b00d0ad0 2024-04-25T09:56:32.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.397+0000 7f529effd640 1 -- 172.21.15.19:0/3057610266 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v184) v1 ==== 143+0+0 (secure 0 0 0) 0x7f52a4087e20 con 0x7f52b00d0ad0 2024-04-25T09:56:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 -- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f529006b500 msgr2=0x7f529006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f529006b500 0x7f529006d9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f52a0034000 tx=0x7f52a002e000 comp rx=0 tx=0).stop 2024-04-25T09:56:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 -- 172.21.15.19:0/3057610266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b00d0ad0 msgr2=0x7f52b0152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b00d0ad0 0x7f52b0152ed0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c700 tx=0x7f52a400cbc0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 -- 172.21.15.19:0/3057610266 shutdown_connections 2024-04-25T09:56:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f529006b500 0x7f529006d9c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52b015dd30 0x7f52b0156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b00d0ad0 0x7f52b0152ed0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 --2- 172.21.15.19:0/3057610266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52b00b9770 0x7f52b0160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 -- 172.21.15.19:0/3057610266 >> 172.21.15.19:0/3057610266 conn(0x7f52b0061c30 msgr2=0x7f52b00c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 -- 172.21.15.19:0/3057610266 shutdown_connections 2024-04-25T09:56:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.399+0000 7f52b4a22640 1 -- 172.21.15.19:0/3057610266 wait complete. 2024-04-25T09:56:32.414 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T09:56:32.414 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:56:32.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:32.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.513+0000 7f3807ccd640 1 Processor -- start 2024-04-25T09:56:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.513+0000 7f3807ccd640 1 -- start start 2024-04-25T09:56:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.514+0000 7f3807ccd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38000afba0 0x7f38000d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.514+0000 7f3807ccd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380006fe00 con 0x7f3800158220 2024-04-25T09:56:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.514+0000 7f3807ccd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38000700e0 con 0x7f38000af720 2024-04-25T09:56:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.514+0000 7f3807ccd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38000b3670 con 0x7f38000afba0 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.514+0000 7f3805a69640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3800158220 0x7f380015ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51180/0 (socket says 172.21.15.19:51180) 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.514+0000 7f3805a69640 1 -- 172.21.15.19:0/1804212694 learned_addr learned my addr 172.21.15.19:0/1804212694 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f380626a640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38000afba0 0x7f38000d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450104630 0 0) 0x7f380006fe00 con 0x7f3800158220 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e8003520 con 0x7f3800158220 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2682239544 0 0) 0x7f37e8003520 con 0x7f3800158220 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38000afba0 msgr2=0x7f38000d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38000afba0 0x7f38000d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 >> v1:172.21.15.112:6789/0 conn(0x7f38000af720 legacy=0x7f38000d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38000703c0 con 0x7f3800158220 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f37f8001670 con 0x7f3800158220 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 >> v1:172.21.15.19:6789/0 conn(0x7f3800158220 legacy=0x7f380015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e80039e0 0x7f37e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38000afba0 0x7f37e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e8004d40 0x7f37e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e800b9c0 con 0x7f37e80039e0 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e800bb40 con 0x7f37e8004d40 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e800bd40 con 0x7f38000afba0 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3805a69640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e80039e0 0x7f37e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f380626a640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e8004d40 0x7f37e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3805268640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38000afba0 0x7f37e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3805a69640 1 -- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38000afba0 msgr2=0x7f37e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3805a69640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38000afba0 0x7f37e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3805a69640 1 -- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e8004d40 msgr2=0x7f37e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3805a69640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e8004d40 0x7f37e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3805a69640 1 -- 172.21.15.19:0/1804212694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e8003520 con 0x7f37e80039e0 2024-04-25T09:56:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f3805268640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38000afba0 0x7f37e8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.515+0000 7f380626a640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e8004d40 0x7f37e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.516+0000 7f3805a69640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e80039e0 0x7f37e8003dc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f37f800c190 tx=0x7f37e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d562426ef9c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.516+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8007cc0 con 0x7f37e80039e0 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.516+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f8005ad0 con 0x7f37e80039e0 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.516+0000 7f3804a67640 1 -- 172.21.15.19:0/1804212694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8007070 con 0x7f37e80039e0 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.517+0000 7f3807ccd640 1 -- 172.21.15.19:0/1804212694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e80039e0 msgr2=0x7f37e8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.517+0000 7f3807ccd640 1 --2- 172.21.15.19:0/1804212694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37e80039e0 0x7f37e8003dc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f37f800c190 tx=0x7f37e8005830 comp rx=0 tx=0).stop 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.517+0000 7f380626a640 1 -- 172.21.15.19:0/1804212694 reap_dead start 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.517+0000 7f3807ccd640 1 -- 172.21.15.19:0/1804212694 shutdown_connections 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.517+0000 7f3807ccd640 1 -- 172.21.15.19:0/1804212694 >> 172.21.15.19:0/1804212694 conn(0x7f38000b2370 msgr2=0x7f38000b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.517+0000 7f3807ccd640 1 -- 172.21.15.19:0/1804212694 shutdown_connections 2024-04-25T09:56:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.517+0000 7f3807ccd640 1 -- 172.21.15.19:0/1804212694 wait complete. 2024-04-25T09:56:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.518+0000 7f3807ccd640 1 Processor -- start 2024-04-25T09:56:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.518+0000 7f3807ccd640 1 -- start start 2024-04-25T09:56:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f3807ccd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37e80039e0 0x7f380015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f3807ccd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38000cb5f0 0x7f380014a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f3807ccd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f380015f690 0x7f380015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f3807ccd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000b0010 con 0x7f380015f690 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f3807ccd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3800070a10 con 0x7f38000cb5f0 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f3807ccd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3800158600 con 0x7f37e80039e0 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f380626a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f380015f690 0x7f380015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f380626a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f380015f690 0x7f380015fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49442/0 (socket says 172.21.15.19:49442) 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f380626a640 1 -- 172.21.15.19:0/1436670085 learned_addr learned my addr 172.21.15.19:0/1436670085 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f3805268640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38000cb5f0 0x7f380014a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f3805a69640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37e80039e0 0x7f380015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f380626a640 1 -- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37e80039e0 msgr2=0x7f380015d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f380626a640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37e80039e0 0x7f380015d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f380626a640 1 -- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38000cb5f0 msgr2=0x7f380014a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f380626a640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38000cb5f0 0x7f380014a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.519+0000 7f380626a640 1 -- 172.21.15.19:0/1436670085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380015e0d0 con 0x7f380015f690 2024-04-25T09:56:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.520+0000 7f380626a640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f380015f690 0x7f380015fb10 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f38000b5300 tx=0x7f37f0010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.520+0000 7f37fe7fc640 1 -- 172.21.15.19:0/1436670085 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0010d80 con 0x7f380015f690 2024-04-25T09:56:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.520+0000 7f37fe7fc640 1 -- 172.21.15.19:0/1436670085 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f0003990 con 0x7f380015f690 2024-04-25T09:56:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.520+0000 7f3807ccd640 1 -- 172.21.15.19:0/1436670085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380015e3b0 con 0x7f380015f690 2024-04-25T09:56:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.520+0000 7f37fe7fc640 1 -- 172.21.15.19:0/1436670085 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0003dc0 con 0x7f380015f690 2024-04-25T09:56:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.520+0000 7f3807ccd640 1 -- 172.21.15.19:0/1436670085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380015e970 con 0x7f380015f690 2024-04-25T09:56:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.521+0000 7f3807ccd640 1 -- 172.21.15.19:0/1436670085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380006fe00 con 0x7f380015f690 2024-04-25T09:56:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.521+0000 7f37fe7fc640 1 -- 172.21.15.19:0/1436670085 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37f0005d10 con 0x7f380015f690 2024-04-25T09:56:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.522+0000 7f37fe7fc640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37ec06b500 0x7f37ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.522+0000 7f37fe7fc640 1 -- 172.21.15.19:0/1436670085 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f37f008a650 con 0x7f380015f690 2024-04-25T09:56:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.522+0000 7f3805a69640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37ec06b500 0x7f37ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.522+0000 7f3805a69640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37ec06b500 0x7f37ec06d9c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f37f8032000 tx=0x7f37f802d000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.526+0000 7f37fe7fc640 1 -- 172.21.15.19:0/1436670085 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f380006fe00 con 0x7f380015f690 2024-04-25T09:56:32.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.705+0000 7f3807ccd640 1 -- 172.21.15.19:0/1436670085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3800073990 con 0x7f380015f690 2024-04-25T09:56:32.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.706+0000 7f37fe7fc640 1 -- 172.21.15.19:0/1436670085 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+13281 (secure 0 0 0) 0x7f37f0025020 con 0x7f380015f690 2024-04-25T09:56:32.709 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:32.709 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":56,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:56:31.392137+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/2436445353":"2024-04-26T09:56:28.384375+0000","172.21.15.19:6834/2436445353":"2024-04-26T09:56:28.384375+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 -- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37ec06b500 msgr2=0x7f37ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37ec06b500 0x7f37ec06d9c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f37f8032000 tx=0x7f37f802d000 comp rx=0 tx=0).stop 2024-04-25T09:56:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 -- 172.21.15.19:0/1436670085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f380015f690 msgr2=0x7f380015fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f380015f690 0x7f380015fb10 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f38000b5300 tx=0x7f37f0010590 comp rx=0 tx=0).stop 2024-04-25T09:56:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 -- 172.21.15.19:0/1436670085 shutdown_connections 2024-04-25T09:56:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f37ec06b500 0x7f37ec06d9c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f380015f690 0x7f380015fb10 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38000cb5f0 0x7f380014a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 --2- 172.21.15.19:0/1436670085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37e80039e0 0x7f380015d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 -- 172.21.15.19:0/1436670085 >> 172.21.15.19:0/1436670085 conn(0x7f38000b2370 msgr2=0x7f380006b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 -- 172.21.15.19:0/1436670085 shutdown_connections 2024-04-25T09:56:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.708+0000 7f3807ccd640 1 -- 172.21.15.19:0/1436670085 wait complete. 2024-04-25T09:56:32.722 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:56:32.723 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:56:32.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.824+0000 7fcb995e0640 1 Processor -- start 2024-04-25T09:56:32.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.824+0000 7fcb995e0640 1 -- start start 2024-04-25T09:56:32.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.824+0000 7fcb995e0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb9415dc20 0x7fcb940ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.824+0000 7fcb995e0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb94161710 0x7fcb940cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.824+0000 7fcb995e0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb9415e000 con 0x7fcb940b9330 2024-04-25T09:56:32.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.824+0000 7fcb995e0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb940d1de0 con 0x7fcb94161710 2024-04-25T09:56:32.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.824+0000 7fcb995e0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb94161c60 con 0x7fcb9415dc20 2024-04-25T09:56:32.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.825+0000 7fcb927fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb94161710 0x7fcb940cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.825+0000 7fcb927fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb94161710 0x7fcb940cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49452/0 (socket says 172.21.15.19:49452) 2024-04-25T09:56:32.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.825+0000 7fcb927fc640 1 -- 172.21.15.19:0/2704895785 learned_addr learned my addr 172.21.15.19:0/2704895785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:32.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.825+0000 7fcb927fc640 1 -- 172.21.15.19:0/2704895785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb9415dc20 msgr2=0x7fcb940ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.825+0000 7fcb92ffd640 1 --2- 172.21.15.19:0/2704895785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb9415dc20 0x7fcb940ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.825+0000 7fcb927fc640 1 --2- 172.21.15.19:0/2704895785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb9415dc20 0x7fcb940ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.825+0000 7fcb927fc640 1 -- 172.21.15.19:0/2704895785 >> v1:172.21.15.112:6789/0 conn(0x7fcb940b9330 legacy=0x7fcb9406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.825+0000 7fcb927fc640 1 -- 172.21.15.19:0/2704895785 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb94161f40 con 0x7fcb94161710 2024-04-25T09:56:32.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb927fc640 1 --2- 172.21.15.19:0/2704895785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb94161710 0x7fcb940cebb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009870 tx=0x7fcb8802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a97b8ef737eb2b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb91ffb640 1 -- 172.21.15.19:0/2704895785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88037070 con 0x7fcb94161710 2024-04-25T09:56:32.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb91ffb640 1 -- 172.21.15.19:0/2704895785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb88032440 con 0x7fcb94161710 2024-04-25T09:56:32.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb91ffb640 1 -- 172.21.15.19:0/2704895785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88032810 con 0x7fcb94161710 2024-04-25T09:56:32.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb995e0640 1 -- 172.21.15.19:0/2704895785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb94161710 msgr2=0x7fcb940cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb995e0640 1 --2- 172.21.15.19:0/2704895785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb94161710 0x7fcb940cebb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009870 tx=0x7fcb8802a210 comp rx=0 tx=0).stop 2024-04-25T09:56:32.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb995e0640 1 -- 172.21.15.19:0/2704895785 shutdown_connections 2024-04-25T09:56:32.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb995e0640 1 --2- 172.21.15.19:0/2704895785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb94161710 0x7fcb940cebb0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb995e0640 1 --2- 172.21.15.19:0/2704895785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb9415dc20 0x7fcb940ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb995e0640 1 -- 172.21.15.19:0/2704895785 >> 172.21.15.19:0/2704895785 conn(0x7fcb940d0420 msgr2=0x7fcb940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:32.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb995e0640 1 -- 172.21.15.19:0/2704895785 shutdown_connections 2024-04-25T09:56:32.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.826+0000 7fcb995e0640 1 -- 172.21.15.19:0/2704895785 wait complete. 2024-04-25T09:56:32.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.828+0000 7fcb995e0640 1 Processor -- start 2024-04-25T09:56:32.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.828+0000 7fcb995e0640 1 -- start start 2024-04-25T09:56:32.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.828+0000 7fcb995e0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb940b9330 0x7fcb940b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.828+0000 7fcb995e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb9415dc20 0x7fcb940cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.828+0000 7fcb995e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb94161710 0x7fcb940cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.828+0000 7fcb995e0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940b5e50 con 0x7fcb9415dc20 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.828+0000 7fcb995e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb94162500 con 0x7fcb94161710 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.828+0000 7fcb995e0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940b5fd0 con 0x7fcb940b9330 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb9415dc20 0x7fcb940cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb9415dc20 0x7fcb940cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49466/0 (socket says 172.21.15.19:49466) 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb927fc640 1 -- 172.21.15.19:0/3854373428 learned_addr learned my addr 172.21.15.19:0/3854373428 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb937fe640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb94161710 0x7fcb940cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb92ffd640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb940b9330 0x7fcb940b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb927fc640 1 -- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb940b9330 msgr2=0x7fcb940b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb927fc640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb940b9330 0x7fcb940b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb927fc640 1 -- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb94161710 msgr2=0x7fcb940cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb927fc640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb94161710 0x7fcb940cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb927fc640 1 -- 172.21.15.19:0/3854373428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb940cdb00 con 0x7fcb9415dc20 2024-04-25T09:56:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb92ffd640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb940b9330 0x7fcb940b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb927fc640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb9415dc20 0x7fcb940cd080 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000c00 tx=0x7fcb880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb7b7fe640 1 -- 172.21.15.19:0/3854373428 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88037040 con 0x7fcb9415dc20 2024-04-25T09:56:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb7b7fe640 1 -- 172.21.15.19:0/3854373428 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb88042070 con 0x7fcb9415dc20 2024-04-25T09:56:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb995e0640 1 -- 172.21.15.19:0/3854373428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb940cbb60 con 0x7fcb9415dc20 2024-04-25T09:56:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb7b7fe640 1 -- 172.21.15.19:0/3854373428 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88030bd0 con 0x7fcb9415dc20 2024-04-25T09:56:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.829+0000 7fcb995e0640 1 -- 172.21.15.19:0/3854373428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb940cc120 con 0x7fcb9415dc20 2024-04-25T09:56:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.830+0000 7fcb995e0640 1 -- 172.21.15.19:0/3854373428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb94073a00 con 0x7fcb9415dc20 2024-04-25T09:56:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.831+0000 7fcb7b7fe640 1 -- 172.21.15.19:0/3854373428 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb88047460 con 0x7fcb9415dc20 2024-04-25T09:56:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.831+0000 7fcb7b7fe640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb8006b4d0 0x7fcb8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.831+0000 7fcb7b7fe640 1 -- 172.21.15.19:0/3854373428 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcb880aad30 con 0x7fcb9415dc20 2024-04-25T09:56:32.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.831+0000 7fcb92ffd640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb8006b4d0 0x7fcb8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.832+0000 7fcb92ffd640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb8006b4d0 0x7fcb8006d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400b820 tx=0x7fcb84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:32.836+0000 7fcb7b7fe640 1 -- 172.21.15.19:0/3854373428 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb88045360 con 0x7fcb9415dc20 2024-04-25T09:56:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.015+0000 7fcb995e0640 1 -- 172.21.15.19:0/3854373428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb9406b010 con 0x7fcb9415dc20 2024-04-25T09:56:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.015+0000 7fcb7b7fe640 1 -- 172.21.15.19:0/3854373428 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb88079d20 con 0x7fcb9415dc20 2024-04-25T09:56:33.019 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:56:33.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 -- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb8006b4d0 msgr2=0x7fcb8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb8006b4d0 0x7fcb8006d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400b820 tx=0x7fcb84008040 comp rx=0 tx=0).stop 2024-04-25T09:56:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 -- 172.21.15.19:0/3854373428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb9415dc20 msgr2=0x7fcb940cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb9415dc20 0x7fcb940cd080 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000c00 tx=0x7fcb880046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 -- 172.21.15.19:0/3854373428 shutdown_connections 2024-04-25T09:56:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb8006b4d0 0x7fcb8006d990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb94161710 0x7fcb940cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb9415dc20 0x7fcb940cd080 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 --2- 172.21.15.19:0/3854373428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb940b9330 0x7fcb940b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 -- 172.21.15.19:0/3854373428 >> 172.21.15.19:0/3854373428 conn(0x7fcb940d0420 msgr2=0x7fcb940b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 -- 172.21.15.19:0/3854373428 shutdown_connections 2024-04-25T09:56:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.017+0000 7fcb995e0640 1 -- 172.21.15.19:0/3854373428 wait complete. 2024-04-25T09:56:33.032 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T09:56:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.129+0000 7f38bb89a640 1 Processor -- start 2024-04-25T09:56:33.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.130+0000 7f38bb89a640 1 -- start start 2024-04-25T09:56:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.130+0000 7f38bb89a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38b415ba00 0x7f38b409a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.130+0000 7f38bb89a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f38b415f5f0 0x7f38b415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.130+0000 7f38bb89a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b40cd710 con 0x7f38b41602e0 2024-04-25T09:56:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.130+0000 7f38bb89a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b415ff80 con 0x7f38b415ba00 2024-04-25T09:56:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.130+0000 7f38bb89a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b40cd4b0 con 0x7f38b415f5f0 2024-04-25T09:56:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.130+0000 7f38b9636640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38b415ba00 0x7f38b409a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.131+0000 7f38b9636640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38b415ba00 0x7f38b409a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49478/0 (socket says 172.21.15.19:49478) 2024-04-25T09:56:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.131+0000 7f38b9636640 1 -- 172.21.15.19:0/1824894952 learned_addr learned my addr 172.21.15.19:0/1824894952 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.131+0000 7f38b9636640 1 -- 172.21.15.19:0/1824894952 >> v2:172.21.15.112:3300/0 conn(0x7f38b415f5f0 msgr2=0x7f38b415fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.131+0000 7f38b9e37640 1 --2- 172.21.15.19:0/1824894952 >> v2:172.21.15.112:3300/0 conn(0x7f38b415f5f0 0x7f38b415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.131+0000 7f38b9636640 1 --2- 172.21.15.19:0/1824894952 >> v2:172.21.15.112:3300/0 conn(0x7f38b415f5f0 0x7f38b415fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.131+0000 7f38b9636640 1 -- 172.21.15.19:0/1824894952 >> v1:172.21.15.112:6789/0 conn(0x7f38b41602e0 legacy=0x7f38b40ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.131+0000 7f38b9636640 1 -- 172.21.15.19:0/1824894952 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b4099b70 con 0x7f38b415ba00 2024-04-25T09:56:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.131+0000 7f38b9e37640 1 --2- 172.21.15.19:0/1824894952 >> v2:172.21.15.112:3300/0 conn(0x7f38b415f5f0 0x7f38b415fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.132+0000 7f38b9636640 1 --2- 172.21.15.19:0/1824894952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38b415ba00 0x7f38b409a930 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f38b4072b10 tx=0x7f38b002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a4a0ab7fd19fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.132+0000 7f38abfff640 1 -- 172.21.15.19:0/1824894952 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b0037070 con 0x7f38b415ba00 2024-04-25T09:56:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.132+0000 7f38abfff640 1 -- 172.21.15.19:0/1824894952 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b002ae40 con 0x7f38b415ba00 2024-04-25T09:56:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.132+0000 7f38abfff640 1 -- 172.21.15.19:0/1824894952 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b0032660 con 0x7f38b415ba00 2024-04-25T09:56:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.132+0000 7f38bb89a640 1 -- 172.21.15.19:0/1824894952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38b415ba00 msgr2=0x7f38b409a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.132+0000 7f38bb89a640 1 --2- 172.21.15.19:0/1824894952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38b415ba00 0x7f38b409a930 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f38b4072b10 tx=0x7f38b002a130 comp rx=0 tx=0).stop 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.132+0000 7f38bb89a640 1 -- 172.21.15.19:0/1824894952 shutdown_connections 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.132+0000 7f38bb89a640 1 --2- 172.21.15.19:0/1824894952 >> v2:172.21.15.112:3300/0 conn(0x7f38b415f5f0 0x7f38b415fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.132+0000 7f38bb89a640 1 --2- 172.21.15.19:0/1824894952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38b415ba00 0x7f38b409a930 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.132+0000 7f38bb89a640 1 -- 172.21.15.19:0/1824894952 >> 172.21.15.19:0/1824894952 conn(0x7f38b40b3270 msgr2=0x7f38b40b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.133+0000 7f38bb89a640 1 -- 172.21.15.19:0/1824894952 shutdown_connections 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.133+0000 7f38bb89a640 1 -- 172.21.15.19:0/1824894952 wait complete. 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.134+0000 7f38bb89a640 1 Processor -- start 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.134+0000 7f38bb89a640 1 -- start start 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.134+0000 7f38bb89a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b415ba00 0x7f38b4157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.134+0000 7f38bb89a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38b415f5f0 0x7f38b40c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.134+0000 7f38bb89a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38b41602e0 0x7f38b415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.134+0000 7f38bb89a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b409ae70 con 0x7f38b41602e0 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.134+0000 7f38bb89a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b409a130 con 0x7f38b415ba00 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.134+0000 7f38bb89a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b415bde0 con 0x7f38b415f5f0 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38b9e37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38b41602e0 0x7f38b415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38b9e37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38b41602e0 0x7f38b415ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49482/0 (socket says 172.21.15.19:49482) 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38b9e37640 1 -- 172.21.15.19:0/3320334944 learned_addr learned my addr 172.21.15.19:0/3320334944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38b9636640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b415ba00 0x7f38b4157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38b8e35640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38b415f5f0 0x7f38b40c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38b9e37640 1 -- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38b415f5f0 msgr2=0x7f38b40c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38b9e37640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38b415f5f0 0x7f38b40c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38b9e37640 1 -- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b415ba00 msgr2=0x7f38b4157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38b9e37640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b415ba00 0x7f38b4157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38b9e37640 1 -- 172.21.15.19:0/3320334944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b40c8850 con 0x7f38b41602e0 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38b9e37640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38b41602e0 0x7f38b415ca50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00cf20 tx=0x7f38ac00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38a9ffb640 1 -- 172.21.15.19:0/3320334944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac01a070 con 0x7f38b41602e0 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38a9ffb640 1 -- 172.21.15.19:0/3320334944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac00d840 con 0x7f38b41602e0 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38bb89a640 1 -- 172.21.15.19:0/3320334944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b40c8b30 con 0x7f38b41602e0 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38a9ffb640 1 -- 172.21.15.19:0/3320334944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00dc30 con 0x7f38b41602e0 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.135+0000 7f38bb89a640 1 -- 172.21.15.19:0/3320334944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b415c250 con 0x7f38b41602e0 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.137+0000 7f38bb89a640 1 -- 172.21.15.19:0/3320334944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b4073990 con 0x7f38b41602e0 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.137+0000 7f38a9ffb640 1 -- 172.21.15.19:0/3320334944 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38ac01d070 con 0x7f38b41602e0 2024-04-25T09:56:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.137+0000 7f38a9ffb640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f389806b4d0 0x7f389806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.137+0000 7f38a9ffb640 1 -- 172.21.15.19:0/3320334944 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f38ac08a8a0 con 0x7f38b41602e0 2024-04-25T09:56:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.138+0000 7f38b9636640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f389806b4d0 0x7f389806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.138+0000 7f38b9636640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f389806b4d0 0x7f389806d990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f38b0034000 tx=0x7f38b002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.142+0000 7f38a9ffb640 1 -- 172.21.15.19:0/3320334944 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38ac059240 con 0x7f38b41602e0 2024-04-25T09:56:33.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.388+0000 7f38bb89a640 1 -- 172.21.15.19:0/3320334944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f38b40b40a0 con 0x7f38b41602e0 2024-04-25T09:56:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.403+0000 7f38a9ffb640 1 -- 172.21.15.19:0/3320334944 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v185) v1 ==== 146+0+0 (secure 0 0 0) 0x7f38ac05d2e0 con 0x7f38b41602e0 2024-04-25T09:56:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.405+0000 7f38bb89a640 1 -- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f389806b4d0 msgr2=0x7f389806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.405+0000 7f38bb89a640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f389806b4d0 0x7f389806d990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f38b0034000 tx=0x7f38b002e000 comp rx=0 tx=0).stop 2024-04-25T09:56:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.405+0000 7f38bb89a640 1 -- 172.21.15.19:0/3320334944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38b41602e0 msgr2=0x7f38b415ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.405+0000 7f38bb89a640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38b41602e0 0x7f38b415ca50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00cf20 tx=0x7f38ac00a7d0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.406+0000 7f38bb89a640 1 -- 172.21.15.19:0/3320334944 shutdown_connections 2024-04-25T09:56:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.406+0000 7f38bb89a640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f389806b4d0 0x7f389806d990 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.406+0000 7f38bb89a640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38b41602e0 0x7f38b415ca50 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.406+0000 7f38bb89a640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38b415f5f0 0x7f38b40c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.406+0000 7f38bb89a640 1 --2- 172.21.15.19:0/3320334944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b415ba00 0x7f38b4157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.406+0000 7f38bb89a640 1 -- 172.21.15.19:0/3320334944 >> 172.21.15.19:0/3320334944 conn(0x7f38b40b3270 msgr2=0x7f38b40cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.406+0000 7f38bb89a640 1 -- 172.21.15.19:0/3320334944 shutdown_connections 2024-04-25T09:56:33.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.406+0000 7f38bb89a640 1 -- 172.21.15.19:0/3320334944 wait complete. 2024-04-25T09:56:33.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:56:33.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.520+0000 7f9728fc8640 1 Processor -- start 2024-04-25T09:56:33.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.521+0000 7f9728fc8640 1 -- start start 2024-04-25T09:56:33.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.521+0000 7f9728fc8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f972415b7d0 0x7f97240cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.521+0000 7f9728fc8640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f97240b5620 0x7f97240b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.521+0000 7f9728fc8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f972406c2d0 0x7f972406c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9728fc8640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97240b5fc0 con 0x7f97240b5620 2024-04-25T09:56:33.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9728fc8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f972406cca0 con 0x7f972406c2d0 2024-04-25T09:56:33.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9728fc8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972406cf50 con 0x7f972415b7d0 2024-04-25T09:56:33.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9721d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f97240b5620 0x7f97240b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9721d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f97240b5620 0x7f97240b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49494/0 (socket says 172.21.15.19:49494) 2024-04-25T09:56:33.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9721d9b640 1 -- 172.21.15.19:0/2018596730 learned_addr learned my addr 172.21.15.19:0/2018596730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9722d9d640 1 --2- 172.21.15.19:0/2018596730 >> v2:172.21.15.112:3300/0 conn(0x7f972406c2d0 0x7f972406c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f972259c640 1 --2- 172.21.15.19:0/2018596730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f972415b7d0 0x7f97240cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9721d9b640 1 -- 172.21.15.19:0/2018596730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f972415b7d0 msgr2=0x7f97240cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9721d9b640 1 --2- 172.21.15.19:0/2018596730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f972415b7d0 0x7f97240cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9721d9b640 1 -- 172.21.15.19:0/2018596730 >> v2:172.21.15.112:3300/0 conn(0x7f972406c2d0 msgr2=0x7f972406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9721d9b640 1 --2- 172.21.15.19:0/2018596730 >> v2:172.21.15.112:3300/0 conn(0x7f972406c2d0 0x7f972406c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9721d9b640 1 -- 172.21.15.19:0/2018596730 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240ce530 con 0x7f97240b5620 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.522+0000 7f9722d9d640 1 --2- 172.21.15.19:0/2018596730 >> v2:172.21.15.112:3300/0 conn(0x7f972406c2d0 0x7f972406c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.523+0000 7f9721d9b640 1 --2- 172.21.15.19:0/2018596730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97240b5620 0x7f97240b5a80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f972415e340 tx=0x7f971802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d6a58dd0c69fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.523+0000 7f972159a640 1 -- 172.21.15.19:0/2018596730 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718037070 con 0x7f97240b5620 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.523+0000 7f972159a640 1 -- 172.21.15.19:0/2018596730 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9718032440 con 0x7f97240b5620 2024-04-25T09:56:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.523+0000 7f972159a640 1 -- 172.21.15.19:0/2018596730 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718032810 con 0x7f97240b5620 2024-04-25T09:56:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.523+0000 7f9728fc8640 1 -- 172.21.15.19:0/2018596730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97240b5620 msgr2=0x7f97240b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.523+0000 7f9728fc8640 1 --2- 172.21.15.19:0/2018596730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97240b5620 0x7f97240b5a80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f972415e340 tx=0x7f971802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.524+0000 7f9728fc8640 1 -- 172.21.15.19:0/2018596730 shutdown_connections 2024-04-25T09:56:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.524+0000 7f9728fc8640 1 --2- 172.21.15.19:0/2018596730 >> v2:172.21.15.112:3300/0 conn(0x7f972406c2d0 0x7f972406c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.524+0000 7f9728fc8640 1 --2- 172.21.15.19:0/2018596730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97240b5620 0x7f97240b5a80 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.524+0000 7f9728fc8640 1 --2- 172.21.15.19:0/2018596730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f972415b7d0 0x7f97240cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.524+0000 7f9728fc8640 1 -- 172.21.15.19:0/2018596730 >> 172.21.15.19:0/2018596730 conn(0x7f97240d0800 msgr2=0x7f97240d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.524+0000 7f9728fc8640 1 -- 172.21.15.19:0/2018596730 shutdown_connections 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.524+0000 7f9728fc8640 1 -- 172.21.15.19:0/2018596730 wait complete. 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.525+0000 7f9728fc8640 1 Processor -- start 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.525+0000 7f9728fc8640 1 -- start start 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9728fc8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f972406c2d0 0x7f97240b88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9728fc8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97240b5620 0x7f97240cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9728fc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972415b7d0 0x7f9724147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9728fc8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972406d230 con 0x7f97240b5620 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9728fc8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97240ceaf0 con 0x7f972415b7d0 2024-04-25T09:56:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9728fc8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97240cf550 con 0x7f972406c2d0 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9721d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97240b5620 0x7f97240cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9721d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97240b5620 0x7f97240cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49508/0 (socket says 172.21.15.19:49508) 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9721d9b640 1 -- 172.21.15.19:0/2763112511 learned_addr learned my addr 172.21.15.19:0/2763112511 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9722d9d640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972415b7d0 0x7f9724147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f972259c640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f972406c2d0 0x7f97240b88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9721d9b640 1 -- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f972406c2d0 msgr2=0x7f97240b88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9721d9b640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f972406c2d0 0x7f97240b88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9721d9b640 1 -- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972415b7d0 msgr2=0x7f9724147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9721d9b640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972415b7d0 0x7f9724147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9721d9b640 1 -- 172.21.15.19:0/2763112511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97241550c0 con 0x7f97240b5620 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9722d9d640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972415b7d0 0x7f9724147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9721d9b640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97240b5620 0x7f97240cd890 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9718000c00 tx=0x7f97180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9712ffd640 1 -- 172.21.15.19:0/2763112511 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718037040 con 0x7f97240b5620 2024-04-25T09:56:33.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9712ffd640 1 -- 172.21.15.19:0/2763112511 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9718042070 con 0x7f97240b5620 2024-04-25T09:56:33.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9728fc8640 1 -- 172.21.15.19:0/2763112511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97240c7820 con 0x7f97240b5620 2024-04-25T09:56:33.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.526+0000 7f9712ffd640 1 -- 172.21.15.19:0/2763112511 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718030bb0 con 0x7f97240b5620 2024-04-25T09:56:33.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.527+0000 7f9728fc8640 1 -- 172.21.15.19:0/2763112511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97240b80c0 con 0x7f97240b5620 2024-04-25T09:56:33.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.527+0000 7f9728fc8640 1 -- 172.21.15.19:0/2763112511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9724073a00 con 0x7f97240b5620 2024-04-25T09:56:33.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.528+0000 7f9712ffd640 1 -- 172.21.15.19:0/2763112511 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9718047460 con 0x7f97240b5620 2024-04-25T09:56:33.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.528+0000 7f9712ffd640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f970406b4d0 0x7f970406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.528+0000 7f9712ffd640 1 -- 172.21.15.19:0/2763112511 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f97180aac70 con 0x7f97240b5620 2024-04-25T09:56:33.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.529+0000 7f972259c640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f970406b4d0 0x7f970406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.529+0000 7f972259c640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f970406b4d0 0x7f970406d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9714006fd0 tx=0x7f9714008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.532+0000 7f9712ffd640 1 -- 172.21.15.19:0/2763112511 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9718045360 con 0x7f97240b5620 2024-04-25T09:56:33.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.712+0000 7f9728fc8640 1 -- 172.21.15.19:0/2763112511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f97240b3810 con 0x7f97240b5620 2024-04-25T09:56:33.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.717+0000 7f9712ffd640 1 -- 172.21.15.19:0/2763112511 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9718079cf0 con 0x7f97240b5620 2024-04-25T09:56:33.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 -- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f970406b4d0 msgr2=0x7f970406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f970406b4d0 0x7f970406d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9714006fd0 tx=0x7f9714008040 comp rx=0 tx=0).stop 2024-04-25T09:56:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 -- 172.21.15.19:0/2763112511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97240b5620 msgr2=0x7f97240cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97240b5620 0x7f97240cd890 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9718000c00 tx=0x7f97180046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 -- 172.21.15.19:0/2763112511 shutdown_connections 2024-04-25T09:56:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f970406b4d0 0x7f970406d990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972415b7d0 0x7f9724147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97240b5620 0x7f97240cd890 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 --2- 172.21.15.19:0/2763112511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f972406c2d0 0x7f97240b88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 -- 172.21.15.19:0/2763112511 >> 172.21.15.19:0/2763112511 conn(0x7f97240d0800 msgr2=0x7f972415afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 -- 172.21.15.19:0/2763112511 shutdown_connections 2024-04-25T09:56:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:33.719+0000 7f9728fc8640 1 -- 172.21.15.19:0/2763112511 wait complete. 2024-04-25T09:56:33.733 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:56:33.733 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:56:33.767 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:56:33.815 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:56:33.815 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:56:33.815 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:33.832 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:56:33.832 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:56:33.833 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:33.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:56:33.850 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:56:33.850 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:33.888 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:56:33.888 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:56:33.888 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:33.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:56:33.909 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:56:33.909 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:34.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.026+0000 7f043fd53640 1 Processor -- start 2024-04-25T09:56:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.026+0000 7f043fd53640 1 -- start start 2024-04-25T09:56:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.027+0000 7f043fd53640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f043815ba00 0x7f043809a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.027+0000 7f043fd53640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380cd710 con 0x7f043815f5f0 2024-04-25T09:56:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.027+0000 7f043fd53640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0438099b70 con 0x7f04381602e0 2024-04-25T09:56:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.027+0000 7f043fd53640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04380cd4b0 con 0x7f043815ba00 2024-04-25T09:56:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.027+0000 7f043daef640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f043815ba00 0x7f043809a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.027+0000 7f043e2f0640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f043815f5f0 0x7f04380c52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51192/0 (socket says 172.21.15.19:51192) 2024-04-25T09:56:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.027+0000 7f043e2f0640 1 -- 172.21.15.19:0/166742505 learned_addr learned my addr 172.21.15.19:0/166742505 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.028+0000 7f043daef640 1 -- 172.21.15.19:0/166742505 >> v1:172.21.15.112:6789/0 conn(0x7f04381602e0 legacy=0x7f04380ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.028+0000 7f043daef640 1 -- 172.21.15.19:0/166742505 >> v1:172.21.15.19:6789/0 conn(0x7f043815f5f0 legacy=0x7f04380c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.028+0000 7f043daef640 1 -- 172.21.15.19:0/166742505 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0438099e50 con 0x7f043815ba00 2024-04-25T09:56:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.028+0000 7f043daef640 1 --2- 172.21.15.19:0/166742505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f043815ba00 0x7f043809a930 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f04300097a0 tx=0x7f043002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1b31f3006e965b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.029+0000 7f043caed640 1 -- 172.21.15.19:0/166742505 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430037070 con 0x7f043815ba00 2024-04-25T09:56:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.029+0000 7f043caed640 1 -- 172.21.15.19:0/166742505 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0430032440 con 0x7f043815ba00 2024-04-25T09:56:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.029+0000 7f043caed640 1 -- 172.21.15.19:0/166742505 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043002ae60 con 0x7f043815ba00 2024-04-25T09:56:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.029+0000 7f043fd53640 1 -- 172.21.15.19:0/166742505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f043815ba00 msgr2=0x7f043809a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.029+0000 7f043fd53640 1 --2- 172.21.15.19:0/166742505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f043815ba00 0x7f043809a930 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f04300097a0 tx=0x7f043002a110 comp rx=0 tx=0).stop 2024-04-25T09:56:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.029+0000 7f043fd53640 1 -- 172.21.15.19:0/166742505 shutdown_connections 2024-04-25T09:56:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.029+0000 7f043fd53640 1 --2- 172.21.15.19:0/166742505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f043815ba00 0x7f043809a930 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.029+0000 7f043fd53640 1 -- 172.21.15.19:0/166742505 >> 172.21.15.19:0/166742505 conn(0x7f04380b3270 msgr2=0x7f04380b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.029+0000 7f043fd53640 1 -- 172.21.15.19:0/166742505 shutdown_connections 2024-04-25T09:56:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.029+0000 7f043fd53640 1 -- 172.21.15.19:0/166742505 wait complete. 2024-04-25T09:56:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.031+0000 7f043fd53640 1 Processor -- start 2024-04-25T09:56:34.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.031+0000 7f043fd53640 1 -- start start 2024-04-25T09:56:34.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.031+0000 7f043fd53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f043815ba00 0x7f04381588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:34.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.031+0000 7f043fd53640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f043815f5f0 0x7f04380cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:34.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.031+0000 7f043fd53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04381602e0 0x7f043815e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:34.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.031+0000 7f043fd53640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043815bde0 con 0x7f04381602e0 2024-04-25T09:56:34.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.031+0000 7f043fd53640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043809ae70 con 0x7f043815ba00 2024-04-25T09:56:34.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.031+0000 7f043fd53640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04380b5fa0 con 0x7f043815f5f0 2024-04-25T09:56:34.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.031+0000 7f043e2f0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04381602e0 0x7f043815e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:34.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.031+0000 7f043e2f0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04381602e0 0x7f043815e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49526/0 (socket says 172.21.15.19:49526) 2024-04-25T09:56:34.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.031+0000 7f043e2f0640 1 -- 172.21.15.19:0/1365063939 learned_addr learned my addr 172.21.15.19:0/1365063939 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:34.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f043e2f0640 1 -- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f043815f5f0 msgr2=0x7f04380cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f043e2f0640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f043815f5f0 0x7f04380cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f043e2f0640 1 -- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f043815ba00 msgr2=0x7f04381588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f043daef640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f043815ba00 0x7f04381588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f043e2f0640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f043815ba00 0x7f04381588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f043e2f0640 1 -- 172.21.15.19:0/1365063939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04380b6240 con 0x7f04381602e0 2024-04-25T09:56:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f043e2f0640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04381602e0 0x7f043815e100 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f042800cbc0 tx=0x7f042800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f04367fc640 1 -- 172.21.15.19:0/1365063939 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428017070 con 0x7f04381602e0 2024-04-25T09:56:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f04367fc640 1 -- 172.21.15.19:0/1365063939 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042800ad40 con 0x7f04381602e0 2024-04-25T09:56:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f043fd53640 1 -- 172.21.15.19:0/1365063939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04380b6520 con 0x7f04381602e0 2024-04-25T09:56:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f04367fc640 1 -- 172.21.15.19:0/1365063939 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428012600 con 0x7f04381602e0 2024-04-25T09:56:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.032+0000 7f043fd53640 1 -- 172.21.15.19:0/1365063939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0438158e10 con 0x7f04381602e0 2024-04-25T09:56:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.033+0000 7f043fd53640 1 -- 172.21.15.19:0/1365063939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0438073990 con 0x7f04381602e0 2024-04-25T09:56:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.034+0000 7f04367fc640 1 -- 172.21.15.19:0/1365063939 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f042801a070 con 0x7f04381602e0 2024-04-25T09:56:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.034+0000 7f04367fc640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f042406b4d0 0x7f042406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.034+0000 7f04367fc640 1 -- 172.21.15.19:0/1365063939 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0428089cb0 con 0x7f04381602e0 2024-04-25T09:56:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.034+0000 7f043daef640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f042406b4d0 0x7f042406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:34.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.035+0000 7f043daef640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f042406b4d0 0x7f042406d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0430034000 tx=0x7f043002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:34.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.038+0000 7f04367fc640 1 -- 172.21.15.19:0/1365063939 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04380cd710 con 0x7f04381602e0 2024-04-25T09:56:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.258+0000 7f043fd53640 1 -- 172.21.15.19:0/1365063939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04380b40a0 con 0x7f04381602e0 2024-04-25T09:56:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.258+0000 7f04367fc640 1 -- 172.21.15.19:0/1365063939 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f04280586e0 con 0x7f04381602e0 2024-04-25T09:56:34.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:34.262 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/231287293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":231287293},{"type":"v1","addr":"172.21.15.112:6836","nonce":231287293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2175655653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2175655653},{"type":"v1","addr":"172.21.15.112:6837","nonce":2175655653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3745496684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3745496684},{"type":"v1","addr":"172.21.15.19:6833","nonce":3745496684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6498,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/281749585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":281749585},{"type":"v1","addr":"172.21.15.19:6835","nonce":281749585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[]} 2024-04-25T09:56:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-04-25T09:56:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.260+0000 7f043fd53640 1 -- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f042406b4d0 msgr2=0x7f042406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.260+0000 7f043fd53640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f042406b4d0 0x7f042406d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0430034000 tx=0x7f043002e000 comp rx=0 tx=0).stop 2024-04-25T09:56:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.260+0000 7f043fd53640 1 -- 172.21.15.19:0/1365063939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04381602e0 msgr2=0x7f043815e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.260+0000 7f043fd53640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04381602e0 0x7f043815e100 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f042800cbc0 tx=0x7f042800a430 comp rx=0 tx=0).stop 2024-04-25T09:56:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.260+0000 7f043fd53640 1 -- 172.21.15.19:0/1365063939 shutdown_connections 2024-04-25T09:56:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.260+0000 7f043fd53640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f042406b4d0 0x7f042406d990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.260+0000 7f043fd53640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04381602e0 0x7f043815e100 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.260+0000 7f043fd53640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f043815f5f0 0x7f04380cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.260+0000 7f043fd53640 1 --2- 172.21.15.19:0/1365063939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f043815ba00 0x7f04381588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.260+0000 7f043fd53640 1 -- 172.21.15.19:0/1365063939 >> 172.21.15.19:0/1365063939 conn(0x7f04380b3270 msgr2=0x7f04380c7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:34.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.260+0000 7f043fd53640 1 -- 172.21.15.19:0/1365063939 shutdown_connections 2024-04-25T09:56:34.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.261+0000 7f043fd53640 1 -- 172.21.15.19:0/1365063939 wait complete. 2024-04-25T09:56:34.275 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:56:34.275 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:56:34.276 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:56:34.276 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:56:34.276 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:56:34.276 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:56:34.276 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:56:34.276 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:56:34.277 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:56:34.277 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:56:34.277 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:56:34.278 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:56:34.278 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:56:34.278 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:56:34.278 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:56:34.279 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:56:34.286 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:56:34.286 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:56:34.287 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:56:34.287 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:56:34.289 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:56:34.289 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:56:34.290 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:56:34.290 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:56:34.329 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:56:34.330 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:34.331 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:56:34.331 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:34.332 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:56:34.332 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:34.332 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:56:34.332 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:34.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:56:34.407 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T09:56:34.407 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T09:56:34.407 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T09:56:34.405+0000 7f49f2cdf640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T09:56:34.412 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T09:56:34.413 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T09:56:34.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.436+0000 7f2ed2a2a640 1 Processor -- start 2024-04-25T09:56:34.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.437+0000 7f2ed2a2a640 1 -- start start 2024-04-25T09:56:34.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.437+0000 7f2ed2a2a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ecc1616f0 0x7f2ecc161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:34.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.437+0000 7f2ed2a2a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ecc0ce0a0 0x7f2ecc0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:34.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.437+0000 7f2ed2a2a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ecc06c8d0 con 0x7f2ecc1610f0 2024-04-25T09:56:34.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.437+0000 7f2ed0fc7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ecc0ce0a0 0x7f2ecc0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:34.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.438+0000 7f2ed0fc7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ecc0ce0a0 0x7f2ecc0ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49578/0 (socket says 172.21.15.19:49578) 2024-04-25T09:56:34.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.437+0000 7f2ed2a2a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc1614d0 con 0x7f2ecc0ce0a0 2024-04-25T09:56:34.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.438+0000 7f2ed0fc7640 1 -- 172.21.15.19:0/896897724 learned_addr learned my addr 172.21.15.19:0/896897724 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:34.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.438+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/896897724 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0c6960 con 0x7f2ecc1616f0 2024-04-25T09:56:34.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.438+0000 7f2ecbfff640 1 --1- 172.21.15.19:0/896897724 >> v1:172.21.15.19:6789/0 conn(0x7f2ecc1610f0 0x7f2ecc06c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51200/0 (socket says 172.21.15.19:51200) 2024-04-25T09:56:34.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.438+0000 7f2ecb7fe640 1 --2- 172.21.15.19:0/896897724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ecc1616f0 0x7f2ecc161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:34.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.438+0000 7f2ed0fc7640 1 -- 172.21.15.19:0/896897724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ecc1616f0 msgr2=0x7f2ecc161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:34.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.438+0000 7f2ed0fc7640 1 --2- 172.21.15.19:0/896897724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ecc1616f0 0x7f2ecc161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.438+0000 7f2ed0fc7640 1 -- 172.21.15.19:0/896897724 >> v1:172.21.15.19:6789/0 conn(0x7f2ecc1610f0 legacy=0x7f2ecc06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:34.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.438+0000 7f2ed0fc7640 1 -- 172.21.15.19:0/896897724 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc162260 con 0x7f2ecc0ce0a0 2024-04-25T09:56:34.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ed0fc7640 1 --2- 172.21.15.19:0/896897724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ecc0ce0a0 0x7f2ecc0ce530 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009e50 tx=0x7f2ec402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432569700dc615d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:34.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ecaffd640 1 -- 172.21.15.19:0/896897724 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4036070 con 0x7f2ecc0ce0a0 2024-04-25T09:56:34.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ecaffd640 1 -- 172.21.15.19:0/896897724 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec4031600 con 0x7f2ecc0ce0a0 2024-04-25T09:56:34.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ecaffd640 1 -- 172.21.15.19:0/896897724 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40319d0 con 0x7f2ecc0ce0a0 2024-04-25T09:56:34.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/896897724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ecc0ce0a0 msgr2=0x7f2ecc0ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:34.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ed2a2a640 1 --2- 172.21.15.19:0/896897724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ecc0ce0a0 0x7f2ecc0ce530 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009e50 tx=0x7f2ec402a710 comp rx=0 tx=0).stop 2024-04-25T09:56:34.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/896897724 shutdown_connections 2024-04-25T09:56:34.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ed2a2a640 1 --2- 172.21.15.19:0/896897724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ecc0ce0a0 0x7f2ecc0ce530 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ed2a2a640 1 --2- 172.21.15.19:0/896897724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ecc1616f0 0x7f2ecc161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/896897724 >> 172.21.15.19:0/896897724 conn(0x7f2ecc06f500 msgr2=0x7f2ecc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:34.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/896897724 shutdown_connections 2024-04-25T09:56:34.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.439+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/896897724 wait complete. 2024-04-25T09:56:34.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.441+0000 7f2ed2a2a640 1 Processor -- start 2024-04-25T09:56:34.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.441+0000 7f2ed2a2a640 1 -- start start 2024-04-25T09:56:34.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ed2a2a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ecc0ce0a0 0x7f2ecc147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:34.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ed2a2a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ecc1610f0 0x7f2ecc1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ed2a2a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ecc1616f0 0x7f2ecc1f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ed2a2a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0cfce0 con 0x7f2ecc1610f0 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ed2a2a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0cfa00 con 0x7f2ecc0ce0a0 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ecc1610f0 0x7f2ecc1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ed2a2a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0cf690 con 0x7f2ecc1616f0 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ecc1610f0 0x7f2ecc1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49588/0 (socket says 172.21.15.19:49588) 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ecb7fe640 1 -- 172.21.15.19:0/2394738202 learned_addr learned my addr 172.21.15.19:0/2394738202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ecb7fe640 1 -- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ecc1616f0 msgr2=0x7f2ecc1f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ed0fc7640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ecc1616f0 0x7f2ecc1f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ecbfff640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ecc0ce0a0 0x7f2ecc147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ecb7fe640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ecc1616f0 0x7f2ecc1f7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ecb7fe640 1 -- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ecc0ce0a0 msgr2=0x7f2ecc147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:34.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ecb7fe640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ecc0ce0a0 0x7f2ecc147980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:34.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ecb7fe640 1 -- 172.21.15.19:0/2394738202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc159d50 con 0x7f2ecc1610f0 2024-04-25T09:56:34.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ecb7fe640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ecc1610f0 0x7f2ecc1595b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000e870 tx=0x7f2ec000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:34.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ec8ff9640 1 -- 172.21.15.19:0/2394738202 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0017070 con 0x7f2ecc1610f0 2024-04-25T09:56:34.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ec8ff9640 1 -- 172.21.15.19:0/2394738202 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec000c940 con 0x7f2ecc1610f0 2024-04-25T09:56:34.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/2394738202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ecc15a030 con 0x7f2ecc1610f0 2024-04-25T09:56:34.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.442+0000 7f2ec8ff9640 1 -- 172.21.15.19:0/2394738202 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec000cd30 con 0x7f2ecc1610f0 2024-04-25T09:56:34.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.443+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/2394738202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc15a5f0 con 0x7f2ecc1610f0 2024-04-25T09:56:34.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.444+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/2394738202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ecc073a00 con 0x7f2ecc1610f0 2024-04-25T09:56:34.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.444+0000 7f2ec8ff9640 1 -- 172.21.15.19:0/2394738202 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ec001a070 con 0x7f2ecc1610f0 2024-04-25T09:56:34.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.444+0000 7f2ec8ff9640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eb806b4d0 0x7f2eb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:34.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.445+0000 7f2ec8ff9640 1 -- 172.21.15.19:0/2394738202 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2ec008a320 con 0x7f2ecc1610f0 2024-04-25T09:56:34.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.445+0000 7f2ecbfff640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eb806b4d0 0x7f2eb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:34.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.445+0000 7f2ecbfff640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eb806b4d0 0x7f2eb806d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc007fd0 tx=0x7f2ebc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:34.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.449+0000 7f2ec8ff9640 1 -- 172.21.15.19:0/2394738202 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ec0059300 con 0x7f2ecc1610f0 2024-04-25T09:56:34.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:34.642+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/2394738202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2ecc06ff00 con 0x7f2ecc1610f0 2024-04-25T09:56:35.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.420+0000 7f2ec8ff9640 1 -- 172.21.15.19:0/2394738202 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2ec0058c00 con 0x7f2ecc1610f0 2024-04-25T09:56:35.423 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T09:56:35.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eb806b4d0 msgr2=0x7f2eb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eb806b4d0 0x7f2eb806d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc007fd0 tx=0x7f2ebc009040 comp rx=0 tx=0).stop 2024-04-25T09:56:35.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/2394738202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ecc1610f0 msgr2=0x7f2ecc1595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ecc1610f0 0x7f2ecc1595b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000e870 tx=0x7f2ec000ed30 comp rx=0 tx=0).stop 2024-04-25T09:56:35.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/2394738202 shutdown_connections 2024-04-25T09:56:35.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2eb806b4d0 0x7f2eb806d990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ecc1616f0 0x7f2ecc1f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ecc1610f0 0x7f2ecc1595b0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 --2- 172.21.15.19:0/2394738202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ecc0ce0a0 0x7f2ecc147980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/2394738202 >> 172.21.15.19:0/2394738202 conn(0x7f2ecc06f500 msgr2=0x7f2ecc06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/2394738202 shutdown_connections 2024-04-25T09:56:35.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.424+0000 7f2ed2a2a640 1 -- 172.21.15.19:0/2394738202 wait complete. 2024-04-25T09:56:35.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:56:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.547+0000 7f33ad570640 1 Processor -- start 2024-04-25T09:56:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.547+0000 7f33ad570640 1 -- start start 2024-04-25T09:56:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.548+0000 7f33ad570640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f33a815db70 0x7f33a8162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.548+0000 7f33ad570640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a8162690 con 0x7f33a80b6cb0 2024-04-25T09:56:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.548+0000 7f33ad570640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a80ceee0 con 0x7f33a80b5c30 2024-04-25T09:56:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.548+0000 7f33ad570640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a80d1d10 con 0x7f33a815db70 2024-04-25T09:56:35.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.548+0000 7f33a6ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f33a815db70 0x7f33a8162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:35.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.548+0000 7f33a67fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f33a80b6cb0 0x7f33a80b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51208/0 (socket says 172.21.15.19:51208) 2024-04-25T09:56:35.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.548+0000 7f33a67fc640 1 -- 172.21.15.19:0/2714591555 learned_addr learned my addr 172.21.15.19:0/2714591555 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:35.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.548+0000 7f33a6ffd640 1 -- 172.21.15.19:0/2714591555 >> v1:172.21.15.112:6789/0 conn(0x7f33a80b5c30 legacy=0x7f33a80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:35.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.548+0000 7f33a6ffd640 1 -- 172.21.15.19:0/2714591555 >> v1:172.21.15.19:6789/0 conn(0x7f33a80b6cb0 legacy=0x7f33a80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.548+0000 7f33a6ffd640 1 -- 172.21.15.19:0/2714591555 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a80d1ff0 con 0x7f33a815db70 2024-04-25T09:56:35.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.548+0000 7f33a5ffb640 1 -- 172.21.15.19:0/2714591555 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333695950 0 0) 0x7f33a8162690 con 0x7f33a80b6cb0 2024-04-25T09:56:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.549+0000 7f33a6ffd640 1 --2- 172.21.15.19:0/2714591555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33a815db70 0x7f33a8162150 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f33980097a0 tx=0x7f339802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cc2d862052153c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.549+0000 7f33a5ffb640 1 -- 172.21.15.19:0/2714591555 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398037070 con 0x7f33a815db70 2024-04-25T09:56:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.549+0000 7f33a5ffb640 1 -- 172.21.15.19:0/2714591555 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3398032440 con 0x7f33a815db70 2024-04-25T09:56:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.549+0000 7f33a5ffb640 1 -- 172.21.15.19:0/2714591555 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398032810 con 0x7f33a815db70 2024-04-25T09:56:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.550+0000 7f33ad570640 1 -- 172.21.15.19:0/2714591555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33a815db70 msgr2=0x7f33a8162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.550+0000 7f33ad570640 1 --2- 172.21.15.19:0/2714591555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33a815db70 0x7f33a8162150 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f33980097a0 tx=0x7f339802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.550+0000 7f33ad570640 1 -- 172.21.15.19:0/2714591555 shutdown_connections 2024-04-25T09:56:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.550+0000 7f33ad570640 1 --2- 172.21.15.19:0/2714591555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33a815db70 0x7f33a8162150 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.550+0000 7f33ad570640 1 -- 172.21.15.19:0/2714591555 >> 172.21.15.19:0/2714591555 conn(0x7f33a80d0520 msgr2=0x7f33a80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.550+0000 7f33ad570640 1 -- 172.21.15.19:0/2714591555 shutdown_connections 2024-04-25T09:56:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.550+0000 7f33ad570640 1 -- 172.21.15.19:0/2714591555 wait complete. 2024-04-25T09:56:35.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.551+0000 7f33ad570640 1 Processor -- start 2024-04-25T09:56:35.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.551+0000 7f33ad570640 1 -- start start 2024-04-25T09:56:35.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33ad570640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33a80b5c30 0x7f33a81547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:35.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33ad570640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33a80b6cb0 0x7f33a8151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:35.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33ad570640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33a815db70 0x7f33a8152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:35.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33ad570640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a80ce530 con 0x7f33a80b6cb0 2024-04-25T09:56:35.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33ad570640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a815df50 con 0x7f33a815db70 2024-04-25T09:56:35.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33ad570640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a80cf1c0 con 0x7f33a80b5c30 2024-04-25T09:56:35.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33a80b6cb0 0x7f33a8151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:35.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33a80b6cb0 0x7f33a8151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49598/0 (socket says 172.21.15.19:49598) 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a67fc640 1 -- 172.21.15.19:0/2597620772 learned_addr learned my addr 172.21.15.19:0/2597620772 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a77fe640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33a815db70 0x7f33a8152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a6ffd640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33a80b5c30 0x7f33a81547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a67fc640 1 -- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33a80b5c30 msgr2=0x7f33a81547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a67fc640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33a80b5c30 0x7f33a81547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a67fc640 1 -- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33a815db70 msgr2=0x7f33a8152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a67fc640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33a815db70 0x7f33a8152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a67fc640 1 -- 172.21.15.19:0/2597620772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a81524d0 con 0x7f33a80b6cb0 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a77fe640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33a815db70 0x7f33a8152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.552+0000 7f33a67fc640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33a80b6cb0 0x7f33a8151f90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f339c00c4b0 tx=0x7f339c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.553+0000 7f338f7fe640 1 -- 172.21.15.19:0/2597620772 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c018070 con 0x7f33a80b6cb0 2024-04-25T09:56:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.553+0000 7f338f7fe640 1 -- 172.21.15.19:0/2597620772 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c007500 con 0x7f33a80b6cb0 2024-04-25T09:56:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.553+0000 7f33ad570640 1 -- 172.21.15.19:0/2597620772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a8153750 con 0x7f33a80b6cb0 2024-04-25T09:56:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.553+0000 7f338f7fe640 1 -- 172.21.15.19:0/2597620772 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c004a20 con 0x7f33a80b6cb0 2024-04-25T09:56:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.553+0000 7f33ad570640 1 -- 172.21.15.19:0/2597620772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a8153d10 con 0x7f33a80b6cb0 2024-04-25T09:56:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.554+0000 7f33ad570640 1 -- 172.21.15.19:0/2597620772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a8073990 con 0x7f33a80b6cb0 2024-04-25T09:56:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.554+0000 7f338f7fe640 1 -- 172.21.15.19:0/2597620772 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f339c004bc0 con 0x7f33a80b6cb0 2024-04-25T09:56:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.555+0000 7f338f7fe640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f339406b4d0 0x7f339406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.555+0000 7f338f7fe640 1 -- 172.21.15.19:0/2597620772 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f339c089380 con 0x7f33a80b6cb0 2024-04-25T09:56:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.555+0000 7f33a6ffd640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f339406b4d0 0x7f339406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.555+0000 7f33a6ffd640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f339406b4d0 0x7f339406d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3398034000 tx=0x7f339802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.559+0000 7f338f7fe640 1 -- 172.21.15.19:0/2597620772 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33a81524d0 con 0x7f33a80b6cb0 2024-04-25T09:56:35.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.805+0000 7f33ad570640 1 -- 172.21.15.19:0/2597620772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f33a80ce310 con 0x7f33a80b6cb0 2024-04-25T09:56:35.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.805+0000 7f338f7fe640 1 -- 172.21.15.19:0/2597620772 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f339c058400 con 0x7f33a80b6cb0 2024-04-25T09:56:35.808 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.809 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T09:56:35.810 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:56:35.811 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:56:35.813 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:56:35.815 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T09:56:35.816 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T09:56:35.817 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T09:56:35.819 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:56:35.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:56:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T09:56:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T09:56:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:56:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:56:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:56:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:56:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:35.822 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T09:56:35.822 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 -- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f339406b4d0 msgr2=0x7f339406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f339406b4d0 0x7f339406d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3398034000 tx=0x7f339802e000 comp rx=0 tx=0).stop 2024-04-25T09:56:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 -- 172.21.15.19:0/2597620772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33a80b6cb0 msgr2=0x7f33a8151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33a80b6cb0 0x7f33a8151f90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f339c00c4b0 tx=0x7f339c00c970 comp rx=0 tx=0).stop 2024-04-25T09:56:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 -- 172.21.15.19:0/2597620772 shutdown_connections 2024-04-25T09:56:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f339406b4d0 0x7f339406d990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33a815db70 0x7f33a8152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33a80b6cb0 0x7f33a8151f90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 --2- 172.21.15.19:0/2597620772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33a80b5c30 0x7f33a81547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 -- 172.21.15.19:0/2597620772 >> 172.21.15.19:0/2597620772 conn(0x7f33a80d0520 msgr2=0x7f33a815a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 -- 172.21.15.19:0/2597620772 shutdown_connections 2024-04-25T09:56:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.807+0000 7f33ad570640 1 -- 172.21.15.19:0/2597620772 wait complete. 2024-04-25T09:56:35.825 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.923+0000 7f3aae45d640 1 Processor -- start 2024-04-25T09:56:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.923+0000 7f3aae45d640 1 -- start start 2024-04-25T09:56:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.924+0000 7f3aae45d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3aa81597d0 0x7f3aa806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.924+0000 7f3aae45d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa80d0480 con 0x7f3aa80cc320 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.924+0000 7f3aae45d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa80cf0e0 con 0x7f3aa80b2d40 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.924+0000 7f3aae45d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa815c1b0 con 0x7f3aa81597d0 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.924+0000 7f3aac9fa640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3aa80cc320 0x7f3aa80ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51222/0 (socket says 172.21.15.19:51222) 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.924+0000 7f3aac9fa640 1 -- 172.21.15.19:0/4002775730 learned_addr learned my addr 172.21.15.19:0/4002775730 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.924+0000 7f3aa7fff640 1 --2- 172.21.15.19:0/4002775730 >> v2:172.21.15.112:3300/0 conn(0x7f3aa81597d0 0x7f3aa806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.924+0000 7f3aa6ffd640 1 -- 172.21.15.19:0/4002775730 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494559548 0 0) 0x7f3aa80d0480 con 0x7f3aa80cc320 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.924+0000 7f3aa6ffd640 1 -- 172.21.15.19:0/4002775730 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a90003520 con 0x7f3aa80cc320 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.925+0000 7f3aa6ffd640 1 -- 172.21.15.19:0/4002775730 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696452480 0 0) 0x7f3aa80cf0e0 con 0x7f3aa80b2d40 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.925+0000 7f3aa6ffd640 1 -- 172.21.15.19:0/4002775730 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aa80d0480 con 0x7f3aa80b2d40 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.925+0000 7f3aa7fff640 1 -- 172.21.15.19:0/4002775730 >> v1:172.21.15.112:6789/0 conn(0x7f3aa80b2d40 legacy=0x7f3aa80ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.925+0000 7f3aa7fff640 1 -- 172.21.15.19:0/4002775730 >> v1:172.21.15.19:6789/0 conn(0x7f3aa80cc320 legacy=0x7f3aa80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.925+0000 7f3aa7fff640 1 -- 172.21.15.19:0/4002775730 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa815c490 con 0x7f3aa81597d0 2024-04-25T09:56:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.925+0000 7f3aa7fff640 1 --2- 172.21.15.19:0/4002775730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa81597d0 0x7f3aa806abe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3a980097a0 tx=0x7f3a9802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45d2334a3f02ff9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.926+0000 7f3aa6ffd640 1 -- 172.21.15.19:0/4002775730 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98037070 con 0x7f3aa81597d0 2024-04-25T09:56:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.926+0000 7f3aa6ffd640 1 -- 172.21.15.19:0/4002775730 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a98032440 con 0x7f3aa81597d0 2024-04-25T09:56:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.926+0000 7f3aa6ffd640 1 -- 172.21.15.19:0/4002775730 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a980327f0 con 0x7f3aa81597d0 2024-04-25T09:56:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.926+0000 7f3aae45d640 1 -- 172.21.15.19:0/4002775730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa81597d0 msgr2=0x7f3aa806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.926+0000 7f3aae45d640 1 --2- 172.21.15.19:0/4002775730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa81597d0 0x7f3aa806abe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3a980097a0 tx=0x7f3a9802a230 comp rx=0 tx=0).stop 2024-04-25T09:56:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.926+0000 7f3aae45d640 1 -- 172.21.15.19:0/4002775730 shutdown_connections 2024-04-25T09:56:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.926+0000 7f3aae45d640 1 --2- 172.21.15.19:0/4002775730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa81597d0 0x7f3aa806abe0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.926+0000 7f3aae45d640 1 -- 172.21.15.19:0/4002775730 >> 172.21.15.19:0/4002775730 conn(0x7f3aa80b2220 msgr2=0x7f3aa80b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.926+0000 7f3aae45d640 1 -- 172.21.15.19:0/4002775730 shutdown_connections 2024-04-25T09:56:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.926+0000 7f3aae45d640 1 -- 172.21.15.19:0/4002775730 wait complete. 2024-04-25T09:56:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.927+0000 7f3aae45d640 1 Processor -- start 2024-04-25T09:56:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.928+0000 7f3aae45d640 1 -- start start 2024-04-25T09:56:35.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.928+0000 7f3aae45d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3aa80b2d40 0x7f3aa80c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:35.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.928+0000 7f3aae45d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aa80cc320 0x7f3aa80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:35.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.928+0000 7f3aae45d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa81597d0 0x7f3aa80b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:35.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.928+0000 7f3aae45d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa806b7a0 con 0x7f3aa80cc320 2024-04-25T09:56:35.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.928+0000 7f3aae45d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa815ca50 con 0x7f3aa81597d0 2024-04-25T09:56:35.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.928+0000 7f3aae45d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa806b920 con 0x7f3aa80b2d40 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.928+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aa80cc320 0x7f3aa80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.928+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aa80cc320 0x7f3aa80b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49606/0 (socket says 172.21.15.19:49606) 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.928+0000 7f3aa77fe640 1 -- 172.21.15.19:0/1733082658 learned_addr learned my addr 172.21.15.19:0/1733082658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aa7fff640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3aa80b2d40 0x7f3aa80c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aac9fa640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa81597d0 0x7f3aa80b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aa77fe640 1 -- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3aa80b2d40 msgr2=0x7f3aa80c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aa77fe640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3aa80b2d40 0x7f3aa80c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aa77fe640 1 -- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa81597d0 msgr2=0x7f3aa80b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aa77fe640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa81597d0 0x7f3aa80b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aa77fe640 1 -- 172.21.15.19:0/1733082658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80b4000 con 0x7f3aa80cc320 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aac9fa640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa81597d0 0x7f3aa80b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aa77fe640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aa80cc320 0x7f3aa80b4fd0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00c400 tx=0x7f3a9c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aa4ff9640 1 -- 172.21.15.19:0/1733082658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c017070 con 0x7f3aa80cc320 2024-04-25T09:56:35.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aa4ff9640 1 -- 172.21.15.19:0/1733082658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c0052d0 con 0x7f3aa80cc320 2024-04-25T09:56:35.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aae45d640 1 -- 172.21.15.19:0/1733082658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa80b42e0 con 0x7f3aa80cc320 2024-04-25T09:56:35.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aa4ff9640 1 -- 172.21.15.19:0/1733082658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c003810 con 0x7f3aa80cc320 2024-04-25T09:56:35.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.929+0000 7f3aae45d640 1 -- 172.21.15.19:0/1733082658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa80b48a0 con 0x7f3aa80cc320 2024-04-25T09:56:35.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.930+0000 7f3aae45d640 1 -- 172.21.15.19:0/1733082658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa8073990 con 0x7f3aa80cc320 2024-04-25T09:56:35.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.930+0000 7f3aa4ff9640 1 -- 172.21.15.19:0/1733082658 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a9c01a070 con 0x7f3aa80cc320 2024-04-25T09:56:35.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.931+0000 7f3aa4ff9640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a9406b4d0 0x7f3a9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:35.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.931+0000 7f3aa4ff9640 1 -- 172.21.15.19:0/1733082658 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3a9c0893f0 con 0x7f3aa80cc320 2024-04-25T09:56:35.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.931+0000 7f3aa7fff640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a9406b4d0 0x7f3a9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:35.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.931+0000 7f3aa7fff640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a9406b4d0 0x7f3a9406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3a98034000 tx=0x7f3a9802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:35.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:35.935+0000 7f3aa4ff9640 1 -- 172.21.15.19:0/1733082658 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a9c0584c0 con 0x7f3aa80cc320 2024-04-25T09:56:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.117+0000 7f3aae45d640 1 -- 172.21.15.19:0/1733082658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3aa80c6fd0 con 0x7f3aa80cc320 2024-04-25T09:56:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.118+0000 7f3aa4ff9640 1 -- 172.21.15.19:0/1733082658 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+13152 (secure 0 0 0) 0x7f3a9c057dc0 con 0x7f3aa80cc320 2024-04-25T09:56:36.121 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:36.121 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":60,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:56:35.428464+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:36.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.120+0000 7f3aae45d640 1 -- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a9406b4d0 msgr2=0x7f3a9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.120+0000 7f3aae45d640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a9406b4d0 0x7f3a9406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3a98034000 tx=0x7f3a9802e000 comp rx=0 tx=0).stop 2024-04-25T09:56:36.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.120+0000 7f3aae45d640 1 -- 172.21.15.19:0/1733082658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aa80cc320 msgr2=0x7f3aa80b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.120+0000 7f3aae45d640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aa80cc320 0x7f3aa80b4fd0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00c400 tx=0x7f3a9c00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.121+0000 7f3aae45d640 1 -- 172.21.15.19:0/1733082658 shutdown_connections 2024-04-25T09:56:36.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.121+0000 7f3aae45d640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a9406b4d0 0x7f3a9406d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.121+0000 7f3aae45d640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3aa81597d0 0x7f3aa80b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.121+0000 7f3aae45d640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aa80cc320 0x7f3aa80b4fd0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.121+0000 7f3aae45d640 1 --2- 172.21.15.19:0/1733082658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3aa80b2d40 0x7f3aa80c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.121+0000 7f3aae45d640 1 -- 172.21.15.19:0/1733082658 >> 172.21.15.19:0/1733082658 conn(0x7f3aa80b2220 msgr2=0x7f3aa815e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:36.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.121+0000 7f3aae45d640 1 -- 172.21.15.19:0/1733082658 shutdown_connections 2024-04-25T09:56:36.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.121+0000 7f3aae45d640 1 -- 172.21.15.19:0/1733082658 wait complete. 2024-04-25T09:56:36.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:56:36.135 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:56:36.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.236+0000 7fdb6469c640 1 Processor -- start 2024-04-25T09:56:36.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.236+0000 7fdb6469c640 1 -- start start 2024-04-25T09:56:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb6469c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdb5c15b7d0 0x7fdb5c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb6469c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c06c9e0 0x7fdb5c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb6469c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c06cf50 con 0x7fdb5c0b5620 2024-04-25T09:56:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb6469c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5c0ce530 con 0x7fdb5c15b7d0 2024-04-25T09:56:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb6469c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0ce810 con 0x7fdb5c06c9e0 2024-04-25T09:56:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61c37640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdb5c0b5620 0x7fdb5c06c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51226/0 (socket says 172.21.15.19:51226) 2024-04-25T09:56:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61c37640 1 -- 172.21.15.19:0/3337625416 learned_addr learned my addr 172.21.15.19:0/3337625416 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104962576 0 0) 0x7fdb5c06cf50 con 0x7fdb5c0b5620 2024-04-25T09:56:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb62438640 1 --2- 172.21.15.19:0/3337625416 >> v2:172.21.15.112:3300/0 conn(0x7fdb5c15b7d0 0x7fdb5c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb62c39640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c06c9e0 0x7fdb5c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb48003520 con 0x7fdb5c0b5620 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2804219888 0 0) 0x7fdb48003520 con 0x7fdb5c0b5620 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c06c9e0 msgr2=0x7fdb5c0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c06c9e0 0x7fdb5c0b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 >> v2:172.21.15.112:3300/0 conn(0x7fdb5c15b7d0 msgr2=0x7fdb5c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 --2- 172.21.15.19:0/3337625416 >> v2:172.21.15.112:3300/0 conn(0x7fdb5c15b7d0 0x7fdb5c0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c0ceaf0 con 0x7fdb5c0b5620 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fdb4c001690 con 0x7fdb5c0b5620 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 >> v1:172.21.15.19:6789/0 conn(0x7fdb5c0b5620 legacy=0x7fdb5c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c06c9e0 0x7fdb480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 0x7fdb48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb48004d40 0x7fdb480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0ce530 con 0x7fdb48003f30 2024-04-25T09:56:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.237+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0ce810 con 0x7fdb48004d40 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.238+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4800b9c0 con 0x7fdb5c06c9e0 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.238+0000 7fdb61c37640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 0x7fdb48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.238+0000 7fdb62438640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c06c9e0 0x7fdb480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.238+0000 7fdb62c39640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb48004d40 0x7fdb480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.238+0000 7fdb61c37640 1 -- 172.21.15.19:0/3337625416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c06c9e0 msgr2=0x7fdb480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.238+0000 7fdb61c37640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c06c9e0 0x7fdb480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.238+0000 7fdb61c37640 1 -- 172.21.15.19:0/3337625416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb48004d40 msgr2=0x7fdb480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.238+0000 7fdb61c37640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb48004d40 0x7fdb480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.238+0000 7fdb61c37640 1 -- 172.21.15.19:0/3337625416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb48003520 con 0x7fdb48003f30 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.239+0000 7fdb61c37640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 0x7fdb48004800 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c007fd0 tx=0x7fdb48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69d137aaa6361a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.239+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c007d50 con 0x7fdb48003f30 2024-04-25T09:56:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.239+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4c0058d0 con 0x7fdb48003f30 2024-04-25T09:56:36.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.239+0000 7fdb61436640 1 -- 172.21.15.19:0/3337625416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c005ca0 con 0x7fdb48003f30 2024-04-25T09:56:36.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.239+0000 7fdb6469c640 1 -- 172.21.15.19:0/3337625416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 msgr2=0x7fdb48004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.239+0000 7fdb6469c640 1 --2- 172.21.15.19:0/3337625416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 0x7fdb48004800 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c007fd0 tx=0x7fdb48005830 comp rx=0 tx=0).stop 2024-04-25T09:56:36.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.239+0000 7fdb62c39640 1 -- 172.21.15.19:0/3337625416 reap_dead start 2024-04-25T09:56:36.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.239+0000 7fdb6469c640 1 -- 172.21.15.19:0/3337625416 shutdown_connections 2024-04-25T09:56:36.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.239+0000 7fdb6469c640 1 -- 172.21.15.19:0/3337625416 >> 172.21.15.19:0/3337625416 conn(0x7fdb5c0d0800 msgr2=0x7fdb5c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:36.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.239+0000 7fdb6469c640 1 -- 172.21.15.19:0/3337625416 shutdown_connections 2024-04-25T09:56:36.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.239+0000 7fdb6469c640 1 -- 172.21.15.19:0/3337625416 wait complete. 2024-04-25T09:56:36.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.241+0000 7fdb6469c640 1 Processor -- start 2024-04-25T09:56:36.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.241+0000 7fdb6469c640 1 -- start start 2024-04-25T09:56:36.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.241+0000 7fdb6469c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 0x7fdb5c0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.241+0000 7fdb6469c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c0c9d80 0x7fdb5c0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.241+0000 7fdb6469c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c14cb30 0x7fdb5c14cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.241+0000 7fdb6469c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0cf6d0 con 0x7fdb48003f30 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.241+0000 7fdb6469c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0cf550 con 0x7fdb5c14cb30 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.241+0000 7fdb6469c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c15bbb0 con 0x7fdb5c0c9d80 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.241+0000 7fdb62438640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 0x7fdb5c0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.241+0000 7fdb62438640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 0x7fdb5c0c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49622/0 (socket says 172.21.15.19:49622) 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.241+0000 7fdb62438640 1 -- 172.21.15.19:0/3722290649 learned_addr learned my addr 172.21.15.19:0/3722290649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb61c37640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c0c9d80 0x7fdb5c0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb62c39640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c14cb30 0x7fdb5c14cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb62438640 1 -- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c0c9d80 msgr2=0x7fdb5c0c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb62438640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c0c9d80 0x7fdb5c0c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb62438640 1 -- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c14cb30 msgr2=0x7fdb5c14cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb62438640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c14cb30 0x7fdb5c14cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb62438640 1 -- 172.21.15.19:0/3722290649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c14d4f0 con 0x7fdb48003f30 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb61c37640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c0c9d80 0x7fdb5c0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb62438640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 0x7fdb5c0c7570 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdb58011960 tx=0x7fdb58011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb52ffd640 1 -- 172.21.15.19:0/3722290649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800faf0 con 0x7fdb48003f30 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb52ffd640 1 -- 172.21.15.19:0/3722290649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5800fc90 con 0x7fdb48003f30 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb6469c640 1 -- 172.21.15.19:0/3722290649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c0cbe40 con 0x7fdb48003f30 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb52ffd640 1 -- 172.21.15.19:0/3722290649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58014440 con 0x7fdb48003f30 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.242+0000 7fdb6469c640 1 -- 172.21.15.19:0/3722290649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c0cc400 con 0x7fdb48003f30 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.243+0000 7fdb6469c640 1 -- 172.21.15.19:0/3722290649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c073a00 con 0x7fdb48003f30 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.244+0000 7fdb52ffd640 1 -- 172.21.15.19:0/3722290649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb58012040 con 0x7fdb48003f30 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.244+0000 7fdb52ffd640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb4006b4d0 0x7fdb4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.244+0000 7fdb52ffd640 1 -- 172.21.15.19:0/3722290649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fdb5808a780 con 0x7fdb48003f30 2024-04-25T09:56:36.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.244+0000 7fdb61c37640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb4006b4d0 0x7fdb4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.245+0000 7fdb61c37640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb4006b4d0 0x7fdb4006d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c033000 tx=0x7fdb4c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:36.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.248+0000 7fdb52ffd640 1 -- 172.21.15.19:0/3722290649 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb58019030 con 0x7fdb48003f30 2024-04-25T09:56:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.429+0000 7fdb6469c640 1 -- 172.21.15.19:0/3722290649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb5c15b3f0 con 0x7fdb48003f30 2024-04-25T09:56:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.430+0000 7fdb52ffd640 1 -- 172.21.15.19:0/3722290649 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb580590e0 con 0x7fdb48003f30 2024-04-25T09:56:36.433 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:56:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 -- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb4006b4d0 msgr2=0x7fdb4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb4006b4d0 0x7fdb4006d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c033000 tx=0x7fdb4c009030 comp rx=0 tx=0).stop 2024-04-25T09:56:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 -- 172.21.15.19:0/3722290649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 msgr2=0x7fdb5c0c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 0x7fdb5c0c7570 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdb58011960 tx=0x7fdb58011e20 comp rx=0 tx=0).stop 2024-04-25T09:56:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 -- 172.21.15.19:0/3722290649 shutdown_connections 2024-04-25T09:56:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb4006b4d0 0x7fdb4006d990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb5c14cb30 0x7fdb5c14cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb5c0c9d80 0x7fdb5c0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 --2- 172.21.15.19:0/3722290649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb48003f30 0x7fdb5c0c7570 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 -- 172.21.15.19:0/3722290649 >> 172.21.15.19:0/3722290649 conn(0x7fdb5c0d0800 msgr2=0x7fdb5c15f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:36.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 -- 172.21.15.19:0/3722290649 shutdown_connections 2024-04-25T09:56:36.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.432+0000 7fdb6469c640 1 -- 172.21.15.19:0/3722290649 wait complete. 2024-04-25T09:56:36.446 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:56:36.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.543+0000 7f9fe6ca1640 1 Processor -- start 2024-04-25T09:56:36.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.543+0000 7f9fe6ca1640 1 -- start start 2024-04-25T09:56:36.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.544+0000 7f9fe6ca1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9fe015dc70 0x7f9fe00d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.544+0000 7f9fe6ca1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9fe006ce10 0x7f9fe00cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.544+0000 7f9fe6ca1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe006c220 con 0x7f9fe009a450 2024-04-25T09:56:36.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.544+0000 7f9fe6ca1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe00d0c90 con 0x7f9fe015dc70 2024-04-25T09:56:36.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.544+0000 7f9fe6ca1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe00d0080 con 0x7f9fe006ce10 2024-04-25T09:56:36.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.544+0000 7f9fe4a3d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9fe015dc70 0x7f9fe00d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.544+0000 7f9fdffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9fe009a450 0x7f9fe006c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51238/0 (socket says 172.21.15.19:51238) 2024-04-25T09:56:36.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.544+0000 7f9fe4a3d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9fe015dc70 0x7f9fe00d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49630/0 (socket says 172.21.15.19:49630) 2024-04-25T09:56:36.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.544+0000 7f9fdffff640 1 -- 172.21.15.19:0/126640637 learned_addr learned my addr 172.21.15.19:0/126640637 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:36.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.545+0000 7f9fe523e640 1 --2- 172.21.15.19:0/126640637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9fe006ce10 0x7f9fe00cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.545+0000 7f9fe4a3d640 1 -- 172.21.15.19:0/126640637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9fe006ce10 msgr2=0x7f9fe00cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:36.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.545+0000 7f9fe4a3d640 1 --2- 172.21.15.19:0/126640637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9fe006ce10 0x7f9fe00cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.545+0000 7f9fe4a3d640 1 -- 172.21.15.19:0/126640637 >> v1:172.21.15.19:6789/0 conn(0x7f9fe009a450 legacy=0x7f9fe006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.545+0000 7f9fe4a3d640 1 -- 172.21.15.19:0/126640637 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe00d0360 con 0x7f9fe015dc70 2024-04-25T09:56:36.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.545+0000 7f9fdf7fe640 1 -- 172.21.15.19:0/126640637 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804924843 0 0) 0x7f9fe006c220 con 0x7f9fe009a450 2024-04-25T09:56:36.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.545+0000 7f9fe4a3d640 1 --2- 172.21.15.19:0/126640637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe015dc70 0x7f9fe00d1120 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009d90 tx=0x7f9fd802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e5639b5e4acd72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:36.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.546+0000 7f9fdf7fe640 1 -- 172.21.15.19:0/126640637 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8007740 con 0x7f9fe015dc70 2024-04-25T09:56:36.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.546+0000 7f9fdf7fe640 1 -- 172.21.15.19:0/126640637 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd80078e0 con 0x7f9fe015dc70 2024-04-25T09:56:36.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.546+0000 7f9fdf7fe640 1 -- 172.21.15.19:0/126640637 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8007cb0 con 0x7f9fe015dc70 2024-04-25T09:56:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.546+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/126640637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe015dc70 msgr2=0x7f9fe00d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.546+0000 7f9fe6ca1640 1 --2- 172.21.15.19:0/126640637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe015dc70 0x7f9fe00d1120 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009d90 tx=0x7f9fd802a660 comp rx=0 tx=0).stop 2024-04-25T09:56:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.546+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/126640637 shutdown_connections 2024-04-25T09:56:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.546+0000 7f9fe6ca1640 1 --2- 172.21.15.19:0/126640637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9fe006ce10 0x7f9fe00cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.546+0000 7f9fe6ca1640 1 --2- 172.21.15.19:0/126640637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe015dc70 0x7f9fe00d1120 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.546+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/126640637 >> 172.21.15.19:0/126640637 conn(0x7f9fe00b3270 msgr2=0x7f9fe00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.546+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/126640637 shutdown_connections 2024-04-25T09:56:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.546+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/126640637 wait complete. 2024-04-25T09:56:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.547+0000 7f9fe6ca1640 1 Processor -- start 2024-04-25T09:56:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.547+0000 7f9fe6ca1640 1 -- start start 2024-04-25T09:56:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.548+0000 7f9fe6ca1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9fe006ce10 0x7f9fe0144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.548+0000 7f9fe6ca1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe009a450 0x7f9fe0157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.548+0000 7f9fe6ca1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe015dc70 0x7f9fe01464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.548+0000 7f9fe6ca1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe015e050 con 0x7f9fe009a450 2024-04-25T09:56:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.548+0000 7f9fe6ca1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe00d0920 con 0x7f9fe015dc70 2024-04-25T09:56:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.548+0000 7f9fe6ca1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe00cbdd0 con 0x7f9fe006ce10 2024-04-25T09:56:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.548+0000 7f9fdffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe009a450 0x7f9fe0157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.548+0000 7f9fdffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe009a450 0x7f9fe0157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49638/0 (socket says 172.21.15.19:49638) 2024-04-25T09:56:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.548+0000 7f9fdffff640 1 -- 172.21.15.19:0/2172632190 learned_addr learned my addr 172.21.15.19:0/2172632190 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fdffff640 1 -- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9fe006ce10 msgr2=0x7f9fe0144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fe4a3d640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9fe006ce10 0x7f9fe0144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fe523e640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe015dc70 0x7f9fe01464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fdffff640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9fe006ce10 0x7f9fe0144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fdffff640 1 -- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe015dc70 msgr2=0x7f9fe01464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fdffff640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe015dc70 0x7f9fe01464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fdffff640 1 -- 172.21.15.19:0/2172632190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe0145400 con 0x7f9fe009a450 2024-04-25T09:56:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fe523e640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe015dc70 0x7f9fe01464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fdffff640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe009a450 0x7f9fe0157e80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400c500 tx=0x7f9fd400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fdd7fa640 1 -- 172.21.15.19:0/2172632190 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4018070 con 0x7f9fe009a450 2024-04-25T09:56:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fdd7fa640 1 -- 172.21.15.19:0/2172632190 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd4007580 con 0x7f9fe009a450 2024-04-25T09:56:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/2172632190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe01456e0 con 0x7f9fe009a450 2024-04-25T09:56:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fdd7fa640 1 -- 172.21.15.19:0/2172632190 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4004a20 con 0x7f9fe009a450 2024-04-25T09:56:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.549+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/2172632190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe0145ca0 con 0x7f9fe009a450 2024-04-25T09:56:36.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.550+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/2172632190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe0073990 con 0x7f9fe009a450 2024-04-25T09:56:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.551+0000 7f9fdd7fa640 1 -- 172.21.15.19:0/2172632190 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fd401b070 con 0x7f9fe009a450 2024-04-25T09:56:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.551+0000 7f9fdd7fa640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9fcc06b4d0 0x7f9fcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.551+0000 7f9fdd7fa640 1 -- 172.21.15.19:0/2172632190 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9fd40892e0 con 0x7f9fe009a450 2024-04-25T09:56:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.551+0000 7f9fe4a3d640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9fcc06b4d0 0x7f9fcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.552+0000 7f9fe4a3d640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9fcc06b4d0 0x7f9fcc06d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8034000 tx=0x7f9fd8033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.555+0000 7f9fdd7fa640 1 -- 172.21.15.19:0/2172632190 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9fd40583b0 con 0x7f9fe009a450 2024-04-25T09:56:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.741+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/2172632190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9fe00b40a0 con 0x7f9fe009a450 2024-04-25T09:56:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.741+0000 7f9fdd7fa640 1 -- 172.21.15.19:0/2172632190 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9fd4057cb0 con 0x7f9fe009a450 2024-04-25T09:56:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:36.745 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T09:56:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:56:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T09:56:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T09:56:36.745 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:56:36.745 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T09:56:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9fcc06b4d0 msgr2=0x7f9fcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9fcc06b4d0 0x7f9fcc06d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8034000 tx=0x7f9fd8033040 comp rx=0 tx=0).stop 2024-04-25T09:56:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/2172632190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe009a450 msgr2=0x7f9fe0157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe009a450 0x7f9fe0157e80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400c500 tx=0x7f9fd400c9c0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/2172632190 shutdown_connections 2024-04-25T09:56:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9fcc06b4d0 0x7f9fcc06d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fe015dc70 0x7f9fe01464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fe009a450 0x7f9fe0157e80 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 --2- 172.21.15.19:0/2172632190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9fe006ce10 0x7f9fe0144e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/2172632190 >> 172.21.15.19:0/2172632190 conn(0x7f9fe00b3270 msgr2=0x7f9fe00b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/2172632190 shutdown_connections 2024-04-25T09:56:36.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.744+0000 7f9fe6ca1640 1 -- 172.21.15.19:0/2172632190 wait complete. 2024-04-25T09:56:36.757 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:56:36.758 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:56:36.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.859+0000 7f2dd3b6a640 1 Processor -- start 2024-04-25T09:56:36.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.859+0000 7f2dd3b6a640 1 -- start start 2024-04-25T09:56:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.860+0000 7f2dd3b6a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc15dc20 0x7f2dcc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.860+0000 7f2dd3b6a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2dcc161710 0x7f2dcc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.860+0000 7f2dd3b6a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2dcc0b9330 0x7f2dcc06bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.860+0000 7f2dd3b6a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc15e000 con 0x7f2dcc0b9330 2024-04-25T09:56:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.860+0000 7f2dd3b6a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc0d1de0 con 0x7f2dcc161710 2024-04-25T09:56:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.860+0000 7f2dd3b6a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc161c60 con 0x7f2dcc15dc20 2024-04-25T09:56:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.860+0000 7f2dd2107640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2dcc0b9330 0x7f2dcc06bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.860+0000 7f2dd2107640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2dcc0b9330 0x7f2dcc06bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49640/0 (socket says 172.21.15.19:49640) 2024-04-25T09:56:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.860+0000 7f2dd2107640 1 -- 172.21.15.19:0/1615181042 learned_addr learned my addr 172.21.15.19:0/1615181042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.861+0000 7f2dd1105640 1 --2- 172.21.15.19:0/1615181042 >> v2:172.21.15.112:3300/0 conn(0x7f2dcc161710 0x7f2dcc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.861+0000 7f2dd1906640 1 --2- 172.21.15.19:0/1615181042 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc15dc20 0x7f2dcc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.861+0000 7f2dd2107640 1 -- 172.21.15.19:0/1615181042 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc15dc20 msgr2=0x7f2dcc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.861+0000 7f2dd2107640 1 --2- 172.21.15.19:0/1615181042 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc15dc20 0x7f2dcc0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.861+0000 7f2dd2107640 1 -- 172.21.15.19:0/1615181042 >> v2:172.21.15.112:3300/0 conn(0x7f2dcc161710 msgr2=0x7f2dcc0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.861+0000 7f2dd2107640 1 --2- 172.21.15.19:0/1615181042 >> v2:172.21.15.112:3300/0 conn(0x7f2dcc161710 0x7f2dcc0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.861+0000 7f2dd2107640 1 -- 172.21.15.19:0/1615181042 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc161f40 con 0x7f2dcc0b9330 2024-04-25T09:56:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.861+0000 7f2dd2107640 1 --2- 172.21.15.19:0/1615181042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dcc0b9330 0x7f2dcc06bf80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc06d9a0 tx=0x7f2dbc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c36da6979b31bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd0904640 1 -- 172.21.15.19:0/1615181042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc036070 con 0x7f2dcc0b9330 2024-04-25T09:56:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd0904640 1 -- 172.21.15.19:0/1615181042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc031600 con 0x7f2dcc0b9330 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd0904640 1 -- 172.21.15.19:0/1615181042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0319d0 con 0x7f2dcc0b9330 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/1615181042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dcc0b9330 msgr2=0x7f2dcc06bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd3b6a640 1 --2- 172.21.15.19:0/1615181042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dcc0b9330 0x7f2dcc06bf80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc06d9a0 tx=0x7f2dbc02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/1615181042 shutdown_connections 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd3b6a640 1 --2- 172.21.15.19:0/1615181042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dcc0b9330 0x7f2dcc06bf80 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd3b6a640 1 --2- 172.21.15.19:0/1615181042 >> v2:172.21.15.112:3300/0 conn(0x7f2dcc161710 0x7f2dcc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd3b6a640 1 --2- 172.21.15.19:0/1615181042 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc15dc20 0x7f2dcc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/1615181042 >> 172.21.15.19:0/1615181042 conn(0x7f2dcc0d0420 msgr2=0x7f2dcc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/1615181042 shutdown_connections 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.862+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/1615181042 wait complete. 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.863+0000 7f2dd3b6a640 1 Processor -- start 2024-04-25T09:56:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.863+0000 7f2dd3b6a640 1 -- start start 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.864+0000 7f2dd3b6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dcc0b9330 0x7f2dcc0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.864+0000 7f2dd3b6a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc15dc20 0x7f2dcc0aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.864+0000 7f2dd3b6a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dcc161710 0x7f2dcc159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.864+0000 7f2dd3b6a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0b5e50 con 0x7f2dcc0b9330 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.864+0000 7f2dd3b6a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc162500 con 0x7f2dcc161710 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.864+0000 7f2dd3b6a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0b5fd0 con 0x7f2dcc15dc20 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.864+0000 7f2dd1906640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dcc0b9330 0x7f2dcc0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.864+0000 7f2dd1906640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dcc0b9330 0x7f2dcc0b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49652/0 (socket says 172.21.15.19:49652) 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.864+0000 7f2dd1906640 1 -- 172.21.15.19:0/2335745546 learned_addr learned my addr 172.21.15.19:0/2335745546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dd2107640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dcc161710 0x7f2dcc159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dd1105640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc15dc20 0x7f2dcc0aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dd1906640 1 -- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc15dc20 msgr2=0x7f2dcc0aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dd1906640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc15dc20 0x7f2dcc0aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dd1906640 1 -- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dcc161710 msgr2=0x7f2dcc159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dd1906640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dcc161710 0x7f2dcc159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dd1906640 1 -- 172.21.15.19:0/2335745546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc15ffe0 con 0x7f2dcc0b9330 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dd1105640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc15dc20 0x7f2dcc0aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dd1906640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dcc0b9330 0x7f2dcc0b2110 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400d830 tx=0x7f2dc400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dca7fc640 1 -- 172.21.15.19:0/2335745546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4018030 con 0x7f2dcc0b9330 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dca7fc640 1 -- 172.21.15.19:0/2335745546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc400be10 con 0x7f2dcc0b9330 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/2335745546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc1602c0 con 0x7f2dcc0b9330 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dca7fc640 1 -- 172.21.15.19:0/2335745546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc40098b0 con 0x7f2dcc0b9330 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.865+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/2335745546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc160880 con 0x7f2dcc0b9330 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.866+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/2335745546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc073a00 con 0x7f2dcc0b9330 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.866+0000 7f2dca7fc640 1 -- 172.21.15.19:0/2335745546 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2dc401b070 con 0x7f2dcc0b9330 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.867+0000 7f2dca7fc640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2db806b4d0 0x7f2db806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.867+0000 7f2dca7fc640 1 -- 172.21.15.19:0/2335745546 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2dc408a2f0 con 0x7f2dcc0b9330 2024-04-25T09:56:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.867+0000 7f2dd1105640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2db806b4d0 0x7f2db806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.867+0000 7f2dd1105640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2db806b4d0 0x7f2db806d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006fd0 tx=0x7f2dc0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:36.871+0000 7f2dca7fc640 1 -- 172.21.15.19:0/2335745546 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2dc4059290 con 0x7f2dcc0b9330 2024-04-25T09:56:37.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.056+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/2335745546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2dcc0d16b0 con 0x7f2dcc0b9330 2024-04-25T09:56:37.301 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:56:37.304 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:56:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.425+0000 7f2dca7fc640 1 -- 172.21.15.19:0/2335745546 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2dc4058b90 con 0x7f2dcc0b9330 2024-04-25T09:56:37.428 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2db806b4d0 msgr2=0x7f2db806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2db806b4d0 0x7f2db806d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006fd0 tx=0x7f2dc0008040 comp rx=0 tx=0).stop 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/2335745546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dcc0b9330 msgr2=0x7f2dcc0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dcc0b9330 0x7f2dcc0b2110 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400d830 tx=0x7f2dc400dcf0 comp rx=0 tx=0).stop 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/2335745546 shutdown_connections 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2db806b4d0 0x7f2db806d990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dcc161710 0x7f2dcc159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc15dc20 0x7f2dcc0aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 --2- 172.21.15.19:0/2335745546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dcc0b9330 0x7f2dcc0b2110 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/2335745546 >> 172.21.15.19:0/2335745546 conn(0x7f2dcc0d0420 msgr2=0x7f2dcc0b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/2335745546 shutdown_connections 2024-04-25T09:56:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.427+0000 7f2dd3b6a640 1 -- 172.21.15.19:0/2335745546 wait complete. 2024-04-25T09:56:37.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:56:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.550+0000 7f0a7786b640 1 Processor -- start 2024-04-25T09:56:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.550+0000 7f0a7786b640 1 -- start start 2024-04-25T09:56:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a7786b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a7015da90 0x7f0a70161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a7786b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a70162040 con 0x7f0a70161ba0 2024-04-25T09:56:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a7786b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a70162320 con 0x7f0a700b9750 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a7786b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a70162600 con 0x7f0a7015da90 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a75e08640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0a70161ba0 0x7f0a700d2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51248/0 (socket says 172.21.15.19:51248) 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a75e08640 1 -- 172.21.15.19:0/1840249118 learned_addr learned my addr 172.21.15.19:0/1840249118 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a75607640 1 --2- 172.21.15.19:0/1840249118 >> v2:172.21.15.112:3300/0 conn(0x7f0a7015da90 0x7f0a70161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048887885 0 0) 0x7f0a70162040 con 0x7f0a70161ba0 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a5c003520 con 0x7f0a70161ba0 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2079227687 0 0) 0x7f0a5c003520 con 0x7f0a70161ba0 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 >> v2:172.21.15.112:3300/0 conn(0x7f0a7015da90 msgr2=0x7f0a70161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a67fff640 1 --2- 172.21.15.19:0/1840249118 >> v2:172.21.15.112:3300/0 conn(0x7f0a7015da90 0x7f0a70161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 >> v1:172.21.15.112:6789/0 conn(0x7f0a700b9750 legacy=0x7f0a700ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a700d1b10 con 0x7f0a70161ba0 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0a680019f0 con 0x7f0a70161ba0 2024-04-25T09:56:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 >> v1:172.21.15.19:6789/0 conn(0x7f0a70161ba0 legacy=0x7f0a700d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a67fff640 1 --2- 172.21.15.19:0/1840249118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a5c0039b0 0x7f0a5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.551+0000 7f0a67fff640 1 --2- 172.21.15.19:0/1840249118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a5c004800 0x7f0a5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a67fff640 1 --2- 172.21.15.19:0/1840249118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 0x7f0a5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a70162600 con 0x7f0a5c0051c0 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c00c160 con 0x7f0a5c004800 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c00c360 con 0x7f0a5c0039b0 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a75e08640 1 --2- 172.21.15.19:0/1840249118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 0x7f0a5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a75607640 1 --2- 172.21.15.19:0/1840249118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a5c0039b0 0x7f0a5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a74e06640 1 --2- 172.21.15.19:0/1840249118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a5c004800 0x7f0a5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a75e08640 1 -- 172.21.15.19:0/1840249118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a5c0039b0 msgr2=0x7f0a5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a75e08640 1 --2- 172.21.15.19:0/1840249118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a5c0039b0 0x7f0a5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a75e08640 1 -- 172.21.15.19:0/1840249118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a5c004800 msgr2=0x7f0a5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a75e08640 1 --2- 172.21.15.19:0/1840249118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a5c004800 0x7f0a5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.552+0000 7f0a75e08640 1 -- 172.21.15.19:0/1840249118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c003520 con 0x7f0a5c0051c0 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a75e08640 1 -- 172.21.15.19:0/1840249118 reap_dead start 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a75e08640 1 --2- 172.21.15.19:0/1840249118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 0x7f0a5c00b9c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800a860 tx=0x7f0a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35f554f82390c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6800a480 con 0x7f0a5c0051c0 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6802f070 con 0x7f0a5c0051c0 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a67fff640 1 -- 172.21.15.19:0/1840249118 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6802bcc0 con 0x7f0a5c0051c0 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a7786b640 1 -- 172.21.15.19:0/1840249118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 msgr2=0x7f0a5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a7786b640 1 --2- 172.21.15.19:0/1840249118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 0x7f0a5c00b9c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800a860 tx=0x7f0a5c005830 comp rx=0 tx=0).stop 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a7786b640 1 -- 172.21.15.19:0/1840249118 shutdown_connections 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a7786b640 1 --2- 172.21.15.19:0/1840249118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 0x7f0a5c00b9c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a7786b640 1 -- 172.21.15.19:0/1840249118 >> 172.21.15.19:0/1840249118 conn(0x7f0a70061a20 msgr2=0x7f0a70061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a7786b640 1 -- 172.21.15.19:0/1840249118 shutdown_connections 2024-04-25T09:56:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.553+0000 7f0a7786b640 1 -- 172.21.15.19:0/1840249118 wait complete. 2024-04-25T09:56:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.555+0000 7f0a7786b640 1 Processor -- start 2024-04-25T09:56:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.555+0000 7f0a7786b640 1 -- start start 2024-04-25T09:56:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.555+0000 7f0a7786b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 0x7f0a701517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.555+0000 7f0a7786b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a7015e380 0x7f0a700c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.555+0000 7f0a7786b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a700c7a10 0x7f0a7015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.555+0000 7f0a7786b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a70161660 con 0x7f0a5c0051c0 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.555+0000 7f0a7786b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7015de70 con 0x7f0a700c7a10 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.555+0000 7f0a7786b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a701617e0 con 0x7f0a7015e380 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.555+0000 7f0a75607640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 0x7f0a701517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.555+0000 7f0a75607640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 0x7f0a701517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49670/0 (socket says 172.21.15.19:49670) 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a75607640 1 -- 172.21.15.19:0/3048070944 learned_addr learned my addr 172.21.15.19:0/3048070944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a75e08640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a700c7a10 0x7f0a7015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a74e06640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a7015e380 0x7f0a700c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a75607640 1 -- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a7015e380 msgr2=0x7f0a700c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a75607640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a7015e380 0x7f0a700c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a75607640 1 -- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a700c7a10 msgr2=0x7f0a7015a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a75607640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a700c7a10 0x7f0a7015a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a75607640 1 -- 172.21.15.19:0/3048070944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a70152160 con 0x7f0a5c0051c0 2024-04-25T09:56:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a75607640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 0x7f0a701517a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c011910 tx=0x7f0a6c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a65ffb640 1 -- 172.21.15.19:0/3048070944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c00faf0 con 0x7f0a5c0051c0 2024-04-25T09:56:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a65ffb640 1 -- 172.21.15.19:0/3048070944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6c00fc90 con 0x7f0a5c0051c0 2024-04-25T09:56:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a7786b640 1 -- 172.21.15.19:0/3048070944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a700c8600 con 0x7f0a5c0051c0 2024-04-25T09:56:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a65ffb640 1 -- 172.21.15.19:0/3048070944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c014440 con 0x7f0a5c0051c0 2024-04-25T09:56:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.556+0000 7f0a7786b640 1 -- 172.21.15.19:0/3048070944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a700c8320 con 0x7f0a5c0051c0 2024-04-25T09:56:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.557+0000 7f0a7786b640 1 -- 172.21.15.19:0/3048070944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a70073a00 con 0x7f0a5c0051c0 2024-04-25T09:56:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.558+0000 7f0a65ffb640 1 -- 172.21.15.19:0/3048070944 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a6c012040 con 0x7f0a5c0051c0 2024-04-25T09:56:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.558+0000 7f0a65ffb640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a5406b4d0 0x7f0a5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.558+0000 7f0a65ffb640 1 -- 172.21.15.19:0/3048070944 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f0a6c08a800 con 0x7f0a5c0051c0 2024-04-25T09:56:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.559+0000 7f0a74e06640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a5406b4d0 0x7f0a5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.559+0000 7f0a74e06640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a5406b4d0 0x7f0a5406d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0a60000c00 tx=0x7f0a60009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:37.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.563+0000 7f0a65ffb640 1 -- 172.21.15.19:0/3048070944 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a6c019030 con 0x7f0a5c0051c0 2024-04-25T09:56:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:37.744+0000 7f0a7786b640 1 -- 172.21.15.19:0/3048070944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0a700d1590 con 0x7f0a5c0051c0 2024-04-25T09:56:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.427+0000 7f0a65ffb640 1 -- 172.21.15.19:0/3048070944 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0a6c0596d0 con 0x7f0a5c0051c0 2024-04-25T09:56:38.430 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T09:56:38.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 -- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a5406b4d0 msgr2=0x7f0a5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:38.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a5406b4d0 0x7f0a5406d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0a60000c00 tx=0x7f0a60009040 comp rx=0 tx=0).stop 2024-04-25T09:56:38.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 -- 172.21.15.19:0/3048070944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 msgr2=0x7f0a701517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:38.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 0x7f0a701517a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c011910 tx=0x7f0a6c011dd0 comp rx=0 tx=0).stop 2024-04-25T09:56:38.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 -- 172.21.15.19:0/3048070944 shutdown_connections 2024-04-25T09:56:38.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a5406b4d0 0x7f0a5406d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:38.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a700c7a10 0x7f0a7015a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:38.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a7015e380 0x7f0a700c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:38.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 --2- 172.21.15.19:0/3048070944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0051c0 0x7f0a701517a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:38.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 -- 172.21.15.19:0/3048070944 >> 172.21.15.19:0/3048070944 conn(0x7f0a70061a20 msgr2=0x7f0a7006c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:38.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 -- 172.21.15.19:0/3048070944 shutdown_connections 2024-04-25T09:56:38.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.430+0000 7f0a7786b640 1 -- 172.21.15.19:0/3048070944 wait complete. 2024-04-25T09:56:38.457 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:56:38.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.564+0000 7fe5952c4640 1 Processor -- start 2024-04-25T09:56:38.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.564+0000 7fe5952c4640 1 -- start start 2024-04-25T09:56:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.565+0000 7fe5952c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe59015dc20 0x7fe5900ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.565+0000 7fe5952c4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe5900d32f0 0x7fe5900cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.565+0000 7fe5952c4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59015e000 con 0x7fe590161710 2024-04-25T09:56:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.565+0000 7fe5952c4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5900d1de0 con 0x7fe5900d32f0 2024-04-25T09:56:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.565+0000 7fe5952c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe590161c60 con 0x7fe59015dc20 2024-04-25T09:56:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.565+0000 7fe58effd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe59015dc20 0x7fe5900ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.565+0000 7fe58f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe5900d32f0 0x7fe5900cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.565+0000 7fe58effd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe59015dc20 0x7fe5900ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:58014/0 (socket says 172.21.15.19:58014) 2024-04-25T09:56:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.565+0000 7fe58effd640 1 -- 172.21.15.19:0/3019941418 learned_addr learned my addr 172.21.15.19:0/3019941418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.565+0000 7fe587fff640 1 --1- 172.21.15.19:0/3019941418 >> v1:172.21.15.112:6789/0 conn(0x7fe590161710 0x7fe59006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:55006/0 (socket says 172.21.15.19:55006) 2024-04-25T09:56:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.566+0000 7fe58f7fe640 1 -- 172.21.15.19:0/3019941418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe59015dc20 msgr2=0x7fe5900ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.566+0000 7fe58f7fe640 1 --2- 172.21.15.19:0/3019941418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe59015dc20 0x7fe5900ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.566+0000 7fe58f7fe640 1 -- 172.21.15.19:0/3019941418 >> v1:172.21.15.112:6789/0 conn(0x7fe590161710 legacy=0x7fe59006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.566+0000 7fe58f7fe640 1 -- 172.21.15.19:0/3019941418 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590161f40 con 0x7fe5900d32f0 2024-04-25T09:56:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.566+0000 7fe58e7fc640 1 -- 172.21.15.19:0/3019941418 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418875223 0 0) 0x7fe59015e000 con 0x7fe590161710 2024-04-25T09:56:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.566+0000 7fe58f7fe640 1 --2- 172.21.15.19:0/3019941418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5900d32f0 0x7fe5900cebb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe588009d90 tx=0x7fe58802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bff19b1c1e4f8d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.566+0000 7fe58e7fc640 1 -- 172.21.15.19:0/3019941418 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588007740 con 0x7fe5900d32f0 2024-04-25T09:56:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.567+0000 7fe58e7fc640 1 -- 172.21.15.19:0/3019941418 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5880078e0 con 0x7fe5900d32f0 2024-04-25T09:56:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.567+0000 7fe58e7fc640 1 -- 172.21.15.19:0/3019941418 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588007cb0 con 0x7fe5900d32f0 2024-04-25T09:56:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.567+0000 7fe5952c4640 1 -- 172.21.15.19:0/3019941418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5900d32f0 msgr2=0x7fe5900cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.567+0000 7fe5952c4640 1 --2- 172.21.15.19:0/3019941418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5900d32f0 0x7fe5900cebb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe588009d90 tx=0x7fe58802a660 comp rx=0 tx=0).stop 2024-04-25T09:56:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.567+0000 7fe5952c4640 1 -- 172.21.15.19:0/3019941418 shutdown_connections 2024-04-25T09:56:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.567+0000 7fe5952c4640 1 --2- 172.21.15.19:0/3019941418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5900d32f0 0x7fe5900cebb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.567+0000 7fe5952c4640 1 --2- 172.21.15.19:0/3019941418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe59015dc20 0x7fe5900ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.567+0000 7fe5952c4640 1 -- 172.21.15.19:0/3019941418 >> 172.21.15.19:0/3019941418 conn(0x7fe5900d0420 msgr2=0x7fe5900d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.567+0000 7fe5952c4640 1 -- 172.21.15.19:0/3019941418 shutdown_connections 2024-04-25T09:56:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.567+0000 7fe5952c4640 1 -- 172.21.15.19:0/3019941418 wait complete. 2024-04-25T09:56:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.568+0000 7fe5952c4640 1 Processor -- start 2024-04-25T09:56:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.568+0000 7fe5952c4640 1 -- start start 2024-04-25T09:56:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.569+0000 7fe5952c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5900d32f0 0x7fe5900caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.569+0000 7fe5952c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe59015dc20 0x7fe5900cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.569+0000 7fe5952c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590161710 0x7fe5900cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.569+0000 7fe5952c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900b5e50 con 0x7fe590161710 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.569+0000 7fe5952c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe590162500 con 0x7fe5900d32f0 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.569+0000 7fe5952c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900b5fd0 con 0x7fe59015dc20 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.569+0000 7fe58f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590161710 0x7fe5900cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.569+0000 7fe58f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590161710 0x7fe5900cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53140/0 (socket says 172.21.15.19:53140) 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.569+0000 7fe58f7fe640 1 -- 172.21.15.19:0/1769388413 learned_addr learned my addr 172.21.15.19:0/1769388413 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.569+0000 7fe58effd640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5900d32f0 0x7fe5900caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.569+0000 7fe587fff640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe59015dc20 0x7fe5900cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.570+0000 7fe58f7fe640 1 -- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe59015dc20 msgr2=0x7fe5900cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.570+0000 7fe58f7fe640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe59015dc20 0x7fe5900cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.570+0000 7fe58f7fe640 1 -- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5900d32f0 msgr2=0x7fe5900caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.570+0000 7fe58f7fe640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5900d32f0 0x7fe5900caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.570+0000 7fe58f7fe640 1 -- 172.21.15.19:0/1769388413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59015fd80 con 0x7fe590161710 2024-04-25T09:56:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.570+0000 7fe58f7fe640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590161710 0x7fe5900cd270 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe588009d90 tx=0x7fe588004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.570+0000 7fe5877fe640 1 -- 172.21.15.19:0/1769388413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588040070 con 0x7fe590161710 2024-04-25T09:56:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.570+0000 7fe5877fe640 1 -- 172.21.15.19:0/1769388413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe588031c40 con 0x7fe590161710 2024-04-25T09:56:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.570+0000 7fe5952c4640 1 -- 172.21.15.19:0/1769388413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe590160060 con 0x7fe590161710 2024-04-25T09:56:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.570+0000 7fe5877fe640 1 -- 172.21.15.19:0/1769388413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588009450 con 0x7fe590161710 2024-04-25T09:56:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.570+0000 7fe5952c4640 1 -- 172.21.15.19:0/1769388413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe590160620 con 0x7fe590161710 2024-04-25T09:56:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.571+0000 7fe5952c4640 1 -- 172.21.15.19:0/1769388413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe590073a00 con 0x7fe590161710 2024-04-25T09:56:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.572+0000 7fe5877fe640 1 -- 172.21.15.19:0/1769388413 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe588039040 con 0x7fe590161710 2024-04-25T09:56:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.572+0000 7fe5877fe640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe57006b4d0 0x7fe57006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.572+0000 7fe5877fe640 1 -- 172.21.15.19:0/1769388413 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fe5880ab040 con 0x7fe590161710 2024-04-25T09:56:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.572+0000 7fe58effd640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe57006b4d0 0x7fe57006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.573+0000 7fe58effd640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe57006b4d0 0x7fe57006d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe580009b30 tx=0x7fe580009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.576+0000 7fe5877fe640 1 -- 172.21.15.19:0/1769388413 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe588079740 con 0x7fe590161710 2024-04-25T09:56:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:38.801+0000 7fe5952c4640 1 -- 172.21.15.19:0/1769388413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe5900d16b0 con 0x7fe590161710 2024-04-25T09:56:39.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.446+0000 7fe5877fe640 1 -- 172.21.15.19:0/1769388413 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v33) v1 ==== 173+0+0 (secure 0 0 0) 0x7fe58807d7e0 con 0x7fe590161710 2024-04-25T09:56:39.449 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T09:56:39.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.452+0000 7fe5952c4640 1 -- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe57006b4d0 msgr2=0x7fe57006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.452+0000 7fe5952c4640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe57006b4d0 0x7fe57006d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe580009b30 tx=0x7fe580009240 comp rx=0 tx=0).stop 2024-04-25T09:56:39.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.452+0000 7fe5952c4640 1 -- 172.21.15.19:0/1769388413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590161710 msgr2=0x7fe5900cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.452+0000 7fe5952c4640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590161710 0x7fe5900cd270 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe588009d90 tx=0x7fe588004a40 comp rx=0 tx=0).stop 2024-04-25T09:56:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.459+0000 7fe5952c4640 1 -- 172.21.15.19:0/1769388413 shutdown_connections 2024-04-25T09:56:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.459+0000 7fe5952c4640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe57006b4d0 0x7fe57006d990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.459+0000 7fe5952c4640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590161710 0x7fe5900cd270 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.459+0000 7fe5952c4640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe59015dc20 0x7fe5900cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.459+0000 7fe5952c4640 1 --2- 172.21.15.19:0/1769388413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5900d32f0 0x7fe5900caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.459+0000 7fe5952c4640 1 -- 172.21.15.19:0/1769388413 >> 172.21.15.19:0/1769388413 conn(0x7fe5900d0420 msgr2=0x7fe5900b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:39.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.461+0000 7fe5952c4640 1 -- 172.21.15.19:0/1769388413 shutdown_connections 2024-04-25T09:56:39.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.462+0000 7fe5952c4640 1 -- 172.21.15.19:0/1769388413 wait complete. 2024-04-25T09:56:39.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:39.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.582+0000 7f36576b3640 1 Processor -- start 2024-04-25T09:56:39.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.582+0000 7f36576b3640 1 -- start start 2024-04-25T09:56:39.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.582+0000 7f36576b3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f36500b93a0 0x7f36501616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f36576b3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365015e000 con 0x7f36500cedf0 2024-04-25T09:56:39.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f36576b3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36500b6bb0 con 0x7f365015dc20 2024-04-25T09:56:39.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f36576b3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36500b6030 con 0x7f36500b93a0 2024-04-25T09:56:39.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f3654c4e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f36500cedf0 0x7f36500d2510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36576/0 (socket says 172.21.15.19:36576) 2024-04-25T09:56:39.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f3654c4e640 1 -- 172.21.15.19:0/2921792032 learned_addr learned my addr 172.21.15.19:0/2921792032 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f3655c50640 1 --2- 172.21.15.19:0/2921792032 >> v2:172.21.15.112:3300/0 conn(0x7f36500b93a0 0x7f36501616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130081652 0 0) 0x7f3644001110 con 0x7f36500cedf0 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3638003520 con 0x7f36500cedf0 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2930832115 0 0) 0x7f3638003520 con 0x7f36500cedf0 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 >> v2:172.21.15.112:3300/0 conn(0x7f36500b93a0 msgr2=0x7f36501616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 --2- 172.21.15.19:0/2921792032 >> v2:172.21.15.112:3300/0 conn(0x7f36500b93a0 0x7f36501616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 >> v1:172.21.15.112:6789/0 conn(0x7f365015dc20 legacy=0x7f36500ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3650161de0 con 0x7f36500cedf0 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f3644001840 con 0x7f36500cedf0 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 >> v1:172.21.15.19:6789/0 conn(0x7f36500cedf0 legacy=0x7f36500d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 --2- 172.21.15.19:0/2921792032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36380039b0 0x7f3638003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 --2- 172.21.15.19:0/2921792032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3638004800 0x7f3638004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 --2- 172.21.15.19:0/2921792032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36380051c0 0x7f363800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36500b6030 con 0x7f3638004800 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363800c060 con 0x7f36380051c0 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363800c260 con 0x7f36380039b0 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.583+0000 7f3654c4e640 1 --2- 172.21.15.19:0/2921792032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3638004800 0x7f3638004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.584+0000 7f3655c50640 1 --2- 172.21.15.19:0/2921792032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36380051c0 0x7f363800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.584+0000 7f365544f640 1 --2- 172.21.15.19:0/2921792032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36380039b0 0x7f3638003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.584+0000 7f3654c4e640 1 -- 172.21.15.19:0/2921792032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36380039b0 msgr2=0x7f3638003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.584+0000 7f3654c4e640 1 --2- 172.21.15.19:0/2921792032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36380039b0 0x7f3638003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.584+0000 7f3654c4e640 1 -- 172.21.15.19:0/2921792032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36380051c0 msgr2=0x7f363800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.584+0000 7f3654c4e640 1 --2- 172.21.15.19:0/2921792032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36380051c0 0x7f363800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.584+0000 7f3654c4e640 1 -- 172.21.15.19:0/2921792032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638003520 con 0x7f3638004800 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.584+0000 7f3655c50640 1 -- 172.21.15.19:0/2921792032 reap_dead start 2024-04-25T09:56:39.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.584+0000 7f3654c4e640 1 --2- 172.21.15.19:0/2921792032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3638004800 0x7f3638004c80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f364400c4c0 tx=0x7f3638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d186748c5e84fd86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.585+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364400c0e0 con 0x7f3638004800 2024-04-25T09:56:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.585+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3644034070 con 0x7f3638004800 2024-04-25T09:56:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.585+0000 7f364ffff640 1 -- 172.21.15.19:0/2921792032 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644005d70 con 0x7f3638004800 2024-04-25T09:56:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.585+0000 7f36576b3640 1 -- 172.21.15.19:0/2921792032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3638004800 msgr2=0x7f3638004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.585+0000 7f36576b3640 1 --2- 172.21.15.19:0/2921792032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3638004800 0x7f3638004c80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f364400c4c0 tx=0x7f3638005830 comp rx=0 tx=0).stop 2024-04-25T09:56:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.585+0000 7f36576b3640 1 -- 172.21.15.19:0/2921792032 shutdown_connections 2024-04-25T09:56:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.585+0000 7f36576b3640 1 --2- 172.21.15.19:0/2921792032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3638004800 0x7f3638004c80 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.585+0000 7f36576b3640 1 -- 172.21.15.19:0/2921792032 >> 172.21.15.19:0/2921792032 conn(0x7f36500d0490 msgr2=0x7f36500d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.585+0000 7f36576b3640 1 -- 172.21.15.19:0/2921792032 shutdown_connections 2024-04-25T09:56:39.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.585+0000 7f36576b3640 1 -- 172.21.15.19:0/2921792032 wait complete. 2024-04-25T09:56:39.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.586+0000 7f36576b3640 1 Processor -- start 2024-04-25T09:56:39.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.586+0000 7f36576b3640 1 -- start start 2024-04-25T09:56:39.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.587+0000 7f36576b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3638004800 0x7f365015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.587+0000 7f36576b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36500b51f0 0x7f36500b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.587+0000 7f36576b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36501e7fa0 0x7f36501e8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.587+0000 7f36576b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36501616b0 con 0x7f36500b51f0 2024-04-25T09:56:39.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.587+0000 7f36576b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36501623a0 con 0x7f3638004800 2024-04-25T09:56:39.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.587+0000 7f36576b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3650161830 con 0x7f36501e7fa0 2024-04-25T09:56:39.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.587+0000 7f3654c4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36500b51f0 0x7f36500b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.587+0000 7f3654c4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36500b51f0 0x7f36500b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53150/0 (socket says 172.21.15.19:53150) 2024-04-25T09:56:39.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.587+0000 7f3654c4e640 1 -- 172.21.15.19:0/2555548005 learned_addr learned my addr 172.21.15.19:0/2555548005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:39.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.587+0000 7f3655c50640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36501e7fa0 0x7f36501e8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.587+0000 7f365544f640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3638004800 0x7f365015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.588+0000 7f3654c4e640 1 -- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36501e7fa0 msgr2=0x7f36501e8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.588+0000 7f3654c4e640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36501e7fa0 0x7f36501e8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.588+0000 7f3654c4e640 1 -- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3638004800 msgr2=0x7f365015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.588+0000 7f3654c4e640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3638004800 0x7f365015c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.588+0000 7f3654c4e640 1 -- 172.21.15.19:0/2555548005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36500ab690 con 0x7f36500b51f0 2024-04-25T09:56:39.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.588+0000 7f3654c4e640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36500b51f0 0x7f36500b5670 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f364400c4c0 tx=0x7f3650161fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:39.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.588+0000 7f364dffb640 1 -- 172.21.15.19:0/2555548005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644007270 con 0x7f36500b51f0 2024-04-25T09:56:39.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.588+0000 7f364dffb640 1 -- 172.21.15.19:0/2555548005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364403f030 con 0x7f36500b51f0 2024-04-25T09:56:39.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.588+0000 7f36576b3640 1 -- 172.21.15.19:0/2555548005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36500ab910 con 0x7f36500b51f0 2024-04-25T09:56:39.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.588+0000 7f364dffb640 1 -- 172.21.15.19:0/2555548005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644007630 con 0x7f36500b51f0 2024-04-25T09:56:39.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.588+0000 7f36576b3640 1 -- 172.21.15.19:0/2555548005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36500abed0 con 0x7f36500b51f0 2024-04-25T09:56:39.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.589+0000 7f36576b3640 1 -- 172.21.15.19:0/2555548005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36500ca3b0 con 0x7f36500b51f0 2024-04-25T09:56:39.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.590+0000 7f364dffb640 1 -- 172.21.15.19:0/2555548005 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f364402e070 con 0x7f36500b51f0 2024-04-25T09:56:39.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.590+0000 7f364dffb640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f363c06b4d0 0x7f363c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.590+0000 7f364dffb640 1 -- 172.21.15.19:0/2555548005 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f36440aa7b0 con 0x7f36500b51f0 2024-04-25T09:56:39.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.590+0000 7f365544f640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f363c06b4d0 0x7f363c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.591+0000 7f365544f640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f363c06b4d0 0x7f363c06d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3648000c00 tx=0x7f3648009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:39.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.594+0000 7f364dffb640 1 -- 172.21.15.19:0/2555548005 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36500ca3b0 con 0x7f36500b51f0 2024-04-25T09:56:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.775+0000 7f36576b3640 1 -- 172.21.15.19:0/2555548005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3650073990 con 0x7f36500b51f0 2024-04-25T09:56:39.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.776+0000 7f364dffb640 1 -- 172.21.15.19:0/2555548005 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f3644049030 con 0x7f36500b51f0 2024-04-25T09:56:39.780 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:39.780 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":63,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:56:38.802193+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:56:37.087339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:56:37.775183+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.778+0000 7f36576b3640 1 -- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f363c06b4d0 msgr2=0x7f363c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.778+0000 7f36576b3640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f363c06b4d0 0x7f363c06d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3648000c00 tx=0x7f3648009040 comp rx=0 tx=0).stop 2024-04-25T09:56:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.778+0000 7f36576b3640 1 -- 172.21.15.19:0/2555548005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36500b51f0 msgr2=0x7f36500b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.778+0000 7f36576b3640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36500b51f0 0x7f36500b5670 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f364400c4c0 tx=0x7f3650161fb0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.779+0000 7f36576b3640 1 -- 172.21.15.19:0/2555548005 shutdown_connections 2024-04-25T09:56:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.779+0000 7f36576b3640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f363c06b4d0 0x7f363c06d990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.779+0000 7f36576b3640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36501e7fa0 0x7f36501e8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.779+0000 7f36576b3640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36500b51f0 0x7f36500b5670 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.779+0000 7f36576b3640 1 --2- 172.21.15.19:0/2555548005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3638004800 0x7f365015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.779+0000 7f36576b3640 1 -- 172.21.15.19:0/2555548005 >> 172.21.15.19:0/2555548005 conn(0x7f36500d0490 msgr2=0x7f36500d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:39.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.779+0000 7f36576b3640 1 -- 172.21.15.19:0/2555548005 shutdown_connections 2024-04-25T09:56:39.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.779+0000 7f36576b3640 1 -- 172.21.15.19:0/2555548005 wait complete. 2024-04-25T09:56:39.793 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.891+0000 7fbf69f5c640 1 Processor -- start 2024-04-25T09:56:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.891+0000 7fbf69f5c640 1 -- start start 2024-04-25T09:56:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.892+0000 7fbf69f5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbf6415dc00 0x7fbf640c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.892+0000 7fbf69f5c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6415dfe0 con 0x7fbf640ceb20 2024-04-25T09:56:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.892+0000 7fbf69f5c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf640d00c0 con 0x7fbf640d1dd0 2024-04-25T09:56:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.892+0000 7fbf69f5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf640d03a0 con 0x7fbf6415dc00 2024-04-25T09:56:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.892+0000 7fbf62ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbf640ceb20 0x7fbf640d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36588/0 (socket says 172.21.15.19:36588) 2024-04-25T09:56:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.892+0000 7fbf62ffd640 1 -- 172.21.15.19:0/479106749 learned_addr learned my addr 172.21.15.19:0/479106749 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.892+0000 7fbf637fe640 1 --2- 172.21.15.19:0/479106749 >> v2:172.21.15.112:3300/0 conn(0x7fbf6415dc00 0x7fbf640c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.892+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568432861 0 0) 0x7fbf58001110 con 0x7fbf640ceb20 2024-04-25T09:56:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.892+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf4c003520 con 0x7fbf640ceb20 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3096188749 0 0) 0x7fbf4c003520 con 0x7fbf640ceb20 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 >> v2:172.21.15.112:3300/0 conn(0x7fbf6415dc00 msgr2=0x7fbf640c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 --2- 172.21.15.19:0/479106749 >> v2:172.21.15.112:3300/0 conn(0x7fbf6415dc00 0x7fbf640c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 >> v1:172.21.15.112:6789/0 conn(0x7fbf640d1dd0 legacy=0x7fbf640ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf640cc440 con 0x7fbf640ceb20 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fbf58001840 con 0x7fbf640ceb20 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 >> v1:172.21.15.19:6789/0 conn(0x7fbf640ceb20 legacy=0x7fbf640d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 --2- 172.21.15.19:0/479106749 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf4c0039b0 0x7fbf4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 --2- 172.21.15.19:0/479106749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf4c004800 0x7fbf4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 --2- 172.21.15.19:0/479106749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf4c0051c0 0x7fbf4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640d03a0 con 0x7fbf4c004800 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c00c060 con 0x7fbf4c0051c0 2024-04-25T09:56:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c00c260 con 0x7fbf4c0039b0 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf62ffd640 1 --2- 172.21.15.19:0/479106749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf4c004800 0x7fbf4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf637fe640 1 --2- 172.21.15.19:0/479106749 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf4c0039b0 0x7fbf4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf63fff640 1 --2- 172.21.15.19:0/479106749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf4c0051c0 0x7fbf4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf62ffd640 1 -- 172.21.15.19:0/479106749 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf4c0039b0 msgr2=0x7fbf4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf62ffd640 1 --2- 172.21.15.19:0/479106749 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf4c0039b0 0x7fbf4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf62ffd640 1 -- 172.21.15.19:0/479106749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf4c0051c0 msgr2=0x7fbf4c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf62ffd640 1 --2- 172.21.15.19:0/479106749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf4c0051c0 0x7fbf4c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf62ffd640 1 -- 172.21.15.19:0/479106749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c003520 con 0x7fbf4c004800 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.893+0000 7fbf63fff640 1 -- 172.21.15.19:0/479106749 reap_dead start 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.894+0000 7fbf62ffd640 1 --2- 172.21.15.19:0/479106749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf4c004800 0x7fbf4c004c80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c4c0 tx=0x7fbf4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914f9713bb534861 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.894+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800c0e0 con 0x7fbf4c004800 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.894+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58034070 con 0x7fbf4c004800 2024-04-25T09:56:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.894+0000 7fbf627fc640 1 -- 172.21.15.19:0/479106749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58005d70 con 0x7fbf4c004800 2024-04-25T09:56:39.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.895+0000 7fbf69f5c640 1 -- 172.21.15.19:0/479106749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf4c004800 msgr2=0x7fbf4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.895+0000 7fbf69f5c640 1 --2- 172.21.15.19:0/479106749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf4c004800 0x7fbf4c004c80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c4c0 tx=0x7fbf4c005830 comp rx=0 tx=0).stop 2024-04-25T09:56:39.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.895+0000 7fbf69f5c640 1 -- 172.21.15.19:0/479106749 shutdown_connections 2024-04-25T09:56:39.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.895+0000 7fbf69f5c640 1 --2- 172.21.15.19:0/479106749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf4c004800 0x7fbf4c004c80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.895+0000 7fbf69f5c640 1 -- 172.21.15.19:0/479106749 >> 172.21.15.19:0/479106749 conn(0x7fbf6406f5d0 msgr2=0x7fbf6406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:39.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.895+0000 7fbf69f5c640 1 -- 172.21.15.19:0/479106749 shutdown_connections 2024-04-25T09:56:39.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.895+0000 7fbf69f5c640 1 -- 172.21.15.19:0/479106749 wait complete. 2024-04-25T09:56:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.896+0000 7fbf69f5c640 1 Processor -- start 2024-04-25T09:56:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.896+0000 7fbf69f5c640 1 -- start start 2024-04-25T09:56:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf69f5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf4c004800 0x7fbf64145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf69f5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf641e98c0 0x7fbf6415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf69f5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6415cf90 0x7fbf64159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf69f5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640cfd90 con 0x7fbf641e98c0 2024-04-25T09:56:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf69f5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640cc130 con 0x7fbf6415cf90 2024-04-25T09:56:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf69f5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640cdd70 con 0x7fbf4c004800 2024-04-25T09:56:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf62ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf641e98c0 0x7fbf6415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf62ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf641e98c0 0x7fbf6415ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53162/0 (socket says 172.21.15.19:53162) 2024-04-25T09:56:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf62ffd640 1 -- 172.21.15.19:0/1954914466 learned_addr learned my addr 172.21.15.19:0/1954914466 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf63fff640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6415cf90 0x7fbf64159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf637fe640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf4c004800 0x7fbf64145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf62ffd640 1 -- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf4c004800 msgr2=0x7fbf64145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf62ffd640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf4c004800 0x7fbf64145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf62ffd640 1 -- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6415cf90 msgr2=0x7fbf64159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf62ffd640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6415cf90 0x7fbf64159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf62ffd640 1 -- 172.21.15.19:0/1954914466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf641e8820 con 0x7fbf641e98c0 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.897+0000 7fbf62ffd640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf641e98c0 0x7fbf6415ca50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c4c0 tx=0x7fbf58001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.898+0000 7fbf4bfff640 1 -- 172.21.15.19:0/1954914466 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58007270 con 0x7fbf641e98c0 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.898+0000 7fbf4bfff640 1 -- 172.21.15.19:0/1954914466 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5803f030 con 0x7fbf641e98c0 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.898+0000 7fbf69f5c640 1 -- 172.21.15.19:0/1954914466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf641e8b00 con 0x7fbf641e98c0 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.898+0000 7fbf4bfff640 1 -- 172.21.15.19:0/1954914466 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58007630 con 0x7fbf641e98c0 2024-04-25T09:56:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.898+0000 7fbf69f5c640 1 -- 172.21.15.19:0/1954914466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf641e90c0 con 0x7fbf641e98c0 2024-04-25T09:56:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.899+0000 7fbf69f5c640 1 -- 172.21.15.19:0/1954914466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6415d370 con 0x7fbf641e98c0 2024-04-25T09:56:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.899+0000 7fbf4bfff640 1 -- 172.21.15.19:0/1954914466 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf5802f480 con 0x7fbf641e98c0 2024-04-25T09:56:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.899+0000 7fbf4bfff640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf5006b500 0x7fbf5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.900+0000 7fbf4bfff640 1 -- 172.21.15.19:0/1954914466 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbf580aabe0 con 0x7fbf641e98c0 2024-04-25T09:56:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.900+0000 7fbf637fe640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf5006b500 0x7fbf5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.900+0000 7fbf637fe640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf5006b500 0x7fbf5006d9c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009c40 tx=0x7fbf54009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:39.904+0000 7fbf4bfff640 1 -- 172.21.15.19:0/1954914466 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf58049030 con 0x7fbf641e98c0 2024-04-25T09:56:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.089+0000 7fbf69f5c640 1 -- 172.21.15.19:0/1954914466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf64073990 con 0x7fbf641e98c0 2024-04-25T09:56:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.090+0000 7fbf4bfff640 1 -- 172.21.15.19:0/1954914466 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16885 (secure 0 0 0) 0x7fbf6415dfe0 con 0x7fbf641e98c0 2024-04-25T09:56:40.094 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:40.094 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":63,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:56:38.802193+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:56:37.087339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:56:37.775183+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.092+0000 7fbf69f5c640 1 -- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf5006b500 msgr2=0x7fbf5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.092+0000 7fbf69f5c640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf5006b500 0x7fbf5006d9c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009c40 tx=0x7fbf54009190 comp rx=0 tx=0).stop 2024-04-25T09:56:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.092+0000 7fbf69f5c640 1 -- 172.21.15.19:0/1954914466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf641e98c0 msgr2=0x7fbf6415ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.092+0000 7fbf69f5c640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf641e98c0 0x7fbf6415ca50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c4c0 tx=0x7fbf58001c10 comp rx=0 tx=0).stop 2024-04-25T09:56:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.092+0000 7fbf69f5c640 1 -- 172.21.15.19:0/1954914466 shutdown_connections 2024-04-25T09:56:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.092+0000 7fbf69f5c640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbf5006b500 0x7fbf5006d9c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.092+0000 7fbf69f5c640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf6415cf90 0x7fbf64159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.092+0000 7fbf69f5c640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf641e98c0 0x7fbf6415ca50 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.092+0000 7fbf69f5c640 1 --2- 172.21.15.19:0/1954914466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbf4c004800 0x7fbf64145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.093+0000 7fbf69f5c640 1 -- 172.21.15.19:0/1954914466 >> 172.21.15.19:0/1954914466 conn(0x7fbf6406f5d0 msgr2=0x7fbf640ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.093+0000 7fbf69f5c640 1 -- 172.21.15.19:0/1954914466 shutdown_connections 2024-04-25T09:56:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.093+0000 7fbf69f5c640 1 -- 172.21.15.19:0/1954914466 wait complete. 2024-04-25T09:56:40.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:56:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.207+0000 7f6887ecb640 1 Processor -- start 2024-04-25T09:56:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.208+0000 7f6887ecb640 1 -- start start 2024-04-25T09:56:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.208+0000 7f6887ecb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68800cadb0 0x7f68800cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.208+0000 7f6887ecb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f68800ccfe0 0x7f68800cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.208+0000 7f6887ecb640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68800afc80 con 0x7f688015de80 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.208+0000 7f6887ecb640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68800cb210 con 0x7f68800ccfe0 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.208+0000 7f6887ecb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68800ce4d0 con 0x7f68800cadb0 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6885c67640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f688015de80 0x7f68800cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36598/0 (socket says 172.21.15.19:36598) 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6885c67640 1 -- 172.21.15.19:0/979753433 learned_addr learned my addr 172.21.15.19:0/979753433 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6885466640 1 --2- 172.21.15.19:0/979753433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68800cadb0 0x7f68800cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6886468640 1 --2- 172.21.15.19:0/979753433 >> v2:172.21.15.112:3300/0 conn(0x7f68800ccfe0 0x7f68800cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6884c65640 1 -- 172.21.15.19:0/979753433 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656991816 0 0) 0x7f68800afc80 con 0x7f688015de80 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6884c65640 1 -- 172.21.15.19:0/979753433 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f686c003520 con 0x7f688015de80 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6884c65640 1 -- 172.21.15.19:0/979753433 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3233878502 0 0) 0x7f686c003520 con 0x7f688015de80 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6885466640 1 -- 172.21.15.19:0/979753433 >> v2:172.21.15.112:3300/0 conn(0x7f68800ccfe0 msgr2=0x7f68800cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6885466640 1 --2- 172.21.15.19:0/979753433 >> v2:172.21.15.112:3300/0 conn(0x7f68800ccfe0 0x7f68800cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6885466640 1 -- 172.21.15.19:0/979753433 >> v1:172.21.15.19:6789/0 conn(0x7f688015de80 legacy=0x7f68800cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6885466640 1 -- 172.21.15.19:0/979753433 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68800ce7b0 con 0x7f68800cadb0 2024-04-25T09:56:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.209+0000 7f6886468640 1 --2- 172.21.15.19:0/979753433 >> v2:172.21.15.112:3300/0 conn(0x7f68800ccfe0 0x7f68800cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.210+0000 7f6885466640 1 --2- 172.21.15.19:0/979753433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68800cadb0 0x7f68800cf6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f68700097a0 tx=0x7f687002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f18efda07f223bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.210+0000 7f6884c65640 1 -- 172.21.15.19:0/979753433 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6870037070 con 0x7f68800cadb0 2024-04-25T09:56:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.210+0000 7f6884c65640 1 -- 172.21.15.19:0/979753433 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687002aba0 con 0x7f68800cadb0 2024-04-25T09:56:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.210+0000 7f6884c65640 1 -- 172.21.15.19:0/979753433 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c001670 con 0x7f68800cadb0 2024-04-25T09:56:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.211+0000 7f6887ecb640 1 -- 172.21.15.19:0/979753433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68800cadb0 msgr2=0x7f68800cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.211+0000 7f6887ecb640 1 --2- 172.21.15.19:0/979753433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68800cadb0 0x7f68800cf6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f68700097a0 tx=0x7f687002a110 comp rx=0 tx=0).stop 2024-04-25T09:56:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.211+0000 7f6887ecb640 1 -- 172.21.15.19:0/979753433 shutdown_connections 2024-04-25T09:56:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.211+0000 7f6887ecb640 1 --2- 172.21.15.19:0/979753433 >> v2:172.21.15.112:3300/0 conn(0x7f68800ccfe0 0x7f68800cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.211+0000 7f6887ecb640 1 --2- 172.21.15.19:0/979753433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68800cadb0 0x7f68800cf6c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.211+0000 7f6887ecb640 1 -- 172.21.15.19:0/979753433 >> 172.21.15.19:0/979753433 conn(0x7f68800b2390 msgr2=0x7f68800b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.211+0000 7f6887ecb640 1 -- 172.21.15.19:0/979753433 shutdown_connections 2024-04-25T09:56:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.211+0000 7f6887ecb640 1 -- 172.21.15.19:0/979753433 wait complete. 2024-04-25T09:56:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.212+0000 7f6887ecb640 1 Processor -- start 2024-04-25T09:56:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.212+0000 7f6887ecb640 1 -- start start 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6887ecb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68800cadb0 0x7f68800ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6887ecb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68800ccfe0 0x7f68801f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6887ecb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f688015de80 0x7f68801ed190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6887ecb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688015e260 con 0x7f688015de80 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6887ecb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68800cfc00 con 0x7f68800cadb0 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6887ecb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68800b2eb0 con 0x7f68800ccfe0 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6886468640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f688015de80 0x7f68801ed190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6886468640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f688015de80 0x7f68801ed190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53178/0 (socket says 172.21.15.19:53178) 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6886468640 1 -- 172.21.15.19:0/1672813229 learned_addr learned my addr 172.21.15.19:0/1672813229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6885c67640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68800cadb0 0x7f68800ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6885466640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68800ccfe0 0x7f68801f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6886468640 1 -- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68800ccfe0 msgr2=0x7f68801f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6886468640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68800ccfe0 0x7f68801f4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6886468640 1 -- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68800cadb0 msgr2=0x7f68800ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6886468640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68800cadb0 0x7f68800ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6886468640 1 -- 172.21.15.19:0/1672813229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68801f45a0 con 0x7f688015de80 2024-04-25T09:56:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.213+0000 7f6886468640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f688015de80 0x7f68801ed190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f687800efc0 tx=0x7f6878009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.214+0000 7f68767fc640 1 -- 172.21.15.19:0/1672813229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878018030 con 0x7f688015de80 2024-04-25T09:56:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.214+0000 7f68767fc640 1 -- 172.21.15.19:0/1672813229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687800cde0 con 0x7f688015de80 2024-04-25T09:56:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.214+0000 7f6887ecb640 1 -- 172.21.15.19:0/1672813229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68800adc00 con 0x7f688015de80 2024-04-25T09:56:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.214+0000 7f68767fc640 1 -- 172.21.15.19:0/1672813229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878003bb0 con 0x7f688015de80 2024-04-25T09:56:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.214+0000 7f6887ecb640 1 -- 172.21.15.19:0/1672813229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68800ae4a0 con 0x7f688015de80 2024-04-25T09:56:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.215+0000 7f6887ecb640 1 -- 172.21.15.19:0/1672813229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6880073a00 con 0x7f688015de80 2024-04-25T09:56:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.215+0000 7f68767fc640 1 -- 172.21.15.19:0/1672813229 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6878005d10 con 0x7f688015de80 2024-04-25T09:56:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.216+0000 7f68767fc640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f686406b4d0 0x7f686406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.216+0000 7f68767fc640 1 -- 172.21.15.19:0/1672813229 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f687808ad00 con 0x7f688015de80 2024-04-25T09:56:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.216+0000 7f6885c67640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f686406b4d0 0x7f686406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.216+0000 7f6885c67640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f686406b4d0 0x7f686406d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f687c005880 tx=0x7f687c0057f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:40.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.220+0000 7f68767fc640 1 -- 172.21.15.19:0/1672813229 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6878022070 con 0x7f688015de80 2024-04-25T09:56:40.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.438+0000 7f6887ecb640 1 -- 172.21.15.19:0/1672813229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f688015c9c0 con 0x7f688015de80 2024-04-25T09:56:40.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.447+0000 7f68767fc640 1 -- 172.21.15.19:0/1672813229 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f68801f45a0 con 0x7f688015de80 2024-04-25T09:56:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 -- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f686406b4d0 msgr2=0x7f686406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f686406b4d0 0x7f686406d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f687c005880 tx=0x7f687c0057f0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 -- 172.21.15.19:0/1672813229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f688015de80 msgr2=0x7f68801ed190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f688015de80 0x7f68801ed190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f687800efc0 tx=0x7f6878009f90 comp rx=0 tx=0).stop 2024-04-25T09:56:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 -- 172.21.15.19:0/1672813229 shutdown_connections 2024-04-25T09:56:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f686406b4d0 0x7f686406d990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f688015de80 0x7f68801ed190 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68800ccfe0 0x7f68801f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 --2- 172.21.15.19:0/1672813229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68800cadb0 0x7f68800ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 -- 172.21.15.19:0/1672813229 >> 172.21.15.19:0/1672813229 conn(0x7f68800b2390 msgr2=0x7f68800b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 -- 172.21.15.19:0/1672813229 shutdown_connections 2024-04-25T09:56:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.449+0000 7f6887ecb640 1 -- 172.21.15.19:0/1672813229 wait complete. 2024-04-25T09:56:40.464 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:56:40.464 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:56:40.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.564+0000 7f826d9d5640 1 Processor -- start 2024-04-25T09:56:40.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.564+0000 7f826d9d5640 1 -- start start 2024-04-25T09:56:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.565+0000 7f826d9d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82680d00e0 0x7f82680ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.565+0000 7f826d9d5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82680cb190 con 0x7f82680ccdd0 2024-04-25T09:56:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.565+0000 7f826d9d5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82680cb470 con 0x7f826815bb20 2024-04-25T09:56:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.565+0000 7f826d9d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82680cb750 con 0x7f82680d00e0 2024-04-25T09:56:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.565+0000 7f82667fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f82680ccdd0 0x7f82680cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36602/0 (socket says 172.21.15.19:36602) 2024-04-25T09:56:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.565+0000 7f82667fc640 1 -- 172.21.15.19:0/1402517080 learned_addr learned my addr 172.21.15.19:0/1402517080 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f82677fe640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82680d00e0 0x7f82680ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835224463 0 0) 0x7f82680cb190 con 0x7f82680ccdd0 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8254003520 con 0x7f82680ccdd0 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2987846827 0 0) 0x7f8254003520 con 0x7f82680ccdd0 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82680d00e0 msgr2=0x7f82680ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82680d00e0 0x7f82680ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 >> v1:172.21.15.112:6789/0 conn(0x7f826815bb20 legacy=0x7f82680cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82680cba30 con 0x7f82680ccdd0 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8250001670 con 0x7f82680ccdd0 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 >> v1:172.21.15.19:6789/0 conn(0x7f82680ccdd0 legacy=0x7f82680cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82540039e0 0x7f8254003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82680d00e0 0x7f8254004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8254004d40 0x7f82540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82680cb750 con 0x7f8254004d40 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825400baa0 con 0x7f82540039e0 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825400bca0 con 0x7f82680d00e0 2024-04-25T09:56:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f82677fe640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8254004d40 0x7f82540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f82667fc640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82680d00e0 0x7f8254004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f8266ffd640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82540039e0 0x7f8254003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f82677fe640 1 -- 172.21.15.19:0/1402517080 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82680d00e0 msgr2=0x7f8254004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f82677fe640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82680d00e0 0x7f8254004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.566+0000 7f82677fe640 1 -- 172.21.15.19:0/1402517080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82540039e0 msgr2=0x7f8254003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.567+0000 7f82677fe640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82540039e0 0x7f8254003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.567+0000 7f82677fe640 1 -- 172.21.15.19:0/1402517080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8254003520 con 0x7f8254004d40 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.567+0000 7f82677fe640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8254004d40 0x7f82540051c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8258009fc0 tx=0x7f8254005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd06f705b472a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.567+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258009c10 con 0x7f8254004d40 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.567+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8258036070 con 0x7f8254004d40 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.567+0000 7f8265ffb640 1 -- 172.21.15.19:0/1402517080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258031880 con 0x7f8254004d40 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.568+0000 7f826d9d5640 1 -- 172.21.15.19:0/1402517080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8254004d40 msgr2=0x7f82540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.568+0000 7f826d9d5640 1 --2- 172.21.15.19:0/1402517080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8254004d40 0x7f82540051c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8258009fc0 tx=0x7f8254005830 comp rx=0 tx=0).stop 2024-04-25T09:56:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.568+0000 7f82677fe640 1 -- 172.21.15.19:0/1402517080 reap_dead start 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.568+0000 7f826d9d5640 1 -- 172.21.15.19:0/1402517080 shutdown_connections 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.568+0000 7f826d9d5640 1 -- 172.21.15.19:0/1402517080 >> 172.21.15.19:0/1402517080 conn(0x7f82680b2390 msgr2=0x7f82680b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.568+0000 7f826d9d5640 1 -- 172.21.15.19:0/1402517080 shutdown_connections 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.568+0000 7f826d9d5640 1 -- 172.21.15.19:0/1402517080 wait complete. 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.569+0000 7f826d9d5640 1 Processor -- start 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.569+0000 7f826d9d5640 1 -- start start 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f826d9d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8254004d40 0x7f82680b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f826d9d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82681f1dd0 0x7f82680c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f826d9d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f826809a780 0x7f826809ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f826d9d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826815c080 con 0x7f82681f1dd0 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f826d9d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826815bf00 con 0x7f8254004d40 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f826d9d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82680cadb0 con 0x7f826809a780 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f82667fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82681f1dd0 0x7f82680c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f82667fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82681f1dd0 0x7f82680c9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53196/0 (socket says 172.21.15.19:53196) 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f82667fc640 1 -- 172.21.15.19:0/1826695182 learned_addr learned my addr 172.21.15.19:0/1826695182 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f8266ffd640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8254004d40 0x7f82680b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f82677fe640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f826809a780 0x7f826809ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f82667fc640 1 -- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f826809a780 msgr2=0x7f826809ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f82667fc640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f826809a780 0x7f826809ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f82667fc640 1 -- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8254004d40 msgr2=0x7f82680b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f82667fc640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8254004d40 0x7f82680b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f82667fc640 1 -- 172.21.15.19:0/1826695182 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82680c9ae0 con 0x7f82681f1dd0 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.570+0000 7f82677fe640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f826809a780 0x7f826809ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.571+0000 7f82667fc640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82681f1dd0 0x7f82680c9290 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f82500072c0 tx=0x7f8250007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.571+0000 7f825ffff640 1 -- 172.21.15.19:0/1826695182 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000f960 con 0x7f82681f1dd0 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.571+0000 7f825ffff640 1 -- 172.21.15.19:0/1826695182 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8250018070 con 0x7f82681f1dd0 2024-04-25T09:56:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.571+0000 7f826d9d5640 1 -- 172.21.15.19:0/1826695182 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82680c9dc0 con 0x7f82681f1dd0 2024-04-25T09:56:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.571+0000 7f825ffff640 1 -- 172.21.15.19:0/1826695182 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000fd60 con 0x7f82681f1dd0 2024-04-25T09:56:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.571+0000 7f826d9d5640 1 -- 172.21.15.19:0/1826695182 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826809c9d0 con 0x7f82681f1dd0 2024-04-25T09:56:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.572+0000 7f826d9d5640 1 -- 172.21.15.19:0/1826695182 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826809cbd0 con 0x7f82681f1dd0 2024-04-25T09:56:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.572+0000 7f825ffff640 1 -- 172.21.15.19:0/1826695182 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f825001d460 con 0x7f82681f1dd0 2024-04-25T09:56:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.573+0000 7f825ffff640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f824806b4d0 0x7f824806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.573+0000 7f825ffff640 1 -- 172.21.15.19:0/1826695182 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f825008a3c0 con 0x7f82681f1dd0 2024-04-25T09:56:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.573+0000 7f8266ffd640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f824806b4d0 0x7f824806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.573+0000 7f8266ffd640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f824806b4d0 0x7f824806d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8268070200 tx=0x7f8260009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.577+0000 7f825ffff640 1 -- 172.21.15.19:0/1826695182 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f826809cbd0 con 0x7f82681f1dd0 2024-04-25T09:56:40.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:40.801+0000 7f826d9d5640 1 -- 172.21.15.19:0/1826695182 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8268073a00 con 0x7f82681f1dd0 2024-04-25T09:56:41.424 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T09:56:41.421+0000 7fabbbda2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:56:41.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.455+0000 7f825ffff640 1 -- 172.21.15.19:0/1826695182 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8250059010 con 0x7f82681f1dd0 2024-04-25T09:56:41.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.458+0000 7f826d9d5640 1 -- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f824806b4d0 msgr2=0x7f824806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.458+0000 7f826d9d5640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f824806b4d0 0x7f824806d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8268070200 tx=0x7f8260009040 comp rx=0 tx=0).stop 2024-04-25T09:56:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.458+0000 7f826d9d5640 1 -- 172.21.15.19:0/1826695182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82681f1dd0 msgr2=0x7f82680c9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.458+0000 7f826d9d5640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82681f1dd0 0x7f82680c9290 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f82500072c0 tx=0x7f8250007780 comp rx=0 tx=0).stop 2024-04-25T09:56:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.458+0000 7f826d9d5640 1 -- 172.21.15.19:0/1826695182 shutdown_connections 2024-04-25T09:56:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.458+0000 7f826d9d5640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f824806b4d0 0x7f824806d990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.458+0000 7f826d9d5640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f826809a780 0x7f826809ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.458+0000 7f826d9d5640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82681f1dd0 0x7f82680c9290 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.458+0000 7f826d9d5640 1 --2- 172.21.15.19:0/1826695182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8254004d40 0x7f82680b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.458+0000 7f826d9d5640 1 -- 172.21.15.19:0/1826695182 >> 172.21.15.19:0/1826695182 conn(0x7f82680b2390 msgr2=0x7f82680d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.459+0000 7f826d9d5640 1 -- 172.21.15.19:0/1826695182 shutdown_connections 2024-04-25T09:56:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.459+0000 7f826d9d5640 1 -- 172.21.15.19:0/1826695182 wait complete. 2024-04-25T09:56:41.475 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:56:41.475 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.579+0000 7feb47789640 1 Processor -- start 2024-04-25T09:56:41.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.580+0000 7feb47789640 1 -- start start 2024-04-25T09:56:41.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.580+0000 7feb47789640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4015ef00 0x7feb400d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.580+0000 7feb47789640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feb400d1ca0 0x7feb400c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb47789640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb400d20e0 con 0x7feb400c7140 2024-04-25T09:56:41.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb47789640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4015f2e0 con 0x7feb400d1ca0 2024-04-25T09:56:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb47789640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb400c7970 con 0x7feb4015ef00 2024-04-25T09:56:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb45525640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4015ef00 0x7feb400d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb44d24640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feb400d1ca0 0x7feb400c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb45d26640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7feb400c7140 0x7feb4015f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:55052/0 (socket says 172.21.15.19:55052) 2024-04-25T09:56:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb45525640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4015ef00 0x7feb400d1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:58114/0 (socket says 172.21.15.19:58114) 2024-04-25T09:56:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb45d26640 1 -- 172.21.15.19:0/3111691494 learned_addr learned my addr 172.21.15.19:0/3111691494 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb44d24640 1 -- 172.21.15.19:0/3111691494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4015ef00 msgr2=0x7feb400d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb44d24640 1 --2- 172.21.15.19:0/3111691494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4015ef00 0x7feb400d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb44d24640 1 -- 172.21.15.19:0/3111691494 >> v1:172.21.15.112:6789/0 conn(0x7feb400c7140 legacy=0x7feb4015f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb44d24640 1 -- 172.21.15.19:0/3111691494 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400c7c50 con 0x7feb400d1ca0 2024-04-25T09:56:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.581+0000 7feb45525640 1 --2- 172.21.15.19:0/3111691494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4015ef00 0x7feb400d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:41.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.582+0000 7feb44d24640 1 --2- 172.21.15.19:0/3111691494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb400d1ca0 0x7feb400c6c00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7feb300097a0 tx=0x7feb3002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f79e12eb776e8ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:41.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.582+0000 7feb37fff640 1 -- 172.21.15.19:0/3111691494 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30037070 con 0x7feb400d1ca0 2024-04-25T09:56:41.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.582+0000 7feb37fff640 1 -- 172.21.15.19:0/3111691494 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb30032440 con 0x7feb400d1ca0 2024-04-25T09:56:41.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.582+0000 7feb37fff640 1 -- 172.21.15.19:0/3111691494 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30032810 con 0x7feb400d1ca0 2024-04-25T09:56:41.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.583+0000 7feb47789640 1 -- 172.21.15.19:0/3111691494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb400d1ca0 msgr2=0x7feb400c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:41.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.583+0000 7feb47789640 1 --2- 172.21.15.19:0/3111691494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb400d1ca0 0x7feb400c6c00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7feb300097a0 tx=0x7feb3002a210 comp rx=0 tx=0).stop 2024-04-25T09:56:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.583+0000 7feb47789640 1 -- 172.21.15.19:0/3111691494 shutdown_connections 2024-04-25T09:56:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.583+0000 7feb47789640 1 --2- 172.21.15.19:0/3111691494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb400d1ca0 0x7feb400c6c00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.583+0000 7feb47789640 1 --2- 172.21.15.19:0/3111691494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4015ef00 0x7feb400d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.583+0000 7feb47789640 1 -- 172.21.15.19:0/3111691494 >> 172.21.15.19:0/3111691494 conn(0x7feb4006f5d0 msgr2=0x7feb4006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.583+0000 7feb47789640 1 -- 172.21.15.19:0/3111691494 shutdown_connections 2024-04-25T09:56:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.583+0000 7feb47789640 1 -- 172.21.15.19:0/3111691494 wait complete. 2024-04-25T09:56:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.584+0000 7feb47789640 1 Processor -- start 2024-04-25T09:56:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.584+0000 7feb47789640 1 -- start start 2024-04-25T09:56:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.585+0000 7feb47789640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb400c7140 0x7feb400b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.585+0000 7feb47789640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb400d1ca0 0x7feb400c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.585+0000 7feb47789640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4015ef00 0x7feb4014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.585+0000 7feb47789640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400c84f0 con 0x7feb4015ef00 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.585+0000 7feb47789640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400c8210 con 0x7feb400c7140 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.585+0000 7feb47789640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb400c7560 con 0x7feb400d1ca0 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.585+0000 7feb45d26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4015ef00 0x7feb4014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.585+0000 7feb45d26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4015ef00 0x7feb4014f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53198/0 (socket says 172.21.15.19:53198) 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.585+0000 7feb45d26640 1 -- 172.21.15.19:0/1590688624 learned_addr learned my addr 172.21.15.19:0/1590688624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.585+0000 7feb44d24640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb400d1ca0 0x7feb400c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.585+0000 7feb45525640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb400c7140 0x7feb400b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.586+0000 7feb45d26640 1 -- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb400d1ca0 msgr2=0x7feb400c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.586+0000 7feb45d26640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb400d1ca0 0x7feb400c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.586+0000 7feb45d26640 1 -- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb400c7140 msgr2=0x7feb400b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.586+0000 7feb45d26640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb400c7140 0x7feb400b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.586+0000 7feb45d26640 1 -- 172.21.15.19:0/1590688624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400c8e30 con 0x7feb4015ef00 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.586+0000 7feb45d26640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4015ef00 0x7feb4014f6d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7feb3800cbc0 tx=0x7feb3800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.586+0000 7feb35ffb640 1 -- 172.21.15.19:0/1590688624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38017070 con 0x7feb4015ef00 2024-04-25T09:56:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.586+0000 7feb35ffb640 1 -- 172.21.15.19:0/1590688624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3800ad40 con 0x7feb4015ef00 2024-04-25T09:56:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.586+0000 7feb47789640 1 -- 172.21.15.19:0/1590688624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb400c90d0 con 0x7feb4015ef00 2024-04-25T09:56:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.586+0000 7feb35ffb640 1 -- 172.21.15.19:0/1590688624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38012600 con 0x7feb4015ef00 2024-04-25T09:56:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.586+0000 7feb47789640 1 -- 172.21.15.19:0/1590688624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb40156e50 con 0x7feb4015ef00 2024-04-25T09:56:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.587+0000 7feb47789640 1 -- 172.21.15.19:0/1590688624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb40073990 con 0x7feb4015ef00 2024-04-25T09:56:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.587+0000 7feb35ffb640 1 -- 172.21.15.19:0/1590688624 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb3801a070 con 0x7feb4015ef00 2024-04-25T09:56:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.588+0000 7feb35ffb640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb2406b4d0 0x7feb2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.588+0000 7feb35ffb640 1 -- 172.21.15.19:0/1590688624 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb3808a000 con 0x7feb4015ef00 2024-04-25T09:56:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.588+0000 7feb45525640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb2406b4d0 0x7feb2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.589+0000 7feb45525640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb2406b4d0 0x7feb2406d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009560 tx=0x7feb3c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.592+0000 7feb35ffb640 1 -- 172.21.15.19:0/1590688624 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb380586a0 con 0x7feb4015ef00 2024-04-25T09:56:41.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.812+0000 7feb47789640 1 -- 172.21.15.19:0/1590688624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb4015e3d0 con 0x7feb4015ef00 2024-04-25T09:56:41.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.812+0000 7feb35ffb640 1 -- 172.21.15.19:0/1590688624 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7feb3805c740 con 0x7feb4015ef00 2024-04-25T09:56:41.816 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:41.816 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3520485752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3520485752},{"type":"v1","addr":"172.21.15.19:6835","nonce":3520485752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5594,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1340729764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1340729764},{"type":"v1","addr":"172.21.15.112:6837","nonce":1340729764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/335673057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":335673057},{"type":"v1","addr":"172.21.15.112:6835","nonce":335673057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:56:38.802500+0000","modified":"2024-04-25T09:56:40.802847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6582":{"gid":6582,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/591570980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":591570980},{"type":"v1","addr":"172.21.15.19:6834","nonce":591570980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:56:41.816 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-04-25T09:56:41.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 -- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb2406b4d0 msgr2=0x7feb2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:41.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb2406b4d0 0x7feb2406d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009560 tx=0x7feb3c009190 comp rx=0 tx=0).stop 2024-04-25T09:56:41.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 -- 172.21.15.19:0/1590688624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4015ef00 msgr2=0x7feb4014f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:41.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4015ef00 0x7feb4014f6d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7feb3800cbc0 tx=0x7feb3800a430 comp rx=0 tx=0).stop 2024-04-25T09:56:41.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 -- 172.21.15.19:0/1590688624 shutdown_connections 2024-04-25T09:56:41.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb2406b4d0 0x7feb2406d990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4015ef00 0x7feb4014f6d0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb400d1ca0 0x7feb400c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 --2- 172.21.15.19:0/1590688624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb400c7140 0x7feb400b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 -- 172.21.15.19:0/1590688624 >> 172.21.15.19:0/1590688624 conn(0x7feb4006f5d0 msgr2=0x7feb400ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:41.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 -- 172.21.15.19:0/1590688624 shutdown_connections 2024-04-25T09:56:41.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.815+0000 7feb47789640 1 -- 172.21.15.19:0/1590688624 wait complete. 2024-04-25T09:56:41.829 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:41.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.926+0000 7f9700a6b640 1 Processor -- start 2024-04-25T09:56:41.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.926+0000 7f9700a6b640 1 -- start start 2024-04-25T09:56:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.927+0000 7f9700a6b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96fc158220 0x7f96fc15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.927+0000 7f9700a6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96fc0afba0 0x7f96fc161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.927+0000 7f9700a6b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96fc15f180 con 0x7f96fc0af720 2024-04-25T09:56:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.927+0000 7f9700a6b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96fc0b3670 con 0x7f96fc158220 2024-04-25T09:56:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.927+0000 7f9700a6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc070a10 con 0x7f96fc0afba0 2024-04-25T09:56:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.927+0000 7f96fa59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96fc158220 0x7f96fc15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.928+0000 7f96fa59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96fc158220 0x7f96fc15ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53210/0 (socket says 172.21.15.19:53210) 2024-04-25T09:56:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.928+0000 7f96fa59c640 1 -- 172.21.15.19:0/1523675294 learned_addr learned my addr 172.21.15.19:0/1523675294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.928+0000 7f96fa59c640 1 -- 172.21.15.19:0/1523675294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96fc0afba0 msgr2=0x7f96fc161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.928+0000 7f96fa59c640 1 --2- 172.21.15.19:0/1523675294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96fc0afba0 0x7f96fc161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.928+0000 7f96fa59c640 1 -- 172.21.15.19:0/1523675294 >> v1:172.21.15.112:6789/0 conn(0x7f96fc0af720 legacy=0x7f96fc1617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.928+0000 7f96fa59c640 1 -- 172.21.15.19:0/1523675294 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc06fe00 con 0x7f96fc158220 2024-04-25T09:56:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.928+0000 7f96fa59c640 1 --2- 172.21.15.19:0/1523675294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96fc158220 0x7f96fc15ec40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009870 tx=0x7f96f402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e55503f0cf1862 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.929+0000 7f96f959a640 1 -- 172.21.15.19:0/1523675294 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4037070 con 0x7f96fc158220 2024-04-25T09:56:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.929+0000 7f96f959a640 1 -- 172.21.15.19:0/1523675294 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f4032440 con 0x7f96fc158220 2024-04-25T09:56:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.929+0000 7f96f959a640 1 -- 172.21.15.19:0/1523675294 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4032810 con 0x7f96fc158220 2024-04-25T09:56:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.929+0000 7f9700a6b640 1 -- 172.21.15.19:0/1523675294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96fc158220 msgr2=0x7f96fc15ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.929+0000 7f9700a6b640 1 --2- 172.21.15.19:0/1523675294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96fc158220 0x7f96fc15ec40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009870 tx=0x7f96f402a210 comp rx=0 tx=0).stop 2024-04-25T09:56:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.929+0000 7f9700a6b640 1 -- 172.21.15.19:0/1523675294 shutdown_connections 2024-04-25T09:56:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.929+0000 7f9700a6b640 1 --2- 172.21.15.19:0/1523675294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96fc0afba0 0x7f96fc161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.929+0000 7f9700a6b640 1 --2- 172.21.15.19:0/1523675294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96fc158220 0x7f96fc15ec40 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.929+0000 7f9700a6b640 1 -- 172.21.15.19:0/1523675294 >> 172.21.15.19:0/1523675294 conn(0x7f96fc0b2370 msgr2=0x7f96fc0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:41.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.929+0000 7f9700a6b640 1 -- 172.21.15.19:0/1523675294 shutdown_connections 2024-04-25T09:56:41.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.929+0000 7f9700a6b640 1 -- 172.21.15.19:0/1523675294 wait complete. 2024-04-25T09:56:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.931+0000 7f9700a6b640 1 Processor -- start 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.931+0000 7f9700a6b640 1 -- start start 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.931+0000 7f9700a6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96fc0af720 0x7f96fc1f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.931+0000 7f9700a6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96fc0afba0 0x7f96fc1f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.931+0000 7f9700a6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96fc158220 0x7f96fc0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.931+0000 7f9700a6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0706a0 con 0x7f96fc0afba0 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.931+0000 7f9700a6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0703c0 con 0x7f96fc0af720 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.931+0000 7f9700a6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0b0010 con 0x7f96fc158220 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96fc0afba0 0x7f96fc1f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96fc0afba0 0x7f96fc1f2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53226/0 (socket says 172.21.15.19:53226) 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f9d9b640 1 -- 172.21.15.19:0/2478721681 learned_addr learned my addr 172.21.15.19:0/2478721681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96fa59c640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96fc0af720 0x7f96fc1f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96fad9d640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96fc158220 0x7f96fc0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f9d9b640 1 -- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96fc158220 msgr2=0x7f96fc0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f9d9b640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96fc158220 0x7f96fc0b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f9d9b640 1 -- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96fc0af720 msgr2=0x7f96fc1f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f9d9b640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96fc0af720 0x7f96fc1f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f9d9b640 1 -- 172.21.15.19:0/2478721681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc159860 con 0x7f96fc0afba0 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96fa59c640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96fc0af720 0x7f96fc1f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f9d9b640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96fc0afba0 0x7f96fc1f2e90 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c410 tx=0x7f96e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f2ffd640 1 -- 172.21.15.19:0/2478721681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4017070 con 0x7f96fc0afba0 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f2ffd640 1 -- 172.21.15.19:0/2478721681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e40035f0 con 0x7f96fc0afba0 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f96f2ffd640 1 -- 172.21.15.19:0/2478721681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e40039e0 con 0x7f96fc0afba0 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f9700a6b640 1 -- 172.21.15.19:0/2478721681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96fc159b40 con 0x7f96fc0afba0 2024-04-25T09:56:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.932+0000 7f9700a6b640 1 -- 172.21.15.19:0/2478721681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96fc15a100 con 0x7f96fc0afba0 2024-04-25T09:56:41.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.933+0000 7f9700a6b640 1 -- 172.21.15.19:0/2478721681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96fc073990 con 0x7f96fc0afba0 2024-04-25T09:56:41.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.934+0000 7f96f2ffd640 1 -- 172.21.15.19:0/2478721681 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96e401a070 con 0x7f96fc0afba0 2024-04-25T09:56:41.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.934+0000 7f96f2ffd640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96dc06b4d0 0x7f96dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:41.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.934+0000 7f96f2ffd640 1 -- 172.21.15.19:0/2478721681 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f96e4089800 con 0x7f96fc0afba0 2024-04-25T09:56:41.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.934+0000 7f96fa59c640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96dc06b4d0 0x7f96dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:41.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.935+0000 7f96fa59c640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96dc06b4d0 0x7f96dc06d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f96f4034000 tx=0x7f96f402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:41.938+0000 7f96f2ffd640 1 -- 172.21.15.19:0/2478721681 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96e40584f0 con 0x7f96fc0afba0 2024-04-25T09:56:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.118+0000 7f9700a6b640 1 -- 172.21.15.19:0/2478721681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96fc06d330 con 0x7f96fc0afba0 2024-04-25T09:56:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.119+0000 7f96f2ffd640 1 -- 172.21.15.19:0/2478721681 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f96e4057df0 con 0x7f96fc0afba0 2024-04-25T09:56:42.123 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:42.123 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":64,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:56:40.436884+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:56:37.087339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:56:37.775183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 -- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96dc06b4d0 msgr2=0x7f96dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96dc06b4d0 0x7f96dc06d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f96f4034000 tx=0x7f96f402e000 comp rx=0 tx=0).stop 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 -- 172.21.15.19:0/2478721681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96fc0afba0 msgr2=0x7f96fc1f2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96fc0afba0 0x7f96fc1f2e90 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c410 tx=0x7f96e400c8d0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 -- 172.21.15.19:0/2478721681 shutdown_connections 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96dc06b4d0 0x7f96dc06d990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96fc158220 0x7f96fc0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96fc0afba0 0x7f96fc1f2e90 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 --2- 172.21.15.19:0/2478721681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96fc0af720 0x7f96fc1f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 -- 172.21.15.19:0/2478721681 >> 172.21.15.19:0/2478721681 conn(0x7f96fc0b2370 msgr2=0x7f96fc0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 -- 172.21.15.19:0/2478721681 shutdown_connections 2024-04-25T09:56:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.121+0000 7f9700a6b640 1 -- 172.21.15.19:0/2478721681 wait complete. 2024-04-25T09:56:42.136 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:56:42.136 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:56:42.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.234+0000 7f3149f29640 1 Processor -- start 2024-04-25T09:56:42.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.234+0000 7f3149f29640 1 -- start start 2024-04-25T09:56:42.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3149f29640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f314415dbe0 0x7f31440d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3149f29640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f31440b2cb0 0x7f31440d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3149f29640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31440b97a0 con 0x7f31440b92a0 2024-04-25T09:56:42.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3149f29640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314415dfc0 con 0x7f31440b2cb0 2024-04-25T09:56:42.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3149f29640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31440d1a00 con 0x7f314415dbe0 2024-04-25T09:56:42.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3142ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f31440b2cb0 0x7f31440d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3143fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f31440b92a0 0x7f31440b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36604/0 (socket says 172.21.15.19:36604) 2024-04-25T09:56:42.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3142ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f31440b2cb0 0x7f31440d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53238/0 (socket says 172.21.15.19:53238) 2024-04-25T09:56:42.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3143fff640 1 -- 172.21.15.19:0/69603897 learned_addr learned my addr 172.21.15.19:0/69603897 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:42.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f31437fe640 1 --2- 172.21.15.19:0/69603897 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f314415dbe0 0x7f31440d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3142ffd640 1 -- 172.21.15.19:0/69603897 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f314415dbe0 msgr2=0x7f31440d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:42.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3142ffd640 1 --2- 172.21.15.19:0/69603897 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f314415dbe0 0x7f31440d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3142ffd640 1 -- 172.21.15.19:0/69603897 >> v1:172.21.15.19:6789/0 conn(0x7f31440b92a0 legacy=0x7f31440b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f3142ffd640 1 -- 172.21.15.19:0/69603897 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31440d1c50 con 0x7f31440b2cb0 2024-04-25T09:56:42.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.235+0000 7f31427fc640 1 -- 172.21.15.19:0/69603897 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358266310 0 0) 0x7f31440b97a0 con 0x7f31440b92a0 2024-04-25T09:56:42.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.236+0000 7f3142ffd640 1 --2- 172.21.15.19:0/69603897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31440b2cb0 0x7f31440d0e50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f31340097a0 tx=0x7f313402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c789cb3830d5c35a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:42.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.236+0000 7f31427fc640 1 -- 172.21.15.19:0/69603897 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134037070 con 0x7f31440b2cb0 2024-04-25T09:56:42.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.236+0000 7f31427fc640 1 -- 172.21.15.19:0/69603897 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3134032440 con 0x7f31440b2cb0 2024-04-25T09:56:42.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.236+0000 7f31427fc640 1 -- 172.21.15.19:0/69603897 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134032810 con 0x7f31440b2cb0 2024-04-25T09:56:42.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.236+0000 7f3149f29640 1 -- 172.21.15.19:0/69603897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31440b2cb0 msgr2=0x7f31440d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.236+0000 7f3149f29640 1 --2- 172.21.15.19:0/69603897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31440b2cb0 0x7f31440d0e50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f31340097a0 tx=0x7f313402a180 comp rx=0 tx=0).stop 2024-04-25T09:56:42.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.237+0000 7f3149f29640 1 -- 172.21.15.19:0/69603897 shutdown_connections 2024-04-25T09:56:42.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.237+0000 7f3149f29640 1 --2- 172.21.15.19:0/69603897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31440b2cb0 0x7f31440d0e50 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.237+0000 7f3149f29640 1 --2- 172.21.15.19:0/69603897 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f314415dbe0 0x7f31440d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.237+0000 7f3149f29640 1 -- 172.21.15.19:0/69603897 >> 172.21.15.19:0/69603897 conn(0x7f3144061a20 msgr2=0x7f3144061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:42.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.237+0000 7f3149f29640 1 -- 172.21.15.19:0/69603897 shutdown_connections 2024-04-25T09:56:42.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.237+0000 7f3149f29640 1 -- 172.21.15.19:0/69603897 wait complete. 2024-04-25T09:56:42.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.238+0000 7f3149f29640 1 Processor -- start 2024-04-25T09:56:42.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.238+0000 7f3149f29640 1 -- start start 2024-04-25T09:56:42.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3149f29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31440b2cb0 0x7f314415f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3149f29640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31440b92a0 0x7f31440cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3149f29640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f314415dbe0 0x7f31440b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3149f29640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31440b6ab0 con 0x7f31440b92a0 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3149f29640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31440d2210 con 0x7f31440b2cb0 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3149f29640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31440b6c30 con 0x7f314415dbe0 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3142ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31440b92a0 0x7f31440cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3142ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31440b92a0 0x7f31440cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53248/0 (socket says 172.21.15.19:53248) 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3142ffd640 1 -- 172.21.15.19:0/420575482 learned_addr learned my addr 172.21.15.19:0/420575482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3143fff640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f314415dbe0 0x7f31440b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f31437fe640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31440b2cb0 0x7f314415f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3142ffd640 1 -- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f314415dbe0 msgr2=0x7f31440b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3142ffd640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f314415dbe0 0x7f31440b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3142ffd640 1 -- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31440b2cb0 msgr2=0x7f314415f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3142ffd640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31440b2cb0 0x7f314415f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3142ffd640 1 -- 172.21.15.19:0/420575482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31440cd110 con 0x7f31440b92a0 2024-04-25T09:56:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f31437fe640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31440b2cb0 0x7f314415f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f3142ffd640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31440b92a0 0x7f31440cc830 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3134000c00 tx=0x7f31340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f312bfff640 1 -- 172.21.15.19:0/420575482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134037040 con 0x7f31440b92a0 2024-04-25T09:56:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f312bfff640 1 -- 172.21.15.19:0/420575482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3134042070 con 0x7f31440b92a0 2024-04-25T09:56:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.239+0000 7f312bfff640 1 -- 172.21.15.19:0/420575482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134030bb0 con 0x7f31440b92a0 2024-04-25T09:56:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.240+0000 7f3149f29640 1 -- 172.21.15.19:0/420575482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31440b41e0 con 0x7f31440b92a0 2024-04-25T09:56:42.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.240+0000 7f3149f29640 1 -- 172.21.15.19:0/420575482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31440b47a0 con 0x7f31440b92a0 2024-04-25T09:56:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.241+0000 7f3149f29640 1 -- 172.21.15.19:0/420575482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3144073990 con 0x7f31440b92a0 2024-04-25T09:56:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.241+0000 7f312bfff640 1 -- 172.21.15.19:0/420575482 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3134047460 con 0x7f31440b92a0 2024-04-25T09:56:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.241+0000 7f312bfff640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313006b4d0 0x7f313006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.242+0000 7f312bfff640 1 -- 172.21.15.19:0/420575482 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f31340aafa0 con 0x7f31440b92a0 2024-04-25T09:56:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.242+0000 7f31437fe640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313006b4d0 0x7f313006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.242+0000 7f31437fe640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313006b4d0 0x7f313006d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3138006fd0 tx=0x7f3138008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:42.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.246+0000 7f312bfff640 1 -- 172.21.15.19:0/420575482 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3134032e20 con 0x7f31440b92a0 2024-04-25T09:56:42.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.423+0000 7f3149f29640 1 -- 172.21.15.19:0/420575482 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31440b7fc0 con 0x7f313006b4d0 2024-04-25T09:56:42.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.429+0000 7f312bfff640 1 -- 172.21.15.19:0/420575482 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246700 (secure 0 0 0) 0x7f31440b7fc0 con 0x7f313006b4d0 2024-04-25T09:56:42.435 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:42.437 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 -- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313006b4d0 msgr2=0x7f313006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313006b4d0 0x7f313006d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3138006fd0 tx=0x7f3138008040 comp rx=0 tx=0).stop 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 -- 172.21.15.19:0/420575482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31440b92a0 msgr2=0x7f31440cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31440b92a0 0x7f31440cc830 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3134000c00 tx=0x7f31340046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 -- 172.21.15.19:0/420575482 shutdown_connections 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313006b4d0 0x7f313006d990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f314415dbe0 0x7f31440b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31440b92a0 0x7f31440cc830 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 --2- 172.21.15.19:0/420575482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31440b2cb0 0x7f314415f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 -- 172.21.15.19:0/420575482 >> 172.21.15.19:0/420575482 conn(0x7f3144061a20 msgr2=0x7f314415b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 -- 172.21.15.19:0/420575482 shutdown_connections 2024-04-25T09:56:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.432+0000 7f3149f29640 1 -- 172.21.15.19:0/420575482 wait complete. 2024-04-25T09:56:42.446 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-04-25T09:56:40.961147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289720,"kb_used_data":6980,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685000,"statfs":{"total":773094113280,"available":772797440000,"internally_reserved":0,"allocated":7147520,"data_stored":3857879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000948"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437683+0000","last_change":"2024-04-25T09:56:38.437683+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437683+0000","last_undegraded":"2024-04-25T09:56:38.437683+0000","last_fullsized":"2024-04-25T09:56:38.437683+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436642+0000","last_change":"2024-04-25T09:56:38.436642+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436642+0000","last_undegraded":"2024-04-25T09:56:38.436642+0000","last_fullsized":"2024-04-25T09:56:38.436642+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433788+0000","last_change":"2024-04-25T09:56:38.433788+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433788+0000","last_undegraded":"2024-04-25T09:56:38.433788+0000","last_fullsized":"2024-04-25T09:56:38.433788+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431818+0000","last_change":"2024-04-25T09:56:38.431818+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431818+0000","last_undegraded":"2024-04-25T09:56:38.431818+0000","last_fullsized":"2024-04-25T09:56:38.431818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437599+0000","last_change":"2024-04-25T09:56:38.437599+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437599+0000","last_undegraded":"2024-04-25T09:56:38.437599+0000","last_fullsized":"2024-04-25T09:56:38.437599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436685+0000","last_change":"2024-04-25T09:56:38.436685+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436685+0000","last_undegraded":"2024-04-25T09:56:38.436685+0000","last_fullsized":"2024-04-25T09:56:38.436685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439909+0000","last_change":"2024-04-25T09:56:38.439909+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439909+0000","last_undegraded":"2024-04-25T09:56:38.439909+0000","last_fullsized":"2024-04-25T09:56:38.439909+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438373+0000","last_change":"2024-04-25T09:56:38.438373+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438373+0000","last_undegraded":"2024-04-25T09:56:38.438373+0000","last_fullsized":"2024-04-25T09:56:38.438373+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433700+0000","last_change":"2024-04-25T09:56:38.433700+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433700+0000","last_undegraded":"2024-04-25T09:56:38.433700+0000","last_fullsized":"2024-04-25T09:56:38.433700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435532+0000","last_change":"2024-04-25T09:56:38.435532+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435532+0000","last_undegraded":"2024-04-25T09:56:38.435532+0000","last_fullsized":"2024-04-25T09:56:38.435532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435197+0000","last_change":"2024-04-25T09:56:38.435197+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435197+0000","last_undegraded":"2024-04-25T09:56:38.435197+0000","last_fullsized":"2024-04-25T09:56:38.435197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434689+0000","last_change":"2024-04-25T09:56:38.434689+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434689+0000","last_undegraded":"2024-04-25T09:56:38.434689+0000","last_fullsized":"2024-04-25T09:56:38.434689+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436051+0000","last_change":"2024-04-25T09:56:38.436051+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436051+0000","last_undegraded":"2024-04-25T09:56:38.436051+0000","last_fullsized":"2024-04-25T09:56:38.436051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433377+0000","last_change":"2024-04-25T09:56:38.433377+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433377+0000","last_undegraded":"2024-04-25T09:56:38.433377+0000","last_fullsized":"2024-04-25T09:56:38.433377+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433505+0000","last_change":"2024-04-25T09:56:38.433505+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433505+0000","last_undegraded":"2024-04-25T09:56:38.433505+0000","last_fullsized":"2024-04-25T09:56:38.433505+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439675+0000","last_change":"2024-04-25T09:56:38.439675+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439675+0000","last_undegraded":"2024-04-25T09:56:38.439675+0000","last_fullsized":"2024-04-25T09:56:38.439675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434662+0000","last_change":"2024-04-25T09:56:38.434662+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434662+0000","last_undegraded":"2024-04-25T09:56:38.434662+0000","last_fullsized":"2024-04-25T09:56:38.434662+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435818+0000","last_change":"2024-04-25T09:56:38.435818+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435818+0000","last_undegraded":"2024-04-25T09:56:38.435818+0000","last_fullsized":"2024-04-25T09:56:38.435818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437438+0000","last_change":"2024-04-25T09:56:38.437438+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437438+0000","last_undegraded":"2024-04-25T09:56:38.437438+0000","last_fullsized":"2024-04-25T09:56:38.437438+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434623+0000","last_change":"2024-04-25T09:56:38.434623+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434623+0000","last_undegraded":"2024-04-25T09:56:38.434623+0000","last_fullsized":"2024-04-25T09:56:38.434623+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434027+0000","last_change":"2024-04-25T09:56:38.434027+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434027+0000","last_undegraded":"2024-04-25T09:56:38.434027+0000","last_fullsized":"2024-04-25T09:56:38.434027+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436029+0000","last_change":"2024-04-25T09:56:38.436029+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436029+0000","last_undegraded":"2024-04-25T09:56:38.436029+0000","last_fullsized":"2024-04-25T09:56:38.436029+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434596+0000","last_change":"2024-04-25T09:56:38.434596+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434596+0000","last_undegraded":"2024-04-25T09:56:38.434596+0000","last_fullsized":"2024-04-25T09:56:38.434596+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433635+0000","last_change":"2024-04-25T09:56:38.433635+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433635+0000","last_undegraded":"2024-04-25T09:56:38.433635+0000","last_fullsized":"2024-04-25T09:56:38.433635+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437925+0000","last_change":"2024-04-25T09:56:38.437925+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437925+0000","last_undegraded":"2024-04-25T09:56:38.437925+0000","last_fullsized":"2024-04-25T09:56:38.437925+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434898+0000","last_change":"2024-04-25T09:56:38.434898+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434898+0000","last_undegraded":"2024-04-25T09:56:38.434898+0000","last_fullsized":"2024-04-25T09:56:38.434898+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434806+0000","last_change":"2024-04-25T09:56:38.434806+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434806+0000","last_undegraded":"2024-04-25T09:56:38.434806+0000","last_fullsized":"2024-04-25T09:56:38.434806+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434169+0000","last_change":"2024-04-25T09:56:38.434169+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434169+0000","last_undegraded":"2024-04-25T09:56:38.434169+0000","last_fullsized":"2024-04-25T09:56:38.434169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433478+0000","last_change":"2024-04-25T09:56:38.433478+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433478+0000","last_undegraded":"2024-04-25T09:56:38.433478+0000","last_fullsized":"2024-04-25T09:56:38.433478+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435681+0000","last_change":"2024-04-25T09:56:38.435681+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435681+0000","last_undegraded":"2024-04-25T09:56:38.435681+0000","last_fullsized":"2024-04-25T09:56:38.435681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431678+0000","last_change":"2024-04-25T09:56:38.431678+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431678+0000","last_undegraded":"2024-04-25T09:56:38.431678+0000","last_fullsized":"2024-04-25T09:56:38.431678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434572+0000","last_change":"2024-04-25T09:56:38.434572+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434572+0000","last_undegraded":"2024-04-25T09:56:38.434572+0000","last_fullsized":"2024-04-25T09:56:38.434572+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434920+0000","last_change":"2024-04-25T09:56:38.434920+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434920+0000","last_undegraded":"2024-04-25T09:56:38.434920+0000","last_fullsized":"2024-04-25T09:56:38.434920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.451222+0000","last_change":"2024-04-25T09:56:38.448802+0000","last_active":"2024-04-25T09:56:38.451222+0000","last_peered":"2024-04-25T09:56:38.451222+0000","last_clean":"2024-04-25T09:56:38.451222+0000","last_became_active":"2024-04-25T09:56:38.447596+0000","last_became_peered":"2024-04-25T09:56:38.447596+0000","last_unstale":"2024-04-25T09:56:38.451222+0000","last_undegraded":"2024-04-25T09:56:38.451222+0000","last_fullsized":"2024-04-25T09:56:38.451222+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:56.119011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.432113+0000","last_change":"2024-04-25T09:56:38.432113+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.432113+0000","last_undegraded":"2024-04-25T09:56:38.432113+0000","last_fullsized":"2024-04-25T09:56:38.432113+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450513+0000","last_change":"2024-04-25T09:56:38.446203+0000","last_active":"2024-04-25T09:56:38.450513+0000","last_peered":"2024-04-25T09:56:38.450513+0000","last_clean":"2024-04-25T09:56:38.450513+0000","last_became_active":"2024-04-25T09:56:38.445931+0000","last_became_peered":"2024-04-25T09:56:38.445931+0000","last_unstale":"2024-04-25T09:56:38.450513+0000","last_undegraded":"2024-04-25T09:56:38.450513+0000","last_fullsized":"2024-04-25T09:56:38.450513+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:48.759756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434485+0000","last_change":"2024-04-25T09:56:38.434485+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434485+0000","last_undegraded":"2024-04-25T09:56:38.434485+0000","last_fullsized":"2024-04-25T09:56:38.434485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449712+0000","last_change":"2024-04-25T09:56:38.442763+0000","last_active":"2024-04-25T09:56:38.449712+0000","last_peered":"2024-04-25T09:56:38.449712+0000","last_clean":"2024-04-25T09:56:38.449712+0000","last_became_active":"2024-04-25T09:56:38.442494+0000","last_became_peered":"2024-04-25T09:56:38.442494+0000","last_unstale":"2024-04-25T09:56:38.449712+0000","last_undegraded":"2024-04-25T09:56:38.449712+0000","last_fullsized":"2024-04-25T09:56:38.449712+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.557978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434490+0000","last_change":"2024-04-25T09:56:38.434490+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434490+0000","last_undegraded":"2024-04-25T09:56:38.434490+0000","last_fullsized":"2024-04-25T09:56:38.434490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443747+0000","last_change":"2024-04-25T09:56:38.441504+0000","last_active":"2024-04-25T09:56:38.443747+0000","last_peered":"2024-04-25T09:56:38.443747+0000","last_clean":"2024-04-25T09:56:38.443747+0000","last_became_active":"2024-04-25T09:56:38.441178+0000","last_became_peered":"2024-04-25T09:56:38.441178+0000","last_unstale":"2024-04-25T09:56:38.443747+0000","last_undegraded":"2024-04-25T09:56:38.443747+0000","last_fullsized":"2024-04-25T09:56:38.443747+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:49.974254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439169+0000","last_change":"2024-04-25T09:56:38.439169+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439169+0000","last_undegraded":"2024-04-25T09:56:38.439169+0000","last_fullsized":"2024-04-25T09:56:38.439169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443433+0000","last_change":"2024-04-25T09:56:38.442701+0000","last_active":"2024-04-25T09:56:38.443433+0000","last_peered":"2024-04-25T09:56:38.443433+0000","last_clean":"2024-04-25T09:56:38.443433+0000","last_became_active":"2024-04-25T09:56:38.442433+0000","last_became_peered":"2024-04-25T09:56:38.442433+0000","last_unstale":"2024-04-25T09:56:38.443433+0000","last_undegraded":"2024-04-25T09:56:38.443433+0000","last_fullsized":"2024-04-25T09:56:38.443433+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:55.484379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434138+0000","last_change":"2024-04-25T09:56:38.434138+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434138+0000","last_undegraded":"2024-04-25T09:56:38.434138+0000","last_fullsized":"2024-04-25T09:56:38.434138+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448317+0000","last_change":"2024-04-25T09:56:38.446402+0000","last_active":"2024-04-25T09:56:38.448317+0000","last_peered":"2024-04-25T09:56:38.448317+0000","last_clean":"2024-04-25T09:56:38.448317+0000","last_became_active":"2024-04-25T09:56:38.446088+0000","last_became_peered":"2024-04-25T09:56:38.446088+0000","last_unstale":"2024-04-25T09:56:38.448317+0000","last_undegraded":"2024-04-25T09:56:38.448317+0000","last_fullsized":"2024-04-25T09:56:38.448317+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.933751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438728+0000","last_change":"2024-04-25T09:56:38.438728+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438728+0000","last_undegraded":"2024-04-25T09:56:38.438728+0000","last_fullsized":"2024-04-25T09:56:38.438728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.447423+0000","last_change":"2024-04-25T09:56:38.445015+0000","last_active":"2024-04-25T09:56:38.447423+0000","last_peered":"2024-04-25T09:56:38.447423+0000","last_clean":"2024-04-25T09:56:38.447423+0000","last_became_active":"2024-04-25T09:56:38.444713+0000","last_became_peered":"2024-04-25T09:56:38.444713+0000","last_unstale":"2024-04-25T09:56:38.447423+0000","last_undegraded":"2024-04-25T09:56:38.447423+0000","last_fullsized":"2024-04-25T09:56:38.447423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435118+0000","last_change":"2024-04-25T09:56:38.435118+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435118+0000","last_undegraded":"2024-04-25T09:56:38.435118+0000","last_fullsized":"2024-04-25T09:56:38.435118+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448373+0000","last_change":"2024-04-25T09:56:38.446581+0000","last_active":"2024-04-25T09:56:38.448373+0000","last_peered":"2024-04-25T09:56:38.448373+0000","last_clean":"2024-04-25T09:56:38.448373+0000","last_became_active":"2024-04-25T09:56:38.444721+0000","last_became_peered":"2024-04-25T09:56:38.444721+0000","last_unstale":"2024-04-25T09:56:38.448373+0000","last_undegraded":"2024-04-25T09:56:38.448373+0000","last_fullsized":"2024-04-25T09:56:38.448373+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:40.119587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433629+0000","last_change":"2024-04-25T09:56:38.433629+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433629+0000","last_undegraded":"2024-04-25T09:56:38.433629+0000","last_fullsized":"2024-04-25T09:56:38.433629+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454111+0000","last_change":"2024-04-25T09:56:38.452304+0000","last_active":"2024-04-25T09:56:38.454111+0000","last_peered":"2024-04-25T09:56:38.454111+0000","last_clean":"2024-04-25T09:56:38.454111+0000","last_became_active":"2024-04-25T09:56:38.452001+0000","last_became_peered":"2024-04-25T09:56:38.452001+0000","last_unstale":"2024-04-25T09:56:38.454111+0000","last_undegraded":"2024-04-25T09:56:38.454111+0000","last_fullsized":"2024-04-25T09:56:38.454111+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:07.043227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437792+0000","last_change":"2024-04-25T09:56:38.437792+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437792+0000","last_undegraded":"2024-04-25T09:56:38.437792+0000","last_fullsized":"2024-04-25T09:56:38.437792+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452779+0000","last_change":"2024-04-25T09:56:38.451554+0000","last_active":"2024-04-25T09:56:38.452779+0000","last_peered":"2024-04-25T09:56:38.452779+0000","last_clean":"2024-04-25T09:56:38.452779+0000","last_became_active":"2024-04-25T09:56:38.451253+0000","last_became_peered":"2024-04-25T09:56:38.451253+0000","last_unstale":"2024-04-25T09:56:38.452779+0000","last_undegraded":"2024-04-25T09:56:38.452779+0000","last_fullsized":"2024-04-25T09:56:38.452779+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:44.723554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438457+0000","last_change":"2024-04-25T09:56:38.438457+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438457+0000","last_undegraded":"2024-04-25T09:56:38.438457+0000","last_fullsized":"2024-04-25T09:56:38.438457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454400+0000","last_change":"2024-04-25T09:56:38.452908+0000","last_active":"2024-04-25T09:56:38.454400+0000","last_peered":"2024-04-25T09:56:38.454400+0000","last_clean":"2024-04-25T09:56:38.454400+0000","last_became_active":"2024-04-25T09:56:38.452672+0000","last_became_peered":"2024-04-25T09:56:38.452672+0000","last_unstale":"2024-04-25T09:56:38.454400+0000","last_undegraded":"2024-04-25T09:56:38.454400+0000","last_fullsized":"2024-04-25T09:56:38.454400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.108179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431918+0000","last_change":"2024-04-25T09:56:38.431918+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431918+0000","last_undegraded":"2024-04-25T09:56:38.431918+0000","last_fullsized":"2024-04-25T09:56:38.431918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450753+0000","last_change":"2024-04-25T09:56:38.441289+0000","last_active":"2024-04-25T09:56:38.450753+0000","last_peered":"2024-04-25T09:56:38.450753+0000","last_clean":"2024-04-25T09:56:38.450753+0000","last_became_active":"2024-04-25T09:56:38.440974+0000","last_became_peered":"2024-04-25T09:56:38.440974+0000","last_unstale":"2024-04-25T09:56:38.450753+0000","last_undegraded":"2024-04-25T09:56:38.450753+0000","last_fullsized":"2024-04-25T09:56:38.450753+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:06.200902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431725+0000","last_change":"2024-04-25T09:56:38.431725+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431725+0000","last_undegraded":"2024-04-25T09:56:38.431725+0000","last_fullsized":"2024-04-25T09:56:38.431725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.444935+0000","last_change":"2024-04-25T09:56:38.444152+0000","last_active":"2024-04-25T09:56:38.444935+0000","last_peered":"2024-04-25T09:56:38.444935+0000","last_clean":"2024-04-25T09:56:38.444935+0000","last_became_active":"2024-04-25T09:56:38.443891+0000","last_became_peered":"2024-04-25T09:56:38.443891+0000","last_unstale":"2024-04-25T09:56:38.444935+0000","last_undegraded":"2024-04-25T09:56:38.444935+0000","last_fullsized":"2024-04-25T09:56:38.444935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.635113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436850+0000","last_change":"2024-04-25T09:56:38.436850+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436850+0000","last_undegraded":"2024-04-25T09:56:38.436850+0000","last_fullsized":"2024-04-25T09:56:38.436850+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454025+0000","last_change":"2024-04-25T09:56:38.452182+0000","last_active":"2024-04-25T09:56:38.454025+0000","last_peered":"2024-04-25T09:56:38.454025+0000","last_clean":"2024-04-25T09:56:38.454025+0000","last_became_active":"2024-04-25T09:56:38.451492+0000","last_became_peered":"2024-04-25T09:56:38.451492+0000","last_unstale":"2024-04-25T09:56:38.454025+0000","last_undegraded":"2024-04-25T09:56:38.454025+0000","last_fullsized":"2024-04-25T09:56:38.454025+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:33.806055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438322+0000","last_change":"2024-04-25T09:56:38.438322+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438322+0000","last_undegraded":"2024-04-25T09:56:38.438322+0000","last_fullsized":"2024-04-25T09:56:38.438322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452814+0000","last_change":"2024-04-25T09:56:38.449599+0000","last_active":"2024-04-25T09:56:38.452814+0000","last_peered":"2024-04-25T09:56:38.452814+0000","last_clean":"2024-04-25T09:56:38.452814+0000","last_became_active":"2024-04-25T09:56:38.449302+0000","last_became_peered":"2024-04-25T09:56:38.449302+0000","last_unstale":"2024-04-25T09:56:38.452814+0000","last_undegraded":"2024-04-25T09:56:38.452814+0000","last_fullsized":"2024-04-25T09:56:38.452814+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.404639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433556+0000","last_change":"2024-04-25T09:56:38.433556+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433556+0000","last_undegraded":"2024-04-25T09:56:38.433556+0000","last_fullsized":"2024-04-25T09:56:38.433556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448561+0000","last_change":"2024-04-25T09:56:38.447935+0000","last_active":"2024-04-25T09:56:38.448561+0000","last_peered":"2024-04-25T09:56:38.448561+0000","last_clean":"2024-04-25T09:56:38.448561+0000","last_became_active":"2024-04-25T09:56:38.447648+0000","last_became_peered":"2024-04-25T09:56:38.447648+0000","last_unstale":"2024-04-25T09:56:38.448561+0000","last_undegraded":"2024-04-25T09:56:38.448561+0000","last_fullsized":"2024-04-25T09:56:38.448561+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:32.556063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433134+0000","last_change":"2024-04-25T09:56:38.433134+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433134+0000","last_undegraded":"2024-04-25T09:56:38.433134+0000","last_fullsized":"2024-04-25T09:56:38.433134+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452679+0000","last_change":"2024-04-25T09:56:38.451735+0000","last_active":"2024-04-25T09:56:38.452679+0000","last_peered":"2024-04-25T09:56:38.452679+0000","last_clean":"2024-04-25T09:56:38.452679+0000","last_became_active":"2024-04-25T09:56:38.451108+0000","last_became_peered":"2024-04-25T09:56:38.451108+0000","last_unstale":"2024-04-25T09:56:38.452679+0000","last_undegraded":"2024-04-25T09:56:38.452679+0000","last_fullsized":"2024-04-25T09:56:38.452679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:34.259055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436880+0000","last_change":"2024-04-25T09:56:38.436880+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436880+0000","last_undegraded":"2024-04-25T09:56:38.436880+0000","last_fullsized":"2024-04-25T09:56:38.436880+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452467+0000","last_change":"2024-04-25T09:56:38.449467+0000","last_active":"2024-04-25T09:56:38.452467+0000","last_peered":"2024-04-25T09:56:38.452467+0000","last_clean":"2024-04-25T09:56:38.452467+0000","last_became_active":"2024-04-25T09:56:38.449110+0000","last_became_peered":"2024-04-25T09:56:38.449110+0000","last_unstale":"2024-04-25T09:56:38.452467+0000","last_undegraded":"2024-04-25T09:56:38.452467+0000","last_fullsized":"2024-04-25T09:56:38.452467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:34.259055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435023+0000","last_change":"2024-04-25T09:56:38.435023+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435023+0000","last_undegraded":"2024-04-25T09:56:38.435023+0000","last_fullsized":"2024-04-25T09:56:38.435023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454387+0000","last_change":"2024-04-25T09:56:38.452721+0000","last_active":"2024-04-25T09:56:38.454387+0000","last_peered":"2024-04-25T09:56:38.454387+0000","last_clean":"2024-04-25T09:56:38.454387+0000","last_became_active":"2024-04-25T09:56:38.452370+0000","last_became_peered":"2024-04-25T09:56:38.452370+0000","last_unstale":"2024-04-25T09:56:38.454387+0000","last_undegraded":"2024-04-25T09:56:38.454387+0000","last_fullsized":"2024-04-25T09:56:38.454387+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:59.539232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437996+0000","last_change":"2024-04-25T09:56:38.437996+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437996+0000","last_undegraded":"2024-04-25T09:56:38.437996+0000","last_fullsized":"2024-04-25T09:56:38.437996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.441954+0000","last_change":"2024-04-25T09:56:38.441046+0000","last_active":"2024-04-25T09:56:38.441954+0000","last_peered":"2024-04-25T09:56:38.441954+0000","last_clean":"2024-04-25T09:56:38.441954+0000","last_became_active":"2024-04-25T09:56:38.440736+0000","last_became_peered":"2024-04-25T09:56:38.440736+0000","last_unstale":"2024-04-25T09:56:38.441954+0000","last_undegraded":"2024-04-25T09:56:38.441954+0000","last_fullsized":"2024-04-25T09:56:38.441954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:10.273708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434042+0000","last_change":"2024-04-25T09:56:38.434042+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434042+0000","last_undegraded":"2024-04-25T09:56:38.434042+0000","last_fullsized":"2024-04-25T09:56:38.434042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450475+0000","last_change":"2024-04-25T09:56:38.443577+0000","last_active":"2024-04-25T09:56:38.450475+0000","last_peered":"2024-04-25T09:56:38.450475+0000","last_clean":"2024-04-25T09:56:38.450475+0000","last_became_active":"2024-04-25T09:56:38.443304+0000","last_became_peered":"2024-04-25T09:56:38.443304+0000","last_unstale":"2024-04-25T09:56:38.450475+0000","last_undegraded":"2024-04-25T09:56:38.450475+0000","last_fullsized":"2024-04-25T09:56:38.450475+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.933751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436494+0000","last_change":"2024-04-25T09:56:38.436494+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436494+0000","last_undegraded":"2024-04-25T09:56:38.436494+0000","last_fullsized":"2024-04-25T09:56:38.436494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454005+0000","last_change":"2024-04-25T09:56:38.451937+0000","last_active":"2024-04-25T09:56:38.454005+0000","last_peered":"2024-04-25T09:56:38.454005+0000","last_clean":"2024-04-25T09:56:38.454005+0000","last_became_active":"2024-04-25T09:56:38.451248+0000","last_became_peered":"2024-04-25T09:56:38.451248+0000","last_unstale":"2024-04-25T09:56:38.454005+0000","last_undegraded":"2024-04-25T09:56:38.454005+0000","last_fullsized":"2024-04-25T09:56:38.454005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.404639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435576+0000","last_change":"2024-04-25T09:56:38.435576+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435576+0000","last_undegraded":"2024-04-25T09:56:38.435576+0000","last_fullsized":"2024-04-25T09:56:38.435576+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450527+0000","last_change":"2024-04-25T09:56:38.446452+0000","last_active":"2024-04-25T09:56:38.450527+0000","last_peered":"2024-04-25T09:56:38.450527+0000","last_clean":"2024-04-25T09:56:38.450527+0000","last_became_active":"2024-04-25T09:56:38.446166+0000","last_became_peered":"2024-04-25T09:56:38.446166+0000","last_unstale":"2024-04-25T09:56:38.450527+0000","last_undegraded":"2024-04-25T09:56:38.450527+0000","last_fullsized":"2024-04-25T09:56:38.450527+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:10.912571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435303+0000","last_change":"2024-04-25T09:56:38.435303+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435303+0000","last_undegraded":"2024-04-25T09:56:38.435303+0000","last_fullsized":"2024-04-25T09:56:38.435303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448987+0000","last_change":"2024-04-25T09:56:38.447857+0000","last_active":"2024-04-25T09:56:38.448987+0000","last_peered":"2024-04-25T09:56:38.448987+0000","last_clean":"2024-04-25T09:56:38.448987+0000","last_became_active":"2024-04-25T09:56:38.447519+0000","last_became_peered":"2024-04-25T09:56:38.447519+0000","last_unstale":"2024-04-25T09:56:38.448987+0000","last_undegraded":"2024-04-25T09:56:38.448987+0000","last_fullsized":"2024-04-25T09:56:38.448987+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:32.728995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.443451+0000","last_change":"2024-04-25T09:56:38.443451+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.443451+0000","last_undegraded":"2024-04-25T09:56:38.443451+0000","last_fullsized":"2024-04-25T09:56:38.443451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449375+0000","last_change":"2024-04-25T09:56:38.448408+0000","last_active":"2024-04-25T09:56:38.449375+0000","last_peered":"2024-04-25T09:56:38.449375+0000","last_clean":"2024-04-25T09:56:38.449375+0000","last_became_active":"2024-04-25T09:56:38.448024+0000","last_became_peered":"2024-04-25T09:56:38.448024+0000","last_unstale":"2024-04-25T09:56:38.449375+0000","last_undegraded":"2024-04-25T09:56:38.449375+0000","last_fullsized":"2024-04-25T09:56:38.449375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.635113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434466+0000","last_change":"2024-04-25T09:56:38.434466+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434466+0000","last_undegraded":"2024-04-25T09:56:38.434466+0000","last_fullsized":"2024-04-25T09:56:38.434466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449665+0000","last_change":"2024-04-25T09:56:38.442585+0000","last_active":"2024-04-25T09:56:38.449665+0000","last_peered":"2024-04-25T09:56:38.449665+0000","last_clean":"2024-04-25T09:56:38.449665+0000","last_became_active":"2024-04-25T09:56:38.442310+0000","last_became_peered":"2024-04-25T09:56:38.442310+0000","last_unstale":"2024-04-25T09:56:38.449665+0000","last_undegraded":"2024-04-25T09:56:38.449665+0000","last_fullsized":"2024-04-25T09:56:38.449665+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437364+0000","last_change":"2024-04-25T09:56:38.437364+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437364+0000","last_undegraded":"2024-04-25T09:56:38.437364+0000","last_fullsized":"2024-04-25T09:56:38.437364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450830+0000","last_change":"2024-04-25T09:56:38.442079+0000","last_active":"2024-04-25T09:56:38.450830+0000","last_peered":"2024-04-25T09:56:38.450830+0000","last_clean":"2024-04-25T09:56:38.450830+0000","last_became_active":"2024-04-25T09:56:38.441787+0000","last_became_peered":"2024-04-25T09:56:38.441787+0000","last_unstale":"2024-04-25T09:56:38.450830+0000","last_undegraded":"2024-04-25T09:56:38.450830+0000","last_fullsized":"2024-04-25T09:56:38.450830+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.776063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436133+0000","last_change":"2024-04-25T09:56:38.436133+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436133+0000","last_undegraded":"2024-04-25T09:56:38.436133+0000","last_fullsized":"2024-04-25T09:56:38.436133+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452759+0000","last_change":"2024-04-25T09:56:38.450312+0000","last_active":"2024-04-25T09:56:38.452759+0000","last_peered":"2024-04-25T09:56:38.452759+0000","last_clean":"2024-04-25T09:56:38.452759+0000","last_became_active":"2024-04-25T09:56:38.450049+0000","last_became_peered":"2024-04-25T09:56:38.450049+0000","last_unstale":"2024-04-25T09:56:38.452759+0000","last_undegraded":"2024-04-25T09:56:38.452759+0000","last_fullsized":"2024-04-25T09:56:38.452759+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:33.806055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431039+0000","last_change":"2024-04-25T09:56:38.431039+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431039+0000","last_undegraded":"2024-04-25T09:56:38.431039+0000","last_fullsized":"2024-04-25T09:56:38.431039+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443404+0000","last_change":"2024-04-25T09:56:38.441913+0000","last_active":"2024-04-25T09:56:38.443404+0000","last_peered":"2024-04-25T09:56:38.443404+0000","last_clean":"2024-04-25T09:56:38.443404+0000","last_became_active":"2024-04-25T09:56:38.441584+0000","last_became_peered":"2024-04-25T09:56:38.441584+0000","last_unstale":"2024-04-25T09:56:38.443404+0000","last_undegraded":"2024-04-25T09:56:38.443404+0000","last_fullsized":"2024-04-25T09:56:38.443404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:06.302131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433502+0000","last_change":"2024-04-25T09:56:38.433502+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433502+0000","last_undegraded":"2024-04-25T09:56:38.433502+0000","last_fullsized":"2024-04-25T09:56:38.433502+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.447445+0000","last_change":"2024-04-25T09:56:38.444951+0000","last_active":"2024-04-25T09:56:38.447445+0000","last_peered":"2024-04-25T09:56:38.447445+0000","last_clean":"2024-04-25T09:56:38.447445+0000","last_became_active":"2024-04-25T09:56:38.444617+0000","last_became_peered":"2024-04-25T09:56:38.444617+0000","last_unstale":"2024-04-25T09:56:38.447445+0000","last_undegraded":"2024-04-25T09:56:38.447445+0000","last_fullsized":"2024-04-25T09:56:38.447445+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.557978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.443248+0000","last_change":"2024-04-25T09:56:38.443248+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.443248+0000","last_undegraded":"2024-04-25T09:56:38.443248+0000","last_fullsized":"2024-04-25T09:56:38.443248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.451069+0000","last_change":"2024-04-25T09:56:38.447517+0000","last_active":"2024-04-25T09:56:38.451069+0000","last_peered":"2024-04-25T09:56:38.451069+0000","last_clean":"2024-04-25T09:56:38.451069+0000","last_became_active":"2024-04-25T09:56:38.447204+0000","last_became_peered":"2024-04-25T09:56:38.447204+0000","last_unstale":"2024-04-25T09:56:38.451069+0000","last_undegraded":"2024-04-25T09:56:38.451069+0000","last_fullsized":"2024-04-25T09:56:38.451069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:34.630001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436966+0000","last_change":"2024-04-25T09:56:38.436966+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436966+0000","last_undegraded":"2024-04-25T09:56:38.436966+0000","last_fullsized":"2024-04-25T09:56:38.436966+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.446681+0000","last_change":"2024-04-25T09:56:38.443664+0000","last_active":"2024-04-25T09:56:38.446681+0000","last_peered":"2024-04-25T09:56:38.446681+0000","last_clean":"2024-04-25T09:56:38.446681+0000","last_became_active":"2024-04-25T09:56:38.443253+0000","last_became_peered":"2024-04-25T09:56:38.443253+0000","last_unstale":"2024-04-25T09:56:38.446681+0000","last_undegraded":"2024-04-25T09:56:38.446681+0000","last_fullsized":"2024-04-25T09:56:38.446681+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:49.974254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":255,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.435835+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:56:38.435835+0000","last_peered":"2024-04-25T09:56:38.435835+0000","last_clean":"2024-04-25T09:56:38.435835+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:56:38.435835+0000","last_undegraded":"2024-04-25T09:56:38.435835+0000","last_fullsized":"2024-04-25T09:56:38.435835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36652,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335188,"statfs":{"total":96636764160,"available":96599232512,"internally_reserved":0,"allocated":1343488,"data_stored":927508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36076,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335764,"statfs":{"total":96636764160,"available":96599822336,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36076,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335764,"statfs":{"total":96636764160,"available":96599822336,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":737280,"data_stored":330481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36080,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335760,"statfs":{"total":96636764160,"available":96599818240,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":737280,"data_stored":330481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36644,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335196,"statfs":{"total":96636764160,"available":96599240704,"internally_reserved":0,"allocated":1331200,"data_stored":920849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:56:42.451 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:56:42.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.584+0000 7fde0530a640 1 Processor -- start 2024-04-25T09:56:42.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.585+0000 7fde0530a640 1 -- start start 2024-04-25T09:56:42.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.585+0000 7fde0530a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde000b2cb0 0x7fde000d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.585+0000 7fde0530a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde000b97a0 con 0x7fde0015dbe0 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.585+0000 7fde0530a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde000d1c50 con 0x7fde000d1020 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.585+0000 7fde0530a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0015dfc0 con 0x7fde000b2cb0 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.585+0000 7fddfe7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde000b2cb0 0x7fde000d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.585+0000 7fddfeffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fde0015dbe0 0x7fde000d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36610/0 (socket says 172.21.15.19:36610) 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.585+0000 7fddfeffd640 1 -- 172.21.15.19:0/4159733823 learned_addr learned my addr 172.21.15.19:0/4159733823 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.586+0000 7fddfe7fc640 1 -- 172.21.15.19:0/4159733823 >> v1:172.21.15.112:6789/0 conn(0x7fde000d1020 legacy=0x7fde000d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.586+0000 7fddfe7fc640 1 -- 172.21.15.19:0/4159733823 >> v1:172.21.15.19:6789/0 conn(0x7fde0015dbe0 legacy=0x7fde000d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.586+0000 7fddfdffb640 1 -- 172.21.15.19:0/4159733823 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474617340 0 0) 0x7fde000b97a0 con 0x7fde0015dbe0 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.586+0000 7fddfe7fc640 1 -- 172.21.15.19:0/4159733823 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000d1f30 con 0x7fde000b2cb0 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.586+0000 7fddfe7fc640 1 --2- 172.21.15.19:0/4159733823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde000b2cb0 0x7fde000d2730 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fdde80097a0 tx=0x7fdde802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b933e7ad1a2801 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.587+0000 7fddfdffb640 1 -- 172.21.15.19:0/4159733823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8037070 con 0x7fde000b2cb0 2024-04-25T09:56:42.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.587+0000 7fddfdffb640 1 -- 172.21.15.19:0/4159733823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde8032440 con 0x7fde000b2cb0 2024-04-25T09:56:42.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.587+0000 7fddfdffb640 1 -- 172.21.15.19:0/4159733823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8032810 con 0x7fde000b2cb0 2024-04-25T09:56:42.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.587+0000 7fde0530a640 1 -- 172.21.15.19:0/4159733823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde000b2cb0 msgr2=0x7fde000d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.587+0000 7fde0530a640 1 --2- 172.21.15.19:0/4159733823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde000b2cb0 0x7fde000d2730 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fdde80097a0 tx=0x7fdde802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.587+0000 7fde0530a640 1 -- 172.21.15.19:0/4159733823 shutdown_connections 2024-04-25T09:56:42.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.587+0000 7fde0530a640 1 --2- 172.21.15.19:0/4159733823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde000b2cb0 0x7fde000d2730 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.587+0000 7fde0530a640 1 -- 172.21.15.19:0/4159733823 >> 172.21.15.19:0/4159733823 conn(0x7fde00061a20 msgr2=0x7fde00061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.587+0000 7fde0530a640 1 -- 172.21.15.19:0/4159733823 shutdown_connections 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.587+0000 7fde0530a640 1 -- 172.21.15.19:0/4159733823 wait complete. 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.589+0000 7fde0530a640 1 Processor -- start 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.589+0000 7fde0530a640 1 -- start start 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.589+0000 7fde0530a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde000b2cb0 0x7fde0014b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.589+0000 7fde0530a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde000d1020 0x7fde0015e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.589+0000 7fde0530a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0015dbe0 0x7fde0015fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.589+0000 7fde0530a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000b92a0 con 0x7fde000d1020 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.589+0000 7fde0530a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000d1a00 con 0x7fde0015dbe0 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.589+0000 7fde0530a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde000b9420 con 0x7fde000b2cb0 2024-04-25T09:56:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.589+0000 7fddfe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde000d1020 0x7fde0015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.589+0000 7fddfe7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde000d1020 0x7fde0015e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53264/0 (socket says 172.21.15.19:53264) 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.589+0000 7fddfe7fc640 1 -- 172.21.15.19:0/562283748 learned_addr learned my addr 172.21.15.19:0/562283748 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fddff7fe640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0015dbe0 0x7fde0015fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fddfeffd640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde000b2cb0 0x7fde0014b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fddfe7fc640 1 -- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde000b2cb0 msgr2=0x7fde0014b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fddfe7fc640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde000b2cb0 0x7fde0014b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fddfe7fc640 1 -- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0015dbe0 msgr2=0x7fde0015fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fddfe7fc640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0015dbe0 0x7fde0015fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fddfe7fc640 1 -- 172.21.15.19:0/562283748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0015eb10 con 0x7fde000d1020 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fddfe7fc640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde000d1020 0x7fde0015e150 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fdde80097a0 tx=0x7fdde80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fddf77fe640 1 -- 172.21.15.19:0/562283748 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8037040 con 0x7fde000d1020 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fddf77fe640 1 -- 172.21.15.19:0/562283748 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde8004250 con 0x7fde000d1020 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fde0530a640 1 -- 172.21.15.19:0/562283748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0015edf0 con 0x7fde000d1020 2024-04-25T09:56:42.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fddf77fe640 1 -- 172.21.15.19:0/562283748 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8032e20 con 0x7fde000d1020 2024-04-25T09:56:42.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.590+0000 7fde0530a640 1 -- 172.21.15.19:0/562283748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0015f3b0 con 0x7fde000d1020 2024-04-25T09:56:42.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.591+0000 7fde0530a640 1 -- 172.21.15.19:0/562283748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde00073990 con 0x7fde000d1020 2024-04-25T09:56:42.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.592+0000 7fddf77fe640 1 -- 172.21.15.19:0/562283748 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdde8030770 con 0x7fde000d1020 2024-04-25T09:56:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.592+0000 7fddf77fe640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdde006b4d0 0x7fdde006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.592+0000 7fddf77fe640 1 -- 172.21.15.19:0/562283748 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdde80ab5b0 con 0x7fde000d1020 2024-04-25T09:56:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.592+0000 7fddfeffd640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdde006b4d0 0x7fdde006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.593+0000 7fddfeffd640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdde006b4d0 0x7fdde006d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fddf8000f60 tx=0x7fddf8007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:42.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.596+0000 7fddf77fe640 1 -- 172.21.15.19:0/562283748 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdde807a330 con 0x7fde000d1020 2024-04-25T09:56:42.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.773+0000 7fde0530a640 1 -- 172.21.15.19:0/562283748 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde0015ac00 con 0x7fdde006b4d0 2024-04-25T09:56:42.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.780+0000 7fddf77fe640 1 -- 172.21.15.19:0/562283748 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246700 (secure 0 0 0) 0x7fde0015ac00 con 0x7fdde006b4d0 2024-04-25T09:56:42.785 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:42.786 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:56:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.782+0000 7fde0530a640 1 -- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdde006b4d0 msgr2=0x7fdde006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.782+0000 7fde0530a640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdde006b4d0 0x7fdde006d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fddf8000f60 tx=0x7fddf8007c00 comp rx=0 tx=0).stop 2024-04-25T09:56:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.782+0000 7fde0530a640 1 -- 172.21.15.19:0/562283748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde000d1020 msgr2=0x7fde0015e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.782+0000 7fde0530a640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde000d1020 0x7fde0015e150 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fdde80097a0 tx=0x7fdde80046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.782+0000 7fde0530a640 1 -- 172.21.15.19:0/562283748 shutdown_connections 2024-04-25T09:56:42.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.782+0000 7fde0530a640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdde006b4d0 0x7fdde006d990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.782+0000 7fde0530a640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde0015dbe0 0x7fde0015fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.782+0000 7fde0530a640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde000d1020 0x7fde0015e150 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.782+0000 7fde0530a640 1 --2- 172.21.15.19:0/562283748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde000b2cb0 0x7fde0014b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.783+0000 7fde0530a640 1 -- 172.21.15.19:0/562283748 >> 172.21.15.19:0/562283748 conn(0x7fde00061a20 msgr2=0x7fde00153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:42.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.783+0000 7fde0530a640 1 -- 172.21.15.19:0/562283748 shutdown_connections 2024-04-25T09:56:42.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.783+0000 7fde0530a640 1 -- 172.21.15.19:0/562283748 wait complete. 2024-04-25T09:56:42.797 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-04-25T09:56:40.961147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289720,"kb_used_data":6980,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685000,"statfs":{"total":773094113280,"available":772797440000,"internally_reserved":0,"allocated":7147520,"data_stored":3857879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000948"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437683+0000","last_change":"2024-04-25T09:56:38.437683+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437683+0000","last_undegraded":"2024-04-25T09:56:38.437683+0000","last_fullsized":"2024-04-25T09:56:38.437683+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436642+0000","last_change":"2024-04-25T09:56:38.436642+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436642+0000","last_undegraded":"2024-04-25T09:56:38.436642+0000","last_fullsized":"2024-04-25T09:56:38.436642+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433788+0000","last_change":"2024-04-25T09:56:38.433788+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433788+0000","last_undegraded":"2024-04-25T09:56:38.433788+0000","last_fullsized":"2024-04-25T09:56:38.433788+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431818+0000","last_change":"2024-04-25T09:56:38.431818+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431818+0000","last_undegraded":"2024-04-25T09:56:38.431818+0000","last_fullsized":"2024-04-25T09:56:38.431818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437599+0000","last_change":"2024-04-25T09:56:38.437599+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437599+0000","last_undegraded":"2024-04-25T09:56:38.437599+0000","last_fullsized":"2024-04-25T09:56:38.437599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436685+0000","last_change":"2024-04-25T09:56:38.436685+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436685+0000","last_undegraded":"2024-04-25T09:56:38.436685+0000","last_fullsized":"2024-04-25T09:56:38.436685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439909+0000","last_change":"2024-04-25T09:56:38.439909+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439909+0000","last_undegraded":"2024-04-25T09:56:38.439909+0000","last_fullsized":"2024-04-25T09:56:38.439909+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438373+0000","last_change":"2024-04-25T09:56:38.438373+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438373+0000","last_undegraded":"2024-04-25T09:56:38.438373+0000","last_fullsized":"2024-04-25T09:56:38.438373+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433700+0000","last_change":"2024-04-25T09:56:38.433700+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433700+0000","last_undegraded":"2024-04-25T09:56:38.433700+0000","last_fullsized":"2024-04-25T09:56:38.433700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435532+0000","last_change":"2024-04-25T09:56:38.435532+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435532+0000","last_undegraded":"2024-04-25T09:56:38.435532+0000","last_fullsized":"2024-04-25T09:56:38.435532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435197+0000","last_change":"2024-04-25T09:56:38.435197+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435197+0000","last_undegraded":"2024-04-25T09:56:38.435197+0000","last_fullsized":"2024-04-25T09:56:38.435197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434689+0000","last_change":"2024-04-25T09:56:38.434689+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434689+0000","last_undegraded":"2024-04-25T09:56:38.434689+0000","last_fullsized":"2024-04-25T09:56:38.434689+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436051+0000","last_change":"2024-04-25T09:56:38.436051+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436051+0000","last_undegraded":"2024-04-25T09:56:38.436051+0000","last_fullsized":"2024-04-25T09:56:38.436051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433377+0000","last_change":"2024-04-25T09:56:38.433377+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433377+0000","last_undegraded":"2024-04-25T09:56:38.433377+0000","last_fullsized":"2024-04-25T09:56:38.433377+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433505+0000","last_change":"2024-04-25T09:56:38.433505+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433505+0000","last_undegraded":"2024-04-25T09:56:38.433505+0000","last_fullsized":"2024-04-25T09:56:38.433505+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439675+0000","last_change":"2024-04-25T09:56:38.439675+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439675+0000","last_undegraded":"2024-04-25T09:56:38.439675+0000","last_fullsized":"2024-04-25T09:56:38.439675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434662+0000","last_change":"2024-04-25T09:56:38.434662+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434662+0000","last_undegraded":"2024-04-25T09:56:38.434662+0000","last_fullsized":"2024-04-25T09:56:38.434662+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435818+0000","last_change":"2024-04-25T09:56:38.435818+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435818+0000","last_undegraded":"2024-04-25T09:56:38.435818+0000","last_fullsized":"2024-04-25T09:56:38.435818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437438+0000","last_change":"2024-04-25T09:56:38.437438+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437438+0000","last_undegraded":"2024-04-25T09:56:38.437438+0000","last_fullsized":"2024-04-25T09:56:38.437438+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434623+0000","last_change":"2024-04-25T09:56:38.434623+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434623+0000","last_undegraded":"2024-04-25T09:56:38.434623+0000","last_fullsized":"2024-04-25T09:56:38.434623+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434027+0000","last_change":"2024-04-25T09:56:38.434027+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434027+0000","last_undegraded":"2024-04-25T09:56:38.434027+0000","last_fullsized":"2024-04-25T09:56:38.434027+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436029+0000","last_change":"2024-04-25T09:56:38.436029+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436029+0000","last_undegraded":"2024-04-25T09:56:38.436029+0000","last_fullsized":"2024-04-25T09:56:38.436029+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434596+0000","last_change":"2024-04-25T09:56:38.434596+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434596+0000","last_undegraded":"2024-04-25T09:56:38.434596+0000","last_fullsized":"2024-04-25T09:56:38.434596+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433635+0000","last_change":"2024-04-25T09:56:38.433635+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433635+0000","last_undegraded":"2024-04-25T09:56:38.433635+0000","last_fullsized":"2024-04-25T09:56:38.433635+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437925+0000","last_change":"2024-04-25T09:56:38.437925+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437925+0000","last_undegraded":"2024-04-25T09:56:38.437925+0000","last_fullsized":"2024-04-25T09:56:38.437925+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434898+0000","last_change":"2024-04-25T09:56:38.434898+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434898+0000","last_undegraded":"2024-04-25T09:56:38.434898+0000","last_fullsized":"2024-04-25T09:56:38.434898+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434806+0000","last_change":"2024-04-25T09:56:38.434806+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434806+0000","last_undegraded":"2024-04-25T09:56:38.434806+0000","last_fullsized":"2024-04-25T09:56:38.434806+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434169+0000","last_change":"2024-04-25T09:56:38.434169+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434169+0000","last_undegraded":"2024-04-25T09:56:38.434169+0000","last_fullsized":"2024-04-25T09:56:38.434169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433478+0000","last_change":"2024-04-25T09:56:38.433478+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433478+0000","last_undegraded":"2024-04-25T09:56:38.433478+0000","last_fullsized":"2024-04-25T09:56:38.433478+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435681+0000","last_change":"2024-04-25T09:56:38.435681+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435681+0000","last_undegraded":"2024-04-25T09:56:38.435681+0000","last_fullsized":"2024-04-25T09:56:38.435681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431678+0000","last_change":"2024-04-25T09:56:38.431678+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431678+0000","last_undegraded":"2024-04-25T09:56:38.431678+0000","last_fullsized":"2024-04-25T09:56:38.431678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434572+0000","last_change":"2024-04-25T09:56:38.434572+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434572+0000","last_undegraded":"2024-04-25T09:56:38.434572+0000","last_fullsized":"2024-04-25T09:56:38.434572+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434920+0000","last_change":"2024-04-25T09:56:38.434920+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434920+0000","last_undegraded":"2024-04-25T09:56:38.434920+0000","last_fullsized":"2024-04-25T09:56:38.434920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.451222+0000","last_change":"2024-04-25T09:56:38.448802+0000","last_active":"2024-04-25T09:56:38.451222+0000","last_peered":"2024-04-25T09:56:38.451222+0000","last_clean":"2024-04-25T09:56:38.451222+0000","last_became_active":"2024-04-25T09:56:38.447596+0000","last_became_peered":"2024-04-25T09:56:38.447596+0000","last_unstale":"2024-04-25T09:56:38.451222+0000","last_undegraded":"2024-04-25T09:56:38.451222+0000","last_fullsized":"2024-04-25T09:56:38.451222+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:56.119011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.432113+0000","last_change":"2024-04-25T09:56:38.432113+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.432113+0000","last_undegraded":"2024-04-25T09:56:38.432113+0000","last_fullsized":"2024-04-25T09:56:38.432113+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450513+0000","last_change":"2024-04-25T09:56:38.446203+0000","last_active":"2024-04-25T09:56:38.450513+0000","last_peered":"2024-04-25T09:56:38.450513+0000","last_clean":"2024-04-25T09:56:38.450513+0000","last_became_active":"2024-04-25T09:56:38.445931+0000","last_became_peered":"2024-04-25T09:56:38.445931+0000","last_unstale":"2024-04-25T09:56:38.450513+0000","last_undegraded":"2024-04-25T09:56:38.450513+0000","last_fullsized":"2024-04-25T09:56:38.450513+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:48.759756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434485+0000","last_change":"2024-04-25T09:56:38.434485+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434485+0000","last_undegraded":"2024-04-25T09:56:38.434485+0000","last_fullsized":"2024-04-25T09:56:38.434485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449712+0000","last_change":"2024-04-25T09:56:38.442763+0000","last_active":"2024-04-25T09:56:38.449712+0000","last_peered":"2024-04-25T09:56:38.449712+0000","last_clean":"2024-04-25T09:56:38.449712+0000","last_became_active":"2024-04-25T09:56:38.442494+0000","last_became_peered":"2024-04-25T09:56:38.442494+0000","last_unstale":"2024-04-25T09:56:38.449712+0000","last_undegraded":"2024-04-25T09:56:38.449712+0000","last_fullsized":"2024-04-25T09:56:38.449712+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.557978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434490+0000","last_change":"2024-04-25T09:56:38.434490+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434490+0000","last_undegraded":"2024-04-25T09:56:38.434490+0000","last_fullsized":"2024-04-25T09:56:38.434490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443747+0000","last_change":"2024-04-25T09:56:38.441504+0000","last_active":"2024-04-25T09:56:38.443747+0000","last_peered":"2024-04-25T09:56:38.443747+0000","last_clean":"2024-04-25T09:56:38.443747+0000","last_became_active":"2024-04-25T09:56:38.441178+0000","last_became_peered":"2024-04-25T09:56:38.441178+0000","last_unstale":"2024-04-25T09:56:38.443747+0000","last_undegraded":"2024-04-25T09:56:38.443747+0000","last_fullsized":"2024-04-25T09:56:38.443747+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:49.974254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439169+0000","last_change":"2024-04-25T09:56:38.439169+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439169+0000","last_undegraded":"2024-04-25T09:56:38.439169+0000","last_fullsized":"2024-04-25T09:56:38.439169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443433+0000","last_change":"2024-04-25T09:56:38.442701+0000","last_active":"2024-04-25T09:56:38.443433+0000","last_peered":"2024-04-25T09:56:38.443433+0000","last_clean":"2024-04-25T09:56:38.443433+0000","last_became_active":"2024-04-25T09:56:38.442433+0000","last_became_peered":"2024-04-25T09:56:38.442433+0000","last_unstale":"2024-04-25T09:56:38.443433+0000","last_undegraded":"2024-04-25T09:56:38.443433+0000","last_fullsized":"2024-04-25T09:56:38.443433+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:55.484379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434138+0000","last_change":"2024-04-25T09:56:38.434138+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434138+0000","last_undegraded":"2024-04-25T09:56:38.434138+0000","last_fullsized":"2024-04-25T09:56:38.434138+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448317+0000","last_change":"2024-04-25T09:56:38.446402+0000","last_active":"2024-04-25T09:56:38.448317+0000","last_peered":"2024-04-25T09:56:38.448317+0000","last_clean":"2024-04-25T09:56:38.448317+0000","last_became_active":"2024-04-25T09:56:38.446088+0000","last_became_peered":"2024-04-25T09:56:38.446088+0000","last_unstale":"2024-04-25T09:56:38.448317+0000","last_undegraded":"2024-04-25T09:56:38.448317+0000","last_fullsized":"2024-04-25T09:56:38.448317+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.933751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438728+0000","last_change":"2024-04-25T09:56:38.438728+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438728+0000","last_undegraded":"2024-04-25T09:56:38.438728+0000","last_fullsized":"2024-04-25T09:56:38.438728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.447423+0000","last_change":"2024-04-25T09:56:38.445015+0000","last_active":"2024-04-25T09:56:38.447423+0000","last_peered":"2024-04-25T09:56:38.447423+0000","last_clean":"2024-04-25T09:56:38.447423+0000","last_became_active":"2024-04-25T09:56:38.444713+0000","last_became_peered":"2024-04-25T09:56:38.444713+0000","last_unstale":"2024-04-25T09:56:38.447423+0000","last_undegraded":"2024-04-25T09:56:38.447423+0000","last_fullsized":"2024-04-25T09:56:38.447423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435118+0000","last_change":"2024-04-25T09:56:38.435118+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435118+0000","last_undegraded":"2024-04-25T09:56:38.435118+0000","last_fullsized":"2024-04-25T09:56:38.435118+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448373+0000","last_change":"2024-04-25T09:56:38.446581+0000","last_active":"2024-04-25T09:56:38.448373+0000","last_peered":"2024-04-25T09:56:38.448373+0000","last_clean":"2024-04-25T09:56:38.448373+0000","last_became_active":"2024-04-25T09:56:38.444721+0000","last_became_peered":"2024-04-25T09:56:38.444721+0000","last_unstale":"2024-04-25T09:56:38.448373+0000","last_undegraded":"2024-04-25T09:56:38.448373+0000","last_fullsized":"2024-04-25T09:56:38.448373+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:40.119587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433629+0000","last_change":"2024-04-25T09:56:38.433629+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433629+0000","last_undegraded":"2024-04-25T09:56:38.433629+0000","last_fullsized":"2024-04-25T09:56:38.433629+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454111+0000","last_change":"2024-04-25T09:56:38.452304+0000","last_active":"2024-04-25T09:56:38.454111+0000","last_peered":"2024-04-25T09:56:38.454111+0000","last_clean":"2024-04-25T09:56:38.454111+0000","last_became_active":"2024-04-25T09:56:38.452001+0000","last_became_peered":"2024-04-25T09:56:38.452001+0000","last_unstale":"2024-04-25T09:56:38.454111+0000","last_undegraded":"2024-04-25T09:56:38.454111+0000","last_fullsized":"2024-04-25T09:56:38.454111+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:07.043227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437792+0000","last_change":"2024-04-25T09:56:38.437792+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437792+0000","last_undegraded":"2024-04-25T09:56:38.437792+0000","last_fullsized":"2024-04-25T09:56:38.437792+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452779+0000","last_change":"2024-04-25T09:56:38.451554+0000","last_active":"2024-04-25T09:56:38.452779+0000","last_peered":"2024-04-25T09:56:38.452779+0000","last_clean":"2024-04-25T09:56:38.452779+0000","last_became_active":"2024-04-25T09:56:38.451253+0000","last_became_peered":"2024-04-25T09:56:38.451253+0000","last_unstale":"2024-04-25T09:56:38.452779+0000","last_undegraded":"2024-04-25T09:56:38.452779+0000","last_fullsized":"2024-04-25T09:56:38.452779+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:44.723554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438457+0000","last_change":"2024-04-25T09:56:38.438457+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438457+0000","last_undegraded":"2024-04-25T09:56:38.438457+0000","last_fullsized":"2024-04-25T09:56:38.438457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454400+0000","last_change":"2024-04-25T09:56:38.452908+0000","last_active":"2024-04-25T09:56:38.454400+0000","last_peered":"2024-04-25T09:56:38.454400+0000","last_clean":"2024-04-25T09:56:38.454400+0000","last_became_active":"2024-04-25T09:56:38.452672+0000","last_became_peered":"2024-04-25T09:56:38.452672+0000","last_unstale":"2024-04-25T09:56:38.454400+0000","last_undegraded":"2024-04-25T09:56:38.454400+0000","last_fullsized":"2024-04-25T09:56:38.454400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.108179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431918+0000","last_change":"2024-04-25T09:56:38.431918+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431918+0000","last_undegraded":"2024-04-25T09:56:38.431918+0000","last_fullsized":"2024-04-25T09:56:38.431918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450753+0000","last_change":"2024-04-25T09:56:38.441289+0000","last_active":"2024-04-25T09:56:38.450753+0000","last_peered":"2024-04-25T09:56:38.450753+0000","last_clean":"2024-04-25T09:56:38.450753+0000","last_became_active":"2024-04-25T09:56:38.440974+0000","last_became_peered":"2024-04-25T09:56:38.440974+0000","last_unstale":"2024-04-25T09:56:38.450753+0000","last_undegraded":"2024-04-25T09:56:38.450753+0000","last_fullsized":"2024-04-25T09:56:38.450753+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:06.200902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431725+0000","last_change":"2024-04-25T09:56:38.431725+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431725+0000","last_undegraded":"2024-04-25T09:56:38.431725+0000","last_fullsized":"2024-04-25T09:56:38.431725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.444935+0000","last_change":"2024-04-25T09:56:38.444152+0000","last_active":"2024-04-25T09:56:38.444935+0000","last_peered":"2024-04-25T09:56:38.444935+0000","last_clean":"2024-04-25T09:56:38.444935+0000","last_became_active":"2024-04-25T09:56:38.443891+0000","last_became_peered":"2024-04-25T09:56:38.443891+0000","last_unstale":"2024-04-25T09:56:38.444935+0000","last_undegraded":"2024-04-25T09:56:38.444935+0000","last_fullsized":"2024-04-25T09:56:38.444935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.635113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436850+0000","last_change":"2024-04-25T09:56:38.436850+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436850+0000","last_undegraded":"2024-04-25T09:56:38.436850+0000","last_fullsized":"2024-04-25T09:56:38.436850+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454025+0000","last_change":"2024-04-25T09:56:38.452182+0000","last_active":"2024-04-25T09:56:38.454025+0000","last_peered":"2024-04-25T09:56:38.454025+0000","last_clean":"2024-04-25T09:56:38.454025+0000","last_became_active":"2024-04-25T09:56:38.451492+0000","last_became_peered":"2024-04-25T09:56:38.451492+0000","last_unstale":"2024-04-25T09:56:38.454025+0000","last_undegraded":"2024-04-25T09:56:38.454025+0000","last_fullsized":"2024-04-25T09:56:38.454025+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:33.806055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438322+0000","last_change":"2024-04-25T09:56:38.438322+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438322+0000","last_undegraded":"2024-04-25T09:56:38.438322+0000","last_fullsized":"2024-04-25T09:56:38.438322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452814+0000","last_change":"2024-04-25T09:56:38.449599+0000","last_active":"2024-04-25T09:56:38.452814+0000","last_peered":"2024-04-25T09:56:38.452814+0000","last_clean":"2024-04-25T09:56:38.452814+0000","last_became_active":"2024-04-25T09:56:38.449302+0000","last_became_peered":"2024-04-25T09:56:38.449302+0000","last_unstale":"2024-04-25T09:56:38.452814+0000","last_undegraded":"2024-04-25T09:56:38.452814+0000","last_fullsized":"2024-04-25T09:56:38.452814+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.404639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433556+0000","last_change":"2024-04-25T09:56:38.433556+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433556+0000","last_undegraded":"2024-04-25T09:56:38.433556+0000","last_fullsized":"2024-04-25T09:56:38.433556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448561+0000","last_change":"2024-04-25T09:56:38.447935+0000","last_active":"2024-04-25T09:56:38.448561+0000","last_peered":"2024-04-25T09:56:38.448561+0000","last_clean":"2024-04-25T09:56:38.448561+0000","last_became_active":"2024-04-25T09:56:38.447648+0000","last_became_peered":"2024-04-25T09:56:38.447648+0000","last_unstale":"2024-04-25T09:56:38.448561+0000","last_undegraded":"2024-04-25T09:56:38.448561+0000","last_fullsized":"2024-04-25T09:56:38.448561+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:32.556063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433134+0000","last_change":"2024-04-25T09:56:38.433134+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433134+0000","last_undegraded":"2024-04-25T09:56:38.433134+0000","last_fullsized":"2024-04-25T09:56:38.433134+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452679+0000","last_change":"2024-04-25T09:56:38.451735+0000","last_active":"2024-04-25T09:56:38.452679+0000","last_peered":"2024-04-25T09:56:38.452679+0000","last_clean":"2024-04-25T09:56:38.452679+0000","last_became_active":"2024-04-25T09:56:38.451108+0000","last_became_peered":"2024-04-25T09:56:38.451108+0000","last_unstale":"2024-04-25T09:56:38.452679+0000","last_undegraded":"2024-04-25T09:56:38.452679+0000","last_fullsized":"2024-04-25T09:56:38.452679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:34.259055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436880+0000","last_change":"2024-04-25T09:56:38.436880+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436880+0000","last_undegraded":"2024-04-25T09:56:38.436880+0000","last_fullsized":"2024-04-25T09:56:38.436880+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452467+0000","last_change":"2024-04-25T09:56:38.449467+0000","last_active":"2024-04-25T09:56:38.452467+0000","last_peered":"2024-04-25T09:56:38.452467+0000","last_clean":"2024-04-25T09:56:38.452467+0000","last_became_active":"2024-04-25T09:56:38.449110+0000","last_became_peered":"2024-04-25T09:56:38.449110+0000","last_unstale":"2024-04-25T09:56:38.452467+0000","last_undegraded":"2024-04-25T09:56:38.452467+0000","last_fullsized":"2024-04-25T09:56:38.452467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:34.259055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435023+0000","last_change":"2024-04-25T09:56:38.435023+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435023+0000","last_undegraded":"2024-04-25T09:56:38.435023+0000","last_fullsized":"2024-04-25T09:56:38.435023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454387+0000","last_change":"2024-04-25T09:56:38.452721+0000","last_active":"2024-04-25T09:56:38.454387+0000","last_peered":"2024-04-25T09:56:38.454387+0000","last_clean":"2024-04-25T09:56:38.454387+0000","last_became_active":"2024-04-25T09:56:38.452370+0000","last_became_peered":"2024-04-25T09:56:38.452370+0000","last_unstale":"2024-04-25T09:56:38.454387+0000","last_undegraded":"2024-04-25T09:56:38.454387+0000","last_fullsized":"2024-04-25T09:56:38.454387+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:59.539232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437996+0000","last_change":"2024-04-25T09:56:38.437996+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437996+0000","last_undegraded":"2024-04-25T09:56:38.437996+0000","last_fullsized":"2024-04-25T09:56:38.437996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.441954+0000","last_change":"2024-04-25T09:56:38.441046+0000","last_active":"2024-04-25T09:56:38.441954+0000","last_peered":"2024-04-25T09:56:38.441954+0000","last_clean":"2024-04-25T09:56:38.441954+0000","last_became_active":"2024-04-25T09:56:38.440736+0000","last_became_peered":"2024-04-25T09:56:38.440736+0000","last_unstale":"2024-04-25T09:56:38.441954+0000","last_undegraded":"2024-04-25T09:56:38.441954+0000","last_fullsized":"2024-04-25T09:56:38.441954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:10.273708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434042+0000","last_change":"2024-04-25T09:56:38.434042+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434042+0000","last_undegraded":"2024-04-25T09:56:38.434042+0000","last_fullsized":"2024-04-25T09:56:38.434042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450475+0000","last_change":"2024-04-25T09:56:38.443577+0000","last_active":"2024-04-25T09:56:38.450475+0000","last_peered":"2024-04-25T09:56:38.450475+0000","last_clean":"2024-04-25T09:56:38.450475+0000","last_became_active":"2024-04-25T09:56:38.443304+0000","last_became_peered":"2024-04-25T09:56:38.443304+0000","last_unstale":"2024-04-25T09:56:38.450475+0000","last_undegraded":"2024-04-25T09:56:38.450475+0000","last_fullsized":"2024-04-25T09:56:38.450475+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.933751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436494+0000","last_change":"2024-04-25T09:56:38.436494+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436494+0000","last_undegraded":"2024-04-25T09:56:38.436494+0000","last_fullsized":"2024-04-25T09:56:38.436494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454005+0000","last_change":"2024-04-25T09:56:38.451937+0000","last_active":"2024-04-25T09:56:38.454005+0000","last_peered":"2024-04-25T09:56:38.454005+0000","last_clean":"2024-04-25T09:56:38.454005+0000","last_became_active":"2024-04-25T09:56:38.451248+0000","last_became_peered":"2024-04-25T09:56:38.451248+0000","last_unstale":"2024-04-25T09:56:38.454005+0000","last_undegraded":"2024-04-25T09:56:38.454005+0000","last_fullsized":"2024-04-25T09:56:38.454005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.404639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435576+0000","last_change":"2024-04-25T09:56:38.435576+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435576+0000","last_undegraded":"2024-04-25T09:56:38.435576+0000","last_fullsized":"2024-04-25T09:56:38.435576+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450527+0000","last_change":"2024-04-25T09:56:38.446452+0000","last_active":"2024-04-25T09:56:38.450527+0000","last_peered":"2024-04-25T09:56:38.450527+0000","last_clean":"2024-04-25T09:56:38.450527+0000","last_became_active":"2024-04-25T09:56:38.446166+0000","last_became_peered":"2024-04-25T09:56:38.446166+0000","last_unstale":"2024-04-25T09:56:38.450527+0000","last_undegraded":"2024-04-25T09:56:38.450527+0000","last_fullsized":"2024-04-25T09:56:38.450527+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:10.912571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435303+0000","last_change":"2024-04-25T09:56:38.435303+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435303+0000","last_undegraded":"2024-04-25T09:56:38.435303+0000","last_fullsized":"2024-04-25T09:56:38.435303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448987+0000","last_change":"2024-04-25T09:56:38.447857+0000","last_active":"2024-04-25T09:56:38.448987+0000","last_peered":"2024-04-25T09:56:38.448987+0000","last_clean":"2024-04-25T09:56:38.448987+0000","last_became_active":"2024-04-25T09:56:38.447519+0000","last_became_peered":"2024-04-25T09:56:38.447519+0000","last_unstale":"2024-04-25T09:56:38.448987+0000","last_undegraded":"2024-04-25T09:56:38.448987+0000","last_fullsized":"2024-04-25T09:56:38.448987+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:32.728995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.443451+0000","last_change":"2024-04-25T09:56:38.443451+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.443451+0000","last_undegraded":"2024-04-25T09:56:38.443451+0000","last_fullsized":"2024-04-25T09:56:38.443451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449375+0000","last_change":"2024-04-25T09:56:38.448408+0000","last_active":"2024-04-25T09:56:38.449375+0000","last_peered":"2024-04-25T09:56:38.449375+0000","last_clean":"2024-04-25T09:56:38.449375+0000","last_became_active":"2024-04-25T09:56:38.448024+0000","last_became_peered":"2024-04-25T09:56:38.448024+0000","last_unstale":"2024-04-25T09:56:38.449375+0000","last_undegraded":"2024-04-25T09:56:38.449375+0000","last_fullsized":"2024-04-25T09:56:38.449375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.635113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434466+0000","last_change":"2024-04-25T09:56:38.434466+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434466+0000","last_undegraded":"2024-04-25T09:56:38.434466+0000","last_fullsized":"2024-04-25T09:56:38.434466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449665+0000","last_change":"2024-04-25T09:56:38.442585+0000","last_active":"2024-04-25T09:56:38.449665+0000","last_peered":"2024-04-25T09:56:38.449665+0000","last_clean":"2024-04-25T09:56:38.449665+0000","last_became_active":"2024-04-25T09:56:38.442310+0000","last_became_peered":"2024-04-25T09:56:38.442310+0000","last_unstale":"2024-04-25T09:56:38.449665+0000","last_undegraded":"2024-04-25T09:56:38.449665+0000","last_fullsized":"2024-04-25T09:56:38.449665+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437364+0000","last_change":"2024-04-25T09:56:38.437364+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437364+0000","last_undegraded":"2024-04-25T09:56:38.437364+0000","last_fullsized":"2024-04-25T09:56:38.437364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450830+0000","last_change":"2024-04-25T09:56:38.442079+0000","last_active":"2024-04-25T09:56:38.450830+0000","last_peered":"2024-04-25T09:56:38.450830+0000","last_clean":"2024-04-25T09:56:38.450830+0000","last_became_active":"2024-04-25T09:56:38.441787+0000","last_became_peered":"2024-04-25T09:56:38.441787+0000","last_unstale":"2024-04-25T09:56:38.450830+0000","last_undegraded":"2024-04-25T09:56:38.450830+0000","last_fullsized":"2024-04-25T09:56:38.450830+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.776063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436133+0000","last_change":"2024-04-25T09:56:38.436133+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436133+0000","last_undegraded":"2024-04-25T09:56:38.436133+0000","last_fullsized":"2024-04-25T09:56:38.436133+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452759+0000","last_change":"2024-04-25T09:56:38.450312+0000","last_active":"2024-04-25T09:56:38.452759+0000","last_peered":"2024-04-25T09:56:38.452759+0000","last_clean":"2024-04-25T09:56:38.452759+0000","last_became_active":"2024-04-25T09:56:38.450049+0000","last_became_peered":"2024-04-25T09:56:38.450049+0000","last_unstale":"2024-04-25T09:56:38.452759+0000","last_undegraded":"2024-04-25T09:56:38.452759+0000","last_fullsized":"2024-04-25T09:56:38.452759+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:33.806055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431039+0000","last_change":"2024-04-25T09:56:38.431039+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431039+0000","last_undegraded":"2024-04-25T09:56:38.431039+0000","last_fullsized":"2024-04-25T09:56:38.431039+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443404+0000","last_change":"2024-04-25T09:56:38.441913+0000","last_active":"2024-04-25T09:56:38.443404+0000","last_peered":"2024-04-25T09:56:38.443404+0000","last_clean":"2024-04-25T09:56:38.443404+0000","last_became_active":"2024-04-25T09:56:38.441584+0000","last_became_peered":"2024-04-25T09:56:38.441584+0000","last_unstale":"2024-04-25T09:56:38.443404+0000","last_undegraded":"2024-04-25T09:56:38.443404+0000","last_fullsized":"2024-04-25T09:56:38.443404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:06.302131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433502+0000","last_change":"2024-04-25T09:56:38.433502+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433502+0000","last_undegraded":"2024-04-25T09:56:38.433502+0000","last_fullsized":"2024-04-25T09:56:38.433502+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.447445+0000","last_change":"2024-04-25T09:56:38.444951+0000","last_active":"2024-04-25T09:56:38.447445+0000","last_peered":"2024-04-25T09:56:38.447445+0000","last_clean":"2024-04-25T09:56:38.447445+0000","last_became_active":"2024-04-25T09:56:38.444617+0000","last_became_peered":"2024-04-25T09:56:38.444617+0000","last_unstale":"2024-04-25T09:56:38.447445+0000","last_undegraded":"2024-04-25T09:56:38.447445+0000","last_fullsized":"2024-04-25T09:56:38.447445+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.557978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.443248+0000","last_change":"2024-04-25T09:56:38.443248+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.443248+0000","last_undegraded":"2024-04-25T09:56:38.443248+0000","last_fullsized":"2024-04-25T09:56:38.443248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.451069+0000","last_change":"2024-04-25T09:56:38.447517+0000","last_active":"2024-04-25T09:56:38.451069+0000","last_peered":"2024-04-25T09:56:38.451069+0000","last_clean":"2024-04-25T09:56:38.451069+0000","last_became_active":"2024-04-25T09:56:38.447204+0000","last_became_peered":"2024-04-25T09:56:38.447204+0000","last_unstale":"2024-04-25T09:56:38.451069+0000","last_undegraded":"2024-04-25T09:56:38.451069+0000","last_fullsized":"2024-04-25T09:56:38.451069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:34.630001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436966+0000","last_change":"2024-04-25T09:56:38.436966+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436966+0000","last_undegraded":"2024-04-25T09:56:38.436966+0000","last_fullsized":"2024-04-25T09:56:38.436966+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.446681+0000","last_change":"2024-04-25T09:56:38.443664+0000","last_active":"2024-04-25T09:56:38.446681+0000","last_peered":"2024-04-25T09:56:38.446681+0000","last_clean":"2024-04-25T09:56:38.446681+0000","last_became_active":"2024-04-25T09:56:38.443253+0000","last_became_peered":"2024-04-25T09:56:38.443253+0000","last_unstale":"2024-04-25T09:56:38.446681+0000","last_undegraded":"2024-04-25T09:56:38.446681+0000","last_fullsized":"2024-04-25T09:56:38.446681+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:49.974254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":255,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.435835+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:56:38.435835+0000","last_peered":"2024-04-25T09:56:38.435835+0000","last_clean":"2024-04-25T09:56:38.435835+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:56:38.435835+0000","last_undegraded":"2024-04-25T09:56:38.435835+0000","last_fullsized":"2024-04-25T09:56:38.435835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36652,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335188,"statfs":{"total":96636764160,"available":96599232512,"internally_reserved":0,"allocated":1343488,"data_stored":927508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36076,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335764,"statfs":{"total":96636764160,"available":96599822336,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36076,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335764,"statfs":{"total":96636764160,"available":96599822336,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":737280,"data_stored":330481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36080,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335760,"statfs":{"total":96636764160,"available":96599818240,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":737280,"data_stored":330481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36644,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335196,"statfs":{"total":96636764160,"available":96599240704,"internally_reserved":0,"allocated":1331200,"data_stored":920849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:56:42.801 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:56:42.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.936+0000 7fd3cd48f640 1 Processor -- start 2024-04-25T09:56:42.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.936+0000 7fd3cd48f640 1 -- start start 2024-04-25T09:56:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.936+0000 7fd3cd48f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3c815dc00 0x7fd3c80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.936+0000 7fd3cd48f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd3c80d1dd0 0x7fd3c80ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.936+0000 7fd3cd48f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c815dfe0 con 0x7fd3c80ceb20 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.936+0000 7fd3cd48f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c80d00c0 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.936+0000 7fd3cd48f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c80d03a0 con 0x7fd3c815dc00 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.937+0000 7fd3c77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd3c80d1dd0 0x7fd3c80ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.937+0000 7fd3c67fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd3c80ceb20 0x7fd3c80d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36618/0 (socket says 172.21.15.19:36618) 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.937+0000 7fd3c77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd3c80d1dd0 0x7fd3c80ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53266/0 (socket says 172.21.15.19:53266) 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.937+0000 7fd3c67fc640 1 -- 172.21.15.19:0/2349857358 learned_addr learned my addr 172.21.15.19:0/2349857358 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.937+0000 7fd3c77fe640 1 -- 172.21.15.19:0/2349857358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3c815dc00 msgr2=0x7fd3c80c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.937+0000 7fd3c6ffd640 1 --2- 172.21.15.19:0/2349857358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3c815dc00 0x7fd3c80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.937+0000 7fd3c5ffb640 1 -- 172.21.15.19:0/2349857358 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435733832 0 0) 0x7fd3b0001110 con 0x7fd3c80ceb20 2024-04-25T09:56:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.937+0000 7fd3c77fe640 1 --2- 172.21.15.19:0/2349857358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3c815dc00 0x7fd3c80c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.937+0000 7fd3c77fe640 1 -- 172.21.15.19:0/2349857358 >> v1:172.21.15.19:6789/0 conn(0x7fd3c80ceb20 legacy=0x7fd3c80d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.937+0000 7fd3c77fe640 1 -- 172.21.15.19:0/2349857358 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c80cc440 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.937+0000 7fd3c6ffd640 1 --2- 172.21.15.19:0/2349857358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3c815dc00 0x7fd3c80c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3c77fe640 1 --2- 172.21.15.19:0/2349857358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c80d1dd0 0x7fd3c80ccc40 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009d90 tx=0x7fd3b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fab9773d61c8272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3c5ffb640 1 -- 172.21.15.19:0/2349857358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8007740 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3c5ffb640 1 -- 172.21.15.19:0/2349857358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b80078e0 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3c5ffb640 1 -- 172.21.15.19:0/2349857358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8007cb0 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3cd48f640 1 -- 172.21.15.19:0/2349857358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c80d1dd0 msgr2=0x7fd3c80ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3cd48f640 1 --2- 172.21.15.19:0/2349857358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c80d1dd0 0x7fd3c80ccc40 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009d90 tx=0x7fd3b802a660 comp rx=0 tx=0).stop 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3cd48f640 1 -- 172.21.15.19:0/2349857358 shutdown_connections 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3cd48f640 1 --2- 172.21.15.19:0/2349857358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c80d1dd0 0x7fd3c80ccc40 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3cd48f640 1 --2- 172.21.15.19:0/2349857358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3c815dc00 0x7fd3c80c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3cd48f640 1 -- 172.21.15.19:0/2349857358 >> 172.21.15.19:0/2349857358 conn(0x7fd3c806f5d0 msgr2=0x7fd3c806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3cd48f640 1 -- 172.21.15.19:0/2349857358 shutdown_connections 2024-04-25T09:56:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.938+0000 7fd3cd48f640 1 -- 172.21.15.19:0/2349857358 wait complete. 2024-04-25T09:56:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.940+0000 7fd3cd48f640 1 Processor -- start 2024-04-25T09:56:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.940+0000 7fd3cd48f640 1 -- start start 2024-04-25T09:56:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.940+0000 7fd3cd48f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c80ceb20 0x7fd3c80b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.940+0000 7fd3cd48f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c80d1dd0 0x7fd3c81532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.940+0000 7fd3cd48f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3c815dc00 0x7fd3c8158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.940+0000 7fd3cd48f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c80cfd90 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.940+0000 7fd3cd48f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c80cc130 con 0x7fd3c80ceb20 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.940+0000 7fd3cd48f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c80cdd70 con 0x7fd3c815dc00 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c80d1dd0 0x7fd3c81532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c80d1dd0 0x7fd3c81532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53268/0 (socket says 172.21.15.19:53268) 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c67fc640 1 -- 172.21.15.19:0/1653304083 learned_addr learned my addr 172.21.15.19:0/1653304083 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c77fe640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3c815dc00 0x7fd3c8158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c6ffd640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c80ceb20 0x7fd3c80b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c67fc640 1 -- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3c815dc00 msgr2=0x7fd3c8158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c67fc640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3c815dc00 0x7fd3c8158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c67fc640 1 -- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c80ceb20 msgr2=0x7fd3c80b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c67fc640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c80ceb20 0x7fd3c80b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c67fc640 1 -- 172.21.15.19:0/1653304083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c8153830 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c6ffd640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c80ceb20 0x7fd3c80b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3c67fc640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c80d1dd0 0x7fd3c81532f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000c780 tx=0x7fd3b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3bf7fe640 1 -- 172.21.15.19:0/1653304083 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0018070 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3bf7fe640 1 -- 172.21.15.19:0/1653304083 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b00072d0 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3cd48f640 1 -- 172.21.15.19:0/1653304083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c8159070 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3bf7fe640 1 -- 172.21.15.19:0/1653304083 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b00047e0 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.941+0000 7fd3cd48f640 1 -- 172.21.15.19:0/1653304083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c8159630 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.942+0000 7fd3cd48f640 1 -- 172.21.15.19:0/1653304083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c80c66c0 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.943+0000 7fd3bf7fe640 1 -- 172.21.15.19:0/1653304083 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3b0004980 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.943+0000 7fd3bf7fe640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a806b4d0 0x7fd3a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:42.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.943+0000 7fd3bf7fe640 1 -- 172.21.15.19:0/1653304083 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd3b0089740 con 0x7fd3c80d1dd0 2024-04-25T09:56:42.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.943+0000 7fd3c6ffd640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a806b4d0 0x7fd3a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:42.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.944+0000 7fd3c6ffd640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a806b4d0 0x7fd3a806d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000a990 tx=0x7fd3c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:42.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:42.947+0000 7fd3bf7fe640 1 -- 172.21.15.19:0/1653304083 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3c815dfe0 con 0x7fd3c80d1dd0 2024-04-25T09:56:43.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.123+0000 7fd3cd48f640 1 -- 172.21.15.19:0/1653304083 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd3c8158d60 con 0x7fd3a806b4d0 2024-04-25T09:56:43.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.130+0000 7fd3bf7fe640 1 -- 172.21.15.19:0/1653304083 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246700 (secure 0 0 0) 0x7fd3c8158d60 con 0x7fd3a806b4d0 2024-04-25T09:56:43.136 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:56:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.132+0000 7fd3cd48f640 1 -- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a806b4d0 msgr2=0x7fd3a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.132+0000 7fd3cd48f640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a806b4d0 0x7fd3a806d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000a990 tx=0x7fd3c0008040 comp rx=0 tx=0).stop 2024-04-25T09:56:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.132+0000 7fd3cd48f640 1 -- 172.21.15.19:0/1653304083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c80d1dd0 msgr2=0x7fd3c81532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.132+0000 7fd3cd48f640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c80d1dd0 0x7fd3c81532f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000c780 tx=0x7fd3b000cc40 comp rx=0 tx=0).stop 2024-04-25T09:56:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.132+0000 7fd3cd48f640 1 -- 172.21.15.19:0/1653304083 shutdown_connections 2024-04-25T09:56:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.132+0000 7fd3cd48f640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a806b4d0 0x7fd3a806d990 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.132+0000 7fd3cd48f640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3c815dc00 0x7fd3c8158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.132+0000 7fd3cd48f640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c80d1dd0 0x7fd3c81532f0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.132+0000 7fd3cd48f640 1 --2- 172.21.15.19:0/1653304083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c80ceb20 0x7fd3c80b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.132+0000 7fd3cd48f640 1 -- 172.21.15.19:0/1653304083 >> 172.21.15.19:0/1653304083 conn(0x7fd3c806f5d0 msgr2=0x7fd3c80b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.133+0000 7fd3cd48f640 1 -- 172.21.15.19:0/1653304083 shutdown_connections 2024-04-25T09:56:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.133+0000 7fd3cd48f640 1 -- 172.21.15.19:0/1653304083 wait complete. 2024-04-25T09:56:43.147 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-04-25T09:56:42.961633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289720,"kb_used_data":6980,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685000,"statfs":{"total":773094113280,"available":772797440000,"internally_reserved":0,"allocated":7147520,"data_stored":3857879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.534188"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437683+0000","last_change":"2024-04-25T09:56:38.437683+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437683+0000","last_undegraded":"2024-04-25T09:56:38.437683+0000","last_fullsized":"2024-04-25T09:56:38.437683+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436642+0000","last_change":"2024-04-25T09:56:38.436642+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436642+0000","last_undegraded":"2024-04-25T09:56:38.436642+0000","last_fullsized":"2024-04-25T09:56:38.436642+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433788+0000","last_change":"2024-04-25T09:56:38.433788+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433788+0000","last_undegraded":"2024-04-25T09:56:38.433788+0000","last_fullsized":"2024-04-25T09:56:38.433788+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431818+0000","last_change":"2024-04-25T09:56:38.431818+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431818+0000","last_undegraded":"2024-04-25T09:56:38.431818+0000","last_fullsized":"2024-04-25T09:56:38.431818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437599+0000","last_change":"2024-04-25T09:56:38.437599+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437599+0000","last_undegraded":"2024-04-25T09:56:38.437599+0000","last_fullsized":"2024-04-25T09:56:38.437599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436685+0000","last_change":"2024-04-25T09:56:38.436685+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436685+0000","last_undegraded":"2024-04-25T09:56:38.436685+0000","last_fullsized":"2024-04-25T09:56:38.436685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439909+0000","last_change":"2024-04-25T09:56:38.439909+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439909+0000","last_undegraded":"2024-04-25T09:56:38.439909+0000","last_fullsized":"2024-04-25T09:56:38.439909+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438373+0000","last_change":"2024-04-25T09:56:38.438373+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438373+0000","last_undegraded":"2024-04-25T09:56:38.438373+0000","last_fullsized":"2024-04-25T09:56:38.438373+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433700+0000","last_change":"2024-04-25T09:56:38.433700+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433700+0000","last_undegraded":"2024-04-25T09:56:38.433700+0000","last_fullsized":"2024-04-25T09:56:38.433700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435532+0000","last_change":"2024-04-25T09:56:38.435532+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435532+0000","last_undegraded":"2024-04-25T09:56:38.435532+0000","last_fullsized":"2024-04-25T09:56:38.435532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435197+0000","last_change":"2024-04-25T09:56:38.435197+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435197+0000","last_undegraded":"2024-04-25T09:56:38.435197+0000","last_fullsized":"2024-04-25T09:56:38.435197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434689+0000","last_change":"2024-04-25T09:56:38.434689+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434689+0000","last_undegraded":"2024-04-25T09:56:38.434689+0000","last_fullsized":"2024-04-25T09:56:38.434689+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436051+0000","last_change":"2024-04-25T09:56:38.436051+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436051+0000","last_undegraded":"2024-04-25T09:56:38.436051+0000","last_fullsized":"2024-04-25T09:56:38.436051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433377+0000","last_change":"2024-04-25T09:56:38.433377+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433377+0000","last_undegraded":"2024-04-25T09:56:38.433377+0000","last_fullsized":"2024-04-25T09:56:38.433377+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433505+0000","last_change":"2024-04-25T09:56:38.433505+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433505+0000","last_undegraded":"2024-04-25T09:56:38.433505+0000","last_fullsized":"2024-04-25T09:56:38.433505+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439675+0000","last_change":"2024-04-25T09:56:38.439675+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439675+0000","last_undegraded":"2024-04-25T09:56:38.439675+0000","last_fullsized":"2024-04-25T09:56:38.439675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434662+0000","last_change":"2024-04-25T09:56:38.434662+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434662+0000","last_undegraded":"2024-04-25T09:56:38.434662+0000","last_fullsized":"2024-04-25T09:56:38.434662+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435818+0000","last_change":"2024-04-25T09:56:38.435818+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435818+0000","last_undegraded":"2024-04-25T09:56:38.435818+0000","last_fullsized":"2024-04-25T09:56:38.435818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437438+0000","last_change":"2024-04-25T09:56:38.437438+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437438+0000","last_undegraded":"2024-04-25T09:56:38.437438+0000","last_fullsized":"2024-04-25T09:56:38.437438+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434623+0000","last_change":"2024-04-25T09:56:38.434623+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434623+0000","last_undegraded":"2024-04-25T09:56:38.434623+0000","last_fullsized":"2024-04-25T09:56:38.434623+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434027+0000","last_change":"2024-04-25T09:56:38.434027+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434027+0000","last_undegraded":"2024-04-25T09:56:38.434027+0000","last_fullsized":"2024-04-25T09:56:38.434027+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436029+0000","last_change":"2024-04-25T09:56:38.436029+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436029+0000","last_undegraded":"2024-04-25T09:56:38.436029+0000","last_fullsized":"2024-04-25T09:56:38.436029+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434596+0000","last_change":"2024-04-25T09:56:38.434596+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434596+0000","last_undegraded":"2024-04-25T09:56:38.434596+0000","last_fullsized":"2024-04-25T09:56:38.434596+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433635+0000","last_change":"2024-04-25T09:56:38.433635+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433635+0000","last_undegraded":"2024-04-25T09:56:38.433635+0000","last_fullsized":"2024-04-25T09:56:38.433635+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437925+0000","last_change":"2024-04-25T09:56:38.437925+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437925+0000","last_undegraded":"2024-04-25T09:56:38.437925+0000","last_fullsized":"2024-04-25T09:56:38.437925+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434898+0000","last_change":"2024-04-25T09:56:38.434898+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434898+0000","last_undegraded":"2024-04-25T09:56:38.434898+0000","last_fullsized":"2024-04-25T09:56:38.434898+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434806+0000","last_change":"2024-04-25T09:56:38.434806+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434806+0000","last_undegraded":"2024-04-25T09:56:38.434806+0000","last_fullsized":"2024-04-25T09:56:38.434806+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434169+0000","last_change":"2024-04-25T09:56:38.434169+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434169+0000","last_undegraded":"2024-04-25T09:56:38.434169+0000","last_fullsized":"2024-04-25T09:56:38.434169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433478+0000","last_change":"2024-04-25T09:56:38.433478+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433478+0000","last_undegraded":"2024-04-25T09:56:38.433478+0000","last_fullsized":"2024-04-25T09:56:38.433478+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435681+0000","last_change":"2024-04-25T09:56:38.435681+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435681+0000","last_undegraded":"2024-04-25T09:56:38.435681+0000","last_fullsized":"2024-04-25T09:56:38.435681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431678+0000","last_change":"2024-04-25T09:56:38.431678+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431678+0000","last_undegraded":"2024-04-25T09:56:38.431678+0000","last_fullsized":"2024-04-25T09:56:38.431678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434572+0000","last_change":"2024-04-25T09:56:38.434572+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434572+0000","last_undegraded":"2024-04-25T09:56:38.434572+0000","last_fullsized":"2024-04-25T09:56:38.434572+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434920+0000","last_change":"2024-04-25T09:56:38.434920+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434920+0000","last_undegraded":"2024-04-25T09:56:38.434920+0000","last_fullsized":"2024-04-25T09:56:38.434920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.451222+0000","last_change":"2024-04-25T09:56:38.448802+0000","last_active":"2024-04-25T09:56:38.451222+0000","last_peered":"2024-04-25T09:56:38.451222+0000","last_clean":"2024-04-25T09:56:38.451222+0000","last_became_active":"2024-04-25T09:56:38.447596+0000","last_became_peered":"2024-04-25T09:56:38.447596+0000","last_unstale":"2024-04-25T09:56:38.451222+0000","last_undegraded":"2024-04-25T09:56:38.451222+0000","last_fullsized":"2024-04-25T09:56:38.451222+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:56.119011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.432113+0000","last_change":"2024-04-25T09:56:38.432113+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.432113+0000","last_undegraded":"2024-04-25T09:56:38.432113+0000","last_fullsized":"2024-04-25T09:56:38.432113+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450513+0000","last_change":"2024-04-25T09:56:38.446203+0000","last_active":"2024-04-25T09:56:38.450513+0000","last_peered":"2024-04-25T09:56:38.450513+0000","last_clean":"2024-04-25T09:56:38.450513+0000","last_became_active":"2024-04-25T09:56:38.445931+0000","last_became_peered":"2024-04-25T09:56:38.445931+0000","last_unstale":"2024-04-25T09:56:38.450513+0000","last_undegraded":"2024-04-25T09:56:38.450513+0000","last_fullsized":"2024-04-25T09:56:38.450513+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:48.759756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434485+0000","last_change":"2024-04-25T09:56:38.434485+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434485+0000","last_undegraded":"2024-04-25T09:56:38.434485+0000","last_fullsized":"2024-04-25T09:56:38.434485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449712+0000","last_change":"2024-04-25T09:56:38.442763+0000","last_active":"2024-04-25T09:56:38.449712+0000","last_peered":"2024-04-25T09:56:38.449712+0000","last_clean":"2024-04-25T09:56:38.449712+0000","last_became_active":"2024-04-25T09:56:38.442494+0000","last_became_peered":"2024-04-25T09:56:38.442494+0000","last_unstale":"2024-04-25T09:56:38.449712+0000","last_undegraded":"2024-04-25T09:56:38.449712+0000","last_fullsized":"2024-04-25T09:56:38.449712+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.557978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434490+0000","last_change":"2024-04-25T09:56:38.434490+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434490+0000","last_undegraded":"2024-04-25T09:56:38.434490+0000","last_fullsized":"2024-04-25T09:56:38.434490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443747+0000","last_change":"2024-04-25T09:56:38.441504+0000","last_active":"2024-04-25T09:56:38.443747+0000","last_peered":"2024-04-25T09:56:38.443747+0000","last_clean":"2024-04-25T09:56:38.443747+0000","last_became_active":"2024-04-25T09:56:38.441178+0000","last_became_peered":"2024-04-25T09:56:38.441178+0000","last_unstale":"2024-04-25T09:56:38.443747+0000","last_undegraded":"2024-04-25T09:56:38.443747+0000","last_fullsized":"2024-04-25T09:56:38.443747+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:49.974254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439169+0000","last_change":"2024-04-25T09:56:38.439169+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439169+0000","last_undegraded":"2024-04-25T09:56:38.439169+0000","last_fullsized":"2024-04-25T09:56:38.439169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443433+0000","last_change":"2024-04-25T09:56:38.442701+0000","last_active":"2024-04-25T09:56:38.443433+0000","last_peered":"2024-04-25T09:56:38.443433+0000","last_clean":"2024-04-25T09:56:38.443433+0000","last_became_active":"2024-04-25T09:56:38.442433+0000","last_became_peered":"2024-04-25T09:56:38.442433+0000","last_unstale":"2024-04-25T09:56:38.443433+0000","last_undegraded":"2024-04-25T09:56:38.443433+0000","last_fullsized":"2024-04-25T09:56:38.443433+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:55.484379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434138+0000","last_change":"2024-04-25T09:56:38.434138+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434138+0000","last_undegraded":"2024-04-25T09:56:38.434138+0000","last_fullsized":"2024-04-25T09:56:38.434138+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448317+0000","last_change":"2024-04-25T09:56:38.446402+0000","last_active":"2024-04-25T09:56:38.448317+0000","last_peered":"2024-04-25T09:56:38.448317+0000","last_clean":"2024-04-25T09:56:38.448317+0000","last_became_active":"2024-04-25T09:56:38.446088+0000","last_became_peered":"2024-04-25T09:56:38.446088+0000","last_unstale":"2024-04-25T09:56:38.448317+0000","last_undegraded":"2024-04-25T09:56:38.448317+0000","last_fullsized":"2024-04-25T09:56:38.448317+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.933751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438728+0000","last_change":"2024-04-25T09:56:38.438728+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438728+0000","last_undegraded":"2024-04-25T09:56:38.438728+0000","last_fullsized":"2024-04-25T09:56:38.438728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.447423+0000","last_change":"2024-04-25T09:56:38.445015+0000","last_active":"2024-04-25T09:56:38.447423+0000","last_peered":"2024-04-25T09:56:38.447423+0000","last_clean":"2024-04-25T09:56:38.447423+0000","last_became_active":"2024-04-25T09:56:38.444713+0000","last_became_peered":"2024-04-25T09:56:38.444713+0000","last_unstale":"2024-04-25T09:56:38.447423+0000","last_undegraded":"2024-04-25T09:56:38.447423+0000","last_fullsized":"2024-04-25T09:56:38.447423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435118+0000","last_change":"2024-04-25T09:56:38.435118+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435118+0000","last_undegraded":"2024-04-25T09:56:38.435118+0000","last_fullsized":"2024-04-25T09:56:38.435118+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448373+0000","last_change":"2024-04-25T09:56:38.446581+0000","last_active":"2024-04-25T09:56:38.448373+0000","last_peered":"2024-04-25T09:56:38.448373+0000","last_clean":"2024-04-25T09:56:38.448373+0000","last_became_active":"2024-04-25T09:56:38.444721+0000","last_became_peered":"2024-04-25T09:56:38.444721+0000","last_unstale":"2024-04-25T09:56:38.448373+0000","last_undegraded":"2024-04-25T09:56:38.448373+0000","last_fullsized":"2024-04-25T09:56:38.448373+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:40.119587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433629+0000","last_change":"2024-04-25T09:56:38.433629+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433629+0000","last_undegraded":"2024-04-25T09:56:38.433629+0000","last_fullsized":"2024-04-25T09:56:38.433629+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454111+0000","last_change":"2024-04-25T09:56:38.452304+0000","last_active":"2024-04-25T09:56:38.454111+0000","last_peered":"2024-04-25T09:56:38.454111+0000","last_clean":"2024-04-25T09:56:38.454111+0000","last_became_active":"2024-04-25T09:56:38.452001+0000","last_became_peered":"2024-04-25T09:56:38.452001+0000","last_unstale":"2024-04-25T09:56:38.454111+0000","last_undegraded":"2024-04-25T09:56:38.454111+0000","last_fullsized":"2024-04-25T09:56:38.454111+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:07.043227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437792+0000","last_change":"2024-04-25T09:56:38.437792+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437792+0000","last_undegraded":"2024-04-25T09:56:38.437792+0000","last_fullsized":"2024-04-25T09:56:38.437792+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452779+0000","last_change":"2024-04-25T09:56:38.451554+0000","last_active":"2024-04-25T09:56:38.452779+0000","last_peered":"2024-04-25T09:56:38.452779+0000","last_clean":"2024-04-25T09:56:38.452779+0000","last_became_active":"2024-04-25T09:56:38.451253+0000","last_became_peered":"2024-04-25T09:56:38.451253+0000","last_unstale":"2024-04-25T09:56:38.452779+0000","last_undegraded":"2024-04-25T09:56:38.452779+0000","last_fullsized":"2024-04-25T09:56:38.452779+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:44.723554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438457+0000","last_change":"2024-04-25T09:56:38.438457+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438457+0000","last_undegraded":"2024-04-25T09:56:38.438457+0000","last_fullsized":"2024-04-25T09:56:38.438457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454400+0000","last_change":"2024-04-25T09:56:38.452908+0000","last_active":"2024-04-25T09:56:38.454400+0000","last_peered":"2024-04-25T09:56:38.454400+0000","last_clean":"2024-04-25T09:56:38.454400+0000","last_became_active":"2024-04-25T09:56:38.452672+0000","last_became_peered":"2024-04-25T09:56:38.452672+0000","last_unstale":"2024-04-25T09:56:38.454400+0000","last_undegraded":"2024-04-25T09:56:38.454400+0000","last_fullsized":"2024-04-25T09:56:38.454400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.108179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431918+0000","last_change":"2024-04-25T09:56:38.431918+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431918+0000","last_undegraded":"2024-04-25T09:56:38.431918+0000","last_fullsized":"2024-04-25T09:56:38.431918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450753+0000","last_change":"2024-04-25T09:56:38.441289+0000","last_active":"2024-04-25T09:56:38.450753+0000","last_peered":"2024-04-25T09:56:38.450753+0000","last_clean":"2024-04-25T09:56:38.450753+0000","last_became_active":"2024-04-25T09:56:38.440974+0000","last_became_peered":"2024-04-25T09:56:38.440974+0000","last_unstale":"2024-04-25T09:56:38.450753+0000","last_undegraded":"2024-04-25T09:56:38.450753+0000","last_fullsized":"2024-04-25T09:56:38.450753+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:06.200902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431725+0000","last_change":"2024-04-25T09:56:38.431725+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431725+0000","last_undegraded":"2024-04-25T09:56:38.431725+0000","last_fullsized":"2024-04-25T09:56:38.431725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.444935+0000","last_change":"2024-04-25T09:56:38.444152+0000","last_active":"2024-04-25T09:56:38.444935+0000","last_peered":"2024-04-25T09:56:38.444935+0000","last_clean":"2024-04-25T09:56:38.444935+0000","last_became_active":"2024-04-25T09:56:38.443891+0000","last_became_peered":"2024-04-25T09:56:38.443891+0000","last_unstale":"2024-04-25T09:56:38.444935+0000","last_undegraded":"2024-04-25T09:56:38.444935+0000","last_fullsized":"2024-04-25T09:56:38.444935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.635113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436850+0000","last_change":"2024-04-25T09:56:38.436850+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436850+0000","last_undegraded":"2024-04-25T09:56:38.436850+0000","last_fullsized":"2024-04-25T09:56:38.436850+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454025+0000","last_change":"2024-04-25T09:56:38.452182+0000","last_active":"2024-04-25T09:56:38.454025+0000","last_peered":"2024-04-25T09:56:38.454025+0000","last_clean":"2024-04-25T09:56:38.454025+0000","last_became_active":"2024-04-25T09:56:38.451492+0000","last_became_peered":"2024-04-25T09:56:38.451492+0000","last_unstale":"2024-04-25T09:56:38.454025+0000","last_undegraded":"2024-04-25T09:56:38.454025+0000","last_fullsized":"2024-04-25T09:56:38.454025+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:33.806055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438322+0000","last_change":"2024-04-25T09:56:38.438322+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438322+0000","last_undegraded":"2024-04-25T09:56:38.438322+0000","last_fullsized":"2024-04-25T09:56:38.438322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452814+0000","last_change":"2024-04-25T09:56:38.449599+0000","last_active":"2024-04-25T09:56:38.452814+0000","last_peered":"2024-04-25T09:56:38.452814+0000","last_clean":"2024-04-25T09:56:38.452814+0000","last_became_active":"2024-04-25T09:56:38.449302+0000","last_became_peered":"2024-04-25T09:56:38.449302+0000","last_unstale":"2024-04-25T09:56:38.452814+0000","last_undegraded":"2024-04-25T09:56:38.452814+0000","last_fullsized":"2024-04-25T09:56:38.452814+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.404639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433556+0000","last_change":"2024-04-25T09:56:38.433556+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433556+0000","last_undegraded":"2024-04-25T09:56:38.433556+0000","last_fullsized":"2024-04-25T09:56:38.433556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448561+0000","last_change":"2024-04-25T09:56:38.447935+0000","last_active":"2024-04-25T09:56:38.448561+0000","last_peered":"2024-04-25T09:56:38.448561+0000","last_clean":"2024-04-25T09:56:38.448561+0000","last_became_active":"2024-04-25T09:56:38.447648+0000","last_became_peered":"2024-04-25T09:56:38.447648+0000","last_unstale":"2024-04-25T09:56:38.448561+0000","last_undegraded":"2024-04-25T09:56:38.448561+0000","last_fullsized":"2024-04-25T09:56:38.448561+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:32.556063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433134+0000","last_change":"2024-04-25T09:56:38.433134+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433134+0000","last_undegraded":"2024-04-25T09:56:38.433134+0000","last_fullsized":"2024-04-25T09:56:38.433134+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452679+0000","last_change":"2024-04-25T09:56:38.451735+0000","last_active":"2024-04-25T09:56:38.452679+0000","last_peered":"2024-04-25T09:56:38.452679+0000","last_clean":"2024-04-25T09:56:38.452679+0000","last_became_active":"2024-04-25T09:56:38.451108+0000","last_became_peered":"2024-04-25T09:56:38.451108+0000","last_unstale":"2024-04-25T09:56:38.452679+0000","last_undegraded":"2024-04-25T09:56:38.452679+0000","last_fullsized":"2024-04-25T09:56:38.452679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:34.259055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436880+0000","last_change":"2024-04-25T09:56:38.436880+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436880+0000","last_undegraded":"2024-04-25T09:56:38.436880+0000","last_fullsized":"2024-04-25T09:56:38.436880+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452467+0000","last_change":"2024-04-25T09:56:38.449467+0000","last_active":"2024-04-25T09:56:38.452467+0000","last_peered":"2024-04-25T09:56:38.452467+0000","last_clean":"2024-04-25T09:56:38.452467+0000","last_became_active":"2024-04-25T09:56:38.449110+0000","last_became_peered":"2024-04-25T09:56:38.449110+0000","last_unstale":"2024-04-25T09:56:38.452467+0000","last_undegraded":"2024-04-25T09:56:38.452467+0000","last_fullsized":"2024-04-25T09:56:38.452467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:34.259055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435023+0000","last_change":"2024-04-25T09:56:38.435023+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435023+0000","last_undegraded":"2024-04-25T09:56:38.435023+0000","last_fullsized":"2024-04-25T09:56:38.435023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454387+0000","last_change":"2024-04-25T09:56:38.452721+0000","last_active":"2024-04-25T09:56:38.454387+0000","last_peered":"2024-04-25T09:56:38.454387+0000","last_clean":"2024-04-25T09:56:38.454387+0000","last_became_active":"2024-04-25T09:56:38.452370+0000","last_became_peered":"2024-04-25T09:56:38.452370+0000","last_unstale":"2024-04-25T09:56:38.454387+0000","last_undegraded":"2024-04-25T09:56:38.454387+0000","last_fullsized":"2024-04-25T09:56:38.454387+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:59.539232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437996+0000","last_change":"2024-04-25T09:56:38.437996+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437996+0000","last_undegraded":"2024-04-25T09:56:38.437996+0000","last_fullsized":"2024-04-25T09:56:38.437996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.441954+0000","last_change":"2024-04-25T09:56:38.441046+0000","last_active":"2024-04-25T09:56:38.441954+0000","last_peered":"2024-04-25T09:56:38.441954+0000","last_clean":"2024-04-25T09:56:38.441954+0000","last_became_active":"2024-04-25T09:56:38.440736+0000","last_became_peered":"2024-04-25T09:56:38.440736+0000","last_unstale":"2024-04-25T09:56:38.441954+0000","last_undegraded":"2024-04-25T09:56:38.441954+0000","last_fullsized":"2024-04-25T09:56:38.441954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:10.273708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434042+0000","last_change":"2024-04-25T09:56:38.434042+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434042+0000","last_undegraded":"2024-04-25T09:56:38.434042+0000","last_fullsized":"2024-04-25T09:56:38.434042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450475+0000","last_change":"2024-04-25T09:56:38.443577+0000","last_active":"2024-04-25T09:56:38.450475+0000","last_peered":"2024-04-25T09:56:38.450475+0000","last_clean":"2024-04-25T09:56:38.450475+0000","last_became_active":"2024-04-25T09:56:38.443304+0000","last_became_peered":"2024-04-25T09:56:38.443304+0000","last_unstale":"2024-04-25T09:56:38.450475+0000","last_undegraded":"2024-04-25T09:56:38.450475+0000","last_fullsized":"2024-04-25T09:56:38.450475+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.933751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436494+0000","last_change":"2024-04-25T09:56:38.436494+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436494+0000","last_undegraded":"2024-04-25T09:56:38.436494+0000","last_fullsized":"2024-04-25T09:56:38.436494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454005+0000","last_change":"2024-04-25T09:56:38.451937+0000","last_active":"2024-04-25T09:56:38.454005+0000","last_peered":"2024-04-25T09:56:38.454005+0000","last_clean":"2024-04-25T09:56:38.454005+0000","last_became_active":"2024-04-25T09:56:38.451248+0000","last_became_peered":"2024-04-25T09:56:38.451248+0000","last_unstale":"2024-04-25T09:56:38.454005+0000","last_undegraded":"2024-04-25T09:56:38.454005+0000","last_fullsized":"2024-04-25T09:56:38.454005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.404639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435576+0000","last_change":"2024-04-25T09:56:38.435576+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435576+0000","last_undegraded":"2024-04-25T09:56:38.435576+0000","last_fullsized":"2024-04-25T09:56:38.435576+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450527+0000","last_change":"2024-04-25T09:56:38.446452+0000","last_active":"2024-04-25T09:56:38.450527+0000","last_peered":"2024-04-25T09:56:38.450527+0000","last_clean":"2024-04-25T09:56:38.450527+0000","last_became_active":"2024-04-25T09:56:38.446166+0000","last_became_peered":"2024-04-25T09:56:38.446166+0000","last_unstale":"2024-04-25T09:56:38.450527+0000","last_undegraded":"2024-04-25T09:56:38.450527+0000","last_fullsized":"2024-04-25T09:56:38.450527+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:10.912571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435303+0000","last_change":"2024-04-25T09:56:38.435303+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435303+0000","last_undegraded":"2024-04-25T09:56:38.435303+0000","last_fullsized":"2024-04-25T09:56:38.435303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448987+0000","last_change":"2024-04-25T09:56:38.447857+0000","last_active":"2024-04-25T09:56:38.448987+0000","last_peered":"2024-04-25T09:56:38.448987+0000","last_clean":"2024-04-25T09:56:38.448987+0000","last_became_active":"2024-04-25T09:56:38.447519+0000","last_became_peered":"2024-04-25T09:56:38.447519+0000","last_unstale":"2024-04-25T09:56:38.448987+0000","last_undegraded":"2024-04-25T09:56:38.448987+0000","last_fullsized":"2024-04-25T09:56:38.448987+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:32.728995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.443451+0000","last_change":"2024-04-25T09:56:38.443451+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.443451+0000","last_undegraded":"2024-04-25T09:56:38.443451+0000","last_fullsized":"2024-04-25T09:56:38.443451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449375+0000","last_change":"2024-04-25T09:56:38.448408+0000","last_active":"2024-04-25T09:56:38.449375+0000","last_peered":"2024-04-25T09:56:38.449375+0000","last_clean":"2024-04-25T09:56:38.449375+0000","last_became_active":"2024-04-25T09:56:38.448024+0000","last_became_peered":"2024-04-25T09:56:38.448024+0000","last_unstale":"2024-04-25T09:56:38.449375+0000","last_undegraded":"2024-04-25T09:56:38.449375+0000","last_fullsized":"2024-04-25T09:56:38.449375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.635113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434466+0000","last_change":"2024-04-25T09:56:38.434466+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434466+0000","last_undegraded":"2024-04-25T09:56:38.434466+0000","last_fullsized":"2024-04-25T09:56:38.434466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449665+0000","last_change":"2024-04-25T09:56:38.442585+0000","last_active":"2024-04-25T09:56:38.449665+0000","last_peered":"2024-04-25T09:56:38.449665+0000","last_clean":"2024-04-25T09:56:38.449665+0000","last_became_active":"2024-04-25T09:56:38.442310+0000","last_became_peered":"2024-04-25T09:56:38.442310+0000","last_unstale":"2024-04-25T09:56:38.449665+0000","last_undegraded":"2024-04-25T09:56:38.449665+0000","last_fullsized":"2024-04-25T09:56:38.449665+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437364+0000","last_change":"2024-04-25T09:56:38.437364+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437364+0000","last_undegraded":"2024-04-25T09:56:38.437364+0000","last_fullsized":"2024-04-25T09:56:38.437364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450830+0000","last_change":"2024-04-25T09:56:38.442079+0000","last_active":"2024-04-25T09:56:38.450830+0000","last_peered":"2024-04-25T09:56:38.450830+0000","last_clean":"2024-04-25T09:56:38.450830+0000","last_became_active":"2024-04-25T09:56:38.441787+0000","last_became_peered":"2024-04-25T09:56:38.441787+0000","last_unstale":"2024-04-25T09:56:38.450830+0000","last_undegraded":"2024-04-25T09:56:38.450830+0000","last_fullsized":"2024-04-25T09:56:38.450830+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.776063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436133+0000","last_change":"2024-04-25T09:56:38.436133+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436133+0000","last_undegraded":"2024-04-25T09:56:38.436133+0000","last_fullsized":"2024-04-25T09:56:38.436133+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452759+0000","last_change":"2024-04-25T09:56:38.450312+0000","last_active":"2024-04-25T09:56:38.452759+0000","last_peered":"2024-04-25T09:56:38.452759+0000","last_clean":"2024-04-25T09:56:38.452759+0000","last_became_active":"2024-04-25T09:56:38.450049+0000","last_became_peered":"2024-04-25T09:56:38.450049+0000","last_unstale":"2024-04-25T09:56:38.452759+0000","last_undegraded":"2024-04-25T09:56:38.452759+0000","last_fullsized":"2024-04-25T09:56:38.452759+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:33.806055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431039+0000","last_change":"2024-04-25T09:56:38.431039+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431039+0000","last_undegraded":"2024-04-25T09:56:38.431039+0000","last_fullsized":"2024-04-25T09:56:38.431039+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443404+0000","last_change":"2024-04-25T09:56:38.441913+0000","last_active":"2024-04-25T09:56:38.443404+0000","last_peered":"2024-04-25T09:56:38.443404+0000","last_clean":"2024-04-25T09:56:38.443404+0000","last_became_active":"2024-04-25T09:56:38.441584+0000","last_became_peered":"2024-04-25T09:56:38.441584+0000","last_unstale":"2024-04-25T09:56:38.443404+0000","last_undegraded":"2024-04-25T09:56:38.443404+0000","last_fullsized":"2024-04-25T09:56:38.443404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:06.302131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433502+0000","last_change":"2024-04-25T09:56:38.433502+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433502+0000","last_undegraded":"2024-04-25T09:56:38.433502+0000","last_fullsized":"2024-04-25T09:56:38.433502+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.447445+0000","last_change":"2024-04-25T09:56:38.444951+0000","last_active":"2024-04-25T09:56:38.447445+0000","last_peered":"2024-04-25T09:56:38.447445+0000","last_clean":"2024-04-25T09:56:38.447445+0000","last_became_active":"2024-04-25T09:56:38.444617+0000","last_became_peered":"2024-04-25T09:56:38.444617+0000","last_unstale":"2024-04-25T09:56:38.447445+0000","last_undegraded":"2024-04-25T09:56:38.447445+0000","last_fullsized":"2024-04-25T09:56:38.447445+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.557978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.443248+0000","last_change":"2024-04-25T09:56:38.443248+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.443248+0000","last_undegraded":"2024-04-25T09:56:38.443248+0000","last_fullsized":"2024-04-25T09:56:38.443248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.451069+0000","last_change":"2024-04-25T09:56:38.447517+0000","last_active":"2024-04-25T09:56:38.451069+0000","last_peered":"2024-04-25T09:56:38.451069+0000","last_clean":"2024-04-25T09:56:38.451069+0000","last_became_active":"2024-04-25T09:56:38.447204+0000","last_became_peered":"2024-04-25T09:56:38.447204+0000","last_unstale":"2024-04-25T09:56:38.451069+0000","last_undegraded":"2024-04-25T09:56:38.451069+0000","last_fullsized":"2024-04-25T09:56:38.451069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:34.630001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436966+0000","last_change":"2024-04-25T09:56:38.436966+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436966+0000","last_undegraded":"2024-04-25T09:56:38.436966+0000","last_fullsized":"2024-04-25T09:56:38.436966+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.446681+0000","last_change":"2024-04-25T09:56:38.443664+0000","last_active":"2024-04-25T09:56:38.446681+0000","last_peered":"2024-04-25T09:56:38.446681+0000","last_clean":"2024-04-25T09:56:38.446681+0000","last_became_active":"2024-04-25T09:56:38.443253+0000","last_became_peered":"2024-04-25T09:56:38.443253+0000","last_unstale":"2024-04-25T09:56:38.446681+0000","last_undegraded":"2024-04-25T09:56:38.446681+0000","last_fullsized":"2024-04-25T09:56:38.446681+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:49.974254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":255,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.435835+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:56:38.435835+0000","last_peered":"2024-04-25T09:56:38.435835+0000","last_clean":"2024-04-25T09:56:38.435835+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:56:38.435835+0000","last_undegraded":"2024-04-25T09:56:38.435835+0000","last_fullsized":"2024-04-25T09:56:38.435835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36652,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335188,"statfs":{"total":96636764160,"available":96599232512,"internally_reserved":0,"allocated":1343488,"data_stored":927508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36076,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335764,"statfs":{"total":96636764160,"available":96599822336,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36076,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335764,"statfs":{"total":96636764160,"available":96599822336,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":737280,"data_stored":330481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36080,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335760,"statfs":{"total":96636764160,"available":96599818240,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":737280,"data_stored":330481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36644,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335196,"statfs":{"total":96636764160,"available":96599240704,"internally_reserved":0,"allocated":1331200,"data_stored":920849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:56:43.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2024-04-25T09:56:43.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.437683+0000', 'last_change': '2024-04-25T09:56:38.437683+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.437683+0000', 'last_undegraded': '2024-04-25T09:56:38.437683+0000', 'last_fullsized': '2024-04-25T09:56:38.437683+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:56:43.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2024-04-25T09:56:43.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.436642+0000', 'last_change': '2024-04-25T09:56:38.436642+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.436642+0000', 'last_undegraded': '2024-04-25T09:56:38.436642+0000', 'last_fullsized': '2024-04-25T09:56:38.436642+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:56:43.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2024-04-25T09:56:43.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.433788+0000', 'last_change': '2024-04-25T09:56:38.433788+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.433788+0000', 'last_undegraded': '2024-04-25T09:56:38.433788+0000', 'last_fullsized': '2024-04-25T09:56:38.433788+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:56:43.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2024-04-25T09:56:43.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.431818+0000', 'last_change': '2024-04-25T09:56:38.431818+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.431818+0000', 'last_undegraded': '2024-04-25T09:56:38.431818+0000', 'last_fullsized': '2024-04-25T09:56:38.431818+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:56:43.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2024-04-25T09:56:43.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.437599+0000', 'last_change': '2024-04-25T09:56:38.437599+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.437599+0000', 'last_undegraded': '2024-04-25T09:56:38.437599+0000', 'last_fullsized': '2024-04-25T09:56:38.437599+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2024-04-25T09:56:43.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.436685+0000', 'last_change': '2024-04-25T09:56:38.436685+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.436685+0000', 'last_undegraded': '2024-04-25T09:56:38.436685+0000', 'last_fullsized': '2024-04-25T09:56:38.436685+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:56:43.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2024-04-25T09:56:43.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.439909+0000', 'last_change': '2024-04-25T09:56:38.439909+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.439909+0000', 'last_undegraded': '2024-04-25T09:56:38.439909+0000', 'last_fullsized': '2024-04-25T09:56:38.439909+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:56:43.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2024-04-25T09:56:43.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.438373+0000', 'last_change': '2024-04-25T09:56:38.438373+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.438373+0000', 'last_undegraded': '2024-04-25T09:56:38.438373+0000', 'last_fullsized': '2024-04-25T09:56:38.438373+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:56:43.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2024-04-25T09:56:43.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.433700+0000', 'last_change': '2024-04-25T09:56:38.433700+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.433700+0000', 'last_undegraded': '2024-04-25T09:56:38.433700+0000', 'last_fullsized': '2024-04-25T09:56:38.433700+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:56:43.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2024-04-25T09:56:43.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.435532+0000', 'last_change': '2024-04-25T09:56:38.435532+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.435532+0000', 'last_undegraded': '2024-04-25T09:56:38.435532+0000', 'last_fullsized': '2024-04-25T09:56:38.435532+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:56:43.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2024-04-25T09:56:43.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.435197+0000', 'last_change': '2024-04-25T09:56:38.435197+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.435197+0000', 'last_undegraded': '2024-04-25T09:56:38.435197+0000', 'last_fullsized': '2024-04-25T09:56:38.435197+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:56:43.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2024-04-25T09:56:43.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434689+0000', 'last_change': '2024-04-25T09:56:38.434689+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434689+0000', 'last_undegraded': '2024-04-25T09:56:38.434689+0000', 'last_fullsized': '2024-04-25T09:56:38.434689+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:56:43.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2024-04-25T09:56:43.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.436051+0000', 'last_change': '2024-04-25T09:56:38.436051+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.436051+0000', 'last_undegraded': '2024-04-25T09:56:38.436051+0000', 'last_fullsized': '2024-04-25T09:56:38.436051+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:56:43.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2024-04-25T09:56:43.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.433377+0000', 'last_change': '2024-04-25T09:56:38.433377+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.433377+0000', 'last_undegraded': '2024-04-25T09:56:38.433377+0000', 'last_fullsized': '2024-04-25T09:56:38.433377+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:56:43.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2024-04-25T09:56:43.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.433505+0000', 'last_change': '2024-04-25T09:56:38.433505+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.433505+0000', 'last_undegraded': '2024-04-25T09:56:38.433505+0000', 'last_fullsized': '2024-04-25T09:56:38.433505+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:56:43.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2024-04-25T09:56:43.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.439675+0000', 'last_change': '2024-04-25T09:56:38.439675+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.439675+0000', 'last_undegraded': '2024-04-25T09:56:38.439675+0000', 'last_fullsized': '2024-04-25T09:56:38.439675+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:56:43.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2024-04-25T09:56:43.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434662+0000', 'last_change': '2024-04-25T09:56:38.434662+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434662+0000', 'last_undegraded': '2024-04-25T09:56:38.434662+0000', 'last_fullsized': '2024-04-25T09:56:38.434662+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:56:43.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2024-04-25T09:56:43.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.435818+0000', 'last_change': '2024-04-25T09:56:38.435818+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.435818+0000', 'last_undegraded': '2024-04-25T09:56:38.435818+0000', 'last_fullsized': '2024-04-25T09:56:38.435818+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:56:43.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2024-04-25T09:56:43.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.437438+0000', 'last_change': '2024-04-25T09:56:38.437438+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.437438+0000', 'last_undegraded': '2024-04-25T09:56:38.437438+0000', 'last_fullsized': '2024-04-25T09:56:38.437438+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2024-04-25T09:56:43.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434623+0000', 'last_change': '2024-04-25T09:56:38.434623+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434623+0000', 'last_undegraded': '2024-04-25T09:56:38.434623+0000', 'last_fullsized': '2024-04-25T09:56:38.434623+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:56:43.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2024-04-25T09:56:43.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434027+0000', 'last_change': '2024-04-25T09:56:38.434027+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434027+0000', 'last_undegraded': '2024-04-25T09:56:38.434027+0000', 'last_fullsized': '2024-04-25T09:56:38.434027+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:56:43.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2024-04-25T09:56:43.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.436029+0000', 'last_change': '2024-04-25T09:56:38.436029+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.436029+0000', 'last_undegraded': '2024-04-25T09:56:38.436029+0000', 'last_fullsized': '2024-04-25T09:56:38.436029+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:56:43.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2024-04-25T09:56:43.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434596+0000', 'last_change': '2024-04-25T09:56:38.434596+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434596+0000', 'last_undegraded': '2024-04-25T09:56:38.434596+0000', 'last_fullsized': '2024-04-25T09:56:38.434596+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:56:43.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2024-04-25T09:56:43.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.433635+0000', 'last_change': '2024-04-25T09:56:38.433635+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.433635+0000', 'last_undegraded': '2024-04-25T09:56:38.433635+0000', 'last_fullsized': '2024-04-25T09:56:38.433635+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:56:43.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2024-04-25T09:56:43.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.437925+0000', 'last_change': '2024-04-25T09:56:38.437925+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.437925+0000', 'last_undegraded': '2024-04-25T09:56:38.437925+0000', 'last_fullsized': '2024-04-25T09:56:38.437925+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:56:43.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2024-04-25T09:56:43.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434898+0000', 'last_change': '2024-04-25T09:56:38.434898+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434898+0000', 'last_undegraded': '2024-04-25T09:56:38.434898+0000', 'last_fullsized': '2024-04-25T09:56:38.434898+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:56:43.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2024-04-25T09:56:43.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434806+0000', 'last_change': '2024-04-25T09:56:38.434806+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434806+0000', 'last_undegraded': '2024-04-25T09:56:38.434806+0000', 'last_fullsized': '2024-04-25T09:56:38.434806+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:56:43.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2024-04-25T09:56:43.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434169+0000', 'last_change': '2024-04-25T09:56:38.434169+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434169+0000', 'last_undegraded': '2024-04-25T09:56:38.434169+0000', 'last_fullsized': '2024-04-25T09:56:38.434169+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:56:43.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2024-04-25T09:56:43.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.433478+0000', 'last_change': '2024-04-25T09:56:38.433478+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.433478+0000', 'last_undegraded': '2024-04-25T09:56:38.433478+0000', 'last_fullsized': '2024-04-25T09:56:38.433478+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:56:43.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2024-04-25T09:56:43.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.435681+0000', 'last_change': '2024-04-25T09:56:38.435681+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.435681+0000', 'last_undegraded': '2024-04-25T09:56:38.435681+0000', 'last_fullsized': '2024-04-25T09:56:38.435681+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:56:43.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2024-04-25T09:56:43.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.431678+0000', 'last_change': '2024-04-25T09:56:38.431678+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.431678+0000', 'last_undegraded': '2024-04-25T09:56:38.431678+0000', 'last_fullsized': '2024-04-25T09:56:38.431678+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:56:43.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2024-04-25T09:56:43.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434572+0000', 'last_change': '2024-04-25T09:56:38.434572+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434572+0000', 'last_undegraded': '2024-04-25T09:56:38.434572+0000', 'last_fullsized': '2024-04-25T09:56:38.434572+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:56:43.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-04-25T09:56:43.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434920+0000', 'last_change': '2024-04-25T09:56:38.434920+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434920+0000', 'last_undegraded': '2024-04-25T09:56:38.434920+0000', 'last_fullsized': '2024-04-25T09:56:38.434920+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2024-04-25T09:56:43.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.432113+0000', 'last_change': '2024-04-25T09:56:38.432113+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.432113+0000', 'last_undegraded': '2024-04-25T09:56:38.432113+0000', 'last_fullsized': '2024-04-25T09:56:38.432113+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:56:43.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2024-04-25T09:56:43.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434485+0000', 'last_change': '2024-04-25T09:56:38.434485+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434485+0000', 'last_undegraded': '2024-04-25T09:56:38.434485+0000', 'last_fullsized': '2024-04-25T09:56:38.434485+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:56:43.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2024-04-25T09:56:43.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434490+0000', 'last_change': '2024-04-25T09:56:38.434490+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434490+0000', 'last_undegraded': '2024-04-25T09:56:38.434490+0000', 'last_fullsized': '2024-04-25T09:56:38.434490+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:56:43.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2024-04-25T09:56:43.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.439169+0000', 'last_change': '2024-04-25T09:56:38.439169+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.439169+0000', 'last_undegraded': '2024-04-25T09:56:38.439169+0000', 'last_fullsized': '2024-04-25T09:56:38.439169+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:56:43.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2024-04-25T09:56:43.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434138+0000', 'last_change': '2024-04-25T09:56:38.434138+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434138+0000', 'last_undegraded': '2024-04-25T09:56:38.434138+0000', 'last_fullsized': '2024-04-25T09:56:38.434138+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:56:43.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-04-25T09:56:43.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.438728+0000', 'last_change': '2024-04-25T09:56:38.438728+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.438728+0000', 'last_undegraded': '2024-04-25T09:56:38.438728+0000', 'last_fullsized': '2024-04-25T09:56:38.438728+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2024-04-25T09:56:43.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.435118+0000', 'last_change': '2024-04-25T09:56:38.435118+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.435118+0000', 'last_undegraded': '2024-04-25T09:56:38.435118+0000', 'last_fullsized': '2024-04-25T09:56:38.435118+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:56:43.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2024-04-25T09:56:43.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.433629+0000', 'last_change': '2024-04-25T09:56:38.433629+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.433629+0000', 'last_undegraded': '2024-04-25T09:56:38.433629+0000', 'last_fullsized': '2024-04-25T09:56:38.433629+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:56:43.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2024-04-25T09:56:43.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.437792+0000', 'last_change': '2024-04-25T09:56:38.437792+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.437792+0000', 'last_undegraded': '2024-04-25T09:56:38.437792+0000', 'last_fullsized': '2024-04-25T09:56:38.437792+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:56:43.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-04-25T09:56:43.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.438457+0000', 'last_change': '2024-04-25T09:56:38.438457+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.438457+0000', 'last_undegraded': '2024-04-25T09:56:38.438457+0000', 'last_fullsized': '2024-04-25T09:56:38.438457+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2024-04-25T09:56:43.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.431918+0000', 'last_change': '2024-04-25T09:56:38.431918+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.431918+0000', 'last_undegraded': '2024-04-25T09:56:38.431918+0000', 'last_fullsized': '2024-04-25T09:56:38.431918+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:56:43.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2024-04-25T09:56:43.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.431725+0000', 'last_change': '2024-04-25T09:56:38.431725+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.431725+0000', 'last_undegraded': '2024-04-25T09:56:38.431725+0000', 'last_fullsized': '2024-04-25T09:56:38.431725+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:56:43.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2024-04-25T09:56:43.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.436850+0000', 'last_change': '2024-04-25T09:56:38.436850+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.436850+0000', 'last_undegraded': '2024-04-25T09:56:38.436850+0000', 'last_fullsized': '2024-04-25T09:56:38.436850+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:56:43.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-04-25T09:56:43.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.438322+0000', 'last_change': '2024-04-25T09:56:38.438322+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.438322+0000', 'last_undegraded': '2024-04-25T09:56:38.438322+0000', 'last_fullsized': '2024-04-25T09:56:38.438322+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2024-04-25T09:56:43.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.433556+0000', 'last_change': '2024-04-25T09:56:38.433556+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.433556+0000', 'last_undegraded': '2024-04-25T09:56:38.433556+0000', 'last_fullsized': '2024-04-25T09:56:38.433556+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:56:43.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2024-04-25T09:56:43.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.433134+0000', 'last_change': '2024-04-25T09:56:38.433134+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.433134+0000', 'last_undegraded': '2024-04-25T09:56:38.433134+0000', 'last_fullsized': '2024-04-25T09:56:38.433134+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:56:43.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-04-25T09:56:43.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.436880+0000', 'last_change': '2024-04-25T09:56:38.436880+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.436880+0000', 'last_undegraded': '2024-04-25T09:56:38.436880+0000', 'last_fullsized': '2024-04-25T09:56:38.436880+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2024-04-25T09:56:43.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.435023+0000', 'last_change': '2024-04-25T09:56:38.435023+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.435023+0000', 'last_undegraded': '2024-04-25T09:56:38.435023+0000', 'last_fullsized': '2024-04-25T09:56:38.435023+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:56:43.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-04-25T09:56:43.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.437996+0000', 'last_change': '2024-04-25T09:56:38.437996+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.437996+0000', 'last_undegraded': '2024-04-25T09:56:38.437996+0000', 'last_fullsized': '2024-04-25T09:56:38.437996+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2024-04-25T09:56:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434042+0000', 'last_change': '2024-04-25T09:56:38.434042+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434042+0000', 'last_undegraded': '2024-04-25T09:56:38.434042+0000', 'last_fullsized': '2024-04-25T09:56:38.434042+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:56:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2024-04-25T09:56:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.436494+0000', 'last_change': '2024-04-25T09:56:38.436494+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.436494+0000', 'last_undegraded': '2024-04-25T09:56:38.436494+0000', 'last_fullsized': '2024-04-25T09:56:38.436494+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:56:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-04-25T09:56:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.435576+0000', 'last_change': '2024-04-25T09:56:38.435576+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.435576+0000', 'last_undegraded': '2024-04-25T09:56:38.435576+0000', 'last_fullsized': '2024-04-25T09:56:38.435576+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2024-04-25T09:56:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.435303+0000', 'last_change': '2024-04-25T09:56:38.435303+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.435303+0000', 'last_undegraded': '2024-04-25T09:56:38.435303+0000', 'last_fullsized': '2024-04-25T09:56:38.435303+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:56:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2024-04-25T09:56:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.443451+0000', 'last_change': '2024-04-25T09:56:38.443451+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.443451+0000', 'last_undegraded': '2024-04-25T09:56:38.443451+0000', 'last_fullsized': '2024-04-25T09:56:38.443451+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:56:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2024-04-25T09:56:43.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.434466+0000', 'last_change': '2024-04-25T09:56:38.434466+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.434466+0000', 'last_undegraded': '2024-04-25T09:56:38.434466+0000', 'last_fullsized': '2024-04-25T09:56:38.434466+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:56:43.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-04-25T09:56:43.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.437364+0000', 'last_change': '2024-04-25T09:56:38.437364+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.437364+0000', 'last_undegraded': '2024-04-25T09:56:38.437364+0000', 'last_fullsized': '2024-04-25T09:56:38.437364+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-04-25T09:56:43.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.436133+0000', 'last_change': '2024-04-25T09:56:38.436133+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.436133+0000', 'last_undegraded': '2024-04-25T09:56:38.436133+0000', 'last_fullsized': '2024-04-25T09:56:38.436133+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2024-04-25T09:56:43.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.431039+0000', 'last_change': '2024-04-25T09:56:38.431039+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.431039+0000', 'last_undegraded': '2024-04-25T09:56:38.431039+0000', 'last_fullsized': '2024-04-25T09:56:38.431039+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:56:43.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2024-04-25T09:56:43.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.433502+0000', 'last_change': '2024-04-25T09:56:38.433502+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.433502+0000', 'last_undegraded': '2024-04-25T09:56:38.433502+0000', 'last_fullsized': '2024-04-25T09:56:38.433502+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:56:43.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2024-04-25T09:56:43.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.443248+0000', 'last_change': '2024-04-25T09:56:38.443248+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.443248+0000', 'last_undegraded': '2024-04-25T09:56:38.443248+0000', 'last_fullsized': '2024-04-25T09:56:38.443248+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:56:43.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-04-25T09:56:43.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:56:38.436966+0000', 'last_change': '2024-04-25T09:56:38.436966+0000', 'last_active': '2024-04-25T09:56:38.425967+0000', 'last_peered': '2024-04-25T09:56:38.425967+0000', 'last_clean': '2024-04-25T09:56:38.425967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:56:38.436966+0000', 'last_undegraded': '2024-04-25T09:56:38.436966+0000', 'last_fullsized': '2024-04-25T09:56:38.436966+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'last_clean_scrub_stamp': '2024-04-25T09:56:38.425967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:56:43.166 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:56:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.283+0000 7f96b65c1640 1 Processor -- start 2024-04-25T09:56:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.283+0000 7f96b65c1640 1 -- start start 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.284+0000 7f96b65c1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f96b00b6cb0 0x7f96b00b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.284+0000 7f96b65c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b5a60 0x7f96b00d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.284+0000 7f96b65c1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b00ceee0 con 0x7f96b015db70 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.284+0000 7f96b65c1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b00d1d10 con 0x7f96b00b6cb0 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.284+0000 7f96b65c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00d1ff0 con 0x7f96b00b5a60 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.284+0000 7f96af7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f96b00b6cb0 0x7f96b00b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.284+0000 7f96b4b5e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b5a60 0x7f96b00d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.284+0000 7f96b4b5e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b5a60 0x7f96b00d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:58186/0 (socket says 172.21.15.19:58186) 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.284+0000 7f96b4b5e640 1 -- 172.21.15.19:0/937433035 learned_addr learned my addr 172.21.15.19:0/937433035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.284+0000 7f96af7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f96b00b6cb0 0x7f96b00b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:47172/0 (socket says 172.21.15.19:47172) 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.284+0000 7f96affff640 1 --1- 172.21.15.19:0/937433035 >> v1:172.21.15.112:6789/0 conn(0x7f96b015db70 0x7f96b0162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:55086/0 (socket says 172.21.15.19:55086) 2024-04-25T09:56:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.285+0000 7f96b4b5e640 1 -- 172.21.15.19:0/937433035 >> v2:172.21.15.112:3300/0 conn(0x7f96b00b6cb0 msgr2=0x7f96b00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.285+0000 7f96b4b5e640 1 --2- 172.21.15.19:0/937433035 >> v2:172.21.15.112:3300/0 conn(0x7f96b00b6cb0 0x7f96b00b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.285+0000 7f96b4b5e640 1 -- 172.21.15.19:0/937433035 >> v1:172.21.15.112:6789/0 conn(0x7f96b015db70 legacy=0x7f96b0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.285+0000 7f96b4b5e640 1 -- 172.21.15.19:0/937433035 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00d22d0 con 0x7f96b00b5a60 2024-04-25T09:56:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.285+0000 7f96af7fe640 1 --2- 172.21.15.19:0/937433035 >> v2:172.21.15.112:3300/0 conn(0x7f96b00b6cb0 0x7f96b00b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.285+0000 7f96b4b5e640 1 --2- 172.21.15.19:0/937433035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b5a60 0x7f96b00d27f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009e50 tx=0x7f96a802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f12bb89d16297e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.286+0000 7f96aeffd640 1 -- 172.21.15.19:0/937433035 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8036070 con 0x7f96b00b5a60 2024-04-25T09:56:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.286+0000 7f96aeffd640 1 -- 172.21.15.19:0/937433035 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a8031600 con 0x7f96b00b5a60 2024-04-25T09:56:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.286+0000 7f96aeffd640 1 -- 172.21.15.19:0/937433035 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a80319d0 con 0x7f96b00b5a60 2024-04-25T09:56:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.286+0000 7f96b65c1640 1 -- 172.21.15.19:0/937433035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b5a60 msgr2=0x7f96b00d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.286+0000 7f96b65c1640 1 --2- 172.21.15.19:0/937433035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b5a60 0x7f96b00d27f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009e50 tx=0x7f96a802a710 comp rx=0 tx=0).stop 2024-04-25T09:56:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.286+0000 7f96b65c1640 1 -- 172.21.15.19:0/937433035 shutdown_connections 2024-04-25T09:56:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.286+0000 7f96b65c1640 1 --2- 172.21.15.19:0/937433035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b5a60 0x7f96b00d27f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.286+0000 7f96b65c1640 1 --2- 172.21.15.19:0/937433035 >> v2:172.21.15.112:3300/0 conn(0x7f96b00b6cb0 0x7f96b00b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.286+0000 7f96b65c1640 1 -- 172.21.15.19:0/937433035 >> 172.21.15.19:0/937433035 conn(0x7f96b00d0520 msgr2=0x7f96b00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.286+0000 7f96b65c1640 1 -- 172.21.15.19:0/937433035 shutdown_connections 2024-04-25T09:56:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.286+0000 7f96b65c1640 1 -- 172.21.15.19:0/937433035 wait complete. 2024-04-25T09:56:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.287+0000 7f96b65c1640 1 Processor -- start 2024-04-25T09:56:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.288+0000 7f96b65c1640 1 -- start start 2024-04-25T09:56:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.288+0000 7f96b65c1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b00b5a60 0x7f96b0155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.288+0000 7f96b65c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b6cb0 0x7f96b01f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.288+0000 7f96b65c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b015db70 0x7f96b01f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.288+0000 7f96b65c1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00cf1c0 con 0x7f96b00b5a60 2024-04-25T09:56:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.288+0000 7f96b65c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00ce530 con 0x7f96b015db70 2024-04-25T09:56:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.288+0000 7f96b65c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00cf340 con 0x7f96b00b6cb0 2024-04-25T09:56:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.288+0000 7f96affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b00b5a60 0x7f96b0155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.288+0000 7f96affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b00b5a60 0x7f96b0155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53278/0 (socket says 172.21.15.19:53278) 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.288+0000 7f96affff640 1 -- 172.21.15.19:0/2403575193 learned_addr learned my addr 172.21.15.19:0/2403575193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96af7fe640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b6cb0 0x7f96b01f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96b4b5e640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b015db70 0x7f96b01f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96affff640 1 -- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b6cb0 msgr2=0x7f96b01f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96affff640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b6cb0 0x7f96b01f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96affff640 1 -- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b015db70 msgr2=0x7f96b01f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96affff640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b015db70 0x7f96b01f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96affff640 1 -- 172.21.15.19:0/2403575193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b01561e0 con 0x7f96b00b5a60 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96affff640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b00b5a60 0x7f96b0155ca0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c400 tx=0x7f96a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96acff9640 1 -- 172.21.15.19:0/2403575193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a40074e0 con 0x7f96b00b5a60 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96acff9640 1 -- 172.21.15.19:0/2403575193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a4007680 con 0x7f96b00b5a60 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96b65c1640 1 -- 172.21.15.19:0/2403575193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b00cc9b0 con 0x7f96b00b5a60 2024-04-25T09:56:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96acff9640 1 -- 172.21.15.19:0/2403575193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4004a00 con 0x7f96b00b5a60 2024-04-25T09:56:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.289+0000 7f96b65c1640 1 -- 172.21.15.19:0/2403575193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b00ccf70 con 0x7f96b00b5a60 2024-04-25T09:56:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.290+0000 7f96b65c1640 1 -- 172.21.15.19:0/2403575193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b0073990 con 0x7f96b00b5a60 2024-04-25T09:56:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.291+0000 7f96acff9640 1 -- 172.21.15.19:0/2403575193 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96a401a020 con 0x7f96b00b5a60 2024-04-25T09:56:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.291+0000 7f96acff9640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f969c06b4d0 0x7f969c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.291+0000 7f96acff9640 1 -- 172.21.15.19:0/2403575193 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f96a40899f0 con 0x7f96b00b5a60 2024-04-25T09:56:43.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.291+0000 7f96af7fe640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f969c06b4d0 0x7f969c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.292+0000 7f96af7fe640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f969c06b4d0 0x7f969c06d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009560 tx=0x7f96a0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:43.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.295+0000 7f96acff9640 1 -- 172.21.15.19:0/2403575193 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96a4058120 con 0x7f96b00b5a60 2024-04-25T09:56:43.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.546+0000 7f96b65c1640 1 -- 172.21.15.19:0/2403575193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f96b015bca0 con 0x7f96b00b5a60 2024-04-25T09:56:43.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.547+0000 7f96acff9640 1 -- 172.21.15.19:0/2403575193 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1453 (secure 0 0 0) 0x7f96a4089080 con 0x7f96b00b5a60 2024-04-25T09:56:43.550 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:43.550 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":240,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":296673280,"bytes_avail":772797440000,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":35,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6582}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-25T09:56:28.958406+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:56:43.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 -- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f969c06b4d0 msgr2=0x7f969c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f969c06b4d0 0x7f969c06d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009560 tx=0x7f96a0009190 comp rx=0 tx=0).stop 2024-04-25T09:56:43.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 -- 172.21.15.19:0/2403575193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b00b5a60 msgr2=0x7f96b0155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b00b5a60 0x7f96b0155ca0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c400 tx=0x7f96a400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 -- 172.21.15.19:0/2403575193 shutdown_connections 2024-04-25T09:56:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f969c06b4d0 0x7f969c06d990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96b015db70 0x7f96b01f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96b00b6cb0 0x7f96b01f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 --2- 172.21.15.19:0/2403575193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b00b5a60 0x7f96b0155ca0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 -- 172.21.15.19:0/2403575193 >> 172.21.15.19:0/2403575193 conn(0x7f96b00d0520 msgr2=0x7f96b0156e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 -- 172.21.15.19:0/2403575193 shutdown_connections 2024-04-25T09:56:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.549+0000 7f96b65c1640 1 -- 172.21.15.19:0/2403575193 wait complete. 2024-04-25T09:56:43.563 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:56:43.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:56:43.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.661+0000 7fd36b729640 1 Processor -- start 2024-04-25T09:56:43.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.661+0000 7fd36b729640 1 -- start start 2024-04-25T09:56:43.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd36b729640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd36415db70 0x7fd364162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd36b729640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd3640b5c30 0x7fd3640d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd36b729640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd364162690 con 0x7fd3640b6cb0 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd36b729640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3640ceee0 con 0x7fd3640b5c30 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd36b729640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640d1d10 con 0x7fd36415db70 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd369cc6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd3640b5c30 0x7fd3640d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd369cc6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd3640b5c30 0x7fd3640d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53294/0 (socket says 172.21.15.19:53294) 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd368cc4640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd3640b6cb0 0x7fd3640b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36628/0 (socket says 172.21.15.19:36628) 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd369cc6640 1 -- 172.21.15.19:0/3291534782 learned_addr learned my addr 172.21.15.19:0/3291534782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd3694c5640 1 --2- 172.21.15.19:0/3291534782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd36415db70 0x7fd364162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd369cc6640 1 -- 172.21.15.19:0/3291534782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd36415db70 msgr2=0x7fd364162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd369cc6640 1 --2- 172.21.15.19:0/3291534782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd36415db70 0x7fd364162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd369cc6640 1 -- 172.21.15.19:0/3291534782 >> v1:172.21.15.19:6789/0 conn(0x7fd3640b6cb0 legacy=0x7fd3640b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd369cc6640 1 -- 172.21.15.19:0/3291534782 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640d1ff0 con 0x7fd3640b5c30 2024-04-25T09:56:43.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.662+0000 7fd363fff640 1 -- 172.21.15.19:0/3291534782 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837947614 0 0) 0x7fd364162690 con 0x7fd3640b6cb0 2024-04-25T09:56:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.663+0000 7fd369cc6640 1 --2- 172.21.15.19:0/3291534782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3640b5c30 0x7fd3640d27f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009870 tx=0x7fd35c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230170d25b26b9e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.663+0000 7fd363fff640 1 -- 172.21.15.19:0/3291534782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c037070 con 0x7fd3640b5c30 2024-04-25T09:56:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.663+0000 7fd363fff640 1 -- 172.21.15.19:0/3291534782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c032440 con 0x7fd3640b5c30 2024-04-25T09:56:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.663+0000 7fd363fff640 1 -- 172.21.15.19:0/3291534782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c032810 con 0x7fd3640b5c30 2024-04-25T09:56:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.664+0000 7fd36b729640 1 -- 172.21.15.19:0/3291534782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3640b5c30 msgr2=0x7fd3640d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.664+0000 7fd36b729640 1 --2- 172.21.15.19:0/3291534782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3640b5c30 0x7fd3640d27f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009870 tx=0x7fd35c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.664+0000 7fd36b729640 1 -- 172.21.15.19:0/3291534782 shutdown_connections 2024-04-25T09:56:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.664+0000 7fd36b729640 1 --2- 172.21.15.19:0/3291534782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3640b5c30 0x7fd3640d27f0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.664+0000 7fd36b729640 1 --2- 172.21.15.19:0/3291534782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd36415db70 0x7fd364162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.664+0000 7fd36b729640 1 -- 172.21.15.19:0/3291534782 >> 172.21.15.19:0/3291534782 conn(0x7fd3640d0520 msgr2=0x7fd3640d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.664+0000 7fd36b729640 1 -- 172.21.15.19:0/3291534782 shutdown_connections 2024-04-25T09:56:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.664+0000 7fd36b729640 1 -- 172.21.15.19:0/3291534782 wait complete. 2024-04-25T09:56:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.665+0000 7fd36b729640 1 Processor -- start 2024-04-25T09:56:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.665+0000 7fd36b729640 1 -- start start 2024-04-25T09:56:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd36b729640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3640b5c30 0x7fd36415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd36b729640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3640b6cb0 0x7fd3640cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd36b729640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd36415db70 0x7fd36415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd36b729640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640ce530 con 0x7fd3640b6cb0 2024-04-25T09:56:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd36b729640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36415df50 con 0x7fd3640b5c30 2024-04-25T09:56:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd36b729640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640cf1c0 con 0x7fd36415db70 2024-04-25T09:56:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd368cc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3640b6cb0 0x7fd3640cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd368cc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3640b6cb0 0x7fd3640cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53308/0 (socket says 172.21.15.19:53308) 2024-04-25T09:56:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd368cc4640 1 -- 172.21.15.19:0/3010781930 learned_addr learned my addr 172.21.15.19:0/3010781930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd369cc6640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd36415db70 0x7fd36415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd3694c5640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3640b5c30 0x7fd36415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd368cc4640 1 -- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd36415db70 msgr2=0x7fd36415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd368cc4640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd36415db70 0x7fd36415c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd368cc4640 1 -- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3640b5c30 msgr2=0x7fd36415f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd368cc4640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3640b5c30 0x7fd36415f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd368cc4640 1 -- 172.21.15.19:0/3010781930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640ccbe0 con 0x7fd3640b6cb0 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.666+0000 7fd368cc4640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3640b6cb0 0x7fd3640cc460 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd35800c460 tx=0x7fd35800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.667+0000 7fd361ffb640 1 -- 172.21.15.19:0/3010781930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358018070 con 0x7fd3640b6cb0 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.667+0000 7fd361ffb640 1 -- 172.21.15.19:0/3010781930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd358007520 con 0x7fd3640b6cb0 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.667+0000 7fd361ffb640 1 -- 172.21.15.19:0/3010781930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358004a20 con 0x7fd3640b6cb0 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.667+0000 7fd36b729640 1 -- 172.21.15.19:0/3010781930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3640ccec0 con 0x7fd3640b6cb0 2024-04-25T09:56:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.667+0000 7fd36b729640 1 -- 172.21.15.19:0/3010781930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3640cd480 con 0x7fd3640b6cb0 2024-04-25T09:56:43.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.668+0000 7fd36b729640 1 -- 172.21.15.19:0/3010781930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd364073990 con 0x7fd3640b6cb0 2024-04-25T09:56:43.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.668+0000 7fd361ffb640 1 -- 172.21.15.19:0/3010781930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd358004bc0 con 0x7fd3640b6cb0 2024-04-25T09:56:43.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.669+0000 7fd361ffb640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd35006b4d0 0x7fd35006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:43.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.669+0000 7fd361ffb640 1 -- 172.21.15.19:0/3010781930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd358089550 con 0x7fd3640b6cb0 2024-04-25T09:56:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.669+0000 7fd3694c5640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd35006b4d0 0x7fd35006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.669+0000 7fd3694c5640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd35006b4d0 0x7fd35006d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd354009e90 tx=0x7fd35400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.673+0000 7fd361ffb640 1 -- 172.21.15.19:0/3010781930 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd358057c80 con 0x7fd3640b6cb0 2024-04-25T09:56:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.851+0000 7fd36b729640 1 -- 172.21.15.19:0/3010781930 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd3640d1820 con 0x7fd35006b4d0 2024-04-25T09:56:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.857+0000 7fd361ffb640 1 -- 172.21.15.19:0/3010781930 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246700 (secure 0 0 0) 0x7fd3640d1820 con 0x7fd35006b4d0 2024-04-25T09:56:43.865 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:43.867 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:56:43.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 -- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd35006b4d0 msgr2=0x7fd35006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd35006b4d0 0x7fd35006d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd354009e90 tx=0x7fd35400b040 comp rx=0 tx=0).stop 2024-04-25T09:56:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 -- 172.21.15.19:0/3010781930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3640b6cb0 msgr2=0x7fd3640cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3640b6cb0 0x7fd3640cc460 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd35800c460 tx=0x7fd35800c920 comp rx=0 tx=0).stop 2024-04-25T09:56:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 -- 172.21.15.19:0/3010781930 shutdown_connections 2024-04-25T09:56:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd35006b4d0 0x7fd35006d990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd36415db70 0x7fd36415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3640b6cb0 0x7fd3640cc460 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 --2- 172.21.15.19:0/3010781930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3640b5c30 0x7fd36415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 -- 172.21.15.19:0/3010781930 >> 172.21.15.19:0/3010781930 conn(0x7fd3640d0520 msgr2=0x7fd3641560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 -- 172.21.15.19:0/3010781930 shutdown_connections 2024-04-25T09:56:43.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:43.860+0000 7fd36b729640 1 -- 172.21.15.19:0/3010781930 wait complete. 2024-04-25T09:56:43.875 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-04-25T09:56:42.961633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289720,"kb_used_data":6980,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685000,"statfs":{"total":773094113280,"available":772797440000,"internally_reserved":0,"allocated":7147520,"data_stored":3857879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.534188"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437683+0000","last_change":"2024-04-25T09:56:38.437683+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437683+0000","last_undegraded":"2024-04-25T09:56:38.437683+0000","last_fullsized":"2024-04-25T09:56:38.437683+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436642+0000","last_change":"2024-04-25T09:56:38.436642+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436642+0000","last_undegraded":"2024-04-25T09:56:38.436642+0000","last_fullsized":"2024-04-25T09:56:38.436642+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433788+0000","last_change":"2024-04-25T09:56:38.433788+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433788+0000","last_undegraded":"2024-04-25T09:56:38.433788+0000","last_fullsized":"2024-04-25T09:56:38.433788+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431818+0000","last_change":"2024-04-25T09:56:38.431818+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431818+0000","last_undegraded":"2024-04-25T09:56:38.431818+0000","last_fullsized":"2024-04-25T09:56:38.431818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437599+0000","last_change":"2024-04-25T09:56:38.437599+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437599+0000","last_undegraded":"2024-04-25T09:56:38.437599+0000","last_fullsized":"2024-04-25T09:56:38.437599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436685+0000","last_change":"2024-04-25T09:56:38.436685+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436685+0000","last_undegraded":"2024-04-25T09:56:38.436685+0000","last_fullsized":"2024-04-25T09:56:38.436685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439909+0000","last_change":"2024-04-25T09:56:38.439909+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439909+0000","last_undegraded":"2024-04-25T09:56:38.439909+0000","last_fullsized":"2024-04-25T09:56:38.439909+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438373+0000","last_change":"2024-04-25T09:56:38.438373+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438373+0000","last_undegraded":"2024-04-25T09:56:38.438373+0000","last_fullsized":"2024-04-25T09:56:38.438373+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433700+0000","last_change":"2024-04-25T09:56:38.433700+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433700+0000","last_undegraded":"2024-04-25T09:56:38.433700+0000","last_fullsized":"2024-04-25T09:56:38.433700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435532+0000","last_change":"2024-04-25T09:56:38.435532+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435532+0000","last_undegraded":"2024-04-25T09:56:38.435532+0000","last_fullsized":"2024-04-25T09:56:38.435532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435197+0000","last_change":"2024-04-25T09:56:38.435197+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435197+0000","last_undegraded":"2024-04-25T09:56:38.435197+0000","last_fullsized":"2024-04-25T09:56:38.435197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434689+0000","last_change":"2024-04-25T09:56:38.434689+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434689+0000","last_undegraded":"2024-04-25T09:56:38.434689+0000","last_fullsized":"2024-04-25T09:56:38.434689+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436051+0000","last_change":"2024-04-25T09:56:38.436051+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436051+0000","last_undegraded":"2024-04-25T09:56:38.436051+0000","last_fullsized":"2024-04-25T09:56:38.436051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433377+0000","last_change":"2024-04-25T09:56:38.433377+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433377+0000","last_undegraded":"2024-04-25T09:56:38.433377+0000","last_fullsized":"2024-04-25T09:56:38.433377+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433505+0000","last_change":"2024-04-25T09:56:38.433505+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433505+0000","last_undegraded":"2024-04-25T09:56:38.433505+0000","last_fullsized":"2024-04-25T09:56:38.433505+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439675+0000","last_change":"2024-04-25T09:56:38.439675+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439675+0000","last_undegraded":"2024-04-25T09:56:38.439675+0000","last_fullsized":"2024-04-25T09:56:38.439675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434662+0000","last_change":"2024-04-25T09:56:38.434662+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434662+0000","last_undegraded":"2024-04-25T09:56:38.434662+0000","last_fullsized":"2024-04-25T09:56:38.434662+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435818+0000","last_change":"2024-04-25T09:56:38.435818+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435818+0000","last_undegraded":"2024-04-25T09:56:38.435818+0000","last_fullsized":"2024-04-25T09:56:38.435818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437438+0000","last_change":"2024-04-25T09:56:38.437438+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437438+0000","last_undegraded":"2024-04-25T09:56:38.437438+0000","last_fullsized":"2024-04-25T09:56:38.437438+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434623+0000","last_change":"2024-04-25T09:56:38.434623+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434623+0000","last_undegraded":"2024-04-25T09:56:38.434623+0000","last_fullsized":"2024-04-25T09:56:38.434623+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434027+0000","last_change":"2024-04-25T09:56:38.434027+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434027+0000","last_undegraded":"2024-04-25T09:56:38.434027+0000","last_fullsized":"2024-04-25T09:56:38.434027+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436029+0000","last_change":"2024-04-25T09:56:38.436029+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436029+0000","last_undegraded":"2024-04-25T09:56:38.436029+0000","last_fullsized":"2024-04-25T09:56:38.436029+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434596+0000","last_change":"2024-04-25T09:56:38.434596+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434596+0000","last_undegraded":"2024-04-25T09:56:38.434596+0000","last_fullsized":"2024-04-25T09:56:38.434596+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433635+0000","last_change":"2024-04-25T09:56:38.433635+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433635+0000","last_undegraded":"2024-04-25T09:56:38.433635+0000","last_fullsized":"2024-04-25T09:56:38.433635+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437925+0000","last_change":"2024-04-25T09:56:38.437925+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437925+0000","last_undegraded":"2024-04-25T09:56:38.437925+0000","last_fullsized":"2024-04-25T09:56:38.437925+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434898+0000","last_change":"2024-04-25T09:56:38.434898+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434898+0000","last_undegraded":"2024-04-25T09:56:38.434898+0000","last_fullsized":"2024-04-25T09:56:38.434898+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434806+0000","last_change":"2024-04-25T09:56:38.434806+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434806+0000","last_undegraded":"2024-04-25T09:56:38.434806+0000","last_fullsized":"2024-04-25T09:56:38.434806+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434169+0000","last_change":"2024-04-25T09:56:38.434169+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434169+0000","last_undegraded":"2024-04-25T09:56:38.434169+0000","last_fullsized":"2024-04-25T09:56:38.434169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433478+0000","last_change":"2024-04-25T09:56:38.433478+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433478+0000","last_undegraded":"2024-04-25T09:56:38.433478+0000","last_fullsized":"2024-04-25T09:56:38.433478+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435681+0000","last_change":"2024-04-25T09:56:38.435681+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435681+0000","last_undegraded":"2024-04-25T09:56:38.435681+0000","last_fullsized":"2024-04-25T09:56:38.435681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431678+0000","last_change":"2024-04-25T09:56:38.431678+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431678+0000","last_undegraded":"2024-04-25T09:56:38.431678+0000","last_fullsized":"2024-04-25T09:56:38.431678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434572+0000","last_change":"2024-04-25T09:56:38.434572+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434572+0000","last_undegraded":"2024-04-25T09:56:38.434572+0000","last_fullsized":"2024-04-25T09:56:38.434572+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434920+0000","last_change":"2024-04-25T09:56:38.434920+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434920+0000","last_undegraded":"2024-04-25T09:56:38.434920+0000","last_fullsized":"2024-04-25T09:56:38.434920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.451222+0000","last_change":"2024-04-25T09:56:38.448802+0000","last_active":"2024-04-25T09:56:38.451222+0000","last_peered":"2024-04-25T09:56:38.451222+0000","last_clean":"2024-04-25T09:56:38.451222+0000","last_became_active":"2024-04-25T09:56:38.447596+0000","last_became_peered":"2024-04-25T09:56:38.447596+0000","last_unstale":"2024-04-25T09:56:38.451222+0000","last_undegraded":"2024-04-25T09:56:38.451222+0000","last_fullsized":"2024-04-25T09:56:38.451222+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:56.119011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.432113+0000","last_change":"2024-04-25T09:56:38.432113+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.432113+0000","last_undegraded":"2024-04-25T09:56:38.432113+0000","last_fullsized":"2024-04-25T09:56:38.432113+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450513+0000","last_change":"2024-04-25T09:56:38.446203+0000","last_active":"2024-04-25T09:56:38.450513+0000","last_peered":"2024-04-25T09:56:38.450513+0000","last_clean":"2024-04-25T09:56:38.450513+0000","last_became_active":"2024-04-25T09:56:38.445931+0000","last_became_peered":"2024-04-25T09:56:38.445931+0000","last_unstale":"2024-04-25T09:56:38.450513+0000","last_undegraded":"2024-04-25T09:56:38.450513+0000","last_fullsized":"2024-04-25T09:56:38.450513+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:48.759756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434485+0000","last_change":"2024-04-25T09:56:38.434485+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434485+0000","last_undegraded":"2024-04-25T09:56:38.434485+0000","last_fullsized":"2024-04-25T09:56:38.434485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449712+0000","last_change":"2024-04-25T09:56:38.442763+0000","last_active":"2024-04-25T09:56:38.449712+0000","last_peered":"2024-04-25T09:56:38.449712+0000","last_clean":"2024-04-25T09:56:38.449712+0000","last_became_active":"2024-04-25T09:56:38.442494+0000","last_became_peered":"2024-04-25T09:56:38.442494+0000","last_unstale":"2024-04-25T09:56:38.449712+0000","last_undegraded":"2024-04-25T09:56:38.449712+0000","last_fullsized":"2024-04-25T09:56:38.449712+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.557978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434490+0000","last_change":"2024-04-25T09:56:38.434490+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434490+0000","last_undegraded":"2024-04-25T09:56:38.434490+0000","last_fullsized":"2024-04-25T09:56:38.434490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443747+0000","last_change":"2024-04-25T09:56:38.441504+0000","last_active":"2024-04-25T09:56:38.443747+0000","last_peered":"2024-04-25T09:56:38.443747+0000","last_clean":"2024-04-25T09:56:38.443747+0000","last_became_active":"2024-04-25T09:56:38.441178+0000","last_became_peered":"2024-04-25T09:56:38.441178+0000","last_unstale":"2024-04-25T09:56:38.443747+0000","last_undegraded":"2024-04-25T09:56:38.443747+0000","last_fullsized":"2024-04-25T09:56:38.443747+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:49.974254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.439169+0000","last_change":"2024-04-25T09:56:38.439169+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.439169+0000","last_undegraded":"2024-04-25T09:56:38.439169+0000","last_fullsized":"2024-04-25T09:56:38.439169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443433+0000","last_change":"2024-04-25T09:56:38.442701+0000","last_active":"2024-04-25T09:56:38.443433+0000","last_peered":"2024-04-25T09:56:38.443433+0000","last_clean":"2024-04-25T09:56:38.443433+0000","last_became_active":"2024-04-25T09:56:38.442433+0000","last_became_peered":"2024-04-25T09:56:38.442433+0000","last_unstale":"2024-04-25T09:56:38.443433+0000","last_undegraded":"2024-04-25T09:56:38.443433+0000","last_fullsized":"2024-04-25T09:56:38.443433+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:55.484379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434138+0000","last_change":"2024-04-25T09:56:38.434138+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434138+0000","last_undegraded":"2024-04-25T09:56:38.434138+0000","last_fullsized":"2024-04-25T09:56:38.434138+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448317+0000","last_change":"2024-04-25T09:56:38.446402+0000","last_active":"2024-04-25T09:56:38.448317+0000","last_peered":"2024-04-25T09:56:38.448317+0000","last_clean":"2024-04-25T09:56:38.448317+0000","last_became_active":"2024-04-25T09:56:38.446088+0000","last_became_peered":"2024-04-25T09:56:38.446088+0000","last_unstale":"2024-04-25T09:56:38.448317+0000","last_undegraded":"2024-04-25T09:56:38.448317+0000","last_fullsized":"2024-04-25T09:56:38.448317+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.933751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438728+0000","last_change":"2024-04-25T09:56:38.438728+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438728+0000","last_undegraded":"2024-04-25T09:56:38.438728+0000","last_fullsized":"2024-04-25T09:56:38.438728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.447423+0000","last_change":"2024-04-25T09:56:38.445015+0000","last_active":"2024-04-25T09:56:38.447423+0000","last_peered":"2024-04-25T09:56:38.447423+0000","last_clean":"2024-04-25T09:56:38.447423+0000","last_became_active":"2024-04-25T09:56:38.444713+0000","last_became_peered":"2024-04-25T09:56:38.444713+0000","last_unstale":"2024-04-25T09:56:38.447423+0000","last_undegraded":"2024-04-25T09:56:38.447423+0000","last_fullsized":"2024-04-25T09:56:38.447423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435118+0000","last_change":"2024-04-25T09:56:38.435118+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435118+0000","last_undegraded":"2024-04-25T09:56:38.435118+0000","last_fullsized":"2024-04-25T09:56:38.435118+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448373+0000","last_change":"2024-04-25T09:56:38.446581+0000","last_active":"2024-04-25T09:56:38.448373+0000","last_peered":"2024-04-25T09:56:38.448373+0000","last_clean":"2024-04-25T09:56:38.448373+0000","last_became_active":"2024-04-25T09:56:38.444721+0000","last_became_peered":"2024-04-25T09:56:38.444721+0000","last_unstale":"2024-04-25T09:56:38.448373+0000","last_undegraded":"2024-04-25T09:56:38.448373+0000","last_fullsized":"2024-04-25T09:56:38.448373+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:40.119587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433629+0000","last_change":"2024-04-25T09:56:38.433629+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433629+0000","last_undegraded":"2024-04-25T09:56:38.433629+0000","last_fullsized":"2024-04-25T09:56:38.433629+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454111+0000","last_change":"2024-04-25T09:56:38.452304+0000","last_active":"2024-04-25T09:56:38.454111+0000","last_peered":"2024-04-25T09:56:38.454111+0000","last_clean":"2024-04-25T09:56:38.454111+0000","last_became_active":"2024-04-25T09:56:38.452001+0000","last_became_peered":"2024-04-25T09:56:38.452001+0000","last_unstale":"2024-04-25T09:56:38.454111+0000","last_undegraded":"2024-04-25T09:56:38.454111+0000","last_fullsized":"2024-04-25T09:56:38.454111+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:07.043227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437792+0000","last_change":"2024-04-25T09:56:38.437792+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437792+0000","last_undegraded":"2024-04-25T09:56:38.437792+0000","last_fullsized":"2024-04-25T09:56:38.437792+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452779+0000","last_change":"2024-04-25T09:56:38.451554+0000","last_active":"2024-04-25T09:56:38.452779+0000","last_peered":"2024-04-25T09:56:38.452779+0000","last_clean":"2024-04-25T09:56:38.452779+0000","last_became_active":"2024-04-25T09:56:38.451253+0000","last_became_peered":"2024-04-25T09:56:38.451253+0000","last_unstale":"2024-04-25T09:56:38.452779+0000","last_undegraded":"2024-04-25T09:56:38.452779+0000","last_fullsized":"2024-04-25T09:56:38.452779+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:44.723554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438457+0000","last_change":"2024-04-25T09:56:38.438457+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438457+0000","last_undegraded":"2024-04-25T09:56:38.438457+0000","last_fullsized":"2024-04-25T09:56:38.438457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454400+0000","last_change":"2024-04-25T09:56:38.452908+0000","last_active":"2024-04-25T09:56:38.454400+0000","last_peered":"2024-04-25T09:56:38.454400+0000","last_clean":"2024-04-25T09:56:38.454400+0000","last_became_active":"2024-04-25T09:56:38.452672+0000","last_became_peered":"2024-04-25T09:56:38.452672+0000","last_unstale":"2024-04-25T09:56:38.454400+0000","last_undegraded":"2024-04-25T09:56:38.454400+0000","last_fullsized":"2024-04-25T09:56:38.454400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.108179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431918+0000","last_change":"2024-04-25T09:56:38.431918+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431918+0000","last_undegraded":"2024-04-25T09:56:38.431918+0000","last_fullsized":"2024-04-25T09:56:38.431918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450753+0000","last_change":"2024-04-25T09:56:38.441289+0000","last_active":"2024-04-25T09:56:38.450753+0000","last_peered":"2024-04-25T09:56:38.450753+0000","last_clean":"2024-04-25T09:56:38.450753+0000","last_became_active":"2024-04-25T09:56:38.440974+0000","last_became_peered":"2024-04-25T09:56:38.440974+0000","last_unstale":"2024-04-25T09:56:38.450753+0000","last_undegraded":"2024-04-25T09:56:38.450753+0000","last_fullsized":"2024-04-25T09:56:38.450753+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:06.200902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431725+0000","last_change":"2024-04-25T09:56:38.431725+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431725+0000","last_undegraded":"2024-04-25T09:56:38.431725+0000","last_fullsized":"2024-04-25T09:56:38.431725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.444935+0000","last_change":"2024-04-25T09:56:38.444152+0000","last_active":"2024-04-25T09:56:38.444935+0000","last_peered":"2024-04-25T09:56:38.444935+0000","last_clean":"2024-04-25T09:56:38.444935+0000","last_became_active":"2024-04-25T09:56:38.443891+0000","last_became_peered":"2024-04-25T09:56:38.443891+0000","last_unstale":"2024-04-25T09:56:38.444935+0000","last_undegraded":"2024-04-25T09:56:38.444935+0000","last_fullsized":"2024-04-25T09:56:38.444935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.635113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436850+0000","last_change":"2024-04-25T09:56:38.436850+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436850+0000","last_undegraded":"2024-04-25T09:56:38.436850+0000","last_fullsized":"2024-04-25T09:56:38.436850+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454025+0000","last_change":"2024-04-25T09:56:38.452182+0000","last_active":"2024-04-25T09:56:38.454025+0000","last_peered":"2024-04-25T09:56:38.454025+0000","last_clean":"2024-04-25T09:56:38.454025+0000","last_became_active":"2024-04-25T09:56:38.451492+0000","last_became_peered":"2024-04-25T09:56:38.451492+0000","last_unstale":"2024-04-25T09:56:38.454025+0000","last_undegraded":"2024-04-25T09:56:38.454025+0000","last_fullsized":"2024-04-25T09:56:38.454025+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:33.806055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.438322+0000","last_change":"2024-04-25T09:56:38.438322+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.438322+0000","last_undegraded":"2024-04-25T09:56:38.438322+0000","last_fullsized":"2024-04-25T09:56:38.438322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452814+0000","last_change":"2024-04-25T09:56:38.449599+0000","last_active":"2024-04-25T09:56:38.452814+0000","last_peered":"2024-04-25T09:56:38.452814+0000","last_clean":"2024-04-25T09:56:38.452814+0000","last_became_active":"2024-04-25T09:56:38.449302+0000","last_became_peered":"2024-04-25T09:56:38.449302+0000","last_unstale":"2024-04-25T09:56:38.452814+0000","last_undegraded":"2024-04-25T09:56:38.452814+0000","last_fullsized":"2024-04-25T09:56:38.452814+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.404639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433556+0000","last_change":"2024-04-25T09:56:38.433556+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433556+0000","last_undegraded":"2024-04-25T09:56:38.433556+0000","last_fullsized":"2024-04-25T09:56:38.433556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448561+0000","last_change":"2024-04-25T09:56:38.447935+0000","last_active":"2024-04-25T09:56:38.448561+0000","last_peered":"2024-04-25T09:56:38.448561+0000","last_clean":"2024-04-25T09:56:38.448561+0000","last_became_active":"2024-04-25T09:56:38.447648+0000","last_became_peered":"2024-04-25T09:56:38.447648+0000","last_unstale":"2024-04-25T09:56:38.448561+0000","last_undegraded":"2024-04-25T09:56:38.448561+0000","last_fullsized":"2024-04-25T09:56:38.448561+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:32.556063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433134+0000","last_change":"2024-04-25T09:56:38.433134+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433134+0000","last_undegraded":"2024-04-25T09:56:38.433134+0000","last_fullsized":"2024-04-25T09:56:38.433134+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452679+0000","last_change":"2024-04-25T09:56:38.451735+0000","last_active":"2024-04-25T09:56:38.452679+0000","last_peered":"2024-04-25T09:56:38.452679+0000","last_clean":"2024-04-25T09:56:38.452679+0000","last_became_active":"2024-04-25T09:56:38.451108+0000","last_became_peered":"2024-04-25T09:56:38.451108+0000","last_unstale":"2024-04-25T09:56:38.452679+0000","last_undegraded":"2024-04-25T09:56:38.452679+0000","last_fullsized":"2024-04-25T09:56:38.452679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:34.259055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436880+0000","last_change":"2024-04-25T09:56:38.436880+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436880+0000","last_undegraded":"2024-04-25T09:56:38.436880+0000","last_fullsized":"2024-04-25T09:56:38.436880+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452467+0000","last_change":"2024-04-25T09:56:38.449467+0000","last_active":"2024-04-25T09:56:38.452467+0000","last_peered":"2024-04-25T09:56:38.452467+0000","last_clean":"2024-04-25T09:56:38.452467+0000","last_became_active":"2024-04-25T09:56:38.449110+0000","last_became_peered":"2024-04-25T09:56:38.449110+0000","last_unstale":"2024-04-25T09:56:38.452467+0000","last_undegraded":"2024-04-25T09:56:38.452467+0000","last_fullsized":"2024-04-25T09:56:38.452467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:34.259055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435023+0000","last_change":"2024-04-25T09:56:38.435023+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435023+0000","last_undegraded":"2024-04-25T09:56:38.435023+0000","last_fullsized":"2024-04-25T09:56:38.435023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454387+0000","last_change":"2024-04-25T09:56:38.452721+0000","last_active":"2024-04-25T09:56:38.454387+0000","last_peered":"2024-04-25T09:56:38.454387+0000","last_clean":"2024-04-25T09:56:38.454387+0000","last_became_active":"2024-04-25T09:56:38.452370+0000","last_became_peered":"2024-04-25T09:56:38.452370+0000","last_unstale":"2024-04-25T09:56:38.454387+0000","last_undegraded":"2024-04-25T09:56:38.454387+0000","last_fullsized":"2024-04-25T09:56:38.454387+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:59.539232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437996+0000","last_change":"2024-04-25T09:56:38.437996+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437996+0000","last_undegraded":"2024-04-25T09:56:38.437996+0000","last_fullsized":"2024-04-25T09:56:38.437996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.441954+0000","last_change":"2024-04-25T09:56:38.441046+0000","last_active":"2024-04-25T09:56:38.441954+0000","last_peered":"2024-04-25T09:56:38.441954+0000","last_clean":"2024-04-25T09:56:38.441954+0000","last_became_active":"2024-04-25T09:56:38.440736+0000","last_became_peered":"2024-04-25T09:56:38.440736+0000","last_unstale":"2024-04-25T09:56:38.441954+0000","last_undegraded":"2024-04-25T09:56:38.441954+0000","last_fullsized":"2024-04-25T09:56:38.441954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:10.273708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434042+0000","last_change":"2024-04-25T09:56:38.434042+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434042+0000","last_undegraded":"2024-04-25T09:56:38.434042+0000","last_fullsized":"2024-04-25T09:56:38.434042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450475+0000","last_change":"2024-04-25T09:56:38.443577+0000","last_active":"2024-04-25T09:56:38.450475+0000","last_peered":"2024-04-25T09:56:38.450475+0000","last_clean":"2024-04-25T09:56:38.450475+0000","last_became_active":"2024-04-25T09:56:38.443304+0000","last_became_peered":"2024-04-25T09:56:38.443304+0000","last_unstale":"2024-04-25T09:56:38.450475+0000","last_undegraded":"2024-04-25T09:56:38.450475+0000","last_fullsized":"2024-04-25T09:56:38.450475+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.933751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436494+0000","last_change":"2024-04-25T09:56:38.436494+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436494+0000","last_undegraded":"2024-04-25T09:56:38.436494+0000","last_fullsized":"2024-04-25T09:56:38.436494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.454005+0000","last_change":"2024-04-25T09:56:38.451937+0000","last_active":"2024-04-25T09:56:38.454005+0000","last_peered":"2024-04-25T09:56:38.454005+0000","last_clean":"2024-04-25T09:56:38.454005+0000","last_became_active":"2024-04-25T09:56:38.451248+0000","last_became_peered":"2024-04-25T09:56:38.451248+0000","last_unstale":"2024-04-25T09:56:38.454005+0000","last_undegraded":"2024-04-25T09:56:38.454005+0000","last_fullsized":"2024-04-25T09:56:38.454005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:04.404639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435576+0000","last_change":"2024-04-25T09:56:38.435576+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435576+0000","last_undegraded":"2024-04-25T09:56:38.435576+0000","last_fullsized":"2024-04-25T09:56:38.435576+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450527+0000","last_change":"2024-04-25T09:56:38.446452+0000","last_active":"2024-04-25T09:56:38.450527+0000","last_peered":"2024-04-25T09:56:38.450527+0000","last_clean":"2024-04-25T09:56:38.450527+0000","last_became_active":"2024-04-25T09:56:38.446166+0000","last_became_peered":"2024-04-25T09:56:38.446166+0000","last_unstale":"2024-04-25T09:56:38.450527+0000","last_undegraded":"2024-04-25T09:56:38.450527+0000","last_fullsized":"2024-04-25T09:56:38.450527+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:10.912571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.435303+0000","last_change":"2024-04-25T09:56:38.435303+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.435303+0000","last_undegraded":"2024-04-25T09:56:38.435303+0000","last_fullsized":"2024-04-25T09:56:38.435303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.448987+0000","last_change":"2024-04-25T09:56:38.447857+0000","last_active":"2024-04-25T09:56:38.448987+0000","last_peered":"2024-04-25T09:56:38.448987+0000","last_clean":"2024-04-25T09:56:38.448987+0000","last_became_active":"2024-04-25T09:56:38.447519+0000","last_became_peered":"2024-04-25T09:56:38.447519+0000","last_unstale":"2024-04-25T09:56:38.448987+0000","last_undegraded":"2024-04-25T09:56:38.448987+0000","last_fullsized":"2024-04-25T09:56:38.448987+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:32.728995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.443451+0000","last_change":"2024-04-25T09:56:38.443451+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.443451+0000","last_undegraded":"2024-04-25T09:56:38.443451+0000","last_fullsized":"2024-04-25T09:56:38.443451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449375+0000","last_change":"2024-04-25T09:56:38.448408+0000","last_active":"2024-04-25T09:56:38.449375+0000","last_peered":"2024-04-25T09:56:38.449375+0000","last_clean":"2024-04-25T09:56:38.449375+0000","last_became_active":"2024-04-25T09:56:38.448024+0000","last_became_peered":"2024-04-25T09:56:38.448024+0000","last_unstale":"2024-04-25T09:56:38.449375+0000","last_undegraded":"2024-04-25T09:56:38.449375+0000","last_fullsized":"2024-04-25T09:56:38.449375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.635113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.434466+0000","last_change":"2024-04-25T09:56:38.434466+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.434466+0000","last_undegraded":"2024-04-25T09:56:38.434466+0000","last_fullsized":"2024-04-25T09:56:38.434466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.449665+0000","last_change":"2024-04-25T09:56:38.442585+0000","last_active":"2024-04-25T09:56:38.449665+0000","last_peered":"2024-04-25T09:56:38.449665+0000","last_clean":"2024-04-25T09:56:38.449665+0000","last_became_active":"2024-04-25T09:56:38.442310+0000","last_became_peered":"2024-04-25T09:56:38.442310+0000","last_unstale":"2024-04-25T09:56:38.449665+0000","last_undegraded":"2024-04-25T09:56:38.449665+0000","last_fullsized":"2024-04-25T09:56:38.449665+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.437364+0000","last_change":"2024-04-25T09:56:38.437364+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.437364+0000","last_undegraded":"2024-04-25T09:56:38.437364+0000","last_fullsized":"2024-04-25T09:56:38.437364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.450830+0000","last_change":"2024-04-25T09:56:38.442079+0000","last_active":"2024-04-25T09:56:38.450830+0000","last_peered":"2024-04-25T09:56:38.450830+0000","last_clean":"2024-04-25T09:56:38.450830+0000","last_became_active":"2024-04-25T09:56:38.441787+0000","last_became_peered":"2024-04-25T09:56:38.441787+0000","last_unstale":"2024-04-25T09:56:38.450830+0000","last_undegraded":"2024-04-25T09:56:38.450830+0000","last_fullsized":"2024-04-25T09:56:38.450830+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.776063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436133+0000","last_change":"2024-04-25T09:56:38.436133+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436133+0000","last_undegraded":"2024-04-25T09:56:38.436133+0000","last_fullsized":"2024-04-25T09:56:38.436133+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.452759+0000","last_change":"2024-04-25T09:56:38.450312+0000","last_active":"2024-04-25T09:56:38.452759+0000","last_peered":"2024-04-25T09:56:38.452759+0000","last_clean":"2024-04-25T09:56:38.452759+0000","last_became_active":"2024-04-25T09:56:38.450049+0000","last_became_peered":"2024-04-25T09:56:38.450049+0000","last_unstale":"2024-04-25T09:56:38.452759+0000","last_undegraded":"2024-04-25T09:56:38.452759+0000","last_fullsized":"2024-04-25T09:56:38.452759+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:33.806055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.431039+0000","last_change":"2024-04-25T09:56:38.431039+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.431039+0000","last_undegraded":"2024-04-25T09:56:38.431039+0000","last_fullsized":"2024-04-25T09:56:38.431039+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.443404+0000","last_change":"2024-04-25T09:56:38.441913+0000","last_active":"2024-04-25T09:56:38.443404+0000","last_peered":"2024-04-25T09:56:38.443404+0000","last_clean":"2024-04-25T09:56:38.443404+0000","last_became_active":"2024-04-25T09:56:38.441584+0000","last_became_peered":"2024-04-25T09:56:38.441584+0000","last_unstale":"2024-04-25T09:56:38.443404+0000","last_undegraded":"2024-04-25T09:56:38.443404+0000","last_fullsized":"2024-04-25T09:56:38.443404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:06.302131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.433502+0000","last_change":"2024-04-25T09:56:38.433502+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.433502+0000","last_undegraded":"2024-04-25T09:56:38.433502+0000","last_fullsized":"2024-04-25T09:56:38.433502+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.447445+0000","last_change":"2024-04-25T09:56:38.444951+0000","last_active":"2024-04-25T09:56:38.447445+0000","last_peered":"2024-04-25T09:56:38.447445+0000","last_clean":"2024-04-25T09:56:38.447445+0000","last_became_active":"2024-04-25T09:56:38.444617+0000","last_became_peered":"2024-04-25T09:56:38.444617+0000","last_unstale":"2024-04-25T09:56:38.447445+0000","last_undegraded":"2024-04-25T09:56:38.447445+0000","last_fullsized":"2024-04-25T09:56:38.447445+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.557978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.443248+0000","last_change":"2024-04-25T09:56:38.443248+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.443248+0000","last_undegraded":"2024-04-25T09:56:38.443248+0000","last_fullsized":"2024-04-25T09:56:38.443248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.451069+0000","last_change":"2024-04-25T09:56:38.447517+0000","last_active":"2024-04-25T09:56:38.451069+0000","last_peered":"2024-04-25T09:56:38.451069+0000","last_clean":"2024-04-25T09:56:38.451069+0000","last_became_active":"2024-04-25T09:56:38.447204+0000","last_became_peered":"2024-04-25T09:56:38.447204+0000","last_unstale":"2024-04-25T09:56:38.451069+0000","last_undegraded":"2024-04-25T09:56:38.451069+0000","last_fullsized":"2024-04-25T09:56:38.451069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:34.630001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-04-25T09:56:38.436966+0000","last_change":"2024-04-25T09:56:38.436966+0000","last_active":"2024-04-25T09:56:38.425967+0000","last_peered":"2024-04-25T09:56:38.425967+0000","last_clean":"2024-04-25T09:56:38.425967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:56:38.436966+0000","last_undegraded":"2024-04-25T09:56:38.436966+0000","last_fullsized":"2024-04-25T09:56:38.436966+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.446681+0000","last_change":"2024-04-25T09:56:38.443664+0000","last_active":"2024-04-25T09:56:38.446681+0000","last_peered":"2024-04-25T09:56:38.446681+0000","last_clean":"2024-04-25T09:56:38.446681+0000","last_became_active":"2024-04-25T09:56:38.443253+0000","last_became_peered":"2024-04-25T09:56:38.443253+0000","last_unstale":"2024-04-25T09:56:38.446681+0000","last_undegraded":"2024-04-25T09:56:38.446681+0000","last_fullsized":"2024-04-25T09:56:38.446681+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:49.974254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":255,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:56:38.435835+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:56:38.435835+0000","last_peered":"2024-04-25T09:56:38.435835+0000","last_clean":"2024-04-25T09:56:38.435835+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:56:38.435835+0000","last_undegraded":"2024-04-25T09:56:38.435835+0000","last_fullsized":"2024-04-25T09:56:38.435835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36652,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335188,"statfs":{"total":96636764160,"available":96599232512,"internally_reserved":0,"allocated":1343488,"data_stored":927508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36076,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335764,"statfs":{"total":96636764160,"available":96599822336,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36076,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335764,"statfs":{"total":96636764160,"available":96599822336,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":737280,"data_stored":330481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36080,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335760,"statfs":{"total":96636764160,"available":96599818240,"internally_reserved":0,"allocated":749568,"data_stored":337140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":737280,"data_stored":330481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36644,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335196,"statfs":{"total":96636764160,"available":96599240704,"internally_reserved":0,"allocated":1331200,"data_stored":920849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:56:46.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:56:46.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.993+0000 7fecd13f5640 1 Processor -- start 2024-04-25T09:56:46.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.994+0000 7fecd13f5640 1 -- start start 2024-04-25T09:56:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.994+0000 7fecd13f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feccc154ee0 0x7feccc0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.994+0000 7fecd13f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feccc0cfcf0 0x7feccc0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.994+0000 7fecd13f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc0ce3c0 0x7feccc15e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.994+0000 7fecd13f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feccc0ce830 con 0x7feccc0cfcf0 2024-04-25T09:56:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.994+0000 7fecd13f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feccc072090 con 0x7feccc154ee0 2024-04-25T09:56:46.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.994+0000 7fecd13f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feccc071230 con 0x7feccc0ce3c0 2024-04-25T09:56:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.995+0000 7fecca7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feccc0cfcf0 0x7feccc0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.995+0000 7fecca7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feccc0cfcf0 0x7feccc0cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53324/0 (socket says 172.21.15.19:53324) 2024-04-25T09:56:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.995+0000 7fecca7fc640 1 -- 172.21.15.19:0/203635467 learned_addr learned my addr 172.21.15.19:0/203635467 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.995+0000 7feccaffd640 1 --2- 172.21.15.19:0/203635467 >> v2:172.21.15.112:3300/0 conn(0x7feccc154ee0 0x7feccc0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.995+0000 7feccb7fe640 1 --2- 172.21.15.19:0/203635467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc0ce3c0 0x7feccc15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.995+0000 7fecca7fc640 1 -- 172.21.15.19:0/203635467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc0ce3c0 msgr2=0x7feccc15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.995+0000 7fecca7fc640 1 --2- 172.21.15.19:0/203635467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc0ce3c0 0x7feccc15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.995+0000 7fecca7fc640 1 -- 172.21.15.19:0/203635467 >> v2:172.21.15.112:3300/0 conn(0x7feccc154ee0 msgr2=0x7feccc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.995+0000 7fecca7fc640 1 --2- 172.21.15.19:0/203635467 >> v2:172.21.15.112:3300/0 conn(0x7feccc154ee0 0x7feccc0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.995+0000 7fecca7fc640 1 -- 172.21.15.19:0/203635467 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc071480 con 0x7feccc0cfcf0 2024-04-25T09:56:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.995+0000 7feccaffd640 1 --2- 172.21.15.19:0/203635467 >> v2:172.21.15.112:3300/0 conn(0x7feccc154ee0 0x7feccc0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:47.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.996+0000 7fecca7fc640 1 --2- 172.21.15.19:0/203635467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0cfcf0 0x7feccc0cde80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7feccc0d02e0 tx=0x7fecb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26c10ae3fc996af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:47.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.996+0000 7fecc9ffb640 1 -- 172.21.15.19:0/203635467 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4037070 con 0x7feccc0cfcf0 2024-04-25T09:56:47.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.996+0000 7fecc9ffb640 1 -- 172.21.15.19:0/203635467 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb4032440 con 0x7feccc0cfcf0 2024-04-25T09:56:47.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.996+0000 7fecc9ffb640 1 -- 172.21.15.19:0/203635467 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4032810 con 0x7feccc0cfcf0 2024-04-25T09:56:47.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.996+0000 7fecd13f5640 1 -- 172.21.15.19:0/203635467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0cfcf0 msgr2=0x7feccc0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.996+0000 7fecd13f5640 1 --2- 172.21.15.19:0/203635467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0cfcf0 0x7feccc0cde80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7feccc0d02e0 tx=0x7fecb402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.997+0000 7fecd13f5640 1 -- 172.21.15.19:0/203635467 shutdown_connections 2024-04-25T09:56:47.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.997+0000 7fecd13f5640 1 --2- 172.21.15.19:0/203635467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc0ce3c0 0x7feccc15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.997+0000 7fecd13f5640 1 --2- 172.21.15.19:0/203635467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0cfcf0 0x7feccc0cde80 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.997+0000 7fecd13f5640 1 --2- 172.21.15.19:0/203635467 >> v2:172.21.15.112:3300/0 conn(0x7feccc154ee0 0x7feccc0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.997+0000 7fecd13f5640 1 -- 172.21.15.19:0/203635467 >> 172.21.15.19:0/203635467 conn(0x7feccc06abe0 msgr2=0x7feccc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:47.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.997+0000 7fecd13f5640 1 -- 172.21.15.19:0/203635467 shutdown_connections 2024-04-25T09:56:47.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.997+0000 7fecd13f5640 1 -- 172.21.15.19:0/203635467 wait complete. 2024-04-25T09:56:47.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.998+0000 7fecd13f5640 1 Processor -- start 2024-04-25T09:56:47.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.998+0000 7fecd13f5640 1 -- start start 2024-04-25T09:56:47.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7fecd13f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0ce3c0 0x7feccc14af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7fecd13f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feccc0cfcf0 0x7feccc15d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7fecd13f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc154ee0 0x7feccc15d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7fecd13f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc071d20 con 0x7feccc0ce3c0 2024-04-25T09:56:47.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7fecd13f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc071a40 con 0x7feccc0cfcf0 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7fecd13f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feccc1552c0 con 0x7feccc154ee0 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7feccaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0ce3c0 0x7feccc14af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7feccaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0ce3c0 0x7feccc14af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53340/0 (socket says 172.21.15.19:53340) 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7feccaffd640 1 -- 172.21.15.19:0/4286642633 learned_addr learned my addr 172.21.15.19:0/4286642633 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7feccb7fe640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc154ee0 0x7feccc15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7fecca7fc640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feccc0cfcf0 0x7feccc15d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7feccaffd640 1 -- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc154ee0 msgr2=0x7feccc15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7feccaffd640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc154ee0 0x7feccc15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7feccaffd640 1 -- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feccc0cfcf0 msgr2=0x7feccc15d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7feccaffd640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feccc0cfcf0 0x7feccc15d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:46.999+0000 7feccaffd640 1 -- 172.21.15.19:0/4286642633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc156c90 con 0x7feccc0ce3c0 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.000+0000 7feccaffd640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0ce3c0 0x7feccc14af90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fecc4007ef0 tx=0x7fecc400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.000+0000 7fecc37fe640 1 -- 172.21.15.19:0/4286642633 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc401a070 con 0x7feccc0ce3c0 2024-04-25T09:56:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.000+0000 7fecc37fe640 1 -- 172.21.15.19:0/4286642633 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc4003b90 con 0x7feccc0ce3c0 2024-04-25T09:56:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.000+0000 7fecc37fe640 1 -- 172.21.15.19:0/4286642633 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4004030 con 0x7feccc0ce3c0 2024-04-25T09:56:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.000+0000 7fecd13f5640 1 -- 172.21.15.19:0/4286642633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feccc0b3350 con 0x7feccc0ce3c0 2024-04-25T09:56:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.000+0000 7fecd13f5640 1 -- 172.21.15.19:0/4286642633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feccc0b3910 con 0x7feccc0ce3c0 2024-04-25T09:56:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.001+0000 7fecd13f5640 1 -- 172.21.15.19:0/4286642633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feccc0b1d10 con 0x7feccc0ce3c0 2024-04-25T09:56:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.001+0000 7fecc37fe640 1 -- 172.21.15.19:0/4286642633 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecc401d070 con 0x7feccc0ce3c0 2024-04-25T09:56:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.002+0000 7fecc37fe640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecac06b4d0 0x7fecac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.002+0000 7fecc37fe640 1 -- 172.21.15.19:0/4286642633 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fecc408a480 con 0x7feccc0ce3c0 2024-04-25T09:56:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.002+0000 7fecca7fc640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecac06b4d0 0x7fecac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.002+0000 7fecca7fc640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecac06b4d0 0x7fecac06d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fecb4034000 tx=0x7fecb402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:47.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.006+0000 7fecc37fe640 1 -- 172.21.15.19:0/4286642633 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feccc0b1d10 con 0x7feccc0ce3c0 2024-04-25T09:56:47.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.185+0000 7fecd13f5640 1 -- 172.21.15.19:0/4286642633 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feccc073990 con 0x7fecac06b4d0 2024-04-25T09:56:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.191+0000 7fecc37fe640 1 -- 172.21.15.19:0/4286642633 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252019 (secure 0 0 0) 0x7feccc073990 con 0x7fecac06b4d0 2024-04-25T09:56:47.197 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:56:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.194+0000 7fecd13f5640 1 -- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecac06b4d0 msgr2=0x7fecac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.194+0000 7fecd13f5640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecac06b4d0 0x7fecac06d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fecb4034000 tx=0x7fecb402e000 comp rx=0 tx=0).stop 2024-04-25T09:56:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.194+0000 7fecd13f5640 1 -- 172.21.15.19:0/4286642633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0ce3c0 msgr2=0x7feccc14af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.194+0000 7fecd13f5640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0ce3c0 0x7feccc14af90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fecc4007ef0 tx=0x7fecc400d4a0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.195+0000 7fecd13f5640 1 -- 172.21.15.19:0/4286642633 shutdown_connections 2024-04-25T09:56:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.195+0000 7fecd13f5640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecac06b4d0 0x7fecac06d990 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.195+0000 7fecd13f5640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc154ee0 0x7feccc15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.195+0000 7fecd13f5640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feccc0cfcf0 0x7feccc15d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.195+0000 7fecd13f5640 1 --2- 172.21.15.19:0/4286642633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0ce3c0 0x7feccc14af90 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.195+0000 7fecd13f5640 1 -- 172.21.15.19:0/4286642633 >> 172.21.15.19:0/4286642633 conn(0x7feccc06abe0 msgr2=0x7feccc06e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.195+0000 7fecd13f5640 1 -- 172.21.15.19:0/4286642633 shutdown_connections 2024-04-25T09:56:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.195+0000 7fecd13f5640 1 -- 172.21.15.19:0/4286642633 wait complete. 2024-04-25T09:56:47.209 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-04-25T09:56:46.962928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290008,"kb_used_data":7272,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754684712,"statfs":{"total":773094113280,"available":772797145088,"internally_reserved":0,"allocated":7446528,"data_stored":3993548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002340"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758179+0000","last_change":"2024-04-25T09:56:39.460914+0000","last_active":"2024-04-25T09:56:41.758179+0000","last_peered":"2024-04-25T09:56:41.758179+0000","last_clean":"2024-04-25T09:56:41.758179+0000","last_became_active":"2024-04-25T09:56:39.460608+0000","last_became_peered":"2024-04-25T09:56:39.460608+0000","last_unstale":"2024-04-25T09:56:41.758179+0000","last_undegraded":"2024-04-25T09:56:41.758179+0000","last_fullsized":"2024-04-25T09:56:41.758179+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:03.657765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758102+0000","last_change":"2024-04-25T09:56:39.462242+0000","last_active":"2024-04-25T09:56:41.758102+0000","last_peered":"2024-04-25T09:56:41.758102+0000","last_clean":"2024-04-25T09:56:41.758102+0000","last_became_active":"2024-04-25T09:56:39.461561+0000","last_became_peered":"2024-04-25T09:56:39.461561+0000","last_unstale":"2024-04-25T09:56:41.758102+0000","last_undegraded":"2024-04-25T09:56:41.758102+0000","last_fullsized":"2024-04-25T09:56:41.758102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:20.949554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.442071+0000","last_change":"2024-04-25T09:56:39.461847+0000","last_active":"2024-04-25T09:56:40.442071+0000","last_peered":"2024-04-25T09:56:40.442071+0000","last_clean":"2024-04-25T09:56:40.442071+0000","last_became_active":"2024-04-25T09:56:39.461468+0000","last_became_peered":"2024-04-25T09:56:39.461468+0000","last_unstale":"2024-04-25T09:56:40.442071+0000","last_undegraded":"2024-04-25T09:56:40.442071+0000","last_fullsized":"2024-04-25T09:56:40.442071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:53.184942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.442260+0000","last_change":"2024-04-25T09:56:39.462873+0000","last_active":"2024-04-25T09:56:40.442260+0000","last_peered":"2024-04-25T09:56:40.442260+0000","last_clean":"2024-04-25T09:56:40.442260+0000","last_became_active":"2024-04-25T09:56:39.461408+0000","last_became_peered":"2024-04-25T09:56:39.461408+0000","last_unstale":"2024-04-25T09:56:40.442260+0000","last_undegraded":"2024-04-25T09:56:40.442260+0000","last_fullsized":"2024-04-25T09:56:40.442260+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:59.080083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758424+0000","last_change":"2024-04-25T09:56:39.464422+0000","last_active":"2024-04-25T09:56:41.758424+0000","last_peered":"2024-04-25T09:56:41.758424+0000","last_clean":"2024-04-25T09:56:41.758424+0000","last_became_active":"2024-04-25T09:56:39.464148+0000","last_became_peered":"2024-04-25T09:56:39.464148+0000","last_unstale":"2024-04-25T09:56:41.758424+0000","last_undegraded":"2024-04-25T09:56:41.758424+0000","last_fullsized":"2024-04-25T09:56:41.758424+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:22.403165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758556+0000","last_change":"2024-04-25T09:56:39.466015+0000","last_active":"2024-04-25T09:56:41.758556+0000","last_peered":"2024-04-25T09:56:41.758556+0000","last_clean":"2024-04-25T09:56:41.758556+0000","last_became_active":"2024-04-25T09:56:39.465121+0000","last_became_peered":"2024-04-25T09:56:39.465121+0000","last_unstale":"2024-04-25T09:56:41.758556+0000","last_undegraded":"2024-04-25T09:56:41.758556+0000","last_fullsized":"2024-04-25T09:56:41.758556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:04.998080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.458870+0000","last_change":"2024-04-25T09:56:39.454310+0000","last_active":"2024-04-25T09:56:40.458870+0000","last_peered":"2024-04-25T09:56:40.458870+0000","last_clean":"2024-04-25T09:56:40.458870+0000","last_became_active":"2024-04-25T09:56:39.454045+0000","last_became_peered":"2024-04-25T09:56:39.454045+0000","last_unstale":"2024-04-25T09:56:40.458870+0000","last_undegraded":"2024-04-25T09:56:40.458870+0000","last_fullsized":"2024-04-25T09:56:40.458870+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:47.914647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.457100+0000","last_change":"2024-04-25T09:56:39.454163+0000","last_active":"2024-04-25T09:56:40.457100+0000","last_peered":"2024-04-25T09:56:40.457100+0000","last_clean":"2024-04-25T09:56:40.457100+0000","last_became_active":"2024-04-25T09:56:39.453557+0000","last_became_peered":"2024-04-25T09:56:39.453557+0000","last_unstale":"2024-04-25T09:56:40.457100+0000","last_undegraded":"2024-04-25T09:56:40.457100+0000","last_fullsized":"2024-04-25T09:56:40.457100+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:05.783663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441550+0000","last_change":"2024-04-25T09:56:39.460218+0000","last_active":"2024-04-25T09:56:40.441550+0000","last_peered":"2024-04-25T09:56:40.441550+0000","last_clean":"2024-04-25T09:56:40.441550+0000","last_became_active":"2024-04-25T09:56:39.459863+0000","last_became_peered":"2024-04-25T09:56:39.459863+0000","last_unstale":"2024-04-25T09:56:40.441550+0000","last_undegraded":"2024-04-25T09:56:40.441550+0000","last_fullsized":"2024-04-25T09:56:40.441550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:33.971258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441054+0000","last_change":"2024-04-25T09:56:39.452402+0000","last_active":"2024-04-25T09:56:40.441054+0000","last_peered":"2024-04-25T09:56:40.441054+0000","last_clean":"2024-04-25T09:56:40.441054+0000","last_became_active":"2024-04-25T09:56:39.452063+0000","last_became_peered":"2024-04-25T09:56:39.452063+0000","last_unstale":"2024-04-25T09:56:40.441054+0000","last_undegraded":"2024-04-25T09:56:40.441054+0000","last_fullsized":"2024-04-25T09:56:40.441054+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:50.146820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441107+0000","last_change":"2024-04-25T09:56:39.455896+0000","last_active":"2024-04-25T09:56:40.441107+0000","last_peered":"2024-04-25T09:56:40.441107+0000","last_clean":"2024-04-25T09:56:40.441107+0000","last_became_active":"2024-04-25T09:56:39.455610+0000","last_became_peered":"2024-04-25T09:56:39.455610+0000","last_unstale":"2024-04-25T09:56:40.441107+0000","last_undegraded":"2024-04-25T09:56:40.441107+0000","last_fullsized":"2024-04-25T09:56:40.441107+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:40.385464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.457049+0000","last_change":"2024-04-25T09:56:39.453096+0000","last_active":"2024-04-25T09:56:40.457049+0000","last_peered":"2024-04-25T09:56:40.457049+0000","last_clean":"2024-04-25T09:56:40.457049+0000","last_became_active":"2024-04-25T09:56:39.452824+0000","last_became_peered":"2024-04-25T09:56:39.452824+0000","last_unstale":"2024-04-25T09:56:40.457049+0000","last_undegraded":"2024-04-25T09:56:40.457049+0000","last_fullsized":"2024-04-25T09:56:40.457049+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:10.800936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.456749+0000","last_change":"2024-04-25T09:56:39.460984+0000","last_active":"2024-04-25T09:56:40.456749+0000","last_peered":"2024-04-25T09:56:40.456749+0000","last_clean":"2024-04-25T09:56:40.456749+0000","last_became_active":"2024-04-25T09:56:39.460713+0000","last_became_peered":"2024-04-25T09:56:39.460713+0000","last_unstale":"2024-04-25T09:56:40.456749+0000","last_undegraded":"2024-04-25T09:56:40.456749+0000","last_fullsized":"2024-04-25T09:56:40.456749+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:49.764274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441960+0000","last_change":"2024-04-25T09:56:39.463024+0000","last_active":"2024-04-25T09:56:40.441960+0000","last_peered":"2024-04-25T09:56:40.441960+0000","last_clean":"2024-04-25T09:56:40.441960+0000","last_became_active":"2024-04-25T09:56:39.462734+0000","last_became_peered":"2024-04-25T09:56:39.462734+0000","last_unstale":"2024-04-25T09:56:40.441960+0000","last_undegraded":"2024-04-25T09:56:40.441960+0000","last_fullsized":"2024-04-25T09:56:40.441960+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:06.108481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.442186+0000","last_change":"2024-04-25T09:56:39.449074+0000","last_active":"2024-04-25T09:56:40.442186+0000","last_peered":"2024-04-25T09:56:40.442186+0000","last_clean":"2024-04-25T09:56:40.442186+0000","last_became_active":"2024-04-25T09:56:39.448207+0000","last_became_peered":"2024-04-25T09:56:39.448207+0000","last_unstale":"2024-04-25T09:56:40.442186+0000","last_undegraded":"2024-04-25T09:56:40.442186+0000","last_fullsized":"2024-04-25T09:56:40.442186+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:02.621288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.458848+0000","last_change":"2024-04-25T09:56:39.457682+0000","last_active":"2024-04-25T09:56:40.458848+0000","last_peered":"2024-04-25T09:56:40.458848+0000","last_clean":"2024-04-25T09:56:40.458848+0000","last_became_active":"2024-04-25T09:56:39.456397+0000","last_became_peered":"2024-04-25T09:56:39.456397+0000","last_unstale":"2024-04-25T09:56:40.458848+0000","last_undegraded":"2024-04-25T09:56:40.458848+0000","last_fullsized":"2024-04-25T09:56:40.458848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:06.111221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758371+0000","last_change":"2024-04-25T09:56:39.460288+0000","last_active":"2024-04-25T09:56:41.758371+0000","last_peered":"2024-04-25T09:56:41.758371+0000","last_clean":"2024-04-25T09:56:41.758371+0000","last_became_active":"2024-04-25T09:56:39.460031+0000","last_became_peered":"2024-04-25T09:56:39.460031+0000","last_unstale":"2024-04-25T09:56:41.758371+0000","last_undegraded":"2024-04-25T09:56:41.758371+0000","last_fullsized":"2024-04-25T09:56:41.758371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:11.278226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441847+0000","last_change":"2024-04-25T09:56:39.461983+0000","last_active":"2024-04-25T09:56:40.441847+0000","last_peered":"2024-04-25T09:56:40.441847+0000","last_clean":"2024-04-25T09:56:40.441847+0000","last_became_active":"2024-04-25T09:56:39.461688+0000","last_became_peered":"2024-04-25T09:56:39.461688+0000","last_unstale":"2024-04-25T09:56:40.441847+0000","last_undegraded":"2024-04-25T09:56:40.441847+0000","last_fullsized":"2024-04-25T09:56:40.441847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:48.315384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758609+0000","last_change":"2024-04-25T09:56:39.459522+0000","last_active":"2024-04-25T09:56:41.758609+0000","last_peered":"2024-04-25T09:56:41.758609+0000","last_clean":"2024-04-25T09:56:41.758609+0000","last_became_active":"2024-04-25T09:56:39.459226+0000","last_became_peered":"2024-04-25T09:56:39.459226+0000","last_unstale":"2024-04-25T09:56:41.758609+0000","last_undegraded":"2024-04-25T09:56:41.758609+0000","last_fullsized":"2024-04-25T09:56:41.758609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:34.810573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.440172+0000","last_change":"2024-04-25T09:56:39.457655+0000","last_active":"2024-04-25T09:56:40.440172+0000","last_peered":"2024-04-25T09:56:40.440172+0000","last_clean":"2024-04-25T09:56:40.440172+0000","last_became_active":"2024-04-25T09:56:39.457375+0000","last_became_peered":"2024-04-25T09:56:39.457375+0000","last_unstale":"2024-04-25T09:56:40.440172+0000","last_undegraded":"2024-04-25T09:56:40.440172+0000","last_fullsized":"2024-04-25T09:56:40.440172+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:53.256258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.456867+0000","last_change":"2024-04-25T09:56:39.454180+0000","last_active":"2024-04-25T09:56:40.456867+0000","last_peered":"2024-04-25T09:56:40.456867+0000","last_clean":"2024-04-25T09:56:40.456867+0000","last_became_active":"2024-04-25T09:56:39.453875+0000","last_became_peered":"2024-04-25T09:56:39.453875+0000","last_unstale":"2024-04-25T09:56:40.456867+0000","last_undegraded":"2024-04-25T09:56:40.456867+0000","last_fullsized":"2024-04-25T09:56:40.456867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:50.146820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.439973+0000","last_change":"2024-04-25T09:56:39.458960+0000","last_active":"2024-04-25T09:56:40.439973+0000","last_peered":"2024-04-25T09:56:40.439973+0000","last_clean":"2024-04-25T09:56:40.439973+0000","last_became_active":"2024-04-25T09:56:39.458671+0000","last_became_peered":"2024-04-25T09:56:39.458671+0000","last_unstale":"2024-04-25T09:56:40.439973+0000","last_undegraded":"2024-04-25T09:56:40.439973+0000","last_fullsized":"2024-04-25T09:56:40.439973+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:32.243065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.439948+0000","last_change":"2024-04-25T09:56:39.459114+0000","last_active":"2024-04-25T09:56:40.439948+0000","last_peered":"2024-04-25T09:56:40.439948+0000","last_clean":"2024-04-25T09:56:40.439948+0000","last_became_active":"2024-04-25T09:56:39.458173+0000","last_became_peered":"2024-04-25T09:56:39.458173+0000","last_unstale":"2024-04-25T09:56:40.439948+0000","last_undegraded":"2024-04-25T09:56:40.439948+0000","last_fullsized":"2024-04-25T09:56:40.439948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:49.569548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441218+0000","last_change":"2024-04-25T09:56:39.462881+0000","last_active":"2024-04-25T09:56:40.441218+0000","last_peered":"2024-04-25T09:56:40.441218+0000","last_clean":"2024-04-25T09:56:40.441218+0000","last_became_active":"2024-04-25T09:56:39.462289+0000","last_became_peered":"2024-04-25T09:56:39.462289+0000","last_unstale":"2024-04-25T09:56:40.441218+0000","last_undegraded":"2024-04-25T09:56:40.441218+0000","last_fullsized":"2024-04-25T09:56:40.441218+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:08.461836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.457061+0000","last_change":"2024-04-25T09:56:39.453907+0000","last_active":"2024-04-25T09:56:40.457061+0000","last_peered":"2024-04-25T09:56:40.457061+0000","last_clean":"2024-04-25T09:56:40.457061+0000","last_became_active":"2024-04-25T09:56:39.453582+0000","last_became_peered":"2024-04-25T09:56:39.453582+0000","last_unstale":"2024-04-25T09:56:40.457061+0000","last_undegraded":"2024-04-25T09:56:40.457061+0000","last_fullsized":"2024-04-25T09:56:40.457061+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:33.971258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.456936+0000","last_change":"2024-04-25T09:56:39.444711+0000","last_active":"2024-04-25T09:56:40.456936+0000","last_peered":"2024-04-25T09:56:40.456936+0000","last_clean":"2024-04-25T09:56:40.456936+0000","last_became_active":"2024-04-25T09:56:39.444360+0000","last_became_peered":"2024-04-25T09:56:39.444360+0000","last_unstale":"2024-04-25T09:56:40.456936+0000","last_undegraded":"2024-04-25T09:56:40.456936+0000","last_fullsized":"2024-04-25T09:56:40.456936+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:08.461836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441285+0000","last_change":"2024-04-25T09:56:39.458514+0000","last_active":"2024-04-25T09:56:40.441285+0000","last_peered":"2024-04-25T09:56:40.441285+0000","last_clean":"2024-04-25T09:56:40.441285+0000","last_became_active":"2024-04-25T09:56:39.458242+0000","last_became_peered":"2024-04-25T09:56:39.458242+0000","last_unstale":"2024-04-25T09:56:40.441285+0000","last_undegraded":"2024-04-25T09:56:40.441285+0000","last_fullsized":"2024-04-25T09:56:40.441285+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:07.205419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.757965+0000","last_change":"2024-04-25T09:56:39.457370+0000","last_active":"2024-04-25T09:56:41.757965+0000","last_peered":"2024-04-25T09:56:41.757965+0000","last_clean":"2024-04-25T09:56:41.757965+0000","last_became_active":"2024-04-25T09:56:39.457111+0000","last_became_peered":"2024-04-25T09:56:39.457111+0000","last_unstale":"2024-04-25T09:56:41.757965+0000","last_undegraded":"2024-04-25T09:56:41.757965+0000","last_fullsized":"2024-04-25T09:56:41.757965+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:18.251416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441315+0000","last_change":"2024-04-25T09:56:39.460817+0000","last_active":"2024-04-25T09:56:40.441315+0000","last_peered":"2024-04-25T09:56:40.441315+0000","last_clean":"2024-04-25T09:56:40.441315+0000","last_became_active":"2024-04-25T09:56:39.460556+0000","last_became_peered":"2024-04-25T09:56:39.460556+0000","last_unstale":"2024-04-25T09:56:40.441315+0000","last_undegraded":"2024-04-25T09:56:40.441315+0000","last_fullsized":"2024-04-25T09:56:40.441315+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:21.806485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.456715+0000","last_change":"2024-04-25T09:56:39.451288+0000","last_active":"2024-04-25T09:56:40.456715+0000","last_peered":"2024-04-25T09:56:40.456715+0000","last_clean":"2024-04-25T09:56:40.456715+0000","last_became_active":"2024-04-25T09:56:39.451008+0000","last_became_peered":"2024-04-25T09:56:39.451008+0000","last_unstale":"2024-04-25T09:56:40.456715+0000","last_undegraded":"2024-04-25T09:56:40.456715+0000","last_fullsized":"2024-04-25T09:56:40.456715+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:52.938269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441375+0000","last_change":"2024-04-25T09:56:39.458802+0000","last_active":"2024-04-25T09:56:40.441375+0000","last_peered":"2024-04-25T09:56:40.441375+0000","last_clean":"2024-04-25T09:56:40.441375+0000","last_became_active":"2024-04-25T09:56:39.458532+0000","last_became_peered":"2024-04-25T09:56:39.458532+0000","last_unstale":"2024-04-25T09:56:40.441375+0000","last_undegraded":"2024-04-25T09:56:40.441375+0000","last_fullsized":"2024-04-25T09:56:40.441375+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:09.780581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.757877+0000","last_change":"2024-04-25T09:56:39.465753+0000","last_active":"2024-04-25T09:56:41.757877+0000","last_peered":"2024-04-25T09:56:41.757877+0000","last_clean":"2024-04-25T09:56:41.757877+0000","last_became_active":"2024-04-25T09:56:39.465403+0000","last_became_peered":"2024-04-25T09:56:39.465403+0000","last_unstale":"2024-04-25T09:56:41.757877+0000","last_undegraded":"2024-04-25T09:56:41.757877+0000","last_fullsized":"2024-04-25T09:56:41.757877+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:55.209433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758665+0000","last_change":"2024-04-25T09:56:39.460333+0000","last_active":"2024-04-25T09:56:41.758665+0000","last_peered":"2024-04-25T09:56:41.758665+0000","last_clean":"2024-04-25T09:56:41.758665+0000","last_became_active":"2024-04-25T09:56:39.460057+0000","last_became_peered":"2024-04-25T09:56:39.460057+0000","last_unstale":"2024-04-25T09:56:41.758665+0000","last_undegraded":"2024-04-25T09:56:41.758665+0000","last_fullsized":"2024-04-25T09:56:41.758665+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:11.180982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758297+0000","last_change":"2024-04-25T09:56:38.448802+0000","last_active":"2024-04-25T09:56:41.758297+0000","last_peered":"2024-04-25T09:56:41.758297+0000","last_clean":"2024-04-25T09:56:41.758297+0000","last_became_active":"2024-04-25T09:56:38.447596+0000","last_became_peered":"2024-04-25T09:56:38.447596+0000","last_unstale":"2024-04-25T09:56:41.758297+0000","last_undegraded":"2024-04-25T09:56:41.758297+0000","last_fullsized":"2024-04-25T09:56:41.758297+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:21.398647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441422+0000","last_change":"2024-04-25T09:56:39.457798+0000","last_active":"2024-04-25T09:56:40.441422+0000","last_peered":"2024-04-25T09:56:40.441422+0000","last_clean":"2024-04-25T09:56:40.441422+0000","last_became_active":"2024-04-25T09:56:39.457504+0000","last_became_peered":"2024-04-25T09:56:39.457504+0000","last_unstale":"2024-04-25T09:56:40.441422+0000","last_undegraded":"2024-04-25T09:56:40.441422+0000","last_fullsized":"2024-04-25T09:56:40.441422+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:35.634518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758865+0000","last_change":"2024-04-25T09:56:38.446203+0000","last_active":"2024-04-25T09:56:41.758865+0000","last_peered":"2024-04-25T09:56:41.758865+0000","last_clean":"2024-04-25T09:56:41.758865+0000","last_became_active":"2024-04-25T09:56:38.445931+0000","last_became_peered":"2024-04-25T09:56:38.445931+0000","last_unstale":"2024-04-25T09:56:41.758865+0000","last_undegraded":"2024-04-25T09:56:41.758865+0000","last_fullsized":"2024-04-25T09:56:41.758865+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:04.779145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758759+0000","last_change":"2024-04-25T09:56:39.466011+0000","last_active":"2024-04-25T09:56:41.758759+0000","last_peered":"2024-04-25T09:56:41.758759+0000","last_clean":"2024-04-25T09:56:41.758759+0000","last_became_active":"2024-04-25T09:56:39.465654+0000","last_became_peered":"2024-04-25T09:56:39.465654+0000","last_unstale":"2024-04-25T09:56:41.758759+0000","last_undegraded":"2024-04-25T09:56:41.758759+0000","last_fullsized":"2024-04-25T09:56:41.758759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:33.081726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758567+0000","last_change":"2024-04-25T09:56:38.442763+0000","last_active":"2024-04-25T09:56:41.758567+0000","last_peered":"2024-04-25T09:56:41.758567+0000","last_clean":"2024-04-25T09:56:41.758567+0000","last_became_active":"2024-04-25T09:56:38.442494+0000","last_became_peered":"2024-04-25T09:56:38.442494+0000","last_unstale":"2024-04-25T09:56:41.758567+0000","last_undegraded":"2024-04-25T09:56:41.758567+0000","last_fullsized":"2024-04-25T09:56:41.758567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:09.796418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441677+0000","last_change":"2024-04-25T09:56:39.457184+0000","last_active":"2024-04-25T09:56:40.441677+0000","last_peered":"2024-04-25T09:56:40.441677+0000","last_clean":"2024-04-25T09:56:40.441677+0000","last_became_active":"2024-04-25T09:56:39.456893+0000","last_became_peered":"2024-04-25T09:56:39.456893+0000","last_unstale":"2024-04-25T09:56:40.441677+0000","last_undegraded":"2024-04-25T09:56:40.441677+0000","last_fullsized":"2024-04-25T09:56:40.441677+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:05.783663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758685+0000","last_change":"2024-04-25T09:56:38.441504+0000","last_active":"2024-04-25T09:56:41.758685+0000","last_peered":"2024-04-25T09:56:41.758685+0000","last_clean":"2024-04-25T09:56:41.758685+0000","last_became_active":"2024-04-25T09:56:38.441178+0000","last_became_peered":"2024-04-25T09:56:38.441178+0000","last_unstale":"2024-04-25T09:56:41.758685+0000","last_undegraded":"2024-04-25T09:56:41.758685+0000","last_fullsized":"2024-04-25T09:56:41.758685+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:07.457318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.458989+0000","last_change":"2024-04-25T09:56:39.448406+0000","last_active":"2024-04-25T09:56:40.458989+0000","last_peered":"2024-04-25T09:56:40.458989+0000","last_clean":"2024-04-25T09:56:40.458989+0000","last_became_active":"2024-04-25T09:56:39.448110+0000","last_became_peered":"2024-04-25T09:56:39.448110+0000","last_unstale":"2024-04-25T09:56:40.458989+0000","last_undegraded":"2024-04-25T09:56:40.458989+0000","last_fullsized":"2024-04-25T09:56:40.458989+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:08.145693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.440113+0000","last_change":"2024-04-25T09:56:38.442701+0000","last_active":"2024-04-25T09:56:40.440113+0000","last_peered":"2024-04-25T09:56:40.440113+0000","last_clean":"2024-04-25T09:56:40.440113+0000","last_became_active":"2024-04-25T09:56:38.442433+0000","last_became_peered":"2024-04-25T09:56:38.442433+0000","last_unstale":"2024-04-25T09:56:40.440113+0000","last_undegraded":"2024-04-25T09:56:40.440113+0000","last_fullsized":"2024-04-25T09:56:40.440113+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:32.966740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.757788+0000","last_change":"2024-04-25T09:56:39.454510+0000","last_active":"2024-04-25T09:56:41.757788+0000","last_peered":"2024-04-25T09:56:41.757788+0000","last_clean":"2024-04-25T09:56:41.757788+0000","last_became_active":"2024-04-25T09:56:39.454204+0000","last_became_peered":"2024-04-25T09:56:39.454204+0000","last_unstale":"2024-04-25T09:56:41.757788+0000","last_undegraded":"2024-04-25T09:56:41.757788+0000","last_fullsized":"2024-04-25T09:56:41.757788+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:31.528678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.439950+0000","last_change":"2024-04-25T09:56:38.446402+0000","last_active":"2024-04-25T09:56:40.439950+0000","last_peered":"2024-04-25T09:56:40.439950+0000","last_clean":"2024-04-25T09:56:40.439950+0000","last_became_active":"2024-04-25T09:56:38.446088+0000","last_became_peered":"2024-04-25T09:56:38.446088+0000","last_unstale":"2024-04-25T09:56:40.439950+0000","last_undegraded":"2024-04-25T09:56:40.439950+0000","last_fullsized":"2024-04-25T09:56:40.439950+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:48.759756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758294+0000","last_change":"2024-04-25T09:56:39.461240+0000","last_active":"2024-04-25T09:56:41.758294+0000","last_peered":"2024-04-25T09:56:41.758294+0000","last_clean":"2024-04-25T09:56:41.758294+0000","last_became_active":"2024-04-25T09:56:39.460957+0000","last_became_peered":"2024-04-25T09:56:39.460957+0000","last_unstale":"2024-04-25T09:56:41.758294+0000","last_undegraded":"2024-04-25T09:56:41.758294+0000","last_fullsized":"2024-04-25T09:56:41.758294+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:03.935410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.439954+0000","last_change":"2024-04-25T09:56:38.445015+0000","last_active":"2024-04-25T09:56:40.439954+0000","last_peered":"2024-04-25T09:56:40.439954+0000","last_clean":"2024-04-25T09:56:40.439954+0000","last_became_active":"2024-04-25T09:56:38.444713+0000","last_became_peered":"2024-04-25T09:56:38.444713+0000","last_unstale":"2024-04-25T09:56:40.439954+0000","last_undegraded":"2024-04-25T09:56:40.439954+0000","last_fullsized":"2024-04-25T09:56:40.439954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:10.912571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.440056+0000","last_change":"2024-04-25T09:56:39.458046+0000","last_active":"2024-04-25T09:56:40.440056+0000","last_peered":"2024-04-25T09:56:40.440056+0000","last_clean":"2024-04-25T09:56:40.440056+0000","last_became_active":"2024-04-25T09:56:39.457740+0000","last_became_peered":"2024-04-25T09:56:39.457740+0000","last_unstale":"2024-04-25T09:56:40.440056+0000","last_undegraded":"2024-04-25T09:56:40.440056+0000","last_fullsized":"2024-04-25T09:56:40.440056+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:55.209433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758249+0000","last_change":"2024-04-25T09:56:38.446581+0000","last_active":"2024-04-25T09:56:41.758249+0000","last_peered":"2024-04-25T09:56:41.758249+0000","last_clean":"2024-04-25T09:56:41.758249+0000","last_became_active":"2024-04-25T09:56:38.444721+0000","last_became_peered":"2024-04-25T09:56:38.444721+0000","last_unstale":"2024-04-25T09:56:41.758249+0000","last_undegraded":"2024-04-25T09:56:41.758249+0000","last_fullsized":"2024-04-25T09:56:41.758249+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:53.694683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758124+0000","last_change":"2024-04-25T09:56:39.461463+0000","last_active":"2024-04-25T09:56:41.758124+0000","last_peered":"2024-04-25T09:56:41.758124+0000","last_clean":"2024-04-25T09:56:41.758124+0000","last_became_active":"2024-04-25T09:56:39.461205+0000","last_became_peered":"2024-04-25T09:56:39.461205+0000","last_unstale":"2024-04-25T09:56:41.758124+0000","last_undegraded":"2024-04-25T09:56:41.758124+0000","last_fullsized":"2024-04-25T09:56:41.758124+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:23.348161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441939+0000","last_change":"2024-04-25T09:56:38.452304+0000","last_active":"2024-04-25T09:56:40.441939+0000","last_peered":"2024-04-25T09:56:40.441939+0000","last_clean":"2024-04-25T09:56:40.441939+0000","last_became_active":"2024-04-25T09:56:38.452001+0000","last_became_peered":"2024-04-25T09:56:38.452001+0000","last_unstale":"2024-04-25T09:56:40.441939+0000","last_undegraded":"2024-04-25T09:56:40.441939+0000","last_fullsized":"2024-04-25T09:56:40.441939+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:22.343643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.440209+0000","last_change":"2024-04-25T09:56:39.455583+0000","last_active":"2024-04-25T09:56:40.440209+0000","last_peered":"2024-04-25T09:56:40.440209+0000","last_clean":"2024-04-25T09:56:40.440209+0000","last_became_active":"2024-04-25T09:56:39.455267+0000","last_became_peered":"2024-04-25T09:56:39.455267+0000","last_unstale":"2024-04-25T09:56:40.440209+0000","last_undegraded":"2024-04-25T09:56:40.440209+0000","last_fullsized":"2024-04-25T09:56:40.440209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:04.998080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.442246+0000","last_change":"2024-04-25T09:56:38.451554+0000","last_active":"2024-04-25T09:56:40.442246+0000","last_peered":"2024-04-25T09:56:40.442246+0000","last_clean":"2024-04-25T09:56:40.442246+0000","last_became_active":"2024-04-25T09:56:38.451253+0000","last_became_peered":"2024-04-25T09:56:38.451253+0000","last_unstale":"2024-04-25T09:56:40.442246+0000","last_undegraded":"2024-04-25T09:56:40.442246+0000","last_fullsized":"2024-04-25T09:56:40.442246+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:13.560188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758663+0000","last_change":"2024-04-25T09:56:39.455558+0000","last_active":"2024-04-25T09:56:41.758663+0000","last_peered":"2024-04-25T09:56:41.758663+0000","last_clean":"2024-04-25T09:56:41.758663+0000","last_became_active":"2024-04-25T09:56:39.455289+0000","last_became_peered":"2024-04-25T09:56:39.455289+0000","last_unstale":"2024-04-25T09:56:41.758663+0000","last_undegraded":"2024-04-25T09:56:41.758663+0000","last_fullsized":"2024-04-25T09:56:41.758663+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:00.543750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.442087+0000","last_change":"2024-04-25T09:56:38.452908+0000","last_active":"2024-04-25T09:56:40.442087+0000","last_peered":"2024-04-25T09:56:40.442087+0000","last_clean":"2024-04-25T09:56:40.442087+0000","last_became_active":"2024-04-25T09:56:38.452672+0000","last_became_peered":"2024-04-25T09:56:38.452672+0000","last_unstale":"2024-04-25T09:56:40.442087+0000","last_undegraded":"2024-04-25T09:56:40.442087+0000","last_fullsized":"2024-04-25T09:56:40.442087+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:19.310816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441735+0000","last_change":"2024-04-25T09:56:39.456111+0000","last_active":"2024-04-25T09:56:40.441735+0000","last_peered":"2024-04-25T09:56:40.441735+0000","last_clean":"2024-04-25T09:56:40.441735+0000","last_became_active":"2024-04-25T09:56:39.455815+0000","last_became_peered":"2024-04-25T09:56:39.455815+0000","last_unstale":"2024-04-25T09:56:40.441735+0000","last_undegraded":"2024-04-25T09:56:40.441735+0000","last_fullsized":"2024-04-25T09:56:40.441735+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:22.403165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758218+0000","last_change":"2024-04-25T09:56:38.441289+0000","last_active":"2024-04-25T09:56:41.758218+0000","last_peered":"2024-04-25T09:56:41.758218+0000","last_clean":"2024-04-25T09:56:41.758218+0000","last_became_active":"2024-04-25T09:56:38.440974+0000","last_became_peered":"2024-04-25T09:56:38.440974+0000","last_unstale":"2024-04-25T09:56:41.758218+0000","last_undegraded":"2024-04-25T09:56:41.758218+0000","last_fullsized":"2024-04-25T09:56:41.758218+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:58.075565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.440848+0000","last_change":"2024-04-25T09:56:39.457026+0000","last_active":"2024-04-25T09:56:40.440848+0000","last_peered":"2024-04-25T09:56:40.440848+0000","last_clean":"2024-04-25T09:56:40.440848+0000","last_became_active":"2024-04-25T09:56:39.456738+0000","last_became_peered":"2024-04-25T09:56:39.456738+0000","last_unstale":"2024-04-25T09:56:40.440848+0000","last_undegraded":"2024-04-25T09:56:40.440848+0000","last_fullsized":"2024-04-25T09:56:40.440848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:22.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441278+0000","last_change":"2024-04-25T09:56:38.444152+0000","last_active":"2024-04-25T09:56:40.441278+0000","last_peered":"2024-04-25T09:56:40.441278+0000","last_clean":"2024-04-25T09:56:40.441278+0000","last_became_active":"2024-04-25T09:56:38.443891+0000","last_became_peered":"2024-04-25T09:56:38.443891+0000","last_unstale":"2024-04-25T09:56:40.441278+0000","last_undegraded":"2024-04-25T09:56:40.441278+0000","last_fullsized":"2024-04-25T09:56:40.441278+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:10.176464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.456714+0000","last_change":"2024-04-25T09:56:39.463694+0000","last_active":"2024-04-25T09:56:40.456714+0000","last_peered":"2024-04-25T09:56:40.456714+0000","last_clean":"2024-04-25T09:56:40.456714+0000","last_became_active":"2024-04-25T09:56:39.463417+0000","last_became_peered":"2024-04-25T09:56:39.463417+0000","last_unstale":"2024-04-25T09:56:40.456714+0000","last_undegraded":"2024-04-25T09:56:40.456714+0000","last_fullsized":"2024-04-25T09:56:40.456714+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:40.562496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.456697+0000","last_change":"2024-04-25T09:56:38.452182+0000","last_active":"2024-04-25T09:56:40.456697+0000","last_peered":"2024-04-25T09:56:40.456697+0000","last_clean":"2024-04-25T09:56:40.456697+0000","last_became_active":"2024-04-25T09:56:38.451492+0000","last_became_peered":"2024-04-25T09:56:38.451492+0000","last_unstale":"2024-04-25T09:56:40.456697+0000","last_undegraded":"2024-04-25T09:56:40.456697+0000","last_fullsized":"2024-04-25T09:56:40.456697+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:59.539232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758300+0000","last_change":"2024-04-25T09:56:39.459035+0000","last_active":"2024-04-25T09:56:41.758300+0000","last_peered":"2024-04-25T09:56:41.758300+0000","last_clean":"2024-04-25T09:56:41.758300+0000","last_became_active":"2024-04-25T09:56:39.458698+0000","last_became_peered":"2024-04-25T09:56:39.458698+0000","last_unstale":"2024-04-25T09:56:41.758300+0000","last_undegraded":"2024-04-25T09:56:41.758300+0000","last_fullsized":"2024-04-25T09:56:41.758300+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:59.080083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.442229+0000","last_change":"2024-04-25T09:56:38.449599+0000","last_active":"2024-04-25T09:56:40.442229+0000","last_peered":"2024-04-25T09:56:40.442229+0000","last_clean":"2024-04-25T09:56:40.442229+0000","last_became_active":"2024-04-25T09:56:38.449302+0000","last_became_peered":"2024-04-25T09:56:38.449302+0000","last_unstale":"2024-04-25T09:56:40.442229+0000","last_undegraded":"2024-04-25T09:56:40.442229+0000","last_fullsized":"2024-04-25T09:56:40.442229+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:17.246898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.757976+0000","last_change":"2024-04-25T09:56:39.459233+0000","last_active":"2024-04-25T09:56:41.757976+0000","last_peered":"2024-04-25T09:56:41.757976+0000","last_clean":"2024-04-25T09:56:41.757976+0000","last_became_active":"2024-04-25T09:56:39.458950+0000","last_became_peered":"2024-04-25T09:56:39.458950+0000","last_unstale":"2024-04-25T09:56:41.757976+0000","last_undegraded":"2024-04-25T09:56:41.757976+0000","last_fullsized":"2024-04-25T09:56:41.757976+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:07.074947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.440962+0000","last_change":"2024-04-25T09:56:38.447935+0000","last_active":"2024-04-25T09:56:40.440962+0000","last_peered":"2024-04-25T09:56:40.440962+0000","last_clean":"2024-04-25T09:56:40.440962+0000","last_became_active":"2024-04-25T09:56:38.447648+0000","last_became_peered":"2024-04-25T09:56:38.447648+0000","last_unstale":"2024-04-25T09:56:40.440962+0000","last_undegraded":"2024-04-25T09:56:40.440962+0000","last_fullsized":"2024-04-25T09:56:40.440962+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:08.406378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441548+0000","last_change":"2024-04-25T09:56:39.456478+0000","last_active":"2024-04-25T09:56:40.441548+0000","last_peered":"2024-04-25T09:56:40.441548+0000","last_clean":"2024-04-25T09:56:40.441548+0000","last_became_active":"2024-04-25T09:56:39.456137+0000","last_became_peered":"2024-04-25T09:56:39.456137+0000","last_unstale":"2024-04-25T09:56:40.441548+0000","last_undegraded":"2024-04-25T09:56:40.441548+0000","last_fullsized":"2024-04-25T09:56:40.441548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:10.800936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.456993+0000","last_change":"2024-04-25T09:56:38.451735+0000","last_active":"2024-04-25T09:56:40.456993+0000","last_peered":"2024-04-25T09:56:40.456993+0000","last_clean":"2024-04-25T09:56:40.456993+0000","last_became_active":"2024-04-25T09:56:38.451108+0000","last_became_peered":"2024-04-25T09:56:38.451108+0000","last_unstale":"2024-04-25T09:56:40.456993+0000","last_undegraded":"2024-04-25T09:56:40.456993+0000","last_fullsized":"2024-04-25T09:56:40.456993+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.524161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758568+0000","last_change":"2024-04-25T09:56:39.457632+0000","last_active":"2024-04-25T09:56:41.758568+0000","last_peered":"2024-04-25T09:56:41.758568+0000","last_clean":"2024-04-25T09:56:41.758568+0000","last_became_active":"2024-04-25T09:56:39.457017+0000","last_became_peered":"2024-04-25T09:56:39.457017+0000","last_unstale":"2024-04-25T09:56:41.758568+0000","last_undegraded":"2024-04-25T09:56:41.758568+0000","last_fullsized":"2024-04-25T09:56:41.758568+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:35.263572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441184+0000","last_change":"2024-04-25T09:56:38.449467+0000","last_active":"2024-04-25T09:56:40.441184+0000","last_peered":"2024-04-25T09:56:40.441184+0000","last_clean":"2024-04-25T09:56:40.441184+0000","last_became_active":"2024-04-25T09:56:38.449110+0000","last_became_peered":"2024-04-25T09:56:38.449110+0000","last_unstale":"2024-04-25T09:56:40.441184+0000","last_undegraded":"2024-04-25T09:56:40.441184+0000","last_fullsized":"2024-04-25T09:56:40.441184+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:06.070430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.442388+0000","last_change":"2024-04-25T09:56:39.457741+0000","last_active":"2024-04-25T09:56:40.442388+0000","last_peered":"2024-04-25T09:56:40.442388+0000","last_clean":"2024-04-25T09:56:40.442388+0000","last_became_active":"2024-04-25T09:56:39.457457+0000","last_became_peered":"2024-04-25T09:56:39.457457+0000","last_unstale":"2024-04-25T09:56:40.442388+0000","last_undegraded":"2024-04-25T09:56:40.442388+0000","last_fullsized":"2024-04-25T09:56:40.442388+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:03.935410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.442201+0000","last_change":"2024-04-25T09:56:38.452721+0000","last_active":"2024-04-25T09:56:40.442201+0000","last_peered":"2024-04-25T09:56:40.442201+0000","last_clean":"2024-04-25T09:56:40.442201+0000","last_became_active":"2024-04-25T09:56:38.452370+0000","last_became_peered":"2024-04-25T09:56:38.452370+0000","last_unstale":"2024-04-25T09:56:40.442201+0000","last_undegraded":"2024-04-25T09:56:40.442201+0000","last_fullsized":"2024-04-25T09:56:40.442201+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:19.945036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758601+0000","last_change":"2024-04-25T09:56:39.455192+0000","last_active":"2024-04-25T09:56:41.758601+0000","last_peered":"2024-04-25T09:56:41.758601+0000","last_clean":"2024-04-25T09:56:41.758601+0000","last_became_active":"2024-04-25T09:56:39.454906+0000","last_became_peered":"2024-04-25T09:56:39.454906+0000","last_unstale":"2024-04-25T09:56:41.758601+0000","last_undegraded":"2024-04-25T09:56:41.758601+0000","last_fullsized":"2024-04-25T09:56:41.758601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.409157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.440164+0000","last_change":"2024-04-25T09:56:38.441046+0000","last_active":"2024-04-25T09:56:40.440164+0000","last_peered":"2024-04-25T09:56:40.440164+0000","last_clean":"2024-04-25T09:56:40.440164+0000","last_became_active":"2024-04-25T09:56:38.440736+0000","last_became_peered":"2024-04-25T09:56:38.440736+0000","last_unstale":"2024-04-25T09:56:40.440164+0000","last_undegraded":"2024-04-25T09:56:40.440164+0000","last_fullsized":"2024-04-25T09:56:40.440164+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:20.801968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758031+0000","last_change":"2024-04-25T09:56:39.459803+0000","last_active":"2024-04-25T09:56:41.758031+0000","last_peered":"2024-04-25T09:56:41.758031+0000","last_clean":"2024-04-25T09:56:41.758031+0000","last_became_active":"2024-04-25T09:56:39.459538+0000","last_became_peered":"2024-04-25T09:56:39.459538+0000","last_unstale":"2024-04-25T09:56:41.758031+0000","last_undegraded":"2024-04-25T09:56:41.758031+0000","last_fullsized":"2024-04-25T09:56:41.758031+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:53.256258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758417+0000","last_change":"2024-04-25T09:56:38.443577+0000","last_active":"2024-04-25T09:56:41.758417+0000","last_peered":"2024-04-25T09:56:41.758417+0000","last_clean":"2024-04-25T09:56:41.758417+0000","last_became_active":"2024-04-25T09:56:38.443304+0000","last_became_peered":"2024-04-25T09:56:38.443304+0000","last_unstale":"2024-04-25T09:56:41.758417+0000","last_undegraded":"2024-04-25T09:56:41.758417+0000","last_fullsized":"2024-04-25T09:56:41.758417+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:20.801968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.456834+0000","last_change":"2024-04-25T09:56:39.445727+0000","last_active":"2024-04-25T09:56:40.456834+0000","last_peered":"2024-04-25T09:56:40.456834+0000","last_clean":"2024-04-25T09:56:40.456834+0000","last_became_active":"2024-04-25T09:56:39.445489+0000","last_became_peered":"2024-04-25T09:56:39.445489+0000","last_unstale":"2024-04-25T09:56:40.456834+0000","last_undegraded":"2024-04-25T09:56:40.456834+0000","last_fullsized":"2024-04-25T09:56:40.456834+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:11.917089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.456852+0000","last_change":"2024-04-25T09:56:38.451937+0000","last_active":"2024-04-25T09:56:40.456852+0000","last_peered":"2024-04-25T09:56:40.456852+0000","last_clean":"2024-04-25T09:56:40.456852+0000","last_became_active":"2024-04-25T09:56:38.451248+0000","last_became_peered":"2024-04-25T09:56:38.451248+0000","last_unstale":"2024-04-25T09:56:40.456852+0000","last_undegraded":"2024-04-25T09:56:40.456852+0000","last_fullsized":"2024-04-25T09:56:40.456852+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.108179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758647+0000","last_change":"2024-04-25T09:56:39.461933+0000","last_active":"2024-04-25T09:56:41.758647+0000","last_peered":"2024-04-25T09:56:41.758647+0000","last_clean":"2024-04-25T09:56:41.758647+0000","last_became_active":"2024-04-25T09:56:39.461675+0000","last_became_peered":"2024-04-25T09:56:39.461675+0000","last_unstale":"2024-04-25T09:56:41.758647+0000","last_undegraded":"2024-04-25T09:56:41.758647+0000","last_fullsized":"2024-04-25T09:56:41.758647+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:08.047745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758277+0000","last_change":"2024-04-25T09:56:38.446452+0000","last_active":"2024-04-25T09:56:41.758277+0000","last_peered":"2024-04-25T09:56:41.758277+0000","last_clean":"2024-04-25T09:56:41.758277+0000","last_became_active":"2024-04-25T09:56:38.446166+0000","last_became_peered":"2024-04-25T09:56:38.446166+0000","last_unstale":"2024-04-25T09:56:41.758277+0000","last_undegraded":"2024-04-25T09:56:41.758277+0000","last_fullsized":"2024-04-25T09:56:41.758277+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:05.103963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758599+0000","last_change":"2024-04-25T09:56:39.459859+0000","last_active":"2024-04-25T09:56:41.758599+0000","last_peered":"2024-04-25T09:56:41.758599+0000","last_clean":"2024-04-25T09:56:41.758599+0000","last_became_active":"2024-04-25T09:56:39.459591+0000","last_became_peered":"2024-04-25T09:56:39.459591+0000","last_unstale":"2024-04-25T09:56:41.758599+0000","last_undegraded":"2024-04-25T09:56:41.758599+0000","last_fullsized":"2024-04-25T09:56:41.758599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:48.312124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.457209+0000","last_change":"2024-04-25T09:56:38.447857+0000","last_active":"2024-04-25T09:56:40.457209+0000","last_peered":"2024-04-25T09:56:40.457209+0000","last_clean":"2024-04-25T09:56:40.457209+0000","last_became_active":"2024-04-25T09:56:38.447519+0000","last_became_peered":"2024-04-25T09:56:38.447519+0000","last_unstale":"2024-04-25T09:56:40.457209+0000","last_undegraded":"2024-04-25T09:56:40.457209+0000","last_fullsized":"2024-04-25T09:56:40.457209+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:17.246898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.458983+0000","last_change":"2024-04-25T09:56:39.463510+0000","last_active":"2024-04-25T09:56:40.458983+0000","last_peered":"2024-04-25T09:56:40.458983+0000","last_clean":"2024-04-25T09:56:40.458983+0000","last_became_active":"2024-04-25T09:56:39.463214+0000","last_became_peered":"2024-04-25T09:56:39.463214+0000","last_unstale":"2024-04-25T09:56:40.458983+0000","last_undegraded":"2024-04-25T09:56:40.458983+0000","last_fullsized":"2024-04-25T09:56:40.458983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:27.797861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"64'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.457183+0000","last_change":"2024-04-25T09:56:38.448408+0000","last_active":"2024-04-25T09:56:40.457183+0000","last_peered":"2024-04-25T09:56:40.457183+0000","last_clean":"2024-04-25T09:56:40.457183+0000","last_became_active":"2024-04-25T09:56:38.448024+0000","last_became_peered":"2024-04-25T09:56:38.448024+0000","last_unstale":"2024-04-25T09:56:40.457183+0000","last_undegraded":"2024-04-25T09:56:40.457183+0000","last_fullsized":"2024-04-25T09:56:40.457183+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:10.176464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.457024+0000","last_change":"2024-04-25T09:56:39.464022+0000","last_active":"2024-04-25T09:56:40.457024+0000","last_peered":"2024-04-25T09:56:40.457024+0000","last_clean":"2024-04-25T09:56:40.457024+0000","last_became_active":"2024-04-25T09:56:39.463757+0000","last_became_peered":"2024-04-25T09:56:39.463757+0000","last_unstale":"2024-04-25T09:56:40.457024+0000","last_undegraded":"2024-04-25T09:56:40.457024+0000","last_fullsized":"2024-04-25T09:56:40.457024+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:21.806485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758629+0000","last_change":"2024-04-25T09:56:38.442585+0000","last_active":"2024-04-25T09:56:41.758629+0000","last_peered":"2024-04-25T09:56:41.758629+0000","last_clean":"2024-04-25T09:56:41.758629+0000","last_became_active":"2024-04-25T09:56:38.442310+0000","last_became_peered":"2024-04-25T09:56:38.442310+0000","last_unstale":"2024-04-25T09:56:41.758629+0000","last_undegraded":"2024-04-25T09:56:41.758629+0000","last_fullsized":"2024-04-25T09:56:41.758629+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:32.966740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758672+0000","last_change":"2024-04-25T09:56:39.456191+0000","last_active":"2024-04-25T09:56:41.758672+0000","last_peered":"2024-04-25T09:56:41.758672+0000","last_clean":"2024-04-25T09:56:41.758672+0000","last_became_active":"2024-04-25T09:56:39.455576+0000","last_became_peered":"2024-04-25T09:56:39.455576+0000","last_unstale":"2024-04-25T09:56:41.758672+0000","last_undegraded":"2024-04-25T09:56:41.758672+0000","last_fullsized":"2024-04-25T09:56:41.758672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:42.112697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758335+0000","last_change":"2024-04-25T09:56:38.442079+0000","last_active":"2024-04-25T09:56:41.758335+0000","last_peered":"2024-04-25T09:56:41.758335+0000","last_clean":"2024-04-25T09:56:41.758335+0000","last_became_active":"2024-04-25T09:56:38.441787+0000","last_became_peered":"2024-04-25T09:56:38.441787+0000","last_unstale":"2024-04-25T09:56:41.758335+0000","last_undegraded":"2024-04-25T09:56:41.758335+0000","last_fullsized":"2024-04-25T09:56:41.758335+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:02.930892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758510+0000","last_change":"2024-04-25T09:56:39.459609+0000","last_active":"2024-04-25T09:56:41.758510+0000","last_peered":"2024-04-25T09:56:41.758510+0000","last_clean":"2024-04-25T09:56:41.758510+0000","last_became_active":"2024-04-25T09:56:39.459336+0000","last_became_peered":"2024-04-25T09:56:39.459336+0000","last_unstale":"2024-04-25T09:56:41.758510+0000","last_undegraded":"2024-04-25T09:56:41.758510+0000","last_fullsized":"2024-04-25T09:56:41.758510+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:57.639631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441639+0000","last_change":"2024-04-25T09:56:38.450312+0000","last_active":"2024-04-25T09:56:40.441639+0000","last_peered":"2024-04-25T09:56:40.441639+0000","last_clean":"2024-04-25T09:56:40.441639+0000","last_became_active":"2024-04-25T09:56:38.450049+0000","last_became_peered":"2024-04-25T09:56:38.450049+0000","last_unstale":"2024-04-25T09:56:40.441639+0000","last_undegraded":"2024-04-25T09:56:40.441639+0000","last_fullsized":"2024-04-25T09:56:40.441639+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.524161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441512+0000","last_change":"2024-04-25T09:56:39.458184+0000","last_active":"2024-04-25T09:56:40.441512+0000","last_peered":"2024-04-25T09:56:40.441512+0000","last_clean":"2024-04-25T09:56:40.441512+0000","last_became_active":"2024-04-25T09:56:39.457877+0000","last_became_peered":"2024-04-25T09:56:39.457877+0000","last_unstale":"2024-04-25T09:56:40.441512+0000","last_undegraded":"2024-04-25T09:56:40.441512+0000","last_fullsized":"2024-04-25T09:56:40.441512+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.123529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.440079+0000","last_change":"2024-04-25T09:56:38.441913+0000","last_active":"2024-04-25T09:56:40.440079+0000","last_peered":"2024-04-25T09:56:40.440079+0000","last_clean":"2024-04-25T09:56:40.440079+0000","last_became_active":"2024-04-25T09:56:38.441584+0000","last_became_peered":"2024-04-25T09:56:38.441584+0000","last_unstale":"2024-04-25T09:56:40.440079+0000","last_undegraded":"2024-04-25T09:56:40.440079+0000","last_fullsized":"2024-04-25T09:56:40.440079+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:49.142302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758068+0000","last_change":"2024-04-25T09:56:39.453828+0000","last_active":"2024-04-25T09:56:41.758068+0000","last_peered":"2024-04-25T09:56:41.758068+0000","last_clean":"2024-04-25T09:56:41.758068+0000","last_became_active":"2024-04-25T09:56:39.453509+0000","last_became_peered":"2024-04-25T09:56:39.453509+0000","last_unstale":"2024-04-25T09:56:41.758068+0000","last_undegraded":"2024-04-25T09:56:41.758068+0000","last_fullsized":"2024-04-25T09:56:41.758068+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:20.315333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.440062+0000","last_change":"2024-04-25T09:56:38.444951+0000","last_active":"2024-04-25T09:56:40.440062+0000","last_peered":"2024-04-25T09:56:40.440062+0000","last_clean":"2024-04-25T09:56:40.440062+0000","last_became_active":"2024-04-25T09:56:38.444617+0000","last_became_peered":"2024-04-25T09:56:38.444617+0000","last_unstale":"2024-04-25T09:56:40.440062+0000","last_undegraded":"2024-04-25T09:56:40.440062+0000","last_fullsized":"2024-04-25T09:56:40.440062+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:07.457318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.459141+0000","last_change":"2024-04-25T09:56:39.446045+0000","last_active":"2024-04-25T09:56:40.459141+0000","last_peered":"2024-04-25T09:56:40.459141+0000","last_clean":"2024-04-25T09:56:40.459141+0000","last_became_active":"2024-04-25T09:56:39.445709+0000","last_became_peered":"2024-04-25T09:56:39.445709+0000","last_unstale":"2024-04-25T09:56:40.459141+0000","last_undegraded":"2024-04-25T09:56:40.459141+0000","last_fullsized":"2024-04-25T09:56:40.459141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:29.668415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.757715+0000","last_change":"2024-04-25T09:56:38.447517+0000","last_active":"2024-04-25T09:56:41.757715+0000","last_peered":"2024-04-25T09:56:41.757715+0000","last_clean":"2024-04-25T09:56:41.757715+0000","last_became_active":"2024-04-25T09:56:38.447204+0000","last_became_peered":"2024-04-25T09:56:38.447204+0000","last_unstale":"2024-04-25T09:56:41.757715+0000","last_undegraded":"2024-04-25T09:56:41.757715+0000","last_fullsized":"2024-04-25T09:56:41.757715+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:47.310866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:41.758470+0000","last_change":"2024-04-25T09:56:39.461139+0000","last_active":"2024-04-25T09:56:41.758470+0000","last_peered":"2024-04-25T09:56:41.758470+0000","last_clean":"2024-04-25T09:56:41.758470+0000","last_became_active":"2024-04-25T09:56:39.460815+0000","last_became_peered":"2024-04-25T09:56:39.460815+0000","last_unstale":"2024-04-25T09:56:41.758470+0000","last_undegraded":"2024-04-25T09:56:41.758470+0000","last_fullsized":"2024-04-25T09:56:41.758470+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:38.425967+0000","last_clean_scrub_stamp":"2024-04-25T09:56:38.425967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:33.733512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.439872+0000","last_change":"2024-04-25T09:56:38.443664+0000","last_active":"2024-04-25T09:56:40.439872+0000","last_peered":"2024-04-25T09:56:40.439872+0000","last_clean":"2024-04-25T09:56:40.439872+0000","last_became_active":"2024-04-25T09:56:38.443253+0000","last_became_peered":"2024-04-25T09:56:38.443253+0000","last_unstale":"2024-04-25T09:56:40.439872+0000","last_undegraded":"2024-04-25T09:56:40.439872+0000","last_fullsized":"2024-04-25T09:56:40.439872+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:56:37.421450+0000","last_clean_scrub_stamp":"2024-04-25T09:56:37.421450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:09.796418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":259,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T09:56:40.441504+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:56:40.441504+0000","last_peered":"2024-04-25T09:56:40.441504+0000","last_clean":"2024-04-25T09:56:40.441504+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:56:40.441504+0000","last_undegraded":"2024-04-25T09:56:40.441504+0000","last_fullsized":"2024-04-25T09:56:40.441504+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1384448,"data_stored":942143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36100,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335740,"statfs":{"total":96636764160,"available":96599797760,"internally_reserved":0,"allocated":778240,"data_stored":351033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36100,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335740,"statfs":{"total":96636764160,"available":96599797760,"internally_reserved":0,"allocated":778240,"data_stored":351021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":786432,"data_stored":352611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36100,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335740,"statfs":{"total":96636764160,"available":96599797760,"internally_reserved":0,"allocated":774144,"data_stored":350999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36100,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335740,"statfs":{"total":96636764160,"available":96599797760,"internally_reserved":0,"allocated":774144,"data_stored":350999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":802816,"data_stored":353375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36676,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335164,"statfs":{"total":96636764160,"available":96599207936,"internally_reserved":0,"allocated":1368064,"data_stored":941367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:56:47.213 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:56:47.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:56:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.354+0000 7f353a688640 1 Processor -- start 2024-04-25T09:56:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.354+0000 7f353a688640 1 -- start start 2024-04-25T09:56:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f353a688640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f353415dc20 0x7f35340ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f353a688640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f35340d32f0 0x7f35340cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f353a688640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f353415e000 con 0x7f3534161710 2024-04-25T09:56:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f353a688640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35340d1de0 con 0x7f353415dc20 2024-04-25T09:56:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f353a688640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3534161c60 con 0x7f35340d32f0 2024-04-25T09:56:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f3533fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f353415dc20 0x7f35340ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f35337fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3534161710 0x7f353406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36630/0 (socket says 172.21.15.19:36630) 2024-04-25T09:56:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f35337fe640 1 -- 172.21.15.19:0/1172562751 learned_addr learned my addr 172.21.15.19:0/1172562751 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f3533fff640 1 --2- 172.21.15.19:0/1172562751 >> v2:172.21.15.19:3300/0 conn(0x7f353415dc20 0x7f35340ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53354/0 (socket says 172.21.15.19:53354) 2024-04-25T09:56:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f3538c25640 1 --2- 172.21.15.19:0/1172562751 >> v2:172.21.15.112:3300/0 conn(0x7f35340d32f0 0x7f35340cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f3533fff640 1 -- 172.21.15.19:0/1172562751 >> v2:172.21.15.112:3300/0 conn(0x7f35340d32f0 msgr2=0x7f35340cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f3533fff640 1 --2- 172.21.15.19:0/1172562751 >> v2:172.21.15.112:3300/0 conn(0x7f35340d32f0 0x7f35340cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f3533fff640 1 -- 172.21.15.19:0/1172562751 >> v1:172.21.15.19:6789/0 conn(0x7f3534161710 legacy=0x7f353406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f3532ffd640 1 -- 172.21.15.19:0/1172562751 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373613713 0 0) 0x7f353415e000 con 0x7f3534161710 2024-04-25T09:56:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f3533fff640 1 -- 172.21.15.19:0/1172562751 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3534161f40 con 0x7f353415dc20 2024-04-25T09:56:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.355+0000 7f3538c25640 1 --2- 172.21.15.19:0/1172562751 >> v2:172.21.15.112:3300/0 conn(0x7f35340d32f0 0x7f35340cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.356+0000 7f3533fff640 1 --2- 172.21.15.19:0/1172562751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f353415dc20 0x7f35340ce670 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f35340b6910 tx=0x7f352402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f338549ea1995cef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.356+0000 7f3532ffd640 1 -- 172.21.15.19:0/1172562751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524037070 con 0x7f353415dc20 2024-04-25T09:56:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.356+0000 7f3532ffd640 1 -- 172.21.15.19:0/1172562751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3524032440 con 0x7f353415dc20 2024-04-25T09:56:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.356+0000 7f3532ffd640 1 -- 172.21.15.19:0/1172562751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524032810 con 0x7f353415dc20 2024-04-25T09:56:47.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.356+0000 7f353a688640 1 -- 172.21.15.19:0/1172562751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f353415dc20 msgr2=0x7f35340ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.356+0000 7f353a688640 1 --2- 172.21.15.19:0/1172562751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f353415dc20 0x7f35340ce670 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f35340b6910 tx=0x7f352402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.357+0000 7f353a688640 1 -- 172.21.15.19:0/1172562751 shutdown_connections 2024-04-25T09:56:47.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.357+0000 7f353a688640 1 --2- 172.21.15.19:0/1172562751 >> v2:172.21.15.112:3300/0 conn(0x7f35340d32f0 0x7f35340cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.357+0000 7f353a688640 1 --2- 172.21.15.19:0/1172562751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f353415dc20 0x7f35340ce670 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.357+0000 7f353a688640 1 -- 172.21.15.19:0/1172562751 >> 172.21.15.19:0/1172562751 conn(0x7f35340d0420 msgr2=0x7f35340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:47.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.357+0000 7f353a688640 1 -- 172.21.15.19:0/1172562751 shutdown_connections 2024-04-25T09:56:47.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.357+0000 7f353a688640 1 -- 172.21.15.19:0/1172562751 wait complete. 2024-04-25T09:56:47.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.358+0000 7f353a688640 1 Processor -- start 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.358+0000 7f353a688640 1 -- start start 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f353a688640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340d32f0 0x7f35340b7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f353a688640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353415dc20 0x7f35341f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f353a688640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3534161710 0x7f35341f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f353a688640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35340b5e50 con 0x7f35340d32f0 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f353a688640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3534162500 con 0x7f353415dc20 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f353a688640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35340b5fd0 con 0x7f3534161710 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3533fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340d32f0 0x7f35340b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3533fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340d32f0 0x7f35340b7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53358/0 (socket says 172.21.15.19:53358) 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3533fff640 1 -- 172.21.15.19:0/1746575516 learned_addr learned my addr 172.21.15.19:0/1746575516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f35337fe640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353415dc20 0x7f35341f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3538c25640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3534161710 0x7f35341f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3533fff640 1 -- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3534161710 msgr2=0x7f35341f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3533fff640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3534161710 0x7f35341f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3533fff640 1 -- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353415dc20 msgr2=0x7f35341f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3533fff640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353415dc20 0x7f35341f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3533fff640 1 -- 172.21.15.19:0/1746575516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35341f7690 con 0x7f35340d32f0 2024-04-25T09:56:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f35337fe640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353415dc20 0x7f35341f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3538c25640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3534161710 0x7f35341f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3533fff640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340d32f0 0x7f35340b7890 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3524009890 tx=0x7f35240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3530ff9640 1 -- 172.21.15.19:0/1746575516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524037040 con 0x7f35340d32f0 2024-04-25T09:56:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3530ff9640 1 -- 172.21.15.19:0/1746575516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3524042070 con 0x7f35340d32f0 2024-04-25T09:56:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f353a688640 1 -- 172.21.15.19:0/1746575516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35341f4720 con 0x7f35340d32f0 2024-04-25T09:56:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.359+0000 7f3530ff9640 1 -- 172.21.15.19:0/1746575516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524030a50 con 0x7f35340d32f0 2024-04-25T09:56:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.360+0000 7f353a688640 1 -- 172.21.15.19:0/1746575516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35341f4ce0 con 0x7f35340d32f0 2024-04-25T09:56:47.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.361+0000 7f353a688640 1 -- 172.21.15.19:0/1746575516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3534073a00 con 0x7f35340d32f0 2024-04-25T09:56:47.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.361+0000 7f3530ff9640 1 -- 172.21.15.19:0/1746575516 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3524030bf0 con 0x7f35340d32f0 2024-04-25T09:56:47.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.361+0000 7f3530ff9640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 0x7f352006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.361+0000 7f3530ff9640 1 -- 172.21.15.19:0/1746575516 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f35240aaf40 con 0x7f35340d32f0 2024-04-25T09:56:47.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.362+0000 7f35337fe640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 0x7f352006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.362+0000 7f35337fe640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 0x7f352006d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3528000f60 tx=0x7f3528009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:47.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.365+0000 7f3530ff9640 1 -- 172.21.15.19:0/1746575516 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3524045360 con 0x7f35340d32f0 2024-04-25T09:56:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.619+0000 7f353a688640 1 -- 172.21.15.19:0/1746575516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f35340d2f20 con 0x7f35340d32f0 2024-04-25T09:56:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.622+0000 7f3530ff9640 1 -- 172.21.15.19:0/1746575516 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3524079b90 con 0x7f35340d32f0 2024-04-25T09:56:47.625 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T09:56:47.625 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T09:56:47.625 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T09:56:47.625 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T09:56:47.625 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:56:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T09:56:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.623+0000 7f353a688640 1 -- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 msgr2=0x7f352006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.623+0000 7f353a688640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 0x7f352006d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3528000f60 tx=0x7f3528009040 comp rx=0 tx=0).stop 2024-04-25T09:56:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.624+0000 7f353a688640 1 -- 172.21.15.19:0/1746575516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340d32f0 msgr2=0x7f35340b7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.624+0000 7f353a688640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340d32f0 0x7f35340b7890 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3524009890 tx=0x7f35240046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.624+0000 7f353a688640 1 -- 172.21.15.19:0/1746575516 shutdown_connections 2024-04-25T09:56:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.624+0000 7f353a688640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 0x7f352006d990 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.624+0000 7f353a688640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3534161710 0x7f35341f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.624+0000 7f353a688640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353415dc20 0x7f35341f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.624+0000 7f353a688640 1 --2- 172.21.15.19:0/1746575516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340d32f0 0x7f35340b7890 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.624+0000 7f353a688640 1 -- 172.21.15.19:0/1746575516 >> 172.21.15.19:0/1746575516 conn(0x7f35340d0420 msgr2=0x7f35340b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.624+0000 7f353a688640 1 -- 172.21.15.19:0/1746575516 shutdown_connections 2024-04-25T09:56:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.624+0000 7f353a688640 1 -- 172.21.15.19:0/1746575516 wait complete. 2024-04-25T09:56:47.638 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:47.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.736+0000 7f6d1b7c1640 1 Processor -- start 2024-04-25T09:56:47.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.736+0000 7f6d1b7c1640 1 -- start start 2024-04-25T09:56:47.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d1b7c1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6d140cfec0 0x7f6d140cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d1b7c1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d14071480 con 0x7f6d140ce3c0 2024-04-25T09:56:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d1b7c1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d14071760 con 0x7f6d14154ee0 2024-04-25T09:56:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d1b7c1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d140ce830 con 0x7f6d140cfec0 2024-04-25T09:56:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d18d5c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6d140cfec0 0x7f6d140cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d19d5e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6d140ce3c0 0x7f6d1406d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58052/0 (socket says 172.21.15.19:58052) 2024-04-25T09:56:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d18d5c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6d140cfec0 0x7f6d140cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54028/0 (socket says 172.21.15.19:54028) 2024-04-25T09:56:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d19d5e640 1 -- 172.21.15.19:0/4138163322 learned_addr learned my addr 172.21.15.19:0/4138163322 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d18d5c640 1 -- 172.21.15.19:0/4138163322 >> v1:172.21.15.112:6789/0 conn(0x7f6d14154ee0 legacy=0x7f6d140cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d18d5c640 1 -- 172.21.15.19:0/4138163322 >> v1:172.21.15.19:6789/0 conn(0x7f6d140ce3c0 legacy=0x7f6d1406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d18d5c640 1 -- 172.21.15.19:0/4138163322 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d14071a40 con 0x7f6d140cfec0 2024-04-25T09:56:47.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.737+0000 7f6d0bfff640 1 -- 172.21.15.19:0/4138163322 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165159519 0 0) 0x7f6d14071480 con 0x7f6d140ce3c0 2024-04-25T09:56:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.738+0000 7f6d18d5c640 1 --2- 172.21.15.19:0/4138163322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d140cfec0 0x7f6d140cde80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6d040097a0 tx=0x7f6d0402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad4356378ab0f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.738+0000 7f6d0bfff640 1 -- 172.21.15.19:0/4138163322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04037070 con 0x7f6d140cfec0 2024-04-25T09:56:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.738+0000 7f6d0bfff640 1 -- 172.21.15.19:0/4138163322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d04032440 con 0x7f6d140cfec0 2024-04-25T09:56:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.738+0000 7f6d0bfff640 1 -- 172.21.15.19:0/4138163322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04032810 con 0x7f6d140cfec0 2024-04-25T09:56:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.738+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/4138163322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d140cfec0 msgr2=0x7f6d140cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.738+0000 7f6d1b7c1640 1 --2- 172.21.15.19:0/4138163322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d140cfec0 0x7f6d140cde80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6d040097a0 tx=0x7f6d0402a230 comp rx=0 tx=0).stop 2024-04-25T09:56:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.739+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/4138163322 shutdown_connections 2024-04-25T09:56:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.739+0000 7f6d1b7c1640 1 --2- 172.21.15.19:0/4138163322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d140cfec0 0x7f6d140cde80 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.739+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/4138163322 >> 172.21.15.19:0/4138163322 conn(0x7f6d1406abe0 msgr2=0x7f6d1406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.739+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/4138163322 shutdown_connections 2024-04-25T09:56:47.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.739+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/4138163322 wait complete. 2024-04-25T09:56:47.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.740+0000 7f6d1b7c1640 1 Processor -- start 2024-04-25T09:56:47.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.740+0000 7f6d1b7c1640 1 -- start start 2024-04-25T09:56:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d1b7c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d140ce3c0 0x7f6d14153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d1b7c1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d140cfec0 0x7f6d141566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d1b7c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d14154ee0 0x7f6d141e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d1b7c1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d14071230 con 0x7f6d140cfec0 2024-04-25T09:56:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d1b7c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d141552c0 con 0x7f6d14154ee0 2024-04-25T09:56:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d1b7c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d14155440 con 0x7f6d140ce3c0 2024-04-25T09:56:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d18d5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d140cfec0 0x7f6d141566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d18d5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d140cfec0 0x7f6d141566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54034/0 (socket says 172.21.15.19:54034) 2024-04-25T09:56:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d18d5c640 1 -- 172.21.15.19:0/2562071995 learned_addr learned my addr 172.21.15.19:0/2562071995 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d1955d640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d140ce3c0 0x7f6d14153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d19d5e640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d14154ee0 0x7f6d141e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d18d5c640 1 -- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d140ce3c0 msgr2=0x7f6d14153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d18d5c640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d140ce3c0 0x7f6d14153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d18d5c640 1 -- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d14154ee0 msgr2=0x7f6d141e61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d18d5c640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d14154ee0 0x7f6d141e61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d18d5c640 1 -- 172.21.15.19:0/2562071995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d141570d0 con 0x7f6d140cfec0 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d18d5c640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d140cfec0 0x7f6d141566f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6d04000c00 tx=0x7f6d040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.741+0000 7f6d09ffb640 1 -- 172.21.15.19:0/2562071995 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04037040 con 0x7f6d140cfec0 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.742+0000 7f6d09ffb640 1 -- 172.21.15.19:0/2562071995 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d04042070 con 0x7f6d140cfec0 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.742+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/2562071995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d141573b0 con 0x7f6d140cfec0 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.742+0000 7f6d09ffb640 1 -- 172.21.15.19:0/2562071995 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04030a70 con 0x7f6d140cfec0 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.742+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/2562071995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d14157970 con 0x7f6d140cfec0 2024-04-25T09:56:47.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.743+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/2562071995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d14073990 con 0x7f6d140cfec0 2024-04-25T09:56:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.743+0000 7f6d09ffb640 1 -- 172.21.15.19:0/2562071995 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d04030c10 con 0x7f6d140cfec0 2024-04-25T09:56:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.744+0000 7f6d09ffb640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6cf806b4d0 0x7f6cf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:47.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.744+0000 7f6d09ffb640 1 -- 172.21.15.19:0/2562071995 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6d040aae80 con 0x7f6d140cfec0 2024-04-25T09:56:47.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.744+0000 7f6d1955d640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6cf806b4d0 0x7f6cf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:47.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.744+0000 7f6d1955d640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6cf806b4d0 0x7f6cf806d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6d141548d0 tx=0x7f6d0c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:47.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.748+0000 7f6d09ffb640 1 -- 172.21.15.19:0/2562071995 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d04045360 con 0x7f6d140cfec0 2024-04-25T09:56:47.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.963+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/2562071995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d1406d360 con 0x7f6d140cfec0 2024-04-25T09:56:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.964+0000 7f6d09ffb640 1 -- 172.21.15.19:0/2562071995 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f6d04079b90 con 0x7f6d140cfec0 2024-04-25T09:56:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:47.967 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3520485752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3520485752},{"type":"v1","addr":"172.21.15.19:6835","nonce":3520485752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5594,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1340729764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1340729764},{"type":"v1","addr":"172.21.15.112:6837","nonce":1340729764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/335673057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":335673057},{"type":"v1","addr":"172.21.15.112:6835","nonce":335673057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:56:38.802500+0000","modified":"2024-04-25T09:56:40.802847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6582":{"gid":6582,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/591570980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":591570980},{"type":"v1","addr":"172.21.15.19:6834","nonce":591570980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:56:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-04-25T09:56:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.965+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6cf806b4d0 msgr2=0x7f6cf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.965+0000 7f6d1b7c1640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6cf806b4d0 0x7f6cf806d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6d141548d0 tx=0x7f6d0c009040 comp rx=0 tx=0).stop 2024-04-25T09:56:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.965+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/2562071995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d140cfec0 msgr2=0x7f6d141566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.965+0000 7f6d1b7c1640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d140cfec0 0x7f6d141566f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6d04000c00 tx=0x7f6d040046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.965+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/2562071995 shutdown_connections 2024-04-25T09:56:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.965+0000 7f6d1b7c1640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6cf806b4d0 0x7f6cf806d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.965+0000 7f6d1b7c1640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d14154ee0 0x7f6d141e61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.965+0000 7f6d1b7c1640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d140cfec0 0x7f6d141566f0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.965+0000 7f6d1b7c1640 1 --2- 172.21.15.19:0/2562071995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d140ce3c0 0x7f6d14153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.965+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/2562071995 >> 172.21.15.19:0/2562071995 conn(0x7f6d1406abe0 msgr2=0x7f6d140b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.966+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/2562071995 shutdown_connections 2024-04-25T09:56:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:47.966+0000 7f6d1b7c1640 1 -- 172.21.15.19:0/2562071995 wait complete. 2024-04-25T09:56:48.383 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:56:38.802500+0000', 'modified': '2024-04-25T09:56:40.802847+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6582}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6582': {'gid': 6582, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6834/591570980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 591570980}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 591570980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:56:48.384 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:56:48.384 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:56:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.492+0000 7f36ff702640 1 Processor -- start 2024-04-25T09:56:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.492+0000 7f36ff702640 1 -- start start 2024-04-25T09:56:48.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.493+0000 7f36ff702640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36f815dc20 0x7f36f80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.493+0000 7f36ff702640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f36f80d32f0 0x7f36f80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.493+0000 7f36ff702640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f815e000 con 0x7f36f8161710 2024-04-25T09:56:48.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.493+0000 7f36ff702640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f80d1de0 con 0x7f36f80d32f0 2024-04-25T09:56:48.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.493+0000 7f36ff702640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f8161c60 con 0x7f36f815dc20 2024-04-25T09:56:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.493+0000 7f36fdc9f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f36f80d32f0 0x7f36f80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.494+0000 7f36fdc9f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f36f80d32f0 0x7f36f80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54040/0 (socket says 172.21.15.19:54040) 2024-04-25T09:56:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.494+0000 7f36fdc9f640 1 -- 172.21.15.19:0/114523457 learned_addr learned my addr 172.21.15.19:0/114523457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.494+0000 7f36fd49e640 1 --2- 172.21.15.19:0/114523457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36f815dc20 0x7f36f80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.494+0000 7f36fdc9f640 1 -- 172.21.15.19:0/114523457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36f815dc20 msgr2=0x7f36f80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.494+0000 7f36fdc9f640 1 --2- 172.21.15.19:0/114523457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36f815dc20 0x7f36f80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.494+0000 7f36fdc9f640 1 -- 172.21.15.19:0/114523457 >> v1:172.21.15.112:6789/0 conn(0x7f36f8161710 legacy=0x7f36f806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.494+0000 7f36fdc9f640 1 -- 172.21.15.19:0/114523457 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f8161f40 con 0x7f36f80d32f0 2024-04-25T09:56:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.494+0000 7f36fdc9f640 1 --2- 172.21.15.19:0/114523457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f80d32f0 0x7f36f80cebb0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009e50 tx=0x7f36f002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfbddbbc347b8ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.495+0000 7f36effff640 1 -- 172.21.15.19:0/114523457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0036070 con 0x7f36f80d32f0 2024-04-25T09:56:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.495+0000 7f36effff640 1 -- 172.21.15.19:0/114523457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f0031600 con 0x7f36f80d32f0 2024-04-25T09:56:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.495+0000 7f36effff640 1 -- 172.21.15.19:0/114523457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f00319d0 con 0x7f36f80d32f0 2024-04-25T09:56:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.495+0000 7f36ff702640 1 -- 172.21.15.19:0/114523457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f80d32f0 msgr2=0x7f36f80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.495+0000 7f36ff702640 1 --2- 172.21.15.19:0/114523457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f80d32f0 0x7f36f80cebb0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009e50 tx=0x7f36f002a710 comp rx=0 tx=0).stop 2024-04-25T09:56:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.495+0000 7f36ff702640 1 -- 172.21.15.19:0/114523457 shutdown_connections 2024-04-25T09:56:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.495+0000 7f36ff702640 1 --2- 172.21.15.19:0/114523457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f80d32f0 0x7f36f80cebb0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.495+0000 7f36ff702640 1 --2- 172.21.15.19:0/114523457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36f815dc20 0x7f36f80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.495+0000 7f36ff702640 1 -- 172.21.15.19:0/114523457 >> 172.21.15.19:0/114523457 conn(0x7f36f80d0420 msgr2=0x7f36f80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.496+0000 7f36ff702640 1 -- 172.21.15.19:0/114523457 shutdown_connections 2024-04-25T09:56:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.496+0000 7f36ff702640 1 -- 172.21.15.19:0/114523457 wait complete. 2024-04-25T09:56:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.497+0000 7f36ff702640 1 Processor -- start 2024-04-25T09:56:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.497+0000 7f36ff702640 1 -- start start 2024-04-25T09:56:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36ff702640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f80d32f0 0x7f36f814c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36ff702640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f815dc20 0x7f36f80acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36ff702640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36f8161710 0x7f36f80ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36ff702640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f80b5e50 con 0x7f36f815dc20 2024-04-25T09:56:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36ff702640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f8162500 con 0x7f36f80d32f0 2024-04-25T09:56:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36ff702640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f80b5fd0 con 0x7f36f8161710 2024-04-25T09:56:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36fcc9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f815dc20 0x7f36f80acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36fcc9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f815dc20 0x7f36f80acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54042/0 (socket says 172.21.15.19:54042) 2024-04-25T09:56:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36fcc9d640 1 -- 172.21.15.19:0/100715968 learned_addr learned my addr 172.21.15.19:0/100715968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36fd49e640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f80d32f0 0x7f36f814c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36fdc9f640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36f8161710 0x7f36f80ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36fcc9d640 1 -- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36f8161710 msgr2=0x7f36f80ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36fcc9d640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36f8161710 0x7f36f80ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36fcc9d640 1 -- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f80d32f0 msgr2=0x7f36f814c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36fcc9d640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f80d32f0 0x7f36f814c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36fcc9d640 1 -- 172.21.15.19:0/100715968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f80cd350 con 0x7f36f815dc20 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36fcc9d640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f815dc20 0x7f36f80acf20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f36e800c3c0 tx=0x7f36e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36edffb640 1 -- 172.21.15.19:0/100715968 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e80052d0 con 0x7f36f815dc20 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36edffb640 1 -- 172.21.15.19:0/100715968 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e8005490 con 0x7f36f815dc20 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36ff702640 1 -- 172.21.15.19:0/100715968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f814cc30 con 0x7f36f815dc20 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.498+0000 7f36edffb640 1 -- 172.21.15.19:0/100715968 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8007260 con 0x7f36f815dc20 2024-04-25T09:56:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.499+0000 7f36ff702640 1 -- 172.21.15.19:0/100715968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f80cc5d0 con 0x7f36f815dc20 2024-04-25T09:56:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.500+0000 7f36edffb640 1 -- 172.21.15.19:0/100715968 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36e8007400 con 0x7f36f815dc20 2024-04-25T09:56:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.501+0000 7f36edffb640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f36dc06b500 0x7f36dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.501+0000 7f36edffb640 1 -- 172.21.15.19:0/100715968 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f36e8089710 con 0x7f36f815dc20 2024-04-25T09:56:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.501+0000 7f36fd49e640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f36dc06b500 0x7f36dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.501+0000 7f36fd49e640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f36dc06b500 0x7f36dc06d9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f36f80aa590 tx=0x7f36f4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.502+0000 7f37023fd740 1 Processor -- start 2024-04-25T09:56:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.502+0000 7f37023fd740 1 -- start start 2024-04-25T09:56:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.502+0000 7f37023fd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb89230 0x55938fb89610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.502+0000 7f37023fd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fba7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.502+0000 7f37023fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb5ef10 0x55938fba5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.502+0000 7f37023fd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55938fac1a10 con 0x55938fba6ad0 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.502+0000 7f37023fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55938fa6ab30 con 0x55938fb5ef10 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.502+0000 7f37023fd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55938fabdc50 con 0x55938fb89230 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36fcc9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fba7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36fcc9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fba7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54048/0 (socket says 172.21.15.19:54048) 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36fcc9d640 1 -- 172.21.15.19:0/3637021509 learned_addr learned my addr 172.21.15.19:0/3637021509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36fdc9f640 1 --2- 172.21.15.19:0/3637021509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb5ef10 0x55938fba5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36fd49e640 1 --2- 172.21.15.19:0/3637021509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb89230 0x55938fb89610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36fcc9d640 1 -- 172.21.15.19:0/3637021509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb89230 msgr2=0x55938fb89610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36fcc9d640 1 --2- 172.21.15.19:0/3637021509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb89230 0x55938fb89610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36fcc9d640 1 -- 172.21.15.19:0/3637021509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb5ef10 msgr2=0x55938fba5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36fcc9d640 1 --2- 172.21.15.19:0/3637021509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb5ef10 0x55938fba5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36fcc9d640 1 -- 172.21.15.19:0/3637021509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55938fa7fe10 con 0x55938fba6ad0 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36fcc9d640 1 --2- 172.21.15.19:0/3637021509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fba7020 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f36e800cfd0 tx=0x7f36e8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e383f59bcad74cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36e2ffd640 1 -- 172.21.15.19:0/3637021509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8087e60 con 0x55938fba6ad0 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36e2ffd640 1 -- 172.21.15.19:0/3637021509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e805e0c0 con 0x55938fba6ad0 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.503+0000 7f36e2ffd640 1 -- 172.21.15.19:0/3637021509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8054440 con 0x55938fba6ad0 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.504+0000 7f37023fd740 1 -- 172.21.15.19:0/3637021509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 msgr2=0x55938fba7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.504+0000 7f37023fd740 1 --2- 172.21.15.19:0/3637021509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fba7020 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f36e800cfd0 tx=0x7f36e8011000 comp rx=0 tx=0).stop 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.504+0000 7f37023fd740 1 -- 172.21.15.19:0/3637021509 shutdown_connections 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.504+0000 7f37023fd740 1 --2- 172.21.15.19:0/3637021509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb5ef10 0x55938fba5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.504+0000 7f37023fd740 1 --2- 172.21.15.19:0/3637021509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fba7020 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.504+0000 7f37023fd740 1 --2- 172.21.15.19:0/3637021509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb89230 0x55938fb89610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.504+0000 7f37023fd740 1 -- 172.21.15.19:0/3637021509 >> 172.21.15.19:0/3637021509 conn(0x55938fb5b840 msgr2=0x55938fba4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.504+0000 7f37023fd740 1 -- 172.21.15.19:0/3637021509 shutdown_connections 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.504+0000 7f37023fd740 1 -- 172.21.15.19:0/3637021509 wait complete. 2024-04-25T09:56:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.504+0000 7f37023fd740 1 -- start start 2024-04-25T09:56:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.505+0000 7f37023fd740 1 Processor -- start 2024-04-25T09:56:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.505+0000 7f37023fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb89230 0x55938fb77a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.505+0000 7f37023fd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb9ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.505+0000 7f37023fd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb9d3d0 0x55938fb8cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.505+0000 7f37023fd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55938fb5fcb0 con 0x55938fba6ad0 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.505+0000 7f37023fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55938fa7e0c0 con 0x55938fb89230 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.505+0000 7f37023fd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55938fb59770 con 0x55938fb9d3d0 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f37023fd740 10 client.0 _ll_register_callbacks cb 0x55938fa71ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36fcc9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb9ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36fcc9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb9ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54064/0 (socket says 172.21.15.19:54064) 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36fcc9d640 1 -- 172.21.15.19:0/2932092795 learned_addr learned my addr 172.21.15.19:0/2932092795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36fd49e640 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb89230 0x55938fb77a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36fdc9f640 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb9d3d0 0x55938fb8cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36fcc9d640 1 -- 172.21.15.19:0/2932092795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb9d3d0 msgr2=0x55938fb8cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36fcc9d640 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb9d3d0 0x55938fb8cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36fcc9d640 1 -- 172.21.15.19:0/2932092795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb89230 msgr2=0x55938fb77a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36fcc9d640 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb89230 0x55938fb77a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36fcc9d640 1 -- 172.21.15.19:0/2932092795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55938f7d62d0 con 0x55938fba6ad0 2024-04-25T09:56:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36fcc9d640 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb9ce90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f36e8054230 tx=0x7f36e80879a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adda0c8dd5770a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36d7fff640 1 -- 172.21.15.19:0/2932092795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8085070 con 0x55938fba6ad0 2024-04-25T09:56:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36d7fff640 1 -- 172.21.15.19:0/2932092795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f36e8057220 con 0x55938fba6ad0 2024-04-25T09:56:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f37023fd740 1 -- 172.21.15.19:0/2932092795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36e805e0c0 con 0x55938fba6ad0 2024-04-25T09:56:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f36d7fff640 1 -- 172.21.15.19:0/2932092795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36e8087e60 con 0x55938fba6ad0 2024-04-25T09:56:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f37023fd740 10 client.6753 fetch_fsmap learned FSMap version 35 2024-04-25T09:56:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.506+0000 7f37023fd740 1 -- 172.21.15.19:0/2932092795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55938f8a06b0 con 0x55938fba6ad0 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.507+0000 7f36d7fff640 1 -- 172.21.15.19:0/2932092795 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e805e0c0 con 0x55938fba6ad0 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.507+0000 7f36d7fff640 1 -- 172.21.15.19:0/2932092795 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f36e806bce0 con 0x55938fba6ad0 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.507+0000 7f37023fd740 10 client.6753 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.507+0000 7f37023fd740 10 client.6753 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.507+0000 7f37023fd740 20 client.6753 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.507+0000 7f37023fd740 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.19:6832/591570980,v1:172.21.15.19:6834/591570980] conn(0x55938fbb7820 0x55938fbb7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.507+0000 7f37023fd740 4 client.6753 mds_command: new command op to 6582 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.507+0000 7f37023fd740 1 -- 172.21.15.19:0/2932092795 --> [v2:172.21.15.19:6832/591570980,v1:172.21.15.19:6834/591570980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55938fb20920 con 0x55938fbb7820 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.507+0000 7f36fd49e640 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.19:6832/591570980,v1:172.21.15.19:6834/591570980] conn(0x55938fbb7820 0x55938fbb7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.507+0000 7f36d7fff640 10 client.6753 ms_handle_connect on v2:172.21.15.19:6832/591570980 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.507+0000 7f36fd49e640 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.19:6832/591570980,v1:172.21.15.19:6834/591570980] conn(0x55938fbb7820 0x55938fbb7c00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=208db160eaba4c15 server_cookie=180191b23e02cf60 in_seq=0 out_seq=0 2024-04-25T09:56:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f36d7fff640 1 -- 172.21.15.19:0/2932092795 <== mds.0 v2:172.21.15.19:6832/591570980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55938fb20920 con 0x55938fbb7820 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f36d7fff640 10 client.6753 handle_command_reply: tid=0 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 1 client.6753 shutdown 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 1 -- 172.21.15.19:0/2932092795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 msgr2=0x55938fb9ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb9ce90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f36e8054230 tx=0x7f36e80879a0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 1 -- 172.21.15.19:0/2932092795 shutdown_connections 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.19:6832/591570980,v1:172.21.15.19:6834/591570980] conn(0x55938fbb7820 0x55938fbb7c00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb9d3d0 0x55938fb8cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f36d7fff640 0 client.6753 ms_handle_reset on v2:172.21.15.19:6832/591570980 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb9ce90 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 1 --2- 172.21.15.19:0/2932092795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb89230 0x55938fb77a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 1 -- 172.21.15.19:0/2932092795 >> 172.21.15.19:0/2932092795 conn(0x55938fb5ef10 msgr2=0x55938fba7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 1 -- 172.21.15.19:0/2932092795 shutdown_connections 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 1 -- 172.21.15.19:0/2932092795 wait complete. 2024-04-25T09:56:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.508+0000 7f37023fd740 20 client.6753 trim_cache size 0 max 16384 2024-04-25T09:56:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.524+0000 7f37023fd740 1 Processor -- start 2024-04-25T09:56:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.524+0000 7f37023fd740 1 -- start start 2024-04-25T09:56:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f37023fd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fb89230 0x55938fb5e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f37023fd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb9d3d0 0x55938fba5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f37023fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fba6ad0 0x55938fba5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f37023fd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f81ff7f0 con 0x55938fb89230 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f37023fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f814e5a0 con 0x55938fba6ad0 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f37023fd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f81ff1f0 con 0x55938fb9d3d0 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36fd49e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fb89230 0x55938fb5e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36fd49e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fb89230 0x55938fb5e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54068/0 (socket says 172.21.15.19:54068) 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36fd49e640 1 -- 172.21.15.19:0/1728497875 learned_addr learned my addr 172.21.15.19:0/1728497875 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36fd49e640 1 -- 172.21.15.19:0/1728497875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb9d3d0 msgr2=0x55938fba5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36fd49e640 1 --2- 172.21.15.19:0/1728497875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb9d3d0 0x55938fba5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36fd49e640 1 -- 172.21.15.19:0/1728497875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fba6ad0 msgr2=0x55938fba5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36fd49e640 1 --2- 172.21.15.19:0/1728497875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fba6ad0 0x55938fba5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36fd49e640 1 -- 172.21.15.19:0/1728497875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55938fb9b830 con 0x55938fb89230 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36fd49e640 1 --2- 172.21.15.19:0/1728497875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fb89230 0x55938fb5e730 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f36f4016040 tx=0x7f36f4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b235f725212409f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36e27fc640 1 -- 172.21.15.19:0/1728497875 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f402ab70 con 0x55938fb89230 2024-04-25T09:56:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36e27fc640 1 -- 172.21.15.19:0/1728497875 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f401b040 con 0x55938fb89230 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.525+0000 7f36e27fc640 1 -- 172.21.15.19:0/1728497875 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f402a2c0 con 0x55938fb89230 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.526+0000 7f37023fd740 1 -- 172.21.15.19:0/1728497875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fb89230 msgr2=0x55938fb5e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.526+0000 7f37023fd740 1 --2- 172.21.15.19:0/1728497875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fb89230 0x55938fb5e730 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f36f4016040 tx=0x7f36f4018000 comp rx=0 tx=0).stop 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.526+0000 7f37023fd740 1 -- 172.21.15.19:0/1728497875 shutdown_connections 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.526+0000 7f37023fd740 1 --2- 172.21.15.19:0/1728497875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fba6ad0 0x55938fba5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.526+0000 7f37023fd740 1 --2- 172.21.15.19:0/1728497875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb9d3d0 0x55938fba5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.526+0000 7f37023fd740 1 --2- 172.21.15.19:0/1728497875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fb89230 0x55938fb5e730 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.526+0000 7f37023fd740 1 -- 172.21.15.19:0/1728497875 >> 172.21.15.19:0/1728497875 conn(0x55938fb5ef10 msgr2=0x55938fba6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.526+0000 7f37023fd740 1 -- 172.21.15.19:0/1728497875 shutdown_connections 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.526+0000 7f37023fd740 1 -- 172.21.15.19:0/1728497875 wait complete. 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.526+0000 7f37023fd740 1 -- start start 2024-04-25T09:56:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.527+0000 7f37023fd740 1 Processor -- start 2024-04-25T09:56:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.527+0000 7f37023fd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb89230 0x55938fb88240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.527+0000 7f37023fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb9d3d0 0x55938fb88780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.527+0000 7f37023fd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb5cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.527+0000 7f37023fd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55938fb5fcb0 con 0x55938fba6ad0 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.527+0000 7f37023fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55938fa7e0c0 con 0x55938fb9d3d0 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.527+0000 7f37023fd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55938fb8a530 con 0x55938fb89230 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.527+0000 7f37023fd740 10 client.0 _ll_register_callbacks cb 0x55938fb59c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36fdc9f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb5cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36fdc9f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb5cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54080/0 (socket says 172.21.15.19:54080) 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36fdc9f640 1 -- 172.21.15.19:0/156162464 learned_addr learned my addr 172.21.15.19:0/156162464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36fd49e640 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb89230 0x55938fb88240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36fcc9d640 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb9d3d0 0x55938fb88780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36fdc9f640 1 -- 172.21.15.19:0/156162464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb89230 msgr2=0x55938fb88240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36fdc9f640 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb89230 0x55938fb88240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36fdc9f640 1 -- 172.21.15.19:0/156162464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb9d3d0 msgr2=0x55938fb88780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36fdc9f640 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb9d3d0 0x55938fb88780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36fdc9f640 1 -- 172.21.15.19:0/156162464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55938fb59fa0 con 0x55938fba6ad0 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36fdc9f640 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb5cde0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009e50 tx=0x7f36f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3689f75c5daf3dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36d7fff640 1 -- 172.21.15.19:0/156162464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f004c060 con 0x55938fba6ad0 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36d7fff640 1 -- 172.21.15.19:0/156162464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f36f003e6d0 con 0x55938fba6ad0 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f37023fd740 1 -- 172.21.15.19:0/156162464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36f401b040 con 0x55938fba6ad0 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f36d7fff640 1 -- 172.21.15.19:0/156162464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36f0052050 con 0x55938fba6ad0 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f37023fd740 10 client.6759 fetch_fsmap learned FSMap version 35 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.528+0000 7f37023fd740 1 -- 172.21.15.19:0/156162464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55938fc10090 con 0x55938fba6ad0 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f36d7fff640 1 -- 172.21.15.19:0/156162464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f401b040 con 0x55938fba6ad0 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f36d7fff640 1 -- 172.21.15.19:0/156162464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f36f0055020 con 0x55938fba6ad0 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 10 client.6759 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 10 client.6759 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 20 client.6759 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.19:6832/591570980,v1:172.21.15.19:6834/591570980] conn(0x55938fb5d920 0x55938fc0f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 4 client.6759 mds_command: new command op to 6582 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 1 -- 172.21.15.19:0/156162464 --> [v2:172.21.15.19:6832/591570980,v1:172.21.15.19:6834/591570980] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55938facef50 con 0x55938fb5d920 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f36fd49e640 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.19:6832/591570980,v1:172.21.15.19:6834/591570980] conn(0x55938fb5d920 0x55938fc0f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f36fd49e640 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.19:6832/591570980,v1:172.21.15.19:6834/591570980] conn(0x55938fb5d920 0x55938fc0f010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d032f1a0420317 server_cookie=8c001a4d057f5c68 in_seq=0 out_seq=0 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f36d7fff640 10 client.6759 ms_handle_connect on v2:172.21.15.19:6832/591570980 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f36d7fff640 1 -- 172.21.15.19:0/156162464 <== mds.0 v2:172.21.15.19:6832/591570980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55938facef50 con 0x55938fb5d920 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f36d7fff640 10 client.6759 handle_command_reply: tid=0 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 1 client.6759 shutdown 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 1 -- 172.21.15.19:0/156162464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 msgr2=0x55938fb5cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb5cde0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009e50 tx=0x7f36f0004a40 comp rx=0 tx=0).stop 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 1 -- 172.21.15.19:0/156162464 shutdown_connections 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.19:6832/591570980,v1:172.21.15.19:6834/591570980] conn(0x55938fb5d920 0x55938fc0f010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55938fba6ad0 0x55938fb5cde0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938fb9d3d0 0x55938fb88780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f36d7fff640 0 client.6759 ms_handle_reset on v2:172.21.15.19:6832/591570980 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 1 --2- 172.21.15.19:0/156162464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55938fb89230 0x55938fb88240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.529+0000 7f37023fd740 1 -- 172.21.15.19:0/156162464 >> 172.21.15.19:0/156162464 conn(0x55938fb5ef10 msgr2=0x55938fba6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f37023fd740 1 -- 172.21.15.19:0/156162464 shutdown_connections 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f37023fd740 1 -- 172.21.15.19:0/156162464 wait complete. 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f37023fd740 20 client.6759 trim_cache size 0 max 16384 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 6582, 2024-04-25T09:56:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T09:56:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T09:56:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T09:56:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 9.0816868280000005, 2024-04-25T09:56:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 35, 2024-04-25T09:56:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 66, 2024-04-25T09:56:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 64, 2024-04-25T09:56:48.540 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 14.124042313 2024-04-25T09:56:48.540 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 -- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f36dc06b500 msgr2=0x7f36dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f36dc06b500 0x7f36dc06d9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f36f80aa590 tx=0x7f36f4009190 comp rx=0 tx=0).stop 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 -- 172.21.15.19:0/100715968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f815dc20 msgr2=0x7f36f80acf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f815dc20 0x7f36f80acf20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f36e800c3c0 tx=0x7f36e800c880 comp rx=0 tx=0).stop 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 -- 172.21.15.19:0/100715968 shutdown_connections 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f36dc06b500 0x7f36dc06d9c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36f8161710 0x7f36f80ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36f815dc20 0x7f36f80acf20 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 --2- 172.21.15.19:0/100715968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36f80d32f0 0x7f36f814c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 -- 172.21.15.19:0/100715968 >> 172.21.15.19:0/100715968 conn(0x7f36f80d0420 msgr2=0x7f36f80b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 -- 172.21.15.19:0/100715968 shutdown_connections 2024-04-25T09:56:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.530+0000 7f36ff702640 1 -- 172.21.15.19:0/100715968 wait complete. 2024-04-25T09:56:48.545 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:56:48.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.642+0000 7fb356d49640 1 Processor -- start 2024-04-25T09:56:48.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.643+0000 7fb356d49640 1 -- start start 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.643+0000 7fb356d49640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3501602e0 0x7fb3500ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.643+0000 7fb356d49640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb35015f5f0 0x7fb35015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.643+0000 7fb356d49640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3500cd710 con 0x7fb35015ba00 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.643+0000 7fb356d49640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb350099b70 con 0x7fb35015f5f0 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.643+0000 7fb356d49640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb350099e50 con 0x7fb3501602e0 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.643+0000 7fb3552e6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb35015f5f0 0x7fb35015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.643+0000 7fb3552e6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb35015f5f0 0x7fb35015faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54082/0 (socket says 172.21.15.19:54082) 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.643+0000 7fb3552e6640 1 -- 172.21.15.19:0/2409214700 learned_addr learned my addr 172.21.15.19:0/2409214700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.644+0000 7fb34ffff640 1 --2- 172.21.15.19:0/2409214700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3501602e0 0x7fb3500ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.644+0000 7fb3552e6640 1 -- 172.21.15.19:0/2409214700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3501602e0 msgr2=0x7fb3500ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.644+0000 7fb3552e6640 1 --2- 172.21.15.19:0/2409214700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3501602e0 0x7fb3500ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.644+0000 7fb3552e6640 1 -- 172.21.15.19:0/2409214700 >> v1:172.21.15.112:6789/0 conn(0x7fb35015ba00 legacy=0x7fb35009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:48.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.644+0000 7fb3552e6640 1 -- 172.21.15.19:0/2409214700 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35009a130 con 0x7fb35015f5f0 2024-04-25T09:56:48.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.644+0000 7fb34ffff640 1 --2- 172.21.15.19:0/2409214700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3501602e0 0x7fb3500ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:48.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.644+0000 7fb3552e6640 1 --2- 172.21.15.19:0/2409214700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35015f5f0 0x7fb35015faa0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb340009870 tx=0x7fb34002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff75fc89389a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:48.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.645+0000 7fb34f7fe640 1 -- 172.21.15.19:0/2409214700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340037070 con 0x7fb35015f5f0 2024-04-25T09:56:48.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.645+0000 7fb34f7fe640 1 -- 172.21.15.19:0/2409214700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34002ae10 con 0x7fb35015f5f0 2024-04-25T09:56:48.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.645+0000 7fb34f7fe640 1 -- 172.21.15.19:0/2409214700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340032660 con 0x7fb35015f5f0 2024-04-25T09:56:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.645+0000 7fb356d49640 1 -- 172.21.15.19:0/2409214700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35015f5f0 msgr2=0x7fb35015faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.645+0000 7fb356d49640 1 --2- 172.21.15.19:0/2409214700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35015f5f0 0x7fb35015faa0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb340009870 tx=0x7fb34002a100 comp rx=0 tx=0).stop 2024-04-25T09:56:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.645+0000 7fb356d49640 1 -- 172.21.15.19:0/2409214700 shutdown_connections 2024-04-25T09:56:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.645+0000 7fb356d49640 1 --2- 172.21.15.19:0/2409214700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35015f5f0 0x7fb35015faa0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.645+0000 7fb356d49640 1 --2- 172.21.15.19:0/2409214700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3501602e0 0x7fb3500ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.645+0000 7fb356d49640 1 -- 172.21.15.19:0/2409214700 >> 172.21.15.19:0/2409214700 conn(0x7fb3500b3270 msgr2=0x7fb3500b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.645+0000 7fb356d49640 1 -- 172.21.15.19:0/2409214700 shutdown_connections 2024-04-25T09:56:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.645+0000 7fb356d49640 1 -- 172.21.15.19:0/2409214700 wait complete. 2024-04-25T09:56:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.646+0000 7fb356d49640 1 Processor -- start 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.646+0000 7fb356d49640 1 -- start start 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.647+0000 7fb356d49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35015ba00 0x7fb3500b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.647+0000 7fb356d49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb35015f5f0 0x7fb3500c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.647+0000 7fb356d49640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3501602e0 0x7fb35015e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.647+0000 7fb356d49640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500cd460 con 0x7fb35015ba00 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.647+0000 7fb356d49640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500cd2e0 con 0x7fb35015f5f0 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.647+0000 7fb356d49640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500c52d0 con 0x7fb3501602e0 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.647+0000 7fb354ae5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35015ba00 0x7fb3500b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.647+0000 7fb354ae5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35015ba00 0x7fb3500b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54090/0 (socket says 172.21.15.19:54090) 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.647+0000 7fb354ae5640 1 -- 172.21.15.19:0/684728975 learned_addr learned my addr 172.21.15.19:0/684728975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb3552e6640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3501602e0 0x7fb35015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb34ffff640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb35015f5f0 0x7fb3500c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb354ae5640 1 -- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3501602e0 msgr2=0x7fb35015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb354ae5640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3501602e0 0x7fb35015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb354ae5640 1 -- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb35015f5f0 msgr2=0x7fb3500c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb354ae5640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb35015f5f0 0x7fb3500c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb354ae5640 1 -- 172.21.15.19:0/684728975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3500c9130 con 0x7fb35015ba00 2024-04-25T09:56:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb354ae5640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35015ba00 0x7fb3500b5070 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb34800c7f0 tx=0x7fb34800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb34d7fa640 1 -- 172.21.15.19:0/684728975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348016020 con 0x7fb35015ba00 2024-04-25T09:56:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb34d7fa640 1 -- 172.21.15.19:0/684728975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34800aa50 con 0x7fb35015ba00 2024-04-25T09:56:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb356d49640 1 -- 172.21.15.19:0/684728975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3500c9410 con 0x7fb35015ba00 2024-04-25T09:56:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb34d7fa640 1 -- 172.21.15.19:0/684728975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34800ae40 con 0x7fb35015ba00 2024-04-25T09:56:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.648+0000 7fb356d49640 1 -- 172.21.15.19:0/684728975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3500b58f0 con 0x7fb35015ba00 2024-04-25T09:56:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.649+0000 7fb356d49640 1 -- 172.21.15.19:0/684728975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb350073990 con 0x7fb35015ba00 2024-04-25T09:56:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.650+0000 7fb34d7fa640 1 -- 172.21.15.19:0/684728975 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb348019020 con 0x7fb35015ba00 2024-04-25T09:56:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.650+0000 7fb34d7fa640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb33c06b4d0 0x7fb33c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.650+0000 7fb34d7fa640 1 -- 172.21.15.19:0/684728975 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fb34808a510 con 0x7fb35015ba00 2024-04-25T09:56:48.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.650+0000 7fb34ffff640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb33c06b4d0 0x7fb33c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:48.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.651+0000 7fb34ffff640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb33c06b4d0 0x7fb33c06d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb350158930 tx=0x7fb344008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:48.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.654+0000 7fb34d7fa640 1 -- 172.21.15.19:0/684728975 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb348020030 con 0x7fb35015ba00 2024-04-25T09:56:48.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.875+0000 7fb356d49640 1 -- 172.21.15.19:0/684728975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb3500b4080 con 0x7fb35015ba00 2024-04-25T09:56:48.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.875+0000 7fb34d7fa640 1 -- 172.21.15.19:0/684728975 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb348059100 con 0x7fb35015ba00 2024-04-25T09:56:48.878 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:56:48.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 -- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb33c06b4d0 msgr2=0x7fb33c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb33c06b4d0 0x7fb33c06d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb350158930 tx=0x7fb344008040 comp rx=0 tx=0).stop 2024-04-25T09:56:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 -- 172.21.15.19:0/684728975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35015ba00 msgr2=0x7fb3500b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35015ba00 0x7fb3500b5070 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb34800c7f0 tx=0x7fb34800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 -- 172.21.15.19:0/684728975 shutdown_connections 2024-04-25T09:56:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb33c06b4d0 0x7fb33c06d990 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3501602e0 0x7fb35015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb35015f5f0 0x7fb3500c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 --2- 172.21.15.19:0/684728975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35015ba00 0x7fb3500b5070 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 -- 172.21.15.19:0/684728975 >> 172.21.15.19:0/684728975 conn(0x7fb3500b3270 msgr2=0x7fb3500c5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 -- 172.21.15.19:0/684728975 shutdown_connections 2024-04-25T09:56:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:48.877+0000 7fb356d49640 1 -- 172.21.15.19:0/684728975 wait complete. 2024-04-25T09:56:48.891 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:56:48.891 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:56:48.891 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T09:56:48.891 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:56:48.891 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:48.891 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:56:48.892 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:56:48.892 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:56:48.892 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:56:48.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:48.892 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T09:56:48.907 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T09:56:48.908 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42537sec preferred_lft 42537sec 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:48.909 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:56:48.910 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:56:48.910 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T09:56:48.910 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:48.910 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:56:48.910 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:56:48.910 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T09:56:48.910 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:48.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:48.910 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:56:48.971 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T09:56:48.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:48.973 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T09:56:49.029 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T09:56:49.029 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:56:49.029 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:56:49.029 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:56:49.029 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:56:49.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:49.030 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:56:49.030 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:56:49.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:56:49.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:56:49.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:56:49.030 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:56:49.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:49.115 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:56:49.130 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:56:49.131 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:56:49.131 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:56:49.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:49.131 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T09:56:49.195 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:56:49.195 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:49.251 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:56:49.251 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:56:49.251 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:49.306 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:56:49.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:49.306 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:56:49.388 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:56:49.389 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:56:49.397 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:56:49.398 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:56:49.398 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T09:56:49.398 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T09:56:49.398 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T09:56:49.398 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:56:49.398 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:56:49.398 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:56:49.398 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:56:49.398 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:56:49.400 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:56:49.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:49.400 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:49.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:49.465 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:56:49.526 INFO:teuthology.orchestra.run.smithi019.stdout:client5693 2024-04-25T09:56:49.527 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:56:49.527 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:49.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:49.586 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T09:56:49.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:49.644 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T09:56:49.701 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T09:56:49.701 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T09:56:49.701 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T09:56:49.701 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:56:49.701 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T09:56:49.701 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T09:56:49.807 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627776 2024-04-25T09:56:49.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:56:49.816 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:56:49.816 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:49.873 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:56:49.873 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:49.929 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:50.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.066+0000 7f9c64012640 1 Processor -- start 2024-04-25T09:56:50.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.066+0000 7f9c64012640 1 -- start start 2024-04-25T09:56:50.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.067+0000 7f9c64012640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c15dc20 0x7f9c5c1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.067+0000 7f9c64012640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9c5c0b93a0 0x7f9c5c0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.067+0000 7f9c64012640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c5c0cec20 0x7f9c5c06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.067+0000 7f9c64012640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c15e000 con 0x7f9c5c0cec20 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.067+0000 7f9c64012640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c0b6bb0 con 0x7f9c5c0b93a0 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.067+0000 7f9c64012640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0b6030 con 0x7f9c5c15dc20 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.067+0000 7f9c625af640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c5c0cec20 0x7f9c5c06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.067+0000 7f9c625af640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c5c0cec20 0x7f9c5c06c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54106/0 (socket says 172.21.15.19:54106) 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.067+0000 7f9c625af640 1 -- 172.21.15.19:0/4225005182 learned_addr learned my addr 172.21.15.19:0/4225005182 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.067+0000 7f9c615ad640 1 --2- 172.21.15.19:0/4225005182 >> v2:172.21.15.112:3300/0 conn(0x7f9c5c0b93a0 0x7f9c5c0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.067+0000 7f9c61dae640 1 --2- 172.21.15.19:0/4225005182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c15dc20 0x7f9c5c1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.068+0000 7f9c625af640 1 -- 172.21.15.19:0/4225005182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c15dc20 msgr2=0x7f9c5c1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.068+0000 7f9c625af640 1 --2- 172.21.15.19:0/4225005182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c15dc20 0x7f9c5c1616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.068+0000 7f9c625af640 1 -- 172.21.15.19:0/4225005182 >> v2:172.21.15.112:3300/0 conn(0x7f9c5c0b93a0 msgr2=0x7f9c5c0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.068+0000 7f9c625af640 1 --2- 172.21.15.19:0/4225005182 >> v2:172.21.15.112:3300/0 conn(0x7f9c5c0b93a0 0x7f9c5c0ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.068+0000 7f9c625af640 1 -- 172.21.15.19:0/4225005182 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c161de0 con 0x7f9c5c0cec20 2024-04-25T09:56:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.068+0000 7f9c625af640 1 --2- 172.21.15.19:0/4225005182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c0cec20 0x7f9c5c06c0d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0d4a20 tx=0x7f9c5802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8554f74d851e7936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:50.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.068+0000 7f9c60dac640 1 -- 172.21.15.19:0/4225005182 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58036070 con 0x7f9c5c0cec20 2024-04-25T09:56:50.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.068+0000 7f9c60dac640 1 -- 172.21.15.19:0/4225005182 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c58031600 con 0x7f9c5c0cec20 2024-04-25T09:56:50.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.069+0000 7f9c60dac640 1 -- 172.21.15.19:0/4225005182 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c580319d0 con 0x7f9c5c0cec20 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.069+0000 7f9c64012640 1 -- 172.21.15.19:0/4225005182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c0cec20 msgr2=0x7f9c5c06c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.069+0000 7f9c64012640 1 --2- 172.21.15.19:0/4225005182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c0cec20 0x7f9c5c06c0d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0d4a20 tx=0x7f9c5802a6a0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.069+0000 7f9c64012640 1 -- 172.21.15.19:0/4225005182 shutdown_connections 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.069+0000 7f9c64012640 1 --2- 172.21.15.19:0/4225005182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c0cec20 0x7f9c5c06c0d0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.069+0000 7f9c64012640 1 --2- 172.21.15.19:0/4225005182 >> v2:172.21.15.112:3300/0 conn(0x7f9c5c0b93a0 0x7f9c5c0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.069+0000 7f9c64012640 1 --2- 172.21.15.19:0/4225005182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c15dc20 0x7f9c5c1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.069+0000 7f9c64012640 1 -- 172.21.15.19:0/4225005182 >> 172.21.15.19:0/4225005182 conn(0x7f9c5c0d0490 msgr2=0x7f9c5c0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.069+0000 7f9c64012640 1 -- 172.21.15.19:0/4225005182 shutdown_connections 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.069+0000 7f9c64012640 1 -- 172.21.15.19:0/4225005182 wait complete. 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.070+0000 7f9c64012640 1 Processor -- start 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.070+0000 7f9c64012640 1 -- start start 2024-04-25T09:56:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.071+0000 7f9c64012640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0b93a0 0x7f9c5c149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.071+0000 7f9c64012640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c0cec20 0x7f9c5c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.071+0000 7f9c64012640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c15dc20 0x7f9c5c1f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.071+0000 7f9c64012640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c06c610 con 0x7f9c5c0cec20 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.071+0000 7f9c64012640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c1623a0 con 0x7f9c5c15dc20 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.071+0000 7f9c64012640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c06c790 con 0x7f9c5c0b93a0 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.071+0000 7f9c615ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c0cec20 0x7f9c5c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.071+0000 7f9c615ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c0cec20 0x7f9c5c0cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54116/0 (socket says 172.21.15.19:54116) 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.071+0000 7f9c615ad640 1 -- 172.21.15.19:0/3192696514 learned_addr learned my addr 172.21.15.19:0/3192696514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.071+0000 7f9c61dae640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0b93a0 0x7f9c5c149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.071+0000 7f9c625af640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c15dc20 0x7f9c5c1f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c615ad640 1 -- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0b93a0 msgr2=0x7f9c5c149a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c615ad640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0b93a0 0x7f9c5c149a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c615ad640 1 -- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c15dc20 msgr2=0x7f9c5c1f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c615ad640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c15dc20 0x7f9c5c1f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c615ad640 1 -- 172.21.15.19:0/3192696514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c0cca60 con 0x7f9c5c0cec20 2024-04-25T09:56:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c61dae640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0b93a0 0x7f9c5c149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c625af640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c15dc20 0x7f9c5c1f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c615ad640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c0cec20 0x7f9c5c0cd7b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00d830 tx=0x7f9c4c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c527fc640 1 -- 172.21.15.19:0/3192696514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c018020 con 0x7f9c5c0cec20 2024-04-25T09:56:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c527fc640 1 -- 172.21.15.19:0/3192696514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c0096a0 con 0x7f9c5c0cec20 2024-04-25T09:56:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c64012640 1 -- 172.21.15.19:0/3192696514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c0cccd0 con 0x7f9c5c0cec20 2024-04-25T09:56:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c527fc640 1 -- 172.21.15.19:0/3192696514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c009a70 con 0x7f9c5c0cec20 2024-04-25T09:56:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.072+0000 7f9c64012640 1 -- 172.21.15.19:0/3192696514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c0cd290 con 0x7f9c5c0cec20 2024-04-25T09:56:50.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.073+0000 7f9c64012640 1 -- 172.21.15.19:0/3192696514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c073990 con 0x7f9c5c0cec20 2024-04-25T09:56:50.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.073+0000 7f9c527fc640 1 -- 172.21.15.19:0/3192696514 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c4c01b020 con 0x7f9c5c0cec20 2024-04-25T09:56:50.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.074+0000 7f9c527fc640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c4006b500 0x7f9c4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.074+0000 7f9c527fc640 1 -- 172.21.15.19:0/3192696514 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f9c4c08a780 con 0x7f9c5c0cec20 2024-04-25T09:56:50.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.074+0000 7f9c61dae640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c4006b500 0x7f9c4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.075+0000 7f9c61dae640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c4006b500 0x7f9c4006d9c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c1508d0 tx=0x7f9c54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.078+0000 7f9c527fc640 1 -- 172.21.15.19:0/3192696514 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c4c022030 con 0x7f9c5c0cec20 2024-04-25T09:56:50.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.257+0000 7f9c64012640 1 -- 172.21.15.19:0/3192696514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c5c161470 con 0x7f9c5c0cec20 2024-04-25T09:56:50.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.258+0000 7f9c527fc640 1 -- 172.21.15.19:0/3192696514 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+17064 (secure 0 0 0) 0x7f9c4c0102e0 con 0x7f9c5c0cec20 2024-04-25T09:56:50.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:50.262 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":66,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:56:48.480791+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:56:37.087339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:56:37.775183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 -- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c4006b500 msgr2=0x7f9c4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c4006b500 0x7f9c4006d9c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c1508d0 tx=0x7f9c54008040 comp rx=0 tx=0).stop 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 -- 172.21.15.19:0/3192696514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c0cec20 msgr2=0x7f9c5c0cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c0cec20 0x7f9c5c0cd7b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00d830 tx=0x7f9c4c00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 -- 172.21.15.19:0/3192696514 shutdown_connections 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c4006b500 0x7f9c4006d9c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c15dc20 0x7f9c5c1f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c0cec20 0x7f9c5c0cd7b0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 --2- 172.21.15.19:0/3192696514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0b93a0 0x7f9c5c149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 -- 172.21.15.19:0/3192696514 >> 172.21.15.19:0/3192696514 conn(0x7f9c5c0d0490 msgr2=0x7f9c5c0b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 -- 172.21.15.19:0/3192696514 shutdown_connections 2024-04-25T09:56:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.260+0000 7f9c64012640 1 -- 172.21.15.19:0/3192696514 wait complete. 2024-04-25T09:56:50.274 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:56:37.087339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:56:37.775183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:56:50.275 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:56:50.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.375+0000 7f875e95c640 1 Processor -- start 2024-04-25T09:56:50.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.375+0000 7f875e95c640 1 -- start start 2024-04-25T09:56:50.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.376+0000 7f875e95c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f875806ccd0 0x7f87580b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.376+0000 7f875e95c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87580d1770 con 0x7f8758070dc0 2024-04-25T09:56:50.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.376+0000 7f875e95c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8758162740 con 0x7f875815deb0 2024-04-25T09:56:50.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.376+0000 7f875e95c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87581622b0 con 0x7f875806ccd0 2024-04-25T09:56:50.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.376+0000 7f8757fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8758070dc0 0x7f87580711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58064/0 (socket says 172.21.15.19:58064) 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.376+0000 7f8757fff640 1 -- 172.21.15.19:0/2845607510 learned_addr learned my addr 172.21.15.19:0/2845607510 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f87577fe640 1 --2- 172.21.15.19:0/2845607510 >> v2:172.21.15.112:3300/0 conn(0x7f875806ccd0 0x7f87580b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346306261 0 0) 0x7f87580d1770 con 0x7f8758070dc0 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8740003520 con 0x7f8758070dc0 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2725970650 0 0) 0x7f8740003520 con 0x7f8758070dc0 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 >> v2:172.21.15.112:3300/0 conn(0x7f875806ccd0 msgr2=0x7f87580b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 --2- 172.21.15.19:0/2845607510 >> v2:172.21.15.112:3300/0 conn(0x7f875806ccd0 0x7f87580b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 >> v1:172.21.15.112:6789/0 conn(0x7f875815deb0 legacy=0x7f875815e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758161980 con 0x7f8758070dc0 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8748001670 con 0x7f8758070dc0 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 >> v1:172.21.15.19:6789/0 conn(0x7f8758070dc0 legacy=0x7f87580711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87400039b0 0x7f8740003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8740004800 0x7f8740004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87400052e0 0x7f874000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87581622b0 con 0x7f87400039b0 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874000c1b0 con 0x7f87400052e0 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874000c3b0 con 0x7f8740004800 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f8757fff640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87400039b0 0x7f8740003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f87577fe640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8740004800 0x7f8740004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.377+0000 7f875cef9640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87400052e0 0x7f874000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.378+0000 7f8757fff640 1 -- 172.21.15.19:0/2845607510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8740004800 msgr2=0x7f8740004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.378+0000 7f8757fff640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8740004800 0x7f8740004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.378+0000 7f8757fff640 1 -- 172.21.15.19:0/2845607510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87400052e0 msgr2=0x7f874000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.378+0000 7f8757fff640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87400052e0 0x7f874000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.378+0000 7f8757fff640 1 -- 172.21.15.19:0/2845607510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8740003520 con 0x7f87400039b0 2024-04-25T09:56:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.378+0000 7f875cef9640 1 -- 172.21.15.19:0/2845607510 reap_dead start 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.378+0000 7f87577fe640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8740004800 0x7f8740004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.378+0000 7f8757fff640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87400039b0 0x7f8740003dd0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f874800c210 tx=0x7f8740005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da75fe55874daf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.378+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748007cf0 con 0x7f87400039b0 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.378+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8748005af0 con 0x7f87400039b0 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.378+0000 7f8756ffd640 1 -- 172.21.15.19:0/2845607510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748007070 con 0x7f87400039b0 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.379+0000 7f875e95c640 1 -- 172.21.15.19:0/2845607510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87400039b0 msgr2=0x7f8740003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.379+0000 7f875e95c640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87400039b0 0x7f8740003dd0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f874800c210 tx=0x7f8740005830 comp rx=0 tx=0).stop 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.379+0000 7f875e95c640 1 -- 172.21.15.19:0/2845607510 shutdown_connections 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.379+0000 7f875e95c640 1 --2- 172.21.15.19:0/2845607510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87400039b0 0x7f8740003dd0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.379+0000 7f875e95c640 1 -- 172.21.15.19:0/2845607510 >> 172.21.15.19:0/2845607510 conn(0x7f87580b3530 msgr2=0x7f87580b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.379+0000 7f875e95c640 1 -- 172.21.15.19:0/2845607510 shutdown_connections 2024-04-25T09:56:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.379+0000 7f875e95c640 1 -- 172.21.15.19:0/2845607510 wait complete. 2024-04-25T09:56:50.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.380+0000 7f875e95c640 1 Processor -- start 2024-04-25T09:56:50.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.380+0000 7f875e95c640 1 -- start start 2024-04-25T09:56:50.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f875e95c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87400039b0 0x7f87580cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f875e95c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758160d80 0x7f875814b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f875e95c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87580cc310 0x7f87580ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f875e95c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8758161f40 con 0x7f8758160d80 2024-04-25T09:56:50.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f875e95c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87580b10f0 con 0x7f87580cc310 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f875e95c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87580711a0 con 0x7f87400039b0 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f87577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758160d80 0x7f875814b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f87577fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758160d80 0x7f875814b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54132/0 (socket says 172.21.15.19:54132) 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f87577fe640 1 -- 172.21.15.19:0/2410545440 learned_addr learned my addr 172.21.15.19:0/2410545440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f875cef9640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87580cc310 0x7f87580ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f8757fff640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87400039b0 0x7f87580cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f87577fe640 1 -- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87400039b0 msgr2=0x7f87580cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f87577fe640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87400039b0 0x7f87580cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f87577fe640 1 -- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87580cc310 msgr2=0x7f87580ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f87577fe640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87580cc310 0x7f87580ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.381+0000 7f87577fe640 1 -- 172.21.15.19:0/2410545440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758161200 con 0x7f8758160d80 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.382+0000 7f87577fe640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758160d80 0x7f875814b490 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f874c0114e0 tx=0x7f874c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.382+0000 7f8754ff9640 1 -- 172.21.15.19:0/2410545440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c015870 con 0x7f8758160d80 2024-04-25T09:56:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.382+0000 7f8754ff9640 1 -- 172.21.15.19:0/2410545440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c015a10 con 0x7f8758160d80 2024-04-25T09:56:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.382+0000 7f875e95c640 1 -- 172.21.15.19:0/2410545440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87580cb270 con 0x7f8758160d80 2024-04-25T09:56:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.382+0000 7f8754ff9640 1 -- 172.21.15.19:0/2410545440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c015e00 con 0x7f8758160d80 2024-04-25T09:56:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.382+0000 7f875e95c640 1 -- 172.21.15.19:0/2410545440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87580cb830 con 0x7f8758160d80 2024-04-25T09:56:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.383+0000 7f875e95c640 1 -- 172.21.15.19:0/2410545440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8758073a00 con 0x7f8758160d80 2024-04-25T09:56:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.383+0000 7f8754ff9640 1 -- 172.21.15.19:0/2410545440 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f874c01d400 con 0x7f8758160d80 2024-04-25T09:56:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.384+0000 7f8754ff9640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f874406b4d0 0x7f874406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.384+0000 7f8754ff9640 1 -- 172.21.15.19:0/2410545440 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f874c08ac20 con 0x7f8758160d80 2024-04-25T09:56:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.384+0000 7f8757fff640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f874406b4d0 0x7f874406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.385+0000 7f8757fff640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f874406b4d0 0x7f874406d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8748032000 tx=0x7f8748009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.388+0000 7f8754ff9640 1 -- 172.21.15.19:0/2410545440 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f874c019030 con 0x7f8758160d80 2024-04-25T09:56:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.571+0000 7f875e95c640 1 -- 172.21.15.19:0/2410545440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87580b63c0 con 0x7f8758160d80 2024-04-25T09:56:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.571+0000 7f8754ff9640 1 -- 172.21.15.19:0/2410545440 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f874c059250 con 0x7f8758160d80 2024-04-25T09:56:50.575 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:56:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.573+0000 7f875e95c640 1 -- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f874406b4d0 msgr2=0x7f874406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.573+0000 7f875e95c640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f874406b4d0 0x7f874406d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8748032000 tx=0x7f8748009030 comp rx=0 tx=0).stop 2024-04-25T09:56:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.573+0000 7f875e95c640 1 -- 172.21.15.19:0/2410545440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758160d80 msgr2=0x7f875814b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.573+0000 7f875e95c640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758160d80 0x7f875814b490 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f874c0114e0 tx=0x7f874c0119a0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.574+0000 7f875e95c640 1 -- 172.21.15.19:0/2410545440 shutdown_connections 2024-04-25T09:56:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.574+0000 7f875e95c640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f874406b4d0 0x7f874406d990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.574+0000 7f875e95c640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87580cc310 0x7f87580ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.574+0000 7f875e95c640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8758160d80 0x7f875814b490 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.574+0000 7f875e95c640 1 --2- 172.21.15.19:0/2410545440 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87400039b0 0x7f87580cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.574+0000 7f875e95c640 1 -- 172.21.15.19:0/2410545440 >> 172.21.15.19:0/2410545440 conn(0x7f87580b3530 msgr2=0x7f875815d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:50.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.574+0000 7f875e95c640 1 -- 172.21.15.19:0/2410545440 shutdown_connections 2024-04-25T09:56:50.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.574+0000 7f875e95c640 1 -- 172.21.15.19:0/2410545440 wait complete. 2024-04-25T09:56:50.588 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:56:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.688+0000 7f7357d41640 1 Processor -- start 2024-04-25T09:56:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.688+0000 7f7357d41640 1 -- start start 2024-04-25T09:56:50.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f7357d41640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73500b5620 0x7f73500b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f7357d41640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f735006c2d0 0x7f735006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f7357d41640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735006cc80 con 0x7f735015b7d0 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f7357d41640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735006cf50 con 0x7f73500b5620 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f7357d41640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73500ce530 con 0x7f735006c2d0 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f73552dc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73500b5620 0x7f73500b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f73552dc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73500b5620 0x7f73500b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54136/0 (socket says 172.21.15.19:54136) 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f73552dc640 1 -- 172.21.15.19:0/1383289539 learned_addr learned my addr 172.21.15.19:0/1383289539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f73562de640 1 --2- 172.21.15.19:0/1383289539 >> v2:172.21.15.112:3300/0 conn(0x7f735006c2d0 0x7f735006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f73552dc640 1 -- 172.21.15.19:0/1383289539 >> v2:172.21.15.112:3300/0 conn(0x7f735006c2d0 msgr2=0x7f735006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f73552dc640 1 --2- 172.21.15.19:0/1383289539 >> v2:172.21.15.112:3300/0 conn(0x7f735006c2d0 0x7f735006c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f73552dc640 1 -- 172.21.15.19:0/1383289539 >> v1:172.21.15.112:6789/0 conn(0x7f735015b7d0 legacy=0x7f73500cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.689+0000 7f73552dc640 1 -- 172.21.15.19:0/1383289539 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73500ce810 con 0x7f73500b5620 2024-04-25T09:56:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.690+0000 7f73552dc640 1 --2- 172.21.15.19:0/1383289539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73500b5620 0x7f73500b5ab0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7350161890 tx=0x7f734402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923e7368bb1e78e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.690+0000 7f7354adb640 1 -- 172.21.15.19:0/1383289539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344037070 con 0x7f73500b5620 2024-04-25T09:56:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.690+0000 7f7354adb640 1 -- 172.21.15.19:0/1383289539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7344032440 con 0x7f73500b5620 2024-04-25T09:56:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.690+0000 7f7354adb640 1 -- 172.21.15.19:0/1383289539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344032810 con 0x7f73500b5620 2024-04-25T09:56:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.691+0000 7f7357d41640 1 -- 172.21.15.19:0/1383289539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73500b5620 msgr2=0x7f73500b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.691+0000 7f7357d41640 1 --2- 172.21.15.19:0/1383289539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73500b5620 0x7f73500b5ab0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7350161890 tx=0x7f734402a230 comp rx=0 tx=0).stop 2024-04-25T09:56:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.691+0000 7f7357d41640 1 -- 172.21.15.19:0/1383289539 shutdown_connections 2024-04-25T09:56:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.691+0000 7f7357d41640 1 --2- 172.21.15.19:0/1383289539 >> v2:172.21.15.112:3300/0 conn(0x7f735006c2d0 0x7f735006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.691+0000 7f7357d41640 1 --2- 172.21.15.19:0/1383289539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73500b5620 0x7f73500b5ab0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.691+0000 7f7357d41640 1 -- 172.21.15.19:0/1383289539 >> 172.21.15.19:0/1383289539 conn(0x7f73500d0800 msgr2=0x7f73500d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:50.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.691+0000 7f7357d41640 1 -- 172.21.15.19:0/1383289539 shutdown_connections 2024-04-25T09:56:50.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.691+0000 7f7357d41640 1 -- 172.21.15.19:0/1383289539 wait complete. 2024-04-25T09:56:50.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.692+0000 7f7357d41640 1 Processor -- start 2024-04-25T09:56:50.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.692+0000 7f7357d41640 1 -- start start 2024-04-25T09:56:50.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7357d41640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f735006c2d0 0x7f7350157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7357d41640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73500b5620 0x7f73500c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7357d41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f735015b7d0 0x7f73500c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7357d41640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735015bbb0 con 0x7f735006c2d0 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7357d41640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735006d230 con 0x7f735015b7d0 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7357d41640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735015bd30 con 0x7f73500b5620 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7355add640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f735006c2d0 0x7f7350157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7355add640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f735006c2d0 0x7f7350157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54140/0 (socket says 172.21.15.19:54140) 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7355add640 1 -- 172.21.15.19:0/2778636259 learned_addr learned my addr 172.21.15.19:0/2778636259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f73552dc640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73500b5620 0x7f73500c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f73562de640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f735015b7d0 0x7f73500c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7355add640 1 -- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73500b5620 msgr2=0x7f73500c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7355add640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73500b5620 0x7f73500c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7355add640 1 -- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f735015b7d0 msgr2=0x7f73500c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7355add640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f735015b7d0 0x7f73500c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7355add640 1 -- 172.21.15.19:0/2778636259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735015a6b0 con 0x7f735006c2d0 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f73562de640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f735015b7d0 0x7f73500c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.693+0000 7f7355add640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f735006c2d0 0x7f7350157ea0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f734000ba50 tx=0x7f734000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.694+0000 7f734e7fc640 1 -- 172.21.15.19:0/2778636259 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340019070 con 0x7f735006c2d0 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.694+0000 7f734e7fc640 1 -- 172.21.15.19:0/2778636259 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734000cc20 con 0x7f735006c2d0 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.694+0000 7f7357d41640 1 -- 172.21.15.19:0/2778636259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73500b71a0 con 0x7f735006c2d0 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.694+0000 7f734e7fc640 1 -- 172.21.15.19:0/2778636259 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340014680 con 0x7f735006c2d0 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.694+0000 7f7357d41640 1 -- 172.21.15.19:0/2778636259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73500b7a00 con 0x7f735006c2d0 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.695+0000 7f7357d41640 1 -- 172.21.15.19:0/2778636259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7350073a00 con 0x7f735006c2d0 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.695+0000 7f734e7fc640 1 -- 172.21.15.19:0/2778636259 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f734001c070 con 0x7f735006c2d0 2024-04-25T09:56:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.696+0000 7f734e7fc640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f733c06b4d0 0x7f733c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.696+0000 7f734e7fc640 1 -- 172.21.15.19:0/2778636259 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f7340089ff0 con 0x7f735006c2d0 2024-04-25T09:56:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.696+0000 7f73552dc640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f733c06b4d0 0x7f733c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.696+0000 7f73552dc640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f733c06b4d0 0x7f733c06d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7344034000 tx=0x7f734402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.700+0000 7f734e7fc640 1 -- 172.21.15.19:0/2778636259 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f735015a6b0 con 0x7f735006c2d0 2024-04-25T09:56:50.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.878+0000 7f7357d41640 1 -- 172.21.15.19:0/2778636259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73500b74f0 con 0x7f735006c2d0 2024-04-25T09:56:50.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.879+0000 7f734e7fc640 1 -- 172.21.15.19:0/2778636259 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73400586b0 con 0x7f735006c2d0 2024-04-25T09:56:50.882 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:56:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 -- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f733c06b4d0 msgr2=0x7f733c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f733c06b4d0 0x7f733c06d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7344034000 tx=0x7f734402e000 comp rx=0 tx=0).stop 2024-04-25T09:56:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 -- 172.21.15.19:0/2778636259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f735006c2d0 msgr2=0x7f7350157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f735006c2d0 0x7f7350157ea0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f734000ba50 tx=0x7f734000bf10 comp rx=0 tx=0).stop 2024-04-25T09:56:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 -- 172.21.15.19:0/2778636259 shutdown_connections 2024-04-25T09:56:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f733c06b4d0 0x7f733c06d990 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f735015b7d0 0x7f73500c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73500b5620 0x7f73500c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 --2- 172.21.15.19:0/2778636259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f735006c2d0 0x7f7350157ea0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 -- 172.21.15.19:0/2778636259 >> 172.21.15.19:0/2778636259 conn(0x7f73500d0800 msgr2=0x7f73500b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 -- 172.21.15.19:0/2778636259 shutdown_connections 2024-04-25T09:56:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.881+0000 7f7357d41640 1 -- 172.21.15.19:0/2778636259 wait complete. 2024-04-25T09:56:50.895 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:56:50.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.995+0000 7f998e3a5640 1 Processor -- start 2024-04-25T09:56:50.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.995+0000 7f998e3a5640 1 -- start start 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f998e3a5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f998815dac0 0x7f99880d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f998e3a5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f99880d0ab0 0x7f99880b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f998e3a5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99880d2420 con 0x7f99880d1f30 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f998e3a5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99880d3030 con 0x7f99880d0ab0 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f998e3a5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99880d2700 con 0x7f998815dac0 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f998c942640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f99880d0ab0 0x7f99880b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f99877fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f99880d1f30 0x7f99880d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58076/0 (socket says 172.21.15.19:58076) 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f99877fe640 1 -- 172.21.15.19:0/3281737398 learned_addr learned my addr 172.21.15.19:0/3281737398 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f998c942640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f99880d0ab0 0x7f99880b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54150/0 (socket says 172.21.15.19:54150) 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f9987fff640 1 --2- 172.21.15.19:0/3281737398 >> v2:172.21.15.112:3300/0 conn(0x7f998815dac0 0x7f99880d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f998c942640 1 -- 172.21.15.19:0/3281737398 >> v2:172.21.15.112:3300/0 conn(0x7f998815dac0 msgr2=0x7f99880d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f998c942640 1 --2- 172.21.15.19:0/3281737398 >> v2:172.21.15.112:3300/0 conn(0x7f998815dac0 0x7f99880d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f998c942640 1 -- 172.21.15.19:0/3281737398 >> v1:172.21.15.19:6789/0 conn(0x7f99880d1f30 legacy=0x7f99880d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f998c942640 1 -- 172.21.15.19:0/3281737398 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99880d29e0 con 0x7f99880d0ab0 2024-04-25T09:56:51.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.996+0000 7f9986ffd640 1 -- 172.21.15.19:0/3281737398 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397398547 0 0) 0x7f99880d2420 con 0x7f99880d1f30 2024-04-25T09:56:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.997+0000 7f998c942640 1 --2- 172.21.15.19:0/3281737398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99880d0ab0 0x7f99880b6990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f99880b6520 tx=0x7f998002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558a0497da407a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.997+0000 7f9986ffd640 1 -- 172.21.15.19:0/3281737398 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9980007740 con 0x7f99880d0ab0 2024-04-25T09:56:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.997+0000 7f9986ffd640 1 -- 172.21.15.19:0/3281737398 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99800078e0 con 0x7f99880d0ab0 2024-04-25T09:56:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.997+0000 7f9986ffd640 1 -- 172.21.15.19:0/3281737398 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9980007cb0 con 0x7f99880d0ab0 2024-04-25T09:56:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.998+0000 7f998e3a5640 1 -- 172.21.15.19:0/3281737398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99880d0ab0 msgr2=0x7f99880b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.998+0000 7f998e3a5640 1 --2- 172.21.15.19:0/3281737398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99880d0ab0 0x7f99880b6990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f99880b6520 tx=0x7f998002a710 comp rx=0 tx=0).stop 2024-04-25T09:56:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.998+0000 7f998e3a5640 1 -- 172.21.15.19:0/3281737398 shutdown_connections 2024-04-25T09:56:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.998+0000 7f998e3a5640 1 --2- 172.21.15.19:0/3281737398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99880d0ab0 0x7f99880b6990 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.998+0000 7f998e3a5640 1 --2- 172.21.15.19:0/3281737398 >> v2:172.21.15.112:3300/0 conn(0x7f998815dac0 0x7f99880d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.998+0000 7f998e3a5640 1 -- 172.21.15.19:0/3281737398 >> 172.21.15.19:0/3281737398 conn(0x7f9988061c30 msgr2=0x7f998806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.998+0000 7f998e3a5640 1 -- 172.21.15.19:0/3281737398 shutdown_connections 2024-04-25T09:56:51.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.998+0000 7f998e3a5640 1 -- 172.21.15.19:0/3281737398 wait complete. 2024-04-25T09:56:51.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.999+0000 7f998e3a5640 1 Processor -- start 2024-04-25T09:56:51.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:50.999+0000 7f998e3a5640 1 -- start start 2024-04-25T09:56:51.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f998e3a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99880d0ab0 0x7f9988152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f998e3a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99880d1f30 0x7f9988154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f998e3a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998815dac0 0x7f9988160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f998e3a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99880b97d0 con 0x7f99880d1f30 2024-04-25T09:56:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f998e3a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998815dea0 con 0x7f99880d0ab0 2024-04-25T09:56:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f998e3a5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99880b9950 con 0x7f998815dac0 2024-04-25T09:56:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f99877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99880d1f30 0x7f9988154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f99877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99880d1f30 0x7f9988154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54154/0 (socket says 172.21.15.19:54154) 2024-04-25T09:56:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f99877fe640 1 -- 172.21.15.19:0/1425116221 learned_addr learned my addr 172.21.15.19:0/1425116221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f998c942640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998815dac0 0x7f9988160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f9987fff640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99880d0ab0 0x7f9988152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f99877fe640 1 -- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998815dac0 msgr2=0x7f9988160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f99877fe640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998815dac0 0x7f9988160660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f99877fe640 1 -- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99880d0ab0 msgr2=0x7f9988152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f99877fe640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99880d0ab0 0x7f9988152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f99877fe640 1 -- 172.21.15.19:0/1425116221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99881533f0 con 0x7f99880d1f30 2024-04-25T09:56:51.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.000+0000 7f9987fff640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99880d0ab0 0x7f9988152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:51.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.001+0000 7f99877fe640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99880d1f30 0x7f9988154d30 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f997800c4d0 tx=0x7f997800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:51.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.001+0000 7f9984ff9640 1 -- 172.21.15.19:0/1425116221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978007480 con 0x7f99880d1f30 2024-04-25T09:56:51.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.001+0000 7f9984ff9640 1 -- 172.21.15.19:0/1425116221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99780057a0 con 0x7f99880d1f30 2024-04-25T09:56:51.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.001+0000 7f998e3a5640 1 -- 172.21.15.19:0/1425116221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9988153c90 con 0x7f99880d1f30 2024-04-25T09:56:51.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.001+0000 7f9984ff9640 1 -- 172.21.15.19:0/1425116221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978007620 con 0x7f99880d1f30 2024-04-25T09:56:51.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.001+0000 7f998e3a5640 1 -- 172.21.15.19:0/1425116221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9988154250 con 0x7f99880d1f30 2024-04-25T09:56:51.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.002+0000 7f998e3a5640 1 -- 172.21.15.19:0/1425116221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9988073a00 con 0x7f99880d1f30 2024-04-25T09:56:51.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.002+0000 7f9984ff9640 1 -- 172.21.15.19:0/1425116221 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9978005b50 con 0x7f99880d1f30 2024-04-25T09:56:51.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.003+0000 7f9984ff9640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997406b4d0 0x7f997406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.003+0000 7f9984ff9640 1 -- 172.21.15.19:0/1425116221 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f9978089ef0 con 0x7f99880d1f30 2024-04-25T09:56:51.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.003+0000 7f9987fff640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997406b4d0 0x7f997406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.003+0000 7f9987fff640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997406b4d0 0x7f997406d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f997c009af0 tx=0x7f997c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.007+0000 7f9984ff9640 1 -- 172.21.15.19:0/1425116221 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9978058c00 con 0x7f99880d1f30 2024-04-25T09:56:51.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.185+0000 7f998e3a5640 1 -- 172.21.15.19:0/1425116221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f99880b4740 con 0x7f99880d1f30 2024-04-25T09:56:51.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.186+0000 7f9984ff9640 1 -- 172.21.15.19:0/1425116221 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9978058500 con 0x7f99880d1f30 2024-04-25T09:56:51.189 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:56:51.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 -- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997406b4d0 msgr2=0x7f997406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997406b4d0 0x7f997406d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f997c009af0 tx=0x7f997c008040 comp rx=0 tx=0).stop 2024-04-25T09:56:51.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 -- 172.21.15.19:0/1425116221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99880d1f30 msgr2=0x7f9988154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99880d1f30 0x7f9988154d30 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f997800c4d0 tx=0x7f997800c990 comp rx=0 tx=0).stop 2024-04-25T09:56:51.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 -- 172.21.15.19:0/1425116221 shutdown_connections 2024-04-25T09:56:51.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f997406b4d0 0x7f997406d990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f998815dac0 0x7f9988160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99880d1f30 0x7f9988154d30 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 --2- 172.21.15.19:0/1425116221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99880d0ab0 0x7f9988152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 -- 172.21.15.19:0/1425116221 >> 172.21.15.19:0/1425116221 conn(0x7f9988061c30 msgr2=0x7f99880b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:51.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 -- 172.21.15.19:0/1425116221 shutdown_connections 2024-04-25T09:56:51.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.188+0000 7f998e3a5640 1 -- 172.21.15.19:0/1425116221 wait complete. 2024-04-25T09:56:51.202 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:56:51.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.300+0000 7f5712c12640 1 Processor -- start 2024-04-25T09:56:51.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.300+0000 7f5712c12640 1 -- start start 2024-04-25T09:56:51.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.301+0000 7f5712c12640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1616f0 0x7f570c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.301+0000 7f5712c12640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f570c0ce0a0 0x7f570c0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.301+0000 7f5712c12640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570c06c8d0 con 0x7f570c1610f0 2024-04-25T09:56:51.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.301+0000 7f5712c12640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570c1614d0 con 0x7f570c0ce0a0 2024-04-25T09:56:51.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.301+0000 7f5712c12640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0c6960 con 0x7f570c1616f0 2024-04-25T09:56:51.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.301+0000 7f570bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1616f0 0x7f570c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.302+0000 7f57111af640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f570c0ce0a0 0x7f570c0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.302+0000 7f570bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1616f0 0x7f570c161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:37942/0 (socket says 172.21.15.19:37942) 2024-04-25T09:56:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.302+0000 7f570bfff640 1 -- 172.21.15.19:0/3237455372 learned_addr learned my addr 172.21.15.19:0/3237455372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.302+0000 7f570bfff640 1 -- 172.21.15.19:0/3237455372 >> v2:172.21.15.112:3300/0 conn(0x7f570c0ce0a0 msgr2=0x7f570c0ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.302+0000 7f570bfff640 1 --2- 172.21.15.19:0/3237455372 >> v2:172.21.15.112:3300/0 conn(0x7f570c0ce0a0 0x7f570c0ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.302+0000 7f570bfff640 1 -- 172.21.15.19:0/3237455372 >> v1:172.21.15.112:6789/0 conn(0x7f570c1610f0 legacy=0x7f570c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.302+0000 7f570bfff640 1 -- 172.21.15.19:0/3237455372 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c162260 con 0x7f570c1616f0 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f570bfff640 1 --2- 172.21.15.19:0/3237455372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1616f0 0x7f570c161b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5700009870 tx=0x7f570002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e1c3d2127743de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f570b7fe640 1 -- 172.21.15.19:0/3237455372 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700037070 con 0x7f570c1616f0 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f570b7fe640 1 -- 172.21.15.19:0/3237455372 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5700032440 con 0x7f570c1616f0 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f570b7fe640 1 -- 172.21.15.19:0/3237455372 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700032810 con 0x7f570c1616f0 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f5712c12640 1 -- 172.21.15.19:0/3237455372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1616f0 msgr2=0x7f570c161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f5712c12640 1 --2- 172.21.15.19:0/3237455372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1616f0 0x7f570c161b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5700009870 tx=0x7f570002a270 comp rx=0 tx=0).stop 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f5712c12640 1 -- 172.21.15.19:0/3237455372 shutdown_connections 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f5712c12640 1 --2- 172.21.15.19:0/3237455372 >> v2:172.21.15.112:3300/0 conn(0x7f570c0ce0a0 0x7f570c0ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f5712c12640 1 --2- 172.21.15.19:0/3237455372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1616f0 0x7f570c161b50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f5712c12640 1 -- 172.21.15.19:0/3237455372 >> 172.21.15.19:0/3237455372 conn(0x7f570c06f500 msgr2=0x7f570c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f5712c12640 1 -- 172.21.15.19:0/3237455372 shutdown_connections 2024-04-25T09:56:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.303+0000 7f5712c12640 1 -- 172.21.15.19:0/3237455372 wait complete. 2024-04-25T09:56:51.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.304+0000 7f5712c12640 1 Processor -- start 2024-04-25T09:56:51.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.305+0000 7f5712c12640 1 -- start start 2024-04-25T09:56:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.305+0000 7f5712c12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c0ce0a0 0x7f570c156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.305+0000 7f5712c12640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1610f0 0x7f570c0afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.305+0000 7f5712c12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f570c1616f0 0x7f570c0c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.305+0000 7f5712c12640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0cfce0 con 0x7f570c0ce0a0 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.305+0000 7f5712c12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0cfa00 con 0x7f570c1616f0 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.305+0000 7f5712c12640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0cf690 con 0x7f570c1610f0 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57109ae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c0ce0a0 0x7f570c156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57109ae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c0ce0a0 0x7f570c156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54170/0 (socket says 172.21.15.19:54170) 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57109ae640 1 -- 172.21.15.19:0/2497231218 learned_addr learned my addr 172.21.15.19:0/2497231218 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f570bfff640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1610f0 0x7f570c0afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57111af640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f570c1616f0 0x7f570c0c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57109ae640 1 -- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1610f0 msgr2=0x7f570c0afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57109ae640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1610f0 0x7f570c0afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57109ae640 1 -- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f570c1616f0 msgr2=0x7f570c0c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57109ae640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f570c1616f0 0x7f570c0c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57109ae640 1 -- 172.21.15.19:0/2497231218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c0affe0 con 0x7f570c0ce0a0 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57111af640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f570c1616f0 0x7f570c0c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57109ae640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c0ce0a0 0x7f570c156f50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f570400c790 tx=0x7f570400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57097fa640 1 -- 172.21.15.19:0/2497231218 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704016020 con 0x7f570c0ce0a0 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57097fa640 1 -- 172.21.15.19:0/2497231218 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704006d50 con 0x7f570c0ce0a0 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f5712c12640 1 -- 172.21.15.19:0/2497231218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570c0aea00 con 0x7f570c0ce0a0 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f57097fa640 1 -- 172.21.15.19:0/2497231218 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704003bb0 con 0x7f570c0ce0a0 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.306+0000 7f5712c12640 1 -- 172.21.15.19:0/2497231218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570c0aefc0 con 0x7f570c0ce0a0 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.307+0000 7f5712c12640 1 -- 172.21.15.19:0/2497231218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570c073a00 con 0x7f570c0ce0a0 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.308+0000 7f57097fa640 1 -- 172.21.15.19:0/2497231218 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f570400a850 con 0x7f570c0ce0a0 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.308+0000 7f57097fa640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56f806b4d0 0x7f56f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.308+0000 7f57097fa640 1 -- 172.21.15.19:0/2497231218 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f570408a000 con 0x7f570c0ce0a0 2024-04-25T09:56:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.308+0000 7f570bfff640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56f806b4d0 0x7f56f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.309+0000 7f570bfff640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56f806b4d0 0x7f56f806d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5700034000 tx=0x7f570002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.312+0000 7f57097fa640 1 -- 172.21.15.19:0/2497231218 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5704020020 con 0x7f570c0ce0a0 2024-04-25T09:56:51.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.507+0000 7f5712c12640 1 -- 172.21.15.19:0/2497231218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f570c0cf150 con 0x7f570c0ce0a0 2024-04-25T09:56:51.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.507+0000 7f57097fa640 1 -- 172.21.15.19:0/2497231218 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5704058d10 con 0x7f570c0ce0a0 2024-04-25T09:56:51.510 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 -- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56f806b4d0 msgr2=0x7f56f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56f806b4d0 0x7f56f806d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5700034000 tx=0x7f570002e000 comp rx=0 tx=0).stop 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 -- 172.21.15.19:0/2497231218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c0ce0a0 msgr2=0x7f570c156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c0ce0a0 0x7f570c156f50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f570400c790 tx=0x7f570400cc50 comp rx=0 tx=0).stop 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 -- 172.21.15.19:0/2497231218 shutdown_connections 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56f806b4d0 0x7f56f806d990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f570c1616f0 0x7f570c0c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f570c1610f0 0x7f570c0afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 --2- 172.21.15.19:0/2497231218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f570c0ce0a0 0x7f570c156f50 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 -- 172.21.15.19:0/2497231218 >> 172.21.15.19:0/2497231218 conn(0x7f570c06f500 msgr2=0x7f570c0b69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 -- 172.21.15.19:0/2497231218 shutdown_connections 2024-04-25T09:56:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.509+0000 7f5712c12640 1 -- 172.21.15.19:0/2497231218 wait complete. 2024-04-25T09:56:51.523 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:56:51.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:51.524 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:51.566 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:56:51.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:51.566 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:56:51.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:51.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.760+0000 7fe79bedd640 1 Processor -- start 2024-04-25T09:56:51.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.760+0000 7fe79bedd640 1 -- start start 2024-04-25T09:56:51.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.761+0000 7fe79bedd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe79415dd30 0x7fe7940b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.761+0000 7fe79bedd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe7940b9770 0x7fe7940d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.761+0000 7fe79bedd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7940d1df0 con 0x7fe7940d2e90 2024-04-25T09:56:51.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.761+0000 7fe79bedd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7940b7a50 con 0x7fe7940b9770 2024-04-25T09:56:51.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.761+0000 7fe79bedd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940d20d0 con 0x7fe79415dd30 2024-04-25T09:56:51.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.761+0000 7fe79a47a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe7940d2e90 0x7fe7940b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58088/0 (socket says 172.21.15.19:58088) 2024-04-25T09:56:51.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.761+0000 7fe79a47a640 1 -- 172.21.15.19:0/1259355472 learned_addr learned my addr 172.21.15.19:0/1259355472 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe799c79640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe79415dd30 0x7fe7940b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825171200 0 0) 0x7fe7940d1df0 con 0x7fe7940d2e90 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe799478640 1 --2- 172.21.15.19:0/1259355472 >> v2:172.21.15.112:3300/0 conn(0x7fe7940b9770 0x7fe7940d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe77c003520 con 0x7fe7940d2e90 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1208584961 0 0) 0x7fe77c003520 con 0x7fe7940d2e90 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe79415dd30 msgr2=0x7fe7940b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe79415dd30 0x7fe7940b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 >> v2:172.21.15.112:3300/0 conn(0x7fe7940b9770 msgr2=0x7fe7940d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 --2- 172.21.15.19:0/1259355472 >> v2:172.21.15.112:3300/0 conn(0x7fe7940b9770 0x7fe7940d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7940d23b0 con 0x7fe7940d2e90 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fe7840019f0 con 0x7fe7940d2e90 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 >> v1:172.21.15.19:6789/0 conn(0x7fe7940d2e90 legacy=0x7fe7940b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe79415dd30 0x7fe77c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 0x7fe77c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77c004d40 0x7fe77c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940b7a50 con 0x7fe77c003f30 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940d20d0 con 0x7fe77c004d40 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c00ba10 con 0x7fe79415dd30 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe799478640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 0x7fe77c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe799c79640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe79415dd30 0x7fe77c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe79a47a640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77c004d40 0x7fe77c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe799478640 1 -- 172.21.15.19:0/1259355472 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe79415dd30 msgr2=0x7fe77c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe799478640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe79415dd30 0x7fe77c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe799478640 1 -- 172.21.15.19:0/1259355472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77c004d40 msgr2=0x7fe77c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe799478640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77c004d40 0x7fe77c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.762+0000 7fe799478640 1 -- 172.21.15.19:0/1259355472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c003520 con 0x7fe77c003f30 2024-04-25T09:56:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.763+0000 7fe799478640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 0x7fe77c004800 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe788009ae0 tx=0x7fe77c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b938975c7fd2e16b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.763+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788036070 con 0x7fe77c003f30 2024-04-25T09:56:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.763+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe788005990 con 0x7fe77c003f30 2024-04-25T09:56:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.763+0000 7fe798c77640 1 -- 172.21.15.19:0/1259355472 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788009680 con 0x7fe77c003f30 2024-04-25T09:56:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.764+0000 7fe79bedd640 1 -- 172.21.15.19:0/1259355472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 msgr2=0x7fe77c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.764+0000 7fe79bedd640 1 --2- 172.21.15.19:0/1259355472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 0x7fe77c004800 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe788009ae0 tx=0x7fe77c005830 comp rx=0 tx=0).stop 2024-04-25T09:56:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.764+0000 7fe79a47a640 1 -- 172.21.15.19:0/1259355472 reap_dead start 2024-04-25T09:56:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.764+0000 7fe79bedd640 1 -- 172.21.15.19:0/1259355472 shutdown_connections 2024-04-25T09:56:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.764+0000 7fe79bedd640 1 -- 172.21.15.19:0/1259355472 >> 172.21.15.19:0/1259355472 conn(0x7fe794061c30 msgr2=0x7fe79406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.764+0000 7fe79bedd640 1 -- 172.21.15.19:0/1259355472 shutdown_connections 2024-04-25T09:56:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.764+0000 7fe79bedd640 1 -- 172.21.15.19:0/1259355472 wait complete. 2024-04-25T09:56:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.765+0000 7fe79bedd640 1 Processor -- start 2024-04-25T09:56:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.765+0000 7fe79bedd640 1 -- start start 2024-04-25T09:56:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe79bedd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 0x7fe79414daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe79bedd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79406c1a0 0x7fe79406c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe79bedd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7940cad20 0x7fe7940cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe79bedd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940d1a10 con 0x7fe77c003f30 2024-04-25T09:56:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe79bedd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940d2970 con 0x7fe79406c1a0 2024-04-25T09:56:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe79bedd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940d1b90 con 0x7fe7940cad20 2024-04-25T09:56:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe799c79640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 0x7fe79414daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe799c79640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 0x7fe79414daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54194/0 (socket says 172.21.15.19:54194) 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe799c79640 1 -- 172.21.15.19:0/4105606881 learned_addr learned my addr 172.21.15.19:0/4105606881 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe79a47a640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7940cad20 0x7fe7940cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe799478640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79406c1a0 0x7fe79406c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe799c79640 1 -- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7940cad20 msgr2=0x7fe7940cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe799c79640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7940cad20 0x7fe7940cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe799c79640 1 -- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79406c1a0 msgr2=0x7fe79406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe799c79640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79406c1a0 0x7fe79406c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe799c79640 1 -- 172.21.15.19:0/4105606881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7940cc220 con 0x7fe77c003f30 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.766+0000 7fe799c79640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 0x7fe79414daa0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0119d0 tx=0x7fe78c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.767+0000 7fe7927fc640 1 -- 172.21.15.19:0/4105606881 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00fb50 con 0x7fe77c003f30 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.767+0000 7fe7927fc640 1 -- 172.21.15.19:0/4105606881 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c00fcf0 con 0x7fe77c003f30 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.767+0000 7fe79bedd640 1 -- 172.21.15.19:0/4105606881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7940cc500 con 0x7fe77c003f30 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.767+0000 7fe7927fc640 1 -- 172.21.15.19:0/4105606881 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00fea0 con 0x7fe77c003f30 2024-04-25T09:56:51.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.767+0000 7fe79bedd640 1 -- 172.21.15.19:0/4105606881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7940acda0 con 0x7fe77c003f30 2024-04-25T09:56:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.768+0000 7fe79bedd640 1 -- 172.21.15.19:0/4105606881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe794073990 con 0x7fe77c003f30 2024-04-25T09:56:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.768+0000 7fe7927fc640 1 -- 172.21.15.19:0/4105606881 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe78c012040 con 0x7fe77c003f30 2024-04-25T09:56:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.769+0000 7fe7927fc640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe78006b4d0 0x7fe78006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.769+0000 7fe7927fc640 1 -- 172.21.15.19:0/4105606881 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fe78c08a9e0 con 0x7fe77c003f30 2024-04-25T09:56:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.769+0000 7fe799478640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe78006b4d0 0x7fe78006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.769+0000 7fe799478640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe78006b4d0 0x7fe78006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe788033000 tx=0x7fe788008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.773+0000 7fe7927fc640 1 -- 172.21.15.19:0/4105606881 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe78c019030 con 0x7fe77c003f30 2024-04-25T09:56:51.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.991+0000 7fe79bedd640 1 -- 172.21.15.19:0/4105606881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe79415e660 con 0x7fe77c003f30 2024-04-25T09:56:51.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.992+0000 7fe7927fc640 1 -- 172.21.15.19:0/4105606881 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fe78c0596f0 con 0x7fe77c003f30 2024-04-25T09:56:51.995 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:51.995 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3520485752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3520485752},{"type":"v1","addr":"172.21.15.19:6835","nonce":3520485752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5594,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1340729764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1340729764},{"type":"v1","addr":"172.21.15.112:6837","nonce":1340729764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/335673057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":335673057},{"type":"v1","addr":"172.21.15.112:6835","nonce":335673057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:56:38.802500+0000","modified":"2024-04-25T09:56:40.802847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6582":{"gid":6582,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/591570980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":591570980},{"type":"v1","addr":"172.21.15.19:6834","nonce":591570980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:56:51.996 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-04-25T09:56:51.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 -- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe78006b4d0 msgr2=0x7fe78006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe78006b4d0 0x7fe78006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe788033000 tx=0x7fe788008030 comp rx=0 tx=0).stop 2024-04-25T09:56:51.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 -- 172.21.15.19:0/4105606881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 msgr2=0x7fe79414daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:51.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 0x7fe79414daa0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0119d0 tx=0x7fe78c011e90 comp rx=0 tx=0).stop 2024-04-25T09:56:51.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 -- 172.21.15.19:0/4105606881 shutdown_connections 2024-04-25T09:56:51.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe78006b4d0 0x7fe78006d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7940cad20 0x7fe7940cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79406c1a0 0x7fe79406c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 --2- 172.21.15.19:0/4105606881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe77c003f30 0x7fe79414daa0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:51.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 -- 172.21.15.19:0/4105606881 >> 172.21.15.19:0/4105606881 conn(0x7fe794061c30 msgr2=0x7fe79415fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:51.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 -- 172.21.15.19:0/4105606881 shutdown_connections 2024-04-25T09:56:51.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:51.994+0000 7fe79bedd640 1 -- 172.21.15.19:0/4105606881 wait complete. 2024-04-25T09:56:52.008 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T09:56:52.138 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-04-25T09:56:52.138 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:56:52.139 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:56:52.139 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:56:52.139 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:56:52.140 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:56:52.140 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:56:52.140 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:56:52.140 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:56:52.140 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:56:52.149 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:56:52.149 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:52.150 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:56:52.150 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:52.153 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:56:52.154 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:52.154 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:56:52.154 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:52.154 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T09:56:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.231+0000 7fc214520f40 1 Processor -- start 2024-04-25T09:56:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.231+0000 7fc214520f40 1 -- start start 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc214520f40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc002c80 0x55b840376320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc214520f40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b840412a40 0x55b840411e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc214520f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840345dc0 0x55b8404123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc214520f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8402eddf0 con 0x55b840345dc0 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc214520f40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8402f1d50 con 0x55b840412a40 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc214520f40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b84038ff30 con 0x7fc1fc002c80 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc211f07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840345dc0 0x55b8404123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc211706640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc002c80 0x55b840376320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc210f05640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b840412a40 0x55b840411e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc211f07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840345dc0 0x55b8404123d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54204/0 (socket says 172.21.15.19:54204) 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc211f07640 1 -- 172.21.15.19:0/893481759 learned_addr learned my addr 172.21.15.19:0/893481759 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc211f07640 1 -- 172.21.15.19:0/893481759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc002c80 msgr2=0x55b840376320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc211f07640 1 --2- 172.21.15.19:0/893481759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc002c80 0x55b840376320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc211f07640 1 -- 172.21.15.19:0/893481759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b840412a40 msgr2=0x55b840411e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc211f07640 1 --2- 172.21.15.19:0/893481759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b840412a40 0x55b840411e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc211f07640 1 -- 172.21.15.19:0/893481759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b840370c80 con 0x55b840345dc0 2024-04-25T09:56:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.232+0000 7fc211f07640 1 --2- 172.21.15.19:0/893481759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840345dc0 0x55b8404123d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc20c002680 tx=0x7fc20c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbb85e763bbe332 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc2037fe640 1 -- 172.21.15.19:0/893481759 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c017070 con 0x55b840345dc0 2024-04-25T09:56:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc2037fe640 1 -- 172.21.15.19:0/893481759 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c00f7a0 con 0x55b840345dc0 2024-04-25T09:56:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc2037fe640 1 -- 172.21.15.19:0/893481759 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00fb70 con 0x55b840345dc0 2024-04-25T09:56:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc214520f40 1 -- 172.21.15.19:0/893481759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840345dc0 msgr2=0x55b8404123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc214520f40 1 --2- 172.21.15.19:0/893481759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840345dc0 0x55b8404123d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc20c002680 tx=0x7fc20c005d80 comp rx=0 tx=0).stop 2024-04-25T09:56:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc214520f40 1 -- 172.21.15.19:0/893481759 shutdown_connections 2024-04-25T09:56:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc214520f40 1 --2- 172.21.15.19:0/893481759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840345dc0 0x55b8404123d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc214520f40 1 --2- 172.21.15.19:0/893481759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b840412a40 0x55b840411e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc214520f40 1 --2- 172.21.15.19:0/893481759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc002c80 0x55b840376320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc214520f40 1 -- 172.21.15.19:0/893481759 >> 172.21.15.19:0/893481759 conn(0x55b8403459b0 msgr2=0x55b84038f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc214520f40 1 -- 172.21.15.19:0/893481759 shutdown_connections 2024-04-25T09:56:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.233+0000 7fc214520f40 1 -- 172.21.15.19:0/893481759 wait complete. 2024-04-25T09:56:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.234+0000 7fc214520f40 1 Processor -- start 2024-04-25T09:56:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.234+0000 7fc214520f40 1 -- start start 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.234+0000 7fc214520f40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b840345dc0 0x55b84038b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.234+0000 7fc214520f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840412a40 0x55b840402870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.234+0000 7fc214520f40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc002c80 0x55b8403479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.234+0000 7fc214520f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b840370a30 con 0x55b840412a40 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.234+0000 7fc214520f40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b840372250 con 0x55b840345dc0 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.234+0000 7fc214520f40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8404186d0 con 0x7fc1fc002c80 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc210f05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840412a40 0x55b840402870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc210f05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840412a40 0x55b840402870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54212/0 (socket says 172.21.15.19:54212) 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc210f05640 1 -- 172.21.15.19:0/3878582081 learned_addr learned my addr 172.21.15.19:0/3878582081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc211706640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b840345dc0 0x55b84038b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc211f07640 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc002c80 0x55b8403479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc210f05640 1 -- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc002c80 msgr2=0x55b8403479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc210f05640 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc002c80 0x55b8403479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc210f05640 1 -- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b840345dc0 msgr2=0x55b84038b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc210f05640 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b840345dc0 0x55b84038b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc210f05640 1 -- 172.21.15.19:0/3878582081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8404104f0 con 0x55b840412a40 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc211706640 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b840345dc0 0x55b84038b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc210f05640 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840412a40 0x55b840402870 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc2080092c0 tx=0x7fc2080019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc2027fc640 1 -- 172.21.15.19:0/3878582081 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080096d0 con 0x55b840412a40 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc2027fc640 1 -- 172.21.15.19:0/3878582081 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208062630 con 0x55b840412a40 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc214520f40 1 -- 172.21.15.19:0/3878582081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b84040ff50 con 0x55b840412a40 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc2027fc640 1 -- 172.21.15.19:0/3878582081 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208062a00 con 0x55b840412a40 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.235+0000 7fc214520f40 1 -- 172.21.15.19:0/3878582081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8404031a0 con 0x55b840412a40 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.237+0000 7fc2027fc640 1 -- 172.21.15.19:0/3878582081 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc20805e040 con 0x55b840412a40 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.237+0000 7fc2027fc640 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc1f4069eb0 0x7fc1f406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.237+0000 7fc2027fc640 1 -- 172.21.15.19:0/3878582081 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc2080ce300 con 0x55b840412a40 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.237+0000 7fc211706640 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc1f4069eb0 0x7fc1f406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.237+0000 7fc214520f40 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x55b840415480 0x55b840411c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.237+0000 7fc214520f40 1 -- 172.21.15.19:0/3878582081 --> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] -- osd_op(unknown.0.0:1 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v9 -- 0x55b8404121a0 con 0x55b840415480 2024-04-25T09:56:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.238+0000 7fc211f07640 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x55b840415480 0x55b840411c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.238+0000 7fc211706640 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc1f4069eb0 0x7fc1f406c370 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc204002fd0 tx=0x7fc204002c60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.238+0000 7fc211f07640 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x55b840415480 0x55b840411c60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc211f07640 1 -- 172.21.15.19:0/3878582081 <== osd.4 v2:172.21.15.112:6800/1718655451 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v66'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc20c019030 con 0x55b840415480 2024-04-25T09:56:52.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc214520f40 1 -- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x55b840415480 msgr2=0x55b840411c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc214520f40 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x55b840415480 0x55b840411c60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc214520f40 1 -- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc1f4069eb0 msgr2=0x7fc1f406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc214520f40 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc1f4069eb0 0x7fc1f406c370 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc204002fd0 tx=0x7fc204002c60 comp rx=0 tx=0).stop 2024-04-25T09:56:52.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc214520f40 1 -- 172.21.15.19:0/3878582081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840412a40 msgr2=0x55b840402870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc214520f40 1 --2- 172.21.15.19:0/3878582081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b840412a40 0x55b840402870 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc2080092c0 tx=0x7fc2080019c0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc211f07640 1 -- 172.21.15.19:0/3878582081 reap_dead start 2024-04-25T09:56:52.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc214520f40 1 -- 172.21.15.19:0/3878582081 shutdown_connections 2024-04-25T09:56:52.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc214520f40 1 -- 172.21.15.19:0/3878582081 >> 172.21.15.19:0/3878582081 conn(0x55b8403459b0 msgr2=0x55b840376320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:52.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc214520f40 1 -- 172.21.15.19:0/3878582081 shutdown_connections 2024-04-25T09:56:52.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.239+0000 7fc214520f40 1 -- 172.21.15.19:0/3878582081 wait complete. 2024-04-25T09:56:52.246 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T09:56:52.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T09:56:52.250 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T09:56:52.250 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T09:56:52.250 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T09:56:52.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T09:56:52.251 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T09:56:52.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T09:56:52.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.390+0000 7fd28cb34640 1 Processor -- start 2024-04-25T09:56:52.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.390+0000 7fd28cb34640 1 -- start start 2024-04-25T09:56:52.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.390+0000 7f462ae9e640 1 Processor -- start 2024-04-25T09:56:52.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.390+0000 7f24ad370640 1 Processor -- start 2024-04-25T09:56:52.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.390+0000 7f3168922640 1 Processor -- start 2024-04-25T09:56:52.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.390+0000 7f462ae9e640 1 -- start start 2024-04-25T09:56:52.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.390+0000 7f24ad370640 1 -- start start 2024-04-25T09:56:52.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.390+0000 7f3168922640 1 -- start start 2024-04-25T09:56:52.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f3168922640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f316415dc20 0x7f31640ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f24ad370640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f24a815de80 0x7f24a80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f3168922640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3164161710 0x7f31640cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f24ad370640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f24a80cefb0 0x7f24a80cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f24ad370640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cf980 0x7f24a806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f24ad370640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a80afc80 con 0x7f24a80cefb0 2024-04-25T09:56:52.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7fd28cb34640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2880ce320 0x7fd28815b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f24ad370640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a80ce4d0 con 0x7f24a815de80 2024-04-25T09:56:52.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7fd28cb34640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2880720c0 con 0x7fd288071350 2024-04-25T09:56:52.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7fd28cb34640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28806da70 con 0x7fd2880cdf40 2024-04-25T09:56:52.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7fd28cb34640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2880b2910 con 0x7fd2880ce320 2024-04-25T09:56:52.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f462ae9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4624144d50 0x7f462406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f462ae9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46241452a0 0x7f462406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f462ae9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46240cccc0 con 0x7f46241446d0 2024-04-25T09:56:52.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f462ae9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46240b2b20 con 0x7f4624144d50 2024-04-25T09:56:52.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f462ae9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462406d400 con 0x7f46241452a0 2024-04-25T09:56:52.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f24ad370640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80ce7b0 con 0x7f24a80cf980 2024-04-25T09:56:52.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f3168922640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316415e000 con 0x7f31640b9330 2024-04-25T09:56:52.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f3168922640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31640d1de0 con 0x7f3164161710 2024-04-25T09:56:52.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f3168922640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3164161c60 con 0x7f316415dc20 2024-04-25T09:56:52.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7fd285d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2880ce320 0x7fd28815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7fd285d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2880ce320 0x7fd28815b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54234/0 (socket says 172.21.15.19:54234) 2024-04-25T09:56:52.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7fd285d9b640 1 -- 172.21.15.19:0/1072884283 learned_addr learned my addr 172.21.15.19:0/1072884283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:52.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f4628c3a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f46241446d0 0x7f46240b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58100/0 (socket says 172.21.15.19:58100) 2024-04-25T09:56:52.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f4628c3a640 1 -- 172.21.15.19:0/1525643545 learned_addr learned my addr 172.21.15.19:0/1525643545 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:52.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f24a67fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f24a80cefb0 0x7f24a80cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f315bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3164161710 0x7f31640cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f4623fff640 1 --2- 172.21.15.19:0/1525643545 >> v2:172.21.15.112:3300/0 conn(0x7f4624144d50 0x7f462406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f24a67fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f24a80cefb0 0x7f24a80cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54236/0 (socket says 172.21.15.19:54236) 2024-04-25T09:56:52.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f24a67fc640 1 -- 172.21.15.19:0/1324529916 learned_addr learned my addr 172.21.15.19:0/1324529916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:52.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f24a77fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cf980 0x7f24a806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f24a6ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f24a815de80 0x7f24a80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f3162d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f31640b9330 0x7f316406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58102/0 (socket says 172.21.15.19:58102) 2024-04-25T09:56:52.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f316259c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f316415dc20 0x7f31640ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f3162d9d640 1 -- 172.21.15.19:0/1566360204 learned_addr learned my addr 172.21.15.19:0/1566360204 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:52.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.391+0000 7f462943b640 1 --2- 172.21.15.19:0/1525643545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46241452a0 0x7f462406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f24a77fe640 1 -- 172.21.15.19:0/1324529916 >> v2:172.21.15.112:3300/0 conn(0x7f24a815de80 msgr2=0x7f24a80cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:52.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f24a77fe640 1 --2- 172.21.15.19:0/1324529916 >> v2:172.21.15.112:3300/0 conn(0x7f24a815de80 0x7f24a80cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f24a77fe640 1 -- 172.21.15.19:0/1324529916 >> v2:172.21.15.19:3300/0 conn(0x7f24a80cefb0 msgr2=0x7f24a80cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:52.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f24a77fe640 1 --2- 172.21.15.19:0/1324529916 >> v2:172.21.15.19:3300/0 conn(0x7f24a80cefb0 0x7f24a80cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f24a77fe640 1 -- 172.21.15.19:0/1324529916 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a80cea90 con 0x7f24a80cf980 2024-04-25T09:56:52.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7fd285d9b640 1 -- 172.21.15.19:0/1072884283 >> v1:172.21.15.112:6789/0 conn(0x7fd2880cdf40 legacy=0x7fd28806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f315bfff640 1 -- 172.21.15.19:0/1566360204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f316415dc20 msgr2=0x7f31640ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:52.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f315bfff640 1 --2- 172.21.15.19:0/1566360204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f316415dc20 0x7f31640ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f315bfff640 1 -- 172.21.15.19:0/1566360204 >> v1:172.21.15.19:6789/0 conn(0x7f31640b9330 legacy=0x7f316406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f462943b640 1 -- 172.21.15.19:0/1525643545 >> v2:172.21.15.112:3300/0 conn(0x7f4624144d50 msgr2=0x7f462406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:52.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f462943b640 1 --2- 172.21.15.19:0/1525643545 >> v2:172.21.15.112:3300/0 conn(0x7f4624144d50 0x7f462406d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f462943b640 1 -- 172.21.15.19:0/1525643545 >> v1:172.21.15.19:6789/0 conn(0x7f46241446d0 legacy=0x7f46240b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f315bfff640 1 -- 172.21.15.19:0/1566360204 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3164161f40 con 0x7f3164161710 2024-04-25T09:56:52.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7f462943b640 1 -- 172.21.15.19:0/1525643545 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240cc140 con 0x7f46241452a0 2024-04-25T09:56:52.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7fd285d9b640 1 -- 172.21.15.19:0/1072884283 >> v1:172.21.15.19:6789/0 conn(0x7fd288071350 legacy=0x7fd2880717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.392+0000 7fd285d9b640 1 -- 172.21.15.19:0/1072884283 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28806dd50 con 0x7fd2880ce320 2024-04-25T09:56:52.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f24a77fe640 1 --2- 172.21.15.19:0/1324529916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cf980 0x7f24a806abe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24a80d1800 tx=0x7f24a002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2c33495f05867db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f462943b640 1 --2- 172.21.15.19:0/1525643545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46241452a0 0x7f462406ddd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f461c009d90 tx=0x7f461c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e775c8842df03e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7fd285d9b640 1 --2- 172.21.15.19:0/1072884283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2880ce320 0x7fd28815b740 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009870 tx=0x7fd27c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fa8793cbcf09c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f315bfff640 1 --2- 172.21.15.19:0/1566360204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3164161710 0x7f31640cebb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f314c0097a0 tx=0x7f314c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdfd52878c8aba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f24a5ffb640 1 -- 172.21.15.19:0/1324529916 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0031600 con 0x7f24a80cf980 2024-04-25T09:56:52.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f46237fe640 1 -- 172.21.15.19:0/1525643545 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c007740 con 0x7f46241452a0 2024-04-25T09:56:52.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7fd28559a640 1 -- 172.21.15.19:0/1072884283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c037070 con 0x7fd2880ce320 2024-04-25T09:56:52.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7fd28559a640 1 -- 172.21.15.19:0/1072884283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c032440 con 0x7fd2880ce320 2024-04-25T09:56:52.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7fd28559a640 1 -- 172.21.15.19:0/1072884283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c032810 con 0x7fd2880ce320 2024-04-25T09:56:52.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f3161d9b640 1 -- 172.21.15.19:0/1566360204 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c037070 con 0x7f3164161710 2024-04-25T09:56:52.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f46237fe640 1 -- 172.21.15.19:0/1525643545 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c0078e0 con 0x7f46241452a0 2024-04-25T09:56:52.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f24a5ffb640 1 -- 172.21.15.19:0/1324529916 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a00317a0 con 0x7f24a80cf980 2024-04-25T09:56:52.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f3161d9b640 1 -- 172.21.15.19:0/1566360204 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c032440 con 0x7f3164161710 2024-04-25T09:56:52.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f46237fe640 1 -- 172.21.15.19:0/1525643545 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c007cb0 con 0x7f46241452a0 2024-04-25T09:56:52.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f24a5ffb640 1 -- 172.21.15.19:0/1324529916 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0031b70 con 0x7f24a80cf980 2024-04-25T09:56:52.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.393+0000 7f3161d9b640 1 -- 172.21.15.19:0/1566360204 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c032810 con 0x7f3164161710 2024-04-25T09:56:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f462ae9e640 1 -- 172.21.15.19:0/1525643545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46241452a0 msgr2=0x7f462406ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f462ae9e640 1 --2- 172.21.15.19:0/1525643545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46241452a0 0x7f462406ddd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f461c009d90 tx=0x7f461c02a660 comp rx=0 tx=0).stop 2024-04-25T09:56:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7fd28cb34640 1 -- 172.21.15.19:0/1072884283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2880ce320 msgr2=0x7fd28815b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7fd28cb34640 1 --2- 172.21.15.19:0/1072884283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2880ce320 0x7fd28815b740 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009870 tx=0x7fd27c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f3168922640 1 -- 172.21.15.19:0/1566360204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3164161710 msgr2=0x7f31640cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f3168922640 1 --2- 172.21.15.19:0/1566360204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3164161710 0x7f31640cebb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f314c0097a0 tx=0x7f314c02a180 comp rx=0 tx=0).stop 2024-04-25T09:56:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f24ad370640 1 -- 172.21.15.19:0/1324529916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cf980 msgr2=0x7f24a806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f24ad370640 1 --2- 172.21.15.19:0/1324529916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cf980 0x7f24a806abe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24a80d1800 tx=0x7f24a002a5d0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f462ae9e640 1 -- 172.21.15.19:0/1525643545 shutdown_connections 2024-04-25T09:56:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f462ae9e640 1 --2- 172.21.15.19:0/1525643545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46241452a0 0x7f462406ddd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f462ae9e640 1 --2- 172.21.15.19:0/1525643545 >> v2:172.21.15.112:3300/0 conn(0x7f4624144d50 0x7f462406d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f462ae9e640 1 -- 172.21.15.19:0/1525643545 >> 172.21.15.19:0/1525643545 conn(0x7f462406abe0 msgr2=0x7f462406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f3168922640 1 -- 172.21.15.19:0/1566360204 shutdown_connections 2024-04-25T09:56:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f3168922640 1 --2- 172.21.15.19:0/1566360204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3164161710 0x7f31640cebb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f3168922640 1 --2- 172.21.15.19:0/1566360204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f316415dc20 0x7f31640ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f3168922640 1 -- 172.21.15.19:0/1566360204 >> 172.21.15.19:0/1566360204 conn(0x7f31640d0420 msgr2=0x7f31640d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f462ae9e640 1 -- 172.21.15.19:0/1525643545 shutdown_connections 2024-04-25T09:56:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f24ad370640 1 -- 172.21.15.19:0/1324529916 shutdown_connections 2024-04-25T09:56:52.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f462ae9e640 1 -- 172.21.15.19:0/1525643545 wait complete. 2024-04-25T09:56:52.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f24ad370640 1 --2- 172.21.15.19:0/1324529916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cf980 0x7f24a806abe0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f24ad370640 1 --2- 172.21.15.19:0/1324529916 >> v2:172.21.15.19:3300/0 conn(0x7f24a80cefb0 0x7f24a80cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f24ad370640 1 --2- 172.21.15.19:0/1324529916 >> v2:172.21.15.112:3300/0 conn(0x7f24a815de80 0x7f24a80cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f24ad370640 1 -- 172.21.15.19:0/1324529916 >> 172.21.15.19:0/1324529916 conn(0x7f24a80b2390 msgr2=0x7f24a80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:52.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7fd28cb34640 1 -- 172.21.15.19:0/1072884283 shutdown_connections 2024-04-25T09:56:52.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f3168922640 1 -- 172.21.15.19:0/1566360204 shutdown_connections 2024-04-25T09:56:52.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7fd28cb34640 1 --2- 172.21.15.19:0/1072884283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2880ce320 0x7fd28815b740 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7fd28cb34640 1 -- 172.21.15.19:0/1072884283 >> 172.21.15.19:0/1072884283 conn(0x7fd28806abe0 msgr2=0x7fd28806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:52.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f3168922640 1 -- 172.21.15.19:0/1566360204 wait complete. 2024-04-25T09:56:52.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f24ad370640 1 -- 172.21.15.19:0/1324529916 shutdown_connections 2024-04-25T09:56:52.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7f24ad370640 1 -- 172.21.15.19:0/1324529916 wait complete. 2024-04-25T09:56:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7fd28cb34640 1 -- 172.21.15.19:0/1072884283 shutdown_connections 2024-04-25T09:56:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.394+0000 7fd28cb34640 1 -- 172.21.15.19:0/1072884283 wait complete. 2024-04-25T09:56:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.395+0000 7f462ae9e640 1 Processor -- start 2024-04-25T09:56:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.395+0000 7f462ae9e640 1 -- start start 2024-04-25T09:56:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.395+0000 7f3168922640 1 Processor -- start 2024-04-25T09:56:52.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f3168922640 1 -- start start 2024-04-25T09:56:52.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7fd28cb34640 1 Processor -- start 2024-04-25T09:56:52.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f24ad370640 1 Processor -- start 2024-04-25T09:56:52.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7fd28cb34640 1 -- start start 2024-04-25T09:56:52.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f24ad370640 1 -- start start 2024-04-25T09:56:52.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f462ae9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46241446d0 0x7f46240ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f3168922640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31640b9330 0x7f3164155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f3168922640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f316415dc20 0x7f316415f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f462ae9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4624144d50 0x7f4624155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f462ae9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241452a0 0x7f462413ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f462ae9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46240cc9e0 con 0x7f46241452a0 2024-04-25T09:56:52.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f462ae9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46240cc700 con 0x7f46241446d0 2024-04-25T09:56:52.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f462ae9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4624144ab0 con 0x7f4624144d50 2024-04-25T09:56:52.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f3168922640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3164161710 0x7f316414abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f3168922640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31640b5e50 con 0x7f3164161710 2024-04-25T09:56:52.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f3168922640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3164162500 con 0x7f31640b9330 2024-04-25T09:56:52.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f3168922640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31640b5fd0 con 0x7f316415dc20 2024-04-25T09:56:52.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.396+0000 7f462943b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241452a0 0x7f462413ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f462943b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241452a0 0x7f462413ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54238/0 (socket says 172.21.15.19:54238) 2024-04-25T09:56:52.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f462943b640 1 -- 172.21.15.19:0/4001378434 learned_addr learned my addr 172.21.15.19:0/4001378434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:52.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f24ad370640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cefb0 0x7f24a80c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f24ad370640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a80cf980 0x7f24a8151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f4628c3a640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46241446d0 0x7f46240ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f4623fff640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4624144d50 0x7f4624155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f24ad370640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a815de80 0x7f24a80c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f24ad370640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80b3030 con 0x7f24a80cf980 2024-04-25T09:56:52.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f3162d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3164161710 0x7f316414abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f3162d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3164161710 0x7f316414abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54250/0 (socket says 172.21.15.19:54250) 2024-04-25T09:56:52.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f3162d9d640 1 -- 172.21.15.19:0/3997095543 learned_addr learned my addr 172.21.15.19:0/3997095543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:52.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f316259c640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31640b9330 0x7f3164155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f315bfff640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f316415dc20 0x7f316415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f24ad370640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80b2eb0 con 0x7f24a815de80 2024-04-25T09:56:52.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7fd28cb34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd288071350 0x7fd288153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7fd28cb34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2880cdf40 0x7fd288156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7fd28cb34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2880ce320 0x7fd288153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7fd28cb34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28815bc80 con 0x7fd2880cdf40 2024-04-25T09:56:52.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7fd28cb34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28806d820 con 0x7fd2880ce320 2024-04-25T09:56:52.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7fd28cb34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28815be00 con 0x7fd288071350 2024-04-25T09:56:52.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f24a6ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cefb0 0x7f24a80c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f24ad370640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80c5600 con 0x7f24a80cefb0 2024-04-25T09:56:52.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7fd28659c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd288071350 0x7fd288153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7fd286d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2880ce320 0x7fd288153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7fd286d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2880ce320 0x7fd288153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:49170/0 (socket says 172.21.15.19:49170) 2024-04-25T09:56:52.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7fd286d9d640 1 -- 172.21.15.19:0/3390011005 learned_addr learned my addr 172.21.15.19:0/3390011005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:52.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f24a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a815de80 0x7f24a80c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f24a6ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cefb0 0x7f24a80c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:38028/0 (socket says 172.21.15.19:38028) 2024-04-25T09:56:52.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f24a6ffd640 1 -- 172.21.15.19:0/3776030526 learned_addr learned my addr 172.21.15.19:0/3776030526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:52.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f4628c3a640 1 -- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4624144d50 msgr2=0x7f4624155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f4628c3a640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4624144d50 0x7f4624155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f4628c3a640 1 -- 172.21.15.19:0/4001378434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241452a0 msgr2=0x7f462413ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f4628c3a640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241452a0 0x7f462413ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f4628c3a640 1 -- 172.21.15.19:0/4001378434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462413f2e0 con 0x7f46241446d0 2024-04-25T09:56:52.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f462943b640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241452a0 0x7f462413ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:52.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f4623fff640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4624144d50 0x7f4624155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:52.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7fd285d9b640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2880cdf40 0x7fd288156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f3162d9d640 1 -- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f316415dc20 msgr2=0x7f316415f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f3162d9d640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f316415dc20 0x7f316415f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f3162d9d640 1 -- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31640b9330 msgr2=0x7f3164155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f3162d9d640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31640b9330 0x7f3164155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f3162d9d640 1 -- 172.21.15.19:0/3997095543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3164160380 con 0x7f3164161710 2024-04-25T09:56:52.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f316259c640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31640b9330 0x7f3164155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:52.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.397+0000 7f24a67fc640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a80cf980 0x7f24a8151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f4628c3a640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46241446d0 0x7f46240ceef0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f461800c440 tx=0x7f461800c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7fd286d9d640 1 -- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd288071350 msgr2=0x7fd288153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7fd286d9d640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd288071350 0x7fd288153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7fd286d9d640 1 -- 172.21.15.19:0/3390011005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2880cdf40 msgr2=0x7fd288156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7fd286d9d640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2880cdf40 0x7fd288156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7fd286d9d640 1 -- 172.21.15.19:0/3390011005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd288159d40 con 0x7fd2880ce320 2024-04-25T09:56:52.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f3162d9d640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3164161710 0x7f316414abf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f315c00caa0 tx=0x7f315c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f24a77fe640 1 -- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cefb0 msgr2=0x7f24a80c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f24a77fe640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cefb0 0x7f24a80c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f24a77fe640 1 -- 172.21.15.19:0/3776030526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a80cf980 msgr2=0x7f24a8151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:52.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f24a77fe640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a80cf980 0x7f24a8151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:52.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f24a77fe640 1 -- 172.21.15.19:0/3776030526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a815c790 con 0x7f24a815de80 2024-04-25T09:56:52.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f24a6ffd640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cefb0 0x7f24a80c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:52.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f46217fa640 1 -- 172.21.15.19:0/4001378434 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618017070 con 0x7f46241446d0 2024-04-25T09:56:52.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f46217fa640 1 -- 172.21.15.19:0/4001378434 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4618005ce0 con 0x7f46241446d0 2024-04-25T09:56:52.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7fd286d9d640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2880ce320 0x7fd288153290 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd28000c7a0 tx=0x7fd28000cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f462ae9e640 1 -- 172.21.15.19:0/4001378434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462413f5c0 con 0x7f46241446d0 2024-04-25T09:56:52.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f24a77fe640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a815de80 0x7f24a80c8490 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f24a0031000 tx=0x7f24a0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f462ae9e640 1 -- 172.21.15.19:0/4001378434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462413fb80 con 0x7f46241446d0 2024-04-25T09:56:52.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f315affd640 1 -- 172.21.15.19:0/3997095543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c017070 con 0x7f3164161710 2024-04-25T09:56:52.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f315affd640 1 -- 172.21.15.19:0/3997095543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c00aae0 con 0x7f3164161710 2024-04-25T09:56:52.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f315affd640 1 -- 172.21.15.19:0/3997095543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c012400 con 0x7f3164161710 2024-04-25T09:56:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f46217fa640 1 -- 172.21.15.19:0/4001378434 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180072d0 con 0x7f46241446d0 2024-04-25T09:56:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f3168922640 1 -- 172.21.15.19:0/3997095543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3164160660 con 0x7f3164161710 2024-04-25T09:56:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7fd276ffd640 1 -- 172.21.15.19:0/3390011005 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280003b70 con 0x7fd2880ce320 2024-04-25T09:56:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7fd276ffd640 1 -- 172.21.15.19:0/3390011005 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280003d30 con 0x7fd2880ce320 2024-04-25T09:56:52.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.398+0000 7f3168922640 1 -- 172.21.15.19:0/3997095543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316415ef60 con 0x7f3164161710 2024-04-25T09:56:52.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.399+0000 7fd276ffd640 1 -- 172.21.15.19:0/3390011005 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28000aa90 con 0x7fd2880ce320 2024-04-25T09:56:52.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.399+0000 7fd28cb34640 1 -- 172.21.15.19:0/3390011005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2881f80b0 con 0x7fd2880ce320 2024-04-25T09:56:52.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.399+0000 7fd28cb34640 1 -- 172.21.15.19:0/3390011005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2881f7810 con 0x7fd2880ce320 2024-04-25T09:56:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.399+0000 7f248f7fe640 1 -- 172.21.15.19:0/3776030526 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0040070 con 0x7f24a815de80 2024-04-25T09:56:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.399+0000 7f248f7fe640 1 -- 172.21.15.19:0/3776030526 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a0030c00 con 0x7f24a815de80 2024-04-25T09:56:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.399+0000 7f248f7fe640 1 -- 172.21.15.19:0/3776030526 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0009450 con 0x7f24a815de80 2024-04-25T09:56:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.399+0000 7f24ad370640 1 -- 172.21.15.19:0/3776030526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a815ca70 con 0x7f24a815de80 2024-04-25T09:56:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.400+0000 7f24ad370640 1 -- 172.21.15.19:0/3776030526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a815d030 con 0x7f24a815de80 2024-04-25T09:56:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.400+0000 7f462ae9e640 1 -- 172.21.15.19:0/4001378434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4604005290 con 0x7f46241446d0 2024-04-25T09:56:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.400+0000 7f3168922640 1 -- 172.21.15.19:0/3997095543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3164073a00 con 0x7f3164161710 2024-04-25T09:56:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.400+0000 7fd28cb34640 1 -- 172.21.15.19:0/3390011005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd288073a00 con 0x7fd2880ce320 2024-04-25T09:56:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.400+0000 7f46217fa640 1 -- 172.21.15.19:0/4001378434 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f461801a070 con 0x7f46241446d0 2024-04-25T09:56:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.401+0000 7f24ad370640 1 -- 172.21.15.19:0/3776030526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a8073a00 con 0x7f24a815de80 2024-04-25T09:56:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.401+0000 7f46217fa640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f460806b520 0x7f460806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.401+0000 7f248f7fe640 1 -- 172.21.15.19:0/3776030526 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24a0039040 con 0x7f24a815de80 2024-04-25T09:56:52.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.401+0000 7fd276ffd640 1 -- 172.21.15.19:0/3390011005 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd28000ac30 con 0x7fd2880ce320 2024-04-25T09:56:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.401+0000 7f46217fa640 1 -- 172.21.15.19:0/4001378434 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f4618089580 con 0x7f46241446d0 2024-04-25T09:56:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.401+0000 7f4623fff640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f460806b520 0x7f460806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.401+0000 7f315affd640 1 -- 172.21.15.19:0/3997095543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f315c01a070 con 0x7f3164161710 2024-04-25T09:56:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.402+0000 7fd276ffd640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd26806b520 0x7fd26806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.402+0000 7f4623fff640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f460806b520 0x7f460806d9e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4624140f60 tx=0x7f4610008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.402+0000 7f248f7fe640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f249406b520 0x7f249406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.402+0000 7f315affd640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f314406b5a0 0x7f314406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.402+0000 7f248f7fe640 1 -- 172.21.15.19:0/3776030526 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f24a00ab4f0 con 0x7f24a815de80 2024-04-25T09:56:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.402+0000 7f24a6ffd640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f249406b520 0x7f249406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.402+0000 7fd276ffd640 1 -- 172.21.15.19:0/3390011005 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fd28008ac50 con 0x7fd2880ce320 2024-04-25T09:56:52.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.402+0000 7fd28659c640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd26806b520 0x7fd26806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.402+0000 7f315affd640 1 -- 172.21.15.19:0/3997095543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f315c089ff0 con 0x7f3164161710 2024-04-25T09:56:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.402+0000 7f316259c640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f314406b5a0 0x7f314406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.402+0000 7f24a6ffd640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f249406b520 0x7f249406d9e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2498007e50 tx=0x7f2498008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.403+0000 7fd28659c640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd26806b520 0x7fd26806d9e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd278000f60 tx=0x7fd278009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.403+0000 7f316259c640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f314406b5a0 0x7f314406da60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f31540097f0 tx=0x7f3154016040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.406+0000 7fd276ffd640 1 -- 172.21.15.19:0/3390011005 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd280039030 con 0x7fd2880ce320 2024-04-25T09:56:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.406+0000 7f315affd640 1 -- 172.21.15.19:0/3997095543 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f315c058600 con 0x7f3164161710 2024-04-25T09:56:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.410+0000 7f46217fa640 1 -- 172.21.15.19:0/4001378434 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4618057c20 con 0x7f46241446d0 2024-04-25T09:56:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.415+0000 7f248f7fe640 1 -- 172.21.15.19:0/3776030526 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f24a007a250 con 0x7f24a815de80 2024-04-25T09:56:52.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.660+0000 7fd28cb34640 1 -- 172.21.15.19:0/3390011005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fd28806d3d0 con 0x7fd2880ce320 2024-04-25T09:56:52.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.667+0000 7f24ad370640 1 -- 172.21.15.19:0/3776030526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f24a806fbf0 con 0x7f24a815de80 2024-04-25T09:56:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.721+0000 7f3168922640 1 -- 172.21.15.19:0/3997095543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f31640d16b0 con 0x7f3164161710 2024-04-25T09:56:52.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:52.723+0000 7f462ae9e640 1 -- 172.21.15.19:0/4001378434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f4604005580 con 0x7f46241446d0 2024-04-25T09:56:53.515 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T09:56:53.511+0000 7fc4eb66a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:56:53.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.512+0000 7fd276ffd640 1 -- 172.21.15.19:0/3390011005 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6097 v36) v1 ==== 95+0+0 (secure 0 0 0) 0x7fd280059260 con 0x7fd2880ce320 2024-04-25T09:56:53.515 INFO:teuthology.orchestra.run.smithi019.stderr:failed mds gid 6097 2024-04-25T09:56:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 -- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd26806b520 msgr2=0x7fd26806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd26806b520 0x7fd26806d9e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd278000f60 tx=0x7fd278009040 comp rx=0 tx=0).stop 2024-04-25T09:56:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 -- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2880ce320 msgr2=0x7fd288153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2880ce320 0x7fd288153290 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd28000c7a0 tx=0x7fd28000cc60 comp rx=0 tx=0).stop 2024-04-25T09:56:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 -- 172.21.15.19:0/3390011005 shutdown_connections 2024-04-25T09:56:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd26806b520 0x7fd26806d9e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2880ce320 0x7fd288153290 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2880cdf40 0x7fd288156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 --2- 172.21.15.19:0/3390011005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd288071350 0x7fd288153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 -- 172.21.15.19:0/3390011005 >> 172.21.15.19:0/3390011005 conn(0x7fd28806abe0 msgr2=0x7fd2880b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 -- 172.21.15.19:0/3390011005 shutdown_connections 2024-04-25T09:56:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.515+0000 7fd28cb34640 1 -- 172.21.15.19:0/3390011005 wait complete. 2024-04-25T09:56:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.520+0000 7f248f7fe640 1 -- 172.21.15.19:0/3776030526 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6582 v37) v1 ==== 95+0+0 (secure 0 0 0) 0x7f24a0079b50 con 0x7f24a815de80 2024-04-25T09:56:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:failed mds gid 6582 2024-04-25T09:56:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.521+0000 7f315affd640 1 -- 172.21.15.19:0/3997095543 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5594 v38) v1 ==== 95+0+0 (secure 0 0 0) 0x7f315c05c6a0 con 0x7f3164161710 2024-04-25T09:56:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:failed mds gid 5594 2024-04-25T09:56:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 -- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f249406b520 msgr2=0x7f249406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:53.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f249406b520 0x7f249406d9e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2498007e50 tx=0x7f2498008040 comp rx=0 tx=0).stop 2024-04-25T09:56:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 -- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a815de80 msgr2=0x7f24a80c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a815de80 0x7f24a80c8490 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f24a0031000 tx=0x7f24a0004a40 comp rx=0 tx=0).stop 2024-04-25T09:56:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 -- 172.21.15.19:0/3776030526 shutdown_connections 2024-04-25T09:56:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f249406b520 0x7f249406d9e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a815de80 0x7f24a80c8490 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a80cf980 0x7f24a8151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 --2- 172.21.15.19:0/3776030526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24a80cefb0 0x7f24a80c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 -- 172.21.15.19:0/3776030526 >> 172.21.15.19:0/3776030526 conn(0x7f24a80b2390 msgr2=0x7f24a80d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 -- 172.21.15.19:0/3776030526 shutdown_connections 2024-04-25T09:56:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.523+0000 7f24ad370640 1 -- 172.21.15.19:0/3776030526 wait complete. 2024-04-25T09:56:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.524+0000 7f3168922640 1 -- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f314406b5a0 msgr2=0x7f314406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.524+0000 7f3168922640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f314406b5a0 0x7f314406da60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f31540097f0 tx=0x7f3154016040 comp rx=0 tx=0).stop 2024-04-25T09:56:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.524+0000 7f3168922640 1 -- 172.21.15.19:0/3997095543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3164161710 msgr2=0x7f316414abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.524+0000 7f3168922640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3164161710 0x7f316414abf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f315c00caa0 tx=0x7f315c00cf60 comp rx=0 tx=0).stop 2024-04-25T09:56:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.524+0000 7f3168922640 1 -- 172.21.15.19:0/3997095543 shutdown_connections 2024-04-25T09:56:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.524+0000 7f3168922640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f314406b5a0 0x7f314406da60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.524+0000 7f3168922640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3164161710 0x7f316414abf0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.524+0000 7f3168922640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f316415dc20 0x7f316415f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.524+0000 7f3168922640 1 --2- 172.21.15.19:0/3997095543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31640b9330 0x7f3164155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.524+0000 7f3168922640 1 -- 172.21.15.19:0/3997095543 >> 172.21.15.19:0/3997095543 conn(0x7f31640d0420 msgr2=0x7f31640b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.524+0000 7f3168922640 1 -- 172.21.15.19:0/3997095543 shutdown_connections 2024-04-25T09:56:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:53.525+0000 7f3168922640 1 -- 172.21.15.19:0/3997095543 wait complete. 2024-04-25T09:56:53.535 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:56:53.535 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:56:53.538 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:56:53.538 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:53.542 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:56:53.542 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:56:53.543 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:56:53.543 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:56:53.545 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:56:53.545 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:53.584 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:56:53.584 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:53.616 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T09:56:53.620 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T09:56:53.662 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T09:56:54.517 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T09:56:54.514+0000 7fc4eb66a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:56:54.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.524+0000 7f46217fa640 1 -- 172.21.15.19:0/4001378434 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 5588 v39) v1 ==== 95+0+0 (secure 0 0 0) 0x7f461805bcc0 con 0x7f46241446d0 2024-04-25T09:56:54.528 INFO:teuthology.orchestra.run.smithi019.stderr:failed mds gid 5588 2024-04-25T09:56:54.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 -- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f460806b520 msgr2=0x7f460806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f460806b520 0x7f460806d9e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4624140f60 tx=0x7f4610008040 comp rx=0 tx=0).stop 2024-04-25T09:56:54.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 -- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46241446d0 msgr2=0x7f46240ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46241446d0 0x7f46240ceef0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f461800c440 tx=0x7f461800c900 comp rx=0 tx=0).stop 2024-04-25T09:56:54.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 -- 172.21.15.19:0/4001378434 shutdown_connections 2024-04-25T09:56:54.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f460806b520 0x7f460806d9e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46241452a0 0x7f462413ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4624144d50 0x7f4624155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 --2- 172.21.15.19:0/4001378434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46241446d0 0x7f46240ceef0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 -- 172.21.15.19:0/4001378434 >> 172.21.15.19:0/4001378434 conn(0x7f462406abe0 msgr2=0x7f46240af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 -- 172.21.15.19:0/4001378434 shutdown_connections 2024-04-25T09:56:54.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.527+0000 7f462ae9e640 1 -- 172.21.15.19:0/4001378434 wait complete. 2024-04-25T09:56:54.548 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:56:54.548 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:56:54.590 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:56:54.591 DEBUG:teuthology.parallel:result is None 2024-04-25T09:56:54.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:54.670 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T09:56:54.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.696+0000 7fae77a1f640 1 Processor -- start 2024-04-25T09:56:54.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.696+0000 7fae77a1f640 1 -- start start 2024-04-25T09:56:54.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae77a1f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fae7015db60 0x7fae700ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae77a1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fae700b5f00 0x7fae700d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae77a1f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae700cec00 con 0x7fae7015df40 2024-04-25T09:56:54.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae77a1f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae700ce930 con 0x7fae7015db60 2024-04-25T09:56:54.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae77a1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700b6450 con 0x7fae700b5f00 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae757bb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fae7015db60 0x7fae700ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae74fba640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fae700b5f00 0x7fae700d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae757bb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fae7015db60 0x7fae700ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:49200/0 (socket says 172.21.15.19:49200) 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae757bb640 1 -- 172.21.15.19:0/80848590 learned_addr learned my addr 172.21.15.19:0/80848590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae75fbc640 1 --1- 172.21.15.19:0/80848590 >> v1:172.21.15.112:6789/0 conn(0x7fae7015df40 0x7fae70161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:60274/0 (socket says 172.21.15.19:60274) 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae757bb640 1 -- 172.21.15.19:0/80848590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fae700b5f00 msgr2=0x7fae700d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae757bb640 1 --2- 172.21.15.19:0/80848590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fae700b5f00 0x7fae700d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae757bb640 1 -- 172.21.15.19:0/80848590 >> v1:172.21.15.112:6789/0 conn(0x7fae7015df40 legacy=0x7fae70161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae757bb640 1 -- 172.21.15.19:0/80848590 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700ceee0 con 0x7fae7015db60 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.697+0000 7fae67fff640 1 -- 172.21.15.19:0/80848590 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362024708 0 0) 0x7fae700cec00 con 0x7fae7015df40 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.698+0000 7fae757bb640 1 --2- 172.21.15.19:0/80848590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae7015db60 0x7fae700ce3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae600098a0 tx=0x7fae6002a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29da0b6eace1c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.698+0000 7fae67fff640 1 -- 172.21.15.19:0/80848590 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60037070 con 0x7fae7015db60 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.698+0000 7fae67fff640 1 -- 172.21.15.19:0/80848590 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae60032440 con 0x7fae7015db60 2024-04-25T09:56:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.698+0000 7fae67fff640 1 -- 172.21.15.19:0/80848590 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60032810 con 0x7fae7015db60 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.699+0000 7fae77a1f640 1 -- 172.21.15.19:0/80848590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae7015db60 msgr2=0x7fae700ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.699+0000 7fae77a1f640 1 --2- 172.21.15.19:0/80848590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae7015db60 0x7fae700ce3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae600098a0 tx=0x7fae6002a200 comp rx=0 tx=0).stop 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.699+0000 7fae77a1f640 1 -- 172.21.15.19:0/80848590 shutdown_connections 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.699+0000 7fae77a1f640 1 --2- 172.21.15.19:0/80848590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fae700b5f00 0x7fae700d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.699+0000 7fae77a1f640 1 --2- 172.21.15.19:0/80848590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae7015db60 0x7fae700ce3f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.699+0000 7fae77a1f640 1 -- 172.21.15.19:0/80848590 >> 172.21.15.19:0/80848590 conn(0x7fae70061600 msgr2=0x7fae70061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.699+0000 7fae77a1f640 1 -- 172.21.15.19:0/80848590 shutdown_connections 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.699+0000 7fae77a1f640 1 -- 172.21.15.19:0/80848590 wait complete. 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.700+0000 7fae77a1f640 1 Processor -- start 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.700+0000 7fae77a1f640 1 -- start start 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.701+0000 7fae77a1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae700b5f00 0x7fae700c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.701+0000 7fae77a1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae7015db60 0x7fae700b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.701+0000 7fae77a1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fae7015df40 0x7fae70151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.701+0000 7fae77a1f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700d3280 con 0x7fae700b5f00 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.701+0000 7fae77a1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700d1d50 con 0x7fae7015db60 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.701+0000 7fae77a1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700d3400 con 0x7fae7015df40 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.701+0000 7fae757bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae700b5f00 0x7fae700c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.701+0000 7fae757bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae700b5f00 0x7fae700c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54310/0 (socket says 172.21.15.19:54310) 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.701+0000 7fae757bb640 1 -- 172.21.15.19:0/2304453865 learned_addr learned my addr 172.21.15.19:0/2304453865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.701+0000 7fae74fba640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae7015db60 0x7fae700b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.701+0000 7fae75fbc640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fae7015df40 0x7fae70151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae757bb640 1 -- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fae7015df40 msgr2=0x7fae70151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae757bb640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fae7015df40 0x7fae70151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae757bb640 1 -- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae7015db60 msgr2=0x7fae700b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae757bb640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae7015db60 0x7fae700b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae757bb640 1 -- 172.21.15.19:0/2304453865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700c84c0 con 0x7fae700b5f00 2024-04-25T09:56:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae75fbc640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fae7015df40 0x7fae70151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae757bb640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae700b5f00 0x7fae700c7f80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fae600098a0 tx=0x7fae600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae65ffb640 1 -- 172.21.15.19:0/2304453865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60037040 con 0x7fae700b5f00 2024-04-25T09:56:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae65ffb640 1 -- 172.21.15.19:0/2304453865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae60030440 con 0x7fae700b5f00 2024-04-25T09:56:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae77a1f640 1 -- 172.21.15.19:0/2304453865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae700cafd0 con 0x7fae700b5f00 2024-04-25T09:56:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae65ffb640 1 -- 172.21.15.19:0/2304453865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60030830 con 0x7fae700b5f00 2024-04-25T09:56:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.702+0000 7fae77a1f640 1 -- 172.21.15.19:0/2304453865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae700cb870 con 0x7fae700b5f00 2024-04-25T09:56:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.703+0000 7fae77a1f640 1 -- 172.21.15.19:0/2304453865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae700cb3c0 con 0x7fae700b5f00 2024-04-25T09:56:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.704+0000 7fae65ffb640 1 -- 172.21.15.19:0/2304453865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae60044030 con 0x7fae700b5f00 2024-04-25T09:56:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.704+0000 7fae65ffb640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fae5406b500 0x7fae5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.704+0000 7fae65ffb640 1 -- 172.21.15.19:0/2304453865 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fae600aa3f0 con 0x7fae700b5f00 2024-04-25T09:56:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.705+0000 7fae74fba640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fae5406b500 0x7fae5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.705+0000 7fae74fba640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fae5406b500 0x7fae5406d9c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fae700b9b60 tx=0x7fae6c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.709+0000 7fae65ffb640 1 -- 172.21.15.19:0/2304453865 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae6004b060 con 0x7fae700b5f00 2024-04-25T09:56:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.932+0000 7fae77a1f640 1 -- 172.21.15.19:0/2304453865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae700cc260 con 0x7fae700b5f00 2024-04-25T09:56:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.932+0000 7fae65ffb640 1 -- 172.21.15.19:0/2304453865 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4155 (secure 0 0 0) 0x7fae60078ea0 con 0x7fae700b5f00 2024-04-25T09:56:54.936 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:54.936 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2433112551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2433112551},{"type":"v1","addr":"172.21.15.112:6837","nonce":2433112551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6295,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2458476655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2458476655},{"type":"v1","addr":"172.21.15.112:6835","nonce":2458476655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:56:38.802500+0000","modified":"2024-04-25T09:56:54.522215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":40,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6833/2482429280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2482429280},{"type":"v1","addr":"172.21.15.19:6833","nonce":2482429280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:56:54.936 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 40 2024-04-25T09:56:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.934+0000 7fae77a1f640 1 -- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fae5406b500 msgr2=0x7fae5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.934+0000 7fae77a1f640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fae5406b500 0x7fae5406d9c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fae700b9b60 tx=0x7fae6c009190 comp rx=0 tx=0).stop 2024-04-25T09:56:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.934+0000 7fae77a1f640 1 -- 172.21.15.19:0/2304453865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae700b5f00 msgr2=0x7fae700c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.934+0000 7fae77a1f640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae700b5f00 0x7fae700c7f80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fae600098a0 tx=0x7fae600046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.935+0000 7fae77a1f640 1 -- 172.21.15.19:0/2304453865 shutdown_connections 2024-04-25T09:56:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.935+0000 7fae77a1f640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fae5406b500 0x7fae5406d9c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.935+0000 7fae77a1f640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fae7015df40 0x7fae70151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.935+0000 7fae77a1f640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae7015db60 0x7fae700b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.935+0000 7fae77a1f640 1 --2- 172.21.15.19:0/2304453865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae700b5f00 0x7fae700c7f80 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.935+0000 7fae77a1f640 1 -- 172.21.15.19:0/2304453865 >> 172.21.15.19:0/2304453865 conn(0x7fae70061600 msgr2=0x7fae7015d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.935+0000 7fae77a1f640 1 -- 172.21.15.19:0/2304453865 shutdown_connections 2024-04-25T09:56:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:54.935+0000 7fae77a1f640 1 -- 172.21.15.19:0/2304453865 wait complete. 2024-04-25T09:56:55.352 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:56:38.802500+0000', 'modified': '2024-04-25T09:56:54.522215+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6858}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6858': {'gid': 6858, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6833/2482429280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2482429280}, {'type': 'v1', 'addr': '172.21.15.19:6833', 'nonce': 2482429280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:56:55.352 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6858:up:replay 2024-04-25T09:56:56.353 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.464+0000 7f56ce789640 1 Processor -- start 2024-04-25T09:56:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.464+0000 7f56ce789640 1 -- start start 2024-04-25T09:56:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.465+0000 7f56ce789640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f56c815dd30 0x7f56c80b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.465+0000 7f56ce789640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56c80d0ad0 0x7f56c80b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.465+0000 7f56ce789640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c80d1df0 con 0x7f56c80b9770 2024-04-25T09:56:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.465+0000 7f56ce789640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c80d20d0 con 0x7f56c815dd30 2024-04-25T09:56:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.465+0000 7f56ce789640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80d23b0 con 0x7f56c80d0ad0 2024-04-25T09:56:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.465+0000 7f56c7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f56c815dd30 0x7f56c80b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.465+0000 7f56ccd26640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56c80d0ad0 0x7f56c80b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.465+0000 7f56c7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f56c815dd30 0x7f56c80b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:49218/0 (socket says 172.21.15.19:49218) 2024-04-25T09:56:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.465+0000 7f56c7fff640 1 -- 172.21.15.19:0/932737757 learned_addr learned my addr 172.21.15.19:0/932737757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.465+0000 7f56c77fe640 1 --1- 172.21.15.19:0/932737757 >> v1:172.21.15.112:6789/0 conn(0x7f56c80b9770 0x7f56c80d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:60282/0 (socket says 172.21.15.19:60282) 2024-04-25T09:56:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.466+0000 7f56ccd26640 1 -- 172.21.15.19:0/932737757 >> v2:172.21.15.112:3300/0 conn(0x7f56c815dd30 msgr2=0x7f56c80b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.466+0000 7f56ccd26640 1 --2- 172.21.15.19:0/932737757 >> v2:172.21.15.112:3300/0 conn(0x7f56c815dd30 0x7f56c80b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.466+0000 7f56ccd26640 1 -- 172.21.15.19:0/932737757 >> v1:172.21.15.112:6789/0 conn(0x7f56c80b9770 legacy=0x7f56c80d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.466+0000 7f56ccd26640 1 -- 172.21.15.19:0/932737757 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c80d2690 con 0x7f56c80d0ad0 2024-04-25T09:56:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.466+0000 7f56c6ffd640 1 -- 172.21.15.19:0/932737757 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342848213 0 0) 0x7f56c80d1df0 con 0x7f56c80b9770 2024-04-25T09:56:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.466+0000 7f56c7fff640 1 --2- 172.21.15.19:0/932737757 >> v2:172.21.15.112:3300/0 conn(0x7f56c815dd30 0x7f56c80b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56ccd26640 1 --2- 172.21.15.19:0/932737757 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56c80d0ad0 0x7f56c80b6a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009d90 tx=0x7f56c002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab33383eb2bda62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56c6ffd640 1 -- 172.21.15.19:0/932737757 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0031600 con 0x7f56c80d0ad0 2024-04-25T09:56:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56c6ffd640 1 -- 172.21.15.19:0/932737757 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c00317a0 con 0x7f56c80d0ad0 2024-04-25T09:56:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56c6ffd640 1 -- 172.21.15.19:0/932737757 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0031b70 con 0x7f56c80d0ad0 2024-04-25T09:56:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56ce789640 1 -- 172.21.15.19:0/932737757 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56c80d0ad0 msgr2=0x7f56c80b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56ce789640 1 --2- 172.21.15.19:0/932737757 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56c80d0ad0 0x7f56c80b6a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009d90 tx=0x7f56c002a5d0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56ce789640 1 -- 172.21.15.19:0/932737757 shutdown_connections 2024-04-25T09:56:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56ce789640 1 --2- 172.21.15.19:0/932737757 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56c80d0ad0 0x7f56c80b6a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56ce789640 1 --2- 172.21.15.19:0/932737757 >> v2:172.21.15.112:3300/0 conn(0x7f56c815dd30 0x7f56c80b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56ce789640 1 -- 172.21.15.19:0/932737757 >> 172.21.15.19:0/932737757 conn(0x7f56c8061c30 msgr2=0x7f56c806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56ce789640 1 -- 172.21.15.19:0/932737757 shutdown_connections 2024-04-25T09:56:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.467+0000 7f56ce789640 1 -- 172.21.15.19:0/932737757 wait complete. 2024-04-25T09:56:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.468+0000 7f56ce789640 1 Processor -- start 2024-04-25T09:56:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.469+0000 7f56ce789640 1 -- start start 2024-04-25T09:56:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.469+0000 7f56ce789640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c80b9770 0x7f56c815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.469+0000 7f56ce789640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c80d0ad0 0x7f56c80ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.469+0000 7f56ce789640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56c815dd30 0x7f56c80c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.469+0000 7f56ce789640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80d1b90 con 0x7f56c80d0ad0 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.469+0000 7f56ce789640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80d1a10 con 0x7f56c80b9770 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.469+0000 7f56ce789640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80d2e90 con 0x7f56c815dd30 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c80d0ad0 0x7f56c80ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c80d0ad0 0x7f56c80ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54320/0 (socket says 172.21.15.19:54320) 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c77fe640 1 -- 172.21.15.19:0/1815492107 learned_addr learned my addr 172.21.15.19:0/1815492107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56ccd26640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56c815dd30 0x7f56c80c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c7fff640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c80b9770 0x7f56c815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c77fe640 1 -- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56c815dd30 msgr2=0x7f56c80c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c77fe640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56c815dd30 0x7f56c80c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c77fe640 1 -- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c80b9770 msgr2=0x7f56c815a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c77fe640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c80b9770 0x7f56c815a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c77fe640 1 -- 172.21.15.19:0/1815492107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c80b56d0 con 0x7f56c80d0ad0 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c7fff640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c80b9770 0x7f56c815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c77fe640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c80d0ad0 0x7f56c80ce220 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f56b800c490 tx=0x7f56b800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c4ff9640 1 -- 172.21.15.19:0/1815492107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8018070 con 0x7f56c80d0ad0 2024-04-25T09:56:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56ce789640 1 -- 172.21.15.19:0/1815492107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c80b4630 con 0x7f56c80d0ad0 2024-04-25T09:56:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c4ff9640 1 -- 172.21.15.19:0/1815492107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b80074e0 con 0x7f56c80d0ad0 2024-04-25T09:56:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56c4ff9640 1 -- 172.21.15.19:0/1815492107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8004a50 con 0x7f56c80d0ad0 2024-04-25T09:56:56.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.470+0000 7f56ce789640 1 -- 172.21.15.19:0/1815492107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c80b4bf0 con 0x7f56c80d0ad0 2024-04-25T09:56:56.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.471+0000 7f56ce789640 1 -- 172.21.15.19:0/1815492107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c8073990 con 0x7f56c80d0ad0 2024-04-25T09:56:56.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.472+0000 7f56c4ff9640 1 -- 172.21.15.19:0/1815492107 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56b8004bf0 con 0x7f56c80d0ad0 2024-04-25T09:56:56.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.472+0000 7f56c4ff9640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a806b4d0 0x7f56a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.472+0000 7f56c4ff9640 1 -- 172.21.15.19:0/1815492107 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f56b8089720 con 0x7f56c80d0ad0 2024-04-25T09:56:56.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.473+0000 7f56c7fff640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a806b4d0 0x7f56a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.473+0000 7f56c7fff640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a806b4d0 0x7f56a806d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009b80 tx=0x7f56b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.477+0000 7f56c4ff9640 1 -- 172.21.15.19:0/1815492107 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56b8058330 con 0x7f56c80d0ad0 2024-04-25T09:56:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.694+0000 7f56ce789640 1 -- 172.21.15.19:0/1815492107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56c80d4510 con 0x7f56c80d0ad0 2024-04-25T09:56:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.694+0000 7f56c4ff9640 1 -- 172.21.15.19:0/1815492107 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f56b8057c30 con 0x7f56c80d0ad0 2024-04-25T09:56:56.697 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:56.697 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2433112551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2433112551},{"type":"v1","addr":"172.21.15.112:6837","nonce":2433112551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6295,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2458476655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2458476655},{"type":"v1","addr":"172.21.15.112:6835","nonce":2458476655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6867,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1321165720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1321165720},{"type":"v1","addr":"172.21.15.19:6835","nonce":1321165720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:56:38.802500+0000","modified":"2024-04-25T09:56:55.535033+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":40,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.19:6833/2482429280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2482429280},{"type":"v1","addr":"172.21.15.19:6833","nonce":2482429280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:56:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 42 2024-04-25T09:56:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.696+0000 7f56ce789640 1 -- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a806b4d0 msgr2=0x7f56a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.696+0000 7f56ce789640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a806b4d0 0x7f56a806d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009b80 tx=0x7f56b0008040 comp rx=0 tx=0).stop 2024-04-25T09:56:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.696+0000 7f56ce789640 1 -- 172.21.15.19:0/1815492107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c80d0ad0 msgr2=0x7f56c80ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.696+0000 7f56ce789640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c80d0ad0 0x7f56c80ce220 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f56b800c490 tx=0x7f56b800c950 comp rx=0 tx=0).stop 2024-04-25T09:56:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.696+0000 7f56ce789640 1 -- 172.21.15.19:0/1815492107 shutdown_connections 2024-04-25T09:56:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.696+0000 7f56ce789640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a806b4d0 0x7f56a806d990 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.696+0000 7f56ce789640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56c815dd30 0x7f56c80c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.696+0000 7f56ce789640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56c80d0ad0 0x7f56c80ce220 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.696+0000 7f56ce789640 1 --2- 172.21.15.19:0/1815492107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c80b9770 0x7f56c815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.697+0000 7f56ce789640 1 -- 172.21.15.19:0/1815492107 >> 172.21.15.19:0/1815492107 conn(0x7f56c8061c30 msgr2=0x7f56c815cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.697+0000 7f56ce789640 1 -- 172.21.15.19:0/1815492107 shutdown_connections 2024-04-25T09:56:56.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:56.697+0000 7f56ce789640 1 -- 172.21.15.19:0/1815492107 wait complete. 2024-04-25T09:56:57.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:56:38.802500+0000', 'modified': '2024-04-25T09:56:55.535033+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6858}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6858': {'gid': 6858, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.19:6833/2482429280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2482429280}, {'type': 'v1', 'addr': '172.21.15.19:6833', 'nonce': 2482429280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:56:57.115 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6858:up:rejoin 2024-04-25T09:56:58.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.228+0000 7fc2ab28f640 1 Processor -- start 2024-04-25T09:56:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.228+0000 7fc2ab28f640 1 -- start start 2024-04-25T09:56:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.229+0000 7fc2ab28f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2a40b96d0 0x7fc2a40d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.229+0000 7fc2ab28f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc2a40d0fe0 0x7fc2a40d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.229+0000 7fc2ab28f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a415e810 con 0x7fc2a415dd20 2024-04-25T09:56:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.229+0000 7fc2ab28f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a40b9b30 con 0x7fc2a40d0fe0 2024-04-25T09:56:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.229+0000 7fc2ab28f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40d1c00 con 0x7fc2a40b96d0 2024-04-25T09:56:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.229+0000 7fc2a902b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc2a415dd20 0x7fc2a415e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40972/0 (socket says 172.21.15.19:40972) 2024-04-25T09:56:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.229+0000 7fc2a902b640 1 -- 172.21.15.19:0/679798763 learned_addr learned my addr 172.21.15.19:0/679798763 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.229+0000 7fc2a882a640 1 --2- 172.21.15.19:0/679798763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2a40b96d0 0x7fc2a40d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.229+0000 7fc2a982c640 1 --2- 172.21.15.19:0/679798763 >> v2:172.21.15.112:3300/0 conn(0x7fc2a40d0fe0 0x7fc2a40d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.229+0000 7fc29bfff640 1 -- 172.21.15.19:0/679798763 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227471023 0 0) 0x7fc2a415e810 con 0x7fc2a415dd20 2024-04-25T09:56:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.229+0000 7fc29bfff640 1 -- 172.21.15.19:0/679798763 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc290003520 con 0x7fc2a415dd20 2024-04-25T09:56:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.230+0000 7fc29bfff640 1 -- 172.21.15.19:0/679798763 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3350931256 0 0) 0x7fc290003520 con 0x7fc2a415dd20 2024-04-25T09:56:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.230+0000 7fc2a882a640 1 -- 172.21.15.19:0/679798763 >> v2:172.21.15.112:3300/0 conn(0x7fc2a40d0fe0 msgr2=0x7fc2a40d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.230+0000 7fc2a882a640 1 --2- 172.21.15.19:0/679798763 >> v2:172.21.15.112:3300/0 conn(0x7fc2a40d0fe0 0x7fc2a40d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.230+0000 7fc2a882a640 1 -- 172.21.15.19:0/679798763 >> v1:172.21.15.19:6789/0 conn(0x7fc2a415dd20 legacy=0x7fc2a415e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.230+0000 7fc2a882a640 1 -- 172.21.15.19:0/679798763 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a415eac0 con 0x7fc2a40b96d0 2024-04-25T09:56:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.231+0000 7fc2a882a640 1 --2- 172.21.15.19:0/679798763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2a40b96d0 0x7fc2a40d0aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc2940097a0 tx=0x7fc29402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a9eee1885c7b98a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.231+0000 7fc29bfff640 1 -- 172.21.15.19:0/679798763 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294037070 con 0x7fc2a40b96d0 2024-04-25T09:56:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.231+0000 7fc29bfff640 1 -- 172.21.15.19:0/679798763 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29402ab10 con 0x7fc2a40b96d0 2024-04-25T09:56:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.231+0000 7fc29bfff640 1 -- 172.21.15.19:0/679798763 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0001670 con 0x7fc2a40b96d0 2024-04-25T09:56:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.231+0000 7fc2ab28f640 1 -- 172.21.15.19:0/679798763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2a40b96d0 msgr2=0x7fc2a40d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.231+0000 7fc2ab28f640 1 --2- 172.21.15.19:0/679798763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2a40b96d0 0x7fc2a40d0aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc2940097a0 tx=0x7fc29402a080 comp rx=0 tx=0).stop 2024-04-25T09:56:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.231+0000 7fc2ab28f640 1 -- 172.21.15.19:0/679798763 shutdown_connections 2024-04-25T09:56:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.231+0000 7fc2ab28f640 1 --2- 172.21.15.19:0/679798763 >> v2:172.21.15.112:3300/0 conn(0x7fc2a40d0fe0 0x7fc2a40d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.231+0000 7fc2ab28f640 1 --2- 172.21.15.19:0/679798763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2a40b96d0 0x7fc2a40d0aa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.231+0000 7fc2ab28f640 1 -- 172.21.15.19:0/679798763 >> 172.21.15.19:0/679798763 conn(0x7fc2a4061a20 msgr2=0x7fc2a4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.231+0000 7fc2ab28f640 1 -- 172.21.15.19:0/679798763 shutdown_connections 2024-04-25T09:56:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.232+0000 7fc2ab28f640 1 -- 172.21.15.19:0/679798763 wait complete. 2024-04-25T09:56:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.233+0000 7fc2ab28f640 1 Processor -- start 2024-04-25T09:56:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.233+0000 7fc2ab28f640 1 -- start start 2024-04-25T09:56:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.233+0000 7fc2ab28f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a40b96d0 0x7fc2a40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.233+0000 7fc2ab28f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2a40d0fe0 0x7fc2a40aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.233+0000 7fc2ab28f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2a415dd20 0x7fc2a4150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.233+0000 7fc2ab28f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40d1e50 con 0x7fc2a40b96d0 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.233+0000 7fc2ab28f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a415f080 con 0x7fc2a415dd20 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.233+0000 7fc2ab28f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40d28b0 con 0x7fc2a40d0fe0 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2a902b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a40b96d0 0x7fc2a40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2a902b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a40b96d0 0x7fc2a40cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33312/0 (socket says 172.21.15.19:33312) 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2a902b640 1 -- 172.21.15.19:0/3267462621 learned_addr learned my addr 172.21.15.19:0/3267462621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2a882a640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2a40d0fe0 0x7fc2a40aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2a982c640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2a415dd20 0x7fc2a4150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2a902b640 1 -- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2a40d0fe0 msgr2=0x7fc2a40aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2a902b640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2a40d0fe0 0x7fc2a40aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2a902b640 1 -- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2a415dd20 msgr2=0x7fc2a4150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2a902b640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2a415dd20 0x7fc2a4150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2a902b640 1 -- 172.21.15.19:0/3267462621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a41505b0 con 0x7fc2a40b96d0 2024-04-25T09:56:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2a902b640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a40b96d0 0x7fc2a40cd240 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c700 tx=0x7fc2a000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc299ffb640 1 -- 172.21.15.19:0/3267462621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0017070 con 0x7fc2a40b96d0 2024-04-25T09:56:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc299ffb640 1 -- 172.21.15.19:0/3267462621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a0005ad0 con 0x7fc2a40b96d0 2024-04-25T09:56:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc2ab28f640 1 -- 172.21.15.19:0/3267462621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a40ab620 con 0x7fc2a40b96d0 2024-04-25T09:56:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.234+0000 7fc299ffb640 1 -- 172.21.15.19:0/3267462621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0007070 con 0x7fc2a40b96d0 2024-04-25T09:56:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.235+0000 7fc2ab28f640 1 -- 172.21.15.19:0/3267462621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a40abbe0 con 0x7fc2a40b96d0 2024-04-25T09:56:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.236+0000 7fc2ab28f640 1 -- 172.21.15.19:0/3267462621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a4073990 con 0x7fc2a40b96d0 2024-04-25T09:56:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.236+0000 7fc299ffb640 1 -- 172.21.15.19:0/3267462621 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2a001a070 con 0x7fc2a40b96d0 2024-04-25T09:56:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.236+0000 7fc299ffb640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc28806b4d0 0x7fc28806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.237+0000 7fc299ffb640 1 -- 172.21.15.19:0/3267462621 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fc2a00897a0 con 0x7fc2a40b96d0 2024-04-25T09:56:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.237+0000 7fc2a882a640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc28806b4d0 0x7fc28806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.237+0000 7fc2a882a640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc28806b4d0 0x7fc28806d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc294034000 tx=0x7fc29402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.241+0000 7fc299ffb640 1 -- 172.21.15.19:0/3267462621 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2a0057d30 con 0x7fc2a40b96d0 2024-04-25T09:56:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.460+0000 7fc2ab28f640 1 -- 172.21.15.19:0/3267462621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2a40d2370 con 0x7fc2a40b96d0 2024-04-25T09:56:58.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.460+0000 7fc299ffb640 1 -- 172.21.15.19:0/3267462621 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fc2a005bdd0 con 0x7fc2a40b96d0 2024-04-25T09:56:58.464 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:56:58.464 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2433112551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2433112551},{"type":"v1","addr":"172.21.15.112:6837","nonce":2433112551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6295,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2458476655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2458476655},{"type":"v1","addr":"172.21.15.112:6835","nonce":2458476655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6867,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1321165720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1321165720},{"type":"v1","addr":"172.21.15.19:6835","nonce":1321165720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:56:38.802500+0000","modified":"2024-04-25T09:56:56.539131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6833/2482429280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2482429280},{"type":"v1","addr":"172.21.15.19:6833","nonce":2482429280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:56:58.464 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 43 2024-04-25T09:56:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 -- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc28806b4d0 msgr2=0x7fc28806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc28806b4d0 0x7fc28806d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc294034000 tx=0x7fc29402e000 comp rx=0 tx=0).stop 2024-04-25T09:56:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 -- 172.21.15.19:0/3267462621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a40b96d0 msgr2=0x7fc2a40cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a40b96d0 0x7fc2a40cd240 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c700 tx=0x7fc2a000cbc0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 -- 172.21.15.19:0/3267462621 shutdown_connections 2024-04-25T09:56:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc28806b4d0 0x7fc28806d990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2a415dd20 0x7fc2a4150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2a40d0fe0 0x7fc2a40aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 --2- 172.21.15.19:0/3267462621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a40b96d0 0x7fc2a40cd240 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 -- 172.21.15.19:0/3267462621 >> 172.21.15.19:0/3267462621 conn(0x7fc2a4061a20 msgr2=0x7fc2a40ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 -- 172.21.15.19:0/3267462621 shutdown_connections 2024-04-25T09:56:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.462+0000 7fc2ab28f640 1 -- 172.21.15.19:0/3267462621 wait complete. 2024-04-25T09:56:58.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:56:38.802500+0000', 'modified': '2024-04-25T09:56:56.539131+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6858}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6858': {'gid': 6858, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6833/2482429280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2482429280}, {'type': 'v1', 'addr': '172.21.15.19:6833', 'nonce': 2482429280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:56:58.880 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:56:58.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:56:58.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.984+0000 7fe7f2df9640 1 Processor -- start 2024-04-25T09:56:58.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.984+0000 7fe7f2df9640 1 -- start start 2024-04-25T09:56:58.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.985+0000 7fe7f2df9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7ec0d0ab0 0x7fe7ec168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.985+0000 7fe7f2df9640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec0d2420 con 0x7fe7ec0d34c0 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.985+0000 7fe7f2df9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec0d2700 con 0x7fe7ec15dac0 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.985+0000 7fe7f2df9640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ec0d3030 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.985+0000 7fe7f1396640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7ec0d0ab0 0x7fe7ec168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.985+0000 7fe7ebfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe7ec0d34c0 0x7fe7ec0d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40986/0 (socket says 172.21.15.19:40986) 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.985+0000 7fe7f1396640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7ec0d0ab0 0x7fe7ec168670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33316/0 (socket says 172.21.15.19:33316) 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.985+0000 7fe7ebfff640 1 -- 172.21.15.19:0/1784918726 learned_addr learned my addr 172.21.15.19:0/1784918726 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.986+0000 7fe7f1396640 1 -- 172.21.15.19:0/1784918726 >> v1:172.21.15.112:6789/0 conn(0x7fe7ec15dac0 legacy=0x7fe7ec0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.986+0000 7fe7f1396640 1 -- 172.21.15.19:0/1784918726 >> v1:172.21.15.19:6789/0 conn(0x7fe7ec0d34c0 legacy=0x7fe7ec0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.986+0000 7fe7eb7fe640 1 -- 172.21.15.19:0/1784918726 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472967978 0 0) 0x7fe7ec0d2420 con 0x7fe7ec0d34c0 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.986+0000 7fe7f1396640 1 -- 172.21.15.19:0/1784918726 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec0d29e0 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.986+0000 7fe7f1396640 1 --2- 172.21.15.19:0/1784918726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec0d0ab0 0x7fe7ec168670 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0d31b0 tx=0x7fe7dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d8aa4bd1c88024 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.986+0000 7fe7eb7fe640 1 -- 172.21.15.19:0/1784918726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc037070 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.987+0000 7fe7eb7fe640 1 -- 172.21.15.19:0/1784918726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc032440 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.987+0000 7fe7eb7fe640 1 -- 172.21.15.19:0/1784918726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc032810 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.987+0000 7fe7f2df9640 1 -- 172.21.15.19:0/1784918726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec0d0ab0 msgr2=0x7fe7ec168670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.987+0000 7fe7f2df9640 1 --2- 172.21.15.19:0/1784918726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec0d0ab0 0x7fe7ec168670 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0d31b0 tx=0x7fe7dc02a270 comp rx=0 tx=0).stop 2024-04-25T09:56:58.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.987+0000 7fe7f2df9640 1 -- 172.21.15.19:0/1784918726 shutdown_connections 2024-04-25T09:56:58.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.987+0000 7fe7f2df9640 1 --2- 172.21.15.19:0/1784918726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec0d0ab0 0x7fe7ec168670 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.987+0000 7fe7f2df9640 1 -- 172.21.15.19:0/1784918726 >> 172.21.15.19:0/1784918726 conn(0x7fe7ec061c30 msgr2=0x7fe7ec06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:58.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.987+0000 7fe7f2df9640 1 -- 172.21.15.19:0/1784918726 shutdown_connections 2024-04-25T09:56:58.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.987+0000 7fe7f2df9640 1 -- 172.21.15.19:0/1784918726 wait complete. 2024-04-25T09:56:58.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.988+0000 7fe7f2df9640 1 Processor -- start 2024-04-25T09:56:58.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.988+0000 7fe7f2df9640 1 -- start start 2024-04-25T09:56:58.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.989+0000 7fe7f2df9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec0d0ab0 0x7fe7ec1607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.989+0000 7fe7f2df9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7ec0d34c0 0x7fe7ec152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.989+0000 7fe7f2df9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7ec15dac0 0x7fe7ec152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.989+0000 7fe7f2df9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec15dea0 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.989+0000 7fe7f2df9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0d2100 con 0x7fe7ec0d34c0 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.989+0000 7fe7f2df9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b97d0 con 0x7fe7ec15dac0 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.989+0000 7fe7f0b95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec0d0ab0 0x7fe7ec1607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.989+0000 7fe7f0b95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec0d0ab0 0x7fe7ec1607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33332/0 (socket says 172.21.15.19:33332) 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.989+0000 7fe7f0b95640 1 -- 172.21.15.19:0/3162607160 learned_addr learned my addr 172.21.15.19:0/3162607160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7f1396640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7ec15dac0 0x7fe7ec152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7ec0d34c0 0x7fe7ec152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7f0b95640 1 -- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7ec15dac0 msgr2=0x7fe7ec152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7ec15dac0 0x7fe7ec152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7f0b95640 1 -- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7ec0d34c0 msgr2=0x7fe7ec152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7ec0d34c0 0x7fe7ec152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7f0b95640 1 -- 172.21.15.19:0/3162607160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec0b5280 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7f1396640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7ec15dac0 0x7fe7ec152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7ec0d34c0 0x7fe7ec152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec0d0ab0 0x7fe7ec1607c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400ba80 tx=0x7fe7e400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7e97fa640 1 -- 172.21.15.19:0/3162607160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e400cc00 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7e97fa640 1 -- 172.21.15.19:0/3162607160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e400cda0 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7f2df9640 1 -- 172.21.15.19:0/3162607160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ec0b5560 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7e97fa640 1 -- 172.21.15.19:0/3162607160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4003bb0 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.990+0000 7fe7f2df9640 1 -- 172.21.15.19:0/3162607160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec0b6040 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.992+0000 7fe7e97fa640 1 -- 172.21.15.19:0/3162607160 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7e4014440 con 0x7fe7ec0d0ab0 2024-04-25T09:56:58.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.992+0000 7fe7e97fa640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d806b4d0 0x7fe7d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.992+0000 7fe7e97fa640 1 -- 172.21.15.19:0/3162607160 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe7e408a310 con 0x7fe7ec0d0ab0 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.993+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d806b4d0 0x7fe7d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.993+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d806b4d0 0x7fe7d806d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0cb890 tx=0x7fe7e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.993+0000 7fe7f5bdb740 1 Processor -- start 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.993+0000 7fe7f5bdb740 1 -- start start 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af5400610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af541dad0 0x556af541e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af53d5f10 0x556af541c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af5338a10 con 0x556af5400230 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af52e1b30 con 0x556af53d5f10 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x556af5334c50 con 0x556af541dad0 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f0b95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af5400610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f0b95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af5400610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33338/0 (socket says 172.21.15.19:33338) 2024-04-25T09:56:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f0b95640 1 -- 172.21.15.19:0/2201509424 learned_addr learned my addr 172.21.15.19:0/2201509424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/2201509424 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af541dad0 0x556af541e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f0b95640 1 -- 172.21.15.19:0/2201509424 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af541dad0 msgr2=0x556af541e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/2201509424 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af541dad0 0x556af541e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f0b95640 1 -- 172.21.15.19:0/2201509424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af53d5f10 msgr2=0x556af541c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/2201509424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af53d5f10 0x556af541c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7f0b95640 1 -- 172.21.15.19:0/2201509424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556af52f6e10 con 0x556af5400230 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.994+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/2201509424 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af541dad0 0x556af541e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/2201509424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af5400610 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4060a50 tx=0x7fe7e4058930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fae1b092043e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7d27fc640 1 -- 172.21.15.19:0/2201509424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e408a0d0 con 0x556af5400230 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7d27fc640 1 -- 172.21.15.19:0/2201509424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e4056b70 con 0x556af5400230 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7d27fc640 1 -- 172.21.15.19:0/2201509424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4060af0 con 0x556af5400230 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/2201509424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 msgr2=0x556af5400610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/2201509424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af5400610 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4060a50 tx=0x7fe7e4058930 comp rx=0 tx=0).stop 2024-04-25T09:56:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/2201509424 shutdown_connections 2024-04-25T09:56:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/2201509424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af53d5f10 0x556af541c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/2201509424 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af541dad0 0x556af541e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/2201509424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af5400610 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/2201509424 >> 172.21.15.19:0/2201509424 conn(0x556af53d2840 msgr2=0x556af541b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/2201509424 shutdown_connections 2024-04-25T09:56:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.995+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/2201509424 wait complete. 2024-04-25T09:56:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.996+0000 7fe7f5bdb740 1 -- start start 2024-04-25T09:56:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.996+0000 7fe7f5bdb740 1 Processor -- start 2024-04-25T09:56:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af53ed970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af541dad0 0x556af53edeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af5401060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af53d6cb0 con 0x556af5400230 2024-04-25T09:56:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af52f50c0 con 0x556af541dad0 2024-04-25T09:56:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x556af53d0770 con 0x556af53ee3f0 2024-04-25T09:56:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f5bdb740 10 client.0 _ll_register_callbacks cb 0x556af52e8ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f0b95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af53ed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f0b95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af53ed970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33348/0 (socket says 172.21.15.19:33348) 2024-04-25T09:56:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f0b95640 1 -- 172.21.15.19:0/216673156 learned_addr learned my addr 172.21.15.19:0/216673156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f1396640 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af5401060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af541dad0 0x556af53edeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f0b95640 1 -- 172.21.15.19:0/216673156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 msgr2=0x556af5401060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af5401060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f0b95640 1 -- 172.21.15.19:0/216673156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af541dad0 msgr2=0x556af53edeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af541dad0 0x556af53edeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f0b95640 1 -- 172.21.15.19:0/216673156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556af504d2d0 con 0x556af5400230 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af541dad0 0x556af53edeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f1396640 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af5401060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af53ed970 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4088670 tx=0x7fe7e4004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c810b8f86398def server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/216673156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4060cf0 con 0x556af5400230 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/216673156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe7e4059ba0 con 0x556af5400230 2024-04-25T09:56:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.997+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/216673156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7e4056b70 con 0x556af5400230 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/216673156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7e4088300 con 0x556af5400230 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7f5bdb740 10 client.6894 fetch_fsmap learned FSMap version 43 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/216673156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x556af51176b0 con 0x556af5400230 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/216673156 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e4056b70 con 0x556af5400230 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/216673156 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe7e40772f0 con 0x556af5400230 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7f5bdb740 10 client.6894 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7f5bdb740 10 client.6894 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7f5bdb740 20 client.6894 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x556af5437210 0x556af5439600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7f5bdb740 4 client.6894 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/216673156 --> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556af5397920 con 0x556af5437210 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x556af5437210 0x556af5439600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7cb7fe640 10 client.6894 ms_handle_connect on v2:172.21.15.19:6832/2482429280 2024-04-25T09:56:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.998+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x556af5437210 0x556af5439600 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1dd6a859789657 server_cookie=2e0bf58b6e1c0f5b in_seq=0 out_seq=0 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/216673156 <== mds.0 v2:172.21.15.19:6832/2482429280 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556af5397920 con 0x556af5437210 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7cb7fe640 10 client.6894 handle_command_reply: tid=0 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 1 client.6894 shutdown 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/216673156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 msgr2=0x556af53ed970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af53ed970 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4088670 tx=0x7fe7e4004a00 comp rx=0 tx=0).stop 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/216673156 shutdown_connections 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x556af5437210 0x556af5439600 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af5401060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af541dad0 0x556af53edeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7cb7fe640 0 client.6894 ms_handle_reset on v2:172.21.15.19:6832/2482429280 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/216673156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af5400230 0x556af53ed970 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/216673156 >> 172.21.15.19:0/216673156 conn(0x556af53d5f10 msgr2=0x556af541e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/216673156 shutdown_connections 2024-04-25T09:56:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/216673156 wait complete. 2024-04-25T09:56:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:58.999+0000 7fe7f5bdb740 20 client.6894 trim_cache size 0 max 16384 2024-04-25T09:56:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.015+0000 7fe7f5bdb740 1 Processor -- start 2024-04-25T09:56:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.015+0000 7fe7f5bdb740 1 -- start start 2024-04-25T09:56:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af53ed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 0x556af54023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af54028e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec200770 con 0x556af541dad0 2024-04-25T09:56:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec1ff8d0 con 0x556af5400230 2024-04-25T09:56:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec1fed60 con 0x556af53ee3f0 2024-04-25T09:56:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f1396640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af54028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f1396640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af54028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33356/0 (socket says 172.21.15.19:33356) 2024-04-25T09:56:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f1396640 1 -- 172.21.15.19:0/728392216 learned_addr learned my addr 172.21.15.19:0/728392216 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/728392216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 0x556af54023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/728392216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af53ed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f1396640 1 -- 172.21.15.19:0/728392216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 msgr2=0x556af53ed930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f1396640 1 --2- 172.21.15.19:0/728392216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af53ed930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f1396640 1 -- 172.21.15.19:0/728392216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 msgr2=0x556af54023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f1396640 1 --2- 172.21.15.19:0/728392216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 0x556af54023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f1396640 1 -- 172.21.15.19:0/728392216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556af5404440 con 0x556af541dad0 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/728392216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af53ed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/728392216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 0x556af54023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.016+0000 7fe7f1396640 1 --2- 172.21.15.19:0/728392216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af54028e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc035040 tx=0x7fe7dc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c39be6655206f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7d1ffb640 1 -- 172.21.15.19:0/728392216 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc037040 con 0x556af541dad0 2024-04-25T09:56:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7d1ffb640 1 -- 172.21.15.19:0/728392216 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc032820 con 0x556af541dad0 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7d1ffb640 1 -- 172.21.15.19:0/728392216 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc032c10 con 0x556af541dad0 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/728392216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 msgr2=0x556af54028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/728392216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af54028e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc035040 tx=0x7fe7dc02c000 comp rx=0 tx=0).stop 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/728392216 shutdown_connections 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/728392216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af54028e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/728392216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 0x556af54023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/728392216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af53ed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/728392216 >> 172.21.15.19:0/728392216 conn(0x556af53d5f10 msgr2=0x556af541e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/728392216 shutdown_connections 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.017+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/728392216 wait complete. 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.018+0000 7fe7f5bdb740 1 -- start start 2024-04-25T09:56:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.018+0000 7fe7f5bdb740 1 Processor -- start 2024-04-25T09:56:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af541e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 0x556af5435830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f5bdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af5479b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af53d6cb0 con 0x556af541dad0 2024-04-25T09:56:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec1ff2d0 con 0x556af5400230 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f5bdb740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x556af54020e0 con 0x556af53ee3f0 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f5bdb740 10 client.0 _ll_register_callbacks cb 0x556af53d0c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f1396640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af5479b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f1396640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af5479b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33366/0 (socket says 172.21.15.19:33366) 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f1396640 1 -- 172.21.15.19:0/1071553582 learned_addr learned my addr 172.21.15.19:0/1071553582 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af541e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 0x556af5435830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f1396640 1 -- 172.21.15.19:0/1071553582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 msgr2=0x556af541e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f1396640 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af541e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f1396640 1 -- 172.21.15.19:0/1071553582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 msgr2=0x556af5435830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f1396640 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 0x556af5435830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f1396640 1 -- 172.21.15.19:0/1071553582 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556af541df80 con 0x556af541dad0 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af541e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 0x556af5435830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7f1396640 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af5479b20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009890 tx=0x7fe7dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5996051029f64ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.019+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/1071553582 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc04b070 con 0x556af541dad0 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/1071553582 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe7dc03d640 con 0x556af541dad0 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/1071553582 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7dc032820 con 0x556af541dad0 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/1071553582 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7dc056050 con 0x556af541dad0 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7f5bdb740 10 client.6900 fetch_fsmap learned FSMap version 43 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/1071553582 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x556af53d0270 con 0x556af541dad0 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/1071553582 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc032820 con 0x556af541dad0 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/1071553582 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe7dc059020 con 0x556af541dad0 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7f5bdb740 10 client.6900 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7f5bdb740 10 client.6900 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7f5bdb740 20 client.6900 populate_metadata read hostname 'smithi019' 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x556af542e490 0x556af54862a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7f5bdb740 4 client.6900 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:56:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/1071553582 --> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556af5345f50 con 0x556af542e490 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x556af542e490 0x556af54862a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7cb7fe640 10 client.6900 ms_handle_connect on v2:172.21.15.19:6832/2482429280 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.020+0000 7fe7f0b95640 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x556af542e490 0x556af54862a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=609a580c67ad42cd server_cookie=ab5815841a7b811a in_seq=0 out_seq=0 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7cb7fe640 1 -- 172.21.15.19:0/1071553582 <== mds.0 v2:172.21.15.19:6832/2482429280 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x556af5345f50 con 0x556af542e490 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7cb7fe640 10 client.6900 handle_command_reply: tid=0 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 1 client.6900 shutdown 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/1071553582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 msgr2=0x556af5479b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af5479b20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009890 tx=0x7fe7dc004730 comp rx=0 tx=0).stop 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/1071553582 shutdown_connections 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x556af542e490 0x556af54862a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556af541dad0 0x556af5479b20 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556af5400230 0x556af5435830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7cb7fe640 0 client.6900 ms_handle_reset on v2:172.21.15.19:6832/2482429280 2024-04-25T09:56:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 1 --2- 172.21.15.19:0/1071553582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556af53ee3f0 0x556af541e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/1071553582 >> 172.21.15.19:0/1071553582 conn(0x556af53d5f10 msgr2=0x556af53ed930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/1071553582 shutdown_connections 2024-04-25T09:56:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 1 -- 172.21.15.19:0/1071553582 wait complete. 2024-04-25T09:56:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f5bdb740 20 client.6900 trim_cache size 0 max 16384 2024-04-25T09:56:59.031 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:56:59.031 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T09:56:59.031 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T09:56:59.031 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 6858, 2024-04-25T09:56:59.032 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T09:56:59.032 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T09:56:59.032 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T09:56:59.032 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 4.4922809880000001, 2024-04-25T09:56:59.032 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 43, 2024-04-25T09:56:59.032 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 70, 2024-04-25T09:56:59.032 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 70, 2024-04-25T09:56:59.032 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 5.3602211889999998 2024-04-25T09:56:59.032 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:56:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f2df9640 1 -- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d806b4d0 msgr2=0x7fe7d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f2df9640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d806b4d0 0x7fe7d806d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0cb890 tx=0x7fe7e0009040 comp rx=0 tx=0).stop 2024-04-25T09:56:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f2df9640 1 -- 172.21.15.19:0/3162607160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec0d0ab0 msgr2=0x7fe7ec1607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.021+0000 7fe7f2df9640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec0d0ab0 0x7fe7ec1607c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400ba80 tx=0x7fe7e400bf40 comp rx=0 tx=0).stop 2024-04-25T09:56:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.022+0000 7fe7f2df9640 1 -- 172.21.15.19:0/3162607160 shutdown_connections 2024-04-25T09:56:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.022+0000 7fe7f2df9640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d806b4d0 0x7fe7d806d990 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.022+0000 7fe7f2df9640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7ec15dac0 0x7fe7ec152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.022+0000 7fe7f2df9640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7ec0d34c0 0x7fe7ec152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.022+0000 7fe7f2df9640 1 --2- 172.21.15.19:0/3162607160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7ec0d0ab0 0x7fe7ec1607c0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.022+0000 7fe7f2df9640 1 -- 172.21.15.19:0/3162607160 >> 172.21.15.19:0/3162607160 conn(0x7fe7ec061c30 msgr2=0x7fe7ec0b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.022+0000 7fe7f2df9640 1 -- 172.21.15.19:0/3162607160 shutdown_connections 2024-04-25T09:56:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.022+0000 7fe7f2df9640 1 -- 172.21.15.19:0/3162607160 wait complete. 2024-04-25T09:56:59.036 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:56:59.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.133+0000 7ffbed10a640 1 Processor -- start 2024-04-25T09:56:59.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.133+0000 7ffbed10a640 1 -- start start 2024-04-25T09:56:59.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbed10a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffbe815da70 0x7ffbe8161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbed10a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbe80b9730 0x7ffbe80ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbed10a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffbe8161b80 0x7ffbe80ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbed10a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe81625e0 con 0x7ffbe8161b80 2024-04-25T09:56:59.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbed10a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe8162020 con 0x7ffbe815da70 2024-04-25T09:56:59.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbed10a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe8162300 con 0x7ffbe80b9730 2024-04-25T09:56:59.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbe759e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffbe8161b80 0x7ffbe80ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbe759e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffbe8161b80 0x7ffbe80ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33378/0 (socket says 172.21.15.19:33378) 2024-04-25T09:56:59.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbe759e640 1 -- 172.21.15.19:0/2435454373 learned_addr learned my addr 172.21.15.19:0/2435454373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbe6d9d640 1 --2- 172.21.15.19:0/2435454373 >> v2:172.21.15.112:3300/0 conn(0x7ffbe815da70 0x7ffbe8161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbe659c640 1 --2- 172.21.15.19:0/2435454373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbe80b9730 0x7ffbe80ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbe759e640 1 -- 172.21.15.19:0/2435454373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbe80b9730 msgr2=0x7ffbe80ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbe759e640 1 --2- 172.21.15.19:0/2435454373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbe80b9730 0x7ffbe80ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbe759e640 1 -- 172.21.15.19:0/2435454373 >> v2:172.21.15.112:3300/0 conn(0x7ffbe815da70 msgr2=0x7ffbe8161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbe759e640 1 --2- 172.21.15.19:0/2435454373 >> v2:172.21.15.112:3300/0 conn(0x7ffbe815da70 0x7ffbe8161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.134+0000 7ffbe759e640 1 -- 172.21.15.19:0/2435454373 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe80d1af0 con 0x7ffbe8161b80 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.135+0000 7ffbe759e640 1 --2- 172.21.15.19:0/2435454373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8161b80 0x7ffbe80ceb90 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80b61a0 tx=0x7ffbe002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6ffddb85486e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.135+0000 7ffbe5d9b640 1 -- 172.21.15.19:0/2435454373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0037070 con 0x7ffbe8161b80 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.135+0000 7ffbe5d9b640 1 -- 172.21.15.19:0/2435454373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe0032440 con 0x7ffbe8161b80 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.135+0000 7ffbe5d9b640 1 -- 172.21.15.19:0/2435454373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0032810 con 0x7ffbe8161b80 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.136+0000 7ffbed10a640 1 -- 172.21.15.19:0/2435454373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8161b80 msgr2=0x7ffbe80ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.136+0000 7ffbed10a640 1 --2- 172.21.15.19:0/2435454373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8161b80 0x7ffbe80ceb90 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80b61a0 tx=0x7ffbe002a140 comp rx=0 tx=0).stop 2024-04-25T09:56:59.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.136+0000 7ffbed10a640 1 -- 172.21.15.19:0/2435454373 shutdown_connections 2024-04-25T09:56:59.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.136+0000 7ffbed10a640 1 --2- 172.21.15.19:0/2435454373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbe80b9730 0x7ffbe80ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.136+0000 7ffbed10a640 1 --2- 172.21.15.19:0/2435454373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8161b80 0x7ffbe80ceb90 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.136+0000 7ffbed10a640 1 --2- 172.21.15.19:0/2435454373 >> v2:172.21.15.112:3300/0 conn(0x7ffbe815da70 0x7ffbe8161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.136+0000 7ffbed10a640 1 -- 172.21.15.19:0/2435454373 >> 172.21.15.19:0/2435454373 conn(0x7ffbe8061a20 msgr2=0x7ffbe8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.136+0000 7ffbed10a640 1 -- 172.21.15.19:0/2435454373 shutdown_connections 2024-04-25T09:56:59.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.136+0000 7ffbed10a640 1 -- 172.21.15.19:0/2435454373 wait complete. 2024-04-25T09:56:59.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.137+0000 7ffbed10a640 1 Processor -- start 2024-04-25T09:56:59.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.137+0000 7ffbed10a640 1 -- start start 2024-04-25T09:56:59.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbed10a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbe815da70 0x7ffbe815f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbed10a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8161b80 0x7ffbe80c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbed10a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe815dfd0 0x7ffbe8158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbed10a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe806c120 con 0x7ffbe8161b80 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbed10a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe815de50 con 0x7ffbe815dfd0 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbed10a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe806c2a0 con 0x7ffbe815da70 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbe659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8161b80 0x7ffbe80c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbe659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8161b80 0x7ffbe80c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33382/0 (socket says 172.21.15.19:33382) 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbe659c640 1 -- 172.21.15.19:0/1338941297 learned_addr learned my addr 172.21.15.19:0/1338941297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbe759e640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe815dfd0 0x7ffbe8158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbe6d9d640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbe815da70 0x7ffbe815f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbe659c640 1 -- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbe815da70 msgr2=0x7ffbe815f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbe659c640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbe815da70 0x7ffbe815f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbe659c640 1 -- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe815dfd0 msgr2=0x7ffbe8158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbe659c640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe815dfd0 0x7ffbe8158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.138+0000 7ffbe659c640 1 -- 172.21.15.19:0/1338941297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe815fa90 con 0x7ffbe8161b80 2024-04-25T09:56:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.139+0000 7ffbe659c640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8161b80 0x7ffbe80c8370 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00d7d0 tx=0x7ffbdc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.139+0000 7ffbcf7fe640 1 -- 172.21.15.19:0/1338941297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc004020 con 0x7ffbe8161b80 2024-04-25T09:56:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.139+0000 7ffbcf7fe640 1 -- 172.21.15.19:0/1338941297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc0041e0 con 0x7ffbe8161b80 2024-04-25T09:56:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.139+0000 7ffbed10a640 1 -- 172.21.15.19:0/1338941297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe815e4b0 con 0x7ffbe8161b80 2024-04-25T09:56:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.139+0000 7ffbcf7fe640 1 -- 172.21.15.19:0/1338941297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0043d0 con 0x7ffbe8161b80 2024-04-25T09:56:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.139+0000 7ffbed10a640 1 -- 172.21.15.19:0/1338941297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe815ea70 con 0x7ffbe8161b80 2024-04-25T09:56:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.140+0000 7ffbed10a640 1 -- 172.21.15.19:0/1338941297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe8073990 con 0x7ffbe8161b80 2024-04-25T09:56:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.140+0000 7ffbcf7fe640 1 -- 172.21.15.19:0/1338941297 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbdc01b020 con 0x7ffbe8161b80 2024-04-25T09:56:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.141+0000 7ffbcf7fe640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbd406b4d0 0x7ffbd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.141+0000 7ffbcf7fe640 1 -- 172.21.15.19:0/1338941297 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7ffbdc08a200 con 0x7ffbe8161b80 2024-04-25T09:56:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.141+0000 7ffbe6d9d640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbd406b4d0 0x7ffbd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.141+0000 7ffbe6d9d640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbd406b4d0 0x7ffbd406d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8003aa0 tx=0x7ffbd80098d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.145+0000 7ffbcf7fe640 1 -- 172.21.15.19:0/1338941297 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbdc0586f0 con 0x7ffbe8161b80 2024-04-25T09:56:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.361+0000 7ffbed10a640 1 -- 172.21.15.19:0/1338941297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffbe80d21f0 con 0x7ffbe8161b80 2024-04-25T09:56:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.361+0000 7ffbcf7fe640 1 -- 172.21.15.19:0/1338941297 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffbdc05c790 con 0x7ffbe8161b80 2024-04-25T09:56:59.365 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 -- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbd406b4d0 msgr2=0x7ffbd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbd406b4d0 0x7ffbd406d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8003aa0 tx=0x7ffbd80098d0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 -- 172.21.15.19:0/1338941297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8161b80 msgr2=0x7ffbe80c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8161b80 0x7ffbe80c8370 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00d7d0 tx=0x7ffbdc00dc90 comp rx=0 tx=0).stop 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 -- 172.21.15.19:0/1338941297 shutdown_connections 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffbd406b4d0 0x7ffbd406d990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe815dfd0 0x7ffbe8158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8161b80 0x7ffbe80c8370 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 --2- 172.21.15.19:0/1338941297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffbe815da70 0x7ffbe815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 -- 172.21.15.19:0/1338941297 >> 172.21.15.19:0/1338941297 conn(0x7ffbe8061a20 msgr2=0x7ffbe80b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 -- 172.21.15.19:0/1338941297 shutdown_connections 2024-04-25T09:56:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:56:59.363+0000 7ffbed10a640 1 -- 172.21.15.19:0/1338941297 wait complete. 2024-04-25T09:56:59.377 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:56:59.377 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:56:59.377 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T09:56:59.377 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:56:59.378 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:59.378 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:56:59.378 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:56:59.378 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:56:59.378 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:56:59.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:59.378 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T09:56:59.394 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:56:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:56:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:56:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:59.394 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42526sec preferred_lft 42526sec 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T09:56:59.395 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:59.397 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:56:59.458 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T09:56:59.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:59.459 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T09:56:59.516 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T09:56:59.516 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:56:59.516 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:56:59.516 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:56:59.516 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:56:59.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:59.517 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:56:59.517 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:56:59.517 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:56:59.517 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:56:59.517 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:56:59.517 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:56:59.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:59.602 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:56:59.617 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:56:59.617 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:56:59.617 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:56:59.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:59.617 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T09:56:59.681 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:56:59.682 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:59.737 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:56:59.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:56:59.737 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:59.792 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:56:59.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:59.793 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:56:59.875 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:56:59.875 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:56:59.883 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:56:59.883 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:56:59.884 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T09:56:59.884 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T09:56:59.884 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T09:56:59.884 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:56:59.884 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:56:59.885 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:56:59.885 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:56:59.885 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:56:59.886 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:56:59.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:59.887 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:59.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:59.952 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:57:00.009 INFO:teuthology.orchestra.run.smithi019.stdout:client5798 2024-04-25T09:57:00.009 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:57:00.009 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:00.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:00.065 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T09:57:00.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:57:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T09:57:00.122 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T09:57:00.122 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:00.122 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T09:57:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.264+0000 7f92ec329640 1 Processor -- start 2024-04-25T09:57:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.264+0000 7f92ec329640 1 -- start start 2024-04-25T09:57:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ec329640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92e40d34c0 0x7f92e40d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ec329640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92e40d08e0 0x7f92e40b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ec329640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e40d2420 con 0x7f92e415dac0 2024-04-25T09:57:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ec329640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e40d3030 con 0x7f92e40d08e0 2024-04-25T09:57:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ec329640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40d2700 con 0x7f92e40d34c0 2024-04-25T09:57:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ea8c6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92e40d08e0 0x7f92e40b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ea8c6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92e40d08e0 0x7f92e40b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33394/0 (socket says 172.21.15.19:33394) 2024-04-25T09:57:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ea8c6640 1 -- 172.21.15.19:0/1421985246 learned_addr learned my addr 172.21.15.19:0/1421985246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92e98c4640 1 --2- 172.21.15.19:0/1421985246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92e40d34c0 0x7f92e40d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ea8c6640 1 -- 172.21.15.19:0/1421985246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92e40d34c0 msgr2=0x7f92e40d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/1421985246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92e40d34c0 0x7f92e40d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ea8c6640 1 -- 172.21.15.19:0/1421985246 >> v1:172.21.15.112:6789/0 conn(0x7f92e415dac0 legacy=0x7f92e40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:00.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.265+0000 7f92ea8c6640 1 -- 172.21.15.19:0/1421985246 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40d29e0 con 0x7f92e40d08e0 2024-04-25T09:57:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.266+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/1421985246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e40d08e0 0x7f92e40b6990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009e50 tx=0x7f92dc02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56181513b5c0b2b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.266+0000 7f92e90c3640 1 -- 172.21.15.19:0/1421985246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc036070 con 0x7f92e40d08e0 2024-04-25T09:57:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.266+0000 7f92e90c3640 1 -- 172.21.15.19:0/1421985246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc031600 con 0x7f92e40d08e0 2024-04-25T09:57:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.266+0000 7f92e90c3640 1 -- 172.21.15.19:0/1421985246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0319d0 con 0x7f92e40d08e0 2024-04-25T09:57:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.267+0000 7f92ec329640 1 -- 172.21.15.19:0/1421985246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e40d08e0 msgr2=0x7f92e40b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.267+0000 7f92ec329640 1 --2- 172.21.15.19:0/1421985246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e40d08e0 0x7f92e40b6990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009e50 tx=0x7f92dc02a750 comp rx=0 tx=0).stop 2024-04-25T09:57:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.267+0000 7f92ec329640 1 -- 172.21.15.19:0/1421985246 shutdown_connections 2024-04-25T09:57:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.267+0000 7f92ec329640 1 --2- 172.21.15.19:0/1421985246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e40d08e0 0x7f92e40b6990 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.267+0000 7f92ec329640 1 --2- 172.21.15.19:0/1421985246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92e40d34c0 0x7f92e40d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.267+0000 7f92ec329640 1 -- 172.21.15.19:0/1421985246 >> 172.21.15.19:0/1421985246 conn(0x7f92e4061c30 msgr2=0x7f92e406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.267+0000 7f92ec329640 1 -- 172.21.15.19:0/1421985246 shutdown_connections 2024-04-25T09:57:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.267+0000 7f92ec329640 1 -- 172.21.15.19:0/1421985246 wait complete. 2024-04-25T09:57:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.268+0000 7f92ec329640 1 Processor -- start 2024-04-25T09:57:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.268+0000 7f92ec329640 1 -- start start 2024-04-25T09:57:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ec329640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e40d08e0 0x7f92e40aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ec329640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92e40d34c0 0x7f92e414f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ec329640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e415dac0 0x7f92e414c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ec329640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e415dea0 con 0x7f92e415dac0 2024-04-25T09:57:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ec329640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40d2100 con 0x7f92e40d08e0 2024-04-25T09:57:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ec329640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40b97d0 con 0x7f92e40d34c0 2024-04-25T09:57:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ea8c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e415dac0 0x7f92e414c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ea8c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e415dac0 0x7f92e414c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33400/0 (socket says 172.21.15.19:33400) 2024-04-25T09:57:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3063990269 learned_addr learned my addr 172.21.15.19:0/3063990269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ea0c5640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e40d08e0 0x7f92e40aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92e98c4640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92e40d34c0 0x7f92e414f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92e40d34c0 msgr2=0x7f92e414f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92e40d34c0 0x7f92e414f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e40d08e0 msgr2=0x7f92e40aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e40d08e0 0x7f92e40aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3063990269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e415eef0 con 0x7f92e415dac0 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.269+0000 7f92e98c4640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92e40d34c0 0x7f92e414f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.270+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e415dac0 0x7f92e414c740 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009e50 tx=0x7f92dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.270+0000 7f92daffd640 1 -- 172.21.15.19:0/3063990269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc036050 con 0x7f92e415dac0 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.270+0000 7f92daffd640 1 -- 172.21.15.19:0/3063990269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc042070 con 0x7f92e415dac0 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.270+0000 7f92ec329640 1 -- 172.21.15.19:0/3063990269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e40aa900 con 0x7f92e415dac0 2024-04-25T09:57:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.270+0000 7f92daffd640 1 -- 172.21.15.19:0/3063990269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc009510 con 0x7f92e415dac0 2024-04-25T09:57:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.270+0000 7f92ec329640 1 -- 172.21.15.19:0/3063990269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e415f470 con 0x7f92e415dac0 2024-04-25T09:57:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.271+0000 7f92daffd640 1 -- 172.21.15.19:0/3063990269 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92dc007bd0 con 0x7f92e415dac0 2024-04-25T09:57:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.272+0000 7f92daffd640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92c806b500 0x7f92c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.272+0000 7f92daffd640 1 -- 172.21.15.19:0/3063990269 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f92dc0ab390 con 0x7f92e415dac0 2024-04-25T09:57:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.272+0000 7f92ea0c5640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92c806b500 0x7f92c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.272+0000 7f92ea0c5640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92c806b500 0x7f92c806d9c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f92e0006fd0 tx=0x7f92e0009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:00.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.273+0000 7f92ef02b740 1 Processor -- start 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.273+0000 7f92ef02b740 1 -- start start 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.273+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f535d20 0x55ba5f4e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f518e40 0x55ba5f5181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f518700 0x55ba5f4ed510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92ef02b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba5f450f40 con 0x55ba5f518e40 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92ef02b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba5f3f00d0 con 0x55ba5f518700 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92ef02b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba5f44cfe0 con 0x55ba5f535d20 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92e98c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f518e40 0x55ba5f5181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92e98c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f518e40 0x55ba5f5181c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33410/0 (socket says 172.21.15.19:33410) 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92e98c4640 1 -- 172.21.15.19:0/2861309308 learned_addr learned my addr 172.21.15.19:0/2861309308 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/2861309308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f518700 0x55ba5f4ed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92ea0c5640 1 --2- 172.21.15.19:0/2861309308 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f535d20 0x55ba5f4e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92e98c4640 1 -- 172.21.15.19:0/2861309308 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f535d20 msgr2=0x55ba5f4e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92e98c4640 1 --2- 172.21.15.19:0/2861309308 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f535d20 0x55ba5f4e86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92e98c4640 1 -- 172.21.15.19:0/2861309308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f518700 msgr2=0x55ba5f4ed510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92e98c4640 1 --2- 172.21.15.19:0/2861309308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f518700 0x55ba5f4ed510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92e98c4640 1 -- 172.21.15.19:0/2861309308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba5f231af0 con 0x55ba5f518e40 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92e98c4640 1 --2- 172.21.15.19:0/2861309308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f518e40 0x55ba5f5181c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f92d4007960 tx=0x7f92d4007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e67607a30542f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92c7fff640 1 -- 172.21.15.19:0/2861309308 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4018030 con 0x55ba5f518e40 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92c7fff640 1 -- 172.21.15.19:0/2861309308 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d4004020 con 0x55ba5f518e40 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.274+0000 7f92c7fff640 1 -- 172.21.15.19:0/2861309308 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40043f0 con 0x55ba5f518e40 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.275+0000 7f92ef02b740 1 -- 172.21.15.19:0/2861309308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f518e40 msgr2=0x55ba5f5181c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.275+0000 7f92ef02b740 1 --2- 172.21.15.19:0/2861309308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f518e40 0x55ba5f5181c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f92d4007960 tx=0x7f92d4007e20 comp rx=0 tx=0).stop 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.275+0000 7f92ef02b740 1 -- 172.21.15.19:0/2861309308 shutdown_connections 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.275+0000 7f92ef02b740 1 --2- 172.21.15.19:0/2861309308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f518700 0x55ba5f4ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.275+0000 7f92ef02b740 1 --2- 172.21.15.19:0/2861309308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f518e40 0x55ba5f5181c0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.275+0000 7f92ef02b740 1 --2- 172.21.15.19:0/2861309308 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f535d20 0x55ba5f4e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.275+0000 7f92ef02b740 1 -- 172.21.15.19:0/2861309308 >> 172.21.15.19:0/2861309308 conn(0x55ba5f4ea920 msgr2=0x55ba5f5331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.275+0000 7f92ef02b740 1 -- 172.21.15.19:0/2861309308 shutdown_connections 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.275+0000 7f92ef02b740 1 -- 172.21.15.19:0/2861309308 wait complete. 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.276+0000 7f92ef02b740 1 -- start start 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.276+0000 7f92ef02b740 1 Processor -- start 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f518e40 0x55ba5f5100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f535d20 0x55ba5f531bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ef02b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba5f4ecad0 con 0x55ba5f547170 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ef02b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba5f3edba0 con 0x55ba5f535d20 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ef02b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba5f4ebec0 con 0x55ba5f518e40 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ef02b740 10 client.0 _ll_register_callbacks cb 0x55ba5f3ed570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ea8c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ea8c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33414/0 (socket says 172.21.15.19:33414) 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ea8c6640 1 -- 172.21.15.19:0/368739372 learned_addr learned my addr 172.21.15.19:0/368739372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92e98c4640 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f535d20 0x55ba5f531bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ea0c5640 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f518e40 0x55ba5f5100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ea8c6640 1 -- 172.21.15.19:0/368739372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f518e40 msgr2=0x55ba5f5100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f518e40 0x55ba5f5100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ea8c6640 1 -- 172.21.15.19:0/368739372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f535d20 msgr2=0x55ba5f531bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f535d20 0x55ba5f531bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ea8c6640 1 -- 172.21.15.19:0/368739372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba5f1253e0 con 0x55ba5f547170 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a8a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0794c0 tx=0x7f92dc009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8e6848d688e49d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92c57fa640 1 -- 172.21.15.19:0/368739372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc091020 con 0x55ba5f547170 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92c57fa640 1 -- 172.21.15.19:0/368739372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f92dc0911c0 con 0x55ba5f547170 2024-04-25T09:57:00.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.277+0000 7f92ef02b740 1 -- 172.21.15.19:0/368739372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92d4004020 con 0x55ba5f547170 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92c57fa640 1 -- 172.21.15.19:0/368739372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92dc07d910 con 0x55ba5f547170 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92ef02b740 10 client.6918 fetch_fsmap learned FSMap version 43 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92ef02b740 1 -- 172.21.15.19:0/368739372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55ba5f292280 con 0x55ba5f547170 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92c57fa640 1 -- 172.21.15.19:0/368739372 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d4004020 con 0x55ba5f547170 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92c57fa640 1 -- 172.21.15.19:0/368739372 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f92dc0a9080 con 0x55ba5f547170 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92ef02b740 10 client.6918 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92ef02b740 10 client.6918 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92ef02b740 20 client.6918 populate_metadata read hostname 'smithi019' 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92ef02b740 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x55ba5f4eced0 0x55ba5f4ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92ef02b740 4 client.6918 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92ef02b740 1 -- 172.21.15.19:0/368739372 --> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba5f45e480 con 0x55ba5f4eced0 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92ea0c5640 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x55ba5f4eced0 0x55ba5f4ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92c57fa640 10 client.6918 ms_handle_connect on v2:172.21.15.19:6832/2482429280 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.278+0000 7f92ea0c5640 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x55ba5f4eced0 0x55ba5f4ed2b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d2f1cc38b6f58e server_cookie=d38904ba430a36e2 in_seq=0 out_seq=0 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92c57fa640 1 -- 172.21.15.19:0/368739372 <== mds.0 v2:172.21.15.19:6832/2482429280 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba5f45e480 con 0x55ba5f4eced0 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92c57fa640 10 client.6918 handle_command_reply: tid=0 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 1 client.6918 shutdown 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 1 -- 172.21.15.19:0/368739372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 msgr2=0x55ba5f51a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a8a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0794c0 tx=0x7f92dc009b10 comp rx=0 tx=0).stop 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 1 -- 172.21.15.19:0/368739372 shutdown_connections 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x55ba5f4eced0 0x55ba5f4ed2b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a8a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92c57fa640 0 client.6918 ms_handle_reset on v2:172.21.15.19:6832/2482429280 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f535d20 0x55ba5f531bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 1 --2- 172.21.15.19:0/368739372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f518e40 0x55ba5f5100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 1 -- 172.21.15.19:0/368739372 >> 172.21.15.19:0/368739372 conn(0x55ba5f518700 msgr2=0x55ba5f5331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:00.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 1 -- 172.21.15.19:0/368739372 shutdown_connections 2024-04-25T09:57:00.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 1 -- 172.21.15.19:0/368739372 wait complete. 2024-04-25T09:57:00.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.279+0000 7f92ef02b740 20 client.6918 trim_cache size 0 max 16384 2024-04-25T09:57:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.295+0000 7f92ef02b740 1 Processor -- start 2024-04-25T09:57:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.295+0000 7f92ef02b740 1 -- start start 2024-04-25T09:57:00.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.295+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f518e40 0x55ba5f542cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.295+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f535d20 0x55ba5f51a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.295+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.295+0000 7f92ef02b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4200880 con 0x55ba5f547170 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.295+0000 7f92ef02b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41ffa50 con 0x55ba5f518e40 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.295+0000 7f92ef02b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41f7ce0 con 0x55ba5f535d20 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.295+0000 7f92ea8c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92ea8c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33428/0 (socket says 172.21.15.19:33428) 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3022754400 learned_addr learned my addr 172.21.15.19:0/3022754400 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92ea0c5640 1 --2- 172.21.15.19:0/3022754400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f518e40 0x55ba5f542cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92e98c4640 1 --2- 172.21.15.19:0/3022754400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f535d20 0x55ba5f51a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3022754400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f535d20 msgr2=0x55ba5f51a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/3022754400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f535d20 0x55ba5f51a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3022754400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f518e40 msgr2=0x55ba5f542cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/3022754400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f518e40 0x55ba5f542cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3022754400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba5f0fe560 con 0x55ba5f547170 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92e98c4640 1 --2- 172.21.15.19:0/3022754400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f535d20 0x55ba5f51a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/3022754400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a880 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0094a0 tx=0x7f92dc07d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85df668e8410e58e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92c77fe640 1 -- 172.21.15.19:0/3022754400 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc091040 con 0x55ba5f547170 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92c77fe640 1 -- 172.21.15.19:0/3022754400 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc037040 con 0x55ba5f547170 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.296+0000 7f92c77fe640 1 -- 172.21.15.19:0/3022754400 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0911e0 con 0x55ba5f547170 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.297+0000 7f92ef02b740 1 -- 172.21.15.19:0/3022754400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 msgr2=0x55ba5f51a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.297+0000 7f92ef02b740 1 --2- 172.21.15.19:0/3022754400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a880 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0094a0 tx=0x7f92dc07d000 comp rx=0 tx=0).stop 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.297+0000 7f92ef02b740 1 -- 172.21.15.19:0/3022754400 shutdown_connections 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.297+0000 7f92ef02b740 1 --2- 172.21.15.19:0/3022754400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f51a880 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.297+0000 7f92ef02b740 1 --2- 172.21.15.19:0/3022754400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f535d20 0x55ba5f51a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.297+0000 7f92ef02b740 1 --2- 172.21.15.19:0/3022754400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f518e40 0x55ba5f542cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.297+0000 7f92ef02b740 1 -- 172.21.15.19:0/3022754400 >> 172.21.15.19:0/3022754400 conn(0x55ba5f518700 msgr2=0x55ba5f50fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.297+0000 7f92ef02b740 1 -- 172.21.15.19:0/3022754400 shutdown_connections 2024-04-25T09:57:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.297+0000 7f92ef02b740 1 -- 172.21.15.19:0/3022754400 wait complete. 2024-04-25T09:57:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.297+0000 7f92ef02b740 1 -- start start 2024-04-25T09:57:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.297+0000 7f92ef02b740 1 Processor -- start 2024-04-25T09:57:00.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f518e40 0x55ba5f50e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f535d20 0x55ba5f545f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92ef02b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f545730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92ef02b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba5f4ef6e0 con 0x55ba5f547170 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92ef02b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41ff450 con 0x55ba5f535d20 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92ef02b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba5f4e7ec0 con 0x55ba5f518e40 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92ef02b740 10 client.0 _ll_register_callbacks cb 0x7f92dc079660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92ea8c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f545730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92ea8c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f545730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33444/0 (socket says 172.21.15.19:33444) 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3854351783 learned_addr learned my addr 172.21.15.19:0/3854351783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92e98c4640 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f535d20 0x55ba5f545f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.298+0000 7f92ea0c5640 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f518e40 0x55ba5f50e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3854351783 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f518e40 msgr2=0x55ba5f50e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f518e40 0x55ba5f50e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3854351783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f535d20 msgr2=0x55ba5f545f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f535d20 0x55ba5f545f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ea8c6640 1 -- 172.21.15.19:0/3854351783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba5f545c70 con 0x55ba5f547170 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ea8c6640 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f545730 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f92dc077a70 tx=0x7f92dc0a9440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909c7fc5e2a1a11d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92c57fa640 1 -- 172.21.15.19:0/3854351783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc081ed0 con 0x55ba5f547170 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92c57fa640 1 -- 172.21.15.19:0/3854351783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f92dc0ab0b0 con 0x55ba5f547170 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ef02b740 1 -- 172.21.15.19:0/3854351783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92dc037040 con 0x55ba5f547170 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92c57fa640 1 -- 172.21.15.19:0/3854351783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92dc08b050 con 0x55ba5f547170 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ef02b740 10 client.6924 fetch_fsmap learned FSMap version 43 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ef02b740 1 -- 172.21.15.19:0/3854351783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55ba5f545c70 con 0x55ba5f547170 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92c57fa640 1 -- 172.21.15.19:0/3854351783 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc037040 con 0x55ba5f547170 2024-04-25T09:57:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92c57fa640 1 -- 172.21.15.19:0/3854351783 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f92dc0a9080 con 0x55ba5f547170 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ef02b740 10 client.6924 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ef02b740 10 client.6924 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ef02b740 20 client.6924 populate_metadata read hostname 'smithi019' 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ef02b740 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x55ba5f4ed2b0 0x55ba5f5422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ef02b740 4 client.6924 mds_command: new command op to 6858 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ef02b740 1 -- 172.21.15.19:0/3854351783 --> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x55ba5f4587d0 con 0x55ba5f4ed2b0 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.299+0000 7f92ea0c5640 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x55ba5f4ed2b0 0x55ba5f5422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ea0c5640 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x55ba5f4ed2b0 0x55ba5f5422d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ed72dcd66cc3b4 server_cookie=1553c1fd867b234e in_seq=0 out_seq=0 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92c57fa640 10 client.6924 ms_handle_connect on v2:172.21.15.19:6832/2482429280 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92c57fa640 1 -- 172.21.15.19:0/3854351783 <== mds.0 v2:172.21.15.19:6832/2482429280 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ba5f4587d0 con 0x55ba5f4ed2b0 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92c57fa640 10 client.6924 handle_command_reply: tid=0 2024-04-25T09:57:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ef02b740 1 client.6924 shutdown 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ef02b740 1 -- 172.21.15.19:0/3854351783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 msgr2=0x55ba5f545730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ef02b740 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f545730 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f92dc077a70 tx=0x7f92dc0a9440 comp rx=0 tx=0).stop 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ef02b740 1 -- 172.21.15.19:0/3854351783 shutdown_connections 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ef02b740 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.19:6832/2482429280,v1:172.21.15.19:6833/2482429280] conn(0x55ba5f4ed2b0 0x55ba5f5422d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ef02b740 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba5f547170 0x55ba5f545730 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ef02b740 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba5f535d20 0x55ba5f545f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ef02b740 1 --2- 172.21.15.19:0/3854351783 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ba5f518e40 0x55ba5f50e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92c57fa640 0 client.6924 ms_handle_reset on v2:172.21.15.19:6832/2482429280 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ef02b740 1 -- 172.21.15.19:0/3854351783 >> 172.21.15.19:0/3854351783 conn(0x55ba5f518700 msgr2=0x55ba5f529c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ef02b740 1 -- 172.21.15.19:0/3854351783 shutdown_connections 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.300+0000 7f92ef02b740 1 -- 172.21.15.19:0/3854351783 wait complete. 2024-04-25T09:57:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ef02b740 20 client.6924 trim_cache size 0 max 16384 2024-04-25T09:57:00.311 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:57:00.311 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:57:00.311 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "66b05c29-fa18-4923-a8cf-4e9f26d33841", 2024-04-25T09:57:00.311 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:57:00.311 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 -- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92c806b500 msgr2=0x7f92c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92c806b500 0x7f92c806d9c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f92e0006fd0 tx=0x7f92e0009060 comp rx=0 tx=0).stop 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 -- 172.21.15.19:0/3063990269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e415dac0 msgr2=0x7f92e414c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e415dac0 0x7f92e414c740 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009e50 tx=0x7f92dc004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 -- 172.21.15.19:0/3063990269 shutdown_connections 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92c806b500 0x7f92c806d9c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e415dac0 0x7f92e414c740 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92e40d34c0 0x7f92e414f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 --2- 172.21.15.19:0/3063990269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92e40d08e0 0x7f92e40aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 -- 172.21.15.19:0/3063990269 >> 172.21.15.19:0/3063990269 conn(0x7f92e4061c30 msgr2=0x7f92e40c7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 -- 172.21.15.19:0/3063990269 shutdown_connections 2024-04-25T09:57:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:00.301+0000 7f92ec329640 1 -- 172.21.15.19:0/3063990269 wait complete. 2024-04-25T09:57:00.315 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': '66b05c29-fa18-4923-a8cf-4e9f26d33841', 'mode': 'asynchronous'}' 2024-04-25T09:57:00.529 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T09:57:00.526+0000 7fe546160640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:57:07.367 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:07.370 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:10.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:10.315 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T09:57:10.337 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:57:10.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:10.416 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:10.464 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:57:10.464 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:57:10.464 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:10.483 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:57:10.483 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:10.542 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.689+0000 7f5af2eb3640 1 Processor -- start 2024-04-25T09:57:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.689+0000 7f5af2eb3640 1 -- start start 2024-04-25T09:57:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.690+0000 7f5af2eb3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5aec15dd20 0x7f5aec0b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.690+0000 7f5af2eb3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec15eac0 con 0x7f5aec15e100 2024-04-25T09:57:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.690+0000 7f5af2eb3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec15eda0 con 0x7f5aec0d11b0 2024-04-25T09:57:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.690+0000 7f5af2eb3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0d1c00 con 0x7f5aec15dd20 2024-04-25T09:57:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.690+0000 7f5aebfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5aec15e100 0x7f5aec0d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37530/0 (socket says 172.21.15.19:37530) 2024-04-25T09:57:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.690+0000 7f5aebfff640 1 -- 172.21.15.19:0/1414001995 learned_addr learned my addr 172.21.15.19:0/1414001995 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.691+0000 7f5af0c4f640 1 --2- 172.21.15.19:0/1414001995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5aec15dd20 0x7f5aec0b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.691+0000 7f5aeb7fe640 1 -- 172.21.15.19:0/1414001995 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963698702 0 0) 0x7f5aec15eac0 con 0x7f5aec15e100 2024-04-25T09:57:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.691+0000 7f5aeb7fe640 1 -- 172.21.15.19:0/1414001995 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad8003520 con 0x7f5aec15e100 2024-04-25T09:57:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.691+0000 7f5aeb7fe640 1 -- 172.21.15.19:0/1414001995 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661822953 0 0) 0x7f5aec15eda0 con 0x7f5aec0d11b0 2024-04-25T09:57:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.691+0000 7f5aeb7fe640 1 -- 172.21.15.19:0/1414001995 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5aec15eac0 con 0x7f5aec0d11b0 2024-04-25T09:57:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.691+0000 7f5af0c4f640 1 -- 172.21.15.19:0/1414001995 >> v1:172.21.15.112:6789/0 conn(0x7f5aec0d11b0 legacy=0x7f5aec0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.691+0000 7f5af0c4f640 1 -- 172.21.15.19:0/1414001995 >> v1:172.21.15.19:6789/0 conn(0x7f5aec15e100 legacy=0x7f5aec0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.691+0000 7f5af0c4f640 1 -- 172.21.15.19:0/1414001995 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec15f080 con 0x7f5aec15dd20 2024-04-25T09:57:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.692+0000 7f5af0c4f640 1 --2- 172.21.15.19:0/1414001995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5aec15dd20 0x7f5aec0b96d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0d1d80 tx=0x7f5ae402a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1673340e70f0d05f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.692+0000 7f5aeb7fe640 1 -- 172.21.15.19:0/1414001995 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4007740 con 0x7f5aec15dd20 2024-04-25T09:57:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.692+0000 7f5aeb7fe640 1 -- 172.21.15.19:0/1414001995 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae402ae50 con 0x7f5aec15dd20 2024-04-25T09:57:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.692+0000 7f5aeb7fe640 1 -- 172.21.15.19:0/1414001995 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4007ae0 con 0x7f5aec15dd20 2024-04-25T09:57:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.692+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1414001995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5aec15dd20 msgr2=0x7f5aec0b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.692+0000 7f5af2eb3640 1 --2- 172.21.15.19:0/1414001995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5aec15dd20 0x7f5aec0b96d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0d1d80 tx=0x7f5ae402a630 comp rx=0 tx=0).stop 2024-04-25T09:57:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.693+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1414001995 shutdown_connections 2024-04-25T09:57:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.693+0000 7f5af2eb3640 1 --2- 172.21.15.19:0/1414001995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5aec15dd20 0x7f5aec0b96d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.693+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1414001995 >> 172.21.15.19:0/1414001995 conn(0x7f5aec061a20 msgr2=0x7f5aec061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.693+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1414001995 shutdown_connections 2024-04-25T09:57:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.693+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1414001995 wait complete. 2024-04-25T09:57:10.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.694+0000 7f5af2eb3640 1 Processor -- start 2024-04-25T09:57:10.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.694+0000 7f5af2eb3640 1 -- start start 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5af2eb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aec0d11b0 0x7f5aec1ed7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5af2eb3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aec15dd20 0x7f5aec0ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5af2eb3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5aec15e100 0x7f5aec1f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5af2eb3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec15e6e0 con 0x7f5aec15dd20 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5af2eb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec15e560 con 0x7f5aec0d11b0 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5af2eb3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec15e860 con 0x7f5aec15e100 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aec15dd20 0x7f5aec0ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aec15dd20 0x7f5aec0ac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50330/0 (socket says 172.21.15.19:50330) 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5aebfff640 1 -- 172.21.15.19:0/1084874635 learned_addr learned my addr 172.21.15.19:0/1084874635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5af0c4f640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aec0d11b0 0x7f5aec1ed7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5af1450640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5aec15e100 0x7f5aec1f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5aebfff640 1 -- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5aec15e100 msgr2=0x7f5aec1f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5aebfff640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5aec15e100 0x7f5aec1f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5aebfff640 1 -- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aec0d11b0 msgr2=0x7f5aec1ed7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5aebfff640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aec0d11b0 0x7f5aec1ed7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.695+0000 7f5aebfff640 1 -- 172.21.15.19:0/1084874635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0ab7e0 con 0x7f5aec15dd20 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.696+0000 7f5aebfff640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aec15dd20 0x7f5aec0ac840 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00c410 tx=0x7f5adc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.696+0000 7f5ae97fa640 1 -- 172.21.15.19:0/1084874635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc017070 con 0x7f5aec15dd20 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.696+0000 7f5ae97fa640 1 -- 172.21.15.19:0/1084874635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc005ce0 con 0x7f5aec15dd20 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.696+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1084874635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec0aba80 con 0x7f5aec15dd20 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.696+0000 7f5ae97fa640 1 -- 172.21.15.19:0/1084874635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0072d0 con 0x7f5aec15dd20 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.696+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1084874635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec0ac040 con 0x7f5aec15dd20 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.697+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1084874635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec073990 con 0x7f5aec15dd20 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.697+0000 7f5ae97fa640 1 -- 172.21.15.19:0/1084874635 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5adc01a070 con 0x7f5aec15dd20 2024-04-25T09:57:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.698+0000 7f5ae97fa640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5acc06b4d0 0x7f5acc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.698+0000 7f5ae97fa640 1 -- 172.21.15.19:0/1084874635 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f5adc0897b0 con 0x7f5aec15dd20 2024-04-25T09:57:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.698+0000 7f5af0c4f640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5acc06b4d0 0x7f5acc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.699+0000 7f5af0c4f640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5acc06b4d0 0x7f5acc06d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4034000 tx=0x7f5ae402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:10.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.702+0000 7f5ae97fa640 1 -- 172.21.15.19:0/1084874635 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5adc057d30 con 0x7f5aec15dd20 2024-04-25T09:57:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.881+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1084874635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5aec0b8ce0 con 0x7f5aec15dd20 2024-04-25T09:57:10.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.883+0000 7f5ae97fa640 1 -- 172.21.15.19:0/1084874635 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17321 (secure 0 0 0) 0x7f5adc05bdd0 con 0x7f5aec15dd20 2024-04-25T09:57:10.886 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:10.886 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":70,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:56:54.515339+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:56:37.087339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:56:37.775183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3520485752":"2024-04-26T09:56:54.515283+0000","172.21.15.19:6833/3520485752":"2024-04-26T09:56:54.515283+0000","172.21.15.19:6834/591570980":"2024-04-26T09:56:53.512369+0000","172.21.15.19:6832/591570980":"2024-04-26T09:56:53.512369+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:10.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.884+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5acc06b4d0 msgr2=0x7f5acc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.884+0000 7f5af2eb3640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5acc06b4d0 0x7f5acc06d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4034000 tx=0x7f5ae402f000 comp rx=0 tx=0).stop 2024-04-25T09:57:10.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.884+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1084874635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aec15dd20 msgr2=0x7f5aec0ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.884+0000 7f5af2eb3640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aec15dd20 0x7f5aec0ac840 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00c410 tx=0x7f5adc00c8d0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.885+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1084874635 shutdown_connections 2024-04-25T09:57:10.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.885+0000 7f5af2eb3640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5acc06b4d0 0x7f5acc06d990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.885+0000 7f5af2eb3640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5aec15e100 0x7f5aec1f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.885+0000 7f5af2eb3640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aec15dd20 0x7f5aec0ac840 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.885+0000 7f5af2eb3640 1 --2- 172.21.15.19:0/1084874635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5aec0d11b0 0x7f5aec1ed7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.885+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1084874635 >> 172.21.15.19:0/1084874635 conn(0x7f5aec061a20 msgr2=0x7f5aec0d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:10.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.885+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1084874635 shutdown_connections 2024-04-25T09:57:10.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:10.885+0000 7f5af2eb3640 1 -- 172.21.15.19:0/1084874635 wait complete. 2024-04-25T09:57:10.901 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:56:37.087339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:56:37.775183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:10.901 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.002+0000 7fe8d66dd640 1 Processor -- start 2024-04-25T09:57:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.002+0000 7fe8d66dd640 1 -- start start 2024-04-25T09:57:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.003+0000 7fe8d66dd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe8d00ce7b0 0x7fe8d0161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.003+0000 7fe8d66dd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d006c8d0 con 0x7fe8d01616f0 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.003+0000 7fe8d66dd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d00c6960 con 0x7fe8d01610f0 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.003+0000 7fe8d66dd640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d01614d0 con 0x7fe8d00ce7b0 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.003+0000 7fe8d4c7a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe8d00ce7b0 0x7fe8d0161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.003+0000 7fe8d4c7a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe8d00ce7b0 0x7fe8d0161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50332/0 (socket says 172.21.15.19:50332) 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.003+0000 7fe8cf7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe8d01616f0 0x7fe8d00ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37534/0 (socket says 172.21.15.19:37534) 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.003+0000 7fe8d4c7a640 1 -- 172.21.15.19:0/3920915699 learned_addr learned my addr 172.21.15.19:0/3920915699 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.004+0000 7fe8d4c7a640 1 -- 172.21.15.19:0/3920915699 >> v1:172.21.15.112:6789/0 conn(0x7fe8d01610f0 legacy=0x7fe8d006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.004+0000 7fe8d4c7a640 1 -- 172.21.15.19:0/3920915699 >> v1:172.21.15.19:6789/0 conn(0x7fe8d01616f0 legacy=0x7fe8d00ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.004+0000 7fe8ceffd640 1 -- 172.21.15.19:0/3920915699 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898652995 0 0) 0x7fe8d006c8d0 con 0x7fe8d01616f0 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.004+0000 7fe8d4c7a640 1 -- 172.21.15.19:0/3920915699 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d0162260 con 0x7fe8d00ce7b0 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.004+0000 7fe8d4c7a640 1 --2- 172.21.15.19:0/3920915699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8d00ce7b0 0x7fe8d0161b50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009e50 tx=0x7fe8c802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98826fab54db720e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.005+0000 7fe8ceffd640 1 -- 172.21.15.19:0/3920915699 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8036070 con 0x7fe8d00ce7b0 2024-04-25T09:57:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.005+0000 7fe8ceffd640 1 -- 172.21.15.19:0/3920915699 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c8031600 con 0x7fe8d00ce7b0 2024-04-25T09:57:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.005+0000 7fe8ceffd640 1 -- 172.21.15.19:0/3920915699 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80319d0 con 0x7fe8d00ce7b0 2024-04-25T09:57:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.005+0000 7fe8d66dd640 1 -- 172.21.15.19:0/3920915699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8d00ce7b0 msgr2=0x7fe8d0161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.005+0000 7fe8d66dd640 1 --2- 172.21.15.19:0/3920915699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8d00ce7b0 0x7fe8d0161b50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009e50 tx=0x7fe8c802a760 comp rx=0 tx=0).stop 2024-04-25T09:57:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.005+0000 7fe8d66dd640 1 -- 172.21.15.19:0/3920915699 shutdown_connections 2024-04-25T09:57:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.005+0000 7fe8d66dd640 1 --2- 172.21.15.19:0/3920915699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8d00ce7b0 0x7fe8d0161b50 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.005+0000 7fe8d66dd640 1 -- 172.21.15.19:0/3920915699 >> 172.21.15.19:0/3920915699 conn(0x7fe8d006f500 msgr2=0x7fe8d006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.005+0000 7fe8d66dd640 1 -- 172.21.15.19:0/3920915699 shutdown_connections 2024-04-25T09:57:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.005+0000 7fe8d66dd640 1 -- 172.21.15.19:0/3920915699 wait complete. 2024-04-25T09:57:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.006+0000 7fe8d66dd640 1 Processor -- start 2024-04-25T09:57:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.007+0000 7fe8d66dd640 1 -- start start 2024-04-25T09:57:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.007+0000 7fe8d66dd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8d00ce7b0 0x7fe8d014c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.007+0000 7fe8d66dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8d01610f0 0x7fe8d00b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.007+0000 7fe8d66dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d01616f0 0x7fe8d00b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.007+0000 7fe8d66dd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00cfce0 con 0x7fe8d01610f0 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.007+0000 7fe8d66dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00cfa00 con 0x7fe8d01616f0 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.007+0000 7fe8d66dd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00cf690 con 0x7fe8d00ce7b0 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.007+0000 7fe8cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8d01610f0 0x7fe8d00b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.007+0000 7fe8cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8d01610f0 0x7fe8d00b4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50344/0 (socket says 172.21.15.19:50344) 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.007+0000 7fe8cf7fe640 1 -- 172.21.15.19:0/4241699086 learned_addr learned my addr 172.21.15.19:0/4241699086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8d4c7a640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d01616f0 0x7fe8d00b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8cffff640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8d00ce7b0 0x7fe8d014c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8cf7fe640 1 -- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8d00ce7b0 msgr2=0x7fe8d014c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8cf7fe640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8d00ce7b0 0x7fe8d014c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8cf7fe640 1 -- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d01616f0 msgr2=0x7fe8d00b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8cf7fe640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d01616f0 0x7fe8d00b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8cf7fe640 1 -- 172.21.15.19:0/4241699086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d01f3cd0 con 0x7fe8d01610f0 2024-04-25T09:57:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8cf7fe640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8d01610f0 0x7fe8d00b4520 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c4d0 tx=0x7fe8c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8ccff9640 1 -- 172.21.15.19:0/4241699086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4018070 con 0x7fe8d01610f0 2024-04-25T09:57:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8ccff9640 1 -- 172.21.15.19:0/4241699086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c4007500 con 0x7fe8d01610f0 2024-04-25T09:57:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8d66dd640 1 -- 172.21.15.19:0/4241699086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d00b4e70 con 0x7fe8d01610f0 2024-04-25T09:57:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8ccff9640 1 -- 172.21.15.19:0/4241699086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4004a20 con 0x7fe8d01610f0 2024-04-25T09:57:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.008+0000 7fe8d66dd640 1 -- 172.21.15.19:0/4241699086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d00b5430 con 0x7fe8d01610f0 2024-04-25T09:57:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.009+0000 7fe8d66dd640 1 -- 172.21.15.19:0/4241699086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d0073a00 con 0x7fe8d01610f0 2024-04-25T09:57:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.009+0000 7fe8ccff9640 1 -- 172.21.15.19:0/4241699086 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8c4004bc0 con 0x7fe8d01610f0 2024-04-25T09:57:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.010+0000 7fe8ccff9640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8bc06b4d0 0x7fe8bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.010+0000 7fe8ccff9640 1 -- 172.21.15.19:0/4241699086 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe8c40897c0 con 0x7fe8d01610f0 2024-04-25T09:57:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.010+0000 7fe8cffff640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8bc06b4d0 0x7fe8bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.011+0000 7fe8cffff640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8bc06b4d0 0x7fe8bc06d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0004ea0 tx=0x7fe8c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.014+0000 7fe8ccff9640 1 -- 172.21.15.19:0/4241699086 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8c4058300 con 0x7fe8d01610f0 2024-04-25T09:57:11.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.194+0000 7fe8d66dd640 1 -- 172.21.15.19:0/4241699086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8d0071d00 con 0x7fe8d01610f0 2024-04-25T09:57:11.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.194+0000 7fe8ccff9640 1 -- 172.21.15.19:0/4241699086 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8c4057c00 con 0x7fe8d01610f0 2024-04-25T09:57:11.198 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:57:11.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 -- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8bc06b4d0 msgr2=0x7fe8bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8bc06b4d0 0x7fe8bc06d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0004ea0 tx=0x7fe8c0009040 comp rx=0 tx=0).stop 2024-04-25T09:57:11.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 -- 172.21.15.19:0/4241699086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8d01610f0 msgr2=0x7fe8d00b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8d01610f0 0x7fe8d00b4520 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c4d0 tx=0x7fe8c400c990 comp rx=0 tx=0).stop 2024-04-25T09:57:11.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 -- 172.21.15.19:0/4241699086 shutdown_connections 2024-04-25T09:57:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe8bc06b4d0 0x7fe8bc06d990 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8d01616f0 0x7fe8d00b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8d01610f0 0x7fe8d00b4520 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 --2- 172.21.15.19:0/4241699086 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8d00ce7b0 0x7fe8d014c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 -- 172.21.15.19:0/4241699086 >> 172.21.15.19:0/4241699086 conn(0x7fe8d006f500 msgr2=0x7fe8d00c7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 -- 172.21.15.19:0/4241699086 shutdown_connections 2024-04-25T09:57:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.196+0000 7fe8d66dd640 1 -- 172.21.15.19:0/4241699086 wait complete. 2024-04-25T09:57:11.210 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:57:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.310+0000 7f8c89a6f640 1 Processor -- start 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.311+0000 7f8c89a6f640 1 -- start start 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.311+0000 7f8c89a6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 0x7f8c840c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.311+0000 7f8c89a6f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c840c6940 con 0x7f8c840cd6f0 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.311+0000 7f8c89a6f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8415add0 con 0x7f8c840cdbf0 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.311+0000 7f8c89a6f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840b0020 con 0x7f8c8415a9f0 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.311+0000 7f8c83fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8c840cd6f0 0x7f8c840cba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37546/0 (socket says 172.21.15.19:37546) 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.311+0000 7f8c83fff640 1 -- 172.21.15.19:0/843050058 learned_addr learned my addr 172.21.15.19:0/843050058 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.312+0000 7f8c837fe640 1 --2- 172.21.15.19:0/843050058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 0x7f8c840c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.312+0000 7f8c827fc640 1 -- 172.21.15.19:0/843050058 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429419705 0 0) 0x7f8c7c001490 con 0x7f8c840cd6f0 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.312+0000 7f8c827fc640 1 -- 172.21.15.19:0/843050058 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c6c005730 con 0x7f8c840cd6f0 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.312+0000 7f8c827fc640 1 -- 172.21.15.19:0/843050058 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090776912 0 0) 0x7f8c8415add0 con 0x7f8c840cdbf0 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.312+0000 7f8c827fc640 1 -- 172.21.15.19:0/843050058 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c7c001490 con 0x7f8c840cdbf0 2024-04-25T09:57:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.312+0000 7f8c827fc640 1 -- 172.21.15.19:0/843050058 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3044887714 0 0) 0x7f8c6c005730 con 0x7f8c840cd6f0 2024-04-25T09:57:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.312+0000 7f8c837fe640 1 -- 172.21.15.19:0/843050058 >> v1:172.21.15.112:6789/0 conn(0x7f8c840cdbf0 legacy=0x7f8c840ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.312+0000 7f8c837fe640 1 -- 172.21.15.19:0/843050058 >> v1:172.21.15.19:6789/0 conn(0x7f8c840cd6f0 legacy=0x7f8c840cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.312+0000 7f8c837fe640 1 -- 172.21.15.19:0/843050058 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840c6660 con 0x7f8c8415a9f0 2024-04-25T09:57:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.313+0000 7f8c837fe640 1 --2- 172.21.15.19:0/843050058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 0x7f8c840c6010 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8c740097d0 tx=0x7f8c7402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=452e1e1098b86fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.313+0000 7f8c827fc640 1 -- 172.21.15.19:0/843050058 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74037070 con 0x7f8c8415a9f0 2024-04-25T09:57:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.313+0000 7f8c827fc640 1 -- 172.21.15.19:0/843050058 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7402ac30 con 0x7f8c8415a9f0 2024-04-25T09:57:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.313+0000 7f8c827fc640 1 -- 172.21.15.19:0/843050058 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c001bc0 con 0x7f8c8415a9f0 2024-04-25T09:57:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.313+0000 7f8c89a6f640 1 -- 172.21.15.19:0/843050058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 msgr2=0x7f8c840c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.313+0000 7f8c89a6f640 1 --2- 172.21.15.19:0/843050058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 0x7f8c840c6010 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8c740097d0 tx=0x7f8c7402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.314+0000 7f8c89a6f640 1 -- 172.21.15.19:0/843050058 shutdown_connections 2024-04-25T09:57:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.314+0000 7f8c89a6f640 1 --2- 172.21.15.19:0/843050058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 0x7f8c840c6010 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.314+0000 7f8c89a6f640 1 -- 172.21.15.19:0/843050058 >> 172.21.15.19:0/843050058 conn(0x7f8c840b2390 msgr2=0x7f8c840b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.314+0000 7f8c89a6f640 1 -- 172.21.15.19:0/843050058 shutdown_connections 2024-04-25T09:57:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.314+0000 7f8c89a6f640 1 -- 172.21.15.19:0/843050058 wait complete. 2024-04-25T09:57:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.315+0000 7f8c89a6f640 1 Processor -- start 2024-04-25T09:57:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.315+0000 7f8c89a6f640 1 -- start start 2024-04-25T09:57:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c89a6f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c840cd6f0 0x7f8c840ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c89a6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c840cdbf0 0x7f8c840b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c89a6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 0x7f8c840b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c89a6f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840cb230 con 0x7f8c840cd6f0 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c89a6f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840cb5a0 con 0x7f8c840cdbf0 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c89a6f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840ce070 con 0x7f8c8415a9f0 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c837fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c840cd6f0 0x7f8c840ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c837fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c840cd6f0 0x7f8c840ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50350/0 (socket says 172.21.15.19:50350) 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c837fe640 1 -- 172.21.15.19:0/3033093437 learned_addr learned my addr 172.21.15.19:0/3033093437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c837fe640 1 -- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 msgr2=0x7f8c840b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c83fff640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 0x7f8c840b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c82ffd640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c840cdbf0 0x7f8c840b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c837fe640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 0x7f8c840b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c837fe640 1 -- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c840cdbf0 msgr2=0x7f8c840b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c837fe640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c840cdbf0 0x7f8c840b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c837fe640 1 -- 172.21.15.19:0/3033093437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840c7c40 con 0x7f8c840cd6f0 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c82ffd640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c840cdbf0 0x7f8c840b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c83fff640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 0x7f8c840b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c837fe640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c840cd6f0 0x7f8c840ca050 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8c74035040 tx=0x7f8c740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c6bfff640 1 -- 172.21.15.19:0/3033093437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74041020 con 0x7f8c840cd6f0 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c6bfff640 1 -- 172.21.15.19:0/3033093437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7403d640 con 0x7f8c840cd6f0 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c6bfff640 1 -- 172.21.15.19:0/3033093437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74037040 con 0x7f8c840cd6f0 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.316+0000 7f8c89a6f640 1 -- 172.21.15.19:0/3033093437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c840b6260 con 0x7f8c840cd6f0 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.317+0000 7f8c89a6f640 1 -- 172.21.15.19:0/3033093437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c840b5f80 con 0x7f8c840cd6f0 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.318+0000 7f8c89a6f640 1 -- 172.21.15.19:0/3033093437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c84073a00 con 0x7f8c840cd6f0 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.318+0000 7f8c6bfff640 1 -- 172.21.15.19:0/3033093437 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c74044020 con 0x7f8c840cd6f0 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.318+0000 7f8c6bfff640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.318+0000 7f8c6bfff640 1 -- 172.21.15.19:0/3033093437 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f8c740aa9e0 con 0x7f8c840cd6f0 2024-04-25T09:57:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.319+0000 7f8c82ffd640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.319+0000 7f8c82ffd640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8c840af8a0 tx=0x7f8c78009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.322+0000 7f8c6bfff640 1 -- 172.21.15.19:0/3033093437 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c7404b030 con 0x7f8c840cd6f0 2024-04-25T09:57:11.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.503+0000 7f8c89a6f640 1 -- 172.21.15.19:0/3033093437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c840b1f30 con 0x7f8c840cd6f0 2024-04-25T09:57:11.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.504+0000 7f8c6bfff640 1 -- 172.21.15.19:0/3033093437 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c840c7c40 con 0x7f8c840cd6f0 2024-04-25T09:57:11.507 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:57:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 -- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c7006b4d0 msgr2=0x7f8c7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8c840af8a0 tx=0x7f8c78009040 comp rx=0 tx=0).stop 2024-04-25T09:57:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 -- 172.21.15.19:0/3033093437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c840cd6f0 msgr2=0x7f8c840ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c840cd6f0 0x7f8c840ca050 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8c74035040 tx=0x7f8c740046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 -- 172.21.15.19:0/3033093437 shutdown_connections 2024-04-25T09:57:11.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c8415a9f0 0x7f8c840b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c840cdbf0 0x7f8c840b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 --2- 172.21.15.19:0/3033093437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c840cd6f0 0x7f8c840ca050 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 -- 172.21.15.19:0/3033093437 >> 172.21.15.19:0/3033093437 conn(0x7f8c840b2390 msgr2=0x7f8c840cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 -- 172.21.15.19:0/3033093437 shutdown_connections 2024-04-25T09:57:11.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.506+0000 7f8c89a6f640 1 -- 172.21.15.19:0/3033093437 wait complete. 2024-04-25T09:57:11.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:57:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.621+0000 7f1800122640 1 Processor -- start 2024-04-25T09:57:11.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.621+0000 7f1800122640 1 -- start start 2024-04-25T09:57:11.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.622+0000 7f1800122640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f17f815bb50 0x7f17f80c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.622+0000 7f1800122640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17f80c70d0 0x7f17f80cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.622+0000 7f1800122640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17f80cfb40 0x7f17f8070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.622+0000 7f1800122640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f80c8680 con 0x7f17f80cfb40 2024-04-25T09:57:11.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.622+0000 7f1800122640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f80c75e0 con 0x7f17f815bb50 2024-04-25T09:57:11.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.622+0000 7f1800122640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f80c78c0 con 0x7f17f80c70d0 2024-04-25T09:57:11.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.622+0000 7f17fe6bf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17f80cfb40 0x7f17f8070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.622+0000 7f17fe6bf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17f80cfb40 0x7f17f8070cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50362/0 (socket says 172.21.15.19:50362) 2024-04-25T09:57:11.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.623+0000 7f17fe6bf640 1 -- 172.21.15.19:0/239644598 learned_addr learned my addr 172.21.15.19:0/239644598 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:11.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.623+0000 7f17fd6bd640 1 --2- 172.21.15.19:0/239644598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17f80c70d0 0x7f17f80cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.623+0000 7f17fdebe640 1 --2- 172.21.15.19:0/239644598 >> v2:172.21.15.112:3300/0 conn(0x7f17f815bb50 0x7f17f80c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.623+0000 7f17fe6bf640 1 -- 172.21.15.19:0/239644598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17f80c70d0 msgr2=0x7f17f80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:11.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.623+0000 7f17fe6bf640 1 --2- 172.21.15.19:0/239644598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17f80c70d0 0x7f17f80cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.623+0000 7f17fe6bf640 1 -- 172.21.15.19:0/239644598 >> v2:172.21.15.112:3300/0 conn(0x7f17f815bb50 msgr2=0x7f17f80c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:11.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.623+0000 7f17fe6bf640 1 --2- 172.21.15.19:0/239644598 >> v2:172.21.15.112:3300/0 conn(0x7f17f815bb50 0x7f17f80c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.623+0000 7f17fe6bf640 1 -- 172.21.15.19:0/239644598 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f80c7ba0 con 0x7f17f80cfb40 2024-04-25T09:57:11.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f17fe6bf640 1 --2- 172.21.15.19:0/239644598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f80cfb40 0x7f17f8070cf0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f17f8070ab0 tx=0x7f17e802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d848671b926061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f17fcebc640 1 -- 172.21.15.19:0/239644598 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8036070 con 0x7f17f80cfb40 2024-04-25T09:57:11.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f17fcebc640 1 -- 172.21.15.19:0/239644598 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e8031600 con 0x7f17f80cfb40 2024-04-25T09:57:11.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f17fcebc640 1 -- 172.21.15.19:0/239644598 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e80319d0 con 0x7f17f80cfb40 2024-04-25T09:57:11.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f1800122640 1 -- 172.21.15.19:0/239644598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f80cfb40 msgr2=0x7f17f8070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f1800122640 1 --2- 172.21.15.19:0/239644598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f80cfb40 0x7f17f8070cf0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f17f8070ab0 tx=0x7f17e802a730 comp rx=0 tx=0).stop 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f1800122640 1 -- 172.21.15.19:0/239644598 shutdown_connections 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f1800122640 1 --2- 172.21.15.19:0/239644598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17f80c70d0 0x7f17f80cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f1800122640 1 --2- 172.21.15.19:0/239644598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f80cfb40 0x7f17f8070cf0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f1800122640 1 --2- 172.21.15.19:0/239644598 >> v2:172.21.15.112:3300/0 conn(0x7f17f815bb50 0x7f17f80c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f1800122640 1 -- 172.21.15.19:0/239644598 >> 172.21.15.19:0/239644598 conn(0x7f17f806f500 msgr2=0x7f17f806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f1800122640 1 -- 172.21.15.19:0/239644598 shutdown_connections 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.624+0000 7f1800122640 1 -- 172.21.15.19:0/239644598 wait complete. 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.625+0000 7f1800122640 1 Processor -- start 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.626+0000 7f1800122640 1 -- start start 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.626+0000 7f1800122640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17f80c70d0 0x7f17f814ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.626+0000 7f1800122640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f80cfb40 0x7f17f815e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.626+0000 7f1800122640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17f815bb50 0x7f17f80ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.626+0000 7f1800122640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f815f6d0 con 0x7f17f80cfb40 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.626+0000 7f1800122640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f80c8160 con 0x7f17f80c70d0 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.626+0000 7f1800122640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f815f850 con 0x7f17f815bb50 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.626+0000 7f17fd6bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f80cfb40 0x7f17f815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.626+0000 7f17fd6bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f80cfb40 0x7f17f815e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50374/0 (socket says 172.21.15.19:50374) 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.626+0000 7f17fd6bd640 1 -- 172.21.15.19:0/371253910 learned_addr learned my addr 172.21.15.19:0/371253910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f17fdebe640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17f80c70d0 0x7f17f814ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f17fe6bf640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17f815bb50 0x7f17f80ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f17fd6bd640 1 -- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17f815bb50 msgr2=0x7f17f80ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f17fd6bd640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17f815bb50 0x7f17f80ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f17fd6bd640 1 -- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17f80c70d0 msgr2=0x7f17f814ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f17fd6bd640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17f80c70d0 0x7f17f814ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f17fd6bd640 1 -- 172.21.15.19:0/371253910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f814d340 con 0x7f17f80cfb40 2024-04-25T09:57:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f17fd6bd640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f80cfb40 0x7f17f815e860 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00e870 tx=0x7f17ec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f17f67fc640 1 -- 172.21.15.19:0/371253910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec017070 con 0x7f17f80cfb40 2024-04-25T09:57:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f17f67fc640 1 -- 172.21.15.19:0/371253910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec00ee60 con 0x7f17f80cfb40 2024-04-25T09:57:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f17f67fc640 1 -- 172.21.15.19:0/371253910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00cb60 con 0x7f17f80cfb40 2024-04-25T09:57:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f1800122640 1 -- 172.21.15.19:0/371253910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f815d4e0 con 0x7f17f80cfb40 2024-04-25T09:57:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.627+0000 7f1800122640 1 -- 172.21.15.19:0/371253910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f815dd80 con 0x7f17f80cfb40 2024-04-25T09:57:11.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.628+0000 7f1800122640 1 -- 172.21.15.19:0/371253910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f8073a00 con 0x7f17f80cfb40 2024-04-25T09:57:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.629+0000 7f17f67fc640 1 -- 172.21.15.19:0/371253910 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17ec01a070 con 0x7f17f80cfb40 2024-04-25T09:57:11.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.629+0000 7f17f67fc640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.629+0000 7f17f67fc640 1 -- 172.21.15.19:0/371253910 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f17ec08a2e0 con 0x7f17f80cfb40 2024-04-25T09:57:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.629+0000 7f17fdebe640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.630+0000 7f17fdebe640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 0x7f17e406d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f17f8153ff0 tx=0x7f17f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.633+0000 7f17f67fc640 1 -- 172.21.15.19:0/371253910 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17f814d340 con 0x7f17f80cfb40 2024-04-25T09:57:11.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.811+0000 7f1800122640 1 -- 172.21.15.19:0/371253910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f17f80b0c90 con 0x7f17f80cfb40 2024-04-25T09:57:11.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.812+0000 7f17f67fc640 1 -- 172.21.15.19:0/371253910 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f17ec058e90 con 0x7f17f80cfb40 2024-04-25T09:57:11.815 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:57:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.813+0000 7f1800122640 1 -- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 msgr2=0x7f17e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.813+0000 7f1800122640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 0x7f17e406d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f17f8153ff0 tx=0x7f17f0008040 comp rx=0 tx=0).stop 2024-04-25T09:57:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.813+0000 7f1800122640 1 -- 172.21.15.19:0/371253910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f80cfb40 msgr2=0x7f17f815e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.813+0000 7f1800122640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f80cfb40 0x7f17f815e860 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00e870 tx=0x7f17ec00ed30 comp rx=0 tx=0).stop 2024-04-25T09:57:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.813+0000 7f1800122640 1 -- 172.21.15.19:0/371253910 shutdown_connections 2024-04-25T09:57:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.813+0000 7f1800122640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.813+0000 7f1800122640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17f815bb50 0x7f17f80ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.813+0000 7f1800122640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f80cfb40 0x7f17f815e860 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.813+0000 7f1800122640 1 --2- 172.21.15.19:0/371253910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17f80c70d0 0x7f17f814ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.814+0000 7f1800122640 1 -- 172.21.15.19:0/371253910 >> 172.21.15.19:0/371253910 conn(0x7f17f806f500 msgr2=0x7f17f8070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.814+0000 7f1800122640 1 -- 172.21.15.19:0/371253910 shutdown_connections 2024-04-25T09:57:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.814+0000 7f1800122640 1 -- 172.21.15.19:0/371253910 wait complete. 2024-04-25T09:57:11.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:57:11.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.927+0000 7fa9806fc640 1 Processor -- start 2024-04-25T09:57:11.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.928+0000 7fa9806fc640 1 -- start start 2024-04-25T09:57:11.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.928+0000 7fa9806fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa97815dc20 0x7fa9780ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.928+0000 7fa9806fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9780d32f0 0x7fa9780cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.928+0000 7fa9806fc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97815e000 con 0x7fa978161710 2024-04-25T09:57:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.928+0000 7fa9806fc640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9780d1de0 con 0x7fa97815dc20 2024-04-25T09:57:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.928+0000 7fa9806fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa978161c60 con 0x7fa9780d32f0 2024-04-25T09:57:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.929+0000 7fa97e498640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa97815dc20 0x7fa9780ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.929+0000 7fa97e498640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa97815dc20 0x7fa9780ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50384/0 (socket says 172.21.15.19:50384) 2024-04-25T09:57:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.929+0000 7fa97e498640 1 -- 172.21.15.19:0/1230463088 learned_addr learned my addr 172.21.15.19:0/1230463088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.929+0000 7fa97ec99640 1 --2- 172.21.15.19:0/1230463088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9780d32f0 0x7fa9780cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.929+0000 7fa97e498640 1 -- 172.21.15.19:0/1230463088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9780d32f0 msgr2=0x7fa9780cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.929+0000 7fa97e498640 1 --2- 172.21.15.19:0/1230463088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9780d32f0 0x7fa9780cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.929+0000 7fa97e498640 1 -- 172.21.15.19:0/1230463088 >> v1:172.21.15.112:6789/0 conn(0x7fa978161710 legacy=0x7fa97806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.929+0000 7fa97e498640 1 -- 172.21.15.19:0/1230463088 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa978161f40 con 0x7fa97815dc20 2024-04-25T09:57:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.930+0000 7fa97e498640 1 --2- 172.21.15.19:0/1230463088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa97815dc20 0x7fa9780ce670 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fa968009870 tx=0x7fa96802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bec9bd4062b09f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.930+0000 7fa97d496640 1 -- 172.21.15.19:0/1230463088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968037070 con 0x7fa97815dc20 2024-04-25T09:57:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.930+0000 7fa97d496640 1 -- 172.21.15.19:0/1230463088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa968032440 con 0x7fa97815dc20 2024-04-25T09:57:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.930+0000 7fa97d496640 1 -- 172.21.15.19:0/1230463088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968032810 con 0x7fa97815dc20 2024-04-25T09:57:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.930+0000 7fa9806fc640 1 -- 172.21.15.19:0/1230463088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa97815dc20 msgr2=0x7fa9780ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.930+0000 7fa9806fc640 1 --2- 172.21.15.19:0/1230463088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa97815dc20 0x7fa9780ce670 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fa968009870 tx=0x7fa96802a270 comp rx=0 tx=0).stop 2024-04-25T09:57:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.931+0000 7fa9806fc640 1 -- 172.21.15.19:0/1230463088 shutdown_connections 2024-04-25T09:57:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.931+0000 7fa9806fc640 1 --2- 172.21.15.19:0/1230463088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9780d32f0 0x7fa9780cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.931+0000 7fa9806fc640 1 --2- 172.21.15.19:0/1230463088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa97815dc20 0x7fa9780ce670 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.931+0000 7fa9806fc640 1 -- 172.21.15.19:0/1230463088 >> 172.21.15.19:0/1230463088 conn(0x7fa9780d0420 msgr2=0x7fa9780d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.931+0000 7fa9806fc640 1 -- 172.21.15.19:0/1230463088 shutdown_connections 2024-04-25T09:57:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.931+0000 7fa9806fc640 1 -- 172.21.15.19:0/1230463088 wait complete. 2024-04-25T09:57:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.932+0000 7fa9806fc640 1 Processor -- start 2024-04-25T09:57:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.932+0000 7fa9806fc640 1 -- start start 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa9806fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9780d32f0 0x7fa97815c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa9806fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa97815dc20 0x7fa9781e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa9806fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa978161710 0x7fa9781e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa9806fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780b5e50 con 0x7fa97815dc20 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa9806fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa978162500 con 0x7fa978161710 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa9806fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780b5fd0 con 0x7fa9780d32f0 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa97dc97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa97815dc20 0x7fa9781e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa97dc97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa97815dc20 0x7fa9781e82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50400/0 (socket says 172.21.15.19:50400) 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa97dc97640 1 -- 172.21.15.19:0/1076495557 learned_addr learned my addr 172.21.15.19:0/1076495557 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa97e498640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9780d32f0 0x7fa97815c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa97ec99640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa978161710 0x7fa9781e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa97dc97640 1 -- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9780d32f0 msgr2=0x7fa97815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa97dc97640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9780d32f0 0x7fa97815c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa97dc97640 1 -- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa978161710 msgr2=0x7fa9781e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa97dc97640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa978161710 0x7fa9781e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa97dc97640 1 -- 172.21.15.19:0/1076495557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9781e90c0 con 0x7fa97815dc20 2024-04-25T09:57:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.933+0000 7fa97dc97640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa97815dc20 0x7fa9781e82d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fa97400c3c0 tx=0x7fa97400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.934+0000 7fa96effd640 1 -- 172.21.15.19:0/1076495557 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9740052d0 con 0x7fa97815dc20 2024-04-25T09:57:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.934+0000 7fa96effd640 1 -- 172.21.15.19:0/1076495557 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa974005490 con 0x7fa97815dc20 2024-04-25T09:57:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.934+0000 7fa96effd640 1 -- 172.21.15.19:0/1076495557 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974007260 con 0x7fa97815dc20 2024-04-25T09:57:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.934+0000 7fa9806fc640 1 -- 172.21.15.19:0/1076495557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9781f4070 con 0x7fa97815dc20 2024-04-25T09:57:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.934+0000 7fa9806fc640 1 -- 172.21.15.19:0/1076495557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97815cd90 con 0x7fa97815dc20 2024-04-25T09:57:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.935+0000 7fa9806fc640 1 -- 172.21.15.19:0/1076495557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa978073a00 con 0x7fa97815dc20 2024-04-25T09:57:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.935+0000 7fa96effd640 1 -- 172.21.15.19:0/1076495557 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa974007400 con 0x7fa97815dc20 2024-04-25T09:57:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.935+0000 7fa96effd640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa95c06b4d0 0x7fa95c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.936+0000 7fa96effd640 1 -- 172.21.15.19:0/1076495557 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fa974089850 con 0x7fa97815dc20 2024-04-25T09:57:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.936+0000 7fa97e498640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa95c06b4d0 0x7fa95c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.936+0000 7fa97e498640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa95c06b4d0 0x7fa95c06d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa968034000 tx=0x7fa96802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:11.940+0000 7fa96effd640 1 -- 172.21.15.19:0/1076495557 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa974057e60 con 0x7fa97815dc20 2024-04-25T09:57:12.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.133+0000 7fa9806fc640 1 -- 172.21.15.19:0/1076495557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa9780d0cc0 con 0x7fa97815dc20 2024-04-25T09:57:12.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.133+0000 7fa96effd640 1 -- 172.21.15.19:0/1076495557 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v70) v1 ==== 80+0+242 (secure 0 0 0) 0x7fa97405ce20 con 0x7fa97815dc20 2024-04-25T09:57:12.137 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3520485752 2024-04-26T09:56:54.515283+0000 2024-04-25T09:57:12.137 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6833/3520485752 2024-04-26T09:56:54.515283+0000 2024-04-25T09:57:12.137 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/591570980 2024-04-26T09:56:53.512369+0000 2024-04-25T09:57:12.137 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6832/591570980 2024-04-26T09:56:53.512369+0000 2024-04-25T09:57:12.137 INFO:teuthology.orchestra.run.smithi019.stderr:listed 4 entries 2024-04-25T09:57:12.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 -- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa95c06b4d0 msgr2=0x7fa95c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa95c06b4d0 0x7fa95c06d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa968034000 tx=0x7fa96802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 -- 172.21.15.19:0/1076495557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa97815dc20 msgr2=0x7fa9781e82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa97815dc20 0x7fa9781e82d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fa97400c3c0 tx=0x7fa97400c880 comp rx=0 tx=0).stop 2024-04-25T09:57:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 -- 172.21.15.19:0/1076495557 shutdown_connections 2024-04-25T09:57:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa95c06b4d0 0x7fa95c06d990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa978161710 0x7fa9781e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa97815dc20 0x7fa9781e82d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 --2- 172.21.15.19:0/1076495557 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9780d32f0 0x7fa97815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 -- 172.21.15.19:0/1076495557 >> 172.21.15.19:0/1076495557 conn(0x7fa9780d0420 msgr2=0x7fa9780b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 -- 172.21.15.19:0/1076495557 shutdown_connections 2024-04-25T09:57:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.135+0000 7fa9806fc640 1 -- 172.21.15.19:0/1076495557 wait complete. 2024-04-25T09:57:12.149 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:57:12.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:12.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:12.179 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:12.179 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:57:12.235 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:57:12.235 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:12.254 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:12.254 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:57:12.254 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:12.290 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:57:12.290 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:57:12.290 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:12.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:12.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:12.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.428+0000 7fe2d00b5640 1 Processor -- start 2024-04-25T09:57:12.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.428+0000 7fe2d00b5640 1 -- start start 2024-04-25T09:57:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.429+0000 7fe2d00b5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe2c815dbe0 0x7fe2c80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.429+0000 7fe2d00b5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe2c80b2cb0 0x7fe2c80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.429+0000 7fe2d00b5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c80b97a0 con 0x7fe2c80b92a0 2024-04-25T09:57:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.429+0000 7fe2d00b5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c815dfc0 con 0x7fe2c80b2cb0 2024-04-25T09:57:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.429+0000 7fe2d00b5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c80d1a00 con 0x7fe2c815dbe0 2024-04-25T09:57:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.429+0000 7fe2cd650640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe2c80b2cb0 0x7fe2c80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.429+0000 7fe2ce652640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe2c80b92a0 0x7fe2c80b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37562/0 (socket says 172.21.15.19:37562) 2024-04-25T09:57:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.429+0000 7fe2ce652640 1 -- 172.21.15.19:0/906253712 learned_addr learned my addr 172.21.15.19:0/906253712 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.429+0000 7fe2cd650640 1 --2- 172.21.15.19:0/906253712 >> v2:172.21.15.19:3300/0 conn(0x7fe2c80b2cb0 0x7fe2c80d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50408/0 (socket says 172.21.15.19:50408) 2024-04-25T09:57:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.430+0000 7fe2cde51640 1 --2- 172.21.15.19:0/906253712 >> v2:172.21.15.112:3300/0 conn(0x7fe2c815dbe0 0x7fe2c80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.430+0000 7fe2cd650640 1 -- 172.21.15.19:0/906253712 >> v2:172.21.15.112:3300/0 conn(0x7fe2c815dbe0 msgr2=0x7fe2c80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.430+0000 7fe2cd650640 1 --2- 172.21.15.19:0/906253712 >> v2:172.21.15.112:3300/0 conn(0x7fe2c815dbe0 0x7fe2c80d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.430+0000 7fe2cd650640 1 -- 172.21.15.19:0/906253712 >> v1:172.21.15.19:6789/0 conn(0x7fe2c80b92a0 legacy=0x7fe2c80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.430+0000 7fe2cce4f640 1 -- 172.21.15.19:0/906253712 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045862208 0 0) 0x7fe2c80b97a0 con 0x7fe2c80b92a0 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.430+0000 7fe2cd650640 1 -- 172.21.15.19:0/906253712 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80d1c50 con 0x7fe2c80b2cb0 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2cd650640 1 --2- 172.21.15.19:0/906253712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c80b2cb0 0x7fe2c80d0e50 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe2c806c140 tx=0x7fe2bc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8ca000e0892007 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2cce4f640 1 -- 172.21.15.19:0/906253712 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc037070 con 0x7fe2c80b2cb0 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2cce4f640 1 -- 172.21.15.19:0/906253712 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2bc032440 con 0x7fe2c80b2cb0 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2cce4f640 1 -- 172.21.15.19:0/906253712 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc02ae60 con 0x7fe2c80b2cb0 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2d00b5640 1 -- 172.21.15.19:0/906253712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c80b2cb0 msgr2=0x7fe2c80d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2d00b5640 1 --2- 172.21.15.19:0/906253712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c80b2cb0 0x7fe2c80d0e50 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe2c806c140 tx=0x7fe2bc02a110 comp rx=0 tx=0).stop 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2d00b5640 1 -- 172.21.15.19:0/906253712 shutdown_connections 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2d00b5640 1 --2- 172.21.15.19:0/906253712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c80b2cb0 0x7fe2c80d0e50 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2d00b5640 1 --2- 172.21.15.19:0/906253712 >> v2:172.21.15.112:3300/0 conn(0x7fe2c815dbe0 0x7fe2c80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2d00b5640 1 -- 172.21.15.19:0/906253712 >> 172.21.15.19:0/906253712 conn(0x7fe2c8061a20 msgr2=0x7fe2c8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2d00b5640 1 -- 172.21.15.19:0/906253712 shutdown_connections 2024-04-25T09:57:12.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.431+0000 7fe2d00b5640 1 -- 172.21.15.19:0/906253712 wait complete. 2024-04-25T09:57:12.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.433+0000 7fe2d00b5640 1 Processor -- start 2024-04-25T09:57:12.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.433+0000 7fe2d00b5640 1 -- start start 2024-04-25T09:57:12.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.433+0000 7fe2d00b5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2c80b2cb0 0x7fe2c8160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.433+0000 7fe2d00b5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c80b92a0 0x7fe2c80b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.433+0000 7fe2d00b5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c815dbe0 0x7fe2c8151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.433+0000 7fe2d00b5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80b6ab0 con 0x7fe2c815dbe0 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.433+0000 7fe2d00b5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80d2210 con 0x7fe2c80b92a0 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.433+0000 7fe2d00b5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80b6c30 con 0x7fe2c80b2cb0 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.433+0000 7fe2ce652640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c815dbe0 0x7fe2c8151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.433+0000 7fe2ce652640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c815dbe0 0x7fe2c8151a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50412/0 (socket says 172.21.15.19:50412) 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.433+0000 7fe2ce652640 1 -- 172.21.15.19:0/3676760491 learned_addr learned my addr 172.21.15.19:0/3676760491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2cd650640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c80b92a0 0x7fe2c80b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2cde51640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2c80b2cb0 0x7fe2c8160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2ce652640 1 -- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2c80b2cb0 msgr2=0x7fe2c8160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2ce652640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2c80b2cb0 0x7fe2c8160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2ce652640 1 -- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c80b92a0 msgr2=0x7fe2c80b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2ce652640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c80b92a0 0x7fe2c80b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2ce652640 1 -- 172.21.15.19:0/3676760491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80b40d0 con 0x7fe2c815dbe0 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2ce652640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c815dbe0 0x7fe2c8151a60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800c7e0 tx=0x7fe2b800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2c67fc640 1 -- 172.21.15.19:0/3676760491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8018070 con 0x7fe2c815dbe0 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2c67fc640 1 -- 172.21.15.19:0/3676760491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b800a640 con 0x7fe2c815dbe0 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2d00b5640 1 -- 172.21.15.19:0/3676760491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c80b43b0 con 0x7fe2c815dbe0 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2c67fc640 1 -- 172.21.15.19:0/3676760491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b800aa10 con 0x7fe2c815dbe0 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.434+0000 7fe2d00b5640 1 -- 172.21.15.19:0/3676760491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c80cbc80 con 0x7fe2c815dbe0 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.435+0000 7fe2d00b5640 1 -- 172.21.15.19:0/3676760491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c8073990 con 0x7fe2c815dbe0 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.435+0000 7fe2c67fc640 1 -- 172.21.15.19:0/3676760491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2b801b070 con 0x7fe2c815dbe0 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.436+0000 7fe2c67fc640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2b406b4d0 0x7fe2b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.436+0000 7fe2c67fc640 1 -- 172.21.15.19:0/3676760491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe2b8089a50 con 0x7fe2c815dbe0 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.436+0000 7fe2cde51640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2b406b4d0 0x7fe2b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.437+0000 7fe2cde51640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2b406b4d0 0x7fe2b406d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009a30 tx=0x7fe2c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.440+0000 7fe2c67fc640 1 -- 172.21.15.19:0/3676760491 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2b80586b0 con 0x7fe2c815dbe0 2024-04-25T09:57:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.662+0000 7fe2d00b5640 1 -- 172.21.15.19:0/3676760491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2c80cba50 con 0x7fe2c815dbe0 2024-04-25T09:57:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.663+0000 7fe2c67fc640 1 -- 172.21.15.19:0/3676760491 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe2b8057fb0 con 0x7fe2c815dbe0 2024-04-25T09:57:12.666 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:12.666 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2433112551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2433112551},{"type":"v1","addr":"172.21.15.112:6837","nonce":2433112551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6295,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2458476655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2458476655},{"type":"v1","addr":"172.21.15.112:6835","nonce":2458476655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6867,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1321165720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1321165720},{"type":"v1","addr":"172.21.15.19:6835","nonce":1321165720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:56:38.802500+0000","modified":"2024-04-25T09:56:56.539131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6833/2482429280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2482429280},{"type":"v1","addr":"172.21.15.19:6833","nonce":2482429280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:57:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 43 2024-04-25T09:57:12.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 -- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2b406b4d0 msgr2=0x7fe2b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2b406b4d0 0x7fe2b406d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009a30 tx=0x7fe2c0008040 comp rx=0 tx=0).stop 2024-04-25T09:57:12.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 -- 172.21.15.19:0/3676760491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c815dbe0 msgr2=0x7fe2c8151a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c815dbe0 0x7fe2c8151a60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800c7e0 tx=0x7fe2b800cca0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 -- 172.21.15.19:0/3676760491 shutdown_connections 2024-04-25T09:57:12.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2b406b4d0 0x7fe2b406d990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2c815dbe0 0x7fe2c8151a60 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2c80b92a0 0x7fe2c80b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 --2- 172.21.15.19:0/3676760491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2c80b2cb0 0x7fe2c8160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 -- 172.21.15.19:0/3676760491 >> 172.21.15.19:0/3676760491 conn(0x7fe2c8061a20 msgr2=0x7fe2c815c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:12.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 -- 172.21.15.19:0/3676760491 shutdown_connections 2024-04-25T09:57:12.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.665+0000 7fe2d00b5640 1 -- 172.21.15.19:0/3676760491 wait complete. 2024-04-25T09:57:12.679 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:12.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.777+0000 7f777aa1b640 1 Processor -- start 2024-04-25T09:57:12.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.777+0000 7f777aa1b640 1 -- start start 2024-04-25T09:57:12.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.778+0000 7f777aa1b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f777415dd20 0x7f77740b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.778+0000 7f777aa1b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f777415e100 0x7f77740d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.778+0000 7f777aa1b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777415eac0 con 0x7f777415e590 2024-04-25T09:57:12.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.778+0000 7f777aa1b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77740d1c00 con 0x7f777415e100 2024-04-25T09:57:12.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.778+0000 7f777aa1b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777415eda0 con 0x7f777415dd20 2024-04-25T09:57:12.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.778+0000 7f77737fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f777415e100 0x7f77740d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.778+0000 7f77737fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f777415e100 0x7f77740d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50426/0 (socket says 172.21.15.19:50426) 2024-04-25T09:57:12.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.778+0000 7f77737fe640 1 -- 172.21.15.19:0/965386162 learned_addr learned my addr 172.21.15.19:0/965386162 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:12.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.778+0000 7f7773fff640 1 --2- 172.21.15.19:0/965386162 >> v2:172.21.15.112:3300/0 conn(0x7f777415dd20 0x7f77740b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.779+0000 7f77737fe640 1 -- 172.21.15.19:0/965386162 >> v2:172.21.15.112:3300/0 conn(0x7f777415dd20 msgr2=0x7f77740b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:12.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.779+0000 7f77737fe640 1 --2- 172.21.15.19:0/965386162 >> v2:172.21.15.112:3300/0 conn(0x7f777415dd20 0x7f77740b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.779+0000 7f77737fe640 1 -- 172.21.15.19:0/965386162 >> v1:172.21.15.112:6789/0 conn(0x7f777415e590 legacy=0x7f77740d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.779+0000 7f77737fe640 1 -- 172.21.15.19:0/965386162 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777415f080 con 0x7f777415e100 2024-04-25T09:57:12.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.779+0000 7f77737fe640 1 --2- 172.21.15.19:0/965386162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f777415e100 0x7f77740d0aa0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f77740b6520 tx=0x7f776402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc8ebe4acce35a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.779+0000 7f7772ffd640 1 -- 172.21.15.19:0/965386162 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764037070 con 0x7f777415e100 2024-04-25T09:57:12.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.779+0000 7f7772ffd640 1 -- 172.21.15.19:0/965386162 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7764032440 con 0x7f777415e100 2024-04-25T09:57:12.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.779+0000 7f7772ffd640 1 -- 172.21.15.19:0/965386162 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764032810 con 0x7f777415e100 2024-04-25T09:57:12.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.780+0000 7f777aa1b640 1 -- 172.21.15.19:0/965386162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f777415e100 msgr2=0x7f77740d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.780+0000 7f777aa1b640 1 --2- 172.21.15.19:0/965386162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f777415e100 0x7f77740d0aa0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f77740b6520 tx=0x7f776402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.780+0000 7f777aa1b640 1 -- 172.21.15.19:0/965386162 shutdown_connections 2024-04-25T09:57:12.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.780+0000 7f777aa1b640 1 --2- 172.21.15.19:0/965386162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f777415e100 0x7f77740d0aa0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.780+0000 7f777aa1b640 1 --2- 172.21.15.19:0/965386162 >> v2:172.21.15.112:3300/0 conn(0x7f777415dd20 0x7f77740b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.780+0000 7f777aa1b640 1 -- 172.21.15.19:0/965386162 >> 172.21.15.19:0/965386162 conn(0x7f7774061a20 msgr2=0x7f7774061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:12.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.780+0000 7f777aa1b640 1 -- 172.21.15.19:0/965386162 shutdown_connections 2024-04-25T09:57:12.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.780+0000 7f777aa1b640 1 -- 172.21.15.19:0/965386162 wait complete. 2024-04-25T09:57:12.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.781+0000 7f777aa1b640 1 Processor -- start 2024-04-25T09:57:12.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.781+0000 7f777aa1b640 1 -- start start 2024-04-25T09:57:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.782+0000 7f777aa1b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f777415dd20 0x7f77740c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.782+0000 7f777aa1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777415e100 0x7f7774155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.782+0000 7f777aa1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f777415e590 0x7f77740c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.782+0000 7f777aa1b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740d3500 con 0x7f777415e590 2024-04-25T09:57:12.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.782+0000 7f777aa1b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740d3380 con 0x7f777415e100 2024-04-25T09:57:12.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.782+0000 7f777aa1b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77740d3680 con 0x7f777415dd20 2024-04-25T09:57:12.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.782+0000 7f7778fb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f777415e590 0x7f77740c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.782+0000 7f7773fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f777415dd20 0x7f77740c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.782+0000 7f7778fb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f777415e590 0x7f77740c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50442/0 (socket says 172.21.15.19:50442) 2024-04-25T09:57:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.782+0000 7f7778fb8640 1 -- 172.21.15.19:0/2630734523 learned_addr learned my addr 172.21.15.19:0/2630734523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.782+0000 7f77737fe640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777415e100 0x7f7774155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.783+0000 7f7778fb8640 1 -- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f777415dd20 msgr2=0x7f77740c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.783+0000 7f7778fb8640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f777415dd20 0x7f77740c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.783+0000 7f7778fb8640 1 -- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777415e100 msgr2=0x7f7774155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.783+0000 7f7778fb8640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777415e100 0x7f7774155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.783+0000 7f7778fb8640 1 -- 172.21.15.19:0/2630734523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77740c8590 con 0x7f777415e590 2024-04-25T09:57:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.783+0000 7f7778fb8640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f777415e590 0x7f77740c8050 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c6e0 tx=0x7f776c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.783+0000 7f7770ff9640 1 -- 172.21.15.19:0/2630734523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c003b70 con 0x7f777415e590 2024-04-25T09:57:12.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.783+0000 7f7770ff9640 1 -- 172.21.15.19:0/2630734523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c003d30 con 0x7f777415e590 2024-04-25T09:57:12.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.783+0000 7f777aa1b640 1 -- 172.21.15.19:0/2630734523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77740c90b0 con 0x7f777415e590 2024-04-25T09:57:12.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.783+0000 7f7770ff9640 1 -- 172.21.15.19:0/2630734523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c005310 con 0x7f777415e590 2024-04-25T09:57:12.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.783+0000 7f777aa1b640 1 -- 172.21.15.19:0/2630734523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77740c8810 con 0x7f777415e590 2024-04-25T09:57:12.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.784+0000 7f777aa1b640 1 -- 172.21.15.19:0/2630734523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7774073990 con 0x7f777415e590 2024-04-25T09:57:12.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.784+0000 7f7770ff9640 1 -- 172.21.15.19:0/2630734523 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f776c0054b0 con 0x7f777415e590 2024-04-25T09:57:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.785+0000 7f7770ff9640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f775406b4d0 0x7f775406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.785+0000 7f7770ff9640 1 -- 172.21.15.19:0/2630734523 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f776c08a110 con 0x7f777415e590 2024-04-25T09:57:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.785+0000 7f7773fff640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f775406b4d0 0x7f775406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.786+0000 7f7773fff640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f775406b4d0 0x7f775406d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f775c009b40 tx=0x7f775c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.789+0000 7f7770ff9640 1 -- 172.21.15.19:0/2630734523 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f77740c8590 con 0x7f777415e590 2024-04-25T09:57:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.969+0000 7f777aa1b640 1 -- 172.21.15.19:0/2630734523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77740c72a0 con 0x7f777415e590 2024-04-25T09:57:12.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.970+0000 7f7770ff9640 1 -- 172.21.15.19:0/2630734523 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17321 (secure 0 0 0) 0x7f776c058720 con 0x7f777415e590 2024-04-25T09:57:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:12.974 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":70,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:56:54.515339+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:56:37.087339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:56:37.775183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3520485752":"2024-04-26T09:56:54.515283+0000","172.21.15.19:6833/3520485752":"2024-04-26T09:56:54.515283+0000","172.21.15.19:6834/591570980":"2024-04-26T09:56:53.512369+0000","172.21.15.19:6832/591570980":"2024-04-26T09:56:53.512369+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.972+0000 7f777aa1b640 1 -- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f775406b4d0 msgr2=0x7f775406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.972+0000 7f777aa1b640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f775406b4d0 0x7f775406d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f775c009b40 tx=0x7f775c009190 comp rx=0 tx=0).stop 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.972+0000 7f777aa1b640 1 -- 172.21.15.19:0/2630734523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f777415e590 msgr2=0x7f77740c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.972+0000 7f777aa1b640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f777415e590 0x7f77740c8050 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c6e0 tx=0x7f776c00cba0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.972+0000 7f777aa1b640 1 -- 172.21.15.19:0/2630734523 shutdown_connections 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.972+0000 7f777aa1b640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f775406b4d0 0x7f775406d990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.972+0000 7f777aa1b640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f777415e590 0x7f77740c8050 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.972+0000 7f777aa1b640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777415e100 0x7f7774155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.972+0000 7f777aa1b640 1 --2- 172.21.15.19:0/2630734523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f777415dd20 0x7f77740c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.972+0000 7f777aa1b640 1 -- 172.21.15.19:0/2630734523 >> 172.21.15.19:0/2630734523 conn(0x7f7774061a20 msgr2=0x7f77740d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.972+0000 7f777aa1b640 1 -- 172.21.15.19:0/2630734523 shutdown_connections 2024-04-25T09:57:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:12.973+0000 7f777aa1b640 1 -- 172.21.15.19:0/2630734523 wait complete. 2024-04-25T09:57:12.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:56:37.087339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:56:37.775183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:12.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:13.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.088+0000 7f16d7407640 1 Processor -- start 2024-04-25T09:57:13.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.088+0000 7f16d7407640 1 -- start start 2024-04-25T09:57:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d7407640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f16d015b7d0 0x7f16d00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d7407640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f16d00b5620 0x7f16d00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d7407640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d006c2d0 0x7f16d006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d7407640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d006cc80 con 0x7f16d015b7d0 2024-04-25T09:57:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d7407640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d006cf50 con 0x7f16d00b5620 2024-04-25T09:57:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d7407640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d00ce530 con 0x7f16d006c2d0 2024-04-25T09:57:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d51a3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f16d015b7d0 0x7f16d00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d51a3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f16d015b7d0 0x7f16d00cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50450/0 (socket says 172.21.15.19:50450) 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d51a3640 1 -- 172.21.15.19:0/3631995162 learned_addr learned my addr 172.21.15.19:0/3631995162 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d49a2640 1 --2- 172.21.15.19:0/3631995162 >> v2:172.21.15.112:3300/0 conn(0x7f16d00b5620 0x7f16d00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d51a3640 1 -- 172.21.15.19:0/3631995162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d006c2d0 msgr2=0x7f16d006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d59a4640 1 --2- 172.21.15.19:0/3631995162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d006c2d0 0x7f16d006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d51a3640 1 --2- 172.21.15.19:0/3631995162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d006c2d0 0x7f16d006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d51a3640 1 -- 172.21.15.19:0/3631995162 >> v2:172.21.15.112:3300/0 conn(0x7f16d00b5620 msgr2=0x7f16d00b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d51a3640 1 --2- 172.21.15.19:0/3631995162 >> v2:172.21.15.112:3300/0 conn(0x7f16d00b5620 0x7f16d00b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d51a3640 1 -- 172.21.15.19:0/3631995162 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d00ce810 con 0x7f16d015b7d0 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d49a2640 1 --2- 172.21.15.19:0/3631995162 >> v2:172.21.15.112:3300/0 conn(0x7f16d00b5620 0x7f16d00b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.089+0000 7f16d59a4640 1 --2- 172.21.15.19:0/3631995162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d006c2d0 0x7f16d006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.090+0000 7f16d51a3640 1 --2- 172.21.15.19:0/3631995162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d015b7d0 0x7f16d00cf010 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f16d00ce180 tx=0x7f16cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d395379487f45d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.090+0000 7f16c7fff640 1 -- 172.21.15.19:0/3631995162 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc037070 con 0x7f16d015b7d0 2024-04-25T09:57:13.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.090+0000 7f16c7fff640 1 -- 172.21.15.19:0/3631995162 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc032440 con 0x7f16d015b7d0 2024-04-25T09:57:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.090+0000 7f16c7fff640 1 -- 172.21.15.19:0/3631995162 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc032810 con 0x7f16d015b7d0 2024-04-25T09:57:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.090+0000 7f16d7407640 1 -- 172.21.15.19:0/3631995162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d015b7d0 msgr2=0x7f16d00cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.090+0000 7f16d7407640 1 --2- 172.21.15.19:0/3631995162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d015b7d0 0x7f16d00cf010 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f16d00ce180 tx=0x7f16cc02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.091+0000 7f16d7407640 1 -- 172.21.15.19:0/3631995162 shutdown_connections 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.091+0000 7f16d7407640 1 --2- 172.21.15.19:0/3631995162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d006c2d0 0x7f16d006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.091+0000 7f16d7407640 1 --2- 172.21.15.19:0/3631995162 >> v2:172.21.15.112:3300/0 conn(0x7f16d00b5620 0x7f16d00b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.091+0000 7f16d7407640 1 --2- 172.21.15.19:0/3631995162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d015b7d0 0x7f16d00cf010 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.091+0000 7f16d7407640 1 -- 172.21.15.19:0/3631995162 >> 172.21.15.19:0/3631995162 conn(0x7f16d00d0800 msgr2=0x7f16d00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.091+0000 7f16d7407640 1 -- 172.21.15.19:0/3631995162 shutdown_connections 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.091+0000 7f16d7407640 1 -- 172.21.15.19:0/3631995162 wait complete. 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.092+0000 7f16d7407640 1 Processor -- start 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.092+0000 7f16d7407640 1 -- start start 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.092+0000 7f16d7407640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d006c2d0 0x7f16d01559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.092+0000 7f16d7407640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d00b5620 0x7f16d00c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.092+0000 7f16d7407640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d015b7d0 0x7f16d00c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.092+0000 7f16d7407640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d00cf550 con 0x7f16d006c2d0 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.092+0000 7f16d7407640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d006d230 con 0x7f16d00b5620 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.092+0000 7f16d7407640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d00cf6d0 con 0x7f16d015b7d0 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d51a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d006c2d0 0x7f16d01559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d51a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d006c2d0 0x7f16d01559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50456/0 (socket says 172.21.15.19:50456) 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d51a3640 1 -- 172.21.15.19:0/1635513784 learned_addr learned my addr 172.21.15.19:0/1635513784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d49a2640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d00b5620 0x7f16d00c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d59a4640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d015b7d0 0x7f16d00c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d51a3640 1 -- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d015b7d0 msgr2=0x7f16d00c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d51a3640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d015b7d0 0x7f16d00c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d51a3640 1 -- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d00b5620 msgr2=0x7f16d00c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d51a3640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d00b5620 0x7f16d00c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d51a3640 1 -- 172.21.15.19:0/1635513784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d00c8700 con 0x7f16d006c2d0 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d59a4640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d015b7d0 0x7f16d00c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:13.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d51a3640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d006c2d0 0x7f16d01559e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f16cc02a220 tx=0x7f16cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16c5ffb640 1 -- 172.21.15.19:0/1635513784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc037040 con 0x7f16d006c2d0 2024-04-25T09:57:13.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16c5ffb640 1 -- 172.21.15.19:0/1635513784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc042070 con 0x7f16d006c2d0 2024-04-25T09:57:13.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16d7407640 1 -- 172.21.15.19:0/1635513784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d01f3190 con 0x7f16d006c2d0 2024-04-25T09:57:13.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.093+0000 7f16c5ffb640 1 -- 172.21.15.19:0/1635513784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc030bb0 con 0x7f16d006c2d0 2024-04-25T09:57:13.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.094+0000 7f16d7407640 1 -- 172.21.15.19:0/1635513784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d00caec0 con 0x7f16d006c2d0 2024-04-25T09:57:13.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.094+0000 7f16d7407640 1 -- 172.21.15.19:0/1635513784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d0073a00 con 0x7f16d006c2d0 2024-04-25T09:57:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.095+0000 7f16c5ffb640 1 -- 172.21.15.19:0/1635513784 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16cc047460 con 0x7f16d006c2d0 2024-04-25T09:57:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.095+0000 7f16c5ffb640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16b406b4d0 0x7f16b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.095+0000 7f16c5ffb640 1 -- 172.21.15.19:0/1635513784 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f16cc0ab0d0 con 0x7f16d006c2d0 2024-04-25T09:57:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.096+0000 7f16d49a2640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16b406b4d0 0x7f16b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.096+0000 7f16d49a2640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16b406b4d0 0x7f16b406d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f16d0155f20 tx=0x7f16c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.100+0000 7f16c5ffb640 1 -- 172.21.15.19:0/1635513784 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16cc045360 con 0x7f16d006c2d0 2024-04-25T09:57:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.279+0000 7f16d7407640 1 -- 172.21.15.19:0/1635513784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16d006c7c0 con 0x7f16d006c2d0 2024-04-25T09:57:13.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.280+0000 7f16c5ffb640 1 -- 172.21.15.19:0/1635513784 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16cc079ca0 con 0x7f16d006c2d0 2024-04-25T09:57:13.283 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:57:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 -- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16b406b4d0 msgr2=0x7f16b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16b406b4d0 0x7f16b406d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f16d0155f20 tx=0x7f16c0008040 comp rx=0 tx=0).stop 2024-04-25T09:57:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 -- 172.21.15.19:0/1635513784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d006c2d0 msgr2=0x7f16d01559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d006c2d0 0x7f16d01559e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f16cc02a220 tx=0x7f16cc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 -- 172.21.15.19:0/1635513784 shutdown_connections 2024-04-25T09:57:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16b406b4d0 0x7f16b406d990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16d015b7d0 0x7f16d00c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d00b5620 0x7f16d00c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 --2- 172.21.15.19:0/1635513784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d006c2d0 0x7f16d01559e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 -- 172.21.15.19:0/1635513784 >> 172.21.15.19:0/1635513784 conn(0x7f16d00d0800 msgr2=0x7f16d015ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 -- 172.21.15.19:0/1635513784 shutdown_connections 2024-04-25T09:57:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.282+0000 7f16d7407640 1 -- 172.21.15.19:0/1635513784 wait complete. 2024-04-25T09:57:13.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:57:13.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.396+0000 7f84cd3e2640 1 Processor -- start 2024-04-25T09:57:13.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.396+0000 7f84cd3e2640 1 -- start start 2024-04-25T09:57:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.397+0000 7f84cd3e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84c80cfa40 0x7f84c80cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.397+0000 7f84cd3e2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c80c75e0 con 0x7f84c80cf600 2024-04-25T09:57:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.397+0000 7f84cd3e2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c80c78c0 con 0x7f84c815bb50 2024-04-25T09:57:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.397+0000 7f84cd3e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80c8680 con 0x7f84c80cfa40 2024-04-25T09:57:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.397+0000 7f84c67fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f84c80cf600 0x7f84c80cbf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37572/0 (socket says 172.21.15.19:37572) 2024-04-25T09:57:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.397+0000 7f84c67fc640 1 -- 172.21.15.19:0/652078456 learned_addr learned my addr 172.21.15.19:0/652078456 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.397+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740204277 0 0) 0x7f84c80c75e0 con 0x7f84c80cf600 2024-04-25T09:57:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.397+0000 7f84c77fe640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84c80cfa40 0x7f84c80cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.397+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b4003520 con 0x7f84c80cf600 2024-04-25T09:57:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 174323388 0 0) 0x7f84b4003520 con 0x7f84c80cf600 2024-04-25T09:57:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84c80cfa40 msgr2=0x7f84c80cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84c80cfa40 0x7f84c80cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 >> v1:172.21.15.112:6789/0 conn(0x7f84c815bb50 legacy=0x7f84c80c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c80c7ba0 con 0x7f84c80cf600 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f84b0001670 con 0x7f84c80cf600 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 >> v1:172.21.15.19:6789/0 conn(0x7f84c80cf600 legacy=0x7f84c80cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b40039e0 0x7f84b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84b4004390 0x7f84b4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84c80cfa40 0x7f84b40050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80c8680 con 0x7f84b4004390 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b400b9c0 con 0x7f84b40039e0 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b400bbc0 con 0x7f84c80cfa40 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c67fc640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84b4004390 0x7f84b4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c6ffd640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b40039e0 0x7f84b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c77fe640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84c80cfa40 0x7f84b40050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c67fc640 1 -- 172.21.15.19:0/652078456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84c80cfa40 msgr2=0x7f84b40050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c67fc640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84c80cfa40 0x7f84b40050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c67fc640 1 -- 172.21.15.19:0/652078456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b40039e0 msgr2=0x7f84b4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c67fc640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b40039e0 0x7f84b4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.398+0000 7f84c67fc640 1 -- 172.21.15.19:0/652078456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4003520 con 0x7f84b4004390 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.399+0000 7f84c67fc640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84b4004390 0x7f84b4004860 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c190 tx=0x7f84b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bffdf5e074616bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.399+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0007cc0 con 0x7f84b4004390 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.399+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b0005ad0 con 0x7f84b4004390 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.399+0000 7f84c5ffb640 1 -- 172.21.15.19:0/652078456 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0007070 con 0x7f84b4004390 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.400+0000 7f84cd3e2640 1 -- 172.21.15.19:0/652078456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84b4004390 msgr2=0x7f84b4004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.400+0000 7f84cd3e2640 1 --2- 172.21.15.19:0/652078456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84b4004390 0x7f84b4004860 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c190 tx=0x7f84b4005830 comp rx=0 tx=0).stop 2024-04-25T09:57:13.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.400+0000 7f84c77fe640 1 -- 172.21.15.19:0/652078456 reap_dead start 2024-04-25T09:57:13.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.400+0000 7f84cd3e2640 1 -- 172.21.15.19:0/652078456 shutdown_connections 2024-04-25T09:57:13.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.400+0000 7f84cd3e2640 1 -- 172.21.15.19:0/652078456 >> 172.21.15.19:0/652078456 conn(0x7f84c806f500 msgr2=0x7f84c806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.400+0000 7f84cd3e2640 1 -- 172.21.15.19:0/652078456 shutdown_connections 2024-04-25T09:57:13.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.400+0000 7f84cd3e2640 1 -- 172.21.15.19:0/652078456 wait complete. 2024-04-25T09:57:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.401+0000 7f84cd3e2640 1 Processor -- start 2024-04-25T09:57:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.401+0000 7f84cd3e2640 1 -- start start 2024-04-25T09:57:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84cd3e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84b4004390 0x7f84c81e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84cd3e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c81515f0 0x7f84c815df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84cd3e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c815e480 0x7f84c8156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84cd3e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80cfed0 con 0x7f84c81515f0 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84cd3e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80c8160 con 0x7f84c815e480 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84cd3e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80c72a0 con 0x7f84b4004390 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84c67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c81515f0 0x7f84c815df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84c67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c81515f0 0x7f84c815df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50474/0 (socket says 172.21.15.19:50474) 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84c67fc640 1 -- 172.21.15.19:0/2714087098 learned_addr learned my addr 172.21.15.19:0/2714087098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84c6ffd640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84b4004390 0x7f84c81e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84c77fe640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c815e480 0x7f84c8156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84c67fc640 1 -- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84b4004390 msgr2=0x7f84c81e5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84c67fc640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84b4004390 0x7f84c81e5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84c67fc640 1 -- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c815e480 msgr2=0x7f84c8156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84c67fc640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c815e480 0x7f84c8156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84c67fc640 1 -- 172.21.15.19:0/2714087098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c815e970 con 0x7f84c81515f0 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.402+0000 7f84c67fc640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c81515f0 0x7f84c815df40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f84b002acf0 tx=0x7f84b00019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.403+0000 7f84bf7fe640 1 -- 172.21.15.19:0/2714087098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0043070 con 0x7f84c81515f0 2024-04-25T09:57:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.403+0000 7f84bf7fe640 1 -- 172.21.15.19:0/2714087098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b0038040 con 0x7f84c81515f0 2024-04-25T09:57:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.403+0000 7f84cd3e2640 1 -- 172.21.15.19:0/2714087098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c815ec50 con 0x7f84c81515f0 2024-04-25T09:57:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.403+0000 7f84bf7fe640 1 -- 172.21.15.19:0/2714087098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0007070 con 0x7f84c81515f0 2024-04-25T09:57:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.403+0000 7f84cd3e2640 1 -- 172.21.15.19:0/2714087098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c81e5570 con 0x7f84c81515f0 2024-04-25T09:57:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.404+0000 7f84cd3e2640 1 -- 172.21.15.19:0/2714087098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c8073a00 con 0x7f84c81515f0 2024-04-25T09:57:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.404+0000 7f84bf7fe640 1 -- 172.21.15.19:0/2714087098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84b0007230 con 0x7f84c81515f0 2024-04-25T09:57:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.404+0000 7f84bf7fe640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84a806b500 0x7f84a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.405+0000 7f84bf7fe640 1 -- 172.21.15.19:0/2714087098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f84b00aa040 con 0x7f84c81515f0 2024-04-25T09:57:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.405+0000 7f84c6ffd640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84a806b500 0x7f84a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.405+0000 7f84c6ffd640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84a806b500 0x7f84a806d9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f84c0003640 tx=0x7f84c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.409+0000 7f84bf7fe640 1 -- 172.21.15.19:0/2714087098 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84b0005aa0 con 0x7f84c81515f0 2024-04-25T09:57:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.587+0000 7f84cd3e2640 1 -- 172.21.15.19:0/2714087098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84c80cde80 con 0x7f84c81515f0 2024-04-25T09:57:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.587+0000 7f84bf7fe640 1 -- 172.21.15.19:0/2714087098 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84b00785c0 con 0x7f84c81515f0 2024-04-25T09:57:13.591 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 -- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84a806b500 msgr2=0x7f84a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84a806b500 0x7f84a806d9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f84c0003640 tx=0x7f84c0009040 comp rx=0 tx=0).stop 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 -- 172.21.15.19:0/2714087098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c81515f0 msgr2=0x7f84c815df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c81515f0 0x7f84c815df40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f84b002acf0 tx=0x7f84b00019c0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 -- 172.21.15.19:0/2714087098 shutdown_connections 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84a806b500 0x7f84a806d9c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c815e480 0x7f84c8156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84c81515f0 0x7f84c815df40 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 --2- 172.21.15.19:0/2714087098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84b4004390 0x7f84c81e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 -- 172.21.15.19:0/2714087098 >> 172.21.15.19:0/2714087098 conn(0x7f84c806f500 msgr2=0x7f84c80b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 -- 172.21.15.19:0/2714087098 shutdown_connections 2024-04-25T09:57:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.589+0000 7f84cd3e2640 1 -- 172.21.15.19:0/2714087098 wait complete. 2024-04-25T09:57:13.603 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:57:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.703+0000 7f2fc82ac640 1 Processor -- start 2024-04-25T09:57:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.703+0000 7f2fc82ac640 1 -- start start 2024-04-25T09:57:13.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.704+0000 7f2fc82ac640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2fc00cf600 0x7f2fc00cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.704+0000 7f2fc82ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc00cbf20 0x7f2fc00cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.704+0000 7f2fc82ac640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc00c75e0 con 0x7f2fc015bb50 2024-04-25T09:57:13.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.704+0000 7f2fc82ac640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00c8680 con 0x7f2fc00cf600 2024-04-25T09:57:13.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.704+0000 7f2fc82ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00cff80 con 0x7f2fc00cbf20 2024-04-25T09:57:13.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.705+0000 7f2fc6048640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2fc015bb50 0x7f2fc00c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37580/0 (socket says 172.21.15.19:37580) 2024-04-25T09:57:13.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.705+0000 7f2fc6048640 1 -- 172.21.15.19:0/1612341426 learned_addr learned my addr 172.21.15.19:0/1612341426 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:13.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.705+0000 7f2fc5847640 1 --2- 172.21.15.19:0/1612341426 >> v2:172.21.15.19:3300/0 conn(0x7f2fc00cf600 0x7f2fc00cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.705+0000 7f2fc5847640 1 -- 172.21.15.19:0/1612341426 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc00cbf20 msgr2=0x7f2fc00cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.705+0000 7f2fc5847640 1 --2- 172.21.15.19:0/1612341426 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc00cbf20 0x7f2fc00cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.705+0000 7f2fc5847640 1 -- 172.21.15.19:0/1612341426 >> v1:172.21.15.19:6789/0 conn(0x7f2fc015bb50 legacy=0x7f2fc00c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.705+0000 7f2fc5847640 1 -- 172.21.15.19:0/1612341426 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00c78c0 con 0x7f2fc00cf600 2024-04-25T09:57:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc5847640 1 --2- 172.21.15.19:0/1612341426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc00cf600 0x7f2fc00cfa40 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009d90 tx=0x7f2fb402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e983963dd5eea7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc5046640 1 -- 172.21.15.19:0/1612341426 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4007740 con 0x7f2fc00cf600 2024-04-25T09:57:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc5046640 1 -- 172.21.15.19:0/1612341426 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb40078e0 con 0x7f2fc00cf600 2024-04-25T09:57:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc5046640 1 -- 172.21.15.19:0/1612341426 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4007cb0 con 0x7f2fc00cf600 2024-04-25T09:57:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc82ac640 1 -- 172.21.15.19:0/1612341426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc00cf600 msgr2=0x7f2fc00cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc82ac640 1 --2- 172.21.15.19:0/1612341426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc00cf600 0x7f2fc00cfa40 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009d90 tx=0x7f2fb402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc82ac640 1 -- 172.21.15.19:0/1612341426 shutdown_connections 2024-04-25T09:57:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc82ac640 1 --2- 172.21.15.19:0/1612341426 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc00cbf20 0x7f2fc00cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc82ac640 1 --2- 172.21.15.19:0/1612341426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc00cf600 0x7f2fc00cfa40 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc82ac640 1 -- 172.21.15.19:0/1612341426 >> 172.21.15.19:0/1612341426 conn(0x7f2fc006f500 msgr2=0x7f2fc006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc82ac640 1 -- 172.21.15.19:0/1612341426 shutdown_connections 2024-04-25T09:57:13.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.706+0000 7f2fc82ac640 1 -- 172.21.15.19:0/1612341426 wait complete. 2024-04-25T09:57:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.708+0000 7f2fc82ac640 1 Processor -- start 2024-04-25T09:57:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.708+0000 7f2fc82ac640 1 -- start start 2024-04-25T09:57:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.708+0000 7f2fc82ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc00cbf20 0x7f2fc0155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.708+0000 7f2fc82ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc00cf600 0x7f2fc00cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.708+0000 7f2fc82ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc015bb50 0x7f2fc00cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.708+0000 7f2fc82ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00c8160 con 0x7f2fc015bb50 2024-04-25T09:57:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.708+0000 7f2fc82ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00c7e80 con 0x7f2fc00cbf20 2024-04-25T09:57:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.708+0000 7f2fc82ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00c72a0 con 0x7f2fc00cf600 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.708+0000 7f2fc6849640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc015bb50 0x7f2fc00cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.708+0000 7f2fc6849640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc015bb50 0x7f2fc00cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50484/0 (socket says 172.21.15.19:50484) 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.708+0000 7f2fc6849640 1 -- 172.21.15.19:0/841581822 learned_addr learned my addr 172.21.15.19:0/841581822 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fc5847640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc00cf600 0x7f2fc00cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fc6048640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc00cbf20 0x7f2fc0155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fc6849640 1 -- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc00cf600 msgr2=0x7f2fc00cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fc6849640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc00cf600 0x7f2fc00cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fc6849640 1 -- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc00cbf20 msgr2=0x7f2fc0155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fc6849640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc00cbf20 0x7f2fc0155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fc6849640 1 -- 172.21.15.19:0/841581822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc0155d40 con 0x7f2fc015bb50 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fc6849640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc015bb50 0x7f2fc00cacb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009fd0 tx=0x7f2fb000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fbeffd640 1 -- 172.21.15.19:0/841581822 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0011830 con 0x7f2fc015bb50 2024-04-25T09:57:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fbeffd640 1 -- 172.21.15.19:0/841581822 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb0016070 con 0x7f2fc015bb50 2024-04-25T09:57:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fc82ac640 1 -- 172.21.15.19:0/841581822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc00c9cb0 con 0x7f2fc015bb50 2024-04-25T09:57:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fbeffd640 1 -- 172.21.15.19:0/841581822 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0011bf0 con 0x7f2fc015bb50 2024-04-25T09:57:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.709+0000 7f2fc82ac640 1 -- 172.21.15.19:0/841581822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc00ca270 con 0x7f2fc015bb50 2024-04-25T09:57:13.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.710+0000 7f2fc82ac640 1 -- 172.21.15.19:0/841581822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc0073a00 con 0x7f2fc015bb50 2024-04-25T09:57:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.711+0000 7f2fbeffd640 1 -- 172.21.15.19:0/841581822 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2fb001b460 con 0x7f2fc015bb50 2024-04-25T09:57:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.711+0000 7f2fbeffd640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 0x7f2fac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.711+0000 7f2fbeffd640 1 -- 172.21.15.19:0/841581822 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f2fb008a1c0 con 0x7f2fc015bb50 2024-04-25T09:57:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.711+0000 7f2fc6048640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 0x7f2fac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.712+0000 7f2fc6048640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 0x7f2fac06d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8000f60 tx=0x7f2fb8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.715+0000 7f2fbeffd640 1 -- 172.21.15.19:0/841581822 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2fb0058e20 con 0x7f2fc015bb50 2024-04-25T09:57:13.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.893+0000 7f2fc82ac640 1 -- 172.21.15.19:0/841581822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2fc00cf150 con 0x7f2fc015bb50 2024-04-25T09:57:13.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.894+0000 7f2fbeffd640 1 -- 172.21.15.19:0/841581822 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2fb0058720 con 0x7f2fc015bb50 2024-04-25T09:57:13.897 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:57:13.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.895+0000 7f2fc82ac640 1 -- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 msgr2=0x7f2fac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.895+0000 7f2fc82ac640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 0x7f2fac06d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8000f60 tx=0x7f2fb8009040 comp rx=0 tx=0).stop 2024-04-25T09:57:13.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.895+0000 7f2fc82ac640 1 -- 172.21.15.19:0/841581822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc015bb50 msgr2=0x7f2fc00cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.895+0000 7f2fc82ac640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc015bb50 0x7f2fc00cacb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009fd0 tx=0x7f2fb000b8e0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.895+0000 7f2fc82ac640 1 -- 172.21.15.19:0/841581822 shutdown_connections 2024-04-25T09:57:13.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.896+0000 7f2fc82ac640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 0x7f2fac06d990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.896+0000 7f2fc82ac640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc015bb50 0x7f2fc00cacb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.896+0000 7f2fc82ac640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc00cf600 0x7f2fc00cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.896+0000 7f2fc82ac640 1 --2- 172.21.15.19:0/841581822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc00cbf20 0x7f2fc0155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.896+0000 7f2fc82ac640 1 -- 172.21.15.19:0/841581822 >> 172.21.15.19:0/841581822 conn(0x7f2fc006f500 msgr2=0x7f2fc00d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.896+0000 7f2fc82ac640 1 -- 172.21.15.19:0/841581822 shutdown_connections 2024-04-25T09:57:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:13.896+0000 7f2fc82ac640 1 -- 172.21.15.19:0/841581822 wait complete. 2024-04-25T09:57:13.910 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.007+0000 7f96794a3640 1 Processor -- start 2024-04-25T09:57:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.007+0000 7f96794a3640 1 -- start start 2024-04-25T09:57:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.008+0000 7f96794a3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f967406c1d0 0x7f9674168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.008+0000 7f96794a3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96740adcf0 con 0x7f9674071ac0 2024-04-25T09:57:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.008+0000 7f96794a3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967415ec60 con 0x7f9674154ed0 2024-04-25T09:57:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.008+0000 7f96794a3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967415ef40 con 0x7f967406c1d0 2024-04-25T09:57:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.008+0000 7f966bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9674071ac0 0x7f96740ad5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37594/0 (socket says 172.21.15.19:37594) 2024-04-25T09:57:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.008+0000 7f966bfff640 1 -- 172.21.15.19:0/1030931342 learned_addr learned my addr 172.21.15.19:0/1030931342 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.008+0000 7f96737fe640 1 --2- 172.21.15.19:0/1030931342 >> v2:172.21.15.112:3300/0 conn(0x7f967406c1d0 0x7f9674168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.008+0000 7f96727fc640 1 -- 172.21.15.19:0/1030931342 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521886459 0 0) 0x7f96740adcf0 con 0x7f9674071ac0 2024-04-25T09:57:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.008+0000 7f96727fc640 1 -- 172.21.15.19:0/1030931342 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9660003520 con 0x7f9674071ac0 2024-04-25T09:57:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.009+0000 7f96727fc640 1 -- 172.21.15.19:0/1030931342 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1324528521 0 0) 0x7f9660003520 con 0x7f9674071ac0 2024-04-25T09:57:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.009+0000 7f96737fe640 1 -- 172.21.15.19:0/1030931342 >> v1:172.21.15.112:6789/0 conn(0x7f9674154ed0 legacy=0x7f96740aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.009+0000 7f96737fe640 1 -- 172.21.15.19:0/1030931342 >> v1:172.21.15.19:6789/0 conn(0x7f9674071ac0 legacy=0x7f96740ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.009+0000 7f96737fe640 1 -- 172.21.15.19:0/1030931342 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967406db90 con 0x7f967406c1d0 2024-04-25T09:57:14.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.009+0000 7f96737fe640 1 --2- 172.21.15.19:0/1030931342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f967406c1d0 0x7f9674168930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f966c009d90 tx=0x7f966c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fdc20c3fa66790c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.010+0000 7f96727fc640 1 -- 172.21.15.19:0/1030931342 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c001670 con 0x7f967406c1d0 2024-04-25T09:57:14.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.010+0000 7f96727fc640 1 -- 172.21.15.19:0/1030931342 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966c007540 con 0x7f967406c1d0 2024-04-25T09:57:14.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.010+0000 7f96727fc640 1 -- 172.21.15.19:0/1030931342 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c0078f0 con 0x7f967406c1d0 2024-04-25T09:57:14.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.010+0000 7f96794a3640 1 -- 172.21.15.19:0/1030931342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f967406c1d0 msgr2=0x7f9674168930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.010+0000 7f96794a3640 1 --2- 172.21.15.19:0/1030931342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f967406c1d0 0x7f9674168930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f966c009d90 tx=0x7f966c02a5f0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.010+0000 7f96794a3640 1 -- 172.21.15.19:0/1030931342 shutdown_connections 2024-04-25T09:57:14.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.010+0000 7f96794a3640 1 --2- 172.21.15.19:0/1030931342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f967406c1d0 0x7f9674168930 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.010+0000 7f96794a3640 1 -- 172.21.15.19:0/1030931342 >> 172.21.15.19:0/1030931342 conn(0x7f96740af670 msgr2=0x7f96740b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.010+0000 7f96794a3640 1 -- 172.21.15.19:0/1030931342 shutdown_connections 2024-04-25T09:57:14.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.010+0000 7f96794a3640 1 -- 172.21.15.19:0/1030931342 wait complete. 2024-04-25T09:57:14.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.011+0000 7f96794a3640 1 Processor -- start 2024-04-25T09:57:14.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.011+0000 7f96794a3640 1 -- start start 2024-04-25T09:57:14.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f96794a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674071ac0 0x7f96740b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f96794a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9674154ed0 0x7f96740cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f96794a3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740cd4d0 0x7f96740cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f96794a3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9674168ff0 con 0x7f9674071ac0 2024-04-25T09:57:14.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f96794a3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9674168e70 con 0x7f9674154ed0 2024-04-25T09:57:14.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f96794a3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9674169170 con 0x7f96740cd4d0 2024-04-25T09:57:14.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f9672ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674071ac0 0x7f96740b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f9672ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674071ac0 0x7f96740b1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50494/0 (socket says 172.21.15.19:50494) 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f9672ffd640 1 -- 172.21.15.19:0/2217105277 learned_addr learned my addr 172.21.15.19:0/2217105277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f96737fe640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740cd4d0 0x7f96740cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f966bfff640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9674154ed0 0x7f96740cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f9672ffd640 1 -- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740cd4d0 msgr2=0x7f96740cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f9672ffd640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740cd4d0 0x7f96740cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f9672ffd640 1 -- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9674154ed0 msgr2=0x7f96740cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f9672ffd640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9674154ed0 0x7f96740cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f9672ffd640 1 -- 172.21.15.19:0/2217105277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96740b08a0 con 0x7f9674071ac0 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.012+0000 7f966bfff640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9674154ed0 0x7f96740cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.013+0000 7f9672ffd640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674071ac0 0x7f96740b1940 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f966400c400 tx=0x7f966400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.013+0000 7f966b7fe640 1 -- 172.21.15.19:0/2217105277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664017070 con 0x7f9674071ac0 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.013+0000 7f966b7fe640 1 -- 172.21.15.19:0/2217105277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96640052d0 con 0x7f9674071ac0 2024-04-25T09:57:14.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.013+0000 7f96794a3640 1 -- 172.21.15.19:0/2217105277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96740b0b80 con 0x7f9674071ac0 2024-04-25T09:57:14.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.013+0000 7f966b7fe640 1 -- 172.21.15.19:0/2217105277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664003830 con 0x7f9674071ac0 2024-04-25T09:57:14.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.013+0000 7f96794a3640 1 -- 172.21.15.19:0/2217105277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96740b1140 con 0x7f9674071ac0 2024-04-25T09:57:14.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.014+0000 7f96794a3640 1 -- 172.21.15.19:0/2217105277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96740b2b10 con 0x7f9674071ac0 2024-04-25T09:57:14.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.014+0000 7f966b7fe640 1 -- 172.21.15.19:0/2217105277 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f966401a070 con 0x7f9674071ac0 2024-04-25T09:57:14.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.015+0000 7f966b7fe640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.015+0000 7f966b7fe640 1 -- 172.21.15.19:0/2217105277 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f9664089810 con 0x7f9674071ac0 2024-04-25T09:57:14.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.015+0000 7f966bfff640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.015+0000 7f966bfff640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9674147f80 tx=0x7f965c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.019+0000 7f966b7fe640 1 -- 172.21.15.19:0/2217105277 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96640583e0 con 0x7f9674071ac0 2024-04-25T09:57:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.244+0000 7f96794a3640 1 -- 172.21.15.19:0/2217105277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96740b51d0 con 0x7f9674071ac0 2024-04-25T09:57:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.244+0000 7f966b7fe640 1 -- 172.21.15.19:0/2217105277 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9664057ce0 con 0x7f9674071ac0 2024-04-25T09:57:14.248 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:14.248 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2433112551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2433112551},{"type":"v1","addr":"172.21.15.112:6837","nonce":2433112551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6295,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2458476655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2458476655},{"type":"v1","addr":"172.21.15.112:6835","nonce":2458476655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6867,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1321165720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1321165720},{"type":"v1","addr":"172.21.15.19:6835","nonce":1321165720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:56:38.802500+0000","modified":"2024-04-25T09:56:56.539131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6833/2482429280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2482429280},{"type":"v1","addr":"172.21.15.19:6833","nonce":2482429280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:57:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 43 2024-04-25T09:57:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.246+0000 7f96794a3640 1 -- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 msgr2=0x7f965406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.246+0000 7f96794a3640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9674147f80 tx=0x7f965c009040 comp rx=0 tx=0).stop 2024-04-25T09:57:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.246+0000 7f96794a3640 1 -- 172.21.15.19:0/2217105277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674071ac0 msgr2=0x7f96740b1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.246+0000 7f96794a3640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674071ac0 0x7f96740b1940 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f966400c400 tx=0x7f966400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.247+0000 7f96794a3640 1 -- 172.21.15.19:0/2217105277 shutdown_connections 2024-04-25T09:57:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.247+0000 7f96794a3640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.247+0000 7f96794a3640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96740cd4d0 0x7f96740cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.247+0000 7f96794a3640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9674154ed0 0x7f96740cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.247+0000 7f96794a3640 1 --2- 172.21.15.19:0/2217105277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9674071ac0 0x7f96740b1940 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.247+0000 7f96794a3640 1 -- 172.21.15.19:0/2217105277 >> 172.21.15.19:0/2217105277 conn(0x7f967406c1d0 msgr2=0x7f96740d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.247+0000 7f96794a3640 1 -- 172.21.15.19:0/2217105277 shutdown_connections 2024-04-25T09:57:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.247+0000 7f96794a3640 1 -- 172.21.15.19:0/2217105277 wait complete. 2024-04-25T09:57:14.261 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:14.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.358+0000 7fc967012640 1 Processor -- start 2024-04-25T09:57:14.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.358+0000 7fc967012640 1 -- start start 2024-04-25T09:57:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc967012640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc96015dc00 0x7fc9600c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc967012640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9600ceb20 0x7fc9600d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc967012640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96015dfe0 con 0x7fc9600d1c00 2024-04-25T09:57:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc967012640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9600d2060 con 0x7fc9600ceb20 2024-04-25T09:57:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc967012640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600d00c0 con 0x7fc96015dc00 2024-04-25T09:57:14.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc95ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9600ceb20 0x7fc9600d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc9655af640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc9600d1c00 0x7fc9600ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37604/0 (socket says 172.21.15.19:37604) 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc9655af640 1 -- 172.21.15.19:0/4279092231 learned_addr learned my addr 172.21.15.19:0/4279092231 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc964dae640 1 --2- 172.21.15.19:0/4279092231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc96015dc00 0x7fc9600c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc95ffff640 1 -- 172.21.15.19:0/4279092231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc96015dc00 msgr2=0x7fc9600c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc95ffff640 1 --2- 172.21.15.19:0/4279092231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc96015dc00 0x7fc9600c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc95ffff640 1 -- 172.21.15.19:0/4279092231 >> v1:172.21.15.19:6789/0 conn(0x7fc9600d1c00 legacy=0x7fc9600ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc95ffff640 1 -- 172.21.15.19:0/4279092231 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9600d03a0 con 0x7fc9600ceb20 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.359+0000 7fc95f7fe640 1 -- 172.21.15.19:0/4279092231 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761974549 0 0) 0x7fc950001110 con 0x7fc9600d1c00 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.360+0000 7fc95ffff640 1 --2- 172.21.15.19:0/4279092231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9600ceb20 0x7fc9600d16c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc9540097a0 tx=0x7fc95402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38ad82ad2c2f0ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.360+0000 7fc95f7fe640 1 -- 172.21.15.19:0/4279092231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954037070 con 0x7fc9600ceb20 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.360+0000 7fc95f7fe640 1 -- 172.21.15.19:0/4279092231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc954032440 con 0x7fc9600ceb20 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.360+0000 7fc95f7fe640 1 -- 172.21.15.19:0/4279092231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954032810 con 0x7fc9600ceb20 2024-04-25T09:57:14.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.361+0000 7fc967012640 1 -- 172.21.15.19:0/4279092231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9600ceb20 msgr2=0x7fc9600d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.361+0000 7fc967012640 1 --2- 172.21.15.19:0/4279092231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9600ceb20 0x7fc9600d16c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc9540097a0 tx=0x7fc95402a180 comp rx=0 tx=0).stop 2024-04-25T09:57:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.361+0000 7fc967012640 1 -- 172.21.15.19:0/4279092231 shutdown_connections 2024-04-25T09:57:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.361+0000 7fc967012640 1 --2- 172.21.15.19:0/4279092231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9600ceb20 0x7fc9600d16c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.361+0000 7fc967012640 1 --2- 172.21.15.19:0/4279092231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc96015dc00 0x7fc9600c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.361+0000 7fc967012640 1 -- 172.21.15.19:0/4279092231 >> 172.21.15.19:0/4279092231 conn(0x7fc96006f5d0 msgr2=0x7fc96006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.361+0000 7fc967012640 1 -- 172.21.15.19:0/4279092231 shutdown_connections 2024-04-25T09:57:14.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.361+0000 7fc967012640 1 -- 172.21.15.19:0/4279092231 wait complete. 2024-04-25T09:57:14.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.362+0000 7fc967012640 1 Processor -- start 2024-04-25T09:57:14.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.362+0000 7fc967012640 1 -- start start 2024-04-25T09:57:14.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc967012640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9600ceb20 0x7fc960156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc967012640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9600d1c00 0x7fc9601e7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc967012640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc96015dc00 0x7fc960156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc967012640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600cc130 con 0x7fc96015dc00 2024-04-25T09:57:14.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc967012640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600cc720 con 0x7fc9600d1c00 2024-04-25T09:57:14.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc967012640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600cfd90 con 0x7fc9600ceb20 2024-04-25T09:57:14.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc9655af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc96015dc00 0x7fc960156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc9655af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc96015dc00 0x7fc960156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50520/0 (socket says 172.21.15.19:50520) 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc9655af640 1 -- 172.21.15.19:0/1326675241 learned_addr learned my addr 172.21.15.19:0/1326675241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc964dae640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9600ceb20 0x7fc960156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc95ffff640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9600d1c00 0x7fc9601e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc9655af640 1 -- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9600ceb20 msgr2=0x7fc960156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc9655af640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9600ceb20 0x7fc960156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc9655af640 1 -- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9600d1c00 msgr2=0x7fc9601e7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc9655af640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9600d1c00 0x7fc9601e7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc9655af640 1 -- 172.21.15.19:0/1326675241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9601e7ac0 con 0x7fc96015dc00 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.363+0000 7fc95ffff640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9600d1c00 0x7fc9601e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.364+0000 7fc9655af640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc96015dc00 0x7fc960156720 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc95000c780 tx=0x7fc95000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.364+0000 7fc95d7fa640 1 -- 172.21.15.19:0/1326675241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950018070 con 0x7fc96015dc00 2024-04-25T09:57:14.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.364+0000 7fc95d7fa640 1 -- 172.21.15.19:0/1326675241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9500070d0 con 0x7fc96015dc00 2024-04-25T09:57:14.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.364+0000 7fc967012640 1 -- 172.21.15.19:0/1326675241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9600b5690 con 0x7fc96015dc00 2024-04-25T09:57:14.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.364+0000 7fc95d7fa640 1 -- 172.21.15.19:0/1326675241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950007500 con 0x7fc96015dc00 2024-04-25T09:57:14.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.364+0000 7fc967012640 1 -- 172.21.15.19:0/1326675241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9600b5c50 con 0x7fc96015dc00 2024-04-25T09:57:14.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.365+0000 7fc967012640 1 -- 172.21.15.19:0/1326675241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc960073990 con 0x7fc96015dc00 2024-04-25T09:57:14.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.365+0000 7fc95d7fa640 1 -- 172.21.15.19:0/1326675241 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc95001b070 con 0x7fc96015dc00 2024-04-25T09:57:14.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.366+0000 7fc95d7fa640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc94c06b4d0 0x7fc94c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.366+0000 7fc95d7fa640 1 -- 172.21.15.19:0/1326675241 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fc9500895b0 con 0x7fc96015dc00 2024-04-25T09:57:14.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.366+0000 7fc964dae640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc94c06b4d0 0x7fc94c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.366+0000 7fc964dae640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc94c06b4d0 0x7fc94c06d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc96015d000 tx=0x7fc958009830 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.370+0000 7fc95d7fa640 1 -- 172.21.15.19:0/1326675241 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc96015dfe0 con 0x7fc96015dc00 2024-04-25T09:57:14.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.551+0000 7fc967012640 1 -- 172.21.15.19:0/1326675241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9600c66c0 con 0x7fc96015dc00 2024-04-25T09:57:14.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.552+0000 7fc95d7fa640 1 -- 172.21.15.19:0/1326675241 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17321 (secure 0 0 0) 0x7fc950057bc0 con 0x7fc96015dc00 2024-04-25T09:57:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:14.556 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":70,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:56:54.515339+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:56:37.087339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:56:37.775183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3520485752":"2024-04-26T09:56:54.515283+0000","172.21.15.19:6833/3520485752":"2024-04-26T09:56:54.515283+0000","172.21.15.19:6834/591570980":"2024-04-26T09:56:53.512369+0000","172.21.15.19:6832/591570980":"2024-04-26T09:56:53.512369+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 -- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc94c06b4d0 msgr2=0x7fc94c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc94c06b4d0 0x7fc94c06d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc96015d000 tx=0x7fc958009830 comp rx=0 tx=0).stop 2024-04-25T09:57:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 -- 172.21.15.19:0/1326675241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc96015dc00 msgr2=0x7fc960156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc96015dc00 0x7fc960156720 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc95000c780 tx=0x7fc95000cc40 comp rx=0 tx=0).stop 2024-04-25T09:57:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 -- 172.21.15.19:0/1326675241 shutdown_connections 2024-04-25T09:57:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc94c06b4d0 0x7fc94c06d990 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc96015dc00 0x7fc960156720 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9600d1c00 0x7fc9601e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 --2- 172.21.15.19:0/1326675241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9600ceb20 0x7fc960156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 -- 172.21.15.19:0/1326675241 >> 172.21.15.19:0/1326675241 conn(0x7fc96006f5d0 msgr2=0x7fc9600b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 -- 172.21.15.19:0/1326675241 shutdown_connections 2024-04-25T09:57:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.554+0000 7fc967012640 1 -- 172.21.15.19:0/1326675241 wait complete. 2024-04-25T09:57:14.569 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:57:14.569 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.666+0000 7fbc02a5d640 1 Processor -- start 2024-04-25T09:57:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.666+0000 7fbc02a5d640 1 -- start start 2024-04-25T09:57:14.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc02a5d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dc00 0x7fbbfc0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc02a5d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbfc0d1dd0 0x7fbbfc0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc02a5d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc15dfe0 con 0x7fbbfc0ceb20 2024-04-25T09:57:14.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc02a5d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbfc0d00c0 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc02a5d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0d03a0 con 0x7fbbfc15dc00 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc00ffa640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbfc0d1dd0 0x7fbbfc0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc00ffa640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbfc0d1dd0 0x7fbbfc0ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50522/0 (socket says 172.21.15.19:50522) 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc00ffa640 1 -- 172.21.15.19:0/604732644 learned_addr learned my addr 172.21.15.19:0/604732644 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc00ffa640 1 -- 172.21.15.19:0/604732644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dc00 msgr2=0x7fbbfc0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbbfbfff640 1 --2- 172.21.15.19:0/604732644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dc00 0x7fbbfc0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc00ffa640 1 --2- 172.21.15.19:0/604732644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dc00 0x7fbbfc0c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc00ffa640 1 -- 172.21.15.19:0/604732644 >> v1:172.21.15.19:6789/0 conn(0x7fbbfc0ceb20 legacy=0x7fbbfc0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbbfaffd640 1 -- 172.21.15.19:0/604732644 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946882760 0 0) 0x7fbbec001110 con 0x7fbbfc0ceb20 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.667+0000 7fbc00ffa640 1 -- 172.21.15.19:0/604732644 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc0cc440 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.668+0000 7fbc00ffa640 1 --2- 172.21.15.19:0/604732644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0d1dd0 0x7fbbfc0ccc40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009e50 tx=0x7fbbf402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a045ce980cf16ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.668+0000 7fbbfaffd640 1 -- 172.21.15.19:0/604732644 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4036070 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.668+0000 7fbbfaffd640 1 -- 172.21.15.19:0/604732644 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4031600 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.668+0000 7fbbfaffd640 1 -- 172.21.15.19:0/604732644 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40319d0 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.669+0000 7fbc02a5d640 1 -- 172.21.15.19:0/604732644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0d1dd0 msgr2=0x7fbbfc0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.669+0000 7fbc02a5d640 1 --2- 172.21.15.19:0/604732644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0d1dd0 0x7fbbfc0ccc40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009e50 tx=0x7fbbf402a710 comp rx=0 tx=0).stop 2024-04-25T09:57:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.669+0000 7fbc02a5d640 1 -- 172.21.15.19:0/604732644 shutdown_connections 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.669+0000 7fbc02a5d640 1 --2- 172.21.15.19:0/604732644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0d1dd0 0x7fbbfc0ccc40 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.669+0000 7fbc02a5d640 1 --2- 172.21.15.19:0/604732644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dc00 0x7fbbfc0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.669+0000 7fbc02a5d640 1 -- 172.21.15.19:0/604732644 >> 172.21.15.19:0/604732644 conn(0x7fbbfc06f5d0 msgr2=0x7fbbfc06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.669+0000 7fbc02a5d640 1 -- 172.21.15.19:0/604732644 shutdown_connections 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.669+0000 7fbc02a5d640 1 -- 172.21.15.19:0/604732644 wait complete. 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.670+0000 7fbc02a5d640 1 Processor -- start 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.670+0000 7fbc02a5d640 1 -- start start 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbc02a5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbfc0ceb20 0x7fbbfc153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbc02a5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0d1dd0 0x7fbbfc0b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbc02a5d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dc00 0x7fbbfc0b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbc02a5d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0cfd90 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbc02a5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0cc130 con 0x7fbbfc0ceb20 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbc02a5d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0cdd70 con 0x7fbbfc15dc00 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbbfb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0d1dd0 0x7fbbfc0b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbbfb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0d1dd0 0x7fbbfc0b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50530/0 (socket says 172.21.15.19:50530) 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbbfb7fe640 1 -- 172.21.15.19:0/3706449241 learned_addr learned my addr 172.21.15.19:0/3706449241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbc00ffa640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dc00 0x7fbbfc0b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbbfbfff640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbfc0ceb20 0x7fbbfc153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbbfb7fe640 1 -- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dc00 msgr2=0x7fbbfc0b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbbfb7fe640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dc00 0x7fbbfc0b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbbfb7fe640 1 -- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbfc0ceb20 msgr2=0x7fbbfc153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbbfb7fe640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbfc0ceb20 0x7fbbfc153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbbfb7fe640 1 -- 172.21.15.19:0/3706449241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc06bb30 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbc00ffa640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dc00 0x7fbbfc0b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.671+0000 7fbbfb7fe640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0d1dd0 0x7fbbfc0b44a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c750 tx=0x7fbbec00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.672+0000 7fbbf8ff9640 1 -- 172.21.15.19:0/3706449241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec017070 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.672+0000 7fbbf8ff9640 1 -- 172.21.15.19:0/3706449241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec0052b0 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.672+0000 7fbc02a5d640 1 -- 172.21.15.19:0/3706449241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc06bdd0 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.672+0000 7fbbf8ff9640 1 -- 172.21.15.19:0/3706449241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec00a880 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.672+0000 7fbc02a5d640 1 -- 172.21.15.19:0/3706449241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc06c390 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.673+0000 7fbc02a5d640 1 -- 172.21.15.19:0/3706449241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc073990 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.673+0000 7fbbf8ff9640 1 -- 172.21.15.19:0/3706449241 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbec01a070 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.674+0000 7fbbf8ff9640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.674+0000 7fbbf8ff9640 1 -- 172.21.15.19:0/3706449241 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fbbec089780 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.674+0000 7fbbfbfff640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.674+0000 7fbbfbfff640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4006fd0 tx=0x7fbbe4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.678+0000 7fbbf8ff9640 1 -- 172.21.15.19:0/3706449241 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbbfc15dfe0 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.896+0000 7fbc02a5d640 1 -- 172.21.15.19:0/3706449241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbfc15afb0 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.896+0000 7fbbf8ff9640 1 -- 172.21.15.19:0/3706449241 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fbbec087e20 con 0x7fbbfc0d1dd0 2024-04-25T09:57:14.899 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:14.900 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2433112551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2433112551},{"type":"v1","addr":"172.21.15.112:6837","nonce":2433112551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6295,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2458476655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2458476655},{"type":"v1","addr":"172.21.15.112:6835","nonce":2458476655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6867,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1321165720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1321165720},{"type":"v1","addr":"172.21.15.19:6835","nonce":1321165720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:56:38.802500+0000","modified":"2024-04-25T09:56:56.539131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6833/2482429280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2482429280},{"type":"v1","addr":"172.21.15.19:6833","nonce":2482429280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:57:14.900 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 43 2024-04-25T09:57:14.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 -- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbdc06b4d0 msgr2=0x7fbbdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4006fd0 tx=0x7fbbe4008040 comp rx=0 tx=0).stop 2024-04-25T09:57:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 -- 172.21.15.19:0/3706449241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0d1dd0 msgr2=0x7fbbfc0b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0d1dd0 0x7fbbfc0b44a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c750 tx=0x7fbbec00cc10 comp rx=0 tx=0).stop 2024-04-25T09:57:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 -- 172.21.15.19:0/3706449241 shutdown_connections 2024-04-25T09:57:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbbfc15dc00 0x7fbbfc0b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbfc0d1dd0 0x7fbbfc0b44a0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 --2- 172.21.15.19:0/3706449241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbfc0ceb20 0x7fbbfc153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 -- 172.21.15.19:0/3706449241 >> 172.21.15.19:0/3706449241 conn(0x7fbbfc06f5d0 msgr2=0x7fbbfc0b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 -- 172.21.15.19:0/3706449241 shutdown_connections 2024-04-25T09:57:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:14.898+0000 7fbc02a5d640 1 -- 172.21.15.19:0/3706449241 wait complete. 2024-04-25T09:57:14.912 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.009+0000 7fa524109640 1 Processor -- start 2024-04-25T09:57:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.009+0000 7fa524109640 1 -- start start 2024-04-25T09:57:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.010+0000 7fa524109640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c15a790 0x7fa51c0d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.010+0000 7fa524109640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51c0b3710 con 0x7fa51c0b4eb0 2024-04-25T09:57:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.010+0000 7fa524109640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51c073120 con 0x7fa51c06fea0 2024-04-25T09:57:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.010+0000 7fa524109640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0cf0e0 con 0x7fa51c15a790 2024-04-25T09:57:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.010+0000 7fa5226a6640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa51c0b4eb0 0x7fa51c072a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37624/0 (socket says 172.21.15.19:37624) 2024-04-25T09:57:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.010+0000 7fa5226a6640 1 -- 172.21.15.19:0/3931345976 learned_addr learned my addr 172.21.15.19:0/3931345976 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa521ea5640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c15a790 0x7fa51c0d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638790016 0 0) 0x7fa51c0b3710 con 0x7fa51c0b4eb0 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa504003520 con 0x7fa51c0b4eb0 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 613359607 0 0) 0x7fa504003520 con 0x7fa51c0b4eb0 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c15a790 msgr2=0x7fa51c0d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c15a790 0x7fa51c0d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 >> v1:172.21.15.112:6789/0 conn(0x7fa51c06fea0 legacy=0x7fa51c072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c0cf3c0 con 0x7fa51c0b4eb0 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fa5140019f0 con 0x7fa51c0b4eb0 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 >> v1:172.21.15.19:6789/0 conn(0x7fa51c0b4eb0 legacy=0x7fa51c072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5040039e0 0x7fa504003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c15a790 0x7fa504004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa504004d40 0x7fa5040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0cf0e0 con 0x7fa504004d40 2024-04-25T09:57:15.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50400ba70 con 0x7fa5040039e0 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50400de80 con 0x7fa51c15a790 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa5226a6640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa504004d40 0x7fa5040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa521ea5640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5040039e0 0x7fa504003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa5216a4640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c15a790 0x7fa504004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa5226a6640 1 -- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c15a790 msgr2=0x7fa504004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa5226a6640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c15a790 0x7fa504004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa5226a6640 1 -- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5040039e0 msgr2=0x7fa504003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa5226a6640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5040039e0 0x7fa504003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa5226a6640 1 -- 172.21.15.19:0/3931345976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa504003520 con 0x7fa504004d40 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.011+0000 7fa5216a4640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c15a790 0x7fa504004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.012+0000 7fa5226a6640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa504004d40 0x7fa5040051c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa51400a830 tx=0x7fa504005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d1773bb692bdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.012+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51400a480 con 0x7fa504004d40 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.012+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51402f070 con 0x7fa504004d40 2024-04-25T09:57:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.012+0000 7fa520ea3640 1 -- 172.21.15.19:0/3931345976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51402bcb0 con 0x7fa504004d40 2024-04-25T09:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.013+0000 7fa524109640 1 -- 172.21.15.19:0/3931345976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa504004d40 msgr2=0x7fa5040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.013+0000 7fa524109640 1 --2- 172.21.15.19:0/3931345976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa504004d40 0x7fa5040051c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa51400a830 tx=0x7fa504005830 comp rx=0 tx=0).stop 2024-04-25T09:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.013+0000 7fa5226a6640 1 -- 172.21.15.19:0/3931345976 reap_dead start 2024-04-25T09:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.013+0000 7fa524109640 1 -- 172.21.15.19:0/3931345976 shutdown_connections 2024-04-25T09:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.013+0000 7fa524109640 1 -- 172.21.15.19:0/3931345976 >> 172.21.15.19:0/3931345976 conn(0x7fa51c0b2370 msgr2=0x7fa51c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.013+0000 7fa524109640 1 -- 172.21.15.19:0/3931345976 shutdown_connections 2024-04-25T09:57:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.013+0000 7fa524109640 1 -- 172.21.15.19:0/3931345976 wait complete. 2024-04-25T09:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.014+0000 7fa524109640 1 Processor -- start 2024-04-25T09:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.014+0000 7fa524109640 1 -- start start 2024-04-25T09:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa524109640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c146090 0x7fa51c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa524109640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa51c0a84a0 0x7fa51c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa524109640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c158860 0x7fa51c158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa524109640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0cf980 con 0x7fa51c0a84a0 2024-04-25T09:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa524109640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0b3980 con 0x7fa51c146090 2024-04-25T09:57:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa524109640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c15ab70 con 0x7fa51c158860 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa5216a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa51c0a84a0 0x7fa51c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa5216a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa51c0a84a0 0x7fa51c15da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50540/0 (socket says 172.21.15.19:50540) 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa5216a4640 1 -- 172.21.15.19:0/2931351027 learned_addr learned my addr 172.21.15.19:0/2931351027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa521ea5640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c146090 0x7fa51c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa5226a6640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c158860 0x7fa51c158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa5216a4640 1 -- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c158860 msgr2=0x7fa51c158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa5216a4640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c158860 0x7fa51c158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa5216a4640 1 -- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c146090 msgr2=0x7fa51c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa5216a4640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c146090 0x7fa51c15d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa5216a4640 1 -- 172.21.15.19:0/2931351027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c1f0960 con 0x7fa51c0a84a0 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa521ea5640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c146090 0x7fa51c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.015+0000 7fa5216a4640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa51c0a84a0 0x7fa51c15da20 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa51000e5a0 tx=0x7fa51000ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.016+0000 7fa51a7fc640 1 -- 172.21.15.19:0/2931351027 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100052e0 con 0x7fa51c0a84a0 2024-04-25T09:57:15.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.016+0000 7fa51a7fc640 1 -- 172.21.15.19:0/2931351027 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa510018070 con 0x7fa51c0a84a0 2024-04-25T09:57:15.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.016+0000 7fa524109640 1 -- 172.21.15.19:0/2931351027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51c1f0c40 con 0x7fa51c0a84a0 2024-04-25T09:57:15.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.016+0000 7fa51a7fc640 1 -- 172.21.15.19:0/2931351027 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510003830 con 0x7fa51c0a84a0 2024-04-25T09:57:15.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.016+0000 7fa524109640 1 -- 172.21.15.19:0/2931351027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51c1f1200 con 0x7fa51c0a84a0 2024-04-25T09:57:15.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.017+0000 7fa524109640 1 -- 172.21.15.19:0/2931351027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51c073990 con 0x7fa51c0a84a0 2024-04-25T09:57:15.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.017+0000 7fa51a7fc640 1 -- 172.21.15.19:0/2931351027 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5100039d0 con 0x7fa51c0a84a0 2024-04-25T09:57:15.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.018+0000 7fa51a7fc640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa50806b500 0x7fa50806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.018+0000 7fa51a7fc640 1 -- 172.21.15.19:0/2931351027 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fa51008a280 con 0x7fa51c0a84a0 2024-04-25T09:57:15.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.018+0000 7fa521ea5640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa50806b500 0x7fa50806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.018+0000 7fa521ea5640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa50806b500 0x7fa50806d9c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009b40 tx=0x7fa50c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.022+0000 7fa51a7fc640 1 -- 172.21.15.19:0/2931351027 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa510058db0 con 0x7fa51c0a84a0 2024-04-25T09:57:15.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.237+0000 7fa524109640 1 -- 172.21.15.19:0/2931351027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa51c0cece0 con 0x7fa51c0a84a0 2024-04-25T09:57:15.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.238+0000 7fa51a7fc640 1 -- 172.21.15.19:0/2931351027 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fa51001be40 con 0x7fa51c0a84a0 2024-04-25T09:57:15.241 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:15.241 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2433112551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2433112551},{"type":"v1","addr":"172.21.15.112:6837","nonce":2433112551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6295,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2458476655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2458476655},{"type":"v1","addr":"172.21.15.112:6835","nonce":2458476655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6867,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1321165720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1321165720},{"type":"v1","addr":"172.21.15.19:6835","nonce":1321165720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:56:38.802500+0000","modified":"2024-04-25T09:56:56.539131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6833/2482429280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2482429280},{"type":"v1","addr":"172.21.15.19:6833","nonce":2482429280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:57:15.242 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 43 2024-04-25T09:57:15.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 -- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa50806b500 msgr2=0x7fa50806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa50806b500 0x7fa50806d9c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009b40 tx=0x7fa50c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:15.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 -- 172.21.15.19:0/2931351027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa51c0a84a0 msgr2=0x7fa51c15da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa51c0a84a0 0x7fa51c15da20 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa51000e5a0 tx=0x7fa51000ea60 comp rx=0 tx=0).stop 2024-04-25T09:57:15.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 -- 172.21.15.19:0/2931351027 shutdown_connections 2024-04-25T09:57:15.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa50806b500 0x7fa50806d9c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa51c158860 0x7fa51c158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa51c0a84a0 0x7fa51c15da20 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 --2- 172.21.15.19:0/2931351027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c146090 0x7fa51c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 -- 172.21.15.19:0/2931351027 >> 172.21.15.19:0/2931351027 conn(0x7fa51c0b2370 msgr2=0x7fa51c156190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 -- 172.21.15.19:0/2931351027 shutdown_connections 2024-04-25T09:57:15.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.240+0000 7fa524109640 1 -- 172.21.15.19:0/2931351027 wait complete. 2024-04-25T09:57:15.254 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.351+0000 7f06e2dc4640 1 Processor -- start 2024-04-25T09:57:15.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.351+0000 7f06e2dc4640 1 -- start start 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.352+0000 7f06e2dc4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06dc15dc70 0x7f06dc0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.352+0000 7f06e2dc4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f06dc06ce10 0x7f06dc0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.352+0000 7f06e2dc4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06dc06c220 con 0x7f06dc09a450 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.352+0000 7f06e2dc4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06dc09a8b0 con 0x7f06dc06ce10 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.352+0000 7f06e2dc4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0d0c90 con 0x7f06dc15dc70 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.352+0000 7f06e1361640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f06dc06ce10 0x7f06dc0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.352+0000 7f06dbfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f06dc09a450 0x7f06dc06c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37630/0 (socket says 172.21.15.19:37630) 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.352+0000 7f06e1361640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f06dc06ce10 0x7f06dc0cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50544/0 (socket says 172.21.15.19:50544) 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.352+0000 7f06dbfff640 1 -- 172.21.15.19:0/2988135736 learned_addr learned my addr 172.21.15.19:0/2988135736 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.353+0000 7f06e0b60640 1 --2- 172.21.15.19:0/2988135736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06dc15dc70 0x7f06dc0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.353+0000 7f06e1361640 1 -- 172.21.15.19:0/2988135736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06dc15dc70 msgr2=0x7f06dc0d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:15.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.353+0000 7f06e1361640 1 --2- 172.21.15.19:0/2988135736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06dc15dc70 0x7f06dc0d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.353+0000 7f06e1361640 1 -- 172.21.15.19:0/2988135736 >> v1:172.21.15.19:6789/0 conn(0x7f06dc09a450 legacy=0x7f06dc06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.353+0000 7f06e1361640 1 -- 172.21.15.19:0/2988135736 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc0d0080 con 0x7f06dc06ce10 2024-04-25T09:57:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.353+0000 7f06db7fe640 1 -- 172.21.15.19:0/2988135736 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575602034 0 0) 0x7f06dc06c220 con 0x7f06dc09a450 2024-04-25T09:57:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.353+0000 7f06e1361640 1 --2- 172.21.15.19:0/2988135736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06dc06ce10 0x7f06dc0cf9c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009d90 tx=0x7f06d402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb3d980ea4b368a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.354+0000 7f06db7fe640 1 -- 172.21.15.19:0/2988135736 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4007740 con 0x7f06dc06ce10 2024-04-25T09:57:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.354+0000 7f06db7fe640 1 -- 172.21.15.19:0/2988135736 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d40078e0 con 0x7f06dc06ce10 2024-04-25T09:57:15.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.354+0000 7f06db7fe640 1 -- 172.21.15.19:0/2988135736 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4007cb0 con 0x7f06dc06ce10 2024-04-25T09:57:15.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.354+0000 7f06e2dc4640 1 -- 172.21.15.19:0/2988135736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06dc06ce10 msgr2=0x7f06dc0cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.354+0000 7f06e2dc4640 1 --2- 172.21.15.19:0/2988135736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06dc06ce10 0x7f06dc0cf9c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009d90 tx=0x7f06d402a660 comp rx=0 tx=0).stop 2024-04-25T09:57:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.354+0000 7f06e2dc4640 1 -- 172.21.15.19:0/2988135736 shutdown_connections 2024-04-25T09:57:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.354+0000 7f06e2dc4640 1 --2- 172.21.15.19:0/2988135736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06dc06ce10 0x7f06dc0cf9c0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.354+0000 7f06e2dc4640 1 --2- 172.21.15.19:0/2988135736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06dc15dc70 0x7f06dc0d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.354+0000 7f06e2dc4640 1 -- 172.21.15.19:0/2988135736 >> 172.21.15.19:0/2988135736 conn(0x7f06dc0b3270 msgr2=0x7f06dc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.354+0000 7f06e2dc4640 1 -- 172.21.15.19:0/2988135736 shutdown_connections 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.354+0000 7f06e2dc4640 1 -- 172.21.15.19:0/2988135736 wait complete. 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.355+0000 7f06e2dc4640 1 Processor -- start 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.356+0000 7f06e2dc4640 1 -- start start 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.356+0000 7f06e2dc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06dc06ce10 0x7f06dc0c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.356+0000 7f06e2dc4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06dc09a450 0x7f06dc09eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.356+0000 7f06e2dc4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06dc15dc70 0x7f06dc0c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.356+0000 7f06e2dc4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0d0920 con 0x7f06dc06ce10 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.356+0000 7f06e2dc4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0d0640 con 0x7f06dc09a450 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.356+0000 7f06e2dc4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc15e050 con 0x7f06dc15dc70 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e0b60640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06dc06ce10 0x7f06dc0c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e0b60640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06dc06ce10 0x7f06dc0c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50554/0 (socket says 172.21.15.19:50554) 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e0b60640 1 -- 172.21.15.19:0/62579428 learned_addr learned my addr 172.21.15.19:0/62579428 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06dbfff640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06dc09a450 0x7f06dc09eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e1361640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06dc15dc70 0x7f06dc0c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e0b60640 1 -- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06dc15dc70 msgr2=0x7f06dc0c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e0b60640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06dc15dc70 0x7f06dc0c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e0b60640 1 -- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06dc09a450 msgr2=0x7f06dc09eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e0b60640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06dc09a450 0x7f06dc09eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e0b60640 1 -- 172.21.15.19:0/62579428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc09f410 con 0x7f06dc06ce10 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e0b60640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06dc06ce10 0x7f06dc0c8850 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00d800 tx=0x7f06cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06d97fa640 1 -- 172.21.15.19:0/62579428 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00b6c0 con 0x7f06dc06ce10 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06d97fa640 1 -- 172.21.15.19:0/62579428 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06cc019070 con 0x7f06dc06ce10 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e2dc4640 1 -- 172.21.15.19:0/62579428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc0c9780 con 0x7f06dc06ce10 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06d97fa640 1 -- 172.21.15.19:0/62579428 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00ba80 con 0x7f06dc06ce10 2024-04-25T09:57:15.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.357+0000 7f06e2dc4640 1 -- 172.21.15.19:0/62579428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc0c91c0 con 0x7f06dc06ce10 2024-04-25T09:57:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.358+0000 7f06e2dc4640 1 -- 172.21.15.19:0/62579428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06dc073990 con 0x7f06dc06ce10 2024-04-25T09:57:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.359+0000 7f06d97fa640 1 -- 172.21.15.19:0/62579428 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06cc00bc20 con 0x7f06dc06ce10 2024-04-25T09:57:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.359+0000 7f06d97fa640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f06c806b4d0 0x7f06c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.359+0000 7f06d97fa640 1 -- 172.21.15.19:0/62579428 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f06cc08a990 con 0x7f06dc06ce10 2024-04-25T09:57:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.359+0000 7f06dbfff640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f06c806b4d0 0x7f06c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.360+0000 7f06dbfff640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f06c806b4d0 0x7f06c806d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0c9bf0 tx=0x7f06d0007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.363+0000 7f06d97fa640 1 -- 172.21.15.19:0/62579428 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06cc01c360 con 0x7f06dc06ce10 2024-04-25T09:57:15.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.542+0000 7f06e2dc4640 1 -- 172.21.15.19:0/62579428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06dc0b72a0 con 0x7f06dc06ce10 2024-04-25T09:57:15.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.543+0000 7f06d97fa640 1 -- 172.21.15.19:0/62579428 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17321 (secure 0 0 0) 0x7f06cc059560 con 0x7f06dc06ce10 2024-04-25T09:57:15.547 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:15.547 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":70,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:56:54.515339+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:56:37.087339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:56:37.775183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3520485752":"2024-04-26T09:56:54.515283+0000","172.21.15.19:6833/3520485752":"2024-04-26T09:56:54.515283+0000","172.21.15.19:6834/591570980":"2024-04-26T09:56:53.512369+0000","172.21.15.19:6832/591570980":"2024-04-26T09:56:53.512369+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:15.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.545+0000 7f06e2dc4640 1 -- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f06c806b4d0 msgr2=0x7f06c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.545+0000 7f06e2dc4640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f06c806b4d0 0x7f06c806d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0c9bf0 tx=0x7f06d0007b50 comp rx=0 tx=0).stop 2024-04-25T09:57:15.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.545+0000 7f06e2dc4640 1 -- 172.21.15.19:0/62579428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06dc06ce10 msgr2=0x7f06dc0c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.545+0000 7f06e2dc4640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06dc06ce10 0x7f06dc0c8850 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00d800 tx=0x7f06cc00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.546+0000 7f06e2dc4640 1 -- 172.21.15.19:0/62579428 shutdown_connections 2024-04-25T09:57:15.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.546+0000 7f06e2dc4640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f06c806b4d0 0x7f06c806d990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.546+0000 7f06e2dc4640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06dc15dc70 0x7f06dc0c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.546+0000 7f06e2dc4640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06dc09a450 0x7f06dc09eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.546+0000 7f06e2dc4640 1 --2- 172.21.15.19:0/62579428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06dc06ce10 0x7f06dc0c8850 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.546+0000 7f06e2dc4640 1 -- 172.21.15.19:0/62579428 >> 172.21.15.19:0/62579428 conn(0x7f06dc0b3270 msgr2=0x7f06dc0b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.546+0000 7f06e2dc4640 1 -- 172.21.15.19:0/62579428 shutdown_connections 2024-04-25T09:57:15.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.546+0000 7f06e2dc4640 1 -- 172.21.15.19:0/62579428 wait complete. 2024-04-25T09:57:15.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:57:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.659+0000 7f973b5e2640 1 Processor -- start 2024-04-25T09:57:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.659+0000 7f973b5e2640 1 -- start start 2024-04-25T09:57:15.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f973b5e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f973406c9e0 0x7f97340b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f973b5e2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973406cf50 con 0x7f973415b7d0 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f973b5e2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97340ce530 con 0x7f97340b5620 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f973b5e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97340ce810 con 0x7f973406c9e0 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f973937e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f973415b7d0 0x7f97340cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37638/0 (socket says 172.21.15.19:37638) 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f973937e640 1 -- 172.21.15.19:0/2882714013 learned_addr learned my addr 172.21.15.19:0/2882714013 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f9739b7f640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f973406c9e0 0x7f97340b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077952793 0 0) 0x7f973406cf50 con 0x7f973415b7d0 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9720003520 con 0x7f973415b7d0 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 699748349 0 0) 0x7f9720003520 con 0x7f973415b7d0 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f973406c9e0 msgr2=0x7f97340b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f973406c9e0 0x7f97340b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 >> v1:172.21.15.112:6789/0 conn(0x7f97340b5620 legacy=0x7f973406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97340ceaf0 con 0x7f973415b7d0 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f9730001690 con 0x7f973415b7d0 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 >> v1:172.21.15.19:6789/0 conn(0x7f973415b7d0 legacy=0x7f97340cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f973406c9e0 0x7f97200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720003f20 0x7f9720004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9720004d40 0x7f97200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97340ce810 con 0x7f9720003f20 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972000ba20 con 0x7f9720004d40 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.660+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972000bc20 con 0x7f973406c9e0 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.661+0000 7f9738b7d640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720003f20 0x7f9720004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.661+0000 7f973937e640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f973406c9e0 0x7f97200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.661+0000 7f9739b7f640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9720004d40 0x7f97200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.661+0000 7f9738b7d640 1 -- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f973406c9e0 msgr2=0x7f97200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.661+0000 7f9738b7d640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f973406c9e0 0x7f97200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.661+0000 7f9738b7d640 1 -- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9720004d40 msgr2=0x7f97200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.661+0000 7f9738b7d640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9720004d40 0x7f97200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.661+0000 7f9738b7d640 1 -- 172.21.15.19:0/2882714013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9720003520 con 0x7f9720003f20 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.661+0000 7f9739b7f640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9720004d40 0x7f97200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:15.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.662+0000 7f9738b7d640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720003f20 0x7f9720004800 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9724001860 tx=0x7f9720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5019b784d0597435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.662+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724037070 con 0x7f9720003f20 2024-04-25T09:57:15.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.662+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972402ab60 con 0x7f9720003f20 2024-04-25T09:57:15.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.662+0000 7f972bfff640 1 -- 172.21.15.19:0/2882714013 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724001430 con 0x7f9720003f20 2024-04-25T09:57:15.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.662+0000 7f973b5e2640 1 -- 172.21.15.19:0/2882714013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720003f20 msgr2=0x7f9720004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.662+0000 7f973b5e2640 1 --2- 172.21.15.19:0/2882714013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9720003f20 0x7f9720004800 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9724001860 tx=0x7f9720005830 comp rx=0 tx=0).stop 2024-04-25T09:57:15.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.662+0000 7f9739b7f640 1 -- 172.21.15.19:0/2882714013 reap_dead start 2024-04-25T09:57:15.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.662+0000 7f973b5e2640 1 -- 172.21.15.19:0/2882714013 shutdown_connections 2024-04-25T09:57:15.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.662+0000 7f973b5e2640 1 -- 172.21.15.19:0/2882714013 >> 172.21.15.19:0/2882714013 conn(0x7f97340d0800 msgr2=0x7f97340d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.662+0000 7f973b5e2640 1 -- 172.21.15.19:0/2882714013 shutdown_connections 2024-04-25T09:57:15.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.662+0000 7f973b5e2640 1 -- 172.21.15.19:0/2882714013 wait complete. 2024-04-25T09:57:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.663+0000 7f973b5e2640 1 Processor -- start 2024-04-25T09:57:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.664+0000 7f973b5e2640 1 -- start start 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.664+0000 7f973b5e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9720003f20 0x7f9734150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.664+0000 7f973b5e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734158eb0 0x7f9734159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.664+0000 7f973b5e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97340c8ed0 0x7f97340c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.664+0000 7f973b5e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973415bd30 con 0x7f9734158eb0 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.664+0000 7f973b5e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973415bbb0 con 0x7f9720003f20 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.664+0000 7f973b5e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97340b5ab0 con 0x7f97340c8ed0 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.664+0000 7f9738b7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734158eb0 0x7f9734159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.664+0000 7f9738b7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734158eb0 0x7f9734159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50564/0 (socket says 172.21.15.19:50564) 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.664+0000 7f9738b7d640 1 -- 172.21.15.19:0/1486137991 learned_addr learned my addr 172.21.15.19:0/1486137991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f973937e640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9720003f20 0x7f9734150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f9739b7f640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97340c8ed0 0x7f97340c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f9738b7d640 1 -- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97340c8ed0 msgr2=0x7f97340c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f9738b7d640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97340c8ed0 0x7f97340c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f9738b7d640 1 -- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9720003f20 msgr2=0x7f9734150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f9738b7d640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9720003f20 0x7f9734150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f9738b7d640 1 -- 172.21.15.19:0/1486137991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97340b7210 con 0x7f9734158eb0 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f9738b7d640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734158eb0 0x7f9734159310 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f97240018c0 tx=0x7f9724001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f9729ffb640 1 -- 172.21.15.19:0/1486137991 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972403f020 con 0x7f9734158eb0 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f9729ffb640 1 -- 172.21.15.19:0/1486137991 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9724005270 con 0x7f9734158eb0 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f973b5e2640 1 -- 172.21.15.19:0/1486137991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97340b74f0 con 0x7f9734158eb0 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f9729ffb640 1 -- 172.21.15.19:0/1486137991 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724037040 con 0x7f9734158eb0 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.665+0000 7f973b5e2640 1 -- 172.21.15.19:0/1486137991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97340b7ab0 con 0x7f9734158eb0 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.666+0000 7f973b5e2640 1 -- 172.21.15.19:0/1486137991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9734073a00 con 0x7f9734158eb0 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.667+0000 7f9729ffb640 1 -- 172.21.15.19:0/1486137991 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9724042020 con 0x7f9734158eb0 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.667+0000 7f9729ffb640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f971806b500 0x7f971806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.667+0000 7f9729ffb640 1 -- 172.21.15.19:0/1486137991 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f97240aa2b0 con 0x7f9734158eb0 2024-04-25T09:57:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.668+0000 7f973937e640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f971806b500 0x7f971806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.668+0000 7f973937e640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f971806b500 0x7f971806d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f973406ce80 tx=0x7f9730009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.672+0000 7f9729ffb640 1 -- 172.21.15.19:0/1486137991 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9724049030 con 0x7f9734158eb0 2024-04-25T09:57:15.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.892+0000 7f973b5e2640 1 -- 172.21.15.19:0/1486137991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f973406c470 con 0x7f9734158eb0 2024-04-25T09:57:15.896 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T09:57:15.893+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:57:15.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.894+0000 7f9729ffb640 1 -- 172.21.15.19:0/1486137991 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9724030e20 con 0x7f9734158eb0 2024-04-25T09:57:15.898 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:57:15.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 -- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f971806b500 msgr2=0x7f971806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f971806b500 0x7f971806d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f973406ce80 tx=0x7f9730009040 comp rx=0 tx=0).stop 2024-04-25T09:57:15.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 -- 172.21.15.19:0/1486137991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734158eb0 msgr2=0x7f9734159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734158eb0 0x7f9734159310 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f97240018c0 tx=0x7f9724001230 comp rx=0 tx=0).stop 2024-04-25T09:57:15.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 -- 172.21.15.19:0/1486137991 shutdown_connections 2024-04-25T09:57:15.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f971806b500 0x7f971806d9c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97340c8ed0 0x7f97340c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9734158eb0 0x7f9734159310 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 --2- 172.21.15.19:0/1486137991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9720003f20 0x7f9734150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 -- 172.21.15.19:0/1486137991 >> 172.21.15.19:0/1486137991 conn(0x7f97340d0800 msgr2=0x7f97340cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 -- 172.21.15.19:0/1486137991 shutdown_connections 2024-04-25T09:57:15.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:15.898+0000 7f973b5e2640 1 -- 172.21.15.19:0/1486137991 wait complete. 2024-04-25T09:57:15.904 INFO:tasks.ceph.mds.c.smithi019.stderr: -1685> 2024-04-25T09:57:00.526+0000 7fe546160640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:57:15.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:57:15.933 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T09:57:16.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.015+0000 7f342dcf5640 1 Processor -- start 2024-04-25T09:57:16.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.015+0000 7f342dcf5640 1 -- start start 2024-04-25T09:57:16.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.015+0000 7f342dcf5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f34280b38d0 0x7f34280b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.015+0000 7f342dcf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342806bd20 0x7f342806c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.015+0000 7f342dcf5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342815e020 con 0x7f342806e360 2024-04-25T09:57:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.015+0000 7f342dcf5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342806b890 con 0x7f34280b38d0 2024-04-25T09:57:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.015+0000 7f342dcf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342806ac80 con 0x7f342806bd20 2024-04-25T09:57:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34277fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f342806e360 0x7f342806e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37648/0 (socket says 172.21.15.19:37648) 2024-04-25T09:57:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34277fe640 1 -- 172.21.15.19:0/1464734452 learned_addr learned my addr 172.21.15.19:0/1464734452 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f3426ffd640 1 --2- 172.21.15.19:0/1464734452 >> v2:172.21.15.112:3300/0 conn(0x7f34280b38d0 0x7f34280b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f3427fff640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342806bd20 0x7f342806c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413016409 0 0) 0x7f342815e020 con 0x7f342806e360 2024-04-25T09:57:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3410003520 con 0x7f342806e360 2024-04-25T09:57:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4174415741 0 0) 0x7f3410003520 con 0x7f342806e360 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342806bd20 msgr2=0x7f342806c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342806bd20 0x7f342806c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 >> v2:172.21.15.112:3300/0 conn(0x7f34280b38d0 msgr2=0x7f34280b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 --2- 172.21.15.19:0/1464734452 >> v2:172.21.15.112:3300/0 conn(0x7f34280b38d0 0x7f34280b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342806af60 con 0x7f342806e360 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f3418001670 con 0x7f342806e360 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 >> v1:172.21.15.19:6789/0 conn(0x7f342806e360 legacy=0x7f342806e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34100039b0 0x7f3410003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342806bd20 0x7f3410004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3410004d40 0x7f34100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342806b890 con 0x7f34100039b0 2024-04-25T09:57:16.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342806ac80 con 0x7f3410004d40 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341000b9c0 con 0x7f342806bd20 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f34277fe640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34100039b0 0x7f3410003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f3427fff640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3410004d40 0x7f34100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.016+0000 7f3426ffd640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342806bd20 0x7f3410004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.017+0000 7f34277fe640 1 -- 172.21.15.19:0/1464734452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342806bd20 msgr2=0x7f3410004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.017+0000 7f34277fe640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f342806bd20 0x7f3410004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.017+0000 7f34277fe640 1 -- 172.21.15.19:0/1464734452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3410004d40 msgr2=0x7f34100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.017+0000 7f34277fe640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3410004d40 0x7f34100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.017+0000 7f34277fe640 1 -- 172.21.15.19:0/1464734452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3410003520 con 0x7f34100039b0 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.017+0000 7f34277fe640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34100039b0 0x7f3410003dd0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f341800c240 tx=0x7f3410005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fa053d98f7c598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.017+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341802e070 con 0x7f34100039b0 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.018+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3418005af0 con 0x7f34100039b0 2024-04-25T09:57:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.018+0000 7f34267fc640 1 -- 172.21.15.19:0/1464734452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418007d30 con 0x7f34100039b0 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.018+0000 7f342dcf5640 1 -- 172.21.15.19:0/1464734452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34100039b0 msgr2=0x7f3410003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.018+0000 7f342dcf5640 1 --2- 172.21.15.19:0/1464734452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34100039b0 0x7f3410003dd0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f341800c240 tx=0x7f3410005830 comp rx=0 tx=0).stop 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.018+0000 7f3427fff640 1 -- 172.21.15.19:0/1464734452 reap_dead start 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.018+0000 7f342dcf5640 1 -- 172.21.15.19:0/1464734452 shutdown_connections 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.018+0000 7f342dcf5640 1 -- 172.21.15.19:0/1464734452 >> 172.21.15.19:0/1464734452 conn(0x7f34280cfec0 msgr2=0x7f34280d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.018+0000 7f342dcf5640 1 -- 172.21.15.19:0/1464734452 shutdown_connections 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.018+0000 7f342dcf5640 1 -- 172.21.15.19:0/1464734452 wait complete. 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.019+0000 7f342dcf5640 1 Processor -- start 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.020+0000 7f342dcf5640 1 -- start start 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.020+0000 7f342dcf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34100039b0 0x7f34281e52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.020+0000 7f342dcf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34281f4ef0 0x7f34280b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.020+0000 7f342dcf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34280b1430 0x7f34281f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.020+0000 7f342dcf5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342806ee50 con 0x7f34281f4ef0 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.020+0000 7f342dcf5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342806b520 con 0x7f34100039b0 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.020+0000 7f342dcf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34280b2fd0 con 0x7f34280b1430 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.020+0000 7f3426ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34281f4ef0 0x7f34280b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.020+0000 7f3426ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34281f4ef0 0x7f34280b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50598/0 (socket says 172.21.15.19:50598) 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.020+0000 7f3426ffd640 1 -- 172.21.15.19:0/2720452623 learned_addr learned my addr 172.21.15.19:0/2720452623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f3427fff640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34280b1430 0x7f34281f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f34277fe640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34100039b0 0x7f34281e52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f3426ffd640 1 -- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34280b1430 msgr2=0x7f34281f77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f3426ffd640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34280b1430 0x7f34281f77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f3426ffd640 1 -- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34100039b0 msgr2=0x7f34281e52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f3426ffd640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34100039b0 0x7f34281e52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f3426ffd640 1 -- 172.21.15.19:0/2720452623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34281f8500 con 0x7f34281f4ef0 2024-04-25T09:57:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f34277fe640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34100039b0 0x7f34281e52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f3426ffd640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34281f4ef0 0x7f34280b0ef0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f341c011960 tx=0x7f341c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f340ffff640 1 -- 172.21.15.19:0/2720452623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c00faf0 con 0x7f34281f4ef0 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f340ffff640 1 -- 172.21.15.19:0/2720452623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c00fc90 con 0x7f34281f4ef0 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f342dcf5640 1 -- 172.21.15.19:0/2720452623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34281f4130 con 0x7f34281f4ef0 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f340ffff640 1 -- 172.21.15.19:0/2720452623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c014440 con 0x7f34281f4ef0 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.021+0000 7f342dcf5640 1 -- 172.21.15.19:0/2720452623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34281f46f0 con 0x7f34281f4ef0 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.022+0000 7f342dcf5640 1 -- 172.21.15.19:0/2720452623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3428073a00 con 0x7f34281f4ef0 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.023+0000 7f340ffff640 1 -- 172.21.15.19:0/2720452623 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f341c012040 con 0x7f34281f4ef0 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.023+0000 7f340ffff640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f341406b4d0 0x7f341406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.023+0000 7f340ffff640 1 -- 172.21.15.19:0/2720452623 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f341c08ab10 con 0x7f34281f4ef0 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.023+0000 7f34277fe640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f341406b4d0 0x7f341406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.024+0000 7f34277fe640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f341406b4d0 0x7f341406d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3418033000 tx=0x7f3418009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:16.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.027+0000 7f340ffff640 1 -- 172.21.15.19:0/2720452623 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f341c019030 con 0x7f34281f4ef0 2024-04-25T09:57:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.245+0000 7f342dcf5640 1 -- 172.21.15.19:0/2720452623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f34280b1810 con 0x7f34281f4ef0 2024-04-25T09:57:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.896+0000 7f340ffff640 1 -- 172.21.15.19:0/2720452623 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7f341c059040 con 0x7f34281f4ef0 2024-04-25T09:57:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 -- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f341406b4d0 msgr2=0x7f341406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f341406b4d0 0x7f341406d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3418033000 tx=0x7f3418009030 comp rx=0 tx=0).stop 2024-04-25T09:57:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 -- 172.21.15.19:0/2720452623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34281f4ef0 msgr2=0x7f34280b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34281f4ef0 0x7f34280b0ef0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f341c011960 tx=0x7f341c011e20 comp rx=0 tx=0).stop 2024-04-25T09:57:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 -- 172.21.15.19:0/2720452623 shutdown_connections 2024-04-25T09:57:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f341406b4d0 0x7f341406d990 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34280b1430 0x7f34281f77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34281f4ef0 0x7f34280b0ef0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 --2- 172.21.15.19:0/2720452623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34100039b0 0x7f34281e52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 -- 172.21.15.19:0/2720452623 >> 172.21.15.19:0/2720452623 conn(0x7f34280cfec0 msgr2=0x7f34280ace60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 -- 172.21.15.19:0/2720452623 shutdown_connections 2024-04-25T09:57:16.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:16.899+0000 7f342dcf5640 1 -- 172.21.15.19:0/2720452623 wait complete. 2024-04-25T09:57:16.916 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:57:17.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.064+0000 7feb84bb7640 1 Processor -- start 2024-04-25T09:57:17.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.064+0000 7feb84bb7640 1 -- start start 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb84bb7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb800b5620 0x7feb800b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb84bb7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feb8006c2d0 0x7feb8006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb84bb7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8006cc80 con 0x7feb8015b7d0 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb84bb7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8006cf50 con 0x7feb800b5620 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb84bb7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb800ce530 con 0x7feb8006c2d0 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb7dd9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb800b5620 0x7feb800b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb7e59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feb8015b7d0 0x7feb800cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37650/0 (socket says 172.21.15.19:37650) 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb7e59c640 1 -- 172.21.15.19:0/3650133624 learned_addr learned my addr 172.21.15.19:0/3650133624 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb7ed9d640 1 --2- 172.21.15.19:0/3650133624 >> v2:172.21.15.112:3300/0 conn(0x7feb8006c2d0 0x7feb8006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb7d59a640 1 -- 172.21.15.19:0/3650133624 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412179372 0 0) 0x7feb8006cc80 con 0x7feb8015b7d0 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb7dd9b640 1 -- 172.21.15.19:0/3650133624 >> v2:172.21.15.112:3300/0 conn(0x7feb8006c2d0 msgr2=0x7feb8006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb7dd9b640 1 --2- 172.21.15.19:0/3650133624 >> v2:172.21.15.112:3300/0 conn(0x7feb8006c2d0 0x7feb8006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb7dd9b640 1 -- 172.21.15.19:0/3650133624 >> v1:172.21.15.19:6789/0 conn(0x7feb8015b7d0 legacy=0x7feb800cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.065+0000 7feb7dd9b640 1 -- 172.21.15.19:0/3650133624 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb800ce810 con 0x7feb800b5620 2024-04-25T09:57:17.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.066+0000 7feb7dd9b640 1 --2- 172.21.15.19:0/3650133624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb800b5620 0x7feb800b5ab0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7feb80161890 tx=0x7feb7402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd7af74d3225b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.066+0000 7feb7d59a640 1 -- 172.21.15.19:0/3650133624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74037070 con 0x7feb800b5620 2024-04-25T09:57:17.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.066+0000 7feb7d59a640 1 -- 172.21.15.19:0/3650133624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb74032440 con 0x7feb800b5620 2024-04-25T09:57:17.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.066+0000 7feb7d59a640 1 -- 172.21.15.19:0/3650133624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74032810 con 0x7feb800b5620 2024-04-25T09:57:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.067+0000 7feb84bb7640 1 -- 172.21.15.19:0/3650133624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb800b5620 msgr2=0x7feb800b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.067+0000 7feb84bb7640 1 --2- 172.21.15.19:0/3650133624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb800b5620 0x7feb800b5ab0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7feb80161890 tx=0x7feb7402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.067+0000 7feb84bb7640 1 -- 172.21.15.19:0/3650133624 shutdown_connections 2024-04-25T09:57:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.067+0000 7feb84bb7640 1 --2- 172.21.15.19:0/3650133624 >> v2:172.21.15.112:3300/0 conn(0x7feb8006c2d0 0x7feb8006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.067+0000 7feb84bb7640 1 --2- 172.21.15.19:0/3650133624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb800b5620 0x7feb800b5ab0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.067+0000 7feb84bb7640 1 -- 172.21.15.19:0/3650133624 >> 172.21.15.19:0/3650133624 conn(0x7feb800d0800 msgr2=0x7feb800d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.067+0000 7feb84bb7640 1 -- 172.21.15.19:0/3650133624 shutdown_connections 2024-04-25T09:57:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.067+0000 7feb84bb7640 1 -- 172.21.15.19:0/3650133624 wait complete. 2024-04-25T09:57:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.068+0000 7feb84bb7640 1 Processor -- start 2024-04-25T09:57:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.068+0000 7feb84bb7640 1 -- start start 2024-04-25T09:57:17.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb84bb7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb8006c2d0 0x7feb80157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb84bb7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb800b5620 0x7feb800c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb84bb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb8015b7d0 0x7feb80160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb84bb7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8015bbb0 con 0x7feb800b5620 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb84bb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8006d230 con 0x7feb8015b7d0 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb84bb7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8015bd30 con 0x7feb8006c2d0 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb800b5620 0x7feb800c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb7dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb800b5620 0x7feb800c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50622/0 (socket says 172.21.15.19:50622) 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb7dd9b640 1 -- 172.21.15.19:0/1467513965 learned_addr learned my addr 172.21.15.19:0/1467513965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb7ed9d640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb8015b7d0 0x7feb80160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb7e59c640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb8006c2d0 0x7feb80157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb7dd9b640 1 -- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb8006c2d0 msgr2=0x7feb80157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb7dd9b640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb8006c2d0 0x7feb80157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb7dd9b640 1 -- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb8015b7d0 msgr2=0x7feb80160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb7dd9b640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb8015b7d0 0x7feb80160050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb7dd9b640 1 -- 172.21.15.19:0/1467513965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb800cd050 con 0x7feb800b5620 2024-04-25T09:57:17.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.069+0000 7feb7e59c640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb8006c2d0 0x7feb80157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:17.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.070+0000 7feb7dd9b640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb800b5620 0x7feb800c9ca0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7feb74000c00 tx=0x7feb740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.070+0000 7feb6effd640 1 -- 172.21.15.19:0/1467513965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74037040 con 0x7feb800b5620 2024-04-25T09:57:17.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.070+0000 7feb6effd640 1 -- 172.21.15.19:0/1467513965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb74042070 con 0x7feb800b5620 2024-04-25T09:57:17.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.070+0000 7feb84bb7640 1 -- 172.21.15.19:0/1467513965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb800cd330 con 0x7feb800b5620 2024-04-25T09:57:17.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.070+0000 7feb6effd640 1 -- 172.21.15.19:0/1467513965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74030960 con 0x7feb800b5620 2024-04-25T09:57:17.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.070+0000 7feb84bb7640 1 -- 172.21.15.19:0/1467513965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb800cdbd0 con 0x7feb800b5620 2024-04-25T09:57:17.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.071+0000 7feb84bb7640 1 -- 172.21.15.19:0/1467513965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb800cd500 con 0x7feb800b5620 2024-04-25T09:57:17.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.071+0000 7feb6effd640 1 -- 172.21.15.19:0/1467513965 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb74030b00 con 0x7feb800b5620 2024-04-25T09:57:17.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.072+0000 7feb6effd640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6006b4d0 0x7feb6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.072+0000 7feb6effd640 1 -- 172.21.15.19:0/1467513965 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6475+0+0 (secure 0 0 0) 0x7feb740aad90 con 0x7feb800b5620 2024-04-25T09:57:17.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.072+0000 7feb7e59c640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6006b4d0 0x7feb6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.072+0000 7feb7e59c640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6006b4d0 0x7feb6006d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feb70000f60 tx=0x7feb70007ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.076+0000 7feb6effd640 1 -- 172.21.15.19:0/1467513965 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb74079910 con 0x7feb800b5620 2024-04-25T09:57:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.255+0000 7feb84bb7640 1 -- 172.21.15.19:0/1467513965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feb80073a00 con 0x7feb800b5620 2024-04-25T09:57:17.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.900+0000 7feb6effd640 1 -- 172.21.15.19:0/1467513965 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) v1 ==== 181+0+0 (secure 0 0 0) 0x7feb74079210 con 0x7feb800b5620 2024-04-25T09:57:17.904 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:57:17.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.904+0000 7feb84bb7640 1 -- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6006b4d0 msgr2=0x7feb6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.904+0000 7feb84bb7640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6006b4d0 0x7feb6006d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feb70000f60 tx=0x7feb70007ca0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.904+0000 7feb84bb7640 1 -- 172.21.15.19:0/1467513965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb800b5620 msgr2=0x7feb800c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.904+0000 7feb84bb7640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb800b5620 0x7feb800c9ca0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7feb74000c00 tx=0x7feb740046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.905+0000 7feb84bb7640 1 -- 172.21.15.19:0/1467513965 shutdown_connections 2024-04-25T09:57:17.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.905+0000 7feb84bb7640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6006b4d0 0x7feb6006d990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.905+0000 7feb84bb7640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb8015b7d0 0x7feb80160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.905+0000 7feb84bb7640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb800b5620 0x7feb800c9ca0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.905+0000 7feb84bb7640 1 --2- 172.21.15.19:0/1467513965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb8006c2d0 0x7feb80157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.905+0000 7feb84bb7640 1 -- 172.21.15.19:0/1467513965 >> 172.21.15.19:0/1467513965 conn(0x7feb800d0800 msgr2=0x7feb8006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:17.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.905+0000 7feb84bb7640 1 -- 172.21.15.19:0/1467513965 shutdown_connections 2024-04-25T09:57:17.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:17.905+0000 7feb84bb7640 1 -- 172.21.15.19:0/1467513965 wait complete. 2024-04-25T09:57:17.923 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:57:18.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.031+0000 7ffa5d728640 1 Processor -- start 2024-04-25T09:57:18.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.031+0000 7ffa5d728640 1 -- start start 2024-04-25T09:57:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.032+0000 7ffa5d728640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa5815b7d0 0x7ffa580cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.032+0000 7ffa5d728640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffa5806c9e0 0x7ffa580b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.032+0000 7ffa5d728640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5806cf50 con 0x7ffa580b5620 2024-04-25T09:57:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.032+0000 7ffa5d728640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa580b5fc0 con 0x7ffa5806c9e0 2024-04-25T09:57:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.032+0000 7ffa5d728640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa580ce530 con 0x7ffa5815b7d0 2024-04-25T09:57:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.032+0000 7ffa567fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ffa580b5620 0x7ffa5806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39874/0 (socket says 172.21.15.19:39874) 2024-04-25T09:57:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.032+0000 7ffa567fc640 1 -- 172.21.15.19:0/3786423613 learned_addr learned my addr 172.21.15.19:0/3786423613 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.032+0000 7ffa56ffd640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa5815b7d0 0x7ffa580cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.032+0000 7ffa577fe640 1 --2- 172.21.15.19:0/3786423613 >> v2:172.21.15.112:3300/0 conn(0x7ffa5806c9e0 0x7ffa580b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.032+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771761959 0 0) 0x7ffa5806cf50 con 0x7ffa580b5620 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.032+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa44003520 con 0x7ffa580b5620 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 625661043 0 0) 0x7ffa44003520 con 0x7ffa580b5620 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa5815b7d0 msgr2=0x7ffa580cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa5815b7d0 0x7ffa580cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 >> v2:172.21.15.112:3300/0 conn(0x7ffa5806c9e0 msgr2=0x7ffa580b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 --2- 172.21.15.19:0/3786423613 >> v2:172.21.15.112:3300/0 conn(0x7ffa5806c9e0 0x7ffa580b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580ce810 con 0x7ffa580b5620 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7ffa40001670 con 0x7ffa580b5620 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 >> v1:172.21.15.19:6789/0 conn(0x7ffa580b5620 legacy=0x7ffa5806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa440039b0 0x7ffa44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa5815b7d0 0x7ffa44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44004d40 0x7ffa440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa580ce530 con 0x7ffa440039b0 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4400b9c0 con 0x7ffa44004d40 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4400bb40 con 0x7ffa5815b7d0 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa56ffd640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa440039b0 0x7ffa44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa567fc640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa5815b7d0 0x7ffa44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa577fe640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44004d40 0x7ffa440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa56ffd640 1 -- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa5815b7d0 msgr2=0x7ffa44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa56ffd640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa5815b7d0 0x7ffa44004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa56ffd640 1 -- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44004d40 msgr2=0x7ffa440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa56ffd640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44004d40 0x7ffa440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa56ffd640 1 -- 172.21.15.19:0/3786423613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa44003520 con 0x7ffa440039b0 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.033+0000 7ffa577fe640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa44004d40 0x7ffa440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.034+0000 7ffa56ffd640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa440039b0 0x7ffa44003dd0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009ac0 tx=0x7ffa44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5d37d786872231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.034+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50036070 con 0x7ffa440039b0 2024-04-25T09:57:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.034+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa50031400 con 0x7ffa440039b0 2024-04-25T09:57:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.034+0000 7ffa55ffb640 1 -- 172.21.15.19:0/3786423613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50009690 con 0x7ffa440039b0 2024-04-25T09:57:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.035+0000 7ffa5d728640 1 -- 172.21.15.19:0/3786423613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa440039b0 msgr2=0x7ffa44003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.035+0000 7ffa5d728640 1 --2- 172.21.15.19:0/3786423613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa440039b0 0x7ffa44003dd0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009ac0 tx=0x7ffa44005830 comp rx=0 tx=0).stop 2024-04-25T09:57:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.035+0000 7ffa577fe640 1 -- 172.21.15.19:0/3786423613 reap_dead start 2024-04-25T09:57:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.035+0000 7ffa5d728640 1 -- 172.21.15.19:0/3786423613 shutdown_connections 2024-04-25T09:57:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.035+0000 7ffa5d728640 1 -- 172.21.15.19:0/3786423613 >> 172.21.15.19:0/3786423613 conn(0x7ffa580d0800 msgr2=0x7ffa580d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.035+0000 7ffa5d728640 1 -- 172.21.15.19:0/3786423613 shutdown_connections 2024-04-25T09:57:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.035+0000 7ffa5d728640 1 -- 172.21.15.19:0/3786423613 wait complete. 2024-04-25T09:57:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.036+0000 7ffa5d728640 1 Processor -- start 2024-04-25T09:57:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.036+0000 7ffa5d728640 1 -- start start 2024-04-25T09:57:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa5d728640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa440039b0 0x7ffa5815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa5d728640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa58159320 0x7ffa58158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa5d728640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa580b81a0 0x7ffa58158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa5d728640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5815bbb0 con 0x7ffa58159320 2024-04-25T09:57:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa5d728640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5806d230 con 0x7ffa580b81a0 2024-04-25T09:57:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa5d728640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5815bd30 con 0x7ffa440039b0 2024-04-25T09:57:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa567fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa58159320 0x7ffa58158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa567fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa58159320 0x7ffa58158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44448/0 (socket says 172.21.15.19:44448) 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa567fc640 1 -- 172.21.15.19:0/3342635958 learned_addr learned my addr 172.21.15.19:0/3342635958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa56ffd640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa440039b0 0x7ffa5815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa577fe640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa580b81a0 0x7ffa58158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa567fc640 1 -- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa440039b0 msgr2=0x7ffa5815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa567fc640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa440039b0 0x7ffa5815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa567fc640 1 -- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa580b81a0 msgr2=0x7ffa58158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa567fc640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa580b81a0 0x7ffa58158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.037+0000 7ffa567fc640 1 -- 172.21.15.19:0/3342635958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580b8630 con 0x7ffa58159320 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.038+0000 7ffa567fc640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa58159320 0x7ffa58158110 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ffa40007310 tx=0x7ffa400077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.038+0000 7ffa4f7fe640 1 -- 172.21.15.19:0/3342635958 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4000fa80 con 0x7ffa58159320 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.038+0000 7ffa4f7fe640 1 -- 172.21.15.19:0/3342635958 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa40018070 con 0x7ffa58159320 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.038+0000 7ffa5d728640 1 -- 172.21.15.19:0/3342635958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa580b8910 con 0x7ffa58159320 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.038+0000 7ffa4f7fe640 1 -- 172.21.15.19:0/3342635958 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4000fea0 con 0x7ffa58159320 2024-04-25T09:57:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.038+0000 7ffa5d728640 1 -- 172.21.15.19:0/3342635958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa580b8ed0 con 0x7ffa58159320 2024-04-25T09:57:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.039+0000 7ffa5d728640 1 -- 172.21.15.19:0/3342635958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa580cad20 con 0x7ffa58159320 2024-04-25T09:57:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.039+0000 7ffa4f7fe640 1 -- 172.21.15.19:0/3342635958 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa4001d460 con 0x7ffa58159320 2024-04-25T09:57:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.040+0000 7ffa4f7fe640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa3806b4d0 0x7ffa3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.040+0000 7ffa4f7fe640 1 -- 172.21.15.19:0/3342635958 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6008+0+0 (secure 0 0 0) 0x7ffa4008a290 con 0x7ffa58159320 2024-04-25T09:57:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.040+0000 7ffa56ffd640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa3806b4d0 0x7ffa3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.040+0000 7ffa56ffd640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa3806b4d0 0x7ffa3806d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffa50033000 tx=0x7ffa50008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.044+0000 7ffa4f7fe640 1 -- 172.21.15.19:0/3342635958 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa40058a20 con 0x7ffa58159320 2024-04-25T09:57:18.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.227+0000 7ffa5d728640 1 -- 172.21.15.19:0/3342635958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffa581602e0 con 0x7ffa58159320 2024-04-25T09:57:18.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.902+0000 7ffa4f7fe640 1 -- 172.21.15.19:0/3342635958 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) v1 ==== 169+0+0 (secure 0 0 0) 0x7ffa4005cac0 con 0x7ffa58159320 2024-04-25T09:57:18.905 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T09:57:18.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.905+0000 7ffa5d728640 1 -- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa3806b4d0 msgr2=0x7ffa3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.905+0000 7ffa5d728640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa3806b4d0 0x7ffa3806d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffa50033000 tx=0x7ffa50008030 comp rx=0 tx=0).stop 2024-04-25T09:57:18.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.905+0000 7ffa5d728640 1 -- 172.21.15.19:0/3342635958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa58159320 msgr2=0x7ffa58158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.905+0000 7ffa5d728640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa58159320 0x7ffa58158110 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ffa40007310 tx=0x7ffa400077d0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.906+0000 7ffa5d728640 1 -- 172.21.15.19:0/3342635958 shutdown_connections 2024-04-25T09:57:18.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.906+0000 7ffa5d728640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa3806b4d0 0x7ffa3806d990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.906+0000 7ffa5d728640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa580b81a0 0x7ffa58158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.906+0000 7ffa5d728640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa58159320 0x7ffa58158110 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.906+0000 7ffa5d728640 1 --2- 172.21.15.19:0/3342635958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa440039b0 0x7ffa5815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.906+0000 7ffa5d728640 1 -- 172.21.15.19:0/3342635958 >> 172.21.15.19:0/3342635958 conn(0x7ffa580d0800 msgr2=0x7ffa5815f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:18.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.906+0000 7ffa5d728640 1 -- 172.21.15.19:0/3342635958 shutdown_connections 2024-04-25T09:57:18.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:18.906+0000 7ffa5d728640 1 -- 172.21.15.19:0/3342635958 wait complete. 2024-04-25T09:57:18.925 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:57:19.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.030+0000 7fe7f9a2a640 1 Processor -- start 2024-04-25T09:57:19.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.030+0000 7fe7f9a2a640 1 -- start start 2024-04-25T09:57:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.031+0000 7fe7f9a2a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7f415bb50 0x7fe7f40c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.031+0000 7fe7f9a2a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe7f40cfd10 0x7fe7f40cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.031+0000 7fe7f9a2a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f40c75e0 con 0x7fe7f40c70d0 2024-04-25T09:57:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.031+0000 7fe7f9a2a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f40c8680 con 0x7fe7f415bb50 2024-04-25T09:57:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.031+0000 7fe7f9a2a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f40c78c0 con 0x7fe7f40cfd10 2024-04-25T09:57:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.031+0000 7fe7f2ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7f415bb50 0x7fe7f40c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.031+0000 7fe7f2ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7f415bb50 0x7fe7f40c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44454/0 (socket says 172.21.15.19:44454) 2024-04-25T09:57:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.031+0000 7fe7f2ffd640 1 -- 172.21.15.19:0/3382214205 learned_addr learned my addr 172.21.15.19:0/3382214205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.032+0000 7fe7f2ffd640 1 -- 172.21.15.19:0/3382214205 >> v2:172.21.15.112:3300/0 conn(0x7fe7f40cfd10 msgr2=0x7fe7f40cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.032+0000 7fe7f2ffd640 1 --2- 172.21.15.19:0/3382214205 >> v2:172.21.15.112:3300/0 conn(0x7fe7f40cfd10 0x7fe7f40cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.032+0000 7fe7f2ffd640 1 -- 172.21.15.19:0/3382214205 >> v1:172.21.15.112:6789/0 conn(0x7fe7f40c70d0 legacy=0x7fe7f40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.032+0000 7fe7f2ffd640 1 -- 172.21.15.19:0/3382214205 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f40c7ba0 con 0x7fe7f415bb50 2024-04-25T09:57:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.032+0000 7fe7f2ffd640 1 --2- 172.21.15.19:0/3382214205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f415bb50 0x7fe7f40c6b90 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4071f40 tx=0x7fe7ec02a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69da2b1d3d220234 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.033+0000 7fe7f1ffb640 1 -- 172.21.15.19:0/3382214205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec037070 con 0x7fe7f415bb50 2024-04-25T09:57:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.033+0000 7fe7f1ffb640 1 -- 172.21.15.19:0/3382214205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ec032440 con 0x7fe7f415bb50 2024-04-25T09:57:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.033+0000 7fe7f1ffb640 1 -- 172.21.15.19:0/3382214205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec032810 con 0x7fe7f415bb50 2024-04-25T09:57:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.033+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/3382214205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f415bb50 msgr2=0x7fe7f40c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.033+0000 7fe7f9a2a640 1 --2- 172.21.15.19:0/3382214205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f415bb50 0x7fe7f40c6b90 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4071f40 tx=0x7fe7ec02a2e0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.033+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/3382214205 shutdown_connections 2024-04-25T09:57:19.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.033+0000 7fe7f9a2a640 1 --2- 172.21.15.19:0/3382214205 >> v2:172.21.15.112:3300/0 conn(0x7fe7f40cfd10 0x7fe7f40cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.033+0000 7fe7f9a2a640 1 --2- 172.21.15.19:0/3382214205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f415bb50 0x7fe7f40c6b90 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.033+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/3382214205 >> 172.21.15.19:0/3382214205 conn(0x7fe7f406f500 msgr2=0x7fe7f406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.033+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/3382214205 shutdown_connections 2024-04-25T09:57:19.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.033+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/3382214205 wait complete. 2024-04-25T09:57:19.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.034+0000 7fe7f9a2a640 1 Processor -- start 2024-04-25T09:57:19.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.035+0000 7fe7f9a2a640 1 -- start start 2024-04-25T09:57:19.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.035+0000 7fe7f9a2a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7f40c70d0 0x7fe7f40b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.035+0000 7fe7f9a2a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f40cfd10 0x7fe7f415d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.035+0000 7fe7f9a2a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7f415bb50 0x7fe7f40c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.035+0000 7fe7f9a2a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f415f6d0 con 0x7fe7f40cfd10 2024-04-25T09:57:19.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.035+0000 7fe7f9a2a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f40c8160 con 0x7fe7f40c70d0 2024-04-25T09:57:19.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.035+0000 7fe7f9a2a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f415f850 con 0x7fe7f415bb50 2024-04-25T09:57:19.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.035+0000 7fe7f27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f40cfd10 0x7fe7f415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f40cfd10 0x7fe7f415d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44466/0 (socket says 172.21.15.19:44466) 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f27fc640 1 -- 172.21.15.19:0/121978339 learned_addr learned my addr 172.21.15.19:0/121978339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f2ffd640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7f40c70d0 0x7fe7f40b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f37fe640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7f415bb50 0x7fe7f40c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f27fc640 1 -- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7f415bb50 msgr2=0x7fe7f40c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f27fc640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7f415bb50 0x7fe7f40c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f27fc640 1 -- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7f40c70d0 msgr2=0x7fe7f40b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f27fc640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7f40c70d0 0x7fe7f40b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f27fc640 1 -- 172.21.15.19:0/121978339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f406cd20 con 0x7fe7f40cfd10 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f27fc640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f40cfd10 0x7fe7f415d820 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c430 tx=0x7fe7dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7ebfff640 1 -- 172.21.15.19:0/121978339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc017070 con 0x7fe7f40cfd10 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7ebfff640 1 -- 172.21.15.19:0/121978339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc0035f0 con 0x7fe7f40cfd10 2024-04-25T09:57:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/121978339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f406d000 con 0x7fe7f40cfd10 2024-04-25T09:57:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7ebfff640 1 -- 172.21.15.19:0/121978339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc0039e0 con 0x7fe7f40cfd10 2024-04-25T09:57:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.036+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/121978339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f40b6440 con 0x7fe7f40cfd10 2024-04-25T09:57:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.037+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/121978339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f4073a00 con 0x7fe7f40cfd10 2024-04-25T09:57:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.038+0000 7fe7ebfff640 1 -- 172.21.15.19:0/121978339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7dc01a070 con 0x7fe7f40cfd10 2024-04-25T09:57:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.038+0000 7fe7ebfff640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d406b4d0 0x7fe7d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.038+0000 7fe7ebfff640 1 -- 172.21.15.19:0/121978339 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe7dc0894f0 con 0x7fe7f40cfd10 2024-04-25T09:57:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.039+0000 7fe7f2ffd640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d406b4d0 0x7fe7d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.039+0000 7fe7f2ffd640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d406b4d0 0x7fe7d406d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec034000 tx=0x7fe7ec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.042+0000 7fe7ebfff640 1 -- 172.21.15.19:0/121978339 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7dc057d40 con 0x7fe7f40cfd10 2024-04-25T09:57:19.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.232+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/121978339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe7f40cf150 con 0x7fe7f40cfd10 2024-04-25T09:57:19.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.235+0000 7fe7ebfff640 1 -- 172.21.15.19:0/121978339 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe7dc05bde0 con 0x7fe7f40cfd10 2024-04-25T09:57:19.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d406b4d0 msgr2=0x7fe7d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d406b4d0 0x7fe7d406d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec034000 tx=0x7fe7ec02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:19.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/121978339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f40cfd10 msgr2=0x7fe7f415d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f40cfd10 0x7fe7f415d820 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c430 tx=0x7fe7dc00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/121978339 shutdown_connections 2024-04-25T09:57:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe7d406b4d0 0x7fe7d406d990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7f415bb50 0x7fe7f40c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f40cfd10 0x7fe7f415d820 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 --2- 172.21.15.19:0/121978339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7f40c70d0 0x7fe7f40b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/121978339 >> 172.21.15.19:0/121978339 conn(0x7fe7f406f500 msgr2=0x7fe7f40ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/121978339 shutdown_connections 2024-04-25T09:57:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.237+0000 7fe7f9a2a640 1 -- 172.21.15.19:0/121978339 wait complete. 2024-04-25T09:57:19.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T09:57:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.348+0000 7fb64805d640 1 Processor -- start 2024-04-25T09:57:19.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.348+0000 7fb64805d640 1 -- start start 2024-04-25T09:57:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.349+0000 7fb64805d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6400d1020 0x7fb6400d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.349+0000 7fb64805d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6400b97a0 con 0x7fb6400b2cb0 2024-04-25T09:57:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.349+0000 7fb64805d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6400d1c50 con 0x7fb64015dbe0 2024-04-25T09:57:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.349+0000 7fb64805d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64015dfc0 con 0x7fb6400d1020 2024-04-25T09:57:19.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.349+0000 7fb6455f8640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb6400b2cb0 0x7fb6400d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39886/0 (socket says 172.21.15.19:39886) 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.349+0000 7fb6455f8640 1 -- 172.21.15.19:0/1279221768 learned_addr learned my addr 172.21.15.19:0/1279221768 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577238001 0 0) 0x7fb6400b97a0 con 0x7fb6400b2cb0 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb62c003520 con 0x7fb6400b2cb0 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb6465fa640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6400d1020 0x7fb6400d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3111039318 0 0) 0x7fb62c003520 con 0x7fb6400b2cb0 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6400d1020 msgr2=0x7fb6400d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6400d1020 0x7fb6400d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 >> v1:172.21.15.112:6789/0 conn(0x7fb64015dbe0 legacy=0x7fb6400d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6400d1f30 con 0x7fb6400b2cb0 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fb630001670 con 0x7fb6400b2cb0 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 >> v1:172.21.15.19:6789/0 conn(0x7fb6400b2cb0 legacy=0x7fb6400d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb62c0039e0 0x7fb62c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 0x7fb62c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6400d1020 0x7fb62c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c00ba70 con 0x7fb62c004800 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c00bc20 con 0x7fb62c0039e0 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c00be20 con 0x7fb6400d1020 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb6455f8640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 0x7fb62c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb645df9640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb62c0039e0 0x7fb62c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb6465fa640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6400d1020 0x7fb62c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb6455f8640 1 -- 172.21.15.19:0/1279221768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6400d1020 msgr2=0x7fb62c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb6455f8640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6400d1020 0x7fb62c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb6455f8640 1 -- 172.21.15.19:0/1279221768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb62c0039e0 msgr2=0x7fb62c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb6455f8640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb62c0039e0 0x7fb62c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.350+0000 7fb6455f8640 1 -- 172.21.15.19:0/1279221768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c003520 con 0x7fb62c004800 2024-04-25T09:57:19.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.351+0000 7fb6455f8640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 0x7fb62c004c80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb6400d3930 tx=0x7fb62c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58d8fd76b8aa8ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.351+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630007d10 con 0x7fb62c004800 2024-04-25T09:57:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.351+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb630005ad0 con 0x7fb62c004800 2024-04-25T09:57:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.351+0000 7fb644df7640 1 -- 172.21.15.19:0/1279221768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630007070 con 0x7fb62c004800 2024-04-25T09:57:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.352+0000 7fb64805d640 1 -- 172.21.15.19:0/1279221768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 msgr2=0x7fb62c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.352+0000 7fb64805d640 1 --2- 172.21.15.19:0/1279221768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 0x7fb62c004c80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb6400d3930 tx=0x7fb62c005830 comp rx=0 tx=0).stop 2024-04-25T09:57:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.352+0000 7fb6465fa640 1 -- 172.21.15.19:0/1279221768 reap_dead start 2024-04-25T09:57:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.352+0000 7fb64805d640 1 -- 172.21.15.19:0/1279221768 shutdown_connections 2024-04-25T09:57:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.352+0000 7fb64805d640 1 -- 172.21.15.19:0/1279221768 >> 172.21.15.19:0/1279221768 conn(0x7fb640061a20 msgr2=0x7fb640061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.352+0000 7fb64805d640 1 -- 172.21.15.19:0/1279221768 shutdown_connections 2024-04-25T09:57:19.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.352+0000 7fb64805d640 1 -- 172.21.15.19:0/1279221768 wait complete. 2024-04-25T09:57:19.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.353+0000 7fb64805d640 1 Processor -- start 2024-04-25T09:57:19.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.353+0000 7fb64805d640 1 -- start start 2024-04-25T09:57:19.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb64805d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 0x7fb6400c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb64805d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb64015e9d0 0x7fb6400cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb64805d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb64015ee50 0x7fb640153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb64805d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400b92a0 con 0x7fb62c004800 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb64805d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400d1a00 con 0x7fb64015e9d0 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb64805d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400b9420 con 0x7fb64015ee50 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb645df9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 0x7fb6400c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb645df9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 0x7fb6400c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44490/0 (socket says 172.21.15.19:44490) 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb645df9640 1 -- 172.21.15.19:0/3939473318 learned_addr learned my addr 172.21.15.19:0/3939473318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb6455f8640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb64015e9d0 0x7fb6400cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb6465fa640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb64015ee50 0x7fb640153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb645df9640 1 -- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb64015ee50 msgr2=0x7fb640153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb645df9640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb64015ee50 0x7fb640153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb645df9640 1 -- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb64015e9d0 msgr2=0x7fb6400cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb645df9640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb64015e9d0 0x7fb6400cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb645df9640 1 -- 172.21.15.19:0/3939473318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6400cae10 con 0x7fb62c004800 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.354+0000 7fb645df9640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 0x7fb6400c9cd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00f650 tx=0x7fb63c00fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.355+0000 7fb6367fc640 1 -- 172.21.15.19:0/3939473318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0050d0 con 0x7fb62c004800 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.355+0000 7fb6367fc640 1 -- 172.21.15.19:0/3939473318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c018070 con 0x7fb62c004800 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.355+0000 7fb64805d640 1 -- 172.21.15.19:0/3939473318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6400cb0f0 con 0x7fb62c004800 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.355+0000 7fb6367fc640 1 -- 172.21.15.19:0/3939473318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c005490 con 0x7fb62c004800 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.355+0000 7fb64805d640 1 -- 172.21.15.19:0/3939473318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6400cb6b0 con 0x7fb62c004800 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.356+0000 7fb64805d640 1 -- 172.21.15.19:0/3939473318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb640073990 con 0x7fb62c004800 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.356+0000 7fb6367fc640 1 -- 172.21.15.19:0/3939473318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb63c00d730 con 0x7fb62c004800 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.356+0000 7fb6367fc640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62406b500 0x7fb62406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.357+0000 7fb6367fc640 1 -- 172.21.15.19:0/3939473318 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fb63c089cf0 con 0x7fb62c004800 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.357+0000 7fb6455f8640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62406b500 0x7fb62406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.357+0000 7fb6455f8640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62406b500 0x7fb62406d9c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0057e0 tx=0x7fb63002d000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.361+0000 7fb6367fc640 1 -- 172.21.15.19:0/3939473318 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb63c08e070 con 0x7fb62c004800 2024-04-25T09:57:19.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.614+0000 7fb64805d640 1 -- 172.21.15.19:0/3939473318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7fb6400621a0 con 0x7fb62c004800 2024-04-25T09:57:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.907+0000 7fb6367fc640 1 -- 172.21.15.19:0/3939473318 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v227) v1 ==== 143+0+0 (secure 0 0 0) 0x7fb63c058670 con 0x7fb62c004800 2024-04-25T09:57:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 -- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62406b500 msgr2=0x7fb62406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62406b500 0x7fb62406d9c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0057e0 tx=0x7fb63002d000 comp rx=0 tx=0).stop 2024-04-25T09:57:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 -- 172.21.15.19:0/3939473318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 msgr2=0x7fb6400c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 0x7fb6400c9cd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00f650 tx=0x7fb63c00fb10 comp rx=0 tx=0).stop 2024-04-25T09:57:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 -- 172.21.15.19:0/3939473318 shutdown_connections 2024-04-25T09:57:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb62406b500 0x7fb62406d9c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb64015ee50 0x7fb640153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb64015e9d0 0x7fb6400cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 --2- 172.21.15.19:0/3939473318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb62c004800 0x7fb6400c9cd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 -- 172.21.15.19:0/3939473318 >> 172.21.15.19:0/3939473318 conn(0x7fb640061a20 msgr2=0x7fb6400b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 -- 172.21.15.19:0/3939473318 shutdown_connections 2024-04-25T09:57:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:19.909+0000 7fb64805d640 1 -- 172.21.15.19:0/3939473318 wait complete. 2024-04-25T09:57:19.924 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T09:57:19.924 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T09:57:19.924 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:57:19.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.023+0000 7f66f8815640 1 Processor -- start 2024-04-25T09:57:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.023+0000 7f66f8815640 1 -- start start 2024-04-25T09:57:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.023+0000 7f66f8815640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66f415dc90 0x7f66f41625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.023+0000 7f66f8815640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66f40cc8e0 0x7f66f415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.023+0000 7f66f8815640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f415f390 con 0x7f66f40d1810 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.023+0000 7f66f8815640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f40cce50 con 0x7f66f40cc8e0 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.023+0000 7f66f8815640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4161af0 con 0x7f66f415dc90 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.024+0000 7f66ebfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66f40cc8e0 0x7f66f415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.024+0000 7f66ebfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66f40cc8e0 0x7f66f415ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:35392/0 (socket says 172.21.15.19:35392) 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.024+0000 7f66ebfff640 1 -- 172.21.15.19:0/649371654 learned_addr learned my addr 172.21.15.19:0/649371654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.024+0000 7f66f2d9d640 1 --1- 172.21.15.19:0/649371654 >> v1:172.21.15.112:6789/0 conn(0x7f66f40d1810 0x7f66f40cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:39592/0 (socket says 172.21.15.19:39592) 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.024+0000 7f66f259c640 1 --2- 172.21.15.19:0/649371654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66f415dc90 0x7f66f41625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.024+0000 7f66f259c640 1 -- 172.21.15.19:0/649371654 >> v2:172.21.15.112:3300/0 conn(0x7f66f40cc8e0 msgr2=0x7f66f415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.024+0000 7f66f259c640 1 --2- 172.21.15.19:0/649371654 >> v2:172.21.15.112:3300/0 conn(0x7f66f40cc8e0 0x7f66f415ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.024+0000 7f66f259c640 1 -- 172.21.15.19:0/649371654 >> v1:172.21.15.112:6789/0 conn(0x7f66f40d1810 legacy=0x7f66f40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.024+0000 7f66f259c640 1 -- 172.21.15.19:0/649371654 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4161dd0 con 0x7f66f415dc90 2024-04-25T09:57:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.024+0000 7f66ebfff640 1 --2- 172.21.15.19:0/649371654 >> v2:172.21.15.112:3300/0 conn(0x7f66f40cc8e0 0x7f66f415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:20.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.025+0000 7f66f259c640 1 --2- 172.21.15.19:0/649371654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66f415dc90 0x7f66f41625d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66e40097a0 tx=0x7f66e402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64aaa44eeb6959b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.025+0000 7f66f1d9b640 1 -- 172.21.15.19:0/649371654 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4037070 con 0x7f66f415dc90 2024-04-25T09:57:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.025+0000 7f66f1d9b640 1 -- 172.21.15.19:0/649371654 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e402ae50 con 0x7f66f415dc90 2024-04-25T09:57:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.025+0000 7f66f1d9b640 1 -- 172.21.15.19:0/649371654 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4032660 con 0x7f66f415dc90 2024-04-25T09:57:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.026+0000 7f66f8815640 1 -- 172.21.15.19:0/649371654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66f415dc90 msgr2=0x7f66f41625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.026+0000 7f66f8815640 1 --2- 172.21.15.19:0/649371654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66f415dc90 0x7f66f41625d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66e40097a0 tx=0x7f66e402a100 comp rx=0 tx=0).stop 2024-04-25T09:57:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.026+0000 7f66f8815640 1 -- 172.21.15.19:0/649371654 shutdown_connections 2024-04-25T09:57:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.026+0000 7f66f8815640 1 --2- 172.21.15.19:0/649371654 >> v2:172.21.15.112:3300/0 conn(0x7f66f40cc8e0 0x7f66f415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.026+0000 7f66f8815640 1 --2- 172.21.15.19:0/649371654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66f415dc90 0x7f66f41625d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.026+0000 7f66f8815640 1 -- 172.21.15.19:0/649371654 >> 172.21.15.19:0/649371654 conn(0x7f66f406f590 msgr2=0x7f66f406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.026+0000 7f66f8815640 1 -- 172.21.15.19:0/649371654 shutdown_connections 2024-04-25T09:57:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.026+0000 7f66f8815640 1 -- 172.21.15.19:0/649371654 wait complete. 2024-04-25T09:57:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.027+0000 7f66f8815640 1 Processor -- start 2024-04-25T09:57:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.027+0000 7f66f8815640 1 -- start start 2024-04-25T09:57:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f8815640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cc8e0 0x7f66f41f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f8815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66f40d1810 0x7f66f40ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f8815640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66f415dc90 0x7f66f40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f8815640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f415e070 con 0x7f66f40cc8e0 2024-04-25T09:57:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f8815640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40cd130 con 0x7f66f40d1810 2024-04-25T09:57:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f8815640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f415e1f0 con 0x7f66f415dc90 2024-04-25T09:57:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cc8e0 0x7f66f41f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cc8e0 0x7f66f41f3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44504/0 (socket says 172.21.15.19:44504) 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f259c640 1 -- 172.21.15.19:0/2816123528 learned_addr learned my addr 172.21.15.19:0/2816123528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66ebfff640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66f40d1810 0x7f66f40ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f2d9d640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66f415dc90 0x7f66f40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f259c640 1 -- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66f415dc90 msgr2=0x7f66f40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f259c640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66f415dc90 0x7f66f40cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f259c640 1 -- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66f40d1810 msgr2=0x7f66f40ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f259c640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66f40d1810 0x7f66f40ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f259c640 1 -- 172.21.15.19:0/2816123528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f415a230 con 0x7f66f40cc8e0 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.028+0000 7f66f259c640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cc8e0 0x7f66f41f3970 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f66e4000c00 tx=0x7f66e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.029+0000 7f66eaffd640 1 -- 172.21.15.19:0/2816123528 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4041070 con 0x7f66f40cc8e0 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.029+0000 7f66eaffd640 1 -- 172.21.15.19:0/2816123528 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e403c460 con 0x7f66f40cc8e0 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.029+0000 7f66f8815640 1 -- 172.21.15.19:0/2816123528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f415a510 con 0x7f66f40cc8e0 2024-04-25T09:57:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.029+0000 7f66eaffd640 1 -- 172.21.15.19:0/2816123528 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4037040 con 0x7f66f40cc8e0 2024-04-25T09:57:20.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.029+0000 7f66f8815640 1 -- 172.21.15.19:0/2816123528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f415adb0 con 0x7f66f40cc8e0 2024-04-25T09:57:20.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.030+0000 7f66f8815640 1 -- 172.21.15.19:0/2816123528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f4073990 con 0x7f66f40cc8e0 2024-04-25T09:57:20.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.030+0000 7f66eaffd640 1 -- 172.21.15.19:0/2816123528 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66e4044070 con 0x7f66f40cc8e0 2024-04-25T09:57:20.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.031+0000 7f66eaffd640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66d406b4d0 0x7f66d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.031+0000 7f66eaffd640 1 -- 172.21.15.19:0/2816123528 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f66e40aa650 con 0x7f66f40cc8e0 2024-04-25T09:57:20.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.031+0000 7f66ebfff640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66d406b4d0 0x7f66d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.031+0000 7f66ebfff640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66d406b4d0 0x7f66d406d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f66f4154860 tx=0x7f66dc009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.035+0000 7f66eaffd640 1 -- 172.21.15.19:0/2816123528 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66e4078fd0 con 0x7f66f40cc8e0 2024-04-25T09:57:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.218+0000 7f66f8815640 1 -- 172.21.15.19:0/2816123528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66f40cf6b0 con 0x7f66f40cc8e0 2024-04-25T09:57:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.219+0000 7f66eaffd640 1 -- 172.21.15.19:0/2816123528 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+13540 (secure 0 0 0) 0x7f66e407d070 con 0x7f66f40cc8e0 2024-04-25T09:57:20.222 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:20.223 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":73,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:57:18.901491+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6833/2482429280":"2024-04-26T09:57:15.893941+0000","172.21.15.19:6832/2482429280":"2024-04-26T09:57:15.893941+0000","172.21.15.19:6835/3520485752":"2024-04-26T09:56:54.515283+0000","172.21.15.19:6833/3520485752":"2024-04-26T09:56:54.515283+0000","172.21.15.19:6834/591570980":"2024-04-26T09:56:53.512369+0000","172.21.15.19:6832/591570980":"2024-04-26T09:56:53.512369+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 -- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66d406b4d0 msgr2=0x7f66d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66d406b4d0 0x7f66d406d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f66f4154860 tx=0x7f66dc009240 comp rx=0 tx=0).stop 2024-04-25T09:57:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 -- 172.21.15.19:0/2816123528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cc8e0 msgr2=0x7f66f41f3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cc8e0 0x7f66f41f3970 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f66e4000c00 tx=0x7f66e40046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 -- 172.21.15.19:0/2816123528 shutdown_connections 2024-04-25T09:57:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f66d406b4d0 0x7f66d406d990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66f415dc90 0x7f66f40cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66f40d1810 0x7f66f40ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 --2- 172.21.15.19:0/2816123528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cc8e0 0x7f66f41f3970 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 -- 172.21.15.19:0/2816123528 >> 172.21.15.19:0/2816123528 conn(0x7f66f406f590 msgr2=0x7f66f4155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:20.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 -- 172.21.15.19:0/2816123528 shutdown_connections 2024-04-25T09:57:20.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.221+0000 7f66f8815640 1 -- 172.21.15.19:0/2816123528 wait complete. 2024-04-25T09:57:20.236 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:20.236 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:20.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.336+0000 7f2c569b0640 1 Processor -- start 2024-04-25T09:57:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.336+0000 7f2c569b0640 1 -- start start 2024-04-25T09:57:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.337+0000 7f2c569b0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2c5015b7d0 0x7f2c500cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.337+0000 7f2c569b0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5006cf50 con 0x7f2c500b5620 2024-04-25T09:57:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.337+0000 7f2c569b0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c500ce530 con 0x7f2c5006c9e0 2024-04-25T09:57:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.337+0000 7f2c569b0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c500ce810 con 0x7f2c5015b7d0 2024-04-25T09:57:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.337+0000 7f2c4ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2c5015b7d0 0x7f2c500cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.337+0000 7f2c4f7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2c500b5620 0x7f2c5006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39896/0 (socket says 172.21.15.19:39896) 2024-04-25T09:57:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.337+0000 7f2c4f7fe640 1 -- 172.21.15.19:0/2037013474 learned_addr learned my addr 172.21.15.19:0/2037013474 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.337+0000 7f2c4ffff640 1 -- 172.21.15.19:0/2037013474 >> v1:172.21.15.112:6789/0 conn(0x7f2c5006c9e0 legacy=0x7f2c500d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.337+0000 7f2c4ffff640 1 -- 172.21.15.19:0/2037013474 >> v1:172.21.15.19:6789/0 conn(0x7f2c500b5620 legacy=0x7f2c5006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.337+0000 7f2c4effd640 1 -- 172.21.15.19:0/2037013474 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339321546 0 0) 0x7f2c5006cf50 con 0x7f2c500b5620 2024-04-25T09:57:20.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.337+0000 7f2c4ffff640 1 -- 172.21.15.19:0/2037013474 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500ceaf0 con 0x7f2c5015b7d0 2024-04-25T09:57:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.338+0000 7f2c4ffff640 1 --2- 172.21.15.19:0/2037013474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c5015b7d0 0x7f2c500cf010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2c380097a0 tx=0x7f2c3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed920a1267998ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.338+0000 7f2c4effd640 1 -- 172.21.15.19:0/2037013474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38037070 con 0x7f2c5015b7d0 2024-04-25T09:57:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.338+0000 7f2c4effd640 1 -- 172.21.15.19:0/2037013474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c38032440 con 0x7f2c5015b7d0 2024-04-25T09:57:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.338+0000 7f2c4effd640 1 -- 172.21.15.19:0/2037013474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38032810 con 0x7f2c5015b7d0 2024-04-25T09:57:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.339+0000 7f2c569b0640 1 -- 172.21.15.19:0/2037013474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c5015b7d0 msgr2=0x7f2c500cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.339+0000 7f2c569b0640 1 --2- 172.21.15.19:0/2037013474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c5015b7d0 0x7f2c500cf010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2c380097a0 tx=0x7f2c3802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.339+0000 7f2c569b0640 1 -- 172.21.15.19:0/2037013474 shutdown_connections 2024-04-25T09:57:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.339+0000 7f2c569b0640 1 --2- 172.21.15.19:0/2037013474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c5015b7d0 0x7f2c500cf010 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.339+0000 7f2c569b0640 1 -- 172.21.15.19:0/2037013474 >> 172.21.15.19:0/2037013474 conn(0x7f2c500d0800 msgr2=0x7f2c500d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.339+0000 7f2c569b0640 1 -- 172.21.15.19:0/2037013474 shutdown_connections 2024-04-25T09:57:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.339+0000 7f2c569b0640 1 -- 172.21.15.19:0/2037013474 wait complete. 2024-04-25T09:57:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.340+0000 7f2c569b0640 1 Processor -- start 2024-04-25T09:57:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.340+0000 7f2c569b0640 1 -- start start 2024-04-25T09:57:20.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c569b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c5006c9e0 0x7f2c50159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c569b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c500b5620 0x7f2c500b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c569b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c5015b7d0 0x7f2c500b8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c569b0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500cf6d0 con 0x7f2c500b5620 2024-04-25T09:57:20.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c569b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500cf550 con 0x7f2c5006c9e0 2024-04-25T09:57:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c569b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5015bbb0 con 0x7f2c5015b7d0 2024-04-25T09:57:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c4f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c500b5620 0x7f2c500b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c4f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c500b5620 0x7f2c500b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44528/0 (socket says 172.21.15.19:44528) 2024-04-25T09:57:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c4f7fe640 1 -- 172.21.15.19:0/2202750003 learned_addr learned my addr 172.21.15.19:0/2202750003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c4ffff640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c5006c9e0 0x7f2c50159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c4f7fe640 1 -- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c5015b7d0 msgr2=0x7f2c500b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c54f4d640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c5015b7d0 0x7f2c500b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c4f7fe640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c5015b7d0 0x7f2c500b8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c4f7fe640 1 -- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c5006c9e0 msgr2=0x7f2c50159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c4f7fe640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c5006c9e0 0x7f2c50159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c4f7fe640 1 -- 172.21.15.19:0/2202750003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500cad10 con 0x7f2c500b5620 2024-04-25T09:57:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.341+0000 7f2c4ffff640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c5006c9e0 0x7f2c50159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.342+0000 7f2c4f7fe640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c500b5620 0x7f2c500b8050 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c4d0 tx=0x7f2c4000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.342+0000 7f2c4cff9640 1 -- 172.21.15.19:0/2202750003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40018070 con 0x7f2c500b5620 2024-04-25T09:57:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.342+0000 7f2c4cff9640 1 -- 172.21.15.19:0/2202750003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c40007500 con 0x7f2c500b5620 2024-04-25T09:57:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.342+0000 7f2c569b0640 1 -- 172.21.15.19:0/2202750003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c500caff0 con 0x7f2c500b5620 2024-04-25T09:57:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.342+0000 7f2c4cff9640 1 -- 172.21.15.19:0/2202750003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40004a20 con 0x7f2c500b5620 2024-04-25T09:57:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.342+0000 7f2c569b0640 1 -- 172.21.15.19:0/2202750003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c501f4710 con 0x7f2c500b5620 2024-04-25T09:57:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.343+0000 7f2c569b0640 1 -- 172.21.15.19:0/2202750003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c50073a00 con 0x7f2c500b5620 2024-04-25T09:57:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.343+0000 7f2c4cff9640 1 -- 172.21.15.19:0/2202750003 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c4001b070 con 0x7f2c500b5620 2024-04-25T09:57:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.344+0000 7f2c4cff9640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c3006b4d0 0x7f2c3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.344+0000 7f2c4cff9640 1 -- 172.21.15.19:0/2202750003 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f2c40089480 con 0x7f2c500b5620 2024-04-25T09:57:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.344+0000 7f2c4ffff640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c3006b4d0 0x7f2c3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.345+0000 7f2c4ffff640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c3006b4d0 0x7f2c3006d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2c38034000 tx=0x7f2c3802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.348+0000 7f2c4cff9640 1 -- 172.21.15.19:0/2202750003 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c400583c0 con 0x7f2c500b5620 2024-04-25T09:57:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.526+0000 7f2c569b0640 1 -- 172.21.15.19:0/2202750003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c500b9470 con 0x7f2c500b5620 2024-04-25T09:57:20.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.526+0000 7f2c4cff9640 1 -- 172.21.15.19:0/2202750003 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c40057cc0 con 0x7f2c500b5620 2024-04-25T09:57:20.529 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:57:20.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 -- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c3006b4d0 msgr2=0x7f2c3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c3006b4d0 0x7f2c3006d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2c38034000 tx=0x7f2c3802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:20.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 -- 172.21.15.19:0/2202750003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c500b5620 msgr2=0x7f2c500b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c500b5620 0x7f2c500b8050 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c4d0 tx=0x7f2c4000c990 comp rx=0 tx=0).stop 2024-04-25T09:57:20.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 -- 172.21.15.19:0/2202750003 shutdown_connections 2024-04-25T09:57:20.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c3006b4d0 0x7f2c3006d990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c5015b7d0 0x7f2c500b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c500b5620 0x7f2c500b8050 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 --2- 172.21.15.19:0/2202750003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c5006c9e0 0x7f2c50159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 -- 172.21.15.19:0/2202750003 >> 172.21.15.19:0/2202750003 conn(0x7f2c500d0800 msgr2=0x7f2c50160750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:20.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 -- 172.21.15.19:0/2202750003 shutdown_connections 2024-04-25T09:57:20.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.528+0000 7f2c569b0640 1 -- 172.21.15.19:0/2202750003 wait complete. 2024-04-25T09:57:20.542 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T09:57:20.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.640+0000 7f07a061b640 1 Processor -- start 2024-04-25T09:57:20.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.641+0000 7f07a061b640 1 -- start start 2024-04-25T09:57:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.641+0000 7f07a061b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f079813d190 0x7f07980cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.641+0000 7f07a061b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f079815b320 0x7f07980cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.641+0000 7f07a061b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0798071ed0 con 0x7f0798071230 2024-04-25T09:57:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.642+0000 7f07a061b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079813d570 con 0x7f079813d190 2024-04-25T09:57:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.642+0000 7f07a061b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079815a840 con 0x7f079815b320 2024-04-25T09:57:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.642+0000 7f079e3b7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f079813d190 0x7f07980cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.642+0000 7f079e3b7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f079813d190 0x7f07980cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44530/0 (socket says 172.21.15.19:44530) 2024-04-25T09:57:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.642+0000 7f079e3b7640 1 -- 172.21.15.19:0/3980060125 learned_addr learned my addr 172.21.15.19:0/3980060125 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.642+0000 7f079dbb6640 1 --2- 172.21.15.19:0/3980060125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f079815b320 0x7f07980cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.642+0000 7f079e3b7640 1 -- 172.21.15.19:0/3980060125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f079815b320 msgr2=0x7f07980cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.642+0000 7f079e3b7640 1 --2- 172.21.15.19:0/3980060125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f079815b320 0x7f07980cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.642+0000 7f079e3b7640 1 -- 172.21.15.19:0/3980060125 >> v1:172.21.15.112:6789/0 conn(0x7f0798071230 legacy=0x7f07980716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.642+0000 7f079e3b7640 1 -- 172.21.15.19:0/3980060125 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079815ab20 con 0x7f079813d190 2024-04-25T09:57:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.643+0000 7f079e3b7640 1 --2- 172.21.15.19:0/3980060125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079813d190 0x7f07980cbfb0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0794009870 tx=0x7f079402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cc6d0406e2534f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.643+0000 7f079d3b5640 1 -- 172.21.15.19:0/3980060125 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794037070 con 0x7f079813d190 2024-04-25T09:57:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.643+0000 7f079d3b5640 1 -- 172.21.15.19:0/3980060125 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0794032440 con 0x7f079813d190 2024-04-25T09:57:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.643+0000 7f079d3b5640 1 -- 172.21.15.19:0/3980060125 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794032810 con 0x7f079813d190 2024-04-25T09:57:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.643+0000 7f07a061b640 1 -- 172.21.15.19:0/3980060125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079813d190 msgr2=0x7f07980cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.643+0000 7f07a061b640 1 --2- 172.21.15.19:0/3980060125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079813d190 0x7f07980cbfb0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0794009870 tx=0x7f079402a1c0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.644+0000 7f07a061b640 1 -- 172.21.15.19:0/3980060125 shutdown_connections 2024-04-25T09:57:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.644+0000 7f07a061b640 1 --2- 172.21.15.19:0/3980060125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f079815b320 0x7f07980cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.644+0000 7f07a061b640 1 --2- 172.21.15.19:0/3980060125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079813d190 0x7f07980cbfb0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.644+0000 7f07a061b640 1 -- 172.21.15.19:0/3980060125 >> 172.21.15.19:0/3980060125 conn(0x7f079806abe0 msgr2=0x7f079806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.644+0000 7f07a061b640 1 -- 172.21.15.19:0/3980060125 shutdown_connections 2024-04-25T09:57:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.644+0000 7f07a061b640 1 -- 172.21.15.19:0/3980060125 wait complete. 2024-04-25T09:57:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.645+0000 7f07a061b640 1 Processor -- start 2024-04-25T09:57:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.645+0000 7f07a061b640 1 -- start start 2024-04-25T09:57:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.645+0000 7f07a061b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0798071230 0x7f079813a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.645+0000 7f07a061b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f079813d190 0x7f07981594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.645+0000 7f07a061b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079815b320 0x7f0798159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.645+0000 7f07a061b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079815dda0 con 0x7f079815b320 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.645+0000 7f07a061b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079815dc20 con 0x7f0798071230 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.645+0000 7f07a061b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079815df20 con 0x7f079813d190 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079ebb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079815b320 0x7f0798159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079ebb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079815b320 0x7f0798159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44532/0 (socket says 172.21.15.19:44532) 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079ebb8640 1 -- 172.21.15.19:0/2327459812 learned_addr learned my addr 172.21.15.19:0/2327459812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079e3b7640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0798071230 0x7f079813a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079dbb6640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f079813d190 0x7f07981594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079ebb8640 1 -- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f079813d190 msgr2=0x7f07981594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079ebb8640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f079813d190 0x7f07981594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079ebb8640 1 -- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0798071230 msgr2=0x7f079813a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079ebb8640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0798071230 0x7f079813a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079ebb8640 1 -- 172.21.15.19:0/2327459812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079813a580 con 0x7f079815b320 2024-04-25T09:57:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079e3b7640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0798071230 0x7f079813a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f079ebb8640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079815b320 0x7f0798159a10 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f079000c7a0 tx=0x7f079000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f078effd640 1 -- 172.21.15.19:0/2327459812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790003b70 con 0x7f079815b320 2024-04-25T09:57:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f078effd640 1 -- 172.21.15.19:0/2327459812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0790003d30 con 0x7f079815b320 2024-04-25T09:57:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f07a061b640 1 -- 172.21.15.19:0/2327459812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07981e6500 con 0x7f079815b320 2024-04-25T09:57:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f078effd640 1 -- 172.21.15.19:0/2327459812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000aa90 con 0x7f079815b320 2024-04-25T09:57:20.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.646+0000 7f07a061b640 1 -- 172.21.15.19:0/2327459812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07981e5c60 con 0x7f079815b320 2024-04-25T09:57:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.647+0000 7f07a061b640 1 -- 172.21.15.19:0/2327459812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0798073990 con 0x7f079815b320 2024-04-25T09:57:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.648+0000 7f078effd640 1 -- 172.21.15.19:0/2327459812 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f079000ac30 con 0x7f079815b320 2024-04-25T09:57:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.648+0000 7f078effd640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f077c06b4d0 0x7f077c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.648+0000 7f078effd640 1 -- 172.21.15.19:0/2327459812 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f079008a050 con 0x7f079815b320 2024-04-25T09:57:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.648+0000 7f079e3b7640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f077c06b4d0 0x7f077c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.649+0000 7f079e3b7640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f077c06b4d0 0x7f077c06d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0794034000 tx=0x7f079402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.652+0000 7f078effd640 1 -- 172.21.15.19:0/2327459812 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f079813a580 con 0x7f079815b320 2024-04-25T09:57:20.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.905+0000 7f07a061b640 1 -- 172.21.15.19:0/2327459812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f07980cc510 con 0x7f079815b320 2024-04-25T09:57:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.908+0000 7f078effd640 1 -- 172.21.15.19:0/2327459812 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v228) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0790058f90 con 0x7f079815b320 2024-04-25T09:57:20.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 -- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f077c06b4d0 msgr2=0x7f077c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f077c06b4d0 0x7f077c06d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0794034000 tx=0x7f079402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:20.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 -- 172.21.15.19:0/2327459812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079815b320 msgr2=0x7f0798159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079815b320 0x7f0798159a10 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f079000c7a0 tx=0x7f079000cc60 comp rx=0 tx=0).stop 2024-04-25T09:57:20.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 -- 172.21.15.19:0/2327459812 shutdown_connections 2024-04-25T09:57:20.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f077c06b4d0 0x7f077c06d990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079815b320 0x7f0798159a10 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f079813d190 0x7f07981594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 --2- 172.21.15.19:0/2327459812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0798071230 0x7f079813a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 -- 172.21.15.19:0/2327459812 >> 172.21.15.19:0/2327459812 conn(0x7f079806abe0 msgr2=0x7f079815b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:20.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 -- 172.21.15.19:0/2327459812 shutdown_connections 2024-04-25T09:57:20.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:20.910+0000 7f07a061b640 1 -- 172.21.15.19:0/2327459812 wait complete. 2024-04-25T09:57:20.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:57:21.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.024+0000 7f9c8801f640 1 Processor -- start 2024-04-25T09:57:21.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.025+0000 7f9c8801f640 1 -- start start 2024-04-25T09:57:21.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.025+0000 7f9c8801f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9c8015b7d0 0x7f9c800cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.025+0000 7f9c8801f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c800b5620 0x7f9c800b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.025+0000 7f9c8801f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8006c2d0 0x7f9c8006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.025+0000 7f9c8801f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8006cc80 con 0x7f9c800b5620 2024-04-25T09:57:21.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.025+0000 7f9c8801f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8006cf50 con 0x7f9c8015b7d0 2024-04-25T09:57:21.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.025+0000 7f9c8801f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800ce530 con 0x7f9c8006c2d0 2024-04-25T09:57:21.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c855ba640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c800b5620 0x7f9c800b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c855ba640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c800b5620 0x7f9c800b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44536/0 (socket says 172.21.15.19:44536) 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c855ba640 1 -- 172.21.15.19:0/2141902319 learned_addr learned my addr 172.21.15.19:0/2141902319 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c855ba640 1 -- 172.21.15.19:0/2141902319 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8006c2d0 msgr2=0x7f9c8006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c865bc640 1 --2- 172.21.15.19:0/2141902319 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8006c2d0 0x7f9c8006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c85dbb640 1 --2- 172.21.15.19:0/2141902319 >> v2:172.21.15.112:3300/0 conn(0x7f9c8015b7d0 0x7f9c800cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c855ba640 1 --2- 172.21.15.19:0/2141902319 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8006c2d0 0x7f9c8006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c855ba640 1 -- 172.21.15.19:0/2141902319 >> v2:172.21.15.112:3300/0 conn(0x7f9c8015b7d0 msgr2=0x7f9c800cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c855ba640 1 --2- 172.21.15.19:0/2141902319 >> v2:172.21.15.112:3300/0 conn(0x7f9c8015b7d0 0x7f9c800cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c855ba640 1 -- 172.21.15.19:0/2141902319 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c800ce810 con 0x7f9c800b5620 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c85dbb640 1 --2- 172.21.15.19:0/2141902319 >> v2:172.21.15.112:3300/0 conn(0x7f9c8015b7d0 0x7f9c800cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.026+0000 7f9c865bc640 1 --2- 172.21.15.19:0/2141902319 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8006c2d0 0x7f9c8006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c855ba640 1 --2- 172.21.15.19:0/2141902319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c800b5620 0x7f9c800b5ab0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9c8008c0d0 tx=0x7f9c7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c477623fe3467f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c84db9640 1 -- 172.21.15.19:0/2141902319 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70037070 con 0x7f9c800b5620 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c84db9640 1 -- 172.21.15.19:0/2141902319 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c70032440 con 0x7f9c800b5620 2024-04-25T09:57:21.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c84db9640 1 -- 172.21.15.19:0/2141902319 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70032810 con 0x7f9c800b5620 2024-04-25T09:57:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c8801f640 1 -- 172.21.15.19:0/2141902319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c800b5620 msgr2=0x7f9c800b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c8801f640 1 --2- 172.21.15.19:0/2141902319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c800b5620 0x7f9c800b5ab0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9c8008c0d0 tx=0x7f9c7002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c8801f640 1 -- 172.21.15.19:0/2141902319 shutdown_connections 2024-04-25T09:57:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c8801f640 1 --2- 172.21.15.19:0/2141902319 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c8006c2d0 0x7f9c8006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c8801f640 1 --2- 172.21.15.19:0/2141902319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c800b5620 0x7f9c800b5ab0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c8801f640 1 --2- 172.21.15.19:0/2141902319 >> v2:172.21.15.112:3300/0 conn(0x7f9c8015b7d0 0x7f9c800cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c8801f640 1 -- 172.21.15.19:0/2141902319 >> 172.21.15.19:0/2141902319 conn(0x7f9c800d0800 msgr2=0x7f9c800d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c8801f640 1 -- 172.21.15.19:0/2141902319 shutdown_connections 2024-04-25T09:57:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.027+0000 7f9c8801f640 1 -- 172.21.15.19:0/2141902319 wait complete. 2024-04-25T09:57:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.029+0000 7f9c8801f640 1 Processor -- start 2024-04-25T09:57:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.029+0000 7f9c8801f640 1 -- start start 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.029+0000 7f9c8801f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8006c2d0 0x7f9c80159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.029+0000 7f9c8801f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c800b5620 0x7f9c800c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.029+0000 7f9c8801f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c8015b7d0 0x7f9c800cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.029+0000 7f9c8801f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800cf550 con 0x7f9c8006c2d0 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.029+0000 7f9c8801f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8006d230 con 0x7f9c8015b7d0 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.029+0000 7f9c8801f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800cf6d0 con 0x7f9c800b5620 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.029+0000 7f9c85dbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8006c2d0 0x7f9c80159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.029+0000 7f9c85dbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8006c2d0 0x7f9c80159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44544/0 (socket says 172.21.15.19:44544) 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.029+0000 7f9c85dbb640 1 -- 172.21.15.19:0/2796518966 learned_addr learned my addr 172.21.15.19:0/2796518966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.030+0000 7f9c85dbb640 1 -- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c800b5620 msgr2=0x7f9c800c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.030+0000 7f9c85dbb640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c800b5620 0x7f9c800c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.030+0000 7f9c85dbb640 1 -- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c8015b7d0 msgr2=0x7f9c800cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.030+0000 7f9c85dbb640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c8015b7d0 0x7f9c800cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.030+0000 7f9c85dbb640 1 -- 172.21.15.19:0/2796518966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c800cb680 con 0x7f9c8006c2d0 2024-04-25T09:57:21.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.030+0000 7f9c85dbb640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8006c2d0 0x7f9c80159120 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c7f0 tx=0x7f9c7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.030+0000 7f9c767fc640 1 -- 172.21.15.19:0/2796518966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c015070 con 0x7f9c8006c2d0 2024-04-25T09:57:21.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.030+0000 7f9c767fc640 1 -- 172.21.15.19:0/2796518966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c007820 con 0x7f9c8006c2d0 2024-04-25T09:57:21.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.030+0000 7f9c767fc640 1 -- 172.21.15.19:0/2796518966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c007bd0 con 0x7f9c8006c2d0 2024-04-25T09:57:21.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.030+0000 7f9c8801f640 1 -- 172.21.15.19:0/2796518966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c800cb960 con 0x7f9c8006c2d0 2024-04-25T09:57:21.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.030+0000 7f9c8801f640 1 -- 172.21.15.19:0/2796518966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c800cc200 con 0x7f9c8006c2d0 2024-04-25T09:57:21.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.031+0000 7f9c8801f640 1 -- 172.21.15.19:0/2796518966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c80073a00 con 0x7f9c8006c2d0 2024-04-25T09:57:21.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.031+0000 7f9c767fc640 1 -- 172.21.15.19:0/2796518966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c7c018070 con 0x7f9c8006c2d0 2024-04-25T09:57:21.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.032+0000 7f9c767fc640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c6406b4d0 0x7f9c6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.032+0000 7f9c767fc640 1 -- 172.21.15.19:0/2796518966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f9c7c08a320 con 0x7f9c8006c2d0 2024-04-25T09:57:21.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.032+0000 7f9c855ba640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c6406b4d0 0x7f9c6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.033+0000 7f9c855ba640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c6406b4d0 0x7f9c6406d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9c70034000 tx=0x7f9c7002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.036+0000 7f9c767fc640 1 -- 172.21.15.19:0/2796518966 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c7c0591b0 con 0x7f9c8006c2d0 2024-04-25T09:57:21.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.214+0000 7f9c8801f640 1 -- 172.21.15.19:0/2796518966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9c80161190 con 0x7f9c8006c2d0 2024-04-25T09:57:21.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.217+0000 7f9c767fc640 1 -- 172.21.15.19:0/2796518966 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9c7c058ab0 con 0x7f9c8006c2d0 2024-04-25T09:57:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.219+0000 7f9c8801f640 1 -- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c6406b4d0 msgr2=0x7f9c6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.219+0000 7f9c8801f640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c6406b4d0 0x7f9c6406d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9c70034000 tx=0x7f9c7002c000 comp rx=0 tx=0).stop 2024-04-25T09:57:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.219+0000 7f9c8801f640 1 -- 172.21.15.19:0/2796518966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8006c2d0 msgr2=0x7f9c80159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.219+0000 7f9c8801f640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8006c2d0 0x7f9c80159120 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c7f0 tx=0x7f9c7c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.219+0000 7f9c8801f640 1 -- 172.21.15.19:0/2796518966 shutdown_connections 2024-04-25T09:57:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.219+0000 7f9c8801f640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c6406b4d0 0x7f9c6406d990 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.219+0000 7f9c8801f640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c8015b7d0 0x7f9c800cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.219+0000 7f9c8801f640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c800b5620 0x7f9c800c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.219+0000 7f9c8801f640 1 --2- 172.21.15.19:0/2796518966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c8006c2d0 0x7f9c80159120 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.219+0000 7f9c8801f640 1 -- 172.21.15.19:0/2796518966 >> 172.21.15.19:0/2796518966 conn(0x7f9c800d0800 msgr2=0x7f9c8015e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.219+0000 7f9c8801f640 1 -- 172.21.15.19:0/2796518966 shutdown_connections 2024-04-25T09:57:21.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.220+0000 7f9c8801f640 1 -- 172.21.15.19:0/2796518966 wait complete. 2024-04-25T09:57:21.234 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:57:21.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:21.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:21.328 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:57:21.389 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:57:21.389 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:21.406 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:57:21.406 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:57:21.406 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:21.424 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:21.424 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:57:21.424 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:21.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:57:21.461 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:57:21.461 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:21.481 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:21.481 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:57:21.481 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:21.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.599+0000 7f9dbace1640 1 Processor -- start 2024-04-25T09:57:21.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.599+0000 7f9dbace1640 1 -- start start 2024-04-25T09:57:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.600+0000 7f9dbace1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9db40d0ad0 0x7f9db4168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.600+0000 7f9dbace1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db40b9940 con 0x7f9db40d2e90 2024-04-25T09:57:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.600+0000 7f9dbace1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db40d1df0 con 0x7f9db415dd30 2024-04-25T09:57:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.600+0000 7f9dbace1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db40b7a50 con 0x7f9db40d0ad0 2024-04-25T09:57:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.600+0000 7f9db3fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9db40d2e90 0x7f9db40d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39906/0 (socket says 172.21.15.19:39906) 2024-04-25T09:57:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.600+0000 7f9db3fff640 1 -- 172.21.15.19:0/664089669 learned_addr learned my addr 172.21.15.19:0/664089669 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.600+0000 7f9db927e640 1 --2- 172.21.15.19:0/664089669 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9db40d0ad0 0x7f9db4168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.600+0000 7f9db37fe640 1 -- 172.21.15.19:0/664089669 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759462959 0 0) 0x7f9db40b9940 con 0x7f9db40d2e90 2024-04-25T09:57:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.600+0000 7f9db37fe640 1 -- 172.21.15.19:0/664089669 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d9c003520 con 0x7f9db40d2e90 2024-04-25T09:57:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.601+0000 7f9db37fe640 1 -- 172.21.15.19:0/664089669 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523059065 0 0) 0x7f9db40d1df0 con 0x7f9db415dd30 2024-04-25T09:57:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.601+0000 7f9db37fe640 1 -- 172.21.15.19:0/664089669 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9db40b9940 con 0x7f9db415dd30 2024-04-25T09:57:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.601+0000 7f9db37fe640 1 -- 172.21.15.19:0/664089669 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3490875043 0 0) 0x7f9d9c003520 con 0x7f9db40d2e90 2024-04-25T09:57:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.601+0000 7f9db927e640 1 -- 172.21.15.19:0/664089669 >> v1:172.21.15.112:6789/0 conn(0x7f9db415dd30 legacy=0x7f9db40b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.601+0000 7f9db927e640 1 -- 172.21.15.19:0/664089669 >> v1:172.21.15.19:6789/0 conn(0x7f9db40d2e90 legacy=0x7f9db40d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.601+0000 7f9db927e640 1 -- 172.21.15.19:0/664089669 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40d20d0 con 0x7f9db40d0ad0 2024-04-25T09:57:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.601+0000 7f9db927e640 1 --2- 172.21.15.19:0/664089669 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9db40d0ad0 0x7f9db4168760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009dc0 tx=0x7f9dac02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=310846de68a90d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.602+0000 7f9db37fe640 1 -- 172.21.15.19:0/664089669 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da80016c0 con 0x7f9db40d0ad0 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.602+0000 7f9db37fe640 1 -- 172.21.15.19:0/664089669 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac007540 con 0x7f9db40d0ad0 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.602+0000 7f9db37fe640 1 -- 172.21.15.19:0/664089669 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0078f0 con 0x7f9db40d0ad0 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.602+0000 7f9dbace1640 1 -- 172.21.15.19:0/664089669 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9db40d0ad0 msgr2=0x7f9db4168760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.602+0000 7f9dbace1640 1 --2- 172.21.15.19:0/664089669 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9db40d0ad0 0x7f9db4168760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009dc0 tx=0x7f9dac02a5d0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.602+0000 7f9dbace1640 1 -- 172.21.15.19:0/664089669 shutdown_connections 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.602+0000 7f9dbace1640 1 --2- 172.21.15.19:0/664089669 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9db40d0ad0 0x7f9db4168760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.602+0000 7f9dbace1640 1 -- 172.21.15.19:0/664089669 >> 172.21.15.19:0/664089669 conn(0x7f9db4061c30 msgr2=0x7f9db406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.602+0000 7f9dbace1640 1 -- 172.21.15.19:0/664089669 shutdown_connections 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.602+0000 7f9dbace1640 1 -- 172.21.15.19:0/664089669 wait complete. 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.603+0000 7f9dbace1640 1 Processor -- start 2024-04-25T09:57:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.603+0000 7f9dbace1640 1 -- start start 2024-04-25T09:57:21.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.604+0000 7f9dbace1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40d0ad0 0x7f9db4160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.604+0000 7f9dbace1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9db40d2e90 0x7f9db415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.604+0000 7f9dbace1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9db415dd30 0x7f9db415eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.604+0000 7f9dbace1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db40d2970 con 0x7f9db415dd30 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.604+0000 7f9dbace1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db40d2690 con 0x7f9db40d0ad0 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.604+0000 7f9dbace1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db40d1a10 con 0x7f9db40d2e90 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.604+0000 7f9db927e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9db415dd30 0x7f9db415eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.604+0000 7f9db927e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9db415dd30 0x7f9db415eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44554/0 (socket says 172.21.15.19:44554) 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.604+0000 7f9db927e640 1 -- 172.21.15.19:0/4245700028 learned_addr learned my addr 172.21.15.19:0/4245700028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9db3fff640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9db40d2e90 0x7f9db415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9db8a7d640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40d0ad0 0x7f9db4160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9db927e640 1 -- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9db40d2e90 msgr2=0x7f9db415b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9db927e640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9db40d2e90 0x7f9db415b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9db927e640 1 -- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40d0ad0 msgr2=0x7f9db4160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9db927e640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40d0ad0 0x7f9db4160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9db927e640 1 -- 172.21.15.19:0/4245700028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db415ffc0 con 0x7f9db415dd30 2024-04-25T09:57:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9db927e640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9db415dd30 0x7f9db415eb20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009ee0 tx=0x7f9dac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9db17fa640 1 -- 172.21.15.19:0/4245700028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac040070 con 0x7f9db415dd30 2024-04-25T09:57:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9db17fa640 1 -- 172.21.15.19:0/4245700028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac031c40 con 0x7f9db415dd30 2024-04-25T09:57:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9db17fa640 1 -- 172.21.15.19:0/4245700028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac009450 con 0x7f9db415dd30 2024-04-25T09:57:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9dbace1640 1 -- 172.21.15.19:0/4245700028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db41602a0 con 0x7f9db415dd30 2024-04-25T09:57:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.605+0000 7f9dbace1640 1 -- 172.21.15.19:0/4245700028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db4160860 con 0x7f9db415dd30 2024-04-25T09:57:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.606+0000 7f9dbace1640 1 -- 172.21.15.19:0/4245700028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db4073990 con 0x7f9db415dd30 2024-04-25T09:57:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.607+0000 7f9db17fa640 1 -- 172.21.15.19:0/4245700028 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9dac039040 con 0x7f9db415dd30 2024-04-25T09:57:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.607+0000 7f9db17fa640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9da006b4d0 0x7f9da006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.607+0000 7f9db17fa640 1 -- 172.21.15.19:0/4245700028 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f9dac0ab300 con 0x7f9db415dd30 2024-04-25T09:57:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.607+0000 7f9db8a7d640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9da006b4d0 0x7f9da006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.608+0000 7f9db8a7d640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9da006b4d0 0x7f9da006d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9da40011e0 tx=0x7f9da4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.611+0000 7f9db17fa640 1 -- 172.21.15.19:0/4245700028 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9dac079d10 con 0x7f9db415dd30 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.829+0000 7f9dbace1640 1 -- 172.21.15.19:0/4245700028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9db40d4570 con 0x7f9db415dd30 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.829+0000 7f9db17fa640 1 -- 172.21.15.19:0/4245700028 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f9dac07ddb0 con 0x7f9db415dd30 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5756,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2433112551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2433112551},{"type":"v1","addr":"172.21.15.112:6837","nonce":2433112551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6295,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2458476655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2458476655},{"type":"v1","addr":"172.21.15.112:6835","nonce":2458476655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6867,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1321165720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1321165720},{"type":"v1","addr":"172.21.15.19:6835","nonce":1321165720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7044,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2506126577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2506126577},{"type":"v1","addr":"172.21.15.19:6833","nonce":2506126577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[]} 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 45 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.831+0000 7f9dbace1640 1 -- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9da006b4d0 msgr2=0x7f9da006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.831+0000 7f9dbace1640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9da006b4d0 0x7f9da006d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9da40011e0 tx=0x7f9da4009040 comp rx=0 tx=0).stop 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.831+0000 7f9dbace1640 1 -- 172.21.15.19:0/4245700028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9db415dd30 msgr2=0x7f9db415eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.831+0000 7f9dbace1640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9db415dd30 0x7f9db415eb20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009ee0 tx=0x7f9dac004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.831+0000 7f9dbace1640 1 -- 172.21.15.19:0/4245700028 shutdown_connections 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.831+0000 7f9dbace1640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9da006b4d0 0x7f9da006d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.831+0000 7f9dbace1640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9db415dd30 0x7f9db415eb20 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.831+0000 7f9dbace1640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9db40d2e90 0x7f9db415b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.831+0000 7f9dbace1640 1 --2- 172.21.15.19:0/4245700028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40d0ad0 0x7f9db4160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.831+0000 7f9dbace1640 1 -- 172.21.15.19:0/4245700028 >> 172.21.15.19:0/4245700028 conn(0x7f9db4061c30 msgr2=0x7f9db40c77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:21.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.831+0000 7f9dbace1640 1 -- 172.21.15.19:0/4245700028 shutdown_connections 2024-04-25T09:57:21.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:21.832+0000 7f9dbace1640 1 -- 172.21.15.19:0/4245700028 wait complete. 2024-04-25T09:57:21.846 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:57:21.846 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:57:21.847 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:57:21.847 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:21.847 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:57:21.847 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:57:21.847 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:57:21.847 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:21.847 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:57:21.848 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:57:21.848 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:57:21.848 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:21.848 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:57:21.848 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:57:21.848 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:57:21.848 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:21.857 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:57:21.857 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:57:21.858 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:57:21.858 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:57:21.860 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:57:21.860 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:57:21.861 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:57:21.861 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:57:21.900 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:57:21.900 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:21.900 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:57:21.900 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:21.902 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:57:21.903 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:21.903 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:57:21.903 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:21.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:57:21.976 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T09:57:21.976 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T09:57:21.984 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T09:57:21.984 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T09:57:22.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.008+0000 7fa37b4bd640 1 Processor -- start 2024-04-25T09:57:22.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.009+0000 7fa37b4bd640 1 -- start start 2024-04-25T09:57:22.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.009+0000 7fa37b4bd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa374161890 0x7fa3740b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.009+0000 7fa37b4bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3740b54a0 0x7fa3740b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.009+0000 7fa37b4bd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3740cf1d0 con 0x7fa374161240 2024-04-25T09:57:22.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.009+0000 7fa37b4bd640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3740b7120 con 0x7fa374161890 2024-04-25T09:57:22.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.009+0000 7fa37b4bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa374161620 con 0x7fa3740b54a0 2024-04-25T09:57:22.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.009+0000 7fa378a58640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa374161890 0x7fa3740b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.010+0000 7fa378a58640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa374161890 0x7fa3740b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44608/0 (socket says 172.21.15.19:44608) 2024-04-25T09:57:22.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.010+0000 7fa378a58640 1 -- 172.21.15.19:0/2489139946 learned_addr learned my addr 172.21.15.19:0/2489139946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.010+0000 7fa379a5a640 1 --2- 172.21.15.19:0/2489139946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3740b54a0 0x7fa3740b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.010+0000 7fa378a58640 1 -- 172.21.15.19:0/2489139946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3740b54a0 msgr2=0x7fa3740b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.010+0000 7fa378a58640 1 --2- 172.21.15.19:0/2489139946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3740b54a0 0x7fa3740b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.010+0000 7fa378a58640 1 -- 172.21.15.19:0/2489139946 >> v1:172.21.15.112:6789/0 conn(0x7fa374161240 legacy=0x7fa3740ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.010+0000 7fa378a58640 1 -- 172.21.15.19:0/2489139946 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374161e00 con 0x7fa374161890 2024-04-25T09:57:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.011+0000 7fa378a58640 1 --2- 172.21.15.19:0/2489139946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa374161890 0x7fa3740b6be0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa364009870 tx=0x7fa36402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8728f43006cd7c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.011+0000 7fa36bfff640 1 -- 172.21.15.19:0/2489139946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364037070 con 0x7fa374161890 2024-04-25T09:57:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.011+0000 7fa36bfff640 1 -- 172.21.15.19:0/2489139946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa364032440 con 0x7fa374161890 2024-04-25T09:57:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.011+0000 7fa36bfff640 1 -- 172.21.15.19:0/2489139946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364032810 con 0x7fa374161890 2024-04-25T09:57:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.011+0000 7fa37b4bd640 1 -- 172.21.15.19:0/2489139946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa374161890 msgr2=0x7fa3740b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.011+0000 7fa37b4bd640 1 --2- 172.21.15.19:0/2489139946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa374161890 0x7fa3740b6be0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa364009870 tx=0x7fa36402a270 comp rx=0 tx=0).stop 2024-04-25T09:57:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.011+0000 7fa37b4bd640 1 -- 172.21.15.19:0/2489139946 shutdown_connections 2024-04-25T09:57:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.011+0000 7fa37b4bd640 1 --2- 172.21.15.19:0/2489139946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3740b54a0 0x7fa3740b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.011+0000 7fa37b4bd640 1 --2- 172.21.15.19:0/2489139946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa374161890 0x7fa3740b6be0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.011+0000 7fa37b4bd640 1 -- 172.21.15.19:0/2489139946 >> 172.21.15.19:0/2489139946 conn(0x7fa3740d04a0 msgr2=0x7fa3740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.012+0000 7fa37b4bd640 1 -- 172.21.15.19:0/2489139946 shutdown_connections 2024-04-25T09:57:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.012+0000 7fa37b4bd640 1 -- 172.21.15.19:0/2489139946 wait complete. 2024-04-25T09:57:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.013+0000 7fa37b4bd640 1 Processor -- start 2024-04-25T09:57:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.013+0000 7fa37b4bd640 1 -- start start 2024-04-25T09:57:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.013+0000 7fa37b4bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3740b54a0 0x7fa374152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa37b4bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa374161240 0x7fa37415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa37b4bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374161890 0x7fa37415ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa37b4bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3741626a0 con 0x7fa3740b54a0 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa37b4bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3741623c0 con 0x7fa374161890 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa37b4bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740d15f0 con 0x7fa374161240 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa379259640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3740b54a0 0x7fa374152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa379259640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3740b54a0 0x7fa374152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44610/0 (socket says 172.21.15.19:44610) 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa379259640 1 -- 172.21.15.19:0/756398891 learned_addr learned my addr 172.21.15.19:0/756398891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa378a58640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa374161240 0x7fa37415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa379a5a640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374161890 0x7fa37415ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa379259640 1 -- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa374161240 msgr2=0x7fa37415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa379259640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa374161240 0x7fa37415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa379259640 1 -- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374161890 msgr2=0x7fa37415ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa379259640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374161890 0x7fa37415ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa379259640 1 -- 172.21.15.19:0/756398891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3741f7320 con 0x7fa3740b54a0 2024-04-25T09:57:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa379259640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3740b54a0 0x7fa374152030 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c400 tx=0x7fa37000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa369ffb640 1 -- 172.21.15.19:0/756398891 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3700052d0 con 0x7fa3740b54a0 2024-04-25T09:57:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa37b4bd640 1 -- 172.21.15.19:0/756398891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3741f7600 con 0x7fa3740b54a0 2024-04-25T09:57:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa369ffb640 1 -- 172.21.15.19:0/756398891 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa370005490 con 0x7fa3740b54a0 2024-04-25T09:57:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa369ffb640 1 -- 172.21.15.19:0/756398891 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370007260 con 0x7fa3740b54a0 2024-04-25T09:57:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.014+0000 7fa37b4bd640 1 -- 172.21.15.19:0/756398891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3741f7bc0 con 0x7fa3740b54a0 2024-04-25T09:57:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.016+0000 7fa37b4bd640 1 -- 172.21.15.19:0/756398891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa374073a00 con 0x7fa3740b54a0 2024-04-25T09:57:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.016+0000 7fa369ffb640 1 -- 172.21.15.19:0/756398891 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa370007400 con 0x7fa3740b54a0 2024-04-25T09:57:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.016+0000 7fa369ffb640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa35806b700 0x7fa35806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.017+0000 7fa369ffb640 1 -- 172.21.15.19:0/756398891 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa370007020 con 0x7fa3740b54a0 2024-04-25T09:57:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.017+0000 7fa378a58640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa35806b700 0x7fa35806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.017+0000 7fa378a58640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa35806b700 0x7fa35806dbc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa364034000 tx=0x7fa36402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.021+0000 7fa369ffb640 1 -- 172.21.15.19:0/756398891 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa370058300 con 0x7fa3740b54a0 2024-04-25T09:57:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.212+0000 7fa37b4bd640 1 -- 172.21.15.19:0/756398891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa37406d240 con 0x7fa3740b54a0 2024-04-25T09:57:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.922+0000 7fa369ffb640 1 -- 172.21.15.19:0/756398891 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa370057c00 con 0x7fa3740b54a0 2024-04-25T09:57:22.925 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T09:57:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 -- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa35806b700 msgr2=0x7fa35806dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa35806b700 0x7fa35806dbc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa364034000 tx=0x7fa36402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 -- 172.21.15.19:0/756398891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3740b54a0 msgr2=0x7fa374152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3740b54a0 0x7fa374152030 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c400 tx=0x7fa37000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 -- 172.21.15.19:0/756398891 shutdown_connections 2024-04-25T09:57:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa35806b700 0x7fa35806dbc0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa374161890 0x7fa37415ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa374161240 0x7fa37415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 --2- 172.21.15.19:0/756398891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3740b54a0 0x7fa374152030 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 -- 172.21.15.19:0/756398891 >> 172.21.15.19:0/756398891 conn(0x7fa3740d04a0 msgr2=0x7fa3740d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 -- 172.21.15.19:0/756398891 shutdown_connections 2024-04-25T09:57:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:22.925+0000 7fa37b4bd640 1 -- 172.21.15.19:0/756398891 wait complete. 2024-04-25T09:57:22.943 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:57:23.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.046+0000 7f9bc1bf4640 1 Processor -- start 2024-04-25T09:57:23.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.046+0000 7f9bc1bf4640 1 -- start start 2024-04-25T09:57:23.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.047+0000 7f9bc1bf4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9bbc15db70 0x7f9bbc162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.047+0000 7f9bc1bf4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bbc0b6cb0 0x7f9bbc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.047+0000 7f9bc1bf4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9bbc0b5a60 0x7f9bbc06c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.047+0000 7f9bc1bf4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc162690 con 0x7f9bbc0b5a60 2024-04-25T09:57:23.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.047+0000 7f9bc1bf4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc0ceee0 con 0x7f9bbc15db70 2024-04-25T09:57:23.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.047+0000 7f9bc1bf4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0d1d10 con 0x7f9bbc0b6cb0 2024-04-25T09:57:23.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.047+0000 7f9bbbfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9bbc0b5a60 0x7f9bbc06c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.047+0000 7f9bbbfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9bbc0b5a60 0x7f9bbc06c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44618/0 (socket says 172.21.15.19:44618) 2024-04-25T09:57:23.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.047+0000 7f9bbbfff640 1 -- 172.21.15.19:0/2671734922 learned_addr learned my addr 172.21.15.19:0/2671734922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:23.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.048+0000 7f9bbaffd640 1 --2- 172.21.15.19:0/2671734922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bbc0b6cb0 0x7f9bbc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.048+0000 7f9bbb7fe640 1 --2- 172.21.15.19:0/2671734922 >> v2:172.21.15.112:3300/0 conn(0x7f9bbc15db70 0x7f9bbc162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.048+0000 7f9bbbfff640 1 -- 172.21.15.19:0/2671734922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bbc0b6cb0 msgr2=0x7f9bbc0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:23.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.048+0000 7f9bbbfff640 1 --2- 172.21.15.19:0/2671734922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bbc0b6cb0 0x7f9bbc0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.048+0000 7f9bbbfff640 1 -- 172.21.15.19:0/2671734922 >> v2:172.21.15.112:3300/0 conn(0x7f9bbc15db70 msgr2=0x7f9bbc162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:23.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.048+0000 7f9bbbfff640 1 --2- 172.21.15.19:0/2671734922 >> v2:172.21.15.112:3300/0 conn(0x7f9bbc15db70 0x7f9bbc162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.048+0000 7f9bbbfff640 1 -- 172.21.15.19:0/2671734922 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc0d1ff0 con 0x7f9bbc0b5a60 2024-04-25T09:57:23.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.048+0000 7f9bbb7fe640 1 --2- 172.21.15.19:0/2671734922 >> v2:172.21.15.112:3300/0 conn(0x7f9bbc15db70 0x7f9bbc162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:23.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.048+0000 7f9bbbfff640 1 --2- 172.21.15.19:0/2671734922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bbc0b5a60 0x7f9bbc06c2d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0b6570 tx=0x7f9bb402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c443f94f876b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bba7fc640 1 -- 172.21.15.19:0/2671734922 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4036070 con 0x7f9bbc0b5a60 2024-04-25T09:57:23.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bba7fc640 1 -- 172.21.15.19:0/2671734922 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb4031600 con 0x7f9bbc0b5a60 2024-04-25T09:57:23.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bba7fc640 1 -- 172.21.15.19:0/2671734922 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40319d0 con 0x7f9bbc0b5a60 2024-04-25T09:57:23.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2671734922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bbc0b5a60 msgr2=0x7f9bbc06c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bc1bf4640 1 --2- 172.21.15.19:0/2671734922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bbc0b5a60 0x7f9bbc06c2d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0b6570 tx=0x7f9bb402a6a0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2671734922 shutdown_connections 2024-04-25T09:57:23.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bc1bf4640 1 --2- 172.21.15.19:0/2671734922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bbc0b6cb0 0x7f9bbc0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bc1bf4640 1 --2- 172.21.15.19:0/2671734922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bbc0b5a60 0x7f9bbc06c2d0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bc1bf4640 1 --2- 172.21.15.19:0/2671734922 >> v2:172.21.15.112:3300/0 conn(0x7f9bbc15db70 0x7f9bbc162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2671734922 >> 172.21.15.19:0/2671734922 conn(0x7f9bbc0d0520 msgr2=0x7f9bbc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2671734922 shutdown_connections 2024-04-25T09:57:23.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.049+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2671734922 wait complete. 2024-04-25T09:57:23.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.050+0000 7f9bc1bf4640 1 Processor -- start 2024-04-25T09:57:23.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.051+0000 7f9bc1bf4640 1 -- start start 2024-04-25T09:57:23.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.051+0000 7f9bc1bf4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bbc0b5a60 0x7f9bbc159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.051+0000 7f9bc1bf4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bbc0b6cb0 0x7f9bbc15f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.051+0000 7f9bc1bf4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bbc15db70 0x7f9bbc0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.051+0000 7f9bc1bf4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0ce530 con 0x7f9bbc15db70 2024-04-25T09:57:23.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.051+0000 7f9bc1bf4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc15df50 con 0x7f9bbc0b5a60 2024-04-25T09:57:23.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.051+0000 7f9bc1bf4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0cf1c0 con 0x7f9bbc0b6cb0 2024-04-25T09:57:23.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bbc15db70 0x7f9bbc0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bbc15db70 0x7f9bbc0ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44620/0 (socket says 172.21.15.19:44620) 2024-04-25T09:57:23.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbbfff640 1 -- 172.21.15.19:0/2198808591 learned_addr learned my addr 172.21.15.19:0/2198808591 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:23.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbaffd640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bbc0b6cb0 0x7f9bbc15f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbb7fe640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bbc0b5a60 0x7f9bbc159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbbfff640 1 -- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bbc0b6cb0 msgr2=0x7f9bbc15f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbbfff640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bbc0b6cb0 0x7f9bbc15f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbbfff640 1 -- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bbc0b5a60 msgr2=0x7f9bbc159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbbfff640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bbc0b5a60 0x7f9bbc159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbbfff640 1 -- 172.21.15.19:0/2198808591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc15e200 con 0x7f9bbc15db70 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbaffd640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bbc0b6cb0 0x7f9bbc15f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bbbfff640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bbc15db70 0x7f9bbc0ccd40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4034040 tx=0x7f9bb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9ba3fff640 1 -- 172.21.15.19:0/2198808591 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4041020 con 0x7f9bbc15db70 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9ba3fff640 1 -- 172.21.15.19:0/2198808591 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb4007780 con 0x7f9bbc15db70 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2198808591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc15e4e0 con 0x7f9bbc15db70 2024-04-25T09:57:23.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9ba3fff640 1 -- 172.21.15.19:0/2198808591 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4036040 con 0x7f9bbc15db70 2024-04-25T09:57:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.052+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2198808591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc15eaa0 con 0x7f9bbc15db70 2024-04-25T09:57:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.053+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2198808591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc073990 con 0x7f9bbc15db70 2024-04-25T09:57:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.054+0000 7f9ba3fff640 1 -- 172.21.15.19:0/2198808591 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bb4044020 con 0x7f9bbc15db70 2024-04-25T09:57:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.054+0000 7f9ba3fff640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b4d0 0x7f9ba806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.054+0000 7f9ba3fff640 1 -- 172.21.15.19:0/2198808591 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9bb40aae20 con 0x7f9bbc15db70 2024-04-25T09:57:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.055+0000 7f9bbb7fe640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b4d0 0x7f9ba806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.055+0000 7f9bbb7fe640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b4d0 0x7f9ba806d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00a950 tx=0x7f9bac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.059+0000 7f9ba3fff640 1 -- 172.21.15.19:0/2198808591 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9bb404b030 con 0x7f9bbc15db70 2024-04-25T09:57:23.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.308+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2198808591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9bbc0d1820 con 0x7f9bbc15db70 2024-04-25T09:57:23.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.308+0000 7f9ba3fff640 1 -- 172.21.15.19:0/2198808591 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9bb4079fb0 con 0x7f9bbc15db70 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:57:23.313 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:57:23.314 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T09:57:23.315 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.317 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.318 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.319 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.320 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T09:57:23.322 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:57:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T09:57:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T09:57:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:57:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:57:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:57:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:57:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T09:57:23.324 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:57:23.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.310+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b4d0 msgr2=0x7f9ba806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.310+0000 7f9bc1bf4640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b4d0 0x7f9ba806d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00a950 tx=0x7f9bac008040 comp rx=0 tx=0).stop 2024-04-25T09:57:23.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.310+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2198808591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bbc15db70 msgr2=0x7f9bbc0ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.310+0000 7f9bc1bf4640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bbc15db70 0x7f9bbc0ccd40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4034040 tx=0x7f9bb4004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.310+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2198808591 shutdown_connections 2024-04-25T09:57:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.310+0000 7f9bc1bf4640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b4d0 0x7f9ba806d990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.310+0000 7f9bc1bf4640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bbc15db70 0x7f9bbc0ccd40 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.310+0000 7f9bc1bf4640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bbc0b6cb0 0x7f9bbc15f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.310+0000 7f9bc1bf4640 1 --2- 172.21.15.19:0/2198808591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bbc0b5a60 0x7f9bbc159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.310+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2198808591 >> 172.21.15.19:0/2198808591 conn(0x7f9bbc0d0520 msgr2=0x7f9bbc06c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.310+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2198808591 shutdown_connections 2024-04-25T09:57:23.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.311+0000 7f9bc1bf4640 1 -- 172.21.15.19:0/2198808591 wait complete. 2024-04-25T09:57:23.327 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.424+0000 7f933386f640 1 Processor -- start 2024-04-25T09:57:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.424+0000 7f933386f640 1 -- start start 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f933386f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f932c0b6cb0 0x7f932c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f933386f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c0ceee0 con 0x7f932c0b5a60 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f933386f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c0d1d10 con 0x7f932c15db70 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f933386f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932c0d1ff0 con 0x7f932c0b6cb0 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f9331e0c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f932c0b5a60 0x7f932c0d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39914/0 (socket says 172.21.15.19:39914) 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f9331e0c640 1 -- 172.21.15.19:0/2224343926 learned_addr learned my addr 172.21.15.19:0/2224343926 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f9330e0a640 1 --2- 172.21.15.19:0/2224343926 >> v2:172.21.15.112:3300/0 conn(0x7f932c0b6cb0 0x7f932c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f9323fff640 1 -- 172.21.15.19:0/2224343926 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996298103 0 0) 0x7f932c0ceee0 con 0x7f932c0b5a60 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f9323fff640 1 -- 172.21.15.19:0/2224343926 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9318003520 con 0x7f932c0b5a60 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f9330e0a640 1 -- 172.21.15.19:0/2224343926 >> v1:172.21.15.112:6789/0 conn(0x7f932c15db70 legacy=0x7f932c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f9330e0a640 1 -- 172.21.15.19:0/2224343926 >> v1:172.21.15.19:6789/0 conn(0x7f932c0b5a60 legacy=0x7f932c0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.425+0000 7f9330e0a640 1 -- 172.21.15.19:0/2224343926 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c0d22d0 con 0x7f932c0b6cb0 2024-04-25T09:57:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.426+0000 7f9330e0a640 1 --2- 172.21.15.19:0/2224343926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932c0b6cb0 0x7f932c0b5520 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f931c0097a0 tx=0x7f931c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb76e72a88d4607 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.426+0000 7f9323fff640 1 -- 172.21.15.19:0/2224343926 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c037070 con 0x7f932c0b6cb0 2024-04-25T09:57:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.426+0000 7f9323fff640 1 -- 172.21.15.19:0/2224343926 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c032440 con 0x7f932c0b6cb0 2024-04-25T09:57:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.426+0000 7f9323fff640 1 -- 172.21.15.19:0/2224343926 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c0327f0 con 0x7f932c0b6cb0 2024-04-25T09:57:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.427+0000 7f933386f640 1 -- 172.21.15.19:0/2224343926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932c0b6cb0 msgr2=0x7f932c0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.427+0000 7f933386f640 1 --2- 172.21.15.19:0/2224343926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932c0b6cb0 0x7f932c0b5520 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f931c0097a0 tx=0x7f931c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.427+0000 7f933386f640 1 -- 172.21.15.19:0/2224343926 shutdown_connections 2024-04-25T09:57:23.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.427+0000 7f933386f640 1 --2- 172.21.15.19:0/2224343926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932c0b6cb0 0x7f932c0b5520 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.427+0000 7f933386f640 1 -- 172.21.15.19:0/2224343926 >> 172.21.15.19:0/2224343926 conn(0x7f932c0d0520 msgr2=0x7f932c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.427+0000 7f933386f640 1 -- 172.21.15.19:0/2224343926 shutdown_connections 2024-04-25T09:57:23.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.427+0000 7f933386f640 1 -- 172.21.15.19:0/2224343926 wait complete. 2024-04-25T09:57:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.428+0000 7f933386f640 1 Processor -- start 2024-04-25T09:57:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.428+0000 7f933386f640 1 -- start start 2024-04-25T09:57:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.429+0000 7f933386f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f932c0b5a60 0x7f932c0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.429+0000 7f933386f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932c0b6cb0 0x7f932c1f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.429+0000 7f933386f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f932c15db70 0x7f932c1601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.429+0000 7f933386f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c0cf1c0 con 0x7f932c15db70 2024-04-25T09:57:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.429+0000 7f933386f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c0ce530 con 0x7f932c0b6cb0 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.429+0000 7f933386f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c0cf340 con 0x7f932c0b5a60 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.429+0000 7f9331e0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f932c15db70 0x7f932c1601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.429+0000 7f9331e0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f932c15db70 0x7f932c1601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44628/0 (socket says 172.21.15.19:44628) 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.429+0000 7f9331e0c640 1 -- 172.21.15.19:0/2245034456 learned_addr learned my addr 172.21.15.19:0/2245034456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.429+0000 7f9330e0a640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932c0b6cb0 0x7f932c1f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.429+0000 7f933160b640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f932c0b5a60 0x7f932c0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.430+0000 7f9331e0c640 1 -- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f932c0b5a60 msgr2=0x7f932c0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.430+0000 7f9331e0c640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f932c0b5a60 0x7f932c0cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.430+0000 7f9331e0c640 1 -- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932c0b6cb0 msgr2=0x7f932c1f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.430+0000 7f9331e0c640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932c0b6cb0 0x7f932c1f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.430+0000 7f9331e0c640 1 -- 172.21.15.19:0/2245034456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c0cb250 con 0x7f932c15db70 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.430+0000 7f9331e0c640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f932c15db70 0x7f932c1601d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f932400cb40 tx=0x7f932400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.430+0000 7f9321ffb640 1 -- 172.21.15.19:0/2245034456 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324017070 con 0x7f932c15db70 2024-04-25T09:57:23.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.430+0000 7f9321ffb640 1 -- 172.21.15.19:0/2245034456 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932400aca0 con 0x7f932c15db70 2024-04-25T09:57:23.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.430+0000 7f933386f640 1 -- 172.21.15.19:0/2245034456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c0cb530 con 0x7f932c15db70 2024-04-25T09:57:23.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.430+0000 7f9321ffb640 1 -- 172.21.15.19:0/2245034456 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324012620 con 0x7f932c15db70 2024-04-25T09:57:23.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.430+0000 7f933386f640 1 -- 172.21.15.19:0/2245034456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c0cbaf0 con 0x7f932c15db70 2024-04-25T09:57:23.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.431+0000 7f933386f640 1 -- 172.21.15.19:0/2245034456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932c073990 con 0x7f932c15db70 2024-04-25T09:57:23.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.431+0000 7f9321ffb640 1 -- 172.21.15.19:0/2245034456 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f932401a070 con 0x7f932c15db70 2024-04-25T09:57:23.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.432+0000 7f9321ffb640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f931006b500 0x7f931006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.432+0000 7f9321ffb640 1 -- 172.21.15.19:0/2245034456 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9324089d60 con 0x7f932c15db70 2024-04-25T09:57:23.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.432+0000 7f933160b640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f931006b500 0x7f931006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.433+0000 7f933160b640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f931006b500 0x7f931006d9c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f932c06cba0 tx=0x7f9328009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.436+0000 7f9321ffb640 1 -- 172.21.15.19:0/2245034456 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93240587e0 con 0x7f932c15db70 2024-04-25T09:57:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.620+0000 7f933386f640 1 -- 172.21.15.19:0/2245034456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f932c0d1820 con 0x7f932c15db70 2024-04-25T09:57:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.621+0000 7f9321ffb640 1 -- 172.21.15.19:0/2245034456 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+13153 (secure 0 0 0) 0x7f932405c880 con 0x7f932c15db70 2024-04-25T09:57:23.624 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:23.624 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":77,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:57:22.935680+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 -- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f931006b500 msgr2=0x7f931006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f931006b500 0x7f931006d9c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f932c06cba0 tx=0x7f9328009040 comp rx=0 tx=0).stop 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 -- 172.21.15.19:0/2245034456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f932c15db70 msgr2=0x7f932c1601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f932c15db70 0x7f932c1601d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f932400cb40 tx=0x7f932400a060 comp rx=0 tx=0).stop 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 -- 172.21.15.19:0/2245034456 shutdown_connections 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f931006b500 0x7f931006d9c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f932c15db70 0x7f932c1601d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932c0b6cb0 0x7f932c1f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 --2- 172.21.15.19:0/2245034456 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f932c0b5a60 0x7f932c0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 -- 172.21.15.19:0/2245034456 >> 172.21.15.19:0/2245034456 conn(0x7f932c0d0520 msgr2=0x7f932c15c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 -- 172.21.15.19:0/2245034456 shutdown_connections 2024-04-25T09:57:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.623+0000 7f933386f640 1 -- 172.21.15.19:0/2245034456 wait complete. 2024-04-25T09:57:23.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:23.638 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:23.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.738+0000 7fb574113640 1 Processor -- start 2024-04-25T09:57:23.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.738+0000 7fb574113640 1 -- start start 2024-04-25T09:57:23.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb574113640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb56c158230 0x7fb56c0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb574113640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb56c0d1b80 0x7fb56c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb574113640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb56c0d2020 con 0x7fb56c06c870 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb574113640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56c158610 con 0x7fb56c0d1b80 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb574113640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56c06bea0 con 0x7fb56c158230 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb5716ae640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb56c0d1b80 0x7fb56c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb5716ae640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb56c0d1b80 0x7fb56c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44630/0 (socket says 172.21.15.19:44630) 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb5716ae640 1 -- 172.21.15.19:0/1007498490 learned_addr learned my addr 172.21.15.19:0/1007498490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb571eaf640 1 --2- 172.21.15.19:0/1007498490 >> v2:172.21.15.112:3300/0 conn(0x7fb56c158230 0x7fb56c0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb5716ae640 1 -- 172.21.15.19:0/1007498490 >> v2:172.21.15.112:3300/0 conn(0x7fb56c158230 msgr2=0x7fb56c0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb5716ae640 1 --2- 172.21.15.19:0/1007498490 >> v2:172.21.15.112:3300/0 conn(0x7fb56c158230 0x7fb56c0d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb5716ae640 1 -- 172.21.15.19:0/1007498490 >> v1:172.21.15.112:6789/0 conn(0x7fb56c06c870 legacy=0x7fb56c0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.739+0000 7fb5716ae640 1 -- 172.21.15.19:0/1007498490 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c0d2300 con 0x7fb56c0d1b80 2024-04-25T09:57:23.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.740+0000 7fb5716ae640 1 --2- 172.21.15.19:0/1007498490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c0d1b80 0x7fb56c06c330 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb56c070460 tx=0x7fb55c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e7381c488eee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.740+0000 7fb570ead640 1 -- 172.21.15.19:0/1007498490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c004030 con 0x7fb56c0d1b80 2024-04-25T09:57:23.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.740+0000 7fb570ead640 1 -- 172.21.15.19:0/1007498490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c0041d0 con 0x7fb56c0d1b80 2024-04-25T09:57:23.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.740+0000 7fb570ead640 1 -- 172.21.15.19:0/1007498490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c003610 con 0x7fb56c0d1b80 2024-04-25T09:57:23.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.740+0000 7fb574113640 1 -- 172.21.15.19:0/1007498490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c0d1b80 msgr2=0x7fb56c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.740+0000 7fb574113640 1 --2- 172.21.15.19:0/1007498490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c0d1b80 0x7fb56c06c330 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb56c070460 tx=0x7fb55c02a270 comp rx=0 tx=0).stop 2024-04-25T09:57:23.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.741+0000 7fb574113640 1 -- 172.21.15.19:0/1007498490 shutdown_connections 2024-04-25T09:57:23.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.741+0000 7fb574113640 1 --2- 172.21.15.19:0/1007498490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c0d1b80 0x7fb56c06c330 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.741+0000 7fb574113640 1 --2- 172.21.15.19:0/1007498490 >> v2:172.21.15.112:3300/0 conn(0x7fb56c158230 0x7fb56c0d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.741+0000 7fb574113640 1 -- 172.21.15.19:0/1007498490 >> 172.21.15.19:0/1007498490 conn(0x7fb56c061600 msgr2=0x7fb56c0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.741+0000 7fb574113640 1 -- 172.21.15.19:0/1007498490 shutdown_connections 2024-04-25T09:57:23.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.741+0000 7fb574113640 1 -- 172.21.15.19:0/1007498490 wait complete. 2024-04-25T09:57:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.742+0000 7fb574113640 1 Processor -- start 2024-04-25T09:57:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.742+0000 7fb574113640 1 -- start start 2024-04-25T09:57:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb574113640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c06c870 0x7fb56c1e7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb574113640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb56c0d1b80 0x7fb56c146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb574113640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb56c158230 0x7fb56c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb574113640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c15e320 con 0x7fb56c06c870 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb574113640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c0d25e0 con 0x7fb56c0d1b80 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb574113640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c15e4a0 con 0x7fb56c158230 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb571eaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c06c870 0x7fb56c1e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb571eaf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c06c870 0x7fb56c1e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44634/0 (socket says 172.21.15.19:44634) 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb571eaf640 1 -- 172.21.15.19:0/841500370 learned_addr learned my addr 172.21.15.19:0/841500370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb5716ae640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb56c0d1b80 0x7fb56c146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb5726b0640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb56c158230 0x7fb56c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb571eaf640 1 -- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb56c158230 msgr2=0x7fb56c155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb571eaf640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb56c158230 0x7fb56c155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb571eaf640 1 -- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb56c0d1b80 msgr2=0x7fb56c146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb571eaf640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb56c0d1b80 0x7fb56c146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb571eaf640 1 -- 172.21.15.19:0/841500370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c146650 con 0x7fb56c06c870 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb5716ae640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb56c0d1b80 0x7fb56c146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb5726b0640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb56c158230 0x7fb56c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb571eaf640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c06c870 0x7fb56c1e7090 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb56800e9b0 tx=0x7fb56800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb5627fc640 1 -- 172.21.15.19:0/841500370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568017070 con 0x7fb56c06c870 2024-04-25T09:57:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb5627fc640 1 -- 172.21.15.19:0/841500370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56800c9a0 con 0x7fb56c06c870 2024-04-25T09:57:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb574113640 1 -- 172.21.15.19:0/841500370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56c1e5ff0 con 0x7fb56c06c870 2024-04-25T09:57:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb5627fc640 1 -- 172.21.15.19:0/841500370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56800cd90 con 0x7fb56c06c870 2024-04-25T09:57:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.743+0000 7fb574113640 1 -- 172.21.15.19:0/841500370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56c1e65b0 con 0x7fb56c06c870 2024-04-25T09:57:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.744+0000 7fb574113640 1 -- 172.21.15.19:0/841500370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56c06de50 con 0x7fb56c06c870 2024-04-25T09:57:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.745+0000 7fb5627fc640 1 -- 172.21.15.19:0/841500370 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb56801a070 con 0x7fb56c06c870 2024-04-25T09:57:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.745+0000 7fb5627fc640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb55006b4d0 0x7fb55006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.745+0000 7fb5627fc640 1 -- 172.21.15.19:0/841500370 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb568089fd0 con 0x7fb56c06c870 2024-04-25T09:57:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.746+0000 7fb5716ae640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb55006b4d0 0x7fb55006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.746+0000 7fb5716ae640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb55006b4d0 0x7fb55006d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb55c033000 tx=0x7fb55c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.750+0000 7fb5627fc640 1 -- 172.21.15.19:0/841500370 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb568059010 con 0x7fb56c06c870 2024-04-25T09:57:23.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.929+0000 7fb574113640 1 -- 172.21.15.19:0/841500370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb56c0b56e0 con 0x7fb56c06c870 2024-04-25T09:57:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.929+0000 7fb5627fc640 1 -- 172.21.15.19:0/841500370 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb568058910 con 0x7fb56c06c870 2024-04-25T09:57:23.933 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:57:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 -- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb55006b4d0 msgr2=0x7fb55006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb55006b4d0 0x7fb55006d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb55c033000 tx=0x7fb55c02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 -- 172.21.15.19:0/841500370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c06c870 msgr2=0x7fb56c1e7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c06c870 0x7fb56c1e7090 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb56800e9b0 tx=0x7fb56800ee70 comp rx=0 tx=0).stop 2024-04-25T09:57:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 -- 172.21.15.19:0/841500370 shutdown_connections 2024-04-25T09:57:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb55006b4d0 0x7fb55006d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb56c158230 0x7fb56c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb56c0d1b80 0x7fb56c146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 --2- 172.21.15.19:0/841500370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56c06c870 0x7fb56c1e7090 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 -- 172.21.15.19:0/841500370 >> 172.21.15.19:0/841500370 conn(0x7fb56c061600 msgr2=0x7fb56c0cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 -- 172.21.15.19:0/841500370 shutdown_connections 2024-04-25T09:57:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:23.931+0000 7fb574113640 1 -- 172.21.15.19:0/841500370 wait complete. 2024-04-25T09:57:23.946 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:57:24.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.043+0000 7f3aced6a640 1 Processor -- start 2024-04-25T09:57:24.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.043+0000 7f3aced6a640 1 -- start start 2024-04-25T09:57:24.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3aced6a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac815dd30 0x7f3ac80b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3aced6a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3ac80b9770 0x7f3ac80d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3aced6a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac80d1df0 con 0x7f3ac80d2e90 2024-04-25T09:57:24.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3aced6a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac80b7a50 con 0x7f3ac80b9770 2024-04-25T09:57:24.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3aced6a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80d20d0 con 0x7f3ac815dd30 2024-04-25T09:57:24.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3acd307640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3ac80d2e90 0x7f3ac80b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39922/0 (socket says 172.21.15.19:39922) 2024-04-25T09:57:24.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3acd307640 1 -- 172.21.15.19:0/985164854 learned_addr learned my addr 172.21.15.19:0/985164854 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:24.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac7fff640 1 --2- 172.21.15.19:0/985164854 >> v2:172.21.15.112:3300/0 conn(0x7f3ac80b9770 0x7f3ac80d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3accb06640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac815dd30 0x7f3ac80b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376844126 0 0) 0x7f3ac80d1df0 con 0x7f3ac80d2e90 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab8003520 con 0x7f3ac80d2e90 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 6366325 0 0) 0x7f3ab8003520 con 0x7f3ac80d2e90 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac815dd30 msgr2=0x7f3ac80b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac77fe640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac815dd30 0x7f3ac80b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 >> v2:172.21.15.112:3300/0 conn(0x7f3ac80b9770 msgr2=0x7f3ac80d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac77fe640 1 --2- 172.21.15.19:0/985164854 >> v2:172.21.15.112:3300/0 conn(0x7f3ac80b9770 0x7f3ac80d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac80d23b0 con 0x7f3ac80d2e90 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f3ac00019f0 con 0x7f3ac80d2e90 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 >> v1:172.21.15.19:6789/0 conn(0x7f3ac80d2e90 legacy=0x7f3ac80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.044+0000 7f3ac77fe640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ab80039b0 0x7f3ab8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3ac77fe640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac815dd30 0x7f3ab8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3ac77fe640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab8004d40 0x7f3ab80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80b7a50 con 0x7f3ab80039b0 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80d20d0 con 0x7f3ab8004d40 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab800b9f0 con 0x7f3ac815dd30 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3accb06640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ab80039b0 0x7f3ab8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3ac7fff640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac815dd30 0x7f3ab8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3acd307640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab8004d40 0x7f3ab80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3accb06640 1 -- 172.21.15.19:0/985164854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac815dd30 msgr2=0x7f3ab8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3accb06640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac815dd30 0x7f3ab8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3accb06640 1 -- 172.21.15.19:0/985164854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab8004d40 msgr2=0x7f3ab80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3accb06640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab8004d40 0x7f3ab80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.045+0000 7f3accb06640 1 -- 172.21.15.19:0/985164854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab8003520 con 0x7f3ab80039b0 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.046+0000 7f3accb06640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ab80039b0 0x7f3ab8003dd0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009ac0 tx=0x7f3ab8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f765aeb08bcca06c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.046+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc036070 con 0x7f3ab80039b0 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.046+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc031400 con 0x7f3ab80039b0 2024-04-25T09:57:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.046+0000 7f3ac77fe640 1 -- 172.21.15.19:0/985164854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc009690 con 0x7f3ab80039b0 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.046+0000 7f3aced6a640 1 -- 172.21.15.19:0/985164854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ab80039b0 msgr2=0x7f3ab8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.046+0000 7f3aced6a640 1 --2- 172.21.15.19:0/985164854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ab80039b0 0x7f3ab8003dd0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009ac0 tx=0x7f3ab8005830 comp rx=0 tx=0).stop 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.046+0000 7f3acd307640 1 -- 172.21.15.19:0/985164854 reap_dead start 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.046+0000 7f3aced6a640 1 -- 172.21.15.19:0/985164854 shutdown_connections 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.046+0000 7f3aced6a640 1 -- 172.21.15.19:0/985164854 >> 172.21.15.19:0/985164854 conn(0x7f3ac8061c30 msgr2=0x7f3ac806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.046+0000 7f3aced6a640 1 -- 172.21.15.19:0/985164854 shutdown_connections 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.046+0000 7f3aced6a640 1 -- 172.21.15.19:0/985164854 wait complete. 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.048+0000 7f3aced6a640 1 Processor -- start 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.048+0000 7f3aced6a640 1 -- start start 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.048+0000 7f3aced6a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab80039b0 0x7f3ac8151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.048+0000 7f3aced6a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac80b5440 0x7f3ac80b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.048+0000 7f3aced6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac80b31f0 0x7f3ac80b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.048+0000 7f3aced6a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80d1a10 con 0x7f3ac80b31f0 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.048+0000 7f3aced6a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80d2970 con 0x7f3ab80039b0 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.048+0000 7f3aced6a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80d1b90 con 0x7f3ac80b5440 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3acd307640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac80b31f0 0x7f3ac80b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3acd307640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac80b31f0 0x7f3ac80b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44654/0 (socket says 172.21.15.19:44654) 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3acd307640 1 -- 172.21.15.19:0/2141700134 learned_addr learned my addr 172.21.15.19:0/2141700134 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3accb06640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab80039b0 0x7f3ac8151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3ac7fff640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac80b5440 0x7f3ac80b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3acd307640 1 -- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac80b5440 msgr2=0x7f3ac80b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3acd307640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac80b5440 0x7f3ac80b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3acd307640 1 -- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab80039b0 msgr2=0x7f3ac8151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3acd307640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab80039b0 0x7f3ac8151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3acd307640 1 -- 172.21.15.19:0/2141700134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac80b4ae0 con 0x7f3ac80b31f0 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3ac7fff640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac80b5440 0x7f3ac80b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3acd307640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac80b31f0 0x7f3ac80b3670 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00013d0 tx=0x7f3ac0007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3ac57fa640 1 -- 172.21.15.19:0/2141700134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000ac20 con 0x7f3ac80b31f0 2024-04-25T09:57:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3ac57fa640 1 -- 172.21.15.19:0/2141700134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac0018070 con 0x7f3ac80b31f0 2024-04-25T09:57:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3ac57fa640 1 -- 172.21.15.19:0/2141700134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00133f0 con 0x7f3ac80b31f0 2024-04-25T09:57:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3aced6a640 1 -- 172.21.15.19:0/2141700134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac80b3f60 con 0x7f3ac80b31f0 2024-04-25T09:57:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.049+0000 7f3aced6a640 1 -- 172.21.15.19:0/2141700134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac80b4520 con 0x7f3ac80b31f0 2024-04-25T09:57:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.050+0000 7f3aced6a640 1 -- 172.21.15.19:0/2141700134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac8073990 con 0x7f3ac80b31f0 2024-04-25T09:57:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.051+0000 7f3ac57fa640 1 -- 172.21.15.19:0/2141700134 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ac0013590 con 0x7f3ac80b31f0 2024-04-25T09:57:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.051+0000 7f3ac57fa640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3aac06b4d0 0x7f3aac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.051+0000 7f3ac57fa640 1 -- 172.21.15.19:0/2141700134 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3ac008a580 con 0x7f3ac80b31f0 2024-04-25T09:57:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.052+0000 7f3accb06640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3aac06b4d0 0x7f3aac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.052+0000 7f3accb06640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3aac06b4d0 0x7f3aac06d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3abc033000 tx=0x7f3abc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.055+0000 7f3ac57fa640 1 -- 172.21.15.19:0/2141700134 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ac00595c0 con 0x7f3ac80b31f0 2024-04-25T09:57:24.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.241+0000 7f3aced6a640 1 -- 172.21.15.19:0/2141700134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3ac80b8940 con 0x7f3ac80b31f0 2024-04-25T09:57:24.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.241+0000 7f3ac57fa640 1 -- 172.21.15.19:0/2141700134 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3ac0058ec0 con 0x7f3ac80b31f0 2024-04-25T09:57:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:24.245 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T09:57:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:57:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T09:57:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T09:57:24.245 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:57:24.245 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T09:57:24.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 -- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3aac06b4d0 msgr2=0x7f3aac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3aac06b4d0 0x7f3aac06d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3abc033000 tx=0x7f3abc008030 comp rx=0 tx=0).stop 2024-04-25T09:57:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 -- 172.21.15.19:0/2141700134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac80b31f0 msgr2=0x7f3ac80b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac80b31f0 0x7f3ac80b3670 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00013d0 tx=0x7f3ac0007bb0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 -- 172.21.15.19:0/2141700134 shutdown_connections 2024-04-25T09:57:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3aac06b4d0 0x7f3aac06d990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac80b31f0 0x7f3ac80b3670 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ac80b5440 0x7f3ac80b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 --2- 172.21.15.19:0/2141700134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab80039b0 0x7f3ac8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 -- 172.21.15.19:0/2141700134 >> 172.21.15.19:0/2141700134 conn(0x7f3ac8061c30 msgr2=0x7f3ac80cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 -- 172.21.15.19:0/2141700134 shutdown_connections 2024-04-25T09:57:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.243+0000 7f3aced6a640 1 -- 172.21.15.19:0/2141700134 wait complete. 2024-04-25T09:57:24.257 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:57:24.257 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:57:24.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.358+0000 7fb4915b0640 1 Processor -- start 2024-04-25T09:57:24.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.358+0000 7fb4915b0640 1 -- start start 2024-04-25T09:57:24.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.359+0000 7fb4915b0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb48c0cf600 0x7fb48c0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.359+0000 7fb4915b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb48c0cbf20 0x7fb48c0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.359+0000 7fb4915b0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c0c75e0 con 0x7fb48c15bb50 2024-04-25T09:57:24.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.359+0000 7fb4915b0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c0c8680 con 0x7fb48c0cf600 2024-04-25T09:57:24.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.359+0000 7fb4915b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0cff80 con 0x7fb48c0cbf20 2024-04-25T09:57:24.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.359+0000 7fb48a7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb48c0cf600 0x7fb48c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.359+0000 7fb48a7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb48c0cf600 0x7fb48c0cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44664/0 (socket says 172.21.15.19:44664) 2024-04-25T09:57:24.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.359+0000 7fb48a7fc640 1 -- 172.21.15.19:0/1984684879 learned_addr learned my addr 172.21.15.19:0/1984684879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:24.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.359+0000 7fb48b7fe640 1 --2- 172.21.15.19:0/1984684879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb48c0cbf20 0x7fb48c0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.360+0000 7fb48a7fc640 1 -- 172.21.15.19:0/1984684879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb48c0cbf20 msgr2=0x7fb48c0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.360+0000 7fb48a7fc640 1 --2- 172.21.15.19:0/1984684879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb48c0cbf20 0x7fb48c0cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.360+0000 7fb48a7fc640 1 -- 172.21.15.19:0/1984684879 >> v1:172.21.15.112:6789/0 conn(0x7fb48c15bb50 legacy=0x7fb48c0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.360+0000 7fb48a7fc640 1 -- 172.21.15.19:0/1984684879 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c0c78c0 con 0x7fb48c0cf600 2024-04-25T09:57:24.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.360+0000 7fb48a7fc640 1 --2- 172.21.15.19:0/1984684879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c0cf600 0x7fb48c0cfa40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb480009870 tx=0x7fb48002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f987ec0caf32181 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.361+0000 7fb489ffb640 1 -- 172.21.15.19:0/1984684879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480037070 con 0x7fb48c0cf600 2024-04-25T09:57:24.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.361+0000 7fb489ffb640 1 -- 172.21.15.19:0/1984684879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48002ae50 con 0x7fb48c0cf600 2024-04-25T09:57:24.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.361+0000 7fb489ffb640 1 -- 172.21.15.19:0/1984684879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480032860 con 0x7fb48c0cf600 2024-04-25T09:57:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.361+0000 7fb4915b0640 1 -- 172.21.15.19:0/1984684879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c0cf600 msgr2=0x7fb48c0cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.361+0000 7fb4915b0640 1 --2- 172.21.15.19:0/1984684879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c0cf600 0x7fb48c0cfa40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb480009870 tx=0x7fb48002a300 comp rx=0 tx=0).stop 2024-04-25T09:57:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.361+0000 7fb4915b0640 1 -- 172.21.15.19:0/1984684879 shutdown_connections 2024-04-25T09:57:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.361+0000 7fb4915b0640 1 --2- 172.21.15.19:0/1984684879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb48c0cbf20 0x7fb48c0cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.361+0000 7fb4915b0640 1 --2- 172.21.15.19:0/1984684879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c0cf600 0x7fb48c0cfa40 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.361+0000 7fb4915b0640 1 -- 172.21.15.19:0/1984684879 >> 172.21.15.19:0/1984684879 conn(0x7fb48c06f500 msgr2=0x7fb48c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.361+0000 7fb4915b0640 1 -- 172.21.15.19:0/1984684879 shutdown_connections 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.361+0000 7fb4915b0640 1 -- 172.21.15.19:0/1984684879 wait complete. 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.362+0000 7fb4915b0640 1 Processor -- start 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.363+0000 7fb4915b0640 1 -- start start 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.363+0000 7fb4915b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb48c0cbf20 0x7fb48c1e5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.363+0000 7fb4915b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c0cf600 0x7fb48c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.363+0000 7fb4915b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb48c15bb50 0x7fb48c0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.363+0000 7fb4915b0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0c8160 con 0x7fb48c0cf600 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.363+0000 7fb4915b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0c7e80 con 0x7fb48c0cbf20 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.363+0000 7fb4915b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0c72a0 con 0x7fb48c15bb50 2024-04-25T09:57:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.363+0000 7fb48a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c0cf600 0x7fb48c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.363+0000 7fb48a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c0cf600 0x7fb48c15df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44668/0 (socket says 172.21.15.19:44668) 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.363+0000 7fb48a7fc640 1 -- 172.21.15.19:0/302251871 learned_addr learned my addr 172.21.15.19:0/302251871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb48b7fe640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb48c15bb50 0x7fb48c0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb48affd640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb48c0cbf20 0x7fb48c1e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb48a7fc640 1 -- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb48c15bb50 msgr2=0x7fb48c0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb48a7fc640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb48c15bb50 0x7fb48c0cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb48a7fc640 1 -- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb48c0cbf20 msgr2=0x7fb48c1e5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb48a7fc640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb48c0cbf20 0x7fb48c1e5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb48a7fc640 1 -- 172.21.15.19:0/302251871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c1e59a0 con 0x7fb48c0cf600 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb48a7fc640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c0cf600 0x7fb48c15df70 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb480000c00 tx=0x7fb4800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb4737fe640 1 -- 172.21.15.19:0/302251871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480041070 con 0x7fb48c0cf600 2024-04-25T09:57:24.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb4737fe640 1 -- 172.21.15.19:0/302251871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48003c660 con 0x7fb48c0cf600 2024-04-25T09:57:24.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb4915b0640 1 -- 172.21.15.19:0/302251871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c1e5c80 con 0x7fb48c0cf600 2024-04-25T09:57:24.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb4737fe640 1 -- 172.21.15.19:0/302251871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480037040 con 0x7fb48c0cf600 2024-04-25T09:57:24.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.364+0000 7fb4915b0640 1 -- 172.21.15.19:0/302251871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c1e6240 con 0x7fb48c0cf600 2024-04-25T09:57:24.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.365+0000 7fb4915b0640 1 -- 172.21.15.19:0/302251871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c073a00 con 0x7fb48c0cf600 2024-04-25T09:57:24.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.366+0000 7fb4737fe640 1 -- 172.21.15.19:0/302251871 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb480044070 con 0x7fb48c0cf600 2024-04-25T09:57:24.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.366+0000 7fb4737fe640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb47806b4d0 0x7fb47806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.366+0000 7fb4737fe640 1 -- 172.21.15.19:0/302251871 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb4800aa750 con 0x7fb48c0cf600 2024-04-25T09:57:24.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.366+0000 7fb48affd640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb47806b4d0 0x7fb47806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.367+0000 7fb48affd640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb47806b4d0 0x7fb47806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb484001ca0 tx=0x7fb484009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.370+0000 7fb4737fe640 1 -- 172.21.15.19:0/302251871 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb480079940 con 0x7fb48c0cf600 2024-04-25T09:57:24.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.550+0000 7fb4915b0640 1 -- 172.21.15.19:0/302251871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb48c06fc60 con 0x7fb48c0cf600 2024-04-25T09:57:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.948+0000 7fb4737fe640 1 -- 172.21.15.19:0/302251871 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb480079240 con 0x7fb48c0cf600 2024-04-25T09:57:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T09:57:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 -- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb47806b4d0 msgr2=0x7fb47806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb47806b4d0 0x7fb47806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb484001ca0 tx=0x7fb484009040 comp rx=0 tx=0).stop 2024-04-25T09:57:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 -- 172.21.15.19:0/302251871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c0cf600 msgr2=0x7fb48c15df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c0cf600 0x7fb48c15df70 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb480000c00 tx=0x7fb4800046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 -- 172.21.15.19:0/302251871 shutdown_connections 2024-04-25T09:57:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb47806b4d0 0x7fb47806d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb48c15bb50 0x7fb48c0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb48c0cf600 0x7fb48c15df70 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 --2- 172.21.15.19:0/302251871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb48c0cbf20 0x7fb48c1e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 -- 172.21.15.19:0/302251871 >> 172.21.15.19:0/302251871 conn(0x7fb48c06f500 msgr2=0x7fb48c0c6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 -- 172.21.15.19:0/302251871 shutdown_connections 2024-04-25T09:57:24.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:24.950+0000 7fb4915b0640 1 -- 172.21.15.19:0/302251871 wait complete. 2024-04-25T09:57:24.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:57:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.073+0000 7f88f3ae4640 1 Processor -- start 2024-04-25T09:57:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.073+0000 7f88f3ae4640 1 -- start start 2024-04-25T09:57:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.074+0000 7f88f3ae4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f88ec15dc20 0x7f88ec0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.074+0000 7f88f3ae4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f88ec161710 0x7f88ec0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.074+0000 7f88f3ae4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec15e000 con 0x7f88ec0b9330 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.074+0000 7f88f3ae4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ec0d1de0 con 0x7f88ec161710 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.074+0000 7f88f3ae4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ec161c60 con 0x7f88ec15dc20 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.074+0000 7f88f107f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f88ec161710 0x7f88ec0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.074+0000 7f88f107f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f88ec161710 0x7f88ec0cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44676/0 (socket says 172.21.15.19:44676) 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.074+0000 7f88f107f640 1 -- 172.21.15.19:0/1051171253 learned_addr learned my addr 172.21.15.19:0/1051171253 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.075+0000 7f88f1880640 1 --2- 172.21.15.19:0/1051171253 >> v2:172.21.15.112:3300/0 conn(0x7f88ec15dc20 0x7f88ec0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.075+0000 7f88f107f640 1 -- 172.21.15.19:0/1051171253 >> v2:172.21.15.112:3300/0 conn(0x7f88ec15dc20 msgr2=0x7f88ec0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.075+0000 7f88f107f640 1 --2- 172.21.15.19:0/1051171253 >> v2:172.21.15.112:3300/0 conn(0x7f88ec15dc20 0x7f88ec0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.075+0000 7f88f107f640 1 -- 172.21.15.19:0/1051171253 >> v1:172.21.15.112:6789/0 conn(0x7f88ec0b9330 legacy=0x7f88ec06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.075+0000 7f88f107f640 1 -- 172.21.15.19:0/1051171253 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec161f40 con 0x7f88ec161710 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.075+0000 7f88f1880640 1 --2- 172.21.15.19:0/1051171253 >> v2:172.21.15.112:3300/0 conn(0x7f88ec15dc20 0x7f88ec0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.075+0000 7f88f107f640 1 --2- 172.21.15.19:0/1051171253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88ec161710 0x7f88ec0cebb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0d1520 tx=0x7f88e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207bf1d550948b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.076+0000 7f88f087e640 1 -- 172.21.15.19:0/1051171253 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e0037070 con 0x7f88ec161710 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.076+0000 7f88f087e640 1 -- 172.21.15.19:0/1051171253 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e0032440 con 0x7f88ec161710 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.076+0000 7f88f087e640 1 -- 172.21.15.19:0/1051171253 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e0032810 con 0x7f88ec161710 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.076+0000 7f88f3ae4640 1 -- 172.21.15.19:0/1051171253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88ec161710 msgr2=0x7f88ec0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.076+0000 7f88f3ae4640 1 --2- 172.21.15.19:0/1051171253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88ec161710 0x7f88ec0cebb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0d1520 tx=0x7f88e002a270 comp rx=0 tx=0).stop 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.076+0000 7f88f3ae4640 1 -- 172.21.15.19:0/1051171253 shutdown_connections 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.076+0000 7f88f3ae4640 1 --2- 172.21.15.19:0/1051171253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88ec161710 0x7f88ec0cebb0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.076+0000 7f88f3ae4640 1 --2- 172.21.15.19:0/1051171253 >> v2:172.21.15.112:3300/0 conn(0x7f88ec15dc20 0x7f88ec0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.076+0000 7f88f3ae4640 1 -- 172.21.15.19:0/1051171253 >> 172.21.15.19:0/1051171253 conn(0x7f88ec0d0420 msgr2=0x7f88ec0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.076+0000 7f88f3ae4640 1 -- 172.21.15.19:0/1051171253 shutdown_connections 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.076+0000 7f88f3ae4640 1 -- 172.21.15.19:0/1051171253 wait complete. 2024-04-25T09:57:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.078+0000 7f88f3ae4640 1 Processor -- start 2024-04-25T09:57:25.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.078+0000 7f88f3ae4640 1 -- start start 2024-04-25T09:57:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.078+0000 7f88f3ae4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88ec0b9330 0x7f88ec15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.078+0000 7f88f3ae4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88ec15dc20 0x7f88ec1f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.078+0000 7f88f3ae4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec161710 0x7f88ec0d2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.078+0000 7f88f3ae4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec0b5e50 con 0x7f88ec0b9330 2024-04-25T09:57:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.078+0000 7f88f3ae4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec162500 con 0x7f88ec161710 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.078+0000 7f88f3ae4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec0b5fd0 con 0x7f88ec15dc20 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.078+0000 7f88f1880640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88ec0b9330 0x7f88ec15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.078+0000 7f88f1880640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88ec0b9330 0x7f88ec15ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44692/0 (socket says 172.21.15.19:44692) 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.078+0000 7f88f1880640 1 -- 172.21.15.19:0/3852609604 learned_addr learned my addr 172.21.15.19:0/3852609604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88f107f640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88ec15dc20 0x7f88ec1f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88f2081640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec161710 0x7f88ec0d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88f1880640 1 -- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88ec15dc20 msgr2=0x7f88ec1f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88f1880640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88ec15dc20 0x7f88ec1f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88f1880640 1 -- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec161710 msgr2=0x7f88ec0d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88f1880640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec161710 0x7f88ec0d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88f1880640 1 -- 172.21.15.19:0/3852609604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec15f4a0 con 0x7f88ec0b9330 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88f1880640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88ec0b9330 0x7f88ec15ad80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f88e400d7d0 tx=0x7f88e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88ea7fc640 1 -- 172.21.15.19:0/3852609604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4004030 con 0x7f88ec0b9330 2024-04-25T09:57:25.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88ea7fc640 1 -- 172.21.15.19:0/3852609604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e40041d0 con 0x7f88ec0b9330 2024-04-25T09:57:25.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88ea7fc640 1 -- 172.21.15.19:0/3852609604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e40096a0 con 0x7f88ec0b9330 2024-04-25T09:57:25.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88f3ae4640 1 -- 172.21.15.19:0/3852609604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ec15f740 con 0x7f88ec0b9330 2024-04-25T09:57:25.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.079+0000 7f88f3ae4640 1 -- 172.21.15.19:0/3852609604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ec15fd00 con 0x7f88ec0b9330 2024-04-25T09:57:25.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.080+0000 7f88f3ae4640 1 -- 172.21.15.19:0/3852609604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ec073a00 con 0x7f88ec0b9330 2024-04-25T09:57:25.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.081+0000 7f88ea7fc640 1 -- 172.21.15.19:0/3852609604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88e4009840 con 0x7f88ec0b9330 2024-04-25T09:57:25.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.081+0000 7f88ea7fc640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88d806b4d0 0x7f88d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.081+0000 7f88ea7fc640 1 -- 172.21.15.19:0/3852609604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f88e408a2e0 con 0x7f88ec0b9330 2024-04-25T09:57:25.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.081+0000 7f88f107f640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88d806b4d0 0x7f88d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.082+0000 7f88f107f640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88d806b4d0 0x7f88d806d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f88e0034000 tx=0x7f88e002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.086+0000 7f88ea7fc640 1 -- 172.21.15.19:0/3852609604 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88e4058b60 con 0x7f88ec0b9330 2024-04-25T09:57:25.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.265+0000 7f88f3ae4640 1 -- 172.21.15.19:0/3852609604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f88ec0d16b0 con 0x7f88ec0b9330 2024-04-25T09:57:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.955+0000 7f88ea7fc640 1 -- 172.21.15.19:0/3852609604 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) v1 ==== 170+0+0 (secure 0 0 0) 0x7f88e405cc00 con 0x7f88ec0b9330 2024-04-25T09:57:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 -- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88d806b4d0 msgr2=0x7f88d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88d806b4d0 0x7f88d806d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f88e0034000 tx=0x7f88e002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 -- 172.21.15.19:0/3852609604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88ec0b9330 msgr2=0x7f88ec15ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88ec0b9330 0x7f88ec15ad80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f88e400d7d0 tx=0x7f88e400dc90 comp rx=0 tx=0).stop 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 -- 172.21.15.19:0/3852609604 shutdown_connections 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88d806b4d0 0x7f88d806d990 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec161710 0x7f88ec0d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88ec15dc20 0x7f88ec1f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 --2- 172.21.15.19:0/3852609604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88ec0b9330 0x7f88ec15ad80 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 -- 172.21.15.19:0/3852609604 >> 172.21.15.19:0/3852609604 conn(0x7f88ec0d0420 msgr2=0x7f88ec0b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 -- 172.21.15.19:0/3852609604 shutdown_connections 2024-04-25T09:57:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:25.958+0000 7f88f3ae4640 1 -- 172.21.15.19:0/3852609604 wait complete. 2024-04-25T09:57:25.985 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:57:26.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.091+0000 7fd2ba415640 1 Processor -- start 2024-04-25T09:57:26.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.091+0000 7fd2ba415640 1 -- start start 2024-04-25T09:57:26.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.091+0000 7fd2ba415640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd2b40ced80 0x7fd2b40d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.091+0000 7fd2ba415640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b415e000 con 0x7fd2b415dc20 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.091+0000 7fd2ba415640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b40d1de0 con 0x7fd2b4161710 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.091+0000 7fd2ba415640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b4161c60 con 0x7fd2b40ced80 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.092+0000 7fd2b3fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd2b415dc20 0x7fd2b40ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39936/0 (socket says 172.21.15.19:39936) 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.092+0000 7fd2b3fff640 1 -- 172.21.15.19:0/4151717087 learned_addr learned my addr 172.21.15.19:0/4151717087 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.092+0000 7fd2b37fe640 1 --2- 172.21.15.19:0/4151717087 >> v2:172.21.15.112:3300/0 conn(0x7fd2b40ced80 0x7fd2b40d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.092+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/4151717087 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140531783 0 0) 0x7fd2b415e000 con 0x7fd2b415dc20 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.092+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/4151717087 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd29c003520 con 0x7fd2b415dc20 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.092+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/4151717087 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371293292 0 0) 0x7fd2b40d1de0 con 0x7fd2b4161710 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.092+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/4151717087 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2b415e000 con 0x7fd2b4161710 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.092+0000 7fd2b37fe640 1 -- 172.21.15.19:0/4151717087 >> v1:172.21.15.112:6789/0 conn(0x7fd2b4161710 legacy=0x7fd2b40d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.092+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/4151717087 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3320711125 0 0) 0x7fd29c003520 con 0x7fd2b415dc20 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.092+0000 7fd2b37fe640 1 -- 172.21.15.19:0/4151717087 >> v1:172.21.15.19:6789/0 conn(0x7fd2b415dc20 legacy=0x7fd2b40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.092+0000 7fd2b37fe640 1 -- 172.21.15.19:0/4151717087 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b4161f40 con 0x7fd2b40ced80 2024-04-25T09:57:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.093+0000 7fd2b37fe640 1 --2- 172.21.15.19:0/4151717087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b40ced80 0x7fd2b40d22e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80097a0 tx=0x7fd2a802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b896171929378467 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.093+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/4151717087 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8037070 con 0x7fd2b40ced80 2024-04-25T09:57:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.093+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/4151717087 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a802ac50 con 0x7fd2b40ced80 2024-04-25T09:57:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.093+0000 7fd2b2ffd640 1 -- 172.21.15.19:0/4151717087 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4001670 con 0x7fd2b40ced80 2024-04-25T09:57:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.094+0000 7fd2ba415640 1 -- 172.21.15.19:0/4151717087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b40ced80 msgr2=0x7fd2b40d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.094+0000 7fd2ba415640 1 --2- 172.21.15.19:0/4151717087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b40ced80 0x7fd2b40d22e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80097a0 tx=0x7fd2a802a1c0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.094+0000 7fd2ba415640 1 -- 172.21.15.19:0/4151717087 shutdown_connections 2024-04-25T09:57:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.094+0000 7fd2ba415640 1 --2- 172.21.15.19:0/4151717087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b40ced80 0x7fd2b40d22e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.094+0000 7fd2ba415640 1 -- 172.21.15.19:0/4151717087 >> 172.21.15.19:0/4151717087 conn(0x7fd2b40d0420 msgr2=0x7fd2b40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.094+0000 7fd2ba415640 1 -- 172.21.15.19:0/4151717087 shutdown_connections 2024-04-25T09:57:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.094+0000 7fd2ba415640 1 -- 172.21.15.19:0/4151717087 wait complete. 2024-04-25T09:57:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.095+0000 7fd2ba415640 1 Processor -- start 2024-04-25T09:57:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.095+0000 7fd2ba415640 1 -- start start 2024-04-25T09:57:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2ba415640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b40ced80 0x7fd2b40b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2ba415640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b415dc20 0x7fd2b415fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2ba415640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2b4161710 0x7fd2b4160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2ba415640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b40b5e50 con 0x7fd2b40ced80 2024-04-25T09:57:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2ba415640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4162500 con 0x7fd2b415dc20 2024-04-25T09:57:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2ba415640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b40b5fd0 con 0x7fd2b4161710 2024-04-25T09:57:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b40ced80 0x7fd2b40b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b40ced80 0x7fd2b40b1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44708/0 (socket says 172.21.15.19:44708) 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2b3fff640 1 -- 172.21.15.19:0/1924193759 learned_addr learned my addr 172.21.15.19:0/1924193759 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2b37fe640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b415dc20 0x7fd2b415fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2b89b2640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2b4161710 0x7fd2b4160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2b3fff640 1 -- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2b4161710 msgr2=0x7fd2b4160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2b3fff640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2b4161710 0x7fd2b4160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2b3fff640 1 -- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b415dc20 msgr2=0x7fd2b415fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2b3fff640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b415dc20 0x7fd2b415fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2b3fff640 1 -- 172.21.15.19:0/1924193759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b40cb760 con 0x7fd2b40ced80 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.096+0000 7fd2b3fff640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b40ced80 0x7fd2b40b1940 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400c5a0 tx=0x7fd2a400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.097+0000 7fd2b0ff9640 1 -- 172.21.15.19:0/1924193759 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4017070 con 0x7fd2b40ced80 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.097+0000 7fd2b0ff9640 1 -- 172.21.15.19:0/1924193759 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a4005ce0 con 0x7fd2b40ced80 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.097+0000 7fd2ba415640 1 -- 172.21.15.19:0/1924193759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b40cba40 con 0x7fd2b40ced80 2024-04-25T09:57:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.097+0000 7fd2b0ff9640 1 -- 172.21.15.19:0/1924193759 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40072b0 con 0x7fd2b40ced80 2024-04-25T09:57:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.097+0000 7fd2ba415640 1 -- 172.21.15.19:0/1924193759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b414dba0 con 0x7fd2b40ced80 2024-04-25T09:57:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.098+0000 7fd2ba415640 1 -- 172.21.15.19:0/1924193759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b4073a00 con 0x7fd2b40ced80 2024-04-25T09:57:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.098+0000 7fd2b0ff9640 1 -- 172.21.15.19:0/1924193759 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2a401a070 con 0x7fd2b40ced80 2024-04-25T09:57:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.099+0000 7fd2b0ff9640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd2a006b4d0 0x7fd2a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.099+0000 7fd2b0ff9640 1 -- 172.21.15.19:0/1924193759 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd2a40897b0 con 0x7fd2b40ced80 2024-04-25T09:57:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.099+0000 7fd2b37fe640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd2a006b4d0 0x7fd2a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.099+0000 7fd2b37fe640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd2a006b4d0 0x7fd2a006d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8034000 tx=0x7fd2a802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.103+0000 7fd2b0ff9640 1 -- 172.21.15.19:0/1924193759 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2a4057eb0 con 0x7fd2b40ced80 2024-04-25T09:57:26.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.326+0000 7fd2ba415640 1 -- 172.21.15.19:0/1924193759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd2b40d16b0 con 0x7fd2b40ced80 2024-04-25T09:57:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.957+0000 7fd2b0ff9640 1 -- 172.21.15.19:0/1924193759 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v48) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd2a405bf50 con 0x7fd2b40ced80 2024-04-25T09:57:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T09:57:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.960+0000 7fd2ba415640 1 -- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd2a006b4d0 msgr2=0x7fd2a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.960+0000 7fd2ba415640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd2a006b4d0 0x7fd2a006d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8034000 tx=0x7fd2a802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.960+0000 7fd2ba415640 1 -- 172.21.15.19:0/1924193759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b40ced80 msgr2=0x7fd2b40b1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.960+0000 7fd2ba415640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b40ced80 0x7fd2b40b1940 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400c5a0 tx=0x7fd2a400ca60 comp rx=0 tx=0).stop 2024-04-25T09:57:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.960+0000 7fd2ba415640 1 -- 172.21.15.19:0/1924193759 shutdown_connections 2024-04-25T09:57:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.960+0000 7fd2ba415640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd2a006b4d0 0x7fd2a006d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.960+0000 7fd2ba415640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2b4161710 0x7fd2b4160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.960+0000 7fd2ba415640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b415dc20 0x7fd2b415fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.960+0000 7fd2ba415640 1 --2- 172.21.15.19:0/1924193759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2b40ced80 0x7fd2b40b1940 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.960+0000 7fd2ba415640 1 -- 172.21.15.19:0/1924193759 >> 172.21.15.19:0/1924193759 conn(0x7fd2b40d0420 msgr2=0x7fd2b40d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.961+0000 7fd2ba415640 1 -- 172.21.15.19:0/1924193759 shutdown_connections 2024-04-25T09:57:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:26.962+0000 7fd2ba415640 1 -- 172.21.15.19:0/1924193759 wait complete. 2024-04-25T09:57:26.999 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:27.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.101+0000 7f3899bd5640 1 Processor -- start 2024-04-25T09:57:27.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.101+0000 7f3899bd5640 1 -- start start 2024-04-25T09:57:27.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f3899bd5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38940d2e90 0x7f38940d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f3899bd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38940d0900 0x7f38940b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f3899bd5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38940b9940 con 0x7f389415dd30 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f3899bd5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38940d0d40 con 0x7f38940d2e90 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f3899bd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38940d1df0 con 0x7f38940d0900 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f3892ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38940d2e90 0x7f38940d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f38937fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f389415dd30 0x7f38940b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39940/0 (socket says 172.21.15.19:39940) 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f38937fe640 1 -- 172.21.15.19:0/1952108003 learned_addr learned my addr 172.21.15.19:0/1952108003 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f3893fff640 1 --2- 172.21.15.19:0/1952108003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38940d0900 0x7f38940b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f3892ffd640 1 -- 172.21.15.19:0/1952108003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38940d0900 msgr2=0x7f38940b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f3892ffd640 1 --2- 172.21.15.19:0/1952108003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38940d0900 0x7f38940b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f3892ffd640 1 -- 172.21.15.19:0/1952108003 >> v1:172.21.15.19:6789/0 conn(0x7f389415dd30 legacy=0x7f38940b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.102+0000 7f3892ffd640 1 -- 172.21.15.19:0/1952108003 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38940d20d0 con 0x7f38940d2e90 2024-04-25T09:57:27.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.103+0000 7f3892ffd640 1 --2- 172.21.15.19:0/1952108003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38940d2e90 0x7f38940d03c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f387c009790 tx=0x7f387c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8798db606ce0b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.103+0000 7f38927fc640 1 -- 172.21.15.19:0/1952108003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c037070 con 0x7f38940d2e90 2024-04-25T09:57:27.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.103+0000 7f38927fc640 1 -- 172.21.15.19:0/1952108003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c02ae50 con 0x7f38940d2e90 2024-04-25T09:57:27.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.103+0000 7f38927fc640 1 -- 172.21.15.19:0/1952108003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c032660 con 0x7f38940d2e90 2024-04-25T09:57:27.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.104+0000 7f3899bd5640 1 -- 172.21.15.19:0/1952108003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38940d2e90 msgr2=0x7f38940d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.104+0000 7f3899bd5640 1 --2- 172.21.15.19:0/1952108003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38940d2e90 0x7f38940d03c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f387c009790 tx=0x7f387c02a100 comp rx=0 tx=0).stop 2024-04-25T09:57:27.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.104+0000 7f3899bd5640 1 -- 172.21.15.19:0/1952108003 shutdown_connections 2024-04-25T09:57:27.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.104+0000 7f3899bd5640 1 --2- 172.21.15.19:0/1952108003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38940d0900 0x7f38940b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.104+0000 7f3899bd5640 1 --2- 172.21.15.19:0/1952108003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38940d2e90 0x7f38940d03c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.104+0000 7f3899bd5640 1 -- 172.21.15.19:0/1952108003 >> 172.21.15.19:0/1952108003 conn(0x7f3894061c30 msgr2=0x7f389406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.104+0000 7f3899bd5640 1 -- 172.21.15.19:0/1952108003 shutdown_connections 2024-04-25T09:57:27.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.104+0000 7f3899bd5640 1 -- 172.21.15.19:0/1952108003 wait complete. 2024-04-25T09:57:27.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.105+0000 7f3899bd5640 1 Processor -- start 2024-04-25T09:57:27.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.105+0000 7f3899bd5640 1 -- start start 2024-04-25T09:57:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3899bd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38940d0900 0x7f38941f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3899bd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38940d2e90 0x7f389414d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3899bd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389415dd30 0x7f389414a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3899bd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38940d2970 con 0x7f389415dd30 2024-04-25T09:57:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3899bd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38940d2690 con 0x7f38940d0900 2024-04-25T09:57:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3899bd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38940d1a10 con 0x7f38940d2e90 2024-04-25T09:57:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3893fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389415dd30 0x7f389414a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3893fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389415dd30 0x7f389414a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44732/0 (socket says 172.21.15.19:44732) 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3893fff640 1 -- 172.21.15.19:0/1437156373 learned_addr learned my addr 172.21.15.19:0/1437156373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f38937fe640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38940d0900 0x7f38941f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3892ffd640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38940d2e90 0x7f389414d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3893fff640 1 -- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38940d2e90 msgr2=0x7f389414d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3893fff640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38940d2e90 0x7f389414d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3893fff640 1 -- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38940d0900 msgr2=0x7f38941f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3893fff640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38940d0900 0x7f38941f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.106+0000 7f3893fff640 1 -- 172.21.15.19:0/1437156373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38941f7d10 con 0x7f389415dd30 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.107+0000 7f3893fff640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389415dd30 0x7f389414a5d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f388c00d800 tx=0x7f388c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.107+0000 7f388bfff640 1 -- 172.21.15.19:0/1437156373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c00b6c0 con 0x7f389415dd30 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.107+0000 7f388bfff640 1 -- 172.21.15.19:0/1437156373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c019070 con 0x7f389415dd30 2024-04-25T09:57:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.107+0000 7f3899bd5640 1 -- 172.21.15.19:0/1437156373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38941f7470 con 0x7f389415dd30 2024-04-25T09:57:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.107+0000 7f388bfff640 1 -- 172.21.15.19:0/1437156373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c00bac0 con 0x7f389415dd30 2024-04-25T09:57:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.107+0000 7f3899bd5640 1 -- 172.21.15.19:0/1437156373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38941f7a30 con 0x7f389415dd30 2024-04-25T09:57:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.108+0000 7f3899bd5640 1 -- 172.21.15.19:0/1437156373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3894073990 con 0x7f389415dd30 2024-04-25T09:57:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.108+0000 7f388bfff640 1 -- 172.21.15.19:0/1437156373 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f388c00bc60 con 0x7f389415dd30 2024-04-25T09:57:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.109+0000 7f388bfff640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.109+0000 7f388bfff640 1 -- 172.21.15.19:0/1437156373 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f388c08a8c0 con 0x7f389415dd30 2024-04-25T09:57:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.109+0000 7f38937fe640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.109+0000 7f38937fe640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387406b4d0 0x7f387406d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f38840012e0 tx=0x7f3884009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.113+0000 7f388bfff640 1 -- 172.21.15.19:0/1437156373 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f388c059510 con 0x7f389415dd30 2024-04-25T09:57:27.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.292+0000 7f3899bd5640 1 -- 172.21.15.19:0/1437156373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38940d45c0 con 0x7f389415dd30 2024-04-25T09:57:27.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.293+0000 7f388bfff640 1 -- 172.21.15.19:0/1437156373 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16946 (secure 0 0 0) 0x7f388c058e10 con 0x7f389415dd30 2024-04-25T09:57:27.296 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:27.296 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:57:26.327665+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:24.581855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:25.297009+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:27.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 -- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387406b4d0 msgr2=0x7f387406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387406b4d0 0x7f387406d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f38840012e0 tx=0x7f3884009040 comp rx=0 tx=0).stop 2024-04-25T09:57:27.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 -- 172.21.15.19:0/1437156373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389415dd30 msgr2=0x7f389414a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389415dd30 0x7f389414a5d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f388c00d800 tx=0x7f388c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 -- 172.21.15.19:0/1437156373 shutdown_connections 2024-04-25T09:57:27.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f389415dd30 0x7f389414a5d0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38940d2e90 0x7f389414d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 --2- 172.21.15.19:0/1437156373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38940d0900 0x7f38941f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 -- 172.21.15.19:0/1437156373 >> 172.21.15.19:0/1437156373 conn(0x7f3894061c30 msgr2=0x7f38940b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 -- 172.21.15.19:0/1437156373 shutdown_connections 2024-04-25T09:57:27.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.295+0000 7f3899bd5640 1 -- 172.21.15.19:0/1437156373 wait complete. 2024-04-25T09:57:27.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:27.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.407+0000 7f0421716640 1 Processor -- start 2024-04-25T09:57:27.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.407+0000 7f0421716640 1 -- start start 2024-04-25T09:57:27.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.408+0000 7f0421716640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f041c15c9e0 0x7f041c071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.408+0000 7f0421716640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c072400 con 0x7f041c155fe0 2024-04-25T09:57:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.408+0000 7f0421716640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c06de40 con 0x7f041c072000 2024-04-25T09:57:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.408+0000 7f0421716640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c15ce70 con 0x7f041c15c9e0 2024-04-25T09:57:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.408+0000 7f041a7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f041c15c9e0 0x7f041c071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.408+0000 7f041affd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f041c155fe0 0x7f041c0adbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39950/0 (socket says 172.21.15.19:39950) 2024-04-25T09:57:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.408+0000 7f041affd640 1 -- 172.21.15.19:0/3844788999 learned_addr learned my addr 172.21.15.19:0/3844788999 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.408+0000 7f041a7fc640 1 -- 172.21.15.19:0/3844788999 >> v1:172.21.15.112:6789/0 conn(0x7f041c072000 legacy=0x7f041c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.408+0000 7f041a7fc640 1 -- 172.21.15.19:0/3844788999 >> v1:172.21.15.19:6789/0 conn(0x7f041c155fe0 legacy=0x7f041c0adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.408+0000 7f0419ffb640 1 -- 172.21.15.19:0/3844788999 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676971427 0 0) 0x7f041c072400 con 0x7f041c155fe0 2024-04-25T09:57:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.408+0000 7f041a7fc640 1 -- 172.21.15.19:0/3844788999 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0ad0e0 con 0x7f041c15c9e0 2024-04-25T09:57:27.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.409+0000 7f041a7fc640 1 --2- 172.21.15.19:0/3844788999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f041c15c9e0 0x7f041c071ac0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f04040097a0 tx=0x7f040402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8bd6b27b4c33a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.409+0000 7f0419ffb640 1 -- 172.21.15.19:0/3844788999 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404037070 con 0x7f041c15c9e0 2024-04-25T09:57:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.409+0000 7f0419ffb640 1 -- 172.21.15.19:0/3844788999 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0404032440 con 0x7f041c15c9e0 2024-04-25T09:57:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.409+0000 7f0419ffb640 1 -- 172.21.15.19:0/3844788999 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404032810 con 0x7f041c15c9e0 2024-04-25T09:57:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.410+0000 7f0421716640 1 -- 172.21.15.19:0/3844788999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f041c15c9e0 msgr2=0x7f041c071ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.410+0000 7f0421716640 1 --2- 172.21.15.19:0/3844788999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f041c15c9e0 0x7f041c071ac0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f04040097a0 tx=0x7f040402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.410+0000 7f0421716640 1 -- 172.21.15.19:0/3844788999 shutdown_connections 2024-04-25T09:57:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.410+0000 7f0421716640 1 --2- 172.21.15.19:0/3844788999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f041c15c9e0 0x7f041c071ac0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.410+0000 7f0421716640 1 -- 172.21.15.19:0/3844788999 >> 172.21.15.19:0/3844788999 conn(0x7f041c0af670 msgr2=0x7f041c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.410+0000 7f0421716640 1 -- 172.21.15.19:0/3844788999 shutdown_connections 2024-04-25T09:57:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.410+0000 7f0421716640 1 -- 172.21.15.19:0/3844788999 wait complete. 2024-04-25T09:57:27.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.411+0000 7f0421716640 1 Processor -- start 2024-04-25T09:57:27.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.411+0000 7f0421716640 1 -- start start 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f0421716640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c155fe0 0x7f041c14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f0421716640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c15c9e0 0x7f041c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f0421716640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f041c14eef0 0x7f041c157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f0421716640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1563c0 con 0x7f041c14eef0 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f0421716640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c0ad6a0 con 0x7f041c155fe0 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f0421716640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c06f880 con 0x7f041c15c9e0 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f041b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f041c14eef0 0x7f041c157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f041b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f041c14eef0 0x7f041c157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44744/0 (socket says 172.21.15.19:44744) 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f041b7fe640 1 -- 172.21.15.19:0/154784251 learned_addr learned my addr 172.21.15.19:0/154784251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f041affd640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c155fe0 0x7f041c14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f041a7fc640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c15c9e0 0x7f041c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f041b7fe640 1 -- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c15c9e0 msgr2=0x7f041c1570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f041b7fe640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c15c9e0 0x7f041c1570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f041b7fe640 1 -- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c155fe0 msgr2=0x7f041c14e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f041b7fe640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c155fe0 0x7f041c14e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f041b7fe640 1 -- 172.21.15.19:0/154784251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c1595b0 con 0x7f041c14eef0 2024-04-25T09:57:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.412+0000 7f041a7fc640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c15c9e0 0x7f041c1570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.413+0000 7f041b7fe640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f041c14eef0 0x7f041c157620 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c730 tx=0x7f040c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.413+0000 7f04137fe640 1 -- 172.21.15.19:0/154784251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c003b70 con 0x7f041c14eef0 2024-04-25T09:57:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.413+0000 7f04137fe640 1 -- 172.21.15.19:0/154784251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c003d30 con 0x7f041c14eef0 2024-04-25T09:57:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.413+0000 7f0421716640 1 -- 172.21.15.19:0/154784251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c159890 con 0x7f041c14eef0 2024-04-25T09:57:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.413+0000 7f04137fe640 1 -- 172.21.15.19:0/154784251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c005310 con 0x7f041c14eef0 2024-04-25T09:57:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.413+0000 7f0421716640 1 -- 172.21.15.19:0/154784251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c159e50 con 0x7f041c14eef0 2024-04-25T09:57:27.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.414+0000 7f0421716640 1 -- 172.21.15.19:0/154784251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c073990 con 0x7f041c14eef0 2024-04-25T09:57:27.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.414+0000 7f04137fe640 1 -- 172.21.15.19:0/154784251 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f040c0056c0 con 0x7f041c14eef0 2024-04-25T09:57:27.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.415+0000 7f04137fe640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03fc06b4d0 0x7f03fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.415+0000 7f04137fe640 1 -- 172.21.15.19:0/154784251 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f040c08a370 con 0x7f041c14eef0 2024-04-25T09:57:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.415+0000 7f041affd640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03fc06b4d0 0x7f03fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.415+0000 7f041affd640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03fc06b4d0 0x7f03fc06d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0414007f70 tx=0x7f0414007f00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.419+0000 7f04137fe640 1 -- 172.21.15.19:0/154784251 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f040c059060 con 0x7f041c14eef0 2024-04-25T09:57:27.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.601+0000 7f0421716640 1 -- 172.21.15.19:0/154784251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f041c06bf20 con 0x7f041c14eef0 2024-04-25T09:57:27.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.602+0000 7f04137fe640 1 -- 172.21.15.19:0/154784251 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16946 (secure 0 0 0) 0x7f040c058960 con 0x7f041c14eef0 2024-04-25T09:57:27.606 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:27.606 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:57:26.327665+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:24.581855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:25.297009+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 -- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03fc06b4d0 msgr2=0x7f03fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03fc06b4d0 0x7f03fc06d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0414007f70 tx=0x7f0414007f00 comp rx=0 tx=0).stop 2024-04-25T09:57:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 -- 172.21.15.19:0/154784251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f041c14eef0 msgr2=0x7f041c157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f041c14eef0 0x7f041c157620 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c730 tx=0x7f040c00cbf0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 -- 172.21.15.19:0/154784251 shutdown_connections 2024-04-25T09:57:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03fc06b4d0 0x7f03fc06d990 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f041c14eef0 0x7f041c157620 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c15c9e0 0x7f041c1570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 --2- 172.21.15.19:0/154784251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c155fe0 0x7f041c14e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 -- 172.21.15.19:0/154784251 >> 172.21.15.19:0/154784251 conn(0x7f041c072000 msgr2=0x7f041c153f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 -- 172.21.15.19:0/154784251 shutdown_connections 2024-04-25T09:57:27.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.604+0000 7f0421716640 1 -- 172.21.15.19:0/154784251 wait complete. 2024-04-25T09:57:27.619 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:57:27.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.719+0000 7f747854c640 1 Processor -- start 2024-04-25T09:57:27.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.719+0000 7f747854c640 1 -- start start 2024-04-25T09:57:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.720+0000 7f747854c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74700cf600 0x7f74700cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.720+0000 7f747854c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f74700cbf20 0x7f74700cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.720+0000 7f747854c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74700c75e0 con 0x7f747015bb50 2024-04-25T09:57:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.720+0000 7f747854c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74700c8680 con 0x7f74700cbf20 2024-04-25T09:57:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.720+0000 7f747854c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74700cff80 con 0x7f74700cf600 2024-04-25T09:57:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.720+0000 7f74762e8640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f747015bb50 0x7f74700c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40348/0 (socket says 172.21.15.19:40348) 2024-04-25T09:57:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.720+0000 7f74762e8640 1 -- 172.21.15.19:0/2541410171 learned_addr learned my addr 172.21.15.19:0/2541410171 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f7476ae9640 1 --2- 172.21.15.19:0/2541410171 >> v2:172.21.15.112:3300/0 conn(0x7f74700cbf20 0x7f74700cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f7475ae7640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74700cf600 0x7f74700cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101906855 0 0) 0x7f74700c75e0 con 0x7f747015bb50 2024-04-25T09:57:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f745c003520 con 0x7f747015bb50 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 359338539 0 0) 0x7f745c003520 con 0x7f747015bb50 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74700cf600 msgr2=0x7f74700cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74700cf600 0x7f74700cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 >> v2:172.21.15.112:3300/0 conn(0x7f74700cbf20 msgr2=0x7f74700cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 --2- 172.21.15.19:0/2541410171 >> v2:172.21.15.112:3300/0 conn(0x7f74700cbf20 0x7f74700cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74700c78c0 con 0x7f747015bb50 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f746c001670 con 0x7f747015bb50 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 >> v1:172.21.15.19:6789/0 conn(0x7f747015bb50 legacy=0x7f74700c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f745c0039b0 0x7f745c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74700cf600 0x7f745c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c004d40 0x7f745c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74700c8680 con 0x7f745c0039b0 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74700cff80 con 0x7f745c004d40 2024-04-25T09:57:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c00b9c0 con 0x7f74700cf600 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f74762e8640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f745c0039b0 0x7f745c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f7475ae7640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74700cf600 0x7f745c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.721+0000 7f7476ae9640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c004d40 0x7f745c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.722+0000 7f74762e8640 1 -- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74700cf600 msgr2=0x7f745c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.722+0000 7f74762e8640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74700cf600 0x7f745c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.722+0000 7f74762e8640 1 -- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c004d40 msgr2=0x7f745c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.722+0000 7f74762e8640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c004d40 0x7f745c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.722+0000 7f74762e8640 1 -- 172.21.15.19:0/2541410171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c003520 con 0x7f745c0039b0 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.722+0000 7f7476ae9640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c004d40 0x7f745c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.722+0000 7f74762e8640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f745c0039b0 0x7f745c003dd0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f746c001050 tx=0x7f745c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a184f07fb8353a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.722+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c02e070 con 0x7f745c0039b0 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.722+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c005af0 con 0x7f745c0039b0 2024-04-25T09:57:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.722+0000 7f74752e6640 1 -- 172.21.15.19:0/2541410171 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c007d70 con 0x7f745c0039b0 2024-04-25T09:57:27.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.723+0000 7f747854c640 1 -- 172.21.15.19:0/2541410171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f745c0039b0 msgr2=0x7f745c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.723+0000 7f747854c640 1 --2- 172.21.15.19:0/2541410171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f745c0039b0 0x7f745c003dd0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f746c001050 tx=0x7f745c005830 comp rx=0 tx=0).stop 2024-04-25T09:57:27.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.723+0000 7f7476ae9640 1 -- 172.21.15.19:0/2541410171 reap_dead start 2024-04-25T09:57:27.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.723+0000 7f747854c640 1 -- 172.21.15.19:0/2541410171 shutdown_connections 2024-04-25T09:57:27.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.723+0000 7f747854c640 1 -- 172.21.15.19:0/2541410171 >> 172.21.15.19:0/2541410171 conn(0x7f747006f500 msgr2=0x7f747006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.723+0000 7f747854c640 1 -- 172.21.15.19:0/2541410171 shutdown_connections 2024-04-25T09:57:27.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.723+0000 7f747854c640 1 -- 172.21.15.19:0/2541410171 wait complete. 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.724+0000 7f747854c640 1 Processor -- start 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.724+0000 7f747854c640 1 -- start start 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f747854c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0039b0 0x7f7470155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f747854c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f747015b3a0 0x7f74700b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f747854c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74700b6d60 0x7f74700b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f747854c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74700c8160 con 0x7f74700b6d60 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f747854c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74700c7e80 con 0x7f745c0039b0 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f747854c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74700c72a0 con 0x7f747015b3a0 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f7476ae9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74700b6d60 0x7f74700b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f7476ae9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74700b6d60 0x7f74700b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53586/0 (socket says 172.21.15.19:53586) 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f7476ae9640 1 -- 172.21.15.19:0/3242088240 learned_addr learned my addr 172.21.15.19:0/3242088240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f74762e8640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0039b0 0x7f7470155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f7475ae7640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f747015b3a0 0x7f74700b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f7476ae9640 1 -- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f747015b3a0 msgr2=0x7f74700b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f7476ae9640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f747015b3a0 0x7f74700b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f7476ae9640 1 -- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0039b0 msgr2=0x7f7470155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f7476ae9640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0039b0 0x7f7470155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f7476ae9640 1 -- 172.21.15.19:0/3242088240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74700b7220 con 0x7f74700b6d60 2024-04-25T09:57:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.725+0000 7f74762e8640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0039b0 0x7f7470155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.726+0000 7f7476ae9640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74700b6d60 0x7f74700b47f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f747006f270 tx=0x7f7468010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.726+0000 7f7466ffd640 1 -- 172.21.15.19:0/3242088240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746801a030 con 0x7f74700b6d60 2024-04-25T09:57:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.726+0000 7f7466ffd640 1 -- 172.21.15.19:0/3242088240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7468010d80 con 0x7f74700b6d60 2024-04-25T09:57:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.726+0000 7f7466ffd640 1 -- 172.21.15.19:0/3242088240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468003bd0 con 0x7f74700b6d60 2024-04-25T09:57:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.726+0000 7f747854c640 1 -- 172.21.15.19:0/3242088240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74700b7500 con 0x7f74700b6d60 2024-04-25T09:57:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.726+0000 7f747854c640 1 -- 172.21.15.19:0/3242088240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74700c9ad0 con 0x7f74700b6d60 2024-04-25T09:57:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.727+0000 7f747854c640 1 -- 172.21.15.19:0/3242088240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7470073a00 con 0x7f74700b6d60 2024-04-25T09:57:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.727+0000 7f7466ffd640 1 -- 172.21.15.19:0/3242088240 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f746801d070 con 0x7f74700b6d60 2024-04-25T09:57:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.728+0000 7f7466ffd640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745406b500 0x7f745406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.728+0000 7f7466ffd640 1 -- 172.21.15.19:0/3242088240 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f746808a8a0 con 0x7f74700b6d60 2024-04-25T09:57:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.728+0000 7f74762e8640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745406b500 0x7f745406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.729+0000 7f74762e8640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745406b500 0x7f745406d9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f746c033000 tx=0x7f746c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.732+0000 7f7466ffd640 1 -- 172.21.15.19:0/3242088240 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7468059620 con 0x7f74700b6d60 2024-04-25T09:57:27.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:27.951+0000 7f747854c640 1 -- 172.21.15.19:0/3242088240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f74700c9ca0 con 0x7f74700b6d60 2024-04-25T09:57:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.564+0000 7f7466ffd640 1 -- 172.21.15.19:0/3242088240 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7468058f20 con 0x7f74700b6d60 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 -- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745406b500 msgr2=0x7f745406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745406b500 0x7f745406d9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f746c033000 tx=0x7f746c009030 comp rx=0 tx=0).stop 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 -- 172.21.15.19:0/3242088240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74700b6d60 msgr2=0x7f74700b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74700b6d60 0x7f74700b47f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f747006f270 tx=0x7f7468010590 comp rx=0 tx=0).stop 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 -- 172.21.15.19:0/3242088240 shutdown_connections 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745406b500 0x7f745406d9c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74700b6d60 0x7f74700b47f0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f747015b3a0 0x7f74700b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 --2- 172.21.15.19:0/3242088240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0039b0 0x7f7470155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 -- 172.21.15.19:0/3242088240 >> 172.21.15.19:0/3242088240 conn(0x7f747006f500 msgr2=0x7f74700b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 -- 172.21.15.19:0/3242088240 shutdown_connections 2024-04-25T09:57:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.567+0000 7f747854c640 1 -- 172.21.15.19:0/3242088240 wait complete. 2024-04-25T09:57:28.584 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:57:28.584 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:57:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.691+0000 7f5ecd5a5640 1 Processor -- start 2024-04-25T09:57:28.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.692+0000 7f5ecd5a5640 1 -- start start 2024-04-25T09:57:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.692+0000 7f5ecd5a5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ec80c6010 0x7f5ec80ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.692+0000 7f5ecd5a5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec80c6940 con 0x7f5ec815a9f0 2024-04-25T09:57:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.692+0000 7f5ecd5a5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec815add0 con 0x7f5ec80cd520 2024-04-25T09:57:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.692+0000 7f5ecd5a5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec80b0020 con 0x7f5ec80c6010 2024-04-25T09:57:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.693+0000 7f5ec67fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ec80c6010 0x7f5ec80ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.693+0000 7f5ec6ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5ec815a9f0 0x7f5ec80cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40364/0 (socket says 172.21.15.19:40364) 2024-04-25T09:57:28.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.693+0000 7f5ec6ffd640 1 -- 172.21.15.19:0/114386347 learned_addr learned my addr 172.21.15.19:0/114386347 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:28.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.693+0000 7f5ec67fc640 1 -- 172.21.15.19:0/114386347 >> v1:172.21.15.112:6789/0 conn(0x7f5ec80cd520 legacy=0x7f5ec80cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.693+0000 7f5ec67fc640 1 -- 172.21.15.19:0/114386347 >> v1:172.21.15.19:6789/0 conn(0x7f5ec815a9f0 legacy=0x7f5ec80cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.693+0000 7f5ec67fc640 1 -- 172.21.15.19:0/114386347 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80c6660 con 0x7f5ec80c6010 2024-04-25T09:57:28.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.694+0000 7f5ec67fc640 1 --2- 172.21.15.19:0/114386347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec80c6010 0x7f5ec80ccfe0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00097a0 tx=0x7f5eb002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8d649bfb5d44cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.694+0000 7f5ec5ffb640 1 -- 172.21.15.19:0/114386347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0037070 con 0x7f5ec80c6010 2024-04-25T09:57:28.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.694+0000 7f5ec5ffb640 1 -- 172.21.15.19:0/114386347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb0032440 con 0x7f5ec80c6010 2024-04-25T09:57:28.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.694+0000 7f5ec5ffb640 1 -- 172.21.15.19:0/114386347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0032810 con 0x7f5ec80c6010 2024-04-25T09:57:28.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.694+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/114386347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec80c6010 msgr2=0x7f5ec80ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.694+0000 7f5ecd5a5640 1 --2- 172.21.15.19:0/114386347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec80c6010 0x7f5ec80ccfe0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00097a0 tx=0x7f5eb002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.694+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/114386347 shutdown_connections 2024-04-25T09:57:28.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.694+0000 7f5ecd5a5640 1 --2- 172.21.15.19:0/114386347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec80c6010 0x7f5ec80ccfe0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.694+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/114386347 >> 172.21.15.19:0/114386347 conn(0x7f5ec80b2390 msgr2=0x7f5ec80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.694+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/114386347 shutdown_connections 2024-04-25T09:57:28.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.694+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/114386347 wait complete. 2024-04-25T09:57:28.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.696+0000 7f5ecd5a5640 1 Processor -- start 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.696+0000 7f5ecd5a5640 1 -- start start 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.696+0000 7f5ecd5a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ec80c6010 0x7f5ec81490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.696+0000 7f5ecd5a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec80cd520 0x7f5ec806ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.696+0000 7f5ecd5a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec815a9f0 0x7f5ec806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.696+0000 7f5ecd5a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80cb230 con 0x7f5ec80cd520 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.696+0000 7f5ecd5a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80cb5a0 con 0x7f5ec815a9f0 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.696+0000 7f5ecd5a5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80cc140 con 0x7f5ec80c6010 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ec67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec80cd520 0x7f5ec806ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ec67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec80cd520 0x7f5ec806ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53614/0 (socket says 172.21.15.19:53614) 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ec67fc640 1 -- 172.21.15.19:0/2519784 learned_addr learned my addr 172.21.15.19:0/2519784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ec77fe640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec815a9f0 0x7f5ec806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ec6ffd640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ec80c6010 0x7f5ec81490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ec67fc640 1 -- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ec80c6010 msgr2=0x7f5ec81490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ec67fc640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ec80c6010 0x7f5ec81490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ec67fc640 1 -- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec815a9f0 msgr2=0x7f5ec806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ec67fc640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec815a9f0 0x7f5ec806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ec67fc640 1 -- 172.21.15.19:0/2519784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec814f060 con 0x7f5ec80cd520 2024-04-25T09:57:28.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ec67fc640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec80cd520 0x7f5ec806ac70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0000c00 tx=0x7f5eb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ebf7fe640 1 -- 172.21.15.19:0/2519784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0037040 con 0x7f5ec80cd520 2024-04-25T09:57:28.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ebf7fe640 1 -- 172.21.15.19:0/2519784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb0042070 con 0x7f5ec80cd520 2024-04-25T09:57:28.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/2519784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec814f2d0 con 0x7f5ec80cd520 2024-04-25T09:57:28.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ebf7fe640 1 -- 172.21.15.19:0/2519784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0030a70 con 0x7f5ec80cd520 2024-04-25T09:57:28.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.697+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/2519784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec814f890 con 0x7f5ec80cd520 2024-04-25T09:57:28.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.698+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/2519784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec806cc80 con 0x7f5ec80cd520 2024-04-25T09:57:28.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.699+0000 7f5ebf7fe640 1 -- 172.21.15.19:0/2519784 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5eb0030c10 con 0x7f5ec80cd520 2024-04-25T09:57:28.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.699+0000 7f5ebf7fe640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ea806b500 0x7f5ea806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.700+0000 7f5ebf7fe640 1 -- 172.21.15.19:0/2519784 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5eb00aadc0 con 0x7f5ec80cd520 2024-04-25T09:57:28.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.700+0000 7f5ec6ffd640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ea806b500 0x7f5ea806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.700+0000 7f5ec6ffd640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ea806b500 0x7f5ea806d9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00012e0 tx=0x7f5ec0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.704+0000 7f5ebf7fe640 1 -- 172.21.15.19:0/2519784 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5eb0045360 con 0x7f5ec80cd520 2024-04-25T09:57:28.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:28.921+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/2519784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5ec8073a00 con 0x7f5ec80cd520 2024-04-25T09:57:28.930 INFO:tasks.ceph.mds.b.smithi112.stderr:2024-04-25T09:57:28.927+0000 7f6232203640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:57:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.569+0000 7f5ebf7fe640 1 -- 172.21.15.19:0/2519784 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v51) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5eb0079ab0 con 0x7f5ec80cd520 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ea806b500 msgr2=0x7f5ea806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ea806b500 0x7f5ea806d9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00012e0 tx=0x7f5ec0009040 comp rx=0 tx=0).stop 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/2519784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec80cd520 msgr2=0x7f5ec806ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec80cd520 0x7f5ec806ac70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0000c00 tx=0x7f5eb00046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/2519784 shutdown_connections 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ea806b500 0x7f5ea806d9c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ec815a9f0 0x7f5ec806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec80cd520 0x7f5ec806ac70 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 --2- 172.21.15.19:0/2519784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ec80c6010 0x7f5ec81490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/2519784 >> 172.21.15.19:0/2519784 conn(0x7f5ec80b2390 msgr2=0x7f5ec80cc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/2519784 shutdown_connections 2024-04-25T09:57:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.573+0000 7f5ecd5a5640 1 -- 172.21.15.19:0/2519784 wait complete. 2024-04-25T09:57:29.591 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:57:29.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.698+0000 7f6044a88640 1 Processor -- start 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.698+0000 7f6044a88640 1 -- start start 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f6044a88640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f604015db70 0x7f6040162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f6044a88640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60400b5c30 0x7f60400d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f6044a88640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6040162690 con 0x7f60400b6cb0 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f6044a88640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60400ceee0 con 0x7f604015db70 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f6044a88640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60400d1d10 con 0x7f60400b5c30 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f603e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f604015db70 0x7f6040162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f603dd9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f60400b6cb0 0x7f60400b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40368/0 (socket says 172.21.15.19:40368) 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f603dd9b640 1 -- 172.21.15.19:0/53031371 learned_addr learned my addr 172.21.15.19:0/53031371 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f603ed9d640 1 --2- 172.21.15.19:0/53031371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60400b5c30 0x7f60400d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f603e59c640 1 -- 172.21.15.19:0/53031371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60400b5c30 msgr2=0x7f60400d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f603e59c640 1 --2- 172.21.15.19:0/53031371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60400b5c30 0x7f60400d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.700+0000 7f603e59c640 1 -- 172.21.15.19:0/53031371 >> v1:172.21.15.19:6789/0 conn(0x7f60400b6cb0 legacy=0x7f60400b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.699+0000 7f603d59a640 1 -- 172.21.15.19:0/53031371 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75775578 0 0) 0x7f6040162690 con 0x7f60400b6cb0 2024-04-25T09:57:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.700+0000 7f603e59c640 1 -- 172.21.15.19:0/53031371 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400d1ff0 con 0x7f604015db70 2024-04-25T09:57:29.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.700+0000 7f603ed9d640 1 --2- 172.21.15.19:0/53031371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60400b5c30 0x7f60400d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:29.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.700+0000 7f603e59c640 1 --2- 172.21.15.19:0/53031371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604015db70 0x7f6040162150 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f60300097a0 tx=0x7f603002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a754c275be050f4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.700+0000 7f603d59a640 1 -- 172.21.15.19:0/53031371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030037070 con 0x7f604015db70 2024-04-25T09:57:29.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.700+0000 7f603d59a640 1 -- 172.21.15.19:0/53031371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6030032440 con 0x7f604015db70 2024-04-25T09:57:29.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.700+0000 7f603d59a640 1 -- 172.21.15.19:0/53031371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030032810 con 0x7f604015db70 2024-04-25T09:57:29.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.701+0000 7f6044a88640 1 -- 172.21.15.19:0/53031371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604015db70 msgr2=0x7f6040162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.701+0000 7f6044a88640 1 --2- 172.21.15.19:0/53031371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604015db70 0x7f6040162150 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f60300097a0 tx=0x7f603002a180 comp rx=0 tx=0).stop 2024-04-25T09:57:29.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.701+0000 7f6044a88640 1 -- 172.21.15.19:0/53031371 shutdown_connections 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.701+0000 7f6044a88640 1 --2- 172.21.15.19:0/53031371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60400b5c30 0x7f60400d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.701+0000 7f6044a88640 1 --2- 172.21.15.19:0/53031371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604015db70 0x7f6040162150 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.701+0000 7f6044a88640 1 -- 172.21.15.19:0/53031371 >> 172.21.15.19:0/53031371 conn(0x7f60400d0520 msgr2=0x7f60400d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.701+0000 7f6044a88640 1 -- 172.21.15.19:0/53031371 shutdown_connections 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.701+0000 7f6044a88640 1 -- 172.21.15.19:0/53031371 wait complete. 2024-04-25T09:57:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.702+0000 7f6044a88640 1 Processor -- start 2024-04-25T09:57:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.702+0000 7f6044a88640 1 -- start start 2024-04-25T09:57:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.703+0000 7f6044a88640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60400b5c30 0x7f60401f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.703+0000 7f6044a88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60400b6cb0 0x7f60401f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.703+0000 7f6044a88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604015db70 0x7f60401f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.703+0000 7f6044a88640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60400ce530 con 0x7f604015db70 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.703+0000 7f6044a88640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604015df50 con 0x7f60400b6cb0 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.703+0000 7f6044a88640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60400cf1c0 con 0x7f60400b5c30 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.703+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604015db70 0x7f60401f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.703+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604015db70 0x7f60401f6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53632/0 (socket says 172.21.15.19:53632) 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.703+0000 7f603ed9d640 1 -- 172.21.15.19:0/891837129 learned_addr learned my addr 172.21.15.19:0/891837129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.703+0000 7f603e59c640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60400b5c30 0x7f60401f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.703+0000 7f603dd9b640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60400b6cb0 0x7f60401f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.704+0000 7f603ed9d640 1 -- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60400b5c30 msgr2=0x7f60401f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.704+0000 7f603ed9d640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60400b5c30 0x7f60401f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.704+0000 7f603ed9d640 1 -- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60400b6cb0 msgr2=0x7f60401f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.704+0000 7f603ed9d640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60400b6cb0 0x7f60401f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.704+0000 7f603ed9d640 1 -- 172.21.15.19:0/891837129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6040152970 con 0x7f604015db70 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.704+0000 7f603ed9d640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604015db70 0x7f60401f6da0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f603800cd10 tx=0x7f603800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.704+0000 7f602effd640 1 -- 172.21.15.19:0/891837129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038003b90 con 0x7f604015db70 2024-04-25T09:57:29.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.704+0000 7f602effd640 1 -- 172.21.15.19:0/891837129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6038003d50 con 0x7f604015db70 2024-04-25T09:57:29.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.704+0000 7f6044a88640 1 -- 172.21.15.19:0/891837129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6040152cb0 con 0x7f604015db70 2024-04-25T09:57:29.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.704+0000 7f602effd640 1 -- 172.21.15.19:0/891837129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038004260 con 0x7f604015db70 2024-04-25T09:57:29.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.704+0000 7f6044a88640 1 -- 172.21.15.19:0/891837129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60401534f0 con 0x7f604015db70 2024-04-25T09:57:29.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.705+0000 7f6044a88640 1 -- 172.21.15.19:0/891837129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6040073990 con 0x7f604015db70 2024-04-25T09:57:29.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.706+0000 7f602effd640 1 -- 172.21.15.19:0/891837129 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6038004400 con 0x7f604015db70 2024-04-25T09:57:29.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.706+0000 7f602effd640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f602006b4d0 0x7f602006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.706+0000 7f602effd640 1 -- 172.21.15.19:0/891837129 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f603808ad40 con 0x7f604015db70 2024-04-25T09:57:29.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.706+0000 7f603e59c640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f602006b4d0 0x7f602006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.707+0000 7f603e59c640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f602006b4d0 0x7f602006d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6030034000 tx=0x7f603002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.710+0000 7f602effd640 1 -- 172.21.15.19:0/891837129 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6040152cb0 con 0x7f604015db70 2024-04-25T09:57:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.932+0000 7f6044a88640 1 -- 172.21.15.19:0/891837129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f604015d390 con 0x7f604015db70 2024-04-25T09:57:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.932+0000 7f602effd640 1 -- 172.21.15.19:0/891837129 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6038014030 con 0x7f604015db70 2024-04-25T09:57:29.936 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:29.936 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2023643575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2023643575},{"type":"v1","addr":"172.21.15.19:6835","nonce":2023643575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3683680157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3683680157},{"type":"v1","addr":"172.21.15.112:6836","nonce":3683680157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/309620056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":309620056},{"type":"v1","addr":"172.21.15.19:6833","nonce":309620056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:26.327989+0000","modified":"2024-04-25T09:57:28.922996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7137},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7137":{"gid":7137,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1747929190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1747929190},{"type":"v1","addr":"172.21.15.112:6837","nonce":1747929190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:57:29.936 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 51 2024-04-25T09:57:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 -- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f602006b4d0 msgr2=0x7f602006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f602006b4d0 0x7f602006d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6030034000 tx=0x7f603002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 -- 172.21.15.19:0/891837129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604015db70 msgr2=0x7f60401f6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604015db70 0x7f60401f6da0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f603800cd10 tx=0x7f603800a650 comp rx=0 tx=0).stop 2024-04-25T09:57:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 -- 172.21.15.19:0/891837129 shutdown_connections 2024-04-25T09:57:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f602006b4d0 0x7f602006d990 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604015db70 0x7f60401f6da0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60400b6cb0 0x7f60401f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 --2- 172.21.15.19:0/891837129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60400b5c30 0x7f60401f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 -- 172.21.15.19:0/891837129 >> 172.21.15.19:0/891837129 conn(0x7f60400d0520 msgr2=0x7f604015a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:29.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 -- 172.21.15.19:0/891837129 shutdown_connections 2024-04-25T09:57:29.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:29.934+0000 7f6044a88640 1 -- 172.21.15.19:0/891837129 wait complete. 2024-04-25T09:57:29.949 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.046+0000 7f87eeaf5640 1 Processor -- start 2024-04-25T09:57:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.046+0000 7f87eeaf5640 1 -- start start 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87eeaf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e815ef00 0x7f87e80d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87eeaf5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e80d20e0 con 0x7f87e80d1ca0 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87eeaf5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e80c7970 con 0x7f87e80c7310 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87eeaf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80c7730 con 0x7f87e815ef00 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87dffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f87e80d1ca0 0x7f87e80c6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40376/0 (socket says 172.21.15.19:40376) 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87dffff640 1 -- 172.21.15.19:0/2321000988 learned_addr learned my addr 172.21.15.19:0/2321000988 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420528664 0 0) 0x7f87e80d20e0 con 0x7f87e80d1ca0 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87ec891640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e815ef00 0x7f87e80d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87d0003520 con 0x7f87e80d1ca0 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1546391078 0 0) 0x7f87d0003520 con 0x7f87e80d1ca0 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e815ef00 msgr2=0x7f87e80d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87df7fe640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e815ef00 0x7f87e80d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 >> v1:172.21.15.112:6789/0 conn(0x7f87e80c7310 legacy=0x7f87e80cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.047+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e80c7c50 con 0x7f87e80d1ca0 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f87d8001670 con 0x7f87e80d1ca0 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 >> v1:172.21.15.19:6789/0 conn(0x7f87e80d1ca0 legacy=0x7f87e80c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87df7fe640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 0x7f87d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87df7fe640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e815ef00 0x7f87d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87df7fe640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d0004d40 0x7f87d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80c7730 con 0x7f87d00039e0 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d000b9c0 con 0x7f87d0004d40 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d000bbc0 con 0x7f87e815ef00 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87ec891640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 0x7f87d0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87dffff640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e815ef00 0x7f87d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87ed092640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d0004d40 0x7f87d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87ec891640 1 -- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e815ef00 msgr2=0x7f87d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87ec891640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e815ef00 0x7f87d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87ec891640 1 -- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d0004d40 msgr2=0x7f87d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87ec891640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d0004d40 0x7f87d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87ec891640 1 -- 172.21.15.19:0/2321000988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d0003520 con 0x7f87d00039e0 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.048+0000 7f87ed092640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d0004d40 0x7f87d00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.049+0000 7f87ec891640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 0x7f87d0003dc0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009ac0 tx=0x7f87d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0163aaf86782b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.049+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4036070 con 0x7f87d00039e0 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.049+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e4031400 con 0x7f87d00039e0 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.049+0000 7f87df7fe640 1 -- 172.21.15.19:0/2321000988 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4009690 con 0x7f87d00039e0 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.049+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2321000988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 msgr2=0x7f87d0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.049+0000 7f87eeaf5640 1 --2- 172.21.15.19:0/2321000988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 0x7f87d0003dc0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009ac0 tx=0x7f87d0005830 comp rx=0 tx=0).stop 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.049+0000 7f87ed092640 1 -- 172.21.15.19:0/2321000988 reap_dead start 2024-04-25T09:57:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.049+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2321000988 shutdown_connections 2024-04-25T09:57:30.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.049+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2321000988 >> 172.21.15.19:0/2321000988 conn(0x7f87e806f5d0 msgr2=0x7f87e806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:30.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.049+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2321000988 shutdown_connections 2024-04-25T09:57:30.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.049+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2321000988 wait complete. 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.051+0000 7f87eeaf5640 1 Processor -- start 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.051+0000 7f87eeaf5640 1 -- start start 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.051+0000 7f87eeaf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 0x7f87e8153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.051+0000 7f87eeaf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e81f5600 0x7f87e8150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.051+0000 7f87eeaf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8151480 0x7f87e80b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87eeaf5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80c8210 con 0x7f87d00039e0 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87eeaf5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e815f2e0 con 0x7f87e8151480 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87eeaf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80c84f0 con 0x7f87e81f5600 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87ec891640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 0x7f87e8153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87ec891640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 0x7f87e8153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53658/0 (socket says 172.21.15.19:53658) 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87ec891640 1 -- 172.21.15.19:0/2808667819 learned_addr learned my addr 172.21.15.19:0/2808667819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87dffff640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e81f5600 0x7f87e8150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87ed092640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8151480 0x7f87e80b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87ec891640 1 -- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e81f5600 msgr2=0x7f87e8150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87ec891640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e81f5600 0x7f87e8150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87ec891640 1 -- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8151480 msgr2=0x7f87e80b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87ec891640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8151480 0x7f87e80b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87ec891640 1 -- 172.21.15.19:0/2808667819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e8153db0 con 0x7f87d00039e0 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87dffff640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e81f5600 0x7f87e8150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87ed092640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8151480 0x7f87e80b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87ec891640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 0x7f87e8153500 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f87e4034040 tx=0x7f87e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87dd7fa640 1 -- 172.21.15.19:0/2808667819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4040020 con 0x7f87d00039e0 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87dd7fa640 1 -- 172.21.15.19:0/2808667819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e4003bc0 con 0x7f87d00039e0 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2808667819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e8154090 con 0x7f87d00039e0 2024-04-25T09:57:30.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87dd7fa640 1 -- 172.21.15.19:0/2808667819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4036040 con 0x7f87d00039e0 2024-04-25T09:57:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.052+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2808667819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e8153a40 con 0x7f87d00039e0 2024-04-25T09:57:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.053+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2808667819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e8073990 con 0x7f87d00039e0 2024-04-25T09:57:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.054+0000 7f87dd7fa640 1 -- 172.21.15.19:0/2808667819 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87e4043020 con 0x7f87d00039e0 2024-04-25T09:57:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.054+0000 7f87dd7fa640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f87d406b4d0 0x7f87d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.054+0000 7f87dd7fa640 1 -- 172.21.15.19:0/2808667819 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87e40aa4b0 con 0x7f87d00039e0 2024-04-25T09:57:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.055+0000 7f87dffff640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f87d406b4d0 0x7f87d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.055+0000 7f87dffff640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f87d406b4d0 0x7f87d406d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f87e8159c00 tx=0x7f87d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.059+0000 7f87dd7fa640 1 -- 172.21.15.19:0/2808667819 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87e404a070 con 0x7f87d00039e0 2024-04-25T09:57:30.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.240+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2808667819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87e80b3d20 con 0x7f87d00039e0 2024-04-25T09:57:30.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.241+0000 7f87dd7fa640 1 -- 172.21.15.19:0/2808667819 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f87e40044e0 con 0x7f87d00039e0 2024-04-25T09:57:30.245 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:30.245 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:57:27.970541+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:24.581855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:25.297009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:30.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f87d406b4d0 msgr2=0x7f87d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f87d406b4d0 0x7f87d406d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f87e8159c00 tx=0x7f87d8009040 comp rx=0 tx=0).stop 2024-04-25T09:57:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2808667819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 msgr2=0x7f87e8153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 0x7f87e8153500 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f87e4034040 tx=0x7f87e4004730 comp rx=0 tx=0).stop 2024-04-25T09:57:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2808667819 shutdown_connections 2024-04-25T09:57:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f87d406b4d0 0x7f87d406d990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8151480 0x7f87e80b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87e81f5600 0x7f87e8150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 --2- 172.21.15.19:0/2808667819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d00039e0 0x7f87e8153500 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2808667819 >> 172.21.15.19:0/2808667819 conn(0x7f87e806f5d0 msgr2=0x7f87e80cade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2808667819 shutdown_connections 2024-04-25T09:57:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.243+0000 7f87eeaf5640 1 -- 172.21.15.19:0/2808667819 wait complete. 2024-04-25T09:57:30.259 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:57:30.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:57:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.356+0000 7f6b37329640 1 Processor -- start 2024-04-25T09:57:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.356+0000 7f6b37329640 1 -- start start 2024-04-25T09:57:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b37329640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b3015dd30 0x7f6b300b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b37329640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6b300d0ad0 0x7f6b300b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b37329640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b300d1df0 con 0x7f6b300b9770 2024-04-25T09:57:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b37329640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b300d20d0 con 0x7f6b3015dd30 2024-04-25T09:57:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b37329640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b300d23b0 con 0x7f6b300d0ad0 2024-04-25T09:57:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b350c5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b3015dd30 0x7f6b300b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b348c4640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6b300b9770 0x7f6b300d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40380/0 (socket says 172.21.15.19:40380) 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b348c4640 1 -- 172.21.15.19:0/681954501 learned_addr learned my addr 172.21.15.19:0/681954501 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b358c6640 1 --2- 172.21.15.19:0/681954501 >> v2:172.21.15.112:3300/0 conn(0x7f6b300d0ad0 0x7f6b300b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b350c5640 1 -- 172.21.15.19:0/681954501 >> v2:172.21.15.112:3300/0 conn(0x7f6b300d0ad0 msgr2=0x7f6b300b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b350c5640 1 --2- 172.21.15.19:0/681954501 >> v2:172.21.15.112:3300/0 conn(0x7f6b300d0ad0 0x7f6b300b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b350c5640 1 -- 172.21.15.19:0/681954501 >> v1:172.21.15.19:6789/0 conn(0x7f6b300b9770 legacy=0x7f6b300d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b350c5640 1 -- 172.21.15.19:0/681954501 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b300d2690 con 0x7f6b3015dd30 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b27fff640 1 -- 172.21.15.19:0/681954501 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346858042 0 0) 0x7f6b300d1df0 con 0x7f6b300b9770 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.357+0000 7f6b358c6640 1 --2- 172.21.15.19:0/681954501 >> v2:172.21.15.112:3300/0 conn(0x7f6b300d0ad0 0x7f6b300b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.358+0000 7f6b350c5640 1 --2- 172.21.15.19:0/681954501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3015dd30 0x7f6b300b9230 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6b300ce200 tx=0x7f6b2c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b09e777c2516934 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.358+0000 7f6b27fff640 1 -- 172.21.15.19:0/681954501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c037070 con 0x7f6b3015dd30 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.358+0000 7f6b27fff640 1 -- 172.21.15.19:0/681954501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c02add0 con 0x7f6b3015dd30 2024-04-25T09:57:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.358+0000 7f6b27fff640 1 -- 172.21.15.19:0/681954501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c032660 con 0x7f6b3015dd30 2024-04-25T09:57:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.359+0000 7f6b37329640 1 -- 172.21.15.19:0/681954501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3015dd30 msgr2=0x7f6b300b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.359+0000 7f6b37329640 1 --2- 172.21.15.19:0/681954501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3015dd30 0x7f6b300b9230 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6b300ce200 tx=0x7f6b2c02a080 comp rx=0 tx=0).stop 2024-04-25T09:57:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.359+0000 7f6b37329640 1 -- 172.21.15.19:0/681954501 shutdown_connections 2024-04-25T09:57:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.359+0000 7f6b37329640 1 --2- 172.21.15.19:0/681954501 >> v2:172.21.15.112:3300/0 conn(0x7f6b300d0ad0 0x7f6b300b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.359+0000 7f6b37329640 1 --2- 172.21.15.19:0/681954501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3015dd30 0x7f6b300b9230 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.359+0000 7f6b37329640 1 -- 172.21.15.19:0/681954501 >> 172.21.15.19:0/681954501 conn(0x7f6b30061c30 msgr2=0x7f6b3006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.359+0000 7f6b37329640 1 -- 172.21.15.19:0/681954501 shutdown_connections 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.359+0000 7f6b37329640 1 -- 172.21.15.19:0/681954501 wait complete. 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.360+0000 7f6b37329640 1 Processor -- start 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.360+0000 7f6b37329640 1 -- start start 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b37329640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b300b9770 0x7f6b301f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b37329640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b300d0ad0 0x7f6b30159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b37329640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b3015dd30 0x7f6b30159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b37329640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b300d1b90 con 0x7f6b300b9770 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b37329640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b300d1a10 con 0x7f6b3015dd30 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b37329640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b300d2e90 con 0x7f6b300d0ad0 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b350c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b300b9770 0x7f6b301f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b350c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b300b9770 0x7f6b301f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53682/0 (socket says 172.21.15.19:53682) 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b350c5640 1 -- 172.21.15.19:0/1015824023 learned_addr learned my addr 172.21.15.19:0/1015824023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b358c6640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b3015dd30 0x7f6b30159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b348c4640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b300d0ad0 0x7f6b30159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b350c5640 1 -- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b300d0ad0 msgr2=0x7f6b30159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b350c5640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b300d0ad0 0x7f6b30159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b350c5640 1 -- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b3015dd30 msgr2=0x7f6b30159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b350c5640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b3015dd30 0x7f6b30159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b350c5640 1 -- 172.21.15.19:0/1015824023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b300b3140 con 0x7f6b300b9770 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b350c5640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b300b9770 0x7f6b301f7490 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c000c00 tx=0x7f6b2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.361+0000 7f6b25ffb640 1 -- 172.21.15.19:0/1015824023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c037050 con 0x7f6b300b9770 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.362+0000 7f6b25ffb640 1 -- 172.21.15.19:0/1015824023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c038040 con 0x7f6b300b9770 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.362+0000 7f6b37329640 1 -- 172.21.15.19:0/1015824023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b300b3420 con 0x7f6b300b9770 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.362+0000 7f6b25ffb640 1 -- 172.21.15.19:0/1015824023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c004270 con 0x7f6b300b9770 2024-04-25T09:57:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.362+0000 7f6b37329640 1 -- 172.21.15.19:0/1015824023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b301595a0 con 0x7f6b300b9770 2024-04-25T09:57:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.363+0000 7f6b37329640 1 -- 172.21.15.19:0/1015824023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b30073990 con 0x7f6b300b9770 2024-04-25T09:57:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.363+0000 7f6b25ffb640 1 -- 172.21.15.19:0/1015824023 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b2c003610 con 0x7f6b300b9770 2024-04-25T09:57:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.364+0000 7f6b25ffb640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1406b4d0 0x7f6b1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.364+0000 7f6b25ffb640 1 -- 172.21.15.19:0/1015824023 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b2c0aab30 con 0x7f6b300b9770 2024-04-25T09:57:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.364+0000 7f6b348c4640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1406b4d0 0x7f6b1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.364+0000 7f6b348c4640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1406b4d0 0x7f6b1406d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6b300b7a30 tx=0x7f6b20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.368+0000 7f6b25ffb640 1 -- 172.21.15.19:0/1015824023 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b2c079940 con 0x7f6b300b9770 2024-04-25T09:57:30.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.546+0000 7f6b37329640 1 -- 172.21.15.19:0/1015824023 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b300b35f0 con 0x7f6b1406b4d0 2024-04-25T09:57:30.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.553+0000 7f6b25ffb640 1 -- 172.21.15.19:0/1015824023 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252064 (secure 0 0 0) 0x7f6b300b35f0 con 0x7f6b1406b4d0 2024-04-25T09:57:30.560 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:30.561 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:57:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 -- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1406b4d0 msgr2=0x7f6b1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1406b4d0 0x7f6b1406d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6b300b7a30 tx=0x7f6b20009040 comp rx=0 tx=0).stop 2024-04-25T09:57:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 -- 172.21.15.19:0/1015824023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b300b9770 msgr2=0x7f6b301f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b300b9770 0x7f6b301f7490 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c000c00 tx=0x7f6b2c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 -- 172.21.15.19:0/1015824023 shutdown_connections 2024-04-25T09:57:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1406b4d0 0x7f6b1406d990 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b3015dd30 0x7f6b30159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b300d0ad0 0x7f6b30159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 --2- 172.21.15.19:0/1015824023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b300b9770 0x7f6b301f7490 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 -- 172.21.15.19:0/1015824023 >> 172.21.15.19:0/1015824023 conn(0x7f6b30061c30 msgr2=0x7f6b300b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:30.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 -- 172.21.15.19:0/1015824023 shutdown_connections 2024-04-25T09:57:30.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.556+0000 7f6b37329640 1 -- 172.21.15.19:0/1015824023 wait complete. 2024-04-25T09:57:30.570 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-25T09:57:28.973197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":291652,"kb_used_data":8920,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754683068,"statfs":{"total":773094113280,"available":772795461632,"internally_reserved":0,"allocated":9134080,"data_stored":4828996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023261"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982420+0000","last_change":"2024-04-25T09:57:26.982359+0000","last_active":"2024-04-25T09:57:27.982420+0000","last_peered":"2024-04-25T09:57:27.982420+0000","last_clean":"2024-04-25T09:57:27.982420+0000","last_became_active":"2024-04-25T09:57:26.981833+0000","last_became_peered":"2024-04-25T09:57:26.981833+0000","last_unstale":"2024-04-25T09:57:27.982420+0000","last_undegraded":"2024-04-25T09:57:27.982420+0000","last_fullsized":"2024-04-25T09:57:27.982420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:35.184611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177015+0000","last_change":"2024-04-25T09:57:26.982517+0000","last_active":"2024-04-25T09:57:28.177015+0000","last_peered":"2024-04-25T09:57:28.177015+0000","last_clean":"2024-04-25T09:57:28.177015+0000","last_became_active":"2024-04-25T09:57:26.982119+0000","last_became_peered":"2024-04-25T09:57:26.982119+0000","last_unstale":"2024-04-25T09:57:28.177015+0000","last_undegraded":"2024-04-25T09:57:28.177015+0000","last_fullsized":"2024-04-25T09:57:28.177015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:39.635629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976350+0000","last_change":"2024-04-25T09:57:26.990420+0000","last_active":"2024-04-25T09:57:27.976350+0000","last_peered":"2024-04-25T09:57:27.976350+0000","last_clean":"2024-04-25T09:57:27.976350+0000","last_became_active":"2024-04-25T09:57:26.990216+0000","last_became_peered":"2024-04-25T09:57:26.990216+0000","last_unstale":"2024-04-25T09:57:27.976350+0000","last_undegraded":"2024-04-25T09:57:27.976350+0000","last_fullsized":"2024-04-25T09:57:27.976350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:03.128863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084206+0000","last_change":"2024-04-25T09:57:26.983824+0000","last_active":"2024-04-25T09:57:28.084206+0000","last_peered":"2024-04-25T09:57:28.084206+0000","last_clean":"2024-04-25T09:57:28.084206+0000","last_became_active":"2024-04-25T09:57:26.983564+0000","last_became_peered":"2024-04-25T09:57:26.983564+0000","last_unstale":"2024-04-25T09:57:28.084206+0000","last_undegraded":"2024-04-25T09:57:28.084206+0000","last_fullsized":"2024-04-25T09:57:28.084206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:47.385592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652104+0000","last_change":"2024-04-25T09:57:26.977747+0000","last_active":"2024-04-25T09:57:28.652104+0000","last_peered":"2024-04-25T09:57:28.652104+0000","last_clean":"2024-04-25T09:57:28.652104+0000","last_became_active":"2024-04-25T09:57:26.977403+0000","last_became_peered":"2024-04-25T09:57:26.977403+0000","last_unstale":"2024-04-25T09:57:28.652104+0000","last_undegraded":"2024-04-25T09:57:28.652104+0000","last_fullsized":"2024-04-25T09:57:28.652104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:35.838934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762475+0000","last_change":"2024-04-25T09:57:26.986370+0000","last_active":"2024-04-25T09:57:28.762475+0000","last_peered":"2024-04-25T09:57:28.762475+0000","last_clean":"2024-04-25T09:57:28.762475+0000","last_became_active":"2024-04-25T09:57:26.986114+0000","last_became_peered":"2024-04-25T09:57:26.986114+0000","last_unstale":"2024-04-25T09:57:28.762475+0000","last_undegraded":"2024-04-25T09:57:28.762475+0000","last_fullsized":"2024-04-25T09:57:28.762475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:05.397991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.984287+0000","last_change":"2024-04-25T09:57:26.982140+0000","last_active":"2024-04-25T09:57:26.984287+0000","last_peered":"2024-04-25T09:57:26.984287+0000","last_clean":"2024-04-25T09:57:26.984287+0000","last_became_active":"2024-04-25T09:57:26.980851+0000","last_became_peered":"2024-04-25T09:57:26.980851+0000","last_unstale":"2024-04-25T09:57:26.984287+0000","last_undegraded":"2024-04-25T09:57:26.984287+0000","last_fullsized":"2024-04-25T09:57:26.984287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:20.615201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.651971+0000","last_change":"2024-04-25T09:57:26.976091+0000","last_active":"2024-04-25T09:57:28.651971+0000","last_peered":"2024-04-25T09:57:28.651971+0000","last_clean":"2024-04-25T09:57:28.651971+0000","last_became_active":"2024-04-25T09:57:26.975800+0000","last_became_peered":"2024-04-25T09:57:26.975800+0000","last_unstale":"2024-04-25T09:57:28.651971+0000","last_undegraded":"2024-04-25T09:57:28.651971+0000","last_fullsized":"2024-04-25T09:57:28.651971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:44.647080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983224+0000","last_change":"2024-04-25T09:57:26.975169+0000","last_active":"2024-04-25T09:57:27.983224+0000","last_peered":"2024-04-25T09:57:27.983224+0000","last_clean":"2024-04-25T09:57:27.983224+0000","last_became_active":"2024-04-25T09:57:26.974904+0000","last_became_peered":"2024-04-25T09:57:26.974904+0000","last_unstale":"2024-04-25T09:57:27.983224+0000","last_undegraded":"2024-04-25T09:57:27.983224+0000","last_fullsized":"2024-04-25T09:57:27.983224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:37.093098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084182+0000","last_change":"2024-04-25T09:57:26.981689+0000","last_active":"2024-04-25T09:57:28.084182+0000","last_peered":"2024-04-25T09:57:28.084182+0000","last_clean":"2024-04-25T09:57:28.084182+0000","last_became_active":"2024-04-25T09:57:26.981410+0000","last_became_peered":"2024-04-25T09:57:26.981410+0000","last_unstale":"2024-04-25T09:57:28.084182+0000","last_undegraded":"2024-04-25T09:57:28.084182+0000","last_fullsized":"2024-04-25T09:57:28.084182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:54.451766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176826+0000","last_change":"2024-04-25T09:57:26.995424+0000","last_active":"2024-04-25T09:57:28.176826+0000","last_peered":"2024-04-25T09:57:28.176826+0000","last_clean":"2024-04-25T09:57:28.176826+0000","last_became_active":"2024-04-25T09:57:26.995143+0000","last_became_peered":"2024-04-25T09:57:26.995143+0000","last_unstale":"2024-04-25T09:57:28.176826+0000","last_undegraded":"2024-04-25T09:57:28.176826+0000","last_fullsized":"2024-04-25T09:57:28.176826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:07.019676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084232+0000","last_change":"2024-04-25T09:57:26.989346+0000","last_active":"2024-04-25T09:57:28.084232+0000","last_peered":"2024-04-25T09:57:28.084232+0000","last_clean":"2024-04-25T09:57:28.084232+0000","last_became_active":"2024-04-25T09:57:26.989063+0000","last_became_peered":"2024-04-25T09:57:26.989063+0000","last_unstale":"2024-04-25T09:57:28.084232+0000","last_undegraded":"2024-04-25T09:57:28.084232+0000","last_fullsized":"2024-04-25T09:57:28.084232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:14.824503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652292+0000","last_change":"2024-04-25T09:57:26.974876+0000","last_active":"2024-04-25T09:57:28.652292+0000","last_peered":"2024-04-25T09:57:28.652292+0000","last_clean":"2024-04-25T09:57:28.652292+0000","last_became_active":"2024-04-25T09:57:26.974577+0000","last_became_peered":"2024-04-25T09:57:26.974577+0000","last_unstale":"2024-04-25T09:57:28.652292+0000","last_undegraded":"2024-04-25T09:57:28.652292+0000","last_fullsized":"2024-04-25T09:57:28.652292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:21.257063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762504+0000","last_change":"2024-04-25T09:57:26.985432+0000","last_active":"2024-04-25T09:57:28.762504+0000","last_peered":"2024-04-25T09:57:28.762504+0000","last_clean":"2024-04-25T09:57:28.762504+0000","last_became_active":"2024-04-25T09:57:26.985167+0000","last_became_peered":"2024-04-25T09:57:26.985167+0000","last_unstale":"2024-04-25T09:57:28.762504+0000","last_undegraded":"2024-04-25T09:57:28.762504+0000","last_fullsized":"2024-04-25T09:57:28.762504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:35.184611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176794+0000","last_change":"2024-04-25T09:57:26.981564+0000","last_active":"2024-04-25T09:57:28.176794+0000","last_peered":"2024-04-25T09:57:28.176794+0000","last_clean":"2024-04-25T09:57:28.176794+0000","last_became_active":"2024-04-25T09:57:26.981267+0000","last_became_peered":"2024-04-25T09:57:26.981267+0000","last_unstale":"2024-04-25T09:57:28.176794+0000","last_undegraded":"2024-04-25T09:57:28.176794+0000","last_fullsized":"2024-04-25T09:57:28.176794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:27.935292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982551+0000","last_change":"2024-04-25T09:57:26.971445+0000","last_active":"2024-04-25T09:57:27.982551+0000","last_peered":"2024-04-25T09:57:27.982551+0000","last_clean":"2024-04-25T09:57:27.982551+0000","last_became_active":"2024-04-25T09:57:26.971037+0000","last_became_peered":"2024-04-25T09:57:26.971037+0000","last_unstale":"2024-04-25T09:57:27.982551+0000","last_undegraded":"2024-04-25T09:57:27.982551+0000","last_fullsized":"2024-04-25T09:57:27.982551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:31.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652366+0000","last_change":"2024-04-25T09:57:26.979651+0000","last_active":"2024-04-25T09:57:28.652366+0000","last_peered":"2024-04-25T09:57:28.652366+0000","last_clean":"2024-04-25T09:57:28.652366+0000","last_became_active":"2024-04-25T09:57:26.979363+0000","last_became_peered":"2024-04-25T09:57:26.979363+0000","last_unstale":"2024-04-25T09:57:28.652366+0000","last_undegraded":"2024-04-25T09:57:28.652366+0000","last_fullsized":"2024-04-25T09:57:28.652366+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:52.932708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976404+0000","last_change":"2024-04-25T09:57:26.979642+0000","last_active":"2024-04-25T09:57:27.976404+0000","last_peered":"2024-04-25T09:57:27.976404+0000","last_clean":"2024-04-25T09:57:27.976404+0000","last_became_active":"2024-04-25T09:57:26.979277+0000","last_became_peered":"2024-04-25T09:57:26.979277+0000","last_unstale":"2024-04-25T09:57:27.976404+0000","last_undegraded":"2024-04-25T09:57:27.976404+0000","last_fullsized":"2024-04-25T09:57:27.976404+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:06.092784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762788+0000","last_change":"2024-04-25T09:57:26.976315+0000","last_active":"2024-04-25T09:57:28.762788+0000","last_peered":"2024-04-25T09:57:28.762788+0000","last_clean":"2024-04-25T09:57:28.762788+0000","last_became_active":"2024-04-25T09:57:26.976054+0000","last_became_peered":"2024-04-25T09:57:26.976054+0000","last_unstale":"2024-04-25T09:57:28.762788+0000","last_undegraded":"2024-04-25T09:57:28.762788+0000","last_fullsized":"2024-04-25T09:57:28.762788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:36.159789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652292+0000","last_change":"2024-04-25T09:57:26.977707+0000","last_active":"2024-04-25T09:57:28.652292+0000","last_peered":"2024-04-25T09:57:28.652292+0000","last_clean":"2024-04-25T09:57:28.652292+0000","last_became_active":"2024-04-25T09:57:26.977328+0000","last_became_peered":"2024-04-25T09:57:26.977328+0000","last_unstale":"2024-04-25T09:57:28.652292+0000","last_undegraded":"2024-04-25T09:57:28.652292+0000","last_fullsized":"2024-04-25T09:57:28.652292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:45.163182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982463+0000","last_change":"2024-04-25T09:57:26.972540+0000","last_active":"2024-04-25T09:57:27.982463+0000","last_peered":"2024-04-25T09:57:27.982463+0000","last_clean":"2024-04-25T09:57:27.982463+0000","last_became_active":"2024-04-25T09:57:26.972285+0000","last_became_peered":"2024-04-25T09:57:26.972285+0000","last_unstale":"2024-04-25T09:57:27.982463+0000","last_undegraded":"2024-04-25T09:57:27.982463+0000","last_fullsized":"2024-04-25T09:57:27.982463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:21.494809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652331+0000","last_change":"2024-04-25T09:57:26.980937+0000","last_active":"2024-04-25T09:57:28.652331+0000","last_peered":"2024-04-25T09:57:28.652331+0000","last_clean":"2024-04-25T09:57:28.652331+0000","last_became_active":"2024-04-25T09:57:26.980655+0000","last_became_peered":"2024-04-25T09:57:26.980655+0000","last_unstale":"2024-04-25T09:57:28.652331+0000","last_undegraded":"2024-04-25T09:57:28.652331+0000","last_fullsized":"2024-04-25T09:57:28.652331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:22.787123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982244+0000","last_change":"2024-04-25T09:57:26.970723+0000","last_active":"2024-04-25T09:57:27.982244+0000","last_peered":"2024-04-25T09:57:27.982244+0000","last_clean":"2024-04-25T09:57:27.982244+0000","last_became_active":"2024-04-25T09:57:26.970451+0000","last_became_peered":"2024-04-25T09:57:26.970451+0000","last_unstale":"2024-04-25T09:57:27.982244+0000","last_undegraded":"2024-04-25T09:57:27.982244+0000","last_fullsized":"2024-04-25T09:57:27.982244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:58.324486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177158+0000","last_change":"2024-04-25T09:57:26.985571+0000","last_active":"2024-04-25T09:57:28.177158+0000","last_peered":"2024-04-25T09:57:28.177158+0000","last_clean":"2024-04-25T09:57:28.177158+0000","last_became_active":"2024-04-25T09:57:26.985303+0000","last_became_peered":"2024-04-25T09:57:26.985303+0000","last_unstale":"2024-04-25T09:57:28.177158+0000","last_undegraded":"2024-04-25T09:57:28.177158+0000","last_fullsized":"2024-04-25T09:57:28.177158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:57.219604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177013+0000","last_change":"2024-04-25T09:57:26.981675+0000","last_active":"2024-04-25T09:57:28.177013+0000","last_peered":"2024-04-25T09:57:28.177013+0000","last_clean":"2024-04-25T09:57:28.177013+0000","last_became_active":"2024-04-25T09:57:26.981386+0000","last_became_peered":"2024-04-25T09:57:26.981386+0000","last_unstale":"2024-04-25T09:57:28.177013+0000","last_undegraded":"2024-04-25T09:57:28.177013+0000","last_fullsized":"2024-04-25T09:57:28.177013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:20.012844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084244+0000","last_change":"2024-04-25T09:57:26.989888+0000","last_active":"2024-04-25T09:57:28.084244+0000","last_peered":"2024-04-25T09:57:28.084244+0000","last_clean":"2024-04-25T09:57:28.084244+0000","last_became_active":"2024-04-25T09:57:26.989273+0000","last_became_peered":"2024-04-25T09:57:26.989273+0000","last_unstale":"2024-04-25T09:57:28.084244+0000","last_undegraded":"2024-04-25T09:57:28.084244+0000","last_fullsized":"2024-04-25T09:57:28.084244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:09.210740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982392+0000","last_change":"2024-04-25T09:57:26.971302+0000","last_active":"2024-04-25T09:57:27.982392+0000","last_peered":"2024-04-25T09:57:27.982392+0000","last_clean":"2024-04-25T09:57:27.982392+0000","last_became_active":"2024-04-25T09:57:26.971015+0000","last_became_peered":"2024-04-25T09:57:26.971015+0000","last_unstale":"2024-04-25T09:57:27.982392+0000","last_undegraded":"2024-04-25T09:57:27.982392+0000","last_fullsized":"2024-04-25T09:57:27.982392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.273632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652215+0000","last_change":"2024-04-25T09:57:26.985183+0000","last_active":"2024-04-25T09:57:28.652215+0000","last_peered":"2024-04-25T09:57:28.652215+0000","last_clean":"2024-04-25T09:57:28.652215+0000","last_became_active":"2024-04-25T09:57:26.984888+0000","last_became_peered":"2024-04-25T09:57:26.984888+0000","last_unstale":"2024-04-25T09:57:28.652215+0000","last_undegraded":"2024-04-25T09:57:28.652215+0000","last_fullsized":"2024-04-25T09:57:28.652215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:51.458960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176722+0000","last_change":"2024-04-25T09:57:26.992227+0000","last_active":"2024-04-25T09:57:28.176722+0000","last_peered":"2024-04-25T09:57:28.176722+0000","last_clean":"2024-04-25T09:57:28.176722+0000","last_became_active":"2024-04-25T09:57:26.991944+0000","last_became_peered":"2024-04-25T09:57:26.991944+0000","last_unstale":"2024-04-25T09:57:28.176722+0000","last_undegraded":"2024-04-25T09:57:28.176722+0000","last_fullsized":"2024-04-25T09:57:28.176722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:27.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982024+0000","last_change":"2024-04-25T09:57:26.989327+0000","last_active":"2024-04-25T09:57:27.982024+0000","last_peered":"2024-04-25T09:57:27.982024+0000","last_clean":"2024-04-25T09:57:27.982024+0000","last_became_active":"2024-04-25T09:57:26.989033+0000","last_became_peered":"2024-04-25T09:57:26.989033+0000","last_unstale":"2024-04-25T09:57:27.982024+0000","last_undegraded":"2024-04-25T09:57:27.982024+0000","last_fullsized":"2024-04-25T09:57:27.982024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:05.397991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976110+0000","last_change":"2024-04-25T09:57:26.978153+0000","last_active":"2024-04-25T09:57:27.976110+0000","last_peered":"2024-04-25T09:57:27.976110+0000","last_clean":"2024-04-25T09:57:27.976110+0000","last_became_active":"2024-04-25T09:57:26.977787+0000","last_became_peered":"2024-04-25T09:57:26.977787+0000","last_unstale":"2024-04-25T09:57:27.976110+0000","last_undegraded":"2024-04-25T09:57:27.976110+0000","last_fullsized":"2024-04-25T09:57:27.976110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:21.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982619+0000","last_change":"2024-04-25T09:57:26.982195+0000","last_active":"2024-04-25T09:57:27.982619+0000","last_peered":"2024-04-25T09:57:27.982619+0000","last_clean":"2024-04-25T09:57:27.982619+0000","last_became_active":"2024-04-25T09:57:26.981889+0000","last_became_peered":"2024-04-25T09:57:26.981889+0000","last_unstale":"2024-04-25T09:57:27.982619+0000","last_undegraded":"2024-04-25T09:57:27.982619+0000","last_fullsized":"2024-04-25T09:57:27.982619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:29.840350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976053+0000","last_change":"2024-04-25T09:57:26.974168+0000","last_active":"2024-04-25T09:57:27.976053+0000","last_peered":"2024-04-25T09:57:27.976053+0000","last_clean":"2024-04-25T09:57:27.976053+0000","last_became_active":"2024-04-25T09:57:26.973823+0000","last_became_peered":"2024-04-25T09:57:26.973823+0000","last_unstale":"2024-04-25T09:57:27.976053+0000","last_undegraded":"2024-04-25T09:57:27.976053+0000","last_fullsized":"2024-04-25T09:57:27.976053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:20.605277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652462+0000","last_change":"2024-04-25T09:57:25.980755+0000","last_active":"2024-04-25T09:57:28.652462+0000","last_peered":"2024-04-25T09:57:28.652462+0000","last_clean":"2024-04-25T09:57:28.652462+0000","last_became_active":"2024-04-25T09:57:25.978318+0000","last_became_peered":"2024-04-25T09:57:25.978318+0000","last_unstale":"2024-04-25T09:57:28.652462+0000","last_undegraded":"2024-04-25T09:57:28.652462+0000","last_fullsized":"2024-04-25T09:57:28.652462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:13.731726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176847+0000","last_change":"2024-04-25T09:57:26.989710+0000","last_active":"2024-04-25T09:57:28.176847+0000","last_peered":"2024-04-25T09:57:28.176847+0000","last_clean":"2024-04-25T09:57:28.176847+0000","last_became_active":"2024-04-25T09:57:26.989031+0000","last_became_peered":"2024-04-25T09:57:26.989031+0000","last_unstale":"2024-04-25T09:57:28.176847+0000","last_undegraded":"2024-04-25T09:57:28.176847+0000","last_fullsized":"2024-04-25T09:57:28.176847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:44.942613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982383+0000","last_change":"2024-04-25T09:57:25.978427+0000","last_active":"2024-04-25T09:57:27.982383+0000","last_peered":"2024-04-25T09:57:27.982383+0000","last_clean":"2024-04-25T09:57:27.982383+0000","last_became_active":"2024-04-25T09:57:25.978154+0000","last_became_peered":"2024-04-25T09:57:25.978154+0000","last_unstale":"2024-04-25T09:57:27.982383+0000","last_undegraded":"2024-04-25T09:57:27.982383+0000","last_fullsized":"2024-04-25T09:57:27.982383+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:35.642932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983017+0000","last_change":"2024-04-25T09:57:26.982132+0000","last_active":"2024-04-25T09:57:27.983017+0000","last_peered":"2024-04-25T09:57:27.983017+0000","last_clean":"2024-04-25T09:57:27.983017+0000","last_became_active":"2024-04-25T09:57:26.981880+0000","last_became_peered":"2024-04-25T09:57:26.981880+0000","last_unstale":"2024-04-25T09:57:27.983017+0000","last_undegraded":"2024-04-25T09:57:27.983017+0000","last_fullsized":"2024-04-25T09:57:27.983017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:53.632031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982381+0000","last_change":"2024-04-25T09:57:25.977765+0000","last_active":"2024-04-25T09:57:27.982381+0000","last_peered":"2024-04-25T09:57:27.982381+0000","last_clean":"2024-04-25T09:57:27.982381+0000","last_became_active":"2024-04-25T09:57:25.977508+0000","last_became_peered":"2024-04-25T09:57:25.977508+0000","last_unstale":"2024-04-25T09:57:27.982381+0000","last_undegraded":"2024-04-25T09:57:27.982381+0000","last_fullsized":"2024-04-25T09:57:27.982381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.137562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982591+0000","last_change":"2024-04-25T09:57:26.971424+0000","last_active":"2024-04-25T09:57:27.982591+0000","last_peered":"2024-04-25T09:57:27.982591+0000","last_clean":"2024-04-25T09:57:27.982591+0000","last_became_active":"2024-04-25T09:57:26.971171+0000","last_became_peered":"2024-04-25T09:57:26.971171+0000","last_unstale":"2024-04-25T09:57:27.982591+0000","last_undegraded":"2024-04-25T09:57:27.982591+0000","last_fullsized":"2024-04-25T09:57:27.982591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:53.307213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.979253+0000","last_change":"2024-04-25T09:57:25.980034+0000","last_active":"2024-04-25T09:57:26.979253+0000","last_peered":"2024-04-25T09:57:26.979253+0000","last_clean":"2024-04-25T09:57:26.979253+0000","last_became_active":"2024-04-25T09:57:25.979582+0000","last_became_peered":"2024-04-25T09:57:25.979582+0000","last_unstale":"2024-04-25T09:57:26.979253+0000","last_undegraded":"2024-04-25T09:57:26.979253+0000","last_fullsized":"2024-04-25T09:57:26.979253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:08.325589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177123+0000","last_change":"2024-04-25T09:57:26.988577+0000","last_active":"2024-04-25T09:57:28.177123+0000","last_peered":"2024-04-25T09:57:28.177123+0000","last_clean":"2024-04-25T09:57:28.177123+0000","last_became_active":"2024-04-25T09:57:26.988286+0000","last_became_peered":"2024-04-25T09:57:26.988286+0000","last_unstale":"2024-04-25T09:57:28.177123+0000","last_undegraded":"2024-04-25T09:57:28.177123+0000","last_fullsized":"2024-04-25T09:57:28.177123+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:07.941153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982684+0000","last_change":"2024-04-25T09:57:25.973204+0000","last_active":"2024-04-25T09:57:27.982684+0000","last_peered":"2024-04-25T09:57:27.982684+0000","last_clean":"2024-04-25T09:57:27.982684+0000","last_became_active":"2024-04-25T09:57:25.972931+0000","last_became_peered":"2024-04-25T09:57:25.972931+0000","last_unstale":"2024-04-25T09:57:27.982684+0000","last_undegraded":"2024-04-25T09:57:27.982684+0000","last_fullsized":"2024-04-25T09:57:27.982684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:53.825752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176953+0000","last_change":"2024-04-25T09:57:26.982617+0000","last_active":"2024-04-25T09:57:28.176953+0000","last_peered":"2024-04-25T09:57:28.176953+0000","last_clean":"2024-04-25T09:57:28.176953+0000","last_became_active":"2024-04-25T09:57:26.982270+0000","last_became_peered":"2024-04-25T09:57:26.982270+0000","last_unstale":"2024-04-25T09:57:28.176953+0000","last_undegraded":"2024-04-25T09:57:28.176953+0000","last_fullsized":"2024-04-25T09:57:28.176953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:17.051240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.083946+0000","last_change":"2024-04-25T09:57:25.975563+0000","last_active":"2024-04-25T09:57:28.083946+0000","last_peered":"2024-04-25T09:57:28.083946+0000","last_clean":"2024-04-25T09:57:28.083946+0000","last_became_active":"2024-04-25T09:57:25.974734+0000","last_became_peered":"2024-04-25T09:57:25.974734+0000","last_unstale":"2024-04-25T09:57:28.083946+0000","last_undegraded":"2024-04-25T09:57:28.083946+0000","last_fullsized":"2024-04-25T09:57:28.083946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:47.529460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983186+0000","last_change":"2024-04-25T09:57:26.981481+0000","last_active":"2024-04-25T09:57:27.983186+0000","last_peered":"2024-04-25T09:57:27.983186+0000","last_clean":"2024-04-25T09:57:27.983186+0000","last_became_active":"2024-04-25T09:57:26.978900+0000","last_became_peered":"2024-04-25T09:57:26.978900+0000","last_unstale":"2024-04-25T09:57:27.983186+0000","last_undegraded":"2024-04-25T09:57:27.983186+0000","last_fullsized":"2024-04-25T09:57:27.983186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:50.144838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176911+0000","last_change":"2024-04-25T09:57:25.973998+0000","last_active":"2024-04-25T09:57:28.176911+0000","last_peered":"2024-04-25T09:57:28.176911+0000","last_clean":"2024-04-25T09:57:28.176911+0000","last_became_active":"2024-04-25T09:57:25.973464+0000","last_became_peered":"2024-04-25T09:57:25.973464+0000","last_unstale":"2024-04-25T09:57:28.176911+0000","last_undegraded":"2024-04-25T09:57:28.176911+0000","last_fullsized":"2024-04-25T09:57:28.176911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:57.902028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.980725+0000","last_change":"2024-04-25T09:57:26.979586+0000","last_active":"2024-04-25T09:57:26.980725+0000","last_peered":"2024-04-25T09:57:26.980725+0000","last_clean":"2024-04-25T09:57:26.980725+0000","last_became_active":"2024-04-25T09:57:26.979325+0000","last_became_peered":"2024-04-25T09:57:26.979325+0000","last_unstale":"2024-04-25T09:57:26.980725+0000","last_undegraded":"2024-04-25T09:57:26.980725+0000","last_fullsized":"2024-04-25T09:57:26.980725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:53.632031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652299+0000","last_change":"2024-04-25T09:57:25.980118+0000","last_active":"2024-04-25T09:57:28.652299+0000","last_peered":"2024-04-25T09:57:28.652299+0000","last_clean":"2024-04-25T09:57:28.652299+0000","last_became_active":"2024-04-25T09:57:25.979840+0000","last_became_peered":"2024-04-25T09:57:25.979840+0000","last_unstale":"2024-04-25T09:57:28.652299+0000","last_undegraded":"2024-04-25T09:57:28.652299+0000","last_fullsized":"2024-04-25T09:57:28.652299+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:01.083810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177291+0000","last_change":"2024-04-25T09:57:26.989278+0000","last_active":"2024-04-25T09:57:28.177291+0000","last_peered":"2024-04-25T09:57:28.177291+0000","last_clean":"2024-04-25T09:57:28.177291+0000","last_became_active":"2024-04-25T09:57:26.988983+0000","last_became_peered":"2024-04-25T09:57:26.988983+0000","last_unstale":"2024-04-25T09:57:28.177291+0000","last_undegraded":"2024-04-25T09:57:28.177291+0000","last_fullsized":"2024-04-25T09:57:28.177291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:10.862261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982535+0000","last_change":"2024-04-25T09:57:25.977126+0000","last_active":"2024-04-25T09:57:27.982535+0000","last_peered":"2024-04-25T09:57:27.982535+0000","last_clean":"2024-04-25T09:57:27.982535+0000","last_became_active":"2024-04-25T09:57:25.976841+0000","last_became_peered":"2024-04-25T09:57:25.976841+0000","last_unstale":"2024-04-25T09:57:27.982535+0000","last_undegraded":"2024-04-25T09:57:27.982535+0000","last_fullsized":"2024-04-25T09:57:27.982535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:13.762594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084147+0000","last_change":"2024-04-25T09:57:26.982884+0000","last_active":"2024-04-25T09:57:28.084147+0000","last_peered":"2024-04-25T09:57:28.084147+0000","last_clean":"2024-04-25T09:57:28.084147+0000","last_became_active":"2024-04-25T09:57:26.982608+0000","last_became_peered":"2024-04-25T09:57:26.982608+0000","last_unstale":"2024-04-25T09:57:28.084147+0000","last_undegraded":"2024-04-25T09:57:28.084147+0000","last_fullsized":"2024-04-25T09:57:28.084147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:47.142009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084269+0000","last_change":"2024-04-25T09:57:25.977803+0000","last_active":"2024-04-25T09:57:28.084269+0000","last_peered":"2024-04-25T09:57:28.084269+0000","last_clean":"2024-04-25T09:57:28.084269+0000","last_became_active":"2024-04-25T09:57:25.977522+0000","last_became_peered":"2024-04-25T09:57:25.977522+0000","last_unstale":"2024-04-25T09:57:28.084269+0000","last_undegraded":"2024-04-25T09:57:28.084269+0000","last_fullsized":"2024-04-25T09:57:28.084269+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:16.527873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762724+0000","last_change":"2024-04-25T09:57:26.979198+0000","last_active":"2024-04-25T09:57:28.762724+0000","last_peered":"2024-04-25T09:57:28.762724+0000","last_clean":"2024-04-25T09:57:28.762724+0000","last_became_active":"2024-04-25T09:57:26.972993+0000","last_became_peered":"2024-04-25T09:57:26.972993+0000","last_unstale":"2024-04-25T09:57:28.762724+0000","last_undegraded":"2024-04-25T09:57:28.762724+0000","last_fullsized":"2024-04-25T09:57:28.762724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:31.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652526+0000","last_change":"2024-04-25T09:57:25.982931+0000","last_active":"2024-04-25T09:57:28.652526+0000","last_peered":"2024-04-25T09:57:28.652526+0000","last_clean":"2024-04-25T09:57:28.652526+0000","last_became_active":"2024-04-25T09:57:25.978341+0000","last_became_peered":"2024-04-25T09:57:25.978341+0000","last_unstale":"2024-04-25T09:57:28.652526+0000","last_undegraded":"2024-04-25T09:57:28.652526+0000","last_fullsized":"2024-04-25T09:57:28.652526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:41.218305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976240+0000","last_change":"2024-04-25T09:57:26.991164+0000","last_active":"2024-04-25T09:57:27.976240+0000","last_peered":"2024-04-25T09:57:27.976240+0000","last_clean":"2024-04-25T09:57:27.976240+0000","last_became_active":"2024-04-25T09:57:26.990863+0000","last_became_peered":"2024-04-25T09:57:26.990863+0000","last_unstale":"2024-04-25T09:57:27.976240+0000","last_undegraded":"2024-04-25T09:57:27.976240+0000","last_fullsized":"2024-04-25T09:57:27.976240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:17.725041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"80'6","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.990500+0000","last_change":"2024-04-25T09:57:25.980275+0000","last_active":"2024-04-25T09:57:26.990500+0000","last_peered":"2024-04-25T09:57:26.990500+0000","last_clean":"2024-04-25T09:57:26.990500+0000","last_became_active":"2024-04-25T09:57:25.979873+0000","last_became_peered":"2024-04-25T09:57:25.979873+0000","last_unstale":"2024-04-25T09:57:26.990500+0000","last_undegraded":"2024-04-25T09:57:26.990500+0000","last_fullsized":"2024-04-25T09:57:26.990500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:57.320039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084306+0000","last_change":"2024-04-25T09:57:26.976015+0000","last_active":"2024-04-25T09:57:28.084306+0000","last_peered":"2024-04-25T09:57:28.084306+0000","last_clean":"2024-04-25T09:57:28.084306+0000","last_became_active":"2024-04-25T09:57:26.975715+0000","last_became_peered":"2024-04-25T09:57:26.975715+0000","last_unstale":"2024-04-25T09:57:28.084306+0000","last_undegraded":"2024-04-25T09:57:28.084306+0000","last_fullsized":"2024-04-25T09:57:28.084306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:11.580893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.083975+0000","last_change":"2024-04-25T09:57:25.977037+0000","last_active":"2024-04-25T09:57:28.083975+0000","last_peered":"2024-04-25T09:57:28.083975+0000","last_clean":"2024-04-25T09:57:28.083975+0000","last_became_active":"2024-04-25T09:57:25.976742+0000","last_became_peered":"2024-04-25T09:57:25.976742+0000","last_unstale":"2024-04-25T09:57:28.083975+0000","last_undegraded":"2024-04-25T09:57:28.083975+0000","last_fullsized":"2024-04-25T09:57:28.083975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:33.969019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.977200+0000","last_change":"2024-04-25T09:57:26.973316+0000","last_active":"2024-04-25T09:57:26.977200+0000","last_peered":"2024-04-25T09:57:26.977200+0000","last_clean":"2024-04-25T09:57:26.977200+0000","last_became_active":"2024-04-25T09:57:26.973006+0000","last_became_peered":"2024-04-25T09:57:26.973006+0000","last_unstale":"2024-04-25T09:57:26.977200+0000","last_undegraded":"2024-04-25T09:57:26.977200+0000","last_fullsized":"2024-04-25T09:57:26.977200+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:21.494809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982238+0000","last_change":"2024-04-25T09:57:25.983301+0000","last_active":"2024-04-25T09:57:27.982238+0000","last_peered":"2024-04-25T09:57:27.982238+0000","last_clean":"2024-04-25T09:57:27.982238+0000","last_became_active":"2024-04-25T09:57:25.983059+0000","last_became_peered":"2024-04-25T09:57:25.983059+0000","last_unstale":"2024-04-25T09:57:27.982238+0000","last_undegraded":"2024-04-25T09:57:27.982238+0000","last_fullsized":"2024-04-25T09:57:27.982238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:19.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.984955+0000","last_change":"2024-04-25T09:57:26.983768+0000","last_active":"2024-04-25T09:57:26.984955+0000","last_peered":"2024-04-25T09:57:26.984955+0000","last_clean":"2024-04-25T09:57:26.984955+0000","last_became_active":"2024-04-25T09:57:26.983504+0000","last_became_peered":"2024-04-25T09:57:26.983504+0000","last_unstale":"2024-04-25T09:57:26.984955+0000","last_undegraded":"2024-04-25T09:57:26.984955+0000","last_fullsized":"2024-04-25T09:57:26.984955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:42.732984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652018+0000","last_change":"2024-04-25T09:57:25.980102+0000","last_active":"2024-04-25T09:57:28.652018+0000","last_peered":"2024-04-25T09:57:28.652018+0000","last_clean":"2024-04-25T09:57:28.652018+0000","last_became_active":"2024-04-25T09:57:25.979797+0000","last_became_peered":"2024-04-25T09:57:25.979797+0000","last_unstale":"2024-04-25T09:57:28.652018+0000","last_undegraded":"2024-04-25T09:57:28.652018+0000","last_fullsized":"2024-04-25T09:57:28.652018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:20.839016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.985052+0000","last_change":"2024-04-25T09:57:26.983708+0000","last_active":"2024-04-25T09:57:26.985052+0000","last_peered":"2024-04-25T09:57:26.985052+0000","last_clean":"2024-04-25T09:57:26.985052+0000","last_became_active":"2024-04-25T09:57:26.982989+0000","last_became_peered":"2024-04-25T09:57:26.982989+0000","last_unstale":"2024-04-25T09:57:26.985052+0000","last_undegraded":"2024-04-25T09:57:26.985052+0000","last_fullsized":"2024-04-25T09:57:26.985052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:19.766616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177261+0000","last_change":"2024-04-25T09:57:25.979176+0000","last_active":"2024-04-25T09:57:28.177261+0000","last_peered":"2024-04-25T09:57:28.177261+0000","last_clean":"2024-04-25T09:57:28.177261+0000","last_became_active":"2024-04-25T09:57:25.978881+0000","last_became_peered":"2024-04-25T09:57:25.978881+0000","last_unstale":"2024-04-25T09:57:28.177261+0000","last_undegraded":"2024-04-25T09:57:28.177261+0000","last_fullsized":"2024-04-25T09:57:28.177261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.137562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762276+0000","last_change":"2024-04-25T09:57:26.980663+0000","last_active":"2024-04-25T09:57:28.762276+0000","last_peered":"2024-04-25T09:57:28.762276+0000","last_clean":"2024-04-25T09:57:28.762276+0000","last_became_active":"2024-04-25T09:57:26.980389+0000","last_became_peered":"2024-04-25T09:57:26.980389+0000","last_unstale":"2024-04-25T09:57:28.762276+0000","last_undegraded":"2024-04-25T09:57:28.762276+0000","last_fullsized":"2024-04-25T09:57:28.762276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:20.354301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982656+0000","last_change":"2024-04-25T09:57:25.975386+0000","last_active":"2024-04-25T09:57:27.982656+0000","last_peered":"2024-04-25T09:57:27.982656+0000","last_clean":"2024-04-25T09:57:27.982656+0000","last_became_active":"2024-04-25T09:57:25.975112+0000","last_became_peered":"2024-04-25T09:57:25.975112+0000","last_unstale":"2024-04-25T09:57:27.982656+0000","last_undegraded":"2024-04-25T09:57:27.982656+0000","last_fullsized":"2024-04-25T09:57:27.982656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:57.902028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652278+0000","last_change":"2024-04-25T09:57:26.982146+0000","last_active":"2024-04-25T09:57:28.652278+0000","last_peered":"2024-04-25T09:57:28.652278+0000","last_clean":"2024-04-25T09:57:28.652278+0000","last_became_active":"2024-04-25T09:57:26.981876+0000","last_became_peered":"2024-04-25T09:57:26.981876+0000","last_unstale":"2024-04-25T09:57:28.652278+0000","last_undegraded":"2024-04-25T09:57:28.652278+0000","last_fullsized":"2024-04-25T09:57:28.652278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:09.926715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983202+0000","last_change":"2024-04-25T09:57:25.975455+0000","last_active":"2024-04-25T09:57:27.983202+0000","last_peered":"2024-04-25T09:57:27.983202+0000","last_clean":"2024-04-25T09:57:27.983202+0000","last_became_active":"2024-04-25T09:57:25.975169+0000","last_became_peered":"2024-04-25T09:57:25.975169+0000","last_unstale":"2024-04-25T09:57:27.983202+0000","last_undegraded":"2024-04-25T09:57:27.983202+0000","last_fullsized":"2024-04-25T09:57:27.983202+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:43.008000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652144+0000","last_change":"2024-04-25T09:57:26.990096+0000","last_active":"2024-04-25T09:57:28.652144+0000","last_peered":"2024-04-25T09:57:28.652144+0000","last_clean":"2024-04-25T09:57:28.652144+0000","last_became_active":"2024-04-25T09:57:26.989786+0000","last_became_peered":"2024-04-25T09:57:26.989786+0000","last_unstale":"2024-04-25T09:57:28.652144+0000","last_undegraded":"2024-04-25T09:57:28.652144+0000","last_fullsized":"2024-04-25T09:57:28.652144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:46.603634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652532+0000","last_change":"2024-04-25T09:57:25.981279+0000","last_active":"2024-04-25T09:57:28.652532+0000","last_peered":"2024-04-25T09:57:28.652532+0000","last_clean":"2024-04-25T09:57:28.652532+0000","last_became_active":"2024-04-25T09:57:25.980986+0000","last_became_peered":"2024-04-25T09:57:25.980986+0000","last_unstale":"2024-04-25T09:57:28.652532+0000","last_undegraded":"2024-04-25T09:57:28.652532+0000","last_fullsized":"2024-04-25T09:57:28.652532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:27.643209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762490+0000","last_change":"2024-04-25T09:57:26.974844+0000","last_active":"2024-04-25T09:57:28.762490+0000","last_peered":"2024-04-25T09:57:28.762490+0000","last_clean":"2024-04-25T09:57:28.762490+0000","last_became_active":"2024-04-25T09:57:26.974558+0000","last_became_peered":"2024-04-25T09:57:26.974558+0000","last_unstale":"2024-04-25T09:57:28.762490+0000","last_undegraded":"2024-04-25T09:57:28.762490+0000","last_fullsized":"2024-04-25T09:57:28.762490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.273632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976195+0000","last_change":"2024-04-25T09:57:25.982149+0000","last_active":"2024-04-25T09:57:27.976195+0000","last_peered":"2024-04-25T09:57:27.976195+0000","last_clean":"2024-04-25T09:57:27.976195+0000","last_became_active":"2024-04-25T09:57:25.981765+0000","last_became_peered":"2024-04-25T09:57:25.981765+0000","last_unstale":"2024-04-25T09:57:27.976195+0000","last_undegraded":"2024-04-25T09:57:27.976195+0000","last_fullsized":"2024-04-25T09:57:27.976195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:52.627584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176999+0000","last_change":"2024-04-25T09:57:26.992142+0000","last_active":"2024-04-25T09:57:28.176999+0000","last_peered":"2024-04-25T09:57:28.176999+0000","last_clean":"2024-04-25T09:57:28.176999+0000","last_became_active":"2024-04-25T09:57:26.991842+0000","last_became_peered":"2024-04-25T09:57:26.991842+0000","last_unstale":"2024-04-25T09:57:28.176999+0000","last_undegraded":"2024-04-25T09:57:28.176999+0000","last_fullsized":"2024-04-25T09:57:28.176999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:16.358371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762519+0000","last_change":"2024-04-25T09:57:25.976450+0000","last_active":"2024-04-25T09:57:28.762519+0000","last_peered":"2024-04-25T09:57:28.762519+0000","last_clean":"2024-04-25T09:57:28.762519+0000","last_became_active":"2024-04-25T09:57:25.976171+0000","last_became_peered":"2024-04-25T09:57:25.976171+0000","last_unstale":"2024-04-25T09:57:28.762519+0000","last_undegraded":"2024-04-25T09:57:28.762519+0000","last_fullsized":"2024-04-25T09:57:28.762519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:57.902028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177172+0000","last_change":"2024-04-25T09:57:26.993684+0000","last_active":"2024-04-25T09:57:28.177172+0000","last_peered":"2024-04-25T09:57:28.177172+0000","last_clean":"2024-04-25T09:57:28.177172+0000","last_became_active":"2024-04-25T09:57:26.993403+0000","last_became_peered":"2024-04-25T09:57:26.993403+0000","last_unstale":"2024-04-25T09:57:28.177172+0000","last_undegraded":"2024-04-25T09:57:28.177172+0000","last_fullsized":"2024-04-25T09:57:28.177172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:16.732299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762336+0000","last_change":"2024-04-25T09:57:25.978638+0000","last_active":"2024-04-25T09:57:28.762336+0000","last_peered":"2024-04-25T09:57:28.762336+0000","last_clean":"2024-04-25T09:57:28.762336+0000","last_became_active":"2024-04-25T09:57:25.978352+0000","last_became_peered":"2024-04-25T09:57:25.978352+0000","last_unstale":"2024-04-25T09:57:28.762336+0000","last_undegraded":"2024-04-25T09:57:28.762336+0000","last_fullsized":"2024-04-25T09:57:28.762336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:35.642932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.984892+0000","last_change":"2024-04-25T09:57:26.983315+0000","last_active":"2024-04-25T09:57:26.984892+0000","last_peered":"2024-04-25T09:57:26.984892+0000","last_clean":"2024-04-25T09:57:26.984892+0000","last_became_active":"2024-04-25T09:57:26.982625+0000","last_became_peered":"2024-04-25T09:57:26.982625+0000","last_unstale":"2024-04-25T09:57:26.984892+0000","last_undegraded":"2024-04-25T09:57:26.984892+0000","last_fullsized":"2024-04-25T09:57:26.984892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:37.093098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762308+0000","last_change":"2024-04-25T09:57:25.978782+0000","last_active":"2024-04-25T09:57:28.762308+0000","last_peered":"2024-04-25T09:57:28.762308+0000","last_clean":"2024-04-25T09:57:28.762308+0000","last_became_active":"2024-04-25T09:57:25.978421+0000","last_became_peered":"2024-04-25T09:57:25.978421+0000","last_unstale":"2024-04-25T09:57:28.762308+0000","last_undegraded":"2024-04-25T09:57:28.762308+0000","last_fullsized":"2024-04-25T09:57:28.762308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:13.762594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983196+0000","last_change":"2024-04-25T09:57:26.975675+0000","last_active":"2024-04-25T09:57:27.983196+0000","last_peered":"2024-04-25T09:57:27.983196+0000","last_clean":"2024-04-25T09:57:27.983196+0000","last_became_active":"2024-04-25T09:57:26.975400+0000","last_became_peered":"2024-04-25T09:57:26.975400+0000","last_unstale":"2024-04-25T09:57:27.983196+0000","last_undegraded":"2024-04-25T09:57:27.983196+0000","last_fullsized":"2024-04-25T09:57:27.983196+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:20.615201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084161+0000","last_change":"2024-04-25T09:57:25.976538+0000","last_active":"2024-04-25T09:57:28.084161+0000","last_peered":"2024-04-25T09:57:28.084161+0000","last_clean":"2024-04-25T09:57:28.084161+0000","last_became_active":"2024-04-25T09:57:25.976243+0000","last_became_peered":"2024-04-25T09:57:25.976243+0000","last_unstale":"2024-04-25T09:57:28.084161+0000","last_undegraded":"2024-04-25T09:57:28.084161+0000","last_fullsized":"2024-04-25T09:57:28.084161+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:00.015322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.984264+0000","last_change":"2024-04-25T09:57:26.981900+0000","last_active":"2024-04-25T09:57:26.984264+0000","last_peered":"2024-04-25T09:57:26.984264+0000","last_clean":"2024-04-25T09:57:26.984264+0000","last_became_active":"2024-04-25T09:57:26.981568+0000","last_became_peered":"2024-04-25T09:57:26.981568+0000","last_unstale":"2024-04-25T09:57:26.984264+0000","last_undegraded":"2024-04-25T09:57:26.984264+0000","last_fullsized":"2024-04-25T09:57:26.984264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:50.144838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177159+0000","last_change":"2024-04-25T09:57:25.984072+0000","last_active":"2024-04-25T09:57:28.177159+0000","last_peered":"2024-04-25T09:57:28.177159+0000","last_clean":"2024-04-25T09:57:28.177159+0000","last_became_active":"2024-04-25T09:57:25.983745+0000","last_became_peered":"2024-04-25T09:57:25.983745+0000","last_unstale":"2024-04-25T09:57:28.177159+0000","last_undegraded":"2024-04-25T09:57:28.177159+0000","last_fullsized":"2024-04-25T09:57:28.177159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:19.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983178+0000","last_change":"2024-04-25T09:57:26.981513+0000","last_active":"2024-04-25T09:57:27.983178+0000","last_peered":"2024-04-25T09:57:27.983178+0000","last_clean":"2024-04-25T09:57:27.983178+0000","last_became_active":"2024-04-25T09:57:26.981214+0000","last_became_peered":"2024-04-25T09:57:26.981214+0000","last_unstale":"2024-04-25T09:57:27.983178+0000","last_undegraded":"2024-04-25T09:57:27.983178+0000","last_fullsized":"2024-04-25T09:57:27.983178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:40.708493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982713+0000","last_change":"2024-04-25T09:57:25.974949+0000","last_active":"2024-04-25T09:57:27.982713+0000","last_peered":"2024-04-25T09:57:27.982713+0000","last_clean":"2024-04-25T09:57:27.982713+0000","last_became_active":"2024-04-25T09:57:25.974625+0000","last_became_peered":"2024-04-25T09:57:25.974625+0000","last_unstale":"2024-04-25T09:57:27.982713+0000","last_undegraded":"2024-04-25T09:57:27.982713+0000","last_fullsized":"2024-04-25T09:57:27.982713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:53.447319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176811+0000","last_change":"2024-04-25T09:57:26.988576+0000","last_active":"2024-04-25T09:57:28.176811+0000","last_peered":"2024-04-25T09:57:28.176811+0000","last_clean":"2024-04-25T09:57:28.176811+0000","last_became_active":"2024-04-25T09:57:26.988277+0000","last_became_peered":"2024-04-25T09:57:26.988277+0000","last_unstale":"2024-04-25T09:57:28.176811+0000","last_undegraded":"2024-04-25T09:57:28.176811+0000","last_fullsized":"2024-04-25T09:57:28.176811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:07.334443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982143+0000","last_change":"2024-04-25T09:57:25.977187+0000","last_active":"2024-04-25T09:57:27.982143+0000","last_peered":"2024-04-25T09:57:27.982143+0000","last_clean":"2024-04-25T09:57:27.982143+0000","last_became_active":"2024-04-25T09:57:25.976825+0000","last_became_peered":"2024-04-25T09:57:25.976825+0000","last_unstale":"2024-04-25T09:57:27.982143+0000","last_undegraded":"2024-04-25T09:57:27.982143+0000","last_fullsized":"2024-04-25T09:57:27.982143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:27.866179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.979830+0000","last_change":"2024-04-25T09:57:26.978261+0000","last_active":"2024-04-25T09:57:26.979830+0000","last_peered":"2024-04-25T09:57:26.979830+0000","last_clean":"2024-04-25T09:57:26.979830+0000","last_became_active":"2024-04-25T09:57:26.977918+0000","last_became_peered":"2024-04-25T09:57:26.977918+0000","last_unstale":"2024-04-25T09:57:26.979830+0000","last_undegraded":"2024-04-25T09:57:26.979830+0000","last_fullsized":"2024-04-25T09:57:26.979830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:53.307213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.966465+0000","last_change":"2024-04-25T09:57:25.977540+0000","last_active":"2024-04-25T09:57:26.966465+0000","last_peered":"2024-04-25T09:57:26.966465+0000","last_clean":"2024-04-25T09:57:26.966465+0000","last_became_active":"2024-04-25T09:57:25.977229+0000","last_became_peered":"2024-04-25T09:57:25.977229+0000","last_unstale":"2024-04-25T09:57:26.966465+0000","last_undegraded":"2024-04-25T09:57:26.966465+0000","last_fullsized":"2024-04-25T09:57:26.966465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:36.665923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.984063+0000","last_change":"2024-04-25T09:57:26.981663+0000","last_active":"2024-04-25T09:57:26.984063+0000","last_peered":"2024-04-25T09:57:26.984063+0000","last_clean":"2024-04-25T09:57:26.984063+0000","last_became_active":"2024-04-25T09:57:26.981372+0000","last_became_peered":"2024-04-25T09:57:26.981372+0000","last_unstale":"2024-04-25T09:57:26.984063+0000","last_undegraded":"2024-04-25T09:57:26.984063+0000","last_fullsized":"2024-04-25T09:57:26.984063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:40.708493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652098+0000","last_change":"2024-04-25T09:57:25.983160+0000","last_active":"2024-04-25T09:57:28.652098+0000","last_peered":"2024-04-25T09:57:28.652098+0000","last_clean":"2024-04-25T09:57:28.652098+0000","last_became_active":"2024-04-25T09:57:25.981166+0000","last_became_peered":"2024-04-25T09:57:25.981166+0000","last_unstale":"2024-04-25T09:57:28.652098+0000","last_undegraded":"2024-04-25T09:57:28.652098+0000","last_fullsized":"2024-04-25T09:57:28.652098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:08.287523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976408+0000","last_change":"2024-04-25T09:57:26.990059+0000","last_active":"2024-04-25T09:57:27.976408+0000","last_peered":"2024-04-25T09:57:27.976408+0000","last_clean":"2024-04-25T09:57:27.976408+0000","last_became_active":"2024-04-25T09:57:26.989765+0000","last_became_peered":"2024-04-25T09:57:26.989765+0000","last_unstale":"2024-04-25T09:57:27.976408+0000","last_undegraded":"2024-04-25T09:57:27.976408+0000","last_fullsized":"2024-04-25T09:57:27.976408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:42.956819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.968317+0000","last_change":"2024-04-25T09:57:25.982572+0000","last_active":"2024-04-25T09:57:26.968317+0000","last_peered":"2024-04-25T09:57:26.968317+0000","last_clean":"2024-04-25T09:57:26.968317+0000","last_became_active":"2024-04-25T09:57:25.982318+0000","last_became_peered":"2024-04-25T09:57:25.982318+0000","last_unstale":"2024-04-25T09:57:26.968317+0000","last_undegraded":"2024-04-25T09:57:26.968317+0000","last_fullsized":"2024-04-25T09:57:26.968317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.980939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976470+0000","last_change":"2024-04-25T09:57:26.974674+0000","last_active":"2024-04-25T09:57:27.976470+0000","last_peered":"2024-04-25T09:57:27.976470+0000","last_clean":"2024-04-25T09:57:27.976470+0000","last_became_active":"2024-04-25T09:57:26.974375+0000","last_became_peered":"2024-04-25T09:57:26.974375+0000","last_unstale":"2024-04-25T09:57:27.976470+0000","last_undegraded":"2024-04-25T09:57:27.976470+0000","last_fullsized":"2024-04-25T09:57:27.976470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:35.981102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.975981+0000","last_change":"2024-04-25T09:57:25.973572+0000","last_active":"2024-04-25T09:57:27.975981+0000","last_peered":"2024-04-25T09:57:27.975981+0000","last_clean":"2024-04-25T09:57:27.975981+0000","last_became_active":"2024-04-25T09:57:25.973124+0000","last_became_peered":"2024-04-25T09:57:25.973124+0000","last_unstale":"2024-04-25T09:57:27.975981+0000","last_undegraded":"2024-04-25T09:57:27.975981+0000","last_fullsized":"2024-04-25T09:57:27.975981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:39.704046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982584+0000","last_change":"2024-04-25T09:57:26.974838+0000","last_active":"2024-04-25T09:57:27.982584+0000","last_peered":"2024-04-25T09:57:27.982584+0000","last_clean":"2024-04-25T09:57:27.982584+0000","last_became_active":"2024-04-25T09:57:26.974580+0000","last_became_peered":"2024-04-25T09:57:26.974580+0000","last_unstale":"2024-04-25T09:57:27.982584+0000","last_undegraded":"2024-04-25T09:57:27.982584+0000","last_fullsized":"2024-04-25T09:57:27.982584+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:36.159789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762593+0000","last_change":"2024-04-25T09:57:25.979039+0000","last_active":"2024-04-25T09:57:28.762593+0000","last_peered":"2024-04-25T09:57:28.762593+0000","last_clean":"2024-04-25T09:57:28.762593+0000","last_became_active":"2024-04-25T09:57:25.978715+0000","last_became_peered":"2024-04-25T09:57:25.978715+0000","last_unstale":"2024-04-25T09:57:28.762593+0000","last_undegraded":"2024-04-25T09:57:28.762593+0000","last_fullsized":"2024-04-25T09:57:28.762593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:19.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":294,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982619+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:57:27.982619+0000","last_peered":"2024-04-25T09:57:27.982619+0000","last_clean":"2024-04-25T09:57:27.982619+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:57:27.982619+0000","last_undegraded":"2024-04-25T09:57:27.982619+0000","last_fullsized":"2024-04-25T09:57:27.982619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36900,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334940,"statfs":{"total":96636764160,"available":96598978560,"internally_reserved":0,"allocated":1589248,"data_stored":1049166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":453696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":458810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1011712,"data_stored":460492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335540,"statfs":{"total":96636764160,"available":96599592960,"internally_reserved":0,"allocated":983040,"data_stored":452666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335532,"statfs":{"total":96636764160,"available":96599584768,"internally_reserved":0,"allocated":991232,"data_stored":452826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":999424,"data_stored":458888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1572864,"data_stored":1042452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:57:30.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:57:30.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.707+0000 7f3c20607640 1 Processor -- start 2024-04-25T09:57:30.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.707+0000 7f3c20607640 1 -- start start 2024-04-25T09:57:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.708+0000 7f3c20607640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3c180d0170 0x7f3c180ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.708+0000 7f3c20607640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c180ce700 0x7f3c1806c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.708+0000 7f3c20607640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c180c8030 con 0x7f3c1815da70 2024-04-25T09:57:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.708+0000 7f3c20607640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c180ceb30 con 0x7f3c180d0170 2024-04-25T09:57:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.708+0000 7f3c20607640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180c8310 con 0x7f3c180ce700 2024-04-25T09:57:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.708+0000 7f3c1e3a3640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3c1815da70 0x7f3c180c7730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40396/0 (socket says 172.21.15.19:40396) 2024-04-25T09:57:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.708+0000 7f3c1e3a3640 1 -- 172.21.15.19:0/2551045608 learned_addr learned my addr 172.21.15.19:0/2551045608 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.708+0000 7f3c1dba2640 1 --2- 172.21.15.19:0/2551045608 >> v2:172.21.15.19:3300/0 conn(0x7f3c180d0170 0x7f3c180ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.708+0000 7f3c1eba4640 1 --2- 172.21.15.19:0/2551045608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c180ce700 0x7f3c1806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.708+0000 7f3c1dba2640 1 -- 172.21.15.19:0/2551045608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c180ce700 msgr2=0x7f3c1806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:30.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.709+0000 7f3c1dba2640 1 --2- 172.21.15.19:0/2551045608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c180ce700 0x7f3c1806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.709+0000 7f3c1dba2640 1 -- 172.21.15.19:0/2551045608 >> v1:172.21.15.19:6789/0 conn(0x7f3c1815da70 legacy=0x7f3c180c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.709+0000 7f3c1dba2640 1 -- 172.21.15.19:0/2551045608 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c180c85f0 con 0x7f3c180d0170 2024-04-25T09:57:30.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.709+0000 7f3c1eba4640 1 --2- 172.21.15.19:0/2551045608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c180ce700 0x7f3c1806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.709+0000 7f3c1dba2640 1 --2- 172.21.15.19:0/2551045608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c180d0170 0x7f3c180ce1c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0097a0 tx=0x7f3c0c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af68d7cd5abf4b05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.709+0000 7f3c1d3a1640 1 -- 172.21.15.19:0/2551045608 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c037070 con 0x7f3c180d0170 2024-04-25T09:57:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.709+0000 7f3c1d3a1640 1 -- 172.21.15.19:0/2551045608 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c032440 con 0x7f3c180d0170 2024-04-25T09:57:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.709+0000 7f3c1d3a1640 1 -- 172.21.15.19:0/2551045608 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c032810 con 0x7f3c180d0170 2024-04-25T09:57:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.710+0000 7f3c20607640 1 -- 172.21.15.19:0/2551045608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c180d0170 msgr2=0x7f3c180ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.710+0000 7f3c20607640 1 --2- 172.21.15.19:0/2551045608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c180d0170 0x7f3c180ce1c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0097a0 tx=0x7f3c0c02a180 comp rx=0 tx=0).stop 2024-04-25T09:57:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.710+0000 7f3c20607640 1 -- 172.21.15.19:0/2551045608 shutdown_connections 2024-04-25T09:57:30.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.710+0000 7f3c20607640 1 --2- 172.21.15.19:0/2551045608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c180ce700 0x7f3c1806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.710+0000 7f3c20607640 1 --2- 172.21.15.19:0/2551045608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c180d0170 0x7f3c180ce1c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.710+0000 7f3c20607640 1 -- 172.21.15.19:0/2551045608 >> 172.21.15.19:0/2551045608 conn(0x7f3c180b33e0 msgr2=0x7f3c180b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.710+0000 7f3c20607640 1 -- 172.21.15.19:0/2551045608 shutdown_connections 2024-04-25T09:57:30.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.710+0000 7f3c20607640 1 -- 172.21.15.19:0/2551045608 wait complete. 2024-04-25T09:57:30.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.711+0000 7f3c20607640 1 Processor -- start 2024-04-25T09:57:30.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.711+0000 7f3c20607640 1 -- start start 2024-04-25T09:57:30.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c20607640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c180ce700 0x7f3c180ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c20607640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c180d0170 0x7f3c18148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c20607640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1815da70 0x7f3c180c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c20607640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1815de50 con 0x7f3c1815da70 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c20607640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1806be10 con 0x7f3c180ce700 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c20607640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1806c870 con 0x7f3c180d0170 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c1eba4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1815da70 0x7f3c180c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c1eba4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1815da70 0x7f3c180c8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53700/0 (socket says 172.21.15.19:53700) 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c1eba4640 1 -- 172.21.15.19:0/1121462906 learned_addr learned my addr 172.21.15.19:0/1121462906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c1e3a3640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c180ce700 0x7f3c180ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c1dba2640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c180d0170 0x7f3c18148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c1eba4640 1 -- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c180d0170 msgr2=0x7f3c18148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.712+0000 7f3c1eba4640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c180d0170 0x7f3c18148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.713+0000 7f3c1eba4640 1 -- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c180ce700 msgr2=0x7f3c180ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.713+0000 7f3c1eba4640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c180ce700 0x7f3c180ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.713+0000 7f3c1eba4640 1 -- 172.21.15.19:0/1121462906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c180ca8b0 con 0x7f3c1815da70 2024-04-25T09:57:30.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.713+0000 7f3c1eba4640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1815da70 0x7f3c180c8880 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009fd0 tx=0x7f3c0800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.713+0000 7f3c16ffd640 1 -- 172.21.15.19:0/1121462906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0801a070 con 0x7f3c1815da70 2024-04-25T09:57:30.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.713+0000 7f3c16ffd640 1 -- 172.21.15.19:0/1121462906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c08003bb0 con 0x7f3c1815da70 2024-04-25T09:57:30.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.713+0000 7f3c20607640 1 -- 172.21.15.19:0/1121462906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c180c92d0 con 0x7f3c1815da70 2024-04-25T09:57:30.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.713+0000 7f3c16ffd640 1 -- 172.21.15.19:0/1121462906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08004030 con 0x7f3c1815da70 2024-04-25T09:57:30.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.713+0000 7f3c20607640 1 -- 172.21.15.19:0/1121462906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c180c9890 con 0x7f3c1815da70 2024-04-25T09:57:30.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.714+0000 7f3c20607640 1 -- 172.21.15.19:0/1121462906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c18073990 con 0x7f3c1815da70 2024-04-25T09:57:30.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.714+0000 7f3c16ffd640 1 -- 172.21.15.19:0/1121462906 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c0801d070 con 0x7f3c1815da70 2024-04-25T09:57:30.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.715+0000 7f3c16ffd640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3c0406b4d0 0x7f3c0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.715+0000 7f3c16ffd640 1 -- 172.21.15.19:0/1121462906 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3c0808a5e0 con 0x7f3c1815da70 2024-04-25T09:57:30.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.715+0000 7f3c1e3a3640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3c0406b4d0 0x7f3c0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.715+0000 7f3c1e3a3640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3c0406b4d0 0x7f3c0406d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3c10000f60 tx=0x7f3c10007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.719+0000 7f3c16ffd640 1 -- 172.21.15.19:0/1121462906 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c08010070 con 0x7f3c1815da70 2024-04-25T09:57:30.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.899+0000 7f3c20607640 1 -- 172.21.15.19:0/1121462906 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c180008d0 con 0x7f3c0406b4d0 2024-04-25T09:57:30.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.906+0000 7f3c16ffd640 1 -- 172.21.15.19:0/1121462906 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252064 (secure 0 0 0) 0x7f3c180008d0 con 0x7f3c0406b4d0 2024-04-25T09:57:30.911 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:57:30.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.908+0000 7f3c20607640 1 -- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3c0406b4d0 msgr2=0x7f3c0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.908+0000 7f3c20607640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3c0406b4d0 0x7f3c0406d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3c10000f60 tx=0x7f3c10007e80 comp rx=0 tx=0).stop 2024-04-25T09:57:30.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.908+0000 7f3c20607640 1 -- 172.21.15.19:0/1121462906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1815da70 msgr2=0x7f3c180c8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.908+0000 7f3c20607640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1815da70 0x7f3c180c8880 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009fd0 tx=0x7f3c0800d560 comp rx=0 tx=0).stop 2024-04-25T09:57:30.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.908+0000 7f3c20607640 1 -- 172.21.15.19:0/1121462906 shutdown_connections 2024-04-25T09:57:30.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.908+0000 7f3c20607640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3c0406b4d0 0x7f3c0406d990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.908+0000 7f3c20607640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1815da70 0x7f3c180c8880 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.908+0000 7f3c20607640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3c180d0170 0x7f3c18148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.908+0000 7f3c20607640 1 --2- 172.21.15.19:0/1121462906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c180ce700 0x7f3c180ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.908+0000 7f3c20607640 1 -- 172.21.15.19:0/1121462906 >> 172.21.15.19:0/1121462906 conn(0x7f3c180b33e0 msgr2=0x7f3c180c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:30.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.909+0000 7f3c20607640 1 -- 172.21.15.19:0/1121462906 shutdown_connections 2024-04-25T09:57:30.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:30.909+0000 7f3c20607640 1 -- 172.21.15.19:0/1121462906 wait complete. 2024-04-25T09:57:30.923 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-25T09:57:28.973197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":291652,"kb_used_data":8920,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754683068,"statfs":{"total":773094113280,"available":772795461632,"internally_reserved":0,"allocated":9134080,"data_stored":4828996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023261"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982420+0000","last_change":"2024-04-25T09:57:26.982359+0000","last_active":"2024-04-25T09:57:27.982420+0000","last_peered":"2024-04-25T09:57:27.982420+0000","last_clean":"2024-04-25T09:57:27.982420+0000","last_became_active":"2024-04-25T09:57:26.981833+0000","last_became_peered":"2024-04-25T09:57:26.981833+0000","last_unstale":"2024-04-25T09:57:27.982420+0000","last_undegraded":"2024-04-25T09:57:27.982420+0000","last_fullsized":"2024-04-25T09:57:27.982420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:35.184611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177015+0000","last_change":"2024-04-25T09:57:26.982517+0000","last_active":"2024-04-25T09:57:28.177015+0000","last_peered":"2024-04-25T09:57:28.177015+0000","last_clean":"2024-04-25T09:57:28.177015+0000","last_became_active":"2024-04-25T09:57:26.982119+0000","last_became_peered":"2024-04-25T09:57:26.982119+0000","last_unstale":"2024-04-25T09:57:28.177015+0000","last_undegraded":"2024-04-25T09:57:28.177015+0000","last_fullsized":"2024-04-25T09:57:28.177015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:39.635629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976350+0000","last_change":"2024-04-25T09:57:26.990420+0000","last_active":"2024-04-25T09:57:27.976350+0000","last_peered":"2024-04-25T09:57:27.976350+0000","last_clean":"2024-04-25T09:57:27.976350+0000","last_became_active":"2024-04-25T09:57:26.990216+0000","last_became_peered":"2024-04-25T09:57:26.990216+0000","last_unstale":"2024-04-25T09:57:27.976350+0000","last_undegraded":"2024-04-25T09:57:27.976350+0000","last_fullsized":"2024-04-25T09:57:27.976350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:03.128863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084206+0000","last_change":"2024-04-25T09:57:26.983824+0000","last_active":"2024-04-25T09:57:28.084206+0000","last_peered":"2024-04-25T09:57:28.084206+0000","last_clean":"2024-04-25T09:57:28.084206+0000","last_became_active":"2024-04-25T09:57:26.983564+0000","last_became_peered":"2024-04-25T09:57:26.983564+0000","last_unstale":"2024-04-25T09:57:28.084206+0000","last_undegraded":"2024-04-25T09:57:28.084206+0000","last_fullsized":"2024-04-25T09:57:28.084206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:47.385592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652104+0000","last_change":"2024-04-25T09:57:26.977747+0000","last_active":"2024-04-25T09:57:28.652104+0000","last_peered":"2024-04-25T09:57:28.652104+0000","last_clean":"2024-04-25T09:57:28.652104+0000","last_became_active":"2024-04-25T09:57:26.977403+0000","last_became_peered":"2024-04-25T09:57:26.977403+0000","last_unstale":"2024-04-25T09:57:28.652104+0000","last_undegraded":"2024-04-25T09:57:28.652104+0000","last_fullsized":"2024-04-25T09:57:28.652104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:35.838934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762475+0000","last_change":"2024-04-25T09:57:26.986370+0000","last_active":"2024-04-25T09:57:28.762475+0000","last_peered":"2024-04-25T09:57:28.762475+0000","last_clean":"2024-04-25T09:57:28.762475+0000","last_became_active":"2024-04-25T09:57:26.986114+0000","last_became_peered":"2024-04-25T09:57:26.986114+0000","last_unstale":"2024-04-25T09:57:28.762475+0000","last_undegraded":"2024-04-25T09:57:28.762475+0000","last_fullsized":"2024-04-25T09:57:28.762475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:05.397991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.984287+0000","last_change":"2024-04-25T09:57:26.982140+0000","last_active":"2024-04-25T09:57:26.984287+0000","last_peered":"2024-04-25T09:57:26.984287+0000","last_clean":"2024-04-25T09:57:26.984287+0000","last_became_active":"2024-04-25T09:57:26.980851+0000","last_became_peered":"2024-04-25T09:57:26.980851+0000","last_unstale":"2024-04-25T09:57:26.984287+0000","last_undegraded":"2024-04-25T09:57:26.984287+0000","last_fullsized":"2024-04-25T09:57:26.984287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:20.615201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.651971+0000","last_change":"2024-04-25T09:57:26.976091+0000","last_active":"2024-04-25T09:57:28.651971+0000","last_peered":"2024-04-25T09:57:28.651971+0000","last_clean":"2024-04-25T09:57:28.651971+0000","last_became_active":"2024-04-25T09:57:26.975800+0000","last_became_peered":"2024-04-25T09:57:26.975800+0000","last_unstale":"2024-04-25T09:57:28.651971+0000","last_undegraded":"2024-04-25T09:57:28.651971+0000","last_fullsized":"2024-04-25T09:57:28.651971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:44.647080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983224+0000","last_change":"2024-04-25T09:57:26.975169+0000","last_active":"2024-04-25T09:57:27.983224+0000","last_peered":"2024-04-25T09:57:27.983224+0000","last_clean":"2024-04-25T09:57:27.983224+0000","last_became_active":"2024-04-25T09:57:26.974904+0000","last_became_peered":"2024-04-25T09:57:26.974904+0000","last_unstale":"2024-04-25T09:57:27.983224+0000","last_undegraded":"2024-04-25T09:57:27.983224+0000","last_fullsized":"2024-04-25T09:57:27.983224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:37.093098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084182+0000","last_change":"2024-04-25T09:57:26.981689+0000","last_active":"2024-04-25T09:57:28.084182+0000","last_peered":"2024-04-25T09:57:28.084182+0000","last_clean":"2024-04-25T09:57:28.084182+0000","last_became_active":"2024-04-25T09:57:26.981410+0000","last_became_peered":"2024-04-25T09:57:26.981410+0000","last_unstale":"2024-04-25T09:57:28.084182+0000","last_undegraded":"2024-04-25T09:57:28.084182+0000","last_fullsized":"2024-04-25T09:57:28.084182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:54.451766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176826+0000","last_change":"2024-04-25T09:57:26.995424+0000","last_active":"2024-04-25T09:57:28.176826+0000","last_peered":"2024-04-25T09:57:28.176826+0000","last_clean":"2024-04-25T09:57:28.176826+0000","last_became_active":"2024-04-25T09:57:26.995143+0000","last_became_peered":"2024-04-25T09:57:26.995143+0000","last_unstale":"2024-04-25T09:57:28.176826+0000","last_undegraded":"2024-04-25T09:57:28.176826+0000","last_fullsized":"2024-04-25T09:57:28.176826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:07.019676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084232+0000","last_change":"2024-04-25T09:57:26.989346+0000","last_active":"2024-04-25T09:57:28.084232+0000","last_peered":"2024-04-25T09:57:28.084232+0000","last_clean":"2024-04-25T09:57:28.084232+0000","last_became_active":"2024-04-25T09:57:26.989063+0000","last_became_peered":"2024-04-25T09:57:26.989063+0000","last_unstale":"2024-04-25T09:57:28.084232+0000","last_undegraded":"2024-04-25T09:57:28.084232+0000","last_fullsized":"2024-04-25T09:57:28.084232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:14.824503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652292+0000","last_change":"2024-04-25T09:57:26.974876+0000","last_active":"2024-04-25T09:57:28.652292+0000","last_peered":"2024-04-25T09:57:28.652292+0000","last_clean":"2024-04-25T09:57:28.652292+0000","last_became_active":"2024-04-25T09:57:26.974577+0000","last_became_peered":"2024-04-25T09:57:26.974577+0000","last_unstale":"2024-04-25T09:57:28.652292+0000","last_undegraded":"2024-04-25T09:57:28.652292+0000","last_fullsized":"2024-04-25T09:57:28.652292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:21.257063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762504+0000","last_change":"2024-04-25T09:57:26.985432+0000","last_active":"2024-04-25T09:57:28.762504+0000","last_peered":"2024-04-25T09:57:28.762504+0000","last_clean":"2024-04-25T09:57:28.762504+0000","last_became_active":"2024-04-25T09:57:26.985167+0000","last_became_peered":"2024-04-25T09:57:26.985167+0000","last_unstale":"2024-04-25T09:57:28.762504+0000","last_undegraded":"2024-04-25T09:57:28.762504+0000","last_fullsized":"2024-04-25T09:57:28.762504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:35.184611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176794+0000","last_change":"2024-04-25T09:57:26.981564+0000","last_active":"2024-04-25T09:57:28.176794+0000","last_peered":"2024-04-25T09:57:28.176794+0000","last_clean":"2024-04-25T09:57:28.176794+0000","last_became_active":"2024-04-25T09:57:26.981267+0000","last_became_peered":"2024-04-25T09:57:26.981267+0000","last_unstale":"2024-04-25T09:57:28.176794+0000","last_undegraded":"2024-04-25T09:57:28.176794+0000","last_fullsized":"2024-04-25T09:57:28.176794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:27.935292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982551+0000","last_change":"2024-04-25T09:57:26.971445+0000","last_active":"2024-04-25T09:57:27.982551+0000","last_peered":"2024-04-25T09:57:27.982551+0000","last_clean":"2024-04-25T09:57:27.982551+0000","last_became_active":"2024-04-25T09:57:26.971037+0000","last_became_peered":"2024-04-25T09:57:26.971037+0000","last_unstale":"2024-04-25T09:57:27.982551+0000","last_undegraded":"2024-04-25T09:57:27.982551+0000","last_fullsized":"2024-04-25T09:57:27.982551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:31.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652366+0000","last_change":"2024-04-25T09:57:26.979651+0000","last_active":"2024-04-25T09:57:28.652366+0000","last_peered":"2024-04-25T09:57:28.652366+0000","last_clean":"2024-04-25T09:57:28.652366+0000","last_became_active":"2024-04-25T09:57:26.979363+0000","last_became_peered":"2024-04-25T09:57:26.979363+0000","last_unstale":"2024-04-25T09:57:28.652366+0000","last_undegraded":"2024-04-25T09:57:28.652366+0000","last_fullsized":"2024-04-25T09:57:28.652366+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:52.932708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976404+0000","last_change":"2024-04-25T09:57:26.979642+0000","last_active":"2024-04-25T09:57:27.976404+0000","last_peered":"2024-04-25T09:57:27.976404+0000","last_clean":"2024-04-25T09:57:27.976404+0000","last_became_active":"2024-04-25T09:57:26.979277+0000","last_became_peered":"2024-04-25T09:57:26.979277+0000","last_unstale":"2024-04-25T09:57:27.976404+0000","last_undegraded":"2024-04-25T09:57:27.976404+0000","last_fullsized":"2024-04-25T09:57:27.976404+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:06.092784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762788+0000","last_change":"2024-04-25T09:57:26.976315+0000","last_active":"2024-04-25T09:57:28.762788+0000","last_peered":"2024-04-25T09:57:28.762788+0000","last_clean":"2024-04-25T09:57:28.762788+0000","last_became_active":"2024-04-25T09:57:26.976054+0000","last_became_peered":"2024-04-25T09:57:26.976054+0000","last_unstale":"2024-04-25T09:57:28.762788+0000","last_undegraded":"2024-04-25T09:57:28.762788+0000","last_fullsized":"2024-04-25T09:57:28.762788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:36.159789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652292+0000","last_change":"2024-04-25T09:57:26.977707+0000","last_active":"2024-04-25T09:57:28.652292+0000","last_peered":"2024-04-25T09:57:28.652292+0000","last_clean":"2024-04-25T09:57:28.652292+0000","last_became_active":"2024-04-25T09:57:26.977328+0000","last_became_peered":"2024-04-25T09:57:26.977328+0000","last_unstale":"2024-04-25T09:57:28.652292+0000","last_undegraded":"2024-04-25T09:57:28.652292+0000","last_fullsized":"2024-04-25T09:57:28.652292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:45.163182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982463+0000","last_change":"2024-04-25T09:57:26.972540+0000","last_active":"2024-04-25T09:57:27.982463+0000","last_peered":"2024-04-25T09:57:27.982463+0000","last_clean":"2024-04-25T09:57:27.982463+0000","last_became_active":"2024-04-25T09:57:26.972285+0000","last_became_peered":"2024-04-25T09:57:26.972285+0000","last_unstale":"2024-04-25T09:57:27.982463+0000","last_undegraded":"2024-04-25T09:57:27.982463+0000","last_fullsized":"2024-04-25T09:57:27.982463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:21.494809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652331+0000","last_change":"2024-04-25T09:57:26.980937+0000","last_active":"2024-04-25T09:57:28.652331+0000","last_peered":"2024-04-25T09:57:28.652331+0000","last_clean":"2024-04-25T09:57:28.652331+0000","last_became_active":"2024-04-25T09:57:26.980655+0000","last_became_peered":"2024-04-25T09:57:26.980655+0000","last_unstale":"2024-04-25T09:57:28.652331+0000","last_undegraded":"2024-04-25T09:57:28.652331+0000","last_fullsized":"2024-04-25T09:57:28.652331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:22.787123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982244+0000","last_change":"2024-04-25T09:57:26.970723+0000","last_active":"2024-04-25T09:57:27.982244+0000","last_peered":"2024-04-25T09:57:27.982244+0000","last_clean":"2024-04-25T09:57:27.982244+0000","last_became_active":"2024-04-25T09:57:26.970451+0000","last_became_peered":"2024-04-25T09:57:26.970451+0000","last_unstale":"2024-04-25T09:57:27.982244+0000","last_undegraded":"2024-04-25T09:57:27.982244+0000","last_fullsized":"2024-04-25T09:57:27.982244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:58.324486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177158+0000","last_change":"2024-04-25T09:57:26.985571+0000","last_active":"2024-04-25T09:57:28.177158+0000","last_peered":"2024-04-25T09:57:28.177158+0000","last_clean":"2024-04-25T09:57:28.177158+0000","last_became_active":"2024-04-25T09:57:26.985303+0000","last_became_peered":"2024-04-25T09:57:26.985303+0000","last_unstale":"2024-04-25T09:57:28.177158+0000","last_undegraded":"2024-04-25T09:57:28.177158+0000","last_fullsized":"2024-04-25T09:57:28.177158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:57.219604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177013+0000","last_change":"2024-04-25T09:57:26.981675+0000","last_active":"2024-04-25T09:57:28.177013+0000","last_peered":"2024-04-25T09:57:28.177013+0000","last_clean":"2024-04-25T09:57:28.177013+0000","last_became_active":"2024-04-25T09:57:26.981386+0000","last_became_peered":"2024-04-25T09:57:26.981386+0000","last_unstale":"2024-04-25T09:57:28.177013+0000","last_undegraded":"2024-04-25T09:57:28.177013+0000","last_fullsized":"2024-04-25T09:57:28.177013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:20.012844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084244+0000","last_change":"2024-04-25T09:57:26.989888+0000","last_active":"2024-04-25T09:57:28.084244+0000","last_peered":"2024-04-25T09:57:28.084244+0000","last_clean":"2024-04-25T09:57:28.084244+0000","last_became_active":"2024-04-25T09:57:26.989273+0000","last_became_peered":"2024-04-25T09:57:26.989273+0000","last_unstale":"2024-04-25T09:57:28.084244+0000","last_undegraded":"2024-04-25T09:57:28.084244+0000","last_fullsized":"2024-04-25T09:57:28.084244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:09.210740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982392+0000","last_change":"2024-04-25T09:57:26.971302+0000","last_active":"2024-04-25T09:57:27.982392+0000","last_peered":"2024-04-25T09:57:27.982392+0000","last_clean":"2024-04-25T09:57:27.982392+0000","last_became_active":"2024-04-25T09:57:26.971015+0000","last_became_peered":"2024-04-25T09:57:26.971015+0000","last_unstale":"2024-04-25T09:57:27.982392+0000","last_undegraded":"2024-04-25T09:57:27.982392+0000","last_fullsized":"2024-04-25T09:57:27.982392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.273632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652215+0000","last_change":"2024-04-25T09:57:26.985183+0000","last_active":"2024-04-25T09:57:28.652215+0000","last_peered":"2024-04-25T09:57:28.652215+0000","last_clean":"2024-04-25T09:57:28.652215+0000","last_became_active":"2024-04-25T09:57:26.984888+0000","last_became_peered":"2024-04-25T09:57:26.984888+0000","last_unstale":"2024-04-25T09:57:28.652215+0000","last_undegraded":"2024-04-25T09:57:28.652215+0000","last_fullsized":"2024-04-25T09:57:28.652215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:51.458960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176722+0000","last_change":"2024-04-25T09:57:26.992227+0000","last_active":"2024-04-25T09:57:28.176722+0000","last_peered":"2024-04-25T09:57:28.176722+0000","last_clean":"2024-04-25T09:57:28.176722+0000","last_became_active":"2024-04-25T09:57:26.991944+0000","last_became_peered":"2024-04-25T09:57:26.991944+0000","last_unstale":"2024-04-25T09:57:28.176722+0000","last_undegraded":"2024-04-25T09:57:28.176722+0000","last_fullsized":"2024-04-25T09:57:28.176722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:27.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982024+0000","last_change":"2024-04-25T09:57:26.989327+0000","last_active":"2024-04-25T09:57:27.982024+0000","last_peered":"2024-04-25T09:57:27.982024+0000","last_clean":"2024-04-25T09:57:27.982024+0000","last_became_active":"2024-04-25T09:57:26.989033+0000","last_became_peered":"2024-04-25T09:57:26.989033+0000","last_unstale":"2024-04-25T09:57:27.982024+0000","last_undegraded":"2024-04-25T09:57:27.982024+0000","last_fullsized":"2024-04-25T09:57:27.982024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:05.397991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976110+0000","last_change":"2024-04-25T09:57:26.978153+0000","last_active":"2024-04-25T09:57:27.976110+0000","last_peered":"2024-04-25T09:57:27.976110+0000","last_clean":"2024-04-25T09:57:27.976110+0000","last_became_active":"2024-04-25T09:57:26.977787+0000","last_became_peered":"2024-04-25T09:57:26.977787+0000","last_unstale":"2024-04-25T09:57:27.976110+0000","last_undegraded":"2024-04-25T09:57:27.976110+0000","last_fullsized":"2024-04-25T09:57:27.976110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:21.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982619+0000","last_change":"2024-04-25T09:57:26.982195+0000","last_active":"2024-04-25T09:57:27.982619+0000","last_peered":"2024-04-25T09:57:27.982619+0000","last_clean":"2024-04-25T09:57:27.982619+0000","last_became_active":"2024-04-25T09:57:26.981889+0000","last_became_peered":"2024-04-25T09:57:26.981889+0000","last_unstale":"2024-04-25T09:57:27.982619+0000","last_undegraded":"2024-04-25T09:57:27.982619+0000","last_fullsized":"2024-04-25T09:57:27.982619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:29.840350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976053+0000","last_change":"2024-04-25T09:57:26.974168+0000","last_active":"2024-04-25T09:57:27.976053+0000","last_peered":"2024-04-25T09:57:27.976053+0000","last_clean":"2024-04-25T09:57:27.976053+0000","last_became_active":"2024-04-25T09:57:26.973823+0000","last_became_peered":"2024-04-25T09:57:26.973823+0000","last_unstale":"2024-04-25T09:57:27.976053+0000","last_undegraded":"2024-04-25T09:57:27.976053+0000","last_fullsized":"2024-04-25T09:57:27.976053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:20.605277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652462+0000","last_change":"2024-04-25T09:57:25.980755+0000","last_active":"2024-04-25T09:57:28.652462+0000","last_peered":"2024-04-25T09:57:28.652462+0000","last_clean":"2024-04-25T09:57:28.652462+0000","last_became_active":"2024-04-25T09:57:25.978318+0000","last_became_peered":"2024-04-25T09:57:25.978318+0000","last_unstale":"2024-04-25T09:57:28.652462+0000","last_undegraded":"2024-04-25T09:57:28.652462+0000","last_fullsized":"2024-04-25T09:57:28.652462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:13.731726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176847+0000","last_change":"2024-04-25T09:57:26.989710+0000","last_active":"2024-04-25T09:57:28.176847+0000","last_peered":"2024-04-25T09:57:28.176847+0000","last_clean":"2024-04-25T09:57:28.176847+0000","last_became_active":"2024-04-25T09:57:26.989031+0000","last_became_peered":"2024-04-25T09:57:26.989031+0000","last_unstale":"2024-04-25T09:57:28.176847+0000","last_undegraded":"2024-04-25T09:57:28.176847+0000","last_fullsized":"2024-04-25T09:57:28.176847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:44.942613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982383+0000","last_change":"2024-04-25T09:57:25.978427+0000","last_active":"2024-04-25T09:57:27.982383+0000","last_peered":"2024-04-25T09:57:27.982383+0000","last_clean":"2024-04-25T09:57:27.982383+0000","last_became_active":"2024-04-25T09:57:25.978154+0000","last_became_peered":"2024-04-25T09:57:25.978154+0000","last_unstale":"2024-04-25T09:57:27.982383+0000","last_undegraded":"2024-04-25T09:57:27.982383+0000","last_fullsized":"2024-04-25T09:57:27.982383+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:35.642932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983017+0000","last_change":"2024-04-25T09:57:26.982132+0000","last_active":"2024-04-25T09:57:27.983017+0000","last_peered":"2024-04-25T09:57:27.983017+0000","last_clean":"2024-04-25T09:57:27.983017+0000","last_became_active":"2024-04-25T09:57:26.981880+0000","last_became_peered":"2024-04-25T09:57:26.981880+0000","last_unstale":"2024-04-25T09:57:27.983017+0000","last_undegraded":"2024-04-25T09:57:27.983017+0000","last_fullsized":"2024-04-25T09:57:27.983017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:53.632031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982381+0000","last_change":"2024-04-25T09:57:25.977765+0000","last_active":"2024-04-25T09:57:27.982381+0000","last_peered":"2024-04-25T09:57:27.982381+0000","last_clean":"2024-04-25T09:57:27.982381+0000","last_became_active":"2024-04-25T09:57:25.977508+0000","last_became_peered":"2024-04-25T09:57:25.977508+0000","last_unstale":"2024-04-25T09:57:27.982381+0000","last_undegraded":"2024-04-25T09:57:27.982381+0000","last_fullsized":"2024-04-25T09:57:27.982381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.137562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982591+0000","last_change":"2024-04-25T09:57:26.971424+0000","last_active":"2024-04-25T09:57:27.982591+0000","last_peered":"2024-04-25T09:57:27.982591+0000","last_clean":"2024-04-25T09:57:27.982591+0000","last_became_active":"2024-04-25T09:57:26.971171+0000","last_became_peered":"2024-04-25T09:57:26.971171+0000","last_unstale":"2024-04-25T09:57:27.982591+0000","last_undegraded":"2024-04-25T09:57:27.982591+0000","last_fullsized":"2024-04-25T09:57:27.982591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:53.307213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.979253+0000","last_change":"2024-04-25T09:57:25.980034+0000","last_active":"2024-04-25T09:57:26.979253+0000","last_peered":"2024-04-25T09:57:26.979253+0000","last_clean":"2024-04-25T09:57:26.979253+0000","last_became_active":"2024-04-25T09:57:25.979582+0000","last_became_peered":"2024-04-25T09:57:25.979582+0000","last_unstale":"2024-04-25T09:57:26.979253+0000","last_undegraded":"2024-04-25T09:57:26.979253+0000","last_fullsized":"2024-04-25T09:57:26.979253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:08.325589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177123+0000","last_change":"2024-04-25T09:57:26.988577+0000","last_active":"2024-04-25T09:57:28.177123+0000","last_peered":"2024-04-25T09:57:28.177123+0000","last_clean":"2024-04-25T09:57:28.177123+0000","last_became_active":"2024-04-25T09:57:26.988286+0000","last_became_peered":"2024-04-25T09:57:26.988286+0000","last_unstale":"2024-04-25T09:57:28.177123+0000","last_undegraded":"2024-04-25T09:57:28.177123+0000","last_fullsized":"2024-04-25T09:57:28.177123+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:07.941153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982684+0000","last_change":"2024-04-25T09:57:25.973204+0000","last_active":"2024-04-25T09:57:27.982684+0000","last_peered":"2024-04-25T09:57:27.982684+0000","last_clean":"2024-04-25T09:57:27.982684+0000","last_became_active":"2024-04-25T09:57:25.972931+0000","last_became_peered":"2024-04-25T09:57:25.972931+0000","last_unstale":"2024-04-25T09:57:27.982684+0000","last_undegraded":"2024-04-25T09:57:27.982684+0000","last_fullsized":"2024-04-25T09:57:27.982684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:53.825752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176953+0000","last_change":"2024-04-25T09:57:26.982617+0000","last_active":"2024-04-25T09:57:28.176953+0000","last_peered":"2024-04-25T09:57:28.176953+0000","last_clean":"2024-04-25T09:57:28.176953+0000","last_became_active":"2024-04-25T09:57:26.982270+0000","last_became_peered":"2024-04-25T09:57:26.982270+0000","last_unstale":"2024-04-25T09:57:28.176953+0000","last_undegraded":"2024-04-25T09:57:28.176953+0000","last_fullsized":"2024-04-25T09:57:28.176953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:17.051240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.083946+0000","last_change":"2024-04-25T09:57:25.975563+0000","last_active":"2024-04-25T09:57:28.083946+0000","last_peered":"2024-04-25T09:57:28.083946+0000","last_clean":"2024-04-25T09:57:28.083946+0000","last_became_active":"2024-04-25T09:57:25.974734+0000","last_became_peered":"2024-04-25T09:57:25.974734+0000","last_unstale":"2024-04-25T09:57:28.083946+0000","last_undegraded":"2024-04-25T09:57:28.083946+0000","last_fullsized":"2024-04-25T09:57:28.083946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:47.529460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983186+0000","last_change":"2024-04-25T09:57:26.981481+0000","last_active":"2024-04-25T09:57:27.983186+0000","last_peered":"2024-04-25T09:57:27.983186+0000","last_clean":"2024-04-25T09:57:27.983186+0000","last_became_active":"2024-04-25T09:57:26.978900+0000","last_became_peered":"2024-04-25T09:57:26.978900+0000","last_unstale":"2024-04-25T09:57:27.983186+0000","last_undegraded":"2024-04-25T09:57:27.983186+0000","last_fullsized":"2024-04-25T09:57:27.983186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:50.144838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176911+0000","last_change":"2024-04-25T09:57:25.973998+0000","last_active":"2024-04-25T09:57:28.176911+0000","last_peered":"2024-04-25T09:57:28.176911+0000","last_clean":"2024-04-25T09:57:28.176911+0000","last_became_active":"2024-04-25T09:57:25.973464+0000","last_became_peered":"2024-04-25T09:57:25.973464+0000","last_unstale":"2024-04-25T09:57:28.176911+0000","last_undegraded":"2024-04-25T09:57:28.176911+0000","last_fullsized":"2024-04-25T09:57:28.176911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:57.902028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.980725+0000","last_change":"2024-04-25T09:57:26.979586+0000","last_active":"2024-04-25T09:57:26.980725+0000","last_peered":"2024-04-25T09:57:26.980725+0000","last_clean":"2024-04-25T09:57:26.980725+0000","last_became_active":"2024-04-25T09:57:26.979325+0000","last_became_peered":"2024-04-25T09:57:26.979325+0000","last_unstale":"2024-04-25T09:57:26.980725+0000","last_undegraded":"2024-04-25T09:57:26.980725+0000","last_fullsized":"2024-04-25T09:57:26.980725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:53.632031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652299+0000","last_change":"2024-04-25T09:57:25.980118+0000","last_active":"2024-04-25T09:57:28.652299+0000","last_peered":"2024-04-25T09:57:28.652299+0000","last_clean":"2024-04-25T09:57:28.652299+0000","last_became_active":"2024-04-25T09:57:25.979840+0000","last_became_peered":"2024-04-25T09:57:25.979840+0000","last_unstale":"2024-04-25T09:57:28.652299+0000","last_undegraded":"2024-04-25T09:57:28.652299+0000","last_fullsized":"2024-04-25T09:57:28.652299+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:01.083810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177291+0000","last_change":"2024-04-25T09:57:26.989278+0000","last_active":"2024-04-25T09:57:28.177291+0000","last_peered":"2024-04-25T09:57:28.177291+0000","last_clean":"2024-04-25T09:57:28.177291+0000","last_became_active":"2024-04-25T09:57:26.988983+0000","last_became_peered":"2024-04-25T09:57:26.988983+0000","last_unstale":"2024-04-25T09:57:28.177291+0000","last_undegraded":"2024-04-25T09:57:28.177291+0000","last_fullsized":"2024-04-25T09:57:28.177291+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:10.862261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982535+0000","last_change":"2024-04-25T09:57:25.977126+0000","last_active":"2024-04-25T09:57:27.982535+0000","last_peered":"2024-04-25T09:57:27.982535+0000","last_clean":"2024-04-25T09:57:27.982535+0000","last_became_active":"2024-04-25T09:57:25.976841+0000","last_became_peered":"2024-04-25T09:57:25.976841+0000","last_unstale":"2024-04-25T09:57:27.982535+0000","last_undegraded":"2024-04-25T09:57:27.982535+0000","last_fullsized":"2024-04-25T09:57:27.982535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:13.762594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084147+0000","last_change":"2024-04-25T09:57:26.982884+0000","last_active":"2024-04-25T09:57:28.084147+0000","last_peered":"2024-04-25T09:57:28.084147+0000","last_clean":"2024-04-25T09:57:28.084147+0000","last_became_active":"2024-04-25T09:57:26.982608+0000","last_became_peered":"2024-04-25T09:57:26.982608+0000","last_unstale":"2024-04-25T09:57:28.084147+0000","last_undegraded":"2024-04-25T09:57:28.084147+0000","last_fullsized":"2024-04-25T09:57:28.084147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:47.142009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084269+0000","last_change":"2024-04-25T09:57:25.977803+0000","last_active":"2024-04-25T09:57:28.084269+0000","last_peered":"2024-04-25T09:57:28.084269+0000","last_clean":"2024-04-25T09:57:28.084269+0000","last_became_active":"2024-04-25T09:57:25.977522+0000","last_became_peered":"2024-04-25T09:57:25.977522+0000","last_unstale":"2024-04-25T09:57:28.084269+0000","last_undegraded":"2024-04-25T09:57:28.084269+0000","last_fullsized":"2024-04-25T09:57:28.084269+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:16.527873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762724+0000","last_change":"2024-04-25T09:57:26.979198+0000","last_active":"2024-04-25T09:57:28.762724+0000","last_peered":"2024-04-25T09:57:28.762724+0000","last_clean":"2024-04-25T09:57:28.762724+0000","last_became_active":"2024-04-25T09:57:26.972993+0000","last_became_peered":"2024-04-25T09:57:26.972993+0000","last_unstale":"2024-04-25T09:57:28.762724+0000","last_undegraded":"2024-04-25T09:57:28.762724+0000","last_fullsized":"2024-04-25T09:57:28.762724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:31.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652526+0000","last_change":"2024-04-25T09:57:25.982931+0000","last_active":"2024-04-25T09:57:28.652526+0000","last_peered":"2024-04-25T09:57:28.652526+0000","last_clean":"2024-04-25T09:57:28.652526+0000","last_became_active":"2024-04-25T09:57:25.978341+0000","last_became_peered":"2024-04-25T09:57:25.978341+0000","last_unstale":"2024-04-25T09:57:28.652526+0000","last_undegraded":"2024-04-25T09:57:28.652526+0000","last_fullsized":"2024-04-25T09:57:28.652526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:41.218305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976240+0000","last_change":"2024-04-25T09:57:26.991164+0000","last_active":"2024-04-25T09:57:27.976240+0000","last_peered":"2024-04-25T09:57:27.976240+0000","last_clean":"2024-04-25T09:57:27.976240+0000","last_became_active":"2024-04-25T09:57:26.990863+0000","last_became_peered":"2024-04-25T09:57:26.990863+0000","last_unstale":"2024-04-25T09:57:27.976240+0000","last_undegraded":"2024-04-25T09:57:27.976240+0000","last_fullsized":"2024-04-25T09:57:27.976240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:17.725041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"80'6","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.990500+0000","last_change":"2024-04-25T09:57:25.980275+0000","last_active":"2024-04-25T09:57:26.990500+0000","last_peered":"2024-04-25T09:57:26.990500+0000","last_clean":"2024-04-25T09:57:26.990500+0000","last_became_active":"2024-04-25T09:57:25.979873+0000","last_became_peered":"2024-04-25T09:57:25.979873+0000","last_unstale":"2024-04-25T09:57:26.990500+0000","last_undegraded":"2024-04-25T09:57:26.990500+0000","last_fullsized":"2024-04-25T09:57:26.990500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:57.320039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084306+0000","last_change":"2024-04-25T09:57:26.976015+0000","last_active":"2024-04-25T09:57:28.084306+0000","last_peered":"2024-04-25T09:57:28.084306+0000","last_clean":"2024-04-25T09:57:28.084306+0000","last_became_active":"2024-04-25T09:57:26.975715+0000","last_became_peered":"2024-04-25T09:57:26.975715+0000","last_unstale":"2024-04-25T09:57:28.084306+0000","last_undegraded":"2024-04-25T09:57:28.084306+0000","last_fullsized":"2024-04-25T09:57:28.084306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:11.580893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.083975+0000","last_change":"2024-04-25T09:57:25.977037+0000","last_active":"2024-04-25T09:57:28.083975+0000","last_peered":"2024-04-25T09:57:28.083975+0000","last_clean":"2024-04-25T09:57:28.083975+0000","last_became_active":"2024-04-25T09:57:25.976742+0000","last_became_peered":"2024-04-25T09:57:25.976742+0000","last_unstale":"2024-04-25T09:57:28.083975+0000","last_undegraded":"2024-04-25T09:57:28.083975+0000","last_fullsized":"2024-04-25T09:57:28.083975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:33.969019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.977200+0000","last_change":"2024-04-25T09:57:26.973316+0000","last_active":"2024-04-25T09:57:26.977200+0000","last_peered":"2024-04-25T09:57:26.977200+0000","last_clean":"2024-04-25T09:57:26.977200+0000","last_became_active":"2024-04-25T09:57:26.973006+0000","last_became_peered":"2024-04-25T09:57:26.973006+0000","last_unstale":"2024-04-25T09:57:26.977200+0000","last_undegraded":"2024-04-25T09:57:26.977200+0000","last_fullsized":"2024-04-25T09:57:26.977200+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:21.494809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982238+0000","last_change":"2024-04-25T09:57:25.983301+0000","last_active":"2024-04-25T09:57:27.982238+0000","last_peered":"2024-04-25T09:57:27.982238+0000","last_clean":"2024-04-25T09:57:27.982238+0000","last_became_active":"2024-04-25T09:57:25.983059+0000","last_became_peered":"2024-04-25T09:57:25.983059+0000","last_unstale":"2024-04-25T09:57:27.982238+0000","last_undegraded":"2024-04-25T09:57:27.982238+0000","last_fullsized":"2024-04-25T09:57:27.982238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:19.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.984955+0000","last_change":"2024-04-25T09:57:26.983768+0000","last_active":"2024-04-25T09:57:26.984955+0000","last_peered":"2024-04-25T09:57:26.984955+0000","last_clean":"2024-04-25T09:57:26.984955+0000","last_became_active":"2024-04-25T09:57:26.983504+0000","last_became_peered":"2024-04-25T09:57:26.983504+0000","last_unstale":"2024-04-25T09:57:26.984955+0000","last_undegraded":"2024-04-25T09:57:26.984955+0000","last_fullsized":"2024-04-25T09:57:26.984955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:42.732984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652018+0000","last_change":"2024-04-25T09:57:25.980102+0000","last_active":"2024-04-25T09:57:28.652018+0000","last_peered":"2024-04-25T09:57:28.652018+0000","last_clean":"2024-04-25T09:57:28.652018+0000","last_became_active":"2024-04-25T09:57:25.979797+0000","last_became_peered":"2024-04-25T09:57:25.979797+0000","last_unstale":"2024-04-25T09:57:28.652018+0000","last_undegraded":"2024-04-25T09:57:28.652018+0000","last_fullsized":"2024-04-25T09:57:28.652018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:20.839016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.985052+0000","last_change":"2024-04-25T09:57:26.983708+0000","last_active":"2024-04-25T09:57:26.985052+0000","last_peered":"2024-04-25T09:57:26.985052+0000","last_clean":"2024-04-25T09:57:26.985052+0000","last_became_active":"2024-04-25T09:57:26.982989+0000","last_became_peered":"2024-04-25T09:57:26.982989+0000","last_unstale":"2024-04-25T09:57:26.985052+0000","last_undegraded":"2024-04-25T09:57:26.985052+0000","last_fullsized":"2024-04-25T09:57:26.985052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:19.766616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177261+0000","last_change":"2024-04-25T09:57:25.979176+0000","last_active":"2024-04-25T09:57:28.177261+0000","last_peered":"2024-04-25T09:57:28.177261+0000","last_clean":"2024-04-25T09:57:28.177261+0000","last_became_active":"2024-04-25T09:57:25.978881+0000","last_became_peered":"2024-04-25T09:57:25.978881+0000","last_unstale":"2024-04-25T09:57:28.177261+0000","last_undegraded":"2024-04-25T09:57:28.177261+0000","last_fullsized":"2024-04-25T09:57:28.177261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.137562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762276+0000","last_change":"2024-04-25T09:57:26.980663+0000","last_active":"2024-04-25T09:57:28.762276+0000","last_peered":"2024-04-25T09:57:28.762276+0000","last_clean":"2024-04-25T09:57:28.762276+0000","last_became_active":"2024-04-25T09:57:26.980389+0000","last_became_peered":"2024-04-25T09:57:26.980389+0000","last_unstale":"2024-04-25T09:57:28.762276+0000","last_undegraded":"2024-04-25T09:57:28.762276+0000","last_fullsized":"2024-04-25T09:57:28.762276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:20.354301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982656+0000","last_change":"2024-04-25T09:57:25.975386+0000","last_active":"2024-04-25T09:57:27.982656+0000","last_peered":"2024-04-25T09:57:27.982656+0000","last_clean":"2024-04-25T09:57:27.982656+0000","last_became_active":"2024-04-25T09:57:25.975112+0000","last_became_peered":"2024-04-25T09:57:25.975112+0000","last_unstale":"2024-04-25T09:57:27.982656+0000","last_undegraded":"2024-04-25T09:57:27.982656+0000","last_fullsized":"2024-04-25T09:57:27.982656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:57.902028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652278+0000","last_change":"2024-04-25T09:57:26.982146+0000","last_active":"2024-04-25T09:57:28.652278+0000","last_peered":"2024-04-25T09:57:28.652278+0000","last_clean":"2024-04-25T09:57:28.652278+0000","last_became_active":"2024-04-25T09:57:26.981876+0000","last_became_peered":"2024-04-25T09:57:26.981876+0000","last_unstale":"2024-04-25T09:57:28.652278+0000","last_undegraded":"2024-04-25T09:57:28.652278+0000","last_fullsized":"2024-04-25T09:57:28.652278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:09.926715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983202+0000","last_change":"2024-04-25T09:57:25.975455+0000","last_active":"2024-04-25T09:57:27.983202+0000","last_peered":"2024-04-25T09:57:27.983202+0000","last_clean":"2024-04-25T09:57:27.983202+0000","last_became_active":"2024-04-25T09:57:25.975169+0000","last_became_peered":"2024-04-25T09:57:25.975169+0000","last_unstale":"2024-04-25T09:57:27.983202+0000","last_undegraded":"2024-04-25T09:57:27.983202+0000","last_fullsized":"2024-04-25T09:57:27.983202+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:43.008000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652144+0000","last_change":"2024-04-25T09:57:26.990096+0000","last_active":"2024-04-25T09:57:28.652144+0000","last_peered":"2024-04-25T09:57:28.652144+0000","last_clean":"2024-04-25T09:57:28.652144+0000","last_became_active":"2024-04-25T09:57:26.989786+0000","last_became_peered":"2024-04-25T09:57:26.989786+0000","last_unstale":"2024-04-25T09:57:28.652144+0000","last_undegraded":"2024-04-25T09:57:28.652144+0000","last_fullsized":"2024-04-25T09:57:28.652144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:46.603634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652532+0000","last_change":"2024-04-25T09:57:25.981279+0000","last_active":"2024-04-25T09:57:28.652532+0000","last_peered":"2024-04-25T09:57:28.652532+0000","last_clean":"2024-04-25T09:57:28.652532+0000","last_became_active":"2024-04-25T09:57:25.980986+0000","last_became_peered":"2024-04-25T09:57:25.980986+0000","last_unstale":"2024-04-25T09:57:28.652532+0000","last_undegraded":"2024-04-25T09:57:28.652532+0000","last_fullsized":"2024-04-25T09:57:28.652532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:27.643209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762490+0000","last_change":"2024-04-25T09:57:26.974844+0000","last_active":"2024-04-25T09:57:28.762490+0000","last_peered":"2024-04-25T09:57:28.762490+0000","last_clean":"2024-04-25T09:57:28.762490+0000","last_became_active":"2024-04-25T09:57:26.974558+0000","last_became_peered":"2024-04-25T09:57:26.974558+0000","last_unstale":"2024-04-25T09:57:28.762490+0000","last_undegraded":"2024-04-25T09:57:28.762490+0000","last_fullsized":"2024-04-25T09:57:28.762490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:26.273632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976195+0000","last_change":"2024-04-25T09:57:25.982149+0000","last_active":"2024-04-25T09:57:27.976195+0000","last_peered":"2024-04-25T09:57:27.976195+0000","last_clean":"2024-04-25T09:57:27.976195+0000","last_became_active":"2024-04-25T09:57:25.981765+0000","last_became_peered":"2024-04-25T09:57:25.981765+0000","last_unstale":"2024-04-25T09:57:27.976195+0000","last_undegraded":"2024-04-25T09:57:27.976195+0000","last_fullsized":"2024-04-25T09:57:27.976195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:52.627584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176999+0000","last_change":"2024-04-25T09:57:26.992142+0000","last_active":"2024-04-25T09:57:28.176999+0000","last_peered":"2024-04-25T09:57:28.176999+0000","last_clean":"2024-04-25T09:57:28.176999+0000","last_became_active":"2024-04-25T09:57:26.991842+0000","last_became_peered":"2024-04-25T09:57:26.991842+0000","last_unstale":"2024-04-25T09:57:28.176999+0000","last_undegraded":"2024-04-25T09:57:28.176999+0000","last_fullsized":"2024-04-25T09:57:28.176999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:16.358371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762519+0000","last_change":"2024-04-25T09:57:25.976450+0000","last_active":"2024-04-25T09:57:28.762519+0000","last_peered":"2024-04-25T09:57:28.762519+0000","last_clean":"2024-04-25T09:57:28.762519+0000","last_became_active":"2024-04-25T09:57:25.976171+0000","last_became_peered":"2024-04-25T09:57:25.976171+0000","last_unstale":"2024-04-25T09:57:28.762519+0000","last_undegraded":"2024-04-25T09:57:28.762519+0000","last_fullsized":"2024-04-25T09:57:28.762519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:57.902028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177172+0000","last_change":"2024-04-25T09:57:26.993684+0000","last_active":"2024-04-25T09:57:28.177172+0000","last_peered":"2024-04-25T09:57:28.177172+0000","last_clean":"2024-04-25T09:57:28.177172+0000","last_became_active":"2024-04-25T09:57:26.993403+0000","last_became_peered":"2024-04-25T09:57:26.993403+0000","last_unstale":"2024-04-25T09:57:28.177172+0000","last_undegraded":"2024-04-25T09:57:28.177172+0000","last_fullsized":"2024-04-25T09:57:28.177172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:16.732299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762336+0000","last_change":"2024-04-25T09:57:25.978638+0000","last_active":"2024-04-25T09:57:28.762336+0000","last_peered":"2024-04-25T09:57:28.762336+0000","last_clean":"2024-04-25T09:57:28.762336+0000","last_became_active":"2024-04-25T09:57:25.978352+0000","last_became_peered":"2024-04-25T09:57:25.978352+0000","last_unstale":"2024-04-25T09:57:28.762336+0000","last_undegraded":"2024-04-25T09:57:28.762336+0000","last_fullsized":"2024-04-25T09:57:28.762336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:35.642932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.984892+0000","last_change":"2024-04-25T09:57:26.983315+0000","last_active":"2024-04-25T09:57:26.984892+0000","last_peered":"2024-04-25T09:57:26.984892+0000","last_clean":"2024-04-25T09:57:26.984892+0000","last_became_active":"2024-04-25T09:57:26.982625+0000","last_became_peered":"2024-04-25T09:57:26.982625+0000","last_unstale":"2024-04-25T09:57:26.984892+0000","last_undegraded":"2024-04-25T09:57:26.984892+0000","last_fullsized":"2024-04-25T09:57:26.984892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:37.093098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762308+0000","last_change":"2024-04-25T09:57:25.978782+0000","last_active":"2024-04-25T09:57:28.762308+0000","last_peered":"2024-04-25T09:57:28.762308+0000","last_clean":"2024-04-25T09:57:28.762308+0000","last_became_active":"2024-04-25T09:57:25.978421+0000","last_became_peered":"2024-04-25T09:57:25.978421+0000","last_unstale":"2024-04-25T09:57:28.762308+0000","last_undegraded":"2024-04-25T09:57:28.762308+0000","last_fullsized":"2024-04-25T09:57:28.762308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:13.762594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983196+0000","last_change":"2024-04-25T09:57:26.975675+0000","last_active":"2024-04-25T09:57:27.983196+0000","last_peered":"2024-04-25T09:57:27.983196+0000","last_clean":"2024-04-25T09:57:27.983196+0000","last_became_active":"2024-04-25T09:57:26.975400+0000","last_became_peered":"2024-04-25T09:57:26.975400+0000","last_unstale":"2024-04-25T09:57:27.983196+0000","last_undegraded":"2024-04-25T09:57:27.983196+0000","last_fullsized":"2024-04-25T09:57:27.983196+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:20.615201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.084161+0000","last_change":"2024-04-25T09:57:25.976538+0000","last_active":"2024-04-25T09:57:28.084161+0000","last_peered":"2024-04-25T09:57:28.084161+0000","last_clean":"2024-04-25T09:57:28.084161+0000","last_became_active":"2024-04-25T09:57:25.976243+0000","last_became_peered":"2024-04-25T09:57:25.976243+0000","last_unstale":"2024-04-25T09:57:28.084161+0000","last_undegraded":"2024-04-25T09:57:28.084161+0000","last_fullsized":"2024-04-25T09:57:28.084161+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:00.015322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.984264+0000","last_change":"2024-04-25T09:57:26.981900+0000","last_active":"2024-04-25T09:57:26.984264+0000","last_peered":"2024-04-25T09:57:26.984264+0000","last_clean":"2024-04-25T09:57:26.984264+0000","last_became_active":"2024-04-25T09:57:26.981568+0000","last_became_peered":"2024-04-25T09:57:26.981568+0000","last_unstale":"2024-04-25T09:57:26.984264+0000","last_undegraded":"2024-04-25T09:57:26.984264+0000","last_fullsized":"2024-04-25T09:57:26.984264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:50.144838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.177159+0000","last_change":"2024-04-25T09:57:25.984072+0000","last_active":"2024-04-25T09:57:28.177159+0000","last_peered":"2024-04-25T09:57:28.177159+0000","last_clean":"2024-04-25T09:57:28.177159+0000","last_became_active":"2024-04-25T09:57:25.983745+0000","last_became_peered":"2024-04-25T09:57:25.983745+0000","last_unstale":"2024-04-25T09:57:28.177159+0000","last_undegraded":"2024-04-25T09:57:28.177159+0000","last_fullsized":"2024-04-25T09:57:28.177159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:19.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.983178+0000","last_change":"2024-04-25T09:57:26.981513+0000","last_active":"2024-04-25T09:57:27.983178+0000","last_peered":"2024-04-25T09:57:27.983178+0000","last_clean":"2024-04-25T09:57:27.983178+0000","last_became_active":"2024-04-25T09:57:26.981214+0000","last_became_peered":"2024-04-25T09:57:26.981214+0000","last_unstale":"2024-04-25T09:57:27.983178+0000","last_undegraded":"2024-04-25T09:57:27.983178+0000","last_fullsized":"2024-04-25T09:57:27.983178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:40.708493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982713+0000","last_change":"2024-04-25T09:57:25.974949+0000","last_active":"2024-04-25T09:57:27.982713+0000","last_peered":"2024-04-25T09:57:27.982713+0000","last_clean":"2024-04-25T09:57:27.982713+0000","last_became_active":"2024-04-25T09:57:25.974625+0000","last_became_peered":"2024-04-25T09:57:25.974625+0000","last_unstale":"2024-04-25T09:57:27.982713+0000","last_undegraded":"2024-04-25T09:57:27.982713+0000","last_fullsized":"2024-04-25T09:57:27.982713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:53.447319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.176811+0000","last_change":"2024-04-25T09:57:26.988576+0000","last_active":"2024-04-25T09:57:28.176811+0000","last_peered":"2024-04-25T09:57:28.176811+0000","last_clean":"2024-04-25T09:57:28.176811+0000","last_became_active":"2024-04-25T09:57:26.988277+0000","last_became_peered":"2024-04-25T09:57:26.988277+0000","last_unstale":"2024-04-25T09:57:28.176811+0000","last_undegraded":"2024-04-25T09:57:28.176811+0000","last_fullsized":"2024-04-25T09:57:28.176811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:07.334443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982143+0000","last_change":"2024-04-25T09:57:25.977187+0000","last_active":"2024-04-25T09:57:27.982143+0000","last_peered":"2024-04-25T09:57:27.982143+0000","last_clean":"2024-04-25T09:57:27.982143+0000","last_became_active":"2024-04-25T09:57:25.976825+0000","last_became_peered":"2024-04-25T09:57:25.976825+0000","last_unstale":"2024-04-25T09:57:27.982143+0000","last_undegraded":"2024-04-25T09:57:27.982143+0000","last_fullsized":"2024-04-25T09:57:27.982143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:27.866179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.979830+0000","last_change":"2024-04-25T09:57:26.978261+0000","last_active":"2024-04-25T09:57:26.979830+0000","last_peered":"2024-04-25T09:57:26.979830+0000","last_clean":"2024-04-25T09:57:26.979830+0000","last_became_active":"2024-04-25T09:57:26.977918+0000","last_became_peered":"2024-04-25T09:57:26.977918+0000","last_unstale":"2024-04-25T09:57:26.979830+0000","last_undegraded":"2024-04-25T09:57:26.979830+0000","last_fullsized":"2024-04-25T09:57:26.979830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:53.307213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.966465+0000","last_change":"2024-04-25T09:57:25.977540+0000","last_active":"2024-04-25T09:57:26.966465+0000","last_peered":"2024-04-25T09:57:26.966465+0000","last_clean":"2024-04-25T09:57:26.966465+0000","last_became_active":"2024-04-25T09:57:25.977229+0000","last_became_peered":"2024-04-25T09:57:25.977229+0000","last_unstale":"2024-04-25T09:57:26.966465+0000","last_undegraded":"2024-04-25T09:57:26.966465+0000","last_fullsized":"2024-04-25T09:57:26.966465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:36.665923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.984063+0000","last_change":"2024-04-25T09:57:26.981663+0000","last_active":"2024-04-25T09:57:26.984063+0000","last_peered":"2024-04-25T09:57:26.984063+0000","last_clean":"2024-04-25T09:57:26.984063+0000","last_became_active":"2024-04-25T09:57:26.981372+0000","last_became_peered":"2024-04-25T09:57:26.981372+0000","last_unstale":"2024-04-25T09:57:26.984063+0000","last_undegraded":"2024-04-25T09:57:26.984063+0000","last_fullsized":"2024-04-25T09:57:26.984063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:40.708493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.652098+0000","last_change":"2024-04-25T09:57:25.983160+0000","last_active":"2024-04-25T09:57:28.652098+0000","last_peered":"2024-04-25T09:57:28.652098+0000","last_clean":"2024-04-25T09:57:28.652098+0000","last_became_active":"2024-04-25T09:57:25.981166+0000","last_became_peered":"2024-04-25T09:57:25.981166+0000","last_unstale":"2024-04-25T09:57:28.652098+0000","last_undegraded":"2024-04-25T09:57:28.652098+0000","last_fullsized":"2024-04-25T09:57:28.652098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:08.287523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976408+0000","last_change":"2024-04-25T09:57:26.990059+0000","last_active":"2024-04-25T09:57:27.976408+0000","last_peered":"2024-04-25T09:57:27.976408+0000","last_clean":"2024-04-25T09:57:27.976408+0000","last_became_active":"2024-04-25T09:57:26.989765+0000","last_became_peered":"2024-04-25T09:57:26.989765+0000","last_unstale":"2024-04-25T09:57:27.976408+0000","last_undegraded":"2024-04-25T09:57:27.976408+0000","last_fullsized":"2024-04-25T09:57:27.976408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:42.956819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T09:57:26.968317+0000","last_change":"2024-04-25T09:57:25.982572+0000","last_active":"2024-04-25T09:57:26.968317+0000","last_peered":"2024-04-25T09:57:26.968317+0000","last_clean":"2024-04-25T09:57:26.968317+0000","last_became_active":"2024-04-25T09:57:25.982318+0000","last_became_peered":"2024-04-25T09:57:25.982318+0000","last_unstale":"2024-04-25T09:57:26.968317+0000","last_undegraded":"2024-04-25T09:57:26.968317+0000","last_fullsized":"2024-04-25T09:57:26.968317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.980939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.976470+0000","last_change":"2024-04-25T09:57:26.974674+0000","last_active":"2024-04-25T09:57:27.976470+0000","last_peered":"2024-04-25T09:57:27.976470+0000","last_clean":"2024-04-25T09:57:27.976470+0000","last_became_active":"2024-04-25T09:57:26.974375+0000","last_became_peered":"2024-04-25T09:57:26.974375+0000","last_unstale":"2024-04-25T09:57:27.976470+0000","last_undegraded":"2024-04-25T09:57:27.976470+0000","last_fullsized":"2024-04-25T09:57:27.976470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:35.981102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.975981+0000","last_change":"2024-04-25T09:57:25.973572+0000","last_active":"2024-04-25T09:57:27.975981+0000","last_peered":"2024-04-25T09:57:27.975981+0000","last_clean":"2024-04-25T09:57:27.975981+0000","last_became_active":"2024-04-25T09:57:25.973124+0000","last_became_peered":"2024-04-25T09:57:25.973124+0000","last_unstale":"2024-04-25T09:57:27.975981+0000","last_undegraded":"2024-04-25T09:57:27.975981+0000","last_fullsized":"2024-04-25T09:57:27.975981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:39.704046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982584+0000","last_change":"2024-04-25T09:57:26.974838+0000","last_active":"2024-04-25T09:57:27.982584+0000","last_peered":"2024-04-25T09:57:27.982584+0000","last_clean":"2024-04-25T09:57:27.982584+0000","last_became_active":"2024-04-25T09:57:26.974580+0000","last_became_peered":"2024-04-25T09:57:26.974580+0000","last_unstale":"2024-04-25T09:57:27.982584+0000","last_undegraded":"2024-04-25T09:57:27.982584+0000","last_fullsized":"2024-04-25T09:57:27.982584+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:25.949518+0000","last_clean_scrub_stamp":"2024-04-25T09:57:25.949518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:36.159789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:28.762593+0000","last_change":"2024-04-25T09:57:25.979039+0000","last_active":"2024-04-25T09:57:28.762593+0000","last_peered":"2024-04-25T09:57:28.762593+0000","last_clean":"2024-04-25T09:57:28.762593+0000","last_became_active":"2024-04-25T09:57:25.978715+0000","last_became_peered":"2024-04-25T09:57:25.978715+0000","last_unstale":"2024-04-25T09:57:28.762593+0000","last_undegraded":"2024-04-25T09:57:28.762593+0000","last_fullsized":"2024-04-25T09:57:28.762593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:24.945071+0000","last_clean_scrub_stamp":"2024-04-25T09:57:24.945071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:19.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":294,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T09:57:27.982619+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:57:27.982619+0000","last_peered":"2024-04-25T09:57:27.982619+0000","last_clean":"2024-04-25T09:57:27.982619+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:57:27.982619+0000","last_undegraded":"2024-04-25T09:57:27.982619+0000","last_fullsized":"2024-04-25T09:57:27.982619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36900,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334940,"statfs":{"total":96636764160,"available":96598978560,"internally_reserved":0,"allocated":1589248,"data_stored":1049166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":453696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":995328,"data_stored":458810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1011712,"data_stored":460492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335540,"statfs":{"total":96636764160,"available":96599592960,"internally_reserved":0,"allocated":983040,"data_stored":452666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335532,"statfs":{"total":96636764160,"available":96599584768,"internally_reserved":0,"allocated":991232,"data_stored":452826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":999424,"data_stored":458888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1572864,"data_stored":1042452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:57:30.927 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:57:30.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:57:31.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.068+0000 7fbc1f683640 1 Processor -- start 2024-04-25T09:57:31.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.068+0000 7fbc1f683640 1 -- start start 2024-04-25T09:57:31.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.069+0000 7fbc1f683640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbc1806c9e0 0x7fbc180b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.069+0000 7fbc1f683640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1806cf50 con 0x7fbc180b5620 2024-04-25T09:57:31.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.069+0000 7fbc1f683640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc180ce530 con 0x7fbc1815b7d0 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.069+0000 7fbc1f683640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc180ce810 con 0x7fbc1806c9e0 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.069+0000 7fbc1cc1e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbc180b5620 0x7fbc1806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40402/0 (socket says 172.21.15.19:40402) 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.069+0000 7fbc1cc1e640 1 -- 172.21.15.19:0/3505928016 learned_addr learned my addr 172.21.15.19:0/3505928016 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.069+0000 7fbc1dc20640 1 --2- 172.21.15.19:0/3505928016 >> v2:172.21.15.112:3300/0 conn(0x7fbc1806c9e0 0x7fbc180b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606224116 0 0) 0x7fbc1806cf50 con 0x7fbc180b5620 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc00003520 con 0x7fbc180b5620 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3848291923 0 0) 0x7fbc00003520 con 0x7fbc180b5620 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 >> v2:172.21.15.112:3300/0 conn(0x7fbc1806c9e0 msgr2=0x7fbc180b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 --2- 172.21.15.19:0/3505928016 >> v2:172.21.15.112:3300/0 conn(0x7fbc1806c9e0 0x7fbc180b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 >> v1:172.21.15.112:6789/0 conn(0x7fbc1815b7d0 legacy=0x7fbc180cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc180ceaf0 con 0x7fbc180b5620 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fbc0c001670 con 0x7fbc180b5620 2024-04-25T09:57:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc1dc20640 1 --2- 172.21.15.19:0/3505928016 >> v2:172.21.15.112:3300/0 conn(0x7fbc1806c9e0 0x7fbc180b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 >> v1:172.21.15.19:6789/0 conn(0x7fbc180b5620 legacy=0x7fbc1806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 --2- 172.21.15.19:0/3505928016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc000039b0 0x7fbc00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 --2- 172.21.15.19:0/3505928016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc00004800 0x7fbc00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 --2- 172.21.15.19:0/3505928016 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc000051c0 0x7fbc0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc180ce810 con 0x7fbc00004800 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0000c060 con 0x7fbc000039b0 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0000c260 con 0x7fbc000051c0 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc1cc1e640 1 --2- 172.21.15.19:0/3505928016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc00004800 0x7fbc00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc1d41f640 1 --2- 172.21.15.19:0/3505928016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc000039b0 0x7fbc00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc1dc20640 1 --2- 172.21.15.19:0/3505928016 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc000051c0 0x7fbc0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc1cc1e640 1 -- 172.21.15.19:0/3505928016 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc000051c0 msgr2=0x7fbc0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc1cc1e640 1 --2- 172.21.15.19:0/3505928016 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc000051c0 0x7fbc0000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc1cc1e640 1 -- 172.21.15.19:0/3505928016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc000039b0 msgr2=0x7fbc00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc1cc1e640 1 --2- 172.21.15.19:0/3505928016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc000039b0 0x7fbc00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc1cc1e640 1 -- 172.21.15.19:0/3505928016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc00003520 con 0x7fbc00004800 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.070+0000 7fbc1dc20640 1 -- 172.21.15.19:0/3505928016 reap_dead start 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.071+0000 7fbc1cc1e640 1 --2- 172.21.15.19:0/3505928016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc00004800 0x7fbc00004c80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c240 tx=0x7fbc00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f447232a039274d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.071+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c007cf0 con 0x7fbc00004800 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.071+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c005af0 con 0x7fbc00004800 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.071+0000 7fbc17fff640 1 -- 172.21.15.19:0/3505928016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c007070 con 0x7fbc00004800 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.072+0000 7fbc1f683640 1 -- 172.21.15.19:0/3505928016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc00004800 msgr2=0x7fbc00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.072+0000 7fbc1f683640 1 --2- 172.21.15.19:0/3505928016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc00004800 0x7fbc00004c80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c240 tx=0x7fbc00005830 comp rx=0 tx=0).stop 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.072+0000 7fbc1f683640 1 -- 172.21.15.19:0/3505928016 shutdown_connections 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.072+0000 7fbc1f683640 1 --2- 172.21.15.19:0/3505928016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc00004800 0x7fbc00004c80 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.072+0000 7fbc1f683640 1 -- 172.21.15.19:0/3505928016 >> 172.21.15.19:0/3505928016 conn(0x7fbc180d0800 msgr2=0x7fbc180d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.072+0000 7fbc1f683640 1 -- 172.21.15.19:0/3505928016 shutdown_connections 2024-04-25T09:57:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.072+0000 7fbc1f683640 1 -- 172.21.15.19:0/3505928016 wait complete. 2024-04-25T09:57:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.073+0000 7fbc1f683640 1 Processor -- start 2024-04-25T09:57:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.073+0000 7fbc1f683640 1 -- start start 2024-04-25T09:57:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1f683640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc00004800 0x7fbc1814eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1f683640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc180b4150 0x7fbc180c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1f683640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc180b45d0 0x7fbc180c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1f683640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1815bd30 con 0x7fbc180b4150 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1f683640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1815bbb0 con 0x7fbc180b45d0 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1f683640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc180b5ab0 con 0x7fbc00004800 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1cc1e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc180b4150 0x7fbc180c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1cc1e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc180b4150 0x7fbc180c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53704/0 (socket says 172.21.15.19:53704) 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1cc1e640 1 -- 172.21.15.19:0/4125051416 learned_addr learned my addr 172.21.15.19:0/4125051416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1dc20640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc180b45d0 0x7fbc180c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1d41f640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc00004800 0x7fbc1814eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1cc1e640 1 -- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc00004800 msgr2=0x7fbc1814eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1cc1e640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc00004800 0x7fbc1814eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1cc1e640 1 -- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc180b45d0 msgr2=0x7fbc180c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1cc1e640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc180b45d0 0x7fbc180c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1cc1e640 1 -- 172.21.15.19:0/4125051416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc180cb4e0 con 0x7fbc180b4150 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1dc20640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc180b45d0 0x7fbc180c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc1cc1e640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc180b4150 0x7fbc180c7570 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c02ad80 tx=0x7fbc180cecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc15ffb640 1 -- 172.21.15.19:0/4125051416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c001470 con 0x7fbc180b4150 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.074+0000 7fbc15ffb640 1 -- 172.21.15.19:0/4125051416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c040070 con 0x7fbc180b4150 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.075+0000 7fbc1f683640 1 -- 172.21.15.19:0/4125051416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc180cb7c0 con 0x7fbc180b4150 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.075+0000 7fbc15ffb640 1 -- 172.21.15.19:0/4125051416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0047e0 con 0x7fbc180b4150 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.075+0000 7fbc1f683640 1 -- 172.21.15.19:0/4125051416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc180cbd80 con 0x7fbc180b4150 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.076+0000 7fbc1f683640 1 -- 172.21.15.19:0/4125051416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc18073a00 con 0x7fbc180b4150 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.076+0000 7fbc15ffb640 1 -- 172.21.15.19:0/4125051416 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc0c03a070 con 0x7fbc180b4150 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.077+0000 7fbc15ffb640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0406b4d0 0x7fbc0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.077+0000 7fbc15ffb640 1 -- 172.21.15.19:0/4125051416 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc0c0a9820 con 0x7fbc180b4150 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.077+0000 7fbc1d41f640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0406b4d0 0x7fbc0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.077+0000 7fbc1d41f640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0406b4d0 0x7fbc0406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbc10000c00 tx=0x7fbc10009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.081+0000 7fbc15ffb640 1 -- 172.21.15.19:0/4125051416 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc0c043370 con 0x7fbc180b4150 2024-04-25T09:57:31.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.332+0000 7fbc1f683640 1 -- 172.21.15.19:0/4125051416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbc1815ed90 con 0x7fbc180b4150 2024-04-25T09:57:31.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.334+0000 7fbc15ffb640 1 -- 172.21.15.19:0/4125051416 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbc0c078510 con 0x7fbc180b4150 2024-04-25T09:57:31.338 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T09:57:31.338 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T09:57:31.338 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T09:57:31.338 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T09:57:31.338 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:57:31.338 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T09:57:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 -- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0406b4d0 msgr2=0x7fbc0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0406b4d0 0x7fbc0406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbc10000c00 tx=0x7fbc10009040 comp rx=0 tx=0).stop 2024-04-25T09:57:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 -- 172.21.15.19:0/4125051416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc180b4150 msgr2=0x7fbc180c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc180b4150 0x7fbc180c7570 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c02ad80 tx=0x7fbc180cecc0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 -- 172.21.15.19:0/4125051416 shutdown_connections 2024-04-25T09:57:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc0406b4d0 0x7fbc0406d990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc180b45d0 0x7fbc180c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc180b4150 0x7fbc180c7570 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 --2- 172.21.15.19:0/4125051416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc00004800 0x7fbc1814eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 -- 172.21.15.19:0/4125051416 >> 172.21.15.19:0/4125051416 conn(0x7fbc180d0800 msgr2=0x7fbc18160c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 -- 172.21.15.19:0/4125051416 shutdown_connections 2024-04-25T09:57:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.337+0000 7fbc1f683640 1 -- 172.21.15.19:0/4125051416 wait complete. 2024-04-25T09:57:31.351 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.448+0000 7f2fcf2f5640 1 Processor -- start 2024-04-25T09:57:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.448+0000 7f2fcf2f5640 1 -- start start 2024-04-25T09:57:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.449+0000 7f2fcf2f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc80b6cb0 0x7f2fc80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.449+0000 7f2fcf2f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2fc80b5a60 0x7f2fc80d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.449+0000 7f2fcf2f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc80ceee0 con 0x7f2fc815db70 2024-04-25T09:57:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.449+0000 7f2fcf2f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc80d1d10 con 0x7f2fc80b5a60 2024-04-25T09:57:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.449+0000 7f2fcf2f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80d1ff0 con 0x7f2fc80b6cb0 2024-04-25T09:57:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.449+0000 7f2fcc890640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc80b6cb0 0x7f2fc80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.449+0000 7f2fcd892640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2fc80b5a60 0x7f2fc80d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.449+0000 7f2fcd091640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2fc815db70 0x7f2fc8162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:41304/0 (socket says 172.21.15.19:41304) 2024-04-25T09:57:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.449+0000 7f2fcd091640 1 -- 172.21.15.19:0/1872808519 learned_addr learned my addr 172.21.15.19:0/1872808519 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.450+0000 7f2fcc890640 1 -- 172.21.15.19:0/1872808519 >> v2:172.21.15.112:3300/0 conn(0x7f2fc80b5a60 msgr2=0x7f2fc80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.450+0000 7f2fcc890640 1 --2- 172.21.15.19:0/1872808519 >> v2:172.21.15.112:3300/0 conn(0x7f2fc80b5a60 0x7f2fc80d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.450+0000 7f2fcc890640 1 -- 172.21.15.19:0/1872808519 >> v1:172.21.15.112:6789/0 conn(0x7f2fc815db70 legacy=0x7f2fc8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.450+0000 7f2fcc890640 1 -- 172.21.15.19:0/1872808519 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc80d22d0 con 0x7f2fc80b6cb0 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.450+0000 7f2fcd892640 1 --2- 172.21.15.19:0/1872808519 >> v2:172.21.15.112:3300/0 conn(0x7f2fc80b5a60 0x7f2fc80d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.450+0000 7f2fcc890640 1 --2- 172.21.15.19:0/1872808519 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc80b6cb0 0x7f2fc80b5520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80097a0 tx=0x7f2fb802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd651df3fd9195b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.451+0000 7f2fbffff640 1 -- 172.21.15.19:0/1872808519 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8037070 con 0x7f2fc80b6cb0 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.451+0000 7f2fbffff640 1 -- 172.21.15.19:0/1872808519 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb8032440 con 0x7f2fc80b6cb0 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.451+0000 7f2fbffff640 1 -- 172.21.15.19:0/1872808519 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8032810 con 0x7f2fc80b6cb0 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.451+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/1872808519 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc80b6cb0 msgr2=0x7f2fc80b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.451+0000 7f2fcf2f5640 1 --2- 172.21.15.19:0/1872808519 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc80b6cb0 0x7f2fc80b5520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80097a0 tx=0x7f2fb802a180 comp rx=0 tx=0).stop 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.451+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/1872808519 shutdown_connections 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.451+0000 7f2fcf2f5640 1 --2- 172.21.15.19:0/1872808519 >> v2:172.21.15.112:3300/0 conn(0x7f2fc80b5a60 0x7f2fc80d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.451+0000 7f2fcf2f5640 1 --2- 172.21.15.19:0/1872808519 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc80b6cb0 0x7f2fc80b5520 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.451+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/1872808519 >> 172.21.15.19:0/1872808519 conn(0x7f2fc80d0520 msgr2=0x7f2fc80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.451+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/1872808519 shutdown_connections 2024-04-25T09:57:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.451+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/1872808519 wait complete. 2024-04-25T09:57:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.452+0000 7f2fcf2f5640 1 Processor -- start 2024-04-25T09:57:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.452+0000 7f2fcf2f5640 1 -- start start 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.453+0000 7f2fcf2f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc80b5a60 0x7f2fc8160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.453+0000 7f2fcf2f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc80b6cb0 0x7f2fc8157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.453+0000 7f2fcf2f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc815db70 0x7f2fc8158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.453+0000 7f2fcf2f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80cf1c0 con 0x7f2fc80b5a60 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.453+0000 7f2fcf2f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80ce530 con 0x7f2fc80b6cb0 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.453+0000 7f2fcf2f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80cf340 con 0x7f2fc815db70 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.453+0000 7f2fcd091640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc80b5a60 0x7f2fc8160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.453+0000 7f2fcd091640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc80b5a60 0x7f2fc8160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53720/0 (socket says 172.21.15.19:53720) 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.453+0000 7f2fcd091640 1 -- 172.21.15.19:0/3311277314 learned_addr learned my addr 172.21.15.19:0/3311277314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fcd892640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc815db70 0x7f2fc8158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fcc890640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc80b6cb0 0x7f2fc8157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fcd091640 1 -- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc815db70 msgr2=0x7f2fc8158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fcd091640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc815db70 0x7f2fc8158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fcd091640 1 -- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc80b6cb0 msgr2=0x7f2fc8157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fcd091640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc80b6cb0 0x7f2fc8157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fcd091640 1 -- 172.21.15.19:0/3311277314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8159760 con 0x7f2fc80b5a60 2024-04-25T09:57:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fcd091640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc80b5a60 0x7f2fc8160bb0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c3f0 tx=0x7f2fc400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fbdffb640 1 -- 172.21.15.19:0/3311277314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4017070 con 0x7f2fc80b5a60 2024-04-25T09:57:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fbdffb640 1 -- 172.21.15.19:0/3311277314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc4005ce0 con 0x7f2fc80b5a60 2024-04-25T09:57:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fbdffb640 1 -- 172.21.15.19:0/3311277314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40072f0 con 0x7f2fc80b5a60 2024-04-25T09:57:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/3311277314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc8159a20 con 0x7f2fc80b5a60 2024-04-25T09:57:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.454+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/3311277314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc8159fe0 con 0x7f2fc80b5a60 2024-04-25T09:57:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.455+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/3311277314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc8073990 con 0x7f2fc80b5a60 2024-04-25T09:57:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.455+0000 7f2fbdffb640 1 -- 172.21.15.19:0/3311277314 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2fc401a070 con 0x7f2fc80b5a60 2024-04-25T09:57:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.456+0000 7f2fbdffb640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 0x7f2fac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.456+0000 7f2fbdffb640 1 -- 172.21.15.19:0/3311277314 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2fc40896a0 con 0x7f2fc80b5a60 2024-04-25T09:57:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.456+0000 7f2fcc890640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 0x7f2fac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.457+0000 7f2fcc890640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 0x7f2fac06d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8034000 tx=0x7f2fb802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:31.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.460+0000 7f2fbdffb640 1 -- 172.21.15.19:0/3311277314 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2fc4058420 con 0x7f2fc80b5a60 2024-04-25T09:57:31.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.677+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/3311277314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fc80d1820 con 0x7f2fc80b5a60 2024-04-25T09:57:31.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.677+0000 7f2fbdffb640 1 -- 172.21.15.19:0/3311277314 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2fc4057d20 con 0x7f2fc80b5a60 2024-04-25T09:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2023643575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2023643575},{"type":"v1","addr":"172.21.15.19:6835","nonce":2023643575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3683680157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3683680157},{"type":"v1","addr":"172.21.15.112:6836","nonce":3683680157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/309620056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":309620056},{"type":"v1","addr":"172.21.15.19:6833","nonce":309620056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:26.327989+0000","modified":"2024-04-25T09:57:28.922996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7137},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7137":{"gid":7137,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1747929190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1747929190},{"type":"v1","addr":"172.21.15.112:6837","nonce":1747929190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:57:31.681 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 51 2024-04-25T09:57:31.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 msgr2=0x7f2fac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 0x7f2fac06d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8034000 tx=0x7f2fb802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:31.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/3311277314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc80b5a60 msgr2=0x7f2fc8160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:31.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc80b5a60 0x7f2fc8160bb0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c3f0 tx=0x7f2fc400c8b0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/3311277314 shutdown_connections 2024-04-25T09:57:31.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2fac06b4d0 0x7f2fac06d990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fc815db70 0x7f2fc8158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc80b6cb0 0x7f2fc8157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 --2- 172.21.15.19:0/3311277314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fc80b5a60 0x7f2fc8160bb0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:31.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/3311277314 >> 172.21.15.19:0/3311277314 conn(0x7f2fc80d0520 msgr2=0x7f2fc8157100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:31.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/3311277314 shutdown_connections 2024-04-25T09:57:31.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:31.679+0000 7f2fcf2f5640 1 -- 172.21.15.19:0/3311277314 wait complete. 2024-04-25T09:57:32.096 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:57:26.327989+0000', 'modified': '2024-04-25T09:57:28.922996+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7137}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7137': {'gid': 7137, 'name': 'b', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6837/1747929190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6835', 'nonce': 1747929190}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 1747929190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:57:32.096 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:57:32.096 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T09:57:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.200+0000 7f20c8f5c640 1 Processor -- start 2024-04-25T09:57:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.200+0000 7f20c8f5c640 1 -- start start 2024-04-25T09:57:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.201+0000 7f20c8f5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f20c415dc20 0x7f20c40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.201+0000 7f20c8f5c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20c4161710 0x7f20c40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.201+0000 7f20c8f5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20c40b9330 0x7f20c406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.201+0000 7f20c8f5c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c415e000 con 0x7f20c4161710 2024-04-25T09:57:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.201+0000 7f20c8f5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c40d1de0 con 0x7f20c415dc20 2024-04-25T09:57:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.201+0000 7f20c8f5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c4161c60 con 0x7f20c40b9330 2024-04-25T09:57:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.201+0000 7f20c1d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20c4161710 0x7f20c40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.201+0000 7f20c1d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20c4161710 0x7f20c40cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53732/0 (socket says 172.21.15.19:53732) 2024-04-25T09:57:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.201+0000 7f20c1d9b640 1 -- 172.21.15.19:0/1590634614 learned_addr learned my addr 172.21.15.19:0/1590634614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:32.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.202+0000 7f20c1d9b640 1 -- 172.21.15.19:0/1590634614 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20c40b9330 msgr2=0x7f20c406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:32.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.202+0000 7f20c1d9b640 1 --2- 172.21.15.19:0/1590634614 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20c40b9330 0x7f20c406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.202+0000 7f20c1d9b640 1 -- 172.21.15.19:0/1590634614 >> v2:172.21.15.112:3300/0 conn(0x7f20c415dc20 msgr2=0x7f20c40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:32.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.202+0000 7f20c259c640 1 --2- 172.21.15.19:0/1590634614 >> v2:172.21.15.112:3300/0 conn(0x7f20c415dc20 0x7f20c40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.202+0000 7f20c1d9b640 1 --2- 172.21.15.19:0/1590634614 >> v2:172.21.15.112:3300/0 conn(0x7f20c415dc20 0x7f20c40ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.202+0000 7f20c1d9b640 1 -- 172.21.15.19:0/1590634614 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c4161f40 con 0x7f20c4161710 2024-04-25T09:57:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.202+0000 7f20c1d9b640 1 --2- 172.21.15.19:0/1590634614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c4161710 0x7f20c40cebb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f20c40d1520 tx=0x7f20ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecb0d77ae976276 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.202+0000 7f20c159a640 1 -- 172.21.15.19:0/1590634614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac037070 con 0x7f20c4161710 2024-04-25T09:57:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.203+0000 7f20c159a640 1 -- 172.21.15.19:0/1590634614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac032440 con 0x7f20c4161710 2024-04-25T09:57:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.203+0000 7f20c159a640 1 -- 172.21.15.19:0/1590634614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac032810 con 0x7f20c4161710 2024-04-25T09:57:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.203+0000 7f20c8f5c640 1 -- 172.21.15.19:0/1590634614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c4161710 msgr2=0x7f20c40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.203+0000 7f20c8f5c640 1 --2- 172.21.15.19:0/1590634614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c4161710 0x7f20c40cebb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f20c40d1520 tx=0x7f20ac02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.203+0000 7f20c8f5c640 1 -- 172.21.15.19:0/1590634614 shutdown_connections 2024-04-25T09:57:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.203+0000 7f20c8f5c640 1 --2- 172.21.15.19:0/1590634614 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20c40b9330 0x7f20c406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.203+0000 7f20c8f5c640 1 --2- 172.21.15.19:0/1590634614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c4161710 0x7f20c40cebb0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.203+0000 7f20c8f5c640 1 --2- 172.21.15.19:0/1590634614 >> v2:172.21.15.112:3300/0 conn(0x7f20c415dc20 0x7f20c40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.203+0000 7f20c8f5c640 1 -- 172.21.15.19:0/1590634614 >> 172.21.15.19:0/1590634614 conn(0x7f20c40d0420 msgr2=0x7f20c40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.203+0000 7f20c8f5c640 1 -- 172.21.15.19:0/1590634614 shutdown_connections 2024-04-25T09:57:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.203+0000 7f20c8f5c640 1 -- 172.21.15.19:0/1590634614 wait complete. 2024-04-25T09:57:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.204+0000 7f20c8f5c640 1 Processor -- start 2024-04-25T09:57:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.204+0000 7f20c8f5c640 1 -- start start 2024-04-25T09:57:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.205+0000 7f20c8f5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20c40b9330 0x7f20c40cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.205+0000 7f20c8f5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20c415dc20 0x7f20c41519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.205+0000 7f20c8f5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c4161710 0x7f20c4160780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.205+0000 7f20c8f5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40b5e50 con 0x7f20c4161710 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.205+0000 7f20c8f5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c4162500 con 0x7f20c415dc20 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.205+0000 7f20c8f5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40b5fd0 con 0x7f20c40b9330 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.205+0000 7f20c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c4161710 0x7f20c4160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.205+0000 7f20c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c4161710 0x7f20c4160780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53734/0 (socket says 172.21.15.19:53734) 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.205+0000 7f20c2d9d640 1 -- 172.21.15.19:0/3161977221 learned_addr learned my addr 172.21.15.19:0/3161977221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.205+0000 7f20c1d9b640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20c415dc20 0x7f20c41519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.205+0000 7f20c259c640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20c40b9330 0x7f20c40cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.206+0000 7f20c2d9d640 1 -- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20c40b9330 msgr2=0x7f20c40cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.206+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20c40b9330 0x7f20c40cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.206+0000 7f20c2d9d640 1 -- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20c415dc20 msgr2=0x7f20c41519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.206+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20c415dc20 0x7f20c41519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.206+0000 7f20c2d9d640 1 -- 172.21.15.19:0/3161977221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c41e7760 con 0x7f20c4161710 2024-04-25T09:57:32.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.206+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c4161710 0x7f20c4160780 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f20b400b9e0 tx=0x7f20b400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.206+0000 7f20baffd640 1 -- 172.21.15.19:0/3161977221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4019070 con 0x7f20c4161710 2024-04-25T09:57:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.206+0000 7f20baffd640 1 -- 172.21.15.19:0/3161977221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b400caa0 con 0x7f20c4161710 2024-04-25T09:57:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.206+0000 7f20c8f5c640 1 -- 172.21.15.19:0/3161977221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c41e7a40 con 0x7f20c4161710 2024-04-25T09:57:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.206+0000 7f20baffd640 1 -- 172.21.15.19:0/3161977221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4014440 con 0x7f20c4161710 2024-04-25T09:57:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.206+0000 7f20c8f5c640 1 -- 172.21.15.19:0/3161977221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c41e8000 con 0x7f20c4161710 2024-04-25T09:57:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.208+0000 7f20baffd640 1 -- 172.21.15.19:0/3161977221 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20b401c070 con 0x7f20c4161710 2024-04-25T09:57:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.208+0000 7f20baffd640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20a406b4d0 0x7f20a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.208+0000 7f20baffd640 1 -- 172.21.15.19:0/3161977221 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20b4089f10 con 0x7f20c4161710 2024-04-25T09:57:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.208+0000 7f20c259c640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20a406b4d0 0x7f20a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.209+0000 7f20c259c640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20a406b4d0 0x7f20a406d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f20c40b8c80 tx=0x7f20bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.209+0000 7f20cbc57740 1 Processor -- start 2024-04-25T09:57:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.209+0000 7f20cbc57740 1 -- start start 2024-04-25T09:57:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb44230 0x55acbbb44610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb61ad0 0x55acbbb62020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb19f10 0x55acbbb60190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20cbc57740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acbba7ca10 con 0x55acbbb19f10 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20cbc57740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acbba25b30 con 0x55acbbb44230 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20cbc57740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acbba78c50 con 0x55acbbb61ad0 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb19f10 0x55acbbb60190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20c259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb44230 0x55acbbb44610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb19f10 0x55acbbb60190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53750/0 (socket says 172.21.15.19:53750) 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1347955871 learned_addr learned my addr 172.21.15.19:0/1347955871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20c1d9b640 1 --2- 172.21.15.19:0/1347955871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb61ad0 0x55acbbb62020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1347955871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb61ad0 msgr2=0x55acbbb62020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/1347955871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb61ad0 0x55acbbb62020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1347955871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb44230 msgr2=0x55acbbb44610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/1347955871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb44230 0x55acbbb44610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1347955871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acbba3ae10 con 0x55acbbb19f10 2024-04-25T09:57:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.210+0000 7f20c259c640 1 --2- 172.21.15.19:0/1347955871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb44230 0x55acbbb44610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/1347955871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb19f10 0x55acbbb60190 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f20b40606f0 tx=0x7f20b4058680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d55de9fcbc90d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20a3fff640 1 -- 172.21.15.19:0/1347955871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b40568c0 con 0x55acbbb19f10 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20a3fff640 1 -- 172.21.15.19:0/1347955871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b40770c0 con 0x55acbbb19f10 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20a3fff640 1 -- 172.21.15.19:0/1347955871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4060790 con 0x55acbbb19f10 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20cbc57740 1 -- 172.21.15.19:0/1347955871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb19f10 msgr2=0x55acbbb60190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1347955871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb19f10 0x55acbbb60190 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f20b40606f0 tx=0x7f20b4058680 comp rx=0 tx=0).stop 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20cbc57740 1 -- 172.21.15.19:0/1347955871 shutdown_connections 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1347955871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb19f10 0x55acbbb60190 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1347955871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb61ad0 0x55acbbb62020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1347955871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb44230 0x55acbbb44610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20cbc57740 1 -- 172.21.15.19:0/1347955871 >> 172.21.15.19:0/1347955871 conn(0x55acbbb16840 msgr2=0x55acbbb5f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20cbc57740 1 -- 172.21.15.19:0/1347955871 shutdown_connections 2024-04-25T09:57:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.211+0000 7f20cbc57740 1 -- 172.21.15.19:0/1347955871 wait complete. 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.212+0000 7f20cbc57740 1 -- start start 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.212+0000 7f20cbc57740 1 Processor -- start 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 0x55acbbb46420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb61ad0 0x55acbbb301a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb40dc0 0x55acbbb134a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20cbc57740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acbbb1acb0 con 0x55acbbb40dc0 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20cbc57740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acbba390c0 con 0x55acbbb61ad0 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20cbc57740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acbbb14770 con 0x55acbbb44230 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20cbc57740 10 client.0 _ll_register_callbacks cb 0x55acbba2cec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb40dc0 0x55acbbb134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb40dc0 0x55acbbb134a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53764/0 (socket says 172.21.15.19:53764) 2024-04-25T09:57:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1980848358 learned_addr learned my addr 172.21.15.19:0/1980848358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20c1d9b640 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb61ad0 0x55acbbb301a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20c259c640 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 0x55acbbb46420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 msgr2=0x55acbbb46420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 0x55acbbb46420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb61ad0 msgr2=0x55acbbb301a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb61ad0 0x55acbbb301a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.213+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1980848358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acbb7912d0 con 0x55acbbb40dc0 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20c259c640 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 0x55acbbb46420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb40dc0 0x55acbbb134a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f20b4054a20 tx=0x7f20b400e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4efde274c07d8d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20a17fa640 1 -- 172.21.15.19:0/1980848358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4085020 con 0x55acbbb40dc0 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20a17fa640 1 -- 172.21.15.19:0/1980848358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20b4060990 con 0x55acbbb40dc0 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20cbc57740 1 -- 172.21.15.19:0/1980848358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20b40770c0 con 0x55acbbb40dc0 2024-04-25T09:57:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20a17fa640 1 -- 172.21.15.19:0/1980848358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20b4058390 con 0x55acbbb40dc0 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20cbc57740 10 client.7272 fetch_fsmap learned FSMap version 51 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20cbc57740 1 -- 172.21.15.19:0/1980848358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55acbb85b6b0 con 0x55acbbb40dc0 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20a17fa640 1 -- 172.21.15.19:0/1980848358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b40770c0 con 0x55acbbb40dc0 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20a17fa640 1 -- 172.21.15.19:0/1980848358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f20b4075ce0 con 0x55acbbb40dc0 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20cbc57740 10 client.7272 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20cbc57740 10 client.7272 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20cbc57740 20 client.7272 populate_metadata read hostname 'smithi019' 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55acbbb64ab0 0x55acbbb64e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20cbc57740 4 client.7272 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.214+0000 7f20cbc57740 1 -- 172.21.15.19:0/1980848358 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acbbadb920 con 0x55acbbb64ab0 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.215+0000 7f20c259c640 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55acbbb64ab0 0x55acbbb64e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.215+0000 7f20a17fa640 10 client.7272 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:57:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.215+0000 7f20c259c640 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55acbbb64ab0 0x55acbbb64e90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca47eb57466f3c7 server_cookie=102dcfa81e8150cf in_seq=0 out_seq=0 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20a17fa640 1 -- 172.21.15.19:0/1980848358 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55acbbadb920 con 0x55acbbb64ab0 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20a17fa640 10 client.7272 handle_command_reply: tid=0 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 1 client.7272 shutdown 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 1 -- 172.21.15.19:0/1980848358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb40dc0 msgr2=0x55acbbb134a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb40dc0 0x55acbbb134a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f20b4054a20 tx=0x7f20b400e040 comp rx=0 tx=0).stop 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 1 -- 172.21.15.19:0/1980848358 shutdown_connections 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55acbbb64ab0 0x55acbbb64e90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb40dc0 0x55acbbb134a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb61ad0 0x55acbbb301a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20a17fa640 0 client.7272 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1980848358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 0x55acbbb46420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 1 -- 172.21.15.19:0/1980848358 >> 172.21.15.19:0/1980848358 conn(0x55acbbb19f10 msgr2=0x55acbbb625c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 1 -- 172.21.15.19:0/1980848358 shutdown_connections 2024-04-25T09:57:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 1 -- 172.21.15.19:0/1980848358 wait complete. 2024-04-25T09:57:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.216+0000 7f20cbc57740 20 client.7272 trim_cache size 0 max 16384 2024-04-25T09:57:32.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.232+0000 7f20cbc57740 1 Processor -- start 2024-04-25T09:57:32.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.232+0000 7f20cbc57740 1 -- start start 2024-04-25T09:57:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb40dc0 0x55acbbb659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb44230 0x55acbbb55c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb61ad0 0x55acbbb56190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20cbc57740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c41ff720 con 0x55acbbb44230 2024-04-25T09:57:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20cbc57740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c415eb20 con 0x55acbbb61ad0 2024-04-25T09:57:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20cbc57740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c41ff2f0 con 0x55acbbb40dc0 2024-04-25T09:57:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb44230 0x55acbbb55c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb44230 0x55acbbb55c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53780/0 (socket says 172.21.15.19:53780) 2024-04-25T09:57:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c1d9b640 1 -- 172.21.15.19:0/2550390265 learned_addr learned my addr 172.21.15.19:0/2550390265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/2550390265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb61ad0 0x55acbbb56190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c259c640 1 --2- 172.21.15.19:0/2550390265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb40dc0 0x55acbbb659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c1d9b640 1 -- 172.21.15.19:0/2550390265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb40dc0 msgr2=0x55acbbb659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c1d9b640 1 --2- 172.21.15.19:0/2550390265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb40dc0 0x55acbbb659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c1d9b640 1 -- 172.21.15.19:0/2550390265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb61ad0 msgr2=0x55acbbb56190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c1d9b640 1 --2- 172.21.15.19:0/2550390265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb61ad0 0x55acbbb56190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c1d9b640 1 -- 172.21.15.19:0/2550390265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acbbb82480 con 0x55acbbb44230 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/2550390265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb61ad0 0x55acbbb56190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20c1d9b640 1 --2- 172.21.15.19:0/2550390265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb44230 0x55acbbb55c50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0090b0 tx=0x7f20ac02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad63ea903fec3cab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20a37fe640 1 -- 172.21.15.19:0/2550390265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac04b070 con 0x55acbbb44230 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20a37fe640 1 -- 172.21.15.19:0/2550390265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac032990 con 0x55acbbb44230 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.233+0000 7f20a37fe640 1 -- 172.21.15.19:0/2550390265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac037040 con 0x55acbbb44230 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.234+0000 7f20cbc57740 1 -- 172.21.15.19:0/2550390265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb44230 msgr2=0x55acbbb55c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.234+0000 7f20cbc57740 1 --2- 172.21.15.19:0/2550390265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb44230 0x55acbbb55c50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0090b0 tx=0x7f20ac02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.234+0000 7f20cbc57740 1 -- 172.21.15.19:0/2550390265 shutdown_connections 2024-04-25T09:57:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.234+0000 7f20cbc57740 1 --2- 172.21.15.19:0/2550390265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb61ad0 0x55acbbb56190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.234+0000 7f20cbc57740 1 --2- 172.21.15.19:0/2550390265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb44230 0x55acbbb55c50 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.234+0000 7f20cbc57740 1 --2- 172.21.15.19:0/2550390265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb40dc0 0x55acbbb659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.234+0000 7f20cbc57740 1 -- 172.21.15.19:0/2550390265 >> 172.21.15.19:0/2550390265 conn(0x55acbbb19f10 msgr2=0x55acbbb19430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.234+0000 7f20cbc57740 1 -- 172.21.15.19:0/2550390265 shutdown_connections 2024-04-25T09:57:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.234+0000 7f20cbc57740 1 -- 172.21.15.19:0/2550390265 wait complete. 2024-04-25T09:57:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.234+0000 7f20cbc57740 1 -- start start 2024-04-25T09:57:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.235+0000 7f20cbc57740 1 Processor -- start 2024-04-25T09:57:32.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.235+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb40dc0 0x55acbbbc9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.235+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 0x55acbbb61300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.235+0000 7f20cbc57740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb61ad0 0x55acbbb549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.235+0000 7f20cbc57740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acbbb459e0 con 0x55acbbb61ad0 2024-04-25T09:57:32.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.235+0000 7f20cbc57740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acbbb1acb0 con 0x55acbbb40dc0 2024-04-25T09:57:32.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.235+0000 7f20cbc57740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acbbb2fba0 con 0x55acbbb44230 2024-04-25T09:57:32.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20cbc57740 10 client.0 _ll_register_callbacks cb 0x55acbbb14c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:32.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb61ad0 0x55acbbb549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb61ad0 0x55acbbb549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53794/0 (socket says 172.21.15.19:53794) 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1377793747 learned_addr learned my addr 172.21.15.19:0/1377793747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c259c640 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb40dc0 0x55acbbbc9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c1d9b640 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 0x55acbbb61300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 msgr2=0x55acbbb61300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 0x55acbbb61300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb40dc0 msgr2=0x55acbbbc9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb40dc0 0x55acbbbc9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c2d9d640 1 -- 172.21.15.19:0/1377793747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acbbb61840 con 0x55acbbb61ad0 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c1d9b640 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 0x55acbbb61300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20c2d9d640 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb61ad0 0x55acbbb549d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f20b400c5a0 tx=0x7f20b4071000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a042578cc5fce5b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:57:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20a17fa640 1 -- 172.21.15.19:0/1377793747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4085050 con 0x55acbbb61ad0 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20a17fa640 1 -- 172.21.15.19:0/1377793747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20b4060990 con 0x55acbbb61ad0 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20cbc57740 1 -- 172.21.15.19:0/1377793747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20ac032990 con 0x55acbbb61ad0 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20a17fa640 1 -- 172.21.15.19:0/1377793747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20b4066050 con 0x55acbbb61ad0 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20cbc57740 10 client.7278 fetch_fsmap learned FSMap version 51 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20cbc57740 1 -- 172.21.15.19:0/1377793747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55acbbb61840 con 0x55acbbb61ad0 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.236+0000 7f20a17fa640 1 -- 172.21.15.19:0/1377793747 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac032990 con 0x55acbbb61ad0 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20a17fa640 1 -- 172.21.15.19:0/1377793747 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f20b4069020 con 0x55acbbb61ad0 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20cbc57740 10 client.7278 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20cbc57740 10 client.7278 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20cbc57740 20 client.7278 populate_metadata read hostname 'smithi019' 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55acbbb626b0 0x55acbbb62a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20cbc57740 4 client.7278 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20cbc57740 1 -- 172.21.15.19:0/1377793747 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55acbba89f50 con 0x55acbbb626b0 2024-04-25T09:57:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20c259c640 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55acbbb626b0 0x55acbbb62a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20a17fa640 10 client.7278 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20c259c640 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55acbbb626b0 0x55acbbb62a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831c9f3f8bc5d8e3 server_cookie=4094061be9eb1f49 in_seq=0 out_seq=0 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20a17fa640 1 -- 172.21.15.19:0/1377793747 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55acbba89f50 con 0x55acbbb626b0 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20a17fa640 10 client.7278 handle_command_reply: tid=0 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.237+0000 7f20cbc57740 1 client.7278 shutdown 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20cbc57740 1 -- 172.21.15.19:0/1377793747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb61ad0 msgr2=0x55acbbb549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb61ad0 0x55acbbb549d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f20b400c5a0 tx=0x7f20b4071000 comp rx=0 tx=0).stop 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20cbc57740 1 -- 172.21.15.19:0/1377793747 shutdown_connections 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55acbbb626b0 0x55acbbb62a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acbbb61ad0 0x55acbbb549d0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55acbbb44230 0x55acbbb61300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20a17fa640 0 client.7278 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:57:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20cbc57740 1 --2- 172.21.15.19:0/1377793747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acbbb40dc0 0x55acbbbc9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20cbc57740 1 -- 172.21.15.19:0/1377793747 >> 172.21.15.19:0/1377793747 conn(0x55acbbb19f10 msgr2=0x55acbbb654e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:32.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20cbc57740 1 -- 172.21.15.19:0/1377793747 shutdown_connections 2024-04-25T09:57:32.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20cbc57740 1 -- 172.21.15.19:0/1377793747 wait complete. 2024-04-25T09:57:32.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20cbc57740 20 client.7278 trim_cache size 0 max 16384 2024-04-25T09:57:32.248 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:57:32.248 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T09:57:32.248 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T09:57:32.248 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 7137, 2024-04-25T09:57:32.248 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T09:57:32.248 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T09:57:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T09:57:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.2798882880000004, 2024-04-25T09:57:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 51, 2024-04-25T09:57:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 81, 2024-04-25T09:57:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 81, 2024-04-25T09:57:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 10.255054479 2024-04-25T09:57:32.249 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:57:32.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 -- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20a406b4d0 msgr2=0x7f20a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20a406b4d0 0x7f20a406d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f20c40b8c80 tx=0x7f20bc008040 comp rx=0 tx=0).stop 2024-04-25T09:57:32.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 -- 172.21.15.19:0/3161977221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c4161710 msgr2=0x7f20c4160780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c4161710 0x7f20c4160780 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f20b400b9e0 tx=0x7f20b400bea0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 -- 172.21.15.19:0/3161977221 shutdown_connections 2024-04-25T09:57:32.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20a406b4d0 0x7f20a406d990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c4161710 0x7f20c4160780 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20c415dc20 0x7f20c41519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 --2- 172.21.15.19:0/3161977221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20c40b9330 0x7f20c40cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 -- 172.21.15.19:0/3161977221 >> 172.21.15.19:0/3161977221 conn(0x7f20c40d0420 msgr2=0x7f20c40d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:32.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 -- 172.21.15.19:0/3161977221 shutdown_connections 2024-04-25T09:57:32.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.238+0000 7f20c8f5c640 1 -- 172.21.15.19:0/3161977221 wait complete. 2024-04-25T09:57:32.252 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:57:32.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.349+0000 7fcedc0ee640 1 Processor -- start 2024-04-25T09:57:32.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.349+0000 7fcedc0ee640 1 -- start start 2024-04-25T09:57:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.350+0000 7fcedc0ee640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fced40b6cb0 0x7fced40b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.350+0000 7fcedc0ee640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced40ceee0 con 0x7fced415db70 2024-04-25T09:57:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.350+0000 7fcedc0ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced40d1d10 con 0x7fced40b5a60 2024-04-25T09:57:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.350+0000 7fcedc0ee640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced40d1ff0 con 0x7fced40b6cb0 2024-04-25T09:57:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.350+0000 7fced9e8a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fced415db70 0x7fced4162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40412/0 (socket says 172.21.15.19:40412) 2024-04-25T09:57:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.350+0000 7fced9e8a640 1 -- 172.21.15.19:0/3424262216 learned_addr learned my addr 172.21.15.19:0/3424262216 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.350+0000 7fced9689640 1 --2- 172.21.15.19:0/3424262216 >> v2:172.21.15.19:3300/0 conn(0x7fced40b6cb0 0x7fced40b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.351+0000 7fced8e88640 1 -- 172.21.15.19:0/3424262216 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550191638 0 0) 0x7fced40ceee0 con 0x7fced415db70 2024-04-25T09:57:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.351+0000 7fced8e88640 1 -- 172.21.15.19:0/3424262216 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcebc003520 con 0x7fced415db70 2024-04-25T09:57:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.351+0000 7fced9689640 1 -- 172.21.15.19:0/3424262216 >> v1:172.21.15.112:6789/0 conn(0x7fced40b5a60 legacy=0x7fced40d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.351+0000 7fced9689640 1 -- 172.21.15.19:0/3424262216 >> v1:172.21.15.19:6789/0 conn(0x7fced415db70 legacy=0x7fced4162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.351+0000 7fced9689640 1 -- 172.21.15.19:0/3424262216 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced40d22d0 con 0x7fced40b6cb0 2024-04-25T09:57:32.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.351+0000 7fced9689640 1 --2- 172.21.15.19:0/3424262216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced40b6cb0 0x7fced40b5520 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcec80097a0 tx=0x7fcec802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74cc32c64138d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.351+0000 7fced8e88640 1 -- 172.21.15.19:0/3424262216 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8037070 con 0x7fced40b6cb0 2024-04-25T09:57:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.352+0000 7fced8e88640 1 -- 172.21.15.19:0/3424262216 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec8032440 con 0x7fced40b6cb0 2024-04-25T09:57:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.352+0000 7fced8e88640 1 -- 172.21.15.19:0/3424262216 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8032810 con 0x7fced40b6cb0 2024-04-25T09:57:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.352+0000 7fcedc0ee640 1 -- 172.21.15.19:0/3424262216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced40b6cb0 msgr2=0x7fced40b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.352+0000 7fcedc0ee640 1 --2- 172.21.15.19:0/3424262216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced40b6cb0 0x7fced40b5520 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcec80097a0 tx=0x7fcec802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.352+0000 7fcedc0ee640 1 -- 172.21.15.19:0/3424262216 shutdown_connections 2024-04-25T09:57:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.352+0000 7fcedc0ee640 1 --2- 172.21.15.19:0/3424262216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced40b6cb0 0x7fced40b5520 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.352+0000 7fcedc0ee640 1 -- 172.21.15.19:0/3424262216 >> 172.21.15.19:0/3424262216 conn(0x7fced40d0520 msgr2=0x7fced40d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.352+0000 7fcedc0ee640 1 -- 172.21.15.19:0/3424262216 shutdown_connections 2024-04-25T09:57:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.352+0000 7fcedc0ee640 1 -- 172.21.15.19:0/3424262216 wait complete. 2024-04-25T09:57:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.353+0000 7fcedc0ee640 1 Processor -- start 2024-04-25T09:57:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.353+0000 7fcedc0ee640 1 -- start start 2024-04-25T09:57:32.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.354+0000 7fcedc0ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fced40b5a60 0x7fced40aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.354+0000 7fcedc0ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced40b6cb0 0x7fced41f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.354+0000 7fcedc0ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced415db70 0x7fced41f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.354+0000 7fcedc0ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced40cf1c0 con 0x7fced40b6cb0 2024-04-25T09:57:32.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.354+0000 7fcedc0ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced40ce530 con 0x7fced415db70 2024-04-25T09:57:32.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.354+0000 7fcedc0ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced40cf340 con 0x7fced40b5a60 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.354+0000 7fced9689640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced40b6cb0 0x7fced41f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.354+0000 7fced9689640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced40b6cb0 0x7fced41f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53812/0 (socket says 172.21.15.19:53812) 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.354+0000 7fced9689640 1 -- 172.21.15.19:0/1432110966 learned_addr learned my addr 172.21.15.19:0/1432110966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.354+0000 7fced9e8a640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fced40b5a60 0x7fced40aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.354+0000 7fceda68b640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced415db70 0x7fced41f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fced9689640 1 -- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fced40b5a60 msgr2=0x7fced40aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fced9689640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fced40b5a60 0x7fced40aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fced9689640 1 -- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced415db70 msgr2=0x7fced41f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fced9689640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced415db70 0x7fced41f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fced9689640 1 -- 172.21.15.19:0/1432110966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced41601c0 con 0x7fced40b6cb0 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fced9e8a640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fced40b5a60 0x7fced40aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fceda68b640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced415db70 0x7fced41f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fced9689640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced40b6cb0 0x7fced41f3760 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcec80098c0 tx=0x7fcec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fced27fc640 1 -- 172.21.15.19:0/1432110966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8037040 con 0x7fced40b6cb0 2024-04-25T09:57:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fced27fc640 1 -- 172.21.15.19:0/1432110966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec8030500 con 0x7fced40b6cb0 2024-04-25T09:57:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fced27fc640 1 -- 172.21.15.19:0/1432110966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec80308d0 con 0x7fced40b6cb0 2024-04-25T09:57:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fcedc0ee640 1 -- 172.21.15.19:0/1432110966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced41604a0 con 0x7fced40b6cb0 2024-04-25T09:57:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.355+0000 7fcedc0ee640 1 -- 172.21.15.19:0/1432110966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced40cb670 con 0x7fced40b6cb0 2024-04-25T09:57:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.356+0000 7fcedc0ee640 1 -- 172.21.15.19:0/1432110966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced4160890 con 0x7fced40b6cb0 2024-04-25T09:57:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.356+0000 7fced27fc640 1 -- 172.21.15.19:0/1432110966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcec8044020 con 0x7fced40b6cb0 2024-04-25T09:57:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.357+0000 7fced27fc640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcec006b4d0 0x7fcec006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.357+0000 7fced27fc640 1 -- 172.21.15.19:0/1432110966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcec80aa5f0 con 0x7fced40b6cb0 2024-04-25T09:57:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.357+0000 7fced9e8a640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcec006b4d0 0x7fcec006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.358+0000 7fced9e8a640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcec006b4d0 0x7fcec006d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcecc007fd0 tx=0x7fcecc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.361+0000 7fced27fc640 1 -- 172.21.15.19:0/1432110966 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcec8079370 con 0x7fced40b6cb0 2024-04-25T09:57:32.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.580+0000 7fcedc0ee640 1 -- 172.21.15.19:0/1432110966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fced4160670 con 0x7fced40b6cb0 2024-04-25T09:57:32.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.581+0000 7fced27fc640 1 -- 172.21.15.19:0/1432110966 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcec8078c70 con 0x7fced40b6cb0 2024-04-25T09:57:32.584 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:57:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 -- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcec006b4d0 msgr2=0x7fcec006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcec006b4d0 0x7fcec006d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcecc007fd0 tx=0x7fcecc009040 comp rx=0 tx=0).stop 2024-04-25T09:57:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 -- 172.21.15.19:0/1432110966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced40b6cb0 msgr2=0x7fced41f3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced40b6cb0 0x7fced41f3760 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcec80098c0 tx=0x7fcec80046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 -- 172.21.15.19:0/1432110966 shutdown_connections 2024-04-25T09:57:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcec006b4d0 0x7fcec006d990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fced415db70 0x7fced41f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fced40b6cb0 0x7fced41f3760 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 --2- 172.21.15.19:0/1432110966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fced40b5a60 0x7fced40aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 -- 172.21.15.19:0/1432110966 >> 172.21.15.19:0/1432110966 conn(0x7fced40d0520 msgr2=0x7fced40b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 -- 172.21.15.19:0/1432110966 shutdown_connections 2024-04-25T09:57:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:32.584+0000 7fcedc0ee640 1 -- 172.21.15.19:0/1432110966 wait complete. 2024-04-25T09:57:32.598 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:57:32.598 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:57:32.598 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T09:57:32.598 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:57:32.598 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:32.598 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:57:32.598 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:57:32.599 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:57:32.599 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:32.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:32.599 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42493sec preferred_lft 42493sec 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:32.616 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:32.618 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:57:32.679 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T09:57:32.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:32.680 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T09:57:32.736 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T09:57:32.736 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:57:32.736 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:57:32.736 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:57:32.736 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:57:32.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:32.737 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:57:32.737 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:57:32.737 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:57:32.737 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:57:32.737 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:57:32.737 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:57:32.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:32.822 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:57:32.837 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:57:32.837 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:57:32.837 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:32.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:32.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T09:57:32.902 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:57:32.902 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:32.957 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:32.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:57:32.958 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:33.013 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:57:33.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:33.013 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:57:33.095 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:57:33.095 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:57:33.102 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:57:33.103 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:57:33.103 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T09:57:33.103 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T09:57:33.103 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T09:57:33.103 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:57:33.103 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:57:33.103 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:57:33.104 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:57:33.104 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:57:33.105 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:57:33.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:33.105 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:33.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:33.170 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:57:33.233 INFO:teuthology.orchestra.run.smithi019.stdout:client6628 2024-04-25T09:57:33.233 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:57:33.233 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:33.289 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T09:57:33.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.430+0000 7f710d953640 1 Processor -- start 2024-04-25T09:57:33.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.430+0000 7f710d953640 1 -- start start 2024-04-25T09:57:33.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f710d953640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71080b5620 0x7f71080b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f710d953640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f710806c2d0 0x7f710806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f710d953640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710806cc80 con 0x7f710815b7d0 2024-04-25T09:57:33.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f710d953640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f710806cf50 con 0x7f710806c2d0 2024-04-25T09:57:33.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f710d953640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71080ce530 con 0x7f71080b5620 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f71077fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f710806c2d0 0x7f710806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f7106ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f710815b7d0 0x7f71080cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40414/0 (socket says 172.21.15.19:40414) 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f7106ffd640 1 -- 172.21.15.19:0/2330129400 learned_addr learned my addr 172.21.15.19:0/2330129400 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f71067fc640 1 --2- 172.21.15.19:0/2330129400 >> v2:172.21.15.112:3300/0 conn(0x7f71080b5620 0x7f71080b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f71077fe640 1 -- 172.21.15.19:0/2330129400 >> v2:172.21.15.112:3300/0 conn(0x7f71080b5620 msgr2=0x7f71080b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f71077fe640 1 --2- 172.21.15.19:0/2330129400 >> v2:172.21.15.112:3300/0 conn(0x7f71080b5620 0x7f71080b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f71077fe640 1 -- 172.21.15.19:0/2330129400 >> v1:172.21.15.19:6789/0 conn(0x7f710815b7d0 legacy=0x7f71080cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f71077fe640 1 -- 172.21.15.19:0/2330129400 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71080ce810 con 0x7f710806c2d0 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.431+0000 7f7105ffb640 1 -- 172.21.15.19:0/2330129400 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504408555 0 0) 0x7f710806cc80 con 0x7f710815b7d0 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.432+0000 7f71077fe640 1 --2- 172.21.15.19:0/2330129400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710806c2d0 0x7f710806c740 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f71080ce180 tx=0x7f710002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d040d12b5de48f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.432+0000 7f7105ffb640 1 -- 172.21.15.19:0/2330129400 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100007740 con 0x7f710806c2d0 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.432+0000 7f7105ffb640 1 -- 172.21.15.19:0/2330129400 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71000078e0 con 0x7f710806c2d0 2024-04-25T09:57:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.432+0000 7f7105ffb640 1 -- 172.21.15.19:0/2330129400 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100007cb0 con 0x7f710806c2d0 2024-04-25T09:57:33.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.433+0000 7f710d953640 1 -- 172.21.15.19:0/2330129400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710806c2d0 msgr2=0x7f710806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.433+0000 7f710d953640 1 --2- 172.21.15.19:0/2330129400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710806c2d0 0x7f710806c740 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f71080ce180 tx=0x7f710002a680 comp rx=0 tx=0).stop 2024-04-25T09:57:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.433+0000 7f710d953640 1 -- 172.21.15.19:0/2330129400 shutdown_connections 2024-04-25T09:57:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.433+0000 7f710d953640 1 --2- 172.21.15.19:0/2330129400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710806c2d0 0x7f710806c740 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.433+0000 7f710d953640 1 --2- 172.21.15.19:0/2330129400 >> v2:172.21.15.112:3300/0 conn(0x7f71080b5620 0x7f71080b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.433+0000 7f710d953640 1 -- 172.21.15.19:0/2330129400 >> 172.21.15.19:0/2330129400 conn(0x7f71080d0800 msgr2=0x7f71080d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.433+0000 7f710d953640 1 -- 172.21.15.19:0/2330129400 shutdown_connections 2024-04-25T09:57:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.433+0000 7f710d953640 1 -- 172.21.15.19:0/2330129400 wait complete. 2024-04-25T09:57:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.434+0000 7f710d953640 1 Processor -- start 2024-04-25T09:57:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.434+0000 7f710d953640 1 -- start start 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f710d953640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710806c2d0 0x7f710815d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f710d953640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71080b5620 0x7f710815d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f710d953640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710815b7d0 0x7f7108158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f710d953640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710815bbb0 con 0x7f710815b7d0 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f710d953640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710806d230 con 0x7f710806c2d0 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f710d953640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710815bd30 con 0x7f71080b5620 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f71077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710815b7d0 0x7f7108158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f71077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710815b7d0 0x7f7108158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53840/0 (socket says 172.21.15.19:53840) 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f71077fe640 1 -- 172.21.15.19:0/2047484581 learned_addr learned my addr 172.21.15.19:0/2047484581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f7106ffd640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710806c2d0 0x7f710815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f71067fc640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71080b5620 0x7f710815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f71077fe640 1 -- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71080b5620 msgr2=0x7f710815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f71077fe640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71080b5620 0x7f710815d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f71077fe640 1 -- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710806c2d0 msgr2=0x7f710815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f71077fe640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710806c2d0 0x7f710815d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f71077fe640 1 -- 172.21.15.19:0/2047484581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710815a3f0 con 0x7f710815b7d0 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.435+0000 7f71067fc640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71080b5620 0x7f710815d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.436+0000 7f71077fe640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710815b7d0 0x7f7108158090 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f710002ac70 tx=0x7f7100004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.436+0000 7f70effff640 1 -- 172.21.15.19:0/2047484581 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100040070 con 0x7f710815b7d0 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.436+0000 7f70effff640 1 -- 172.21.15.19:0/2047484581 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7100031bc0 con 0x7f710815b7d0 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.436+0000 7f710d953640 1 -- 172.21.15.19:0/2047484581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71080c76a0 con 0x7f710815b7d0 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.436+0000 7f70effff640 1 -- 172.21.15.19:0/2047484581 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100009450 con 0x7f710815b7d0 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.436+0000 7f710d953640 1 -- 172.21.15.19:0/2047484581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7108159e30 con 0x7f710815b7d0 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.437+0000 7f710d953640 1 -- 172.21.15.19:0/2047484581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7108073a00 con 0x7f710815b7d0 2024-04-25T09:57:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.437+0000 7f70effff640 1 -- 172.21.15.19:0/2047484581 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7100039040 con 0x7f710815b7d0 2024-04-25T09:57:33.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.438+0000 7f70effff640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70f406b4d0 0x7f70f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.438+0000 7f70effff640 1 -- 172.21.15.19:0/2047484581 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71000abad0 con 0x7f710815b7d0 2024-04-25T09:57:33.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.438+0000 7f7106ffd640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70f406b4d0 0x7f70f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.438+0000 7f7106ffd640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70f406b4d0 0x7f70f406d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000f60 tx=0x7f70f8007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.442+0000 7f70effff640 1 -- 172.21.15.19:0/2047484581 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f710007a850 con 0x7f710815b7d0 2024-04-25T09:57:33.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:33.660+0000 7f710d953640 1 -- 172.21.15.19:0/2047484581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f71080cdf40 con 0x7f710815b7d0 2024-04-25T09:57:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.593+0000 7f70effff640 1 -- 172.21.15.19:0/2047484581 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v52) v1 ==== 167+0+0 (secure 0 0 0) 0x7f710007a150 con 0x7f710815b7d0 2024-04-25T09:57:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:enabled new snapshots 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 -- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70f406b4d0 msgr2=0x7f70f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70f406b4d0 0x7f70f406d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000f60 tx=0x7f70f8007c00 comp rx=0 tx=0).stop 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 -- 172.21.15.19:0/2047484581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710815b7d0 msgr2=0x7f7108158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710815b7d0 0x7f7108158090 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f710002ac70 tx=0x7f7100004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 -- 172.21.15.19:0/2047484581 shutdown_connections 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70f406b4d0 0x7f70f406d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f710815b7d0 0x7f7108158090 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71080b5620 0x7f710815d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 --2- 172.21.15.19:0/2047484581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710806c2d0 0x7f710815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 -- 172.21.15.19:0/2047484581 >> 172.21.15.19:0/2047484581 conn(0x7f71080d0800 msgr2=0x7f710815b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 -- 172.21.15.19:0/2047484581 shutdown_connections 2024-04-25T09:57:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:34.596+0000 7f710d953640 1 -- 172.21.15.19:0/2047484581 wait complete. 2024-04-25T09:57:34.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:34.614 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T09:57:34.637 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:57:34.638 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:34.696 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> import uuid 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> n = 2000 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> create_hard_links = False 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> if 3 > 0: 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> create_hard_links = True 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> dpath = os.path.dirname(path) 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> fnameprefix = os.path.basename(path) 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> os.makedirs(dpath, exist_ok=True) 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T09:57:34.697 DEBUG:teuthology.orchestra.run.smithi019:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> for i in range(n): 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> f.write(f"{i}") 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> f.flush() 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> os.fsync(f.fileno()) 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> os.unlink(fpath) 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> os.fsync(dirfd) 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> if create_hard_links: 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> for j in range(3): 2024-04-25T09:57:34.698 DEBUG:teuthology.orchestra.run.smithi019:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T09:57:34.699 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T09:57:34.699 DEBUG:teuthology.orchestra.run.smithi019:> os.fsync(dirfd) 2024-04-25T09:57:34.699 DEBUG:teuthology.orchestra.run.smithi019:> finally: 2024-04-25T09:57:34.699 DEBUG:teuthology.orchestra.run.smithi019:> os.close(dirfd) 2024-04-25T09:57:34.699 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T09:57:37.573 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:37.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:57:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:57:52.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:52.353 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:57:52.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:52.375 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T09:58:07.721 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:07.724 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:19.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:19.518 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:58:19.559 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:58:19.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T09:58:19.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.698+0000 7f8cba9d5640 1 Processor -- start 2024-04-25T09:58:19.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.698+0000 7f8cba9d5640 1 -- start start 2024-04-25T09:58:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.699+0000 7f8cba9d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cb415bb50 0x7f8cb40c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.699+0000 7f8cba9d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb40c70d0 0x7f8cb40cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.699+0000 7f8cba9d5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8cb40cfb40 0x7f8cb4070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.699+0000 7f8cba9d5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb40c8680 con 0x7f8cb415bb50 2024-04-25T09:58:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.699+0000 7f8cba9d5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb40c75e0 con 0x7f8cb40cfb40 2024-04-25T09:58:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.699+0000 7f8cba9d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40c78c0 con 0x7f8cb40c70d0 2024-04-25T09:58:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.699+0000 7f8cb3fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cb415bb50 0x7f8cb40c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.699+0000 7f8cb3fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cb415bb50 0x7f8cb40c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41198/0 (socket says 172.21.15.19:41198) 2024-04-25T09:58:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.699+0000 7f8cb3fff640 1 -- 172.21.15.19:0/476590118 learned_addr learned my addr 172.21.15.19:0/476590118 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.699+0000 7f8cb37fe640 1 --2- 172.21.15.19:0/476590118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb40c70d0 0x7f8cb40cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.699+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/476590118 >> v2:172.21.15.112:3300/0 conn(0x7f8cb40cfb40 0x7f8cb4070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.700+0000 7f8cb3fff640 1 -- 172.21.15.19:0/476590118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb40c70d0 msgr2=0x7f8cb40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.700+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/476590118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb40c70d0 0x7f8cb40cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.700+0000 7f8cb3fff640 1 -- 172.21.15.19:0/476590118 >> v2:172.21.15.112:3300/0 conn(0x7f8cb40cfb40 msgr2=0x7f8cb4070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.700+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/476590118 >> v2:172.21.15.112:3300/0 conn(0x7f8cb40cfb40 0x7f8cb4070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.700+0000 7f8cb3fff640 1 -- 172.21.15.19:0/476590118 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb40c7ba0 con 0x7f8cb415bb50 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.700+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/476590118 >> v2:172.21.15.112:3300/0 conn(0x7f8cb40cfb40 0x7f8cb4070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.700+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/476590118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb415bb50 0x7f8cb40c6b90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40b33e0 tx=0x7f8ca402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1964519bb07b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cb2ffd640 1 -- 172.21.15.19:0/476590118 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4037070 con 0x7f8cb415bb50 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cb2ffd640 1 -- 172.21.15.19:0/476590118 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca4032440 con 0x7f8cb415bb50 2024-04-25T09:58:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cb2ffd640 1 -- 172.21.15.19:0/476590118 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4032810 con 0x7f8cb415bb50 2024-04-25T09:58:19.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cba9d5640 1 -- 172.21.15.19:0/476590118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb415bb50 msgr2=0x7f8cb40c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cba9d5640 1 --2- 172.21.15.19:0/476590118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb415bb50 0x7f8cb40c6b90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40b33e0 tx=0x7f8ca402a270 comp rx=0 tx=0).stop 2024-04-25T09:58:19.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cba9d5640 1 -- 172.21.15.19:0/476590118 shutdown_connections 2024-04-25T09:58:19.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cba9d5640 1 --2- 172.21.15.19:0/476590118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb40c70d0 0x7f8cb40cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cba9d5640 1 --2- 172.21.15.19:0/476590118 >> v2:172.21.15.112:3300/0 conn(0x7f8cb40cfb40 0x7f8cb4070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cba9d5640 1 --2- 172.21.15.19:0/476590118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb415bb50 0x7f8cb40c6b90 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cba9d5640 1 -- 172.21.15.19:0/476590118 >> 172.21.15.19:0/476590118 conn(0x7f8cb406f500 msgr2=0x7f8cb406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cba9d5640 1 -- 172.21.15.19:0/476590118 shutdown_connections 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.701+0000 7f8cba9d5640 1 -- 172.21.15.19:0/476590118 wait complete. 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.702+0000 7f8cba9d5640 1 Processor -- start 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.702+0000 7f8cba9d5640 1 -- start start 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.703+0000 7f8cba9d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb40c70d0 0x7f8cb40c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.703+0000 7f8cba9d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb40cfb40 0x7f8cb40b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.703+0000 7f8cba9d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb415bb50 0x7f8cb414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.703+0000 7f8cba9d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb415f6d0 con 0x7f8cb415bb50 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.703+0000 7f8cba9d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40c8160 con 0x7f8cb40cfb40 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.703+0000 7f8cba9d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb415f850 con 0x7f8cb40c70d0 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.703+0000 7f8cb8f72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb415bb50 0x7f8cb414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.703+0000 7f8cb8f72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb415bb50 0x7f8cb414fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41204/0 (socket says 172.21.15.19:41204) 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.703+0000 7f8cb8f72640 1 -- 172.21.15.19:0/2861047527 learned_addr learned my addr 172.21.15.19:0/2861047527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.703+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb40c70d0 0x7f8cb40c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.703+0000 7f8cb37fe640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb40cfb40 0x7f8cb40b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.704+0000 7f8cb8f72640 1 -- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb40c70d0 msgr2=0x7f8cb40c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.704+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb40c70d0 0x7f8cb40c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.704+0000 7f8cb8f72640 1 -- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb40cfb40 msgr2=0x7f8cb40b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.704+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb40cfb40 0x7f8cb40b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.704+0000 7f8cb8f72640 1 -- 172.21.15.19:0/2861047527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb40b5030 con 0x7f8cb415bb50 2024-04-25T09:58:19.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.704+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb415bb50 0x7f8cb414fa40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8cac007ef0 tx=0x7f8cac00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.704+0000 7f8cb0ff9640 1 -- 172.21.15.19:0/2861047527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac01a070 con 0x7f8cb415bb50 2024-04-25T09:58:19.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.704+0000 7f8cb0ff9640 1 -- 172.21.15.19:0/2861047527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac003b90 con 0x7f8cb415bb50 2024-04-25T09:58:19.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.704+0000 7f8cba9d5640 1 -- 172.21.15.19:0/2861047527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb40b5310 con 0x7f8cb415bb50 2024-04-25T09:58:19.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.704+0000 7f8cb0ff9640 1 -- 172.21.15.19:0/2861047527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac004030 con 0x7f8cb415bb50 2024-04-25T09:58:19.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.704+0000 7f8cba9d5640 1 -- 172.21.15.19:0/2861047527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb40b58d0 con 0x7f8cb415bb50 2024-04-25T09:58:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.705+0000 7f8cb0ff9640 1 -- 172.21.15.19:0/2861047527 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cac01d070 con 0x7f8cb415bb50 2024-04-25T09:58:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.706+0000 7f8cb0ff9640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 0x7f8ca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.706+0000 7f8cb0ff9640 1 -- 172.21.15.19:0/2861047527 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8cac08a5d0 con 0x7f8cb415bb50 2024-04-25T09:58:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.706+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 0x7f8ca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.707+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 0x7f8ca006d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4034000 tx=0x7f8ca402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.707+0000 7f8cbd7b9740 1 Processor -- start 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.707+0000 7f8cbd7b9740 1 -- start start 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217ae7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217ae6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b10be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562217a48b90 con 0x562217b22370 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622179f1e40 con 0x562217ae5ca0 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562217a44c30 con 0x562217b10760 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217ae6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217ae6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41208/0 (socket says 172.21.15.19:41208) 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217ae7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb37fe640 1 -- 172.21.15.19:0/4100314331 learned_addr learned my addr 172.21.15.19:0/4100314331 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/4100314331 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b10be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb37fe640 1 -- 172.21.15.19:0/4100314331 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 msgr2=0x562217b10be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb37fe640 1 --2- 172.21.15.19:0/4100314331 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b10be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb37fe640 1 -- 172.21.15.19:0/4100314331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 msgr2=0x562217ae7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb37fe640 1 --2- 172.21.15.19:0/4100314331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217ae7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb37fe640 1 -- 172.21.15.19:0/4100314331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622178277d0 con 0x562217b22370 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/4100314331 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b10be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.708+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/4100314331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217ae7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8cb37fe640 1 --2- 172.21.15.19:0/4100314331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217ae6080 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009aa0 tx=0x7f8ca8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d7d85ef81fe1d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8c99ffb640 1 -- 172.21.15.19:0/4100314331 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8011cc0 con 0x562217b22370 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8c99ffb640 1 -- 172.21.15.19:0/4100314331 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca8011e60 con 0x562217b22370 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8c99ffb640 1 -- 172.21.15.19:0/4100314331 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8004230 con 0x562217b22370 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/4100314331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 msgr2=0x562217ae6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/4100314331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217ae6080 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009aa0 tx=0x7f8ca8009f60 comp rx=0 tx=0).stop 2024-04-25T09:58:19.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/4100314331 shutdown_connections 2024-04-25T09:58:19.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/4100314331 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b10be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/4100314331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217ae6080 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/4100314331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217ae7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/4100314331 >> 172.21.15.19:0/4100314331 conn(0x562217aad870 msgr2=0x562217b2b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/4100314331 shutdown_connections 2024-04-25T09:58:19.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.709+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/4100314331 wait complete. 2024-04-25T09:58:19.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.710+0000 7f8cbd7b9740 1 -- start start 2024-04-25T09:58:19.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.710+0000 7f8cbd7b9740 1 Processor -- start 2024-04-25T09:58:19.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217affee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b2d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562217a053e0 con 0x562217b22370 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x562217b0e2b0 con 0x562217ae5ca0 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562217b23ed0 con 0x562217b10760 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cbd7b9740 10 client.0 _ll_register_callbacks cb 0x562217a04db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb8f72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b2d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb8f72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b2d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41222/0 (socket says 172.21.15.19:41222) 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb8f72640 1 -- 172.21.15.19:0/2381351610 learned_addr learned my addr 172.21.15.19:0/2381351610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb37fe640 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217affee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb8f72640 1 -- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 msgr2=0x562217b115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b115f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb8f72640 1 -- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 msgr2=0x562217affee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217affee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb8f72640 1 -- 172.21.15.19:0/2381351610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56221771d0f0 con 0x562217b22370 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb37fe640 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217affee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b2d5f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8cac058970 tx=0x7f8cac058db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4daea966552dc169 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.711+0000 7f8c92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8c92ffd640 1 -- 172.21.15.19:0/2381351610 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac0590d0 con 0x562217b22370 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8c92ffd640 1 -- 172.21.15.19:0/2381351610 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8cac060e40 con 0x562217b22370 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2381351610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ca8011e60 con 0x562217b22370 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8c92ffd640 1 -- 172.21.15.19:0/2381351610 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cac061040 con 0x562217b22370 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8cbd7b9740 10 client.7308 fetch_fsmap learned FSMap version 52 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2381351610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x562217887f60 con 0x562217b22370 2024-04-25T09:58:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8c92ffd640 1 -- 172.21.15.19:0/2381351610 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca8011e60 con 0x562217b22370 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8c92ffd640 1 -- 172.21.15.19:0/2381351610 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8cac06cb80 con 0x562217b22370 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8cbd7b9740 10 client.7308 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8cbd7b9740 10 client.7308 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8cbd7b9740 20 client.7308 populate_metadata read hostname 'smithi019' 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x562217b3eda0 0x562217b3f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8cbd7b9740 4 client.7308 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2381351610 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562217a560d0 con 0x562217b3eda0 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.712+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x562217b3eda0 0x562217b3f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8c92ffd640 10 client.7308 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x562217b3eda0 0x562217b3f1b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd3297203431603 server_cookie=66dc77940cecfe89 in_seq=0 out_seq=0 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8c92ffd640 1 -- 172.21.15.19:0/2381351610 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562217a560d0 con 0x562217b3eda0 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8c92ffd640 10 client.7308 handle_command_reply: tid=0 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cbd7b9740 1 client.7308 shutdown 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2381351610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 msgr2=0x562217b2d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b2d5f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8cac058970 tx=0x7f8cac058db0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2381351610 shutdown_connections 2024-04-25T09:58:19.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x562217b3eda0 0x562217b3f1b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b2d5f0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8c92ffd640 0 client.7308 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:19.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/2381351610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217affee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2381351610 >> 172.21.15.19:0/2381351610 conn(0x562217aad870 msgr2=0x562217b0b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2381351610 shutdown_connections 2024-04-25T09:58:19.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.713+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2381351610 wait complete. 2024-04-25T09:58:19.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.714+0000 7f8cbd7b9740 20 client.7308 trim_cache size 0 max 16384 2024-04-25T09:58:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.728+0000 7f8cbd7b9740 1 Processor -- start 2024-04-25T09:58:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.728+0000 7f8cbd7b9740 1 -- start start 2024-04-25T09:58:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.728+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217ae5ca0 0x562217ae4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.728+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217b10760 0x562217b10000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.728+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b22370 0x562217b13c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.728+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb4200580 con 0x562217ae5ca0 2024-04-25T09:58:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.728+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb41ff720 con 0x562217b10760 2024-04-25T09:58:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.728+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40caa70 con 0x562217b22370 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217ae5ca0 0x562217ae4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217ae5ca0 0x562217ae4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41234/0 (socket says 172.21.15.19:41234) 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb3fff640 1 -- 172.21.15.19:0/2660837546 learned_addr learned my addr 172.21.15.19:0/2660837546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb37fe640 1 --2- 172.21.15.19:0/2660837546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217b10760 0x562217b10000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb3fff640 1 -- 172.21.15.19:0/2660837546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b22370 msgr2=0x562217b13c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/2660837546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b22370 0x562217b13c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/2660837546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b22370 0x562217b13c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb3fff640 1 -- 172.21.15.19:0/2660837546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217b10760 msgr2=0x562217b10000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/2660837546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217b10760 0x562217b10000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb3fff640 1 -- 172.21.15.19:0/2660837546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562217b00250 con 0x562217ae5ca0 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/2660837546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b22370 0x562217b13c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/2660837546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217ae5ca0 0x562217ae4460 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8ca403f040 tx=0x7f8ca4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f008196189c830 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8c997fa640 1 -- 172.21.15.19:0/2660837546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4037040 con 0x562217ae5ca0 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8c997fa640 1 -- 172.21.15.19:0/2660837546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca4058070 con 0x562217ae5ca0 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.729+0000 7f8c997fa640 1 -- 172.21.15.19:0/2660837546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40533b0 con 0x562217ae5ca0 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.730+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2660837546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217ae5ca0 msgr2=0x562217ae4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.730+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/2660837546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217ae5ca0 0x562217ae4460 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8ca403f040 tx=0x7f8ca4008030 comp rx=0 tx=0).stop 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.730+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2660837546 shutdown_connections 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.730+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/2660837546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b22370 0x562217b13c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.730+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/2660837546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217b10760 0x562217b10000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.730+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/2660837546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217ae5ca0 0x562217ae4460 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.730+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2660837546 >> 172.21.15.19:0/2660837546 conn(0x562217aad870 msgr2=0x562217b45570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.730+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2660837546 shutdown_connections 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.730+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/2660837546 wait complete. 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.730+0000 7f8cbd7b9740 1 -- start start 2024-04-25T09:58:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.731+0000 7f8cbd7b9740 1 Processor -- start 2024-04-25T09:58:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.731+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217b3e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.731+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b45570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.731+0000 7f8cbd7b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b93b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.731+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562217adb510 con 0x562217b22370 2024-04-25T09:58:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.731+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40cb070 con 0x562217ae5ca0 2024-04-25T09:58:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.731+0000 7f8cbd7b9740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562217b117b0 con 0x562217b10760 2024-04-25T09:58:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.731+0000 7f8cbd7b9740 10 client.0 _ll_register_callbacks cb 0x7f8ca4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.731+0000 7f8cb8f72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b93b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.731+0000 7f8cb8f72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b93b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41242/0 (socket says 172.21.15.19:41242) 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.731+0000 7f8cb8f72640 1 -- 172.21.15.19:0/3646251849 learned_addr learned my addr 172.21.15.19:0/3646251849 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217b3e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8cb37fe640 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b45570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8cb8f72640 1 -- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 msgr2=0x562217b45570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b45570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8cb8f72640 1 -- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 msgr2=0x562217b3e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217b3e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8cb8f72640 1 -- 172.21.15.19:0/3646251849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622176f61b0 con 0x562217b22370 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8cb8f72640 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b93b30 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8cac08a8f0 tx=0x7f8cac017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42cb4a4d55f1aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8c92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8c92ffd640 1 -- 172.21.15.19:0/3646251849 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac0585d0 con 0x562217b22370 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8c92ffd640 1 -- 172.21.15.19:0/3646251849 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8cac058ed0 con 0x562217b22370 2024-04-25T09:58:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/3646251849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ca4058070 con 0x562217b22370 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8c92ffd640 1 -- 172.21.15.19:0/3646251849 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cac0590f0 con 0x562217b22370 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8cbd7b9740 10 client.7314 fetch_fsmap learned FSMap version 52 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/3646251849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x562217afe5e0 con 0x562217b22370 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.732+0000 7f8c92ffd640 1 -- 172.21.15.19:0/3646251849 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca4058070 con 0x562217b22370 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.733+0000 7f8c92ffd640 1 -- 172.21.15.19:0/3646251849 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8cac058110 con 0x562217b22370 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.733+0000 7f8cbd7b9740 10 client.7314 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.733+0000 7f8cbd7b9740 10 client.7314 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.733+0000 7f8cbd7b9740 20 client.7314 populate_metadata read hostname 'smithi019' 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.733+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x562217b13f50 0x562217b966a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.733+0000 7f8cbd7b9740 4 client.7314 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.733+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/3646251849 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x562217b25790 con 0x562217b13f50 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.733+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x562217b13f50 0x562217b966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.733+0000 7f8cb3fff640 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x562217b13f50 0x562217b966a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fb9843eda4d3be0 server_cookie=2a0ce351c00d187d in_seq=0 out_seq=0 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.733+0000 7f8c92ffd640 10 client.7314 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8c92ffd640 1 -- 172.21.15.19:0/3646251849 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17925 (crc 0 0 0) 0x562217b25790 con 0x562217b13f50 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8c92ffd640 10 client.7314 handle_command_reply: tid=0 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 1 client.7314 shutdown 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/3646251849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 msgr2=0x562217b93b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b93b30 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8cac08a8f0 tx=0x7f8cac017040 comp rx=0 tx=0).stop 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/3646251849 shutdown_connections 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x562217b13f50 0x562217b966a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562217b22370 0x562217b93b30 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562217b10760 0x562217b45570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8c92ffd640 0 client.7314 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 1 --2- 172.21.15.19:0/3646251849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562217ae5ca0 0x562217b3e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/3646251849 >> 172.21.15.19:0/3646251849 conn(0x562217aad870 msgr2=0x562217b0a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/3646251849 shutdown_connections 2024-04-25T09:58:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 1 -- 172.21.15.19:0/3646251849 wait complete. 2024-04-25T09:58:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cbd7b9740 20 client.7314 trim_cache size 0 max 16384 2024-04-25T09:58:19.746 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:58:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:58:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1292, 2024-04-25T09:58:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1291, 2024-04-25T09:58:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 528837, 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 17988806, 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 9, 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.156346736, 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.079956968, 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.026806186, 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.040851541, 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1290, 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.344709765, 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000267216 2024-04-25T09:58:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2, 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000131, 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000065 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 28334, 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 50786, 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 9486833, 2024-04-25T09:58:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 57605511, 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 1.878598078, 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.929464991, 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 1.123234784, 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.079842833, 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 50786, 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.217128629, 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000023965 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:58:19.749 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 73674, 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.007872620, 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000106 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 116944, 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 17856 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2401, 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2415, 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 595871, 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 42829991, 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-04-25T09:58:19.750 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.267308721, 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.148658787, 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.038889391, 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.068125093, 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2414, 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.256817556, 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000106386 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.751 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 106592 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1121, 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 14.759709305, 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.013166556 2024-04-25T09:58:19.752 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2095, 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.676576961, 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000322948 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 25516, 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 25516, 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 25516, 2024-04-25T09:58:19.753 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 293.395269380, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.011498482 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 12, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 19803, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 19803, 2024-04-25T09:58:19.754 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 19803, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 159, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 159, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 2, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 46863, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 46453, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 118, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-04-25T09:58:19.755 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:58:19.756 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 1114, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 4041, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 3937, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 3333, 2024-04-25T09:58:19.757 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 51, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 13995 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 2000, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 5, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 4048, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 1843, 2024-04-25T09:58:19.758 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 2132, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 25232, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 25232, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 38, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 38, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 4, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-04-25T09:58:19.760 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 96773678, 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 0, 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2425, 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 82.821951975, 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.034153382 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 0 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 2014, 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 2014, 2024-04-25T09:58:19.761 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 13, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 13, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 19803, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 19804, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 1, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 159, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 7215, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 7056, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 116280, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215396 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-04-25T09:58:19.762 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 28633, 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 25516, 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 3, 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2000, 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.289818615, 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000644909 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 29, 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.005479571, 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000188950 2024-04-25T09:58:19.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 6000, 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 6.656838964, 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001109473 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 7342, 2024-04-25T09:58:19.764 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 6.145017641, 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000836967 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.765 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000051334, 2024-04-25T09:58:19.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000051334 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.002760967, 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.002760967 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-04-25T09:58:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.003549779, 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.003549779 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.768 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8, 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.429034619, 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.178629327 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2132, 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 262.174218910, 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.122971022 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.769 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:58:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.023007362, 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.023007362 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000249627, 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000249627 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.771 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8000, 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 15.665241991, 2024-04-25T09:58:19.773 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001958155 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 1, 2024-04-25T09:58:19.774 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 18643, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 18643, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 43, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:58:19.775 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:58:19.776 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:58:19.777 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 16361756, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 6101, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 440, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 5, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 5424, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 76, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:58:19.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 20786259, 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 872973, 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-04-25T09:58:19.779 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 4495, 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 93051982, 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 37, 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 4495, 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4495, 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 20.490461992, 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.004558500 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4495, 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 4585 2024-04-25T09:58:19.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 4495, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 47, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 4448, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 0, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 11, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 47, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 2472, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 30, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-04-25T09:58:19.781 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 1843, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 34, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 0, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 22, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-04-25T09:58:19.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-04-25T09:58:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 81, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 1, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 8, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 8, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-04-25T09:58:19.784 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 99, 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 0, 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 27 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 1, 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.785 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 32, 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 1843, 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 1843, 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:58:19.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 32021, 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 7724584, 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 32021, 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 32021, 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 7724584, 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.787 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 4495, 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 93252869, 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 4495, 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 93252869, 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:58:19.788 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 4495, 2024-04-25T09:58:19.789 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 4495, 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 4495, 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 4495, 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:58:19.790 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 1843, 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 200887, 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 1843, 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 47, 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 200887, 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:19.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55952ea786e0": { 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 84, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 25232, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 92579374, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 25232, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2425, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 92579290, 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:58:19.792 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:19.793 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:19.793 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:19.793 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:19.793 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:19.793 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:58:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cba9d5640 1 -- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 msgr2=0x7f8ca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cba9d5640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 0x7f8ca006d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4034000 tx=0x7f8ca402e000 comp rx=0 tx=0).stop 2024-04-25T09:58:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cba9d5640 1 -- 172.21.15.19:0/2861047527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb415bb50 msgr2=0x7f8cb414fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.734+0000 7f8cba9d5640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb415bb50 0x7f8cb414fa40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8cac007ef0 tx=0x7f8cac00d4a0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.735+0000 7f8cba9d5640 1 -- 172.21.15.19:0/2861047527 shutdown_connections 2024-04-25T09:58:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.735+0000 7f8cba9d5640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 0x7f8ca006d990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.735+0000 7f8cba9d5640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb415bb50 0x7f8cb414fa40 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.735+0000 7f8cba9d5640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb40cfb40 0x7f8cb40b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.735+0000 7f8cba9d5640 1 --2- 172.21.15.19:0/2861047527 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb40c70d0 0x7f8cb40c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.735+0000 7f8cba9d5640 1 -- 172.21.15.19:0/2861047527 >> 172.21.15.19:0/2861047527 conn(0x7f8cb406f500 msgr2=0x7f8cb4070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.735+0000 7f8cba9d5640 1 -- 172.21.15.19:0/2861047527 shutdown_connections 2024-04-25T09:58:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.735+0000 7f8cba9d5640 1 -- 172.21.15.19:0/2861047527 wait complete. 2024-04-25T09:58:19.796 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T09:58:19.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:19.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T09:58:19.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.897+0000 7fa2cb7e2640 1 Processor -- start 2024-04-25T09:58:19.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.898+0000 7fa2cb7e2640 1 -- start start 2024-04-25T09:58:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.898+0000 7fa2cb7e2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2c4161240 0x7fa2c40b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.898+0000 7fa2cb7e2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa2c40b54a0 0x7fa2c40b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.898+0000 7fa2cb7e2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c40cf1d0 con 0x7fa2c4161890 2024-04-25T09:58:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.898+0000 7fa2cb7e2640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c40b7120 con 0x7fa2c4161240 2024-04-25T09:58:19.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.898+0000 7fa2cb7e2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c40b5e30 con 0x7fa2c40b54a0 2024-04-25T09:58:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.898+0000 7fa2c957e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2c4161240 0x7fa2c40b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.898+0000 7fa2c8d7d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa2c4161890 0x7fa2c40ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46062/0 (socket says 172.21.15.19:46062) 2024-04-25T09:58:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.899+0000 7fa2c8d7d640 1 -- 172.21.15.19:0/1416024533 learned_addr learned my addr 172.21.15.19:0/1416024533 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.899+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/1416024533 >> v2:172.21.15.112:3300/0 conn(0x7fa2c40b54a0 0x7fa2c40b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.899+0000 7fa2c957e640 1 -- 172.21.15.19:0/1416024533 >> v2:172.21.15.112:3300/0 conn(0x7fa2c40b54a0 msgr2=0x7fa2c40b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.899+0000 7fa2c957e640 1 --2- 172.21.15.19:0/1416024533 >> v2:172.21.15.112:3300/0 conn(0x7fa2c40b54a0 0x7fa2c40b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.899+0000 7fa2c957e640 1 -- 172.21.15.19:0/1416024533 >> v1:172.21.15.19:6789/0 conn(0x7fa2c4161890 legacy=0x7fa2c40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.899+0000 7fa2c957e640 1 -- 172.21.15.19:0/1416024533 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c4161620 con 0x7fa2c4161240 2024-04-25T09:58:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.899+0000 7fa2c957e640 1 --2- 172.21.15.19:0/1416024533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4161240 0x7fa2c40b6be0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa2c408ce10 tx=0x7fa2c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b68451947abfbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.900+0000 7fa2bbfff640 1 -- 172.21.15.19:0/1416024533 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0037070 con 0x7fa2c4161240 2024-04-25T09:58:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.900+0000 7fa2bbfff640 1 -- 172.21.15.19:0/1416024533 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c0032440 con 0x7fa2c4161240 2024-04-25T09:58:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.900+0000 7fa2bbfff640 1 -- 172.21.15.19:0/1416024533 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0032810 con 0x7fa2c4161240 2024-04-25T09:58:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.900+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1416024533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4161240 msgr2=0x7fa2c40b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.900+0000 7fa2cb7e2640 1 --2- 172.21.15.19:0/1416024533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4161240 0x7fa2c40b6be0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa2c408ce10 tx=0x7fa2c002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.900+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1416024533 shutdown_connections 2024-04-25T09:58:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.900+0000 7fa2cb7e2640 1 --2- 172.21.15.19:0/1416024533 >> v2:172.21.15.112:3300/0 conn(0x7fa2c40b54a0 0x7fa2c40b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.900+0000 7fa2cb7e2640 1 --2- 172.21.15.19:0/1416024533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4161240 0x7fa2c40b6be0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.900+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1416024533 >> 172.21.15.19:0/1416024533 conn(0x7fa2c40d04a0 msgr2=0x7fa2c40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.900+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1416024533 shutdown_connections 2024-04-25T09:58:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.900+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1416024533 wait complete. 2024-04-25T09:58:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.901+0000 7fa2cb7e2640 1 Processor -- start 2024-04-25T09:58:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.902+0000 7fa2cb7e2640 1 -- start start 2024-04-25T09:58:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.902+0000 7fa2cb7e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa2c40b54a0 0x7fa2c4152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.902+0000 7fa2cb7e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2c4161240 0x7fa2c415a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.902+0000 7fa2cb7e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4161890 0x7fa2c415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.902+0000 7fa2cb7e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41623c0 con 0x7fa2c4161890 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.902+0000 7fa2cb7e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41620e0 con 0x7fa2c4161240 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.902+0000 7fa2cb7e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41626a0 con 0x7fa2c40b54a0 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.902+0000 7fa2c9d7f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4161890 0x7fa2c415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.902+0000 7fa2c9d7f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4161890 0x7fa2c415ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41264/0 (socket says 172.21.15.19:41264) 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.902+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/1440247006 learned_addr learned my addr 172.21.15.19:0/1440247006 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.902+0000 7fa2c8d7d640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2c4161240 0x7fa2c415a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2c957e640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa2c40b54a0 0x7fa2c4152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa2c40b54a0 msgr2=0x7fa2c4152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa2c40b54a0 0x7fa2c4152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2c4161240 msgr2=0x7fa2c415a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2c4161240 0x7fa2c415a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/1440247006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40c97b0 con 0x7fa2c4161890 2024-04-25T09:58:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4161890 0x7fa2c415ac10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc007e70 tx=0x7fa2bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2b9ffb640 1 -- 172.21.15.19:0/1440247006 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc014420 con 0x7fa2c4161890 2024-04-25T09:58:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2b9ffb640 1 -- 172.21.15.19:0/1440247006 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc019070 con 0x7fa2c4161890 2024-04-25T09:58:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1440247006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c40c9a90 con 0x7fa2c4161890 2024-04-25T09:58:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2b9ffb640 1 -- 172.21.15.19:0/1440247006 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc014820 con 0x7fa2c4161890 2024-04-25T09:58:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.903+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1440247006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c40ca050 con 0x7fa2c4161890 2024-04-25T09:58:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.905+0000 7fa2b9ffb640 1 -- 172.21.15.19:0/1440247006 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa2bc0149c0 con 0x7fa2c4161890 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.905+0000 7fa2b9ffb640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa2a806b500 0x7fa2a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.905+0000 7fa2b9ffb640 1 -- 172.21.15.19:0/1440247006 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa2bc08ab40 con 0x7fa2c4161890 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.905+0000 7fa2c957e640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa2a806b500 0x7fa2a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.906+0000 7fa2c957e640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa2a806b500 0x7fa2a806d9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0034000 tx=0x7fa2c002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.906+0000 7fa2ce5bd740 1 Processor -- start 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.906+0000 7fa2ce5bd740 1 -- start start 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bcbd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 0x56421bc8d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bc8d740 0x56421bcb5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421bbf60a0 con 0x56421bc8d740 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421bb9ed40 con 0x56421bcbd080 2024-04-25T09:58:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56421bbf2140 con 0x56421bcbdaa0 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2c9d7f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bc8d740 0x56421bcb5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2c9d7f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bc8d740 0x56421bcb5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41268/0 (socket says 172.21.15.19:41268) 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/2346310631 learned_addr learned my addr 172.21.15.19:0/2346310631 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2c957e640 1 --2- 172.21.15.19:0/2346310631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bcbd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2c8d7d640 1 --2- 172.21.15.19:0/2346310631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 0x56421bc8d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/2346310631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 msgr2=0x56421bc8d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/2346310631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 0x56421bc8d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/2346310631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 msgr2=0x56421bcbd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/2346310631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bcbd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/2346310631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56421b9d6a20 con 0x56421bc8d740 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.907+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/2346310631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bc8d740 0x56421bcb5500 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00c040 tx=0x7fa2bc059140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f0c952c2a222cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2aeffd640 1 -- 172.21.15.19:0/2346310631 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc057440 con 0x56421bc8d740 2024-04-25T09:58:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2aeffd640 1 -- 172.21.15.19:0/2346310631 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc0557b0 con 0x56421bc8d740 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2aeffd640 1 -- 172.21.15.19:0/2346310631 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc061530 con 0x56421bc8d740 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/2346310631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bc8d740 msgr2=0x56421bcb5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/2346310631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bc8d740 0x56421bcb5500 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00c040 tx=0x7fa2bc059140 comp rx=0 tx=0).stop 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/2346310631 shutdown_connections 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/2346310631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bc8d740 0x56421bcb5500 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/2346310631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 0x56421bc8d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/2346310631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bcbd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/2346310631 >> 172.21.15.19:0/2346310631 conn(0x56421bc8f470 msgr2=0x56421bcd7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/2346310631 shutdown_connections 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.908+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/2346310631 wait complete. 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.909+0000 7fa2ce5bd740 1 -- start start 2024-04-25T09:58:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.909+0000 7fa2ce5bd740 1 Processor -- start 2024-04-25T09:58:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bcc0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 0x56421bcb4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcb6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421bcb97d0 con 0x56421bcb4db0 2024-04-25T09:58:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421bb943d0 con 0x56421bcbd080 2024-04-25T09:58:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56421bcb9a70 con 0x56421bcbdaa0 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2ce5bd740 10 client.0 _ll_register_callbacks cb 0x56421b909e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2c9d7f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcb6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2c9d7f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcb6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41274/0 (socket says 172.21.15.19:41274) 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/4283646146 learned_addr learned my addr 172.21.15.19:0/4283646146 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2c8d7d640 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 0x56421bcb4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2c957e640 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bcc0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 msgr2=0x56421bcb4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 0x56421bcb4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 msgr2=0x56421bcc0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bcc0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/4283646146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56421b8ca310 con 0x56421bcb4db0 2024-04-25T09:58:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcb6980 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0555a0 tx=0x7fa2bc016070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335e977cc0dd7920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2a3fff640 1 -- 172.21.15.19:0/4283646146 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc074070 con 0x56421bcb4db0 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2a3fff640 1 -- 172.21.15.19:0/4283646146 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa2bc061510 con 0x56421bcb4db0 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.910+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/4283646146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2bc0557b0 con 0x56421bcb4db0 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2a3fff640 1 -- 172.21.15.19:0/4283646146 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2bc07f050 con 0x56421bcb4db0 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2ce5bd740 10 client.7329 fetch_fsmap learned FSMap version 52 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/4283646146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x56421ba371b0 con 0x56421bcb4db0 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2a3fff640 1 -- 172.21.15.19:0/4283646146 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc0557b0 con 0x56421bcb4db0 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2a3fff640 1 -- 172.21.15.19:0/4283646146 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa2bc082020 con 0x56421bcb4db0 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2ce5bd740 10 client.7329 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2ce5bd740 10 client.7329 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2ce5bd740 20 client.7329 populate_metadata read hostname 'smithi019' 2024-04-25T09:58:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x56421bcf2910 0x56421bcf4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2ce5bd740 4 client.7329 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/4283646146 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56421bc035e0 con 0x56421bcf2910 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.911+0000 7fa2c957e640 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x56421bcf2910 0x56421bcf4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2c957e640 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x56421bcf2910 0x56421bcf4d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f363d9f7f36ab57b server_cookie=79fbb0ce0e3c4aea in_seq=0 out_seq=0 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2a3fff640 10 client.7329 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2a3fff640 1 -- 172.21.15.19:0/4283646146 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56421bc035e0 con 0x56421bcf2910 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2a3fff640 10 client.7329 handle_command_reply: tid=0 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2ce5bd740 1 client.7329 shutdown 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/4283646146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 msgr2=0x56421bcb6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcb6980 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0555a0 tx=0x7fa2bc016070 comp rx=0 tx=0).stop 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/4283646146 shutdown_connections 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x56421bcf2910 0x56421bcf4d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcb6980 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 0x56421bcb4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2a3fff640 0 client.7329 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/4283646146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bcc0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/4283646146 >> 172.21.15.19:0/4283646146 conn(0x56421bc8d740 msgr2=0x56421bc8c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/4283646146 shutdown_connections 2024-04-25T09:58:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.912+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/4283646146 wait complete. 2024-04-25T09:58:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.913+0000 7fa2ce5bd740 20 client.7329 trim_cache size 0 max 16384 2024-04-25T09:58:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.928+0000 7fa2ce5bd740 1 Processor -- start 2024-04-25T09:58:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.928+0000 7fa2ce5bd740 1 -- start start 2024-04-25T09:58:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.928+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcf8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.928+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bc8e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 0x56421bcd7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421bc90380 con 0x56421bcb4db0 2024-04-25T09:58:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c42006a0 con 0x56421bcbd080 2024-04-25T09:58:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41ff860 con 0x56421bcbdaa0 2024-04-25T09:58:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2c957e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcf8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2c957e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcf8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41290/0 (socket says 172.21.15.19:41290) 2024-04-25T09:58:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2c957e640 1 -- 172.21.15.19:0/489963846 learned_addr learned my addr 172.21.15.19:0/489963846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2c957e640 1 -- 172.21.15.19:0/489963846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 msgr2=0x56421bcd7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2c957e640 1 --2- 172.21.15.19:0/489963846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 0x56421bcd7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2c957e640 1 -- 172.21.15.19:0/489963846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 msgr2=0x56421bc8e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2c957e640 1 --2- 172.21.15.19:0/489963846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bc8e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2c957e640 1 -- 172.21.15.19:0/489963846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56421bcc05f0 con 0x56421bcb4db0 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2c957e640 1 --2- 172.21.15.19:0/489963846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcf8e90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0035040 tx=0x7fa2c002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e00d6b17a25f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2ae7fc640 1 -- 172.21.15.19:0/489963846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0037040 con 0x56421bcb4db0 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2ae7fc640 1 -- 172.21.15.19:0/489963846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c0051bc0 con 0x56421bcb4db0 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.929+0000 7fa2ae7fc640 1 -- 172.21.15.19:0/489963846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0051d70 con 0x56421bcb4db0 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.930+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/489963846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 msgr2=0x56421bcf8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.930+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/489963846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcf8e90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0035040 tx=0x7fa2c002c000 comp rx=0 tx=0).stop 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.930+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/489963846 shutdown_connections 2024-04-25T09:58:19.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.930+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/489963846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcbdaa0 0x56421bcd7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.930+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/489963846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bc8e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.930+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/489963846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcb4db0 0x56421bcf8e90 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.930+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/489963846 >> 172.21.15.19:0/489963846 conn(0x56421bc8d740 msgr2=0x56421bcb4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.930+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/489963846 shutdown_connections 2024-04-25T09:58:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.930+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/489963846 wait complete. 2024-04-25T09:58:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.930+0000 7fa2ce5bd740 1 -- start start 2024-04-25T09:58:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.930+0000 7fa2ce5bd740 1 Processor -- start 2024-04-25T09:58:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.931+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcb4db0 0x56421bd28540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.931+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bcba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.931+0000 7fa2ce5bd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcbdaa0 0x56421bcbac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.931+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41ff260 con 0x56421bcbdaa0 2024-04-25T09:58:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.931+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41f4cb0 con 0x56421bcbd080 2024-04-25T09:58:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.931+0000 7fa2ce5bd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56421bc8d460 con 0x56421bcb4db0 2024-04-25T09:58:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.931+0000 7fa2ce5bd740 10 client.0 _ll_register_callbacks cb 0x7fa2bc055460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.931+0000 7fa2c9d7f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcbdaa0 0x56421bcbac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.931+0000 7fa2c9d7f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcbdaa0 0x56421bcbac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41304/0 (socket says 172.21.15.19:41304) 2024-04-25T09:58:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.931+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/443498750 learned_addr learned my addr 172.21.15.19:0/443498750 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2c957e640 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcb4db0 0x56421bd28540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcb4db0 msgr2=0x56421bd28540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcb4db0 0x56421bd28540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 msgr2=0x56421bcba730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bcba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2c9d7f640 1 -- 172.21.15.19:0/443498750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56421bcb8690 con 0x56421bcbdaa0 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2c957e640 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcb4db0 0x56421bd28540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2c9d7f640 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcbdaa0 0x56421bcbac70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0590d0 tx=0x7fa2bc088d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b17cfbb0461a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2a3fff640 1 -- 172.21.15.19:0/443498750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc065030 con 0x56421bcbdaa0 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2a3fff640 1 -- 172.21.15.19:0/443498750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa2bc05f180 con 0x56421bcbdaa0 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/443498750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2c0051bc0 con 0x56421bcbdaa0 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2a3fff640 1 -- 172.21.15.19:0/443498750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2bc074040 con 0x56421bcbdaa0 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2ce5bd740 10 client.7335 fetch_fsmap learned FSMap version 52 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/443498750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x56421b8a3490 con 0x56421bcbdaa0 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2a3fff640 1 -- 172.21.15.19:0/443498750 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c0051bc0 con 0x56421bcbdaa0 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2a3fff640 1 -- 172.21.15.19:0/443498750 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa2bc082030 con 0x56421bcbdaa0 2024-04-25T09:58:19.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2ce5bd740 10 client.7335 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:58:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2ce5bd740 10 client.7335 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:58:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2ce5bd740 20 client.7335 populate_metadata read hostname 'smithi019' 2024-04-25T09:58:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x56421bcbc150 0x56421bcdb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2ce5bd740 4 client.7335 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T09:58:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.932+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/443498750 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x56421bcc0d30 con 0x56421bcbc150 2024-04-25T09:58:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.933+0000 7fa2c957e640 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x56421bcbc150 0x56421bcdb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.933+0000 7fa2c957e640 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x56421bcbc150 0x56421bcdb280 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=421d97bbc134191d server_cookie=325c45e7c672a405 in_seq=0 out_seq=0 2024-04-25T09:58:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.933+0000 7fa2a3fff640 10 client.7335 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2a3fff640 1 -- 172.21.15.19:0/443498750 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56421bcc0d30 con 0x56421bcbc150 2024-04-25T09:58:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2a3fff640 10 client.7335 handle_command_reply: tid=0 2024-04-25T09:58:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 1 client.7335 shutdown 2024-04-25T09:58:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/443498750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcbdaa0 msgr2=0x56421bcbac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcbdaa0 0x56421bcbac70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0590d0 tx=0x7fa2bc088d10 comp rx=0 tx=0).stop 2024-04-25T09:58:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/443498750 shutdown_connections 2024-04-25T09:58:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x56421bcbc150 0x56421bcdb280 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56421bcbdaa0 0x56421bcbac70 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421bcbd080 0x56421bcba730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 1 --2- 172.21.15.19:0/443498750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56421bcb4db0 0x56421bd28540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2a3fff640 0 client.7335 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:19.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/443498750 >> 172.21.15.19:0/443498750 conn(0x56421bc8d740 msgr2=0x56421bcd7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/443498750 shutdown_connections 2024-04-25T09:58:19.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 1 -- 172.21.15.19:0/443498750 wait complete. 2024-04-25T09:58:19.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.945+0000 7fa2ce5bd740 20 client.7335 trim_cache size 0 max 16384 2024-04-25T09:58:19.950 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:58:19.950 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:58:19.950 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "fb457f1b-be70-4eb2-be88-3cfff741377a", 2024-04-25T09:58:19.950 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:58:19.950 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:58:19.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa2a806b500 msgr2=0x7fa2a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa2a806b500 0x7fa2a806d9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0034000 tx=0x7fa2c002e000 comp rx=0 tx=0).stop 2024-04-25T09:58:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1440247006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4161890 msgr2=0x7fa2c415ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4161890 0x7fa2c415ac10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc007e70 tx=0x7fa2bc00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1440247006 shutdown_connections 2024-04-25T09:58:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa2a806b500 0x7fa2a806d9c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4161890 0x7fa2c415ac10 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2c4161240 0x7fa2c415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 --2- 172.21.15.19:0/1440247006 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa2c40b54a0 0x7fa2c4152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1440247006 >> 172.21.15.19:0/1440247006 conn(0x7fa2c40d04a0 msgr2=0x7fa2c406c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1440247006 shutdown_connections 2024-04-25T09:58:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:19.946+0000 7fa2cb7e2640 1 -- 172.21.15.19:0/1440247006 wait complete. 2024-04-25T09:58:19.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:19.960 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:58:20.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.082+0000 7fd61ffe0640 1 Processor -- start 2024-04-25T09:58:20.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.082+0000 7fd61ffe0640 1 -- start start 2024-04-25T09:58:20.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.083+0000 7fd61ffe0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd61815bb50 0x7fd6180c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.083+0000 7fd61ffe0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd6180c70d0 0x7fd6180cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.083+0000 7fd61ffe0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6180c75e0 con 0x7fd6180cfb40 2024-04-25T09:58:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.083+0000 7fd61ffe0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6180c8680 con 0x7fd6180c70d0 2024-04-25T09:58:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.083+0000 7fd61ffe0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180c78c0 con 0x7fd61815bb50 2024-04-25T09:58:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.084+0000 7fd61d57b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd6180c70d0 0x7fd6180cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.084+0000 7fd61e57d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd6180cfb40 0x7fd618070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46076/0 (socket says 172.21.15.19:46076) 2024-04-25T09:58:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.084+0000 7fd61e57d640 1 -- 172.21.15.19:0/3505803220 learned_addr learned my addr 172.21.15.19:0/3505803220 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.084+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/3505803220 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd61815bb50 0x7fd6180c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.084+0000 7fd61d57b640 1 -- 172.21.15.19:0/3505803220 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd61815bb50 msgr2=0x7fd6180c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.084+0000 7fd61d57b640 1 --2- 172.21.15.19:0/3505803220 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd61815bb50 0x7fd6180c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.084+0000 7fd61d57b640 1 -- 172.21.15.19:0/3505803220 >> v1:172.21.15.19:6789/0 conn(0x7fd6180cfb40 legacy=0x7fd618070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:20.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.084+0000 7fd61d57b640 1 -- 172.21.15.19:0/3505803220 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6180c7ba0 con 0x7fd6180c70d0 2024-04-25T09:58:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.085+0000 7fd61d57b640 1 --2- 172.21.15.19:0/3505803220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6180c70d0 0x7fd6180cf600 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd6080097a0 tx=0x7fd60802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6202e82fd39a97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.085+0000 7fd61cd7a640 1 -- 172.21.15.19:0/3505803220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608037070 con 0x7fd6180c70d0 2024-04-25T09:58:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.085+0000 7fd61cd7a640 1 -- 172.21.15.19:0/3505803220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60802ada0 con 0x7fd6180c70d0 2024-04-25T09:58:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.085+0000 7fd61cd7a640 1 -- 172.21.15.19:0/3505803220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608032660 con 0x7fd6180c70d0 2024-04-25T09:58:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.086+0000 7fd61ffe0640 1 -- 172.21.15.19:0/3505803220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6180c70d0 msgr2=0x7fd6180cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.086+0000 7fd61ffe0640 1 --2- 172.21.15.19:0/3505803220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6180c70d0 0x7fd6180cf600 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd6080097a0 tx=0x7fd60802a110 comp rx=0 tx=0).stop 2024-04-25T09:58:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.086+0000 7fd61ffe0640 1 -- 172.21.15.19:0/3505803220 shutdown_connections 2024-04-25T09:58:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.086+0000 7fd61ffe0640 1 --2- 172.21.15.19:0/3505803220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6180c70d0 0x7fd6180cf600 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.086+0000 7fd61ffe0640 1 --2- 172.21.15.19:0/3505803220 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd61815bb50 0x7fd6180c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.086+0000 7fd61ffe0640 1 -- 172.21.15.19:0/3505803220 >> 172.21.15.19:0/3505803220 conn(0x7fd61806f500 msgr2=0x7fd61806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.086+0000 7fd61ffe0640 1 -- 172.21.15.19:0/3505803220 shutdown_connections 2024-04-25T09:58:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.086+0000 7fd61ffe0640 1 -- 172.21.15.19:0/3505803220 wait complete. 2024-04-25T09:58:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.087+0000 7fd61ffe0640 1 Processor -- start 2024-04-25T09:58:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.087+0000 7fd61ffe0640 1 -- start start 2024-04-25T09:58:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.088+0000 7fd61ffe0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd61806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.088+0000 7fd61ffe0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6180cfb40 0x7fd6181fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.088+0000 7fd61ffe0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd6180b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.088+0000 7fd61ffe0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61815f6d0 con 0x7fd61815bb50 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.088+0000 7fd61ffe0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180c8160 con 0x7fd6180cfb40 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.088+0000 7fd61ffe0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61815f850 con 0x7fd6180c70d0 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.088+0000 7fd61dd7c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd61806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.088+0000 7fd61dd7c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd61806ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:58788/0 (socket says 172.21.15.19:58788) 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.088+0000 7fd61e57d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd6180b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.088+0000 7fd61dd7c640 1 -- 172.21.15.19:0/2864032357 learned_addr learned my addr 172.21.15.19:0/2864032357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd61d57b640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6180cfb40 0x7fd6181fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd61dd7c640 1 -- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6180cfb40 msgr2=0x7fd6181fbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6180cfb40 0x7fd6181fbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd61dd7c640 1 -- 172.21.15.19:0/2864032357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 msgr2=0x7fd6180b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd6180b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd61dd7c640 1 -- 172.21.15.19:0/2864032357 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61806d050 con 0x7fd6180c70d0 2024-04-25T09:58:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd61e57d640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd6180b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd61806ca70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e870 tx=0x7fd61400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd60e7fc640 1 -- 172.21.15.19:0/2864032357 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614017070 con 0x7fd6180c70d0 2024-04-25T09:58:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd60e7fc640 1 -- 172.21.15.19:0/2864032357 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61400c940 con 0x7fd6180c70d0 2024-04-25T09:58:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd61ffe0640 1 -- 172.21.15.19:0/2864032357 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd618157b30 con 0x7fd6180c70d0 2024-04-25T09:58:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.089+0000 7fd60e7fc640 1 -- 172.21.15.19:0/2864032357 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61400cd30 con 0x7fd6180c70d0 2024-04-25T09:58:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.090+0000 7fd61ffe0640 1 -- 172.21.15.19:0/2864032357 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6181580f0 con 0x7fd6180c70d0 2024-04-25T09:58:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.091+0000 7fd60e7fc640 1 -- 172.21.15.19:0/2864032357 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd61401a070 con 0x7fd6180c70d0 2024-04-25T09:58:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.092+0000 7fd60e7fc640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5fc06b4d0 0x7fd5fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.092+0000 7fd60e7fc640 1 -- 172.21.15.19:0/2864032357 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd61408a560 con 0x7fd6180c70d0 2024-04-25T09:58:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.092+0000 7fd61d57b640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5fc06b4d0 0x7fd5fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.093+0000 7fd61d57b640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5fc06b4d0 0x7fd5fc06d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd608034000 tx=0x7fd60802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.094+0000 7fd622dc3740 1 Processor -- start 2024-04-25T09:58:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.094+0000 7fd622dc3740 1 -- start start 2024-04-25T09:58:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.094+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912b10 0x55ac9a915320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.094+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a915860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.094+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a8e5770 0x55ac9a8e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.094+0000 7fd622dc3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9a84eda0 con 0x55ac9a912b10 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd622dc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9a7f7b80 con 0x55ac9a8e5770 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd622dc3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac9a84ae40 con 0x55ac9a912ef0 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd61dd7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912b10 0x55ac9a915320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd61dd7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912b10 0x55ac9a915320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41312/0 (socket says 172.21.15.19:41312) 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd61dd7c640 1 -- 172.21.15.19:0/2769083864 learned_addr learned my addr 172.21.15.19:0/2769083864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd61d57b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a915860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd61e57d640 1 --2- 172.21.15.19:0/2769083864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a8e5770 0x55ac9a8e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd61dd7c640 1 -- 172.21.15.19:0/2769083864 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 msgr2=0x55ac9a915860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/2769083864 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a915860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd61dd7c640 1 -- 172.21.15.19:0/2769083864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a8e5770 msgr2=0x55ac9a8e8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/2769083864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a8e5770 0x55ac9a8e8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd61dd7c640 1 -- 172.21.15.19:0/2769083864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac9a801f10 con 0x55ac9a912b10 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.095+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/2769083864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912b10 0x55ac9a915320 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd614055060 tx=0x7fd614058560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983d26014c2228a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd5fb7fe640 1 -- 172.21.15.19:0/2769083864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614021050 con 0x55ac9a912b10 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd5fb7fe640 1 -- 172.21.15.19:0/2769083864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd614061260 con 0x55ac9a912b10 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd5fb7fe640 1 -- 172.21.15.19:0/2769083864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614055270 con 0x55ac9a912b10 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd622dc3740 1 -- 172.21.15.19:0/2769083864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912b10 msgr2=0x55ac9a915320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2769083864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912b10 0x55ac9a915320 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd614055060 tx=0x7fd614058560 comp rx=0 tx=0).stop 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd622dc3740 1 -- 172.21.15.19:0/2769083864 shutdown_connections 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2769083864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a8e5770 0x55ac9a8e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2769083864 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a915860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2769083864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912b10 0x55ac9a915320 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd622dc3740 1 -- 172.21.15.19:0/2769083864 >> 172.21.15.19:0/2769083864 conn(0x55ac9a8e79e0 msgr2=0x55ac9a930320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd622dc3740 1 -- 172.21.15.19:0/2769083864 shutdown_connections 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.096+0000 7fd622dc3740 1 -- 172.21.15.19:0/2769083864 wait complete. 2024-04-25T09:58:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.097+0000 7fd622dc3740 1 -- start start 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.097+0000 7fd622dc3740 1 Processor -- start 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a8fab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a9170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a909f40 0x55ac9a9175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd622dc3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9a84eda0 con 0x55ac9a909f40 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd622dc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9a7ed4b0 con 0x55ac9a912b10 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd622dc3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac9a8eab90 con 0x55ac9a912ef0 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd622dc3740 10 client.0 _ll_register_callbacks cb 0x55ac9a82d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd61e57d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a909f40 0x55ac9a9175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd61e57d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a909f40 0x55ac9a9175e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41322/0 (socket says 172.21.15.19:41322) 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd61e57d640 1 -- 172.21.15.19:0/2095883066 learned_addr learned my addr 172.21.15.19:0/2095883066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd61d57b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a9170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.098+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a8fab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.099+0000 7fd61d57b640 1 -- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 msgr2=0x55ac9a8fab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.099+0000 7fd61d57b640 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a8fab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.099+0000 7fd61d57b640 1 -- 172.21.15.19:0/2095883066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a909f40 msgr2=0x55ac9a9175e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.099+0000 7fd61d57b640 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a909f40 0x55ac9a9175e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.099+0000 7fd61d57b640 1 -- 172.21.15.19:0/2095883066 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac9a62d7d0 con 0x55ac9a912ef0 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.099+0000 7fd61d57b640 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a9170a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd608031000 tx=0x7fd608004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16882859b00210c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.099+0000 7fd5f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3301/0 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.099+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/2095883066 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608037040 con 0x55ac9a912ef0 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.099+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/2095883066 <== mon.2 v2:172.21.15.112:3301/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd608054480 con 0x55ac9a912ef0 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd622dc3740 1 -- 172.21.15.19:0/2095883066 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd614061260 con 0x55ac9a912ef0 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/2095883066 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd608054680 con 0x55ac9a912ef0 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd622dc3740 10 client.6056 fetch_fsmap learned FSMap version 52 2024-04-25T09:58:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd622dc3740 1 -- 172.21.15.19:0/2095883066 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55ac9a5633f0 con 0x55ac9a912ef0 2024-04-25T09:58:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/2095883066 <== mon.2 v2:172.21.15.112:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd614061260 con 0x55ac9a912ef0 2024-04-25T09:58:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/2095883066 <== mon.2 v2:172.21.15.112:3301/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd608049430 con 0x55ac9a912ef0 2024-04-25T09:58:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd622dc3740 10 client.6056 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:58:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd622dc3740 10 client.6056 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:58:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd622dc3740 20 client.6056 populate_metadata read hostname 'smithi019' 2024-04-25T09:58:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ac9a908da0 0x55ac9a9091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd622dc3740 4 client.6056 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd622dc3740 1 -- 172.21.15.19:0/2095883066 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac9a85c2e0 con 0x55ac9a908da0 2024-04-25T09:58:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.100+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ac9a908da0 0x55ac9a9091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.101+0000 7fd5f8ff9640 10 client.6056 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.101+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ac9a908da0 0x55ac9a9091c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3997d46357cb36b server_cookie=6ecc46da0f4fc2e9 in_seq=0 out_seq=0 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/2095883066 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ac9a85c2e0 con 0x55ac9a908da0 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd5f8ff9640 10 client.6056 handle_command_reply: tid=0 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 1 client.6056 shutdown 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 1 -- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 msgr2=0x55ac9a9170a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a9170a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd608031000 tx=0x7fd608004730 comp rx=0 tx=0).stop 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 1 -- 172.21.15.19:0/2095883066 shutdown_connections 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ac9a908da0 0x55ac9a9091c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a909f40 0x55ac9a9175e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a9170a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2095883066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a8fab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 1 -- 172.21.15.19:0/2095883066 >> 172.21.15.19:0/2095883066 conn(0x55ac9a8e5770 msgr2=0x55ac9a8e4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd5f8ff9640 0 client.6056 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 1 -- 172.21.15.19:0/2095883066 shutdown_connections 2024-04-25T09:58:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 1 -- 172.21.15.19:0/2095883066 wait complete. 2024-04-25T09:58:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.102+0000 7fd622dc3740 20 client.6056 trim_cache size 0 max 16384 2024-04-25T09:58:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.123+0000 7fd622dc3740 1 Processor -- start 2024-04-25T09:58:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.123+0000 7fd622dc3740 1 -- start start 2024-04-25T09:58:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a8e4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912ef0 0x55ac9a8e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a9343e0 0x55ac9a916e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd622dc3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618204910 con 0x55ac9a912ef0 2024-04-25T09:58:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd622dc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6181f81d0 con 0x55ac9a912b10 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd622dc3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61815f010 con 0x55ac9a9343e0 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd61e57d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a9343e0 0x55ac9a916e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd61dd7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a8e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd61d57b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912ef0 0x55ac9a8e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd61e57d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a9343e0 0x55ac9a916e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:58822/0 (socket says 172.21.15.19:58822) 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd61dd7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a8e4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:40766/0 (socket says 172.21.15.19:40766) 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd61e57d640 1 -- 172.21.15.19:0/2316720043 learned_addr learned my addr 172.21.15.19:0/2316720043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd61d57b640 1 -- 172.21.15.19:0/2316720043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a9343e0 msgr2=0x55ac9a916e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd61d57b640 1 --2- 172.21.15.19:0/2316720043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a9343e0 0x55ac9a916e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd61d57b640 1 -- 172.21.15.19:0/2316720043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 msgr2=0x55ac9a8e4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd61d57b640 1 --2- 172.21.15.19:0/2316720043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a8e4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.124+0000 7fd61d57b640 1 -- 172.21.15.19:0/2316720043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac9a5230f0 con 0x55ac9a912ef0 2024-04-25T09:58:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.125+0000 7fd61d57b640 1 --2- 172.21.15.19:0/2316720043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912ef0 0x55ac9a8e4b40 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd608000c00 tx=0x7fd60804b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c384bb0373db3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.125+0000 7fd5faffd640 1 -- 172.21.15.19:0/2316720043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608037030 con 0x55ac9a912ef0 2024-04-25T09:58:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.125+0000 7fd5faffd640 1 -- 172.21.15.19:0/2316720043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd608032d20 con 0x55ac9a912ef0 2024-04-25T09:58:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.125+0000 7fd5faffd640 1 -- 172.21.15.19:0/2316720043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608049430 con 0x55ac9a912ef0 2024-04-25T09:58:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.126+0000 7fd622dc3740 1 -- 172.21.15.19:0/2316720043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912ef0 msgr2=0x55ac9a8e4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.126+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2316720043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912ef0 0x55ac9a8e4b40 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd608000c00 tx=0x7fd60804b070 comp rx=0 tx=0).stop 2024-04-25T09:58:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.126+0000 7fd622dc3740 1 -- 172.21.15.19:0/2316720043 shutdown_connections 2024-04-25T09:58:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.126+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2316720043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a9343e0 0x55ac9a916e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.126+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2316720043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a912ef0 0x55ac9a8e4b40 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.126+0000 7fd622dc3740 1 --2- 172.21.15.19:0/2316720043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a8e4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.126+0000 7fd622dc3740 1 -- 172.21.15.19:0/2316720043 >> 172.21.15.19:0/2316720043 conn(0x55ac9a8e5770 msgr2=0x55ac9a9152e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.126+0000 7fd622dc3740 1 -- 172.21.15.19:0/2316720043 shutdown_connections 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.126+0000 7fd622dc3740 1 -- 172.21.15.19:0/2316720043 wait complete. 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.126+0000 7fd622dc3740 1 -- start start 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.127+0000 7fd622dc3740 1 Processor -- start 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a906d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a9116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd622dc3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a9343e0 0x55ac9a911bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd622dc3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180b6000 con 0x55ac9a9343e0 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd622dc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9a930710 con 0x55ac9a912b10 2024-04-25T09:58:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd622dc3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac9a9072c0 con 0x55ac9a912ef0 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd622dc3740 10 client.0 _ll_register_callbacks cb 0x7fd60802af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61e57d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a9343e0 0x55ac9a911bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61d57b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a9116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61dd7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a906d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61d57b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a9116b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:58830/0 (socket says 172.21.15.19:58830) 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61d57b640 1 -- 172.21.15.19:0/509960759 learned_addr learned my addr 172.21.15.19:0/509960759 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61e57d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a9343e0 0x55ac9a911bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41338/0 (socket says 172.21.15.19:41338) 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61e57d640 1 -- 172.21.15.19:0/509960759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 msgr2=0x55ac9a9116b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61e57d640 1 --2- 172.21.15.19:0/509960759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a9116b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61e57d640 1 -- 172.21.15.19:0/509960759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 msgr2=0x55ac9a906d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61e57d640 1 --2- 172.21.15.19:0/509960759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a906d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61e57d640 1 -- 172.21.15.19:0/509960759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac9a8eac40 con 0x55ac9a9343e0 2024-04-25T09:58:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd61e57d640 1 --2- 172.21.15.19:0/509960759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a9343e0 0x55ac9a911bf0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x55ac9a933190 tx=0x7fd610017800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f1d85983a10ece server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.128+0000 7fd5f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/509960759 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610015650 con 0x55ac9a9343e0 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/509960759 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6100157f0 con 0x55ac9a9343e0 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd622dc3740 1 -- 172.21.15.19:0/509960759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd608032d20 con 0x55ac9a9343e0 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/509960759 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd610027050 con 0x55ac9a9343e0 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd622dc3740 10 client.7356 fetch_fsmap learned FSMap version 52 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd622dc3740 1 -- 172.21.15.19:0/509960759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55ac9a930510 con 0x55ac9a9343e0 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/509960759 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd608032d20 con 0x55ac9a9343e0 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/509960759 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd61002a020 con 0x55ac9a9343e0 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd622dc3740 10 client.7356 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd622dc3740 10 client.7356 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:58:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd622dc3740 20 client.7356 populate_metadata read hostname 'smithi019' 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.129+0000 7fd622dc3740 1 --2- 172.21.15.19:0/509960759 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ac9a8ebad0 0x55ac9a93ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.130+0000 7fd622dc3740 4 client.7356 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.130+0000 7fd622dc3740 1 -- 172.21.15.19:0/509960759 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ac9a856630 con 0x55ac9a8ebad0 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.130+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/509960759 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ac9a8ebad0 0x55ac9a93ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.130+0000 7fd5f8ff9640 10 client.7356 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.130+0000 7fd61dd7c640 1 --2- 172.21.15.19:0/509960759 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ac9a8ebad0 0x55ac9a93ec70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a944af4313a3c19 server_cookie=d61be9b32cabd480 in_seq=0 out_seq=0 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd5f8ff9640 1 -- 172.21.15.19:0/509960759 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ac9a856630 con 0x55ac9a8ebad0 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd5f8ff9640 10 client.7356 handle_command_reply: tid=0 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd622dc3740 1 client.7356 shutdown 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd622dc3740 1 -- 172.21.15.19:0/509960759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a9343e0 msgr2=0x55ac9a911bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd622dc3740 1 --2- 172.21.15.19:0/509960759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a9343e0 0x55ac9a911bf0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x55ac9a933190 tx=0x7fd610017800 comp rx=0 tx=0).stop 2024-04-25T09:58:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd622dc3740 1 -- 172.21.15.19:0/509960759 shutdown_connections 2024-04-25T09:58:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd622dc3740 1 --2- 172.21.15.19:0/509960759 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ac9a8ebad0 0x55ac9a93ec70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd622dc3740 1 --2- 172.21.15.19:0/509960759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ac9a9343e0 0x55ac9a911bf0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd5f8ff9640 0 client.7356 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd622dc3740 1 --2- 172.21.15.19:0/509960759 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ac9a912ef0 0x55ac9a9116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd622dc3740 1 --2- 172.21.15.19:0/509960759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac9a912b10 0x55ac9a906d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.131+0000 7fd622dc3740 1 -- 172.21.15.19:0/509960759 >> 172.21.15.19:0/509960759 conn(0x55ac9a8e5770 msgr2=0x55ac9a8ea280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd622dc3740 1 -- 172.21.15.19:0/509960759 shutdown_connections 2024-04-25T09:58:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd622dc3740 1 -- 172.21.15.19:0/509960759 wait complete. 2024-04-25T09:58:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd622dc3740 20 client.7356 trim_cache size 0 max 16384 2024-04-25T09:58:20.141 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:58:20.141 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (184 inodes in the stack)", 2024-04-25T09:58:20.141 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T09:58:20.141 INFO:teuthology.orchestra.run.smithi019.stdout: "fb457f1b-be70-4eb2-be88-3cfff741377a": { 2024-04-25T09:58:20.141 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "~mds0", 2024-04-25T09:58:20.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "fb457f1b-be70-4eb2-be88-3cfff741377a", 2024-04-25T09:58:20.141 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive" 2024-04-25T09:58:20.141 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:20.141 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:20.141 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:58:20.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd61ffe0640 1 -- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5fc06b4d0 msgr2=0x7fd5fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:20.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd61ffe0640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5fc06b4d0 0x7fd5fc06d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd608034000 tx=0x7fd60802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:20.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd61ffe0640 1 -- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 msgr2=0x7fd61806ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:20.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd61ffe0640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd61806ca70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e870 tx=0x7fd61400ed30 comp rx=0 tx=0).stop 2024-04-25T09:58:20.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd61ffe0640 1 -- 172.21.15.19:0/2864032357 shutdown_connections 2024-04-25T09:58:20.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd61ffe0640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5fc06b4d0 0x7fd5fc06d990 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd61ffe0640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd6180b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd61ffe0640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6180cfb40 0x7fd6181fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd61ffe0640 1 --2- 172.21.15.19:0/2864032357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd61806ca70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:20.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd61ffe0640 1 -- 172.21.15.19:0/2864032357 >> 172.21.15.19:0/2864032357 conn(0x7fd61806f500 msgr2=0x7fd6180cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:20.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.132+0000 7fd61ffe0640 1 -- 172.21.15.19:0/2864032357 shutdown_connections 2024-04-25T09:58:20.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:20.133+0000 7fd61ffe0640 1 -- 172.21.15.19:0/2864032357 wait complete. 2024-04-25T09:58:20.148 INFO:tasks.cephfs.filesystem:scrub status for tag:fb457f1b-be70-4eb2-be88-3cfff741377a - {'path': '~mds0', 'tag': 'fb457f1b-be70-4eb2-be88-3cfff741377a', 'options': 'recursive'} 2024-04-25T09:58:37.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:37.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:50.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:50.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:58:50.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.261+0000 7f17336a3640 1 Processor -- start 2024-04-25T09:58:50.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.262+0000 7f17336a3640 1 -- start start 2024-04-25T09:58:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.262+0000 7f17336a3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f172c15dc20 0x7f172c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.262+0000 7f17336a3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f172c0d32f0 0x7f172c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.262+0000 7f17336a3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172c15e000 con 0x7f172c161710 2024-04-25T09:58:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.262+0000 7f17336a3640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172c0d1de0 con 0x7f172c15dc20 2024-04-25T09:58:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.262+0000 7f17336a3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172c161c60 con 0x7f172c0d32f0 2024-04-25T09:58:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.263+0000 7f173143f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f172c15dc20 0x7f172c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.263+0000 7f1730c3e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f172c161710 0x7f172c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52782/0 (socket says 172.21.15.19:52782) 2024-04-25T09:58:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.263+0000 7f1730c3e640 1 -- 172.21.15.19:0/4264822617 learned_addr learned my addr 172.21.15.19:0/4264822617 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.263+0000 7f1731c40640 1 --2- 172.21.15.19:0/4264822617 >> v2:172.21.15.112:3300/0 conn(0x7f172c0d32f0 0x7f172c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.263+0000 7f173143f640 1 -- 172.21.15.19:0/4264822617 >> v2:172.21.15.112:3300/0 conn(0x7f172c0d32f0 msgr2=0x7f172c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.263+0000 7f173143f640 1 --2- 172.21.15.19:0/4264822617 >> v2:172.21.15.112:3300/0 conn(0x7f172c0d32f0 0x7f172c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.263+0000 7f173143f640 1 -- 172.21.15.19:0/4264822617 >> v1:172.21.15.19:6789/0 conn(0x7f172c161710 legacy=0x7f172c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.263+0000 7f173143f640 1 -- 172.21.15.19:0/4264822617 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c161f40 con 0x7f172c15dc20 2024-04-25T09:58:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.263+0000 7f1723fff640 1 -- 172.21.15.19:0/4264822617 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205700283 0 0) 0x7f172c15e000 con 0x7f172c161710 2024-04-25T09:58:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.263+0000 7f1731c40640 1 --2- 172.21.15.19:0/4264822617 >> v2:172.21.15.112:3300/0 conn(0x7f172c0d32f0 0x7f172c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.264+0000 7f173143f640 1 --2- 172.21.15.19:0/4264822617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c15dc20 0x7f172c0ce670 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f172c0b6910 tx=0x7f172802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9ba9834abdac96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.264+0000 7f1723fff640 1 -- 172.21.15.19:0/4264822617 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728037070 con 0x7f172c15dc20 2024-04-25T09:58:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.264+0000 7f1723fff640 1 -- 172.21.15.19:0/4264822617 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1728032440 con 0x7f172c15dc20 2024-04-25T09:58:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.264+0000 7f1723fff640 1 -- 172.21.15.19:0/4264822617 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728032810 con 0x7f172c15dc20 2024-04-25T09:58:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.264+0000 7f17336a3640 1 -- 172.21.15.19:0/4264822617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c15dc20 msgr2=0x7f172c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.264+0000 7f17336a3640 1 --2- 172.21.15.19:0/4264822617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c15dc20 0x7f172c0ce670 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f172c0b6910 tx=0x7f172802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.264+0000 7f17336a3640 1 -- 172.21.15.19:0/4264822617 shutdown_connections 2024-04-25T09:58:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.264+0000 7f17336a3640 1 --2- 172.21.15.19:0/4264822617 >> v2:172.21.15.112:3300/0 conn(0x7f172c0d32f0 0x7f172c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.264+0000 7f17336a3640 1 --2- 172.21.15.19:0/4264822617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c15dc20 0x7f172c0ce670 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.264+0000 7f17336a3640 1 -- 172.21.15.19:0/4264822617 >> 172.21.15.19:0/4264822617 conn(0x7f172c0d0420 msgr2=0x7f172c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.265+0000 7f17336a3640 1 -- 172.21.15.19:0/4264822617 shutdown_connections 2024-04-25T09:58:50.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.265+0000 7f17336a3640 1 -- 172.21.15.19:0/4264822617 wait complete. 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.266+0000 7f17336a3640 1 Processor -- start 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.266+0000 7f17336a3640 1 -- start start 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f17336a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172c0d32f0 0x7f172c0aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f17336a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c15dc20 0x7f172c159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f17336a3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f172c161710 0x7f172c0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f17336a3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0b5e50 con 0x7f172c15dc20 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f17336a3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c162500 con 0x7f172c0d32f0 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f17336a3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0b5fd0 con 0x7f172c161710 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1730c3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c15dc20 0x7f172c159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1730c3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c15dc20 0x7f172c159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40112/0 (socket says 172.21.15.19:40112) 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1730c3e640 1 -- 172.21.15.19:0/2767827163 learned_addr learned my addr 172.21.15.19:0/2767827163 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1731c40640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f172c161710 0x7f172c0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f173143f640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172c0d32f0 0x7f172c0aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1730c3e640 1 -- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f172c161710 msgr2=0x7f172c0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1730c3e640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f172c161710 0x7f172c0b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1730c3e640 1 -- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172c0d32f0 msgr2=0x7f172c0aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1730c3e640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172c0d32f0 0x7f172c0aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1730c3e640 1 -- 172.21.15.19:0/2767827163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c0cbb20 con 0x7f172c15dc20 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1730c3e640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c15dc20 0x7f172c159a50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f171c00c460 tx=0x7f171c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1721ffb640 1 -- 172.21.15.19:0/2767827163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c018070 con 0x7f172c15dc20 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1721ffb640 1 -- 172.21.15.19:0/2767827163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c007500 con 0x7f172c15dc20 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f17336a3640 1 -- 172.21.15.19:0/2767827163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c0cbe00 con 0x7f172c15dc20 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.267+0000 7f1721ffb640 1 -- 172.21.15.19:0/2767827163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c004a20 con 0x7f172c15dc20 2024-04-25T09:58:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.268+0000 7f17336a3640 1 -- 172.21.15.19:0/2767827163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c0cc3c0 con 0x7f172c15dc20 2024-04-25T09:58:50.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.269+0000 7f1721ffb640 1 -- 172.21.15.19:0/2767827163 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f171c01b070 con 0x7f172c15dc20 2024-04-25T09:58:50.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.270+0000 7f1721ffb640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f171006b4d0 0x7f171006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.270+0000 7f1721ffb640 1 -- 172.21.15.19:0/2767827163 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f171c089620 con 0x7f172c15dc20 2024-04-25T09:58:50.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.270+0000 7f173143f640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f171006b4d0 0x7f171006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.270+0000 7f173143f640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f171006b4d0 0x7f171006d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1728034000 tx=0x7f172802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.271+0000 7f173639c740 1 Processor -- start 2024-04-25T09:58:50.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.271+0000 7f173639c740 1 -- start start 2024-04-25T09:58:50.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.271+0000 7f173639c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36cc8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.271+0000 7f173639c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cc5ef0 0x55ab36cc8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.271+0000 7f173639c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36c98770 0x55ab36c9bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.271+0000 7f173639c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab36c01da0 con 0x55ab36cc5b10 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.271+0000 7f173639c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab36baab80 con 0x55ab36c98770 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.271+0000 7f173639c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab36bfde40 con 0x55ab36cc5ef0 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f173143f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36cc8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f173143f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36cc8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40126/0 (socket says 172.21.15.19:40126) 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f173143f640 1 -- 172.21.15.19:0/627602054 learned_addr learned my addr 172.21.15.19:0/627602054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f1730c3e640 1 --2- 172.21.15.19:0/627602054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cc5ef0 0x55ab36cc8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f1731c40640 1 --2- 172.21.15.19:0/627602054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36c98770 0x55ab36c9bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f173143f640 1 -- 172.21.15.19:0/627602054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cc5ef0 msgr2=0x55ab36cc8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f173143f640 1 --2- 172.21.15.19:0/627602054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cc5ef0 0x55ab36cc8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f173143f640 1 -- 172.21.15.19:0/627602054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36c98770 msgr2=0x55ab36c9bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f173143f640 1 --2- 172.21.15.19:0/627602054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36c98770 0x55ab36c9bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f173143f640 1 -- 172.21.15.19:0/627602054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab36bb4f10 con 0x55ab36cc5b10 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f1730c3e640 1 --2- 172.21.15.19:0/627602054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cc5ef0 0x55ab36cc8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f1731c40640 1 --2- 172.21.15.19:0/627602054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36c98770 0x55ab36c9bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f173143f640 1 --2- 172.21.15.19:0/627602054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36cc8320 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f172802a1e0 tx=0x7f172802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f0f9cac77e9208 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f1716ffd640 1 -- 172.21.15.19:0/627602054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172804d070 con 0x55ab36cc5b10 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f1716ffd640 1 -- 172.21.15.19:0/627602054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1728032e20 con 0x55ab36cc5b10 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.272+0000 7f1716ffd640 1 -- 172.21.15.19:0/627602054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728037040 con 0x55ab36cc5b10 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.273+0000 7f173639c740 1 -- 172.21.15.19:0/627602054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 msgr2=0x55ab36cc8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.273+0000 7f173639c740 1 --2- 172.21.15.19:0/627602054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36cc8320 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f172802a1e0 tx=0x7f172802c000 comp rx=0 tx=0).stop 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.273+0000 7f173639c740 1 -- 172.21.15.19:0/627602054 shutdown_connections 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.273+0000 7f173639c740 1 --2- 172.21.15.19:0/627602054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36c98770 0x55ab36c9bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.273+0000 7f173639c740 1 --2- 172.21.15.19:0/627602054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cc5ef0 0x55ab36cc8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.273+0000 7f173639c740 1 --2- 172.21.15.19:0/627602054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36cc8320 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.273+0000 7f173639c740 1 -- 172.21.15.19:0/627602054 >> 172.21.15.19:0/627602054 conn(0x55ab36c9a9e0 msgr2=0x55ab36ce3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.273+0000 7f173639c740 1 -- 172.21.15.19:0/627602054 shutdown_connections 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.273+0000 7f173639c740 1 -- 172.21.15.19:0/627602054 wait complete. 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.273+0000 7f173639c740 1 -- start start 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.274+0000 7f173639c740 1 Processor -- start 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.274+0000 7f173639c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5b10 0x55ab36cadc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f173639c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5ef0 0x55ab36cea370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f173639c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36ce79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f173639c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab36c9db90 con 0x55ab36cc5ef0 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f173639c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab36ba04b0 con 0x55ab36cc5b10 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f173639c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab36c9de70 con 0x55ab36cbcfb0 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f173639c740 10 client.0 _ll_register_callbacks cb 0x55ab36be0180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f1730c3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5ef0 0x55ab36cea370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f1730c3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5ef0 0x55ab36cea370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40134/0 (socket says 172.21.15.19:40134) 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f1730c3e640 1 -- 172.21.15.19:0/3627749952 learned_addr learned my addr 172.21.15.19:0/3627749952 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f1731c40640 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36ce79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f173143f640 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5b10 0x55ab36cadc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f1730c3e640 1 -- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 msgr2=0x55ab36ce79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f1730c3e640 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36ce79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f1730c3e640 1 -- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5b10 msgr2=0x55ab36cadc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f1730c3e640 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5b10 0x55ab36cadc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f1730c3e640 1 -- 172.21.15.19:0/3627749952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab369e07d0 con 0x55ab36cc5ef0 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f173143f640 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5b10 0x55ab36cadc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f1730c3e640 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5ef0 0x55ab36cea370 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f171c000f60 tx=0x7f171c057d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c2f0322b45bbac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f170bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:58:50.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f170bfff640 1 -- 172.21.15.19:0/3627749952 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c055fa0 con 0x55ab36cc5ef0 2024-04-25T09:58:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f170bfff640 1 -- 172.21.15.19:0/3627749952 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f171c06bba0 con 0x55ab36cc5ef0 2024-04-25T09:58:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f173639c740 1 -- 172.21.15.19:0/3627749952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1728032e20 con 0x55ab36cc5ef0 2024-04-25T09:58:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f170bfff640 1 -- 172.21.15.19:0/3627749952 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f171c06bda0 con 0x55ab36cc5ef0 2024-04-25T09:58:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f173639c740 10 client.7371 fetch_fsmap learned FSMap version 52 2024-04-25T09:58:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.275+0000 7f173639c740 1 -- 172.21.15.19:0/3627749952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55ab369163f0 con 0x55ab36cc5ef0 2024-04-25T09:58:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.276+0000 7f170bfff640 1 -- 172.21.15.19:0/3627749952 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1728032e20 con 0x55ab36cc5ef0 2024-04-25T09:58:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.276+0000 7f170bfff640 1 -- 172.21.15.19:0/3627749952 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f171c082020 con 0x55ab36cc5ef0 2024-04-25T09:58:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.276+0000 7f173639c740 10 client.7371 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:58:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.276+0000 7f173639c740 10 client.7371 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:58:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.276+0000 7f173639c740 20 client.7371 populate_metadata read hostname 'smithi019' 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.276+0000 7f173639c740 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ab36ce64f0 0x55ab36ce68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.276+0000 7f173639c740 4 client.7371 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.276+0000 7f173639c740 1 -- 172.21.15.19:0/3627749952 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab36c0f2e0 con 0x55ab36ce64f0 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.276+0000 7f173143f640 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ab36ce64f0 0x55ab36ce68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.276+0000 7f173143f640 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ab36ce64f0 0x55ab36ce68d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128a31b393618d0e server_cookie=e35cbfb1aeeaf929 in_seq=0 out_seq=0 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.276+0000 7f170bfff640 10 client.7371 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f170bfff640 1 -- 172.21.15.19:0/3627749952 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab36c0f2e0 con 0x55ab36ce64f0 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f170bfff640 10 client.7371 handle_command_reply: tid=0 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f173639c740 1 client.7371 shutdown 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f173639c740 1 -- 172.21.15.19:0/3627749952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5ef0 msgr2=0x55ab36cea370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f173639c740 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5ef0 0x55ab36cea370 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f171c000f60 tx=0x7f171c057d20 comp rx=0 tx=0).stop 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f173639c740 1 -- 172.21.15.19:0/3627749952 shutdown_connections 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f173639c740 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ab36ce64f0 0x55ab36ce68d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f173639c740 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36ce79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f173639c740 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5ef0 0x55ab36cea370 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f173639c740 1 --2- 172.21.15.19:0/3627749952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5b10 0x55ab36cadc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f170bfff640 0 client.7371 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:50.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f173639c740 1 -- 172.21.15.19:0/3627749952 >> 172.21.15.19:0/3627749952 conn(0x55ab36c98770 msgr2=0x55ab36c99420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.277+0000 7f173639c740 1 -- 172.21.15.19:0/3627749952 shutdown_connections 2024-04-25T09:58:50.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.278+0000 7f173639c740 1 -- 172.21.15.19:0/3627749952 wait complete. 2024-04-25T09:58:50.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.278+0000 7f173639c740 20 client.7371 trim_cache size 0 max 16384 2024-04-25T09:58:50.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.293+0000 7f173639c740 1 Processor -- start 2024-04-25T09:58:50.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.293+0000 7f173639c740 1 -- start start 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f173639c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36cea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f173639c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36d057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f173639c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5ef0 0x55ab36d45ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f173639c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c200290 con 0x55ab36cc5b10 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f173639c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1ff480 con 0x55ab36cc5ef0 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f173639c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0cd430 con 0x55ab36cbcfb0 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f1730c3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36d057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f1730c3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36d057a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40138/0 (socket says 172.21.15.19:40138) 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f1730c3e640 1 -- 172.21.15.19:0/3796585955 learned_addr learned my addr 172.21.15.19:0/3796585955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f173143f640 1 --2- 172.21.15.19:0/3796585955 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36cea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f1731c40640 1 --2- 172.21.15.19:0/3796585955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5ef0 0x55ab36d45ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f1730c3e640 1 -- 172.21.15.19:0/3796585955 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 msgr2=0x55ab36cea270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f1730c3e640 1 --2- 172.21.15.19:0/3796585955 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36cea270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f1730c3e640 1 -- 172.21.15.19:0/3796585955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5ef0 msgr2=0x55ab36d45ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f1730c3e640 1 --2- 172.21.15.19:0/3796585955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5ef0 0x55ab36d45ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f1730c3e640 1 -- 172.21.15.19:0/3796585955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab368d60f0 con 0x55ab36cc5b10 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f1730c3e640 1 --2- 172.21.15.19:0/3796585955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36d057a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f171c054560 tx=0x7f171c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eeabae54d7e3aad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f17167fc640 1 -- 172.21.15.19:0/3796585955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c053e80 con 0x55ab36cc5b10 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f17167fc640 1 -- 172.21.15.19:0/3796585955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c055fa0 con 0x55ab36cc5b10 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.294+0000 7f17167fc640 1 -- 172.21.15.19:0/3796585955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c0892b0 con 0x55ab36cc5b10 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.295+0000 7f173639c740 1 -- 172.21.15.19:0/3796585955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 msgr2=0x55ab36d057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.295+0000 7f173639c740 1 --2- 172.21.15.19:0/3796585955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36d057a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f171c054560 tx=0x7f171c00a000 comp rx=0 tx=0).stop 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.295+0000 7f173639c740 1 -- 172.21.15.19:0/3796585955 shutdown_connections 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.295+0000 7f173639c740 1 --2- 172.21.15.19:0/3796585955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5ef0 0x55ab36d45ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.295+0000 7f173639c740 1 --2- 172.21.15.19:0/3796585955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36d057a0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.295+0000 7f173639c740 1 --2- 172.21.15.19:0/3796585955 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36cea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.295+0000 7f173639c740 1 -- 172.21.15.19:0/3796585955 >> 172.21.15.19:0/3796585955 conn(0x55ab36c98770 msgr2=0x55ab36c93db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.295+0000 7f173639c740 1 -- 172.21.15.19:0/3796585955 shutdown_connections 2024-04-25T09:58:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.295+0000 7f173639c740 1 -- 172.21.15.19:0/3796585955 wait complete. 2024-04-25T09:58:50.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.296+0000 7f173639c740 1 -- start start 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.296+0000 7f173639c740 1 Processor -- start 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f173639c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36ce1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f173639c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36ce4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f173639c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5ef0 0x55ab36c99170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f173639c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab36ce2bb0 con 0x55ab36cc5b10 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f173639c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0cb640 con 0x55ab36cc5ef0 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f173639c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab36ce3020 con 0x55ab36cbcfb0 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f173639c740 10 client.0 _ll_register_callbacks cb 0x7f1728035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f1730c3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36ce4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f1730c3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36ce4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40146/0 (socket says 172.21.15.19:40146) 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f1730c3e640 1 -- 172.21.15.19:0/2673381986 learned_addr learned my addr 172.21.15.19:0/2673381986 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f1731c40640 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5ef0 0x55ab36c99170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f173143f640 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36ce1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f1730c3e640 1 -- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 msgr2=0x55ab36ce1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f1730c3e640 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36ce1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f1730c3e640 1 -- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5ef0 msgr2=0x55ab36c99170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f1730c3e640 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5ef0 0x55ab36c99170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f1730c3e640 1 -- 172.21.15.19:0/2673381986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab36c97730 con 0x55ab36cc5b10 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f1730c3e640 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36ce4450 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f171c055e90 tx=0x7f171c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb8047bf57f8147 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f170bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f170bfff640 1 -- 172.21.15.19:0/2673381986 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c053e80 con 0x55ab36cc5b10 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f170bfff640 1 -- 172.21.15.19:0/2673381986 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f171c056cb0 con 0x55ab36cc5b10 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.297+0000 7f173639c740 1 -- 172.21.15.19:0/2673381986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab36c09630 con 0x55ab36cc5b10 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f170bfff640 1 -- 172.21.15.19:0/2673381986 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f171c0579b0 con 0x55ab36cc5b10 2024-04-25T09:58:50.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f173639c740 10 client.7377 fetch_fsmap learned FSMap version 52 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f173639c740 1 -- 172.21.15.19:0/2673381986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55ab36ce4d20 con 0x55ab36cc5b10 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f170bfff640 1 -- 172.21.15.19:0/2673381986 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ab36c09630 con 0x55ab36cc5b10 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f170bfff640 1 -- 172.21.15.19:0/2673381986 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f171c082030 con 0x55ab36cc5b10 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f173639c740 10 client.7377 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f173639c740 10 client.7377 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f173639c740 20 client.7377 populate_metadata read hostname 'smithi019' 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f173639c740 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ab36c996b0 0x55ab36ce7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f173639c740 4 client.7377 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f173639c740 1 -- 172.21.15.19:0/2673381986 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ab36b87690 con 0x55ab36c996b0 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f173143f640 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ab36c996b0 0x55ab36ce7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f170bfff640 10 client.7377 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.298+0000 7f173143f640 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ab36c996b0 0x55ab36ce7280 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=489155a7d8d8c9f3 server_cookie=a20a478499942d34 in_seq=0 out_seq=0 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f170bfff640 1 -- 172.21.15.19:0/2673381986 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ab36b87690 con 0x55ab36c996b0 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f170bfff640 10 client.7377 handle_command_reply: tid=0 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 1 client.7377 shutdown 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 1 -- 172.21.15.19:0/2673381986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 msgr2=0x55ab36ce4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36ce4450 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f171c055e90 tx=0x7f171c009040 comp rx=0 tx=0).stop 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 1 -- 172.21.15.19:0/2673381986 shutdown_connections 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x55ab36c996b0 0x55ab36ce7280 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab36cc5ef0 0x55ab36c99170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f170bfff640 0 client.7377 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab36cc5b10 0x55ab36ce4450 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 1 --2- 172.21.15.19:0/2673381986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ab36cbcfb0 0x55ab36ce1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 1 -- 172.21.15.19:0/2673381986 >> 172.21.15.19:0/2673381986 conn(0x55ab36c98770 msgr2=0x55ab36d45ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 1 -- 172.21.15.19:0/2673381986 shutdown_connections 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 1 -- 172.21.15.19:0/2673381986 wait complete. 2024-04-25T09:58:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f173639c740 20 client.7377 trim_cache size 0 max 16384 2024-04-25T09:58:50.309 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:58:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T09:58:50.309 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T09:58:50.309 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:58:50.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f17336a3640 1 -- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f171006b4d0 msgr2=0x7f171006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.299+0000 7f17336a3640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f171006b4d0 0x7f171006d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1728034000 tx=0x7f172802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.300+0000 7f17336a3640 1 -- 172.21.15.19:0/2767827163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c15dc20 msgr2=0x7f172c159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.300+0000 7f17336a3640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c15dc20 0x7f172c159a50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f171c00c460 tx=0x7f171c00c920 comp rx=0 tx=0).stop 2024-04-25T09:58:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.300+0000 7f17336a3640 1 -- 172.21.15.19:0/2767827163 shutdown_connections 2024-04-25T09:58:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.300+0000 7f17336a3640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f171006b4d0 0x7f171006d990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.300+0000 7f17336a3640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f172c161710 0x7f172c0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.300+0000 7f17336a3640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172c15dc20 0x7f172c159a50 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.300+0000 7f17336a3640 1 --2- 172.21.15.19:0/2767827163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f172c0d32f0 0x7f172c0aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.300+0000 7f17336a3640 1 -- 172.21.15.19:0/2767827163 >> 172.21.15.19:0/2767827163 conn(0x7f172c0d0420 msgr2=0x7f172c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.300+0000 7f17336a3640 1 -- 172.21.15.19:0/2767827163 shutdown_connections 2024-04-25T09:58:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.300+0000 7f17336a3640 1 -- 172.21.15.19:0/2767827163 wait complete. 2024-04-25T09:58:50.314 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:58:50.314 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:58:50.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T09:58:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.413+0000 7f298be38640 1 Processor -- start 2024-04-25T09:58:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.413+0000 7f298be38640 1 -- start start 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.414+0000 7f298be38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298415dc20 0x7f29840d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.414+0000 7f298be38640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29840d1ce0 0x7f29840b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.414+0000 7f298be38640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29840d2120 con 0x7f29840ce670 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.414+0000 7f298be38640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29840d23a0 con 0x7f29840d1ce0 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.414+0000 7f298be38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29840d2680 con 0x7f298415dc20 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.414+0000 7f298a3d5640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f29840ce670 0x7f298406c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52792/0 (socket says 172.21.15.19:52792) 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.414+0000 7f298a3d5640 1 -- 172.21.15.19:0/2213796636 learned_addr learned my addr 172.21.15.19:0/2213796636 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.414+0000 7f29893d3640 1 --2- 172.21.15.19:0/2213796636 >> v2:172.21.15.112:3300/0 conn(0x7f29840d1ce0 0x7f29840b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.414+0000 7f2989bd4640 1 --2- 172.21.15.19:0/2213796636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298415dc20 0x7f29840d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.415+0000 7f2988bd2640 1 -- 172.21.15.19:0/2213796636 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585093783 0 0) 0x7f2974001490 con 0x7f29840ce670 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.415+0000 7f2988bd2640 1 -- 172.21.15.19:0/2213796636 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f296c003520 con 0x7f29840ce670 2024-04-25T09:58:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.415+0000 7f29893d3640 1 -- 172.21.15.19:0/2213796636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298415dc20 msgr2=0x7f29840d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.415+0000 7f29893d3640 1 --2- 172.21.15.19:0/2213796636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298415dc20 0x7f29840d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.415+0000 7f29893d3640 1 -- 172.21.15.19:0/2213796636 >> v1:172.21.15.19:6789/0 conn(0x7f29840ce670 legacy=0x7f298406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.415+0000 7f29893d3640 1 -- 172.21.15.19:0/2213796636 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29840d2960 con 0x7f29840d1ce0 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f29893d3640 1 --2- 172.21.15.19:0/2213796636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29840d1ce0 0x7f29840b9490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f29780097a0 tx=0x7f297802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caacec2412d60eca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f2988bd2640 1 -- 172.21.15.19:0/2213796636 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978037070 con 0x7f29840d1ce0 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f2988bd2640 1 -- 172.21.15.19:0/2213796636 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297802ae50 con 0x7f29840d1ce0 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f2988bd2640 1 -- 172.21.15.19:0/2213796636 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978032660 con 0x7f29840d1ce0 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f298be38640 1 -- 172.21.15.19:0/2213796636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29840d1ce0 msgr2=0x7f29840b9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f298be38640 1 --2- 172.21.15.19:0/2213796636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29840d1ce0 0x7f29840b9490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f29780097a0 tx=0x7f297802a100 comp rx=0 tx=0).stop 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f298be38640 1 -- 172.21.15.19:0/2213796636 shutdown_connections 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f298be38640 1 --2- 172.21.15.19:0/2213796636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29840d1ce0 0x7f29840b9490 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f298be38640 1 --2- 172.21.15.19:0/2213796636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f298415dc20 0x7f29840d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f298be38640 1 -- 172.21.15.19:0/2213796636 >> 172.21.15.19:0/2213796636 conn(0x7f2984061a20 msgr2=0x7f2984061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f298be38640 1 -- 172.21.15.19:0/2213796636 shutdown_connections 2024-04-25T09:58:50.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.416+0000 7f298be38640 1 -- 172.21.15.19:0/2213796636 wait complete. 2024-04-25T09:58:50.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.418+0000 7f298be38640 1 Processor -- start 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.418+0000 7f298be38640 1 -- start start 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.418+0000 7f298be38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29840ce670 0x7f29841605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.418+0000 7f298be38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29840d1ce0 0x7f2984159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.418+0000 7f298be38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298415dc20 0x7f298414a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.418+0000 7f298be38640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29840ceb00 con 0x7f298415dc20 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.418+0000 7f298be38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29840d2f20 con 0x7f29840d1ce0 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.418+0000 7f298be38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29840cec80 con 0x7f29840ce670 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.418+0000 7f298a3d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298415dc20 0x7f298414a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.418+0000 7f298a3d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298415dc20 0x7f298414a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40156/0 (socket says 172.21.15.19:40156) 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.418+0000 7f298a3d5640 1 -- 172.21.15.19:0/2474426716 learned_addr learned my addr 172.21.15.19:0/2474426716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f29893d3640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29840d1ce0 0x7f2984159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f2989bd4640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29840ce670 0x7f29841605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f298a3d5640 1 -- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29840ce670 msgr2=0x7f29841605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f298a3d5640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29840ce670 0x7f29841605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f298a3d5640 1 -- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29840d1ce0 msgr2=0x7f2984159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f298a3d5640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29840d1ce0 0x7f2984159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f298a3d5640 1 -- 172.21.15.19:0/2474426716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298415f510 con 0x7f298415dc20 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f298a3d5640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298415dc20 0x7f298414a7d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f297400cb50 tx=0x7f297400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f29827fc640 1 -- 172.21.15.19:0/2474426716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974017070 con 0x7f298415dc20 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f298be38640 1 -- 172.21.15.19:0/2474426716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298415f7f0 con 0x7f298415dc20 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f29827fc640 1 -- 172.21.15.19:0/2474426716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297400ad40 con 0x7f298415dc20 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f29827fc640 1 -- 172.21.15.19:0/2474426716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974012620 con 0x7f298415dc20 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.419+0000 7f298be38640 1 -- 172.21.15.19:0/2474426716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298415fdb0 con 0x7f298415dc20 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.421+0000 7f29827fc640 1 -- 172.21.15.19:0/2474426716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f297401a070 con 0x7f298415dc20 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.421+0000 7f29827fc640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f297006b4d0 0x7f297006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.421+0000 7f29827fc640 1 -- 172.21.15.19:0/2474426716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f297408a2b0 con 0x7f298415dc20 2024-04-25T09:58:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.421+0000 7f2989bd4640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f297006b4d0 0x7f297006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.422+0000 7f2989bd4640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f297006b4d0 0x7f297006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f297c009a30 tx=0x7f297c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.422+0000 7f298ec15740 1 Processor -- start 2024-04-25T09:58:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.422+0000 7f298ec15740 1 -- start start 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f298ec15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a6ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f298ec15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a6d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f298ec15740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a97760 0x5609e4a97be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f298ec15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e49cfb90 con 0x5609e4a6cca0 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f298ec15740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4978e40 con 0x5609e4aa9370 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f298ec15740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609e49cbc30 con 0x5609e4a97760 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f2989bd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a6ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f2989bd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a6ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40160/0 (socket says 172.21.15.19:40160) 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f2989bd4640 1 -- 172.21.15.19:0/1685464172 learned_addr learned my addr 172.21.15.19:0/1685464172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f298a3d5640 1 --2- 172.21.15.19:0/1685464172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a97760 0x5609e4a97be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f29893d3640 1 --2- 172.21.15.19:0/1685464172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a6d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f2989bd4640 1 -- 172.21.15.19:0/1685464172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a97760 msgr2=0x5609e4a97be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f2989bd4640 1 --2- 172.21.15.19:0/1685464172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a97760 0x5609e4a97be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f2989bd4640 1 -- 172.21.15.19:0/1685464172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 msgr2=0x5609e4a6d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f2989bd4640 1 --2- 172.21.15.19:0/1685464172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a6d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f2989bd4640 1 -- 172.21.15.19:0/1685464172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e47ae7d0 con 0x5609e4a6cca0 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.423+0000 7f2989bd4640 1 --2- 172.21.15.19:0/1685464172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a6ee00 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f297c011cf0 tx=0x7f297c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b2e52bf8fd2553 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f296b7fe640 1 -- 172.21.15.19:0/1685464172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c011d20 con 0x5609e4a6cca0 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f296b7fe640 1 -- 172.21.15.19:0/1685464172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c026070 con 0x5609e4a6cca0 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f296b7fe640 1 -- 172.21.15.19:0/1685464172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c021640 con 0x5609e4a6cca0 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f298ec15740 1 -- 172.21.15.19:0/1685464172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 msgr2=0x5609e4a6ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f298ec15740 1 --2- 172.21.15.19:0/1685464172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a6ee00 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f297c011cf0 tx=0x7f297c014000 comp rx=0 tx=0).stop 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f298ec15740 1 -- 172.21.15.19:0/1685464172 shutdown_connections 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f298ec15740 1 --2- 172.21.15.19:0/1685464172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a97760 0x5609e4a97be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f298ec15740 1 --2- 172.21.15.19:0/1685464172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a6d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f298ec15740 1 --2- 172.21.15.19:0/1685464172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a6ee00 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f298ec15740 1 -- 172.21.15.19:0/1685464172 >> 172.21.15.19:0/1685464172 conn(0x5609e4a34870 msgr2=0x5609e4ab2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f298ec15740 1 -- 172.21.15.19:0/1685464172 shutdown_connections 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.424+0000 7f298ec15740 1 -- 172.21.15.19:0/1685464172 wait complete. 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.425+0000 7f298ec15740 1 -- start start 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.425+0000 7f298ec15740 1 Processor -- start 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f298ec15740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a6cca0 0x5609e4a94450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f298ec15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4a94990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f298ec15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a97be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f298ec15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e498c3e0 con 0x5609e4a97760 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f298ec15740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4a952b0 con 0x5609e4aa9370 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f298ec15740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609e4aa98d0 con 0x5609e4a6cca0 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f298ec15740 10 client.0 _ll_register_callbacks cb 0x5609e498bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f29893d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4a94990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f29893d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4a94990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40168/0 (socket says 172.21.15.19:40168) 2024-04-25T09:58:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f29893d3640 1 -- 172.21.15.19:0/3098835324 learned_addr learned my addr 172.21.15.19:0/3098835324 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f298a3d5640 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a97be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f2989bd4640 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a6cca0 0x5609e4a94450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f29893d3640 1 -- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a6cca0 msgr2=0x5609e4a94450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f29893d3640 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a6cca0 0x5609e4a94450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f29893d3640 1 -- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 msgr2=0x5609e4a97be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f29893d3640 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a97be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f29893d3640 1 -- 172.21.15.19:0/3098835324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609e46a40f0 con 0x5609e4a97760 2024-04-25T09:58:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f29893d3640 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4a94990 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f297802a140 tx=0x7f2978004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35248a158f39a486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.426+0000 7f2968ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:58:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f2968ff9640 1 -- 172.21.15.19:0/3098835324 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978037040 con 0x5609e4a97760 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f2968ff9640 1 -- 172.21.15.19:0/3098835324 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f2978032df0 con 0x5609e4a97760 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f298ec15740 1 -- 172.21.15.19:0/3098835324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f297c026070 con 0x5609e4a97760 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f2968ff9640 1 -- 172.21.15.19:0/3098835324 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2978006e10 con 0x5609e4a97760 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f298ec15740 10 client.7389 fetch_fsmap learned FSMap version 52 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f298ec15740 1 -- 172.21.15.19:0/3098835324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x5609e480ef60 con 0x5609e4a97760 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f2968ff9640 1 -- 172.21.15.19:0/3098835324 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c026070 con 0x5609e4a97760 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f2968ff9640 1 -- 172.21.15.19:0/3098835324 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f297804e050 con 0x5609e4a97760 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f298ec15740 10 client.7389 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f298ec15740 10 client.7389 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f298ec15740 20 client.7389 populate_metadata read hostname 'smithi019' 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f298ec15740 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x5609e4a91fa0 0x5609e4a98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f298ec15740 4 client.7389 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f298ec15740 1 -- 172.21.15.19:0/3098835324 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609e49dd0d0 con 0x5609e4a91fa0 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.427+0000 7f2989bd4640 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x5609e4a91fa0 0x5609e4a98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f2968ff9640 10 client.7389 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f2989bd4640 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x5609e4a91fa0 0x5609e4a98120 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4db2e81e027fc8f8 server_cookie=fca13ddf894aa05 in_seq=0 out_seq=0 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f2968ff9640 1 -- 172.21.15.19:0/3098835324 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5609e49dd0d0 con 0x5609e4a91fa0 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f2968ff9640 10 client.7389 handle_command_reply: tid=0 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f298ec15740 1 client.7389 shutdown 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f298ec15740 1 -- 172.21.15.19:0/3098835324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 msgr2=0x5609e4a94990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f298ec15740 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4a94990 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f297802a140 tx=0x7f2978004730 comp rx=0 tx=0).stop 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f298ec15740 1 -- 172.21.15.19:0/3098835324 shutdown_connections 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f298ec15740 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x5609e4a91fa0 0x5609e4a98120 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f298ec15740 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a97be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f298ec15740 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4a94990 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f2968ff9640 0 client.7389 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f298ec15740 1 --2- 172.21.15.19:0/3098835324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a6cca0 0x5609e4a94450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.428+0000 7f298ec15740 1 -- 172.21.15.19:0/3098835324 >> 172.21.15.19:0/3098835324 conn(0x5609e4a34870 msgr2=0x5609e4a95490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.429+0000 7f298ec15740 1 -- 172.21.15.19:0/3098835324 shutdown_connections 2024-04-25T09:58:50.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.429+0000 7f298ec15740 1 -- 172.21.15.19:0/3098835324 wait complete. 2024-04-25T09:58:50.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.429+0000 7f298ec15740 20 client.7389 trim_cache size 0 max 16384 2024-04-25T09:58:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.443+0000 7f298ec15740 1 Processor -- start 2024-04-25T09:58:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.443+0000 7f298ec15740 1 -- start start 2024-04-25T09:58:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f298ec15740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a6cca0 0x5609e4ad48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f298ec15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4acbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f298ec15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a90450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f298ec15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2984200410 con 0x5609e4a97760 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f298ec15740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29841ff5b0 con 0x5609e4aa9370 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f298ec15740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29841f40b0 con 0x5609e4a6cca0 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f29893d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4acbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f29893d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4acbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40176/0 (socket says 172.21.15.19:40176) 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f29893d3640 1 -- 172.21.15.19:0/353094884 learned_addr learned my addr 172.21.15.19:0/353094884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f29893d3640 1 -- 172.21.15.19:0/353094884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a6cca0 msgr2=0x5609e4ad48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f29893d3640 1 --2- 172.21.15.19:0/353094884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a6cca0 0x5609e4ad48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f29893d3640 1 -- 172.21.15.19:0/353094884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 msgr2=0x5609e4a90450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f29893d3640 1 --2- 172.21.15.19:0/353094884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a90450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f29893d3640 1 -- 172.21.15.19:0/353094884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e467d1b0 con 0x5609e4a97760 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f29893d3640 1 --2- 172.21.15.19:0/353094884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4acbf20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f2978000c00 tx=0x7f297803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04ce14fad878119 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f296affd640 1 -- 172.21.15.19:0/353094884 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978037040 con 0x5609e4a97760 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f296affd640 1 -- 172.21.15.19:0/353094884 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2978004030 con 0x5609e4a97760 2024-04-25T09:58:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.444+0000 7f296affd640 1 -- 172.21.15.19:0/353094884 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978004400 con 0x5609e4a97760 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.445+0000 7f298ec15740 1 -- 172.21.15.19:0/353094884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 msgr2=0x5609e4acbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.445+0000 7f298ec15740 1 --2- 172.21.15.19:0/353094884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4acbf20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f2978000c00 tx=0x7f297803d000 comp rx=0 tx=0).stop 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.445+0000 7f298ec15740 1 -- 172.21.15.19:0/353094884 shutdown_connections 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.445+0000 7f298ec15740 1 --2- 172.21.15.19:0/353094884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a90450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.445+0000 7f298ec15740 1 --2- 172.21.15.19:0/353094884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a97760 0x5609e4acbf20 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.445+0000 7f298ec15740 1 --2- 172.21.15.19:0/353094884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a6cca0 0x5609e4ad48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.445+0000 7f298ec15740 1 -- 172.21.15.19:0/353094884 >> 172.21.15.19:0/353094884 conn(0x5609e4a34870 msgr2=0x5609e4ab2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.445+0000 7f298ec15740 1 -- 172.21.15.19:0/353094884 shutdown_connections 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.445+0000 7f298ec15740 1 -- 172.21.15.19:0/353094884 wait complete. 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.445+0000 7f298ec15740 1 -- start start 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.446+0000 7f298ec15740 1 Processor -- start 2024-04-25T09:58:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.446+0000 7f298ec15740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a9abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.446+0000 7f298ec15740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a97760 0x5609e4a8aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.446+0000 7f298ec15740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a8b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.446+0000 7f298ec15740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e4955c30 con 0x5609e4a6cca0 2024-04-25T09:58:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.446+0000 7f298ec15740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29841fefb0 con 0x5609e4aa9370 2024-04-25T09:58:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.446+0000 7f298ec15740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609e4a68f10 con 0x5609e4a97760 2024-04-25T09:58:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.446+0000 7f298ec15740 10 client.0 _ll_register_callbacks cb 0x7f296c0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2989bd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a9abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2989bd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a9abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40184/0 (socket says 172.21.15.19:40184) 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2989bd4640 1 -- 172.21.15.19:0/492233674 learned_addr learned my addr 172.21.15.19:0/492233674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f29893d3640 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a97760 0x5609e4a8aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f298a3d5640 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a8b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2989bd4640 1 -- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a97760 msgr2=0x5609e4a8aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2989bd4640 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a97760 0x5609e4a8aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2989bd4640 1 -- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 msgr2=0x5609e4a8b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2989bd4640 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a8b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2989bd4640 1 -- 172.21.15.19:0/492233674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609e4a9b100 con 0x5609e4a6cca0 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2989bd4640 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a9abc0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f297c015fd0 tx=0x7f297c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf33e4f1732b2756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2968ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2968ff9640 1 -- 172.21.15.19:0/492233674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c011d20 con 0x5609e4a6cca0 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2968ff9640 1 -- 172.21.15.19:0/492233674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f297c037350 con 0x5609e4a6cca0 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f298ec15740 1 -- 172.21.15.19:0/492233674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2978004030 con 0x5609e4a6cca0 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2968ff9640 1 -- 172.21.15.19:0/492233674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f297c045050 con 0x5609e4a6cca0 2024-04-25T09:58:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f298ec15740 10 client.7395 fetch_fsmap learned FSMap version 52 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f298ec15740 1 -- 172.21.15.19:0/492233674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x5609e4a87ba0 con 0x5609e4a6cca0 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.447+0000 7f2968ff9640 1 -- 172.21.15.19:0/492233674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2978004030 con 0x5609e4a6cca0 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.448+0000 7f2968ff9640 1 -- 172.21.15.19:0/492233674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f297c048020 con 0x5609e4a6cca0 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.448+0000 7f298ec15740 10 client.7395 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.448+0000 7f298ec15740 10 client.7395 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.448+0000 7f298ec15740 20 client.7395 populate_metadata read hostname 'smithi019' 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.448+0000 7f298ec15740 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x5609e4b1c200 0x5609e4a6baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.448+0000 7f298ec15740 4 client.7395 mds_command: new command op to 7137 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.448+0000 7f298ec15740 1 -- 172.21.15.19:0/492233674 --> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5609e49d7420 con 0x5609e4b1c200 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.448+0000 7f29893d3640 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x5609e4b1c200 0x5609e4a6baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.448+0000 7f29893d3640 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x5609e4b1c200 0x5609e4a6baa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c07fb7d4800036df server_cookie=d7a00cb53ac7faa2 in_seq=0 out_seq=0 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.448+0000 7f2968ff9640 10 client.7395 ms_handle_connect on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f2968ff9640 1 -- 172.21.15.19:0/492233674 <== mds.0 v2:172.21.15.112:6835/1747929190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17937 (crc 0 0 0) 0x5609e49d7420 con 0x5609e4b1c200 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f2968ff9640 10 client.7395 handle_command_reply: tid=0 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 1 client.7395 shutdown 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 1 -- 172.21.15.19:0/492233674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 msgr2=0x5609e4a9abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a9abc0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f297c015fd0 tx=0x7f297c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 1 -- 172.21.15.19:0/492233674 shutdown_connections 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:6835/1747929190,v1:172.21.15.112:6837/1747929190] conn(0x5609e4b1c200 0x5609e4a6baa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609e4aa9370 0x5609e4a8b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5609e4a97760 0x5609e4a8aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f2968ff9640 0 client.7395 ms_handle_reset on v2:172.21.15.112:6835/1747929190 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 1 --2- 172.21.15.19:0/492233674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609e4a6cca0 0x5609e4a9abc0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 1 -- 172.21.15.19:0/492233674 >> 172.21.15.19:0/492233674 conn(0x5609e4a34870 msgr2=0x5609e4a9a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 1 -- 172.21.15.19:0/492233674 shutdown_connections 2024-04-25T09:58:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 1 -- 172.21.15.19:0/492233674 wait complete. 2024-04-25T09:58:50.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.449+0000 7f298ec15740 20 client.7395 trim_cache size 0 max 16384 2024-04-25T09:58:50.459 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:58:50.459 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:58:50.459 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 3027, 2024-04-25T09:58:50.459 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 3026, 2024-04-25T09:58:50.459 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1171003, 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 20064059, 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 17, 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 6, 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.256684262, 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.118500522, 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.049710893, 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.073605564, 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3025, 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.418245297, 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000138262 2024-04-25T09:58:50.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 10, 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000838, 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000083 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 29295, 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 51716, 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 9739843, 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 57912859, 2024-04-25T09:58:50.461 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 1.943898249, 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.957252233, 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 1.137948357, 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.099394702, 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 51716, 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.262491543, 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000024412 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 73709, 2024-04-25T09:58:50.462 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.007878989, 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000106 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 120976, 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 24272 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 3797, 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 3817, 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 951103, 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 43300238, 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-04-25T09:58:50.463 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.343020993, 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.174754105, 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.057431185, 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.094335423, 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3816, 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.326332416, 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000085516 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 117344 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2638, 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 15.065757653, 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.005711052 2024-04-25T09:58:50.465 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:50.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.466 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-04-25T09:58:50.466 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T09:58:50.466 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T09:58:50.466 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4361, 2024-04-25T09:58:50.466 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.853866278, 2024-04-25T09:58:50.466 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000195795 2024-04-25T09:58:50.466 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:50.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.466 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 25516, 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 25516, 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 25516, 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 293.395269380, 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.011498482 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 12, 2024-04-25T09:58:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 17803, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 17803, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 17803, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 2, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 2, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 2, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 46864, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 46454, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-04-25T09:58:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 118, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 2011, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-04-25T09:58:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 2011, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 11, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 2000, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 1114, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 4041, 2024-04-25T09:58:50.470 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 3937, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 3333, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 52, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 13995 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-04-25T09:58:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 4048, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 3843, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 2132, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-04-25T09:58:50.472 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 27239, 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 27239, 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 40, 2024-04-25T09:58:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 40, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 4, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 97982864, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 0, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2429, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 82.857027583, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.034111579 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 0 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 14, 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 2015, 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 2001, 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 13, 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 13, 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 17803, 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 19804, 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 2001, 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 2, 2024-04-25T09:58:50.475 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 7215, 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 7213, 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 103500, 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215396 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 28633, 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 25516, 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 5, 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2000, 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.289818615, 2024-04-25T09:58:50.476 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000644909 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 29, 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.005479571, 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000188950 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-04-25T09:58:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 6000, 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 6.656838964, 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001109473 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 7342, 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 6.145017641, 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000836967 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000051334, 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000051334 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.002760967, 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.002760967 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.003549779, 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.003549779 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8, 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.429034619, 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.178629327 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2132, 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 262.174218910, 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.122971022 2024-04-25T09:58:50.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.023007362, 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.023007362 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000249627, 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000249627 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.485 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8000, 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 15.665241991, 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001958155 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.487 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 1, 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 18643, 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 18643, 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 73, 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:58:50.488 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T09:58:50.489 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T09:58:50.490 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 250091, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 92, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 440, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 5, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:58:50.491 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 5528, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 78, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 13405181, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 660469, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.492 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 8532, 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 94476085, 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 37, 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 8532, 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8532, 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 31.501172794, 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.003692120 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8532, 2024-04-25T09:58:50.493 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 8625 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 8532, 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 2066, 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 6466, 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 0, 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 11, 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 54, 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 2482, 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 34, 2024-04-25T09:58:50.494 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 1843, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 2034, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 2012, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 22, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-04-25T09:58:50.495 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-04-25T09:58:50.496 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 83, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 3, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 8, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 8, 2024-04-25T09:58:50.497 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-04-25T09:58:50.498 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-04-25T09:58:50.498 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 104, 2024-04-25T09:58:50.498 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 0, 2024-04-25T09:58:50.498 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 29 2024-04-25T09:58:50.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.498 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-04-25T09:58:50.498 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 1, 2024-04-25T09:58:50.498 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:58:50.498 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-04-25T09:58:50.498 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-04-25T09:58:50.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.499 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-04-25T09:58:50.499 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-04-25T09:58:50.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.499 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-04-25T09:58:50.499 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-04-25T09:58:50.499 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:58:50.499 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-04-25T09:58:50.499 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 32, 2024-04-25T09:58:50.499 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 3843, 2024-04-25T09:58:50.499 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 3843, 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 36113, 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 8606049, 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 36113, 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:58:50.500 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 36113, 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 8606049, 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T09:58:50.501 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 8532, 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 94878972, 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 8532, 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 94878972, 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-04-25T09:58:50.502 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 8532, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 8532, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 8532, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 8532, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.503 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.504 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.504 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:50.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.504 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:58:50.504 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:58:50.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T09:58:50.504 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:58:50.504 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 3843, 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 402887, 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 3843, 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 53, 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 402887, 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:50.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55952ea786e0": { 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 27239, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 93788560, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 27239, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2429, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 93788560, 2024-04-25T09:58:50.506 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T09:58:50.507 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T09:58:50.507 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T09:58:50.507 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T09:58:50.507 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:50.507 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:58:50.507 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:58:50.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 -- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f297006b4d0 msgr2=0x7f297006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f297006b4d0 0x7f297006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f297c009a30 tx=0x7f297c008040 comp rx=0 tx=0).stop 2024-04-25T09:58:50.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 -- 172.21.15.19:0/2474426716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298415dc20 msgr2=0x7f298414a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298415dc20 0x7f298414a7d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f297400cb50 tx=0x7f297400a430 comp rx=0 tx=0).stop 2024-04-25T09:58:50.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 -- 172.21.15.19:0/2474426716 shutdown_connections 2024-04-25T09:58:50.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f297006b4d0 0x7f297006d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f298415dc20 0x7f298414a7d0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29840d1ce0 0x7f2984159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 --2- 172.21.15.19:0/2474426716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29840ce670 0x7f29841605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 -- 172.21.15.19:0/2474426716 >> 172.21.15.19:0/2474426716 conn(0x7f2984061a20 msgr2=0x7f2984061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 -- 172.21.15.19:0/2474426716 shutdown_connections 2024-04-25T09:58:50.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.450+0000 7f298be38640 1 -- 172.21.15.19:0/2474426716 wait complete. 2024-04-25T09:58:50.510 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:58:50.510 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:58:50.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:58:50.586 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:58:50.587 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:58:50.587 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:50.602 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:58:50.602 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:50.658 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:50.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.795+0000 7f8da1483640 1 Processor -- start 2024-04-25T09:58:50.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.795+0000 7f8da1483640 1 -- start start 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8da1483640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8d9c15dc20 0x7f8d9c1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8da1483640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8d9c0b93a0 0x7f8d9c0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8da1483640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0cec20 0x7f8d9c06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8da1483640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c15e000 con 0x7f8d9c15dc20 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8da1483640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c0b6bb0 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8da1483640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0b6030 con 0x7f8d9c0cec20 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8d9affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8d9c15dc20 0x7f8d9c1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8d9affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8d9c15dc20 0x7f8d9c1616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40188/0 (socket says 172.21.15.19:40188) 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8d9affd640 1 -- 172.21.15.19:0/188725716 learned_addr learned my addr 172.21.15.19:0/188725716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8d93fff640 1 --2- 172.21.15.19:0/188725716 >> v2:172.21.15.112:3300/0 conn(0x7f8d9c0b93a0 0x7f8d9c0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8d9b7fe640 1 --2- 172.21.15.19:0/188725716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0cec20 0x7f8d9c06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8d9affd640 1 -- 172.21.15.19:0/188725716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0cec20 msgr2=0x7f8d9c06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8d9affd640 1 --2- 172.21.15.19:0/188725716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0cec20 0x7f8d9c06c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8d9affd640 1 -- 172.21.15.19:0/188725716 >> v2:172.21.15.112:3300/0 conn(0x7f8d9c0b93a0 msgr2=0x7f8d9c0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8d9affd640 1 --2- 172.21.15.19:0/188725716 >> v2:172.21.15.112:3300/0 conn(0x7f8d9c0b93a0 0x7f8d9c0ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8d9affd640 1 -- 172.21.15.19:0/188725716 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c161de0 con 0x7f8d9c15dc20 2024-04-25T09:58:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.796+0000 7f8d93fff640 1 --2- 172.21.15.19:0/188725716 >> v2:172.21.15.112:3300/0 conn(0x7f8d9c0b93a0 0x7f8d9c0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.797+0000 7f8d9affd640 1 --2- 172.21.15.19:0/188725716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c15dc20 0x7f8d9c1616b0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0d4b10 tx=0x7f8d8c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4b89d0615d3d0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.797+0000 7f8d9a7fc640 1 -- 172.21.15.19:0/188725716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c037070 con 0x7f8d9c15dc20 2024-04-25T09:58:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.797+0000 7f8d9a7fc640 1 -- 172.21.15.19:0/188725716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c032440 con 0x7f8d9c15dc20 2024-04-25T09:58:50.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.797+0000 7f8d9a7fc640 1 -- 172.21.15.19:0/188725716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c032810 con 0x7f8d9c15dc20 2024-04-25T09:58:50.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.798+0000 7f8da1483640 1 -- 172.21.15.19:0/188725716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c15dc20 msgr2=0x7f8d9c1616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.798+0000 7f8da1483640 1 --2- 172.21.15.19:0/188725716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c15dc20 0x7f8d9c1616b0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0d4b10 tx=0x7f8d8c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.798+0000 7f8da1483640 1 -- 172.21.15.19:0/188725716 shutdown_connections 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.798+0000 7f8da1483640 1 --2- 172.21.15.19:0/188725716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0cec20 0x7f8d9c06c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.798+0000 7f8da1483640 1 --2- 172.21.15.19:0/188725716 >> v2:172.21.15.112:3300/0 conn(0x7f8d9c0b93a0 0x7f8d9c0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.798+0000 7f8da1483640 1 --2- 172.21.15.19:0/188725716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c15dc20 0x7f8d9c1616b0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.798+0000 7f8da1483640 1 -- 172.21.15.19:0/188725716 >> 172.21.15.19:0/188725716 conn(0x7f8d9c0d0490 msgr2=0x7f8d9c0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.798+0000 7f8da1483640 1 -- 172.21.15.19:0/188725716 shutdown_connections 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.798+0000 7f8da1483640 1 -- 172.21.15.19:0/188725716 wait complete. 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.799+0000 7f8da1483640 1 Processor -- start 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.799+0000 7f8da1483640 1 -- start start 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8da1483640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0b93a0 0x7f8d9c1f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8da1483640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0cec20 0x7f8d9c0ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8da1483640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d9c15dc20 0x7f8d9c14e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8da1483640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c06c610 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8da1483640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1623a0 con 0x7f8d9c15dc20 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8da1483640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c06c790 con 0x7f8d9c0cec20 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d9affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0b93a0 0x7f8d9c1f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d9affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0b93a0 0x7f8d9c1f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40192/0 (socket says 172.21.15.19:40192) 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d9affd640 1 -- 172.21.15.19:0/3482787158 learned_addr learned my addr 172.21.15.19:0/3482787158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d93fff640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0cec20 0x7f8d9c0ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d9b7fe640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d9c15dc20 0x7f8d9c14e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d9affd640 1 -- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0cec20 msgr2=0x7f8d9c0ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d9affd640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0cec20 0x7f8d9c0ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d9affd640 1 -- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d9c15dc20 msgr2=0x7f8d9c14e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d9affd640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d9c15dc20 0x7f8d9c14e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d9affd640 1 -- 172.21.15.19:0/3482787158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c0aa370 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d93fff640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0cec20 0x7f8d9c0ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d9affd640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0b93a0 0x7f8d9c1f6a80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c02a220 tx=0x7f8d8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d937fe640 1 -- 172.21.15.19:0/3482787158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c041020 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d937fe640 1 -- 172.21.15.19:0/3482787158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c0306c0 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8da1483640 1 -- 172.21.15.19:0/3482787158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c1f72d0 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8d937fe640 1 -- 172.21.15.19:0/3482787158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c037040 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.800+0000 7f8da1483640 1 -- 172.21.15.19:0/3482787158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c0aa610 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.801+0000 7f8da1483640 1 -- 172.21.15.19:0/3482787158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9c073990 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.802+0000 7f8d937fe640 1 -- 172.21.15.19:0/3482787158 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d8c044020 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.802+0000 7f8d937fe640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d7c06b4d0 0x7f8d7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.802+0000 7f8d937fe640 1 -- 172.21.15.19:0/3482787158 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f8d8c0aabe0 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.802+0000 7f8d93fff640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d7c06b4d0 0x7f8d7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.803+0000 7f8d93fff640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d7c06b4d0 0x7f8d7c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c06c080 tx=0x7f8d84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.807+0000 7f8d937fe640 1 -- 172.21.15.19:0/3482787158 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d8c04b030 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.988+0000 7f8da1483640 1 -- 172.21.15.19:0/3482787158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d9c0d1720 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.990+0000 7f8d937fe640 1 -- 172.21.15.19:0/3482787158 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f8d8c0798b0 con 0x7f8d9c0b93a0 2024-04-25T09:58:50.993 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:58:27.571687+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:24.581855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:25.297009+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":82,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 -- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d7c06b4d0 msgr2=0x7f8d7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d7c06b4d0 0x7f8d7c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c06c080 tx=0x7f8d84008040 comp rx=0 tx=0).stop 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 -- 172.21.15.19:0/3482787158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0b93a0 msgr2=0x7f8d9c1f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0b93a0 0x7f8d9c1f6a80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c02a220 tx=0x7f8d8c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 -- 172.21.15.19:0/3482787158 shutdown_connections 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d7c06b4d0 0x7f8d7c06d990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d9c15dc20 0x7f8d9c14e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0cec20 0x7f8d9c0ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 --2- 172.21.15.19:0/3482787158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0b93a0 0x7f8d9c1f6a80 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 -- 172.21.15.19:0/3482787158 >> 172.21.15.19:0/3482787158 conn(0x7f8d9c0d0490 msgr2=0x7f8d9c0b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 -- 172.21.15.19:0/3482787158 shutdown_connections 2024-04-25T09:58:50.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:50.992+0000 7f8da1483640 1 -- 172.21.15.19:0/3482787158 wait complete. 2024-04-25T09:58:51.007 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:57:24.581855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:57:25.297009+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 82, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:58:51.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:58:51.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.107+0000 7fe11a807640 1 Processor -- start 2024-04-25T09:58:51.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.107+0000 7fe11a807640 1 -- start start 2024-04-25T09:58:51.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe11a807640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140cc4f0 0x7fe1140cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe11a807640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe11415b980 0x7fe1140cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe11a807640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe114072280 con 0x7fe1140cdb20 2024-04-25T09:58:51.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe11a807640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1140ce4c0 con 0x7fe11415b980 2024-04-25T09:58:51.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe11a807640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe114071270 con 0x7fe1140cc4f0 2024-04-25T09:58:51.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe1137fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe1140cdb20 0x7fe11415b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52796/0 (socket says 172.21.15.19:52796) 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe1137fe640 1 -- 172.21.15.19:0/864674381 learned_addr learned my addr 172.21.15.19:0/864674381 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe118da4640 1 --2- 172.21.15.19:0/864674381 >> v2:172.21.15.112:3300/0 conn(0x7fe11415b980 0x7fe1140cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe113fff640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140cc4f0 0x7fe1140cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634009227 0 0) 0x7fe114072280 con 0x7fe1140cdb20 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0fc003520 con 0x7fe1140cdb20 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 978310125 0 0) 0x7fe0fc003520 con 0x7fe1140cdb20 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140cc4f0 msgr2=0x7fe1140cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140cc4f0 0x7fe1140cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 >> v2:172.21.15.112:3300/0 conn(0x7fe11415b980 msgr2=0x7fe1140cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 --2- 172.21.15.19:0/864674381 >> v2:172.21.15.112:3300/0 conn(0x7fe11415b980 0x7fe1140cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1140714c0 con 0x7fe1140cdb20 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fe108001670 con 0x7fe1140cdb20 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe113fff640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140cc4f0 0x7fe1140cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe118da4640 1 --2- 172.21.15.19:0/864674381 >> v2:172.21.15.112:3300/0 conn(0x7fe11415b980 0x7fe1140cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 >> v1:172.21.15.19:6789/0 conn(0x7fe1140cdb20 legacy=0x7fe11415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0fc0039b0 0x7fe0fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0fc004800 0x7fe0fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140cc4f0 0x7fe0fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe114071270 con 0x7fe0fc0039b0 2024-04-25T09:58:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc00b9c0 con 0x7fe0fc004800 2024-04-25T09:58:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.108+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc00bb40 con 0x7fe1140cc4f0 2024-04-25T09:58:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.109+0000 7fe113fff640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0fc0039b0 0x7fe0fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.109+0000 7fe1137fe640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0fc004800 0x7fe0fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.109+0000 7fe118da4640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140cc4f0 0x7fe0fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.109+0000 7fe113fff640 1 -- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140cc4f0 msgr2=0x7fe0fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.109+0000 7fe113fff640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140cc4f0 0x7fe0fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.109+0000 7fe113fff640 1 -- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0fc004800 msgr2=0x7fe0fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.109+0000 7fe113fff640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0fc004800 0x7fe0fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.109+0000 7fe113fff640 1 -- 172.21.15.19:0/864674381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc003520 con 0x7fe0fc0039b0 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.109+0000 7fe1137fe640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0fc004800 0x7fe0fc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.109+0000 7fe113fff640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0fc0039b0 0x7fe0fc003dd0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe104009c90 tx=0x7fe0fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e17171503b5b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.110+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104036070 con 0x7fe0fc0039b0 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.110+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe104031400 con 0x7fe0fc0039b0 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.110+0000 7fe112ffd640 1 -- 172.21.15.19:0/864674381 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104009860 con 0x7fe0fc0039b0 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.110+0000 7fe11a807640 1 -- 172.21.15.19:0/864674381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0fc0039b0 msgr2=0x7fe0fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.110+0000 7fe11a807640 1 --2- 172.21.15.19:0/864674381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0fc0039b0 0x7fe0fc003dd0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe104009c90 tx=0x7fe0fc005830 comp rx=0 tx=0).stop 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.110+0000 7fe118da4640 1 -- 172.21.15.19:0/864674381 reap_dead start 2024-04-25T09:58:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.110+0000 7fe11a807640 1 -- 172.21.15.19:0/864674381 shutdown_connections 2024-04-25T09:58:51.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.110+0000 7fe11a807640 1 -- 172.21.15.19:0/864674381 >> 172.21.15.19:0/864674381 conn(0x7fe11406abe0 msgr2=0x7fe11406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.110+0000 7fe11a807640 1 -- 172.21.15.19:0/864674381 shutdown_connections 2024-04-25T09:58:51.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.110+0000 7fe11a807640 1 -- 172.21.15.19:0/864674381 wait complete. 2024-04-25T09:58:51.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.111+0000 7fe11a807640 1 Processor -- start 2024-04-25T09:58:51.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe11a807640 1 -- start start 2024-04-25T09:58:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe11a807640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0fc0039b0 0x7fe114158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe11a807640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114157330 0x7fe1141577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe11a807640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140c9ef0 0x7fe1140ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe11a807640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe114071a80 con 0x7fe114157330 2024-04-25T09:58:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe11a807640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11415bd90 con 0x7fe0fc0039b0 2024-04-25T09:58:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe11a807640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe114071d60 con 0x7fe1140c9ef0 2024-04-25T09:58:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe1137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114157330 0x7fe1141577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe1137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114157330 0x7fe1141577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40212/0 (socket says 172.21.15.19:40212) 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe1137fe640 1 -- 172.21.15.19:0/3588061654 learned_addr learned my addr 172.21.15.19:0/3588061654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe113fff640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0fc0039b0 0x7fe114158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.112+0000 7fe118da4640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140c9ef0 0x7fe1140ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.113+0000 7fe1137fe640 1 -- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140c9ef0 msgr2=0x7fe1140ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.113+0000 7fe1137fe640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140c9ef0 0x7fe1140ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.113+0000 7fe1137fe640 1 -- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0fc0039b0 msgr2=0x7fe114158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.113+0000 7fe1137fe640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0fc0039b0 0x7fe114158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.113+0000 7fe1137fe640 1 -- 172.21.15.19:0/3588061654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1140c8e50 con 0x7fe114157330 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.113+0000 7fe1137fe640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114157330 0x7fe1141577b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe10800e850 tx=0x7fe10800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.113+0000 7fe110ff9640 1 -- 172.21.15.19:0/3588061654 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108019070 con 0x7fe114157330 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.113+0000 7fe110ff9640 1 -- 172.21.15.19:0/3588061654 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10800c8d0 con 0x7fe114157330 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.113+0000 7fe110ff9640 1 -- 172.21.15.19:0/3588061654 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800cc80 con 0x7fe114157330 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.113+0000 7fe11a807640 1 -- 172.21.15.19:0/3588061654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1140c9130 con 0x7fe114157330 2024-04-25T09:58:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.113+0000 7fe11a807640 1 -- 172.21.15.19:0/3588061654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1140c96f0 con 0x7fe114157330 2024-04-25T09:58:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.114+0000 7fe11a807640 1 -- 172.21.15.19:0/3588061654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1140ce960 con 0x7fe114157330 2024-04-25T09:58:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.114+0000 7fe110ff9640 1 -- 172.21.15.19:0/3588061654 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe10801c070 con 0x7fe114157330 2024-04-25T09:58:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.115+0000 7fe110ff9640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10006b4d0 0x7fe10006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.115+0000 7fe110ff9640 1 -- 172.21.15.19:0/3588061654 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fe108089be0 con 0x7fe114157330 2024-04-25T09:58:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.115+0000 7fe113fff640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10006b4d0 0x7fe10006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.116+0000 7fe113fff640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10006b4d0 0x7fe10006d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe104033000 tx=0x7fe104008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.119+0000 7fe110ff9640 1 -- 172.21.15.19:0/3588061654 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1140ce960 con 0x7fe114157330 2024-04-25T09:58:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.302+0000 7fe11a807640 1 -- 172.21.15.19:0/3588061654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe114073a00 con 0x7fe114157330 2024-04-25T09:58:51.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.303+0000 7fe110ff9640 1 -- 172.21.15.19:0/3588061654 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1080581d0 con 0x7fe114157330 2024-04-25T09:58:51.307 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:58:51.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 -- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10006b4d0 msgr2=0x7fe10006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10006b4d0 0x7fe10006d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe104033000 tx=0x7fe104008030 comp rx=0 tx=0).stop 2024-04-25T09:58:51.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 -- 172.21.15.19:0/3588061654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114157330 msgr2=0x7fe1141577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114157330 0x7fe1141577b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe10800e850 tx=0x7fe10800ed10 comp rx=0 tx=0).stop 2024-04-25T09:58:51.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 -- 172.21.15.19:0/3588061654 shutdown_connections 2024-04-25T09:58:51.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe10006b4d0 0x7fe10006d990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1140c9ef0 0x7fe1140ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114157330 0x7fe1141577b0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 --2- 172.21.15.19:0/3588061654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0fc0039b0 0x7fe114158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 -- 172.21.15.19:0/3588061654 >> 172.21.15.19:0/3588061654 conn(0x7fe11406abe0 msgr2=0x7fe11415c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 -- 172.21.15.19:0/3588061654 shutdown_connections 2024-04-25T09:58:51.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.305+0000 7fe11a807640 1 -- 172.21.15.19:0/3588061654 wait complete. 2024-04-25T09:58:51.320 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:58:51.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.419+0000 7fc90f59e640 1 Processor -- start 2024-04-25T09:58:51.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.419+0000 7fc90f59e640 1 -- start start 2024-04-25T09:58:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.420+0000 7fc90f59e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc90806abe0 0x7fc9080b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.420+0000 7fc90f59e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9080af470 0x7fc90806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.420+0000 7fc90f59e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc90806b500 0x7fc9080adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.420+0000 7fc90f59e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9080b5870 con 0x7fc90806abe0 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.420+0000 7fc90f59e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90806fe30 con 0x7fc90806b500 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.420+0000 7fc90f59e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90806b990 con 0x7fc9080af470 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.421+0000 7fc90e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc90806abe0 0x7fc9080b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.421+0000 7fc90e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc90806abe0 0x7fc9080b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40214/0 (socket says 172.21.15.19:40214) 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.421+0000 7fc90e59c640 1 -- 172.21.15.19:0/3199595827 learned_addr learned my addr 172.21.15.19:0/3199595827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.421+0000 7fc90dd9b640 1 --2- 172.21.15.19:0/3199595827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9080af470 0x7fc90806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.421+0000 7fc90ed9d640 1 --2- 172.21.15.19:0/3199595827 >> v2:172.21.15.112:3300/0 conn(0x7fc90806b500 0x7fc9080adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.421+0000 7fc90e59c640 1 -- 172.21.15.19:0/3199595827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9080af470 msgr2=0x7fc90806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.421+0000 7fc90e59c640 1 --2- 172.21.15.19:0/3199595827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9080af470 0x7fc90806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.421+0000 7fc90e59c640 1 -- 172.21.15.19:0/3199595827 >> v2:172.21.15.112:3300/0 conn(0x7fc90806b500 msgr2=0x7fc9080adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.421+0000 7fc90e59c640 1 --2- 172.21.15.19:0/3199595827 >> v2:172.21.15.112:3300/0 conn(0x7fc90806b500 0x7fc9080adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.421+0000 7fc90e59c640 1 -- 172.21.15.19:0/3199595827 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9080b3650 con 0x7fc90806abe0 2024-04-25T09:58:51.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90e59c640 1 --2- 172.21.15.19:0/3199595827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90806abe0 0x7fc9080b5330 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc9080b47a0 tx=0x7fc8fc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883fb1a9fa55b071 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90d59a640 1 -- 172.21.15.19:0/3199595827 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc037070 con 0x7fc90806abe0 2024-04-25T09:58:51.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90d59a640 1 -- 172.21.15.19:0/3199595827 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc02ae00 con 0x7fc90806abe0 2024-04-25T09:58:51.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90d59a640 1 -- 172.21.15.19:0/3199595827 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc032660 con 0x7fc90806abe0 2024-04-25T09:58:51.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90f59e640 1 -- 172.21.15.19:0/3199595827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90806abe0 msgr2=0x7fc9080b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90f59e640 1 --2- 172.21.15.19:0/3199595827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90806abe0 0x7fc9080b5330 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc9080b47a0 tx=0x7fc8fc02a0b0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90f59e640 1 -- 172.21.15.19:0/3199595827 shutdown_connections 2024-04-25T09:58:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90f59e640 1 --2- 172.21.15.19:0/3199595827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9080af470 0x7fc90806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90f59e640 1 --2- 172.21.15.19:0/3199595827 >> v2:172.21.15.112:3300/0 conn(0x7fc90806b500 0x7fc9080adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90f59e640 1 --2- 172.21.15.19:0/3199595827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90806abe0 0x7fc9080b5330 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90f59e640 1 -- 172.21.15.19:0/3199595827 >> 172.21.15.19:0/3199595827 conn(0x7fc9080b5a40 msgr2=0x7fc9080b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90f59e640 1 -- 172.21.15.19:0/3199595827 shutdown_connections 2024-04-25T09:58:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.422+0000 7fc90f59e640 1 -- 172.21.15.19:0/3199595827 wait complete. 2024-04-25T09:58:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.424+0000 7fc90f59e640 1 Processor -- start 2024-04-25T09:58:51.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.424+0000 7fc90f59e640 1 -- start start 2024-04-25T09:58:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.424+0000 7fc90f59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90806abe0 0x7fc90813fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.424+0000 7fc90f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90806b500 0x7fc908142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.424+0000 7fc90f59e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9080af470 0x7fc908155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.424+0000 7fc90f59e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc908070360 con 0x7fc90806b500 2024-04-25T09:58:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.424+0000 7fc90f59e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc908070080 con 0x7fc90806abe0 2024-04-25T09:58:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.424+0000 7fc90f59e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc908161210 con 0x7fc9080af470 2024-04-25T09:58:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90806b500 0x7fc908142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90806b500 0x7fc908142480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40216/0 (socket says 172.21.15.19:40216) 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90dd9b640 1 -- 172.21.15.19:0/902241554 learned_addr learned my addr 172.21.15.19:0/902241554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90ed9d640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9080af470 0x7fc908155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90e59c640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90806abe0 0x7fc90813fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90dd9b640 1 -- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9080af470 msgr2=0x7fc908155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90dd9b640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9080af470 0x7fc908155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90dd9b640 1 -- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90806abe0 msgr2=0x7fc90813fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90dd9b640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90806abe0 0x7fc90813fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90dd9b640 1 -- 172.21.15.19:0/902241554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc908142ed0 con 0x7fc90806b500 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90e59c640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90806abe0 0x7fc90813fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90ed9d640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9080af470 0x7fc908155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90dd9b640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90806b500 0x7fc908142480 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc90000d5f0 tx=0x7fc90000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc8faffd640 1 -- 172.21.15.19:0/902241554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000125f0 con 0x7fc90806b500 2024-04-25T09:58:51.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc8faffd640 1 -- 172.21.15.19:0/902241554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc900019070 con 0x7fc90806b500 2024-04-25T09:58:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc8faffd640 1 -- 172.21.15.19:0/902241554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000129d0 con 0x7fc90806b500 2024-04-25T09:58:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90f59e640 1 -- 172.21.15.19:0/902241554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9081431b0 con 0x7fc90806b500 2024-04-25T09:58:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.425+0000 7fc90f59e640 1 -- 172.21.15.19:0/902241554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90809a7f0 con 0x7fc90806b500 2024-04-25T09:58:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.426+0000 7fc90f59e640 1 -- 172.21.15.19:0/902241554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc908073990 con 0x7fc90806b500 2024-04-25T09:58:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.427+0000 7fc8faffd640 1 -- 172.21.15.19:0/902241554 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc900012b70 con 0x7fc90806b500 2024-04-25T09:58:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.427+0000 7fc8faffd640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8ec06b500 0x7fc8ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.427+0000 7fc8faffd640 1 -- 172.21.15.19:0/902241554 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fc90008a7d0 con 0x7fc90806b500 2024-04-25T09:58:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.427+0000 7fc90e59c640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8ec06b500 0x7fc8ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.428+0000 7fc90e59c640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8ec06b500 0x7fc8ec06d9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc034000 tx=0x7fc8fc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.432+0000 7fc8faffd640 1 -- 172.21.15.19:0/902241554 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc9000594a0 con 0x7fc90806b500 2024-04-25T09:58:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.610+0000 7fc90f59e640 1 -- 172.21.15.19:0/902241554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc908143380 con 0x7fc90806b500 2024-04-25T09:58:51.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.612+0000 7fc8faffd640 1 -- 172.21.15.19:0/902241554 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc900058da0 con 0x7fc90806b500 2024-04-25T09:58:51.619 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:58:51.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 -- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8ec06b500 msgr2=0x7fc8ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8ec06b500 0x7fc8ec06d9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc034000 tx=0x7fc8fc02e000 comp rx=0 tx=0).stop 2024-04-25T09:58:51.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 -- 172.21.15.19:0/902241554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90806b500 msgr2=0x7fc908142480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90806b500 0x7fc908142480 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc90000d5f0 tx=0x7fc90000dab0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 -- 172.21.15.19:0/902241554 shutdown_connections 2024-04-25T09:58:51.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8ec06b500 0x7fc8ec06d9c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9080af470 0x7fc908155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90806b500 0x7fc908142480 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 --2- 172.21.15.19:0/902241554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90806abe0 0x7fc90813fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 -- 172.21.15.19:0/902241554 >> 172.21.15.19:0/902241554 conn(0x7fc9080b5a40 msgr2=0x7fc9080b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 -- 172.21.15.19:0/902241554 shutdown_connections 2024-04-25T09:58:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.614+0000 7fc90f59e640 1 -- 172.21.15.19:0/902241554 wait complete. 2024-04-25T09:58:51.628 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:58:51.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.728+0000 7fecd279d640 1 Processor -- start 2024-04-25T09:58:51.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.728+0000 7fecd279d640 1 -- start start 2024-04-25T09:58:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7fecd279d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feccc15dc20 0x7feccc0d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7fecd279d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feccc0ced80 0x7feccc0b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7fecd279d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feccc0d2120 con 0x7feccc0d1ce0 2024-04-25T09:58:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7fecd279d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feccc0d23a0 con 0x7feccc0ced80 2024-04-25T09:58:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7fecd279d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feccc0d2680 con 0x7feccc15dc20 2024-04-25T09:58:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7fecd0d3a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feccc0ced80 0x7feccc0b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7feccb7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feccc0d1ce0 0x7feccc0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52808/0 (socket says 172.21.15.19:52808) 2024-04-25T09:58:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7feccb7fe640 1 -- 172.21.15.19:0/3218235237 learned_addr learned my addr 172.21.15.19:0/3218235237 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:51.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7feccbfff640 1 --2- 172.21.15.19:0/3218235237 >> v2:172.21.15.112:3300/0 conn(0x7feccc15dc20 0x7feccc0d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7fecd0d3a640 1 -- 172.21.15.19:0/3218235237 >> v2:172.21.15.112:3300/0 conn(0x7feccc15dc20 msgr2=0x7feccc0d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7fecd0d3a640 1 --2- 172.21.15.19:0/3218235237 >> v2:172.21.15.112:3300/0 conn(0x7feccc15dc20 0x7feccc0d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7fecd0d3a640 1 -- 172.21.15.19:0/3218235237 >> v1:172.21.15.19:6789/0 conn(0x7feccc0d1ce0 legacy=0x7feccc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7fecd0d3a640 1 -- 172.21.15.19:0/3218235237 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc0d2960 con 0x7feccc0ced80 2024-04-25T09:58:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.729+0000 7feccaffd640 1 -- 172.21.15.19:0/3218235237 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227081513 0 0) 0x7fecbc001110 con 0x7feccc0d1ce0 2024-04-25T09:58:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.730+0000 7fecd0d3a640 1 --2- 172.21.15.19:0/3218235237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0ced80 0x7feccc0b9490 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7feccc0d1500 tx=0x7fecc402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e3aa8a541483a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.730+0000 7feccaffd640 1 -- 172.21.15.19:0/3218235237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4007740 con 0x7feccc0ced80 2024-04-25T09:58:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.730+0000 7feccaffd640 1 -- 172.21.15.19:0/3218235237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc40078e0 con 0x7feccc0ced80 2024-04-25T09:58:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.730+0000 7feccaffd640 1 -- 172.21.15.19:0/3218235237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4007cb0 con 0x7feccc0ced80 2024-04-25T09:58:51.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.731+0000 7fecd279d640 1 -- 172.21.15.19:0/3218235237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0ced80 msgr2=0x7feccc0b9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.731+0000 7fecd279d640 1 --2- 172.21.15.19:0/3218235237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0ced80 0x7feccc0b9490 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7feccc0d1500 tx=0x7fecc402a5f0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.731+0000 7fecd279d640 1 -- 172.21.15.19:0/3218235237 shutdown_connections 2024-04-25T09:58:51.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.731+0000 7fecd279d640 1 --2- 172.21.15.19:0/3218235237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0ced80 0x7feccc0b9490 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.731+0000 7fecd279d640 1 --2- 172.21.15.19:0/3218235237 >> v2:172.21.15.112:3300/0 conn(0x7feccc15dc20 0x7feccc0d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.731+0000 7fecd279d640 1 -- 172.21.15.19:0/3218235237 >> 172.21.15.19:0/3218235237 conn(0x7feccc061a20 msgr2=0x7feccc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.731+0000 7fecd279d640 1 -- 172.21.15.19:0/3218235237 shutdown_connections 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.731+0000 7fecd279d640 1 -- 172.21.15.19:0/3218235237 wait complete. 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.732+0000 7fecd279d640 1 Processor -- start 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.732+0000 7fecd279d640 1 -- start start 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd279d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feccc0ced80 0x7feccc0cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd279d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc0d1ce0 0x7feccc14c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd279d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc15dc20 0x7feccc1e6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd279d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc0b6a30 con 0x7feccc15dc20 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd279d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc0d2f20 con 0x7feccc0ced80 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd279d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feccc0b6bb0 con 0x7feccc0d1ce0 2024-04-25T09:58:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd0d3a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc15dc20 0x7feccc1e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd0d3a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc15dc20 0x7feccc1e6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40228/0 (socket says 172.21.15.19:40228) 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd0d3a640 1 -- 172.21.15.19:0/1854474244 learned_addr learned my addr 172.21.15.19:0/1854474244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7feccb7fe640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc0d1ce0 0x7feccc14c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7feccbfff640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feccc0ced80 0x7feccc0cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd0d3a640 1 -- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc0d1ce0 msgr2=0x7feccc14c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd0d3a640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc0d1ce0 0x7feccc14c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd0d3a640 1 -- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feccc0ced80 msgr2=0x7feccc0cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.733+0000 7fecd0d3a640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feccc0ced80 0x7feccc0cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.734+0000 7fecd0d3a640 1 -- 172.21.15.19:0/1854474244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc0cb680 con 0x7feccc15dc20 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.734+0000 7fecd0d3a640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc15dc20 0x7feccc1e6050 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fecc402abe0 tx=0x7fecc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.734+0000 7fecc8ff9640 1 -- 172.21.15.19:0/1854474244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4040070 con 0x7feccc15dc20 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.734+0000 7fecc8ff9640 1 -- 172.21.15.19:0/1854474244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc4031bc0 con 0x7feccc15dc20 2024-04-25T09:58:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.734+0000 7fecd279d640 1 -- 172.21.15.19:0/1854474244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feccc14a710 con 0x7feccc15dc20 2024-04-25T09:58:51.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.734+0000 7fecc8ff9640 1 -- 172.21.15.19:0/1854474244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4045460 con 0x7feccc15dc20 2024-04-25T09:58:51.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.734+0000 7fecd279d640 1 -- 172.21.15.19:0/1854474244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feccc14acd0 con 0x7feccc15dc20 2024-04-25T09:58:51.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.735+0000 7fecd279d640 1 -- 172.21.15.19:0/1854474244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feccc15ce50 con 0x7feccc15dc20 2024-04-25T09:58:51.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.735+0000 7fecc8ff9640 1 -- 172.21.15.19:0/1854474244 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecc4039040 con 0x7feccc15dc20 2024-04-25T09:58:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.736+0000 7fecc8ff9640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecb806b4d0 0x7fecb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.736+0000 7fecc8ff9640 1 -- 172.21.15.19:0/1854474244 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fecc40abb00 con 0x7feccc15dc20 2024-04-25T09:58:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.736+0000 7feccbfff640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecb806b4d0 0x7fecb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.737+0000 7feccbfff640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecb806b4d0 0x7fecb806d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fecc00079a0 tx=0x7fecc0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.740+0000 7fecc8ff9640 1 -- 172.21.15.19:0/1854474244 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fecc407a330 con 0x7feccc15dc20 2024-04-25T09:58:51.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.920+0000 7fecd279d640 1 -- 172.21.15.19:0/1854474244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feccc073a00 con 0x7feccc15dc20 2024-04-25T09:58:51.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.920+0000 7fecc8ff9640 1 -- 172.21.15.19:0/1854474244 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7fecc407e3d0 con 0x7feccc15dc20 2024-04-25T09:58:51.924 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:58:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 -- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecb806b4d0 msgr2=0x7fecb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecb806b4d0 0x7fecb806d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fecc00079a0 tx=0x7fecc0008040 comp rx=0 tx=0).stop 2024-04-25T09:58:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 -- 172.21.15.19:0/1854474244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc15dc20 msgr2=0x7feccc1e6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc15dc20 0x7feccc1e6050 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fecc402abe0 tx=0x7fecc4004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 -- 172.21.15.19:0/1854474244 shutdown_connections 2024-04-25T09:58:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecb806b4d0 0x7fecb806d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc15dc20 0x7feccc1e6050 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feccc0d1ce0 0x7feccc14c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 --2- 172.21.15.19:0/1854474244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feccc0ced80 0x7feccc0cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 -- 172.21.15.19:0/1854474244 >> 172.21.15.19:0/1854474244 conn(0x7feccc061a20 msgr2=0x7feccc0ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 -- 172.21.15.19:0/1854474244 shutdown_connections 2024-04-25T09:58:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:51.922+0000 7fecd279d640 1 -- 172.21.15.19:0/1854474244 wait complete. 2024-04-25T09:58:51.936 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:58:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.034+0000 7f334173b640 1 Processor -- start 2024-04-25T09:58:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.034+0000 7f334173b640 1 -- start start 2024-04-25T09:58:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.035+0000 7f334173b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f333c0ce900 0x7f333c166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.035+0000 7f334173b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0ced00 con 0x7f333c155f80 2024-04-25T09:58:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.035+0000 7f334173b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0b4d80 con 0x7f333c06fbd0 2024-04-25T09:58:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.035+0000 7f334173b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333c0b5390 con 0x7f333c0ce900 2024-04-25T09:58:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.035+0000 7f333affd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f333c155f80 0x7f333c06f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52820/0 (socket says 172.21.15.19:52820) 2024-04-25T09:58:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.035+0000 7f333b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f333c0ce900 0x7f333c166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.035+0000 7f333affd640 1 -- 172.21.15.19:0/2990760493 learned_addr learned my addr 172.21.15.19:0/2990760493 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:52.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.035+0000 7f333b7fe640 1 -- 172.21.15.19:0/2990760493 >> v1:172.21.15.112:6789/0 conn(0x7f333c06fbd0 legacy=0x7f333c0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.035+0000 7f333b7fe640 1 -- 172.21.15.19:0/2990760493 >> v1:172.21.15.19:6789/0 conn(0x7f333c155f80 legacy=0x7f333c06f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.035+0000 7f333b7fe640 1 -- 172.21.15.19:0/2990760493 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0cef70 con 0x7f333c0ce900 2024-04-25T09:58:52.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.035+0000 7f333a7fc640 1 -- 172.21.15.19:0/2990760493 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917522936 0 0) 0x7f333c0ced00 con 0x7f333c155f80 2024-04-25T09:58:52.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.036+0000 7f333b7fe640 1 --2- 172.21.15.19:0/2990760493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c0ce900 0x7f333c166870 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f333c0b5510 tx=0x7f333402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c49bce388fabb95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.036+0000 7f333a7fc640 1 -- 172.21.15.19:0/2990760493 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334007740 con 0x7f333c0ce900 2024-04-25T09:58:52.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.036+0000 7f333a7fc640 1 -- 172.21.15.19:0/2990760493 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33340078e0 con 0x7f333c0ce900 2024-04-25T09:58:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.036+0000 7f333a7fc640 1 -- 172.21.15.19:0/2990760493 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334007cb0 con 0x7f333c0ce900 2024-04-25T09:58:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.037+0000 7f334173b640 1 -- 172.21.15.19:0/2990760493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c0ce900 msgr2=0x7f333c166870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.037+0000 7f334173b640 1 --2- 172.21.15.19:0/2990760493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c0ce900 0x7f333c166870 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f333c0b5510 tx=0x7f333402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.037+0000 7f334173b640 1 -- 172.21.15.19:0/2990760493 shutdown_connections 2024-04-25T09:58:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.037+0000 7f334173b640 1 --2- 172.21.15.19:0/2990760493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c0ce900 0x7f333c166870 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.037+0000 7f334173b640 1 -- 172.21.15.19:0/2990760493 >> 172.21.15.19:0/2990760493 conn(0x7f333c061600 msgr2=0x7f333c0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.037+0000 7f334173b640 1 -- 172.21.15.19:0/2990760493 shutdown_connections 2024-04-25T09:58:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.037+0000 7f334173b640 1 -- 172.21.15.19:0/2990760493 wait complete. 2024-04-25T09:58:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.038+0000 7f334173b640 1 Processor -- start 2024-04-25T09:58:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.038+0000 7f334173b640 1 -- start start 2024-04-25T09:58:52.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f334173b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c06fbd0 0x7f333c14d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f334173b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c0ce900 0x7f333c0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f334173b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c155f80 0x7f333c149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f334173b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c156360 con 0x7f333c0ce900 2024-04-25T09:58:52.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f334173b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0d07f0 con 0x7f333c06fbd0 2024-04-25T09:58:52.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f334173b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0cf250 con 0x7f333c155f80 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f3333fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c0ce900 0x7f333c0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f3333fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c0ce900 0x7f333c0b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40258/0 (socket says 172.21.15.19:40258) 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f3333fff640 1 -- 172.21.15.19:0/3456982405 learned_addr learned my addr 172.21.15.19:0/3456982405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f333b7fe640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c155f80 0x7f333c149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f333affd640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c06fbd0 0x7f333c14d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f3333fff640 1 -- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c155f80 msgr2=0x7f333c149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f3333fff640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c155f80 0x7f333c149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f3333fff640 1 -- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c06fbd0 msgr2=0x7f333c14d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f3333fff640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c06fbd0 0x7f333c14d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f3333fff640 1 -- 172.21.15.19:0/3456982405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0b2700 con 0x7f333c0ce900 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f3333fff640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c0ce900 0x7f333c0b1c60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f332400c450 tx=0x7f332400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.039+0000 7f33337fe640 1 -- 172.21.15.19:0/3456982405 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33240052d0 con 0x7f333c0ce900 2024-04-25T09:58:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.040+0000 7f33337fe640 1 -- 172.21.15.19:0/3456982405 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3324005490 con 0x7f333c0ce900 2024-04-25T09:58:52.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.040+0000 7f33337fe640 1 -- 172.21.15.19:0/3456982405 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324003880 con 0x7f333c0ce900 2024-04-25T09:58:52.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.040+0000 7f334173b640 1 -- 172.21.15.19:0/3456982405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c0b29e0 con 0x7f333c0ce900 2024-04-25T09:58:52.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.040+0000 7f334173b640 1 -- 172.21.15.19:0/3456982405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c14bbc0 con 0x7f333c0ce900 2024-04-25T09:58:52.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.041+0000 7f334173b640 1 -- 172.21.15.19:0/3456982405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333c073a00 con 0x7f333c0ce900 2024-04-25T09:58:52.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.041+0000 7f33337fe640 1 -- 172.21.15.19:0/3456982405 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3324003a20 con 0x7f333c0ce900 2024-04-25T09:58:52.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.041+0000 7f33337fe640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f331c06b500 0x7f331c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.042+0000 7f33337fe640 1 -- 172.21.15.19:0/3456982405 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f3324089f30 con 0x7f333c0ce900 2024-04-25T09:58:52.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.042+0000 7f333affd640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f331c06b500 0x7f331c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.042+0000 7f333affd640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f331c06b500 0x7f331c06d9c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f332c000f60 tx=0x7f332c007e40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.046+0000 7f33337fe640 1 -- 172.21.15.19:0/3456982405 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3324058510 con 0x7f333c0ce900 2024-04-25T09:58:52.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.238+0000 7f334173b640 1 -- 172.21.15.19:0/3456982405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f333c0b1030 con 0x7f333c0ce900 2024-04-25T09:58:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.238+0000 7f33337fe640 1 -- 172.21.15.19:0/3456982405 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) v1 ==== 80+0+0 (secure 0 0 0) 0x7f332405c5b0 con 0x7f333c0ce900 2024-04-25T09:58:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T09:58:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 -- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f331c06b500 msgr2=0x7f331c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f331c06b500 0x7f331c06d9c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f332c000f60 tx=0x7f332c007e40 comp rx=0 tx=0).stop 2024-04-25T09:58:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 -- 172.21.15.19:0/3456982405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c0ce900 msgr2=0x7f333c0b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c0ce900 0x7f333c0b1c60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f332400c450 tx=0x7f332400c910 comp rx=0 tx=0).stop 2024-04-25T09:58:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 -- 172.21.15.19:0/3456982405 shutdown_connections 2024-04-25T09:58:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f331c06b500 0x7f331c06d9c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c155f80 0x7f333c149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c0ce900 0x7f333c0b1c60 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 --2- 172.21.15.19:0/3456982405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c06fbd0 0x7f333c14d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 -- 172.21.15.19:0/3456982405 >> 172.21.15.19:0/3456982405 conn(0x7f333c061600 msgr2=0x7f333c06f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 -- 172.21.15.19:0/3456982405 shutdown_connections 2024-04-25T09:58:52.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.240+0000 7f334173b640 1 -- 172.21.15.19:0/3456982405 wait complete. 2024-04-25T09:58:52.254 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:58:52.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:52.255 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:52.293 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:58:52.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:52.293 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:58:52.349 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:58:52.349 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:52.367 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:58:52.367 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:58:52.367 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:52.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:58:52.405 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:58:52.405 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:52.424 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:58:52.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.542+0000 7f0c270dd640 1 Processor -- start 2024-04-25T09:58:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.542+0000 7f0c270dd640 1 -- start start 2024-04-25T09:58:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.543+0000 7f0c270dd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c2015dbe0 0x7f0c200d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.543+0000 7f0c270dd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c200b2cb0 0x7f0c200d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.543+0000 7f0c270dd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c200b97a0 con 0x7f0c200b92a0 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.543+0000 7f0c270dd640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2015dfc0 con 0x7f0c2015dbe0 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.543+0000 7f0c270dd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200d1a00 con 0x7f0c200b2cb0 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.543+0000 7f0c24e79640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c2015dbe0 0x7f0c200d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.543+0000 7f0c2567a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0c200b92a0 0x7f0c200b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52824/0 (socket says 172.21.15.19:52824) 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.543+0000 7f0c2567a640 1 -- 172.21.15.19:0/685829417 learned_addr learned my addr 172.21.15.19:0/685829417 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.544+0000 7f0c1ffff640 1 --2- 172.21.15.19:0/685829417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c200b2cb0 0x7f0c200d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.544+0000 7f0c24e79640 1 -- 172.21.15.19:0/685829417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c200b2cb0 msgr2=0x7f0c200d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.544+0000 7f0c24e79640 1 --2- 172.21.15.19:0/685829417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c200b2cb0 0x7f0c200d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.544+0000 7f0c24e79640 1 -- 172.21.15.19:0/685829417 >> v1:172.21.15.19:6789/0 conn(0x7f0c200b92a0 legacy=0x7f0c200b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.544+0000 7f0c1f7fe640 1 -- 172.21.15.19:0/685829417 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197060092 0 0) 0x7f0c200b97a0 con 0x7f0c200b92a0 2024-04-25T09:58:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.544+0000 7f0c24e79640 1 -- 172.21.15.19:0/685829417 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200d1c50 con 0x7f0c2015dbe0 2024-04-25T09:58:52.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.544+0000 7f0c1ffff640 1 --2- 172.21.15.19:0/685829417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c200b2cb0 0x7f0c200d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:52.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.544+0000 7f0c24e79640 1 --2- 172.21.15.19:0/685829417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015dbe0 0x7f0c200d0910 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009d90 tx=0x7f0c1802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3d4775b14e2bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.545+0000 7f0c1f7fe640 1 -- 172.21.15.19:0/685829417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18007740 con 0x7f0c2015dbe0 2024-04-25T09:58:52.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.545+0000 7f0c1f7fe640 1 -- 172.21.15.19:0/685829417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c180078e0 con 0x7f0c2015dbe0 2024-04-25T09:58:52.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.545+0000 7f0c1f7fe640 1 -- 172.21.15.19:0/685829417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18007cb0 con 0x7f0c2015dbe0 2024-04-25T09:58:52.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.545+0000 7f0c270dd640 1 -- 172.21.15.19:0/685829417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015dbe0 msgr2=0x7f0c200d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.545+0000 7f0c270dd640 1 --2- 172.21.15.19:0/685829417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015dbe0 0x7f0c200d0910 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009d90 tx=0x7f0c1802a660 comp rx=0 tx=0).stop 2024-04-25T09:58:52.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.545+0000 7f0c270dd640 1 -- 172.21.15.19:0/685829417 shutdown_connections 2024-04-25T09:58:52.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.545+0000 7f0c270dd640 1 --2- 172.21.15.19:0/685829417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c200b2cb0 0x7f0c200d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.545+0000 7f0c270dd640 1 --2- 172.21.15.19:0/685829417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015dbe0 0x7f0c200d0910 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.545+0000 7f0c270dd640 1 -- 172.21.15.19:0/685829417 >> 172.21.15.19:0/685829417 conn(0x7f0c20061a20 msgr2=0x7f0c20061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.545+0000 7f0c270dd640 1 -- 172.21.15.19:0/685829417 shutdown_connections 2024-04-25T09:58:52.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.545+0000 7f0c270dd640 1 -- 172.21.15.19:0/685829417 wait complete. 2024-04-25T09:58:52.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.546+0000 7f0c270dd640 1 Processor -- start 2024-04-25T09:58:52.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.546+0000 7f0c270dd640 1 -- start start 2024-04-25T09:58:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.547+0000 7f0c270dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c200b2cb0 0x7f0c2015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.547+0000 7f0c270dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c200b92a0 0x7f0c200cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.547+0000 7f0c270dd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c2015dbe0 0x7f0c20151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.547+0000 7f0c270dd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200b6ab0 con 0x7f0c200b92a0 2024-04-25T09:58:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.547+0000 7f0c270dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200d2210 con 0x7f0c200b2cb0 2024-04-25T09:58:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.547+0000 7f0c270dd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200b6c30 con 0x7f0c2015dbe0 2024-04-25T09:58:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.547+0000 7f0c1ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c200b92a0 0x7f0c200cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.547+0000 7f0c1ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c200b92a0 0x7f0c200cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40276/0 (socket says 172.21.15.19:40276) 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.547+0000 7f0c1ffff640 1 -- 172.21.15.19:0/2725236142 learned_addr learned my addr 172.21.15.19:0/2725236142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c2567a640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c2015dbe0 0x7f0c20151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c24e79640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c200b2cb0 0x7f0c2015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c1ffff640 1 -- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c2015dbe0 msgr2=0x7f0c20151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c1ffff640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c2015dbe0 0x7f0c20151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c1ffff640 1 -- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c200b2cb0 msgr2=0x7f0c2015f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c1ffff640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c200b2cb0 0x7f0c2015f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c1ffff640 1 -- 172.21.15.19:0/2725236142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c20152420 con 0x7f0c200b92a0 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c1ffff640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c200b92a0 0x7f0c200cac10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c7f0 tx=0x7f0c1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c1d7fa640 1 -- 172.21.15.19:0/2725236142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10015070 con 0x7f0c200b92a0 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c1d7fa640 1 -- 172.21.15.19:0/2725236142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10007a30 con 0x7f0c200b92a0 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c270dd640 1 -- 172.21.15.19:0/2725236142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2015f830 con 0x7f0c200b92a0 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c1d7fa640 1 -- 172.21.15.19:0/2725236142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10007e00 con 0x7f0c200b92a0 2024-04-25T09:58:52.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.548+0000 7f0c270dd640 1 -- 172.21.15.19:0/2725236142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c200b4a50 con 0x7f0c200b92a0 2024-04-25T09:58:52.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.549+0000 7f0c270dd640 1 -- 172.21.15.19:0/2725236142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c20073990 con 0x7f0c200b92a0 2024-04-25T09:58:52.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.550+0000 7f0c1d7fa640 1 -- 172.21.15.19:0/2725236142 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c10018070 con 0x7f0c200b92a0 2024-04-25T09:58:52.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.550+0000 7f0c1d7fa640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c0006b4d0 0x7f0c0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.550+0000 7f0c1d7fa640 1 -- 172.21.15.19:0/2725236142 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f0c1008a2f0 con 0x7f0c200b92a0 2024-04-25T09:58:52.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.551+0000 7f0c24e79640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c0006b4d0 0x7f0c0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.551+0000 7f0c24e79640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c0006b4d0 0x7f0c0006d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0c18034000 tx=0x7f0c18033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.554+0000 7f0c1d7fa640 1 -- 172.21.15.19:0/2725236142 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c10058fd0 con 0x7f0c200b92a0 2024-04-25T09:58:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.772+0000 7f0c270dd640 1 -- 172.21.15.19:0/2725236142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c20160f60 con 0x7f0c200b92a0 2024-04-25T09:58:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.772+0000 7f0c1d7fa640 1 -- 172.21.15.19:0/2725236142 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0c100588d0 con 0x7f0c200b92a0 2024-04-25T09:58:52.776 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:58:52.776 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2023643575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2023643575},{"type":"v1","addr":"172.21.15.19:6835","nonce":2023643575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3683680157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3683680157},{"type":"v1","addr":"172.21.15.112:6836","nonce":3683680157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/309620056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":309620056},{"type":"v1","addr":"172.21.15.19:6833","nonce":309620056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:57:26.327989+0000","modified":"2024-04-25T09:57:33.661321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7137},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7137":{"gid":7137,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1747929190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1747929190},{"type":"v1","addr":"172.21.15.112:6837","nonce":1747929190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 52 2024-04-25T09:58:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 -- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c0006b4d0 msgr2=0x7f0c0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c0006b4d0 0x7f0c0006d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0c18034000 tx=0x7f0c18033040 comp rx=0 tx=0).stop 2024-04-25T09:58:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 -- 172.21.15.19:0/2725236142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c200b92a0 msgr2=0x7f0c200cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c200b92a0 0x7f0c200cac10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c7f0 tx=0x7f0c1000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 -- 172.21.15.19:0/2725236142 shutdown_connections 2024-04-25T09:58:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c0006b4d0 0x7f0c0006d990 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c2015dbe0 0x7f0c20151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c200b92a0 0x7f0c200cac10 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 --2- 172.21.15.19:0/2725236142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c200b2cb0 0x7f0c2015f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 -- 172.21.15.19:0/2725236142 >> 172.21.15.19:0/2725236142 conn(0x7f0c20061a20 msgr2=0x7f0c2015b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 -- 172.21.15.19:0/2725236142 shutdown_connections 2024-04-25T09:58:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.774+0000 7f0c270dd640 1 -- 172.21.15.19:0/2725236142 wait complete. 2024-04-25T09:58:52.789 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:52.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.886+0000 7f8db7e7c640 1 Processor -- start 2024-04-25T09:58:52.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.886+0000 7f8db7e7c640 1 -- start start 2024-04-25T09:58:52.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db7e7c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8db00b5c30 0x7f8db0168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db7e7c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db00ceee0 con 0x7f8db015db70 2024-04-25T09:58:52.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db7e7c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db00d1d10 con 0x7f8db00b6cb0 2024-04-25T09:58:52.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db7e7c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db00d3800 con 0x7f8db00b5c30 2024-04-25T09:58:52.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db5c18640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8db015db70 0x7f8db0162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52834/0 (socket says 172.21.15.19:52834) 2024-04-25T09:58:52.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db5c18640 1 -- 172.21.15.19:0/2055697078 learned_addr learned my addr 172.21.15.19:0/2055697078 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db6419640 1 --2- 172.21.15.19:0/2055697078 >> v2:172.21.15.112:3300/0 conn(0x7f8db00b5c30 0x7f8db0168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150037028 0 0) 0x7f8db00ceee0 con 0x7f8db015db70 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d9c003520 con 0x7f8db015db70 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3388812880 0 0) 0x7f8d9c003520 con 0x7f8db015db70 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 >> v2:172.21.15.112:3300/0 conn(0x7f8db00b5c30 msgr2=0x7f8db0168ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 --2- 172.21.15.19:0/2055697078 >> v2:172.21.15.112:3300/0 conn(0x7f8db00b5c30 0x7f8db0168ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 >> v1:172.21.15.112:6789/0 conn(0x7f8db00b6cb0 legacy=0x7f8db00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db00d1ff0 con 0x7f8db015db70 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8dac001670 con 0x7f8db015db70 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 >> v1:172.21.15.19:6789/0 conn(0x7f8db015db70 legacy=0x7f8db0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 --2- 172.21.15.19:0/2055697078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0039b0 0x7f8d9c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 --2- 172.21.15.19:0/2055697078 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c004800 0x7f8d9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 --2- 172.21.15.19:0/2055697078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d9c0051c0 0x7f8d9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00d3800 con 0x7f8d9c0039b0 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c00c090 con 0x7f8d9c0051c0 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.887+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c00c290 con 0x7f8d9c004800 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.888+0000 7f8db5c18640 1 --2- 172.21.15.19:0/2055697078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0039b0 0x7f8d9c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.888+0000 7f8db6419640 1 --2- 172.21.15.19:0/2055697078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d9c0051c0 0x7f8d9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.888+0000 7f8db5417640 1 --2- 172.21.15.19:0/2055697078 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c004800 0x7f8d9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.888+0000 7f8db5c18640 1 -- 172.21.15.19:0/2055697078 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c004800 msgr2=0x7f8d9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.888+0000 7f8db5c18640 1 --2- 172.21.15.19:0/2055697078 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c004800 0x7f8d9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.888+0000 7f8db5c18640 1 -- 172.21.15.19:0/2055697078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d9c0051c0 msgr2=0x7f8d9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.888+0000 7f8db5c18640 1 --2- 172.21.15.19:0/2055697078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d9c0051c0 0x7f8d9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.888+0000 7f8db6419640 1 -- 172.21.15.19:0/2055697078 reap_dead start 2024-04-25T09:58:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.888+0000 7f8db5c18640 1 -- 172.21.15.19:0/2055697078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c003520 con 0x7f8d9c0039b0 2024-04-25T09:58:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.889+0000 7f8db5c18640 1 --2- 172.21.15.19:0/2055697078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0039b0 0x7f8d9c003df0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00c240 tx=0x7f8d9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b049cd3abc4576 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.889+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac007cf0 con 0x7f8d9c0039b0 2024-04-25T09:58:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.889+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac005af0 con 0x7f8d9c0039b0 2024-04-25T09:58:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.889+0000 7f8db4c16640 1 -- 172.21.15.19:0/2055697078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac007070 con 0x7f8d9c0039b0 2024-04-25T09:58:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.889+0000 7f8db7e7c640 1 -- 172.21.15.19:0/2055697078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0039b0 msgr2=0x7f8d9c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.889+0000 7f8db7e7c640 1 --2- 172.21.15.19:0/2055697078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0039b0 0x7f8d9c003df0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00c240 tx=0x7f8d9c005830 comp rx=0 tx=0).stop 2024-04-25T09:58:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.889+0000 7f8db7e7c640 1 -- 172.21.15.19:0/2055697078 shutdown_connections 2024-04-25T09:58:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.889+0000 7f8db7e7c640 1 --2- 172.21.15.19:0/2055697078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d9c0039b0 0x7f8d9c003df0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.889+0000 7f8db7e7c640 1 -- 172.21.15.19:0/2055697078 >> 172.21.15.19:0/2055697078 conn(0x7f8db00d0520 msgr2=0x7f8db00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.889+0000 7f8db7e7c640 1 -- 172.21.15.19:0/2055697078 shutdown_connections 2024-04-25T09:58:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.889+0000 7f8db7e7c640 1 -- 172.21.15.19:0/2055697078 wait complete. 2024-04-25T09:58:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.891+0000 7f8db7e7c640 1 Processor -- start 2024-04-25T09:58:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.891+0000 7f8db7e7c640 1 -- start start 2024-04-25T09:58:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.891+0000 7f8db7e7c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0039b0 0x7f8db01f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.891+0000 7f8db7e7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db015cb60 0x7f8db01e7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.891+0000 7f8db7e7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db015a9d0 0x7f8db015ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.891+0000 7f8db7e7c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00ce530 con 0x7f8db015cb60 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.891+0000 7f8db7e7c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db015df50 con 0x7f8db015a9d0 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.891+0000 7f8db7e7c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00cf1c0 con 0x7f8d9c0039b0 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.891+0000 7f8db5417640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db015cb60 0x7f8db01e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.891+0000 7f8db5417640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db015cb60 0x7f8db01e7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40296/0 (socket says 172.21.15.19:40296) 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.891+0000 7f8db5417640 1 -- 172.21.15.19:0/293763037 learned_addr learned my addr 172.21.15.19:0/293763037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8db5c18640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0039b0 0x7f8db01f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8db6419640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db015a9d0 0x7f8db015ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8db5417640 1 -- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0039b0 msgr2=0x7f8db01f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8db5417640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0039b0 0x7f8db01f3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8db5417640 1 -- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db015a9d0 msgr2=0x7f8db015ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8db5417640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db015a9d0 0x7f8db015ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8db5417640 1 -- 172.21.15.19:0/293763037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db01e7c90 con 0x7f8db015cb60 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8db6419640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db015a9d0 0x7f8db015ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8db5417640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db015cb60 0x7f8db01e7320 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8da000f5c0 tx=0x7f8da000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8da67fc640 1 -- 172.21.15.19:0/293763037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da00052e0 con 0x7f8db015cb60 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8da67fc640 1 -- 172.21.15.19:0/293763037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da0018070 con 0x7f8db015cb60 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8db7e7c640 1 -- 172.21.15.19:0/293763037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db01e7f70 con 0x7f8db015cb60 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8da67fc640 1 -- 172.21.15.19:0/293763037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0003830 con 0x7f8db015cb60 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.892+0000 7f8db7e7c640 1 -- 172.21.15.19:0/293763037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db0152fb0 con 0x7f8db015cb60 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.893+0000 7f8db7e7c640 1 -- 172.21.15.19:0/293763037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db0073990 con 0x7f8db015cb60 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.894+0000 7f8da67fc640 1 -- 172.21.15.19:0/293763037 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8da00039d0 con 0x7f8db015cb60 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.894+0000 7f8da67fc640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d9406b500 0x7f8d9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.894+0000 7f8da67fc640 1 -- 172.21.15.19:0/293763037 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f8da008a120 con 0x7f8db015cb60 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.894+0000 7f8db5c18640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d9406b500 0x7f8d9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.895+0000 7f8db5c18640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d9406b500 0x7f8d9406d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00c240 tx=0x7f8dac009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:52.898+0000 7f8da67fc640 1 -- 172.21.15.19:0/293763037 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8da0058e80 con 0x7f8db015cb60 2024-04-25T09:58:53.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.078+0000 7f8db7e7c640 1 -- 172.21.15.19:0/293763037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8db00b71f0 con 0x7f8db015cb60 2024-04-25T09:58:53.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.079+0000 7f8da67fc640 1 -- 172.21.15.19:0/293763037 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f8da0058780 con 0x7f8db015cb60 2024-04-25T09:58:53.083 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:58:53.083 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:58:27.571687+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:24.581855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:25.297009+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":82,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:53.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 -- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d9406b500 msgr2=0x7f8d9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d9406b500 0x7f8d9406d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00c240 tx=0x7f8dac009030 comp rx=0 tx=0).stop 2024-04-25T09:58:53.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 -- 172.21.15.19:0/293763037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db015cb60 msgr2=0x7f8db01e7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db015cb60 0x7f8db01e7320 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8da000f5c0 tx=0x7f8da000fa80 comp rx=0 tx=0).stop 2024-04-25T09:58:53.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 -- 172.21.15.19:0/293763037 shutdown_connections 2024-04-25T09:58:53.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8d9406b500 0x7f8d9406d9c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db015a9d0 0x7f8db015ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db015cb60 0x7f8db01e7320 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 --2- 172.21.15.19:0/293763037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8d9c0039b0 0x7f8db01f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 -- 172.21.15.19:0/293763037 >> 172.21.15.19:0/293763037 conn(0x7f8db00d0520 msgr2=0x7f8db01562a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 -- 172.21.15.19:0/293763037 shutdown_connections 2024-04-25T09:58:53.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.081+0000 7f8db7e7c640 1 -- 172.21.15.19:0/293763037 wait complete. 2024-04-25T09:58:53.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:57:24.581855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:57:25.297009+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 82, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:58:53.096 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:58:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.196+0000 7f4cf7e15640 1 Processor -- start 2024-04-25T09:58:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.196+0000 7f4cf7e15640 1 -- start start 2024-04-25T09:58:53.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.197+0000 7f4cf7e15640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf015de30 0x7f4cf00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.197+0000 7f4cf7e15640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4cf00ad570 0x7f4cf00ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.197+0000 7f4cf7e15640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf00b3b70 con 0x7f4cf00b5270 2024-04-25T09:58:53.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.197+0000 7f4cf7e15640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf00b3e50 con 0x7f4cf00ad570 2024-04-25T09:58:53.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.197+0000 7f4cf7e15640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf006ec00 con 0x7f4cf015de30 2024-04-25T09:58:53.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.197+0000 7f4cf53b0640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4cf00b5270 0x7f4cf006f090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52848/0 (socket says 172.21.15.19:52848) 2024-04-25T09:58:53.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.197+0000 7f4cf53b0640 1 -- 172.21.15.19:0/905189701 learned_addr learned my addr 172.21.15.19:0/905189701 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:53.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.197+0000 7f4cf63b2640 1 --2- 172.21.15.19:0/905189701 >> v2:172.21.15.112:3300/0 conn(0x7f4cf00ad570 0x7f4cf00ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.197+0000 7f4cf5bb1640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf015de30 0x7f4cf00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.197+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452179837 0 0) 0x7f4cf00b3b70 con 0x7f4cf00b5270 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cd8003520 con 0x7f4cf00b5270 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 342850803 0 0) 0x7f4cd8003520 con 0x7f4cf00b5270 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf015de30 msgr2=0x7f4cf00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf015de30 0x7f4cf00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 >> v2:172.21.15.112:3300/0 conn(0x7f4cf00ad570 msgr2=0x7f4cf00ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 --2- 172.21.15.19:0/905189701 >> v2:172.21.15.112:3300/0 conn(0x7f4cf00ad570 0x7f4cf00ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf006e5b0 con 0x7f4cf00b5270 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f4ce4001670 con 0x7f4cf00b5270 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf5bb1640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf015de30 0x7f4cf00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf63b2640 1 --2- 172.21.15.19:0/905189701 >> v2:172.21.15.112:3300/0 conn(0x7f4cf00ad570 0x7f4cf00ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 >> v1:172.21.15.19:6789/0 conn(0x7f4cf00b5270 legacy=0x7f4cf006f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf015de30 0x7f4cd80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd8003ef0 0x7f4cd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8004d40 0x7f4cd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00b3e50 con 0x7f4cd8003ef0 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf006ec00 con 0x7f4cd8004d40 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd800b9c0 con 0x7f4cf015de30 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf53b0640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd8003ef0 0x7f4cd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf5bb1640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf015de30 0x7f4cd80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf63b2640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8004d40 0x7f4cd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf53b0640 1 -- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf015de30 msgr2=0x7f4cd80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:53.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf53b0640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf015de30 0x7f4cd80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf53b0640 1 -- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8004d40 msgr2=0x7f4cd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf53b0640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8004d40 0x7f4cd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.198+0000 7f4cf53b0640 1 -- 172.21.15.19:0/905189701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd8003520 con 0x7f4cd8003ef0 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.199+0000 7f4cf53b0640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd8003ef0 0x7f4cd8004800 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400c290 tx=0x7f4cd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b808132957792d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.199+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4007d50 con 0x7f4cd8003ef0 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.199+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce4005af0 con 0x7f4cd8003ef0 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.199+0000 7f4cf4baf640 1 -- 172.21.15.19:0/905189701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4007070 con 0x7f4cd8003ef0 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.200+0000 7f4cf7e15640 1 -- 172.21.15.19:0/905189701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd8003ef0 msgr2=0x7f4cd8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.200+0000 7f4cf7e15640 1 --2- 172.21.15.19:0/905189701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cd8003ef0 0x7f4cd8004800 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400c290 tx=0x7f4cd8005830 comp rx=0 tx=0).stop 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.200+0000 7f4cf63b2640 1 -- 172.21.15.19:0/905189701 reap_dead start 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.200+0000 7f4cf7e15640 1 -- 172.21.15.19:0/905189701 shutdown_connections 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.200+0000 7f4cf7e15640 1 -- 172.21.15.19:0/905189701 >> 172.21.15.19:0/905189701 conn(0x7f4cf00b5af0 msgr2=0x7f4cf00b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.200+0000 7f4cf7e15640 1 -- 172.21.15.19:0/905189701 shutdown_connections 2024-04-25T09:58:53.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.200+0000 7f4cf7e15640 1 -- 172.21.15.19:0/905189701 wait complete. 2024-04-25T09:58:53.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.201+0000 7f4cf7e15640 1 Processor -- start 2024-04-25T09:58:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.201+0000 7f4cf7e15640 1 -- start start 2024-04-25T09:58:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf7e15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8003ef0 0x7f4cf015ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf7e15640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf009e3f0 0x7f4cf01f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf7e15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf0160900 0x7f4cf0160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf7e15640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf006f920 con 0x7f4cf0160900 2024-04-25T09:58:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf7e15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf006f7a0 con 0x7f4cd8003ef0 2024-04-25T09:58:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf7e15640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00b56d0 con 0x7f4cf009e3f0 2024-04-25T09:58:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf63b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf0160900 0x7f4cf0160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf63b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf0160900 0x7f4cf0160d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40318/0 (socket says 172.21.15.19:40318) 2024-04-25T09:58:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf63b2640 1 -- 172.21.15.19:0/2886666662 learned_addr learned my addr 172.21.15.19:0/2886666662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf53b0640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf009e3f0 0x7f4cf01f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf5bb1640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8003ef0 0x7f4cf015ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf63b2640 1 -- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf009e3f0 msgr2=0x7f4cf01f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf63b2640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf009e3f0 0x7f4cf01f2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf63b2640 1 -- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8003ef0 msgr2=0x7f4cf015ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf63b2640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8003ef0 0x7f4cf015ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf63b2640 1 -- 172.21.15.19:0/2886666662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf015f410 con 0x7f4cf0160900 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf5bb1640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8003ef0 0x7f4cf015ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf63b2640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf0160900 0x7f4cf0160d80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4cf006ed80 tx=0x7f4ce000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cee7fc640 1 -- 172.21.15.19:0/2886666662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0003990 con 0x7f4cf0160900 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cee7fc640 1 -- 172.21.15.19:0/2886666662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce0003b30 con 0x7f4cf0160900 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf7e15640 1 -- 172.21.15.19:0/2886666662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf015f6f0 con 0x7f4cf0160900 2024-04-25T09:58:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cee7fc640 1 -- 172.21.15.19:0/2886666662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0004020 con 0x7f4cf0160900 2024-04-25T09:58:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.202+0000 7f4cf7e15640 1 -- 172.21.15.19:0/2886666662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf015fcb0 con 0x7f4cf0160900 2024-04-25T09:58:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.203+0000 7f4cf7e15640 1 -- 172.21.15.19:0/2886666662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf006cc80 con 0x7f4cf0160900 2024-04-25T09:58:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.204+0000 7f4cee7fc640 1 -- 172.21.15.19:0/2886666662 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ce00041c0 con 0x7f4cf0160900 2024-04-25T09:58:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.204+0000 7f4cee7fc640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cdc06b500 0x7f4cdc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.204+0000 7f4cee7fc640 1 -- 172.21.15.19:0/2886666662 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f4ce008b050 con 0x7f4cf0160900 2024-04-25T09:58:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.205+0000 7f4cf5bb1640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cdc06b500 0x7f4cdc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.205+0000 7f4cf5bb1640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cdc06b500 0x7f4cdc06d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80099d0 tx=0x7f4ce8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.208+0000 7f4cee7fc640 1 -- 172.21.15.19:0/2886666662 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ce0019030 con 0x7f4cf0160900 2024-04-25T09:58:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.388+0000 7f4cf7e15640 1 -- 172.21.15.19:0/2886666662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cf015f110 con 0x7f4cf0160900 2024-04-25T09:58:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.388+0000 7f4cee7fc640 1 -- 172.21.15.19:0/2886666662 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ce0010040 con 0x7f4cf0160900 2024-04-25T09:58:53.392 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:58:53.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 -- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cdc06b500 msgr2=0x7f4cdc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cdc06b500 0x7f4cdc06d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80099d0 tx=0x7f4ce8008040 comp rx=0 tx=0).stop 2024-04-25T09:58:53.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 -- 172.21.15.19:0/2886666662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf0160900 msgr2=0x7f4cf0160d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf0160900 0x7f4cf0160d80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4cf006ed80 tx=0x7f4ce000f580 comp rx=0 tx=0).stop 2024-04-25T09:58:53.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 -- 172.21.15.19:0/2886666662 shutdown_connections 2024-04-25T09:58:53.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cdc06b500 0x7f4cdc06d9c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf0160900 0x7f4cf0160d80 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4cf009e3f0 0x7f4cf01f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 --2- 172.21.15.19:0/2886666662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8003ef0 0x7f4cf015ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 -- 172.21.15.19:0/2886666662 >> 172.21.15.19:0/2886666662 conn(0x7f4cf00b5af0 msgr2=0x7f4cf00af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 -- 172.21.15.19:0/2886666662 shutdown_connections 2024-04-25T09:58:53.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.390+0000 7f4cf7e15640 1 -- 172.21.15.19:0/2886666662 wait complete. 2024-04-25T09:58:53.405 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:58:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.504+0000 7f9104b29640 1 Processor -- start 2024-04-25T09:58:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.504+0000 7f9104b29640 1 -- start start 2024-04-25T09:58:53.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f9104b29640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91000b5c40 0x7f91000ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f9104b29640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f910015b980 0x7f91000ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f9104b29640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91000cf320 con 0x7f910015b5a0 2024-04-25T09:58:53.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f9104b29640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91000ced90 con 0x7f910015b980 2024-04-25T09:58:53.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f9104b29640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91000cf040 con 0x7f91000b5c40 2024-04-25T09:58:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f90fed9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f910015b980 0x7f91000ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f90fe59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f910015b5a0 0x7f91000b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52850/0 (socket says 172.21.15.19:52850) 2024-04-25T09:58:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f90fe59c640 1 -- 172.21.15.19:0/3392746051 learned_addr learned my addr 172.21.15.19:0/3392746051 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f90f5d9b640 1 --2- 172.21.15.19:0/3392746051 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91000b5c40 0x7f91000ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f90fed9d640 1 -- 172.21.15.19:0/3392746051 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91000b5c40 msgr2=0x7f91000ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f90fed9d640 1 --2- 172.21.15.19:0/3392746051 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91000b5c40 0x7f91000ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f90fed9d640 1 -- 172.21.15.19:0/3392746051 >> v1:172.21.15.19:6789/0 conn(0x7f910015b5a0 legacy=0x7f91000b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.505+0000 7f90fed9d640 1 -- 172.21.15.19:0/3392746051 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9100161970 con 0x7f910015b980 2024-04-25T09:58:53.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f90fed9d640 1 --2- 172.21.15.19:0/3392746051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910015b980 0x7f91000ce850 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009d90 tx=0x7f90f802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0f55832132c7f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f90fdd9b640 1 -- 172.21.15.19:0/3392746051 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8007740 con 0x7f910015b980 2024-04-25T09:58:53.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f90fdd9b640 1 -- 172.21.15.19:0/3392746051 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f80078e0 con 0x7f910015b980 2024-04-25T09:58:53.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f90fdd9b640 1 -- 172.21.15.19:0/3392746051 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8007cb0 con 0x7f910015b980 2024-04-25T09:58:53.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f9104b29640 1 -- 172.21.15.19:0/3392746051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910015b980 msgr2=0x7f91000ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f9104b29640 1 --2- 172.21.15.19:0/3392746051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910015b980 0x7f91000ce850 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009d90 tx=0x7f90f802a660 comp rx=0 tx=0).stop 2024-04-25T09:58:53.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f9104b29640 1 -- 172.21.15.19:0/3392746051 shutdown_connections 2024-04-25T09:58:53.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f9104b29640 1 --2- 172.21.15.19:0/3392746051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910015b980 0x7f91000ce850 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f9104b29640 1 --2- 172.21.15.19:0/3392746051 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91000b5c40 0x7f91000ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f9104b29640 1 -- 172.21.15.19:0/3392746051 >> 172.21.15.19:0/3392746051 conn(0x7f91000d0530 msgr2=0x7f91000d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f9104b29640 1 -- 172.21.15.19:0/3392746051 shutdown_connections 2024-04-25T09:58:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.506+0000 7f9104b29640 1 -- 172.21.15.19:0/3392746051 wait complete. 2024-04-25T09:58:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.508+0000 7f9104b29640 1 Processor -- start 2024-04-25T09:58:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.508+0000 7f9104b29640 1 -- start start 2024-04-25T09:58:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f9104b29640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91000b5c40 0x7f91000cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f9104b29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910015b5a0 0x7f91000b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f9104b29640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910015b980 0x7f91000cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f9104b29640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9100162450 con 0x7f910015b980 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f9104b29640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9100161f30 con 0x7f910015b5a0 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f9104b29640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91001625d0 con 0x7f91000b5c40 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90fed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910015b980 0x7f91000cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90fed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910015b980 0x7f91000cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40328/0 (socket says 172.21.15.19:40328) 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90fed9d640 1 -- 172.21.15.19:0/1460553218 learned_addr learned my addr 172.21.15.19:0/1460553218 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90fe59c640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91000b5c40 0x7f91000cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90f5d9b640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910015b5a0 0x7f91000b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90fed9d640 1 -- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91000b5c40 msgr2=0x7f91000cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90fed9d640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91000b5c40 0x7f91000cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90fed9d640 1 -- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910015b5a0 msgr2=0x7f91000b7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90fed9d640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910015b5a0 0x7f91000b7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90fed9d640 1 -- 172.21.15.19:0/1460553218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910015f850 con 0x7f910015b980 2024-04-25T09:58:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90fed9d640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910015b980 0x7f91000cab80 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f90f802ac50 tx=0x7f90f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.509+0000 7f90f77fe640 1 -- 172.21.15.19:0/1460553218 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f803f070 con 0x7f910015b980 2024-04-25T09:58:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.510+0000 7f90f77fe640 1 -- 172.21.15.19:0/1460553218 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f8031bc0 con 0x7f910015b980 2024-04-25T09:58:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.510+0000 7f9104b29640 1 -- 172.21.15.19:0/1460553218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910015fb30 con 0x7f910015b980 2024-04-25T09:58:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.510+0000 7f90f77fe640 1 -- 172.21.15.19:0/1460553218 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8009450 con 0x7f910015b980 2024-04-25T09:58:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.510+0000 7f9104b29640 1 -- 172.21.15.19:0/1460553218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91001600f0 con 0x7f910015b980 2024-04-25T09:58:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.511+0000 7f9104b29640 1 -- 172.21.15.19:0/1460553218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9100073a00 con 0x7f910015b980 2024-04-25T09:58:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.511+0000 7f90f77fe640 1 -- 172.21.15.19:0/1460553218 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90f803b410 con 0x7f910015b980 2024-04-25T09:58:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.511+0000 7f90f77fe640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90e006b4d0 0x7f90e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.511+0000 7f90f77fe640 1 -- 172.21.15.19:0/1460553218 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f90f80aba50 con 0x7f910015b980 2024-04-25T09:58:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.512+0000 7f90fe59c640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90e006b4d0 0x7f90e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.512+0000 7f90fe59c640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90e006b4d0 0x7f90e006d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f90f0007fd0 tx=0x7f90f0007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.516+0000 7f90f77fe640 1 -- 172.21.15.19:0/1460553218 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90f807a040 con 0x7f910015b980 2024-04-25T09:58:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.695+0000 7f9104b29640 1 -- 172.21.15.19:0/1460553218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f910014c4d0 con 0x7f910015b980 2024-04-25T09:58:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.695+0000 7f90f77fe640 1 -- 172.21.15.19:0/1460553218 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f90f807e0e0 con 0x7f910015b980 2024-04-25T09:58:53.699 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.697+0000 7f9104b29640 1 -- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90e006b4d0 msgr2=0x7f90e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.698+0000 7f9104b29640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90e006b4d0 0x7f90e006d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f90f0007fd0 tx=0x7f90f0007e80 comp rx=0 tx=0).stop 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.698+0000 7f9104b29640 1 -- 172.21.15.19:0/1460553218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910015b980 msgr2=0x7f91000cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.698+0000 7f9104b29640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910015b980 0x7f91000cab80 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f90f802ac50 tx=0x7f90f8004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.698+0000 7f9104b29640 1 -- 172.21.15.19:0/1460553218 shutdown_connections 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.698+0000 7f9104b29640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90e006b4d0 0x7f90e006d990 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.698+0000 7f9104b29640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910015b980 0x7f91000cab80 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.698+0000 7f9104b29640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910015b5a0 0x7f91000b7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.698+0000 7f9104b29640 1 --2- 172.21.15.19:0/1460553218 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91000b5c40 0x7f91000cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.698+0000 7f9104b29640 1 -- 172.21.15.19:0/1460553218 >> 172.21.15.19:0/1460553218 conn(0x7f91000d0530 msgr2=0x7f91000d2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.698+0000 7f9104b29640 1 -- 172.21.15.19:0/1460553218 shutdown_connections 2024-04-25T09:58:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.698+0000 7f9104b29640 1 -- 172.21.15.19:0/1460553218 wait complete. 2024-04-25T09:58:53.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:58:53.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.812+0000 7f548dc07640 1 Processor -- start 2024-04-25T09:58:53.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.812+0000 7f548dc07640 1 -- start start 2024-04-25T09:58:53.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.813+0000 7f548dc07640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f548815dc20 0x7f54880ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.813+0000 7f548dc07640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5488161710 0x7f54880cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.813+0000 7f548dc07640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548815e000 con 0x7f54880b9330 2024-04-25T09:58:53.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.813+0000 7f548dc07640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54880d1de0 con 0x7f5488161710 2024-04-25T09:58:53.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.813+0000 7f548dc07640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5488161c60 con 0x7f548815dc20 2024-04-25T09:58:53.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.813+0000 7f5486ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5488161710 0x7f54880cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.813+0000 7f5487fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f54880b9330 0x7f548806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52864/0 (socket says 172.21.15.19:52864) 2024-04-25T09:58:53.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.813+0000 7f5487fff640 1 -- 172.21.15.19:0/1238511963 learned_addr learned my addr 172.21.15.19:0/1238511963 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:53.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.813+0000 7f54877fe640 1 --2- 172.21.15.19:0/1238511963 >> v2:172.21.15.112:3300/0 conn(0x7f548815dc20 0x7f54880ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.814+0000 7f5486ffd640 1 -- 172.21.15.19:0/1238511963 >> v2:172.21.15.112:3300/0 conn(0x7f548815dc20 msgr2=0x7f54880ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:53.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.814+0000 7f5486ffd640 1 --2- 172.21.15.19:0/1238511963 >> v2:172.21.15.112:3300/0 conn(0x7f548815dc20 0x7f54880ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.814+0000 7f5486ffd640 1 -- 172.21.15.19:0/1238511963 >> v1:172.21.15.19:6789/0 conn(0x7f54880b9330 legacy=0x7f548806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.814+0000 7f5486ffd640 1 -- 172.21.15.19:0/1238511963 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488161f40 con 0x7f5488161710 2024-04-25T09:58:53.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.814+0000 7f54867fc640 1 -- 172.21.15.19:0/1238511963 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153867416 0 0) 0x7f548815e000 con 0x7f54880b9330 2024-04-25T09:58:53.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.814+0000 7f5486ffd640 1 --2- 172.21.15.19:0/1238511963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5488161710 0x7f54880cebb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f54880d1520 tx=0x7f547c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89caa5bc879c146 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.814+0000 7f54867fc640 1 -- 172.21.15.19:0/1238511963 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c037070 con 0x7f5488161710 2024-04-25T09:58:53.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.814+0000 7f54867fc640 1 -- 172.21.15.19:0/1238511963 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c032440 con 0x7f5488161710 2024-04-25T09:58:53.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.814+0000 7f54867fc640 1 -- 172.21.15.19:0/1238511963 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c032810 con 0x7f5488161710 2024-04-25T09:58:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.815+0000 7f548dc07640 1 -- 172.21.15.19:0/1238511963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5488161710 msgr2=0x7f54880cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.815+0000 7f548dc07640 1 --2- 172.21.15.19:0/1238511963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5488161710 0x7f54880cebb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f54880d1520 tx=0x7f547c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.815+0000 7f548dc07640 1 -- 172.21.15.19:0/1238511963 shutdown_connections 2024-04-25T09:58:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.815+0000 7f548dc07640 1 --2- 172.21.15.19:0/1238511963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5488161710 0x7f54880cebb0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.815+0000 7f548dc07640 1 --2- 172.21.15.19:0/1238511963 >> v2:172.21.15.112:3300/0 conn(0x7f548815dc20 0x7f54880ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.815+0000 7f548dc07640 1 -- 172.21.15.19:0/1238511963 >> 172.21.15.19:0/1238511963 conn(0x7f54880d0420 msgr2=0x7f54880d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.815+0000 7f548dc07640 1 -- 172.21.15.19:0/1238511963 shutdown_connections 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.815+0000 7f548dc07640 1 -- 172.21.15.19:0/1238511963 wait complete. 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.816+0000 7f548dc07640 1 Processor -- start 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.816+0000 7f548dc07640 1 -- start start 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f548dc07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54880b9330 0x7f54881f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f548dc07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548815dc20 0x7f548814d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f548dc07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488161710 0x7f54880cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f548dc07640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54880b5e50 con 0x7f548815dc20 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f548dc07640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488162500 con 0x7f5488161710 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f548dc07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54880b5fd0 con 0x7f54880b9330 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f5486ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548815dc20 0x7f548814d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f5486ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548815dc20 0x7f548814d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40342/0 (socket says 172.21.15.19:40342) 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f5486ffd640 1 -- 172.21.15.19:0/716845991 learned_addr learned my addr 172.21.15.19:0/716845991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f5487fff640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488161710 0x7f54880cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f54877fe640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54880b9330 0x7f54881f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f5486ffd640 1 -- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54880b9330 msgr2=0x7f54881f3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f5486ffd640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54880b9330 0x7f54881f3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f5486ffd640 1 -- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488161710 msgr2=0x7f54880cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f5486ffd640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488161710 0x7f54880cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.817+0000 7f5486ffd640 1 -- 172.21.15.19:0/716845991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54880cbf90 con 0x7f548815dc20 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.818+0000 7f5487fff640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488161710 0x7f54880cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.818+0000 7f5486ffd640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548815dc20 0x7f548814d820 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f547c000c00 tx=0x7f547c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.818+0000 7f546ffff640 1 -- 172.21.15.19:0/716845991 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c037040 con 0x7f548815dc20 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.818+0000 7f546ffff640 1 -- 172.21.15.19:0/716845991 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c042070 con 0x7f548815dc20 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.818+0000 7f548dc07640 1 -- 172.21.15.19:0/716845991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54880cc270 con 0x7f548815dc20 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.818+0000 7f546ffff640 1 -- 172.21.15.19:0/716845991 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c030bd0 con 0x7f548815dc20 2024-04-25T09:58:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.818+0000 7f548dc07640 1 -- 172.21.15.19:0/716845991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54880aa4d0 con 0x7f548815dc20 2024-04-25T09:58:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.819+0000 7f548dc07640 1 -- 172.21.15.19:0/716845991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54880ce3a0 con 0x7f548815dc20 2024-04-25T09:58:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.819+0000 7f546ffff640 1 -- 172.21.15.19:0/716845991 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f547c047460 con 0x7f548815dc20 2024-04-25T09:58:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.820+0000 7f546ffff640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f547406b4d0 0x7f547406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.820+0000 7f546ffff640 1 -- 172.21.15.19:0/716845991 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f547c0aafb0 con 0x7f548815dc20 2024-04-25T09:58:53.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.820+0000 7f54877fe640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f547406b4d0 0x7f547406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.820+0000 7f54877fe640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f547406b4d0 0x7f547406d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5478006fd0 tx=0x7f5478008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:53.824+0000 7f546ffff640 1 -- 172.21.15.19:0/716845991 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54880ce3a0 con 0x7f548815dc20 2024-04-25T09:58:54.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.004+0000 7f548dc07640 1 -- 172.21.15.19:0/716845991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5488073a00 con 0x7f548815dc20 2024-04-25T09:58:54.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.004+0000 7f546ffff640 1 -- 172.21.15.19:0/716845991 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f547c032e20 con 0x7f548815dc20 2024-04-25T09:58:54.008 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:58:54.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 -- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f547406b4d0 msgr2=0x7f547406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f547406b4d0 0x7f547406d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5478006fd0 tx=0x7f5478008040 comp rx=0 tx=0).stop 2024-04-25T09:58:54.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 -- 172.21.15.19:0/716845991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548815dc20 msgr2=0x7f548814d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548815dc20 0x7f548814d820 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f547c000c00 tx=0x7f547c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 -- 172.21.15.19:0/716845991 shutdown_connections 2024-04-25T09:58:54.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f547406b4d0 0x7f547406d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488161710 0x7f54880cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548815dc20 0x7f548814d820 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 --2- 172.21.15.19:0/716845991 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54880b9330 0x7f54881f3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 -- 172.21.15.19:0/716845991 >> 172.21.15.19:0/716845991 conn(0x7f54880d0420 msgr2=0x7f54880d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 -- 172.21.15.19:0/716845991 shutdown_connections 2024-04-25T09:58:54.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.006+0000 7f548dc07640 1 -- 172.21.15.19:0/716845991 wait complete. 2024-04-25T09:58:54.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.117+0000 7fd3f0dc8640 1 Processor -- start 2024-04-25T09:58:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.117+0000 7fd3f0dc8640 1 -- start start 2024-04-25T09:58:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3f0dc8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd3ec15dc90 0x7fd3ec1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3f0dc8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec0cc8e0 0x7fd3ec15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3f0dc8640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ec15f390 con 0x7fd3ec0d1810 2024-04-25T09:58:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3f0dc8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ec0cce50 con 0x7fd3ec15dc90 2024-04-25T09:58:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3f0dc8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec161af0 con 0x7fd3ec0cc8e0 2024-04-25T09:58:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3ead9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd3ec0d1810 0x7fd3ec0cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52868/0 (socket says 172.21.15.19:52868) 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3ead9d640 1 -- 172.21.15.19:0/3668971941 learned_addr learned my addr 172.21.15.19:0/3668971941 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3ea59c640 1 --2- 172.21.15.19:0/3668971941 >> v2:172.21.15.112:3300/0 conn(0x7fd3ec15dc90 0x7fd3ec1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e1d9b640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec0cc8e0 0x7fd3ec15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529478994 0 0) 0x7fd3e4001490 con 0x7fd3ec0d1810 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3d8003520 con 0x7fd3ec0d1810 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1217501836 0 0) 0x7fd3d8003520 con 0x7fd3ec0d1810 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec0cc8e0 msgr2=0x7fd3ec15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec0cc8e0 0x7fd3ec15ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 >> v2:172.21.15.112:3300/0 conn(0x7fd3ec15dc90 msgr2=0x7fd3ec1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 --2- 172.21.15.19:0/3668971941 >> v2:172.21.15.112:3300/0 conn(0x7fd3ec15dc90 0x7fd3ec1625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec161dd0 con 0x7fd3ec0d1810 2024-04-25T09:58:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fd3e4001bc0 con 0x7fd3ec0d1810 2024-04-25T09:58:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3ea59c640 1 --2- 172.21.15.19:0/3668971941 >> v2:172.21.15.112:3300/0 conn(0x7fd3ec15dc90 0x7fd3ec1625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 >> v1:172.21.15.19:6789/0 conn(0x7fd3ec0d1810 legacy=0x7fd3ec0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec0cc8e0 0x7fd3d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d8003f30 0x7fd3d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 0x7fd3d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec0cce50 con 0x7fd3d8004d40 2024-04-25T09:58:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec161af0 con 0x7fd3d8003f30 2024-04-25T09:58:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.118+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d800ba10 con 0x7fd3ec0cc8e0 2024-04-25T09:58:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.119+0000 7fd3ead9d640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 0x7fd3d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.119+0000 7fd3e1d9b640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d8003f30 0x7fd3d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.119+0000 7fd3ea59c640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec0cc8e0 0x7fd3d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.119+0000 7fd3ead9d640 1 -- 172.21.15.19:0/3668971941 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec0cc8e0 msgr2=0x7fd3d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.119+0000 7fd3ead9d640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec0cc8e0 0x7fd3d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.119+0000 7fd3ead9d640 1 -- 172.21.15.19:0/3668971941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d8003f30 msgr2=0x7fd3d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.119+0000 7fd3ead9d640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d8003f30 0x7fd3d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.119+0000 7fd3ead9d640 1 -- 172.21.15.19:0/3668971941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d8003520 con 0x7fd3d8004d40 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.120+0000 7fd3ead9d640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 0x7fd3d80051c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400aaa0 tx=0x7fd3d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ff29ac2216c2a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.120+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e400a690 con 0x7fd3d8004d40 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.120+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e402f070 con 0x7fd3d8004d40 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.120+0000 7fd3e9d9b640 1 -- 172.21.15.19:0/3668971941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4034470 con 0x7fd3d8004d40 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.120+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/3668971941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 msgr2=0x7fd3d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.120+0000 7fd3f0dc8640 1 --2- 172.21.15.19:0/3668971941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 0x7fd3d80051c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400aaa0 tx=0x7fd3d8005830 comp rx=0 tx=0).stop 2024-04-25T09:58:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.120+0000 7fd3ead9d640 1 -- 172.21.15.19:0/3668971941 reap_dead start 2024-04-25T09:58:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.120+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/3668971941 shutdown_connections 2024-04-25T09:58:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.120+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/3668971941 >> 172.21.15.19:0/3668971941 conn(0x7fd3ec06f590 msgr2=0x7fd3ec06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.120+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/3668971941 shutdown_connections 2024-04-25T09:58:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.120+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/3668971941 wait complete. 2024-04-25T09:58:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.122+0000 7fd3f0dc8640 1 Processor -- start 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.122+0000 7fd3f0dc8640 1 -- start start 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.122+0000 7fd3f0dc8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 0x7fd3ec0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.122+0000 7fd3f0dc8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec151a50 0x7fd3ec0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.122+0000 7fd3f0dc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3ec0b5200 0x7fd3ec0b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.122+0000 7fd3f0dc8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec15e070 con 0x7fd3d8004d40 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.122+0000 7fd3f0dc8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec0cd130 con 0x7fd3ec0b5200 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.122+0000 7fd3f0dc8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec15e1f0 con 0x7fd3ec151a50 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3ea59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 0x7fd3ec0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3ea59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 0x7fd3ec0b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40360/0 (socket says 172.21.15.19:40360) 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3ea59c640 1 -- 172.21.15.19:0/2823091003 learned_addr learned my addr 172.21.15.19:0/2823091003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3ea59c640 1 -- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec151a50 msgr2=0x7fd3ec0b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3ea59c640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec151a50 0x7fd3ec0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3ea59c640 1 -- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3ec0b5200 msgr2=0x7fd3ec0b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3ea59c640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3ec0b5200 0x7fd3ec0b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3ea59c640 1 -- 172.21.15.19:0/2823091003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec0c7f00 con 0x7fd3d8004d40 2024-04-25T09:58:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3ea59c640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 0x7fd3ec0b6170 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc011d30 tx=0x7fd3dc00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3e37fe640 1 -- 172.21.15.19:0/2823091003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc00fca0 con 0x7fd3d8004d40 2024-04-25T09:58:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3e37fe640 1 -- 172.21.15.19:0/2823091003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc00fe60 con 0x7fd3d8004d40 2024-04-25T09:58:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/2823091003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ec0ca3d0 con 0x7fd3d8004d40 2024-04-25T09:58:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3e37fe640 1 -- 172.21.15.19:0/2823091003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc004260 con 0x7fd3d8004d40 2024-04-25T09:58:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.123+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/2823091003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ec0b66b0 con 0x7fd3d8004d40 2024-04-25T09:58:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.124+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/2823091003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ec073990 con 0x7fd3d8004d40 2024-04-25T09:58:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.125+0000 7fd3e37fe640 1 -- 172.21.15.19:0/2823091003 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3dc012040 con 0x7fd3d8004d40 2024-04-25T09:58:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.125+0000 7fd3e37fe640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.125+0000 7fd3e37fe640 1 -- 172.21.15.19:0/2823091003 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fd3dc08aa10 con 0x7fd3d8004d40 2024-04-25T09:58:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.125+0000 7fd3e1d9b640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.126+0000 7fd3e1d9b640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0b7540 tx=0x7fd3d4009320 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.130+0000 7fd3e37fe640 1 -- 172.21.15.19:0/2823091003 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3dc019030 con 0x7fd3d8004d40 2024-04-25T09:58:54.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.345+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/2823091003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3ec0cf6b0 con 0x7fd3d8004d40 2024-04-25T09:58:54.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.345+0000 7fd3e37fe640 1 -- 172.21.15.19:0/2823091003 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd3dc059640 con 0x7fd3d8004d40 2024-04-25T09:58:54.349 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:58:54.349 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2023643575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2023643575},{"type":"v1","addr":"172.21.15.19:6835","nonce":2023643575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3683680157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3683680157},{"type":"v1","addr":"172.21.15.112:6836","nonce":3683680157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/309620056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":309620056},{"type":"v1","addr":"172.21.15.19:6833","nonce":309620056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:57:26.327989+0000","modified":"2024-04-25T09:57:33.661321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7137},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7137":{"gid":7137,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1747929190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1747929190},{"type":"v1","addr":"172.21.15.112:6837","nonce":1747929190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:54.349 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 52 2024-04-25T09:58:54.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3cc06b4d0 msgr2=0x7fd3cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0b7540 tx=0x7fd3d4009320 comp rx=0 tx=0).stop 2024-04-25T09:58:54.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/2823091003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 msgr2=0x7fd3ec0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 0x7fd3ec0b6170 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc011d30 tx=0x7fd3dc00f610 comp rx=0 tx=0).stop 2024-04-25T09:58:54.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/2823091003 shutdown_connections 2024-04-25T09:58:54.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3ec0b5200 0x7fd3ec0b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3ec151a50 0x7fd3ec0b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 --2- 172.21.15.19:0/2823091003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3d8004d40 0x7fd3ec0b6170 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/2823091003 >> 172.21.15.19:0/2823091003 conn(0x7fd3ec06f590 msgr2=0x7fd3ec0b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/2823091003 shutdown_connections 2024-04-25T09:58:54.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.347+0000 7fd3f0dc8640 1 -- 172.21.15.19:0/2823091003 wait complete. 2024-04-25T09:58:54.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.458+0000 7f311fe9e640 1 Processor -- start 2024-04-25T09:58:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.458+0000 7f311fe9e640 1 -- start start 2024-04-25T09:58:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311fe9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f311815db70 0x7f3118162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311fe9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f31180b5c30 0x7f31180d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311fe9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3118162690 con 0x7f31180b6cb0 2024-04-25T09:58:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311fe9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31180ceee0 con 0x7f311815db70 2024-04-25T09:58:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311fe9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31180d1d10 con 0x7f31180b5c30 2024-04-25T09:58:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311dc3a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f311815db70 0x7f3118162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311dc3a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f311815db70 0x7f3118162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40370/0 (socket says 172.21.15.19:40370) 2024-04-25T09:58:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311dc3a640 1 -- 172.21.15.19:0/2164469902 learned_addr learned my addr 172.21.15.19:0/2164469902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311e43b640 1 --2- 172.21.15.19:0/2164469902 >> v2:172.21.15.112:3300/0 conn(0x7f31180b5c30 0x7f31180d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311dc3a640 1 -- 172.21.15.19:0/2164469902 >> v2:172.21.15.112:3300/0 conn(0x7f31180b5c30 msgr2=0x7f31180d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311dc3a640 1 --2- 172.21.15.19:0/2164469902 >> v2:172.21.15.112:3300/0 conn(0x7f31180b5c30 0x7f31180d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311dc3a640 1 -- 172.21.15.19:0/2164469902 >> v1:172.21.15.112:6789/0 conn(0x7f31180b6cb0 legacy=0x7f31180b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.459+0000 7f311dc3a640 1 -- 172.21.15.19:0/2164469902 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31180d1ff0 con 0x7f311815db70 2024-04-25T09:58:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.460+0000 7f311dc3a640 1 --2- 172.21.15.19:0/2164469902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f311815db70 0x7f3118162150 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f311808ccc0 tx=0x7f311402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2627911f46b37a65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.460+0000 7f311cc38640 1 -- 172.21.15.19:0/2164469902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114037070 con 0x7f311815db70 2024-04-25T09:58:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.460+0000 7f311cc38640 1 -- 172.21.15.19:0/2164469902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3114032440 con 0x7f311815db70 2024-04-25T09:58:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.460+0000 7f311cc38640 1 -- 172.21.15.19:0/2164469902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114032810 con 0x7f311815db70 2024-04-25T09:58:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.460+0000 7f311fe9e640 1 -- 172.21.15.19:0/2164469902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f311815db70 msgr2=0x7f3118162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.460+0000 7f311fe9e640 1 --2- 172.21.15.19:0/2164469902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f311815db70 0x7f3118162150 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f311808ccc0 tx=0x7f311402a270 comp rx=0 tx=0).stop 2024-04-25T09:58:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.460+0000 7f311fe9e640 1 -- 172.21.15.19:0/2164469902 shutdown_connections 2024-04-25T09:58:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.460+0000 7f311fe9e640 1 --2- 172.21.15.19:0/2164469902 >> v2:172.21.15.112:3300/0 conn(0x7f31180b5c30 0x7f31180d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.460+0000 7f311fe9e640 1 --2- 172.21.15.19:0/2164469902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f311815db70 0x7f3118162150 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.461+0000 7f311fe9e640 1 -- 172.21.15.19:0/2164469902 >> 172.21.15.19:0/2164469902 conn(0x7f31180d0520 msgr2=0x7f31180d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.461+0000 7f311fe9e640 1 -- 172.21.15.19:0/2164469902 shutdown_connections 2024-04-25T09:58:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.461+0000 7f311fe9e640 1 -- 172.21.15.19:0/2164469902 wait complete. 2024-04-25T09:58:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.462+0000 7f311fe9e640 1 Processor -- start 2024-04-25T09:58:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.462+0000 7f311fe9e640 1 -- start start 2024-04-25T09:58:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.462+0000 7f311fe9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31180b5c30 0x7f311815f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.462+0000 7f311fe9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31180b6cb0 0x7f31181545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.462+0000 7f311fe9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f311815db70 0x7f3118154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.462+0000 7f311fe9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31180ce530 con 0x7f31180b6cb0 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311fe9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311815df50 con 0x7f311815db70 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311fe9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31180cf1c0 con 0x7f31180b5c30 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311d439640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31180b6cb0 0x7f31181545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311d439640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31180b6cb0 0x7f31181545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40382/0 (socket says 172.21.15.19:40382) 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311d439640 1 -- 172.21.15.19:0/891948808 learned_addr learned my addr 172.21.15.19:0/891948808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311e43b640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f311815db70 0x7f3118154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311dc3a640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31180b5c30 0x7f311815f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311d439640 1 -- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31180b5c30 msgr2=0x7f311815f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311d439640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31180b5c30 0x7f311815f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311d439640 1 -- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f311815db70 msgr2=0x7f3118154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311d439640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f311815db70 0x7f3118154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311d439640 1 -- 172.21.15.19:0/891948808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31181e7a80 con 0x7f31180b6cb0 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311dc3a640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31180b5c30 0x7f311815f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311e43b640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f311815db70 0x7f3118154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311d439640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31180b6cb0 0x7f31181545d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f310800c440 tx=0x7f3108001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f310e7fc640 1 -- 172.21.15.19:0/891948808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31080052d0 con 0x7f31180b6cb0 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f310e7fc640 1 -- 172.21.15.19:0/891948808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3108005470 con 0x7f31180b6cb0 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311fe9e640 1 -- 172.21.15.19:0/891948808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31181e7d20 con 0x7f31180b6cb0 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f310e7fc640 1 -- 172.21.15.19:0/891948808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108007260 con 0x7f31180b6cb0 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.463+0000 7f311fe9e640 1 -- 172.21.15.19:0/891948808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3118155370 con 0x7f31180b6cb0 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.464+0000 7f311fe9e640 1 -- 172.21.15.19:0/891948808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31181e84d0 con 0x7f31180b6cb0 2024-04-25T09:58:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.465+0000 7f310e7fc640 1 -- 172.21.15.19:0/891948808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3108007400 con 0x7f31180b6cb0 2024-04-25T09:58:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.465+0000 7f310e7fc640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30fc06b4d0 0x7f30fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.465+0000 7f310e7fc640 1 -- 172.21.15.19:0/891948808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f31080898f0 con 0x7f31180b6cb0 2024-04-25T09:58:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.465+0000 7f311dc3a640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30fc06b4d0 0x7f30fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.466+0000 7f311dc3a640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30fc06b4d0 0x7f30fc06d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3114034000 tx=0x7f311402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.469+0000 7f310e7fc640 1 -- 172.21.15.19:0/891948808 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f31080585c0 con 0x7f31180b6cb0 2024-04-25T09:58:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.650+0000 7f311fe9e640 1 -- 172.21.15.19:0/891948808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f311806d2e0 con 0x7f31180b6cb0 2024-04-25T09:58:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.651+0000 7f310e7fc640 1 -- 172.21.15.19:0/891948808 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f3108057ec0 con 0x7f31180b6cb0 2024-04-25T09:58:54.655 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:58:54.655 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:58:27.571687+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:24.581855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:25.297009+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":82,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 -- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30fc06b4d0 msgr2=0x7f30fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30fc06b4d0 0x7f30fc06d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3114034000 tx=0x7f311402e000 comp rx=0 tx=0).stop 2024-04-25T09:58:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 -- 172.21.15.19:0/891948808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31180b6cb0 msgr2=0x7f31181545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31180b6cb0 0x7f31181545d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f310800c440 tx=0x7f3108001e70 comp rx=0 tx=0).stop 2024-04-25T09:58:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 -- 172.21.15.19:0/891948808 shutdown_connections 2024-04-25T09:58:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f30fc06b4d0 0x7f30fc06d990 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f311815db70 0x7f3118154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31180b6cb0 0x7f31181545d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 --2- 172.21.15.19:0/891948808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31180b5c30 0x7f311815f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 -- 172.21.15.19:0/891948808 >> 172.21.15.19:0/891948808 conn(0x7f31180d0520 msgr2=0x7f311806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 -- 172.21.15.19:0/891948808 shutdown_connections 2024-04-25T09:58:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.653+0000 7f311fe9e640 1 -- 172.21.15.19:0/891948808 wait complete. 2024-04-25T09:58:54.668 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:58:54.668 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:54.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.764+0000 7f9cfe65d640 1 Processor -- start 2024-04-25T09:58:54.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.764+0000 7f9cfe65d640 1 -- start start 2024-04-25T09:58:54.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.765+0000 7f9cfe65d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf815dd30 0x7f9cf80b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.765+0000 7f9cfe65d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9cf80d0ad0 0x7f9cf80b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.765+0000 7f9cfe65d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf80d1df0 con 0x7f9cf80b9770 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.765+0000 7f9cfe65d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80d20d0 con 0x7f9cf80d0ad0 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.765+0000 7f9cfe65d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80d23b0 con 0x7f9cf815dd30 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.765+0000 7f9cfcbfa640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9cf80d0ad0 0x7f9cf80b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.765+0000 7f9cf77fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9cf80b9770 0x7f9cf80d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52870/0 (socket says 172.21.15.19:52870) 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.765+0000 7f9cf77fe640 1 -- 172.21.15.19:0/817453240 learned_addr learned my addr 172.21.15.19:0/817453240 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.765+0000 7f9cf7fff640 1 --2- 172.21.15.19:0/817453240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf815dd30 0x7f9cf80b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.766+0000 7f9cfcbfa640 1 -- 172.21.15.19:0/817453240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf815dd30 msgr2=0x7f9cf80b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.766+0000 7f9cfcbfa640 1 --2- 172.21.15.19:0/817453240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf815dd30 0x7f9cf80b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.766+0000 7f9cfcbfa640 1 -- 172.21.15.19:0/817453240 >> v1:172.21.15.19:6789/0 conn(0x7f9cf80b9770 legacy=0x7f9cf80d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.766+0000 7f9cfcbfa640 1 -- 172.21.15.19:0/817453240 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80d2690 con 0x7f9cf80d0ad0 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.766+0000 7f9cf6ffd640 1 -- 172.21.15.19:0/817453240 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996017192 0 0) 0x7f9cf80d1df0 con 0x7f9cf80b9770 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.766+0000 7f9cfcbfa640 1 --2- 172.21.15.19:0/817453240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cf80d0ad0 0x7f9cf80b6a40 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009d90 tx=0x7f9cf002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a4e4f7c4d6730f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.766+0000 7f9cf6ffd640 1 -- 172.21.15.19:0/817453240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0031600 con 0x7f9cf80d0ad0 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.766+0000 7f9cf6ffd640 1 -- 172.21.15.19:0/817453240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf00317a0 con 0x7f9cf80d0ad0 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.766+0000 7f9cf6ffd640 1 -- 172.21.15.19:0/817453240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0031b70 con 0x7f9cf80d0ad0 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.767+0000 7f9cfe65d640 1 -- 172.21.15.19:0/817453240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cf80d0ad0 msgr2=0x7f9cf80b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.767+0000 7f9cfe65d640 1 --2- 172.21.15.19:0/817453240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cf80d0ad0 0x7f9cf80b6a40 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009d90 tx=0x7f9cf002a5d0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.767+0000 7f9cfe65d640 1 -- 172.21.15.19:0/817453240 shutdown_connections 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.767+0000 7f9cfe65d640 1 --2- 172.21.15.19:0/817453240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cf80d0ad0 0x7f9cf80b6a40 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.767+0000 7f9cfe65d640 1 --2- 172.21.15.19:0/817453240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf815dd30 0x7f9cf80b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.767+0000 7f9cfe65d640 1 -- 172.21.15.19:0/817453240 >> 172.21.15.19:0/817453240 conn(0x7f9cf8061c30 msgr2=0x7f9cf806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.767+0000 7f9cfe65d640 1 -- 172.21.15.19:0/817453240 shutdown_connections 2024-04-25T09:58:54.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.767+0000 7f9cfe65d640 1 -- 172.21.15.19:0/817453240 wait complete. 2024-04-25T09:58:54.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.768+0000 7f9cfe65d640 1 Processor -- start 2024-04-25T09:58:54.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.768+0000 7f9cfe65d640 1 -- start start 2024-04-25T09:58:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cfe65d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf80b9770 0x7f9cf81582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cfe65d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf80d0ad0 0x7f9cf8159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cfe65d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cf815dd30 0x7f9cf80b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cfe65d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80d1b90 con 0x7f9cf815dd30 2024-04-25T09:58:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cfe65d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80d1a10 con 0x7f9cf80d0ad0 2024-04-25T09:58:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cfe65d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80d2e90 con 0x7f9cf80b9770 2024-04-25T09:58:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cfcbfa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cf815dd30 0x7f9cf80b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cfcbfa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cf815dd30 0x7f9cf80b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40392/0 (socket says 172.21.15.19:40392) 2024-04-25T09:58:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cfcbfa640 1 -- 172.21.15.19:0/501481683 learned_addr learned my addr 172.21.15.19:0/501481683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cf77fe640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf80d0ad0 0x7f9cf8159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cf7fff640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf80b9770 0x7f9cf81582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.769+0000 7f9cfcbfa640 1 -- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf80b9770 msgr2=0x7f9cf81582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.770+0000 7f9cfcbfa640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf80b9770 0x7f9cf81582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.770+0000 7f9cfcbfa640 1 -- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf80d0ad0 msgr2=0x7f9cf8159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.770+0000 7f9cfcbfa640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf80d0ad0 0x7f9cf8159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.770+0000 7f9cfcbfa640 1 -- 172.21.15.19:0/501481683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8158d40 con 0x7f9cf815dd30 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.770+0000 7f9cfcbfa640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cf815dd30 0x7f9cf80b3150 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009eb0 tx=0x7f9cf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.770+0000 7f9cf4ff9640 1 -- 172.21.15.19:0/501481683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0040070 con 0x7f9cf815dd30 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.770+0000 7f9cf4ff9640 1 -- 172.21.15.19:0/501481683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf0030b60 con 0x7f9cf815dd30 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.770+0000 7f9cfe65d640 1 -- 172.21.15.19:0/501481683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf8159020 con 0x7f9cf815dd30 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.770+0000 7f9cf4ff9640 1 -- 172.21.15.19:0/501481683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0009450 con 0x7f9cf815dd30 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.770+0000 7f9cfe65d640 1 -- 172.21.15.19:0/501481683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf81595e0 con 0x7f9cf815dd30 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.771+0000 7f9cfe65d640 1 -- 172.21.15.19:0/501481683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf8073990 con 0x7f9cf815dd30 2024-04-25T09:58:54.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.771+0000 7f9cf4ff9640 1 -- 172.21.15.19:0/501481683 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9cf0039040 con 0x7f9cf815dd30 2024-04-25T09:58:54.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.772+0000 7f9cf4ff9640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cd806b4d0 0x7f9cd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.772+0000 7f9cf4ff9640 1 -- 172.21.15.19:0/501481683 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f9cf00aba60 con 0x7f9cf815dd30 2024-04-25T09:58:54.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.772+0000 7f9cf7fff640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cd806b4d0 0x7f9cd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.772+0000 7f9cf7fff640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cd806b4d0 0x7f9cd806d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80ce0b0 tx=0x7f9ce0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.776+0000 7f9cf4ff9640 1 -- 172.21.15.19:0/501481683 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9cf007a720 con 0x7f9cf815dd30 2024-04-25T09:58:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.998+0000 7f9cfe65d640 1 -- 172.21.15.19:0/501481683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cf8160790 con 0x7f9cf815dd30 2024-04-25T09:58:55.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:54.998+0000 7f9cf4ff9640 1 -- 172.21.15.19:0/501481683 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9cf007a020 con 0x7f9cf815dd30 2024-04-25T09:58:55.002 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:58:55.002 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2023643575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2023643575},{"type":"v1","addr":"172.21.15.19:6835","nonce":2023643575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3683680157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3683680157},{"type":"v1","addr":"172.21.15.112:6836","nonce":3683680157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/309620056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":309620056},{"type":"v1","addr":"172.21.15.19:6833","nonce":309620056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:57:26.327989+0000","modified":"2024-04-25T09:57:33.661321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7137},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7137":{"gid":7137,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1747929190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1747929190},{"type":"v1","addr":"172.21.15.112:6837","nonce":1747929190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:55.003 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 52 2024-04-25T09:58:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 -- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cd806b4d0 msgr2=0x7f9cd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cd806b4d0 0x7f9cd806d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80ce0b0 tx=0x7f9ce0008040 comp rx=0 tx=0).stop 2024-04-25T09:58:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 -- 172.21.15.19:0/501481683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cf815dd30 msgr2=0x7f9cf80b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cf815dd30 0x7f9cf80b3150 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009eb0 tx=0x7f9cf0004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 -- 172.21.15.19:0/501481683 shutdown_connections 2024-04-25T09:58:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cd806b4d0 0x7f9cd806d990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cf815dd30 0x7f9cf80b3150 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf80d0ad0 0x7f9cf8159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 --2- 172.21.15.19:0/501481683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf80b9770 0x7f9cf81582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 -- 172.21.15.19:0/501481683 >> 172.21.15.19:0/501481683 conn(0x7f9cf8061c30 msgr2=0x7f9cf815cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 -- 172.21.15.19:0/501481683 shutdown_connections 2024-04-25T09:58:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.001+0000 7f9cfe65d640 1 -- 172.21.15.19:0/501481683 wait complete. 2024-04-25T09:58:55.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:55.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.112+0000 7fe331c6f640 1 Processor -- start 2024-04-25T09:58:55.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.112+0000 7fe331c6f640 1 -- start start 2024-04-25T09:58:55.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.113+0000 7fe331c6f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe32c0b6cb0 0x7fe32c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.113+0000 7fe331c6f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c0ceee0 con 0x7fe32c15db70 2024-04-25T09:58:55.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.113+0000 7fe331c6f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c0d1d10 con 0x7fe32c0b5a60 2024-04-25T09:58:55.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.113+0000 7fe331c6f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32c0d1ff0 con 0x7fe32c0b6cb0 2024-04-25T09:58:55.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.113+0000 7fe32affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe32c0b6cb0 0x7fe32c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.113+0000 7fe32b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe32c15db70 0x7fe32c162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52882/0 (socket says 172.21.15.19:52882) 2024-04-25T09:58:55.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.113+0000 7fe32b7fe640 1 -- 172.21.15.19:0/3439577392 learned_addr learned my addr 172.21.15.19:0/3439577392 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:55.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.114+0000 7fe32affd640 1 -- 172.21.15.19:0/3439577392 >> v1:172.21.15.112:6789/0 conn(0x7fe32c0b5a60 legacy=0x7fe32c0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:55.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.114+0000 7fe32affd640 1 -- 172.21.15.19:0/3439577392 >> v1:172.21.15.19:6789/0 conn(0x7fe32c15db70 legacy=0x7fe32c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.114+0000 7fe32affd640 1 -- 172.21.15.19:0/3439577392 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c0d22d0 con 0x7fe32c0b6cb0 2024-04-25T09:58:55.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.114+0000 7fe32a7fc640 1 -- 172.21.15.19:0/3439577392 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120665953 0 0) 0x7fe32c0ceee0 con 0x7fe32c15db70 2024-04-25T09:58:55.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.114+0000 7fe32affd640 1 --2- 172.21.15.19:0/3439577392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe32c0b6cb0 0x7fe32c0b5520 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe3200097a0 tx=0x7fe32002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d2c5c6b425cfd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.115+0000 7fe32a7fc640 1 -- 172.21.15.19:0/3439577392 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320037070 con 0x7fe32c0b6cb0 2024-04-25T09:58:55.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.115+0000 7fe32a7fc640 1 -- 172.21.15.19:0/3439577392 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32002ae50 con 0x7fe32c0b6cb0 2024-04-25T09:58:55.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.115+0000 7fe32a7fc640 1 -- 172.21.15.19:0/3439577392 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320032660 con 0x7fe32c0b6cb0 2024-04-25T09:58:55.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.115+0000 7fe331c6f640 1 -- 172.21.15.19:0/3439577392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe32c0b6cb0 msgr2=0x7fe32c0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.115+0000 7fe331c6f640 1 --2- 172.21.15.19:0/3439577392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe32c0b6cb0 0x7fe32c0b5520 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe3200097a0 tx=0x7fe32002a140 comp rx=0 tx=0).stop 2024-04-25T09:58:55.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.115+0000 7fe331c6f640 1 -- 172.21.15.19:0/3439577392 shutdown_connections 2024-04-25T09:58:55.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.115+0000 7fe331c6f640 1 --2- 172.21.15.19:0/3439577392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe32c0b6cb0 0x7fe32c0b5520 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.115+0000 7fe331c6f640 1 -- 172.21.15.19:0/3439577392 >> 172.21.15.19:0/3439577392 conn(0x7fe32c0d0520 msgr2=0x7fe32c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.115+0000 7fe331c6f640 1 -- 172.21.15.19:0/3439577392 shutdown_connections 2024-04-25T09:58:55.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.115+0000 7fe331c6f640 1 -- 172.21.15.19:0/3439577392 wait complete. 2024-04-25T09:58:55.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.116+0000 7fe331c6f640 1 Processor -- start 2024-04-25T09:58:55.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.116+0000 7fe331c6f640 1 -- start start 2024-04-25T09:58:55.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.117+0000 7fe331c6f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe32c0b5a60 0x7fe32c1f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.117+0000 7fe331c6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c0b6cb0 0x7fe32c156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.117+0000 7fe331c6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe32c15db70 0x7fe32c160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.117+0000 7fe331c6f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c0cf1c0 con 0x7fe32c0b5a60 2024-04-25T09:58:55.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.117+0000 7fe331c6f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c0ce530 con 0x7fe32c0b6cb0 2024-04-25T09:58:55.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.117+0000 7fe331c6f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c0cf340 con 0x7fe32c15db70 2024-04-25T09:58:55.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.117+0000 7fe32b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe32c0b5a60 0x7fe32c1f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.117+0000 7fe32b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe32c0b5a60 0x7fe32c1f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40410/0 (socket says 172.21.15.19:40410) 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.117+0000 7fe32b7fe640 1 -- 172.21.15.19:0/4119813364 learned_addr learned my addr 172.21.15.19:0/4119813364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe32affd640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c0b6cb0 0x7fe32c156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe32bfff640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe32c15db70 0x7fe32c160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe32b7fe640 1 -- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe32c15db70 msgr2=0x7fe32c160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe32b7fe640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe32c15db70 0x7fe32c160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe32b7fe640 1 -- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c0b6cb0 msgr2=0x7fe32c156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe32b7fe640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c0b6cb0 0x7fe32c156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe32b7fe640 1 -- 172.21.15.19:0/4119813364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c1f47d0 con 0x7fe32c0b5a60 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe32affd640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c0b6cb0 0x7fe32c156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe32bfff640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe32c15db70 0x7fe32c160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe32b7fe640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe32c0b5a60 0x7fe32c1f4290 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00c460 tx=0x7fe31c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe313fff640 1 -- 172.21.15.19:0/4119813364 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c018070 con 0x7fe32c0b5a60 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe313fff640 1 -- 172.21.15.19:0/4119813364 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c007500 con 0x7fe32c0b5a60 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe331c6f640 1 -- 172.21.15.19:0/4119813364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c15e780 con 0x7fe32c0b5a60 2024-04-25T09:58:55.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe313fff640 1 -- 172.21.15.19:0/4119813364 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c004a20 con 0x7fe32c0b5a60 2024-04-25T09:58:55.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.118+0000 7fe331c6f640 1 -- 172.21.15.19:0/4119813364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c15ed40 con 0x7fe32c0b5a60 2024-04-25T09:58:55.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.119+0000 7fe331c6f640 1 -- 172.21.15.19:0/4119813364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c073990 con 0x7fe32c0b5a60 2024-04-25T09:58:55.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.119+0000 7fe313fff640 1 -- 172.21.15.19:0/4119813364 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe31c01b070 con 0x7fe32c0b5a60 2024-04-25T09:58:55.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.120+0000 7fe313fff640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe31806b4d0 0x7fe31806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.120+0000 7fe313fff640 1 -- 172.21.15.19:0/4119813364 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fe31c089610 con 0x7fe32c0b5a60 2024-04-25T09:58:55.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.120+0000 7fe32affd640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe31806b4d0 0x7fe31806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.121+0000 7fe32affd640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe31806b4d0 0x7fe31806d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe320034000 tx=0x7fe32002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.124+0000 7fe313fff640 1 -- 172.21.15.19:0/4119813364 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe32c1f47d0 con 0x7fe32c0b5a60 2024-04-25T09:58:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.340+0000 7fe331c6f640 1 -- 172.21.15.19:0/4119813364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe32c0ce180 con 0x7fe32c0b5a60 2024-04-25T09:58:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.341+0000 7fe313fff640 1 -- 172.21.15.19:0/4119813364 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe31c0582e0 con 0x7fe32c0b5a60 2024-04-25T09:58:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:58:55.345 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2023643575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2023643575},{"type":"v1","addr":"172.21.15.19:6835","nonce":2023643575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3683680157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3683680157},{"type":"v1","addr":"172.21.15.112:6836","nonce":3683680157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/309620056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":309620056},{"type":"v1","addr":"172.21.15.19:6833","nonce":309620056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:57:26.327989+0000","modified":"2024-04-25T09:57:33.661321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7137},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7137":{"gid":7137,"name":"b","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1747929190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1747929190},{"type":"v1","addr":"172.21.15.112:6837","nonce":1747929190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 52 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 -- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe31806b4d0 msgr2=0x7fe31806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe31806b4d0 0x7fe31806d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe320034000 tx=0x7fe32002e000 comp rx=0 tx=0).stop 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 -- 172.21.15.19:0/4119813364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe32c0b5a60 msgr2=0x7fe32c1f4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe32c0b5a60 0x7fe32c1f4290 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00c460 tx=0x7fe31c00c920 comp rx=0 tx=0).stop 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 -- 172.21.15.19:0/4119813364 shutdown_connections 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe31806b4d0 0x7fe31806d990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe32c15db70 0x7fe32c160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c0b6cb0 0x7fe32c156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 --2- 172.21.15.19:0/4119813364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe32c0b5a60 0x7fe32c1f4290 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 -- 172.21.15.19:0/4119813364 >> 172.21.15.19:0/4119813364 conn(0x7fe32c0d0520 msgr2=0x7fe32c15b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 -- 172.21.15.19:0/4119813364 shutdown_connections 2024-04-25T09:58:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.343+0000 7fe331c6f640 1 -- 172.21.15.19:0/4119813364 wait complete. 2024-04-25T09:58:55.357 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.454+0000 7f47e3a38640 1 Processor -- start 2024-04-25T09:58:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.454+0000 7f47e3a38640 1 -- start start 2024-04-25T09:58:55.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.455+0000 7f47e3a38640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f47dc0b96d0 0x7f47dc0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.455+0000 7f47e3a38640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc15e810 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.455+0000 7f47e3a38640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc15eac0 con 0x7f47dc15dd20 2024-04-25T09:58:55.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.455+0000 7f47e3a38640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47dc0d1c00 con 0x7f47dc0b96d0 2024-04-25T09:58:55.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.455+0000 7f47e0fd3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f47dc0b96d0 0x7f47dc0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.455+0000 7f47e1fd5640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f47dc0d0fe0 0x7f47dc0d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52894/0 (socket says 172.21.15.19:52894) 2024-04-25T09:58:55.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.455+0000 7f47e1fd5640 1 -- 172.21.15.19:0/96691350 learned_addr learned my addr 172.21.15.19:0/96691350 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:55.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.455+0000 7f47e0fd3640 1 -- 172.21.15.19:0/96691350 >> v1:172.21.15.112:6789/0 conn(0x7f47dc15dd20 legacy=0x7f47dc15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:55.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.455+0000 7f47e0fd3640 1 -- 172.21.15.19:0/96691350 >> v1:172.21.15.19:6789/0 conn(0x7f47dc0d0fe0 legacy=0x7f47dc0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.455+0000 7f47e0fd3640 1 -- 172.21.15.19:0/96691350 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc15eda0 con 0x7f47dc0b96d0 2024-04-25T09:58:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.456+0000 7f47e0fd3640 1 --2- 172.21.15.19:0/96691350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc0b96d0 0x7f47dc0d0aa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f47d00097a0 tx=0x7f47d002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddd39d5568d65b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.456+0000 7f47dbfff640 1 -- 172.21.15.19:0/96691350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0037070 con 0x7f47dc0b96d0 2024-04-25T09:58:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.456+0000 7f47dbfff640 1 -- 172.21.15.19:0/96691350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d002ae50 con 0x7f47dc0b96d0 2024-04-25T09:58:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.456+0000 7f47dbfff640 1 -- 172.21.15.19:0/96691350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0032660 con 0x7f47dc0b96d0 2024-04-25T09:58:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.456+0000 7f47e3a38640 1 -- 172.21.15.19:0/96691350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc0b96d0 msgr2=0x7f47dc0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.456+0000 7f47e3a38640 1 --2- 172.21.15.19:0/96691350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc0b96d0 0x7f47dc0d0aa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f47d00097a0 tx=0x7f47d002a140 comp rx=0 tx=0).stop 2024-04-25T09:58:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.456+0000 7f47e3a38640 1 -- 172.21.15.19:0/96691350 shutdown_connections 2024-04-25T09:58:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.456+0000 7f47e3a38640 1 --2- 172.21.15.19:0/96691350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc0b96d0 0x7f47dc0d0aa0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.456+0000 7f47e3a38640 1 -- 172.21.15.19:0/96691350 >> 172.21.15.19:0/96691350 conn(0x7f47dc061a20 msgr2=0x7f47dc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.457+0000 7f47e3a38640 1 -- 172.21.15.19:0/96691350 shutdown_connections 2024-04-25T09:58:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.457+0000 7f47e3a38640 1 -- 172.21.15.19:0/96691350 wait complete. 2024-04-25T09:58:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.458+0000 7f47e3a38640 1 Processor -- start 2024-04-25T09:58:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.458+0000 7f47e3a38640 1 -- start start 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.458+0000 7f47e3a38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47dc0b96d0 0x7f47dc0c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e3a38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc0d0fe0 0x7f47dc15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e3a38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc15dd20 0x7f47dc1600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e3a38640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0d1e50 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e3a38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0d1420 con 0x7f47dc15dd20 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e3a38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0d2a80 con 0x7f47dc0b96d0 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e0fd3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc0d0fe0 0x7f47dc15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e0fd3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc0d0fe0 0x7f47dc15fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40422/0 (socket says 172.21.15.19:40422) 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e0fd3640 1 -- 172.21.15.19:0/2066763661 learned_addr learned my addr 172.21.15.19:0/2066763661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e1fd5640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc15dd20 0x7f47dc1600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e0fd3640 1 -- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47dc0b96d0 msgr2=0x7f47dc0c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e17d4640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47dc0b96d0 0x7f47dc0c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e0fd3640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47dc0b96d0 0x7f47dc0c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e0fd3640 1 -- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc15dd20 msgr2=0x7f47dc1600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e0fd3640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc15dd20 0x7f47dc1600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e0fd3640 1 -- 172.21.15.19:0/2066763661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc0c8bd0 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e1fd5640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc15dd20 0x7f47dc1600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e17d4640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47dc0b96d0 0x7f47dc0c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e0fd3640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc0d0fe0 0x7f47dc15fb60 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f47d0000c00 tx=0x7f47d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47d9ffb640 1 -- 172.21.15.19:0/2066763661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0041070 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47d9ffb640 1 -- 172.21.15.19:0/2066763661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d003c670 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47e3a38640 1 -- 172.21.15.19:0/2066763661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc0c8ea0 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.459+0000 7f47d9ffb640 1 -- 172.21.15.19:0/2066763661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0037040 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.460+0000 7f47e3a38640 1 -- 172.21.15.19:0/2066763661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc0c9460 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.461+0000 7f47e3a38640 1 -- 172.21.15.19:0/2066763661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc0b9090 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.461+0000 7f47d9ffb640 1 -- 172.21.15.19:0/2066763661 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47d0044070 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.461+0000 7f47d9ffb640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47c806b4d0 0x7f47c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.461+0000 7f47d9ffb640 1 -- 172.21.15.19:0/2066763661 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f47d00aabf0 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.462+0000 7f47e17d4640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47c806b4d0 0x7f47c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.462+0000 7f47e17d4640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47c806b4d0 0x7f47c806d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f47d4007e90 tx=0x7f47d40073a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.465+0000 7f47d9ffb640 1 -- 172.21.15.19:0/2066763661 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47dc0b9090 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.646+0000 7f47e3a38640 1 -- 172.21.15.19:0/2066763661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47dc073990 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.647+0000 7f47d9ffb640 1 -- 172.21.15.19:0/2066763661 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f47d00799e0 con 0x7f47dc0d0fe0 2024-04-25T09:58:55.651 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:58:55.651 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:58:27.571687+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:24.581855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:25.297009+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":82,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.649+0000 7f47e3a38640 1 -- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47c806b4d0 msgr2=0x7f47c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.649+0000 7f47e3a38640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47c806b4d0 0x7f47c806d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f47d4007e90 tx=0x7f47d40073a0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.649+0000 7f47e3a38640 1 -- 172.21.15.19:0/2066763661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc0d0fe0 msgr2=0x7f47dc15fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.649+0000 7f47e3a38640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc0d0fe0 0x7f47dc15fb60 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f47d0000c00 tx=0x7f47d00046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.650+0000 7f47e3a38640 1 -- 172.21.15.19:0/2066763661 shutdown_connections 2024-04-25T09:58:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.650+0000 7f47e3a38640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47c806b4d0 0x7f47c806d990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.650+0000 7f47e3a38640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47dc15dd20 0x7f47dc1600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.650+0000 7f47e3a38640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47dc0d0fe0 0x7f47dc15fb60 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.650+0000 7f47e3a38640 1 --2- 172.21.15.19:0/2066763661 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47dc0b96d0 0x7f47dc0c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.650+0000 7f47e3a38640 1 -- 172.21.15.19:0/2066763661 >> 172.21.15.19:0/2066763661 conn(0x7f47dc061a20 msgr2=0x7f47dc061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.650+0000 7f47e3a38640 1 -- 172.21.15.19:0/2066763661 shutdown_connections 2024-04-25T09:58:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.650+0000 7f47e3a38640 1 -- 172.21.15.19:0/2066763661 wait complete. 2024-04-25T09:58:55.664 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:58:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.762+0000 7f656feac640 1 Processor -- start 2024-04-25T09:58:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.762+0000 7f656feac640 1 -- start start 2024-04-25T09:58:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656feac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65680d34c0 0x7f65680d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656feac640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f65680d08e0 0x7f65680b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656feac640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65680d2420 con 0x7f656815dac0 2024-04-25T09:58:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656feac640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65680d3030 con 0x7f65680d08e0 2024-04-25T09:58:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656feac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65680d2700 con 0x7f65680d34c0 2024-04-25T09:58:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656e449640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f65680d08e0 0x7f65680b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656e449640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f65680d08e0 0x7f65680b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40424/0 (socket says 172.21.15.19:40424) 2024-04-25T09:58:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656e449640 1 -- 172.21.15.19:0/625203704 learned_addr learned my addr 172.21.15.19:0/625203704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656e449640 1 -- 172.21.15.19:0/625203704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65680d34c0 msgr2=0x7f65680d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656d447640 1 --2- 172.21.15.19:0/625203704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65680d34c0 0x7f65680d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656e449640 1 --2- 172.21.15.19:0/625203704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65680d34c0 0x7f65680d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656e449640 1 -- 172.21.15.19:0/625203704 >> v1:172.21.15.112:6789/0 conn(0x7f656815dac0 legacy=0x7f65680d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.763+0000 7f656e449640 1 -- 172.21.15.19:0/625203704 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65680d29e0 con 0x7f65680d08e0 2024-04-25T09:58:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.764+0000 7f656e449640 1 --2- 172.21.15.19:0/625203704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65680d08e0 0x7f65680b6990 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e50 tx=0x7f656002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9ae4b625d14b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.764+0000 7f656cc46640 1 -- 172.21.15.19:0/625203704 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560036070 con 0x7f65680d08e0 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.764+0000 7f656cc46640 1 -- 172.21.15.19:0/625203704 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6560031600 con 0x7f65680d08e0 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.764+0000 7f656cc46640 1 -- 172.21.15.19:0/625203704 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65600319d0 con 0x7f65680d08e0 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.765+0000 7f656feac640 1 -- 172.21.15.19:0/625203704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65680d08e0 msgr2=0x7f65680b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.765+0000 7f656feac640 1 --2- 172.21.15.19:0/625203704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65680d08e0 0x7f65680b6990 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e50 tx=0x7f656002a750 comp rx=0 tx=0).stop 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.765+0000 7f656feac640 1 -- 172.21.15.19:0/625203704 shutdown_connections 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.765+0000 7f656feac640 1 --2- 172.21.15.19:0/625203704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65680d08e0 0x7f65680b6990 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.765+0000 7f656feac640 1 --2- 172.21.15.19:0/625203704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65680d34c0 0x7f65680d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.765+0000 7f656feac640 1 -- 172.21.15.19:0/625203704 >> 172.21.15.19:0/625203704 conn(0x7f6568061c30 msgr2=0x7f656806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.765+0000 7f656feac640 1 -- 172.21.15.19:0/625203704 shutdown_connections 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.765+0000 7f656feac640 1 -- 172.21.15.19:0/625203704 wait complete. 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.766+0000 7f656feac640 1 Processor -- start 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.766+0000 7f656feac640 1 -- start start 2024-04-25T09:58:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656feac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65680d08e0 0x7f6568151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656feac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65680d34c0 0x7f65680b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656feac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656815dac0 0x7f65680caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656feac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656815dea0 con 0x7f656815dac0 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656feac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65680d2100 con 0x7f65680d34c0 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656feac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65680b97d0 con 0x7f65680d08e0 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656e449640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656815dac0 0x7f65680caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656e449640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656815dac0 0x7f65680caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40426/0 (socket says 172.21.15.19:40426) 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656e449640 1 -- 172.21.15.19:0/1432733098 learned_addr learned my addr 172.21.15.19:0/1432733098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656d447640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65680d34c0 0x7f65680b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656dc48640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65680d08e0 0x7f6568151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656e449640 1 -- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65680d08e0 msgr2=0x7f6568151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656e449640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65680d08e0 0x7f6568151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656e449640 1 -- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65680d34c0 msgr2=0x7f65680b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656e449640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65680d34c0 0x7f65680b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656e449640 1 -- 172.21.15.19:0/1432733098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65680ccd80 con 0x7f656815dac0 2024-04-25T09:58:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f656e449640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656815dac0 0x7f65680caf70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e50 tx=0x7f6560004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.767+0000 7f655e7fc640 1 -- 172.21.15.19:0/1432733098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560036050 con 0x7f656815dac0 2024-04-25T09:58:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.768+0000 7f655e7fc640 1 -- 172.21.15.19:0/1432733098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6560042070 con 0x7f656815dac0 2024-04-25T09:58:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.768+0000 7f656feac640 1 -- 172.21.15.19:0/1432733098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65680cd060 con 0x7f656815dac0 2024-04-25T09:58:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.768+0000 7f655e7fc640 1 -- 172.21.15.19:0/1432733098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560009510 con 0x7f656815dac0 2024-04-25T09:58:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.768+0000 7f656feac640 1 -- 172.21.15.19:0/1432733098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65680cd620 con 0x7f656815dac0 2024-04-25T09:58:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.768+0000 7f656feac640 1 -- 172.21.15.19:0/1432733098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656815f060 con 0x7f656815dac0 2024-04-25T09:58:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.769+0000 7f655e7fc640 1 -- 172.21.15.19:0/1432733098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6560007bd0 con 0x7f656815dac0 2024-04-25T09:58:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.769+0000 7f655e7fc640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f654c06b4d0 0x7f654c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.769+0000 7f655e7fc640 1 -- 172.21.15.19:0/1432733098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f65600ab2b0 con 0x7f656815dac0 2024-04-25T09:58:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.770+0000 7f656dc48640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f654c06b4d0 0x7f654c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.770+0000 7f656dc48640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f654c06b4d0 0x7f654c06d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f656400a750 tx=0x7f6564008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.774+0000 7f655e7fc640 1 -- 172.21.15.19:0/1432733098 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65600799c0 con 0x7f656815dac0 2024-04-25T09:58:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.989+0000 7f656feac640 1 -- 172.21.15.19:0/1432733098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6568073a00 con 0x7f656815dac0 2024-04-25T09:58:55.993 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T09:58:55.990+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:58:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.992+0000 7f655e7fc640 1 -- 172.21.15.19:0/1432733098 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) v1 ==== 161+0+0 (secure 0 0 0) 0x7f656007da60 con 0x7f656815dac0 2024-04-25T09:58:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:58:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.994+0000 7f656feac640 1 -- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f654c06b4d0 msgr2=0x7f654c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.994+0000 7f656feac640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f654c06b4d0 0x7f654c06d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f656400a750 tx=0x7f6564008040 comp rx=0 tx=0).stop 2024-04-25T09:58:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.994+0000 7f656feac640 1 -- 172.21.15.19:0/1432733098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656815dac0 msgr2=0x7f65680caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.994+0000 7f656feac640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656815dac0 0x7f65680caf70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e50 tx=0x7f6560004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.994+0000 7f656feac640 1 -- 172.21.15.19:0/1432733098 shutdown_connections 2024-04-25T09:58:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.994+0000 7f656feac640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f654c06b4d0 0x7f654c06d990 secure :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f656400a750 tx=0x7f6564008040 comp rx=0 tx=0).stop 2024-04-25T09:58:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.994+0000 7f656feac640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656815dac0 0x7f65680caf70 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.994+0000 7f656feac640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65680d34c0 0x7f65680b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.994+0000 7f656feac640 1 --2- 172.21.15.19:0/1432733098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65680d08e0 0x7f6568151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.994+0000 7f656feac640 1 -- 172.21.15.19:0/1432733098 >> 172.21.15.19:0/1432733098 conn(0x7f6568061c30 msgr2=0x7f65680b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.995+0000 7f656feac640 1 -- 172.21.15.19:0/1432733098 shutdown_connections 2024-04-25T09:58:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:55.995+0000 7f656feac640 1 -- 172.21.15.19:0/1432733098 wait complete. 2024-04-25T09:58:56.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:58:56.037 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T09:58:56.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.108+0000 7f6fbea5d640 1 Processor -- start 2024-04-25T09:58:56.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.108+0000 7f6fbea5d640 1 -- start start 2024-04-25T09:58:56.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.109+0000 7f6fbea5d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fb815b7d0 0x7f6fb80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:56.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.109+0000 7f6fbea5d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6fb80b5620 0x7f6fb80b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:56.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.109+0000 7f6fbea5d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb806cf50 con 0x7f6fb806c2d0 2024-04-25T09:58:56.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.109+0000 7f6fbea5d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb80b5fc0 con 0x7f6fb80b5620 2024-04-25T09:58:56.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.109+0000 7f6fbea5d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80ce530 con 0x7f6fb815b7d0 2024-04-25T09:58:56.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.110+0000 7f6fb77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6fb80b5620 0x7f6fb80b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:56.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.110+0000 7f6fb7fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fb815b7d0 0x7f6fb80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:56.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.110+0000 7f6fbcffa640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6fb806c2d0 0x7f6fb806c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:38070/0 (socket says 172.21.15.19:38070) 2024-04-25T09:58:56.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.110+0000 7f6fb77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6fb80b5620 0x7f6fb80b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:45328/0 (socket says 172.21.15.19:45328) 2024-04-25T09:58:56.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.110+0000 7f6fbcffa640 1 -- 172.21.15.19:0/3493121391 learned_addr learned my addr 172.21.15.19:0/3493121391 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:56.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.110+0000 7f6fb6ffd640 1 -- 172.21.15.19:0/3493121391 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815479016 0 0) 0x7f6fb806cf50 con 0x7f6fb806c2d0 2024-04-25T09:58:56.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.110+0000 7f6fb6ffd640 1 -- 172.21.15.19:0/3493121391 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa0003520 con 0x7f6fb806c2d0 2024-04-25T09:58:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.110+0000 7f6fb7fff640 1 -- 172.21.15.19:0/3493121391 >> v2:172.21.15.112:3300/0 conn(0x7f6fb80b5620 msgr2=0x7f6fb80b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.110+0000 7f6fb7fff640 1 --2- 172.21.15.19:0/3493121391 >> v2:172.21.15.112:3300/0 conn(0x7f6fb80b5620 0x7f6fb80b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.110+0000 7f6fb7fff640 1 -- 172.21.15.19:0/3493121391 >> v1:172.21.15.112:6789/0 conn(0x7f6fb806c2d0 legacy=0x7f6fb806c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.110+0000 7f6fb7fff640 1 -- 172.21.15.19:0/3493121391 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb80ce810 con 0x7f6fb815b7d0 2024-04-25T09:58:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.111+0000 7f6fb7fff640 1 --2- 172.21.15.19:0/3493121391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fb815b7d0 0x7f6fb80cf010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0097a0 tx=0x7f6fac02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae695910d30a3202 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.111+0000 7f6fb6ffd640 1 -- 172.21.15.19:0/3493121391 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac037070 con 0x7f6fb815b7d0 2024-04-25T09:58:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.111+0000 7f6fb6ffd640 1 -- 172.21.15.19:0/3493121391 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac032440 con 0x7f6fb815b7d0 2024-04-25T09:58:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.111+0000 7f6fb6ffd640 1 -- 172.21.15.19:0/3493121391 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0327f0 con 0x7f6fb815b7d0 2024-04-25T09:58:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.111+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3493121391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fb815b7d0 msgr2=0x7f6fb80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.111+0000 7f6fbea5d640 1 --2- 172.21.15.19:0/3493121391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fb815b7d0 0x7f6fb80cf010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0097a0 tx=0x7f6fac02a180 comp rx=0 tx=0).stop 2024-04-25T09:58:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.111+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3493121391 shutdown_connections 2024-04-25T09:58:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.111+0000 7f6fbea5d640 1 --2- 172.21.15.19:0/3493121391 >> v2:172.21.15.112:3300/0 conn(0x7f6fb80b5620 0x7f6fb80b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.111+0000 7f6fbea5d640 1 --2- 172.21.15.19:0/3493121391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fb815b7d0 0x7f6fb80cf010 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.112+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3493121391 >> 172.21.15.19:0/3493121391 conn(0x7f6fb80d0800 msgr2=0x7f6fb80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:56.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.112+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3493121391 shutdown_connections 2024-04-25T09:58:56.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.112+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3493121391 wait complete. 2024-04-25T09:58:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.113+0000 7f6fbea5d640 1 Processor -- start 2024-04-25T09:58:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.113+0000 7f6fbea5d640 1 -- start start 2024-04-25T09:58:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fbea5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb806c2d0 0x7f6fb81f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fbea5d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fb80b5620 0x7f6fb80cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fbea5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815b7d0 0x7f6fb8158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fbea5d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80cf550 con 0x7f6fb806c2d0 2024-04-25T09:58:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fbea5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb806d230 con 0x7f6fb815b7d0 2024-04-25T09:58:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fbea5d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80cf6d0 con 0x7f6fb80b5620 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb806c2d0 0x7f6fb81f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb806c2d0 0x7f6fb81f2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40436/0 (socket says 172.21.15.19:40436) 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb7fff640 1 -- 172.21.15.19:0/3868174460 learned_addr learned my addr 172.21.15.19:0/3868174460 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb7fff640 1 -- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fb80b5620 msgr2=0x7f6fb80cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb7fff640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fb80b5620 0x7f6fb80cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fbcffa640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815b7d0 0x7f6fb8158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb7fff640 1 -- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815b7d0 msgr2=0x7f6fb8158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb7fff640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815b7d0 0x7f6fb8158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb7fff640 1 -- 172.21.15.19:0/3868174460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb81f3380 con 0x7f6fb806c2d0 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb7fff640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb806c2d0 0x7f6fb81f2e40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6fac02a1c0 tx=0x7f6fac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb4ff9640 1 -- 172.21.15.19:0/3868174460 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac041020 con 0x7f6fb806c2d0 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3868174460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb81e6670 con 0x7f6fb806c2d0 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb4ff9640 1 -- 172.21.15.19:0/3868174460 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac03d620 con 0x7f6fb806c2d0 2024-04-25T09:58:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fb4ff9640 1 -- 172.21.15.19:0/3868174460 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac037040 con 0x7f6fb806c2d0 2024-04-25T09:58:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.114+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3868174460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb81e6c30 con 0x7f6fb806c2d0 2024-04-25T09:58:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.115+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3868174460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb80d1140 con 0x7f6fb806c2d0 2024-04-25T09:58:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.116+0000 7f6fb4ff9640 1 -- 172.21.15.19:0/3868174460 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fac044020 con 0x7f6fb806c2d0 2024-04-25T09:58:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.116+0000 7f6fb4ff9640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fa406b4d0 0x7f6fa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:56.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.117+0000 7f6fb4ff9640 1 -- 172.21.15.19:0/3868174460 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6fac0aab80 con 0x7f6fb806c2d0 2024-04-25T09:58:56.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.117+0000 7f6fb77fe640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fa406b4d0 0x7f6fa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:56.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.117+0000 7f6fb77fe640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fa406b4d0 0x7f6fa406d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8006a90 tx=0x7f6fa8006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:56.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.121+0000 7f6fb4ff9640 1 -- 172.21.15.19:0/3868174460 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fb80d1140 con 0x7f6fb806c2d0 2024-04-25T09:58:56.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.339+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3868174460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6fb8073a00 con 0x7f6fb806c2d0 2024-04-25T09:58:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.993+0000 7f6fb4ff9640 1 -- 172.21.15.19:0/3868174460 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6fac04b030 con 0x7f6fb806c2d0 2024-04-25T09:58:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.995+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fa406b4d0 msgr2=0x7f6fa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.995+0000 7f6fbea5d640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fa406b4d0 0x7f6fa406d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8006a90 tx=0x7f6fa8006a20 comp rx=0 tx=0).stop 2024-04-25T09:58:56.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.995+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3868174460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb806c2d0 msgr2=0x7f6fb81f2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:57.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.995+0000 7f6fbea5d640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb806c2d0 0x7f6fb81f2e40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6fac02a1c0 tx=0x7f6fac0046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.995+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3868174460 shutdown_connections 2024-04-25T09:58:57.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.995+0000 7f6fbea5d640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fa406b4d0 0x7f6fa406d990 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.995+0000 7f6fbea5d640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb815b7d0 0x7f6fb8158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.995+0000 7f6fbea5d640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fb80b5620 0x7f6fb80cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.995+0000 7f6fbea5d640 1 --2- 172.21.15.19:0/3868174460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fb806c2d0 0x7f6fb81f2e40 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.995+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3868174460 >> 172.21.15.19:0/3868174460 conn(0x7f6fb80d0800 msgr2=0x7f6fb815afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:57.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.996+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3868174460 shutdown_connections 2024-04-25T09:58:57.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:56.996+0000 7f6fbea5d640 1 -- 172.21.15.19:0/3868174460 wait complete. 2024-04-25T09:58:57.013 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:58:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.120+0000 7f03295d5640 1 Processor -- start 2024-04-25T09:58:57.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.120+0000 7f03295d5640 1 -- start start 2024-04-25T09:58:57.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.121+0000 7f03295d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03240c6010 0x7f03240ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.121+0000 7f03295d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03240cd520 0x7f03240cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.121+0000 7f03295d5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03240c6940 con 0x7f032415a9f0 2024-04-25T09:58:57.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.121+0000 7f03295d5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03240b0020 con 0x7f03240cd520 2024-04-25T09:58:57.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.121+0000 7f03295d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03240cb5a0 con 0x7f03240c6010 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.121+0000 7f0322ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f032415a9f0 0x7f03240cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52902/0 (socket says 172.21.15.19:52902) 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.121+0000 7f0322ffd640 1 -- 172.21.15.19:0/2649909387 learned_addr learned my addr 172.21.15.19:0/2649909387 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.121+0000 7f03237fe640 1 --2- 172.21.15.19:0/2649909387 >> v2:172.21.15.19:3300/0 conn(0x7f03240cd520 0x7f03240cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.122+0000 7f03227fc640 1 --2- 172.21.15.19:0/2649909387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03240c6010 0x7f03240ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.122+0000 7f0321ffb640 1 -- 172.21.15.19:0/2649909387 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3235241320 0 0) 0x7f031c001110 con 0x7f032415a9f0 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.122+0000 7f0321ffb640 1 -- 172.21.15.19:0/2649909387 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0310003520 con 0x7f032415a9f0 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.122+0000 7f03237fe640 1 -- 172.21.15.19:0/2649909387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03240c6010 msgr2=0x7f03240ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.122+0000 7f03237fe640 1 --2- 172.21.15.19:0/2649909387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03240c6010 0x7f03240ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.122+0000 7f03237fe640 1 -- 172.21.15.19:0/2649909387 >> v1:172.21.15.19:6789/0 conn(0x7f032415a9f0 legacy=0x7f03240cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.122+0000 7f03237fe640 1 -- 172.21.15.19:0/2649909387 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032415add0 con 0x7f03240cd520 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.122+0000 7f03227fc640 1 --2- 172.21.15.19:0/2649909387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03240c6010 0x7f03240ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f03237fe640 1 --2- 172.21.15.19:0/2649909387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03240cd520 0x7f03240cba30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0314009d90 tx=0x7f031402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a783f7d2f46173 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:57.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f0321ffb640 1 -- 172.21.15.19:0/2649909387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314007740 con 0x7f03240cd520 2024-04-25T09:58:57.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f0321ffb640 1 -- 172.21.15.19:0/2649909387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03140078e0 con 0x7f03240cd520 2024-04-25T09:58:57.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f0321ffb640 1 -- 172.21.15.19:0/2649909387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314007c90 con 0x7f03240cd520 2024-04-25T09:58:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f03295d5640 1 -- 172.21.15.19:0/2649909387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03240cd520 msgr2=0x7f03240cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f03295d5640 1 --2- 172.21.15.19:0/2649909387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03240cd520 0x7f03240cba30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0314009d90 tx=0x7f031402a660 comp rx=0 tx=0).stop 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f03295d5640 1 -- 172.21.15.19:0/2649909387 shutdown_connections 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f03295d5640 1 --2- 172.21.15.19:0/2649909387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03240cd520 0x7f03240cba30 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f03295d5640 1 --2- 172.21.15.19:0/2649909387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03240c6010 0x7f03240ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f03295d5640 1 -- 172.21.15.19:0/2649909387 >> 172.21.15.19:0/2649909387 conn(0x7f03240b2390 msgr2=0x7f03240b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f03295d5640 1 -- 172.21.15.19:0/2649909387 shutdown_connections 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.123+0000 7f03295d5640 1 -- 172.21.15.19:0/2649909387 wait complete. 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.125+0000 7f03295d5640 1 Processor -- start 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.125+0000 7f03295d5640 1 -- start start 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.125+0000 7f03295d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03240c6010 0x7f0324157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.125+0000 7f03295d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03240cd520 0x7f032414c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03295d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f032415a9f0 0x7f03240cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03295d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03240cb230 con 0x7f03240cd520 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03295d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03240caf50 con 0x7f03240c6010 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03295d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03240cbf70 con 0x7f032415a9f0 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03227fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03240cd520 0x7f032414c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03227fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03240cd520 0x7f032414c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40454/0 (socket says 172.21.15.19:40454) 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03227fc640 1 -- 172.21.15.19:0/3249263549 learned_addr learned my addr 172.21.15.19:0/3249263549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f0322ffd640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03240c6010 0x7f0324157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03237fe640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f032415a9f0 0x7f03240cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03227fc640 1 -- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f032415a9f0 msgr2=0x7f03240cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03227fc640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f032415a9f0 0x7f03240cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03227fc640 1 -- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03240c6010 msgr2=0x7f0324157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03227fc640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03240c6010 0x7f0324157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03227fc640 1 -- 172.21.15.19:0/3249263549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0324158400 con 0x7f03240cd520 2024-04-25T09:58:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f0322ffd640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03240c6010 0x7f0324157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03227fc640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03240cd520 0x7f032414c100 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f030c00e7e0 tx=0x7f030c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f031b7fe640 1 -- 172.21.15.19:0/3249263549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00f040 con 0x7f03240cd520 2024-04-25T09:58:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f031b7fe640 1 -- 172.21.15.19:0/3249263549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c004030 con 0x7f03240cd520 2024-04-25T09:58:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f031b7fe640 1 -- 172.21.15.19:0/3249263549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c004400 con 0x7f03240cd520 2024-04-25T09:58:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03295d5640 1 -- 172.21.15.19:0/3249263549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03240b4550 con 0x7f03240cd520 2024-04-25T09:58:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.126+0000 7f03295d5640 1 -- 172.21.15.19:0/3249263549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03240b4b10 con 0x7f03240cd520 2024-04-25T09:58:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.128+0000 7f03295d5640 1 -- 172.21.15.19:0/3249263549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0324073a00 con 0x7f03240cd520 2024-04-25T09:58:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.128+0000 7f031b7fe640 1 -- 172.21.15.19:0/3249263549 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f030c00c6d0 con 0x7f03240cd520 2024-04-25T09:58:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.128+0000 7f031b7fe640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f030406b4d0 0x7f030406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.129+0000 7f031b7fe640 1 -- 172.21.15.19:0/3249263549 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f030c08a170 con 0x7f03240cd520 2024-04-25T09:58:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.129+0000 7f0322ffd640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f030406b4d0 0x7f030406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.129+0000 7f0322ffd640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f030406b4d0 0x7f030406d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f031c007fd0 tx=0x7f031c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.133+0000 7f031b7fe640 1 -- 172.21.15.19:0/3249263549 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f030c017020 con 0x7f03240cd520 2024-04-25T09:58:57.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.314+0000 7f03295d5640 1 -- 172.21.15.19:0/3249263549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03240b0810 con 0x7f03240cd520 2024-04-25T09:58:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.994+0000 7f031b7fe640 1 -- 172.21.15.19:0/3249263549 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) v1 ==== 181+0+0 (secure 0 0 0) 0x7f030c058e10 con 0x7f03240cd520 2024-04-25T09:58:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:58:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 -- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f030406b4d0 msgr2=0x7f030406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f030406b4d0 0x7f030406d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f031c007fd0 tx=0x7f031c009040 comp rx=0 tx=0).stop 2024-04-25T09:58:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 -- 172.21.15.19:0/3249263549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03240cd520 msgr2=0x7f032414c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03240cd520 0x7f032414c100 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f030c00e7e0 tx=0x7f030c00eca0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 -- 172.21.15.19:0/3249263549 shutdown_connections 2024-04-25T09:58:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f030406b4d0 0x7f030406d990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f032415a9f0 0x7f03240cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03240cd520 0x7f032414c100 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 --2- 172.21.15.19:0/3249263549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03240c6010 0x7f0324157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 -- 172.21.15.19:0/3249263549 >> 172.21.15.19:0/3249263549 conn(0x7f03240b2390 msgr2=0x7f032406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 -- 172.21.15.19:0/3249263549 shutdown_connections 2024-04-25T09:58:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:57.998+0000 7f03295d5640 1 -- 172.21.15.19:0/3249263549 wait complete. 2024-04-25T09:58:58.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:58:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.124+0000 7fecba942640 1 Processor -- start 2024-04-25T09:58:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.124+0000 7fecba942640 1 -- start start 2024-04-25T09:58:58.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.125+0000 7fecba942640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fecb40b5620 0x7fecb40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.125+0000 7fecba942640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb406c2d0 0x7fecb406c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.125+0000 7fecba942640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb406cc80 con 0x7fecb415b7d0 2024-04-25T09:58:58.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.125+0000 7fecba942640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb406cf50 con 0x7fecb40b5620 2024-04-25T09:58:58.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.125+0000 7fecba942640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb40ce530 con 0x7fecb406c2d0 2024-04-25T09:58:58.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.125+0000 7fecb37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fecb40b5620 0x7fecb40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.125+0000 7fecb8edf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb406c2d0 0x7fecb406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.125+0000 7fecb3fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fecb415b7d0 0x7fecb40cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:36950/0 (socket says 172.21.15.19:36950) 2024-04-25T09:58:58.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.125+0000 7fecb3fff640 1 -- 172.21.15.19:0/815774464 learned_addr learned my addr 172.21.15.19:0/815774464 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:58.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.125+0000 7fecb8edf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb406c2d0 0x7fecb406c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:36030/0 (socket says 172.21.15.19:36030) 2024-04-25T09:58:58.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.126+0000 7fecb2ffd640 1 -- 172.21.15.19:0/815774464 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015933364 0 0) 0x7fecb406cc80 con 0x7fecb415b7d0 2024-04-25T09:58:58.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.126+0000 7fecb2ffd640 1 -- 172.21.15.19:0/815774464 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feca0003520 con 0x7fecb415b7d0 2024-04-25T09:58:58.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.126+0000 7fecb37fe640 1 -- 172.21.15.19:0/815774464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb406c2d0 msgr2=0x7fecb406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:58.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.126+0000 7fecb37fe640 1 --2- 172.21.15.19:0/815774464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb406c2d0 0x7fecb406c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.126+0000 7fecb37fe640 1 -- 172.21.15.19:0/815774464 >> v1:172.21.15.112:6789/0 conn(0x7fecb415b7d0 legacy=0x7fecb40cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.126+0000 7fecb37fe640 1 -- 172.21.15.19:0/815774464 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb40ce810 con 0x7fecb40b5620 2024-04-25T09:58:58.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.126+0000 7fecb8edf640 1 --2- 172.21.15.19:0/815774464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb406c2d0 0x7fecb406c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:58.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecb37fe640 1 --2- 172.21.15.19:0/815774464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecb40b5620 0x7fecb40b5ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feca40097a0 tx=0x7feca402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2386b4c83376d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecb2ffd640 1 -- 172.21.15.19:0/815774464 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4037070 con 0x7fecb40b5620 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecb2ffd640 1 -- 172.21.15.19:0/815774464 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca4032440 con 0x7fecb40b5620 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecb2ffd640 1 -- 172.21.15.19:0/815774464 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4032810 con 0x7fecb40b5620 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecba942640 1 -- 172.21.15.19:0/815774464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecb40b5620 msgr2=0x7fecb40b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecba942640 1 --2- 172.21.15.19:0/815774464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecb40b5620 0x7fecb40b5ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feca40097a0 tx=0x7feca402a180 comp rx=0 tx=0).stop 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecba942640 1 -- 172.21.15.19:0/815774464 shutdown_connections 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecba942640 1 --2- 172.21.15.19:0/815774464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb406c2d0 0x7fecb406c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecba942640 1 --2- 172.21.15.19:0/815774464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecb40b5620 0x7fecb40b5ab0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecba942640 1 -- 172.21.15.19:0/815774464 >> 172.21.15.19:0/815774464 conn(0x7fecb40d0800 msgr2=0x7fecb40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecba942640 1 -- 172.21.15.19:0/815774464 shutdown_connections 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.127+0000 7fecba942640 1 -- 172.21.15.19:0/815774464 wait complete. 2024-04-25T09:58:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.128+0000 7fecba942640 1 Processor -- start 2024-04-25T09:58:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.129+0000 7fecba942640 1 -- start start 2024-04-25T09:58:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.129+0000 7fecba942640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecb406c2d0 0x7fecb414f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.129+0000 7fecba942640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecb40b5620 0x7fecb40c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.129+0000 7fecba942640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb415b7d0 0x7fecb40c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.129+0000 7fecba942640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb415bbb0 con 0x7fecb406c2d0 2024-04-25T09:58:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.129+0000 7fecba942640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb406d230 con 0x7fecb40b5620 2024-04-25T09:58:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.129+0000 7fecba942640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb415bd30 con 0x7fecb415b7d0 2024-04-25T09:58:58.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecb406c2d0 0x7fecb414f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecb406c2d0 0x7fecb414f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49244/0 (socket says 172.21.15.19:49244) 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb3fff640 1 -- 172.21.15.19:0/2572663713 learned_addr learned my addr 172.21.15.19:0/2572663713 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb37fe640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecb40b5620 0x7fecb40c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb8edf640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb415b7d0 0x7fecb40c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb3fff640 1 -- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb415b7d0 msgr2=0x7fecb40c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb3fff640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb415b7d0 0x7fecb40c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb3fff640 1 -- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecb40b5620 msgr2=0x7fecb40c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb3fff640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecb40b5620 0x7fecb40c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb3fff640 1 -- 172.21.15.19:0/2572663713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb40c7ad0 con 0x7fecb406c2d0 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb3fff640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecb406c2d0 0x7fecb414f040 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00c3c0 tx=0x7fec9c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb0ff9640 1 -- 172.21.15.19:0/2572663713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c017070 con 0x7fecb406c2d0 2024-04-25T09:58:58.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecba942640 1 -- 172.21.15.19:0/2572663713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb40cc590 con 0x7fecb406c2d0 2024-04-25T09:58:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb0ff9640 1 -- 172.21.15.19:0/2572663713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c005ce0 con 0x7fecb406c2d0 2024-04-25T09:58:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecb0ff9640 1 -- 172.21.15.19:0/2572663713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0072f0 con 0x7fecb406c2d0 2024-04-25T09:58:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.130+0000 7fecba942640 1 -- 172.21.15.19:0/2572663713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb40ccb50 con 0x7fecb406c2d0 2024-04-25T09:58:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.131+0000 7fecba942640 1 -- 172.21.15.19:0/2572663713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb4073a00 con 0x7fecb406c2d0 2024-04-25T09:58:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.132+0000 7fecb0ff9640 1 -- 172.21.15.19:0/2572663713 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec9c01a070 con 0x7fecb406c2d0 2024-04-25T09:58:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.132+0000 7fecb0ff9640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec9406b500 0x7fec9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.132+0000 7fecb0ff9640 1 -- 172.21.15.19:0/2572663713 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fec9c089620 con 0x7fecb406c2d0 2024-04-25T09:58:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.132+0000 7fecb37fe640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec9406b500 0x7fec9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.133+0000 7fecb37fe640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec9406b500 0x7fec9406d9c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feca4034000 tx=0x7feca402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.137+0000 7fecb0ff9640 1 -- 172.21.15.19:0/2572663713 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec9c057d00 con 0x7fecb406c2d0 2024-04-25T09:58:58.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.316+0000 7fecba942640 1 -- 172.21.15.19:0/2572663713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fecb4160b20 con 0x7fecb406c2d0 2024-04-25T09:58:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:58.996+0000 7fecb0ff9640 1 -- 172.21.15.19:0/2572663713 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) v1 ==== 169+0+0 (secure 0 0 0) 0x7fec9c05bda0 con 0x7fecb406c2d0 2024-04-25T09:58:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T09:58:59.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.000+0000 7fecba942640 1 -- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec9406b500 msgr2=0x7fec9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.000+0000 7fecba942640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec9406b500 0x7fec9406d9c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feca4034000 tx=0x7feca402e000 comp rx=0 tx=0).stop 2024-04-25T09:58:59.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.001+0000 7fecba942640 1 -- 172.21.15.19:0/2572663713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecb406c2d0 msgr2=0x7fecb414f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.001+0000 7fecba942640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecb406c2d0 0x7fecb414f040 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00c3c0 tx=0x7fec9c00c880 comp rx=0 tx=0).stop 2024-04-25T09:58:59.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.001+0000 7fecba942640 1 -- 172.21.15.19:0/2572663713 shutdown_connections 2024-04-25T09:58:59.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.001+0000 7fecba942640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec9406b500 0x7fec9406d9c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.001+0000 7fecba942640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecb415b7d0 0x7fecb40c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.001+0000 7fecba942640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecb40b5620 0x7fecb40c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.001+0000 7fecba942640 1 --2- 172.21.15.19:0/2572663713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecb406c2d0 0x7fecb414f040 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.001+0000 7fecba942640 1 -- 172.21.15.19:0/2572663713 >> 172.21.15.19:0/2572663713 conn(0x7fecb40d0800 msgr2=0x7fecb40d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:59.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.001+0000 7fecba942640 1 -- 172.21.15.19:0/2572663713 shutdown_connections 2024-04-25T09:58:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.001+0000 7fecba942640 1 -- 172.21.15.19:0/2572663713 wait complete. 2024-04-25T09:58:59.022 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:58:59.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.128+0000 7f3855530640 1 Processor -- start 2024-04-25T09:58:59.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.128+0000 7f3855530640 1 -- start start 2024-04-25T09:58:59.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.128+0000 7f3855530640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f385006fbd0 0x7f38500ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.128+0000 7f3855530640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38500ce730 0x7f38500b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.128+0000 7f3855530640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38500b4d80 con 0x7f3850155f80 2024-04-25T09:58:59.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.128+0000 7f3855530640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38500b5390 con 0x7f385006fbd0 2024-04-25T09:58:59.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.129+0000 7f3855530640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38500d07f0 con 0x7f38500ce730 2024-04-25T09:58:59.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.129+0000 7f384e7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f385006fbd0 0x7f38500ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.129+0000 7f384f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38500ce730 0x7f38500b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.129+0000 7f384e7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f385006fbd0 0x7f38500ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:45982/0 (socket says 172.21.15.19:45982) 2024-04-25T09:58:59.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.129+0000 7f384e7fc640 1 -- 172.21.15.19:0/643184827 learned_addr learned my addr 172.21.15.19:0/643184827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:59.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.129+0000 7f384f7fe640 1 -- 172.21.15.19:0/643184827 >> v2:172.21.15.112:3300/0 conn(0x7f385006fbd0 msgr2=0x7f38500ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:59.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.129+0000 7f384f7fe640 1 --2- 172.21.15.19:0/643184827 >> v2:172.21.15.112:3300/0 conn(0x7f385006fbd0 0x7f38500ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.129+0000 7f384f7fe640 1 -- 172.21.15.19:0/643184827 >> v1:172.21.15.112:6789/0 conn(0x7f3850155f80 legacy=0x7f385006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.129+0000 7f384f7fe640 1 -- 172.21.15.19:0/643184827 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38500cef70 con 0x7f38500ce730 2024-04-25T09:58:59.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.129+0000 7f384e7fc640 1 --2- 172.21.15.19:0/643184827 >> v2:172.21.15.112:3300/0 conn(0x7f385006fbd0 0x7f38500ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.130+0000 7f384f7fe640 1 --2- 172.21.15.19:0/643184827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38500ce730 0x7f38500b5820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f38500b5510 tx=0x7f384802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=222ac284c1e6ae25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.130+0000 7f384dffb640 1 -- 172.21.15.19:0/643184827 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384802ae60 con 0x7f38500ce730 2024-04-25T09:58:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.130+0000 7f384dffb640 1 -- 172.21.15.19:0/643184827 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3848007740 con 0x7f38500ce730 2024-04-25T09:58:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.130+0000 7f384dffb640 1 -- 172.21.15.19:0/643184827 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848007b10 con 0x7f38500ce730 2024-04-25T09:58:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.131+0000 7f3855530640 1 -- 172.21.15.19:0/643184827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38500ce730 msgr2=0x7f38500b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.131+0000 7f3855530640 1 --2- 172.21.15.19:0/643184827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38500ce730 0x7f38500b5820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f38500b5510 tx=0x7f384802a640 comp rx=0 tx=0).stop 2024-04-25T09:58:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.131+0000 7f3855530640 1 -- 172.21.15.19:0/643184827 shutdown_connections 2024-04-25T09:58:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.131+0000 7f3855530640 1 --2- 172.21.15.19:0/643184827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38500ce730 0x7f38500b5820 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.131+0000 7f3855530640 1 --2- 172.21.15.19:0/643184827 >> v2:172.21.15.112:3300/0 conn(0x7f385006fbd0 0x7f38500ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.131+0000 7f3855530640 1 -- 172.21.15.19:0/643184827 >> 172.21.15.19:0/643184827 conn(0x7f3850061600 msgr2=0x7f38500619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.131+0000 7f3855530640 1 -- 172.21.15.19:0/643184827 shutdown_connections 2024-04-25T09:58:59.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.131+0000 7f3855530640 1 -- 172.21.15.19:0/643184827 wait complete. 2024-04-25T09:58:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.132+0000 7f3855530640 1 Processor -- start 2024-04-25T09:58:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.132+0000 7f3855530640 1 -- start start 2024-04-25T09:58:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f3855530640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f385006fbd0 0x7f385014b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f3855530640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38500ce730 0x7f385015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f3855530640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3850155f80 0x7f385015aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f3855530640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38500cf250 con 0x7f3850155f80 2024-04-25T09:58:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f3855530640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3850156360 con 0x7f385006fbd0 2024-04-25T09:58:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f3855530640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38500cf3d0 con 0x7f38500ce730 2024-04-25T09:58:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f384f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3850155f80 0x7f385015aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f384f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3850155f80 0x7f385015aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49248/0 (socket says 172.21.15.19:49248) 2024-04-25T09:58:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f384f7fe640 1 -- 172.21.15.19:0/1317314832 learned_addr learned my addr 172.21.15.19:0/1317314832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f384e7fc640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38500ce730 0x7f385015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f384effd640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f385006fbd0 0x7f385014b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f384f7fe640 1 -- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38500ce730 msgr2=0x7f385015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.133+0000 7f384f7fe640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38500ce730 0x7f385015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.134+0000 7f384f7fe640 1 -- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f385006fbd0 msgr2=0x7f385014b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.134+0000 7f384f7fe640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f385006fbd0 0x7f385014b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.134+0000 7f384f7fe640 1 -- 172.21.15.19:0/1317314832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3850151710 con 0x7f3850155f80 2024-04-25T09:58:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.134+0000 7f384f7fe640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3850155f80 0x7f385015aec0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f384802ac30 tx=0x7f3848004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.134+0000 7f38377fe640 1 -- 172.21.15.19:0/1317314832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848040070 con 0x7f3850155f80 2024-04-25T09:58:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.134+0000 7f38377fe640 1 -- 172.21.15.19:0/1317314832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38480319c0 con 0x7f3850155f80 2024-04-25T09:58:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.134+0000 7f3855530640 1 -- 172.21.15.19:0/1317314832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385014bbe0 con 0x7f3850155f80 2024-04-25T09:58:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.134+0000 7f38377fe640 1 -- 172.21.15.19:0/1317314832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848031b90 con 0x7f3850155f80 2024-04-25T09:58:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.134+0000 7f3855530640 1 -- 172.21.15.19:0/1317314832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385014c1a0 con 0x7f3850155f80 2024-04-25T09:58:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.135+0000 7f3855530640 1 -- 172.21.15.19:0/1317314832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38500b4250 con 0x7f3850155f80 2024-04-25T09:58:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.135+0000 7f38377fe640 1 -- 172.21.15.19:0/1317314832 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3848039040 con 0x7f3850155f80 2024-04-25T09:58:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.136+0000 7f38377fe640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f383c06b500 0x7f383c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.136+0000 7f38377fe640 1 -- 172.21.15.19:0/1317314832 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f38480ab670 con 0x7f3850155f80 2024-04-25T09:58:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.136+0000 7f384effd640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f383c06b500 0x7f383c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.136+0000 7f384effd640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f383c06b500 0x7f383c06d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3840000f60 tx=0x7f3840009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.140+0000 7f38377fe640 1 -- 172.21.15.19:0/1317314832 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38500b4250 con 0x7f3850155f80 2024-04-25T09:58:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.326+0000 7f3855530640 1 -- 172.21.15.19:0/1317314832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f38500b0590 con 0x7f3850155f80 2024-04-25T09:58:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.329+0000 7f38377fe640 1 -- 172.21.15.19:0/1317314832 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f384807a010 con 0x7f3850155f80 2024-04-25T09:58:59.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 -- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f383c06b500 msgr2=0x7f383c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f383c06b500 0x7f383c06d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3840000f60 tx=0x7f3840009040 comp rx=0 tx=0).stop 2024-04-25T09:58:59.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 -- 172.21.15.19:0/1317314832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3850155f80 msgr2=0x7f385015aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3850155f80 0x7f385015aec0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f384802ac30 tx=0x7f3848004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 -- 172.21.15.19:0/1317314832 shutdown_connections 2024-04-25T09:58:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f383c06b500 0x7f383c06d9c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3850155f80 0x7f385015aec0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38500ce730 0x7f385015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 --2- 172.21.15.19:0/1317314832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f385006fbd0 0x7f385014b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 -- 172.21.15.19:0/1317314832 >> 172.21.15.19:0/1317314832 conn(0x7f3850061600 msgr2=0x7f38500d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 -- 172.21.15.19:0/1317314832 shutdown_connections 2024-04-25T09:58:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.331+0000 7f3855530640 1 -- 172.21.15.19:0/1317314832 wait complete. 2024-04-25T09:58:59.346 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T09:58:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.442+0000 7f7e0915a640 1 Processor -- start 2024-04-25T09:58:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.443+0000 7f7e0915a640 1 -- start start 2024-04-25T09:58:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.443+0000 7f7e0915a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7e0415db70 0x7f7e04162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.443+0000 7f7e0915a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e04162690 con 0x7f7e040b6cb0 2024-04-25T09:58:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.443+0000 7f7e0915a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e040ceee0 con 0x7f7e040b5c30 2024-04-25T09:58:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.443+0000 7f7e0915a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e040d1d10 con 0x7f7e0415db70 2024-04-25T09:58:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7dfbfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7e040b6cb0 0x7f7e040b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41446/0 (socket says 172.21.15.19:41446) 2024-04-25T09:58:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7dfbfff640 1 -- 172.21.15.19:0/334706085 learned_addr learned my addr 172.21.15.19:0/334706085 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e02d9d640 1 --2- 172.21.15.19:0/334706085 >> v2:172.21.15.112:3300/0 conn(0x7f7e0415db70 0x7f7e04162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919023198 0 0) 0x7f7e04162690 con 0x7f7e040b6cb0 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7df0003520 con 0x7f7e040b6cb0 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 659184759 0 0) 0x7f7df0003520 con 0x7f7e040b6cb0 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 >> v2:172.21.15.112:3300/0 conn(0x7f7e0415db70 msgr2=0x7f7e04162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 --2- 172.21.15.19:0/334706085 >> v2:172.21.15.112:3300/0 conn(0x7f7e0415db70 0x7f7e04162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 >> v1:172.21.15.112:6789/0 conn(0x7f7e040b5c30 legacy=0x7f7e040d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e040d1ff0 con 0x7f7e040b6cb0 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f7dec001670 con 0x7f7e040b6cb0 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 >> v1:172.21.15.19:6789/0 conn(0x7f7e040b6cb0 legacy=0x7f7e040b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 --2- 172.21.15.19:0/334706085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7df00039b0 0x7f7df0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 --2- 172.21.15.19:0/334706085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 --2- 172.21.15.19:0/334706085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df00051c0 0x7f7df000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040d1d10 con 0x7f7df0004800 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df000c060 con 0x7f7df00051c0 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df000c260 con 0x7f7df00039b0 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7dfbfff640 1 --2- 172.21.15.19:0/334706085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e02d9d640 1 --2- 172.21.15.19:0/334706085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7df00039b0 0x7f7df0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.444+0000 7f7e0359e640 1 --2- 172.21.15.19:0/334706085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df00051c0 0x7f7df000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.445+0000 7f7dfbfff640 1 -- 172.21.15.19:0/334706085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7df00039b0 msgr2=0x7f7df0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.445+0000 7f7dfbfff640 1 --2- 172.21.15.19:0/334706085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7df00039b0 0x7f7df0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.445+0000 7f7dfbfff640 1 -- 172.21.15.19:0/334706085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df00051c0 msgr2=0x7f7df000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.445+0000 7f7dfbfff640 1 --2- 172.21.15.19:0/334706085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df00051c0 0x7f7df000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.445+0000 7f7e0359e640 1 -- 172.21.15.19:0/334706085 reap_dead start 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.445+0000 7f7dfbfff640 1 -- 172.21.15.19:0/334706085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df0003520 con 0x7f7df0004800 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.445+0000 7f7dfbfff640 1 --2- 172.21.15.19:0/334706085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00c210 tx=0x7f7df0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f777ee89e7d00d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.445+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec007d10 con 0x7f7df0004800 2024-04-25T09:58:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.446+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec0058d0 con 0x7f7df0004800 2024-04-25T09:58:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.446+0000 7f7e0259c640 1 -- 172.21.15.19:0/334706085 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec005d40 con 0x7f7df0004800 2024-04-25T09:58:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.446+0000 7f7e0915a640 1 -- 172.21.15.19:0/334706085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df0004800 msgr2=0x7f7df0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.446+0000 7f7e0915a640 1 --2- 172.21.15.19:0/334706085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00c210 tx=0x7f7df0005830 comp rx=0 tx=0).stop 2024-04-25T09:58:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.446+0000 7f7e0915a640 1 -- 172.21.15.19:0/334706085 shutdown_connections 2024-04-25T09:58:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.446+0000 7f7e0915a640 1 --2- 172.21.15.19:0/334706085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.446+0000 7f7e0915a640 1 -- 172.21.15.19:0/334706085 >> 172.21.15.19:0/334706085 conn(0x7f7e040d0520 msgr2=0x7f7e040d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.446+0000 7f7e0915a640 1 -- 172.21.15.19:0/334706085 shutdown_connections 2024-04-25T09:58:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.446+0000 7f7e0915a640 1 -- 172.21.15.19:0/334706085 wait complete. 2024-04-25T09:58:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.447+0000 7f7e0915a640 1 Processor -- start 2024-04-25T09:58:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.448+0000 7f7e0915a640 1 -- start start 2024-04-25T09:58:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.448+0000 7f7e0915a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df0004800 0x7f7e040cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.448+0000 7f7e0915a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e04149dc0 0x7f7e0414a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.448+0000 7f7e0915a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e0415fb30 0x7f7e0415ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.448+0000 7f7e0915a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040ce530 con 0x7f7e0415fb30 2024-04-25T09:58:59.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.448+0000 7f7e0915a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0415df50 con 0x7f7df0004800 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.448+0000 7f7e0915a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040cf1c0 con 0x7f7e04149dc0 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.448+0000 7f7e0359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e0415fb30 0x7f7e0415ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.448+0000 7f7e0359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e0415fb30 0x7f7e0415ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49266/0 (socket says 172.21.15.19:49266) 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.448+0000 7f7e0359e640 1 -- 172.21.15.19:0/1949541337 learned_addr learned my addr 172.21.15.19:0/1949541337 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7e02d9d640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df0004800 0x7f7e040cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7dfbfff640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e04149dc0 0x7f7e0414a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7e0359e640 1 -- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e04149dc0 msgr2=0x7f7e0414a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7e0359e640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e04149dc0 0x7f7e0414a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7e0359e640 1 -- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df0004800 msgr2=0x7f7e040cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7e0359e640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df0004800 0x7f7e040cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7e0359e640 1 -- 172.21.15.19:0/1949541337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e041607f0 con 0x7f7e0415fb30 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7e0359e640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e0415fb30 0x7f7e0415ffb0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7e040d0490 tx=0x7f7dfc00fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7dfb7fe640 1 -- 172.21.15.19:0/1949541337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc01a070 con 0x7f7e0415fb30 2024-04-25T09:58:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7dfb7fe640 1 -- 172.21.15.19:0/1949541337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc006b50 con 0x7f7e0415fb30 2024-04-25T09:58:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7dfb7fe640 1 -- 172.21.15.19:0/1949541337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc003970 con 0x7f7e0415fb30 2024-04-25T09:58:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7e0915a640 1 -- 172.21.15.19:0/1949541337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0414ade0 con 0x7f7e0415fb30 2024-04-25T09:58:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.449+0000 7f7e0915a640 1 -- 172.21.15.19:0/1949541337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0414b3a0 con 0x7f7e0415fb30 2024-04-25T09:58:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.450+0000 7f7e0915a640 1 -- 172.21.15.19:0/1949541337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0414b5a0 con 0x7f7e0415fb30 2024-04-25T09:58:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.451+0000 7f7dfb7fe640 1 -- 172.21.15.19:0/1949541337 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7dfc01d070 con 0x7f7e0415fb30 2024-04-25T09:58:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.451+0000 7f7dfb7fe640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7de406b500 0x7f7de406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.451+0000 7f7dfb7fe640 1 -- 172.21.15.19:0/1949541337 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7dfc0893c0 con 0x7f7e0415fb30 2024-04-25T09:58:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.451+0000 7f7e02d9d640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7de406b500 0x7f7de406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.452+0000 7f7e02d9d640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7de406b500 0x7f7de406d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009c40 tx=0x7f7df4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.455+0000 7f7dfb7fe640 1 -- 172.21.15.19:0/1949541337 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e0414b5a0 con 0x7f7e0415fb30 2024-04-25T09:58:59.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.702+0000 7f7e0915a640 1 -- 172.21.15.19:0/1949541337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f7e04073990 con 0x7f7e0415fb30 2024-04-25T09:59:00.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.997+0000 7f7dfb7fe640 1 -- 172.21.15.19:0/1949541337 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v291) v1 ==== 158+0+0 (secure 0 0 0) 0x7f7dfc057d60 con 0x7f7e0415fb30 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.999+0000 7f7e0915a640 1 -- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7de406b500 msgr2=0x7f7de406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:58:59.999+0000 7f7e0915a640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7de406b500 0x7f7de406d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009c40 tx=0x7f7df4009190 comp rx=0 tx=0).stop 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.000+0000 7f7e0915a640 1 -- 172.21.15.19:0/1949541337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e0415fb30 msgr2=0x7f7e0415ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.000+0000 7f7e0915a640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e0415fb30 0x7f7e0415ffb0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7e040d0490 tx=0x7f7dfc00fb40 comp rx=0 tx=0).stop 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.000+0000 7f7e0915a640 1 -- 172.21.15.19:0/1949541337 shutdown_connections 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.000+0000 7f7e0915a640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7de406b500 0x7f7de406d9c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.000+0000 7f7e0915a640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e0415fb30 0x7f7e0415ffb0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.000+0000 7f7e0915a640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e04149dc0 0x7f7e0414a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.000+0000 7f7e0915a640 1 --2- 172.21.15.19:0/1949541337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df0004800 0x7f7e040cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.000+0000 7f7e0915a640 1 -- 172.21.15.19:0/1949541337 >> 172.21.15.19:0/1949541337 conn(0x7f7e040d0520 msgr2=0x7f7e040c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.000+0000 7f7e0915a640 1 -- 172.21.15.19:0/1949541337 shutdown_connections 2024-04-25T09:59:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.000+0000 7f7e0915a640 1 -- 172.21.15.19:0/1949541337 wait complete. 2024-04-25T09:59:00.017 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T09:59:00.018 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:59:00.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:00.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.118+0000 7f9f0c702640 1 Processor -- start 2024-04-25T09:59:00.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.118+0000 7f9f0c702640 1 -- start start 2024-04-25T09:59:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.119+0000 7f9f0c702640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9f040b6cb0 0x7f9f040b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.119+0000 7f9f0c702640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f040b5a60 0x7f9f040d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.119+0000 7f9f0c702640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f040ceee0 con 0x7f9f0415db70 2024-04-25T09:59:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.119+0000 7f9f0c702640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f040d1d10 con 0x7f9f040b6cb0 2024-04-25T09:59:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.119+0000 7f9f0c702640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040d1ff0 con 0x7f9f040b5a60 2024-04-25T09:59:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.119+0000 7f9f0a49e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9f0415db70 0x7f9f04162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41456/0 (socket says 172.21.15.19:41456) 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.119+0000 7f9f0a49e640 1 -- 172.21.15.19:0/3436265261 learned_addr learned my addr 172.21.15.19:0/3436265261 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.119+0000 7f9f0ac9f640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f040b5a60 0x7f9f040d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.119+0000 7f9f09c9d640 1 --2- 172.21.15.19:0/3436265261 >> v2:172.21.15.112:3300/0 conn(0x7f9f040b6cb0 0x7f9f040b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.119+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614276888 0 0) 0x7f9f040ceee0 con 0x7f9f0415db70 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.119+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ef0003520 con 0x7f9f0415db70 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2865916870 0 0) 0x7f9ef0003520 con 0x7f9f0415db70 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f040b5a60 msgr2=0x7f9f040d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f040b5a60 0x7f9f040d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 >> v2:172.21.15.112:3300/0 conn(0x7f9f040b6cb0 msgr2=0x7f9f040b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 --2- 172.21.15.19:0/3436265261 >> v2:172.21.15.112:3300/0 conn(0x7f9f040b6cb0 0x7f9f040b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f040d22d0 con 0x7f9f0415db70 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f9f00001670 con 0x7f9f0415db70 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 >> v1:172.21.15.19:6789/0 conn(0x7f9f0415db70 legacy=0x7f9f04162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f040b5a60 0x7f9ef00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ef0003f30 0x7f9ef0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ef0004d40 0x7f9ef00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040d1d10 con 0x7f9ef0004d40 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040d1ff0 con 0x7f9ef0003f30 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef000b9c0 con 0x7f9f040b5a60 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0ac9f640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ef0004d40 0x7f9ef00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f09c9d640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ef0003f30 0x7f9ef0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0a49e640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f040b5a60 0x7f9ef00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0ac9f640 1 -- 172.21.15.19:0/3436265261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f040b5a60 msgr2=0x7f9ef00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0ac9f640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f040b5a60 0x7f9ef00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0ac9f640 1 -- 172.21.15.19:0/3436265261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ef0003f30 msgr2=0x7f9ef0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0ac9f640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ef0003f30 0x7f9ef0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.120+0000 7f9f0ac9f640 1 -- 172.21.15.19:0/3436265261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef0003520 con 0x7f9ef0004d40 2024-04-25T09:59:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.121+0000 7f9f0ac9f640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ef0004d40 0x7f9ef00051c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009fc0 tx=0x7f9ef0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4075bc25e150dc0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.121+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc009c10 con 0x7f9ef0004d40 2024-04-25T09:59:00.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.121+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc036070 con 0x7f9ef0004d40 2024-04-25T09:59:00.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.121+0000 7f9f0949c640 1 -- 172.21.15.19:0/3436265261 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc031670 con 0x7f9ef0004d40 2024-04-25T09:59:00.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.122+0000 7f9f0c702640 1 -- 172.21.15.19:0/3436265261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ef0004d40 msgr2=0x7f9ef00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.122+0000 7f9f0c702640 1 --2- 172.21.15.19:0/3436265261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ef0004d40 0x7f9ef00051c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009fc0 tx=0x7f9ef0005830 comp rx=0 tx=0).stop 2024-04-25T09:59:00.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.122+0000 7f9f0ac9f640 1 -- 172.21.15.19:0/3436265261 reap_dead start 2024-04-25T09:59:00.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.122+0000 7f9f0c702640 1 -- 172.21.15.19:0/3436265261 shutdown_connections 2024-04-25T09:59:00.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.122+0000 7f9f0c702640 1 -- 172.21.15.19:0/3436265261 >> 172.21.15.19:0/3436265261 conn(0x7f9f040d0520 msgr2=0x7f9f040d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.122+0000 7f9f0c702640 1 -- 172.21.15.19:0/3436265261 shutdown_connections 2024-04-25T09:59:00.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.122+0000 7f9f0c702640 1 -- 172.21.15.19:0/3436265261 wait complete. 2024-04-25T09:59:00.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.123+0000 7f9f0c702640 1 Processor -- start 2024-04-25T09:59:00.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.123+0000 7f9f0c702640 1 -- start start 2024-04-25T09:59:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0c702640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ef0004d40 0x7f9f040b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0c702640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f040cd770 0x7f9f040c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0c702640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f04156ab0 0x7f9f04156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0c702640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040cf1c0 con 0x7f9f04156ab0 2024-04-25T09:59:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0c702640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040ce530 con 0x7f9f040cd770 2024-04-25T09:59:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0c702640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040cf340 con 0x7f9ef0004d40 2024-04-25T09:59:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0ac9f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f04156ab0 0x7f9f04156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0ac9f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f04156ab0 0x7f9f04156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49288/0 (socket says 172.21.15.19:49288) 2024-04-25T09:59:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0ac9f640 1 -- 172.21.15.19:0/3569551929 learned_addr learned my addr 172.21.15.19:0/3569551929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f09c9d640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f040cd770 0x7f9f040c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0a49e640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ef0004d40 0x7f9f040b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0ac9f640 1 -- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ef0004d40 msgr2=0x7f9f040b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0ac9f640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ef0004d40 0x7f9f040b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0ac9f640 1 -- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f040cd770 msgr2=0x7f9f040c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0ac9f640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f040cd770 0x7f9f040c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0ac9f640 1 -- 172.21.15.19:0/3569551929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f040cbbc0 con 0x7f9f04156ab0 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0ac9f640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f04156ab0 0x7f9f04156f60 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00e230 tx=0x7f9efc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9efaffd640 1 -- 172.21.15.19:0/3569551929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc009c10 con 0x7f9f04156ab0 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9efaffd640 1 -- 172.21.15.19:0/3569551929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc031dd0 con 0x7f9f04156ab0 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0c702640 1 -- 172.21.15.19:0/3569551929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f041614b0 con 0x7f9f04156ab0 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9efaffd640 1 -- 172.21.15.19:0/3569551929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc003bd0 con 0x7f9f04156ab0 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.124+0000 7f9f0c702640 1 -- 172.21.15.19:0/3569551929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f041f7240 con 0x7f9f04156ab0 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.125+0000 7f9f0c702640 1 -- 172.21.15.19:0/3569551929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f04073990 con 0x7f9f04156ab0 2024-04-25T09:59:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.126+0000 7f9efaffd640 1 -- 172.21.15.19:0/3569551929 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9efc005d10 con 0x7f9f04156ab0 2024-04-25T09:59:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.126+0000 7f9efaffd640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ee806b4d0 0x7f9ee806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.126+0000 7f9efaffd640 1 -- 172.21.15.19:0/3569551929 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9efc0ab3c0 con 0x7f9f04156ab0 2024-04-25T09:59:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.127+0000 7f9f0a49e640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ee806b4d0 0x7f9ee806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.127+0000 7f9f0a49e640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ee806b4d0 0x7f9ee806d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9f00001050 tx=0x7f9f00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.131+0000 7f9efaffd640 1 -- 172.21.15.19:0/3569551929 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9efc049060 con 0x7f9f04156ab0 2024-04-25T09:59:00.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.313+0000 7f9f0c702640 1 -- 172.21.15.19:0/3569551929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f040d1820 con 0x7f9f04156ab0 2024-04-25T09:59:00.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.314+0000 7f9efaffd640 1 -- 172.21.15.19:0/3569551929 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13284 (secure 0 0 0) 0x7f9f040cbbc0 con 0x7f9f04156ab0 2024-04-25T09:59:00.318 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:00.318 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":86,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:58:58.995529+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6837/1747929190":"2024-04-26T09:58:55.990997+0000","172.21.15.112:6835/1747929190":"2024-04-26T09:58:55.990997+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:00.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 -- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ee806b4d0 msgr2=0x7f9ee806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ee806b4d0 0x7f9ee806d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9f00001050 tx=0x7f9f00009040 comp rx=0 tx=0).stop 2024-04-25T09:59:00.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 -- 172.21.15.19:0/3569551929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f04156ab0 msgr2=0x7f9f04156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f04156ab0 0x7f9f04156f60 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00e230 tx=0x7f9efc009750 comp rx=0 tx=0).stop 2024-04-25T09:59:00.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 -- 172.21.15.19:0/3569551929 shutdown_connections 2024-04-25T09:59:00.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ee806b4d0 0x7f9ee806d990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f04156ab0 0x7f9f04156f60 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f040cd770 0x7f9f040c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 --2- 172.21.15.19:0/3569551929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ef0004d40 0x7f9f040b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 -- 172.21.15.19:0/3569551929 >> 172.21.15.19:0/3569551929 conn(0x7f9f040d0520 msgr2=0x7f9f04158130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 -- 172.21.15.19:0/3569551929 shutdown_connections 2024-04-25T09:59:00.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.316+0000 7f9f0c702640 1 -- 172.21.15.19:0/3569551929 wait complete. 2024-04-25T09:59:00.331 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:59:00.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:59:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.431+0000 7f7100bb6640 1 Processor -- start 2024-04-25T09:59:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.431+0000 7f7100bb6640 1 -- start start 2024-04-25T09:59:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.432+0000 7f7100bb6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f70fc0d34c0 0x7f70fc0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.432+0000 7f7100bb6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f70fc0d08e0 0x7f70fc0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.432+0000 7f7100bb6640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70fc0d2420 con 0x7f70fc15dac0 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.432+0000 7f7100bb6640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70fc0d0d50 con 0x7f70fc0d34c0 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.432+0000 7f7100bb6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70fc0d3030 con 0x7f70fc0d08e0 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.432+0000 7f70f9d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f70fc0d34c0 0x7f70fc0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.432+0000 7f70fa59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f70fc15dac0 0x7f70fc0d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41460/0 (socket says 172.21.15.19:41460) 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.432+0000 7f70fa59c640 1 -- 172.21.15.19:0/128737242 learned_addr learned my addr 172.21.15.19:0/128737242 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.433+0000 7f70f9d9b640 1 -- 172.21.15.19:0/128737242 >> v2:172.21.15.112:3300/0 conn(0x7f70fc0d08e0 msgr2=0x7f70fc0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.433+0000 7f70fad9d640 1 --2- 172.21.15.19:0/128737242 >> v2:172.21.15.112:3300/0 conn(0x7f70fc0d08e0 0x7f70fc0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.433+0000 7f70f9d9b640 1 --2- 172.21.15.19:0/128737242 >> v2:172.21.15.112:3300/0 conn(0x7f70fc0d08e0 0x7f70fc0b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.433+0000 7f70f959a640 1 -- 172.21.15.19:0/128737242 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450317881 0 0) 0x7f70fc0d2420 con 0x7f70fc15dac0 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.433+0000 7f70f9d9b640 1 -- 172.21.15.19:0/128737242 >> v1:172.21.15.19:6789/0 conn(0x7f70fc15dac0 legacy=0x7f70fc0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.433+0000 7f70f9d9b640 1 -- 172.21.15.19:0/128737242 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc0d2700 con 0x7f70fc0d34c0 2024-04-25T09:59:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.433+0000 7f70fad9d640 1 --2- 172.21.15.19:0/128737242 >> v2:172.21.15.112:3300/0 conn(0x7f70fc0d08e0 0x7f70fc0b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.433+0000 7f70f9d9b640 1 --2- 172.21.15.19:0/128737242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc0d34c0 0x7f70fc0d03a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0d2fa0 tx=0x7f70f002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b06861386873de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.433+0000 7f70f959a640 1 -- 172.21.15.19:0/128737242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0037070 con 0x7f70fc0d34c0 2024-04-25T09:59:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.434+0000 7f70f959a640 1 -- 172.21.15.19:0/128737242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0032440 con 0x7f70fc0d34c0 2024-04-25T09:59:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.434+0000 7f70f959a640 1 -- 172.21.15.19:0/128737242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0032810 con 0x7f70fc0d34c0 2024-04-25T09:59:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.434+0000 7f7100bb6640 1 -- 172.21.15.19:0/128737242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc0d34c0 msgr2=0x7f70fc0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.434+0000 7f7100bb6640 1 --2- 172.21.15.19:0/128737242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc0d34c0 0x7f70fc0d03a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0d2fa0 tx=0x7f70f002a230 comp rx=0 tx=0).stop 2024-04-25T09:59:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.434+0000 7f7100bb6640 1 -- 172.21.15.19:0/128737242 shutdown_connections 2024-04-25T09:59:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.434+0000 7f7100bb6640 1 --2- 172.21.15.19:0/128737242 >> v2:172.21.15.112:3300/0 conn(0x7f70fc0d08e0 0x7f70fc0b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.434+0000 7f7100bb6640 1 --2- 172.21.15.19:0/128737242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc0d34c0 0x7f70fc0d03a0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.434+0000 7f7100bb6640 1 -- 172.21.15.19:0/128737242 >> 172.21.15.19:0/128737242 conn(0x7f70fc061c30 msgr2=0x7f70fc06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.434+0000 7f7100bb6640 1 -- 172.21.15.19:0/128737242 shutdown_connections 2024-04-25T09:59:00.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.434+0000 7f7100bb6640 1 -- 172.21.15.19:0/128737242 wait complete. 2024-04-25T09:59:00.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.435+0000 7f7100bb6640 1 Processor -- start 2024-04-25T09:59:00.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.435+0000 7f7100bb6640 1 -- start start 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.436+0000 7f7100bb6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc0d08e0 0x7f70fc152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.436+0000 7f7100bb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70fc0d34c0 0x7f70fc1567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.436+0000 7f7100bb6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc15dac0 0x7f70fc160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.436+0000 7f7100bb6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc0d2100 con 0x7f70fc0d08e0 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.436+0000 7f7100bb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc0d2cc0 con 0x7f70fc0d34c0 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.436+0000 7f7100bb6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc15dea0 con 0x7f70fc15dac0 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.436+0000 7f70fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc0d08e0 0x7f70fc152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.436+0000 7f70fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc0d08e0 0x7f70fc152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49294/0 (socket says 172.21.15.19:49294) 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.436+0000 7f70fa59c640 1 -- 172.21.15.19:0/1473770858 learned_addr learned my addr 172.21.15.19:0/1473770858 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70f9d9b640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70fc0d34c0 0x7f70fc1567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70fad9d640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc15dac0 0x7f70fc160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70fa59c640 1 -- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc15dac0 msgr2=0x7f70fc160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70fa59c640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc15dac0 0x7f70fc160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70fa59c640 1 -- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70fc0d34c0 msgr2=0x7f70fc1567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70fa59c640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70fc0d34c0 0x7f70fc1567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70fa59c640 1 -- 172.21.15.19:0/1473770858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc154680 con 0x7f70fc0d08e0 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70f9d9b640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70fc0d34c0 0x7f70fc1567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70fa59c640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc0d08e0 0x7f70fc152a10 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f70f400c460 tx=0x7f70f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70eaffd640 1 -- 172.21.15.19:0/1473770858 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f4018070 con 0x7f70fc0d08e0 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70eaffd640 1 -- 172.21.15.19:0/1473770858 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f4007500 con 0x7f70fc0d08e0 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f70eaffd640 1 -- 172.21.15.19:0/1473770858 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f4004a20 con 0x7f70fc0d08e0 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f7100bb6640 1 -- 172.21.15.19:0/1473770858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc154960 con 0x7f70fc0d08e0 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.437+0000 7f7100bb6640 1 -- 172.21.15.19:0/1473770858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc156d00 con 0x7f70fc0d08e0 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.438+0000 7f7100bb6640 1 -- 172.21.15.19:0/1473770858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70fc073a00 con 0x7f70fc0d08e0 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.438+0000 7f70eaffd640 1 -- 172.21.15.19:0/1473770858 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70f401b070 con 0x7f70fc0d08e0 2024-04-25T09:59:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.439+0000 7f70eaffd640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70dc06b4d0 0x7f70dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.439+0000 7f70eaffd640 1 -- 172.21.15.19:0/1473770858 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f70f40893e0 con 0x7f70fc0d08e0 2024-04-25T09:59:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.439+0000 7f70f9d9b640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70dc06b4d0 0x7f70dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.440+0000 7f70f9d9b640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70dc06b4d0 0x7f70dc06d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f70f0034000 tx=0x7f70f002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.443+0000 7f70eaffd640 1 -- 172.21.15.19:0/1473770858 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70f4058340 con 0x7f70fc0d08e0 2024-04-25T09:59:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.621+0000 7f7100bb6640 1 -- 172.21.15.19:0/1473770858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70fc154e80 con 0x7f70fc0d08e0 2024-04-25T09:59:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.622+0000 7f70eaffd640 1 -- 172.21.15.19:0/1473770858 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f70f4057c40 con 0x7f70fc0d08e0 2024-04-25T09:59:00.626 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:59:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 -- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70dc06b4d0 msgr2=0x7f70dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70dc06b4d0 0x7f70dc06d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f70f0034000 tx=0x7f70f002e000 comp rx=0 tx=0).stop 2024-04-25T09:59:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 -- 172.21.15.19:0/1473770858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc0d08e0 msgr2=0x7f70fc152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc0d08e0 0x7f70fc152a10 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f70f400c460 tx=0x7f70f400c920 comp rx=0 tx=0).stop 2024-04-25T09:59:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 -- 172.21.15.19:0/1473770858 shutdown_connections 2024-04-25T09:59:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70dc06b4d0 0x7f70dc06d990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc15dac0 0x7f70fc160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70fc0d34c0 0x7f70fc1567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 --2- 172.21.15.19:0/1473770858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc0d08e0 0x7f70fc152a10 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 -- 172.21.15.19:0/1473770858 >> 172.21.15.19:0/1473770858 conn(0x7f70fc061c30 msgr2=0x7f70fc15be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 -- 172.21.15.19:0/1473770858 shutdown_connections 2024-04-25T09:59:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.624+0000 7f7100bb6640 1 -- 172.21.15.19:0/1473770858 wait complete. 2024-04-25T09:59:00.638 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T09:59:00.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.736+0000 7ff6b0eb3640 1 Processor -- start 2024-04-25T09:59:00.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.736+0000 7ff6b0eb3640 1 -- start start 2024-04-25T09:59:00.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.736+0000 7ff6b0eb3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b6cb0 0x7ff6ac0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6b0eb3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6ac0b5a60 0x7ff6ac0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6b0eb3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac0ceee0 con 0x7ff6ac15db70 2024-04-25T09:59:00.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6b0eb3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ac0d1d10 con 0x7ff6ac0b5a60 2024-04-25T09:59:00.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6b0eb3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0d1ff0 con 0x7ff6ac0b6cb0 2024-04-25T09:59:00.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6aad9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6ac0b5a60 0x7ff6ac0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6a9d9b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b6cb0 0x7ff6ac0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6aad9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6ac0b5a60 0x7ff6ac0d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:46076/0 (socket says 172.21.15.19:46076) 2024-04-25T09:59:00.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6aa59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff6ac15db70 0x7ff6ac162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:36992/0 (socket says 172.21.15.19:36992) 2024-04-25T09:59:00.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6aad9d640 1 -- 172.21.15.19:0/2318524950 learned_addr learned my addr 172.21.15.19:0/2318524950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:00.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6a9d9b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b6cb0 0x7ff6ac0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:36118/0 (socket says 172.21.15.19:36118) 2024-04-25T09:59:00.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6a9d9b640 1 -- 172.21.15.19:0/2318524950 >> v2:172.21.15.112:3300/0 conn(0x7ff6ac0b5a60 msgr2=0x7ff6ac0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6a9d9b640 1 --2- 172.21.15.19:0/2318524950 >> v2:172.21.15.112:3300/0 conn(0x7ff6ac0b5a60 0x7ff6ac0d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6a9d9b640 1 -- 172.21.15.19:0/2318524950 >> v1:172.21.15.112:6789/0 conn(0x7ff6ac15db70 legacy=0x7ff6ac162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6a9d9b640 1 -- 172.21.15.19:0/2318524950 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac0d22d0 con 0x7ff6ac0b6cb0 2024-04-25T09:59:00.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.737+0000 7ff6aad9d640 1 --2- 172.21.15.19:0/2318524950 >> v2:172.21.15.112:3300/0 conn(0x7ff6ac0b5a60 0x7ff6ac0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.738+0000 7ff6a9d9b640 1 --2- 172.21.15.19:0/2318524950 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b6cb0 0x7ff6ac0b5520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff6940097a0 tx=0x7ff69402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a935c3cf70a3c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.738+0000 7ff6a959a640 1 -- 172.21.15.19:0/2318524950 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694037070 con 0x7ff6ac0b6cb0 2024-04-25T09:59:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.738+0000 7ff6a959a640 1 -- 172.21.15.19:0/2318524950 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff694032440 con 0x7ff6ac0b6cb0 2024-04-25T09:59:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.738+0000 7ff6a959a640 1 -- 172.21.15.19:0/2318524950 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694032810 con 0x7ff6ac0b6cb0 2024-04-25T09:59:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.739+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/2318524950 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b6cb0 msgr2=0x7ff6ac0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.739+0000 7ff6b0eb3640 1 --2- 172.21.15.19:0/2318524950 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b6cb0 0x7ff6ac0b5520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff6940097a0 tx=0x7ff69402a180 comp rx=0 tx=0).stop 2024-04-25T09:59:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.739+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/2318524950 shutdown_connections 2024-04-25T09:59:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.739+0000 7ff6b0eb3640 1 --2- 172.21.15.19:0/2318524950 >> v2:172.21.15.112:3300/0 conn(0x7ff6ac0b5a60 0x7ff6ac0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.739+0000 7ff6b0eb3640 1 --2- 172.21.15.19:0/2318524950 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b6cb0 0x7ff6ac0b5520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.739+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/2318524950 >> 172.21.15.19:0/2318524950 conn(0x7ff6ac0d0520 msgr2=0x7ff6ac0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.739+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/2318524950 shutdown_connections 2024-04-25T09:59:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.739+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/2318524950 wait complete. 2024-04-25T09:59:00.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.740+0000 7ff6b0eb3640 1 Processor -- start 2024-04-25T09:59:00.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.740+0000 7ff6b0eb3640 1 -- start start 2024-04-25T09:59:00.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.741+0000 7ff6b0eb3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b5a60 0x7ff6ac160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.741+0000 7ff6b0eb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac0b6cb0 0x7ff6ac157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.741+0000 7ff6b0eb3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6ac15db70 0x7ff6ac158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.741+0000 7ff6b0eb3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0cf1c0 con 0x7ff6ac15db70 2024-04-25T09:59:00.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.741+0000 7ff6b0eb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0ce530 con 0x7ff6ac0b6cb0 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.741+0000 7ff6b0eb3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0cf340 con 0x7ff6ac0b5a60 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.741+0000 7ff6aad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6ac15db70 0x7ff6ac158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.741+0000 7ff6aad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6ac15db70 0x7ff6ac158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49304/0 (socket says 172.21.15.19:49304) 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.741+0000 7ff6aad9d640 1 -- 172.21.15.19:0/753257084 learned_addr learned my addr 172.21.15.19:0/753257084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.741+0000 7ff6aa59c640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b5a60 0x7ff6ac160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.741+0000 7ff6a9d9b640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac0b6cb0 0x7ff6ac157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6aad9d640 1 -- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b5a60 msgr2=0x7ff6ac160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6aad9d640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b5a60 0x7ff6ac160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6aad9d640 1 -- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac0b6cb0 msgr2=0x7ff6ac157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6aad9d640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac0b6cb0 0x7ff6ac157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6aad9d640 1 -- 172.21.15.19:0/753257084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac159760 con 0x7ff6ac15db70 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6aa59c640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b5a60 0x7ff6ac160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6a9d9b640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac0b6cb0 0x7ff6ac157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6aad9d640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6ac15db70 0x7ff6ac158120 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400e960 tx=0x7ff6a400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6a2ffd640 1 -- 172.21.15.19:0/753257084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4017070 con 0x7ff6ac15db70 2024-04-25T09:59:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6a2ffd640 1 -- 172.21.15.19:0/753257084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a4004020 con 0x7ff6ac15db70 2024-04-25T09:59:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/753257084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac159a20 con 0x7ff6ac15db70 2024-04-25T09:59:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6a2ffd640 1 -- 172.21.15.19:0/753257084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a40043f0 con 0x7ff6ac15db70 2024-04-25T09:59:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.742+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/753257084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac159fe0 con 0x7ff6ac15db70 2024-04-25T09:59:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.743+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/753257084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac073990 con 0x7ff6ac15db70 2024-04-25T09:59:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.743+0000 7ff6a2ffd640 1 -- 172.21.15.19:0/753257084 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6a400ca00 con 0x7ff6ac15db70 2024-04-25T09:59:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.744+0000 7ff6a2ffd640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff68c06b4d0 0x7ff68c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.744+0000 7ff6a2ffd640 1 -- 172.21.15.19:0/753257084 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff6a4089fd0 con 0x7ff6ac15db70 2024-04-25T09:59:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.744+0000 7ff6aa59c640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff68c06b4d0 0x7ff68c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.744+0000 7ff6aa59c640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff68c06b4d0 0x7ff68c06d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0012e0 tx=0x7ff69c005b70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:00.748+0000 7ff6a2ffd640 1 -- 172.21.15.19:0/753257084 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6a4058f30 con 0x7ff6ac15db70 2024-04-25T09:59:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:01.002+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/753257084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7ff6ac0d1820 con 0x7ff6ac15db70 2024-04-25T09:59:02.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.005+0000 7ff6a2ffd640 1 -- 172.21.15.19:0/753257084 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v293) v1 ==== 147+0+0 (secure 0 0 0) 0x7ff6a4058830 con 0x7ff6ac15db70 2024-04-25T09:59:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff68c06b4d0 msgr2=0x7ff68c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff68c06b4d0 0x7ff68c06d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0012e0 tx=0x7ff69c005b70 comp rx=0 tx=0).stop 2024-04-25T09:59:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/753257084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6ac15db70 msgr2=0x7ff6ac158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6ac15db70 0x7ff6ac158120 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400e960 tx=0x7ff6a400ee20 comp rx=0 tx=0).stop 2024-04-25T09:59:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/753257084 shutdown_connections 2024-04-25T09:59:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff68c06b4d0 0x7ff68c06d990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6ac15db70 0x7ff6ac158120 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6ac0b6cb0 0x7ff6ac157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 --2- 172.21.15.19:0/753257084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6ac0b5a60 0x7ff6ac160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/753257084 >> 172.21.15.19:0/753257084 conn(0x7ff6ac0d0520 msgr2=0x7ff6ac157100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/753257084 shutdown_connections 2024-04-25T09:59:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.008+0000 7ff6b0eb3640 1 -- 172.21.15.19:0/753257084 wait complete. 2024-04-25T09:59:02.025 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:59:02.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.132+0000 7f08d4734640 1 Processor -- start 2024-04-25T09:59:02.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.132+0000 7f08d4734640 1 -- start start 2024-04-25T09:59:02.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d4734640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08cc1454a0 0x7f08cc0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d4734640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f08cc15c970 0x7f08cc06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d4734640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08cc0af540 con 0x7f08cc06b120 2024-04-25T09:59:02.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d4734640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08cc0af820 con 0x7f08cc15c970 2024-04-25T09:59:02.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d4734640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc0afb00 con 0x7f08cc1454a0 2024-04-25T09:59:02.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d1ccf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f08cc15c970 0x7f08cc06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d2cd1640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f08cc06b120 0x7f08cc15f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41462/0 (socket says 172.21.15.19:41462) 2024-04-25T09:59:02.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d2cd1640 1 -- 172.21.15.19:0/1653910408 learned_addr learned my addr 172.21.15.19:0/1653910408 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:02.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d24d0640 1 --2- 172.21.15.19:0/1653910408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08cc1454a0 0x7f08cc0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d1ccf640 1 -- 172.21.15.19:0/1653910408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08cc1454a0 msgr2=0x7f08cc0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:02.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d1ccf640 1 --2- 172.21.15.19:0/1653910408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08cc1454a0 0x7f08cc0aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d1ccf640 1 -- 172.21.15.19:0/1653910408 >> v1:172.21.15.19:6789/0 conn(0x7f08cc06b120 legacy=0x7f08cc15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d1ccf640 1 -- 172.21.15.19:0/1653910408 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc0afde0 con 0x7f08cc15c970 2024-04-25T09:59:02.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.133+0000 7f08d14ce640 1 -- 172.21.15.19:0/1653910408 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543475863 0 0) 0x7f08cc0af540 con 0x7f08cc06b120 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.134+0000 7f08d1ccf640 1 --2- 172.21.15.19:0/1653910408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08cc15c970 0x7f08cc06abe0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0097a0 tx=0x7f08bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35e881b98fd25cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.134+0000 7f08d14ce640 1 -- 172.21.15.19:0/1653910408 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc037070 con 0x7f08cc15c970 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.134+0000 7f08d14ce640 1 -- 172.21.15.19:0/1653910408 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08bc032440 con 0x7f08cc15c970 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.134+0000 7f08d14ce640 1 -- 172.21.15.19:0/1653910408 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc032810 con 0x7f08cc15c970 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.135+0000 7f08d4734640 1 -- 172.21.15.19:0/1653910408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08cc15c970 msgr2=0x7f08cc06abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.135+0000 7f08d4734640 1 --2- 172.21.15.19:0/1653910408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08cc15c970 0x7f08cc06abe0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0097a0 tx=0x7f08bc02a180 comp rx=0 tx=0).stop 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.135+0000 7f08d4734640 1 -- 172.21.15.19:0/1653910408 shutdown_connections 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.135+0000 7f08d4734640 1 --2- 172.21.15.19:0/1653910408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08cc15c970 0x7f08cc06abe0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.135+0000 7f08d4734640 1 --2- 172.21.15.19:0/1653910408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08cc1454a0 0x7f08cc0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.135+0000 7f08d4734640 1 -- 172.21.15.19:0/1653910408 >> 172.21.15.19:0/1653910408 conn(0x7f08cc0b5af0 msgr2=0x7f08cc0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.135+0000 7f08d4734640 1 -- 172.21.15.19:0/1653910408 shutdown_connections 2024-04-25T09:59:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.135+0000 7f08d4734640 1 -- 172.21.15.19:0/1653910408 wait complete. 2024-04-25T09:59:02.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.136+0000 7f08d4734640 1 Processor -- start 2024-04-25T09:59:02.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.136+0000 7f08d4734640 1 -- start start 2024-04-25T09:59:02.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d4734640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08cc06b120 0x7f08cc15b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d4734640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08cc1454a0 0x7f08cc06efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d4734640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08cc15c970 0x7f08cc1f3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d4734640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc15e620 con 0x7f08cc06b120 2024-04-25T09:59:02.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d4734640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc15e4a0 con 0x7f08cc1454a0 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d4734640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc15e7a0 con 0x7f08cc15c970 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d24d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08cc06b120 0x7f08cc15b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d24d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08cc06b120 0x7f08cc15b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49328/0 (socket says 172.21.15.19:49328) 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d24d0640 1 -- 172.21.15.19:0/348978214 learned_addr learned my addr 172.21.15.19:0/348978214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d2cd1640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08cc15c970 0x7f08cc1f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d1ccf640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08cc1454a0 0x7f08cc06efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d24d0640 1 -- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08cc15c970 msgr2=0x7f08cc1f3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d24d0640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08cc15c970 0x7f08cc1f3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d24d0640 1 -- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08cc1454a0 msgr2=0x7f08cc06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d24d0640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08cc1454a0 0x7f08cc06efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d24d0640 1 -- 172.21.15.19:0/348978214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc143f70 con 0x7f08cc06b120 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d2cd1640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08cc15c970 0x7f08cc1f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08d24d0640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08cc06b120 0x7f08cc15b3e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f08c800d800 tx=0x7f08c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.137+0000 7f08c2ffd640 1 -- 172.21.15.19:0/348978214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800b6c0 con 0x7f08cc06b120 2024-04-25T09:59:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.138+0000 7f08c2ffd640 1 -- 172.21.15.19:0/348978214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c8019070 con 0x7f08cc06b120 2024-04-25T09:59:02.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.138+0000 7f08d4734640 1 -- 172.21.15.19:0/348978214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08cc144250 con 0x7f08cc06b120 2024-04-25T09:59:02.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.138+0000 7f08c2ffd640 1 -- 172.21.15.19:0/348978214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800ba80 con 0x7f08cc06b120 2024-04-25T09:59:02.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.138+0000 7f08d4734640 1 -- 172.21.15.19:0/348978214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc144810 con 0x7f08cc06b120 2024-04-25T09:59:02.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.139+0000 7f08d4734640 1 -- 172.21.15.19:0/348978214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08cc06cc80 con 0x7f08cc06b120 2024-04-25T09:59:02.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.139+0000 7f08c2ffd640 1 -- 172.21.15.19:0/348978214 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08c800bc20 con 0x7f08cc06b120 2024-04-25T09:59:02.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.140+0000 7f08c2ffd640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08b006b4d0 0x7f08b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.140+0000 7f08c2ffd640 1 -- 172.21.15.19:0/348978214 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f08c808a610 con 0x7f08cc06b120 2024-04-25T09:59:02.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.140+0000 7f08d1ccf640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08b006b4d0 0x7f08b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.140+0000 7f08d1ccf640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08b006b4d0 0x7f08b006d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f08bc034000 tx=0x7f08bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:02.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.144+0000 7f08c2ffd640 1 -- 172.21.15.19:0/348978214 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08c8059560 con 0x7f08cc06b120 2024-04-25T09:59:02.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.328+0000 7f08d4734640 1 -- 172.21.15.19:0/348978214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f08cc073a00 con 0x7f08cc06b120 2024-04-25T09:59:02.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.332+0000 7f08c2ffd640 1 -- 172.21.15.19:0/348978214 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f08c8058e60 con 0x7f08cc06b120 2024-04-25T09:59:02.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 -- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08b006b4d0 msgr2=0x7f08b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08b006b4d0 0x7f08b006d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f08bc034000 tx=0x7f08bc02e000 comp rx=0 tx=0).stop 2024-04-25T09:59:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 -- 172.21.15.19:0/348978214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08cc06b120 msgr2=0x7f08cc15b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08cc06b120 0x7f08cc15b3e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f08c800d800 tx=0x7f08c800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 -- 172.21.15.19:0/348978214 shutdown_connections 2024-04-25T09:59:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08b006b4d0 0x7f08b006d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08cc15c970 0x7f08cc1f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08cc1454a0 0x7f08cc06efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 --2- 172.21.15.19:0/348978214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08cc06b120 0x7f08cc15b3e0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 -- 172.21.15.19:0/348978214 >> 172.21.15.19:0/348978214 conn(0x7f08cc0b5af0 msgr2=0x7f08cc0b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 -- 172.21.15.19:0/348978214 shutdown_connections 2024-04-25T09:59:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.334+0000 7f08d4734640 1 -- 172.21.15.19:0/348978214 wait complete. 2024-04-25T09:59:02.348 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:59:02.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:59:02.383 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:59:02.435 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:59:02.435 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:59:02.435 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:02.450 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:59:02.450 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:59:02.450 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:02.468 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:59:02.468 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:59:02.468 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:02.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:59:02.505 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:59:02.506 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:02.524 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:59:02.525 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:59:02.525 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:02.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.642+0000 7efdad161640 1 Processor -- start 2024-04-25T09:59:02.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.642+0000 7efdad161640 1 -- start start 2024-04-25T09:59:02.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efdad161640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efda80b6cb0 0x7efda80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efdad161640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efda80b5a60 0x7efda80d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efdad161640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda80ceee0 con 0x7efda815db70 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efdad161640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda80d1d10 con 0x7efda80b6cb0 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efdad161640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda80d1ff0 con 0x7efda80b5a60 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efda659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efda80b6cb0 0x7efda80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efda659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efda80b6cb0 0x7efda80b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49334/0 (socket says 172.21.15.19:49334) 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efda659c640 1 -- 172.21.15.19:0/629274824 learned_addr learned my addr 172.21.15.19:0/629274824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efda659c640 1 -- 172.21.15.19:0/629274824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efda80b5a60 msgr2=0x7efda80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efda659c640 1 --2- 172.21.15.19:0/629274824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efda80b5a60 0x7efda80d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efda659c640 1 -- 172.21.15.19:0/629274824 >> v1:172.21.15.112:6789/0 conn(0x7efda815db70 legacy=0x7efda8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.643+0000 7efda659c640 1 -- 172.21.15.19:0/629274824 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda80d22d0 con 0x7efda80b6cb0 2024-04-25T09:59:02.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.644+0000 7efda659c640 1 --2- 172.21.15.19:0/629274824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda80b6cb0 0x7efda80b5520 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009870 tx=0x7efd9c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21a56fa9622ab80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:02.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.644+0000 7efda5d9b640 1 -- 172.21.15.19:0/629274824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c037070 con 0x7efda80b6cb0 2024-04-25T09:59:02.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.644+0000 7efda5d9b640 1 -- 172.21.15.19:0/629274824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c032440 con 0x7efda80b6cb0 2024-04-25T09:59:02.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.644+0000 7efda5d9b640 1 -- 172.21.15.19:0/629274824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c032810 con 0x7efda80b6cb0 2024-04-25T09:59:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.644+0000 7efdad161640 1 -- 172.21.15.19:0/629274824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda80b6cb0 msgr2=0x7efda80b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.644+0000 7efdad161640 1 --2- 172.21.15.19:0/629274824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda80b6cb0 0x7efda80b5520 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009870 tx=0x7efd9c02a210 comp rx=0 tx=0).stop 2024-04-25T09:59:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.645+0000 7efdad161640 1 -- 172.21.15.19:0/629274824 shutdown_connections 2024-04-25T09:59:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.645+0000 7efdad161640 1 --2- 172.21.15.19:0/629274824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efda80b5a60 0x7efda80d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.645+0000 7efdad161640 1 --2- 172.21.15.19:0/629274824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda80b6cb0 0x7efda80b5520 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.645+0000 7efdad161640 1 -- 172.21.15.19:0/629274824 >> 172.21.15.19:0/629274824 conn(0x7efda80d0520 msgr2=0x7efda80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.645+0000 7efdad161640 1 -- 172.21.15.19:0/629274824 shutdown_connections 2024-04-25T09:59:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.645+0000 7efdad161640 1 -- 172.21.15.19:0/629274824 wait complete. 2024-04-25T09:59:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.646+0000 7efdad161640 1 Processor -- start 2024-04-25T09:59:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.646+0000 7efdad161640 1 -- start start 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efdad161640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda80b5a60 0x7efda815e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efdad161640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda80b6cb0 0x7efda815fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efdad161640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efda815db70 0x7efda8154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efdad161640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda80cf1c0 con 0x7efda80b5a60 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efdad161640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda80ce530 con 0x7efda80b6cb0 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efdad161640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda80cf340 con 0x7efda815db70 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda80b5a60 0x7efda815e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda80b5a60 0x7efda815e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49336/0 (socket says 172.21.15.19:49336) 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda6d9d640 1 -- 172.21.15.19:0/2968117707 learned_addr learned my addr 172.21.15.19:0/2968117707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda659c640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda80b6cb0 0x7efda815fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda759e640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efda815db70 0x7efda8154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda6d9d640 1 -- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efda815db70 msgr2=0x7efda8154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda6d9d640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efda815db70 0x7efda8154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda6d9d640 1 -- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda80b6cb0 msgr2=0x7efda815fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda6d9d640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda80b6cb0 0x7efda815fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda6d9d640 1 -- 172.21.15.19:0/2968117707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda815e860 con 0x7efda80b5a60 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda659c640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda80b6cb0 0x7efda815fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efda6d9d640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda80b5a60 0x7efda815e160 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7efda000b3e0 tx=0x7efda000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.647+0000 7efd8f7fe640 1 -- 172.21.15.19:0/2968117707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0015020 con 0x7efda80b5a60 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.648+0000 7efd8f7fe640 1 -- 172.21.15.19:0/2968117707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda0011850 con 0x7efda80b5a60 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.648+0000 7efdad161640 1 -- 172.21.15.19:0/2968117707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda815eb40 con 0x7efda80b5a60 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.648+0000 7efd8f7fe640 1 -- 172.21.15.19:0/2968117707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0011c40 con 0x7efda80b5a60 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.648+0000 7efdad161640 1 -- 172.21.15.19:0/2968117707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda80cac10 con 0x7efda80b5a60 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.648+0000 7efdad161640 1 -- 172.21.15.19:0/2968117707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda8073990 con 0x7efda80b5a60 2024-04-25T09:59:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.649+0000 7efd8f7fe640 1 -- 172.21.15.19:0/2968117707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efda0018020 con 0x7efda80b5a60 2024-04-25T09:59:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.649+0000 7efd8f7fe640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd9406b4d0 0x7efd9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.650+0000 7efd8f7fe640 1 -- 172.21.15.19:0/2968117707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efda008a010 con 0x7efda80b5a60 2024-04-25T09:59:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.650+0000 7efda659c640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd9406b4d0 0x7efd9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.650+0000 7efda659c640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd9406b4d0 0x7efd9406d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efd9c034000 tx=0x7efd9c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.654+0000 7efd8f7fe640 1 -- 172.21.15.19:0/2968117707 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efda001f020 con 0x7efda80b5a60 2024-04-25T09:59:02.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.871+0000 7efdad161640 1 -- 172.21.15.19:0/2968117707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efda80d1820 con 0x7efda80b5a60 2024-04-25T09:59:02.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.871+0000 7efd8f7fe640 1 -- 172.21.15.19:0/2968117707 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+3702 (secure 0 0 0) 0x7efda00172c0 con 0x7efda80b5a60 2024-04-25T09:59:02.878 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:02.878 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":54,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2023643575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2023643575},{"type":"v1","addr":"172.21.15.19:6835","nonce":2023643575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3683680157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3683680157},{"type":"v1","addr":"172.21.15.112:6836","nonce":3683680157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2248535890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2248535890},{"type":"v1","addr":"172.21.15.112:6837","nonce":2248535890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/309620056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":309620056},{"type":"v1","addr":"172.21.15.19:6833","nonce":309620056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[]} 2024-04-25T09:59:02.879 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 54 2024-04-25T09:59:02.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 -- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd9406b4d0 msgr2=0x7efd9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd9406b4d0 0x7efd9406d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efd9c034000 tx=0x7efd9c02e000 comp rx=0 tx=0).stop 2024-04-25T09:59:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 -- 172.21.15.19:0/2968117707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda80b5a60 msgr2=0x7efda815e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda80b5a60 0x7efda815e160 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7efda000b3e0 tx=0x7efda000b8a0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 -- 172.21.15.19:0/2968117707 shutdown_connections 2024-04-25T09:59:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd9406b4d0 0x7efd9406d990 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efda815db70 0x7efda8154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda80b6cb0 0x7efda815fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 --2- 172.21.15.19:0/2968117707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efda80b5a60 0x7efda815e160 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 -- 172.21.15.19:0/2968117707 >> 172.21.15.19:0/2968117707 conn(0x7efda80d0520 msgr2=0x7efda806c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 -- 172.21.15.19:0/2968117707 shutdown_connections 2024-04-25T09:59:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:02.873+0000 7efdad161640 1 -- 172.21.15.19:0/2968117707 wait complete. 2024-04-25T09:59:02.888 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:59:02.888 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:59:02.888 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:59:02.888 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:59:02.888 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:59:02.888 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:59:02.889 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:59:02.889 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:59:02.889 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:59:02.889 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:59:02.890 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:59:02.890 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:59:02.890 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:59:02.890 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:59:02.890 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:59:02.890 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:59:02.900 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:59:02.900 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:59:02.901 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:59:02.901 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:59:02.902 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:59:02.902 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:59:02.903 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:59:02.903 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:59:02.943 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:59:02.943 DEBUG:teuthology.parallel:result is None 2024-04-25T09:59:02.944 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:59:02.944 DEBUG:teuthology.parallel:result is None 2024-04-25T09:59:02.945 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:59:02.946 DEBUG:teuthology.parallel:result is None 2024-04-25T09:59:02.946 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:59:02.946 DEBUG:teuthology.parallel:result is None 2024-04-25T09:59:02.946 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:59:03.019 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T09:59:03.020 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T09:59:03.026 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T09:59:03.026 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T09:59:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.049+0000 7fab566e2640 1 Processor -- start 2024-04-25T09:59:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.049+0000 7fab566e2640 1 -- start start 2024-04-25T09:59:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.050+0000 7fab566e2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fab5015bb50 0x7fab500c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.050+0000 7fab566e2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fab500c70d0 0x7fab500cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.050+0000 7fab566e2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab500c75e0 con 0x7fab500cfb40 2024-04-25T09:59:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.050+0000 7fab566e2640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab500c8680 con 0x7fab5015bb50 2024-04-25T09:59:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.050+0000 7fab566e2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab500c78c0 con 0x7fab500c70d0 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.050+0000 7fab4ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fab5015bb50 0x7fab500c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.050+0000 7fab54c7f640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fab500cfb40 0x7fab50070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41466/0 (socket says 172.21.15.19:41466) 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.050+0000 7fab54c7f640 1 -- 172.21.15.19:0/818214957 learned_addr learned my addr 172.21.15.19:0/818214957 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.050+0000 7fab4ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fab5015bb50 0x7fab500c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49372/0 (socket says 172.21.15.19:49372) 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.051+0000 7fab4f7fe640 1 --2- 172.21.15.19:0/818214957 >> v2:172.21.15.112:3300/0 conn(0x7fab500c70d0 0x7fab500cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.051+0000 7fab4ffff640 1 -- 172.21.15.19:0/818214957 >> v2:172.21.15.112:3300/0 conn(0x7fab500c70d0 msgr2=0x7fab500cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.051+0000 7fab4ffff640 1 --2- 172.21.15.19:0/818214957 >> v2:172.21.15.112:3300/0 conn(0x7fab500c70d0 0x7fab500cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.051+0000 7fab4ffff640 1 -- 172.21.15.19:0/818214957 >> v1:172.21.15.19:6789/0 conn(0x7fab500cfb40 legacy=0x7fab50070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.051+0000 7fab4ffff640 1 -- 172.21.15.19:0/818214957 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab500c7ba0 con 0x7fab5015bb50 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.051+0000 7fab4ffff640 1 --2- 172.21.15.19:0/818214957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5015bb50 0x7fab500c6b90 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fab5006f220 tx=0x7fab3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b734461634fc8f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.052+0000 7fab4effd640 1 -- 172.21.15.19:0/818214957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38037070 con 0x7fab5015bb50 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.052+0000 7fab4effd640 1 -- 172.21.15.19:0/818214957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab38032440 con 0x7fab5015bb50 2024-04-25T09:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.052+0000 7fab4effd640 1 -- 172.21.15.19:0/818214957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38032810 con 0x7fab5015bb50 2024-04-25T09:59:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.052+0000 7fab566e2640 1 -- 172.21.15.19:0/818214957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5015bb50 msgr2=0x7fab500c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.052+0000 7fab566e2640 1 --2- 172.21.15.19:0/818214957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5015bb50 0x7fab500c6b90 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fab5006f220 tx=0x7fab3802a230 comp rx=0 tx=0).stop 2024-04-25T09:59:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.052+0000 7fab566e2640 1 -- 172.21.15.19:0/818214957 shutdown_connections 2024-04-25T09:59:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.052+0000 7fab566e2640 1 --2- 172.21.15.19:0/818214957 >> v2:172.21.15.112:3300/0 conn(0x7fab500c70d0 0x7fab500cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.052+0000 7fab566e2640 1 --2- 172.21.15.19:0/818214957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab5015bb50 0x7fab500c6b90 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.052+0000 7fab566e2640 1 -- 172.21.15.19:0/818214957 >> 172.21.15.19:0/818214957 conn(0x7fab5006f500 msgr2=0x7fab5006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.052+0000 7fab566e2640 1 -- 172.21.15.19:0/818214957 shutdown_connections 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.052+0000 7fab566e2640 1 -- 172.21.15.19:0/818214957 wait complete. 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.053+0000 7fab566e2640 1 Processor -- start 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.054+0000 7fab566e2640 1 -- start start 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.054+0000 7fab566e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab500c70d0 0x7fab5014d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.054+0000 7fab566e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab500cfb40 0x7fab50142be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.054+0000 7fab566e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5015bb50 0x7fab5015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.054+0000 7fab566e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5015f6d0 con 0x7fab500cfb40 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.054+0000 7fab566e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab500c8160 con 0x7fab5015bb50 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.054+0000 7fab566e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5015f850 con 0x7fab500c70d0 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.054+0000 7fab4f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab500cfb40 0x7fab50142be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab500cfb40 0x7fab50142be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49378/0 (socket says 172.21.15.19:49378) 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4f7fe640 1 -- 172.21.15.19:0/1401720649 learned_addr learned my addr 172.21.15.19:0/1401720649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4ffff640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab500c70d0 0x7fab5014d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab54c7f640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5015bb50 0x7fab5015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4f7fe640 1 -- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab500c70d0 msgr2=0x7fab5014d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4f7fe640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab500c70d0 0x7fab5014d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4f7fe640 1 -- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5015bb50 msgr2=0x7fab5015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4f7fe640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5015bb50 0x7fab5015bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4f7fe640 1 -- 172.21.15.19:0/1401720649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab501588f0 con 0x7fab500cfb40 2024-04-25T09:59:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4f7fe640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab500cfb40 0x7fab50142be0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fab4000d800 tx=0x7fab4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4cff9640 1 -- 172.21.15.19:0/1401720649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40004020 con 0x7fab500cfb40 2024-04-25T09:59:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4cff9640 1 -- 172.21.15.19:0/1401720649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab400041c0 con 0x7fab500cfb40 2024-04-25T09:59:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab566e2640 1 -- 172.21.15.19:0/1401720649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab500b64f0 con 0x7fab500cfb40 2024-04-25T09:59:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab4cff9640 1 -- 172.21.15.19:0/1401720649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40004390 con 0x7fab500cfb40 2024-04-25T09:59:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.055+0000 7fab566e2640 1 -- 172.21.15.19:0/1401720649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab500b6ab0 con 0x7fab500cfb40 2024-04-25T09:59:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.056+0000 7fab566e2640 1 -- 172.21.15.19:0/1401720649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab50073a00 con 0x7fab500cfb40 2024-04-25T09:59:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.057+0000 7fab4cff9640 1 -- 172.21.15.19:0/1401720649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab4001b020 con 0x7fab500cfb40 2024-04-25T09:59:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.057+0000 7fab4cff9640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab3006b4d0 0x7fab3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.057+0000 7fab4cff9640 1 -- 172.21.15.19:0/1401720649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fab40089f40 con 0x7fab500cfb40 2024-04-25T09:59:03.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.057+0000 7fab4ffff640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab3006b4d0 0x7fab3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:03.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.058+0000 7fab4ffff640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab3006b4d0 0x7fab3006d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fab38034000 tx=0x7fab3802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:03.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.061+0000 7fab4cff9640 1 -- 172.21.15.19:0/1401720649 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab40058eb0 con 0x7fab500cfb40 2024-04-25T09:59:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:03.253+0000 7fab566e2640 1 -- 172.21.15.19:0/1401720649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fab500cf150 con 0x7fab500cfb40 2024-04-25T09:59:04.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.352+0000 7fab4cff9640 1 -- 172.21.15.19:0/1401720649 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v91) v1 ==== 97+0+0 (secure 0 0 0) 0x7fab400587b0 con 0x7fab500cfb40 2024-04-25T09:59:04.356 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T09:59:04.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 -- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab3006b4d0 msgr2=0x7fab3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab3006b4d0 0x7fab3006d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fab38034000 tx=0x7fab3802e000 comp rx=0 tx=0).stop 2024-04-25T09:59:04.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 -- 172.21.15.19:0/1401720649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab500cfb40 msgr2=0x7fab50142be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab500cfb40 0x7fab50142be0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fab4000d800 tx=0x7fab4000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 -- 172.21.15.19:0/1401720649 shutdown_connections 2024-04-25T09:59:04.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab3006b4d0 0x7fab3006d990 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab5015bb50 0x7fab5015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab500cfb40 0x7fab50142be0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 --2- 172.21.15.19:0/1401720649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab500c70d0 0x7fab5014d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 -- 172.21.15.19:0/1401720649 >> 172.21.15.19:0/1401720649 conn(0x7fab5006f500 msgr2=0x7fab500cbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:04.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 -- 172.21.15.19:0/1401720649 shutdown_connections 2024-04-25T09:59:04.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.355+0000 7fab566e2640 1 -- 172.21.15.19:0/1401720649 wait complete. 2024-04-25T09:59:04.372 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:59:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.475+0000 7f7f68a5a640 1 Processor -- start 2024-04-25T09:59:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.476+0000 7f7f68a5a640 1 -- start start 2024-04-25T09:59:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.476+0000 7f7f68a5a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7f640b2cb0 0x7f7f640d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.476+0000 7f7f68a5a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f640b97a0 con 0x7f7f6415dbe0 2024-04-25T09:59:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.476+0000 7f7f68a5a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f640d1c50 con 0x7f7f640d1020 2024-04-25T09:59:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.476+0000 7f7f68a5a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6415dfc0 con 0x7f7f640b2cb0 2024-04-25T09:59:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6259c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7f6415dbe0 0x7f7f640d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41468/0 (socket says 172.21.15.19:41468) 2024-04-25T09:59:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6259c640 1 -- 172.21.15.19:0/907493206 learned_addr learned my addr 172.21.15.19:0/907493206 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544958761 0 0) 0x7f7f640b97a0 con 0x7f7f6415dbe0 2024-04-25T09:59:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f61d9b640 1 --2- 172.21.15.19:0/907493206 >> v2:172.21.15.112:3300/0 conn(0x7f7f640b2cb0 0x7f7f640d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f50003520 con 0x7f7f6415dbe0 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2262598689 0 0) 0x7f7f50003520 con 0x7f7f6415dbe0 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 >> v2:172.21.15.112:3300/0 conn(0x7f7f640b2cb0 msgr2=0x7f7f640d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 --2- 172.21.15.19:0/907493206 >> v2:172.21.15.112:3300/0 conn(0x7f7f640b2cb0 0x7f7f640d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 >> v1:172.21.15.112:6789/0 conn(0x7f7f640d1020 legacy=0x7f7f640d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f640d1f30 con 0x7f7f6415dbe0 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f7f5c001670 con 0x7f7f6415dbe0 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 >> v1:172.21.15.19:6789/0 conn(0x7f7f6415dbe0 legacy=0x7f7f640d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f500039b0 0x7f7f50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f50004800 0x7f7f50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f500051c0 0x7f7f5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5000c030 con 0x7f7f50004800 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5000c1e0 con 0x7f7f500051c0 2024-04-25T09:59:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5000c3e0 con 0x7f7f500039b0 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.477+0000 7f7f61d9b640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f50004800 0x7f7f50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.478+0000 7f7f62d9d640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f500051c0 0x7f7f5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.478+0000 7f7f6259c640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f500039b0 0x7f7f50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.478+0000 7f7f61d9b640 1 -- 172.21.15.19:0/907493206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f500039b0 msgr2=0x7f7f50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.478+0000 7f7f61d9b640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f500039b0 0x7f7f50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.478+0000 7f7f61d9b640 1 -- 172.21.15.19:0/907493206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f500051c0 msgr2=0x7f7f5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.478+0000 7f7f61d9b640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f500051c0 0x7f7f5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.478+0000 7f7f61d9b640 1 -- 172.21.15.19:0/907493206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f50003520 con 0x7f7f50004800 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.478+0000 7f7f62d9d640 1 -- 172.21.15.19:0/907493206 reap_dead start 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.478+0000 7f7f6259c640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f500039b0 0x7f7f50003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.478+0000 7f7f61d9b640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f50004800 0x7f7f50004c80 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009a90 tx=0x7f7f50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8f6d547a689b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.479+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c036070 con 0x7f7f50004800 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.479+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c031400 con 0x7f7f50004800 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.479+0000 7f7f6159a640 1 -- 172.21.15.19:0/907493206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c009690 con 0x7f7f50004800 2024-04-25T09:59:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.479+0000 7f7f68a5a640 1 -- 172.21.15.19:0/907493206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f50004800 msgr2=0x7f7f50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.479+0000 7f7f68a5a640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f50004800 0x7f7f50004c80 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009a90 tx=0x7f7f50005830 comp rx=0 tx=0).stop 2024-04-25T09:59:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.479+0000 7f7f68a5a640 1 -- 172.21.15.19:0/907493206 shutdown_connections 2024-04-25T09:59:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.479+0000 7f7f68a5a640 1 --2- 172.21.15.19:0/907493206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f50004800 0x7f7f50004c80 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.479+0000 7f7f68a5a640 1 -- 172.21.15.19:0/907493206 >> 172.21.15.19:0/907493206 conn(0x7f7f64061a20 msgr2=0x7f7f64061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.479+0000 7f7f68a5a640 1 -- 172.21.15.19:0/907493206 shutdown_connections 2024-04-25T09:59:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.479+0000 7f7f68a5a640 1 -- 172.21.15.19:0/907493206 wait complete. 2024-04-25T09:59:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.481+0000 7f7f68a5a640 1 Processor -- start 2024-04-25T09:59:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.481+0000 7f7f68a5a640 1 -- start start 2024-04-25T09:59:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.481+0000 7f7f68a5a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f50004800 0x7f7f64154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.481+0000 7f7f68a5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f641597d0 0x7f7f64159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.481+0000 7f7f68a5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6415b040 0x7f7f6415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.481+0000 7f7f68a5a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640b92a0 con 0x7f7f641597d0 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.481+0000 7f7f68a5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640d1a00 con 0x7f7f6415b040 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.481+0000 7f7f68a5a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640b9420 con 0x7f7f50004800 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.481+0000 7f7f61d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f641597d0 0x7f7f64159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.481+0000 7f7f61d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f641597d0 0x7f7f64159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49394/0 (socket says 172.21.15.19:49394) 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.481+0000 7f7f61d9b640 1 -- 172.21.15.19:0/1714047043 learned_addr learned my addr 172.21.15.19:0/1714047043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f62d9d640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6415b040 0x7f7f6415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f6259c640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f50004800 0x7f7f64154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f61d9b640 1 -- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f50004800 msgr2=0x7f7f64154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f61d9b640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f50004800 0x7f7f64154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f61d9b640 1 -- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6415b040 msgr2=0x7f7f6415b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f61d9b640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6415b040 0x7f7f6415b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f61d9b640 1 -- 172.21.15.19:0/1714047043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6415a2c0 con 0x7f7f641597d0 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f62d9d640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6415b040 0x7f7f6415b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f61d9b640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f641597d0 0x7f7f64159c30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c034040 tx=0x7f7f4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f5affd640 1 -- 172.21.15.19:0/1714047043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c03f020 con 0x7f7f641597d0 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f5affd640 1 -- 172.21.15.19:0/1714047043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c037040 con 0x7f7f641597d0 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f68a5a640 1 -- 172.21.15.19:0/1714047043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6415a5a0 con 0x7f7f641597d0 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f5affd640 1 -- 172.21.15.19:0/1714047043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c036070 con 0x7f7f641597d0 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.482+0000 7f7f68a5a640 1 -- 172.21.15.19:0/1714047043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f64155780 con 0x7f7f641597d0 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.483+0000 7f7f68a5a640 1 -- 172.21.15.19:0/1714047043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f64073990 con 0x7f7f641597d0 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.484+0000 7f7f5affd640 1 -- 172.21.15.19:0/1714047043 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f4c042020 con 0x7f7f641597d0 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.484+0000 7f7f5affd640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f4406b500 0x7f7f4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.484+0000 7f7f5affd640 1 -- 172.21.15.19:0/1714047043 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7f4c0a9d60 con 0x7f7f641597d0 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.484+0000 7f7f6259c640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f4406b500 0x7f7f4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.485+0000 7f7f6259c640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f4406b500 0x7f7f4406d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7f640692f0 tx=0x7f7f5c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.488+0000 7f7f5affd640 1 -- 172.21.15.19:0/1714047043 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f4c049070 con 0x7f7f641597d0 2024-04-25T09:59:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.734+0000 7f7f68a5a640 1 -- 172.21.15.19:0/1714047043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7f640b7ce0 con 0x7f7f641597d0 2024-04-25T09:59:04.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.735+0000 7f7f5affd640 1 -- 172.21.15.19:0/1714047043 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7f4c078e30 con 0x7f7f641597d0 2024-04-25T09:59:04.739 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:04.740 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:59:04.740 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T09:59:04.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.740 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T09:59:04.740 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T09:59:04.741 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.741 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:04.741 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:59:04.741 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:59:04.741 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.741 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.742 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T09:59:04.742 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T09:59:04.742 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.742 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:04.742 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:59:04.743 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:59:04.743 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.743 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.743 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T09:59:04.743 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T09:59:04.743 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.743 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:04.744 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:59:04.744 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:59:04.744 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.744 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.744 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T09:59:04.744 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T09:59:04.744 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T09:59:04.746 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.748 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T09:59:04.748 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T09:59:04.748 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.748 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.750 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.751 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.752 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T09:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T09:59:04.755 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.755 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:59:04.755 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:59:04.755 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.755 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:04.755 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:04.755 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T09:59:04.756 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T09:59:04.756 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:04.756 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:59:04.756 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:59:04.756 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:04.756 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.756 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:04.756 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T09:59:04.756 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:59:04.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.736+0000 7f7f68a5a640 1 -- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f4406b500 msgr2=0x7f7f4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.736+0000 7f7f68a5a640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f4406b500 0x7f7f4406d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7f640692f0 tx=0x7f7f5c009040 comp rx=0 tx=0).stop 2024-04-25T09:59:04.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.736+0000 7f7f68a5a640 1 -- 172.21.15.19:0/1714047043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f641597d0 msgr2=0x7f7f64159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.737+0000 7f7f68a5a640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f641597d0 0x7f7f64159c30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c034040 tx=0x7f7f4c004730 comp rx=0 tx=0).stop 2024-04-25T09:59:04.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.737+0000 7f7f68a5a640 1 -- 172.21.15.19:0/1714047043 shutdown_connections 2024-04-25T09:59:04.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.737+0000 7f7f68a5a640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7f4406b500 0x7f7f4406d9c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.737+0000 7f7f68a5a640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f6415b040 0x7f7f6415b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.737+0000 7f7f68a5a640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f641597d0 0x7f7f64159c30 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.737+0000 7f7f68a5a640 1 --2- 172.21.15.19:0/1714047043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f50004800 0x7f7f64154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.737+0000 7f7f68a5a640 1 -- 172.21.15.19:0/1714047043 >> 172.21.15.19:0/1714047043 conn(0x7f7f64061a20 msgr2=0x7f7f640b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:04.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.737+0000 7f7f68a5a640 1 -- 172.21.15.19:0/1714047043 shutdown_connections 2024-04-25T09:59:04.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.737+0000 7f7f68a5a640 1 -- 172.21.15.19:0/1714047043 wait complete. 2024-04-25T09:59:04.759 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:04.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.856+0000 7f5c266bd640 1 Processor -- start 2024-04-25T09:59:04.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.856+0000 7f5c266bd640 1 -- start start 2024-04-25T09:59:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.857+0000 7f5c266bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c200b2cb0 0x7f5c200d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.857+0000 7f5c266bd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c200d1020 0x7f5c200d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.857+0000 7f5c266bd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c200b97a0 con 0x7f5c2015dbe0 2024-04-25T09:59:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.857+0000 7f5c266bd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2015dfc0 con 0x7f5c200d1020 2024-04-25T09:59:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.857+0000 7f5c266bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200d1a00 con 0x7f5c200b2cb0 2024-04-25T09:59:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c1f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c200b2cb0 0x7f5c200d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c24c5a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c200d1020 0x7f5c200d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c24c5a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c200d1020 0x7f5c200d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:46186/0 (socket says 172.21.15.19:46186) 2024-04-25T09:59:04.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c24c5a640 1 -- 172.21.15.19:0/3171039206 learned_addr learned my addr 172.21.15.19:0/3171039206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:04.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c1f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c200b2cb0 0x7f5c200d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:36240/0 (socket says 172.21.15.19:36240) 2024-04-25T09:59:04.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c1ffff640 1 --1- 172.21.15.19:0/3171039206 >> v1:172.21.15.112:6789/0 conn(0x7f5c2015dbe0 0x7f5c200d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:37040/0 (socket says 172.21.15.19:37040) 2024-04-25T09:59:04.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c24c5a640 1 -- 172.21.15.19:0/3171039206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c200b2cb0 msgr2=0x7f5c200d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:04.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c24c5a640 1 --2- 172.21.15.19:0/3171039206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c200b2cb0 0x7f5c200d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c24c5a640 1 -- 172.21.15.19:0/3171039206 >> v1:172.21.15.112:6789/0 conn(0x7f5c2015dbe0 legacy=0x7f5c200d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c24c5a640 1 -- 172.21.15.19:0/3171039206 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200d1c50 con 0x7f5c200d1020 2024-04-25T09:59:04.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c1effd640 1 -- 172.21.15.19:0/3171039206 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220236854 0 0) 0x7f5c200b97a0 con 0x7f5c2015dbe0 2024-04-25T09:59:04.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.858+0000 7f5c1f7fe640 1 --2- 172.21.15.19:0/3171039206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c200b2cb0 0x7f5c200d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:04.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.859+0000 7f5c24c5a640 1 --2- 172.21.15.19:0/3171039206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c200d1020 0x7f5c200d2c70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009e50 tx=0x7f5c1802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38cca304a2936c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:04.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.859+0000 7f5c1effd640 1 -- 172.21.15.19:0/3171039206 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18036070 con 0x7f5c200d1020 2024-04-25T09:59:04.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.859+0000 7f5c1effd640 1 -- 172.21.15.19:0/3171039206 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c18031600 con 0x7f5c200d1020 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.859+0000 7f5c1effd640 1 -- 172.21.15.19:0/3171039206 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c180319d0 con 0x7f5c200d1020 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.859+0000 7f5c266bd640 1 -- 172.21.15.19:0/3171039206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c200d1020 msgr2=0x7f5c200d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.859+0000 7f5c266bd640 1 --2- 172.21.15.19:0/3171039206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c200d1020 0x7f5c200d2c70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009e50 tx=0x7f5c1802a730 comp rx=0 tx=0).stop 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.859+0000 7f5c266bd640 1 -- 172.21.15.19:0/3171039206 shutdown_connections 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.859+0000 7f5c266bd640 1 --2- 172.21.15.19:0/3171039206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c200d1020 0x7f5c200d2c70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.859+0000 7f5c266bd640 1 --2- 172.21.15.19:0/3171039206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c200b2cb0 0x7f5c200d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.859+0000 7f5c266bd640 1 -- 172.21.15.19:0/3171039206 >> 172.21.15.19:0/3171039206 conn(0x7f5c20061a20 msgr2=0x7f5c20061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.860+0000 7f5c266bd640 1 -- 172.21.15.19:0/3171039206 shutdown_connections 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.860+0000 7f5c266bd640 1 -- 172.21.15.19:0/3171039206 wait complete. 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.861+0000 7f5c266bd640 1 Processor -- start 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.861+0000 7f5c266bd640 1 -- start start 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c266bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c200b2cb0 0x7f5c201560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c266bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c200d1020 0x7f5c20154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c266bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c2015dbe0 0x7f5c200ce200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c266bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200b92a0 con 0x7f5c200d1020 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c266bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200d2210 con 0x7f5c200b2cb0 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c266bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200b9420 con 0x7f5c2015dbe0 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c200d1020 0x7f5c20154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c200d1020 0x7f5c20154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49400/0 (socket says 172.21.15.19:49400) 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c1f7fe640 1 -- 172.21.15.19:0/926642853 learned_addr learned my addr 172.21.15.19:0/926642853 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c1ffff640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c200b2cb0 0x7f5c201560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c24c5a640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c2015dbe0 0x7f5c200ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c1f7fe640 1 -- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c2015dbe0 msgr2=0x7f5c200ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c1f7fe640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c2015dbe0 0x7f5c200ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c1f7fe640 1 -- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c200b2cb0 msgr2=0x7f5c201560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c1f7fe640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c200b2cb0 0x7f5c201560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c1f7fe640 1 -- 172.21.15.19:0/926642853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c20154fb0 con 0x7f5c200d1020 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c1ffff640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c200b2cb0 0x7f5c201560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.862+0000 7f5c1f7fe640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c200d1020 0x7f5c20154a70 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000e990 tx=0x7f5c1000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.863+0000 7f5c1cff9640 1 -- 172.21.15.19:0/926642853 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10017070 con 0x7f5c200d1020 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.863+0000 7f5c1cff9640 1 -- 172.21.15.19:0/926642853 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1000c9e0 con 0x7f5c200d1020 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.863+0000 7f5c266bd640 1 -- 172.21.15.19:0/926642853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c200b3530 con 0x7f5c200d1020 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.863+0000 7f5c1cff9640 1 -- 172.21.15.19:0/926642853 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1000cdb0 con 0x7f5c200d1020 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.863+0000 7f5c266bd640 1 -- 172.21.15.19:0/926642853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c200b3af0 con 0x7f5c200d1020 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.864+0000 7f5c266bd640 1 -- 172.21.15.19:0/926642853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c20073990 con 0x7f5c200d1020 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.864+0000 7f5c1cff9640 1 -- 172.21.15.19:0/926642853 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c1001a070 con 0x7f5c200d1020 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.864+0000 7f5c1cff9640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c0c06b4d0 0x7f5c0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.864+0000 7f5c1cff9640 1 -- 172.21.15.19:0/926642853 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5c10089f00 con 0x7f5c200d1020 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.865+0000 7f5c1ffff640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c0c06b4d0 0x7f5c0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:04.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.865+0000 7f5c1ffff640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c0c06b4d0 0x7f5c0c06d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5c14000f60 tx=0x7f5c14007c10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:04.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:04.869+0000 7f5c1cff9640 1 -- 172.21.15.19:0/926642853 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c10058f40 con 0x7f5c200d1020 2024-04-25T09:59:05.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.051+0000 7f5c266bd640 1 -- 172.21.15.19:0/926642853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c2006b080 con 0x7f5c200d1020 2024-04-25T09:59:05.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.051+0000 7f5c1cff9640 1 -- 172.21.15.19:0/926642853 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+13153 (secure 0 0 0) 0x7f5c10058840 con 0x7f5c200d1020 2024-04-25T09:59:05.055 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:05.055 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:04.351134+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:05.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 -- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c0c06b4d0 msgr2=0x7f5c0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c0c06b4d0 0x7f5c0c06d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5c14000f60 tx=0x7f5c14007c10 comp rx=0 tx=0).stop 2024-04-25T09:59:05.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 -- 172.21.15.19:0/926642853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c200d1020 msgr2=0x7f5c20154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c200d1020 0x7f5c20154a70 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000e990 tx=0x7f5c1000ee50 comp rx=0 tx=0).stop 2024-04-25T09:59:05.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 -- 172.21.15.19:0/926642853 shutdown_connections 2024-04-25T09:59:05.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c0c06b4d0 0x7f5c0c06d990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c2015dbe0 0x7f5c200ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c200d1020 0x7f5c20154a70 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 --2- 172.21.15.19:0/926642853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c200b2cb0 0x7f5c201560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 -- 172.21.15.19:0/926642853 >> 172.21.15.19:0/926642853 conn(0x7f5c20061a20 msgr2=0x7f5c200b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:05.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 -- 172.21.15.19:0/926642853 shutdown_connections 2024-04-25T09:59:05.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.054+0000 7f5c266bd640 1 -- 172.21.15.19:0/926642853 wait complete. 2024-04-25T09:59:05.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:59:05.069 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:59:05.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.169+0000 7f324db02640 1 Processor -- start 2024-04-25T09:59:05.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.169+0000 7f324db02640 1 -- start start 2024-04-25T09:59:05.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.169+0000 7f324db02640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f324815dc20 0x7f32480ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f324db02640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f32480d32f0 0x7f32480cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f324db02640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f324815e000 con 0x7f3248161710 2024-04-25T09:59:05.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f324db02640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32480d1de0 con 0x7f324815dc20 2024-04-25T09:59:05.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f324db02640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3248161c60 con 0x7f32480d32f0 2024-04-25T09:59:05.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f32477fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f324815dc20 0x7f32480ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f32477fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f324815dc20 0x7f32480ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49412/0 (socket says 172.21.15.19:49412) 2024-04-25T09:59:05.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f32477fe640 1 -- 172.21.15.19:0/4054696420 learned_addr learned my addr 172.21.15.19:0/4054696420 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f32477fe640 1 -- 172.21.15.19:0/4054696420 >> v2:172.21.15.112:3300/0 conn(0x7f32480d32f0 msgr2=0x7f32480cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f3247fff640 1 --2- 172.21.15.19:0/4054696420 >> v2:172.21.15.112:3300/0 conn(0x7f32480d32f0 0x7f32480cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f32477fe640 1 --2- 172.21.15.19:0/4054696420 >> v2:172.21.15.112:3300/0 conn(0x7f32480d32f0 0x7f32480cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f32477fe640 1 -- 172.21.15.19:0/4054696420 >> v1:172.21.15.112:6789/0 conn(0x7f3248161710 legacy=0x7f324806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f32477fe640 1 -- 172.21.15.19:0/4054696420 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3248161f40 con 0x7f324815dc20 2024-04-25T09:59:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.170+0000 7f3247fff640 1 --2- 172.21.15.19:0/4054696420 >> v2:172.21.15.112:3300/0 conn(0x7f32480d32f0 0x7f32480cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.171+0000 7f32477fe640 1 --2- 172.21.15.19:0/4054696420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324815dc20 0x7f32480ce670 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f32480b6910 tx=0x7f323802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ffcf1bcceffaff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.171+0000 7f32467fc640 1 -- 172.21.15.19:0/4054696420 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238036070 con 0x7f324815dc20 2024-04-25T09:59:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.171+0000 7f32467fc640 1 -- 172.21.15.19:0/4054696420 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3238031600 con 0x7f324815dc20 2024-04-25T09:59:05.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.171+0000 7f32467fc640 1 -- 172.21.15.19:0/4054696420 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32380319d0 con 0x7f324815dc20 2024-04-25T09:59:05.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.171+0000 7f324db02640 1 -- 172.21.15.19:0/4054696420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324815dc20 msgr2=0x7f32480ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.171+0000 7f324db02640 1 --2- 172.21.15.19:0/4054696420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324815dc20 0x7f32480ce670 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f32480b6910 tx=0x7f323802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.172+0000 7f324db02640 1 -- 172.21.15.19:0/4054696420 shutdown_connections 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.172+0000 7f324db02640 1 --2- 172.21.15.19:0/4054696420 >> v2:172.21.15.112:3300/0 conn(0x7f32480d32f0 0x7f32480cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.172+0000 7f324db02640 1 --2- 172.21.15.19:0/4054696420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324815dc20 0x7f32480ce670 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.172+0000 7f324db02640 1 -- 172.21.15.19:0/4054696420 >> 172.21.15.19:0/4054696420 conn(0x7f32480d0420 msgr2=0x7f32480d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.172+0000 7f324db02640 1 -- 172.21.15.19:0/4054696420 shutdown_connections 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.172+0000 7f324db02640 1 -- 172.21.15.19:0/4054696420 wait complete. 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.173+0000 7f324db02640 1 Processor -- start 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.173+0000 7f324db02640 1 -- start start 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f324db02640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32480d32f0 0x7f3248159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f324db02640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324815dc20 0x7f32481f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f324db02640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3248161710 0x7f32480cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f324db02640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32480b5e50 con 0x7f324815dc20 2024-04-25T09:59:05.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f324db02640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3248162500 con 0x7f32480d32f0 2024-04-25T09:59:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f324db02640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32480b5fd0 con 0x7f3248161710 2024-04-25T09:59:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f3246ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324815dc20 0x7f32481f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f3246ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324815dc20 0x7f32481f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49420/0 (socket says 172.21.15.19:49420) 2024-04-25T09:59:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f3246ffd640 1 -- 172.21.15.19:0/1653313096 learned_addr learned my addr 172.21.15.19:0/1653313096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f32477fe640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32480d32f0 0x7f3248159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f3247fff640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3248161710 0x7f32480cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f3246ffd640 1 -- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3248161710 msgr2=0x7f32480cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f3246ffd640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3248161710 0x7f32480cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f3246ffd640 1 -- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32480d32f0 msgr2=0x7f3248159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f3246ffd640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32480d32f0 0x7f3248159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f3246ffd640 1 -- 172.21.15.19:0/1653313096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32480cb580 con 0x7f324815dc20 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.174+0000 7f3246ffd640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324815dc20 0x7f32481f41e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f323c00b3f0 tx=0x7f323c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.175+0000 7f322ffff640 1 -- 172.21.15.19:0/1653313096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c015020 con 0x7f324815dc20 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.175+0000 7f322ffff640 1 -- 172.21.15.19:0/1653313096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c011830 con 0x7f324815dc20 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.175+0000 7f324db02640 1 -- 172.21.15.19:0/1653313096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32480cb860 con 0x7f324815dc20 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.175+0000 7f322ffff640 1 -- 172.21.15.19:0/1653313096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c011c20 con 0x7f324815dc20 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.175+0000 7f324db02640 1 -- 172.21.15.19:0/1653313096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32480cbe20 con 0x7f324815dc20 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.176+0000 7f324db02640 1 -- 172.21.15.19:0/1653313096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3248073a00 con 0x7f324815dc20 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.176+0000 7f322ffff640 1 -- 172.21.15.19:0/1653313096 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f323c018020 con 0x7f324815dc20 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.177+0000 7f322ffff640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f323406b4d0 0x7f323406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.177+0000 7f322ffff640 1 -- 172.21.15.19:0/1653313096 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f323c089aa0 con 0x7f324815dc20 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.177+0000 7f32477fe640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f323406b4d0 0x7f323406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.178+0000 7f32477fe640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f323406b4d0 0x7f323406d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3238033000 tx=0x7f3238009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.181+0000 7f322ffff640 1 -- 172.21.15.19:0/1653313096 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f323c01f030 con 0x7f324815dc20 2024-04-25T09:59:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.359+0000 7f324db02640 1 -- 172.21.15.19:0/1653313096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32480d16b0 con 0x7f324815dc20 2024-04-25T09:59:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.359+0000 7f322ffff640 1 -- 172.21.15.19:0/1653313096 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f323c058520 con 0x7f324815dc20 2024-04-25T09:59:05.363 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:59:05.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 -- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f323406b4d0 msgr2=0x7f323406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f323406b4d0 0x7f323406d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3238033000 tx=0x7f3238009000 comp rx=0 tx=0).stop 2024-04-25T09:59:05.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 -- 172.21.15.19:0/1653313096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324815dc20 msgr2=0x7f32481f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324815dc20 0x7f32481f41e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f323c00b3f0 tx=0x7f323c00b8b0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 -- 172.21.15.19:0/1653313096 shutdown_connections 2024-04-25T09:59:05.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f323406b4d0 0x7f323406d990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3248161710 0x7f32480cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324815dc20 0x7f32481f41e0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 --2- 172.21.15.19:0/1653313096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32480d32f0 0x7f3248159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 -- 172.21.15.19:0/1653313096 >> 172.21.15.19:0/1653313096 conn(0x7f32480d0420 msgr2=0x7f32480b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:05.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 -- 172.21.15.19:0/1653313096 shutdown_connections 2024-04-25T09:59:05.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.361+0000 7f324db02640 1 -- 172.21.15.19:0/1653313096 wait complete. 2024-04-25T09:59:05.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.473+0000 7f9a0aff9640 1 Processor -- start 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.473+0000 7f9a0aff9640 1 -- start start 2024-04-25T09:59:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a0aff9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04154ee0 0x7f9a040cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a0aff9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9a040cfcf0 0x7f9a040cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a0aff9640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a04071480 con 0x7f9a040ce3c0 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a0aff9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a040ce830 con 0x7f9a040cfcf0 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a0aff9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04072090 con 0x7f9a04154ee0 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a09596640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9a040ce3c0 0x7f9a0415e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41470/0 (socket says 172.21.15.19:41470) 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a09596640 1 -- 172.21.15.19:0/656701404 learned_addr learned my addr 172.21.15.19:0/656701404 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a03fff640 1 --2- 172.21.15.19:0/656701404 >> v2:172.21.15.112:3300/0 conn(0x7f9a040cfcf0 0x7f9a040cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a08d95640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04154ee0 0x7f9a040cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957137547 0 0) 0x7f9a04071480 con 0x7f9a040ce3c0 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99f0003520 con 0x7f9a040ce3c0 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.474+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1029849736 0 0) 0x7f99f0003520 con 0x7f9a040ce3c0 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04154ee0 msgr2=0x7f9a040cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04154ee0 0x7f9a040cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 >> v2:172.21.15.112:3300/0 conn(0x7f9a040cfcf0 msgr2=0x7f9a040cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 --2- 172.21.15.19:0/656701404 >> v2:172.21.15.112:3300/0 conn(0x7f9a040cfcf0 0x7f9a040cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04071760 con 0x7f9a040ce3c0 2024-04-25T09:59:05.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f99ec001670 con 0x7f9a040ce3c0 2024-04-25T09:59:05.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 >> v1:172.21.15.19:6789/0 conn(0x7f9a040ce3c0 legacy=0x7f9a0415e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 0x7f99f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04154ee0 0x7f99f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f0004d40 0x7f99f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040ce830 con 0x7f99f00039b0 2024-04-25T09:59:05.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f000b9c0 con 0x7f99f0004d40 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f000bb40 con 0x7f9a04154ee0 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a08d95640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 0x7f99f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a03fff640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04154ee0 0x7f99f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a09596640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f0004d40 0x7f99f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a08d95640 1 -- 172.21.15.19:0/656701404 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04154ee0 msgr2=0x7f99f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a08d95640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04154ee0 0x7f99f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a08d95640 1 -- 172.21.15.19:0/656701404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f0004d40 msgr2=0x7f99f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a08d95640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f0004d40 0x7f99f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.475+0000 7f9a08d95640 1 -- 172.21.15.19:0/656701404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f0003520 con 0x7f99f00039b0 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.476+0000 7f9a08d95640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 0x7f99f0003dd0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009fc0 tx=0x7f99f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5ad6c61a2a0591 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.476+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc009c10 con 0x7f99f00039b0 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.476+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc036070 con 0x7f99f00039b0 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.476+0000 7f9a037fe640 1 -- 172.21.15.19:0/656701404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc031880 con 0x7f99f00039b0 2024-04-25T09:59:05.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.476+0000 7f9a0aff9640 1 -- 172.21.15.19:0/656701404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 msgr2=0x7f99f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.476+0000 7f9a0aff9640 1 --2- 172.21.15.19:0/656701404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 0x7f99f0003dd0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009fc0 tx=0x7f99f0005830 comp rx=0 tx=0).stop 2024-04-25T09:59:05.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.476+0000 7f9a09596640 1 -- 172.21.15.19:0/656701404 reap_dead start 2024-04-25T09:59:05.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.477+0000 7f9a0aff9640 1 -- 172.21.15.19:0/656701404 shutdown_connections 2024-04-25T09:59:05.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.477+0000 7f9a0aff9640 1 -- 172.21.15.19:0/656701404 >> 172.21.15.19:0/656701404 conn(0x7f9a0406abe0 msgr2=0x7f9a0406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:05.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.477+0000 7f9a0aff9640 1 -- 172.21.15.19:0/656701404 shutdown_connections 2024-04-25T09:59:05.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.477+0000 7f9a0aff9640 1 -- 172.21.15.19:0/656701404 wait complete. 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.478+0000 7f9a0aff9640 1 Processor -- start 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.478+0000 7f9a0aff9640 1 -- start start 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a0aff9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 0x7f9a04157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a0aff9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a04149ea0 0x7f9a041584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a0aff9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04158a20 0x7f9a0414cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a0aff9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04071230 con 0x7f99f00039b0 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a0aff9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04071d20 con 0x7f9a04149ea0 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a0aff9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041552c0 con 0x7f9a04158a20 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a08d95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 0x7f9a04157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a08d95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 0x7f9a04157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49438/0 (socket says 172.21.15.19:49438) 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a08d95640 1 -- 172.21.15.19:0/3101267117 learned_addr learned my addr 172.21.15.19:0/3101267117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a03fff640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a04149ea0 0x7f9a041584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a09596640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04158a20 0x7f9a0414cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a08d95640 1 -- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04158a20 msgr2=0x7f9a0414cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a08d95640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04158a20 0x7f9a0414cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a08d95640 1 -- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a04149ea0 msgr2=0x7f9a041584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a08d95640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a04149ea0 0x7f9a041584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a08d95640 1 -- 172.21.15.19:0/3101267117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04158ed0 con 0x7f99f00039b0 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a08d95640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 0x7f9a04157fa0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00e230 tx=0x7f99fc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a017fa640 1 -- 172.21.15.19:0/3101267117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc009c10 con 0x7f99f00039b0 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.479+0000 7f9a017fa640 1 -- 172.21.15.19:0/3101267117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc003990 con 0x7f99f00039b0 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.480+0000 7f9a017fa640 1 -- 172.21.15.19:0/3101267117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc003d60 con 0x7f99f00039b0 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.480+0000 7f9a0aff9640 1 -- 172.21.15.19:0/3101267117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a04156560 con 0x7f99f00039b0 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.480+0000 7f9a0aff9640 1 -- 172.21.15.19:0/3101267117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a04156b20 con 0x7f99f00039b0 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.480+0000 7f9a0aff9640 1 -- 172.21.15.19:0/3101267117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a04073990 con 0x7f99f00039b0 2024-04-25T09:59:05.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.481+0000 7f9a017fa640 1 -- 172.21.15.19:0/3101267117 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99fc005d10 con 0x7f99f00039b0 2024-04-25T09:59:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.481+0000 7f9a017fa640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f99e406b500 0x7f99e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.482+0000 7f9a017fa640 1 -- 172.21.15.19:0/3101267117 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f99fc0ab160 con 0x7f99f00039b0 2024-04-25T09:59:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.482+0000 7f9a03fff640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f99e406b500 0x7f99e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.482+0000 7f9a03fff640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f99e406b500 0x7f99e406d9c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9a0414a280 tx=0x7f99f4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.486+0000 7f9a017fa640 1 -- 172.21.15.19:0/3101267117 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99fc07a230 con 0x7f99f00039b0 2024-04-25T09:59:05.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.667+0000 7f9a0aff9640 1 -- 172.21.15.19:0/3101267117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9a0406d360 con 0x7f99f00039b0 2024-04-25T09:59:05.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.667+0000 7f9a017fa640 1 -- 172.21.15.19:0/3101267117 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v91) v1 ==== 84+0+65 (secure 0 0 0) 0x7f99fc079b30 con 0x7f99f00039b0 2024-04-25T09:59:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:05.671 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T09:59:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T09:59:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T09:59:05.672 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:05.672 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T09:59:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 -- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f99e406b500 msgr2=0x7f99e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f99e406b500 0x7f99e406d9c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9a0414a280 tx=0x7f99f4009190 comp rx=0 tx=0).stop 2024-04-25T09:59:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 -- 172.21.15.19:0/3101267117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 msgr2=0x7f9a04157fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 0x7f9a04157fa0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00e230 tx=0x7f99fc009750 comp rx=0 tx=0).stop 2024-04-25T09:59:05.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 -- 172.21.15.19:0/3101267117 shutdown_connections 2024-04-25T09:59:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f99e406b500 0x7f99e406d9c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a04158a20 0x7f9a0414cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a04149ea0 0x7f9a041584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 --2- 172.21.15.19:0/3101267117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f00039b0 0x7f9a04157fa0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 -- 172.21.15.19:0/3101267117 >> 172.21.15.19:0/3101267117 conn(0x7f9a0406abe0 msgr2=0x7f9a040b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 -- 172.21.15.19:0/3101267117 shutdown_connections 2024-04-25T09:59:05.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.669+0000 7f9a0aff9640 1 -- 172.21.15.19:0/3101267117 wait complete. 2024-04-25T09:59:05.684 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:59:05.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:59:05.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.785+0000 7fecfecbd640 1 Processor -- start 2024-04-25T09:59:05.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.785+0000 7fecfecbd640 1 -- start start 2024-04-25T09:59:05.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfecbd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fecf815dc70 0x7fecf80af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfecbd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf806f190 0x7fecf806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfecbd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fecf806f5f0 0x7fecf806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfecbd640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf80b3590 con 0x7fecf806f5f0 2024-04-25T09:59:05.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfecbd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf80b56e0 con 0x7fecf815dc70 2024-04-25T09:59:05.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfecbd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80b37e0 con 0x7fecf806f190 2024-04-25T09:59:05.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfd25a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fecf806f5f0 0x7fecf806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfd25a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fecf806f5f0 0x7fecf806b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49442/0 (socket says 172.21.15.19:49442) 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfd25a640 1 -- 172.21.15.19:0/1747605590 learned_addr learned my addr 172.21.15.19:0/1747605590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecf7fff640 1 --2- 172.21.15.19:0/1747605590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf806f190 0x7fecf806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfca59640 1 --2- 172.21.15.19:0/1747605590 >> v2:172.21.15.112:3300/0 conn(0x7fecf815dc70 0x7fecf80af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfd25a640 1 -- 172.21.15.19:0/1747605590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf806f190 msgr2=0x7fecf806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfd25a640 1 --2- 172.21.15.19:0/1747605590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf806f190 0x7fecf806abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfd25a640 1 -- 172.21.15.19:0/1747605590 >> v2:172.21.15.112:3300/0 conn(0x7fecf815dc70 msgr2=0x7fecf80af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfd25a640 1 --2- 172.21.15.19:0/1747605590 >> v2:172.21.15.112:3300/0 conn(0x7fecf815dc70 0x7fecf80af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecfd25a640 1 -- 172.21.15.19:0/1747605590 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf80b3ac0 con 0x7fecf806f5f0 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.786+0000 7fecf7fff640 1 --2- 172.21.15.19:0/1747605590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf806f190 0x7fecf806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.787+0000 7fecfd25a640 1 --2- 172.21.15.19:0/1747605590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecf806f5f0 0x7fecf806b120 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fecf80625e0 tx=0x7fecf002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0617ee6403bbda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.787+0000 7fecf77fe640 1 -- 172.21.15.19:0/1747605590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0036070 con 0x7fecf806f5f0 2024-04-25T09:59:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.787+0000 7fecf77fe640 1 -- 172.21.15.19:0/1747605590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf0031600 con 0x7fecf806f5f0 2024-04-25T09:59:05.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.787+0000 7fecf77fe640 1 -- 172.21.15.19:0/1747605590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf00319d0 con 0x7fecf806f5f0 2024-04-25T09:59:05.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.788+0000 7fecfecbd640 1 -- 172.21.15.19:0/1747605590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecf806f5f0 msgr2=0x7fecf806b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.788+0000 7fecfecbd640 1 --2- 172.21.15.19:0/1747605590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecf806f5f0 0x7fecf806b120 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fecf80625e0 tx=0x7fecf002a570 comp rx=0 tx=0).stop 2024-04-25T09:59:05.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.788+0000 7fecfecbd640 1 -- 172.21.15.19:0/1747605590 shutdown_connections 2024-04-25T09:59:05.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.788+0000 7fecfecbd640 1 --2- 172.21.15.19:0/1747605590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf806f190 0x7fecf806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.788+0000 7fecfecbd640 1 --2- 172.21.15.19:0/1747605590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecf806f5f0 0x7fecf806b120 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.788+0000 7fecfecbd640 1 --2- 172.21.15.19:0/1747605590 >> v2:172.21.15.112:3300/0 conn(0x7fecf815dc70 0x7fecf80af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.788+0000 7fecfecbd640 1 -- 172.21.15.19:0/1747605590 >> 172.21.15.19:0/1747605590 conn(0x7fecf80b5af0 msgr2=0x7fecf80b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:05.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.788+0000 7fecfecbd640 1 -- 172.21.15.19:0/1747605590 shutdown_connections 2024-04-25T09:59:05.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.788+0000 7fecfecbd640 1 -- 172.21.15.19:0/1747605590 wait complete. 2024-04-25T09:59:05.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.789+0000 7fecfecbd640 1 Processor -- start 2024-04-25T09:59:05.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.789+0000 7fecfecbd640 1 -- start start 2024-04-25T09:59:05.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfecbd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecf806f190 0x7fecf815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfecbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf806f5f0 0x7fecf815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfecbd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf815dc70 0x7fecf8147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfecbd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80701d0 con 0x7fecf806f190 2024-04-25T09:59:05.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfecbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf815e050 con 0x7fecf806f5f0 2024-04-25T09:59:05.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfecbd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf8070350 con 0x7fecf815dc70 2024-04-25T09:59:05.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfca59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecf806f190 0x7fecf815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfca59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecf806f190 0x7fecf815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49452/0 (socket says 172.21.15.19:49452) 2024-04-25T09:59:05.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfca59640 1 -- 172.21.15.19:0/3414783758 learned_addr learned my addr 172.21.15.19:0/3414783758 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:05.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfd25a640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf815dc70 0x7fecf8147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecf7fff640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf806f5f0 0x7fecf815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfca59640 1 -- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf815dc70 msgr2=0x7fecf8147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfca59640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf815dc70 0x7fecf8147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfca59640 1 -- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf806f5f0 msgr2=0x7fecf815c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfca59640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf806f5f0 0x7fecf815c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfca59640 1 -- 172.21.15.19:0/3414783758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf81475d0 con 0x7fecf806f190 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecf7fff640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf806f5f0 0x7fecf815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.790+0000 7fecfca59640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecf806f190 0x7fecf815bcd0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fecec00d620 tx=0x7fecec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.791+0000 7fecf57fa640 1 -- 172.21.15.19:0/3414783758 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec00b620 con 0x7fecf806f190 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.791+0000 7fecf57fa640 1 -- 172.21.15.19:0/3414783758 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec019070 con 0x7fecf806f190 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.791+0000 7fecfecbd640 1 -- 172.21.15.19:0/3414783758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf81478b0 con 0x7fecf806f190 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.791+0000 7fecf57fa640 1 -- 172.21.15.19:0/3414783758 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec00ba00 con 0x7fecf806f190 2024-04-25T09:59:05.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.791+0000 7fecfecbd640 1 -- 172.21.15.19:0/3414783758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf809c8a0 con 0x7fecf806f190 2024-04-25T09:59:05.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.792+0000 7fecfecbd640 1 -- 172.21.15.19:0/3414783758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf806cc80 con 0x7fecf806f190 2024-04-25T09:59:05.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.792+0000 7fecf57fa640 1 -- 172.21.15.19:0/3414783758 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecec00bba0 con 0x7fecf806f190 2024-04-25T09:59:05.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.793+0000 7fecf57fa640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecdc06b500 0x7fecdc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.793+0000 7fecf57fa640 1 -- 172.21.15.19:0/3414783758 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fecec08a270 con 0x7fecf806f190 2024-04-25T09:59:05.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.793+0000 7fecf7fff640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecdc06b500 0x7fecdc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.793+0000 7fecf7fff640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecdc06b500 0x7fecdc06d9c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fecf809e890 tx=0x7fece4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.797+0000 7fecf57fa640 1 -- 172.21.15.19:0/3414783758 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fecec059340 con 0x7fecf806f190 2024-04-25T09:59:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:05.977+0000 7fecfecbd640 1 -- 172.21.15.19:0/3414783758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fecf8073a00 con 0x7fecf806f190 2024-04-25T09:59:06.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.355+0000 7fecf57fa640 1 -- 172.21.15.19:0/3414783758 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v92) v1 ==== 138+0+0 (secure 0 0 0) 0x7fecec058c40 con 0x7fecf806f190 2024-04-25T09:59:06.363 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T09:59:06.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 -- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecdc06b500 msgr2=0x7fecdc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecdc06b500 0x7fecdc06d9c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fecf809e890 tx=0x7fece4008040 comp rx=0 tx=0).stop 2024-04-25T09:59:06.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 -- 172.21.15.19:0/3414783758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecf806f190 msgr2=0x7fecf815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecf806f190 0x7fecf815bcd0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fecec00d620 tx=0x7fecec00dae0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 -- 172.21.15.19:0/3414783758 shutdown_connections 2024-04-25T09:59:06.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fecdc06b500 0x7fecdc06d9c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf815dc70 0x7fecf8147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf806f5f0 0x7fecf815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 --2- 172.21.15.19:0/3414783758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecf806f190 0x7fecf815bcd0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 -- 172.21.15.19:0/3414783758 >> 172.21.15.19:0/3414783758 conn(0x7fecf80b5af0 msgr2=0x7fecf80b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:06.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 -- 172.21.15.19:0/3414783758 shutdown_connections 2024-04-25T09:59:06.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.357+0000 7fecfecbd640 1 -- 172.21.15.19:0/3414783758 wait complete. 2024-04-25T09:59:06.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:59:06.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.482+0000 7fe96a0e1640 1 Processor -- start 2024-04-25T09:59:06.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.482+0000 7fe96a0e1640 1 -- start start 2024-04-25T09:59:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe96a0e1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe96415dc20 0x7fe9640ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe96a0e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9640d32f0 0x7fe9640cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe96a0e1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96415e000 con 0x7fe964161710 2024-04-25T09:59:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe96a0e1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9640d1de0 con 0x7fe96415dc20 2024-04-25T09:59:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe96a0e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe964161c60 con 0x7fe9640d32f0 2024-04-25T09:59:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe9637fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe96415dc20 0x7fe9640ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe9637fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe96415dc20 0x7fe9640ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49464/0 (socket says 172.21.15.19:49464) 2024-04-25T09:59:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe9637fe640 1 -- 172.21.15.19:0/2825070064 learned_addr learned my addr 172.21.15.19:0/2825070064 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe963fff640 1 --2- 172.21.15.19:0/2825070064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9640d32f0 0x7fe9640cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe9637fe640 1 -- 172.21.15.19:0/2825070064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9640d32f0 msgr2=0x7fe9640cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe9637fe640 1 --2- 172.21.15.19:0/2825070064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9640d32f0 0x7fe9640cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe9637fe640 1 -- 172.21.15.19:0/2825070064 >> v1:172.21.15.112:6789/0 conn(0x7fe964161710 legacy=0x7fe96406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.483+0000 7fe9637fe640 1 -- 172.21.15.19:0/2825070064 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe964161f40 con 0x7fe96415dc20 2024-04-25T09:59:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.484+0000 7fe9637fe640 1 --2- 172.21.15.19:0/2825070064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96415dc20 0x7fe9640ce670 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fe954009870 tx=0x7fe95402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76047acb682362a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.484+0000 7fe9627fc640 1 -- 172.21.15.19:0/2825070064 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954037070 con 0x7fe96415dc20 2024-04-25T09:59:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.484+0000 7fe9627fc640 1 -- 172.21.15.19:0/2825070064 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe954032440 con 0x7fe96415dc20 2024-04-25T09:59:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.484+0000 7fe9627fc640 1 -- 172.21.15.19:0/2825070064 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954032810 con 0x7fe96415dc20 2024-04-25T09:59:06.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.485+0000 7fe96a0e1640 1 -- 172.21.15.19:0/2825070064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96415dc20 msgr2=0x7fe9640ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.485+0000 7fe96a0e1640 1 --2- 172.21.15.19:0/2825070064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96415dc20 0x7fe9640ce670 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fe954009870 tx=0x7fe95402a270 comp rx=0 tx=0).stop 2024-04-25T09:59:06.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.485+0000 7fe96a0e1640 1 -- 172.21.15.19:0/2825070064 shutdown_connections 2024-04-25T09:59:06.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.485+0000 7fe96a0e1640 1 --2- 172.21.15.19:0/2825070064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9640d32f0 0x7fe9640cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.485+0000 7fe96a0e1640 1 --2- 172.21.15.19:0/2825070064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96415dc20 0x7fe9640ce670 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.485+0000 7fe96a0e1640 1 -- 172.21.15.19:0/2825070064 >> 172.21.15.19:0/2825070064 conn(0x7fe9640d0420 msgr2=0x7fe9640d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.485+0000 7fe96a0e1640 1 -- 172.21.15.19:0/2825070064 shutdown_connections 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.485+0000 7fe96a0e1640 1 -- 172.21.15.19:0/2825070064 wait complete. 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.486+0000 7fe96a0e1640 1 Processor -- start 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.486+0000 7fe96a0e1640 1 -- start start 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe96a0e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9640d32f0 0x7fe96415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe96a0e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe96415dc20 0x7fe9641e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe96a0e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe964161710 0x7fe9641e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe96a0e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640b5e50 con 0x7fe9640d32f0 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe96a0e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe964162500 con 0x7fe964161710 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe96a0e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640b5fd0 con 0x7fe96415dc20 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe9637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9640d32f0 0x7fe96415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe9637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9640d32f0 0x7fe96415c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49466/0 (socket says 172.21.15.19:49466) 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe9637fe640 1 -- 172.21.15.19:0/759755301 learned_addr learned my addr 172.21.15.19:0/759755301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:06.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe962ffd640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe96415dc20 0x7fe9641e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe963fff640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe964161710 0x7fe9641e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe9637fe640 1 -- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe96415dc20 msgr2=0x7fe9641e82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe9637fe640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe96415dc20 0x7fe9641e82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe9637fe640 1 -- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe964161710 msgr2=0x7fe9641e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe9637fe640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe964161710 0x7fe9641e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.487+0000 7fe9637fe640 1 -- 172.21.15.19:0/759755301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9641e90c0 con 0x7fe9640d32f0 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.488+0000 7fe9637fe640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9640d32f0 0x7fe96415c850 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fe954000c00 tx=0x7fe9540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.488+0000 7fe94bfff640 1 -- 172.21.15.19:0/759755301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954041020 con 0x7fe9640d32f0 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.488+0000 7fe94bfff640 1 -- 172.21.15.19:0/759755301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9540306c0 con 0x7fe9640d32f0 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.488+0000 7fe96a0e1640 1 -- 172.21.15.19:0/759755301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9641f4070 con 0x7fe9640d32f0 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.488+0000 7fe94bfff640 1 -- 172.21.15.19:0/759755301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954037040 con 0x7fe9640d32f0 2024-04-25T09:59:06.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.488+0000 7fe96a0e1640 1 -- 172.21.15.19:0/759755301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96415cd90 con 0x7fe9640d32f0 2024-04-25T09:59:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.489+0000 7fe96a0e1640 1 -- 172.21.15.19:0/759755301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe964073a00 con 0x7fe9640d32f0 2024-04-25T09:59:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.489+0000 7fe94bfff640 1 -- 172.21.15.19:0/759755301 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe954044020 con 0x7fe9640d32f0 2024-04-25T09:59:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.490+0000 7fe94bfff640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe95006b4d0 0x7fe95006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.490+0000 7fe94bfff640 1 -- 172.21.15.19:0/759755301 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fe9540aa840 con 0x7fe9640d32f0 2024-04-25T09:59:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.490+0000 7fe962ffd640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe95006b4d0 0x7fe95006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.490+0000 7fe962ffd640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe95006b4d0 0x7fe95006d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe9640d1980 tx=0x7fe958009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.494+0000 7fe94bfff640 1 -- 172.21.15.19:0/759755301 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe95404b030 con 0x7fe9640d32f0 2024-04-25T09:59:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:06.674+0000 7fe96a0e1640 1 -- 172.21.15.19:0/759755301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe9640d0cc0 con 0x7fe9640d32f0 2024-04-25T09:59:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.360+0000 7fe94bfff640 1 -- 172.21.15.19:0/759755301 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v93) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe9540790b0 con 0x7fe9640d32f0 2024-04-25T09:59:07.364 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T09:59:07.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.363+0000 7fe96a0e1640 1 -- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe95006b4d0 msgr2=0x7fe95006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.363+0000 7fe96a0e1640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe95006b4d0 0x7fe95006d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe9640d1980 tx=0x7fe958009040 comp rx=0 tx=0).stop 2024-04-25T09:59:07.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.363+0000 7fe96a0e1640 1 -- 172.21.15.19:0/759755301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9640d32f0 msgr2=0x7fe96415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.363+0000 7fe96a0e1640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9640d32f0 0x7fe96415c850 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fe954000c00 tx=0x7fe9540046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.363+0000 7fe96a0e1640 1 -- 172.21.15.19:0/759755301 shutdown_connections 2024-04-25T09:59:07.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.363+0000 7fe96a0e1640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe95006b4d0 0x7fe95006d990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.363+0000 7fe96a0e1640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe964161710 0x7fe9641e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.363+0000 7fe96a0e1640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe96415dc20 0x7fe9641e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.363+0000 7fe96a0e1640 1 --2- 172.21.15.19:0/759755301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9640d32f0 0x7fe96415c850 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.363+0000 7fe96a0e1640 1 -- 172.21.15.19:0/759755301 >> 172.21.15.19:0/759755301 conn(0x7fe9640d0420 msgr2=0x7fe9640b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:07.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.364+0000 7fe96a0e1640 1 -- 172.21.15.19:0/759755301 shutdown_connections 2024-04-25T09:59:07.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.364+0000 7fe96a0e1640 1 -- 172.21.15.19:0/759755301 wait complete. 2024-04-25T09:59:07.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:59:07.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.494+0000 7fa27e4e4640 1 Processor -- start 2024-04-25T09:59:07.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.495+0000 7fa27e4e4640 1 -- start start 2024-04-25T09:59:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.495+0000 7fa27e4e4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa278162140 0x7fa2780cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.495+0000 7fa27e4e4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2780cd540 0x7fa2780caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.495+0000 7fa27e4e4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27815e3a0 con 0x7fa2780d0a00 2024-04-25T09:59:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.495+0000 7fa27e4e4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2781625c0 con 0x7fa2780cd540 2024-04-25T09:59:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.495+0000 7fa27e4e4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2780cd9d0 con 0x7fa278162140 2024-04-25T09:59:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.496+0000 7fa27ca81640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2780cd540 0x7fa2780caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.496+0000 7fa27ca81640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2780cd540 0x7fa2780caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49470/0 (socket says 172.21.15.19:49470) 2024-04-25T09:59:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.496+0000 7fa27ca81640 1 -- 172.21.15.19:0/2564859589 learned_addr learned my addr 172.21.15.19:0/2564859589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.496+0000 7fa277fff640 1 --1- 172.21.15.19:0/2564859589 >> v1:172.21.15.19:6789/0 conn(0x7fa2780d0a00 0x7fa2780d0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41484/0 (socket says 172.21.15.19:41484) 2024-04-25T09:59:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.496+0000 7fa2777fe640 1 --2- 172.21.15.19:0/2564859589 >> v2:172.21.15.112:3300/0 conn(0x7fa278162140 0x7fa2780cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.496+0000 7fa27ca81640 1 -- 172.21.15.19:0/2564859589 >> v2:172.21.15.112:3300/0 conn(0x7fa278162140 msgr2=0x7fa2780cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:07.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.496+0000 7fa27ca81640 1 --2- 172.21.15.19:0/2564859589 >> v2:172.21.15.112:3300/0 conn(0x7fa278162140 0x7fa2780cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.496+0000 7fa27ca81640 1 -- 172.21.15.19:0/2564859589 >> v1:172.21.15.19:6789/0 conn(0x7fa2780d0a00 legacy=0x7fa2780d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.496+0000 7fa27ca81640 1 -- 172.21.15.19:0/2564859589 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2780cdc70 con 0x7fa2780cd540 2024-04-25T09:59:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa27ca81640 1 --2- 172.21.15.19:0/2564859589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2780cd540 0x7fa2780caad0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa27806ded0 tx=0x7fa27002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef30db9124bbbc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa276ffd640 1 -- 172.21.15.19:0/2564859589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270036070 con 0x7fa2780cd540 2024-04-25T09:59:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa276ffd640 1 -- 172.21.15.19:0/2564859589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa270031600 con 0x7fa2780cd540 2024-04-25T09:59:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa276ffd640 1 -- 172.21.15.19:0/2564859589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2700319d0 con 0x7fa2780cd540 2024-04-25T09:59:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa27e4e4640 1 -- 172.21.15.19:0/2564859589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2780cd540 msgr2=0x7fa2780caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa27e4e4640 1 --2- 172.21.15.19:0/2564859589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2780cd540 0x7fa2780caad0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa27806ded0 tx=0x7fa27002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa27e4e4640 1 -- 172.21.15.19:0/2564859589 shutdown_connections 2024-04-25T09:59:07.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa27e4e4640 1 --2- 172.21.15.19:0/2564859589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2780cd540 0x7fa2780caad0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa27e4e4640 1 --2- 172.21.15.19:0/2564859589 >> v2:172.21.15.112:3300/0 conn(0x7fa278162140 0x7fa2780cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa27e4e4640 1 -- 172.21.15.19:0/2564859589 >> 172.21.15.19:0/2564859589 conn(0x7fa2780d17c0 msgr2=0x7fa2780d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:07.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa27e4e4640 1 -- 172.21.15.19:0/2564859589 shutdown_connections 2024-04-25T09:59:07.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.497+0000 7fa27e4e4640 1 -- 172.21.15.19:0/2564859589 wait complete. 2024-04-25T09:59:07.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.499+0000 7fa27e4e4640 1 Processor -- start 2024-04-25T09:59:07.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.499+0000 7fa27e4e4640 1 -- start start 2024-04-25T09:59:07.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.499+0000 7fa27e4e4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2780cd540 0x7fa2781eada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:07.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.499+0000 7fa27e4e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2780d0a00 0x7fa2780b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:07.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.499+0000 7fa27e4e4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa278162140 0x7fa2781e7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:07.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.499+0000 7fa27e4e4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa278161940 con 0x7fa2780cd540 2024-04-25T09:59:07.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.499+0000 7fa27e4e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa278161660 con 0x7fa2780d0a00 2024-04-25T09:59:07.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.499+0000 7fa27e4e4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa278161c20 con 0x7fa278162140 2024-04-25T09:59:07.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa277fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2780cd540 0x7fa2781eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:07.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa277fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2780cd540 0x7fa2781eada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49474/0 (socket says 172.21.15.19:49474) 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa277fff640 1 -- 172.21.15.19:0/185806577 learned_addr learned my addr 172.21.15.19:0/185806577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa2777fe640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2780d0a00 0x7fa2780b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa27ca81640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa278162140 0x7fa2781e7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa277fff640 1 -- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa278162140 msgr2=0x7fa2781e7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa277fff640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa278162140 0x7fa2781e7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa277fff640 1 -- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2780d0a00 msgr2=0x7fa2780b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa277fff640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2780d0a00 0x7fa2780b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa277fff640 1 -- 172.21.15.19:0/185806577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2780b2c10 con 0x7fa2780cd540 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa27ca81640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa278162140 0x7fa2781e7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa277fff640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2780cd540 0x7fa2781eada0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fa26000c400 tx=0x7fa26000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa274ff9640 1 -- 172.21.15.19:0/185806577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2600074e0 con 0x7fa2780cd540 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa274ff9640 1 -- 172.21.15.19:0/185806577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa260007680 con 0x7fa2780cd540 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa27e4e4640 1 -- 172.21.15.19:0/185806577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2780b2370 con 0x7fa2780cd540 2024-04-25T09:59:07.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa274ff9640 1 -- 172.21.15.19:0/185806577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260004a00 con 0x7fa2780cd540 2024-04-25T09:59:07.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.500+0000 7fa27e4e4640 1 -- 172.21.15.19:0/185806577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2780b2930 con 0x7fa2780cd540 2024-04-25T09:59:07.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.501+0000 7fa27e4e4640 1 -- 172.21.15.19:0/185806577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa278073a00 con 0x7fa2780cd540 2024-04-25T09:59:07.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.502+0000 7fa274ff9640 1 -- 172.21.15.19:0/185806577 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa26001a020 con 0x7fa2780cd540 2024-04-25T09:59:07.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.502+0000 7fa274ff9640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa25806b4d0 0x7fa25806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:07.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.502+0000 7fa274ff9640 1 -- 172.21.15.19:0/185806577 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fa260089b50 con 0x7fa2780cd540 2024-04-25T09:59:07.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.503+0000 7fa2777fe640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa25806b4d0 0x7fa25806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:07.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.503+0000 7fa2777fe640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa25806b4d0 0x7fa25806d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa268006fd0 tx=0x7fa268008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:07.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.507+0000 7fa274ff9640 1 -- 172.21.15.19:0/185806577 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2600588a0 con 0x7fa2780cd540 2024-04-25T09:59:07.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:07.728+0000 7fa27e4e4640 1 -- 172.21.15.19:0/185806577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa278071d60 con 0x7fa2780cd540 2024-04-25T09:59:07.925 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:59:07.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:59:08.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.362+0000 7fa274ff9640 1 -- 172.21.15.19:0/185806577 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v56) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa2600581a0 con 0x7fa2780cd540 2024-04-25T09:59:08.366 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T09:59:08.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 -- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa25806b4d0 msgr2=0x7fa25806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa25806b4d0 0x7fa25806d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa268006fd0 tx=0x7fa268008040 comp rx=0 tx=0).stop 2024-04-25T09:59:08.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 -- 172.21.15.19:0/185806577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2780cd540 msgr2=0x7fa2781eada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2780cd540 0x7fa2781eada0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fa26000c400 tx=0x7fa26000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 -- 172.21.15.19:0/185806577 shutdown_connections 2024-04-25T09:59:08.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa25806b4d0 0x7fa25806d990 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa278162140 0x7fa2781e7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2780d0a00 0x7fa2780b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 --2- 172.21.15.19:0/185806577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2780cd540 0x7fa2781eada0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 -- 172.21.15.19:0/185806577 >> 172.21.15.19:0/185806577 conn(0x7fa2780d17c0 msgr2=0x7fa278160270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:08.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 -- 172.21.15.19:0/185806577 shutdown_connections 2024-04-25T09:59:08.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.367+0000 7fa27e4e4640 1 -- 172.21.15.19:0/185806577 wait complete. 2024-04-25T09:59:08.407 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.507+0000 7f8f55b34640 1 Processor -- start 2024-04-25T09:59:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.508+0000 7f8f55b34640 1 -- start start 2024-04-25T09:59:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.508+0000 7f8f55b34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5015b320 0x7f8f50164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.508+0000 7f8f55b34640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f50071ed0 con 0x7f8f500cc6c0 2024-04-25T09:59:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.508+0000 7f8f55b34640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5015a840 con 0x7f8f5013d190 2024-04-25T09:59:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.508+0000 7f8f55b34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5013d570 con 0x7f8f5015b320 2024-04-25T09:59:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.509+0000 7f8f4effd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8f500cc6c0 0x7f8f50071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50624/0 (socket says 172.21.15.19:50624) 2024-04-25T09:59:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.509+0000 7f8f4effd640 1 -- 172.21.15.19:0/4220584360 learned_addr learned my addr 172.21.15.19:0/4220584360 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.509+0000 7f8f4ffff640 1 --2- 172.21.15.19:0/4220584360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5015b320 0x7f8f50164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.509+0000 7f8f4e7fc640 1 -- 172.21.15.19:0/4220584360 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792429873 0 0) 0x7f8f50071ed0 con 0x7f8f500cc6c0 2024-04-25T09:59:08.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.509+0000 7f8f4e7fc640 1 -- 172.21.15.19:0/4220584360 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f38003520 con 0x7f8f500cc6c0 2024-04-25T09:59:08.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.509+0000 7f8f4e7fc640 1 -- 172.21.15.19:0/4220584360 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080433770 0 0) 0x7f8f5015a840 con 0x7f8f5013d190 2024-04-25T09:59:08.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.509+0000 7f8f4e7fc640 1 -- 172.21.15.19:0/4220584360 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f50071ed0 con 0x7f8f5013d190 2024-04-25T09:59:08.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.509+0000 7f8f4ffff640 1 -- 172.21.15.19:0/4220584360 >> v1:172.21.15.112:6789/0 conn(0x7f8f5013d190 legacy=0x7f8f500cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.509+0000 7f8f4ffff640 1 -- 172.21.15.19:0/4220584360 >> v1:172.21.15.19:6789/0 conn(0x7f8f500cc6c0 legacy=0x7f8f50071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.509+0000 7f8f4ffff640 1 -- 172.21.15.19:0/4220584360 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5015ab20 con 0x7f8f5015b320 2024-04-25T09:59:08.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.510+0000 7f8f4ffff640 1 --2- 172.21.15.19:0/4220584360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5015b320 0x7f8f50164fb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800aed0 tx=0x7f8f4802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d2463c96c00f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.510+0000 7f8f4e7fc640 1 -- 172.21.15.19:0/4220584360 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48036070 con 0x7f8f5015b320 2024-04-25T09:59:08.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.510+0000 7f8f4e7fc640 1 -- 172.21.15.19:0/4220584360 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f48008640 con 0x7f8f5015b320 2024-04-25T09:59:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.510+0000 7f8f4e7fc640 1 -- 172.21.15.19:0/4220584360 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4802ae60 con 0x7f8f5015b320 2024-04-25T09:59:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.510+0000 7f8f55b34640 1 -- 172.21.15.19:0/4220584360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5015b320 msgr2=0x7f8f50164fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.510+0000 7f8f55b34640 1 --2- 172.21.15.19:0/4220584360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5015b320 0x7f8f50164fb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800aed0 tx=0x7f8f4802a680 comp rx=0 tx=0).stop 2024-04-25T09:59:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.511+0000 7f8f55b34640 1 -- 172.21.15.19:0/4220584360 shutdown_connections 2024-04-25T09:59:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.511+0000 7f8f55b34640 1 --2- 172.21.15.19:0/4220584360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f5015b320 0x7f8f50164fb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.511+0000 7f8f55b34640 1 -- 172.21.15.19:0/4220584360 >> 172.21.15.19:0/4220584360 conn(0x7f8f5006abe0 msgr2=0x7f8f5006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.511+0000 7f8f55b34640 1 -- 172.21.15.19:0/4220584360 shutdown_connections 2024-04-25T09:59:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.511+0000 7f8f55b34640 1 -- 172.21.15.19:0/4220584360 wait complete. 2024-04-25T09:59:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.512+0000 7f8f55b34640 1 Processor -- start 2024-04-25T09:59:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.512+0000 7f8f55b34640 1 -- start start 2024-04-25T09:59:08.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f55b34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f500cc6c0 0x7f8f50141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f55b34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5013d190 0x7f8f50144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f55b34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f5015b320 0x7f8f501595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f55b34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50071ac0 con 0x7f8f5013d190 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f55b34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50071940 con 0x7f8f5015b320 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f55b34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50071c40 con 0x7f8f500cc6c0 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5013d190 0x7f8f50144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5013d190 0x7f8f50144910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40936/0 (socket says 172.21.15.19:40936) 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4effd640 1 -- 172.21.15.19:0/2361014298 learned_addr learned my addr 172.21.15.19:0/2361014298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4ffff640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f5015b320 0x7f8f501595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4f7fe640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f500cc6c0 0x7f8f50141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4effd640 1 -- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f500cc6c0 msgr2=0x7f8f50141de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4effd640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f500cc6c0 0x7f8f50141de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4effd640 1 -- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f5015b320 msgr2=0x7f8f501595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4effd640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f5015b320 0x7f8f501595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4effd640 1 -- 172.21.15.19:0/2361014298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5006d590 con 0x7f8f5013d190 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4f7fe640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f500cc6c0 0x7f8f50141de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f4effd640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5013d190 0x7f8f50144910 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400c3e0 tx=0x7f8f4400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f37fff640 1 -- 172.21.15.19:0/2361014298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44017070 con 0x7f8f5013d190 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.513+0000 7f8f37fff640 1 -- 172.21.15.19:0/2361014298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f44005ce0 con 0x7f8f5013d190 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.514+0000 7f8f55b34640 1 -- 172.21.15.19:0/2361014298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5006eff0 con 0x7f8f5013d190 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.514+0000 7f8f37fff640 1 -- 172.21.15.19:0/2361014298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f440072d0 con 0x7f8f5013d190 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.514+0000 7f8f55b34640 1 -- 172.21.15.19:0/2361014298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f50151960 con 0x7f8f5013d190 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.515+0000 7f8f55b34640 1 -- 172.21.15.19:0/2361014298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f50073990 con 0x7f8f5013d190 2024-04-25T09:59:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.515+0000 7f8f37fff640 1 -- 172.21.15.19:0/2361014298 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f4401a070 con 0x7f8f5013d190 2024-04-25T09:59:08.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.516+0000 7f8f37fff640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f3c06b4d0 0x7f8f3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.516+0000 7f8f37fff640 1 -- 172.21.15.19:0/2361014298 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f440895e0 con 0x7f8f5013d190 2024-04-25T09:59:08.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.516+0000 7f8f4f7fe640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f3c06b4d0 0x7f8f3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.516+0000 7f8f4f7fe640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f3c06b4d0 0x7f8f3c06d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8f40001710 tx=0x7f8f40009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.520+0000 7f8f37fff640 1 -- 172.21.15.19:0/2361014298 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f44057d10 con 0x7f8f5013d190 2024-04-25T09:59:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.703+0000 7f8f55b34640 1 -- 172.21.15.19:0/2361014298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f50151450 con 0x7f8f5013d190 2024-04-25T09:59:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.704+0000 7f8f37fff640 1 -- 172.21.15.19:0/2361014298 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f8f4405bdb0 con 0x7f8f5013d190 2024-04-25T09:59:08.708 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:08.708 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":94,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:07.729311+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:06.008725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:06.705899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 -- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f3c06b4d0 msgr2=0x7f8f3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f3c06b4d0 0x7f8f3c06d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8f40001710 tx=0x7f8f40009040 comp rx=0 tx=0).stop 2024-04-25T09:59:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 -- 172.21.15.19:0/2361014298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5013d190 msgr2=0x7f8f50144910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5013d190 0x7f8f50144910 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400c3e0 tx=0x7f8f4400c8a0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 -- 172.21.15.19:0/2361014298 shutdown_connections 2024-04-25T09:59:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f3c06b4d0 0x7f8f3c06d990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f5015b320 0x7f8f501595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f5013d190 0x7f8f50144910 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 --2- 172.21.15.19:0/2361014298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f500cc6c0 0x7f8f50141de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 -- 172.21.15.19:0/2361014298 >> 172.21.15.19:0/2361014298 conn(0x7f8f5006abe0 msgr2=0x7f8f50154440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:08.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 -- 172.21.15.19:0/2361014298 shutdown_connections 2024-04-25T09:59:08.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.706+0000 7f8f55b34640 1 -- 172.21.15.19:0/2361014298 wait complete. 2024-04-25T09:59:08.721 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:08.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.817+0000 7f2c84ab3640 1 Processor -- start 2024-04-25T09:59:08.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.817+0000 7f2c84ab3640 1 -- start start 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.818+0000 7f2c84ab3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2c8015dbe0 0x7f2c800d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.818+0000 7f2c84ab3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b2cb0 0x7f2c800d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.818+0000 7f2c84ab3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c800b97a0 con 0x7f2c800b92a0 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.818+0000 7f2c84ab3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8015dfc0 con 0x7f2c8015dbe0 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.818+0000 7f2c84ab3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800d1a00 con 0x7f2c800b2cb0 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.818+0000 7f2c7ed9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2c800b92a0 0x7f2c800b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50628/0 (socket says 172.21.15.19:50628) 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.818+0000 7f2c7ed9d640 1 -- 172.21.15.19:0/1575987453 learned_addr learned my addr 172.21.15.19:0/1575987453 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.819+0000 7f2c7dd9b640 1 --2- 172.21.15.19:0/1575987453 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b2cb0 0x7f2c800d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.819+0000 7f2c7e59c640 1 --2- 172.21.15.19:0/1575987453 >> v2:172.21.15.112:3300/0 conn(0x7f2c8015dbe0 0x7f2c800d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.819+0000 7f2c7d59a640 1 -- 172.21.15.19:0/1575987453 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217148498 0 0) 0x7f2c800b97a0 con 0x7f2c800b92a0 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.819+0000 7f2c7d59a640 1 -- 172.21.15.19:0/1575987453 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c6c003520 con 0x7f2c800b92a0 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.819+0000 7f2c7dd9b640 1 -- 172.21.15.19:0/1575987453 >> v2:172.21.15.112:3300/0 conn(0x7f2c8015dbe0 msgr2=0x7f2c800d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.819+0000 7f2c7dd9b640 1 --2- 172.21.15.19:0/1575987453 >> v2:172.21.15.112:3300/0 conn(0x7f2c8015dbe0 0x7f2c800d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.819+0000 7f2c7dd9b640 1 -- 172.21.15.19:0/1575987453 >> v1:172.21.15.19:6789/0 conn(0x7f2c800b92a0 legacy=0x7f2c800b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.819+0000 7f2c7dd9b640 1 -- 172.21.15.19:0/1575987453 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800d1c50 con 0x7f2c800b2cb0 2024-04-25T09:59:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c7dd9b640 1 --2- 172.21.15.19:0/1575987453 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b2cb0 0x7f2c800d0e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2c680097a0 tx=0x7f2c6802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec3ff1a50340706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c7d59a640 1 -- 172.21.15.19:0/1575987453 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68037070 con 0x7f2c800b2cb0 2024-04-25T09:59:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c7d59a640 1 -- 172.21.15.19:0/1575987453 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c68032440 con 0x7f2c800b2cb0 2024-04-25T09:59:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c7d59a640 1 -- 172.21.15.19:0/1575987453 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c680327f0 con 0x7f2c800b2cb0 2024-04-25T09:59:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1575987453 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b2cb0 msgr2=0x7f2c800d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c84ab3640 1 --2- 172.21.15.19:0/1575987453 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b2cb0 0x7f2c800d0e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2c680097a0 tx=0x7f2c6802a180 comp rx=0 tx=0).stop 2024-04-25T09:59:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1575987453 shutdown_connections 2024-04-25T09:59:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c84ab3640 1 --2- 172.21.15.19:0/1575987453 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b2cb0 0x7f2c800d0e50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c84ab3640 1 --2- 172.21.15.19:0/1575987453 >> v2:172.21.15.112:3300/0 conn(0x7f2c8015dbe0 0x7f2c800d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1575987453 >> 172.21.15.19:0/1575987453 conn(0x7f2c80061a20 msgr2=0x7f2c80061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1575987453 shutdown_connections 2024-04-25T09:59:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.820+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1575987453 wait complete. 2024-04-25T09:59:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.822+0000 7f2c84ab3640 1 Processor -- start 2024-04-25T09:59:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.822+0000 7f2c84ab3640 1 -- start start 2024-04-25T09:59:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.822+0000 7f2c84ab3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c800b2cb0 0x7f2c80157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.822+0000 7f2c84ab3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b92a0 0x7f2c800b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.822+0000 7f2c84ab3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8015dbe0 0x7f2c801560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.822+0000 7f2c84ab3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800b6ab0 con 0x7f2c8015dbe0 2024-04-25T09:59:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.822+0000 7f2c84ab3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800d2210 con 0x7f2c800b2cb0 2024-04-25T09:59:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.822+0000 7f2c84ab3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800b6c30 con 0x7f2c800b92a0 2024-04-25T09:59:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8015dbe0 0x7f2c801560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8015dbe0 0x7f2c801560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40940/0 (socket says 172.21.15.19:40940) 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7ed9d640 1 -- 172.21.15.19:0/1193825207 learned_addr learned my addr 172.21.15.19:0/1193825207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7e59c640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c800b2cb0 0x7f2c80157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7dd9b640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b92a0 0x7f2c800b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7ed9d640 1 -- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b92a0 msgr2=0x7f2c800b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7ed9d640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b92a0 0x7f2c800b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7ed9d640 1 -- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c800b2cb0 msgr2=0x7f2c80157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7ed9d640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c800b2cb0 0x7f2c80157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7ed9d640 1 -- 172.21.15.19:0/1193825207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c80156fb0 con 0x7f2c8015dbe0 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7dd9b640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b92a0 0x7f2c800b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7e59c640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c800b2cb0 0x7f2c80157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c7ed9d640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8015dbe0 0x7f2c801560f0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c430 tx=0x7f2c7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c76ffd640 1 -- 172.21.15.19:0/1193825207 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78017070 con 0x7f2c8015dbe0 2024-04-25T09:59:08.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c76ffd640 1 -- 172.21.15.19:0/1193825207 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c78005ce0 con 0x7f2c8015dbe0 2024-04-25T09:59:08.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1193825207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c80157290 con 0x7f2c8015dbe0 2024-04-25T09:59:08.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c76ffd640 1 -- 172.21.15.19:0/1193825207 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78007310 con 0x7f2c8015dbe0 2024-04-25T09:59:08.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.823+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1193825207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c80154370 con 0x7f2c8015dbe0 2024-04-25T09:59:08.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.824+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1193825207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c80073990 con 0x7f2c8015dbe0 2024-04-25T09:59:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.825+0000 7f2c76ffd640 1 -- 172.21.15.19:0/1193825207 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c7801a070 con 0x7f2c8015dbe0 2024-04-25T09:59:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.825+0000 7f2c76ffd640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c6006b4d0 0x7f2c6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.825+0000 7f2c76ffd640 1 -- 172.21.15.19:0/1193825207 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2c78089870 con 0x7f2c8015dbe0 2024-04-25T09:59:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.825+0000 7f2c7e59c640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c6006b4d0 0x7f2c6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.826+0000 7f2c7e59c640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c6006b4d0 0x7f2c6006d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2c70007520 tx=0x7f2c7000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:08.830+0000 7f2c76ffd640 1 -- 172.21.15.19:0/1193825207 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c780584c0 con 0x7f2c8015dbe0 2024-04-25T09:59:09.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.010+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1193825207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c80062010 con 0x7f2c8015dbe0 2024-04-25T09:59:09.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.011+0000 7f2c76ffd640 1 -- 172.21.15.19:0/1193825207 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f2c78057dc0 con 0x7f2c8015dbe0 2024-04-25T09:59:09.020 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:09.020 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":94,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:07.729311+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:06.008725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:06.705899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.013+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c6006b4d0 msgr2=0x7f2c6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.013+0000 7f2c84ab3640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c6006b4d0 0x7f2c6006d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2c70007520 tx=0x7f2c7000b040 comp rx=0 tx=0).stop 2024-04-25T09:59:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.013+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1193825207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8015dbe0 msgr2=0x7f2c801560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.013+0000 7f2c84ab3640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8015dbe0 0x7f2c801560f0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c430 tx=0x7f2c7800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.013+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1193825207 shutdown_connections 2024-04-25T09:59:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.013+0000 7f2c84ab3640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2c6006b4d0 0x7f2c6006d990 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.014+0000 7f2c84ab3640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8015dbe0 0x7f2c801560f0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.014+0000 7f2c84ab3640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c800b92a0 0x7f2c800b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.014+0000 7f2c84ab3640 1 --2- 172.21.15.19:0/1193825207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c800b2cb0 0x7f2c80157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.014+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1193825207 >> 172.21.15.19:0/1193825207 conn(0x7f2c80061a20 msgr2=0x7f2c800b8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.014+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1193825207 shutdown_connections 2024-04-25T09:59:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.014+0000 7f2c84ab3640 1 -- 172.21.15.19:0/1193825207 wait complete. 2024-04-25T09:59:09.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:59:09.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.130+0000 7f43eb7e2640 1 Processor -- start 2024-04-25T09:59:09.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.130+0000 7f43eb7e2640 1 -- start start 2024-04-25T09:59:09.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43eb7e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 0x7f43e40d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43eb7e2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f43e40d0ab0 0x7f43e40b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43eb7e2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e40d2420 con 0x7f43e40d1f30 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43eb7e2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e40d3030 con 0x7f43e40d0ab0 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43eb7e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40d2700 con 0x7f43e415dac0 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43e957e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 0x7f43e40d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43e9d7f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f43e40d0ab0 0x7f43e40b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43e957e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 0x7f43e40d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:40900/0 (socket says 172.21.15.19:40900) 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43e9d7f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f43e40d0ab0 0x7f43e40b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:44914/0 (socket says 172.21.15.19:44914) 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43e8d7d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f43e40d1f30 0x7f43e40d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:59582/0 (socket says 172.21.15.19:59582) 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43e957e640 1 -- 172.21.15.19:0/1358424483 learned_addr learned my addr 172.21.15.19:0/1358424483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43e957e640 1 -- 172.21.15.19:0/1358424483 >> v2:172.21.15.112:3300/0 conn(0x7f43e40d0ab0 msgr2=0x7f43e40b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43e957e640 1 --2- 172.21.15.19:0/1358424483 >> v2:172.21.15.112:3300/0 conn(0x7f43e40d0ab0 0x7f43e40b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43e957e640 1 -- 172.21.15.19:0/1358424483 >> v1:172.21.15.112:6789/0 conn(0x7f43e40d1f30 legacy=0x7f43e40d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43e957e640 1 -- 172.21.15.19:0/1358424483 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e40d29e0 con 0x7f43e415dac0 2024-04-25T09:59:09.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.131+0000 7f43e9d7f640 1 --2- 172.21.15.19:0/1358424483 >> v2:172.21.15.112:3300/0 conn(0x7f43e40d0ab0 0x7f43e40b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.132+0000 7f43e957e640 1 --2- 172.21.15.19:0/1358424483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 0x7f43e40d19f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009e50 tx=0x7f43e002a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e89697be4b4bbe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.132+0000 7f43dbfff640 1 -- 172.21.15.19:0/1358424483 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0036070 con 0x7f43e415dac0 2024-04-25T09:59:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.132+0000 7f43dbfff640 1 -- 172.21.15.19:0/1358424483 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e0031600 con 0x7f43e415dac0 2024-04-25T09:59:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.132+0000 7f43dbfff640 1 -- 172.21.15.19:0/1358424483 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e00319d0 con 0x7f43e415dac0 2024-04-25T09:59:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.133+0000 7f43eb7e2640 1 -- 172.21.15.19:0/1358424483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 msgr2=0x7f43e40d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.133+0000 7f43eb7e2640 1 --2- 172.21.15.19:0/1358424483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 0x7f43e40d19f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009e50 tx=0x7f43e002a7a0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.133+0000 7f43eb7e2640 1 -- 172.21.15.19:0/1358424483 shutdown_connections 2024-04-25T09:59:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.133+0000 7f43eb7e2640 1 --2- 172.21.15.19:0/1358424483 >> v2:172.21.15.112:3300/0 conn(0x7f43e40d0ab0 0x7f43e40b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.133+0000 7f43eb7e2640 1 --2- 172.21.15.19:0/1358424483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 0x7f43e40d19f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.133+0000 7f43eb7e2640 1 -- 172.21.15.19:0/1358424483 >> 172.21.15.19:0/1358424483 conn(0x7f43e4061c30 msgr2=0x7f43e406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.133+0000 7f43eb7e2640 1 -- 172.21.15.19:0/1358424483 shutdown_connections 2024-04-25T09:59:09.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.133+0000 7f43eb7e2640 1 -- 172.21.15.19:0/1358424483 wait complete. 2024-04-25T09:59:09.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.134+0000 7f43eb7e2640 1 Processor -- start 2024-04-25T09:59:09.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.134+0000 7f43eb7e2640 1 -- start start 2024-04-25T09:59:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43eb7e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e40d0ab0 0x7f43e41e7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43eb7e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e40d1f30 0x7f43e414ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43eb7e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 0x7f43e415e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43eb7e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40b97d0 con 0x7f43e40d1f30 2024-04-25T09:59:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43eb7e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e415dea0 con 0x7f43e40d0ab0 2024-04-25T09:59:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43eb7e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40b9950 con 0x7f43e415dac0 2024-04-25T09:59:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43e8d7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e40d1f30 0x7f43e414ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43e8d7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e40d1f30 0x7f43e414ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40944/0 (socket says 172.21.15.19:40944) 2024-04-25T09:59:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43e8d7d640 1 -- 172.21.15.19:0/528407884 learned_addr learned my addr 172.21.15.19:0/528407884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43e957e640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e40d0ab0 0x7f43e41e7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43e9d7f640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 0x7f43e415e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43e8d7d640 1 -- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 msgr2=0x7f43e415e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43e8d7d640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 0x7f43e415e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43e8d7d640 1 -- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e40d0ab0 msgr2=0x7f43e41e7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43e8d7d640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e40d0ab0 0x7f43e41e7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.135+0000 7f43e8d7d640 1 -- 172.21.15.19:0/528407884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e41e7610 con 0x7f43e40d1f30 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.136+0000 7f43e9d7f640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 0x7f43e415e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.136+0000 7f43e8d7d640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e40d1f30 0x7f43e414ac40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f43d400c3c0 tx=0x7f43d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.136+0000 7f43d9ffb640 1 -- 172.21.15.19:0/528407884 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4017070 con 0x7f43e40d1f30 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.136+0000 7f43d9ffb640 1 -- 172.21.15.19:0/528407884 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d4005c70 con 0x7f43e40d1f30 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.136+0000 7f43eb7e2640 1 -- 172.21.15.19:0/528407884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e41e78f0 con 0x7f43e40d1f30 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.136+0000 7f43d9ffb640 1 -- 172.21.15.19:0/528407884 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d40072f0 con 0x7f43e40d1f30 2024-04-25T09:59:09.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.136+0000 7f43eb7e2640 1 -- 172.21.15.19:0/528407884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e41e7eb0 con 0x7f43e40d1f30 2024-04-25T09:59:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.137+0000 7f43eb7e2640 1 -- 172.21.15.19:0/528407884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e4073a00 con 0x7f43e40d1f30 2024-04-25T09:59:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.137+0000 7f43d9ffb640 1 -- 172.21.15.19:0/528407884 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43d401a070 con 0x7f43e40d1f30 2024-04-25T09:59:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.138+0000 7f43d9ffb640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43c806b4d0 0x7f43c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.138+0000 7f43d9ffb640 1 -- 172.21.15.19:0/528407884 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f43d40895b0 con 0x7f43e40d1f30 2024-04-25T09:59:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.138+0000 7f43e957e640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43c806b4d0 0x7f43c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.138+0000 7f43e957e640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43c806b4d0 0x7f43c806d990 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f43e0033000 tx=0x7f43e0009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.142+0000 7f43d9ffb640 1 -- 172.21.15.19:0/528407884 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43d4005e20 con 0x7f43e40d1f30 2024-04-25T09:59:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.358+0000 7f43eb7e2640 1 -- 172.21.15.19:0/528407884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f43e40b8af0 con 0x7f43e40d1f30 2024-04-25T09:59:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.369+0000 7f43d9ffb640 1 -- 172.21.15.19:0/528407884 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7f43d4057ce0 con 0x7f43e40d1f30 2024-04-25T09:59:09.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 -- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43c806b4d0 msgr2=0x7f43c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43c806b4d0 0x7f43c806d990 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f43e0033000 tx=0x7f43e0009000 comp rx=0 tx=0).stop 2024-04-25T09:59:09.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 -- 172.21.15.19:0/528407884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e40d1f30 msgr2=0x7f43e414ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e40d1f30 0x7f43e414ac40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f43d400c3c0 tx=0x7f43d400c880 comp rx=0 tx=0).stop 2024-04-25T09:59:09.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 -- 172.21.15.19:0/528407884 shutdown_connections 2024-04-25T09:59:09.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43c806b4d0 0x7f43c806d990 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43e415dac0 0x7f43e415e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e40d1f30 0x7f43e414ac40 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 --2- 172.21.15.19:0/528407884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e40d0ab0 0x7f43e41e7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 -- 172.21.15.19:0/528407884 >> 172.21.15.19:0/528407884 conn(0x7f43e4061c30 msgr2=0x7f43e40b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:09.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 -- 172.21.15.19:0/528407884 shutdown_connections 2024-04-25T09:59:09.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.372+0000 7f43eb7e2640 1 -- 172.21.15.19:0/528407884 wait complete. 2024-04-25T09:59:09.387 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:59:09.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:59:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.488+0000 7f8ecac83640 1 Processor -- start 2024-04-25T09:59:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.488+0000 7f8ecac83640 1 -- start start 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ecac83640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ec415b7d0 0x7f8ec40cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ecac83640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8ec40b5620 0x7f8ec40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ecac83640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec406cf50 con 0x7f8ec406c2d0 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ecac83640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec40ce530 con 0x7f8ec415b7d0 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ecac83640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec40ce810 con 0x7f8ec40b5620 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ec8a1f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ec415b7d0 0x7f8ec40cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ec8a1f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ec415b7d0 0x7f8ec40cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40960/0 (socket says 172.21.15.19:40960) 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ec8a1f640 1 -- 172.21.15.19:0/904072652 learned_addr learned my addr 172.21.15.19:0/904072652 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ebbfff640 1 --2- 172.21.15.19:0/904072652 >> v2:172.21.15.112:3300/0 conn(0x7f8ec40b5620 0x7f8ec40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ec8a1f640 1 -- 172.21.15.19:0/904072652 >> v2:172.21.15.112:3300/0 conn(0x7f8ec40b5620 msgr2=0x7f8ec40b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ec8a1f640 1 --2- 172.21.15.19:0/904072652 >> v2:172.21.15.112:3300/0 conn(0x7f8ec40b5620 0x7f8ec40b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ec8a1f640 1 -- 172.21.15.19:0/904072652 >> v1:172.21.15.112:6789/0 conn(0x7f8ec406c2d0 legacy=0x7f8ec406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.489+0000 7f8ec8a1f640 1 -- 172.21.15.19:0/904072652 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec40ceaf0 con 0x7f8ec415b7d0 2024-04-25T09:59:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.490+0000 7f8ec8a1f640 1 --2- 172.21.15.19:0/904072652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec415b7d0 0x7f8ec40cf010 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40ce180 tx=0x7f8ec002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3d7e0e5f4a3a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.490+0000 7f8ebb7fe640 1 -- 172.21.15.19:0/904072652 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0037070 con 0x7f8ec415b7d0 2024-04-25T09:59:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.490+0000 7f8ebb7fe640 1 -- 172.21.15.19:0/904072652 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec0032440 con 0x7f8ec415b7d0 2024-04-25T09:59:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.490+0000 7f8ebb7fe640 1 -- 172.21.15.19:0/904072652 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0032810 con 0x7f8ec415b7d0 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.490+0000 7f8ecac83640 1 -- 172.21.15.19:0/904072652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec415b7d0 msgr2=0x7f8ec40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.490+0000 7f8ecac83640 1 --2- 172.21.15.19:0/904072652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec415b7d0 0x7f8ec40cf010 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40ce180 tx=0x7f8ec002a230 comp rx=0 tx=0).stop 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.490+0000 7f8ecac83640 1 -- 172.21.15.19:0/904072652 shutdown_connections 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.490+0000 7f8ecac83640 1 --2- 172.21.15.19:0/904072652 >> v2:172.21.15.112:3300/0 conn(0x7f8ec40b5620 0x7f8ec40b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.490+0000 7f8ecac83640 1 --2- 172.21.15.19:0/904072652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec415b7d0 0x7f8ec40cf010 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.490+0000 7f8ecac83640 1 -- 172.21.15.19:0/904072652 >> 172.21.15.19:0/904072652 conn(0x7f8ec40d0800 msgr2=0x7f8ec40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.491+0000 7f8ecac83640 1 -- 172.21.15.19:0/904072652 shutdown_connections 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.491+0000 7f8ecac83640 1 -- 172.21.15.19:0/904072652 wait complete. 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.492+0000 7f8ecac83640 1 Processor -- start 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.492+0000 7f8ecac83640 1 -- start start 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ecac83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec406c2d0 0x7f8ec4148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ecac83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40b5620 0x7f8ec40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ecac83640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ec415b7d0 0x7f8ec41f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ecac83640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40cf6d0 con 0x7f8ec40b5620 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ecac83640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40cf550 con 0x7f8ec406c2d0 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ecac83640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec415bbb0 con 0x7f8ec415b7d0 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ebbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40b5620 0x7f8ec40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ebbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40b5620 0x7f8ec40cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40966/0 (socket says 172.21.15.19:40966) 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ebbfff640 1 -- 172.21.15.19:0/1506954250 learned_addr learned my addr 172.21.15.19:0/1506954250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ec8a1f640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec406c2d0 0x7f8ec4148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ec9220640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ec415b7d0 0x7f8ec41f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ebbfff640 1 -- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ec415b7d0 msgr2=0x7f8ec41f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ebbfff640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ec415b7d0 0x7f8ec41f6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ebbfff640 1 -- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec406c2d0 msgr2=0x7f8ec4148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ebbfff640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec406c2d0 0x7f8ec4148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ebbfff640 1 -- 172.21.15.19:0/1506954250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec40cc990 con 0x7f8ec40b5620 2024-04-25T09:59:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ebbfff640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40b5620 0x7f8ec40cae70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400d800 tx=0x7f8eb400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8eb97fa640 1 -- 172.21.15.19:0/1506954250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400b6c0 con 0x7f8ec40b5620 2024-04-25T09:59:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8eb97fa640 1 -- 172.21.15.19:0/1506954250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb4019070 con 0x7f8ec40b5620 2024-04-25T09:59:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.493+0000 7f8ecac83640 1 -- 172.21.15.19:0/1506954250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec40ccc10 con 0x7f8ec40b5620 2024-04-25T09:59:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.494+0000 7f8eb97fa640 1 -- 172.21.15.19:0/1506954250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400ba80 con 0x7f8ec40b5620 2024-04-25T09:59:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.494+0000 7f8ecac83640 1 -- 172.21.15.19:0/1506954250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec40cced0 con 0x7f8ec40b5620 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.495+0000 7f8ecac83640 1 -- 172.21.15.19:0/1506954250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec415a730 con 0x7f8ec40b5620 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.497+0000 7f8eb97fa640 1 -- 172.21.15.19:0/1506954250 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8eb400bc20 con 0x7f8ec40b5620 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.497+0000 7f8eb97fa640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8eb006b4d0 0x7f8eb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.497+0000 7f8eb97fa640 1 -- 172.21.15.19:0/1506954250 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8eb408a880 con 0x7f8ec40b5620 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.498+0000 7f8ec8a1f640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8eb006b4d0 0x7f8eb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.498+0000 7f8ec8a1f640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8eb006b4d0 0x7f8eb006d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0034000 tx=0x7f8ec002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.501+0000 7f8eb97fa640 1 -- 172.21.15.19:0/1506954250 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ec40cc990 con 0x7f8ec40b5620 2024-04-25T09:59:09.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:09.719+0000 7f8ecac83640 1 -- 172.21.15.19:0/1506954250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8ec40739e0 con 0x7f8ec40b5620 2024-04-25T09:59:10.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.378+0000 7f8eb97fa640 1 -- 172.21.15.19:0/1506954250 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8eb4059560 con 0x7f8ec40b5620 2024-04-25T09:59:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 -- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8eb006b4d0 msgr2=0x7f8eb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8eb006b4d0 0x7f8eb006d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0034000 tx=0x7f8ec002e000 comp rx=0 tx=0).stop 2024-04-25T09:59:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 -- 172.21.15.19:0/1506954250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40b5620 msgr2=0x7f8ec40cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40b5620 0x7f8ec40cae70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400d800 tx=0x7f8eb400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 -- 172.21.15.19:0/1506954250 shutdown_connections 2024-04-25T09:59:10.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8eb006b4d0 0x7f8eb006d990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ec415b7d0 0x7f8ec41f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40b5620 0x7f8ec40cae70 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 --2- 172.21.15.19:0/1506954250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec406c2d0 0x7f8ec4148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 -- 172.21.15.19:0/1506954250 >> 172.21.15.19:0/1506954250 conn(0x7f8ec40d0800 msgr2=0x7f8ec40d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:10.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 -- 172.21.15.19:0/1506954250 shutdown_connections 2024-04-25T09:59:10.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.381+0000 7f8ecac83640 1 -- 172.21.15.19:0/1506954250 wait complete. 2024-04-25T09:59:10.398 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:59:10.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.502+0000 7f3419e38640 1 Processor -- start 2024-04-25T09:59:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.502+0000 7f3419e38640 1 -- start start 2024-04-25T09:59:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.503+0000 7f3419e38640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f341415dc90 0x7f34141625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.503+0000 7f3419e38640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34140cce50 con 0x7f34140cc8e0 2024-04-25T09:59:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.503+0000 7f3419e38640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3414161af0 con 0x7f34140d1810 2024-04-25T09:59:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.503+0000 7f3419e38640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3414161dd0 con 0x7f341415dc90 2024-04-25T09:59:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.503+0000 7f34137fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f341415dc90 0x7f34141625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.503+0000 7f34137fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f341415dc90 0x7f34141625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40970/0 (socket says 172.21.15.19:40970) 2024-04-25T09:59:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.503+0000 7f34137fe640 1 -- 172.21.15.19:0/3976937654 learned_addr learned my addr 172.21.15.19:0/3976937654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.503+0000 7f3412ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f34140cc8e0 0x7f341415ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50642/0 (socket says 172.21.15.19:50642) 2024-04-25T09:59:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.504+0000 7f34137fe640 1 -- 172.21.15.19:0/3976937654 >> v1:172.21.15.112:6789/0 conn(0x7f34140d1810 legacy=0x7f34140cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:10.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.504+0000 7f34137fe640 1 -- 172.21.15.19:0/3976937654 >> v1:172.21.15.19:6789/0 conn(0x7f34140cc8e0 legacy=0x7f341415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.504+0000 7f34137fe640 1 -- 172.21.15.19:0/3976937654 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34141620b0 con 0x7f341415dc90 2024-04-25T09:59:10.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.504+0000 7f34127fc640 1 -- 172.21.15.19:0/3976937654 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522076308 0 0) 0x7f34140cce50 con 0x7f34140cc8e0 2024-04-25T09:59:10.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.504+0000 7f34137fe640 1 --2- 172.21.15.19:0/3976937654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f341415dc90 0x7f34141625d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3404009870 tx=0x7f340402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b3c602424aefa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.505+0000 7f34127fc640 1 -- 172.21.15.19:0/3976937654 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404037070 con 0x7f341415dc90 2024-04-25T09:59:10.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.505+0000 7f34127fc640 1 -- 172.21.15.19:0/3976937654 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3404032440 con 0x7f341415dc90 2024-04-25T09:59:10.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.505+0000 7f34127fc640 1 -- 172.21.15.19:0/3976937654 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404032810 con 0x7f341415dc90 2024-04-25T09:59:10.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.505+0000 7f3419e38640 1 -- 172.21.15.19:0/3976937654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f341415dc90 msgr2=0x7f34141625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.505+0000 7f3419e38640 1 --2- 172.21.15.19:0/3976937654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f341415dc90 0x7f34141625d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3404009870 tx=0x7f340402a210 comp rx=0 tx=0).stop 2024-04-25T09:59:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.505+0000 7f3419e38640 1 -- 172.21.15.19:0/3976937654 shutdown_connections 2024-04-25T09:59:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.505+0000 7f3419e38640 1 --2- 172.21.15.19:0/3976937654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f341415dc90 0x7f34141625d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.505+0000 7f3419e38640 1 -- 172.21.15.19:0/3976937654 >> 172.21.15.19:0/3976937654 conn(0x7f341406f590 msgr2=0x7f341406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.505+0000 7f3419e38640 1 -- 172.21.15.19:0/3976937654 shutdown_connections 2024-04-25T09:59:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.505+0000 7f3419e38640 1 -- 172.21.15.19:0/3976937654 wait complete. 2024-04-25T09:59:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.506+0000 7f3419e38640 1 Processor -- start 2024-04-25T09:59:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.507+0000 7f3419e38640 1 -- start start 2024-04-25T09:59:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.507+0000 7f3419e38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34140cc8e0 0x7f34141f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.507+0000 7f3419e38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34140d1810 0x7f34140c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.507+0000 7f3419e38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f341415dc90 0x7f34140c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.507+0000 7f3419e38640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341415e1f0 con 0x7f341415dc90 2024-04-25T09:59:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.507+0000 7f3419e38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341415e070 con 0x7f34140cc8e0 2024-04-25T09:59:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.507+0000 7f3419e38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34140d1c80 con 0x7f34140d1810 2024-04-25T09:59:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.507+0000 7f3413fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f341415dc90 0x7f34140c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f3413fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f341415dc90 0x7f34140c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40982/0 (socket says 172.21.15.19:40982) 2024-04-25T09:59:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f3413fff640 1 -- 172.21.15.19:0/190204849 learned_addr learned my addr 172.21.15.19:0/190204849 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f34137fe640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34140cc8e0 0x7f34141f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f3413fff640 1 -- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34140d1810 msgr2=0x7f34140c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f3413fff640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34140d1810 0x7f34140c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f3413fff640 1 -- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34140cc8e0 msgr2=0x7f34141f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f3413fff640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34140cc8e0 0x7f34141f7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f3413fff640 1 -- 172.21.15.19:0/190204849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34141f8090 con 0x7f341415dc90 2024-04-25T09:59:10.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f3413fff640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f341415dc90 0x7f34140c7310 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c780 tx=0x7f340c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f33fbfff640 1 -- 172.21.15.19:0/190204849 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c004b10 con 0x7f341415dc90 2024-04-25T09:59:10.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f33fbfff640 1 -- 172.21.15.19:0/190204849 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c004cb0 con 0x7f341415dc90 2024-04-25T09:59:10.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f33fbfff640 1 -- 172.21.15.19:0/190204849 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c004e60 con 0x7f341415dc90 2024-04-25T09:59:10.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f3419e38640 1 -- 172.21.15.19:0/190204849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34141f8370 con 0x7f341415dc90 2024-04-25T09:59:10.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.508+0000 7f3419e38640 1 -- 172.21.15.19:0/190204849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34140c7850 con 0x7f341415dc90 2024-04-25T09:59:10.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.509+0000 7f3419e38640 1 -- 172.21.15.19:0/190204849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3414073990 con 0x7f341415dc90 2024-04-25T09:59:10.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.510+0000 7f33fbfff640 1 -- 172.21.15.19:0/190204849 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f340c00aa90 con 0x7f341415dc90 2024-04-25T09:59:10.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.510+0000 7f33fbfff640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 0x7f340006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.510+0000 7f33fbfff640 1 -- 172.21.15.19:0/190204849 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f340c08a6c0 con 0x7f341415dc90 2024-04-25T09:59:10.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.510+0000 7f34137fe640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 0x7f340006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.511+0000 7f34137fe640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 0x7f340006d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3404034000 tx=0x7f340402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.514+0000 7f33fbfff640 1 -- 172.21.15.19:0/190204849 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f340c020070 con 0x7f341415dc90 2024-04-25T09:59:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.735+0000 7f3419e38640 1 -- 172.21.15.19:0/190204849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34140b0c40 con 0x7f341415dc90 2024-04-25T09:59:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.736+0000 7f33fbfff640 1 -- 172.21.15.19:0/190204849 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f340c0102e0 con 0x7f341415dc90 2024-04-25T09:59:10.740 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:10.740 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3431952054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3431952054},{"type":"v1","addr":"172.21.15.112:6836","nonce":3431952054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2758109099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2758109099},{"type":"v1","addr":"172.21.15.112:6837","nonce":2758109099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7590,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3063446116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3063446116},{"type":"v1","addr":"172.21.15.19:6834","nonce":3063446116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:07.729627+0000","modified":"2024-04-25T09:59:09.720756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3589967477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3589967477},{"type":"v1","addr":"172.21.15.19:6835","nonce":3589967477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-04-25T09:59:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 -- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 msgr2=0x7f340006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 0x7f340006d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3404034000 tx=0x7f340402e000 comp rx=0 tx=0).stop 2024-04-25T09:59:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 -- 172.21.15.19:0/190204849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f341415dc90 msgr2=0x7f34140c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f341415dc90 0x7f34140c7310 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c780 tx=0x7f340c00cc40 comp rx=0 tx=0).stop 2024-04-25T09:59:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 -- 172.21.15.19:0/190204849 shutdown_connections 2024-04-25T09:59:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f340006b4d0 0x7f340006d990 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f341415dc90 0x7f34140c7310 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34140d1810 0x7f34140c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 --2- 172.21.15.19:0/190204849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34140cc8e0 0x7f34141f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 -- 172.21.15.19:0/190204849 >> 172.21.15.19:0/190204849 conn(0x7f341406f590 msgr2=0x7f341406f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 -- 172.21.15.19:0/190204849 shutdown_connections 2024-04-25T09:59:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.738+0000 7f3419e38640 1 -- 172.21.15.19:0/190204849 wait complete. 2024-04-25T09:59:10.753 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:10.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.849+0000 7f8b41a6f640 1 Processor -- start 2024-04-25T09:59:10.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.850+0000 7f8b41a6f640 1 -- start start 2024-04-25T09:59:10.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.850+0000 7f8b41a6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b3c15db50 0x7f8b3c0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.850+0000 7f8b41a6f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8b3c0c7050 0x7f8b3c0cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.850+0000 7f8b41a6f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b3c0cd5b0 0x7f8b3c0cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.850+0000 7f8b41a6f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c15df30 con 0x7f8b3c0cd5b0 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.850+0000 7f8b41a6f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c1625d0 con 0x7f8b3c0c7050 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.850+0000 7f8b41a6f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0cc2b0 con 0x7f8b3c15db50 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.851+0000 7f8b3bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b3c0cd5b0 0x7f8b3c0cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.851+0000 7f8b3bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b3c0cd5b0 0x7f8b3c0cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40996/0 (socket says 172.21.15.19:40996) 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.851+0000 7f8b3bfff640 1 -- 172.21.15.19:0/1749905587 learned_addr learned my addr 172.21.15.19:0/1749905587 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.851+0000 7f8b3affd640 1 --2- 172.21.15.19:0/1749905587 >> v2:172.21.15.112:3300/0 conn(0x7f8b3c0c7050 0x7f8b3c0cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.851+0000 7f8b3b7fe640 1 --2- 172.21.15.19:0/1749905587 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b3c15db50 0x7f8b3c0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.851+0000 7f8b3bfff640 1 -- 172.21.15.19:0/1749905587 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b3c15db50 msgr2=0x7f8b3c0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.851+0000 7f8b3bfff640 1 --2- 172.21.15.19:0/1749905587 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b3c15db50 0x7f8b3c0c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.851+0000 7f8b3bfff640 1 -- 172.21.15.19:0/1749905587 >> v2:172.21.15.112:3300/0 conn(0x7f8b3c0c7050 msgr2=0x7f8b3c0cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.851+0000 7f8b3bfff640 1 --2- 172.21.15.19:0/1749905587 >> v2:172.21.15.112:3300/0 conn(0x7f8b3c0c7050 0x7f8b3c0cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.851+0000 7f8b3bfff640 1 -- 172.21.15.19:0/1749905587 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c0cc590 con 0x7f8b3c0cd5b0 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.851+0000 7f8b3b7fe640 1 --2- 172.21.15.19:0/1749905587 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b3c15db50 0x7f8b3c0c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b3bfff640 1 --2- 172.21.15.19:0/1749905587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0cd5b0 0x7f8b3c0cfb30 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009e50 tx=0x7f8b3402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feeec0f55dc53f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b3a7fc640 1 -- 172.21.15.19:0/1749905587 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34036070 con 0x7f8b3c0cd5b0 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b3a7fc640 1 -- 172.21.15.19:0/1749905587 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b34031600 con 0x7f8b3c0cd5b0 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b3a7fc640 1 -- 172.21.15.19:0/1749905587 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b340319d0 con 0x7f8b3c0cd5b0 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b41a6f640 1 -- 172.21.15.19:0/1749905587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0cd5b0 msgr2=0x7f8b3c0cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b41a6f640 1 --2- 172.21.15.19:0/1749905587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0cd5b0 0x7f8b3c0cfb30 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009e50 tx=0x7f8b3402a630 comp rx=0 tx=0).stop 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b41a6f640 1 -- 172.21.15.19:0/1749905587 shutdown_connections 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b41a6f640 1 --2- 172.21.15.19:0/1749905587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0cd5b0 0x7f8b3c0cfb30 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b41a6f640 1 --2- 172.21.15.19:0/1749905587 >> v2:172.21.15.112:3300/0 conn(0x7f8b3c0c7050 0x7f8b3c0cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b41a6f640 1 --2- 172.21.15.19:0/1749905587 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b3c15db50 0x7f8b3c0c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b41a6f640 1 -- 172.21.15.19:0/1749905587 >> 172.21.15.19:0/1749905587 conn(0x7f8b3c0d1800 msgr2=0x7f8b3c0d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b41a6f640 1 -- 172.21.15.19:0/1749905587 shutdown_connections 2024-04-25T09:59:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.852+0000 7f8b41a6f640 1 -- 172.21.15.19:0/1749905587 wait complete. 2024-04-25T09:59:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.853+0000 7f8b41a6f640 1 Processor -- start 2024-04-25T09:59:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.854+0000 7f8b41a6f640 1 -- start start 2024-04-25T09:59:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.854+0000 7f8b41a6f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0c7050 0x7f8b3c1e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.854+0000 7f8b41a6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b3c0cd5b0 0x7f8b3c0b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.854+0000 7f8b41a6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b3c15db50 0x7f8b3c158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.854+0000 7f8b41a6f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0d0070 con 0x7f8b3c0c7050 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.854+0000 7f8b41a6f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0ccb50 con 0x7f8b3c15db50 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.854+0000 7f8b41a6f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0d01f0 con 0x7f8b3c0cd5b0 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0c7050 0x7f8b3c1e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0c7050 0x7f8b3c1e74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41000/0 (socket says 172.21.15.19:41000) 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3b7fe640 1 -- 172.21.15.19:0/258596606 learned_addr learned my addr 172.21.15.19:0/258596606 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3bfff640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b3c15db50 0x7f8b3c158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3affd640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b3c0cd5b0 0x7f8b3c0b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3b7fe640 1 -- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b3c0cd5b0 msgr2=0x7f8b3c0b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3b7fe640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b3c0cd5b0 0x7f8b3c0b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3b7fe640 1 -- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b3c15db50 msgr2=0x7f8b3c158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3b7fe640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b3c15db50 0x7f8b3c158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3b7fe640 1 -- 172.21.15.19:0/258596606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c155ca0 con 0x7f8b3c0c7050 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3bfff640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b3c15db50 0x7f8b3c158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b3b7fe640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0c7050 0x7f8b3c1e74b0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c7f0 tx=0x7f8b2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b23fff640 1 -- 172.21.15.19:0/258596606 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c015070 con 0x7f8b3c0c7050 2024-04-25T09:59:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b23fff640 1 -- 172.21.15.19:0/258596606 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c007820 con 0x7f8b3c0c7050 2024-04-25T09:59:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b41a6f640 1 -- 172.21.15.19:0/258596606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3c1e7b20 con 0x7f8b3c0c7050 2024-04-25T09:59:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b23fff640 1 -- 172.21.15.19:0/258596606 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c007bd0 con 0x7f8b3c0c7050 2024-04-25T09:59:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.855+0000 7f8b41a6f640 1 -- 172.21.15.19:0/258596606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c1e80e0 con 0x7f8b3c0c7050 2024-04-25T09:59:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.856+0000 7f8b41a6f640 1 -- 172.21.15.19:0/258596606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3c1579c0 con 0x7f8b3c0c7050 2024-04-25T09:59:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.857+0000 7f8b23fff640 1 -- 172.21.15.19:0/258596606 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b2c018070 con 0x7f8b3c0c7050 2024-04-25T09:59:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.857+0000 7f8b23fff640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.857+0000 7f8b23fff640 1 -- 172.21.15.19:0/258596606 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8b2c08a370 con 0x7f8b3c0c7050 2024-04-25T09:59:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.858+0000 7f8b3affd640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.858+0000 7f8b3affd640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b2806b4d0 0x7f8b2806d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c15a3a0 tx=0x7f8b30008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:10.862+0000 7f8b23fff640 1 -- 172.21.15.19:0/258596606 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b3c1579c0 con 0x7f8b3c0c7050 2024-04-25T09:59:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.044+0000 7f8b41a6f640 1 -- 172.21.15.19:0/258596606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b3c0d0c60 con 0x7f8b3c0c7050 2024-04-25T09:59:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.045+0000 7f8b23fff640 1 -- 172.21.15.19:0/258596606 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f8b2c0590f0 con 0x7f8b3c0c7050 2024-04-25T09:59:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:11.049 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":95,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:09.369855+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:06.008725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:06.705899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 -- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b2806b4d0 msgr2=0x7f8b2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b2806b4d0 0x7f8b2806d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c15a3a0 tx=0x7f8b30008040 comp rx=0 tx=0).stop 2024-04-25T09:59:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 -- 172.21.15.19:0/258596606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0c7050 msgr2=0x7f8b3c1e74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0c7050 0x7f8b3c1e74b0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c7f0 tx=0x7f8b2c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 -- 172.21.15.19:0/258596606 shutdown_connections 2024-04-25T09:59:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b3c15db50 0x7f8b3c158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b3c0cd5b0 0x7f8b3c0b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 --2- 172.21.15.19:0/258596606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0c7050 0x7f8b3c1e74b0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 -- 172.21.15.19:0/258596606 >> 172.21.15.19:0/258596606 conn(0x7f8b3c0d1800 msgr2=0x7f8b3c0cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 -- 172.21.15.19:0/258596606 shutdown_connections 2024-04-25T09:59:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.047+0000 7f8b41a6f640 1 -- 172.21.15.19:0/258596606 wait complete. 2024-04-25T09:59:11.062 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:59:11.062 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:59:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.160+0000 7f84dcc8f640 1 Processor -- start 2024-04-25T09:59:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.160+0000 7f84dcc8f640 1 -- start start 2024-04-25T09:59:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.161+0000 7f84dcc8f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 0x7f84d80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.161+0000 7f84dcc8f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f84d80b2cb0 0x7f84d80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.161+0000 7f84dcc8f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d80b97a0 con 0x7f84d80b92a0 2024-04-25T09:59:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.161+0000 7f84dcc8f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d815dfc0 con 0x7f84d80b2cb0 2024-04-25T09:59:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.161+0000 7f84dcc8f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80d1a00 con 0x7f84d815dbe0 2024-04-25T09:59:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.161+0000 7f84d659c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 0x7f84d80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.161+0000 7f84d659c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 0x7f84d80d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:40956/0 (socket says 172.21.15.19:40956) 2024-04-25T09:59:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.161+0000 7f84d659c640 1 -- 172.21.15.19:0/3429457554 learned_addr learned my addr 172.21.15.19:0/3429457554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.161+0000 7f84cffff640 1 --2- 172.21.15.19:0/3429457554 >> v2:172.21.15.112:3300/0 conn(0x7f84d80b2cb0 0x7f84d80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.162+0000 7f84d659c640 1 -- 172.21.15.19:0/3429457554 >> v2:172.21.15.112:3300/0 conn(0x7f84d80b2cb0 msgr2=0x7f84d80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.162+0000 7f84d659c640 1 --2- 172.21.15.19:0/3429457554 >> v2:172.21.15.112:3300/0 conn(0x7f84d80b2cb0 0x7f84d80d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.162+0000 7f84d659c640 1 -- 172.21.15.19:0/3429457554 >> v1:172.21.15.112:6789/0 conn(0x7f84d80b92a0 legacy=0x7f84d80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.162+0000 7f84d659c640 1 -- 172.21.15.19:0/3429457554 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d80d1c50 con 0x7f84d815dbe0 2024-04-25T09:59:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.162+0000 7f84cffff640 1 --2- 172.21.15.19:0/3429457554 >> v2:172.21.15.112:3300/0 conn(0x7f84d80b2cb0 0x7f84d80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.162+0000 7f84d659c640 1 --2- 172.21.15.19:0/3429457554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 0x7f84d80d0910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009870 tx=0x7f84c802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e23e46e944742d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.163+0000 7f84d5d9b640 1 -- 172.21.15.19:0/3429457554 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8037070 con 0x7f84d815dbe0 2024-04-25T09:59:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.163+0000 7f84d5d9b640 1 -- 172.21.15.19:0/3429457554 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c8032440 con 0x7f84d815dbe0 2024-04-25T09:59:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.163+0000 7f84d5d9b640 1 -- 172.21.15.19:0/3429457554 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8032810 con 0x7f84d815dbe0 2024-04-25T09:59:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.163+0000 7f84dcc8f640 1 -- 172.21.15.19:0/3429457554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 msgr2=0x7f84d80d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.163+0000 7f84dcc8f640 1 --2- 172.21.15.19:0/3429457554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 0x7f84d80d0910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009870 tx=0x7f84c802a270 comp rx=0 tx=0).stop 2024-04-25T09:59:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.163+0000 7f84dcc8f640 1 -- 172.21.15.19:0/3429457554 shutdown_connections 2024-04-25T09:59:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.163+0000 7f84dcc8f640 1 --2- 172.21.15.19:0/3429457554 >> v2:172.21.15.112:3300/0 conn(0x7f84d80b2cb0 0x7f84d80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.163+0000 7f84dcc8f640 1 --2- 172.21.15.19:0/3429457554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 0x7f84d80d0910 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.163+0000 7f84dcc8f640 1 -- 172.21.15.19:0/3429457554 >> 172.21.15.19:0/3429457554 conn(0x7f84d8061a20 msgr2=0x7f84d8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.163+0000 7f84dcc8f640 1 -- 172.21.15.19:0/3429457554 shutdown_connections 2024-04-25T09:59:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.163+0000 7f84dcc8f640 1 -- 172.21.15.19:0/3429457554 wait complete. 2024-04-25T09:59:11.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.165+0000 7f84dcc8f640 1 Processor -- start 2024-04-25T09:59:11.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.165+0000 7f84dcc8f640 1 -- start start 2024-04-25T09:59:11.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.165+0000 7f84dcc8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84d80b2cb0 0x7f84d81605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.165+0000 7f84dcc8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d80b92a0 0x7f84d8142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.165+0000 7f84dcc8f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 0x7f84d80c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.165+0000 7f84dcc8f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80b6ab0 con 0x7f84d80b92a0 2024-04-25T09:59:11.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.165+0000 7f84dcc8f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80d2210 con 0x7f84d80b2cb0 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.165+0000 7f84dcc8f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80b6c30 con 0x7f84d815dbe0 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.165+0000 7f84cffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d80b92a0 0x7f84d8142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84cffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d80b92a0 0x7f84d8142520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41008/0 (socket says 172.21.15.19:41008) 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84cffff640 1 -- 172.21.15.19:0/1621614967 learned_addr learned my addr 172.21.15.19:0/1621614967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84d659c640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84d80b2cb0 0x7f84d81605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84d6d9d640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 0x7f84d80c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84cffff640 1 -- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 msgr2=0x7f84d80c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84cffff640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 0x7f84d80c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84cffff640 1 -- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84d80b2cb0 msgr2=0x7f84d81605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84cffff640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84d80b2cb0 0x7f84d81605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84cffff640 1 -- 172.21.15.19:0/1621614967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d81f7210 con 0x7f84d80b92a0 2024-04-25T09:59:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84cffff640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d80b92a0 0x7f84d8142520 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f84c000ebe0 tx=0x7f84c000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84ceffd640 1 -- 172.21.15.19:0/1621614967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0017070 con 0x7f84d80b92a0 2024-04-25T09:59:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84ceffd640 1 -- 172.21.15.19:0/1621614967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c000cc30 con 0x7f84d80b92a0 2024-04-25T09:59:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84dcc8f640 1 -- 172.21.15.19:0/1621614967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d81f74f0 con 0x7f84d80b92a0 2024-04-25T09:59:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84ceffd640 1 -- 172.21.15.19:0/1621614967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0013470 con 0x7f84d80b92a0 2024-04-25T09:59:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.166+0000 7f84dcc8f640 1 -- 172.21.15.19:0/1621614967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d81f7ab0 con 0x7f84d80b92a0 2024-04-25T09:59:11.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.167+0000 7f84dcc8f640 1 -- 172.21.15.19:0/1621614967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d8073990 con 0x7f84d80b92a0 2024-04-25T09:59:11.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.168+0000 7f84ceffd640 1 -- 172.21.15.19:0/1621614967 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84c001a070 con 0x7f84d80b92a0 2024-04-25T09:59:11.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.168+0000 7f84ceffd640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84b806b4d0 0x7f84b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.168+0000 7f84ceffd640 1 -- 172.21.15.19:0/1621614967 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f84c008a5e0 con 0x7f84d80b92a0 2024-04-25T09:59:11.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.168+0000 7f84d659c640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84b806b4d0 0x7f84b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.169+0000 7f84d659c640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84b806b4d0 0x7f84b806d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f84c8034000 tx=0x7f84c802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.172+0000 7f84ceffd640 1 -- 172.21.15.19:0/1621614967 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84c0058c80 con 0x7f84d80b92a0 2024-04-25T09:59:11.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.349+0000 7f84dcc8f640 1 -- 172.21.15.19:0/1621614967 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84d80d3da0 con 0x7f84b806b4d0 2024-04-25T09:59:11.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.355+0000 7f84ceffd640 1 -- 172.21.15.19:0/1621614967 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252005 (secure 0 0 0) 0x7f84d80d3da0 con 0x7f84b806b4d0 2024-04-25T09:59:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:59:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.358+0000 7f84dcc8f640 1 -- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84b806b4d0 msgr2=0x7f84b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.358+0000 7f84dcc8f640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84b806b4d0 0x7f84b806d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f84c8034000 tx=0x7f84c802e000 comp rx=0 tx=0).stop 2024-04-25T09:59:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.358+0000 7f84dcc8f640 1 -- 172.21.15.19:0/1621614967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d80b92a0 msgr2=0x7f84d8142520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.358+0000 7f84dcc8f640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d80b92a0 0x7f84d8142520 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f84c000ebe0 tx=0x7f84c000c520 comp rx=0 tx=0).stop 2024-04-25T09:59:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.358+0000 7f84dcc8f640 1 -- 172.21.15.19:0/1621614967 shutdown_connections 2024-04-25T09:59:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.358+0000 7f84dcc8f640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f84b806b4d0 0x7f84b806d990 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.359+0000 7f84dcc8f640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84d815dbe0 0x7f84d80c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.359+0000 7f84dcc8f640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d80b92a0 0x7f84d8142520 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.359+0000 7f84dcc8f640 1 --2- 172.21.15.19:0/1621614967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84d80b2cb0 0x7f84d81605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.359+0000 7f84dcc8f640 1 -- 172.21.15.19:0/1621614967 >> 172.21.15.19:0/1621614967 conn(0x7f84d8061a20 msgr2=0x7f84d815af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.359+0000 7f84dcc8f640 1 -- 172.21.15.19:0/1621614967 shutdown_connections 2024-04-25T09:59:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.359+0000 7f84dcc8f640 1 -- 172.21.15.19:0/1621614967 wait complete. 2024-04-25T09:59:11.374 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-04-25T09:59:10.994106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379036,"kb_used_data":10208,"kb_used_omap":12,"kb_used_meta":368691,"kb_avail":754595684,"statfs":{"total":773094113280,"available":772705980416,"internally_reserved":0,"allocated":10452992,"data_stored":5468995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":377540146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.632253"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.399311+0000","last_change":"2024-04-25T09:59:08.398299+0000","last_active":"2024-04-25T09:59:08.399311+0000","last_peered":"2024-04-25T09:59:08.399311+0000","last_clean":"2024-04-25T09:59:08.399311+0000","last_became_active":"2024-04-25T09:59:08.398030+0000","last_became_peered":"2024-04-25T09:59:08.398030+0000","last_unstale":"2024-04-25T09:59:08.399311+0000","last_undegraded":"2024-04-25T09:59:08.399311+0000","last_fullsized":"2024-04-25T09:59:08.399311+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:53.748733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.380727+0000","last_change":"2024-04-25T09:59:08.376005+0000","last_active":"2024-04-25T09:59:08.380727+0000","last_peered":"2024-04-25T09:59:08.380727+0000","last_clean":"2024-04-25T09:59:08.380727+0000","last_became_active":"2024-04-25T09:59:08.375340+0000","last_became_peered":"2024-04-25T09:59:08.375340+0000","last_unstale":"2024-04-25T09:59:08.380727+0000","last_undegraded":"2024-04-25T09:59:08.380727+0000","last_fullsized":"2024-04-25T09:59:08.380727+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:16.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.393238+0000","last_change":"2024-04-25T09:59:08.391855+0000","last_active":"2024-04-25T09:59:08.393238+0000","last_peered":"2024-04-25T09:59:08.393238+0000","last_clean":"2024-04-25T09:59:08.393238+0000","last_became_active":"2024-04-25T09:59:08.391575+0000","last_became_peered":"2024-04-25T09:59:08.391575+0000","last_unstale":"2024-04-25T09:59:08.393238+0000","last_undegraded":"2024-04-25T09:59:08.393238+0000","last_fullsized":"2024-04-25T09:59:08.393238+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:19.379486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.385140+0000","last_change":"2024-04-25T09:59:08.380075+0000","last_active":"2024-04-25T09:59:08.385140+0000","last_peered":"2024-04-25T09:59:08.385140+0000","last_clean":"2024-04-25T09:59:08.385140+0000","last_became_active":"2024-04-25T09:59:08.378283+0000","last_became_peered":"2024-04-25T09:59:08.378283+0000","last_unstale":"2024-04-25T09:59:08.385140+0000","last_undegraded":"2024-04-25T09:59:08.385140+0000","last_fullsized":"2024-04-25T09:59:08.385140+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:31.394397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387669+0000","last_change":"2024-04-25T09:59:08.382237+0000","last_active":"2024-04-25T09:59:08.387669+0000","last_peered":"2024-04-25T09:59:08.387669+0000","last_clean":"2024-04-25T09:59:08.387669+0000","last_became_active":"2024-04-25T09:59:08.381844+0000","last_became_peered":"2024-04-25T09:59:08.381844+0000","last_unstale":"2024-04-25T09:59:08.387669+0000","last_undegraded":"2024-04-25T09:59:08.387669+0000","last_fullsized":"2024-04-25T09:59:08.387669+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:08.149260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.391644+0000","last_change":"2024-04-25T09:59:08.380960+0000","last_active":"2024-04-25T09:59:08.391644+0000","last_peered":"2024-04-25T09:59:08.391644+0000","last_clean":"2024-04-25T09:59:08.391644+0000","last_became_active":"2024-04-25T09:59:08.380475+0000","last_became_peered":"2024-04-25T09:59:08.380475+0000","last_unstale":"2024-04-25T09:59:08.391644+0000","last_undegraded":"2024-04-25T09:59:08.391644+0000","last_fullsized":"2024-04-25T09:59:08.391644+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:08.200701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.395271+0000","last_change":"2024-04-25T09:59:08.392868+0000","last_active":"2024-04-25T09:59:08.395271+0000","last_peered":"2024-04-25T09:59:08.395271+0000","last_clean":"2024-04-25T09:59:08.395271+0000","last_became_active":"2024-04-25T09:59:08.392583+0000","last_became_peered":"2024-04-25T09:59:08.392583+0000","last_unstale":"2024-04-25T09:59:08.395271+0000","last_undegraded":"2024-04-25T09:59:08.395271+0000","last_fullsized":"2024-04-25T09:59:08.395271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:36.021887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387590+0000","last_change":"2024-04-25T09:59:08.384291+0000","last_active":"2024-04-25T09:59:08.387590+0000","last_peered":"2024-04-25T09:59:08.387590+0000","last_clean":"2024-04-25T09:59:08.387590+0000","last_became_active":"2024-04-25T09:59:08.383932+0000","last_became_peered":"2024-04-25T09:59:08.383932+0000","last_unstale":"2024-04-25T09:59:08.387590+0000","last_undegraded":"2024-04-25T09:59:08.387590+0000","last_fullsized":"2024-04-25T09:59:08.387590+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:40.629470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.386626+0000","last_change":"2024-04-25T09:59:08.382131+0000","last_active":"2024-04-25T09:59:08.386626+0000","last_peered":"2024-04-25T09:59:08.386626+0000","last_clean":"2024-04-25T09:59:08.386626+0000","last_became_active":"2024-04-25T09:59:08.381820+0000","last_became_peered":"2024-04-25T09:59:08.381820+0000","last_unstale":"2024-04-25T09:59:08.386626+0000","last_undegraded":"2024-04-25T09:59:08.386626+0000","last_fullsized":"2024-04-25T09:59:08.386626+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:43.402769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392532+0000","last_change":"2024-04-25T09:59:08.389725+0000","last_active":"2024-04-25T09:59:08.392532+0000","last_peered":"2024-04-25T09:59:08.392532+0000","last_clean":"2024-04-25T09:59:08.392532+0000","last_became_active":"2024-04-25T09:59:08.389421+0000","last_became_peered":"2024-04-25T09:59:08.389421+0000","last_unstale":"2024-04-25T09:59:08.392532+0000","last_undegraded":"2024-04-25T09:59:08.392532+0000","last_fullsized":"2024-04-25T09:59:08.392532+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:42.426714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.388981+0000","last_change":"2024-04-25T09:59:08.382381+0000","last_active":"2024-04-25T09:59:08.388981+0000","last_peered":"2024-04-25T09:59:08.388981+0000","last_clean":"2024-04-25T09:59:08.388981+0000","last_became_active":"2024-04-25T09:59:08.382063+0000","last_became_peered":"2024-04-25T09:59:08.382063+0000","last_unstale":"2024-04-25T09:59:08.388981+0000","last_undegraded":"2024-04-25T09:59:08.388981+0000","last_fullsized":"2024-04-25T09:59:08.388981+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:04.838824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.388082+0000","last_change":"2024-04-25T09:59:08.385871+0000","last_active":"2024-04-25T09:59:08.388082+0000","last_peered":"2024-04-25T09:59:08.388082+0000","last_clean":"2024-04-25T09:59:08.388082+0000","last_became_active":"2024-04-25T09:59:08.385567+0000","last_became_peered":"2024-04-25T09:59:08.385567+0000","last_unstale":"2024-04-25T09:59:08.388082+0000","last_undegraded":"2024-04-25T09:59:08.388082+0000","last_fullsized":"2024-04-25T09:59:08.388082+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:24.363765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.385193+0000","last_change":"2024-04-25T09:59:08.379637+0000","last_active":"2024-04-25T09:59:08.385193+0000","last_peered":"2024-04-25T09:59:08.385193+0000","last_clean":"2024-04-25T09:59:08.385193+0000","last_became_active":"2024-04-25T09:59:08.377345+0000","last_became_peered":"2024-04-25T09:59:08.377345+0000","last_unstale":"2024-04-25T09:59:08.385193+0000","last_undegraded":"2024-04-25T09:59:08.385193+0000","last_fullsized":"2024-04-25T09:59:08.385193+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:40.564913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389557+0000","last_change":"2024-04-25T09:59:08.385458+0000","last_active":"2024-04-25T09:59:08.389557+0000","last_peered":"2024-04-25T09:59:08.389557+0000","last_clean":"2024-04-25T09:59:08.389557+0000","last_became_active":"2024-04-25T09:59:08.385146+0000","last_became_peered":"2024-04-25T09:59:08.385146+0000","last_unstale":"2024-04-25T09:59:08.389557+0000","last_undegraded":"2024-04-25T09:59:08.389557+0000","last_fullsized":"2024-04-25T09:59:08.389557+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:40.564913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390978+0000","last_change":"2024-04-25T09:59:08.387926+0000","last_active":"2024-04-25T09:59:08.390978+0000","last_peered":"2024-04-25T09:59:08.390978+0000","last_clean":"2024-04-25T09:59:08.390978+0000","last_became_active":"2024-04-25T09:59:08.387621+0000","last_became_peered":"2024-04-25T09:59:08.387621+0000","last_unstale":"2024-04-25T09:59:08.390978+0000","last_undegraded":"2024-04-25T09:59:08.390978+0000","last_fullsized":"2024-04-25T09:59:08.390978+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:51.099297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.397468+0000","last_change":"2024-04-25T09:59:08.394954+0000","last_active":"2024-04-25T09:59:08.397468+0000","last_peered":"2024-04-25T09:59:08.397468+0000","last_clean":"2024-04-25T09:59:08.397468+0000","last_became_active":"2024-04-25T09:59:08.394436+0000","last_became_peered":"2024-04-25T09:59:08.394436+0000","last_unstale":"2024-04-25T09:59:08.397468+0000","last_undegraded":"2024-04-25T09:59:08.397468+0000","last_fullsized":"2024-04-25T09:59:08.397468+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:21.951308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.396896+0000","last_change":"2024-04-25T09:59:08.394184+0000","last_active":"2024-04-25T09:59:08.396896+0000","last_peered":"2024-04-25T09:59:08.396896+0000","last_clean":"2024-04-25T09:59:08.396896+0000","last_became_active":"2024-04-25T09:59:08.393908+0000","last_became_peered":"2024-04-25T09:59:08.393908+0000","last_unstale":"2024-04-25T09:59:08.396896+0000","last_undegraded":"2024-04-25T09:59:08.396896+0000","last_fullsized":"2024-04-25T09:59:08.396896+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:00.996221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392982+0000","last_change":"2024-04-25T09:59:08.391159+0000","last_active":"2024-04-25T09:59:08.392982+0000","last_peered":"2024-04-25T09:59:08.392982+0000","last_clean":"2024-04-25T09:59:08.392982+0000","last_became_active":"2024-04-25T09:59:08.390893+0000","last_became_peered":"2024-04-25T09:59:08.390893+0000","last_unstale":"2024-04-25T09:59:08.392982+0000","last_undegraded":"2024-04-25T09:59:08.392982+0000","last_fullsized":"2024-04-25T09:59:08.392982+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:35.879053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389453+0000","last_change":"2024-04-25T09:59:08.384613+0000","last_active":"2024-04-25T09:59:08.389453+0000","last_peered":"2024-04-25T09:59:08.389453+0000","last_clean":"2024-04-25T09:59:08.389453+0000","last_became_active":"2024-04-25T09:59:08.384340+0000","last_became_peered":"2024-04-25T09:59:08.384340+0000","last_unstale":"2024-04-25T09:59:08.389453+0000","last_undegraded":"2024-04-25T09:59:08.389453+0000","last_fullsized":"2024-04-25T09:59:08.389453+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:36.989614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387336+0000","last_change":"2024-04-25T09:59:08.385910+0000","last_active":"2024-04-25T09:59:08.387336+0000","last_peered":"2024-04-25T09:59:08.387336+0000","last_clean":"2024-04-25T09:59:08.387336+0000","last_became_active":"2024-04-25T09:59:08.385641+0000","last_became_peered":"2024-04-25T09:59:08.385641+0000","last_unstale":"2024-04-25T09:59:08.387336+0000","last_undegraded":"2024-04-25T09:59:08.387336+0000","last_fullsized":"2024-04-25T09:59:08.387336+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:35.939258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387690+0000","last_change":"2024-04-25T09:59:08.383111+0000","last_active":"2024-04-25T09:59:08.387690+0000","last_peered":"2024-04-25T09:59:08.387690+0000","last_clean":"2024-04-25T09:59:08.387690+0000","last_became_active":"2024-04-25T09:59:08.382805+0000","last_became_peered":"2024-04-25T09:59:08.382805+0000","last_unstale":"2024-04-25T09:59:08.387690+0000","last_undegraded":"2024-04-25T09:59:08.387690+0000","last_fullsized":"2024-04-25T09:59:08.387690+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:57.024245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383778+0000","last_change":"2024-04-25T09:59:08.379959+0000","last_active":"2024-04-25T09:59:08.383778+0000","last_peered":"2024-04-25T09:59:08.383778+0000","last_clean":"2024-04-25T09:59:08.383778+0000","last_became_active":"2024-04-25T09:59:08.379625+0000","last_became_peered":"2024-04-25T09:59:08.379625+0000","last_unstale":"2024-04-25T09:59:08.383778+0000","last_undegraded":"2024-04-25T09:59:08.383778+0000","last_fullsized":"2024-04-25T09:59:08.383778+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:46.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.381259+0000","last_change":"2024-04-25T09:59:08.380204+0000","last_active":"2024-04-25T09:59:08.381259+0000","last_peered":"2024-04-25T09:59:08.381259+0000","last_clean":"2024-04-25T09:59:08.381259+0000","last_became_active":"2024-04-25T09:59:08.379933+0000","last_became_peered":"2024-04-25T09:59:08.379933+0000","last_unstale":"2024-04-25T09:59:08.381259+0000","last_undegraded":"2024-04-25T09:59:08.381259+0000","last_fullsized":"2024-04-25T09:59:08.381259+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:40.889027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.385534+0000","last_change":"2024-04-25T09:59:08.384253+0000","last_active":"2024-04-25T09:59:08.385534+0000","last_peered":"2024-04-25T09:59:08.385534+0000","last_clean":"2024-04-25T09:59:08.385534+0000","last_became_active":"2024-04-25T09:59:08.383960+0000","last_became_peered":"2024-04-25T09:59:08.383960+0000","last_unstale":"2024-04-25T09:59:08.385534+0000","last_undegraded":"2024-04-25T09:59:08.385534+0000","last_fullsized":"2024-04-25T09:59:08.385534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:50.966310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.374063+0000","last_change":"2024-04-25T09:59:08.370958+0000","last_active":"2024-04-25T09:59:08.374063+0000","last_peered":"2024-04-25T09:59:08.374063+0000","last_clean":"2024-04-25T09:59:08.374063+0000","last_became_active":"2024-04-25T09:59:08.370652+0000","last_became_peered":"2024-04-25T09:59:08.370652+0000","last_unstale":"2024-04-25T09:59:08.374063+0000","last_undegraded":"2024-04-25T09:59:08.374063+0000","last_fullsized":"2024-04-25T09:59:08.374063+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:12.078167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389508+0000","last_change":"2024-04-25T09:59:08.385454+0000","last_active":"2024-04-25T09:59:08.389508+0000","last_peered":"2024-04-25T09:59:08.389508+0000","last_clean":"2024-04-25T09:59:08.389508+0000","last_became_active":"2024-04-25T09:59:08.385070+0000","last_became_peered":"2024-04-25T09:59:08.385070+0000","last_unstale":"2024-04-25T09:59:08.389508+0000","last_undegraded":"2024-04-25T09:59:08.389508+0000","last_fullsized":"2024-04-25T09:59:08.389508+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:50.470219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.381771+0000","last_change":"2024-04-25T09:59:08.379785+0000","last_active":"2024-04-25T09:59:08.381771+0000","last_peered":"2024-04-25T09:59:08.381771+0000","last_clean":"2024-04-25T09:59:08.381771+0000","last_became_active":"2024-04-25T09:59:08.379471+0000","last_became_peered":"2024-04-25T09:59:08.379471+0000","last_unstale":"2024-04-25T09:59:08.381771+0000","last_undegraded":"2024-04-25T09:59:08.381771+0000","last_fullsized":"2024-04-25T09:59:08.381771+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:47.499730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.386043+0000","last_change":"2024-04-25T09:59:08.384162+0000","last_active":"2024-04-25T09:59:08.386043+0000","last_peered":"2024-04-25T09:59:08.386043+0000","last_clean":"2024-04-25T09:59:08.386043+0000","last_became_active":"2024-04-25T09:59:08.383852+0000","last_became_peered":"2024-04-25T09:59:08.383852+0000","last_unstale":"2024-04-25T09:59:08.386043+0000","last_undegraded":"2024-04-25T09:59:08.386043+0000","last_fullsized":"2024-04-25T09:59:08.386043+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:40.889027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387724+0000","last_change":"2024-04-25T09:59:08.381228+0000","last_active":"2024-04-25T09:59:08.387724+0000","last_peered":"2024-04-25T09:59:08.387724+0000","last_clean":"2024-04-25T09:59:08.387724+0000","last_became_active":"2024-04-25T09:59:08.380934+0000","last_became_peered":"2024-04-25T09:59:08.380934+0000","last_unstale":"2024-04-25T09:59:08.387724+0000","last_undegraded":"2024-04-25T09:59:08.387724+0000","last_fullsized":"2024-04-25T09:59:08.387724+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:51.221958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.386653+0000","last_change":"2024-04-25T09:59:08.381918+0000","last_active":"2024-04-25T09:59:08.386653+0000","last_peered":"2024-04-25T09:59:08.386653+0000","last_clean":"2024-04-25T09:59:08.386653+0000","last_became_active":"2024-04-25T09:59:08.381322+0000","last_became_peered":"2024-04-25T09:59:08.381322+0000","last_unstale":"2024-04-25T09:59:08.386653+0000","last_undegraded":"2024-04-25T09:59:08.386653+0000","last_fullsized":"2024-04-25T09:59:08.386653+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:25.226164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383568+0000","last_change":"2024-04-25T09:59:08.379238+0000","last_active":"2024-04-25T09:59:08.383568+0000","last_peered":"2024-04-25T09:59:08.383568+0000","last_clean":"2024-04-25T09:59:08.383568+0000","last_became_active":"2024-04-25T09:59:08.378761+0000","last_became_peered":"2024-04-25T09:59:08.378761+0000","last_unstale":"2024-04-25T09:59:08.383568+0000","last_undegraded":"2024-04-25T09:59:08.383568+0000","last_fullsized":"2024-04-25T09:59:08.383568+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:06.141494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.397446+0000","last_change":"2024-04-25T09:59:08.394715+0000","last_active":"2024-04-25T09:59:08.397446+0000","last_peered":"2024-04-25T09:59:08.397446+0000","last_clean":"2024-04-25T09:59:08.397446+0000","last_became_active":"2024-04-25T09:59:08.394044+0000","last_became_peered":"2024-04-25T09:59:08.394044+0000","last_unstale":"2024-04-25T09:59:08.397446+0000","last_undegraded":"2024-04-25T09:59:08.397446+0000","last_fullsized":"2024-04-25T09:59:08.397446+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:48.624669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.396096+0000","last_change":"2024-04-25T09:59:08.387696+0000","last_active":"2024-04-25T09:59:08.396096+0000","last_peered":"2024-04-25T09:59:08.396096+0000","last_clean":"2024-04-25T09:59:08.396096+0000","last_became_active":"2024-04-25T09:59:08.387398+0000","last_became_peered":"2024-04-25T09:59:08.387398+0000","last_unstale":"2024-04-25T09:59:08.396096+0000","last_undegraded":"2024-04-25T09:59:08.396096+0000","last_fullsized":"2024-04-25T09:59:08.396096+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:30.941125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"94'4","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.400744+0000","last_change":"2024-04-25T09:59:07.387484+0000","last_active":"2024-04-25T09:59:08.400744+0000","last_peered":"2024-04-25T09:59:08.400744+0000","last_clean":"2024-04-25T09:59:08.400744+0000","last_became_active":"2024-04-25T09:59:07.386943+0000","last_became_peered":"2024-04-25T09:59:07.386943+0000","last_unstale":"2024-04-25T09:59:08.400744+0000","last_undegraded":"2024-04-25T09:59:08.400744+0000","last_fullsized":"2024-04-25T09:59:08.400744+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:03.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389417+0000","last_change":"2024-04-25T09:59:08.382317+0000","last_active":"2024-04-25T09:59:08.389417+0000","last_peered":"2024-04-25T09:59:08.389417+0000","last_clean":"2024-04-25T09:59:08.389417+0000","last_became_active":"2024-04-25T09:59:08.382047+0000","last_became_peered":"2024-04-25T09:59:08.382047+0000","last_unstale":"2024-04-25T09:59:08.389417+0000","last_undegraded":"2024-04-25T09:59:08.389417+0000","last_fullsized":"2024-04-25T09:59:08.389417+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:11.798267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390050+0000","last_change":"2024-04-25T09:59:07.376083+0000","last_active":"2024-04-25T09:59:08.390050+0000","last_peered":"2024-04-25T09:59:08.390050+0000","last_clean":"2024-04-25T09:59:08.390050+0000","last_became_active":"2024-04-25T09:59:07.375195+0000","last_became_peered":"2024-04-25T09:59:07.375195+0000","last_unstale":"2024-04-25T09:59:08.390050+0000","last_undegraded":"2024-04-25T09:59:08.390050+0000","last_fullsized":"2024-04-25T09:59:08.390050+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:37.624313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.384698+0000","last_change":"2024-04-25T09:59:08.380359+0000","last_active":"2024-04-25T09:59:08.384698+0000","last_peered":"2024-04-25T09:59:08.384698+0000","last_clean":"2024-04-25T09:59:08.384698+0000","last_became_active":"2024-04-25T09:59:08.380051+0000","last_became_peered":"2024-04-25T09:59:08.380051+0000","last_unstale":"2024-04-25T09:59:08.384698+0000","last_undegraded":"2024-04-25T09:59:08.384698+0000","last_fullsized":"2024-04-25T09:59:08.384698+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:52.908151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390934+0000","last_change":"2024-04-25T09:59:07.374210+0000","last_active":"2024-04-25T09:59:08.390934+0000","last_peered":"2024-04-25T09:59:08.390934+0000","last_clean":"2024-04-25T09:59:08.390934+0000","last_became_active":"2024-04-25T09:59:07.373928+0000","last_became_peered":"2024-04-25T09:59:07.373928+0000","last_unstale":"2024-04-25T09:59:08.390934+0000","last_undegraded":"2024-04-25T09:59:08.390934+0000","last_fullsized":"2024-04-25T09:59:08.390934+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:00.551940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.391677+0000","last_change":"2024-04-25T09:59:08.379002+0000","last_active":"2024-04-25T09:59:08.391677+0000","last_peered":"2024-04-25T09:59:08.391677+0000","last_clean":"2024-04-25T09:59:08.391677+0000","last_became_active":"2024-04-25T09:59:08.378721+0000","last_became_peered":"2024-04-25T09:59:08.378721+0000","last_unstale":"2024-04-25T09:59:08.391677+0000","last_undegraded":"2024-04-25T09:59:08.391677+0000","last_fullsized":"2024-04-25T09:59:08.391677+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:56.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.395201+0000","last_change":"2024-04-25T09:59:07.387257+0000","last_active":"2024-04-25T09:59:08.395201+0000","last_peered":"2024-04-25T09:59:08.395201+0000","last_clean":"2024-04-25T09:59:08.395201+0000","last_became_active":"2024-04-25T09:59:07.386974+0000","last_became_peered":"2024-04-25T09:59:07.386974+0000","last_unstale":"2024-04-25T09:59:08.395201+0000","last_undegraded":"2024-04-25T09:59:08.395201+0000","last_fullsized":"2024-04-25T09:59:08.395201+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:35.987378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.397788+0000","last_change":"2024-04-25T09:59:08.388723+0000","last_active":"2024-04-25T09:59:08.397788+0000","last_peered":"2024-04-25T09:59:08.397788+0000","last_clean":"2024-04-25T09:59:08.397788+0000","last_became_active":"2024-04-25T09:59:08.388284+0000","last_became_peered":"2024-04-25T09:59:08.388284+0000","last_unstale":"2024-04-25T09:59:08.397788+0000","last_undegraded":"2024-04-25T09:59:08.397788+0000","last_fullsized":"2024-04-25T09:59:08.397788+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.520303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390182+0000","last_change":"2024-04-25T09:59:07.372735+0000","last_active":"2024-04-25T09:59:08.390182+0000","last_peered":"2024-04-25T09:59:08.390182+0000","last_clean":"2024-04-25T09:59:08.390182+0000","last_became_active":"2024-04-25T09:59:07.372449+0000","last_became_peered":"2024-04-25T09:59:07.372449+0000","last_unstale":"2024-04-25T09:59:08.390182+0000","last_undegraded":"2024-04-25T09:59:08.390182+0000","last_fullsized":"2024-04-25T09:59:08.390182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:48.345862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389209+0000","last_change":"2024-04-25T09:59:08.383430+0000","last_active":"2024-04-25T09:59:08.389209+0000","last_peered":"2024-04-25T09:59:08.389209+0000","last_clean":"2024-04-25T09:59:08.389209+0000","last_became_active":"2024-04-25T09:59:08.382898+0000","last_became_peered":"2024-04-25T09:59:08.382898+0000","last_unstale":"2024-04-25T09:59:08.389209+0000","last_undegraded":"2024-04-25T09:59:08.389209+0000","last_fullsized":"2024-04-25T09:59:08.389209+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:40.629470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.395465+0000","last_change":"2024-04-25T09:59:07.383660+0000","last_active":"2024-04-25T09:59:08.395465+0000","last_peered":"2024-04-25T09:59:08.395465+0000","last_clean":"2024-04-25T09:59:08.395465+0000","last_became_active":"2024-04-25T09:59:07.383362+0000","last_became_peered":"2024-04-25T09:59:07.383362+0000","last_unstale":"2024-04-25T09:59:08.395465+0000","last_undegraded":"2024-04-25T09:59:08.395465+0000","last_fullsized":"2024-04-25T09:59:08.395465+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:03.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.391619+0000","last_change":"2024-04-25T09:59:08.380730+0000","last_active":"2024-04-25T09:59:08.391619+0000","last_peered":"2024-04-25T09:59:08.391619+0000","last_clean":"2024-04-25T09:59:08.391619+0000","last_became_active":"2024-04-25T09:59:08.380133+0000","last_became_peered":"2024-04-25T09:59:08.380133+0000","last_unstale":"2024-04-25T09:59:08.391619+0000","last_undegraded":"2024-04-25T09:59:08.391619+0000","last_fullsized":"2024-04-25T09:59:08.391619+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:02.462568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.381809+0000","last_change":"2024-04-25T09:59:07.380367+0000","last_active":"2024-04-25T09:59:08.381809+0000","last_peered":"2024-04-25T09:59:08.381809+0000","last_clean":"2024-04-25T09:59:08.381809+0000","last_became_active":"2024-04-25T09:59:07.380062+0000","last_became_peered":"2024-04-25T09:59:07.380062+0000","last_unstale":"2024-04-25T09:59:08.381809+0000","last_undegraded":"2024-04-25T09:59:08.381809+0000","last_fullsized":"2024-04-25T09:59:08.381809+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:29.376037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.385173+0000","last_change":"2024-04-25T09:59:08.378461+0000","last_active":"2024-04-25T09:59:08.385173+0000","last_peered":"2024-04-25T09:59:08.385173+0000","last_clean":"2024-04-25T09:59:08.385173+0000","last_became_active":"2024-04-25T09:59:08.377804+0000","last_became_peered":"2024-04-25T09:59:08.377804+0000","last_unstale":"2024-04-25T09:59:08.385173+0000","last_undegraded":"2024-04-25T09:59:08.385173+0000","last_fullsized":"2024-04-25T09:59:08.385173+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:50.470219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.366467+0000","last_change":"2024-04-25T09:59:07.382231+0000","last_active":"2024-04-25T09:59:08.366467+0000","last_peered":"2024-04-25T09:59:08.366467+0000","last_clean":"2024-04-25T09:59:08.366467+0000","last_became_active":"2024-04-25T09:59:07.381450+0000","last_became_peered":"2024-04-25T09:59:07.381450+0000","last_unstale":"2024-04-25T09:59:08.366467+0000","last_undegraded":"2024-04-25T09:59:08.366467+0000","last_fullsized":"2024-04-25T09:59:08.366467+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:59.759144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383882+0000","last_change":"2024-04-25T09:59:08.378636+0000","last_active":"2024-04-25T09:59:08.383882+0000","last_peered":"2024-04-25T09:59:08.383882+0000","last_clean":"2024-04-25T09:59:08.383882+0000","last_became_active":"2024-04-25T09:59:08.378184+0000","last_became_peered":"2024-04-25T09:59:08.378184+0000","last_unstale":"2024-04-25T09:59:08.383882+0000","last_undegraded":"2024-04-25T09:59:08.383882+0000","last_fullsized":"2024-04-25T09:59:08.383882+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:49.324390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.367861+0000","last_change":"2024-04-25T09:59:07.379106+0000","last_active":"2024-04-25T09:59:08.367861+0000","last_peered":"2024-04-25T09:59:08.367861+0000","last_clean":"2024-04-25T09:59:08.367861+0000","last_became_active":"2024-04-25T09:59:07.378811+0000","last_became_peered":"2024-04-25T09:59:07.378811+0000","last_unstale":"2024-04-25T09:59:08.367861+0000","last_undegraded":"2024-04-25T09:59:08.367861+0000","last_fullsized":"2024-04-25T09:59:08.367861+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:17.563606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387424+0000","last_change":"2024-04-25T09:59:08.383214+0000","last_active":"2024-04-25T09:59:08.387424+0000","last_peered":"2024-04-25T09:59:08.387424+0000","last_clean":"2024-04-25T09:59:08.387424+0000","last_became_active":"2024-04-25T09:59:08.382906+0000","last_became_peered":"2024-04-25T09:59:08.382906+0000","last_unstale":"2024-04-25T09:59:08.387424+0000","last_undegraded":"2024-04-25T09:59:08.387424+0000","last_fullsized":"2024-04-25T09:59:08.387424+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:09.753812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390645+0000","last_change":"2024-04-25T09:59:07.387919+0000","last_active":"2024-04-25T09:59:08.390645+0000","last_peered":"2024-04-25T09:59:08.390645+0000","last_clean":"2024-04-25T09:59:08.390645+0000","last_became_active":"2024-04-25T09:59:07.387590+0000","last_became_peered":"2024-04-25T09:59:07.387590+0000","last_unstale":"2024-04-25T09:59:08.390645+0000","last_undegraded":"2024-04-25T09:59:08.390645+0000","last_fullsized":"2024-04-25T09:59:08.390645+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:20.158839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.398662+0000","last_change":"2024-04-25T09:59:08.397792+0000","last_active":"2024-04-25T09:59:08.398662+0000","last_peered":"2024-04-25T09:59:08.398662+0000","last_clean":"2024-04-25T09:59:08.398662+0000","last_became_active":"2024-04-25T09:59:08.397482+0000","last_became_peered":"2024-04-25T09:59:08.397482+0000","last_unstale":"2024-04-25T09:59:08.398662+0000","last_undegraded":"2024-04-25T09:59:08.398662+0000","last_fullsized":"2024-04-25T09:59:08.398662+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:00.761380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.396726+0000","last_change":"2024-04-25T09:59:07.381794+0000","last_active":"2024-04-25T09:59:08.396726+0000","last_peered":"2024-04-25T09:59:08.396726+0000","last_clean":"2024-04-25T09:59:08.396726+0000","last_became_active":"2024-04-25T09:59:07.381451+0000","last_became_peered":"2024-04-25T09:59:07.381451+0000","last_unstale":"2024-04-25T09:59:08.396726+0000","last_undegraded":"2024-04-25T09:59:08.396726+0000","last_fullsized":"2024-04-25T09:59:08.396726+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:56.763080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.385342+0000","last_change":"2024-04-25T09:59:08.382317+0000","last_active":"2024-04-25T09:59:08.385342+0000","last_peered":"2024-04-25T09:59:08.385342+0000","last_clean":"2024-04-25T09:59:08.385342+0000","last_became_active":"2024-04-25T09:59:08.382044+0000","last_became_peered":"2024-04-25T09:59:08.382044+0000","last_unstale":"2024-04-25T09:59:08.385342+0000","last_undegraded":"2024-04-25T09:59:08.385342+0000","last_fullsized":"2024-04-25T09:59:08.385342+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:29.918692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.402536+0000","last_change":"2024-04-25T09:59:07.384024+0000","last_active":"2024-04-25T09:59:08.402536+0000","last_peered":"2024-04-25T09:59:08.402536+0000","last_clean":"2024-04-25T09:59:08.402536+0000","last_became_active":"2024-04-25T09:59:07.383613+0000","last_became_peered":"2024-04-25T09:59:07.383613+0000","last_unstale":"2024-04-25T09:59:08.402536+0000","last_undegraded":"2024-04-25T09:59:08.402536+0000","last_fullsized":"2024-04-25T09:59:08.402536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:40.703547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.384457+0000","last_change":"2024-04-25T09:59:08.377089+0000","last_active":"2024-04-25T09:59:08.384457+0000","last_peered":"2024-04-25T09:59:08.384457+0000","last_clean":"2024-04-25T09:59:08.384457+0000","last_became_active":"2024-04-25T09:59:08.376764+0000","last_became_peered":"2024-04-25T09:59:08.376764+0000","last_unstale":"2024-04-25T09:59:08.384457+0000","last_undegraded":"2024-04-25T09:59:08.384457+0000","last_fullsized":"2024-04-25T09:59:08.384457+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:36.989614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.397753+0000","last_change":"2024-04-25T09:59:07.375328+0000","last_active":"2024-04-25T09:59:08.397753+0000","last_peered":"2024-04-25T09:59:08.397753+0000","last_clean":"2024-04-25T09:59:08.397753+0000","last_became_active":"2024-04-25T09:59:07.375041+0000","last_became_peered":"2024-04-25T09:59:07.375041+0000","last_unstale":"2024-04-25T09:59:08.397753+0000","last_undegraded":"2024-04-25T09:59:08.397753+0000","last_fullsized":"2024-04-25T09:59:08.397753+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:34.876817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383773+0000","last_change":"2024-04-25T09:59:08.378356+0000","last_active":"2024-04-25T09:59:08.383773+0000","last_peered":"2024-04-25T09:59:08.383773+0000","last_clean":"2024-04-25T09:59:08.383773+0000","last_became_active":"2024-04-25T09:59:08.378075+0000","last_became_peered":"2024-04-25T09:59:08.378075+0000","last_unstale":"2024-04-25T09:59:08.383773+0000","last_undegraded":"2024-04-25T09:59:08.383773+0000","last_fullsized":"2024-04-25T09:59:08.383773+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:52.269207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.367770+0000","last_change":"2024-04-25T09:59:07.374251+0000","last_active":"2024-04-25T09:59:08.367770+0000","last_peered":"2024-04-25T09:59:08.367770+0000","last_clean":"2024-04-25T09:59:08.367770+0000","last_became_active":"2024-04-25T09:59:07.373928+0000","last_became_peered":"2024-04-25T09:59:07.373928+0000","last_unstale":"2024-04-25T09:59:08.367770+0000","last_undegraded":"2024-04-25T09:59:08.367770+0000","last_fullsized":"2024-04-25T09:59:08.367770+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:43.226355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387989+0000","last_change":"2024-04-25T09:59:08.387063+0000","last_active":"2024-04-25T09:59:08.387989+0000","last_peered":"2024-04-25T09:59:08.387989+0000","last_clean":"2024-04-25T09:59:08.387989+0000","last_became_active":"2024-04-25T09:59:08.386749+0000","last_became_peered":"2024-04-25T09:59:08.386749+0000","last_unstale":"2024-04-25T09:59:08.387989+0000","last_undegraded":"2024-04-25T09:59:08.387989+0000","last_fullsized":"2024-04-25T09:59:08.387989+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:17.388048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.376405+0000","last_change":"2024-04-25T09:59:07.386944+0000","last_active":"2024-04-25T09:59:08.376405+0000","last_peered":"2024-04-25T09:59:08.376405+0000","last_clean":"2024-04-25T09:59:08.376405+0000","last_became_active":"2024-04-25T09:59:07.386625+0000","last_became_peered":"2024-04-25T09:59:07.386625+0000","last_unstale":"2024-04-25T09:59:08.376405+0000","last_undegraded":"2024-04-25T09:59:08.376405+0000","last_fullsized":"2024-04-25T09:59:08.376405+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:56.022008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.388023+0000","last_change":"2024-04-25T09:59:08.387077+0000","last_active":"2024-04-25T09:59:08.388023+0000","last_peered":"2024-04-25T09:59:08.388023+0000","last_clean":"2024-04-25T09:59:08.388023+0000","last_became_active":"2024-04-25T09:59:08.386810+0000","last_became_peered":"2024-04-25T09:59:08.386810+0000","last_unstale":"2024-04-25T09:59:08.388023+0000","last_undegraded":"2024-04-25T09:59:08.388023+0000","last_fullsized":"2024-04-25T09:59:08.388023+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:16.380411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.399849+0000","last_change":"2024-04-25T09:59:07.385026+0000","last_active":"2024-04-25T09:59:08.399849+0000","last_peered":"2024-04-25T09:59:08.399849+0000","last_clean":"2024-04-25T09:59:08.399849+0000","last_became_active":"2024-04-25T09:59:07.381589+0000","last_became_peered":"2024-04-25T09:59:07.381589+0000","last_unstale":"2024-04-25T09:59:08.399849+0000","last_undegraded":"2024-04-25T09:59:08.399849+0000","last_fullsized":"2024-04-25T09:59:08.399849+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:07.147024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.391797+0000","last_change":"2024-04-25T09:59:08.379283+0000","last_active":"2024-04-25T09:59:08.391797+0000","last_peered":"2024-04-25T09:59:08.391797+0000","last_clean":"2024-04-25T09:59:08.391797+0000","last_became_active":"2024-04-25T09:59:08.379018+0000","last_became_peered":"2024-04-25T09:59:08.379018+0000","last_unstale":"2024-04-25T09:59:08.391797+0000","last_undegraded":"2024-04-25T09:59:08.391797+0000","last_fullsized":"2024-04-25T09:59:08.391797+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:27.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392338+0000","last_change":"2024-04-25T09:59:07.377488+0000","last_active":"2024-04-25T09:59:08.392338+0000","last_peered":"2024-04-25T09:59:08.392338+0000","last_clean":"2024-04-25T09:59:08.392338+0000","last_became_active":"2024-04-25T09:59:07.377084+0000","last_became_peered":"2024-04-25T09:59:07.377084+0000","last_unstale":"2024-04-25T09:59:08.392338+0000","last_undegraded":"2024-04-25T09:59:08.392338+0000","last_fullsized":"2024-04-25T09:59:08.392338+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:56.022008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.375484+0000","last_change":"2024-04-25T09:59:08.371460+0000","last_active":"2024-04-25T09:59:08.375484+0000","last_peered":"2024-04-25T09:59:08.375484+0000","last_clean":"2024-04-25T09:59:08.375484+0000","last_became_active":"2024-04-25T09:59:08.371199+0000","last_became_peered":"2024-04-25T09:59:08.371199+0000","last_unstale":"2024-04-25T09:59:08.375484+0000","last_undegraded":"2024-04-25T09:59:08.375484+0000","last_fullsized":"2024-04-25T09:59:08.375484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:40.564913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392739+0000","last_change":"2024-04-25T09:59:07.379302+0000","last_active":"2024-04-25T09:59:08.392739+0000","last_peered":"2024-04-25T09:59:08.392739+0000","last_clean":"2024-04-25T09:59:08.392739+0000","last_became_active":"2024-04-25T09:59:07.379019+0000","last_became_peered":"2024-04-25T09:59:07.379019+0000","last_unstale":"2024-04-25T09:59:08.392739+0000","last_undegraded":"2024-04-25T09:59:08.392739+0000","last_fullsized":"2024-04-25T09:59:08.392739+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:18.377249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.386881+0000","last_change":"2024-04-25T09:59:08.382384+0000","last_active":"2024-04-25T09:59:08.386881+0000","last_peered":"2024-04-25T09:59:08.386881+0000","last_clean":"2024-04-25T09:59:08.386881+0000","last_became_active":"2024-04-25T09:59:08.381626+0000","last_became_peered":"2024-04-25T09:59:08.381626+0000","last_unstale":"2024-04-25T09:59:08.386881+0000","last_undegraded":"2024-04-25T09:59:08.386881+0000","last_fullsized":"2024-04-25T09:59:08.386881+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:53.728412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392429+0000","last_change":"2024-04-25T09:59:07.386644+0000","last_active":"2024-04-25T09:59:08.392429+0000","last_peered":"2024-04-25T09:59:08.392429+0000","last_clean":"2024-04-25T09:59:08.392429+0000","last_became_active":"2024-04-25T09:59:07.386411+0000","last_became_peered":"2024-04-25T09:59:07.386411+0000","last_unstale":"2024-04-25T09:59:08.392429+0000","last_undegraded":"2024-04-25T09:59:08.392429+0000","last_fullsized":"2024-04-25T09:59:08.392429+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:20.040338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.395600+0000","last_change":"2024-04-25T09:59:08.388679+0000","last_active":"2024-04-25T09:59:08.395600+0000","last_peered":"2024-04-25T09:59:08.395600+0000","last_clean":"2024-04-25T09:59:08.395600+0000","last_became_active":"2024-04-25T09:59:08.388421+0000","last_became_peered":"2024-04-25T09:59:08.388421+0000","last_unstale":"2024-04-25T09:59:08.395600+0000","last_undegraded":"2024-04-25T09:59:08.395600+0000","last_fullsized":"2024-04-25T09:59:08.395600+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:53.428739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.395135+0000","last_change":"2024-04-25T09:59:07.376556+0000","last_active":"2024-04-25T09:59:08.395135+0000","last_peered":"2024-04-25T09:59:08.395135+0000","last_clean":"2024-04-25T09:59:08.395135+0000","last_became_active":"2024-04-25T09:59:07.376289+0000","last_became_peered":"2024-04-25T09:59:07.376289+0000","last_unstale":"2024-04-25T09:59:08.395135+0000","last_undegraded":"2024-04-25T09:59:08.395135+0000","last_fullsized":"2024-04-25T09:59:08.395135+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:52.082835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.386401+0000","last_change":"2024-04-25T09:59:08.381709+0000","last_active":"2024-04-25T09:59:08.386401+0000","last_peered":"2024-04-25T09:59:08.386401+0000","last_clean":"2024-04-25T09:59:08.386401+0000","last_became_active":"2024-04-25T09:59:08.381426+0000","last_became_peered":"2024-04-25T09:59:08.381426+0000","last_unstale":"2024-04-25T09:59:08.386401+0000","last_undegraded":"2024-04-25T09:59:08.386401+0000","last_fullsized":"2024-04-25T09:59:08.386401+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:50.966310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.372625+0000","last_change":"2024-04-25T09:59:07.384848+0000","last_active":"2024-04-25T09:59:08.372625+0000","last_peered":"2024-04-25T09:59:08.372625+0000","last_clean":"2024-04-25T09:59:08.372625+0000","last_became_active":"2024-04-25T09:59:07.384509+0000","last_became_peered":"2024-04-25T09:59:07.384509+0000","last_unstale":"2024-04-25T09:59:08.372625+0000","last_undegraded":"2024-04-25T09:59:08.372625+0000","last_fullsized":"2024-04-25T09:59:08.372625+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:43.533573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383521+0000","last_change":"2024-04-25T09:59:08.379154+0000","last_active":"2024-04-25T09:59:08.383521+0000","last_peered":"2024-04-25T09:59:08.383521+0000","last_clean":"2024-04-25T09:59:08.383521+0000","last_became_active":"2024-04-25T09:59:08.378850+0000","last_became_peered":"2024-04-25T09:59:08.378850+0000","last_unstale":"2024-04-25T09:59:08.383521+0000","last_undegraded":"2024-04-25T09:59:08.383521+0000","last_fullsized":"2024-04-25T09:59:08.383521+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:08.974675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.367156+0000","last_change":"2024-04-25T09:59:07.381785+0000","last_active":"2024-04-25T09:59:08.367156+0000","last_peered":"2024-04-25T09:59:08.367156+0000","last_clean":"2024-04-25T09:59:08.367156+0000","last_became_active":"2024-04-25T09:59:07.381331+0000","last_became_peered":"2024-04-25T09:59:07.381331+0000","last_unstale":"2024-04-25T09:59:08.367156+0000","last_undegraded":"2024-04-25T09:59:08.367156+0000","last_fullsized":"2024-04-25T09:59:08.367156+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:36.178681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.393801+0000","last_change":"2024-04-25T09:59:08.387928+0000","last_active":"2024-04-25T09:59:08.393801+0000","last_peered":"2024-04-25T09:59:08.393801+0000","last_clean":"2024-04-25T09:59:08.393801+0000","last_became_active":"2024-04-25T09:59:08.386590+0000","last_became_peered":"2024-04-25T09:59:08.386590+0000","last_unstale":"2024-04-25T09:59:08.393801+0000","last_undegraded":"2024-04-25T09:59:08.393801+0000","last_fullsized":"2024-04-25T09:59:08.393801+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:34.820077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.372527+0000","last_change":"2024-04-25T09:59:07.377929+0000","last_active":"2024-04-25T09:59:08.372527+0000","last_peered":"2024-04-25T09:59:08.372527+0000","last_clean":"2024-04-25T09:59:08.372527+0000","last_became_active":"2024-04-25T09:59:07.377400+0000","last_became_peered":"2024-04-25T09:59:07.377400+0000","last_unstale":"2024-04-25T09:59:08.372527+0000","last_undegraded":"2024-04-25T09:59:08.372527+0000","last_fullsized":"2024-04-25T09:59:08.372527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:58.129751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.378298+0000","last_change":"2024-04-25T09:59:08.375048+0000","last_active":"2024-04-25T09:59:08.378298+0000","last_peered":"2024-04-25T09:59:08.378298+0000","last_clean":"2024-04-25T09:59:08.378298+0000","last_became_active":"2024-04-25T09:59:08.374686+0000","last_became_peered":"2024-04-25T09:59:08.374686+0000","last_unstale":"2024-04-25T09:59:08.378298+0000","last_undegraded":"2024-04-25T09:59:08.378298+0000","last_fullsized":"2024-04-25T09:59:08.378298+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:31.394397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392442+0000","last_change":"2024-04-25T09:59:07.376765+0000","last_active":"2024-04-25T09:59:08.392442+0000","last_peered":"2024-04-25T09:59:08.392442+0000","last_clean":"2024-04-25T09:59:08.392442+0000","last_became_active":"2024-04-25T09:59:07.376505+0000","last_became_peered":"2024-04-25T09:59:07.376505+0000","last_unstale":"2024-04-25T09:59:08.392442+0000","last_undegraded":"2024-04-25T09:59:08.392442+0000","last_fullsized":"2024-04-25T09:59:08.392442+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:52.746497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390391+0000","last_change":"2024-04-25T09:59:08.386619+0000","last_active":"2024-04-25T09:59:08.390391+0000","last_peered":"2024-04-25T09:59:08.390391+0000","last_clean":"2024-04-25T09:59:08.390391+0000","last_became_active":"2024-04-25T09:59:08.386340+0000","last_became_peered":"2024-04-25T09:59:08.386340+0000","last_unstale":"2024-04-25T09:59:08.390391+0000","last_undegraded":"2024-04-25T09:59:08.390391+0000","last_fullsized":"2024-04-25T09:59:08.390391+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:09.818893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.376477+0000","last_change":"2024-04-25T09:59:07.378817+0000","last_active":"2024-04-25T09:59:08.376477+0000","last_peered":"2024-04-25T09:59:08.376477+0000","last_clean":"2024-04-25T09:59:08.376477+0000","last_became_active":"2024-04-25T09:59:07.378407+0000","last_became_peered":"2024-04-25T09:59:07.378407+0000","last_unstale":"2024-04-25T09:59:08.376477+0000","last_undegraded":"2024-04-25T09:59:08.376477+0000","last_fullsized":"2024-04-25T09:59:08.376477+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:57.137009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389772+0000","last_change":"2024-04-25T09:59:08.381635+0000","last_active":"2024-04-25T09:59:08.389772+0000","last_peered":"2024-04-25T09:59:08.389772+0000","last_clean":"2024-04-25T09:59:08.389772+0000","last_became_active":"2024-04-25T09:59:08.381357+0000","last_became_peered":"2024-04-25T09:59:08.381357+0000","last_unstale":"2024-04-25T09:59:08.389772+0000","last_undegraded":"2024-04-25T09:59:08.389772+0000","last_fullsized":"2024-04-25T09:59:08.389772+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.156095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.375632+0000","last_change":"2024-04-25T09:59:07.374478+0000","last_active":"2024-04-25T09:59:08.375632+0000","last_peered":"2024-04-25T09:59:08.375632+0000","last_clean":"2024-04-25T09:59:08.375632+0000","last_became_active":"2024-04-25T09:59:07.374171+0000","last_became_peered":"2024-04-25T09:59:07.374171+0000","last_unstale":"2024-04-25T09:59:08.375632+0000","last_undegraded":"2024-04-25T09:59:08.375632+0000","last_fullsized":"2024-04-25T09:59:08.375632+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:00.417554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383508+0000","last_change":"2024-04-25T09:59:08.379323+0000","last_active":"2024-04-25T09:59:08.383508+0000","last_peered":"2024-04-25T09:59:08.383508+0000","last_clean":"2024-04-25T09:59:08.383508+0000","last_became_active":"2024-04-25T09:59:08.378830+0000","last_became_peered":"2024-04-25T09:59:08.378830+0000","last_unstale":"2024-04-25T09:59:08.383508+0000","last_undegraded":"2024-04-25T09:59:08.383508+0000","last_fullsized":"2024-04-25T09:59:08.383508+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:30.378274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.371297+0000","last_change":"2024-04-25T09:59:07.379821+0000","last_active":"2024-04-25T09:59:08.371297+0000","last_peered":"2024-04-25T09:59:08.371297+0000","last_clean":"2024-04-25T09:59:08.371297+0000","last_became_active":"2024-04-25T09:59:07.379552+0000","last_became_peered":"2024-04-25T09:59:07.379552+0000","last_unstale":"2024-04-25T09:59:08.371297+0000","last_undegraded":"2024-04-25T09:59:08.371297+0000","last_fullsized":"2024-04-25T09:59:08.371297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:49.467983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.399502+0000","last_change":"2024-04-25T09:59:08.397446+0000","last_active":"2024-04-25T09:59:08.399502+0000","last_peered":"2024-04-25T09:59:08.399502+0000","last_clean":"2024-04-25T09:59:08.399502+0000","last_became_active":"2024-04-25T09:59:08.397179+0000","last_became_peered":"2024-04-25T09:59:08.397179+0000","last_unstale":"2024-04-25T09:59:08.399502+0000","last_undegraded":"2024-04-25T09:59:08.399502+0000","last_fullsized":"2024-04-25T09:59:08.399502+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:29.940853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.366265+0000","last_change":"2024-04-25T09:59:07.374941+0000","last_active":"2024-04-25T09:59:08.366265+0000","last_peered":"2024-04-25T09:59:08.366265+0000","last_clean":"2024-04-25T09:59:08.366265+0000","last_became_active":"2024-04-25T09:59:07.374656+0000","last_became_peered":"2024-04-25T09:59:07.374656+0000","last_unstale":"2024-04-25T09:59:08.366265+0000","last_undegraded":"2024-04-25T09:59:08.366265+0000","last_fullsized":"2024-04-25T09:59:08.366265+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:26.164653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392954+0000","last_change":"2024-04-25T09:59:08.390100+0000","last_active":"2024-04-25T09:59:08.392954+0000","last_peered":"2024-04-25T09:59:08.392954+0000","last_clean":"2024-04-25T09:59:08.392954+0000","last_became_active":"2024-04-25T09:59:08.389808+0000","last_became_peered":"2024-04-25T09:59:08.389808+0000","last_unstale":"2024-04-25T09:59:08.392954+0000","last_undegraded":"2024-04-25T09:59:08.392954+0000","last_fullsized":"2024-04-25T09:59:08.392954+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:37.411559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.367315+0000","last_change":"2024-04-25T09:59:07.382161+0000","last_active":"2024-04-25T09:59:08.367315+0000","last_peered":"2024-04-25T09:59:08.367315+0000","last_clean":"2024-04-25T09:59:08.367315+0000","last_became_active":"2024-04-25T09:59:07.381863+0000","last_became_peered":"2024-04-25T09:59:07.381863+0000","last_unstale":"2024-04-25T09:59:08.367315+0000","last_undegraded":"2024-04-25T09:59:08.367315+0000","last_fullsized":"2024-04-25T09:59:08.367315+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:24.588204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.381271+0000","last_change":"2024-04-25T09:59:08.379558+0000","last_active":"2024-04-25T09:59:08.381271+0000","last_peered":"2024-04-25T09:59:08.381271+0000","last_clean":"2024-04-25T09:59:08.381271+0000","last_became_active":"2024-04-25T09:59:08.379279+0000","last_became_peered":"2024-04-25T09:59:08.379279+0000","last_unstale":"2024-04-25T09:59:08.381271+0000","last_undegraded":"2024-04-25T09:59:08.381271+0000","last_fullsized":"2024-04-25T09:59:08.381271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:29.918692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.369442+0000","last_change":"2024-04-25T09:59:07.379230+0000","last_active":"2024-04-25T09:59:08.369442+0000","last_peered":"2024-04-25T09:59:08.369442+0000","last_clean":"2024-04-25T09:59:08.369442+0000","last_became_active":"2024-04-25T09:59:07.378954+0000","last_became_peered":"2024-04-25T09:59:07.378954+0000","last_unstale":"2024-04-25T09:59:08.369442+0000","last_undegraded":"2024-04-25T09:59:08.369442+0000","last_fullsized":"2024-04-25T09:59:08.369442+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:52.726175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.397395+0000","last_change":"2024-04-25T09:59:08.395443+0000","last_active":"2024-04-25T09:59:08.397395+0000","last_peered":"2024-04-25T09:59:08.397395+0000","last_clean":"2024-04-25T09:59:08.397395+0000","last_became_active":"2024-04-25T09:59:08.393442+0000","last_became_peered":"2024-04-25T09:59:08.393442+0000","last_unstale":"2024-04-25T09:59:08.397395+0000","last_undegraded":"2024-04-25T09:59:08.397395+0000","last_fullsized":"2024-04-25T09:59:08.397395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:13.019243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.366614+0000","last_change":"2024-04-25T09:59:07.378762+0000","last_active":"2024-04-25T09:59:08.366614+0000","last_peered":"2024-04-25T09:59:08.366614+0000","last_clean":"2024-04-25T09:59:08.366614+0000","last_became_active":"2024-04-25T09:59:07.378489+0000","last_became_peered":"2024-04-25T09:59:07.378489+0000","last_unstale":"2024-04-25T09:59:08.366614+0000","last_undegraded":"2024-04-25T09:59:08.366614+0000","last_fullsized":"2024-04-25T09:59:08.366614+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:54.423535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.380840+0000","last_change":"2024-04-25T09:59:08.376800+0000","last_active":"2024-04-25T09:59:08.380840+0000","last_peered":"2024-04-25T09:59:08.380840+0000","last_clean":"2024-04-25T09:59:08.380840+0000","last_became_active":"2024-04-25T09:59:08.376531+0000","last_became_peered":"2024-04-25T09:59:08.376531+0000","last_unstale":"2024-04-25T09:59:08.380840+0000","last_undegraded":"2024-04-25T09:59:08.380840+0000","last_fullsized":"2024-04-25T09:59:08.380840+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:40.120681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.375392+0000","last_change":"2024-04-25T09:59:07.386210+0000","last_active":"2024-04-25T09:59:08.375392+0000","last_peered":"2024-04-25T09:59:08.375392+0000","last_clean":"2024-04-25T09:59:08.375392+0000","last_became_active":"2024-04-25T09:59:07.385823+0000","last_became_peered":"2024-04-25T09:59:07.385823+0000","last_unstale":"2024-04-25T09:59:08.375392+0000","last_undegraded":"2024-04-25T09:59:08.375392+0000","last_fullsized":"2024-04-25T09:59:08.375392+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:52.726175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":321,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.366730+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:59:08.366730+0000","last_peered":"2024-04-25T09:59:08.366730+0000","last_clean":"2024-04-25T09:59:08.366730+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:59:08.366730+0000","last_undegraded":"2024-04-25T09:59:08.366730+0000","last_fullsized":"2024-04-25T09:59:08.366730+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41220,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330620,"statfs":{"total":96636764160,"available":96594554880,"internally_reserved":0,"allocated":1761280,"data_stored":1128860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40652,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331188,"statfs":{"total":96636764160,"available":96595136512,"internally_reserved":0,"allocated":1179648,"data_stored":540652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40640,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331200,"statfs":{"total":96636764160,"available":96595148800,"internally_reserved":0,"allocated":1159168,"data_stored":538436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50352,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321488,"statfs":{"total":96636764160,"available":96585203712,"internally_reserved":0,"allocated":1146880,"data_stored":531269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50360,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321480,"statfs":{"total":96636764160,"available":96585195520,"internally_reserved":0,"allocated":1159168,"data_stored":538436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46204,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325636,"statfs":{"total":96636764160,"available":96589451264,"internally_reserved":0,"allocated":1159168,"data_stored":538436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62836,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309004,"statfs":{"total":96636764160,"available":96572420096,"internally_reserved":0,"allocated":1146880,"data_stored":531269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46772,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325068,"statfs":{"total":96636764160,"available":96588869632,"internally_reserved":0,"allocated":1740800,"data_stored":1121637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:59:11.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:59:11.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.510+0000 7f120c4fc640 1 Processor -- start 2024-04-25T09:59:11.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.510+0000 7f120c4fc640 1 -- start start 2024-04-25T09:59:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120c4fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f12040d2e90 0x7f12040d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120c4fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12040d0900 0x7f12040b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120c4fc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12040b9940 con 0x7f120415dd30 2024-04-25T09:59:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120c4fc640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12040d0d40 con 0x7f12040d0900 2024-04-25T09:59:11.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120c4fc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12040d1df0 con 0x7f12040d2e90 2024-04-25T09:59:11.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120aa99640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12040d0900 0x7f12040b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120aa99640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12040d0900 0x7f12040b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41014/0 (socket says 172.21.15.19:41014) 2024-04-25T09:59:11.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120aa99640 1 -- 172.21.15.19:0/2836361448 learned_addr learned my addr 172.21.15.19:0/2836361448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:11.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f1209a97640 1 --2- 172.21.15.19:0/2836361448 >> v2:172.21.15.112:3300/0 conn(0x7f12040d2e90 0x7f12040d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120aa99640 1 -- 172.21.15.19:0/2836361448 >> v2:172.21.15.112:3300/0 conn(0x7f12040d2e90 msgr2=0x7f12040d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120aa99640 1 --2- 172.21.15.19:0/2836361448 >> v2:172.21.15.112:3300/0 conn(0x7f12040d2e90 0x7f12040d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120aa99640 1 -- 172.21.15.19:0/2836361448 >> v1:172.21.15.112:6789/0 conn(0x7f120415dd30 legacy=0x7f12040b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.511+0000 7f120aa99640 1 -- 172.21.15.19:0/2836361448 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12040d20d0 con 0x7f12040d0900 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.512+0000 7f120aa99640 1 --2- 172.21.15.19:0/2836361448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12040d0900 0x7f12040b6a40 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f12040b6540 tx=0x7f11f402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f955ec168aa993e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.512+0000 7f1209296640 1 -- 172.21.15.19:0/2836361448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4036070 con 0x7f12040d0900 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.512+0000 7f1209296640 1 -- 172.21.15.19:0/2836361448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4031600 con 0x7f12040d0900 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.512+0000 7f1209296640 1 -- 172.21.15.19:0/2836361448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f40319d0 con 0x7f12040d0900 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.513+0000 7f120c4fc640 1 -- 172.21.15.19:0/2836361448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12040d0900 msgr2=0x7f12040b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.513+0000 7f120c4fc640 1 --2- 172.21.15.19:0/2836361448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12040d0900 0x7f12040b6a40 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f12040b6540 tx=0x7f11f402a610 comp rx=0 tx=0).stop 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.513+0000 7f120c4fc640 1 -- 172.21.15.19:0/2836361448 shutdown_connections 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.513+0000 7f120c4fc640 1 --2- 172.21.15.19:0/2836361448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12040d0900 0x7f12040b6a40 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.513+0000 7f120c4fc640 1 --2- 172.21.15.19:0/2836361448 >> v2:172.21.15.112:3300/0 conn(0x7f12040d2e90 0x7f12040d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.513+0000 7f120c4fc640 1 -- 172.21.15.19:0/2836361448 >> 172.21.15.19:0/2836361448 conn(0x7f1204061c30 msgr2=0x7f120406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.513+0000 7f120c4fc640 1 -- 172.21.15.19:0/2836361448 shutdown_connections 2024-04-25T09:59:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.513+0000 7f120c4fc640 1 -- 172.21.15.19:0/2836361448 wait complete. 2024-04-25T09:59:11.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.514+0000 7f120c4fc640 1 Processor -- start 2024-04-25T09:59:11.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.514+0000 7f120c4fc640 1 -- start start 2024-04-25T09:59:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120c4fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12040d0900 0x7f120415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120c4fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12040d2e90 0x7f12040b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120c4fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f120415dd30 0x7f1204158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120c4fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12040d2970 con 0x7f120415dd30 2024-04-25T09:59:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120c4fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12040d2690 con 0x7f12040d0900 2024-04-25T09:59:11.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120c4fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12040d1a10 con 0x7f12040d2e90 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120aa99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f120415dd30 0x7f1204158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120aa99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f120415dd30 0x7f1204158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41022/0 (socket says 172.21.15.19:41022) 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120aa99640 1 -- 172.21.15.19:0/2575125974 learned_addr learned my addr 172.21.15.19:0/2575125974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120a298640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12040d0900 0x7f120415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f1209a97640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12040d2e90 0x7f12040b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120aa99640 1 -- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12040d2e90 msgr2=0x7f12040b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120aa99640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12040d2e90 0x7f12040b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120aa99640 1 -- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12040d0900 msgr2=0x7f120415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120aa99640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12040d0900 0x7f120415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120aa99640 1 -- 172.21.15.19:0/2575125974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12040b4260 con 0x7f120415dd30 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120a298640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12040d0900 0x7f120415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.515+0000 7f120aa99640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f120415dd30 0x7f1204158f40 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009f40 tx=0x7f11f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.516+0000 7f1202ffd640 1 -- 172.21.15.19:0/2575125974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4036050 con 0x7f120415dd30 2024-04-25T09:59:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.516+0000 7f1202ffd640 1 -- 172.21.15.19:0/2575125974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4042070 con 0x7f120415dd30 2024-04-25T09:59:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.516+0000 7f120c4fc640 1 -- 172.21.15.19:0/2575125974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12040b4540 con 0x7f120415dd30 2024-04-25T09:59:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.516+0000 7f1202ffd640 1 -- 172.21.15.19:0/2575125974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4007bd0 con 0x7f120415dd30 2024-04-25T09:59:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.516+0000 7f120c4fc640 1 -- 172.21.15.19:0/2575125974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1204158a20 con 0x7f120415dd30 2024-04-25T09:59:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.517+0000 7f120c4fc640 1 -- 172.21.15.19:0/2575125974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1204073990 con 0x7f120415dd30 2024-04-25T09:59:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.517+0000 7f1202ffd640 1 -- 172.21.15.19:0/2575125974 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11f4047460 con 0x7f120415dd30 2024-04-25T09:59:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.517+0000 7f1202ffd640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f11f006b4d0 0x7f11f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.518+0000 7f1202ffd640 1 -- 172.21.15.19:0/2575125974 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f11f40ab3f0 con 0x7f120415dd30 2024-04-25T09:59:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.518+0000 7f120a298640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f11f006b4d0 0x7f11f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.518+0000 7f120a298640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f11f006b4d0 0x7f11f006d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f12040b8c70 tx=0x7f11fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.522+0000 7f1202ffd640 1 -- 172.21.15.19:0/2575125974 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11f407a040 con 0x7f120415dd30 2024-04-25T09:59:11.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.699+0000 7f120c4fc640 1 -- 172.21.15.19:0/2575125974 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f12040c7170 con 0x7f11f006b4d0 2024-04-25T09:59:11.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.706+0000 7f1202ffd640 1 -- 172.21.15.19:0/2575125974 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252005 (secure 0 0 0) 0x7f12040c7170 con 0x7f11f006b4d0 2024-04-25T09:59:11.713 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:11.714 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:59:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 -- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f11f006b4d0 msgr2=0x7f11f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f11f006b4d0 0x7f11f006d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f12040b8c70 tx=0x7f11fc009040 comp rx=0 tx=0).stop 2024-04-25T09:59:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 -- 172.21.15.19:0/2575125974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f120415dd30 msgr2=0x7f1204158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f120415dd30 0x7f1204158f40 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009f40 tx=0x7f11f4004a40 comp rx=0 tx=0).stop 2024-04-25T09:59:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 -- 172.21.15.19:0/2575125974 shutdown_connections 2024-04-25T09:59:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f11f006b4d0 0x7f11f006d990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f120415dd30 0x7f1204158f40 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12040d2e90 0x7f12040b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 --2- 172.21.15.19:0/2575125974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12040d0900 0x7f120415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 -- 172.21.15.19:0/2575125974 >> 172.21.15.19:0/2575125974 conn(0x7f1204061c30 msgr2=0x7f12040c8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 -- 172.21.15.19:0/2575125974 shutdown_connections 2024-04-25T09:59:11.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.709+0000 7f120c4fc640 1 -- 172.21.15.19:0/2575125974 wait complete. 2024-04-25T09:59:11.723 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-04-25T09:59:10.994106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379036,"kb_used_data":10208,"kb_used_omap":12,"kb_used_meta":368691,"kb_avail":754595684,"statfs":{"total":773094113280,"available":772705980416,"internally_reserved":0,"allocated":10452992,"data_stored":5468995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":377540146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.632253"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.399311+0000","last_change":"2024-04-25T09:59:08.398299+0000","last_active":"2024-04-25T09:59:08.399311+0000","last_peered":"2024-04-25T09:59:08.399311+0000","last_clean":"2024-04-25T09:59:08.399311+0000","last_became_active":"2024-04-25T09:59:08.398030+0000","last_became_peered":"2024-04-25T09:59:08.398030+0000","last_unstale":"2024-04-25T09:59:08.399311+0000","last_undegraded":"2024-04-25T09:59:08.399311+0000","last_fullsized":"2024-04-25T09:59:08.399311+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:53.748733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.380727+0000","last_change":"2024-04-25T09:59:08.376005+0000","last_active":"2024-04-25T09:59:08.380727+0000","last_peered":"2024-04-25T09:59:08.380727+0000","last_clean":"2024-04-25T09:59:08.380727+0000","last_became_active":"2024-04-25T09:59:08.375340+0000","last_became_peered":"2024-04-25T09:59:08.375340+0000","last_unstale":"2024-04-25T09:59:08.380727+0000","last_undegraded":"2024-04-25T09:59:08.380727+0000","last_fullsized":"2024-04-25T09:59:08.380727+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:16.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.393238+0000","last_change":"2024-04-25T09:59:08.391855+0000","last_active":"2024-04-25T09:59:08.393238+0000","last_peered":"2024-04-25T09:59:08.393238+0000","last_clean":"2024-04-25T09:59:08.393238+0000","last_became_active":"2024-04-25T09:59:08.391575+0000","last_became_peered":"2024-04-25T09:59:08.391575+0000","last_unstale":"2024-04-25T09:59:08.393238+0000","last_undegraded":"2024-04-25T09:59:08.393238+0000","last_fullsized":"2024-04-25T09:59:08.393238+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:19.379486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.385140+0000","last_change":"2024-04-25T09:59:08.380075+0000","last_active":"2024-04-25T09:59:08.385140+0000","last_peered":"2024-04-25T09:59:08.385140+0000","last_clean":"2024-04-25T09:59:08.385140+0000","last_became_active":"2024-04-25T09:59:08.378283+0000","last_became_peered":"2024-04-25T09:59:08.378283+0000","last_unstale":"2024-04-25T09:59:08.385140+0000","last_undegraded":"2024-04-25T09:59:08.385140+0000","last_fullsized":"2024-04-25T09:59:08.385140+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:31.394397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387669+0000","last_change":"2024-04-25T09:59:08.382237+0000","last_active":"2024-04-25T09:59:08.387669+0000","last_peered":"2024-04-25T09:59:08.387669+0000","last_clean":"2024-04-25T09:59:08.387669+0000","last_became_active":"2024-04-25T09:59:08.381844+0000","last_became_peered":"2024-04-25T09:59:08.381844+0000","last_unstale":"2024-04-25T09:59:08.387669+0000","last_undegraded":"2024-04-25T09:59:08.387669+0000","last_fullsized":"2024-04-25T09:59:08.387669+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:08.149260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.391644+0000","last_change":"2024-04-25T09:59:08.380960+0000","last_active":"2024-04-25T09:59:08.391644+0000","last_peered":"2024-04-25T09:59:08.391644+0000","last_clean":"2024-04-25T09:59:08.391644+0000","last_became_active":"2024-04-25T09:59:08.380475+0000","last_became_peered":"2024-04-25T09:59:08.380475+0000","last_unstale":"2024-04-25T09:59:08.391644+0000","last_undegraded":"2024-04-25T09:59:08.391644+0000","last_fullsized":"2024-04-25T09:59:08.391644+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:08.200701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.395271+0000","last_change":"2024-04-25T09:59:08.392868+0000","last_active":"2024-04-25T09:59:08.395271+0000","last_peered":"2024-04-25T09:59:08.395271+0000","last_clean":"2024-04-25T09:59:08.395271+0000","last_became_active":"2024-04-25T09:59:08.392583+0000","last_became_peered":"2024-04-25T09:59:08.392583+0000","last_unstale":"2024-04-25T09:59:08.395271+0000","last_undegraded":"2024-04-25T09:59:08.395271+0000","last_fullsized":"2024-04-25T09:59:08.395271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:36.021887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387590+0000","last_change":"2024-04-25T09:59:08.384291+0000","last_active":"2024-04-25T09:59:08.387590+0000","last_peered":"2024-04-25T09:59:08.387590+0000","last_clean":"2024-04-25T09:59:08.387590+0000","last_became_active":"2024-04-25T09:59:08.383932+0000","last_became_peered":"2024-04-25T09:59:08.383932+0000","last_unstale":"2024-04-25T09:59:08.387590+0000","last_undegraded":"2024-04-25T09:59:08.387590+0000","last_fullsized":"2024-04-25T09:59:08.387590+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:40.629470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.386626+0000","last_change":"2024-04-25T09:59:08.382131+0000","last_active":"2024-04-25T09:59:08.386626+0000","last_peered":"2024-04-25T09:59:08.386626+0000","last_clean":"2024-04-25T09:59:08.386626+0000","last_became_active":"2024-04-25T09:59:08.381820+0000","last_became_peered":"2024-04-25T09:59:08.381820+0000","last_unstale":"2024-04-25T09:59:08.386626+0000","last_undegraded":"2024-04-25T09:59:08.386626+0000","last_fullsized":"2024-04-25T09:59:08.386626+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:43.402769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392532+0000","last_change":"2024-04-25T09:59:08.389725+0000","last_active":"2024-04-25T09:59:08.392532+0000","last_peered":"2024-04-25T09:59:08.392532+0000","last_clean":"2024-04-25T09:59:08.392532+0000","last_became_active":"2024-04-25T09:59:08.389421+0000","last_became_peered":"2024-04-25T09:59:08.389421+0000","last_unstale":"2024-04-25T09:59:08.392532+0000","last_undegraded":"2024-04-25T09:59:08.392532+0000","last_fullsized":"2024-04-25T09:59:08.392532+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:42.426714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.388981+0000","last_change":"2024-04-25T09:59:08.382381+0000","last_active":"2024-04-25T09:59:08.388981+0000","last_peered":"2024-04-25T09:59:08.388981+0000","last_clean":"2024-04-25T09:59:08.388981+0000","last_became_active":"2024-04-25T09:59:08.382063+0000","last_became_peered":"2024-04-25T09:59:08.382063+0000","last_unstale":"2024-04-25T09:59:08.388981+0000","last_undegraded":"2024-04-25T09:59:08.388981+0000","last_fullsized":"2024-04-25T09:59:08.388981+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:04.838824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.388082+0000","last_change":"2024-04-25T09:59:08.385871+0000","last_active":"2024-04-25T09:59:08.388082+0000","last_peered":"2024-04-25T09:59:08.388082+0000","last_clean":"2024-04-25T09:59:08.388082+0000","last_became_active":"2024-04-25T09:59:08.385567+0000","last_became_peered":"2024-04-25T09:59:08.385567+0000","last_unstale":"2024-04-25T09:59:08.388082+0000","last_undegraded":"2024-04-25T09:59:08.388082+0000","last_fullsized":"2024-04-25T09:59:08.388082+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:24.363765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.385193+0000","last_change":"2024-04-25T09:59:08.379637+0000","last_active":"2024-04-25T09:59:08.385193+0000","last_peered":"2024-04-25T09:59:08.385193+0000","last_clean":"2024-04-25T09:59:08.385193+0000","last_became_active":"2024-04-25T09:59:08.377345+0000","last_became_peered":"2024-04-25T09:59:08.377345+0000","last_unstale":"2024-04-25T09:59:08.385193+0000","last_undegraded":"2024-04-25T09:59:08.385193+0000","last_fullsized":"2024-04-25T09:59:08.385193+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:40.564913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389557+0000","last_change":"2024-04-25T09:59:08.385458+0000","last_active":"2024-04-25T09:59:08.389557+0000","last_peered":"2024-04-25T09:59:08.389557+0000","last_clean":"2024-04-25T09:59:08.389557+0000","last_became_active":"2024-04-25T09:59:08.385146+0000","last_became_peered":"2024-04-25T09:59:08.385146+0000","last_unstale":"2024-04-25T09:59:08.389557+0000","last_undegraded":"2024-04-25T09:59:08.389557+0000","last_fullsized":"2024-04-25T09:59:08.389557+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:40.564913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390978+0000","last_change":"2024-04-25T09:59:08.387926+0000","last_active":"2024-04-25T09:59:08.390978+0000","last_peered":"2024-04-25T09:59:08.390978+0000","last_clean":"2024-04-25T09:59:08.390978+0000","last_became_active":"2024-04-25T09:59:08.387621+0000","last_became_peered":"2024-04-25T09:59:08.387621+0000","last_unstale":"2024-04-25T09:59:08.390978+0000","last_undegraded":"2024-04-25T09:59:08.390978+0000","last_fullsized":"2024-04-25T09:59:08.390978+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:51.099297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.397468+0000","last_change":"2024-04-25T09:59:08.394954+0000","last_active":"2024-04-25T09:59:08.397468+0000","last_peered":"2024-04-25T09:59:08.397468+0000","last_clean":"2024-04-25T09:59:08.397468+0000","last_became_active":"2024-04-25T09:59:08.394436+0000","last_became_peered":"2024-04-25T09:59:08.394436+0000","last_unstale":"2024-04-25T09:59:08.397468+0000","last_undegraded":"2024-04-25T09:59:08.397468+0000","last_fullsized":"2024-04-25T09:59:08.397468+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:21.951308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.396896+0000","last_change":"2024-04-25T09:59:08.394184+0000","last_active":"2024-04-25T09:59:08.396896+0000","last_peered":"2024-04-25T09:59:08.396896+0000","last_clean":"2024-04-25T09:59:08.396896+0000","last_became_active":"2024-04-25T09:59:08.393908+0000","last_became_peered":"2024-04-25T09:59:08.393908+0000","last_unstale":"2024-04-25T09:59:08.396896+0000","last_undegraded":"2024-04-25T09:59:08.396896+0000","last_fullsized":"2024-04-25T09:59:08.396896+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:00.996221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392982+0000","last_change":"2024-04-25T09:59:08.391159+0000","last_active":"2024-04-25T09:59:08.392982+0000","last_peered":"2024-04-25T09:59:08.392982+0000","last_clean":"2024-04-25T09:59:08.392982+0000","last_became_active":"2024-04-25T09:59:08.390893+0000","last_became_peered":"2024-04-25T09:59:08.390893+0000","last_unstale":"2024-04-25T09:59:08.392982+0000","last_undegraded":"2024-04-25T09:59:08.392982+0000","last_fullsized":"2024-04-25T09:59:08.392982+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:35.879053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389453+0000","last_change":"2024-04-25T09:59:08.384613+0000","last_active":"2024-04-25T09:59:08.389453+0000","last_peered":"2024-04-25T09:59:08.389453+0000","last_clean":"2024-04-25T09:59:08.389453+0000","last_became_active":"2024-04-25T09:59:08.384340+0000","last_became_peered":"2024-04-25T09:59:08.384340+0000","last_unstale":"2024-04-25T09:59:08.389453+0000","last_undegraded":"2024-04-25T09:59:08.389453+0000","last_fullsized":"2024-04-25T09:59:08.389453+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:36.989614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387336+0000","last_change":"2024-04-25T09:59:08.385910+0000","last_active":"2024-04-25T09:59:08.387336+0000","last_peered":"2024-04-25T09:59:08.387336+0000","last_clean":"2024-04-25T09:59:08.387336+0000","last_became_active":"2024-04-25T09:59:08.385641+0000","last_became_peered":"2024-04-25T09:59:08.385641+0000","last_unstale":"2024-04-25T09:59:08.387336+0000","last_undegraded":"2024-04-25T09:59:08.387336+0000","last_fullsized":"2024-04-25T09:59:08.387336+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:35.939258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387690+0000","last_change":"2024-04-25T09:59:08.383111+0000","last_active":"2024-04-25T09:59:08.387690+0000","last_peered":"2024-04-25T09:59:08.387690+0000","last_clean":"2024-04-25T09:59:08.387690+0000","last_became_active":"2024-04-25T09:59:08.382805+0000","last_became_peered":"2024-04-25T09:59:08.382805+0000","last_unstale":"2024-04-25T09:59:08.387690+0000","last_undegraded":"2024-04-25T09:59:08.387690+0000","last_fullsized":"2024-04-25T09:59:08.387690+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:57.024245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383778+0000","last_change":"2024-04-25T09:59:08.379959+0000","last_active":"2024-04-25T09:59:08.383778+0000","last_peered":"2024-04-25T09:59:08.383778+0000","last_clean":"2024-04-25T09:59:08.383778+0000","last_became_active":"2024-04-25T09:59:08.379625+0000","last_became_peered":"2024-04-25T09:59:08.379625+0000","last_unstale":"2024-04-25T09:59:08.383778+0000","last_undegraded":"2024-04-25T09:59:08.383778+0000","last_fullsized":"2024-04-25T09:59:08.383778+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:46.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.381259+0000","last_change":"2024-04-25T09:59:08.380204+0000","last_active":"2024-04-25T09:59:08.381259+0000","last_peered":"2024-04-25T09:59:08.381259+0000","last_clean":"2024-04-25T09:59:08.381259+0000","last_became_active":"2024-04-25T09:59:08.379933+0000","last_became_peered":"2024-04-25T09:59:08.379933+0000","last_unstale":"2024-04-25T09:59:08.381259+0000","last_undegraded":"2024-04-25T09:59:08.381259+0000","last_fullsized":"2024-04-25T09:59:08.381259+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:40.889027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.385534+0000","last_change":"2024-04-25T09:59:08.384253+0000","last_active":"2024-04-25T09:59:08.385534+0000","last_peered":"2024-04-25T09:59:08.385534+0000","last_clean":"2024-04-25T09:59:08.385534+0000","last_became_active":"2024-04-25T09:59:08.383960+0000","last_became_peered":"2024-04-25T09:59:08.383960+0000","last_unstale":"2024-04-25T09:59:08.385534+0000","last_undegraded":"2024-04-25T09:59:08.385534+0000","last_fullsized":"2024-04-25T09:59:08.385534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:50.966310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.374063+0000","last_change":"2024-04-25T09:59:08.370958+0000","last_active":"2024-04-25T09:59:08.374063+0000","last_peered":"2024-04-25T09:59:08.374063+0000","last_clean":"2024-04-25T09:59:08.374063+0000","last_became_active":"2024-04-25T09:59:08.370652+0000","last_became_peered":"2024-04-25T09:59:08.370652+0000","last_unstale":"2024-04-25T09:59:08.374063+0000","last_undegraded":"2024-04-25T09:59:08.374063+0000","last_fullsized":"2024-04-25T09:59:08.374063+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:12.078167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389508+0000","last_change":"2024-04-25T09:59:08.385454+0000","last_active":"2024-04-25T09:59:08.389508+0000","last_peered":"2024-04-25T09:59:08.389508+0000","last_clean":"2024-04-25T09:59:08.389508+0000","last_became_active":"2024-04-25T09:59:08.385070+0000","last_became_peered":"2024-04-25T09:59:08.385070+0000","last_unstale":"2024-04-25T09:59:08.389508+0000","last_undegraded":"2024-04-25T09:59:08.389508+0000","last_fullsized":"2024-04-25T09:59:08.389508+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:50.470219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.381771+0000","last_change":"2024-04-25T09:59:08.379785+0000","last_active":"2024-04-25T09:59:08.381771+0000","last_peered":"2024-04-25T09:59:08.381771+0000","last_clean":"2024-04-25T09:59:08.381771+0000","last_became_active":"2024-04-25T09:59:08.379471+0000","last_became_peered":"2024-04-25T09:59:08.379471+0000","last_unstale":"2024-04-25T09:59:08.381771+0000","last_undegraded":"2024-04-25T09:59:08.381771+0000","last_fullsized":"2024-04-25T09:59:08.381771+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:47.499730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.386043+0000","last_change":"2024-04-25T09:59:08.384162+0000","last_active":"2024-04-25T09:59:08.386043+0000","last_peered":"2024-04-25T09:59:08.386043+0000","last_clean":"2024-04-25T09:59:08.386043+0000","last_became_active":"2024-04-25T09:59:08.383852+0000","last_became_peered":"2024-04-25T09:59:08.383852+0000","last_unstale":"2024-04-25T09:59:08.386043+0000","last_undegraded":"2024-04-25T09:59:08.386043+0000","last_fullsized":"2024-04-25T09:59:08.386043+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:40.889027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387724+0000","last_change":"2024-04-25T09:59:08.381228+0000","last_active":"2024-04-25T09:59:08.387724+0000","last_peered":"2024-04-25T09:59:08.387724+0000","last_clean":"2024-04-25T09:59:08.387724+0000","last_became_active":"2024-04-25T09:59:08.380934+0000","last_became_peered":"2024-04-25T09:59:08.380934+0000","last_unstale":"2024-04-25T09:59:08.387724+0000","last_undegraded":"2024-04-25T09:59:08.387724+0000","last_fullsized":"2024-04-25T09:59:08.387724+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:51.221958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.386653+0000","last_change":"2024-04-25T09:59:08.381918+0000","last_active":"2024-04-25T09:59:08.386653+0000","last_peered":"2024-04-25T09:59:08.386653+0000","last_clean":"2024-04-25T09:59:08.386653+0000","last_became_active":"2024-04-25T09:59:08.381322+0000","last_became_peered":"2024-04-25T09:59:08.381322+0000","last_unstale":"2024-04-25T09:59:08.386653+0000","last_undegraded":"2024-04-25T09:59:08.386653+0000","last_fullsized":"2024-04-25T09:59:08.386653+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:25.226164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383568+0000","last_change":"2024-04-25T09:59:08.379238+0000","last_active":"2024-04-25T09:59:08.383568+0000","last_peered":"2024-04-25T09:59:08.383568+0000","last_clean":"2024-04-25T09:59:08.383568+0000","last_became_active":"2024-04-25T09:59:08.378761+0000","last_became_peered":"2024-04-25T09:59:08.378761+0000","last_unstale":"2024-04-25T09:59:08.383568+0000","last_undegraded":"2024-04-25T09:59:08.383568+0000","last_fullsized":"2024-04-25T09:59:08.383568+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:06.141494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.397446+0000","last_change":"2024-04-25T09:59:08.394715+0000","last_active":"2024-04-25T09:59:08.397446+0000","last_peered":"2024-04-25T09:59:08.397446+0000","last_clean":"2024-04-25T09:59:08.397446+0000","last_became_active":"2024-04-25T09:59:08.394044+0000","last_became_peered":"2024-04-25T09:59:08.394044+0000","last_unstale":"2024-04-25T09:59:08.397446+0000","last_undegraded":"2024-04-25T09:59:08.397446+0000","last_fullsized":"2024-04-25T09:59:08.397446+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:48.624669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.396096+0000","last_change":"2024-04-25T09:59:08.387696+0000","last_active":"2024-04-25T09:59:08.396096+0000","last_peered":"2024-04-25T09:59:08.396096+0000","last_clean":"2024-04-25T09:59:08.396096+0000","last_became_active":"2024-04-25T09:59:08.387398+0000","last_became_peered":"2024-04-25T09:59:08.387398+0000","last_unstale":"2024-04-25T09:59:08.396096+0000","last_undegraded":"2024-04-25T09:59:08.396096+0000","last_fullsized":"2024-04-25T09:59:08.396096+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:30.941125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"94'4","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.400744+0000","last_change":"2024-04-25T09:59:07.387484+0000","last_active":"2024-04-25T09:59:08.400744+0000","last_peered":"2024-04-25T09:59:08.400744+0000","last_clean":"2024-04-25T09:59:08.400744+0000","last_became_active":"2024-04-25T09:59:07.386943+0000","last_became_peered":"2024-04-25T09:59:07.386943+0000","last_unstale":"2024-04-25T09:59:08.400744+0000","last_undegraded":"2024-04-25T09:59:08.400744+0000","last_fullsized":"2024-04-25T09:59:08.400744+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:03.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389417+0000","last_change":"2024-04-25T09:59:08.382317+0000","last_active":"2024-04-25T09:59:08.389417+0000","last_peered":"2024-04-25T09:59:08.389417+0000","last_clean":"2024-04-25T09:59:08.389417+0000","last_became_active":"2024-04-25T09:59:08.382047+0000","last_became_peered":"2024-04-25T09:59:08.382047+0000","last_unstale":"2024-04-25T09:59:08.389417+0000","last_undegraded":"2024-04-25T09:59:08.389417+0000","last_fullsized":"2024-04-25T09:59:08.389417+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:11.798267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390050+0000","last_change":"2024-04-25T09:59:07.376083+0000","last_active":"2024-04-25T09:59:08.390050+0000","last_peered":"2024-04-25T09:59:08.390050+0000","last_clean":"2024-04-25T09:59:08.390050+0000","last_became_active":"2024-04-25T09:59:07.375195+0000","last_became_peered":"2024-04-25T09:59:07.375195+0000","last_unstale":"2024-04-25T09:59:08.390050+0000","last_undegraded":"2024-04-25T09:59:08.390050+0000","last_fullsized":"2024-04-25T09:59:08.390050+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:37.624313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.384698+0000","last_change":"2024-04-25T09:59:08.380359+0000","last_active":"2024-04-25T09:59:08.384698+0000","last_peered":"2024-04-25T09:59:08.384698+0000","last_clean":"2024-04-25T09:59:08.384698+0000","last_became_active":"2024-04-25T09:59:08.380051+0000","last_became_peered":"2024-04-25T09:59:08.380051+0000","last_unstale":"2024-04-25T09:59:08.384698+0000","last_undegraded":"2024-04-25T09:59:08.384698+0000","last_fullsized":"2024-04-25T09:59:08.384698+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:52.908151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390934+0000","last_change":"2024-04-25T09:59:07.374210+0000","last_active":"2024-04-25T09:59:08.390934+0000","last_peered":"2024-04-25T09:59:08.390934+0000","last_clean":"2024-04-25T09:59:08.390934+0000","last_became_active":"2024-04-25T09:59:07.373928+0000","last_became_peered":"2024-04-25T09:59:07.373928+0000","last_unstale":"2024-04-25T09:59:08.390934+0000","last_undegraded":"2024-04-25T09:59:08.390934+0000","last_fullsized":"2024-04-25T09:59:08.390934+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:00.551940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.391677+0000","last_change":"2024-04-25T09:59:08.379002+0000","last_active":"2024-04-25T09:59:08.391677+0000","last_peered":"2024-04-25T09:59:08.391677+0000","last_clean":"2024-04-25T09:59:08.391677+0000","last_became_active":"2024-04-25T09:59:08.378721+0000","last_became_peered":"2024-04-25T09:59:08.378721+0000","last_unstale":"2024-04-25T09:59:08.391677+0000","last_undegraded":"2024-04-25T09:59:08.391677+0000","last_fullsized":"2024-04-25T09:59:08.391677+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:56.991323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.395201+0000","last_change":"2024-04-25T09:59:07.387257+0000","last_active":"2024-04-25T09:59:08.395201+0000","last_peered":"2024-04-25T09:59:08.395201+0000","last_clean":"2024-04-25T09:59:08.395201+0000","last_became_active":"2024-04-25T09:59:07.386974+0000","last_became_peered":"2024-04-25T09:59:07.386974+0000","last_unstale":"2024-04-25T09:59:08.395201+0000","last_undegraded":"2024-04-25T09:59:08.395201+0000","last_fullsized":"2024-04-25T09:59:08.395201+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:35.987378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.397788+0000","last_change":"2024-04-25T09:59:08.388723+0000","last_active":"2024-04-25T09:59:08.397788+0000","last_peered":"2024-04-25T09:59:08.397788+0000","last_clean":"2024-04-25T09:59:08.397788+0000","last_became_active":"2024-04-25T09:59:08.388284+0000","last_became_peered":"2024-04-25T09:59:08.388284+0000","last_unstale":"2024-04-25T09:59:08.397788+0000","last_undegraded":"2024-04-25T09:59:08.397788+0000","last_fullsized":"2024-04-25T09:59:08.397788+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.520303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390182+0000","last_change":"2024-04-25T09:59:07.372735+0000","last_active":"2024-04-25T09:59:08.390182+0000","last_peered":"2024-04-25T09:59:08.390182+0000","last_clean":"2024-04-25T09:59:08.390182+0000","last_became_active":"2024-04-25T09:59:07.372449+0000","last_became_peered":"2024-04-25T09:59:07.372449+0000","last_unstale":"2024-04-25T09:59:08.390182+0000","last_undegraded":"2024-04-25T09:59:08.390182+0000","last_fullsized":"2024-04-25T09:59:08.390182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:48.345862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389209+0000","last_change":"2024-04-25T09:59:08.383430+0000","last_active":"2024-04-25T09:59:08.389209+0000","last_peered":"2024-04-25T09:59:08.389209+0000","last_clean":"2024-04-25T09:59:08.389209+0000","last_became_active":"2024-04-25T09:59:08.382898+0000","last_became_peered":"2024-04-25T09:59:08.382898+0000","last_unstale":"2024-04-25T09:59:08.389209+0000","last_undegraded":"2024-04-25T09:59:08.389209+0000","last_fullsized":"2024-04-25T09:59:08.389209+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:40.629470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.395465+0000","last_change":"2024-04-25T09:59:07.383660+0000","last_active":"2024-04-25T09:59:08.395465+0000","last_peered":"2024-04-25T09:59:08.395465+0000","last_clean":"2024-04-25T09:59:08.395465+0000","last_became_active":"2024-04-25T09:59:07.383362+0000","last_became_peered":"2024-04-25T09:59:07.383362+0000","last_unstale":"2024-04-25T09:59:08.395465+0000","last_undegraded":"2024-04-25T09:59:08.395465+0000","last_fullsized":"2024-04-25T09:59:08.395465+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:03.836587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.391619+0000","last_change":"2024-04-25T09:59:08.380730+0000","last_active":"2024-04-25T09:59:08.391619+0000","last_peered":"2024-04-25T09:59:08.391619+0000","last_clean":"2024-04-25T09:59:08.391619+0000","last_became_active":"2024-04-25T09:59:08.380133+0000","last_became_peered":"2024-04-25T09:59:08.380133+0000","last_unstale":"2024-04-25T09:59:08.391619+0000","last_undegraded":"2024-04-25T09:59:08.391619+0000","last_fullsized":"2024-04-25T09:59:08.391619+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:02.462568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.381809+0000","last_change":"2024-04-25T09:59:07.380367+0000","last_active":"2024-04-25T09:59:08.381809+0000","last_peered":"2024-04-25T09:59:08.381809+0000","last_clean":"2024-04-25T09:59:08.381809+0000","last_became_active":"2024-04-25T09:59:07.380062+0000","last_became_peered":"2024-04-25T09:59:07.380062+0000","last_unstale":"2024-04-25T09:59:08.381809+0000","last_undegraded":"2024-04-25T09:59:08.381809+0000","last_fullsized":"2024-04-25T09:59:08.381809+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:29.376037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.385173+0000","last_change":"2024-04-25T09:59:08.378461+0000","last_active":"2024-04-25T09:59:08.385173+0000","last_peered":"2024-04-25T09:59:08.385173+0000","last_clean":"2024-04-25T09:59:08.385173+0000","last_became_active":"2024-04-25T09:59:08.377804+0000","last_became_peered":"2024-04-25T09:59:08.377804+0000","last_unstale":"2024-04-25T09:59:08.385173+0000","last_undegraded":"2024-04-25T09:59:08.385173+0000","last_fullsized":"2024-04-25T09:59:08.385173+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:50.470219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.366467+0000","last_change":"2024-04-25T09:59:07.382231+0000","last_active":"2024-04-25T09:59:08.366467+0000","last_peered":"2024-04-25T09:59:08.366467+0000","last_clean":"2024-04-25T09:59:08.366467+0000","last_became_active":"2024-04-25T09:59:07.381450+0000","last_became_peered":"2024-04-25T09:59:07.381450+0000","last_unstale":"2024-04-25T09:59:08.366467+0000","last_undegraded":"2024-04-25T09:59:08.366467+0000","last_fullsized":"2024-04-25T09:59:08.366467+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:59.759144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383882+0000","last_change":"2024-04-25T09:59:08.378636+0000","last_active":"2024-04-25T09:59:08.383882+0000","last_peered":"2024-04-25T09:59:08.383882+0000","last_clean":"2024-04-25T09:59:08.383882+0000","last_became_active":"2024-04-25T09:59:08.378184+0000","last_became_peered":"2024-04-25T09:59:08.378184+0000","last_unstale":"2024-04-25T09:59:08.383882+0000","last_undegraded":"2024-04-25T09:59:08.383882+0000","last_fullsized":"2024-04-25T09:59:08.383882+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:49.324390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.367861+0000","last_change":"2024-04-25T09:59:07.379106+0000","last_active":"2024-04-25T09:59:08.367861+0000","last_peered":"2024-04-25T09:59:08.367861+0000","last_clean":"2024-04-25T09:59:08.367861+0000","last_became_active":"2024-04-25T09:59:07.378811+0000","last_became_peered":"2024-04-25T09:59:07.378811+0000","last_unstale":"2024-04-25T09:59:08.367861+0000","last_undegraded":"2024-04-25T09:59:08.367861+0000","last_fullsized":"2024-04-25T09:59:08.367861+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:17.563606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387424+0000","last_change":"2024-04-25T09:59:08.383214+0000","last_active":"2024-04-25T09:59:08.387424+0000","last_peered":"2024-04-25T09:59:08.387424+0000","last_clean":"2024-04-25T09:59:08.387424+0000","last_became_active":"2024-04-25T09:59:08.382906+0000","last_became_peered":"2024-04-25T09:59:08.382906+0000","last_unstale":"2024-04-25T09:59:08.387424+0000","last_undegraded":"2024-04-25T09:59:08.387424+0000","last_fullsized":"2024-04-25T09:59:08.387424+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:09.753812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390645+0000","last_change":"2024-04-25T09:59:07.387919+0000","last_active":"2024-04-25T09:59:08.390645+0000","last_peered":"2024-04-25T09:59:08.390645+0000","last_clean":"2024-04-25T09:59:08.390645+0000","last_became_active":"2024-04-25T09:59:07.387590+0000","last_became_peered":"2024-04-25T09:59:07.387590+0000","last_unstale":"2024-04-25T09:59:08.390645+0000","last_undegraded":"2024-04-25T09:59:08.390645+0000","last_fullsized":"2024-04-25T09:59:08.390645+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:20.158839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.398662+0000","last_change":"2024-04-25T09:59:08.397792+0000","last_active":"2024-04-25T09:59:08.398662+0000","last_peered":"2024-04-25T09:59:08.398662+0000","last_clean":"2024-04-25T09:59:08.398662+0000","last_became_active":"2024-04-25T09:59:08.397482+0000","last_became_peered":"2024-04-25T09:59:08.397482+0000","last_unstale":"2024-04-25T09:59:08.398662+0000","last_undegraded":"2024-04-25T09:59:08.398662+0000","last_fullsized":"2024-04-25T09:59:08.398662+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:00.761380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.396726+0000","last_change":"2024-04-25T09:59:07.381794+0000","last_active":"2024-04-25T09:59:08.396726+0000","last_peered":"2024-04-25T09:59:08.396726+0000","last_clean":"2024-04-25T09:59:08.396726+0000","last_became_active":"2024-04-25T09:59:07.381451+0000","last_became_peered":"2024-04-25T09:59:07.381451+0000","last_unstale":"2024-04-25T09:59:08.396726+0000","last_undegraded":"2024-04-25T09:59:08.396726+0000","last_fullsized":"2024-04-25T09:59:08.396726+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:56.763080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.385342+0000","last_change":"2024-04-25T09:59:08.382317+0000","last_active":"2024-04-25T09:59:08.385342+0000","last_peered":"2024-04-25T09:59:08.385342+0000","last_clean":"2024-04-25T09:59:08.385342+0000","last_became_active":"2024-04-25T09:59:08.382044+0000","last_became_peered":"2024-04-25T09:59:08.382044+0000","last_unstale":"2024-04-25T09:59:08.385342+0000","last_undegraded":"2024-04-25T09:59:08.385342+0000","last_fullsized":"2024-04-25T09:59:08.385342+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:29.918692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.402536+0000","last_change":"2024-04-25T09:59:07.384024+0000","last_active":"2024-04-25T09:59:08.402536+0000","last_peered":"2024-04-25T09:59:08.402536+0000","last_clean":"2024-04-25T09:59:08.402536+0000","last_became_active":"2024-04-25T09:59:07.383613+0000","last_became_peered":"2024-04-25T09:59:07.383613+0000","last_unstale":"2024-04-25T09:59:08.402536+0000","last_undegraded":"2024-04-25T09:59:08.402536+0000","last_fullsized":"2024-04-25T09:59:08.402536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:40.703547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.384457+0000","last_change":"2024-04-25T09:59:08.377089+0000","last_active":"2024-04-25T09:59:08.384457+0000","last_peered":"2024-04-25T09:59:08.384457+0000","last_clean":"2024-04-25T09:59:08.384457+0000","last_became_active":"2024-04-25T09:59:08.376764+0000","last_became_peered":"2024-04-25T09:59:08.376764+0000","last_unstale":"2024-04-25T09:59:08.384457+0000","last_undegraded":"2024-04-25T09:59:08.384457+0000","last_fullsized":"2024-04-25T09:59:08.384457+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:36.989614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.397753+0000","last_change":"2024-04-25T09:59:07.375328+0000","last_active":"2024-04-25T09:59:08.397753+0000","last_peered":"2024-04-25T09:59:08.397753+0000","last_clean":"2024-04-25T09:59:08.397753+0000","last_became_active":"2024-04-25T09:59:07.375041+0000","last_became_peered":"2024-04-25T09:59:07.375041+0000","last_unstale":"2024-04-25T09:59:08.397753+0000","last_undegraded":"2024-04-25T09:59:08.397753+0000","last_fullsized":"2024-04-25T09:59:08.397753+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:34.876817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383773+0000","last_change":"2024-04-25T09:59:08.378356+0000","last_active":"2024-04-25T09:59:08.383773+0000","last_peered":"2024-04-25T09:59:08.383773+0000","last_clean":"2024-04-25T09:59:08.383773+0000","last_became_active":"2024-04-25T09:59:08.378075+0000","last_became_peered":"2024-04-25T09:59:08.378075+0000","last_unstale":"2024-04-25T09:59:08.383773+0000","last_undegraded":"2024-04-25T09:59:08.383773+0000","last_fullsized":"2024-04-25T09:59:08.383773+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:52.269207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.367770+0000","last_change":"2024-04-25T09:59:07.374251+0000","last_active":"2024-04-25T09:59:08.367770+0000","last_peered":"2024-04-25T09:59:08.367770+0000","last_clean":"2024-04-25T09:59:08.367770+0000","last_became_active":"2024-04-25T09:59:07.373928+0000","last_became_peered":"2024-04-25T09:59:07.373928+0000","last_unstale":"2024-04-25T09:59:08.367770+0000","last_undegraded":"2024-04-25T09:59:08.367770+0000","last_fullsized":"2024-04-25T09:59:08.367770+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:43.226355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.387989+0000","last_change":"2024-04-25T09:59:08.387063+0000","last_active":"2024-04-25T09:59:08.387989+0000","last_peered":"2024-04-25T09:59:08.387989+0000","last_clean":"2024-04-25T09:59:08.387989+0000","last_became_active":"2024-04-25T09:59:08.386749+0000","last_became_peered":"2024-04-25T09:59:08.386749+0000","last_unstale":"2024-04-25T09:59:08.387989+0000","last_undegraded":"2024-04-25T09:59:08.387989+0000","last_fullsized":"2024-04-25T09:59:08.387989+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:17.388048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.376405+0000","last_change":"2024-04-25T09:59:07.386944+0000","last_active":"2024-04-25T09:59:08.376405+0000","last_peered":"2024-04-25T09:59:08.376405+0000","last_clean":"2024-04-25T09:59:08.376405+0000","last_became_active":"2024-04-25T09:59:07.386625+0000","last_became_peered":"2024-04-25T09:59:07.386625+0000","last_unstale":"2024-04-25T09:59:08.376405+0000","last_undegraded":"2024-04-25T09:59:08.376405+0000","last_fullsized":"2024-04-25T09:59:08.376405+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:56.022008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.388023+0000","last_change":"2024-04-25T09:59:08.387077+0000","last_active":"2024-04-25T09:59:08.388023+0000","last_peered":"2024-04-25T09:59:08.388023+0000","last_clean":"2024-04-25T09:59:08.388023+0000","last_became_active":"2024-04-25T09:59:08.386810+0000","last_became_peered":"2024-04-25T09:59:08.386810+0000","last_unstale":"2024-04-25T09:59:08.388023+0000","last_undegraded":"2024-04-25T09:59:08.388023+0000","last_fullsized":"2024-04-25T09:59:08.388023+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:16.380411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.399849+0000","last_change":"2024-04-25T09:59:07.385026+0000","last_active":"2024-04-25T09:59:08.399849+0000","last_peered":"2024-04-25T09:59:08.399849+0000","last_clean":"2024-04-25T09:59:08.399849+0000","last_became_active":"2024-04-25T09:59:07.381589+0000","last_became_peered":"2024-04-25T09:59:07.381589+0000","last_unstale":"2024-04-25T09:59:08.399849+0000","last_undegraded":"2024-04-25T09:59:08.399849+0000","last_fullsized":"2024-04-25T09:59:08.399849+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:07.147024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.391797+0000","last_change":"2024-04-25T09:59:08.379283+0000","last_active":"2024-04-25T09:59:08.391797+0000","last_peered":"2024-04-25T09:59:08.391797+0000","last_clean":"2024-04-25T09:59:08.391797+0000","last_became_active":"2024-04-25T09:59:08.379018+0000","last_became_peered":"2024-04-25T09:59:08.379018+0000","last_unstale":"2024-04-25T09:59:08.391797+0000","last_undegraded":"2024-04-25T09:59:08.391797+0000","last_fullsized":"2024-04-25T09:59:08.391797+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:27.434412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392338+0000","last_change":"2024-04-25T09:59:07.377488+0000","last_active":"2024-04-25T09:59:08.392338+0000","last_peered":"2024-04-25T09:59:08.392338+0000","last_clean":"2024-04-25T09:59:08.392338+0000","last_became_active":"2024-04-25T09:59:07.377084+0000","last_became_peered":"2024-04-25T09:59:07.377084+0000","last_unstale":"2024-04-25T09:59:08.392338+0000","last_undegraded":"2024-04-25T09:59:08.392338+0000","last_fullsized":"2024-04-25T09:59:08.392338+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:56.022008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.375484+0000","last_change":"2024-04-25T09:59:08.371460+0000","last_active":"2024-04-25T09:59:08.375484+0000","last_peered":"2024-04-25T09:59:08.375484+0000","last_clean":"2024-04-25T09:59:08.375484+0000","last_became_active":"2024-04-25T09:59:08.371199+0000","last_became_peered":"2024-04-25T09:59:08.371199+0000","last_unstale":"2024-04-25T09:59:08.375484+0000","last_undegraded":"2024-04-25T09:59:08.375484+0000","last_fullsized":"2024-04-25T09:59:08.375484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:40.564913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392739+0000","last_change":"2024-04-25T09:59:07.379302+0000","last_active":"2024-04-25T09:59:08.392739+0000","last_peered":"2024-04-25T09:59:08.392739+0000","last_clean":"2024-04-25T09:59:08.392739+0000","last_became_active":"2024-04-25T09:59:07.379019+0000","last_became_peered":"2024-04-25T09:59:07.379019+0000","last_unstale":"2024-04-25T09:59:08.392739+0000","last_undegraded":"2024-04-25T09:59:08.392739+0000","last_fullsized":"2024-04-25T09:59:08.392739+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:18.377249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.386881+0000","last_change":"2024-04-25T09:59:08.382384+0000","last_active":"2024-04-25T09:59:08.386881+0000","last_peered":"2024-04-25T09:59:08.386881+0000","last_clean":"2024-04-25T09:59:08.386881+0000","last_became_active":"2024-04-25T09:59:08.381626+0000","last_became_peered":"2024-04-25T09:59:08.381626+0000","last_unstale":"2024-04-25T09:59:08.386881+0000","last_undegraded":"2024-04-25T09:59:08.386881+0000","last_fullsized":"2024-04-25T09:59:08.386881+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:53.728412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392429+0000","last_change":"2024-04-25T09:59:07.386644+0000","last_active":"2024-04-25T09:59:08.392429+0000","last_peered":"2024-04-25T09:59:08.392429+0000","last_clean":"2024-04-25T09:59:08.392429+0000","last_became_active":"2024-04-25T09:59:07.386411+0000","last_became_peered":"2024-04-25T09:59:07.386411+0000","last_unstale":"2024-04-25T09:59:08.392429+0000","last_undegraded":"2024-04-25T09:59:08.392429+0000","last_fullsized":"2024-04-25T09:59:08.392429+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:20.040338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.395600+0000","last_change":"2024-04-25T09:59:08.388679+0000","last_active":"2024-04-25T09:59:08.395600+0000","last_peered":"2024-04-25T09:59:08.395600+0000","last_clean":"2024-04-25T09:59:08.395600+0000","last_became_active":"2024-04-25T09:59:08.388421+0000","last_became_peered":"2024-04-25T09:59:08.388421+0000","last_unstale":"2024-04-25T09:59:08.395600+0000","last_undegraded":"2024-04-25T09:59:08.395600+0000","last_fullsized":"2024-04-25T09:59:08.395600+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:53.428739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.395135+0000","last_change":"2024-04-25T09:59:07.376556+0000","last_active":"2024-04-25T09:59:08.395135+0000","last_peered":"2024-04-25T09:59:08.395135+0000","last_clean":"2024-04-25T09:59:08.395135+0000","last_became_active":"2024-04-25T09:59:07.376289+0000","last_became_peered":"2024-04-25T09:59:07.376289+0000","last_unstale":"2024-04-25T09:59:08.395135+0000","last_undegraded":"2024-04-25T09:59:08.395135+0000","last_fullsized":"2024-04-25T09:59:08.395135+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:52.082835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.386401+0000","last_change":"2024-04-25T09:59:08.381709+0000","last_active":"2024-04-25T09:59:08.386401+0000","last_peered":"2024-04-25T09:59:08.386401+0000","last_clean":"2024-04-25T09:59:08.386401+0000","last_became_active":"2024-04-25T09:59:08.381426+0000","last_became_peered":"2024-04-25T09:59:08.381426+0000","last_unstale":"2024-04-25T09:59:08.386401+0000","last_undegraded":"2024-04-25T09:59:08.386401+0000","last_fullsized":"2024-04-25T09:59:08.386401+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:50.966310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.372625+0000","last_change":"2024-04-25T09:59:07.384848+0000","last_active":"2024-04-25T09:59:08.372625+0000","last_peered":"2024-04-25T09:59:08.372625+0000","last_clean":"2024-04-25T09:59:08.372625+0000","last_became_active":"2024-04-25T09:59:07.384509+0000","last_became_peered":"2024-04-25T09:59:07.384509+0000","last_unstale":"2024-04-25T09:59:08.372625+0000","last_undegraded":"2024-04-25T09:59:08.372625+0000","last_fullsized":"2024-04-25T09:59:08.372625+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:43.533573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383521+0000","last_change":"2024-04-25T09:59:08.379154+0000","last_active":"2024-04-25T09:59:08.383521+0000","last_peered":"2024-04-25T09:59:08.383521+0000","last_clean":"2024-04-25T09:59:08.383521+0000","last_became_active":"2024-04-25T09:59:08.378850+0000","last_became_peered":"2024-04-25T09:59:08.378850+0000","last_unstale":"2024-04-25T09:59:08.383521+0000","last_undegraded":"2024-04-25T09:59:08.383521+0000","last_fullsized":"2024-04-25T09:59:08.383521+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:08.974675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.367156+0000","last_change":"2024-04-25T09:59:07.381785+0000","last_active":"2024-04-25T09:59:08.367156+0000","last_peered":"2024-04-25T09:59:08.367156+0000","last_clean":"2024-04-25T09:59:08.367156+0000","last_became_active":"2024-04-25T09:59:07.381331+0000","last_became_peered":"2024-04-25T09:59:07.381331+0000","last_unstale":"2024-04-25T09:59:08.367156+0000","last_undegraded":"2024-04-25T09:59:08.367156+0000","last_fullsized":"2024-04-25T09:59:08.367156+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:36.178681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.393801+0000","last_change":"2024-04-25T09:59:08.387928+0000","last_active":"2024-04-25T09:59:08.393801+0000","last_peered":"2024-04-25T09:59:08.393801+0000","last_clean":"2024-04-25T09:59:08.393801+0000","last_became_active":"2024-04-25T09:59:08.386590+0000","last_became_peered":"2024-04-25T09:59:08.386590+0000","last_unstale":"2024-04-25T09:59:08.393801+0000","last_undegraded":"2024-04-25T09:59:08.393801+0000","last_fullsized":"2024-04-25T09:59:08.393801+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:34.820077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.372527+0000","last_change":"2024-04-25T09:59:07.377929+0000","last_active":"2024-04-25T09:59:08.372527+0000","last_peered":"2024-04-25T09:59:08.372527+0000","last_clean":"2024-04-25T09:59:08.372527+0000","last_became_active":"2024-04-25T09:59:07.377400+0000","last_became_peered":"2024-04-25T09:59:07.377400+0000","last_unstale":"2024-04-25T09:59:08.372527+0000","last_undegraded":"2024-04-25T09:59:08.372527+0000","last_fullsized":"2024-04-25T09:59:08.372527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:58.129751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.378298+0000","last_change":"2024-04-25T09:59:08.375048+0000","last_active":"2024-04-25T09:59:08.378298+0000","last_peered":"2024-04-25T09:59:08.378298+0000","last_clean":"2024-04-25T09:59:08.378298+0000","last_became_active":"2024-04-25T09:59:08.374686+0000","last_became_peered":"2024-04-25T09:59:08.374686+0000","last_unstale":"2024-04-25T09:59:08.378298+0000","last_undegraded":"2024-04-25T09:59:08.378298+0000","last_fullsized":"2024-04-25T09:59:08.378298+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:31.394397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392442+0000","last_change":"2024-04-25T09:59:07.376765+0000","last_active":"2024-04-25T09:59:08.392442+0000","last_peered":"2024-04-25T09:59:08.392442+0000","last_clean":"2024-04-25T09:59:08.392442+0000","last_became_active":"2024-04-25T09:59:07.376505+0000","last_became_peered":"2024-04-25T09:59:07.376505+0000","last_unstale":"2024-04-25T09:59:08.392442+0000","last_undegraded":"2024-04-25T09:59:08.392442+0000","last_fullsized":"2024-04-25T09:59:08.392442+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:52.746497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.390391+0000","last_change":"2024-04-25T09:59:08.386619+0000","last_active":"2024-04-25T09:59:08.390391+0000","last_peered":"2024-04-25T09:59:08.390391+0000","last_clean":"2024-04-25T09:59:08.390391+0000","last_became_active":"2024-04-25T09:59:08.386340+0000","last_became_peered":"2024-04-25T09:59:08.386340+0000","last_unstale":"2024-04-25T09:59:08.390391+0000","last_undegraded":"2024-04-25T09:59:08.390391+0000","last_fullsized":"2024-04-25T09:59:08.390391+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:09.818893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.376477+0000","last_change":"2024-04-25T09:59:07.378817+0000","last_active":"2024-04-25T09:59:08.376477+0000","last_peered":"2024-04-25T09:59:08.376477+0000","last_clean":"2024-04-25T09:59:08.376477+0000","last_became_active":"2024-04-25T09:59:07.378407+0000","last_became_peered":"2024-04-25T09:59:07.378407+0000","last_unstale":"2024-04-25T09:59:08.376477+0000","last_undegraded":"2024-04-25T09:59:08.376477+0000","last_fullsized":"2024-04-25T09:59:08.376477+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:57.137009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.389772+0000","last_change":"2024-04-25T09:59:08.381635+0000","last_active":"2024-04-25T09:59:08.389772+0000","last_peered":"2024-04-25T09:59:08.389772+0000","last_clean":"2024-04-25T09:59:08.389772+0000","last_became_active":"2024-04-25T09:59:08.381357+0000","last_became_peered":"2024-04-25T09:59:08.381357+0000","last_unstale":"2024-04-25T09:59:08.389772+0000","last_undegraded":"2024-04-25T09:59:08.389772+0000","last_fullsized":"2024-04-25T09:59:08.389772+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.156095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.375632+0000","last_change":"2024-04-25T09:59:07.374478+0000","last_active":"2024-04-25T09:59:08.375632+0000","last_peered":"2024-04-25T09:59:08.375632+0000","last_clean":"2024-04-25T09:59:08.375632+0000","last_became_active":"2024-04-25T09:59:07.374171+0000","last_became_peered":"2024-04-25T09:59:07.374171+0000","last_unstale":"2024-04-25T09:59:08.375632+0000","last_undegraded":"2024-04-25T09:59:08.375632+0000","last_fullsized":"2024-04-25T09:59:08.375632+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:00.417554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.383508+0000","last_change":"2024-04-25T09:59:08.379323+0000","last_active":"2024-04-25T09:59:08.383508+0000","last_peered":"2024-04-25T09:59:08.383508+0000","last_clean":"2024-04-25T09:59:08.383508+0000","last_became_active":"2024-04-25T09:59:08.378830+0000","last_became_peered":"2024-04-25T09:59:08.378830+0000","last_unstale":"2024-04-25T09:59:08.383508+0000","last_undegraded":"2024-04-25T09:59:08.383508+0000","last_fullsized":"2024-04-25T09:59:08.383508+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:30.378274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.371297+0000","last_change":"2024-04-25T09:59:07.379821+0000","last_active":"2024-04-25T09:59:08.371297+0000","last_peered":"2024-04-25T09:59:08.371297+0000","last_clean":"2024-04-25T09:59:08.371297+0000","last_became_active":"2024-04-25T09:59:07.379552+0000","last_became_peered":"2024-04-25T09:59:07.379552+0000","last_unstale":"2024-04-25T09:59:08.371297+0000","last_undegraded":"2024-04-25T09:59:08.371297+0000","last_fullsized":"2024-04-25T09:59:08.371297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:49.467983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.399502+0000","last_change":"2024-04-25T09:59:08.397446+0000","last_active":"2024-04-25T09:59:08.399502+0000","last_peered":"2024-04-25T09:59:08.399502+0000","last_clean":"2024-04-25T09:59:08.399502+0000","last_became_active":"2024-04-25T09:59:08.397179+0000","last_became_peered":"2024-04-25T09:59:08.397179+0000","last_unstale":"2024-04-25T09:59:08.399502+0000","last_undegraded":"2024-04-25T09:59:08.399502+0000","last_fullsized":"2024-04-25T09:59:08.399502+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:29.940853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.366265+0000","last_change":"2024-04-25T09:59:07.374941+0000","last_active":"2024-04-25T09:59:08.366265+0000","last_peered":"2024-04-25T09:59:08.366265+0000","last_clean":"2024-04-25T09:59:08.366265+0000","last_became_active":"2024-04-25T09:59:07.374656+0000","last_became_peered":"2024-04-25T09:59:07.374656+0000","last_unstale":"2024-04-25T09:59:08.366265+0000","last_undegraded":"2024-04-25T09:59:08.366265+0000","last_fullsized":"2024-04-25T09:59:08.366265+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:26.164653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.392954+0000","last_change":"2024-04-25T09:59:08.390100+0000","last_active":"2024-04-25T09:59:08.392954+0000","last_peered":"2024-04-25T09:59:08.392954+0000","last_clean":"2024-04-25T09:59:08.392954+0000","last_became_active":"2024-04-25T09:59:08.389808+0000","last_became_peered":"2024-04-25T09:59:08.389808+0000","last_unstale":"2024-04-25T09:59:08.392954+0000","last_undegraded":"2024-04-25T09:59:08.392954+0000","last_fullsized":"2024-04-25T09:59:08.392954+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:37.411559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.367315+0000","last_change":"2024-04-25T09:59:07.382161+0000","last_active":"2024-04-25T09:59:08.367315+0000","last_peered":"2024-04-25T09:59:08.367315+0000","last_clean":"2024-04-25T09:59:08.367315+0000","last_became_active":"2024-04-25T09:59:07.381863+0000","last_became_peered":"2024-04-25T09:59:07.381863+0000","last_unstale":"2024-04-25T09:59:08.367315+0000","last_undegraded":"2024-04-25T09:59:08.367315+0000","last_fullsized":"2024-04-25T09:59:08.367315+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:24.588204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.381271+0000","last_change":"2024-04-25T09:59:08.379558+0000","last_active":"2024-04-25T09:59:08.381271+0000","last_peered":"2024-04-25T09:59:08.381271+0000","last_clean":"2024-04-25T09:59:08.381271+0000","last_became_active":"2024-04-25T09:59:08.379279+0000","last_became_peered":"2024-04-25T09:59:08.379279+0000","last_unstale":"2024-04-25T09:59:08.381271+0000","last_undegraded":"2024-04-25T09:59:08.381271+0000","last_fullsized":"2024-04-25T09:59:08.381271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:29.918692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.369442+0000","last_change":"2024-04-25T09:59:07.379230+0000","last_active":"2024-04-25T09:59:08.369442+0000","last_peered":"2024-04-25T09:59:08.369442+0000","last_clean":"2024-04-25T09:59:08.369442+0000","last_became_active":"2024-04-25T09:59:07.378954+0000","last_became_peered":"2024-04-25T09:59:07.378954+0000","last_unstale":"2024-04-25T09:59:08.369442+0000","last_undegraded":"2024-04-25T09:59:08.369442+0000","last_fullsized":"2024-04-25T09:59:08.369442+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:52.726175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.397395+0000","last_change":"2024-04-25T09:59:08.395443+0000","last_active":"2024-04-25T09:59:08.397395+0000","last_peered":"2024-04-25T09:59:08.397395+0000","last_clean":"2024-04-25T09:59:08.397395+0000","last_became_active":"2024-04-25T09:59:08.393442+0000","last_became_peered":"2024-04-25T09:59:08.393442+0000","last_unstale":"2024-04-25T09:59:08.397395+0000","last_undegraded":"2024-04-25T09:59:08.397395+0000","last_fullsized":"2024-04-25T09:59:08.397395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:13.019243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.366614+0000","last_change":"2024-04-25T09:59:07.378762+0000","last_active":"2024-04-25T09:59:08.366614+0000","last_peered":"2024-04-25T09:59:08.366614+0000","last_clean":"2024-04-25T09:59:08.366614+0000","last_became_active":"2024-04-25T09:59:07.378489+0000","last_became_peered":"2024-04-25T09:59:07.378489+0000","last_unstale":"2024-04-25T09:59:08.366614+0000","last_undegraded":"2024-04-25T09:59:08.366614+0000","last_fullsized":"2024-04-25T09:59:08.366614+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:54.423535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.380840+0000","last_change":"2024-04-25T09:59:08.376800+0000","last_active":"2024-04-25T09:59:08.380840+0000","last_peered":"2024-04-25T09:59:08.380840+0000","last_clean":"2024-04-25T09:59:08.380840+0000","last_became_active":"2024-04-25T09:59:08.376531+0000","last_became_peered":"2024-04-25T09:59:08.376531+0000","last_unstale":"2024-04-25T09:59:08.380840+0000","last_undegraded":"2024-04-25T09:59:08.380840+0000","last_fullsized":"2024-04-25T09:59:08.380840+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:07.356464+0000","last_clean_scrub_stamp":"2024-04-25T09:59:07.356464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:40.120681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.375392+0000","last_change":"2024-04-25T09:59:07.386210+0000","last_active":"2024-04-25T09:59:08.375392+0000","last_peered":"2024-04-25T09:59:08.375392+0000","last_clean":"2024-04-25T09:59:08.375392+0000","last_became_active":"2024-04-25T09:59:07.385823+0000","last_became_peered":"2024-04-25T09:59:07.385823+0000","last_unstale":"2024-04-25T09:59:08.375392+0000","last_undegraded":"2024-04-25T09:59:08.375392+0000","last_fullsized":"2024-04-25T09:59:08.375392+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:06.354227+0000","last_clean_scrub_stamp":"2024-04-25T09:59:06.354227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:52.726175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":321,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T09:59:08.366730+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:59:08.366730+0000","last_peered":"2024-04-25T09:59:08.366730+0000","last_clean":"2024-04-25T09:59:08.366730+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:59:08.366730+0000","last_undegraded":"2024-04-25T09:59:08.366730+0000","last_fullsized":"2024-04-25T09:59:08.366730+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41220,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330620,"statfs":{"total":96636764160,"available":96594554880,"internally_reserved":0,"allocated":1761280,"data_stored":1128860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40652,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331188,"statfs":{"total":96636764160,"available":96595136512,"internally_reserved":0,"allocated":1179648,"data_stored":540652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40640,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331200,"statfs":{"total":96636764160,"available":96595148800,"internally_reserved":0,"allocated":1159168,"data_stored":538436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50352,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321488,"statfs":{"total":96636764160,"available":96585203712,"internally_reserved":0,"allocated":1146880,"data_stored":531269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50360,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321480,"statfs":{"total":96636764160,"available":96585195520,"internally_reserved":0,"allocated":1159168,"data_stored":538436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46204,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325636,"statfs":{"total":96636764160,"available":96589451264,"internally_reserved":0,"allocated":1159168,"data_stored":538436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62836,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309004,"statfs":{"total":96636764160,"available":96572420096,"internally_reserved":0,"allocated":1146880,"data_stored":531269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46772,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325068,"statfs":{"total":96636764160,"available":96588869632,"internally_reserved":0,"allocated":1740800,"data_stored":1121637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:59:11.727 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:59:11.727 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:59:11.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.868+0000 7f521c71b640 1 Processor -- start 2024-04-25T09:59:11.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.868+0000 7f521c71b640 1 -- start start 2024-04-25T09:59:11.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f521c71b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5214144d50 0x7f521406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f521c71b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52141452a0 0x7f521406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f521c71b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52140cccc0 con 0x7f52141446d0 2024-04-25T09:59:11.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f521c71b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52140b2b20 con 0x7f5214144d50 2024-04-25T09:59:11.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f521c71b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521406d400 con 0x7f52141452a0 2024-04-25T09:59:11.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f5219cb6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5214144d50 0x7f521406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f521a4b7640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f52141446d0 0x7f52140b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50654/0 (socket says 172.21.15.19:50654) 2024-04-25T09:59:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f521a4b7640 1 -- 172.21.15.19:0/2816442957 learned_addr learned my addr 172.21.15.19:0/2816442957 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f521acb8640 1 --2- 172.21.15.19:0/2816442957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52141452a0 0x7f521406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f5219cb6640 1 -- 172.21.15.19:0/2816442957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52141452a0 msgr2=0x7f521406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f5219cb6640 1 --2- 172.21.15.19:0/2816442957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52141452a0 0x7f521406ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f5219cb6640 1 -- 172.21.15.19:0/2816442957 >> v1:172.21.15.19:6789/0 conn(0x7f52141446d0 legacy=0x7f52140b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f5219cb6640 1 -- 172.21.15.19:0/2816442957 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52140cc140 con 0x7f5214144d50 2024-04-25T09:59:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.869+0000 7f52194b5640 1 -- 172.21.15.19:0/2816442957 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977066954 0 0) 0x7f52140cccc0 con 0x7f52141446d0 2024-04-25T09:59:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.870+0000 7f5219cb6640 1 --2- 172.21.15.19:0/2816442957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5214144d50 0x7f521406d890 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f52040097a0 tx=0x7f520402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3664b638c25c9292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.870+0000 7f52194b5640 1 -- 172.21.15.19:0/2816442957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204037070 con 0x7f5214144d50 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.870+0000 7f52194b5640 1 -- 172.21.15.19:0/2816442957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5204032440 con 0x7f5214144d50 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.870+0000 7f52194b5640 1 -- 172.21.15.19:0/2816442957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204032810 con 0x7f5214144d50 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.870+0000 7f521c71b640 1 -- 172.21.15.19:0/2816442957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5214144d50 msgr2=0x7f521406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.870+0000 7f521c71b640 1 --2- 172.21.15.19:0/2816442957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5214144d50 0x7f521406d890 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f52040097a0 tx=0x7f520402a180 comp rx=0 tx=0).stop 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.871+0000 7f521c71b640 1 -- 172.21.15.19:0/2816442957 shutdown_connections 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.871+0000 7f521c71b640 1 --2- 172.21.15.19:0/2816442957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52141452a0 0x7f521406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.871+0000 7f521c71b640 1 --2- 172.21.15.19:0/2816442957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5214144d50 0x7f521406d890 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.871+0000 7f521c71b640 1 -- 172.21.15.19:0/2816442957 >> 172.21.15.19:0/2816442957 conn(0x7f521406abe0 msgr2=0x7f521406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.871+0000 7f521c71b640 1 -- 172.21.15.19:0/2816442957 shutdown_connections 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.871+0000 7f521c71b640 1 -- 172.21.15.19:0/2816442957 wait complete. 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.872+0000 7f521c71b640 1 Processor -- start 2024-04-25T09:59:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.872+0000 7f521c71b640 1 -- start start 2024-04-25T09:59:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521c71b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52141446d0 0x7f521413ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521c71b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5214144d50 0x7f52140cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521c71b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52141452a0 0x7f52141f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521c71b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140cc9e0 con 0x7f52141452a0 2024-04-25T09:59:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521c71b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140cc700 con 0x7f52141446d0 2024-04-25T09:59:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521c71b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5214144ab0 con 0x7f5214144d50 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521acb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52141452a0 0x7f52141f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521acb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52141452a0 0x7f52141f64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41048/0 (socket says 172.21.15.19:41048) 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521acb8640 1 -- 172.21.15.19:0/1409304558 learned_addr learned my addr 172.21.15.19:0/1409304558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f5219cb6640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5214144d50 0x7f52140cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521a4b7640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52141446d0 0x7f521413ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521acb8640 1 -- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5214144d50 msgr2=0x7f52140cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521acb8640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5214144d50 0x7f52140cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521acb8640 1 -- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52141446d0 msgr2=0x7f521413ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521acb8640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52141446d0 0x7f521413ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521acb8640 1 -- 172.21.15.19:0/1409304558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521413d4c0 con 0x7f52141452a0 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521a4b7640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52141446d0 0x7f521413ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521acb8640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52141452a0 0x7f52141f64e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f520c00cd70 tx=0x7f520c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f520affd640 1 -- 172.21.15.19:0/1409304558 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c003b90 con 0x7f52141452a0 2024-04-25T09:59:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f520affd640 1 -- 172.21.15.19:0/1409304558 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c003d30 con 0x7f52141452a0 2024-04-25T09:59:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521c71b640 1 -- 172.21.15.19:0/1409304558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521413d7a0 con 0x7f52141452a0 2024-04-25T09:59:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f520affd640 1 -- 172.21.15.19:0/1409304558 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c004240 con 0x7f52141452a0 2024-04-25T09:59:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.873+0000 7f521c71b640 1 -- 172.21.15.19:0/1409304558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52140aca50 con 0x7f52141452a0 2024-04-25T09:59:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.874+0000 7f521c71b640 1 -- 172.21.15.19:0/1409304558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5214073a00 con 0x7f52141452a0 2024-04-25T09:59:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.875+0000 7f520affd640 1 -- 172.21.15.19:0/1409304558 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f520c0043e0 con 0x7f52141452a0 2024-04-25T09:59:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.875+0000 7f520affd640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51f806b4d0 0x7f51f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.875+0000 7f520affd640 1 -- 172.21.15.19:0/1409304558 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f520c08af10 con 0x7f52141452a0 2024-04-25T09:59:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.875+0000 7f521a4b7640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51f806b4d0 0x7f51f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.876+0000 7f521a4b7640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51f806b4d0 0x7f51f806d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5210000f60 tx=0x7f5210009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:11.879+0000 7f520affd640 1 -- 172.21.15.19:0/1409304558 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f520c01d340 con 0x7f52141452a0 2024-04-25T09:59:12.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.126+0000 7f521c71b640 1 -- 172.21.15.19:0/1409304558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f521413dcc0 con 0x7f52141452a0 2024-04-25T09:59:12.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.128+0000 7f520affd640 1 -- 172.21.15.19:0/1409304558 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f520c059c00 con 0x7f52141452a0 2024-04-25T09:59:12.132 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T09:59:12.132 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T09:59:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T09:59:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T09:59:12.133 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:59:12.133 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T09:59:12.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.130+0000 7f521c71b640 1 -- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51f806b4d0 msgr2=0x7f51f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.130+0000 7f521c71b640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51f806b4d0 0x7f51f806d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5210000f60 tx=0x7f5210009040 comp rx=0 tx=0).stop 2024-04-25T09:59:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.130+0000 7f521c71b640 1 -- 172.21.15.19:0/1409304558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52141452a0 msgr2=0x7f52141f64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.130+0000 7f521c71b640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52141452a0 0x7f52141f64e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f520c00cd70 tx=0x7f520c00a660 comp rx=0 tx=0).stop 2024-04-25T09:59:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.131+0000 7f521c71b640 1 -- 172.21.15.19:0/1409304558 shutdown_connections 2024-04-25T09:59:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.131+0000 7f521c71b640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51f806b4d0 0x7f51f806d990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.131+0000 7f521c71b640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52141452a0 0x7f52141f64e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.131+0000 7f521c71b640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5214144d50 0x7f52140cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.131+0000 7f521c71b640 1 --2- 172.21.15.19:0/1409304558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52141446d0 0x7f521413ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.131+0000 7f521c71b640 1 -- 172.21.15.19:0/1409304558 >> 172.21.15.19:0/1409304558 conn(0x7f521406abe0 msgr2=0x7f52140b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.131+0000 7f521c71b640 1 -- 172.21.15.19:0/1409304558 shutdown_connections 2024-04-25T09:59:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.131+0000 7f521c71b640 1 -- 172.21.15.19:0/1409304558 wait complete. 2024-04-25T09:59:12.145 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:12.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.243+0000 7ff702ac6640 1 Processor -- start 2024-04-25T09:59:12.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.243+0000 7ff702ac6640 1 -- start start 2024-04-25T09:59:12.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.244+0000 7ff702ac6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6fc0b6cb0 0x7ff6fc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.244+0000 7ff702ac6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6fc0b5a60 0x7ff6fc0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.244+0000 7ff702ac6640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6fc0ceee0 con 0x7ff6fc15db70 2024-04-25T09:59:12.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.244+0000 7ff702ac6640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6fc0d1d10 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.244+0000 7ff702ac6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0d1ff0 con 0x7ff6fc0b6cb0 2024-04-25T09:59:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.244+0000 7ff701063640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6fc0b5a60 0x7ff6fc0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.244+0000 7ff700862640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff6fc15db70 0x7ff6fc162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50656/0 (socket says 172.21.15.19:50656) 2024-04-25T09:59:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.244+0000 7ff700862640 1 -- 172.21.15.19:0/213757418 learned_addr learned my addr 172.21.15.19:0/213757418 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.245+0000 7ff6fbfff640 1 --2- 172.21.15.19:0/213757418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6fc0b6cb0 0x7ff6fc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.245+0000 7ff701063640 1 -- 172.21.15.19:0/213757418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6fc0b6cb0 msgr2=0x7ff6fc0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.245+0000 7ff701063640 1 --2- 172.21.15.19:0/213757418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6fc0b6cb0 0x7ff6fc0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.245+0000 7ff701063640 1 -- 172.21.15.19:0/213757418 >> v1:172.21.15.19:6789/0 conn(0x7ff6fc15db70 legacy=0x7ff6fc162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.245+0000 7ff701063640 1 -- 172.21.15.19:0/213757418 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc0d22d0 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.245+0000 7ff6fb7fe640 1 -- 172.21.15.19:0/213757418 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864826308 0 0) 0x7ff6fc0ceee0 con 0x7ff6fc15db70 2024-04-25T09:59:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.245+0000 7ff701063640 1 --2- 172.21.15.19:0/213757418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6fc0b5a60 0x7ff6fc0d27f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0097a0 tx=0x7ff6ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9b255d94fa0fa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.245+0000 7ff6fb7fe640 1 -- 172.21.15.19:0/213757418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec037070 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.245+0000 7ff6fb7fe640 1 -- 172.21.15.19:0/213757418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ec032440 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.245+0000 7ff6fb7fe640 1 -- 172.21.15.19:0/213757418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec032810 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.246+0000 7ff702ac6640 1 -- 172.21.15.19:0/213757418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6fc0b5a60 msgr2=0x7ff6fc0d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.246+0000 7ff702ac6640 1 --2- 172.21.15.19:0/213757418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6fc0b5a60 0x7ff6fc0d27f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0097a0 tx=0x7ff6ec02a180 comp rx=0 tx=0).stop 2024-04-25T09:59:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.246+0000 7ff702ac6640 1 -- 172.21.15.19:0/213757418 shutdown_connections 2024-04-25T09:59:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.246+0000 7ff702ac6640 1 --2- 172.21.15.19:0/213757418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6fc0b5a60 0x7ff6fc0d27f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.246+0000 7ff702ac6640 1 --2- 172.21.15.19:0/213757418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6fc0b6cb0 0x7ff6fc0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.246+0000 7ff702ac6640 1 -- 172.21.15.19:0/213757418 >> 172.21.15.19:0/213757418 conn(0x7ff6fc0d0520 msgr2=0x7ff6fc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.246+0000 7ff702ac6640 1 -- 172.21.15.19:0/213757418 shutdown_connections 2024-04-25T09:59:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.246+0000 7ff702ac6640 1 -- 172.21.15.19:0/213757418 wait complete. 2024-04-25T09:59:12.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.247+0000 7ff702ac6640 1 Processor -- start 2024-04-25T09:59:12.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.247+0000 7ff702ac6640 1 -- start start 2024-04-25T09:59:12.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.248+0000 7ff702ac6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6fc0b5a60 0x7ff6fc149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.248+0000 7ff702ac6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6fc0b6cb0 0x7ff6fc0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.248+0000 7ff702ac6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6fc15db70 0x7ff6fc0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.248+0000 7ff702ac6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0cf1c0 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.248+0000 7ff702ac6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0ce530 con 0x7ff6fc0b6cb0 2024-04-25T09:59:12.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.248+0000 7ff702ac6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0cf340 con 0x7ff6fc15db70 2024-04-25T09:59:12.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.248+0000 7ff700862640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6fc0b5a60 0x7ff6fc149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.248+0000 7ff700862640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6fc0b5a60 0x7ff6fc149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41066/0 (socket says 172.21.15.19:41066) 2024-04-25T09:59:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.248+0000 7ff700862640 1 -- 172.21.15.19:0/2338648231 learned_addr learned my addr 172.21.15.19:0/2338648231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.248+0000 7ff6fbfff640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6fc0b6cb0 0x7ff6fc0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.248+0000 7ff701063640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6fc15db70 0x7ff6fc0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.249+0000 7ff700862640 1 -- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6fc15db70 msgr2=0x7ff6fc0cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.249+0000 7ff700862640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6fc15db70 0x7ff6fc0cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.249+0000 7ff700862640 1 -- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6fc0b6cb0 msgr2=0x7ff6fc0c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.249+0000 7ff700862640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6fc0b6cb0 0x7ff6fc0c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.249+0000 7ff700862640 1 -- 172.21.15.19:0/2338648231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc0cbc70 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.249+0000 7ff700862640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6fc0b5a60 0x7ff6fc149840 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400c7f0 tx=0x7ff6f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.249+0000 7ff6f97fa640 1 -- 172.21.15.19:0/2338648231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4018070 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.249+0000 7ff6f97fa640 1 -- 172.21.15.19:0/2338648231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f400a640 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.249+0000 7ff6f97fa640 1 -- 172.21.15.19:0/2338648231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400aa30 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.249+0000 7ff702ac6640 1 -- 172.21.15.19:0/2338648231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6fc0cbf50 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.249+0000 7ff702ac6640 1 -- 172.21.15.19:0/2338648231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6fc0cc510 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.250+0000 7ff702ac6640 1 -- 172.21.15.19:0/2338648231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6fc068410 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.250+0000 7ff6f97fa640 1 -- 172.21.15.19:0/2338648231 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6f401b070 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.251+0000 7ff6f97fa640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6e806b4d0 0x7ff6e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.251+0000 7ff6f97fa640 1 -- 172.21.15.19:0/2338648231 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff6f4089950 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.251+0000 7ff6fbfff640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6e806b4d0 0x7ff6e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.252+0000 7ff6fbfff640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6e806b4d0 0x7ff6e806d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0cd190 tx=0x7ff6f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:12.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.255+0000 7ff6f97fa640 1 -- 172.21.15.19:0/2338648231 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6fc068410 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.473+0000 7ff702ac6640 1 -- 172.21.15.19:0/2338648231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6fc073990 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.473+0000 7ff6f97fa640 1 -- 172.21.15.19:0/2338648231 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff6f4057ff0 con 0x7ff6fc0b5a60 2024-04-25T09:59:12.477 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:12.477 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3431952054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3431952054},{"type":"v1","addr":"172.21.15.112:6836","nonce":3431952054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2758109099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2758109099},{"type":"v1","addr":"172.21.15.112:6837","nonce":2758109099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7590,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3063446116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3063446116},{"type":"v1","addr":"172.21.15.19:6834","nonce":3063446116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:07.729627+0000","modified":"2024-04-25T09:59:09.720756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3589967477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3589967477},{"type":"v1","addr":"172.21.15.19:6835","nonce":3589967477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:12.478 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-04-25T09:59:12.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 -- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6e806b4d0 msgr2=0x7ff6e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6e806b4d0 0x7ff6e806d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0cd190 tx=0x7ff6f0008040 comp rx=0 tx=0).stop 2024-04-25T09:59:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 -- 172.21.15.19:0/2338648231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6fc0b5a60 msgr2=0x7ff6fc149840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6fc0b5a60 0x7ff6fc149840 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400c7f0 tx=0x7ff6f400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 -- 172.21.15.19:0/2338648231 shutdown_connections 2024-04-25T09:59:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6e806b4d0 0x7ff6e806d990 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6fc15db70 0x7ff6fc0cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6fc0b6cb0 0x7ff6fc0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 --2- 172.21.15.19:0/2338648231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6fc0b5a60 0x7ff6fc149840 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 -- 172.21.15.19:0/2338648231 >> 172.21.15.19:0/2338648231 conn(0x7ff6fc0d0520 msgr2=0x7ff6fc0b90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 -- 172.21.15.19:0/2338648231 shutdown_connections 2024-04-25T09:59:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.475+0000 7ff702ac6640 1 -- 172.21.15.19:0/2338648231 wait complete. 2024-04-25T09:59:12.896 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:59:07.729627+0000', 'modified': '2024-04-25T09:59:09.720756+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7593}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7593': {'gid': 7593, 'name': 'c', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/3589967477', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 3589967477}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3589967477}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:59:12.896 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:59:12.896 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:59:13.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.997+0000 7f7393046640 1 Processor -- start 2024-04-25T09:59:13.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.997+0000 7f7393046640 1 -- start start 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.998+0000 7f7393046640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f738c1446d0 0x7f738c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.998+0000 7f7393046640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f738c06e8a0 0x7f738c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.998+0000 7f7393046640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738c0cccc0 con 0x7f738c144d50 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.998+0000 7f7393046640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c0b2b20 con 0x7f738c1446d0 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.998+0000 7f7393046640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c06d400 con 0x7f738c06e8a0 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.998+0000 7f7390de2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f738c1446d0 0x7f738c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.998+0000 7f738bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f738c144d50 0x7f738c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:59636/0 (socket says 172.21.15.19:59636) 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.998+0000 7f738bfff640 1 -- 172.21.15.19:0/2978015643 learned_addr learned my addr 172.21.15.19:0/2978015643 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.998+0000 7f73915e3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f738c06e8a0 0x7f738c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.999+0000 7f7390de2640 1 -- 172.21.15.19:0/2978015643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f738c06e8a0 msgr2=0x7f738c0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.999+0000 7f7390de2640 1 --2- 172.21.15.19:0/2978015643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f738c06e8a0 0x7f738c0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.999+0000 7f7390de2640 1 -- 172.21.15.19:0/2978015643 >> v1:172.21.15.112:6789/0 conn(0x7f738c144d50 legacy=0x7f738c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.999+0000 7f7390de2640 1 -- 172.21.15.19:0/2978015643 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c0cc140 con 0x7f738c1446d0 2024-04-25T09:59:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:12.999+0000 7f73915e3640 1 --2- 172.21.15.19:0/2978015643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f738c06e8a0 0x7f738c0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f7390de2640 1 --2- 172.21.15.19:0/2978015643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738c1446d0 0x7f738c0b3f20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7384009d90 tx=0x7f738402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d06af3c0f9a99f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f738b7fe640 1 -- 172.21.15.19:0/2978015643 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384007740 con 0x7f738c1446d0 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f738b7fe640 1 -- 172.21.15.19:0/2978015643 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73840078e0 con 0x7f738c1446d0 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f738b7fe640 1 -- 172.21.15.19:0/2978015643 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384007cb0 con 0x7f738c1446d0 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f7393046640 1 -- 172.21.15.19:0/2978015643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738c1446d0 msgr2=0x7f738c0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f7393046640 1 --2- 172.21.15.19:0/2978015643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738c1446d0 0x7f738c0b3f20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7384009d90 tx=0x7f738402a660 comp rx=0 tx=0).stop 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f7393046640 1 -- 172.21.15.19:0/2978015643 shutdown_connections 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f7393046640 1 --2- 172.21.15.19:0/2978015643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f738c06e8a0 0x7f738c0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f7393046640 1 --2- 172.21.15.19:0/2978015643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738c1446d0 0x7f738c0b3f20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f7393046640 1 -- 172.21.15.19:0/2978015643 >> 172.21.15.19:0/2978015643 conn(0x7f738c06abe0 msgr2=0x7f738c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f7393046640 1 -- 172.21.15.19:0/2978015643 shutdown_connections 2024-04-25T09:59:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.000+0000 7f7393046640 1 -- 172.21.15.19:0/2978015643 wait complete. 2024-04-25T09:59:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.002+0000 7f7393046640 1 Processor -- start 2024-04-25T09:59:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.002+0000 7f7393046640 1 -- start start 2024-04-25T09:59:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.002+0000 7f7393046640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f738c06e8a0 0x7f738c15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.002+0000 7f7393046640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738c1446d0 0x7f738c138da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.002+0000 7f7393046640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c144d50 0x7f738c1392e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.002+0000 7f7393046640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0cc700 con 0x7f738c144d50 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.002+0000 7f7393046640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0b49a0 con 0x7f738c1446d0 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.002+0000 7f7393046640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0cc9e0 con 0x7f738c06e8a0 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.002+0000 7f73915e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c144d50 0x7f738c1392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f73915e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c144d50 0x7f738c1392e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41070/0 (socket says 172.21.15.19:41070) 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f73915e3640 1 -- 172.21.15.19:0/4068904600 learned_addr learned my addr 172.21.15.19:0/4068904600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f7390de2640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f738c06e8a0 0x7f738c15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f738bfff640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738c1446d0 0x7f738c138da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f73915e3640 1 -- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f738c06e8a0 msgr2=0x7f738c15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f73915e3640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f738c06e8a0 0x7f738c15b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f73915e3640 1 -- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738c1446d0 msgr2=0x7f738c138da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f73915e3640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738c1446d0 0x7f738c138da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f73915e3640 1 -- 172.21.15.19:0/4068904600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c138950 con 0x7f738c144d50 2024-04-25T09:59:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f738bfff640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738c1446d0 0x7f738c138da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f73915e3640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c144d50 0x7f738c1392e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f737c00e880 tx=0x7f737c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f73897fa640 1 -- 172.21.15.19:0/4068904600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c017070 con 0x7f738c144d50 2024-04-25T09:59:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f73897fa640 1 -- 172.21.15.19:0/4068904600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c00c940 con 0x7f738c144d50 2024-04-25T09:59:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f7393046640 1 -- 172.21.15.19:0/4068904600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c13ef70 con 0x7f738c144d50 2024-04-25T09:59:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f73897fa640 1 -- 172.21.15.19:0/4068904600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c00cd30 con 0x7f738c144d50 2024-04-25T09:59:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.003+0000 7f7393046640 1 -- 172.21.15.19:0/4068904600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c13f530 con 0x7f738c144d50 2024-04-25T09:59:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.005+0000 7f73897fa640 1 -- 172.21.15.19:0/4068904600 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f737c01a070 con 0x7f738c144d50 2024-04-25T09:59:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.005+0000 7f73897fa640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737806b4d0 0x7f737806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.005+0000 7f73897fa640 1 -- 172.21.15.19:0/4068904600 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f737c08a4f0 con 0x7f738c144d50 2024-04-25T09:59:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.006+0000 7f7390de2640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737806b4d0 0x7f737806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.006+0000 7f7395e21740 1 Processor -- start 2024-04-25T09:59:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.006+0000 7f7395e21740 1 -- start start 2024-04-25T09:59:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.006+0000 7f7390de2640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737806b4d0 0x7f737806d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7384034000 tx=0x7f738402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7395e21740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52ac2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7395e21740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52adfad0 0x559b52ae0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7395e21740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52a97f10 0x559b52ade190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7395e21740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b529faa10 con 0x559b52ac2230 2024-04-25T09:59:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7395e21740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b529a3b30 con 0x559b52a97f10 2024-04-25T09:59:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7395e21740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b529f6c50 con 0x559b52adfad0 2024-04-25T09:59:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7390de2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52ac2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7390de2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52ac2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41078/0 (socket says 172.21.15.19:41078) 2024-04-25T09:59:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7390de2640 1 -- 172.21.15.19:0/733085593 learned_addr learned my addr 172.21.15.19:0/733085593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f73915e3640 1 --2- 172.21.15.19:0/733085593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52a97f10 0x559b52ade190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f738bfff640 1 --2- 172.21.15.19:0/733085593 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52adfad0 0x559b52ae0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7390de2640 1 -- 172.21.15.19:0/733085593 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52adfad0 msgr2=0x559b52ae0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7390de2640 1 --2- 172.21.15.19:0/733085593 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52adfad0 0x559b52ae0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7390de2640 1 -- 172.21.15.19:0/733085593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52a97f10 msgr2=0x559b52ade190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7390de2640 1 --2- 172.21.15.19:0/733085593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52a97f10 0x559b52ade190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.007+0000 7f7390de2640 1 -- 172.21.15.19:0/733085593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b529b8e10 con 0x559b52ac2230 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f7390de2640 1 --2- 172.21.15.19:0/733085593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52ac2610 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f738402ac50 tx=0x7f738402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693009b77cdd69af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f73727fc640 1 -- 172.21.15.19:0/733085593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738403d6f0 con 0x559b52ac2230 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f73727fc640 1 -- 172.21.15.19:0/733085593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738403d890 con 0x559b52ac2230 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f73727fc640 1 -- 172.21.15.19:0/733085593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738403dc40 con 0x559b52ac2230 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f7395e21740 1 -- 172.21.15.19:0/733085593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 msgr2=0x559b52ac2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f7395e21740 1 --2- 172.21.15.19:0/733085593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52ac2610 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f738402ac50 tx=0x7f738402e000 comp rx=0 tx=0).stop 2024-04-25T09:59:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f7395e21740 1 -- 172.21.15.19:0/733085593 shutdown_connections 2024-04-25T09:59:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f7395e21740 1 --2- 172.21.15.19:0/733085593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52a97f10 0x559b52ade190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f7395e21740 1 --2- 172.21.15.19:0/733085593 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52adfad0 0x559b52ae0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f7395e21740 1 --2- 172.21.15.19:0/733085593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52ac2610 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f7395e21740 1 -- 172.21.15.19:0/733085593 >> 172.21.15.19:0/733085593 conn(0x559b52a94840 msgr2=0x559b52add040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f7395e21740 1 -- 172.21.15.19:0/733085593 shutdown_connections 2024-04-25T09:59:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.008+0000 7f7395e21740 1 -- 172.21.15.19:0/733085593 wait complete. 2024-04-25T09:59:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.009+0000 7f7395e21740 1 -- start start 2024-04-25T09:59:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.009+0000 7f7395e21740 1 Processor -- start 2024-04-25T09:59:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f7395e21740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac2230 0x559b52ab0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f7395e21740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52a91940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f7395e21740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52ac5040 0x559b52a91e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f7395e21740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b529faa10 con 0x559b52adfad0 2024-04-25T09:59:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f7395e21740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b529b70c0 con 0x559b52ac5040 2024-04-25T09:59:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f7395e21740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b52a98cb0 con 0x559b52ac2230 2024-04-25T09:59:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f7395e21740 10 client.0 _ll_register_callbacks cb 0x559b529aaec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f738bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52a91940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f738bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52a91940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41094/0 (socket says 172.21.15.19:41094) 2024-04-25T09:59:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f738bfff640 1 -- 172.21.15.19:0/347183824 learned_addr learned my addr 172.21.15.19:0/347183824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f73915e3640 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52ac5040 0x559b52a91e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f7390de2640 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac2230 0x559b52ab0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f738bfff640 1 -- 172.21.15.19:0/347183824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac2230 msgr2=0x559b52ab0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f738bfff640 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac2230 0x559b52ab0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f738bfff640 1 -- 172.21.15.19:0/347183824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52ac5040 msgr2=0x559b52a91e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f738bfff640 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52ac5040 0x559b52a91e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.010+0000 7f738bfff640 1 -- 172.21.15.19:0/347183824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b5270f2d0 con 0x559b52adfad0 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f738bfff640 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52a91940 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f7380000f60 tx=0x7f73800116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6713540faf9becf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f736b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f736b7fe640 1 -- 172.21.15.19:0/347183824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7380005aa0 con 0x559b52adfad0 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f736b7fe640 1 -- 172.21.15.19:0/347183824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7380005c40 con 0x559b52adfad0 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f7395e21740 1 -- 172.21.15.19:0/347183824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f738403d890 con 0x559b52adfad0 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f736b7fe640 1 -- 172.21.15.19:0/347183824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7380022050 con 0x559b52adfad0 2024-04-25T09:59:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f7395e21740 10 client.7728 fetch_fsmap learned FSMap version 58 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f7395e21740 1 -- 172.21.15.19:0/347183824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x559b527d96b0 con 0x559b52adfad0 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f736b7fe640 1 -- 172.21.15.19:0/347183824 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738403d890 con 0x559b52adfad0 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f736b7fe640 1 -- 172.21.15.19:0/347183824 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7380025020 con 0x559b52adfad0 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f7395e21740 10 client.7728 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f7395e21740 10 client.7728 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f7395e21740 20 client.7728 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f7395e21740 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x559b52ac0f30 0x559b52aadc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f7395e21740 4 client.7728 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f7395e21740 1 -- 172.21.15.19:0/347183824 --> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b52a59920 con 0x559b52ac0f30 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.011+0000 7f7390de2640 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x559b52ac0f30 0x559b52aadc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.012+0000 7f736b7fe640 10 client.7728 ms_handle_connect on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.012+0000 7f7390de2640 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x559b52ac0f30 0x559b52aadc40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff5afe15606944d server_cookie=6e0ec6295fb04ae in_seq=0 out_seq=0 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.012+0000 7f736b7fe640 1 -- 172.21.15.19:0/347183824 <== mds.0 v2:172.21.15.19:6832/3589967477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b52a59920 con 0x559b52ac0f30 2024-04-25T09:59:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.012+0000 7f736b7fe640 10 client.7728 handle_command_reply: tid=0 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.012+0000 7f7395e21740 1 client.7728 shutdown 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f7395e21740 1 -- 172.21.15.19:0/347183824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 msgr2=0x559b52a91940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f7395e21740 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52a91940 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f7380000f60 tx=0x7f73800116e0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f7395e21740 1 -- 172.21.15.19:0/347183824 shutdown_connections 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f7395e21740 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x559b52ac0f30 0x559b52aadc40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f7395e21740 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52ac5040 0x559b52a91e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f7395e21740 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52a91940 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f736b7fe640 0 client.7728 ms_handle_reset on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f7395e21740 1 --2- 172.21.15.19:0/347183824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac2230 0x559b52ab0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f7395e21740 1 -- 172.21.15.19:0/347183824 >> 172.21.15.19:0/347183824 conn(0x559b52a97f10 msgr2=0x559b52a971b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f7395e21740 1 -- 172.21.15.19:0/347183824 shutdown_connections 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f7395e21740 1 -- 172.21.15.19:0/347183824 wait complete. 2024-04-25T09:59:13.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.013+0000 7f7395e21740 20 client.7728 trim_cache size 0 max 16384 2024-04-25T09:59:13.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.029+0000 7f7395e21740 1 Processor -- start 2024-04-25T09:59:13.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.029+0000 7f7395e21740 1 -- start start 2024-04-25T09:59:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.029+0000 7f7395e21740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52a91be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.029+0000 7f7395e21740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac5040 0x559b52aade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.029+0000 7f7395e21740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52adfad0 0x559b52aae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.029+0000 7f7395e21740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c1fd5c0 con 0x559b52ac2230 2024-04-25T09:59:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.029+0000 7f7395e21740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c138030 con 0x559b52adfad0 2024-04-25T09:59:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.029+0000 7f7395e21740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c1f5cf0 con 0x559b52ac5040 2024-04-25T09:59:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.029+0000 7f7390de2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52a91be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.030+0000 7f7390de2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52a91be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41102/0 (socket says 172.21.15.19:41102) 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.030+0000 7f7390de2640 1 -- 172.21.15.19:0/3518373834 learned_addr learned my addr 172.21.15.19:0/3518373834 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.030+0000 7f7390de2640 1 -- 172.21.15.19:0/3518373834 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac5040 msgr2=0x559b52aade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.030+0000 7f7390de2640 1 --2- 172.21.15.19:0/3518373834 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac5040 0x559b52aade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.030+0000 7f7390de2640 1 -- 172.21.15.19:0/3518373834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52adfad0 msgr2=0x559b52aae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.030+0000 7f7390de2640 1 --2- 172.21.15.19:0/3518373834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52adfad0 0x559b52aae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.030+0000 7f7390de2640 1 -- 172.21.15.19:0/3518373834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b52af77c0 con 0x559b52ac2230 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.030+0000 7f7390de2640 1 --2- 172.21.15.19:0/3518373834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52a91be0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7384000f80 tx=0x7f738403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bd292f76c1ed93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.030+0000 7f7371ffb640 1 -- 172.21.15.19:0/3518373834 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384007720 con 0x559b52ac2230 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.030+0000 7f7371ffb640 1 -- 172.21.15.19:0/3518373834 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738405d460 con 0x559b52ac2230 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.030+0000 7f7371ffb640 1 -- 172.21.15.19:0/3518373834 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738405d810 con 0x559b52ac2230 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.031+0000 7f7395e21740 1 -- 172.21.15.19:0/3518373834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 msgr2=0x559b52a91be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.031+0000 7f7395e21740 1 --2- 172.21.15.19:0/3518373834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52a91be0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7384000f80 tx=0x7f738403f040 comp rx=0 tx=0).stop 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.031+0000 7f7395e21740 1 -- 172.21.15.19:0/3518373834 shutdown_connections 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.031+0000 7f7395e21740 1 --2- 172.21.15.19:0/3518373834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52adfad0 0x559b52aae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.031+0000 7f7395e21740 1 --2- 172.21.15.19:0/3518373834 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac5040 0x559b52aade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.031+0000 7f7395e21740 1 --2- 172.21.15.19:0/3518373834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52ac2230 0x559b52a91be0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.031+0000 7f7395e21740 1 -- 172.21.15.19:0/3518373834 >> 172.21.15.19:0/3518373834 conn(0x559b52a97f10 msgr2=0x559b52a982f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.031+0000 7f7395e21740 1 -- 172.21.15.19:0/3518373834 shutdown_connections 2024-04-25T09:59:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.031+0000 7f7395e21740 1 -- 172.21.15.19:0/3518373834 wait complete. 2024-04-25T09:59:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.031+0000 7f7395e21740 1 -- start start 2024-04-25T09:59:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.031+0000 7f7395e21740 1 Processor -- start 2024-04-25T09:59:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.032+0000 7f7395e21740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52ac2230 0x559b52aade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.032+0000 7f7395e21740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac5040 0x559b52aae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.032+0000 7f7395e21740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52b508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.032+0000 7f7395e21740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b52ac0db0 con 0x559b52adfad0 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.032+0000 7f7395e21740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b52ac6010 con 0x559b52ac2230 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.032+0000 7f7395e21740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b52ac1380 con 0x559b52ac5040 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.032+0000 7f7395e21740 10 client.0 _ll_register_callbacks cb 0x559b52a92c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.032+0000 7f73915e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52b508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.032+0000 7f73915e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52b508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41110/0 (socket says 172.21.15.19:41110) 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.032+0000 7f73915e3640 1 -- 172.21.15.19:0/3048832066 learned_addr learned my addr 172.21.15.19:0/3048832066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f73915e3640 1 -- 172.21.15.19:0/3048832066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac5040 msgr2=0x559b52aae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f7390de2640 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52ac2230 0x559b52aade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f73915e3640 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac5040 0x559b52aae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f73915e3640 1 -- 172.21.15.19:0/3048832066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52ac2230 msgr2=0x559b52aade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f73915e3640 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52ac2230 0x559b52aade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f73915e3640 1 -- 172.21.15.19:0/3048832066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b52b4d900 con 0x559b52adfad0 2024-04-25T09:59:13.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f73915e3640 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52b508a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f737c055000 tx=0x7f737c058ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d694b9f422290920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f736b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:59:13.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f736b7fe640 1 -- 172.21.15.19:0/3048832066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c021040 con 0x559b52adfad0 2024-04-25T09:59:13.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f736b7fe640 1 -- 172.21.15.19:0/3048832066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f737c08a2b0 con 0x559b52adfad0 2024-04-25T09:59:13.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f7395e21740 1 -- 172.21.15.19:0/3048832066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f738405d460 con 0x559b52adfad0 2024-04-25T09:59:13.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f736b7fe640 1 -- 172.21.15.19:0/3048832066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f737c05db60 con 0x559b52adfad0 2024-04-25T09:59:13.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f7395e21740 10 client.7734 fetch_fsmap learned FSMap version 58 2024-04-25T09:59:13.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f7395e21740 1 -- 172.21.15.19:0/3048832066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x559b52aaf1e0 con 0x559b52adfad0 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f736b7fe640 1 -- 172.21.15.19:0/3048832066 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738405d460 con 0x559b52adfad0 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f736b7fe640 1 -- 172.21.15.19:0/3048832066 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f737c057ff0 con 0x559b52adfad0 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f7395e21740 10 client.7734 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f7395e21740 10 client.7734 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f7395e21740 20 client.7734 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f7395e21740 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x559b52b51190 0x559b52a95de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f7395e21740 4 client.7734 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.033+0000 7f7395e21740 1 -- 172.21.15.19:0/3048832066 --> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559b52a07f50 con 0x559b52b51190 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7390de2640 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x559b52b51190 0x559b52a95de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f736b7fe640 10 client.7734 ms_handle_connect on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7390de2640 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x559b52b51190 0x559b52a95de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a206adf140f34e0d server_cookie=1283512e8c811af in_seq=0 out_seq=0 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f736b7fe640 1 -- 172.21.15.19:0/3048832066 <== mds.0 v2:172.21.15.19:6832/3589967477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x559b52a07f50 con 0x559b52b51190 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f736b7fe640 10 client.7734 handle_command_reply: tid=0 2024-04-25T09:59:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 1 client.7734 shutdown 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 1 -- 172.21.15.19:0/3048832066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 msgr2=0x559b52b508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52b508a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f737c055000 tx=0x7f737c058ce0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 1 -- 172.21.15.19:0/3048832066 shutdown_connections 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x559b52b51190 0x559b52a95de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b52adfad0 0x559b52b508a0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559b52ac5040 0x559b52aae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f736b7fe640 0 client.7734 ms_handle_reset on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 1 --2- 172.21.15.19:0/3048832066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b52ac2230 0x559b52aade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 1 -- 172.21.15.19:0/3048832066 >> 172.21.15.19:0/3048832066 conn(0x559b52a97f10 msgr2=0x559b52a91be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 1 -- 172.21.15.19:0/3048832066 shutdown_connections 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 1 -- 172.21.15.19:0/3048832066 wait complete. 2024-04-25T09:59:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.034+0000 7f7395e21740 20 client.7734 trim_cache size 0 max 16384 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 7593, 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 4.67181815, 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 58, 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 95, 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 95, 2024-04-25T09:59:13.045 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 10.016863515000001 2024-04-25T09:59:13.046 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:59:13.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 -- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737806b4d0 msgr2=0x7f737806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737806b4d0 0x7f737806d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7384034000 tx=0x7f738402f000 comp rx=0 tx=0).stop 2024-04-25T09:59:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 -- 172.21.15.19:0/4068904600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c144d50 msgr2=0x7f738c1392e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c144d50 0x7f738c1392e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f737c00e880 tx=0x7f737c00ed40 comp rx=0 tx=0).stop 2024-04-25T09:59:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 -- 172.21.15.19:0/4068904600 shutdown_connections 2024-04-25T09:59:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737806b4d0 0x7f737806d990 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f738c144d50 0x7f738c1392e0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f738c1446d0 0x7f738c138da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 --2- 172.21.15.19:0/4068904600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f738c06e8a0 0x7f738c15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 -- 172.21.15.19:0/4068904600 >> 172.21.15.19:0/4068904600 conn(0x7f738c06abe0 msgr2=0x7f738c0b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 -- 172.21.15.19:0/4068904600 shutdown_connections 2024-04-25T09:59:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.035+0000 7f7393046640 1 -- 172.21.15.19:0/4068904600 wait complete. 2024-04-25T09:59:13.050 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:59:13.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.146+0000 7ff8450f7640 1 Processor -- start 2024-04-25T09:59:13.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.146+0000 7ff8450f7640 1 -- start start 2024-04-25T09:59:13.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff8450f7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8400d0ad0 0x7ff840168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff8450f7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8400b9940 con 0x7ff8400d2e90 2024-04-25T09:59:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff8450f7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8400d1df0 con 0x7ff84015dd30 2024-04-25T09:59:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff8450f7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400b7a50 con 0x7ff8400d0ad0 2024-04-25T09:59:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83e59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff8400d2e90 0x7ff8400d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50672/0 (socket says 172.21.15.19:50672) 2024-04-25T09:59:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83e59c640 1 -- 172.21.15.19:0/1599154965 learned_addr learned my addr 172.21.15.19:0/1599154965 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83f59e640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8400d0ad0 0x7ff840168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310589791 0 0) 0x7ff8400b9940 con 0x7ff8400d2e90 2024-04-25T09:59:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff828003520 con 0x7ff8400d2e90 2024-04-25T09:59:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187474970 0 0) 0x7ff8400d1df0 con 0x7ff84015dd30 2024-04-25T09:59:13.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8400b9940 con 0x7ff84015dd30 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2335490358 0 0) 0x7ff828003520 con 0x7ff8400d2e90 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8400d0ad0 msgr2=0x7ff840168760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83dd9b640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8400d0ad0 0x7ff840168760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 >> v1:172.21.15.112:6789/0 conn(0x7ff84015dd30 legacy=0x7ff8400b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.147+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8400d20d0 con 0x7ff8400d2e90 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7ff8340017f0 con 0x7ff8400d2e90 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 >> v1:172.21.15.19:6789/0 conn(0x7ff8400d2e90 legacy=0x7ff8400d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83dd9b640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff828003b70 0x7ff828003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83dd9b640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8400d0ad0 0x7ff828004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83dd9b640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff828004d40 0x7ff8280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400b7a50 con 0x7ff828003b70 2024-04-25T09:59:13.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82800bb50 con 0x7ff828004d40 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82800bd20 con 0x7ff8400d0ad0 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83ed9d640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff828003b70 0x7ff828003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83e59c640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8400d0ad0 0x7ff828004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83f59e640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff828004d40 0x7ff8280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83ed9d640 1 -- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8400d0ad0 msgr2=0x7ff828004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83ed9d640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8400d0ad0 0x7ff828004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83ed9d640 1 -- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff828004d40 msgr2=0x7ff8280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83ed9d640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff828004d40 0x7ff8280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83ed9d640 1 -- 172.21.15.19:0/1599154965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff828003520 con 0x7ff828003b70 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.148+0000 7ff83f59e640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff828004d40 0x7ff8280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.149+0000 7ff83ed9d640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff828003b70 0x7ff828003f70 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff830001900 tx=0x7ff828005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd426af43227350 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.149+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8300014f0 con 0x7ff828003b70 2024-04-25T09:59:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.149+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8300052d0 con 0x7ff828003b70 2024-04-25T09:59:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.149+0000 7ff83dd9b640 1 -- 172.21.15.19:0/1599154965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83002ac80 con 0x7ff828003b70 2024-04-25T09:59:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.149+0000 7ff8450f7640 1 -- 172.21.15.19:0/1599154965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff828003b70 msgr2=0x7ff828003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.149+0000 7ff8450f7640 1 --2- 172.21.15.19:0/1599154965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff828003b70 0x7ff828003f70 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff830001900 tx=0x7ff828005830 comp rx=0 tx=0).stop 2024-04-25T09:59:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.149+0000 7ff83f59e640 1 -- 172.21.15.19:0/1599154965 reap_dead start 2024-04-25T09:59:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.149+0000 7ff8450f7640 1 -- 172.21.15.19:0/1599154965 shutdown_connections 2024-04-25T09:59:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.149+0000 7ff8450f7640 1 -- 172.21.15.19:0/1599154965 >> 172.21.15.19:0/1599154965 conn(0x7ff840061c30 msgr2=0x7ff84006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.149+0000 7ff8450f7640 1 -- 172.21.15.19:0/1599154965 shutdown_connections 2024-04-25T09:59:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.149+0000 7ff8450f7640 1 -- 172.21.15.19:0/1599154965 wait complete. 2024-04-25T09:59:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.151+0000 7ff8450f7640 1 Processor -- start 2024-04-25T09:59:13.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.151+0000 7ff8450f7640 1 -- start start 2024-04-25T09:59:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff8450f7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff828003b70 0x7ff84014b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff8450f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8401e7a70 0x7ff8400ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff8450f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8401e7ef0 0x7ff8400ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff8450f7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400d2970 con 0x7ff8401e7ef0 2024-04-25T09:59:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff8450f7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400d2690 con 0x7ff8401e7a70 2024-04-25T09:59:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff8450f7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400d1a10 con 0x7ff828003b70 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff83f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8401e7ef0 0x7ff8400ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff83f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8401e7ef0 0x7ff8400ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41114/0 (socket says 172.21.15.19:41114) 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff83f59e640 1 -- 172.21.15.19:0/1697769379 learned_addr learned my addr 172.21.15.19:0/1697769379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff83e59c640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8401e7a70 0x7ff8400ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff83ed9d640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff828003b70 0x7ff84014b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff83f59e640 1 -- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff828003b70 msgr2=0x7ff84014b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff83f59e640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff828003b70 0x7ff84014b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff83f59e640 1 -- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8401e7a70 msgr2=0x7ff8400ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff83f59e640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8401e7a70 0x7ff8400ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff83f59e640 1 -- 172.21.15.19:0/1697769379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8401e84a0 con 0x7ff8401e7ef0 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff83f59e640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8401e7ef0 0x7ff8400ac900 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff84008ce70 tx=0x7ff838010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff8277fe640 1 -- 172.21.15.19:0/1697769379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838003990 con 0x7ff8401e7ef0 2024-04-25T09:59:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff8277fe640 1 -- 172.21.15.19:0/1697769379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff838003b50 con 0x7ff8401e7ef0 2024-04-25T09:59:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff8450f7640 1 -- 172.21.15.19:0/1697769379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8401e8780 con 0x7ff8401e7ef0 2024-04-25T09:59:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.152+0000 7ff8277fe640 1 -- 172.21.15.19:0/1697769379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838004020 con 0x7ff8401e7ef0 2024-04-25T09:59:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.153+0000 7ff8450f7640 1 -- 172.21.15.19:0/1697769379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff840149a40 con 0x7ff8401e7ef0 2024-04-25T09:59:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.154+0000 7ff8450f7640 1 -- 172.21.15.19:0/1697769379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff840160210 con 0x7ff8401e7ef0 2024-04-25T09:59:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.154+0000 7ff8277fe640 1 -- 172.21.15.19:0/1697769379 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff838005d10 con 0x7ff8401e7ef0 2024-04-25T09:59:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.154+0000 7ff8277fe640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff82c06b500 0x7ff82c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.155+0000 7ff8277fe640 1 -- 172.21.15.19:0/1697769379 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff83808a9a0 con 0x7ff8401e7ef0 2024-04-25T09:59:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.155+0000 7ff83ed9d640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff82c06b500 0x7ff82c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.155+0000 7ff83ed9d640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff82c06b500 0x7ff82c06d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff830032000 tx=0x7ff830009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.159+0000 7ff8277fe640 1 -- 172.21.15.19:0/1697769379 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff840160210 con 0x7ff8401e7ef0 2024-04-25T09:59:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.379+0000 7ff8450f7640 1 -- 172.21.15.19:0/1697769379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff840073990 con 0x7ff8401e7ef0 2024-04-25T09:59:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.380+0000 7ff8277fe640 1 -- 172.21.15.19:0/1697769379 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff838025020 con 0x7ff8401e7ef0 2024-04-25T09:59:13.384 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:59:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 -- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff82c06b500 msgr2=0x7ff82c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff82c06b500 0x7ff82c06d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff830032000 tx=0x7ff830009030 comp rx=0 tx=0).stop 2024-04-25T09:59:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 -- 172.21.15.19:0/1697769379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8401e7ef0 msgr2=0x7ff8400ac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8401e7ef0 0x7ff8400ac900 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff84008ce70 tx=0x7ff838010590 comp rx=0 tx=0).stop 2024-04-25T09:59:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 -- 172.21.15.19:0/1697769379 shutdown_connections 2024-04-25T09:59:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff82c06b500 0x7ff82c06d9c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8401e7ef0 0x7ff8400ac900 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8401e7a70 0x7ff8400ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 --2- 172.21.15.19:0/1697769379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff828003b70 0x7ff84014b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 -- 172.21.15.19:0/1697769379 >> 172.21.15.19:0/1697769379 conn(0x7ff840061c30 msgr2=0x7ff8400d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 -- 172.21.15.19:0/1697769379 shutdown_connections 2024-04-25T09:59:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:13.382+0000 7ff8450f7640 1 -- 172.21.15.19:0/1697769379 wait complete. 2024-04-25T09:59:13.397 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:59:13.397 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:59:13.397 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T09:59:13.397 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:59:13.397 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:13.397 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:59:13.397 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:59:13.397 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:59:13.398 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:13.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:13.398 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T09:59:13.413 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T09:59:13.414 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42392sec preferred_lft 42392sec 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:59:13.415 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:13.416 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T09:59:13.416 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:13.416 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:59:13.416 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:59:13.416 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T09:59:13.416 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:13.416 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:59:13.416 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:59:13.416 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T09:59:13.416 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:13.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:13.417 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:59:13.476 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T09:59:13.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:13.478 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T09:59:13.535 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T09:59:13.535 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:59:13.535 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:59:13.535 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:59:13.535 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:59:13.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:13.536 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:59:13.536 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:13.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:59:13.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:59:13.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:59:13.536 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:13.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:13.620 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:59:13.636 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:59:13.636 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:59:13.636 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:13.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:13.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T09:59:13.701 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:59:13.701 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:13.757 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:13.757 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:59:13.757 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:13.812 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:59:13.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:13.813 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:13.895 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:13.895 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T09:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T09:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T09:59:13.904 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:59:13.904 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:59:13.904 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:59:13.904 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:59:13.904 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:59:13.905 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:59:13.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:13.906 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:13.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:13.971 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:59:14.022 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T09:59:14.018+0000 7fa0757a2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:59:14.033 INFO:teuthology.orchestra.run.smithi019.stdout:client6284 2024-04-25T09:59:14.033 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:59:14.033 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:14.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:14.088 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:14.088 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:14.088 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:59:14.088 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:14.088 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:14.088 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:14.089 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:14.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785134 s, 134 MB/s 2024-04-25T09:59:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901025 s, 116 MB/s 2024-04-25T09:59:14.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.180 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086087 s, 122 MB/s 2024-04-25T09:59:14.190 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.190 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.190 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880005 s, 119 MB/s 2024-04-25T09:59:14.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.199 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755437 s, 139 MB/s 2024-04-25T09:59:14.206 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.206 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.206 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641497 s, 163 MB/s 2024-04-25T09:59:14.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.214 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774867 s, 135 MB/s 2024-04-25T09:59:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613874 s, 171 MB/s 2024-04-25T09:59:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007213 s, 145 MB/s 2024-04-25T09:59:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824343 s, 127 MB/s 2024-04-25T09:59:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693174 s, 151 MB/s 2024-04-25T09:59:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787831 s, 133 MB/s 2024-04-25T09:59:14.263 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.263 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.263 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717901 s, 146 MB/s 2024-04-25T09:59:14.270 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.271 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.271 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643981 s, 163 MB/s 2024-04-25T09:59:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695702 s, 151 MB/s 2024-04-25T09:59:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756235 s, 139 MB/s 2024-04-25T09:59:14.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.295 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074425 s, 141 MB/s 2024-04-25T09:59:14.303 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.303 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.303 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067057 s, 156 MB/s 2024-04-25T09:59:14.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.311 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783003 s, 134 MB/s 2024-04-25T09:59:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651604 s, 161 MB/s 2024-04-25T09:59:14.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.327 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745535 s, 141 MB/s 2024-04-25T09:59:14.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.336 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073182 s, 143 MB/s 2024-04-25T09:59:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.343 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644062 s, 163 MB/s 2024-04-25T09:59:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714141 s, 147 MB/s 2024-04-25T09:59:14.358 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.359 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067324 s, 156 MB/s 2024-04-25T09:59:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.368 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780757 s, 134 MB/s 2024-04-25T09:59:14.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.375 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686928 s, 153 MB/s 2024-04-25T09:59:14.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.383 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686125 s, 153 MB/s 2024-04-25T09:59:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766289 s, 137 MB/s 2024-04-25T09:59:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663521 s, 158 MB/s 2024-04-25T09:59:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682008 s, 154 MB/s 2024-04-25T09:59:14.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.416 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771633 s, 136 MB/s 2024-04-25T09:59:14.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:14.417 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:14.417 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:14.417 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:59:14.417 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:14.417 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:14.417 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:14.417 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:14.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.483 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741435 s, 141 MB/s 2024-04-25T09:59:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673673 s, 156 MB/s 2024-04-25T09:59:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725196 s, 145 MB/s 2024-04-25T09:59:14.507 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077674 s, 135 MB/s 2024-04-25T09:59:14.516 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.516 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.516 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072489 s, 145 MB/s 2024-04-25T09:59:14.525 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.525 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.525 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845871 s, 124 MB/s 2024-04-25T09:59:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.534 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760143 s, 138 MB/s 2024-04-25T09:59:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631698 s, 166 MB/s 2024-04-25T09:59:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.548 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639416 s, 164 MB/s 2024-04-25T09:59:14.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.556 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706494 s, 148 MB/s 2024-04-25T09:59:14.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.565 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730468 s, 144 MB/s 2024-04-25T09:59:14.574 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.574 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.574 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863929 s, 121 MB/s 2024-04-25T09:59:14.582 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.582 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.582 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734373 s, 143 MB/s 2024-04-25T09:59:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.590 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616821 s, 170 MB/s 2024-04-25T09:59:14.597 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.597 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.598 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712108 s, 147 MB/s 2024-04-25T09:59:14.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.606 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807702 s, 130 MB/s 2024-04-25T09:59:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703585 s, 149 MB/s 2024-04-25T09:59:14.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.622 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645078 s, 163 MB/s 2024-04-25T09:59:14.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.630 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763585 s, 137 MB/s 2024-04-25T09:59:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691065 s, 152 MB/s 2024-04-25T09:59:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766107 s, 137 MB/s 2024-04-25T09:59:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660183 s, 159 MB/s 2024-04-25T09:59:14.661 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.661 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.662 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630922 s, 166 MB/s 2024-04-25T09:59:14.670 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.670 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.670 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754144 s, 139 MB/s 2024-04-25T09:59:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656742 s, 160 MB/s 2024-04-25T09:59:14.685 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.685 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.685 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718069 s, 146 MB/s 2024-04-25T09:59:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691314 s, 152 MB/s 2024-04-25T09:59:14.700 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.700 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.701 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621754 s, 169 MB/s 2024-04-25T09:59:14.707 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.708 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.708 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620054 s, 169 MB/s 2024-04-25T09:59:14.716 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.716 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.716 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753637 s, 139 MB/s 2024-04-25T09:59:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717136 s, 146 MB/s 2024-04-25T09:59:14.732 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.733 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.733 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741439 s, 141 MB/s 2024-04-25T09:59:14.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:14.734 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:14.734 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:14.734 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:59:14.734 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:14.734 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:14.734 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:14.734 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626223 s, 167 MB/s 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649162 s, 162 MB/s 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062989 s, 166 MB/s 2024-04-25T09:59:14.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.820 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644593 s, 163 MB/s 2024-04-25T09:59:14.827 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.827 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.827 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620569 s, 169 MB/s 2024-04-25T09:59:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638826 s, 164 MB/s 2024-04-25T09:59:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716165 s, 146 MB/s 2024-04-25T09:59:14.849 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.849 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.850 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610415 s, 172 MB/s 2024-04-25T09:59:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653677 s, 160 MB/s 2024-04-25T09:59:14.864 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.864 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.864 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642399 s, 163 MB/s 2024-04-25T09:59:14.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.871 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612587 s, 171 MB/s 2024-04-25T09:59:14.878 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.879 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625763 s, 168 MB/s 2024-04-25T09:59:14.886 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.887 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717499 s, 146 MB/s 2024-04-25T09:59:14.894 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.894 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.894 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064674 s, 162 MB/s 2024-04-25T09:59:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721193 s, 145 MB/s 2024-04-25T09:59:14.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706699 s, 148 MB/s 2024-04-25T09:59:14.917 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.917 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.917 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620429 s, 169 MB/s 2024-04-25T09:59:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731786 s, 143 MB/s 2024-04-25T09:59:14.934 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.934 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.934 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784406 s, 134 MB/s 2024-04-25T09:59:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609619 s, 172 MB/s 2024-04-25T09:59:14.949 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.949 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.949 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640222 s, 164 MB/s 2024-04-25T09:59:14.957 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.957 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.957 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750952 s, 140 MB/s 2024-04-25T09:59:14.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.966 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729492 s, 144 MB/s 2024-04-25T09:59:14.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.974 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752551 s, 139 MB/s 2024-04-25T09:59:14.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.982 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721118 s, 145 MB/s 2024-04-25T09:59:14.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.989 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610458 s, 172 MB/s 2024-04-25T09:59:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:14.996 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606928 s, 173 MB/s 2024-04-25T09:59:15.004 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.004 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.005 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753377 s, 139 MB/s 2024-04-25T09:59:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.012 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626363 s, 167 MB/s 2024-04-25T09:59:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742598 s, 141 MB/s 2024-04-25T09:59:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710652 s, 148 MB/s 2024-04-25T09:59:15.035 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.035 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.035 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646716 s, 162 MB/s 2024-04-25T09:59:15.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:15.037 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:15.037 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:15.037 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:59:15.037 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:15.037 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:15.037 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:15.037 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:15.102 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.102 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.102 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719806 s, 146 MB/s 2024-04-25T09:59:15.109 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.109 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.109 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064215 s, 163 MB/s 2024-04-25T09:59:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.117 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681251 s, 154 MB/s 2024-04-25T09:59:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.125 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.125 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650209 s, 161 MB/s 2024-04-25T09:59:15.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.133 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784154 s, 134 MB/s 2024-04-25T09:59:15.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.140 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601596 s, 174 MB/s 2024-04-25T09:59:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709475 s, 148 MB/s 2024-04-25T09:59:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685133 s, 153 MB/s 2024-04-25T09:59:15.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.164 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.164 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651558 s, 161 MB/s 2024-04-25T09:59:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.172 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717759 s, 146 MB/s 2024-04-25T09:59:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656663 s, 160 MB/s 2024-04-25T09:59:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683106 s, 154 MB/s 2024-04-25T09:59:15.195 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.195 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.195 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756184 s, 139 MB/s 2024-04-25T09:59:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066444 s, 158 MB/s 2024-04-25T09:59:15.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675968 s, 155 MB/s 2024-04-25T09:59:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.218 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677103 s, 155 MB/s 2024-04-25T09:59:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735448 s, 143 MB/s 2024-04-25T09:59:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653447 s, 160 MB/s 2024-04-25T09:59:15.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.243 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767055 s, 137 MB/s 2024-04-25T09:59:15.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.250 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688608 s, 152 MB/s 2024-04-25T09:59:15.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.258 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739491 s, 142 MB/s 2024-04-25T09:59:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647676 s, 162 MB/s 2024-04-25T09:59:15.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.273 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666693 s, 157 MB/s 2024-04-25T09:59:15.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.283 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880196 s, 119 MB/s 2024-04-25T09:59:15.290 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.290 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.290 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618269 s, 170 MB/s 2024-04-25T09:59:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679841 s, 154 MB/s 2024-04-25T09:59:15.306 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.307 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767223 s, 137 MB/s 2024-04-25T09:59:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739395 s, 142 MB/s 2024-04-25T09:59:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695422 s, 151 MB/s 2024-04-25T09:59:15.330 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.330 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.330 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621841 s, 169 MB/s 2024-04-25T09:59:15.338 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.338 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.338 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712053 s, 147 MB/s 2024-04-25T09:59:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928843 s, 113 MB/s 2024-04-25T09:59:15.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:15.350 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:15.350 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:15.350 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:59:15.350 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:15.351 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:15.351 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:15.351 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:15.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.416 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064247 s, 163 MB/s 2024-04-25T09:59:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648399 s, 162 MB/s 2024-04-25T09:59:15.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.431 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.431 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644533 s, 163 MB/s 2024-04-25T09:59:15.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.438 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658333 s, 159 MB/s 2024-04-25T09:59:15.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.446 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710515 s, 148 MB/s 2024-04-25T09:59:15.454 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.454 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.454 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693998 s, 151 MB/s 2024-04-25T09:59:15.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661032 s, 159 MB/s 2024-04-25T09:59:15.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.469 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678576 s, 155 MB/s 2024-04-25T09:59:15.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.477 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670758 s, 156 MB/s 2024-04-25T09:59:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598888 s, 175 MB/s 2024-04-25T09:59:15.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.491 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607223 s, 173 MB/s 2024-04-25T09:59:15.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.498 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631027 s, 166 MB/s 2024-04-25T09:59:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074028 s, 142 MB/s 2024-04-25T09:59:15.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.514 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593151 s, 177 MB/s 2024-04-25T09:59:15.522 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.522 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.522 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748831 s, 140 MB/s 2024-04-25T09:59:15.529 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.530 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.530 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695623 s, 151 MB/s 2024-04-25T09:59:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635302 s, 165 MB/s 2024-04-25T09:59:15.544 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.545 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677029 s, 155 MB/s 2024-04-25T09:59:15.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.553 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763472 s, 137 MB/s 2024-04-25T09:59:15.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.561 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712122 s, 147 MB/s 2024-04-25T09:59:15.569 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.569 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738677 s, 142 MB/s 2024-04-25T09:59:15.577 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.578 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711322 s, 147 MB/s 2024-04-25T09:59:15.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.586 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703444 s, 149 MB/s 2024-04-25T09:59:15.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.593 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664506 s, 158 MB/s 2024-04-25T09:59:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071174 s, 147 MB/s 2024-04-25T09:59:15.608 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.609 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.609 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660216 s, 159 MB/s 2024-04-25T09:59:15.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.617 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745905 s, 141 MB/s 2024-04-25T09:59:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681317 s, 154 MB/s 2024-04-25T09:59:15.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.634 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813318 s, 129 MB/s 2024-04-25T09:59:15.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.641 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599934 s, 175 MB/s 2024-04-25T09:59:15.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.648 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662831 s, 158 MB/s 2024-04-25T09:59:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653852 s, 160 MB/s 2024-04-25T09:59:15.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:15.657 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:15.657 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:15.657 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:59:15.657 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:15.657 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:15.657 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:15.657 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722688 s, 145 MB/s 2024-04-25T09:59:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735484 s, 143 MB/s 2024-04-25T09:59:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072049 s, 146 MB/s 2024-04-25T09:59:15.746 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.747 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.747 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638207 s, 164 MB/s 2024-04-25T09:59:15.754 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.754 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.754 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650554 s, 161 MB/s 2024-04-25T09:59:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.762 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690135 s, 152 MB/s 2024-04-25T09:59:15.769 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.769 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.769 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685899 s, 153 MB/s 2024-04-25T09:59:15.777 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.777 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.777 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692124 s, 152 MB/s 2024-04-25T09:59:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.785 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670818 s, 156 MB/s 2024-04-25T09:59:15.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.792 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609784 s, 172 MB/s 2024-04-25T09:59:15.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.800 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696802 s, 150 MB/s 2024-04-25T09:59:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072676 s, 144 MB/s 2024-04-25T09:59:15.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.815 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609362 s, 172 MB/s 2024-04-25T09:59:15.822 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.822 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.822 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649083 s, 162 MB/s 2024-04-25T09:59:15.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.831 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736119 s, 142 MB/s 2024-04-25T09:59:15.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.838 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689429 s, 152 MB/s 2024-04-25T09:59:15.846 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.847 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724375 s, 145 MB/s 2024-04-25T09:59:15.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.854 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620729 s, 169 MB/s 2024-04-25T09:59:15.862 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.863 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.863 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790345 s, 133 MB/s 2024-04-25T09:59:15.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.871 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723543 s, 145 MB/s 2024-04-25T09:59:15.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.879 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738293 s, 142 MB/s 2024-04-25T09:59:15.886 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.886 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.886 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612781 s, 171 MB/s 2024-04-25T09:59:15.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.893 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601325 s, 174 MB/s 2024-04-25T09:59:15.900 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.900 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.900 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642117 s, 163 MB/s 2024-04-25T09:59:15.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.909 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720092 s, 146 MB/s 2024-04-25T09:59:15.917 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.917 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.917 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734795 s, 143 MB/s 2024-04-25T09:59:15.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.925 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714676 s, 147 MB/s 2024-04-25T09:59:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723697 s, 145 MB/s 2024-04-25T09:59:15.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.941 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717723 s, 146 MB/s 2024-04-25T09:59:15.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.951 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085257 s, 123 MB/s 2024-04-25T09:59:15.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.958 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652011 s, 161 MB/s 2024-04-25T09:59:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061583 s, 170 MB/s 2024-04-25T09:59:15.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:15.966 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:15.966 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:15.966 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:59:15.966 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:15.966 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:15.966 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:15.967 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740781 s, 142 MB/s 2024-04-25T09:59:16.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.041 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717435 s, 146 MB/s 2024-04-25T09:59:16.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.048 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655723 s, 160 MB/s 2024-04-25T09:59:16.055 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.055 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.056 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629118 s, 167 MB/s 2024-04-25T09:59:16.064 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.064 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.064 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734092 s, 143 MB/s 2024-04-25T09:59:16.071 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.071 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.071 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647701 s, 162 MB/s 2024-04-25T09:59:16.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.079 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.079 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649999 s, 161 MB/s 2024-04-25T09:59:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689483 s, 152 MB/s 2024-04-25T09:59:16.096 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.096 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.096 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859561 s, 122 MB/s 2024-04-25T09:59:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592611 s, 177 MB/s 2024-04-25T09:59:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.111 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.111 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699906 s, 150 MB/s 2024-04-25T09:59:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.118 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635032 s, 165 MB/s 2024-04-25T09:59:16.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.126 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718811 s, 146 MB/s 2024-04-25T09:59:16.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.133 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634696 s, 165 MB/s 2024-04-25T09:59:16.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.142 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725532 s, 145 MB/s 2024-04-25T09:59:16.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.149 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612078 s, 171 MB/s 2024-04-25T09:59:16.157 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.157 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.157 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725021 s, 145 MB/s 2024-04-25T09:59:16.164 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.165 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.165 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683879 s, 153 MB/s 2024-04-25T09:59:16.173 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.173 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.173 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649522 s, 161 MB/s 2024-04-25T09:59:16.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.180 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688342 s, 152 MB/s 2024-04-25T09:59:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651816 s, 161 MB/s 2024-04-25T09:59:16.194 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.195 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.195 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661918 s, 158 MB/s 2024-04-25T09:59:16.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.202 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686921 s, 153 MB/s 2024-04-25T09:59:16.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.210 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705945 s, 149 MB/s 2024-04-25T09:59:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060898 s, 172 MB/s 2024-04-25T09:59:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722583 s, 145 MB/s 2024-04-25T09:59:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691775 s, 152 MB/s 2024-04-25T09:59:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629515 s, 167 MB/s 2024-04-25T09:59:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071046 s, 148 MB/s 2024-04-25T09:59:16.257 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.257 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.257 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712838 s, 147 MB/s 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706425 s, 148 MB/s 2024-04-25T09:59:16.274 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.274 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.274 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623391 s, 168 MB/s 2024-04-25T09:59:16.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:16.274 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:16.274 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:16.274 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:59:16.275 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:16.275 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:16.275 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:16.275 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:16.340 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.340 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.340 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667677 s, 157 MB/s 2024-04-25T09:59:16.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.348 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678617 s, 155 MB/s 2024-04-25T09:59:16.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.356 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070333 s, 149 MB/s 2024-04-25T09:59:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.364 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718533 s, 146 MB/s 2024-04-25T09:59:16.371 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.371 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.371 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629263 s, 167 MB/s 2024-04-25T09:59:16.380 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.380 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.380 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792648 s, 132 MB/s 2024-04-25T09:59:16.388 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.388 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.388 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715675 s, 147 MB/s 2024-04-25T09:59:16.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.396 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711769 s, 147 MB/s 2024-04-25T09:59:16.404 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.404 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.405 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730568 s, 144 MB/s 2024-04-25T09:59:16.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.413 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727521 s, 144 MB/s 2024-04-25T09:59:16.419 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.420 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593189 s, 177 MB/s 2024-04-25T09:59:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747703 s, 140 MB/s 2024-04-25T09:59:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642173 s, 163 MB/s 2024-04-25T09:59:16.443 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.444 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.444 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725702 s, 144 MB/s 2024-04-25T09:59:16.451 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.451 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.451 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645386 s, 162 MB/s 2024-04-25T09:59:16.459 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.459 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.459 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705954 s, 149 MB/s 2024-04-25T09:59:16.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.466 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671792 s, 156 MB/s 2024-04-25T09:59:16.481 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.482 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.482 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137608 s, 76.2 MB/s 2024-04-25T09:59:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.489 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651936 s, 161 MB/s 2024-04-25T09:59:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591455 s, 177 MB/s 2024-04-25T09:59:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668268 s, 157 MB/s 2024-04-25T09:59:16.510 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.510 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.510 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625905 s, 168 MB/s 2024-04-25T09:59:16.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.517 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636088 s, 165 MB/s 2024-04-25T09:59:16.525 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.525 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.525 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693114 s, 151 MB/s 2024-04-25T09:59:16.533 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.533 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.533 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066857 s, 157 MB/s 2024-04-25T09:59:16.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.541 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.541 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691303 s, 152 MB/s 2024-04-25T09:59:16.548 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.548 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.548 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648394 s, 162 MB/s 2024-04-25T09:59:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006054 s, 173 MB/s 2024-04-25T09:59:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628361 s, 167 MB/s 2024-04-25T09:59:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682817 s, 154 MB/s 2024-04-25T09:59:16.578 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.578 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.578 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689319 s, 152 MB/s 2024-04-25T09:59:16.586 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.586 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.586 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071225 s, 147 MB/s 2024-04-25T09:59:16.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:16.587 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:16.587 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:16.587 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:59:16.587 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:16.587 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:16.587 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:16.587 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:16.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.652 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.652 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653932 s, 160 MB/s 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668977 s, 157 MB/s 2024-04-25T09:59:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.668 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672867 s, 156 MB/s 2024-04-25T09:59:16.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.675 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700063 s, 150 MB/s 2024-04-25T09:59:16.683 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.683 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.683 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718943 s, 146 MB/s 2024-04-25T09:59:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704548 s, 149 MB/s 2024-04-25T09:59:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063485 s, 165 MB/s 2024-04-25T09:59:16.706 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.706 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.706 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718331 s, 146 MB/s 2024-04-25T09:59:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703135 s, 149 MB/s 2024-04-25T09:59:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646767 s, 162 MB/s 2024-04-25T09:59:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622063 s, 169 MB/s 2024-04-25T09:59:16.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.737 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731391 s, 143 MB/s 2024-04-25T09:59:16.745 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.745 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.745 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698263 s, 150 MB/s 2024-04-25T09:59:16.752 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.752 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.752 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617312 s, 170 MB/s 2024-04-25T09:59:16.759 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.759 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.759 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603728 s, 174 MB/s 2024-04-25T09:59:16.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.766 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620451 s, 169 MB/s 2024-04-25T09:59:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.774 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.774 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690659 s, 152 MB/s 2024-04-25T09:59:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657894 s, 159 MB/s 2024-04-25T09:59:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693402 s, 151 MB/s 2024-04-25T09:59:16.797 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.797 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.797 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707465 s, 148 MB/s 2024-04-25T09:59:16.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.805 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703904 s, 149 MB/s 2024-04-25T09:59:16.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.812 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616379 s, 170 MB/s 2024-04-25T09:59:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725654 s, 145 MB/s 2024-04-25T09:59:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.828 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627465 s, 167 MB/s 2024-04-25T09:59:16.835 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.835 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.835 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656225 s, 160 MB/s 2024-04-25T09:59:16.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.843 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070772 s, 148 MB/s 2024-04-25T09:59:16.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.850 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609813 s, 172 MB/s 2024-04-25T09:59:16.858 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.858 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.858 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738942 s, 142 MB/s 2024-04-25T09:59:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695018 s, 151 MB/s 2024-04-25T09:59:16.874 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.874 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.874 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712001 s, 147 MB/s 2024-04-25T09:59:16.882 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.882 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.882 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727334 s, 144 MB/s 2024-04-25T09:59:16.889 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.889 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.889 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622295 s, 169 MB/s 2024-04-25T09:59:16.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:16.890 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:16.890 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:16.891 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:59:16.891 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:16.891 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:16.891 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:16.891 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:16.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.956 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071326 s, 147 MB/s 2024-04-25T09:59:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702985 s, 149 MB/s 2024-04-25T09:59:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689475 s, 152 MB/s 2024-04-25T09:59:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620848 s, 169 MB/s 2024-04-25T09:59:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774839 s, 135 MB/s 2024-04-25T09:59:16.995 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:16.996 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:16.996 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719874 s, 146 MB/s 2024-04-25T09:59:17.003 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.004 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.004 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673345 s, 156 MB/s 2024-04-25T09:59:17.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.010 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595411 s, 176 MB/s 2024-04-25T09:59:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684007 s, 153 MB/s 2024-04-25T09:59:17.025 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.025 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.025 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676516 s, 155 MB/s 2024-04-25T09:59:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.033 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677793 s, 155 MB/s 2024-04-25T09:59:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662167 s, 158 MB/s 2024-04-25T09:59:17.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.049 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.049 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710115 s, 148 MB/s 2024-04-25T09:59:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610675 s, 172 MB/s 2024-04-25T09:59:17.064 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.064 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.064 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713186 s, 147 MB/s 2024-04-25T09:59:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074549 s, 141 MB/s 2024-04-25T09:59:17.080 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.080 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.080 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685061 s, 153 MB/s 2024-04-25T09:59:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722922 s, 145 MB/s 2024-04-25T09:59:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710766 s, 148 MB/s 2024-04-25T09:59:17.104 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.104 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.104 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703652 s, 149 MB/s 2024-04-25T09:59:17.111 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.111 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.111 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627579 s, 167 MB/s 2024-04-25T09:59:17.119 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.119 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.119 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070275 s, 149 MB/s 2024-04-25T09:59:17.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.127 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.127 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644805 s, 163 MB/s 2024-04-25T09:59:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625803 s, 168 MB/s 2024-04-25T09:59:17.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.141 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698581 s, 150 MB/s 2024-04-25T09:59:17.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.149 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646568 s, 162 MB/s 2024-04-25T09:59:17.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.156 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670054 s, 156 MB/s 2024-04-25T09:59:17.164 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.164 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.165 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725383 s, 145 MB/s 2024-04-25T09:59:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624893 s, 168 MB/s 2024-04-25T09:59:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.179 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590315 s, 178 MB/s 2024-04-25T09:59:17.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.187 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.187 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721153 s, 145 MB/s 2024-04-25T09:59:17.194 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.194 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.194 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678304 s, 155 MB/s 2024-04-25T09:59:17.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:17.196 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:17.197 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:17.197 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:59:17.197 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:17.197 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:17.197 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:17.197 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:17.261 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.261 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.261 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706198 s, 148 MB/s 2024-04-25T09:59:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630939 s, 166 MB/s 2024-04-25T09:59:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634966 s, 165 MB/s 2024-04-25T09:59:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737162 s, 142 MB/s 2024-04-25T09:59:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633189 s, 166 MB/s 2024-04-25T09:59:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695097 s, 151 MB/s 2024-04-25T09:59:17.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.307 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074252 s, 141 MB/s 2024-04-25T09:59:17.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.314 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610121 s, 172 MB/s 2024-04-25T09:59:17.321 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.321 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.321 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634176 s, 165 MB/s 2024-04-25T09:59:17.329 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.329 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.329 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658543 s, 159 MB/s 2024-04-25T09:59:17.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.337 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.337 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702056 s, 149 MB/s 2024-04-25T09:59:17.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.345 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696061 s, 151 MB/s 2024-04-25T09:59:17.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.353 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.353 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723007 s, 145 MB/s 2024-04-25T09:59:17.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.360 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638445 s, 164 MB/s 2024-04-25T09:59:17.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.367 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646973 s, 162 MB/s 2024-04-25T09:59:17.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.375 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654964 s, 160 MB/s 2024-04-25T09:59:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069224 s, 151 MB/s 2024-04-25T09:59:17.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.390 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660445 s, 159 MB/s 2024-04-25T09:59:17.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.398 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720761 s, 145 MB/s 2024-04-25T09:59:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.406 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069777 s, 150 MB/s 2024-04-25T09:59:17.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.414 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629493 s, 167 MB/s 2024-04-25T09:59:17.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.423 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826941 s, 127 MB/s 2024-04-25T09:59:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.430 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631388 s, 166 MB/s 2024-04-25T09:59:17.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.438 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754245 s, 139 MB/s 2024-04-25T09:59:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.446 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641706 s, 163 MB/s 2024-04-25T09:59:17.452 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.453 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598836 s, 175 MB/s 2024-04-25T09:59:17.460 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.460 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.460 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688218 s, 152 MB/s 2024-04-25T09:59:17.468 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.468 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.469 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717032 s, 146 MB/s 2024-04-25T09:59:17.475 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.475 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.475 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005951 s, 176 MB/s 2024-04-25T09:59:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601408 s, 174 MB/s 2024-04-25T09:59:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704138 s, 149 MB/s 2024-04-25T09:59:17.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.498 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668594 s, 157 MB/s 2024-04-25T09:59:17.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:17.499 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:17.499 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:17.499 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:59:17.499 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:17.499 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:17.499 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:17.499 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:17.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646622 s, 162 MB/s 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064387 s, 163 MB/s 2024-04-25T09:59:17.579 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.580 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.580 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708233 s, 148 MB/s 2024-04-25T09:59:17.587 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.587 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.587 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633112 s, 166 MB/s 2024-04-25T09:59:17.594 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.594 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.594 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632506 s, 166 MB/s 2024-04-25T09:59:17.602 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.602 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.602 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741858 s, 141 MB/s 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719552 s, 146 MB/s 2024-04-25T09:59:17.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.619 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611521 s, 171 MB/s 2024-04-25T09:59:17.626 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.626 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.626 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735669 s, 143 MB/s 2024-04-25T09:59:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748357 s, 140 MB/s 2024-04-25T09:59:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634625 s, 165 MB/s 2024-04-25T09:59:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748903 s, 140 MB/s 2024-04-25T09:59:17.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.658 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697203 s, 150 MB/s 2024-04-25T09:59:17.665 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.665 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684993 s, 153 MB/s 2024-04-25T09:59:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605152 s, 173 MB/s 2024-04-25T09:59:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612181 s, 171 MB/s 2024-04-25T09:59:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648065 s, 162 MB/s 2024-04-25T09:59:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065776 s, 159 MB/s 2024-04-25T09:59:17.702 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.702 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.702 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694778 s, 151 MB/s 2024-04-25T09:59:17.709 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.710 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063896 s, 164 MB/s 2024-04-25T09:59:17.717 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714392 s, 147 MB/s 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.725 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.725 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599838 s, 175 MB/s 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636488 s, 165 MB/s 2024-04-25T09:59:17.740 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.740 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.740 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726767 s, 144 MB/s 2024-04-25T09:59:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.748 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688342 s, 152 MB/s 2024-04-25T09:59:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.756 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702103 s, 149 MB/s 2024-04-25T09:59:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714193 s, 147 MB/s 2024-04-25T09:59:17.771 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.771 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.771 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631595 s, 166 MB/s 2024-04-25T09:59:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.778 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614816 s, 171 MB/s 2024-04-25T09:59:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662664 s, 158 MB/s 2024-04-25T09:59:17.793 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.793 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.793 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638274 s, 164 MB/s 2024-04-25T09:59:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734571 s, 143 MB/s 2024-04-25T09:59:17.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:17.802 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:17.803 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:17.803 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:59:17.803 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:17.803 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:17.803 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:17.803 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.868 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700804 s, 150 MB/s 2024-04-25T09:59:17.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.876 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717493 s, 146 MB/s 2024-04-25T09:59:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070294 s, 149 MB/s 2024-04-25T09:59:17.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070566 s, 149 MB/s 2024-04-25T09:59:17.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.899 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688953 s, 152 MB/s 2024-04-25T09:59:17.907 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.907 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.907 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688607 s, 152 MB/s 2024-04-25T09:59:17.915 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.915 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.916 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706427 s, 148 MB/s 2024-04-25T09:59:17.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.923 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689943 s, 152 MB/s 2024-04-25T09:59:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628705 s, 167 MB/s 2024-04-25T09:59:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613856 s, 171 MB/s 2024-04-25T09:59:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627647 s, 167 MB/s 2024-04-25T09:59:17.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.952 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700423 s, 150 MB/s 2024-04-25T09:59:17.960 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.960 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.960 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663559 s, 158 MB/s 2024-04-25T09:59:17.967 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.967 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.967 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622519 s, 168 MB/s 2024-04-25T09:59:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.974 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645535 s, 162 MB/s 2024-04-25T09:59:17.981 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.982 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645162 s, 163 MB/s 2024-04-25T09:59:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695542 s, 151 MB/s 2024-04-25T09:59:17.996 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:17.997 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:17.997 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615064 s, 170 MB/s 2024-04-25T09:59:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630981 s, 166 MB/s 2024-04-25T09:59:18.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.011 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636038 s, 165 MB/s 2024-04-25T09:59:18.019 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.019 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.019 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689041 s, 152 MB/s 2024-04-25T09:59:18.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677058 s, 155 MB/s 2024-04-25T09:59:18.034 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.034 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.034 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630364 s, 166 MB/s 2024-04-25T09:59:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678394 s, 155 MB/s 2024-04-25T09:59:18.050 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.050 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.052 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075982 s, 138 MB/s 2024-04-25T09:59:18.058 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.058 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.058 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073085 s, 143 MB/s 2024-04-25T09:59:18.066 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.066 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.066 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070601 s, 149 MB/s 2024-04-25T09:59:18.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.075 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754091 s, 139 MB/s 2024-04-25T09:59:18.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.083 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.083 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668381 s, 157 MB/s 2024-04-25T09:59:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654265 s, 160 MB/s 2024-04-25T09:59:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696886 s, 150 MB/s 2024-04-25T09:59:18.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.106 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657244 s, 160 MB/s 2024-04-25T09:59:18.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:18.107 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:18.107 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:18.107 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:59:18.107 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:18.107 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:18.107 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:18.108 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062472 s, 168 MB/s 2024-04-25T09:59:18.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.180 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785941 s, 133 MB/s 2024-04-25T09:59:18.187 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.187 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.187 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623062 s, 168 MB/s 2024-04-25T09:59:18.196 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.196 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.196 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792143 s, 132 MB/s 2024-04-25T09:59:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680113 s, 154 MB/s 2024-04-25T09:59:18.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.212 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064163 s, 163 MB/s 2024-04-25T09:59:18.219 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.219 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.219 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701351 s, 150 MB/s 2024-04-25T09:59:18.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.227 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644692 s, 163 MB/s 2024-04-25T09:59:18.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.234 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622632 s, 168 MB/s 2024-04-25T09:59:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689878 s, 152 MB/s 2024-04-25T09:59:18.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.251 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842792 s, 124 MB/s 2024-04-25T09:59:18.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.258 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651655 s, 161 MB/s 2024-04-25T09:59:18.265 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.265 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.265 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601594 s, 174 MB/s 2024-04-25T09:59:18.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.274 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704742 s, 149 MB/s 2024-04-25T09:59:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670872 s, 156 MB/s 2024-04-25T09:59:18.288 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.288 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.288 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631835 s, 166 MB/s 2024-04-25T09:59:18.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.295 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624718 s, 168 MB/s 2024-04-25T09:59:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.303 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709064 s, 148 MB/s 2024-04-25T09:59:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692584 s, 151 MB/s 2024-04-25T09:59:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712589 s, 147 MB/s 2024-04-25T09:59:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618886 s, 169 MB/s 2024-04-25T09:59:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668867 s, 157 MB/s 2024-04-25T09:59:18.341 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.342 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704649 s, 149 MB/s 2024-04-25T09:59:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631421 s, 166 MB/s 2024-04-25T09:59:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697205 s, 150 MB/s 2024-04-25T09:59:18.365 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.365 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.365 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706274 s, 148 MB/s 2024-04-25T09:59:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692541 s, 151 MB/s 2024-04-25T09:59:18.380 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.380 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.380 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068196 s, 154 MB/s 2024-04-25T09:59:18.394 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.394 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.394 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129112 s, 81.2 MB/s 2024-04-25T09:59:18.402 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.402 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744011 s, 141 MB/s 2024-04-25T09:59:18.410 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.410 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.410 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709717 s, 148 MB/s 2024-04-25T09:59:18.418 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.418 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.418 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630746 s, 166 MB/s 2024-04-25T09:59:18.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:18.419 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:18.419 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:18.419 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:59:18.419 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:18.419 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:18.419 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:18.420 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:18.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.485 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725584 s, 145 MB/s 2024-04-25T09:59:18.492 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.492 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.492 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669794 s, 157 MB/s 2024-04-25T09:59:18.499 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.499 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.499 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628823 s, 167 MB/s 2024-04-25T09:59:18.507 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.507 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.507 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705243 s, 149 MB/s 2024-04-25T09:59:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647316 s, 162 MB/s 2024-04-25T09:59:18.523 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.523 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.523 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723713 s, 145 MB/s 2024-04-25T09:59:18.530 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.530 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.530 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623818 s, 168 MB/s 2024-04-25T09:59:18.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.537 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612276 s, 171 MB/s 2024-04-25T09:59:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735724 s, 143 MB/s 2024-04-25T09:59:18.552 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.552 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.552 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601573 s, 174 MB/s 2024-04-25T09:59:18.560 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.560 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.560 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645373 s, 162 MB/s 2024-04-25T09:59:18.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.568 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722405 s, 145 MB/s 2024-04-25T09:59:18.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.575 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670433 s, 156 MB/s 2024-04-25T09:59:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654815 s, 160 MB/s 2024-04-25T09:59:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730375 s, 144 MB/s 2024-04-25T09:59:18.599 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.599 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.599 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674921 s, 155 MB/s 2024-04-25T09:59:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628929 s, 167 MB/s 2024-04-25T09:59:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625879 s, 168 MB/s 2024-04-25T09:59:18.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.621 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688193 s, 152 MB/s 2024-04-25T09:59:18.629 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.629 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.629 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742214 s, 141 MB/s 2024-04-25T09:59:18.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.637 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671381 s, 156 MB/s 2024-04-25T09:59:18.644 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.645 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.645 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690857 s, 152 MB/s 2024-04-25T09:59:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640374 s, 164 MB/s 2024-04-25T09:59:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705694 s, 149 MB/s 2024-04-25T09:59:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709364 s, 148 MB/s 2024-04-25T09:59:18.676 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.676 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.676 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689395 s, 152 MB/s 2024-04-25T09:59:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.684 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.684 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712193 s, 147 MB/s 2024-04-25T09:59:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742314 s, 141 MB/s 2024-04-25T09:59:18.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.699 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632846 s, 166 MB/s 2024-04-25T09:59:18.707 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726593 s, 144 MB/s 2024-04-25T09:59:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621398 s, 169 MB/s 2024-04-25T09:59:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749421 s, 140 MB/s 2024-04-25T09:59:18.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:18.724 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:18.724 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:18.724 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:59:18.724 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:18.725 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:18.725 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:18.725 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062405 s, 168 MB/s 2024-04-25T09:59:18.796 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.796 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.796 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642002 s, 163 MB/s 2024-04-25T09:59:18.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.804 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685538 s, 153 MB/s 2024-04-25T09:59:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.811 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629895 s, 166 MB/s 2024-04-25T09:59:18.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.818 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063501 s, 165 MB/s 2024-04-25T09:59:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669445 s, 157 MB/s 2024-04-25T09:59:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620886 s, 169 MB/s 2024-04-25T09:59:18.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.841 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645103 s, 163 MB/s 2024-04-25T09:59:18.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.849 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702211 s, 149 MB/s 2024-04-25T09:59:18.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.857 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729749 s, 144 MB/s 2024-04-25T09:59:18.866 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.866 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.866 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651231 s, 161 MB/s 2024-04-25T09:59:18.872 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.872 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.872 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072771 s, 144 MB/s 2024-04-25T09:59:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630489 s, 166 MB/s 2024-04-25T09:59:18.886 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.887 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631579 s, 166 MB/s 2024-04-25T09:59:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629792 s, 166 MB/s 2024-04-25T09:59:18.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.902 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709768 s, 148 MB/s 2024-04-25T09:59:18.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.910 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689063 s, 152 MB/s 2024-04-25T09:59:18.917 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718516 s, 146 MB/s 2024-04-25T09:59:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.925 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594571 s, 176 MB/s 2024-04-25T09:59:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736377 s, 142 MB/s 2024-04-25T09:59:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701884 s, 149 MB/s 2024-04-25T09:59:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006773 s, 155 MB/s 2024-04-25T09:59:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066332 s, 158 MB/s 2024-04-25T09:59:18.964 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.964 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.964 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705597 s, 149 MB/s 2024-04-25T09:59:18.972 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.972 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.972 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734233 s, 143 MB/s 2024-04-25T09:59:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.980 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.981 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742274 s, 141 MB/s 2024-04-25T09:59:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739651 s, 142 MB/s 2024-04-25T09:59:18.997 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:18.997 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:18.997 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752224 s, 139 MB/s 2024-04-25T09:59:19.005 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.005 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.006 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073588 s, 142 MB/s 2024-04-25T09:59:19.014 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.014 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.014 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006742 s, 156 MB/s 2024-04-25T09:59:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698015 s, 150 MB/s 2024-04-25T09:59:19.028 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630314 s, 166 MB/s 2024-04-25T09:59:19.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:19.030 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:19.030 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:19.030 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:59:19.030 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:19.030 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:19.030 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:19.030 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:19.094 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.095 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.095 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661778 s, 158 MB/s 2024-04-25T09:59:19.102 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.102 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.103 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717729 s, 146 MB/s 2024-04-25T09:59:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744393 s, 141 MB/s 2024-04-25T09:59:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.118 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617877 s, 170 MB/s 2024-04-25T09:59:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690949 s, 152 MB/s 2024-04-25T09:59:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067395 s, 156 MB/s 2024-04-25T09:59:19.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.140 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615138 s, 170 MB/s 2024-04-25T09:59:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656323 s, 160 MB/s 2024-04-25T09:59:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.156 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704056 s, 149 MB/s 2024-04-25T09:59:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628239 s, 167 MB/s 2024-04-25T09:59:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731607 s, 143 MB/s 2024-04-25T09:59:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622679 s, 168 MB/s 2024-04-25T09:59:19.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.186 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705334 s, 149 MB/s 2024-04-25T09:59:19.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.194 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.194 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659251 s, 159 MB/s 2024-04-25T09:59:19.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.201 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067122 s, 156 MB/s 2024-04-25T09:59:19.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.209 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682897 s, 154 MB/s 2024-04-25T09:59:19.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.217 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720083 s, 146 MB/s 2024-04-25T09:59:19.225 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.225 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.225 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699126 s, 150 MB/s 2024-04-25T09:59:19.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.233 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074845 s, 140 MB/s 2024-04-25T09:59:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.241 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685352 s, 153 MB/s 2024-04-25T09:59:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.248 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.249 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641181 s, 164 MB/s 2024-04-25T09:59:19.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.257 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627005 s, 167 MB/s 2024-04-25T09:59:19.264 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.264 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.264 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693879 s, 151 MB/s 2024-04-25T09:59:19.272 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.272 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.272 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745853 s, 141 MB/s 2024-04-25T09:59:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.279 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654123 s, 160 MB/s 2024-04-25T09:59:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.286 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617769 s, 170 MB/s 2024-04-25T09:59:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715429 s, 147 MB/s 2024-04-25T09:59:19.301 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619995 s, 169 MB/s 2024-04-25T09:59:19.309 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.309 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.310 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694266 s, 151 MB/s 2024-04-25T09:59:19.317 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.317 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.317 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652318 s, 161 MB/s 2024-04-25T09:59:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.324 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065116 s, 161 MB/s 2024-04-25T09:59:19.332 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.332 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.332 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730138 s, 144 MB/s 2024-04-25T09:59:19.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:19.334 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:19.334 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:19.334 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:59:19.334 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:19.334 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:19.334 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:19.334 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:19.400 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.400 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.400 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069714 s, 150 MB/s 2024-04-25T09:59:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.408 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707706 s, 148 MB/s 2024-04-25T09:59:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613294 s, 171 MB/s 2024-04-25T09:59:19.422 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.423 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658529 s, 159 MB/s 2024-04-25T09:59:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770436 s, 136 MB/s 2024-04-25T09:59:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.438 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635224 s, 165 MB/s 2024-04-25T09:59:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.446 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644446 s, 163 MB/s 2024-04-25T09:59:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.454 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663416 s, 158 MB/s 2024-04-25T09:59:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070603 s, 149 MB/s 2024-04-25T09:59:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.470 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732998 s, 143 MB/s 2024-04-25T09:59:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.478 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710427 s, 148 MB/s 2024-04-25T09:59:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.485 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631716 s, 166 MB/s 2024-04-25T09:59:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634519 s, 165 MB/s 2024-04-25T09:59:19.500 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.500 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.500 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720359 s, 146 MB/s 2024-04-25T09:59:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696386 s, 151 MB/s 2024-04-25T09:59:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745712 s, 141 MB/s 2024-04-25T09:59:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702967 s, 149 MB/s 2024-04-25T09:59:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.533 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779853 s, 134 MB/s 2024-04-25T09:59:19.541 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.542 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.542 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075059 s, 140 MB/s 2024-04-25T09:59:19.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.549 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688183 s, 152 MB/s 2024-04-25T09:59:19.557 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.557 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.557 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661028 s, 159 MB/s 2024-04-25T09:59:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.565 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670807 s, 156 MB/s 2024-04-25T09:59:19.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.573 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747704 s, 140 MB/s 2024-04-25T09:59:19.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.581 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680898 s, 154 MB/s 2024-04-25T09:59:19.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.589 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710084 s, 148 MB/s 2024-04-25T09:59:19.595 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.596 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599442 s, 175 MB/s 2024-04-25T09:59:19.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692937 s, 151 MB/s 2024-04-25T09:59:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815365 s, 129 MB/s 2024-04-25T09:59:19.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.620 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646068 s, 162 MB/s 2024-04-25T09:59:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705706 s, 149 MB/s 2024-04-25T09:59:19.636 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.636 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.636 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695431 s, 151 MB/s 2024-04-25T09:59:19.643 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.643 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.644 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683428 s, 153 MB/s 2024-04-25T09:59:19.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:19.645 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:19.645 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:19.645 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:59:19.646 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:19.646 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:19.646 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:19.646 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:19.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.712 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779513 s, 135 MB/s 2024-04-25T09:59:19.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.719 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718617 s, 146 MB/s 2024-04-25T09:59:19.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.727 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711292 s, 147 MB/s 2024-04-25T09:59:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.735 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685396 s, 153 MB/s 2024-04-25T09:59:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758201 s, 138 MB/s 2024-04-25T09:59:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692043 s, 152 MB/s 2024-04-25T09:59:19.759 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.759 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.759 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687966 s, 152 MB/s 2024-04-25T09:59:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729396 s, 144 MB/s 2024-04-25T09:59:19.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.776 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716634 s, 146 MB/s 2024-04-25T09:59:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640146 s, 164 MB/s 2024-04-25T09:59:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682116 s, 154 MB/s 2024-04-25T09:59:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628434 s, 167 MB/s 2024-04-25T09:59:19.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.805 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626936 s, 167 MB/s 2024-04-25T09:59:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724877 s, 145 MB/s 2024-04-25T09:59:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694236 s, 151 MB/s 2024-04-25T09:59:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066951 s, 157 MB/s 2024-04-25T09:59:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622341 s, 168 MB/s 2024-04-25T09:59:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597012 s, 176 MB/s 2024-04-25T09:59:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707745 s, 148 MB/s 2024-04-25T09:59:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723274 s, 145 MB/s 2024-04-25T09:59:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822997 s, 127 MB/s 2024-04-25T09:59:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634922 s, 165 MB/s 2024-04-25T09:59:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700893 s, 150 MB/s 2024-04-25T09:59:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688156 s, 152 MB/s 2024-04-25T09:59:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069477 s, 151 MB/s 2024-04-25T09:59:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604049 s, 174 MB/s 2024-04-25T09:59:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691594 s, 152 MB/s 2024-04-25T09:59:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787665 s, 133 MB/s 2024-04-25T09:59:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.930 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631244 s, 166 MB/s 2024-04-25T09:59:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734211 s, 143 MB/s 2024-04-25T09:59:19.945 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.946 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694411 s, 151 MB/s 2024-04-25T09:59:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664795 s, 158 MB/s 2024-04-25T09:59:19.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:19.955 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:19.955 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:19.955 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:59:19.955 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:19.955 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:19.955 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:19.955 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:20.019 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.019 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.019 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660728 s, 159 MB/s 2024-04-25T09:59:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631875 s, 166 MB/s 2024-04-25T09:59:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679221 s, 154 MB/s 2024-04-25T09:59:20.042 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.042 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.042 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700827 s, 150 MB/s 2024-04-25T09:59:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701317 s, 150 MB/s 2024-04-25T09:59:20.058 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.058 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.058 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739903 s, 142 MB/s 2024-04-25T09:59:20.065 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636387 s, 165 MB/s 2024-04-25T09:59:20.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.074 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649264 s, 162 MB/s 2024-04-25T09:59:20.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.082 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769117 s, 136 MB/s 2024-04-25T09:59:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.090 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705359 s, 149 MB/s 2024-04-25T09:59:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661462 s, 159 MB/s 2024-04-25T09:59:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832887 s, 126 MB/s 2024-04-25T09:59:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645092 s, 163 MB/s 2024-04-25T09:59:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679816 s, 154 MB/s 2024-04-25T09:59:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640542 s, 164 MB/s 2024-04-25T09:59:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713635 s, 147 MB/s 2024-04-25T09:59:20.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.145 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720357 s, 146 MB/s 2024-04-25T09:59:20.152 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.152 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.152 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630833 s, 166 MB/s 2024-04-25T09:59:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720333 s, 146 MB/s 2024-04-25T09:59:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646519 s, 162 MB/s 2024-04-25T09:59:20.175 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.175 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.175 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633199 s, 166 MB/s 2024-04-25T09:59:20.183 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.183 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.183 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702597 s, 149 MB/s 2024-04-25T09:59:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718062 s, 146 MB/s 2024-04-25T09:59:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736172 s, 142 MB/s 2024-04-25T09:59:20.206 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.207 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.207 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639233 s, 164 MB/s 2024-04-25T09:59:20.215 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.215 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.215 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718419 s, 146 MB/s 2024-04-25T09:59:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069485 s, 151 MB/s 2024-04-25T09:59:20.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.230 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663121 s, 158 MB/s 2024-04-25T09:59:20.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.238 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640207 s, 164 MB/s 2024-04-25T09:59:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724523 s, 145 MB/s 2024-04-25T09:59:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709368 s, 148 MB/s 2024-04-25T09:59:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733741 s, 143 MB/s 2024-04-25T09:59:20.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:20.264 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:20.264 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:20.264 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:59:20.264 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:20.264 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:20.264 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:20.264 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:20.328 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.329 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.329 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712007 s, 147 MB/s 2024-04-25T09:59:20.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621161 s, 169 MB/s 2024-04-25T09:59:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692223 s, 151 MB/s 2024-04-25T09:59:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742634 s, 141 MB/s 2024-04-25T09:59:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.359 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609331 s, 172 MB/s 2024-04-25T09:59:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742997 s, 141 MB/s 2024-04-25T09:59:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706332 s, 148 MB/s 2024-04-25T09:59:20.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.383 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691648 s, 152 MB/s 2024-04-25T09:59:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701052 s, 150 MB/s 2024-04-25T09:59:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065468 s, 160 MB/s 2024-04-25T09:59:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705522 s, 149 MB/s 2024-04-25T09:59:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622576 s, 168 MB/s 2024-04-25T09:59:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648048 s, 162 MB/s 2024-04-25T09:59:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597337 s, 176 MB/s 2024-04-25T09:59:20.435 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.435 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.435 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629829 s, 166 MB/s 2024-04-25T09:59:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689683 s, 152 MB/s 2024-04-25T09:59:20.450 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.450 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.450 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636462 s, 165 MB/s 2024-04-25T09:59:20.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.458 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723927 s, 145 MB/s 2024-04-25T09:59:20.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.466 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649849 s, 161 MB/s 2024-04-25T09:59:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.474 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720325 s, 146 MB/s 2024-04-25T09:59:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.482 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.482 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695825 s, 151 MB/s 2024-04-25T09:59:20.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.490 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728693 s, 144 MB/s 2024-04-25T09:59:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.497 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615456 s, 170 MB/s 2024-04-25T09:59:20.504 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.504 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.504 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601429 s, 174 MB/s 2024-04-25T09:59:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690592 s, 152 MB/s 2024-04-25T09:59:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.521 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828707 s, 127 MB/s 2024-04-25T09:59:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.528 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620275 s, 169 MB/s 2024-04-25T09:59:20.536 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.536 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.536 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708839 s, 148 MB/s 2024-04-25T09:59:20.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.543 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658711 s, 159 MB/s 2024-04-25T09:59:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635925 s, 165 MB/s 2024-04-25T09:59:20.558 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.559 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704335 s, 149 MB/s 2024-04-25T09:59:20.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.567 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637756 s, 164 MB/s 2024-04-25T09:59:20.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:20.567 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:20.567 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:20.568 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:59:20.568 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:20.568 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:20.568 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:20.568 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642414 s, 163 MB/s 2024-04-25T09:59:20.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.641 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719208 s, 146 MB/s 2024-04-25T09:59:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671231 s, 156 MB/s 2024-04-25T09:59:20.656 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.656 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.656 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701596 s, 149 MB/s 2024-04-25T09:59:20.663 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062468 s, 168 MB/s 2024-04-25T09:59:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645175 s, 163 MB/s 2024-04-25T09:59:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620132 s, 169 MB/s 2024-04-25T09:59:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082306 s, 127 MB/s 2024-04-25T09:59:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.695 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719179 s, 146 MB/s 2024-04-25T09:59:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707027 s, 148 MB/s 2024-04-25T09:59:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715871 s, 146 MB/s 2024-04-25T09:59:20.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.720 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702247 s, 149 MB/s 2024-04-25T09:59:20.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.728 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713076 s, 147 MB/s 2024-04-25T09:59:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637227 s, 165 MB/s 2024-04-25T09:59:20.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.744 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768964 s, 136 MB/s 2024-04-25T09:59:20.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.752 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716669 s, 146 MB/s 2024-04-25T09:59:20.759 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.759 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.759 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629039 s, 167 MB/s 2024-04-25T09:59:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.766 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596649 s, 176 MB/s 2024-04-25T09:59:20.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.774 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697794 s, 150 MB/s 2024-04-25T09:59:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.782 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751307 s, 140 MB/s 2024-04-25T09:59:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702469 s, 149 MB/s 2024-04-25T09:59:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647743 s, 162 MB/s 2024-04-25T09:59:20.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.805 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638547 s, 164 MB/s 2024-04-25T09:59:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683709 s, 153 MB/s 2024-04-25T09:59:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620158 s, 169 MB/s 2024-04-25T09:59:20.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.827 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651176 s, 161 MB/s 2024-04-25T09:59:20.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.834 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699937 s, 150 MB/s 2024-04-25T09:59:20.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.842 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691867 s, 152 MB/s 2024-04-25T09:59:20.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.850 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632227 s, 166 MB/s 2024-04-25T09:59:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.857 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646916 s, 162 MB/s 2024-04-25T09:59:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632185 s, 166 MB/s 2024-04-25T09:59:20.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.872 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631754 s, 166 MB/s 2024-04-25T09:59:20.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:20.873 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:20.873 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:20.873 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:59:20.873 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:20.873 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:20.873 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:20.873 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612748 s, 171 MB/s 2024-04-25T09:59:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063257 s, 166 MB/s 2024-04-25T09:59:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651227 s, 161 MB/s 2024-04-25T09:59:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062391 s, 168 MB/s 2024-04-25T09:59:20.967 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.967 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.967 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700353 s, 150 MB/s 2024-04-25T09:59:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667591 s, 157 MB/s 2024-04-25T09:59:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071627 s, 146 MB/s 2024-04-25T09:59:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696797 s, 150 MB/s 2024-04-25T09:59:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663423 s, 158 MB/s 2024-04-25T09:59:21.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.006 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068383 s, 153 MB/s 2024-04-25T09:59:21.013 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.013 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.013 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644644 s, 163 MB/s 2024-04-25T09:59:21.021 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.022 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738499 s, 142 MB/s 2024-04-25T09:59:21.029 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.029 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.029 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698055 s, 150 MB/s 2024-04-25T09:59:21.037 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.037 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.037 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681417 s, 154 MB/s 2024-04-25T09:59:21.044 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.044 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.045 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063981 s, 164 MB/s 2024-04-25T09:59:21.052 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.052 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.052 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066857 s, 157 MB/s 2024-04-25T09:59:21.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.060 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740665 s, 142 MB/s 2024-04-25T09:59:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.068 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.068 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645053 s, 163 MB/s 2024-04-25T09:59:21.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.075 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668742 s, 157 MB/s 2024-04-25T09:59:21.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.083 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.083 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658548 s, 159 MB/s 2024-04-25T09:59:21.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.090 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650564 s, 161 MB/s 2024-04-25T09:59:21.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.098 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658414 s, 159 MB/s 2024-04-25T09:59:21.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.105 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671771 s, 156 MB/s 2024-04-25T09:59:21.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.113 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682495 s, 154 MB/s 2024-04-25T09:59:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.121 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660846 s, 159 MB/s 2024-04-25T09:59:21.128 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.129 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724473 s, 145 MB/s 2024-04-25T09:59:21.136 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.137 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700549 s, 150 MB/s 2024-04-25T09:59:21.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.146 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836362 s, 125 MB/s 2024-04-25T09:59:21.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.153 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620714 s, 169 MB/s 2024-04-25T09:59:21.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692623 s, 151 MB/s 2024-04-25T09:59:21.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689623 s, 152 MB/s 2024-04-25T09:59:21.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.176 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636593 s, 165 MB/s 2024-04-25T09:59:21.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:21.177 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:21.177 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:21.177 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:59:21.177 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:21.177 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:21.177 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:21.178 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695761 s, 151 MB/s 2024-04-25T09:59:21.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.250 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693859 s, 151 MB/s 2024-04-25T09:59:21.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.258 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069148 s, 152 MB/s 2024-04-25T09:59:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704745 s, 149 MB/s 2024-04-25T09:59:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.273 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604787 s, 173 MB/s 2024-04-25T09:59:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.280 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627161 s, 167 MB/s 2024-04-25T09:59:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066978 s, 157 MB/s 2024-04-25T09:59:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655241 s, 160 MB/s 2024-04-25T09:59:21.303 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.303 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.303 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692105 s, 152 MB/s 2024-04-25T09:59:21.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.310 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062673 s, 167 MB/s 2024-04-25T09:59:21.318 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.319 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759206 s, 138 MB/s 2024-04-25T09:59:21.326 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.326 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.326 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636541 s, 165 MB/s 2024-04-25T09:59:21.333 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.333 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.333 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601095 s, 174 MB/s 2024-04-25T09:59:21.340 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.340 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.340 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630969 s, 166 MB/s 2024-04-25T09:59:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.348 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745594 s, 141 MB/s 2024-04-25T09:59:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690782 s, 152 MB/s 2024-04-25T09:59:21.364 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.364 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.364 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715345 s, 147 MB/s 2024-04-25T09:59:21.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.372 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698807 s, 150 MB/s 2024-04-25T09:59:21.380 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.380 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.380 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067703 s, 155 MB/s 2024-04-25T09:59:21.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.387 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634512 s, 165 MB/s 2024-04-25T09:59:21.395 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.395 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.395 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071157 s, 147 MB/s 2024-04-25T09:59:21.403 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.403 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.404 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073325 s, 143 MB/s 2024-04-25T09:59:21.411 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.411 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.411 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704867 s, 149 MB/s 2024-04-25T09:59:21.419 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.419 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.419 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666619 s, 157 MB/s 2024-04-25T09:59:21.427 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.427 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.427 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684974 s, 153 MB/s 2024-04-25T09:59:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069258 s, 151 MB/s 2024-04-25T09:59:21.443 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.443 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.443 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716356 s, 146 MB/s 2024-04-25T09:59:21.450 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.450 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.450 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623349 s, 168 MB/s 2024-04-25T09:59:21.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.458 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705088 s, 149 MB/s 2024-04-25T09:59:21.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.466 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718089 s, 146 MB/s 2024-04-25T09:59:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641752 s, 163 MB/s 2024-04-25T09:59:21.481 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.481 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.481 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695267 s, 151 MB/s 2024-04-25T09:59:21.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:21.482 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:21.482 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:21.483 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:59:21.483 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:21.483 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:21.483 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:21.483 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:21.546 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.547 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.547 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598353 s, 175 MB/s 2024-04-25T09:59:21.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.554 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607164 s, 173 MB/s 2024-04-25T09:59:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736712 s, 142 MB/s 2024-04-25T09:59:21.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.570 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693794 s, 151 MB/s 2024-04-25T09:59:21.577 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.578 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.578 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694575 s, 151 MB/s 2024-04-25T09:59:21.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703588 s, 149 MB/s 2024-04-25T09:59:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068235 s, 154 MB/s 2024-04-25T09:59:21.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.601 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.601 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641243 s, 164 MB/s 2024-04-25T09:59:21.608 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.609 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.609 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708825 s, 148 MB/s 2024-04-25T09:59:21.616 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.616 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.616 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647121 s, 162 MB/s 2024-04-25T09:59:21.623 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.623 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.623 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627027 s, 167 MB/s 2024-04-25T09:59:21.631 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.631 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.631 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729735 s, 144 MB/s 2024-04-25T09:59:21.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.638 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614958 s, 171 MB/s 2024-04-25T09:59:21.646 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.646 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.646 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699358 s, 150 MB/s 2024-04-25T09:59:21.654 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.654 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.655 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754114 s, 139 MB/s 2024-04-25T09:59:21.661 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.662 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.662 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629473 s, 167 MB/s 2024-04-25T09:59:21.669 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.669 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.669 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659803 s, 159 MB/s 2024-04-25T09:59:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007064 s, 148 MB/s 2024-04-25T09:59:21.685 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772707 s, 136 MB/s 2024-04-25T09:59:21.693 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.693 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.693 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665701 s, 158 MB/s 2024-04-25T09:59:21.700 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.700 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625386 s, 168 MB/s 2024-04-25T09:59:21.708 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.708 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.708 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671313 s, 156 MB/s 2024-04-25T09:59:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647222 s, 162 MB/s 2024-04-25T09:59:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061188 s, 171 MB/s 2024-04-25T09:59:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664264 s, 158 MB/s 2024-04-25T09:59:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063111 s, 166 MB/s 2024-04-25T09:59:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725831 s, 144 MB/s 2024-04-25T09:59:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709862 s, 148 MB/s 2024-04-25T09:59:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757445 s, 138 MB/s 2024-04-25T09:59:21.770 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.771 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.771 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722439 s, 145 MB/s 2024-04-25T09:59:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691313 s, 152 MB/s 2024-04-25T09:59:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694546 s, 151 MB/s 2024-04-25T09:59:21.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:21.787 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:21.788 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:21.788 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:59:21.788 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:21.788 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:21.788 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:21.788 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:21.853 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.853 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.853 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070115 s, 150 MB/s 2024-04-25T09:59:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759496 s, 138 MB/s 2024-04-25T09:59:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724529 s, 145 MB/s 2024-04-25T09:59:21.877 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.878 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.878 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072413 s, 145 MB/s 2024-04-25T09:59:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.885 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693306 s, 151 MB/s 2024-04-25T09:59:21.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.893 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693752 s, 151 MB/s 2024-04-25T09:59:21.901 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.901 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.901 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714983 s, 147 MB/s 2024-04-25T09:59:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638847 s, 164 MB/s 2024-04-25T09:59:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070427 s, 149 MB/s 2024-04-25T09:59:21.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060812 s, 172 MB/s 2024-04-25T09:59:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070658 s, 148 MB/s 2024-04-25T09:59:21.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.939 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703098 s, 149 MB/s 2024-04-25T09:59:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.947 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.947 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642638 s, 163 MB/s 2024-04-25T09:59:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.954 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687558 s, 153 MB/s 2024-04-25T09:59:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065362 s, 160 MB/s 2024-04-25T09:59:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064023 s, 164 MB/s 2024-04-25T09:59:21.976 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639187 s, 164 MB/s 2024-04-25T09:59:21.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.985 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686773 s, 153 MB/s 2024-04-25T09:59:21.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:21.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:21.992 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686736 s, 153 MB/s 2024-04-25T09:59:22.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.000 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661878 s, 158 MB/s 2024-04-25T09:59:22.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.008 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740902 s, 142 MB/s 2024-04-25T09:59:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720808 s, 145 MB/s 2024-04-25T09:59:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685642 s, 153 MB/s 2024-04-25T09:59:22.031 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.031 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.031 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624048 s, 168 MB/s 2024-04-25T09:59:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691747 s, 152 MB/s 2024-04-25T09:59:22.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.046 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006467 s, 162 MB/s 2024-04-25T09:59:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739532 s, 142 MB/s 2024-04-25T09:59:22.063 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.063 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.063 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742654 s, 141 MB/s 2024-04-25T09:59:22.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.070 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669744 s, 157 MB/s 2024-04-25T09:59:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843823 s, 124 MB/s 2024-04-25T09:59:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722723 s, 145 MB/s 2024-04-25T09:59:22.095 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.095 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.095 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610108 s, 172 MB/s 2024-04-25T09:59:22.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:22.096 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:22.096 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:22.096 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:59:22.096 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:22.096 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:22.096 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:22.096 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:22.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703502 s, 149 MB/s 2024-04-25T09:59:22.168 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.168 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.168 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686114 s, 153 MB/s 2024-04-25T09:59:22.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.176 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625555 s, 168 MB/s 2024-04-25T09:59:22.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.184 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708453 s, 148 MB/s 2024-04-25T09:59:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006618 s, 158 MB/s 2024-04-25T09:59:22.199 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.199 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.199 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656079 s, 160 MB/s 2024-04-25T09:59:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739627 s, 142 MB/s 2024-04-25T09:59:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.214 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639713 s, 164 MB/s 2024-04-25T09:59:22.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061936 s, 169 MB/s 2024-04-25T09:59:22.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.229 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063496 s, 165 MB/s 2024-04-25T09:59:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630049 s, 166 MB/s 2024-04-25T09:59:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062294 s, 168 MB/s 2024-04-25T09:59:22.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.250 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612329 s, 171 MB/s 2024-04-25T09:59:22.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.258 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703681 s, 149 MB/s 2024-04-25T09:59:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069071 s, 152 MB/s 2024-04-25T09:59:22.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.274 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.274 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652744 s, 161 MB/s 2024-04-25T09:59:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678309 s, 155 MB/s 2024-04-25T09:59:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.289 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.289 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639395 s, 164 MB/s 2024-04-25T09:59:22.296 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.296 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.296 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657813 s, 159 MB/s 2024-04-25T09:59:22.303 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.304 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.304 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650604 s, 161 MB/s 2024-04-25T09:59:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069825 s, 150 MB/s 2024-04-25T09:59:22.318 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.318 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.318 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612607 s, 171 MB/s 2024-04-25T09:59:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641245 s, 164 MB/s 2024-04-25T09:59:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748196 s, 140 MB/s 2024-04-25T09:59:22.341 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.341 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.342 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634865 s, 165 MB/s 2024-04-25T09:59:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072042 s, 146 MB/s 2024-04-25T09:59:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632268 s, 166 MB/s 2024-04-25T09:59:22.365 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.365 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.365 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713246 s, 147 MB/s 2024-04-25T09:59:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669395 s, 157 MB/s 2024-04-25T09:59:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072264 s, 145 MB/s 2024-04-25T09:59:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.389 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716366 s, 146 MB/s 2024-04-25T09:59:22.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.396 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652758 s, 161 MB/s 2024-04-25T09:59:22.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:22.397 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:22.398 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:22.398 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:59:22.398 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:22.398 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:22.398 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:22.398 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641935 s, 163 MB/s 2024-04-25T09:59:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630566 s, 166 MB/s 2024-04-25T09:59:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060096 s, 174 MB/s 2024-04-25T09:59:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741854 s, 141 MB/s 2024-04-25T09:59:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.492 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637921 s, 164 MB/s 2024-04-25T09:59:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065071 s, 161 MB/s 2024-04-25T09:59:22.507 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.508 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.508 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715965 s, 146 MB/s 2024-04-25T09:59:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605339 s, 173 MB/s 2024-04-25T09:59:22.522 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.522 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.523 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641854 s, 163 MB/s 2024-04-25T09:59:22.529 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.529 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.529 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602353 s, 174 MB/s 2024-04-25T09:59:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749498 s, 140 MB/s 2024-04-25T09:59:22.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.546 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723242 s, 145 MB/s 2024-04-25T09:59:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682558 s, 154 MB/s 2024-04-25T09:59:22.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.561 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689954 s, 152 MB/s 2024-04-25T09:59:22.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.568 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655109 s, 160 MB/s 2024-04-25T09:59:22.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.576 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.576 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634428 s, 165 MB/s 2024-04-25T09:59:22.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.583 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618952 s, 169 MB/s 2024-04-25T09:59:22.591 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.591 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.591 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721932 s, 145 MB/s 2024-04-25T09:59:22.598 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.598 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.598 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631741 s, 166 MB/s 2024-04-25T09:59:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687282 s, 153 MB/s 2024-04-25T09:59:22.613 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.613 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.613 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619354 s, 169 MB/s 2024-04-25T09:59:22.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.621 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711148 s, 147 MB/s 2024-04-25T09:59:22.628 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.628 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.628 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616821 s, 170 MB/s 2024-04-25T09:59:22.636 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.636 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.636 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672013 s, 156 MB/s 2024-04-25T09:59:22.644 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.644 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.644 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703605 s, 149 MB/s 2024-04-25T09:59:22.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.651 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663935 s, 158 MB/s 2024-04-25T09:59:22.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.659 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633724 s, 165 MB/s 2024-04-25T09:59:22.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.667 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.667 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675539 s, 155 MB/s 2024-04-25T09:59:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774269 s, 135 MB/s 2024-04-25T09:59:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823154 s, 127 MB/s 2024-04-25T09:59:22.691 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.691 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.692 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629408 s, 167 MB/s 2024-04-25T09:59:22.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.700 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669526 s, 157 MB/s 2024-04-25T09:59:22.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:22.702 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:22.702 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:22.702 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:59:22.702 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:22.702 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:22.702 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:22.703 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696681 s, 151 MB/s 2024-04-25T09:59:22.775 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.775 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.775 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074241 s, 141 MB/s 2024-04-25T09:59:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.783 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715964 s, 146 MB/s 2024-04-25T09:59:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671215 s, 156 MB/s 2024-04-25T09:59:22.799 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.799 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.799 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073682 s, 142 MB/s 2024-04-25T09:59:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681529 s, 154 MB/s 2024-04-25T09:59:22.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.815 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699099 s, 150 MB/s 2024-04-25T09:59:22.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.823 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719813 s, 146 MB/s 2024-04-25T09:59:22.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.831 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689477 s, 152 MB/s 2024-04-25T09:59:22.839 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.839 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.839 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767361 s, 137 MB/s 2024-04-25T09:59:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685089 s, 153 MB/s 2024-04-25T09:59:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.855 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699423 s, 150 MB/s 2024-04-25T09:59:22.862 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.863 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637415 s, 165 MB/s 2024-04-25T09:59:22.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.870 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.870 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614676 s, 171 MB/s 2024-04-25T09:59:22.877 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.877 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.877 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630781 s, 166 MB/s 2024-04-25T09:59:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715869 s, 146 MB/s 2024-04-25T09:59:22.892 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.892 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.892 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629992 s, 166 MB/s 2024-04-25T09:59:22.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.899 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620155 s, 169 MB/s 2024-04-25T09:59:22.907 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.907 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.907 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695079 s, 151 MB/s 2024-04-25T09:59:22.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.915 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689744 s, 152 MB/s 2024-04-25T09:59:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797943 s, 131 MB/s 2024-04-25T09:59:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691293 s, 152 MB/s 2024-04-25T09:59:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689516 s, 152 MB/s 2024-04-25T09:59:22.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.948 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713554 s, 147 MB/s 2024-04-25T09:59:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644251 s, 163 MB/s 2024-04-25T09:59:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638833 s, 164 MB/s 2024-04-25T09:59:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.970 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691798 s, 152 MB/s 2024-04-25T09:59:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.977 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634495 s, 165 MB/s 2024-04-25T09:59:22.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.984 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622703 s, 168 MB/s 2024-04-25T09:59:22.991 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.991 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.991 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632255 s, 166 MB/s 2024-04-25T09:59:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:22.999 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728383 s, 144 MB/s 2024-04-25T09:59:23.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.007 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685676 s, 153 MB/s 2024-04-25T09:59:23.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:23.008 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:23.008 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:23.009 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:59:23.009 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:23.009 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:23.009 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:23.009 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:23.152 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.152 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.152 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632312 s, 166 MB/s 2024-04-25T09:59:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684241 s, 153 MB/s 2024-04-25T09:59:23.167 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.167 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.168 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664896 s, 158 MB/s 2024-04-25T09:59:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640694 s, 164 MB/s 2024-04-25T09:59:23.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.182 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006543 s, 160 MB/s 2024-04-25T09:59:23.189 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.189 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.190 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659441 s, 159 MB/s 2024-04-25T09:59:23.197 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691627 s, 152 MB/s 2024-04-25T09:59:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061463 s, 171 MB/s 2024-04-25T09:59:23.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.212 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672776 s, 156 MB/s 2024-04-25T09:59:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719039 s, 146 MB/s 2024-04-25T09:59:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714338 s, 147 MB/s 2024-04-25T09:59:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700482 s, 150 MB/s 2024-04-25T09:59:23.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.244 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065721 s, 160 MB/s 2024-04-25T09:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712223 s, 147 MB/s 2024-04-25T09:59:23.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.259 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627187 s, 167 MB/s 2024-04-25T09:59:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.266 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064532 s, 162 MB/s 2024-04-25T09:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.274 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708217 s, 148 MB/s 2024-04-25T09:59:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673618 s, 156 MB/s 2024-04-25T09:59:23.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.291 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837758 s, 125 MB/s 2024-04-25T09:59:23.300 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.300 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.300 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736784 s, 142 MB/s 2024-04-25T09:59:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.307 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661621 s, 158 MB/s 2024-04-25T09:59:23.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007432 s, 141 MB/s 2024-04-25T09:59:23.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.324 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701729 s, 149 MB/s 2024-04-25T09:59:23.332 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.332 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.332 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794615 s, 132 MB/s 2024-04-25T09:59:23.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617639 s, 170 MB/s 2024-04-25T09:59:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626442 s, 167 MB/s 2024-04-25T09:59:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720923 s, 145 MB/s 2024-04-25T09:59:23.362 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.362 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.362 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648046 s, 162 MB/s 2024-04-25T09:59:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.370 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647002 s, 162 MB/s 2024-04-25T09:59:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738812 s, 142 MB/s 2024-04-25T09:59:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610145 s, 172 MB/s 2024-04-25T09:59:23.393 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.393 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.393 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676392 s, 155 MB/s 2024-04-25T09:59:23.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:23.394 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:23.394 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:23.394 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:59:23.394 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:23.394 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:23.394 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:23.394 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:23.419 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.419 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.419 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646434 s, 162 MB/s 2024-04-25T09:59:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740796 s, 142 MB/s 2024-04-25T09:59:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623075 s, 168 MB/s 2024-04-25T09:59:23.442 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707499 s, 148 MB/s 2024-04-25T09:59:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701378 s, 150 MB/s 2024-04-25T09:59:23.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.458 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716969 s, 146 MB/s 2024-04-25T09:59:23.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.466 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653041 s, 161 MB/s 2024-04-25T09:59:23.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.474 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074681 s, 140 MB/s 2024-04-25T09:59:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730545 s, 144 MB/s 2024-04-25T09:59:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.490 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688643 s, 152 MB/s 2024-04-25T09:59:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719324 s, 146 MB/s 2024-04-25T09:59:23.507 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.507 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.507 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749622 s, 140 MB/s 2024-04-25T09:59:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712763 s, 147 MB/s 2024-04-25T09:59:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.523 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671394 s, 156 MB/s 2024-04-25T09:59:23.530 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.530 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.530 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655212 s, 160 MB/s 2024-04-25T09:59:23.538 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.538 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.539 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746409 s, 140 MB/s 2024-04-25T09:59:23.547 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.547 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.547 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756163 s, 139 MB/s 2024-04-25T09:59:23.554 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.554 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.554 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659789 s, 159 MB/s 2024-04-25T09:59:23.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.562 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658811 s, 159 MB/s 2024-04-25T09:59:23.569 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.569 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.570 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656996 s, 160 MB/s 2024-04-25T09:59:23.577 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.577 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.577 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649054 s, 162 MB/s 2024-04-25T09:59:23.584 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.585 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696473 s, 151 MB/s 2024-04-25T09:59:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632361 s, 166 MB/s 2024-04-25T09:59:23.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.600 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736201 s, 142 MB/s 2024-04-25T09:59:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652145 s, 161 MB/s 2024-04-25T09:59:23.616 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.616 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.616 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756742 s, 139 MB/s 2024-04-25T09:59:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676964 s, 155 MB/s 2024-04-25T09:59:23.632 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.632 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.632 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069834 s, 150 MB/s 2024-04-25T09:59:23.640 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.640 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.640 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738477 s, 142 MB/s 2024-04-25T09:59:23.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.648 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068589 s, 153 MB/s 2024-04-25T09:59:23.655 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.656 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.656 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679812 s, 154 MB/s 2024-04-25T09:59:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745898 s, 141 MB/s 2024-04-25T09:59:23.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:23.665 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:23.665 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:23.665 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:59:23.665 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:23.666 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:23.666 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:23.666 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694369 s, 151 MB/s 2024-04-25T09:59:23.739 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.739 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.739 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726546 s, 144 MB/s 2024-04-25T09:59:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660215 s, 159 MB/s 2024-04-25T09:59:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714674 s, 147 MB/s 2024-04-25T09:59:23.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.762 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633275 s, 166 MB/s 2024-04-25T09:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722366 s, 145 MB/s 2024-04-25T09:59:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657418 s, 159 MB/s 2024-04-25T09:59:23.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.786 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691249 s, 152 MB/s 2024-04-25T09:59:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642339 s, 163 MB/s 2024-04-25T09:59:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.801 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631945 s, 166 MB/s 2024-04-25T09:59:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.808 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684582 s, 153 MB/s 2024-04-25T09:59:23.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.816 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652952 s, 161 MB/s 2024-04-25T09:59:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765248 s, 137 MB/s 2024-04-25T09:59:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066789 s, 157 MB/s 2024-04-25T09:59:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622321 s, 168 MB/s 2024-04-25T09:59:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680778 s, 154 MB/s 2024-04-25T09:59:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.854 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694677 s, 151 MB/s 2024-04-25T09:59:23.862 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.862 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.862 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632823 s, 166 MB/s 2024-04-25T09:59:23.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.870 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.870 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695023 s, 151 MB/s 2024-04-25T09:59:23.878 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.878 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.878 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710258 s, 148 MB/s 2024-04-25T09:59:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716807 s, 146 MB/s 2024-04-25T09:59:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627488 s, 167 MB/s 2024-04-25T09:59:23.900 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.900 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.900 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596989 s, 176 MB/s 2024-04-25T09:59:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.908 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076576 s, 137 MB/s 2024-04-25T09:59:23.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.918 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843741 s, 124 MB/s 2024-04-25T09:59:23.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.927 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807521 s, 130 MB/s 2024-04-25T09:59:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808152 s, 130 MB/s 2024-04-25T09:59:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717832 s, 146 MB/s 2024-04-25T09:59:23.953 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.953 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.953 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833501 s, 126 MB/s 2024-04-25T09:59:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676246 s, 155 MB/s 2024-04-25T09:59:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738344 s, 142 MB/s 2024-04-25T09:59:23.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:23.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:23.978 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006629 s, 158 MB/s 2024-04-25T09:59:23.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:23.979 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:59:24.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.119+0000 7f4b80807640 1 Processor -- start 2024-04-25T09:59:24.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.119+0000 7f4b80807640 1 -- start start 2024-04-25T09:59:24.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.119+0000 7f4b80807640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b7c161710 0x7f4b7c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b80807640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7c15e000 con 0x7f4b7c0ced80 2024-04-25T09:59:24.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b80807640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7c0d1de0 con 0x7f4b7c15dc20 2024-04-25T09:59:24.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b80807640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c161c60 con 0x7f4b7c161710 2024-04-25T09:59:24.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b73fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4b7c0ced80 0x7f4b7c0d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33338/0 (socket says 172.21.15.19:33338) 2024-04-25T09:59:24.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b73fff640 1 -- 172.21.15.19:0/1020568445 learned_addr learned my addr 172.21.15.19:0/1020568445 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b7ad9d640 1 --2- 172.21.15.19:0/1020568445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b7c161710 0x7f4b7c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b79d9b640 1 -- 172.21.15.19:0/1020568445 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558462831 0 0) 0x7f4b7c15e000 con 0x7f4b7c0ced80 2024-04-25T09:59:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b79d9b640 1 -- 172.21.15.19:0/1020568445 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b68003520 con 0x7f4b7c0ced80 2024-04-25T09:59:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b79d9b640 1 -- 172.21.15.19:0/1020568445 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901529992 0 0) 0x7f4b7c0d1de0 con 0x7f4b7c15dc20 2024-04-25T09:59:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b79d9b640 1 -- 172.21.15.19:0/1020568445 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b7c15e000 con 0x7f4b7c15dc20 2024-04-25T09:59:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b7ad9d640 1 -- 172.21.15.19:0/1020568445 >> v1:172.21.15.112:6789/0 conn(0x7f4b7c15dc20 legacy=0x7f4b7c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b7ad9d640 1 -- 172.21.15.19:0/1020568445 >> v1:172.21.15.19:6789/0 conn(0x7f4b7c0ced80 legacy=0x7f4b7c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.120+0000 7f4b7ad9d640 1 -- 172.21.15.19:0/1020568445 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c161f40 con 0x7f4b7c161710 2024-04-25T09:59:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.121+0000 7f4b7ad9d640 1 --2- 172.21.15.19:0/1020568445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b7c161710 0x7f4b7c0d29f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009dc0 tx=0x7f4b7402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4616ac005c93067d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.121+0000 7f4b79d9b640 1 -- 172.21.15.19:0/1020568445 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74007740 con 0x7f4b7c161710 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.121+0000 7f4b79d9b640 1 -- 172.21.15.19:0/1020568445 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b740078e0 con 0x7f4b7c161710 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.121+0000 7f4b79d9b640 1 -- 172.21.15.19:0/1020568445 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74007c90 con 0x7f4b7c161710 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.122+0000 7f4b80807640 1 -- 172.21.15.19:0/1020568445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b7c161710 msgr2=0x7f4b7c0d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.122+0000 7f4b80807640 1 --2- 172.21.15.19:0/1020568445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b7c161710 0x7f4b7c0d29f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009dc0 tx=0x7f4b7402a6d0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.122+0000 7f4b80807640 1 -- 172.21.15.19:0/1020568445 shutdown_connections 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.122+0000 7f4b80807640 1 --2- 172.21.15.19:0/1020568445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b7c161710 0x7f4b7c0d29f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.122+0000 7f4b80807640 1 -- 172.21.15.19:0/1020568445 >> 172.21.15.19:0/1020568445 conn(0x7f4b7c0d0420 msgr2=0x7f4b7c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.122+0000 7f4b80807640 1 -- 172.21.15.19:0/1020568445 shutdown_connections 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.122+0000 7f4b80807640 1 -- 172.21.15.19:0/1020568445 wait complete. 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.123+0000 7f4b80807640 1 Processor -- start 2024-04-25T09:59:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.123+0000 7f4b80807640 1 -- start start 2024-04-25T09:59:24.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b80807640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b7c0ced80 0x7f4b7c0b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b80807640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b7c15dc20 0x7f4b7c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b80807640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b7c161710 0x7f4b7c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b80807640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c0b5e50 con 0x7f4b7c0ced80 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b80807640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c162500 con 0x7f4b7c15dc20 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b80807640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c0b5fd0 con 0x7f4b7c161710 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b7a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b7c0ced80 0x7f4b7c0b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b7a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b7c0ced80 0x7f4b7c0b82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38148/0 (socket says 172.21.15.19:38148) 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2939576933 learned_addr learned my addr 172.21.15.19:0/2939576933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b7ad9d640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b7c161710 0x7f4b7c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b73fff640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b7c15dc20 0x7f4b7c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b7c161710 msgr2=0x7f4b7c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b7c161710 0x7f4b7c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b7c15dc20 msgr2=0x7f4b7c15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b7c15dc20 0x7f4b7c15e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2939576933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c15ec90 con 0x7f4b7c0ced80 2024-04-25T09:59:24.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.124+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b7c0ced80 0x7f4b7c0b82d0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00c460 tx=0x7f4b6c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.125+0000 7f4b72ffd640 1 -- 172.21.15.19:0/2939576933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c017070 con 0x7f4b7c0ced80 2024-04-25T09:59:24.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.125+0000 7f4b72ffd640 1 -- 172.21.15.19:0/2939576933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c0052f0 con 0x7f4b7c0ced80 2024-04-25T09:59:24.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.125+0000 7f4b72ffd640 1 -- 172.21.15.19:0/2939576933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c003830 con 0x7f4b7c0ced80 2024-04-25T09:59:24.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.125+0000 7f4b80807640 1 -- 172.21.15.19:0/2939576933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c15ef40 con 0x7f4b7c0ced80 2024-04-25T09:59:24.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.125+0000 7f4b80807640 1 -- 172.21.15.19:0/2939576933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7c0c7fa0 con 0x7f4b7c0ced80 2024-04-25T09:59:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.126+0000 7f4b72ffd640 1 -- 172.21.15.19:0/2939576933 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b6c01a070 con 0x7f4b7c0ced80 2024-04-25T09:59:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.127+0000 7f4b72ffd640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.127+0000 7f4b72ffd640 1 -- 172.21.15.19:0/2939576933 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b6c089640 con 0x7f4b7c0ced80 2024-04-25T09:59:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.127+0000 7f4b73fff640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.127+0000 7f4b73fff640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b640011e0 tx=0x7f4b64007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.127+0000 7f4b835e7740 1 Processor -- start 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.128+0000 7f4b835e7740 1 -- start start 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.128+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24abe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.128+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24af88d0 0x560a24abc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.128+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24ae6cc0 0x560a24ae7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.128+0000 7f4b835e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a24a20000 con 0x560a24abc200 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.128+0000 7f4b835e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a249c8f20 con 0x560a24ae6cc0 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.128+0000 7f4b835e7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a24a1c0a0 con 0x560a24af88d0 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.128+0000 7f4b7a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24abe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.128+0000 7f4b7a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24abe360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38154/0 (socket says 172.21.15.19:38154) 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.128+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2729029865 learned_addr learned my addr 172.21.15.19:0/2729029865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b73fff640 1 --2- 172.21.15.19:0/2729029865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24af88d0 0x560a24abc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b7ad9d640 1 --2- 172.21.15.19:0/2729029865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24ae6cc0 0x560a24ae7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2729029865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24af88d0 msgr2=0x560a24abc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/2729029865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24af88d0 0x560a24abc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2729029865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24ae6cc0 msgr2=0x560a24ae7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/2729029865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24ae6cc0 0x560a24ae7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2729029865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a24800ba0 con 0x560a24abc200 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b7ad9d640 1 --2- 172.21.15.19:0/2729029865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24ae6cc0 0x560a24ae7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/2729029865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24abe360 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c05fea0 tx=0x7f4b6c05be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5763fa9b6c5d097 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b5bfff640 1 -- 172.21.15.19:0/2729029865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c05e0c0 con 0x560a24abc200 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b5bfff640 1 -- 172.21.15.19:0/2729029865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c055e10 con 0x560a24abc200 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.129+0000 7f4b5bfff640 1 -- 172.21.15.19:0/2729029865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0543e0 con 0x560a24abc200 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.130+0000 7f4b835e7740 1 -- 172.21.15.19:0/2729029865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 msgr2=0x560a24abe360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.130+0000 7f4b835e7740 1 --2- 172.21.15.19:0/2729029865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24abe360 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c05fea0 tx=0x7f4b6c05be10 comp rx=0 tx=0).stop 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.130+0000 7f4b835e7740 1 -- 172.21.15.19:0/2729029865 shutdown_connections 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.130+0000 7f4b835e7740 1 --2- 172.21.15.19:0/2729029865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24ae6cc0 0x560a24ae7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.130+0000 7f4b835e7740 1 --2- 172.21.15.19:0/2729029865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24af88d0 0x560a24abc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.130+0000 7f4b835e7740 1 --2- 172.21.15.19:0/2729029865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24abe360 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.130+0000 7f4b835e7740 1 -- 172.21.15.19:0/2729029865 >> 172.21.15.19:0/2729029865 conn(0x560a24a84050 msgr2=0x560a24b01cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.130+0000 7f4b835e7740 1 -- 172.21.15.19:0/2729029865 shutdown_connections 2024-04-25T09:59:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.130+0000 7f4b835e7740 1 -- 172.21.15.19:0/2729029865 wait complete. 2024-04-25T09:59:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.130+0000 7f4b835e7740 1 -- start start 2024-04-25T09:59:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.130+0000 7f4b835e7740 1 Processor -- start 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.131+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24abc200 0x560a24ad6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.131+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24ae6cc0 0x560a24ae8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.131+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24af88d0 0x560a24afa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.131+0000 7f4b835e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a249be610 con 0x560a24af88d0 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.131+0000 7f4b835e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a24ae4810 con 0x560a24abc200 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.131+0000 7f4b835e7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a24ae7f20 con 0x560a24ae6cc0 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.131+0000 7f4b835e7740 10 client.0 _ll_register_callbacks cb 0x560a249d6f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.131+0000 7f4b7ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24af88d0 0x560a24afa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b7a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24abc200 0x560a24ad6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b7ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24af88d0 0x560a24afa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38158/0 (socket says 172.21.15.19:38158) 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b7ad9d640 1 -- 172.21.15.19:0/239518490 learned_addr learned my addr 172.21.15.19:0/239518490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b73fff640 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24ae6cc0 0x560a24ae8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b7ad9d640 1 -- 172.21.15.19:0/239518490 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24ae6cc0 msgr2=0x560a24ae8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b7ad9d640 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24ae6cc0 0x560a24ae8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b7ad9d640 1 -- 172.21.15.19:0/239518490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24abc200 msgr2=0x560a24ad6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b7ad9d640 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24abc200 0x560a24ad6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b7ad9d640 1 -- 172.21.15.19:0/239518490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a246f4490 con 0x560a24af88d0 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b7ad9d640 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24af88d0 0x560a24afa910 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009ee0 tx=0x7f4b74004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6305c700148233d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b597fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b597fa640 1 -- 172.21.15.19:0/239518490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74009450 con 0x560a24af88d0 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b597fa640 1 -- 172.21.15.19:0/239518490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b7403c420 con 0x560a24af88d0 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b835e7740 1 -- 172.21.15.19:0/239518490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b6c055e10 con 0x560a24af88d0 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b597fa640 1 -- 172.21.15.19:0/239518490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b74050050 con 0x560a24af88d0 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b835e7740 10 client.7752 fetch_fsmap learned FSMap version 58 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b835e7740 1 -- 172.21.15.19:0/239518490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x560a24861330 con 0x560a24af88d0 2024-04-25T09:59:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b597fa640 1 -- 172.21.15.19:0/239518490 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c055e10 con 0x560a24af88d0 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.132+0000 7f4b597fa640 1 -- 172.21.15.19:0/239518490 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4b7403c620 con 0x560a24af88d0 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b835e7740 10 client.7752 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b835e7740 10 client.7752 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b835e7740 20 client.7752 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b835e7740 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x560a24afae50 0x560a24ae7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b835e7740 4 client.7752 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b835e7740 1 -- 172.21.15.19:0/239518490 --> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a24a2d540 con 0x560a24afae50 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x560a24afae50 0x560a24ae7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b597fa640 10 client.7752 ms_handle_connect on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x560a24afae50 0x560a24ae7140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774130c0d4ceec4 server_cookie=d62003d0474e05 in_seq=0 out_seq=0 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b597fa640 1 -- 172.21.15.19:0/239518490 <== mds.0 v2:172.21.15.19:6832/3589967477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a24a2d540 con 0x560a24afae50 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b597fa640 10 client.7752 handle_command_reply: tid=0 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.133+0000 7f4b835e7740 1 client.7752 shutdown 2024-04-25T09:59:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b835e7740 1 -- 172.21.15.19:0/239518490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24af88d0 msgr2=0x560a24afa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b835e7740 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24af88d0 0x560a24afa910 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009ee0 tx=0x7f4b74004ab0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b835e7740 1 -- 172.21.15.19:0/239518490 shutdown_connections 2024-04-25T09:59:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b835e7740 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x560a24afae50 0x560a24ae7140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b835e7740 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24af88d0 0x560a24afa910 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b835e7740 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24ae6cc0 0x560a24ae8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b597fa640 0 client.7752 ms_handle_reset on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b835e7740 1 --2- 172.21.15.19:0/239518490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24abc200 0x560a24ad6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b835e7740 1 -- 172.21.15.19:0/239518490 >> 172.21.15.19:0/239518490 conn(0x560a24a84050 msgr2=0x560a24ae1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b835e7740 1 -- 172.21.15.19:0/239518490 shutdown_connections 2024-04-25T09:59:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b835e7740 1 -- 172.21.15.19:0/239518490 wait complete. 2024-04-25T09:59:24.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.134+0000 7f4b835e7740 20 client.7752 trim_cache size 0 max 16384 2024-04-25T09:59:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.149+0000 7f4b835e7740 1 Processor -- start 2024-04-25T09:59:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.149+0000 7f4b835e7740 1 -- start start 2024-04-25T09:59:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24b14570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24ae6cc0 0x560a24ae05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24af88d0 0x560a24ae0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b835e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c2003f0 con 0x560a24abc200 2024-04-25T09:59:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b835e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c0c9660 con 0x560a24ae6cc0 2024-04-25T09:59:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b835e7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c1f8040 con 0x560a24af88d0 2024-04-25T09:59:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b7a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24b14570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b7a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24b14570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38174/0 (socket says 172.21.15.19:38174) 2024-04-25T09:59:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2785087005 learned_addr learned my addr 172.21.15.19:0/2785087005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b73fff640 1 --2- 172.21.15.19:0/2785087005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24ae6cc0 0x560a24ae05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b7ad9d640 1 --2- 172.21.15.19:0/2785087005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24af88d0 0x560a24ae0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2785087005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24af88d0 msgr2=0x560a24ae0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/2785087005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24af88d0 0x560a24ae0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2785087005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24ae6cc0 msgr2=0x560a24ae05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/2785087005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24ae6cc0 0x560a24ae05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b7a59c640 1 -- 172.21.15.19:0/2785087005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a246cd610 con 0x560a24abc200 2024-04-25T09:59:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/2785087005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24b14570 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c057d00 tx=0x7f4b6c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1429eeaf4b339b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b5b7fe640 1 -- 172.21.15.19:0/2785087005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c055e10 con 0x560a24abc200 2024-04-25T09:59:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b5b7fe640 1 -- 172.21.15.19:0/2785087005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c0543e0 con 0x560a24abc200 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.150+0000 7f4b5b7fe640 1 -- 172.21.15.19:0/2785087005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0573c0 con 0x560a24abc200 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.151+0000 7f4b835e7740 1 -- 172.21.15.19:0/2785087005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 msgr2=0x560a24b14570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.151+0000 7f4b835e7740 1 --2- 172.21.15.19:0/2785087005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24b14570 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c057d00 tx=0x7f4b6c01d000 comp rx=0 tx=0).stop 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.151+0000 7f4b835e7740 1 -- 172.21.15.19:0/2785087005 shutdown_connections 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.151+0000 7f4b835e7740 1 --2- 172.21.15.19:0/2785087005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24af88d0 0x560a24ae0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.151+0000 7f4b835e7740 1 --2- 172.21.15.19:0/2785087005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24ae6cc0 0x560a24ae05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.151+0000 7f4b835e7740 1 --2- 172.21.15.19:0/2785087005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24abc200 0x560a24b14570 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.151+0000 7f4b835e7740 1 -- 172.21.15.19:0/2785087005 >> 172.21.15.19:0/2785087005 conn(0x560a24a84050 msgr2=0x560a24abd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.151+0000 7f4b835e7740 1 -- 172.21.15.19:0/2785087005 shutdown_connections 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.151+0000 7f4b835e7740 1 -- 172.21.15.19:0/2785087005 wait complete. 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.152+0000 7f4b835e7740 1 -- start start 2024-04-25T09:59:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.152+0000 7f4b835e7740 1 Processor -- start 2024-04-25T09:59:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24abc200 0x560a24b14570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24ae6cc0 0x560a24b1b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b835e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24af88d0 0x560a24b1b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b835e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a24b051a0 con 0x560a24ae6cc0 2024-04-25T09:59:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b835e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c0c9060 con 0x560a24af88d0 2024-04-25T09:59:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b835e7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a24b24d90 con 0x560a24abc200 2024-04-25T09:59:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b835e7740 10 client.0 _ll_register_callbacks cb 0x7f4b7c062560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:24.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b73fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24ae6cc0 0x560a24b1b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b73fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24ae6cc0 0x560a24b1b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38190/0 (socket says 172.21.15.19:38190) 2024-04-25T09:59:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b73fff640 1 -- 172.21.15.19:0/521869619 learned_addr learned my addr 172.21.15.19:0/521869619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24abc200 0x560a24b14570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.153+0000 7f4b7ad9d640 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24af88d0 0x560a24b1b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b73fff640 1 -- 172.21.15.19:0/521869619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24abc200 msgr2=0x560a24b14570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b73fff640 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24abc200 0x560a24b14570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b73fff640 1 -- 172.21.15.19:0/521869619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24af88d0 msgr2=0x560a24b1b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b73fff640 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24af88d0 0x560a24b1b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b73fff640 1 -- 172.21.15.19:0/521869619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a24ae2fe0 con 0x560a24ae6cc0 2024-04-25T09:59:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b7ad9d640 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24af88d0 0x560a24b1b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:24.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b73fff640 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24ae6cc0 0x560a24b1b050 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c067ea0 tx=0x7f4b64001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ea71cdfa030eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b597fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b597fa640 1 -- 172.21.15.19:0/521869619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6402d420 con 0x560a24ae6cc0 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b597fa640 1 -- 172.21.15.19:0/521869619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b64010040 con 0x560a24ae6cc0 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b835e7740 1 -- 172.21.15.19:0/521869619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b6c0543e0 con 0x560a24ae6cc0 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b597fa640 1 -- 172.21.15.19:0/521869619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6402d5c0 con 0x560a24ae6cc0 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b835e7740 10 client.7758 fetch_fsmap learned FSMap version 58 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b835e7740 1 -- 172.21.15.19:0/521869619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x560a24afee60 con 0x560a24ae6cc0 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b597fa640 1 -- 172.21.15.19:0/521869619 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c0543e0 con 0x560a24ae6cc0 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b597fa640 1 -- 172.21.15.19:0/521869619 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4b64036070 con 0x560a24ae6cc0 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b835e7740 10 client.7758 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b835e7740 10 client.7758 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T09:59:24.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b835e7740 20 client.7758 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b835e7740 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x560a24b6f180 0x560a24afe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b835e7740 4 client.7758 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:59:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.154+0000 7f4b835e7740 1 -- 172.21.15.19:0/521869619 --> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x560a24a27890 con 0x560a24b6f180 2024-04-25T09:59:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.155+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x560a24b6f180 0x560a24afe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.155+0000 7f4b597fa640 10 client.7758 ms_handle_connect on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.155+0000 7f4b7a59c640 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x560a24b6f180 0x560a24afe2b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a97b89bf36bfb1 server_cookie=76819ea695a8d16e in_seq=0 out_seq=0 2024-04-25T09:59:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b597fa640 1 -- 172.21.15.19:0/521869619 <== mds.0 v2:172.21.15.19:6832/3589967477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560a24a27890 con 0x560a24b6f180 2024-04-25T09:59:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b597fa640 10 client.7758 handle_command_reply: tid=0 2024-04-25T09:59:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b835e7740 1 client.7758 shutdown 2024-04-25T09:59:24.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b835e7740 1 -- 172.21.15.19:0/521869619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24ae6cc0 msgr2=0x560a24b1b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b835e7740 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24ae6cc0 0x560a24b1b050 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c067ea0 tx=0x7f4b64001150 comp rx=0 tx=0).stop 2024-04-25T09:59:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b835e7740 1 -- 172.21.15.19:0/521869619 shutdown_connections 2024-04-25T09:59:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b835e7740 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x560a24b6f180 0x560a24afe2b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b835e7740 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a24af88d0 0x560a24b1b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b835e7740 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a24ae6cc0 0x560a24b1b050 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b597fa640 0 client.7758 ms_handle_reset on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b835e7740 1 --2- 172.21.15.19:0/521869619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560a24abc200 0x560a24b14570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b835e7740 1 -- 172.21.15.19:0/521869619 >> 172.21.15.19:0/521869619 conn(0x560a24a84050 msgr2=0x560a24abd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b835e7740 1 -- 172.21.15.19:0/521869619 shutdown_connections 2024-04-25T09:59:24.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.157+0000 7f4b835e7740 1 -- 172.21.15.19:0/521869619 wait complete. 2024-04-25T09:59:24.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b835e7740 20 client.7758 trim_cache size 0 max 16384 2024-04-25T09:59:24.169 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:59:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T09:59:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "2b92e880-549f-4d1c-b216-53729801a885", 2024-04-25T09:59:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T09:59:24.169 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:59:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 -- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5c06b4d0 msgr2=0x7f4b5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b640011e0 tx=0x7f4b64007d30 comp rx=0 tx=0).stop 2024-04-25T09:59:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 -- 172.21.15.19:0/2939576933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b7c0ced80 msgr2=0x7f4b7c0b82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b7c0ced80 0x7f4b7c0b82d0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00c460 tx=0x7f4b6c00c920 comp rx=0 tx=0).stop 2024-04-25T09:59:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 -- 172.21.15.19:0/2939576933 shutdown_connections 2024-04-25T09:59:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b7c161710 0x7f4b7c15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b7c15dc20 0x7f4b7c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 --2- 172.21.15.19:0/2939576933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b7c0ced80 0x7f4b7c0b82d0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 -- 172.21.15.19:0/2939576933 >> 172.21.15.19:0/2939576933 conn(0x7f4b7c0d0420 msgr2=0x7f4b7c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 -- 172.21.15.19:0/2939576933 shutdown_connections 2024-04-25T09:59:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.158+0000 7f4b80807640 1 -- 172.21.15.19:0/2939576933 wait complete. 2024-04-25T09:59:24.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:24.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T09:59:24.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.308+0000 7f5f21610640 1 Processor -- start 2024-04-25T09:59:24.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.308+0000 7f5f21610640 1 -- start start 2024-04-25T09:59:24.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.309+0000 7f5f21610640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5f1c1610f0 0x7f5f1c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.309+0000 7f5f21610640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0c6960 con 0x7f5f1c0ce7b0 2024-04-25T09:59:24.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.309+0000 7f5f21610640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c162260 con 0x7f5f1c1616f0 2024-04-25T09:59:24.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.309+0000 7f5f21610640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c1614d0 con 0x7f5f1c1610f0 2024-04-25T09:59:24.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.309+0000 7f5f1affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5f1c1610f0 0x7f5f1c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.309+0000 7f5f1b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5f1c0ce7b0 0x7f5f1c0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33354/0 (socket says 172.21.15.19:33354) 2024-04-25T09:59:24.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.309+0000 7f5f1affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5f1c1610f0 0x7f5f1c06c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38206/0 (socket says 172.21.15.19:38206) 2024-04-25T09:59:24.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.309+0000 7f5f1b7fe640 1 -- 172.21.15.19:0/2751703074 learned_addr learned my addr 172.21.15.19:0/2751703074 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:24.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.310+0000 7f5f1affd640 1 -- 172.21.15.19:0/2751703074 >> v1:172.21.15.112:6789/0 conn(0x7f5f1c1616f0 legacy=0x7f5f1c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.310+0000 7f5f1affd640 1 -- 172.21.15.19:0/2751703074 >> v1:172.21.15.19:6789/0 conn(0x7f5f1c0ce7b0 legacy=0x7f5f1c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.310+0000 7f5f19ffb640 1 -- 172.21.15.19:0/2751703074 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890490793 0 0) 0x7f5f1c162260 con 0x7f5f1c1616f0 2024-04-25T09:59:24.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.310+0000 7f5f1affd640 1 -- 172.21.15.19:0/2751703074 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c162540 con 0x7f5f1c1610f0 2024-04-25T09:59:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.310+0000 7f5f1affd640 1 --2- 172.21.15.19:0/2751703074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f1c1610f0 0x7f5f1c06c1c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0097a0 tx=0x7f5f0c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d97b6473021a72d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.311+0000 7f5f19ffb640 1 -- 172.21.15.19:0/2751703074 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c037070 con 0x7f5f1c1610f0 2024-04-25T09:59:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.311+0000 7f5f19ffb640 1 -- 172.21.15.19:0/2751703074 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c02adc0 con 0x7f5f1c1610f0 2024-04-25T09:59:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.311+0000 7f5f19ffb640 1 -- 172.21.15.19:0/2751703074 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c032660 con 0x7f5f1c1610f0 2024-04-25T09:59:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.311+0000 7f5f21610640 1 -- 172.21.15.19:0/2751703074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f1c1610f0 msgr2=0x7f5f1c06c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.311+0000 7f5f21610640 1 --2- 172.21.15.19:0/2751703074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f1c1610f0 0x7f5f1c06c1c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0097a0 tx=0x7f5f0c02a130 comp rx=0 tx=0).stop 2024-04-25T09:59:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.311+0000 7f5f21610640 1 -- 172.21.15.19:0/2751703074 shutdown_connections 2024-04-25T09:59:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.311+0000 7f5f21610640 1 --2- 172.21.15.19:0/2751703074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f1c1610f0 0x7f5f1c06c1c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.311+0000 7f5f21610640 1 -- 172.21.15.19:0/2751703074 >> 172.21.15.19:0/2751703074 conn(0x7f5f1c06f500 msgr2=0x7f5f1c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.311+0000 7f5f21610640 1 -- 172.21.15.19:0/2751703074 shutdown_connections 2024-04-25T09:59:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.311+0000 7f5f21610640 1 -- 172.21.15.19:0/2751703074 wait complete. 2024-04-25T09:59:24.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.313+0000 7f5f21610640 1 Processor -- start 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.313+0000 7f5f21610640 1 -- start start 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.313+0000 7f5f21610640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0ce7b0 0x7f5f1c06ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.313+0000 7f5f21610640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f1c1610f0 0x7f5f1c14aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.313+0000 7f5f21610640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f1c1616f0 0x7f5f1c14b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.313+0000 7f5f21610640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0cf690 con 0x7f5f1c1610f0 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.313+0000 7f5f21610640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0cfce0 con 0x7f5f1c0ce7b0 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.313+0000 7f5f21610640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c06c700 con 0x7f5f1c1616f0 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.313+0000 7f5f1a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f1c1610f0 0x7f5f1c14aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f1a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f1c1610f0 0x7f5f1c14aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38218/0 (socket says 172.21.15.19:38218) 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f1a7fc640 1 -- 172.21.15.19:0/1032410346 learned_addr learned my addr 172.21.15.19:0/1032410346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f1b7fe640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f1c1616f0 0x7f5f1c14b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f1affd640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0ce7b0 0x7f5f1c06ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f1a7fc640 1 -- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f1c1616f0 msgr2=0x7f5f1c14b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f1c1616f0 0x7f5f1c14b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f1a7fc640 1 -- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0ce7b0 msgr2=0x7f5f1c06ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0ce7b0 0x7f5f1c06ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f1a7fc640 1 -- 172.21.15.19:0/1032410346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c1f7240 con 0x7f5f1c1610f0 2024-04-25T09:59:24.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f1c1610f0 0x7f5f1c14aed0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400c520 tx=0x7f5f0400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f137fe640 1 -- 172.21.15.19:0/1032410346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04017070 con 0x7f5f1c1610f0 2024-04-25T09:59:24.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f137fe640 1 -- 172.21.15.19:0/1032410346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f04005480 con 0x7f5f1c1610f0 2024-04-25T09:59:24.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f137fe640 1 -- 172.21.15.19:0/1032410346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04003830 con 0x7f5f1c1610f0 2024-04-25T09:59:24.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f21610640 1 -- 172.21.15.19:0/1032410346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c1f74e0 con 0x7f5f1c1610f0 2024-04-25T09:59:24.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.314+0000 7f5f21610640 1 -- 172.21.15.19:0/1032410346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c1f7aa0 con 0x7f5f1c1610f0 2024-04-25T09:59:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.316+0000 7f5f137fe640 1 -- 172.21.15.19:0/1032410346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f0401a070 con 0x7f5f1c1610f0 2024-04-25T09:59:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.316+0000 7f5f137fe640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5efc06b4d0 0x7f5efc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.317+0000 7f5f1affd640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5efc06b4d0 0x7f5efc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.317+0000 7f5f137fe640 1 -- 172.21.15.19:0/1032410346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5f04089720 con 0x7f5f1c1610f0 2024-04-25T09:59:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.317+0000 7f5f1affd640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5efc06b4d0 0x7f5efc06d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c034000 tx=0x7f5f0c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.317+0000 7f5f243ea740 1 Processor -- start 2024-04-25T09:59:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f243ea740 1 -- start start 2024-04-25T09:59:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5651942e6ca0 0x5651942e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 0x5651942e7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565194311760 0x565194311be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f243ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x565194249b90 con 0x565194311760 2024-04-25T09:59:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f243ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651941f2e40 con 0x565194323370 2024-04-25T09:59:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f243ea740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x565194245c30 con 0x5651942e6ca0 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f1b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565194311760 0x565194311be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f1b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565194311760 0x565194311be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38224/0 (socket says 172.21.15.19:38224) 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f1b7fe640 1 -- 172.21.15.19:0/3139081602 learned_addr learned my addr 172.21.15.19:0/3139081602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f1a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 0x5651942e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.318+0000 7f5f1affd640 1 --2- 172.21.15.19:0/3139081602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5651942e6ca0 0x5651942e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.319+0000 7f5f1b7fe640 1 -- 172.21.15.19:0/3139081602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5651942e6ca0 msgr2=0x5651942e8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.319+0000 7f5f1b7fe640 1 --2- 172.21.15.19:0/3139081602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5651942e6ca0 0x5651942e8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.319+0000 7f5f1b7fe640 1 -- 172.21.15.19:0/3139081602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 msgr2=0x5651942e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.319+0000 7f5f1b7fe640 1 --2- 172.21.15.19:0/3139081602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 0x5651942e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.319+0000 7f5f1b7fe640 1 -- 172.21.15.19:0/3139081602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651940287d0 con 0x565194311760 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.319+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/3139081602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 0x5651942e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.319+0000 7f5f1affd640 1 --2- 172.21.15.19:0/3139081602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5651942e6ca0 0x5651942e8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.319+0000 7f5f1b7fe640 1 --2- 172.21.15.19:0/3139081602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565194311760 0x565194311be0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f5f14001b40 tx=0x7f5f14002000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f5b2d8b58e269c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.319+0000 7f5f10ff9640 1 -- 172.21.15.19:0/3139081602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f140078b0 con 0x565194311760 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.319+0000 7f5f10ff9640 1 -- 172.21.15.19:0/3139081602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f14018070 con 0x565194311760 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.319+0000 7f5f10ff9640 1 -- 172.21.15.19:0/3139081602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14013630 con 0x565194311760 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.320+0000 7f5f243ea740 1 -- 172.21.15.19:0/3139081602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565194311760 msgr2=0x565194311be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.320+0000 7f5f243ea740 1 --2- 172.21.15.19:0/3139081602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565194311760 0x565194311be0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f5f14001b40 tx=0x7f5f14002000 comp rx=0 tx=0).stop 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.320+0000 7f5f243ea740 1 -- 172.21.15.19:0/3139081602 shutdown_connections 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.320+0000 7f5f243ea740 1 --2- 172.21.15.19:0/3139081602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565194311760 0x565194311be0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.320+0000 7f5f243ea740 1 --2- 172.21.15.19:0/3139081602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 0x5651942e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.320+0000 7f5f243ea740 1 --2- 172.21.15.19:0/3139081602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5651942e6ca0 0x5651942e8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.320+0000 7f5f243ea740 1 -- 172.21.15.19:0/3139081602 >> 172.21.15.19:0/3139081602 conn(0x5651942ae870 msgr2=0x56519432c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.320+0000 7f5f243ea740 1 -- 172.21.15.19:0/3139081602 shutdown_connections 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.320+0000 7f5f243ea740 1 -- 172.21.15.19:0/3139081602 wait complete. 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.320+0000 7f5f243ea740 1 -- start start 2024-04-25T09:59:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.321+0000 7f5f243ea740 1 Processor -- start 2024-04-25T09:59:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.321+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x5651943137e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.321+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194311760 0x565194313d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.321+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194323370 0x565194326c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.321+0000 7f5f243ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651942063e0 con 0x5651942e6ca0 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.321+0000 7f5f243ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519430f2b0 con 0x565194311760 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.321+0000 7f5f243ea740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x565194324550 con 0x565194323370 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.321+0000 7f5f243ea740 10 client.0 _ll_register_callbacks cb 0x565194205db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.321+0000 7f5f1affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x5651943137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.321+0000 7f5f1affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x5651943137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38228/0 (socket says 172.21.15.19:38228) 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.321+0000 7f5f1affd640 1 -- 172.21.15.19:0/3644787740 learned_addr learned my addr 172.21.15.19:0/3644787740 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f1b7fe640 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194323370 0x565194326c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194311760 0x565194313d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f1affd640 1 -- 172.21.15.19:0/3644787740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194323370 msgr2=0x565194326c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f1affd640 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194323370 0x565194326c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f1affd640 1 -- 172.21.15.19:0/3644787740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194311760 msgr2=0x565194313d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f1affd640 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194311760 0x565194313d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f1affd640 1 -- 172.21.15.19:0/3644787740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565193f1e0f0 con 0x5651942e6ca0 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f1b7fe640 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194323370 0x565194326c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194311760 0x565194313d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f1affd640 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x5651943137e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0098c0 tx=0x7f5f0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5758cc43b26a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5ef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/3644787740 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c050020 con 0x5651942e6ca0 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/3644787740 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5f0c04c460 con 0x5651942e6ca0 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f243ea740 1 -- 172.21.15.19:0/3644787740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f14018070 con 0x5651942e6ca0 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/3644787740 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f0c05a050 con 0x5651942e6ca0 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f243ea740 10 client.7773 fetch_fsmap learned FSMap version 58 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5f243ea740 1 -- 172.21.15.19:0/3644787740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x565194088f60 con 0x5651942e6ca0 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.322+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/3644787740 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f14018070 con 0x5651942e6ca0 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.323+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/3644787740 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5f0c05d020 con 0x5651942e6ca0 2024-04-25T09:59:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.323+0000 7f5f243ea740 10 client.7773 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.323+0000 7f5f243ea740 10 client.7773 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.323+0000 7f5f243ea740 20 client.7773 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.323+0000 7f5f243ea740 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x5651942fcd00 0x5651942fe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.323+0000 7f5f243ea740 4 client.7773 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.323+0000 7f5f243ea740 1 -- 172.21.15.19:0/3644787740 --> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651942570d0 con 0x5651942fcd00 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.323+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x5651942fcd00 0x5651942fe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.323+0000 7f5ef9ffb640 10 client.7773 ms_handle_connect on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.323+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x5651942fcd00 0x5651942fe480 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8654579afd37f923 server_cookie=374e804f285a8183 in_seq=0 out_seq=0 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/3644787740 <== mds.0 v2:172.21.15.19:6832/3589967477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5651942570d0 con 0x5651942fcd00 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5ef9ffb640 10 client.7773 handle_command_reply: tid=0 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 1 client.7773 shutdown 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 1 -- 172.21.15.19:0/3644787740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 msgr2=0x5651943137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x5651943137e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0098c0 tx=0x7f5f0c004730 comp rx=0 tx=0).stop 2024-04-25T09:59:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 1 -- 172.21.15.19:0/3644787740 shutdown_connections 2024-04-25T09:59:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x5651942fcd00 0x5651942fe480 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194323370 0x565194326c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5ef9ffb640 0 client.7773 ms_handle_reset on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194311760 0x565194313d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 1 --2- 172.21.15.19:0/3644787740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x5651943137e0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 1 -- 172.21.15.19:0/3644787740 >> 172.21.15.19:0/3644787740 conn(0x5651942ae870 msgr2=0x56519430ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 1 -- 172.21.15.19:0/3644787740 shutdown_connections 2024-04-25T09:59:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 1 -- 172.21.15.19:0/3644787740 wait complete. 2024-04-25T09:59:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.324+0000 7f5f243ea740 20 client.7773 trim_cache size 0 max 16384 2024-04-25T09:59:24.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.339+0000 7f5f243ea740 1 Processor -- start 2024-04-25T09:59:24.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.339+0000 7f5f243ea740 1 -- start start 2024-04-25T09:59:24.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.339+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194310a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.339+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194311760 0x565194310f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.339+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 0x565194327990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.339+0000 7f5f243ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c2002f0 con 0x5651942e6ca0 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.339+0000 7f5f243ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c1ff4d0 con 0x565194323370 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.339+0000 7f5f243ea740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0b0240 con 0x565194311760 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194310a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194310a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38234/0 (socket says 172.21.15.19:38234) 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1affd640 1 -- 172.21.15.19:0/996804018 learned_addr learned my addr 172.21.15.19:0/996804018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1b7fe640 1 --2- 172.21.15.19:0/996804018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 0x565194327990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/996804018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194311760 0x565194310f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1affd640 1 -- 172.21.15.19:0/996804018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194311760 msgr2=0x565194310f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1affd640 1 --2- 172.21.15.19:0/996804018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194311760 0x565194310f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1affd640 1 -- 172.21.15.19:0/996804018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 msgr2=0x565194327990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1affd640 1 --2- 172.21.15.19:0/996804018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 0x565194327990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1affd640 1 -- 172.21.15.19:0/996804018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565193ef71b0 con 0x5651942e6ca0 2024-04-25T09:59:24.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1b7fe640 1 --2- 172.21.15.19:0/996804018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 0x565194327990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/996804018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194311760 0x565194310f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5f1affd640 1 --2- 172.21.15.19:0/996804018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194310a20 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c004400 tx=0x7f5f0c049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab75b47ccb649e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5efbfff640 1 -- 172.21.15.19:0/996804018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c063070 con 0x5651942e6ca0 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5efbfff640 1 -- 172.21.15.19:0/996804018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c04c7f0 con 0x5651942e6ca0 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.340+0000 7f5efbfff640 1 -- 172.21.15.19:0/996804018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c041030 con 0x5651942e6ca0 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.341+0000 7f5f243ea740 1 -- 172.21.15.19:0/996804018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 msgr2=0x565194310a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.341+0000 7f5f243ea740 1 --2- 172.21.15.19:0/996804018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194310a20 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c004400 tx=0x7f5f0c049000 comp rx=0 tx=0).stop 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.341+0000 7f5f243ea740 1 -- 172.21.15.19:0/996804018 shutdown_connections 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.341+0000 7f5f243ea740 1 --2- 172.21.15.19:0/996804018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194323370 0x565194327990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.341+0000 7f5f243ea740 1 --2- 172.21.15.19:0/996804018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194311760 0x565194310f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.341+0000 7f5f243ea740 1 --2- 172.21.15.19:0/996804018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194310a20 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.341+0000 7f5f243ea740 1 -- 172.21.15.19:0/996804018 >> 172.21.15.19:0/996804018 conn(0x5651942ae870 msgr2=0x565194314bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.341+0000 7f5f243ea740 1 -- 172.21.15.19:0/996804018 shutdown_connections 2024-04-25T09:59:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.341+0000 7f5f243ea740 1 -- 172.21.15.19:0/996804018 wait complete. 2024-04-25T09:59:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.341+0000 7f5f243ea740 1 -- start start 2024-04-25T09:59:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.341+0000 7f5f243ea740 1 Processor -- start 2024-04-25T09:59:24.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194396b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194311760 0x5651943220f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f243ea740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194323370 0x565194322630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f243ea740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0b0840 con 0x5651942e6ca0 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f243ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651942dd030 con 0x565194311760 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f243ea740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56519430d470 con 0x565194323370 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f243ea740 10 client.0 _ll_register_callbacks cb 0x7f5f0c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f1affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194396b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f1affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194396b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38240/0 (socket says 172.21.15.19:38240) 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f1affd640 1 -- 172.21.15.19:0/531231656 learned_addr learned my addr 172.21.15.19:0/531231656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194311760 0x5651943220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.342+0000 7f5f1b7fe640 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194323370 0x565194322630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f1affd640 1 -- 172.21.15.19:0/531231656 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194323370 msgr2=0x565194322630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f1affd640 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194323370 0x565194322630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f1affd640 1 -- 172.21.15.19:0/531231656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194311760 msgr2=0x5651943220f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f1affd640 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194311760 0x5651943220f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f1affd640 1 -- 172.21.15.19:0/531231656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56519430e400 con 0x5651942e6ca0 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f1affd640 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194396b60 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c02a720 tx=0x7f5f0c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1879ff3d00e05964 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5ef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/531231656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c06e060 con 0x5651942e6ca0 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/531231656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5f0c03d650 con 0x5651942e6ca0 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f243ea740 1 -- 172.21.15.19:0/531231656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f0c04c7f0 con 0x5651942e6ca0 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/531231656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f0c074050 con 0x5651942e6ca0 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f243ea740 10 client.7779 fetch_fsmap learned FSMap version 58 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f243ea740 1 -- 172.21.15.19:0/531231656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x5651942e8720 con 0x5651942e6ca0 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/531231656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c04c7f0 con 0x5651942e6ca0 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/531231656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5f0c077020 con 0x5651942e6ca0 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f243ea740 10 client.7779 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T09:59:24.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f243ea740 10 client.7779 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f243ea740 20 client.7779 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f243ea740 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x565194313e40 0x56519439a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f243ea740 4 client.7779 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.343+0000 7f5f243ea740 1 -- 172.21.15.19:0/531231656 --> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x565194251420 con 0x565194313e40 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x565194313e40 0x56519439a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f1a7fc640 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x565194313e40 0x56519439a5d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5a211860fd3fdf2 server_cookie=f316975c53ffe1b9 in_seq=0 out_seq=0 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5ef9ffb640 10 client.7779 ms_handle_connect on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5ef9ffb640 1 -- 172.21.15.19:0/531231656 <== mds.0 v2:172.21.15.19:6832/3589967477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x565194251420 con 0x565194313e40 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5ef9ffb640 10 client.7779 handle_command_reply: tid=0 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 1 client.7779 shutdown 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 1 -- 172.21.15.19:0/531231656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 msgr2=0x565194396b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194396b60 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c02a720 tx=0x7f5f0c009190 comp rx=0 tx=0).stop 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 1 -- 172.21.15.19:0/531231656 shutdown_connections 2024-04-25T09:59:24.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x565194313e40 0x56519439a5d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x565194323370 0x565194322630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565194311760 0x5651943220f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5ef9ffb640 0 client.7779 ms_handle_reset on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 1 --2- 172.21.15.19:0/531231656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651942e6ca0 0x565194396b60 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 1 -- 172.21.15.19:0/531231656 >> 172.21.15.19:0/531231656 conn(0x5651942ae870 msgr2=0x56519434e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 1 -- 172.21.15.19:0/531231656 shutdown_connections 2024-04-25T09:59:24.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 1 -- 172.21.15.19:0/531231656 wait complete. 2024-04-25T09:59:24.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.344+0000 7f5f243ea740 20 client.7779 trim_cache size 0 max 16384 2024-04-25T09:59:24.356 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:59:24.356 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T09:59:24.357 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:59:24.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 -- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5efc06b4d0 msgr2=0x7f5efc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5efc06b4d0 0x7f5efc06d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c034000 tx=0x7f5f0c02e000 comp rx=0 tx=0).stop 2024-04-25T09:59:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 -- 172.21.15.19:0/1032410346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f1c1610f0 msgr2=0x7f5f1c14aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f1c1610f0 0x7f5f1c14aed0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400c520 tx=0x7f5f0400c9e0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 -- 172.21.15.19:0/1032410346 shutdown_connections 2024-04-25T09:59:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5efc06b4d0 0x7f5efc06d990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f1c1616f0 0x7f5f1c14b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f1c1610f0 0x7f5f1c14aed0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 --2- 172.21.15.19:0/1032410346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0ce7b0 0x7f5f1c06ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 -- 172.21.15.19:0/1032410346 >> 172.21.15.19:0/1032410346 conn(0x7f5f1c06f500 msgr2=0x7f5f1c071820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 -- 172.21.15.19:0/1032410346 shutdown_connections 2024-04-25T09:59:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.345+0000 7f5f21610640 1 -- 172.21.15.19:0/1032410346 wait complete. 2024-04-25T09:59:24.360 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:59:24.360 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:59:24.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.460+0000 7fbb5b57e640 1 Processor -- start 2024-04-25T09:59:24.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.460+0000 7fbb5b57e640 1 -- start start 2024-04-25T09:59:24.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.461+0000 7fbb5b57e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbb5415dac0 0x7fbb540d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.461+0000 7fbb5b57e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb540d0ab0 0x7fbb540b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.461+0000 7fbb5b57e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb540d2420 con 0x7fbb540d1f30 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.461+0000 7fbb5b57e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb540d3030 con 0x7fbb5415dac0 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.461+0000 7fbb5b57e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540d2700 con 0x7fbb540d0ab0 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.461+0000 7fbb59b1b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb540d0ab0 0x7fbb540b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.461+0000 7fbb5931a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbb5415dac0 0x7fbb540d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.461+0000 7fbb59b1b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb540d0ab0 0x7fbb540b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:58554/0 (socket says 172.21.15.19:58554) 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.461+0000 7fbb58b19640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbb540d1f30 0x7fbb540d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:40664/0 (socket says 172.21.15.19:40664) 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.461+0000 7fbb59b1b640 1 -- 172.21.15.19:0/3040170540 learned_addr learned my addr 172.21.15.19:0/3040170540 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.462+0000 7fbb53fff640 1 -- 172.21.15.19:0/3040170540 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599878012 0 0) 0x7fbb540d2420 con 0x7fbb540d1f30 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.462+0000 7fbb59b1b640 1 -- 172.21.15.19:0/3040170540 >> v2:172.21.15.112:3300/0 conn(0x7fbb5415dac0 msgr2=0x7fbb540d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.462+0000 7fbb59b1b640 1 --2- 172.21.15.19:0/3040170540 >> v2:172.21.15.112:3300/0 conn(0x7fbb5415dac0 0x7fbb540d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.462+0000 7fbb59b1b640 1 -- 172.21.15.19:0/3040170540 >> v1:172.21.15.112:6789/0 conn(0x7fbb540d1f30 legacy=0x7fbb540d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.462+0000 7fbb59b1b640 1 -- 172.21.15.19:0/3040170540 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540d29e0 con 0x7fbb540d0ab0 2024-04-25T09:59:24.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.462+0000 7fbb5931a640 1 --2- 172.21.15.19:0/3040170540 >> v2:172.21.15.112:3300/0 conn(0x7fbb5415dac0 0x7fbb540d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.462+0000 7fbb59b1b640 1 --2- 172.21.15.19:0/3040170540 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb540d0ab0 0x7fbb540b6990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009e50 tx=0x7fbb4402a7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e52bf5f8bd97008a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.463+0000 7fbb53fff640 1 -- 172.21.15.19:0/3040170540 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44036070 con 0x7fbb540d0ab0 2024-04-25T09:59:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.463+0000 7fbb53fff640 1 -- 172.21.15.19:0/3040170540 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb44031600 con 0x7fbb540d0ab0 2024-04-25T09:59:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.463+0000 7fbb53fff640 1 -- 172.21.15.19:0/3040170540 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440319f0 con 0x7fbb540d0ab0 2024-04-25T09:59:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.463+0000 7fbb5b57e640 1 -- 172.21.15.19:0/3040170540 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb540d0ab0 msgr2=0x7fbb540b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.463+0000 7fbb5b57e640 1 --2- 172.21.15.19:0/3040170540 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb540d0ab0 0x7fbb540b6990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009e50 tx=0x7fbb4402a7c0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.463+0000 7fbb5b57e640 1 -- 172.21.15.19:0/3040170540 shutdown_connections 2024-04-25T09:59:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.463+0000 7fbb5b57e640 1 --2- 172.21.15.19:0/3040170540 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb540d0ab0 0x7fbb540b6990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.463+0000 7fbb5b57e640 1 --2- 172.21.15.19:0/3040170540 >> v2:172.21.15.112:3300/0 conn(0x7fbb5415dac0 0x7fbb540d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.463+0000 7fbb5b57e640 1 -- 172.21.15.19:0/3040170540 >> 172.21.15.19:0/3040170540 conn(0x7fbb54061c30 msgr2=0x7fbb5406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.463+0000 7fbb5b57e640 1 -- 172.21.15.19:0/3040170540 shutdown_connections 2024-04-25T09:59:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.463+0000 7fbb5b57e640 1 -- 172.21.15.19:0/3040170540 wait complete. 2024-04-25T09:59:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.464+0000 7fbb5b57e640 1 Processor -- start 2024-04-25T09:59:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.464+0000 7fbb5b57e640 1 -- start start 2024-04-25T09:59:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.465+0000 7fbb5b57e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb540d0ab0 0x7fbb54152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.465+0000 7fbb5b57e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb540d1f30 0x7fbb54154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.465+0000 7fbb5b57e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb5415dac0 0x7fbb54156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.465+0000 7fbb5b57e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540b97d0 con 0x7fbb540d1f30 2024-04-25T09:59:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.465+0000 7fbb5b57e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5415dea0 con 0x7fbb540d0ab0 2024-04-25T09:59:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.465+0000 7fbb5b57e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540b9950 con 0x7fbb5415dac0 2024-04-25T09:59:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.465+0000 7fbb58b19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb540d1f30 0x7fbb54154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.465+0000 7fbb58b19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb540d1f30 0x7fbb54154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38246/0 (socket says 172.21.15.19:38246) 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.465+0000 7fbb58b19640 1 -- 172.21.15.19:0/4065911346 learned_addr learned my addr 172.21.15.19:0/4065911346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.465+0000 7fbb59b1b640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb5415dac0 0x7fbb54156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.465+0000 7fbb5931a640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb540d0ab0 0x7fbb54152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.466+0000 7fbb58b19640 1 -- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb5415dac0 msgr2=0x7fbb54156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.466+0000 7fbb58b19640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb5415dac0 0x7fbb54156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.466+0000 7fbb58b19640 1 -- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb540d0ab0 msgr2=0x7fbb54152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.466+0000 7fbb58b19640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb540d0ab0 0x7fbb54152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.466+0000 7fbb58b19640 1 -- 172.21.15.19:0/4065911346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb541570b0 con 0x7fbb540d1f30 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.466+0000 7fbb58b19640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb540d1f30 0x7fbb54154910 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c460 tx=0x7fbb4800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.466+0000 7fbb51ffb640 1 -- 172.21.15.19:0/4065911346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48018070 con 0x7fbb540d1f30 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.466+0000 7fbb51ffb640 1 -- 172.21.15.19:0/4065911346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb480074e0 con 0x7fbb540d1f30 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.466+0000 7fbb5b57e640 1 -- 172.21.15.19:0/4065911346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb54156af0 con 0x7fbb540d1f30 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.466+0000 7fbb51ffb640 1 -- 172.21.15.19:0/4065911346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48004a50 con 0x7fbb540d1f30 2024-04-25T09:59:24.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.466+0000 7fbb5b57e640 1 -- 172.21.15.19:0/4065911346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb540b3ff0 con 0x7fbb540d1f30 2024-04-25T09:59:24.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.468+0000 7fbb51ffb640 1 -- 172.21.15.19:0/4065911346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb4801b070 con 0x7fbb540d1f30 2024-04-25T09:59:24.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.468+0000 7fbb51ffb640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb4006b4d0 0x7fbb4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.468+0000 7fbb51ffb640 1 -- 172.21.15.19:0/4065911346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb48089540 con 0x7fbb540d1f30 2024-04-25T09:59:24.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.469+0000 7fbb5931a640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb4006b4d0 0x7fbb4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.469+0000 7fbb5931a640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb4006b4d0 0x7fbb4006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009560 tx=0x7fbb4c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.469+0000 7fbb5e366740 1 Processor -- start 2024-04-25T09:59:24.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb5e366740 1 -- start start 2024-04-25T09:59:24.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad95320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523ad95860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523ad65770 0x56523ad68f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb5e366740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523acceda0 con 0x56523ad92b10 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb5e366740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523ac77b80 con 0x56523ad65770 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb5e366740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56523accae40 con 0x56523ad92ef0 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb5931a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad95320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb5931a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad95320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38252/0 (socket says 172.21.15.19:38252) 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb5931a640 1 -- 172.21.15.19:0/2870513949 learned_addr learned my addr 172.21.15.19:0/2870513949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb59b1b640 1 --2- 172.21.15.19:0/2870513949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523ad65770 0x56523ad68f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.470+0000 7fbb58b19640 1 --2- 172.21.15.19:0/2870513949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523ad95860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.471+0000 7fbb5931a640 1 -- 172.21.15.19:0/2870513949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 msgr2=0x56523ad95860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.471+0000 7fbb5931a640 1 --2- 172.21.15.19:0/2870513949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523ad95860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.471+0000 7fbb5931a640 1 -- 172.21.15.19:0/2870513949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523ad65770 msgr2=0x56523ad68f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.471+0000 7fbb5931a640 1 --2- 172.21.15.19:0/2870513949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523ad65770 0x56523ad68f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.471+0000 7fbb5931a640 1 -- 172.21.15.19:0/2870513949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56523ac81f10 con 0x56523ad92b10 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.471+0000 7fbb5931a640 1 --2- 172.21.15.19:0/2870513949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad95320 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c011dc0 tx=0x7fbb4c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4800769b0cd4ea42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.471+0000 7fbb3affd640 1 -- 172.21.15.19:0/2870513949 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c014bd0 con 0x56523ad92b10 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.471+0000 7fbb3affd640 1 -- 172.21.15.19:0/2870513949 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c01b070 con 0x56523ad92b10 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.471+0000 7fbb3affd640 1 -- 172.21.15.19:0/2870513949 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c004030 con 0x56523ad92b10 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.472+0000 7fbb5e366740 1 -- 172.21.15.19:0/2870513949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 msgr2=0x56523ad95320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.472+0000 7fbb5e366740 1 --2- 172.21.15.19:0/2870513949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad95320 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c011dc0 tx=0x7fbb4c014000 comp rx=0 tx=0).stop 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.472+0000 7fbb5e366740 1 -- 172.21.15.19:0/2870513949 shutdown_connections 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.472+0000 7fbb5e366740 1 --2- 172.21.15.19:0/2870513949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523ad65770 0x56523ad68f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.472+0000 7fbb5e366740 1 --2- 172.21.15.19:0/2870513949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523ad95860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.472+0000 7fbb5e366740 1 --2- 172.21.15.19:0/2870513949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad95320 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.472+0000 7fbb5e366740 1 -- 172.21.15.19:0/2870513949 >> 172.21.15.19:0/2870513949 conn(0x56523ad679e0 msgr2=0x56523adb0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.472+0000 7fbb5e366740 1 -- 172.21.15.19:0/2870513949 shutdown_connections 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.472+0000 7fbb5e366740 1 -- 172.21.15.19:0/2870513949 wait complete. 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.472+0000 7fbb5e366740 1 -- start start 2024-04-25T09:59:24.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.472+0000 7fbb5e366740 1 Processor -- start 2024-04-25T09:59:24.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad90d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523ad912a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523adb4a60 0x56523ad7c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb5e366740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523ad6ab90 con 0x56523ad92b10 2024-04-25T09:59:24.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb5e366740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523ac6d4b0 con 0x56523adb4a60 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb5e366740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56523ad6ae70 con 0x56523ad92ef0 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb5e366740 10 client.0 _ll_register_callbacks cb 0x56523acad180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb5931a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad90d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb5931a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad90d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38264/0 (socket says 172.21.15.19:38264) 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb58b19640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523ad912a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb5931a640 1 -- 172.21.15.19:0/2694318029 learned_addr learned my addr 172.21.15.19:0/2694318029 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.473+0000 7fbb59b1b640 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523adb4a60 0x56523ad7c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb58b19640 1 -- 172.21.15.19:0/2694318029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523adb4a60 msgr2=0x56523ad7c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb58b19640 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523adb4a60 0x56523ad7c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb58b19640 1 -- 172.21.15.19:0/2694318029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 msgr2=0x56523ad90d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb58b19640 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad90d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb58b19640 1 -- 172.21.15.19:0/2694318029 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56523aaad7d0 con 0x56523ad92ef0 2024-04-25T09:59:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb5931a640 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad90d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb58b19640 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523ad912a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb48087940 tx=0x7fbb4805be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=173e53251c905130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb33fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3301/0 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb33fff640 1 -- 172.21.15.19:0/2694318029 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4805e070 con 0x56523ad92ef0 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb33fff640 1 -- 172.21.15.19:0/2694318029 <== mon.2 v2:172.21.15.112:3301/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb480890b0 con 0x56523ad92ef0 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb5e366740 1 -- 172.21.15.19:0/2694318029 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb4c01b070 con 0x56523ad92ef0 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb33fff640 1 -- 172.21.15.19:0/2694318029 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb480892b0 con 0x56523ad92ef0 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb5e366740 10 client.6326 fetch_fsmap learned FSMap version 58 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb5e366740 1 -- 172.21.15.19:0/2694318029 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x56523a9e33f0 con 0x56523ad92ef0 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb33fff640 1 -- 172.21.15.19:0/2694318029 <== mon.2 v2:172.21.15.112:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c01b070 con 0x56523ad92ef0 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb33fff640 1 -- 172.21.15.19:0/2694318029 <== mon.2 v2:172.21.15.112:3301/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb48057180 con 0x56523ad92ef0 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb5e366740 10 client.6326 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.474+0000 7fbb5e366740 10 client.6326 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.475+0000 7fbb5e366740 20 client.6326 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:24.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.475+0000 7fbb5e366740 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x56523ad7f600 0x56523adcd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.475+0000 7fbb5e366740 4 client.6326 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.475+0000 7fbb5e366740 1 -- 172.21.15.19:0/2694318029 --> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56523acdc2e0 con 0x56523ad7f600 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.475+0000 7fbb5931a640 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x56523ad7f600 0x56523adcd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.475+0000 7fbb33fff640 10 client.6326 ms_handle_connect on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.475+0000 7fbb5931a640 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x56523ad7f600 0x56523adcd600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df35ae96fde86d7 server_cookie=2a97ee2a8ecb46c6 in_seq=0 out_seq=0 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.475+0000 7fbb33fff640 1 -- 172.21.15.19:0/2694318029 <== mds.0 v2:172.21.15.19:6832/3589967477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56523acdc2e0 con 0x56523ad7f600 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.475+0000 7fbb33fff640 10 client.6326 handle_command_reply: tid=0 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 1 client.6326 shutdown 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 1 -- 172.21.15.19:0/2694318029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 msgr2=0x56523ad912a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523ad912a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb48087940 tx=0x7fbb4805be40 comp rx=0 tx=0).stop 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 1 -- 172.21.15.19:0/2694318029 shutdown_connections 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x56523ad7f600 0x56523adcd600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523adb4a60 0x56523ad7c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523ad912a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb33fff640 0 client.6326 ms_handle_reset on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 1 --2- 172.21.15.19:0/2694318029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ad90d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 1 -- 172.21.15.19:0/2694318029 >> 172.21.15.19:0/2694318029 conn(0x56523ad65770 msgr2=0x56523ad66400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 1 -- 172.21.15.19:0/2694318029 shutdown_connections 2024-04-25T09:59:24.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 1 -- 172.21.15.19:0/2694318029 wait complete. 2024-04-25T09:59:24.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.476+0000 7fbb5e366740 20 client.6326 trim_cache size 0 max 16384 2024-04-25T09:59:24.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.491+0000 7fbb5e366740 1 Processor -- start 2024-04-25T09:59:24.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.491+0000 7fbb5e366740 1 -- start start 2024-04-25T09:59:24.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92b10 0x56523ad90400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523ad92ef0 0x56523adb1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523adb4a60 0x56523ad90b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb5e366740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523adca2f0 con 0x56523adb4a60 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb5e366740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54200300 con 0x56523ad92ef0 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb5e366740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541ff4f0 con 0x56523ad92b10 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb59b1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523adb4a60 0x56523ad90b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb59b1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523adb4a60 0x56523ad90b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38270/0 (socket says 172.21.15.19:38270) 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb59b1b640 1 -- 172.21.15.19:0/4004627395 learned_addr learned my addr 172.21.15.19:0/4004627395 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb5931a640 1 --2- 172.21.15.19:0/4004627395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92b10 0x56523ad90400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb58b19640 1 --2- 172.21.15.19:0/4004627395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523ad92ef0 0x56523adb1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb59b1b640 1 -- 172.21.15.19:0/4004627395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92b10 msgr2=0x56523ad90400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb59b1b640 1 --2- 172.21.15.19:0/4004627395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92b10 0x56523ad90400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb59b1b640 1 -- 172.21.15.19:0/4004627395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523ad92ef0 msgr2=0x56523adb1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb59b1b640 1 --2- 172.21.15.19:0/4004627395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523ad92ef0 0x56523adb1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb59b1b640 1 -- 172.21.15.19:0/4004627395 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56523ad7cd50 con 0x56523adb4a60 2024-04-25T09:59:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb58b19640 1 --2- 172.21.15.19:0/4004627395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523ad92ef0 0x56523adb1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.492+0000 7fbb59b1b640 1 --2- 172.21.15.19:0/4004627395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523adb4a60 0x56523ad90b70 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fbb44034040 tx=0x7fbb4402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb79f0a977a7198 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb3a7fc640 1 -- 172.21.15.19:0/4004627395 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44036040 con 0x56523adb4a60 2024-04-25T09:59:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb3a7fc640 1 -- 172.21.15.19:0/4004627395 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb44037040 con 0x56523adb4a60 2024-04-25T09:59:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb3a7fc640 1 -- 172.21.15.19:0/4004627395 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4403fae0 con 0x56523adb4a60 2024-04-25T09:59:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb5e366740 1 -- 172.21.15.19:0/4004627395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523adb4a60 msgr2=0x56523ad90b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb5e366740 1 --2- 172.21.15.19:0/4004627395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523adb4a60 0x56523ad90b70 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fbb44034040 tx=0x7fbb4402e000 comp rx=0 tx=0).stop 2024-04-25T09:59:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb5e366740 1 -- 172.21.15.19:0/4004627395 shutdown_connections 2024-04-25T09:59:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb5e366740 1 --2- 172.21.15.19:0/4004627395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523adb4a60 0x56523ad90b70 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb5e366740 1 --2- 172.21.15.19:0/4004627395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523ad92ef0 0x56523adb1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb5e366740 1 --2- 172.21.15.19:0/4004627395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92b10 0x56523ad90400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb5e366740 1 -- 172.21.15.19:0/4004627395 >> 172.21.15.19:0/4004627395 conn(0x56523ad65770 msgr2=0x56523ad6ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb5e366740 1 -- 172.21.15.19:0/4004627395 shutdown_connections 2024-04-25T09:59:24.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.493+0000 7fbb5e366740 1 -- 172.21.15.19:0/4004627395 wait complete. 2024-04-25T09:59:24.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.494+0000 7fbb5e366740 1 -- start start 2024-04-25T09:59:24.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.494+0000 7fbb5e366740 1 Processor -- start 2024-04-25T09:59:24.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ae19f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523adad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5e366740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523adb4a60 0x56523adadbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5e366740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540b5b10 con 0x56523ad92b10 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5e366740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540b5510 con 0x56523adb4a60 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5e366740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56523adad450 con 0x56523ad92ef0 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5e366740 10 client.0 _ll_register_callbacks cb 0x7fbb4c009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5931a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ae19f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5931a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ae19f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38274/0 (socket says 172.21.15.19:38274) 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5931a640 1 -- 172.21.15.19:0/54093012 learned_addr learned my addr 172.21.15.19:0/54093012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb58b19640 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523adad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb59b1b640 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523adb4a60 0x56523adadbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5931a640 1 -- 172.21.15.19:0/54093012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 msgr2=0x56523adad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5931a640 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523adad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5931a640 1 -- 172.21.15.19:0/54093012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523adb4a60 msgr2=0x56523adadbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:24.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5931a640 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523adb4a60 0x56523adadbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5931a640 1 -- 172.21.15.19:0/54093012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56523a9a30f0 con 0x56523ad92b10 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb58b19640 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523adad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5931a640 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ae19f50 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c014fd0 tx=0x7fbb4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e24fb1b036448f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb33fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb33fff640 1 -- 172.21.15.19:0/54093012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c039070 con 0x56523ad92b10 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb33fff640 1 -- 172.21.15.19:0/54093012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb4c0353f0 con 0x56523ad92b10 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb5e366740 1 -- 172.21.15.19:0/54093012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb44037040 con 0x56523ad92b10 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.495+0000 7fbb33fff640 1 -- 172.21.15.19:0/54093012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb4c045050 con 0x56523ad92b10 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb5e366740 10 client.7794 fetch_fsmap learned FSMap version 58 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb5e366740 1 -- 172.21.15.19:0/54093012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x56523ada6750 con 0x56523ad92b10 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb33fff640 1 -- 172.21.15.19:0/54093012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb44037040 con 0x56523ad92b10 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb33fff640 1 -- 172.21.15.19:0/54093012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb4c035ce0 con 0x56523ad92b10 2024-04-25T09:59:24.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb5e366740 10 client.7794 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb5e366740 10 client.7794 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb5e366740 20 client.7794 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb5e366740 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x56523ad66560 0x56523adb0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb5e366740 4 client.7794 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb5e366740 1 -- 172.21.15.19:0/54093012 --> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56523acd6630 con 0x56523ad66560 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb58b19640 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x56523ad66560 0x56523adb0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb33fff640 10 client.7794 ms_handle_connect on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb58b19640 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x56523ad66560 0x56523adb0320 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76c5222b35b14a server_cookie=a015ae365e6caf01 in_seq=0 out_seq=0 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb33fff640 1 -- 172.21.15.19:0/54093012 <== mds.0 v2:172.21.15.19:6832/3589967477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x56523acd6630 con 0x56523ad66560 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb33fff640 10 client.7794 handle_command_reply: tid=0 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb5e366740 1 client.7794 shutdown 2024-04-25T09:59:24.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb5e366740 1 -- 172.21.15.19:0/54093012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 msgr2=0x56523ae19f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.496+0000 7fbb5e366740 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ae19f50 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c014fd0 tx=0x7fbb4c004730 comp rx=0 tx=0).stop 2024-04-25T09:59:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5e366740 1 -- 172.21.15.19:0/54093012 shutdown_connections 2024-04-25T09:59:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5e366740 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x56523ad66560 0x56523adb0320 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5e366740 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56523adb4a60 0x56523adadbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5e366740 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56523ad92ef0 0x56523adad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb33fff640 0 client.7794 ms_handle_reset on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5e366740 1 --2- 172.21.15.19:0/54093012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56523ad92b10 0x56523ae19f50 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5e366740 1 -- 172.21.15.19:0/54093012 >> 172.21.15.19:0/54093012 conn(0x56523ad65770 msgr2=0x56523add17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5e366740 1 -- 172.21.15.19:0/54093012 shutdown_connections 2024-04-25T09:59:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5e366740 1 -- 172.21.15.19:0/54093012 wait complete. 2024-04-25T09:59:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5e366740 20 client.7794 trim_cache size 0 max 16384 2024-04-25T09:59:24.510 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:59:24.510 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:59:24.510 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T09:59:24.510 INFO:teuthology.orchestra.run.smithi019.stdout: "2b92e880-549f-4d1c-b216-53729801a885": { 2024-04-25T09:59:24.510 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:59:24.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "2b92e880-549f-4d1c-b216-53729801a885", 2024-04-25T09:59:24.510 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive" 2024-04-25T09:59:24.510 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:24.510 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:24.510 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:59:24.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 -- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb4006b4d0 msgr2=0x7fbb4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb4006b4d0 0x7fbb4006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009560 tx=0x7fbb4c009190 comp rx=0 tx=0).stop 2024-04-25T09:59:24.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 -- 172.21.15.19:0/4065911346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb540d1f30 msgr2=0x7fbb54154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:24.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb540d1f30 0x7fbb54154910 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c460 tx=0x7fbb4800c920 comp rx=0 tx=0).stop 2024-04-25T09:59:24.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 -- 172.21.15.19:0/4065911346 shutdown_connections 2024-04-25T09:59:24.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb4006b4d0 0x7fbb4006d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb5415dac0 0x7fbb54156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb540d1f30 0x7fbb54154910 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 --2- 172.21.15.19:0/4065911346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb540d0ab0 0x7fbb54152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:24.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 -- 172.21.15.19:0/4065911346 >> 172.21.15.19:0/4065911346 conn(0x7fbb54061c30 msgr2=0x7fbb540ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:24.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 -- 172.21.15.19:0/4065911346 shutdown_connections 2024-04-25T09:59:24.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:24.497+0000 7fbb5b57e640 1 -- 172.21.15.19:0/4065911346 wait complete. 2024-04-25T09:59:29.514 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:59:29.514 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:59:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.625+0000 7f5519b61640 1 Processor -- start 2024-04-25T09:59:29.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.625+0000 7f5519b61640 1 -- start start 2024-04-25T09:59:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5519b61640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551415b7d0 0x7f55140cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5519b61640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f55140b5620 0x7f55140b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5519b61640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551406cf50 con 0x7f551406c2d0 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5519b61640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55140b5fc0 con 0x7f55140b5620 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5519b61640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55140ce530 con 0x7f551415b7d0 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5512ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f55140b5620 0x7f55140b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5513fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f551406c2d0 0x7f551406c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45722/0 (socket says 172.21.15.19:45722) 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5513fff640 1 -- 172.21.15.19:0/1993399230 learned_addr learned my addr 172.21.15.19:0/1993399230 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f55137fe640 1 --2- 172.21.15.19:0/1993399230 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551415b7d0 0x7f55140cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5512ffd640 1 -- 172.21.15.19:0/1993399230 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551415b7d0 msgr2=0x7f55140cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5512ffd640 1 --2- 172.21.15.19:0/1993399230 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551415b7d0 0x7f55140cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5512ffd640 1 -- 172.21.15.19:0/1993399230 >> v1:172.21.15.19:6789/0 conn(0x7f551406c2d0 legacy=0x7f551406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f55127fc640 1 -- 172.21.15.19:0/1993399230 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959036727 0 0) 0x7f551406cf50 con 0x7f551406c2d0 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.626+0000 7f5512ffd640 1 -- 172.21.15.19:0/1993399230 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140ce810 con 0x7f55140b5620 2024-04-25T09:59:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.627+0000 7f5512ffd640 1 --2- 172.21.15.19:0/1993399230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55140b5620 0x7f55140b5a80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f55080097a0 tx=0x7f550802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5fe9ec546f1793 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.627+0000 7f55127fc640 1 -- 172.21.15.19:0/1993399230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508037070 con 0x7f55140b5620 2024-04-25T09:59:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.627+0000 7f55127fc640 1 -- 172.21.15.19:0/1993399230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508032440 con 0x7f55140b5620 2024-04-25T09:59:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.627+0000 7f55127fc640 1 -- 172.21.15.19:0/1993399230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508032810 con 0x7f55140b5620 2024-04-25T09:59:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.628+0000 7f5519b61640 1 -- 172.21.15.19:0/1993399230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55140b5620 msgr2=0x7f55140b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.628+0000 7f5519b61640 1 --2- 172.21.15.19:0/1993399230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55140b5620 0x7f55140b5a80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f55080097a0 tx=0x7f550802a180 comp rx=0 tx=0).stop 2024-04-25T09:59:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.628+0000 7f5519b61640 1 -- 172.21.15.19:0/1993399230 shutdown_connections 2024-04-25T09:59:29.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.628+0000 7f5519b61640 1 --2- 172.21.15.19:0/1993399230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55140b5620 0x7f55140b5a80 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.628+0000 7f5519b61640 1 --2- 172.21.15.19:0/1993399230 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551415b7d0 0x7f55140cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.628+0000 7f5519b61640 1 -- 172.21.15.19:0/1993399230 >> 172.21.15.19:0/1993399230 conn(0x7f55140d0800 msgr2=0x7f55140d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:29.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.628+0000 7f5519b61640 1 -- 172.21.15.19:0/1993399230 shutdown_connections 2024-04-25T09:59:29.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.628+0000 7f5519b61640 1 -- 172.21.15.19:0/1993399230 wait complete. 2024-04-25T09:59:29.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.629+0000 7f5519b61640 1 Processor -- start 2024-04-25T09:59:29.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.629+0000 7f5519b61640 1 -- start start 2024-04-25T09:59:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.630+0000 7f5519b61640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551406c2d0 0x7f55140b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.630+0000 7f5519b61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55140b5620 0x7f55141591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.630+0000 7f5519b61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f551415b7d0 0x7f55140cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.630+0000 7f5519b61640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55140cf550 con 0x7f55140b5620 2024-04-25T09:59:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.630+0000 7f5519b61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551406d230 con 0x7f551415b7d0 2024-04-25T09:59:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.630+0000 7f5519b61640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55140cf6d0 con 0x7f551406c2d0 2024-04-25T09:59:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.630+0000 7f5512ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55140b5620 0x7f55141591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.630+0000 7f5512ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55140b5620 0x7f55141591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46962/0 (socket says 172.21.15.19:46962) 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.630+0000 7f5512ffd640 1 -- 172.21.15.19:0/387975522 learned_addr learned my addr 172.21.15.19:0/387975522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.630+0000 7f55137fe640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551406c2d0 0x7f55140b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.630+0000 7f5513fff640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f551415b7d0 0x7f55140cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.631+0000 7f5512ffd640 1 -- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551406c2d0 msgr2=0x7f55140b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.631+0000 7f5512ffd640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551406c2d0 0x7f55140b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.631+0000 7f5512ffd640 1 -- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f551415b7d0 msgr2=0x7f55140cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.631+0000 7f5512ffd640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f551415b7d0 0x7f55140cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.631+0000 7f5512ffd640 1 -- 172.21.15.19:0/387975522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55141f6800 con 0x7f55140b5620 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.631+0000 7f5512ffd640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55140b5620 0x7f55141591b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5508035040 tx=0x7f55080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.631+0000 7f54fbfff640 1 -- 172.21.15.19:0/387975522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508037040 con 0x7f55140b5620 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.631+0000 7f54fbfff640 1 -- 172.21.15.19:0/387975522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508030770 con 0x7f55140b5620 2024-04-25T09:59:29.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.631+0000 7f5519b61640 1 -- 172.21.15.19:0/387975522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551415b5a0 con 0x7f55140b5620 2024-04-25T09:59:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.631+0000 7f54fbfff640 1 -- 172.21.15.19:0/387975522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508030b40 con 0x7f55140b5620 2024-04-25T09:59:29.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.631+0000 7f5519b61640 1 -- 172.21.15.19:0/387975522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55141f6240 con 0x7f55140b5620 2024-04-25T09:59:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.633+0000 7f54fbfff640 1 -- 172.21.15.19:0/387975522 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5508044020 con 0x7f55140b5620 2024-04-25T09:59:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.633+0000 7f54fbfff640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550006b4d0 0x7f550006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.633+0000 7f54fbfff640 1 -- 172.21.15.19:0/387975522 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55080aa6b0 con 0x7f55140b5620 2024-04-25T09:59:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.633+0000 7f55137fe640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550006b4d0 0x7f550006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.634+0000 7f55137fe640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550006b4d0 0x7f550006d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f55040079a0 tx=0x7f5504008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:29.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.634+0000 7f551c85d740 1 Processor -- start 2024-04-25T09:59:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.634+0000 7f551c85d740 1 -- start start 2024-04-25T09:59:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f551c85d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880b10 0x55fdf7883320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f551c85d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf7883860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f551c85d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7853770 0x55fdf7856f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f551c85d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf77bcda0 con 0x55fdf7880ef0 2024-04-25T09:59:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f551c85d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf7765b80 con 0x55fdf7880b10 2024-04-25T09:59:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f551c85d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdf77b8e40 con 0x55fdf7853770 2024-04-25T09:59:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f5512ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf7883860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f5512ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf7883860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46974/0 (socket says 172.21.15.19:46974) 2024-04-25T09:59:29.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f5512ffd640 1 -- 172.21.15.19:0/28007751 learned_addr learned my addr 172.21.15.19:0/28007751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f55137fe640 1 --2- 172.21.15.19:0/28007751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880b10 0x55fdf7883320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f5513fff640 1 --2- 172.21.15.19:0/28007751 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7853770 0x55fdf7856f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f5512ffd640 1 -- 172.21.15.19:0/28007751 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7853770 msgr2=0x55fdf7856f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f5512ffd640 1 --2- 172.21.15.19:0/28007751 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7853770 0x55fdf7856f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f5512ffd640 1 -- 172.21.15.19:0/28007751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880b10 msgr2=0x55fdf7883320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f5512ffd640 1 --2- 172.21.15.19:0/28007751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880b10 0x55fdf7883320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.635+0000 7f5512ffd640 1 -- 172.21.15.19:0/28007751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdf776ff10 con 0x55fdf7880ef0 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.636+0000 7f5512ffd640 1 --2- 172.21.15.19:0/28007751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf7883860 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5508080f10 tx=0x7f55080751b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b30d7219a4eb91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.636+0000 7f54f97fa640 1 -- 172.21.15.19:0/28007751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508081440 con 0x55fdf7880ef0 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.636+0000 7f54f97fa640 1 -- 172.21.15.19:0/28007751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550807f0b0 con 0x55fdf7880ef0 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.636+0000 7f54f97fa640 1 -- 172.21.15.19:0/28007751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508030ce0 con 0x55fdf7880ef0 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.636+0000 7f551c85d740 1 -- 172.21.15.19:0/28007751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 msgr2=0x55fdf7883860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.636+0000 7f551c85d740 1 --2- 172.21.15.19:0/28007751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf7883860 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5508080f10 tx=0x7f55080751b0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.636+0000 7f551c85d740 1 -- 172.21.15.19:0/28007751 shutdown_connections 2024-04-25T09:59:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.636+0000 7f551c85d740 1 --2- 172.21.15.19:0/28007751 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7853770 0x55fdf7856f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.636+0000 7f551c85d740 1 --2- 172.21.15.19:0/28007751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf7883860 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.636+0000 7f551c85d740 1 --2- 172.21.15.19:0/28007751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880b10 0x55fdf7883320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.637+0000 7f551c85d740 1 -- 172.21.15.19:0/28007751 >> 172.21.15.19:0/28007751 conn(0x55fdf78559e0 msgr2=0x55fdf789e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.637+0000 7f551c85d740 1 -- 172.21.15.19:0/28007751 shutdown_connections 2024-04-25T09:59:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.637+0000 7f551c85d740 1 -- 172.21.15.19:0/28007751 wait complete. 2024-04-25T09:59:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.637+0000 7f551c85d740 1 -- start start 2024-04-25T09:59:29.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.637+0000 7f551c85d740 1 Processor -- start 2024-04-25T09:59:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f551c85d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7880b10 0x55fdf789b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f551c85d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880ef0 0x55fdf78767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f551c85d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7876d30 0x55fdf7854400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f551c85d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf7858b90 con 0x55fdf7876d30 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f551c85d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf775b4b0 con 0x55fdf7880ef0 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f551c85d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdf7858e70 con 0x55fdf7880b10 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f551c85d740 10 client.0 _ll_register_callbacks cb 0x55fdf779b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f5513fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7876d30 0x55fdf7854400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f5513fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7876d30 0x55fdf7854400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46976/0 (socket says 172.21.15.19:46976) 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f5513fff640 1 -- 172.21.15.19:0/396600151 learned_addr learned my addr 172.21.15.19:0/396600151 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f55137fe640 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7880b10 0x55fdf789b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f5512ffd640 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880ef0 0x55fdf78767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.638+0000 7f5513fff640 1 -- 172.21.15.19:0/396600151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7880b10 msgr2=0x55fdf789b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f5513fff640 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7880b10 0x55fdf789b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f5513fff640 1 -- 172.21.15.19:0/396600151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880ef0 msgr2=0x55fdf78767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f5513fff640 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880ef0 0x55fdf78767f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f5513fff640 1 -- 172.21.15.19:0/396600151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdf759b7d0 con 0x55fdf7876d30 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f5513fff640 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7876d30 0x55fdf7854400 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f550c007310 tx=0x7f550c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f15a4f18fe563f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f54f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f54f27fc640 1 -- 172.21.15.19:0/396600151 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c004cf0 con 0x55fdf7876d30 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f54f27fc640 1 -- 172.21.15.19:0/396600151 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f550c0117d0 con 0x55fdf7876d30 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f551c85d740 1 -- 172.21.15.19:0/396600151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f550807f0b0 con 0x55fdf7876d30 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f54f27fc640 1 -- 172.21.15.19:0/396600151 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f550c022050 con 0x55fdf7876d30 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f551c85d740 10 client.7809 fetch_fsmap learned FSMap version 58 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f551c85d740 1 -- 172.21.15.19:0/396600151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55fdf74d13f0 con 0x55fdf7876d30 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f54f27fc640 1 -- 172.21.15.19:0/396600151 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550807f0b0 con 0x55fdf7876d30 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f54f27fc640 1 -- 172.21.15.19:0/396600151 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f550c025020 con 0x55fdf7876d30 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f551c85d740 10 client.7809 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T09:59:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f551c85d740 10 client.7809 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f551c85d740 20 client.7809 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f551c85d740 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x55fdf7898e20 0x55fdf7899230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f551c85d740 4 client.7809 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.639+0000 7f551c85d740 1 -- 172.21.15.19:0/396600151 --> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdf77ca2e0 con 0x55fdf7898e20 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.640+0000 7f55137fe640 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x55fdf7898e20 0x55fdf7899230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.640+0000 7f55137fe640 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x55fdf7898e20 0x55fdf7899230 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62547e2a28fda33f server_cookie=a0f1a1ccaba71ce4 in_seq=0 out_seq=0 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.640+0000 7f54f27fc640 10 client.7809 ms_handle_connect on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.640+0000 7f54f27fc640 1 -- 172.21.15.19:0/396600151 <== mds.0 v2:172.21.15.19:6832/3589967477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fdf77ca2e0 con 0x55fdf7898e20 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.640+0000 7f54f27fc640 10 client.7809 handle_command_reply: tid=0 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.640+0000 7f551c85d740 1 client.7809 shutdown 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f551c85d740 1 -- 172.21.15.19:0/396600151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7876d30 msgr2=0x55fdf7854400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f551c85d740 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7876d30 0x55fdf7854400 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f550c007310 tx=0x7f550c0077d0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f551c85d740 1 -- 172.21.15.19:0/396600151 shutdown_connections 2024-04-25T09:59:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f551c85d740 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x55fdf7898e20 0x55fdf7899230 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f551c85d740 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7876d30 0x55fdf7854400 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f551c85d740 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880ef0 0x55fdf78767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f54f27fc640 0 client.7809 ms_handle_reset on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f551c85d740 1 --2- 172.21.15.19:0/396600151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7880b10 0x55fdf789b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f551c85d740 1 -- 172.21.15.19:0/396600151 >> 172.21.15.19:0/396600151 conn(0x55fdf7853770 msgr2=0x55fdf787fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f551c85d740 1 -- 172.21.15.19:0/396600151 shutdown_connections 2024-04-25T09:59:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f551c85d740 1 -- 172.21.15.19:0/396600151 wait complete. 2024-04-25T09:59:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.641+0000 7f551c85d740 20 client.7809 trim_cache size 0 max 16384 2024-04-25T09:59:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.656+0000 7f551c85d740 1 Processor -- start 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.656+0000 7f551c85d740 1 -- start start 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f551c85d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7876d30 0x55fdf787fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f551c85d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880b10 0x55fdf78801d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f551c85d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf78b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f551c85d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5514200440 con 0x55fdf7880ef0 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f551c85d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55141ff5c0 con 0x55fdf7880b10 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f551c85d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55141f5be0 con 0x55fdf7876d30 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f5513fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf78b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f5513fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf78b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46992/0 (socket says 172.21.15.19:46992) 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f5513fff640 1 -- 172.21.15.19:0/2267924510 learned_addr learned my addr 172.21.15.19:0/2267924510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f5512ffd640 1 --2- 172.21.15.19:0/2267924510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880b10 0x55fdf78801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f55137fe640 1 --2- 172.21.15.19:0/2267924510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7876d30 0x55fdf787fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f5513fff640 1 -- 172.21.15.19:0/2267924510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7876d30 msgr2=0x55fdf787fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f5513fff640 1 --2- 172.21.15.19:0/2267924510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7876d30 0x55fdf787fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f5513fff640 1 -- 172.21.15.19:0/2267924510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880b10 msgr2=0x55fdf78801d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f5513fff640 1 --2- 172.21.15.19:0/2267924510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880b10 0x55fdf78801d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f5513fff640 1 -- 172.21.15.19:0/2267924510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdf74910f0 con 0x55fdf7880ef0 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.657+0000 7f5513fff640 1 --2- 172.21.15.19:0/2267924510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf78b73c0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f550c0072e0 tx=0x7f550c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4f7e6ee6b12f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f54f8ff9640 1 -- 172.21.15.19:0/2267924510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c011820 con 0x55fdf7880ef0 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f54f8ff9640 1 -- 172.21.15.19:0/2267924510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550c01a070 con 0x55fdf7880ef0 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f54f8ff9640 1 -- 172.21.15.19:0/2267924510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c015470 con 0x55fdf7880ef0 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f551c85d740 1 -- 172.21.15.19:0/2267924510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 msgr2=0x55fdf78b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f551c85d740 1 --2- 172.21.15.19:0/2267924510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf78b73c0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f550c0072e0 tx=0x7f550c00c000 comp rx=0 tx=0).stop 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f551c85d740 1 -- 172.21.15.19:0/2267924510 shutdown_connections 2024-04-25T09:59:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f551c85d740 1 --2- 172.21.15.19:0/2267924510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880ef0 0x55fdf78b73c0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f551c85d740 1 --2- 172.21.15.19:0/2267924510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880b10 0x55fdf78801d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f551c85d740 1 --2- 172.21.15.19:0/2267924510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7876d30 0x55fdf787fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f551c85d740 1 -- 172.21.15.19:0/2267924510 >> 172.21.15.19:0/2267924510 conn(0x55fdf7853770 msgr2=0x55fdf78597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:29.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f551c85d740 1 -- 172.21.15.19:0/2267924510 shutdown_connections 2024-04-25T09:59:29.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.658+0000 7f551c85d740 1 -- 172.21.15.19:0/2267924510 wait complete. 2024-04-25T09:59:29.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.659+0000 7f551c85d740 1 -- start start 2024-04-25T09:59:29.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.659+0000 7f551c85d740 1 Processor -- start 2024-04-25T09:59:29.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f551c85d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7876d30 0x55fdf78751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f551c85d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880b10 0x55fdf787a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f551c85d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880ef0 0x55fdf7853b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f551c85d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf7852600 con 0x55fdf7880b10 2024-04-25T09:59:29.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f551c85d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55141feff0 con 0x55fdf7880ef0 2024-04-25T09:59:29.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f551c85d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdf7879690 con 0x55fdf7876d30 2024-04-25T09:59:29.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f551c85d740 10 client.0 _ll_register_callbacks cb 0x7f5508009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:29.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f5512ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880b10 0x55fdf787a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f5512ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880b10 0x55fdf787a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47000/0 (socket says 172.21.15.19:47000) 2024-04-25T09:59:29.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f5512ffd640 1 -- 172.21.15.19:0/844618694 learned_addr learned my addr 172.21.15.19:0/844618694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:29.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f5513fff640 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880ef0 0x55fdf7853b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f55137fe640 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7876d30 0x55fdf78751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f5512ffd640 1 -- 172.21.15.19:0/844618694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7876d30 msgr2=0x55fdf78751c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f5512ffd640 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7876d30 0x55fdf78751c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f5512ffd640 1 -- 172.21.15.19:0/844618694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880ef0 msgr2=0x55fdf7853b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f5512ffd640 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880ef0 0x55fdf7853b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f5512ffd640 1 -- 172.21.15.19:0/844618694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdf7894bd0 con 0x55fdf7880b10 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f5512ffd640 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880b10 0x55fdf787a2b0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f5508000c00 tx=0x7f5508008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25bba387ed181f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f54f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f54f27fc640 1 -- 172.21.15.19:0/844618694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508081440 con 0x55fdf7880b10 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f54f27fc640 1 -- 172.21.15.19:0/844618694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5508030ce0 con 0x55fdf7880b10 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.660+0000 7f551c85d740 1 -- 172.21.15.19:0/844618694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f550c01a070 con 0x55fdf7880b10 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f54f27fc640 1 -- 172.21.15.19:0/844618694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5508075450 con 0x55fdf7880b10 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f551c85d740 10 client.7815 fetch_fsmap learned FSMap version 58 2024-04-25T09:59:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f551c85d740 1 -- 172.21.15.19:0/844618694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55fdf78a1660 con 0x55fdf7880b10 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f54f27fc640 1 -- 172.21.15.19:0/844618694 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550c01a070 con 0x55fdf7880b10 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f54f27fc640 1 -- 172.21.15.19:0/844618694 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f550803a030 con 0x55fdf7880b10 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f551c85d740 10 client.7815 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f551c85d740 10 client.7815 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f551c85d740 20 client.7815 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f551c85d740 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x55fdf790d050 0x55fdf7854090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f551c85d740 4 client.7815 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f551c85d740 1 -- 172.21.15.19:0/844618694 --> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fdf789aac0 con 0x55fdf790d050 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f55137fe640 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x55fdf790d050 0x55fdf7854090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f55137fe640 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x55fdf790d050 0x55fdf7854090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e073763ff6ee16b server_cookie=9b31ebdfe824397c in_seq=0 out_seq=0 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f54f27fc640 10 client.7815 ms_handle_connect on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f54f27fc640 1 -- 172.21.15.19:0/844618694 <== mds.0 v2:172.21.15.19:6832/3589967477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fdf789aac0 con 0x55fdf790d050 2024-04-25T09:59:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f54f27fc640 10 client.7815 handle_command_reply: tid=0 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.661+0000 7f551c85d740 1 client.7815 shutdown 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f551c85d740 1 -- 172.21.15.19:0/844618694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880b10 msgr2=0x55fdf787a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f551c85d740 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880b10 0x55fdf787a2b0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f5508000c00 tx=0x7f5508008070 comp rx=0 tx=0).stop 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f551c85d740 1 -- 172.21.15.19:0/844618694 shutdown_connections 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f551c85d740 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.19:6832/3589967477,v1:172.21.15.19:6835/3589967477] conn(0x55fdf790d050 0x55fdf7854090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f551c85d740 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf7880ef0 0x55fdf7853b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f551c85d740 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fdf7880b10 0x55fdf787a2b0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f551c85d740 1 --2- 172.21.15.19:0/844618694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fdf7876d30 0x55fdf78751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f54f27fc640 0 client.7815 ms_handle_reset on v2:172.21.15.19:6832/3589967477 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f551c85d740 1 -- 172.21.15.19:0/844618694 >> 172.21.15.19:0/844618694 conn(0x55fdf7853770 msgr2=0x55fdf78597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f551c85d740 1 -- 172.21.15.19:0/844618694 shutdown_connections 2024-04-25T09:59:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f551c85d740 1 -- 172.21.15.19:0/844618694 wait complete. 2024-04-25T09:59:29.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f551c85d740 20 client.7815 trim_cache size 0 max 16384 2024-04-25T09:59:29.675 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:59:29.675 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T09:59:29.675 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T09:59:29.675 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:59:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f5519b61640 1 -- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550006b4d0 msgr2=0x7f550006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f5519b61640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550006b4d0 0x7f550006d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f55040079a0 tx=0x7f5504008040 comp rx=0 tx=0).stop 2024-04-25T09:59:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f5519b61640 1 -- 172.21.15.19:0/387975522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55140b5620 msgr2=0x7f55141591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f5519b61640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55140b5620 0x7f55141591b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5508035040 tx=0x7f55080046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f5519b61640 1 -- 172.21.15.19:0/387975522 shutdown_connections 2024-04-25T09:59:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f5519b61640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550006b4d0 0x7f550006d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f5519b61640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f551415b7d0 0x7f55140cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f5519b61640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55140b5620 0x7f55141591b0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f5519b61640 1 --2- 172.21.15.19:0/387975522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551406c2d0 0x7f55140b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.662+0000 7f5519b61640 1 -- 172.21.15.19:0/387975522 >> 172.21.15.19:0/387975522 conn(0x7f55140d0800 msgr2=0x7f551406c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:29.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.663+0000 7f5519b61640 1 -- 172.21.15.19:0/387975522 shutdown_connections 2024-04-25T09:59:29.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.663+0000 7f5519b61640 1 -- 172.21.15.19:0/387975522 wait complete. 2024-04-25T09:59:29.678 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:59:29.678 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:29.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.774+0000 7fe35b688640 1 Processor -- start 2024-04-25T09:59:29.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.774+0000 7fe35b688640 1 -- start start 2024-04-25T09:59:29.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe35b688640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe35415dc90 0x7fe3541625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe35b688640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3540d1810 0x7fe3540d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe35b688640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3540cce50 con 0x7fe3540cc8e0 2024-04-25T09:59:29.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe35b688640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3540d21c0 con 0x7fe35415dc90 2024-04-25T09:59:29.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe35b688640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe354161af0 con 0x7fe3540d1810 2024-04-25T09:59:29.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe359424640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe35415dc90 0x7fe3541625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe359424640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe35415dc90 0x7fe3541625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47014/0 (socket says 172.21.15.19:47014) 2024-04-25T09:59:29.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe359424640 1 -- 172.21.15.19:0/729430757 learned_addr learned my addr 172.21.15.19:0/729430757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:29.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe359c25640 1 --2- 172.21.15.19:0/729430757 >> v2:172.21.15.112:3300/0 conn(0x7fe3540d1810 0x7fe3540d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe359424640 1 -- 172.21.15.19:0/729430757 >> v2:172.21.15.112:3300/0 conn(0x7fe3540d1810 msgr2=0x7fe3540d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:29.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe359424640 1 --2- 172.21.15.19:0/729430757 >> v2:172.21.15.112:3300/0 conn(0x7fe3540d1810 0x7fe3540d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe359424640 1 -- 172.21.15.19:0/729430757 >> v1:172.21.15.112:6789/0 conn(0x7fe3540cc8e0 legacy=0x7fe35415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.775+0000 7fe359424640 1 -- 172.21.15.19:0/729430757 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe354161dd0 con 0x7fe35415dc90 2024-04-25T09:59:29.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.776+0000 7fe359424640 1 --2- 172.21.15.19:0/729430757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe35415dc90 0x7fe3541625d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fe3540cdbb0 tx=0x7fe35002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8a2b3894112601 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:29.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.776+0000 7fe34bfff640 1 -- 172.21.15.19:0/729430757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350037070 con 0x7fe35415dc90 2024-04-25T09:59:29.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.776+0000 7fe34bfff640 1 -- 172.21.15.19:0/729430757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe350032440 con 0x7fe35415dc90 2024-04-25T09:59:29.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.776+0000 7fe34bfff640 1 -- 172.21.15.19:0/729430757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350032810 con 0x7fe35415dc90 2024-04-25T09:59:29.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.777+0000 7fe35b688640 1 -- 172.21.15.19:0/729430757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe35415dc90 msgr2=0x7fe3541625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.777+0000 7fe35b688640 1 --2- 172.21.15.19:0/729430757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe35415dc90 0x7fe3541625d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fe3540cdbb0 tx=0x7fe35002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.777+0000 7fe35b688640 1 -- 172.21.15.19:0/729430757 shutdown_connections 2024-04-25T09:59:29.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.777+0000 7fe35b688640 1 --2- 172.21.15.19:0/729430757 >> v2:172.21.15.112:3300/0 conn(0x7fe3540d1810 0x7fe3540d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.777+0000 7fe35b688640 1 --2- 172.21.15.19:0/729430757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe35415dc90 0x7fe3541625d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.777+0000 7fe35b688640 1 -- 172.21.15.19:0/729430757 >> 172.21.15.19:0/729430757 conn(0x7fe35406f590 msgr2=0x7fe35406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:29.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.777+0000 7fe35b688640 1 -- 172.21.15.19:0/729430757 shutdown_connections 2024-04-25T09:59:29.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.777+0000 7fe35b688640 1 -- 172.21.15.19:0/729430757 wait complete. 2024-04-25T09:59:29.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.778+0000 7fe35b688640 1 Processor -- start 2024-04-25T09:59:29.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.778+0000 7fe35b688640 1 -- start start 2024-04-25T09:59:29.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe35b688640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3540cc8e0 0x7fe354155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe35b688640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3540d1810 0x7fe3540b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe35b688640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe35415dc90 0x7fe35415aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe35b688640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35415e070 con 0x7fe35415dc90 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe35b688640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3540cd130 con 0x7fe3540cc8e0 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe35b688640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35415e1f0 con 0x7fe3540d1810 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe359c25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe35415dc90 0x7fe35415aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe359c25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe35415dc90 0x7fe35415aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47024/0 (socket says 172.21.15.19:47024) 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe359c25640 1 -- 172.21.15.19:0/3400491995 learned_addr learned my addr 172.21.15.19:0/3400491995 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe359424640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3540cc8e0 0x7fe354155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe358c23640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3540d1810 0x7fe3540b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe359c25640 1 -- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3540d1810 msgr2=0x7fe3540b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe359c25640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3540d1810 0x7fe3540b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe359c25640 1 -- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3540cc8e0 msgr2=0x7fe354155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe359c25640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3540cc8e0 0x7fe354155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe359c25640 1 -- 172.21.15.19:0/3400491995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3540b7250 con 0x7fe35415dc90 2024-04-25T09:59:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe358c23640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3540d1810 0x7fe3540b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe359c25640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe35415dc90 0x7fe35415aea0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00efc0 tx=0x7fe34c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe349ffb640 1 -- 172.21.15.19:0/3400491995 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c018020 con 0x7fe35415dc90 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe349ffb640 1 -- 172.21.15.19:0/3400491995 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c009b10 con 0x7fe35415dc90 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.779+0000 7fe35b688640 1 -- 172.21.15.19:0/3400491995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3540b61b0 con 0x7fe35415dc90 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.780+0000 7fe349ffb640 1 -- 172.21.15.19:0/3400491995 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0146c0 con 0x7fe35415dc90 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.780+0000 7fe35b688640 1 -- 172.21.15.19:0/3400491995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3540b6770 con 0x7fe35415dc90 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.781+0000 7fe35b688640 1 -- 172.21.15.19:0/3400491995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe354073990 con 0x7fe35415dc90 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.781+0000 7fe349ffb640 1 -- 172.21.15.19:0/3400491995 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe34c005d10 con 0x7fe35415dc90 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.782+0000 7fe349ffb640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe33806b4d0 0x7fe33806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.782+0000 7fe349ffb640 1 -- 172.21.15.19:0/3400491995 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe34c08afb0 con 0x7fe35415dc90 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.782+0000 7fe359424640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe33806b4d0 0x7fe33806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:29.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.782+0000 7fe359424640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe33806b4d0 0x7fe33806d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe350034000 tx=0x7fe35002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:29.786+0000 7fe349ffb640 1 -- 172.21.15.19:0/3400491995 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe34c022030 con 0x7fe35415dc90 2024-04-25T09:59:30.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.003+0000 7fe35b688640 1 -- 172.21.15.19:0/3400491995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3540cf6b0 con 0x7fe35415dc90 2024-04-25T09:59:30.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.003+0000 7fe349ffb640 1 -- 172.21.15.19:0/3400491995 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe34c010040 con 0x7fe35415dc90 2024-04-25T09:59:30.009 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:30.009 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3431952054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3431952054},{"type":"v1","addr":"172.21.15.112:6836","nonce":3431952054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2758109099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2758109099},{"type":"v1","addr":"172.21.15.112:6837","nonce":2758109099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7590,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3063446116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3063446116},{"type":"v1","addr":"172.21.15.19:6834","nonce":3063446116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:07.729627+0000","modified":"2024-04-25T09:59:09.720756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3589967477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3589967477},{"type":"v1","addr":"172.21.15.19:6835","nonce":3589967477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-04-25T09:59:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 -- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe33806b4d0 msgr2=0x7fe33806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe33806b4d0 0x7fe33806d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe350034000 tx=0x7fe35002e000 comp rx=0 tx=0).stop 2024-04-25T09:59:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 -- 172.21.15.19:0/3400491995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe35415dc90 msgr2=0x7fe35415aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe35415dc90 0x7fe35415aea0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00efc0 tx=0x7fe34c00c490 comp rx=0 tx=0).stop 2024-04-25T09:59:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 -- 172.21.15.19:0/3400491995 shutdown_connections 2024-04-25T09:59:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe33806b4d0 0x7fe33806d990 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe35415dc90 0x7fe35415aea0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3540d1810 0x7fe3540b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 --2- 172.21.15.19:0/3400491995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3540cc8e0 0x7fe354155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 -- 172.21.15.19:0/3400491995 >> 172.21.15.19:0/3400491995 conn(0x7fe35406f590 msgr2=0x7fe3540b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 -- 172.21.15.19:0/3400491995 shutdown_connections 2024-04-25T09:59:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.006+0000 7fe35b688640 1 -- 172.21.15.19:0/3400491995 wait complete. 2024-04-25T09:59:30.021 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:59:30.021 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:59:30.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.116+0000 7f0ac1efc640 1 Processor -- start 2024-04-25T09:59:30.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.116+0000 7f0ac1efc640 1 -- start start 2024-04-25T09:59:30.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.117+0000 7f0ac1efc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc15dbe0 0x7f0abc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.117+0000 7f0ac1efc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0abc0b92a0 0x7f0abc0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.117+0000 7f0ac1efc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc0b97a0 con 0x7f0abc0b2cb0 2024-04-25T09:59:30.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.117+0000 7f0ac1efc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0abc15dfc0 con 0x7f0abc0b92a0 2024-04-25T09:59:30.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.117+0000 7f0ac1efc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0d1a00 con 0x7f0abc15dbe0 2024-04-25T09:59:30.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.117+0000 7f0abaffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0abc0b2cb0 0x7f0abc0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45738/0 (socket says 172.21.15.19:45738) 2024-04-25T09:59:30.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.117+0000 7f0abaffd640 1 -- 172.21.15.19:0/2167965709 learned_addr learned my addr 172.21.15.19:0/2167965709 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0abbfff640 1 --2- 172.21.15.19:0/2167965709 >> v2:172.21.15.112:3300/0 conn(0x7f0abc0b92a0 0x7f0abc0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0abb7fe640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc15dbe0 0x7f0abc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138754145 0 0) 0x7f0abc0b97a0 con 0x7f0abc0b2cb0 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa4003520 con 0x7f0abc0b2cb0 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 588083963 0 0) 0x7f0aa4003520 con 0x7f0abc0b2cb0 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc15dbe0 msgr2=0x7f0abc0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc15dbe0 0x7f0abc0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 >> v2:172.21.15.112:3300/0 conn(0x7f0abc0b92a0 msgr2=0x7f0abc0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 --2- 172.21.15.19:0/2167965709 >> v2:172.21.15.112:3300/0 conn(0x7f0abc0b92a0 0x7f0abc0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc0d1c50 con 0x7f0abc0b2cb0 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0ab0001670 con 0x7f0abc0b2cb0 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 >> v1:172.21.15.19:6789/0 conn(0x7f0abc0b2cb0 legacy=0x7f0abc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc15dbe0 0x7f0aa40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4003f30 0x7f0aa4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa4004d40 0x7f0aa40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0d1a00 con 0x7f0aa4004d40 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa400b9c0 con 0x7f0aa4003f30 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa400bb40 con 0x7f0abc15dbe0 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0abbfff640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa4004d40 0x7f0aa40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0abaffd640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4003f30 0x7f0aa4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0abb7fe640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc15dbe0 0x7f0aa40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0abbfff640 1 -- 172.21.15.19:0/2167965709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc15dbe0 msgr2=0x7f0aa40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0abbfff640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc15dbe0 0x7f0aa40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0abbfff640 1 -- 172.21.15.19:0/2167965709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4003f30 msgr2=0x7f0aa4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0abbfff640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4003f30 0x7f0aa4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.118+0000 7f0abbfff640 1 -- 172.21.15.19:0/2167965709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa4003520 con 0x7f0aa4004d40 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.119+0000 7f0abbfff640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa4004d40 0x7f0aa40051c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0b25c0 tx=0x7f0aa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ba0e0788f79335 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.119+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4009c10 con 0x7f0aa4004d40 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.119+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab4036070 con 0x7f0aa4004d40 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.119+0000 7f0aba7fc640 1 -- 172.21.15.19:0/2167965709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4031630 con 0x7f0aa4004d40 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.120+0000 7f0ac1efc640 1 -- 172.21.15.19:0/2167965709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa4004d40 msgr2=0x7f0aa40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.120+0000 7f0ac1efc640 1 --2- 172.21.15.19:0/2167965709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0aa4004d40 0x7f0aa40051c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0b25c0 tx=0x7f0aa4005830 comp rx=0 tx=0).stop 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.120+0000 7f0abbfff640 1 -- 172.21.15.19:0/2167965709 reap_dead start 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.120+0000 7f0ac1efc640 1 -- 172.21.15.19:0/2167965709 shutdown_connections 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.120+0000 7f0ac1efc640 1 -- 172.21.15.19:0/2167965709 >> 172.21.15.19:0/2167965709 conn(0x7f0abc061a20 msgr2=0x7f0abc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.120+0000 7f0ac1efc640 1 -- 172.21.15.19:0/2167965709 shutdown_connections 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.120+0000 7f0ac1efc640 1 -- 172.21.15.19:0/2167965709 wait complete. 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.121+0000 7f0ac1efc640 1 Processor -- start 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.121+0000 7f0ac1efc640 1 -- start start 2024-04-25T09:59:30.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0ac1efc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4004d40 0x7f0abc15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0ac1efc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc0caa50 0x7f0abc0cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0ac1efc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc0c94d0 0x7f0abc0b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0ac1efc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0b6ab0 con 0x7f0abc0caa50 2024-04-25T09:59:30.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0ac1efc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0d2210 con 0x7f0aa4004d40 2024-04-25T09:59:30.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0ac1efc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0b6c30 con 0x7f0abc0c94d0 2024-04-25T09:59:30.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0abaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc0caa50 0x7f0abc0cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0abaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc0caa50 0x7f0abc0cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47040/0 (socket says 172.21.15.19:47040) 2024-04-25T09:59:30.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0abaffd640 1 -- 172.21.15.19:0/1696324569 learned_addr learned my addr 172.21.15.19:0/1696324569 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:30.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0abb7fe640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4004d40 0x7f0abc15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0abbfff640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc0c94d0 0x7f0abc0b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0abaffd640 1 -- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc0c94d0 msgr2=0x7f0abc0b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0abaffd640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc0c94d0 0x7f0abc0b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0abaffd640 1 -- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4004d40 msgr2=0x7f0abc15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0abaffd640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4004d40 0x7f0abc15eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.122+0000 7f0abaffd640 1 -- 172.21.15.19:0/1696324569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc0caeb0 con 0x7f0abc0caa50 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.123+0000 7f0abaffd640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc0caa50 0x7f0abc0cc220 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007310 tx=0x7f0ab0007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.123+0000 7f0aa3fff640 1 -- 172.21.15.19:0/1696324569 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000f9c0 con 0x7f0abc0caa50 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.123+0000 7f0aa3fff640 1 -- 172.21.15.19:0/1696324569 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab0018070 con 0x7f0abc0caa50 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.123+0000 7f0ac1efc640 1 -- 172.21.15.19:0/1696324569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc0cb180 con 0x7f0abc0caa50 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.123+0000 7f0aa3fff640 1 -- 172.21.15.19:0/1696324569 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000fda0 con 0x7f0abc0caa50 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.123+0000 7f0ac1efc640 1 -- 172.21.15.19:0/1696324569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc0cb740 con 0x7f0abc0caa50 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.124+0000 7f0ac1efc640 1 -- 172.21.15.19:0/1696324569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0abc073990 con 0x7f0abc0caa50 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.124+0000 7f0aa3fff640 1 -- 172.21.15.19:0/1696324569 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ab001d460 con 0x7f0abc0caa50 2024-04-25T09:59:30.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.125+0000 7f0aa3fff640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0aa806b4d0 0x7f0aa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.125+0000 7f0aa3fff640 1 -- 172.21.15.19:0/1696324569 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ab008a550 con 0x7f0abc0caa50 2024-04-25T09:59:30.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.125+0000 7f0abb7fe640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0aa806b4d0 0x7f0aa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.125+0000 7f0abb7fe640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0aa806b4d0 0x7f0aa806d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009bc0 tx=0x7f0aac009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:30.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.129+0000 7f0aa3fff640 1 -- 172.21.15.19:0/1696324569 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ab00591a0 con 0x7f0abc0caa50 2024-04-25T09:59:30.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.380+0000 7f0ac1efc640 1 -- 172.21.15.19:0/1696324569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0abc15ad50 con 0x7f0abc0caa50 2024-04-25T09:59:30.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.380+0000 7f0aa3fff640 1 -- 172.21.15.19:0/1696324569 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1026 (secure 0 0 0) 0x7f0ab0058aa0 con 0x7f0abc0caa50 2024-04-25T09:59:30.385 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:30.386 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":407,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":95,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2523136000,"bytes_avail":770570977280,"bytes_total":773094113280,"write_bytes_sec":89656690,"read_op_per_sec":0,"write_op_per_sec":87},"fsmap":{"epoch":58,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7593}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-25T09:58:56.991196+0000","services":{}},"progress_events":{}} 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.382+0000 7f0ac1efc640 1 -- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0aa806b4d0 msgr2=0x7f0aa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.382+0000 7f0ac1efc640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0aa806b4d0 0x7f0aa806d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009bc0 tx=0x7f0aac009190 comp rx=0 tx=0).stop 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.382+0000 7f0ac1efc640 1 -- 172.21.15.19:0/1696324569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc0caa50 msgr2=0x7f0abc0cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.382+0000 7f0ac1efc640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc0caa50 0x7f0abc0cc220 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007310 tx=0x7f0ab0007710 comp rx=0 tx=0).stop 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.382+0000 7f0ac1efc640 1 -- 172.21.15.19:0/1696324569 shutdown_connections 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.382+0000 7f0ac1efc640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0aa806b4d0 0x7f0aa806d990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.382+0000 7f0ac1efc640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0abc0c94d0 0x7f0abc0b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.383+0000 7f0ac1efc640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc0caa50 0x7f0abc0cc220 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.383+0000 7f0ac1efc640 1 --2- 172.21.15.19:0/1696324569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4004d40 0x7f0abc15eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.383+0000 7f0ac1efc640 1 -- 172.21.15.19:0/1696324569 >> 172.21.15.19:0/1696324569 conn(0x7f0abc061a20 msgr2=0x7f0abc0d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.383+0000 7f0ac1efc640 1 -- 172.21.15.19:0/1696324569 shutdown_connections 2024-04-25T09:59:30.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.383+0000 7f0ac1efc640 1 -- 172.21.15.19:0/1696324569 wait complete. 2024-04-25T09:59:30.398 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:59:30.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:59:30.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:59:30.481 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:59:30.482 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:59:30.482 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:30.497 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:59:30.498 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:30.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:30.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.690+0000 7fa6ffafc640 1 Processor -- start 2024-04-25T09:59:30.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.690+0000 7fa6ffafc640 1 -- start start 2024-04-25T09:59:30.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.691+0000 7fa6ffafc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa6f80b5c30 0x7fa6f8168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.691+0000 7fa6ffafc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f80ceee0 con 0x7fa6f815db70 2024-04-25T09:59:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.691+0000 7fa6ffafc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f80d1d10 con 0x7fa6f80b6cb0 2024-04-25T09:59:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.691+0000 7fa6ffafc640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80d3800 con 0x7fa6f80b5c30 2024-04-25T09:59:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.691+0000 7fa6fe099640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa6f80b5c30 0x7fa6f8168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.691+0000 7fa6fd898640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa6f815db70 0x7fa6f8162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45748/0 (socket says 172.21.15.19:45748) 2024-04-25T09:59:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.691+0000 7fa6fd898640 1 -- 172.21.15.19:0/659844715 learned_addr learned my addr 172.21.15.19:0/659844715 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.692+0000 7fa6fe099640 1 -- 172.21.15.19:0/659844715 >> v1:172.21.15.112:6789/0 conn(0x7fa6f80b6cb0 legacy=0x7fa6f80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.692+0000 7fa6fe099640 1 -- 172.21.15.19:0/659844715 >> v1:172.21.15.19:6789/0 conn(0x7fa6f815db70 legacy=0x7fa6f8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.692+0000 7fa6fc896640 1 -- 172.21.15.19:0/659844715 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33921415 0 0) 0x7fa6f80ceee0 con 0x7fa6f815db70 2024-04-25T09:59:30.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.692+0000 7fa6fe099640 1 -- 172.21.15.19:0/659844715 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80d1ff0 con 0x7fa6f80b5c30 2024-04-25T09:59:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.692+0000 7fa6fe099640 1 --2- 172.21.15.19:0/659844715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6f80b5c30 0x7fa6f8168ad0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80d3980 tx=0x7fa6f402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf370d0111d965 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.692+0000 7fa6fc896640 1 -- 172.21.15.19:0/659844715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4007740 con 0x7fa6f80b5c30 2024-04-25T09:59:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.692+0000 7fa6fc896640 1 -- 172.21.15.19:0/659844715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40078e0 con 0x7fa6f80b5c30 2024-04-25T09:59:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.692+0000 7fa6fc896640 1 -- 172.21.15.19:0/659844715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4007cb0 con 0x7fa6f80b5c30 2024-04-25T09:59:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.693+0000 7fa6ffafc640 1 -- 172.21.15.19:0/659844715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6f80b5c30 msgr2=0x7fa6f8168ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.693+0000 7fa6ffafc640 1 --2- 172.21.15.19:0/659844715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6f80b5c30 0x7fa6f8168ad0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80d3980 tx=0x7fa6f402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.693+0000 7fa6ffafc640 1 -- 172.21.15.19:0/659844715 shutdown_connections 2024-04-25T09:59:30.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.693+0000 7fa6ffafc640 1 --2- 172.21.15.19:0/659844715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6f80b5c30 0x7fa6f8168ad0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.693+0000 7fa6ffafc640 1 -- 172.21.15.19:0/659844715 >> 172.21.15.19:0/659844715 conn(0x7fa6f80d0520 msgr2=0x7fa6f80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:30.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.693+0000 7fa6ffafc640 1 -- 172.21.15.19:0/659844715 shutdown_connections 2024-04-25T09:59:30.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.693+0000 7fa6ffafc640 1 -- 172.21.15.19:0/659844715 wait complete. 2024-04-25T09:59:30.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.694+0000 7fa6ffafc640 1 Processor -- start 2024-04-25T09:59:30.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.694+0000 7fa6ffafc640 1 -- start start 2024-04-25T09:59:30.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6ffafc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80b5c30 0x7fa6f8154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6ffafc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6f80b6cb0 0x7fa6f81511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6ffafc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6f815db70 0x7fa6f81582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6ffafc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80ce530 con 0x7fa6f815db70 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6ffafc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f815df50 con 0x7fa6f80b5c30 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6ffafc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80cf1c0 con 0x7fa6f80b6cb0 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6fe099640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6f815db70 0x7fa6f81582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6fe099640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6f815db70 0x7fa6f81582d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47056/0 (socket says 172.21.15.19:47056) 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6fe099640 1 -- 172.21.15.19:0/2549122416 learned_addr learned my addr 172.21.15.19:0/2549122416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6fd898640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80b5c30 0x7fa6f8154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6fe099640 1 -- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6f80b6cb0 msgr2=0x7fa6f81511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6fe099640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6f80b6cb0 0x7fa6f81511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6fe099640 1 -- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80b5c30 msgr2=0x7fa6f8154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6fe099640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80b5c30 0x7fa6f8154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.695+0000 7fa6fe099640 1 -- 172.21.15.19:0/2549122416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f8158b20 con 0x7fa6f815db70 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.696+0000 7fa6fe099640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6f815db70 0x7fa6f81582d0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009e80 tx=0x7fa6f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.696+0000 7fa6ee7fc640 1 -- 172.21.15.19:0/2549122416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4040070 con 0x7fa6f815db70 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.696+0000 7fa6ee7fc640 1 -- 172.21.15.19:0/2549122416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f4031c40 con 0x7fa6f815db70 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.696+0000 7fa6ee7fc640 1 -- 172.21.15.19:0/2549122416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4003990 con 0x7fa6f815db70 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.696+0000 7fa6ffafc640 1 -- 172.21.15.19:0/2549122416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f8158e00 con 0x7fa6f815db70 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.696+0000 7fa6ffafc640 1 -- 172.21.15.19:0/2549122416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f8158810 con 0x7fa6f815db70 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.697+0000 7fa6ffafc640 1 -- 172.21.15.19:0/2549122416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f815f6b0 con 0x7fa6f815db70 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.697+0000 7fa6ee7fc640 1 -- 172.21.15.19:0/2549122416 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6f4039040 con 0x7fa6f815db70 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.698+0000 7fa6ee7fc640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6dc06b4d0 0x7fa6dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:30.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.698+0000 7fa6ee7fc640 1 -- 172.21.15.19:0/2549122416 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa6f40ab3f0 con 0x7fa6f815db70 2024-04-25T09:59:30.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.698+0000 7fa6fd898640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6dc06b4d0 0x7fa6dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:30.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.699+0000 7fa6fd898640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6dc06b4d0 0x7fa6dc06d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0000f60 tx=0x7fa6f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:30.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.702+0000 7fa6ee7fc640 1 -- 172.21.15.19:0/2549122416 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6f407a0d0 con 0x7fa6f815db70 2024-04-25T09:59:30.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.881+0000 7fa6ffafc640 1 -- 172.21.15.19:0/2549122416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6f8073990 con 0x7fa6f815db70 2024-04-25T09:59:30.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.882+0000 7fa6ee7fc640 1 -- 172.21.15.19:0/2549122416 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16873 (secure 0 0 0) 0x7fa6f40799d0 con 0x7fa6f815db70 2024-04-25T09:59:30.887 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:30.888 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":95,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:09.369855+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:06.008725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:06.705899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:30.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 -- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6dc06b4d0 msgr2=0x7fa6dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6dc06b4d0 0x7fa6dc06d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0000f60 tx=0x7fa6f0009040 comp rx=0 tx=0).stop 2024-04-25T09:59:30.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 -- 172.21.15.19:0/2549122416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6f815db70 msgr2=0x7fa6f81582d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:30.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6f815db70 0x7fa6f81582d0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009e80 tx=0x7fa6f4004a40 comp rx=0 tx=0).stop 2024-04-25T09:59:30.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 -- 172.21.15.19:0/2549122416 shutdown_connections 2024-04-25T09:59:30.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6dc06b4d0 0x7fa6dc06d990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6f815db70 0x7fa6f81582d0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6f80b6cb0 0x7fa6f81511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 --2- 172.21.15.19:0/2549122416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6f80b5c30 0x7fa6f8154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:30.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 -- 172.21.15.19:0/2549122416 >> 172.21.15.19:0/2549122416 conn(0x7fa6f80d0520 msgr2=0x7fa6f80b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:30.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 -- 172.21.15.19:0/2549122416 shutdown_connections 2024-04-25T09:59:30.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:30.884+0000 7fa6ffafc640 1 -- 172.21.15.19:0/2549122416 wait complete. 2024-04-25T09:59:30.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:59:06.008725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:59:06.705899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:59:30.901 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:59:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.000+0000 7f4d57053640 1 Processor -- start 2024-04-25T09:59:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.000+0000 7f4d57053640 1 -- start start 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d57053640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d5015a030 0x7f4d5006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d57053640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4d5015dc20 0x7f4d5015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d57053640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5006f9d0 con 0x7f4d5006ef30 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d57053640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d500ada20 con 0x7f4d5015dc20 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d57053640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500ae260 con 0x7f4d5015a030 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d555f0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4d5015dc20 0x7f4d5015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d555f0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4d5015dc20 0x7f4d5015e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47068/0 (socket says 172.21.15.19:47068) 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d555f0640 1 -- 172.21.15.19:0/2810843695 learned_addr learned my addr 172.21.15.19:0/2810843695 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d54def640 1 --2- 172.21.15.19:0/2810843695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d5015a030 0x7f4d5006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d555f0640 1 -- 172.21.15.19:0/2810843695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d5015a030 msgr2=0x7f4d5006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d555f0640 1 --2- 172.21.15.19:0/2810843695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d5015a030 0x7f4d5006abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d555f0640 1 -- 172.21.15.19:0/2810843695 >> v1:172.21.15.112:6789/0 conn(0x7f4d5006ef30 legacy=0x7f4d5006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.001+0000 7f4d555f0640 1 -- 172.21.15.19:0/2810843695 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d500adc50 con 0x7f4d5015dc20 2024-04-25T09:59:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.002+0000 7f4d555f0640 1 --2- 172.21.15.19:0/2810843695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5015dc20 0x7f4d5015e0b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800d150 tx=0x7f4d4802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa837b5c923a033 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.002+0000 7f4d4f7fe640 1 -- 172.21.15.19:0/2810843695 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48039070 con 0x7f4d5015dc20 2024-04-25T09:59:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.002+0000 7f4d4f7fe640 1 -- 172.21.15.19:0/2810843695 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d48009600 con 0x7f4d5015dc20 2024-04-25T09:59:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.002+0000 7f4d4f7fe640 1 -- 172.21.15.19:0/2810843695 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d480099d0 con 0x7f4d5015dc20 2024-04-25T09:59:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.003+0000 7f4d57053640 1 -- 172.21.15.19:0/2810843695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5015dc20 msgr2=0x7f4d5015e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.003+0000 7f4d57053640 1 --2- 172.21.15.19:0/2810843695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5015dc20 0x7f4d5015e0b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800d150 tx=0x7f4d4802c890 comp rx=0 tx=0).stop 2024-04-25T09:59:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.003+0000 7f4d57053640 1 -- 172.21.15.19:0/2810843695 shutdown_connections 2024-04-25T09:59:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.003+0000 7f4d57053640 1 --2- 172.21.15.19:0/2810843695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5015dc20 0x7f4d5015e0b0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.003+0000 7f4d57053640 1 --2- 172.21.15.19:0/2810843695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d5015a030 0x7f4d5006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.003+0000 7f4d57053640 1 -- 172.21.15.19:0/2810843695 >> 172.21.15.19:0/2810843695 conn(0x7f4d500cfec0 msgr2=0x7f4d500d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.003+0000 7f4d57053640 1 -- 172.21.15.19:0/2810843695 shutdown_connections 2024-04-25T09:59:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.003+0000 7f4d57053640 1 -- 172.21.15.19:0/2810843695 wait complete. 2024-04-25T09:59:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.004+0000 7f4d57053640 1 Processor -- start 2024-04-25T09:59:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.004+0000 7f4d57053640 1 -- start start 2024-04-25T09:59:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d57053640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d5006ef30 0x7f4d501466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d57053640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5015a030 0x7f4d500b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d57053640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d5015dc20 0x7f4d500ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d57053640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500adef0 con 0x7f4d5015a030 2024-04-25T09:59:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d57053640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5006f6f0 con 0x7f4d5015dc20 2024-04-25T09:59:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d57053640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5015a410 con 0x7f4d5006ef30 2024-04-25T09:59:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d4ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5015a030 0x7f4d500b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d4ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5015a030 0x7f4d500b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47078/0 (socket says 172.21.15.19:47078) 2024-04-25T09:59:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d4ffff640 1 -- 172.21.15.19:0/1145761431 learned_addr learned my addr 172.21.15.19:0/1145761431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d54def640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d5006ef30 0x7f4d501466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d555f0640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d5015dc20 0x7f4d500ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d4ffff640 1 -- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d5006ef30 msgr2=0x7f4d501466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d4ffff640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d5006ef30 0x7f4d501466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d4ffff640 1 -- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d5015dc20 msgr2=0x7f4d500ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d4ffff640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d5015dc20 0x7f4d500ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d4ffff640 1 -- 172.21.15.19:0/1145761431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d50146c30 con 0x7f4d5015a030 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.005+0000 7f4d4ffff640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5015a030 0x7f4d500b0400 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00c440 tx=0x7f4d3c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.006+0000 7f4d4d7fa640 1 -- 172.21.15.19:0/1145761431 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0052d0 con 0x7f4d5015a030 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.006+0000 7f4d4d7fa640 1 -- 172.21.15.19:0/1145761431 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3c005470 con 0x7f4d5015a030 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.006+0000 7f4d57053640 1 -- 172.21.15.19:0/1145761431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d500b0d30 con 0x7f4d5015a030 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.006+0000 7f4d4d7fa640 1 -- 172.21.15.19:0/1145761431 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c007240 con 0x7f4d5015a030 2024-04-25T09:59:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.006+0000 7f4d57053640 1 -- 172.21.15.19:0/1145761431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d500b12f0 con 0x7f4d5015a030 2024-04-25T09:59:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.007+0000 7f4d57053640 1 -- 172.21.15.19:0/1145761431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d500b1630 con 0x7f4d5015a030 2024-04-25T09:59:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.007+0000 7f4d4d7fa640 1 -- 172.21.15.19:0/1145761431 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d3c0073e0 con 0x7f4d5015a030 2024-04-25T09:59:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.008+0000 7f4d4d7fa640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d3406b4d0 0x7f4d3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.008+0000 7f4d4d7fa640 1 -- 172.21.15.19:0/1145761431 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d3c0896f0 con 0x7f4d5015a030 2024-04-25T09:59:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.008+0000 7f4d54def640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d3406b4d0 0x7f4d3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.008+0000 7f4d54def640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d3406b4d0 0x7f4d3406d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009b40 tx=0x7f4d44009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.012+0000 7f4d4d7fa640 1 -- 172.21.15.19:0/1145761431 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d500b1630 con 0x7f4d5015a030 2024-04-25T09:59:31.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.190+0000 7f4d57053640 1 -- 172.21.15.19:0/1145761431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d50073a00 con 0x7f4d5015a030 2024-04-25T09:59:31.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.190+0000 7f4d4d7fa640 1 -- 172.21.15.19:0/1145761431 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d3c057e20 con 0x7f4d5015a030 2024-04-25T09:59:31.195 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:59:31.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 -- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d3406b4d0 msgr2=0x7f4d3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d3406b4d0 0x7f4d3406d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009b40 tx=0x7f4d44009190 comp rx=0 tx=0).stop 2024-04-25T09:59:31.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 -- 172.21.15.19:0/1145761431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5015a030 msgr2=0x7f4d500b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5015a030 0x7f4d500b0400 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00c440 tx=0x7f4d3c001e70 comp rx=0 tx=0).stop 2024-04-25T09:59:31.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 -- 172.21.15.19:0/1145761431 shutdown_connections 2024-04-25T09:59:31.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d3406b4d0 0x7f4d3406d990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d5015dc20 0x7f4d500ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5015a030 0x7f4d500b0400 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 --2- 172.21.15.19:0/1145761431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d5006ef30 0x7f4d501466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 -- 172.21.15.19:0/1145761431 >> 172.21.15.19:0/1145761431 conn(0x7f4d500cfec0 msgr2=0x7f4d500b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:31.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 -- 172.21.15.19:0/1145761431 shutdown_connections 2024-04-25T09:59:31.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.192+0000 7f4d57053640 1 -- 172.21.15.19:0/1145761431 wait complete. 2024-04-25T09:59:31.208 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:59:31.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.306+0000 7fe93d89e640 1 Processor -- start 2024-04-25T09:59:31.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.306+0000 7fe93d89e640 1 -- start start 2024-04-25T09:59:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.307+0000 7fe93d89e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe9380ce7b0 0x7fe938161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.307+0000 7fe93d89e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe93806c8d0 con 0x7fe9381616f0 2024-04-25T09:59:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.307+0000 7fe93d89e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9380c6960 con 0x7fe9381610f0 2024-04-25T09:59:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.307+0000 7fe93d89e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9381614d0 con 0x7fe9380ce7b0 2024-04-25T09:59:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.307+0000 7fe9367fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe9381616f0 0x7fe9380ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45762/0 (socket says 172.21.15.19:45762) 2024-04-25T09:59:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.307+0000 7fe9367fc640 1 -- 172.21.15.19:0/1335321111 learned_addr learned my addr 172.21.15.19:0/1335321111 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.308+0000 7fe9377fe640 1 --2- 172.21.15.19:0/1335321111 >> v2:172.21.15.112:3300/0 conn(0x7fe9380ce7b0 0x7fe938161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.308+0000 7fe935ffb640 1 -- 172.21.15.19:0/1335321111 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2851040803 0 0) 0x7fe93806c8d0 con 0x7fe9381616f0 2024-04-25T09:59:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.308+0000 7fe935ffb640 1 -- 172.21.15.19:0/1335321111 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe924003520 con 0x7fe9381616f0 2024-04-25T09:59:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.308+0000 7fe935ffb640 1 -- 172.21.15.19:0/1335321111 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 381865754 0 0) 0x7fe924003520 con 0x7fe9381616f0 2024-04-25T09:59:31.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.308+0000 7fe9377fe640 1 -- 172.21.15.19:0/1335321111 >> v1:172.21.15.112:6789/0 conn(0x7fe9381610f0 legacy=0x7fe93806c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.308+0000 7fe9377fe640 1 -- 172.21.15.19:0/1335321111 >> v1:172.21.15.19:6789/0 conn(0x7fe9381616f0 legacy=0x7fe9380ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.308+0000 7fe9377fe640 1 -- 172.21.15.19:0/1335321111 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe938162260 con 0x7fe9380ce7b0 2024-04-25T09:59:31.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.309+0000 7fe9377fe640 1 --2- 172.21.15.19:0/1335321111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380ce7b0 0x7fe938161b50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe9300097a0 tx=0x7fe93002a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637f000abfbd10d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.309+0000 7fe935ffb640 1 -- 172.21.15.19:0/1335321111 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930037070 con 0x7fe9380ce7b0 2024-04-25T09:59:31.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.309+0000 7fe935ffb640 1 -- 172.21.15.19:0/1335321111 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93002ab60 con 0x7fe9380ce7b0 2024-04-25T09:59:31.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.309+0000 7fe935ffb640 1 -- 172.21.15.19:0/1335321111 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920001670 con 0x7fe9380ce7b0 2024-04-25T09:59:31.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.309+0000 7fe93d89e640 1 -- 172.21.15.19:0/1335321111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380ce7b0 msgr2=0x7fe938161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.309+0000 7fe93d89e640 1 --2- 172.21.15.19:0/1335321111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380ce7b0 0x7fe938161b50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe9300097a0 tx=0x7fe93002a0d0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.310+0000 7fe93d89e640 1 -- 172.21.15.19:0/1335321111 shutdown_connections 2024-04-25T09:59:31.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.310+0000 7fe93d89e640 1 --2- 172.21.15.19:0/1335321111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380ce7b0 0x7fe938161b50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.310+0000 7fe93d89e640 1 -- 172.21.15.19:0/1335321111 >> 172.21.15.19:0/1335321111 conn(0x7fe93806f500 msgr2=0x7fe93806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:31.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.310+0000 7fe93d89e640 1 -- 172.21.15.19:0/1335321111 shutdown_connections 2024-04-25T09:59:31.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.310+0000 7fe93d89e640 1 -- 172.21.15.19:0/1335321111 wait complete. 2024-04-25T09:59:31.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.311+0000 7fe93d89e640 1 Processor -- start 2024-04-25T09:59:31.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.311+0000 7fe93d89e640 1 -- start start 2024-04-25T09:59:31.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe93d89e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9380ce7b0 0x7fe93815fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe93d89e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9381610f0 0x7fe9380af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe93d89e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9381616f0 0x7fe9380acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe93d89e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9380cfce0 con 0x7fe9381610f0 2024-04-25T09:59:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe93d89e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9380cfa00 con 0x7fe9381616f0 2024-04-25T09:59:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe93d89e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9380cf690 con 0x7fe9380ce7b0 2024-04-25T09:59:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe9367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9381610f0 0x7fe9380af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe9367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9381610f0 0x7fe9380af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47084/0 (socket says 172.21.15.19:47084) 2024-04-25T09:59:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe9367fc640 1 -- 172.21.15.19:0/1096369495 learned_addr learned my addr 172.21.15.19:0/1096369495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe9377fe640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9381616f0 0x7fe9380acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe936ffd640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9380ce7b0 0x7fe93815fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe9367fc640 1 -- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9380ce7b0 msgr2=0x7fe93815fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe9367fc640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9380ce7b0 0x7fe93815fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe9367fc640 1 -- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9381616f0 msgr2=0x7fe9380acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe9367fc640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9381616f0 0x7fe9380acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe9367fc640 1 -- 172.21.15.19:0/1096369495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9380afe70 con 0x7fe9381610f0 2024-04-25T09:59:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe9367fc640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9381610f0 0x7fe9380af440 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fe92000c780 tx=0x7fe92000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe92ffff640 1 -- 172.21.15.19:0/1096369495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920018070 con 0x7fe9381610f0 2024-04-25T09:59:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe92ffff640 1 -- 172.21.15.19:0/1096369495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9200072d0 con 0x7fe9381610f0 2024-04-25T09:59:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe93d89e640 1 -- 172.21.15.19:0/1096369495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9380b0150 con 0x7fe9381610f0 2024-04-25T09:59:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe92ffff640 1 -- 172.21.15.19:0/1096369495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200047e0 con 0x7fe9381610f0 2024-04-25T09:59:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.312+0000 7fe93d89e640 1 -- 172.21.15.19:0/1096369495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9380b0710 con 0x7fe9381610f0 2024-04-25T09:59:31.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.313+0000 7fe93d89e640 1 -- 172.21.15.19:0/1096369495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9380b0320 con 0x7fe9381610f0 2024-04-25T09:59:31.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.314+0000 7fe92ffff640 1 -- 172.21.15.19:0/1096369495 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe92001b070 con 0x7fe9381610f0 2024-04-25T09:59:31.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.314+0000 7fe92ffff640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe91806b4d0 0x7fe91806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.315+0000 7fe92ffff640 1 -- 172.21.15.19:0/1096369495 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe920089800 con 0x7fe9381610f0 2024-04-25T09:59:31.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.315+0000 7fe936ffd640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe91806b4d0 0x7fe91806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.315+0000 7fe936ffd640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe91806b4d0 0x7fe91806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe928007fd0 tx=0x7fe928009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.319+0000 7fe92ffff640 1 -- 172.21.15.19:0/1096369495 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9200584f0 con 0x7fe9381610f0 2024-04-25T09:59:31.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.499+0000 7fe93d89e640 1 -- 172.21.15.19:0/1096369495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9380cef40 con 0x7fe9381610f0 2024-04-25T09:59:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.499+0000 7fe92ffff640 1 -- 172.21.15.19:0/1096369495 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9380cef40 con 0x7fe9381610f0 2024-04-25T09:59:31.504 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:59:31.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 -- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe91806b4d0 msgr2=0x7fe91806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe91806b4d0 0x7fe91806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe928007fd0 tx=0x7fe928009040 comp rx=0 tx=0).stop 2024-04-25T09:59:31.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 -- 172.21.15.19:0/1096369495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9381610f0 msgr2=0x7fe9380af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9381610f0 0x7fe9380af440 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fe92000c780 tx=0x7fe92000cc40 comp rx=0 tx=0).stop 2024-04-25T09:59:31.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 -- 172.21.15.19:0/1096369495 shutdown_connections 2024-04-25T09:59:31.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe91806b4d0 0x7fe91806d990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9381616f0 0x7fe9380acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9381610f0 0x7fe9380af440 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 --2- 172.21.15.19:0/1096369495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9380ce7b0 0x7fe93815fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 -- 172.21.15.19:0/1096369495 >> 172.21.15.19:0/1096369495 conn(0x7fe93806f500 msgr2=0x7fe938071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:31.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 -- 172.21.15.19:0/1096369495 shutdown_connections 2024-04-25T09:59:31.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.501+0000 7fe93d89e640 1 -- 172.21.15.19:0/1096369495 wait complete. 2024-04-25T09:59:31.517 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:59:31.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.615+0000 7f1d9806f640 1 Processor -- start 2024-04-25T09:59:31.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.616+0000 7f1d9806f640 1 -- start start 2024-04-25T09:59:31.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.616+0000 7f1d9806f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1d900d0ab0 0x7f1d90168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.616+0000 7f1d9806f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d900d2420 con 0x7f1d900d34c0 2024-04-25T09:59:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.616+0000 7f1d9806f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d900d2700 con 0x7f1d9015dac0 2024-04-25T09:59:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.616+0000 7f1d9806f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d900d3030 con 0x7f1d900d0ab0 2024-04-25T09:59:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.617+0000 7f1d9660c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1d900d0ab0 0x7f1d90168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.617+0000 7f1d9560a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1d900d34c0 0x7f1d900d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45768/0 (socket says 172.21.15.19:45768) 2024-04-25T09:59:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.617+0000 7f1d9660c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1d900d0ab0 0x7f1d90168670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47096/0 (socket says 172.21.15.19:47096) 2024-04-25T09:59:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.617+0000 7f1d9560a640 1 -- 172.21.15.19:0/1339619191 learned_addr learned my addr 172.21.15.19:0/1339619191 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.617+0000 7f1d9660c640 1 -- 172.21.15.19:0/1339619191 >> v1:172.21.15.112:6789/0 conn(0x7f1d9015dac0 legacy=0x7f1d900d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.617+0000 7f1d9660c640 1 -- 172.21.15.19:0/1339619191 >> v1:172.21.15.19:6789/0 conn(0x7f1d900d34c0 legacy=0x7f1d900d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.617+0000 7f1d9660c640 1 -- 172.21.15.19:0/1339619191 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d900d29e0 con 0x7f1d900d0ab0 2024-04-25T09:59:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.617+0000 7f1d94e09640 1 -- 172.21.15.19:0/1339619191 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16044389 0 0) 0x7f1d900d2420 con 0x7f1d900d34c0 2024-04-25T09:59:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.618+0000 7f1d9660c640 1 --2- 172.21.15.19:0/1339619191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d900d0ab0 0x7f1d90168670 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f1d900d31b0 tx=0x7f1d8802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a6b1eafb3e1885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.618+0000 7f1d94e09640 1 -- 172.21.15.19:0/1339619191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88037070 con 0x7f1d900d0ab0 2024-04-25T09:59:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.618+0000 7f1d94e09640 1 -- 172.21.15.19:0/1339619191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d88007740 con 0x7f1d900d0ab0 2024-04-25T09:59:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.618+0000 7f1d94e09640 1 -- 172.21.15.19:0/1339619191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88007b10 con 0x7f1d900d0ab0 2024-04-25T09:59:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.618+0000 7f1d9806f640 1 -- 172.21.15.19:0/1339619191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d900d0ab0 msgr2=0x7f1d90168670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.618+0000 7f1d9806f640 1 --2- 172.21.15.19:0/1339619191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d900d0ab0 0x7f1d90168670 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f1d900d31b0 tx=0x7f1d8802a780 comp rx=0 tx=0).stop 2024-04-25T09:59:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.618+0000 7f1d9806f640 1 -- 172.21.15.19:0/1339619191 shutdown_connections 2024-04-25T09:59:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.618+0000 7f1d9806f640 1 --2- 172.21.15.19:0/1339619191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d900d0ab0 0x7f1d90168670 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.618+0000 7f1d9806f640 1 -- 172.21.15.19:0/1339619191 >> 172.21.15.19:0/1339619191 conn(0x7f1d90061c30 msgr2=0x7f1d9006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.618+0000 7f1d9806f640 1 -- 172.21.15.19:0/1339619191 shutdown_connections 2024-04-25T09:59:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.618+0000 7f1d9806f640 1 -- 172.21.15.19:0/1339619191 wait complete. 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.620+0000 7f1d9806f640 1 Processor -- start 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.620+0000 7f1d9806f640 1 -- start start 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.620+0000 7f1d9806f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d900d0ab0 0x7f1d90144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.620+0000 7f1d9806f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d900d34c0 0x7f1d9015fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.620+0000 7f1d9806f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d9015dac0 0x7f1d9015ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.620+0000 7f1d9806f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9015dea0 con 0x7f1d900d0ab0 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.620+0000 7f1d9806f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900d2100 con 0x7f1d9015dac0 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.620+0000 7f1d9806f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900b97d0 con 0x7f1d900d34c0 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.620+0000 7f1d95e0b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d900d0ab0 0x7f1d90144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d95e0b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d900d0ab0 0x7f1d90144c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47100/0 (socket says 172.21.15.19:47100) 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d95e0b640 1 -- 172.21.15.19:0/4033299713 learned_addr learned my addr 172.21.15.19:0/4033299713 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d9660c640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d9015dac0 0x7f1d9015ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d9560a640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d900d34c0 0x7f1d9015fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d95e0b640 1 -- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d900d34c0 msgr2=0x7f1d9015fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d95e0b640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d900d34c0 0x7f1d9015fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d95e0b640 1 -- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d9015dac0 msgr2=0x7f1d9015ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d95e0b640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d9015dac0 0x7f1d9015ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d95e0b640 1 -- 172.21.15.19:0/4033299713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d90153200 con 0x7f1d900d0ab0 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d9660c640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d9015dac0 0x7f1d9015ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d95e0b640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d900d0ab0 0x7f1d90144c10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00c400 tx=0x7f1d8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d867fc640 1 -- 172.21.15.19:0/4033299713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c017070 con 0x7f1d900d0ab0 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d867fc640 1 -- 172.21.15.19:0/4033299713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c0052d0 con 0x7f1d900d0ab0 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d9806f640 1 -- 172.21.15.19:0/4033299713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d90159d00 con 0x7f1d900d0ab0 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d867fc640 1 -- 172.21.15.19:0/4033299713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c003830 con 0x7f1d900d0ab0 2024-04-25T09:59:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.621+0000 7f1d9806f640 1 -- 172.21.15.19:0/4033299713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d901608d0 con 0x7f1d900d0ab0 2024-04-25T09:59:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.622+0000 7f1d9806f640 1 -- 172.21.15.19:0/4033299713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d90073a00 con 0x7f1d900d0ab0 2024-04-25T09:59:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.623+0000 7f1d867fc640 1 -- 172.21.15.19:0/4033299713 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d8c01a070 con 0x7f1d900d0ab0 2024-04-25T09:59:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.623+0000 7f1d867fc640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d7406b4d0 0x7f1d7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.623+0000 7f1d867fc640 1 -- 172.21.15.19:0/4033299713 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1d8c089640 con 0x7f1d900d0ab0 2024-04-25T09:59:31.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.623+0000 7f1d9560a640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d7406b4d0 0x7f1d7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.624+0000 7f1d9560a640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d7406b4d0 0x7f1d7406d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1d80007fd0 tx=0x7f1d80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.627+0000 7f1d867fc640 1 -- 172.21.15.19:0/4033299713 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d8c057ce0 con 0x7f1d900d0ab0 2024-04-25T09:59:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.808+0000 7f1d9806f640 1 -- 172.21.15.19:0/4033299713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d900c6bd0 con 0x7f1d900d0ab0 2024-04-25T09:59:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.809+0000 7f1d867fc640 1 -- 172.21.15.19:0/4033299713 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d8c05bd80 con 0x7f1d900d0ab0 2024-04-25T09:59:31.814 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:59:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.810+0000 7f1d9806f640 1 -- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d7406b4d0 msgr2=0x7f1d7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.810+0000 7f1d9806f640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d7406b4d0 0x7f1d7406d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1d80007fd0 tx=0x7f1d80009040 comp rx=0 tx=0).stop 2024-04-25T09:59:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.811+0000 7f1d9806f640 1 -- 172.21.15.19:0/4033299713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d900d0ab0 msgr2=0x7f1d90144c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.811+0000 7f1d9806f640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d900d0ab0 0x7f1d90144c10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00c400 tx=0x7f1d8c00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.811+0000 7f1d9806f640 1 -- 172.21.15.19:0/4033299713 shutdown_connections 2024-04-25T09:59:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.811+0000 7f1d9806f640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d7406b4d0 0x7f1d7406d990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.811+0000 7f1d9806f640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d9015dac0 0x7f1d9015ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.811+0000 7f1d9806f640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d900d34c0 0x7f1d9015fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.811+0000 7f1d9806f640 1 --2- 172.21.15.19:0/4033299713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d900d0ab0 0x7f1d90144c10 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.811+0000 7f1d9806f640 1 -- 172.21.15.19:0/4033299713 >> 172.21.15.19:0/4033299713 conn(0x7f1d90061c30 msgr2=0x7f1d900b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.811+0000 7f1d9806f640 1 -- 172.21.15.19:0/4033299713 shutdown_connections 2024-04-25T09:59:31.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.811+0000 7f1d9806f640 1 -- 172.21.15.19:0/4033299713 wait complete. 2024-04-25T09:59:31.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:59:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35f3809640 1 Processor -- start 2024-04-25T09:59:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35f3809640 1 -- start start 2024-04-25T09:59:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35f3809640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec0ce230 0x7f35ec0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35f3809640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f35ec15b270 0x7f35ec15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35f3809640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ec072280 con 0x7f35ec0cc4f0 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35f3809640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ec071270 con 0x7f35ec15b270 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35f3809640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec0714c0 con 0x7f35ec0ce230 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35f15a5640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f35ec0cc4f0 0x7f35ec0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45774/0 (socket says 172.21.15.19:45774) 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35f15a5640 1 -- 172.21.15.19:0/3871143699 learned_addr learned my addr 172.21.15.19:0/3871143699 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35f1da6640 1 --2- 172.21.15.19:0/3871143699 >> v2:172.21.15.112:3300/0 conn(0x7f35ec15b270 0x7f35ec15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897234036 0 0) 0x7f35ec072280 con 0x7f35ec0cc4f0 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d8003520 con 0x7f35ec0cc4f0 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35f0da4640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec0ce230 0x7f35ec0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707162866 0 0) 0x7f35d8003520 con 0x7f35ec0cc4f0 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec0ce230 msgr2=0x7f35ec0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec0ce230 0x7f35ec0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 >> v2:172.21.15.112:3300/0 conn(0x7f35ec15b270 msgr2=0x7f35ec15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 --2- 172.21.15.19:0/3871143699 >> v2:172.21.15.112:3300/0 conn(0x7f35ec15b270 0x7f35ec15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec0717a0 con 0x7f35ec0cc4f0 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f35e8001670 con 0x7f35ec0cc4f0 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 >> v1:172.21.15.19:6789/0 conn(0x7f35ec0cc4f0 legacy=0x7f35ec0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec0ce230 0x7f35d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 0x7f35d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d8004d40 0x7f35d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec071270 con 0x7f35d8003f30 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec0714c0 con 0x7f35d8004d40 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.924+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d800b9c0 con 0x7f35ec0ce230 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.925+0000 7f35f0da4640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 0x7f35d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.925+0000 7f35f15a5640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec0ce230 0x7f35d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.925+0000 7f35f1da6640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d8004d40 0x7f35d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.925+0000 7f35f0da4640 1 -- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec0ce230 msgr2=0x7f35d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.925+0000 7f35f0da4640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec0ce230 0x7f35d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.925+0000 7f35f0da4640 1 -- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d8004d40 msgr2=0x7f35d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.925+0000 7f35f0da4640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d8004d40 0x7f35d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.925+0000 7f35f0da4640 1 -- 172.21.15.19:0/3871143699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d8003520 con 0x7f35d8003f30 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.925+0000 7f35f1da6640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d8004d40 0x7f35d80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.925+0000 7f35f0da4640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 0x7f35d8004800 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009ac0 tx=0x7f35d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edeaf79502a09640 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.926+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc036070 con 0x7f35d8003f30 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.926+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc031400 con 0x7f35d8003f30 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.926+0000 7f35e3fff640 1 -- 172.21.15.19:0/3871143699 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc009690 con 0x7f35d8003f30 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.926+0000 7f35f3809640 1 -- 172.21.15.19:0/3871143699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 msgr2=0x7f35d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.926+0000 7f35f3809640 1 --2- 172.21.15.19:0/3871143699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 0x7f35d8004800 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009ac0 tx=0x7f35d8005830 comp rx=0 tx=0).stop 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.926+0000 7f35f1da6640 1 -- 172.21.15.19:0/3871143699 reap_dead start 2024-04-25T09:59:31.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.926+0000 7f35f3809640 1 -- 172.21.15.19:0/3871143699 shutdown_connections 2024-04-25T09:59:31.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.926+0000 7f35f3809640 1 -- 172.21.15.19:0/3871143699 >> 172.21.15.19:0/3871143699 conn(0x7f35ec06abe0 msgr2=0x7f35ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:31.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.926+0000 7f35f3809640 1 -- 172.21.15.19:0/3871143699 shutdown_connections 2024-04-25T09:59:31.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.926+0000 7f35f3809640 1 -- 172.21.15.19:0/3871143699 wait complete. 2024-04-25T09:59:31.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.927+0000 7f35f3809640 1 Processor -- start 2024-04-25T09:59:31.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.927+0000 7f35f3809640 1 -- start start 2024-04-25T09:59:31.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.928+0000 7f35f3809640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 0x7f35ec1485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.928+0000 7f35f3809640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ec14fb30 0x7f35ec0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.928+0000 7f35f3809640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec148e70 0x7f35ec1492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.928+0000 7f35f3809640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec15bc00 con 0x7f35d8003f30 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.928+0000 7f35f3809640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec071d60 con 0x7f35ec14fb30 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.928+0000 7f35f3809640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec15bd80 con 0x7f35ec148e70 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.928+0000 7f35f15a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 0x7f35ec1485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.928+0000 7f35f15a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 0x7f35ec1485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47108/0 (socket says 172.21.15.19:47108) 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.928+0000 7f35f15a5640 1 -- 172.21.15.19:0/3558231098 learned_addr learned my addr 172.21.15.19:0/3558231098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.928+0000 7f35f0da4640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ec14fb30 0x7f35ec0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.928+0000 7f35f1da6640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec148e70 0x7f35ec1492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35f15a5640 1 -- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec148e70 msgr2=0x7f35ec1492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35f15a5640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec148e70 0x7f35ec1492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35f15a5640 1 -- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ec14fb30 msgr2=0x7f35ec0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35f15a5640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ec14fb30 0x7f35ec0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35f15a5640 1 -- 172.21.15.19:0/3558231098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec1502a0 con 0x7f35d8003f30 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35f1da6640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec148e70 0x7f35ec1492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35f15a5640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 0x7f35ec1485d0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f35e8007340 tx=0x7f35e8007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35e1ffb640 1 -- 172.21.15.19:0/3558231098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e800f940 con 0x7f35d8003f30 2024-04-25T09:59:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35e1ffb640 1 -- 172.21.15.19:0/3558231098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e8018070 con 0x7f35d8003f30 2024-04-25T09:59:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35f3809640 1 -- 172.21.15.19:0/3558231098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ec150580 con 0x7f35d8003f30 2024-04-25T09:59:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35e1ffb640 1 -- 172.21.15.19:0/3558231098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e800fd20 con 0x7f35d8003f30 2024-04-25T09:59:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.929+0000 7f35f3809640 1 -- 172.21.15.19:0/3558231098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec150b40 con 0x7f35d8003f30 2024-04-25T09:59:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.930+0000 7f35f3809640 1 -- 172.21.15.19:0/3558231098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ec073a00 con 0x7f35d8003f30 2024-04-25T09:59:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.930+0000 7f35e1ffb640 1 -- 172.21.15.19:0/3558231098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35e801d460 con 0x7f35d8003f30 2024-04-25T09:59:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.931+0000 7f35e1ffb640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35d006b4d0 0x7f35d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.931+0000 7f35e1ffb640 1 -- 172.21.15.19:0/3558231098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f35e808a3d0 con 0x7f35d8003f30 2024-04-25T09:59:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.931+0000 7f35f0da4640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35d006b4d0 0x7f35d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.931+0000 7f35f0da4640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35d006b4d0 0x7f35d006d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f35dc033000 tx=0x7f35dc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:31.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:31.935+0000 7f35e1ffb640 1 -- 172.21.15.19:0/3558231098 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f35e8059150 con 0x7f35d8003f30 2024-04-25T09:59:32.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.128+0000 7f35f3809640 1 -- 172.21.15.19:0/3558231098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f35ec0cb060 con 0x7f35d8003f30 2024-04-25T09:59:32.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.128+0000 7f35e1ffb640 1 -- 172.21.15.19:0/3558231098 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v95) v1 ==== 80+0+0 (secure 0 0 0) 0x7f35e8058a50 con 0x7f35d8003f30 2024-04-25T09:59:32.133 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T09:59:32.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 -- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35d006b4d0 msgr2=0x7f35d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:32.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35d006b4d0 0x7f35d006d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f35dc033000 tx=0x7f35dc008030 comp rx=0 tx=0).stop 2024-04-25T09:59:32.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 -- 172.21.15.19:0/3558231098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 msgr2=0x7f35ec1485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:32.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 0x7f35ec1485d0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f35e8007340 tx=0x7f35e8007800 comp rx=0 tx=0).stop 2024-04-25T09:59:32.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 -- 172.21.15.19:0/3558231098 shutdown_connections 2024-04-25T09:59:32.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35d006b4d0 0x7f35d006d990 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:32.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ec148e70 0x7f35ec1492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:32.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ec14fb30 0x7f35ec0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:32.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 --2- 172.21.15.19:0/3558231098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d8003f30 0x7f35ec1485d0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:32.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 -- 172.21.15.19:0/3558231098 >> 172.21.15.19:0/3558231098 conn(0x7f35ec06abe0 msgr2=0x7f35ec15c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:32.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 -- 172.21.15.19:0/3558231098 shutdown_connections 2024-04-25T09:59:32.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:32.130+0000 7f35f3809640 1 -- 172.21.15.19:0/3558231098 wait complete. 2024-04-25T09:59:32.145 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:59:32.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:32.145 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:32.804 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T09:59:32.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:32.804 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:59:32.822 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:59:32.822 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:32.840 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:59:32.840 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:59:32.840 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:32.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:59:32.879 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:59:32.879 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:32.896 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:59:32.896 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.014+0000 7f64107c4640 1 Processor -- start 2024-04-25T09:59:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.014+0000 7f64107c4640 1 -- start start 2024-04-25T09:59:33.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f64107c4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64080af720 0x7f64080afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f64107c4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640806fe00 con 0x7f6408158220 2024-04-25T09:59:33.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f64107c4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64080700e0 con 0x7f64080d02f0 2024-04-25T09:59:33.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f64107c4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64080b3670 con 0x7f64080af720 2024-04-25T09:59:33.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640e560640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6408158220 0x7f640815ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45778/0 (socket says 172.21.15.19:45778) 2024-04-25T09:59:33.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640e560640 1 -- 172.21.15.19:0/3427756479 learned_addr learned my addr 172.21.15.19:0/3427756479 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:33.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640dd5f640 1 --2- 172.21.15.19:0/3427756479 >> v2:172.21.15.112:3300/0 conn(0x7f64080af720 0x7f64080afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520464781 0 0) 0x7f640806fe00 con 0x7f6408158220 2024-04-25T09:59:33.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63f4003520 con 0x7f6408158220 2024-04-25T09:59:33.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 754314551 0 0) 0x7f63f4003520 con 0x7f6408158220 2024-04-25T09:59:33.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 >> v2:172.21.15.112:3300/0 conn(0x7f64080af720 msgr2=0x7f64080afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:33.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 --2- 172.21.15.19:0/3427756479 >> v2:172.21.15.112:3300/0 conn(0x7f64080af720 0x7f64080afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 >> v1:172.21.15.112:6789/0 conn(0x7f64080d02f0 legacy=0x7f64080d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080703c0 con 0x7f6408158220 2024-04-25T09:59:33.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f6404001670 con 0x7f6408158220 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 >> v1:172.21.15.19:6789/0 conn(0x7f6408158220 legacy=0x7f640815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 --2- 172.21.15.19:0/3427756479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f40039b0 0x7f63f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 --2- 172.21.15.19:0/3427756479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f4004800 0x7f63f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 --2- 172.21.15.19:0/3427756479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f40051c0 0x7f63f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f400c030 con 0x7f63f4004800 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f400c1e0 con 0x7f63f40039b0 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.015+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f400c3e0 con 0x7f63f40051c0 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.016+0000 7f640dd5f640 1 --2- 172.21.15.19:0/3427756479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f4004800 0x7f63f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.016+0000 7f640e560640 1 --2- 172.21.15.19:0/3427756479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f40039b0 0x7f63f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.016+0000 7f640ed61640 1 --2- 172.21.15.19:0/3427756479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f40051c0 0x7f63f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.016+0000 7f640dd5f640 1 -- 172.21.15.19:0/3427756479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f40051c0 msgr2=0x7f63f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.016+0000 7f640dd5f640 1 --2- 172.21.15.19:0/3427756479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f40051c0 0x7f63f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.016+0000 7f640dd5f640 1 -- 172.21.15.19:0/3427756479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f40039b0 msgr2=0x7f63f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.016+0000 7f640dd5f640 1 --2- 172.21.15.19:0/3427756479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f40039b0 0x7f63f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.016+0000 7f640dd5f640 1 -- 172.21.15.19:0/3427756479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f4003520 con 0x7f63f4004800 2024-04-25T09:59:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.016+0000 7f640ed61640 1 -- 172.21.15.19:0/3427756479 reap_dead start 2024-04-25T09:59:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.016+0000 7f640dd5f640 1 --2- 172.21.15.19:0/3427756479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f4004800 0x7f63f4004c80 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009ac0 tx=0x7f63f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a046e4e99ba45997 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.017+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8036070 con 0x7f63f4004800 2024-04-25T09:59:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.017+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f8031400 con 0x7f63f4004800 2024-04-25T09:59:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.017+0000 7f640d55e640 1 -- 172.21.15.19:0/3427756479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8009690 con 0x7f63f4004800 2024-04-25T09:59:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.017+0000 7f64107c4640 1 -- 172.21.15.19:0/3427756479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f4004800 msgr2=0x7f63f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.017+0000 7f64107c4640 1 --2- 172.21.15.19:0/3427756479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f4004800 0x7f63f4004c80 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009ac0 tx=0x7f63f4005830 comp rx=0 tx=0).stop 2024-04-25T09:59:33.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.017+0000 7f64107c4640 1 -- 172.21.15.19:0/3427756479 shutdown_connections 2024-04-25T09:59:33.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.017+0000 7f64107c4640 1 --2- 172.21.15.19:0/3427756479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f4004800 0x7f63f4004c80 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.017+0000 7f64107c4640 1 -- 172.21.15.19:0/3427756479 >> 172.21.15.19:0/3427756479 conn(0x7f64080b2370 msgr2=0x7f64080b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:33.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.017+0000 7f64107c4640 1 -- 172.21.15.19:0/3427756479 shutdown_connections 2024-04-25T09:59:33.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.017+0000 7f64107c4640 1 -- 172.21.15.19:0/3427756479 wait complete. 2024-04-25T09:59:33.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.018+0000 7f64107c4640 1 Processor -- start 2024-04-25T09:59:33.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.019+0000 7f64107c4640 1 -- start start 2024-04-25T09:59:33.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.019+0000 7f64107c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f4004800 0x7f64081f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.019+0000 7f64107c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64080ce860 0x7f64080b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.019+0000 7f64107c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640815f580 0x7f640815fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.019+0000 7f64107c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6408158600 con 0x7f640815f580 2024-04-25T09:59:33.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.019+0000 7f64107c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6408070a10 con 0x7f64080ce860 2024-04-25T09:59:33.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.019+0000 7f64107c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6408072960 con 0x7f63f4004800 2024-04-25T09:59:33.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.019+0000 7f640ed61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640815f580 0x7f640815fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.019+0000 7f640ed61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640815f580 0x7f640815fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47134/0 (socket says 172.21.15.19:47134) 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.019+0000 7f640ed61640 1 -- 172.21.15.19:0/3599322342 learned_addr learned my addr 172.21.15.19:0/3599322342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.019+0000 7f640e560640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f4004800 0x7f64081f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f640dd5f640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64080ce860 0x7f64080b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f640ed61640 1 -- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f4004800 msgr2=0x7f64081f3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f640ed61640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f4004800 0x7f64081f3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f640ed61640 1 -- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64080ce860 msgr2=0x7f64080b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f640ed61640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64080ce860 0x7f64080b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f640ed61640 1 -- 172.21.15.19:0/3599322342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64081f4100 con 0x7f640815f580 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f640ed61640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640815f580 0x7f640815fa10 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6400007fd0 tx=0x7f640000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f63feffd640 1 -- 172.21.15.19:0/3599322342 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6400003970 con 0x7f640815f580 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f63feffd640 1 -- 172.21.15.19:0/3599322342 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6400018070 con 0x7f640815f580 2024-04-25T09:59:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f64107c4640 1 -- 172.21.15.19:0/3599322342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64080cd7c0 con 0x7f640815f580 2024-04-25T09:59:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f63feffd640 1 -- 172.21.15.19:0/3599322342 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6400003d70 con 0x7f640815f580 2024-04-25T09:59:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.020+0000 7f64107c4640 1 -- 172.21.15.19:0/3599322342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64080cdd80 con 0x7f640815f580 2024-04-25T09:59:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.021+0000 7f64107c4640 1 -- 172.21.15.19:0/3599322342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6408073990 con 0x7f640815f580 2024-04-25T09:59:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.022+0000 7f63feffd640 1 -- 172.21.15.19:0/3599322342 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f64000050d0 con 0x7f640815f580 2024-04-25T09:59:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.022+0000 7f63feffd640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63ec06b4d0 0x7f63ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.022+0000 7f63feffd640 1 -- 172.21.15.19:0/3599322342 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6400089da0 con 0x7f640815f580 2024-04-25T09:59:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.023+0000 7f640e560640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63ec06b4d0 0x7f63ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.023+0000 7f640e560640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63ec06b4d0 0x7f63ec06d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6404001cd0 tx=0x7f6404007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.028+0000 7f63feffd640 1 -- 172.21.15.19:0/3599322342 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f64081f4100 con 0x7f640815f580 2024-04-25T09:59:33.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.243+0000 7f64107c4640 1 -- 172.21.15.19:0/3599322342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6408155d10 con 0x7f640815f580 2024-04-25T09:59:33.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.243+0000 7f63feffd640 1 -- 172.21.15.19:0/3599322342 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f6400058430 con 0x7f640815f580 2024-04-25T09:59:33.249 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:33.249 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3431952054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3431952054},{"type":"v1","addr":"172.21.15.112:6836","nonce":3431952054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2758109099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2758109099},{"type":"v1","addr":"172.21.15.112:6837","nonce":2758109099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7590,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3063446116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3063446116},{"type":"v1","addr":"172.21.15.19:6834","nonce":3063446116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:07.729627+0000","modified":"2024-04-25T09:59:09.720756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3589967477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3589967477},{"type":"v1","addr":"172.21.15.19:6835","nonce":3589967477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:33.249 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-04-25T09:59:33.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.245+0000 7f64107c4640 1 -- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63ec06b4d0 msgr2=0x7f63ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.245+0000 7f64107c4640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63ec06b4d0 0x7f63ec06d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6404001cd0 tx=0x7f6404007040 comp rx=0 tx=0).stop 2024-04-25T09:59:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.245+0000 7f64107c4640 1 -- 172.21.15.19:0/3599322342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640815f580 msgr2=0x7f640815fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.245+0000 7f64107c4640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640815f580 0x7f640815fa10 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6400007fd0 tx=0x7f640000eb90 comp rx=0 tx=0).stop 2024-04-25T09:59:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.245+0000 7f64107c4640 1 -- 172.21.15.19:0/3599322342 shutdown_connections 2024-04-25T09:59:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.245+0000 7f64107c4640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f63ec06b4d0 0x7f63ec06d990 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.245+0000 7f64107c4640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f640815f580 0x7f640815fa10 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.245+0000 7f64107c4640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64080ce860 0x7f64080b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.245+0000 7f64107c4640 1 --2- 172.21.15.19:0/3599322342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f4004800 0x7f64081f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.245+0000 7f64107c4640 1 -- 172.21.15.19:0/3599322342 >> 172.21.15.19:0/3599322342 conn(0x7f64080b2370 msgr2=0x7f64080d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.245+0000 7f64107c4640 1 -- 172.21.15.19:0/3599322342 shutdown_connections 2024-04-25T09:59:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.246+0000 7f64107c4640 1 -- 172.21.15.19:0/3599322342 wait complete. 2024-04-25T09:59:33.262 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.358+0000 7f5ad9c07640 1 Processor -- start 2024-04-25T09:59:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.358+0000 7f5ad9c07640 1 -- start start 2024-04-25T09:59:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.360+0000 7f5ad9c07640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ad415a790 0x7f5ad40d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.360+0000 7f5ad9c07640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad40b3710 con 0x7f5ad40b4eb0 2024-04-25T09:59:33.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.360+0000 7f5ad9c07640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad4073120 con 0x7f5ad406fea0 2024-04-25T09:59:33.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.360+0000 7f5ad9c07640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad40cf0e0 con 0x7f5ad415a790 2024-04-25T09:59:33.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.360+0000 7f5ad37fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ad415a790 0x7f5ad40d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.360+0000 7f5ad3fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5ad40b4eb0 0x7f5ad4072a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45786/0 (socket says 172.21.15.19:45786) 2024-04-25T09:59:33.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.360+0000 7f5ad3fff640 1 -- 172.21.15.19:0/2887928310 learned_addr learned my addr 172.21.15.19:0/2887928310 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:33.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.361+0000 7f5ad37fe640 1 -- 172.21.15.19:0/2887928310 >> v1:172.21.15.112:6789/0 conn(0x7f5ad406fea0 legacy=0x7f5ad4072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:33.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.361+0000 7f5ad37fe640 1 -- 172.21.15.19:0/2887928310 >> v1:172.21.15.19:6789/0 conn(0x7f5ad40b4eb0 legacy=0x7f5ad4072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.361+0000 7f5ad37fe640 1 -- 172.21.15.19:0/2887928310 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad40cf3c0 con 0x7f5ad415a790 2024-04-25T09:59:33.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.361+0000 7f5ad27fc640 1 -- 172.21.15.19:0/2887928310 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401846457 0 0) 0x7f5ad40b3710 con 0x7f5ad40b4eb0 2024-04-25T09:59:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.361+0000 7f5ad37fe640 1 --2- 172.21.15.19:0/2887928310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad415a790 0x7f5ad40d0390 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40097a0 tx=0x7f5ac402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d396714325f4a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.362+0000 7f5ad27fc640 1 -- 172.21.15.19:0/2887928310 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4037070 con 0x7f5ad415a790 2024-04-25T09:59:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.362+0000 7f5ad27fc640 1 -- 172.21.15.19:0/2887928310 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac4032440 con 0x7f5ad415a790 2024-04-25T09:59:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.362+0000 7f5ad27fc640 1 -- 172.21.15.19:0/2887928310 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4032810 con 0x7f5ad415a790 2024-04-25T09:59:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.362+0000 7f5ad9c07640 1 -- 172.21.15.19:0/2887928310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad415a790 msgr2=0x7f5ad40d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.362+0000 7f5ad9c07640 1 --2- 172.21.15.19:0/2887928310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad415a790 0x7f5ad40d0390 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40097a0 tx=0x7f5ac402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.362+0000 7f5ad9c07640 1 -- 172.21.15.19:0/2887928310 shutdown_connections 2024-04-25T09:59:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.362+0000 7f5ad9c07640 1 --2- 172.21.15.19:0/2887928310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad415a790 0x7f5ad40d0390 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.362+0000 7f5ad9c07640 1 -- 172.21.15.19:0/2887928310 >> 172.21.15.19:0/2887928310 conn(0x7f5ad40b2370 msgr2=0x7f5ad40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.362+0000 7f5ad9c07640 1 -- 172.21.15.19:0/2887928310 shutdown_connections 2024-04-25T09:59:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.362+0000 7f5ad9c07640 1 -- 172.21.15.19:0/2887928310 wait complete. 2024-04-25T09:59:33.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.363+0000 7f5ad9c07640 1 Processor -- start 2024-04-25T09:59:33.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.363+0000 7f5ad9c07640 1 -- start start 2024-04-25T09:59:33.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad9c07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad406fea0 0x7f5ad413ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad9c07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ad40b4eb0 0x7f5ad4154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad9c07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad415a790 0x7f5ad4154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad9c07640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40cf980 con 0x7f5ad406fea0 2024-04-25T09:59:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad9c07640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40b3980 con 0x7f5ad415a790 2024-04-25T09:59:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad9c07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad415ab70 con 0x7f5ad40b4eb0 2024-04-25T09:59:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad406fea0 0x7f5ad413ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad406fea0 0x7f5ad413ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47150/0 (socket says 172.21.15.19:47150) 2024-04-25T09:59:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad37fe640 1 -- 172.21.15.19:0/4052069355 learned_addr learned my addr 172.21.15.19:0/4052069355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad3fff640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad415a790 0x7f5ad4154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad2ffd640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ad40b4eb0 0x7f5ad4154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad37fe640 1 -- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ad40b4eb0 msgr2=0x7f5ad4154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad37fe640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ad40b4eb0 0x7f5ad4154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad37fe640 1 -- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad415a790 msgr2=0x7f5ad4154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad37fe640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad415a790 0x7f5ad4154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad37fe640 1 -- 172.21.15.19:0/4052069355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad4141900 con 0x7f5ad406fea0 2024-04-25T09:59:33.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad37fe640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad406fea0 0x7f5ad413ef60 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40097a0 tx=0x7f5ac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:33.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5abbfff640 1 -- 172.21.15.19:0/4052069355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4037040 con 0x7f5ad406fea0 2024-04-25T09:59:33.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5abbfff640 1 -- 172.21.15.19:0/4052069355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac4004250 con 0x7f5ad406fea0 2024-04-25T09:59:33.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad9c07640 1 -- 172.21.15.19:0/4052069355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad413f4a0 con 0x7f5ad406fea0 2024-04-25T09:59:33.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5abbfff640 1 -- 172.21.15.19:0/4052069355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4032e20 con 0x7f5ad406fea0 2024-04-25T09:59:33.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.364+0000 7f5ad9c07640 1 -- 172.21.15.19:0/4052069355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad4141e50 con 0x7f5ad406fea0 2024-04-25T09:59:33.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.366+0000 7f5ad9c07640 1 -- 172.21.15.19:0/4052069355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad4073990 con 0x7f5ad406fea0 2024-04-25T09:59:33.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.367+0000 7f5abbfff640 1 -- 172.21.15.19:0/4052069355 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ac4030770 con 0x7f5ad406fea0 2024-04-25T09:59:33.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.367+0000 7f5abbfff640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac006b4d0 0x7f5ac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.367+0000 7f5abbfff640 1 -- 172.21.15.19:0/4052069355 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ac40ab560 con 0x7f5ad406fea0 2024-04-25T09:59:33.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.367+0000 7f5ad2ffd640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac006b4d0 0x7f5ac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.368+0000 7f5ad2ffd640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac006b4d0 0x7f5ac006d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5ad414ef00 tx=0x7f5ac8007e30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:33.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.371+0000 7f5abbfff640 1 -- 172.21.15.19:0/4052069355 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ac4079c00 con 0x7f5ad406fea0 2024-04-25T09:59:33.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.556+0000 7f5ad9c07640 1 -- 172.21.15.19:0/4052069355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ad40b3ea0 con 0x7f5ad406fea0 2024-04-25T09:59:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.557+0000 7f5abbfff640 1 -- 172.21.15.19:0/4052069355 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f5ac407dca0 con 0x7f5ad406fea0 2024-04-25T09:59:33.562 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:33.562 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":95,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:09.369855+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:06.008725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:06.705899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:33.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.559+0000 7f5ad9c07640 1 -- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac006b4d0 msgr2=0x7f5ac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.559+0000 7f5ad9c07640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac006b4d0 0x7f5ac006d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5ad414ef00 tx=0x7f5ac8007e30 comp rx=0 tx=0).stop 2024-04-25T09:59:33.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.559+0000 7f5ad9c07640 1 -- 172.21.15.19:0/4052069355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad406fea0 msgr2=0x7f5ad413ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.559+0000 7f5ad9c07640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad406fea0 0x7f5ad413ef60 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40097a0 tx=0x7f5ac40046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.559+0000 7f5ad9c07640 1 -- 172.21.15.19:0/4052069355 shutdown_connections 2024-04-25T09:59:33.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.559+0000 7f5ad9c07640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac006b4d0 0x7f5ac006d990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.559+0000 7f5ad9c07640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ad415a790 0x7f5ad4154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.559+0000 7f5ad9c07640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ad40b4eb0 0x7f5ad4154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.559+0000 7f5ad9c07640 1 --2- 172.21.15.19:0/4052069355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ad406fea0 0x7f5ad413ef60 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.559+0000 7f5ad9c07640 1 -- 172.21.15.19:0/4052069355 >> 172.21.15.19:0/4052069355 conn(0x7f5ad40b2370 msgr2=0x7f5ad40b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:33.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.560+0000 7f5ad9c07640 1 -- 172.21.15.19:0/4052069355 shutdown_connections 2024-04-25T09:59:33.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.560+0000 7f5ad9c07640 1 -- 172.21.15.19:0/4052069355 wait complete. 2024-04-25T09:59:33.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:59:06.008725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:59:06.705899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:59:33.575 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:59:33.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.675+0000 7f09df46b640 1 Processor -- start 2024-04-25T09:59:33.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.675+0000 7f09df46b640 1 -- start start 2024-04-25T09:59:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.676+0000 7f09df46b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81452a0 0x7f09d806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.676+0000 7f09df46b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d80cccc0 con 0x7f09d81446d0 2024-04-25T09:59:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.676+0000 7f09df46b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d80cc140 con 0x7f09d8144d50 2024-04-25T09:59:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.676+0000 7f09df46b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80b2b20 con 0x7f09d81452a0 2024-04-25T09:59:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.676+0000 7f09dd207640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f09d81446d0 0x7f09d80b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45792/0 (socket says 172.21.15.19:45792) 2024-04-25T09:59:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.676+0000 7f09dd207640 1 -- 172.21.15.19:0/194566583 learned_addr learned my addr 172.21.15.19:0/194566583 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.676+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774942065 0 0) 0x7f09d80cccc0 con 0x7f09d81446d0 2024-04-25T09:59:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.676+0000 7f09dda08640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81452a0 0x7f09d806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.676+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c0003520 con 0x7f09d81446d0 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3120396170 0 0) 0x7f09c0003520 con 0x7f09d81446d0 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81452a0 msgr2=0x7f09d806dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81452a0 0x7f09d806dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 >> v1:172.21.15.112:6789/0 conn(0x7f09d8144d50 legacy=0x7f09d806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80cc420 con 0x7f09d81446d0 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f09d0001670 con 0x7f09d81446d0 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 >> v1:172.21.15.19:6789/0 conn(0x7f09d81446d0 legacy=0x7f09d80b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c00039e0 0x7f09c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81452a0 0x7f09c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c0004d40 0x7f09c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80b2b20 con 0x7f09c00039e0 2024-04-25T09:59:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c000baa0 con 0x7f09c0004d40 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c000bca0 con 0x7f09d81452a0 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09dd207640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c00039e0 0x7f09c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09dca06640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81452a0 0x7f09c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09dda08640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c0004d40 0x7f09c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09dd207640 1 -- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81452a0 msgr2=0x7f09c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09dd207640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81452a0 0x7f09c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09dd207640 1 -- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c0004d40 msgr2=0x7f09c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09dd207640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c0004d40 0x7f09c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09dd207640 1 -- 172.21.15.19:0/194566583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0003520 con 0x7f09c00039e0 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.677+0000 7f09dda08640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c0004d40 0x7f09c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.678+0000 7f09dd207640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c00039e0 0x7f09c0003e00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f09d000c210 tx=0x7f09c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939497d432a803da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.678+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0007cf0 con 0x7f09c00039e0 2024-04-25T09:59:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.678+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d0005af0 con 0x7f09c00039e0 2024-04-25T09:59:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.678+0000 7f09d7fff640 1 -- 172.21.15.19:0/194566583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0007070 con 0x7f09c00039e0 2024-04-25T09:59:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.679+0000 7f09df46b640 1 -- 172.21.15.19:0/194566583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c00039e0 msgr2=0x7f09c0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.679+0000 7f09df46b640 1 --2- 172.21.15.19:0/194566583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09c00039e0 0x7f09c0003e00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f09d000c210 tx=0x7f09c0005830 comp rx=0 tx=0).stop 2024-04-25T09:59:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.679+0000 7f09dda08640 1 -- 172.21.15.19:0/194566583 reap_dead start 2024-04-25T09:59:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.679+0000 7f09df46b640 1 -- 172.21.15.19:0/194566583 shutdown_connections 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.679+0000 7f09df46b640 1 -- 172.21.15.19:0/194566583 >> 172.21.15.19:0/194566583 conn(0x7f09d806abe0 msgr2=0x7f09d806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.679+0000 7f09df46b640 1 -- 172.21.15.19:0/194566583 shutdown_connections 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.679+0000 7f09df46b640 1 -- 172.21.15.19:0/194566583 wait complete. 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.680+0000 7f09df46b640 1 Processor -- start 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.680+0000 7f09df46b640 1 -- start start 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09df46b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c00039e0 0x7f09d81f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09df46b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d80c5720 0x7f09d81f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09df46b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81f4240 0x7f09d80b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09df46b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80cc9e0 con 0x7f09d80c5720 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09df46b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d806d400 con 0x7f09c00039e0 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09df46b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d8144ab0 con 0x7f09d81f4240 2024-04-25T09:59:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09dca06640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d80c5720 0x7f09d81f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09dca06640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d80c5720 0x7f09d81f1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47168/0 (socket says 172.21.15.19:47168) 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09dca06640 1 -- 172.21.15.19:0/2271237046 learned_addr learned my addr 172.21.15.19:0/2271237046 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09dda08640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81f4240 0x7f09d80b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09dd207640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c00039e0 0x7f09d81f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09dca06640 1 -- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81f4240 msgr2=0x7f09d80b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09dca06640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81f4240 0x7f09d80b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09dca06640 1 -- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c00039e0 msgr2=0x7f09d81f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09dca06640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c00039e0 0x7f09d81f26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09dca06640 1 -- 172.21.15.19:0/2271237046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80ac4d0 con 0x7f09d80c5720 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09dca06640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d80c5720 0x7f09d81f1fc0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00f850 tx=0x7f09cc00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09d5ffb640 1 -- 172.21.15.19:0/2271237046 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00fe40 con 0x7f09d80c5720 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.681+0000 7f09d5ffb640 1 -- 172.21.15.19:0/2271237046 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc018070 con 0x7f09d80c5720 2024-04-25T09:59:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.682+0000 7f09d5ffb640 1 -- 172.21.15.19:0/2271237046 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00d730 con 0x7f09d80c5720 2024-04-25T09:59:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.682+0000 7f09df46b640 1 -- 172.21.15.19:0/2271237046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d80ac7b0 con 0x7f09d80c5720 2024-04-25T09:59:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.682+0000 7f09df46b640 1 -- 172.21.15.19:0/2271237046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d80acd70 con 0x7f09d80c5720 2024-04-25T09:59:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.683+0000 7f09df46b640 1 -- 172.21.15.19:0/2271237046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d8073a00 con 0x7f09d80c5720 2024-04-25T09:59:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.683+0000 7f09d5ffb640 1 -- 172.21.15.19:0/2271237046 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09cc00d8d0 con 0x7f09d80c5720 2024-04-25T09:59:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.683+0000 7f09d5ffb640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f09c406b4d0 0x7f09c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.684+0000 7f09d5ffb640 1 -- 172.21.15.19:0/2271237046 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09cc08a180 con 0x7f09d80c5720 2024-04-25T09:59:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.684+0000 7f09dd207640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f09c406b4d0 0x7f09c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.684+0000 7f09dd207640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f09c406b4d0 0x7f09c406d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f09d0032000 tx=0x7f09d0009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.688+0000 7f09d5ffb640 1 -- 172.21.15.19:0/2271237046 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09cc058780 con 0x7f09d80c5720 2024-04-25T09:59:33.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.867+0000 7f09df46b640 1 -- 172.21.15.19:0/2271237046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09d80afef0 con 0x7f09d80c5720 2024-04-25T09:59:33.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.868+0000 7f09d5ffb640 1 -- 172.21.15.19:0/2271237046 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09cc05c820 con 0x7f09d80c5720 2024-04-25T09:59:33.872 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:59:33.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 -- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f09c406b4d0 msgr2=0x7f09c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f09c406b4d0 0x7f09c406d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f09d0032000 tx=0x7f09d0009030 comp rx=0 tx=0).stop 2024-04-25T09:59:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 -- 172.21.15.19:0/2271237046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d80c5720 msgr2=0x7f09d81f1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d80c5720 0x7f09d81f1fc0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00f850 tx=0x7f09cc00fd10 comp rx=0 tx=0).stop 2024-04-25T09:59:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 -- 172.21.15.19:0/2271237046 shutdown_connections 2024-04-25T09:59:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f09c406b4d0 0x7f09c406d990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09d81f4240 0x7f09d80b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d80c5720 0x7f09d81f1fc0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 --2- 172.21.15.19:0/2271237046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c00039e0 0x7f09d81f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 -- 172.21.15.19:0/2271237046 >> 172.21.15.19:0/2271237046 conn(0x7f09d806abe0 msgr2=0x7f09d806e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 -- 172.21.15.19:0/2271237046 shutdown_connections 2024-04-25T09:59:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.870+0000 7f09df46b640 1 -- 172.21.15.19:0/2271237046 wait complete. 2024-04-25T09:59:33.885 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:59:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.984+0000 7f989ed9e640 1 Processor -- start 2024-04-25T09:59:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.985+0000 7f989ed9e640 1 -- start start 2024-04-25T09:59:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.985+0000 7f989ed9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815dd30 0x7f98980cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.985+0000 7f989ed9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f989806bff0 0x7f98980b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.985+0000 7f989ed9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989815e110 con 0x7f98980b5970 2024-04-25T09:59:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.985+0000 7f989ed9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989806d000 con 0x7f989806bff0 2024-04-25T09:59:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.985+0000 7f989ed9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98980b2fd0 con 0x7f989815dd30 2024-04-25T09:59:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989e59d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f98980b5970 0x7f9898161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45806/0 (socket says 172.21.15.19:45806) 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989e59d640 1 -- 172.21.15.19:0/2307446501 learned_addr learned my addr 172.21.15.19:0/2307446501 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989d59b640 1 --2- 172.21.15.19:0/2307446501 >> v2:172.21.15.112:3300/0 conn(0x7f989806bff0 0x7f98980b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913134394 0 0) 0x7f989815e110 con 0x7f98980b5970 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989dd9c640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815dd30 0x7f98980cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9880003520 con 0x7f98980b5970 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2834735449 0 0) 0x7f9880003520 con 0x7f98980b5970 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815dd30 msgr2=0x7f98980cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815dd30 0x7f98980cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 >> v2:172.21.15.112:3300/0 conn(0x7f989806bff0 msgr2=0x7f98980b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 --2- 172.21.15.19:0/2307446501 >> v2:172.21.15.112:3300/0 conn(0x7f989806bff0 0x7f98980b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989806c480 con 0x7f98980b5970 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f9888001670 con 0x7f98980b5970 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 >> v1:172.21.15.19:6789/0 conn(0x7f98980b5970 legacy=0x7f9898161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98800039b0 0x7f9880003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 0x7f9880004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815dd30 0x7f98800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989806d000 con 0x7f9880004800 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98980b2fd0 con 0x7f98800039b0 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000b9c0 con 0x7f989815dd30 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989d59b640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 0x7f9880004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989dd9c640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98800039b0 0x7f9880003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.986+0000 7f989e59d640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815dd30 0x7f98800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.987+0000 7f989d59b640 1 -- 172.21.15.19:0/2307446501 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815dd30 msgr2=0x7f98800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.987+0000 7f989d59b640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815dd30 0x7f98800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.987+0000 7f989d59b640 1 -- 172.21.15.19:0/2307446501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98800039b0 msgr2=0x7f9880003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.987+0000 7f989d59b640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98800039b0 0x7f9880003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.987+0000 7f989d59b640 1 -- 172.21.15.19:0/2307446501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9880003520 con 0x7f9880004800 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.987+0000 7f989d59b640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 0x7f9880004c80 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f988c009ac0 tx=0x7f9880005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e11471c289b82a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.987+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c036070 con 0x7f9880004800 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.988+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c031400 con 0x7f9880004800 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.988+0000 7f989cd9a640 1 -- 172.21.15.19:0/2307446501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c009690 con 0x7f9880004800 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.988+0000 7f989ed9e640 1 -- 172.21.15.19:0/2307446501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 msgr2=0x7f9880004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.988+0000 7f989ed9e640 1 --2- 172.21.15.19:0/2307446501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 0x7f9880004c80 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f988c009ac0 tx=0x7f9880005830 comp rx=0 tx=0).stop 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.988+0000 7f989e59d640 1 -- 172.21.15.19:0/2307446501 reap_dead start 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.988+0000 7f989ed9e640 1 -- 172.21.15.19:0/2307446501 shutdown_connections 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.988+0000 7f989ed9e640 1 -- 172.21.15.19:0/2307446501 >> 172.21.15.19:0/2307446501 conn(0x7f98980d0610 msgr2=0x7f98980d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.988+0000 7f989ed9e640 1 -- 172.21.15.19:0/2307446501 shutdown_connections 2024-04-25T09:59:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.988+0000 7f989ed9e640 1 -- 172.21.15.19:0/2307446501 wait complete. 2024-04-25T09:59:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.989+0000 7f989ed9e640 1 Processor -- start 2024-04-25T09:59:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989ed9e640 1 -- start start 2024-04-25T09:59:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989ed9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 0x7f98981607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989ed9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989815e2c0 0x7f9898155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989ed9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815e740 0x7f9898155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989ed9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989806cd20 con 0x7f9880004800 2024-04-25T09:59:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989ed9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989806ca40 con 0x7f989815e2c0 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989ed9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98980d3170 con 0x7f989815e740 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989dd9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 0x7f98981607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989dd9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 0x7f98981607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47188/0 (socket says 172.21.15.19:47188) 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989dd9c640 1 -- 172.21.15.19:0/2700315851 learned_addr learned my addr 172.21.15.19:0/2700315851 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989d59b640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989815e2c0 0x7f9898155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.990+0000 7f989e59d640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815e740 0x7f9898155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f989dd9c640 1 -- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815e740 msgr2=0x7f9898155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f989dd9c640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815e740 0x7f9898155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f989dd9c640 1 -- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989815e2c0 msgr2=0x7f9898155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f989dd9c640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989815e2c0 0x7f9898155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f989dd9c640 1 -- 172.21.15.19:0/2700315851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989815ec80 con 0x7f9880004800 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f989d59b640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989815e2c0 0x7f9898155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f989e59d640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815e740 0x7f9898155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f989dd9c640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 0x7f98981607c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9890009f90 tx=0x7f989000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f98967fc640 1 -- 172.21.15.19:0/2700315851 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989000fea0 con 0x7f9880004800 2024-04-25T09:59:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f98967fc640 1 -- 172.21.15.19:0/2700315851 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9890015630 con 0x7f9880004800 2024-04-25T09:59:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f989ed9e640 1 -- 172.21.15.19:0/2700315851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989815ef60 con 0x7f9880004800 2024-04-25T09:59:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f98967fc640 1 -- 172.21.15.19:0/2700315851 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890015a20 con 0x7f9880004800 2024-04-25T09:59:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.991+0000 7f989ed9e640 1 -- 172.21.15.19:0/2700315851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989815f520 con 0x7f9880004800 2024-04-25T09:59:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.992+0000 7f989ed9e640 1 -- 172.21.15.19:0/2700315851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9898073990 con 0x7f9880004800 2024-04-25T09:59:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.992+0000 7f98967fc640 1 -- 172.21.15.19:0/2700315851 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9890015bc0 con 0x7f9880004800 2024-04-25T09:59:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.993+0000 7f98967fc640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988406b500 0x7f988406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.993+0000 7f98967fc640 1 -- 172.21.15.19:0/2700315851 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f989008b4a0 con 0x7f9880004800 2024-04-25T09:59:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.993+0000 7f989d59b640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988406b500 0x7f988406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.994+0000 7f989d59b640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988406b500 0x7f988406d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f988c033000 tx=0x7f988c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:34.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:33.997+0000 7f98967fc640 1 -- 172.21.15.19:0/2700315851 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9890019030 con 0x7f9880004800 2024-04-25T09:59:34.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.176+0000 7f989ed9e640 1 -- 172.21.15.19:0/2700315851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f989815cf60 con 0x7f9880004800 2024-04-25T09:59:34.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.176+0000 7f98967fc640 1 -- 172.21.15.19:0/2700315851 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9890010040 con 0x7f9880004800 2024-04-25T09:59:34.180 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T09:59:34.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 -- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988406b500 msgr2=0x7f988406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988406b500 0x7f988406d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f988c033000 tx=0x7f988c008030 comp rx=0 tx=0).stop 2024-04-25T09:59:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 -- 172.21.15.19:0/2700315851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 msgr2=0x7f98981607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 0x7f98981607c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9890009f90 tx=0x7f989000f580 comp rx=0 tx=0).stop 2024-04-25T09:59:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 -- 172.21.15.19:0/2700315851 shutdown_connections 2024-04-25T09:59:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988406b500 0x7f988406d9c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f989815e740 0x7f9898155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989815e2c0 0x7f9898155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 --2- 172.21.15.19:0/2700315851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9880004800 0x7f98981607c0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 -- 172.21.15.19:0/2700315851 >> 172.21.15.19:0/2700315851 conn(0x7f98980d0610 msgr2=0x7f98980b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 -- 172.21.15.19:0/2700315851 shutdown_connections 2024-04-25T09:59:34.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.178+0000 7f989ed9e640 1 -- 172.21.15.19:0/2700315851 wait complete. 2024-04-25T09:59:34.193 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:59:34.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.293+0000 7facdc9d5640 1 Processor -- start 2024-04-25T09:59:34.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.293+0000 7facdc9d5640 1 -- start start 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facdc9d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7facd80cd4c0 0x7facd80cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facdc9d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facd8150b90 0x7facd8151020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facdc9d5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd815e6e0 con 0x7facd815b1c0 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facdc9d5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd8151560 con 0x7facd80cd4c0 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facdc9d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd815e9c0 con 0x7facd8150b90 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facd659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7facd80cd4c0 0x7facd80cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facd659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7facd80cd4c0 0x7facd80cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47204/0 (socket says 172.21.15.19:47204) 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facd659c640 1 -- 172.21.15.19:0/3708858339 learned_addr learned my addr 172.21.15.19:0/3708858339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facd6d9d640 1 --2- 172.21.15.19:0/3708858339 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facd8150b90 0x7facd8151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facd659c640 1 -- 172.21.15.19:0/3708858339 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facd8150b90 msgr2=0x7facd8151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facd659c640 1 --2- 172.21.15.19:0/3708858339 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facd8150b90 0x7facd8151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facd659c640 1 -- 172.21.15.19:0/3708858339 >> v1:172.21.15.112:6789/0 conn(0x7facd815b1c0 legacy=0x7facd815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facd659c640 1 -- 172.21.15.19:0/3708858339 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd815a9c0 con 0x7facd80cd4c0 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.294+0000 7facd6d9d640 1 --2- 172.21.15.19:0/3708858339 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facd8150b90 0x7facd8151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.295+0000 7facd659c640 1 --2- 172.21.15.19:0/3708858339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facd80cd4c0 0x7facd80cd8a0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7facc800bb10 tx=0x7facc802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e903d20a4faf623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.295+0000 7facd5d9b640 1 -- 172.21.15.19:0/3708858339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8036020 con 0x7facd80cd4c0 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.295+0000 7facd5d9b640 1 -- 172.21.15.19:0/3708858339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc8032820 con 0x7facd80cd4c0 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.295+0000 7facd5d9b640 1 -- 172.21.15.19:0/3708858339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8032bf0 con 0x7facd80cd4c0 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.296+0000 7facdc9d5640 1 -- 172.21.15.19:0/3708858339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facd80cd4c0 msgr2=0x7facd80cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.296+0000 7facdc9d5640 1 --2- 172.21.15.19:0/3708858339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facd80cd4c0 0x7facd80cd8a0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7facc800bb10 tx=0x7facc802c920 comp rx=0 tx=0).stop 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.296+0000 7facdc9d5640 1 -- 172.21.15.19:0/3708858339 shutdown_connections 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.296+0000 7facdc9d5640 1 --2- 172.21.15.19:0/3708858339 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facd8150b90 0x7facd8151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.296+0000 7facdc9d5640 1 --2- 172.21.15.19:0/3708858339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facd80cd4c0 0x7facd80cd8a0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.296+0000 7facdc9d5640 1 -- 172.21.15.19:0/3708858339 >> 172.21.15.19:0/3708858339 conn(0x7facd806abe0 msgr2=0x7facd806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.296+0000 7facdc9d5640 1 -- 172.21.15.19:0/3708858339 shutdown_connections 2024-04-25T09:59:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.296+0000 7facdc9d5640 1 -- 172.21.15.19:0/3708858339 wait complete. 2024-04-25T09:59:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.297+0000 7facdc9d5640 1 Processor -- start 2024-04-25T09:59:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.297+0000 7facdc9d5640 1 -- start start 2024-04-25T09:59:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facdc9d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facd80cd4c0 0x7facd81f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facdc9d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8150b90 0x7facd80aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facdc9d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facd815b1c0 0x7facd81f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facdc9d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd815e370 con 0x7facd815b1c0 2024-04-25T09:59:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facdc9d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd815a670 con 0x7facd8150b90 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facdc9d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd806eca0 con 0x7facd80cd4c0 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facd6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facd815b1c0 0x7facd81f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facd6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facd815b1c0 0x7facd81f7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47216/0 (socket says 172.21.15.19:47216) 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facd6d9d640 1 -- 172.21.15.19:0/2037993771 learned_addr learned my addr 172.21.15.19:0/2037993771 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7faccffff640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8150b90 0x7facd80aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facd659c640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facd80cd4c0 0x7facd81f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facd6d9d640 1 -- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facd80cd4c0 msgr2=0x7facd81f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facd6d9d640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facd80cd4c0 0x7facd81f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facd6d9d640 1 -- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8150b90 msgr2=0x7facd80aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facd6d9d640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8150b90 0x7facd80aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facd6d9d640 1 -- 172.21.15.19:0/2037993771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd80b0490 con 0x7facd815b1c0 2024-04-25T09:59:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facd6d9d640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facd815b1c0 0x7facd81f7030 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7facd000cef0 tx=0x7facd000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.298+0000 7facceffd640 1 -- 172.21.15.19:0/2037993771 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd001a070 con 0x7facd815b1c0 2024-04-25T09:59:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.299+0000 7facceffd640 1 -- 172.21.15.19:0/2037993771 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd000d860 con 0x7facd815b1c0 2024-04-25T09:59:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.299+0000 7facdc9d5640 1 -- 172.21.15.19:0/2037993771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd81f7bf0 con 0x7facd815b1c0 2024-04-25T09:59:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.299+0000 7facceffd640 1 -- 172.21.15.19:0/2037993771 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd000dc50 con 0x7facd815b1c0 2024-04-25T09:59:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.299+0000 7facdc9d5640 1 -- 172.21.15.19:0/2037993771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd80af470 con 0x7facd815b1c0 2024-04-25T09:59:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.300+0000 7facdc9d5640 1 -- 172.21.15.19:0/2037993771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd8073a00 con 0x7facd815b1c0 2024-04-25T09:59:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.300+0000 7facceffd640 1 -- 172.21.15.19:0/2037993771 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7facd001d070 con 0x7facd815b1c0 2024-04-25T09:59:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.300+0000 7facceffd640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7facb806b4d0 0x7facb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.300+0000 7facceffd640 1 -- 172.21.15.19:0/2037993771 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7facd008aac0 con 0x7facd815b1c0 2024-04-25T09:59:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.301+0000 7facd659c640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7facb806b4d0 0x7facb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.301+0000 7facd659c640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7facb806b4d0 0x7facb806d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7facc8037000 tx=0x7facc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:34.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.305+0000 7facceffd640 1 -- 172.21.15.19:0/2037993771 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7facd00592b0 con 0x7facd815b1c0 2024-04-25T09:59:34.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.483+0000 7facdc9d5640 1 -- 172.21.15.19:0/2037993771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7facd80b1770 con 0x7facd815b1c0 2024-04-25T09:59:34.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.483+0000 7facceffd640 1 -- 172.21.15.19:0/2037993771 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7facd005d350 con 0x7facd815b1c0 2024-04-25T09:59:34.488 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T09:59:34.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 -- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7facb806b4d0 msgr2=0x7facb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7facb806b4d0 0x7facb806d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7facc8037000 tx=0x7facc8008040 comp rx=0 tx=0).stop 2024-04-25T09:59:34.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 -- 172.21.15.19:0/2037993771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facd815b1c0 msgr2=0x7facd81f7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facd815b1c0 0x7facd81f7030 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7facd000cef0 tx=0x7facd000a710 comp rx=0 tx=0).stop 2024-04-25T09:59:34.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 -- 172.21.15.19:0/2037993771 shutdown_connections 2024-04-25T09:59:34.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7facb806b4d0 0x7facb806d990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7facd815b1c0 0x7facd81f7030 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8150b90 0x7facd80aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 --2- 172.21.15.19:0/2037993771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facd80cd4c0 0x7facd81f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 -- 172.21.15.19:0/2037993771 >> 172.21.15.19:0/2037993771 conn(0x7facd806abe0 msgr2=0x7facd814fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:34.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 -- 172.21.15.19:0/2037993771 shutdown_connections 2024-04-25T09:59:34.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.485+0000 7facdc9d5640 1 -- 172.21.15.19:0/2037993771 wait complete. 2024-04-25T09:59:34.500 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:34.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.597+0000 7f2f0bf34640 1 Processor -- start 2024-04-25T09:59:34.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.597+0000 7f2f0bf34640 1 -- start start 2024-04-25T09:59:34.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f0bf34640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f0415dbe0 0x7f2f040d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f0bf34640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f040b2cb0 0x7f2f040d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f0bf34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f040b92a0 0x7f2f040b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f0bf34640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0415dfc0 con 0x7f2f040b2cb0 2024-04-25T09:59:34.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f0bf34640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f040d1a00 con 0x7f2f0415dbe0 2024-04-25T09:59:34.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f0bf34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040b97a0 con 0x7f2f040b92a0 2024-04-25T09:59:34.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f094cf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f040b2cb0 0x7f2f040d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f094cf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f040b2cb0 0x7f2f040d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47230/0 (socket says 172.21.15.19:47230) 2024-04-25T09:59:34.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f094cf640 1 -- 172.21.15.19:0/3024313968 learned_addr learned my addr 172.21.15.19:0/3024313968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:34.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f0a4d1640 1 --2- 172.21.15.19:0/3024313968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f040b92a0 0x7f2f040b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f09cd0640 1 --2- 172.21.15.19:0/3024313968 >> v2:172.21.15.112:3300/0 conn(0x7f2f0415dbe0 0x7f2f040d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.598+0000 7f2f094cf640 1 -- 172.21.15.19:0/3024313968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f040b92a0 msgr2=0x7f2f040b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:34.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.599+0000 7f2f094cf640 1 --2- 172.21.15.19:0/3024313968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f040b92a0 0x7f2f040b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.599+0000 7f2f094cf640 1 -- 172.21.15.19:0/3024313968 >> v2:172.21.15.112:3300/0 conn(0x7f2f0415dbe0 msgr2=0x7f2f040d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:34.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.599+0000 7f2f094cf640 1 --2- 172.21.15.19:0/3024313968 >> v2:172.21.15.112:3300/0 conn(0x7f2f0415dbe0 0x7f2f040d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.599+0000 7f2f094cf640 1 -- 172.21.15.19:0/3024313968 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f040d1c50 con 0x7f2f040b2cb0 2024-04-25T09:59:34.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.599+0000 7f2f094cf640 1 --2- 172.21.15.19:0/3024313968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f040b2cb0 0x7f2f040d0e50 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2f0406c140 tx=0x7f2ef802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf3e9d105294064 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:34.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.599+0000 7f2f08cce640 1 -- 172.21.15.19:0/3024313968 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8037070 con 0x7f2f040b2cb0 2024-04-25T09:59:34.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.599+0000 7f2f08cce640 1 -- 172.21.15.19:0/3024313968 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef8032440 con 0x7f2f040b2cb0 2024-04-25T09:59:34.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.599+0000 7f2f08cce640 1 -- 172.21.15.19:0/3024313968 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8032810 con 0x7f2f040b2cb0 2024-04-25T09:59:34.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.600+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3024313968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f040b2cb0 msgr2=0x7f2f040d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.600+0000 7f2f0bf34640 1 --2- 172.21.15.19:0/3024313968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f040b2cb0 0x7f2f040d0e50 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2f0406c140 tx=0x7f2ef802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.600+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3024313968 shutdown_connections 2024-04-25T09:59:34.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.600+0000 7f2f0bf34640 1 --2- 172.21.15.19:0/3024313968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f040b92a0 0x7f2f040b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.600+0000 7f2f0bf34640 1 --2- 172.21.15.19:0/3024313968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f040b2cb0 0x7f2f040d0e50 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.600+0000 7f2f0bf34640 1 --2- 172.21.15.19:0/3024313968 >> v2:172.21.15.112:3300/0 conn(0x7f2f0415dbe0 0x7f2f040d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.600+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3024313968 >> 172.21.15.19:0/3024313968 conn(0x7f2f04061a20 msgr2=0x7f2f04061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:34.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.600+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3024313968 shutdown_connections 2024-04-25T09:59:34.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.600+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3024313968 wait complete. 2024-04-25T09:59:34.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.601+0000 7f2f0bf34640 1 Processor -- start 2024-04-25T09:59:34.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.601+0000 7f2f0bf34640 1 -- start start 2024-04-25T09:59:34.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f0bf34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f040b2cb0 0x7f2f040c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f0bf34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f040b92a0 0x7f2f0415c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f0bf34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0415dbe0 0x7f2f0415cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f0bf34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040b6ab0 con 0x7f2f0415dbe0 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f0bf34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040d2210 con 0x7f2f040b2cb0 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f0bf34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040b6c30 con 0x7f2f040b92a0 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f0a4d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0415dbe0 0x7f2f0415cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f0a4d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0415dbe0 0x7f2f0415cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47238/0 (socket says 172.21.15.19:47238) 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f0a4d1640 1 -- 172.21.15.19:0/3580556613 learned_addr learned my addr 172.21.15.19:0/3580556613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f09cd0640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f040b2cb0 0x7f2f040c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f094cf640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f040b92a0 0x7f2f0415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.602+0000 7f2f0a4d1640 1 -- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f040b92a0 msgr2=0x7f2f0415c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.603+0000 7f2f0a4d1640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f040b92a0 0x7f2f0415c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.603+0000 7f2f0a4d1640 1 -- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f040b2cb0 msgr2=0x7f2f040c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.603+0000 7f2f0a4d1640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f040b2cb0 0x7f2f040c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.603+0000 7f2f0a4d1640 1 -- 172.21.15.19:0/3580556613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f040cabf0 con 0x7f2f0415dbe0 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.603+0000 7f2f0a4d1640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0415dbe0 0x7f2f0415cdd0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4007e60 tx=0x7f2ef400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.603+0000 7f2f027fc640 1 -- 172.21.15.19:0/3580556613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4014620 con 0x7f2f0415dbe0 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.603+0000 7f2f027fc640 1 -- 172.21.15.19:0/3580556613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef4019070 con 0x7f2f0415dbe0 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.603+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3580556613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f040caed0 con 0x7f2f0415dbe0 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.603+0000 7f2f027fc640 1 -- 172.21.15.19:0/3580556613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4014a00 con 0x7f2f0415dbe0 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.603+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3580556613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f040cb490 con 0x7f2f0415dbe0 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.604+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3580556613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f04073990 con 0x7f2f0415dbe0 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.604+0000 7f2f027fc640 1 -- 172.21.15.19:0/3580556613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ef4014ba0 con 0x7f2f0415dbe0 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.605+0000 7f2f027fc640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef006b4d0 0x7f2ef006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.605+0000 7f2f027fc640 1 -- 172.21.15.19:0/3580556613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ef408ab50 con 0x7f2f0415dbe0 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.605+0000 7f2f09cd0640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef006b4d0 0x7f2ef006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.606+0000 7f2f09cd0640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef006b4d0 0x7f2ef006d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2efc007920 tx=0x7f2efc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:34.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.609+0000 7f2f027fc640 1 -- 172.21.15.19:0/3580556613 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ef40591e0 con 0x7f2f0415dbe0 2024-04-25T09:59:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.825+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3580556613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f040b7960 con 0x7f2f0415dbe0 2024-04-25T09:59:34.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.825+0000 7f2f027fc640 1 -- 172.21.15.19:0/3580556613 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f2ef405d280 con 0x7f2f0415dbe0 2024-04-25T09:59:34.829 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:34.829 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3431952054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3431952054},{"type":"v1","addr":"172.21.15.112:6836","nonce":3431952054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2758109099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2758109099},{"type":"v1","addr":"172.21.15.112:6837","nonce":2758109099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7590,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3063446116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3063446116},{"type":"v1","addr":"172.21.15.19:6834","nonce":3063446116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:07.729627+0000","modified":"2024-04-25T09:59:09.720756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3589967477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3589967477},{"type":"v1","addr":"172.21.15.19:6835","nonce":3589967477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:34.830 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-04-25T09:59:34.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef006b4d0 msgr2=0x7f2ef006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef006b4d0 0x7f2ef006d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2efc007920 tx=0x7f2efc008040 comp rx=0 tx=0).stop 2024-04-25T09:59:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3580556613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0415dbe0 msgr2=0x7f2f0415cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0415dbe0 0x7f2f0415cdd0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4007e60 tx=0x7f2ef400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3580556613 shutdown_connections 2024-04-25T09:59:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef006b4d0 0x7f2ef006d990 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0415dbe0 0x7f2f0415cdd0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f040b92a0 0x7f2f0415c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 --2- 172.21.15.19:0/3580556613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f040b2cb0 0x7f2f040c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3580556613 >> 172.21.15.19:0/3580556613 conn(0x7f2f04061a20 msgr2=0x7f2f040b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3580556613 shutdown_connections 2024-04-25T09:59:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.827+0000 7f2f0bf34640 1 -- 172.21.15.19:0/3580556613 wait complete. 2024-04-25T09:59:34.842 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:34.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.939+0000 7f73aa1d5640 1 Processor -- start 2024-04-25T09:59:34.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.939+0000 7f73aa1d5640 1 -- start start 2024-04-25T09:59:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73aa1d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73a415db70 0x7f73a4162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73aa1d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73a40b6cb0 0x7f73a40b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73aa1d5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f73a40b5a60 0x7f73a406c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73aa1d5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a4162690 con 0x7f73a415db70 2024-04-25T09:59:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73aa1d5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a40ceee0 con 0x7f73a40b5a60 2024-04-25T09:59:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73aa1d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40d1d10 con 0x7f73a40b6cb0 2024-04-25T09:59:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73a37fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73a415db70 0x7f73a4162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73a37fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73a415db70 0x7f73a4162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47242/0 (socket says 172.21.15.19:47242) 2024-04-25T09:59:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73a37fe640 1 -- 172.21.15.19:0/573568195 learned_addr learned my addr 172.21.15.19:0/573568195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73a37fe640 1 -- 172.21.15.19:0/573568195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73a40b6cb0 msgr2=0x7f73a40b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73a3fff640 1 --2- 172.21.15.19:0/573568195 >> v2:172.21.15.112:3300/0 conn(0x7f73a40b5a60 0x7f73a406c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73a2ffd640 1 --2- 172.21.15.19:0/573568195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73a40b6cb0 0x7f73a40b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73a37fe640 1 --2- 172.21.15.19:0/573568195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73a40b6cb0 0x7f73a40b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73a37fe640 1 -- 172.21.15.19:0/573568195 >> v2:172.21.15.112:3300/0 conn(0x7f73a40b5a60 msgr2=0x7f73a406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73a37fe640 1 --2- 172.21.15.19:0/573568195 >> v2:172.21.15.112:3300/0 conn(0x7f73a40b5a60 0x7f73a406c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.940+0000 7f73a37fe640 1 -- 172.21.15.19:0/573568195 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a40d1ff0 con 0x7f73a415db70 2024-04-25T09:59:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.941+0000 7f73a37fe640 1 --2- 172.21.15.19:0/573568195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a415db70 0x7f73a4162150 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f73a408bfb0 tx=0x7f738c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a013dfd4747955 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.941+0000 7f73a27fc640 1 -- 172.21.15.19:0/573568195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c037070 con 0x7f73a415db70 2024-04-25T09:59:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.941+0000 7f73a27fc640 1 -- 172.21.15.19:0/573568195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738c032440 con 0x7f73a415db70 2024-04-25T09:59:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.941+0000 7f73a27fc640 1 -- 172.21.15.19:0/573568195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c032810 con 0x7f73a415db70 2024-04-25T09:59:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.941+0000 7f73aa1d5640 1 -- 172.21.15.19:0/573568195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a415db70 msgr2=0x7f73a4162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.942+0000 7f73aa1d5640 1 --2- 172.21.15.19:0/573568195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a415db70 0x7f73a4162150 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f73a408bfb0 tx=0x7f738c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.942+0000 7f73aa1d5640 1 -- 172.21.15.19:0/573568195 shutdown_connections 2024-04-25T09:59:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.942+0000 7f73aa1d5640 1 --2- 172.21.15.19:0/573568195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73a40b6cb0 0x7f73a40b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.942+0000 7f73aa1d5640 1 --2- 172.21.15.19:0/573568195 >> v2:172.21.15.112:3300/0 conn(0x7f73a40b5a60 0x7f73a406c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.942+0000 7f73aa1d5640 1 --2- 172.21.15.19:0/573568195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a415db70 0x7f73a4162150 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.942+0000 7f73aa1d5640 1 -- 172.21.15.19:0/573568195 >> 172.21.15.19:0/573568195 conn(0x7f73a40d0520 msgr2=0x7f73a40d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.942+0000 7f73aa1d5640 1 -- 172.21.15.19:0/573568195 shutdown_connections 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.942+0000 7f73aa1d5640 1 -- 172.21.15.19:0/573568195 wait complete. 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.943+0000 7f73aa1d5640 1 Processor -- start 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.943+0000 7f73aa1d5640 1 -- start start 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73aa1d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a40b5a60 0x7f73a4158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73aa1d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a40b6cb0 0x7f73a40cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73aa1d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73a415db70 0x7f73a415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73aa1d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40ce530 con 0x7f73a40b5a60 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73aa1d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a415df50 con 0x7f73a40b6cb0 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73aa1d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40cf1c0 con 0x7f73a415db70 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a40b5a60 0x7f73a4158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a40b5a60 0x7f73a4158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47250/0 (socket says 172.21.15.19:47250) 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a37fe640 1 -- 172.21.15.19:0/2636563224 learned_addr learned my addr 172.21.15.19:0/2636563224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a3fff640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73a415db70 0x7f73a415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a2ffd640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a40b6cb0 0x7f73a40cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a37fe640 1 -- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73a415db70 msgr2=0x7f73a415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a37fe640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73a415db70 0x7f73a415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a37fe640 1 -- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a40b6cb0 msgr2=0x7f73a40cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a37fe640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a40b6cb0 0x7f73a40cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a37fe640 1 -- 172.21.15.19:0/2636563224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a41f7970 con 0x7f73a40b5a60 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a2ffd640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a40b6cb0 0x7f73a40cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f73a37fe640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a40b5a60 0x7f73a4158e40 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f738c000c00 tx=0x7f738c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.944+0000 7f739bfff640 1 -- 172.21.15.19:0/2636563224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c041070 con 0x7f73a40b5a60 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.945+0000 7f739bfff640 1 -- 172.21.15.19:0/2636563224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738c03c870 con 0x7f73a40b5a60 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.945+0000 7f73aa1d5640 1 -- 172.21.15.19:0/2636563224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a4159c20 con 0x7f73a40b5a60 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.945+0000 7f739bfff640 1 -- 172.21.15.19:0/2636563224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c037040 con 0x7f73a40b5a60 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.945+0000 7f73aa1d5640 1 -- 172.21.15.19:0/2636563224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a415a1e0 con 0x7f73a40b5a60 2024-04-25T09:59:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.945+0000 7f73aa1d5640 1 -- 172.21.15.19:0/2636563224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a4073990 con 0x7f73a40b5a60 2024-04-25T09:59:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.946+0000 7f739bfff640 1 -- 172.21.15.19:0/2636563224 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f738c044070 con 0x7f73a40b5a60 2024-04-25T09:59:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.946+0000 7f739bfff640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f738406b4d0 0x7f738406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.947+0000 7f739bfff640 1 -- 172.21.15.19:0/2636563224 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f738c0aae80 con 0x7f73a40b5a60 2024-04-25T09:59:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.947+0000 7f73a2ffd640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f738406b4d0 0x7f738406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.947+0000 7f73a2ffd640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f738406b4d0 0x7f738406d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f73a406cb80 tx=0x7f7394008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:34.951+0000 7f739bfff640 1 -- 172.21.15.19:0/2636563224 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f738c079640 con 0x7f73a40b5a60 2024-04-25T09:59:35.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.134+0000 7f73aa1d5640 1 -- 172.21.15.19:0/2636563224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73a40b8f70 con 0x7f73a40b5a60 2024-04-25T09:59:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.135+0000 7f739bfff640 1 -- 172.21.15.19:0/2636563224 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f738c07d6e0 con 0x7f73a40b5a60 2024-04-25T09:59:35.139 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:35.140 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":95,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:09.369855+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:06.008725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:06.705899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.137+0000 7f73aa1d5640 1 -- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f738406b4d0 msgr2=0x7f738406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.137+0000 7f73aa1d5640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f738406b4d0 0x7f738406d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f73a406cb80 tx=0x7f7394008040 comp rx=0 tx=0).stop 2024-04-25T09:59:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.137+0000 7f73aa1d5640 1 -- 172.21.15.19:0/2636563224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a40b5a60 msgr2=0x7f73a4158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.137+0000 7f73aa1d5640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a40b5a60 0x7f73a4158e40 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f738c000c00 tx=0x7f738c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.137+0000 7f73aa1d5640 1 -- 172.21.15.19:0/2636563224 shutdown_connections 2024-04-25T09:59:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.137+0000 7f73aa1d5640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f738406b4d0 0x7f738406d990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.137+0000 7f73aa1d5640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73a415db70 0x7f73a415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.137+0000 7f73aa1d5640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a40b6cb0 0x7f73a40cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.137+0000 7f73aa1d5640 1 --2- 172.21.15.19:0/2636563224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73a40b5a60 0x7f73a4158e40 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.137+0000 7f73aa1d5640 1 -- 172.21.15.19:0/2636563224 >> 172.21.15.19:0/2636563224 conn(0x7f73a40d0520 msgr2=0x7f73a40b7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.138+0000 7f73aa1d5640 1 -- 172.21.15.19:0/2636563224 shutdown_connections 2024-04-25T09:59:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.138+0000 7f73aa1d5640 1 -- 172.21.15.19:0/2636563224 wait complete. 2024-04-25T09:59:35.153 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:59:35.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.250+0000 7ff05be5d640 1 Processor -- start 2024-04-25T09:59:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.250+0000 7ff05be5d640 1 -- start start 2024-04-25T09:59:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.251+0000 7ff05be5d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff0540b2cb0 0x7ff0540d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.251+0000 7ff05be5d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff0540d1020 0x7ff0540d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.251+0000 7ff05be5d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0540b97a0 con 0x7ff05415dbe0 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.251+0000 7ff05be5d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05415dfc0 con 0x7ff0540d1020 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.251+0000 7ff05be5d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0540d1a00 con 0x7ff0540b2cb0 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.251+0000 7ff05a3fa640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff0540d1020 0x7ff0540d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.251+0000 7ff05a3fa640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff0540d1020 0x7ff0540d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47256/0 (socket says 172.21.15.19:47256) 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.251+0000 7ff059bf9640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff05415dbe0 0x7ff0540d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45810/0 (socket says 172.21.15.19:45810) 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.251+0000 7ff05a3fa640 1 -- 172.21.15.19:0/2851065075 learned_addr learned my addr 172.21.15.19:0/2851065075 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.252+0000 7ff0593f8640 1 --2- 172.21.15.19:0/2851065075 >> v2:172.21.15.112:3300/0 conn(0x7ff0540b2cb0 0x7ff0540d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.252+0000 7ff058bf7640 1 -- 172.21.15.19:0/2851065075 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132858039 0 0) 0x7ff0540b97a0 con 0x7ff05415dbe0 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.252+0000 7ff05a3fa640 1 -- 172.21.15.19:0/2851065075 >> v2:172.21.15.112:3300/0 conn(0x7ff0540b2cb0 msgr2=0x7ff0540d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.252+0000 7ff05a3fa640 1 --2- 172.21.15.19:0/2851065075 >> v2:172.21.15.112:3300/0 conn(0x7ff0540b2cb0 0x7ff0540d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.252+0000 7ff05a3fa640 1 -- 172.21.15.19:0/2851065075 >> v1:172.21.15.19:6789/0 conn(0x7ff05415dbe0 legacy=0x7ff0540d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.252+0000 7ff05a3fa640 1 -- 172.21.15.19:0/2851065075 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540d1c50 con 0x7ff0540d1020 2024-04-25T09:59:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.252+0000 7ff05a3fa640 1 --2- 172.21.15.19:0/2851065075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0540d1020 0x7ff0540d2c70 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff05406cff0 tx=0x7ff04c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8441943bc9d2224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.253+0000 7ff058bf7640 1 -- 172.21.15.19:0/2851065075 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c036070 con 0x7ff0540d1020 2024-04-25T09:59:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.253+0000 7ff058bf7640 1 -- 172.21.15.19:0/2851065075 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c031600 con 0x7ff0540d1020 2024-04-25T09:59:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.253+0000 7ff058bf7640 1 -- 172.21.15.19:0/2851065075 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c0319d0 con 0x7ff0540d1020 2024-04-25T09:59:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.253+0000 7ff05be5d640 1 -- 172.21.15.19:0/2851065075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0540d1020 msgr2=0x7ff0540d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.253+0000 7ff05be5d640 1 --2- 172.21.15.19:0/2851065075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0540d1020 0x7ff0540d2c70 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff05406cff0 tx=0x7ff04c02a730 comp rx=0 tx=0).stop 2024-04-25T09:59:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.253+0000 7ff05be5d640 1 -- 172.21.15.19:0/2851065075 shutdown_connections 2024-04-25T09:59:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.253+0000 7ff05be5d640 1 --2- 172.21.15.19:0/2851065075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0540d1020 0x7ff0540d2c70 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.253+0000 7ff05be5d640 1 --2- 172.21.15.19:0/2851065075 >> v2:172.21.15.112:3300/0 conn(0x7ff0540b2cb0 0x7ff0540d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.253+0000 7ff05be5d640 1 -- 172.21.15.19:0/2851065075 >> 172.21.15.19:0/2851065075 conn(0x7ff054061a20 msgr2=0x7ff054061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.253+0000 7ff05be5d640 1 -- 172.21.15.19:0/2851065075 shutdown_connections 2024-04-25T09:59:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.253+0000 7ff05be5d640 1 -- 172.21.15.19:0/2851065075 wait complete. 2024-04-25T09:59:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.254+0000 7ff05be5d640 1 Processor -- start 2024-04-25T09:59:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.254+0000 7ff05be5d640 1 -- start start 2024-04-25T09:59:35.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.255+0000 7ff05be5d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0540b2cb0 0x7ff05415fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.255+0000 7ff05be5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0540d1020 0x7ff054160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.255+0000 7ff05be5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05415dbe0 0x7ff054157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.255+0000 7ff05be5d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540b92a0 con 0x7ff05415dbe0 2024-04-25T09:59:35.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.255+0000 7ff05be5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540d2210 con 0x7ff0540d1020 2024-04-25T09:59:35.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.255+0000 7ff05be5d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540b9420 con 0x7ff0540b2cb0 2024-04-25T09:59:35.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.255+0000 7ff05a3fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05415dbe0 0x7ff054157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.255+0000 7ff05a3fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05415dbe0 0x7ff054157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47266/0 (socket says 172.21.15.19:47266) 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.255+0000 7ff05a3fa640 1 -- 172.21.15.19:0/3327609664 learned_addr learned my addr 172.21.15.19:0/3327609664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff0593f8640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0540d1020 0x7ff054160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff059bf9640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0540b2cb0 0x7ff05415fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff05a3fa640 1 -- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0540b2cb0 msgr2=0x7ff05415fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff05a3fa640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0540b2cb0 0x7ff05415fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff05a3fa640 1 -- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0540d1020 msgr2=0x7ff054160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff05a3fa640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0540d1020 0x7ff054160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff05a3fa640 1 -- 172.21.15.19:0/3327609664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05414c540 con 0x7ff05415dbe0 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff05a3fa640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05415dbe0 0x7ff054157c40 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff04c02ad20 tx=0x7ff04c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff04a7fc640 1 -- 172.21.15.19:0/3327609664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c036050 con 0x7ff05415dbe0 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff04a7fc640 1 -- 172.21.15.19:0/3327609664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c042070 con 0x7ff05415dbe0 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff05be5d640 1 -- 172.21.15.19:0/3327609664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05414c820 con 0x7ff05415dbe0 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff04a7fc640 1 -- 172.21.15.19:0/3327609664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c009510 con 0x7ff05415dbe0 2024-04-25T09:59:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.256+0000 7ff05be5d640 1 -- 172.21.15.19:0/3327609664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05414cde0 con 0x7ff05415dbe0 2024-04-25T09:59:35.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.257+0000 7ff05be5d640 1 -- 172.21.15.19:0/3327609664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff054073990 con 0x7ff05415dbe0 2024-04-25T09:59:35.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.257+0000 7ff04a7fc640 1 -- 172.21.15.19:0/3327609664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff04c007bf0 con 0x7ff05415dbe0 2024-04-25T09:59:35.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.258+0000 7ff04a7fc640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03806b4d0 0x7ff03806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.258+0000 7ff04a7fc640 1 -- 172.21.15.19:0/3327609664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff04c0ab1a0 con 0x7ff05415dbe0 2024-04-25T09:59:35.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.258+0000 7ff059bf9640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03806b4d0 0x7ff03806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.258+0000 7ff059bf9640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03806b4d0 0x7ff03806d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff050000f60 tx=0x7ff050007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:35.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.262+0000 7ff04a7fc640 1 -- 172.21.15.19:0/3327609664 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff04c045360 con 0x7ff05415dbe0 2024-04-25T09:59:35.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.484+0000 7ff05be5d640 1 -- 172.21.15.19:0/3327609664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff05414cfb0 con 0x7ff05415dbe0 2024-04-25T09:59:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.484+0000 7ff04a7fc640 1 -- 172.21.15.19:0/3327609664 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff04c079e90 con 0x7ff05415dbe0 2024-04-25T09:59:35.488 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:35.488 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3431952054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3431952054},{"type":"v1","addr":"172.21.15.112:6836","nonce":3431952054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2758109099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2758109099},{"type":"v1","addr":"172.21.15.112:6837","nonce":2758109099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7590,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3063446116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3063446116},{"type":"v1","addr":"172.21.15.19:6834","nonce":3063446116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:07.729627+0000","modified":"2024-04-25T09:59:09.720756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3589967477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3589967477},{"type":"v1","addr":"172.21.15.19:6835","nonce":3589967477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-04-25T09:59:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 -- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03806b4d0 msgr2=0x7ff03806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03806b4d0 0x7ff03806d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff050000f60 tx=0x7ff050007c00 comp rx=0 tx=0).stop 2024-04-25T09:59:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 -- 172.21.15.19:0/3327609664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05415dbe0 msgr2=0x7ff054157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05415dbe0 0x7ff054157c40 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff04c02ad20 tx=0x7ff04c004a40 comp rx=0 tx=0).stop 2024-04-25T09:59:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 -- 172.21.15.19:0/3327609664 shutdown_connections 2024-04-25T09:59:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff03806b4d0 0x7ff03806d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff05415dbe0 0x7ff054157c40 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0540d1020 0x7ff054160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 --2- 172.21.15.19:0/3327609664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0540b2cb0 0x7ff05415fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 -- 172.21.15.19:0/3327609664 >> 172.21.15.19:0/3327609664 conn(0x7ff054061a20 msgr2=0x7ff054159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 -- 172.21.15.19:0/3327609664 shutdown_connections 2024-04-25T09:59:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.486+0000 7ff05be5d640 1 -- 172.21.15.19:0/3327609664 wait complete. 2024-04-25T09:59:35.501 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:35.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.598+0000 7f8cb999e640 1 Processor -- start 2024-04-25T09:59:35.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.598+0000 7f8cb999e640 1 -- start start 2024-04-25T09:59:35.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb999e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb415dc20 0x7f8cb41616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb999e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cb40b93a0 0x7f8cb40ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb999e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb415e000 con 0x7f8cb40cec20 2024-04-25T09:59:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb999e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb40b6bb0 con 0x7f8cb40b93a0 2024-04-25T09:59:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb999e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40b6030 con 0x7f8cb415dc20 2024-04-25T09:59:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb27fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cb40b93a0 0x7f8cb40ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb27fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cb40b93a0 0x7f8cb40ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47282/0 (socket says 172.21.15.19:47282) 2024-04-25T09:59:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb27fc640 1 -- 172.21.15.19:0/1998947725 learned_addr learned my addr 172.21.15.19:0/1998947725 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb2ffd640 1 --2- 172.21.15.19:0/1998947725 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb415dc20 0x7f8cb41616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb27fc640 1 -- 172.21.15.19:0/1998947725 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb415dc20 msgr2=0x7f8cb41616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb27fc640 1 --2- 172.21.15.19:0/1998947725 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb415dc20 0x7f8cb41616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb27fc640 1 -- 172.21.15.19:0/1998947725 >> v1:172.21.15.112:6789/0 conn(0x7f8cb40cec20 legacy=0x7f8cb406c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:35.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.599+0000 7f8cb27fc640 1 -- 172.21.15.19:0/1998947725 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb4161de0 con 0x7f8cb40b93a0 2024-04-25T09:59:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.600+0000 7f8cb27fc640 1 --2- 172.21.15.19:0/1998947725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40b93a0 0x7f8cb40ce6e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009870 tx=0x7f8ca802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db18dc190ddb09a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.600+0000 7f8cb1ffb640 1 -- 172.21.15.19:0/1998947725 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8037070 con 0x7f8cb40b93a0 2024-04-25T09:59:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.600+0000 7f8cb1ffb640 1 -- 172.21.15.19:0/1998947725 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca8032440 con 0x7f8cb40b93a0 2024-04-25T09:59:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.600+0000 7f8cb1ffb640 1 -- 172.21.15.19:0/1998947725 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8032810 con 0x7f8cb40b93a0 2024-04-25T09:59:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.601+0000 7f8cb999e640 1 -- 172.21.15.19:0/1998947725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40b93a0 msgr2=0x7f8cb40ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.601+0000 7f8cb999e640 1 --2- 172.21.15.19:0/1998947725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40b93a0 0x7f8cb40ce6e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009870 tx=0x7f8ca802a210 comp rx=0 tx=0).stop 2024-04-25T09:59:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.601+0000 7f8cb999e640 1 -- 172.21.15.19:0/1998947725 shutdown_connections 2024-04-25T09:59:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.601+0000 7f8cb999e640 1 --2- 172.21.15.19:0/1998947725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40b93a0 0x7f8cb40ce6e0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.601+0000 7f8cb999e640 1 --2- 172.21.15.19:0/1998947725 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb415dc20 0x7f8cb41616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.601+0000 7f8cb999e640 1 -- 172.21.15.19:0/1998947725 >> 172.21.15.19:0/1998947725 conn(0x7f8cb40d0490 msgr2=0x7f8cb40d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.601+0000 7f8cb999e640 1 -- 172.21.15.19:0/1998947725 shutdown_connections 2024-04-25T09:59:35.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.601+0000 7f8cb999e640 1 -- 172.21.15.19:0/1998947725 wait complete. 2024-04-25T09:59:35.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.602+0000 7f8cb999e640 1 Processor -- start 2024-04-25T09:59:35.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.602+0000 7f8cb999e640 1 -- start start 2024-04-25T09:59:35.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb999e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40b93a0 0x7f8cb415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb999e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb40cec20 0x7f8cb4159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb999e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb415dc20 0x7f8cb40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb999e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40d3520 con 0x7f8cb40b93a0 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb999e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb41623a0 con 0x7f8cb40cec20 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb999e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40d36a0 con 0x7f8cb415dc20 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40b93a0 0x7f8cb415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40b93a0 0x7f8cb415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47284/0 (socket says 172.21.15.19:47284) 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb2ffd640 1 -- 172.21.15.19:0/1144260276 learned_addr learned my addr 172.21.15.19:0/1144260276 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb37fe640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb415dc20 0x7f8cb40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb27fc640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb40cec20 0x7f8cb4159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb2ffd640 1 -- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb415dc20 msgr2=0x7f8cb40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb2ffd640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb415dc20 0x7f8cb40cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb2ffd640 1 -- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb40cec20 msgr2=0x7f8cb4159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb2ffd640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb40cec20 0x7f8cb4159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb2ffd640 1 -- 172.21.15.19:0/1144260276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb415a450 con 0x7f8cb40b93a0 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb2ffd640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40b93a0 0x7f8cb415f320 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400c820 tx=0x7f8ca400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8c9bfff640 1 -- 172.21.15.19:0/1144260276 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4015070 con 0x7f8cb40b93a0 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8c9bfff640 1 -- 172.21.15.19:0/1144260276 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca4004240 con 0x7f8cb40b93a0 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.603+0000 7f8cb999e640 1 -- 172.21.15.19:0/1144260276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb415e280 con 0x7f8cb40b93a0 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.604+0000 7f8c9bfff640 1 -- 172.21.15.19:0/1144260276 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400ce10 con 0x7f8cb40b93a0 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.604+0000 7f8cb999e640 1 -- 172.21.15.19:0/1144260276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb415e840 con 0x7f8cb40b93a0 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.605+0000 7f8cb999e640 1 -- 172.21.15.19:0/1144260276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb4073990 con 0x7f8cb40b93a0 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.605+0000 7f8c9bfff640 1 -- 172.21.15.19:0/1144260276 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ca4007830 con 0x7f8cb40b93a0 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.605+0000 7f8c9bfff640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 0x7f8ca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.605+0000 7f8c9bfff640 1 -- 172.21.15.19:0/1144260276 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ca408a200 con 0x7f8cb40b93a0 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.606+0000 7f8cb27fc640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 0x7f8ca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.606+0000 7f8cb27fc640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 0x7f8ca006d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8034000 tx=0x7f8ca802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:35.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.610+0000 7f8c9bfff640 1 -- 172.21.15.19:0/1144260276 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ca4058ef0 con 0x7f8cb40b93a0 2024-04-25T09:59:35.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.827+0000 7f8cb999e640 1 -- 172.21.15.19:0/1144260276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cb40cf1e0 con 0x7f8cb40b93a0 2024-04-25T09:59:35.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.828+0000 7f8c9bfff640 1 -- 172.21.15.19:0/1144260276 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f8ca40587f0 con 0x7f8cb40b93a0 2024-04-25T09:59:35.832 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:35.832 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3431952054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3431952054},{"type":"v1","addr":"172.21.15.112:6836","nonce":3431952054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2758109099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2758109099},{"type":"v1","addr":"172.21.15.112:6837","nonce":2758109099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7590,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3063446116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3063446116},{"type":"v1","addr":"172.21.15.19:6834","nonce":3063446116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:07.729627+0000","modified":"2024-04-25T09:59:09.720756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3589967477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3589967477},{"type":"v1","addr":"172.21.15.19:6835","nonce":3589967477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:35.832 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-04-25T09:59:35.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.829+0000 7f8cb999e640 1 -- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 msgr2=0x7f8ca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.829+0000 7f8cb999e640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 0x7f8ca006d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8034000 tx=0x7f8ca802e000 comp rx=0 tx=0).stop 2024-04-25T09:59:35.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.830+0000 7f8cb999e640 1 -- 172.21.15.19:0/1144260276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40b93a0 msgr2=0x7f8cb415f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.830+0000 7f8cb999e640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40b93a0 0x7f8cb415f320 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400c820 tx=0x7f8ca400cce0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.830+0000 7f8cb999e640 1 -- 172.21.15.19:0/1144260276 shutdown_connections 2024-04-25T09:59:35.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.830+0000 7f8cb999e640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ca006b4d0 0x7f8ca006d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.830+0000 7f8cb999e640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cb415dc20 0x7f8cb40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.830+0000 7f8cb999e640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb40cec20 0x7f8cb4159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.830+0000 7f8cb999e640 1 --2- 172.21.15.19:0/1144260276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40b93a0 0x7f8cb415f320 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.830+0000 7f8cb999e640 1 -- 172.21.15.19:0/1144260276 >> 172.21.15.19:0/1144260276 conn(0x7f8cb40d0490 msgr2=0x7f8cb40b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:35.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.830+0000 7f8cb999e640 1 -- 172.21.15.19:0/1144260276 shutdown_connections 2024-04-25T09:59:35.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.830+0000 7f8cb999e640 1 -- 172.21.15.19:0/1144260276 wait complete. 2024-04-25T09:59:35.847 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.943+0000 7fde05038640 1 Processor -- start 2024-04-25T09:59:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.943+0000 7fde05038640 1 -- start start 2024-04-25T09:59:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.944+0000 7fde05038640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde0015ba00 0x7fde0009a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.944+0000 7fde05038640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde001602e0 0x7fde000ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.944+0000 7fde05038640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde000cd710 con 0x7fde0015f5f0 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.944+0000 7fde05038640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde00099b70 con 0x7fde001602e0 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.944+0000 7fde05038640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde00099e50 con 0x7fde0015ba00 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.944+0000 7fddfe59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde001602e0 0x7fde000ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.944+0000 7fddfe59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde001602e0 0x7fde000ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47296/0 (socket says 172.21.15.19:47296) 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.944+0000 7fddff59e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fde0015f5f0 0x7fde000c52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45826/0 (socket says 172.21.15.19:45826) 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.944+0000 7fddfe59c640 1 -- 172.21.15.19:0/2195237659 learned_addr learned my addr 172.21.15.19:0/2195237659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.945+0000 7fddfed9d640 1 --2- 172.21.15.19:0/2195237659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde0015ba00 0x7fde0009a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.945+0000 7fddfe59c640 1 -- 172.21.15.19:0/2195237659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde0015ba00 msgr2=0x7fde0009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.945+0000 7fddfe59c640 1 --2- 172.21.15.19:0/2195237659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde0015ba00 0x7fde0009a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.945+0000 7fddfe59c640 1 -- 172.21.15.19:0/2195237659 >> v1:172.21.15.19:6789/0 conn(0x7fde0015f5f0 legacy=0x7fde000c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.945+0000 7fddfe59c640 1 -- 172.21.15.19:0/2195237659 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0009a130 con 0x7fde001602e0 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.945+0000 7fddfdd9b640 1 -- 172.21.15.19:0/2195237659 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651400820 0 0) 0x7fddf8001490 con 0x7fde0015f5f0 2024-04-25T09:59:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.945+0000 7fddfed9d640 1 --2- 172.21.15.19:0/2195237659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde0015ba00 0x7fde0009a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.945+0000 7fddfe59c640 1 --2- 172.21.15.19:0/2195237659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde001602e0 0x7fde000ccda0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009870 tx=0x7fddf402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed9f40e12e77d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.946+0000 7fddfdd9b640 1 -- 172.21.15.19:0/2195237659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4037070 con 0x7fde001602e0 2024-04-25T09:59:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.946+0000 7fddfdd9b640 1 -- 172.21.15.19:0/2195237659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf4032440 con 0x7fde001602e0 2024-04-25T09:59:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.946+0000 7fddfdd9b640 1 -- 172.21.15.19:0/2195237659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4032810 con 0x7fde001602e0 2024-04-25T09:59:35.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.946+0000 7fde05038640 1 -- 172.21.15.19:0/2195237659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde001602e0 msgr2=0x7fde000ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.946+0000 7fde05038640 1 --2- 172.21.15.19:0/2195237659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde001602e0 0x7fde000ccda0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009870 tx=0x7fddf402a130 comp rx=0 tx=0).stop 2024-04-25T09:59:35.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.946+0000 7fde05038640 1 -- 172.21.15.19:0/2195237659 shutdown_connections 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.946+0000 7fde05038640 1 --2- 172.21.15.19:0/2195237659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde001602e0 0x7fde000ccda0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.946+0000 7fde05038640 1 --2- 172.21.15.19:0/2195237659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde0015ba00 0x7fde0009a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.946+0000 7fde05038640 1 -- 172.21.15.19:0/2195237659 >> 172.21.15.19:0/2195237659 conn(0x7fde000b3270 msgr2=0x7fde000b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.946+0000 7fde05038640 1 -- 172.21.15.19:0/2195237659 shutdown_connections 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.946+0000 7fde05038640 1 -- 172.21.15.19:0/2195237659 wait complete. 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.948+0000 7fde05038640 1 Processor -- start 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.948+0000 7fde05038640 1 -- start start 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.948+0000 7fde05038640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde0015ba00 0x7fde001f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.948+0000 7fde05038640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde0015f5f0 0x7fde001589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.948+0000 7fde05038640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde001602e0 0x7fde000c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.948+0000 7fde05038640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0015bde0 con 0x7fde0015ba00 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.948+0000 7fde05038640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000cd2e0 con 0x7fde001602e0 2024-04-25T09:59:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.948+0000 7fde05038640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde000b4510 con 0x7fde0015f5f0 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fddfed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde0015ba00 0x7fde001f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fddfed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde0015ba00 0x7fde001f3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47308/0 (socket says 172.21.15.19:47308) 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fddfed9d640 1 -- 172.21.15.19:0/2865018559 learned_addr learned my addr 172.21.15.19:0/2865018559 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fddff59e640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde001602e0 0x7fde000c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fddfe59c640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde0015f5f0 0x7fde001589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fddfed9d640 1 -- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde0015f5f0 msgr2=0x7fde001589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fddfed9d640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde0015f5f0 0x7fde001589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fddfed9d640 1 -- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde001602e0 msgr2=0x7fde000c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fddfed9d640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde001602e0 0x7fde000c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fddfed9d640 1 -- 172.21.15.19:0/2865018559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000c8850 con 0x7fde0015ba00 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fddfed9d640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde0015ba00 0x7fde001f3ac0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fddf000c7f0 tx=0x7fddf000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fdde77fe640 1 -- 172.21.15.19:0/2865018559 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0015070 con 0x7fde0015ba00 2024-04-25T09:59:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fdde77fe640 1 -- 172.21.15.19:0/2865018559 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf0007820 con 0x7fde0015ba00 2024-04-25T09:59:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fde05038640 1 -- 172.21.15.19:0/2865018559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde000c8b30 con 0x7fde0015ba00 2024-04-25T09:59:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fdde77fe640 1 -- 172.21.15.19:0/2865018559 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0007bd0 con 0x7fde0015ba00 2024-04-25T09:59:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.949+0000 7fde05038640 1 -- 172.21.15.19:0/2865018559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde000c90f0 con 0x7fde0015ba00 2024-04-25T09:59:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.950+0000 7fde05038640 1 -- 172.21.15.19:0/2865018559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde00073990 con 0x7fde0015ba00 2024-04-25T09:59:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.951+0000 7fdde77fe640 1 -- 172.21.15.19:0/2865018559 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddf0018070 con 0x7fde0015ba00 2024-04-25T09:59:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.951+0000 7fdde77fe640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fddec06b4d0 0x7fddec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.951+0000 7fdde77fe640 1 -- 172.21.15.19:0/2865018559 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fddf008a2d0 con 0x7fde0015ba00 2024-04-25T09:59:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.951+0000 7fddfe59c640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fddec06b4d0 0x7fddec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.952+0000 7fddfe59c640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fddec06b4d0 0x7fddec06d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fddf4034000 tx=0x7fddf402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:35.955+0000 7fdde77fe640 1 -- 172.21.15.19:0/2865018559 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fddf00588e0 con 0x7fde0015ba00 2024-04-25T09:59:36.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.139+0000 7fde05038640 1 -- 172.21.15.19:0/2865018559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde000b40a0 con 0x7fde0015ba00 2024-04-25T09:59:36.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.141+0000 7fdde77fe640 1 -- 172.21.15.19:0/2865018559 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16873 (secure 0 0 0) 0x7fddf005c980 con 0x7fde0015ba00 2024-04-25T09:59:36.146 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:36.146 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":95,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:09.369855+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:06.008725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:06.705899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:36.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 -- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fddec06b4d0 msgr2=0x7fddec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fddec06b4d0 0x7fddec06d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fddf4034000 tx=0x7fddf402e000 comp rx=0 tx=0).stop 2024-04-25T09:59:36.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 -- 172.21.15.19:0/2865018559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde0015ba00 msgr2=0x7fde001f3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde0015ba00 0x7fde001f3ac0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fddf000c7f0 tx=0x7fddf000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 -- 172.21.15.19:0/2865018559 shutdown_connections 2024-04-25T09:59:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fddec06b4d0 0x7fddec06d990 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde001602e0 0x7fde000c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde0015f5f0 0x7fde001589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 --2- 172.21.15.19:0/2865018559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde0015ba00 0x7fde001f3ac0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 -- 172.21.15.19:0/2865018559 >> 172.21.15.19:0/2865018559 conn(0x7fde000b3270 msgr2=0x7fde0015e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 -- 172.21.15.19:0/2865018559 shutdown_connections 2024-04-25T09:59:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.143+0000 7fde05038640 1 -- 172.21.15.19:0/2865018559 wait complete. 2024-04-25T09:59:36.158 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:59:36.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.256+0000 7fdfced70640 1 Processor -- start 2024-04-25T09:59:36.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.256+0000 7fdfced70640 1 -- start start 2024-04-25T09:59:36.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.257+0000 7fdfced70640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdfc81610f0 0x7fdfc806c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.257+0000 7fdfced70640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc80c6960 con 0x7fdfc80ce7b0 2024-04-25T09:59:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.257+0000 7fdfced70640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc8162260 con 0x7fdfc81616f0 2024-04-25T09:59:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.257+0000 7fdfced70640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc81614d0 con 0x7fdfc81610f0 2024-04-25T09:59:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.257+0000 7fdfcd30d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdfc80ce7b0 0x7fdfc80d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45834/0 (socket says 172.21.15.19:45834) 2024-04-25T09:59:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.257+0000 7fdfcd30d640 1 -- 172.21.15.19:0/2398603538 learned_addr learned my addr 172.21.15.19:0/2398603538 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.257+0000 7fdfccb0c640 1 --2- 172.21.15.19:0/2398603538 >> v2:172.21.15.112:3300/0 conn(0x7fdfc81610f0 0x7fdfc806c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.257+0000 7fdfc77fe640 1 -- 172.21.15.19:0/2398603538 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627538336 0 0) 0x7fdfc80c6960 con 0x7fdfc80ce7b0 2024-04-25T09:59:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.257+0000 7fdfc77fe640 1 -- 172.21.15.19:0/2398603538 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb8003520 con 0x7fdfc80ce7b0 2024-04-25T09:59:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.257+0000 7fdfc77fe640 1 -- 172.21.15.19:0/2398603538 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678593407 0 0) 0x7fdfc8162260 con 0x7fdfc81616f0 2024-04-25T09:59:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.258+0000 7fdfc77fe640 1 -- 172.21.15.19:0/2398603538 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfc80c6960 con 0x7fdfc81616f0 2024-04-25T09:59:36.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.258+0000 7fdfc77fe640 1 -- 172.21.15.19:0/2398603538 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1804874810 0 0) 0x7fdfb8003520 con 0x7fdfc80ce7b0 2024-04-25T09:59:36.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.258+0000 7fdfccb0c640 1 -- 172.21.15.19:0/2398603538 >> v1:172.21.15.112:6789/0 conn(0x7fdfc81616f0 legacy=0x7fdfc80ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.258+0000 7fdfccb0c640 1 -- 172.21.15.19:0/2398603538 >> v1:172.21.15.19:6789/0 conn(0x7fdfc80ce7b0 legacy=0x7fdfc80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.258+0000 7fdfccb0c640 1 -- 172.21.15.19:0/2398603538 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc8162540 con 0x7fdfc81610f0 2024-04-25T09:59:36.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.258+0000 7fdfccb0c640 1 --2- 172.21.15.19:0/2398603538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc81610f0 0x7fdfc806c1c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0097a0 tx=0x7fdfbc02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dece7bd891134cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:36.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.259+0000 7fdfc77fe640 1 -- 172.21.15.19:0/2398603538 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc037070 con 0x7fdfc81610f0 2024-04-25T09:59:36.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.259+0000 7fdfc77fe640 1 -- 172.21.15.19:0/2398603538 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc02ac50 con 0x7fdfc81610f0 2024-04-25T09:59:36.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.259+0000 7fdfc77fe640 1 -- 172.21.15.19:0/2398603538 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc00019f0 con 0x7fdfc81610f0 2024-04-25T09:59:36.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.259+0000 7fdfced70640 1 -- 172.21.15.19:0/2398603538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc81610f0 msgr2=0x7fdfc806c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.259+0000 7fdfced70640 1 --2- 172.21.15.19:0/2398603538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc81610f0 0x7fdfc806c1c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0097a0 tx=0x7fdfbc02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.259+0000 7fdfced70640 1 -- 172.21.15.19:0/2398603538 shutdown_connections 2024-04-25T09:59:36.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.259+0000 7fdfced70640 1 --2- 172.21.15.19:0/2398603538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc81610f0 0x7fdfc806c1c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.259+0000 7fdfced70640 1 -- 172.21.15.19:0/2398603538 >> 172.21.15.19:0/2398603538 conn(0x7fdfc806f500 msgr2=0x7fdfc806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:36.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.259+0000 7fdfced70640 1 -- 172.21.15.19:0/2398603538 shutdown_connections 2024-04-25T09:59:36.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.259+0000 7fdfced70640 1 -- 172.21.15.19:0/2398603538 wait complete. 2024-04-25T09:59:36.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.260+0000 7fdfced70640 1 Processor -- start 2024-04-25T09:59:36.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.260+0000 7fdfced70640 1 -- start start 2024-04-25T09:59:36.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.261+0000 7fdfced70640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfc80ce7b0 0x7fdfc80b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.261+0000 7fdfced70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81610f0 0x7fdfc8154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.261+0000 7fdfced70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc81616f0 0x7fdfc80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.261+0000 7fdfced70640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80cf690 con 0x7fdfc81610f0 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.261+0000 7fdfced70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80cfce0 con 0x7fdfc81616f0 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.261+0000 7fdfced70640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc806c700 con 0x7fdfc80ce7b0 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.261+0000 7fdfc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81610f0 0x7fdfc8154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.261+0000 7fdfc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81610f0 0x7fdfc8154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47316/0 (socket says 172.21.15.19:47316) 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.261+0000 7fdfc7fff640 1 -- 172.21.15.19:0/1279484715 learned_addr learned my addr 172.21.15.19:0/1279484715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfcd30d640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc81616f0 0x7fdfc80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfccb0c640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfc80ce7b0 0x7fdfc80b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfc7fff640 1 -- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfc80ce7b0 msgr2=0x7fdfc80b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfc7fff640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfc80ce7b0 0x7fdfc80b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfc7fff640 1 -- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc81616f0 msgr2=0x7fdfc80b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfc7fff640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc81616f0 0x7fdfc80b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfc7fff640 1 -- 172.21.15.19:0/1279484715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc80b53a0 con 0x7fdfc81610f0 2024-04-25T09:59:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfcd30d640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc81616f0 0x7fdfc80b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfccb0c640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfc80ce7b0 0x7fdfc80b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfc7fff640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81610f0 0x7fdfc8154d00 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400c460 tx=0x7fdfb400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfc57fa640 1 -- 172.21.15.19:0/1279484715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40052d0 con 0x7fdfc81610f0 2024-04-25T09:59:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfc57fa640 1 -- 172.21.15.19:0/1279484715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb4005490 con 0x7fdfc81610f0 2024-04-25T09:59:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfced70640 1 -- 172.21.15.19:0/1279484715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc80b0aa0 con 0x7fdfc81610f0 2024-04-25T09:59:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfc57fa640 1 -- 172.21.15.19:0/1279484715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4012870 con 0x7fdfc81610f0 2024-04-25T09:59:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.262+0000 7fdfced70640 1 -- 172.21.15.19:0/1279484715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc81573e0 con 0x7fdfc81610f0 2024-04-25T09:59:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.263+0000 7fdfced70640 1 -- 172.21.15.19:0/1279484715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc8073a00 con 0x7fdfc81610f0 2024-04-25T09:59:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.264+0000 7fdfc57fa640 1 -- 172.21.15.19:0/1279484715 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfb4012a10 con 0x7fdfc81610f0 2024-04-25T09:59:36.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.264+0000 7fdfc57fa640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfac06b500 0x7fdfac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.264+0000 7fdfc57fa640 1 -- 172.21.15.19:0/1279484715 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdfb408a000 con 0x7fdfc81610f0 2024-04-25T09:59:36.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.264+0000 7fdfccb0c640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfac06b500 0x7fdfac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.265+0000 7fdfccb0c640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfac06b500 0x7fdfac06d9c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc034000 tx=0x7fdfbc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:36.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.268+0000 7fdfc57fa640 1 -- 172.21.15.19:0/1279484715 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfb4058c70 con 0x7fdfc81610f0 2024-04-25T09:59:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.486+0000 7fdfced70640 1 -- 172.21.15.19:0/1279484715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdfc8161fa0 con 0x7fdfc81610f0 2024-04-25T09:59:36.490 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T09:59:36.486+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:59:36.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.489+0000 7fdfc57fa640 1 -- 172.21.15.19:0/1279484715 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdfb4058570 con 0x7fdfc81610f0 2024-04-25T09:59:36.493 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:59:36.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.491+0000 7fdfced70640 1 -- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfac06b500 msgr2=0x7fdfac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.491+0000 7fdfced70640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfac06b500 0x7fdfac06d9c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc034000 tx=0x7fdfbc02e000 comp rx=0 tx=0).stop 2024-04-25T09:59:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.492+0000 7fdfced70640 1 -- 172.21.15.19:0/1279484715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81610f0 msgr2=0x7fdfc8154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.492+0000 7fdfced70640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81610f0 0x7fdfc8154d00 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400c460 tx=0x7fdfb400c920 comp rx=0 tx=0).stop 2024-04-25T09:59:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.492+0000 7fdfced70640 1 -- 172.21.15.19:0/1279484715 shutdown_connections 2024-04-25T09:59:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.492+0000 7fdfced70640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdfac06b500 0x7fdfac06d9c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.492+0000 7fdfced70640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc81616f0 0x7fdfc80b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.492+0000 7fdfced70640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc81610f0 0x7fdfc8154d00 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.492+0000 7fdfced70640 1 --2- 172.21.15.19:0/1279484715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfc80ce7b0 0x7fdfc80b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.492+0000 7fdfced70640 1 -- 172.21.15.19:0/1279484715 >> 172.21.15.19:0/1279484715 conn(0x7fdfc806f500 msgr2=0x7fdfc80713f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.492+0000 7fdfced70640 1 -- 172.21.15.19:0/1279484715 shutdown_connections 2024-04-25T09:59:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.492+0000 7fdfced70640 1 -- 172.21.15.19:0/1279484715 wait complete. 2024-04-25T09:59:36.508 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:59:36.536 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T09:59:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.609+0000 7f1923cd8640 1 Processor -- start 2024-04-25T09:59:36.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.609+0000 7f1923cd8640 1 -- start start 2024-04-25T09:59:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.610+0000 7f1923cd8640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f191c1446d0 0x7f191c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.610+0000 7f1923cd8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f191c144d50 0x7f191c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.610+0000 7f1923cd8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c1452a0 0x7f191c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.610+0000 7f1923cd8640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191c0b2b20 con 0x7f191c1446d0 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.610+0000 7f1923cd8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191c06d400 con 0x7f191c144d50 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.610+0000 7f1923cd8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0b49a0 con 0x7f191c1452a0 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.610+0000 7f1921a74640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f191c1446d0 0x7f191c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.610+0000 7f1921a74640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f191c1446d0 0x7f191c0b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47340/0 (socket says 172.21.15.19:47340) 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.610+0000 7f1921a74640 1 -- 172.21.15.19:0/2927094225 learned_addr learned my addr 172.21.15.19:0/2927094225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.611+0000 7f1921273640 1 --2- 172.21.15.19:0/2927094225 >> v2:172.21.15.112:3300/0 conn(0x7f191c144d50 0x7f191c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.611+0000 7f1922275640 1 --2- 172.21.15.19:0/2927094225 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c1452a0 0x7f191c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.611+0000 7f1921a74640 1 -- 172.21.15.19:0/2927094225 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c1452a0 msgr2=0x7f191c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.611+0000 7f1921a74640 1 --2- 172.21.15.19:0/2927094225 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c1452a0 0x7f191c0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.611+0000 7f1921a74640 1 -- 172.21.15.19:0/2927094225 >> v2:172.21.15.112:3300/0 conn(0x7f191c144d50 msgr2=0x7f191c0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.611+0000 7f1921a74640 1 --2- 172.21.15.19:0/2927094225 >> v2:172.21.15.112:3300/0 conn(0x7f191c144d50 0x7f191c0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.611+0000 7f1921a74640 1 -- 172.21.15.19:0/2927094225 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c0cccc0 con 0x7f191c1446d0 2024-04-25T09:59:36.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.611+0000 7f1921273640 1 --2- 172.21.15.19:0/2927094225 >> v2:172.21.15.112:3300/0 conn(0x7f191c144d50 0x7f191c0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:36.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.611+0000 7f1922275640 1 --2- 172.21.15.19:0/2927094225 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c1452a0 0x7f191c0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:36.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.611+0000 7f1921a74640 1 --2- 172.21.15.19:0/2927094225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191c1446d0 0x7f191c0b3f20 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f191c071c70 tx=0x7f190c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f429ca0bf82023 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:36.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1920a72640 1 -- 172.21.15.19:0/2927094225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c036070 con 0x7f191c1446d0 2024-04-25T09:59:36.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1920a72640 1 -- 172.21.15.19:0/2927094225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c031600 con 0x7f191c1446d0 2024-04-25T09:59:36.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1920a72640 1 -- 172.21.15.19:0/2927094225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c0319d0 con 0x7f191c1446d0 2024-04-25T09:59:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1923cd8640 1 -- 172.21.15.19:0/2927094225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191c1446d0 msgr2=0x7f191c0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1923cd8640 1 --2- 172.21.15.19:0/2927094225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191c1446d0 0x7f191c0b3f20 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f191c071c70 tx=0x7f190c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1923cd8640 1 -- 172.21.15.19:0/2927094225 shutdown_connections 2024-04-25T09:59:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1923cd8640 1 --2- 172.21.15.19:0/2927094225 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c1452a0 0x7f191c0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1923cd8640 1 --2- 172.21.15.19:0/2927094225 >> v2:172.21.15.112:3300/0 conn(0x7f191c144d50 0x7f191c0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1923cd8640 1 --2- 172.21.15.19:0/2927094225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191c1446d0 0x7f191c0b3f20 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1923cd8640 1 -- 172.21.15.19:0/2927094225 >> 172.21.15.19:0/2927094225 conn(0x7f191c06abe0 msgr2=0x7f191c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1923cd8640 1 -- 172.21.15.19:0/2927094225 shutdown_connections 2024-04-25T09:59:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.612+0000 7f1923cd8640 1 -- 172.21.15.19:0/2927094225 wait complete. 2024-04-25T09:59:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.613+0000 7f1923cd8640 1 Processor -- start 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.613+0000 7f1923cd8640 1 -- start start 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.614+0000 7f1923cd8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191c1446d0 0x7f191c135730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.614+0000 7f1923cd8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c144d50 0x7f191c138820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.614+0000 7f1923cd8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f191c1452a0 0x7f191c15b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.614+0000 7f1923cd8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0cc700 con 0x7f191c1446d0 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.614+0000 7f1923cd8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0cc420 con 0x7f191c1452a0 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.614+0000 7f1923cd8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0cc9e0 con 0x7f191c144d50 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.614+0000 7f1921a74640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191c1446d0 0x7f191c135730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.614+0000 7f1921a74640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191c1446d0 0x7f191c135730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47344/0 (socket says 172.21.15.19:47344) 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.614+0000 7f1921a74640 1 -- 172.21.15.19:0/4147872207 learned_addr learned my addr 172.21.15.19:0/4147872207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.614+0000 7f1921273640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c144d50 0x7f191c138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.614+0000 7f1922275640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f191c1452a0 0x7f191c15b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.615+0000 7f1921a74640 1 -- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c144d50 msgr2=0x7f191c138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.615+0000 7f1921a74640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c144d50 0x7f191c138820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.615+0000 7f1921a74640 1 -- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f191c1452a0 msgr2=0x7f191c15b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.615+0000 7f1921a74640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f191c1452a0 0x7f191c15b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.615+0000 7f1921a74640 1 -- 172.21.15.19:0/4147872207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c136c70 con 0x7f191c1446d0 2024-04-25T09:59:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.615+0000 7f1921a74640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191c1446d0 0x7f191c135730 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f190c034040 tx=0x7f190c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.615+0000 7f191a7fc640 1 -- 172.21.15.19:0/4147872207 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c036050 con 0x7f191c1446d0 2024-04-25T09:59:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.615+0000 7f191a7fc640 1 -- 172.21.15.19:0/4147872207 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c042070 con 0x7f191c1446d0 2024-04-25T09:59:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.615+0000 7f191a7fc640 1 -- 172.21.15.19:0/4147872207 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c009510 con 0x7f191c1446d0 2024-04-25T09:59:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.615+0000 7f1923cd8640 1 -- 172.21.15.19:0/4147872207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191c13ebd0 con 0x7f191c1446d0 2024-04-25T09:59:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.615+0000 7f1923cd8640 1 -- 172.21.15.19:0/4147872207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191c136f50 con 0x7f191c1446d0 2024-04-25T09:59:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.616+0000 7f1923cd8640 1 -- 172.21.15.19:0/4147872207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191c073a00 con 0x7f191c1446d0 2024-04-25T09:59:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.617+0000 7f191a7fc640 1 -- 172.21.15.19:0/4147872207 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f190c007c30 con 0x7f191c1446d0 2024-04-25T09:59:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.617+0000 7f191a7fc640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f190806b4d0 0x7f190806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.617+0000 7f191a7fc640 1 -- 172.21.15.19:0/4147872207 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f190c0ab120 con 0x7f191c1446d0 2024-04-25T09:59:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.617+0000 7f1921273640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f190806b4d0 0x7f190806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.618+0000 7f1921273640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f190806b4d0 0x7f190806d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f191c132a30 tx=0x7f1910008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:36.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.621+0000 7f191a7fc640 1 -- 172.21.15.19:0/4147872207 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f190c079800 con 0x7f191c1446d0 2024-04-25T09:59:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:36.838+0000 7f1923cd8640 1 -- 172.21.15.19:0/4147872207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f191c1426d0 con 0x7f191c1446d0 2024-04-25T09:59:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.499+0000 7f191a7fc640 1 -- 172.21.15.19:0/4147872207 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7f190c07d8a0 con 0x7f191c1446d0 2024-04-25T09:59:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 -- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f190806b4d0 msgr2=0x7f190806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f190806b4d0 0x7f190806d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f191c132a30 tx=0x7f1910008040 comp rx=0 tx=0).stop 2024-04-25T09:59:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 -- 172.21.15.19:0/4147872207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191c1446d0 msgr2=0x7f191c135730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191c1446d0 0x7f191c135730 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f190c034040 tx=0x7f190c004a40 comp rx=0 tx=0).stop 2024-04-25T09:59:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 -- 172.21.15.19:0/4147872207 shutdown_connections 2024-04-25T09:59:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f190806b4d0 0x7f190806d990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f191c1452a0 0x7f191c15b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c144d50 0x7f191c138820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 --2- 172.21.15.19:0/4147872207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191c1446d0 0x7f191c135730 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 -- 172.21.15.19:0/4147872207 >> 172.21.15.19:0/4147872207 conn(0x7f191c06abe0 msgr2=0x7f191c0af440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:37.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 -- 172.21.15.19:0/4147872207 shutdown_connections 2024-04-25T09:59:37.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.502+0000 7f1923cd8640 1 -- 172.21.15.19:0/4147872207 wait complete. 2024-04-25T09:59:37.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:59:37.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.671+0000 7f2b005b0640 1 Processor -- start 2024-04-25T09:59:37.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.671+0000 7f2b005b0640 1 -- start start 2024-04-25T09:59:37.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.672+0000 7f2b005b0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2af815b7d0 0x7f2af80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:37.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.672+0000 7f2b005b0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2af806c9e0 0x7f2af80b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:37.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.672+0000 7f2b005b0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af806cf50 con 0x7f2af80b5620 2024-04-25T09:59:37.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.672+0000 7f2b005b0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af80ce530 con 0x7f2af815b7d0 2024-04-25T09:59:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.672+0000 7f2b005b0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af80ce810 con 0x7f2af806c9e0 2024-04-25T09:59:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.672+0000 7f2afe34c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2af815b7d0 0x7f2af80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.672+0000 7f2afe34c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2af815b7d0 0x7f2af80cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51702/0 (socket says 172.21.15.19:51702) 2024-04-25T09:59:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.672+0000 7f2afe34c640 1 -- 172.21.15.19:0/4139956529 learned_addr learned my addr 172.21.15.19:0/4139956529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.672+0000 7f2afeb4d640 1 --2- 172.21.15.19:0/4139956529 >> v2:172.21.15.112:3300/0 conn(0x7f2af806c9e0 0x7f2af80b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.672+0000 7f2afe34c640 1 -- 172.21.15.19:0/4139956529 >> v2:172.21.15.112:3300/0 conn(0x7f2af806c9e0 msgr2=0x7f2af80b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.672+0000 7f2afe34c640 1 --2- 172.21.15.19:0/4139956529 >> v2:172.21.15.112:3300/0 conn(0x7f2af806c9e0 0x7f2af80b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.673+0000 7f2afe34c640 1 -- 172.21.15.19:0/4139956529 >> v1:172.21.15.112:6789/0 conn(0x7f2af80b5620 legacy=0x7f2af806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.673+0000 7f2afe34c640 1 -- 172.21.15.19:0/4139956529 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80ceaf0 con 0x7f2af815b7d0 2024-04-25T09:59:37.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.673+0000 7f2afe34c640 1 --2- 172.21.15.19:0/4139956529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af815b7d0 0x7f2af80cf010 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f2af8161890 tx=0x7f2af402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df318ef6e0813b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.673+0000 7f2afd34a640 1 -- 172.21.15.19:0/4139956529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4037070 con 0x7f2af815b7d0 2024-04-25T09:59:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.673+0000 7f2afd34a640 1 -- 172.21.15.19:0/4139956529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af4032440 con 0x7f2af815b7d0 2024-04-25T09:59:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.673+0000 7f2afd34a640 1 -- 172.21.15.19:0/4139956529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4032810 con 0x7f2af815b7d0 2024-04-25T09:59:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.674+0000 7f2b005b0640 1 -- 172.21.15.19:0/4139956529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af815b7d0 msgr2=0x7f2af80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.674+0000 7f2b005b0640 1 --2- 172.21.15.19:0/4139956529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af815b7d0 0x7f2af80cf010 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f2af8161890 tx=0x7f2af402a270 comp rx=0 tx=0).stop 2024-04-25T09:59:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.674+0000 7f2b005b0640 1 -- 172.21.15.19:0/4139956529 shutdown_connections 2024-04-25T09:59:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.674+0000 7f2b005b0640 1 --2- 172.21.15.19:0/4139956529 >> v2:172.21.15.112:3300/0 conn(0x7f2af806c9e0 0x7f2af80b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.674+0000 7f2b005b0640 1 --2- 172.21.15.19:0/4139956529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af815b7d0 0x7f2af80cf010 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.674+0000 7f2b005b0640 1 -- 172.21.15.19:0/4139956529 >> 172.21.15.19:0/4139956529 conn(0x7f2af80d0800 msgr2=0x7f2af80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.674+0000 7f2b005b0640 1 -- 172.21.15.19:0/4139956529 shutdown_connections 2024-04-25T09:59:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.674+0000 7f2b005b0640 1 -- 172.21.15.19:0/4139956529 wait complete. 2024-04-25T09:59:37.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.675+0000 7f2b005b0640 1 Processor -- start 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.675+0000 7f2b005b0640 1 -- start start 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.676+0000 7f2b005b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af806c9e0 0x7f2af80c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.676+0000 7f2b005b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af80b5620 0x7f2af80b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.676+0000 7f2b005b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2af815b7d0 0x7f2af80c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.676+0000 7f2b005b0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80cf6d0 con 0x7f2af80b5620 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.676+0000 7f2b005b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80cf550 con 0x7f2af806c9e0 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.676+0000 7f2b005b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af815bbb0 con 0x7f2af815b7d0 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.676+0000 7f2afdb4b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af80b5620 0x7f2af80b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.676+0000 7f2afdb4b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af80b5620 0x7f2af80b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51706/0 (socket says 172.21.15.19:51706) 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.676+0000 7f2afdb4b640 1 -- 172.21.15.19:0/4130972227 learned_addr learned my addr 172.21.15.19:0/4130972227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.676+0000 7f2afe34c640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af806c9e0 0x7f2af80c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.676+0000 7f2afeb4d640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2af815b7d0 0x7f2af80c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.677+0000 7f2afdb4b640 1 -- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2af815b7d0 msgr2=0x7f2af80c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.677+0000 7f2afdb4b640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2af815b7d0 0x7f2af80c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.677+0000 7f2afdb4b640 1 -- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af806c9e0 msgr2=0x7f2af80c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.677+0000 7f2afdb4b640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af806c9e0 0x7f2af80c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.677+0000 7f2afdb4b640 1 -- 172.21.15.19:0/4130972227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af81f39b0 con 0x7f2af80b5620 2024-04-25T09:59:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.677+0000 7f2afdb4b640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af80b5620 0x7f2af80b82c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c450 tx=0x7f2ae800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.677+0000 7f2aeeffd640 1 -- 172.21.15.19:0/4130972227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80052d0 con 0x7f2af80b5620 2024-04-25T09:59:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.677+0000 7f2aeeffd640 1 -- 172.21.15.19:0/4130972227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae8005470 con 0x7f2af80b5620 2024-04-25T09:59:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.677+0000 7f2b005b0640 1 -- 172.21.15.19:0/4130972227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af81f3c90 con 0x7f2af80b5620 2024-04-25T09:59:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.677+0000 7f2aeeffd640 1 -- 172.21.15.19:0/4130972227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8012870 con 0x7f2af80b5620 2024-04-25T09:59:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.677+0000 7f2b005b0640 1 -- 172.21.15.19:0/4130972227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af81f4250 con 0x7f2af80b5620 2024-04-25T09:59:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.678+0000 7f2b005b0640 1 -- 172.21.15.19:0/4130972227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af8068410 con 0x7f2af80b5620 2024-04-25T09:59:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.679+0000 7f2aeeffd640 1 -- 172.21.15.19:0/4130972227 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ae8012a10 con 0x7f2af80b5620 2024-04-25T09:59:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.679+0000 7f2aeeffd640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2adc06b4d0 0x7f2adc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.679+0000 7f2aeeffd640 1 -- 172.21.15.19:0/4130972227 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2ae8089e90 con 0x7f2af80b5620 2024-04-25T09:59:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.679+0000 7f2afe34c640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2adc06b4d0 0x7f2adc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.680+0000 7f2afe34c640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2adc06b4d0 0x7f2adc06d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2af4034000 tx=0x7f2af402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:37.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.684+0000 7f2aeeffd640 1 -- 172.21.15.19:0/4130972227 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2af8068410 con 0x7f2af80b5620 2024-04-25T09:59:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:37.865+0000 7f2b005b0640 1 -- 172.21.15.19:0/4130972227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2af815b300 con 0x7f2af80b5620 2024-04-25T09:59:38.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:59:38.018 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:59:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.502+0000 7f2aeeffd640 1 -- 172.21.15.19:0/4130972227 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2ae80584e0 con 0x7f2af80b5620 2024-04-25T09:59:38.506 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 -- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2adc06b4d0 msgr2=0x7f2adc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2adc06b4d0 0x7f2adc06d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2af4034000 tx=0x7f2af402e000 comp rx=0 tx=0).stop 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 -- 172.21.15.19:0/4130972227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af80b5620 msgr2=0x7f2af80b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af80b5620 0x7f2af80b82c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c450 tx=0x7f2ae800c910 comp rx=0 tx=0).stop 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 -- 172.21.15.19:0/4130972227 shutdown_connections 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2adc06b4d0 0x7f2adc06d990 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2af815b7d0 0x7f2af80c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af80b5620 0x7f2af80b82c0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 --2- 172.21.15.19:0/4130972227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af806c9e0 0x7f2af80c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 -- 172.21.15.19:0/4130972227 >> 172.21.15.19:0/4130972227 conn(0x7f2af80d0800 msgr2=0x7f2af80d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 -- 172.21.15.19:0/4130972227 shutdown_connections 2024-04-25T09:59:38.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.504+0000 7f2b005b0640 1 -- 172.21.15.19:0/4130972227 wait complete. 2024-04-25T09:59:38.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:59:38.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.631+0000 7f0cfdbe7640 1 Processor -- start 2024-04-25T09:59:38.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.631+0000 7f0cfdbe7640 1 -- start start 2024-04-25T09:59:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.632+0000 7f0cfdbe7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 0x7f0cf80ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.632+0000 7f0cfdbe7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0cf815b980 0x7f0cf80ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.632+0000 7f0cfdbe7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf80cf320 con 0x7f0cf815b5a0 2024-04-25T09:59:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.632+0000 7f0cfdbe7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf80ced90 con 0x7f0cf815b980 2024-04-25T09:59:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.632+0000 7f0cfdbe7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80cf040 con 0x7f0cf80b5c40 2024-04-25T09:59:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.632+0000 7f0cf6ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 0x7f0cf80ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.632+0000 7f0cf7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0cf815b980 0x7f0cf80ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.632+0000 7f0cf7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0cf815b980 0x7f0cf80ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:50098/0 (socket says 172.21.15.19:50098) 2024-04-25T09:59:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.632+0000 7f0cf77fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0cf815b5a0 0x7f0cf80b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:53876/0 (socket says 172.21.15.19:53876) 2024-04-25T09:59:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.632+0000 7f0cf7fff640 1 -- 172.21.15.19:0/3261089627 learned_addr learned my addr 172.21.15.19:0/3261089627 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:38.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.632+0000 7f0cf6ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 0x7f0cf80ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:54352/0 (socket says 172.21.15.19:54352) 2024-04-25T09:59:38.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.633+0000 7f0cf6ffd640 1 -- 172.21.15.19:0/3261089627 >> v2:172.21.15.112:3300/0 conn(0x7f0cf815b980 msgr2=0x7f0cf80ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.633+0000 7f0cf6ffd640 1 --2- 172.21.15.19:0/3261089627 >> v2:172.21.15.112:3300/0 conn(0x7f0cf815b980 0x7f0cf80ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.633+0000 7f0cf6ffd640 1 -- 172.21.15.19:0/3261089627 >> v1:172.21.15.112:6789/0 conn(0x7f0cf815b5a0 legacy=0x7f0cf80b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.633+0000 7f0cf6ffd640 1 -- 172.21.15.19:0/3261089627 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8161970 con 0x7f0cf80b5c40 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.633+0000 7f0cf7fff640 1 --2- 172.21.15.19:0/3261089627 >> v2:172.21.15.112:3300/0 conn(0x7f0cf815b980 0x7f0cf80ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.633+0000 7f0cf6ffd640 1 --2- 172.21.15.19:0/3261089627 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 0x7f0cf80ce310 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009790 tx=0x7f0cec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3d5d11045cea2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.634+0000 7f0cf67fc640 1 -- 172.21.15.19:0/3261089627 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec037070 con 0x7f0cf80b5c40 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.634+0000 7f0cf67fc640 1 -- 172.21.15.19:0/3261089627 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec032440 con 0x7f0cf80b5c40 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.634+0000 7f0cf67fc640 1 -- 172.21.15.19:0/3261089627 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec032810 con 0x7f0cf80b5c40 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.634+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/3261089627 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 msgr2=0x7f0cf80ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.634+0000 7f0cfdbe7640 1 --2- 172.21.15.19:0/3261089627 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 0x7f0cf80ce310 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009790 tx=0x7f0cec02a180 comp rx=0 tx=0).stop 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.634+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/3261089627 shutdown_connections 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.634+0000 7f0cfdbe7640 1 --2- 172.21.15.19:0/3261089627 >> v2:172.21.15.112:3300/0 conn(0x7f0cf815b980 0x7f0cf80ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.634+0000 7f0cfdbe7640 1 --2- 172.21.15.19:0/3261089627 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 0x7f0cf80ce310 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.634+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/3261089627 >> 172.21.15.19:0/3261089627 conn(0x7f0cf80d0530 msgr2=0x7f0cf80d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.634+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/3261089627 shutdown_connections 2024-04-25T09:59:38.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.634+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/3261089627 wait complete. 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.635+0000 7f0cfdbe7640 1 Processor -- start 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.636+0000 7f0cfdbe7640 1 -- start start 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.636+0000 7f0cfdbe7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 0x7f0cf814b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.636+0000 7f0cfdbe7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf815b5a0 0x7f0cf8149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.636+0000 7f0cfdbe7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cf815b980 0x7f0cf80c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.636+0000 7f0cfdbe7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf8162450 con 0x7f0cf815b5a0 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.636+0000 7f0cfdbe7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf8161f30 con 0x7f0cf815b980 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.636+0000 7f0cfdbe7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf81625d0 con 0x7f0cf80b5c40 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.636+0000 7f0cf6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf815b5a0 0x7f0cf8149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.636+0000 7f0cf6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf815b5a0 0x7f0cf8149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51720/0 (socket says 172.21.15.19:51720) 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.636+0000 7f0cf6ffd640 1 -- 172.21.15.19:0/2682103701 learned_addr learned my addr 172.21.15.19:0/2682103701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:38.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cf77fe640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 0x7f0cf814b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cf7fff640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cf815b980 0x7f0cf80c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cf6ffd640 1 -- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 msgr2=0x7f0cf814b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cf6ffd640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 0x7f0cf814b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cf6ffd640 1 -- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cf815b980 msgr2=0x7f0cf80c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cf6ffd640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cf815b980 0x7f0cf80c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cf6ffd640 1 -- 172.21.15.19:0/2682103701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf80c7830 con 0x7f0cf815b5a0 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cf7fff640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cf815b980 0x7f0cf80c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cf6ffd640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf815b5a0 0x7f0cf8149750 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0cec035040 tx=0x7f0cec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cdffff640 1 -- 172.21.15.19:0/2682103701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec037040 con 0x7f0cf815b5a0 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cdffff640 1 -- 172.21.15.19:0/2682103701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec005b90 con 0x7f0cf815b5a0 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/2682103701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf80c7b10 con 0x7f0cf815b5a0 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cdffff640 1 -- 172.21.15.19:0/2682103701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec006030 con 0x7f0cf815b5a0 2024-04-25T09:59:38.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.637+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/2682103701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf8149c90 con 0x7f0cf815b5a0 2024-04-25T09:59:38.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.638+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/2682103701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf80c75c0 con 0x7f0cf815b5a0 2024-04-25T09:59:38.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.639+0000 7f0cdffff640 1 -- 172.21.15.19:0/2682103701 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0cec0061d0 con 0x7f0cf815b5a0 2024-04-25T09:59:38.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.639+0000 7f0cdffff640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ce406b4d0 0x7f0ce406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.639+0000 7f0cdffff640 1 -- 172.21.15.19:0/2682103701 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f0cec0aaaa0 con 0x7f0cf815b5a0 2024-04-25T09:59:38.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.639+0000 7f0cf77fe640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ce406b4d0 0x7f0ce406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.640+0000 7f0cf77fe640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ce406b4d0 0x7f0ce406d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8000f60 tx=0x7f0ce8007cc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.643+0000 7f0cdffff640 1 -- 172.21.15.19:0/2682103701 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0cec079870 con 0x7f0cf815b5a0 2024-04-25T09:59:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:38.826+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/2682103701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0cf8073a00 con 0x7f0cf815b5a0 2024-04-25T09:59:39.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.507+0000 7f0cdffff640 1 -- 172.21.15.19:0/2682103701 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0cec079170 con 0x7f0cf815b5a0 2024-04-25T09:59:39.511 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T09:59:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.511+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ce406b4d0 msgr2=0x7f0ce406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.511+0000 7f0cfdbe7640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ce406b4d0 0x7f0ce406d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8000f60 tx=0x7f0ce8007cc0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.511+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/2682103701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf815b5a0 msgr2=0x7f0cf8149750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.511+0000 7f0cfdbe7640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf815b5a0 0x7f0cf8149750 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0cec035040 tx=0x7f0cec0046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.512+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/2682103701 shutdown_connections 2024-04-25T09:59:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.512+0000 7f0cfdbe7640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ce406b4d0 0x7f0ce406d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.512+0000 7f0cfdbe7640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cf815b980 0x7f0cf80c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.512+0000 7f0cfdbe7640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cf815b5a0 0x7f0cf8149750 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.512+0000 7f0cfdbe7640 1 --2- 172.21.15.19:0/2682103701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0cf80b5c40 0x7f0cf814b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.512+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/2682103701 >> 172.21.15.19:0/2682103701 conn(0x7f0cf80d0530 msgr2=0x7f0cf80d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:39.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.512+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/2682103701 shutdown_connections 2024-04-25T09:59:39.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.512+0000 7f0cfdbe7640 1 -- 172.21.15.19:0/2682103701 wait complete. 2024-04-25T09:59:39.534 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:59:39.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.673+0000 7f27edced640 1 Processor -- start 2024-04-25T09:59:39.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.674+0000 7f27edced640 1 -- start start 2024-04-25T09:59:39.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.675+0000 7f27edced640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e806b2f0 0x7f27e816c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.675+0000 7f27edced640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e8162280 con 0x7f27e8161ea0 2024-04-25T09:59:39.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.675+0000 7f27edced640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e80b3460 con 0x7f27e80b60d0 2024-04-25T09:59:39.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.675+0000 7f27edced640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80b3a20 con 0x7f27e806b2f0 2024-04-25T09:59:39.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.675+0000 7f27ed4ec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e806b2f0 0x7f27e816c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.675+0000 7f27ed4ec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e806b2f0 0x7f27e816c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:54368/0 (socket says 172.21.15.19:54368) 2024-04-25T09:59:39.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.675+0000 7f27ed4ec640 1 -- 172.21.15.19:0/4029614115 learned_addr learned my addr 172.21.15.19:0/4029614115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:39.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.676+0000 7f27ed4ec640 1 -- 172.21.15.19:0/4029614115 >> v1:172.21.15.112:6789/0 conn(0x7f27e80b60d0 legacy=0x7f27e806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.676+0000 7f27ed4ec640 1 -- 172.21.15.19:0/4029614115 >> v1:172.21.15.19:6789/0 conn(0x7f27e8161ea0 legacy=0x7f27e80b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:39.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.676+0000 7f27ed4ec640 1 -- 172.21.15.19:0/4029614115 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e80b3740 con 0x7f27e806b2f0 2024-04-25T09:59:39.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.677+0000 7f27ed4ec640 1 --2- 172.21.15.19:0/4029614115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e806b2f0 0x7f27e816c9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009e80 tx=0x7f27e002a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c84135a3415fb19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:39.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.677+0000 7f27e77fe640 1 -- 172.21.15.19:0/4029614115 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0036070 con 0x7f27e806b2f0 2024-04-25T09:59:39.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.677+0000 7f27e77fe640 1 -- 172.21.15.19:0/4029614115 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e0031600 con 0x7f27e806b2f0 2024-04-25T09:59:39.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.677+0000 7f27e77fe640 1 -- 172.21.15.19:0/4029614115 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00319d0 con 0x7f27e806b2f0 2024-04-25T09:59:39.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.678+0000 7f27edced640 1 -- 172.21.15.19:0/4029614115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e806b2f0 msgr2=0x7f27e816c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.678+0000 7f27edced640 1 --2- 172.21.15.19:0/4029614115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e806b2f0 0x7f27e816c9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009e80 tx=0x7f27e002a750 comp rx=0 tx=0).stop 2024-04-25T09:59:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.679+0000 7f27edced640 1 -- 172.21.15.19:0/4029614115 shutdown_connections 2024-04-25T09:59:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.679+0000 7f27edced640 1 --2- 172.21.15.19:0/4029614115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e806b2f0 0x7f27e816c9c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.679+0000 7f27edced640 1 -- 172.21.15.19:0/4029614115 >> 172.21.15.19:0/4029614115 conn(0x7f27e8099a40 msgr2=0x7f27e80b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.679+0000 7f27edced640 1 -- 172.21.15.19:0/4029614115 shutdown_connections 2024-04-25T09:59:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.679+0000 7f27edced640 1 -- 172.21.15.19:0/4029614115 wait complete. 2024-04-25T09:59:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.680+0000 7f27edced640 1 Processor -- start 2024-04-25T09:59:39.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.681+0000 7f27edced640 1 -- start start 2024-04-25T09:59:39.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.682+0000 7f27edced640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27e806b2f0 0x7f27e809bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.682+0000 7f27edced640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e80b60d0 0x7f27e80cf790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.682+0000 7f27edced640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e8161ea0 0x7f27e8151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.682+0000 7f27edced640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80ae3b0 con 0x7f27e80b60d0 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.682+0000 7f27edced640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80b5450 con 0x7f27e806b2f0 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.682+0000 7f27edced640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80ae530 con 0x7f27e8161ea0 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.682+0000 7f27e7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e80b60d0 0x7f27e80cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.682+0000 7f27e7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e80b60d0 0x7f27e80cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51734/0 (socket says 172.21.15.19:51734) 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.682+0000 7f27e7fff640 1 -- 172.21.15.19:0/2427033307 learned_addr learned my addr 172.21.15.19:0/2427033307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.682+0000 7f27ecceb640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27e806b2f0 0x7f27e809bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.683+0000 7f27ed4ec640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e8161ea0 0x7f27e8151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.683+0000 7f27ed4ec640 1 -- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27e806b2f0 msgr2=0x7f27e809bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.683+0000 7f27ed4ec640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27e806b2f0 0x7f27e809bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.683+0000 7f27ed4ec640 1 -- 172.21.15.19:0/2427033307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e80b60d0 msgr2=0x7f27e80cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.683+0000 7f27ed4ec640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e80b60d0 0x7f27e80cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.683+0000 7f27ed4ec640 1 -- 172.21.15.19:0/2427033307 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e80cff00 con 0x7f27e8161ea0 2024-04-25T09:59:39.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.683+0000 7f27ed4ec640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e8161ea0 0x7f27e8151730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f27e0030000 tx=0x7f27e0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:39.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.684+0000 7f27e57fa640 1 -- 172.21.15.19:0/2427033307 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0036050 con 0x7f27e8161ea0 2024-04-25T09:59:39.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.684+0000 7f27e57fa640 1 -- 172.21.15.19:0/2427033307 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e0042070 con 0x7f27e8161ea0 2024-04-25T09:59:39.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.684+0000 7f27e57fa640 1 -- 172.21.15.19:0/2427033307 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0009510 con 0x7f27e8161ea0 2024-04-25T09:59:39.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.684+0000 7f27edced640 1 -- 172.21.15.19:0/2427033307 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e80d01e0 con 0x7f27e8161ea0 2024-04-25T09:59:39.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.684+0000 7f27edced640 1 -- 172.21.15.19:0/2427033307 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e80cc7c0 con 0x7f27e8161ea0 2024-04-25T09:59:39.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.685+0000 7f27edced640 1 -- 172.21.15.19:0/2427033307 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e80d1340 con 0x7f27e8161ea0 2024-04-25T09:59:39.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.686+0000 7f27e57fa640 1 -- 172.21.15.19:0/2427033307 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27e0007bb0 con 0x7f27e8161ea0 2024-04-25T09:59:39.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.687+0000 7f27e57fa640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f27cc06b4d0 0x7f27cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.687+0000 7f27e57fa640 1 -- 172.21.15.19:0/2427033307 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f27e00ab000 con 0x7f27e8161ea0 2024-04-25T09:59:39.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.687+0000 7f27ecceb640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f27cc06b4d0 0x7f27cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.688+0000 7f27ecceb640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f27cc06b4d0 0x7f27cc06d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f27e815f980 tx=0x7f27dc009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.691+0000 7f27e57fa640 1 -- 172.21.15.19:0/2427033307 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27e80d1340 con 0x7f27e8161ea0 2024-04-25T09:59:39.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.873+0000 7f27edced640 1 -- 172.21.15.19:0/2427033307 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f27e8073990 con 0x7f27e8161ea0 2024-04-25T09:59:39.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.879+0000 7f27e57fa640 1 -- 172.21.15.19:0/2427033307 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f27e0079f50 con 0x7f27e8161ea0 2024-04-25T09:59:39.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 -- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f27cc06b4d0 msgr2=0x7f27cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f27cc06b4d0 0x7f27cc06d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f27e815f980 tx=0x7f27dc009060 comp rx=0 tx=0).stop 2024-04-25T09:59:39.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 -- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e8161ea0 msgr2=0x7f27e8151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e8161ea0 0x7f27e8151730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f27e0030000 tx=0x7f27e0004a40 comp rx=0 tx=0).stop 2024-04-25T09:59:39.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 -- 172.21.15.19:0/2427033307 shutdown_connections 2024-04-25T09:59:39.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f27cc06b4d0 0x7f27cc06d990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f27e8161ea0 0x7f27e8151730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e80b60d0 0x7f27e80cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 --2- 172.21.15.19:0/2427033307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27e806b2f0 0x7f27e809bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 -- 172.21.15.19:0/2427033307 >> 172.21.15.19:0/2427033307 conn(0x7f27e8099a40 msgr2=0x7f27e80b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:39.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 -- 172.21.15.19:0/2427033307 shutdown_connections 2024-04-25T09:59:39.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.881+0000 7f27edced640 1 -- 172.21.15.19:0/2427033307 wait complete. 2024-04-25T09:59:39.896 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T09:59:39.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.993+0000 7fad433f4640 1 Processor -- start 2024-04-25T09:59:39.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.993+0000 7fad433f4640 1 -- start start 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad433f4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fad3c0b6cb0 0x7fad3c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad433f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad3c0b5a60 0x7fad3c0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad433f4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3c0ceee0 con 0x7fad3c15db70 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad433f4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3c0d1d10 con 0x7fad3c0b6cb0 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad433f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c0d1ff0 con 0x7fad3c0b5a60 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad4098f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fad3c0b6cb0 0x7fad3c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad4098f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fad3c0b6cb0 0x7fad3c0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51742/0 (socket says 172.21.15.19:51742) 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad4098f640 1 -- 172.21.15.19:0/1732876961 learned_addr learned my addr 172.21.15.19:0/1732876961 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad41190640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fad3c15db70 0x7fad3c162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50752/0 (socket says 172.21.15.19:50752) 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad41991640 1 --2- 172.21.15.19:0/1732876961 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad3c0b5a60 0x7fad3c0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad4098f640 1 -- 172.21.15.19:0/1732876961 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad3c0b5a60 msgr2=0x7fad3c0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad4098f640 1 --2- 172.21.15.19:0/1732876961 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad3c0b5a60 0x7fad3c0d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad4098f640 1 -- 172.21.15.19:0/1732876961 >> v1:172.21.15.19:6789/0 conn(0x7fad3c15db70 legacy=0x7fad3c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad4098f640 1 -- 172.21.15.19:0/1732876961 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c0d22d0 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.994+0000 7fad33fff640 1 -- 172.21.15.19:0/1732876961 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794979343 0 0) 0x7fad3c0ceee0 con 0x7fad3c15db70 2024-04-25T09:59:40.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.995+0000 7fad4098f640 1 --2- 172.21.15.19:0/1732876961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad3c0b6cb0 0x7fad3c0b5520 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009870 tx=0x7fad2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30df0c2a43b3f937 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:40.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.995+0000 7fad33fff640 1 -- 172.21.15.19:0/1732876961 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c037070 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.995+0000 7fad33fff640 1 -- 172.21.15.19:0/1732876961 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c032440 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.995+0000 7fad33fff640 1 -- 172.21.15.19:0/1732876961 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c032810 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.996+0000 7fad433f4640 1 -- 172.21.15.19:0/1732876961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad3c0b6cb0 msgr2=0x7fad3c0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.996+0000 7fad433f4640 1 --2- 172.21.15.19:0/1732876961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad3c0b6cb0 0x7fad3c0b5520 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009870 tx=0x7fad2c02a270 comp rx=0 tx=0).stop 2024-04-25T09:59:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.996+0000 7fad433f4640 1 -- 172.21.15.19:0/1732876961 shutdown_connections 2024-04-25T09:59:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.996+0000 7fad433f4640 1 --2- 172.21.15.19:0/1732876961 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad3c0b5a60 0x7fad3c0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.996+0000 7fad433f4640 1 --2- 172.21.15.19:0/1732876961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad3c0b6cb0 0x7fad3c0b5520 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.996+0000 7fad433f4640 1 -- 172.21.15.19:0/1732876961 >> 172.21.15.19:0/1732876961 conn(0x7fad3c0d0520 msgr2=0x7fad3c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.996+0000 7fad433f4640 1 -- 172.21.15.19:0/1732876961 shutdown_connections 2024-04-25T09:59:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.996+0000 7fad433f4640 1 -- 172.21.15.19:0/1732876961 wait complete. 2024-04-25T09:59:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.997+0000 7fad433f4640 1 Processor -- start 2024-04-25T09:59:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.997+0000 7fad433f4640 1 -- start start 2024-04-25T09:59:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad433f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad3c0b5a60 0x7fad3c1e6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad433f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad3c0b6cb0 0x7fad3c0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad433f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad3c15db70 0x7fad3c1f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad433f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c0cf1c0 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad433f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c0ce530 con 0x7fad3c15db70 2024-04-25T09:59:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad433f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c0cf340 con 0x7fad3c0b5a60 2024-04-25T09:59:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad4098f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad3c0b6cb0 0x7fad3c0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad4098f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad3c0b6cb0 0x7fad3c0cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51746/0 (socket says 172.21.15.19:51746) 2024-04-25T09:59:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad4098f640 1 -- 172.21.15.19:0/1996181370 learned_addr learned my addr 172.21.15.19:0/1996181370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad41190640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad3c0b5a60 0x7fad3c1e6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad41991640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad3c15db70 0x7fad3c1f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad4098f640 1 -- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad3c0b5a60 msgr2=0x7fad3c1e6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad4098f640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad3c0b5a60 0x7fad3c1e6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad4098f640 1 -- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad3c15db70 msgr2=0x7fad3c1f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad4098f640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad3c15db70 0x7fad3c1f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad4098f640 1 -- 172.21.15.19:0/1996181370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c0cb580 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad41190640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad3c0b5a60 0x7fad3c1e6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:40.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad4098f640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad3c0b6cb0 0x7fad3c0cc620 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fad2c000c00 tx=0x7fad2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:40.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad31ffb640 1 -- 172.21.15.19:0/1996181370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c037040 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad31ffb640 1 -- 172.21.15.19:0/1996181370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c042070 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad31ffb640 1 -- 172.21.15.19:0/1996181370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c030880 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.998+0000 7fad433f4640 1 -- 172.21.15.19:0/1996181370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3c0cb860 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:39.999+0000 7fad433f4640 1 -- 172.21.15.19:0/1996181370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3c0cbe20 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.000+0000 7fad433f4640 1 -- 172.21.15.19:0/1996181370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3c0cef80 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.000+0000 7fad31ffb640 1 -- 172.21.15.19:0/1996181370 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad2c030a20 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.001+0000 7fad31ffb640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad2006b4d0 0x7fad2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.001+0000 7fad31ffb640 1 -- 172.21.15.19:0/1996181370 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fad2c0aa8e0 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.001+0000 7fad41190640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad2006b4d0 0x7fad2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.001+0000 7fad41190640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad2006b4d0 0x7fad2006d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fad38000f60 tx=0x7fad38007ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:40.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.005+0000 7fad31ffb640 1 -- 172.21.15.19:0/1996181370 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad3c0cef80 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.263+0000 7fad433f4640 1 -- 172.21.15.19:0/1996181370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7fad3c073990 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.509+0000 7fad31ffb640 1 -- 172.21.15.19:0/1996181370 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v325) v1 ==== 144+0+0 (secure 0 0 0) 0x7fad2c0798d0 con 0x7fad3c0b6cb0 2024-04-25T09:59:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 -- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad2006b4d0 msgr2=0x7fad2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad2006b4d0 0x7fad2006d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fad38000f60 tx=0x7fad38007ca0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 -- 172.21.15.19:0/1996181370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad3c0b6cb0 msgr2=0x7fad3c0cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad3c0b6cb0 0x7fad3c0cc620 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fad2c000c00 tx=0x7fad2c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 -- 172.21.15.19:0/1996181370 shutdown_connections 2024-04-25T09:59:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad2006b4d0 0x7fad2006d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad3c15db70 0x7fad3c1f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad3c0b6cb0 0x7fad3c0cc620 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 --2- 172.21.15.19:0/1996181370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad3c0b5a60 0x7fad3c1e6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 -- 172.21.15.19:0/1996181370 >> 172.21.15.19:0/1996181370 conn(0x7fad3c0d0520 msgr2=0x7fad3c0b90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 -- 172.21.15.19:0/1996181370 shutdown_connections 2024-04-25T09:59:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.511+0000 7fad433f4640 1 -- 172.21.15.19:0/1996181370 wait complete. 2024-04-25T09:59:40.527 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:59:40.527 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:59:40.527 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:40.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.624+0000 7f1f5922a640 1 Processor -- start 2024-04-25T09:59:40.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.624+0000 7f1f5922a640 1 -- start start 2024-04-25T09:59:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.625+0000 7f1f5922a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5415a790 0x7f1f540d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.625+0000 7f1f5922a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f5406fea0 0x7f1f540d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.625+0000 7f1f5922a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f540b3710 con 0x7f1f540af9c0 2024-04-25T09:59:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.625+0000 7f1f5922a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f540cf0e0 con 0x7f1f5406fea0 2024-04-25T09:59:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.625+0000 7f1f5922a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f540cf3c0 con 0x7f1f5415a790 2024-04-25T09:59:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.625+0000 7f1f52d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5415a790 0x7f1f540d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.625+0000 7f1f5259c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f5406fea0 0x7f1f540d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.626+0000 7f1f52d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5415a790 0x7f1f540d0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:54408/0 (socket says 172.21.15.19:54408) 2024-04-25T09:59:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.626+0000 7f1f52d9d640 1 -- 172.21.15.19:0/2660627215 learned_addr learned my addr 172.21.15.19:0/2660627215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.626+0000 7f1f52d9d640 1 -- 172.21.15.19:0/2660627215 >> v2:172.21.15.112:3300/0 conn(0x7f1f5406fea0 msgr2=0x7f1f540d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:40.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.626+0000 7f1f52d9d640 1 --2- 172.21.15.19:0/2660627215 >> v2:172.21.15.112:3300/0 conn(0x7f1f5406fea0 0x7f1f540d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.626+0000 7f1f52d9d640 1 -- 172.21.15.19:0/2660627215 >> v1:172.21.15.112:6789/0 conn(0x7f1f540af9c0 legacy=0x7f1f54072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.626+0000 7f1f52d9d640 1 -- 172.21.15.19:0/2660627215 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f540cf6a0 con 0x7f1f5415a790 2024-04-25T09:59:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f52d9d640 1 --2- 172.21.15.19:0/2660627215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5415a790 0x7f1f540d0390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009870 tx=0x7f1f4c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2293a6df0d13003a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f51d9b640 1 -- 172.21.15.19:0/2660627215 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c037070 con 0x7f1f5415a790 2024-04-25T09:59:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f51d9b640 1 -- 172.21.15.19:0/2660627215 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c032440 con 0x7f1f5415a790 2024-04-25T09:59:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f51d9b640 1 -- 172.21.15.19:0/2660627215 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c032810 con 0x7f1f5415a790 2024-04-25T09:59:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f5922a640 1 -- 172.21.15.19:0/2660627215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5415a790 msgr2=0x7f1f540d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f5922a640 1 --2- 172.21.15.19:0/2660627215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5415a790 0x7f1f540d0390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009870 tx=0x7f1f4c02a270 comp rx=0 tx=0).stop 2024-04-25T09:59:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f5922a640 1 -- 172.21.15.19:0/2660627215 shutdown_connections 2024-04-25T09:59:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f5922a640 1 --2- 172.21.15.19:0/2660627215 >> v2:172.21.15.112:3300/0 conn(0x7f1f5406fea0 0x7f1f540d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f5922a640 1 --2- 172.21.15.19:0/2660627215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5415a790 0x7f1f540d0390 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f5922a640 1 -- 172.21.15.19:0/2660627215 >> 172.21.15.19:0/2660627215 conn(0x7f1f540b2370 msgr2=0x7f1f540b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f5922a640 1 -- 172.21.15.19:0/2660627215 shutdown_connections 2024-04-25T09:59:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.627+0000 7f1f5922a640 1 -- 172.21.15.19:0/2660627215 wait complete. 2024-04-25T09:59:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.629+0000 7f1f5922a640 1 Processor -- start 2024-04-25T09:59:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.629+0000 7f1f5922a640 1 -- start start 2024-04-25T09:59:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5922a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5406fea0 0x7f1f54145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5922a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f540af9c0 0x7f1f5415aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5922a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f5415a790 0x7f1f541e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5922a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5415ab70 con 0x7f1f540af9c0 2024-04-25T09:59:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5922a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f540cf980 con 0x7f1f5415a790 2024-04-25T09:59:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5922a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5415acf0 con 0x7f1f5406fea0 2024-04-25T09:59:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f540af9c0 0x7f1f5415aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f540af9c0 0x7f1f5415aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51756/0 (socket says 172.21.15.19:51756) 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5259c640 1 -- 172.21.15.19:0/3692148020 learned_addr learned my addr 172.21.15.19:0/3692148020 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f52d9d640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5406fea0 0x7f1f54145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5359e640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f5415a790 0x7f1f541e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5259c640 1 -- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5406fea0 msgr2=0x7f1f54145080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5259c640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5406fea0 0x7f1f54145080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5259c640 1 -- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f5415a790 msgr2=0x7f1f541e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5259c640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f5415a790 0x7f1f541e49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5259c640 1 -- 172.21.15.19:0/3692148020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5415c920 con 0x7f1f540af9c0 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f5259c640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f540af9c0 0x7f1f5415aea0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00e840 tx=0x7f1f3c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f4b7fe640 1 -- 172.21.15.19:0/3692148020 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c017070 con 0x7f1f540af9c0 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.630+0000 7f1f4b7fe640 1 -- 172.21.15.19:0/3692148020 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c00c940 con 0x7f1f540af9c0 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.631+0000 7f1f5922a640 1 -- 172.21.15.19:0/3692148020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5415b880 con 0x7f1f540af9c0 2024-04-25T09:59:40.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.631+0000 7f1f4b7fe640 1 -- 172.21.15.19:0/3692148020 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c00cd10 con 0x7f1f540af9c0 2024-04-25T09:59:40.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.631+0000 7f1f5922a640 1 -- 172.21.15.19:0/3692148020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5415be40 con 0x7f1f540af9c0 2024-04-25T09:59:40.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.632+0000 7f1f5922a640 1 -- 172.21.15.19:0/3692148020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f54073990 con 0x7f1f540af9c0 2024-04-25T09:59:40.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.632+0000 7f1f4b7fe640 1 -- 172.21.15.19:0/3692148020 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f3c01a070 con 0x7f1f540af9c0 2024-04-25T09:59:40.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.632+0000 7f1f4b7fe640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f3406b4d0 0x7f1f3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.633+0000 7f1f4b7fe640 1 -- 172.21.15.19:0/3692148020 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1f3c089ee0 con 0x7f1f540af9c0 2024-04-25T09:59:40.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.633+0000 7f1f52d9d640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f3406b4d0 0x7f1f3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.633+0000 7f1f52d9d640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f3406b4d0 0x7f1f3406d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c034000 tx=0x7f1f4c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:40.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.637+0000 7f1f4b7fe640 1 -- 172.21.15.19:0/3692148020 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f3c058800 con 0x7f1f540af9c0 2024-04-25T09:59:40.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.819+0000 7f1f5922a640 1 -- 172.21.15.19:0/3692148020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f540d0040 con 0x7f1f540af9c0 2024-04-25T09:59:40.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.820+0000 7f1f4b7fe640 1 -- 172.21.15.19:0/3692148020 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+13282 (secure 0 0 0) 0x7f1f3c05c8a0 con 0x7f1f540af9c0 2024-04-25T09:59:40.823 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:40.824 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":98,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:39.503431+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3589967477":"2024-04-26T09:59:36.487179+0000","172.21.15.19:6832/3589967477":"2024-04-26T09:59:36.487179+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:40.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.821+0000 7f1f5922a640 1 -- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f3406b4d0 msgr2=0x7f1f3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.821+0000 7f1f5922a640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f3406b4d0 0x7f1f3406d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c034000 tx=0x7f1f4c02e000 comp rx=0 tx=0).stop 2024-04-25T09:59:40.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.821+0000 7f1f5922a640 1 -- 172.21.15.19:0/3692148020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f540af9c0 msgr2=0x7f1f5415aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.821+0000 7f1f5922a640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f540af9c0 0x7f1f5415aea0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00e840 tx=0x7f1f3c00ed00 comp rx=0 tx=0).stop 2024-04-25T09:59:40.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.822+0000 7f1f5922a640 1 -- 172.21.15.19:0/3692148020 shutdown_connections 2024-04-25T09:59:40.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.822+0000 7f1f5922a640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1f3406b4d0 0x7f1f3406d990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.822+0000 7f1f5922a640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f5415a790 0x7f1f541e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.822+0000 7f1f5922a640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f540af9c0 0x7f1f5415aea0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.822+0000 7f1f5922a640 1 --2- 172.21.15.19:0/3692148020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f5406fea0 0x7f1f54145080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.822+0000 7f1f5922a640 1 -- 172.21.15.19:0/3692148020 >> 172.21.15.19:0/3692148020 conn(0x7f1f540b2370 msgr2=0x7f1f540b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:40.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.822+0000 7f1f5922a640 1 -- 172.21.15.19:0/3692148020 shutdown_connections 2024-04-25T09:59:40.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.822+0000 7f1f5922a640 1 -- 172.21.15.19:0/3692148020 wait complete. 2024-04-25T09:59:40.837 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:59:40.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:59:40.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.937+0000 7f62cd4ee640 1 Processor -- start 2024-04-25T09:59:40.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.937+0000 7f62cd4ee640 1 -- start start 2024-04-25T09:59:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.938+0000 7f62cd4ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80cf600 0x7f62c80cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.938+0000 7f62cd4ee640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c80c75e0 con 0x7f62c815bb50 2024-04-25T09:59:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.938+0000 7f62cd4ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c80c78c0 con 0x7f62c80cbf20 2024-04-25T09:59:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.938+0000 7f62cd4ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c80c8680 con 0x7f62c80cf600 2024-04-25T09:59:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.938+0000 7f62c6ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f62c815bb50 0x7f62c80c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50764/0 (socket says 172.21.15.19:50764) 2024-04-25T09:59:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.938+0000 7f62c6ffd640 1 -- 172.21.15.19:0/742353867 learned_addr learned my addr 172.21.15.19:0/742353867 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.938+0000 7f62c67fc640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80cf600 0x7f62c80cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.938+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720824650 0 0) 0x7f62c80c75e0 con 0x7f62c815bb50 2024-04-25T09:59:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.938+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62b4003520 con 0x7f62c815bb50 2024-04-25T09:59:40.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1492966997 0 0) 0x7f62b4003520 con 0x7f62c815bb50 2024-04-25T09:59:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80cf600 msgr2=0x7f62c80cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80cf600 0x7f62c80cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 >> v1:172.21.15.112:6789/0 conn(0x7f62c80cbf20 legacy=0x7f62c80cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c80c7ba0 con 0x7f62c815bb50 2024-04-25T09:59:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f62c0001670 con 0x7f62c815bb50 2024-04-25T09:59:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 >> v1:172.21.15.19:6789/0 conn(0x7f62c815bb50 legacy=0x7f62c80c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 0x7f62b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80cf600 0x7f62b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b4004d40 0x7f62b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c80c8680 con 0x7f62b40039e0 2024-04-25T09:59:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b400bb00 con 0x7f62b4004d40 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b400bd00 con 0x7f62c80cf600 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c6ffd640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 0x7f62b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c77fe640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b4004d40 0x7f62b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c67fc640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80cf600 0x7f62b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c6ffd640 1 -- 172.21.15.19:0/742353867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80cf600 msgr2=0x7f62b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c6ffd640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80cf600 0x7f62b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c6ffd640 1 -- 172.21.15.19:0/742353867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b4004d40 msgr2=0x7f62b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c6ffd640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b4004d40 0x7f62b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.939+0000 7f62c6ffd640 1 -- 172.21.15.19:0/742353867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b4003520 con 0x7f62b40039e0 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.940+0000 7f62c6ffd640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 0x7f62b4003e00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f62c0001eb0 tx=0x7f62b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9365e66b40600050 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.940+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0007d30 con 0x7f62b40039e0 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.940+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c00058d0 con 0x7f62b40039e0 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.940+0000 7f62c5ffb640 1 -- 172.21.15.19:0/742353867 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0005ce0 con 0x7f62b40039e0 2024-04-25T09:59:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.941+0000 7f62cd4ee640 1 -- 172.21.15.19:0/742353867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 msgr2=0x7f62b4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.941+0000 7f62cd4ee640 1 --2- 172.21.15.19:0/742353867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 0x7f62b4003e00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f62c0001eb0 tx=0x7f62b4005830 comp rx=0 tx=0).stop 2024-04-25T09:59:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.941+0000 7f62c77fe640 1 -- 172.21.15.19:0/742353867 reap_dead start 2024-04-25T09:59:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.941+0000 7f62cd4ee640 1 -- 172.21.15.19:0/742353867 shutdown_connections 2024-04-25T09:59:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.941+0000 7f62cd4ee640 1 -- 172.21.15.19:0/742353867 >> 172.21.15.19:0/742353867 conn(0x7f62c806f500 msgr2=0x7f62c806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.941+0000 7f62cd4ee640 1 -- 172.21.15.19:0/742353867 shutdown_connections 2024-04-25T09:59:40.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.941+0000 7f62cd4ee640 1 -- 172.21.15.19:0/742353867 wait complete. 2024-04-25T09:59:40.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.942+0000 7f62cd4ee640 1 Processor -- start 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.942+0000 7f62cd4ee640 1 -- start start 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62cd4ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 0x7f62c81578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62cd4ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62c80b0210 0x7f62c80c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62cd4ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80b17b0 0x7f62c80b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62cd4ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c80c72a0 con 0x7f62b40039e0 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62cd4ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c80c8160 con 0x7f62c80b0210 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62cd4ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c815f6d0 con 0x7f62c80b17b0 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 0x7f62c81578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 0x7f62c81578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51772/0 (socket says 172.21.15.19:51772) 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62c6ffd640 1 -- 172.21.15.19:0/2365793671 learned_addr learned my addr 172.21.15.19:0/2365793671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62c67fc640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62c80b0210 0x7f62c80c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62c77fe640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80b17b0 0x7f62c80b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62c6ffd640 1 -- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80b17b0 msgr2=0x7f62c80b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62c6ffd640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80b17b0 0x7f62c80b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62c6ffd640 1 -- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62c80b0210 msgr2=0x7f62c80c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62c6ffd640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62c80b0210 0x7f62c80c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62c6ffd640 1 -- 172.21.15.19:0/2365793671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c815b3a0 con 0x7f62b40039e0 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.943+0000 7f62c6ffd640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 0x7f62c81578f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f62c000c300 tx=0x7f62c0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.944+0000 7f62bf7fe640 1 -- 172.21.15.19:0/2365793671 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0038040 con 0x7f62b40039e0 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.944+0000 7f62bf7fe640 1 -- 172.21.15.19:0/2365793671 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c0000ed0 con 0x7f62b40039e0 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.944+0000 7f62cd4ee640 1 -- 172.21.15.19:0/2365793671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c80cb5e0 con 0x7f62b40039e0 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.944+0000 7f62bf7fe640 1 -- 172.21.15.19:0/2365793671 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0007490 con 0x7f62b40039e0 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.944+0000 7f62cd4ee640 1 -- 172.21.15.19:0/2365793671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c806c490 con 0x7f62b40039e0 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.945+0000 7f62cd4ee640 1 -- 172.21.15.19:0/2365793671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c80b05f0 con 0x7f62b40039e0 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.945+0000 7f62bf7fe640 1 -- 172.21.15.19:0/2365793671 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62c0033420 con 0x7f62b40039e0 2024-04-25T09:59:40.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.945+0000 7f62bf7fe640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62a806b4d0 0x7f62a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:40.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.946+0000 7f62bf7fe640 1 -- 172.21.15.19:0/2365793671 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f62c00aa570 con 0x7f62b40039e0 2024-04-25T09:59:40.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.946+0000 7f62c67fc640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62a806b4d0 0x7f62a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:40.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.946+0000 7f62c67fc640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62a806b4d0 0x7f62a806d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f62c80b7170 tx=0x7f62b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:40.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:40.950+0000 7f62bf7fe640 1 -- 172.21.15.19:0/2365793671 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62c0079560 con 0x7f62b40039e0 2024-04-25T09:59:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.133+0000 7f62cd4ee640 1 -- 172.21.15.19:0/2365793671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62c8073a00 con 0x7f62b40039e0 2024-04-25T09:59:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.133+0000 7f62bf7fe640 1 -- 172.21.15.19:0/2365793671 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62c0078e60 con 0x7f62b40039e0 2024-04-25T09:59:41.137 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 -- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62a806b4d0 msgr2=0x7f62a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62a806b4d0 0x7f62a806d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f62c80b7170 tx=0x7f62b0008040 comp rx=0 tx=0).stop 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 -- 172.21.15.19:0/2365793671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 msgr2=0x7f62c81578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 0x7f62c81578f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f62c000c300 tx=0x7f62c0001a40 comp rx=0 tx=0).stop 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 -- 172.21.15.19:0/2365793671 shutdown_connections 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62a806b4d0 0x7f62a806d990 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62c80b17b0 0x7f62c80b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62c80b0210 0x7f62c80c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 --2- 172.21.15.19:0/2365793671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b40039e0 0x7f62c81578f0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 -- 172.21.15.19:0/2365793671 >> 172.21.15.19:0/2365793671 conn(0x7f62c806f500 msgr2=0x7f62c80cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 -- 172.21.15.19:0/2365793671 shutdown_connections 2024-04-25T09:59:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.135+0000 7f62cd4ee640 1 -- 172.21.15.19:0/2365793671 wait complete. 2024-04-25T09:59:41.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T09:59:41.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.247+0000 7f56c413b640 1 Processor -- start 2024-04-25T09:59:41.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.247+0000 7f56c413b640 1 -- start start 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.248+0000 7f56c413b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f56bc0d2e90 0x7f56bc0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.248+0000 7f56c413b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc0b9940 con 0x7f56bc15dd30 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.248+0000 7f56c413b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc0d1df0 con 0x7f56bc0d0900 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.248+0000 7f56c413b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56bc0d0d40 con 0x7f56bc0d2e90 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.248+0000 7f56c1ed7640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f56bc15dd30 0x7f56bc0b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50778/0 (socket says 172.21.15.19:50778) 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.248+0000 7f56c1ed7640 1 -- 172.21.15.19:0/1883816947 learned_addr learned my addr 172.21.15.19:0/1883816947 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.248+0000 7f56c16d6640 1 --2- 172.21.15.19:0/1883816947 >> v2:172.21.15.112:3300/0 conn(0x7f56bc0d2e90 0x7f56bc0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.248+0000 7f56c0ed5640 1 -- 172.21.15.19:0/1883816947 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568304432 0 0) 0x7f56bc0b9940 con 0x7f56bc15dd30 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.248+0000 7f56c0ed5640 1 -- 172.21.15.19:0/1883816947 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a8003520 con 0x7f56bc15dd30 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.249+0000 7f56c16d6640 1 -- 172.21.15.19:0/1883816947 >> v1:172.21.15.112:6789/0 conn(0x7f56bc0d0900 legacy=0x7f56bc0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.249+0000 7f56c16d6640 1 -- 172.21.15.19:0/1883816947 >> v1:172.21.15.19:6789/0 conn(0x7f56bc15dd30 legacy=0x7f56bc0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.249+0000 7f56c0ed5640 1 -- 172.21.15.19:0/1883816947 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1061469727 0 0) 0x7f56a8003520 con 0x7f56bc15dd30 2024-04-25T09:59:41.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.249+0000 7f56c16d6640 1 -- 172.21.15.19:0/1883816947 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc0d20d0 con 0x7f56bc0d2e90 2024-04-25T09:59:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.249+0000 7f56c16d6640 1 --2- 172.21.15.19:0/1883816947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc0d2e90 0x7f56bc0d03c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009790 tx=0x7f56ac02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ea5c4cd8fa1a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.250+0000 7f56c0ed5640 1 -- 172.21.15.19:0/1883816947 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac037070 con 0x7f56bc0d2e90 2024-04-25T09:59:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.250+0000 7f56c0ed5640 1 -- 172.21.15.19:0/1883816947 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac02ab10 con 0x7f56bc0d2e90 2024-04-25T09:59:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.250+0000 7f56c0ed5640 1 -- 172.21.15.19:0/1883816947 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8001670 con 0x7f56bc0d2e90 2024-04-25T09:59:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.250+0000 7f56c413b640 1 -- 172.21.15.19:0/1883816947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc0d2e90 msgr2=0x7f56bc0d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.250+0000 7f56c413b640 1 --2- 172.21.15.19:0/1883816947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc0d2e90 0x7f56bc0d03c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009790 tx=0x7f56ac02a080 comp rx=0 tx=0).stop 2024-04-25T09:59:41.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.250+0000 7f56c413b640 1 -- 172.21.15.19:0/1883816947 shutdown_connections 2024-04-25T09:59:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.250+0000 7f56c413b640 1 --2- 172.21.15.19:0/1883816947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc0d2e90 0x7f56bc0d03c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.250+0000 7f56c413b640 1 -- 172.21.15.19:0/1883816947 >> 172.21.15.19:0/1883816947 conn(0x7f56bc061c30 msgr2=0x7f56bc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.250+0000 7f56c413b640 1 -- 172.21.15.19:0/1883816947 shutdown_connections 2024-04-25T09:59:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.250+0000 7f56c413b640 1 -- 172.21.15.19:0/1883816947 wait complete. 2024-04-25T09:59:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.251+0000 7f56c413b640 1 Processor -- start 2024-04-25T09:59:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.251+0000 7f56c413b640 1 -- start start 2024-04-25T09:59:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.252+0000 7f56c413b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc0d0900 0x7f56bc1f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.252+0000 7f56c413b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56bc0d2e90 0x7f56bc150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.252+0000 7f56c413b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56bc15dd30 0x7f56bc0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.252+0000 7f56c413b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0d2970 con 0x7f56bc15dd30 2024-04-25T09:59:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.252+0000 7f56c413b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0d2690 con 0x7f56bc0d0900 2024-04-25T09:59:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.252+0000 7f56c413b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0d1a10 con 0x7f56bc0d2e90 2024-04-25T09:59:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.252+0000 7f56c26d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56bc15dd30 0x7f56bc0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.252+0000 7f56c26d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56bc15dd30 0x7f56bc0b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51774/0 (socket says 172.21.15.19:51774) 2024-04-25T09:59:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.252+0000 7f56c26d8640 1 -- 172.21.15.19:0/1418986112 learned_addr learned my addr 172.21.15.19:0/1418986112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.252+0000 7f56c1ed7640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc0d0900 0x7f56bc1f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.252+0000 7f56c16d6640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56bc0d2e90 0x7f56bc150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56c26d8640 1 -- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56bc0d2e90 msgr2=0x7f56bc150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56c26d8640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56bc0d2e90 0x7f56bc150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56c26d8640 1 -- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc0d0900 msgr2=0x7f56bc1f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56c26d8640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc0d0900 0x7f56bc1f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56c26d8640 1 -- 172.21.15.19:0/1418986112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc0cae20 con 0x7f56bc15dd30 2024-04-25T09:59:41.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56c16d6640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56bc0d2e90 0x7f56bc150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56c26d8640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56bc15dd30 0x7f56bc0b53b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f56b400c6b0 tx=0x7f56b400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56b27fc640 1 -- 172.21.15.19:0/1418986112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4003b70 con 0x7f56bc15dd30 2024-04-25T09:59:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56b27fc640 1 -- 172.21.15.19:0/1418986112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b4003d30 con 0x7f56bc15dd30 2024-04-25T09:59:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56c413b640 1 -- 172.21.15.19:0/1418986112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc0cb100 con 0x7f56bc15dd30 2024-04-25T09:59:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56b27fc640 1 -- 172.21.15.19:0/1418986112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4005310 con 0x7f56bc15dd30 2024-04-25T09:59:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.253+0000 7f56c413b640 1 -- 172.21.15.19:0/1418986112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc0cb9a0 con 0x7f56bc15dd30 2024-04-25T09:59:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.254+0000 7f56c413b640 1 -- 172.21.15.19:0/1418986112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc0cb2d0 con 0x7f56bc15dd30 2024-04-25T09:59:41.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.254+0000 7f56b27fc640 1 -- 172.21.15.19:0/1418986112 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56b40054b0 con 0x7f56bc15dd30 2024-04-25T09:59:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.255+0000 7f56b27fc640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a006b4d0 0x7f56a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.255+0000 7f56b27fc640 1 -- 172.21.15.19:0/1418986112 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f56b4089d70 con 0x7f56bc15dd30 2024-04-25T09:59:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.255+0000 7f56c1ed7640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a006b4d0 0x7f56a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.256+0000 7f56c1ed7640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a006b4d0 0x7f56a006d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f56b8005880 tx=0x7f56b8000bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:41.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.259+0000 7f56b27fc640 1 -- 172.21.15.19:0/1418986112 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56b4058df0 con 0x7f56bc15dd30 2024-04-25T09:59:41.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.505+0000 7f56c413b640 1 -- 172.21.15.19:0/1418986112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f56bc073990 con 0x7f56bc15dd30 2024-04-25T09:59:41.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.516+0000 7f56b27fc640 1 -- 172.21.15.19:0/1418986112 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v326) v1 ==== 158+0+0 (secure 0 0 0) 0x7f56b40586f0 con 0x7f56bc15dd30 2024-04-25T09:59:41.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 -- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a006b4d0 msgr2=0x7f56a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a006b4d0 0x7f56a006d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f56b8005880 tx=0x7f56b8000bc0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 -- 172.21.15.19:0/1418986112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56bc15dd30 msgr2=0x7f56bc0b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56bc15dd30 0x7f56bc0b53b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f56b400c6b0 tx=0x7f56b400cb70 comp rx=0 tx=0).stop 2024-04-25T09:59:41.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 -- 172.21.15.19:0/1418986112 shutdown_connections 2024-04-25T09:59:41.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f56a006b4d0 0x7f56a006d990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f56bc15dd30 0x7f56bc0b53b0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56bc0d2e90 0x7f56bc150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 --2- 172.21.15.19:0/1418986112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56bc0d0900 0x7f56bc1f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 -- 172.21.15.19:0/1418986112 >> 172.21.15.19:0/1418986112 conn(0x7f56bc061c30 msgr2=0x7f56bc0c7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:41.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 -- 172.21.15.19:0/1418986112 shutdown_connections 2024-04-25T09:59:41.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.519+0000 7f56c413b640 1 -- 172.21.15.19:0/1418986112 wait complete. 2024-04-25T09:59:41.534 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:59:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.634+0000 7f67972db640 1 Processor -- start 2024-04-25T09:59:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.634+0000 7f67972db640 1 -- start start 2024-04-25T09:59:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f67972db640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f679015da90 0x7f6790161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f67972db640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f67900b9750 0x7f67900ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f67972db640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6790162040 con 0x7f6790161ba0 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f67972db640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6790162600 con 0x7f67900b9750 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f67972db640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6790162320 con 0x7f679015da90 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f6794876640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f67900b9750 0x7f67900ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f6795878640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6790161ba0 0x7f679006c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50786/0 (socket says 172.21.15.19:50786) 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f6795878640 1 -- 172.21.15.19:0/3211343793 learned_addr learned my addr 172.21.15.19:0/3211343793 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f6795077640 1 --2- 172.21.15.19:0/3211343793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f679015da90 0x7f6790161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f6794876640 1 -- 172.21.15.19:0/3211343793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f679015da90 msgr2=0x7f6790161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f6794876640 1 --2- 172.21.15.19:0/3211343793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f679015da90 0x7f6790161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f6794876640 1 -- 172.21.15.19:0/3211343793 >> v1:172.21.15.19:6789/0 conn(0x7f6790161ba0 legacy=0x7f679006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f6794876640 1 -- 172.21.15.19:0/3211343793 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67900d1b10 con 0x7f67900b9750 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.635+0000 7f6787fff640 1 -- 172.21.15.19:0/3211343793 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739208523 0 0) 0x7f6790162040 con 0x7f6790161ba0 2024-04-25T09:59:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.636+0000 7f6794876640 1 --2- 172.21.15.19:0/3211343793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900b9750 0x7f67900ce670 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f67800097a0 tx=0x7f678002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86eede2f3cfad2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.636+0000 7f6787fff640 1 -- 172.21.15.19:0/3211343793 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780037070 con 0x7f67900b9750 2024-04-25T09:59:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.636+0000 7f6787fff640 1 -- 172.21.15.19:0/3211343793 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678002ae50 con 0x7f67900b9750 2024-04-25T09:59:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.636+0000 7f6787fff640 1 -- 172.21.15.19:0/3211343793 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780032660 con 0x7f67900b9750 2024-04-25T09:59:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.637+0000 7f67972db640 1 -- 172.21.15.19:0/3211343793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900b9750 msgr2=0x7f67900ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.637+0000 7f67972db640 1 --2- 172.21.15.19:0/3211343793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900b9750 0x7f67900ce670 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f67800097a0 tx=0x7f678002a100 comp rx=0 tx=0).stop 2024-04-25T09:59:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.637+0000 7f67972db640 1 -- 172.21.15.19:0/3211343793 shutdown_connections 2024-04-25T09:59:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.637+0000 7f67972db640 1 --2- 172.21.15.19:0/3211343793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900b9750 0x7f67900ce670 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.637+0000 7f67972db640 1 --2- 172.21.15.19:0/3211343793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f679015da90 0x7f6790161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.637+0000 7f67972db640 1 -- 172.21.15.19:0/3211343793 >> 172.21.15.19:0/3211343793 conn(0x7f6790061a20 msgr2=0x7f6790061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.637+0000 7f67972db640 1 -- 172.21.15.19:0/3211343793 shutdown_connections 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.637+0000 7f67972db640 1 -- 172.21.15.19:0/3211343793 wait complete. 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.638+0000 7f67972db640 1 Processor -- start 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.638+0000 7f67972db640 1 -- start start 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f67972db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679015da90 0x7f67900c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f67972db640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6790161ba0 0x7f67901f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f67972db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900cad50 0x7f67900c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f67972db640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67900d3320 con 0x7f67900cad50 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f67972db640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679015de70 con 0x7f679015da90 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f67972db640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67900d34a0 con 0x7f6790161ba0 2024-04-25T09:59:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f6795878640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900cad50 0x7f67900c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f6795878640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900cad50 0x7f67900c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51802/0 (socket says 172.21.15.19:51802) 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f6795878640 1 -- 172.21.15.19:0/2264962294 learned_addr learned my addr 172.21.15.19:0/2264962294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f6794876640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6790161ba0 0x7f67901f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f6795077640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679015da90 0x7f67900c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f6795878640 1 -- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6790161ba0 msgr2=0x7f67901f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f6795878640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6790161ba0 0x7f67901f2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f6795878640 1 -- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679015da90 msgr2=0x7f67900c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f6795878640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679015da90 0x7f67900c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f6795878640 1 -- 172.21.15.19:0/2264962294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679015e190 con 0x7f67900cad50 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.639+0000 7f6795878640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900cad50 0x7f67900c8fa0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f678800c7e0 tx=0x7f678800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.640+0000 7f6785ffb640 1 -- 172.21.15.19:0/2264962294 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788018070 con 0x7f67900cad50 2024-04-25T09:59:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.640+0000 7f6785ffb640 1 -- 172.21.15.19:0/2264962294 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678800a640 con 0x7f67900cad50 2024-04-25T09:59:41.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.640+0000 7f67972db640 1 -- 172.21.15.19:0/2264962294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679015e430 con 0x7f67900cad50 2024-04-25T09:59:41.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.640+0000 7f6785ffb640 1 -- 172.21.15.19:0/2264962294 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678800aa10 con 0x7f67900cad50 2024-04-25T09:59:41.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.640+0000 7f67972db640 1 -- 172.21.15.19:0/2264962294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679015e9f0 con 0x7f67900cad50 2024-04-25T09:59:41.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.641+0000 7f67972db640 1 -- 172.21.15.19:0/2264962294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6790073a00 con 0x7f67900cad50 2024-04-25T09:59:41.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.641+0000 7f6785ffb640 1 -- 172.21.15.19:0/2264962294 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f678801b070 con 0x7f67900cad50 2024-04-25T09:59:41.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.641+0000 7f6785ffb640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f677406b4d0 0x7f677406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:41.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.642+0000 7f6785ffb640 1 -- 172.21.15.19:0/2264962294 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f67880895b0 con 0x7f67900cad50 2024-04-25T09:59:41.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.642+0000 7f6795077640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f677406b4d0 0x7f677406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:41.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.642+0000 7f6795077640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f677406b4d0 0x7f677406d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f678c006fd0 tx=0x7f678c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.646+0000 7f6785ffb640 1 -- 172.21.15.19:0/2264962294 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6788057f50 con 0x7f67900cad50 2024-04-25T09:59:41.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.826+0000 7f67972db640 1 -- 172.21.15.19:0/2264962294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f67900d1590 con 0x7f67900cad50 2024-04-25T09:59:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.829+0000 7f6785ffb640 1 -- 172.21.15.19:0/2264962294 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f678805bff0 con 0x7f67900cad50 2024-04-25T09:59:41.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 -- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f677406b4d0 msgr2=0x7f677406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f677406b4d0 0x7f677406d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f678c006fd0 tx=0x7f678c008040 comp rx=0 tx=0).stop 2024-04-25T09:59:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 -- 172.21.15.19:0/2264962294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900cad50 msgr2=0x7f67900c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900cad50 0x7f67900c8fa0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f678800c7e0 tx=0x7f678800cca0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 -- 172.21.15.19:0/2264962294 shutdown_connections 2024-04-25T09:59:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f677406b4d0 0x7f677406d990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900cad50 0x7f67900c8fa0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6790161ba0 0x7f67901f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 --2- 172.21.15.19:0/2264962294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f679015da90 0x7f67900c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 -- 172.21.15.19:0/2264962294 >> 172.21.15.19:0/2264962294 conn(0x7f6790061a20 msgr2=0x7f679015b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 -- 172.21.15.19:0/2264962294 shutdown_connections 2024-04-25T09:59:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:41.831+0000 7f67972db640 1 -- 172.21.15.19:0/2264962294 wait complete. 2024-04-25T09:59:41.846 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:59:41.846 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:59:41.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:59:41.933 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:59:41.933 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:59:41.933 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:41.948 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:59:41.949 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:59:41.949 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:41.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:59:41.991 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:59:41.991 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:42.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:59:42.007 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T09:59:42.007 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:42.048 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:59:42.048 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:59:42.048 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:42.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.145+0000 7f8414b9e640 1 Processor -- start 2024-04-25T09:59:42.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.145+0000 7f8414b9e640 1 -- start start 2024-04-25T09:59:42.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.145+0000 7f8414b9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f841015dd30 0x7f84100b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.145+0000 7f8414b9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f84100b9770 0x7f84100d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.145+0000 7f8414b9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84100d1df0 con 0x7f84100d2e90 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.145+0000 7f8414b9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84100b7a50 con 0x7f84100b9770 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.145+0000 7f8414b9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84100d20d0 con 0x7f841015dd30 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.146+0000 7f840dd9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f84100b9770 0x7f84100d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.146+0000 7f840dd9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f84100b9770 0x7f84100d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51812/0 (socket says 172.21.15.19:51812) 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.146+0000 7f840dd9b640 1 -- 172.21.15.19:0/4078978509 learned_addr learned my addr 172.21.15.19:0/4078978509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.146+0000 7f840ed9d640 1 --1- 172.21.15.19:0/4078978509 >> v1:172.21.15.19:6789/0 conn(0x7f84100d2e90 0x7f84100b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50802/0 (socket says 172.21.15.19:50802) 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.146+0000 7f840e59c640 1 --2- 172.21.15.19:0/4078978509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f841015dd30 0x7f84100b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.146+0000 7f840dd9b640 1 -- 172.21.15.19:0/4078978509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f841015dd30 msgr2=0x7f84100b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.146+0000 7f840dd9b640 1 --2- 172.21.15.19:0/4078978509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f841015dd30 0x7f84100b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.146+0000 7f840dd9b640 1 -- 172.21.15.19:0/4078978509 >> v1:172.21.15.19:6789/0 conn(0x7f84100d2e90 legacy=0x7f84100b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.146+0000 7f840dd9b640 1 -- 172.21.15.19:0/4078978509 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84100d23b0 con 0x7f84100b9770 2024-04-25T09:59:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.146+0000 7f840e59c640 1 --2- 172.21.15.19:0/4078978509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f841015dd30 0x7f84100b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.147+0000 7f840dd9b640 1 --2- 172.21.15.19:0/4078978509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84100b9770 0x7f84100d03c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009850 tx=0x7f83f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04b8ce09ef05c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.147+0000 7f840d59a640 1 -- 172.21.15.19:0/4078978509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8004030 con 0x7f84100b9770 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.147+0000 7f840d59a640 1 -- 172.21.15.19:0/4078978509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f80041d0 con 0x7f84100b9770 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.147+0000 7f840d59a640 1 -- 172.21.15.19:0/4078978509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8003610 con 0x7f84100b9770 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.147+0000 7f8414b9e640 1 -- 172.21.15.19:0/4078978509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84100b9770 msgr2=0x7f84100d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.147+0000 7f8414b9e640 1 --2- 172.21.15.19:0/4078978509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84100b9770 0x7f84100d03c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009850 tx=0x7f83f802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.147+0000 7f8414b9e640 1 -- 172.21.15.19:0/4078978509 shutdown_connections 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.147+0000 7f8414b9e640 1 --2- 172.21.15.19:0/4078978509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84100b9770 0x7f84100d03c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.147+0000 7f8414b9e640 1 --2- 172.21.15.19:0/4078978509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f841015dd30 0x7f84100b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.147+0000 7f8414b9e640 1 -- 172.21.15.19:0/4078978509 >> 172.21.15.19:0/4078978509 conn(0x7f8410061c30 msgr2=0x7f841006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.148+0000 7f8414b9e640 1 -- 172.21.15.19:0/4078978509 shutdown_connections 2024-04-25T09:59:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.148+0000 7f8414b9e640 1 -- 172.21.15.19:0/4078978509 wait complete. 2024-04-25T09:59:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.149+0000 7f8414b9e640 1 Processor -- start 2024-04-25T09:59:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.149+0000 7f8414b9e640 1 -- start start 2024-04-25T09:59:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f8414b9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84100b9770 0x7f84100aa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f8414b9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84100d2e90 0x7f841014b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f8414b9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f841015dd30 0x7f841014ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f8414b9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100d1a10 con 0x7f841015dd30 2024-04-25T09:59:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f8414b9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100d2970 con 0x7f84100d2e90 2024-04-25T09:59:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f8414b9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84100d1b90 con 0x7f84100b9770 2024-04-25T09:59:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f841015dd30 0x7f841014ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f841015dd30 0x7f841014ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51814/0 (socket says 172.21.15.19:51814) 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840ed9d640 1 -- 172.21.15.19:0/446467650 learned_addr learned my addr 172.21.15.19:0/446467650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840dd9b640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84100d2e90 0x7f841014b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840e59c640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84100b9770 0x7f84100aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840ed9d640 1 -- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84100b9770 msgr2=0x7f84100aa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840ed9d640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84100b9770 0x7f84100aa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840ed9d640 1 -- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84100d2e90 msgr2=0x7f841014b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840ed9d640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84100d2e90 0x7f841014b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840ed9d640 1 -- 172.21.15.19:0/446467650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841014f6c0 con 0x7f841015dd30 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840e59c640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84100b9770 0x7f84100aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840dd9b640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84100d2e90 0x7f841014b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f840ed9d640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f841015dd30 0x7f841014ba20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f840800c3c0 tx=0x7f840800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f8406ffd640 1 -- 172.21.15.19:0/446467650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080074e0 con 0x7f841015dd30 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f8406ffd640 1 -- 172.21.15.19:0/446467650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8408007680 con 0x7f841015dd30 2024-04-25T09:59:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f8406ffd640 1 -- 172.21.15.19:0/446467650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408004a00 con 0x7f841015dd30 2024-04-25T09:59:42.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.150+0000 7f8414b9e640 1 -- 172.21.15.19:0/446467650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84100cd860 con 0x7f841015dd30 2024-04-25T09:59:42.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.151+0000 7f8414b9e640 1 -- 172.21.15.19:0/446467650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841014c700 con 0x7f841015dd30 2024-04-25T09:59:42.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.151+0000 7f8414b9e640 1 -- 172.21.15.19:0/446467650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8410073990 con 0x7f841015dd30 2024-04-25T09:59:42.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.152+0000 7f8406ffd640 1 -- 172.21.15.19:0/446467650 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f840801a020 con 0x7f841015dd30 2024-04-25T09:59:42.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.152+0000 7f8406ffd640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83f006b4d0 0x7f83f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.153+0000 7f8406ffd640 1 -- 172.21.15.19:0/446467650 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8408089790 con 0x7f841015dd30 2024-04-25T09:59:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.153+0000 7f840e59c640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83f006b4d0 0x7f83f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.153+0000 7f840e59c640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83f006b4d0 0x7f83f006d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8400007f40 tx=0x7f8400007490 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.157+0000 7f8406ffd640 1 -- 172.21.15.19:0/446467650 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f841014f6c0 con 0x7f841015dd30 2024-04-25T09:59:42.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.374+0000 7f8414b9e640 1 -- 172.21.15.19:0/446467650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f841006b210 con 0x7f841015dd30 2024-04-25T09:59:42.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.374+0000 7f8406ffd640 1 -- 172.21.15.19:0/446467650 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f8408058810 con 0x7f841015dd30 2024-04-25T09:59:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:42.379 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3431952054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3431952054},{"type":"v1","addr":"172.21.15.112:6836","nonce":3431952054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2758109099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2758109099},{"type":"v1","addr":"172.21.15.112:6837","nonce":2758109099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7590,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3063446116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3063446116},{"type":"v1","addr":"172.21.15.19:6834","nonce":3063446116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7944,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1189171433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1189171433},{"type":"v1","addr":"172.21.15.19:6835","nonce":1189171433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[]} 2024-04-25T09:59:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 60 2024-04-25T09:59:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 -- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83f006b4d0 msgr2=0x7f83f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83f006b4d0 0x7f83f006d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8400007f40 tx=0x7f8400007490 comp rx=0 tx=0).stop 2024-04-25T09:59:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 -- 172.21.15.19:0/446467650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f841015dd30 msgr2=0x7f841014ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f841015dd30 0x7f841014ba20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f840800c3c0 tx=0x7f840800c880 comp rx=0 tx=0).stop 2024-04-25T09:59:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 -- 172.21.15.19:0/446467650 shutdown_connections 2024-04-25T09:59:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83f006b4d0 0x7f83f006d990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f841015dd30 0x7f841014ba20 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84100d2e90 0x7f841014b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 --2- 172.21.15.19:0/446467650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84100b9770 0x7f84100aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 -- 172.21.15.19:0/446467650 >> 172.21.15.19:0/446467650 conn(0x7f8410061c30 msgr2=0x7f84100c8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 -- 172.21.15.19:0/446467650 shutdown_connections 2024-04-25T09:59:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.377+0000 7f8414b9e640 1 -- 172.21.15.19:0/446467650 wait complete. 2024-04-25T09:59:42.392 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:59:42.392 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:59:42.392 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:59:42.392 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:59:42.392 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:59:42.393 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:59:42.393 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:59:42.393 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:59:42.393 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:59:42.393 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:59:42.393 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:59:42.393 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:59:42.394 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:59:42.394 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:59:42.394 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:59:42.394 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:59:42.401 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:59:42.401 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:59:42.402 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:59:42.402 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:59:42.405 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:59:42.405 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:59:42.406 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:59:42.406 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:59:42.445 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:59:42.445 DEBUG:teuthology.parallel:result is None 2024-04-25T09:59:42.446 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:59:42.446 DEBUG:teuthology.parallel:result is None 2024-04-25T09:59:42.447 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:59:42.447 DEBUG:teuthology.parallel:result is None 2024-04-25T09:59:42.448 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:59:42.448 DEBUG:teuthology.parallel:result is None 2024-04-25T09:59:42.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:59:42.520 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T09:59:42.521 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T09:59:42.526 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T09:59:42.527 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T09:59:42.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.550+0000 7fb25f26f640 1 Processor -- start 2024-04-25T09:59:42.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.550+0000 7fb25f26f640 1 -- start start 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.551+0000 7fb25f26f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb25815dc20 0x7fb2580d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.551+0000 7fb25f26f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2580d1ce0 0x7fb2580b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.551+0000 7fb25f26f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2580d2120 con 0x7fb2580ce670 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.551+0000 7fb25f26f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2580d23a0 con 0x7fb25815dc20 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.551+0000 7fb25f26f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2580d2680 con 0x7fb2580d1ce0 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.552+0000 7fb25c80a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2580d1ce0 0x7fb2580b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.552+0000 7fb25d80c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb2580ce670 0x7fb25806c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:53924/0 (socket says 172.21.15.19:53924) 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.552+0000 7fb25d00b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb25815dc20 0x7fb2580d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.552+0000 7fb25d80c640 1 -- 172.21.15.19:0/2816528981 learned_addr learned my addr 172.21.15.19:0/2816528981 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.552+0000 7fb25c80a640 1 -- 172.21.15.19:0/2816528981 >> v2:172.21.15.112:3300/0 conn(0x7fb25815dc20 msgr2=0x7fb2580d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.552+0000 7fb25c80a640 1 --2- 172.21.15.19:0/2816528981 >> v2:172.21.15.112:3300/0 conn(0x7fb25815dc20 0x7fb2580d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.552+0000 7fb25c80a640 1 -- 172.21.15.19:0/2816528981 >> v1:172.21.15.112:6789/0 conn(0x7fb2580ce670 legacy=0x7fb25806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:42.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.552+0000 7fb25c80a640 1 -- 172.21.15.19:0/2816528981 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2580d2960 con 0x7fb2580d1ce0 2024-04-25T09:59:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb25c80a640 1 --2- 172.21.15.19:0/2816528981 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2580d1ce0 0x7fb2580b9490 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb2480097a0 tx=0x7fb24802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cade3994dbb744a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb24ffff640 1 -- 172.21.15.19:0/2816528981 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248037070 con 0x7fb2580d1ce0 2024-04-25T09:59:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb24ffff640 1 -- 172.21.15.19:0/2816528981 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24802ae50 con 0x7fb2580d1ce0 2024-04-25T09:59:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb24ffff640 1 -- 172.21.15.19:0/2816528981 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248032660 con 0x7fb2580d1ce0 2024-04-25T09:59:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb25f26f640 1 -- 172.21.15.19:0/2816528981 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2580d1ce0 msgr2=0x7fb2580b9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb25f26f640 1 --2- 172.21.15.19:0/2816528981 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2580d1ce0 0x7fb2580b9490 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb2480097a0 tx=0x7fb24802a100 comp rx=0 tx=0).stop 2024-04-25T09:59:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb25f26f640 1 -- 172.21.15.19:0/2816528981 shutdown_connections 2024-04-25T09:59:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb25f26f640 1 --2- 172.21.15.19:0/2816528981 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2580d1ce0 0x7fb2580b9490 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb25f26f640 1 --2- 172.21.15.19:0/2816528981 >> v2:172.21.15.112:3300/0 conn(0x7fb25815dc20 0x7fb2580d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb25f26f640 1 -- 172.21.15.19:0/2816528981 >> 172.21.15.19:0/2816528981 conn(0x7fb258061a20 msgr2=0x7fb258061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb25f26f640 1 -- 172.21.15.19:0/2816528981 shutdown_connections 2024-04-25T09:59:42.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.553+0000 7fb25f26f640 1 -- 172.21.15.19:0/2816528981 wait complete. 2024-04-25T09:59:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.555+0000 7fb25f26f640 1 Processor -- start 2024-04-25T09:59:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.555+0000 7fb25f26f640 1 -- start start 2024-04-25T09:59:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.555+0000 7fb25f26f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2580ce670 0x7fb2580b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.555+0000 7fb25f26f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2580d1ce0 0x7fb258150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.555+0000 7fb25f26f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb25815dc20 0x7fb258151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.555+0000 7fb25f26f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2580ceb00 con 0x7fb2580d1ce0 2024-04-25T09:59:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.555+0000 7fb25f26f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2580d2f20 con 0x7fb2580ce670 2024-04-25T09:59:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.555+0000 7fb25f26f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2580cec80 con 0x7fb25815dc20 2024-04-25T09:59:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.555+0000 7fb25c80a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2580d1ce0 0x7fb258150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.555+0000 7fb25c80a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2580d1ce0 0x7fb258150d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51856/0 (socket says 172.21.15.19:51856) 2024-04-25T09:59:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.555+0000 7fb25c80a640 1 -- 172.21.15.19:0/151329887 learned_addr learned my addr 172.21.15.19:0/151329887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb25d00b640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2580ce670 0x7fb2580b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb25d80c640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb25815dc20 0x7fb258151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb25c80a640 1 -- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb25815dc20 msgr2=0x7fb258151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb25c80a640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb25815dc20 0x7fb258151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb25c80a640 1 -- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2580ce670 msgr2=0x7fb2580b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb25c80a640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2580ce670 0x7fb2580b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb25c80a640 1 -- 172.21.15.19:0/151329887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25815a370 con 0x7fb2580d1ce0 2024-04-25T09:59:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb25c80a640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2580d1ce0 0x7fb258150d30 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb248035040 tx=0x7fb2480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb24dffb640 1 -- 172.21.15.19:0/151329887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248041070 con 0x7fb2580d1ce0 2024-04-25T09:59:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb24dffb640 1 -- 172.21.15.19:0/151329887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24803c460 con 0x7fb2580d1ce0 2024-04-25T09:59:42.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb24dffb640 1 -- 172.21.15.19:0/151329887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248037040 con 0x7fb2580d1ce0 2024-04-25T09:59:42.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb25f26f640 1 -- 172.21.15.19:0/151329887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2580cd370 con 0x7fb2580d1ce0 2024-04-25T09:59:42.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.556+0000 7fb25f26f640 1 -- 172.21.15.19:0/151329887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2580cd930 con 0x7fb2580d1ce0 2024-04-25T09:59:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.558+0000 7fb25f26f640 1 -- 172.21.15.19:0/151329887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb258073a00 con 0x7fb2580d1ce0 2024-04-25T09:59:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.558+0000 7fb24dffb640 1 -- 172.21.15.19:0/151329887 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb24803a070 con 0x7fb2580d1ce0 2024-04-25T09:59:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.558+0000 7fb24dffb640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb23c06b4d0 0x7fb23c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.559+0000 7fb24dffb640 1 -- 172.21.15.19:0/151329887 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb2480aa4e0 con 0x7fb2580d1ce0 2024-04-25T09:59:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.559+0000 7fb25d00b640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb23c06b4d0 0x7fb23c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.559+0000 7fb25d00b640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb23c06b4d0 0x7fb23c06d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb254009aa0 tx=0x7fb254009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:42.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.563+0000 7fb24dffb640 1 -- 172.21.15.19:0/151329887 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb248078de0 con 0x7fb2580d1ce0 2024-04-25T09:59:42.607 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T09:59:42.607 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T09:59:42.607 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T09:59:42.607 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T09:59:42.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:42.756+0000 7fb25f26f640 1 -- 172.21.15.19:0/151329887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb2580cddd0 con 0x7fb2580d1ce0 2024-04-25T09:59:43.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.534+0000 7fb24dffb640 1 -- 172.21.15.19:0/151329887 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v100) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb24807ce80 con 0x7fb2580d1ce0 2024-04-25T09:59:43.538 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T09:59:43.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 -- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb23c06b4d0 msgr2=0x7fb23c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:43.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb23c06b4d0 0x7fb23c06d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb254009aa0 tx=0x7fb254009190 comp rx=0 tx=0).stop 2024-04-25T09:59:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 -- 172.21.15.19:0/151329887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2580d1ce0 msgr2=0x7fb258150d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2580d1ce0 0x7fb258150d30 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb248035040 tx=0x7fb2480046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 -- 172.21.15.19:0/151329887 shutdown_connections 2024-04-25T09:59:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb23c06b4d0 0x7fb23c06d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb25815dc20 0x7fb258151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2580d1ce0 0x7fb258150d30 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 --2- 172.21.15.19:0/151329887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2580ce670 0x7fb2580b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 -- 172.21.15.19:0/151329887 >> 172.21.15.19:0/151329887 conn(0x7fb258061a20 msgr2=0x7fb25815c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 -- 172.21.15.19:0/151329887 shutdown_connections 2024-04-25T09:59:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.538+0000 7fb25f26f640 1 -- 172.21.15.19:0/151329887 wait complete. 2024-04-25T09:59:43.557 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:59:43.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.660+0000 7f9bed79e640 1 Processor -- start 2024-04-25T09:59:43.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.660+0000 7f9bed79e640 1 -- start start 2024-04-25T09:59:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.661+0000 7f9bed79e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be80b6cb0 0x7f9be80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.661+0000 7f9bed79e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be80ceee0 con 0x7f9be815db70 2024-04-25T09:59:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.661+0000 7f9bed79e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be80d1d10 con 0x7f9be80b5a60 2024-04-25T09:59:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.661+0000 7f9bed79e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80d1ff0 con 0x7f9be80b6cb0 2024-04-25T09:59:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.661+0000 7f9be6ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9be815db70 0x7f9be8162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50818/0 (socket says 172.21.15.19:50818) 2024-04-25T09:59:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.661+0000 7f9be6ffd640 1 -- 172.21.15.19:0/3868770738 learned_addr learned my addr 172.21.15.19:0/3868770738 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.661+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825396509 0 0) 0x7f9be80ceee0 con 0x7f9be815db70 2024-04-25T09:59:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.661+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bd4003520 con 0x7f9be815db70 2024-04-25T09:59:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.661+0000 7f9be67fc640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be80b6cb0 0x7f9be80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 749104564 0 0) 0x7f9bd4003520 con 0x7f9be815db70 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be80b6cb0 msgr2=0x7f9be80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be80b6cb0 0x7f9be80b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 >> v1:172.21.15.112:6789/0 conn(0x7f9be80b5a60 legacy=0x7f9be80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80d22d0 con 0x7f9be815db70 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f9be0001670 con 0x7f9be815db70 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 >> v1:172.21.15.19:6789/0 conn(0x7f9be815db70 legacy=0x7f9be8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 0x7f9bd4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be80b6cb0 0x7f9bd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd4004d40 0x7f9bd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80d1ff0 con 0x7f9bd40039e0 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd400ba70 con 0x7f9bd4004d40 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd400bc20 con 0x7f9be80b6cb0 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be6ffd640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 0x7f9bd4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be67fc640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be80b6cb0 0x7f9bd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be77fe640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd4004d40 0x7f9bd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be6ffd640 1 -- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be80b6cb0 msgr2=0x7f9bd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be6ffd640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be80b6cb0 0x7f9bd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be6ffd640 1 -- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd4004d40 msgr2=0x7f9bd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be6ffd640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd4004d40 0x7f9bd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be6ffd640 1 -- 172.21.15.19:0/3868770738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd4003520 con 0x7f9bd40039e0 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.662+0000 7f9be77fe640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bd4004d40 0x7f9bd40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.663+0000 7f9be6ffd640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 0x7f9bd4003dc0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c110 tx=0x7f9bd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c0fc49a8e388e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.663+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0007c40 con 0x7f9bd40039e0 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.663+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be0005ad0 con 0x7f9bd40039e0 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.663+0000 7f9be5ffb640 1 -- 172.21.15.19:0/3868770738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0007070 con 0x7f9bd40039e0 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.664+0000 7f9bed79e640 1 -- 172.21.15.19:0/3868770738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 msgr2=0x7f9bd4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.664+0000 7f9bed79e640 1 --2- 172.21.15.19:0/3868770738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 0x7f9bd4003dc0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c110 tx=0x7f9bd4005830 comp rx=0 tx=0).stop 2024-04-25T09:59:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.664+0000 7f9be77fe640 1 -- 172.21.15.19:0/3868770738 reap_dead start 2024-04-25T09:59:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.664+0000 7f9bed79e640 1 -- 172.21.15.19:0/3868770738 shutdown_connections 2024-04-25T09:59:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.664+0000 7f9bed79e640 1 -- 172.21.15.19:0/3868770738 >> 172.21.15.19:0/3868770738 conn(0x7f9be80d0520 msgr2=0x7f9be80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.664+0000 7f9bed79e640 1 -- 172.21.15.19:0/3868770738 shutdown_connections 2024-04-25T09:59:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.664+0000 7f9bed79e640 1 -- 172.21.15.19:0/3868770738 wait complete. 2024-04-25T09:59:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.665+0000 7f9bed79e640 1 Processor -- start 2024-04-25T09:59:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.665+0000 7f9bed79e640 1 -- start start 2024-04-25T09:59:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9bed79e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 0x7f9be8160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9bed79e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be815e160 0x7f9be80c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9bed79e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9be81558a0 0x7f9be8155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9bed79e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80cf1c0 con 0x7f9bd40039e0 2024-04-25T09:59:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9bed79e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80ce530 con 0x7f9be81558a0 2024-04-25T09:59:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9bed79e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80cf340 con 0x7f9be815e160 2024-04-25T09:59:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9be6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 0x7f9be8160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9be6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 0x7f9be8160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51870/0 (socket says 172.21.15.19:51870) 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9be6ffd640 1 -- 172.21.15.19:0/1971266763 learned_addr learned my addr 172.21.15.19:0/1971266763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9be67fc640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be815e160 0x7f9be80c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9be77fe640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9be81558a0 0x7f9be8155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9be6ffd640 1 -- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be815e160 msgr2=0x7f9be80c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9be6ffd640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be815e160 0x7f9be80c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9be6ffd640 1 -- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9be81558a0 msgr2=0x7f9be8155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9be6ffd640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9be81558a0 0x7f9be8155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.666+0000 7f9be6ffd640 1 -- 172.21.15.19:0/1971266763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be815e5f0 con 0x7f9bd40039e0 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.667+0000 7f9be6ffd640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 0x7f9be8160db0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c230 tx=0x7f9be80d1ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.667+0000 7f9bdf7fe640 1 -- 172.21.15.19:0/1971266763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0040020 con 0x7f9bd40039e0 2024-04-25T09:59:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.667+0000 7f9bdf7fe640 1 -- 172.21.15.19:0/1971266763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be0001430 con 0x7f9bd40039e0 2024-04-25T09:59:43.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.667+0000 7f9bed79e640 1 -- 172.21.15.19:0/1971266763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be815e8d0 con 0x7f9bd40039e0 2024-04-25T09:59:43.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.667+0000 7f9bdf7fe640 1 -- 172.21.15.19:0/1971266763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0037040 con 0x7f9bd40039e0 2024-04-25T09:59:43.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.667+0000 7f9bed79e640 1 -- 172.21.15.19:0/1971266763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be815ee90 con 0x7f9bd40039e0 2024-04-25T09:59:43.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.668+0000 7f9bed79e640 1 -- 172.21.15.19:0/1971266763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be8073990 con 0x7f9bd40039e0 2024-04-25T09:59:43.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.668+0000 7f9bdf7fe640 1 -- 172.21.15.19:0/1971266763 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9be0043020 con 0x7f9bd40039e0 2024-04-25T09:59:43.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.669+0000 7f9bdf7fe640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9bc806b500 0x7f9bc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:43.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.669+0000 7f9bdf7fe640 1 -- 172.21.15.19:0/1971266763 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9be00a9c80 con 0x7f9bd40039e0 2024-04-25T09:59:43.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.669+0000 7f9be67fc640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9bc806b500 0x7f9bc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:43.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.669+0000 7f9be67fc640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9bc806b500 0x7f9bc806d9c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9be80d31f0 tx=0x7f9bd0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:43.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.673+0000 7f9bdf7fe640 1 -- 172.21.15.19:0/1971266763 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9be0005aa0 con 0x7f9bd40039e0 2024-04-25T09:59:43.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.919+0000 7f9bed79e640 1 -- 172.21.15.19:0/1971266763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9be80ce310 con 0x7f9bd40039e0 2024-04-25T09:59:43.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.920+0000 7f9bdf7fe640 1 -- 172.21.15.19:0/1971266763 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9be0078cd0 con 0x7f9bd40039e0 2024-04-25T09:59:43.924 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:43.924 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:59:43.924 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T09:59:43.924 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.924 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T09:59:43.924 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T09:59:43.924 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.924 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:43.924 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:59:43.924 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:59:43.924 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.925 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.926 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T09:59:43.926 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T09:59:43.926 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.926 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:43.926 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:59:43.926 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:59:43.926 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.926 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:43.927 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T09:59:43.928 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.929 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T09:59:43.930 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.931 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T09:59:43.932 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T09:59:43.933 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T09:59:43.934 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.936 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.936 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T09:59:43.936 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T09:59:43.936 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.936 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:59:43.936 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:59:43.936 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.936 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T09:59:43.936 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:43.936 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T09:59:43.937 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T09:59:43.937 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T09:59:43.937 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T09:59:43.937 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T09:59:43.937 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T09:59:43.937 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.937 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:43.937 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T09:59:43.937 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:59:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 -- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9bc806b500 msgr2=0x7f9bc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9bc806b500 0x7f9bc806d9c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9be80d31f0 tx=0x7f9bd0009190 comp rx=0 tx=0).stop 2024-04-25T09:59:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 -- 172.21.15.19:0/1971266763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 msgr2=0x7f9be8160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 0x7f9be8160db0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c230 tx=0x7f9be80d1ee0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 -- 172.21.15.19:0/1971266763 shutdown_connections 2024-04-25T09:59:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9bc806b500 0x7f9bc806d9c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9be81558a0 0x7f9be8155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9be815e160 0x7f9be80c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 --2- 172.21.15.19:0/1971266763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bd40039e0 0x7f9be8160db0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 -- 172.21.15.19:0/1971266763 >> 172.21.15.19:0/1971266763 conn(0x7f9be80d0520 msgr2=0x7f9be81581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 -- 172.21.15.19:0/1971266763 shutdown_connections 2024-04-25T09:59:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:43.921+0000 7f9bed79e640 1 -- 172.21.15.19:0/1971266763 wait complete. 2024-04-25T09:59:43.943 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:44.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.040+0000 7f6be77ee640 1 Processor -- start 2024-04-25T09:59:44.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.040+0000 7f6be77ee640 1 -- start start 2024-04-25T09:59:44.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.041+0000 7f6be77ee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6be00cc8e0 0x7f6be01625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.041+0000 7f6be77ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6be00d1810 0x7f6be00d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.041+0000 7f6be77ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be00cce50 con 0x7f6be015dc90 2024-04-25T09:59:44.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.041+0000 7f6be77ee640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be00d21c0 con 0x7f6be00cc8e0 2024-04-25T09:59:44.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.041+0000 7f6be77ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be0161af0 con 0x7f6be00d1810 2024-04-25T09:59:44.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.041+0000 7f6be5d8b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6be00d1810 0x7f6be00d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.041+0000 7f6be4d89640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6be00cc8e0 0x7f6be01625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.042+0000 7f6be4d89640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6be00cc8e0 0x7f6be01625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:50228/0 (socket says 172.21.15.19:50228) 2024-04-25T09:59:44.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.042+0000 7f6be558a640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6be015dc90 0x7f6be015ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:53946/0 (socket says 172.21.15.19:53946) 2024-04-25T09:59:44.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.042+0000 7f6be4d89640 1 -- 172.21.15.19:0/1500042498 learned_addr learned my addr 172.21.15.19:0/1500042498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:44.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.042+0000 7f6be5d8b640 1 --2- 172.21.15.19:0/1500042498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6be00d1810 0x7f6be00d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:54544/0 (socket says 172.21.15.19:54544) 2024-04-25T09:59:44.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.042+0000 7f6be4d89640 1 -- 172.21.15.19:0/1500042498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6be00d1810 msgr2=0x7f6be00d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.042+0000 7f6be4d89640 1 --2- 172.21.15.19:0/1500042498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6be00d1810 0x7f6be00d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.042+0000 7f6be4d89640 1 -- 172.21.15.19:0/1500042498 >> v1:172.21.15.112:6789/0 conn(0x7f6be015dc90 legacy=0x7f6be015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.042+0000 7f6bd7fff640 1 -- 172.21.15.19:0/1500042498 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616564127 0 0) 0x7f6be00cce50 con 0x7f6be015dc90 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.042+0000 7f6be4d89640 1 -- 172.21.15.19:0/1500042498 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be0161dd0 con 0x7f6be00cc8e0 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.043+0000 7f6be4d89640 1 --2- 172.21.15.19:0/1500042498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be00cc8e0 0x7f6be01625d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009870 tx=0x7f6bd002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6322f0018920233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.043+0000 7f6bd7fff640 1 -- 172.21.15.19:0/1500042498 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0037070 con 0x7f6be00cc8e0 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.043+0000 7f6bd7fff640 1 -- 172.21.15.19:0/1500042498 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd0032440 con 0x7f6be00cc8e0 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.043+0000 7f6bd7fff640 1 -- 172.21.15.19:0/1500042498 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0032810 con 0x7f6be00cc8e0 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.043+0000 7f6be77ee640 1 -- 172.21.15.19:0/1500042498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be00cc8e0 msgr2=0x7f6be01625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.043+0000 7f6be77ee640 1 --2- 172.21.15.19:0/1500042498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be00cc8e0 0x7f6be01625d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009870 tx=0x7f6bd002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.043+0000 7f6be77ee640 1 -- 172.21.15.19:0/1500042498 shutdown_connections 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.043+0000 7f6be77ee640 1 --2- 172.21.15.19:0/1500042498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6be00d1810 0x7f6be00d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.043+0000 7f6be77ee640 1 --2- 172.21.15.19:0/1500042498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be00cc8e0 0x7f6be01625d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.043+0000 7f6be77ee640 1 -- 172.21.15.19:0/1500042498 >> 172.21.15.19:0/1500042498 conn(0x7f6be006f590 msgr2=0x7f6be006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:44.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.043+0000 7f6be77ee640 1 -- 172.21.15.19:0/1500042498 shutdown_connections 2024-04-25T09:59:44.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.044+0000 7f6be77ee640 1 -- 172.21.15.19:0/1500042498 wait complete. 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.045+0000 7f6be77ee640 1 Processor -- start 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.045+0000 7f6be77ee640 1 -- start start 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.045+0000 7f6be77ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be00cc8e0 0x7f6be015f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.045+0000 7f6be77ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6be00d1810 0x7f6be00c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.045+0000 7f6be77ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6be015dc90 0x7f6be00ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.045+0000 7f6be77ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be015e070 con 0x7f6be00d1810 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.045+0000 7f6be77ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00cd130 con 0x7f6be00cc8e0 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.045+0000 7f6be77ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be015e1f0 con 0x7f6be015dc90 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.045+0000 7f6be4d89640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6be00d1810 0x7f6be00c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be4d89640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6be00d1810 0x7f6be00c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51878/0 (socket says 172.21.15.19:51878) 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be4d89640 1 -- 172.21.15.19:0/921394986 learned_addr learned my addr 172.21.15.19:0/921394986 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:44.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be558a640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be00cc8e0 0x7f6be015f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be5d8b640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6be015dc90 0x7f6be00ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be4d89640 1 -- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6be015dc90 msgr2=0x7f6be00ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be4d89640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6be015dc90 0x7f6be00ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be4d89640 1 -- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be00cc8e0 msgr2=0x7f6be015f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be4d89640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be00cc8e0 0x7f6be015f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be4d89640 1 -- 172.21.15.19:0/921394986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be014c130 con 0x7f6be00d1810 2024-04-25T09:59:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be4d89640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6be00d1810 0x7f6be00c9f10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0035040 tx=0x7f6bd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6bd5ffb640 1 -- 172.21.15.19:0/921394986 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0037040 con 0x7f6be00d1810 2024-04-25T09:59:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6bd5ffb640 1 -- 172.21.15.19:0/921394986 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd0004230 con 0x7f6be00d1810 2024-04-25T09:59:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6bd5ffb640 1 -- 172.21.15.19:0/921394986 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0003610 con 0x7f6be00d1810 2024-04-25T09:59:44.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be77ee640 1 -- 172.21.15.19:0/921394986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be0160010 con 0x7f6be00d1810 2024-04-25T09:59:44.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.046+0000 7f6be77ee640 1 -- 172.21.15.19:0/921394986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be01605d0 con 0x7f6be00d1810 2024-04-25T09:59:44.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.047+0000 7f6be77ee640 1 -- 172.21.15.19:0/921394986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be0073990 con 0x7f6be00d1810 2024-04-25T09:59:44.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.048+0000 7f6bd5ffb640 1 -- 172.21.15.19:0/921394986 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6bd002ad10 con 0x7f6be00d1810 2024-04-25T09:59:44.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.048+0000 7f6bd5ffb640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc406b4d0 0x7f6bc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.048+0000 7f6bd5ffb640 1 -- 172.21.15.19:0/921394986 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6bd00aaaa0 con 0x7f6be00d1810 2024-04-25T09:59:44.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.048+0000 7f6be558a640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc406b4d0 0x7f6bc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.049+0000 7f6be558a640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc406b4d0 0x7f6bc406d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc000f60 tx=0x7f6bdc007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.052+0000 7f6bd5ffb640 1 -- 172.21.15.19:0/921394986 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6bd00ace20 con 0x7f6be00d1810 2024-04-25T09:59:44.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.233+0000 7f6be77ee640 1 -- 172.21.15.19:0/921394986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6be00b4280 con 0x7f6be00d1810 2024-04-25T09:59:44.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.234+0000 7f6bd5ffb640 1 -- 172.21.15.19:0/921394986 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+13154 (secure 0 0 0) 0x7f6bd0079bf0 con 0x7f6be00d1810 2024-04-25T09:59:44.238 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:44.238 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":102,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:43.546010+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 -- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc406b4d0 msgr2=0x7f6bc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc406b4d0 0x7f6bc406d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc000f60 tx=0x7f6bdc007b50 comp rx=0 tx=0).stop 2024-04-25T09:59:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 -- 172.21.15.19:0/921394986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6be00d1810 msgr2=0x7f6be00c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6be00d1810 0x7f6be00c9f10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0035040 tx=0x7f6bd00046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 -- 172.21.15.19:0/921394986 shutdown_connections 2024-04-25T09:59:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc406b4d0 0x7f6bc406d990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6be015dc90 0x7f6be00ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6be00d1810 0x7f6be00c9f10 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 --2- 172.21.15.19:0/921394986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6be00cc8e0 0x7f6be015f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 -- 172.21.15.19:0/921394986 >> 172.21.15.19:0/921394986 conn(0x7f6be006f590 msgr2=0x7f6be00b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 -- 172.21.15.19:0/921394986 shutdown_connections 2024-04-25T09:59:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.236+0000 7f6be77ee640 1 -- 172.21.15.19:0/921394986 wait complete. 2024-04-25T09:59:44.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:59:44.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:59:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.351+0000 7fb7d13b7640 1 Processor -- start 2024-04-25T09:59:44.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.351+0000 7fb7d13b7640 1 -- start start 2024-04-25T09:59:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.352+0000 7fb7d13b7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc0ced80 0x7fb7cc0d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.352+0000 7fb7d13b7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7cc0d23a0 con 0x7fb7cc0d1ce0 2024-04-25T09:59:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.352+0000 7fb7d13b7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7cc0d2680 con 0x7fb7cc15dc20 2024-04-25T09:59:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.352+0000 7fb7d13b7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0d2960 con 0x7fb7cc0ced80 2024-04-25T09:59:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.352+0000 7fb7cb7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb7cc0d1ce0 0x7fb7cc0b6a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50828/0 (socket says 172.21.15.19:50828) 2024-04-25T09:59:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.352+0000 7fb7cb7fe640 1 -- 172.21.15.19:0/3958610317 learned_addr learned my addr 172.21.15.19:0/3958610317 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.352+0000 7fb7ca7fc640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc0ced80 0x7fb7cc0d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.352+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649956580 0 0) 0x7fb7cc0d23a0 con 0x7fb7cc0d1ce0 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.352+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7b4003520 con 0x7fb7cc0d1ce0 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805669255 0 0) 0x7fb7cc0d2680 con 0x7fb7cc15dc20 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7cc0d23a0 con 0x7fb7cc15dc20 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3971658968 0 0) 0x7fb7b4003520 con 0x7fb7cc0d1ce0 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc0ced80 msgr2=0x7fb7cc0d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc0ced80 0x7fb7cc0d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 >> v1:172.21.15.112:6789/0 conn(0x7fb7cc15dc20 legacy=0x7fb7cc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0d2c40 con 0x7fb7cc0d1ce0 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fb7c40019f0 con 0x7fb7cc0d1ce0 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7ca7fc640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc0ced80 0x7fb7cc0d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 >> v1:172.21.15.19:6789/0 conn(0x7fb7cc0d1ce0 legacy=0x7fb7cc0b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b4003b70 0x7fb7b4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b4004800 0x7fb7b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc0ced80 0x7fb7b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0d2960 con 0x7fb7b4003b70 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b400baa0 con 0x7fb7b4004800 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b400bca0 con 0x7fb7cc0ced80 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7caffd640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b4003b70 0x7fb7b4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7ca7fc640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b4004800 0x7fb7b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7cb7fe640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc0ced80 0x7fb7b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7caffd640 1 -- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc0ced80 msgr2=0x7fb7b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7caffd640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc0ced80 0x7fb7b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7caffd640 1 -- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b4004800 msgr2=0x7fb7b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7caffd640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b4004800 0x7fb7b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7caffd640 1 -- 172.21.15.19:0/3958610317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b4003520 con 0x7fb7b4003b70 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.353+0000 7fb7cb7fe640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc0ced80 0x7fb7b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.354+0000 7fb7caffd640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b4003b70 0x7fb7b4003f70 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0018a0 tx=0x7fb7b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c761479ae9c660cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.354+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0014f0 con 0x7fb7b4003b70 2024-04-25T09:59:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.354+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc0052d0 con 0x7fb7b4003b70 2024-04-25T09:59:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.354+0000 7fb7c9ffb640 1 -- 172.21.15.19:0/3958610317 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc02ac40 con 0x7fb7b4003b70 2024-04-25T09:59:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.355+0000 7fb7d13b7640 1 -- 172.21.15.19:0/3958610317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b4003b70 msgr2=0x7fb7b4003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.355+0000 7fb7d13b7640 1 --2- 172.21.15.19:0/3958610317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b4003b70 0x7fb7b4003f70 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0018a0 tx=0x7fb7b4005830 comp rx=0 tx=0).stop 2024-04-25T09:59:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.355+0000 7fb7cb7fe640 1 -- 172.21.15.19:0/3958610317 reap_dead start 2024-04-25T09:59:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.355+0000 7fb7d13b7640 1 -- 172.21.15.19:0/3958610317 shutdown_connections 2024-04-25T09:59:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.355+0000 7fb7d13b7640 1 -- 172.21.15.19:0/3958610317 >> 172.21.15.19:0/3958610317 conn(0x7fb7cc061a20 msgr2=0x7fb7cc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.355+0000 7fb7d13b7640 1 -- 172.21.15.19:0/3958610317 shutdown_connections 2024-04-25T09:59:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.355+0000 7fb7d13b7640 1 -- 172.21.15.19:0/3958610317 wait complete. 2024-04-25T09:59:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.356+0000 7fb7d13b7640 1 Processor -- start 2024-04-25T09:59:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.356+0000 7fb7d13b7640 1 -- start start 2024-04-25T09:59:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7d13b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b4003b70 0x7fb7cc0b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7d13b7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc158240 0x7fb7cc0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7d13b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7cc15bfd0 0x7fb7cc15c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7d13b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0d35c0 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7d13b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0d3440 con 0x7fb7b4003b70 2024-04-25T09:59:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7d13b7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc0d3740 con 0x7fb7cc158240 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7cb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7cc15bfd0 0x7fb7cc15c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7cb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7cc15bfd0 0x7fb7cc15c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51890/0 (socket says 172.21.15.19:51890) 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7cb7fe640 1 -- 172.21.15.19:0/1970942888 learned_addr learned my addr 172.21.15.19:0/1970942888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7caffd640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b4003b70 0x7fb7cc0b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7ca7fc640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc158240 0x7fb7cc0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7cb7fe640 1 -- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc158240 msgr2=0x7fb7cc0c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7cb7fe640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc158240 0x7fb7cc0c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7cb7fe640 1 -- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b4003b70 msgr2=0x7fb7cc0b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7cb7fe640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b4003b70 0x7fb7cc0b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7cb7fe640 1 -- 172.21.15.19:0/1970942888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0c90b0 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.357+0000 7fb7cb7fe640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7cc15bfd0 0x7fb7cc15c450 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0b8fe0 tx=0x7fb7c4007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.358+0000 7fb7b37fe640 1 -- 172.21.15.19:0/1970942888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c400eb00 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.358+0000 7fb7b37fe640 1 -- 172.21.15.19:0/1970942888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c4018070 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.358+0000 7fb7d13b7640 1 -- 172.21.15.19:0/1970942888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7cc0cbac0 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.358+0000 7fb7b37fe640 1 -- 172.21.15.19:0/1970942888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4013420 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.358+0000 7fb7d13b7640 1 -- 172.21.15.19:0/1970942888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc0cc080 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.359+0000 7fb7d13b7640 1 -- 172.21.15.19:0/1970942888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7cc073a00 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.359+0000 7fb7b37fe640 1 -- 172.21.15.19:0/1970942888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7c40135c0 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.360+0000 7fb7b37fe640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7b806b4d0 0x7fb7b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.360+0000 7fb7b37fe640 1 -- 172.21.15.19:0/1970942888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb7c408a1b0 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.360+0000 7fb7caffd640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7b806b4d0 0x7fb7b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.361+0000 7fb7caffd640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7b806b4d0 0x7fb7b806d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc032000 tx=0x7fb7bc009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.364+0000 7fb7b37fe640 1 -- 172.21.15.19:0/1970942888 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7c4058ba0 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.548+0000 7fb7d13b7640 1 -- 172.21.15.19:0/1970942888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7cc0b5ac0 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.549+0000 7fb7b37fe640 1 -- 172.21.15.19:0/1970942888 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7c405cc40 con 0x7fb7cc15bfd0 2024-04-25T09:59:44.553 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T09:59:44.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 -- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7b806b4d0 msgr2=0x7fb7b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7b806b4d0 0x7fb7b806d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc032000 tx=0x7fb7bc009030 comp rx=0 tx=0).stop 2024-04-25T09:59:44.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 -- 172.21.15.19:0/1970942888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7cc15bfd0 msgr2=0x7fb7cc15c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7cc15bfd0 0x7fb7cc15c450 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0b8fe0 tx=0x7fb7c4007bc0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 -- 172.21.15.19:0/1970942888 shutdown_connections 2024-04-25T09:59:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7b806b4d0 0x7fb7b806d990 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7cc15bfd0 0x7fb7cc15c450 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7cc158240 0x7fb7cc0c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 --2- 172.21.15.19:0/1970942888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b4003b70 0x7fb7cc0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 -- 172.21.15.19:0/1970942888 >> 172.21.15.19:0/1970942888 conn(0x7fb7cc061a20 msgr2=0x7fb7cc160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 -- 172.21.15.19:0/1970942888 shutdown_connections 2024-04-25T09:59:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.551+0000 7fb7d13b7640 1 -- 172.21.15.19:0/1970942888 wait complete. 2024-04-25T09:59:44.566 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:59:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.663+0000 7fe77c7f5640 1 Processor -- start 2024-04-25T09:59:44.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.663+0000 7fe77c7f5640 1 -- start start 2024-04-25T09:59:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.664+0000 7fe77c7f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7740d1020 0x7fe7740d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.664+0000 7fe77c7f5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7740b97a0 con 0x7fe77415dbe0 2024-04-25T09:59:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.664+0000 7fe77c7f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7740d1c50 con 0x7fe7740b2cb0 2024-04-25T09:59:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.664+0000 7fe77c7f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77415dfc0 con 0x7fe7740d1020 2024-04-25T09:59:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.664+0000 7fe77ad92640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7740d1020 0x7fe7740d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.664+0000 7fe77a591640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe77415dbe0 0x7fe7740d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50840/0 (socket says 172.21.15.19:50840) 2024-04-25T09:59:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.664+0000 7fe77a591640 1 -- 172.21.15.19:0/3755960561 learned_addr learned my addr 172.21.15.19:0/3755960561 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.664+0000 7fe77ad92640 1 -- 172.21.15.19:0/3755960561 >> v1:172.21.15.112:6789/0 conn(0x7fe7740b2cb0 legacy=0x7fe7740d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.664+0000 7fe77ad92640 1 -- 172.21.15.19:0/3755960561 >> v1:172.21.15.19:6789/0 conn(0x7fe77415dbe0 legacy=0x7fe7740d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.665+0000 7fe77ad92640 1 -- 172.21.15.19:0/3755960561 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740d1f30 con 0x7fe7740d1020 2024-04-25T09:59:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.665+0000 7fe77958f640 1 -- 172.21.15.19:0/3755960561 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092390680 0 0) 0x7fe7740b97a0 con 0x7fe77415dbe0 2024-04-25T09:59:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.665+0000 7fe77ad92640 1 --2- 172.21.15.19:0/3755960561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7740d1020 0x7fe7740d2e40 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009d90 tx=0x7fe76c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f142bd4064267fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.665+0000 7fe77958f640 1 -- 172.21.15.19:0/3755960561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c007740 con 0x7fe7740d1020 2024-04-25T09:59:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.665+0000 7fe77958f640 1 -- 172.21.15.19:0/3755960561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76c0078e0 con 0x7fe7740d1020 2024-04-25T09:59:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.665+0000 7fe77958f640 1 -- 172.21.15.19:0/3755960561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c007cb0 con 0x7fe7740d1020 2024-04-25T09:59:44.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.666+0000 7fe77c7f5640 1 -- 172.21.15.19:0/3755960561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7740d1020 msgr2=0x7fe7740d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.666+0000 7fe77c7f5640 1 --2- 172.21.15.19:0/3755960561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7740d1020 0x7fe7740d2e40 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009d90 tx=0x7fe76c02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.666+0000 7fe77c7f5640 1 -- 172.21.15.19:0/3755960561 shutdown_connections 2024-04-25T09:59:44.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.666+0000 7fe77c7f5640 1 --2- 172.21.15.19:0/3755960561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7740d1020 0x7fe7740d2e40 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.666+0000 7fe77c7f5640 1 -- 172.21.15.19:0/3755960561 >> 172.21.15.19:0/3755960561 conn(0x7fe774061a20 msgr2=0x7fe774061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:44.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.666+0000 7fe77c7f5640 1 -- 172.21.15.19:0/3755960561 shutdown_connections 2024-04-25T09:59:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.666+0000 7fe77c7f5640 1 -- 172.21.15.19:0/3755960561 wait complete. 2024-04-25T09:59:44.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.667+0000 7fe77c7f5640 1 Processor -- start 2024-04-25T09:59:44.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.667+0000 7fe77c7f5640 1 -- start start 2024-04-25T09:59:44.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77c7f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7740b2cb0 0x7fe7740cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77c7f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7740d1020 0x7fe7740caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77c7f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77415dbe0 0x7fe7740c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77c7f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740b92a0 con 0x7fe7740b2cb0 2024-04-25T09:59:44.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77c7f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740d1a00 con 0x7fe77415dbe0 2024-04-25T09:59:44.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77c7f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740b9420 con 0x7fe7740d1020 2024-04-25T09:59:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77a591640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7740b2cb0 0x7fe7740cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77a591640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7740b2cb0 0x7fe7740cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51906/0 (socket says 172.21.15.19:51906) 2024-04-25T09:59:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77a591640 1 -- 172.21.15.19:0/1465921838 learned_addr learned my addr 172.21.15.19:0/1465921838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77ad92640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77415dbe0 0x7fe7740c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe779d90640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7740d1020 0x7fe7740caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77a591640 1 -- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7740d1020 msgr2=0x7fe7740caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77a591640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7740d1020 0x7fe7740caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77a591640 1 -- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77415dbe0 msgr2=0x7fe7740c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77a591640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77415dbe0 0x7fe7740c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77a591640 1 -- 172.21.15.19:0/1465921838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740cb1c0 con 0x7fe7740b2cb0 2024-04-25T09:59:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.668+0000 7fe77ad92640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77415dbe0 0x7fe7740c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.669+0000 7fe77a591640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7740b2cb0 0x7fe7740cccd0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe77000c3f0 tx=0x7fe77000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.669+0000 7fe76affd640 1 -- 172.21.15.19:0/1465921838 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700074e0 con 0x7fe7740b2cb0 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.669+0000 7fe76affd640 1 -- 172.21.15.19:0/1465921838 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7700047e0 con 0x7fe7740b2cb0 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.669+0000 7fe77c7f5640 1 -- 172.21.15.19:0/1465921838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7740cb4a0 con 0x7fe7740b2cb0 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.669+0000 7fe76affd640 1 -- 172.21.15.19:0/1465921838 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770004c10 con 0x7fe7740b2cb0 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.669+0000 7fe77c7f5640 1 -- 172.21.15.19:0/1465921838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7740cba60 con 0x7fe7740b2cb0 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.670+0000 7fe77c7f5640 1 -- 172.21.15.19:0/1465921838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe774073990 con 0x7fe7740b2cb0 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.670+0000 7fe76affd640 1 -- 172.21.15.19:0/1465921838 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe770011070 con 0x7fe7740b2cb0 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.671+0000 7fe76affd640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe75806b500 0x7fe75806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.671+0000 7fe76affd640 1 -- 172.21.15.19:0/1465921838 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe770089c50 con 0x7fe7740b2cb0 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.671+0000 7fe779d90640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe75806b500 0x7fe75806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.671+0000 7fe779d90640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe75806b500 0x7fe75806d9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe774160250 tx=0x7fe764009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.675+0000 7fe76affd640 1 -- 172.21.15.19:0/1465921838 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe77008f020 con 0x7fe7740b2cb0 2024-04-25T09:59:44.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.859+0000 7fe77c7f5640 1 -- 172.21.15.19:0/1465921838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe77415baf0 con 0x7fe7740b2cb0 2024-04-25T09:59:44.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.860+0000 7fe76affd640 1 -- 172.21.15.19:0/1465921838 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v102) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe77001a2b0 con 0x7fe7740b2cb0 2024-04-25T09:59:44.864 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:44.864 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T09:59:44.864 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T09:59:44.864 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T09:59:44.864 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T09:59:44.864 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T09:59:44.864 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T09:59:44.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 -- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe75806b500 msgr2=0x7fe75806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe75806b500 0x7fe75806d9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe774160250 tx=0x7fe764009040 comp rx=0 tx=0).stop 2024-04-25T09:59:44.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 -- 172.21.15.19:0/1465921838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7740b2cb0 msgr2=0x7fe7740cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7740b2cb0 0x7fe7740cccd0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe77000c3f0 tx=0x7fe77000c8b0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 -- 172.21.15.19:0/1465921838 shutdown_connections 2024-04-25T09:59:44.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe75806b500 0x7fe75806d9c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe77415dbe0 0x7fe7740c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7740d1020 0x7fe7740caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 --2- 172.21.15.19:0/1465921838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7740b2cb0 0x7fe7740cccd0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 -- 172.21.15.19:0/1465921838 >> 172.21.15.19:0/1465921838 conn(0x7fe774061a20 msgr2=0x7fe7740b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:44.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 -- 172.21.15.19:0/1465921838 shutdown_connections 2024-04-25T09:59:44.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.862+0000 7fe77c7f5640 1 -- 172.21.15.19:0/1465921838 wait complete. 2024-04-25T09:59:44.876 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:59:44.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:59:44.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.978+0000 7f9174f5c640 1 Processor -- start 2024-04-25T09:59:44.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.978+0000 7f9174f5c640 1 -- start start 2024-04-25T09:59:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.978+0000 7f9174f5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f917015dac0 0x7f91700d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.979+0000 7f9174f5c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91700d2420 con 0x7f91700d0ab0 2024-04-25T09:59:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.979+0000 7f9174f5c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91700d2700 con 0x7f91700d1f30 2024-04-25T09:59:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.979+0000 7f9174f5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91700d3030 con 0x7f917015dac0 2024-04-25T09:59:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.979+0000 7f916ed9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f91700d0ab0 0x7f91700b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50850/0 (socket says 172.21.15.19:50850) 2024-04-25T09:59:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.979+0000 7f916ed9d640 1 -- 172.21.15.19:0/1810715717 learned_addr learned my addr 172.21.15.19:0/1810715717 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.979+0000 7f916e59c640 1 --2- 172.21.15.19:0/1810715717 >> v2:172.21.15.112:3300/0 conn(0x7f917015dac0 0x7f91700d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.979+0000 7f916dd9b640 1 -- 172.21.15.19:0/1810715717 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266804713 0 0) 0x7f91700d2420 con 0x7f91700d0ab0 2024-04-25T09:59:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.979+0000 7f916dd9b640 1 -- 172.21.15.19:0/1810715717 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915c003520 con 0x7f91700d0ab0 2024-04-25T09:59:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.979+0000 7f916e59c640 1 -- 172.21.15.19:0/1810715717 >> v1:172.21.15.112:6789/0 conn(0x7f91700d1f30 legacy=0x7f91700d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.979+0000 7f916e59c640 1 -- 172.21.15.19:0/1810715717 >> v1:172.21.15.19:6789/0 conn(0x7f91700d0ab0 legacy=0x7f91700b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.979+0000 7f916e59c640 1 -- 172.21.15.19:0/1810715717 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91700d29e0 con 0x7f917015dac0 2024-04-25T09:59:44.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.980+0000 7f916e59c640 1 --2- 172.21.15.19:0/1810715717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917015dac0 0x7f91700d19f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f91600097a0 tx=0x7f916002a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3f2d4bde714a2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.980+0000 7f916dd9b640 1 -- 172.21.15.19:0/1810715717 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160037070 con 0x7f917015dac0 2024-04-25T09:59:44.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.980+0000 7f916dd9b640 1 -- 172.21.15.19:0/1810715717 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9160032640 con 0x7f917015dac0 2024-04-25T09:59:44.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.980+0000 7f916dd9b640 1 -- 172.21.15.19:0/1810715717 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91600329f0 con 0x7f917015dac0 2024-04-25T09:59:44.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.981+0000 7f9174f5c640 1 -- 172.21.15.19:0/1810715717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917015dac0 msgr2=0x7f91700d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.981+0000 7f9174f5c640 1 --2- 172.21.15.19:0/1810715717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917015dac0 0x7f91700d19f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f91600097a0 tx=0x7f916002a2c0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.981+0000 7f9174f5c640 1 -- 172.21.15.19:0/1810715717 shutdown_connections 2024-04-25T09:59:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.981+0000 7f9174f5c640 1 --2- 172.21.15.19:0/1810715717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917015dac0 0x7f91700d19f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.981+0000 7f9174f5c640 1 -- 172.21.15.19:0/1810715717 >> 172.21.15.19:0/1810715717 conn(0x7f9170061c30 msgr2=0x7f917006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.981+0000 7f9174f5c640 1 -- 172.21.15.19:0/1810715717 shutdown_connections 2024-04-25T09:59:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.981+0000 7f9174f5c640 1 -- 172.21.15.19:0/1810715717 wait complete. 2024-04-25T09:59:44.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.982+0000 7f9174f5c640 1 Processor -- start 2024-04-25T09:59:44.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.982+0000 7f9174f5c640 1 -- start start 2024-04-25T09:59:44.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f9174f5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91700d0ab0 0x7f91700cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f9174f5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91700d1f30 0x7f9170156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f9174f5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917015dac0 0x7f91701f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f9174f5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91700b97d0 con 0x7f917015dac0 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f9174f5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917015dea0 con 0x7f91700d1f30 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f9174f5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91700b9950 con 0x7f91700d0ab0 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f916ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917015dac0 0x7f91701f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f916ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917015dac0 0x7f91701f37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51922/0 (socket says 172.21.15.19:51922) 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f916ed9d640 1 -- 172.21.15.19:0/507203352 learned_addr learned my addr 172.21.15.19:0/507203352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f9167fff640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91700d1f30 0x7f9170156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f916e59c640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91700d0ab0 0x7f91700cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f916ed9d640 1 -- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91700d0ab0 msgr2=0x7f91700cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f916ed9d640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91700d0ab0 0x7f91700cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f916ed9d640 1 -- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91700d1f30 msgr2=0x7f9170156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f916ed9d640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91700d1f30 0x7f9170156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:44.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f916ed9d640 1 -- 172.21.15.19:0/507203352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9170153360 con 0x7f917015dac0 2024-04-25T09:59:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f916e59c640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91700d0ab0 0x7f91700cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.983+0000 7f916ed9d640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917015dac0 0x7f91701f37f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f916800cb40 tx=0x7f916800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.984+0000 7f9166ffd640 1 -- 172.21.15.19:0/507203352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168017070 con 0x7f917015dac0 2024-04-25T09:59:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.984+0000 7f9166ffd640 1 -- 172.21.15.19:0/507203352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916800aca0 con 0x7f917015dac0 2024-04-25T09:59:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.984+0000 7f9174f5c640 1 -- 172.21.15.19:0/507203352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91701f3e80 con 0x7f917015dac0 2024-04-25T09:59:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.984+0000 7f9166ffd640 1 -- 172.21.15.19:0/507203352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168012620 con 0x7f917015dac0 2024-04-25T09:59:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.984+0000 7f9174f5c640 1 -- 172.21.15.19:0/507203352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91701f4440 con 0x7f917015dac0 2024-04-25T09:59:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.985+0000 7f9174f5c640 1 -- 172.21.15.19:0/507203352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9170073a00 con 0x7f917015dac0 2024-04-25T09:59:44.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.985+0000 7f9166ffd640 1 -- 172.21.15.19:0/507203352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f916801a070 con 0x7f917015dac0 2024-04-25T09:59:44.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.985+0000 7f9166ffd640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f915006b4d0 0x7f915006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:44.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.986+0000 7f9166ffd640 1 -- 172.21.15.19:0/507203352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9168089f20 con 0x7f917015dac0 2024-04-25T09:59:44.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.986+0000 7f916e59c640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f915006b4d0 0x7f915006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:44.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.986+0000 7f916e59c640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f915006b4d0 0x7f915006d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9160034000 tx=0x7f916002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:44.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:44.990+0000 7f9166ffd640 1 -- 172.21.15.19:0/507203352 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91680587f0 con 0x7f917015dac0 2024-04-25T09:59:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.169+0000 7f9174f5c640 1 -- 172.21.15.19:0/507203352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f91700b5b20 con 0x7f917015dac0 2024-04-25T09:59:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.534+0000 7f9166ffd640 1 -- 172.21.15.19:0/507203352 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v103) v1 ==== 138+0+0 (secure 0 0 0) 0x7f916805c890 con 0x7f917015dac0 2024-04-25T09:59:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T09:59:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 -- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f915006b4d0 msgr2=0x7f915006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f915006b4d0 0x7f915006d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9160034000 tx=0x7f916002e000 comp rx=0 tx=0).stop 2024-04-25T09:59:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 -- 172.21.15.19:0/507203352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917015dac0 msgr2=0x7f91701f37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917015dac0 0x7f91701f37f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f916800cb40 tx=0x7f916800a060 comp rx=0 tx=0).stop 2024-04-25T09:59:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 -- 172.21.15.19:0/507203352 shutdown_connections 2024-04-25T09:59:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f915006b4d0 0x7f915006d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917015dac0 0x7f91701f37f0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91700d1f30 0x7f9170156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 --2- 172.21.15.19:0/507203352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91700d0ab0 0x7f91700cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 -- 172.21.15.19:0/507203352 >> 172.21.15.19:0/507203352 conn(0x7f9170061c30 msgr2=0x7f91700b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 -- 172.21.15.19:0/507203352 shutdown_connections 2024-04-25T09:59:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.538+0000 7f9174f5c640 1 -- 172.21.15.19:0/507203352 wait complete. 2024-04-25T09:59:45.557 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:59:45.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.662+0000 7ff67881b640 1 Processor -- start 2024-04-25T09:59:45.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.662+0000 7ff67881b640 1 -- start start 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.662+0000 7ff67881b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6740b5620 0x7ff6740b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.662+0000 7ff67881b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff67406c2d0 0x7ff67406c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.662+0000 7ff67881b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67406cc80 con 0x7ff67415b7d0 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.662+0000 7ff67881b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67406cf50 con 0x7ff6740b5620 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.662+0000 7ff67881b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6740ce530 con 0x7ff67406c2d0 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.663+0000 7ff66bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6740b5620 0x7ff6740b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.663+0000 7ff66bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6740b5620 0x7ff6740b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51930/0 (socket says 172.21.15.19:51930) 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.663+0000 7ff66bfff640 1 -- 172.21.15.19:0/1760769793 learned_addr learned my addr 172.21.15.19:0/1760769793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.663+0000 7ff672d9d640 1 --2- 172.21.15.19:0/1760769793 >> v2:172.21.15.112:3300/0 conn(0x7ff67406c2d0 0x7ff67406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.663+0000 7ff66bfff640 1 -- 172.21.15.19:0/1760769793 >> v2:172.21.15.112:3300/0 conn(0x7ff67406c2d0 msgr2=0x7ff67406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.663+0000 7ff66bfff640 1 --2- 172.21.15.19:0/1760769793 >> v2:172.21.15.112:3300/0 conn(0x7ff67406c2d0 0x7ff67406c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.663+0000 7ff66bfff640 1 -- 172.21.15.19:0/1760769793 >> v1:172.21.15.112:6789/0 conn(0x7ff67415b7d0 legacy=0x7ff6740cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:45.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.663+0000 7ff66bfff640 1 -- 172.21.15.19:0/1760769793 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740ce810 con 0x7ff6740b5620 2024-04-25T09:59:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.663+0000 7ff672d9d640 1 --2- 172.21.15.19:0/1760769793 >> v2:172.21.15.112:3300/0 conn(0x7ff67406c2d0 0x7ff67406c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff66bfff640 1 --2- 172.21.15.19:0/1760769793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6740b5620 0x7ff6740b5ab0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7ff674161890 tx=0x7ff65c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d8c6e1fc7281d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff671d9b640 1 -- 172.21.15.19:0/1760769793 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c037070 con 0x7ff6740b5620 2024-04-25T09:59:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff671d9b640 1 -- 172.21.15.19:0/1760769793 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65c032440 con 0x7ff6740b5620 2024-04-25T09:59:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff671d9b640 1 -- 172.21.15.19:0/1760769793 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c032810 con 0x7ff6740b5620 2024-04-25T09:59:45.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff67881b640 1 -- 172.21.15.19:0/1760769793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6740b5620 msgr2=0x7ff6740b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:45.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff67881b640 1 --2- 172.21.15.19:0/1760769793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6740b5620 0x7ff6740b5ab0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7ff674161890 tx=0x7ff65c02a230 comp rx=0 tx=0).stop 2024-04-25T09:59:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff67881b640 1 -- 172.21.15.19:0/1760769793 shutdown_connections 2024-04-25T09:59:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff67881b640 1 --2- 172.21.15.19:0/1760769793 >> v2:172.21.15.112:3300/0 conn(0x7ff67406c2d0 0x7ff67406c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff67881b640 1 --2- 172.21.15.19:0/1760769793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6740b5620 0x7ff6740b5ab0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff67881b640 1 -- 172.21.15.19:0/1760769793 >> 172.21.15.19:0/1760769793 conn(0x7ff6740d0800 msgr2=0x7ff6740d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff67881b640 1 -- 172.21.15.19:0/1760769793 shutdown_connections 2024-04-25T09:59:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.664+0000 7ff67881b640 1 -- 172.21.15.19:0/1760769793 wait complete. 2024-04-25T09:59:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.666+0000 7ff67881b640 1 Processor -- start 2024-04-25T09:59:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.666+0000 7ff67881b640 1 -- start start 2024-04-25T09:59:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.666+0000 7ff67881b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff67406c2d0 0x7ff6740b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.666+0000 7ff67881b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6740b5620 0x7ff6740cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.666+0000 7ff67881b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff67415b7d0 0x7ff6740cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.666+0000 7ff67881b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67415bbb0 con 0x7ff67415b7d0 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.666+0000 7ff67881b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67406d230 con 0x7ff6740b5620 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.666+0000 7ff67881b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67415bd30 con 0x7ff67406c2d0 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff672d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff67415b7d0 0x7ff6740cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff672d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff67415b7d0 0x7ff6740cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51936/0 (socket says 172.21.15.19:51936) 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff672d9d640 1 -- 172.21.15.19:0/2164093749 learned_addr learned my addr 172.21.15.19:0/2164093749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff672d9d640 1 -- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff67406c2d0 msgr2=0x7ff6740b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff672d9d640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff67406c2d0 0x7ff6740b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff672d9d640 1 -- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6740b5620 msgr2=0x7ff6740cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff66bfff640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6740b5620 0x7ff6740cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff672d9d640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6740b5620 0x7ff6740cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff672d9d640 1 -- 172.21.15.19:0/2164093749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740b7750 con 0x7ff67415b7d0 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff672d9d640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff67415b7d0 0x7ff6740cb1b0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00ce70 tx=0x7ff66c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:45.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff66affd640 1 -- 172.21.15.19:0/2164093749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c01a070 con 0x7ff67415b7d0 2024-04-25T09:59:45.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff66affd640 1 -- 172.21.15.19:0/2164093749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c00d840 con 0x7ff67415b7d0 2024-04-25T09:59:45.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff67881b640 1 -- 172.21.15.19:0/2164093749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6740b7a30 con 0x7ff67415b7d0 2024-04-25T09:59:45.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff66affd640 1 -- 172.21.15.19:0/2164093749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c00dc30 con 0x7ff67415b7d0 2024-04-25T09:59:45.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.667+0000 7ff67881b640 1 -- 172.21.15.19:0/2164093749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6740b7ff0 con 0x7ff67415b7d0 2024-04-25T09:59:45.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.668+0000 7ff67881b640 1 -- 172.21.15.19:0/2164093749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff674073a00 con 0x7ff67415b7d0 2024-04-25T09:59:45.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.669+0000 7ff66affd640 1 -- 172.21.15.19:0/2164093749 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff66c01d070 con 0x7ff67415b7d0 2024-04-25T09:59:45.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.669+0000 7ff66affd640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff65406b4d0 0x7ff65406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:45.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.669+0000 7ff66affd640 1 -- 172.21.15.19:0/2164093749 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5619+0+0 (secure 0 0 0) 0x7ff66c08a950 con 0x7ff67415b7d0 2024-04-25T09:59:45.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.669+0000 7ff67259c640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff65406b4d0 0x7ff65406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:45.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.670+0000 7ff67259c640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff65406b4d0 0x7ff65406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff664006fd0 tx=0x7ff664006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:45.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.674+0000 7ff66affd640 1 -- 172.21.15.19:0/2164093749 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff66c0598b0 con 0x7ff67415b7d0 2024-04-25T09:59:45.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:45.853+0000 7ff67881b640 1 -- 172.21.15.19:0/2164093749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff6740d0d60 con 0x7ff67415b7d0 2024-04-25T09:59:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.537+0000 7ff66affd640 1 -- 172.21.15.19:0/2164093749 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v104) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff66c0591b0 con 0x7ff67415b7d0 2024-04-25T09:59:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T09:59:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 -- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff65406b4d0 msgr2=0x7ff65406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff65406b4d0 0x7ff65406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff664006fd0 tx=0x7ff664006bc0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 -- 172.21.15.19:0/2164093749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff67415b7d0 msgr2=0x7ff6740cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff67415b7d0 0x7ff6740cb1b0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00ce70 tx=0x7ff66c00a710 comp rx=0 tx=0).stop 2024-04-25T09:59:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 -- 172.21.15.19:0/2164093749 shutdown_connections 2024-04-25T09:59:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff65406b4d0 0x7ff65406d990 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff67415b7d0 0x7ff6740cb1b0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6740b5620 0x7ff6740cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 --2- 172.21.15.19:0/2164093749 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff67406c2d0 0x7ff6740b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 -- 172.21.15.19:0/2164093749 >> 172.21.15.19:0/2164093749 conn(0x7ff6740d0800 msgr2=0x7ff67415f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 -- 172.21.15.19:0/2164093749 shutdown_connections 2024-04-25T09:59:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.540+0000 7ff67881b640 1 -- 172.21.15.19:0/2164093749 wait complete. 2024-04-25T09:59:46.571 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:59:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.676+0000 7fad2d838640 1 Processor -- start 2024-04-25T09:59:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.676+0000 7fad2d838640 1 -- start start 2024-04-25T09:59:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad2d838640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fad281610f0 0x7fad2806c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad2d838640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad281616f0 0x7fad28161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad2d838640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fad280ce0a0 0x7fad280ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:46.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad2d838640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad281614d0 con 0x7fad281610f0 2024-04-25T09:59:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad2d838640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280c6960 con 0x7fad280ce0a0 2024-04-25T09:59:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad2d838640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad28162260 con 0x7fad281616f0 2024-04-25T09:59:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad26ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fad281610f0 0x7fad2806c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad26ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fad281610f0 0x7fad2806c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51948/0 (socket says 172.21.15.19:51948) 2024-04-25T09:59:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad26ffd640 1 -- 172.21.15.19:0/4201159329 learned_addr learned my addr 172.21.15.19:0/4201159329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad1ffff640 1 --2- 172.21.15.19:0/4201159329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad281616f0 0x7fad28161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad277fe640 1 --2- 172.21.15.19:0/4201159329 >> v2:172.21.15.112:3300/0 conn(0x7fad280ce0a0 0x7fad280ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:46.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad26ffd640 1 -- 172.21.15.19:0/4201159329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad281616f0 msgr2=0x7fad28161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:46.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad26ffd640 1 --2- 172.21.15.19:0/4201159329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad281616f0 0x7fad28161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad26ffd640 1 -- 172.21.15.19:0/4201159329 >> v2:172.21.15.112:3300/0 conn(0x7fad280ce0a0 msgr2=0x7fad280ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:46.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad26ffd640 1 --2- 172.21.15.19:0/4201159329 >> v2:172.21.15.112:3300/0 conn(0x7fad280ce0a0 0x7fad280ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad26ffd640 1 -- 172.21.15.19:0/4201159329 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad28162540 con 0x7fad281610f0 2024-04-25T09:59:46.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad1ffff640 1 --2- 172.21.15.19:0/4201159329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad281616f0 0x7fad28161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:46.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.677+0000 7fad277fe640 1 --2- 172.21.15.19:0/4201159329 >> v2:172.21.15.112:3300/0 conn(0x7fad280ce0a0 0x7fad280ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:46.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.678+0000 7fad26ffd640 1 --2- 172.21.15.19:0/4201159329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad281610f0 0x7fad2806c1c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fad280cee80 tx=0x7fad1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32713d5087523a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:46.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.678+0000 7fad267fc640 1 -- 172.21.15.19:0/4201159329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18037070 con 0x7fad281610f0 2024-04-25T09:59:46.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.678+0000 7fad267fc640 1 -- 172.21.15.19:0/4201159329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18032440 con 0x7fad281610f0 2024-04-25T09:59:46.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.678+0000 7fad267fc640 1 -- 172.21.15.19:0/4201159329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18032810 con 0x7fad281610f0 2024-04-25T09:59:46.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.679+0000 7fad2d838640 1 -- 172.21.15.19:0/4201159329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad281610f0 msgr2=0x7fad2806c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:46.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.679+0000 7fad2d838640 1 --2- 172.21.15.19:0/4201159329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad281610f0 0x7fad2806c1c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fad280cee80 tx=0x7fad1802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.679+0000 7fad2d838640 1 -- 172.21.15.19:0/4201159329 shutdown_connections 2024-04-25T09:59:46.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.679+0000 7fad2d838640 1 --2- 172.21.15.19:0/4201159329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad281616f0 0x7fad28161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.679+0000 7fad2d838640 1 --2- 172.21.15.19:0/4201159329 >> v2:172.21.15.112:3300/0 conn(0x7fad280ce0a0 0x7fad280ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.679+0000 7fad2d838640 1 --2- 172.21.15.19:0/4201159329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad281610f0 0x7fad2806c1c0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.679+0000 7fad2d838640 1 -- 172.21.15.19:0/4201159329 >> 172.21.15.19:0/4201159329 conn(0x7fad2806f500 msgr2=0x7fad2806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:46.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.679+0000 7fad2d838640 1 -- 172.21.15.19:0/4201159329 shutdown_connections 2024-04-25T09:59:46.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.679+0000 7fad2d838640 1 -- 172.21.15.19:0/4201159329 wait complete. 2024-04-25T09:59:46.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.680+0000 7fad2d838640 1 Processor -- start 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.680+0000 7fad2d838640 1 -- start start 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad2d838640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad280ce0a0 0x7fad280aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad2d838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad281610f0 0x7fad280af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad2d838640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad281616f0 0x7fad281f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad2d838640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad280cf690 con 0x7fad280ce0a0 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad2d838640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad280cfce0 con 0x7fad281610f0 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad2d838640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2806c700 con 0x7fad281616f0 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad26ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad280ce0a0 0x7fad280aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad26ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad280ce0a0 0x7fad280aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51950/0 (socket says 172.21.15.19:51950) 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad26ffd640 1 -- 172.21.15.19:0/2394226589 learned_addr learned my addr 172.21.15.19:0/2394226589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad26ffd640 1 -- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad281616f0 msgr2=0x7fad281f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad26ffd640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad281616f0 0x7fad281f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad26ffd640 1 -- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad281610f0 msgr2=0x7fad280af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad1ffff640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad281610f0 0x7fad280af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad26ffd640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad281610f0 0x7fad280af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad26ffd640 1 -- 172.21.15.19:0/2394226589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280b5450 con 0x7fad280ce0a0 2024-04-25T09:59:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad1ffff640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad281610f0 0x7fad280af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad26ffd640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad280ce0a0 0x7fad280aec10 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fad1802a220 tx=0x7fad180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad24ff9640 1 -- 172.21.15.19:0/2394226589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18037040 con 0x7fad280ce0a0 2024-04-25T09:59:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad24ff9640 1 -- 172.21.15.19:0/2394226589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18042070 con 0x7fad280ce0a0 2024-04-25T09:59:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.681+0000 7fad2d838640 1 -- 172.21.15.19:0/2394226589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad280b5730 con 0x7fad280ce0a0 2024-04-25T09:59:46.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.682+0000 7fad24ff9640 1 -- 172.21.15.19:0/2394226589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18030bb0 con 0x7fad280ce0a0 2024-04-25T09:59:46.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.682+0000 7fad2d838640 1 -- 172.21.15.19:0/2394226589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad280b7930 con 0x7fad280ce0a0 2024-04-25T09:59:46.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.683+0000 7fad2d838640 1 -- 172.21.15.19:0/2394226589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2815ce60 con 0x7fad280ce0a0 2024-04-25T09:59:46.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.683+0000 7fad24ff9640 1 -- 172.21.15.19:0/2394226589 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad18047460 con 0x7fad280ce0a0 2024-04-25T09:59:46.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.683+0000 7fad24ff9640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad0806b4d0 0x7fad0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:46.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.684+0000 7fad24ff9640 1 -- 172.21.15.19:0/2394226589 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fad180ab050 con 0x7fad280ce0a0 2024-04-25T09:59:46.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.684+0000 7fad1ffff640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad0806b4d0 0x7fad0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:46.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.684+0000 7fad1ffff640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad0806b4d0 0x7fad0806d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fad10006fd0 tx=0x7fad10008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:46.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.688+0000 7fad24ff9640 1 -- 172.21.15.19:0/2394226589 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad2815ce60 con 0x7fad280ce0a0 2024-04-25T09:59:46.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:46.905+0000 7fad2d838640 1 -- 172.21.15.19:0/2394226589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fad280cf150 con 0x7fad280ce0a0 2024-04-25T09:59:47.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.561+0000 7fad24ff9640 1 -- 172.21.15.19:0/2394226589 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7fad18045360 con 0x7fad280ce0a0 2024-04-25T09:59:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T09:59:47.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.569+0000 7fad2d838640 1 -- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad0806b4d0 msgr2=0x7fad0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.569+0000 7fad2d838640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad0806b4d0 0x7fad0806d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fad10006fd0 tx=0x7fad10008040 comp rx=0 tx=0).stop 2024-04-25T09:59:47.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.569+0000 7fad2d838640 1 -- 172.21.15.19:0/2394226589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad280ce0a0 msgr2=0x7fad280aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.569+0000 7fad2d838640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad280ce0a0 0x7fad280aec10 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fad1802a220 tx=0x7fad180046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.570+0000 7fad2d838640 1 -- 172.21.15.19:0/2394226589 shutdown_connections 2024-04-25T09:59:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.570+0000 7fad2d838640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fad0806b4d0 0x7fad0806d990 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.570+0000 7fad2d838640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fad281616f0 0x7fad281f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.570+0000 7fad2d838640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad281610f0 0x7fad280af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.570+0000 7fad2d838640 1 --2- 172.21.15.19:0/2394226589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad280ce0a0 0x7fad280aec10 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.570+0000 7fad2d838640 1 -- 172.21.15.19:0/2394226589 >> 172.21.15.19:0/2394226589 conn(0x7fad2806f500 msgr2=0x7fad2806f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.571+0000 7fad2d838640 1 -- 172.21.15.19:0/2394226589 shutdown_connections 2024-04-25T09:59:47.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.571+0000 7fad2d838640 1 -- 172.21.15.19:0/2394226589 wait complete. 2024-04-25T09:59:47.600 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:47.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.701+0000 7f7d6e10a640 1 Processor -- start 2024-04-25T09:59:47.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.701+0000 7f7d6e10a640 1 -- start start 2024-04-25T09:59:47.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.702+0000 7f7d6e10a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7d680d0aa0 0x7f7d680b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.702+0000 7f7d6e10a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d6806fea0 0x7f7d680b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.702+0000 7f7d6e10a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d680b3710 con 0x7f7d6815a790 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.702+0000 7f7d6e10a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d680cf0e0 con 0x7f7d680d0aa0 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.702+0000 7f7d6e10a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680cf3c0 con 0x7f7d6806fea0 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.702+0000 7f7d66ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7d680d0aa0 0x7f7d680b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.702+0000 7f7d677fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7d6815a790 0x7f7d680d0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35080/0 (socket says 172.21.15.19:35080) 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.702+0000 7f7d677fe640 1 -- 172.21.15.19:0/2908500 learned_addr learned my addr 172.21.15.19:0/2908500 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.703+0000 7f7d67fff640 1 --2- 172.21.15.19:0/2908500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d6806fea0 0x7f7d680b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.703+0000 7f7d66ffd640 1 -- 172.21.15.19:0/2908500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d6806fea0 msgr2=0x7f7d680b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.703+0000 7f7d66ffd640 1 --2- 172.21.15.19:0/2908500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d6806fea0 0x7f7d680b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.703+0000 7f7d66ffd640 1 -- 172.21.15.19:0/2908500 >> v1:172.21.15.19:6789/0 conn(0x7f7d6815a790 legacy=0x7f7d680d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.703+0000 7f7d66ffd640 1 -- 172.21.15.19:0/2908500 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d680cf6a0 con 0x7f7d680d0aa0 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.703+0000 7f7d667fc640 1 -- 172.21.15.19:0/2908500 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908373593 0 0) 0x7f7d680b3710 con 0x7f7d6815a790 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.703+0000 7f7d66ffd640 1 --2- 172.21.15.19:0/2908500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d680d0aa0 0x7f7d680b3ea0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0097a0 tx=0x7f7d5c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5319ecb64d37ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:47.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.704+0000 7f7d667fc640 1 -- 172.21.15.19:0/2908500 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c037070 con 0x7f7d680d0aa0 2024-04-25T09:59:47.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.704+0000 7f7d667fc640 1 -- 172.21.15.19:0/2908500 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5c032440 con 0x7f7d680d0aa0 2024-04-25T09:59:47.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.704+0000 7f7d667fc640 1 -- 172.21.15.19:0/2908500 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c032810 con 0x7f7d680d0aa0 2024-04-25T09:59:47.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.704+0000 7f7d6e10a640 1 -- 172.21.15.19:0/2908500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d680d0aa0 msgr2=0x7f7d680b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.704+0000 7f7d6e10a640 1 --2- 172.21.15.19:0/2908500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d680d0aa0 0x7f7d680b3ea0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0097a0 tx=0x7f7d5c02a180 comp rx=0 tx=0).stop 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.704+0000 7f7d6e10a640 1 -- 172.21.15.19:0/2908500 shutdown_connections 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.704+0000 7f7d6e10a640 1 --2- 172.21.15.19:0/2908500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d6806fea0 0x7f7d680b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.704+0000 7f7d6e10a640 1 --2- 172.21.15.19:0/2908500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d680d0aa0 0x7f7d680b3ea0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.704+0000 7f7d6e10a640 1 -- 172.21.15.19:0/2908500 >> 172.21.15.19:0/2908500 conn(0x7f7d680b2370 msgr2=0x7f7d680b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.704+0000 7f7d6e10a640 1 -- 172.21.15.19:0/2908500 shutdown_connections 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.704+0000 7f7d6e10a640 1 -- 172.21.15.19:0/2908500 wait complete. 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.705+0000 7f7d6e10a640 1 Processor -- start 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.706+0000 7f7d6e10a640 1 -- start start 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.706+0000 7f7d6e10a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d6806fea0 0x7f7d6813f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.706+0000 7f7d6e10a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d680d0aa0 0x7f7d68157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.706+0000 7f7d6e10a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6815a790 0x7f7d68157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.706+0000 7f7d6e10a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6815ab70 con 0x7f7d6806fea0 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.706+0000 7f7d6e10a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680cf980 con 0x7f7d6815a790 2024-04-25T09:59:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.706+0000 7f7d6e10a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6815acf0 con 0x7f7d680d0aa0 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.706+0000 7f7d677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d6806fea0 0x7f7d6813f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.706+0000 7f7d677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d6806fea0 0x7f7d6813f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38446/0 (socket says 172.21.15.19:38446) 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.706+0000 7f7d677fe640 1 -- 172.21.15.19:0/695782423 learned_addr learned my addr 172.21.15.19:0/695782423 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d66ffd640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d680d0aa0 0x7f7d68157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d67fff640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6815a790 0x7f7d68157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d677fe640 1 -- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d680d0aa0 msgr2=0x7f7d68157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d677fe640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d680d0aa0 0x7f7d68157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d677fe640 1 -- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6815a790 msgr2=0x7f7d68157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d677fe640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6815a790 0x7f7d68157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d677fe640 1 -- 172.21.15.19:0/695782423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6813f5d0 con 0x7f7d6806fea0 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d677fe640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d6806fea0 0x7f7d6813f090 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c460 tx=0x7f7d5800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d4ffff640 1 -- 172.21.15.19:0/695782423 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58018070 con 0x7f7d6806fea0 2024-04-25T09:59:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d4ffff640 1 -- 172.21.15.19:0/695782423 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58007500 con 0x7f7d6806fea0 2024-04-25T09:59:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d6e10a640 1 -- 172.21.15.19:0/695782423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d68155780 con 0x7f7d6806fea0 2024-04-25T09:59:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d4ffff640 1 -- 172.21.15.19:0/695782423 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58004a20 con 0x7f7d6806fea0 2024-04-25T09:59:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.707+0000 7f7d6e10a640 1 -- 172.21.15.19:0/695782423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6813fc30 con 0x7f7d6806fea0 2024-04-25T09:59:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.708+0000 7f7d6e10a640 1 -- 172.21.15.19:0/695782423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d68073990 con 0x7f7d6806fea0 2024-04-25T09:59:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.709+0000 7f7d4ffff640 1 -- 172.21.15.19:0/695782423 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d5801b070 con 0x7f7d6806fea0 2024-04-25T09:59:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.709+0000 7f7d4ffff640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d5406b4d0 0x7f7d5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.709+0000 7f7d4ffff640 1 -- 172.21.15.19:0/695782423 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7d580895b0 con 0x7f7d6806fea0 2024-04-25T09:59:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.709+0000 7f7d66ffd640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d5406b4d0 0x7f7d5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.710+0000 7f7d66ffd640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d5406b4d0 0x7f7d5406d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c034000 tx=0x7f7d5c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.714+0000 7f7d4ffff640 1 -- 172.21.15.19:0/695782423 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d6813f5d0 con 0x7f7d6806fea0 2024-04-25T09:59:47.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.899+0000 7f7d6e10a640 1 -- 172.21.15.19:0/695782423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d6806d330 con 0x7f7d6806fea0 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.900+0000 7f7d4ffff640 1 -- 172.21.15.19:0/695782423 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f7d58057bc0 con 0x7f7d6806fea0 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":105,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:46.906269+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:45.201424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:45.885235+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:47.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 -- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d5406b4d0 msgr2=0x7f7d5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d5406b4d0 0x7f7d5406d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c034000 tx=0x7f7d5c02e000 comp rx=0 tx=0).stop 2024-04-25T09:59:47.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 -- 172.21.15.19:0/695782423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d6806fea0 msgr2=0x7f7d6813f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d6806fea0 0x7f7d6813f090 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c460 tx=0x7f7d5800c920 comp rx=0 tx=0).stop 2024-04-25T09:59:47.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 -- 172.21.15.19:0/695782423 shutdown_connections 2024-04-25T09:59:47.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7d5406b4d0 0x7f7d5406d990 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6815a790 0x7f7d68157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d680d0aa0 0x7f7d68157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 --2- 172.21.15.19:0/695782423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d6806fea0 0x7f7d6813f090 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 -- 172.21.15.19:0/695782423 >> 172.21.15.19:0/695782423 conn(0x7f7d680b2370 msgr2=0x7f7d6806b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:47.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 -- 172.21.15.19:0/695782423 shutdown_connections 2024-04-25T09:59:47.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:47.903+0000 7f7d6e10a640 1 -- 172.21.15.19:0/695782423 wait complete. 2024-04-25T09:59:47.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.016+0000 7f2bcfd63640 1 Processor -- start 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.016+0000 7f2bcfd63640 1 -- start start 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.017+0000 7f2bcfd63640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2bc80b2cb0 0x7f2bc80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.017+0000 7f2bcfd63640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc80b97a0 con 0x7f2bc815dbe0 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.017+0000 7f2bcfd63640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc80d1c50 con 0x7f2bc80d1020 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.017+0000 7f2bcfd63640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc815dfc0 con 0x7f2bc80b2cb0 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.017+0000 7f2bcd2fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2bc80b2cb0 0x7f2bc80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.017+0000 7f2bcdaff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2bc815dbe0 0x7f2bc80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35092/0 (socket says 172.21.15.19:35092) 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.017+0000 7f2bcdaff640 1 -- 172.21.15.19:0/133742345 learned_addr learned my addr 172.21.15.19:0/133742345 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.017+0000 7f2bcd2fe640 1 -- 172.21.15.19:0/133742345 >> v1:172.21.15.112:6789/0 conn(0x7f2bc80d1020 legacy=0x7f2bc80d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.017+0000 7f2bcd2fe640 1 -- 172.21.15.19:0/133742345 >> v1:172.21.15.19:6789/0 conn(0x7f2bc815dbe0 legacy=0x7f2bc80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.017+0000 7f2bcd2fe640 1 -- 172.21.15.19:0/133742345 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80d1f30 con 0x7f2bc80b2cb0 2024-04-25T09:59:48.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.018+0000 7f2bcd2fe640 1 --2- 172.21.15.19:0/133742345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc80b2cb0 0x7f2bc80d2730 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80097a0 tx=0x7f2bb802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1742d7faedd83911 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.018+0000 7f2bccafd640 1 -- 172.21.15.19:0/133742345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8037070 con 0x7f2bc80b2cb0 2024-04-25T09:59:48.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.018+0000 7f2bccafd640 1 -- 172.21.15.19:0/133742345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb8032440 con 0x7f2bc80b2cb0 2024-04-25T09:59:48.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.018+0000 7f2bccafd640 1 -- 172.21.15.19:0/133742345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8032810 con 0x7f2bc80b2cb0 2024-04-25T09:59:48.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.019+0000 7f2bcfd63640 1 -- 172.21.15.19:0/133742345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc80b2cb0 msgr2=0x7f2bc80d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.019+0000 7f2bcfd63640 1 --2- 172.21.15.19:0/133742345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc80b2cb0 0x7f2bc80d2730 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80097a0 tx=0x7f2bb802a230 comp rx=0 tx=0).stop 2024-04-25T09:59:48.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.019+0000 7f2bcfd63640 1 -- 172.21.15.19:0/133742345 shutdown_connections 2024-04-25T09:59:48.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.019+0000 7f2bcfd63640 1 --2- 172.21.15.19:0/133742345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc80b2cb0 0x7f2bc80d2730 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.019+0000 7f2bcfd63640 1 -- 172.21.15.19:0/133742345 >> 172.21.15.19:0/133742345 conn(0x7f2bc8061a20 msgr2=0x7f2bc8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.019+0000 7f2bcfd63640 1 -- 172.21.15.19:0/133742345 shutdown_connections 2024-04-25T09:59:48.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.019+0000 7f2bcfd63640 1 -- 172.21.15.19:0/133742345 wait complete. 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.020+0000 7f2bcfd63640 1 Processor -- start 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.020+0000 7f2bcfd63640 1 -- start start 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcfd63640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2bc80b2cb0 0x7f2bc8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcfd63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc80d1020 0x7f2bc8160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcfd63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc815dbe0 0x7f2bc815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcfd63640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc80b92a0 con 0x7f2bc80d1020 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcfd63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc80d1a00 con 0x7f2bc815dbe0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcfd63640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc80b9420 con 0x7f2bc80b2cb0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcd2fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc80d1020 0x7f2bc8160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcd2fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc80d1020 0x7f2bc8160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38454/0 (socket says 172.21.15.19:38454) 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcd2fe640 1 -- 172.21.15.19:0/2375911014 learned_addr learned my addr 172.21.15.19:0/2375911014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcdaff640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2bc80b2cb0 0x7f2bc8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bce300640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc815dbe0 0x7f2bc815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcd2fe640 1 -- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2bc80b2cb0 msgr2=0x7f2bc8151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcd2fe640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2bc80b2cb0 0x7f2bc8151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcd2fe640 1 -- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc815dbe0 msgr2=0x7f2bc815f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcd2fe640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc815dbe0 0x7f2bc815f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcd2fe640 1 -- 172.21.15.19:0/2375911014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc815e2d0 con 0x7f2bc80d1020 2024-04-25T09:59:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.021+0000 7f2bcd2fe640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc80d1020 0x7f2bc8160940 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2bb802a270 tx=0x7f2bb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.022+0000 7f2bbe7fc640 1 -- 172.21.15.19:0/2375911014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8037040 con 0x7f2bc80d1020 2024-04-25T09:59:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.022+0000 7f2bbe7fc640 1 -- 172.21.15.19:0/2375911014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb8042070 con 0x7f2bc80d1020 2024-04-25T09:59:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.022+0000 7f2bcfd63640 1 -- 172.21.15.19:0/2375911014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc815e5b0 con 0x7f2bc80d1020 2024-04-25T09:59:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.022+0000 7f2bbe7fc640 1 -- 172.21.15.19:0/2375911014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8030bb0 con 0x7f2bc80d1020 2024-04-25T09:59:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.022+0000 7f2bcfd63640 1 -- 172.21.15.19:0/2375911014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc815eb70 con 0x7f2bc80d1020 2024-04-25T09:59:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.022+0000 7f2bcfd63640 1 -- 172.21.15.19:0/2375911014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc8073990 con 0x7f2bc80d1020 2024-04-25T09:59:48.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.023+0000 7f2bbe7fc640 1 -- 172.21.15.19:0/2375911014 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bb8047460 con 0x7f2bc80d1020 2024-04-25T09:59:48.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.023+0000 7f2bbe7fc640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bac06b500 0x7f2bac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.024+0000 7f2bbe7fc640 1 -- 172.21.15.19:0/2375911014 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2bb80ab190 con 0x7f2bc80d1020 2024-04-25T09:59:48.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.024+0000 7f2bcdaff640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bac06b500 0x7f2bac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.024+0000 7f2bcdaff640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bac06b500 0x7f2bac06d9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4005880 tx=0x7f2bc4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.028+0000 7f2bbe7fc640 1 -- 172.21.15.19:0/2375911014 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2bb8079d50 con 0x7f2bc80d1020 2024-04-25T09:59:48.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.207+0000 7f2bcfd63640 1 -- 172.21.15.19:0/2375911014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bc80b25d0 con 0x7f2bc80d1020 2024-04-25T09:59:48.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.208+0000 7f2bbe7fc640 1 -- 172.21.15.19:0/2375911014 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f2bb8079650 con 0x7f2bc80d1020 2024-04-25T09:59:48.212 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:48.212 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":105,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:46.906269+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:45.201424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:45.885235+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:48.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 -- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bac06b500 msgr2=0x7f2bac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bac06b500 0x7f2bac06d9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4005880 tx=0x7f2bc4009040 comp rx=0 tx=0).stop 2024-04-25T09:59:48.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 -- 172.21.15.19:0/2375911014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc80d1020 msgr2=0x7f2bc8160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc80d1020 0x7f2bc8160940 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2bb802a270 tx=0x7f2bb80046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 -- 172.21.15.19:0/2375911014 shutdown_connections 2024-04-25T09:59:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bac06b500 0x7f2bac06d9c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bc815dbe0 0x7f2bc815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc80d1020 0x7f2bc8160940 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 --2- 172.21.15.19:0/2375911014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2bc80b2cb0 0x7f2bc8151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 -- 172.21.15.19:0/2375911014 >> 172.21.15.19:0/2375911014 conn(0x7f2bc8061a20 msgr2=0x7f2bc80d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 -- 172.21.15.19:0/2375911014 shutdown_connections 2024-04-25T09:59:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.210+0000 7f2bcfd63640 1 -- 172.21.15.19:0/2375911014 wait complete. 2024-04-25T09:59:48.225 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:59:48.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.325+0000 7fa38b65d640 1 Processor -- start 2024-04-25T09:59:48.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.325+0000 7fa38b65d640 1 -- start start 2024-04-25T09:59:48.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.326+0000 7fa38b65d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa38415dac0 0x7fa3840d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.326+0000 7fa38b65d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3840d2420 con 0x7fa3840d0ab0 2024-04-25T09:59:48.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.326+0000 7fa38b65d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3840d2700 con 0x7fa3840d1f30 2024-04-25T09:59:48.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.326+0000 7fa38b65d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3840d3030 con 0x7fa38415dac0 2024-04-25T09:59:48.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.326+0000 7fa3893f9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa38415dac0 0x7fa3840d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.326+0000 7fa389bfa640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa3840d0ab0 0x7fa3840b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35098/0 (socket says 172.21.15.19:35098) 2024-04-25T09:59:48.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.326+0000 7fa389bfa640 1 -- 172.21.15.19:0/1048397219 learned_addr learned my addr 172.21.15.19:0/1048397219 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:48.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.326+0000 7fa3893f9640 1 -- 172.21.15.19:0/1048397219 >> v1:172.21.15.112:6789/0 conn(0x7fa3840d1f30 legacy=0x7fa3840d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.326+0000 7fa3893f9640 1 -- 172.21.15.19:0/1048397219 >> v1:172.21.15.19:6789/0 conn(0x7fa3840d0ab0 legacy=0x7fa3840b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.326+0000 7fa3893f9640 1 -- 172.21.15.19:0/1048397219 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3840d29e0 con 0x7fa38415dac0 2024-04-25T09:59:48.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.326+0000 7fa37bfff640 1 -- 172.21.15.19:0/1048397219 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069183847 0 0) 0x7fa3840d2420 con 0x7fa3840d0ab0 2024-04-25T09:59:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.327+0000 7fa3893f9640 1 --2- 172.21.15.19:0/1048397219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa38415dac0 0x7fa3840d19f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa3800097a0 tx=0x7fa38002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b084c62777ca402c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.327+0000 7fa37bfff640 1 -- 172.21.15.19:0/1048397219 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380037070 con 0x7fa38415dac0 2024-04-25T09:59:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.327+0000 7fa37bfff640 1 -- 172.21.15.19:0/1048397219 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa380032640 con 0x7fa38415dac0 2024-04-25T09:59:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.327+0000 7fa37bfff640 1 -- 172.21.15.19:0/1048397219 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380032a10 con 0x7fa38415dac0 2024-04-25T09:59:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.328+0000 7fa38b65d640 1 -- 172.21.15.19:0/1048397219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa38415dac0 msgr2=0x7fa3840d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.328+0000 7fa38b65d640 1 --2- 172.21.15.19:0/1048397219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa38415dac0 0x7fa3840d19f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa3800097a0 tx=0x7fa38002a2c0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.328+0000 7fa38b65d640 1 -- 172.21.15.19:0/1048397219 shutdown_connections 2024-04-25T09:59:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.328+0000 7fa38b65d640 1 --2- 172.21.15.19:0/1048397219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa38415dac0 0x7fa3840d19f0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.328+0000 7fa38b65d640 1 -- 172.21.15.19:0/1048397219 >> 172.21.15.19:0/1048397219 conn(0x7fa384061c30 msgr2=0x7fa38406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.328+0000 7fa38b65d640 1 -- 172.21.15.19:0/1048397219 shutdown_connections 2024-04-25T09:59:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.328+0000 7fa38b65d640 1 -- 172.21.15.19:0/1048397219 wait complete. 2024-04-25T09:59:48.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.329+0000 7fa38b65d640 1 Processor -- start 2024-04-25T09:59:48.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.329+0000 7fa38b65d640 1 -- start start 2024-04-25T09:59:48.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa38b65d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3840d0ab0 0x7fa38415f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa38b65d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3840d1f30 0x7fa3840c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa38b65d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa38415dac0 0x7fa3840c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa38b65d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3840b97d0 con 0x7fa3840d1f30 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa38b65d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38415dea0 con 0x7fa38415dac0 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa38b65d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3840b9950 con 0x7fa3840d0ab0 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa388bf8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3840d1f30 0x7fa3840c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa388bf8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3840d1f30 0x7fa3840c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38478/0 (socket says 172.21.15.19:38478) 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa388bf8640 1 -- 172.21.15.19:0/3757148480 learned_addr learned my addr 172.21.15.19:0/3757148480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa3893f9640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3840d0ab0 0x7fa38415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa389bfa640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa38415dac0 0x7fa3840c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa388bf8640 1 -- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3840d0ab0 msgr2=0x7fa38415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa388bf8640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3840d0ab0 0x7fa38415f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa388bf8640 1 -- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa38415dac0 msgr2=0x7fa3840c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa388bf8640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa38415dac0 0x7fa3840c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa388bf8640 1 -- 172.21.15.19:0/3757148480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38415cca0 con 0x7fa3840d1f30 2024-04-25T09:59:48.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.330+0000 7fa388bf8640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3840d1f30 0x7fa3840c8230 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fa37400c450 tx=0x7fa37400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.331+0000 7fa379ffb640 1 -- 172.21.15.19:0/3757148480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3740052d0 con 0x7fa3840d1f30 2024-04-25T09:59:48.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.331+0000 7fa379ffb640 1 -- 172.21.15.19:0/3757148480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa374005490 con 0x7fa3840d1f30 2024-04-25T09:59:48.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.331+0000 7fa38b65d640 1 -- 172.21.15.19:0/3757148480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38415c9c0 con 0x7fa3840d1f30 2024-04-25T09:59:48.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.331+0000 7fa379ffb640 1 -- 172.21.15.19:0/3757148480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa374003880 con 0x7fa3840d1f30 2024-04-25T09:59:48.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.331+0000 7fa38b65d640 1 -- 172.21.15.19:0/3757148480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3841603d0 con 0x7fa3840d1f30 2024-04-25T09:59:48.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.332+0000 7fa38b65d640 1 -- 172.21.15.19:0/3757148480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38415af10 con 0x7fa3840d1f30 2024-04-25T09:59:48.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.332+0000 7fa379ffb640 1 -- 172.21.15.19:0/3757148480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa374003a20 con 0x7fa3840d1f30 2024-04-25T09:59:48.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.333+0000 7fa379ffb640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa36806b4d0 0x7fa36806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.333+0000 7fa379ffb640 1 -- 172.21.15.19:0/3757148480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa374089fa0 con 0x7fa3840d1f30 2024-04-25T09:59:48.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.333+0000 7fa3893f9640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa36806b4d0 0x7fa36806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.333+0000 7fa3893f9640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa36806b4d0 0x7fa36806d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa380034000 tx=0x7fa38002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.337+0000 7fa379ffb640 1 -- 172.21.15.19:0/3757148480 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa374058510 con 0x7fa3840d1f30 2024-04-25T09:59:48.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.552+0000 7fa38b65d640 1 -- 172.21.15.19:0/3757148480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa384073a00 con 0x7fa3840d1f30 2024-04-25T09:59:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.558+0000 7fa379ffb640 1 -- 172.21.15.19:0/3757148480 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa37405c5b0 con 0x7fa3840d1f30 2024-04-25T09:59:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.560+0000 7fa38b65d640 1 -- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa36806b4d0 msgr2=0x7fa36806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.560+0000 7fa38b65d640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa36806b4d0 0x7fa36806d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa380034000 tx=0x7fa38002e000 comp rx=0 tx=0).stop 2024-04-25T09:59:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.560+0000 7fa38b65d640 1 -- 172.21.15.19:0/3757148480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3840d1f30 msgr2=0x7fa3840c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.560+0000 7fa38b65d640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3840d1f30 0x7fa3840c8230 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fa37400c450 tx=0x7fa37400c910 comp rx=0 tx=0).stop 2024-04-25T09:59:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.560+0000 7fa38b65d640 1 -- 172.21.15.19:0/3757148480 shutdown_connections 2024-04-25T09:59:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.560+0000 7fa38b65d640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa36806b4d0 0x7fa36806d990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.560+0000 7fa38b65d640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa38415dac0 0x7fa3840c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.560+0000 7fa38b65d640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3840d1f30 0x7fa3840c8230 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.560+0000 7fa38b65d640 1 --2- 172.21.15.19:0/3757148480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3840d0ab0 0x7fa38415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.560+0000 7fa38b65d640 1 -- 172.21.15.19:0/3757148480 >> 172.21.15.19:0/3757148480 conn(0x7fa384061c30 msgr2=0x7fa3840b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.561+0000 7fa38b65d640 1 -- 172.21.15.19:0/3757148480 shutdown_connections 2024-04-25T09:59:48.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.561+0000 7fa38b65d640 1 -- 172.21.15.19:0/3757148480 wait complete. 2024-04-25T09:59:48.576 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:59:48.576 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:59:48.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.676+0000 7f32fe734640 1 Processor -- start 2024-04-25T09:59:48.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.676+0000 7f32fe734640 1 -- start start 2024-04-25T09:59:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32fe734640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f81446d0 0x7f32f80b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32fe734640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32f8144d50 0x7f32f80b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32fe734640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f32f81452a0 0x7f32f80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32fe734640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f80b2b20 con 0x7f32f8144d50 2024-04-25T09:59:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32fe734640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f806d400 con 0x7f32f81452a0 2024-04-25T09:59:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32fe734640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80b49a0 con 0x7f32f81446d0 2024-04-25T09:59:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32f77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32f8144d50 0x7f32f80b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32f77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32f8144d50 0x7f32f80b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38480/0 (socket says 172.21.15.19:38480) 2024-04-25T09:59:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32f77fe640 1 -- 172.21.15.19:0/237093312 learned_addr learned my addr 172.21.15.19:0/237093312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32f7fff640 1 --2- 172.21.15.19:0/237093312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f81446d0 0x7f32f80b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32fccd1640 1 --2- 172.21.15.19:0/237093312 >> v2:172.21.15.112:3300/0 conn(0x7f32f81452a0 0x7f32f80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32f77fe640 1 -- 172.21.15.19:0/237093312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f81446d0 msgr2=0x7f32f80b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32f77fe640 1 --2- 172.21.15.19:0/237093312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f81446d0 0x7f32f80b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32f77fe640 1 -- 172.21.15.19:0/237093312 >> v2:172.21.15.112:3300/0 conn(0x7f32f81452a0 msgr2=0x7f32f80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32f77fe640 1 --2- 172.21.15.19:0/237093312 >> v2:172.21.15.112:3300/0 conn(0x7f32f81452a0 0x7f32f80cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.677+0000 7f32f77fe640 1 -- 172.21.15.19:0/237093312 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f80cccc0 con 0x7f32f8144d50 2024-04-25T09:59:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.678+0000 7f32f77fe640 1 --2- 172.21.15.19:0/237093312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f8144d50 0x7f32f80b4460 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f32f8145880 tx=0x7f32e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a5dac0bee5cffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.678+0000 7f32f6ffd640 1 -- 172.21.15.19:0/237093312 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8037070 con 0x7f32f8144d50 2024-04-25T09:59:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.678+0000 7f32f6ffd640 1 -- 172.21.15.19:0/237093312 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e8032440 con 0x7f32f8144d50 2024-04-25T09:59:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.678+0000 7f32f6ffd640 1 -- 172.21.15.19:0/237093312 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8032810 con 0x7f32f8144d50 2024-04-25T09:59:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.679+0000 7f32fe734640 1 -- 172.21.15.19:0/237093312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f8144d50 msgr2=0x7f32f80b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.679+0000 7f32fe734640 1 --2- 172.21.15.19:0/237093312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f8144d50 0x7f32f80b4460 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f32f8145880 tx=0x7f32e802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.679+0000 7f32fe734640 1 -- 172.21.15.19:0/237093312 shutdown_connections 2024-04-25T09:59:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.679+0000 7f32fe734640 1 --2- 172.21.15.19:0/237093312 >> v2:172.21.15.112:3300/0 conn(0x7f32f81452a0 0x7f32f80cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.679+0000 7f32fe734640 1 --2- 172.21.15.19:0/237093312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f8144d50 0x7f32f80b4460 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.679+0000 7f32fe734640 1 --2- 172.21.15.19:0/237093312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f81446d0 0x7f32f80b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.679+0000 7f32fe734640 1 -- 172.21.15.19:0/237093312 >> 172.21.15.19:0/237093312 conn(0x7f32f806abe0 msgr2=0x7f32f806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.679+0000 7f32fe734640 1 -- 172.21.15.19:0/237093312 shutdown_connections 2024-04-25T09:59:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.679+0000 7f32fe734640 1 -- 172.21.15.19:0/237093312 wait complete. 2024-04-25T09:59:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.680+0000 7f32fe734640 1 Processor -- start 2024-04-25T09:59:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.680+0000 7f32fe734640 1 -- start start 2024-04-25T09:59:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32fe734640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f81446d0 0x7f32f806da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32fe734640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f8144d50 0x7f32f81f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32fe734640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f81452a0 0x7f32f81f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32fe734640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80cc700 con 0x7f32f8144d50 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32fe734640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80cc420 con 0x7f32f81452a0 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32fe734640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80cc9e0 con 0x7f32f81446d0 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f8144d50 0x7f32f81f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f8144d50 0x7f32f81f47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38496/0 (socket says 172.21.15.19:38496) 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f77fe640 1 -- 172.21.15.19:0/2972976775 learned_addr learned my addr 172.21.15.19:0/2972976775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32fccd1640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f81452a0 0x7f32f81f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f7fff640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f81446d0 0x7f32f806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f77fe640 1 -- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f81446d0 msgr2=0x7f32f806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f77fe640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f81446d0 0x7f32f806da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f77fe640 1 -- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f81452a0 msgr2=0x7f32f81f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f77fe640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f81452a0 0x7f32f81f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f77fe640 1 -- 172.21.15.19:0/2972976775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f81f53a0 con 0x7f32f8144d50 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f7fff640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f81446d0 0x7f32f806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f77fe640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f8144d50 0x7f32f81f47c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f32e802a220 tx=0x7f32e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.681+0000 7f32f4ff9640 1 -- 172.21.15.19:0/2972976775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8041070 con 0x7f32f8144d50 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.682+0000 7f32f4ff9640 1 -- 172.21.15.19:0/2972976775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e8032e50 con 0x7f32f8144d50 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.682+0000 7f32fe734640 1 -- 172.21.15.19:0/2972976775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f81f5680 con 0x7f32f8144d50 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.682+0000 7f32f4ff9640 1 -- 172.21.15.19:0/2972976775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8037040 con 0x7f32f8144d50 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.682+0000 7f32fe734640 1 -- 172.21.15.19:0/2972976775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f80aba10 con 0x7f32f8144d50 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.683+0000 7f32fe734640 1 -- 172.21.15.19:0/2972976775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f8073a00 con 0x7f32f8144d50 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.683+0000 7f32f4ff9640 1 -- 172.21.15.19:0/2972976775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32e8044070 con 0x7f32f8144d50 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.683+0000 7f32f4ff9640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32e406b4d0 0x7f32e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.683+0000 7f32f4ff9640 1 -- 172.21.15.19:0/2972976775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32e80aafa0 con 0x7f32f8144d50 2024-04-25T09:59:48.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.684+0000 7f32f7fff640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32e406b4d0 0x7f32e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.684+0000 7f32f7fff640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32e406b4d0 0x7f32e406d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009a40 tx=0x7f32ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.688+0000 7f32f4ff9640 1 -- 172.21.15.19:0/2972976775 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32e8079e40 con 0x7f32f8144d50 2024-04-25T09:59:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:48.904+0000 7f32fe734640 1 -- 172.21.15.19:0/2972976775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f32f806c250 con 0x7f32f8144d50 2024-04-25T09:59:49.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T09:59:49.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T09:59:49.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T09:59:49.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T09:59:49.538 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T09:59:49.534+0000 7fee671f2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:59:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.572+0000 7f32f4ff9640 1 -- 172.21.15.19:0/2972976775 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) v1 ==== 122+0+0 (secure 0 0 0) 0x7f32e8079740 con 0x7f32f8144d50 2024-04-25T09:59:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.574+0000 7f32fe734640 1 -- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32e406b4d0 msgr2=0x7f32e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.575+0000 7f32fe734640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32e406b4d0 0x7f32e406d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009a40 tx=0x7f32ec008040 comp rx=0 tx=0).stop 2024-04-25T09:59:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.575+0000 7f32fe734640 1 -- 172.21.15.19:0/2972976775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f8144d50 msgr2=0x7f32f81f47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.575+0000 7f32fe734640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f8144d50 0x7f32f81f47c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f32e802a220 tx=0x7f32e80046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.575+0000 7f32fe734640 1 -- 172.21.15.19:0/2972976775 shutdown_connections 2024-04-25T09:59:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.575+0000 7f32fe734640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32e406b4d0 0x7f32e406d990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.575+0000 7f32fe734640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f81452a0 0x7f32f81f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.575+0000 7f32fe734640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32f8144d50 0x7f32f81f47c0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.575+0000 7f32fe734640 1 --2- 172.21.15.19:0/2972976775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32f81446d0 0x7f32f806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.575+0000 7f32fe734640 1 -- 172.21.15.19:0/2972976775 >> 172.21.15.19:0/2972976775 conn(0x7f32f806abe0 msgr2=0x7f32f80b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.575+0000 7f32fe734640 1 -- 172.21.15.19:0/2972976775 shutdown_connections 2024-04-25T09:59:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.575+0000 7f32fe734640 1 -- 172.21.15.19:0/2972976775 wait complete. 2024-04-25T09:59:49.592 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:59:49.592 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:49.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.696+0000 7f1aaef5c640 1 Processor -- start 2024-04-25T09:59:49.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.696+0000 7f1aaef5c640 1 -- start start 2024-04-25T09:59:49.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aaef5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1aa815dbe0 0x7f1aa80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aaef5c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa80b6220 con 0x7f1aa80b92a0 2024-04-25T09:59:49.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aaef5c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa80b97a0 con 0x7f1aa80b2cb0 2024-04-25T09:59:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aaef5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa815dfc0 con 0x7f1aa815dbe0 2024-04-25T09:59:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aad4f9640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1aa80b92a0 0x7f1aa80b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35114/0 (socket says 172.21.15.19:35114) 2024-04-25T09:59:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aad4f9640 1 -- 172.21.15.19:0/3509731787 learned_addr learned my addr 172.21.15.19:0/3509731787 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285896664 0 0) 0x7f1aa80b6220 con 0x7f1aa80b92a0 2024-04-25T09:59:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aaccf8640 1 --2- 172.21.15.19:0/3509731787 >> v2:172.21.15.112:3300/0 conn(0x7f1aa815dbe0 0x7f1aa80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a98003520 con 0x7f1aa80b92a0 2024-04-25T09:59:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3364998014 0 0) 0x7f1a98003520 con 0x7f1aa80b92a0 2024-04-25T09:59:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 >> v2:172.21.15.112:3300/0 conn(0x7f1aa815dbe0 msgr2=0x7f1aa80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aa77fe640 1 --2- 172.21.15.19:0/3509731787 >> v2:172.21.15.112:3300/0 conn(0x7f1aa815dbe0 0x7f1aa80d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 >> v1:172.21.15.112:6789/0 conn(0x7f1aa80b2cb0 legacy=0x7f1aa80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.697+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa80d1c50 con 0x7f1aa80b92a0 2024-04-25T09:59:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f1aa00019f0 con 0x7f1aa80b92a0 2024-04-25T09:59:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 >> v1:172.21.15.19:6789/0 conn(0x7f1aa80b92a0 legacy=0x7f1aa80b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aa77fe640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 0x7f1a98003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aa77fe640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a980043b0 0x7f1a98004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aa77fe640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a98005120 0x7f1a9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9800c030 con 0x7f1a980039b0 2024-04-25T09:59:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9800c1e0 con 0x7f1a980043b0 2024-04-25T09:59:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9800c3e0 con 0x7f1a98005120 2024-04-25T09:59:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aaccf8640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 0x7f1a98003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aa7fff640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a980043b0 0x7f1a98004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aad4f9640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a98005120 0x7f1a9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aaccf8640 1 -- 172.21.15.19:0/3509731787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a98005120 msgr2=0x7f1a9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aaccf8640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a98005120 0x7f1a9800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aaccf8640 1 -- 172.21.15.19:0/3509731787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a980043b0 msgr2=0x7f1a98004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aaccf8640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a980043b0 0x7f1a98004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aaccf8640 1 -- 172.21.15.19:0/3509731787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a98003520 con 0x7f1a980039b0 2024-04-25T09:59:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aad4f9640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a98005120 0x7f1a9800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aa7fff640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a980043b0 0x7f1a98004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.698+0000 7f1aad4f9640 1 -- 172.21.15.19:0/3509731787 reap_dead start 2024-04-25T09:59:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.699+0000 7f1aaccf8640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 0x7f1a98003d90 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009ac0 tx=0x7f1a98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ca3e2a2c462d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.699+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c036070 con 0x7f1a980039b0 2024-04-25T09:59:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.699+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c031400 con 0x7f1a980039b0 2024-04-25T09:59:49.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.699+0000 7f1aa77fe640 1 -- 172.21.15.19:0/3509731787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c009690 con 0x7f1a980039b0 2024-04-25T09:59:49.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.699+0000 7f1aaef5c640 1 -- 172.21.15.19:0/3509731787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 msgr2=0x7f1a98003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.699+0000 7f1aaef5c640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 0x7f1a98003d90 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009ac0 tx=0x7f1a98005830 comp rx=0 tx=0).stop 2024-04-25T09:59:49.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.699+0000 7f1aaef5c640 1 -- 172.21.15.19:0/3509731787 shutdown_connections 2024-04-25T09:59:49.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.699+0000 7f1aaef5c640 1 --2- 172.21.15.19:0/3509731787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 0x7f1a98003d90 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.699+0000 7f1aaef5c640 1 -- 172.21.15.19:0/3509731787 >> 172.21.15.19:0/3509731787 conn(0x7f1aa8061a20 msgr2=0x7f1aa8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:49.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.699+0000 7f1aaef5c640 1 -- 172.21.15.19:0/3509731787 shutdown_connections 2024-04-25T09:59:49.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.699+0000 7f1aaef5c640 1 -- 172.21.15.19:0/3509731787 wait complete. 2024-04-25T09:59:49.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.701+0000 7f1aaef5c640 1 Processor -- start 2024-04-25T09:59:49.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.701+0000 7f1aaef5c640 1 -- start start 2024-04-25T09:59:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaef5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 0x7f1aa815e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaef5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa8160a50 0x7f1aa8153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaef5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa815c9b0 0x7f1aa815cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaef5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80d2210 con 0x7f1a980039b0 2024-04-25T09:59:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaef5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80d1a00 con 0x7f1aa815c9b0 2024-04-25T09:59:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaef5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80b6ab0 con 0x7f1aa8160a50 2024-04-25T09:59:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaccf8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 0x7f1aa815e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaccf8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 0x7f1aa815e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38518/0 (socket says 172.21.15.19:38518) 2024-04-25T09:59:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaccf8640 1 -- 172.21.15.19:0/2482083554 learned_addr learned my addr 172.21.15.19:0/2482083554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aa7fff640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa8160a50 0x7f1aa8153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aad4f9640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa815c9b0 0x7f1aa815cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaccf8640 1 -- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa8160a50 msgr2=0x7f1aa8153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaccf8640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa8160a50 0x7f1aa8153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaccf8640 1 -- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa815c9b0 msgr2=0x7f1aa815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaccf8640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa815c9b0 0x7f1aa815cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaccf8640 1 -- 172.21.15.19:0/2482083554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa81572c0 con 0x7f1a980039b0 2024-04-25T09:59:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aa7fff640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa8160a50 0x7f1aa8153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aad4f9640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa815c9b0 0x7f1aa815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaccf8640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 0x7f1aa815e960 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c034040 tx=0x7f1a9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aa57fa640 1 -- 172.21.15.19:0/2482083554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c03f020 con 0x7f1a980039b0 2024-04-25T09:59:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aa57fa640 1 -- 172.21.15.19:0/2482083554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c037040 con 0x7f1a980039b0 2024-04-25T09:59:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaef5c640 1 -- 172.21.15.19:0/2482083554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa81575a0 con 0x7f1a980039b0 2024-04-25T09:59:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aa57fa640 1 -- 172.21.15.19:0/2482083554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c036070 con 0x7f1a980039b0 2024-04-25T09:59:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.702+0000 7f1aaef5c640 1 -- 172.21.15.19:0/2482083554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa815e160 con 0x7f1a980039b0 2024-04-25T09:59:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.703+0000 7f1aaef5c640 1 -- 172.21.15.19:0/2482083554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa8157ac0 con 0x7f1a980039b0 2024-04-25T09:59:49.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.704+0000 7f1aa57fa640 1 -- 172.21.15.19:0/2482083554 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a9c042020 con 0x7f1a980039b0 2024-04-25T09:59:49.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.704+0000 7f1aa57fa640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8c06b4d0 0x7f1a8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.704+0000 7f1aa57fa640 1 -- 172.21.15.19:0/2482083554 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a9c0aa380 con 0x7f1a980039b0 2024-04-25T09:59:49.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.704+0000 7f1aa7fff640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8c06b4d0 0x7f1a8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.705+0000 7f1aa7fff640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8c06b4d0 0x7f1a8c06d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1a94007b40 tx=0x7f1a94009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:49.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.709+0000 7f1aa57fa640 1 -- 172.21.15.19:0/2482083554 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1aa8157ac0 con 0x7f1a980039b0 2024-04-25T09:59:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.924+0000 7f1aaef5c640 1 -- 172.21.15.19:0/2482083554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1aa8073990 con 0x7f1a980039b0 2024-04-25T09:59:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.925+0000 7f1aa57fa640 1 -- 172.21.15.19:0/2482083554 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1a9c049030 con 0x7f1a980039b0 2024-04-25T09:59:49.929 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:49.929 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:59:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T09:59:49.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 -- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8c06b4d0 msgr2=0x7f1a8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8c06b4d0 0x7f1a8c06d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1a94007b40 tx=0x7f1a94009040 comp rx=0 tx=0).stop 2024-04-25T09:59:49.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 -- 172.21.15.19:0/2482083554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 msgr2=0x7f1aa815e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 0x7f1aa815e960 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c034040 tx=0x7f1a9c004730 comp rx=0 tx=0).stop 2024-04-25T09:59:49.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 -- 172.21.15.19:0/2482083554 shutdown_connections 2024-04-25T09:59:49.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8c06b4d0 0x7f1a8c06d990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa815c9b0 0x7f1aa815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa8160a50 0x7f1aa8153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 --2- 172.21.15.19:0/2482083554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a980039b0 0x7f1aa815e960 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 -- 172.21.15.19:0/2482083554 >> 172.21.15.19:0/2482083554 conn(0x7f1aa8061a20 msgr2=0x7f1aa80b8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 -- 172.21.15.19:0/2482083554 shutdown_connections 2024-04-25T09:59:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:49.927+0000 7f1aaef5c640 1 -- 172.21.15.19:0/2482083554 wait complete. 2024-04-25T09:59:49.942 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:50.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.039+0000 7f425d15c640 1 Processor -- start 2024-04-25T09:59:50.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.039+0000 7f425d15c640 1 -- start start 2024-04-25T09:59:50.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.039+0000 7f425d15c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4258154ee0 0x7f42580cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.039+0000 7f425d15c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f42580cfcf0 0x7f42580cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.039+0000 7f425d15c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4258071480 con 0x7f42580ce3c0 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.039+0000 7f425d15c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42580ce830 con 0x7f4258154ee0 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.039+0000 7f425d15c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4258072090 con 0x7f42580cfcf0 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.040+0000 7f4256d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4258154ee0 0x7f42580cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.040+0000 7f4256d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4258154ee0 0x7f42580cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38528/0 (socket says 172.21.15.19:38528) 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.040+0000 7f4256d9d640 1 -- 172.21.15.19:0/1150961529 learned_addr learned my addr 172.21.15.19:0/1150961529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.040+0000 7f425659c640 1 --2- 172.21.15.19:0/1150961529 >> v2:172.21.15.112:3300/0 conn(0x7f42580cfcf0 0x7f42580cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.040+0000 7f4256d9d640 1 -- 172.21.15.19:0/1150961529 >> v2:172.21.15.112:3300/0 conn(0x7f42580cfcf0 msgr2=0x7f42580cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.040+0000 7f4256d9d640 1 --2- 172.21.15.19:0/1150961529 >> v2:172.21.15.112:3300/0 conn(0x7f42580cfcf0 0x7f42580cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.040+0000 7f4256d9d640 1 -- 172.21.15.19:0/1150961529 >> v1:172.21.15.112:6789/0 conn(0x7f42580ce3c0 legacy=0x7f425815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.040+0000 7f4256d9d640 1 -- 172.21.15.19:0/1150961529 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4258071760 con 0x7f4258154ee0 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f4256d9d640 1 --2- 172.21.15.19:0/1150961529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4258154ee0 0x7f42580cf7b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f42580cf560 tx=0x7f424802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af56bf4612faf7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f4255d9b640 1 -- 172.21.15.19:0/1150961529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248037070 con 0x7f4258154ee0 2024-04-25T09:59:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f4255d9b640 1 -- 172.21.15.19:0/1150961529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4248032440 con 0x7f4258154ee0 2024-04-25T09:59:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f4255d9b640 1 -- 172.21.15.19:0/1150961529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248032810 con 0x7f4258154ee0 2024-04-25T09:59:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f425d15c640 1 -- 172.21.15.19:0/1150961529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4258154ee0 msgr2=0x7f42580cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f425d15c640 1 --2- 172.21.15.19:0/1150961529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4258154ee0 0x7f42580cf7b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f42580cf560 tx=0x7f424802a270 comp rx=0 tx=0).stop 2024-04-25T09:59:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f425d15c640 1 -- 172.21.15.19:0/1150961529 shutdown_connections 2024-04-25T09:59:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f425d15c640 1 --2- 172.21.15.19:0/1150961529 >> v2:172.21.15.112:3300/0 conn(0x7f42580cfcf0 0x7f42580cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f425d15c640 1 --2- 172.21.15.19:0/1150961529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4258154ee0 0x7f42580cf7b0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f425d15c640 1 -- 172.21.15.19:0/1150961529 >> 172.21.15.19:0/1150961529 conn(0x7f425806abe0 msgr2=0x7f425806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f425d15c640 1 -- 172.21.15.19:0/1150961529 shutdown_connections 2024-04-25T09:59:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.041+0000 7f425d15c640 1 -- 172.21.15.19:0/1150961529 wait complete. 2024-04-25T09:59:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.043+0000 7f425d15c640 1 Processor -- start 2024-04-25T09:59:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.043+0000 7f425d15c640 1 -- start start 2024-04-25T09:59:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425d15c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580ce3c0 0x7f4258144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425d15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580cfcf0 0x7f425814bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425d15c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4258154ee0 0x7f4258150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425d15c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258071d20 con 0x7f42580cfcf0 2024-04-25T09:59:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425d15c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258071230 con 0x7f42580ce3c0 2024-04-25T09:59:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425d15c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42581552c0 con 0x7f4258154ee0 2024-04-25T09:59:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580cfcf0 0x7f425814bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580cfcf0 0x7f425814bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38538/0 (socket says 172.21.15.19:38538) 2024-04-25T09:59:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425659c640 1 -- 172.21.15.19:0/3522408030 learned_addr learned my addr 172.21.15.19:0/3522408030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425759e640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4258154ee0 0x7f4258150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f4256d9d640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580ce3c0 0x7f4258144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425659c640 1 -- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4258154ee0 msgr2=0x7f4258150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425659c640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4258154ee0 0x7f4258150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425659c640 1 -- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580ce3c0 msgr2=0x7f4258144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425659c640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580ce3c0 0x7f4258144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425659c640 1 -- 172.21.15.19:0/3522408030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425814d7a0 con 0x7f42580cfcf0 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425759e640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4258154ee0 0x7f4258150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f4256d9d640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580ce3c0 0x7f4258144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425659c640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580cfcf0 0x7f425814bb20 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f424c00d830 tx=0x7f424c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f423f7fe640 1 -- 172.21.15.19:0/3522408030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00b6c0 con 0x7f42580cfcf0 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f423f7fe640 1 -- 172.21.15.19:0/3522408030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c019070 con 0x7f42580cfcf0 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f423f7fe640 1 -- 172.21.15.19:0/3522408030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00ba80 con 0x7f42580cfcf0 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.044+0000 7f425d15c640 1 -- 172.21.15.19:0/3522408030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4258149260 con 0x7f42580cfcf0 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.045+0000 7f425d15c640 1 -- 172.21.15.19:0/3522408030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4258144a90 con 0x7f42580cfcf0 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.046+0000 7f425d15c640 1 -- 172.21.15.19:0/3522408030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4258072240 con 0x7f42580cfcf0 2024-04-25T09:59:50.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.046+0000 7f423f7fe640 1 -- 172.21.15.19:0/3522408030 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f424c01e460 con 0x7f42580cfcf0 2024-04-25T09:59:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.046+0000 7f423f7fe640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f424406b4d0 0x7f424406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.046+0000 7f423f7fe640 1 -- 172.21.15.19:0/3522408030 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f424c08a9b0 con 0x7f42580cfcf0 2024-04-25T09:59:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.047+0000 7f4256d9d640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f424406b4d0 0x7f424406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.047+0000 7f4256d9d640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f424406b4d0 0x7f424406d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4248034000 tx=0x7f424802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.051+0000 7f423f7fe640 1 -- 172.21.15.19:0/3522408030 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4258072240 con 0x7f42580cfcf0 2024-04-25T09:59:50.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.232+0000 7f425d15c640 1 -- 172.21.15.19:0/3522408030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42580d04b0 con 0x7f42580cfcf0 2024-04-25T09:59:50.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.233+0000 7f423f7fe640 1 -- 172.21.15.19:0/3522408030 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f42580d04b0 con 0x7f42580cfcf0 2024-04-25T09:59:50.238 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:50.238 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":106,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:48.558694+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:45.201424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:45.885235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:50.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 -- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f424406b4d0 msgr2=0x7f424406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f424406b4d0 0x7f424406d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4248034000 tx=0x7f424802e000 comp rx=0 tx=0).stop 2024-04-25T09:59:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 -- 172.21.15.19:0/3522408030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580cfcf0 msgr2=0x7f425814bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580cfcf0 0x7f425814bb20 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f424c00d830 tx=0x7f424c00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 -- 172.21.15.19:0/3522408030 shutdown_connections 2024-04-25T09:59:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f424406b4d0 0x7f424406d990 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4258154ee0 0x7f4258150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580cfcf0 0x7f425814bb20 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 --2- 172.21.15.19:0/3522408030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42580ce3c0 0x7f4258144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 -- 172.21.15.19:0/3522408030 >> 172.21.15.19:0/3522408030 conn(0x7f425806abe0 msgr2=0x7f425815cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 -- 172.21.15.19:0/3522408030 shutdown_connections 2024-04-25T09:59:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.235+0000 7f425d15c640 1 -- 172.21.15.19:0/3522408030 wait complete. 2024-04-25T09:59:50.250 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:59:50.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:59:50.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.348+0000 7ff4e917e640 1 Processor -- start 2024-04-25T09:59:50.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.348+0000 7ff4e917e640 1 -- start start 2024-04-25T09:59:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.348+0000 7ff4e917e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff4e40b9730 0x7ff4e40ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.348+0000 7ff4e917e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e40cf040 con 0x7ff4e40ceb90 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.348+0000 7ff4e917e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e4161d50 con 0x7ff4e415da70 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.348+0000 7ff4e917e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e41625e0 con 0x7ff4e40b9730 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.349+0000 7ff4e259c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff4e40b9730 0x7ff4e40ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.349+0000 7ff4e359e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff4e40ceb90 0x7ff4e40d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35118/0 (socket says 172.21.15.19:35118) 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.349+0000 7ff4e359e640 1 -- 172.21.15.19:0/3602014122 learned_addr learned my addr 172.21.15.19:0/3602014122 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.349+0000 7ff4e259c640 1 -- 172.21.15.19:0/3602014122 >> v1:172.21.15.112:6789/0 conn(0x7ff4e415da70 legacy=0x7ff4e4161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.349+0000 7ff4e259c640 1 -- 172.21.15.19:0/3602014122 >> v1:172.21.15.19:6789/0 conn(0x7ff4e40ceb90 legacy=0x7ff4e40d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.349+0000 7ff4e259c640 1 -- 172.21.15.19:0/3602014122 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e4162020 con 0x7ff4e40b9730 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.350+0000 7ff4e259c640 1 --2- 172.21.15.19:0/3602014122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e40b9730 0x7ff4e40ce650 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0097a0 tx=0x7ff4cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159e72c7d489dda4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.350+0000 7ff4e1d9b640 1 -- 172.21.15.19:0/3602014122 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc037070 con 0x7ff4e40b9730 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.350+0000 7ff4e1d9b640 1 -- 172.21.15.19:0/3602014122 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc02ae50 con 0x7ff4e40b9730 2024-04-25T09:59:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.350+0000 7ff4e1d9b640 1 -- 172.21.15.19:0/3602014122 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc032660 con 0x7ff4e40b9730 2024-04-25T09:59:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.350+0000 7ff4e917e640 1 -- 172.21.15.19:0/3602014122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e40b9730 msgr2=0x7ff4e40ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.350+0000 7ff4e917e640 1 --2- 172.21.15.19:0/3602014122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e40b9730 0x7ff4e40ce650 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0097a0 tx=0x7ff4cc02a140 comp rx=0 tx=0).stop 2024-04-25T09:59:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.350+0000 7ff4e917e640 1 -- 172.21.15.19:0/3602014122 shutdown_connections 2024-04-25T09:59:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.350+0000 7ff4e917e640 1 --2- 172.21.15.19:0/3602014122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e40b9730 0x7ff4e40ce650 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.350+0000 7ff4e917e640 1 -- 172.21.15.19:0/3602014122 >> 172.21.15.19:0/3602014122 conn(0x7ff4e4061a20 msgr2=0x7ff4e4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.350+0000 7ff4e917e640 1 -- 172.21.15.19:0/3602014122 shutdown_connections 2024-04-25T09:59:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.350+0000 7ff4e917e640 1 -- 172.21.15.19:0/3602014122 wait complete. 2024-04-25T09:59:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.352+0000 7ff4e917e640 1 Processor -- start 2024-04-25T09:59:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.352+0000 7ff4e917e640 1 -- start start 2024-04-25T09:59:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.352+0000 7ff4e917e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e40ceb90 0x7ff4e40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.352+0000 7ff4e917e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4e415da70 0x7ff4e40c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.352+0000 7ff4e917e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4e4160910 0x7ff4e40d3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.352+0000 7ff4e917e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e40d1dd0 con 0x7ff4e40ceb90 2024-04-25T09:59:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.352+0000 7ff4e917e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e40d1af0 con 0x7ff4e4160910 2024-04-25T09:59:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.352+0000 7ff4e917e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e415de50 con 0x7ff4e415da70 2024-04-25T09:59:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e40ceb90 0x7ff4e40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e40ceb90 0x7ff4e40cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38560/0 (socket says 172.21.15.19:38560) 2024-04-25T09:59:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e2d9d640 1 -- 172.21.15.19:0/4093214885 learned_addr learned my addr 172.21.15.19:0/4093214885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e2d9d640 1 -- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4e415da70 msgr2=0x7ff4e40c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e2d9d640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4e415da70 0x7ff4e40c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e2d9d640 1 -- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4e4160910 msgr2=0x7ff4e40d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e359e640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4e4160910 0x7ff4e40d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e2d9d640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4e4160910 0x7ff4e40d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e2d9d640 1 -- 172.21.15.19:0/4093214885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e40ccc20 con 0x7ff4e40ceb90 2024-04-25T09:59:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e359e640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4e4160910 0x7ff4e40d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e2d9d640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e40ceb90 0x7ff4e40cc4b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c7c0 tx=0x7ff4dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4db7fe640 1 -- 172.21.15.19:0/4093214885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc016020 con 0x7ff4e40ceb90 2024-04-25T09:59:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4db7fe640 1 -- 172.21.15.19:0/4093214885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc00aa50 con 0x7ff4e40ceb90 2024-04-25T09:59:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4db7fe640 1 -- 172.21.15.19:0/4093214885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc00ae40 con 0x7ff4e40ceb90 2024-04-25T09:59:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e917e640 1 -- 172.21.15.19:0/4093214885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e40ccf00 con 0x7ff4e40ceb90 2024-04-25T09:59:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.353+0000 7ff4e917e640 1 -- 172.21.15.19:0/4093214885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e40cd4c0 con 0x7ff4e40ceb90 2024-04-25T09:59:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.354+0000 7ff4e917e640 1 -- 172.21.15.19:0/4093214885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e40622a0 con 0x7ff4e40ceb90 2024-04-25T09:59:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.355+0000 7ff4db7fe640 1 -- 172.21.15.19:0/4093214885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4dc019020 con 0x7ff4e40ceb90 2024-04-25T09:59:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.355+0000 7ff4db7fe640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c406b4d0 0x7ff4c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.355+0000 7ff4db7fe640 1 -- 172.21.15.19:0/4093214885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4dc08a3c0 con 0x7ff4e40ceb90 2024-04-25T09:59:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.355+0000 7ff4e259c640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c406b4d0 0x7ff4c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.356+0000 7ff4e259c640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c406b4d0 0x7ff4c406d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc034000 tx=0x7ff4cc02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:50.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.359+0000 7ff4db7fe640 1 -- 172.21.15.19:0/4093214885 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4e40622a0 con 0x7ff4e40ceb90 2024-04-25T09:59:50.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.535+0000 7ff4e917e640 1 -- 172.21.15.19:0/4093214885 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4e40c9a70 con 0x7ff4c406b4d0 2024-04-25T09:59:50.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.541+0000 7ff4db7fe640 1 -- 172.21.15.19:0/4093214885 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252434 (secure 0 0 0) 0x7ff4cc032670 con 0x7ff4c406b4d0 2024-04-25T09:59:50.548 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:50.550 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 -- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c406b4d0 msgr2=0x7ff4c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c406b4d0 0x7ff4c406d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc034000 tx=0x7ff4cc02c000 comp rx=0 tx=0).stop 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 -- 172.21.15.19:0/4093214885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e40ceb90 msgr2=0x7ff4e40cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e40ceb90 0x7ff4e40cc4b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c7c0 tx=0x7ff4dc00cc80 comp rx=0 tx=0).stop 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 -- 172.21.15.19:0/4093214885 shutdown_connections 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff4c406b4d0 0x7ff4c406d990 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4e4160910 0x7ff4e40d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4e415da70 0x7ff4e40c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 --2- 172.21.15.19:0/4093214885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4e40ceb90 0x7ff4e40cc4b0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 -- 172.21.15.19:0/4093214885 >> 172.21.15.19:0/4093214885 conn(0x7ff4e4061a20 msgr2=0x7ff4e415bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 -- 172.21.15.19:0/4093214885 shutdown_connections 2024-04-25T09:59:50.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.544+0000 7ff4e917e640 1 -- 172.21.15.19:0/4093214885 wait complete. 2024-04-25T09:59:50.559 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-04-25T09:59:49.003316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":367768,"kb_used_data":11424,"kb_used_omap":12,"kb_used_meta":356211,"kb_avail":754606952,"statfs":{"total":773094113280,"available":772717518848,"internally_reserved":0,"allocated":11698176,"data_stored":6046524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":364760626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000882"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.571890+0000","last_change":"2024-04-25T09:59:47.568633+0000","last_active":"2024-04-25T09:59:47.571890+0000","last_peered":"2024-04-25T09:59:47.571890+0000","last_clean":"2024-04-25T09:59:47.571890+0000","last_became_active":"2024-04-25T09:59:47.568342+0000","last_became_peered":"2024-04-25T09:59:47.568342+0000","last_unstale":"2024-04-25T09:59:47.571890+0000","last_undegraded":"2024-04-25T09:59:47.571890+0000","last_fullsized":"2024-04-25T09:59:47.571890+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:08.652513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.572833+0000","last_change":"2024-04-25T09:59:47.565668+0000","last_active":"2024-04-25T09:59:47.572833+0000","last_peered":"2024-04-25T09:59:47.572833+0000","last_clean":"2024-04-25T09:59:47.572833+0000","last_became_active":"2024-04-25T09:59:47.565208+0000","last_became_peered":"2024-04-25T09:59:47.565208+0000","last_unstale":"2024-04-25T09:59:47.572833+0000","last_undegraded":"2024-04-25T09:59:47.572833+0000","last_fullsized":"2024-04-25T09:59:47.572833+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:37.762977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.574185+0000","last_change":"2024-04-25T09:59:47.573625+0000","last_active":"2024-04-25T09:59:47.574185+0000","last_peered":"2024-04-25T09:59:47.574185+0000","last_clean":"2024-04-25T09:59:47.574185+0000","last_became_active":"2024-04-25T09:59:47.573329+0000","last_became_peered":"2024-04-25T09:59:47.573329+0000","last_unstale":"2024-04-25T09:59:47.574185+0000","last_undegraded":"2024-04-25T09:59:47.574185+0000","last_fullsized":"2024-04-25T09:59:47.574185+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:50.259657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569753+0000","last_change":"2024-04-25T09:59:47.571166+0000","last_active":"2024-04-25T09:59:48.569753+0000","last_peered":"2024-04-25T09:59:48.569753+0000","last_clean":"2024-04-25T09:59:48.569753+0000","last_became_active":"2024-04-25T09:59:47.570602+0000","last_became_peered":"2024-04-25T09:59:47.570602+0000","last_unstale":"2024-04-25T09:59:48.569753+0000","last_undegraded":"2024-04-25T09:59:48.569753+0000","last_fullsized":"2024-04-25T09:59:48.569753+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:15.406180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.575937+0000","last_change":"2024-04-25T09:59:47.573759+0000","last_active":"2024-04-25T09:59:47.575937+0000","last_peered":"2024-04-25T09:59:47.575937+0000","last_clean":"2024-04-25T09:59:47.575937+0000","last_became_active":"2024-04-25T09:59:47.573469+0000","last_became_peered":"2024-04-25T09:59:47.573469+0000","last_unstale":"2024-04-25T09:59:47.575937+0000","last_undegraded":"2024-04-25T09:59:47.575937+0000","last_fullsized":"2024-04-25T09:59:47.575937+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.584375+0000","last_change":"2024-04-25T09:59:47.581066+0000","last_active":"2024-04-25T09:59:47.584375+0000","last_peered":"2024-04-25T09:59:47.584375+0000","last_clean":"2024-04-25T09:59:47.584375+0000","last_became_active":"2024-04-25T09:59:47.580786+0000","last_became_peered":"2024-04-25T09:59:47.580786+0000","last_unstale":"2024-04-25T09:59:47.584375+0000","last_undegraded":"2024-04-25T09:59:47.584375+0000","last_fullsized":"2024-04-25T09:59:47.584375+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:41.641450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586996+0000","last_change":"2024-04-25T09:59:47.582629+0000","last_active":"2024-04-25T09:59:47.586996+0000","last_peered":"2024-04-25T09:59:47.586996+0000","last_clean":"2024-04-25T09:59:47.586996+0000","last_became_active":"2024-04-25T09:59:47.581549+0000","last_became_peered":"2024-04-25T09:59:47.581549+0000","last_unstale":"2024-04-25T09:59:47.586996+0000","last_undegraded":"2024-04-25T09:59:47.586996+0000","last_fullsized":"2024-04-25T09:59:47.586996+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:47.379583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.589927+0000","last_change":"2024-04-25T09:59:47.588123+0000","last_active":"2024-04-25T09:59:47.589927+0000","last_peered":"2024-04-25T09:59:47.589927+0000","last_clean":"2024-04-25T09:59:47.589927+0000","last_became_active":"2024-04-25T09:59:47.587863+0000","last_became_peered":"2024-04-25T09:59:47.587863+0000","last_unstale":"2024-04-25T09:59:47.589927+0000","last_undegraded":"2024-04-25T09:59:47.589927+0000","last_fullsized":"2024-04-25T09:59:47.589927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:29.649101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573586+0000","last_change":"2024-04-25T09:59:47.569623+0000","last_active":"2024-04-25T09:59:47.573586+0000","last_peered":"2024-04-25T09:59:47.573586+0000","last_clean":"2024-04-25T09:59:47.573586+0000","last_became_active":"2024-04-25T09:59:47.568827+0000","last_became_peered":"2024-04-25T09:59:47.568827+0000","last_unstale":"2024-04-25T09:59:47.573586+0000","last_undegraded":"2024-04-25T09:59:47.573586+0000","last_fullsized":"2024-04-25T09:59:47.573586+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:20.884665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586665+0000","last_change":"2024-04-25T09:59:47.582003+0000","last_active":"2024-04-25T09:59:47.586665+0000","last_peered":"2024-04-25T09:59:47.586665+0000","last_clean":"2024-04-25T09:59:47.586665+0000","last_became_active":"2024-04-25T09:59:47.581718+0000","last_became_peered":"2024-04-25T09:59:47.581718+0000","last_unstale":"2024-04-25T09:59:47.586665+0000","last_undegraded":"2024-04-25T09:59:47.586665+0000","last_fullsized":"2024-04-25T09:59:47.586665+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:41.641450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.577474+0000","last_change":"2024-04-25T09:59:47.576496+0000","last_active":"2024-04-25T09:59:47.577474+0000","last_peered":"2024-04-25T09:59:47.577474+0000","last_clean":"2024-04-25T09:59:47.577474+0000","last_became_active":"2024-04-25T09:59:47.575888+0000","last_became_peered":"2024-04-25T09:59:47.575888+0000","last_unstale":"2024-04-25T09:59:47.577474+0000","last_undegraded":"2024-04-25T09:59:47.577474+0000","last_fullsized":"2024-04-25T09:59:47.577474+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:10.120007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571491+0000","last_change":"2024-04-25T09:59:47.566465+0000","last_active":"2024-04-25T09:59:48.571491+0000","last_peered":"2024-04-25T09:59:48.571491+0000","last_clean":"2024-04-25T09:59:48.571491+0000","last_became_active":"2024-04-25T09:59:47.566170+0000","last_became_peered":"2024-04-25T09:59:47.566170+0000","last_unstale":"2024-04-25T09:59:48.571491+0000","last_undegraded":"2024-04-25T09:59:48.571491+0000","last_fullsized":"2024-04-25T09:59:48.571491+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:10.120007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569446+0000","last_change":"2024-04-25T09:59:47.572484+0000","last_active":"2024-04-25T09:59:48.569446+0000","last_peered":"2024-04-25T09:59:48.569446+0000","last_clean":"2024-04-25T09:59:48.569446+0000","last_became_active":"2024-04-25T09:59:47.571791+0000","last_became_peered":"2024-04-25T09:59:47.571791+0000","last_unstale":"2024-04-25T09:59:48.569446+0000","last_undegraded":"2024-04-25T09:59:48.569446+0000","last_fullsized":"2024-04-25T09:59:48.569446+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:21.208467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571388+0000","last_change":"2024-04-25T09:59:47.568591+0000","last_active":"2024-04-25T09:59:48.571388+0000","last_peered":"2024-04-25T09:59:48.571388+0000","last_clean":"2024-04-25T09:59:48.571388+0000","last_became_active":"2024-04-25T09:59:47.568317+0000","last_became_peered":"2024-04-25T09:59:47.568317+0000","last_unstale":"2024-04-25T09:59:48.571388+0000","last_undegraded":"2024-04-25T09:59:48.571388+0000","last_fullsized":"2024-04-25T09:59:48.571388+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:59.271674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.570977+0000","last_change":"2024-04-25T09:59:47.573408+0000","last_active":"2024-04-25T09:59:48.570977+0000","last_peered":"2024-04-25T09:59:48.570977+0000","last_clean":"2024-04-25T09:59:48.570977+0000","last_became_active":"2024-04-25T09:59:47.573053+0000","last_became_peered":"2024-04-25T09:59:47.573053+0000","last_unstale":"2024-04-25T09:59:48.570977+0000","last_undegraded":"2024-04-25T09:59:48.570977+0000","last_fullsized":"2024-04-25T09:59:48.570977+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:20.286657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571047+0000","last_change":"2024-04-25T09:59:47.578390+0000","last_active":"2024-04-25T09:59:48.571047+0000","last_peered":"2024-04-25T09:59:48.571047+0000","last_clean":"2024-04-25T09:59:48.571047+0000","last_became_active":"2024-04-25T09:59:47.578053+0000","last_became_peered":"2024-04-25T09:59:47.578053+0000","last_unstale":"2024-04-25T09:59:48.571047+0000","last_undegraded":"2024-04-25T09:59:48.571047+0000","last_fullsized":"2024-04-25T09:59:48.571047+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:25.819100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.576867+0000","last_change":"2024-04-25T09:59:47.575824+0000","last_active":"2024-04-25T09:59:47.576867+0000","last_peered":"2024-04-25T09:59:47.576867+0000","last_clean":"2024-04-25T09:59:47.576867+0000","last_became_active":"2024-04-25T09:59:47.574522+0000","last_became_peered":"2024-04-25T09:59:47.574522+0000","last_unstale":"2024-04-25T09:59:47.576867+0000","last_undegraded":"2024-04-25T09:59:47.576867+0000","last_fullsized":"2024-04-25T09:59:47.576867+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:20.069929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586706+0000","last_change":"2024-04-25T09:59:47.582917+0000","last_active":"2024-04-25T09:59:47.586706+0000","last_peered":"2024-04-25T09:59:47.586706+0000","last_clean":"2024-04-25T09:59:47.586706+0000","last_became_active":"2024-04-25T09:59:47.582618+0000","last_became_peered":"2024-04-25T09:59:47.582618+0000","last_unstale":"2024-04-25T09:59:47.586706+0000","last_undegraded":"2024-04-25T09:59:47.586706+0000","last_fullsized":"2024-04-25T09:59:47.586706+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:11.334977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573908+0000","last_change":"2024-04-25T09:59:47.571000+0000","last_active":"2024-04-25T09:59:47.573908+0000","last_peered":"2024-04-25T09:59:47.573908+0000","last_clean":"2024-04-25T09:59:47.573908+0000","last_became_active":"2024-04-25T09:59:47.570352+0000","last_became_peered":"2024-04-25T09:59:47.570352+0000","last_unstale":"2024-04-25T09:59:47.573908+0000","last_undegraded":"2024-04-25T09:59:47.573908+0000","last_fullsized":"2024-04-25T09:59:47.573908+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:22.082005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571643+0000","last_change":"2024-04-25T09:59:47.565164+0000","last_active":"2024-04-25T09:59:48.571643+0000","last_peered":"2024-04-25T09:59:48.571643+0000","last_clean":"2024-04-25T09:59:48.571643+0000","last_became_active":"2024-04-25T09:59:47.564866+0000","last_became_peered":"2024-04-25T09:59:47.564866+0000","last_unstale":"2024-04-25T09:59:48.571643+0000","last_undegraded":"2024-04-25T09:59:48.571643+0000","last_fullsized":"2024-04-25T09:59:48.571643+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:13.998959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579393+0000","last_change":"2024-04-25T09:59:47.576249+0000","last_active":"2024-04-25T09:59:47.579393+0000","last_peered":"2024-04-25T09:59:47.579393+0000","last_clean":"2024-04-25T09:59:47.579393+0000","last_became_active":"2024-04-25T09:59:47.575932+0000","last_became_peered":"2024-04-25T09:59:47.575932+0000","last_unstale":"2024-04-25T09:59:47.579393+0000","last_undegraded":"2024-04-25T09:59:47.579393+0000","last_fullsized":"2024-04-25T09:59:47.579393+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:26.135650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.587131+0000","last_change":"2024-04-25T09:59:47.584187+0000","last_active":"2024-04-25T09:59:47.587131+0000","last_peered":"2024-04-25T09:59:47.587131+0000","last_clean":"2024-04-25T09:59:47.587131+0000","last_became_active":"2024-04-25T09:59:47.583039+0000","last_became_peered":"2024-04-25T09:59:47.583039+0000","last_unstale":"2024-04-25T09:59:47.587131+0000","last_undegraded":"2024-04-25T09:59:47.587131+0000","last_fullsized":"2024-04-25T09:59:47.587131+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:50.977149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.575440+0000","last_change":"2024-04-25T09:59:47.573685+0000","last_active":"2024-04-25T09:59:47.575440+0000","last_peered":"2024-04-25T09:59:47.575440+0000","last_clean":"2024-04-25T09:59:47.575440+0000","last_became_active":"2024-04-25T09:59:47.573406+0000","last_became_peered":"2024-04-25T09:59:47.573406+0000","last_unstale":"2024-04-25T09:59:47.575440+0000","last_undegraded":"2024-04-25T09:59:47.575440+0000","last_fullsized":"2024-04-25T09:59:47.575440+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:10.018295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586735+0000","last_change":"2024-04-25T09:59:47.582096+0000","last_active":"2024-04-25T09:59:47.586735+0000","last_peered":"2024-04-25T09:59:47.586735+0000","last_clean":"2024-04-25T09:59:47.586735+0000","last_became_active":"2024-04-25T09:59:47.573784+0000","last_became_peered":"2024-04-25T09:59:47.573784+0000","last_unstale":"2024-04-25T09:59:47.586735+0000","last_undegraded":"2024-04-25T09:59:47.586735+0000","last_fullsized":"2024-04-25T09:59:47.586735+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:11.334977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571730+0000","last_change":"2024-04-25T09:59:47.573537+0000","last_active":"2024-04-25T09:59:48.571730+0000","last_peered":"2024-04-25T09:59:48.571730+0000","last_clean":"2024-04-25T09:59:48.571730+0000","last_became_active":"2024-04-25T09:59:47.572762+0000","last_became_peered":"2024-04-25T09:59:47.572762+0000","last_unstale":"2024-04-25T09:59:48.571730+0000","last_undegraded":"2024-04-25T09:59:48.571730+0000","last_fullsized":"2024-04-25T09:59:48.571730+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:32.607621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571375+0000","last_change":"2024-04-25T09:59:47.568385+0000","last_active":"2024-04-25T09:59:48.571375+0000","last_peered":"2024-04-25T09:59:48.571375+0000","last_clean":"2024-04-25T09:59:48.571375+0000","last_became_active":"2024-04-25T09:59:47.568087+0000","last_became_peered":"2024-04-25T09:59:47.568087+0000","last_unstale":"2024-04-25T09:59:48.571375+0000","last_undegraded":"2024-04-25T09:59:48.571375+0000","last_fullsized":"2024-04-25T09:59:48.571375+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:48.997774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.571516+0000","last_change":"2024-04-25T09:59:47.570263+0000","last_active":"2024-04-25T09:59:47.571516+0000","last_peered":"2024-04-25T09:59:47.571516+0000","last_clean":"2024-04-25T09:59:47.571516+0000","last_became_active":"2024-04-25T09:59:47.569540+0000","last_became_peered":"2024-04-25T09:59:47.569540+0000","last_unstale":"2024-04-25T09:59:47.571516+0000","last_undegraded":"2024-04-25T09:59:47.571516+0000","last_fullsized":"2024-04-25T09:59:47.571516+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:41.641450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.577878+0000","last_change":"2024-04-25T09:59:47.575640+0000","last_active":"2024-04-25T09:59:47.577878+0000","last_peered":"2024-04-25T09:59:47.577878+0000","last_clean":"2024-04-25T09:59:47.577878+0000","last_became_active":"2024-04-25T09:59:47.575199+0000","last_became_peered":"2024-04-25T09:59:47.575199+0000","last_unstale":"2024-04-25T09:59:47.577878+0000","last_undegraded":"2024-04-25T09:59:47.577878+0000","last_fullsized":"2024-04-25T09:59:47.577878+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:06.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579370+0000","last_change":"2024-04-25T09:59:47.576121+0000","last_active":"2024-04-25T09:59:47.579370+0000","last_peered":"2024-04-25T09:59:47.579370+0000","last_clean":"2024-04-25T09:59:47.579370+0000","last_became_active":"2024-04-25T09:59:47.575537+0000","last_became_peered":"2024-04-25T09:59:47.575537+0000","last_unstale":"2024-04-25T09:59:47.579370+0000","last_undegraded":"2024-04-25T09:59:47.579370+0000","last_fullsized":"2024-04-25T09:59:47.579370+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:28.503272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571946+0000","last_change":"2024-04-25T09:59:47.572253+0000","last_active":"2024-04-25T09:59:48.571946+0000","last_peered":"2024-04-25T09:59:48.571946+0000","last_clean":"2024-04-25T09:59:48.571946+0000","last_became_active":"2024-04-25T09:59:47.571973+0000","last_became_peered":"2024-04-25T09:59:47.571973+0000","last_unstale":"2024-04-25T09:59:48.571946+0000","last_undegraded":"2024-04-25T09:59:48.571946+0000","last_fullsized":"2024-04-25T09:59:48.571946+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:40.175103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.574766+0000","last_change":"2024-04-25T09:59:47.571757+0000","last_active":"2024-04-25T09:59:47.574766+0000","last_peered":"2024-04-25T09:59:47.574766+0000","last_clean":"2024-04-25T09:59:47.574766+0000","last_became_active":"2024-04-25T09:59:47.570664+0000","last_became_peered":"2024-04-25T09:59:47.570664+0000","last_unstale":"2024-04-25T09:59:47.574766+0000","last_undegraded":"2024-04-25T09:59:47.574766+0000","last_fullsized":"2024-04-25T09:59:47.574766+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579910+0000","last_change":"2024-04-25T09:59:47.578705+0000","last_active":"2024-04-25T09:59:47.579910+0000","last_peered":"2024-04-25T09:59:47.579910+0000","last_clean":"2024-04-25T09:59:47.579910+0000","last_became_active":"2024-04-25T09:59:47.577826+0000","last_became_peered":"2024-04-25T09:59:47.577826+0000","last_unstale":"2024-04-25T09:59:47.579910+0000","last_undegraded":"2024-04-25T09:59:47.579910+0000","last_fullsized":"2024-04-25T09:59:47.579910+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:44.017705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.571841+0000","last_change":"2024-04-25T09:59:47.568929+0000","last_active":"2024-04-25T09:59:47.571841+0000","last_peered":"2024-04-25T09:59:47.571841+0000","last_clean":"2024-04-25T09:59:47.571841+0000","last_became_active":"2024-04-25T09:59:47.567921+0000","last_became_peered":"2024-04-25T09:59:47.567921+0000","last_unstale":"2024-04-25T09:59:47.571841+0000","last_undegraded":"2024-04-25T09:59:47.571841+0000","last_fullsized":"2024-04-25T09:59:47.571841+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:00.118302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.570023+0000","last_change":"2024-04-25T09:59:46.553598+0000","last_active":"2024-04-25T09:59:48.570023+0000","last_peered":"2024-04-25T09:59:48.570023+0000","last_clean":"2024-04-25T09:59:48.570023+0000","last_became_active":"2024-04-25T09:59:46.553335+0000","last_became_peered":"2024-04-25T09:59:46.553335+0000","last_unstale":"2024-04-25T09:59:48.570023+0000","last_undegraded":"2024-04-25T09:59:48.570023+0000","last_fullsized":"2024-04-25T09:59:48.570023+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:09.115929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579325+0000","last_change":"2024-04-25T09:59:47.577578+0000","last_active":"2024-04-25T09:59:47.579325+0000","last_peered":"2024-04-25T09:59:47.579325+0000","last_clean":"2024-04-25T09:59:47.579325+0000","last_became_active":"2024-04-25T09:59:47.577267+0000","last_became_peered":"2024-04-25T09:59:47.577267+0000","last_unstale":"2024-04-25T09:59:47.579325+0000","last_undegraded":"2024-04-25T09:59:47.579325+0000","last_fullsized":"2024-04-25T09:59:47.579325+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:04.405045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.591261+0000","last_change":"2024-04-25T09:59:46.548719+0000","last_active":"2024-04-25T09:59:47.591261+0000","last_peered":"2024-04-25T09:59:47.591261+0000","last_clean":"2024-04-25T09:59:47.591261+0000","last_became_active":"2024-04-25T09:59:46.548431+0000","last_became_peered":"2024-04-25T09:59:46.548431+0000","last_unstale":"2024-04-25T09:59:47.591261+0000","last_undegraded":"2024-04-25T09:59:47.591261+0000","last_fullsized":"2024-04-25T09:59:47.591261+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:04.524363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571299+0000","last_change":"2024-04-25T09:59:47.574967+0000","last_active":"2024-04-25T09:59:48.571299+0000","last_peered":"2024-04-25T09:59:48.571299+0000","last_clean":"2024-04-25T09:59:48.571299+0000","last_became_active":"2024-04-25T09:59:47.574044+0000","last_became_peered":"2024-04-25T09:59:47.574044+0000","last_unstale":"2024-04-25T09:59:48.571299+0000","last_undegraded":"2024-04-25T09:59:48.571299+0000","last_fullsized":"2024-04-25T09:59:48.571299+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:50.977149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.572018+0000","last_change":"2024-04-25T09:59:46.557167+0000","last_active":"2024-04-25T09:59:48.572018+0000","last_peered":"2024-04-25T09:59:48.572018+0000","last_clean":"2024-04-25T09:59:48.572018+0000","last_became_active":"2024-04-25T09:59:46.556864+0000","last_became_peered":"2024-04-25T09:59:46.556864+0000","last_unstale":"2024-04-25T09:59:48.572018+0000","last_undegraded":"2024-04-25T09:59:48.572018+0000","last_fullsized":"2024-04-25T09:59:48.572018+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:45.239556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.576264+0000","last_change":"2024-04-25T09:59:47.574559+0000","last_active":"2024-04-25T09:59:47.576264+0000","last_peered":"2024-04-25T09:59:47.576264+0000","last_clean":"2024-04-25T09:59:47.576264+0000","last_became_active":"2024-04-25T09:59:47.574247+0000","last_became_peered":"2024-04-25T09:59:47.574247+0000","last_unstale":"2024-04-25T09:59:47.576264+0000","last_undegraded":"2024-04-25T09:59:47.576264+0000","last_fullsized":"2024-04-25T09:59:47.576264+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:48.997774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.587470+0000","last_change":"2024-04-25T09:59:46.561068+0000","last_active":"2024-04-25T09:59:47.587470+0000","last_peered":"2024-04-25T09:59:47.587470+0000","last_clean":"2024-04-25T09:59:47.587470+0000","last_became_active":"2024-04-25T09:59:46.560492+0000","last_became_peered":"2024-04-25T09:59:46.560492+0000","last_unstale":"2024-04-25T09:59:47.587470+0000","last_undegraded":"2024-04-25T09:59:47.587470+0000","last_fullsized":"2024-04-25T09:59:47.587470+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:45.239556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569777+0000","last_change":"2024-04-25T09:59:47.581294+0000","last_active":"2024-04-25T09:59:48.569777+0000","last_peered":"2024-04-25T09:59:48.569777+0000","last_clean":"2024-04-25T09:59:48.569777+0000","last_became_active":"2024-04-25T09:59:47.581002+0000","last_became_peered":"2024-04-25T09:59:47.581002+0000","last_unstale":"2024-04-25T09:59:48.569777+0000","last_undegraded":"2024-04-25T09:59:48.569777+0000","last_fullsized":"2024-04-25T09:59:48.569777+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:20.286657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571761+0000","last_change":"2024-04-25T09:59:46.550941+0000","last_active":"2024-04-25T09:59:48.571761+0000","last_peered":"2024-04-25T09:59:48.571761+0000","last_clean":"2024-04-25T09:59:48.571761+0000","last_became_active":"2024-04-25T09:59:46.550575+0000","last_became_peered":"2024-04-25T09:59:46.550575+0000","last_unstale":"2024-04-25T09:59:48.571761+0000","last_undegraded":"2024-04-25T09:59:48.571761+0000","last_fullsized":"2024-04-25T09:59:48.571761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:26.799473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573890+0000","last_change":"2024-04-25T09:59:47.570624+0000","last_active":"2024-04-25T09:59:47.573890+0000","last_peered":"2024-04-25T09:59:47.573890+0000","last_clean":"2024-04-25T09:59:47.573890+0000","last_became_active":"2024-04-25T09:59:47.569985+0000","last_became_peered":"2024-04-25T09:59:47.569985+0000","last_unstale":"2024-04-25T09:59:47.573890+0000","last_undegraded":"2024-04-25T09:59:47.573890+0000","last_fullsized":"2024-04-25T09:59:47.573890+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:33.869614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.582408+0000","last_change":"2024-04-25T09:59:46.554738+0000","last_active":"2024-04-25T09:59:47.582408+0000","last_peered":"2024-04-25T09:59:47.582408+0000","last_clean":"2024-04-25T09:59:47.582408+0000","last_became_active":"2024-04-25T09:59:46.554280+0000","last_became_peered":"2024-04-25T09:59:46.554280+0000","last_unstale":"2024-04-25T09:59:47.582408+0000","last_undegraded":"2024-04-25T09:59:47.582408+0000","last_fullsized":"2024-04-25T09:59:47.582408+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:05.609216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.575662+0000","last_change":"2024-04-25T09:59:47.574359+0000","last_active":"2024-04-25T09:59:47.575662+0000","last_peered":"2024-04-25T09:59:47.575662+0000","last_clean":"2024-04-25T09:59:47.575662+0000","last_became_active":"2024-04-25T09:59:47.574069+0000","last_became_peered":"2024-04-25T09:59:47.574069+0000","last_unstale":"2024-04-25T09:59:47.575662+0000","last_undegraded":"2024-04-25T09:59:47.575662+0000","last_fullsized":"2024-04-25T09:59:47.575662+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:36.170205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571052+0000","last_change":"2024-04-25T09:59:46.552210+0000","last_active":"2024-04-25T09:59:48.571052+0000","last_peered":"2024-04-25T09:59:48.571052+0000","last_clean":"2024-04-25T09:59:48.571052+0000","last_became_active":"2024-04-25T09:59:46.551764+0000","last_became_peered":"2024-04-25T09:59:46.551764+0000","last_unstale":"2024-04-25T09:59:48.571052+0000","last_undegraded":"2024-04-25T09:59:48.571052+0000","last_fullsized":"2024-04-25T09:59:48.571052+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:02.605211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569927+0000","last_change":"2024-04-25T09:59:47.572553+0000","last_active":"2024-04-25T09:59:48.569927+0000","last_peered":"2024-04-25T09:59:48.569927+0000","last_clean":"2024-04-25T09:59:48.569927+0000","last_became_active":"2024-04-25T09:59:47.572272+0000","last_became_peered":"2024-04-25T09:59:47.572272+0000","last_unstale":"2024-04-25T09:59:48.569927+0000","last_undegraded":"2024-04-25T09:59:48.569927+0000","last_fullsized":"2024-04-25T09:59:48.569927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:25.819100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569340+0000","last_change":"2024-04-25T09:59:46.560393+0000","last_active":"2024-04-25T09:59:48.569340+0000","last_peered":"2024-04-25T09:59:48.569340+0000","last_clean":"2024-04-25T09:59:48.569340+0000","last_became_active":"2024-04-25T09:59:46.560107+0000","last_became_peered":"2024-04-25T09:59:46.560107+0000","last_unstale":"2024-04-25T09:59:48.569340+0000","last_undegraded":"2024-04-25T09:59:48.569340+0000","last_fullsized":"2024-04-25T09:59:48.569340+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:47.993697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.572243+0000","last_change":"2024-04-25T09:59:47.570071+0000","last_active":"2024-04-25T09:59:47.572243+0000","last_peered":"2024-04-25T09:59:47.572243+0000","last_clean":"2024-04-25T09:59:47.572243+0000","last_became_active":"2024-04-25T09:59:47.568308+0000","last_became_peered":"2024-04-25T09:59:47.568308+0000","last_unstale":"2024-04-25T09:59:47.572243+0000","last_undegraded":"2024-04-25T09:59:47.572243+0000","last_fullsized":"2024-04-25T09:59:47.572243+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:37.523348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571824+0000","last_change":"2024-04-25T09:59:46.553828+0000","last_active":"2024-04-25T09:59:48.571824+0000","last_peered":"2024-04-25T09:59:48.571824+0000","last_clean":"2024-04-25T09:59:48.571824+0000","last_became_active":"2024-04-25T09:59:46.553567+0000","last_became_peered":"2024-04-25T09:59:46.553567+0000","last_unstale":"2024-04-25T09:59:48.571824+0000","last_undegraded":"2024-04-25T09:59:48.571824+0000","last_fullsized":"2024-04-25T09:59:48.571824+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:36.137754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586702+0000","last_change":"2024-04-25T09:59:47.582086+0000","last_active":"2024-04-25T09:59:47.586702+0000","last_peered":"2024-04-25T09:59:47.586702+0000","last_clean":"2024-04-25T09:59:47.586702+0000","last_became_active":"2024-04-25T09:59:47.581709+0000","last_became_peered":"2024-04-25T09:59:47.581709+0000","last_unstale":"2024-04-25T09:59:47.586702+0000","last_undegraded":"2024-04-25T09:59:47.586702+0000","last_fullsized":"2024-04-25T09:59:47.586702+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:47.379583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.554973+0000","last_change":"2024-04-25T09:59:46.553720+0000","last_active":"2024-04-25T09:59:47.554973+0000","last_peered":"2024-04-25T09:59:47.554973+0000","last_clean":"2024-04-25T09:59:47.554973+0000","last_became_active":"2024-04-25T09:59:46.553457+0000","last_became_peered":"2024-04-25T09:59:46.553457+0000","last_unstale":"2024-04-25T09:59:47.554973+0000","last_undegraded":"2024-04-25T09:59:47.554973+0000","last_fullsized":"2024-04-25T09:59:47.554973+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:32.975313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.588938+0000","last_change":"2024-04-25T09:59:47.587730+0000","last_active":"2024-04-25T09:59:47.588938+0000","last_peered":"2024-04-25T09:59:47.588938+0000","last_clean":"2024-04-25T09:59:47.588938+0000","last_became_active":"2024-04-25T09:59:47.587455+0000","last_became_peered":"2024-04-25T09:59:47.587455+0000","last_unstale":"2024-04-25T09:59:47.588938+0000","last_undegraded":"2024-04-25T09:59:47.588938+0000","last_fullsized":"2024-04-25T09:59:47.588938+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:16.168496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571813+0000","last_change":"2024-04-25T09:59:46.547192+0000","last_active":"2024-04-25T09:59:48.571813+0000","last_peered":"2024-04-25T09:59:48.571813+0000","last_clean":"2024-04-25T09:59:48.571813+0000","last_became_active":"2024-04-25T09:59:46.545232+0000","last_became_peered":"2024-04-25T09:59:46.545232+0000","last_unstale":"2024-04-25T09:59:48.571813+0000","last_undegraded":"2024-04-25T09:59:48.571813+0000","last_fullsized":"2024-04-25T09:59:48.571813+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:00.126112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586288+0000","last_change":"2024-04-25T09:59:47.580730+0000","last_active":"2024-04-25T09:59:47.586288+0000","last_peered":"2024-04-25T09:59:47.586288+0000","last_clean":"2024-04-25T09:59:47.586288+0000","last_became_active":"2024-04-25T09:59:47.580423+0000","last_became_peered":"2024-04-25T09:59:47.580423+0000","last_unstale":"2024-04-25T09:59:47.586288+0000","last_undegraded":"2024-04-25T09:59:47.586288+0000","last_fullsized":"2024-04-25T09:59:47.586288+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:06.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.590617+0000","last_change":"2024-04-25T09:59:46.556227+0000","last_active":"2024-04-25T09:59:47.590617+0000","last_peered":"2024-04-25T09:59:47.590617+0000","last_clean":"2024-04-25T09:59:47.590617+0000","last_became_active":"2024-04-25T09:59:46.555907+0000","last_became_peered":"2024-04-25T09:59:46.555907+0000","last_unstale":"2024-04-25T09:59:47.590617+0000","last_undegraded":"2024-04-25T09:59:47.590617+0000","last_fullsized":"2024-04-25T09:59:47.590617+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:47.928616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573503+0000","last_change":"2024-04-25T09:59:47.572164+0000","last_active":"2024-04-25T09:59:47.573503+0000","last_peered":"2024-04-25T09:59:47.573503+0000","last_clean":"2024-04-25T09:59:47.573503+0000","last_became_active":"2024-04-25T09:59:47.571869+0000","last_became_peered":"2024-04-25T09:59:47.571869+0000","last_unstale":"2024-04-25T09:59:47.573503+0000","last_undegraded":"2024-04-25T09:59:47.573503+0000","last_fullsized":"2024-04-25T09:59:47.573503+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:50.977149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.588946+0000","last_change":"2024-04-25T09:59:46.558626+0000","last_active":"2024-04-25T09:59:47.588946+0000","last_peered":"2024-04-25T09:59:47.588946+0000","last_clean":"2024-04-25T09:59:47.588946+0000","last_became_active":"2024-04-25T09:59:46.558303+0000","last_became_peered":"2024-04-25T09:59:46.558303+0000","last_unstale":"2024-04-25T09:59:47.588946+0000","last_undegraded":"2024-04-25T09:59:47.588946+0000","last_fullsized":"2024-04-25T09:59:47.588946+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:19.880588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573993+0000","last_change":"2024-04-25T09:59:47.568843+0000","last_active":"2024-04-25T09:59:47.573993+0000","last_peered":"2024-04-25T09:59:47.573993+0000","last_clean":"2024-04-25T09:59:47.573993+0000","last_became_active":"2024-04-25T09:59:47.568326+0000","last_became_peered":"2024-04-25T09:59:47.568326+0000","last_unstale":"2024-04-25T09:59:47.573993+0000","last_undegraded":"2024-04-25T09:59:47.573993+0000","last_fullsized":"2024-04-25T09:59:47.573993+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:46.243634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569456+0000","last_change":"2024-04-25T09:59:46.546542+0000","last_active":"2024-04-25T09:59:48.569456+0000","last_peered":"2024-04-25T09:59:48.569456+0000","last_clean":"2024-04-25T09:59:48.569456+0000","last_became_active":"2024-04-25T09:59:46.546242+0000","last_became_peered":"2024-04-25T09:59:46.546242+0000","last_unstale":"2024-04-25T09:59:48.569456+0000","last_undegraded":"2024-04-25T09:59:48.569456+0000","last_fullsized":"2024-04-25T09:59:48.569456+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:49.973071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.577744+0000","last_change":"2024-04-25T09:59:47.575505+0000","last_active":"2024-04-25T09:59:47.577744+0000","last_peered":"2024-04-25T09:59:47.577744+0000","last_clean":"2024-04-25T09:59:47.577744+0000","last_became_active":"2024-04-25T09:59:47.575197+0000","last_became_peered":"2024-04-25T09:59:47.575197+0000","last_unstale":"2024-04-25T09:59:47.577744+0000","last_undegraded":"2024-04-25T09:59:47.577744+0000","last_fullsized":"2024-04-25T09:59:47.577744+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:31.448089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.562838+0000","last_change":"2024-04-25T09:59:46.560493+0000","last_active":"2024-04-25T09:59:47.562838+0000","last_peered":"2024-04-25T09:59:47.562838+0000","last_clean":"2024-04-25T09:59:47.562838+0000","last_became_active":"2024-04-25T09:59:46.560168+0000","last_became_peered":"2024-04-25T09:59:46.560168+0000","last_unstale":"2024-04-25T09:59:47.562838+0000","last_undegraded":"2024-04-25T09:59:47.562838+0000","last_fullsized":"2024-04-25T09:59:47.562838+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:39.594594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.576888+0000","last_change":"2024-04-25T09:59:47.576087+0000","last_active":"2024-04-25T09:59:47.576888+0000","last_peered":"2024-04-25T09:59:47.576888+0000","last_clean":"2024-04-25T09:59:47.576888+0000","last_became_active":"2024-04-25T09:59:47.574638+0000","last_became_peered":"2024-04-25T09:59:47.574638+0000","last_unstale":"2024-04-25T09:59:47.576888+0000","last_undegraded":"2024-04-25T09:59:47.576888+0000","last_fullsized":"2024-04-25T09:59:47.576888+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:12.602237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.570759+0000","last_change":"2024-04-25T09:59:46.556274+0000","last_active":"2024-04-25T09:59:48.570759+0000","last_peered":"2024-04-25T09:59:48.570759+0000","last_clean":"2024-04-25T09:59:48.570759+0000","last_became_active":"2024-04-25T09:59:46.556000+0000","last_became_peered":"2024-04-25T09:59:46.556000+0000","last_unstale":"2024-04-25T09:59:48.570759+0000","last_undegraded":"2024-04-25T09:59:48.570759+0000","last_fullsized":"2024-04-25T09:59:48.570759+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:11.966549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579931+0000","last_change":"2024-04-25T09:59:47.578895+0000","last_active":"2024-04-25T09:59:47.579931+0000","last_peered":"2024-04-25T09:59:47.579931+0000","last_clean":"2024-04-25T09:59:47.579931+0000","last_became_active":"2024-04-25T09:59:47.578093+0000","last_became_peered":"2024-04-25T09:59:47.578093+0000","last_unstale":"2024-04-25T09:59:47.579931+0000","last_undegraded":"2024-04-25T09:59:47.579931+0000","last_fullsized":"2024-04-25T09:59:47.579931+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:36.203126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586793+0000","last_change":"2024-04-25T09:59:46.559170+0000","last_active":"2024-04-25T09:59:47.586793+0000","last_peered":"2024-04-25T09:59:47.586793+0000","last_clean":"2024-04-25T09:59:47.586793+0000","last_became_active":"2024-04-25T09:59:46.558877+0000","last_became_peered":"2024-04-25T09:59:46.558877+0000","last_unstale":"2024-04-25T09:59:47.586793+0000","last_undegraded":"2024-04-25T09:59:47.586793+0000","last_fullsized":"2024-04-25T09:59:47.586793+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:36.137754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573991+0000","last_change":"2024-04-25T09:59:47.568967+0000","last_active":"2024-04-25T09:59:47.573991+0000","last_peered":"2024-04-25T09:59:47.573991+0000","last_clean":"2024-04-25T09:59:47.573991+0000","last_became_active":"2024-04-25T09:59:47.567986+0000","last_became_peered":"2024-04-25T09:59:47.567986+0000","last_unstale":"2024-04-25T09:59:47.573991+0000","last_undegraded":"2024-04-25T09:59:47.573991+0000","last_fullsized":"2024-04-25T09:59:47.573991+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:06.345771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571913+0000","last_change":"2024-04-25T09:59:46.559231+0000","last_active":"2024-04-25T09:59:48.571913+0000","last_peered":"2024-04-25T09:59:48.571913+0000","last_clean":"2024-04-25T09:59:48.571913+0000","last_became_active":"2024-04-25T09:59:46.558836+0000","last_became_peered":"2024-04-25T09:59:46.558836+0000","last_unstale":"2024-04-25T09:59:48.571913+0000","last_undegraded":"2024-04-25T09:59:48.571913+0000","last_fullsized":"2024-04-25T09:59:48.571913+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:51.194047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.582753+0000","last_change":"2024-04-25T09:59:47.575787+0000","last_active":"2024-04-25T09:59:47.582753+0000","last_peered":"2024-04-25T09:59:47.582753+0000","last_clean":"2024-04-25T09:59:47.582753+0000","last_became_active":"2024-04-25T09:59:47.572949+0000","last_became_peered":"2024-04-25T09:59:47.572949+0000","last_unstale":"2024-04-25T09:59:47.582753+0000","last_undegraded":"2024-04-25T09:59:47.582753+0000","last_fullsized":"2024-04-25T09:59:47.582753+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:36.170205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.568172+0000","last_change":"2024-04-25T09:59:46.553605+0000","last_active":"2024-04-25T09:59:47.568172+0000","last_peered":"2024-04-25T09:59:47.568172+0000","last_clean":"2024-04-25T09:59:47.568172+0000","last_became_active":"2024-04-25T09:59:46.553338+0000","last_became_peered":"2024-04-25T09:59:46.553338+0000","last_unstale":"2024-04-25T09:59:47.568172+0000","last_undegraded":"2024-04-25T09:59:47.568172+0000","last_fullsized":"2024-04-25T09:59:47.568172+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:35.166127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.569139+0000","last_change":"2024-04-25T09:59:47.566375+0000","last_active":"2024-04-25T09:59:47.569139+0000","last_peered":"2024-04-25T09:59:47.569139+0000","last_clean":"2024-04-25T09:59:47.569139+0000","last_became_active":"2024-04-25T09:59:47.566109+0000","last_became_peered":"2024-04-25T09:59:47.566109+0000","last_unstale":"2024-04-25T09:59:47.569139+0000","last_undegraded":"2024-04-25T09:59:47.569139+0000","last_fullsized":"2024-04-25T09:59:47.569139+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:49.066163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.591013+0000","last_change":"2024-04-25T09:59:46.556827+0000","last_active":"2024-04-25T09:59:47.591013+0000","last_peered":"2024-04-25T09:59:47.591013+0000","last_clean":"2024-04-25T09:59:47.591013+0000","last_became_active":"2024-04-25T09:59:46.556393+0000","last_became_peered":"2024-04-25T09:59:46.556393+0000","last_unstale":"2024-04-25T09:59:47.591013+0000","last_undegraded":"2024-04-25T09:59:47.591013+0000","last_fullsized":"2024-04-25T09:59:47.591013+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:35.940120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.576514+0000","last_change":"2024-04-25T09:59:47.575238+0000","last_active":"2024-04-25T09:59:47.576514+0000","last_peered":"2024-04-25T09:59:47.576514+0000","last_clean":"2024-04-25T09:59:47.576514+0000","last_became_active":"2024-04-25T09:59:47.574541+0000","last_became_peered":"2024-04-25T09:59:47.574541+0000","last_unstale":"2024-04-25T09:59:47.576514+0000","last_undegraded":"2024-04-25T09:59:47.576514+0000","last_fullsized":"2024-04-25T09:59:47.576514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:30.031204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.564666+0000","last_change":"2024-04-25T09:59:46.561911+0000","last_active":"2024-04-25T09:59:47.564666+0000","last_peered":"2024-04-25T09:59:47.564666+0000","last_clean":"2024-04-25T09:59:47.564666+0000","last_became_active":"2024-04-25T09:59:46.561335+0000","last_became_peered":"2024-04-25T09:59:46.561335+0000","last_unstale":"2024-04-25T09:59:47.564666+0000","last_undegraded":"2024-04-25T09:59:47.564666+0000","last_fullsized":"2024-04-25T09:59:47.564666+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:59.217379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569741+0000","last_change":"2024-04-25T09:59:47.569180+0000","last_active":"2024-04-25T09:59:48.569741+0000","last_peered":"2024-04-25T09:59:48.569741+0000","last_clean":"2024-04-25T09:59:48.569741+0000","last_became_active":"2024-04-25T09:59:47.568833+0000","last_became_peered":"2024-04-25T09:59:47.568833+0000","last_unstale":"2024-04-25T09:59:48.569741+0000","last_undegraded":"2024-04-25T09:59:48.569741+0000","last_fullsized":"2024-04-25T09:59:48.569741+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:40.267433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.565343+0000","last_change":"2024-04-25T09:59:46.557402+0000","last_active":"2024-04-25T09:59:47.565343+0000","last_peered":"2024-04-25T09:59:47.565343+0000","last_clean":"2024-04-25T09:59:47.565343+0000","last_became_active":"2024-04-25T09:59:46.556975+0000","last_became_peered":"2024-04-25T09:59:46.556975+0000","last_unstale":"2024-04-25T09:59:47.565343+0000","last_undegraded":"2024-04-25T09:59:47.565343+0000","last_fullsized":"2024-04-25T09:59:47.565343+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:02.605211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579368+0000","last_change":"2024-04-25T09:59:47.578193+0000","last_active":"2024-04-25T09:59:47.579368+0000","last_peered":"2024-04-25T09:59:47.579368+0000","last_clean":"2024-04-25T09:59:47.579368+0000","last_became_active":"2024-04-25T09:59:47.577892+0000","last_became_peered":"2024-04-25T09:59:47.577892+0000","last_unstale":"2024-04-25T09:59:47.579368+0000","last_undegraded":"2024-04-25T09:59:47.579368+0000","last_fullsized":"2024-04-25T09:59:47.579368+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:32.907293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.566402+0000","last_change":"2024-04-25T09:59:46.561750+0000","last_active":"2024-04-25T09:59:47.566402+0000","last_peered":"2024-04-25T09:59:47.566402+0000","last_clean":"2024-04-25T09:59:47.566402+0000","last_became_active":"2024-04-25T09:59:46.561449+0000","last_became_peered":"2024-04-25T09:59:46.561449+0000","last_unstale":"2024-04-25T09:59:47.566402+0000","last_undegraded":"2024-04-25T09:59:47.566402+0000","last_fullsized":"2024-04-25T09:59:47.566402+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:16.801353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.572223+0000","last_change":"2024-04-25T09:59:47.569555+0000","last_active":"2024-04-25T09:59:47.572223+0000","last_peered":"2024-04-25T09:59:47.572223+0000","last_clean":"2024-04-25T09:59:47.572223+0000","last_became_active":"2024-04-25T09:59:47.569288+0000","last_became_peered":"2024-04-25T09:59:47.569288+0000","last_unstale":"2024-04-25T09:59:47.572223+0000","last_undegraded":"2024-04-25T09:59:47.572223+0000","last_fullsized":"2024-04-25T09:59:47.572223+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:42.798250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.588696+0000","last_change":"2024-04-25T09:59:46.559599+0000","last_active":"2024-04-25T09:59:47.588696+0000","last_peered":"2024-04-25T09:59:47.588696+0000","last_clean":"2024-04-25T09:59:47.588696+0000","last_became_active":"2024-04-25T09:59:46.559315+0000","last_became_peered":"2024-04-25T09:59:46.559315+0000","last_unstale":"2024-04-25T09:59:47.588696+0000","last_undegraded":"2024-04-25T09:59:47.588696+0000","last_fullsized":"2024-04-25T09:59:47.588696+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:36.137754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.570889+0000","last_change":"2024-04-25T09:59:47.570319+0000","last_active":"2024-04-25T09:59:48.570889+0000","last_peered":"2024-04-25T09:59:48.570889+0000","last_clean":"2024-04-25T09:59:48.570889+0000","last_became_active":"2024-04-25T09:59:47.570012+0000","last_became_peered":"2024-04-25T09:59:47.570012+0000","last_unstale":"2024-04-25T09:59:48.570889+0000","last_undegraded":"2024-04-25T09:59:48.570889+0000","last_fullsized":"2024-04-25T09:59:48.570889+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:11.334977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569862+0000","last_change":"2024-04-25T09:59:46.544164+0000","last_active":"2024-04-25T09:59:48.569862+0000","last_peered":"2024-04-25T09:59:48.569862+0000","last_clean":"2024-04-25T09:59:48.569862+0000","last_became_active":"2024-04-25T09:59:46.543861+0000","last_became_peered":"2024-04-25T09:59:46.543861+0000","last_unstale":"2024-04-25T09:59:48.569862+0000","last_undegraded":"2024-04-25T09:59:48.569862+0000","last_fullsized":"2024-04-25T09:59:48.569862+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:58.127110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569481+0000","last_change":"2024-04-25T09:59:47.567748+0000","last_active":"2024-04-25T09:59:48.569481+0000","last_peered":"2024-04-25T09:59:48.569481+0000","last_clean":"2024-04-25T09:59:48.569481+0000","last_became_active":"2024-04-25T09:59:47.567454+0000","last_became_peered":"2024-04-25T09:59:47.567454+0000","last_unstale":"2024-04-25T09:59:48.569481+0000","last_undegraded":"2024-04-25T09:59:48.569481+0000","last_fullsized":"2024-04-25T09:59:48.569481+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:47.522460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.565271+0000","last_change":"2024-04-25T09:59:46.557280+0000","last_active":"2024-04-25T09:59:47.565271+0000","last_peered":"2024-04-25T09:59:47.565271+0000","last_clean":"2024-04-25T09:59:47.565271+0000","last_became_active":"2024-04-25T09:59:46.556995+0000","last_became_peered":"2024-04-25T09:59:46.556995+0000","last_unstale":"2024-04-25T09:59:47.565271+0000","last_undegraded":"2024-04-25T09:59:47.565271+0000","last_fullsized":"2024-04-25T09:59:47.565271+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:14.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.587634+0000","last_change":"2024-04-25T09:59:47.587237+0000","last_active":"2024-04-25T09:59:47.587634+0000","last_peered":"2024-04-25T09:59:47.587634+0000","last_clean":"2024-04-25T09:59:47.587634+0000","last_became_active":"2024-04-25T09:59:47.586951+0000","last_became_peered":"2024-04-25T09:59:47.586951+0000","last_unstale":"2024-04-25T09:59:47.587634+0000","last_undegraded":"2024-04-25T09:59:47.587634+0000","last_fullsized":"2024-04-25T09:59:47.587634+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:13.998959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.565165+0000","last_change":"2024-04-25T09:59:46.559198+0000","last_active":"2024-04-25T09:59:47.565165+0000","last_peered":"2024-04-25T09:59:47.565165+0000","last_clean":"2024-04-25T09:59:47.565165+0000","last_became_active":"2024-04-25T09:59:46.558917+0000","last_became_peered":"2024-04-25T09:59:46.558917+0000","last_unstale":"2024-04-25T09:59:47.565165+0000","last_undegraded":"2024-04-25T09:59:47.565165+0000","last_fullsized":"2024-04-25T09:59:47.565165+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:05.341694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579296+0000","last_change":"2024-04-25T09:59:47.577779+0000","last_active":"2024-04-25T09:59:47.579296+0000","last_peered":"2024-04-25T09:59:47.579296+0000","last_clean":"2024-04-25T09:59:47.579296+0000","last_became_active":"2024-04-25T09:59:47.577385+0000","last_became_peered":"2024-04-25T09:59:47.577385+0000","last_unstale":"2024-04-25T09:59:47.579296+0000","last_undegraded":"2024-04-25T09:59:47.579296+0000","last_fullsized":"2024-04-25T09:59:47.579296+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:22.581651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571115+0000","last_change":"2024-04-25T09:59:46.558348+0000","last_active":"2024-04-25T09:59:48.571115+0000","last_peered":"2024-04-25T09:59:48.571115+0000","last_clean":"2024-04-25T09:59:48.571115+0000","last_became_active":"2024-04-25T09:59:46.557904+0000","last_became_peered":"2024-04-25T09:59:46.557904+0000","last_unstale":"2024-04-25T09:59:48.571115+0000","last_undegraded":"2024-04-25T09:59:48.571115+0000","last_fullsized":"2024-04-25T09:59:48.571115+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:36.758899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579946+0000","last_change":"2024-04-25T09:59:47.577960+0000","last_active":"2024-04-25T09:59:47.579946+0000","last_peered":"2024-04-25T09:59:47.579946+0000","last_clean":"2024-04-25T09:59:47.579946+0000","last_became_active":"2024-04-25T09:59:47.577274+0000","last_became_peered":"2024-04-25T09:59:47.577274+0000","last_unstale":"2024-04-25T09:59:47.579946+0000","last_undegraded":"2024-04-25T09:59:47.579946+0000","last_fullsized":"2024-04-25T09:59:47.579946+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:47.328142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.564840+0000","last_change":"2024-04-25T09:59:46.561665+0000","last_active":"2024-04-25T09:59:47.564840+0000","last_peered":"2024-04-25T09:59:47.564840+0000","last_clean":"2024-04-25T09:59:47.564840+0000","last_became_active":"2024-04-25T09:59:46.560895+0000","last_became_peered":"2024-04-25T09:59:46.560895+0000","last_unstale":"2024-04-25T09:59:47.564840+0000","last_undegraded":"2024-04-25T09:59:47.564840+0000","last_fullsized":"2024-04-25T09:59:47.564840+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:27.522902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571776+0000","last_change":"2024-04-25T09:59:47.572605+0000","last_active":"2024-04-25T09:59:48.571776+0000","last_peered":"2024-04-25T09:59:48.571776+0000","last_clean":"2024-04-25T09:59:48.571776+0000","last_became_active":"2024-04-25T09:59:47.572318+0000","last_became_peered":"2024-04-25T09:59:47.572318+0000","last_unstale":"2024-04-25T09:59:48.571776+0000","last_undegraded":"2024-04-25T09:59:48.571776+0000","last_fullsized":"2024-04-25T09:59:48.571776+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:22.684006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.565485+0000","last_change":"2024-04-25T09:59:46.558022+0000","last_active":"2024-04-25T09:59:47.565485+0000","last_peered":"2024-04-25T09:59:47.565485+0000","last_clean":"2024-04-25T09:59:47.565485+0000","last_became_active":"2024-04-25T09:59:46.557729+0000","last_became_peered":"2024-04-25T09:59:46.557729+0000","last_unstale":"2024-04-25T09:59:47.565485+0000","last_undegraded":"2024-04-25T09:59:47.565485+0000","last_fullsized":"2024-04-25T09:59:47.565485+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:11.966549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.572821+0000","last_change":"2024-04-25T09:59:47.571852+0000","last_active":"2024-04-25T09:59:47.572821+0000","last_peered":"2024-04-25T09:59:47.572821+0000","last_clean":"2024-04-25T09:59:47.572821+0000","last_became_active":"2024-04-25T09:59:47.571379+0000","last_became_peered":"2024-04-25T09:59:47.571379+0000","last_unstale":"2024-04-25T09:59:47.572821+0000","last_undegraded":"2024-04-25T09:59:47.572821+0000","last_fullsized":"2024-04-25T09:59:47.572821+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:11.334977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.566555+0000","last_change":"2024-04-25T09:59:46.562075+0000","last_active":"2024-04-25T09:59:47.566555+0000","last_peered":"2024-04-25T09:59:47.566555+0000","last_clean":"2024-04-25T09:59:47.566555+0000","last_became_active":"2024-04-25T09:59:46.561771+0000","last_became_peered":"2024-04-25T09:59:46.561771+0000","last_unstale":"2024-04-25T09:59:47.566555+0000","last_undegraded":"2024-04-25T09:59:47.566555+0000","last_fullsized":"2024-04-25T09:59:47.566555+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:36.314049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.572677+0000","last_change":"2024-04-25T09:59:47.570526+0000","last_active":"2024-04-25T09:59:47.572677+0000","last_peered":"2024-04-25T09:59:47.572677+0000","last_clean":"2024-04-25T09:59:47.572677+0000","last_became_active":"2024-04-25T09:59:47.570185+0000","last_became_peered":"2024-04-25T09:59:47.570185+0000","last_unstale":"2024-04-25T09:59:47.572677+0000","last_undegraded":"2024-04-25T09:59:47.572677+0000","last_fullsized":"2024-04-25T09:59:47.572677+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:47.379583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.590878+0000","last_change":"2024-04-25T09:59:46.560247+0000","last_active":"2024-04-25T09:59:47.590878+0000","last_peered":"2024-04-25T09:59:47.590878+0000","last_clean":"2024-04-25T09:59:47.590878+0000","last_became_active":"2024-04-25T09:59:46.559938+0000","last_became_peered":"2024-04-25T09:59:46.559938+0000","last_unstale":"2024-04-25T09:59:47.590878+0000","last_undegraded":"2024-04-25T09:59:47.590878+0000","last_fullsized":"2024-04-25T09:59:47.590878+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:36.632990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":345,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571106+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:59:48.571106+0000","last_peered":"2024-04-25T09:59:48.571106+0000","last_clean":"2024-04-25T09:59:48.571106+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:59:48.571106+0000","last_undegraded":"2024-04-25T09:59:48.571106+0000","last_fullsized":"2024-04-25T09:59:48.571106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41360,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330480,"statfs":{"total":96636764160,"available":96594411520,"internally_reserved":0,"allocated":1904640,"data_stored":1196476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40792,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331048,"statfs":{"total":96636764160,"available":96594993152,"internally_reserved":0,"allocated":1323008,"data_stored":608164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40780,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331060,"statfs":{"total":96636764160,"available":96595005440,"internally_reserved":0,"allocated":1302528,"data_stored":605948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50520,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321320,"statfs":{"total":96636764160,"available":96585031680,"internally_reserved":0,"allocated":1323008,"data_stored":612732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50520,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321320,"statfs":{"total":96636764160,"available":96585031680,"internally_reserved":0,"allocated":1318912,"data_stored":613222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50508,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321332,"statfs":{"total":96636764160,"available":96585043968,"internally_reserved":0,"allocated":1306624,"data_stored":605982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46344,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325496,"statfs":{"total":96636764160,"available":96589307904,"internally_reserved":0,"allocated":1306624,"data_stored":605982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46944,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324896,"statfs":{"total":96636764160,"available":96588693504,"internally_reserved":0,"allocated":1912832,"data_stored":1198018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:59:50.564 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:59:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.697+0000 7fb11625d640 1 Processor -- start 2024-04-25T09:59:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.697+0000 7fb11625d640 1 -- start start 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.697+0000 7fb11625d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb11015dbe0 0x7fb1100d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.697+0000 7fb11625d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb1100b2cb0 0x7fb1100d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.697+0000 7fb11625d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1100b97a0 con 0x7fb1100b92a0 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.697+0000 7fb11625d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11015dfc0 con 0x7fb1100b2cb0 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.697+0000 7fb11625d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100d1a00 con 0x7fb11015dbe0 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.698+0000 7fb10effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb1100b2cb0 0x7fb1100d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.698+0000 7fb10ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb1100b92a0 0x7fb1100b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35132/0 (socket says 172.21.15.19:35132) 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.698+0000 7fb10ffff640 1 -- 172.21.15.19:0/4008686558 learned_addr learned my addr 172.21.15.19:0/4008686558 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.698+0000 7fb10f7fe640 1 --2- 172.21.15.19:0/4008686558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb11015dbe0 0x7fb1100d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.698+0000 7fb10effd640 1 -- 172.21.15.19:0/4008686558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb11015dbe0 msgr2=0x7fb1100d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.698+0000 7fb10effd640 1 --2- 172.21.15.19:0/4008686558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb11015dbe0 0x7fb1100d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.698+0000 7fb10effd640 1 -- 172.21.15.19:0/4008686558 >> v1:172.21.15.19:6789/0 conn(0x7fb1100b92a0 legacy=0x7fb1100b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.698+0000 7fb10effd640 1 -- 172.21.15.19:0/4008686558 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1100d1c50 con 0x7fb1100b2cb0 2024-04-25T09:59:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb10effd640 1 --2- 172.21.15.19:0/4008686558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1100b2cb0 0x7fb1100d0e50 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fb1040097a0 tx=0x7fb10402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7983e1779cac2ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb10e7fc640 1 -- 172.21.15.19:0/4008686558 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104037070 con 0x7fb1100b2cb0 2024-04-25T09:59:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb10e7fc640 1 -- 172.21.15.19:0/4008686558 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb104032440 con 0x7fb1100b2cb0 2024-04-25T09:59:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb10e7fc640 1 -- 172.21.15.19:0/4008686558 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104032810 con 0x7fb1100b2cb0 2024-04-25T09:59:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb11625d640 1 -- 172.21.15.19:0/4008686558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1100b2cb0 msgr2=0x7fb1100d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb11625d640 1 --2- 172.21.15.19:0/4008686558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1100b2cb0 0x7fb1100d0e50 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fb1040097a0 tx=0x7fb10402a180 comp rx=0 tx=0).stop 2024-04-25T09:59:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb11625d640 1 -- 172.21.15.19:0/4008686558 shutdown_connections 2024-04-25T09:59:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb11625d640 1 --2- 172.21.15.19:0/4008686558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1100b2cb0 0x7fb1100d0e50 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb11625d640 1 --2- 172.21.15.19:0/4008686558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb11015dbe0 0x7fb1100d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb11625d640 1 -- 172.21.15.19:0/4008686558 >> 172.21.15.19:0/4008686558 conn(0x7fb110061a20 msgr2=0x7fb110061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb11625d640 1 -- 172.21.15.19:0/4008686558 shutdown_connections 2024-04-25T09:59:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.699+0000 7fb11625d640 1 -- 172.21.15.19:0/4008686558 wait complete. 2024-04-25T09:59:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.700+0000 7fb11625d640 1 Processor -- start 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.701+0000 7fb11625d640 1 -- start start 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.701+0000 7fb11625d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1100b2cb0 0x7fb110160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.701+0000 7fb11625d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1100b92a0 0x7fb11015f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.701+0000 7fb11625d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb11015dbe0 0x7fb110143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.701+0000 7fb11625d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100b6ab0 con 0x7fb11015dbe0 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.701+0000 7fb11625d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100d2210 con 0x7fb1100b2cb0 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.701+0000 7fb11625d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100b6c30 con 0x7fb1100b92a0 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.701+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb11015dbe0 0x7fb110143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb11015dbe0 0x7fb110143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38570/0 (socket says 172.21.15.19:38570) 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10ffff640 1 -- 172.21.15.19:0/1937908412 learned_addr learned my addr 172.21.15.19:0/1937908412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10effd640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1100b92a0 0x7fb11015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10f7fe640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1100b2cb0 0x7fb110160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10ffff640 1 -- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1100b92a0 msgr2=0x7fb11015f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10ffff640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1100b92a0 0x7fb11015f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10ffff640 1 -- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1100b2cb0 msgr2=0x7fb110160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10ffff640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1100b2cb0 0x7fb110160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10ffff640 1 -- 172.21.15.19:0/1937908412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11015fac0 con 0x7fb11015dbe0 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10effd640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1100b92a0 0x7fb11015f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10f7fe640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1100b2cb0 0x7fb110160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb10ffff640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb11015dbe0 0x7fb110143b90 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb10800ca60 tx=0x7fb10800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb0f7fff640 1 -- 172.21.15.19:0/1937908412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108017070 con 0x7fb11015dbe0 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb0f7fff640 1 -- 172.21.15.19:0/1937908412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10800ab70 con 0x7fb11015dbe0 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb11625d640 1 -- 172.21.15.19:0/1937908412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11015fda0 con 0x7fb11015dbe0 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb0f7fff640 1 -- 172.21.15.19:0/1937908412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108012400 con 0x7fb11015dbe0 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.702+0000 7fb11625d640 1 -- 172.21.15.19:0/1937908412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb110160360 con 0x7fb11015dbe0 2024-04-25T09:59:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.703+0000 7fb11625d640 1 -- 172.21.15.19:0/1937908412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb110073990 con 0x7fb11015dbe0 2024-04-25T09:59:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.703+0000 7fb0f7fff640 1 -- 172.21.15.19:0/1937908412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb10801a070 con 0x7fb11015dbe0 2024-04-25T09:59:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.704+0000 7fb0f7fff640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0fc06b4d0 0x7fb0fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.704+0000 7fb0f7fff640 1 -- 172.21.15.19:0/1937908412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb10808a0b0 con 0x7fb11015dbe0 2024-04-25T09:59:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.704+0000 7fb10f7fe640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0fc06b4d0 0x7fb0fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.705+0000 7fb10f7fe640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0fc06b4d0 0x7fb0fc06d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb100006fd0 tx=0x7fb100008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.708+0000 7fb0f7fff640 1 -- 172.21.15.19:0/1937908412 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb108058d00 con 0x7fb11015dbe0 2024-04-25T09:59:50.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.884+0000 7fb11625d640 1 -- 172.21.15.19:0/1937908412 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1100b62e0 con 0x7fb0fc06b4d0 2024-04-25T09:59:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.891+0000 7fb0f7fff640 1 -- 172.21.15.19:0/1937908412 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252434 (secure 0 0 0) 0x7fb1100b62e0 con 0x7fb0fc06b4d0 2024-04-25T09:59:50.897 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T09:59:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 -- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0fc06b4d0 msgr2=0x7fb0fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0fc06b4d0 0x7fb0fc06d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb100006fd0 tx=0x7fb100008040 comp rx=0 tx=0).stop 2024-04-25T09:59:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 -- 172.21.15.19:0/1937908412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb11015dbe0 msgr2=0x7fb110143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb11015dbe0 0x7fb110143b90 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb10800ca60 tx=0x7fb10800cf20 comp rx=0 tx=0).stop 2024-04-25T09:59:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 -- 172.21.15.19:0/1937908412 shutdown_connections 2024-04-25T09:59:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb0fc06b4d0 0x7fb0fc06d990 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb11015dbe0 0x7fb110143b90 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1100b92a0 0x7fb11015f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 --2- 172.21.15.19:0/1937908412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1100b2cb0 0x7fb110160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 -- 172.21.15.19:0/1937908412 >> 172.21.15.19:0/1937908412 conn(0x7fb110061a20 msgr2=0x7fb11015ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 -- 172.21.15.19:0/1937908412 shutdown_connections 2024-04-25T09:59:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:50.894+0000 7fb11625d640 1 -- 172.21.15.19:0/1937908412 wait complete. 2024-04-25T09:59:50.908 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-04-25T09:59:49.003316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":367768,"kb_used_data":11424,"kb_used_omap":12,"kb_used_meta":356211,"kb_avail":754606952,"statfs":{"total":773094113280,"available":772717518848,"internally_reserved":0,"allocated":11698176,"data_stored":6046524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":364760626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000882"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.571890+0000","last_change":"2024-04-25T09:59:47.568633+0000","last_active":"2024-04-25T09:59:47.571890+0000","last_peered":"2024-04-25T09:59:47.571890+0000","last_clean":"2024-04-25T09:59:47.571890+0000","last_became_active":"2024-04-25T09:59:47.568342+0000","last_became_peered":"2024-04-25T09:59:47.568342+0000","last_unstale":"2024-04-25T09:59:47.571890+0000","last_undegraded":"2024-04-25T09:59:47.571890+0000","last_fullsized":"2024-04-25T09:59:47.571890+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:08.652513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.572833+0000","last_change":"2024-04-25T09:59:47.565668+0000","last_active":"2024-04-25T09:59:47.572833+0000","last_peered":"2024-04-25T09:59:47.572833+0000","last_clean":"2024-04-25T09:59:47.572833+0000","last_became_active":"2024-04-25T09:59:47.565208+0000","last_became_peered":"2024-04-25T09:59:47.565208+0000","last_unstale":"2024-04-25T09:59:47.572833+0000","last_undegraded":"2024-04-25T09:59:47.572833+0000","last_fullsized":"2024-04-25T09:59:47.572833+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:37.762977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.574185+0000","last_change":"2024-04-25T09:59:47.573625+0000","last_active":"2024-04-25T09:59:47.574185+0000","last_peered":"2024-04-25T09:59:47.574185+0000","last_clean":"2024-04-25T09:59:47.574185+0000","last_became_active":"2024-04-25T09:59:47.573329+0000","last_became_peered":"2024-04-25T09:59:47.573329+0000","last_unstale":"2024-04-25T09:59:47.574185+0000","last_undegraded":"2024-04-25T09:59:47.574185+0000","last_fullsized":"2024-04-25T09:59:47.574185+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:50.259657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569753+0000","last_change":"2024-04-25T09:59:47.571166+0000","last_active":"2024-04-25T09:59:48.569753+0000","last_peered":"2024-04-25T09:59:48.569753+0000","last_clean":"2024-04-25T09:59:48.569753+0000","last_became_active":"2024-04-25T09:59:47.570602+0000","last_became_peered":"2024-04-25T09:59:47.570602+0000","last_unstale":"2024-04-25T09:59:48.569753+0000","last_undegraded":"2024-04-25T09:59:48.569753+0000","last_fullsized":"2024-04-25T09:59:48.569753+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:15.406180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.575937+0000","last_change":"2024-04-25T09:59:47.573759+0000","last_active":"2024-04-25T09:59:47.575937+0000","last_peered":"2024-04-25T09:59:47.575937+0000","last_clean":"2024-04-25T09:59:47.575937+0000","last_became_active":"2024-04-25T09:59:47.573469+0000","last_became_peered":"2024-04-25T09:59:47.573469+0000","last_unstale":"2024-04-25T09:59:47.575937+0000","last_undegraded":"2024-04-25T09:59:47.575937+0000","last_fullsized":"2024-04-25T09:59:47.575937+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.584375+0000","last_change":"2024-04-25T09:59:47.581066+0000","last_active":"2024-04-25T09:59:47.584375+0000","last_peered":"2024-04-25T09:59:47.584375+0000","last_clean":"2024-04-25T09:59:47.584375+0000","last_became_active":"2024-04-25T09:59:47.580786+0000","last_became_peered":"2024-04-25T09:59:47.580786+0000","last_unstale":"2024-04-25T09:59:47.584375+0000","last_undegraded":"2024-04-25T09:59:47.584375+0000","last_fullsized":"2024-04-25T09:59:47.584375+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:41.641450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586996+0000","last_change":"2024-04-25T09:59:47.582629+0000","last_active":"2024-04-25T09:59:47.586996+0000","last_peered":"2024-04-25T09:59:47.586996+0000","last_clean":"2024-04-25T09:59:47.586996+0000","last_became_active":"2024-04-25T09:59:47.581549+0000","last_became_peered":"2024-04-25T09:59:47.581549+0000","last_unstale":"2024-04-25T09:59:47.586996+0000","last_undegraded":"2024-04-25T09:59:47.586996+0000","last_fullsized":"2024-04-25T09:59:47.586996+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:47.379583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.589927+0000","last_change":"2024-04-25T09:59:47.588123+0000","last_active":"2024-04-25T09:59:47.589927+0000","last_peered":"2024-04-25T09:59:47.589927+0000","last_clean":"2024-04-25T09:59:47.589927+0000","last_became_active":"2024-04-25T09:59:47.587863+0000","last_became_peered":"2024-04-25T09:59:47.587863+0000","last_unstale":"2024-04-25T09:59:47.589927+0000","last_undegraded":"2024-04-25T09:59:47.589927+0000","last_fullsized":"2024-04-25T09:59:47.589927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:29.649101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573586+0000","last_change":"2024-04-25T09:59:47.569623+0000","last_active":"2024-04-25T09:59:47.573586+0000","last_peered":"2024-04-25T09:59:47.573586+0000","last_clean":"2024-04-25T09:59:47.573586+0000","last_became_active":"2024-04-25T09:59:47.568827+0000","last_became_peered":"2024-04-25T09:59:47.568827+0000","last_unstale":"2024-04-25T09:59:47.573586+0000","last_undegraded":"2024-04-25T09:59:47.573586+0000","last_fullsized":"2024-04-25T09:59:47.573586+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:20.884665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586665+0000","last_change":"2024-04-25T09:59:47.582003+0000","last_active":"2024-04-25T09:59:47.586665+0000","last_peered":"2024-04-25T09:59:47.586665+0000","last_clean":"2024-04-25T09:59:47.586665+0000","last_became_active":"2024-04-25T09:59:47.581718+0000","last_became_peered":"2024-04-25T09:59:47.581718+0000","last_unstale":"2024-04-25T09:59:47.586665+0000","last_undegraded":"2024-04-25T09:59:47.586665+0000","last_fullsized":"2024-04-25T09:59:47.586665+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:41.641450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.577474+0000","last_change":"2024-04-25T09:59:47.576496+0000","last_active":"2024-04-25T09:59:47.577474+0000","last_peered":"2024-04-25T09:59:47.577474+0000","last_clean":"2024-04-25T09:59:47.577474+0000","last_became_active":"2024-04-25T09:59:47.575888+0000","last_became_peered":"2024-04-25T09:59:47.575888+0000","last_unstale":"2024-04-25T09:59:47.577474+0000","last_undegraded":"2024-04-25T09:59:47.577474+0000","last_fullsized":"2024-04-25T09:59:47.577474+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:10.120007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571491+0000","last_change":"2024-04-25T09:59:47.566465+0000","last_active":"2024-04-25T09:59:48.571491+0000","last_peered":"2024-04-25T09:59:48.571491+0000","last_clean":"2024-04-25T09:59:48.571491+0000","last_became_active":"2024-04-25T09:59:47.566170+0000","last_became_peered":"2024-04-25T09:59:47.566170+0000","last_unstale":"2024-04-25T09:59:48.571491+0000","last_undegraded":"2024-04-25T09:59:48.571491+0000","last_fullsized":"2024-04-25T09:59:48.571491+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:10.120007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569446+0000","last_change":"2024-04-25T09:59:47.572484+0000","last_active":"2024-04-25T09:59:48.569446+0000","last_peered":"2024-04-25T09:59:48.569446+0000","last_clean":"2024-04-25T09:59:48.569446+0000","last_became_active":"2024-04-25T09:59:47.571791+0000","last_became_peered":"2024-04-25T09:59:47.571791+0000","last_unstale":"2024-04-25T09:59:48.569446+0000","last_undegraded":"2024-04-25T09:59:48.569446+0000","last_fullsized":"2024-04-25T09:59:48.569446+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:21.208467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571388+0000","last_change":"2024-04-25T09:59:47.568591+0000","last_active":"2024-04-25T09:59:48.571388+0000","last_peered":"2024-04-25T09:59:48.571388+0000","last_clean":"2024-04-25T09:59:48.571388+0000","last_became_active":"2024-04-25T09:59:47.568317+0000","last_became_peered":"2024-04-25T09:59:47.568317+0000","last_unstale":"2024-04-25T09:59:48.571388+0000","last_undegraded":"2024-04-25T09:59:48.571388+0000","last_fullsized":"2024-04-25T09:59:48.571388+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:59.271674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.570977+0000","last_change":"2024-04-25T09:59:47.573408+0000","last_active":"2024-04-25T09:59:48.570977+0000","last_peered":"2024-04-25T09:59:48.570977+0000","last_clean":"2024-04-25T09:59:48.570977+0000","last_became_active":"2024-04-25T09:59:47.573053+0000","last_became_peered":"2024-04-25T09:59:47.573053+0000","last_unstale":"2024-04-25T09:59:48.570977+0000","last_undegraded":"2024-04-25T09:59:48.570977+0000","last_fullsized":"2024-04-25T09:59:48.570977+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:20.286657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571047+0000","last_change":"2024-04-25T09:59:47.578390+0000","last_active":"2024-04-25T09:59:48.571047+0000","last_peered":"2024-04-25T09:59:48.571047+0000","last_clean":"2024-04-25T09:59:48.571047+0000","last_became_active":"2024-04-25T09:59:47.578053+0000","last_became_peered":"2024-04-25T09:59:47.578053+0000","last_unstale":"2024-04-25T09:59:48.571047+0000","last_undegraded":"2024-04-25T09:59:48.571047+0000","last_fullsized":"2024-04-25T09:59:48.571047+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:25.819100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.576867+0000","last_change":"2024-04-25T09:59:47.575824+0000","last_active":"2024-04-25T09:59:47.576867+0000","last_peered":"2024-04-25T09:59:47.576867+0000","last_clean":"2024-04-25T09:59:47.576867+0000","last_became_active":"2024-04-25T09:59:47.574522+0000","last_became_peered":"2024-04-25T09:59:47.574522+0000","last_unstale":"2024-04-25T09:59:47.576867+0000","last_undegraded":"2024-04-25T09:59:47.576867+0000","last_fullsized":"2024-04-25T09:59:47.576867+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:20.069929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586706+0000","last_change":"2024-04-25T09:59:47.582917+0000","last_active":"2024-04-25T09:59:47.586706+0000","last_peered":"2024-04-25T09:59:47.586706+0000","last_clean":"2024-04-25T09:59:47.586706+0000","last_became_active":"2024-04-25T09:59:47.582618+0000","last_became_peered":"2024-04-25T09:59:47.582618+0000","last_unstale":"2024-04-25T09:59:47.586706+0000","last_undegraded":"2024-04-25T09:59:47.586706+0000","last_fullsized":"2024-04-25T09:59:47.586706+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:11.334977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573908+0000","last_change":"2024-04-25T09:59:47.571000+0000","last_active":"2024-04-25T09:59:47.573908+0000","last_peered":"2024-04-25T09:59:47.573908+0000","last_clean":"2024-04-25T09:59:47.573908+0000","last_became_active":"2024-04-25T09:59:47.570352+0000","last_became_peered":"2024-04-25T09:59:47.570352+0000","last_unstale":"2024-04-25T09:59:47.573908+0000","last_undegraded":"2024-04-25T09:59:47.573908+0000","last_fullsized":"2024-04-25T09:59:47.573908+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:22.082005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571643+0000","last_change":"2024-04-25T09:59:47.565164+0000","last_active":"2024-04-25T09:59:48.571643+0000","last_peered":"2024-04-25T09:59:48.571643+0000","last_clean":"2024-04-25T09:59:48.571643+0000","last_became_active":"2024-04-25T09:59:47.564866+0000","last_became_peered":"2024-04-25T09:59:47.564866+0000","last_unstale":"2024-04-25T09:59:48.571643+0000","last_undegraded":"2024-04-25T09:59:48.571643+0000","last_fullsized":"2024-04-25T09:59:48.571643+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:13.998959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579393+0000","last_change":"2024-04-25T09:59:47.576249+0000","last_active":"2024-04-25T09:59:47.579393+0000","last_peered":"2024-04-25T09:59:47.579393+0000","last_clean":"2024-04-25T09:59:47.579393+0000","last_became_active":"2024-04-25T09:59:47.575932+0000","last_became_peered":"2024-04-25T09:59:47.575932+0000","last_unstale":"2024-04-25T09:59:47.579393+0000","last_undegraded":"2024-04-25T09:59:47.579393+0000","last_fullsized":"2024-04-25T09:59:47.579393+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:26.135650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.587131+0000","last_change":"2024-04-25T09:59:47.584187+0000","last_active":"2024-04-25T09:59:47.587131+0000","last_peered":"2024-04-25T09:59:47.587131+0000","last_clean":"2024-04-25T09:59:47.587131+0000","last_became_active":"2024-04-25T09:59:47.583039+0000","last_became_peered":"2024-04-25T09:59:47.583039+0000","last_unstale":"2024-04-25T09:59:47.587131+0000","last_undegraded":"2024-04-25T09:59:47.587131+0000","last_fullsized":"2024-04-25T09:59:47.587131+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:50.977149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.575440+0000","last_change":"2024-04-25T09:59:47.573685+0000","last_active":"2024-04-25T09:59:47.575440+0000","last_peered":"2024-04-25T09:59:47.575440+0000","last_clean":"2024-04-25T09:59:47.575440+0000","last_became_active":"2024-04-25T09:59:47.573406+0000","last_became_peered":"2024-04-25T09:59:47.573406+0000","last_unstale":"2024-04-25T09:59:47.575440+0000","last_undegraded":"2024-04-25T09:59:47.575440+0000","last_fullsized":"2024-04-25T09:59:47.575440+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:10.018295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586735+0000","last_change":"2024-04-25T09:59:47.582096+0000","last_active":"2024-04-25T09:59:47.586735+0000","last_peered":"2024-04-25T09:59:47.586735+0000","last_clean":"2024-04-25T09:59:47.586735+0000","last_became_active":"2024-04-25T09:59:47.573784+0000","last_became_peered":"2024-04-25T09:59:47.573784+0000","last_unstale":"2024-04-25T09:59:47.586735+0000","last_undegraded":"2024-04-25T09:59:47.586735+0000","last_fullsized":"2024-04-25T09:59:47.586735+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:11.334977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571730+0000","last_change":"2024-04-25T09:59:47.573537+0000","last_active":"2024-04-25T09:59:48.571730+0000","last_peered":"2024-04-25T09:59:48.571730+0000","last_clean":"2024-04-25T09:59:48.571730+0000","last_became_active":"2024-04-25T09:59:47.572762+0000","last_became_peered":"2024-04-25T09:59:47.572762+0000","last_unstale":"2024-04-25T09:59:48.571730+0000","last_undegraded":"2024-04-25T09:59:48.571730+0000","last_fullsized":"2024-04-25T09:59:48.571730+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:32.607621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571375+0000","last_change":"2024-04-25T09:59:47.568385+0000","last_active":"2024-04-25T09:59:48.571375+0000","last_peered":"2024-04-25T09:59:48.571375+0000","last_clean":"2024-04-25T09:59:48.571375+0000","last_became_active":"2024-04-25T09:59:47.568087+0000","last_became_peered":"2024-04-25T09:59:47.568087+0000","last_unstale":"2024-04-25T09:59:48.571375+0000","last_undegraded":"2024-04-25T09:59:48.571375+0000","last_fullsized":"2024-04-25T09:59:48.571375+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:48.997774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.571516+0000","last_change":"2024-04-25T09:59:47.570263+0000","last_active":"2024-04-25T09:59:47.571516+0000","last_peered":"2024-04-25T09:59:47.571516+0000","last_clean":"2024-04-25T09:59:47.571516+0000","last_became_active":"2024-04-25T09:59:47.569540+0000","last_became_peered":"2024-04-25T09:59:47.569540+0000","last_unstale":"2024-04-25T09:59:47.571516+0000","last_undegraded":"2024-04-25T09:59:47.571516+0000","last_fullsized":"2024-04-25T09:59:47.571516+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:41.641450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.577878+0000","last_change":"2024-04-25T09:59:47.575640+0000","last_active":"2024-04-25T09:59:47.577878+0000","last_peered":"2024-04-25T09:59:47.577878+0000","last_clean":"2024-04-25T09:59:47.577878+0000","last_became_active":"2024-04-25T09:59:47.575199+0000","last_became_peered":"2024-04-25T09:59:47.575199+0000","last_unstale":"2024-04-25T09:59:47.577878+0000","last_undegraded":"2024-04-25T09:59:47.577878+0000","last_fullsized":"2024-04-25T09:59:47.577878+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:06.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579370+0000","last_change":"2024-04-25T09:59:47.576121+0000","last_active":"2024-04-25T09:59:47.579370+0000","last_peered":"2024-04-25T09:59:47.579370+0000","last_clean":"2024-04-25T09:59:47.579370+0000","last_became_active":"2024-04-25T09:59:47.575537+0000","last_became_peered":"2024-04-25T09:59:47.575537+0000","last_unstale":"2024-04-25T09:59:47.579370+0000","last_undegraded":"2024-04-25T09:59:47.579370+0000","last_fullsized":"2024-04-25T09:59:47.579370+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:28.503272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571946+0000","last_change":"2024-04-25T09:59:47.572253+0000","last_active":"2024-04-25T09:59:48.571946+0000","last_peered":"2024-04-25T09:59:48.571946+0000","last_clean":"2024-04-25T09:59:48.571946+0000","last_became_active":"2024-04-25T09:59:47.571973+0000","last_became_peered":"2024-04-25T09:59:47.571973+0000","last_unstale":"2024-04-25T09:59:48.571946+0000","last_undegraded":"2024-04-25T09:59:48.571946+0000","last_fullsized":"2024-04-25T09:59:48.571946+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:40.175103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.574766+0000","last_change":"2024-04-25T09:59:47.571757+0000","last_active":"2024-04-25T09:59:47.574766+0000","last_peered":"2024-04-25T09:59:47.574766+0000","last_clean":"2024-04-25T09:59:47.574766+0000","last_became_active":"2024-04-25T09:59:47.570664+0000","last_became_peered":"2024-04-25T09:59:47.570664+0000","last_unstale":"2024-04-25T09:59:47.574766+0000","last_undegraded":"2024-04-25T09:59:47.574766+0000","last_fullsized":"2024-04-25T09:59:47.574766+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.407473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579910+0000","last_change":"2024-04-25T09:59:47.578705+0000","last_active":"2024-04-25T09:59:47.579910+0000","last_peered":"2024-04-25T09:59:47.579910+0000","last_clean":"2024-04-25T09:59:47.579910+0000","last_became_active":"2024-04-25T09:59:47.577826+0000","last_became_peered":"2024-04-25T09:59:47.577826+0000","last_unstale":"2024-04-25T09:59:47.579910+0000","last_undegraded":"2024-04-25T09:59:47.579910+0000","last_fullsized":"2024-04-25T09:59:47.579910+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:44.017705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.571841+0000","last_change":"2024-04-25T09:59:47.568929+0000","last_active":"2024-04-25T09:59:47.571841+0000","last_peered":"2024-04-25T09:59:47.571841+0000","last_clean":"2024-04-25T09:59:47.571841+0000","last_became_active":"2024-04-25T09:59:47.567921+0000","last_became_peered":"2024-04-25T09:59:47.567921+0000","last_unstale":"2024-04-25T09:59:47.571841+0000","last_undegraded":"2024-04-25T09:59:47.571841+0000","last_fullsized":"2024-04-25T09:59:47.571841+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:00.118302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.570023+0000","last_change":"2024-04-25T09:59:46.553598+0000","last_active":"2024-04-25T09:59:48.570023+0000","last_peered":"2024-04-25T09:59:48.570023+0000","last_clean":"2024-04-25T09:59:48.570023+0000","last_became_active":"2024-04-25T09:59:46.553335+0000","last_became_peered":"2024-04-25T09:59:46.553335+0000","last_unstale":"2024-04-25T09:59:48.570023+0000","last_undegraded":"2024-04-25T09:59:48.570023+0000","last_fullsized":"2024-04-25T09:59:48.570023+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:09.115929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579325+0000","last_change":"2024-04-25T09:59:47.577578+0000","last_active":"2024-04-25T09:59:47.579325+0000","last_peered":"2024-04-25T09:59:47.579325+0000","last_clean":"2024-04-25T09:59:47.579325+0000","last_became_active":"2024-04-25T09:59:47.577267+0000","last_became_peered":"2024-04-25T09:59:47.577267+0000","last_unstale":"2024-04-25T09:59:47.579325+0000","last_undegraded":"2024-04-25T09:59:47.579325+0000","last_fullsized":"2024-04-25T09:59:47.579325+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:04.405045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.591261+0000","last_change":"2024-04-25T09:59:46.548719+0000","last_active":"2024-04-25T09:59:47.591261+0000","last_peered":"2024-04-25T09:59:47.591261+0000","last_clean":"2024-04-25T09:59:47.591261+0000","last_became_active":"2024-04-25T09:59:46.548431+0000","last_became_peered":"2024-04-25T09:59:46.548431+0000","last_unstale":"2024-04-25T09:59:47.591261+0000","last_undegraded":"2024-04-25T09:59:47.591261+0000","last_fullsized":"2024-04-25T09:59:47.591261+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:04.524363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571299+0000","last_change":"2024-04-25T09:59:47.574967+0000","last_active":"2024-04-25T09:59:48.571299+0000","last_peered":"2024-04-25T09:59:48.571299+0000","last_clean":"2024-04-25T09:59:48.571299+0000","last_became_active":"2024-04-25T09:59:47.574044+0000","last_became_peered":"2024-04-25T09:59:47.574044+0000","last_unstale":"2024-04-25T09:59:48.571299+0000","last_undegraded":"2024-04-25T09:59:48.571299+0000","last_fullsized":"2024-04-25T09:59:48.571299+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:50.977149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.572018+0000","last_change":"2024-04-25T09:59:46.557167+0000","last_active":"2024-04-25T09:59:48.572018+0000","last_peered":"2024-04-25T09:59:48.572018+0000","last_clean":"2024-04-25T09:59:48.572018+0000","last_became_active":"2024-04-25T09:59:46.556864+0000","last_became_peered":"2024-04-25T09:59:46.556864+0000","last_unstale":"2024-04-25T09:59:48.572018+0000","last_undegraded":"2024-04-25T09:59:48.572018+0000","last_fullsized":"2024-04-25T09:59:48.572018+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:45.239556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.576264+0000","last_change":"2024-04-25T09:59:47.574559+0000","last_active":"2024-04-25T09:59:47.576264+0000","last_peered":"2024-04-25T09:59:47.576264+0000","last_clean":"2024-04-25T09:59:47.576264+0000","last_became_active":"2024-04-25T09:59:47.574247+0000","last_became_peered":"2024-04-25T09:59:47.574247+0000","last_unstale":"2024-04-25T09:59:47.576264+0000","last_undegraded":"2024-04-25T09:59:47.576264+0000","last_fullsized":"2024-04-25T09:59:47.576264+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:48.997774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.587470+0000","last_change":"2024-04-25T09:59:46.561068+0000","last_active":"2024-04-25T09:59:47.587470+0000","last_peered":"2024-04-25T09:59:47.587470+0000","last_clean":"2024-04-25T09:59:47.587470+0000","last_became_active":"2024-04-25T09:59:46.560492+0000","last_became_peered":"2024-04-25T09:59:46.560492+0000","last_unstale":"2024-04-25T09:59:47.587470+0000","last_undegraded":"2024-04-25T09:59:47.587470+0000","last_fullsized":"2024-04-25T09:59:47.587470+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:45.239556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569777+0000","last_change":"2024-04-25T09:59:47.581294+0000","last_active":"2024-04-25T09:59:48.569777+0000","last_peered":"2024-04-25T09:59:48.569777+0000","last_clean":"2024-04-25T09:59:48.569777+0000","last_became_active":"2024-04-25T09:59:47.581002+0000","last_became_peered":"2024-04-25T09:59:47.581002+0000","last_unstale":"2024-04-25T09:59:48.569777+0000","last_undegraded":"2024-04-25T09:59:48.569777+0000","last_fullsized":"2024-04-25T09:59:48.569777+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:20.286657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571761+0000","last_change":"2024-04-25T09:59:46.550941+0000","last_active":"2024-04-25T09:59:48.571761+0000","last_peered":"2024-04-25T09:59:48.571761+0000","last_clean":"2024-04-25T09:59:48.571761+0000","last_became_active":"2024-04-25T09:59:46.550575+0000","last_became_peered":"2024-04-25T09:59:46.550575+0000","last_unstale":"2024-04-25T09:59:48.571761+0000","last_undegraded":"2024-04-25T09:59:48.571761+0000","last_fullsized":"2024-04-25T09:59:48.571761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:26.799473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573890+0000","last_change":"2024-04-25T09:59:47.570624+0000","last_active":"2024-04-25T09:59:47.573890+0000","last_peered":"2024-04-25T09:59:47.573890+0000","last_clean":"2024-04-25T09:59:47.573890+0000","last_became_active":"2024-04-25T09:59:47.569985+0000","last_became_peered":"2024-04-25T09:59:47.569985+0000","last_unstale":"2024-04-25T09:59:47.573890+0000","last_undegraded":"2024-04-25T09:59:47.573890+0000","last_fullsized":"2024-04-25T09:59:47.573890+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:33.869614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.582408+0000","last_change":"2024-04-25T09:59:46.554738+0000","last_active":"2024-04-25T09:59:47.582408+0000","last_peered":"2024-04-25T09:59:47.582408+0000","last_clean":"2024-04-25T09:59:47.582408+0000","last_became_active":"2024-04-25T09:59:46.554280+0000","last_became_peered":"2024-04-25T09:59:46.554280+0000","last_unstale":"2024-04-25T09:59:47.582408+0000","last_undegraded":"2024-04-25T09:59:47.582408+0000","last_fullsized":"2024-04-25T09:59:47.582408+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:05.609216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.575662+0000","last_change":"2024-04-25T09:59:47.574359+0000","last_active":"2024-04-25T09:59:47.575662+0000","last_peered":"2024-04-25T09:59:47.575662+0000","last_clean":"2024-04-25T09:59:47.575662+0000","last_became_active":"2024-04-25T09:59:47.574069+0000","last_became_peered":"2024-04-25T09:59:47.574069+0000","last_unstale":"2024-04-25T09:59:47.575662+0000","last_undegraded":"2024-04-25T09:59:47.575662+0000","last_fullsized":"2024-04-25T09:59:47.575662+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:36.170205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571052+0000","last_change":"2024-04-25T09:59:46.552210+0000","last_active":"2024-04-25T09:59:48.571052+0000","last_peered":"2024-04-25T09:59:48.571052+0000","last_clean":"2024-04-25T09:59:48.571052+0000","last_became_active":"2024-04-25T09:59:46.551764+0000","last_became_peered":"2024-04-25T09:59:46.551764+0000","last_unstale":"2024-04-25T09:59:48.571052+0000","last_undegraded":"2024-04-25T09:59:48.571052+0000","last_fullsized":"2024-04-25T09:59:48.571052+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:02.605211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569927+0000","last_change":"2024-04-25T09:59:47.572553+0000","last_active":"2024-04-25T09:59:48.569927+0000","last_peered":"2024-04-25T09:59:48.569927+0000","last_clean":"2024-04-25T09:59:48.569927+0000","last_became_active":"2024-04-25T09:59:47.572272+0000","last_became_peered":"2024-04-25T09:59:47.572272+0000","last_unstale":"2024-04-25T09:59:48.569927+0000","last_undegraded":"2024-04-25T09:59:48.569927+0000","last_fullsized":"2024-04-25T09:59:48.569927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:25.819100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569340+0000","last_change":"2024-04-25T09:59:46.560393+0000","last_active":"2024-04-25T09:59:48.569340+0000","last_peered":"2024-04-25T09:59:48.569340+0000","last_clean":"2024-04-25T09:59:48.569340+0000","last_became_active":"2024-04-25T09:59:46.560107+0000","last_became_peered":"2024-04-25T09:59:46.560107+0000","last_unstale":"2024-04-25T09:59:48.569340+0000","last_undegraded":"2024-04-25T09:59:48.569340+0000","last_fullsized":"2024-04-25T09:59:48.569340+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:47.993697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.572243+0000","last_change":"2024-04-25T09:59:47.570071+0000","last_active":"2024-04-25T09:59:47.572243+0000","last_peered":"2024-04-25T09:59:47.572243+0000","last_clean":"2024-04-25T09:59:47.572243+0000","last_became_active":"2024-04-25T09:59:47.568308+0000","last_became_peered":"2024-04-25T09:59:47.568308+0000","last_unstale":"2024-04-25T09:59:47.572243+0000","last_undegraded":"2024-04-25T09:59:47.572243+0000","last_fullsized":"2024-04-25T09:59:47.572243+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:37.523348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571824+0000","last_change":"2024-04-25T09:59:46.553828+0000","last_active":"2024-04-25T09:59:48.571824+0000","last_peered":"2024-04-25T09:59:48.571824+0000","last_clean":"2024-04-25T09:59:48.571824+0000","last_became_active":"2024-04-25T09:59:46.553567+0000","last_became_peered":"2024-04-25T09:59:46.553567+0000","last_unstale":"2024-04-25T09:59:48.571824+0000","last_undegraded":"2024-04-25T09:59:48.571824+0000","last_fullsized":"2024-04-25T09:59:48.571824+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:36.137754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586702+0000","last_change":"2024-04-25T09:59:47.582086+0000","last_active":"2024-04-25T09:59:47.586702+0000","last_peered":"2024-04-25T09:59:47.586702+0000","last_clean":"2024-04-25T09:59:47.586702+0000","last_became_active":"2024-04-25T09:59:47.581709+0000","last_became_peered":"2024-04-25T09:59:47.581709+0000","last_unstale":"2024-04-25T09:59:47.586702+0000","last_undegraded":"2024-04-25T09:59:47.586702+0000","last_fullsized":"2024-04-25T09:59:47.586702+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:47.379583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.554973+0000","last_change":"2024-04-25T09:59:46.553720+0000","last_active":"2024-04-25T09:59:47.554973+0000","last_peered":"2024-04-25T09:59:47.554973+0000","last_clean":"2024-04-25T09:59:47.554973+0000","last_became_active":"2024-04-25T09:59:46.553457+0000","last_became_peered":"2024-04-25T09:59:46.553457+0000","last_unstale":"2024-04-25T09:59:47.554973+0000","last_undegraded":"2024-04-25T09:59:47.554973+0000","last_fullsized":"2024-04-25T09:59:47.554973+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:32.975313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.588938+0000","last_change":"2024-04-25T09:59:47.587730+0000","last_active":"2024-04-25T09:59:47.588938+0000","last_peered":"2024-04-25T09:59:47.588938+0000","last_clean":"2024-04-25T09:59:47.588938+0000","last_became_active":"2024-04-25T09:59:47.587455+0000","last_became_peered":"2024-04-25T09:59:47.587455+0000","last_unstale":"2024-04-25T09:59:47.588938+0000","last_undegraded":"2024-04-25T09:59:47.588938+0000","last_fullsized":"2024-04-25T09:59:47.588938+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:16.168496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571813+0000","last_change":"2024-04-25T09:59:46.547192+0000","last_active":"2024-04-25T09:59:48.571813+0000","last_peered":"2024-04-25T09:59:48.571813+0000","last_clean":"2024-04-25T09:59:48.571813+0000","last_became_active":"2024-04-25T09:59:46.545232+0000","last_became_peered":"2024-04-25T09:59:46.545232+0000","last_unstale":"2024-04-25T09:59:48.571813+0000","last_undegraded":"2024-04-25T09:59:48.571813+0000","last_fullsized":"2024-04-25T09:59:48.571813+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:00.126112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586288+0000","last_change":"2024-04-25T09:59:47.580730+0000","last_active":"2024-04-25T09:59:47.586288+0000","last_peered":"2024-04-25T09:59:47.586288+0000","last_clean":"2024-04-25T09:59:47.586288+0000","last_became_active":"2024-04-25T09:59:47.580423+0000","last_became_peered":"2024-04-25T09:59:47.580423+0000","last_unstale":"2024-04-25T09:59:47.586288+0000","last_undegraded":"2024-04-25T09:59:47.586288+0000","last_fullsized":"2024-04-25T09:59:47.586288+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:06.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.590617+0000","last_change":"2024-04-25T09:59:46.556227+0000","last_active":"2024-04-25T09:59:47.590617+0000","last_peered":"2024-04-25T09:59:47.590617+0000","last_clean":"2024-04-25T09:59:47.590617+0000","last_became_active":"2024-04-25T09:59:46.555907+0000","last_became_peered":"2024-04-25T09:59:46.555907+0000","last_unstale":"2024-04-25T09:59:47.590617+0000","last_undegraded":"2024-04-25T09:59:47.590617+0000","last_fullsized":"2024-04-25T09:59:47.590617+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:47.928616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573503+0000","last_change":"2024-04-25T09:59:47.572164+0000","last_active":"2024-04-25T09:59:47.573503+0000","last_peered":"2024-04-25T09:59:47.573503+0000","last_clean":"2024-04-25T09:59:47.573503+0000","last_became_active":"2024-04-25T09:59:47.571869+0000","last_became_peered":"2024-04-25T09:59:47.571869+0000","last_unstale":"2024-04-25T09:59:47.573503+0000","last_undegraded":"2024-04-25T09:59:47.573503+0000","last_fullsized":"2024-04-25T09:59:47.573503+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:50.977149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.588946+0000","last_change":"2024-04-25T09:59:46.558626+0000","last_active":"2024-04-25T09:59:47.588946+0000","last_peered":"2024-04-25T09:59:47.588946+0000","last_clean":"2024-04-25T09:59:47.588946+0000","last_became_active":"2024-04-25T09:59:46.558303+0000","last_became_peered":"2024-04-25T09:59:46.558303+0000","last_unstale":"2024-04-25T09:59:47.588946+0000","last_undegraded":"2024-04-25T09:59:47.588946+0000","last_fullsized":"2024-04-25T09:59:47.588946+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:19.880588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573993+0000","last_change":"2024-04-25T09:59:47.568843+0000","last_active":"2024-04-25T09:59:47.573993+0000","last_peered":"2024-04-25T09:59:47.573993+0000","last_clean":"2024-04-25T09:59:47.573993+0000","last_became_active":"2024-04-25T09:59:47.568326+0000","last_became_peered":"2024-04-25T09:59:47.568326+0000","last_unstale":"2024-04-25T09:59:47.573993+0000","last_undegraded":"2024-04-25T09:59:47.573993+0000","last_fullsized":"2024-04-25T09:59:47.573993+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:46.243634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569456+0000","last_change":"2024-04-25T09:59:46.546542+0000","last_active":"2024-04-25T09:59:48.569456+0000","last_peered":"2024-04-25T09:59:48.569456+0000","last_clean":"2024-04-25T09:59:48.569456+0000","last_became_active":"2024-04-25T09:59:46.546242+0000","last_became_peered":"2024-04-25T09:59:46.546242+0000","last_unstale":"2024-04-25T09:59:48.569456+0000","last_undegraded":"2024-04-25T09:59:48.569456+0000","last_fullsized":"2024-04-25T09:59:48.569456+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:49.973071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.577744+0000","last_change":"2024-04-25T09:59:47.575505+0000","last_active":"2024-04-25T09:59:47.577744+0000","last_peered":"2024-04-25T09:59:47.577744+0000","last_clean":"2024-04-25T09:59:47.577744+0000","last_became_active":"2024-04-25T09:59:47.575197+0000","last_became_peered":"2024-04-25T09:59:47.575197+0000","last_unstale":"2024-04-25T09:59:47.577744+0000","last_undegraded":"2024-04-25T09:59:47.577744+0000","last_fullsized":"2024-04-25T09:59:47.577744+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:31.448089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.562838+0000","last_change":"2024-04-25T09:59:46.560493+0000","last_active":"2024-04-25T09:59:47.562838+0000","last_peered":"2024-04-25T09:59:47.562838+0000","last_clean":"2024-04-25T09:59:47.562838+0000","last_became_active":"2024-04-25T09:59:46.560168+0000","last_became_peered":"2024-04-25T09:59:46.560168+0000","last_unstale":"2024-04-25T09:59:47.562838+0000","last_undegraded":"2024-04-25T09:59:47.562838+0000","last_fullsized":"2024-04-25T09:59:47.562838+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:39.594594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.576888+0000","last_change":"2024-04-25T09:59:47.576087+0000","last_active":"2024-04-25T09:59:47.576888+0000","last_peered":"2024-04-25T09:59:47.576888+0000","last_clean":"2024-04-25T09:59:47.576888+0000","last_became_active":"2024-04-25T09:59:47.574638+0000","last_became_peered":"2024-04-25T09:59:47.574638+0000","last_unstale":"2024-04-25T09:59:47.576888+0000","last_undegraded":"2024-04-25T09:59:47.576888+0000","last_fullsized":"2024-04-25T09:59:47.576888+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:12.602237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.570759+0000","last_change":"2024-04-25T09:59:46.556274+0000","last_active":"2024-04-25T09:59:48.570759+0000","last_peered":"2024-04-25T09:59:48.570759+0000","last_clean":"2024-04-25T09:59:48.570759+0000","last_became_active":"2024-04-25T09:59:46.556000+0000","last_became_peered":"2024-04-25T09:59:46.556000+0000","last_unstale":"2024-04-25T09:59:48.570759+0000","last_undegraded":"2024-04-25T09:59:48.570759+0000","last_fullsized":"2024-04-25T09:59:48.570759+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:11.966549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579931+0000","last_change":"2024-04-25T09:59:47.578895+0000","last_active":"2024-04-25T09:59:47.579931+0000","last_peered":"2024-04-25T09:59:47.579931+0000","last_clean":"2024-04-25T09:59:47.579931+0000","last_became_active":"2024-04-25T09:59:47.578093+0000","last_became_peered":"2024-04-25T09:59:47.578093+0000","last_unstale":"2024-04-25T09:59:47.579931+0000","last_undegraded":"2024-04-25T09:59:47.579931+0000","last_fullsized":"2024-04-25T09:59:47.579931+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:36.203126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.586793+0000","last_change":"2024-04-25T09:59:46.559170+0000","last_active":"2024-04-25T09:59:47.586793+0000","last_peered":"2024-04-25T09:59:47.586793+0000","last_clean":"2024-04-25T09:59:47.586793+0000","last_became_active":"2024-04-25T09:59:46.558877+0000","last_became_peered":"2024-04-25T09:59:46.558877+0000","last_unstale":"2024-04-25T09:59:47.586793+0000","last_undegraded":"2024-04-25T09:59:47.586793+0000","last_fullsized":"2024-04-25T09:59:47.586793+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:36.137754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.573991+0000","last_change":"2024-04-25T09:59:47.568967+0000","last_active":"2024-04-25T09:59:47.573991+0000","last_peered":"2024-04-25T09:59:47.573991+0000","last_clean":"2024-04-25T09:59:47.573991+0000","last_became_active":"2024-04-25T09:59:47.567986+0000","last_became_peered":"2024-04-25T09:59:47.567986+0000","last_unstale":"2024-04-25T09:59:47.573991+0000","last_undegraded":"2024-04-25T09:59:47.573991+0000","last_fullsized":"2024-04-25T09:59:47.573991+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:06.345771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571913+0000","last_change":"2024-04-25T09:59:46.559231+0000","last_active":"2024-04-25T09:59:48.571913+0000","last_peered":"2024-04-25T09:59:48.571913+0000","last_clean":"2024-04-25T09:59:48.571913+0000","last_became_active":"2024-04-25T09:59:46.558836+0000","last_became_peered":"2024-04-25T09:59:46.558836+0000","last_unstale":"2024-04-25T09:59:48.571913+0000","last_undegraded":"2024-04-25T09:59:48.571913+0000","last_fullsized":"2024-04-25T09:59:48.571913+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:51.194047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.582753+0000","last_change":"2024-04-25T09:59:47.575787+0000","last_active":"2024-04-25T09:59:47.582753+0000","last_peered":"2024-04-25T09:59:47.582753+0000","last_clean":"2024-04-25T09:59:47.582753+0000","last_became_active":"2024-04-25T09:59:47.572949+0000","last_became_peered":"2024-04-25T09:59:47.572949+0000","last_unstale":"2024-04-25T09:59:47.582753+0000","last_undegraded":"2024-04-25T09:59:47.582753+0000","last_fullsized":"2024-04-25T09:59:47.582753+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:36.170205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.568172+0000","last_change":"2024-04-25T09:59:46.553605+0000","last_active":"2024-04-25T09:59:47.568172+0000","last_peered":"2024-04-25T09:59:47.568172+0000","last_clean":"2024-04-25T09:59:47.568172+0000","last_became_active":"2024-04-25T09:59:46.553338+0000","last_became_peered":"2024-04-25T09:59:46.553338+0000","last_unstale":"2024-04-25T09:59:47.568172+0000","last_undegraded":"2024-04-25T09:59:47.568172+0000","last_fullsized":"2024-04-25T09:59:47.568172+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:35.166127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.569139+0000","last_change":"2024-04-25T09:59:47.566375+0000","last_active":"2024-04-25T09:59:47.569139+0000","last_peered":"2024-04-25T09:59:47.569139+0000","last_clean":"2024-04-25T09:59:47.569139+0000","last_became_active":"2024-04-25T09:59:47.566109+0000","last_became_peered":"2024-04-25T09:59:47.566109+0000","last_unstale":"2024-04-25T09:59:47.569139+0000","last_undegraded":"2024-04-25T09:59:47.569139+0000","last_fullsized":"2024-04-25T09:59:47.569139+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:49.066163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.591013+0000","last_change":"2024-04-25T09:59:46.556827+0000","last_active":"2024-04-25T09:59:47.591013+0000","last_peered":"2024-04-25T09:59:47.591013+0000","last_clean":"2024-04-25T09:59:47.591013+0000","last_became_active":"2024-04-25T09:59:46.556393+0000","last_became_peered":"2024-04-25T09:59:46.556393+0000","last_unstale":"2024-04-25T09:59:47.591013+0000","last_undegraded":"2024-04-25T09:59:47.591013+0000","last_fullsized":"2024-04-25T09:59:47.591013+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:35.940120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.576514+0000","last_change":"2024-04-25T09:59:47.575238+0000","last_active":"2024-04-25T09:59:47.576514+0000","last_peered":"2024-04-25T09:59:47.576514+0000","last_clean":"2024-04-25T09:59:47.576514+0000","last_became_active":"2024-04-25T09:59:47.574541+0000","last_became_peered":"2024-04-25T09:59:47.574541+0000","last_unstale":"2024-04-25T09:59:47.576514+0000","last_undegraded":"2024-04-25T09:59:47.576514+0000","last_fullsized":"2024-04-25T09:59:47.576514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:30.031204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.564666+0000","last_change":"2024-04-25T09:59:46.561911+0000","last_active":"2024-04-25T09:59:47.564666+0000","last_peered":"2024-04-25T09:59:47.564666+0000","last_clean":"2024-04-25T09:59:47.564666+0000","last_became_active":"2024-04-25T09:59:46.561335+0000","last_became_peered":"2024-04-25T09:59:46.561335+0000","last_unstale":"2024-04-25T09:59:47.564666+0000","last_undegraded":"2024-04-25T09:59:47.564666+0000","last_fullsized":"2024-04-25T09:59:47.564666+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:59.217379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569741+0000","last_change":"2024-04-25T09:59:47.569180+0000","last_active":"2024-04-25T09:59:48.569741+0000","last_peered":"2024-04-25T09:59:48.569741+0000","last_clean":"2024-04-25T09:59:48.569741+0000","last_became_active":"2024-04-25T09:59:47.568833+0000","last_became_peered":"2024-04-25T09:59:47.568833+0000","last_unstale":"2024-04-25T09:59:48.569741+0000","last_undegraded":"2024-04-25T09:59:48.569741+0000","last_fullsized":"2024-04-25T09:59:48.569741+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:40.267433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.565343+0000","last_change":"2024-04-25T09:59:46.557402+0000","last_active":"2024-04-25T09:59:47.565343+0000","last_peered":"2024-04-25T09:59:47.565343+0000","last_clean":"2024-04-25T09:59:47.565343+0000","last_became_active":"2024-04-25T09:59:46.556975+0000","last_became_peered":"2024-04-25T09:59:46.556975+0000","last_unstale":"2024-04-25T09:59:47.565343+0000","last_undegraded":"2024-04-25T09:59:47.565343+0000","last_fullsized":"2024-04-25T09:59:47.565343+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:02.605211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579368+0000","last_change":"2024-04-25T09:59:47.578193+0000","last_active":"2024-04-25T09:59:47.579368+0000","last_peered":"2024-04-25T09:59:47.579368+0000","last_clean":"2024-04-25T09:59:47.579368+0000","last_became_active":"2024-04-25T09:59:47.577892+0000","last_became_peered":"2024-04-25T09:59:47.577892+0000","last_unstale":"2024-04-25T09:59:47.579368+0000","last_undegraded":"2024-04-25T09:59:47.579368+0000","last_fullsized":"2024-04-25T09:59:47.579368+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:32.907293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.566402+0000","last_change":"2024-04-25T09:59:46.561750+0000","last_active":"2024-04-25T09:59:47.566402+0000","last_peered":"2024-04-25T09:59:47.566402+0000","last_clean":"2024-04-25T09:59:47.566402+0000","last_became_active":"2024-04-25T09:59:46.561449+0000","last_became_peered":"2024-04-25T09:59:46.561449+0000","last_unstale":"2024-04-25T09:59:47.566402+0000","last_undegraded":"2024-04-25T09:59:47.566402+0000","last_fullsized":"2024-04-25T09:59:47.566402+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:16.801353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.572223+0000","last_change":"2024-04-25T09:59:47.569555+0000","last_active":"2024-04-25T09:59:47.572223+0000","last_peered":"2024-04-25T09:59:47.572223+0000","last_clean":"2024-04-25T09:59:47.572223+0000","last_became_active":"2024-04-25T09:59:47.569288+0000","last_became_peered":"2024-04-25T09:59:47.569288+0000","last_unstale":"2024-04-25T09:59:47.572223+0000","last_undegraded":"2024-04-25T09:59:47.572223+0000","last_fullsized":"2024-04-25T09:59:47.572223+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:42.798250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.588696+0000","last_change":"2024-04-25T09:59:46.559599+0000","last_active":"2024-04-25T09:59:47.588696+0000","last_peered":"2024-04-25T09:59:47.588696+0000","last_clean":"2024-04-25T09:59:47.588696+0000","last_became_active":"2024-04-25T09:59:46.559315+0000","last_became_peered":"2024-04-25T09:59:46.559315+0000","last_unstale":"2024-04-25T09:59:47.588696+0000","last_undegraded":"2024-04-25T09:59:47.588696+0000","last_fullsized":"2024-04-25T09:59:47.588696+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:36.137754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.570889+0000","last_change":"2024-04-25T09:59:47.570319+0000","last_active":"2024-04-25T09:59:48.570889+0000","last_peered":"2024-04-25T09:59:48.570889+0000","last_clean":"2024-04-25T09:59:48.570889+0000","last_became_active":"2024-04-25T09:59:47.570012+0000","last_became_peered":"2024-04-25T09:59:47.570012+0000","last_unstale":"2024-04-25T09:59:48.570889+0000","last_undegraded":"2024-04-25T09:59:48.570889+0000","last_fullsized":"2024-04-25T09:59:48.570889+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:11.334977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569862+0000","last_change":"2024-04-25T09:59:46.544164+0000","last_active":"2024-04-25T09:59:48.569862+0000","last_peered":"2024-04-25T09:59:48.569862+0000","last_clean":"2024-04-25T09:59:48.569862+0000","last_became_active":"2024-04-25T09:59:46.543861+0000","last_became_peered":"2024-04-25T09:59:46.543861+0000","last_unstale":"2024-04-25T09:59:48.569862+0000","last_undegraded":"2024-04-25T09:59:48.569862+0000","last_fullsized":"2024-04-25T09:59:48.569862+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:58.127110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.569481+0000","last_change":"2024-04-25T09:59:47.567748+0000","last_active":"2024-04-25T09:59:48.569481+0000","last_peered":"2024-04-25T09:59:48.569481+0000","last_clean":"2024-04-25T09:59:48.569481+0000","last_became_active":"2024-04-25T09:59:47.567454+0000","last_became_peered":"2024-04-25T09:59:47.567454+0000","last_unstale":"2024-04-25T09:59:48.569481+0000","last_undegraded":"2024-04-25T09:59:48.569481+0000","last_fullsized":"2024-04-25T09:59:48.569481+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:47.522460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.565271+0000","last_change":"2024-04-25T09:59:46.557280+0000","last_active":"2024-04-25T09:59:47.565271+0000","last_peered":"2024-04-25T09:59:47.565271+0000","last_clean":"2024-04-25T09:59:47.565271+0000","last_became_active":"2024-04-25T09:59:46.556995+0000","last_became_peered":"2024-04-25T09:59:46.556995+0000","last_unstale":"2024-04-25T09:59:47.565271+0000","last_undegraded":"2024-04-25T09:59:47.565271+0000","last_fullsized":"2024-04-25T09:59:47.565271+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:14.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.587634+0000","last_change":"2024-04-25T09:59:47.587237+0000","last_active":"2024-04-25T09:59:47.587634+0000","last_peered":"2024-04-25T09:59:47.587634+0000","last_clean":"2024-04-25T09:59:47.587634+0000","last_became_active":"2024-04-25T09:59:47.586951+0000","last_became_peered":"2024-04-25T09:59:47.586951+0000","last_unstale":"2024-04-25T09:59:47.587634+0000","last_undegraded":"2024-04-25T09:59:47.587634+0000","last_fullsized":"2024-04-25T09:59:47.587634+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:13.998959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.565165+0000","last_change":"2024-04-25T09:59:46.559198+0000","last_active":"2024-04-25T09:59:47.565165+0000","last_peered":"2024-04-25T09:59:47.565165+0000","last_clean":"2024-04-25T09:59:47.565165+0000","last_became_active":"2024-04-25T09:59:46.558917+0000","last_became_peered":"2024-04-25T09:59:46.558917+0000","last_unstale":"2024-04-25T09:59:47.565165+0000","last_undegraded":"2024-04-25T09:59:47.565165+0000","last_fullsized":"2024-04-25T09:59:47.565165+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:05.341694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579296+0000","last_change":"2024-04-25T09:59:47.577779+0000","last_active":"2024-04-25T09:59:47.579296+0000","last_peered":"2024-04-25T09:59:47.579296+0000","last_clean":"2024-04-25T09:59:47.579296+0000","last_became_active":"2024-04-25T09:59:47.577385+0000","last_became_peered":"2024-04-25T09:59:47.577385+0000","last_unstale":"2024-04-25T09:59:47.579296+0000","last_undegraded":"2024-04-25T09:59:47.579296+0000","last_fullsized":"2024-04-25T09:59:47.579296+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:22.581651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571115+0000","last_change":"2024-04-25T09:59:46.558348+0000","last_active":"2024-04-25T09:59:48.571115+0000","last_peered":"2024-04-25T09:59:48.571115+0000","last_clean":"2024-04-25T09:59:48.571115+0000","last_became_active":"2024-04-25T09:59:46.557904+0000","last_became_peered":"2024-04-25T09:59:46.557904+0000","last_unstale":"2024-04-25T09:59:48.571115+0000","last_undegraded":"2024-04-25T09:59:48.571115+0000","last_fullsized":"2024-04-25T09:59:48.571115+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:36.758899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.579946+0000","last_change":"2024-04-25T09:59:47.577960+0000","last_active":"2024-04-25T09:59:47.579946+0000","last_peered":"2024-04-25T09:59:47.579946+0000","last_clean":"2024-04-25T09:59:47.579946+0000","last_became_active":"2024-04-25T09:59:47.577274+0000","last_became_peered":"2024-04-25T09:59:47.577274+0000","last_unstale":"2024-04-25T09:59:47.579946+0000","last_undegraded":"2024-04-25T09:59:47.579946+0000","last_fullsized":"2024-04-25T09:59:47.579946+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:47.328142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.564840+0000","last_change":"2024-04-25T09:59:46.561665+0000","last_active":"2024-04-25T09:59:47.564840+0000","last_peered":"2024-04-25T09:59:47.564840+0000","last_clean":"2024-04-25T09:59:47.564840+0000","last_became_active":"2024-04-25T09:59:46.560895+0000","last_became_peered":"2024-04-25T09:59:46.560895+0000","last_unstale":"2024-04-25T09:59:47.564840+0000","last_undegraded":"2024-04-25T09:59:47.564840+0000","last_fullsized":"2024-04-25T09:59:47.564840+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:27.522902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571776+0000","last_change":"2024-04-25T09:59:47.572605+0000","last_active":"2024-04-25T09:59:48.571776+0000","last_peered":"2024-04-25T09:59:48.571776+0000","last_clean":"2024-04-25T09:59:48.571776+0000","last_became_active":"2024-04-25T09:59:47.572318+0000","last_became_peered":"2024-04-25T09:59:47.572318+0000","last_unstale":"2024-04-25T09:59:48.571776+0000","last_undegraded":"2024-04-25T09:59:48.571776+0000","last_fullsized":"2024-04-25T09:59:48.571776+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:22.684006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.565485+0000","last_change":"2024-04-25T09:59:46.558022+0000","last_active":"2024-04-25T09:59:47.565485+0000","last_peered":"2024-04-25T09:59:47.565485+0000","last_clean":"2024-04-25T09:59:47.565485+0000","last_became_active":"2024-04-25T09:59:46.557729+0000","last_became_peered":"2024-04-25T09:59:46.557729+0000","last_unstale":"2024-04-25T09:59:47.565485+0000","last_undegraded":"2024-04-25T09:59:47.565485+0000","last_fullsized":"2024-04-25T09:59:47.565485+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:11.966549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.572821+0000","last_change":"2024-04-25T09:59:47.571852+0000","last_active":"2024-04-25T09:59:47.572821+0000","last_peered":"2024-04-25T09:59:47.572821+0000","last_clean":"2024-04-25T09:59:47.572821+0000","last_became_active":"2024-04-25T09:59:47.571379+0000","last_became_peered":"2024-04-25T09:59:47.571379+0000","last_unstale":"2024-04-25T09:59:47.572821+0000","last_undegraded":"2024-04-25T09:59:47.572821+0000","last_fullsized":"2024-04-25T09:59:47.572821+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:11.334977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.566555+0000","last_change":"2024-04-25T09:59:46.562075+0000","last_active":"2024-04-25T09:59:47.566555+0000","last_peered":"2024-04-25T09:59:47.566555+0000","last_clean":"2024-04-25T09:59:47.566555+0000","last_became_active":"2024-04-25T09:59:46.561771+0000","last_became_peered":"2024-04-25T09:59:46.561771+0000","last_unstale":"2024-04-25T09:59:47.566555+0000","last_undegraded":"2024-04-25T09:59:47.566555+0000","last_fullsized":"2024-04-25T09:59:47.566555+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:36.314049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.572677+0000","last_change":"2024-04-25T09:59:47.570526+0000","last_active":"2024-04-25T09:59:47.572677+0000","last_peered":"2024-04-25T09:59:47.572677+0000","last_clean":"2024-04-25T09:59:47.572677+0000","last_became_active":"2024-04-25T09:59:47.570185+0000","last_became_peered":"2024-04-25T09:59:47.570185+0000","last_unstale":"2024-04-25T09:59:47.572677+0000","last_undegraded":"2024-04-25T09:59:47.572677+0000","last_fullsized":"2024-04-25T09:59:47.572677+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:46.535345+0000","last_clean_scrub_stamp":"2024-04-25T09:59:46.535345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:47.379583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T09:59:47.590878+0000","last_change":"2024-04-25T09:59:46.560247+0000","last_active":"2024-04-25T09:59:47.590878+0000","last_peered":"2024-04-25T09:59:47.590878+0000","last_clean":"2024-04-25T09:59:47.590878+0000","last_became_active":"2024-04-25T09:59:46.559938+0000","last_became_peered":"2024-04-25T09:59:46.559938+0000","last_unstale":"2024-04-25T09:59:47.590878+0000","last_undegraded":"2024-04-25T09:59:47.590878+0000","last_fullsized":"2024-04-25T09:59:47.590878+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:59:45.531268+0000","last_clean_scrub_stamp":"2024-04-25T09:59:45.531268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:36.632990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":345,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T09:59:48.571106+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T09:59:48.571106+0000","last_peered":"2024-04-25T09:59:48.571106+0000","last_clean":"2024-04-25T09:59:48.571106+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T09:59:48.571106+0000","last_undegraded":"2024-04-25T09:59:48.571106+0000","last_fullsized":"2024-04-25T09:59:48.571106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41360,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330480,"statfs":{"total":96636764160,"available":96594411520,"internally_reserved":0,"allocated":1904640,"data_stored":1196476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40792,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331048,"statfs":{"total":96636764160,"available":96594993152,"internally_reserved":0,"allocated":1323008,"data_stored":608164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40780,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331060,"statfs":{"total":96636764160,"available":96595005440,"internally_reserved":0,"allocated":1302528,"data_stored":605948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50520,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321320,"statfs":{"total":96636764160,"available":96585031680,"internally_reserved":0,"allocated":1323008,"data_stored":612732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50520,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321320,"statfs":{"total":96636764160,"available":96585031680,"internally_reserved":0,"allocated":1318912,"data_stored":613222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50508,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321332,"statfs":{"total":96636764160,"available":96585043968,"internally_reserved":0,"allocated":1306624,"data_stored":605982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46344,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325496,"statfs":{"total":96636764160,"available":96589307904,"internally_reserved":0,"allocated":1306624,"data_stored":605982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46944,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324896,"statfs":{"total":96636764160,"available":96588693504,"internally_reserved":0,"allocated":1912832,"data_stored":1198018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:59:50.912 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:59:50.913 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:59:51.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.053+0000 7f3d9dab3640 1 Processor -- start 2024-04-25T09:59:51.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.053+0000 7f3d9dab3640 1 -- start start 2024-04-25T09:59:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.054+0000 7f3d9dab3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3d9815b7d0 0x7f3d980cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.054+0000 7f3d9dab3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9806cf50 con 0x7f3d9806c9e0 2024-04-25T09:59:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.054+0000 7f3d9dab3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d980ce530 con 0x7f3d980b5620 2024-04-25T09:59:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.054+0000 7f3d9dab3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d980ce810 con 0x7f3d9815b7d0 2024-04-25T09:59:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.054+0000 7f3d97fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3d9806c9e0 0x7f3d980d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35148/0 (socket says 172.21.15.19:35148) 2024-04-25T09:59:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.054+0000 7f3d97fff640 1 -- 172.21.15.19:0/1525922974 learned_addr learned my addr 172.21.15.19:0/1525922974 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.054+0000 7f3d977fe640 1 --2- 172.21.15.19:0/1525922974 >> v2:172.21.15.112:3300/0 conn(0x7f3d9815b7d0 0x7f3d980cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.054+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592358090 0 0) 0x7f3d9806cf50 con 0x7f3d9806c9e0 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.054+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d80003520 con 0x7f3d9806c9e0 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.054+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538846147 0 0) 0x7f3d980ce530 con 0x7f3d980b5620 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d9806cf50 con 0x7f3d980b5620 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1958676180 0 0) 0x7f3d80003520 con 0x7f3d9806c9e0 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 >> v2:172.21.15.112:3300/0 conn(0x7f3d9815b7d0 msgr2=0x7f3d980cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 --2- 172.21.15.19:0/1525922974 >> v2:172.21.15.112:3300/0 conn(0x7f3d9815b7d0 0x7f3d980cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 >> v1:172.21.15.112:6789/0 conn(0x7f3d980b5620 legacy=0x7f3d9806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d980ceaf0 con 0x7f3d9806c9e0 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f3d900019f0 con 0x7f3d9806c9e0 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 >> v1:172.21.15.19:6789/0 conn(0x7f3d9806c9e0 legacy=0x7f3d980d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 --2- 172.21.15.19:0/1525922974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d80003b90 0x7f3d80003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 --2- 172.21.15.19:0/1525922974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d80004800 0x7f3d80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 --2- 172.21.15.19:0/1525922974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d800051c0 0x7f3d8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980ce810 con 0x7f3d80003b90 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8000c090 con 0x7f3d80004800 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8000c290 con 0x7f3d800051c0 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d977fe640 1 --2- 172.21.15.19:0/1525922974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d80003b90 0x7f3d80003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d97fff640 1 --2- 172.21.15.19:0/1525922974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d800051c0 0x7f3d8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d96ffd640 1 --2- 172.21.15.19:0/1525922974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d80004800 0x7f3d80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d977fe640 1 -- 172.21.15.19:0/1525922974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d800051c0 msgr2=0x7f3d8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d977fe640 1 --2- 172.21.15.19:0/1525922974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d800051c0 0x7f3d8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d977fe640 1 -- 172.21.15.19:0/1525922974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d80004800 msgr2=0x7f3d80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d977fe640 1 --2- 172.21.15.19:0/1525922974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d80004800 0x7f3d80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d97fff640 1 -- 172.21.15.19:0/1525922974 reap_dead start 2024-04-25T09:59:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.055+0000 7f3d977fe640 1 -- 172.21.15.19:0/1525922974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d80003520 con 0x7f3d80003b90 2024-04-25T09:59:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.056+0000 7f3d977fe640 1 --2- 172.21.15.19:0/1525922974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d80003b90 0x7f3d80003f90 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009ac0 tx=0x7f3d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c605db8f3d0efff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.056+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88036070 con 0x7f3d80003b90 2024-04-25T09:59:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.056+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d88031400 con 0x7f3d80003b90 2024-04-25T09:59:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.056+0000 7f3d967fc640 1 -- 172.21.15.19:0/1525922974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88009690 con 0x7f3d80003b90 2024-04-25T09:59:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.056+0000 7f3d9dab3640 1 -- 172.21.15.19:0/1525922974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d80003b90 msgr2=0x7f3d80003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.056+0000 7f3d9dab3640 1 --2- 172.21.15.19:0/1525922974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d80003b90 0x7f3d80003f90 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009ac0 tx=0x7f3d80005830 comp rx=0 tx=0).stop 2024-04-25T09:59:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.057+0000 7f3d9dab3640 1 -- 172.21.15.19:0/1525922974 shutdown_connections 2024-04-25T09:59:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.057+0000 7f3d9dab3640 1 --2- 172.21.15.19:0/1525922974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d80003b90 0x7f3d80003f90 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.057+0000 7f3d9dab3640 1 -- 172.21.15.19:0/1525922974 >> 172.21.15.19:0/1525922974 conn(0x7f3d980d0800 msgr2=0x7f3d980d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.057+0000 7f3d9dab3640 1 -- 172.21.15.19:0/1525922974 shutdown_connections 2024-04-25T09:59:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.057+0000 7f3d9dab3640 1 -- 172.21.15.19:0/1525922974 wait complete. 2024-04-25T09:59:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.058+0000 7f3d9dab3640 1 Processor -- start 2024-04-25T09:59:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.058+0000 7f3d9dab3640 1 -- start start 2024-04-25T09:59:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d9dab3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d80003b90 0x7f3d9814c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d9dab3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9814c890 0x7f3d980c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d9dab3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d980c8c90 0x7f3d9815cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d9dab3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9815bd30 con 0x7f3d9814c890 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d9dab3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9815bbb0 con 0x7f3d80003b90 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d9dab3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980b5ab0 con 0x7f3d980c8c90 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9814c890 0x7f3d980c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d96ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9814c890 0x7f3d980c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38584/0 (socket says 172.21.15.19:38584) 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d96ffd640 1 -- 172.21.15.19:0/316232751 learned_addr learned my addr 172.21.15.19:0/316232751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d97fff640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d980c8c90 0x7f3d9815cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d977fe640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d80003b90 0x7f3d9814c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d96ffd640 1 -- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d980c8c90 msgr2=0x7f3d9815cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d96ffd640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d980c8c90 0x7f3d9815cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d96ffd640 1 -- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d80003b90 msgr2=0x7f3d9814c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d96ffd640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d80003b90 0x7f3d9814c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d96ffd640 1 -- 172.21.15.19:0/316232751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9814cd60 con 0x7f3d9814c890 2024-04-25T09:59:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d97fff640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d980c8c90 0x7f3d9815cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d977fe640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d80003b90 0x7f3d9814c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d96ffd640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9814c890 0x7f3d980c8750 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00f7a0 tx=0x7f3d8c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.059+0000 7f3d7ffff640 1 -- 172.21.15.19:0/316232751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c01a070 con 0x7f3d9814c890 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.060+0000 7f3d7ffff640 1 -- 172.21.15.19:0/316232751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c005320 con 0x7f3d9814c890 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.060+0000 7f3d9dab3640 1 -- 172.21.15.19:0/316232751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9814d040 con 0x7f3d9814c890 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.060+0000 7f3d7ffff640 1 -- 172.21.15.19:0/316232751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c007270 con 0x7f3d9814c890 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.060+0000 7f3d9dab3640 1 -- 172.21.15.19:0/316232751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9814d600 con 0x7f3d9814c890 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.060+0000 7f3d9dab3640 1 -- 172.21.15.19:0/316232751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d98073a00 con 0x7f3d9814c890 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.061+0000 7f3d7ffff640 1 -- 172.21.15.19:0/316232751 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d8c01d070 con 0x7f3d9814c890 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.061+0000 7f3d7ffff640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d8406b4d0 0x7f3d8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.062+0000 7f3d7ffff640 1 -- 172.21.15.19:0/316232751 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d8c089c40 con 0x7f3d9814c890 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.062+0000 7f3d977fe640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d8406b4d0 0x7f3d8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.062+0000 7f3d977fe640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d8406b4d0 0x7f3d8406d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3d88033000 tx=0x7f3d88008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:51.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.066+0000 7f3d7ffff640 1 -- 172.21.15.19:0/316232751 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d8c058920 con 0x7f3d9814c890 2024-04-25T09:59:51.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.318+0000 7f3d9dab3640 1 -- 172.21.15.19:0/316232751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3d980c7330 con 0x7f3d9814c890 2024-04-25T09:59:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.320+0000 7f3d7ffff640 1 -- 172.21.15.19:0/316232751 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3d8c058220 con 0x7f3d9814c890 2024-04-25T09:59:51.324 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T09:59:51.325 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T09:59:51.325 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T09:59:51.325 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T09:59:51.325 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:59:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T09:59:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 -- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d8406b4d0 msgr2=0x7f3d8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d8406b4d0 0x7f3d8406d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3d88033000 tx=0x7f3d88008030 comp rx=0 tx=0).stop 2024-04-25T09:59:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 -- 172.21.15.19:0/316232751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9814c890 msgr2=0x7f3d980c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9814c890 0x7f3d980c8750 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00f7a0 tx=0x7f3d8c00fc60 comp rx=0 tx=0).stop 2024-04-25T09:59:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 -- 172.21.15.19:0/316232751 shutdown_connections 2024-04-25T09:59:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3d8406b4d0 0x7f3d8406d990 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d980c8c90 0x7f3d9815cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d9814c890 0x7f3d980c8750 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 --2- 172.21.15.19:0/316232751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d80003b90 0x7f3d9814c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 -- 172.21.15.19:0/316232751 >> 172.21.15.19:0/316232751 conn(0x7f3d980d0800 msgr2=0x7f3d980b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 -- 172.21.15.19:0/316232751 shutdown_connections 2024-04-25T09:59:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.323+0000 7f3d9dab3640 1 -- 172.21.15.19:0/316232751 wait complete. 2024-04-25T09:59:51.338 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.436+0000 7fd3c8eb3640 1 Processor -- start 2024-04-25T09:59:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.436+0000 7fd3c8eb3640 1 -- start start 2024-04-25T09:59:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c8eb3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd3c40b2cb0 0x7fd3c40d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c8eb3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c40b97a0 con 0x7fd3c40d1020 2024-04-25T09:59:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c8eb3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c40d1c50 con 0x7fd3c415dbe0 2024-04-25T09:59:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c8eb3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c415dfc0 con 0x7fd3c40b2cb0 2024-04-25T09:59:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c2d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd3c40d1020 0x7fd3c40d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35164/0 (socket says 172.21.15.19:35164) 2024-04-25T09:59:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c2d9d640 1 -- 172.21.15.19:0/1266297010 learned_addr learned my addr 172.21.15.19:0/1266297010 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3bbfff640 1 --2- 172.21.15.19:0/1266297010 >> v2:172.21.15.112:3300/0 conn(0x7fd3c40b2cb0 0x7fd3c40d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304908771 0 0) 0x7fd3c40b97a0 con 0x7fd3c40d1020 2024-04-25T09:59:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3b0003520 con 0x7fd3c40d1020 2024-04-25T09:59:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 387452785 0 0) 0x7fd3b0003520 con 0x7fd3c40d1020 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 >> v2:172.21.15.112:3300/0 conn(0x7fd3c40b2cb0 msgr2=0x7fd3c40d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c1d9b640 1 --2- 172.21.15.19:0/1266297010 >> v2:172.21.15.112:3300/0 conn(0x7fd3c40b2cb0 0x7fd3c40d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.437+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 >> v1:172.21.15.112:6789/0 conn(0x7fd3c415dbe0 legacy=0x7fd3c40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c40d1f30 con 0x7fd3c40d1020 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fd3bc0019f0 con 0x7fd3c40d1020 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 >> v1:172.21.15.19:6789/0 conn(0x7fd3c40d1020 legacy=0x7fd3c40d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c1d9b640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3b00039b0 0x7fd3b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c1d9b640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3b00043b0 0x7fd3b0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c1d9b640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3b0005080 0x7fd3b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b000bfb0 con 0x7fd3b00039b0 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b000c160 con 0x7fd3b0005080 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b000c360 con 0x7fd3b00043b0 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3bbfff640 1 --2- 172.21.15.19:0/1266297010 >> v2:172.21.15.112:3300/0 conn(0x7fd3c40b2cb0 0x7fd3c40d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c259c640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3b00039b0 0x7fd3b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c2d9d640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3b0005080 0x7fd3b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3bbfff640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3b00043b0 0x7fd3b0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c259c640 1 -- 172.21.15.19:0/1266297010 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3b00043b0 msgr2=0x7fd3b0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c259c640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3b00043b0 0x7fd3b0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c259c640 1 -- 172.21.15.19:0/1266297010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3b0005080 msgr2=0x7fd3b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c259c640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3b0005080 0x7fd3b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c259c640 1 -- 172.21.15.19:0/1266297010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b0003520 con 0x7fd3b00039b0 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c2d9d640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3b0005080 0x7fd3b000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.438+0000 7fd3c2d9d640 1 -- 172.21.15.19:0/1266297010 reap_dead start 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.439+0000 7fd3c259c640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3b00039b0 0x7fd3b0003d90 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4001860 tx=0x7fd3b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92759206ffb82ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.439+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4037070 con 0x7fd3b00039b0 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.439+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b402abf0 con 0x7fd3b00039b0 2024-04-25T09:59:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.439+0000 7fd3c1d9b640 1 -- 172.21.15.19:0/1266297010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4001430 con 0x7fd3b00039b0 2024-04-25T09:59:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.439+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1266297010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3b00039b0 msgr2=0x7fd3b0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.439+0000 7fd3c8eb3640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3b00039b0 0x7fd3b0003d90 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4001860 tx=0x7fd3b0005830 comp rx=0 tx=0).stop 2024-04-25T09:59:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.439+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1266297010 shutdown_connections 2024-04-25T09:59:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.439+0000 7fd3c8eb3640 1 --2- 172.21.15.19:0/1266297010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3b00039b0 0x7fd3b0003d90 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.439+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1266297010 >> 172.21.15.19:0/1266297010 conn(0x7fd3c4061a20 msgr2=0x7fd3c4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.439+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1266297010 shutdown_connections 2024-04-25T09:59:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.439+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1266297010 wait complete. 2024-04-25T09:59:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.441+0000 7fd3c8eb3640 1 Processor -- start 2024-04-25T09:59:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.441+0000 7fd3c8eb3640 1 -- start start 2024-04-25T09:59:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c8eb3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3b00039b0 0x7fd3c4159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c8eb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c40cb690 0x7fd3c41f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c8eb3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c40aa370 0x7fd3c40aa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c8eb3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c40b92a0 con 0x7fd3c40aa370 2024-04-25T09:59:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c8eb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c40d1a00 con 0x7fd3c40cb690 2024-04-25T09:59:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c8eb3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c40b9420 con 0x7fd3b00039b0 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c40aa370 0x7fd3c40aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c40aa370 0x7fd3c40aa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38600/0 (socket says 172.21.15.19:38600) 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c2d9d640 1 -- 172.21.15.19:0/1936122693 learned_addr learned my addr 172.21.15.19:0/1936122693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3bbfff640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c40cb690 0x7fd3c41f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c259c640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3b00039b0 0x7fd3c4159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c2d9d640 1 -- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3b00039b0 msgr2=0x7fd3c4159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c2d9d640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3b00039b0 0x7fd3c4159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c2d9d640 1 -- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c40cb690 msgr2=0x7fd3c41f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c2d9d640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c40cb690 0x7fd3c41f3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c2d9d640 1 -- 172.21.15.19:0/1936122693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c4159d40 con 0x7fd3c40aa370 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c259c640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3b00039b0 0x7fd3c4159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3bbfff640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c40cb690 0x7fd3c41f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c2d9d640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c40aa370 0x7fd3c40aa820 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40d4a60 tx=0x7fd3bc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3baffd640 1 -- 172.21.15.19:0/1936122693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00eb90 con 0x7fd3c40aa370 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3baffd640 1 -- 172.21.15.19:0/1936122693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc018070 con 0x7fd3c40aa370 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3baffd640 1 -- 172.21.15.19:0/1936122693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc013420 con 0x7fd3c40aa370 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1936122693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c41f4200 con 0x7fd3c40aa370 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.442+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1936122693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c40cbee0 con 0x7fd3c40aa370 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.443+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1936122693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c4073990 con 0x7fd3c40aa370 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.444+0000 7fd3baffd640 1 -- 172.21.15.19:0/1936122693 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3bc0135c0 con 0x7fd3c40aa370 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.444+0000 7fd3baffd640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a406b500 0x7fd3a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.444+0000 7fd3baffd640 1 -- 172.21.15.19:0/1936122693 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd3bc08a660 con 0x7fd3c40aa370 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.444+0000 7fd3c259c640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a406b500 0x7fd3a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.445+0000 7fd3c259c640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a406b500 0x7fd3a406d9c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4034000 tx=0x7fd3b4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.448+0000 7fd3baffd640 1 -- 172.21.15.19:0/1936122693 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3bc0593e0 con 0x7fd3c40aa370 2024-04-25T09:59:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.664+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1936122693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3c40b2970 con 0x7fd3c40aa370 2024-04-25T09:59:51.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.664+0000 7fd3baffd640 1 -- 172.21.15.19:0/1936122693 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd3bc088e20 con 0x7fd3c40aa370 2024-04-25T09:59:51.669 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T09:59:51.669 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:59:51.669 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a406b500 msgr2=0x7fd3a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a406b500 0x7fd3a406d9c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4034000 tx=0x7fd3b4009030 comp rx=0 tx=0).stop 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1936122693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c40aa370 msgr2=0x7fd3c40aa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c40aa370 0x7fd3c40aa820 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40d4a60 tx=0x7fd3bc00e530 comp rx=0 tx=0).stop 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1936122693 shutdown_connections 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd3a406b500 0x7fd3a406d9c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3c40aa370 0x7fd3c40aa820 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3c40cb690 0x7fd3c41f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 --2- 172.21.15.19:0/1936122693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3b00039b0 0x7fd3c4159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1936122693 >> 172.21.15.19:0/1936122693 conn(0x7fd3c4061a20 msgr2=0x7fd3c40b6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1936122693 shutdown_connections 2024-04-25T09:59:51.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:51.666+0000 7fd3c8eb3640 1 -- 172.21.15.19:0/1936122693 wait complete. 2024-04-25T09:59:52.084 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:59:46.906578+0000', 'modified': '2024-04-25T09:59:48.905936+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8025}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8025': {'gid': 8025, 'name': 'a', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/4224193015', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 4224193015}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 4224193015}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:59:52.084 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:59:52.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:59:52.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.189+0000 7f4063fd5640 1 Processor -- start 2024-04-25T09:59:52.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.189+0000 7f4063fd5640 1 -- start start 2024-04-25T09:59:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4063fd5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f405c15b7d0 0x7f405c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4063fd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f405c0b5620 0x7f405c0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4063fd5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f405c06c2d0 0x7f405c06c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4063fd5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405c0b5fc0 con 0x7f405c06c2d0 2024-04-25T09:59:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4063fd5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405c06cca0 con 0x7f405c15b7d0 2024-04-25T09:59:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4063fd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405c06cf50 con 0x7f405c0b5620 2024-04-25T09:59:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4062572640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f405c06c2d0 0x7f405c06c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4062572640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f405c06c2d0 0x7f405c06c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38602/0 (socket says 172.21.15.19:38602) 2024-04-25T09:59:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4062572640 1 -- 172.21.15.19:0/3214670014 learned_addr learned my addr 172.21.15.19:0/3214670014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4061d71640 1 --2- 172.21.15.19:0/3214670014 >> v2:172.21.15.112:3300/0 conn(0x7f405c15b7d0 0x7f405c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4061570640 1 --2- 172.21.15.19:0/3214670014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f405c0b5620 0x7f405c0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4062572640 1 -- 172.21.15.19:0/3214670014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f405c0b5620 msgr2=0x7f405c0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4062572640 1 --2- 172.21.15.19:0/3214670014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f405c0b5620 0x7f405c0b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4062572640 1 -- 172.21.15.19:0/3214670014 >> v2:172.21.15.112:3300/0 conn(0x7f405c15b7d0 msgr2=0x7f405c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4062572640 1 --2- 172.21.15.19:0/3214670014 >> v2:172.21.15.112:3300/0 conn(0x7f405c15b7d0 0x7f405c0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4062572640 1 -- 172.21.15.19:0/3214670014 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c0ce530 con 0x7f405c06c2d0 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4061570640 1 --2- 172.21.15.19:0/3214670014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f405c0b5620 0x7f405c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.190+0000 7f4061d71640 1 --2- 172.21.15.19:0/3214670014 >> v2:172.21.15.112:3300/0 conn(0x7f405c15b7d0 0x7f405c0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.191+0000 7f4062572640 1 --2- 172.21.15.19:0/3214670014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f405c06c2d0 0x7f405c06c760 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f405c08c0d0 tx=0x7f405402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea25c32501ca1a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.191+0000 7f4060d6f640 1 -- 172.21.15.19:0/3214670014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054036070 con 0x7f405c06c2d0 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.191+0000 7f4060d6f640 1 -- 172.21.15.19:0/3214670014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054031600 con 0x7f405c06c2d0 2024-04-25T09:59:52.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.191+0000 7f4060d6f640 1 -- 172.21.15.19:0/3214670014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40540319d0 con 0x7f405c06c2d0 2024-04-25T09:59:52.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.192+0000 7f4063fd5640 1 -- 172.21.15.19:0/3214670014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f405c06c2d0 msgr2=0x7f405c06c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.192+0000 7f4063fd5640 1 --2- 172.21.15.19:0/3214670014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f405c06c2d0 0x7f405c06c760 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f405c08c0d0 tx=0x7f405402a6a0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.192+0000 7f4063fd5640 1 -- 172.21.15.19:0/3214670014 shutdown_connections 2024-04-25T09:59:52.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.192+0000 7f4063fd5640 1 --2- 172.21.15.19:0/3214670014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f405c0b5620 0x7f405c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.192+0000 7f4063fd5640 1 --2- 172.21.15.19:0/3214670014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f405c06c2d0 0x7f405c06c760 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.192+0000 7f4063fd5640 1 --2- 172.21.15.19:0/3214670014 >> v2:172.21.15.112:3300/0 conn(0x7f405c15b7d0 0x7f405c0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.192+0000 7f4063fd5640 1 -- 172.21.15.19:0/3214670014 >> 172.21.15.19:0/3214670014 conn(0x7f405c0d0800 msgr2=0x7f405c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:52.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.192+0000 7f4063fd5640 1 -- 172.21.15.19:0/3214670014 shutdown_connections 2024-04-25T09:59:52.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.192+0000 7f4063fd5640 1 -- 172.21.15.19:0/3214670014 wait complete. 2024-04-25T09:59:52.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.193+0000 7f4063fd5640 1 Processor -- start 2024-04-25T09:59:52.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.193+0000 7f4063fd5640 1 -- start start 2024-04-25T09:59:52.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4063fd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f405c06c2d0 0x7f405c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4063fd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f405c0b5620 0x7f405c157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4063fd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f405c15b7d0 0x7f405c15d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4063fd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c06d230 con 0x7f405c06c2d0 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4063fd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c0ceaf0 con 0x7f405c0b5620 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4063fd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405c0cf550 con 0x7f405c15b7d0 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4061d71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f405c06c2d0 0x7f405c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4061d71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f405c06c2d0 0x7f405c157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38618/0 (socket says 172.21.15.19:38618) 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4061d71640 1 -- 172.21.15.19:0/741101209 learned_addr learned my addr 172.21.15.19:0/741101209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4061570640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f405c0b5620 0x7f405c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4062572640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f405c15b7d0 0x7f405c15d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4061d71640 1 -- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f405c15b7d0 msgr2=0x7f405c15d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4061d71640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f405c15b7d0 0x7f405c15d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4061d71640 1 -- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f405c0b5620 msgr2=0x7f405c157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4061d71640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f405c0b5620 0x7f405c157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4061d71640 1 -- 172.21.15.19:0/741101209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c155b50 con 0x7f405c06c2d0 2024-04-25T09:59:52.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f4061d71640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f405c06c2d0 0x7f405c157210 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f405800c7f0 tx=0x7f405800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:52.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.194+0000 7f40527fc640 1 -- 172.21.15.19:0/741101209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058015070 con 0x7f405c06c2d0 2024-04-25T09:59:52.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.195+0000 7f40527fc640 1 -- 172.21.15.19:0/741101209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4058007820 con 0x7f405c06c2d0 2024-04-25T09:59:52.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.195+0000 7f4063fd5640 1 -- 172.21.15.19:0/741101209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405c155e30 con 0x7f405c06c2d0 2024-04-25T09:59:52.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.195+0000 7f40527fc640 1 -- 172.21.15.19:0/741101209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058007bd0 con 0x7f405c06c2d0 2024-04-25T09:59:52.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.195+0000 7f4063fd5640 1 -- 172.21.15.19:0/741101209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405c1563f0 con 0x7f405c06c2d0 2024-04-25T09:59:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.196+0000 7f40527fc640 1 -- 172.21.15.19:0/741101209 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4058018070 con 0x7f405c06c2d0 2024-04-25T09:59:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.197+0000 7f40527fc640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f404006b4d0 0x7f404006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.197+0000 7f40527fc640 1 -- 172.21.15.19:0/741101209 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f405808a500 con 0x7f405c06c2d0 2024-04-25T09:59:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.197+0000 7f4061570640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f404006b4d0 0x7f404006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.198+0000 7f4061570640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f404006b4d0 0x7f404006d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f405c1f6910 tx=0x7f404c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.198+0000 7f4066db5740 1 Processor -- start 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.198+0000 7f4066db5740 1 -- start start 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4066db5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff08610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4066db5740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff25ad0 0x5571dff26020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4066db5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dfeddf10 0x5571dff24190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4066db5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571dfe40a10 con 0x5571dfeddf10 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4066db5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571dfde9b30 con 0x5571dff08230 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4066db5740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571dfe3cc50 con 0x5571dff25ad0 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4062572640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dfeddf10 0x5571dff24190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4062572640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dfeddf10 0x5571dff24190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38632/0 (socket says 172.21.15.19:38632) 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4062572640 1 -- 172.21.15.19:0/3769571469 learned_addr learned my addr 172.21.15.19:0/3769571469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4061570640 1 --2- 172.21.15.19:0/3769571469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff25ad0 0x5571dff26020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4061d71640 1 --2- 172.21.15.19:0/3769571469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff08610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4062572640 1 -- 172.21.15.19:0/3769571469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff25ad0 msgr2=0x5571dff26020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4062572640 1 --2- 172.21.15.19:0/3769571469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff25ad0 0x5571dff26020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4062572640 1 -- 172.21.15.19:0/3769571469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 msgr2=0x5571dff08610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4062572640 1 --2- 172.21.15.19:0/3769571469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff08610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4062572640 1 -- 172.21.15.19:0/3769571469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571dfdfee10 con 0x5571dfeddf10 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f4062572640 1 --2- 172.21.15.19:0/3769571469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dfeddf10 0x5571dff24190 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f4054009f40 tx=0x7f405402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f676e0b013820f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f403f7fe640 1 -- 172.21.15.19:0/3769571469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054041070 con 0x5571dfeddf10 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f403f7fe640 1 -- 172.21.15.19:0/3769571469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405403c640 con 0x5571dfeddf10 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.199+0000 7f403f7fe640 1 -- 172.21.15.19:0/3769571469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054036040 con 0x5571dfeddf10 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.200+0000 7f4066db5740 1 -- 172.21.15.19:0/3769571469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dfeddf10 msgr2=0x5571dff24190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.200+0000 7f4066db5740 1 --2- 172.21.15.19:0/3769571469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dfeddf10 0x5571dff24190 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f4054009f40 tx=0x7f405402e000 comp rx=0 tx=0).stop 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.200+0000 7f4066db5740 1 -- 172.21.15.19:0/3769571469 shutdown_connections 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.200+0000 7f4066db5740 1 --2- 172.21.15.19:0/3769571469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dfeddf10 0x5571dff24190 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.200+0000 7f4066db5740 1 --2- 172.21.15.19:0/3769571469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff25ad0 0x5571dff26020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.200+0000 7f4066db5740 1 --2- 172.21.15.19:0/3769571469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff08610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.200+0000 7f4066db5740 1 -- 172.21.15.19:0/3769571469 >> 172.21.15.19:0/3769571469 conn(0x5571dfeda840 msgr2=0x5571dff23040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.200+0000 7f4066db5740 1 -- 172.21.15.19:0/3769571469 shutdown_connections 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.200+0000 7f4066db5740 1 -- 172.21.15.19:0/3769571469 wait complete. 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.201+0000 7f4066db5740 1 -- start start 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.201+0000 7f4066db5740 1 Processor -- start 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4066db5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff330d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4066db5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dfed74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4066db5740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff33610 0x5571dfed79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4066db5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571dfedecb0 con 0x5571dff25ad0 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4066db5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571dfdfd0c0 con 0x5571dff08230 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4066db5740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571dfed8770 con 0x5571dff33610 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4066db5740 10 client.0 _ll_register_callbacks cb 0x5571dfdf0ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4061570640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dfed74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4061570640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dfed74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38642/0 (socket says 172.21.15.19:38642) 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4061570640 1 -- 172.21.15.19:0/2895942740 learned_addr learned my addr 172.21.15.19:0/2895942740 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4062572640 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff33610 0x5571dfed79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4061d71640 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4061570640 1 -- 172.21.15.19:0/2895942740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff33610 msgr2=0x5571dfed79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4061570640 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff33610 0x5571dfed79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4061570640 1 -- 172.21.15.19:0/2895942740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 msgr2=0x5571dff330d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4061570640 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff330d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4061570640 1 -- 172.21.15.19:0/2895942740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571dfb552d0 con 0x5571dff25ad0 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4061570640 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dfed74a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f404c0137d0 tx=0x7f404c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee4b50f76c4acea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f403cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f403cff9640 1 -- 172.21.15.19:0/2895942740 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c024070 con 0x5571dff25ad0 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f403cff9640 1 -- 172.21.15.19:0/2895942740 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f404c014900 con 0x5571dff25ad0 2024-04-25T09:59:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.202+0000 7f4066db5740 1 -- 172.21.15.19:0/2895942740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f405403c640 con 0x5571dff25ad0 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f403cff9640 1 -- 172.21.15.19:0/2895942740 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f404c02f050 con 0x5571dff25ad0 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f4066db5740 10 client.8163 fetch_fsmap learned FSMap version 65 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f4066db5740 1 -- 172.21.15.19:0/2895942740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5571dfc1f6b0 con 0x5571dff25ad0 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f403cff9640 1 -- 172.21.15.19:0/2895942740 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405403c640 con 0x5571dff25ad0 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f403cff9640 1 -- 172.21.15.19:0/2895942740 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f404c032020 con 0x5571dff25ad0 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f4066db5740 10 client.8163 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f4066db5740 10 client.8163 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f4066db5740 20 client.8163 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f4066db5740 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5571dff29030 0x5571dff0a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f4066db5740 4 client.8163 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f4066db5740 1 -- 172.21.15.19:0/2895942740 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571dfe9f920 con 0x5571dff29030 2024-04-25T09:59:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f4061d71640 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5571dff29030 0x5571dff0a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f4061d71640 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5571dff29030 0x5571dff0a080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=913624a32e68609c server_cookie=fb39850d8dba323b in_seq=0 out_seq=0 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.203+0000 7f403cff9640 10 client.8163 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f403cff9640 1 -- 172.21.15.19:0/2895942740 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5571dfe9f920 con 0x5571dff29030 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f403cff9640 10 client.8163 handle_command_reply: tid=0 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 1 client.8163 shutdown 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 1 -- 172.21.15.19:0/2895942740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 msgr2=0x5571dfed74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dfed74a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f404c0137d0 tx=0x7f404c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 1 -- 172.21.15.19:0/2895942740 shutdown_connections 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5571dff29030 0x5571dff0a080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff33610 0x5571dfed79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dfed74a0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 1 --2- 172.21.15.19:0/2895942740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff330d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f403cff9640 0 client.8163 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T09:59:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 1 -- 172.21.15.19:0/2895942740 >> 172.21.15.19:0/2895942740 conn(0x5571dfeddf10 msgr2=0x5571dff265c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 1 -- 172.21.15.19:0/2895942740 shutdown_connections 2024-04-25T09:59:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 1 -- 172.21.15.19:0/2895942740 wait complete. 2024-04-25T09:59:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.204+0000 7f4066db5740 20 client.8163 trim_cache size 0 max 16384 2024-04-25T09:59:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.220+0000 7f4066db5740 1 Processor -- start 2024-04-25T09:59:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.220+0000 7f4066db5740 1 -- start start 2024-04-25T09:59:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4066db5740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff08230 0x5571dff07240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4066db5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff29960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4066db5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff33610 0x5571dff29ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4066db5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c1ff3c0 con 0x5571dff25ad0 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4066db5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c153ba0 con 0x5571dff33610 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4066db5740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405c1fedc0 con 0x5571dff08230 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4061570640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff29960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4061570640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff29960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38658/0 (socket says 172.21.15.19:38658) 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4061570640 1 -- 172.21.15.19:0/3826789042 learned_addr learned my addr 172.21.15.19:0/3826789042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4062572640 1 --2- 172.21.15.19:0/3826789042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff33610 0x5571dff29ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4061d71640 1 --2- 172.21.15.19:0/3826789042 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff08230 0x5571dff07240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4061570640 1 -- 172.21.15.19:0/3826789042 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff08230 msgr2=0x5571dff07240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4061570640 1 --2- 172.21.15.19:0/3826789042 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff08230 0x5571dff07240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4061570640 1 -- 172.21.15.19:0/3826789042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff33610 msgr2=0x5571dff29ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4061570640 1 --2- 172.21.15.19:0/3826789042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff33610 0x5571dff29ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.221+0000 7f4061570640 1 -- 172.21.15.19:0/3826789042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571dfed8400 con 0x5571dff25ad0 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f4061570640 1 --2- 172.21.15.19:0/3826789042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff29960 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f404c006b10 tx=0x7f404c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20e792b4033c171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f403effd640 1 -- 172.21.15.19:0/3826789042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c024040 con 0x5571dff25ad0 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f403effd640 1 -- 172.21.15.19:0/3826789042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404c01b040 con 0x5571dff25ad0 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f403effd640 1 -- 172.21.15.19:0/3826789042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c01a040 con 0x5571dff25ad0 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f4066db5740 1 -- 172.21.15.19:0/3826789042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 msgr2=0x5571dff29960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f4066db5740 1 --2- 172.21.15.19:0/3826789042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff29960 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f404c006b10 tx=0x7f404c021000 comp rx=0 tx=0).stop 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f4066db5740 1 -- 172.21.15.19:0/3826789042 shutdown_connections 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f4066db5740 1 --2- 172.21.15.19:0/3826789042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff33610 0x5571dff29ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f4066db5740 1 --2- 172.21.15.19:0/3826789042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff29960 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f4066db5740 1 --2- 172.21.15.19:0/3826789042 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff08230 0x5571dff07240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f4066db5740 1 -- 172.21.15.19:0/3826789042 >> 172.21.15.19:0/3826789042 conn(0x5571dfeddf10 msgr2=0x5571dfed77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f4066db5740 1 -- 172.21.15.19:0/3826789042 shutdown_connections 2024-04-25T09:59:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.222+0000 7f4066db5740 1 -- 172.21.15.19:0/3826789042 wait complete. 2024-04-25T09:59:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.223+0000 7f4066db5740 1 -- start start 2024-04-25T09:59:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.223+0000 7f4066db5740 1 Processor -- start 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4066db5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff8db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4066db5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff3c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4066db5740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff33610 0x5571dff07190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4066db5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571dff28eb0 con 0x5571dff25ad0 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4066db5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571dff09270 con 0x5571dff08230 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4066db5740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571dff29480 con 0x5571dff33610 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4066db5740 10 client.0 _ll_register_callbacks cb 0x5571dfed8c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4061570640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff3c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4061570640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff3c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38664/0 (socket says 172.21.15.19:38664) 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4061570640 1 -- 172.21.15.19:0/1543509689 learned_addr learned my addr 172.21.15.19:0/1543509689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4061d71640 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff8db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4062572640 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff33610 0x5571dff07190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4061570640 1 -- 172.21.15.19:0/1543509689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff33610 msgr2=0x5571dff07190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4061570640 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff33610 0x5571dff07190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4061570640 1 -- 172.21.15.19:0/1543509689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 msgr2=0x5571dff8db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4061570640 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff8db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4061570640 1 -- 172.21.15.19:0/1543509689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571dff27a40 con 0x5571dff25ad0 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4061570640 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff3c8a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f404c00a870 tx=0x7f404c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c524407009774b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f403cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f403cff9640 1 -- 172.21.15.19:0/1543509689 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c042060 con 0x5571dff25ad0 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f403cff9640 1 -- 172.21.15.19:0/1543509689 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f404c029650 con 0x5571dff25ad0 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.224+0000 7f4066db5740 1 -- 172.21.15.19:0/1543509689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f404c01b040 con 0x5571dff25ad0 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f403cff9640 1 -- 172.21.15.19:0/1543509689 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f404c048050 con 0x5571dff25ad0 2024-04-25T09:59:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f4066db5740 10 client.8169 fetch_fsmap learned FSMap version 65 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f4066db5740 1 -- 172.21.15.19:0/1543509689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5571dff8f520 con 0x5571dff25ad0 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f403cff9640 1 -- 172.21.15.19:0/1543509689 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404c01b040 con 0x5571dff25ad0 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f403cff9640 1 -- 172.21.15.19:0/1543509689 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f404c04b020 con 0x5571dff25ad0 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f4066db5740 10 client.8169 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f4066db5740 10 client.8169 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f4066db5740 20 client.8169 populate_metadata read hostname 'smithi019' 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f4066db5740 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5571dff19380 0x5571dff8e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f4066db5740 4 client.8169 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f4066db5740 1 -- 172.21.15.19:0/1543509689 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5571dfe4df50 con 0x5571dff19380 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f4061d71640 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5571dff19380 0x5571dff8e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f403cff9640 10 client.8169 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.225+0000 7f4061d71640 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5571dff19380 0x5571dff8e440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d586ee8045101e29 server_cookie=57933463245d6fb1 in_seq=0 out_seq=0 2024-04-25T09:59:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f403cff9640 1 -- 172.21.15.19:0/1543509689 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5571dfe4df50 con 0x5571dff19380 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f403cff9640 10 client.8169 handle_command_reply: tid=0 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 1 client.8169 shutdown 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 1 -- 172.21.15.19:0/1543509689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 msgr2=0x5571dff3c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff3c8a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f404c00a870 tx=0x7f404c015000 comp rx=0 tx=0).stop 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 1 -- 172.21.15.19:0/1543509689 shutdown_connections 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5571dff19380 0x5571dff8e440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571dff33610 0x5571dff07190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571dff25ad0 0x5571dff3c8a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 1 --2- 172.21.15.19:0/1543509689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571dff08230 0x5571dff8db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f403cff9640 0 client.8169 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 1 -- 172.21.15.19:0/1543509689 >> 172.21.15.19:0/1543509689 conn(0x5571dfeddf10 msgr2=0x5571dfed77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 1 -- 172.21.15.19:0/1543509689 shutdown_connections 2024-04-25T09:59:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 1 -- 172.21.15.19:0/1543509689 wait complete. 2024-04-25T09:59:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4066db5740 20 client.8169 trim_cache size 0 max 16384 2024-04-25T09:59:52.237 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T09:59:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T09:59:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T09:59:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 8025, 2024-04-25T09:59:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T09:59:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T09:59:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T09:59:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 4.6672896919999998, 2024-04-25T09:59:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 65, 2024-04-25T09:59:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 106, 2024-04-25T09:59:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 106, 2024-04-25T09:59:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 9.7067480249999996 2024-04-25T09:59:52.238 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T09:59:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4063fd5640 1 -- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f404006b4d0 msgr2=0x7f404006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4063fd5640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f404006b4d0 0x7f404006d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f405c1f6910 tx=0x7f404c008040 comp rx=0 tx=0).stop 2024-04-25T09:59:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4063fd5640 1 -- 172.21.15.19:0/741101209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f405c06c2d0 msgr2=0x7f405c157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4063fd5640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f405c06c2d0 0x7f405c157210 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f405800c7f0 tx=0x7f405800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4063fd5640 1 -- 172.21.15.19:0/741101209 shutdown_connections 2024-04-25T09:59:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4063fd5640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f404006b4d0 0x7f404006d990 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4063fd5640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f405c15b7d0 0x7f405c15d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4063fd5640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f405c0b5620 0x7f405c157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4063fd5640 1 --2- 172.21.15.19:0/741101209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f405c06c2d0 0x7f405c157210 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.226+0000 7f4063fd5640 1 -- 172.21.15.19:0/741101209 >> 172.21.15.19:0/741101209 conn(0x7f405c0d0800 msgr2=0x7f405c160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.227+0000 7f4063fd5640 1 -- 172.21.15.19:0/741101209 shutdown_connections 2024-04-25T09:59:52.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.227+0000 7f4063fd5640 1 -- 172.21.15.19:0/741101209 wait complete. 2024-04-25T09:59:52.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:59:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.337+0000 7f1513916640 1 Processor -- start 2024-04-25T09:59:52.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.337+0000 7f1513916640 1 -- start start 2024-04-25T09:59:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.338+0000 7f1513916640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f150c15dc90 0x7f150c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.338+0000 7f1513916640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f150c0cc8e0 0x7f150c15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.338+0000 7f1513916640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150c15f390 con 0x7f150c0d1810 2024-04-25T09:59:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.338+0000 7f1513916640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c0cce50 con 0x7f150c15dc90 2024-04-25T09:59:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.338+0000 7f1513916640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c161af0 con 0x7f150c0cc8e0 2024-04-25T09:59:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.338+0000 7f15116b2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f150c15dc90 0x7f150c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.338+0000 7f1511eb3640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f150c0d1810 0x7f150c0cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35178/0 (socket says 172.21.15.19:35178) 2024-04-25T09:59:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.338+0000 7f1511eb3640 1 -- 172.21.15.19:0/1241941214 learned_addr learned my addr 172.21.15.19:0/1241941214 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T09:59:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.338+0000 7f1510eb1640 1 --2- 172.21.15.19:0/1241941214 >> v2:172.21.15.112:3300/0 conn(0x7f150c0cc8e0 0x7f150c15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.339+0000 7f15116b2640 1 -- 172.21.15.19:0/1241941214 >> v2:172.21.15.112:3300/0 conn(0x7f150c0cc8e0 msgr2=0x7f150c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.339+0000 7f15116b2640 1 --2- 172.21.15.19:0/1241941214 >> v2:172.21.15.112:3300/0 conn(0x7f150c0cc8e0 0x7f150c15ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.339+0000 7f15116b2640 1 -- 172.21.15.19:0/1241941214 >> v1:172.21.15.19:6789/0 conn(0x7f150c0d1810 legacy=0x7f150c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.339+0000 7f150bfff640 1 -- 172.21.15.19:0/1241941214 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766097320 0 0) 0x7f14fc001490 con 0x7f150c0d1810 2024-04-25T09:59:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.339+0000 7f15116b2640 1 -- 172.21.15.19:0/1241941214 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c161dd0 con 0x7f150c15dc90 2024-04-25T09:59:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.339+0000 7f1510eb1640 1 --2- 172.21.15.19:0/1241941214 >> v2:172.21.15.112:3300/0 conn(0x7f150c0cc8e0 0x7f150c15ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.339+0000 7f15116b2640 1 --2- 172.21.15.19:0/1241941214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c15dc90 0x7f150c1625d0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f150c0c88a0 tx=0x7f150402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918c2973cbbcb85f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.340+0000 7f150bfff640 1 -- 172.21.15.19:0/1241941214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504037070 con 0x7f150c15dc90 2024-04-25T09:59:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.340+0000 7f150bfff640 1 -- 172.21.15.19:0/1241941214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1504032440 con 0x7f150c15dc90 2024-04-25T09:59:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.340+0000 7f150bfff640 1 -- 172.21.15.19:0/1241941214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150402ae60 con 0x7f150c15dc90 2024-04-25T09:59:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.340+0000 7f1513916640 1 -- 172.21.15.19:0/1241941214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c15dc90 msgr2=0x7f150c1625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.340+0000 7f1513916640 1 --2- 172.21.15.19:0/1241941214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c15dc90 0x7f150c1625d0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f150c0c88a0 tx=0x7f150402a110 comp rx=0 tx=0).stop 2024-04-25T09:59:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.340+0000 7f1513916640 1 -- 172.21.15.19:0/1241941214 shutdown_connections 2024-04-25T09:59:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.340+0000 7f1513916640 1 --2- 172.21.15.19:0/1241941214 >> v2:172.21.15.112:3300/0 conn(0x7f150c0cc8e0 0x7f150c15ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.340+0000 7f1513916640 1 --2- 172.21.15.19:0/1241941214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c15dc90 0x7f150c1625d0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.340+0000 7f1513916640 1 -- 172.21.15.19:0/1241941214 >> 172.21.15.19:0/1241941214 conn(0x7f150c06f590 msgr2=0x7f150c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.340+0000 7f1513916640 1 -- 172.21.15.19:0/1241941214 shutdown_connections 2024-04-25T09:59:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.340+0000 7f1513916640 1 -- 172.21.15.19:0/1241941214 wait complete. 2024-04-25T09:59:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.341+0000 7f1513916640 1 Processor -- start 2024-04-25T09:59:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.341+0000 7f1513916640 1 -- start start 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f1513916640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c0cc8e0 0x7f150c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f1513916640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f150c0d1810 0x7f150c0ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f1513916640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f150c15dc90 0x7f150c0b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f1513916640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c15e070 con 0x7f150c0cc8e0 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f1513916640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c0cd130 con 0x7f150c0d1810 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f1513916640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c15e1f0 con 0x7f150c15dc90 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f15116b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c0cc8e0 0x7f150c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f15116b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c0cc8e0 0x7f150c155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38674/0 (socket says 172.21.15.19:38674) 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f15116b2640 1 -- 172.21.15.19:0/1327421448 learned_addr learned my addr 172.21.15.19:0/1327421448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f1510eb1640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f150c0d1810 0x7f150c0ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f1511eb3640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f150c15dc90 0x7f150c0b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f15116b2640 1 -- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f150c15dc90 msgr2=0x7f150c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f15116b2640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f150c15dc90 0x7f150c0b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f15116b2640 1 -- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f150c0d1810 msgr2=0x7f150c0ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f15116b2640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f150c0d1810 0x7f150c0ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.342+0000 7f15116b2640 1 -- 172.21.15.19:0/1327421448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c0b5400 con 0x7f150c0cc8e0 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.343+0000 7f15116b2640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c0cc8e0 0x7f150c155aa0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1504000c00 tx=0x7f15040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.343+0000 7f1509ffb640 1 -- 172.21.15.19:0/1327421448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504037050 con 0x7f150c0cc8e0 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.343+0000 7f1509ffb640 1 -- 172.21.15.19:0/1327421448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1504038040 con 0x7f150c0cc8e0 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.343+0000 7f1513916640 1 -- 172.21.15.19:0/1327421448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c0b4680 con 0x7f150c0cc8e0 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.343+0000 7f1509ffb640 1 -- 172.21.15.19:0/1327421448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504004450 con 0x7f150c0cc8e0 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.343+0000 7f1513916640 1 -- 172.21.15.19:0/1327421448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c0b4c00 con 0x7f150c0cc8e0 2024-04-25T09:59:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.344+0000 7f1513916640 1 -- 172.21.15.19:0/1327421448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c073990 con 0x7f150c0cc8e0 2024-04-25T09:59:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.344+0000 7f1509ffb640 1 -- 172.21.15.19:0/1327421448 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1504030980 con 0x7f150c0cc8e0 2024-04-25T09:59:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.345+0000 7f1509ffb640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14f806b500 0x7f14f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.345+0000 7f1509ffb640 1 -- 172.21.15.19:0/1327421448 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f15040ab0e0 con 0x7f150c0cc8e0 2024-04-25T09:59:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.345+0000 7f1510eb1640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14f806b500 0x7f14f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.346+0000 7f1510eb1640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14f806b500 0x7f14f806d9c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f150c0b5640 tx=0x7f1500008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.349+0000 7f1509ffb640 1 -- 172.21.15.19:0/1327421448 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1504079e60 con 0x7f150c0cc8e0 2024-04-25T09:59:52.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.568+0000 7f1513916640 1 -- 172.21.15.19:0/1327421448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f150c0cf6b0 con 0x7f150c0cc8e0 2024-04-25T09:59:52.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.568+0000 7f1509ffb640 1 -- 172.21.15.19:0/1327421448 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1504079760 con 0x7f150c0cc8e0 2024-04-25T09:59:52.572 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:59:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 -- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14f806b500 msgr2=0x7f14f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14f806b500 0x7f14f806d9c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f150c0b5640 tx=0x7f1500008040 comp rx=0 tx=0).stop 2024-04-25T09:59:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 -- 172.21.15.19:0/1327421448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c0cc8e0 msgr2=0x7f150c155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c0cc8e0 0x7f150c155aa0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1504000c00 tx=0x7f15040046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 -- 172.21.15.19:0/1327421448 shutdown_connections 2024-04-25T09:59:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14f806b500 0x7f14f806d9c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f150c15dc90 0x7f150c0b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f150c0d1810 0x7f150c0ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 --2- 172.21.15.19:0/1327421448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c0cc8e0 0x7f150c155aa0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 -- 172.21.15.19:0/1327421448 >> 172.21.15.19:0/1327421448 conn(0x7f150c06f590 msgr2=0x7f150c06d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 -- 172.21.15.19:0/1327421448 shutdown_connections 2024-04-25T09:59:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T09:59:52.570+0000 7f1513916640 1 -- 172.21.15.19:0/1327421448 wait complete. 2024-04-25T09:59:52.585 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:59:52.585 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:59:52.585 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T09:59:52.585 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:59:52.585 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:52.585 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:59:52.585 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:59:52.585 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:59:52.585 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:52.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:52.586 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T09:59:52.601 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:59:52.601 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42353sec preferred_lft 42353sec 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:52.603 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:59:52.604 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:59:52.604 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T09:59:52.604 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:52.604 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:59:52.604 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:59:52.604 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T09:59:52.604 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:52.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:52.604 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:59:52.664 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T09:59:52.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:52.666 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T09:59:52.723 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T09:59:52.723 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:59:52.723 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:59:52.723 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:59:52.723 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:59:52.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:52.724 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T09:59:52.724 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:52.724 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:59:52.724 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:59:52.724 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:59:52.724 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:52.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:52.810 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T09:59:52.825 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:59:52.825 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:59:52.825 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:52.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:52.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T09:59:52.890 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:59:52.890 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:52.946 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:52.946 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:59:52.946 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:53.001 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:59:53.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:53.001 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:53.083 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:53.084 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:59:53.091 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:53.091 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:59:53.091 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T09:59:53.091 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T09:59:53.091 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T09:59:53.092 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:59:53.092 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:59:53.092 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:59:53.092 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:59:53.092 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:59:53.093 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:59:53.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:53.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:53.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:53.159 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:59:53.219 INFO:teuthology.orchestra.run.smithi019.stdout:client8181 2024-04-25T09:59:53.220 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T09:59:53.220 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:53.279 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:53.279 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T09:59:53.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:53.279 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:53.279 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:53.279 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:59:53.279 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:53.279 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:53.280 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:53.280 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958243 s, 109 MB/s 2024-04-25T09:59:53.362 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.362 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.362 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871282 s, 120 MB/s 2024-04-25T09:59:53.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.371 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785021 s, 134 MB/s 2024-04-25T09:59:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747699 s, 140 MB/s 2024-04-25T09:59:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841942 s, 125 MB/s 2024-04-25T09:59:53.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.396 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710547 s, 148 MB/s 2024-04-25T09:59:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073807 s, 142 MB/s 2024-04-25T09:59:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.414 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824165 s, 127 MB/s 2024-04-25T09:59:53.422 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.422 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.422 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694229 s, 151 MB/s 2024-04-25T09:59:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744536 s, 141 MB/s 2024-04-25T09:59:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688592 s, 152 MB/s 2024-04-25T09:59:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.445 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639962 s, 164 MB/s 2024-04-25T09:59:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749907 s, 140 MB/s 2024-04-25T09:59:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762619 s, 137 MB/s 2024-04-25T09:59:53.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.470 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734697 s, 143 MB/s 2024-04-25T09:59:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700894 s, 150 MB/s 2024-04-25T09:59:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727812 s, 144 MB/s 2024-04-25T09:59:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687605 s, 152 MB/s 2024-04-25T09:59:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.501 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652632 s, 161 MB/s 2024-04-25T09:59:53.510 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.510 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.510 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007436 s, 141 MB/s 2024-04-25T09:59:53.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686234 s, 153 MB/s 2024-04-25T09:59:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675953 s, 155 MB/s 2024-04-25T09:59:53.533 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.533 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.534 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747798 s, 140 MB/s 2024-04-25T09:59:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758776 s, 138 MB/s 2024-04-25T09:59:53.550 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.550 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759108 s, 138 MB/s 2024-04-25T09:59:53.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.559 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071429 s, 147 MB/s 2024-04-25T09:59:53.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.567 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716862 s, 146 MB/s 2024-04-25T09:59:53.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.575 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749429 s, 140 MB/s 2024-04-25T09:59:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698574 s, 150 MB/s 2024-04-25T09:59:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778556 s, 135 MB/s 2024-04-25T09:59:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718896 s, 146 MB/s 2024-04-25T09:59:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006305 s, 166 MB/s 2024-04-25T09:59:53.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:53.608 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:53.608 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:53.609 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:59:53.609 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:53.609 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:53.609 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:53.609 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:53.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.674 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706152 s, 148 MB/s 2024-04-25T09:59:53.683 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.683 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.683 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771789 s, 136 MB/s 2024-04-25T09:59:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.691 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699918 s, 150 MB/s 2024-04-25T09:59:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.699 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725989 s, 144 MB/s 2024-04-25T09:59:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634499 s, 165 MB/s 2024-04-25T09:59:53.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.714 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705299 s, 149 MB/s 2024-04-25T09:59:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711024 s, 147 MB/s 2024-04-25T09:59:53.729 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643602 s, 163 MB/s 2024-04-25T09:59:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657321 s, 160 MB/s 2024-04-25T09:59:53.745 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.745 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.745 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075758 s, 138 MB/s 2024-04-25T09:59:53.753 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.753 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.753 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682069 s, 154 MB/s 2024-04-25T09:59:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.761 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679104 s, 154 MB/s 2024-04-25T09:59:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654207 s, 160 MB/s 2024-04-25T09:59:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715813 s, 146 MB/s 2024-04-25T09:59:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.784 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695399 s, 151 MB/s 2024-04-25T09:59:53.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.791 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643165 s, 163 MB/s 2024-04-25T09:59:53.799 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.800 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740355 s, 142 MB/s 2024-04-25T09:59:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646862 s, 162 MB/s 2024-04-25T09:59:53.814 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.814 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.814 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634298 s, 165 MB/s 2024-04-25T09:59:53.821 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.821 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.821 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616295 s, 170 MB/s 2024-04-25T09:59:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675738 s, 155 MB/s 2024-04-25T09:59:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745833 s, 141 MB/s 2024-04-25T09:59:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.845 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643257 s, 163 MB/s 2024-04-25T09:59:53.852 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.852 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.852 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694664 s, 151 MB/s 2024-04-25T09:59:53.859 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.859 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.860 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637863 s, 164 MB/s 2024-04-25T09:59:53.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.867 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695182 s, 151 MB/s 2024-04-25T09:59:53.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.875 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709695 s, 148 MB/s 2024-04-25T09:59:53.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.884 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718345 s, 146 MB/s 2024-04-25T09:59:53.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.893 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649844 s, 161 MB/s 2024-04-25T09:59:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645882 s, 162 MB/s 2024-04-25T09:59:53.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.906 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685389 s, 153 MB/s 2024-04-25T09:59:53.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.914 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693851 s, 151 MB/s 2024-04-25T09:59:53.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:53.915 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:53.915 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:53.915 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:59:53.915 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:53.915 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:53.915 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:53.915 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:53.979 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.979 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.979 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653282 s, 161 MB/s 2024-04-25T09:59:53.988 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.988 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.988 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797865 s, 131 MB/s 2024-04-25T09:59:53.996 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:53.996 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:53.996 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668062 s, 157 MB/s 2024-04-25T09:59:54.003 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.004 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.004 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684586 s, 153 MB/s 2024-04-25T09:59:54.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.012 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721892 s, 145 MB/s 2024-04-25T09:59:54.020 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.020 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.020 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691239 s, 152 MB/s 2024-04-25T09:59:54.027 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.027 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.027 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650499 s, 161 MB/s 2024-04-25T09:59:54.035 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.035 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.035 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695286 s, 151 MB/s 2024-04-25T09:59:54.043 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.043 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.043 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070164 s, 149 MB/s 2024-04-25T09:59:54.051 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.051 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.051 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747229 s, 140 MB/s 2024-04-25T09:59:54.059 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.059 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.059 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704945 s, 149 MB/s 2024-04-25T09:59:54.068 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.068 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.068 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813325 s, 129 MB/s 2024-04-25T09:59:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068548 s, 153 MB/s 2024-04-25T09:59:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705406 s, 149 MB/s 2024-04-25T09:59:54.092 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.092 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.092 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747069 s, 140 MB/s 2024-04-25T09:59:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678761 s, 154 MB/s 2024-04-25T09:59:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.108 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674775 s, 155 MB/s 2024-04-25T09:59:54.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724889 s, 145 MB/s 2024-04-25T09:59:54.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.123 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638536 s, 164 MB/s 2024-04-25T09:59:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665557 s, 158 MB/s 2024-04-25T09:59:54.138 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.138 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.138 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706572 s, 148 MB/s 2024-04-25T09:59:54.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.146 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069488 s, 151 MB/s 2024-04-25T09:59:54.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.154 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641105 s, 164 MB/s 2024-04-25T09:59:54.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.162 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705561 s, 149 MB/s 2024-04-25T09:59:54.168 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.168 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058813 s, 178 MB/s 2024-04-25T09:59:54.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.176 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662087 s, 158 MB/s 2024-04-25T09:59:54.183 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.183 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.183 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641412 s, 163 MB/s 2024-04-25T09:59:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699836 s, 150 MB/s 2024-04-25T09:59:54.199 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.199 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719651 s, 146 MB/s 2024-04-25T09:59:54.207 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.207 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703366 s, 149 MB/s 2024-04-25T09:59:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069525 s, 151 MB/s 2024-04-25T09:59:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067439 s, 155 MB/s 2024-04-25T09:59:54.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:54.225 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:54.225 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:54.225 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:59:54.226 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:54.226 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:54.226 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:54.226 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741146 s, 141 MB/s 2024-04-25T09:59:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659485 s, 159 MB/s 2024-04-25T09:59:54.305 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.305 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.305 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070689 s, 148 MB/s 2024-04-25T09:59:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751782 s, 139 MB/s 2024-04-25T09:59:54.321 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.321 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.321 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632929 s, 166 MB/s 2024-04-25T09:59:54.328 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.329 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.329 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683039 s, 154 MB/s 2024-04-25T09:59:54.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.336 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693056 s, 151 MB/s 2024-04-25T09:59:54.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.344 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063047 s, 166 MB/s 2024-04-25T09:59:54.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.352 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709662 s, 148 MB/s 2024-04-25T09:59:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666046 s, 157 MB/s 2024-04-25T09:59:54.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.367 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688863 s, 152 MB/s 2024-04-25T09:59:54.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.375 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069869 s, 150 MB/s 2024-04-25T09:59:54.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.382 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661195 s, 159 MB/s 2024-04-25T09:59:54.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.390 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711438 s, 147 MB/s 2024-04-25T09:59:54.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.398 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712813 s, 147 MB/s 2024-04-25T09:59:54.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.407 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713877 s, 147 MB/s 2024-04-25T09:59:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700226 s, 150 MB/s 2024-04-25T09:59:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.422 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650292 s, 161 MB/s 2024-04-25T09:59:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623118 s, 168 MB/s 2024-04-25T09:59:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699554 s, 150 MB/s 2024-04-25T09:59:54.444 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665027 s, 158 MB/s 2024-04-25T09:59:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668916 s, 157 MB/s 2024-04-25T09:59:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608145 s, 172 MB/s 2024-04-25T09:59:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737312 s, 142 MB/s 2024-04-25T09:59:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652167 s, 161 MB/s 2024-04-25T09:59:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.483 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657009 s, 160 MB/s 2024-04-25T09:59:54.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.490 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686738 s, 153 MB/s 2024-04-25T09:59:54.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.499 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728797 s, 144 MB/s 2024-04-25T09:59:54.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.506 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.506 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611851 s, 171 MB/s 2024-04-25T09:59:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633635 s, 165 MB/s 2024-04-25T09:59:54.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.520 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629989 s, 166 MB/s 2024-04-25T09:59:54.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.528 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724148 s, 145 MB/s 2024-04-25T09:59:54.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:54.529 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:54.529 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:54.529 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:59:54.530 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:54.530 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:54.530 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:54.530 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:54.595 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.596 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769183 s, 136 MB/s 2024-04-25T09:59:54.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.603 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690847 s, 152 MB/s 2024-04-25T09:59:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684972 s, 153 MB/s 2024-04-25T09:59:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655177 s, 160 MB/s 2024-04-25T09:59:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072238 s, 145 MB/s 2024-04-25T09:59:54.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.634 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649239 s, 162 MB/s 2024-04-25T09:59:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755553 s, 139 MB/s 2024-04-25T09:59:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066783 s, 157 MB/s 2024-04-25T09:59:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073868 s, 142 MB/s 2024-04-25T09:59:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685271 s, 153 MB/s 2024-04-25T09:59:54.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.674 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695662 s, 151 MB/s 2024-04-25T09:59:54.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.682 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693784 s, 151 MB/s 2024-04-25T09:59:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631164 s, 166 MB/s 2024-04-25T09:59:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.697 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673816 s, 156 MB/s 2024-04-25T09:59:54.705 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.705 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.705 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711104 s, 147 MB/s 2024-04-25T09:59:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612384 s, 171 MB/s 2024-04-25T09:59:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751649 s, 140 MB/s 2024-04-25T09:59:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677956 s, 155 MB/s 2024-04-25T09:59:54.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.736 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.736 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686238 s, 153 MB/s 2024-04-25T09:59:54.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.743 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628991 s, 167 MB/s 2024-04-25T09:59:54.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.751 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712461 s, 147 MB/s 2024-04-25T09:59:54.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.758 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676824 s, 155 MB/s 2024-04-25T09:59:54.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.766 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694509 s, 151 MB/s 2024-04-25T09:59:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073366 s, 143 MB/s 2024-04-25T09:59:54.782 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.782 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.782 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683674 s, 153 MB/s 2024-04-25T09:59:54.789 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.789 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.789 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623578 s, 168 MB/s 2024-04-25T09:59:54.797 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722475 s, 145 MB/s 2024-04-25T09:59:54.806 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.806 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.806 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701363 s, 150 MB/s 2024-04-25T09:59:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687401 s, 153 MB/s 2024-04-25T09:59:54.821 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.821 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.821 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700974 s, 150 MB/s 2024-04-25T09:59:54.829 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.829 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.829 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710813 s, 148 MB/s 2024-04-25T09:59:54.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.837 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682971 s, 154 MB/s 2024-04-25T09:59:54.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:54.838 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:54.838 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:54.838 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:59:54.838 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:54.838 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:54.838 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:54.839 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:54.905 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.905 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.905 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692966 s, 151 MB/s 2024-04-25T09:59:54.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.914 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698464 s, 150 MB/s 2024-04-25T09:59:54.920 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.921 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.921 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707686 s, 148 MB/s 2024-04-25T09:59:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.928 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665903 s, 157 MB/s 2024-04-25T09:59:54.936 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.936 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.936 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680067 s, 154 MB/s 2024-04-25T09:59:54.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.944 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695648 s, 151 MB/s 2024-04-25T09:59:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636872 s, 165 MB/s 2024-04-25T09:59:54.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.959 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703971 s, 149 MB/s 2024-04-25T09:59:54.967 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761466 s, 138 MB/s 2024-04-25T09:59:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705409 s, 149 MB/s 2024-04-25T09:59:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857327 s, 122 MB/s 2024-04-25T09:59:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689641 s, 152 MB/s 2024-04-25T09:59:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071427 s, 147 MB/s 2024-04-25T09:59:55.009 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.009 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.009 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701666 s, 149 MB/s 2024-04-25T09:59:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.016 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648785 s, 162 MB/s 2024-04-25T09:59:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624881 s, 168 MB/s 2024-04-25T09:59:55.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.033 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829295 s, 126 MB/s 2024-04-25T09:59:55.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.041 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697884 s, 150 MB/s 2024-04-25T09:59:55.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.048 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687505 s, 153 MB/s 2024-04-25T09:59:55.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.057 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738916 s, 142 MB/s 2024-04-25T09:59:55.064 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.064 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.064 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649434 s, 161 MB/s 2024-04-25T09:59:55.072 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.072 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.072 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678793 s, 154 MB/s 2024-04-25T09:59:55.079 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.079 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.079 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663294 s, 158 MB/s 2024-04-25T09:59:55.087 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.087 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.087 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645326 s, 162 MB/s 2024-04-25T09:59:55.095 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.095 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.095 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702167 s, 149 MB/s 2024-04-25T09:59:55.103 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.103 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.103 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692473 s, 151 MB/s 2024-04-25T09:59:55.111 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.111 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.111 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709239 s, 148 MB/s 2024-04-25T09:59:55.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.119 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673568 s, 156 MB/s 2024-04-25T09:59:55.127 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.128 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.128 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844853 s, 124 MB/s 2024-04-25T09:59:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.135 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654084 s, 160 MB/s 2024-04-25T09:59:55.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644254 s, 163 MB/s 2024-04-25T09:59:55.150 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.150 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.150 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630603 s, 166 MB/s 2024-04-25T09:59:55.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:55.151 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:55.151 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:55.151 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:59:55.151 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:55.151 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:55.152 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:55.152 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:55.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.218 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.218 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640064 s, 164 MB/s 2024-04-25T09:59:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736403 s, 142 MB/s 2024-04-25T09:59:55.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.233 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624046 s, 168 MB/s 2024-04-25T09:59:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704256 s, 149 MB/s 2024-04-25T09:59:55.249 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.249 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.249 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713851 s, 147 MB/s 2024-04-25T09:59:55.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.256 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639678 s, 164 MB/s 2024-04-25T09:59:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713906 s, 147 MB/s 2024-04-25T09:59:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663593 s, 158 MB/s 2024-04-25T09:59:55.279 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.279 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.279 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636532 s, 165 MB/s 2024-04-25T09:59:55.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.288 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747218 s, 140 MB/s 2024-04-25T09:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.295 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637486 s, 164 MB/s 2024-04-25T09:59:55.302 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.302 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.302 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666178 s, 157 MB/s 2024-04-25T09:59:55.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.310 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708767 s, 148 MB/s 2024-04-25T09:59:55.318 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.322 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070108 s, 150 MB/s 2024-04-25T09:59:55.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.327 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727411 s, 144 MB/s 2024-04-25T09:59:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728448 s, 144 MB/s 2024-04-25T09:59:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684772 s, 153 MB/s 2024-04-25T09:59:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706873 s, 148 MB/s 2024-04-25T09:59:55.358 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.358 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.358 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681069 s, 154 MB/s 2024-04-25T09:59:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706736 s, 148 MB/s 2024-04-25T09:59:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741815 s, 141 MB/s 2024-04-25T09:59:55.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717049 s, 146 MB/s 2024-04-25T09:59:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.390 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617019 s, 170 MB/s 2024-04-25T09:59:55.397 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.397 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.397 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634996 s, 165 MB/s 2024-04-25T09:59:55.404 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.404 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.404 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635221 s, 165 MB/s 2024-04-25T09:59:55.412 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.412 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.412 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692475 s, 151 MB/s 2024-04-25T09:59:55.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.420 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652927 s, 161 MB/s 2024-04-25T09:59:55.427 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.427 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.427 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619301 s, 169 MB/s 2024-04-25T09:59:55.434 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.434 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.434 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672588 s, 156 MB/s 2024-04-25T09:59:55.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063025 s, 166 MB/s 2024-04-25T09:59:55.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.450 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.450 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699608 s, 150 MB/s 2024-04-25T09:59:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658882 s, 159 MB/s 2024-04-25T09:59:55.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:55.458 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:55.458 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:55.458 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:59:55.458 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:55.459 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:55.459 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:55.459 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683474 s, 153 MB/s 2024-04-25T09:59:55.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.532 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711736 s, 147 MB/s 2024-04-25T09:59:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676652 s, 155 MB/s 2024-04-25T09:59:55.550 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.550 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.550 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864097 s, 121 MB/s 2024-04-25T09:59:55.557 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.557 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.557 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663365 s, 158 MB/s 2024-04-25T09:59:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614337 s, 171 MB/s 2024-04-25T09:59:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.572 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066471 s, 158 MB/s 2024-04-25T09:59:55.580 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.580 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.580 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751021 s, 140 MB/s 2024-04-25T09:59:55.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.588 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692844 s, 151 MB/s 2024-04-25T09:59:55.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.596 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732948 s, 143 MB/s 2024-04-25T09:59:55.604 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.604 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.604 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063972 s, 164 MB/s 2024-04-25T09:59:55.612 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.612 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.612 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693272 s, 151 MB/s 2024-04-25T09:59:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689296 s, 152 MB/s 2024-04-25T09:59:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715519 s, 147 MB/s 2024-04-25T09:59:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692824 s, 151 MB/s 2024-04-25T09:59:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704415 s, 149 MB/s 2024-04-25T09:59:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716453 s, 146 MB/s 2024-04-25T09:59:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655971 s, 160 MB/s 2024-04-25T09:59:55.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665752 s, 158 MB/s 2024-04-25T09:59:55.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.675 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747382 s, 140 MB/s 2024-04-25T09:59:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.683 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688012 s, 152 MB/s 2024-04-25T09:59:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070282 s, 149 MB/s 2024-04-25T09:59:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654017 s, 160 MB/s 2024-04-25T09:59:55.707 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.707 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.707 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824311 s, 127 MB/s 2024-04-25T09:59:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686273 s, 153 MB/s 2024-04-25T09:59:55.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.723 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715548 s, 147 MB/s 2024-04-25T09:59:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676406 s, 155 MB/s 2024-04-25T09:59:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062494 s, 168 MB/s 2024-04-25T09:59:55.746 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.746 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.746 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733902 s, 143 MB/s 2024-04-25T09:59:55.753 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.753 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.753 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632289 s, 166 MB/s 2024-04-25T09:59:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.763 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808785 s, 130 MB/s 2024-04-25T09:59:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007074 s, 148 MB/s 2024-04-25T09:59:55.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:55.771 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:55.772 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:55.772 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:59:55.772 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:55.772 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:55.772 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:55.772 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689389 s, 152 MB/s 2024-04-25T09:59:55.844 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.844 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.844 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671846 s, 156 MB/s 2024-04-25T09:59:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658561 s, 159 MB/s 2024-04-25T09:59:55.860 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.860 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.860 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066494 s, 158 MB/s 2024-04-25T09:59:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678392 s, 155 MB/s 2024-04-25T09:59:55.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.875 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690037 s, 152 MB/s 2024-04-25T09:59:55.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.883 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712819 s, 147 MB/s 2024-04-25T09:59:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641257 s, 164 MB/s 2024-04-25T09:59:55.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.899 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750176 s, 140 MB/s 2024-04-25T09:59:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641808 s, 163 MB/s 2024-04-25T09:59:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681608 s, 154 MB/s 2024-04-25T09:59:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625264 s, 168 MB/s 2024-04-25T09:59:55.928 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.928 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.928 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644718 s, 163 MB/s 2024-04-25T09:59:55.936 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.936 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.936 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621071 s, 169 MB/s 2024-04-25T09:59:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707041 s, 148 MB/s 2024-04-25T09:59:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686252 s, 153 MB/s 2024-04-25T09:59:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667849 s, 157 MB/s 2024-04-25T09:59:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634768 s, 165 MB/s 2024-04-25T09:59:55.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.974 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722136 s, 145 MB/s 2024-04-25T09:59:55.981 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.981 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.982 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639237 s, 164 MB/s 2024-04-25T09:59:55.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.990 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719881 s, 146 MB/s 2024-04-25T09:59:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713921 s, 147 MB/s 2024-04-25T09:59:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658825 s, 159 MB/s 2024-04-25T09:59:56.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.013 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619705 s, 169 MB/s 2024-04-25T09:59:56.020 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.020 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.020 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684202 s, 153 MB/s 2024-04-25T09:59:56.027 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.027 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.027 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634401 s, 165 MB/s 2024-04-25T09:59:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672834 s, 156 MB/s 2024-04-25T09:59:56.042 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683227 s, 153 MB/s 2024-04-25T09:59:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797063 s, 132 MB/s 2024-04-25T09:59:56.059 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.059 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725506 s, 145 MB/s 2024-04-25T09:59:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626334 s, 167 MB/s 2024-04-25T09:59:56.074 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.074 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.074 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648023 s, 162 MB/s 2024-04-25T09:59:56.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:56.075 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:56.075 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:56.075 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:59:56.076 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:56.076 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:56.076 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:56.076 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:56.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.140 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620712 s, 169 MB/s 2024-04-25T09:59:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656109 s, 160 MB/s 2024-04-25T09:59:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069729 s, 150 MB/s 2024-04-25T09:59:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064756 s, 162 MB/s 2024-04-25T09:59:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.171 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713479 s, 147 MB/s 2024-04-25T09:59:56.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.178 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619951 s, 169 MB/s 2024-04-25T09:59:56.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.186 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717583 s, 146 MB/s 2024-04-25T09:59:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628745 s, 167 MB/s 2024-04-25T09:59:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692097 s, 152 MB/s 2024-04-25T09:59:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662706 s, 158 MB/s 2024-04-25T09:59:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702327 s, 149 MB/s 2024-04-25T09:59:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650558 s, 161 MB/s 2024-04-25T09:59:56.231 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.231 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.231 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656567 s, 160 MB/s 2024-04-25T09:59:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716808 s, 146 MB/s 2024-04-25T09:59:56.247 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.247 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709523 s, 148 MB/s 2024-04-25T09:59:56.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.256 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731108 s, 143 MB/s 2024-04-25T09:59:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066153 s, 159 MB/s 2024-04-25T09:59:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629788 s, 166 MB/s 2024-04-25T09:59:56.278 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.278 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.278 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637909 s, 164 MB/s 2024-04-25T09:59:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819881 s, 128 MB/s 2024-04-25T09:59:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710233 s, 148 MB/s 2024-04-25T09:59:56.303 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.303 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.303 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708019 s, 148 MB/s 2024-04-25T09:59:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719619 s, 146 MB/s 2024-04-25T09:59:56.320 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.320 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.320 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791954 s, 132 MB/s 2024-04-25T09:59:56.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.328 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652532 s, 161 MB/s 2024-04-25T09:59:56.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.335 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665802 s, 157 MB/s 2024-04-25T09:59:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806452 s, 130 MB/s 2024-04-25T09:59:56.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.353 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.353 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719422 s, 146 MB/s 2024-04-25T09:59:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620972 s, 169 MB/s 2024-04-25T09:59:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.367 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641795 s, 163 MB/s 2024-04-25T09:59:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078762 s, 133 MB/s 2024-04-25T09:59:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677961 s, 155 MB/s 2024-04-25T09:59:56.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:56.385 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:56.385 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:56.385 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:59:56.385 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:56.385 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:56.385 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:56.385 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606073 s, 173 MB/s 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819658 s, 128 MB/s 2024-04-25T09:59:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697087 s, 150 MB/s 2024-04-25T09:59:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827837 s, 127 MB/s 2024-04-25T09:59:56.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.485 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839508 s, 125 MB/s 2024-04-25T09:59:56.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.494 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.494 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814868 s, 129 MB/s 2024-04-25T09:59:56.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.501 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650294 s, 161 MB/s 2024-04-25T09:59:56.510 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.510 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.510 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829917 s, 126 MB/s 2024-04-25T09:59:56.518 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.518 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.518 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735673 s, 143 MB/s 2024-04-25T09:59:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726654 s, 144 MB/s 2024-04-25T09:59:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705143 s, 149 MB/s 2024-04-25T09:59:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.543 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735554 s, 143 MB/s 2024-04-25T09:59:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070059 s, 150 MB/s 2024-04-25T09:59:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680177 s, 154 MB/s 2024-04-25T09:59:56.565 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.566 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.566 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640453 s, 164 MB/s 2024-04-25T09:59:56.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.573 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065978 s, 159 MB/s 2024-04-25T09:59:56.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.581 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707657 s, 148 MB/s 2024-04-25T09:59:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705846 s, 149 MB/s 2024-04-25T09:59:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703364 s, 149 MB/s 2024-04-25T09:59:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638035 s, 164 MB/s 2024-04-25T09:59:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637036 s, 165 MB/s 2024-04-25T09:59:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725846 s, 144 MB/s 2024-04-25T09:59:56.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.628 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.628 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670929 s, 156 MB/s 2024-04-25T09:59:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742198 s, 141 MB/s 2024-04-25T09:59:56.644 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.644 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.644 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695714 s, 151 MB/s 2024-04-25T09:59:56.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.652 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677547 s, 155 MB/s 2024-04-25T09:59:56.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.659 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666489 s, 157 MB/s 2024-04-25T09:59:56.667 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.667 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.668 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764343 s, 137 MB/s 2024-04-25T09:59:56.676 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.676 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.676 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717762 s, 146 MB/s 2024-04-25T09:59:56.683 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.683 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.683 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644772 s, 163 MB/s 2024-04-25T09:59:56.691 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.691 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.691 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073193 s, 143 MB/s 2024-04-25T09:59:56.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750722 s, 140 MB/s 2024-04-25T09:59:56.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:56.701 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:56.701 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:56.701 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:59:56.701 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:56.701 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:56.701 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:56.701 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:56.767 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.767 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.767 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061782 s, 170 MB/s 2024-04-25T09:59:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720876 s, 145 MB/s 2024-04-25T09:59:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617722 s, 170 MB/s 2024-04-25T09:59:56.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.787 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637187 s, 165 MB/s 2024-04-25T09:59:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630691 s, 166 MB/s 2024-04-25T09:59:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857592 s, 122 MB/s 2024-04-25T09:59:56.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.812 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684481 s, 153 MB/s 2024-04-25T09:59:56.821 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.821 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.821 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845602 s, 124 MB/s 2024-04-25T09:59:56.829 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.829 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.829 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716875 s, 146 MB/s 2024-04-25T09:59:56.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.837 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730049 s, 144 MB/s 2024-04-25T09:59:56.844 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.845 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.845 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633177 s, 166 MB/s 2024-04-25T09:59:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695578 s, 151 MB/s 2024-04-25T09:59:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689999 s, 152 MB/s 2024-04-25T09:59:56.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.867 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624428 s, 168 MB/s 2024-04-25T09:59:56.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.875 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687635 s, 152 MB/s 2024-04-25T09:59:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.883 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668366 s, 157 MB/s 2024-04-25T09:59:56.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.890 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631567 s, 166 MB/s 2024-04-25T09:59:56.897 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.897 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.897 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618286 s, 170 MB/s 2024-04-25T09:59:56.905 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.905 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.905 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006864 s, 153 MB/s 2024-04-25T09:59:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064255 s, 163 MB/s 2024-04-25T09:59:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645719 s, 162 MB/s 2024-04-25T09:59:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682175 s, 154 MB/s 2024-04-25T09:59:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073621 s, 142 MB/s 2024-04-25T09:59:56.942 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.942 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.942 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617884 s, 170 MB/s 2024-04-25T09:59:56.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.950 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673381 s, 156 MB/s 2024-04-25T09:59:56.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.958 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693498 s, 151 MB/s 2024-04-25T09:59:56.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.965 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620259 s, 169 MB/s 2024-04-25T09:59:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700456 s, 150 MB/s 2024-04-25T09:59:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689079 s, 152 MB/s 2024-04-25T09:59:56.988 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671035 s, 156 MB/s 2024-04-25T09:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006248 s, 168 MB/s 2024-04-25T09:59:57.003 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.003 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.003 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705013 s, 149 MB/s 2024-04-25T09:59:57.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:57.005 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:57.005 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:57.005 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:59:57.005 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:57.005 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:57.005 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:57.005 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:57.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.070 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715584 s, 147 MB/s 2024-04-25T09:59:57.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.078 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691864 s, 152 MB/s 2024-04-25T09:59:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730664 s, 144 MB/s 2024-04-25T09:59:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720346 s, 146 MB/s 2024-04-25T09:59:57.102 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.102 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.102 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700222 s, 150 MB/s 2024-04-25T09:59:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006782 s, 155 MB/s 2024-04-25T09:59:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710387 s, 148 MB/s 2024-04-25T09:59:57.125 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.125 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.125 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629918 s, 166 MB/s 2024-04-25T09:59:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679322 s, 154 MB/s 2024-04-25T09:59:57.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.140 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065056 s, 161 MB/s 2024-04-25T09:59:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630195 s, 166 MB/s 2024-04-25T09:59:57.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.155 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633363 s, 166 MB/s 2024-04-25T09:59:57.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.162 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678475 s, 155 MB/s 2024-04-25T09:59:57.170 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.170 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.170 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667626 s, 157 MB/s 2024-04-25T09:59:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710805 s, 148 MB/s 2024-04-25T09:59:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678854 s, 154 MB/s 2024-04-25T09:59:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715572 s, 147 MB/s 2024-04-25T09:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.202 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670223 s, 156 MB/s 2024-04-25T09:59:57.208 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.209 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006398 s, 164 MB/s 2024-04-25T09:59:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650483 s, 161 MB/s 2024-04-25T09:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.223 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627635 s, 167 MB/s 2024-04-25T09:59:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703136 s, 149 MB/s 2024-04-25T09:59:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680514 s, 154 MB/s 2024-04-25T09:59:57.246 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.246 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.246 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656282 s, 160 MB/s 2024-04-25T09:59:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728483 s, 144 MB/s 2024-04-25T09:59:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.263 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718856 s, 146 MB/s 2024-04-25T09:59:57.270 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.270 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.270 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690569 s, 152 MB/s 2024-04-25T09:59:57.279 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.279 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.279 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756763 s, 139 MB/s 2024-04-25T09:59:57.286 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703433 s, 149 MB/s 2024-04-25T09:59:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656757 s, 160 MB/s 2024-04-25T09:59:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632418 s, 166 MB/s 2024-04-25T09:59:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732924 s, 143 MB/s 2024-04-25T09:59:57.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:57.311 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:57.311 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:57.311 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:59:57.311 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:57.311 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:57.311 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:57.311 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842642 s, 124 MB/s 2024-04-25T09:59:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.385 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660119 s, 159 MB/s 2024-04-25T09:59:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701445 s, 149 MB/s 2024-04-25T09:59:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747589 s, 140 MB/s 2024-04-25T09:59:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731742 s, 143 MB/s 2024-04-25T09:59:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689086 s, 152 MB/s 2024-04-25T09:59:57.425 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.425 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.425 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696178 s, 151 MB/s 2024-04-25T09:59:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615086 s, 170 MB/s 2024-04-25T09:59:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668045 s, 157 MB/s 2024-04-25T09:59:57.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.448 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070147 s, 149 MB/s 2024-04-25T09:59:57.455 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.455 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.455 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660259 s, 159 MB/s 2024-04-25T09:59:57.463 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.463 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.463 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702481 s, 149 MB/s 2024-04-25T09:59:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790232 s, 133 MB/s 2024-04-25T09:59:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715537 s, 147 MB/s 2024-04-25T09:59:57.488 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.488 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.488 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074927 s, 140 MB/s 2024-04-25T09:59:57.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.497 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078065 s, 134 MB/s 2024-04-25T09:59:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698313 s, 150 MB/s 2024-04-25T09:59:57.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.513 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006996 s, 150 MB/s 2024-04-25T09:59:57.521 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.521 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.521 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719857 s, 146 MB/s 2024-04-25T09:59:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006119 s, 171 MB/s 2024-04-25T09:59:57.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.535 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643738 s, 163 MB/s 2024-04-25T09:59:57.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.545 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865394 s, 121 MB/s 2024-04-25T09:59:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071128 s, 147 MB/s 2024-04-25T09:59:57.560 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.560 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.560 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646245 s, 162 MB/s 2024-04-25T09:59:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702497 s, 149 MB/s 2024-04-25T09:59:57.577 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.577 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.577 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772208 s, 136 MB/s 2024-04-25T09:59:57.586 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.586 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.586 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763739 s, 137 MB/s 2024-04-25T09:59:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.596 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849693 s, 123 MB/s 2024-04-25T09:59:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655922 s, 160 MB/s 2024-04-25T09:59:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757073 s, 139 MB/s 2024-04-25T09:59:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737872 s, 142 MB/s 2024-04-25T09:59:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655888 s, 160 MB/s 2024-04-25T09:59:57.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:57.629 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:57.629 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:57.629 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:59:57.629 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:57.629 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:57.629 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:57.629 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:57.693 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.693 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.693 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669531 s, 157 MB/s 2024-04-25T09:59:57.701 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.701 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.701 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691409 s, 152 MB/s 2024-04-25T09:59:57.709 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.709 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.709 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720542 s, 146 MB/s 2024-04-25T09:59:57.717 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.717 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.717 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710634 s, 148 MB/s 2024-04-25T09:59:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707641 s, 148 MB/s 2024-04-25T09:59:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676669 s, 155 MB/s 2024-04-25T09:59:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843484 s, 124 MB/s 2024-04-25T09:59:57.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.750 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.750 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638683 s, 164 MB/s 2024-04-25T09:59:57.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.758 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070308 s, 149 MB/s 2024-04-25T09:59:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704658 s, 149 MB/s 2024-04-25T09:59:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.773 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662826 s, 158 MB/s 2024-04-25T09:59:57.781 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750335 s, 140 MB/s 2024-04-25T09:59:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708309 s, 148 MB/s 2024-04-25T09:59:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061915 s, 169 MB/s 2024-04-25T09:59:57.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.805 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711336 s, 147 MB/s 2024-04-25T09:59:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063176 s, 166 MB/s 2024-04-25T09:59:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704482 s, 149 MB/s 2024-04-25T09:59:57.827 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.827 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.827 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665408 s, 158 MB/s 2024-04-25T09:59:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631146 s, 166 MB/s 2024-04-25T09:59:57.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.842 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700816 s, 150 MB/s 2024-04-25T09:59:57.849 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.850 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635696 s, 165 MB/s 2024-04-25T09:59:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684924 s, 153 MB/s 2024-04-25T09:59:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661589 s, 158 MB/s 2024-04-25T09:59:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683088 s, 154 MB/s 2024-04-25T09:59:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618513 s, 170 MB/s 2024-04-25T09:59:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682779 s, 154 MB/s 2024-04-25T09:59:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065696 s, 160 MB/s 2024-04-25T09:59:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707029 s, 148 MB/s 2024-04-25T09:59:57.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.911 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672928 s, 156 MB/s 2024-04-25T09:59:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690029 s, 152 MB/s 2024-04-25T09:59:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701008 s, 150 MB/s 2024-04-25T09:59:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:57.934 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611851 s, 171 MB/s 2024-04-25T09:59:57.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:57.935 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:57.935 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:57.935 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:59:57.935 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:57.935 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:57.935 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:57.935 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713577 s, 147 MB/s 2024-04-25T09:59:58.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.009 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774279 s, 135 MB/s 2024-04-25T09:59:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689541 s, 152 MB/s 2024-04-25T09:59:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693824 s, 151 MB/s 2024-04-25T09:59:58.032 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.032 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.032 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684512 s, 153 MB/s 2024-04-25T09:59:58.039 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.039 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.039 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629233 s, 167 MB/s 2024-04-25T09:59:58.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.047 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653212 s, 161 MB/s 2024-04-25T09:59:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752001 s, 139 MB/s 2024-04-25T09:59:58.062 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.062 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.062 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652931 s, 161 MB/s 2024-04-25T09:59:58.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.070 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647918 s, 162 MB/s 2024-04-25T09:59:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708957 s, 148 MB/s 2024-04-25T09:59:58.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.086 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706844 s, 148 MB/s 2024-04-25T09:59:58.094 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.094 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.094 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718023 s, 146 MB/s 2024-04-25T09:59:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702949 s, 149 MB/s 2024-04-25T09:59:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707405 s, 148 MB/s 2024-04-25T09:59:58.117 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.118 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703788 s, 149 MB/s 2024-04-25T09:59:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701004 s, 150 MB/s 2024-04-25T09:59:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.134 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723773 s, 145 MB/s 2024-04-25T09:59:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706584 s, 148 MB/s 2024-04-25T09:59:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063694 s, 165 MB/s 2024-04-25T09:59:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.157 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638003 s, 164 MB/s 2024-04-25T09:59:58.164 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.164 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.164 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642355 s, 163 MB/s 2024-04-25T09:59:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699326 s, 150 MB/s 2024-04-25T09:59:58.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.179 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606184 s, 173 MB/s 2024-04-25T09:59:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692808 s, 151 MB/s 2024-04-25T09:59:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659573 s, 159 MB/s 2024-04-25T09:59:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.202 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065074 s, 161 MB/s 2024-04-25T09:59:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750291 s, 140 MB/s 2024-04-25T09:59:58.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.217 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628629 s, 167 MB/s 2024-04-25T09:59:58.225 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.225 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.225 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715551 s, 147 MB/s 2024-04-25T09:59:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695319 s, 151 MB/s 2024-04-25T09:59:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645089 s, 163 MB/s 2024-04-25T09:59:58.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:58.242 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:58.242 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:58.242 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:59:58.242 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:58.242 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:58.242 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:58.242 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.308 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728567 s, 144 MB/s 2024-04-25T09:59:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684973 s, 153 MB/s 2024-04-25T09:59:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662283 s, 158 MB/s 2024-04-25T09:59:58.330 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.331 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.331 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070792 s, 148 MB/s 2024-04-25T09:59:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689621 s, 152 MB/s 2024-04-25T09:59:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643575 s, 163 MB/s 2024-04-25T09:59:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657068 s, 160 MB/s 2024-04-25T09:59:58.361 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.361 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.361 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658271 s, 159 MB/s 2024-04-25T09:59:58.368 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.368 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.369 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068266 s, 154 MB/s 2024-04-25T09:59:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726782 s, 144 MB/s 2024-04-25T09:59:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631364 s, 166 MB/s 2024-04-25T09:59:58.391 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.391 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.391 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649824 s, 161 MB/s 2024-04-25T09:59:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627582 s, 167 MB/s 2024-04-25T09:59:58.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.407 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697805 s, 150 MB/s 2024-04-25T09:59:58.414 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.414 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.414 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649715 s, 161 MB/s 2024-04-25T09:59:58.421 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.422 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.422 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692743 s, 151 MB/s 2024-04-25T09:59:58.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.430 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717223 s, 146 MB/s 2024-04-25T09:59:58.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.438 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070016 s, 150 MB/s 2024-04-25T09:59:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659471 s, 159 MB/s 2024-04-25T09:59:58.452 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.452 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.452 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640921 s, 164 MB/s 2024-04-25T09:59:58.460 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.460 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.460 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064019 s, 164 MB/s 2024-04-25T09:59:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.467 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637872 s, 164 MB/s 2024-04-25T09:59:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.475 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720693 s, 145 MB/s 2024-04-25T09:59:58.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.483 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698366 s, 150 MB/s 2024-04-25T09:59:58.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.491 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639399 s, 164 MB/s 2024-04-25T09:59:58.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.498 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622416 s, 168 MB/s 2024-04-25T09:59:58.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.505 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725229 s, 145 MB/s 2024-04-25T09:59:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.514 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.514 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710644 s, 148 MB/s 2024-04-25T09:59:58.522 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.522 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.522 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714773 s, 147 MB/s 2024-04-25T09:59:58.529 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.529 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.529 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653249 s, 161 MB/s 2024-04-25T09:59:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680768 s, 154 MB/s 2024-04-25T09:59:58.546 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.546 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.546 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682844 s, 154 MB/s 2024-04-25T09:59:58.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:58.546 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:58.547 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:58.547 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:59:58.547 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:58.547 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:58.547 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:58.547 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683596 s, 153 MB/s 2024-04-25T09:59:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731976 s, 143 MB/s 2024-04-25T09:59:58.626 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.627 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686389 s, 153 MB/s 2024-04-25T09:59:58.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.634 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636513 s, 165 MB/s 2024-04-25T09:59:58.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.641 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609474 s, 172 MB/s 2024-04-25T09:59:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082738 s, 127 MB/s 2024-04-25T09:59:58.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.658 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727692 s, 144 MB/s 2024-04-25T09:59:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632739 s, 166 MB/s 2024-04-25T09:59:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.673 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607647 s, 173 MB/s 2024-04-25T09:59:58.679 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061881 s, 169 MB/s 2024-04-25T09:59:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660885 s, 159 MB/s 2024-04-25T09:59:58.695 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.695 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.695 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071367 s, 147 MB/s 2024-04-25T09:59:58.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.703 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694627 s, 151 MB/s 2024-04-25T09:59:58.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.711 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679939 s, 154 MB/s 2024-04-25T09:59:58.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.719 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685511 s, 153 MB/s 2024-04-25T09:59:58.726 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.726 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.726 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652027 s, 161 MB/s 2024-04-25T09:59:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646673 s, 162 MB/s 2024-04-25T09:59:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656673 s, 160 MB/s 2024-04-25T09:59:58.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.748 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629944 s, 166 MB/s 2024-04-25T09:59:58.755 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.755 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.755 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621224 s, 169 MB/s 2024-04-25T09:59:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646557 s, 162 MB/s 2024-04-25T09:59:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649875 s, 161 MB/s 2024-04-25T09:59:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654597 s, 160 MB/s 2024-04-25T09:59:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665954 s, 157 MB/s 2024-04-25T09:59:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700615 s, 150 MB/s 2024-04-25T09:59:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699818 s, 150 MB/s 2024-04-25T09:59:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638753 s, 164 MB/s 2024-04-25T09:59:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.818 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683292 s, 153 MB/s 2024-04-25T09:59:58.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.823 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637047 s, 165 MB/s 2024-04-25T09:59:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626645 s, 167 MB/s 2024-04-25T09:59:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680071 s, 154 MB/s 2024-04-25T09:59:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655499 s, 160 MB/s 2024-04-25T09:59:58.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:58.846 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:58.846 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:58.846 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:59:58.846 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:58.847 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:58.847 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:58.847 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649869 s, 161 MB/s 2024-04-25T09:59:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694847 s, 151 MB/s 2024-04-25T09:59:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074142 s, 141 MB/s 2024-04-25T09:59:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.934 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.935 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626627 s, 167 MB/s 2024-04-25T09:59:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.943 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.943 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701657 s, 149 MB/s 2024-04-25T09:59:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694745 s, 151 MB/s 2024-04-25T09:59:58.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.958 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666097 s, 157 MB/s 2024-04-25T09:59:58.966 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.966 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.966 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695573 s, 151 MB/s 2024-04-25T09:59:58.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.974 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700449 s, 150 MB/s 2024-04-25T09:59:58.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.982 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708949 s, 148 MB/s 2024-04-25T09:59:58.991 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.991 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.991 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858857 s, 122 MB/s 2024-04-25T09:59:58.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:58.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:58.999 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702944 s, 149 MB/s 2024-04-25T09:59:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623503 s, 168 MB/s 2024-04-25T09:59:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066485 s, 158 MB/s 2024-04-25T09:59:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644053 s, 163 MB/s 2024-04-25T09:59:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675812 s, 155 MB/s 2024-04-25T09:59:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694306 s, 151 MB/s 2024-04-25T09:59:59.043 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.044 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.044 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062783 s, 167 MB/s 2024-04-25T09:59:59.052 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.052 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.052 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738547 s, 142 MB/s 2024-04-25T09:59:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642667 s, 163 MB/s 2024-04-25T09:59:59.067 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.067 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730143 s, 144 MB/s 2024-04-25T09:59:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684857 s, 153 MB/s 2024-04-25T09:59:59.083 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.084 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754852 s, 139 MB/s 2024-04-25T09:59:59.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.091 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659113 s, 159 MB/s 2024-04-25T09:59:59.099 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.099 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.099 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652055 s, 161 MB/s 2024-04-25T09:59:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064251 s, 163 MB/s 2024-04-25T09:59:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738081 s, 142 MB/s 2024-04-25T09:59:59.122 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.122 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.122 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708166 s, 148 MB/s 2024-04-25T09:59:59.130 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.130 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.130 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715351 s, 147 MB/s 2024-04-25T09:59:59.138 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.138 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.138 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709712 s, 148 MB/s 2024-04-25T09:59:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.146 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638155 s, 164 MB/s 2024-04-25T09:59:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705715 s, 149 MB/s 2024-04-25T09:59:59.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:59.155 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:59.155 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:59.155 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:59:59.155 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:59.155 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:59.155 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:59.155 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:59.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722398 s, 145 MB/s 2024-04-25T09:59:59.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.228 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.228 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618169 s, 170 MB/s 2024-04-25T09:59:59.235 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.235 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.235 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063064 s, 166 MB/s 2024-04-25T09:59:59.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.242 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066233 s, 158 MB/s 2024-04-25T09:59:59.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.250 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706204 s, 148 MB/s 2024-04-25T09:59:59.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.258 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699531 s, 150 MB/s 2024-04-25T09:59:59.265 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.265 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.265 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633562 s, 166 MB/s 2024-04-25T09:59:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694903 s, 151 MB/s 2024-04-25T09:59:59.281 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.281 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.281 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724834 s, 145 MB/s 2024-04-25T09:59:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704648 s, 149 MB/s 2024-04-25T09:59:59.297 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.297 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.297 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071524 s, 147 MB/s 2024-04-25T09:59:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717132 s, 146 MB/s 2024-04-25T09:59:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.314 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699664 s, 150 MB/s 2024-04-25T09:59:59.321 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.321 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.321 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712909 s, 147 MB/s 2024-04-25T09:59:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.329 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657197 s, 160 MB/s 2024-04-25T09:59:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064171 s, 163 MB/s 2024-04-25T09:59:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653745 s, 160 MB/s 2024-04-25T09:59:59.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.352 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764299 s, 137 MB/s 2024-04-25T09:59:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068075 s, 154 MB/s 2024-04-25T09:59:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067665 s, 155 MB/s 2024-04-25T09:59:59.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.375 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629185 s, 167 MB/s 2024-04-25T09:59:59.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.382 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612721 s, 171 MB/s 2024-04-25T09:59:59.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.390 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717626 s, 146 MB/s 2024-04-25T09:59:59.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.398 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712084 s, 147 MB/s 2024-04-25T09:59:59.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.406 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631174 s, 166 MB/s 2024-04-25T09:59:59.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.413 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714546 s, 147 MB/s 2024-04-25T09:59:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.421 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681112 s, 154 MB/s 2024-04-25T09:59:59.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.431 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.431 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716537 s, 146 MB/s 2024-04-25T09:59:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687376 s, 153 MB/s 2024-04-25T09:59:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721759 s, 145 MB/s 2024-04-25T09:59:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711576 s, 147 MB/s 2024-04-25T09:59:59.460 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.460 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.460 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614908 s, 171 MB/s 2024-04-25T09:59:59.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:59.461 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:59.461 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:59.461 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:59:59.462 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:59.462 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:59.462 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:59.462 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:59.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.527 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073938 s, 142 MB/s 2024-04-25T09:59:59.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.535 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710555 s, 148 MB/s 2024-04-25T09:59:59.542 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.542 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.542 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633045 s, 166 MB/s 2024-04-25T09:59:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750721 s, 140 MB/s 2024-04-25T09:59:59.560 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.560 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.560 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080431 s, 130 MB/s 2024-04-25T09:59:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681888 s, 154 MB/s 2024-04-25T09:59:59.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.576 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.576 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712275 s, 147 MB/s 2024-04-25T09:59:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697172 s, 150 MB/s 2024-04-25T09:59:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692442 s, 151 MB/s 2024-04-25T09:59:59.599 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.599 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.599 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643627 s, 163 MB/s 2024-04-25T09:59:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069456 s, 151 MB/s 2024-04-25T09:59:59.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.615 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749006 s, 140 MB/s 2024-04-25T09:59:59.623 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.623 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.623 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711912 s, 147 MB/s 2024-04-25T09:59:59.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.630 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654317 s, 160 MB/s 2024-04-25T09:59:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619157 s, 169 MB/s 2024-04-25T09:59:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.645 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.646 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704709 s, 149 MB/s 2024-04-25T09:59:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667854 s, 157 MB/s 2024-04-25T09:59:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810487 s, 129 MB/s 2024-04-25T09:59:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751302 s, 140 MB/s 2024-04-25T09:59:59.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.678 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660044 s, 159 MB/s 2024-04-25T09:59:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725579 s, 145 MB/s 2024-04-25T09:59:59.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.694 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691882 s, 152 MB/s 2024-04-25T09:59:59.702 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.702 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.702 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070213 s, 149 MB/s 2024-04-25T09:59:59.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.710 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741622 s, 141 MB/s 2024-04-25T09:59:59.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.719 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730925 s, 143 MB/s 2024-04-25T09:59:59.726 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.726 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.726 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694559 s, 151 MB/s 2024-04-25T09:59:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699231 s, 150 MB/s 2024-04-25T09:59:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636588 s, 165 MB/s 2024-04-25T09:59:59.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.749 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651279 s, 161 MB/s 2024-04-25T09:59:59.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.757 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672822 s, 156 MB/s 2024-04-25T09:59:59.764 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.764 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691739 s, 152 MB/s 2024-04-25T09:59:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682537 s, 154 MB/s 2024-04-25T09:59:59.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:59.773 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:59:59.774 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T09:59:59.774 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:59:59.774 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:59:59.774 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:59:59.774 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T09:59:59.774 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T09:59:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.839 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674353 s, 155 MB/s 2024-04-25T09:59:59.846 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.846 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.846 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674701 s, 155 MB/s 2024-04-25T09:59:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709412 s, 148 MB/s 2024-04-25T09:59:59.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.862 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.862 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666723 s, 157 MB/s 2024-04-25T09:59:59.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.869 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705377 s, 149 MB/s 2024-04-25T09:59:59.877 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.877 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.877 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657623 s, 159 MB/s 2024-04-25T09:59:59.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.885 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698882 s, 150 MB/s 2024-04-25T09:59:59.892 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.892 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.893 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696674 s, 151 MB/s 2024-04-25T09:59:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639245 s, 164 MB/s 2024-04-25T09:59:59.907 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.907 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.907 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597269 s, 176 MB/s 2024-04-25T09:59:59.915 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.915 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.915 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696508 s, 151 MB/s 2024-04-25T09:59:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697803 s, 150 MB/s 2024-04-25T09:59:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665305 s, 158 MB/s 2024-04-25T09:59:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646788 s, 162 MB/s 2024-04-25T09:59:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074363 s, 141 MB/s 2024-04-25T09:59:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725509 s, 145 MB/s 2024-04-25T09:59:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679877 s, 154 MB/s 2024-04-25T09:59:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.969 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670991 s, 156 MB/s 2024-04-25T09:59:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721835 s, 145 MB/s 2024-04-25T09:59:59.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.985 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.985 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621103 s, 169 MB/s 2024-04-25T09:59:59.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T09:59:59.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T09:59:59.992 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677702 s, 155 MB/s 2024-04-25T09:59:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064929 s, 161 MB/s 2024-04-25T10:00:00.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.007 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703702 s, 149 MB/s 2024-04-25T10:00:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663209 s, 158 MB/s 2024-04-25T10:00:00.023 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.023 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.023 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697736 s, 150 MB/s 2024-04-25T10:00:00.031 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.031 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.031 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751715 s, 139 MB/s 2024-04-25T10:00:00.038 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639284 s, 164 MB/s 2024-04-25T10:00:00.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.047 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717697 s, 146 MB/s 2024-04-25T10:00:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654495 s, 160 MB/s 2024-04-25T10:00:00.062 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.062 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.062 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726402 s, 144 MB/s 2024-04-25T10:00:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679277 s, 154 MB/s 2024-04-25T10:00:00.079 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.079 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.079 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697295 s, 150 MB/s 2024-04-25T10:00:00.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:00.079 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:00.079 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:00.079 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:00:00.080 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:00.080 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:00.080 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:00.080 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:00.143 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.143 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.143 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655787 s, 160 MB/s 2024-04-25T10:00:00.152 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.152 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.152 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823279 s, 127 MB/s 2024-04-25T10:00:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741893 s, 141 MB/s 2024-04-25T10:00:00.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698643 s, 150 MB/s 2024-04-25T10:00:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692816 s, 151 MB/s 2024-04-25T10:00:00.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.184 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699087 s, 150 MB/s 2024-04-25T10:00:00.191 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.191 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626739 s, 167 MB/s 2024-04-25T10:00:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.199 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.199 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621953 s, 169 MB/s 2024-04-25T10:00:00.206 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.206 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.206 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691706 s, 152 MB/s 2024-04-25T10:00:00.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.215 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695167 s, 151 MB/s 2024-04-25T10:00:00.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.222 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695504 s, 151 MB/s 2024-04-25T10:00:00.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.230 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714828 s, 147 MB/s 2024-04-25T10:00:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653746 s, 160 MB/s 2024-04-25T10:00:00.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.245 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674467 s, 155 MB/s 2024-04-25T10:00:00.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.252 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611776 s, 171 MB/s 2024-04-25T10:00:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697626 s, 150 MB/s 2024-04-25T10:00:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741273 s, 141 MB/s 2024-04-25T10:00:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.277 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704711 s, 149 MB/s 2024-04-25T10:00:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.285 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745664 s, 141 MB/s 2024-04-25T10:00:00.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.292 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648117 s, 162 MB/s 2024-04-25T10:00:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827252 s, 127 MB/s 2024-04-25T10:00:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649577 s, 161 MB/s 2024-04-25T10:00:00.317 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.317 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.317 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701695 s, 149 MB/s 2024-04-25T10:00:00.325 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.325 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.325 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706262 s, 148 MB/s 2024-04-25T10:00:00.333 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.333 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.333 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732192 s, 143 MB/s 2024-04-25T10:00:00.340 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.340 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.341 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638535 s, 164 MB/s 2024-04-25T10:00:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703756 s, 149 MB/s 2024-04-25T10:00:00.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.356 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070432 s, 149 MB/s 2024-04-25T10:00:00.364 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.364 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.364 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692254 s, 151 MB/s 2024-04-25T10:00:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610584 s, 172 MB/s 2024-04-25T10:00:00.378 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.379 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670111 s, 156 MB/s 2024-04-25T10:00:00.386 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.386 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.386 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675051 s, 155 MB/s 2024-04-25T10:00:00.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:00.387 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:00.387 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:00.388 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:00:00.388 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:00.388 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:00.388 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:00.388 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:00.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.453 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068994 s, 152 MB/s 2024-04-25T10:00:00.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.461 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669371 s, 157 MB/s 2024-04-25T10:00:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657066 s, 160 MB/s 2024-04-25T10:00:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704244 s, 149 MB/s 2024-04-25T10:00:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644727 s, 163 MB/s 2024-04-25T10:00:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676552 s, 155 MB/s 2024-04-25T10:00:00.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.498 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605288 s, 173 MB/s 2024-04-25T10:00:00.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.505 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632104 s, 166 MB/s 2024-04-25T10:00:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731366 s, 143 MB/s 2024-04-25T10:00:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.521 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.521 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645199 s, 163 MB/s 2024-04-25T10:00:00.530 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.530 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.530 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818669 s, 128 MB/s 2024-04-25T10:00:00.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.537 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647811 s, 162 MB/s 2024-04-25T10:00:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705186 s, 149 MB/s 2024-04-25T10:00:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712732 s, 147 MB/s 2024-04-25T10:00:00.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.562 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723135 s, 145 MB/s 2024-04-25T10:00:00.569 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.569 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.569 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669509 s, 157 MB/s 2024-04-25T10:00:00.579 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.579 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.579 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887337 s, 118 MB/s 2024-04-25T10:00:00.586 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.586 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.586 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006185 s, 170 MB/s 2024-04-25T10:00:00.594 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.594 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.594 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752337 s, 139 MB/s 2024-04-25T10:00:00.602 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.602 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.602 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672735 s, 156 MB/s 2024-04-25T10:00:00.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.610 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662769 s, 158 MB/s 2024-04-25T10:00:00.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.617 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666625 s, 157 MB/s 2024-04-25T10:00:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846732 s, 124 MB/s 2024-04-25T10:00:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071589 s, 146 MB/s 2024-04-25T10:00:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.642 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694504 s, 151 MB/s 2024-04-25T10:00:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684565 s, 153 MB/s 2024-04-25T10:00:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636984 s, 165 MB/s 2024-04-25T10:00:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785701 s, 133 MB/s 2024-04-25T10:00:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667193 s, 157 MB/s 2024-04-25T10:00:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620312 s, 169 MB/s 2024-04-25T10:00:00.688 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.688 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.689 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669225 s, 157 MB/s 2024-04-25T10:00:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795801 s, 132 MB/s 2024-04-25T10:00:00.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:00.698 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:00.699 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:00.699 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:00:00.699 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:00.699 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:00.699 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:00.699 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:00.764 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.764 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.764 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675889 s, 155 MB/s 2024-04-25T10:00:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701129 s, 150 MB/s 2024-04-25T10:00:00.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.779 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654065 s, 160 MB/s 2024-04-25T10:00:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672769 s, 156 MB/s 2024-04-25T10:00:00.795 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.795 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.795 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720924 s, 145 MB/s 2024-04-25T10:00:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809756 s, 129 MB/s 2024-04-25T10:00:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682877 s, 154 MB/s 2024-04-25T10:00:00.819 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.819 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.819 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652517 s, 161 MB/s 2024-04-25T10:00:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820824 s, 128 MB/s 2024-04-25T10:00:00.835 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.836 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.836 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648262 s, 162 MB/s 2024-04-25T10:00:00.844 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.844 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.844 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723187 s, 145 MB/s 2024-04-25T10:00:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735437 s, 143 MB/s 2024-04-25T10:00:00.860 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.860 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.860 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690059 s, 152 MB/s 2024-04-25T10:00:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.868 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733245 s, 143 MB/s 2024-04-25T10:00:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732686 s, 143 MB/s 2024-04-25T10:00:00.884 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.884 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.884 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653974 s, 160 MB/s 2024-04-25T10:00:00.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.895 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100802 s, 104 MB/s 2024-04-25T10:00:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794537 s, 132 MB/s 2024-04-25T10:00:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.912 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705183 s, 149 MB/s 2024-04-25T10:00:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.919 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619083 s, 169 MB/s 2024-04-25T10:00:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663291 s, 158 MB/s 2024-04-25T10:00:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753068 s, 139 MB/s 2024-04-25T10:00:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.943 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782642 s, 134 MB/s 2024-04-25T10:00:00.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.953 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.953 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831386 s, 126 MB/s 2024-04-25T10:00:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.960 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687647 s, 152 MB/s 2024-04-25T10:00:00.968 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.968 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.968 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694526 s, 151 MB/s 2024-04-25T10:00:00.976 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.976 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.976 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072028 s, 146 MB/s 2024-04-25T10:00:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.985 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.985 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731016 s, 143 MB/s 2024-04-25T10:00:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:00.992 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638281 s, 164 MB/s 2024-04-25T10:00:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709586 s, 148 MB/s 2024-04-25T10:00:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716455 s, 146 MB/s 2024-04-25T10:00:01.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.016 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703923 s, 149 MB/s 2024-04-25T10:00:01.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:01.017 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:01.017 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:01.017 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:00:01.018 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:01.018 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:01.018 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:01.018 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:01.083 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.083 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.083 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704664 s, 149 MB/s 2024-04-25T10:00:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773872 s, 135 MB/s 2024-04-25T10:00:01.099 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.100 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.100 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681216 s, 154 MB/s 2024-04-25T10:00:01.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.107 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685269 s, 153 MB/s 2024-04-25T10:00:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674891 s, 155 MB/s 2024-04-25T10:00:01.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.123 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708133 s, 148 MB/s 2024-04-25T10:00:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626588 s, 167 MB/s 2024-04-25T10:00:01.138 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.138 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.138 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724981 s, 145 MB/s 2024-04-25T10:00:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.146 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659342 s, 159 MB/s 2024-04-25T10:00:01.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.153 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641936 s, 163 MB/s 2024-04-25T10:00:01.160 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672789 s, 156 MB/s 2024-04-25T10:00:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772286 s, 136 MB/s 2024-04-25T10:00:01.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.177 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641589 s, 163 MB/s 2024-04-25T10:00:01.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.184 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671146 s, 156 MB/s 2024-04-25T10:00:01.192 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.192 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.192 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710798 s, 148 MB/s 2024-04-25T10:00:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913031 s, 115 MB/s 2024-04-25T10:00:01.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.210 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694215 s, 151 MB/s 2024-04-25T10:00:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.219 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.220 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824118 s, 127 MB/s 2024-04-25T10:00:01.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.227 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683642 s, 153 MB/s 2024-04-25T10:00:01.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.234 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665153 s, 158 MB/s 2024-04-25T10:00:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692689 s, 151 MB/s 2024-04-25T10:00:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072808 s, 144 MB/s 2024-04-25T10:00:01.261 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.261 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.261 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931627 s, 113 MB/s 2024-04-25T10:00:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.268 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657386 s, 160 MB/s 2024-04-25T10:00:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695774 s, 151 MB/s 2024-04-25T10:00:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633799 s, 165 MB/s 2024-04-25T10:00:01.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.291 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071714 s, 146 MB/s 2024-04-25T10:00:01.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.299 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695517 s, 151 MB/s 2024-04-25T10:00:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.307 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663779 s, 158 MB/s 2024-04-25T10:00:01.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.314 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634333 s, 165 MB/s 2024-04-25T10:00:01.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.327 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115549 s, 90.7 MB/s 2024-04-25T10:00:01.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.335 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716137 s, 146 MB/s 2024-04-25T10:00:01.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:01.336 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:01.336 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:01.336 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:00:01.336 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:01.336 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:01.336 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:01.336 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834971 s, 126 MB/s 2024-04-25T10:00:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.410 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062374 s, 168 MB/s 2024-04-25T10:00:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.417 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067064 s, 156 MB/s 2024-04-25T10:00:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697588 s, 150 MB/s 2024-04-25T10:00:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913767 s, 115 MB/s 2024-04-25T10:00:01.443 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.444 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.444 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768911 s, 136 MB/s 2024-04-25T10:00:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843965 s, 124 MB/s 2024-04-25T10:00:01.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.461 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762978 s, 137 MB/s 2024-04-25T10:00:01.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.469 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679213 s, 154 MB/s 2024-04-25T10:00:01.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.477 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663736 s, 158 MB/s 2024-04-25T10:00:01.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.486 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719258 s, 146 MB/s 2024-04-25T10:00:01.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.494 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.494 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796967 s, 132 MB/s 2024-04-25T10:00:01.504 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.504 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.504 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907596 s, 116 MB/s 2024-04-25T10:00:01.511 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.511 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.511 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069643 s, 151 MB/s 2024-04-25T10:00:01.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.519 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645114 s, 163 MB/s 2024-04-25T10:00:01.526 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.526 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.526 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648672 s, 162 MB/s 2024-04-25T10:00:01.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.534 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730737 s, 143 MB/s 2024-04-25T10:00:01.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.543 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746865 s, 140 MB/s 2024-04-25T10:00:01.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.553 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911361 s, 115 MB/s 2024-04-25T10:00:01.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.561 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682342 s, 154 MB/s 2024-04-25T10:00:01.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.568 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661244 s, 159 MB/s 2024-04-25T10:00:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704836 s, 149 MB/s 2024-04-25T10:00:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775076 s, 135 MB/s 2024-04-25T10:00:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684349 s, 153 MB/s 2024-04-25T10:00:01.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.600 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697669 s, 150 MB/s 2024-04-25T10:00:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675047 s, 155 MB/s 2024-04-25T10:00:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631688 s, 166 MB/s 2024-04-25T10:00:01.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.623 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064794 s, 162 MB/s 2024-04-25T10:00:01.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.630 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688351 s, 152 MB/s 2024-04-25T10:00:01.639 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.639 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.639 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733445 s, 143 MB/s 2024-04-25T10:00:01.647 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.647 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.647 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706089 s, 149 MB/s 2024-04-25T10:00:01.656 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.656 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.657 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867177 s, 121 MB/s 2024-04-25T10:00:01.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:01.658 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:01.658 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:01.658 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:00:01.658 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:01.658 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:01.658 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:01.658 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:01.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.723 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732054 s, 143 MB/s 2024-04-25T10:00:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726376 s, 144 MB/s 2024-04-25T10:00:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785005 s, 134 MB/s 2024-04-25T10:00:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.749 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739468 s, 142 MB/s 2024-04-25T10:00:01.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714894 s, 147 MB/s 2024-04-25T10:00:01.764 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.764 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.765 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698131 s, 150 MB/s 2024-04-25T10:00:01.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.772 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662402 s, 158 MB/s 2024-04-25T10:00:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667149 s, 157 MB/s 2024-04-25T10:00:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707817 s, 148 MB/s 2024-04-25T10:00:01.799 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.799 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.799 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104244 s, 101 MB/s 2024-04-25T10:00:01.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.807 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653928 s, 160 MB/s 2024-04-25T10:00:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705485 s, 149 MB/s 2024-04-25T10:00:01.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.823 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750012 s, 140 MB/s 2024-04-25T10:00:01.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.831 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722316 s, 145 MB/s 2024-04-25T10:00:01.839 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.840 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.840 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740122 s, 142 MB/s 2024-04-25T10:00:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698337 s, 150 MB/s 2024-04-25T10:00:01.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.857 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871452 s, 120 MB/s 2024-04-25T10:00:01.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.865 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071203 s, 147 MB/s 2024-04-25T10:00:01.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.873 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693847 s, 151 MB/s 2024-04-25T10:00:01.880 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648425 s, 162 MB/s 2024-04-25T10:00:01.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.890 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070684 s, 148 MB/s 2024-04-25T10:00:01.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.899 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999311 s, 105 MB/s 2024-04-25T10:00:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717293 s, 146 MB/s 2024-04-25T10:00:01.915 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.915 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.915 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732628 s, 143 MB/s 2024-04-25T10:00:01.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.923 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656234 s, 160 MB/s 2024-04-25T10:00:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692767 s, 151 MB/s 2024-04-25T10:00:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850028 s, 123 MB/s 2024-04-25T10:00:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.948 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710177 s, 148 MB/s 2024-04-25T10:00:01.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.956 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694944 s, 151 MB/s 2024-04-25T10:00:01.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.965 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795207 s, 132 MB/s 2024-04-25T10:00:01.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.973 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739266 s, 142 MB/s 2024-04-25T10:00:01.983 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:01.983 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:01.983 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878356 s, 119 MB/s 2024-04-25T10:00:01.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:01.984 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:01.984 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:01.984 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:00:01.984 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:01.984 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:01.984 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:01.985 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696132 s, 151 MB/s 2024-04-25T10:00:02.058 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.058 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.058 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758007 s, 138 MB/s 2024-04-25T10:00:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629764 s, 167 MB/s 2024-04-25T10:00:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648923 s, 162 MB/s 2024-04-25T10:00:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.081 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684181 s, 153 MB/s 2024-04-25T10:00:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714766 s, 147 MB/s 2024-04-25T10:00:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722481 s, 145 MB/s 2024-04-25T10:00:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691795 s, 152 MB/s 2024-04-25T10:00:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.112 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621449 s, 169 MB/s 2024-04-25T10:00:02.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.123 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975474 s, 107 MB/s 2024-04-25T10:00:02.130 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.130 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.131 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712967 s, 147 MB/s 2024-04-25T10:00:02.138 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.139 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.139 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689232 s, 152 MB/s 2024-04-25T10:00:02.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.146 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708958 s, 148 MB/s 2024-04-25T10:00:02.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.154 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676446 s, 155 MB/s 2024-04-25T10:00:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724201 s, 145 MB/s 2024-04-25T10:00:02.173 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.173 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.173 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970959 s, 108 MB/s 2024-04-25T10:00:02.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.180 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622032 s, 169 MB/s 2024-04-25T10:00:02.188 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.188 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.188 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749338 s, 140 MB/s 2024-04-25T10:00:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698355 s, 150 MB/s 2024-04-25T10:00:02.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.205 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.205 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730546 s, 144 MB/s 2024-04-25T10:00:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652797 s, 161 MB/s 2024-04-25T10:00:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712823 s, 147 MB/s 2024-04-25T10:00:02.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.227 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624845 s, 168 MB/s 2024-04-25T10:00:02.235 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.235 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.235 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706716 s, 148 MB/s 2024-04-25T10:00:02.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.243 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.243 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652031 s, 161 MB/s 2024-04-25T10:00:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849072 s, 123 MB/s 2024-04-25T10:00:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715869 s, 146 MB/s 2024-04-25T10:00:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608788 s, 172 MB/s 2024-04-25T10:00:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704938 s, 149 MB/s 2024-04-25T10:00:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705111 s, 149 MB/s 2024-04-25T10:00:02.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.292 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744378 s, 141 MB/s 2024-04-25T10:00:02.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.299 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680811 s, 154 MB/s 2024-04-25T10:00:02.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:02.300 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:02.300 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:02.300 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:00:02.301 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:02.301 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:02.301 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:02.301 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:02.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.366 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765103 s, 137 MB/s 2024-04-25T10:00:02.374 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.374 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.374 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647248 s, 162 MB/s 2024-04-25T10:00:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064327 s, 163 MB/s 2024-04-25T10:00:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726509 s, 144 MB/s 2024-04-25T10:00:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864986 s, 121 MB/s 2024-04-25T10:00:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857631 s, 122 MB/s 2024-04-25T10:00:02.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.416 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713758 s, 147 MB/s 2024-04-25T10:00:02.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.424 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648313 s, 162 MB/s 2024-04-25T10:00:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.432 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079154 s, 132 MB/s 2024-04-25T10:00:02.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.441 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757764 s, 138 MB/s 2024-04-25T10:00:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698758 s, 150 MB/s 2024-04-25T10:00:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766303 s, 137 MB/s 2024-04-25T10:00:02.465 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.466 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727242 s, 144 MB/s 2024-04-25T10:00:02.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.474 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738062 s, 142 MB/s 2024-04-25T10:00:02.481 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.481 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.481 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625216 s, 168 MB/s 2024-04-25T10:00:02.489 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.489 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.489 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689062 s, 152 MB/s 2024-04-25T10:00:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690676 s, 152 MB/s 2024-04-25T10:00:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671358 s, 156 MB/s 2024-04-25T10:00:02.512 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.512 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.512 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703609 s, 149 MB/s 2024-04-25T10:00:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721658 s, 145 MB/s 2024-04-25T10:00:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737667 s, 142 MB/s 2024-04-25T10:00:02.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.537 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700999 s, 150 MB/s 2024-04-25T10:00:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.545 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769141 s, 136 MB/s 2024-04-25T10:00:02.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.553 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683082 s, 154 MB/s 2024-04-25T10:00:02.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.562 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797862 s, 131 MB/s 2024-04-25T10:00:02.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.574 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996178 s, 105 MB/s 2024-04-25T10:00:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01017 s, 103 MB/s 2024-04-25T10:00:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704051 s, 149 MB/s 2024-04-25T10:00:02.601 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.601 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.601 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715998 s, 146 MB/s 2024-04-25T10:00:02.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.610 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775662 s, 135 MB/s 2024-04-25T10:00:02.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.617 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657233 s, 160 MB/s 2024-04-25T10:00:02.625 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.625 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.625 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649649 s, 161 MB/s 2024-04-25T10:00:02.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:02.626 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:02.626 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:02.626 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:00:02.626 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:02.626 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:02.626 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:02.626 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737944 s, 142 MB/s 2024-04-25T10:00:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627145 s, 167 MB/s 2024-04-25T10:00:02.707 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.707 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.707 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648349 s, 162 MB/s 2024-04-25T10:00:02.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.714 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703272 s, 149 MB/s 2024-04-25T10:00:02.722 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.722 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.722 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728975 s, 144 MB/s 2024-04-25T10:00:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713125 s, 147 MB/s 2024-04-25T10:00:02.738 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.738 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.738 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068469 s, 153 MB/s 2024-04-25T10:00:02.745 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.746 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.746 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649305 s, 161 MB/s 2024-04-25T10:00:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987817 s, 106 MB/s 2024-04-25T10:00:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617839 s, 170 MB/s 2024-04-25T10:00:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.772 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737301 s, 142 MB/s 2024-04-25T10:00:02.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.779 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661096 s, 159 MB/s 2024-04-25T10:00:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685624 s, 153 MB/s 2024-04-25T10:00:02.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.794 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641614 s, 163 MB/s 2024-04-25T10:00:02.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.804 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913303 s, 115 MB/s 2024-04-25T10:00:02.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.812 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734528 s, 143 MB/s 2024-04-25T10:00:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713428 s, 147 MB/s 2024-04-25T10:00:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688762 s, 152 MB/s 2024-04-25T10:00:02.836 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.836 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.836 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694709 s, 151 MB/s 2024-04-25T10:00:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656745 s, 160 MB/s 2024-04-25T10:00:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855986 s, 122 MB/s 2024-04-25T10:00:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702262 s, 149 MB/s 2024-04-25T10:00:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665037 s, 158 MB/s 2024-04-25T10:00:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675663 s, 155 MB/s 2024-04-25T10:00:02.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.883 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616196 s, 170 MB/s 2024-04-25T10:00:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686758 s, 153 MB/s 2024-04-25T10:00:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698602 s, 150 MB/s 2024-04-25T10:00:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692481 s, 151 MB/s 2024-04-25T10:00:02.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.915 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714078 s, 147 MB/s 2024-04-25T10:00:02.922 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.922 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.922 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679047 s, 154 MB/s 2024-04-25T10:00:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690135 s, 152 MB/s 2024-04-25T10:00:02.938 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:02.938 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:02.938 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708622 s, 148 MB/s 2024-04-25T10:00:02.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:02.939 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:02.939 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:02.940 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:00:02.940 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:02.940 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:02.940 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:02.940 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:03.004 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.004 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.004 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651149 s, 161 MB/s 2024-04-25T10:00:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.012 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723502 s, 145 MB/s 2024-04-25T10:00:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611642 s, 171 MB/s 2024-04-25T10:00:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.028 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754256 s, 139 MB/s 2024-04-25T10:00:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734354 s, 143 MB/s 2024-04-25T10:00:03.044 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.044 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.044 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698482 s, 150 MB/s 2024-04-25T10:00:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772645 s, 136 MB/s 2024-04-25T10:00:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626476 s, 167 MB/s 2024-04-25T10:00:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.067 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672202 s, 156 MB/s 2024-04-25T10:00:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.076 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712267 s, 147 MB/s 2024-04-25T10:00:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726225 s, 144 MB/s 2024-04-25T10:00:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652442 s, 161 MB/s 2024-04-25T10:00:03.100 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.100 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.101 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846674 s, 124 MB/s 2024-04-25T10:00:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069443 s, 151 MB/s 2024-04-25T10:00:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688622 s, 152 MB/s 2024-04-25T10:00:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680177 s, 154 MB/s 2024-04-25T10:00:03.131 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.131 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.131 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650202 s, 161 MB/s 2024-04-25T10:00:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698718 s, 150 MB/s 2024-04-25T10:00:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.147 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715213 s, 147 MB/s 2024-04-25T10:00:03.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.155 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664459 s, 158 MB/s 2024-04-25T10:00:03.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.162 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622867 s, 168 MB/s 2024-04-25T10:00:03.170 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.170 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.170 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720054 s, 146 MB/s 2024-04-25T10:00:03.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.178 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732935 s, 143 MB/s 2024-04-25T10:00:03.185 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.185 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.185 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621359 s, 169 MB/s 2024-04-25T10:00:03.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.194 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717072 s, 146 MB/s 2024-04-25T10:00:03.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.201 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677592 s, 155 MB/s 2024-04-25T10:00:03.208 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.209 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640449 s, 164 MB/s 2024-04-25T10:00:03.218 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.218 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.218 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733647 s, 143 MB/s 2024-04-25T10:00:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716744 s, 146 MB/s 2024-04-25T10:00:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890805 s, 118 MB/s 2024-04-25T10:00:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705645 s, 149 MB/s 2024-04-25T10:00:03.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071316 s, 147 MB/s 2024-04-25T10:00:03.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:03.252 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:00:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.393+0000 7f197d9f4640 1 Processor -- start 2024-04-25T10:00:03.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.393+0000 7f197d9f4640 1 -- start start 2024-04-25T10:00:03.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.394+0000 7f197d9f4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f197815b7d0 0x7f19780cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.394+0000 7f197d9f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f197806c9e0 0x7f19780b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.394+0000 7f197d9f4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197806cf50 con 0x7f19780b5620 2024-04-25T10:00:03.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.394+0000 7f197d9f4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19780ce530 con 0x7f197815b7d0 2024-04-25T10:00:03.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.394+0000 7f197d9f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19780ce810 con 0x7f197806c9e0 2024-04-25T10:00:03.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.394+0000 7f1976ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f197815b7d0 0x7f19780cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.394+0000 7f1976ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f197815b7d0 0x7f19780cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47234/0 (socket says 172.21.15.19:47234) 2024-04-25T10:00:03.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.394+0000 7f1976ffd640 1 -- 172.21.15.19:0/2996206804 learned_addr learned my addr 172.21.15.19:0/2996206804 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.395+0000 7f19777fe640 1 --2- 172.21.15.19:0/2996206804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f197806c9e0 0x7f19780b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.395+0000 7f1976ffd640 1 -- 172.21.15.19:0/2996206804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f197806c9e0 msgr2=0x7f19780b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.395+0000 7f1976ffd640 1 --2- 172.21.15.19:0/2996206804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f197806c9e0 0x7f19780b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.395+0000 7f1976ffd640 1 -- 172.21.15.19:0/2996206804 >> v1:172.21.15.112:6789/0 conn(0x7f19780b5620 legacy=0x7f197806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.395+0000 7f1976ffd640 1 -- 172.21.15.19:0/2996206804 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19780ceaf0 con 0x7f197815b7d0 2024-04-25T10:00:03.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.395+0000 7f19777fe640 1 --2- 172.21.15.19:0/2996206804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f197806c9e0 0x7f19780b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:03.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.395+0000 7f1976ffd640 1 --2- 172.21.15.19:0/2996206804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f197815b7d0 0x7f19780cf010 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f1968009870 tx=0x7f196802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b610e31945a84a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.395+0000 7f19767fc640 1 -- 172.21.15.19:0/2996206804 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968037070 con 0x7f197815b7d0 2024-04-25T10:00:03.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.396+0000 7f19767fc640 1 -- 172.21.15.19:0/2996206804 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1968032440 con 0x7f197815b7d0 2024-04-25T10:00:03.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.396+0000 7f19767fc640 1 -- 172.21.15.19:0/2996206804 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968032810 con 0x7f197815b7d0 2024-04-25T10:00:03.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.396+0000 7f197d9f4640 1 -- 172.21.15.19:0/2996206804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f197815b7d0 msgr2=0x7f19780cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.396+0000 7f197d9f4640 1 --2- 172.21.15.19:0/2996206804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f197815b7d0 0x7f19780cf010 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f1968009870 tx=0x7f196802a270 comp rx=0 tx=0).stop 2024-04-25T10:00:03.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.396+0000 7f197d9f4640 1 -- 172.21.15.19:0/2996206804 shutdown_connections 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.396+0000 7f197d9f4640 1 --2- 172.21.15.19:0/2996206804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f197806c9e0 0x7f19780b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.396+0000 7f197d9f4640 1 --2- 172.21.15.19:0/2996206804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f197815b7d0 0x7f19780cf010 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.396+0000 7f197d9f4640 1 -- 172.21.15.19:0/2996206804 >> 172.21.15.19:0/2996206804 conn(0x7f19780d0800 msgr2=0x7f19780d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.396+0000 7f197d9f4640 1 -- 172.21.15.19:0/2996206804 shutdown_connections 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.396+0000 7f197d9f4640 1 -- 172.21.15.19:0/2996206804 wait complete. 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.398+0000 7f197d9f4640 1 Processor -- start 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.398+0000 7f197d9f4640 1 -- start start 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.398+0000 7f197d9f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f197806c9e0 0x7f19780c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.398+0000 7f197d9f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19780b5620 0x7f19780ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.398+0000 7f197d9f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f197815b7d0 0x7f19780b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.398+0000 7f197d9f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780cf6d0 con 0x7f19780b5620 2024-04-25T10:00:03.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.398+0000 7f197d9f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780cf550 con 0x7f197815b7d0 2024-04-25T10:00:03.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.398+0000 7f197d9f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197815bbb0 con 0x7f197806c9e0 2024-04-25T10:00:03.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.398+0000 7f196ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19780b5620 0x7f19780ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.398+0000 7f196ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19780b5620 0x7f19780ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47244/0 (socket says 172.21.15.19:47244) 2024-04-25T10:00:03.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.398+0000 7f196ffff640 1 -- 172.21.15.19:0/2502443246 learned_addr learned my addr 172.21.15.19:0/2502443246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f1976ffd640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f197806c9e0 0x7f19780c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f19777fe640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f197815b7d0 0x7f19780b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f196ffff640 1 -- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f197806c9e0 msgr2=0x7f19780c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f196ffff640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f197806c9e0 0x7f19780c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f196ffff640 1 -- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f197815b7d0 msgr2=0x7f19780b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f196ffff640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f197815b7d0 0x7f19780b7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f196ffff640 1 -- 172.21.15.19:0/2502443246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19781f3220 con 0x7f19780b5620 2024-04-25T10:00:03.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f196ffff640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19780b5620 0x7f19780ca290 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f196000c460 tx=0x7f196000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f1974ff9640 1 -- 172.21.15.19:0/2502443246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960017070 con 0x7f19780b5620 2024-04-25T10:00:03.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f1974ff9640 1 -- 172.21.15.19:0/2502443246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1960005310 con 0x7f19780b5620 2024-04-25T10:00:03.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f1974ff9640 1 -- 172.21.15.19:0/2502443246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960003850 con 0x7f19780b5620 2024-04-25T10:00:03.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f197d9f4640 1 -- 172.21.15.19:0/2502443246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19780c91f0 con 0x7f19780b5620 2024-04-25T10:00:03.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.399+0000 7f197d9f4640 1 -- 172.21.15.19:0/2502443246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19780c97b0 con 0x7f19780b5620 2024-04-25T10:00:03.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.401+0000 7f1974ff9640 1 -- 172.21.15.19:0/2502443246 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f196001a070 con 0x7f19780b5620 2024-04-25T10:00:03.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.401+0000 7f1974ff9640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f195806b4d0 0x7f195806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.401+0000 7f1974ff9640 1 -- 172.21.15.19:0/2502443246 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f19600895a0 con 0x7f19780b5620 2024-04-25T10:00:03.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.401+0000 7f1976ffd640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f195806b4d0 0x7f195806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.402+0000 7f1976ffd640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f195806b4d0 0x7f195806d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1968034000 tx=0x7f196802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.402+0000 7f19807ce740 1 Processor -- start 2024-04-25T10:00:03.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.402+0000 7f19807ce740 1 -- start start 2024-04-25T10:00:03.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f19807ce740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c5e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f19807ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c5c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f19807ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c86cc0 0x5619e8c87140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f19807ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e8bc0000 con 0x5619e8c988d0 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f19807ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e8b68f20 con 0x5619e8c86cc0 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f19807ce740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619e8bbc0a0 con 0x5619e8c5c200 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c5c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c5c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47248/0 (socket says 172.21.15.19:47248) 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196ffff640 1 -- 172.21.15.19:0/569949261 learned_addr learned my addr 172.21.15.19:0/569949261 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f19777fe640 1 --2- 172.21.15.19:0/569949261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c86cc0 0x5619e8c87140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f1976ffd640 1 --2- 172.21.15.19:0/569949261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c5e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196ffff640 1 -- 172.21.15.19:0/569949261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 msgr2=0x5619e8c5e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196ffff640 1 --2- 172.21.15.19:0/569949261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c5e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196ffff640 1 -- 172.21.15.19:0/569949261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c86cc0 msgr2=0x5619e8c87140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196ffff640 1 --2- 172.21.15.19:0/569949261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c86cc0 0x5619e8c87140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196ffff640 1 -- 172.21.15.19:0/569949261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e89a0ba0 con 0x5619e8c988d0 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196ffff640 1 --2- 172.21.15.19:0/569949261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c5c5e0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1960000ff0 tx=0x7f196000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0960d08541b360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196d7fa640 1 -- 172.21.15.19:0/569949261 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960055f40 con 0x5619e8c988d0 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196d7fa640 1 -- 172.21.15.19:0/569949261 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19600542b0 con 0x5619e8c988d0 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.403+0000 7f196d7fa640 1 -- 172.21.15.19:0/569949261 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196006bd90 con 0x5619e8c988d0 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.404+0000 7f19807ce740 1 -- 172.21.15.19:0/569949261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 msgr2=0x5619e8c5c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.404+0000 7f19807ce740 1 --2- 172.21.15.19:0/569949261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c5c5e0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1960000ff0 tx=0x7f196000a000 comp rx=0 tx=0).stop 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.404+0000 7f19807ce740 1 -- 172.21.15.19:0/569949261 shutdown_connections 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.404+0000 7f19807ce740 1 --2- 172.21.15.19:0/569949261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c86cc0 0x5619e8c87140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.404+0000 7f19807ce740 1 --2- 172.21.15.19:0/569949261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c5c5e0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.404+0000 7f19807ce740 1 --2- 172.21.15.19:0/569949261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c5e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.404+0000 7f19807ce740 1 -- 172.21.15.19:0/569949261 >> 172.21.15.19:0/569949261 conn(0x5619e8c24050 msgr2=0x5619e8ca1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.404+0000 7f19807ce740 1 -- 172.21.15.19:0/569949261 shutdown_connections 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.404+0000 7f19807ce740 1 -- 172.21.15.19:0/569949261 wait complete. 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.405+0000 7f19807ce740 1 -- start start 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.405+0000 7f19807ce740 1 Processor -- start 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.405+0000 7f19807ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c5c200 0x5619e8c756e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19807ce740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c86cc0 0x5619e8c88f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19807ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c89460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19807ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e8b5e610 con 0x5619e8c988d0 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19807ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e8c84810 con 0x5619e8c5c200 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19807ce740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619e8c9bd60 con 0x5619e8c86cc0 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19807ce740 10 client.0 _ll_register_callbacks cb 0x5619e8b76f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19777fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c89460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19777fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c89460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47252/0 (socket says 172.21.15.19:47252) 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f196ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c86cc0 0x5619e8c88f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19777fe640 1 -- 172.21.15.19:0/364764916 learned_addr learned my addr 172.21.15.19:0/364764916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f1976ffd640 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c5c200 0x5619e8c756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19777fe640 1 -- 172.21.15.19:0/364764916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c86cc0 msgr2=0x5619e8c88f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19777fe640 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c86cc0 0x5619e8c88f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19777fe640 1 -- 172.21.15.19:0/364764916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c5c200 msgr2=0x5619e8c756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19777fe640 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c5c200 0x5619e8c756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19777fe640 1 -- 172.21.15.19:0/364764916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619e8894490 con 0x5619e8c988d0 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19777fe640 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c89460 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f197000dea0 tx=0x7f197000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667a1b2c1b351014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19567fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19567fc640 1 -- 172.21.15.19:0/364764916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970018640 con 0x5619e8c988d0 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19567fc640 1 -- 172.21.15.19:0/364764916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f19700187e0 con 0x5619e8c988d0 2024-04-25T10:00:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19807ce740 1 -- 172.21.15.19:0/364764916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19600542b0 con 0x5619e8c988d0 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19567fc640 1 -- 172.21.15.19:0/364764916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1970024050 con 0x5619e8c988d0 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19807ce740 10 client.8193 fetch_fsmap learned FSMap version 65 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.406+0000 7f19807ce740 1 -- 172.21.15.19:0/364764916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5619e8a01330 con 0x5619e8c988d0 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.407+0000 7f19567fc640 1 -- 172.21.15.19:0/364764916 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19600542b0 con 0x5619e8c988d0 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.407+0000 7f19567fc640 1 -- 172.21.15.19:0/364764916 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1970027020 con 0x5619e8c988d0 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.407+0000 7f19807ce740 10 client.8193 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.407+0000 7f19807ce740 10 client.8193 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.407+0000 7f19807ce740 20 client.8193 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.407+0000 7f19807ce740 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5619e8c9c8d0 0x5619e8cb5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.407+0000 7f19807ce740 4 client.8193 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.407+0000 7f19807ce740 1 -- 172.21.15.19:0/364764916 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619e8bcd540 con 0x5619e8c9c8d0 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.407+0000 7f1976ffd640 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5619e8c9c8d0 0x5619e8cb5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.407+0000 7f1976ffd640 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5619e8c9c8d0 0x5619e8cb5490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a374210dafe445 server_cookie=ca38a8c4f2c0f7dd in_seq=0 out_seq=0 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.407+0000 7f19567fc640 10 client.8193 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19567fc640 1 -- 172.21.15.19:0/364764916 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619e8bcd540 con 0x5619e8c9c8d0 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19567fc640 10 client.8193 handle_command_reply: tid=0 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 1 client.8193 shutdown 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 1 -- 172.21.15.19:0/364764916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 msgr2=0x5619e8c89460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c89460 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f197000dea0 tx=0x7f197000df90 comp rx=0 tx=0).stop 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 1 -- 172.21.15.19:0/364764916 shutdown_connections 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5619e8c9c8d0 0x5619e8cb5490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c988d0 0x5619e8c89460 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c86cc0 0x5619e8c88f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 1 --2- 172.21.15.19:0/364764916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c5c200 0x5619e8c756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19567fc640 0 client.8193 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 1 -- 172.21.15.19:0/364764916 >> 172.21.15.19:0/364764916 conn(0x5619e8c24050 msgr2=0x5619e8c83fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 1 -- 172.21.15.19:0/364764916 shutdown_connections 2024-04-25T10:00:03.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 1 -- 172.21.15.19:0/364764916 wait complete. 2024-04-25T10:00:03.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.408+0000 7f19807ce740 20 client.8193 trim_cache size 0 max 16384 2024-04-25T10:00:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.423+0000 7f19807ce740 1 Processor -- start 2024-04-25T10:00:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.423+0000 7f19807ce740 1 -- start start 2024-04-25T10:00:03.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f19807ce740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c861c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f19807ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c86700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f19807ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c988d0 0x5619e8c97650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f19807ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1978200470 con 0x5619e8c86cc0 2024-04-25T10:00:03.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f19807ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781ff610 con 0x5619e8c988d0 2024-04-25T10:00:03.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f19807ce740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197814b950 con 0x5619e8c5c200 2024-04-25T10:00:03.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f196ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c86700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f196ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c86700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47258/0 (socket says 172.21.15.19:47258) 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f196ffff640 1 -- 172.21.15.19:0/1267846377 learned_addr learned my addr 172.21.15.19:0/1267846377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f19777fe640 1 --2- 172.21.15.19:0/1267846377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c988d0 0x5619e8c97650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f1976ffd640 1 --2- 172.21.15.19:0/1267846377 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c861c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f196ffff640 1 -- 172.21.15.19:0/1267846377 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 msgr2=0x5619e8c861c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f196ffff640 1 --2- 172.21.15.19:0/1267846377 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c861c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f196ffff640 1 -- 172.21.15.19:0/1267846377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c988d0 msgr2=0x5619e8c97650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f196ffff640 1 --2- 172.21.15.19:0/1267846377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c988d0 0x5619e8c97650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f196ffff640 1 -- 172.21.15.19:0/1267846377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e886d610 con 0x5619e8c86cc0 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f19777fe640 1 --2- 172.21.15.19:0/1267846377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c988d0 0x5619e8c97650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.424+0000 7f1976ffd640 1 --2- 172.21.15.19:0/1267846377 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c861c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f196ffff640 1 --2- 172.21.15.19:0/1267846377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c86700 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f19600578f0 tx=0x7f196001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d662f5111c05f64f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f196cff9640 1 -- 172.21.15.19:0/1267846377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960074070 con 0x5619e8c86cc0 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f196cff9640 1 -- 172.21.15.19:0/1267846377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19600542b0 con 0x5619e8c86cc0 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f196cff9640 1 -- 172.21.15.19:0/1267846377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196006bd90 con 0x5619e8c86cc0 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f19807ce740 1 -- 172.21.15.19:0/1267846377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 msgr2=0x5619e8c86700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f19807ce740 1 --2- 172.21.15.19:0/1267846377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c86700 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f19600578f0 tx=0x7f196001e000 comp rx=0 tx=0).stop 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f19807ce740 1 -- 172.21.15.19:0/1267846377 shutdown_connections 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f19807ce740 1 --2- 172.21.15.19:0/1267846377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c988d0 0x5619e8c97650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f19807ce740 1 --2- 172.21.15.19:0/1267846377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c86700 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f19807ce740 1 --2- 172.21.15.19:0/1267846377 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c861c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f19807ce740 1 -- 172.21.15.19:0/1267846377 >> 172.21.15.19:0/1267846377 conn(0x5619e8c24050 msgr2=0x5619e8c98d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f19807ce740 1 -- 172.21.15.19:0/1267846377 shutdown_connections 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.425+0000 7f19807ce740 1 -- 172.21.15.19:0/1267846377 wait complete. 2024-04-25T10:00:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.426+0000 7f19807ce740 1 -- start start 2024-04-25T10:00:03.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.426+0000 7f19807ce740 1 Processor -- start 2024-04-25T10:00:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19807ce740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c5b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19807ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19807ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c988d0 0x5619e8c875e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19807ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e8c999e0 con 0x5619e8c86cc0 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19807ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781ff010 con 0x5619e8c988d0 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19807ce740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619e8cba5a0 con 0x5619e8c5c200 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19807ce740 10 client.0 _ll_register_callbacks cb 0x7f1960053ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f196ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f196ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c870a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47268/0 (socket says 172.21.15.19:47268) 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f196ffff640 1 -- 172.21.15.19:0/1451011590 learned_addr learned my addr 172.21.15.19:0/1451011590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19777fe640 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c988d0 0x5619e8c875e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f1976ffd640 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c5b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f196ffff640 1 -- 172.21.15.19:0/1451011590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 msgr2=0x5619e8c5b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f196ffff640 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c5b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f196ffff640 1 -- 172.21.15.19:0/1451011590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c988d0 msgr2=0x5619e8c875e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f196ffff640 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c988d0 0x5619e8c875e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f196ffff640 1 -- 172.21.15.19:0/1451011590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619e8cb0ce0 con 0x5619e8c86cc0 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f196ffff640 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c870a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f196000c430 tx=0x7f1960000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577955dd60048ada server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19567fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19567fc640 1 -- 172.21.15.19:0/1451011590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960074070 con 0x5619e8c86cc0 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19567fc640 1 -- 172.21.15.19:0/1451011590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1960057090 con 0x5619e8c86cc0 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.427+0000 7f19807ce740 1 -- 172.21.15.19:0/1451011590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19600542b0 con 0x5619e8c86cc0 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19567fc640 1 -- 172.21.15.19:0/1451011590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1960057290 con 0x5619e8c86cc0 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19807ce740 10 client.8199 fetch_fsmap learned FSMap version 65 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19807ce740 1 -- 172.21.15.19:0/1451011590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5619e8cba3a0 con 0x5619e8c86cc0 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19567fc640 1 -- 172.21.15.19:0/1451011590 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19600542b0 con 0x5619e8c86cc0 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19567fc640 1 -- 172.21.15.19:0/1451011590 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f19600890c0 con 0x5619e8c86cc0 2024-04-25T10:00:03.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19807ce740 10 client.8199 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:00:03.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19807ce740 10 client.8199 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:00:03.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19807ce740 20 client.8199 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:03.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19807ce740 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5619e8cf6b20 0x5619e8c9a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19807ce740 4 client.8199 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:00:03.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19807ce740 1 -- 172.21.15.19:0/1451011590 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5619e8bc7890 con 0x5619e8cf6b20 2024-04-25T10:00:03.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f1976ffd640 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5619e8cf6b20 0x5619e8c9a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f19567fc640 10 client.8199 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.428+0000 7f1976ffd640 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5619e8cf6b20 0x5619e8c9a6a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61bf5e24ae7f908f server_cookie=3ca177e52707da55 in_seq=0 out_seq=0 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.432+0000 7f19567fc640 1 -- 172.21.15.19:0/1451011590 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5619e8bc7890 con 0x5619e8cf6b20 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.432+0000 7f19567fc640 10 client.8199 handle_command_reply: tid=0 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.432+0000 7f19807ce740 1 client.8199 shutdown 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19807ce740 1 -- 172.21.15.19:0/1451011590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 msgr2=0x5619e8c870a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19807ce740 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c870a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f196000c430 tx=0x7f1960000bc0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19807ce740 1 -- 172.21.15.19:0/1451011590 shutdown_connections 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19807ce740 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5619e8cf6b20 0x5619e8c9a6a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19807ce740 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619e8c988d0 0x5619e8c875e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19807ce740 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5619e8c86cc0 0x5619e8c870a0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19567fc640 0 client.8199 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19807ce740 1 --2- 172.21.15.19:0/1451011590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5619e8c5c200 0x5619e8c5b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19807ce740 1 -- 172.21.15.19:0/1451011590 >> 172.21.15.19:0/1451011590 conn(0x5619e8c24050 msgr2=0x5619e8c861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19807ce740 1 -- 172.21.15.19:0/1451011590 shutdown_connections 2024-04-25T10:00:03.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19807ce740 1 -- 172.21.15.19:0/1451011590 wait complete. 2024-04-25T10:00:03.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f19807ce740 20 client.8199 trim_cache size 0 max 16384 2024-04-25T10:00:03.441 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:03.441 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:00:03.441 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "79a1f363-2990-4ff8-af83-45799bd6247b", 2024-04-25T10:00:03.441 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:00:03.441 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:03.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f197d9f4640 1 -- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f195806b4d0 msgr2=0x7f195806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f197d9f4640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f195806b4d0 0x7f195806d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1968034000 tx=0x7f196802e000 comp rx=0 tx=0).stop 2024-04-25T10:00:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f197d9f4640 1 -- 172.21.15.19:0/2502443246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19780b5620 msgr2=0x7f19780ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f197d9f4640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19780b5620 0x7f19780ca290 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f196000c460 tx=0x7f196000c920 comp rx=0 tx=0).stop 2024-04-25T10:00:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f197d9f4640 1 -- 172.21.15.19:0/2502443246 shutdown_connections 2024-04-25T10:00:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.433+0000 7f197d9f4640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f195806b4d0 0x7f195806d990 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.434+0000 7f197d9f4640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f197815b7d0 0x7f19780b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.434+0000 7f197d9f4640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19780b5620 0x7f19780ca290 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.434+0000 7f197d9f4640 1 --2- 172.21.15.19:0/2502443246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f197806c9e0 0x7f19780c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.434+0000 7f197d9f4640 1 -- 172.21.15.19:0/2502443246 >> 172.21.15.19:0/2502443246 conn(0x7f19780d0800 msgr2=0x7f19780d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.434+0000 7f197d9f4640 1 -- 172.21.15.19:0/2502443246 shutdown_connections 2024-04-25T10:00:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.434+0000 7f197d9f4640 1 -- 172.21.15.19:0/2502443246 wait complete. 2024-04-25T10:00:03.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:03.452 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T10:00:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.586+0000 7f1725cc4640 1 Processor -- start 2024-04-25T10:00:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.586+0000 7f1725cc4640 1 -- start start 2024-04-25T10:00:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f1725cc4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f172015dc20 0x7f17200ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f1725cc4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1720161710 0x7f17200cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f1725cc4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172015e000 con 0x7f17200b9330 2024-04-25T10:00:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f1725cc4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17200d1de0 con 0x7f172015dc20 2024-04-25T10:00:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f1725cc4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1720161c60 con 0x7f1720161710 2024-04-25T10:00:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f171f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f172015dc20 0x7f17200ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f171f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f172015dc20 0x7f17200ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47280/0 (socket says 172.21.15.19:47280) 2024-04-25T10:00:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f171f7fe640 1 -- 172.21.15.19:0/1243873609 learned_addr learned my addr 172.21.15.19:0/1243873609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f171effd640 1 --2- 172.21.15.19:0/1243873609 >> v2:172.21.15.112:3300/0 conn(0x7f1720161710 0x7f17200cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f171f7fe640 1 -- 172.21.15.19:0/1243873609 >> v2:172.21.15.112:3300/0 conn(0x7f1720161710 msgr2=0x7f17200cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f171f7fe640 1 --2- 172.21.15.19:0/1243873609 >> v2:172.21.15.112:3300/0 conn(0x7f1720161710 0x7f17200cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f171f7fe640 1 -- 172.21.15.19:0/1243873609 >> v1:172.21.15.112:6789/0 conn(0x7f17200b9330 legacy=0x7f172006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.587+0000 7f171f7fe640 1 -- 172.21.15.19:0/1243873609 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1720161f40 con 0x7f172015dc20 2024-04-25T10:00:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.588+0000 7f171f7fe640 1 --2- 172.21.15.19:0/1243873609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172015dc20 0x7f17200ce670 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f172006da90 tx=0x7f171002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3685c58eb6c40390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.588+0000 7f171e7fc640 1 -- 172.21.15.19:0/1243873609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710037070 con 0x7f172015dc20 2024-04-25T10:00:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.588+0000 7f171e7fc640 1 -- 172.21.15.19:0/1243873609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1710032440 con 0x7f172015dc20 2024-04-25T10:00:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.588+0000 7f171e7fc640 1 -- 172.21.15.19:0/1243873609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710032810 con 0x7f172015dc20 2024-04-25T10:00:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.589+0000 7f1725cc4640 1 -- 172.21.15.19:0/1243873609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172015dc20 msgr2=0x7f17200ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.589+0000 7f1725cc4640 1 --2- 172.21.15.19:0/1243873609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172015dc20 0x7f17200ce670 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f172006da90 tx=0x7f171002a270 comp rx=0 tx=0).stop 2024-04-25T10:00:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.589+0000 7f1725cc4640 1 -- 172.21.15.19:0/1243873609 shutdown_connections 2024-04-25T10:00:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.589+0000 7f1725cc4640 1 --2- 172.21.15.19:0/1243873609 >> v2:172.21.15.112:3300/0 conn(0x7f1720161710 0x7f17200cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.589+0000 7f1725cc4640 1 --2- 172.21.15.19:0/1243873609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172015dc20 0x7f17200ce670 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.589+0000 7f1725cc4640 1 -- 172.21.15.19:0/1243873609 >> 172.21.15.19:0/1243873609 conn(0x7f17200d0420 msgr2=0x7f17200d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.589+0000 7f1725cc4640 1 -- 172.21.15.19:0/1243873609 shutdown_connections 2024-04-25T10:00:03.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.589+0000 7f1725cc4640 1 -- 172.21.15.19:0/1243873609 wait complete. 2024-04-25T10:00:03.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.590+0000 7f1725cc4640 1 Processor -- start 2024-04-25T10:00:03.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.590+0000 7f1725cc4640 1 -- start start 2024-04-25T10:00:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f1725cc4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17200b9330 0x7f172015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f1725cc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172015dc20 0x7f17201e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f1725cc4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1720161710 0x7f172014fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f1725cc4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17200b5e50 con 0x7f172015dc20 2024-04-25T10:00:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f1725cc4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1720162500 con 0x7f1720161710 2024-04-25T10:00:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f1725cc4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17200b5fd0 con 0x7f17200b9330 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f171effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172015dc20 0x7f17201e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f171effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172015dc20 0x7f17201e74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47294/0 (socket says 172.21.15.19:47294) 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f171effd640 1 -- 172.21.15.19:0/1260858821 learned_addr learned my addr 172.21.15.19:0/1260858821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f171ffff640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1720161710 0x7f172014fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f171f7fe640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17200b9330 0x7f172015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f171effd640 1 -- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17200b9330 msgr2=0x7f172015fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f171effd640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17200b9330 0x7f172015fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f171effd640 1 -- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1720161710 msgr2=0x7f172014fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f171effd640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1720161710 0x7f172014fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.591+0000 7f171effd640 1 -- 172.21.15.19:0/1260858821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17201e7a30 con 0x7f172015dc20 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.592+0000 7f171effd640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172015dc20 0x7f17201e74f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f171400e8a0 tx=0x7f171400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.592+0000 7f1707fff640 1 -- 172.21.15.19:0/1260858821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714017070 con 0x7f172015dc20 2024-04-25T10:00:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.592+0000 7f1707fff640 1 -- 172.21.15.19:0/1260858821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171400c940 con 0x7f172015dc20 2024-04-25T10:00:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.592+0000 7f1725cc4640 1 -- 172.21.15.19:0/1260858821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17201e7d00 con 0x7f172015dc20 2024-04-25T10:00:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.592+0000 7f1707fff640 1 -- 172.21.15.19:0/1260858821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171400cd30 con 0x7f172015dc20 2024-04-25T10:00:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.592+0000 7f1725cc4640 1 -- 172.21.15.19:0/1260858821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17201e82c0 con 0x7f172015dc20 2024-04-25T10:00:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.593+0000 7f1707fff640 1 -- 172.21.15.19:0/1260858821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f171401a070 con 0x7f172015dc20 2024-04-25T10:00:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.594+0000 7f1707fff640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f170c06b4d0 0x7f170c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.594+0000 7f1707fff640 1 -- 172.21.15.19:0/1260858821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f171408a0e0 con 0x7f172015dc20 2024-04-25T10:00:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.594+0000 7f171f7fe640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f170c06b4d0 0x7f170c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.594+0000 7f171f7fe640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f170c06b4d0 0x7f170c06d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1710034000 tx=0x7f171002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.595+0000 7f17289c7740 1 Processor -- start 2024-04-25T10:00:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.595+0000 7f17289c7740 1 -- start start 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.596+0000 7f17289c7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3b63e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.596+0000 7f17289c7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3b62080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.596+0000 7f17289c7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b8cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.596+0000 7f17289c7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b3ac4b90 con 0x5626b3b8c760 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.596+0000 7f17289c7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b3a6de40 con 0x5626b3b9e370 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.596+0000 7f17289c7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626b3ac0c30 con 0x5626b3b61ca0 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b8cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3b63e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3b62080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b8cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47298/0 (socket says 172.21.15.19:47298) 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171ffff640 1 -- 172.21.15.19:0/91296094 learned_addr learned my addr 172.21.15.19:0/91296094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171ffff640 1 -- 172.21.15.19:0/91296094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 msgr2=0x5626b3b63e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171ffff640 1 --2- 172.21.15.19:0/91296094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3b63e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171ffff640 1 -- 172.21.15.19:0/91296094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 msgr2=0x5626b3b62080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171ffff640 1 --2- 172.21.15.19:0/91296094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3b62080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171ffff640 1 -- 172.21.15.19:0/91296094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626b38a37d0 con 0x5626b3b8c760 2024-04-25T10:00:03.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171effd640 1 --2- 172.21.15.19:0/91296094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3b62080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:03.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.597+0000 7f171ffff640 1 --2- 172.21.15.19:0/91296094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b8cbe0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f1718001e60 tx=0x7f17180014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f3df316bb58724 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.599+0000 7f17057fa640 1 -- 172.21.15.19:0/91296094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718002070 con 0x5626b3b8c760 2024-04-25T10:00:03.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.599+0000 7f17057fa640 1 -- 172.21.15.19:0/91296094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1718018070 con 0x5626b3b8c760 2024-04-25T10:00:03.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.599+0000 7f17057fa640 1 -- 172.21.15.19:0/91296094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718003970 con 0x5626b3b8c760 2024-04-25T10:00:03.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.600+0000 7f17289c7740 1 -- 172.21.15.19:0/91296094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 msgr2=0x5626b3b8cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.600+0000 7f17289c7740 1 --2- 172.21.15.19:0/91296094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b8cbe0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f1718001e60 tx=0x7f17180014b0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.600+0000 7f17289c7740 1 -- 172.21.15.19:0/91296094 shutdown_connections 2024-04-25T10:00:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.600+0000 7f17289c7740 1 --2- 172.21.15.19:0/91296094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b8cbe0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.600+0000 7f17289c7740 1 --2- 172.21.15.19:0/91296094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3b62080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.600+0000 7f17289c7740 1 --2- 172.21.15.19:0/91296094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3b63e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.600+0000 7f17289c7740 1 -- 172.21.15.19:0/91296094 >> 172.21.15.19:0/91296094 conn(0x5626b3b29870 msgr2=0x5626b3ba7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.600+0000 7f17289c7740 1 -- 172.21.15.19:0/91296094 shutdown_connections 2024-04-25T10:00:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.600+0000 7f17289c7740 1 -- 172.21.15.19:0/91296094 wait complete. 2024-04-25T10:00:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.601+0000 7f17289c7740 1 -- start start 2024-04-25T10:00:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.601+0000 7f17289c7740 1 Processor -- start 2024-04-25T10:00:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.601+0000 7f17289c7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3bab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.601+0000 7f17289c7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3ba24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.601+0000 7f17289c7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3bb75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.601+0000 7f17289c7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b3a813e0 con 0x5626b3b8c760 2024-04-25T10:00:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.601+0000 7f17289c7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b3b8a2b0 con 0x5626b3b9e370 2024-04-25T10:00:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.601+0000 7f17289c7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626b3bb8100 con 0x5626b3b61ca0 2024-04-25T10:00:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.601+0000 7f17289c7740 10 client.0 _ll_register_callbacks cb 0x5626b3a80db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f171effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3ba24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f171f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3bab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f171effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3ba24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47312/0 (socket says 172.21.15.19:47312) 2024-04-25T10:00:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f171effd640 1 -- 172.21.15.19:0/4270026651 learned_addr learned my addr 172.21.15.19:0/4270026651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f171ffff640 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3bb75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f171effd640 1 -- 172.21.15.19:0/4270026651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 msgr2=0x5626b3bab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f171effd640 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3bab570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f171effd640 1 -- 172.21.15.19:0/4270026651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 msgr2=0x5626b3bb75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f171effd640 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3bb75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f171effd640 1 -- 172.21.15.19:0/4270026651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626b37990f0 con 0x5626b3b8c760 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f171effd640 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3ba24f0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f171400c520 tx=0x7f1714058950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6816bfefbaf7d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f16fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f16fe7fc640 1 -- 172.21.15.19:0/4270026651 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714070020 con 0x5626b3b8c760 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f16fe7fc640 1 -- 172.21.15.19:0/4270026651 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f17140701c0 con 0x5626b3b8c760 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f17289c7740 1 -- 172.21.15.19:0/4270026651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1718018070 con 0x5626b3b8c760 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f16fe7fc640 1 -- 172.21.15.19:0/4270026651 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1714021070 con 0x5626b3b8c760 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f17289c7740 10 client.8211 fetch_fsmap learned FSMap version 65 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f17289c7740 1 -- 172.21.15.19:0/4270026651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5626b3903f60 con 0x5626b3b8c760 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.602+0000 7f16fe7fc640 1 -- 172.21.15.19:0/4270026651 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1718018070 con 0x5626b3b8c760 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.603+0000 7f16fe7fc640 1 -- 172.21.15.19:0/4270026651 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1714024060 con 0x5626b3b8c760 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.603+0000 7f17289c7740 10 client.8211 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:00:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.603+0000 7f17289c7740 10 client.8211 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:00:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.603+0000 7f17289c7740 20 client.8211 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.603+0000 7f17289c7740 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5626b3bb7b00 0x5626b3b8f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.603+0000 7f17289c7740 4 client.8211 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.603+0000 7f17289c7740 1 -- 172.21.15.19:0/4270026651 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626b3ad20d0 con 0x5626b3bb7b00 2024-04-25T10:00:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.603+0000 7f171f7fe640 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5626b3bb7b00 0x5626b3b8f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.603+0000 7f16fe7fc640 10 client.8211 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.603+0000 7f171f7fe640 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5626b3bb7b00 0x5626b3b8f2e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cbd25d33908992e server_cookie=75031b0569bb605c in_seq=0 out_seq=0 2024-04-25T10:00:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.603+0000 7f16fe7fc640 1 -- 172.21.15.19:0/4270026651 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626b3ad20d0 con 0x5626b3bb7b00 2024-04-25T10:00:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f16fe7fc640 10 client.8211 handle_command_reply: tid=0 2024-04-25T10:00:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 1 client.8211 shutdown 2024-04-25T10:00:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 1 -- 172.21.15.19:0/4270026651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 msgr2=0x5626b3ba24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3ba24f0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f171400c520 tx=0x7f1714058950 comp rx=0 tx=0).stop 2024-04-25T10:00:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 1 -- 172.21.15.19:0/4270026651 shutdown_connections 2024-04-25T10:00:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5626b3bb7b00 0x5626b3b8f2e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3bb75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3ba24f0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f16fe7fc640 0 client.8211 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 1 --2- 172.21.15.19:0/4270026651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3bab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 1 -- 172.21.15.19:0/4270026651 >> 172.21.15.19:0/4270026651 conn(0x5626b3b29870 msgr2=0x5626b3b8a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 1 -- 172.21.15.19:0/4270026651 shutdown_connections 2024-04-25T10:00:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 1 -- 172.21.15.19:0/4270026651 wait complete. 2024-04-25T10:00:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.604+0000 7f17289c7740 20 client.8211 trim_cache size 0 max 16384 2024-04-25T10:00:03.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.619+0000 7f17289c7740 1 Processor -- start 2024-04-25T10:00:03.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.619+0000 7f17289c7740 1 -- start start 2024-04-25T10:00:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f17289c7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3bc9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f17289c7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3bb72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f17289c7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3b8fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f17289c7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17202005f0 con 0x5626b3b8c760 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f17289c7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17201ff7c0 con 0x5626b3b9e370 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f17289c7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17201f4c10 con 0x5626b3b61ca0 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3bb72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3bb72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47318/0 (socket says 172.21.15.19:47318) 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171effd640 1 -- 172.21.15.19:0/2455470898 learned_addr learned my addr 172.21.15.19:0/2455470898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171ffff640 1 --2- 172.21.15.19:0/2455470898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3b8fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171f7fe640 1 --2- 172.21.15.19:0/2455470898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3bc9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171effd640 1 -- 172.21.15.19:0/2455470898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 msgr2=0x5626b3bc9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171effd640 1 --2- 172.21.15.19:0/2455470898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3bc9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171effd640 1 -- 172.21.15.19:0/2455470898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 msgr2=0x5626b3b8fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171effd640 1 --2- 172.21.15.19:0/2455470898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3b8fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171effd640 1 -- 172.21.15.19:0/2455470898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626b37721b0 con 0x5626b3b8c760 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171f7fe640 1 --2- 172.21.15.19:0/2455470898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3bc9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f171effd640 1 --2- 172.21.15.19:0/2455470898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3bb72b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f1714009160 tx=0x7f171405a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1fb745fbf47ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f1704ff9640 1 -- 172.21.15.19:0/2455470898 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714070040 con 0x5626b3b8c760 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f1704ff9640 1 -- 172.21.15.19:0/2455470898 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714077340 con 0x5626b3b8c760 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.620+0000 7f1704ff9640 1 -- 172.21.15.19:0/2455470898 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17140701e0 con 0x5626b3b8c760 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.621+0000 7f17289c7740 1 -- 172.21.15.19:0/2455470898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 msgr2=0x5626b3bb72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.621+0000 7f17289c7740 1 --2- 172.21.15.19:0/2455470898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3bb72b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f1714009160 tx=0x7f171405a000 comp rx=0 tx=0).stop 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.621+0000 7f17289c7740 1 -- 172.21.15.19:0/2455470898 shutdown_connections 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.621+0000 7f17289c7740 1 --2- 172.21.15.19:0/2455470898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b9e370 0x5626b3b8fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.621+0000 7f17289c7740 1 --2- 172.21.15.19:0/2455470898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3bb72b0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.621+0000 7f17289c7740 1 --2- 172.21.15.19:0/2455470898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b61ca0 0x5626b3bc9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.621+0000 7f17289c7740 1 -- 172.21.15.19:0/2455470898 >> 172.21.15.19:0/2455470898 conn(0x5626b3b29870 msgr2=0x5626b3b8f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.621+0000 7f17289c7740 1 -- 172.21.15.19:0/2455470898 shutdown_connections 2024-04-25T10:00:03.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.621+0000 7f17289c7740 1 -- 172.21.15.19:0/2455470898 wait complete. 2024-04-25T10:00:03.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.622+0000 7f17289c7740 1 -- start start 2024-04-25T10:00:03.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.622+0000 7f17289c7740 1 Processor -- start 2024-04-25T10:00:03.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.622+0000 7f17289c7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b61ca0 0x5626b3b8bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.622+0000 7f17289c7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b882c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.622+0000 7f17289c7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b9e370 0x5626b3ba8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.622+0000 7f17289c7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b3ba9bc0 con 0x5626b3b8c760 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.622+0000 7f17289c7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17201ff310 con 0x5626b3b61ca0 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.622+0000 7f17289c7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626b3ba8230 con 0x5626b3b9e370 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.622+0000 7f17289c7740 10 client.0 _ll_register_callbacks cb 0x7f1714054b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f171effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b882c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f171effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b882c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47328/0 (socket says 172.21.15.19:47328) 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f171ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b9e370 0x5626b3ba8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f171effd640 1 -- 172.21.15.19:0/295598685 learned_addr learned my addr 172.21.15.19:0/295598685 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f171f7fe640 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b61ca0 0x5626b3b8bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f171effd640 1 -- 172.21.15.19:0/295598685 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b9e370 msgr2=0x5626b3ba8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f171effd640 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b9e370 0x5626b3ba8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f171effd640 1 -- 172.21.15.19:0/295598685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b61ca0 msgr2=0x5626b3b8bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f171effd640 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b61ca0 0x5626b3b8bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f171effd640 1 -- 172.21.15.19:0/295598685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626b3bb5e80 con 0x5626b3b8c760 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f171effd640 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b882c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f171400e870 tx=0x7f17140883e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942cf27255111022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f16fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f16fe7fc640 1 -- 172.21.15.19:0/295598685 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714054560 con 0x5626b3b8c760 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f16fe7fc640 1 -- 172.21.15.19:0/295598685 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1714054700 con 0x5626b3b8c760 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f17289c7740 1 -- 172.21.15.19:0/295598685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1714077340 con 0x5626b3b8c760 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f16fe7fc640 1 -- 172.21.15.19:0/295598685 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17140576d0 con 0x5626b3b8c760 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f17289c7740 10 client.8217 fetch_fsmap learned FSMap version 65 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f17289c7740 1 -- 172.21.15.19:0/295598685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5626b3ba7710 con 0x5626b3b8c760 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.623+0000 7f16fe7fc640 1 -- 172.21.15.19:0/295598685 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714077340 con 0x5626b3b8c760 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f16fe7fc640 1 -- 172.21.15.19:0/295598685 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1714059a90 con 0x5626b3b8c760 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 10 client.8217 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 10 client.8217 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:00:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 20 client.8217 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5626b3ba7c60 0x5626b3ba9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 4 client.8217 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 1 -- 172.21.15.19:0/295598685 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x5626b3b5cc10 con 0x5626b3ba7c60 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f171f7fe640 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5626b3ba7c60 0x5626b3ba9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f16fe7fc640 10 client.8217 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f171f7fe640 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5626b3ba7c60 0x5626b3ba9470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62bfdb3938a53703 server_cookie=30ed8680b07d5821 in_seq=0 out_seq=0 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f16fe7fc640 1 -- 172.21.15.19:0/295598685 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5626b3b5cc10 con 0x5626b3ba7c60 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f16fe7fc640 10 client.8217 handle_command_reply: tid=0 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 1 client.8217 shutdown 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 1 -- 172.21.15.19:0/295598685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 msgr2=0x5626b3b882c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b882c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f171400e870 tx=0x7f17140883e0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 1 -- 172.21.15.19:0/295598685 shutdown_connections 2024-04-25T10:00:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5626b3ba7c60 0x5626b3ba9470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5626b3b9e370 0x5626b3ba8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626b3b8c760 0x5626b3b882c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 1 --2- 172.21.15.19:0/295598685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626b3b61ca0 0x5626b3b8bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f16fe7fc640 0 client.8217 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.624+0000 7f17289c7740 1 -- 172.21.15.19:0/295598685 >> 172.21.15.19:0/295598685 conn(0x5626b3b29870 msgr2=0x5626b3bab000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f17289c7740 1 -- 172.21.15.19:0/295598685 shutdown_connections 2024-04-25T10:00:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f17289c7740 1 -- 172.21.15.19:0/295598685 wait complete. 2024-04-25T10:00:03.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f17289c7740 20 client.8217 trim_cache size 0 max 16384 2024-04-25T10:00:03.635 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:03.635 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T10:00:03.636 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 -- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f170c06b4d0 msgr2=0x7f170c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f170c06b4d0 0x7f170c06d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1710034000 tx=0x7f171002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 -- 172.21.15.19:0/1260858821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172015dc20 msgr2=0x7f17201e74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172015dc20 0x7f17201e74f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f171400e8a0 tx=0x7f171400ed60 comp rx=0 tx=0).stop 2024-04-25T10:00:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 -- 172.21.15.19:0/1260858821 shutdown_connections 2024-04-25T10:00:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f170c06b4d0 0x7f170c06d990 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1720161710 0x7f172014fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f172015dc20 0x7f17201e74f0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 --2- 172.21.15.19:0/1260858821 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17200b9330 0x7f172015fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 -- 172.21.15.19:0/1260858821 >> 172.21.15.19:0/1260858821 conn(0x7f17200d0420 msgr2=0x7f17200b8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 -- 172.21.15.19:0/1260858821 shutdown_connections 2024-04-25T10:00:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.625+0000 7f1725cc4640 1 -- 172.21.15.19:0/1260858821 wait complete. 2024-04-25T10:00:03.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:03.640 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:00:03.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.738+0000 7f090ade7640 1 Processor -- start 2024-04-25T10:00:03.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.738+0000 7f090ade7640 1 -- start start 2024-04-25T10:00:03.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.739+0000 7f090ade7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090415de80 0x7f09040cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.739+0000 7f090ade7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f09040cefb0 0x7f09040cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.739+0000 7f090ade7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09040afc80 con 0x7f09040cf950 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.739+0000 7f090ade7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09040ce4d0 con 0x7f09040cefb0 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.739+0000 7f090ade7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09040ce7b0 con 0x7f090415de80 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.740+0000 7f0903fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f09040cefb0 0x7f09040cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.740+0000 7f0908b83640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090415de80 0x7f09040cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.740+0000 7f0909384640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f09040cf950 0x7f090406abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:33970/0 (socket says 172.21.15.19:33970) 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.740+0000 7f0909384640 1 -- 172.21.15.19:0/1558710412 learned_addr learned my addr 172.21.15.19:0/1558710412 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.740+0000 7f0903fff640 1 -- 172.21.15.19:0/1558710412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090415de80 msgr2=0x7f09040cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.740+0000 7f0903fff640 1 --2- 172.21.15.19:0/1558710412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090415de80 0x7f09040cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.740+0000 7f0903fff640 1 -- 172.21.15.19:0/1558710412 >> v1:172.21.15.112:6789/0 conn(0x7f09040cf950 legacy=0x7f090406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.740+0000 7f0903fff640 1 -- 172.21.15.19:0/1558710412 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040cea90 con 0x7f09040cefb0 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.740+0000 7f09037fe640 1 -- 172.21.15.19:0/1558710412 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299915408 0 0) 0x7f09040afc80 con 0x7f09040cf950 2024-04-25T10:00:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.741+0000 7f0903fff640 1 --2- 172.21.15.19:0/1558710412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09040cefb0 0x7f09040cf410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f08f40097a0 tx=0x7f08f402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a38eea38692d1125 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.741+0000 7f09037fe640 1 -- 172.21.15.19:0/1558710412 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4037070 con 0x7f09040cefb0 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.741+0000 7f09037fe640 1 -- 172.21.15.19:0/1558710412 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f4032440 con 0x7f09040cefb0 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.741+0000 7f09037fe640 1 -- 172.21.15.19:0/1558710412 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4032810 con 0x7f09040cefb0 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.741+0000 7f090ade7640 1 -- 172.21.15.19:0/1558710412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09040cefb0 msgr2=0x7f09040cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.741+0000 7f090ade7640 1 --2- 172.21.15.19:0/1558710412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09040cefb0 0x7f09040cf410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f08f40097a0 tx=0x7f08f402a180 comp rx=0 tx=0).stop 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.741+0000 7f090ade7640 1 -- 172.21.15.19:0/1558710412 shutdown_connections 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.741+0000 7f090ade7640 1 --2- 172.21.15.19:0/1558710412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09040cefb0 0x7f09040cf410 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.741+0000 7f090ade7640 1 --2- 172.21.15.19:0/1558710412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090415de80 0x7f09040cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.742+0000 7f090ade7640 1 -- 172.21.15.19:0/1558710412 >> 172.21.15.19:0/1558710412 conn(0x7f09040b2390 msgr2=0x7f09040b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.742+0000 7f090ade7640 1 -- 172.21.15.19:0/1558710412 shutdown_connections 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.742+0000 7f090ade7640 1 -- 172.21.15.19:0/1558710412 wait complete. 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.743+0000 7f090ade7640 1 Processor -- start 2024-04-25T10:00:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.743+0000 7f090ade7640 1 -- start start 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f090ade7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09040cefb0 0x7f09040ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f090ade7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09040cf950 0x7f090414e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f090ade7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f090415de80 0x7f09040ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f090ade7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040b3030 con 0x7f09040cf950 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f090ade7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040b2eb0 con 0x7f090415de80 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f090ade7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09040c5600 con 0x7f09040cefb0 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0903fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09040cf950 0x7f090414e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0903fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09040cf950 0x7f090414e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47346/0 (socket says 172.21.15.19:47346) 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0903fff640 1 -- 172.21.15.19:0/2279102117 learned_addr learned my addr 172.21.15.19:0/2279102117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0909384640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f090415de80 0x7f09040ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0908b83640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09040cefb0 0x7f09040ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0909384640 1 -- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09040cefb0 msgr2=0x7f09040ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0909384640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09040cefb0 0x7f09040ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0909384640 1 -- 172.21.15.19:0/2279102117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09040cf950 msgr2=0x7f090414e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0909384640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09040cf950 0x7f090414e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0909384640 1 -- 172.21.15.19:0/2279102117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040ca5f0 con 0x7f090415de80 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0903fff640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09040cf950 0x7f090414e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f0909384640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f090415de80 0x7f09040ae650 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f08ec007fd0 tx=0x7f08ec00c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.744+0000 7f09017fa640 1 -- 172.21.15.19:0/2279102117 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0074e0 con 0x7f090415de80 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.745+0000 7f09017fa640 1 -- 172.21.15.19:0/2279102117 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec007680 con 0x7f090415de80 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.745+0000 7f090ade7640 1 -- 172.21.15.19:0/2279102117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09040c9010 con 0x7f090415de80 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.745+0000 7f09017fa640 1 -- 172.21.15.19:0/2279102117 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec004a20 con 0x7f090415de80 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.745+0000 7f090ade7640 1 -- 172.21.15.19:0/2279102117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09040c98b0 con 0x7f090415de80 2024-04-25T10:00:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.746+0000 7f09017fa640 1 -- 172.21.15.19:0/2279102117 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08ec011070 con 0x7f090415de80 2024-04-25T10:00:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.746+0000 7f09017fa640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b500 0x7f08e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.746+0000 7f09017fa640 1 -- 172.21.15.19:0/2279102117 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f08ec089dd0 con 0x7f090415de80 2024-04-25T10:00:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.747+0000 7f0908b83640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b500 0x7f08e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.747+0000 7f0908b83640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b500 0x7f08e406d9c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f08fc011340 tx=0x7f08fc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.747+0000 7f090dbca740 1 Processor -- start 2024-04-25T10:00:03.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.747+0000 7f090dbca740 1 -- start start 2024-04-25T10:00:03.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.748+0000 7f090dbca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15b10 0x55d34dc18320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.748+0000 7f090dbca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15ef0 0x55d34dc18860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.748+0000 7f090dbca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dbe8770 0x55d34dbebf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.748+0000 7f090dbca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34db51da0 con 0x55d34dc15ef0 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.748+0000 7f090dbca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34dafab80 con 0x55d34dc15b10 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.748+0000 7f090dbca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d34db4de40 con 0x55d34dbe8770 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.748+0000 7f0903fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15ef0 0x55d34dc18860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.748+0000 7f0903fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15ef0 0x55d34dc18860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47354/0 (socket says 172.21.15.19:47354) 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.748+0000 7f0903fff640 1 -- 172.21.15.19:0/295592826 learned_addr learned my addr 172.21.15.19:0/295592826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.748+0000 7f0908b83640 1 --2- 172.21.15.19:0/295592826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15b10 0x55d34dc18320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.748+0000 7f0909384640 1 --2- 172.21.15.19:0/295592826 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dbe8770 0x55d34dbebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f0909384640 1 -- 172.21.15.19:0/295592826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15b10 msgr2=0x55d34dc18320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f0909384640 1 --2- 172.21.15.19:0/295592826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15b10 0x55d34dc18320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f0909384640 1 -- 172.21.15.19:0/295592826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15ef0 msgr2=0x55d34dc18860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f0909384640 1 --2- 172.21.15.19:0/295592826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15ef0 0x55d34dc18860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f0909384640 1 -- 172.21.15.19:0/295592826 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d34db04f10 con 0x55d34dbe8770 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f0903fff640 1 --2- 172.21.15.19:0/295592826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15ef0 0x55d34dc18860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f0909384640 1 --2- 172.21.15.19:0/295592826 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dbe8770 0x55d34dbebf80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0881c0 tx=0x7f08ec057e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53574cd1c38d06ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f08fa7fc640 1 -- 172.21.15.19:0/295592826 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0566e0 con 0x55d34dbe8770 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f08fa7fc640 1 -- 172.21.15.19:0/295592826 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec054a50 con 0x55d34dbe8770 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f08fa7fc640 1 -- 172.21.15.19:0/295592826 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec004bc0 con 0x55d34dbe8770 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f090dbca740 1 -- 172.21.15.19:0/295592826 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dbe8770 msgr2=0x55d34dbebf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f090dbca740 1 --2- 172.21.15.19:0/295592826 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dbe8770 0x55d34dbebf80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0881c0 tx=0x7f08ec057e30 comp rx=0 tx=0).stop 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f090dbca740 1 -- 172.21.15.19:0/295592826 shutdown_connections 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f090dbca740 1 --2- 172.21.15.19:0/295592826 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dbe8770 0x55d34dbebf80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f090dbca740 1 --2- 172.21.15.19:0/295592826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15ef0 0x55d34dc18860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f090dbca740 1 --2- 172.21.15.19:0/295592826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15b10 0x55d34dc18320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.749+0000 7f090dbca740 1 -- 172.21.15.19:0/295592826 >> 172.21.15.19:0/295592826 conn(0x55d34dbea9e0 msgr2=0x55d34dc33320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.750+0000 7f090dbca740 1 -- 172.21.15.19:0/295592826 shutdown_connections 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.750+0000 7f090dbca740 1 -- 172.21.15.19:0/295592826 wait complete. 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.750+0000 7f090dbca740 1 -- start start 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.750+0000 7f090dbca740 1 Processor -- start 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f090dbca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15b10 0x55d34dc2f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f090dbca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15ef0 0x55d34dc35080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f090dbca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc0e820 0x55d34dc14190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f090dbca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34dbedb90 con 0x55d34dc15b10 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f090dbca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34daf04b0 con 0x55d34dc15ef0 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f090dbca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d34dbede70 con 0x55d34dc0e820 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f090dbca740 10 client.0 _ll_register_callbacks cb 0x55d34db30180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f0908b83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15b10 0x55d34dc2f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f0908b83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15b10 0x55d34dc2f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47368/0 (socket says 172.21.15.19:47368) 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f0908b83640 1 -- 172.21.15.19:0/518277046 learned_addr learned my addr 172.21.15.19:0/518277046 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f0909384640 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc0e820 0x55d34dc14190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f0903fff640 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15ef0 0x55d34dc35080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f0908b83640 1 -- 172.21.15.19:0/518277046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc0e820 msgr2=0x55d34dc14190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f0908b83640 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc0e820 0x55d34dc14190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f0908b83640 1 -- 172.21.15.19:0/518277046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15ef0 msgr2=0x55d34dc35080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.751+0000 7f0908b83640 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15ef0 0x55d34dc35080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f0908b83640 1 -- 172.21.15.19:0/518277046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d34d9307d0 con 0x55d34dc15b10 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f0903fff640 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15ef0 0x55d34dc35080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f0909384640 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc0e820 0x55d34dc14190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f0908b83640 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15b10 0x55d34dc2f240 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f08fc004620 tx=0x7f08fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e340851bb4e7977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f08e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f08e37fe640 1 -- 172.21.15.19:0/518277046 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc027070 con 0x55d34dc15b10 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f08e37fe640 1 -- 172.21.15.19:0/518277046 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f08fc023620 con 0x55d34dc15b10 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f090dbca740 1 -- 172.21.15.19:0/518277046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08ec054a50 con 0x55d34dc15b10 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f08e37fe640 1 -- 172.21.15.19:0/518277046 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08fc032050 con 0x55d34dc15b10 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f090dbca740 10 client.8226 fetch_fsmap learned FSMap version 65 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f090dbca740 1 -- 172.21.15.19:0/518277046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55d34d8663f0 con 0x55d34dc15b10 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f08e37fe640 1 -- 172.21.15.19:0/518277046 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec054a50 con 0x55d34dc15b10 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f08e37fe640 1 -- 172.21.15.19:0/518277046 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f08fc035020 con 0x55d34dc15b10 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f090dbca740 10 client.8226 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f090dbca740 10 client.8226 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f090dbca740 20 client.8226 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f090dbca740 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55d34dc46a00 0x55d34dc46e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f090dbca740 4 client.8226 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f090dbca740 1 -- 172.21.15.19:0/518277046 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d34db5f2e0 con 0x55d34dc46a00 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.752+0000 7f0903fff640 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55d34dc46a00 0x55d34dc46e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.753+0000 7f0903fff640 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55d34dc46a00 0x55d34dc46e20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7655985081b0fa65 server_cookie=eb40e04833897042 in_seq=0 out_seq=0 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.753+0000 7f08e37fe640 10 client.8226 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.753+0000 7f08e37fe640 1 -- 172.21.15.19:0/518277046 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d34db5f2e0 con 0x55d34dc46a00 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.753+0000 7f08e37fe640 10 client.8226 handle_command_reply: tid=0 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.753+0000 7f090dbca740 1 client.8226 shutdown 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f090dbca740 1 -- 172.21.15.19:0/518277046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15b10 msgr2=0x55d34dc2f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f090dbca740 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15b10 0x55d34dc2f240 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f08fc004620 tx=0x7f08fc004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f090dbca740 1 -- 172.21.15.19:0/518277046 shutdown_connections 2024-04-25T10:00:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f090dbca740 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55d34dc46a00 0x55d34dc46e20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f090dbca740 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc0e820 0x55d34dc14190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f090dbca740 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15ef0 0x55d34dc35080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f08e37fe640 0 client.8226 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f090dbca740 1 --2- 172.21.15.19:0/518277046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc15b10 0x55d34dc2f240 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f090dbca740 1 -- 172.21.15.19:0/518277046 >> 172.21.15.19:0/518277046 conn(0x55d34dbe8770 msgr2=0x55d34dc10380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f090dbca740 1 -- 172.21.15.19:0/518277046 shutdown_connections 2024-04-25T10:00:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f090dbca740 1 -- 172.21.15.19:0/518277046 wait complete. 2024-04-25T10:00:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.754+0000 7f090dbca740 20 client.8226 trim_cache size 0 max 16384 2024-04-25T10:00:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.769+0000 7f090dbca740 1 Processor -- start 2024-04-25T10:00:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.769+0000 7f090dbca740 1 -- start start 2024-04-25T10:00:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f090dbca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbe77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f090dbca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15b10 0x55d34dc2d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f090dbca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc15ef0 0x55d34dc2de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f090dbca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0904200500 con 0x55d34dc0e820 2024-04-25T10:00:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f090dbca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09041ff6a0 con 0x55d34dc15b10 2024-04-25T10:00:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f090dbca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09040c8380 con 0x55d34dc15ef0 2024-04-25T10:00:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f0908b83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbe77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f0908b83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbe77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47370/0 (socket says 172.21.15.19:47370) 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f0908b83640 1 -- 172.21.15.19:0/3807632036 learned_addr learned my addr 172.21.15.19:0/3807632036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f0909384640 1 --2- 172.21.15.19:0/3807632036 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc15ef0 0x55d34dc2de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f0903fff640 1 --2- 172.21.15.19:0/3807632036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15b10 0x55d34dc2d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f0908b83640 1 -- 172.21.15.19:0/3807632036 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc15ef0 msgr2=0x55d34dc2de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f0908b83640 1 --2- 172.21.15.19:0/3807632036 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc15ef0 0x55d34dc2de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f0908b83640 1 -- 172.21.15.19:0/3807632036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15b10 msgr2=0x55d34dc2d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f0908b83640 1 --2- 172.21.15.19:0/3807632036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15b10 0x55d34dc2d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f0908b83640 1 -- 172.21.15.19:0/3807632036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d34d8260f0 con 0x55d34dc0e820 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f0908b83640 1 --2- 172.21.15.19:0/3807632036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbe77f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0045f0 tx=0x7f08fc023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84dfe3b57a4006ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f08f9ffb640 1 -- 172.21.15.19:0/3807632036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc027040 con 0x55d34dc0e820 2024-04-25T10:00:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f08f9ffb640 1 -- 172.21.15.19:0/3807632036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc023620 con 0x55d34dc0e820 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.770+0000 7f08f9ffb640 1 -- 172.21.15.19:0/3807632036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc023b10 con 0x55d34dc0e820 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.771+0000 7f090dbca740 1 -- 172.21.15.19:0/3807632036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 msgr2=0x55d34dbe77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.771+0000 7f090dbca740 1 --2- 172.21.15.19:0/3807632036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbe77f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0045f0 tx=0x7f08fc023000 comp rx=0 tx=0).stop 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.771+0000 7f090dbca740 1 -- 172.21.15.19:0/3807632036 shutdown_connections 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.771+0000 7f090dbca740 1 --2- 172.21.15.19:0/3807632036 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc15ef0 0x55d34dc2de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.771+0000 7f090dbca740 1 --2- 172.21.15.19:0/3807632036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15b10 0x55d34dc2d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.771+0000 7f090dbca740 1 --2- 172.21.15.19:0/3807632036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbe77f0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.771+0000 7f090dbca740 1 -- 172.21.15.19:0/3807632036 >> 172.21.15.19:0/3807632036 conn(0x55d34dbe8770 msgr2=0x55d34dc2f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.771+0000 7f090dbca740 1 -- 172.21.15.19:0/3807632036 shutdown_connections 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.771+0000 7f090dbca740 1 -- 172.21.15.19:0/3807632036 wait complete. 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.771+0000 7f090dbca740 1 -- start start 2024-04-25T10:00:03.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.772+0000 7f090dbca740 1 Processor -- start 2024-04-25T10:00:03.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.772+0000 7f090dbca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbecc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.772+0000 7f090dbca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc15b10 0x55d34dc9d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.772+0000 7f090dbca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15ef0 0x55d34dc9dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.772+0000 7f090dbca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09041ff0a0 con 0x55d34dc0e820 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.772+0000 7f090dbca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34dc2f9e0 con 0x55d34dc15ef0 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.772+0000 7f090dbca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d34dc9e520 con 0x55d34dc15b10 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.772+0000 7f090dbca740 10 client.0 _ll_register_callbacks cb 0x7f08ec054680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f0908b83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f0908b83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbecc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47382/0 (socket says 172.21.15.19:47382) 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f0908b83640 1 -- 172.21.15.19:0/2351697300 learned_addr learned my addr 172.21.15.19:0/2351697300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f0903fff640 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc15b10 0x55d34dc9d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f0909384640 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15ef0 0x55d34dc9dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f0908b83640 1 -- 172.21.15.19:0/2351697300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc15b10 msgr2=0x55d34dc9d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f0908b83640 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc15b10 0x55d34dc9d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f0908b83640 1 -- 172.21.15.19:0/2351697300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15ef0 msgr2=0x55d34dc9dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f0908b83640 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15ef0 0x55d34dc9dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f0908b83640 1 -- 172.21.15.19:0/2351697300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d34dc338b0 con 0x55d34dc0e820 2024-04-25T10:00:03.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f0908b83640 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbecc00 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00b010 tx=0x7f08fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264389f31f1de3ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f08e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f08e37fe640 1 -- 172.21.15.19:0/2351697300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc027070 con 0x55d34dc0e820 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f08e37fe640 1 -- 172.21.15.19:0/2351697300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f08fc023d20 con 0x55d34dc0e820 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f090dbca740 1 -- 172.21.15.19:0/2351697300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08fc023620 con 0x55d34dc0e820 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f08e37fe640 1 -- 172.21.15.19:0/2351697300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08fc04f050 con 0x55d34dc0e820 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f090dbca740 10 client.8232 fetch_fsmap learned FSMap version 65 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.773+0000 7f090dbca740 1 -- 172.21.15.19:0/2351697300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55d34dc9e260 con 0x55d34dc0e820 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f08e37fe640 1 -- 172.21.15.19:0/2351697300 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc023620 con 0x55d34dc0e820 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f08e37fe640 1 -- 172.21.15.19:0/2351697300 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f08fc052020 con 0x55d34dc0e820 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f090dbca740 10 client.8232 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f090dbca740 10 client.8232 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f090dbca740 20 client.8232 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f090dbca740 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55d34dbe7a80 0x55d34dc32bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f090dbca740 4 client.8232 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f090dbca740 1 -- 172.21.15.19:0/2351697300 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d34db59630 con 0x55d34dbe7a80 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f0903fff640 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55d34dbe7a80 0x55d34dc32bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f08e37fe640 10 client.8232 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f0903fff640 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55d34dbe7a80 0x55d34dc32bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fdc4e10f1a1a63f server_cookie=99daf05702486782 in_seq=0 out_seq=0 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f08e37fe640 1 -- 172.21.15.19:0/2351697300 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55d34db59630 con 0x55d34dbe7a80 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f08e37fe640 10 client.8232 handle_command_reply: tid=0 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.774+0000 7f090dbca740 1 client.8232 shutdown 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090dbca740 1 -- 172.21.15.19:0/2351697300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 msgr2=0x55d34dbecc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090dbca740 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbecc00 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00b010 tx=0x7f08fc016000 comp rx=0 tx=0).stop 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090dbca740 1 -- 172.21.15.19:0/2351697300 shutdown_connections 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090dbca740 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55d34dbe7a80 0x55d34dc32bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090dbca740 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d34dc15ef0 0x55d34dc9dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090dbca740 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d34dc15b10 0x55d34dc9d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f08e37fe640 0 client.8232 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090dbca740 1 --2- 172.21.15.19:0/2351697300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d34dc0e820 0x55d34dbecc00 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090dbca740 1 -- 172.21.15.19:0/2351697300 >> 172.21.15.19:0/2351697300 conn(0x55d34dbe8770 msgr2=0x55d34dc2c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090dbca740 1 -- 172.21.15.19:0/2351697300 shutdown_connections 2024-04-25T10:00:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090dbca740 1 -- 172.21.15.19:0/2351697300 wait complete. 2024-04-25T10:00:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090dbca740 20 client.8232 trim_cache size 0 max 16384 2024-04-25T10:00:03.785 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:03.785 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:00:03.785 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:00:03.785 INFO:teuthology.orchestra.run.smithi019.stdout: "79a1f363-2990-4ff8-af83-45799bd6247b": { 2024-04-25T10:00:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:00:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "79a1f363-2990-4ff8-af83-45799bd6247b", 2024-04-25T10:00:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive" 2024-04-25T10:00:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:03.786 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 -- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b500 msgr2=0x7f08e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b500 0x7f08e406d9c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f08fc011340 tx=0x7f08fc00b040 comp rx=0 tx=0).stop 2024-04-25T10:00:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 -- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f090415de80 msgr2=0x7f09040ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f090415de80 0x7f09040ae650 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f08ec007fd0 tx=0x7f08ec00c870 comp rx=0 tx=0).stop 2024-04-25T10:00:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 -- 172.21.15.19:0/2279102117 shutdown_connections 2024-04-25T10:00:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b500 0x7f08e406d9c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f090415de80 0x7f09040ae650 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09040cf950 0x7f090414e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 --2- 172.21.15.19:0/2279102117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09040cefb0 0x7f09040ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 -- 172.21.15.19:0/2279102117 >> 172.21.15.19:0/2279102117 conn(0x7f09040b2390 msgr2=0x7f09041587c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 -- 172.21.15.19:0/2279102117 shutdown_connections 2024-04-25T10:00:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.775+0000 7f090ade7640 1 -- 172.21.15.19:0/2279102117 wait complete. 2024-04-25T10:00:03.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:03.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.887+0000 7f1c681a5640 1 Processor -- start 2024-04-25T10:00:03.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.887+0000 7f1c681a5640 1 -- start start 2024-04-25T10:00:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c681a5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c6015dd30 0x7f1c600b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c681a5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1c600b9770 0x7f1c600d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c681a5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c600d1df0 con 0x7f1c600d2e90 2024-04-25T10:00:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c681a5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c600b7a50 con 0x7f1c6015dd30 2024-04-25T10:00:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c681a5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c600d20d0 con 0x7f1c600b9770 2024-04-25T10:00:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c65f41640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c6015dd30 0x7f1c600b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c65f41640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c6015dd30 0x7f1c600b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47396/0 (socket says 172.21.15.19:47396) 2024-04-25T10:00:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c65f41640 1 -- 172.21.15.19:0/4247529415 learned_addr learned my addr 172.21.15.19:0/4247529415 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c65740640 1 --2- 172.21.15.19:0/4247529415 >> v2:172.21.15.112:3300/0 conn(0x7f1c600b9770 0x7f1c600d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c65f41640 1 -- 172.21.15.19:0/4247529415 >> v2:172.21.15.112:3300/0 conn(0x7f1c600b9770 msgr2=0x7f1c600d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c65f41640 1 --2- 172.21.15.19:0/4247529415 >> v2:172.21.15.112:3300/0 conn(0x7f1c600b9770 0x7f1c600d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c65f41640 1 -- 172.21.15.19:0/4247529415 >> v1:172.21.15.112:6789/0 conn(0x7f1c600d2e90 legacy=0x7f1c600b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c65f41640 1 -- 172.21.15.19:0/4247529415 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c600d23b0 con 0x7f1c6015dd30 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c64f3f640 1 -- 172.21.15.19:0/4247529415 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055073118 0 0) 0x7f1c600d1df0 con 0x7f1c600d2e90 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.888+0000 7f1c65740640 1 --2- 172.21.15.19:0/4247529415 >> v2:172.21.15.112:3300/0 conn(0x7f1c600b9770 0x7f1c600d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.889+0000 7f1c65f41640 1 --2- 172.21.15.19:0/4247529415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6015dd30 0x7f1c600b9230 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f1c600b6540 tx=0x7f1c5c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcbdf55bc39aa27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.889+0000 7f1c64f3f640 1 -- 172.21.15.19:0/4247529415 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c037070 con 0x7f1c6015dd30 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.889+0000 7f1c64f3f640 1 -- 172.21.15.19:0/4247529415 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c032440 con 0x7f1c6015dd30 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.889+0000 7f1c64f3f640 1 -- 172.21.15.19:0/4247529415 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c032810 con 0x7f1c6015dd30 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.890+0000 7f1c681a5640 1 -- 172.21.15.19:0/4247529415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6015dd30 msgr2=0x7f1c600b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.890+0000 7f1c681a5640 1 --2- 172.21.15.19:0/4247529415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6015dd30 0x7f1c600b9230 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f1c600b6540 tx=0x7f1c5c02a150 comp rx=0 tx=0).stop 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.890+0000 7f1c681a5640 1 -- 172.21.15.19:0/4247529415 shutdown_connections 2024-04-25T10:00:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.890+0000 7f1c681a5640 1 --2- 172.21.15.19:0/4247529415 >> v2:172.21.15.112:3300/0 conn(0x7f1c600b9770 0x7f1c600d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.890+0000 7f1c681a5640 1 --2- 172.21.15.19:0/4247529415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6015dd30 0x7f1c600b9230 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.890+0000 7f1c681a5640 1 -- 172.21.15.19:0/4247529415 >> 172.21.15.19:0/4247529415 conn(0x7f1c60061c30 msgr2=0x7f1c6006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.890+0000 7f1c681a5640 1 -- 172.21.15.19:0/4247529415 shutdown_connections 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.890+0000 7f1c681a5640 1 -- 172.21.15.19:0/4247529415 wait complete. 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.891+0000 7f1c681a5640 1 Processor -- start 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.891+0000 7f1c681a5640 1 -- start start 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c681a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c600b9770 0x7f1c60151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c681a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c600d2e90 0x7f1c600cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c681a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c6015dd30 0x7f1c600cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c681a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c600d1a10 con 0x7f1c600d2e90 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c681a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c600d2970 con 0x7f1c600b9770 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c681a5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c600d1b90 con 0x7f1c6015dd30 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c65740640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c600d2e90 0x7f1c600cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c65740640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c600d2e90 0x7f1c600cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47404/0 (socket says 172.21.15.19:47404) 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c65740640 1 -- 172.21.15.19:0/3184173920 learned_addr learned my addr 172.21.15.19:0/3184173920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c65f41640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c600b9770 0x7f1c60151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c66742640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c6015dd30 0x7f1c600cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c65740640 1 -- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c6015dd30 msgr2=0x7f1c600cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c65740640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c6015dd30 0x7f1c600cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c65740640 1 -- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c600b9770 msgr2=0x7f1c60151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c65740640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c600b9770 0x7f1c60151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c65740640 1 -- 172.21.15.19:0/3184173920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c60151840 con 0x7f1c600d2e90 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.892+0000 7f1c65740640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c600d2e90 0x7f1c600cce90 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000ec10 tx=0x7f1c5000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.893+0000 7f1c567fc640 1 -- 172.21.15.19:0/3184173920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5000cc30 con 0x7f1c600d2e90 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.893+0000 7f1c567fc640 1 -- 172.21.15.19:0/3184173920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5000cdd0 con 0x7f1c600d2e90 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.893+0000 7f1c681a5640 1 -- 172.21.15.19:0/3184173920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c601f4d50 con 0x7f1c600d2e90 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.893+0000 7f1c567fc640 1 -- 172.21.15.19:0/3184173920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50005bb0 con 0x7f1c600d2e90 2024-04-25T10:00:03.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.893+0000 7f1c681a5640 1 -- 172.21.15.19:0/3184173920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6015ff60 con 0x7f1c600d2e90 2024-04-25T10:00:03.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.894+0000 7f1c681a5640 1 -- 172.21.15.19:0/3184173920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c60073990 con 0x7f1c600d2e90 2024-04-25T10:00:03.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.894+0000 7f1c567fc640 1 -- 172.21.15.19:0/3184173920 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c50006030 con 0x7f1c600d2e90 2024-04-25T10:00:03.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.894+0000 7f1c567fc640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c4406b4d0 0x7f1c4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.895+0000 7f1c567fc640 1 -- 172.21.15.19:0/3184173920 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c5008a6f0 con 0x7f1c600d2e90 2024-04-25T10:00:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.895+0000 7f1c65f41640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c4406b4d0 0x7f1c4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.895+0000 7f1c65f41640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c4406b4d0 0x7f1c4406d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c034000 tx=0x7f1c5c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:03.899+0000 7f1c567fc640 1 -- 172.21.15.19:0/3184173920 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c50017020 con 0x7f1c600d2e90 2024-04-25T10:00:04.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.122+0000 7f1c681a5640 1 -- 172.21.15.19:0/3184173920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c600b8750 con 0x7f1c600d2e90 2024-04-25T10:00:04.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.122+0000 7f1c567fc640 1 -- 172.21.15.19:0/3184173920 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1c50058e20 con 0x7f1c600d2e90 2024-04-25T10:00:04.127 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:04.127 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:04.127 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:04.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 -- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c4406b4d0 msgr2=0x7f1c4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:04.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c4406b4d0 0x7f1c4406d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c034000 tx=0x7f1c5c02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:04.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 -- 172.21.15.19:0/3184173920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c600d2e90 msgr2=0x7f1c600cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:04.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c600d2e90 0x7f1c600cce90 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000ec10 tx=0x7f1c5000c520 comp rx=0 tx=0).stop 2024-04-25T10:00:04.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 -- 172.21.15.19:0/3184173920 shutdown_connections 2024-04-25T10:00:04.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c4406b4d0 0x7f1c4406d990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c6015dd30 0x7f1c600cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c600d2e90 0x7f1c600cce90 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 --2- 172.21.15.19:0/3184173920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c600b9770 0x7f1c60151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 -- 172.21.15.19:0/3184173920 >> 172.21.15.19:0/3184173920 conn(0x7f1c60061c30 msgr2=0x7f1c600d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:04.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 -- 172.21.15.19:0/3184173920 shutdown_connections 2024-04-25T10:00:04.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.124+0000 7f1c681a5640 1 -- 172.21.15.19:0/3184173920 wait complete. 2024-04-25T10:00:04.139 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:00:04.139 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:04.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.235+0000 7f5ca92cb640 1 Processor -- start 2024-04-25T10:00:04.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.235+0000 7f5ca92cb640 1 -- start start 2024-04-25T10:00:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca92cb640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ca415dbe0 0x7f5ca40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca92cb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ca40b2cb0 0x7f5ca40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca92cb640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca40b97a0 con 0x7f5ca40b92a0 2024-04-25T10:00:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca92cb640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca415dfc0 con 0x7f5ca415dbe0 2024-04-25T10:00:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca92cb640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca40d1a00 con 0x7f5ca40b2cb0 2024-04-25T10:00:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca2ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ca415dbe0 0x7f5ca40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca37fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5ca40b92a0 0x7f5ca40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60278/0 (socket says 172.21.15.19:60278) 2024-04-25T10:00:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca2ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ca415dbe0 0x7f5ca40d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47418/0 (socket says 172.21.15.19:47418) 2024-04-25T10:00:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca37fe640 1 -- 172.21.15.19:0/608915996 learned_addr learned my addr 172.21.15.19:0/608915996 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:04.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5c9a7fc640 1 --2- 172.21.15.19:0/608915996 >> v2:172.21.15.112:3300/0 conn(0x7f5ca40b2cb0 0x7f5ca40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:04.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca2ffd640 1 -- 172.21.15.19:0/608915996 >> v2:172.21.15.112:3300/0 conn(0x7f5ca40b2cb0 msgr2=0x7f5ca40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:04.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca2ffd640 1 --2- 172.21.15.19:0/608915996 >> v2:172.21.15.112:3300/0 conn(0x7f5ca40b2cb0 0x7f5ca40d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca2ffd640 1 -- 172.21.15.19:0/608915996 >> v1:172.21.15.19:6789/0 conn(0x7f5ca40b92a0 legacy=0x7f5ca40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:04.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca27fc640 1 -- 172.21.15.19:0/608915996 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675066847 0 0) 0x7f5ca40b97a0 con 0x7f5ca40b92a0 2024-04-25T10:00:04.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.236+0000 7f5ca2ffd640 1 -- 172.21.15.19:0/608915996 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca40d1c50 con 0x7f5ca415dbe0 2024-04-25T10:00:04.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.237+0000 7f5ca2ffd640 1 --2- 172.21.15.19:0/608915996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca415dbe0 0x7f5ca40d0910 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f5ca406cff0 tx=0x7f5c9402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0556dedf030f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:04.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.237+0000 7f5ca27fc640 1 -- 172.21.15.19:0/608915996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94037070 con 0x7f5ca415dbe0 2024-04-25T10:00:04.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.237+0000 7f5ca27fc640 1 -- 172.21.15.19:0/608915996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c94032440 con 0x7f5ca415dbe0 2024-04-25T10:00:04.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.237+0000 7f5ca27fc640 1 -- 172.21.15.19:0/608915996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9402ae60 con 0x7f5ca415dbe0 2024-04-25T10:00:04.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.238+0000 7f5ca92cb640 1 -- 172.21.15.19:0/608915996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca415dbe0 msgr2=0x7f5ca40d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:04.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.238+0000 7f5ca92cb640 1 --2- 172.21.15.19:0/608915996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca415dbe0 0x7f5ca40d0910 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f5ca406cff0 tx=0x7f5c9402a110 comp rx=0 tx=0).stop 2024-04-25T10:00:04.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.238+0000 7f5ca92cb640 1 -- 172.21.15.19:0/608915996 shutdown_connections 2024-04-25T10:00:04.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.238+0000 7f5ca92cb640 1 --2- 172.21.15.19:0/608915996 >> v2:172.21.15.112:3300/0 conn(0x7f5ca40b2cb0 0x7f5ca40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.238+0000 7f5ca92cb640 1 --2- 172.21.15.19:0/608915996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca415dbe0 0x7f5ca40d0910 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.238+0000 7f5ca92cb640 1 -- 172.21.15.19:0/608915996 >> 172.21.15.19:0/608915996 conn(0x7f5ca4061a20 msgr2=0x7f5ca4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:04.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.238+0000 7f5ca92cb640 1 -- 172.21.15.19:0/608915996 shutdown_connections 2024-04-25T10:00:04.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.238+0000 7f5ca92cb640 1 -- 172.21.15.19:0/608915996 wait complete. 2024-04-25T10:00:04.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.239+0000 7f5ca92cb640 1 Processor -- start 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.239+0000 7f5ca92cb640 1 -- start start 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5ca92cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca40b2cb0 0x7f5ca4153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5ca92cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca40b92a0 0x7f5ca40cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5ca92cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ca415dbe0 0x7f5ca40cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5ca92cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40b6ab0 con 0x7f5ca40b92a0 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5ca92cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40d2210 con 0x7f5ca40b2cb0 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5ca92cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40b6c30 con 0x7f5ca415dbe0 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5c9a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca40b92a0 0x7f5ca40cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5c9a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca40b92a0 0x7f5ca40cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47430/0 (socket says 172.21.15.19:47430) 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5c9a7fc640 1 -- 172.21.15.19:0/3321817181 learned_addr learned my addr 172.21.15.19:0/3321817181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5ca2ffd640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca40b2cb0 0x7f5ca4153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5ca37fe640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ca415dbe0 0x7f5ca40cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5c9a7fc640 1 -- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ca415dbe0 msgr2=0x7f5ca40cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:04.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5c9a7fc640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ca415dbe0 0x7f5ca40cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5c9a7fc640 1 -- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca40b2cb0 msgr2=0x7f5ca4153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:04.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5c9a7fc640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca40b2cb0 0x7f5ca4153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5c9a7fc640 1 -- 172.21.15.19:0/3321817181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca40cc2f0 con 0x7f5ca40b92a0 2024-04-25T10:00:04.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5c9a7fc640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca40b92a0 0x7f5ca40cb7a0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00d7d0 tx=0x7f5c8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:04.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.240+0000 7f5c9bfff640 1 -- 172.21.15.19:0/3321817181 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00b660 con 0x7f5ca40b92a0 2024-04-25T10:00:04.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.241+0000 7f5c9bfff640 1 -- 172.21.15.19:0/3321817181 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c019070 con 0x7f5ca40b92a0 2024-04-25T10:00:04.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.241+0000 7f5ca92cb640 1 -- 172.21.15.19:0/3321817181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca40cc5d0 con 0x7f5ca40b92a0 2024-04-25T10:00:04.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.241+0000 7f5c9bfff640 1 -- 172.21.15.19:0/3321817181 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00ba60 con 0x7f5ca40b92a0 2024-04-25T10:00:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.241+0000 7f5ca92cb640 1 -- 172.21.15.19:0/3321817181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca40ccb90 con 0x7f5ca40b92a0 2024-04-25T10:00:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.242+0000 7f5ca92cb640 1 -- 172.21.15.19:0/3321817181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca4073990 con 0x7f5ca40b92a0 2024-04-25T10:00:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.242+0000 7f5c9bfff640 1 -- 172.21.15.19:0/3321817181 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c8c00bc00 con 0x7f5ca40b92a0 2024-04-25T10:00:04.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.243+0000 7f5c9bfff640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c8406b500 0x7f5c8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:04.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.243+0000 7f5c9bfff640 1 -- 172.21.15.19:0/3321817181 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5c8c08a610 con 0x7f5ca40b92a0 2024-04-25T10:00:04.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.243+0000 7f5ca2ffd640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c8406b500 0x7f5c8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:04.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.243+0000 7f5ca2ffd640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c8406b500 0x7f5c8406d9c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5c94034000 tx=0x7f5c9402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:04.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.247+0000 7f5c9bfff640 1 -- 172.21.15.19:0/3321817181 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c8c058cc0 con 0x7f5ca40b92a0 2024-04-25T10:00:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.503+0000 7f5ca92cb640 1 -- 172.21.15.19:0/3321817181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5ca415b040 con 0x7f5ca40b92a0 2024-04-25T10:00:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.504+0000 7f5c9bfff640 1 -- 172.21.15.19:0/3321817181 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7f5c8c05cd60 con 0x7f5ca40b92a0 2024-04-25T10:00:04.508 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:04.508 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":441,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":608,"data_bytes":613909344,"bytes_used":1482391552,"bytes_avail":771611721728,"bytes_total":773094113280,"write_bytes_sec":51104945,"read_op_per_sec":0,"write_op_per_sec":49},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8025}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-04-25T09:59:37.000521+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 -- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c8406b500 msgr2=0x7f5c8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c8406b500 0x7f5c8406d9c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5c94034000 tx=0x7f5c9402e000 comp rx=0 tx=0).stop 2024-04-25T10:00:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 -- 172.21.15.19:0/3321817181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca40b92a0 msgr2=0x7f5ca40cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca40b92a0 0x7f5ca40cb7a0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00d7d0 tx=0x7f5c8c00dc90 comp rx=0 tx=0).stop 2024-04-25T10:00:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 -- 172.21.15.19:0/3321817181 shutdown_connections 2024-04-25T10:00:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5c8406b500 0x7f5c8406d9c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ca415dbe0 0x7f5ca40cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ca40b92a0 0x7f5ca40cb7a0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 --2- 172.21.15.19:0/3321817181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ca40b2cb0 0x7f5ca4153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 -- 172.21.15.19:0/3321817181 >> 172.21.15.19:0/3321817181 conn(0x7f5ca4061a20 msgr2=0x7f5ca40d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 -- 172.21.15.19:0/3321817181 shutdown_connections 2024-04-25T10:00:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:04.506+0000 7f5ca92cb640 1 -- 172.21.15.19:0/3321817181 wait complete. 2024-04-25T10:00:04.520 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:00:05.521 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.632+0000 7efd1824d640 1 Processor -- start 2024-04-25T10:00:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.632+0000 7efd1824d640 1 -- start start 2024-04-25T10:00:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.633+0000 7efd1824d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd1015db70 0x7efd10162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.633+0000 7efd1824d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efd100b6cb0 0x7efd100b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.633+0000 7efd1824d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd10162690 con 0x7efd100b5a60 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.633+0000 7efd1824d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd100d3800 con 0x7efd100b6cb0 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.633+0000 7efd1824d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd100ceee0 con 0x7efd1015db70 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.633+0000 7efd167ea640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efd100b5a60 0x7efd1006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60288/0 (socket says 172.21.15.19:60288) 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.633+0000 7efd167ea640 1 -- 172.21.15.19:0/2073033417 learned_addr learned my addr 172.21.15.19:0/2073033417 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.633+0000 7efd15fe9640 1 --2- 172.21.15.19:0/2073033417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd1015db70 0x7efd10162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.633+0000 7efd157e8640 1 --2- 172.21.15.19:0/2073033417 >> v2:172.21.15.112:3300/0 conn(0x7efd100b6cb0 0x7efd100b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.633+0000 7efd14fe7640 1 -- 172.21.15.19:0/2073033417 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857197834 0 0) 0x7efd10162690 con 0x7efd100b5a60 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.633+0000 7efd14fe7640 1 -- 172.21.15.19:0/2073033417 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcfc003520 con 0x7efd100b5a60 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.634+0000 7efd15fe9640 1 -- 172.21.15.19:0/2073033417 >> v2:172.21.15.112:3300/0 conn(0x7efd100b6cb0 msgr2=0x7efd100b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.634+0000 7efd15fe9640 1 --2- 172.21.15.19:0/2073033417 >> v2:172.21.15.112:3300/0 conn(0x7efd100b6cb0 0x7efd100b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.634+0000 7efd15fe9640 1 -- 172.21.15.19:0/2073033417 >> v1:172.21.15.19:6789/0 conn(0x7efd100b5a60 legacy=0x7efd1006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.634+0000 7efd15fe9640 1 -- 172.21.15.19:0/2073033417 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd100d1d10 con 0x7efd1015db70 2024-04-25T10:00:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd15fe9640 1 --2- 172.21.15.19:0/2073033417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd1015db70 0x7efd10162150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0097a0 tx=0x7efd0c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e07581b5037e7ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd14fe7640 1 -- 172.21.15.19:0/2073033417 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c037070 con 0x7efd1015db70 2024-04-25T10:00:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd14fe7640 1 -- 172.21.15.19:0/2073033417 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c032440 con 0x7efd1015db70 2024-04-25T10:00:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd14fe7640 1 -- 172.21.15.19:0/2073033417 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c032810 con 0x7efd1015db70 2024-04-25T10:00:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd1824d640 1 -- 172.21.15.19:0/2073033417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd1015db70 msgr2=0x7efd10162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd1824d640 1 --2- 172.21.15.19:0/2073033417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd1015db70 0x7efd10162150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0097a0 tx=0x7efd0c02a180 comp rx=0 tx=0).stop 2024-04-25T10:00:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd1824d640 1 -- 172.21.15.19:0/2073033417 shutdown_connections 2024-04-25T10:00:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd1824d640 1 --2- 172.21.15.19:0/2073033417 >> v2:172.21.15.112:3300/0 conn(0x7efd100b6cb0 0x7efd100b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd1824d640 1 --2- 172.21.15.19:0/2073033417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd1015db70 0x7efd10162150 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd1824d640 1 -- 172.21.15.19:0/2073033417 >> 172.21.15.19:0/2073033417 conn(0x7efd100d0520 msgr2=0x7efd100d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd1824d640 1 -- 172.21.15.19:0/2073033417 shutdown_connections 2024-04-25T10:00:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.635+0000 7efd1824d640 1 -- 172.21.15.19:0/2073033417 wait complete. 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.637+0000 7efd1824d640 1 Processor -- start 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.637+0000 7efd1824d640 1 -- start start 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.637+0000 7efd1824d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd100b5a60 0x7efd101f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.637+0000 7efd1824d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd100b6cb0 0x7efd100cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.637+0000 7efd1824d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1015db70 0x7efd100ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.637+0000 7efd1824d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1015df50 con 0x7efd100b6cb0 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.637+0000 7efd1824d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd100d22d0 con 0x7efd1015db70 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.637+0000 7efd1824d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd100ce530 con 0x7efd100b5a60 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd157e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd100b6cb0 0x7efd100cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd157e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd100b6cb0 0x7efd100cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47440/0 (socket says 172.21.15.19:47440) 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd157e8640 1 -- 172.21.15.19:0/3664766120 learned_addr learned my addr 172.21.15.19:0/3664766120 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd15fe9640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd100b5a60 0x7efd101f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd167ea640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1015db70 0x7efd100ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd157e8640 1 -- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd100b5a60 msgr2=0x7efd101f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd157e8640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd100b5a60 0x7efd101f0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd157e8640 1 -- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1015db70 msgr2=0x7efd100ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd157e8640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1015db70 0x7efd100ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd157e8640 1 -- 172.21.15.19:0/3664766120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd101f45f0 con 0x7efd100b6cb0 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd157e8640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd100b6cb0 0x7efd100cc980 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7efd0000e980 tx=0x7efd0000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd067fc640 1 -- 172.21.15.19:0/3664766120 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00017070 con 0x7efd100b6cb0 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd1824d640 1 -- 172.21.15.19:0/3664766120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd101f1010 con 0x7efd100b6cb0 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd067fc640 1 -- 172.21.15.19:0/3664766120 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0000ca00 con 0x7efd100b6cb0 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd067fc640 1 -- 172.21.15.19:0/3664766120 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0000cdd0 con 0x7efd100b6cb0 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.638+0000 7efd1824d640 1 -- 172.21.15.19:0/3664766120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd101f7090 con 0x7efd100b6cb0 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.639+0000 7efd1824d640 1 -- 172.21.15.19:0/3664766120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd10073990 con 0x7efd100b6cb0 2024-04-25T10:00:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.640+0000 7efd067fc640 1 -- 172.21.15.19:0/3664766120 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd0001a070 con 0x7efd100b6cb0 2024-04-25T10:00:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.640+0000 7efd067fc640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcf406b4d0 0x7efcf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.640+0000 7efd067fc640 1 -- 172.21.15.19:0/3664766120 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd0008a360 con 0x7efd100b6cb0 2024-04-25T10:00:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.641+0000 7efd15fe9640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcf406b4d0 0x7efcf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.641+0000 7efd15fe9640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcf406b4d0 0x7efcf406d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efd0c034000 tx=0x7efd0c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:05.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.645+0000 7efd067fc640 1 -- 172.21.15.19:0/3664766120 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd00058970 con 0x7efd100b6cb0 2024-04-25T10:00:05.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.861+0000 7efd1824d640 1 -- 172.21.15.19:0/3664766120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd100d1820 con 0x7efd100b6cb0 2024-04-25T10:00:05.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.862+0000 7efd067fc640 1 -- 172.21.15.19:0/3664766120 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7efd0005ca10 con 0x7efd100b6cb0 2024-04-25T10:00:05.866 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:05.866 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:05.866 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 -- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcf406b4d0 msgr2=0x7efcf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcf406b4d0 0x7efcf406d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efd0c034000 tx=0x7efd0c02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 -- 172.21.15.19:0/3664766120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd100b6cb0 msgr2=0x7efd100cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd100b6cb0 0x7efd100cc980 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7efd0000e980 tx=0x7efd0000ee40 comp rx=0 tx=0).stop 2024-04-25T10:00:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 -- 172.21.15.19:0/3664766120 shutdown_connections 2024-04-25T10:00:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efcf406b4d0 0x7efcf406d990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1015db70 0x7efd100ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd100b6cb0 0x7efd100cc980 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 --2- 172.21.15.19:0/3664766120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd100b5a60 0x7efd101f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 -- 172.21.15.19:0/3664766120 >> 172.21.15.19:0/3664766120 conn(0x7efd100d0520 msgr2=0x7efd100d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 -- 172.21.15.19:0/3664766120 shutdown_connections 2024-04-25T10:00:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.864+0000 7efd1824d640 1 -- 172.21.15.19:0/3664766120 wait complete. 2024-04-25T10:00:05.879 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:00:05.879 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:05.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.974+0000 7fe200c38640 1 Processor -- start 2024-04-25T10:00:05.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.974+0000 7fe200c38640 1 -- start start 2024-04-25T10:00:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.975+0000 7fe200c38640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe1fc15dd30 0x7fe1fc0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.975+0000 7fe200c38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1fc0b9770 0x7fe1fc0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.975+0000 7fe200c38640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1fc0d1df0 con 0x7fe1fc0d2e90 2024-04-25T10:00:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.975+0000 7fe200c38640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1fc0b7a50 con 0x7fe1fc15dd30 2024-04-25T10:00:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.975+0000 7fe200c38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0d20d0 con 0x7fe1fc0b9770 2024-04-25T10:00:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.975+0000 7fe1fa59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe1fc15dd30 0x7fe1fc0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.975+0000 7fe1fa59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe1fc15dd30 0x7fe1fc0b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47448/0 (socket says 172.21.15.19:47448) 2024-04-25T10:00:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.975+0000 7fe1fa59c640 1 -- 172.21.15.19:0/2949676273 learned_addr learned my addr 172.21.15.19:0/2949676273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.975+0000 7fe1f9d9b640 1 --2- 172.21.15.19:0/2949676273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1fc0b9770 0x7fe1fc0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.976+0000 7fe1fa59c640 1 -- 172.21.15.19:0/2949676273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1fc0b9770 msgr2=0x7fe1fc0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:05.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.976+0000 7fe1fa59c640 1 --2- 172.21.15.19:0/2949676273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1fc0b9770 0x7fe1fc0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.976+0000 7fe1fa59c640 1 -- 172.21.15.19:0/2949676273 >> v1:172.21.15.112:6789/0 conn(0x7fe1fc0d2e90 legacy=0x7fe1fc0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:05.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.976+0000 7fe1fa59c640 1 -- 172.21.15.19:0/2949676273 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0d23b0 con 0x7fe1fc15dd30 2024-04-25T10:00:05.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.976+0000 7fe1fa59c640 1 --2- 172.21.15.19:0/2949676273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1fc15dd30 0x7fe1fc0b9230 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009e50 tx=0x7fe1f402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca41f95e39b08f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:05.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.977+0000 7fe1f959a640 1 -- 172.21.15.19:0/2949676273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4036070 con 0x7fe1fc15dd30 2024-04-25T10:00:05.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.977+0000 7fe1f959a640 1 -- 172.21.15.19:0/2949676273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4031600 con 0x7fe1fc15dd30 2024-04-25T10:00:05.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.977+0000 7fe1f959a640 1 -- 172.21.15.19:0/2949676273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40319d0 con 0x7fe1fc15dd30 2024-04-25T10:00:05.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.977+0000 7fe200c38640 1 -- 172.21.15.19:0/2949676273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1fc15dd30 msgr2=0x7fe1fc0b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:05.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.977+0000 7fe200c38640 1 --2- 172.21.15.19:0/2949676273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1fc15dd30 0x7fe1fc0b9230 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009e50 tx=0x7fe1f402a680 comp rx=0 tx=0).stop 2024-04-25T10:00:05.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.977+0000 7fe200c38640 1 -- 172.21.15.19:0/2949676273 shutdown_connections 2024-04-25T10:00:05.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.977+0000 7fe200c38640 1 --2- 172.21.15.19:0/2949676273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1fc0b9770 0x7fe1fc0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.977+0000 7fe200c38640 1 --2- 172.21.15.19:0/2949676273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1fc15dd30 0x7fe1fc0b9230 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.977+0000 7fe200c38640 1 -- 172.21.15.19:0/2949676273 >> 172.21.15.19:0/2949676273 conn(0x7fe1fc061c30 msgr2=0x7fe1fc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.977+0000 7fe200c38640 1 -- 172.21.15.19:0/2949676273 shutdown_connections 2024-04-25T10:00:05.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.977+0000 7fe200c38640 1 -- 172.21.15.19:0/2949676273 wait complete. 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.978+0000 7fe200c38640 1 Processor -- start 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.979+0000 7fe200c38640 1 -- start start 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.979+0000 7fe200c38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1fc0b9770 0x7fe1fc15f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.979+0000 7fe200c38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1fc0d2e90 0x7fe1fc153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.979+0000 7fe200c38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1fc15dd30 0x7fe1fc154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.979+0000 7fe200c38640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0d1a10 con 0x7fe1fc0d2e90 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.979+0000 7fe200c38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0d2970 con 0x7fe1fc0b9770 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.979+0000 7fe200c38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0d1b90 con 0x7fe1fc15dd30 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.979+0000 7fe1f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1fc0d2e90 0x7fe1fc153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.979+0000 7fe1f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1fc0d2e90 0x7fe1fc153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47464/0 (socket says 172.21.15.19:47464) 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.979+0000 7fe1f9d9b640 1 -- 172.21.15.19:0/734380433 learned_addr learned my addr 172.21.15.19:0/734380433 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe1fa59c640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1fc0b9770 0x7fe1fc15f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe1fad9d640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1fc15dd30 0x7fe1fc154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe1f9d9b640 1 -- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1fc15dd30 msgr2=0x7fe1fc154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe1f9d9b640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1fc15dd30 0x7fe1fc154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe1f9d9b640 1 -- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1fc0b9770 msgr2=0x7fe1fc15f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe1f9d9b640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1fc0b9770 0x7fe1fc15f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe1f9d9b640 1 -- 172.21.15.19:0/734380433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc15f740 con 0x7fe1fc0d2e90 2024-04-25T10:00:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe1f9d9b640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1fc0d2e90 0x7fe1fc153b40 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000e900 tx=0x7fe1f000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe1eaffd640 1 -- 172.21.15.19:0/734380433 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0017070 con 0x7fe1fc0d2e90 2024-04-25T10:00:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe1eaffd640 1 -- 172.21.15.19:0/734380433 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f000ca00 con 0x7fe1fc0d2e90 2024-04-25T10:00:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe1eaffd640 1 -- 172.21.15.19:0/734380433 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000cdb0 con 0x7fe1fc0d2e90 2024-04-25T10:00:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe200c38640 1 -- 172.21.15.19:0/734380433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1fc15fa20 con 0x7fe1fc0d2e90 2024-04-25T10:00:05.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.980+0000 7fe200c38640 1 -- 172.21.15.19:0/734380433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1fc1602c0 con 0x7fe1fc0d2e90 2024-04-25T10:00:05.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.981+0000 7fe200c38640 1 -- 172.21.15.19:0/734380433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1fc15fbf0 con 0x7fe1fc0d2e90 2024-04-25T10:00:05.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.982+0000 7fe1eaffd640 1 -- 172.21.15.19:0/734380433 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1f001a070 con 0x7fe1fc0d2e90 2024-04-25T10:00:05.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.982+0000 7fe1eaffd640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe1dc06b4d0 0x7fe1dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:05.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.982+0000 7fe1eaffd640 1 -- 172.21.15.19:0/734380433 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1f008a7d0 con 0x7fe1fc0d2e90 2024-04-25T10:00:05.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.982+0000 7fe1fa59c640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe1dc06b4d0 0x7fe1dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:05.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.983+0000 7fe1fa59c640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe1dc06b4d0 0x7fe1dc06d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4033000 tx=0x7fe1f4009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:05.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:05.986+0000 7fe1eaffd640 1 -- 172.21.15.19:0/734380433 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1f00594c0 con 0x7fe1fc0d2e90 2024-04-25T10:00:06.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.238+0000 7fe200c38640 1 -- 172.21.15.19:0/734380433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe1fc073990 con 0x7fe1fc0d2e90 2024-04-25T10:00:06.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.239+0000 7fe1eaffd640 1 -- 172.21.15.19:0/734380433 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1637 (secure 0 0 0) 0x7fe1f0058dc0 con 0x7fe1fc0d2e90 2024-04-25T10:00:06.245 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:06.245 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":443,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076423385,"bytes_used":2527879168,"bytes_avail":770566234112,"bytes_total":773094113280,"write_bytes_sec":89646096,"read_op_per_sec":0,"write_op_per_sec":87},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8025}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T10:00:05.006768+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:06.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 -- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe1dc06b4d0 msgr2=0x7fe1dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:06.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe1dc06b4d0 0x7fe1dc06d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4033000 tx=0x7fe1f4009000 comp rx=0 tx=0).stop 2024-04-25T10:00:06.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 -- 172.21.15.19:0/734380433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1fc0d2e90 msgr2=0x7fe1fc153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:06.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1fc0d2e90 0x7fe1fc153b40 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000e900 tx=0x7fe1f000edc0 comp rx=0 tx=0).stop 2024-04-25T10:00:06.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 -- 172.21.15.19:0/734380433 shutdown_connections 2024-04-25T10:00:06.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe1dc06b4d0 0x7fe1dc06d990 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:06.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1fc15dd30 0x7fe1fc154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:06.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1fc0d2e90 0x7fe1fc153b40 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:06.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 --2- 172.21.15.19:0/734380433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1fc0b9770 0x7fe1fc15f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:06.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 -- 172.21.15.19:0/734380433 >> 172.21.15.19:0/734380433 conn(0x7fe1fc061c30 msgr2=0x7fe1fc0c9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:06.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 -- 172.21.15.19:0/734380433 shutdown_connections 2024-04-25T10:00:06.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:06.241+0000 7fe200c38640 1 -- 172.21.15.19:0/734380433 wait complete. 2024-04-25T10:00:06.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:00:07.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:07.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.365+0000 7f303339e640 1 Processor -- start 2024-04-25T10:00:07.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.366+0000 7f303339e640 1 -- start start 2024-04-25T10:00:07.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.366+0000 7f303339e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f302c0cdfd0 0x7f302c161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.366+0000 7f303339e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302c06ce10 con 0x7f302c06c270 2024-04-25T10:00:07.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.366+0000 7f303339e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302c1624f0 con 0x7f302c0ce3b0 2024-04-25T10:00:07.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.366+0000 7f303339e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302c06fdf0 con 0x7f302c0cdfd0 2024-04-25T10:00:07.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f3032b9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f302c06c270 0x7f302c06c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60296/0 (socket says 172.21.15.19:60296) 2024-04-25T10:00:07.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f3032b9d640 1 -- 172.21.15.19:0/506097626 learned_addr learned my addr 172.21.15.19:0/506097626 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:07.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303239c640 1 --2- 172.21.15.19:0/506097626 >> v2:172.21.15.112:3300/0 conn(0x7f302c0cdfd0 0x7f302c161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553085051 0 0) 0x7f3028001490 con 0x7f302c06c270 2024-04-25T10:00:07.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3018003520 con 0x7f302c06c270 2024-04-25T10:00:07.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 502223497 0 0) 0x7f3018003520 con 0x7f302c06c270 2024-04-25T10:00:07.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 >> v2:172.21.15.112:3300/0 conn(0x7f302c0cdfd0 msgr2=0x7f302c161fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:07.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 --2- 172.21.15.19:0/506097626 >> v2:172.21.15.112:3300/0 conn(0x7f302c0cdfd0 0x7f302c161fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 >> v1:172.21.15.112:6789/0 conn(0x7f302c0ce3b0 legacy=0x7f302c070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c0d0320 con 0x7f302c06c270 2024-04-25T10:00:07.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f3028001bc0 con 0x7f302c06c270 2024-04-25T10:00:07.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 >> v1:172.21.15.19:6789/0 conn(0x7f302c06c270 legacy=0x7f302c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180039b0 0x7f3018003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3018004800 0x7f3018004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30180051c0 0x7f301800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c06fdf0 con 0x7f3018004800 2024-04-25T10:00:07.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301800c110 con 0x7f30180039b0 2024-04-25T10:00:07.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301800c310 con 0x7f30180051c0 2024-04-25T10:00:07.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f3031b9b640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3018004800 0x7f3018004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.367+0000 7f303239c640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180039b0 0x7f3018003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.368+0000 7f3032b9d640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30180051c0 0x7f301800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.368+0000 7f303239c640 1 -- 172.21.15.19:0/506097626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30180051c0 msgr2=0x7f301800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:07.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.368+0000 7f303239c640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30180051c0 0x7f301800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.368+0000 7f303239c640 1 -- 172.21.15.19:0/506097626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3018004800 msgr2=0x7f3018004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:07.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.368+0000 7f303239c640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3018004800 0x7f3018004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.368+0000 7f3032b9d640 1 -- 172.21.15.19:0/506097626 reap_dead start 2024-04-25T10:00:07.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.368+0000 7f303239c640 1 -- 172.21.15.19:0/506097626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3018003520 con 0x7f30180039b0 2024-04-25T10:00:07.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.368+0000 7f3031b9b640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3018004800 0x7f3018004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:07.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.369+0000 7f303239c640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180039b0 0x7f3018003d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3024009ac0 tx=0x7f3018005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73d4734832692b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:07.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.369+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024036070 con 0x7f30180039b0 2024-04-25T10:00:07.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.369+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3024031400 con 0x7f30180039b0 2024-04-25T10:00:07.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.369+0000 7f303139a640 1 -- 172.21.15.19:0/506097626 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024009690 con 0x7f30180039b0 2024-04-25T10:00:07.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.369+0000 7f303339e640 1 -- 172.21.15.19:0/506097626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180039b0 msgr2=0x7f3018003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.369+0000 7f303339e640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180039b0 0x7f3018003d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3024009ac0 tx=0x7f3018005830 comp rx=0 tx=0).stop 2024-04-25T10:00:07.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.369+0000 7f303339e640 1 -- 172.21.15.19:0/506097626 shutdown_connections 2024-04-25T10:00:07.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.369+0000 7f303339e640 1 --2- 172.21.15.19:0/506097626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30180039b0 0x7f3018003d90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.369+0000 7f303339e640 1 -- 172.21.15.19:0/506097626 >> 172.21.15.19:0/506097626 conn(0x7f302c0d1630 msgr2=0x7f302c0d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:07.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.369+0000 7f303339e640 1 -- 172.21.15.19:0/506097626 shutdown_connections 2024-04-25T10:00:07.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.369+0000 7f303339e640 1 -- 172.21.15.19:0/506097626 wait complete. 2024-04-25T10:00:07.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.371+0000 7f303339e640 1 Processor -- start 2024-04-25T10:00:07.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.371+0000 7f303339e640 1 -- start start 2024-04-25T10:00:07.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303339e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30180039b0 0x7f302c155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303339e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f302c0c5160 0x7f302c0c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303339e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f302c15e480 0x7f302c15e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303339e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0cf940 con 0x7f30180039b0 2024-04-25T10:00:07.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303339e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c06fa80 con 0x7f302c0c5160 2024-04-25T10:00:07.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303339e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0d0600 con 0x7f302c15e480 2024-04-25T10:00:07.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303239c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30180039b0 0x7f302c155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303239c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30180039b0 0x7f302c155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47482/0 (socket says 172.21.15.19:47482) 2024-04-25T10:00:07.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303239c640 1 -- 172.21.15.19:0/2576589978 learned_addr learned my addr 172.21.15.19:0/2576589978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:07.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f3031b9b640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f302c0c5160 0x7f302c0c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f3032b9d640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f302c15e480 0x7f302c15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303239c640 1 -- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f302c15e480 msgr2=0x7f302c15e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303239c640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f302c15e480 0x7f302c15e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303239c640 1 -- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f302c0c5160 msgr2=0x7f302c0c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303239c640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f302c0c5160 0x7f302c0c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303239c640 1 -- 172.21.15.19:0/2576589978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c156370 con 0x7f30180039b0 2024-04-25T10:00:07.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f3031b9b640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f302c0c5160 0x7f302c0c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:07.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f3032b9d640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f302c15e480 0x7f302c15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:07.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f303239c640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30180039b0 0x7f302c155e30 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3024009be0 tx=0x7f3024004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:07.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.372+0000 7f3022ffd640 1 -- 172.21.15.19:0/2576589978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302403f020 con 0x7f30180039b0 2024-04-25T10:00:07.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.373+0000 7f3022ffd640 1 -- 172.21.15.19:0/2576589978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3024037040 con 0x7f30180039b0 2024-04-25T10:00:07.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.373+0000 7f303339e640 1 -- 172.21.15.19:0/2576589978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c0c56d0 con 0x7f30180039b0 2024-04-25T10:00:07.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.373+0000 7f3022ffd640 1 -- 172.21.15.19:0/2576589978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024036070 con 0x7f30180039b0 2024-04-25T10:00:07.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.373+0000 7f303339e640 1 -- 172.21.15.19:0/2576589978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c0c5c90 con 0x7f30180039b0 2024-04-25T10:00:07.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.374+0000 7f303339e640 1 -- 172.21.15.19:0/2576589978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302c073990 con 0x7f30180039b0 2024-04-25T10:00:07.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.374+0000 7f3022ffd640 1 -- 172.21.15.19:0/2576589978 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3024042020 con 0x7f30180039b0 2024-04-25T10:00:07.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.375+0000 7f3022ffd640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f301006b500 0x7f301006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.375+0000 7f3022ffd640 1 -- 172.21.15.19:0/2576589978 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f30240a9fd0 con 0x7f30180039b0 2024-04-25T10:00:07.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.375+0000 7f3031b9b640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f301006b500 0x7f301006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.375+0000 7f3031b9b640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f301006b500 0x7f301006d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f302c0c6650 tx=0x7f301c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:07.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.379+0000 7f3022ffd640 1 -- 172.21.15.19:0/2576589978 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3024049070 con 0x7f30180039b0 2024-04-25T10:00:07.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.596+0000 7f303339e640 1 -- 172.21.15.19:0/2576589978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f302c06f4e0 con 0x7f30180039b0 2024-04-25T10:00:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.597+0000 7f3022ffd640 1 -- 172.21.15.19:0/2576589978 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3024078d50 con 0x7f30180039b0 2024-04-25T10:00:07.601 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:07.601 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.598+0000 7f303339e640 1 -- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f301006b500 msgr2=0x7f301006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.598+0000 7f303339e640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f301006b500 0x7f301006d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f302c0c6650 tx=0x7f301c009040 comp rx=0 tx=0).stop 2024-04-25T10:00:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.598+0000 7f303339e640 1 -- 172.21.15.19:0/2576589978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30180039b0 msgr2=0x7f302c155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.598+0000 7f303339e640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30180039b0 0x7f302c155e30 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3024009be0 tx=0x7f3024004730 comp rx=0 tx=0).stop 2024-04-25T10:00:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.599+0000 7f303339e640 1 -- 172.21.15.19:0/2576589978 shutdown_connections 2024-04-25T10:00:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.599+0000 7f303339e640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f301006b500 0x7f301006d9c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.599+0000 7f303339e640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f302c15e480 0x7f302c15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.599+0000 7f303339e640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f302c0c5160 0x7f302c0c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.599+0000 7f303339e640 1 --2- 172.21.15.19:0/2576589978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30180039b0 0x7f302c155e30 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.599+0000 7f303339e640 1 -- 172.21.15.19:0/2576589978 >> 172.21.15.19:0/2576589978 conn(0x7f302c0d1630 msgr2=0x7f302c0cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.599+0000 7f303339e640 1 -- 172.21.15.19:0/2576589978 shutdown_connections 2024-04-25T10:00:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.599+0000 7f303339e640 1 -- 172.21.15.19:0/2576589978 wait complete. 2024-04-25T10:00:07.614 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:00:07.614 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:07.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.710+0000 7fca1e215640 1 Processor -- start 2024-04-25T10:00:07.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.710+0000 7fca1e215640 1 -- start start 2024-04-25T10:00:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca1e215640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca180cedf0 0x7fca181616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca1e215640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1815e000 con 0x7fca1815dc20 2024-04-25T10:00:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca1e215640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca180b6bb0 con 0x7fca180b93a0 2024-04-25T10:00:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca1e215640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca180b6030 con 0x7fca180cedf0 2024-04-25T10:00:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca177fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fca1815dc20 0x7fca180ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55014/0 (socket says 172.21.15.19:55014) 2024-04-25T10:00:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca177fe640 1 -- 172.21.15.19:0/4100625025 learned_addr learned my addr 172.21.15.19:0/4100625025 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca16ffd640 1 --2- 172.21.15.19:0/4100625025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca180cedf0 0x7fca181616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca167fc640 1 -- 172.21.15.19:0/4100625025 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666039670 0 0) 0x7fca08001110 con 0x7fca1815dc20 2024-04-25T10:00:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca167fc640 1 -- 172.21.15.19:0/4100625025 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca00003520 con 0x7fca1815dc20 2024-04-25T10:00:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca167fc640 1 -- 172.21.15.19:0/4100625025 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282406910 0 0) 0x7fca10001490 con 0x7fca180b93a0 2024-04-25T10:00:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca167fc640 1 -- 172.21.15.19:0/4100625025 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca08001110 con 0x7fca180b93a0 2024-04-25T10:00:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca16ffd640 1 -- 172.21.15.19:0/4100625025 >> v1:172.21.15.112:6789/0 conn(0x7fca180b93a0 legacy=0x7fca180d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.711+0000 7fca16ffd640 1 -- 172.21.15.19:0/4100625025 >> v1:172.21.15.19:6789/0 conn(0x7fca1815dc20 legacy=0x7fca180ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.712+0000 7fca167fc640 1 -- 172.21.15.19:0/4100625025 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1586916804 0 0) 0x7fca00003520 con 0x7fca1815dc20 2024-04-25T10:00:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.712+0000 7fca16ffd640 1 -- 172.21.15.19:0/4100625025 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca18161de0 con 0x7fca180cedf0 2024-04-25T10:00:07.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.712+0000 7fca16ffd640 1 --2- 172.21.15.19:0/4100625025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca180cedf0 0x7fca181616b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0097d0 tx=0x7fca0c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afdbce8050383439 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:07.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.712+0000 7fca167fc640 1 -- 172.21.15.19:0/4100625025 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c037070 con 0x7fca180cedf0 2024-04-25T10:00:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.712+0000 7fca167fc640 1 -- 172.21.15.19:0/4100625025 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c02ac30 con 0x7fca180cedf0 2024-04-25T10:00:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.713+0000 7fca167fc640 1 -- 172.21.15.19:0/4100625025 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08001840 con 0x7fca180cedf0 2024-04-25T10:00:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.713+0000 7fca1e215640 1 -- 172.21.15.19:0/4100625025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca180cedf0 msgr2=0x7fca181616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.713+0000 7fca1e215640 1 --2- 172.21.15.19:0/4100625025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca180cedf0 0x7fca181616b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0097d0 tx=0x7fca0c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.713+0000 7fca1e215640 1 -- 172.21.15.19:0/4100625025 shutdown_connections 2024-04-25T10:00:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.713+0000 7fca1e215640 1 --2- 172.21.15.19:0/4100625025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca180cedf0 0x7fca181616b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.713+0000 7fca1e215640 1 -- 172.21.15.19:0/4100625025 >> 172.21.15.19:0/4100625025 conn(0x7fca180d0490 msgr2=0x7fca180d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.713+0000 7fca1e215640 1 -- 172.21.15.19:0/4100625025 shutdown_connections 2024-04-25T10:00:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.713+0000 7fca1e215640 1 -- 172.21.15.19:0/4100625025 wait complete. 2024-04-25T10:00:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.714+0000 7fca1e215640 1 Processor -- start 2024-04-25T10:00:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.714+0000 7fca1e215640 1 -- start start 2024-04-25T10:00:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.715+0000 7fca1e215640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca180b93a0 0x7fca18149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.715+0000 7fca1e215640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca180cedf0 0x7fca180ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.715+0000 7fca1e215640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca1815dc20 0x7fca180cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.715+0000 7fca1e215640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca180d2c20 con 0x7fca1815dc20 2024-04-25T10:00:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.715+0000 7fca1e215640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca181623a0 con 0x7fca180b93a0 2024-04-25T10:00:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.715+0000 7fca1e215640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca180d2da0 con 0x7fca180cedf0 2024-04-25T10:00:07.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.715+0000 7fca17fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca1815dc20 0x7fca180cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.715+0000 7fca17fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca1815dc20 0x7fca180cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38788/0 (socket says 172.21.15.19:38788) 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.715+0000 7fca17fff640 1 -- 172.21.15.19:0/2467195258 learned_addr learned my addr 172.21.15.19:0/2467195258 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.715+0000 7fca177fe640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca180b93a0 0x7fca18149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.715+0000 7fca16ffd640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca180cedf0 0x7fca180ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.716+0000 7fca17fff640 1 -- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca180cedf0 msgr2=0x7fca180ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.716+0000 7fca17fff640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca180cedf0 0x7fca180ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.716+0000 7fca17fff640 1 -- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca180b93a0 msgr2=0x7fca18149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.716+0000 7fca17fff640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca180b93a0 0x7fca18149300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.716+0000 7fca17fff640 1 -- 172.21.15.19:0/2467195258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca18149840 con 0x7fca1815dc20 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.716+0000 7fca17fff640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca1815dc20 0x7fca180cd300 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fca10001630 tx=0x7fca1000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.716+0000 7fc9fffff640 1 -- 172.21.15.19:0/2467195258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10003b70 con 0x7fca1815dc20 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.716+0000 7fc9fffff640 1 -- 172.21.15.19:0/2467195258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca10003d10 con 0x7fca1815dc20 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.716+0000 7fca1e215640 1 -- 172.21.15.19:0/2467195258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca180cd950 con 0x7fca1815dc20 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.716+0000 7fc9fffff640 1 -- 172.21.15.19:0/2467195258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1000aa50 con 0x7fca1815dc20 2024-04-25T10:00:07.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.716+0000 7fca1e215640 1 -- 172.21.15.19:0/2467195258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca18146d10 con 0x7fca1815dc20 2024-04-25T10:00:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.717+0000 7fca1e215640 1 -- 172.21.15.19:0/2467195258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca18073990 con 0x7fca1815dc20 2024-04-25T10:00:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.718+0000 7fc9fffff640 1 -- 172.21.15.19:0/2467195258 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca1000abf0 con 0x7fca1815dc20 2024-04-25T10:00:07.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.718+0000 7fc9fffff640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca0406b4d0 0x7fca0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:07.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.718+0000 7fc9fffff640 1 -- 172.21.15.19:0/2467195258 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fca1008a2b0 con 0x7fca1815dc20 2024-04-25T10:00:07.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.718+0000 7fca177fe640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca0406b4d0 0x7fca0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:07.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.719+0000 7fca177fe640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca0406b4d0 0x7fca0406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fca08005880 tx=0x7fca08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:07.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.722+0000 7fc9fffff640 1 -- 172.21.15.19:0/2467195258 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca180b6bb0 con 0x7fca1815dc20 2024-04-25T10:00:07.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.971+0000 7fca1e215640 1 -- 172.21.15.19:0/2467195258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fca180d1720 con 0x7fca1815dc20 2024-04-25T10:00:07.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.972+0000 7fc9fffff640 1 -- 172.21.15.19:0/2467195258 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1637 (secure 0 0 0) 0x7fca10059030 con 0x7fca1815dc20 2024-04-25T10:00:07.977 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:07.977 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":445,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076423385,"bytes_used":2527879168,"bytes_avail":770566234112,"bytes_total":773094113280,"write_bytes_sec":84053542,"read_op_per_sec":0,"write_op_per_sec":82},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8025}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T10:00:05.006768+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:07.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 -- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca0406b4d0 msgr2=0x7fca0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca0406b4d0 0x7fca0406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fca08005880 tx=0x7fca08009040 comp rx=0 tx=0).stop 2024-04-25T10:00:07.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 -- 172.21.15.19:0/2467195258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca1815dc20 msgr2=0x7fca180cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:07.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca1815dc20 0x7fca180cd300 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fca10001630 tx=0x7fca1000ccd0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 -- 172.21.15.19:0/2467195258 shutdown_connections 2024-04-25T10:00:07.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca0406b4d0 0x7fca0406d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca1815dc20 0x7fca180cd300 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca180cedf0 0x7fca180ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 --2- 172.21.15.19:0/2467195258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca180b93a0 0x7fca18149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 -- 172.21.15.19:0/2467195258 >> 172.21.15.19:0/2467195258 conn(0x7fca180d0490 msgr2=0x7fca180d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 -- 172.21.15.19:0/2467195258 shutdown_connections 2024-04-25T10:00:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:07.974+0000 7fca1e215640 1 -- 172.21.15.19:0/2467195258 wait complete. 2024-04-25T10:00:07.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:00:08.089 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:08.091 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:08.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.100+0000 7fda39a6f640 1 Processor -- start 2024-04-25T10:00:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.100+0000 7fda39a6f640 1 -- start start 2024-04-25T10:00:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.101+0000 7fda39a6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda340d2e90 0x7fda340d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.101+0000 7fda39a6f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda340d0900 0x7fda340b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.101+0000 7fda39a6f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda340b9940 con 0x7fda3415dd30 2024-04-25T10:00:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.101+0000 7fda39a6f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda340d0d70 con 0x7fda340d0900 2024-04-25T10:00:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.101+0000 7fda39a6f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda340d1df0 con 0x7fda340d2e90 2024-04-25T10:00:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.101+0000 7fda33fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda340d0900 0x7fda340b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.101+0000 7fda33fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda340d0900 0x7fda340b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38796/0 (socket says 172.21.15.19:38796) 2024-04-25T10:00:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.101+0000 7fda33fff640 1 -- 172.21.15.19:0/1357230684 learned_addr learned my addr 172.21.15.19:0/1357230684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.101+0000 7fda32ffd640 1 --2- 172.21.15.19:0/1357230684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda340d2e90 0x7fda340d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.102+0000 7fda33fff640 1 -- 172.21.15.19:0/1357230684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda340d2e90 msgr2=0x7fda340d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.102+0000 7fda33fff640 1 --2- 172.21.15.19:0/1357230684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda340d2e90 0x7fda340d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.102+0000 7fda33fff640 1 -- 172.21.15.19:0/1357230684 >> v1:172.21.15.112:6789/0 conn(0x7fda3415dd30 legacy=0x7fda340b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.102+0000 7fda33fff640 1 -- 172.21.15.19:0/1357230684 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda340d20d0 con 0x7fda340d0900 2024-04-25T10:00:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.102+0000 7fda33fff640 1 --2- 172.21.15.19:0/1357230684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda340d0900 0x7fda340b6a40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009e50 tx=0x7fda2c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f6472bcd9d1e4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.102+0000 7fda327fc640 1 -- 172.21.15.19:0/1357230684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c036070 con 0x7fda340d0900 2024-04-25T10:00:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.103+0000 7fda327fc640 1 -- 172.21.15.19:0/1357230684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c031600 con 0x7fda340d0900 2024-04-25T10:00:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.103+0000 7fda327fc640 1 -- 172.21.15.19:0/1357230684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0319d0 con 0x7fda340d0900 2024-04-25T10:00:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.103+0000 7fda39a6f640 1 -- 172.21.15.19:0/1357230684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda340d0900 msgr2=0x7fda340b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.103+0000 7fda39a6f640 1 --2- 172.21.15.19:0/1357230684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda340d0900 0x7fda340b6a40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009e50 tx=0x7fda2c02a680 comp rx=0 tx=0).stop 2024-04-25T10:00:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.103+0000 7fda39a6f640 1 -- 172.21.15.19:0/1357230684 shutdown_connections 2024-04-25T10:00:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.103+0000 7fda39a6f640 1 --2- 172.21.15.19:0/1357230684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda340d0900 0x7fda340b6a40 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.103+0000 7fda39a6f640 1 --2- 172.21.15.19:0/1357230684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda340d2e90 0x7fda340d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.103+0000 7fda39a6f640 1 -- 172.21.15.19:0/1357230684 >> 172.21.15.19:0/1357230684 conn(0x7fda34061c30 msgr2=0x7fda3406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.103+0000 7fda39a6f640 1 -- 172.21.15.19:0/1357230684 shutdown_connections 2024-04-25T10:00:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.103+0000 7fda39a6f640 1 -- 172.21.15.19:0/1357230684 wait complete. 2024-04-25T10:00:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.104+0000 7fda39a6f640 1 Processor -- start 2024-04-25T10:00:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.105+0000 7fda39a6f640 1 -- start start 2024-04-25T10:00:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.105+0000 7fda39a6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda340d0900 0x7fda340b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.105+0000 7fda39a6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda340d2e90 0x7fda341566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.105+0000 7fda39a6f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3415dd30 0x7fda341601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.105+0000 7fda39a6f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340d2970 con 0x7fda3415dd30 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.105+0000 7fda39a6f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340d2690 con 0x7fda340d2e90 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.105+0000 7fda39a6f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda340d1a10 con 0x7fda340d0900 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.105+0000 7fda33fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3415dd30 0x7fda341601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.105+0000 7fda33fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3415dd30 0x7fda341601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38812/0 (socket says 172.21.15.19:38812) 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.105+0000 7fda33fff640 1 -- 172.21.15.19:0/2826814678 learned_addr learned my addr 172.21.15.19:0/2826814678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda32ffd640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda340d2e90 0x7fda341566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda337fe640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda340d0900 0x7fda340b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda33fff640 1 -- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda340d0900 msgr2=0x7fda340b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda33fff640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda340d0900 0x7fda340b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda33fff640 1 -- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda340d2e90 msgr2=0x7fda341566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda33fff640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda340d2e90 0x7fda341566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda33fff640 1 -- 172.21.15.19:0/2826814678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda341f3c00 con 0x7fda3415dd30 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda33fff640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3415dd30 0x7fda341601c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fda2c034040 tx=0x7fda2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda1bfff640 1 -- 172.21.15.19:0/2826814678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c036050 con 0x7fda3415dd30 2024-04-25T10:00:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda1bfff640 1 -- 172.21.15.19:0/2826814678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c042070 con 0x7fda3415dd30 2024-04-25T10:00:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda39a6f640 1 -- 172.21.15.19:0/2826814678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda341f3ee0 con 0x7fda3415dd30 2024-04-25T10:00:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda1bfff640 1 -- 172.21.15.19:0/2826814678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c009510 con 0x7fda3415dd30 2024-04-25T10:00:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.106+0000 7fda39a6f640 1 -- 172.21.15.19:0/2826814678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda341f44a0 con 0x7fda3415dd30 2024-04-25T10:00:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.107+0000 7fda39a6f640 1 -- 172.21.15.19:0/2826814678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda340b8b40 con 0x7fda3415dd30 2024-04-25T10:00:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.108+0000 7fda1bfff640 1 -- 172.21.15.19:0/2826814678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda2c007c50 con 0x7fda3415dd30 2024-04-25T10:00:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.108+0000 7fda1bfff640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda2006b4d0 0x7fda2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.108+0000 7fda1bfff640 1 -- 172.21.15.19:0/2826814678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda2c0ab260 con 0x7fda3415dd30 2024-04-25T10:00:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.108+0000 7fda337fe640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda2006b4d0 0x7fda2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.109+0000 7fda337fe640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda2006b4d0 0x7fda2006d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fda24007fd0 tx=0x7fda24009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.112+0000 7fda1bfff640 1 -- 172.21.15.19:0/2826814678 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda340b8b40 con 0x7fda3415dd30 2024-04-25T10:00:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.328+0000 7fda39a6f640 1 -- 172.21.15.19:0/2826814678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda34073990 con 0x7fda3415dd30 2024-04-25T10:00:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.329+0000 7fda1bfff640 1 -- 172.21.15.19:0/2826814678 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fda2c079f50 con 0x7fda3415dd30 2024-04-25T10:00:09.333 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:09.333 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 -- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda2006b4d0 msgr2=0x7fda2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda2006b4d0 0x7fda2006d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fda24007fd0 tx=0x7fda24009040 comp rx=0 tx=0).stop 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 -- 172.21.15.19:0/2826814678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3415dd30 msgr2=0x7fda341601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3415dd30 0x7fda341601c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fda2c034040 tx=0x7fda2c004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 -- 172.21.15.19:0/2826814678 shutdown_connections 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda2006b4d0 0x7fda2006d990 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3415dd30 0x7fda341601c0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda340d2e90 0x7fda341566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 --2- 172.21.15.19:0/2826814678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda340d0900 0x7fda340b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 -- 172.21.15.19:0/2826814678 >> 172.21.15.19:0/2826814678 conn(0x7fda34061c30 msgr2=0x7fda340b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 -- 172.21.15.19:0/2826814678 shutdown_connections 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.331+0000 7fda39a6f640 1 -- 172.21.15.19:0/2826814678 wait complete. 2024-04-25T10:00:09.345 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:00:09.346 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:09.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.442+0000 7f9ed5c5d640 1 Processor -- start 2024-04-25T10:00:09.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.442+0000 7f9ed5c5d640 1 -- start start 2024-04-25T10:00:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9ed5c5d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ed015dbe0 0x7f9ed00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9ed5c5d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed00b6220 con 0x7f9ed00b92a0 2024-04-25T10:00:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9ed5c5d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed00b97a0 con 0x7f9ed00b2cb0 2024-04-25T10:00:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9ed5c5d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed015dfc0 con 0x7f9ed015dbe0 2024-04-25T10:00:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9ecffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9ed00b92a0 0x7f9ed00b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55030/0 (socket says 172.21.15.19:55030) 2024-04-25T10:00:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9ecffff640 1 -- 172.21.15.19:0/2917360543 learned_addr learned my addr 172.21.15.19:0/2917360543 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9ecf7fe640 1 --2- 172.21.15.19:0/2917360543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ed015dbe0 0x7f9ed00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9eceffd640 1 -- 172.21.15.19:0/2917360543 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255375549 0 0) 0x7f9ed00b6220 con 0x7f9ed00b92a0 2024-04-25T10:00:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9eceffd640 1 -- 172.21.15.19:0/2917360543 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ebc003520 con 0x7f9ed00b92a0 2024-04-25T10:00:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9eceffd640 1 -- 172.21.15.19:0/2917360543 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174704804 0 0) 0x7f9ed00b97a0 con 0x7f9ed00b2cb0 2024-04-25T10:00:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9eceffd640 1 -- 172.21.15.19:0/2917360543 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed00b6220 con 0x7f9ed00b2cb0 2024-04-25T10:00:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9ecf7fe640 1 -- 172.21.15.19:0/2917360543 >> v1:172.21.15.112:6789/0 conn(0x7f9ed00b2cb0 legacy=0x7f9ed00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9ecf7fe640 1 -- 172.21.15.19:0/2917360543 >> v1:172.21.15.19:6789/0 conn(0x7f9ed00b92a0 legacy=0x7f9ed00b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.443+0000 7f9ecf7fe640 1 -- 172.21.15.19:0/2917360543 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed00d1c50 con 0x7f9ed015dbe0 2024-04-25T10:00:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.444+0000 7f9ecf7fe640 1 --2- 172.21.15.19:0/2917360543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ed015dbe0 0x7f9ed00d0910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00097d0 tx=0x7f9ec002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd61e232f50edee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.444+0000 7f9eceffd640 1 -- 172.21.15.19:0/2917360543 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0037070 con 0x7f9ed015dbe0 2024-04-25T10:00:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.444+0000 7f9eceffd640 1 -- 172.21.15.19:0/2917360543 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0032440 con 0x7f9ed015dbe0 2024-04-25T10:00:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.444+0000 7f9eceffd640 1 -- 172.21.15.19:0/2917360543 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00327f0 con 0x7f9ed015dbe0 2024-04-25T10:00:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.445+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/2917360543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ed015dbe0 msgr2=0x7f9ed00d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.445+0000 7f9ed5c5d640 1 --2- 172.21.15.19:0/2917360543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ed015dbe0 0x7f9ed00d0910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00097d0 tx=0x7f9ec002a210 comp rx=0 tx=0).stop 2024-04-25T10:00:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.445+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/2917360543 shutdown_connections 2024-04-25T10:00:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.445+0000 7f9ed5c5d640 1 --2- 172.21.15.19:0/2917360543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ed015dbe0 0x7f9ed00d0910 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.445+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/2917360543 >> 172.21.15.19:0/2917360543 conn(0x7f9ed0061a20 msgr2=0x7f9ed0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.445+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/2917360543 shutdown_connections 2024-04-25T10:00:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.445+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/2917360543 wait complete. 2024-04-25T10:00:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.446+0000 7f9ed5c5d640 1 Processor -- start 2024-04-25T10:00:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.446+0000 7f9ed5c5d640 1 -- start start 2024-04-25T10:00:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ed5c5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ed00b2cb0 0x7f9ed014bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ed5c5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed00b92a0 0x7f9ed0160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ed5c5d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ed015dbe0 0x7f9ed00cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ed5c5d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00d2210 con 0x7f9ed00b92a0 2024-04-25T10:00:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ed5c5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00d1a00 con 0x7f9ed00b2cb0 2024-04-25T10:00:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ed5c5d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00b6ab0 con 0x7f9ed015dbe0 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ec7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed00b92a0 0x7f9ed0160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ec7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed00b92a0 0x7f9ed0160460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38824/0 (socket says 172.21.15.19:38824) 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ec7fff640 1 -- 172.21.15.19:0/3690476967 learned_addr learned my addr 172.21.15.19:0/3690476967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ecf7fe640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ed00b2cb0 0x7f9ed014bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ecffff640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ed015dbe0 0x7f9ed00cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ec7fff640 1 -- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ed015dbe0 msgr2=0x7f9ed00cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ec7fff640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ed015dbe0 0x7f9ed00cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ec7fff640 1 -- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ed00b2cb0 msgr2=0x7f9ed014bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ec7fff640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ed00b2cb0 0x7f9ed014bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ec7fff640 1 -- 172.21.15.19:0/3690476967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed00cb460 con 0x7f9ed00b92a0 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.447+0000 7f9ec7fff640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed00b92a0 0x7f9ed0160460 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c490 tx=0x7f9eb800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.448+0000 7f9eccff9640 1 -- 172.21.15.19:0/3690476967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb80052d0 con 0x7f9ed00b92a0 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.448+0000 7f9eccff9640 1 -- 172.21.15.19:0/3690476967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb8005470 con 0x7f9ed00b92a0 2024-04-25T10:00:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.448+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/3690476967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed00cb740 con 0x7f9ed00b92a0 2024-04-25T10:00:09.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.448+0000 7f9eccff9640 1 -- 172.21.15.19:0/3690476967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8012850 con 0x7f9ed00b92a0 2024-04-25T10:00:09.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.448+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/3690476967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed00cbd00 con 0x7f9ed00b92a0 2024-04-25T10:00:09.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.449+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/3690476967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed00cbf40 con 0x7f9ed00b92a0 2024-04-25T10:00:09.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.449+0000 7f9eccff9640 1 -- 172.21.15.19:0/3690476967 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9eb80129f0 con 0x7f9ed00b92a0 2024-04-25T10:00:09.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.449+0000 7f9eccff9640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9eb006b4d0 0x7f9eb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.449+0000 7f9eccff9640 1 -- 172.21.15.19:0/3690476967 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9eb8089f00 con 0x7f9ed00b92a0 2024-04-25T10:00:09.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.450+0000 7f9ecf7fe640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9eb006b4d0 0x7f9eb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.450+0000 7f9ecf7fe640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9eb006b4d0 0x7f9eb006d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0034000 tx=0x7f9ec002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.453+0000 7f9eccff9640 1 -- 172.21.15.19:0/3690476967 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ed00cbf40 con 0x7f9ed00b92a0 2024-04-25T10:00:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.702+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/3690476967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9ed0073990 con 0x7f9ed00b92a0 2024-04-25T10:00:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.702+0000 7f9eccff9640 1 -- 172.21.15.19:0/3690476967 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1637 (secure 0 0 0) 0x7f9eb80585a0 con 0x7f9ed00b92a0 2024-04-25T10:00:09.707 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:09.707 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":446,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2547023872,"bytes_avail":770547089408,"bytes_total":773094113280,"write_bytes_sec":84066604,"read_op_per_sec":0,"write_op_per_sec":82},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8025}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T10:00:05.006768+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:09.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9eb006b4d0 msgr2=0x7f9eb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9eb006b4d0 0x7f9eb006d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0034000 tx=0x7f9ec002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/3690476967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed00b92a0 msgr2=0x7f9ed0160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed00b92a0 0x7f9ed0160460 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c490 tx=0x7f9eb800c950 comp rx=0 tx=0).stop 2024-04-25T10:00:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/3690476967 shutdown_connections 2024-04-25T10:00:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9eb006b4d0 0x7f9eb006d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ed015dbe0 0x7f9ed00cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed00b92a0 0x7f9ed0160460 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 --2- 172.21.15.19:0/3690476967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ed00b2cb0 0x7f9ed014bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/3690476967 >> 172.21.15.19:0/3690476967 conn(0x7f9ed0061a20 msgr2=0x7f9ed015aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/3690476967 shutdown_connections 2024-04-25T10:00:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:09.704+0000 7f9ed5c5d640 1 -- 172.21.15.19:0/3690476967 wait complete. 2024-04-25T10:00:09.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:00:10.720 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:10.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.830+0000 7f6c42407640 1 Processor -- start 2024-04-25T10:00:10.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.830+0000 7f6c42407640 1 -- start start 2024-04-25T10:00:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.831+0000 7f6c42407640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6c3c0cfec0 0x7f6c3c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.831+0000 7f6c42407640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c3c0ce3c0 0x7f6c3c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.831+0000 7f6c42407640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c071480 con 0x7f6c3c154ee0 2024-04-25T10:00:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.831+0000 7f6c42407640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3c0ce830 con 0x7f6c3c0cfec0 2024-04-25T10:00:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.831+0000 7f6c42407640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c072090 con 0x7f6c3c0ce3c0 2024-04-25T10:00:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.831+0000 7f6c3bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6c3c154ee0 0x7f6c3c0cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55038/0 (socket says 172.21.15.19:55038) 2024-04-25T10:00:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.831+0000 7f6c3bfff640 1 -- 172.21.15.19:0/3872060274 learned_addr learned my addr 172.21.15.19:0/3872060274 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.832+0000 7f6c3b7fe640 1 --2- 172.21.15.19:0/3872060274 >> v2:172.21.15.112:3300/0 conn(0x7f6c3c0cfec0 0x7f6c3c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.832+0000 7f6c409a4640 1 --2- 172.21.15.19:0/3872060274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c3c0ce3c0 0x7f6c3c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.832+0000 7f6c3affd640 1 -- 172.21.15.19:0/3872060274 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207457640 0 0) 0x7f6c3c071480 con 0x7f6c3c154ee0 2024-04-25T10:00:10.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.832+0000 7f6c3affd640 1 -- 172.21.15.19:0/3872060274 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c24003520 con 0x7f6c3c154ee0 2024-04-25T10:00:10.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.832+0000 7f6c3b7fe640 1 -- 172.21.15.19:0/3872060274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c3c0ce3c0 msgr2=0x7f6c3c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:10.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.832+0000 7f6c3b7fe640 1 --2- 172.21.15.19:0/3872060274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c3c0ce3c0 0x7f6c3c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.832+0000 7f6c3affd640 1 -- 172.21.15.19:0/3872060274 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1262547083 0 0) 0x7f6c24003520 con 0x7f6c3c154ee0 2024-04-25T10:00:10.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.832+0000 7f6c3b7fe640 1 -- 172.21.15.19:0/3872060274 >> v1:172.21.15.19:6789/0 conn(0x7f6c3c154ee0 legacy=0x7f6c3c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.832+0000 7f6c3b7fe640 1 -- 172.21.15.19:0/3872060274 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c071760 con 0x7f6c3c0cfec0 2024-04-25T10:00:10.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.832+0000 7f6c409a4640 1 --2- 172.21.15.19:0/3872060274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c3c0ce3c0 0x7f6c3c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:10.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.833+0000 7f6c3b7fe640 1 --2- 172.21.15.19:0/3872060274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c0cfec0 0x7f6c3c0cde80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6c300097a0 tx=0x7f6c3002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d96ada6c86ac185e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.833+0000 7f6c3affd640 1 -- 172.21.15.19:0/3872060274 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30037070 con 0x7f6c3c0cfec0 2024-04-25T10:00:10.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.833+0000 7f6c3affd640 1 -- 172.21.15.19:0/3872060274 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3002aba0 con 0x7f6c3c0cfec0 2024-04-25T10:00:10.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.833+0000 7f6c3affd640 1 -- 172.21.15.19:0/3872060274 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c001670 con 0x7f6c3c0cfec0 2024-04-25T10:00:10.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.833+0000 7f6c42407640 1 -- 172.21.15.19:0/3872060274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c0cfec0 msgr2=0x7f6c3c0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.833+0000 7f6c42407640 1 --2- 172.21.15.19:0/3872060274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c0cfec0 0x7f6c3c0cde80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6c300097a0 tx=0x7f6c3002a110 comp rx=0 tx=0).stop 2024-04-25T10:00:10.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.834+0000 7f6c42407640 1 -- 172.21.15.19:0/3872060274 shutdown_connections 2024-04-25T10:00:10.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.834+0000 7f6c42407640 1 --2- 172.21.15.19:0/3872060274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c3c0ce3c0 0x7f6c3c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.834+0000 7f6c42407640 1 --2- 172.21.15.19:0/3872060274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c0cfec0 0x7f6c3c0cde80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.834+0000 7f6c42407640 1 -- 172.21.15.19:0/3872060274 >> 172.21.15.19:0/3872060274 conn(0x7f6c3c06abe0 msgr2=0x7f6c3c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:10.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.834+0000 7f6c42407640 1 -- 172.21.15.19:0/3872060274 shutdown_connections 2024-04-25T10:00:10.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.834+0000 7f6c42407640 1 -- 172.21.15.19:0/3872060274 wait complete. 2024-04-25T10:00:10.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.835+0000 7f6c42407640 1 Processor -- start 2024-04-25T10:00:10.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.835+0000 7f6c42407640 1 -- start start 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c42407640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3c0ce3c0 0x7f6c3c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c42407640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c3c0cfec0 0x7f6c3c1f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c42407640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c154ee0 0x7f6c3c150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c42407640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c071d20 con 0x7f6c3c0ce3c0 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c42407640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c071230 con 0x7f6c3c154ee0 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c42407640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c1552c0 con 0x7f6c3c0cfec0 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c3bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3c0ce3c0 0x7f6c3c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c3bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3c0ce3c0 0x7f6c3c158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38828/0 (socket says 172.21.15.19:38828) 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c3bfff640 1 -- 172.21.15.19:0/1347637641 learned_addr learned my addr 172.21.15.19:0/1347637641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c3b7fe640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c3c0cfec0 0x7f6c3c1f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c409a4640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c154ee0 0x7f6c3c150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c3bfff640 1 -- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c3c0cfec0 msgr2=0x7f6c3c1f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c3bfff640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c3c0cfec0 0x7f6c3c1f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c3bfff640 1 -- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c154ee0 msgr2=0x7f6c3c150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c3bfff640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c154ee0 0x7f6c3c150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c3bfff640 1 -- 172.21.15.19:0/1347637641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c1588c0 con 0x7f6c3c0ce3c0 2024-04-25T10:00:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.836+0000 7f6c3bfff640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3c0ce3c0 0x7f6c3c158070 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00c700 tx=0x7f6c2c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.837+0000 7f6c38ff9640 1 -- 172.21.15.19:0/1347637641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c017070 con 0x7f6c3c0ce3c0 2024-04-25T10:00:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.837+0000 7f6c38ff9640 1 -- 172.21.15.19:0/1347637641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c005ad0 con 0x7f6c3c0ce3c0 2024-04-25T10:00:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.837+0000 7f6c42407640 1 -- 172.21.15.19:0/1347637641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c156790 con 0x7f6c3c0ce3c0 2024-04-25T10:00:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.837+0000 7f6c38ff9640 1 -- 172.21.15.19:0/1347637641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c007070 con 0x7f6c3c0ce3c0 2024-04-25T10:00:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.837+0000 7f6c42407640 1 -- 172.21.15.19:0/1347637641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c1585b0 con 0x7f6c3c0ce3c0 2024-04-25T10:00:10.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.838+0000 7f6c42407640 1 -- 172.21.15.19:0/1347637641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c073990 con 0x7f6c3c0ce3c0 2024-04-25T10:00:10.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.838+0000 7f6c38ff9640 1 -- 172.21.15.19:0/1347637641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c2c01a070 con 0x7f6c3c0ce3c0 2024-04-25T10:00:10.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.839+0000 7f6c38ff9640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6c2806b4d0 0x7f6c2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.839+0000 7f6c38ff9640 1 -- 172.21.15.19:0/1347637641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c2c089710 con 0x7f6c3c0ce3c0 2024-04-25T10:00:10.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.839+0000 7f6c3b7fe640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6c2806b4d0 0x7f6c2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.840+0000 7f6c3b7fe640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6c2806b4d0 0x7f6c2806d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6c30034000 tx=0x7f6c3002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:10.843+0000 7f6c38ff9640 1 -- 172.21.15.19:0/1347637641 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c2c057db0 con 0x7f6c3c0ce3c0 2024-04-25T10:00:11.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.066+0000 7f6c42407640 1 -- 172.21.15.19:0/1347637641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c3c06d360 con 0x7f6c3c0ce3c0 2024-04-25T10:00:11.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.067+0000 7f6c38ff9640 1 -- 172.21.15.19:0/1347637641 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6c2c05be50 con 0x7f6c3c0ce3c0 2024-04-25T10:00:11.071 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:11.071 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:11.071 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:11.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 -- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6c2806b4d0 msgr2=0x7f6c2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6c2806b4d0 0x7f6c2806d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6c30034000 tx=0x7f6c3002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:11.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 -- 172.21.15.19:0/1347637641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3c0ce3c0 msgr2=0x7f6c3c158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3c0ce3c0 0x7f6c3c158070 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00c700 tx=0x7f6c2c00cbc0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 -- 172.21.15.19:0/1347637641 shutdown_connections 2024-04-25T10:00:11.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6c2806b4d0 0x7f6c2806d990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c154ee0 0x7f6c3c150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c3c0cfec0 0x7f6c3c1f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 --2- 172.21.15.19:0/1347637641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c3c0ce3c0 0x7f6c3c158070 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 -- 172.21.15.19:0/1347637641 >> 172.21.15.19:0/1347637641 conn(0x7f6c3c06abe0 msgr2=0x7f6c3c15d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:11.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 -- 172.21.15.19:0/1347637641 shutdown_connections 2024-04-25T10:00:11.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.069+0000 7f6c42407640 1 -- 172.21.15.19:0/1347637641 wait complete. 2024-04-25T10:00:11.084 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:00:11.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:11.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.180+0000 7f4c77cac640 1 Processor -- start 2024-04-25T10:00:11.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.180+0000 7f4c77cac640 1 -- start start 2024-04-25T10:00:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.181+0000 7f4c77cac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c7015ba00 0x7f4c7009a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.181+0000 7f4c77cac640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4c701602e0 0x7f4c700ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.181+0000 7f4c77cac640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c700cd710 con 0x7f4c7015f5f0 2024-04-25T10:00:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.181+0000 7f4c77cac640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c70099b70 con 0x7f4c701602e0 2024-04-25T10:00:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.181+0000 7f4c77cac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c70099e50 con 0x7f4c7015ba00 2024-04-25T10:00:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.181+0000 7f4c75247640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4c701602e0 0x7f4c700ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.181+0000 7f4c75247640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4c701602e0 0x7f4c700ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38840/0 (socket says 172.21.15.19:38840) 2024-04-25T10:00:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.181+0000 7f4c76249640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4c7015f5f0 0x7f4c700c52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55052/0 (socket says 172.21.15.19:55052) 2024-04-25T10:00:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.181+0000 7f4c75247640 1 -- 172.21.15.19:0/1868557091 learned_addr learned my addr 172.21.15.19:0/1868557091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.182+0000 7f4c75a48640 1 --2- 172.21.15.19:0/1868557091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c7015ba00 0x7f4c7009a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.182+0000 7f4c75247640 1 -- 172.21.15.19:0/1868557091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c7015ba00 msgr2=0x7f4c7009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.182+0000 7f4c75247640 1 --2- 172.21.15.19:0/1868557091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c7015ba00 0x7f4c7009a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.182+0000 7f4c75247640 1 -- 172.21.15.19:0/1868557091 >> v1:172.21.15.19:6789/0 conn(0x7f4c7015f5f0 legacy=0x7f4c700c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.182+0000 7f4c74a46640 1 -- 172.21.15.19:0/1868557091 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393839564 0 0) 0x7f4c60001490 con 0x7f4c7015f5f0 2024-04-25T10:00:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.182+0000 7f4c75247640 1 -- 172.21.15.19:0/1868557091 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7009a130 con 0x7f4c701602e0 2024-04-25T10:00:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.182+0000 7f4c75a48640 1 --2- 172.21.15.19:0/1868557091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c7015ba00 0x7f4c7009a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.182+0000 7f4c75247640 1 --2- 172.21.15.19:0/1868557091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c701602e0 0x7f4c700ccda0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009870 tx=0x7f4c6402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa22ee2e2e07b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.183+0000 7f4c74a46640 1 -- 172.21.15.19:0/1868557091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64037070 con 0x7f4c701602e0 2024-04-25T10:00:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.183+0000 7f4c74a46640 1 -- 172.21.15.19:0/1868557091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c64032440 con 0x7f4c701602e0 2024-04-25T10:00:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.183+0000 7f4c74a46640 1 -- 172.21.15.19:0/1868557091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64032810 con 0x7f4c701602e0 2024-04-25T10:00:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.183+0000 7f4c77cac640 1 -- 172.21.15.19:0/1868557091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c701602e0 msgr2=0x7f4c700ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.183+0000 7f4c77cac640 1 --2- 172.21.15.19:0/1868557091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c701602e0 0x7f4c700ccda0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009870 tx=0x7f4c6402a130 comp rx=0 tx=0).stop 2024-04-25T10:00:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.183+0000 7f4c77cac640 1 -- 172.21.15.19:0/1868557091 shutdown_connections 2024-04-25T10:00:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.183+0000 7f4c77cac640 1 --2- 172.21.15.19:0/1868557091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c701602e0 0x7f4c700ccda0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.183+0000 7f4c77cac640 1 --2- 172.21.15.19:0/1868557091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c7015ba00 0x7f4c7009a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.183+0000 7f4c77cac640 1 -- 172.21.15.19:0/1868557091 >> 172.21.15.19:0/1868557091 conn(0x7f4c700b3270 msgr2=0x7f4c700b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.183+0000 7f4c77cac640 1 -- 172.21.15.19:0/1868557091 shutdown_connections 2024-04-25T10:00:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.183+0000 7f4c77cac640 1 -- 172.21.15.19:0/1868557091 wait complete. 2024-04-25T10:00:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.184+0000 7f4c77cac640 1 Processor -- start 2024-04-25T10:00:11.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.184+0000 7f4c77cac640 1 -- start start 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.185+0000 7f4c77cac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c7015ba00 0x7f4c70158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.185+0000 7f4c77cac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c7015f5f0 0x7f4c700aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.185+0000 7f4c77cac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c701602e0 0x7f4c700af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.185+0000 7f4c77cac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7015bde0 con 0x7f4c701602e0 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.185+0000 7f4c77cac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c700cd2e0 con 0x7f4c7015f5f0 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.185+0000 7f4c77cac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c700b4b60 con 0x7f4c7015ba00 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.185+0000 7f4c76249640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c701602e0 0x7f4c700af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.185+0000 7f4c76249640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c701602e0 0x7f4c700af3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38848/0 (socket says 172.21.15.19:38848) 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.185+0000 7f4c76249640 1 -- 172.21.15.19:0/2687011193 learned_addr learned my addr 172.21.15.19:0/2687011193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.185+0000 7f4c75a48640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c7015ba00 0x7f4c70158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.185+0000 7f4c75247640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c7015f5f0 0x7f4c700aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.186+0000 7f4c76249640 1 -- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c7015ba00 msgr2=0x7f4c70158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.186+0000 7f4c76249640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c7015ba00 0x7f4c70158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.186+0000 7f4c76249640 1 -- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c7015f5f0 msgr2=0x7f4c700aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.186+0000 7f4c76249640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c7015f5f0 0x7f4c700aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.186+0000 7f4c76249640 1 -- 172.21.15.19:0/2687011193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c701f5720 con 0x7f4c701602e0 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.186+0000 7f4c76249640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c701602e0 0x7f4c700af3b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c700 tx=0x7f4c6000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.186+0000 7f4c6e7fc640 1 -- 172.21.15.19:0/2687011193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60018070 con 0x7f4c701602e0 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.186+0000 7f4c6e7fc640 1 -- 172.21.15.19:0/2687011193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6000ce10 con 0x7f4c701602e0 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.186+0000 7f4c6e7fc640 1 -- 172.21.15.19:0/2687011193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6000a860 con 0x7f4c701602e0 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.186+0000 7f4c77cac640 1 -- 172.21.15.19:0/2687011193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c700b4510 con 0x7f4c701602e0 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.186+0000 7f4c77cac640 1 -- 172.21.15.19:0/2687011193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c701594d0 con 0x7f4c701602e0 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.187+0000 7f4c77cac640 1 -- 172.21.15.19:0/2687011193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c700b4ce0 con 0x7f4c701602e0 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.187+0000 7f4c6e7fc640 1 -- 172.21.15.19:0/2687011193 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c6001b070 con 0x7f4c701602e0 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.188+0000 7f4c6e7fc640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4c5c06b4d0 0x7f4c5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.188+0000 7f4c6e7fc640 1 -- 172.21.15.19:0/2687011193 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c60089990 con 0x7f4c701602e0 2024-04-25T10:00:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.188+0000 7f4c75a48640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4c5c06b4d0 0x7f4c5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.188+0000 7f4c75a48640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4c5c06b4d0 0x7f4c5c06d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800a950 tx=0x7f4c68008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.192+0000 7f4c6e7fc640 1 -- 172.21.15.19:0/2687011193 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c700cd710 con 0x7f4c701602e0 2024-04-25T10:00:11.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.439+0000 7f4c77cac640 1 -- 172.21.15.19:0/2687011193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4c70073990 con 0x7f4c701602e0 2024-04-25T10:00:11.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.439+0000 7f4c6e7fc640 1 -- 172.21.15.19:0/2687011193 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1637 (secure 0 0 0) 0x7f4c60058680 con 0x7f4c701602e0 2024-04-25T10:00:11.444 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:11.444 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":448,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2547023872,"bytes_avail":770547089408,"bytes_total":773094113280,"write_bytes_sec":84066604,"read_op_per_sec":0,"write_op_per_sec":82},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8025}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T10:00:05.006768+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:11.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 -- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4c5c06b4d0 msgr2=0x7f4c5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4c5c06b4d0 0x7f4c5c06d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800a950 tx=0x7f4c68008040 comp rx=0 tx=0).stop 2024-04-25T10:00:11.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 -- 172.21.15.19:0/2687011193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c701602e0 msgr2=0x7f4c700af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c701602e0 0x7f4c700af3b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c700 tx=0x7f4c6000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 -- 172.21.15.19:0/2687011193 shutdown_connections 2024-04-25T10:00:11.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4c5c06b4d0 0x7f4c5c06d990 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c701602e0 0x7f4c700af3b0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c7015f5f0 0x7f4c700aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 --2- 172.21.15.19:0/2687011193 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c7015ba00 0x7f4c70158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 -- 172.21.15.19:0/2687011193 >> 172.21.15.19:0/2687011193 conn(0x7f4c700b3270 msgr2=0x7f4c700cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:11.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 -- 172.21.15.19:0/2687011193 shutdown_connections 2024-04-25T10:00:11.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:11.441+0000 7f4c77cac640 1 -- 172.21.15.19:0/2687011193 wait complete. 2024-04-25T10:00:11.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:00:12.457 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:12.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.568+0000 7efc1e2fc640 1 Processor -- start 2024-04-25T10:00:12.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.568+0000 7efc1e2fc640 1 -- start start 2024-04-25T10:00:12.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.568+0000 7efc1e2fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efc180d0b10 0x7efc180ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.568+0000 7efc1e2fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc180b5f00 0x7efc180d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.569+0000 7efc1e2fc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc180cec00 con 0x7efc1815db60 2024-04-25T10:00:12.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.569+0000 7efc1e2fc640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc180ce930 con 0x7efc180d0b10 2024-04-25T10:00:12.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.569+0000 7efc1e2fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc180b6450 con 0x7efc180b5f00 2024-04-25T10:00:12.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.569+0000 7efc177fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efc180d0b10 0x7efc180ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.569+0000 7efc177fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efc180d0b10 0x7efc180ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38852/0 (socket says 172.21.15.19:38852) 2024-04-25T10:00:12.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.569+0000 7efc177fe640 1 -- 172.21.15.19:0/3880681861 learned_addr learned my addr 172.21.15.19:0/3880681861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:12.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.569+0000 7efc1c899640 1 --2- 172.21.15.19:0/3880681861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc180b5f00 0x7efc180d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.569+0000 7efc177fe640 1 -- 172.21.15.19:0/3880681861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc180b5f00 msgr2=0x7efc180d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:12.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.569+0000 7efc177fe640 1 --2- 172.21.15.19:0/3880681861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc180b5f00 0x7efc180d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.569+0000 7efc177fe640 1 -- 172.21.15.19:0/3880681861 >> v1:172.21.15.112:6789/0 conn(0x7efc1815db60 legacy=0x7efc180d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.569+0000 7efc177fe640 1 -- 172.21.15.19:0/3880681861 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc180ceee0 con 0x7efc180d0b10 2024-04-25T10:00:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.570+0000 7efc177fe640 1 --2- 172.21.15.19:0/3880681861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc180d0b10 0x7efc180ce3f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0098d0 tx=0x7efc0c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a2fa1ad5198687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.570+0000 7efc16ffd640 1 -- 172.21.15.19:0/3880681861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c037070 con 0x7efc180d0b10 2024-04-25T10:00:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.570+0000 7efc16ffd640 1 -- 172.21.15.19:0/3880681861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c032440 con 0x7efc180d0b10 2024-04-25T10:00:12.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.570+0000 7efc16ffd640 1 -- 172.21.15.19:0/3880681861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c032810 con 0x7efc180d0b10 2024-04-25T10:00:12.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.570+0000 7efc1e2fc640 1 -- 172.21.15.19:0/3880681861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc180d0b10 msgr2=0x7efc180ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.570+0000 7efc1e2fc640 1 --2- 172.21.15.19:0/3880681861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc180d0b10 0x7efc180ce3f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0098d0 tx=0x7efc0c02a230 comp rx=0 tx=0).stop 2024-04-25T10:00:12.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.571+0000 7efc1e2fc640 1 -- 172.21.15.19:0/3880681861 shutdown_connections 2024-04-25T10:00:12.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.571+0000 7efc1e2fc640 1 --2- 172.21.15.19:0/3880681861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc180b5f00 0x7efc180d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.571+0000 7efc1e2fc640 1 --2- 172.21.15.19:0/3880681861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc180d0b10 0x7efc180ce3f0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.571+0000 7efc1e2fc640 1 -- 172.21.15.19:0/3880681861 >> 172.21.15.19:0/3880681861 conn(0x7efc18061600 msgr2=0x7efc18061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:12.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.571+0000 7efc1e2fc640 1 -- 172.21.15.19:0/3880681861 shutdown_connections 2024-04-25T10:00:12.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.571+0000 7efc1e2fc640 1 -- 172.21.15.19:0/3880681861 wait complete. 2024-04-25T10:00:12.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.572+0000 7efc1e2fc640 1 Processor -- start 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.572+0000 7efc1e2fc640 1 -- start start 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1e2fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc180b5f00 0x7efc1815f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1e2fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc180d0b10 0x7efc181f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1e2fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc1815db60 0x7efc180d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1e2fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1815df40 con 0x7efc1815db60 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1e2fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc180d1d50 con 0x7efc180b5f00 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1e2fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1815e0c0 con 0x7efc180d0b10 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1c899640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc1815db60 0x7efc180d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1c899640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc1815db60 0x7efc180d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38858/0 (socket says 172.21.15.19:38858) 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1c899640 1 -- 172.21.15.19:0/1921793567 learned_addr learned my addr 172.21.15.19:0/1921793567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc177fe640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc180d0b10 0x7efc181f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc17fff640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc180b5f00 0x7efc1815f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1c899640 1 -- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc180d0b10 msgr2=0x7efc181f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1c899640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc180d0b10 0x7efc181f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1c899640 1 -- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc180b5f00 msgr2=0x7efc1815f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1c899640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc180b5f00 0x7efc1815f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.573+0000 7efc1c899640 1 -- 172.21.15.19:0/1921793567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc18160640 con 0x7efc1815db60 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.574+0000 7efc1c899640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc1815db60 0x7efc180d26c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7efc10007e60 tx=0x7efc1000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.574+0000 7efc14ff9640 1 -- 172.21.15.19:0/1921793567 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10014420 con 0x7efc1815db60 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.574+0000 7efc14ff9640 1 -- 172.21.15.19:0/1921793567 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc10019070 con 0x7efc1815db60 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.574+0000 7efc1e2fc640 1 -- 172.21.15.19:0/1921793567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1815fac0 con 0x7efc1815db60 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.574+0000 7efc14ff9640 1 -- 172.21.15.19:0/1921793567 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10014820 con 0x7efc1815db60 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.574+0000 7efc1e2fc640 1 -- 172.21.15.19:0/1921793567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc18160080 con 0x7efc1815db60 2024-04-25T10:00:12.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.575+0000 7efc1e2fc640 1 -- 172.21.15.19:0/1921793567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc18073270 con 0x7efc1815db60 2024-04-25T10:00:12.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.575+0000 7efc14ff9640 1 -- 172.21.15.19:0/1921793567 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc100149c0 con 0x7efc1815db60 2024-04-25T10:00:12.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.576+0000 7efc14ff9640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc0406b4d0 0x7efc0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.576+0000 7efc14ff9640 1 -- 172.21.15.19:0/1921793567 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efc1008aa20 con 0x7efc1815db60 2024-04-25T10:00:12.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.576+0000 7efc17fff640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc0406b4d0 0x7efc0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.577+0000 7efc17fff640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc0406b4d0 0x7efc0406d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efc181609c0 tx=0x7efc08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.580+0000 7efc14ff9640 1 -- 172.21.15.19:0/1921793567 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc10059150 con 0x7efc1815db60 2024-04-25T10:00:12.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.798+0000 7efc1e2fc640 1 -- 172.21.15.19:0/1921793567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc18073990 con 0x7efc1815db60 2024-04-25T10:00:12.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.799+0000 7efc14ff9640 1 -- 172.21.15.19:0/1921793567 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7efc1005d1f0 con 0x7efc1815db60 2024-04-25T10:00:12.803 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:12.803 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:12.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 -- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc0406b4d0 msgr2=0x7efc0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc0406b4d0 0x7efc0406d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efc181609c0 tx=0x7efc08009040 comp rx=0 tx=0).stop 2024-04-25T10:00:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 -- 172.21.15.19:0/1921793567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc1815db60 msgr2=0x7efc180d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc1815db60 0x7efc180d26c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7efc10007e60 tx=0x7efc1000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 -- 172.21.15.19:0/1921793567 shutdown_connections 2024-04-25T10:00:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc0406b4d0 0x7efc0406d990 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc1815db60 0x7efc180d26c0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc180d0b10 0x7efc181f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 --2- 172.21.15.19:0/1921793567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc180b5f00 0x7efc1815f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 -- 172.21.15.19:0/1921793567 >> 172.21.15.19:0/1921793567 conn(0x7efc18061600 msgr2=0x7efc180b5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 -- 172.21.15.19:0/1921793567 shutdown_connections 2024-04-25T10:00:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.801+0000 7efc1e2fc640 1 -- 172.21.15.19:0/1921793567 wait complete. 2024-04-25T10:00:12.816 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:00:12.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.912+0000 7fcef17d5640 1 Processor -- start 2024-04-25T10:00:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.912+0000 7fcef17d5640 1 -- start start 2024-04-25T10:00:12.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.912+0000 7fcef17d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fceec0c6df0 0x7fceec0d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.912+0000 7fcef17d5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceec15dfe0 con 0x7fceec15dc00 2024-04-25T10:00:12.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.912+0000 7fcef17d5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceec0d00c0 con 0x7fceec0d1c00 2024-04-25T10:00:12.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.912+0000 7fcef17d5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceec0d2070 con 0x7fceec0c6df0 2024-04-25T10:00:12.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.913+0000 7fceea7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fceec0c6df0 0x7fceec0d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.913+0000 7fceeaffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fceec15dc00 0x7fceec0ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55056/0 (socket says 172.21.15.19:55056) 2024-04-25T10:00:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.913+0000 7fceeaffd640 1 -- 172.21.15.19:0/3991573749 learned_addr learned my addr 172.21.15.19:0/3991573749 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.913+0000 7fceea7fc640 1 -- 172.21.15.19:0/3991573749 >> v1:172.21.15.112:6789/0 conn(0x7fceec0d1c00 legacy=0x7fceec0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.913+0000 7fceea7fc640 1 -- 172.21.15.19:0/3991573749 >> v1:172.21.15.19:6789/0 conn(0x7fceec15dc00 legacy=0x7fceec0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.913+0000 7fcee9ffb640 1 -- 172.21.15.19:0/3991573749 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357263875 0 0) 0x7fcedc001110 con 0x7fceec15dc00 2024-04-25T10:00:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.913+0000 7fceea7fc640 1 -- 172.21.15.19:0/3991573749 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec0d03a0 con 0x7fceec0c6df0 2024-04-25T10:00:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.914+0000 7fceea7fc640 1 --2- 172.21.15.19:0/3991573749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceec0c6df0 0x7fceec0d16c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fcee00097a0 tx=0x7fcee002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0a681f2c4c543a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.914+0000 7fcee9ffb640 1 -- 172.21.15.19:0/3991573749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0037070 con 0x7fceec0c6df0 2024-04-25T10:00:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.914+0000 7fcee9ffb640 1 -- 172.21.15.19:0/3991573749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee0032440 con 0x7fceec0c6df0 2024-04-25T10:00:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.914+0000 7fcee9ffb640 1 -- 172.21.15.19:0/3991573749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0032810 con 0x7fceec0c6df0 2024-04-25T10:00:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.914+0000 7fcef17d5640 1 -- 172.21.15.19:0/3991573749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceec0c6df0 msgr2=0x7fceec0d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.914+0000 7fcef17d5640 1 --2- 172.21.15.19:0/3991573749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceec0c6df0 0x7fceec0d16c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fcee00097a0 tx=0x7fcee002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.914+0000 7fcef17d5640 1 -- 172.21.15.19:0/3991573749 shutdown_connections 2024-04-25T10:00:12.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.914+0000 7fcef17d5640 1 --2- 172.21.15.19:0/3991573749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceec0c6df0 0x7fceec0d16c0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.914+0000 7fcef17d5640 1 -- 172.21.15.19:0/3991573749 >> 172.21.15.19:0/3991573749 conn(0x7fceec06f5d0 msgr2=0x7fceec06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:12.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.914+0000 7fcef17d5640 1 -- 172.21.15.19:0/3991573749 shutdown_connections 2024-04-25T10:00:12.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.915+0000 7fcef17d5640 1 -- 172.21.15.19:0/3991573749 wait complete. 2024-04-25T10:00:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.916+0000 7fcef17d5640 1 Processor -- start 2024-04-25T10:00:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.916+0000 7fcef17d5640 1 -- start start 2024-04-25T10:00:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.916+0000 7fcef17d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec0c6df0 0x7fceec0b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.916+0000 7fcef17d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fceec0d1c00 0x7fceec1e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.916+0000 7fcef17d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceec15dc00 0x7fceec156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.916+0000 7fcef17d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec0cc130 con 0x7fceec15dc00 2024-04-25T10:00:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fcef17d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec0cc720 con 0x7fceec0c6df0 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fcef17d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceec0cfd90 con 0x7fceec0d1c00 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceeb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceec15dc00 0x7fceec156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceeb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceec15dc00 0x7fceec156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38876/0 (socket says 172.21.15.19:38876) 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceeb7fe640 1 -- 172.21.15.19:0/2886866265 learned_addr learned my addr 172.21.15.19:0/2886866265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceeaffd640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec0c6df0 0x7fceec0b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceea7fc640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fceec0d1c00 0x7fceec1e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceeb7fe640 1 -- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fceec0d1c00 msgr2=0x7fceec1e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceeb7fe640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fceec0d1c00 0x7fceec1e8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceeb7fe640 1 -- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec0c6df0 msgr2=0x7fceec0b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceeb7fe640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec0c6df0 0x7fceec0b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceeb7fe640 1 -- 172.21.15.19:0/2886866265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec0b6180 con 0x7fceec15dc00 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceea7fc640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fceec0d1c00 0x7fceec1e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceeaffd640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec0c6df0 0x7fceec0b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fceeb7fe640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceec15dc00 0x7fceec156440 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fcee400c780 tx=0x7fcee400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fced37fe640 1 -- 172.21.15.19:0/2886866265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4004b60 con 0x7fceec15dc00 2024-04-25T10:00:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fced37fe640 1 -- 172.21.15.19:0/2886866265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee4004d00 con 0x7fceec15dc00 2024-04-25T10:00:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fced37fe640 1 -- 172.21.15.19:0/2886866265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee400aa90 con 0x7fceec15dc00 2024-04-25T10:00:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fcef17d5640 1 -- 172.21.15.19:0/2886866265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceec0b6460 con 0x7fceec15dc00 2024-04-25T10:00:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.917+0000 7fcef17d5640 1 -- 172.21.15.19:0/2886866265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceec0b6a20 con 0x7fceec15dc00 2024-04-25T10:00:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.918+0000 7fcef17d5640 1 -- 172.21.15.19:0/2886866265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceec073990 con 0x7fceec15dc00 2024-04-25T10:00:12.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.919+0000 7fced37fe640 1 -- 172.21.15.19:0/2886866265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcee400ac30 con 0x7fceec15dc00 2024-04-25T10:00:12.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.919+0000 7fced37fe640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fced806b4d0 0x7fced806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.919+0000 7fced37fe640 1 -- 172.21.15.19:0/2886866265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcee408a670 con 0x7fceec15dc00 2024-04-25T10:00:12.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.920+0000 7fceeaffd640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fced806b4d0 0x7fced806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.920+0000 7fceeaffd640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fced806b4d0 0x7fced806d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcedc000f60 tx=0x7fcedc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:12.924+0000 7fced37fe640 1 -- 172.21.15.19:0/2886866265 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcee4020070 con 0x7fceec15dc00 2024-04-25T10:00:13.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.172+0000 7fcef17d5640 1 -- 172.21.15.19:0/2886866265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fceec0c66c0 con 0x7fceec15dc00 2024-04-25T10:00:13.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.173+0000 7fced37fe640 1 -- 172.21.15.19:0/2886866265 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fcee4059360 con 0x7fceec15dc00 2024-04-25T10:00:13.177 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:13.177 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":450,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2547023872,"bytes_avail":770547089408,"bytes_total":773094113280,"write_bytes_sec":38551869,"read_op_per_sec":0,"write_op_per_sec":38},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8025}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T10:00:11.008241+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:13.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.174+0000 7fcef17d5640 1 -- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fced806b4d0 msgr2=0x7fced806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.175+0000 7fcef17d5640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fced806b4d0 0x7fced806d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcedc000f60 tx=0x7fcedc009040 comp rx=0 tx=0).stop 2024-04-25T10:00:13.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.175+0000 7fcef17d5640 1 -- 172.21.15.19:0/2886866265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceec15dc00 msgr2=0x7fceec156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.175+0000 7fcef17d5640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceec15dc00 0x7fceec156440 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fcee400c780 tx=0x7fcee400cc40 comp rx=0 tx=0).stop 2024-04-25T10:00:13.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.175+0000 7fcef17d5640 1 -- 172.21.15.19:0/2886866265 shutdown_connections 2024-04-25T10:00:13.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.175+0000 7fcef17d5640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fced806b4d0 0x7fced806d990 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.175+0000 7fcef17d5640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceec15dc00 0x7fceec156440 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.175+0000 7fcef17d5640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fceec0d1c00 0x7fceec1e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.175+0000 7fcef17d5640 1 --2- 172.21.15.19:0/2886866265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceec0c6df0 0x7fceec0b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.175+0000 7fcef17d5640 1 -- 172.21.15.19:0/2886866265 >> 172.21.15.19:0/2886866265 conn(0x7fceec06f5d0 msgr2=0x7fceec0c7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.175+0000 7fcef17d5640 1 -- 172.21.15.19:0/2886866265 shutdown_connections 2024-04-25T10:00:13.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.175+0000 7fcef17d5640 1 -- 172.21.15.19:0/2886866265 wait complete. 2024-04-25T10:00:13.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T10:00:13.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:13.190 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T10:00:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.288+0000 7f5279fc8640 1 Processor -- start 2024-04-25T10:00:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.288+0000 7f5279fc8640 1 -- start start 2024-04-25T10:00:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.289+0000 7f5279fc8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f527415b7d0 0x7f52740cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.289+0000 7f5279fc8640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527406cf50 con 0x7f52740b5620 2024-04-25T10:00:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.289+0000 7f5279fc8640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52740ce530 con 0x7f527406c9e0 2024-04-25T10:00:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.289+0000 7f5279fc8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52740ce810 con 0x7f527415b7d0 2024-04-25T10:00:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.289+0000 7f5272ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f52740b5620 0x7f527406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55070/0 (socket says 172.21.15.19:55070) 2024-04-25T10:00:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.289+0000 7f5272ffd640 1 -- 172.21.15.19:0/3001170973 learned_addr learned my addr 172.21.15.19:0/3001170973 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.289+0000 7f52737fe640 1 --2- 172.21.15.19:0/3001170973 >> v2:172.21.15.112:3300/0 conn(0x7f527415b7d0 0x7f52740cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.290+0000 7f52727fc640 1 -- 172.21.15.19:0/3001170973 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319547821 0 0) 0x7f527406cf50 con 0x7f52740b5620 2024-04-25T10:00:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.290+0000 7f52727fc640 1 -- 172.21.15.19:0/3001170973 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f525c003520 con 0x7f52740b5620 2024-04-25T10:00:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.290+0000 7f52737fe640 1 -- 172.21.15.19:0/3001170973 >> v1:172.21.15.112:6789/0 conn(0x7f527406c9e0 legacy=0x7f52740d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.290+0000 7f52737fe640 1 -- 172.21.15.19:0/3001170973 >> v1:172.21.15.19:6789/0 conn(0x7f52740b5620 legacy=0x7f527406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.290+0000 7f52737fe640 1 -- 172.21.15.19:0/3001170973 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740ceaf0 con 0x7f527415b7d0 2024-04-25T10:00:13.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.290+0000 7f52737fe640 1 --2- 172.21.15.19:0/3001170973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f527415b7d0 0x7f52740cf010 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f52640097a0 tx=0x7f526402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13fabcee9528fe96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.291+0000 7f52727fc640 1 -- 172.21.15.19:0/3001170973 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5264037070 con 0x7f527415b7d0 2024-04-25T10:00:13.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.291+0000 7f52727fc640 1 -- 172.21.15.19:0/3001170973 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5264032440 con 0x7f527415b7d0 2024-04-25T10:00:13.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.291+0000 7f52727fc640 1 -- 172.21.15.19:0/3001170973 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52640327f0 con 0x7f527415b7d0 2024-04-25T10:00:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.291+0000 7f5279fc8640 1 -- 172.21.15.19:0/3001170973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f527415b7d0 msgr2=0x7f52740cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.291+0000 7f5279fc8640 1 --2- 172.21.15.19:0/3001170973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f527415b7d0 0x7f52740cf010 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f52640097a0 tx=0x7f526402a230 comp rx=0 tx=0).stop 2024-04-25T10:00:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.291+0000 7f5279fc8640 1 -- 172.21.15.19:0/3001170973 shutdown_connections 2024-04-25T10:00:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.291+0000 7f5279fc8640 1 --2- 172.21.15.19:0/3001170973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f527415b7d0 0x7f52740cf010 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.291+0000 7f5279fc8640 1 -- 172.21.15.19:0/3001170973 >> 172.21.15.19:0/3001170973 conn(0x7f52740d0800 msgr2=0x7f52740d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.291+0000 7f5279fc8640 1 -- 172.21.15.19:0/3001170973 shutdown_connections 2024-04-25T10:00:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.291+0000 7f5279fc8640 1 -- 172.21.15.19:0/3001170973 wait complete. 2024-04-25T10:00:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.292+0000 7f5279fc8640 1 Processor -- start 2024-04-25T10:00:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.293+0000 7f5279fc8640 1 -- start start 2024-04-25T10:00:13.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.293+0000 7f5279fc8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f527406c9e0 0x7f52740b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.293+0000 7f5279fc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52740b5620 0x7f52740c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.293+0000 7f5279fc8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f527415b7d0 0x7f5274148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.293+0000 7f5279fc8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740cf6d0 con 0x7f527415b7d0 2024-04-25T10:00:13.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.293+0000 7f5279fc8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740cf550 con 0x7f52740b5620 2024-04-25T10:00:13.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.293+0000 7f5279fc8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527415bbb0 con 0x7f527406c9e0 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.293+0000 7f5273fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f527415b7d0 0x7f5274148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.293+0000 7f5273fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f527415b7d0 0x7f5274148c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38890/0 (socket says 172.21.15.19:38890) 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.293+0000 7f5273fff640 1 -- 172.21.15.19:0/634137633 learned_addr learned my addr 172.21.15.19:0/634137633 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f5272ffd640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52740b5620 0x7f52740c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f52737fe640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f527406c9e0 0x7f52740b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f5273fff640 1 -- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f527406c9e0 msgr2=0x7f52740b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f5273fff640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f527406c9e0 0x7f52740b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f5273fff640 1 -- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52740b5620 msgr2=0x7f52740c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f5273fff640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52740b5620 0x7f52740c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f5273fff640 1 -- 172.21.15.19:0/634137633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740ca3d0 con 0x7f527415b7d0 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f5272ffd640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52740b5620 0x7f52740c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f5273fff640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f527415b7d0 0x7f5274148c80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f526c00c760 tx=0x7f526c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f525bfff640 1 -- 172.21.15.19:0/634137633 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c003b70 con 0x7f527415b7d0 2024-04-25T10:00:13.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f525bfff640 1 -- 172.21.15.19:0/634137633 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c003d30 con 0x7f527415b7d0 2024-04-25T10:00:13.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f5279fc8640 1 -- 172.21.15.19:0/634137633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52740b40b0 con 0x7f527415b7d0 2024-04-25T10:00:13.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f525bfff640 1 -- 172.21.15.19:0/634137633 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c00aa90 con 0x7f527415b7d0 2024-04-25T10:00:13.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.294+0000 7f5279fc8640 1 -- 172.21.15.19:0/634137633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527414efd0 con 0x7f527415b7d0 2024-04-25T10:00:13.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.295+0000 7f525bfff640 1 -- 172.21.15.19:0/634137633 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f526c00ac30 con 0x7f527415b7d0 2024-04-25T10:00:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.296+0000 7f525bfff640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f526006b4d0 0x7f526006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.296+0000 7f525bfff640 1 -- 172.21.15.19:0/634137633 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f526c08a250 con 0x7f527415b7d0 2024-04-25T10:00:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.296+0000 7f52737fe640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f526006b4d0 0x7f526006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.296+0000 7f52737fe640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f526006b4d0 0x7f526006d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5264034000 tx=0x7f526402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.297+0000 7f527cda4740 1 Processor -- start 2024-04-25T10:00:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.297+0000 7f527cda4740 1 -- start start 2024-04-25T10:00:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f527cda4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd4eb10 0x55a9bbd51320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f527cda4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd51860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f527cda4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd21770 0x55a9bbd24f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f527cda4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bbc8ada0 con 0x55a9bbd21770 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f527cda4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bbc33b80 con 0x55a9bbd4eef0 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f527cda4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9bbc86e40 con 0x55a9bbd4eb10 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f5273fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd21770 0x55a9bbd24f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f5273fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd21770 0x55a9bbd24f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38900/0 (socket says 172.21.15.19:38900) 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f5273fff640 1 -- 172.21.15.19:0/469535861 learned_addr learned my addr 172.21.15.19:0/469535861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f52737fe640 1 --2- 172.21.15.19:0/469535861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd4eb10 0x55a9bbd51320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f5272ffd640 1 --2- 172.21.15.19:0/469535861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd51860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f5273fff640 1 -- 172.21.15.19:0/469535861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd4eb10 msgr2=0x55a9bbd51320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f5273fff640 1 --2- 172.21.15.19:0/469535861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd4eb10 0x55a9bbd51320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f5273fff640 1 -- 172.21.15.19:0/469535861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eef0 msgr2=0x55a9bbd51860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f5273fff640 1 --2- 172.21.15.19:0/469535861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd51860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f5273fff640 1 -- 172.21.15.19:0/469535861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9bbc3df10 con 0x55a9bbd21770 2024-04-25T10:00:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f5273fff640 1 --2- 172.21.15.19:0/469535861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd21770 0x55a9bbd24f80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f526c060a10 tx=0x7f526c058f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ce78feff9b38d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f52597fa640 1 -- 172.21.15.19:0/469535861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c05c980 con 0x55a9bbd21770 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f52597fa640 1 -- 172.21.15.19:0/469535861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c060f40 con 0x55a9bbd21770 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.298+0000 7f52597fa640 1 -- 172.21.15.19:0/469535861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c00ce10 con 0x55a9bbd21770 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.299+0000 7f527cda4740 1 -- 172.21.15.19:0/469535861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd21770 msgr2=0x55a9bbd24f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.299+0000 7f527cda4740 1 --2- 172.21.15.19:0/469535861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd21770 0x55a9bbd24f80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f526c060a10 tx=0x7f526c058f30 comp rx=0 tx=0).stop 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.299+0000 7f527cda4740 1 -- 172.21.15.19:0/469535861 shutdown_connections 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.299+0000 7f527cda4740 1 --2- 172.21.15.19:0/469535861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd21770 0x55a9bbd24f80 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.299+0000 7f527cda4740 1 --2- 172.21.15.19:0/469535861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd51860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.299+0000 7f527cda4740 1 --2- 172.21.15.19:0/469535861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd4eb10 0x55a9bbd51320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.299+0000 7f527cda4740 1 -- 172.21.15.19:0/469535861 >> 172.21.15.19:0/469535861 conn(0x55a9bbd239e0 msgr2=0x55a9bbd6c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.299+0000 7f527cda4740 1 -- 172.21.15.19:0/469535861 shutdown_connections 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.299+0000 7f527cda4740 1 -- 172.21.15.19:0/469535861 wait complete. 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.300+0000 7f527cda4740 1 -- start start 2024-04-25T10:00:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.300+0000 7f527cda4740 1 Processor -- start 2024-04-25T10:00:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f527cda4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd47ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f527cda4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd6fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f527cda4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd70180 0x55a9bbd481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f527cda4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bbd26b90 con 0x55a9bbd4eef0 2024-04-25T10:00:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f527cda4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bbc294b0 con 0x55a9bbd4eb10 2024-04-25T10:00:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f527cda4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9bbd26e70 con 0x55a9bbd70180 2024-04-25T10:00:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f527cda4740 10 client.0 _ll_register_callbacks cb 0x55a9bbc69180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f5272ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd6fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f5272ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd6fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38902/0 (socket says 172.21.15.19:38902) 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f5272ffd640 1 -- 172.21.15.19:0/4245519592 learned_addr learned my addr 172.21.15.19:0/4245519592 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f52737fe640 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd47ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f5272ffd640 1 -- 172.21.15.19:0/4245519592 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd70180 msgr2=0x55a9bbd481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f5272ffd640 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd70180 0x55a9bbd481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f5272ffd640 1 -- 172.21.15.19:0/4245519592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eb10 msgr2=0x55a9bbd47ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f5272ffd640 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd47ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f5272ffd640 1 -- 172.21.15.19:0/4245519592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9bba697d0 con 0x55a9bbd4eef0 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f5272ffd640 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd6fc40 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f52741f6a30 tx=0x7f5268011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a8fa558ef5ed63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f52527fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f52527fc640 1 -- 172.21.15.19:0/4245519592 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268005ac0 con 0x55a9bbd4eef0 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f52527fc640 1 -- 172.21.15.19:0/4245519592 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5268005c60 con 0x55a9bbd4eef0 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f527cda4740 1 -- 172.21.15.19:0/4245519592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f526c060f40 con 0x55a9bbd4eef0 2024-04-25T10:00:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f52527fc640 1 -- 172.21.15.19:0/4245519592 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5268022050 con 0x55a9bbd4eef0 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f527cda4740 10 client.8325 fetch_fsmap learned FSMap version 65 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.301+0000 7f527cda4740 1 -- 172.21.15.19:0/4245519592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55a9bb99f3f0 con 0x55a9bbd4eef0 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.302+0000 7f52527fc640 1 -- 172.21.15.19:0/4245519592 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c060f40 con 0x55a9bbd4eef0 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.302+0000 7f52527fc640 1 -- 172.21.15.19:0/4245519592 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5268025020 con 0x55a9bbd4eef0 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.302+0000 7f527cda4740 10 client.8325 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.302+0000 7f527cda4740 10 client.8325 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.302+0000 7f527cda4740 20 client.8325 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.302+0000 7f527cda4740 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55a9bbd6df70 0x55a9bbd6e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.302+0000 7f527cda4740 4 client.8325 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.302+0000 7f527cda4740 1 -- 172.21.15.19:0/4245519592 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9bbc982e0 con 0x55a9bbd6df70 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.302+0000 7f52737fe640 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55a9bbd6df70 0x55a9bbd6e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.302+0000 7f52737fe640 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55a9bbd6df70 0x55a9bbd6e390 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76c10bf01ceb056e server_cookie=790505be8d65e8f2 in_seq=0 out_seq=0 2024-04-25T10:00:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.302+0000 7f52527fc640 10 client.8325 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f52527fc640 1 -- 172.21.15.19:0/4245519592 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9bbc982e0 con 0x55a9bbd6df70 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f52527fc640 10 client.8325 handle_command_reply: tid=0 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 1 client.8325 shutdown 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 1 -- 172.21.15.19:0/4245519592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eef0 msgr2=0x55a9bbd6fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd6fc40 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f52741f6a30 tx=0x7f5268011910 comp rx=0 tx=0).stop 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 1 -- 172.21.15.19:0/4245519592 shutdown_connections 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55a9bbd6df70 0x55a9bbd6e390 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd70180 0x55a9bbd481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd6fc40 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f52527fc640 0 client.8325 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 1 --2- 172.21.15.19:0/4245519592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd47ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 1 -- 172.21.15.19:0/4245519592 >> 172.21.15.19:0/4245519592 conn(0x55a9bbd21770 msgr2=0x55a9bbd4ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 1 -- 172.21.15.19:0/4245519592 shutdown_connections 2024-04-25T10:00:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 1 -- 172.21.15.19:0/4245519592 wait complete. 2024-04-25T10:00:13.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.303+0000 7f527cda4740 20 client.8325 trim_cache size 0 max 16384 2024-04-25T10:00:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.318+0000 7f527cda4740 1 Processor -- start 2024-04-25T10:00:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.318+0000 7f527cda4740 1 -- start start 2024-04-25T10:00:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f527cda4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd20600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f527cda4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd20b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f527cda4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd70180 0x55a9bbd4ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f527cda4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52742002d0 con 0x55a9bbd4eb10 2024-04-25T10:00:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f527cda4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52741ff490 con 0x55a9bbd4eef0 2024-04-25T10:00:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f527cda4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527415dae0 con 0x55a9bbd70180 2024-04-25T10:00:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f52737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd20600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f52737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd20600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38918/0 (socket says 172.21.15.19:38918) 2024-04-25T10:00:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f52737fe640 1 -- 172.21.15.19:0/1904447498 learned_addr learned my addr 172.21.15.19:0/1904447498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f5273fff640 1 --2- 172.21.15.19:0/1904447498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd70180 0x55a9bbd4ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f5272ffd640 1 --2- 172.21.15.19:0/1904447498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd20b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f52737fe640 1 -- 172.21.15.19:0/1904447498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd70180 msgr2=0x55a9bbd4ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f52737fe640 1 --2- 172.21.15.19:0/1904447498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd70180 0x55a9bbd4ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f52737fe640 1 -- 172.21.15.19:0/1904447498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eef0 msgr2=0x55a9bbd20b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f52737fe640 1 --2- 172.21.15.19:0/1904447498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd20b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.319+0000 7f52737fe640 1 -- 172.21.15.19:0/1904447498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9bb95f0f0 con 0x55a9bbd4eb10 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f52737fe640 1 --2- 172.21.15.19:0/1904447498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd20600 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f526402a270 tx=0x7f526402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1a065cd0c9c923 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f5258ff9640 1 -- 172.21.15.19:0/1904447498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5264037050 con 0x55a9bbd4eb10 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f5258ff9640 1 -- 172.21.15.19:0/1904447498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5264058070 con 0x55a9bbd4eb10 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f5258ff9640 1 -- 172.21.15.19:0/1904447498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5264003a90 con 0x55a9bbd4eb10 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f527cda4740 1 -- 172.21.15.19:0/1904447498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 msgr2=0x55a9bbd20600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f527cda4740 1 --2- 172.21.15.19:0/1904447498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd20600 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f526402a270 tx=0x7f526402c000 comp rx=0 tx=0).stop 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f527cda4740 1 -- 172.21.15.19:0/1904447498 shutdown_connections 2024-04-25T10:00:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f527cda4740 1 --2- 172.21.15.19:0/1904447498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd70180 0x55a9bbd4ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f527cda4740 1 --2- 172.21.15.19:0/1904447498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd4eef0 0x55a9bbd20b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f527cda4740 1 --2- 172.21.15.19:0/1904447498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd20600 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f527cda4740 1 -- 172.21.15.19:0/1904447498 >> 172.21.15.19:0/1904447498 conn(0x55a9bbd21770 msgr2=0x55a9bbd277a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f527cda4740 1 -- 172.21.15.19:0/1904447498 shutdown_connections 2024-04-25T10:00:13.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.320+0000 7f527cda4740 1 -- 172.21.15.19:0/1904447498 wait complete. 2024-04-25T10:00:13.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.321+0000 7f527cda4740 1 -- start start 2024-04-25T10:00:13.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.321+0000 7f527cda4740 1 Processor -- start 2024-04-25T10:00:13.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f527cda4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd6ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f527cda4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd4eef0 0x55a9bbd6f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f527cda4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd70180 0x55a9bbdd62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f527cda4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bbd69820 con 0x55a9bbd4eb10 2024-04-25T10:00:13.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f527cda4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52741fee90 con 0x55a9bbd70180 2024-04-25T10:00:13.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f527cda4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9bbdd75f0 con 0x55a9bbd4eef0 2024-04-25T10:00:13.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f527cda4740 10 client.0 _ll_register_callbacks cb 0x7f526c054ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:13.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f52737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd6ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f52737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd6ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38928/0 (socket says 172.21.15.19:38928) 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f52737fe640 1 -- 172.21.15.19:0/4285808419 learned_addr learned my addr 172.21.15.19:0/4285808419 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f5272ffd640 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd4eef0 0x55a9bbd6f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f5273fff640 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd70180 0x55a9bbdd62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f52737fe640 1 -- 172.21.15.19:0/4285808419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd4eef0 msgr2=0x55a9bbd6f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f52737fe640 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd4eef0 0x55a9bbd6f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f52737fe640 1 -- 172.21.15.19:0/4285808419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd70180 msgr2=0x55a9bbdd62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f52737fe640 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd70180 0x55a9bbdd62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f52737fe640 1 -- 172.21.15.19:0/4285808419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9bbd268b0 con 0x55a9bbd4eb10 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f52737fe640 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd6ef70 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f52640091d0 tx=0x7f5264004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7a660469978044 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f52527fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.322+0000 7f52527fc640 1 -- 172.21.15.19:0/4285808419 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5264037070 con 0x55a9bbd4eb10 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f52527fc640 1 -- 172.21.15.19:0/4285808419 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5264003b90 con 0x55a9bbd4eb10 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f527cda4740 1 -- 172.21.15.19:0/4285808419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5264058070 con 0x55a9bbd4eb10 2024-04-25T10:00:13.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f52527fc640 1 -- 172.21.15.19:0/4285808419 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5264069050 con 0x55a9bbd4eb10 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f527cda4740 10 client.8331 fetch_fsmap learned FSMap version 65 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f527cda4740 1 -- 172.21.15.19:0/4285808419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55a9bbdd7390 con 0x55a9bbd4eb10 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f52527fc640 1 -- 172.21.15.19:0/4285808419 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5264058070 con 0x55a9bbd4eb10 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f52527fc640 1 -- 172.21.15.19:0/4285808419 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f526406c020 con 0x55a9bbd4eb10 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f527cda4740 10 client.8331 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f527cda4740 10 client.8331 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f527cda4740 20 client.8331 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f527cda4740 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55a9bbd69220 0x55a9bbdd6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f527cda4740 4 client.8331 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f527cda4740 1 -- 172.21.15.19:0/4285808419 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55a9bbc92630 con 0x55a9bbd69220 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f5272ffd640 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55a9bbd69220 0x55a9bbdd6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f52527fc640 10 client.8331 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.323+0000 7f5272ffd640 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55a9bbd69220 0x55a9bbdd6810 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9914f0f5f798ab3 server_cookie=d87f983fdfcfbb0 in_seq=0 out_seq=0 2024-04-25T10:00:13.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f52527fc640 1 -- 172.21.15.19:0/4285808419 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55a9bbc92630 con 0x55a9bbd69220 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f52527fc640 10 client.8331 handle_command_reply: tid=0 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 1 client.8331 shutdown 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 1 -- 172.21.15.19:0/4285808419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 msgr2=0x55a9bbd6ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd6ef70 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f52640091d0 tx=0x7f5264004730 comp rx=0 tx=0).stop 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 1 -- 172.21.15.19:0/4285808419 shutdown_connections 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x55a9bbd69220 0x55a9bbdd6810 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9bbd70180 0x55a9bbdd62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a9bbd4eef0 0x55a9bbd6f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 1 --2- 172.21.15.19:0/4285808419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9bbd4eb10 0x55a9bbd6ef70 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f52527fc640 0 client.8331 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 1 -- 172.21.15.19:0/4285808419 >> 172.21.15.19:0/4285808419 conn(0x55a9bbd21770 msgr2=0x55a9bbd6c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 1 -- 172.21.15.19:0/4285808419 shutdown_connections 2024-04-25T10:00:13.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 1 -- 172.21.15.19:0/4285808419 wait complete. 2024-04-25T10:00:13.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f527cda4740 20 client.8331 trim_cache size 0 max 16384 2024-04-25T10:00:13.335 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:13.335 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T10:00:13.335 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f5279fc8640 1 -- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f526006b4d0 msgr2=0x7f526006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f5279fc8640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f526006b4d0 0x7f526006d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5264034000 tx=0x7f526402e000 comp rx=0 tx=0).stop 2024-04-25T10:00:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f5279fc8640 1 -- 172.21.15.19:0/634137633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f527415b7d0 msgr2=0x7f5274148c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f5279fc8640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f527415b7d0 0x7f5274148c80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f526c00c760 tx=0x7f526c00cc20 comp rx=0 tx=0).stop 2024-04-25T10:00:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f5279fc8640 1 -- 172.21.15.19:0/634137633 shutdown_connections 2024-04-25T10:00:13.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f5279fc8640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f526006b4d0 0x7f526006d990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f5279fc8640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f527415b7d0 0x7f5274148c80 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f5279fc8640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52740b5620 0x7f52740c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f5279fc8640 1 --2- 172.21.15.19:0/634137633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f527406c9e0 0x7f52740b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.324+0000 7f5279fc8640 1 -- 172.21.15.19:0/634137633 >> 172.21.15.19:0/634137633 conn(0x7f52740d0800 msgr2=0x7f5274160820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.325+0000 7f5279fc8640 1 -- 172.21.15.19:0/634137633 shutdown_connections 2024-04-25T10:00:13.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.325+0000 7f5279fc8640 1 -- 172.21.15.19:0/634137633 wait complete. 2024-04-25T10:00:13.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:13.339 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:00:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.438+0000 7f2d33fb0640 1 Processor -- start 2024-04-25T10:00:13.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.438+0000 7f2d33fb0640 1 -- start start 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d33fb0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2d2c06abe0 0x7f2d2c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d33fb0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2d2c15dd80 0x7f2d2c0b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d33fb0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c06b500 con 0x7f2d2c0ad570 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d33fb0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c06e410 con 0x7f2d2c15dd80 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d33fb0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c0b3930 con 0x7f2d2c06abe0 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d3154b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2d2c15dd80 0x7f2d2c0b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d3154b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2d2c15dd80 0x7f2d2c0b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38938/0 (socket says 172.21.15.19:38938) 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d3154b640 1 -- 172.21.15.19:0/485993366 learned_addr learned my addr 172.21.15.19:0/485993366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/485993366 >> v2:172.21.15.112:3300/0 conn(0x7f2d2c06abe0 0x7f2d2c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d3154b640 1 -- 172.21.15.19:0/485993366 >> v2:172.21.15.112:3300/0 conn(0x7f2d2c06abe0 msgr2=0x7f2d2c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d3154b640 1 --2- 172.21.15.19:0/485993366 >> v2:172.21.15.112:3300/0 conn(0x7f2d2c06abe0 0x7f2d2c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d3154b640 1 -- 172.21.15.19:0/485993366 >> v1:172.21.15.112:6789/0 conn(0x7f2d2c0ad570 legacy=0x7f2d2c06ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.439+0000 7f2d3154b640 1 -- 172.21.15.19:0/485993366 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c0b3c10 con 0x7f2d2c15dd80 2024-04-25T10:00:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.440+0000 7f2d3154b640 1 --2- 172.21.15.19:0/485993366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d2c15dd80 0x7f2d2c0b5290 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c06e590 tx=0x7f2d2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e7e0d0f315bcc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.440+0000 7f2d30d4a640 1 -- 172.21.15.19:0/485993366 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20037070 con 0x7f2d2c15dd80 2024-04-25T10:00:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.440+0000 7f2d30d4a640 1 -- 172.21.15.19:0/485993366 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d20032440 con 0x7f2d2c15dd80 2024-04-25T10:00:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.440+0000 7f2d30d4a640 1 -- 172.21.15.19:0/485993366 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20032810 con 0x7f2d2c15dd80 2024-04-25T10:00:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.441+0000 7f2d33fb0640 1 -- 172.21.15.19:0/485993366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d2c15dd80 msgr2=0x7f2d2c0b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.441+0000 7f2d33fb0640 1 --2- 172.21.15.19:0/485993366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d2c15dd80 0x7f2d2c0b5290 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c06e590 tx=0x7f2d2002a270 comp rx=0 tx=0).stop 2024-04-25T10:00:13.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.441+0000 7f2d33fb0640 1 -- 172.21.15.19:0/485993366 shutdown_connections 2024-04-25T10:00:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.441+0000 7f2d33fb0640 1 --2- 172.21.15.19:0/485993366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d2c15dd80 0x7f2d2c0b5290 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.441+0000 7f2d33fb0640 1 --2- 172.21.15.19:0/485993366 >> v2:172.21.15.112:3300/0 conn(0x7f2d2c06abe0 0x7f2d2c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.441+0000 7f2d33fb0640 1 -- 172.21.15.19:0/485993366 >> 172.21.15.19:0/485993366 conn(0x7f2d2c0b5af0 msgr2=0x7f2d2c0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.441+0000 7f2d33fb0640 1 -- 172.21.15.19:0/485993366 shutdown_connections 2024-04-25T10:00:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.441+0000 7f2d33fb0640 1 -- 172.21.15.19:0/485993366 wait complete. 2024-04-25T10:00:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.442+0000 7f2d33fb0640 1 Processor -- start 2024-04-25T10:00:13.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.442+0000 7f2d33fb0640 1 -- start start 2024-04-25T10:00:13.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d33fb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d2c06abe0 0x7f2d2c149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d33fb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d2c0ad570 0x7f2d2c1e6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d33fb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d2c15dd80 0x7f2d2c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d33fb0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0b57d0 con 0x7f2d2c06abe0 2024-04-25T10:00:13.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d33fb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c06e940 con 0x7f2d2c0ad570 2024-04-25T10:00:13.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d33fb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c070280 con 0x7f2d2c15dd80 2024-04-25T10:00:13.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d31d4c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d2c06abe0 0x7f2d2c149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d31d4c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d2c06abe0 0x7f2d2c149230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38952/0 (socket says 172.21.15.19:38952) 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d31d4c640 1 -- 172.21.15.19:0/837235297 learned_addr learned my addr 172.21.15.19:0/837235297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d3154b640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d2c0ad570 0x7f2d2c1e6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d3254d640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d2c15dd80 0x7f2d2c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d31d4c640 1 -- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d2c15dd80 msgr2=0x7f2d2c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d2c15dd80 0x7f2d2c15a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d31d4c640 1 -- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d2c0ad570 msgr2=0x7f2d2c1e6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d2c0ad570 0x7f2d2c1e6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d31d4c640 1 -- 172.21.15.19:0/837235297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c149770 con 0x7f2d2c06abe0 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.443+0000 7f2d3254d640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d2c15dd80 0x7f2d2c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.444+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d2c06abe0 0x7f2d2c149230 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e9b0 tx=0x7f2d2400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.444+0000 7f2d2a7fc640 1 -- 172.21.15.19:0/837235297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24017070 con 0x7f2d2c06abe0 2024-04-25T10:00:13.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.444+0000 7f2d2a7fc640 1 -- 172.21.15.19:0/837235297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2400c9a0 con 0x7f2d2c06abe0 2024-04-25T10:00:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.444+0000 7f2d2a7fc640 1 -- 172.21.15.19:0/837235297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2400cd90 con 0x7f2d2c06abe0 2024-04-25T10:00:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.444+0000 7f2d33fb0640 1 -- 172.21.15.19:0/837235297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c1e6e10 con 0x7f2d2c06abe0 2024-04-25T10:00:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.444+0000 7f2d33fb0640 1 -- 172.21.15.19:0/837235297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c1e73d0 con 0x7f2d2c06abe0 2024-04-25T10:00:13.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.445+0000 7f2d2a7fc640 1 -- 172.21.15.19:0/837235297 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d2401a070 con 0x7f2d2c06abe0 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.446+0000 7f2d2a7fc640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2d1806b4d0 0x7f2d1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.446+0000 7f2d2a7fc640 1 -- 172.21.15.19:0/837235297 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d2408a290 con 0x7f2d2c06abe0 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.446+0000 7f2d3154b640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2d1806b4d0 0x7f2d1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.447+0000 7f2d36d90740 1 Processor -- start 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.447+0000 7f2d36d90740 1 -- start start 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.447+0000 7f2d3154b640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2d1806b4d0 0x7f2d1806d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2d20034000 tx=0x7f2d2002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.447+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbc77320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.447+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc74ef0 0x5642fbc77860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.447+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc47770 0x5642fbc4af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.447+0000 7f2d36d90740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fbbb0da0 con 0x5642fbc47770 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.447+0000 7f2d36d90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fbb59b80 con 0x5642fbc74b10 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.447+0000 7f2d36d90740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642fbbace40 con 0x5642fbc74ef0 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d3254d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc47770 0x5642fbc4af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d3254d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc47770 0x5642fbc4af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38956/0 (socket says 172.21.15.19:38956) 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d3254d640 1 -- 172.21.15.19:0/2367284521 learned_addr learned my addr 172.21.15.19:0/2367284521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/2367284521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbc77320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d3154b640 1 --2- 172.21.15.19:0/2367284521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc74ef0 0x5642fbc77860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d3254d640 1 -- 172.21.15.19:0/2367284521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc74ef0 msgr2=0x5642fbc77860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d3254d640 1 --2- 172.21.15.19:0/2367284521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc74ef0 0x5642fbc77860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d3254d640 1 -- 172.21.15.19:0/2367284521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 msgr2=0x5642fbc77320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d3254d640 1 --2- 172.21.15.19:0/2367284521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbc77320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d3254d640 1 -- 172.21.15.19:0/2367284521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642fbb63f10 con 0x5642fbc47770 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d3254d640 1 --2- 172.21.15.19:0/2367284521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc47770 0x5642fbc4af80 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c001d20 tx=0x7f2d1c0021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb30da85fbf5801e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d137fe640 1 -- 172.21.15.19:0/2367284521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c019070 con 0x5642fbc47770 2024-04-25T10:00:13.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d137fe640 1 -- 172.21.15.19:0/2367284521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c003b80 con 0x5642fbc47770 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.448+0000 7f2d137fe640 1 -- 172.21.15.19:0/2367284521 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c00e8c0 con 0x5642fbc47770 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.449+0000 7f2d36d90740 1 -- 172.21.15.19:0/2367284521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc47770 msgr2=0x5642fbc4af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.449+0000 7f2d36d90740 1 --2- 172.21.15.19:0/2367284521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc47770 0x5642fbc4af80 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c001d20 tx=0x7f2d1c0021e0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.449+0000 7f2d36d90740 1 -- 172.21.15.19:0/2367284521 shutdown_connections 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.449+0000 7f2d36d90740 1 --2- 172.21.15.19:0/2367284521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc47770 0x5642fbc4af80 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.449+0000 7f2d36d90740 1 --2- 172.21.15.19:0/2367284521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc74ef0 0x5642fbc77860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.449+0000 7f2d36d90740 1 --2- 172.21.15.19:0/2367284521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbc77320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.449+0000 7f2d36d90740 1 -- 172.21.15.19:0/2367284521 >> 172.21.15.19:0/2367284521 conn(0x5642fbc499e0 msgr2=0x5642fbc92320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.449+0000 7f2d36d90740 1 -- 172.21.15.19:0/2367284521 shutdown_connections 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.449+0000 7f2d36d90740 1 -- 172.21.15.19:0/2367284521 wait complete. 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.449+0000 7f2d36d90740 1 -- start start 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.449+0000 7f2d36d90740 1 Processor -- start 2024-04-25T10:00:13.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.450+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74b10 0x5642fbc6bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.450+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc74ef0 0x5642fbc72900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.450+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc949d0 0x5642fbc6c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.450+0000 7f2d36d90740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fbc4cb90 con 0x5642fbc74b10 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.450+0000 7f2d36d90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fbb4f4b0 con 0x5642fbc949d0 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.450+0000 7f2d36d90740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642fbc4ce70 con 0x5642fbc74ef0 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.450+0000 7f2d36d90740 10 client.0 _ll_register_callbacks cb 0x5642fbb8f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.450+0000 7f2d31d4c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74b10 0x5642fbc6bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d31d4c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74b10 0x5642fbc6bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38958/0 (socket says 172.21.15.19:38958) 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d31d4c640 1 -- 172.21.15.19:0/764989120 learned_addr learned my addr 172.21.15.19:0/764989120 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d3154b640 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc74ef0 0x5642fbc72900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d3254d640 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc949d0 0x5642fbc6c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d31d4c640 1 -- 172.21.15.19:0/764989120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc74ef0 msgr2=0x5642fbc72900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc74ef0 0x5642fbc72900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d31d4c640 1 -- 172.21.15.19:0/764989120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc949d0 msgr2=0x5642fbc6c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc949d0 0x5642fbc6c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d31d4c640 1 -- 172.21.15.19:0/764989120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642fb98f7d0 con 0x5642fbc74b10 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d3154b640 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc74ef0 0x5642fbc72900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74b10 0x5642fbc6bec0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c520 tx=0x7f2d2408a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a145c0d74fda75c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d10ff9640 1 -- 172.21.15.19:0/764989120 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24086060 con 0x5642fbc74b10 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d10ff9640 1 -- 172.21.15.19:0/764989120 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d240703c0 con 0x5642fbc74b10 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d36d90740 1 -- 172.21.15.19:0/764989120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d1c003b80 con 0x5642fbc74b10 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d10ff9640 1 -- 172.21.15.19:0/764989120 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d24075050 con 0x5642fbc74b10 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d36d90740 10 client.8343 fetch_fsmap learned FSMap version 65 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d36d90740 1 -- 172.21.15.19:0/764989120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5642fb8c53f0 con 0x5642fbc74b10 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.451+0000 7f2d10ff9640 1 -- 172.21.15.19:0/764989120 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c003b80 con 0x5642fbc74b10 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d10ff9640 1 -- 172.21.15.19:0/764989120 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d24024060 con 0x5642fbc74b10 2024-04-25T10:00:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d36d90740 10 client.8343 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d36d90740 10 client.8343 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d36d90740 20 client.8343 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d36d90740 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5642fbc95d10 0x5642fbc93450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d36d90740 4 client.8343 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d36d90740 1 -- 172.21.15.19:0/764989120 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642fbbbe2e0 con 0x5642fbc95d10 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d3154b640 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5642fbc95d10 0x5642fbc93450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d3154b640 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5642fbc95d10 0x5642fbc93450 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d607db38e376f582 server_cookie=ebd86d18ec671848 in_seq=0 out_seq=0 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d10ff9640 10 client.8343 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d10ff9640 1 -- 172.21.15.19:0/764989120 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642fbbbe2e0 con 0x5642fbc95d10 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.452+0000 7f2d10ff9640 10 client.8343 handle_command_reply: tid=0 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 1 client.8343 shutdown 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 1 -- 172.21.15.19:0/764989120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74b10 msgr2=0x5642fbc6bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74b10 0x5642fbc6bec0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c520 tx=0x7f2d2408a1f0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 1 -- 172.21.15.19:0/764989120 shutdown_connections 2024-04-25T10:00:13.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5642fbc95d10 0x5642fbc93450 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc949d0 0x5642fbc6c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc74ef0 0x5642fbc72900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 1 --2- 172.21.15.19:0/764989120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74b10 0x5642fbc6bec0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 1 -- 172.21.15.19:0/764989120 >> 172.21.15.19:0/764989120 conn(0x5642fbc47770 msgr2=0x5642fbc73190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d10ff9640 0 client.8343 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:13.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 1 -- 172.21.15.19:0/764989120 shutdown_connections 2024-04-25T10:00:13.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 1 -- 172.21.15.19:0/764989120 wait complete. 2024-04-25T10:00:13.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.453+0000 7f2d36d90740 20 client.8343 trim_cache size 0 max 16384 2024-04-25T10:00:13.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.468+0000 7f2d36d90740 1 Processor -- start 2024-04-25T10:00:13.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.468+0000 7f2d36d90740 1 -- start start 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbc6b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc70ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc949d0 0x5642fbc6e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d36d90740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c200420 con 0x5642fbc74ef0 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d36d90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1ff5c0 con 0x5642fbc74b10 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d36d90740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1fe9f0 con 0x5642fbc949d0 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d3154b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc70ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d3154b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc70ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38960/0 (socket says 172.21.15.19:38960) 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d3154b640 1 -- 172.21.15.19:0/3731660046 learned_addr learned my addr 172.21.15.19:0/3731660046 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d3154b640 1 -- 172.21.15.19:0/3731660046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc949d0 msgr2=0x5642fbc6e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d3154b640 1 --2- 172.21.15.19:0/3731660046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc949d0 0x5642fbc6e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/3731660046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbc6b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d3154b640 1 -- 172.21.15.19:0/3731660046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 msgr2=0x5642fbc6b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d3154b640 1 --2- 172.21.15.19:0/3731660046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbc6b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d3154b640 1 -- 172.21.15.19:0/3731660046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642fb8850f0 con 0x5642fbc74ef0 2024-04-25T10:00:13.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.469+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/3731660046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbc6b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:13.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d3154b640 1 --2- 172.21.15.19:0/3731660046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc70ac0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f2d200090b0 tx=0x7f2d20008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1274291ec6d092c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d12ffd640 1 -- 172.21.15.19:0/3731660046 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20037040 con 0x5642fbc74ef0 2024-04-25T10:00:13.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d12ffd640 1 -- 172.21.15.19:0/3731660046 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d20052030 con 0x5642fbc74ef0 2024-04-25T10:00:13.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d12ffd640 1 -- 172.21.15.19:0/3731660046 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2004eb00 con 0x5642fbc74ef0 2024-04-25T10:00:13.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d36d90740 1 -- 172.21.15.19:0/3731660046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 msgr2=0x5642fbc70ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d36d90740 1 --2- 172.21.15.19:0/3731660046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc70ac0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f2d200090b0 tx=0x7f2d20008030 comp rx=0 tx=0).stop 2024-04-25T10:00:13.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d36d90740 1 -- 172.21.15.19:0/3731660046 shutdown_connections 2024-04-25T10:00:13.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d36d90740 1 --2- 172.21.15.19:0/3731660046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc949d0 0x5642fbc6e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d36d90740 1 --2- 172.21.15.19:0/3731660046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc70ac0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d36d90740 1 --2- 172.21.15.19:0/3731660046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbc6b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d36d90740 1 -- 172.21.15.19:0/3731660046 >> 172.21.15.19:0/3731660046 conn(0x5642fbc47770 msgr2=0x5642fbc6bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d36d90740 1 -- 172.21.15.19:0/3731660046 shutdown_connections 2024-04-25T10:00:13.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.470+0000 7f2d36d90740 1 -- 172.21.15.19:0/3731660046 wait complete. 2024-04-25T10:00:13.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.471+0000 7f2d36d90740 1 -- start start 2024-04-25T10:00:13.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.471+0000 7f2d36d90740 1 Processor -- start 2024-04-25T10:00:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbcf9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc69b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d36d90740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc949d0 0x5642fbc6e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d36d90740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1feff0 con 0x5642fbc74ef0 2024-04-25T10:00:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d36d90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fbc91d70 con 0x5642fbc74b10 2024-04-25T10:00:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d36d90740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642fbc6d8e0 con 0x5642fbc949d0 2024-04-25T10:00:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d36d90740 10 client.0 _ll_register_callbacks cb 0x7f2d1c0017c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d3154b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc69b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d3154b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc69b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38964/0 (socket says 172.21.15.19:38964) 2024-04-25T10:00:13.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d3154b640 1 -- 172.21.15.19:0/98961813 learned_addr learned my addr 172.21.15.19:0/98961813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbcf9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d3254d640 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc949d0 0x5642fbc6e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d3154b640 1 -- 172.21.15.19:0/98961813 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc949d0 msgr2=0x5642fbc6e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d3154b640 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc949d0 0x5642fbc6e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d3154b640 1 -- 172.21.15.19:0/98961813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 msgr2=0x5642fbcf9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d3154b640 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbcf9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d3154b640 1 -- 172.21.15.19:0/98961813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642fbc692f0 con 0x5642fbc74ef0 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d3254d640 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc949d0 0x5642fbc6e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d3154b640 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc69b40 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f2d2002a2b0 tx=0x7f2d20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5321d20a56453e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.472+0000 7f2d10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d10ff9640 1 -- 172.21.15.19:0/98961813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20037040 con 0x5642fbc74ef0 2024-04-25T10:00:13.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d10ff9640 1 -- 172.21.15.19:0/98961813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d2004ed00 con 0x5642fbc74ef0 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d36d90740 1 -- 172.21.15.19:0/98961813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d20052030 con 0x5642fbc74ef0 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d10ff9640 1 -- 172.21.15.19:0/98961813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d2004e2c0 con 0x5642fbc74ef0 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d36d90740 10 client.8349 fetch_fsmap learned FSMap version 65 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d36d90740 1 -- 172.21.15.19:0/98961813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5642fbc6d6e0 con 0x5642fbc74ef0 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d10ff9640 1 -- 172.21.15.19:0/98961813 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d20052030 con 0x5642fbc74ef0 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d10ff9640 1 -- 172.21.15.19:0/98961813 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d20046040 con 0x5642fbc74ef0 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d36d90740 10 client.8349 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d36d90740 10 client.8349 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d36d90740 20 client.8349 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d36d90740 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5642fbc70b30 0x5642fbc6eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d36d90740 4 client.8349 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d36d90740 1 -- 172.21.15.19:0/98961813 --> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5642fbbb8630 con 0x5642fbc70b30 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5642fbc70b30 0x5642fbc6eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d10ff9640 10 client.8349 ms_handle_connect on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.473+0000 7f2d31d4c640 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5642fbc70b30 0x5642fbc6eca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12b0e2ddeb6b848 server_cookie=76c73c95c99a0771 in_seq=0 out_seq=0 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d10ff9640 1 -- 172.21.15.19:0/98961813 <== mds.0 v2:172.21.15.19:6833/4224193015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5642fbbb8630 con 0x5642fbc70b30 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d10ff9640 10 client.8349 handle_command_reply: tid=0 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 1 client.8349 shutdown 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 1 -- 172.21.15.19:0/98961813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 msgr2=0x5642fbc69b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc69b40 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f2d2002a2b0 tx=0x7f2d20004730 comp rx=0 tx=0).stop 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 1 -- 172.21.15.19:0/98961813 shutdown_connections 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.19:6833/4224193015,v1:172.21.15.19:6835/4224193015] conn(0x5642fbc70b30 0x5642fbc6eca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642fbc949d0 0x5642fbc6e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642fbc74ef0 0x5642fbc69b40 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 1 --2- 172.21.15.19:0/98961813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642fbc74b10 0x5642fbcf9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 1 -- 172.21.15.19:0/98961813 >> 172.21.15.19:0/98961813 conn(0x5642fbc47770 msgr2=0x5642fbc92320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d10ff9640 0 client.8349 ms_handle_reset on v2:172.21.15.19:6833/4224193015 2024-04-25T10:00:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 1 -- 172.21.15.19:0/98961813 shutdown_connections 2024-04-25T10:00:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 1 -- 172.21.15.19:0/98961813 wait complete. 2024-04-25T10:00:13.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d36d90740 20 client.8349 trim_cache size 0 max 16384 2024-04-25T10:00:13.485 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:13.485 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:00:13.485 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:00:13.485 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:13.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d33fb0640 1 -- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2d1806b4d0 msgr2=0x7f2d1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d33fb0640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2d1806b4d0 0x7f2d1806d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2d20034000 tx=0x7f2d2002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:13.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d33fb0640 1 -- 172.21.15.19:0/837235297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d2c06abe0 msgr2=0x7f2d2c149230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.474+0000 7f2d33fb0640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d2c06abe0 0x7f2d2c149230 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e9b0 tx=0x7f2d2400ee70 comp rx=0 tx=0).stop 2024-04-25T10:00:13.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.475+0000 7f2d33fb0640 1 -- 172.21.15.19:0/837235297 shutdown_connections 2024-04-25T10:00:13.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.475+0000 7f2d33fb0640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2d1806b4d0 0x7f2d1806d990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.475+0000 7f2d33fb0640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d2c15dd80 0x7f2d2c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.475+0000 7f2d33fb0640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d2c0ad570 0x7f2d2c1e6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.475+0000 7f2d33fb0640 1 --2- 172.21.15.19:0/837235297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d2c06abe0 0x7f2d2c149230 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.475+0000 7f2d33fb0640 1 -- 172.21.15.19:0/837235297 >> 172.21.15.19:0/837235297 conn(0x7f2d2c0b5af0 msgr2=0x7f2d2c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.475+0000 7f2d33fb0640 1 -- 172.21.15.19:0/837235297 shutdown_connections 2024-04-25T10:00:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.475+0000 7f2d33fb0640 1 -- 172.21.15.19:0/837235297 wait complete. 2024-04-25T10:00:13.489 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.586+0000 7fabeed34640 1 Processor -- start 2024-04-25T10:00:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.586+0000 7fabeed34640 1 -- start start 2024-04-25T10:00:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.586+0000 7fabeed34640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fabe815ba00 0x7fabe809a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.586+0000 7fabeed34640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fabe81602e0 0x7fabe80ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.586+0000 7fabeed34640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe80cd710 con 0x7fabe815f5f0 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.586+0000 7fabeed34640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe8099b70 con 0x7fabe815ba00 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.586+0000 7fabeed34640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe8099e50 con 0x7fabe81602e0 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.586+0000 7fabecad0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fabe815ba00 0x7fabe809a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.586+0000 7fabed2d1640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fabe815f5f0 0x7fabe80c52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55078/0 (socket says 172.21.15.19:55078) 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.587+0000 7fabed2d1640 1 -- 172.21.15.19:0/2989231311 learned_addr learned my addr 172.21.15.19:0/2989231311 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.587+0000 7fabe7fff640 1 --2- 172.21.15.19:0/2989231311 >> v2:172.21.15.112:3300/0 conn(0x7fabe81602e0 0x7fabe80ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.587+0000 7fabecad0640 1 -- 172.21.15.19:0/2989231311 >> v2:172.21.15.112:3300/0 conn(0x7fabe81602e0 msgr2=0x7fabe80ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.587+0000 7fabecad0640 1 --2- 172.21.15.19:0/2989231311 >> v2:172.21.15.112:3300/0 conn(0x7fabe81602e0 0x7fabe80ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.587+0000 7fabecad0640 1 -- 172.21.15.19:0/2989231311 >> v1:172.21.15.19:6789/0 conn(0x7fabe815f5f0 legacy=0x7fabe80c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.587+0000 7fabecad0640 1 -- 172.21.15.19:0/2989231311 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe809a130 con 0x7fabe815ba00 2024-04-25T10:00:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.587+0000 7fabe77fe640 1 -- 172.21.15.19:0/2989231311 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614609880 0 0) 0x7fabd0001110 con 0x7fabe815f5f0 2024-04-25T10:00:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabecad0640 1 --2- 172.21.15.19:0/2989231311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe815ba00 0x7fabe809a930 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fabe80b7830 tx=0x7fabe002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba54fa63d8b4593a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabe77fe640 1 -- 172.21.15.19:0/2989231311 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0037070 con 0x7fabe815ba00 2024-04-25T10:00:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabe77fe640 1 -- 172.21.15.19:0/2989231311 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe002add0 con 0x7fabe815ba00 2024-04-25T10:00:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabe77fe640 1 -- 172.21.15.19:0/2989231311 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0032660 con 0x7fabe815ba00 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabeed34640 1 -- 172.21.15.19:0/2989231311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe815ba00 msgr2=0x7fabe809a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabeed34640 1 --2- 172.21.15.19:0/2989231311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe815ba00 0x7fabe809a930 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fabe80b7830 tx=0x7fabe002a080 comp rx=0 tx=0).stop 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabeed34640 1 -- 172.21.15.19:0/2989231311 shutdown_connections 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabeed34640 1 --2- 172.21.15.19:0/2989231311 >> v2:172.21.15.112:3300/0 conn(0x7fabe81602e0 0x7fabe80ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabeed34640 1 --2- 172.21.15.19:0/2989231311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe815ba00 0x7fabe809a930 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabeed34640 1 -- 172.21.15.19:0/2989231311 >> 172.21.15.19:0/2989231311 conn(0x7fabe80b3270 msgr2=0x7fabe80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabeed34640 1 -- 172.21.15.19:0/2989231311 shutdown_connections 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.588+0000 7fabeed34640 1 -- 172.21.15.19:0/2989231311 wait complete. 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.589+0000 7fabeed34640 1 Processor -- start 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.590+0000 7fabeed34640 1 -- start start 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.590+0000 7fabeed34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabe815ba00 0x7fabe8152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.590+0000 7fabeed34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe815f5f0 0x7fabe80cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.590+0000 7fabeed34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabe81602e0 0x7fabe80b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.590+0000 7fabeed34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe815bde0 con 0x7fabe815f5f0 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.590+0000 7fabeed34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80cd2e0 con 0x7fabe815ba00 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.590+0000 7fabeed34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80b55f0 con 0x7fabe81602e0 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.590+0000 7fabe7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe815f5f0 0x7fabe80cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabe7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe815f5f0 0x7fabe80cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38982/0 (socket says 172.21.15.19:38982) 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabe7fff640 1 -- 172.21.15.19:0/1710197363 learned_addr learned my addr 172.21.15.19:0/1710197363 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabecad0640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabe815ba00 0x7fabe8152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabed2d1640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabe81602e0 0x7fabe80b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabe7fff640 1 -- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabe81602e0 msgr2=0x7fabe80b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabe7fff640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabe81602e0 0x7fabe80b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabe7fff640 1 -- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabe815ba00 msgr2=0x7fabe8152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabe7fff640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabe815ba00 0x7fabe8152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabe7fff640 1 -- 172.21.15.19:0/1710197363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80b5850 con 0x7fabe815f5f0 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabed2d1640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabe81602e0 0x7fabe80b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabe7fff640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe815f5f0 0x7fabe80cb410 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fabd8007ef0 tx=0x7fabd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabe57fa640 1 -- 172.21.15.19:0/1710197363 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8014620 con 0x7fabe815f5f0 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabe57fa640 1 -- 172.21.15.19:0/1710197363 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd8019070 con 0x7fabe815f5f0 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabeed34640 1 -- 172.21.15.19:0/1710197363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe80b5b30 con 0x7fabe815f5f0 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabe57fa640 1 -- 172.21.15.19:0/1710197363 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8014a00 con 0x7fabe815f5f0 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.591+0000 7fabeed34640 1 -- 172.21.15.19:0/1710197363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe80b63d0 con 0x7fabe815f5f0 2024-04-25T10:00:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.592+0000 7fabeed34640 1 -- 172.21.15.19:0/1710197363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe8073990 con 0x7fabe815f5f0 2024-04-25T10:00:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.593+0000 7fabe57fa640 1 -- 172.21.15.19:0/1710197363 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabd8014ba0 con 0x7fabe815f5f0 2024-04-25T10:00:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.593+0000 7fabe57fa640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabc806b4d0 0x7fabc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.593+0000 7fabe57fa640 1 -- 172.21.15.19:0/1710197363 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fabd808ab70 con 0x7fabe815f5f0 2024-04-25T10:00:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.593+0000 7fabecad0640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabc806b4d0 0x7fabc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.594+0000 7fabecad0640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabc806b4d0 0x7fabc806d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fabe0034000 tx=0x7fabe002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.597+0000 7fabe57fa640 1 -- 172.21.15.19:0/1710197363 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fabd80598f0 con 0x7fabe815f5f0 2024-04-25T10:00:13.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.818+0000 7fabeed34640 1 -- 172.21.15.19:0/1710197363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabe80b40a0 con 0x7fabe815f5f0 2024-04-25T10:00:13.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.818+0000 7fabe57fa640 1 -- 172.21.15.19:0/1710197363 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fabd80591f0 con 0x7fabe815f5f0 2024-04-25T10:00:13.823 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:13.823 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:13.823 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 -- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabc806b4d0 msgr2=0x7fabc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabc806b4d0 0x7fabc806d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fabe0034000 tx=0x7fabe002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 -- 172.21.15.19:0/1710197363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe815f5f0 msgr2=0x7fabe80cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe815f5f0 0x7fabe80cb410 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fabd8007ef0 tx=0x7fabd800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 -- 172.21.15.19:0/1710197363 shutdown_connections 2024-04-25T10:00:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabc806b4d0 0x7fabc806d990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabe81602e0 0x7fabe80b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabe815f5f0 0x7fabe80cb410 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 --2- 172.21.15.19:0/1710197363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabe815ba00 0x7fabe8152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 -- 172.21.15.19:0/1710197363 >> 172.21.15.19:0/1710197363 conn(0x7fabe80b3270 msgr2=0x7fabe80c5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 -- 172.21.15.19:0/1710197363 shutdown_connections 2024-04-25T10:00:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.821+0000 7fabeed34640 1 -- 172.21.15.19:0/1710197363 wait complete. 2024-04-25T10:00:13.836 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:00:13.836 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:13.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.932+0000 7efd393e7640 1 Processor -- start 2024-04-25T10:00:13.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.932+0000 7efd393e7640 1 -- start start 2024-04-25T10:00:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.933+0000 7efd393e7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efd3415dc00 0x7efd340c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.933+0000 7efd393e7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3415dfe0 con 0x7efd340ceb20 2024-04-25T10:00:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.933+0000 7efd393e7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd340d00c0 con 0x7efd340d1dd0 2024-04-25T10:00:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.933+0000 7efd393e7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd340d03a0 con 0x7efd3415dc00 2024-04-25T10:00:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.933+0000 7efd32ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efd3415dc00 0x7efd340c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.933+0000 7efd2bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efd340ceb20 0x7efd340d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55090/0 (socket says 172.21.15.19:55090) 2024-04-25T10:00:13.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.933+0000 7efd32ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efd3415dc00 0x7efd340c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38990/0 (socket says 172.21.15.19:38990) 2024-04-25T10:00:13.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.933+0000 7efd2bfff640 1 -- 172.21.15.19:0/685003496 learned_addr learned my addr 172.21.15.19:0/685003496 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:13.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.933+0000 7efd32ffd640 1 -- 172.21.15.19:0/685003496 >> v1:172.21.15.112:6789/0 conn(0x7efd340d1dd0 legacy=0x7efd340ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.933+0000 7efd32ffd640 1 -- 172.21.15.19:0/685003496 >> v1:172.21.15.19:6789/0 conn(0x7efd340ceb20 legacy=0x7efd340d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.933+0000 7efd32ffd640 1 -- 172.21.15.19:0/685003496 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340cc440 con 0x7efd3415dc00 2024-04-25T10:00:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.934+0000 7efd32ffd640 1 --2- 172.21.15.19:0/685003496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd3415dc00 0x7efd340c6df0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7efd240097a0 tx=0x7efd2402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f432446e4fa39d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.934+0000 7efd327fc640 1 -- 172.21.15.19:0/685003496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24037070 con 0x7efd3415dc00 2024-04-25T10:00:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.934+0000 7efd327fc640 1 -- 172.21.15.19:0/685003496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd24032440 con 0x7efd3415dc00 2024-04-25T10:00:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.934+0000 7efd327fc640 1 -- 172.21.15.19:0/685003496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24032810 con 0x7efd3415dc00 2024-04-25T10:00:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.934+0000 7efd393e7640 1 -- 172.21.15.19:0/685003496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd3415dc00 msgr2=0x7efd340c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.934+0000 7efd393e7640 1 --2- 172.21.15.19:0/685003496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd3415dc00 0x7efd340c6df0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7efd240097a0 tx=0x7efd2402a230 comp rx=0 tx=0).stop 2024-04-25T10:00:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.934+0000 7efd393e7640 1 -- 172.21.15.19:0/685003496 shutdown_connections 2024-04-25T10:00:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.934+0000 7efd393e7640 1 --2- 172.21.15.19:0/685003496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd3415dc00 0x7efd340c6df0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.934+0000 7efd393e7640 1 -- 172.21.15.19:0/685003496 >> 172.21.15.19:0/685003496 conn(0x7efd3406f5d0 msgr2=0x7efd3406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.935+0000 7efd393e7640 1 -- 172.21.15.19:0/685003496 shutdown_connections 2024-04-25T10:00:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.935+0000 7efd393e7640 1 -- 172.21.15.19:0/685003496 wait complete. 2024-04-25T10:00:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.936+0000 7efd393e7640 1 Processor -- start 2024-04-25T10:00:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.936+0000 7efd393e7640 1 -- start start 2024-04-25T10:00:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.936+0000 7efd393e7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd340ceb20 0x7efd3415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.936+0000 7efd393e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd340d1dd0 0x7efd3414a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.936+0000 7efd393e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd3415dc00 0x7efd341476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.936+0000 7efd393e7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd340cfd90 con 0x7efd3415dc00 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.936+0000 7efd393e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd340cc130 con 0x7efd340d1dd0 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.936+0000 7efd393e7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd340cdd70 con 0x7efd340ceb20 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd3415dc00 0x7efd341476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd337fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd3415dc00 0x7efd341476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38994/0 (socket says 172.21.15.19:38994) 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd337fe640 1 -- 172.21.15.19:0/2696431641 learned_addr learned my addr 172.21.15.19:0/2696431641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd2bfff640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd340d1dd0 0x7efd3414a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd32ffd640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd340ceb20 0x7efd3415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd337fe640 1 -- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd340ceb20 msgr2=0x7efd3415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd337fe640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd340ceb20 0x7efd3415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd337fe640 1 -- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd340d1dd0 msgr2=0x7efd3414a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd337fe640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd340d1dd0 0x7efd3414a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd337fe640 1 -- 172.21.15.19:0/2696431641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340b4730 con 0x7efd3415dc00 2024-04-25T10:00:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd337fe640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd3415dc00 0x7efd341476b0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00c730 tx=0x7efd2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd2b7fe640 1 -- 172.21.15.19:0/2696431641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c003b70 con 0x7efd3415dc00 2024-04-25T10:00:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd2b7fe640 1 -- 172.21.15.19:0/2696431641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c003d10 con 0x7efd3415dc00 2024-04-25T10:00:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd2b7fe640 1 -- 172.21.15.19:0/2696431641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c00aa90 con 0x7efd3415dc00 2024-04-25T10:00:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd393e7640 1 -- 172.21.15.19:0/2696431641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd340b4a10 con 0x7efd3415dc00 2024-04-25T10:00:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.937+0000 7efd393e7640 1 -- 172.21.15.19:0/2696431641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd341596d0 con 0x7efd3415dc00 2024-04-25T10:00:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.938+0000 7efd393e7640 1 -- 172.21.15.19:0/2696431641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd34073990 con 0x7efd3415dc00 2024-04-25T10:00:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.939+0000 7efd2b7fe640 1 -- 172.21.15.19:0/2696431641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd2c00ac30 con 0x7efd3415dc00 2024-04-25T10:00:13.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.939+0000 7efd2b7fe640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.939+0000 7efd2b7fe640 1 -- 172.21.15.19:0/2696431641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd2c08a1f0 con 0x7efd3415dc00 2024-04-25T10:00:13.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.939+0000 7efd32ffd640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.940+0000 7efd32ffd640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 0x7efd1406d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efd24034000 tx=0x7efd2402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:13.943+0000 7efd2b7fe640 1 -- 172.21.15.19:0/2696431641 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd2c058f70 con 0x7efd3415dc00 2024-04-25T10:00:14.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.195+0000 7efd393e7640 1 -- 172.21.15.19:0/2696431641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efd340c66c0 con 0x7efd3415dc00 2024-04-25T10:00:14.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.195+0000 7efd2b7fe640 1 -- 172.21.15.19:0/2696431641 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7efd2c058870 con 0x7efd3415dc00 2024-04-25T10:00:14.199 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:14.199 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":451,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2547023872,"bytes_avail":770547089408,"bytes_total":773094113280,"write_bytes_sec":38552050,"read_op_per_sec":0,"write_op_per_sec":38},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8025}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T10:00:11.008241+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:14.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 -- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 msgr2=0x7efd1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 0x7efd1406d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efd24034000 tx=0x7efd2402e000 comp rx=0 tx=0).stop 2024-04-25T10:00:14.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 -- 172.21.15.19:0/2696431641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd3415dc00 msgr2=0x7efd341476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd3415dc00 0x7efd341476b0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00c730 tx=0x7efd2c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 -- 172.21.15.19:0/2696431641 shutdown_connections 2024-04-25T10:00:14.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd3415dc00 0x7efd341476b0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd340d1dd0 0x7efd3414a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 --2- 172.21.15.19:0/2696431641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd340ceb20 0x7efd3415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 -- 172.21.15.19:0/2696431641 >> 172.21.15.19:0/2696431641 conn(0x7efd3406f5d0 msgr2=0x7efd340ca900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:14.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 -- 172.21.15.19:0/2696431641 shutdown_connections 2024-04-25T10:00:14.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:14.197+0000 7efd393e7640 1 -- 172.21.15.19:0/2696431641 wait complete. 2024-04-25T10:00:14.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T10:00:15.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.322+0000 7f7bfcb70640 1 Processor -- start 2024-04-25T10:00:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.322+0000 7f7bfcb70640 1 -- start start 2024-04-25T10:00:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.323+0000 7f7bfcb70640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7bf8144750 0x7f7bf806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.323+0000 7f7bfcb70640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bf80cf030 0x7f7bf806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.323+0000 7f7bfcb70640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf8144b30 con 0x7f7bf806eb10 2024-04-25T10:00:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.323+0000 7f7bfcb70640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf80712d0 con 0x7f7bf8144750 2024-04-25T10:00:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.323+0000 7f7bfcb70640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf80715b0 con 0x7f7bf80cf030 2024-04-25T10:00:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.323+0000 7f7bf659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7bf8144750 0x7f7bf806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.323+0000 7f7bf659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7bf8144750 0x7f7bf806e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38998/0 (socket says 172.21.15.19:38998) 2024-04-25T10:00:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.323+0000 7f7bf659c640 1 -- 172.21.15.19:0/4144086716 learned_addr learned my addr 172.21.15.19:0/4144086716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.323+0000 7f7bf5d9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7bf806eb10 0x7f7bf80ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55096/0 (socket says 172.21.15.19:55096) 2024-04-25T10:00:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.324+0000 7f7bf6d9d640 1 --2- 172.21.15.19:0/4144086716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bf80cf030 0x7f7bf806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.324+0000 7f7bf659c640 1 -- 172.21.15.19:0/4144086716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bf80cf030 msgr2=0x7f7bf806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.324+0000 7f7bf659c640 1 --2- 172.21.15.19:0/4144086716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bf80cf030 0x7f7bf806d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.324+0000 7f7bf659c640 1 -- 172.21.15.19:0/4144086716 >> v1:172.21.15.19:6789/0 conn(0x7f7bf806eb10 legacy=0x7f7bf80ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.324+0000 7f7bf659c640 1 -- 172.21.15.19:0/4144086716 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf8071890 con 0x7f7bf8144750 2024-04-25T10:00:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.324+0000 7f7bf659c640 1 --2- 172.21.15.19:0/4144086716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bf8144750 0x7f7bf806e5d0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009870 tx=0x7f7bf002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcf3b33a9362d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.325+0000 7f7bf559a640 1 -- 172.21.15.19:0/4144086716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0037070 con 0x7f7bf8144750 2024-04-25T10:00:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.325+0000 7f7bf559a640 1 -- 172.21.15.19:0/4144086716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf0032440 con 0x7f7bf8144750 2024-04-25T10:00:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.325+0000 7f7bf559a640 1 -- 172.21.15.19:0/4144086716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0032810 con 0x7f7bf8144750 2024-04-25T10:00:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.325+0000 7f7bfcb70640 1 -- 172.21.15.19:0/4144086716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bf8144750 msgr2=0x7f7bf806e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.325+0000 7f7bfcb70640 1 --2- 172.21.15.19:0/4144086716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bf8144750 0x7f7bf806e5d0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009870 tx=0x7f7bf002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.325+0000 7f7bfcb70640 1 -- 172.21.15.19:0/4144086716 shutdown_connections 2024-04-25T10:00:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.325+0000 7f7bfcb70640 1 --2- 172.21.15.19:0/4144086716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bf80cf030 0x7f7bf806d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.325+0000 7f7bfcb70640 1 --2- 172.21.15.19:0/4144086716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bf8144750 0x7f7bf806e5d0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.325+0000 7f7bfcb70640 1 -- 172.21.15.19:0/4144086716 >> 172.21.15.19:0/4144086716 conn(0x7f7bf806abe0 msgr2=0x7f7bf806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.325+0000 7f7bfcb70640 1 -- 172.21.15.19:0/4144086716 shutdown_connections 2024-04-25T10:00:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.325+0000 7f7bfcb70640 1 -- 172.21.15.19:0/4144086716 wait complete. 2024-04-25T10:00:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.327+0000 7f7bfcb70640 1 Processor -- start 2024-04-25T10:00:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.327+0000 7f7bfcb70640 1 -- start start 2024-04-25T10:00:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bfcb70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bf806eb10 0x7f7bf81f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bfcb70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bf80cf030 0x7f7bf813b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bfcb70640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bf8144750 0x7f7bf815b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bfcb70640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81455d0 con 0x7f7bf806eb10 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bfcb70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf8145450 con 0x7f7bf80cf030 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bfcb70640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf815c000 con 0x7f7bf8144750 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bf659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bf806eb10 0x7f7bf81f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bf659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bf806eb10 0x7f7bf81f16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39002/0 (socket says 172.21.15.19:39002) 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bf659c640 1 -- 172.21.15.19:0/2534188106 learned_addr learned my addr 172.21.15.19:0/2534188106 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bf5d9b640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bf80cf030 0x7f7bf813b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bf6d9d640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bf8144750 0x7f7bf815b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bf659c640 1 -- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bf8144750 msgr2=0x7f7bf815b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bf659c640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bf8144750 0x7f7bf815b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bf659c640 1 -- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bf80cf030 msgr2=0x7f7bf813b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bf659c640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bf80cf030 0x7f7bf813b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bf659c640 1 -- 172.21.15.19:0/2534188106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf813b920 con 0x7f7bf806eb10 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bf659c640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bf806eb10 0x7f7bf81f16e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f7bf002a200 tx=0x7f7bf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7beeffd640 1 -- 172.21.15.19:0/2534188106 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0041020 con 0x7f7bf806eb10 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7beeffd640 1 -- 172.21.15.19:0/2534188106 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf00306c0 con 0x7f7bf806eb10 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7bfcb70640 1 -- 172.21.15.19:0/2534188106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf8140960 con 0x7f7bf806eb10 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.328+0000 7f7beeffd640 1 -- 172.21.15.19:0/2534188106 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0037040 con 0x7f7bf806eb10 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.329+0000 7f7bfcb70640 1 -- 172.21.15.19:0/2534188106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf8143090 con 0x7f7bf806eb10 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.330+0000 7f7bfcb70640 1 -- 172.21.15.19:0/2534188106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf8073990 con 0x7f7bf806eb10 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.330+0000 7f7beeffd640 1 -- 172.21.15.19:0/2534188106 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bf0044020 con 0x7f7bf806eb10 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.330+0000 7f7beeffd640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7bd806b4d0 0x7f7bd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.330+0000 7f7beeffd640 1 -- 172.21.15.19:0/2534188106 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7bf00aab40 con 0x7f7bf806eb10 2024-04-25T10:00:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.331+0000 7f7bf5d9b640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7bd806b4d0 0x7f7bd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.331+0000 7f7bf5d9b640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7bd806b4d0 0x7f7bd806d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7bf813bf20 tx=0x7f7be0007ce0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:15.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.335+0000 7f7beeffd640 1 -- 172.21.15.19:0/2534188106 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bf004b030 con 0x7f7bf806eb10 2024-04-25T10:00:15.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.551+0000 7f7bfcb70640 1 -- 172.21.15.19:0/2534188106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bf80d0510 con 0x7f7bf806eb10 2024-04-25T10:00:15.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.552+0000 7f7beeffd640 1 -- 172.21.15.19:0/2534188106 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f7bf0079830 con 0x7f7bf806eb10 2024-04-25T10:00:15.556 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:15.556 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:15.556 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:15.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.553+0000 7f7bfcb70640 1 -- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7bd806b4d0 msgr2=0x7f7bd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.553+0000 7f7bfcb70640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7bd806b4d0 0x7f7bd806d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7bf813bf20 tx=0x7f7be0007ce0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.553+0000 7f7bfcb70640 1 -- 172.21.15.19:0/2534188106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bf806eb10 msgr2=0x7f7bf81f16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.553+0000 7f7bfcb70640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bf806eb10 0x7f7bf81f16e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f7bf002a200 tx=0x7f7bf00046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.554+0000 7f7bfcb70640 1 -- 172.21.15.19:0/2534188106 shutdown_connections 2024-04-25T10:00:15.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.554+0000 7f7bfcb70640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7bd806b4d0 0x7f7bd806d990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.554+0000 7f7bfcb70640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bf8144750 0x7f7bf815b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.554+0000 7f7bfcb70640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bf80cf030 0x7f7bf813b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.554+0000 7f7bfcb70640 1 --2- 172.21.15.19:0/2534188106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bf806eb10 0x7f7bf81f16e0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.554+0000 7f7bfcb70640 1 -- 172.21.15.19:0/2534188106 >> 172.21.15.19:0/2534188106 conn(0x7f7bf806abe0 msgr2=0x7f7bf80b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:15.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.554+0000 7f7bfcb70640 1 -- 172.21.15.19:0/2534188106 shutdown_connections 2024-04-25T10:00:15.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.554+0000 7f7bfcb70640 1 -- 172.21.15.19:0/2534188106 wait complete. 2024-04-25T10:00:15.568 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:00:15.569 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.666+0000 7f6be2553640 1 Processor -- start 2024-04-25T10:00:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.666+0000 7f6be2553640 1 -- start start 2024-04-25T10:00:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6be2553640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6bdc154ed0 0x7f6bdc0aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6be2553640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bdc06c1d0 0x7f6bdc06afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6be2553640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bdc15ec60 con 0x7f6bdc071ac0 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6be2553640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bdc15ef40 con 0x7f6bdc154ed0 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6be2553640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc06db90 con 0x7f6bdc06c1d0 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6bdbfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6bdc154ed0 0x7f6bdc0aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6bdb7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6bdc071ac0 0x7f6bdc0ad410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55112/0 (socket says 172.21.15.19:55112) 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6bdb7fe640 1 -- 172.21.15.19:0/2833217845 learned_addr learned my addr 172.21.15.19:0/2833217845 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6be0af0640 1 --2- 172.21.15.19:0/2833217845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bdc06c1d0 0x7f6bdc06afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6bdaffd640 1 -- 172.21.15.19:0/2833217845 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873112809 0 0) 0x7f6bdc15ec60 con 0x7f6bdc071ac0 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6bdaffd640 1 -- 172.21.15.19:0/2833217845 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bc4003520 con 0x7f6bdc071ac0 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6bdbfff640 1 -- 172.21.15.19:0/2833217845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bdc06c1d0 msgr2=0x7f6bdc06afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6bdbfff640 1 --2- 172.21.15.19:0/2833217845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bdc06c1d0 0x7f6bdc06afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.667+0000 7f6bdbfff640 1 -- 172.21.15.19:0/2833217845 >> v1:172.21.15.19:6789/0 conn(0x7f6bdc071ac0 legacy=0x7f6bdc0ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.668+0000 7f6bdbfff640 1 -- 172.21.15.19:0/2833217845 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc06de70 con 0x7f6bdc154ed0 2024-04-25T10:00:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.668+0000 7f6bdbfff640 1 --2- 172.21.15.19:0/2833217845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc154ed0 0x7f6bdc0aced0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0097a0 tx=0x7f6bcc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa78cc5c9781244a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:15.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.668+0000 7f6bdaffd640 1 -- 172.21.15.19:0/2833217845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc037070 con 0x7f6bdc154ed0 2024-04-25T10:00:15.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.668+0000 7f6bdaffd640 1 -- 172.21.15.19:0/2833217845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc032440 con 0x7f6bdc154ed0 2024-04-25T10:00:15.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.668+0000 7f6bdaffd640 1 -- 172.21.15.19:0/2833217845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0327f0 con 0x7f6bdc154ed0 2024-04-25T10:00:15.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.669+0000 7f6be2553640 1 -- 172.21.15.19:0/2833217845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc154ed0 msgr2=0x7f6bdc0aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.669+0000 7f6be2553640 1 --2- 172.21.15.19:0/2833217845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc154ed0 0x7f6bdc0aced0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0097a0 tx=0x7f6bcc02a180 comp rx=0 tx=0).stop 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.669+0000 7f6be2553640 1 -- 172.21.15.19:0/2833217845 shutdown_connections 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.669+0000 7f6be2553640 1 --2- 172.21.15.19:0/2833217845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bdc06c1d0 0x7f6bdc06afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.669+0000 7f6be2553640 1 --2- 172.21.15.19:0/2833217845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc154ed0 0x7f6bdc0aced0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.669+0000 7f6be2553640 1 -- 172.21.15.19:0/2833217845 >> 172.21.15.19:0/2833217845 conn(0x7f6bdc0af670 msgr2=0x7f6bdc0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.669+0000 7f6be2553640 1 -- 172.21.15.19:0/2833217845 shutdown_connections 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.669+0000 7f6be2553640 1 -- 172.21.15.19:0/2833217845 wait complete. 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.670+0000 7f6be2553640 1 Processor -- start 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.670+0000 7f6be2553640 1 -- start start 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6be2553640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bdc071ac0 0x7f6bdc14e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6be2553640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc154ed0 0x7f6bdc0cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6be2553640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bdc0ce430 0x7f6bdc0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6be2553640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc06b660 con 0x7f6bdc154ed0 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6be2553640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc06b4e0 con 0x7f6bdc071ac0 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6be2553640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc06b7e0 con 0x7f6bdc0ce430 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc154ed0 0x7f6bdc0cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc154ed0 0x7f6bdc0cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39018/0 (socket says 172.21.15.19:39018) 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6bdb7fe640 1 -- 172.21.15.19:0/3743565775 learned_addr learned my addr 172.21.15.19:0/3743565775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6bdbfff640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bdc071ac0 0x7f6bdc14e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6be0af0640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bdc0ce430 0x7f6bdc0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6bdb7fe640 1 -- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bdc0ce430 msgr2=0x7f6bdc0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6bdb7fe640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bdc0ce430 0x7f6bdc0b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6bdb7fe640 1 -- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bdc071ac0 msgr2=0x7f6bdc14e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6bdb7fe640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bdc071ac0 0x7f6bdc14e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.671+0000 7f6bdb7fe640 1 -- 172.21.15.19:0/3743565775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc0ccc50 con 0x7f6bdc154ed0 2024-04-25T10:00:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.672+0000 7f6be0af0640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bdc0ce430 0x7f6bdc0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.672+0000 7f6bdb7fe640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc154ed0 0x7f6bdc0cc420 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000c400 tx=0x7f6bd000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.672+0000 7f6bd8ff9640 1 -- 172.21.15.19:0/3743565775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0017070 con 0x7f6bdc154ed0 2024-04-25T10:00:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.672+0000 7f6bd8ff9640 1 -- 172.21.15.19:0/3743565775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd0005d00 con 0x7f6bdc154ed0 2024-04-25T10:00:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.672+0000 7f6be2553640 1 -- 172.21.15.19:0/3743565775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc0ccf30 con 0x7f6bdc154ed0 2024-04-25T10:00:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.672+0000 7f6bd8ff9640 1 -- 172.21.15.19:0/3743565775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0007310 con 0x7f6bdc154ed0 2024-04-25T10:00:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.672+0000 7f6be2553640 1 -- 172.21.15.19:0/3743565775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc155460 con 0x7f6bdc154ed0 2024-04-25T10:00:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.673+0000 7f6be2553640 1 -- 172.21.15.19:0/3743565775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bdc0b2b10 con 0x7f6bdc154ed0 2024-04-25T10:00:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.673+0000 7f6bd8ff9640 1 -- 172.21.15.19:0/3743565775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6bd001a070 con 0x7f6bdc154ed0 2024-04-25T10:00:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.674+0000 7f6bd8ff9640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc806b500 0x7f6bc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.674+0000 7f6bd8ff9640 1 -- 172.21.15.19:0/3743565775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6bd0089780 con 0x7f6bdc154ed0 2024-04-25T10:00:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.674+0000 7f6bdbfff640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc806b500 0x7f6bc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.674+0000 7f6bdbfff640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc806b500 0x7f6bc806d9c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc034000 tx=0x7f6bcc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:15.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.678+0000 7f6bd8ff9640 1 -- 172.21.15.19:0/3743565775 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6bd0057da0 con 0x7f6bdc154ed0 2024-04-25T10:00:15.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.926+0000 7f6be2553640 1 -- 172.21.15.19:0/3743565775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6bdc073990 con 0x7f6bdc154ed0 2024-04-25T10:00:15.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.927+0000 7f6bd8ff9640 1 -- 172.21.15.19:0/3743565775 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f6bd005be40 con 0x7f6bdc154ed0 2024-04-25T10:00:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:15.931 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":453,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076566130,"bytes_used":2547040256,"bytes_avail":770547073024,"bytes_total":773094113280,"write_bytes_sec":38552098,"read_op_per_sec":0,"write_op_per_sec":38},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8025}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T10:00:15.009423+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.928+0000 7f6be2553640 1 -- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc806b500 msgr2=0x7f6bc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.928+0000 7f6be2553640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc806b500 0x7f6bc806d9c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc034000 tx=0x7f6bcc02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.928+0000 7f6be2553640 1 -- 172.21.15.19:0/3743565775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc154ed0 msgr2=0x7f6bdc0cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.928+0000 7f6be2553640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc154ed0 0x7f6bdc0cc420 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000c400 tx=0x7f6bd000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.928+0000 7f6be2553640 1 -- 172.21.15.19:0/3743565775 shutdown_connections 2024-04-25T10:00:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.928+0000 7f6be2553640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6bc806b500 0x7f6bc806d9c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.928+0000 7f6be2553640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bdc0ce430 0x7f6bdc0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.928+0000 7f6be2553640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bdc154ed0 0x7f6bdc0cc420 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.928+0000 7f6be2553640 1 --2- 172.21.15.19:0/3743565775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bdc071ac0 0x7f6bdc14e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.928+0000 7f6be2553640 1 -- 172.21.15.19:0/3743565775 >> 172.21.15.19:0/3743565775 conn(0x7f6bdc06c1d0 msgr2=0x7f6bdc1604e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.929+0000 7f6be2553640 1 -- 172.21.15.19:0/3743565775 shutdown_connections 2024-04-25T10:00:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:15.929+0000 7f6be2553640 1 -- 172.21.15.19:0/3743565775 wait complete. 2024-04-25T10:00:15.944 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:00:15.944 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:00:15.944 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:15.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:16.029 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:00:16.029 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:00:16.029 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:16.045 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:00:16.046 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:16.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:16.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.238+0000 7fc55315c640 1 Processor -- start 2024-04-25T10:00:16.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.238+0000 7fc55315c640 1 -- start start 2024-04-25T10:00:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc55315c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c154ee0 0x7fc54c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc55315c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c071480 con 0x7fc54c0ce590 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc55315c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c071760 con 0x7fc54c0cfcf0 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc55315c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c072090 con 0x7fc54c154ee0 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc5516f9640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc54c0ce590 0x7fc54c06d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55120/0 (socket says 172.21.15.19:55120) 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc5516f9640 1 -- 172.21.15.19:0/1170306461 learned_addr learned my addr 172.21.15.19:0/1170306461 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215809028 0 0) 0x7fc54c071480 con 0x7fc54c0ce590 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc550ef8640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c154ee0 0x7fc54c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc53c005730 con 0x7fc54c0ce590 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4219330074 0 0) 0x7fc53c005730 con 0x7fc54c0ce590 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c154ee0 msgr2=0x7fc54c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc54b7fe640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c154ee0 0x7fc54c0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 >> v1:172.21.15.112:6789/0 conn(0x7fc54c0cfcf0 legacy=0x7fc54c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c071a40 con 0x7fc54c0ce590 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fc5440019f0 con 0x7fc54c0ce590 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.239+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 >> v1:172.21.15.19:6789/0 conn(0x7fc54c0ce590 legacy=0x7fc54c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc54b7fe640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc53c005bf0 0x7fc53c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc54b7fe640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c006a10 0x7fc53c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc54b7fe640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c154ee0 0x7fc53c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c00dc80 con 0x7fc53c005bf0 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c00de00 con 0x7fc53c006a10 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c00e000 con 0x7fc54c154ee0 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc550ef8640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc53c005bf0 0x7fc53c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc5516f9640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c154ee0 0x7fc53c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc54bfff640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c006a10 0x7fc53c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc550ef8640 1 -- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c154ee0 msgr2=0x7fc53c0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc550ef8640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c154ee0 0x7fc53c0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc550ef8640 1 -- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c006a10 msgr2=0x7fc53c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc550ef8640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c006a10 0x7fc53c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc550ef8640 1 -- 172.21.15.19:0/1170306461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c005730 con 0x7fc53c005bf0 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.240+0000 7fc54bfff640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c006a10 0x7fc53c006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.241+0000 7fc550ef8640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc53c005bf0 0x7fc53c006010 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fc540009a90 tx=0x7fc53c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba53e940cca07c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.241+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540036070 con 0x7fc53c005bf0 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.241+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc540031400 con 0x7fc53c005bf0 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.241+0000 7fc54b7fe640 1 -- 172.21.15.19:0/1170306461 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540009690 con 0x7fc53c005bf0 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.241+0000 7fc55315c640 1 -- 172.21.15.19:0/1170306461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc53c005bf0 msgr2=0x7fc53c006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.241+0000 7fc55315c640 1 --2- 172.21.15.19:0/1170306461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc53c005bf0 0x7fc53c006010 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fc540009a90 tx=0x7fc53c007a40 comp rx=0 tx=0).stop 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.241+0000 7fc5516f9640 1 -- 172.21.15.19:0/1170306461 reap_dead start 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.241+0000 7fc55315c640 1 -- 172.21.15.19:0/1170306461 shutdown_connections 2024-04-25T10:00:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.241+0000 7fc55315c640 1 -- 172.21.15.19:0/1170306461 >> 172.21.15.19:0/1170306461 conn(0x7fc54c06abe0 msgr2=0x7fc54c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.241+0000 7fc55315c640 1 -- 172.21.15.19:0/1170306461 shutdown_connections 2024-04-25T10:00:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.241+0000 7fc55315c640 1 -- 172.21.15.19:0/1170306461 wait complete. 2024-04-25T10:00:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.243+0000 7fc55315c640 1 Processor -- start 2024-04-25T10:00:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.243+0000 7fc55315c640 1 -- start start 2024-04-25T10:00:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.243+0000 7fc55315c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c005bf0 0x7fc54c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.243+0000 7fc55315c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc54c15c760 0x7fc54c141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.243+0000 7fc55315c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c1422f0 0x7fc54c142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.243+0000 7fc55315c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c071230 con 0x7fc54c15c760 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.243+0000 7fc55315c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c1552c0 con 0x7fc53c005bf0 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.243+0000 7fc55315c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c155440 con 0x7fc54c1422f0 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc54bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc54c15c760 0x7fc54c141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc54bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc54c15c760 0x7fc54c141db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39030/0 (socket says 172.21.15.19:39030) 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc54bfff640 1 -- 172.21.15.19:0/2045176633 learned_addr learned my addr 172.21.15.19:0/2045176633 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc5516f9640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c1422f0 0x7fc54c142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc54bfff640 1 -- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c1422f0 msgr2=0x7fc54c142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc550ef8640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c005bf0 0x7fc54c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc54bfff640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c1422f0 0x7fc54c142770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc54bfff640 1 -- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c005bf0 msgr2=0x7fc54c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc54bfff640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c005bf0 0x7fc54c15c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc54bfff640 1 -- 172.21.15.19:0/2045176633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c1433b0 con 0x7fc54c15c760 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc54bfff640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc54c15c760 0x7fc54c141db0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc53800e530 tx=0x7fc53800e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc5497fa640 1 -- 172.21.15.19:0/2045176633 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5380052e0 con 0x7fc54c15c760 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc5497fa640 1 -- 172.21.15.19:0/2045176633 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc538018070 con 0x7fc54c15c760 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc5497fa640 1 -- 172.21.15.19:0/2045176633 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc538003810 con 0x7fc54c15c760 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc55315c640 1 -- 172.21.15.19:0/2045176633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c143690 con 0x7fc54c15c760 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.244+0000 7fc55315c640 1 -- 172.21.15.19:0/2045176633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c143c50 con 0x7fc54c15c760 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.245+0000 7fc55315c640 1 -- 172.21.15.19:0/2045176633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c073990 con 0x7fc54c15c760 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.246+0000 7fc5497fa640 1 -- 172.21.15.19:0/2045176633 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5380039b0 con 0x7fc54c15c760 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.246+0000 7fc5497fa640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc53006b4d0 0x7fc53006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.246+0000 7fc5497fa640 1 -- 172.21.15.19:0/2045176633 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc53808a0c0 con 0x7fc54c15c760 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.246+0000 7fc550ef8640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc53006b4d0 0x7fc53006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.247+0000 7fc550ef8640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc53006b4d0 0x7fc53006d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc540033000 tx=0x7fc540008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.250+0000 7fc5497fa640 1 -- 172.21.15.19:0/2045176633 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc538058d20 con 0x7fc54c15c760 2024-04-25T10:00:16.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.432+0000 7fc55315c640 1 -- 172.21.15.19:0/2045176633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc54c06d360 con 0x7fc54c15c760 2024-04-25T10:00:16.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.433+0000 7fc5497fa640 1 -- 172.21.15.19:0/2045176633 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16872 (secure 0 0 0) 0x7fc538058620 con 0x7fc54c15c760 2024-04-25T10:00:16.437 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:16.438 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":106,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:48.558694+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:45.201424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:45.885235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 -- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc53006b4d0 msgr2=0x7fc53006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc53006b4d0 0x7fc53006d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc540033000 tx=0x7fc540008030 comp rx=0 tx=0).stop 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 -- 172.21.15.19:0/2045176633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc54c15c760 msgr2=0x7fc54c141db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc54c15c760 0x7fc54c141db0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc53800e530 tx=0x7fc53800e9f0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 -- 172.21.15.19:0/2045176633 shutdown_connections 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc53006b4d0 0x7fc53006d990 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc54c1422f0 0x7fc54c142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc54c15c760 0x7fc54c141db0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 --2- 172.21.15.19:0/2045176633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c005bf0 0x7fc54c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 -- 172.21.15.19:0/2045176633 >> 172.21.15.19:0/2045176633 conn(0x7fc54c06abe0 msgr2=0x7fc54c1535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 -- 172.21.15.19:0/2045176633 shutdown_connections 2024-04-25T10:00:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.435+0000 7fc55315c640 1 -- 172.21.15.19:0/2045176633 wait complete. 2024-04-25T10:00:16.450 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:59:45.201424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:59:45.885235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:00:16.451 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:00:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.550+0000 7ff56e29e640 1 Processor -- start 2024-04-25T10:00:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.550+0000 7ff56e29e640 1 -- start start 2024-04-25T10:00:16.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.551+0000 7ff56e29e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff5680b5620 0x7ff5680b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.551+0000 7ff56e29e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff56806c2d0 0x7ff56806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.551+0000 7ff56e29e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56806cc80 con 0x7ff56815b7d0 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.551+0000 7ff56e29e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56806cf50 con 0x7ff5680b5620 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.551+0000 7ff56e29e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680ce530 con 0x7ff56806c2d0 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.551+0000 7ff5677fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff5680b5620 0x7ff5680b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.551+0000 7ff5677fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff5680b5620 0x7ff5680b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39036/0 (socket says 172.21.15.19:39036) 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.551+0000 7ff5677fe640 1 -- 172.21.15.19:0/628110133 learned_addr learned my addr 172.21.15.19:0/628110133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.551+0000 7ff56c83b640 1 --2- 172.21.15.19:0/628110133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff56806c2d0 0x7ff56806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.551+0000 7ff5677fe640 1 -- 172.21.15.19:0/628110133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff56806c2d0 msgr2=0x7ff56806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.551+0000 7ff5677fe640 1 --2- 172.21.15.19:0/628110133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff56806c2d0 0x7ff56806c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.552+0000 7ff5677fe640 1 -- 172.21.15.19:0/628110133 >> v1:172.21.15.112:6789/0 conn(0x7ff56815b7d0 legacy=0x7ff5680cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.552+0000 7ff5677fe640 1 -- 172.21.15.19:0/628110133 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5680ce810 con 0x7ff5680b5620 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.552+0000 7ff5677fe640 1 --2- 172.21.15.19:0/628110133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5680b5620 0x7ff5680b5ab0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff558009870 tx=0x7ff55802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57adc0304193c00e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.552+0000 7ff566ffd640 1 -- 172.21.15.19:0/628110133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558037070 con 0x7ff5680b5620 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.552+0000 7ff566ffd640 1 -- 172.21.15.19:0/628110133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff558032440 con 0x7ff5680b5620 2024-04-25T10:00:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.552+0000 7ff566ffd640 1 -- 172.21.15.19:0/628110133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558032810 con 0x7ff5680b5620 2024-04-25T10:00:16.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.553+0000 7ff56e29e640 1 -- 172.21.15.19:0/628110133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5680b5620 msgr2=0x7ff5680b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.553+0000 7ff56e29e640 1 --2- 172.21.15.19:0/628110133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5680b5620 0x7ff5680b5ab0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff558009870 tx=0x7ff55802a270 comp rx=0 tx=0).stop 2024-04-25T10:00:16.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.553+0000 7ff56e29e640 1 -- 172.21.15.19:0/628110133 shutdown_connections 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.553+0000 7ff56e29e640 1 --2- 172.21.15.19:0/628110133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff56806c2d0 0x7ff56806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.553+0000 7ff56e29e640 1 --2- 172.21.15.19:0/628110133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5680b5620 0x7ff5680b5ab0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.553+0000 7ff56e29e640 1 -- 172.21.15.19:0/628110133 >> 172.21.15.19:0/628110133 conn(0x7ff5680d0800 msgr2=0x7ff5680d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.553+0000 7ff56e29e640 1 -- 172.21.15.19:0/628110133 shutdown_connections 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.553+0000 7ff56e29e640 1 -- 172.21.15.19:0/628110133 wait complete. 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.554+0000 7ff56e29e640 1 Processor -- start 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.554+0000 7ff56e29e640 1 -- start start 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.555+0000 7ff56e29e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff56806c2d0 0x7ff5680ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.555+0000 7ff56e29e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5680b5620 0x7ff5680cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.555+0000 7ff56e29e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff56815b7d0 0x7ff5680c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.555+0000 7ff56e29e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56815bbb0 con 0x7ff56815b7d0 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.555+0000 7ff56e29e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56806d230 con 0x7ff5680b5620 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.555+0000 7ff56e29e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56815bd30 con 0x7ff56806c2d0 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.555+0000 7ff56c83b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff56815b7d0 0x7ff5680c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.555+0000 7ff56c83b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff56815b7d0 0x7ff5680c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39046/0 (socket says 172.21.15.19:39046) 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.555+0000 7ff56c83b640 1 -- 172.21.15.19:0/279587165 learned_addr learned my addr 172.21.15.19:0/279587165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.555+0000 7ff5677fe640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5680b5620 0x7ff5680cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.555+0000 7ff567fff640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff56806c2d0 0x7ff5680ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff56c83b640 1 -- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff56806c2d0 msgr2=0x7ff5680ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff56c83b640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff56806c2d0 0x7ff5680ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff56c83b640 1 -- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5680b5620 msgr2=0x7ff5680cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff56c83b640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5680b5620 0x7ff5680cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff56c83b640 1 -- 172.21.15.19:0/279587165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5680b4760 con 0x7ff56815b7d0 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff5677fe640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5680b5620 0x7ff5680cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff567fff640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff56806c2d0 0x7ff5680ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff56c83b640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff56815b7d0 0x7ff5680c8090 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ff560007d90 tx=0x7ff56000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff564ff9640 1 -- 172.21.15.19:0/279587165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560014420 con 0x7ff56815b7d0 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff564ff9640 1 -- 172.21.15.19:0/279587165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff560019070 con 0x7ff56815b7d0 2024-04-25T10:00:16.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff564ff9640 1 -- 172.21.15.19:0/279587165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560014820 con 0x7ff56815b7d0 2024-04-25T10:00:16.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff56e29e640 1 -- 172.21.15.19:0/279587165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5680b4a40 con 0x7ff56815b7d0 2024-04-25T10:00:16.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.556+0000 7ff56e29e640 1 -- 172.21.15.19:0/279587165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5680cbc40 con 0x7ff56815b7d0 2024-04-25T10:00:16.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.557+0000 7ff56e29e640 1 -- 172.21.15.19:0/279587165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff568073a00 con 0x7ff56815b7d0 2024-04-25T10:00:16.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.557+0000 7ff564ff9640 1 -- 172.21.15.19:0/279587165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5600149c0 con 0x7ff56815b7d0 2024-04-25T10:00:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.558+0000 7ff564ff9640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff54806b4d0 0x7ff54806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.558+0000 7ff564ff9640 1 -- 172.21.15.19:0/279587165 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff56008abe0 con 0x7ff56815b7d0 2024-04-25T10:00:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.558+0000 7ff567fff640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff54806b4d0 0x7ff54806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.559+0000 7ff567fff640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff54806b4d0 0x7ff54806d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff5500015e0 tx=0x7ff550009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.562+0000 7ff564ff9640 1 -- 172.21.15.19:0/279587165 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5600598d0 con 0x7ff56815b7d0 2024-04-25T10:00:16.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.741+0000 7ff56e29e640 1 -- 172.21.15.19:0/279587165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff56815ecf0 con 0x7ff56815b7d0 2024-04-25T10:00:16.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.742+0000 7ff564ff9640 1 -- 172.21.15.19:0/279587165 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5600591d0 con 0x7ff56815b7d0 2024-04-25T10:00:16.746 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:00:16.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.743+0000 7ff56e29e640 1 -- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff54806b4d0 msgr2=0x7ff54806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.743+0000 7ff56e29e640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff54806b4d0 0x7ff54806d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff5500015e0 tx=0x7ff550009040 comp rx=0 tx=0).stop 2024-04-25T10:00:16.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.743+0000 7ff56e29e640 1 -- 172.21.15.19:0/279587165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff56815b7d0 msgr2=0x7ff5680c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.743+0000 7ff56e29e640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff56815b7d0 0x7ff5680c8090 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ff560007d90 tx=0x7ff56000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.744+0000 7ff56e29e640 1 -- 172.21.15.19:0/279587165 shutdown_connections 2024-04-25T10:00:16.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.744+0000 7ff56e29e640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff54806b4d0 0x7ff54806d990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.744+0000 7ff56e29e640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff56815b7d0 0x7ff5680c8090 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.744+0000 7ff56e29e640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5680b5620 0x7ff5680cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.744+0000 7ff56e29e640 1 --2- 172.21.15.19:0/279587165 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff56806c2d0 0x7ff5680ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.744+0000 7ff56e29e640 1 -- 172.21.15.19:0/279587165 >> 172.21.15.19:0/279587165 conn(0x7ff5680d0800 msgr2=0x7ff5680d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:16.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.744+0000 7ff56e29e640 1 -- 172.21.15.19:0/279587165 shutdown_connections 2024-04-25T10:00:16.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.744+0000 7ff56e29e640 1 -- 172.21.15.19:0/279587165 wait complete. 2024-04-25T10:00:16.759 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:00:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.858+0000 7f47fd170640 1 Processor -- start 2024-04-25T10:00:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.858+0000 7f47fd170640 1 -- start start 2024-04-25T10:00:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.859+0000 7f47fd170640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f47f8161890 0x7f47f80b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.859+0000 7f47fd170640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f80cf1d0 con 0x7f47f80b54a0 2024-04-25T10:00:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.859+0000 7f47fd170640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f8161620 con 0x7f47f8161240 2024-04-25T10:00:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.859+0000 7f47fd170640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f80b7120 con 0x7f47f8161890 2024-04-25T10:00:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.859+0000 7f47f659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f47f8161890 0x7f47f80b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.859+0000 7f47f759e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f47f80b54a0 0x7f47f80d1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55136/0 (socket says 172.21.15.19:55136) 2024-04-25T10:00:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.859+0000 7f47f759e640 1 -- 172.21.15.19:0/610535705 learned_addr learned my addr 172.21.15.19:0/610535705 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.859+0000 7f47f659c640 1 -- 172.21.15.19:0/610535705 >> v1:172.21.15.112:6789/0 conn(0x7f47f8161240 legacy=0x7f47f80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.859+0000 7f47f659c640 1 -- 172.21.15.19:0/610535705 >> v1:172.21.15.19:6789/0 conn(0x7f47f80b54a0 legacy=0x7f47f80d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.859+0000 7f47f659c640 1 -- 172.21.15.19:0/610535705 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f8161e00 con 0x7f47f8161890 2024-04-25T10:00:16.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.860+0000 7f47f659c640 1 --2- 172.21.15.19:0/610535705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f8161890 0x7f47f80b6be0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0097a0 tx=0x7f47ec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9035af512876af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.860+0000 7f47f5d9b640 1 -- 172.21.15.19:0/610535705 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec037070 con 0x7f47f8161890 2024-04-25T10:00:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.860+0000 7f47f5d9b640 1 -- 172.21.15.19:0/610535705 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec032440 con 0x7f47f8161890 2024-04-25T10:00:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.860+0000 7f47f5d9b640 1 -- 172.21.15.19:0/610535705 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec032810 con 0x7f47f8161890 2024-04-25T10:00:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.861+0000 7f47fd170640 1 -- 172.21.15.19:0/610535705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f8161890 msgr2=0x7f47f80b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.861+0000 7f47fd170640 1 --2- 172.21.15.19:0/610535705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f8161890 0x7f47f80b6be0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0097a0 tx=0x7f47ec02a230 comp rx=0 tx=0).stop 2024-04-25T10:00:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.861+0000 7f47fd170640 1 -- 172.21.15.19:0/610535705 shutdown_connections 2024-04-25T10:00:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.861+0000 7f47fd170640 1 --2- 172.21.15.19:0/610535705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f8161890 0x7f47f80b6be0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.861+0000 7f47fd170640 1 -- 172.21.15.19:0/610535705 >> 172.21.15.19:0/610535705 conn(0x7f47f80d04a0 msgr2=0x7f47f80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.861+0000 7f47fd170640 1 -- 172.21.15.19:0/610535705 shutdown_connections 2024-04-25T10:00:16.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.861+0000 7f47fd170640 1 -- 172.21.15.19:0/610535705 wait complete. 2024-04-25T10:00:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.862+0000 7f47fd170640 1 Processor -- start 2024-04-25T10:00:16.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.862+0000 7f47fd170640 1 -- start start 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.863+0000 7f47fd170640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47f80b54a0 0x7f47f8151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.863+0000 7f47fd170640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f8161240 0x7f47f815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.863+0000 7f47fd170640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f8161890 0x7f47f80cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.863+0000 7f47fd170640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f81626a0 con 0x7f47f8161890 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.863+0000 7f47fd170640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f81623c0 con 0x7f47f8161240 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.863+0000 7f47fd170640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80d15f0 con 0x7f47f80b54a0 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.863+0000 7f47f759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f8161890 0x7f47f80cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.863+0000 7f47f759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f8161890 0x7f47f80cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39070/0 (socket says 172.21.15.19:39070) 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.863+0000 7f47f759e640 1 -- 172.21.15.19:0/2717055532 learned_addr learned my addr 172.21.15.19:0/2717055532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.863+0000 7f47f659c640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f8161240 0x7f47f815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.863+0000 7f47f6d9d640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47f80b54a0 0x7f47f8151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.864+0000 7f47f759e640 1 -- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47f80b54a0 msgr2=0x7f47f8151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.864+0000 7f47f759e640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47f80b54a0 0x7f47f8151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.864+0000 7f47f759e640 1 -- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f8161240 msgr2=0x7f47f815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.864+0000 7f47f759e640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f8161240 0x7f47f815a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.864+0000 7f47f759e640 1 -- 172.21.15.19:0/2717055532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f815f920 con 0x7f47f8161890 2024-04-25T10:00:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.864+0000 7f47f759e640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f8161890 0x7f47f80cc220 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f47e800cbc0 tx=0x7f47e800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.864+0000 7f47df7fe640 1 -- 172.21.15.19:0/2717055532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8017070 con 0x7f47f8161890 2024-04-25T10:00:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.864+0000 7f47df7fe640 1 -- 172.21.15.19:0/2717055532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e800ad40 con 0x7f47f8161890 2024-04-25T10:00:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.864+0000 7f47fd170640 1 -- 172.21.15.19:0/2717055532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f815fc00 con 0x7f47f8161890 2024-04-25T10:00:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.864+0000 7f47df7fe640 1 -- 172.21.15.19:0/2717055532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8012600 con 0x7f47f8161890 2024-04-25T10:00:16.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.864+0000 7f47fd170640 1 -- 172.21.15.19:0/2717055532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f81601c0 con 0x7f47f8161890 2024-04-25T10:00:16.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.865+0000 7f47fd170640 1 -- 172.21.15.19:0/2717055532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f8073a00 con 0x7f47f8161890 2024-04-25T10:00:16.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.866+0000 7f47df7fe640 1 -- 172.21.15.19:0/2717055532 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47e801a070 con 0x7f47f8161890 2024-04-25T10:00:16.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.866+0000 7f47df7fe640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47e406b4d0 0x7f47e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.866+0000 7f47df7fe640 1 -- 172.21.15.19:0/2717055532 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f47e8089f70 con 0x7f47f8161890 2024-04-25T10:00:16.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.866+0000 7f47f6d9d640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47e406b4d0 0x7f47e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.867+0000 7f47f6d9d640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47e406b4d0 0x7f47e406d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f47f0001640 tx=0x7f47f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:16.870+0000 7f47df7fe640 1 -- 172.21.15.19:0/2717055532 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47e80586a0 con 0x7f47f8161890 2024-04-25T10:00:17.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.048+0000 7f47fd170640 1 -- 172.21.15.19:0/2717055532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47f815b760 con 0x7f47f8161890 2024-04-25T10:00:17.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.049+0000 7f47df7fe640 1 -- 172.21.15.19:0/2717055532 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f47e805c740 con 0x7f47f8161890 2024-04-25T10:00:17.053 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:00:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 -- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47e406b4d0 msgr2=0x7f47e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47e406b4d0 0x7f47e406d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f47f0001640 tx=0x7f47f0009040 comp rx=0 tx=0).stop 2024-04-25T10:00:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 -- 172.21.15.19:0/2717055532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f8161890 msgr2=0x7f47f80cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f8161890 0x7f47f80cc220 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f47e800cbc0 tx=0x7f47e800a430 comp rx=0 tx=0).stop 2024-04-25T10:00:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 -- 172.21.15.19:0/2717055532 shutdown_connections 2024-04-25T10:00:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f47e406b4d0 0x7f47e406d990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f8161890 0x7f47f80cc220 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47f8161240 0x7f47f815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 --2- 172.21.15.19:0/2717055532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47f80b54a0 0x7f47f8151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 -- 172.21.15.19:0/2717055532 >> 172.21.15.19:0/2717055532 conn(0x7f47f80d04a0 msgr2=0x7f47f815dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 -- 172.21.15.19:0/2717055532 shutdown_connections 2024-04-25T10:00:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.051+0000 7f47fd170640 1 -- 172.21.15.19:0/2717055532 wait complete. 2024-04-25T10:00:17.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:00:17.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.166+0000 7f98a5f34640 1 Processor -- start 2024-04-25T10:00:17.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.166+0000 7f98a5f34640 1 -- start start 2024-04-25T10:00:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.167+0000 7f98a5f34640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98a015b7d0 0x7f98a00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.167+0000 7f98a5f34640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a006cf50 con 0x7f98a006c9e0 2024-04-25T10:00:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.167+0000 7f98a5f34640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a00ce530 con 0x7f98a00b5620 2024-04-25T10:00:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.167+0000 7f98a5f34640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a00ce810 con 0x7f98a015b7d0 2024-04-25T10:00:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.167+0000 7f989f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98a015b7d0 0x7f98a00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.167+0000 7f989ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f98a006c9e0 0x7f98a00d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55150/0 (socket says 172.21.15.19:55150) 2024-04-25T10:00:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.167+0000 7f989f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98a015b7d0 0x7f98a00cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39080/0 (socket says 172.21.15.19:39080) 2024-04-25T10:00:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.167+0000 7f989ffff640 1 -- 172.21.15.19:0/2854279275 learned_addr learned my addr 172.21.15.19:0/2854279275 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.167+0000 7f989f7fe640 1 -- 172.21.15.19:0/2854279275 >> v1:172.21.15.112:6789/0 conn(0x7f98a00b5620 legacy=0x7f98a006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.167+0000 7f989f7fe640 1 -- 172.21.15.19:0/2854279275 >> v1:172.21.15.19:6789/0 conn(0x7f98a006c9e0 legacy=0x7f98a00d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.167+0000 7f989f7fe640 1 -- 172.21.15.19:0/2854279275 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00ceaf0 con 0x7f98a015b7d0 2024-04-25T10:00:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.168+0000 7f989f7fe640 1 --2- 172.21.15.19:0/2854279275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a015b7d0 0x7f98a00cf010 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f98940097a0 tx=0x7f989402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f98b587c7097360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.168+0000 7f989e7fc640 1 -- 172.21.15.19:0/2854279275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894037070 con 0x7f98a015b7d0 2024-04-25T10:00:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.168+0000 7f989e7fc640 1 -- 172.21.15.19:0/2854279275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9894032440 con 0x7f98a015b7d0 2024-04-25T10:00:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.168+0000 7f989e7fc640 1 -- 172.21.15.19:0/2854279275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894032810 con 0x7f98a015b7d0 2024-04-25T10:00:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.169+0000 7f98a5f34640 1 -- 172.21.15.19:0/2854279275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a015b7d0 msgr2=0x7f98a00cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.169+0000 7f98a5f34640 1 --2- 172.21.15.19:0/2854279275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a015b7d0 0x7f98a00cf010 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f98940097a0 tx=0x7f989402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.169+0000 7f98a5f34640 1 -- 172.21.15.19:0/2854279275 shutdown_connections 2024-04-25T10:00:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.169+0000 7f98a5f34640 1 --2- 172.21.15.19:0/2854279275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a015b7d0 0x7f98a00cf010 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.169+0000 7f98a5f34640 1 -- 172.21.15.19:0/2854279275 >> 172.21.15.19:0/2854279275 conn(0x7f98a00d0800 msgr2=0x7f98a00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.169+0000 7f98a5f34640 1 -- 172.21.15.19:0/2854279275 shutdown_connections 2024-04-25T10:00:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.169+0000 7f98a5f34640 1 -- 172.21.15.19:0/2854279275 wait complete. 2024-04-25T10:00:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.170+0000 7f98a5f34640 1 Processor -- start 2024-04-25T10:00:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.170+0000 7f98a5f34640 1 -- start start 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f98a5f34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98a006c9e0 0x7f98a01f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f98a5f34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a00b5620 0x7f98a01f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f98a5f34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98a015b7d0 0x7f98a00cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f98a5f34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00cf6d0 con 0x7f98a00b5620 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f98a5f34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00cf550 con 0x7f98a006c9e0 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f98a5f34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a015bbb0 con 0x7f98a015b7d0 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f989effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a00b5620 0x7f98a01f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f989effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a00b5620 0x7f98a01f6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39092/0 (socket says 172.21.15.19:39092) 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f989effd640 1 -- 172.21.15.19:0/1167900242 learned_addr learned my addr 172.21.15.19:0/1167900242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f989ffff640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98a015b7d0 0x7f98a00cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f989f7fe640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98a006c9e0 0x7f98a01f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f989effd640 1 -- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98a015b7d0 msgr2=0x7f98a00cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f989effd640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98a015b7d0 0x7f98a00cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f989effd640 1 -- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98a006c9e0 msgr2=0x7f98a01f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f989effd640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98a006c9e0 0x7f98a01f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f989effd640 1 -- 172.21.15.19:0/1167900242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00cd1d0 con 0x7f98a00b5620 2024-04-25T10:00:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f989effd640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a00b5620 0x7f98a01f6270 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f989000c460 tx=0x7f989000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f9887fff640 1 -- 172.21.15.19:0/1167900242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890016020 con 0x7f98a00b5620 2024-04-25T10:00:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f9887fff640 1 -- 172.21.15.19:0/1167900242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9890012870 con 0x7f98a00b5620 2024-04-25T10:00:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f98a5f34640 1 -- 172.21.15.19:0/1167900242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a00c8720 con 0x7f98a00b5620 2024-04-25T10:00:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.171+0000 7f9887fff640 1 -- 172.21.15.19:0/1167900242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890012c60 con 0x7f98a00b5620 2024-04-25T10:00:17.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.172+0000 7f98a5f34640 1 -- 172.21.15.19:0/1167900242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a00c8ce0 con 0x7f98a00b5620 2024-04-25T10:00:17.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.173+0000 7f98a5f34640 1 -- 172.21.15.19:0/1167900242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a0073a00 con 0x7f98a00b5620 2024-04-25T10:00:17.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.173+0000 7f9887fff640 1 -- 172.21.15.19:0/1167900242 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9890019020 con 0x7f98a00b5620 2024-04-25T10:00:17.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.174+0000 7f9887fff640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988c06b500 0x7f988c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.174+0000 7f9887fff640 1 -- 172.21.15.19:0/1167900242 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9890089e80 con 0x7f98a00b5620 2024-04-25T10:00:17.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.174+0000 7f989f7fe640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988c06b500 0x7f988c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.174+0000 7f989f7fe640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988c06b500 0x7f988c06d9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9894034000 tx=0x7f989402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:17.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.178+0000 7f9887fff640 1 -- 172.21.15.19:0/1167900242 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9890020070 con 0x7f98a00b5620 2024-04-25T10:00:17.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.355+0000 7f98a5f34640 1 -- 172.21.15.19:0/1167900242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98a015ec80 con 0x7f98a00b5620 2024-04-25T10:00:17.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.355+0000 7f9887fff640 1 -- 172.21.15.19:0/1167900242 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9890058ae0 con 0x7f98a00b5620 2024-04-25T10:00:17.360 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:00:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.357+0000 7f98a5f34640 1 -- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988c06b500 msgr2=0x7f988c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.358+0000 7f98a5f34640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988c06b500 0x7f988c06d9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9894034000 tx=0x7f989402e000 comp rx=0 tx=0).stop 2024-04-25T10:00:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.358+0000 7f98a5f34640 1 -- 172.21.15.19:0/1167900242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a00b5620 msgr2=0x7f98a01f6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.358+0000 7f98a5f34640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a00b5620 0x7f98a01f6270 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f989000c460 tx=0x7f989000c920 comp rx=0 tx=0).stop 2024-04-25T10:00:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.358+0000 7f98a5f34640 1 -- 172.21.15.19:0/1167900242 shutdown_connections 2024-04-25T10:00:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.358+0000 7f98a5f34640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f988c06b500 0x7f988c06d9c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.358+0000 7f98a5f34640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98a015b7d0 0x7f98a00cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.358+0000 7f98a5f34640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a00b5620 0x7f98a01f6270 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.358+0000 7f98a5f34640 1 --2- 172.21.15.19:0/1167900242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98a006c9e0 0x7f98a01f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.358+0000 7f98a5f34640 1 -- 172.21.15.19:0/1167900242 >> 172.21.15.19:0/1167900242 conn(0x7f98a00d0800 msgr2=0x7f98a00b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.358+0000 7f98a5f34640 1 -- 172.21.15.19:0/1167900242 shutdown_connections 2024-04-25T10:00:17.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.358+0000 7f98a5f34640 1 -- 172.21.15.19:0/1167900242 wait complete. 2024-04-25T10:00:17.373 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:00:17.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.470+0000 7fcb4816a640 1 Processor -- start 2024-04-25T10:00:17.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.471+0000 7fcb4816a640 1 -- start start 2024-04-25T10:00:17.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.471+0000 7fcb4816a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb400cd8a0 0x7fcb4015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.471+0000 7fcb4816a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcb40150b90 0x7fcb40151020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.471+0000 7fcb4816a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4015e6e0 con 0x7fcb400cd4c0 2024-04-25T10:00:17.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.471+0000 7fcb4816a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb40151560 con 0x7fcb400cd8a0 2024-04-25T10:00:17.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.471+0000 7fcb4816a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4015e9c0 con 0x7fcb40150b90 2024-04-25T10:00:17.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.472+0000 7fcb45705640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb400cd8a0 0x7fcb4015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.472+0000 7fcb45f06640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcb400cd4c0 0x7fcb4015dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55154/0 (socket says 172.21.15.19:55154) 2024-04-25T10:00:17.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.472+0000 7fcb45705640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb400cd8a0 0x7fcb4015b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39096/0 (socket says 172.21.15.19:39096) 2024-04-25T10:00:17.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.472+0000 7fcb45f06640 1 -- 172.21.15.19:0/2916749818 learned_addr learned my addr 172.21.15.19:0/2916749818 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:17.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.472+0000 7fcb46707640 1 --2- 172.21.15.19:0/2916749818 >> v2:172.21.15.112:3300/0 conn(0x7fcb40150b90 0x7fcb40151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.472+0000 7fcb45705640 1 -- 172.21.15.19:0/2916749818 >> v2:172.21.15.112:3300/0 conn(0x7fcb40150b90 msgr2=0x7fcb40151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:17.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.472+0000 7fcb45705640 1 --2- 172.21.15.19:0/2916749818 >> v2:172.21.15.112:3300/0 conn(0x7fcb40150b90 0x7fcb40151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.472+0000 7fcb45705640 1 -- 172.21.15.19:0/2916749818 >> v1:172.21.15.19:6789/0 conn(0x7fcb400cd4c0 legacy=0x7fcb4015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.472+0000 7fcb44f04640 1 -- 172.21.15.19:0/2916749818 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778617252 0 0) 0x7fcb4015e6e0 con 0x7fcb400cd4c0 2024-04-25T10:00:17.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.472+0000 7fcb45705640 1 -- 172.21.15.19:0/2916749818 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4015a9c0 con 0x7fcb400cd8a0 2024-04-25T10:00:17.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb45705640 1 --2- 172.21.15.19:0/2916749818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400cd8a0 0x7fcb4015b1c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fcb400cec10 tx=0x7fcb3402b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ac2fd6ae414cba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:17.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb44f04640 1 -- 172.21.15.19:0/2916749818 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400b040 con 0x7fcb400cd8a0 2024-04-25T10:00:17.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb44f04640 1 -- 172.21.15.19:0/2916749818 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb34030710 con 0x7fcb400cd8a0 2024-04-25T10:00:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb44f04640 1 -- 172.21.15.19:0/2916749818 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34030ae0 con 0x7fcb400cd8a0 2024-04-25T10:00:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb4816a640 1 -- 172.21.15.19:0/2916749818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400cd8a0 msgr2=0x7fcb4015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb4816a640 1 --2- 172.21.15.19:0/2916749818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400cd8a0 0x7fcb4015b1c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fcb400cec10 tx=0x7fcb3402b560 comp rx=0 tx=0).stop 2024-04-25T10:00:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb4816a640 1 -- 172.21.15.19:0/2916749818 shutdown_connections 2024-04-25T10:00:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb4816a640 1 --2- 172.21.15.19:0/2916749818 >> v2:172.21.15.112:3300/0 conn(0x7fcb40150b90 0x7fcb40151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb4816a640 1 --2- 172.21.15.19:0/2916749818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400cd8a0 0x7fcb4015b1c0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb4816a640 1 -- 172.21.15.19:0/2916749818 >> 172.21.15.19:0/2916749818 conn(0x7fcb4006abe0 msgr2=0x7fcb4006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb4816a640 1 -- 172.21.15.19:0/2916749818 shutdown_connections 2024-04-25T10:00:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.473+0000 7fcb4816a640 1 -- 172.21.15.19:0/2916749818 wait complete. 2024-04-25T10:00:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.475+0000 7fcb4816a640 1 Processor -- start 2024-04-25T10:00:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.475+0000 7fcb4816a640 1 -- start start 2024-04-25T10:00:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.475+0000 7fcb4816a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb400cd4c0 0x7fcb401e4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.475+0000 7fcb4816a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400cd8a0 0x7fcb40154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.475+0000 7fcb4816a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb40150b90 0x7fcb40156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.475+0000 7fcb4816a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4015e370 con 0x7fcb400cd8a0 2024-04-25T10:00:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.475+0000 7fcb4816a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4015a670 con 0x7fcb400cd4c0 2024-04-25T10:00:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.475+0000 7fcb4816a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4006eca0 con 0x7fcb40150b90 2024-04-25T10:00:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.475+0000 7fcb45705640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400cd8a0 0x7fcb40154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb45705640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400cd8a0 0x7fcb40154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39110/0 (socket says 172.21.15.19:39110) 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb45705640 1 -- 172.21.15.19:0/1032905030 learned_addr learned my addr 172.21.15.19:0/1032905030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb46707640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb40150b90 0x7fcb40156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb45f06640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb400cd4c0 0x7fcb401e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb45705640 1 -- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb40150b90 msgr2=0x7fcb40156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb45705640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb40150b90 0x7fcb40156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb45705640 1 -- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb400cd4c0 msgr2=0x7fcb401e4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb45705640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb400cd4c0 0x7fcb401e4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb45705640 1 -- 172.21.15.19:0/1032905030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb401e3d20 con 0x7fcb400cd8a0 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb45705640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400cd8a0 0x7fcb40154180 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400aa30 tx=0x7fcb340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb3e7fc640 1 -- 172.21.15.19:0/1032905030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34040070 con 0x7fcb400cd8a0 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb3e7fc640 1 -- 172.21.15.19:0/1032905030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb34033a50 con 0x7fcb400cd8a0 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb4816a640 1 -- 172.21.15.19:0/1032905030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb401445b0 con 0x7fcb400cd8a0 2024-04-25T10:00:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb3e7fc640 1 -- 172.21.15.19:0/1032905030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34033e40 con 0x7fcb400cd8a0 2024-04-25T10:00:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.476+0000 7fcb4816a640 1 -- 172.21.15.19:0/1032905030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb40144b70 con 0x7fcb400cd8a0 2024-04-25T10:00:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.477+0000 7fcb4816a640 1 -- 172.21.15.19:0/1032905030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb40073a00 con 0x7fcb400cd8a0 2024-04-25T10:00:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.478+0000 7fcb3e7fc640 1 -- 172.21.15.19:0/1032905030 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb34043070 con 0x7fcb400cd8a0 2024-04-25T10:00:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.478+0000 7fcb3e7fc640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb2c06b4d0 0x7fcb2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.478+0000 7fcb3e7fc640 1 -- 172.21.15.19:0/1032905030 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb340aae30 con 0x7fcb400cd8a0 2024-04-25T10:00:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.478+0000 7fcb45f06640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb2c06b4d0 0x7fcb2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.479+0000 7fcb45f06640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb2c06b4d0 0x7fcb2c06d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcb38007fd0 tx=0x7fcb38009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:17.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.482+0000 7fcb3e7fc640 1 -- 172.21.15.19:0/1032905030 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb34079b20 con 0x7fcb400cd8a0 2024-04-25T10:00:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.674+0000 7fcb4816a640 1 -- 172.21.15.19:0/1032905030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcb400ccb10 con 0x7fcb400cd8a0 2024-04-25T10:00:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.675+0000 7fcb3e7fc640 1 -- 172.21.15.19:0/1032905030 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcb34079420 con 0x7fcb400cd8a0 2024-04-25T10:00:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T10:00:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 -- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb2c06b4d0 msgr2=0x7fcb2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb2c06b4d0 0x7fcb2c06d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcb38007fd0 tx=0x7fcb38009040 comp rx=0 tx=0).stop 2024-04-25T10:00:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 -- 172.21.15.19:0/1032905030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400cd8a0 msgr2=0x7fcb40154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400cd8a0 0x7fcb40154180 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400aa30 tx=0x7fcb340046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 -- 172.21.15.19:0/1032905030 shutdown_connections 2024-04-25T10:00:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcb2c06b4d0 0x7fcb2c06d990 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb40150b90 0x7fcb40156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400cd8a0 0x7fcb40154180 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 --2- 172.21.15.19:0/1032905030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb400cd4c0 0x7fcb401e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 -- 172.21.15.19:0/1032905030 >> 172.21.15.19:0/1032905030 conn(0x7fcb4006abe0 msgr2=0x7fcb400adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 -- 172.21.15.19:0/1032905030 shutdown_connections 2024-04-25T10:00:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:17.677+0000 7fcb4816a640 1 -- 172.21.15.19:0/1032905030 wait complete. 2024-04-25T10:00:17.691 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:00:17.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:17.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:18.331 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:00:18.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:18.331 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:00:18.346 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:00:18.346 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:18.363 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:00:18.363 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:00:18.363 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:18.402 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:00:18.403 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:00:18.403 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:18.419 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:00:18.419 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:18.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.539+0000 7fdb2a6e1640 1 Processor -- start 2024-04-25T10:00:18.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.539+0000 7fdb2a6e1640 1 -- start start 2024-04-25T10:00:18.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.540+0000 7fdb2a6e1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb240b6cb0 0x7fdb240b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.540+0000 7fdb2a6e1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb240ceee0 con 0x7fdb2415db70 2024-04-25T10:00:18.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.540+0000 7fdb2a6e1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb240d1d10 con 0x7fdb240b5a60 2024-04-25T10:00:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.540+0000 7fdb2a6e1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb240d1ff0 con 0x7fdb240b6cb0 2024-04-25T10:00:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.540+0000 7fdb237fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb240b6cb0 0x7fdb240b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.540+0000 7fdb23fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdb2415db70 0x7fdb24162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43962/0 (socket says 172.21.15.19:43962) 2024-04-25T10:00:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.540+0000 7fdb237fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb240b6cb0 0x7fdb240b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32978/0 (socket says 172.21.15.19:32978) 2024-04-25T10:00:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.540+0000 7fdb23fff640 1 -- 172.21.15.19:0/2984127026 learned_addr learned my addr 172.21.15.19:0/2984127026 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.540+0000 7fdb237fe640 1 -- 172.21.15.19:0/2984127026 >> v1:172.21.15.112:6789/0 conn(0x7fdb240b5a60 legacy=0x7fdb240d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.540+0000 7fdb237fe640 1 -- 172.21.15.19:0/2984127026 >> v1:172.21.15.19:6789/0 conn(0x7fdb2415db70 legacy=0x7fdb24162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.540+0000 7fdb237fe640 1 -- 172.21.15.19:0/2984127026 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb240d22d0 con 0x7fdb240b6cb0 2024-04-25T10:00:18.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.541+0000 7fdb237fe640 1 --2- 172.21.15.19:0/2984127026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb240b6cb0 0x7fdb240b5520 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0097a0 tx=0x7fdb0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f50f817dc817f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.541+0000 7fdb22ffd640 1 -- 172.21.15.19:0/2984127026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c037070 con 0x7fdb240b6cb0 2024-04-25T10:00:18.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.541+0000 7fdb22ffd640 1 -- 172.21.15.19:0/2984127026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c032440 con 0x7fdb240b6cb0 2024-04-25T10:00:18.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.541+0000 7fdb22ffd640 1 -- 172.21.15.19:0/2984127026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c032810 con 0x7fdb240b6cb0 2024-04-25T10:00:18.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.542+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/2984127026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb240b6cb0 msgr2=0x7fdb240b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.542+0000 7fdb2a6e1640 1 --2- 172.21.15.19:0/2984127026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb240b6cb0 0x7fdb240b5520 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0097a0 tx=0x7fdb0c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.542+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/2984127026 shutdown_connections 2024-04-25T10:00:18.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.542+0000 7fdb2a6e1640 1 --2- 172.21.15.19:0/2984127026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb240b6cb0 0x7fdb240b5520 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.542+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/2984127026 >> 172.21.15.19:0/2984127026 conn(0x7fdb240d0520 msgr2=0x7fdb240d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:18.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.542+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/2984127026 shutdown_connections 2024-04-25T10:00:18.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.542+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/2984127026 wait complete. 2024-04-25T10:00:18.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.543+0000 7fdb2a6e1640 1 Processor -- start 2024-04-25T10:00:18.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.543+0000 7fdb2a6e1640 1 -- start start 2024-04-25T10:00:18.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb2a6e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb240b5a60 0x7fdb240cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb2a6e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb240b6cb0 0x7fdb240ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb2a6e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb2415db70 0x7fdb240cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb2a6e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb240cf1c0 con 0x7fdb240b6cb0 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb2a6e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb240ce530 con 0x7fdb240b5a60 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb2a6e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb240cf340 con 0x7fdb2415db70 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb240b6cb0 0x7fdb240ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb240b6cb0 0x7fdb240ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32984/0 (socket says 172.21.15.19:32984) 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb237fe640 1 -- 172.21.15.19:0/3618124122 learned_addr learned my addr 172.21.15.19:0/3618124122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb23fff640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb240b5a60 0x7fdb240cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb28c7e640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb2415db70 0x7fdb240cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb237fe640 1 -- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb2415db70 msgr2=0x7fdb240cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb237fe640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb2415db70 0x7fdb240cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb237fe640 1 -- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb240b5a60 msgr2=0x7fdb240cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb237fe640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb240b5a60 0x7fdb240cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb237fe640 1 -- 172.21.15.19:0/3618124122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb240ce2e0 con 0x7fdb240b6cb0 2024-04-25T10:00:18.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb237fe640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb240b6cb0 0x7fdb240ccf10 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c035040 tx=0x7fdb0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.544+0000 7fdb20ff9640 1 -- 172.21.15.19:0/3618124122 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c037040 con 0x7fdb240b6cb0 2024-04-25T10:00:18.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.545+0000 7fdb20ff9640 1 -- 172.21.15.19:0/3618124122 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c030770 con 0x7fdb240b6cb0 2024-04-25T10:00:18.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.545+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/3618124122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb240cda40 con 0x7fdb240b6cb0 2024-04-25T10:00:18.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.545+0000 7fdb20ff9640 1 -- 172.21.15.19:0/3618124122 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c030b80 con 0x7fdb240b6cb0 2024-04-25T10:00:18.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.545+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/3618124122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb240ce000 con 0x7fdb240b6cb0 2024-04-25T10:00:18.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.546+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/3618124122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb24068410 con 0x7fdb240b6cb0 2024-04-25T10:00:18.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.546+0000 7fdb20ff9640 1 -- 172.21.15.19:0/3618124122 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb0c044020 con 0x7fdb240b6cb0 2024-04-25T10:00:18.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.546+0000 7fdb20ff9640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb0406b4d0 0x7fdb0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.546+0000 7fdb20ff9640 1 -- 172.21.15.19:0/3618124122 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdb0c0aa810 con 0x7fdb240b6cb0 2024-04-25T10:00:18.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.547+0000 7fdb23fff640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb0406b4d0 0x7fdb0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.547+0000 7fdb23fff640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb0406b4d0 0x7fdb0406d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdb14005880 tx=0x7fdb14009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.551+0000 7fdb20ff9640 1 -- 172.21.15.19:0/3618124122 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb24068410 con 0x7fdb240b6cb0 2024-04-25T10:00:18.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.773+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/3618124122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb24073990 con 0x7fdb240b6cb0 2024-04-25T10:00:18.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.774+0000 7fdb20ff9640 1 -- 172.21.15.19:0/3618124122 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fdb0c032e20 con 0x7fdb240b6cb0 2024-04-25T10:00:18.778 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:18.778 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:18.778 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:18.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb0406b4d0 msgr2=0x7fdb0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb0406b4d0 0x7fdb0406d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdb14005880 tx=0x7fdb14009040 comp rx=0 tx=0).stop 2024-04-25T10:00:18.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/3618124122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb240b6cb0 msgr2=0x7fdb240ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb240b6cb0 0x7fdb240ccf10 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c035040 tx=0x7fdb0c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/3618124122 shutdown_connections 2024-04-25T10:00:18.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdb0406b4d0 0x7fdb0406d990 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb2415db70 0x7fdb240cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb240b6cb0 0x7fdb240ccf10 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 --2- 172.21.15.19:0/3618124122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb240b5a60 0x7fdb240cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/3618124122 >> 172.21.15.19:0/3618124122 conn(0x7fdb240d0520 msgr2=0x7fdb2406c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:18.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/3618124122 shutdown_connections 2024-04-25T10:00:18.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.776+0000 7fdb2a6e1640 1 -- 172.21.15.19:0/3618124122 wait complete. 2024-04-25T10:00:18.791 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:18.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.888+0000 7f68632b3640 1 Processor -- start 2024-04-25T10:00:18.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.888+0000 7f68632b3640 1 -- start start 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.889+0000 7f68632b3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f685c15db70 0x7f685c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.889+0000 7f68632b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685c0b6cb0 0x7f685c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.889+0000 7f68632b3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c162690 con 0x7f685c0b5a60 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.889+0000 7f68632b3640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0d3800 con 0x7f685c15db70 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.889+0000 7f68632b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0ceee0 con 0x7f685c0b6cb0 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.889+0000 7f686104f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f685c15db70 0x7f685c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.889+0000 7f686104f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f685c15db70 0x7f685c162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32986/0 (socket says 172.21.15.19:32986) 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.889+0000 7f686104f640 1 -- 172.21.15.19:0/3165750742 learned_addr learned my addr 172.21.15.19:0/3165750742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.889+0000 7f686084e640 1 --2- 172.21.15.19:0/3165750742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685c0b6cb0 0x7f685c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.889+0000 7f686104f640 1 -- 172.21.15.19:0/3165750742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685c0b6cb0 msgr2=0x7f685c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.889+0000 7f686104f640 1 --2- 172.21.15.19:0/3165750742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685c0b6cb0 0x7f685c0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.890+0000 7f686104f640 1 -- 172.21.15.19:0/3165750742 >> v1:172.21.15.112:6789/0 conn(0x7f685c0b5a60 legacy=0x7f685c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.890+0000 7f686104f640 1 -- 172.21.15.19:0/3165750742 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0d1d10 con 0x7f685c15db70 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.890+0000 7f686104f640 1 --2- 172.21.15.19:0/3165750742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685c15db70 0x7f685c162150 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f6858009e50 tx=0x7f685802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e25ca274d5f68f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.890+0000 7f6853fff640 1 -- 172.21.15.19:0/3165750742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858036070 con 0x7f685c15db70 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.890+0000 7f6853fff640 1 -- 172.21.15.19:0/3165750742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6858031600 con 0x7f685c15db70 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.890+0000 7f6853fff640 1 -- 172.21.15.19:0/3165750742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68580319d0 con 0x7f685c15db70 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.891+0000 7f68632b3640 1 -- 172.21.15.19:0/3165750742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685c15db70 msgr2=0x7f685c162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.891+0000 7f68632b3640 1 --2- 172.21.15.19:0/3165750742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685c15db70 0x7f685c162150 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f6858009e50 tx=0x7f685802a710 comp rx=0 tx=0).stop 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.891+0000 7f68632b3640 1 -- 172.21.15.19:0/3165750742 shutdown_connections 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.891+0000 7f68632b3640 1 --2- 172.21.15.19:0/3165750742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685c0b6cb0 0x7f685c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.891+0000 7f68632b3640 1 --2- 172.21.15.19:0/3165750742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685c15db70 0x7f685c162150 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.891+0000 7f68632b3640 1 -- 172.21.15.19:0/3165750742 >> 172.21.15.19:0/3165750742 conn(0x7f685c0d0520 msgr2=0x7f685c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.891+0000 7f68632b3640 1 -- 172.21.15.19:0/3165750742 shutdown_connections 2024-04-25T10:00:18.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.891+0000 7f68632b3640 1 -- 172.21.15.19:0/3165750742 wait complete. 2024-04-25T10:00:18.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.892+0000 7f68632b3640 1 Processor -- start 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.892+0000 7f68632b3640 1 -- start start 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f68632b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685c0b5a60 0x7f685c0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f68632b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685c0b6cb0 0x7f685c158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f68632b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685c15db70 0x7f685c0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f68632b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c15df50 con 0x7f685c15db70 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f68632b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0d22d0 con 0x7f685c0b5a60 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f68632b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0ce530 con 0x7f685c0b6cb0 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f6861850640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685c15db70 0x7f685c0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f6861850640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685c15db70 0x7f685c0cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32996/0 (socket says 172.21.15.19:32996) 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f6861850640 1 -- 172.21.15.19:0/2906045595 learned_addr learned my addr 172.21.15.19:0/2906045595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f686084e640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685c0b6cb0 0x7f685c158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f686104f640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685c0b5a60 0x7f685c0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f6861850640 1 -- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685c0b6cb0 msgr2=0x7f685c158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f6861850640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685c0b6cb0 0x7f685c158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f6861850640 1 -- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685c0b5a60 msgr2=0x7f685c0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.893+0000 7f6861850640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685c0b5a60 0x7f685c0cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.894+0000 7f6861850640 1 -- 172.21.15.19:0/2906045595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0cbfa0 con 0x7f685c15db70 2024-04-25T10:00:18.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.894+0000 7f6861850640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685c15db70 0x7f685c0cba60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f6854001eb0 tx=0x7f685400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.894+0000 7f6851ffb640 1 -- 172.21.15.19:0/2906045595 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854017070 con 0x7f685c15db70 2024-04-25T10:00:18.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.894+0000 7f6851ffb640 1 -- 172.21.15.19:0/2906045595 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68540052d0 con 0x7f685c15db70 2024-04-25T10:00:18.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.894+0000 7f6851ffb640 1 -- 172.21.15.19:0/2906045595 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854003810 con 0x7f685c15db70 2024-04-25T10:00:18.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.894+0000 7f68632b3640 1 -- 172.21.15.19:0/2906045595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c0cca20 con 0x7f685c15db70 2024-04-25T10:00:18.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.894+0000 7f68632b3640 1 -- 172.21.15.19:0/2906045595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c0cd2c0 con 0x7f685c15db70 2024-04-25T10:00:18.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.895+0000 7f68632b3640 1 -- 172.21.15.19:0/2906045595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c073990 con 0x7f685c15db70 2024-04-25T10:00:18.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.895+0000 7f6851ffb640 1 -- 172.21.15.19:0/2906045595 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f685401a070 con 0x7f685c15db70 2024-04-25T10:00:18.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.896+0000 7f6851ffb640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f684006b4d0 0x7f684006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.896+0000 7f6851ffb640 1 -- 172.21.15.19:0/2906045595 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6854089680 con 0x7f685c15db70 2024-04-25T10:00:18.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.896+0000 7f686104f640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f684006b4d0 0x7f684006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.896+0000 7f686104f640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f684006b4d0 0x7f684006d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6858033000 tx=0x7f6858009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:18.900+0000 7f6851ffb640 1 -- 172.21.15.19:0/2906045595 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f685c0cbfa0 con 0x7f685c15db70 2024-04-25T10:00:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.081+0000 7f68632b3640 1 -- 172.21.15.19:0/2906045595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f685c0d1820 con 0x7f685c15db70 2024-04-25T10:00:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.082+0000 7f6851ffb640 1 -- 172.21.15.19:0/2906045595 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f6854057d20 con 0x7f685c15db70 2024-04-25T10:00:19.086 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:19.087 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":106,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:48.558694+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:45.201424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:45.885235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 -- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f684006b4d0 msgr2=0x7f684006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f684006b4d0 0x7f684006d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6858033000 tx=0x7f6858009000 comp rx=0 tx=0).stop 2024-04-25T10:00:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 -- 172.21.15.19:0/2906045595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685c15db70 msgr2=0x7f685c0cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685c15db70 0x7f685c0cba60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f6854001eb0 tx=0x7f685400c890 comp rx=0 tx=0).stop 2024-04-25T10:00:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 -- 172.21.15.19:0/2906045595 shutdown_connections 2024-04-25T10:00:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f684006b4d0 0x7f684006d990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685c15db70 0x7f685c0cba60 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685c0b6cb0 0x7f685c158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 --2- 172.21.15.19:0/2906045595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685c0b5a60 0x7f685c0cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 -- 172.21.15.19:0/2906045595 >> 172.21.15.19:0/2906045595 conn(0x7f685c0d0520 msgr2=0x7f685c06c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 -- 172.21.15.19:0/2906045595 shutdown_connections 2024-04-25T10:00:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.084+0000 7f68632b3640 1 -- 172.21.15.19:0/2906045595 wait complete. 2024-04-25T10:00:19.099 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:59:45.201424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:59:45.885235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:00:19.099 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:00:19.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.200+0000 7fd298f9e640 1 Processor -- start 2024-04-25T10:00:19.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.200+0000 7fd298f9e640 1 -- start start 2024-04-25T10:00:19.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.200+0000 7fd298f9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2940cecb0 0x7fd294168970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.201+0000 7fd298f9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29406c880 con 0x7fd29415d9c0 2024-04-25T10:00:19.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.201+0000 7fd298f9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29406cb60 con 0x7fd2940b9260 2024-04-25T10:00:19.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.201+0000 7fd298f9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940d1a30 con 0x7fd2940cecb0 2024-04-25T10:00:19.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.201+0000 7fd2937fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd29415d9c0 0x7fd29406c030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43974/0 (socket says 172.21.15.19:43974) 2024-04-25T10:00:19.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.201+0000 7fd293fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2940cecb0 0x7fd294168970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.201+0000 7fd2937fe640 1 -- 172.21.15.19:0/2241500418 learned_addr learned my addr 172.21.15.19:0/2241500418 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:19.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.201+0000 7fd2927fc640 1 -- 172.21.15.19:0/2241500418 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432278328 0 0) 0x7fd29406c880 con 0x7fd29415d9c0 2024-04-25T10:00:19.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.201+0000 7fd2927fc640 1 -- 172.21.15.19:0/2241500418 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd27c003520 con 0x7fd29415d9c0 2024-04-25T10:00:19.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.201+0000 7fd293fff640 1 -- 172.21.15.19:0/2241500418 >> v1:172.21.15.112:6789/0 conn(0x7fd2940b9260 legacy=0x7fd2940ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.201+0000 7fd293fff640 1 -- 172.21.15.19:0/2241500418 >> v1:172.21.15.19:6789/0 conn(0x7fd29415d9c0 legacy=0x7fd29406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.201+0000 7fd293fff640 1 -- 172.21.15.19:0/2241500418 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29406ce40 con 0x7fd2940cecb0 2024-04-25T10:00:19.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.202+0000 7fd293fff640 1 --2- 172.21.15.19:0/2241500418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2940cecb0 0x7fd294168970 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd2940d1bb0 tx=0x7fd28c02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ea09b940a21ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.202+0000 7fd2927fc640 1 -- 172.21.15.19:0/2241500418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c037070 con 0x7fd2940cecb0 2024-04-25T10:00:19.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.202+0000 7fd2927fc640 1 -- 172.21.15.19:0/2241500418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c007740 con 0x7fd2940cecb0 2024-04-25T10:00:19.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.202+0000 7fd2927fc640 1 -- 172.21.15.19:0/2241500418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c007b10 con 0x7fd2940cecb0 2024-04-25T10:00:19.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.202+0000 7fd298f9e640 1 -- 172.21.15.19:0/2241500418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2940cecb0 msgr2=0x7fd294168970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.202+0000 7fd298f9e640 1 --2- 172.21.15.19:0/2241500418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2940cecb0 0x7fd294168970 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd2940d1bb0 tx=0x7fd28c02a780 comp rx=0 tx=0).stop 2024-04-25T10:00:19.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.202+0000 7fd298f9e640 1 -- 172.21.15.19:0/2241500418 shutdown_connections 2024-04-25T10:00:19.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.202+0000 7fd298f9e640 1 --2- 172.21.15.19:0/2241500418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2940cecb0 0x7fd294168970 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.202+0000 7fd298f9e640 1 -- 172.21.15.19:0/2241500418 >> 172.21.15.19:0/2241500418 conn(0x7fd2940d0770 msgr2=0x7fd2940d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.202+0000 7fd298f9e640 1 -- 172.21.15.19:0/2241500418 shutdown_connections 2024-04-25T10:00:19.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.202+0000 7fd298f9e640 1 -- 172.21.15.19:0/2241500418 wait complete. 2024-04-25T10:00:19.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.204+0000 7fd298f9e640 1 Processor -- start 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.204+0000 7fd298f9e640 1 -- start start 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd298f9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2940b9260 0x7fd2940aa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd298f9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940cecb0 0x7fd2940c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd298f9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29415d9c0 0x7fd2940c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd298f9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29415df20 con 0x7fd29415d9c0 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd298f9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29415dda0 con 0x7fd2940cecb0 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd298f9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940d31b0 con 0x7fd2940b9260 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd293fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29415d9c0 0x7fd2940c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd293fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29415d9c0 0x7fd2940c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33022/0 (socket says 172.21.15.19:33022) 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd293fff640 1 -- 172.21.15.19:0/2168013011 learned_addr learned my addr 172.21.15.19:0/2168013011 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd2937fe640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2940b9260 0x7fd2940aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd292ffd640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940cecb0 0x7fd2940c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd293fff640 1 -- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2940b9260 msgr2=0x7fd2940aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd293fff640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2940b9260 0x7fd2940aa320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd293fff640 1 -- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940cecb0 msgr2=0x7fd2940c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd293fff640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940cecb0 0x7fd2940c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd293fff640 1 -- 172.21.15.19:0/2168013011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2941610d0 con 0x7fd29415d9c0 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd292ffd640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940cecb0 0x7fd2940c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd293fff640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29415d9c0 0x7fd2940c9aa0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd28c02ad70 tx=0x7fd28c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.205+0000 7fd27bfff640 1 -- 172.21.15.19:0/2168013011 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c042020 con 0x7fd29415d9c0 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.206+0000 7fd27bfff640 1 -- 172.21.15.19:0/2168013011 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c0327f0 con 0x7fd29415d9c0 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.206+0000 7fd298f9e640 1 -- 172.21.15.19:0/2168013011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd294160df0 con 0x7fd29415d9c0 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.206+0000 7fd27bfff640 1 -- 172.21.15.19:0/2168013011 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c037040 con 0x7fd29415d9c0 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.206+0000 7fd298f9e640 1 -- 172.21.15.19:0/2168013011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29414ce00 con 0x7fd29415d9c0 2024-04-25T10:00:19.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.207+0000 7fd298f9e640 1 -- 172.21.15.19:0/2168013011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd294073990 con 0x7fd29415d9c0 2024-04-25T10:00:19.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.207+0000 7fd27bfff640 1 -- 172.21.15.19:0/2168013011 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd28c005d10 con 0x7fd29415d9c0 2024-04-25T10:00:19.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.207+0000 7fd27bfff640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd28006b4d0 0x7fd28006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.207+0000 7fd27bfff640 1 -- 172.21.15.19:0/2168013011 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd28c0ab760 con 0x7fd29415d9c0 2024-04-25T10:00:19.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.208+0000 7fd2937fe640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd28006b4d0 0x7fd28006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.208+0000 7fd2937fe640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd28006b4d0 0x7fd28006d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd2840011e0 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.212+0000 7fd27bfff640 1 -- 172.21.15.19:0/2168013011 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd28c04c030 con 0x7fd29415d9c0 2024-04-25T10:00:19.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.391+0000 7fd298f9e640 1 -- 172.21.15.19:0/2168013011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd2940d15e0 con 0x7fd29415d9c0 2024-04-25T10:00:19.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.391+0000 7fd27bfff640 1 -- 172.21.15.19:0/2168013011 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd28c07a450 con 0x7fd29415d9c0 2024-04-25T10:00:19.396 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:00:19.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 -- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd28006b4d0 msgr2=0x7fd28006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd28006b4d0 0x7fd28006d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd2840011e0 tx=0x7fd284009040 comp rx=0 tx=0).stop 2024-04-25T10:00:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 -- 172.21.15.19:0/2168013011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29415d9c0 msgr2=0x7fd2940c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29415d9c0 0x7fd2940c9aa0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd28c02ad70 tx=0x7fd28c004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 -- 172.21.15.19:0/2168013011 shutdown_connections 2024-04-25T10:00:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd28006b4d0 0x7fd28006d990 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd29415d9c0 0x7fd2940c9aa0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940cecb0 0x7fd2940c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 --2- 172.21.15.19:0/2168013011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2940b9260 0x7fd2940aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 -- 172.21.15.19:0/2168013011 >> 172.21.15.19:0/2168013011 conn(0x7fd2940d0770 msgr2=0x7fd2940b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 -- 172.21.15.19:0/2168013011 shutdown_connections 2024-04-25T10:00:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.393+0000 7fd298f9e640 1 -- 172.21.15.19:0/2168013011 wait complete. 2024-04-25T10:00:19.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:00:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.508+0000 7ff91dce1640 1 Processor -- start 2024-04-25T10:00:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.508+0000 7ff91dce1640 1 -- start start 2024-04-25T10:00:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff91dce1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff91815b7d0 0x7ff9180cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff91dce1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff91806c9e0 0x7ff9180b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff91dce1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91806cf50 con 0x7ff9180b5620 2024-04-25T10:00:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff91dce1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9180ce530 con 0x7ff91815b7d0 2024-04-25T10:00:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff91dce1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180ce810 con 0x7ff91806c9e0 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff9177fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff91815b7d0 0x7ff9180cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff9177fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff91815b7d0 0x7ff9180cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33038/0 (socket says 172.21.15.19:33038) 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff9177fe640 1 -- 172.21.15.19:0/569013877 learned_addr learned my addr 172.21.15.19:0/569013877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff9177fe640 1 -- 172.21.15.19:0/569013877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff91806c9e0 msgr2=0x7ff9180b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff917fff640 1 --2- 172.21.15.19:0/569013877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff91806c9e0 0x7ff9180b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff9177fe640 1 --2- 172.21.15.19:0/569013877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff91806c9e0 0x7ff9180b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff9177fe640 1 -- 172.21.15.19:0/569013877 >> v1:172.21.15.112:6789/0 conn(0x7ff9180b5620 legacy=0x7ff91806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff9177fe640 1 -- 172.21.15.19:0/569013877 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9180ceaf0 con 0x7ff91815b7d0 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.509+0000 7ff917fff640 1 --2- 172.21.15.19:0/569013877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff91806c9e0 0x7ff9180b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.510+0000 7ff9177fe640 1 --2- 172.21.15.19:0/569013877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff91815b7d0 0x7ff9180cf010 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7ff91808c0d0 tx=0x7ff90802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f111e682d8257448 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.510+0000 7ff9167fc640 1 -- 172.21.15.19:0/569013877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908037070 con 0x7ff91815b7d0 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.510+0000 7ff9167fc640 1 -- 172.21.15.19:0/569013877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff908032440 con 0x7ff91815b7d0 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.510+0000 7ff9167fc640 1 -- 172.21.15.19:0/569013877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908032810 con 0x7ff91815b7d0 2024-04-25T10:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.511+0000 7ff91dce1640 1 -- 172.21.15.19:0/569013877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff91815b7d0 msgr2=0x7ff9180cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.511+0000 7ff91dce1640 1 --2- 172.21.15.19:0/569013877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff91815b7d0 0x7ff9180cf010 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7ff91808c0d0 tx=0x7ff90802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.511+0000 7ff91dce1640 1 -- 172.21.15.19:0/569013877 shutdown_connections 2024-04-25T10:00:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.511+0000 7ff91dce1640 1 --2- 172.21.15.19:0/569013877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff91806c9e0 0x7ff9180b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.511+0000 7ff91dce1640 1 --2- 172.21.15.19:0/569013877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff91815b7d0 0x7ff9180cf010 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.511+0000 7ff91dce1640 1 -- 172.21.15.19:0/569013877 >> 172.21.15.19:0/569013877 conn(0x7ff9180d0800 msgr2=0x7ff9180d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.511+0000 7ff91dce1640 1 -- 172.21.15.19:0/569013877 shutdown_connections 2024-04-25T10:00:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.511+0000 7ff91dce1640 1 -- 172.21.15.19:0/569013877 wait complete. 2024-04-25T10:00:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.512+0000 7ff91dce1640 1 Processor -- start 2024-04-25T10:00:19.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.512+0000 7ff91dce1640 1 -- start start 2024-04-25T10:00:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff91dce1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff91806c9e0 0x7ff918158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff91dce1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9180b5620 0x7ff9180c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff91dce1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff91815b7d0 0x7ff91815d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff91dce1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180cf6d0 con 0x7ff9180b5620 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff91dce1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180cf550 con 0x7ff91815b7d0 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff91dce1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91815bbb0 con 0x7ff91806c9e0 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff916ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9180b5620 0x7ff9180c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff916ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9180b5620 0x7ff9180c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33052/0 (socket says 172.21.15.19:33052) 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff916ffd640 1 -- 172.21.15.19:0/3043278965 learned_addr learned my addr 172.21.15.19:0/3043278965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff9177fe640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff91806c9e0 0x7ff918158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff917fff640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff91815b7d0 0x7ff91815d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff916ffd640 1 -- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff91806c9e0 msgr2=0x7ff918158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff916ffd640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff91806c9e0 0x7ff918158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff916ffd640 1 -- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff91815b7d0 msgr2=0x7ff91815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff916ffd640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff91815b7d0 0x7ff91815d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.513+0000 7ff916ffd640 1 -- 172.21.15.19:0/3043278965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9181585c0 con 0x7ff9180b5620 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.514+0000 7ff916ffd640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9180b5620 0x7ff9180c9e50 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c460 tx=0x7ff90c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.514+0000 7ff8fffff640 1 -- 172.21.15.19:0/3043278965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c016020 con 0x7ff9180b5620 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.514+0000 7ff8fffff640 1 -- 172.21.15.19:0/3043278965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c012870 con 0x7ff9180b5620 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.514+0000 7ff91dce1640 1 -- 172.21.15.19:0/3043278965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9180cb2a0 con 0x7ff9180b5620 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.514+0000 7ff8fffff640 1 -- 172.21.15.19:0/3043278965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c012c60 con 0x7ff9180b5620 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.514+0000 7ff91dce1640 1 -- 172.21.15.19:0/3043278965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9180cb860 con 0x7ff9180b5620 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.515+0000 7ff91dce1640 1 -- 172.21.15.19:0/3043278965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff918073a00 con 0x7ff9180b5620 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.515+0000 7ff8fffff640 1 -- 172.21.15.19:0/3043278965 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff90c019020 con 0x7ff9180b5620 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.516+0000 7ff8fffff640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff90406b4d0 0x7ff90406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.516+0000 7ff8fffff640 1 -- 172.21.15.19:0/3043278965 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff90c089e10 con 0x7ff9180b5620 2024-04-25T10:00:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.516+0000 7ff9177fe640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff90406b4d0 0x7ff90406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.517+0000 7ff9177fe640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff90406b4d0 0x7ff90406d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff908034000 tx=0x7ff90802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.520+0000 7ff8fffff640 1 -- 172.21.15.19:0/3043278965 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff90c020070 con 0x7ff9180b5620 2024-04-25T10:00:19.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.706+0000 7ff91dce1640 1 -- 172.21.15.19:0/3043278965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9180d35f0 con 0x7ff9180b5620 2024-04-25T10:00:19.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.707+0000 7ff8fffff640 1 -- 172.21.15.19:0/3043278965 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff90c058b00 con 0x7ff9180b5620 2024-04-25T10:00:19.711 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:00:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 -- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff90406b4d0 msgr2=0x7ff90406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff90406b4d0 0x7ff90406d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff908034000 tx=0x7ff90802e000 comp rx=0 tx=0).stop 2024-04-25T10:00:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 -- 172.21.15.19:0/3043278965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9180b5620 msgr2=0x7ff9180c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9180b5620 0x7ff9180c9e50 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c460 tx=0x7ff90c00c920 comp rx=0 tx=0).stop 2024-04-25T10:00:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 -- 172.21.15.19:0/3043278965 shutdown_connections 2024-04-25T10:00:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff90406b4d0 0x7ff90406d990 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff91815b7d0 0x7ff91815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9180b5620 0x7ff9180c9e50 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 --2- 172.21.15.19:0/3043278965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff91806c9e0 0x7ff918158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 -- 172.21.15.19:0/3043278965 >> 172.21.15.19:0/3043278965 conn(0x7ff9180d0800 msgr2=0x7ff9181589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 -- 172.21.15.19:0/3043278965 shutdown_connections 2024-04-25T10:00:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.709+0000 7ff91dce1640 1 -- 172.21.15.19:0/3043278965 wait complete. 2024-04-25T10:00:19.724 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:00:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.824+0000 7f0ca67ab640 1 Processor -- start 2024-04-25T10:00:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.824+0000 7f0ca67ab640 1 -- start start 2024-04-25T10:00:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0ca67ab640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0ca015dc20 0x7f0ca00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0ca67ab640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ca0161710 0x7f0ca00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0ca67ab640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0ca00b9330 0x7f0ca006bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0ca67ab640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca015e000 con 0x7f0ca015dc20 2024-04-25T10:00:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0ca67ab640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca00d1de0 con 0x7f0ca00b9330 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0ca67ab640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca0161c60 con 0x7f0ca0161710 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0c9ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0ca015dc20 0x7f0ca00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0c9ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0ca015dc20 0x7f0ca00ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33068/0 (socket says 172.21.15.19:33068) 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0c9ffff640 1 -- 172.21.15.19:0/97863827 learned_addr learned my addr 172.21.15.19:0/97863827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0ca4d48640 1 --2- 172.21.15.19:0/97863827 >> v2:172.21.15.112:3300/0 conn(0x7f0ca00b9330 0x7f0ca006bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0c9f7fe640 1 --2- 172.21.15.19:0/97863827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ca0161710 0x7f0ca00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0c9ffff640 1 -- 172.21.15.19:0/97863827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ca0161710 msgr2=0x7f0ca00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0c9ffff640 1 --2- 172.21.15.19:0/97863827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ca0161710 0x7f0ca00cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0c9ffff640 1 -- 172.21.15.19:0/97863827 >> v2:172.21.15.112:3300/0 conn(0x7f0ca00b9330 msgr2=0x7f0ca006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0c9ffff640 1 --2- 172.21.15.19:0/97863827 >> v2:172.21.15.112:3300/0 conn(0x7f0ca00b9330 0x7f0ca006bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.825+0000 7f0c9ffff640 1 -- 172.21.15.19:0/97863827 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca0161f40 con 0x7f0ca015dc20 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.826+0000 7f0c9ffff640 1 --2- 172.21.15.19:0/97863827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca015dc20 0x7f0ca00ce670 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00b6570 tx=0x7f0c8802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35369dfd54a31da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.826+0000 7f0c9effd640 1 -- 172.21.15.19:0/97863827 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88037070 con 0x7f0ca015dc20 2024-04-25T10:00:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.826+0000 7f0c9effd640 1 -- 172.21.15.19:0/97863827 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c88032440 con 0x7f0ca015dc20 2024-04-25T10:00:19.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.826+0000 7f0c9effd640 1 -- 172.21.15.19:0/97863827 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88032810 con 0x7f0ca015dc20 2024-04-25T10:00:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.827+0000 7f0ca67ab640 1 -- 172.21.15.19:0/97863827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca015dc20 msgr2=0x7f0ca00ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.827+0000 7f0ca67ab640 1 --2- 172.21.15.19:0/97863827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca015dc20 0x7f0ca00ce670 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00b6570 tx=0x7f0c8802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.827+0000 7f0ca67ab640 1 -- 172.21.15.19:0/97863827 shutdown_connections 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.827+0000 7f0ca67ab640 1 --2- 172.21.15.19:0/97863827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ca0161710 0x7f0ca00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.827+0000 7f0ca67ab640 1 --2- 172.21.15.19:0/97863827 >> v2:172.21.15.112:3300/0 conn(0x7f0ca00b9330 0x7f0ca006bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.827+0000 7f0ca67ab640 1 --2- 172.21.15.19:0/97863827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca015dc20 0x7f0ca00ce670 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.827+0000 7f0ca67ab640 1 -- 172.21.15.19:0/97863827 >> 172.21.15.19:0/97863827 conn(0x7f0ca00d0420 msgr2=0x7f0ca00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.827+0000 7f0ca67ab640 1 -- 172.21.15.19:0/97863827 shutdown_connections 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.827+0000 7f0ca67ab640 1 -- 172.21.15.19:0/97863827 wait complete. 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.828+0000 7f0ca67ab640 1 Processor -- start 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.828+0000 7f0ca67ab640 1 -- start start 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0ca67ab640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca00b9330 0x7f0ca01599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0ca67ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca015dc20 0x7f0ca00d3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0ca67ab640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ca0161710 0x7f0ca00d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0ca67ab640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00b5e50 con 0x7f0ca00b9330 2024-04-25T10:00:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0ca67ab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca0162500 con 0x7f0ca015dc20 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0ca67ab640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00b5fd0 con 0x7f0ca0161710 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0c9ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca00b9330 0x7f0ca01599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0c9ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca00b9330 0x7f0ca01599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33080/0 (socket says 172.21.15.19:33080) 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0c9ffff640 1 -- 172.21.15.19:0/1666380043 learned_addr learned my addr 172.21.15.19:0/1666380043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0c9f7fe640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca015dc20 0x7f0ca00d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0ca4d48640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ca0161710 0x7f0ca00d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0c9ffff640 1 -- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ca0161710 msgr2=0x7f0ca00d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0c9ffff640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ca0161710 0x7f0ca00d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0c9ffff640 1 -- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca015dc20 msgr2=0x7f0ca00d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0c9ffff640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca015dc20 0x7f0ca00d3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0c9ffff640 1 -- 172.21.15.19:0/1666380043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca00ab530 con 0x7f0ca00b9330 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.829+0000 7f0c9f7fe640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca015dc20 0x7f0ca00d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.830+0000 7f0c9ffff640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca00b9330 0x7f0ca01599a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f0c88000c00 tx=0x7f0c880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.830+0000 7f0c9cff9640 1 -- 172.21.15.19:0/1666380043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88041070 con 0x7f0ca00b9330 2024-04-25T10:00:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.830+0000 7f0c9cff9640 1 -- 172.21.15.19:0/1666380043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8803c870 con 0x7f0ca00b9330 2024-04-25T10:00:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.830+0000 7f0ca67ab640 1 -- 172.21.15.19:0/1666380043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca00aaf70 con 0x7f0ca00b9330 2024-04-25T10:00:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.830+0000 7f0c9cff9640 1 -- 172.21.15.19:0/1666380043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88037040 con 0x7f0ca00b9330 2024-04-25T10:00:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.830+0000 7f0ca67ab640 1 -- 172.21.15.19:0/1666380043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca00cbc20 con 0x7f0ca00b9330 2024-04-25T10:00:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.831+0000 7f0ca67ab640 1 -- 172.21.15.19:0/1666380043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca0073a00 con 0x7f0ca00b9330 2024-04-25T10:00:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.831+0000 7f0c9cff9640 1 -- 172.21.15.19:0/1666380043 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c88044070 con 0x7f0ca00b9330 2024-04-25T10:00:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.832+0000 7f0c9cff9640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c8006b4d0 0x7f0c8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.832+0000 7f0c9cff9640 1 -- 172.21.15.19:0/1666380043 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c880ab1d0 con 0x7f0ca00b9330 2024-04-25T10:00:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.832+0000 7f0c9f7fe640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c8006b4d0 0x7f0c8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.833+0000 7f0c9f7fe640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c8006b4d0 0x7f0c8006d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0c90006fd0 tx=0x7f0c90008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:19.836+0000 7f0c9cff9640 1 -- 172.21.15.19:0/1666380043 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ca00ab530 con 0x7f0ca00b9330 2024-04-25T10:00:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.014+0000 7f0ca67ab640 1 -- 172.21.15.19:0/1666380043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ca00d16b0 con 0x7f0ca00b9330 2024-04-25T10:00:20.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.015+0000 7f0c9cff9640 1 -- 172.21.15.19:0/1666380043 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0c88079fd0 con 0x7f0ca00b9330 2024-04-25T10:00:20.019 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:00:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.016+0000 7f0ca67ab640 1 -- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c8006b4d0 msgr2=0x7f0c8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.016+0000 7f0ca67ab640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c8006b4d0 0x7f0c8006d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0c90006fd0 tx=0x7f0c90008040 comp rx=0 tx=0).stop 2024-04-25T10:00:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.016+0000 7f0ca67ab640 1 -- 172.21.15.19:0/1666380043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca00b9330 msgr2=0x7f0ca01599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.016+0000 7f0ca67ab640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca00b9330 0x7f0ca01599a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f0c88000c00 tx=0x7f0c880046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.017+0000 7f0ca67ab640 1 -- 172.21.15.19:0/1666380043 shutdown_connections 2024-04-25T10:00:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.017+0000 7f0ca67ab640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c8006b4d0 0x7f0c8006d990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.017+0000 7f0ca67ab640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ca0161710 0x7f0ca00d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.017+0000 7f0ca67ab640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca015dc20 0x7f0ca00d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.017+0000 7f0ca67ab640 1 --2- 172.21.15.19:0/1666380043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca00b9330 0x7f0ca01599a0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.017+0000 7f0ca67ab640 1 -- 172.21.15.19:0/1666380043 >> 172.21.15.19:0/1666380043 conn(0x7f0ca00d0420 msgr2=0x7f0ca00b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.017+0000 7f0ca67ab640 1 -- 172.21.15.19:0/1666380043 shutdown_connections 2024-04-25T10:00:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.017+0000 7f0ca67ab640 1 -- 172.21.15.19:0/1666380043 wait complete. 2024-04-25T10:00:20.032 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:20.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.128+0000 7f01dee9c640 1 Processor -- start 2024-04-25T10:00:20.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.128+0000 7f01dee9c640 1 -- start start 2024-04-25T10:00:20.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.129+0000 7f01dee9c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d815dbe0 0x7f01d80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.129+0000 7f01dee9c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f01d80b2cb0 0x7f01d80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.129+0000 7f01dee9c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d80b97a0 con 0x7f01d80b92a0 2024-04-25T10:00:20.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.129+0000 7f01dee9c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d815dfc0 con 0x7f01d80b2cb0 2024-04-25T10:00:20.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.129+0000 7f01dee9c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80d1a00 con 0x7f01d815dbe0 2024-04-25T10:00:20.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.130+0000 7f01d7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f01d80b2cb0 0x7f01d80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.130+0000 7f01dcc38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d815dbe0 0x7f01d80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.130+0000 7f01dcc38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d815dbe0 0x7f01d80d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:45818/0 (socket says 172.21.15.19:45818) 2024-04-25T10:00:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.130+0000 7f01dcc38640 1 -- 172.21.15.19:0/2792868304 learned_addr learned my addr 172.21.15.19:0/2792868304 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.130+0000 7f01dcc38640 1 -- 172.21.15.19:0/2792868304 >> v2:172.21.15.112:3300/0 conn(0x7f01d80b2cb0 msgr2=0x7f01d80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.130+0000 7f01dcc38640 1 --2- 172.21.15.19:0/2792868304 >> v2:172.21.15.112:3300/0 conn(0x7f01d80b2cb0 0x7f01d80d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.130+0000 7f01dcc38640 1 -- 172.21.15.19:0/2792868304 >> v1:172.21.15.112:6789/0 conn(0x7f01d80b92a0 legacy=0x7f01d80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.130+0000 7f01dcc38640 1 -- 172.21.15.19:0/2792868304 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80d1c50 con 0x7f01d815dbe0 2024-04-25T10:00:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.130+0000 7f01d7fff640 1 --2- 172.21.15.19:0/2792868304 >> v2:172.21.15.112:3300/0 conn(0x7f01d80b2cb0 0x7f01d80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.131+0000 7f01dcc38640 1 --2- 172.21.15.19:0/2792868304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d815dbe0 0x7f01d80d0910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009e50 tx=0x7f01d002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7973e9ab6c4d770 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.131+0000 7f01d77fe640 1 -- 172.21.15.19:0/2792868304 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0036070 con 0x7f01d815dbe0 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.131+0000 7f01d77fe640 1 -- 172.21.15.19:0/2792868304 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d0031600 con 0x7f01d815dbe0 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.131+0000 7f01d77fe640 1 -- 172.21.15.19:0/2792868304 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00319d0 con 0x7f01d815dbe0 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.131+0000 7f01dee9c640 1 -- 172.21.15.19:0/2792868304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d815dbe0 msgr2=0x7f01d80d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.131+0000 7f01dee9c640 1 --2- 172.21.15.19:0/2792868304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d815dbe0 0x7f01d80d0910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009e50 tx=0x7f01d002a710 comp rx=0 tx=0).stop 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.132+0000 7f01dee9c640 1 -- 172.21.15.19:0/2792868304 shutdown_connections 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.132+0000 7f01dee9c640 1 --2- 172.21.15.19:0/2792868304 >> v2:172.21.15.112:3300/0 conn(0x7f01d80b2cb0 0x7f01d80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.132+0000 7f01dee9c640 1 --2- 172.21.15.19:0/2792868304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d815dbe0 0x7f01d80d0910 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.132+0000 7f01dee9c640 1 -- 172.21.15.19:0/2792868304 >> 172.21.15.19:0/2792868304 conn(0x7f01d8061a20 msgr2=0x7f01d8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.132+0000 7f01dee9c640 1 -- 172.21.15.19:0/2792868304 shutdown_connections 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.132+0000 7f01dee9c640 1 -- 172.21.15.19:0/2792868304 wait complete. 2024-04-25T10:00:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.133+0000 7f01dee9c640 1 Processor -- start 2024-04-25T10:00:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.133+0000 7f01dee9c640 1 -- start start 2024-04-25T10:00:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01dee9c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d80b2cb0 0x7f01d81f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01dee9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80b92a0 0x7f01d80b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01dee9c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01d815dbe0 0x7f01d815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01dee9c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80b6ab0 con 0x7f01d80b92a0 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01dee9c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80d2210 con 0x7f01d815dbe0 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01dee9c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80b6c30 con 0x7f01d80b2cb0 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01d7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80b92a0 0x7f01d80b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01d7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80b92a0 0x7f01d80b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33084/0 (socket says 172.21.15.19:33084) 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01d7fff640 1 -- 172.21.15.19:0/1293925160 learned_addr learned my addr 172.21.15.19:0/1293925160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01dcc38640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d80b2cb0 0x7f01d81f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01dd439640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01d815dbe0 0x7f01d815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01d7fff640 1 -- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d80b2cb0 msgr2=0x7f01d81f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01d7fff640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d80b2cb0 0x7f01d81f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01d7fff640 1 -- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01d815dbe0 msgr2=0x7f01d815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01d7fff640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01d815dbe0 0x7f01d815b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01d7fff640 1 -- 172.21.15.19:0/1293925160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80cba40 con 0x7f01d80b92a0 2024-04-25T10:00:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01dd439640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01d815dbe0 0x7f01d815b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01d7fff640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80b92a0 0x7f01d80b5b90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00ebe0 tx=0x7f01cc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01d57fa640 1 -- 172.21.15.19:0/1293925160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc017070 con 0x7f01d80b92a0 2024-04-25T10:00:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.134+0000 7f01d57fa640 1 -- 172.21.15.19:0/1293925160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01cc00cc30 con 0x7f01d80b92a0 2024-04-25T10:00:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.135+0000 7f01dee9c640 1 -- 172.21.15.19:0/1293925160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d80cbd00 con 0x7f01d80b92a0 2024-04-25T10:00:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.135+0000 7f01d57fa640 1 -- 172.21.15.19:0/1293925160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc013470 con 0x7f01d80b92a0 2024-04-25T10:00:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.135+0000 7f01dee9c640 1 -- 172.21.15.19:0/1293925160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d80cc2c0 con 0x7f01d80b92a0 2024-04-25T10:00:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.136+0000 7f01dee9c640 1 -- 172.21.15.19:0/1293925160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d8073990 con 0x7f01d80b92a0 2024-04-25T10:00:20.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.136+0000 7f01d57fa640 1 -- 172.21.15.19:0/1293925160 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01cc01a070 con 0x7f01d80b92a0 2024-04-25T10:00:20.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.137+0000 7f01d57fa640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f01c406b4d0 0x7f01c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.137+0000 7f01d57fa640 1 -- 172.21.15.19:0/1293925160 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f01cc08a570 con 0x7f01d80b92a0 2024-04-25T10:00:20.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.137+0000 7f01dcc38640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f01c406b4d0 0x7f01c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.137+0000 7f01dcc38640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f01c406b4d0 0x7f01c406d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f01d0033000 tx=0x7f01d0009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.141+0000 7f01d57fa640 1 -- 172.21.15.19:0/1293925160 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01cc058ca0 con 0x7f01d80b92a0 2024-04-25T10:00:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.360+0000 7f01dee9c640 1 -- 172.21.15.19:0/1293925160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01d806cae0 con 0x7f01d80b92a0 2024-04-25T10:00:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.361+0000 7f01d57fa640 1 -- 172.21.15.19:0/1293925160 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f01cc05cd40 con 0x7f01d80b92a0 2024-04-25T10:00:20.365 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:20.365 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 -- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f01c406b4d0 msgr2=0x7f01c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f01c406b4d0 0x7f01c406d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f01d0033000 tx=0x7f01d0009000 comp rx=0 tx=0).stop 2024-04-25T10:00:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 -- 172.21.15.19:0/1293925160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80b92a0 msgr2=0x7f01d80b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80b92a0 0x7f01d80b5b90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00ebe0 tx=0x7f01cc00c520 comp rx=0 tx=0).stop 2024-04-25T10:00:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 -- 172.21.15.19:0/1293925160 shutdown_connections 2024-04-25T10:00:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f01c406b4d0 0x7f01c406d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01d815dbe0 0x7f01d815b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80b92a0 0x7f01d80b5b90 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 --2- 172.21.15.19:0/1293925160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01d80b2cb0 0x7f01d81f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 -- 172.21.15.19:0/1293925160 >> 172.21.15.19:0/1293925160 conn(0x7f01d8061a20 msgr2=0x7f01d80d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 -- 172.21.15.19:0/1293925160 shutdown_connections 2024-04-25T10:00:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.363+0000 7f01dee9c640 1 -- 172.21.15.19:0/1293925160 wait complete. 2024-04-25T10:00:20.378 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.475+0000 7efe9c11d640 1 Processor -- start 2024-04-25T10:00:20.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.475+0000 7efe9c11d640 1 -- start start 2024-04-25T10:00:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.476+0000 7efe9c11d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe9415dbe0 0x7efe940d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.476+0000 7efe9c11d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe940b6220 con 0x7efe940b92a0 2024-04-25T10:00:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.476+0000 7efe9c11d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe940b97a0 con 0x7efe940b2cb0 2024-04-25T10:00:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.476+0000 7efe9c11d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9415dfc0 con 0x7efe9415dbe0 2024-04-25T10:00:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.476+0000 7efe99eb9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe9415dbe0 0x7efe940d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.476+0000 7efe9a6ba640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efe940b92a0 0x7efe940b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43988/0 (socket says 172.21.15.19:43988) 2024-04-25T10:00:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.476+0000 7efe9a6ba640 1 -- 172.21.15.19:0/3167487823 learned_addr learned my addr 172.21.15.19:0/3167487823 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.476+0000 7efe99eb9640 1 -- 172.21.15.19:0/3167487823 >> v1:172.21.15.112:6789/0 conn(0x7efe940b2cb0 legacy=0x7efe940b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.476+0000 7efe99eb9640 1 -- 172.21.15.19:0/3167487823 >> v1:172.21.15.19:6789/0 conn(0x7efe940b92a0 legacy=0x7efe940b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.476+0000 7efe99eb9640 1 -- 172.21.15.19:0/3167487823 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940d1c50 con 0x7efe9415dbe0 2024-04-25T10:00:20.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.476+0000 7efe98eb7640 1 -- 172.21.15.19:0/3167487823 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579896873 0 0) 0x7efe940b6220 con 0x7efe940b92a0 2024-04-25T10:00:20.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.477+0000 7efe99eb9640 1 --2- 172.21.15.19:0/3167487823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe9415dbe0 0x7efe940d0910 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7efe900097a0 tx=0x7efe9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5164d705212f1088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.477+0000 7efe98eb7640 1 -- 172.21.15.19:0/3167487823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90037070 con 0x7efe9415dbe0 2024-04-25T10:00:20.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.477+0000 7efe98eb7640 1 -- 172.21.15.19:0/3167487823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe90032440 con 0x7efe9415dbe0 2024-04-25T10:00:20.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.477+0000 7efe98eb7640 1 -- 172.21.15.19:0/3167487823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90032810 con 0x7efe9415dbe0 2024-04-25T10:00:20.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.477+0000 7efe9c11d640 1 -- 172.21.15.19:0/3167487823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe9415dbe0 msgr2=0x7efe940d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.477+0000 7efe9c11d640 1 --2- 172.21.15.19:0/3167487823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe9415dbe0 0x7efe940d0910 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7efe900097a0 tx=0x7efe9002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.478+0000 7efe9c11d640 1 -- 172.21.15.19:0/3167487823 shutdown_connections 2024-04-25T10:00:20.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.478+0000 7efe9c11d640 1 --2- 172.21.15.19:0/3167487823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe9415dbe0 0x7efe940d0910 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.478+0000 7efe9c11d640 1 -- 172.21.15.19:0/3167487823 >> 172.21.15.19:0/3167487823 conn(0x7efe94061a20 msgr2=0x7efe94061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.478+0000 7efe9c11d640 1 -- 172.21.15.19:0/3167487823 shutdown_connections 2024-04-25T10:00:20.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.478+0000 7efe9c11d640 1 -- 172.21.15.19:0/3167487823 wait complete. 2024-04-25T10:00:20.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.479+0000 7efe9c11d640 1 Processor -- start 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.479+0000 7efe9c11d640 1 -- start start 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.479+0000 7efe9c11d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe940b2cb0 0x7efe940caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.479+0000 7efe9c11d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe940b92a0 0x7efe940c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.479+0000 7efe9c11d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe9415dbe0 0x7efe940cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.479+0000 7efe9c11d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940d2210 con 0x7efe9415dbe0 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.479+0000 7efe9c11d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940d1a00 con 0x7efe940b92a0 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.479+0000 7efe9c11d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940b6ab0 con 0x7efe940b2cb0 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe9a6ba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe9415dbe0 0x7efe940cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe9a6ba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe9415dbe0 0x7efe940cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33104/0 (socket says 172.21.15.19:33104) 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe9a6ba640 1 -- 172.21.15.19:0/4086669830 learned_addr learned my addr 172.21.15.19:0/4086669830 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe9a6ba640 1 -- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe940b2cb0 msgr2=0x7efe940caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe9a6ba640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe940b2cb0 0x7efe940caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe9a6ba640 1 -- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe940b92a0 msgr2=0x7efe940c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe9a6ba640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe940b92a0 0x7efe940c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe9a6ba640 1 -- 172.21.15.19:0/4086669830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9415e150 con 0x7efe9415dbe0 2024-04-25T10:00:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe9a6ba640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe9415dbe0 0x7efe940cb820 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c780 tx=0x7efe8c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe8a7fc640 1 -- 172.21.15.19:0/4086669830 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c018070 con 0x7efe9415dbe0 2024-04-25T10:00:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe8a7fc640 1 -- 172.21.15.19:0/4086669830 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c00a640 con 0x7efe9415dbe0 2024-04-25T10:00:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe9c11d640 1 -- 172.21.15.19:0/4086669830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9415e430 con 0x7efe9415dbe0 2024-04-25T10:00:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe8a7fc640 1 -- 172.21.15.19:0/4086669830 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c00aab0 con 0x7efe9415dbe0 2024-04-25T10:00:20.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.480+0000 7efe9c11d640 1 -- 172.21.15.19:0/4086669830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9415e9f0 con 0x7efe9415dbe0 2024-04-25T10:00:20.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.481+0000 7efe9c11d640 1 -- 172.21.15.19:0/4086669830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe94073990 con 0x7efe9415dbe0 2024-04-25T10:00:20.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.482+0000 7efe8a7fc640 1 -- 172.21.15.19:0/4086669830 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe8c00ac50 con 0x7efe9415dbe0 2024-04-25T10:00:20.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.482+0000 7efe8a7fc640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe7806b4d0 0x7efe7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.482+0000 7efe8a7fc640 1 -- 172.21.15.19:0/4086669830 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe8c0899f0 con 0x7efe9415dbe0 2024-04-25T10:00:20.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.482+0000 7efe99eb9640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe7806b4d0 0x7efe7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.483+0000 7efe99eb9640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe7806b4d0 0x7efe7806d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efe90034000 tx=0x7efe9002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.486+0000 7efe8a7fc640 1 -- 172.21.15.19:0/4086669830 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe8c0586f0 con 0x7efe9415dbe0 2024-04-25T10:00:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.667+0000 7efe9c11d640 1 -- 172.21.15.19:0/4086669830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe940ca2e0 con 0x7efe9415dbe0 2024-04-25T10:00:20.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.668+0000 7efe8a7fc640 1 -- 172.21.15.19:0/4086669830 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16872 (secure 0 0 0) 0x7efe8c089070 con 0x7efe9415dbe0 2024-04-25T10:00:20.672 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:20.672 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":106,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:48.558694+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:45.201424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:45.885235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 -- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe7806b4d0 msgr2=0x7efe7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe7806b4d0 0x7efe7806d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efe90034000 tx=0x7efe9002c000 comp rx=0 tx=0).stop 2024-04-25T10:00:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 -- 172.21.15.19:0/4086669830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe9415dbe0 msgr2=0x7efe940cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe9415dbe0 0x7efe940cb820 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c780 tx=0x7efe8c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:00:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 -- 172.21.15.19:0/4086669830 shutdown_connections 2024-04-25T10:00:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe7806b4d0 0x7efe7806d990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe9415dbe0 0x7efe940cb820 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe940b92a0 0x7efe940c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 --2- 172.21.15.19:0/4086669830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe940b2cb0 0x7efe940caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 -- 172.21.15.19:0/4086669830 >> 172.21.15.19:0/4086669830 conn(0x7efe94061a20 msgr2=0x7efe940d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 -- 172.21.15.19:0/4086669830 shutdown_connections 2024-04-25T10:00:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.670+0000 7efe9c11d640 1 -- 172.21.15.19:0/4086669830 wait complete. 2024-04-25T10:00:20.686 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:00:20.686 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:20.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.783+0000 7febbe570640 1 Processor -- start 2024-04-25T10:00:20.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.783+0000 7febbe570640 1 -- start start 2024-04-25T10:00:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febbe570640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febb80b2cb0 0x7febb80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febbe570640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febb80d1020 0x7febb80d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febbe570640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb80b97a0 con 0x7febb815dbe0 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febbe570640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb815dfc0 con 0x7febb80b2cb0 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febbe570640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb80d1a00 con 0x7febb80d1020 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febb77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febb80b2cb0 0x7febb80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febb77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febb80b2cb0 0x7febb80d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33114/0 (socket says 172.21.15.19:33114) 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febb77fe640 1 -- 172.21.15.19:0/2171279701 learned_addr learned my addr 172.21.15.19:0/2171279701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febbcb0d640 1 --2- 172.21.15.19:0/2171279701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febb80d1020 0x7febb80d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febb77fe640 1 -- 172.21.15.19:0/2171279701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febb80d1020 msgr2=0x7febb80d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febb77fe640 1 --2- 172.21.15.19:0/2171279701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febb80d1020 0x7febb80d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febb77fe640 1 -- 172.21.15.19:0/2171279701 >> v1:172.21.15.112:6789/0 conn(0x7febb815dbe0 legacy=0x7febb80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.784+0000 7febb77fe640 1 -- 172.21.15.19:0/2171279701 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb80d1c50 con 0x7febb80b2cb0 2024-04-25T10:00:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.785+0000 7febb77fe640 1 --2- 172.21.15.19:0/2171279701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80b2cb0 0x7febb80d2730 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7febac009870 tx=0x7febac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e28434e52170df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.785+0000 7febb6ffd640 1 -- 172.21.15.19:0/2171279701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac037070 con 0x7febb80b2cb0 2024-04-25T10:00:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.785+0000 7febb6ffd640 1 -- 172.21.15.19:0/2171279701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febac032440 con 0x7febb80b2cb0 2024-04-25T10:00:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.785+0000 7febb6ffd640 1 -- 172.21.15.19:0/2171279701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac032810 con 0x7febb80b2cb0 2024-04-25T10:00:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.786+0000 7febbe570640 1 -- 172.21.15.19:0/2171279701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80b2cb0 msgr2=0x7febb80d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.786+0000 7febbe570640 1 --2- 172.21.15.19:0/2171279701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80b2cb0 0x7febb80d2730 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7febac009870 tx=0x7febac02a210 comp rx=0 tx=0).stop 2024-04-25T10:00:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.786+0000 7febbe570640 1 -- 172.21.15.19:0/2171279701 shutdown_connections 2024-04-25T10:00:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.786+0000 7febbe570640 1 --2- 172.21.15.19:0/2171279701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febb80d1020 0x7febb80d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.786+0000 7febbe570640 1 --2- 172.21.15.19:0/2171279701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80b2cb0 0x7febb80d2730 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.786+0000 7febbe570640 1 -- 172.21.15.19:0/2171279701 >> 172.21.15.19:0/2171279701 conn(0x7febb8061a20 msgr2=0x7febb8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.786+0000 7febbe570640 1 -- 172.21.15.19:0/2171279701 shutdown_connections 2024-04-25T10:00:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.786+0000 7febbe570640 1 -- 172.21.15.19:0/2171279701 wait complete. 2024-04-25T10:00:20.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.787+0000 7febbe570640 1 Processor -- start 2024-04-25T10:00:20.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.787+0000 7febbe570640 1 -- start start 2024-04-25T10:00:20.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febbe570640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80b2cb0 0x7febb80c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febbe570640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb80d1020 0x7febb80caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febbe570640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febb815dbe0 0x7febb80caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febbe570640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb80b92a0 con 0x7febb80b2cb0 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febbe570640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb80d2210 con 0x7febb80d1020 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febbe570640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb80b9420 con 0x7febb815dbe0 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80b2cb0 0x7febb80c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80b2cb0 0x7febb80c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33120/0 (socket says 172.21.15.19:33120) 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb7fff640 1 -- 172.21.15.19:0/475710881 learned_addr learned my addr 172.21.15.19:0/475710881 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febbcb0d640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febb815dbe0 0x7febb80caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb77fe640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb80d1020 0x7febb80caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb7fff640 1 -- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febb815dbe0 msgr2=0x7febb80caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb7fff640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febb815dbe0 0x7febb80caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb7fff640 1 -- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb80d1020 msgr2=0x7febb80caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb7fff640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb80d1020 0x7febb80caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb7fff640 1 -- 172.21.15.19:0/475710881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb8160370 con 0x7febb80b2cb0 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb77fe640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb80d1020 0x7febb80caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb7fff640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80b2cb0 0x7febb80c9080 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7feba800b3f0 tx=0x7feba800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb4ff9640 1 -- 172.21.15.19:0/475710881 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8015020 con 0x7febb80b2cb0 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febb4ff9640 1 -- 172.21.15.19:0/475710881 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba8011830 con 0x7febb80b2cb0 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.788+0000 7febbe570640 1 -- 172.21.15.19:0/475710881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb80b7e90 con 0x7febb80b2cb0 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.789+0000 7febb4ff9640 1 -- 172.21.15.19:0/475710881 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8011c20 con 0x7febb80b2cb0 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.789+0000 7febbe570640 1 -- 172.21.15.19:0/475710881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb80b8450 con 0x7febb80b2cb0 2024-04-25T10:00:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.789+0000 7febbe570640 1 -- 172.21.15.19:0/475710881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb8073990 con 0x7febb80b2cb0 2024-04-25T10:00:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.790+0000 7febb4ff9640 1 -- 172.21.15.19:0/475710881 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feba8018020 con 0x7febb80b2cb0 2024-04-25T10:00:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.790+0000 7febb4ff9640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feba406b4d0 0x7feba406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.790+0000 7febb4ff9640 1 -- 172.21.15.19:0/475710881 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feba808a1e0 con 0x7febb80b2cb0 2024-04-25T10:00:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.791+0000 7febb77fe640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feba406b4d0 0x7feba406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.791+0000 7febb77fe640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feba406b4d0 0x7feba406d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7febac034000 tx=0x7febac02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:20.795+0000 7febb4ff9640 1 -- 172.21.15.19:0/475710881 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feba801f020 con 0x7febb80b2cb0 2024-04-25T10:00:21.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.013+0000 7febbe570640 1 -- 172.21.15.19:0/475710881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febb80d3550 con 0x7febb80b2cb0 2024-04-25T10:00:21.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.013+0000 7febb4ff9640 1 -- 172.21.15.19:0/475710881 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7feba8058f60 con 0x7febb80b2cb0 2024-04-25T10:00:21.017 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:21.017 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:21.018 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 -- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feba406b4d0 msgr2=0x7feba406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feba406b4d0 0x7feba406d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7febac034000 tx=0x7febac02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:21.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 -- 172.21.15.19:0/475710881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80b2cb0 msgr2=0x7febb80c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80b2cb0 0x7febb80c9080 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7feba800b3f0 tx=0x7feba800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 -- 172.21.15.19:0/475710881 shutdown_connections 2024-04-25T10:00:21.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feba406b4d0 0x7feba406d990 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febb815dbe0 0x7febb80caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb80d1020 0x7febb80caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 --2- 172.21.15.19:0/475710881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80b2cb0 0x7febb80c9080 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 -- 172.21.15.19:0/475710881 >> 172.21.15.19:0/475710881 conn(0x7febb8061a20 msgr2=0x7febb80d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 -- 172.21.15.19:0/475710881 shutdown_connections 2024-04-25T10:00:21.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.015+0000 7febbe570640 1 -- 172.21.15.19:0/475710881 wait complete. 2024-04-25T10:00:21.031 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.126+0000 7f315bb1d640 1 Processor -- start 2024-04-25T10:00:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.127+0000 7f315bb1d640 1 -- start start 2024-04-25T10:00:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.127+0000 7f315bb1d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3154154ee0 0x7f31540cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.127+0000 7f315bb1d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31540cfcf0 0x7f31540cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.127+0000 7f315bb1d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3154071480 con 0x7f31540ce3c0 2024-04-25T10:00:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.128+0000 7f315bb1d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31540ce830 con 0x7f3154154ee0 2024-04-25T10:00:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.128+0000 7f315bb1d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3154072090 con 0x7f31540cfcf0 2024-04-25T10:00:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.128+0000 7f31598b9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3154154ee0 0x7f31540cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.128+0000 7f315a0ba640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f31540ce3c0 0x7f315415e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43998/0 (socket says 172.21.15.19:43998) 2024-04-25T10:00:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.128+0000 7f315a0ba640 1 -- 172.21.15.19:0/2551463624 learned_addr learned my addr 172.21.15.19:0/2551463624 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.128+0000 7f31590b8640 1 --2- 172.21.15.19:0/2551463624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31540cfcf0 0x7f31540cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.128+0000 7f31598b9640 1 -- 172.21.15.19:0/2551463624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31540cfcf0 msgr2=0x7f31540cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.128+0000 7f31598b9640 1 --2- 172.21.15.19:0/2551463624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31540cfcf0 0x7f31540cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.128+0000 7f31598b9640 1 -- 172.21.15.19:0/2551463624 >> v1:172.21.15.19:6789/0 conn(0x7f31540ce3c0 legacy=0x7f315415e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.128+0000 7f31598b9640 1 -- 172.21.15.19:0/2551463624 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3154071760 con 0x7f3154154ee0 2024-04-25T10:00:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.128+0000 7f31588b7640 1 -- 172.21.15.19:0/2551463624 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091037389 0 0) 0x7f3154071480 con 0x7f31540ce3c0 2024-04-25T10:00:21.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.129+0000 7f31598b9640 1 --2- 172.21.15.19:0/2551463624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3154154ee0 0x7f31540cf7b0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f31500097a0 tx=0x7f315002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7824a0f578f5eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.129+0000 7f31588b7640 1 -- 172.21.15.19:0/2551463624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150037070 con 0x7f3154154ee0 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.129+0000 7f31588b7640 1 -- 172.21.15.19:0/2551463624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3150032440 con 0x7f3154154ee0 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.129+0000 7f31588b7640 1 -- 172.21.15.19:0/2551463624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150032810 con 0x7f3154154ee0 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.129+0000 7f315bb1d640 1 -- 172.21.15.19:0/2551463624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3154154ee0 msgr2=0x7f31540cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.129+0000 7f315bb1d640 1 --2- 172.21.15.19:0/2551463624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3154154ee0 0x7f31540cf7b0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f31500097a0 tx=0x7f315002a180 comp rx=0 tx=0).stop 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.129+0000 7f315bb1d640 1 -- 172.21.15.19:0/2551463624 shutdown_connections 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.129+0000 7f315bb1d640 1 --2- 172.21.15.19:0/2551463624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31540cfcf0 0x7f31540cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.129+0000 7f315bb1d640 1 --2- 172.21.15.19:0/2551463624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3154154ee0 0x7f31540cf7b0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.129+0000 7f315bb1d640 1 -- 172.21.15.19:0/2551463624 >> 172.21.15.19:0/2551463624 conn(0x7f315406abe0 msgr2=0x7f315406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.130+0000 7f315bb1d640 1 -- 172.21.15.19:0/2551463624 shutdown_connections 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.130+0000 7f315bb1d640 1 -- 172.21.15.19:0/2551463624 wait complete. 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.131+0000 7f315bb1d640 1 Processor -- start 2024-04-25T10:00:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.131+0000 7f315bb1d640 1 -- start start 2024-04-25T10:00:21.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.131+0000 7f315bb1d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31540ce3c0 0x7f3154141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.131+0000 7f315bb1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31540cfcf0 0x7f31541564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.131+0000 7f315bb1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154154ee0 0x7f3154157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.131+0000 7f315bb1d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3154071230 con 0x7f31540cfcf0 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.131+0000 7f315bb1d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3154071d20 con 0x7f3154154ee0 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.131+0000 7f315bb1d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31541552c0 con 0x7f31540ce3c0 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f31590b8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31540cfcf0 0x7f31541564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f31590b8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31540cfcf0 0x7f31541564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33132/0 (socket says 172.21.15.19:33132) 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f31590b8640 1 -- 172.21.15.19:0/2685615546 learned_addr learned my addr 172.21.15.19:0/2685615546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f31598b9640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31540ce3c0 0x7f3154141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f315a0ba640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154154ee0 0x7f3154157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f31590b8640 1 -- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31540ce3c0 msgr2=0x7f3154141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f31590b8640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31540ce3c0 0x7f3154141fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f31590b8640 1 -- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154154ee0 msgr2=0x7f3154157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f31590b8640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154154ee0 0x7f3154157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f31590b8640 1 -- 172.21.15.19:0/2685615546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31541556e0 con 0x7f31540cfcf0 2024-04-25T10:00:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f31598b9640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31540ce3c0 0x7f3154141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f31590b8640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31540cfcf0 0x7f31541564a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f314400d800 tx=0x7f314400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f314a7fc640 1 -- 172.21.15.19:0/2685615546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314400b6c0 con 0x7f31540cfcf0 2024-04-25T10:00:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f314a7fc640 1 -- 172.21.15.19:0/2685615546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3144019070 con 0x7f31540cfcf0 2024-04-25T10:00:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f315bb1d640 1 -- 172.21.15.19:0/2685615546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31541559c0 con 0x7f31540cfcf0 2024-04-25T10:00:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f314a7fc640 1 -- 172.21.15.19:0/2685615546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314400ba80 con 0x7f31540cfcf0 2024-04-25T10:00:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.132+0000 7f315bb1d640 1 -- 172.21.15.19:0/2685615546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315415a320 con 0x7f31540cfcf0 2024-04-25T10:00:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.133+0000 7f315bb1d640 1 -- 172.21.15.19:0/2685615546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3154155db0 con 0x7f31540cfcf0 2024-04-25T10:00:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.134+0000 7f314a7fc640 1 -- 172.21.15.19:0/2685615546 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f314400bc20 con 0x7f31540cfcf0 2024-04-25T10:00:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.134+0000 7f314a7fc640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313806b500 0x7f313806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.134+0000 7f314a7fc640 1 -- 172.21.15.19:0/2685615546 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f314408a7b0 con 0x7f31540cfcf0 2024-04-25T10:00:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.134+0000 7f31598b9640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313806b500 0x7f313806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.135+0000 7f31598b9640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313806b500 0x7f313806d9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3150034000 tx=0x7f315002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.138+0000 7f314a7fc640 1 -- 172.21.15.19:0/2685615546 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3144059530 con 0x7f31540cfcf0 2024-04-25T10:00:21.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.355+0000 7f315bb1d640 1 -- 172.21.15.19:0/2685615546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3154073990 con 0x7f31540cfcf0 2024-04-25T10:00:21.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.356+0000 7f314a7fc640 1 -- 172.21.15.19:0/2685615546 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3144058e30 con 0x7f31540cfcf0 2024-04-25T10:00:21.360 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:21.360 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:59:46.906578+0000","modified":"2024-04-25T09:59:48.905936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/4224193015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4224193015},{"type":"v1","addr":"172.21.15.19:6835","nonce":4224193015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:00:21.361 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T10:00:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 -- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313806b500 msgr2=0x7f313806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313806b500 0x7f313806d9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3150034000 tx=0x7f315002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 -- 172.21.15.19:0/2685615546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31540cfcf0 msgr2=0x7f31541564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31540cfcf0 0x7f31541564a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f314400d800 tx=0x7f314400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 -- 172.21.15.19:0/2685615546 shutdown_connections 2024-04-25T10:00:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f313806b500 0x7f313806d9c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3154154ee0 0x7f3154157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31540cfcf0 0x7f31541564a0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 --2- 172.21.15.19:0/2685615546 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31540ce3c0 0x7f3154141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 -- 172.21.15.19:0/2685615546 >> 172.21.15.19:0/2685615546 conn(0x7f315406abe0 msgr2=0x7f31540cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 -- 172.21.15.19:0/2685615546 shutdown_connections 2024-04-25T10:00:21.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.358+0000 7f315bb1d640 1 -- 172.21.15.19:0/2685615546 wait complete. 2024-04-25T10:00:21.373 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.471+0000 7f5e915c1640 1 Processor -- start 2024-04-25T10:00:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.471+0000 7f5e915c1640 1 -- start start 2024-04-25T10:00:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e915c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c154ee0 0x7f5e8c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e915c1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5e8c0cfcf0 0x7f5e8c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e915c1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8c071480 con 0x7f5e8c0ce3c0 2024-04-25T10:00:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e915c1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8c0ce830 con 0x7f5e8c0cfcf0 2024-04-25T10:00:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e915c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c072090 con 0x7f5e8c154ee0 2024-04-25T10:00:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e83fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5e8c0cfcf0 0x7f5e8c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e83fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5e8c0cfcf0 0x7f5e8c0cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33138/0 (socket says 172.21.15.19:33138) 2024-04-25T10:00:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e83fff640 1 -- 172.21.15.19:0/3195308453 learned_addr learned my addr 172.21.15.19:0/3195308453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e83fff640 1 -- 172.21.15.19:0/3195308453 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c154ee0 msgr2=0x7f5e8c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e8affd640 1 --2- 172.21.15.19:0/3195308453 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c154ee0 0x7f5e8c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e83fff640 1 --2- 172.21.15.19:0/3195308453 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c154ee0 0x7f5e8c0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e83fff640 1 -- 172.21.15.19:0/3195308453 >> v1:172.21.15.112:6789/0 conn(0x7f5e8c0ce3c0 legacy=0x7f5e8c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.472+0000 7f5e83fff640 1 -- 172.21.15.19:0/3195308453 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c071760 con 0x7f5e8c0cfcf0 2024-04-25T10:00:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.473+0000 7f5e83fff640 1 --2- 172.21.15.19:0/3195308453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c0cfcf0 0x7f5e8c0cde80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009870 tx=0x7f5e7402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1064a8183f5246b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.473+0000 7f5e8a7fc640 1 -- 172.21.15.19:0/3195308453 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74037070 con 0x7f5e8c0cfcf0 2024-04-25T10:00:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.473+0000 7f5e8a7fc640 1 -- 172.21.15.19:0/3195308453 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e74032440 con 0x7f5e8c0cfcf0 2024-04-25T10:00:21.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.473+0000 7f5e8a7fc640 1 -- 172.21.15.19:0/3195308453 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74032810 con 0x7f5e8c0cfcf0 2024-04-25T10:00:21.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.474+0000 7f5e915c1640 1 -- 172.21.15.19:0/3195308453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c0cfcf0 msgr2=0x7f5e8c0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.474+0000 7f5e915c1640 1 --2- 172.21.15.19:0/3195308453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c0cfcf0 0x7f5e8c0cde80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009870 tx=0x7f5e7402a210 comp rx=0 tx=0).stop 2024-04-25T10:00:21.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.474+0000 7f5e915c1640 1 -- 172.21.15.19:0/3195308453 shutdown_connections 2024-04-25T10:00:21.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.474+0000 7f5e915c1640 1 --2- 172.21.15.19:0/3195308453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c0cfcf0 0x7f5e8c0cde80 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.474+0000 7f5e915c1640 1 --2- 172.21.15.19:0/3195308453 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c154ee0 0x7f5e8c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.474+0000 7f5e915c1640 1 -- 172.21.15.19:0/3195308453 >> 172.21.15.19:0/3195308453 conn(0x7f5e8c06abe0 msgr2=0x7f5e8c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.474+0000 7f5e915c1640 1 -- 172.21.15.19:0/3195308453 shutdown_connections 2024-04-25T10:00:21.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.474+0000 7f5e915c1640 1 -- 172.21.15.19:0/3195308453 wait complete. 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.475+0000 7f5e915c1640 1 Processor -- start 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.475+0000 7f5e915c1640 1 -- start start 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.475+0000 7f5e915c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c0ce3c0 0x7f5e8c0ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e915c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c0cfcf0 0x7f5e8c0adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e915c1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c154ee0 0x7f5e8c158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e915c1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c071230 con 0x7f5e8c154ee0 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e915c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c071d20 con 0x7f5e8c0cfcf0 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e915c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c1552c0 con 0x7f5e8c0ce3c0 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e8b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c154ee0 0x7f5e8c158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e8b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c154ee0 0x7f5e8c158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33152/0 (socket says 172.21.15.19:33152) 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e8b7fe640 1 -- 172.21.15.19:0/255887498 learned_addr learned my addr 172.21.15.19:0/255887498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e8affd640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c0ce3c0 0x7f5e8c0ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e83fff640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c0cfcf0 0x7f5e8c0adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e8b7fe640 1 -- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c0ce3c0 msgr2=0x7f5e8c0ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e8b7fe640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c0ce3c0 0x7f5e8c0ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e8b7fe640 1 -- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c0cfcf0 msgr2=0x7f5e8c0adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e8b7fe640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c0cfcf0 0x7f5e8c0adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e8b7fe640 1 -- 172.21.15.19:0/255887498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c150360 con 0x7f5e8c154ee0 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e83fff640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c0cfcf0 0x7f5e8c0adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e8b7fe640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c154ee0 0x7f5e8c158630 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400c780 tx=0x7f5e8400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e837fe640 1 -- 172.21.15.19:0/255887498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84004b60 con 0x7f5e8c154ee0 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e837fe640 1 -- 172.21.15.19:0/255887498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e84004d00 con 0x7f5e8c154ee0 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e915c1640 1 -- 172.21.15.19:0/255887498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c150640 con 0x7f5e8c154ee0 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e837fe640 1 -- 172.21.15.19:0/255887498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8400aa90 con 0x7f5e8c154ee0 2024-04-25T10:00:21.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.476+0000 7f5e915c1640 1 -- 172.21.15.19:0/255887498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c150c00 con 0x7f5e8c154ee0 2024-04-25T10:00:21.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.477+0000 7f5e915c1640 1 -- 172.21.15.19:0/255887498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8c073990 con 0x7f5e8c154ee0 2024-04-25T10:00:21.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.478+0000 7f5e837fe640 1 -- 172.21.15.19:0/255887498 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e8400ac30 con 0x7f5e8c154ee0 2024-04-25T10:00:21.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.478+0000 7f5e837fe640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e6c06b4d0 0x7f5e6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.479+0000 7f5e837fe640 1 -- 172.21.15.19:0/255887498 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e8408a660 con 0x7f5e8c154ee0 2024-04-25T10:00:21.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.479+0000 7f5e8affd640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e6c06b4d0 0x7f5e6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.479+0000 7f5e8affd640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e6c06b4d0 0x7f5e6c06d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c006fd0 tx=0x7f5e7c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.483+0000 7f5e837fe640 1 -- 172.21.15.19:0/255887498 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e84020070 con 0x7f5e8c154ee0 2024-04-25T10:00:21.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.664+0000 7f5e915c1640 1 -- 172.21.15.19:0/255887498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e8c1664f0 con 0x7f5e8c154ee0 2024-04-25T10:00:21.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.665+0000 7f5e837fe640 1 -- 172.21.15.19:0/255887498 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f5e84088e20 con 0x7f5e8c154ee0 2024-04-25T10:00:21.669 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:21.669 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":106,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T09:59:48.558694+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:59:45.201424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:59:45.885235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:21.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 -- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e6c06b4d0 msgr2=0x7f5e6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e6c06b4d0 0x7f5e6c06d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c006fd0 tx=0x7f5e7c008040 comp rx=0 tx=0).stop 2024-04-25T10:00:21.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 -- 172.21.15.19:0/255887498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c154ee0 msgr2=0x7f5e8c158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c154ee0 0x7f5e8c158630 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400c780 tx=0x7f5e8400cc40 comp rx=0 tx=0).stop 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 -- 172.21.15.19:0/255887498 shutdown_connections 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e6c06b4d0 0x7f5e6c06d990 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c154ee0 0x7f5e8c158630 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c0cfcf0 0x7f5e8c0adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 --2- 172.21.15.19:0/255887498 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c0ce3c0 0x7f5e8c0ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 -- 172.21.15.19:0/255887498 >> 172.21.15.19:0/255887498 conn(0x7f5e8c06abe0 msgr2=0x7f5e8c0af690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 -- 172.21.15.19:0/255887498 shutdown_connections 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.667+0000 7f5e915c1640 1 -- 172.21.15.19:0/255887498 wait complete. 2024-04-25T10:00:21.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:00:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.780+0000 7f2dfbd0a640 1 Processor -- start 2024-04-25T10:00:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.780+0000 7f2dfbd0a640 1 -- start start 2024-04-25T10:00:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2dfbd0a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2df4144d50 0x7f2df406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2dfbd0a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2df41452a0 0x7f2df406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2dfbd0a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df40cccc0 con 0x7f2df41446d0 2024-04-25T10:00:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2dfbd0a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df40b2b20 con 0x7f2df4144d50 2024-04-25T10:00:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2dfbd0a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df406d400 con 0x7f2df41452a0 2024-04-25T10:00:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2df92a5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2df4144d50 0x7f2df406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2df92a5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2df4144d50 0x7f2df406d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33166/0 (socket says 172.21.15.19:33166) 2024-04-25T10:00:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2df92a5640 1 -- 172.21.15.19:0/4250549048 learned_addr learned my addr 172.21.15.19:0/4250549048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2dfa2a7640 1 --2- 172.21.15.19:0/4250549048 >> v2:172.21.15.112:3300/0 conn(0x7f2df41452a0 0x7f2df406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2df92a5640 1 -- 172.21.15.19:0/4250549048 >> v2:172.21.15.112:3300/0 conn(0x7f2df41452a0 msgr2=0x7f2df406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2df92a5640 1 --2- 172.21.15.19:0/4250549048 >> v2:172.21.15.112:3300/0 conn(0x7f2df41452a0 0x7f2df406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2df92a5640 1 -- 172.21.15.19:0/4250549048 >> v1:172.21.15.112:6789/0 conn(0x7f2df41446d0 legacy=0x7f2df40b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.781+0000 7f2df92a5640 1 -- 172.21.15.19:0/4250549048 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df40cc140 con 0x7f2df4144d50 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.782+0000 7f2df92a5640 1 --2- 172.21.15.19:0/4250549048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df4144d50 0x7f2df406d890 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f2df415bfe0 tx=0x7f2de802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961397306809eb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.782+0000 7f2df8aa4640 1 -- 172.21.15.19:0/4250549048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8037070 con 0x7f2df4144d50 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.782+0000 7f2df8aa4640 1 -- 172.21.15.19:0/4250549048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de8032440 con 0x7f2df4144d50 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.782+0000 7f2df8aa4640 1 -- 172.21.15.19:0/4250549048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8032810 con 0x7f2df4144d50 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.783+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4250549048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df4144d50 msgr2=0x7f2df406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.783+0000 7f2dfbd0a640 1 --2- 172.21.15.19:0/4250549048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df4144d50 0x7f2df406d890 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f2df415bfe0 tx=0x7f2de802a270 comp rx=0 tx=0).stop 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.783+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4250549048 shutdown_connections 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.783+0000 7f2dfbd0a640 1 --2- 172.21.15.19:0/4250549048 >> v2:172.21.15.112:3300/0 conn(0x7f2df41452a0 0x7f2df406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.783+0000 7f2dfbd0a640 1 --2- 172.21.15.19:0/4250549048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df4144d50 0x7f2df406d890 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.783+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4250549048 >> 172.21.15.19:0/4250549048 conn(0x7f2df406abe0 msgr2=0x7f2df406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.783+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4250549048 shutdown_connections 2024-04-25T10:00:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.783+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4250549048 wait complete. 2024-04-25T10:00:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.784+0000 7f2dfbd0a640 1 Processor -- start 2024-04-25T10:00:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.784+0000 7f2dfbd0a640 1 -- start start 2024-04-25T10:00:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfbd0a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df41446d0 0x7f2df40cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfbd0a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2df4144d50 0x7f2df41f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfbd0a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df41452a0 0x7f2df415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfbd0a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40cc9e0 con 0x7f2df41452a0 2024-04-25T10:00:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfbd0a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40cc700 con 0x7f2df41446d0 2024-04-25T10:00:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfbd0a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4144ab0 con 0x7f2df4144d50 2024-04-25T10:00:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfa2a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df41452a0 0x7f2df415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfa2a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df41452a0 0x7f2df415aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33182/0 (socket says 172.21.15.19:33182) 2024-04-25T10:00:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfa2a7640 1 -- 172.21.15.19:0/4074657961 learned_addr learned my addr 172.21.15.19:0/4074657961 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2df9aa6640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df41446d0 0x7f2df40cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2df92a5640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2df4144d50 0x7f2df41f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfa2a7640 1 -- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2df4144d50 msgr2=0x7f2df41f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfa2a7640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2df4144d50 0x7f2df41f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfa2a7640 1 -- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df41446d0 msgr2=0x7f2df40cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfa2a7640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df41446d0 0x7f2df40cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfa2a7640 1 -- 172.21.15.19:0/4074657961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df40ce200 con 0x7f2df41452a0 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.785+0000 7f2dfa2a7640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df41452a0 0x7f2df415aef0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f2de4007e40 tx=0x7f2de400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.786+0000 7f2df27fc640 1 -- 172.21.15.19:0/4074657961 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4014620 con 0x7f2df41452a0 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.786+0000 7f2df27fc640 1 -- 172.21.15.19:0/4074657961 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de4019070 con 0x7f2df41452a0 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.786+0000 7f2df27fc640 1 -- 172.21.15.19:0/4074657961 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4014a20 con 0x7f2df41452a0 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.786+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4074657961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df40ce540 con 0x7f2df41452a0 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.786+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4074657961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df40ced80 con 0x7f2df41452a0 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.787+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4074657961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df4073a00 con 0x7f2df41452a0 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.787+0000 7f2df27fc640 1 -- 172.21.15.19:0/4074657961 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2de4014bc0 con 0x7f2df41452a0 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.787+0000 7f2df27fc640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2de006b4d0 0x7f2de006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.787+0000 7f2df27fc640 1 -- 172.21.15.19:0/4074657961 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2de408ab70 con 0x7f2df41452a0 2024-04-25T10:00:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.788+0000 7f2df9aa6640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2de006b4d0 0x7f2de006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.788+0000 7f2df9aa6640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2de006b4d0 0x7f2de006d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0016e0 tx=0x7f2dec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:21.791+0000 7f2df27fc640 1 -- 172.21.15.19:0/4074657961 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2df40ce540 con 0x7f2df41452a0 2024-04-25T10:00:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.012+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4074657961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2df406e910 con 0x7f2df41452a0 2024-04-25T10:00:22.016 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T10:00:22.012+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:00:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.014+0000 7f2df27fc640 1 -- 172.21.15.19:0/4074657961 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2de4059210 con 0x7f2df41452a0 2024-04-25T10:00:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:00:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2de006b4d0 msgr2=0x7f2de006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2de006b4d0 0x7f2de006d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0016e0 tx=0x7f2dec009040 comp rx=0 tx=0).stop 2024-04-25T10:00:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4074657961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df41452a0 msgr2=0x7f2df415aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df41452a0 0x7f2df415aef0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f2de4007e40 tx=0x7f2de400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4074657961 shutdown_connections 2024-04-25T10:00:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2de006b4d0 0x7f2de006d990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df41452a0 0x7f2df415aef0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2df4144d50 0x7f2df41f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 --2- 172.21.15.19:0/4074657961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2df41446d0 0x7f2df40cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4074657961 >> 172.21.15.19:0/4074657961 conn(0x7f2df406abe0 msgr2=0x7f2df406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4074657961 shutdown_connections 2024-04-25T10:00:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.017+0000 7f2dfbd0a640 1 -- 172.21.15.19:0/4074657961 wait complete. 2024-04-25T10:00:22.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:00:22.064 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T10:00:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.136+0000 7f0824658640 1 Processor -- start 2024-04-25T10:00:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.136+0000 7f0824658640 1 -- start start 2024-04-25T10:00:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.137+0000 7f0824658640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f081c15b8a0 0x7f081c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.137+0000 7f0824658640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c0b4040 0x7f081c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.137+0000 7f0824658640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081c0adf30 con 0x7f081c0af520 2024-04-25T10:00:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.137+0000 7f0824658640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081c0ad970 con 0x7f081c15b8a0 2024-04-25T10:00:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.137+0000 7f0824658640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0adc50 con 0x7f081c0b4040 2024-04-25T10:00:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.137+0000 7f0821bf3640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f081c0af520 0x7f081c06b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44014/0 (socket says 172.21.15.19:44014) 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.137+0000 7f0821bf3640 1 -- 172.21.15.19:0/2226741535 learned_addr learned my addr 172.21.15.19:0/2226741535 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f0822bf5640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c0b4040 0x7f081c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118521907 0 0) 0x7f080c001110 con 0x7f081c0af520 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08223f4640 1 --2- 172.21.15.19:0/2226741535 >> v2:172.21.15.112:3300/0 conn(0x7f081c15b8a0 0x7f081c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0808003520 con 0x7f081c0af520 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 865112066 0 0) 0x7f0808003520 con 0x7f081c0af520 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c0b4040 msgr2=0x7f081c0b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c0b4040 0x7f081c0b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 >> v2:172.21.15.112:3300/0 conn(0x7f081c15b8a0 msgr2=0x7f081c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 --2- 172.21.15.19:0/2226741535 >> v2:172.21.15.112:3300/0 conn(0x7f081c15b8a0 0x7f081c06abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c0b3840 con 0x7f081c0af520 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f080c001840 con 0x7f081c0af520 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 >> v1:172.21.15.19:6789/0 conn(0x7f081c0af520 legacy=0x7f081c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08080039b0 0x7f0808003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c0b4040 0x7f0808004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808004d40 0x7f08080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0ad970 con 0x7f08080039b0 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0adc50 con 0x7f0808004d40 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080800b9c0 con 0x7f081c0b4040 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08223f4640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08080039b0 0x7f0808003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f0821bf3640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c0b4040 0x7f0808004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f0822bf5640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808004d40 0x7f08080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08223f4640 1 -- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c0b4040 msgr2=0x7f0808004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08223f4640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c0b4040 0x7f0808004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08223f4640 1 -- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808004d40 msgr2=0x7f08080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08223f4640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808004d40 0x7f08080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f08223f4640 1 -- 172.21.15.19:0/2226741535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0808003520 con 0x7f08080039b0 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.138+0000 7f0822bf5640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808004d40 0x7f08080051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.139+0000 7f08223f4640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08080039b0 0x7f0808003dd0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f0818009ac0 tx=0x7f0808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eac05fb24cb3761 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.139+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818036070 con 0x7f08080039b0 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.139+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0818031400 con 0x7f08080039b0 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.139+0000 7f08213f2640 1 -- 172.21.15.19:0/2226741535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818009690 con 0x7f08080039b0 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.140+0000 7f0824658640 1 -- 172.21.15.19:0/2226741535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08080039b0 msgr2=0x7f0808003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.140+0000 7f0824658640 1 --2- 172.21.15.19:0/2226741535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08080039b0 0x7f0808003dd0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f0818009ac0 tx=0x7f0808005830 comp rx=0 tx=0).stop 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.140+0000 7f0822bf5640 1 -- 172.21.15.19:0/2226741535 reap_dead start 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.140+0000 7f0824658640 1 -- 172.21.15.19:0/2226741535 shutdown_connections 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.140+0000 7f0824658640 1 -- 172.21.15.19:0/2226741535 >> 172.21.15.19:0/2226741535 conn(0x7f081c0cfec0 msgr2=0x7f081c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.140+0000 7f0824658640 1 -- 172.21.15.19:0/2226741535 shutdown_connections 2024-04-25T10:00:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.140+0000 7f0824658640 1 -- 172.21.15.19:0/2226741535 wait complete. 2024-04-25T10:00:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.141+0000 7f0824658640 1 Processor -- start 2024-04-25T10:00:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.141+0000 7f0824658640 1 -- start start 2024-04-25T10:00:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0824658640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08080039b0 0x7f081c158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0824658640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c158e10 0x7f081c159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0824658640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f081c0cd610 0x7f081c0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0824658640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c06b9b0 con 0x7f081c0cd610 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0824658640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c06b830 con 0x7f08080039b0 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0824658640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c15bc80 con 0x7f081c158e10 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0822bf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f081c0cd610 0x7f081c0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0822bf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f081c0cd610 0x7f081c0cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33228/0 (socket says 172.21.15.19:33228) 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0822bf5640 1 -- 172.21.15.19:0/651788049 learned_addr learned my addr 172.21.15.19:0/651788049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f08223f4640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08080039b0 0x7f081c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0821bf3640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c158e10 0x7f081c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0822bf5640 1 -- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c158e10 msgr2=0x7f081c159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0822bf5640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c158e10 0x7f081c159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0822bf5640 1 -- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08080039b0 msgr2=0x7f081c158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0822bf5640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08080039b0 0x7f081c158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0822bf5640 1 -- 172.21.15.19:0/651788049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c06e8c0 con 0x7f081c0cd610 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.142+0000 7f0822bf5640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f081c0cd610 0x7f081c0cda90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f081c072da0 tx=0x7f0814010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.143+0000 7f0812ffd640 1 -- 172.21.15.19:0/651788049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081401a030 con 0x7f081c0cd610 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.143+0000 7f0812ffd640 1 -- 172.21.15.19:0/651788049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0814010d80 con 0x7f081c0cd610 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.143+0000 7f0824658640 1 -- 172.21.15.19:0/651788049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c06eba0 con 0x7f081c0cd610 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.143+0000 7f0812ffd640 1 -- 172.21.15.19:0/651788049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814003bd0 con 0x7f081c0cd610 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.143+0000 7f0824658640 1 -- 172.21.15.19:0/651788049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c06f160 con 0x7f081c0cd610 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.144+0000 7f0824658640 1 -- 172.21.15.19:0/651788049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081c073a00 con 0x7f081c0cd610 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.144+0000 7f0812ffd640 1 -- 172.21.15.19:0/651788049 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f081401d070 con 0x7f081c0cd610 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.145+0000 7f0812ffd640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080006b4d0 0x7f080006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.145+0000 7f0812ffd640 1 -- 172.21.15.19:0/651788049 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f081408a890 con 0x7f081c0cd610 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.145+0000 7f08223f4640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080006b4d0 0x7f080006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.145+0000 7f08223f4640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080006b4d0 0x7f080006d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0818033000 tx=0x7f0818008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.149+0000 7f0812ffd640 1 -- 172.21.15.19:0/651788049 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0814058f70 con 0x7f081c0cd610 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:22.366+0000 7f0824658640 1 -- 172.21.15.19:0/651788049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f081c15f340 con 0x7f081c0cd610 2024-04-25T10:00:23.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.015+0000 7f0812ffd640 1 -- 172.21.15.19:0/651788049 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f081405d010 con 0x7f081c0cd610 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.018+0000 7f0824658640 1 -- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080006b4d0 msgr2=0x7f080006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.018+0000 7f0824658640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080006b4d0 0x7f080006d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0818033000 tx=0x7f0818008030 comp rx=0 tx=0).stop 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.018+0000 7f0824658640 1 -- 172.21.15.19:0/651788049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f081c0cd610 msgr2=0x7f081c0cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.018+0000 7f0824658640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f081c0cd610 0x7f081c0cda90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f081c072da0 tx=0x7f0814010590 comp rx=0 tx=0).stop 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.018+0000 7f0824658640 1 -- 172.21.15.19:0/651788049 shutdown_connections 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.018+0000 7f0824658640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080006b4d0 0x7f080006d990 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.018+0000 7f0824658640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f081c0cd610 0x7f081c0cda90 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.018+0000 7f0824658640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f081c158e10 0x7f081c159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.019+0000 7f0824658640 1 --2- 172.21.15.19:0/651788049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08080039b0 0x7f081c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.019+0000 7f0824658640 1 -- 172.21.15.19:0/651788049 >> 172.21.15.19:0/651788049 conn(0x7f081c0cfec0 msgr2=0x7f081c0b4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.019+0000 7f0824658640 1 -- 172.21.15.19:0/651788049 shutdown_connections 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.019+0000 7f0824658640 1 -- 172.21.15.19:0/651788049 wait complete. 2024-04-25T10:00:23.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:00:23.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.187+0000 7f7693734640 1 Processor -- start 2024-04-25T10:00:23.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.188+0000 7f7693734640 1 -- start start 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.188+0000 7f7693734640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f768c1610f0 0x7f768c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.188+0000 7f7693734640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f768c1616f0 0x7f768c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.188+0000 7f7693734640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f768c0ce0a0 0x7f768c0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7693734640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768c1614d0 con 0x7f768c1616f0 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7693734640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768c0c6960 con 0x7f768c1610f0 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7693734640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768c162260 con 0x7f768c0ce0a0 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7690ccf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f768c1616f0 0x7f768c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7690ccf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f768c1616f0 0x7f768c161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33232/0 (socket says 172.21.15.19:33232) 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7690ccf640 1 -- 172.21.15.19:0/527544600 learned_addr learned my addr 172.21.15.19:0/527544600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7691cd1640 1 --2- 172.21.15.19:0/527544600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f768c0ce0a0 0x7f768c0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7690ccf640 1 -- 172.21.15.19:0/527544600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f768c0ce0a0 msgr2=0x7f768c0ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f76914d0640 1 --2- 172.21.15.19:0/527544600 >> v2:172.21.15.112:3300/0 conn(0x7f768c1610f0 0x7f768c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7690ccf640 1 --2- 172.21.15.19:0/527544600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f768c0ce0a0 0x7f768c0ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7690ccf640 1 -- 172.21.15.19:0/527544600 >> v2:172.21.15.112:3300/0 conn(0x7f768c1610f0 msgr2=0x7f768c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7690ccf640 1 --2- 172.21.15.19:0/527544600 >> v2:172.21.15.112:3300/0 conn(0x7f768c1610f0 0x7f768c06c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.189+0000 7f7690ccf640 1 -- 172.21.15.19:0/527544600 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c162540 con 0x7f768c1616f0 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.190+0000 7f7690ccf640 1 --2- 172.21.15.19:0/527544600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c1616f0 0x7f768c161b50 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f768c071fb0 tx=0x7f767c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81635c213f09a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.190+0000 7f7683fff640 1 -- 172.21.15.19:0/527544600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c037070 con 0x7f768c1616f0 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.190+0000 7f7683fff640 1 -- 172.21.15.19:0/527544600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767c032440 con 0x7f768c1616f0 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.190+0000 7f7683fff640 1 -- 172.21.15.19:0/527544600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c032810 con 0x7f768c1616f0 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.190+0000 7f7693734640 1 -- 172.21.15.19:0/527544600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c1616f0 msgr2=0x7f768c161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.190+0000 7f7693734640 1 --2- 172.21.15.19:0/527544600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c1616f0 0x7f768c161b50 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f768c071fb0 tx=0x7f767c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.190+0000 7f7693734640 1 -- 172.21.15.19:0/527544600 shutdown_connections 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.190+0000 7f7693734640 1 --2- 172.21.15.19:0/527544600 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f768c0ce0a0 0x7f768c0ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.190+0000 7f7693734640 1 --2- 172.21.15.19:0/527544600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c1616f0 0x7f768c161b50 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.190+0000 7f7693734640 1 --2- 172.21.15.19:0/527544600 >> v2:172.21.15.112:3300/0 conn(0x7f768c1610f0 0x7f768c06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.190+0000 7f7693734640 1 -- 172.21.15.19:0/527544600 >> 172.21.15.19:0/527544600 conn(0x7f768c06f500 msgr2=0x7f768c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.191+0000 7f7693734640 1 -- 172.21.15.19:0/527544600 shutdown_connections 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.191+0000 7f7693734640 1 -- 172.21.15.19:0/527544600 wait complete. 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.192+0000 7f7693734640 1 Processor -- start 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.192+0000 7f7693734640 1 -- start start 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7693734640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c0ce0a0 0x7f768c1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7693734640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f768c1610f0 0x7f768c1551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7693734640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f768c1616f0 0x7f768c1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7693734640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0cf690 con 0x7f768c0ce0a0 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7693734640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0cfce0 con 0x7f768c1610f0 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7693734640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768c06c700 con 0x7f768c1616f0 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f76914d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c0ce0a0 0x7f768c1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f76914d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c0ce0a0 0x7f768c1500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33240/0 (socket says 172.21.15.19:33240) 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f76914d0640 1 -- 172.21.15.19:0/2407694444 learned_addr learned my addr 172.21.15.19:0/2407694444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7691cd1640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f768c1616f0 0x7f768c1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7690ccf640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f768c1610f0 0x7f768c1551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f76914d0640 1 -- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f768c1616f0 msgr2=0x7f768c1556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f76914d0640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f768c1616f0 0x7f768c1556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f76914d0640 1 -- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f768c1610f0 msgr2=0x7f768c1551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f76914d0640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f768c1610f0 0x7f768c1551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f76914d0640 1 -- 172.21.15.19:0/2407694444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c150630 con 0x7f768c0ce0a0 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f76914d0640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c0ce0a0 0x7f768c1500f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f768800c7f0 tx=0x7f768800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7681ffb640 1 -- 172.21.15.19:0/2407694444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688015070 con 0x7f768c0ce0a0 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7681ffb640 1 -- 172.21.15.19:0/2407694444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7688004240 con 0x7f768c0ce0a0 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7693734640 1 -- 172.21.15.19:0/2407694444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768c1595a0 con 0x7f768c0ce0a0 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.193+0000 7f7681ffb640 1 -- 172.21.15.19:0/2407694444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688007620 con 0x7f768c0ce0a0 2024-04-25T10:00:23.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.194+0000 7f7693734640 1 -- 172.21.15.19:0/2407694444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c15e810 con 0x7f768c0ce0a0 2024-04-25T10:00:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.195+0000 7f7693734640 1 -- 172.21.15.19:0/2407694444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768c073a00 con 0x7f768c0ce0a0 2024-04-25T10:00:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.195+0000 7f7681ffb640 1 -- 172.21.15.19:0/2407694444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76880079d0 con 0x7f768c0ce0a0 2024-04-25T10:00:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.195+0000 7f7681ffb640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f767006b4d0 0x7f767006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.195+0000 7f7681ffb640 1 -- 172.21.15.19:0/2407694444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f768808a760 con 0x7f768c0ce0a0 2024-04-25T10:00:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.195+0000 7f7690ccf640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f767006b4d0 0x7f767006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.196+0000 7f7690ccf640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f767006b4d0 0x7f767006d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f767c034000 tx=0x7f767c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.199+0000 7f7681ffb640 1 -- 172.21.15.19:0/2407694444 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7688059490 con 0x7f768c0ce0a0 2024-04-25T10:00:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:23.378+0000 7f7693734640 1 -- 172.21.15.19:0/2407694444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f768c15d170 con 0x7f768c0ce0a0 2024-04-25T10:00:24.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.018+0000 7f7681ffb640 1 -- 172.21.15.19:0/2407694444 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7688058d90 con 0x7f768c0ce0a0 2024-04-25T10:00:24.022 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:00:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.021+0000 7f7693734640 1 -- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f767006b4d0 msgr2=0x7f767006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.021+0000 7f7693734640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f767006b4d0 0x7f767006d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f767c034000 tx=0x7f767c02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.021+0000 7f7693734640 1 -- 172.21.15.19:0/2407694444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c0ce0a0 msgr2=0x7f768c1500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.021+0000 7f7693734640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c0ce0a0 0x7f768c1500f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f768800c7f0 tx=0x7f768800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.022+0000 7f7693734640 1 -- 172.21.15.19:0/2407694444 shutdown_connections 2024-04-25T10:00:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.022+0000 7f7693734640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f767006b4d0 0x7f767006d990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.022+0000 7f7693734640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f768c1616f0 0x7f768c1556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.022+0000 7f7693734640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f768c1610f0 0x7f768c1551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.022+0000 7f7693734640 1 --2- 172.21.15.19:0/2407694444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c0ce0a0 0x7f768c1500f0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.022+0000 7f7693734640 1 -- 172.21.15.19:0/2407694444 >> 172.21.15.19:0/2407694444 conn(0x7f768c06f500 msgr2=0x7f768c15df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:24.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.022+0000 7f7693734640 1 -- 172.21.15.19:0/2407694444 shutdown_connections 2024-04-25T10:00:24.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.022+0000 7f7693734640 1 -- 172.21.15.19:0/2407694444 wait complete. 2024-04-25T10:00:24.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:00:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.147+0000 7f9c38c15640 1 Processor -- start 2024-04-25T10:00:24.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.147+0000 7f9c38c15640 1 -- start start 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.148+0000 7f9c38c15640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3415b7d0 0x7f9c340cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.148+0000 7f9c38c15640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c3406c9e0 0x7f9c340b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.148+0000 7f9c38c15640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3406cf50 con 0x7f9c340b5620 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.148+0000 7f9c38c15640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c340b5fc0 con 0x7f9c3406c9e0 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.148+0000 7f9c38c15640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340ce530 con 0x7f9c3415b7d0 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.148+0000 7f9c32d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c3406c9e0 0x7f9c340b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.148+0000 7f9c32d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c3406c9e0 0x7f9c340b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33242/0 (socket says 172.21.15.19:33242) 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.148+0000 7f9c32d9d640 1 -- 172.21.15.19:0/3450249990 learned_addr learned my addr 172.21.15.19:0/3450249990 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.148+0000 7f9c3259c640 1 --2- 172.21.15.19:0/3450249990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3415b7d0 0x7f9c340cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.149+0000 7f9c32d9d640 1 -- 172.21.15.19:0/3450249990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3415b7d0 msgr2=0x7f9c340cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.149+0000 7f9c32d9d640 1 --2- 172.21.15.19:0/3450249990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3415b7d0 0x7f9c340cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.149+0000 7f9c32d9d640 1 -- 172.21.15.19:0/3450249990 >> v1:172.21.15.112:6789/0 conn(0x7f9c340b5620 legacy=0x7f9c3406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.149+0000 7f9c32d9d640 1 -- 172.21.15.19:0/3450249990 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340ce810 con 0x7f9c3406c9e0 2024-04-25T10:00:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.149+0000 7f9c32d9d640 1 --2- 172.21.15.19:0/3450249990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3406c9e0 0x7f9c340b5a80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009e50 tx=0x7f9c2402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecad0ac4f6a5141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.149+0000 7f9c3159a640 1 -- 172.21.15.19:0/3450249990 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24036070 con 0x7f9c3406c9e0 2024-04-25T10:00:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.149+0000 7f9c3159a640 1 -- 172.21.15.19:0/3450249990 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24031600 con 0x7f9c3406c9e0 2024-04-25T10:00:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.149+0000 7f9c3159a640 1 -- 172.21.15.19:0/3450249990 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240319d0 con 0x7f9c3406c9e0 2024-04-25T10:00:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.150+0000 7f9c38c15640 1 -- 172.21.15.19:0/3450249990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3406c9e0 msgr2=0x7f9c340b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.150+0000 7f9c38c15640 1 --2- 172.21.15.19:0/3450249990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3406c9e0 0x7f9c340b5a80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009e50 tx=0x7f9c2402a710 comp rx=0 tx=0).stop 2024-04-25T10:00:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.150+0000 7f9c38c15640 1 -- 172.21.15.19:0/3450249990 shutdown_connections 2024-04-25T10:00:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.150+0000 7f9c38c15640 1 --2- 172.21.15.19:0/3450249990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3406c9e0 0x7f9c340b5a80 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.150+0000 7f9c38c15640 1 --2- 172.21.15.19:0/3450249990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3415b7d0 0x7f9c340cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.150+0000 7f9c38c15640 1 -- 172.21.15.19:0/3450249990 >> 172.21.15.19:0/3450249990 conn(0x7f9c340d0800 msgr2=0x7f9c340d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:24.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.150+0000 7f9c38c15640 1 -- 172.21.15.19:0/3450249990 shutdown_connections 2024-04-25T10:00:24.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.150+0000 7f9c38c15640 1 -- 172.21.15.19:0/3450249990 wait complete. 2024-04-25T10:00:24.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.151+0000 7f9c38c15640 1 Processor -- start 2024-04-25T10:00:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.151+0000 7f9c38c15640 1 -- start start 2024-04-25T10:00:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.152+0000 7f9c38c15640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3406c9e0 0x7f9c340cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.152+0000 7f9c38c15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c340b5620 0x7f9c341f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.152+0000 7f9c38c15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3415b7d0 0x7f9c34147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.152+0000 7f9c38c15640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340cf550 con 0x7f9c340b5620 2024-04-25T10:00:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.152+0000 7f9c38c15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3406d230 con 0x7f9c3415b7d0 2024-04-25T10:00:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.152+0000 7f9c38c15640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340cf6d0 con 0x7f9c3406c9e0 2024-04-25T10:00:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.152+0000 7f9c31d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c340b5620 0x7f9c341f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.152+0000 7f9c31d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c340b5620 0x7f9c341f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33254/0 (socket says 172.21.15.19:33254) 2024-04-25T10:00:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.152+0000 7f9c31d9b640 1 -- 172.21.15.19:0/3943101095 learned_addr learned my addr 172.21.15.19:0/3943101095 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:24.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.152+0000 7f9c32d9d640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3415b7d0 0x7f9c34147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.152+0000 7f9c3259c640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3406c9e0 0x7f9c340cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.153+0000 7f9c31d9b640 1 -- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3406c9e0 msgr2=0x7f9c340cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.153+0000 7f9c31d9b640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3406c9e0 0x7f9c340cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.153+0000 7f9c31d9b640 1 -- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3415b7d0 msgr2=0x7f9c34147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.153+0000 7f9c31d9b640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3415b7d0 0x7f9c34147de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.153+0000 7f9c31d9b640 1 -- 172.21.15.19:0/3943101095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c341488a0 con 0x7f9c340b5620 2024-04-25T10:00:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.153+0000 7f9c31d9b640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c340b5620 0x7f9c341f66d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800c400 tx=0x7f9c2800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:24.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.153+0000 7f9c22ffd640 1 -- 172.21.15.19:0/3943101095 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c280052d0 con 0x7f9c340b5620 2024-04-25T10:00:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.153+0000 7f9c22ffd640 1 -- 172.21.15.19:0/3943101095 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c28005490 con 0x7f9c340b5620 2024-04-25T10:00:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.153+0000 7f9c38c15640 1 -- 172.21.15.19:0/3943101095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c34148b80 con 0x7f9c340b5620 2024-04-25T10:00:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.153+0000 7f9c22ffd640 1 -- 172.21.15.19:0/3943101095 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28007260 con 0x7f9c340b5620 2024-04-25T10:00:24.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.153+0000 7f9c38c15640 1 -- 172.21.15.19:0/3943101095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c340cd450 con 0x7f9c340b5620 2024-04-25T10:00:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.154+0000 7f9c38c15640 1 -- 172.21.15.19:0/3943101095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c34073a00 con 0x7f9c340b5620 2024-04-25T10:00:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.154+0000 7f9c22ffd640 1 -- 172.21.15.19:0/3943101095 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c28007400 con 0x7f9c340b5620 2024-04-25T10:00:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.155+0000 7f9c22ffd640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c1406b4d0 0x7f9c1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.155+0000 7f9c22ffd640 1 -- 172.21.15.19:0/3943101095 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f9c28089620 con 0x7f9c340b5620 2024-04-25T10:00:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.155+0000 7f9c3259c640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c1406b4d0 0x7f9c1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.156+0000 7f9c3259c640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c1406b4d0 0x7f9c1406d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0079a0 tx=0x7f9c2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:24.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.159+0000 7f9c22ffd640 1 -- 172.21.15.19:0/3943101095 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c28057eb0 con 0x7f9c340b5620 2024-04-25T10:00:24.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:24.339+0000 7f9c38c15640 1 -- 172.21.15.19:0/3943101095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9c340b9280 con 0x7f9c340b5620 2024-04-25T10:00:25.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.019+0000 7f9c22ffd640 1 -- 172.21.15.19:0/3943101095 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9c2805bf50 con 0x7f9c340b5620 2024-04-25T10:00:25.023 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T10:00:25.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 -- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c1406b4d0 msgr2=0x7f9c1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c1406b4d0 0x7f9c1406d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0079a0 tx=0x7f9c2c008040 comp rx=0 tx=0).stop 2024-04-25T10:00:25.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 -- 172.21.15.19:0/3943101095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c340b5620 msgr2=0x7f9c341f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c340b5620 0x7f9c341f66d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800c400 tx=0x7f9c2800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 -- 172.21.15.19:0/3943101095 shutdown_connections 2024-04-25T10:00:25.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c1406b4d0 0x7f9c1406d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3415b7d0 0x7f9c34147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c340b5620 0x7f9c341f66d0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 --2- 172.21.15.19:0/3943101095 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3406c9e0 0x7f9c340cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 -- 172.21.15.19:0/3943101095 >> 172.21.15.19:0/3943101095 conn(0x7f9c340d0800 msgr2=0x7f9c340d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:25.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 -- 172.21.15.19:0/3943101095 shutdown_connections 2024-04-25T10:00:25.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.022+0000 7f9c38c15640 1 -- 172.21.15.19:0/3943101095 wait complete. 2024-04-25T10:00:25.046 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:00:25.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.184+0000 7fedfa770640 1 Processor -- start 2024-04-25T10:00:25.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.184+0000 7fedfa770640 1 -- start start 2024-04-25T10:00:25.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.185+0000 7fedfa770640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf40cef10 0x7fedf40cc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.185+0000 7fedfa770640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf40cd9f0 con 0x7fedf4154150 2024-04-25T10:00:25.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.185+0000 7fedfa770640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf415b1c0 con 0x7fedf40cc680 2024-04-25T10:00:25.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.185+0000 7fedfa770640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf406f0d0 con 0x7fedf40cef10 2024-04-25T10:00:25.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.186+0000 7fedf3fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fedf4154150 0x7fedf40cd2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44026/0 (socket says 172.21.15.19:44026) 2024-04-25T10:00:25.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.186+0000 7fedf8d0d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fedf40cc680 0x7fedf406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:34508/0 (socket says 172.21.15.19:34508) 2024-04-25T10:00:25.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.186+0000 7fedf3fff640 1 -- 172.21.15.19:0/858384041 learned_addr learned my addr 172.21.15.19:0/858384041 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.186+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346928071 0 0) 0x7fedf415b1c0 con 0x7fedf40cc680 2024-04-25T10:00:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.186+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feddc003520 con 0x7fedf40cc680 2024-04-25T10:00:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624406731 0 0) 0x7fedf40cd9f0 con 0x7fedf4154150 2024-04-25T10:00:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feddc003880 con 0x7fedf4154150 2024-04-25T10:00:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 <== mon.1 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 93687096 0 0) 0x7feddc003520 con 0x7fedf40cc680 2024-04-25T10:00:25.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf40cef10 msgr2=0x7fedf40cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 --2- 172.21.15.19:0/858384041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf40cef10 0x7fedf40cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 >> v1:172.21.15.19:6789/0 conn(0x7fedf4154150 legacy=0x7fedf40cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf415a2a0 con 0x7fedf40cc680 2024-04-25T10:00:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 <== mon.1 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fedec001250 con 0x7fedf40cc680 2024-04-25T10:00:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 >> v1:172.21.15.112:6789/0 conn(0x7fedf40cc680 legacy=0x7fedf406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 --2- 172.21.15.19:0/858384041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feddc003d90 0x7feddc0041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 --2- 172.21.15.19:0/858384041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feddc0079a0 0x7feddc009dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 --2- 172.21.15.19:0/858384041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feddc00a670 0x7feddc00cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf406f0d0 con 0x7feddc0079a0 2024-04-25T10:00:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc00d3e0 con 0x7feddc00a670 2024-04-25T10:00:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc00d5e0 con 0x7feddc003d90 2024-04-25T10:00:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf37fe640 1 --2- 172.21.15.19:0/858384041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feddc0079a0 0x7feddc009dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.187+0000 7fedf8d0d640 1 --2- 172.21.15.19:0/858384041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feddc00a670 0x7feddc00cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.188+0000 7fedf3fff640 1 --2- 172.21.15.19:0/858384041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feddc003d90 0x7feddc0041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.188+0000 7fedf37fe640 1 -- 172.21.15.19:0/858384041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feddc003d90 msgr2=0x7feddc0041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.188+0000 7fedf37fe640 1 --2- 172.21.15.19:0/858384041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feddc003d90 0x7feddc0041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.188+0000 7fedf37fe640 1 -- 172.21.15.19:0/858384041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feddc00a670 msgr2=0x7feddc00cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.188+0000 7fedf37fe640 1 --2- 172.21.15.19:0/858384041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feddc00a670 0x7feddc00cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.188+0000 7fedf37fe640 1 -- 172.21.15.19:0/858384041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf415a2a0 con 0x7feddc0079a0 2024-04-25T10:00:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.189+0000 7fedf37fe640 1 --2- 172.21.15.19:0/858384041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feddc0079a0 0x7feddc009dd0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fedf406e8a0 tx=0x7feddc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d08616e4e74c42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.189+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede400b670 con 0x7feddc0079a0 2024-04-25T10:00:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.189+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede4033600 con 0x7feddc0079a0 2024-04-25T10:00:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.189+0000 7fedf2ffd640 1 -- 172.21.15.19:0/858384041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede400b670 con 0x7feddc0079a0 2024-04-25T10:00:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.190+0000 7fedfa770640 1 -- 172.21.15.19:0/858384041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feddc0079a0 msgr2=0x7feddc009dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.190+0000 7fedfa770640 1 --2- 172.21.15.19:0/858384041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feddc0079a0 0x7feddc009dd0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fedf406e8a0 tx=0x7feddc005830 comp rx=0 tx=0).stop 2024-04-25T10:00:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.191+0000 7fedf8d0d640 1 -- 172.21.15.19:0/858384041 reap_dead start 2024-04-25T10:00:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.191+0000 7fedfa770640 1 -- 172.21.15.19:0/858384041 shutdown_connections 2024-04-25T10:00:25.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.191+0000 7fedfa770640 1 -- 172.21.15.19:0/858384041 >> 172.21.15.19:0/858384041 conn(0x7fedf406abe0 msgr2=0x7fedf406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:25.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.193+0000 7fedfa770640 1 -- 172.21.15.19:0/858384041 shutdown_connections 2024-04-25T10:00:25.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.193+0000 7fedfa770640 1 -- 172.21.15.19:0/858384041 wait complete. 2024-04-25T10:00:25.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.195+0000 7fedfa770640 1 Processor -- start 2024-04-25T10:00:25.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.196+0000 7fedfa770640 1 -- start start 2024-04-25T10:00:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.197+0000 7fedfa770640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf41f2810 0x7fedf4158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.197+0000 7fedfa770640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf41f2bf0 0x7fedf42003c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.197+0000 7fedfa770640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedf4200900 0x7fedf4156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.197+0000 7fedfa770640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf415ab00 con 0x7fedf4200900 2024-04-25T10:00:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.197+0000 7fedfa770640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf415a820 con 0x7fedf41f2810 2024-04-25T10:00:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.197+0000 7fedfa770640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf415ade0 con 0x7fedf41f2bf0 2024-04-25T10:00:25.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.198+0000 7fedf37fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf41f2bf0 0x7fedf42003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.198+0000 7fedf37fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf41f2bf0 0x7fedf42003c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:46038/0 (socket says 172.21.15.19:46038) 2024-04-25T10:00:25.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.198+0000 7fedf37fe640 1 -- 172.21.15.19:0/3202364190 learned_addr learned my addr 172.21.15.19:0/3202364190 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:25.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.198+0000 7fedf3fff640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf41f2810 0x7fedf4158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.198+0000 7fedf8d0d640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedf4200900 0x7fedf4156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.198+0000 7fedf37fe640 1 -- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf41f2810 msgr2=0x7fedf4158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.198+0000 7fedf37fe640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf41f2810 0x7fedf4158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.198+0000 7fedf37fe640 1 -- 172.21.15.19:0/3202364190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedf4200900 msgr2=0x7fedf4156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.198+0000 7fedf37fe640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedf4200900 0x7fedf4156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.198+0000 7fedf37fe640 1 -- 172.21.15.19:0/3202364190 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf41576a0 con 0x7fedf41f2bf0 2024-04-25T10:00:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.199+0000 7fedf8d0d640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedf4200900 0x7fedf4156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:25.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.199+0000 7fedf37fe640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf41f2bf0 0x7fedf42003c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fede400bae0 tx=0x7fede40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:25.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.200+0000 7fedf0ff9640 1 -- 172.21.15.19:0/3202364190 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4041020 con 0x7fedf41f2bf0 2024-04-25T10:00:25.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.200+0000 7fedfa770640 1 -- 172.21.15.19:0/3202364190 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf4157980 con 0x7fedf41f2bf0 2024-04-25T10:00:25.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.200+0000 7fedf0ff9640 1 -- 172.21.15.19:0/3202364190 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede4032a50 con 0x7fedf41f2bf0 2024-04-25T10:00:25.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.200+0000 7fedf0ff9640 1 -- 172.21.15.19:0/3202364190 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede400b670 con 0x7fedf41f2bf0 2024-04-25T10:00:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.200+0000 7fedfa770640 1 -- 172.21.15.19:0/3202364190 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf4157f40 con 0x7fedf41f2bf0 2024-04-25T10:00:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.201+0000 7fedfa770640 1 -- 172.21.15.19:0/3202364190 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf4073a00 con 0x7fedf41f2bf0 2024-04-25T10:00:25.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.202+0000 7fedf0ff9640 1 -- 172.21.15.19:0/3202364190 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fede4044020 con 0x7fedf41f2bf0 2024-04-25T10:00:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.202+0000 7fedf0ff9640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fede006b4d0 0x7fede006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.202+0000 7fedf0ff9640 1 -- 172.21.15.19:0/3202364190 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fede40aaf40 con 0x7fedf41f2bf0 2024-04-25T10:00:25.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.202+0000 7fedf3fff640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fede006b4d0 0x7fede006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.203+0000 7fedf3fff640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fede006b4d0 0x7fede006d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fede8007fd0 tx=0x7fede8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:25.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.207+0000 7fedf0ff9640 1 -- 172.21.15.19:0/3202364190 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fede404b030 con 0x7fedf41f2bf0 2024-04-25T10:00:25.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.386+0000 7fedfa770640 1 -- 172.21.15.19:0/3202364190 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fedf40713a0 con 0x7fedf41f2bf0 2024-04-25T10:00:25.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.390+0000 7fedf0ff9640 1 -- 172.21.15.19:0/3202364190 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fede4032e20 con 0x7fedf41f2bf0 2024-04-25T10:00:25.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.392+0000 7fedfa770640 1 -- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fede006b4d0 msgr2=0x7fede006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.392+0000 7fedfa770640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fede006b4d0 0x7fede006d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fede8007fd0 tx=0x7fede8009040 comp rx=0 tx=0).stop 2024-04-25T10:00:25.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.392+0000 7fedfa770640 1 -- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf41f2bf0 msgr2=0x7fedf42003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.392+0000 7fedfa770640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf41f2bf0 0x7fedf42003c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fede400bae0 tx=0x7fede40046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.393+0000 7fedfa770640 1 -- 172.21.15.19:0/3202364190 shutdown_connections 2024-04-25T10:00:25.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.393+0000 7fedfa770640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fede006b4d0 0x7fede006d990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.393+0000 7fedfa770640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedf4200900 0x7fedf4156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.393+0000 7fedfa770640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf41f2bf0 0x7fedf42003c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.393+0000 7fedfa770640 1 --2- 172.21.15.19:0/3202364190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf41f2810 0x7fedf4158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.393+0000 7fedfa770640 1 -- 172.21.15.19:0/3202364190 >> 172.21.15.19:0/3202364190 conn(0x7fedf406abe0 msgr2=0x7fedf415cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:25.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.393+0000 7fedfa770640 1 -- 172.21.15.19:0/3202364190 shutdown_connections 2024-04-25T10:00:25.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.393+0000 7fedfa770640 1 -- 172.21.15.19:0/3202364190 wait complete. 2024-04-25T10:00:25.408 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T10:00:25.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.506+0000 7fb2ef41f640 1 Processor -- start 2024-04-25T10:00:25.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.506+0000 7fb2ef41f640 1 -- start start 2024-04-25T10:00:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2ef41f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb2e80ce590 0x7fb2e8165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2ef41f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e8071480 con 0x7fb2e80cfec0 2024-04-25T10:00:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2ef41f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e8071760 con 0x7fb2e8154ee0 2024-04-25T10:00:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2ef41f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e8072090 con 0x7fb2e80ce590 2024-04-25T10:00:25.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2ec9ba640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb2e80cfec0 0x7fb2e80cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44038/0 (socket says 172.21.15.19:44038) 2024-04-25T10:00:25.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2ec9ba640 1 -- 172.21.15.19:0/3589376152 learned_addr learned my addr 172.21.15.19:0/3589376152 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:25.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2ed9bc640 1 --2- 172.21.15.19:0/3589376152 >> v2:172.21.15.112:3300/0 conn(0x7fb2e80ce590 0x7fb2e8165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2dffff640 1 -- 172.21.15.19:0/3589376152 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276619304 0 0) 0x7fb2e8071480 con 0x7fb2e80cfec0 2024-04-25T10:00:25.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2dffff640 1 -- 172.21.15.19:0/3589376152 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d4003520 con 0x7fb2e80cfec0 2024-04-25T10:00:25.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2ed9bc640 1 -- 172.21.15.19:0/3589376152 >> v1:172.21.15.112:6789/0 conn(0x7fb2e8154ee0 legacy=0x7fb2e80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2ed9bc640 1 -- 172.21.15.19:0/3589376152 >> v1:172.21.15.19:6789/0 conn(0x7fb2e80cfec0 legacy=0x7fb2e80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2dffff640 1 -- 172.21.15.19:0/3589376152 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713745543 0 0) 0x7fb2e8071760 con 0x7fb2e8154ee0 2024-04-25T10:00:25.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.507+0000 7fb2ed9bc640 1 -- 172.21.15.19:0/3589376152 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e8071a40 con 0x7fb2e80ce590 2024-04-25T10:00:25.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.508+0000 7fb2ed9bc640 1 --2- 172.21.15.19:0/3589376152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e80ce590 0x7fb2e8165900 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009d90 tx=0x7fb2e002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da4d6232b3a1ec28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:25.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.508+0000 7fb2dffff640 1 -- 172.21.15.19:0/3589376152 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0007740 con 0x7fb2e80ce590 2024-04-25T10:00:25.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.508+0000 7fb2dffff640 1 -- 172.21.15.19:0/3589376152 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e00078e0 con 0x7fb2e80ce590 2024-04-25T10:00:25.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.508+0000 7fb2dffff640 1 -- 172.21.15.19:0/3589376152 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0007cb0 con 0x7fb2e80ce590 2024-04-25T10:00:25.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.509+0000 7fb2ef41f640 1 -- 172.21.15.19:0/3589376152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e80ce590 msgr2=0x7fb2e8165900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.509+0000 7fb2ef41f640 1 --2- 172.21.15.19:0/3589376152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e80ce590 0x7fb2e8165900 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009d90 tx=0x7fb2e002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.509+0000 7fb2ef41f640 1 -- 172.21.15.19:0/3589376152 shutdown_connections 2024-04-25T10:00:25.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.509+0000 7fb2ef41f640 1 --2- 172.21.15.19:0/3589376152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e80ce590 0x7fb2e8165900 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.509+0000 7fb2ef41f640 1 -- 172.21.15.19:0/3589376152 >> 172.21.15.19:0/3589376152 conn(0x7fb2e806abe0 msgr2=0x7fb2e806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:25.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.509+0000 7fb2ef41f640 1 -- 172.21.15.19:0/3589376152 shutdown_connections 2024-04-25T10:00:25.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.509+0000 7fb2ef41f640 1 -- 172.21.15.19:0/3589376152 wait complete. 2024-04-25T10:00:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.510+0000 7fb2ef41f640 1 Processor -- start 2024-04-25T10:00:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.510+0000 7fb2ef41f640 1 -- start start 2024-04-25T10:00:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ef41f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2e80ce590 0x7fb2e8157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ef41f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e80cfec0 0x7fb2e8145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ef41f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2e8154ee0 0x7fb2e81461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ef41f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e81552c0 con 0x7fb2e8154ee0 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ef41f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8071230 con 0x7fb2e80cfec0 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ef41f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8155440 con 0x7fb2e80ce590 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ed9bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2e8154ee0 0x7fb2e81461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ed9bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2e8154ee0 0x7fb2e81461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33294/0 (socket says 172.21.15.19:33294) 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ed9bc640 1 -- 172.21.15.19:0/1072603695 learned_addr learned my addr 172.21.15.19:0/1072603695 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ec9ba640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e80cfec0 0x7fb2e8145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ed1bb640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2e80ce590 0x7fb2e8157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ed9bc640 1 -- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2e80ce590 msgr2=0x7fb2e8157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ed9bc640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2e80ce590 0x7fb2e8157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ed9bc640 1 -- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e80cfec0 msgr2=0x7fb2e8145c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ed9bc640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e80cfec0 0x7fb2e8145c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ed9bc640 1 -- 172.21.15.19:0/1072603695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e81e4270 con 0x7fb2e8154ee0 2024-04-25T10:00:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ec9ba640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e80cfec0 0x7fb2e8145c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.511+0000 7fb2ed9bc640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2e8154ee0 0x7fb2e81461c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009d90 tx=0x7fb2e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.512+0000 7fb2ddffb640 1 -- 172.21.15.19:0/1072603695 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0040070 con 0x7fb2e8154ee0 2024-04-25T10:00:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.512+0000 7fb2ddffb640 1 -- 172.21.15.19:0/1072603695 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e0031b40 con 0x7fb2e8154ee0 2024-04-25T10:00:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.512+0000 7fb2ef41f640 1 -- 172.21.15.19:0/1072603695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e81e4550 con 0x7fb2e8154ee0 2024-04-25T10:00:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.512+0000 7fb2ddffb640 1 -- 172.21.15.19:0/1072603695 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e003c420 con 0x7fb2e8154ee0 2024-04-25T10:00:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.512+0000 7fb2ef41f640 1 -- 172.21.15.19:0/1072603695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e8157000 con 0x7fb2e8154ee0 2024-04-25T10:00:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.513+0000 7fb2ef41f640 1 -- 172.21.15.19:0/1072603695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e81e4d20 con 0x7fb2e8154ee0 2024-04-25T10:00:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.513+0000 7fb2ddffb640 1 -- 172.21.15.19:0/1072603695 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2e0039040 con 0x7fb2e8154ee0 2024-04-25T10:00:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.514+0000 7fb2ddffb640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb2cc06b4d0 0x7fb2cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.514+0000 7fb2ddffb640 1 -- 172.21.15.19:0/1072603695 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb2e00ab370 con 0x7fb2e8154ee0 2024-04-25T10:00:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.514+0000 7fb2ed1bb640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb2cc06b4d0 0x7fb2cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.514+0000 7fb2ed1bb640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb2cc06b4d0 0x7fb2cc06d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4000f60 tx=0x7fb2e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:25.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.518+0000 7fb2ddffb640 1 -- 172.21.15.19:0/1072603695 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2e0079da0 con 0x7fb2e8154ee0 2024-04-25T10:00:25.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:25.769+0000 7fb2ef41f640 1 -- 172.21.15.19:0/1072603695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7fb2e8073990 con 0x7fb2e8154ee0 2024-04-25T10:00:26.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.020+0000 7fb2ddffb640 1 -- 172.21.15.19:0/1072603695 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v364) v1 ==== 155+0+0 (secure 0 0 0) 0x7fb2e007de40 con 0x7fb2e8154ee0 2024-04-25T10:00:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.022+0000 7fb2ef41f640 1 -- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb2cc06b4d0 msgr2=0x7fb2cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.022+0000 7fb2ef41f640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb2cc06b4d0 0x7fb2cc06d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4000f60 tx=0x7fb2e4009040 comp rx=0 tx=0).stop 2024-04-25T10:00:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.022+0000 7fb2ef41f640 1 -- 172.21.15.19:0/1072603695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2e8154ee0 msgr2=0x7fb2e81461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.022+0000 7fb2ef41f640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2e8154ee0 0x7fb2e81461c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009d90 tx=0x7fb2e0004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.022+0000 7fb2ef41f640 1 -- 172.21.15.19:0/1072603695 shutdown_connections 2024-04-25T10:00:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.022+0000 7fb2ef41f640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb2cc06b4d0 0x7fb2cc06d990 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.022+0000 7fb2ef41f640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2e8154ee0 0x7fb2e81461c0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.022+0000 7fb2ef41f640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e80cfec0 0x7fb2e8145c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.022+0000 7fb2ef41f640 1 --2- 172.21.15.19:0/1072603695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2e80ce590 0x7fb2e8157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.022+0000 7fb2ef41f640 1 -- 172.21.15.19:0/1072603695 >> 172.21.15.19:0/1072603695 conn(0x7fb2e806abe0 msgr2=0x7fb2e81540f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.023+0000 7fb2ef41f640 1 -- 172.21.15.19:0/1072603695 shutdown_connections 2024-04-25T10:00:26.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.023+0000 7fb2ef41f640 1 -- 172.21.15.19:0/1072603695 wait complete. 2024-04-25T10:00:26.038 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:00:26.038 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:00:26.039 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:26.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.137+0000 7fe16d21f640 1 Processor -- start 2024-04-25T10:00:26.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.137+0000 7fe16d21f640 1 -- start start 2024-04-25T10:00:26.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.137+0000 7fe16d21f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe16806b1a0 0x7fe16806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.137+0000 7fe16d21f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1680d0170 0x7fe16815d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.137+0000 7fe16d21f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe168161960 con 0x7fe16806b580 2024-04-25T10:00:26.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.137+0000 7fe16d21f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1680cbd50 con 0x7fe16806b1a0 2024-04-25T10:00:26.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.137+0000 7fe16d21f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe168161c40 con 0x7fe1680d0170 2024-04-25T10:00:26.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.137+0000 7fe166d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe16806b1a0 0x7fe16806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.138+0000 7fe15ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe16806b580 0x7fe1680cfa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44040/0 (socket says 172.21.15.19:44040) 2024-04-25T10:00:26.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.138+0000 7fe15ffff640 1 -- 172.21.15.19:0/1948951046 learned_addr learned my addr 172.21.15.19:0/1948951046 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:26.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.138+0000 7fe16759e640 1 --2- 172.21.15.19:0/1948951046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1680d0170 0x7fe16815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.138+0000 7fe166d9d640 1 -- 172.21.15.19:0/1948951046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1680d0170 msgr2=0x7fe16815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.138+0000 7fe166d9d640 1 --2- 172.21.15.19:0/1948951046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1680d0170 0x7fe16815d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.138+0000 7fe166d9d640 1 -- 172.21.15.19:0/1948951046 >> v1:172.21.15.19:6789/0 conn(0x7fe16806b580 legacy=0x7fe1680cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.138+0000 7fe16659c640 1 -- 172.21.15.19:0/1948951046 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520672623 0 0) 0x7fe168161960 con 0x7fe16806b580 2024-04-25T10:00:26.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.138+0000 7fe166d9d640 1 -- 172.21.15.19:0/1948951046 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe168161f20 con 0x7fe16806b1a0 2024-04-25T10:00:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe166d9d640 1 --2- 172.21.15.19:0/1948951046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe16806b1a0 0x7fe16806f9c0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fe1580097a0 tx=0x7fe15802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36ca762f713d772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe16659c640 1 -- 172.21.15.19:0/1948951046 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158037070 con 0x7fe16806b1a0 2024-04-25T10:00:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe16659c640 1 -- 172.21.15.19:0/1948951046 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15802ae50 con 0x7fe16806b1a0 2024-04-25T10:00:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe16659c640 1 -- 172.21.15.19:0/1948951046 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158032660 con 0x7fe16806b1a0 2024-04-25T10:00:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe16d21f640 1 -- 172.21.15.19:0/1948951046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe16806b1a0 msgr2=0x7fe16806f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe16d21f640 1 --2- 172.21.15.19:0/1948951046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe16806b1a0 0x7fe16806f9c0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fe1580097a0 tx=0x7fe15802a100 comp rx=0 tx=0).stop 2024-04-25T10:00:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe16d21f640 1 -- 172.21.15.19:0/1948951046 shutdown_connections 2024-04-25T10:00:26.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe16d21f640 1 --2- 172.21.15.19:0/1948951046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1680d0170 0x7fe16815d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe16d21f640 1 --2- 172.21.15.19:0/1948951046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe16806b1a0 0x7fe16806f9c0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe16d21f640 1 -- 172.21.15.19:0/1948951046 >> 172.21.15.19:0/1948951046 conn(0x7fe1680b2220 msgr2=0x7fe1680b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe16d21f640 1 -- 172.21.15.19:0/1948951046 shutdown_connections 2024-04-25T10:00:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.139+0000 7fe16d21f640 1 -- 172.21.15.19:0/1948951046 wait complete. 2024-04-25T10:00:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.141+0000 7fe16d21f640 1 Processor -- start 2024-04-25T10:00:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.141+0000 7fe16d21f640 1 -- start start 2024-04-25T10:00:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.141+0000 7fe16d21f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe16806b1a0 0x7fe1680c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.141+0000 7fe16d21f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe16806b580 0x7fe1680b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.141+0000 7fe16d21f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1680d0170 0x7fe1681505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.141+0000 7fe16d21f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe168070330 con 0x7fe1680d0170 2024-04-25T10:00:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.141+0000 7fe16d21f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe168162720 con 0x7fe16806b1a0 2024-04-25T10:00:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.141+0000 7fe16d21f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16806ff00 con 0x7fe16806b580 2024-04-25T10:00:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.141+0000 7fe16759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1680d0170 0x7fe1681505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe16759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1680d0170 0x7fe1681505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33308/0 (socket says 172.21.15.19:33308) 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe16759e640 1 -- 172.21.15.19:0/3696028080 learned_addr learned my addr 172.21.15.19:0/3696028080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe15ffff640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe16806b580 0x7fe1680b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe166d9d640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe16806b1a0 0x7fe1680c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe16759e640 1 -- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe16806b580 msgr2=0x7fe1680b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe16759e640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe16806b580 0x7fe1680b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe16759e640 1 -- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe16806b1a0 msgr2=0x7fe1680c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe16759e640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe16806b1a0 0x7fe1680c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe16759e640 1 -- 172.21.15.19:0/3696028080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680b40b0 con 0x7fe1680d0170 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe16759e640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1680d0170 0x7fe1681505f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe16000cd00 tx=0x7fe16000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe15f7fe640 1 -- 172.21.15.19:0/3696028080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe160003b90 con 0x7fe1680d0170 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe15f7fe640 1 -- 172.21.15.19:0/3696028080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe160003d50 con 0x7fe1680d0170 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe16d21f640 1 -- 172.21.15.19:0/3696028080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1680b4390 con 0x7fe1680d0170 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe15f7fe640 1 -- 172.21.15.19:0/3696028080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe160004260 con 0x7fe1680d0170 2024-04-25T10:00:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.142+0000 7fe16d21f640 1 -- 172.21.15.19:0/3696028080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1680b4950 con 0x7fe1680d0170 2024-04-25T10:00:26.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.143+0000 7fe16d21f640 1 -- 172.21.15.19:0/3696028080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1680afb20 con 0x7fe1680d0170 2024-04-25T10:00:26.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.144+0000 7fe15f7fe640 1 -- 172.21.15.19:0/3696028080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe160004400 con 0x7fe1680d0170 2024-04-25T10:00:26.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.144+0000 7fe15f7fe640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe14806b4d0 0x7fe14806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.144+0000 7fe15f7fe640 1 -- 172.21.15.19:0/3696028080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe16008ab40 con 0x7fe1680d0170 2024-04-25T10:00:26.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.144+0000 7fe166d9d640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe14806b4d0 0x7fe14806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.145+0000 7fe166d9d640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe14806b4d0 0x7fe14806d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe158034000 tx=0x7fe15802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.148+0000 7fe15f7fe640 1 -- 172.21.15.19:0/3696028080 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe160059b30 con 0x7fe1680d0170 2024-04-25T10:00:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.330+0000 7fe16d21f640 1 -- 172.21.15.19:0/3696028080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe16806d370 con 0x7fe1680d0170 2024-04-25T10:00:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.331+0000 7fe15f7fe640 1 -- 172.21.15.19:0/3696028080 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+13291 (secure 0 0 0) 0x7fe160059430 con 0x7fe1680d0170 2024-04-25T10:00:26.335 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:26.335 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:00:25.018508+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/4224193015":"2024-04-26T10:00:22.013444+0000","172.21.15.19:6833/4224193015":"2024-04-26T10:00:22.013444+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.332+0000 7fe16d21f640 1 -- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe14806b4d0 msgr2=0x7fe14806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.332+0000 7fe16d21f640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe14806b4d0 0x7fe14806d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe158034000 tx=0x7fe15802e000 comp rx=0 tx=0).stop 2024-04-25T10:00:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.332+0000 7fe16d21f640 1 -- 172.21.15.19:0/3696028080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1680d0170 msgr2=0x7fe1681505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.332+0000 7fe16d21f640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1680d0170 0x7fe1681505f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe16000cd00 tx=0x7fe16000a660 comp rx=0 tx=0).stop 2024-04-25T10:00:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.332+0000 7fe16d21f640 1 -- 172.21.15.19:0/3696028080 shutdown_connections 2024-04-25T10:00:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.332+0000 7fe16d21f640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe14806b4d0 0x7fe14806d990 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.332+0000 7fe16d21f640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1680d0170 0x7fe1681505f0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.332+0000 7fe16d21f640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe16806b580 0x7fe1680b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.333+0000 7fe16d21f640 1 --2- 172.21.15.19:0/3696028080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe16806b1a0 0x7fe1680c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.333+0000 7fe16d21f640 1 -- 172.21.15.19:0/3696028080 >> 172.21.15.19:0/3696028080 conn(0x7fe1680b2220 msgr2=0x7fe1680b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.333+0000 7fe16d21f640 1 -- 172.21.15.19:0/3696028080 shutdown_connections 2024-04-25T10:00:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.333+0000 7fe16d21f640 1 -- 172.21.15.19:0/3696028080 wait complete. 2024-04-25T10:00:26.348 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:00:26.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:00:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.448+0000 7f5df05c1640 1 Processor -- start 2024-04-25T10:00:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.448+0000 7f5df05c1640 1 -- start start 2024-04-25T10:00:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.449+0000 7f5df05c1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5de80cd4c0 0x7f5de80cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.449+0000 7f5df05c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de815b1c0 0x7f5de815dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.449+0000 7f5df05c1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de815e6e0 con 0x7f5de815e1a0 2024-04-25T10:00:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.449+0000 7f5df05c1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de815b620 con 0x7f5de80cd4c0 2024-04-25T10:00:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.449+0000 7f5df05c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de815e9c0 con 0x7f5de815b1c0 2024-04-25T10:00:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.449+0000 7f5deeb5e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5de815e1a0 0x7f5de8150b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44044/0 (socket says 172.21.15.19:44044) 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.449+0000 7f5deeb5e640 1 -- 172.21.15.19:0/2339903089 learned_addr learned my addr 172.21.15.19:0/2339903089 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.449+0000 7f5dee35d640 1 --2- 172.21.15.19:0/2339903089 >> v2:172.21.15.112:3300/0 conn(0x7f5de80cd4c0 0x7f5de80cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5dedb5c640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de815b1c0 0x7f5de815dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684142041 0 0) 0x7f5de815e6e0 con 0x7f5de815e1a0 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dd0003520 con 0x7f5de815e1a0 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1776429715 0 0) 0x7f5dd0003520 con 0x7f5de815e1a0 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de815b1c0 msgr2=0x7f5de815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de815b1c0 0x7f5de815dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 >> v2:172.21.15.112:3300/0 conn(0x7f5de80cd4c0 msgr2=0x7f5de80cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 --2- 172.21.15.19:0/2339903089 >> v2:172.21.15.112:3300/0 conn(0x7f5de80cd4c0 0x7f5de80cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de815a9c0 con 0x7f5de815e1a0 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f5dd8001a10 con 0x7f5de815e1a0 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5dee35d640 1 --2- 172.21.15.19:0/2339903089 >> v2:172.21.15.112:3300/0 conn(0x7f5de80cd4c0 0x7f5de80cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 >> v1:172.21.15.19:6789/0 conn(0x7f5de815e1a0 legacy=0x7f5de8150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd00039b0 0x7f5dd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd0004800 0x7f5dd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de815b1c0 0x7f5dd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de815b620 con 0x7f5dd00039b0 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de815e9c0 con 0x7f5dd0004800 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd000b9c0 con 0x7f5de815b1c0 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5dee35d640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd00039b0 0x7f5dd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5dedb5c640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd0004800 0x7f5dd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5deeb5e640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de815b1c0 0x7f5dd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5dee35d640 1 -- 172.21.15.19:0/2339903089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de815b1c0 msgr2=0x7f5dd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5dee35d640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de815b1c0 0x7f5dd00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5dee35d640 1 -- 172.21.15.19:0/2339903089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd0004800 msgr2=0x7f5dd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5dee35d640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd0004800 0x7f5dd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.450+0000 7f5dee35d640 1 -- 172.21.15.19:0/2339903089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0003520 con 0x7f5dd00039b0 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.451+0000 7f5dee35d640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd00039b0 0x7f5dd0003dd0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009c90 tx=0x7f5dd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cdb84a4ec55886b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.451+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0036070 con 0x7f5dd00039b0 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.451+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de0031400 con 0x7f5dd00039b0 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.451+0000 7f5ded35b640 1 -- 172.21.15.19:0/2339903089 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0009860 con 0x7f5dd00039b0 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.452+0000 7f5df05c1640 1 -- 172.21.15.19:0/2339903089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd00039b0 msgr2=0x7f5dd0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.452+0000 7f5df05c1640 1 --2- 172.21.15.19:0/2339903089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd00039b0 0x7f5dd0003dd0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009c90 tx=0x7f5dd0005830 comp rx=0 tx=0).stop 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.452+0000 7f5deeb5e640 1 -- 172.21.15.19:0/2339903089 reap_dead start 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.452+0000 7f5df05c1640 1 -- 172.21.15.19:0/2339903089 shutdown_connections 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.452+0000 7f5df05c1640 1 -- 172.21.15.19:0/2339903089 >> 172.21.15.19:0/2339903089 conn(0x7f5de806abe0 msgr2=0x7f5de806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.452+0000 7f5df05c1640 1 -- 172.21.15.19:0/2339903089 shutdown_connections 2024-04-25T10:00:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.452+0000 7f5df05c1640 1 -- 172.21.15.19:0/2339903089 wait complete. 2024-04-25T10:00:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.453+0000 7f5df05c1640 1 Processor -- start 2024-04-25T10:00:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.453+0000 7f5df05c1640 1 -- start start 2024-04-25T10:00:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5df05c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd00039b0 0x7f5de81458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5df05c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de80a84a0 0x7f5de815d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5df05c1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de8157850 0x7f5de8143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5df05c1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de806eca0 con 0x7f5de8157850 2024-04-25T10:00:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5df05c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de815a670 con 0x7f5dd00039b0 2024-04-25T10:00:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5df05c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de806ee20 con 0x7f5de80a84a0 2024-04-25T10:00:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5deeb5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de8157850 0x7f5de8143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5deeb5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de8157850 0x7f5de8143ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33324/0 (socket says 172.21.15.19:33324) 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5deeb5e640 1 -- 172.21.15.19:0/2019556800 learned_addr learned my addr 172.21.15.19:0/2019556800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5dee35d640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd00039b0 0x7f5de81458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5dedb5c640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de80a84a0 0x7f5de815d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5deeb5e640 1 -- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de80a84a0 msgr2=0x7f5de815d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5deeb5e640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de80a84a0 0x7f5de815d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5deeb5e640 1 -- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd00039b0 msgr2=0x7f5de81458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5deeb5e640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd00039b0 0x7f5de81458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5deeb5e640 1 -- 172.21.15.19:0/2019556800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de8157e10 con 0x7f5de8157850 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.454+0000 7f5deeb5e640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de8157850 0x7f5de8143ff0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8000f80 tx=0x7f5dd800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.455+0000 7f5de6ffd640 1 -- 172.21.15.19:0/2019556800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8007a80 con 0x7f5de8157850 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.455+0000 7f5de6ffd640 1 -- 172.21.15.19:0/2019556800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd8018070 con 0x7f5de8157850 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.455+0000 7f5de6ffd640 1 -- 172.21.15.19:0/2019556800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8013420 con 0x7f5de8157850 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.455+0000 7f5df05c1640 1 -- 172.21.15.19:0/2019556800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de8144e10 con 0x7f5de8157850 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.455+0000 7f5df05c1640 1 -- 172.21.15.19:0/2019556800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de8145e30 con 0x7f5de8157850 2024-04-25T10:00:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.456+0000 7f5df05c1640 1 -- 172.21.15.19:0/2019556800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de806e9c0 con 0x7f5de8157850 2024-04-25T10:00:26.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.456+0000 7f5de6ffd640 1 -- 172.21.15.19:0/2019556800 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5dd80135c0 con 0x7f5de8157850 2024-04-25T10:00:26.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.457+0000 7f5de6ffd640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b4d0 0x7f5dd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.457+0000 7f5de6ffd640 1 -- 172.21.15.19:0/2019556800 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5dd808a2e0 con 0x7f5de8157850 2024-04-25T10:00:26.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.457+0000 7f5dee35d640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b4d0 0x7f5dd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.457+0000 7f5dee35d640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b4d0 0x7f5dd406d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5de0033000 tx=0x7f5de0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.461+0000 7f5de6ffd640 1 -- 172.21.15.19:0/2019556800 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5de806e9c0 con 0x7f5de8157850 2024-04-25T10:00:26.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.642+0000 7f5df05c1640 1 -- 172.21.15.19:0/2019556800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5de8073a00 con 0x7f5de8157850 2024-04-25T10:00:26.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.643+0000 7f5de6ffd640 1 -- 172.21.15.19:0/2019556800 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5dd80592d0 con 0x7f5de8157850 2024-04-25T10:00:26.647 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 -- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b4d0 msgr2=0x7f5dd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b4d0 0x7f5dd406d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5de0033000 tx=0x7f5de0008030 comp rx=0 tx=0).stop 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 -- 172.21.15.19:0/2019556800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de8157850 msgr2=0x7f5de8143ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de8157850 0x7f5de8143ff0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8000f80 tx=0x7f5dd800e530 comp rx=0 tx=0).stop 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 -- 172.21.15.19:0/2019556800 shutdown_connections 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b4d0 0x7f5dd406d990 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5de8157850 0x7f5de8143ff0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5de80a84a0 0x7f5de815d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 --2- 172.21.15.19:0/2019556800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dd00039b0 0x7f5de81458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 -- 172.21.15.19:0/2019556800 >> 172.21.15.19:0/2019556800 conn(0x7f5de806abe0 msgr2=0x7f5de80b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 -- 172.21.15.19:0/2019556800 shutdown_connections 2024-04-25T10:00:26.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.644+0000 7f5df05c1640 1 -- 172.21.15.19:0/2019556800 wait complete. 2024-04-25T10:00:26.659 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:00:26.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.756+0000 7fd5c3f3b640 1 Processor -- start 2024-04-25T10:00:26.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.756+0000 7fd5c3f3b640 1 -- start start 2024-04-25T10:00:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c3f3b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0b2cb0 0x7fd5bc0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c3f3b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc0b97a0 con 0x7fd5bc15dbe0 2024-04-25T10:00:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c3f3b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc0d1c50 con 0x7fd5bc0d1020 2024-04-25T10:00:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c3f3b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc15dfc0 con 0x7fd5bc0b2cb0 2024-04-25T10:00:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c1cd7640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd5bc15dbe0 0x7fd5bc0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44056/0 (socket says 172.21.15.19:44056) 2024-04-25T10:00:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c1cd7640 1 -- 172.21.15.19:0/690337952 learned_addr learned my addr 172.21.15.19:0/690337952 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c14d6640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0b2cb0 0x7fd5bc0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084292474 0 0) 0x7fd5bc0b97a0 con 0x7fd5bc15dbe0 2024-04-25T10:00:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5a8003520 con 0x7fd5bc15dbe0 2024-04-25T10:00:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3153848937 0 0) 0x7fd5a8003520 con 0x7fd5bc15dbe0 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0b2cb0 msgr2=0x7fd5bc0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0b2cb0 0x7fd5bc0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 >> v1:172.21.15.112:6789/0 conn(0x7fd5bc0d1020 legacy=0x7fd5bc0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc0d1f30 con 0x7fd5bc15dbe0 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fd5b8001670 con 0x7fd5bc15dbe0 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 >> v1:172.21.15.19:6789/0 conn(0x7fd5bc15dbe0 legacy=0x7fd5bc0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 0x7fd5a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a8004800 0x7fd5a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0b2cb0 0x7fd5a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a800ba70 con 0x7fd5a80039e0 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a800bbf0 con 0x7fd5a8004800 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.757+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a800bdf0 con 0x7fd5bc0b2cb0 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.758+0000 7fd5c1cd7640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 0x7fd5a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.758+0000 7fd5c14d6640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a8004800 0x7fd5a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.758+0000 7fd5c24d8640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0b2cb0 0x7fd5a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.758+0000 7fd5c1cd7640 1 -- 172.21.15.19:0/690337952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0b2cb0 msgr2=0x7fd5a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.758+0000 7fd5c1cd7640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0b2cb0 0x7fd5a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.758+0000 7fd5c1cd7640 1 -- 172.21.15.19:0/690337952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a8004800 msgr2=0x7fd5a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.758+0000 7fd5c1cd7640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5a8004800 0x7fd5a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.758+0000 7fd5c1cd7640 1 -- 172.21.15.19:0/690337952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a8003520 con 0x7fd5a80039e0 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.759+0000 7fd5c1cd7640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 0x7fd5a8003e00 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c210 tx=0x7fd5a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6ca4506d7f8250 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.759+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8007d10 con 0x7fd5a80039e0 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.759+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b8005ad0 con 0x7fd5a80039e0 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.759+0000 7fd5c0cd5640 1 -- 172.21.15.19:0/690337952 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8007070 con 0x7fd5a80039e0 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.759+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/690337952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 msgr2=0x7fd5a8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.759+0000 7fd5c3f3b640 1 --2- 172.21.15.19:0/690337952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 0x7fd5a8003e00 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c210 tx=0x7fd5a8005830 comp rx=0 tx=0).stop 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.759+0000 7fd5c24d8640 1 -- 172.21.15.19:0/690337952 reap_dead start 2024-04-25T10:00:26.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.759+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/690337952 shutdown_connections 2024-04-25T10:00:26.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.759+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/690337952 >> 172.21.15.19:0/690337952 conn(0x7fd5bc061a20 msgr2=0x7fd5bc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.759+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/690337952 shutdown_connections 2024-04-25T10:00:26.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.759+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/690337952 wait complete. 2024-04-25T10:00:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.760+0000 7fd5c3f3b640 1 Processor -- start 2024-04-25T10:00:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.760+0000 7fd5c3f3b640 1 -- start start 2024-04-25T10:00:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.761+0000 7fd5c3f3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 0x7fd5bc0cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.761+0000 7fd5c3f3b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0cd7a0 0x7fd5bc1476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.761+0000 7fd5c3f3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc1e82e0 0x7fd5bc1e8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.761+0000 7fd5c3f3b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0b92a0 con 0x7fd5a80039e0 2024-04-25T10:00:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.761+0000 7fd5c3f3b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0d1a00 con 0x7fd5bc1e82e0 2024-04-25T10:00:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.761+0000 7fd5c3f3b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0b9420 con 0x7fd5bc0cd7a0 2024-04-25T10:00:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.761+0000 7fd5c1cd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 0x7fd5bc0cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.761+0000 7fd5c1cd7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 0x7fd5bc0cb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33338/0 (socket says 172.21.15.19:33338) 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.761+0000 7fd5c1cd7640 1 -- 172.21.15.19:0/237914241 learned_addr learned my addr 172.21.15.19:0/237914241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.761+0000 7fd5c24d8640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc1e82e0 0x7fd5bc1e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.761+0000 7fd5c14d6640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0cd7a0 0x7fd5bc1476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5c1cd7640 1 -- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0cd7a0 msgr2=0x7fd5bc1476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5c1cd7640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0cd7a0 0x7fd5bc1476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5c1cd7640 1 -- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc1e82e0 msgr2=0x7fd5bc1e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5c1cd7640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc1e82e0 0x7fd5bc1e8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5c1cd7640 1 -- 172.21.15.19:0/237914241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc0cba40 con 0x7fd5a80039e0 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5c24d8640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc1e82e0 0x7fd5bc1e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5c14d6640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0cd7a0 0x7fd5bc1476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5c1cd7640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 0x7fd5bc0cb460 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8001970 tx=0x7fd5bc0d1e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5b27fc640 1 -- 172.21.15.19:0/237914241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8040020 con 0x7fd5a80039e0 2024-04-25T10:00:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5b27fc640 1 -- 172.21.15.19:0/237914241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b8001430 con 0x7fd5a80039e0 2024-04-25T10:00:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5b27fc640 1 -- 172.21.15.19:0/237914241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8037040 con 0x7fd5a80039e0 2024-04-25T10:00:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/237914241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc0cbd20 con 0x7fd5a80039e0 2024-04-25T10:00:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.762+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/237914241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc0cdc20 con 0x7fd5a80039e0 2024-04-25T10:00:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.763+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/237914241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5bc073990 con 0x7fd5a80039e0 2024-04-25T10:00:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.763+0000 7fd5b27fc640 1 -- 172.21.15.19:0/237914241 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5b8043020 con 0x7fd5a80039e0 2024-04-25T10:00:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.764+0000 7fd5b27fc640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5a006b4f0 0x7fd5a006d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.764+0000 7fd5b27fc640 1 -- 172.21.15.19:0/237914241 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd5b80a9490 con 0x7fd5a80039e0 2024-04-25T10:00:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.764+0000 7fd5c14d6640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5a006b4f0 0x7fd5a006d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.764+0000 7fd5c14d6640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5a006b4f0 0x7fd5a006d9b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0cc740 tx=0x7fd5ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:26.768+0000 7fd5b27fc640 1 -- 172.21.15.19:0/237914241 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5b8005aa0 con 0x7fd5a80039e0 2024-04-25T10:00:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.016+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/237914241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7fd5bc0b6260 con 0x7fd5a80039e0 2024-04-25T10:00:27.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.025+0000 7fd5b27fc640 1 -- 172.21.15.19:0/237914241 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v365) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd5b8078480 con 0x7fd5a80039e0 2024-04-25T10:00:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.027+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5a006b4f0 msgr2=0x7fd5a006d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.027+0000 7fd5c3f3b640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5a006b4f0 0x7fd5a006d9b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0cc740 tx=0x7fd5ac008040 comp rx=0 tx=0).stop 2024-04-25T10:00:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.027+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/237914241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 msgr2=0x7fd5bc0cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.027+0000 7fd5c3f3b640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 0x7fd5bc0cb460 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8001970 tx=0x7fd5bc0d1e20 comp rx=0 tx=0).stop 2024-04-25T10:00:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.028+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/237914241 shutdown_connections 2024-04-25T10:00:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.028+0000 7fd5c3f3b640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5a006b4f0 0x7fd5a006d9b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.028+0000 7fd5c3f3b640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc1e82e0 0x7fd5bc1e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.028+0000 7fd5c3f3b640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc0cd7a0 0x7fd5bc1476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.028+0000 7fd5c3f3b640 1 --2- 172.21.15.19:0/237914241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a80039e0 0x7fd5bc0cb460 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.028+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/237914241 >> 172.21.15.19:0/237914241 conn(0x7fd5bc061a20 msgr2=0x7fd5bc0b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.028+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/237914241 shutdown_connections 2024-04-25T10:00:27.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.028+0000 7fd5c3f3b640 1 -- 172.21.15.19:0/237914241 wait complete. 2024-04-25T10:00:27.043 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:00:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.144+0000 7f17c6458640 1 Processor -- start 2024-04-25T10:00:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.144+0000 7f17c6458640 1 -- start start 2024-04-25T10:00:27.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17c6458640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f17c015dc20 0x7f17c00d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17c6458640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c00d2120 con 0x7f17c00d1ce0 2024-04-25T10:00:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17c6458640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c00d23a0 con 0x7f17c00ced80 2024-04-25T10:00:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17c6458640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c00d2680 con 0x7f17c015dc20 2024-04-25T10:00:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17bf7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f17c00d1ce0 0x7f17c00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44062/0 (socket says 172.21.15.19:44062) 2024-04-25T10:00:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17bf7fe640 1 -- 172.21.15.19:0/1545832174 learned_addr learned my addr 172.21.15.19:0/1545832174 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17bffff640 1 --2- 172.21.15.19:0/1545832174 >> v2:172.21.15.112:3300/0 conn(0x7f17c015dc20 0x7f17c00d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17beffd640 1 -- 172.21.15.19:0/1545832174 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266354575 0 0) 0x7f17a8001110 con 0x7f17c00d1ce0 2024-04-25T10:00:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17beffd640 1 -- 172.21.15.19:0/1545832174 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17ac003520 con 0x7f17c00d1ce0 2024-04-25T10:00:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17bffff640 1 -- 172.21.15.19:0/1545832174 >> v1:172.21.15.112:6789/0 conn(0x7f17c00ced80 legacy=0x7f17c00b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17bffff640 1 -- 172.21.15.19:0/1545832174 >> v1:172.21.15.19:6789/0 conn(0x7f17c00d1ce0 legacy=0x7f17c00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17beffd640 1 -- 172.21.15.19:0/1545832174 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3925296244 0 0) 0x7f17ac003520 con 0x7f17c00d1ce0 2024-04-25T10:00:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.145+0000 7f17bffff640 1 -- 172.21.15.19:0/1545832174 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c00d2960 con 0x7f17c015dc20 2024-04-25T10:00:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.146+0000 7f17bffff640 1 --2- 172.21.15.19:0/1545832174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c015dc20 0x7f17c00d3440 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f17b00097a0 tx=0x7f17b002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f070424af2635372 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.146+0000 7f17beffd640 1 -- 172.21.15.19:0/1545832174 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0037070 con 0x7f17c015dc20 2024-04-25T10:00:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.146+0000 7f17beffd640 1 -- 172.21.15.19:0/1545832174 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b002abc0 con 0x7f17c015dc20 2024-04-25T10:00:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.146+0000 7f17beffd640 1 -- 172.21.15.19:0/1545832174 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8001840 con 0x7f17c015dc20 2024-04-25T10:00:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.147+0000 7f17c6458640 1 -- 172.21.15.19:0/1545832174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c015dc20 msgr2=0x7f17c00d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.147+0000 7f17c6458640 1 --2- 172.21.15.19:0/1545832174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c015dc20 0x7f17c00d3440 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f17b00097a0 tx=0x7f17b002a130 comp rx=0 tx=0).stop 2024-04-25T10:00:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.147+0000 7f17c6458640 1 -- 172.21.15.19:0/1545832174 shutdown_connections 2024-04-25T10:00:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.147+0000 7f17c6458640 1 --2- 172.21.15.19:0/1545832174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c015dc20 0x7f17c00d3440 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.147+0000 7f17c6458640 1 -- 172.21.15.19:0/1545832174 >> 172.21.15.19:0/1545832174 conn(0x7f17c0061a20 msgr2=0x7f17c0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.147+0000 7f17c6458640 1 -- 172.21.15.19:0/1545832174 shutdown_connections 2024-04-25T10:00:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.147+0000 7f17c6458640 1 -- 172.21.15.19:0/1545832174 wait complete. 2024-04-25T10:00:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.148+0000 7f17c6458640 1 Processor -- start 2024-04-25T10:00:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.148+0000 7f17c6458640 1 -- start start 2024-04-25T10:00:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17c6458640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c00ced80 0x7f17c00cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17c6458640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c00d1ce0 0x7f17c0160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17c6458640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c015dc20 0x7f17c014c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17c6458640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00b9490 con 0x7f17c00ced80 2024-04-25T10:00:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17c6458640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00d2f20 con 0x7f17c00d1ce0 2024-04-25T10:00:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17c6458640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00b9610 con 0x7f17c015dc20 2024-04-25T10:00:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c00ced80 0x7f17c00cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c00ced80 0x7f17c00cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33352/0 (socket says 172.21.15.19:33352) 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17bffff640 1 -- 172.21.15.19:0/250594293 learned_addr learned my addr 172.21.15.19:0/250594293 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17c49f5640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c015dc20 0x7f17c014c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17bf7fe640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c00d1ce0 0x7f17c0160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17bffff640 1 -- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c015dc20 msgr2=0x7f17c014c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17bffff640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c015dc20 0x7f17c014c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17bffff640 1 -- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c00d1ce0 msgr2=0x7f17c0160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.149+0000 7f17bffff640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c00d1ce0 0x7f17c0160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.150+0000 7f17bffff640 1 -- 172.21.15.19:0/250594293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c00cc970 con 0x7f17c00ced80 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.150+0000 7f17c49f5640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c015dc20 0x7f17c014c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.150+0000 7f17bffff640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c00ced80 0x7f17c00cd730 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f17b0035040 tx=0x7f17b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.150+0000 7f17bcff9640 1 -- 172.21.15.19:0/250594293 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0041070 con 0x7f17c00ced80 2024-04-25T10:00:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.150+0000 7f17bcff9640 1 -- 172.21.15.19:0/250594293 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b003c660 con 0x7f17c00ced80 2024-04-25T10:00:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.150+0000 7f17bcff9640 1 -- 172.21.15.19:0/250594293 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0037040 con 0x7f17c00ced80 2024-04-25T10:00:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.150+0000 7f17c6458640 1 -- 172.21.15.19:0/250594293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c00ccc50 con 0x7f17c00ced80 2024-04-25T10:00:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.150+0000 7f17c6458640 1 -- 172.21.15.19:0/250594293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c0160980 con 0x7f17c00ced80 2024-04-25T10:00:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.151+0000 7f17c6458640 1 -- 172.21.15.19:0/250594293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c0073a00 con 0x7f17c00ced80 2024-04-25T10:00:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.151+0000 7f17bcff9640 1 -- 172.21.15.19:0/250594293 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17b0044070 con 0x7f17c00ced80 2024-04-25T10:00:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.152+0000 7f17bcff9640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17a006b4d0 0x7f17a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.152+0000 7f17bcff9640 1 -- 172.21.15.19:0/250594293 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f17b00aac30 con 0x7f17c00ced80 2024-04-25T10:00:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.152+0000 7f17bf7fe640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17a006b4d0 0x7f17a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.153+0000 7f17bf7fe640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17a006b4d0 0x7f17a006d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f17c00b6080 tx=0x7f17a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.156+0000 7f17bcff9640 1 -- 172.21.15.19:0/250594293 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17b0079dd0 con 0x7f17c00ced80 2024-04-25T10:00:27.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.335+0000 7f17c6458640 1 -- 172.21.15.19:0/250594293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f17c00ce360 con 0x7f17c00ced80 2024-04-25T10:00:27.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.338+0000 7f17bcff9640 1 -- 172.21.15.19:0/250594293 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f17b00796d0 con 0x7f17c00ced80 2024-04-25T10:00:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 -- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17a006b4d0 msgr2=0x7f17a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17a006b4d0 0x7f17a006d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f17c00b6080 tx=0x7f17a8009040 comp rx=0 tx=0).stop 2024-04-25T10:00:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 -- 172.21.15.19:0/250594293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c00ced80 msgr2=0x7f17c00cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c00ced80 0x7f17c00cd730 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f17b0035040 tx=0x7f17b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 -- 172.21.15.19:0/250594293 shutdown_connections 2024-04-25T10:00:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17a006b4d0 0x7f17a006d990 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c015dc20 0x7f17c014c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c00d1ce0 0x7f17c0160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 --2- 172.21.15.19:0/250594293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c00ced80 0x7f17c00cd730 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 -- 172.21.15.19:0/250594293 >> 172.21.15.19:0/250594293 conn(0x7f17c0061a20 msgr2=0x7f17c00c7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 -- 172.21.15.19:0/250594293 shutdown_connections 2024-04-25T10:00:27.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.340+0000 7f17c6458640 1 -- 172.21.15.19:0/250594293 wait complete. 2024-04-25T10:00:27.355 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:00:27.356 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:27.397 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:27.440 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:00:27.440 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:00:27.440 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:27.455 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:00:27.456 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:00:27.456 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:27.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:00:27.498 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:00:27.498 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:27.513 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:00:27.514 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:00:27.514 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:27.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:00:27.555 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:00:27.555 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:27.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.652+0000 7f7e71d1d640 1 Processor -- start 2024-04-25T10:00:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.652+0000 7f7e71d1d640 1 -- start start 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.653+0000 7f7e71d1d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7e6c15dbe0 0x7f7e6c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.653+0000 7f7e71d1d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7e6c0b2cb0 0x7f7e6c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.653+0000 7f7e71d1d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b92a0 0x7f7e6c0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.653+0000 7f7e71d1d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6c15dfc0 con 0x7f7e6c0b2cb0 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.653+0000 7f7e71d1d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6c0d1a00 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.653+0000 7f7e71d1d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c0b97a0 con 0x7f7e6c0b92a0 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.653+0000 7f7e6affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7e6c0b2cb0 0x7f7e6c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.653+0000 7f7e6affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7e6c0b2cb0 0x7f7e6c0d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33360/0 (socket says 172.21.15.19:33360) 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.653+0000 7f7e6affd640 1 -- 172.21.15.19:0/648904195 learned_addr learned my addr 172.21.15.19:0/648904195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.654+0000 7f7e6bfff640 1 --2- 172.21.15.19:0/648904195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b92a0 0x7f7e6c0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.654+0000 7f7e6b7fe640 1 --2- 172.21.15.19:0/648904195 >> v2:172.21.15.112:3300/0 conn(0x7f7e6c15dbe0 0x7f7e6c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.654+0000 7f7e6affd640 1 -- 172.21.15.19:0/648904195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b92a0 msgr2=0x7f7e6c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.654+0000 7f7e6affd640 1 --2- 172.21.15.19:0/648904195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b92a0 0x7f7e6c0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.654+0000 7f7e6affd640 1 -- 172.21.15.19:0/648904195 >> v2:172.21.15.112:3300/0 conn(0x7f7e6c15dbe0 msgr2=0x7f7e6c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.654+0000 7f7e6affd640 1 --2- 172.21.15.19:0/648904195 >> v2:172.21.15.112:3300/0 conn(0x7f7e6c15dbe0 0x7f7e6c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.654+0000 7f7e6affd640 1 -- 172.21.15.19:0/648904195 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c0d1c50 con 0x7f7e6c0b2cb0 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.654+0000 7f7e6bfff640 1 --2- 172.21.15.19:0/648904195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b92a0 0x7f7e6c0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.654+0000 7f7e6b7fe640 1 --2- 172.21.15.19:0/648904195 >> v2:172.21.15.112:3300/0 conn(0x7f7e6c15dbe0 0x7f7e6c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.654+0000 7f7e6affd640 1 --2- 172.21.15.19:0/648904195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6c0b2cb0 0x7f7e6c0d0e50 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c06c140 tx=0x7f7e5402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b778073d748446c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e6a7fc640 1 -- 172.21.15.19:0/648904195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54037070 con 0x7f7e6c0b2cb0 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e6a7fc640 1 -- 172.21.15.19:0/648904195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e54032440 con 0x7f7e6c0b2cb0 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e6a7fc640 1 -- 172.21.15.19:0/648904195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54032810 con 0x7f7e6c0b2cb0 2024-04-25T10:00:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e71d1d640 1 -- 172.21.15.19:0/648904195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6c0b2cb0 msgr2=0x7f7e6c0d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e71d1d640 1 --2- 172.21.15.19:0/648904195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6c0b2cb0 0x7f7e6c0d0e50 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c06c140 tx=0x7f7e5402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e71d1d640 1 -- 172.21.15.19:0/648904195 shutdown_connections 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e71d1d640 1 --2- 172.21.15.19:0/648904195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b92a0 0x7f7e6c0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e71d1d640 1 --2- 172.21.15.19:0/648904195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6c0b2cb0 0x7f7e6c0d0e50 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e71d1d640 1 --2- 172.21.15.19:0/648904195 >> v2:172.21.15.112:3300/0 conn(0x7f7e6c15dbe0 0x7f7e6c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e71d1d640 1 -- 172.21.15.19:0/648904195 >> 172.21.15.19:0/648904195 conn(0x7f7e6c061a20 msgr2=0x7f7e6c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e71d1d640 1 -- 172.21.15.19:0/648904195 shutdown_connections 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.655+0000 7f7e71d1d640 1 -- 172.21.15.19:0/648904195 wait complete. 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.656+0000 7f7e71d1d640 1 Processor -- start 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.657+0000 7f7e71d1d640 1 -- start start 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.657+0000 7f7e71d1d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b2cb0 0x7f7e6c157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.657+0000 7f7e71d1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e6c0b92a0 0x7f7e6c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.657+0000 7f7e71d1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6c15dbe0 0x7f7e6c15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.657+0000 7f7e71d1d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c0b6ab0 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.657+0000 7f7e71d1d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c0d2210 con 0x7f7e6c0b92a0 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.657+0000 7f7e71d1d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c0b6c30 con 0x7f7e6c0b2cb0 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.657+0000 7f7e6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6c15dbe0 0x7f7e6c15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.657+0000 7f7e6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6c15dbe0 0x7f7e6c15aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33372/0 (socket says 172.21.15.19:33372) 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.657+0000 7f7e6bfff640 1 -- 172.21.15.19:0/1676391122 learned_addr learned my addr 172.21.15.19:0/1676391122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e6b7fe640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b2cb0 0x7f7e6c157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e6affd640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e6c0b92a0 0x7f7e6c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e6bfff640 1 -- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b2cb0 msgr2=0x7f7e6c157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e6bfff640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b2cb0 0x7f7e6c157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e6bfff640 1 -- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e6c0b92a0 msgr2=0x7f7e6c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e6bfff640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e6c0b92a0 0x7f7e6c15a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e6bfff640 1 -- 172.21.15.19:0/1676391122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c0b32e0 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e6b7fe640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b2cb0 0x7f7e6c157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e6affd640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e6c0b92a0 0x7f7e6c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e6bfff640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6c15dbe0 0x7f7e6c15aa20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400ce80 tx=0x7f7e6400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e63fff640 1 -- 172.21.15.19:0/1676391122 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6401a070 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e63fff640 1 -- 172.21.15.19:0/1676391122 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6400d880 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e71d1d640 1 -- 172.21.15.19:0/1676391122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6c0b35c0 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e63fff640 1 -- 172.21.15.19:0/1676391122 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400dc70 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.658+0000 7f7e71d1d640 1 -- 172.21.15.19:0/1676391122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c0b3b80 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.659+0000 7f7e71d1d640 1 -- 172.21.15.19:0/1676391122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6c073990 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.660+0000 7f7e63fff640 1 -- 172.21.15.19:0/1676391122 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e6401d070 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.660+0000 7f7e63fff640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7e4c06b4d0 0x7f7e4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.660+0000 7f7e63fff640 1 -- 172.21.15.19:0/1676391122 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7e6408a710 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.660+0000 7f7e6b7fe640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7e4c06b4d0 0x7f7e4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.661+0000 7f7e6b7fe640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7e4c06b4d0 0x7f7e4c06d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0b6470 tx=0x7f7e5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.664+0000 7f7e63fff640 1 -- 172.21.15.19:0/1676391122 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e64059140 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.881+0000 7f7e71d1d640 1 -- 172.21.15.19:0/1676391122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e6c15cb90 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.881+0000 7f7e63fff640 1 -- 172.21.15.19:0/1676391122 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f7e6405d1e0 con 0x7f7e6c15dbe0 2024-04-25T10:00:27.886 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:27.886 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2692919547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2692919547},{"type":"v1","addr":"172.21.15.19:6834","nonce":2692919547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/813171200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":813171200},{"type":"v1","addr":"172.21.15.112:6836","nonce":813171200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3009483958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3009483958},{"type":"v1","addr":"172.21.15.112:6837","nonce":3009483958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2556042207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2556042207},{"type":"v1","addr":"172.21.15.19:6835","nonce":2556042207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[]} 2024-04-25T10:00:27.886 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 67 2024-04-25T10:00:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.883+0000 7f7e71d1d640 1 -- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7e4c06b4d0 msgr2=0x7f7e4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.883+0000 7f7e71d1d640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7e4c06b4d0 0x7f7e4c06d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0b6470 tx=0x7f7e5c008040 comp rx=0 tx=0).stop 2024-04-25T10:00:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.883+0000 7f7e71d1d640 1 -- 172.21.15.19:0/1676391122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6c15dbe0 msgr2=0x7f7e6c15aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.883+0000 7f7e71d1d640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6c15dbe0 0x7f7e6c15aa20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400ce80 tx=0x7f7e6400a710 comp rx=0 tx=0).stop 2024-04-25T10:00:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.883+0000 7f7e71d1d640 1 -- 172.21.15.19:0/1676391122 shutdown_connections 2024-04-25T10:00:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.883+0000 7f7e71d1d640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7e4c06b4d0 0x7f7e4c06d990 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.883+0000 7f7e71d1d640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6c15dbe0 0x7f7e6c15aa20 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.883+0000 7f7e71d1d640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e6c0b92a0 0x7f7e6c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.883+0000 7f7e71d1d640 1 --2- 172.21.15.19:0/1676391122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e6c0b2cb0 0x7f7e6c157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.883+0000 7f7e71d1d640 1 -- 172.21.15.19:0/1676391122 >> 172.21.15.19:0/1676391122 conn(0x7f7e6c061a20 msgr2=0x7f7e6c0b74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.883+0000 7f7e71d1d640 1 -- 172.21.15.19:0/1676391122 shutdown_connections 2024-04-25T10:00:27.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:27.884+0000 7f7e71d1d640 1 -- 172.21.15.19:0/1676391122 wait complete. 2024-04-25T10:00:27.899 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:00:27.899 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:00:27.899 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:00:27.899 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:00:27.899 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:00:27.899 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:00:27.900 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:00:27.900 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:00:27.900 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:00:27.900 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:00:27.900 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:00:27.900 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:00:27.901 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:00:27.901 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:00:27.901 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:00:27.901 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:00:27.908 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:00:27.908 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:00:27.909 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:00:27.909 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:00:27.912 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:00:27.912 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:00:27.912 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:00:27.913 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:00:27.951 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:00:27.951 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:27.952 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:00:27.952 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:27.955 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:00:27.955 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:00:27.956 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:27.956 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:27.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:00:28.027 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T10:00:28.028 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:00:28.036 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T10:00:28.037 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T10:00:28.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.058+0000 7fc913dbd640 1 Processor -- start 2024-04-25T10:00:28.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.058+0000 7fc913dbd640 1 -- start start 2024-04-25T10:00:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.059+0000 7fc913dbd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc90c0cf600 0x7fc90c0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.059+0000 7fc913dbd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c0c75e0 con 0x7fc90c0cbf20 2024-04-25T10:00:28.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.059+0000 7fc913dbd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c0c78c0 con 0x7fc90c15bb50 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.059+0000 7fc913dbd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0c8680 con 0x7fc90c0cf600 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.059+0000 7fc91235a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc90c0cbf20 0x7fc90c0cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46982/0 (socket says 172.21.15.19:46982) 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.059+0000 7fc91235a640 1 -- 172.21.15.19:0/3960604781 learned_addr learned my addr 172.21.15.19:0/3960604781 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.060+0000 7fc911358640 1 --2- 172.21.15.19:0/3960604781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc90c0cf600 0x7fc90c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.060+0000 7fc910b57640 1 -- 172.21.15.19:0/3960604781 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501624828 0 0) 0x7fc90c0c75e0 con 0x7fc90c0cbf20 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.060+0000 7fc910b57640 1 -- 172.21.15.19:0/3960604781 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8f8003520 con 0x7fc90c0cbf20 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.060+0000 7fc910b57640 1 -- 172.21.15.19:0/3960604781 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564891552 0 0) 0x7fc90c0c78c0 con 0x7fc90c15bb50 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.060+0000 7fc910b57640 1 -- 172.21.15.19:0/3960604781 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8f8003880 con 0x7fc90c15bb50 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.060+0000 7fc910b57640 1 -- 172.21.15.19:0/3960604781 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3482710370 0 0) 0x7fc8f8003520 con 0x7fc90c0cbf20 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.060+0000 7fc911358640 1 -- 172.21.15.19:0/3960604781 >> v1:172.21.15.112:6789/0 conn(0x7fc90c15bb50 legacy=0x7fc90c0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.060+0000 7fc911358640 1 -- 172.21.15.19:0/3960604781 >> v1:172.21.15.19:6789/0 conn(0x7fc90c0cbf20 legacy=0x7fc90c0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.060+0000 7fc911358640 1 -- 172.21.15.19:0/3960604781 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c0c7ba0 con 0x7fc90c0cf600 2024-04-25T10:00:28.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.061+0000 7fc911358640 1 --2- 172.21.15.19:0/3960604781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc90c0cf600 0x7fc90c0cfa40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0097d0 tx=0x7fc8fc02a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cebcf02c71a2840 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.061+0000 7fc910b57640 1 -- 172.21.15.19:0/3960604781 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc037070 con 0x7fc90c0cf600 2024-04-25T10:00:28.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.061+0000 7fc910b57640 1 -- 172.21.15.19:0/3960604781 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc02acc0 con 0x7fc90c0cf600 2024-04-25T10:00:28.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.061+0000 7fc910b57640 1 -- 172.21.15.19:0/3960604781 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9040019d0 con 0x7fc90c0cf600 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.061+0000 7fc913dbd640 1 -- 172.21.15.19:0/3960604781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc90c0cf600 msgr2=0x7fc90c0cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.061+0000 7fc913dbd640 1 --2- 172.21.15.19:0/3960604781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc90c0cf600 0x7fc90c0cfa40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0097d0 tx=0x7fc8fc02a230 comp rx=0 tx=0).stop 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.062+0000 7fc913dbd640 1 -- 172.21.15.19:0/3960604781 shutdown_connections 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.062+0000 7fc913dbd640 1 --2- 172.21.15.19:0/3960604781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc90c0cf600 0x7fc90c0cfa40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.062+0000 7fc913dbd640 1 -- 172.21.15.19:0/3960604781 >> 172.21.15.19:0/3960604781 conn(0x7fc90c06f500 msgr2=0x7fc90c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.062+0000 7fc913dbd640 1 -- 172.21.15.19:0/3960604781 shutdown_connections 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.062+0000 7fc913dbd640 1 -- 172.21.15.19:0/3960604781 wait complete. 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.063+0000 7fc913dbd640 1 Processor -- start 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.063+0000 7fc913dbd640 1 -- start start 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc913dbd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90c0cbf20 0x7fc90c06cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc913dbd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc90c0cf600 0x7fc90c148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc913dbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c15bb50 0x7fc90c149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc913dbd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0c8160 con 0x7fc90c0cbf20 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc913dbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0cff80 con 0x7fc90c15bb50 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc913dbd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0c72a0 con 0x7fc90c0cf600 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc911b59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90c0cbf20 0x7fc90c06cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc911b59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90c0cbf20 0x7fc90c06cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44344/0 (socket says 172.21.15.19:44344) 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc911b59640 1 -- 172.21.15.19:0/1218546972 learned_addr learned my addr 172.21.15.19:0/1218546972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc911358640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc90c0cf600 0x7fc90c148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc91235a640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c15bb50 0x7fc90c149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc911b59640 1 -- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc90c0cf600 msgr2=0x7fc90c148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc911b59640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc90c0cf600 0x7fc90c148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc911b59640 1 -- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c15bb50 msgr2=0x7fc90c149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc911b59640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c15bb50 0x7fc90c149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc911b59640 1 -- 172.21.15.19:0/1218546972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c06be20 con 0x7fc90c0cbf20 2024-04-25T10:00:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc91235a640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c15bb50 0x7fc90c149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc911b59640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90c0cbf20 0x7fc90c06cec0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c430 tx=0x7fc90800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc9027fc640 1 -- 172.21.15.19:0/1218546972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908017070 con 0x7fc90c0cbf20 2024-04-25T10:00:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc9027fc640 1 -- 172.21.15.19:0/1218546972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9080052f0 con 0x7fc90c0cbf20 2024-04-25T10:00:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc913dbd640 1 -- 172.21.15.19:0/1218546972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c06c100 con 0x7fc90c0cbf20 2024-04-25T10:00:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc9027fc640 1 -- 172.21.15.19:0/1218546972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908003830 con 0x7fc90c0cbf20 2024-04-25T10:00:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.064+0000 7fc913dbd640 1 -- 172.21.15.19:0/1218546972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c06c6c0 con 0x7fc90c0cbf20 2024-04-25T10:00:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.065+0000 7fc913dbd640 1 -- 172.21.15.19:0/1218546972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c073a00 con 0x7fc90c0cbf20 2024-04-25T10:00:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.066+0000 7fc9027fc640 1 -- 172.21.15.19:0/1218546972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc90801a070 con 0x7fc90c0cbf20 2024-04-25T10:00:28.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.066+0000 7fc9027fc640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8f006b500 0x7fc8f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.066+0000 7fc9027fc640 1 -- 172.21.15.19:0/1218546972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc908089370 con 0x7fc90c0cbf20 2024-04-25T10:00:28.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.066+0000 7fc911358640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8f006b500 0x7fc8f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.067+0000 7fc911358640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8f006b500 0x7fc8f006d9c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc034000 tx=0x7fc8fc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.070+0000 7fc9027fc640 1 -- 172.21.15.19:0/1218546972 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc908057da0 con 0x7fc90c0cbf20 2024-04-25T10:00:28.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:28.262+0000 7fc913dbd640 1 -- 172.21.15.19:0/1218546972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc90c0ce490 con 0x7fc90c0cbf20 2024-04-25T10:00:29.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.038+0000 7fc9027fc640 1 -- 172.21.15.19:0/1218546972 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc90805be40 con 0x7fc90c0cbf20 2024-04-25T10:00:29.043 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T10:00:29.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 -- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8f006b500 msgr2=0x7fc8f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8f006b500 0x7fc8f006d9c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc034000 tx=0x7fc8fc02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:29.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 -- 172.21.15.19:0/1218546972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90c0cbf20 msgr2=0x7fc90c06cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90c0cbf20 0x7fc90c06cec0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c430 tx=0x7fc90800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 -- 172.21.15.19:0/1218546972 shutdown_connections 2024-04-25T10:00:29.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8f006b500 0x7fc8f006d9c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c15bb50 0x7fc90c149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc90c0cf600 0x7fc90c148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 --2- 172.21.15.19:0/1218546972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc90c0cbf20 0x7fc90c06cec0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 -- 172.21.15.19:0/1218546972 >> 172.21.15.19:0/1218546972 conn(0x7fc90c06f500 msgr2=0x7fc90c070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 -- 172.21.15.19:0/1218546972 shutdown_connections 2024-04-25T10:00:29.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.042+0000 7fc913dbd640 1 -- 172.21.15.19:0/1218546972 wait complete. 2024-04-25T10:00:29.062 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:00:29.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.165+0000 7f5e9317d640 1 Processor -- start 2024-04-25T10:00:29.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.165+0000 7f5e9317d640 1 -- start start 2024-04-25T10:00:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.165+0000 7f5e9317d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5e8c15dc70 0x7f5e8c0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.165+0000 7f5e9317d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c06ce10 0x7f5e8c0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.165+0000 7f5e9317d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8c06c220 con 0x7f5e8c09a450 2024-04-25T10:00:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.165+0000 7f5e9317d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8c0d0c90 con 0x7f5e8c15dc70 2024-04-25T10:00:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.165+0000 7f5e9317d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0d0080 con 0x7f5e8c06ce10 2024-04-25T10:00:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.166+0000 7f5e90f19640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5e8c15dc70 0x7f5e8c0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.166+0000 7f5e8bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5e8c09a450 0x7f5e8c06c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46994/0 (socket says 172.21.15.19:46994) 2024-04-25T10:00:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.166+0000 7f5e8bfff640 1 -- 172.21.15.19:0/3835014361 learned_addr learned my addr 172.21.15.19:0/3835014361 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.166+0000 7f5e9171a640 1 --2- 172.21.15.19:0/3835014361 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c06ce10 0x7f5e8c0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.166+0000 7f5e90f19640 1 -- 172.21.15.19:0/3835014361 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c06ce10 msgr2=0x7f5e8c0cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:29.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.166+0000 7f5e90f19640 1 --2- 172.21.15.19:0/3835014361 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c06ce10 0x7f5e8c0cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.166+0000 7f5e90f19640 1 -- 172.21.15.19:0/3835014361 >> v1:172.21.15.19:6789/0 conn(0x7f5e8c09a450 legacy=0x7f5e8c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.166+0000 7f5e90f19640 1 -- 172.21.15.19:0/3835014361 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c0d0360 con 0x7f5e8c15dc70 2024-04-25T10:00:29.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.166+0000 7f5e8b7fe640 1 -- 172.21.15.19:0/3835014361 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194739901 0 0) 0x7f5e8c06c220 con 0x7f5e8c09a450 2024-04-25T10:00:29.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e90f19640 1 --2- 172.21.15.19:0/3835014361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c15dc70 0x7f5e8c0d1120 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009d90 tx=0x7f5e8402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375043ab0fd4cc77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e8b7fe640 1 -- 172.21.15.19:0/3835014361 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84007740 con 0x7f5e8c15dc70 2024-04-25T10:00:29.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e8b7fe640 1 -- 172.21.15.19:0/3835014361 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e840078e0 con 0x7f5e8c15dc70 2024-04-25T10:00:29.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e8b7fe640 1 -- 172.21.15.19:0/3835014361 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84007cb0 con 0x7f5e8c15dc70 2024-04-25T10:00:29.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e9317d640 1 -- 172.21.15.19:0/3835014361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c15dc70 msgr2=0x7f5e8c0d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e9317d640 1 --2- 172.21.15.19:0/3835014361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c15dc70 0x7f5e8c0d1120 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009d90 tx=0x7f5e8402a660 comp rx=0 tx=0).stop 2024-04-25T10:00:29.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e9317d640 1 -- 172.21.15.19:0/3835014361 shutdown_connections 2024-04-25T10:00:29.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e9317d640 1 --2- 172.21.15.19:0/3835014361 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c06ce10 0x7f5e8c0cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e9317d640 1 --2- 172.21.15.19:0/3835014361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c15dc70 0x7f5e8c0d1120 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e9317d640 1 -- 172.21.15.19:0/3835014361 >> 172.21.15.19:0/3835014361 conn(0x7f5e8c0b3270 msgr2=0x7f5e8c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e9317d640 1 -- 172.21.15.19:0/3835014361 shutdown_connections 2024-04-25T10:00:29.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.167+0000 7f5e9317d640 1 -- 172.21.15.19:0/3835014361 wait complete. 2024-04-25T10:00:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.169+0000 7f5e9317d640 1 Processor -- start 2024-04-25T10:00:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.169+0000 7f5e9317d640 1 -- start start 2024-04-25T10:00:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.169+0000 7f5e9317d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c06ce10 0x7f5e8c09b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.169+0000 7f5e9317d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c09a450 0x7f5e8c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.169+0000 7f5e9317d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c15dc70 0x7f5e8c0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.169+0000 7f5e9317d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c15e050 con 0x7f5e8c15dc70 2024-04-25T10:00:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.169+0000 7f5e9317d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0d0920 con 0x7f5e8c09a450 2024-04-25T10:00:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.169+0000 7f5e9317d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0cbdd0 con 0x7f5e8c06ce10 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e9171a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c15dc70 0x7f5e8c0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e9171a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c15dc70 0x7f5e8c0c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44368/0 (socket says 172.21.15.19:44368) 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e9171a640 1 -- 172.21.15.19:0/1763388799 learned_addr learned my addr 172.21.15.19:0/1763388799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e8bfff640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c09a450 0x7f5e8c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e90f19640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c06ce10 0x7f5e8c09b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e9171a640 1 -- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c06ce10 msgr2=0x7f5e8c09b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e9171a640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c06ce10 0x7f5e8c09b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e9171a640 1 -- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c09a450 msgr2=0x7f5e8c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e9171a640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c09a450 0x7f5e8c0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e9171a640 1 -- 172.21.15.19:0/1763388799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c1517a0 con 0x7f5e8c15dc70 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e9171a640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c15dc70 0x7f5e8c0c9ba0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00d830 tx=0x7f5e7c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e897fa640 1 -- 172.21.15.19:0/1763388799 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c00b6c0 con 0x7f5e8c15dc70 2024-04-25T10:00:29.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e897fa640 1 -- 172.21.15.19:0/1763388799 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7c019070 con 0x7f5e8c15dc70 2024-04-25T10:00:29.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e897fa640 1 -- 172.21.15.19:0/1763388799 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c00bac0 con 0x7f5e8c15dc70 2024-04-25T10:00:29.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e9317d640 1 -- 172.21.15.19:0/1763388799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c151a80 con 0x7f5e8c15dc70 2024-04-25T10:00:29.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.170+0000 7f5e9317d640 1 -- 172.21.15.19:0/1763388799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c152320 con 0x7f5e8c15dc70 2024-04-25T10:00:29.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.172+0000 7f5e9317d640 1 -- 172.21.15.19:0/1763388799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8c152520 con 0x7f5e8c15dc70 2024-04-25T10:00:29.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.172+0000 7f5e897fa640 1 -- 172.21.15.19:0/1763388799 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e7c01e460 con 0x7f5e8c15dc70 2024-04-25T10:00:29.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.172+0000 7f5e897fa640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e7806b4d0 0x7f5e7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.172+0000 7f5e897fa640 1 -- 172.21.15.19:0/1763388799 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5e7c08a570 con 0x7f5e8c15dc70 2024-04-25T10:00:29.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.173+0000 7f5e90f19640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e7806b4d0 0x7f5e7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.173+0000 7f5e90f19640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e7806b4d0 0x7f5e7806d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5e84034000 tx=0x7f5e8402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.177+0000 7f5e897fa640 1 -- 172.21.15.19:0/1763388799 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e8c152520 con 0x7f5e8c15dc70 2024-04-25T10:00:29.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.424+0000 7f5e9317d640 1 -- 172.21.15.19:0/1763388799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5e8c073990 con 0x7f5e8c15dc70 2024-04-25T10:00:29.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.424+0000 7f5e897fa640 1 -- 172.21.15.19:0/1763388799 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5e7c058f60 con 0x7f5e8c15dc70 2024-04-25T10:00:29.429 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:29.429 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:29.429 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T10:00:29.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.429 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T10:00:29.429 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T10:00:29.429 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.429 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:00:29.429 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.430 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T10:00:29.432 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.434 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:00:29.435 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T10:00:29.437 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:00:29.438 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T10:00:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:00:29.440 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:00:29.441 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:00:29.441 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.441 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T10:00:29.441 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T10:00:29.441 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.441 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:00:29.441 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:00:29.441 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:00:29.442 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T10:00:29.444 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:29.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 -- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e7806b4d0 msgr2=0x7f5e7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e7806b4d0 0x7f5e7806d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5e84034000 tx=0x7f5e8402f000 comp rx=0 tx=0).stop 2024-04-25T10:00:29.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 -- 172.21.15.19:0/1763388799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c15dc70 msgr2=0x7f5e8c0c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c15dc70 0x7f5e8c0c9ba0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00d830 tx=0x7f5e7c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 -- 172.21.15.19:0/1763388799 shutdown_connections 2024-04-25T10:00:29.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e7806b4d0 0x7f5e7806d990 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8c15dc70 0x7f5e8c0c9ba0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e8c09a450 0x7f5e8c0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 --2- 172.21.15.19:0/1763388799 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e8c06ce10 0x7f5e8c09b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 -- 172.21.15.19:0/1763388799 >> 172.21.15.19:0/1763388799 conn(0x7f5e8c0b3270 msgr2=0x7f5e8c0ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 -- 172.21.15.19:0/1763388799 shutdown_connections 2024-04-25T10:00:29.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.426+0000 7f5e9317d640 1 -- 172.21.15.19:0/1763388799 wait complete. 2024-04-25T10:00:29.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.544+0000 7fa0626f7640 1 Processor -- start 2024-04-25T10:00:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.544+0000 7fa0626f7640 1 -- start start 2024-04-25T10:00:29.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.545+0000 7fa0626f7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0b2cb0 0x7fa05c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.545+0000 7fa0626f7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa05c0d1020 0x7fa05c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.545+0000 7fa0626f7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa05c0b97a0 con 0x7fa05c15dbe0 2024-04-25T10:00:29.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.545+0000 7fa0626f7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05c15dfc0 con 0x7fa05c0d1020 2024-04-25T10:00:29.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.545+0000 7fa0626f7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c0d1a00 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.545+0000 7fa05bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa05c15dbe0 0x7fa05c0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47004/0 (socket says 172.21.15.19:47004) 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.545+0000 7fa05bfff640 1 -- 172.21.15.19:0/2725396993 learned_addr learned my addr 172.21.15.19:0/2725396993 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.545+0000 7fa060c94640 1 --2- 172.21.15.19:0/2725396993 >> v2:172.21.15.19:3300/0 conn(0x7fa05c0d1020 0x7fa05c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.545+0000 7fa05b7fe640 1 --2- 172.21.15.19:0/2725396993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0b2cb0 0x7fa05c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.546+0000 7fa060c94640 1 -- 172.21.15.19:0/2725396993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0b2cb0 msgr2=0x7fa05c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.546+0000 7fa060c94640 1 --2- 172.21.15.19:0/2725396993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0b2cb0 0x7fa05c0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.546+0000 7fa060c94640 1 -- 172.21.15.19:0/2725396993 >> v1:172.21.15.19:6789/0 conn(0x7fa05c15dbe0 legacy=0x7fa05c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.546+0000 7fa060c94640 1 -- 172.21.15.19:0/2725396993 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c0d1c50 con 0x7fa05c0d1020 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.546+0000 7fa05affd640 1 -- 172.21.15.19:0/2725396993 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837643362 0 0) 0x7fa05c0b97a0 con 0x7fa05c15dbe0 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.546+0000 7fa05b7fe640 1 --2- 172.21.15.19:0/2725396993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0b2cb0 0x7fa05c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.546+0000 7fa060c94640 1 --2- 172.21.15.19:0/2725396993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c0d1020 0x7fa05c0d2c70 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa054009d90 tx=0x7fa05402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59d3534d1adf5be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.547+0000 7fa05affd640 1 -- 172.21.15.19:0/2725396993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054007740 con 0x7fa05c0d1020 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.547+0000 7fa05affd640 1 -- 172.21.15.19:0/2725396993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0540078e0 con 0x7fa05c0d1020 2024-04-25T10:00:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.547+0000 7fa05affd640 1 -- 172.21.15.19:0/2725396993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054007cb0 con 0x7fa05c0d1020 2024-04-25T10:00:29.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.547+0000 7fa0626f7640 1 -- 172.21.15.19:0/2725396993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c0d1020 msgr2=0x7fa05c0d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.547+0000 7fa0626f7640 1 --2- 172.21.15.19:0/2725396993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c0d1020 0x7fa05c0d2c70 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa054009d90 tx=0x7fa05402a660 comp rx=0 tx=0).stop 2024-04-25T10:00:29.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.547+0000 7fa0626f7640 1 -- 172.21.15.19:0/2725396993 shutdown_connections 2024-04-25T10:00:29.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.547+0000 7fa0626f7640 1 --2- 172.21.15.19:0/2725396993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c0d1020 0x7fa05c0d2c70 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.547+0000 7fa0626f7640 1 --2- 172.21.15.19:0/2725396993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0b2cb0 0x7fa05c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.547+0000 7fa0626f7640 1 -- 172.21.15.19:0/2725396993 >> 172.21.15.19:0/2725396993 conn(0x7fa05c061a20 msgr2=0x7fa05c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.547+0000 7fa0626f7640 1 -- 172.21.15.19:0/2725396993 shutdown_connections 2024-04-25T10:00:29.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.547+0000 7fa0626f7640 1 -- 172.21.15.19:0/2725396993 wait complete. 2024-04-25T10:00:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.548+0000 7fa0626f7640 1 Processor -- start 2024-04-25T10:00:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.548+0000 7fa0626f7640 1 -- start start 2024-04-25T10:00:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.549+0000 7fa0626f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c0b2cb0 0x7fa05c1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.549+0000 7fa0626f7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0d1020 0x7fa05c0cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.549+0000 7fa0626f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa05c15dbe0 0x7fa05c0b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.549+0000 7fa0626f7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c0b92a0 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.549+0000 7fa0626f7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c0d2210 con 0x7fa05c15dbe0 2024-04-25T10:00:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.549+0000 7fa0626f7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c0b9420 con 0x7fa05c0d1020 2024-04-25T10:00:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.549+0000 7fa05bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c0b2cb0 0x7fa05c1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.549+0000 7fa05bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c0b2cb0 0x7fa05c1542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44384/0 (socket says 172.21.15.19:44384) 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.549+0000 7fa05bfff640 1 -- 172.21.15.19:0/2325021349 learned_addr learned my addr 172.21.15.19:0/2325021349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa060c94640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa05c15dbe0 0x7fa05c0b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa05b7fe640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0d1020 0x7fa05c0cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa05bfff640 1 -- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0d1020 msgr2=0x7fa05c0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa05bfff640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0d1020 0x7fa05c0cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa05bfff640 1 -- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa05c15dbe0 msgr2=0x7fa05c0b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa05bfff640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa05c15dbe0 0x7fa05c0b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa05bfff640 1 -- 172.21.15.19:0/2325021349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c154820 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa05b7fe640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0d1020 0x7fa05c0cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa05bfff640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c0b2cb0 0x7fa05c1542e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00c430 tx=0x7fa04c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa058ff9640 1 -- 172.21.15.19:0/2325021349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c0074e0 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa058ff9640 1 -- 172.21.15.19:0/2325021349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c0047e0 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa0626f7640 1 -- 172.21.15.19:0/2325021349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05c1e62f0 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa058ff9640 1 -- 172.21.15.19:0/2325021349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c004bd0 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.550+0000 7fa0626f7640 1 -- 172.21.15.19:0/2325021349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05c1e68b0 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.551+0000 7fa0626f7640 1 -- 172.21.15.19:0/2325021349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05c073990 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.552+0000 7fa058ff9640 1 -- 172.21.15.19:0/2325021349 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa04c011070 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.552+0000 7fa058ff9640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa03c06b4d0 0x7fa03c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.552+0000 7fa058ff9640 1 -- 172.21.15.19:0/2325021349 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa04c089bd0 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.552+0000 7fa05b7fe640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa03c06b4d0 0x7fa03c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.553+0000 7fa05b7fe640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa03c06b4d0 0x7fa03c06d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa044006fd0 tx=0x7fa044008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.556+0000 7fa058ff9640 1 -- 172.21.15.19:0/2325021349 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa04c058ca0 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.736+0000 7fa0626f7640 1 -- 172.21.15.19:0/2325021349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa05c062180 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.737+0000 7fa058ff9640 1 -- 172.21.15.19:0/2325021349 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+13162 (secure 0 0 0) 0x7fa04c0585a0 con 0x7fa05c0b2cb0 2024-04-25T10:00:29.741 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:29.741 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":113,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:00:29.047299+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:29.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 -- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa03c06b4d0 msgr2=0x7fa03c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa03c06b4d0 0x7fa03c06d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa044006fd0 tx=0x7fa044008040 comp rx=0 tx=0).stop 2024-04-25T10:00:29.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 -- 172.21.15.19:0/2325021349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c0b2cb0 msgr2=0x7fa05c1542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c0b2cb0 0x7fa05c1542e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00c430 tx=0x7fa04c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 -- 172.21.15.19:0/2325021349 shutdown_connections 2024-04-25T10:00:29.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa03c06b4d0 0x7fa03c06d990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa05c15dbe0 0x7fa05c0b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa05c0d1020 0x7fa05c0cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 --2- 172.21.15.19:0/2325021349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa05c0b2cb0 0x7fa05c1542e0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 -- 172.21.15.19:0/2325021349 >> 172.21.15.19:0/2325021349 conn(0x7fa05c061a20 msgr2=0x7fa05c0b6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 -- 172.21.15.19:0/2325021349 shutdown_connections 2024-04-25T10:00:29.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.739+0000 7fa0626f7640 1 -- 172.21.15.19:0/2325021349 wait complete. 2024-04-25T10:00:29.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:00:29.756 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:00:29.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.856+0000 7f9e74246640 1 Processor -- start 2024-04-25T10:00:29.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.856+0000 7f9e74246640 1 -- start start 2024-04-25T10:00:29.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.857+0000 7f9e74246640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9e6c15b980 0x7f9e6c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.857+0000 7f9e74246640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6c072280 con 0x7f9e6c0ce230 2024-04-25T10:00:29.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.857+0000 7f9e74246640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6c0714c0 con 0x7f9e6c0cc4f0 2024-04-25T10:00:29.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.857+0000 7f9e74246640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6c071270 con 0x7f9e6c15b980 2024-04-25T10:00:29.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.857+0000 7f9e727e3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9e6c15b980 0x7f9e6c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.857+0000 7f9e717e1640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9e6c0ce230 0x7f9e6c15b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47020/0 (socket says 172.21.15.19:47020) 2024-04-25T10:00:29.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.857+0000 7f9e727e3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9e6c15b980 0x7f9e6c0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44386/0 (socket says 172.21.15.19:44386) 2024-04-25T10:00:29.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.857+0000 7f9e717e1640 1 -- 172.21.15.19:0/2923810335 learned_addr learned my addr 172.21.15.19:0/2923810335 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:29.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.857+0000 7f9e727e3640 1 -- 172.21.15.19:0/2923810335 >> v1:172.21.15.112:6789/0 conn(0x7f9e6c0cc4f0 legacy=0x7f9e6c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.857+0000 7f9e727e3640 1 -- 172.21.15.19:0/2923810335 >> v1:172.21.15.19:6789/0 conn(0x7f9e6c0ce230 legacy=0x7f9e6c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.857+0000 7f9e727e3640 1 -- 172.21.15.19:0/2923810335 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c0717a0 con 0x7f9e6c15b980 2024-04-25T10:00:29.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.858+0000 7f9e727e3640 1 --2- 172.21.15.19:0/2923810335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e6c15b980 0x7f9e6c0cc8d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009d90 tx=0x7f9e6402c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df9903bfef4a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.858+0000 7f9e70fe0640 1 -- 172.21.15.19:0/2923810335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64036020 con 0x7f9e6c15b980 2024-04-25T10:00:29.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.858+0000 7f9e70fe0640 1 -- 172.21.15.19:0/2923810335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e64032800 con 0x7f9e6c15b980 2024-04-25T10:00:29.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.858+0000 7f9e70fe0640 1 -- 172.21.15.19:0/2923810335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64032bd0 con 0x7f9e6c15b980 2024-04-25T10:00:29.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.859+0000 7f9e74246640 1 -- 172.21.15.19:0/2923810335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e6c15b980 msgr2=0x7f9e6c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.859+0000 7f9e74246640 1 --2- 172.21.15.19:0/2923810335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e6c15b980 0x7f9e6c0cc8d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009d90 tx=0x7f9e6402c900 comp rx=0 tx=0).stop 2024-04-25T10:00:29.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.859+0000 7f9e74246640 1 -- 172.21.15.19:0/2923810335 shutdown_connections 2024-04-25T10:00:29.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.859+0000 7f9e74246640 1 --2- 172.21.15.19:0/2923810335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e6c15b980 0x7f9e6c0cc8d0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.859+0000 7f9e74246640 1 -- 172.21.15.19:0/2923810335 >> 172.21.15.19:0/2923810335 conn(0x7f9e6c06abe0 msgr2=0x7f9e6c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.859+0000 7f9e74246640 1 -- 172.21.15.19:0/2923810335 shutdown_connections 2024-04-25T10:00:29.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.859+0000 7f9e74246640 1 -- 172.21.15.19:0/2923810335 wait complete. 2024-04-25T10:00:29.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.860+0000 7f9e74246640 1 Processor -- start 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.860+0000 7f9e74246640 1 -- start start 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e74246640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e6c0cc4f0 0x7f9e6c13d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e74246640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e6c0ce230 0x7f9e6c0b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e74246640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e6c15b980 0x7f9e6c13f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e74246640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c06e8a0 con 0x7f9e6c15b980 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e74246640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c071d60 con 0x7f9e6c0cc4f0 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e74246640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0ae500 con 0x7f9e6c0ce230 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e727e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e6c15b980 0x7f9e6c13f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e727e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e6c15b980 0x7f9e6c13f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44396/0 (socket says 172.21.15.19:44396) 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e727e3640 1 -- 172.21.15.19:0/2784851030 learned_addr learned my addr 172.21.15.19:0/2784851030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e717e1640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e6c0ce230 0x7f9e6c0b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e71fe2640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e6c0cc4f0 0x7f9e6c13d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e727e3640 1 -- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e6c0ce230 msgr2=0x7f9e6c0b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e727e3640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e6c0ce230 0x7f9e6c0b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e727e3640 1 -- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e6c0cc4f0 msgr2=0x7f9e6c13d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e727e3640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e6c0cc4f0 0x7f9e6c13d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e727e3640 1 -- 172.21.15.19:0/2784851030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c151600 con 0x7f9e6c15b980 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e71fe2640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e6c0cc4f0 0x7f9e6c13d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.861+0000 7f9e727e3640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e6c15b980 0x7f9e6c13f210 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009d90 tx=0x7f9e64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.862+0000 7f9e627fc640 1 -- 172.21.15.19:0/2784851030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64036040 con 0x7f9e6c15b980 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.862+0000 7f9e627fc640 1 -- 172.21.15.19:0/2784851030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6403dd00 con 0x7f9e6c15b980 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.862+0000 7f9e74246640 1 -- 172.21.15.19:0/2784851030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6c155a00 con 0x7f9e6c15b980 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.862+0000 7f9e627fc640 1 -- 172.21.15.19:0/2784851030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6403b670 con 0x7f9e6c15b980 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.862+0000 7f9e74246640 1 -- 172.21.15.19:0/2784851030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c13e730 con 0x7f9e6c15b980 2024-04-25T10:00:29.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.863+0000 7f9e74246640 1 -- 172.21.15.19:0/2784851030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6c0ce960 con 0x7f9e6c15b980 2024-04-25T10:00:29.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.863+0000 7f9e627fc640 1 -- 172.21.15.19:0/2784851030 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e64039020 con 0x7f9e6c15b980 2024-04-25T10:00:29.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.864+0000 7f9e627fc640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9e5006b4d0 0x7f9e5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.864+0000 7f9e627fc640 1 -- 172.21.15.19:0/2784851030 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9e640ad700 con 0x7f9e6c15b980 2024-04-25T10:00:29.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.864+0000 7f9e71fe2640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9e5006b4d0 0x7f9e5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.864+0000 7f9e71fe2640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9e5006b4d0 0x7f9e5006d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9e68001660 tx=0x7f9e68009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:29.868+0000 7f9e627fc640 1 -- 172.21.15.19:0/2784851030 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e6c0ce960 con 0x7f9e6c15b980 2024-04-25T10:00:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.047+0000 7f9e74246640 1 -- 172.21.15.19:0/2784851030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e6c0ceee0 con 0x7f9e6c15b980 2024-04-25T10:00:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.047+0000 7f9e627fc640 1 -- 172.21.15.19:0/2784851030 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e6c151600 con 0x7f9e6c15b980 2024-04-25T10:00:30.052 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:00:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 -- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9e5006b4d0 msgr2=0x7f9e5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9e5006b4d0 0x7f9e5006d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9e68001660 tx=0x7f9e68009040 comp rx=0 tx=0).stop 2024-04-25T10:00:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 -- 172.21.15.19:0/2784851030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e6c15b980 msgr2=0x7f9e6c13f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e6c15b980 0x7f9e6c13f210 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009d90 tx=0x7f9e64004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 -- 172.21.15.19:0/2784851030 shutdown_connections 2024-04-25T10:00:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9e5006b4d0 0x7f9e5006d990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e6c15b980 0x7f9e6c13f210 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e6c0ce230 0x7f9e6c0b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 --2- 172.21.15.19:0/2784851030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e6c0cc4f0 0x7f9e6c13d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 -- 172.21.15.19:0/2784851030 >> 172.21.15.19:0/2784851030 conn(0x7f9e6c06abe0 msgr2=0x7f9e6c0cf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 -- 172.21.15.19:0/2784851030 shutdown_connections 2024-04-25T10:00:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.049+0000 7f9e74246640 1 -- 172.21.15.19:0/2784851030 wait complete. 2024-04-25T10:00:30.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:00:30.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.163+0000 7f6855a4c640 1 Processor -- start 2024-04-25T10:00:30.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.163+0000 7f6855a4c640 1 -- start start 2024-04-25T10:00:30.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.163+0000 7f6855a4c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f68500d2e90 0x7f68500d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.163+0000 7f6855a4c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500b9940 con 0x7f68500d0900 2024-04-25T10:00:30.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.163+0000 7f6855a4c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500d1df0 con 0x7f685015dd30 2024-04-25T10:00:30.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.164+0000 7f6855a4c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500d0d40 con 0x7f68500d2e90 2024-04-25T10:00:30.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.164+0000 7f6847fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f68500d2e90 0x7f68500d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.164+0000 7f6847fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f68500d2e90 0x7f68500d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44398/0 (socket says 172.21.15.19:44398) 2024-04-25T10:00:30.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.164+0000 7f6847fff640 1 -- 172.21.15.19:0/664594107 learned_addr learned my addr 172.21.15.19:0/664594107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:30.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.164+0000 7f6847fff640 1 -- 172.21.15.19:0/664594107 >> v1:172.21.15.112:6789/0 conn(0x7f685015dd30 legacy=0x7f68500b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:30.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.164+0000 7f6847fff640 1 -- 172.21.15.19:0/664594107 >> v1:172.21.15.19:6789/0 conn(0x7f68500d0900 legacy=0x7f68500b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.164+0000 7f684e7fc640 1 -- 172.21.15.19:0/664594107 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319543253 0 0) 0x7f68500b9940 con 0x7f68500d0900 2024-04-25T10:00:30.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.164+0000 7f6847fff640 1 -- 172.21.15.19:0/664594107 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500d20d0 con 0x7f68500d2e90 2024-04-25T10:00:30.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.165+0000 7f6847fff640 1 --2- 172.21.15.19:0/664594107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68500d2e90 0x7f68500d03c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f6838009850 tx=0x7f683802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7698557e3e374d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.165+0000 7f684e7fc640 1 -- 172.21.15.19:0/664594107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838004030 con 0x7f68500d2e90 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.165+0000 7f684e7fc640 1 -- 172.21.15.19:0/664594107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68380041d0 con 0x7f68500d2e90 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.165+0000 7f684e7fc640 1 -- 172.21.15.19:0/664594107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838003610 con 0x7f68500d2e90 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.165+0000 7f6855a4c640 1 -- 172.21.15.19:0/664594107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68500d2e90 msgr2=0x7f68500d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.165+0000 7f6855a4c640 1 --2- 172.21.15.19:0/664594107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68500d2e90 0x7f68500d03c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f6838009850 tx=0x7f683802a160 comp rx=0 tx=0).stop 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.165+0000 7f6855a4c640 1 -- 172.21.15.19:0/664594107 shutdown_connections 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.165+0000 7f6855a4c640 1 --2- 172.21.15.19:0/664594107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68500d2e90 0x7f68500d03c0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.166+0000 7f6855a4c640 1 -- 172.21.15.19:0/664594107 >> 172.21.15.19:0/664594107 conn(0x7f6850061c30 msgr2=0x7f685006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.166+0000 7f6855a4c640 1 -- 172.21.15.19:0/664594107 shutdown_connections 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.166+0000 7f6855a4c640 1 -- 172.21.15.19:0/664594107 wait complete. 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.167+0000 7f6855a4c640 1 Processor -- start 2024-04-25T10:00:30.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.167+0000 7f6855a4c640 1 -- start start 2024-04-25T10:00:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.167+0000 7f6855a4c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68500d0900 0x7f6850151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.167+0000 7f6855a4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68500d2e90 0x7f685015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.167+0000 7f6855a4c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685015dd30 0x7f685015f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.167+0000 7f6855a4c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500d2970 con 0x7f685015dd30 2024-04-25T10:00:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.167+0000 7f6855a4c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500d2690 con 0x7f68500d2e90 2024-04-25T10:00:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.167+0000 7f6855a4c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500d1a10 con 0x7f68500d0900 2024-04-25T10:00:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.167+0000 7f684f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685015dd30 0x7f685015f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685015dd30 0x7f685015f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44404/0 (socket says 172.21.15.19:44404) 2024-04-25T10:00:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684f7fe640 1 -- 172.21.15.19:0/689907304 learned_addr learned my addr 172.21.15.19:0/689907304 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f6847fff640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68500d2e90 0x7f685015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684effd640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68500d0900 0x7f6850151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684f7fe640 1 -- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68500d0900 msgr2=0x7f6850151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684f7fe640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68500d0900 0x7f6850151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684f7fe640 1 -- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68500d2e90 msgr2=0x7f685015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684f7fe640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68500d2e90 0x7f685015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684f7fe640 1 -- 172.21.15.19:0/689907304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500b5960 con 0x7f685015dd30 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f6847fff640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68500d2e90 0x7f685015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684f7fe640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685015dd30 0x7f685015f2c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f684800c780 tx=0x7f684800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684cff9640 1 -- 172.21.15.19:0/689907304 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848003b70 con 0x7f685015dd30 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684cff9640 1 -- 172.21.15.19:0/689907304 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6848003d10 con 0x7f685015dd30 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f6855a4c640 1 -- 172.21.15.19:0/689907304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685015f8f0 con 0x7f685015dd30 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f684cff9640 1 -- 172.21.15.19:0/689907304 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848005270 con 0x7f685015dd30 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.168+0000 7f6855a4c640 1 -- 172.21.15.19:0/689907304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68500cac20 con 0x7f685015dd30 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.169+0000 7f6855a4c640 1 -- 172.21.15.19:0/689907304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6850073990 con 0x7f685015dd30 2024-04-25T10:00:30.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.170+0000 7f684cff9640 1 -- 172.21.15.19:0/689907304 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6848005410 con 0x7f685015dd30 2024-04-25T10:00:30.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.170+0000 7f684cff9640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f683006b4d0 0x7f683006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.170+0000 7f684cff9640 1 -- 172.21.15.19:0/689907304 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6848089eb0 con 0x7f685015dd30 2024-04-25T10:00:30.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.170+0000 7f684effd640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f683006b4d0 0x7f683006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.171+0000 7f684effd640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f683006b4d0 0x7f683006d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6840004ea0 tx=0x7f6840009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:30.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.175+0000 7f684cff9640 1 -- 172.21.15.19:0/689907304 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6848058ef0 con 0x7f685015dd30 2024-04-25T10:00:30.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.359+0000 7f6855a4c640 1 -- 172.21.15.19:0/689907304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f685015d3c0 con 0x7f685015dd30 2024-04-25T10:00:30.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.359+0000 7f684cff9640 1 -- 172.21.15.19:0/689907304 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) v1 ==== 84+0+65 (secure 0 0 0) 0x7f68480587f0 con 0x7f685015dd30 2024-04-25T10:00:30.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:30.363 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T10:00:30.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:00:30.364 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T10:00:30.364 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T10:00:30.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:30.364 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T10:00:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 -- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f683006b4d0 msgr2=0x7f683006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f683006b4d0 0x7f683006d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6840004ea0 tx=0x7f6840009040 comp rx=0 tx=0).stop 2024-04-25T10:00:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 -- 172.21.15.19:0/689907304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685015dd30 msgr2=0x7f685015f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685015dd30 0x7f685015f2c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f684800c780 tx=0x7f684800cb80 comp rx=0 tx=0).stop 2024-04-25T10:00:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 -- 172.21.15.19:0/689907304 shutdown_connections 2024-04-25T10:00:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f683006b4d0 0x7f683006d990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f685015dd30 0x7f685015f2c0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68500d2e90 0x7f685015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 --2- 172.21.15.19:0/689907304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68500d0900 0x7f6850151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 -- 172.21.15.19:0/689907304 >> 172.21.15.19:0/689907304 conn(0x7f6850061c30 msgr2=0x7f68500ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 -- 172.21.15.19:0/689907304 shutdown_connections 2024-04-25T10:00:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.361+0000 7f6855a4c640 1 -- 172.21.15.19:0/689907304 wait complete. 2024-04-25T10:00:30.376 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:00:30.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:00:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.477+0000 7fbd677f9640 1 Processor -- start 2024-04-25T10:00:30.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.477+0000 7fbd677f9640 1 -- start start 2024-04-25T10:00:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd677f9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbd6015b7d0 0x7fbd600cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd677f9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbd6006c9e0 0x7fbd600b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd677f9640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6006cf50 con 0x7fbd600b5620 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd677f9640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd600ce530 con 0x7fbd6006c9e0 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd677f9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd600ce810 con 0x7fbd6015b7d0 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd65d96640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbd6006c9e0 0x7fbd600b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd64d94640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbd600b5620 0x7fbd6006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47034/0 (socket says 172.21.15.19:47034) 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd64d94640 1 -- 172.21.15.19:0/833656200 learned_addr learned my addr 172.21.15.19:0/833656200 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd65d96640 1 --2- 172.21.15.19:0/833656200 >> v2:172.21.15.19:3300/0 conn(0x7fbd6006c9e0 0x7fbd600b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44408/0 (socket says 172.21.15.19:44408) 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd65595640 1 --2- 172.21.15.19:0/833656200 >> v2:172.21.15.112:3300/0 conn(0x7fbd6015b7d0 0x7fbd600cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd65d96640 1 -- 172.21.15.19:0/833656200 >> v2:172.21.15.112:3300/0 conn(0x7fbd6015b7d0 msgr2=0x7fbd600cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd65d96640 1 --2- 172.21.15.19:0/833656200 >> v2:172.21.15.112:3300/0 conn(0x7fbd6015b7d0 0x7fbd600cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd65d96640 1 -- 172.21.15.19:0/833656200 >> v1:172.21.15.19:6789/0 conn(0x7fbd600b5620 legacy=0x7fbd6006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd65d96640 1 -- 172.21.15.19:0/833656200 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd600ceaf0 con 0x7fbd6006c9e0 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd57fff640 1 -- 172.21.15.19:0/833656200 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414253258 0 0) 0x7fbd6006cf50 con 0x7fbd600b5620 2024-04-25T10:00:30.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.478+0000 7fbd65595640 1 --2- 172.21.15.19:0/833656200 >> v2:172.21.15.112:3300/0 conn(0x7fbd6015b7d0 0x7fbd600cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:30.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.479+0000 7fbd65d96640 1 --2- 172.21.15.19:0/833656200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd6006c9e0 0x7fbd600b5ab0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fbd6008c0d0 tx=0x7fbd5802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcabdf4bbcf55f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:30.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.479+0000 7fbd57fff640 1 -- 172.21.15.19:0/833656200 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd58007740 con 0x7fbd6006c9e0 2024-04-25T10:00:30.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.479+0000 7fbd57fff640 1 -- 172.21.15.19:0/833656200 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd580078e0 con 0x7fbd6006c9e0 2024-04-25T10:00:30.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.479+0000 7fbd57fff640 1 -- 172.21.15.19:0/833656200 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd58007cb0 con 0x7fbd6006c9e0 2024-04-25T10:00:30.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.479+0000 7fbd677f9640 1 -- 172.21.15.19:0/833656200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd6006c9e0 msgr2=0x7fbd600b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.479+0000 7fbd677f9640 1 --2- 172.21.15.19:0/833656200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd6006c9e0 0x7fbd600b5ab0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fbd6008c0d0 tx=0x7fbd5802a680 comp rx=0 tx=0).stop 2024-04-25T10:00:30.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.480+0000 7fbd677f9640 1 -- 172.21.15.19:0/833656200 shutdown_connections 2024-04-25T10:00:30.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.480+0000 7fbd677f9640 1 --2- 172.21.15.19:0/833656200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd6006c9e0 0x7fbd600b5ab0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.480+0000 7fbd677f9640 1 --2- 172.21.15.19:0/833656200 >> v2:172.21.15.112:3300/0 conn(0x7fbd6015b7d0 0x7fbd600cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.480+0000 7fbd677f9640 1 -- 172.21.15.19:0/833656200 >> 172.21.15.19:0/833656200 conn(0x7fbd600d0800 msgr2=0x7fbd600d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:30.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.480+0000 7fbd677f9640 1 -- 172.21.15.19:0/833656200 shutdown_connections 2024-04-25T10:00:30.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.480+0000 7fbd677f9640 1 -- 172.21.15.19:0/833656200 wait complete. 2024-04-25T10:00:30.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.481+0000 7fbd677f9640 1 Processor -- start 2024-04-25T10:00:30.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.481+0000 7fbd677f9640 1 -- start start 2024-04-25T10:00:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd677f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6006c9e0 0x7fbd60159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd677f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd600b5620 0x7fbd600c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd677f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd6015b7d0 0x7fbd600b8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd677f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600cf6d0 con 0x7fbd600b5620 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd677f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600cf550 con 0x7fbd6006c9e0 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd677f9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6015bbb0 con 0x7fbd6015b7d0 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd64d94640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd600b5620 0x7fbd600c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd64d94640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd600b5620 0x7fbd600c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44420/0 (socket says 172.21.15.19:44420) 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd64d94640 1 -- 172.21.15.19:0/1815228713 learned_addr learned my addr 172.21.15.19:0/1815228713 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd65595640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6006c9e0 0x7fbd60159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd65d96640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd6015b7d0 0x7fbd600b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd64d94640 1 -- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd6015b7d0 msgr2=0x7fbd600b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd64d94640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd6015b7d0 0x7fbd600b8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd64d94640 1 -- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6006c9e0 msgr2=0x7fbd60159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd64d94640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6006c9e0 0x7fbd60159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd64d94640 1 -- 172.21.15.19:0/1815228713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd600b7480 con 0x7fbd600b5620 2024-04-25T10:00:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd65595640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6006c9e0 0x7fbd60159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd64d94640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd600b5620 0x7fbd600c84e0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000c500 tx=0x7fbd5000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd55ffb640 1 -- 172.21.15.19:0/1815228713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50018070 con 0x7fbd600b5620 2024-04-25T10:00:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd55ffb640 1 -- 172.21.15.19:0/1815228713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd50007580 con 0x7fbd600b5620 2024-04-25T10:00:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd677f9640 1 -- 172.21.15.19:0/1815228713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd600b7760 con 0x7fbd600b5620 2024-04-25T10:00:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.482+0000 7fbd55ffb640 1 -- 172.21.15.19:0/1815228713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50004a20 con 0x7fbd600b5620 2024-04-25T10:00:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.483+0000 7fbd677f9640 1 -- 172.21.15.19:0/1815228713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd600b7d20 con 0x7fbd600b5620 2024-04-25T10:00:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.484+0000 7fbd677f9640 1 -- 172.21.15.19:0/1815228713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd60073a00 con 0x7fbd600b5620 2024-04-25T10:00:30.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.484+0000 7fbd55ffb640 1 -- 172.21.15.19:0/1815228713 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd5001b070 con 0x7fbd600b5620 2024-04-25T10:00:30.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.484+0000 7fbd55ffb640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbd4406b4d0 0x7fbd4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.484+0000 7fbd55ffb640 1 -- 172.21.15.19:0/1815228713 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbd500893d0 con 0x7fbd600b5620 2024-04-25T10:00:30.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.485+0000 7fbd65595640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbd4406b4d0 0x7fbd4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.485+0000 7fbd65595640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbd4406b4d0 0x7fbd4406d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c006fd0 tx=0x7fbd5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:30.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.488+0000 7fbd55ffb640 1 -- 172.21.15.19:0/1815228713 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd50058400 con 0x7fbd600b5620 2024-04-25T10:00:30.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:30.670+0000 7fbd677f9640 1 -- 172.21.15.19:0/1815228713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbd600d0d10 con 0x7fbd600b5620 2024-04-25T10:00:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.043+0000 7fbd55ffb640 1 -- 172.21.15.19:0/1815228713 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v114) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbd50057d00 con 0x7fbd600b5620 2024-04-25T10:00:31.048 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T10:00:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 -- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbd4406b4d0 msgr2=0x7fbd4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbd4406b4d0 0x7fbd4406d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c006fd0 tx=0x7fbd5c008040 comp rx=0 tx=0).stop 2024-04-25T10:00:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 -- 172.21.15.19:0/1815228713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd600b5620 msgr2=0x7fbd600c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd600b5620 0x7fbd600c84e0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000c500 tx=0x7fbd5000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:00:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 -- 172.21.15.19:0/1815228713 shutdown_connections 2024-04-25T10:00:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbd4406b4d0 0x7fbd4406d990 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd6015b7d0 0x7fbd600b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd600b5620 0x7fbd600c84e0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 --2- 172.21.15.19:0/1815228713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6006c9e0 0x7fbd60159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 -- 172.21.15.19:0/1815228713 >> 172.21.15.19:0/1815228713 conn(0x7fbd600d0800 msgr2=0x7fbd6015afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 -- 172.21.15.19:0/1815228713 shutdown_connections 2024-04-25T10:00:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.046+0000 7fbd677f9640 1 -- 172.21.15.19:0/1815228713 wait complete. 2024-04-25T10:00:31.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:00:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.170+0000 7f0c31bb7640 1 Processor -- start 2024-04-25T10:00:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.170+0000 7f0c31bb7640 1 -- start start 2024-04-25T10:00:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c31bb7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c2c0d34c0 0x7f0c2c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c31bb7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0c2c0d08e0 0x7f0c2c0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c31bb7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2c0d2420 con 0x7f0c2c15dac0 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c31bb7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2c0d0d50 con 0x7f0c2c0d34c0 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c31bb7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2c0d3030 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c2affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c2c0d34c0 0x7f0c2c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c2affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c2c0d34c0 0x7f0c2c0d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44430/0 (socket says 172.21.15.19:44430) 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c2affd640 1 -- 172.21.15.19:0/2850261217 learned_addr learned my addr 172.21.15.19:0/2850261217 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c2bfff640 1 --2- 172.21.15.19:0/2850261217 >> v2:172.21.15.112:3300/0 conn(0x7f0c2c0d08e0 0x7f0c2c0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c2affd640 1 -- 172.21.15.19:0/2850261217 >> v2:172.21.15.112:3300/0 conn(0x7f0c2c0d08e0 msgr2=0x7f0c2c0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c2affd640 1 --2- 172.21.15.19:0/2850261217 >> v2:172.21.15.112:3300/0 conn(0x7f0c2c0d08e0 0x7f0c2c0b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c2affd640 1 -- 172.21.15.19:0/2850261217 >> v1:172.21.15.112:6789/0 conn(0x7f0c2c15dac0 legacy=0x7f0c2c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.171+0000 7f0c2affd640 1 -- 172.21.15.19:0/2850261217 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c0d2700 con 0x7f0c2c0d34c0 2024-04-25T10:00:31.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.172+0000 7f0c2affd640 1 --2- 172.21.15.19:0/2850261217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2c0d34c0 0x7f0c2c0d03a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0d2fa0 tx=0x7f0c2002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497e2e5f50355671 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:31.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.172+0000 7f0c2a7fc640 1 -- 172.21.15.19:0/2850261217 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20004230 con 0x7f0c2c0d34c0 2024-04-25T10:00:31.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.172+0000 7f0c2a7fc640 1 -- 172.21.15.19:0/2850261217 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2002ae50 con 0x7f0c2c0d34c0 2024-04-25T10:00:31.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.172+0000 7f0c2a7fc640 1 -- 172.21.15.19:0/2850261217 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20003610 con 0x7f0c2c0d34c0 2024-04-25T10:00:31.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.173+0000 7f0c31bb7640 1 -- 172.21.15.19:0/2850261217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2c0d34c0 msgr2=0x7f0c2c0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:31.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.173+0000 7f0c31bb7640 1 --2- 172.21.15.19:0/2850261217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2c0d34c0 0x7f0c2c0d03a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0d2fa0 tx=0x7f0c2002a300 comp rx=0 tx=0).stop 2024-04-25T10:00:31.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.173+0000 7f0c31bb7640 1 -- 172.21.15.19:0/2850261217 shutdown_connections 2024-04-25T10:00:31.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.173+0000 7f0c31bb7640 1 --2- 172.21.15.19:0/2850261217 >> v2:172.21.15.112:3300/0 conn(0x7f0c2c0d08e0 0x7f0c2c0b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:31.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.173+0000 7f0c31bb7640 1 --2- 172.21.15.19:0/2850261217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2c0d34c0 0x7f0c2c0d03a0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:31.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.173+0000 7f0c31bb7640 1 -- 172.21.15.19:0/2850261217 >> 172.21.15.19:0/2850261217 conn(0x7f0c2c061c30 msgr2=0x7f0c2c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:31.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.173+0000 7f0c31bb7640 1 -- 172.21.15.19:0/2850261217 shutdown_connections 2024-04-25T10:00:31.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.173+0000 7f0c31bb7640 1 -- 172.21.15.19:0/2850261217 wait complete. 2024-04-25T10:00:31.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.174+0000 7f0c31bb7640 1 Processor -- start 2024-04-25T10:00:31.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.174+0000 7f0c31bb7640 1 -- start start 2024-04-25T10:00:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c31bb7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2c0d08e0 0x7f0c2c0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c31bb7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c2c0d34c0 0x7f0c2c1f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c31bb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c2c15dac0 0x7f0c2c0cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c31bb7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0d2100 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c31bb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0d2cc0 con 0x7f0c2c15dac0 2024-04-25T10:00:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c31bb7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c15dea0 con 0x7f0c2c0d34c0 2024-04-25T10:00:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2c0d08e0 0x7f0c2c0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2c0d08e0 0x7f0c2c0cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44446/0 (socket says 172.21.15.19:44446) 2024-04-25T10:00:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c2b7fe640 1 -- 172.21.15.19:0/3554599321 learned_addr learned my addr 172.21.15.19:0/3554599321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c2bfff640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c2c15dac0 0x7f0c2c0cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c2affd640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c2c0d34c0 0x7f0c2c1f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c2b7fe640 1 -- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c2c0d34c0 msgr2=0x7f0c2c1f7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c2b7fe640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c2c0d34c0 0x7f0c2c1f7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c2b7fe640 1 -- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c2c15dac0 msgr2=0x7f0c2c0cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c2b7fe640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c2c15dac0 0x7f0c2c0cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.175+0000 7f0c2b7fe640 1 -- 172.21.15.19:0/3554599321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c1f6fb0 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.176+0000 7f0c2b7fe640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2c0d08e0 0x7f0c2c0cda40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c3c0 tx=0x7f0c1c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.176+0000 7f0c13fff640 1 -- 172.21.15.19:0/3554599321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0052d0 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.176+0000 7f0c13fff640 1 -- 172.21.15.19:0/3554599321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c005490 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.176+0000 7f0c31bb7640 1 -- 172.21.15.19:0/3554599321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c0cc020 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.176+0000 7f0c13fff640 1 -- 172.21.15.19:0/3554599321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c007260 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.176+0000 7f0c31bb7640 1 -- 172.21.15.19:0/3554599321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2c1f7500 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.177+0000 7f0c31bb7640 1 -- 172.21.15.19:0/3554599321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2c073a00 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.177+0000 7f0c13fff640 1 -- 172.21.15.19:0/3554599321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c1c007400 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.178+0000 7f0c13fff640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c1806b4d0 0x7f0c1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.178+0000 7f0c13fff640 1 -- 172.21.15.19:0/3554599321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f0c1c0895a0 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.178+0000 7f0c2affd640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c1806b4d0 0x7f0c1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.178+0000 7f0c2affd640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c1806b4d0 0x7f0c1806d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0c20033000 tx=0x7f0c2002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:31.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.182+0000 7f0c13fff640 1 -- 172.21.15.19:0/3554599321 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c1c057e20 con 0x7f0c2c0d08e0 2024-04-25T10:00:31.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:31.364+0000 7f0c31bb7640 1 -- 172.21.15.19:0/3554599321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0c2c1f76d0 con 0x7f0c2c0d08e0 2024-04-25T10:00:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.046+0000 7f0c13fff640 1 -- 172.21.15.19:0/3554599321 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v115) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0c1c05bec0 con 0x7f0c2c0d08e0 2024-04-25T10:00:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T10:00:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 -- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c1806b4d0 msgr2=0x7f0c1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c1806b4d0 0x7f0c1806d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0c20033000 tx=0x7f0c2002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 -- 172.21.15.19:0/3554599321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2c0d08e0 msgr2=0x7f0c2c0cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2c0d08e0 0x7f0c2c0cda40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c3c0 tx=0x7f0c1c00c880 comp rx=0 tx=0).stop 2024-04-25T10:00:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 -- 172.21.15.19:0/3554599321 shutdown_connections 2024-04-25T10:00:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c1806b4d0 0x7f0c1806d990 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c2c15dac0 0x7f0c2c0cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c2c0d34c0 0x7f0c2c1f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 --2- 172.21.15.19:0/3554599321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2c0d08e0 0x7f0c2c0cda40 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 -- 172.21.15.19:0/3554599321 >> 172.21.15.19:0/3554599321 conn(0x7f0c2c061c30 msgr2=0x7f0c2c1600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 -- 172.21.15.19:0/3554599321 shutdown_connections 2024-04-25T10:00:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.049+0000 7f0c31bb7640 1 -- 172.21.15.19:0/3554599321 wait complete. 2024-04-25T10:00:32.078 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:00:32.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.182+0000 7f729b7ce640 1 Processor -- start 2024-04-25T10:00:32.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.182+0000 7f729b7ce640 1 -- start start 2024-04-25T10:00:32.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.183+0000 7f729b7ce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f72941446d0 0x7f72940b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.183+0000 7f729b7ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294144d50 0x7f72940b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.183+0000 7f729b7ce640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72940cccc0 con 0x7f72941452a0 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.183+0000 7f729b7ce640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72940b2b20 con 0x7f72941446d0 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.183+0000 7f729b7ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729406d400 con 0x7f7294144d50 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.183+0000 7f7299d6b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f72941452a0 0x7f72940cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47040/0 (socket says 172.21.15.19:47040) 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.183+0000 7f7299d6b640 1 -- 172.21.15.19:0/1306564677 learned_addr learned my addr 172.21.15.19:0/1306564677 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.183+0000 7f729956a640 1 --2- 172.21.15.19:0/1306564677 >> v2:172.21.15.112:3300/0 conn(0x7f72941446d0 0x7f72940b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.183+0000 7f7298d69640 1 --2- 172.21.15.19:0/1306564677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294144d50 0x7f72940b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.183+0000 7f7293fff640 1 -- 172.21.15.19:0/1306564677 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924981593 0 0) 0x7f72940cccc0 con 0x7f72941452a0 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.183+0000 7f7293fff640 1 -- 172.21.15.19:0/1306564677 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f727c003520 con 0x7f72941452a0 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.184+0000 7f7293fff640 1 -- 172.21.15.19:0/1306564677 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1853878014 0 0) 0x7f727c003520 con 0x7f72941452a0 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.184+0000 7f729956a640 1 -- 172.21.15.19:0/1306564677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294144d50 msgr2=0x7f72940b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.184+0000 7f729956a640 1 --2- 172.21.15.19:0/1306564677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294144d50 0x7f72940b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.184+0000 7f729956a640 1 -- 172.21.15.19:0/1306564677 >> v1:172.21.15.19:6789/0 conn(0x7f72941452a0 legacy=0x7f72940cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.184+0000 7f729956a640 1 -- 172.21.15.19:0/1306564677 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72940cc140 con 0x7f72941446d0 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.184+0000 7f7298d69640 1 --2- 172.21.15.19:0/1306564677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294144d50 0x7f72940b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.184+0000 7f729956a640 1 --2- 172.21.15.19:0/1306564677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72941446d0 0x7f72940b3f20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f728c0097a0 tx=0x7f728c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=288705614ec8dde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.185+0000 7f7293fff640 1 -- 172.21.15.19:0/1306564677 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c037070 con 0x7f72941446d0 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.185+0000 7f7293fff640 1 -- 172.21.15.19:0/1306564677 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c02aba0 con 0x7f72941446d0 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.185+0000 7f7293fff640 1 -- 172.21.15.19:0/1306564677 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72840019f0 con 0x7f72941446d0 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.185+0000 7f729b7ce640 1 -- 172.21.15.19:0/1306564677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72941446d0 msgr2=0x7f72940b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.185+0000 7f729b7ce640 1 --2- 172.21.15.19:0/1306564677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72941446d0 0x7f72940b3f20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f728c0097a0 tx=0x7f728c02a110 comp rx=0 tx=0).stop 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.185+0000 7f729b7ce640 1 -- 172.21.15.19:0/1306564677 shutdown_connections 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.185+0000 7f729b7ce640 1 --2- 172.21.15.19:0/1306564677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294144d50 0x7f72940b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.185+0000 7f729b7ce640 1 --2- 172.21.15.19:0/1306564677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72941446d0 0x7f72940b3f20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.185+0000 7f729b7ce640 1 -- 172.21.15.19:0/1306564677 >> 172.21.15.19:0/1306564677 conn(0x7f729406abe0 msgr2=0x7f729406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.185+0000 7f729b7ce640 1 -- 172.21.15.19:0/1306564677 shutdown_connections 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.185+0000 7f729b7ce640 1 -- 172.21.15.19:0/1306564677 wait complete. 2024-04-25T10:00:32.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.186+0000 7f729b7ce640 1 Processor -- start 2024-04-25T10:00:32.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.187+0000 7f729b7ce640 1 -- start start 2024-04-25T10:00:32.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.187+0000 7f729b7ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72941446d0 0x7f729413e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.187+0000 7f729b7ce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7294144d50 0x7f72941e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.187+0000 7f729b7ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72941452a0 0x7f72941e2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.187+0000 7f729b7ce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72940cc700 con 0x7f7294144d50 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.187+0000 7f729b7ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72940b49a0 con 0x7f72941452a0 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.187+0000 7f729b7ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72940cc9e0 con 0x7f72941446d0 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.187+0000 7f7298d69640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7294144d50 0x7f72941e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.187+0000 7f7298d69640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7294144d50 0x7f72941e0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44458/0 (socket says 172.21.15.19:44458) 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.187+0000 7f7298d69640 1 -- 172.21.15.19:0/3728552452 learned_addr learned my addr 172.21.15.19:0/3728552452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f7299d6b640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72941452a0 0x7f72941e2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f729956a640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72941446d0 0x7f729413e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f7298d69640 1 -- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72941446d0 msgr2=0x7f729413e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f7298d69640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72941446d0 0x7f729413e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f7298d69640 1 -- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72941452a0 msgr2=0x7f72941e2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f7298d69640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72941452a0 0x7f72941e2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f7298d69640 1 -- 172.21.15.19:0/3728552452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729414ea90 con 0x7f7294144d50 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f729956a640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72941446d0 0x7f729413e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f7298d69640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7294144d50 0x7f72941e0f90 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f728800ec10 tx=0x7f728800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f7291ffb640 1 -- 172.21.15.19:0/3728552452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288017070 con 0x7f7294144d50 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f7291ffb640 1 -- 172.21.15.19:0/3728552452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728800cc30 con 0x7f7294144d50 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f729b7ce640 1 -- 172.21.15.19:0/3728552452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72941e1c10 con 0x7f7294144d50 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f7291ffb640 1 -- 172.21.15.19:0/3728552452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288013470 con 0x7f7294144d50 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.188+0000 7f729b7ce640 1 -- 172.21.15.19:0/3728552452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72941e21d0 con 0x7f7294144d50 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.189+0000 7f729b7ce640 1 -- 172.21.15.19:0/3728552452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7294073a00 con 0x7f7294144d50 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.190+0000 7f7291ffb640 1 -- 172.21.15.19:0/3728552452 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f728801a070 con 0x7f7294144d50 2024-04-25T10:00:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.190+0000 7f7291ffb640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728006b4d0 0x7f728006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.190+0000 7f7291ffb640 1 -- 172.21.15.19:0/3728552452 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f728808a5d0 con 0x7f7294144d50 2024-04-25T10:00:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.191+0000 7f729956a640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728006b4d0 0x7f728006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.191+0000 7f729956a640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728006b4d0 0x7f728006d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f728c034000 tx=0x7f728c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.195+0000 7f7291ffb640 1 -- 172.21.15.19:0/3728552452 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7288058c30 con 0x7f7294144d50 2024-04-25T10:00:32.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:32.411+0000 7f729b7ce640 1 -- 172.21.15.19:0/3728552452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7294140c90 con 0x7f7294144d50 2024-04-25T10:00:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.060+0000 7f7291ffb640 1 -- 172.21.15.19:0/3728552452 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v70) v1 ==== 174+0+0 (secure 0 0 0) 0x7f728805ccd0 con 0x7f7294144d50 2024-04-25T10:00:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T10:00:33.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.069+0000 7f729b7ce640 1 -- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728006b4d0 msgr2=0x7f728006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.069+0000 7f729b7ce640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728006b4d0 0x7f728006d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f728c034000 tx=0x7f728c02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:33.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.069+0000 7f729b7ce640 1 -- 172.21.15.19:0/3728552452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7294144d50 msgr2=0x7f72941e0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.069+0000 7f729b7ce640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7294144d50 0x7f72941e0f90 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f728800ec10 tx=0x7f728800c520 comp rx=0 tx=0).stop 2024-04-25T10:00:33.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.071+0000 7f729b7ce640 1 -- 172.21.15.19:0/3728552452 shutdown_connections 2024-04-25T10:00:33.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.071+0000 7f729b7ce640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728006b4d0 0x7f728006d990 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.071+0000 7f729b7ce640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72941452a0 0x7f72941e2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.071+0000 7f729b7ce640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7294144d50 0x7f72941e0f90 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.071+0000 7f729b7ce640 1 --2- 172.21.15.19:0/3728552452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72941446d0 0x7f729413e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.071+0000 7f729b7ce640 1 -- 172.21.15.19:0/3728552452 >> 172.21.15.19:0/3728552452 conn(0x7f729406abe0 msgr2=0x7f72940b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.071+0000 7f729b7ce640 1 -- 172.21.15.19:0/3728552452 shutdown_connections 2024-04-25T10:00:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.072+0000 7f729b7ce640 1 -- 172.21.15.19:0/3728552452 wait complete. 2024-04-25T10:00:33.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.208+0000 7f58e01c1640 1 Processor -- start 2024-04-25T10:00:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.208+0000 7f58e01c1640 1 -- start start 2024-04-25T10:00:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58e01c1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58d80ce330 0x7f58d806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58e01c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58d80700a0 0x7f58d815d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58e01c1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d81626e0 con 0x7f58d806abe0 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58e01c1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d80cd270 con 0x7f58d80ce330 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58e01c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80cd550 con 0x7f58d80700a0 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58dd75c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58d80ce330 0x7f58d806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58dd75c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58d80ce330 0x7f58d806fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44474/0 (socket says 172.21.15.19:44474) 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58dd75c640 1 -- 172.21.15.19:0/231616326 learned_addr learned my addr 172.21.15.19:0/231616326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58ddf5d640 1 --1- 172.21.15.19:0/231616326 >> v1:172.21.15.19:6789/0 conn(0x7f58d806abe0 0x7f58d806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47054/0 (socket says 172.21.15.19:47054) 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58de75e640 1 --2- 172.21.15.19:0/231616326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58d80700a0 0x7f58d815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58dd75c640 1 -- 172.21.15.19:0/231616326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58d80700a0 msgr2=0x7f58d815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58dd75c640 1 --2- 172.21.15.19:0/231616326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58d80700a0 0x7f58d815d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58dd75c640 1 -- 172.21.15.19:0/231616326 >> v1:172.21.15.19:6789/0 conn(0x7f58d806abe0 legacy=0x7f58d806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.210+0000 7f58dd75c640 1 -- 172.21.15.19:0/231616326 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d80cd830 con 0x7f58d80ce330 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.209+0000 7f58dcf5b640 1 -- 172.21.15.19:0/231616326 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1074922956 0 0) 0x7f58d81626e0 con 0x7f58d806abe0 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.210+0000 7f58de75e640 1 --2- 172.21.15.19:0/231616326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58d80700a0 0x7f58d815d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.210+0000 7f58dd75c640 1 --2- 172.21.15.19:0/231616326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80ce330 0x7f58d806fb60 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009870 tx=0x7f58c802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedfacdfc221bbcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.210+0000 7f58dcf5b640 1 -- 172.21.15.19:0/231616326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8037070 con 0x7f58d80ce330 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.210+0000 7f58dcf5b640 1 -- 172.21.15.19:0/231616326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8032440 con 0x7f58d80ce330 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.210+0000 7f58dcf5b640 1 -- 172.21.15.19:0/231616326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8032810 con 0x7f58d80ce330 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.211+0000 7f58e01c1640 1 -- 172.21.15.19:0/231616326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80ce330 msgr2=0x7f58d806fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.211+0000 7f58e01c1640 1 --2- 172.21.15.19:0/231616326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80ce330 0x7f58d806fb60 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009870 tx=0x7f58c802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.211+0000 7f58e01c1640 1 -- 172.21.15.19:0/231616326 shutdown_connections 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.211+0000 7f58e01c1640 1 --2- 172.21.15.19:0/231616326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58d80700a0 0x7f58d815d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.211+0000 7f58e01c1640 1 --2- 172.21.15.19:0/231616326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80ce330 0x7f58d806fb60 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.211+0000 7f58e01c1640 1 -- 172.21.15.19:0/231616326 >> 172.21.15.19:0/231616326 conn(0x7f58d80b2220 msgr2=0x7f58d80b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.211+0000 7f58e01c1640 1 -- 172.21.15.19:0/231616326 shutdown_connections 2024-04-25T10:00:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.211+0000 7f58e01c1640 1 -- 172.21.15.19:0/231616326 wait complete. 2024-04-25T10:00:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.212+0000 7f58e01c1640 1 Processor -- start 2024-04-25T10:00:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.212+0000 7f58e01c1640 1 -- start start 2024-04-25T10:00:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58e01c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58d806abe0 0x7f58d809a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58e01c1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80700a0 0x7f58d809bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58e01c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d80ce330 0x7f58d815fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58e01c1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d815d8b0 con 0x7f58d80700a0 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58e01c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d8161780 con 0x7f58d80ce330 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58e01c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d815da30 con 0x7f58d806abe0 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58dd75c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80700a0 0x7f58d809bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58dd75c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80700a0 0x7f58d809bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44486/0 (socket says 172.21.15.19:44486) 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58dd75c640 1 -- 172.21.15.19:0/2312424630 learned_addr learned my addr 172.21.15.19:0/2312424630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58ddf5d640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58d806abe0 0x7f58d809a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58de75e640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d80ce330 0x7f58d815fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58dd75c640 1 -- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58d806abe0 msgr2=0x7f58d809a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58dd75c640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58d806abe0 0x7f58d809a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58dd75c640 1 -- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d80ce330 msgr2=0x7f58d815fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.213+0000 7f58dd75c640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d80ce330 0x7f58d815fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.214+0000 7f58dd75c640 1 -- 172.21.15.19:0/2312424630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d809ab50 con 0x7f58d80700a0 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.214+0000 7f58de75e640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d80ce330 0x7f58d815fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.214+0000 7f58dd75c640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80700a0 0x7f58d809bfe0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f58c802a200 tx=0x7f58c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.214+0000 7f58ce7fc640 1 -- 172.21.15.19:0/2312424630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8041020 con 0x7f58d80700a0 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.214+0000 7f58ce7fc640 1 -- 172.21.15.19:0/2312424630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c80306e0 con 0x7f58d80700a0 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.214+0000 7f58ce7fc640 1 -- 172.21.15.19:0/2312424630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8037040 con 0x7f58d80700a0 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.214+0000 7f58e01c1640 1 -- 172.21.15.19:0/2312424630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d815e6a0 con 0x7f58d80700a0 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.214+0000 7f58e01c1640 1 -- 172.21.15.19:0/2312424630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d815ec60 con 0x7f58d80700a0 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.215+0000 7f58e01c1640 1 -- 172.21.15.19:0/2312424630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d8073990 con 0x7f58d80700a0 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.216+0000 7f58ce7fc640 1 -- 172.21.15.19:0/2312424630 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58c8044020 con 0x7f58d80700a0 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.216+0000 7f58ce7fc640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58bc06b4d0 0x7f58bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.216+0000 7f58ce7fc640 1 -- 172.21.15.19:0/2312424630 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58c80aabf0 con 0x7f58d80700a0 2024-04-25T10:00:33.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.216+0000 7f58ddf5d640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58bc06b4d0 0x7f58bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.217+0000 7f58ddf5d640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58bc06b4d0 0x7f58bc06d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f58d4000f60 tx=0x7f58d4007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.221+0000 7f58ce7fc640 1 -- 172.21.15.19:0/2312424630 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58c804b030 con 0x7f58d80700a0 2024-04-25T10:00:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.405+0000 7f58e01c1640 1 -- 172.21.15.19:0/2312424630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58d80b2ff0 con 0x7f58d80700a0 2024-04-25T10:00:33.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.406+0000 7f58ce7fc640 1 -- 172.21.15.19:0/2312424630 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16953 (secure 0 0 0) 0x7f58c80798e0 con 0x7f58d80700a0 2024-04-25T10:00:33.410 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:33.410 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":116,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:00:32.412270+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:30.702453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:31.396192+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:33.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 -- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58bc06b4d0 msgr2=0x7f58bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58bc06b4d0 0x7f58bc06d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f58d4000f60 tx=0x7f58d4007b50 comp rx=0 tx=0).stop 2024-04-25T10:00:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 -- 172.21.15.19:0/2312424630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80700a0 msgr2=0x7f58d809bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80700a0 0x7f58d809bfe0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f58c802a200 tx=0x7f58c80046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 -- 172.21.15.19:0/2312424630 shutdown_connections 2024-04-25T10:00:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58bc06b4d0 0x7f58bc06d990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58d80ce330 0x7f58d815fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80700a0 0x7f58d809bfe0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 --2- 172.21.15.19:0/2312424630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58d806abe0 0x7f58d809a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 -- 172.21.15.19:0/2312424630 >> 172.21.15.19:0/2312424630 conn(0x7f58d80b2220 msgr2=0x7f58d806f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 -- 172.21.15.19:0/2312424630 shutdown_connections 2024-04-25T10:00:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.408+0000 7f58e01c1640 1 -- 172.21.15.19:0/2312424630 wait complete. 2024-04-25T10:00:33.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.521+0000 7f082665d640 1 Processor -- start 2024-04-25T10:00:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.521+0000 7f082665d640 1 -- start start 2024-04-25T10:00:33.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f082665d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08200d2e90 0x7f08200d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f082665d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08200b9940 con 0x7f08200d0900 2024-04-25T10:00:33.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f082665d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08200d1df0 con 0x7f082015dd30 2024-04-25T10:00:33.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f082665d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08200d0d70 con 0x7f08200d2e90 2024-04-25T10:00:33.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f0824bfa640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f08200d0900 0x7f08200b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47070/0 (socket says 172.21.15.19:47070) 2024-04-25T10:00:33.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f0824bfa640 1 -- 172.21.15.19:0/2159703073 learned_addr learned my addr 172.21.15.19:0/2159703073 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:33.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081f7fe640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08200d2e90 0x7f08200d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243445260 0 0) 0x7f08200b9940 con 0x7f08200d0900 2024-04-25T10:00:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0808003520 con 0x7f08200d0900 2024-04-25T10:00:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3699793093 0 0) 0x7f08200d1df0 con 0x7f082015dd30 2024-04-25T10:00:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08200b9940 con 0x7f082015dd30 2024-04-25T10:00:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3418343355 0 0) 0x7f0808003520 con 0x7f08200d0900 2024-04-25T10:00:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08200d2e90 msgr2=0x7f08200d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08200d2e90 0x7f08200d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 >> v1:172.21.15.112:6789/0 conn(0x7f082015dd30 legacy=0x7f08200b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08200d20d0 con 0x7f08200d0900 2024-04-25T10:00:33.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f08180019f0 con 0x7f08200d0900 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 >> v1:172.21.15.19:6789/0 conn(0x7f08200d0900 legacy=0x7f08200b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0808003b70 0x7f0808003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808004800 0x7f0808004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08200d2e90 0x7f0808005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08200d0d70 con 0x7f0808003b70 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080800bb50 con 0x7f0808004800 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.522+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080800bd20 con 0x7f08200d2e90 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.523+0000 7f081ffff640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0808003b70 0x7f0808003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.523+0000 7f081f7fe640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808004800 0x7f0808004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.523+0000 7f0824bfa640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08200d2e90 0x7f0808005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.523+0000 7f081ffff640 1 -- 172.21.15.19:0/2159703073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08200d2e90 msgr2=0x7f0808005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.523+0000 7f081ffff640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08200d2e90 0x7f0808005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.523+0000 7f081ffff640 1 -- 172.21.15.19:0/2159703073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808004800 msgr2=0x7f0808004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.523+0000 7f081ffff640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808004800 0x7f0808004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.523+0000 7f081ffff640 1 -- 172.21.15.19:0/2159703073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0808003520 con 0x7f0808003b70 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.524+0000 7f081ffff640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0808003b70 0x7f0808003f70 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f0810001900 tx=0x7f0808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e104e6b08faee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.524+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08100014f0 con 0x7f0808003b70 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.524+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08100052d0 con 0x7f0808003b70 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.524+0000 7f081effd640 1 -- 172.21.15.19:0/2159703073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081002ac80 con 0x7f0808003b70 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.524+0000 7f082665d640 1 -- 172.21.15.19:0/2159703073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0808003b70 msgr2=0x7f0808003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.524+0000 7f082665d640 1 --2- 172.21.15.19:0/2159703073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0808003b70 0x7f0808003f70 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f0810001900 tx=0x7f0808005830 comp rx=0 tx=0).stop 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.524+0000 7f0824bfa640 1 -- 172.21.15.19:0/2159703073 reap_dead start 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.524+0000 7f082665d640 1 -- 172.21.15.19:0/2159703073 shutdown_connections 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.524+0000 7f082665d640 1 -- 172.21.15.19:0/2159703073 >> 172.21.15.19:0/2159703073 conn(0x7f0820061c30 msgr2=0x7f082006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.524+0000 7f082665d640 1 -- 172.21.15.19:0/2159703073 shutdown_connections 2024-04-25T10:00:33.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.524+0000 7f082665d640 1 -- 172.21.15.19:0/2159703073 wait complete. 2024-04-25T10:00:33.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.525+0000 7f082665d640 1 Processor -- start 2024-04-25T10:00:33.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.526+0000 7f082665d640 1 -- start start 2024-04-25T10:00:33.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.526+0000 7f082665d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808003b70 0x7f0820156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.526+0000 7f082665d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0820152e00 0x7f0820156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.526+0000 7f082665d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08201f6f60 0x7f08201f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.526+0000 7f082665d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08200d2970 con 0x7f0820152e00 2024-04-25T10:00:33.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.526+0000 7f082665d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08200d2690 con 0x7f0808003b70 2024-04-25T10:00:33.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.526+0000 7f082665d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08200d1a10 con 0x7f08201f6f60 2024-04-25T10:00:33.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.526+0000 7f081f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0820152e00 0x7f0820156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.526+0000 7f081f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0820152e00 0x7f0820156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44498/0 (socket says 172.21.15.19:44498) 2024-04-25T10:00:33.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081f7fe640 1 -- 172.21.15.19:0/3739400090 learned_addr learned my addr 172.21.15.19:0/3739400090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081ffff640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808003b70 0x7f0820156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f0824bfa640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08201f6f60 0x7f08201f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081f7fe640 1 -- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08201f6f60 msgr2=0x7f08201f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081f7fe640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08201f6f60 0x7f08201f73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081f7fe640 1 -- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808003b70 msgr2=0x7f0820156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081f7fe640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808003b70 0x7f0820156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081f7fe640 1 -- 172.21.15.19:0/3739400090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0820156b40 con 0x7f0820152e00 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081ffff640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808003b70 0x7f0820156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081f7fe640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0820152e00 0x7f0820156d90 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f0814011910 tx=0x7f0814011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081cff9640 1 -- 172.21.15.19:0/3739400090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814005ba0 con 0x7f0820152e00 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081cff9640 1 -- 172.21.15.19:0/3739400090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0814005d40 con 0x7f0820152e00 2024-04-25T10:00:33.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f082665d640 1 -- 172.21.15.19:0/3739400090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0820159560 con 0x7f0820152e00 2024-04-25T10:00:33.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f081cff9640 1 -- 172.21.15.19:0/3739400090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400fb00 con 0x7f0820152e00 2024-04-25T10:00:33.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.527+0000 7f082665d640 1 -- 172.21.15.19:0/3739400090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0820159b20 con 0x7f0820152e00 2024-04-25T10:00:33.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.528+0000 7f082665d640 1 -- 172.21.15.19:0/3739400090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0820073990 con 0x7f0820152e00 2024-04-25T10:00:33.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.529+0000 7f081cff9640 1 -- 172.21.15.19:0/3739400090 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0814012040 con 0x7f0820152e00 2024-04-25T10:00:33.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.529+0000 7f081cff9640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080c06b500 0x7f080c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.529+0000 7f081cff9640 1 -- 172.21.15.19:0/3739400090 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f081408a910 con 0x7f0820152e00 2024-04-25T10:00:33.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.529+0000 7f081ffff640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080c06b500 0x7f080c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.530+0000 7f081ffff640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080c06b500 0x7f080c06d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0810032000 tx=0x7f0810009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.534+0000 7f081cff9640 1 -- 172.21.15.19:0/3739400090 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0814018030 con 0x7f0820152e00 2024-04-25T10:00:33.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.713+0000 7f082665d640 1 -- 172.21.15.19:0/3739400090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08200c9f80 con 0x7f0820152e00 2024-04-25T10:00:33.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.714+0000 7f081cff9640 1 -- 172.21.15.19:0/3739400090 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16953 (secure 0 0 0) 0x7f0814059690 con 0x7f0820152e00 2024-04-25T10:00:33.719 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:33.719 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":116,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:00:32.412270+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:30.702453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:31.396192+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.716+0000 7f082665d640 1 -- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080c06b500 msgr2=0x7f080c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.716+0000 7f082665d640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080c06b500 0x7f080c06d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0810032000 tx=0x7f0810009030 comp rx=0 tx=0).stop 2024-04-25T10:00:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.716+0000 7f082665d640 1 -- 172.21.15.19:0/3739400090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0820152e00 msgr2=0x7f0820156d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.716+0000 7f082665d640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0820152e00 0x7f0820156d90 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f0814011910 tx=0x7f0814011dd0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.716+0000 7f082665d640 1 -- 172.21.15.19:0/3739400090 shutdown_connections 2024-04-25T10:00:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.716+0000 7f082665d640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f080c06b500 0x7f080c06d9c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.716+0000 7f082665d640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08201f6f60 0x7f08201f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.716+0000 7f082665d640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0820152e00 0x7f0820156d90 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.716+0000 7f082665d640 1 --2- 172.21.15.19:0/3739400090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0808003b70 0x7f0820156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.716+0000 7f082665d640 1 -- 172.21.15.19:0/3739400090 >> 172.21.15.19:0/3739400090 conn(0x7f0820061c30 msgr2=0x7f0820160370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:33.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.717+0000 7f082665d640 1 -- 172.21.15.19:0/3739400090 shutdown_connections 2024-04-25T10:00:33.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.717+0000 7f082665d640 1 -- 172.21.15.19:0/3739400090 wait complete. 2024-04-25T10:00:33.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:00:33.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.832+0000 7f4ced3c1640 1 Processor -- start 2024-04-25T10:00:33.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.832+0000 7f4ced3c1640 1 -- start start 2024-04-25T10:00:33.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.832+0000 7f4ced3c1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ce815da90 0x7f4ce8161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.832+0000 7f4ced3c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce80b9750 0x7f4ce80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.832+0000 7f4ced3c1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce8162040 con 0x7f4ce8161ba0 2024-04-25T10:00:33.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.832+0000 7f4ced3c1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce8162600 con 0x7f4ce815da90 2024-04-25T10:00:33.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.832+0000 7f4ced3c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce8162320 con 0x7f4ce80b9750 2024-04-25T10:00:33.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.833+0000 7f4ce67fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce80b9750 0x7f4ce80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.833+0000 7f4ce6ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ce815da90 0x7f4ce8161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.833+0000 7f4ce77fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4ce8161ba0 0x7f4ce806c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:52856/0 (socket says 172.21.15.19:52856) 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.833+0000 7f4ce77fe640 1 -- 172.21.15.19:0/2491878204 learned_addr learned my addr 172.21.15.19:0/2491878204 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.833+0000 7f4ce5ffb640 1 -- 172.21.15.19:0/2491878204 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180533299 0 0) 0x7f4ce8162040 con 0x7f4ce8161ba0 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.833+0000 7f4ce67fc640 1 -- 172.21.15.19:0/2491878204 >> v2:172.21.15.112:3300/0 conn(0x7f4ce815da90 msgr2=0x7f4ce8161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.833+0000 7f4ce67fc640 1 --2- 172.21.15.19:0/2491878204 >> v2:172.21.15.112:3300/0 conn(0x7f4ce815da90 0x7f4ce8161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.833+0000 7f4ce67fc640 1 -- 172.21.15.19:0/2491878204 >> v1:172.21.15.112:6789/0 conn(0x7f4ce8161ba0 legacy=0x7f4ce806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.833+0000 7f4ce67fc640 1 -- 172.21.15.19:0/2491878204 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce80d1b10 con 0x7f4ce80b9750 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.833+0000 7f4ce6ffd640 1 --2- 172.21.15.19:0/2491878204 >> v2:172.21.15.112:3300/0 conn(0x7f4ce815da90 0x7f4ce8161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.834+0000 7f4ce67fc640 1 --2- 172.21.15.19:0/2491878204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce80b9750 0x7f4ce80ce670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0097a0 tx=0x7f4cdc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19347613219677a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.834+0000 7f4ce5ffb640 1 -- 172.21.15.19:0/2491878204 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc037070 con 0x7f4ce80b9750 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.834+0000 7f4ce5ffb640 1 -- 172.21.15.19:0/2491878204 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc02ae50 con 0x7f4ce80b9750 2024-04-25T10:00:33.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.834+0000 7f4ce5ffb640 1 -- 172.21.15.19:0/2491878204 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc032660 con 0x7f4ce80b9750 2024-04-25T10:00:33.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.834+0000 7f4ced3c1640 1 -- 172.21.15.19:0/2491878204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce80b9750 msgr2=0x7f4ce80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.834+0000 7f4ced3c1640 1 --2- 172.21.15.19:0/2491878204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce80b9750 0x7f4ce80ce670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0097a0 tx=0x7f4cdc02a100 comp rx=0 tx=0).stop 2024-04-25T10:00:33.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.835+0000 7f4ced3c1640 1 -- 172.21.15.19:0/2491878204 shutdown_connections 2024-04-25T10:00:33.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.835+0000 7f4ced3c1640 1 --2- 172.21.15.19:0/2491878204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce80b9750 0x7f4ce80ce670 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.835+0000 7f4ced3c1640 1 --2- 172.21.15.19:0/2491878204 >> v2:172.21.15.112:3300/0 conn(0x7f4ce815da90 0x7f4ce8161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.835+0000 7f4ced3c1640 1 -- 172.21.15.19:0/2491878204 >> 172.21.15.19:0/2491878204 conn(0x7f4ce8061a20 msgr2=0x7f4ce8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.835+0000 7f4ced3c1640 1 -- 172.21.15.19:0/2491878204 shutdown_connections 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.835+0000 7f4ced3c1640 1 -- 172.21.15.19:0/2491878204 wait complete. 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.836+0000 7f4ced3c1640 1 Processor -- start 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.836+0000 7f4ced3c1640 1 -- start start 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ced3c1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ce815da90 0x7f4ce8160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ced3c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce8161ba0 0x7f4ce8152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ced3c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce8152590 0x7f4ce81551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ced3c1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce80d3320 con 0x7f4ce815da90 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ced3c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce815de70 con 0x7f4ce8152590 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ced3c1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce80d34a0 con 0x7f4ce8161ba0 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ce6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ce815da90 0x7f4ce8160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ce6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ce815da90 0x7f4ce8160220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44508/0 (socket says 172.21.15.19:44508) 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ce6ffd640 1 -- 172.21.15.19:0/3491220499 learned_addr learned my addr 172.21.15.19:0/3491220499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ce77fe640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce8152590 0x7f4ce81551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ce67fc640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce8161ba0 0x7f4ce8152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ce6ffd640 1 -- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce8161ba0 msgr2=0x7f4ce8152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ce6ffd640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce8161ba0 0x7f4ce8152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ce6ffd640 1 -- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce8152590 msgr2=0x7f4ce81551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ce6ffd640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce8152590 0x7f4ce81551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ce6ffd640 1 -- 172.21.15.19:0/3491220499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce815f1e0 con 0x7f4ce815da90 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ce6ffd640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ce815da90 0x7f4ce8160220 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800ec10 tx=0x7f4cd800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ccf7fe640 1 -- 172.21.15.19:0/3491220499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8017070 con 0x7f4ce815da90 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ccf7fe640 1 -- 172.21.15.19:0/3491220499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd800cc30 con 0x7f4ce815da90 2024-04-25T10:00:33.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ccf7fe640 1 -- 172.21.15.19:0/3491220499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8013470 con 0x7f4ce815da90 2024-04-25T10:00:33.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.837+0000 7f4ced3c1640 1 -- 172.21.15.19:0/3491220499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce815f460 con 0x7f4ce815da90 2024-04-25T10:00:33.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.838+0000 7f4ced3c1640 1 -- 172.21.15.19:0/3491220499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce815fa20 con 0x7f4ce815da90 2024-04-25T10:00:33.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.838+0000 7f4ced3c1640 1 -- 172.21.15.19:0/3491220499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce81529a0 con 0x7f4ce815da90 2024-04-25T10:00:33.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.839+0000 7f4ccf7fe640 1 -- 172.21.15.19:0/3491220499 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cd801a070 con 0x7f4ce815da90 2024-04-25T10:00:33.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.839+0000 7f4ccf7fe640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cd406b4d0 0x7f4cd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.840+0000 7f4ccf7fe640 1 -- 172.21.15.19:0/3491220499 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4cd808a610 con 0x7f4ce815da90 2024-04-25T10:00:33.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.840+0000 7f4ce67fc640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cd406b4d0 0x7f4cd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.840+0000 7f4ce67fc640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cd406b4d0 0x7f4cd406d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc034000 tx=0x7f4cdc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:33.844+0000 7f4ccf7fe640 1 -- 172.21.15.19:0/3491220499 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ce815f1e0 con 0x7f4ce815da90 2024-04-25T10:00:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.061+0000 7f4ced3c1640 1 -- 172.21.15.19:0/3491220499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4ce8073a00 con 0x7f4ce815da90 2024-04-25T10:00:34.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.075+0000 7f4ccf7fe640 1 -- 172.21.15.19:0/3491220499 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v71) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4cd8058cb0 con 0x7f4ce815da90 2024-04-25T10:00:34.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 -- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cd406b4d0 msgr2=0x7f4cd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cd406b4d0 0x7f4cd406d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc034000 tx=0x7f4cdc02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:34.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 -- 172.21.15.19:0/3491220499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ce815da90 msgr2=0x7f4ce8160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ce815da90 0x7f4ce8160220 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800ec10 tx=0x7f4cd800c520 comp rx=0 tx=0).stop 2024-04-25T10:00:34.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 -- 172.21.15.19:0/3491220499 shutdown_connections 2024-04-25T10:00:34.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cd406b4d0 0x7f4cd406d990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce8152590 0x7f4ce81551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ce8161ba0 0x7f4ce8152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 --2- 172.21.15.19:0/3491220499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ce815da90 0x7f4ce8160220 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 -- 172.21.15.19:0/3491220499 >> 172.21.15.19:0/3491220499 conn(0x7f4ce8061a20 msgr2=0x7f4ce80b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:34.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 -- 172.21.15.19:0/3491220499 shutdown_connections 2024-04-25T10:00:34.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.077+0000 7f4ced3c1640 1 -- 172.21.15.19:0/3491220499 wait complete. 2024-04-25T10:00:34.093 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:00:34.093 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:00:34.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.192+0000 7f9ca4c37640 1 Processor -- start 2024-04-25T10:00:34.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.192+0000 7f9ca4c37640 1 -- start start 2024-04-25T10:00:34.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9ca4c37640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9ca015b7d0 0x7f9ca00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9ca4c37640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ca00b5620 0x7f9ca00b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9ca4c37640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9ca006c2d0 0x7f9ca006c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9ca4c37640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca00b5fc0 con 0x7f9ca015b7d0 2024-04-25T10:00:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9ca4c37640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca006cca0 con 0x7f9ca006c2d0 2024-04-25T10:00:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9ca4c37640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca006cf50 con 0x7f9ca00b5620 2024-04-25T10:00:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9c9e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9ca015b7d0 0x7f9ca00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9c9e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9ca015b7d0 0x7f9ca00cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44516/0 (socket says 172.21.15.19:44516) 2024-04-25T10:00:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9c9e59c640 1 -- 172.21.15.19:0/2907776519 learned_addr learned my addr 172.21.15.19:0/2907776519 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9c9e59c640 1 -- 172.21.15.19:0/2907776519 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ca00b5620 msgr2=0x7f9ca00b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9c9e59c640 1 --2- 172.21.15.19:0/2907776519 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ca00b5620 0x7f9ca00b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9c9e59c640 1 -- 172.21.15.19:0/2907776519 >> v2:172.21.15.112:3300/0 conn(0x7f9ca006c2d0 msgr2=0x7f9ca006c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9c9e59c640 1 --2- 172.21.15.19:0/2907776519 >> v2:172.21.15.112:3300/0 conn(0x7f9ca006c2d0 0x7f9ca006c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.193+0000 7f9c9e59c640 1 -- 172.21.15.19:0/2907776519 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca00ce530 con 0x7f9ca015b7d0 2024-04-25T10:00:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.194+0000 7f9c9e59c640 1 --2- 172.21.15.19:0/2907776519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ca015b7d0 0x7f9ca00cf010 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f9ca015e340 tx=0x7f9c9002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2fd88d35a76f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.194+0000 7f9c9dd9b640 1 -- 172.21.15.19:0/2907776519 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90037070 con 0x7f9ca015b7d0 2024-04-25T10:00:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.194+0000 7f9c9dd9b640 1 -- 172.21.15.19:0/2907776519 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c90032440 con 0x7f9ca015b7d0 2024-04-25T10:00:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.194+0000 7f9c9dd9b640 1 -- 172.21.15.19:0/2907776519 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90032810 con 0x7f9ca015b7d0 2024-04-25T10:00:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.194+0000 7f9ca4c37640 1 -- 172.21.15.19:0/2907776519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ca015b7d0 msgr2=0x7f9ca00cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.194+0000 7f9ca4c37640 1 --2- 172.21.15.19:0/2907776519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ca015b7d0 0x7f9ca00cf010 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f9ca015e340 tx=0x7f9c9002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.194+0000 7f9ca4c37640 1 -- 172.21.15.19:0/2907776519 shutdown_connections 2024-04-25T10:00:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.194+0000 7f9ca4c37640 1 --2- 172.21.15.19:0/2907776519 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ca00b5620 0x7f9ca00b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.194+0000 7f9ca4c37640 1 --2- 172.21.15.19:0/2907776519 >> v2:172.21.15.112:3300/0 conn(0x7f9ca006c2d0 0x7f9ca006c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.194+0000 7f9ca4c37640 1 --2- 172.21.15.19:0/2907776519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ca015b7d0 0x7f9ca00cf010 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.194+0000 7f9ca4c37640 1 -- 172.21.15.19:0/2907776519 >> 172.21.15.19:0/2907776519 conn(0x7f9ca00d0800 msgr2=0x7f9ca00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.195+0000 7f9ca4c37640 1 -- 172.21.15.19:0/2907776519 shutdown_connections 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.195+0000 7f9ca4c37640 1 -- 172.21.15.19:0/2907776519 wait complete. 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.196+0000 7f9ca4c37640 1 Processor -- start 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.196+0000 7f9ca4c37640 1 -- start start 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9ca4c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ca006c2d0 0x7f9ca00ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9ca4c37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ca00b5620 0x7f9ca015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9ca4c37640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ca015b7d0 0x7f9ca0150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9ca4c37640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca006d230 con 0x7f9ca00b5620 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9ca4c37640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00ceaf0 con 0x7f9ca006c2d0 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9ca4c37640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00cf550 con 0x7f9ca015b7d0 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9c95d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ca00b5620 0x7f9ca015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9c95d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ca00b5620 0x7f9ca015d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44520/0 (socket says 172.21.15.19:44520) 2024-04-25T10:00:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9c95d9b640 1 -- 172.21.15.19:0/4196384781 learned_addr learned my addr 172.21.15.19:0/4196384781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9c9e59c640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ca006c2d0 0x7f9ca00ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9c9ed9d640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ca015b7d0 0x7f9ca0150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9c95d9b640 1 -- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ca015b7d0 msgr2=0x7f9ca0150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9c95d9b640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ca015b7d0 0x7f9ca0150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9c95d9b640 1 -- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ca006c2d0 msgr2=0x7f9ca00ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9c95d9b640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ca006c2d0 0x7f9ca00ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9c95d9b640 1 -- 172.21.15.19:0/4196384781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca01f7130 con 0x7f9ca00b5620 2024-04-25T10:00:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.197+0000 7f9c95d9b640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ca00b5620 0x7f9ca015d4e0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800b4a0 tx=0x7f9c8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.198+0000 7f9c977fe640 1 -- 172.21.15.19:0/4196384781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88014070 con 0x7f9ca00b5620 2024-04-25T10:00:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.198+0000 7f9c977fe640 1 -- 172.21.15.19:0/4196384781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c88010610 con 0x7f9ca00b5620 2024-04-25T10:00:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.198+0000 7f9ca4c37640 1 -- 172.21.15.19:0/4196384781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca01f7410 con 0x7f9ca00b5620 2024-04-25T10:00:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.198+0000 7f9c977fe640 1 -- 172.21.15.19:0/4196384781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88010a00 con 0x7f9ca00b5620 2024-04-25T10:00:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.198+0000 7f9ca4c37640 1 -- 172.21.15.19:0/4196384781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca01f79d0 con 0x7f9ca00b5620 2024-04-25T10:00:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.199+0000 7f9ca4c37640 1 -- 172.21.15.19:0/4196384781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca0073a00 con 0x7f9ca00b5620 2024-04-25T10:00:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.199+0000 7f9c977fe640 1 -- 172.21.15.19:0/4196384781 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c88017070 con 0x7f9ca00b5620 2024-04-25T10:00:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.200+0000 7f9c977fe640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c8006b4d0 0x7f9c8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.200+0000 7f9c977fe640 1 -- 172.21.15.19:0/4196384781 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c88089c70 con 0x7f9ca00b5620 2024-04-25T10:00:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.200+0000 7f9c9e59c640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c8006b4d0 0x7f9c8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.200+0000 7f9c9e59c640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c8006b4d0 0x7f9c8006d990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9c90034000 tx=0x7f9c9002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.204+0000 7f9c977fe640 1 -- 172.21.15.19:0/4196384781 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c880583a0 con 0x7f9ca00b5620 2024-04-25T10:00:34.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:34.422+0000 7f9ca4c37640 1 -- 172.21.15.19:0/4196384781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9ca015b4b0 con 0x7f9ca00b5620 2024-04-25T10:00:35.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.087+0000 7f9c977fe640 1 -- 172.21.15.19:0/4196384781 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v72) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9c8805c440 con 0x7f9ca00b5620 2024-04-25T10:00:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 -- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c8006b4d0 msgr2=0x7f9c8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c8006b4d0 0x7f9c8006d990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9c90034000 tx=0x7f9c9002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 -- 172.21.15.19:0/4196384781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ca00b5620 msgr2=0x7f9ca015d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ca00b5620 0x7f9ca015d4e0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800b4a0 tx=0x7f9c8800b960 comp rx=0 tx=0).stop 2024-04-25T10:00:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 -- 172.21.15.19:0/4196384781 shutdown_connections 2024-04-25T10:00:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c8006b4d0 0x7f9c8006d990 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ca015b7d0 0x7f9ca0150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ca00b5620 0x7f9ca015d4e0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 --2- 172.21.15.19:0/4196384781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ca006c2d0 0x7f9ca00ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 -- 172.21.15.19:0/4196384781 >> 172.21.15.19:0/4196384781 conn(0x7f9ca00d0800 msgr2=0x7f9ca00d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 -- 172.21.15.19:0/4196384781 shutdown_connections 2024-04-25T10:00:35.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.089+0000 7f9ca4c37640 1 -- 172.21.15.19:0/4196384781 wait complete. 2024-04-25T10:00:35.107 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:00:35.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:35.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.211+0000 7fb3a565d640 1 Processor -- start 2024-04-25T10:00:35.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.211+0000 7fb3a565d640 1 -- start start 2024-04-25T10:00:35.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb3a565d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3a00b2cb0 0x7fb3a00d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb3a565d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb3a00d1020 0x7fb3a00d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb3a565d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a00b97a0 con 0x7fb3a015dbe0 2024-04-25T10:00:35.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb3a565d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a015dfc0 con 0x7fb3a00d1020 2024-04-25T10:00:35.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb3a565d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a00d1a00 con 0x7fb3a00b2cb0 2024-04-25T10:00:35.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb39f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb3a00d1020 0x7fb3a00d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb39effd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb3a015dbe0 0x7fb3a00d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:52872/0 (socket says 172.21.15.19:52872) 2024-04-25T10:00:35.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb39e7fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3a00b2cb0 0x7fb3a00d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb39effd640 1 -- 172.21.15.19:0/764657631 learned_addr learned my addr 172.21.15.19:0/764657631 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb39e7fc640 1 -- 172.21.15.19:0/764657631 >> v2:172.21.15.112:3300/0 conn(0x7fb3a00d1020 msgr2=0x7fb3a00d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb39e7fc640 1 --2- 172.21.15.19:0/764657631 >> v2:172.21.15.112:3300/0 conn(0x7fb3a00d1020 0x7fb3a00d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb39e7fc640 1 -- 172.21.15.19:0/764657631 >> v1:172.21.15.112:6789/0 conn(0x7fb3a015dbe0 legacy=0x7fb3a00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb39e7fc640 1 -- 172.21.15.19:0/764657631 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a00d1c50 con 0x7fb3a00b2cb0 2024-04-25T10:00:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.212+0000 7fb39f7fe640 1 --2- 172.21.15.19:0/764657631 >> v2:172.21.15.112:3300/0 conn(0x7fb3a00d1020 0x7fb3a00d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.213+0000 7fb39e7fc640 1 --2- 172.21.15.19:0/764657631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3a00b2cb0 0x7fb3a00d2730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb3940097a0 tx=0x7fb39402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fa7d983aad3deb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.213+0000 7fb39dffb640 1 -- 172.21.15.19:0/764657631 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394037070 con 0x7fb3a00b2cb0 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.213+0000 7fb39dffb640 1 -- 172.21.15.19:0/764657631 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb394032440 con 0x7fb3a00b2cb0 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.213+0000 7fb39dffb640 1 -- 172.21.15.19:0/764657631 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394032810 con 0x7fb3a00b2cb0 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.214+0000 7fb3a565d640 1 -- 172.21.15.19:0/764657631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3a00b2cb0 msgr2=0x7fb3a00d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.214+0000 7fb3a565d640 1 --2- 172.21.15.19:0/764657631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3a00b2cb0 0x7fb3a00d2730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb3940097a0 tx=0x7fb39402a180 comp rx=0 tx=0).stop 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.214+0000 7fb3a565d640 1 -- 172.21.15.19:0/764657631 shutdown_connections 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.214+0000 7fb3a565d640 1 --2- 172.21.15.19:0/764657631 >> v2:172.21.15.112:3300/0 conn(0x7fb3a00d1020 0x7fb3a00d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.214+0000 7fb3a565d640 1 --2- 172.21.15.19:0/764657631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3a00b2cb0 0x7fb3a00d2730 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.214+0000 7fb3a565d640 1 -- 172.21.15.19:0/764657631 >> 172.21.15.19:0/764657631 conn(0x7fb3a0061a20 msgr2=0x7fb3a0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.214+0000 7fb3a565d640 1 -- 172.21.15.19:0/764657631 shutdown_connections 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.214+0000 7fb3a565d640 1 -- 172.21.15.19:0/764657631 wait complete. 2024-04-25T10:00:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.215+0000 7fb3a565d640 1 Processor -- start 2024-04-25T10:00:35.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.215+0000 7fb3a565d640 1 -- start start 2024-04-25T10:00:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.216+0000 7fb3a565d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3a00b2cb0 0x7fb3a00caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.216+0000 7fb3a565d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a00d1020 0x7fb3a01f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.216+0000 7fb3a565d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3a015dbe0 0x7fb3a015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.216+0000 7fb3a565d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a00b92a0 con 0x7fb3a00d1020 2024-04-25T10:00:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.216+0000 7fb3a565d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a00d2210 con 0x7fb3a015dbe0 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.216+0000 7fb3a565d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a00b9420 con 0x7fb3a00b2cb0 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.216+0000 7fb39e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a00d1020 0x7fb3a01f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.216+0000 7fb39e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a00d1020 0x7fb3a01f78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44522/0 (socket says 172.21.15.19:44522) 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.216+0000 7fb39e7fc640 1 -- 172.21.15.19:0/1756161972 learned_addr learned my addr 172.21.15.19:0/1756161972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.216+0000 7fb39effd640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3a00b2cb0 0x7fb3a00caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.216+0000 7fb39f7fe640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3a015dbe0 0x7fb3a015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb39e7fc640 1 -- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3a00b2cb0 msgr2=0x7fb3a00caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb39e7fc640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3a00b2cb0 0x7fb3a00caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb39e7fc640 1 -- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3a015dbe0 msgr2=0x7fb3a015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb39e7fc640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3a015dbe0 0x7fb3a015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb39e7fc640 1 -- 172.21.15.19:0/1756161972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a00cb0f0 con 0x7fb3a00d1020 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb39f7fe640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3a015dbe0 0x7fb3a015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb39e7fc640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a00d1020 0x7fb3a01f78d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb394035040 tx=0x7fb3940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb3877fe640 1 -- 172.21.15.19:0/1756161972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394041020 con 0x7fb3a00d1020 2024-04-25T10:00:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb3877fe640 1 -- 172.21.15.19:0/1756161972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb394030700 con 0x7fb3a00d1020 2024-04-25T10:00:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb3a565d640 1 -- 172.21.15.19:0/1756161972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a00cb3d0 con 0x7fb3a00d1020 2024-04-25T10:00:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb3877fe640 1 -- 172.21.15.19:0/1756161972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394037040 con 0x7fb3a00d1020 2024-04-25T10:00:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.217+0000 7fb3a565d640 1 -- 172.21.15.19:0/1756161972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a00cb990 con 0x7fb3a00d1020 2024-04-25T10:00:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.218+0000 7fb3a565d640 1 -- 172.21.15.19:0/1756161972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a0073990 con 0x7fb3a00d1020 2024-04-25T10:00:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.219+0000 7fb3877fe640 1 -- 172.21.15.19:0/1756161972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb394044020 con 0x7fb3a00d1020 2024-04-25T10:00:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.219+0000 7fb3877fe640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb38c06b4d0 0x7fb38c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.219+0000 7fb3877fe640 1 -- 172.21.15.19:0/1756161972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3940aab50 con 0x7fb3a00d1020 2024-04-25T10:00:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.219+0000 7fb39effd640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb38c06b4d0 0x7fb38c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.220+0000 7fb39effd640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb38c06b4d0 0x7fb38c06d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb390007fd0 tx=0x7fb390007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.223+0000 7fb3877fe640 1 -- 172.21.15.19:0/1756161972 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb39404b030 con 0x7fb3a00d1020 2024-04-25T10:00:35.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.441+0000 7fb3a565d640 1 -- 172.21.15.19:0/1756161972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3a00cef60 con 0x7fb3a00d1020 2024-04-25T10:00:35.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.442+0000 7fb3877fe640 1 -- 172.21.15.19:0/1756161972 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb394079850 con 0x7fb3a00d1020 2024-04-25T10:00:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:35.446 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:00:35.446 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:00:35.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.443+0000 7fb3a565d640 1 -- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb38c06b4d0 msgr2=0x7fb38c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.443+0000 7fb3a565d640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb38c06b4d0 0x7fb38c06d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb390007fd0 tx=0x7fb390007e80 comp rx=0 tx=0).stop 2024-04-25T10:00:35.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.443+0000 7fb3a565d640 1 -- 172.21.15.19:0/1756161972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a00d1020 msgr2=0x7fb3a01f78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.443+0000 7fb3a565d640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a00d1020 0x7fb3a01f78d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb394035040 tx=0x7fb3940046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.443+0000 7fb3a565d640 1 -- 172.21.15.19:0/1756161972 shutdown_connections 2024-04-25T10:00:35.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.444+0000 7fb3a565d640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb38c06b4d0 0x7fb38c06d990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.444+0000 7fb3a565d640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3a015dbe0 0x7fb3a015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.444+0000 7fb3a565d640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a00d1020 0x7fb3a01f78d0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.444+0000 7fb3a565d640 1 --2- 172.21.15.19:0/1756161972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3a00b2cb0 0x7fb3a00caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.444+0000 7fb3a565d640 1 -- 172.21.15.19:0/1756161972 >> 172.21.15.19:0/1756161972 conn(0x7fb3a0061a20 msgr2=0x7fb3a00d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.444+0000 7fb3a565d640 1 -- 172.21.15.19:0/1756161972 shutdown_connections 2024-04-25T10:00:35.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.444+0000 7fb3a565d640 1 -- 172.21.15.19:0/1756161972 wait complete. 2024-04-25T10:00:35.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.556+0000 7f1a81276640 1 Processor -- start 2024-04-25T10:00:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.556+0000 7f1a81276640 1 -- start start 2024-04-25T10:00:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a81276640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1a7c0b5c30 0x7f1a7c168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a81276640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7c0ceee0 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a81276640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7c0d1d10 con 0x7f1a7c15db70 2024-04-25T10:00:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a81276640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a7c0d3800 con 0x7f1a7c0b5c30 2024-04-25T10:00:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a7a7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1a7c0b6cb0 0x7f1a7c0b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47076/0 (socket says 172.21.15.19:47076) 2024-04-25T10:00:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a7a7fc640 1 -- 172.21.15.19:0/1533461145 learned_addr learned my addr 172.21.15.19:0/1533461145 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:35.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a7b7fe640 1 --2- 172.21.15.19:0/1533461145 >> v2:172.21.15.19:3300/0 conn(0x7f1a7c0b5c30 0x7f1a7c168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a79ffb640 1 -- 172.21.15.19:0/1533461145 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430328255 0 0) 0x7f1a7c0ceee0 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a79ffb640 1 -- 172.21.15.19:0/1533461145 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a64003520 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a7b7fe640 1 -- 172.21.15.19:0/1533461145 >> v1:172.21.15.112:6789/0 conn(0x7f1a7c15db70 legacy=0x7f1a7c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a7b7fe640 1 -- 172.21.15.19:0/1533461145 >> v1:172.21.15.19:6789/0 conn(0x7f1a7c0b6cb0 legacy=0x7f1a7c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.557+0000 7f1a7b7fe640 1 -- 172.21.15.19:0/1533461145 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c0d1ff0 con 0x7f1a7c0b5c30 2024-04-25T10:00:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.558+0000 7f1a7b7fe640 1 --2- 172.21.15.19:0/1533461145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7c0b5c30 0x7f1a7c168ad0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0d3980 tx=0x7f1a7402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafd6a1c1a90c345 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.558+0000 7f1a79ffb640 1 -- 172.21.15.19:0/1533461145 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74007740 con 0x7f1a7c0b5c30 2024-04-25T10:00:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.558+0000 7f1a79ffb640 1 -- 172.21.15.19:0/1533461145 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a740078e0 con 0x7f1a7c0b5c30 2024-04-25T10:00:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.558+0000 7f1a79ffb640 1 -- 172.21.15.19:0/1533461145 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74007cb0 con 0x7f1a7c0b5c30 2024-04-25T10:00:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.559+0000 7f1a81276640 1 -- 172.21.15.19:0/1533461145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7c0b5c30 msgr2=0x7f1a7c168ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.559+0000 7f1a81276640 1 --2- 172.21.15.19:0/1533461145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7c0b5c30 0x7f1a7c168ad0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0d3980 tx=0x7f1a7402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.559+0000 7f1a81276640 1 -- 172.21.15.19:0/1533461145 shutdown_connections 2024-04-25T10:00:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.559+0000 7f1a81276640 1 --2- 172.21.15.19:0/1533461145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7c0b5c30 0x7f1a7c168ad0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.559+0000 7f1a81276640 1 -- 172.21.15.19:0/1533461145 >> 172.21.15.19:0/1533461145 conn(0x7f1a7c0d0520 msgr2=0x7f1a7c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.559+0000 7f1a81276640 1 -- 172.21.15.19:0/1533461145 shutdown_connections 2024-04-25T10:00:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.559+0000 7f1a81276640 1 -- 172.21.15.19:0/1533461145 wait complete. 2024-04-25T10:00:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.560+0000 7f1a81276640 1 Processor -- start 2024-04-25T10:00:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.560+0000 7f1a81276640 1 -- start start 2024-04-25T10:00:35.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a81276640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a7c0b5c30 0x7f1a7c0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a81276640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7c0b6cb0 0x7f1a7c159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a81276640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a7c15db70 0x7f1a7c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a81276640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c0ce530 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a81276640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c15df50 con 0x7f1a7c0b5c30 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a81276640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c0cf1c0 con 0x7f1a7c15db70 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a7a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7c0b6cb0 0x7f1a7c159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a7a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7c0b6cb0 0x7f1a7c159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44538/0 (socket says 172.21.15.19:44538) 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a7a7fc640 1 -- 172.21.15.19:0/2163928291 learned_addr learned my addr 172.21.15.19:0/2163928291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a7a7fc640 1 -- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a7c15db70 msgr2=0x7f1a7c15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a7affd640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a7c0b5c30 0x7f1a7c0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a7a7fc640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a7c15db70 0x7f1a7c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a7a7fc640 1 -- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a7c0b5c30 msgr2=0x7f1a7c0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a7a7fc640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a7c0b5c30 0x7f1a7c0c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a7a7fc640 1 -- 172.21.15.19:0/2163928291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c15ede0 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a7a7fc640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7c0b6cb0 0x7f1a7c159fc0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c400 tx=0x7f1a7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a637fe640 1 -- 172.21.15.19:0/2163928291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70017070 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a637fe640 1 -- 172.21.15.19:0/2163928291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a70005d00 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a637fe640 1 -- 172.21.15.19:0/2163928291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70007310 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a81276640 1 -- 172.21.15.19:0/2163928291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7c15f0c0 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.561+0000 7f1a81276640 1 -- 172.21.15.19:0/2163928291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7c15f680 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.563+0000 7f1a81276640 1 -- 172.21.15.19:0/2163928291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7c073990 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.563+0000 7f1a637fe640 1 -- 172.21.15.19:0/2163928291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a7001a070 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.564+0000 7f1a637fe640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a6806b4d0 0x7f1a6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.564+0000 7f1a637fe640 1 -- 172.21.15.19:0/2163928291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a70089820 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.564+0000 7f1a7affd640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a6806b4d0 0x7f1a6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.564+0000 7f1a7affd640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a6806b4d0 0x7f1a6806d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00a750 tx=0x7f1a6c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.568+0000 7f1a637fe640 1 -- 172.21.15.19:0/2163928291 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a70058510 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.748+0000 7f1a81276640 1 -- 172.21.15.19:0/2163928291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a7c06cb60 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.749+0000 7f1a637fe640 1 -- 172.21.15.19:0/2163928291 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f1a70057e10 con 0x7f1a7c0b6cb0 2024-04-25T10:00:35.754 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:35.754 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":117,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:00:34.061124+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:30.702453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:31.396192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 -- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a6806b4d0 msgr2=0x7f1a6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a6806b4d0 0x7f1a6806d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00a750 tx=0x7f1a6c008040 comp rx=0 tx=0).stop 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 -- 172.21.15.19:0/2163928291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7c0b6cb0 msgr2=0x7f1a7c159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7c0b6cb0 0x7f1a7c159fc0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c400 tx=0x7f1a7000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 -- 172.21.15.19:0/2163928291 shutdown_connections 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a6806b4d0 0x7f1a6806d990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a7c15db70 0x7f1a7c15a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a7c0b6cb0 0x7f1a7c159fc0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 --2- 172.21.15.19:0/2163928291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a7c0b5c30 0x7f1a7c0c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 -- 172.21.15.19:0/2163928291 >> 172.21.15.19:0/2163928291 conn(0x7f1a7c0d0520 msgr2=0x7f1a7c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 -- 172.21.15.19:0/2163928291 shutdown_connections 2024-04-25T10:00:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.751+0000 7f1a81276640 1 -- 172.21.15.19:0/2163928291 wait complete. 2024-04-25T10:00:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:00:35.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:00:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.864+0000 7ff20f0ee640 1 Processor -- start 2024-04-25T10:00:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.864+0000 7ff20f0ee640 1 -- start start 2024-04-25T10:00:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.864+0000 7ff20f0ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2080b6cb0 0x7ff2080b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20f0ee640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2080b5a60 0x7ff2080d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20f0ee640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2080ceee0 con 0x7ff20815db70 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20f0ee640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2080d1d10 con 0x7ff2080b5a60 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20f0ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080d1ff0 con 0x7ff2080b6cb0 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20d68b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2080b5a60 0x7ff2080d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20ce8a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff20815db70 0x7ff208162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47080/0 (socket says 172.21.15.19:47080) 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20ce8a640 1 -- 172.21.15.19:0/1163229867 learned_addr learned my addr 172.21.15.19:0/1163229867 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20d68b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2080b5a60 0x7ff2080d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44544/0 (socket says 172.21.15.19:44544) 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff207fff640 1 --2- 172.21.15.19:0/1163229867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2080b6cb0 0x7ff2080b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20d68b640 1 -- 172.21.15.19:0/1163229867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2080b6cb0 msgr2=0x7ff2080b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20d68b640 1 --2- 172.21.15.19:0/1163229867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2080b6cb0 0x7ff2080b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20d68b640 1 -- 172.21.15.19:0/1163229867 >> v1:172.21.15.19:6789/0 conn(0x7ff20815db70 legacy=0x7ff208162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff20d68b640 1 -- 172.21.15.19:0/1163229867 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080d22d0 con 0x7ff2080b5a60 2024-04-25T10:00:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.865+0000 7ff2077fe640 1 -- 172.21.15.19:0/1163229867 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429301049 0 0) 0x7ff2080ceee0 con 0x7ff20815db70 2024-04-25T10:00:35.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff20d68b640 1 --2- 172.21.15.19:0/1163229867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2080b5a60 0x7ff2080d27f0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80097a0 tx=0x7ff1f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeff10771b2cf8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff2077fe640 1 -- 172.21.15.19:0/1163229867 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8037070 con 0x7ff2080b5a60 2024-04-25T10:00:35.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff2077fe640 1 -- 172.21.15.19:0/1163229867 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f8032440 con 0x7ff2080b5a60 2024-04-25T10:00:35.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff2077fe640 1 -- 172.21.15.19:0/1163229867 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8032810 con 0x7ff2080b5a60 2024-04-25T10:00:35.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff20f0ee640 1 -- 172.21.15.19:0/1163229867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2080b5a60 msgr2=0x7ff2080d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff20f0ee640 1 --2- 172.21.15.19:0/1163229867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2080b5a60 0x7ff2080d27f0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80097a0 tx=0x7ff1f802a180 comp rx=0 tx=0).stop 2024-04-25T10:00:35.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff20f0ee640 1 -- 172.21.15.19:0/1163229867 shutdown_connections 2024-04-25T10:00:35.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff20f0ee640 1 --2- 172.21.15.19:0/1163229867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2080b5a60 0x7ff2080d27f0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff20f0ee640 1 --2- 172.21.15.19:0/1163229867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2080b6cb0 0x7ff2080b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff20f0ee640 1 -- 172.21.15.19:0/1163229867 >> 172.21.15.19:0/1163229867 conn(0x7ff2080d0520 msgr2=0x7ff2080d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff20f0ee640 1 -- 172.21.15.19:0/1163229867 shutdown_connections 2024-04-25T10:00:35.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.866+0000 7ff20f0ee640 1 -- 172.21.15.19:0/1163229867 wait complete. 2024-04-25T10:00:35.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.868+0000 7ff20f0ee640 1 Processor -- start 2024-04-25T10:00:35.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.868+0000 7ff20f0ee640 1 -- start start 2024-04-25T10:00:35.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.868+0000 7ff20f0ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2080b5a60 0x7ff208149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff20f0ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2080b6cb0 0x7ff2080c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff20f0ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff20815db70 0x7ff2080cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff20f0ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080cf1c0 con 0x7ff2080b6cb0 2024-04-25T10:00:35.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff20f0ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080ce530 con 0x7ff20815db70 2024-04-25T10:00:35.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff20f0ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080cf340 con 0x7ff2080b5a60 2024-04-25T10:00:35.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff207fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2080b6cb0 0x7ff2080c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff207fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2080b6cb0 0x7ff2080c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44546/0 (socket says 172.21.15.19:44546) 2024-04-25T10:00:35.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff207fff640 1 -- 172.21.15.19:0/326811350 learned_addr learned my addr 172.21.15.19:0/326811350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:35.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff20d68b640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff20815db70 0x7ff2080cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff20ce8a640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2080b5a60 0x7ff208149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff207fff640 1 -- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2080b5a60 msgr2=0x7ff208149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff207fff640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2080b5a60 0x7ff208149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff207fff640 1 -- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff20815db70 msgr2=0x7ff2080cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff207fff640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff20815db70 0x7ff2080cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff207fff640 1 -- 172.21.15.19:0/326811350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080cbc70 con 0x7ff2080b6cb0 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff20d68b640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff20815db70 0x7ff2080cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff20ce8a640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2080b5a60 0x7ff208149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff207fff640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2080b6cb0 0x7ff2080c6f40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00d800 tx=0x7ff1fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff2057fa640 1 -- 172.21.15.19:0/326811350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc00b6c0 con 0x7ff2080b6cb0 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff2057fa640 1 -- 172.21.15.19:0/326811350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc019070 con 0x7ff2080b6cb0 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff20f0ee640 1 -- 172.21.15.19:0/326811350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2080cbf50 con 0x7ff2080b6cb0 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff2057fa640 1 -- 172.21.15.19:0/326811350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc00ba80 con 0x7ff2080b6cb0 2024-04-25T10:00:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.869+0000 7ff20f0ee640 1 -- 172.21.15.19:0/326811350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2080cc510 con 0x7ff2080b6cb0 2024-04-25T10:00:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.870+0000 7ff20f0ee640 1 -- 172.21.15.19:0/326811350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff208068410 con 0x7ff2080b6cb0 2024-04-25T10:00:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.871+0000 7ff2057fa640 1 -- 172.21.15.19:0/326811350 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1fc00bc20 con 0x7ff2080b6cb0 2024-04-25T10:00:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.871+0000 7ff2057fa640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff1f406b4d0 0x7ff1f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.872+0000 7ff2057fa640 1 -- 172.21.15.19:0/326811350 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1fc08a7c0 con 0x7ff2080b6cb0 2024-04-25T10:00:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.872+0000 7ff20ce8a640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff1f406b4d0 0x7ff1f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.872+0000 7ff20ce8a640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff1f406b4d0 0x7ff1f406d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff200007fd0 tx=0x7ff200009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:35.876+0000 7ff2057fa640 1 -- 172.21.15.19:0/326811350 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff208068410 con 0x7ff2080b6cb0 2024-04-25T10:00:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.051+0000 7ff20f0ee640 1 -- 172.21.15.19:0/326811350 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2080d1820 con 0x7ff1f406b4d0 2024-04-25T10:00:36.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.057+0000 7ff2057fa640 1 -- 172.21.15.19:0/326811350 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252416 (secure 0 0 0) 0x7ff2080d1820 con 0x7ff1f406b4d0 2024-04-25T10:00:36.064 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:36.066 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:00:36.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 -- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff1f406b4d0 msgr2=0x7ff1f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff1f406b4d0 0x7ff1f406d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff200007fd0 tx=0x7ff200009040 comp rx=0 tx=0).stop 2024-04-25T10:00:36.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 -- 172.21.15.19:0/326811350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2080b6cb0 msgr2=0x7ff2080c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2080b6cb0 0x7ff2080c6f40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00d800 tx=0x7ff1fc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 -- 172.21.15.19:0/326811350 shutdown_connections 2024-04-25T10:00:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff1f406b4d0 0x7ff1f406d990 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff20815db70 0x7ff2080cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2080b6cb0 0x7ff2080c6f40 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 --2- 172.21.15.19:0/326811350 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2080b5a60 0x7ff208149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 -- 172.21.15.19:0/326811350 >> 172.21.15.19:0/326811350 conn(0x7ff2080d0520 msgr2=0x7ff2080b90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 -- 172.21.15.19:0/326811350 shutdown_connections 2024-04-25T10:00:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.060+0000 7ff20f0ee640 1 -- 172.21.15.19:0/326811350 wait complete. 2024-04-25T10:00:36.076 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-04-25T10:00:35.013611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":386900,"kb_used_data":12504,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754587820,"statfs":{"total":773094113280,"available":772697927680,"internally_reserved":0,"allocated":12804096,"data_stored":6571590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":383241778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000825"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.088321+0000","last_change":"2024-04-25T10:00:33.086979+0000","last_active":"2024-04-25T10:00:33.088321+0000","last_peered":"2024-04-25T10:00:33.088321+0000","last_clean":"2024-04-25T10:00:33.088321+0000","last_became_active":"2024-04-25T10:00:33.086684+0000","last_became_peered":"2024-04-25T10:00:33.086684+0000","last_unstale":"2024-04-25T10:00:33.088321+0000","last_undegraded":"2024-04-25T10:00:33.088321+0000","last_fullsized":"2024-04-25T10:00:33.088321+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:11.688215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085200+0000","last_change":"2024-04-25T10:00:33.076867+0000","last_active":"2024-04-25T10:00:33.085200+0000","last_peered":"2024-04-25T10:00:33.085200+0000","last_clean":"2024-04-25T10:00:33.085200+0000","last_became_active":"2024-04-25T10:00:33.076544+0000","last_became_peered":"2024-04-25T10:00:33.076544+0000","last_unstale":"2024-04-25T10:00:33.085200+0000","last_undegraded":"2024-04-25T10:00:33.085200+0000","last_fullsized":"2024-04-25T10:00:33.085200+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:19.378845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.088905+0000","last_change":"2024-04-25T10:00:33.083169+0000","last_active":"2024-04-25T10:00:33.088905+0000","last_peered":"2024-04-25T10:00:33.088905+0000","last_clean":"2024-04-25T10:00:33.088905+0000","last_became_active":"2024-04-25T10:00:33.082362+0000","last_became_peered":"2024-04-25T10:00:33.082362+0000","last_unstale":"2024-04-25T10:00:33.088905+0000","last_undegraded":"2024-04-25T10:00:33.088905+0000","last_fullsized":"2024-04-25T10:00:33.088905+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:58.111467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.075411+0000","last_change":"2024-04-25T10:00:33.072589+0000","last_active":"2024-04-25T10:00:33.075411+0000","last_peered":"2024-04-25T10:00:33.075411+0000","last_clean":"2024-04-25T10:00:33.075411+0000","last_became_active":"2024-04-25T10:00:33.072127+0000","last_became_peered":"2024-04-25T10:00:33.072127+0000","last_unstale":"2024-04-25T10:00:33.075411+0000","last_undegraded":"2024-04-25T10:00:33.075411+0000","last_fullsized":"2024-04-25T10:00:33.075411+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:19.378845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086698+0000","last_change":"2024-04-25T10:00:33.083346+0000","last_active":"2024-04-25T10:00:33.086698+0000","last_peered":"2024-04-25T10:00:33.086698+0000","last_clean":"2024-04-25T10:00:33.086698+0000","last_became_active":"2024-04-25T10:00:33.083025+0000","last_became_peered":"2024-04-25T10:00:33.083025+0000","last_unstale":"2024-04-25T10:00:33.086698+0000","last_undegraded":"2024-04-25T10:00:33.086698+0000","last_fullsized":"2024-04-25T10:00:33.086698+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:06.315019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092638+0000","last_change":"2024-04-25T10:00:33.082084+0000","last_active":"2024-04-25T10:00:33.092638+0000","last_peered":"2024-04-25T10:00:33.092638+0000","last_clean":"2024-04-25T10:00:33.092638+0000","last_became_active":"2024-04-25T10:00:33.079288+0000","last_became_peered":"2024-04-25T10:00:33.079288+0000","last_unstale":"2024-04-25T10:00:33.092638+0000","last_undegraded":"2024-04-25T10:00:33.092638+0000","last_fullsized":"2024-04-25T10:00:33.092638+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:19.221838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092701+0000","last_change":"2024-04-25T10:00:33.088018+0000","last_active":"2024-04-25T10:00:33.092701+0000","last_peered":"2024-04-25T10:00:33.092701+0000","last_clean":"2024-04-25T10:00:33.092701+0000","last_became_active":"2024-04-25T10:00:33.082713+0000","last_became_peered":"2024-04-25T10:00:33.082713+0000","last_unstale":"2024-04-25T10:00:33.092701+0000","last_undegraded":"2024-04-25T10:00:33.092701+0000","last_fullsized":"2024-04-25T10:00:33.092701+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:09.650080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.091126+0000","last_change":"2024-04-25T10:00:33.082597+0000","last_active":"2024-04-25T10:00:33.091126+0000","last_peered":"2024-04-25T10:00:33.091126+0000","last_clean":"2024-04-25T10:00:33.091126+0000","last_became_active":"2024-04-25T10:00:33.082075+0000","last_became_peered":"2024-04-25T10:00:33.082075+0000","last_unstale":"2024-04-25T10:00:33.091126+0000","last_undegraded":"2024-04-25T10:00:33.091126+0000","last_fullsized":"2024-04-25T10:00:33.091126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:05.579160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085128+0000","last_change":"2024-04-25T10:00:33.078739+0000","last_active":"2024-04-25T10:00:33.085128+0000","last_peered":"2024-04-25T10:00:33.085128+0000","last_clean":"2024-04-25T10:00:33.085128+0000","last_became_active":"2024-04-25T10:00:33.078474+0000","last_became_peered":"2024-04-25T10:00:33.078474+0000","last_unstale":"2024-04-25T10:00:33.085128+0000","last_undegraded":"2024-04-25T10:00:33.085128+0000","last_fullsized":"2024-04-25T10:00:33.085128+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:05.579160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.095942+0000","last_change":"2024-04-25T10:00:33.094855+0000","last_active":"2024-04-25T10:00:33.095942+0000","last_peered":"2024-04-25T10:00:33.095942+0000","last_clean":"2024-04-25T10:00:33.095942+0000","last_became_active":"2024-04-25T10:00:33.094571+0000","last_became_peered":"2024-04-25T10:00:33.094571+0000","last_unstale":"2024-04-25T10:00:33.095942+0000","last_undegraded":"2024-04-25T10:00:33.095942+0000","last_fullsized":"2024-04-25T10:00:33.095942+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:30.045573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086659+0000","last_change":"2024-04-25T10:00:33.084876+0000","last_active":"2024-04-25T10:00:33.086659+0000","last_peered":"2024-04-25T10:00:33.086659+0000","last_clean":"2024-04-25T10:00:33.086659+0000","last_became_active":"2024-04-25T10:00:33.083238+0000","last_became_peered":"2024-04-25T10:00:33.083238+0000","last_unstale":"2024-04-25T10:00:33.086659+0000","last_undegraded":"2024-04-25T10:00:33.086659+0000","last_fullsized":"2024-04-25T10:00:33.086659+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:41.510869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092880+0000","last_change":"2024-04-25T10:00:33.091045+0000","last_active":"2024-04-25T10:00:33.092880+0000","last_peered":"2024-04-25T10:00:33.092880+0000","last_clean":"2024-04-25T10:00:33.092880+0000","last_became_active":"2024-04-25T10:00:33.090664+0000","last_became_peered":"2024-04-25T10:00:33.090664+0000","last_unstale":"2024-04-25T10:00:33.092880+0000","last_undegraded":"2024-04-25T10:00:33.092880+0000","last_fullsized":"2024-04-25T10:00:33.092880+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:31.173540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093393+0000","last_change":"2024-04-25T10:00:33.090502+0000","last_active":"2024-04-25T10:00:33.093393+0000","last_peered":"2024-04-25T10:00:33.093393+0000","last_clean":"2024-04-25T10:00:33.093393+0000","last_became_active":"2024-04-25T10:00:33.090206+0000","last_became_peered":"2024-04-25T10:00:33.090206+0000","last_unstale":"2024-04-25T10:00:33.093393+0000","last_undegraded":"2024-04-25T10:00:33.093393+0000","last_fullsized":"2024-04-25T10:00:33.093393+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:43.973298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086328+0000","last_change":"2024-04-25T10:00:33.080941+0000","last_active":"2024-04-25T10:00:33.086328+0000","last_peered":"2024-04-25T10:00:33.086328+0000","last_clean":"2024-04-25T10:00:33.086328+0000","last_became_active":"2024-04-25T10:00:33.080503+0000","last_became_peered":"2024-04-25T10:00:33.080503+0000","last_unstale":"2024-04-25T10:00:33.086328+0000","last_undegraded":"2024-04-25T10:00:33.086328+0000","last_fullsized":"2024-04-25T10:00:33.086328+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:15.540434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086729+0000","last_change":"2024-04-25T10:00:33.082453+0000","last_active":"2024-04-25T10:00:33.086729+0000","last_peered":"2024-04-25T10:00:33.086729+0000","last_clean":"2024-04-25T10:00:33.086729+0000","last_became_active":"2024-04-25T10:00:33.082186+0000","last_became_peered":"2024-04-25T10:00:33.082186+0000","last_unstale":"2024-04-25T10:00:33.086729+0000","last_undegraded":"2024-04-25T10:00:33.086729+0000","last_fullsized":"2024-04-25T10:00:33.086729+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:46.888583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092887+0000","last_change":"2024-04-25T10:00:33.078441+0000","last_active":"2024-04-25T10:00:33.092887+0000","last_peered":"2024-04-25T10:00:33.092887+0000","last_clean":"2024-04-25T10:00:33.092887+0000","last_became_active":"2024-04-25T10:00:33.078129+0000","last_became_peered":"2024-04-25T10:00:33.078129+0000","last_unstale":"2024-04-25T10:00:33.092887+0000","last_undegraded":"2024-04-25T10:00:33.092887+0000","last_fullsized":"2024-04-25T10:00:33.092887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:42.968050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.091751+0000","last_change":"2024-04-25T10:00:33.087706+0000","last_active":"2024-04-25T10:00:33.091751+0000","last_peered":"2024-04-25T10:00:33.091751+0000","last_clean":"2024-04-25T10:00:33.091751+0000","last_became_active":"2024-04-25T10:00:33.087409+0000","last_became_peered":"2024-04-25T10:00:33.087409+0000","last_unstale":"2024-04-25T10:00:33.091751+0000","last_undegraded":"2024-04-25T10:00:33.091751+0000","last_fullsized":"2024-04-25T10:00:33.091751+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:11.688215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085178+0000","last_change":"2024-04-25T10:00:33.076667+0000","last_active":"2024-04-25T10:00:33.085178+0000","last_peered":"2024-04-25T10:00:33.085178+0000","last_clean":"2024-04-25T10:00:33.085178+0000","last_became_active":"2024-04-25T10:00:33.076382+0000","last_became_peered":"2024-04-25T10:00:33.076382+0000","last_unstale":"2024-04-25T10:00:33.085178+0000","last_undegraded":"2024-04-25T10:00:33.085178+0000","last_fullsized":"2024-04-25T10:00:33.085178+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:23.032578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086029+0000","last_change":"2024-04-25T10:00:33.081953+0000","last_active":"2024-04-25T10:00:33.086029+0000","last_peered":"2024-04-25T10:00:33.086029+0000","last_clean":"2024-04-25T10:00:33.086029+0000","last_became_active":"2024-04-25T10:00:33.081683+0000","last_became_peered":"2024-04-25T10:00:33.081683+0000","last_unstale":"2024-04-25T10:00:33.086029+0000","last_undegraded":"2024-04-25T10:00:33.086029+0000","last_fullsized":"2024-04-25T10:00:33.086029+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:38.220436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.077998+0000","last_change":"2024-04-25T10:00:33.075098+0000","last_active":"2024-04-25T10:00:33.077998+0000","last_peered":"2024-04-25T10:00:33.077998+0000","last_clean":"2024-04-25T10:00:33.077998+0000","last_became_active":"2024-04-25T10:00:33.074811+0000","last_became_peered":"2024-04-25T10:00:33.074811+0000","last_unstale":"2024-04-25T10:00:33.077998+0000","last_undegraded":"2024-04-25T10:00:33.077998+0000","last_fullsized":"2024-04-25T10:00:33.077998+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:49.118519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085160+0000","last_change":"2024-04-25T10:00:33.077341+0000","last_active":"2024-04-25T10:00:33.085160+0000","last_peered":"2024-04-25T10:00:33.085160+0000","last_clean":"2024-04-25T10:00:33.085160+0000","last_became_active":"2024-04-25T10:00:33.077069+0000","last_became_peered":"2024-04-25T10:00:33.077069+0000","last_unstale":"2024-04-25T10:00:33.085160+0000","last_undegraded":"2024-04-25T10:00:33.085160+0000","last_fullsized":"2024-04-25T10:00:33.085160+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.768887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.079752+0000","last_change":"2024-04-25T10:00:33.075175+0000","last_active":"2024-04-25T10:00:33.079752+0000","last_peered":"2024-04-25T10:00:33.079752+0000","last_clean":"2024-04-25T10:00:33.079752+0000","last_became_active":"2024-04-25T10:00:33.074841+0000","last_became_peered":"2024-04-25T10:00:33.074841+0000","last_unstale":"2024-04-25T10:00:33.079752+0000","last_undegraded":"2024-04-25T10:00:33.079752+0000","last_fullsized":"2024-04-25T10:00:33.079752+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:11.208934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093406+0000","last_change":"2024-04-25T10:00:33.089468+0000","last_active":"2024-04-25T10:00:33.093406+0000","last_peered":"2024-04-25T10:00:33.093406+0000","last_clean":"2024-04-25T10:00:33.093406+0000","last_became_active":"2024-04-25T10:00:33.089177+0000","last_became_peered":"2024-04-25T10:00:33.089177+0000","last_unstale":"2024-04-25T10:00:33.093406+0000","last_undegraded":"2024-04-25T10:00:33.093406+0000","last_fullsized":"2024-04-25T10:00:33.093406+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:58.111467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090803+0000","last_change":"2024-04-25T10:00:33.086644+0000","last_active":"2024-04-25T10:00:33.090803+0000","last_peered":"2024-04-25T10:00:33.090803+0000","last_clean":"2024-04-25T10:00:33.090803+0000","last_became_active":"2024-04-25T10:00:33.086379+0000","last_became_peered":"2024-04-25T10:00:33.086379+0000","last_unstale":"2024-04-25T10:00:33.090803+0000","last_undegraded":"2024-04-25T10:00:33.090803+0000","last_fullsized":"2024-04-25T10:00:33.090803+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:35.272876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093032+0000","last_change":"2024-04-25T10:00:33.088002+0000","last_active":"2024-04-25T10:00:33.093032+0000","last_peered":"2024-04-25T10:00:33.093032+0000","last_clean":"2024-04-25T10:00:33.093032+0000","last_became_active":"2024-04-25T10:00:33.087609+0000","last_became_peered":"2024-04-25T10:00:33.087609+0000","last_unstale":"2024-04-25T10:00:33.093032+0000","last_undegraded":"2024-04-25T10:00:33.093032+0000","last_fullsized":"2024-04-25T10:00:33.093032+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:30.045573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.083043+0000","last_change":"2024-04-25T10:00:33.081210+0000","last_active":"2024-04-25T10:00:33.083043+0000","last_peered":"2024-04-25T10:00:33.083043+0000","last_clean":"2024-04-25T10:00:33.083043+0000","last_became_active":"2024-04-25T10:00:33.080936+0000","last_became_peered":"2024-04-25T10:00:33.080936+0000","last_unstale":"2024-04-25T10:00:33.083043+0000","last_undegraded":"2024-04-25T10:00:33.083043+0000","last_fullsized":"2024-04-25T10:00:33.083043+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:23.272207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.084270+0000","last_change":"2024-04-25T10:00:33.078825+0000","last_active":"2024-04-25T10:00:33.084270+0000","last_peered":"2024-04-25T10:00:33.084270+0000","last_clean":"2024-04-25T10:00:33.084270+0000","last_became_active":"2024-04-25T10:00:33.078494+0000","last_became_peered":"2024-04-25T10:00:33.078494+0000","last_unstale":"2024-04-25T10:00:33.084270+0000","last_undegraded":"2024-04-25T10:00:33.084270+0000","last_fullsized":"2024-04-25T10:00:33.084270+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:10.756822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092830+0000","last_change":"2024-04-25T10:00:33.090807+0000","last_active":"2024-04-25T10:00:33.092830+0000","last_peered":"2024-04-25T10:00:33.092830+0000","last_clean":"2024-04-25T10:00:33.092830+0000","last_became_active":"2024-04-25T10:00:33.089480+0000","last_became_peered":"2024-04-25T10:00:33.089480+0000","last_unstale":"2024-04-25T10:00:33.092830+0000","last_undegraded":"2024-04-25T10:00:33.092830+0000","last_fullsized":"2024-04-25T10:00:33.092830+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:35.704465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.078898+0000","last_change":"2024-04-25T10:00:33.075447+0000","last_active":"2024-04-25T10:00:33.078898+0000","last_peered":"2024-04-25T10:00:33.078898+0000","last_clean":"2024-04-25T10:00:33.078898+0000","last_became_active":"2024-04-25T10:00:33.072693+0000","last_became_peered":"2024-04-25T10:00:33.072693+0000","last_unstale":"2024-04-25T10:00:33.078898+0000","last_undegraded":"2024-04-25T10:00:33.078898+0000","last_fullsized":"2024-04-25T10:00:33.078898+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:58.479858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.078271+0000","last_change":"2024-04-25T10:00:33.075096+0000","last_active":"2024-04-25T10:00:33.078271+0000","last_peered":"2024-04-25T10:00:33.078271+0000","last_clean":"2024-04-25T10:00:33.078271+0000","last_became_active":"2024-04-25T10:00:33.074800+0000","last_became_peered":"2024-04-25T10:00:33.074800+0000","last_unstale":"2024-04-25T10:00:33.078271+0000","last_undegraded":"2024-04-25T10:00:33.078271+0000","last_fullsized":"2024-04-25T10:00:33.078271+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:28.307481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.087681+0000","last_change":"2024-04-25T10:00:33.084268+0000","last_active":"2024-04-25T10:00:33.087681+0000","last_peered":"2024-04-25T10:00:33.087681+0000","last_clean":"2024-04-25T10:00:33.087681+0000","last_became_active":"2024-04-25T10:00:33.083973+0000","last_became_peered":"2024-04-25T10:00:33.083973+0000","last_unstale":"2024-04-25T10:00:33.087681+0000","last_undegraded":"2024-04-25T10:00:33.087681+0000","last_fullsized":"2024-04-25T10:00:33.087681+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:10.756822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090823+0000","last_change":"2024-04-25T10:00:33.087052+0000","last_active":"2024-04-25T10:00:33.090823+0000","last_peered":"2024-04-25T10:00:33.090823+0000","last_clean":"2024-04-25T10:00:33.090823+0000","last_became_active":"2024-04-25T10:00:33.086106+0000","last_became_peered":"2024-04-25T10:00:33.086106+0000","last_unstale":"2024-04-25T10:00:33.090823+0000","last_undegraded":"2024-04-25T10:00:33.090823+0000","last_fullsized":"2024-04-25T10:00:33.090823+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:38.220436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.095834+0000","last_change":"2024-04-25T10:00:33.094838+0000","last_active":"2024-04-25T10:00:33.095834+0000","last_peered":"2024-04-25T10:00:33.095834+0000","last_clean":"2024-04-25T10:00:33.095834+0000","last_became_active":"2024-04-25T10:00:33.094553+0000","last_became_peered":"2024-04-25T10:00:33.094553+0000","last_unstale":"2024-04-25T10:00:33.095834+0000","last_undegraded":"2024-04-25T10:00:33.095834+0000","last_fullsized":"2024-04-25T10:00:33.095834+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:38.220436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"116'4","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.098660+0000","last_change":"2024-04-25T10:00:32.058752+0000","last_active":"2024-04-25T10:00:33.098660+0000","last_peered":"2024-04-25T10:00:33.098660+0000","last_clean":"2024-04-25T10:00:33.098660+0000","last_became_active":"2024-04-25T10:00:32.058483+0000","last_became_peered":"2024-04-25T10:00:32.058483+0000","last_unstale":"2024-04-25T10:00:33.098660+0000","last_undegraded":"2024-04-25T10:00:33.098660+0000","last_fullsized":"2024-04-25T10:00:33.098660+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:45.880926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092642+0000","last_change":"2024-04-25T10:00:33.075965+0000","last_active":"2024-04-25T10:00:33.092642+0000","last_peered":"2024-04-25T10:00:33.092642+0000","last_clean":"2024-04-25T10:00:33.092642+0000","last_became_active":"2024-04-25T10:00:33.075664+0000","last_became_peered":"2024-04-25T10:00:33.075664+0000","last_unstale":"2024-04-25T10:00:33.092642+0000","last_undegraded":"2024-04-25T10:00:33.092642+0000","last_fullsized":"2024-04-25T10:00:33.092642+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:05.969814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"116'1","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.084440+0000","last_change":"2024-04-25T10:00:32.072195+0000","last_active":"2024-04-25T10:00:33.084440+0000","last_peered":"2024-04-25T10:00:33.084440+0000","last_clean":"2024-04-25T10:00:33.084440+0000","last_became_active":"2024-04-25T10:00:32.071408+0000","last_became_peered":"2024-04-25T10:00:32.071408+0000","last_unstale":"2024-04-25T10:00:33.084440+0000","last_undegraded":"2024-04-25T10:00:33.084440+0000","last_fullsized":"2024-04-25T10:00:33.084440+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:25.802978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.098474+0000","last_change":"2024-04-25T10:00:33.095042+0000","last_active":"2024-04-25T10:00:33.098474+0000","last_peered":"2024-04-25T10:00:33.098474+0000","last_clean":"2024-04-25T10:00:33.098474+0000","last_became_active":"2024-04-25T10:00:33.094806+0000","last_became_peered":"2024-04-25T10:00:33.094806+0000","last_unstale":"2024-04-25T10:00:33.098474+0000","last_undegraded":"2024-04-25T10:00:33.098474+0000","last_fullsized":"2024-04-25T10:00:33.098474+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:34.575394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.095853+0000","last_change":"2024-04-25T10:00:32.058554+0000","last_active":"2024-04-25T10:00:33.095853+0000","last_peered":"2024-04-25T10:00:33.095853+0000","last_clean":"2024-04-25T10:00:33.095853+0000","last_became_active":"2024-04-25T10:00:32.058248+0000","last_became_peered":"2024-04-25T10:00:32.058248+0000","last_unstale":"2024-04-25T10:00:33.095853+0000","last_undegraded":"2024-04-25T10:00:33.095853+0000","last_fullsized":"2024-04-25T10:00:33.095853+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:33.567737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085097+0000","last_change":"2024-04-25T10:00:33.077798+0000","last_active":"2024-04-25T10:00:33.085097+0000","last_peered":"2024-04-25T10:00:33.085097+0000","last_clean":"2024-04-25T10:00:33.085097+0000","last_became_active":"2024-04-25T10:00:33.077515+0000","last_became_peered":"2024-04-25T10:00:33.077515+0000","last_unstale":"2024-04-25T10:00:33.085097+0000","last_undegraded":"2024-04-25T10:00:33.085097+0000","last_fullsized":"2024-04-25T10:00:33.085097+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:55.527525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092420+0000","last_change":"2024-04-25T10:00:32.062931+0000","last_active":"2024-04-25T10:00:33.092420+0000","last_peered":"2024-04-25T10:00:33.092420+0000","last_clean":"2024-04-25T10:00:33.092420+0000","last_became_active":"2024-04-25T10:00:32.062624+0000","last_became_peered":"2024-04-25T10:00:32.062624+0000","last_unstale":"2024-04-25T10:00:33.092420+0000","last_undegraded":"2024-04-25T10:00:33.092420+0000","last_fullsized":"2024-04-25T10:00:33.092420+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:39.658994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086905+0000","last_change":"2024-04-25T10:00:33.085492+0000","last_active":"2024-04-25T10:00:33.086905+0000","last_peered":"2024-04-25T10:00:33.086905+0000","last_clean":"2024-04-25T10:00:33.086905+0000","last_became_active":"2024-04-25T10:00:33.085176+0000","last_became_peered":"2024-04-25T10:00:33.085176+0000","last_unstale":"2024-04-25T10:00:33.086905+0000","last_undegraded":"2024-04-25T10:00:33.086905+0000","last_fullsized":"2024-04-25T10:00:33.086905+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:46.888583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"116'1","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.096098+0000","last_change":"2024-04-25T10:00:32.062480+0000","last_active":"2024-04-25T10:00:33.096098+0000","last_peered":"2024-04-25T10:00:33.096098+0000","last_clean":"2024-04-25T10:00:33.096098+0000","last_became_active":"2024-04-25T10:00:32.062196+0000","last_became_peered":"2024-04-25T10:00:32.062196+0000","last_unstale":"2024-04-25T10:00:33.096098+0000","last_undegraded":"2024-04-25T10:00:33.096098+0000","last_fullsized":"2024-04-25T10:00:33.096098+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:51.453678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.080731+0000","last_change":"2024-04-25T10:00:33.074895+0000","last_active":"2024-04-25T10:00:33.080731+0000","last_peered":"2024-04-25T10:00:33.080731+0000","last_clean":"2024-04-25T10:00:33.080731+0000","last_became_active":"2024-04-25T10:00:33.074581+0000","last_became_peered":"2024-04-25T10:00:33.074581+0000","last_unstale":"2024-04-25T10:00:33.080731+0000","last_undegraded":"2024-04-25T10:00:33.080731+0000","last_fullsized":"2024-04-25T10:00:33.080731+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:26.810634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"116'1","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093636+0000","last_change":"2024-04-25T10:00:32.068681+0000","last_active":"2024-04-25T10:00:33.093636+0000","last_peered":"2024-04-25T10:00:33.093636+0000","last_clean":"2024-04-25T10:00:33.093636+0000","last_became_active":"2024-04-25T10:00:32.068230+0000","last_became_peered":"2024-04-25T10:00:32.068230+0000","last_unstale":"2024-04-25T10:00:33.093636+0000","last_undegraded":"2024-04-25T10:00:33.093636+0000","last_fullsized":"2024-04-25T10:00:33.093636+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:39.660320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.087653+0000","last_change":"2024-04-25T10:00:33.084125+0000","last_active":"2024-04-25T10:00:33.087653+0000","last_peered":"2024-04-25T10:00:33.087653+0000","last_clean":"2024-04-25T10:00:33.087653+0000","last_became_active":"2024-04-25T10:00:33.083839+0000","last_became_peered":"2024-04-25T10:00:33.083839+0000","last_unstale":"2024-04-25T10:00:33.087653+0000","last_undegraded":"2024-04-25T10:00:33.087653+0000","last_fullsized":"2024-04-25T10:00:33.087653+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:25.995898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.077403+0000","last_change":"2024-04-25T10:00:32.061863+0000","last_active":"2024-04-25T10:00:33.077403+0000","last_peered":"2024-04-25T10:00:33.077403+0000","last_clean":"2024-04-25T10:00:33.077403+0000","last_became_active":"2024-04-25T10:00:32.061414+0000","last_became_peered":"2024-04-25T10:00:32.061414+0000","last_unstale":"2024-04-25T10:00:33.077403+0000","last_undegraded":"2024-04-25T10:00:33.077403+0000","last_fullsized":"2024-04-25T10:00:33.077403+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:45.880926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093414+0000","last_change":"2024-04-25T10:00:33.092069+0000","last_active":"2024-04-25T10:00:33.093414+0000","last_peered":"2024-04-25T10:00:33.093414+0000","last_clean":"2024-04-25T10:00:33.093414+0000","last_became_active":"2024-04-25T10:00:33.089864+0000","last_became_peered":"2024-04-25T10:00:33.089864+0000","last_unstale":"2024-04-25T10:00:33.093414+0000","last_undegraded":"2024-04-25T10:00:33.093414+0000","last_fullsized":"2024-04-25T10:00:33.093414+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:35.272876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.070482+0000","last_change":"2024-04-25T10:00:32.073711+0000","last_active":"2024-04-25T10:00:33.070482+0000","last_peered":"2024-04-25T10:00:33.070482+0000","last_clean":"2024-04-25T10:00:33.070482+0000","last_became_active":"2024-04-25T10:00:32.073416+0000","last_became_peered":"2024-04-25T10:00:32.073416+0000","last_unstale":"2024-04-25T10:00:33.070482+0000","last_undegraded":"2024-04-25T10:00:33.070482+0000","last_fullsized":"2024-04-25T10:00:33.070482+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:21.757151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085349+0000","last_change":"2024-04-25T10:00:33.081273+0000","last_active":"2024-04-25T10:00:33.085349+0000","last_peered":"2024-04-25T10:00:33.085349+0000","last_clean":"2024-04-25T10:00:33.085349+0000","last_became_active":"2024-04-25T10:00:33.081031+0000","last_became_peered":"2024-04-25T10:00:33.081031+0000","last_unstale":"2024-04-25T10:00:33.085349+0000","last_undegraded":"2024-04-25T10:00:33.085349+0000","last_fullsized":"2024-04-25T10:00:33.085349+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:35.272876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.073730+0000","last_change":"2024-04-25T10:00:32.068048+0000","last_active":"2024-04-25T10:00:33.073730+0000","last_peered":"2024-04-25T10:00:33.073730+0000","last_clean":"2024-04-25T10:00:33.073730+0000","last_became_active":"2024-04-25T10:00:32.067736+0000","last_became_peered":"2024-04-25T10:00:32.067736+0000","last_unstale":"2024-04-25T10:00:33.073730+0000","last_undegraded":"2024-04-25T10:00:33.073730+0000","last_fullsized":"2024-04-25T10:00:33.073730+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:40.503212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093357+0000","last_change":"2024-04-25T10:00:33.089782+0000","last_active":"2024-04-25T10:00:33.093357+0000","last_peered":"2024-04-25T10:00:33.093357+0000","last_clean":"2024-04-25T10:00:33.093357+0000","last_became_active":"2024-04-25T10:00:33.089519+0000","last_became_peered":"2024-04-25T10:00:33.089519+0000","last_unstale":"2024-04-25T10:00:33.093357+0000","last_undegraded":"2024-04-25T10:00:33.093357+0000","last_fullsized":"2024-04-25T10:00:33.093357+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:19.488621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.079505+0000","last_change":"2024-04-25T10:00:32.069176+0000","last_active":"2024-04-25T10:00:33.079505+0000","last_peered":"2024-04-25T10:00:33.079505+0000","last_clean":"2024-04-25T10:00:33.079505+0000","last_became_active":"2024-04-25T10:00:32.068865+0000","last_became_peered":"2024-04-25T10:00:32.068865+0000","last_unstale":"2024-04-25T10:00:33.079505+0000","last_undegraded":"2024-04-25T10:00:33.079505+0000","last_fullsized":"2024-04-25T10:00:33.079505+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:05.307362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086252+0000","last_change":"2024-04-25T10:00:33.079690+0000","last_active":"2024-04-25T10:00:33.086252+0000","last_peered":"2024-04-25T10:00:33.086252+0000","last_clean":"2024-04-25T10:00:33.086252+0000","last_became_active":"2024-04-25T10:00:33.079356+0000","last_became_peered":"2024-04-25T10:00:33.079356+0000","last_unstale":"2024-04-25T10:00:33.086252+0000","last_undegraded":"2024-04-25T10:00:33.086252+0000","last_fullsized":"2024-04-25T10:00:33.086252+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.768887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"116'3","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090965+0000","last_change":"2024-04-25T10:00:32.066394+0000","last_active":"2024-04-25T10:00:33.090965+0000","last_peered":"2024-04-25T10:00:33.090965+0000","last_clean":"2024-04-25T10:00:33.090965+0000","last_became_active":"2024-04-25T10:00:32.066056+0000","last_became_peered":"2024-04-25T10:00:32.066056+0000","last_unstale":"2024-04-25T10:00:33.090965+0000","last_undegraded":"2024-04-25T10:00:33.090965+0000","last_fullsized":"2024-04-25T10:00:33.090965+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:40.503212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090750+0000","last_change":"2024-04-25T10:00:33.086129+0000","last_active":"2024-04-25T10:00:33.090750+0000","last_peered":"2024-04-25T10:00:33.090750+0000","last_clean":"2024-04-25T10:00:33.090750+0000","last_became_active":"2024-04-25T10:00:33.085580+0000","last_became_peered":"2024-04-25T10:00:33.085580+0000","last_unstale":"2024-04-25T10:00:33.090750+0000","last_undegraded":"2024-04-25T10:00:33.090750+0000","last_fullsized":"2024-04-25T10:00:33.090750+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:43.973298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"116'6","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.098564+0000","last_change":"2024-04-25T10:00:32.069720+0000","last_active":"2024-04-25T10:00:33.098564+0000","last_peered":"2024-04-25T10:00:33.098564+0000","last_clean":"2024-04-25T10:00:33.098564+0000","last_became_active":"2024-04-25T10:00:32.069484+0000","last_became_peered":"2024-04-25T10:00:32.069484+0000","last_unstale":"2024-04-25T10:00:33.098564+0000","last_undegraded":"2024-04-25T10:00:33.098564+0000","last_fullsized":"2024-04-25T10:00:33.098564+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:44.619876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.084033+0000","last_change":"2024-04-25T10:00:33.079306+0000","last_active":"2024-04-25T10:00:33.084033+0000","last_peered":"2024-04-25T10:00:33.084033+0000","last_clean":"2024-04-25T10:00:33.084033+0000","last_became_active":"2024-04-25T10:00:33.079057+0000","last_became_peered":"2024-04-25T10:00:33.079057+0000","last_unstale":"2024-04-25T10:00:33.084033+0000","last_undegraded":"2024-04-25T10:00:33.084033+0000","last_fullsized":"2024-04-25T10:00:33.084033+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:43.973298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093811+0000","last_change":"2024-04-25T10:00:32.067975+0000","last_active":"2024-04-25T10:00:33.093811+0000","last_peered":"2024-04-25T10:00:33.093811+0000","last_clean":"2024-04-25T10:00:33.093811+0000","last_became_active":"2024-04-25T10:00:32.067655+0000","last_became_peered":"2024-04-25T10:00:32.067655+0000","last_unstale":"2024-04-25T10:00:33.093811+0000","last_undegraded":"2024-04-25T10:00:33.093811+0000","last_fullsized":"2024-04-25T10:00:33.093811+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:27.904957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093228+0000","last_change":"2024-04-25T10:00:33.089117+0000","last_active":"2024-04-25T10:00:33.093228+0000","last_peered":"2024-04-25T10:00:33.093228+0000","last_clean":"2024-04-25T10:00:33.093228+0000","last_became_active":"2024-04-25T10:00:33.088746+0000","last_became_peered":"2024-04-25T10:00:33.088746+0000","last_unstale":"2024-04-25T10:00:33.093228+0000","last_undegraded":"2024-04-25T10:00:33.093228+0000","last_fullsized":"2024-04-25T10:00:33.093228+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:10.756822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.065860+0000","last_change":"2024-04-25T10:00:32.072530+0000","last_active":"2024-04-25T10:00:33.065860+0000","last_peered":"2024-04-25T10:00:33.065860+0000","last_clean":"2024-04-25T10:00:33.065860+0000","last_became_active":"2024-04-25T10:00:32.072234+0000","last_became_peered":"2024-04-25T10:00:32.072234+0000","last_unstale":"2024-04-25T10:00:33.065860+0000","last_undegraded":"2024-04-25T10:00:33.065860+0000","last_fullsized":"2024-04-25T10:00:33.065860+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:53.154087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.075863+0000","last_change":"2024-04-25T10:00:33.071505+0000","last_active":"2024-04-25T10:00:33.075863+0000","last_peered":"2024-04-25T10:00:33.075863+0000","last_clean":"2024-04-25T10:00:33.075863+0000","last_became_active":"2024-04-25T10:00:33.071242+0000","last_became_peered":"2024-04-25T10:00:33.071242+0000","last_unstale":"2024-04-25T10:00:33.075863+0000","last_undegraded":"2024-04-25T10:00:33.075863+0000","last_fullsized":"2024-04-25T10:00:33.075863+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:00.709999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.065151+0000","last_change":"2024-04-25T10:00:32.058306+0000","last_active":"2024-04-25T10:00:33.065151+0000","last_peered":"2024-04-25T10:00:33.065151+0000","last_clean":"2024-04-25T10:00:33.065151+0000","last_became_active":"2024-04-25T10:00:32.057993+0000","last_became_peered":"2024-04-25T10:00:32.057993+0000","last_unstale":"2024-04-25T10:00:33.065151+0000","last_undegraded":"2024-04-25T10:00:33.065151+0000","last_fullsized":"2024-04-25T10:00:33.065151+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:43.632761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.098453+0000","last_change":"2024-04-25T10:00:33.093924+0000","last_active":"2024-04-25T10:00:33.098453+0000","last_peered":"2024-04-25T10:00:33.098453+0000","last_clean":"2024-04-25T10:00:33.098453+0000","last_became_active":"2024-04-25T10:00:33.093644+0000","last_became_peered":"2024-04-25T10:00:33.093644+0000","last_unstale":"2024-04-25T10:00:33.098453+0000","last_undegraded":"2024-04-25T10:00:33.098453+0000","last_fullsized":"2024-04-25T10:00:33.098453+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:44.640418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"116'1","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.094350+0000","last_change":"2024-04-25T10:00:32.070024+0000","last_active":"2024-04-25T10:00:33.094350+0000","last_peered":"2024-04-25T10:00:33.094350+0000","last_clean":"2024-04-25T10:00:33.094350+0000","last_became_active":"2024-04-25T10:00:32.069723+0000","last_became_peered":"2024-04-25T10:00:32.069723+0000","last_unstale":"2024-04-25T10:00:33.094350+0000","last_undegraded":"2024-04-25T10:00:33.094350+0000","last_fullsized":"2024-04-25T10:00:33.094350+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:10.201277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.082487+0000","last_change":"2024-04-25T10:00:33.075187+0000","last_active":"2024-04-25T10:00:33.082487+0000","last_peered":"2024-04-25T10:00:33.082487+0000","last_clean":"2024-04-25T10:00:33.082487+0000","last_became_active":"2024-04-25T10:00:33.074899+0000","last_became_peered":"2024-04-25T10:00:33.074899+0000","last_unstale":"2024-04-25T10:00:33.082487+0000","last_undegraded":"2024-04-25T10:00:33.082487+0000","last_fullsized":"2024-04-25T10:00:33.082487+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:40.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.095546+0000","last_change":"2024-04-25T10:00:32.068913+0000","last_active":"2024-04-25T10:00:33.095546+0000","last_peered":"2024-04-25T10:00:33.095546+0000","last_clean":"2024-04-25T10:00:33.095546+0000","last_became_active":"2024-04-25T10:00:32.068618+0000","last_became_peered":"2024-04-25T10:00:32.068618+0000","last_unstale":"2024-04-25T10:00:33.095546+0000","last_undegraded":"2024-04-25T10:00:33.095546+0000","last_fullsized":"2024-04-25T10:00:33.095546+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:33.434267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092646+0000","last_change":"2024-04-25T10:00:33.082469+0000","last_active":"2024-04-25T10:00:33.092646+0000","last_peered":"2024-04-25T10:00:33.092646+0000","last_clean":"2024-04-25T10:00:33.092646+0000","last_became_active":"2024-04-25T10:00:33.079407+0000","last_became_peered":"2024-04-25T10:00:33.079407+0000","last_unstale":"2024-04-25T10:00:33.092646+0000","last_undegraded":"2024-04-25T10:00:33.092646+0000","last_fullsized":"2024-04-25T10:00:33.092646+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:52.403120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"116'1","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.096083+0000","last_change":"2024-04-25T10:00:32.065456+0000","last_active":"2024-04-25T10:00:33.096083+0000","last_peered":"2024-04-25T10:00:33.096083+0000","last_clean":"2024-04-25T10:00:33.096083+0000","last_became_active":"2024-04-25T10:00:32.065176+0000","last_became_peered":"2024-04-25T10:00:32.065176+0000","last_unstale":"2024-04-25T10:00:33.096083+0000","last_undegraded":"2024-04-25T10:00:33.096083+0000","last_fullsized":"2024-04-25T10:00:33.096083+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:27.299824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093339+0000","last_change":"2024-04-25T10:00:33.088628+0000","last_active":"2024-04-25T10:00:33.093339+0000","last_peered":"2024-04-25T10:00:33.093339+0000","last_clean":"2024-04-25T10:00:33.093339+0000","last_became_active":"2024-04-25T10:00:33.088330+0000","last_became_peered":"2024-04-25T10:00:33.088330+0000","last_unstale":"2024-04-25T10:00:33.093339+0000","last_undegraded":"2024-04-25T10:00:33.093339+0000","last_fullsized":"2024-04-25T10:00:33.093339+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:28.912614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.067537+0000","last_change":"2024-04-25T10:00:32.066399+0000","last_active":"2024-04-25T10:00:33.067537+0000","last_peered":"2024-04-25T10:00:33.067537+0000","last_clean":"2024-04-25T10:00:33.067537+0000","last_became_active":"2024-04-25T10:00:32.065919+0000","last_became_peered":"2024-04-25T10:00:32.065919+0000","last_unstale":"2024-04-25T10:00:33.067537+0000","last_undegraded":"2024-04-25T10:00:33.067537+0000","last_fullsized":"2024-04-25T10:00:33.067537+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:18.480965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092912+0000","last_change":"2024-04-25T10:00:33.089653+0000","last_active":"2024-04-25T10:00:33.092912+0000","last_peered":"2024-04-25T10:00:33.092912+0000","last_clean":"2024-04-25T10:00:33.092912+0000","last_became_active":"2024-04-25T10:00:33.089322+0000","last_became_peered":"2024-04-25T10:00:33.089322+0000","last_unstale":"2024-04-25T10:00:33.092912+0000","last_undegraded":"2024-04-25T10:00:33.092912+0000","last_fullsized":"2024-04-25T10:00:33.092912+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:48.899768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092983+0000","last_change":"2024-04-25T10:00:32.067452+0000","last_active":"2024-04-25T10:00:33.092983+0000","last_peered":"2024-04-25T10:00:33.092983+0000","last_clean":"2024-04-25T10:00:33.092983+0000","last_became_active":"2024-04-25T10:00:32.067032+0000","last_became_peered":"2024-04-25T10:00:32.067032+0000","last_unstale":"2024-04-25T10:00:33.092983+0000","last_undegraded":"2024-04-25T10:00:33.092983+0000","last_fullsized":"2024-04-25T10:00:33.092983+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:16.507017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.087274+0000","last_change":"2024-04-25T10:00:33.083871+0000","last_active":"2024-04-25T10:00:33.087274+0000","last_peered":"2024-04-25T10:00:33.087274+0000","last_clean":"2024-04-25T10:00:33.087274+0000","last_became_active":"2024-04-25T10:00:33.081695+0000","last_became_peered":"2024-04-25T10:00:33.081695+0000","last_unstale":"2024-04-25T10:00:33.087274+0000","last_undegraded":"2024-04-25T10:00:33.087274+0000","last_fullsized":"2024-04-25T10:00:33.087274+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:14.717224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.068658+0000","last_change":"2024-04-25T10:00:32.058582+0000","last_active":"2024-04-25T10:00:33.068658+0000","last_peered":"2024-04-25T10:00:33.068658+0000","last_clean":"2024-04-25T10:00:33.068658+0000","last_became_active":"2024-04-25T10:00:32.058302+0000","last_became_peered":"2024-04-25T10:00:32.058302+0000","last_unstale":"2024-04-25T10:00:33.068658+0000","last_undegraded":"2024-04-25T10:00:33.068658+0000","last_fullsized":"2024-04-25T10:00:33.068658+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:44.619876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086301+0000","last_change":"2024-04-25T10:00:33.080516+0000","last_active":"2024-04-25T10:00:33.086301+0000","last_peered":"2024-04-25T10:00:33.086301+0000","last_clean":"2024-04-25T10:00:33.086301+0000","last_became_active":"2024-04-25T10:00:33.080135+0000","last_became_peered":"2024-04-25T10:00:33.080135+0000","last_unstale":"2024-04-25T10:00:33.086301+0000","last_undegraded":"2024-04-25T10:00:33.086301+0000","last_fullsized":"2024-04-25T10:00:33.086301+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:55.527525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.065344+0000","last_change":"2024-04-25T10:00:32.069188+0000","last_active":"2024-04-25T10:00:33.065344+0000","last_peered":"2024-04-25T10:00:33.065344+0000","last_clean":"2024-04-25T10:00:33.065344+0000","last_became_active":"2024-04-25T10:00:32.068888+0000","last_became_peered":"2024-04-25T10:00:32.068888+0000","last_unstale":"2024-04-25T10:00:33.065344+0000","last_undegraded":"2024-04-25T10:00:33.065344+0000","last_fullsized":"2024-04-25T10:00:33.065344+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:33.567737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085291+0000","last_change":"2024-04-25T10:00:33.081845+0000","last_active":"2024-04-25T10:00:33.085291+0000","last_peered":"2024-04-25T10:00:33.085291+0000","last_clean":"2024-04-25T10:00:33.085291+0000","last_became_active":"2024-04-25T10:00:33.081595+0000","last_became_peered":"2024-04-25T10:00:33.081595+0000","last_unstale":"2024-04-25T10:00:33.085291+0000","last_undegraded":"2024-04-25T10:00:33.085291+0000","last_fullsized":"2024-04-25T10:00:33.085291+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:11.688215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.067754+0000","last_change":"2024-04-25T10:00:32.059011+0000","last_active":"2024-04-25T10:00:33.067754+0000","last_peered":"2024-04-25T10:00:33.067754+0000","last_clean":"2024-04-25T10:00:33.067754+0000","last_became_active":"2024-04-25T10:00:32.058741+0000","last_became_peered":"2024-04-25T10:00:32.058741+0000","last_unstale":"2024-04-25T10:00:33.067754+0000","last_undegraded":"2024-04-25T10:00:33.067754+0000","last_fullsized":"2024-04-25T10:00:33.067754+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:43.632761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092853+0000","last_change":"2024-04-25T10:00:33.091014+0000","last_active":"2024-04-25T10:00:33.092853+0000","last_peered":"2024-04-25T10:00:33.092853+0000","last_clean":"2024-04-25T10:00:33.092853+0000","last_became_active":"2024-04-25T10:00:33.089860+0000","last_became_peered":"2024-04-25T10:00:33.089860+0000","last_unstale":"2024-04-25T10:00:33.092853+0000","last_undegraded":"2024-04-25T10:00:33.092853+0000","last_fullsized":"2024-04-25T10:00:33.092853+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:23.776136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092837+0000","last_change":"2024-04-25T10:00:32.065946+0000","last_active":"2024-04-25T10:00:33.092837+0000","last_peered":"2024-04-25T10:00:33.092837+0000","last_clean":"2024-04-25T10:00:33.092837+0000","last_became_active":"2024-04-25T10:00:32.065650+0000","last_became_peered":"2024-04-25T10:00:32.065650+0000","last_unstale":"2024-04-25T10:00:33.092837+0000","last_undegraded":"2024-04-25T10:00:33.092837+0000","last_fullsized":"2024-04-25T10:00:33.092837+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:26.788956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.089278+0000","last_change":"2024-04-25T10:00:33.086393+0000","last_active":"2024-04-25T10:00:33.089278+0000","last_peered":"2024-04-25T10:00:33.089278+0000","last_clean":"2024-04-25T10:00:33.089278+0000","last_became_active":"2024-04-25T10:00:33.086049+0000","last_became_peered":"2024-04-25T10:00:33.086049+0000","last_unstale":"2024-04-25T10:00:33.089278+0000","last_undegraded":"2024-04-25T10:00:33.089278+0000","last_fullsized":"2024-04-25T10:00:33.089278+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:41.510869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.073688+0000","last_change":"2024-04-25T10:00:32.068064+0000","last_active":"2024-04-25T10:00:33.073688+0000","last_peered":"2024-04-25T10:00:33.073688+0000","last_clean":"2024-04-25T10:00:33.073688+0000","last_became_active":"2024-04-25T10:00:32.067687+0000","last_became_peered":"2024-04-25T10:00:32.067687+0000","last_unstale":"2024-04-25T10:00:33.073688+0000","last_undegraded":"2024-04-25T10:00:33.073688+0000","last_fullsized":"2024-04-25T10:00:33.073688+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:05.307362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.074099+0000","last_change":"2024-04-25T10:00:33.072598+0000","last_active":"2024-04-25T10:00:33.074099+0000","last_peered":"2024-04-25T10:00:33.074099+0000","last_clean":"2024-04-25T10:00:33.074099+0000","last_became_active":"2024-04-25T10:00:33.072156+0000","last_became_peered":"2024-04-25T10:00:33.072156+0000","last_unstale":"2024-04-25T10:00:33.074099+0000","last_undegraded":"2024-04-25T10:00:33.074099+0000","last_fullsized":"2024-04-25T10:00:33.074099+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:23.032578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.073909+0000","last_change":"2024-04-25T10:00:32.068129+0000","last_active":"2024-04-25T10:00:33.073909+0000","last_peered":"2024-04-25T10:00:33.073909+0000","last_clean":"2024-04-25T10:00:33.073909+0000","last_became_active":"2024-04-25T10:00:32.067680+0000","last_became_peered":"2024-04-25T10:00:32.067680+0000","last_unstale":"2024-04-25T10:00:33.073909+0000","last_undegraded":"2024-04-25T10:00:33.073909+0000","last_fullsized":"2024-04-25T10:00:33.073909+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.680558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090032+0000","last_change":"2024-04-25T10:00:33.084569+0000","last_active":"2024-04-25T10:00:33.090032+0000","last_peered":"2024-04-25T10:00:33.090032+0000","last_clean":"2024-04-25T10:00:33.090032+0000","last_became_active":"2024-04-25T10:00:33.083734+0000","last_became_peered":"2024-04-25T10:00:33.083734+0000","last_unstale":"2024-04-25T10:00:33.090032+0000","last_undegraded":"2024-04-25T10:00:33.090032+0000","last_fullsized":"2024-04-25T10:00:33.090032+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:22.651062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.072898+0000","last_change":"2024-04-25T10:00:32.074426+0000","last_active":"2024-04-25T10:00:33.072898+0000","last_peered":"2024-04-25T10:00:33.072898+0000","last_clean":"2024-04-25T10:00:33.072898+0000","last_became_active":"2024-04-25T10:00:32.074127+0000","last_became_peered":"2024-04-25T10:00:32.074127+0000","last_unstale":"2024-04-25T10:00:33.072898+0000","last_undegraded":"2024-04-25T10:00:33.072898+0000","last_fullsized":"2024-04-25T10:00:33.072898+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:21.454244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.077417+0000","last_change":"2024-04-25T10:00:33.073354+0000","last_active":"2024-04-25T10:00:33.077417+0000","last_peered":"2024-04-25T10:00:33.077417+0000","last_clean":"2024-04-25T10:00:33.077417+0000","last_became_active":"2024-04-25T10:00:33.072772+0000","last_became_peered":"2024-04-25T10:00:33.072772+0000","last_unstale":"2024-04-25T10:00:33.077417+0000","last_undegraded":"2024-04-25T10:00:33.077417+0000","last_fullsized":"2024-04-25T10:00:33.077417+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:47.544864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.067251+0000","last_change":"2024-04-25T10:00:32.053556+0000","last_active":"2024-04-25T10:00:33.067251+0000","last_peered":"2024-04-25T10:00:33.067251+0000","last_clean":"2024-04-25T10:00:33.067251+0000","last_became_active":"2024-04-25T10:00:32.053281+0000","last_became_peered":"2024-04-25T10:00:32.053281+0000","last_unstale":"2024-04-25T10:00:33.067251+0000","last_undegraded":"2024-04-25T10:00:33.067251+0000","last_fullsized":"2024-04-25T10:00:33.067251+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:33.329170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086301+0000","last_change":"2024-04-25T10:00:33.079422+0000","last_active":"2024-04-25T10:00:33.086301+0000","last_peered":"2024-04-25T10:00:33.086301+0000","last_clean":"2024-04-25T10:00:33.086301+0000","last_became_active":"2024-04-25T10:00:33.079123+0000","last_became_peered":"2024-04-25T10:00:33.079123+0000","last_unstale":"2024-04-25T10:00:33.086301+0000","last_undegraded":"2024-04-25T10:00:33.086301+0000","last_fullsized":"2024-04-25T10:00:33.086301+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:07.591236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.068149+0000","last_change":"2024-04-25T10:00:32.063538+0000","last_active":"2024-04-25T10:00:33.068149+0000","last_peered":"2024-04-25T10:00:33.068149+0000","last_clean":"2024-04-25T10:00:33.068149+0000","last_became_active":"2024-04-25T10:00:32.063246+0000","last_became_peered":"2024-04-25T10:00:32.063246+0000","last_unstale":"2024-04-25T10:00:33.068149+0000","last_undegraded":"2024-04-25T10:00:33.068149+0000","last_fullsized":"2024-04-25T10:00:33.068149+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:50.024135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.074715+0000","last_change":"2024-04-25T10:00:33.074039+0000","last_active":"2024-04-25T10:00:33.074715+0000","last_peered":"2024-04-25T10:00:33.074715+0000","last_clean":"2024-04-25T10:00:33.074715+0000","last_became_active":"2024-04-25T10:00:33.073730+0000","last_became_peered":"2024-04-25T10:00:33.073730+0000","last_unstale":"2024-04-25T10:00:33.074715+0000","last_undegraded":"2024-04-25T10:00:33.074715+0000","last_fullsized":"2024-04-25T10:00:33.074715+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:07.591236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.069616+0000","last_change":"2024-04-25T10:00:32.073732+0000","last_active":"2024-04-25T10:00:33.069616+0000","last_peered":"2024-04-25T10:00:33.069616+0000","last_clean":"2024-04-25T10:00:33.069616+0000","last_became_active":"2024-04-25T10:00:32.073432+0000","last_became_peered":"2024-04-25T10:00:32.073432+0000","last_unstale":"2024-04-25T10:00:33.069616+0000","last_undegraded":"2024-04-25T10:00:33.069616+0000","last_fullsized":"2024-04-25T10:00:33.069616+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:18.195763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090389+0000","last_change":"2024-04-25T10:00:33.086386+0000","last_active":"2024-04-25T10:00:33.090389+0000","last_peered":"2024-04-25T10:00:33.090389+0000","last_clean":"2024-04-25T10:00:33.090389+0000","last_became_active":"2024-04-25T10:00:33.086010+0000","last_became_peered":"2024-04-25T10:00:33.086010+0000","last_unstale":"2024-04-25T10:00:33.090389+0000","last_undegraded":"2024-04-25T10:00:33.090389+0000","last_fullsized":"2024-04-25T10:00:33.090389+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:06.315019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.067913+0000","last_change":"2024-04-25T10:00:32.064444+0000","last_active":"2024-04-25T10:00:33.067913+0000","last_peered":"2024-04-25T10:00:33.067913+0000","last_clean":"2024-04-25T10:00:33.067913+0000","last_became_active":"2024-04-25T10:00:32.064160+0000","last_became_peered":"2024-04-25T10:00:32.064160+0000","last_unstale":"2024-04-25T10:00:33.067913+0000","last_undegraded":"2024-04-25T10:00:33.067913+0000","last_fullsized":"2024-04-25T10:00:33.067913+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:53.154087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085180+0000","last_change":"2024-04-25T10:00:33.078274+0000","last_active":"2024-04-25T10:00:33.085180+0000","last_peered":"2024-04-25T10:00:33.085180+0000","last_clean":"2024-04-25T10:00:33.085180+0000","last_became_active":"2024-04-25T10:00:33.077677+0000","last_became_peered":"2024-04-25T10:00:33.077677+0000","last_unstale":"2024-04-25T10:00:33.085180+0000","last_undegraded":"2024-04-25T10:00:33.085180+0000","last_fullsized":"2024-04-25T10:00:33.085180+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:15.540434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"116'8","reported_seq":20,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.096480+0000","last_change":"2024-04-25T10:00:32.068349+0000","last_active":"2024-04-25T10:00:33.096480+0000","last_peered":"2024-04-25T10:00:33.096480+0000","last_clean":"2024-04-25T10:00:33.096480+0000","last_became_active":"2024-04-25T10:00:32.068075+0000","last_became_peered":"2024-04-25T10:00:32.068075+0000","last_unstale":"2024-04-25T10:00:33.096480+0000","last_undegraded":"2024-04-25T10:00:33.096480+0000","last_fullsized":"2024-04-25T10:00:33.096480+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:18.480965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":365,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.068033+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:00:33.068033+0000","last_peered":"2024-04-25T10:00:33.068033+0000","last_clean":"2024-04-25T10:00:33.068033+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:00:33.068033+0000","last_undegraded":"2024-04-25T10:00:33.068033+0000","last_fullsized":"2024-04-25T10:00:33.068033+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41508,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330332,"statfs":{"total":96636764160,"available":96594259968,"internally_reserved":0,"allocated":2056192,"data_stored":1264604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46484,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325356,"statfs":{"total":96636764160,"available":96589164544,"internally_reserved":0,"allocated":1449984,"data_stored":674042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45088,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326752,"statfs":{"total":96636764160,"available":96590594048,"internally_reserved":0,"allocated":1454080,"data_stored":673620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50660,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321180,"statfs":{"total":96636764160,"available":96584888320,"internally_reserved":0,"allocated":1458176,"data_stored":675072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50640,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1445888,"data_stored":673460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50644,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321196,"statfs":{"total":96636764160,"available":96584904704,"internally_reserved":0,"allocated":1445888,"data_stored":673460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50644,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321196,"statfs":{"total":96636764160,"available":96584904704,"internally_reserved":0,"allocated":1449984,"data_stored":673482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51232,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320608,"statfs":{"total":96636764160,"available":96584302592,"internally_reserved":0,"allocated":2043904,"data_stored":1263850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:00:36.080 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:00:36.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.213+0000 7f0b5b21f640 1 Processor -- start 2024-04-25T10:00:36.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.213+0000 7f0b5b21f640 1 -- start start 2024-04-25T10:00:36.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.213+0000 7f0b5b21f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b540d1120 0x7f0b5409a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.213+0000 7f0b5b21f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5406c220 con 0x7f0b5415dc70 2024-04-25T10:00:36.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.213+0000 7f0b5b21f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b540d0080 con 0x7f0b5406ce10 2024-04-25T10:00:36.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.213+0000 7f0b5b21f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b540d0c90 con 0x7f0b540d1120 2024-04-25T10:00:36.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.214+0000 7f0b597bc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b540d1120 0x7f0b5409a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.214+0000 7f0b597bc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b540d1120 0x7f0b5409a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44548/0 (socket says 172.21.15.19:44548) 2024-04-25T10:00:36.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.214+0000 7f0b597bc640 1 -- 172.21.15.19:0/1182978155 learned_addr learned my addr 172.21.15.19:0/1182978155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:36.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.214+0000 7f0b58fbb640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0b5415dc70 0x7f0b5406c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47090/0 (socket says 172.21.15.19:47090) 2024-04-25T10:00:36.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.214+0000 7f0b597bc640 1 -- 172.21.15.19:0/1182978155 >> v1:172.21.15.112:6789/0 conn(0x7f0b5406ce10 legacy=0x7f0b540cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.214+0000 7f0b597bc640 1 -- 172.21.15.19:0/1182978155 >> v1:172.21.15.19:6789/0 conn(0x7f0b5415dc70 legacy=0x7f0b5406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.214+0000 7f0b597bc640 1 -- 172.21.15.19:0/1182978155 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540d0360 con 0x7f0b540d1120 2024-04-25T10:00:36.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.214+0000 7f0b537fe640 1 -- 172.21.15.19:0/1182978155 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899679218 0 0) 0x7f0b5406c220 con 0x7f0b5415dc70 2024-04-25T10:00:36.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.215+0000 7f0b597bc640 1 --2- 172.21.15.19:0/1182978155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b540d1120 0x7f0b5409a450 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009e50 tx=0x7f0b4c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e242fc35ae78ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:36.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.215+0000 7f0b537fe640 1 -- 172.21.15.19:0/1182978155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c036070 con 0x7f0b540d1120 2024-04-25T10:00:36.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.215+0000 7f0b537fe640 1 -- 172.21.15.19:0/1182978155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c031600 con 0x7f0b540d1120 2024-04-25T10:00:36.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.215+0000 7f0b537fe640 1 -- 172.21.15.19:0/1182978155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0319f0 con 0x7f0b540d1120 2024-04-25T10:00:36.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.215+0000 7f0b5b21f640 1 -- 172.21.15.19:0/1182978155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b540d1120 msgr2=0x7f0b5409a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.215+0000 7f0b5b21f640 1 --2- 172.21.15.19:0/1182978155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b540d1120 0x7f0b5409a450 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009e50 tx=0x7f0b4c02a7c0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.215+0000 7f0b5b21f640 1 -- 172.21.15.19:0/1182978155 shutdown_connections 2024-04-25T10:00:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.215+0000 7f0b5b21f640 1 --2- 172.21.15.19:0/1182978155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b540d1120 0x7f0b5409a450 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.215+0000 7f0b5b21f640 1 -- 172.21.15.19:0/1182978155 >> 172.21.15.19:0/1182978155 conn(0x7f0b540b3270 msgr2=0x7f0b540b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.216+0000 7f0b5b21f640 1 -- 172.21.15.19:0/1182978155 shutdown_connections 2024-04-25T10:00:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.216+0000 7f0b5b21f640 1 -- 172.21.15.19:0/1182978155 wait complete. 2024-04-25T10:00:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.217+0000 7f0b5b21f640 1 Processor -- start 2024-04-25T10:00:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.217+0000 7f0b5b21f640 1 -- start start 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b5b21f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b5406ce10 0x7f0b5409c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b5b21f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b540d1120 0x7f0b54151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b5b21f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b5415dc70 0x7f0b540cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b5b21f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5415e050 con 0x7f0b540d1120 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b5b21f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540d0920 con 0x7f0b5406ce10 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b5b21f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540cf9c0 con 0x7f0b5415dc70 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b540d1120 0x7f0b54151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b540d1120 0x7f0b54151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44558/0 (socket says 172.21.15.19:44558) 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b53fff640 1 -- 172.21.15.19:0/2438664768 learned_addr learned my addr 172.21.15.19:0/2438664768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b58fbb640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b5406ce10 0x7f0b5409c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b597bc640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b5415dc70 0x7f0b540cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b53fff640 1 -- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b5415dc70 msgr2=0x7f0b540cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b53fff640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b5415dc70 0x7f0b540cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b53fff640 1 -- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b5406ce10 msgr2=0x7f0b5409c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b53fff640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b5406ce10 0x7f0b5409c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b53fff640 1 -- 172.21.15.19:0/2438664768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5409b640 con 0x7f0b540d1120 2024-04-25T10:00:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b53fff640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b540d1120 0x7f0b54151af0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000c450 tx=0x7f0b4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b517fa640 1 -- 172.21.15.19:0/2438664768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b400052d0 con 0x7f0b540d1120 2024-04-25T10:00:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b517fa640 1 -- 172.21.15.19:0/2438664768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b40005470 con 0x7f0b540d1120 2024-04-25T10:00:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b5b21f640 1 -- 172.21.15.19:0/2438664768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5409b920 con 0x7f0b540d1120 2024-04-25T10:00:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.218+0000 7f0b517fa640 1 -- 172.21.15.19:0/2438664768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40012870 con 0x7f0b540d1120 2024-04-25T10:00:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.219+0000 7f0b5b21f640 1 -- 172.21.15.19:0/2438664768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5409bee0 con 0x7f0b540d1120 2024-04-25T10:00:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.219+0000 7f0b5b21f640 1 -- 172.21.15.19:0/2438664768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b54073990 con 0x7f0b540d1120 2024-04-25T10:00:36.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.220+0000 7f0b517fa640 1 -- 172.21.15.19:0/2438664768 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b40012a10 con 0x7f0b540d1120 2024-04-25T10:00:36.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.220+0000 7f0b517fa640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3806b4d0 0x7f0b3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.221+0000 7f0b517fa640 1 -- 172.21.15.19:0/2438664768 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b40089f50 con 0x7f0b540d1120 2024-04-25T10:00:36.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.221+0000 7f0b58fbb640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3806b4d0 0x7f0b3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.221+0000 7f0b58fbb640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3806b4d0 0x7f0b3806d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0b48000f60 tx=0x7f0b48007e40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:36.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.225+0000 7f0b517fa640 1 -- 172.21.15.19:0/2438664768 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b400584d0 con 0x7f0b540d1120 2024-04-25T10:00:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.401+0000 7f0b5b21f640 1 -- 172.21.15.19:0/2438664768 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b540b40a0 con 0x7f0b3806b4d0 2024-04-25T10:00:36.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.407+0000 7f0b517fa640 1 -- 172.21.15.19:0/2438664768 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252416 (secure 0 0 0) 0x7f0b540b40a0 con 0x7f0b3806b4d0 2024-04-25T10:00:36.414 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:36.415 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.410+0000 7f0b5b21f640 1 -- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3806b4d0 msgr2=0x7f0b3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.410+0000 7f0b5b21f640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3806b4d0 0x7f0b3806d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0b48000f60 tx=0x7f0b48007e40 comp rx=0 tx=0).stop 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.410+0000 7f0b5b21f640 1 -- 172.21.15.19:0/2438664768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b540d1120 msgr2=0x7f0b54151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.410+0000 7f0b5b21f640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b540d1120 0x7f0b54151af0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000c450 tx=0x7f0b4000c910 comp rx=0 tx=0).stop 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.411+0000 7f0b5b21f640 1 -- 172.21.15.19:0/2438664768 shutdown_connections 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.411+0000 7f0b5b21f640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3806b4d0 0x7f0b3806d990 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.411+0000 7f0b5b21f640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b5415dc70 0x7f0b540cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.411+0000 7f0b5b21f640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b540d1120 0x7f0b54151af0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.411+0000 7f0b5b21f640 1 --2- 172.21.15.19:0/2438664768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b5406ce10 0x7f0b5409c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.411+0000 7f0b5b21f640 1 -- 172.21.15.19:0/2438664768 >> 172.21.15.19:0/2438664768 conn(0x7f0b540b3270 msgr2=0x7f0b540cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.411+0000 7f0b5b21f640 1 -- 172.21.15.19:0/2438664768 shutdown_connections 2024-04-25T10:00:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.411+0000 7f0b5b21f640 1 -- 172.21.15.19:0/2438664768 wait complete. 2024-04-25T10:00:36.426 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-04-25T10:00:35.013611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":386900,"kb_used_data":12504,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754587820,"statfs":{"total":773094113280,"available":772697927680,"internally_reserved":0,"allocated":12804096,"data_stored":6571590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":383241778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000825"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.088321+0000","last_change":"2024-04-25T10:00:33.086979+0000","last_active":"2024-04-25T10:00:33.088321+0000","last_peered":"2024-04-25T10:00:33.088321+0000","last_clean":"2024-04-25T10:00:33.088321+0000","last_became_active":"2024-04-25T10:00:33.086684+0000","last_became_peered":"2024-04-25T10:00:33.086684+0000","last_unstale":"2024-04-25T10:00:33.088321+0000","last_undegraded":"2024-04-25T10:00:33.088321+0000","last_fullsized":"2024-04-25T10:00:33.088321+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:11.688215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085200+0000","last_change":"2024-04-25T10:00:33.076867+0000","last_active":"2024-04-25T10:00:33.085200+0000","last_peered":"2024-04-25T10:00:33.085200+0000","last_clean":"2024-04-25T10:00:33.085200+0000","last_became_active":"2024-04-25T10:00:33.076544+0000","last_became_peered":"2024-04-25T10:00:33.076544+0000","last_unstale":"2024-04-25T10:00:33.085200+0000","last_undegraded":"2024-04-25T10:00:33.085200+0000","last_fullsized":"2024-04-25T10:00:33.085200+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:19.378845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.088905+0000","last_change":"2024-04-25T10:00:33.083169+0000","last_active":"2024-04-25T10:00:33.088905+0000","last_peered":"2024-04-25T10:00:33.088905+0000","last_clean":"2024-04-25T10:00:33.088905+0000","last_became_active":"2024-04-25T10:00:33.082362+0000","last_became_peered":"2024-04-25T10:00:33.082362+0000","last_unstale":"2024-04-25T10:00:33.088905+0000","last_undegraded":"2024-04-25T10:00:33.088905+0000","last_fullsized":"2024-04-25T10:00:33.088905+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:58.111467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.075411+0000","last_change":"2024-04-25T10:00:33.072589+0000","last_active":"2024-04-25T10:00:33.075411+0000","last_peered":"2024-04-25T10:00:33.075411+0000","last_clean":"2024-04-25T10:00:33.075411+0000","last_became_active":"2024-04-25T10:00:33.072127+0000","last_became_peered":"2024-04-25T10:00:33.072127+0000","last_unstale":"2024-04-25T10:00:33.075411+0000","last_undegraded":"2024-04-25T10:00:33.075411+0000","last_fullsized":"2024-04-25T10:00:33.075411+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:19.378845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086698+0000","last_change":"2024-04-25T10:00:33.083346+0000","last_active":"2024-04-25T10:00:33.086698+0000","last_peered":"2024-04-25T10:00:33.086698+0000","last_clean":"2024-04-25T10:00:33.086698+0000","last_became_active":"2024-04-25T10:00:33.083025+0000","last_became_peered":"2024-04-25T10:00:33.083025+0000","last_unstale":"2024-04-25T10:00:33.086698+0000","last_undegraded":"2024-04-25T10:00:33.086698+0000","last_fullsized":"2024-04-25T10:00:33.086698+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:06.315019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092638+0000","last_change":"2024-04-25T10:00:33.082084+0000","last_active":"2024-04-25T10:00:33.092638+0000","last_peered":"2024-04-25T10:00:33.092638+0000","last_clean":"2024-04-25T10:00:33.092638+0000","last_became_active":"2024-04-25T10:00:33.079288+0000","last_became_peered":"2024-04-25T10:00:33.079288+0000","last_unstale":"2024-04-25T10:00:33.092638+0000","last_undegraded":"2024-04-25T10:00:33.092638+0000","last_fullsized":"2024-04-25T10:00:33.092638+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:19.221838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092701+0000","last_change":"2024-04-25T10:00:33.088018+0000","last_active":"2024-04-25T10:00:33.092701+0000","last_peered":"2024-04-25T10:00:33.092701+0000","last_clean":"2024-04-25T10:00:33.092701+0000","last_became_active":"2024-04-25T10:00:33.082713+0000","last_became_peered":"2024-04-25T10:00:33.082713+0000","last_unstale":"2024-04-25T10:00:33.092701+0000","last_undegraded":"2024-04-25T10:00:33.092701+0000","last_fullsized":"2024-04-25T10:00:33.092701+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:09.650080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.091126+0000","last_change":"2024-04-25T10:00:33.082597+0000","last_active":"2024-04-25T10:00:33.091126+0000","last_peered":"2024-04-25T10:00:33.091126+0000","last_clean":"2024-04-25T10:00:33.091126+0000","last_became_active":"2024-04-25T10:00:33.082075+0000","last_became_peered":"2024-04-25T10:00:33.082075+0000","last_unstale":"2024-04-25T10:00:33.091126+0000","last_undegraded":"2024-04-25T10:00:33.091126+0000","last_fullsized":"2024-04-25T10:00:33.091126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:05.579160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085128+0000","last_change":"2024-04-25T10:00:33.078739+0000","last_active":"2024-04-25T10:00:33.085128+0000","last_peered":"2024-04-25T10:00:33.085128+0000","last_clean":"2024-04-25T10:00:33.085128+0000","last_became_active":"2024-04-25T10:00:33.078474+0000","last_became_peered":"2024-04-25T10:00:33.078474+0000","last_unstale":"2024-04-25T10:00:33.085128+0000","last_undegraded":"2024-04-25T10:00:33.085128+0000","last_fullsized":"2024-04-25T10:00:33.085128+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:05.579160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.095942+0000","last_change":"2024-04-25T10:00:33.094855+0000","last_active":"2024-04-25T10:00:33.095942+0000","last_peered":"2024-04-25T10:00:33.095942+0000","last_clean":"2024-04-25T10:00:33.095942+0000","last_became_active":"2024-04-25T10:00:33.094571+0000","last_became_peered":"2024-04-25T10:00:33.094571+0000","last_unstale":"2024-04-25T10:00:33.095942+0000","last_undegraded":"2024-04-25T10:00:33.095942+0000","last_fullsized":"2024-04-25T10:00:33.095942+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:30.045573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086659+0000","last_change":"2024-04-25T10:00:33.084876+0000","last_active":"2024-04-25T10:00:33.086659+0000","last_peered":"2024-04-25T10:00:33.086659+0000","last_clean":"2024-04-25T10:00:33.086659+0000","last_became_active":"2024-04-25T10:00:33.083238+0000","last_became_peered":"2024-04-25T10:00:33.083238+0000","last_unstale":"2024-04-25T10:00:33.086659+0000","last_undegraded":"2024-04-25T10:00:33.086659+0000","last_fullsized":"2024-04-25T10:00:33.086659+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:41.510869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092880+0000","last_change":"2024-04-25T10:00:33.091045+0000","last_active":"2024-04-25T10:00:33.092880+0000","last_peered":"2024-04-25T10:00:33.092880+0000","last_clean":"2024-04-25T10:00:33.092880+0000","last_became_active":"2024-04-25T10:00:33.090664+0000","last_became_peered":"2024-04-25T10:00:33.090664+0000","last_unstale":"2024-04-25T10:00:33.092880+0000","last_undegraded":"2024-04-25T10:00:33.092880+0000","last_fullsized":"2024-04-25T10:00:33.092880+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:31.173540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093393+0000","last_change":"2024-04-25T10:00:33.090502+0000","last_active":"2024-04-25T10:00:33.093393+0000","last_peered":"2024-04-25T10:00:33.093393+0000","last_clean":"2024-04-25T10:00:33.093393+0000","last_became_active":"2024-04-25T10:00:33.090206+0000","last_became_peered":"2024-04-25T10:00:33.090206+0000","last_unstale":"2024-04-25T10:00:33.093393+0000","last_undegraded":"2024-04-25T10:00:33.093393+0000","last_fullsized":"2024-04-25T10:00:33.093393+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:43.973298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086328+0000","last_change":"2024-04-25T10:00:33.080941+0000","last_active":"2024-04-25T10:00:33.086328+0000","last_peered":"2024-04-25T10:00:33.086328+0000","last_clean":"2024-04-25T10:00:33.086328+0000","last_became_active":"2024-04-25T10:00:33.080503+0000","last_became_peered":"2024-04-25T10:00:33.080503+0000","last_unstale":"2024-04-25T10:00:33.086328+0000","last_undegraded":"2024-04-25T10:00:33.086328+0000","last_fullsized":"2024-04-25T10:00:33.086328+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:15.540434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086729+0000","last_change":"2024-04-25T10:00:33.082453+0000","last_active":"2024-04-25T10:00:33.086729+0000","last_peered":"2024-04-25T10:00:33.086729+0000","last_clean":"2024-04-25T10:00:33.086729+0000","last_became_active":"2024-04-25T10:00:33.082186+0000","last_became_peered":"2024-04-25T10:00:33.082186+0000","last_unstale":"2024-04-25T10:00:33.086729+0000","last_undegraded":"2024-04-25T10:00:33.086729+0000","last_fullsized":"2024-04-25T10:00:33.086729+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:46.888583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092887+0000","last_change":"2024-04-25T10:00:33.078441+0000","last_active":"2024-04-25T10:00:33.092887+0000","last_peered":"2024-04-25T10:00:33.092887+0000","last_clean":"2024-04-25T10:00:33.092887+0000","last_became_active":"2024-04-25T10:00:33.078129+0000","last_became_peered":"2024-04-25T10:00:33.078129+0000","last_unstale":"2024-04-25T10:00:33.092887+0000","last_undegraded":"2024-04-25T10:00:33.092887+0000","last_fullsized":"2024-04-25T10:00:33.092887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:42.968050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.091751+0000","last_change":"2024-04-25T10:00:33.087706+0000","last_active":"2024-04-25T10:00:33.091751+0000","last_peered":"2024-04-25T10:00:33.091751+0000","last_clean":"2024-04-25T10:00:33.091751+0000","last_became_active":"2024-04-25T10:00:33.087409+0000","last_became_peered":"2024-04-25T10:00:33.087409+0000","last_unstale":"2024-04-25T10:00:33.091751+0000","last_undegraded":"2024-04-25T10:00:33.091751+0000","last_fullsized":"2024-04-25T10:00:33.091751+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:11.688215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085178+0000","last_change":"2024-04-25T10:00:33.076667+0000","last_active":"2024-04-25T10:00:33.085178+0000","last_peered":"2024-04-25T10:00:33.085178+0000","last_clean":"2024-04-25T10:00:33.085178+0000","last_became_active":"2024-04-25T10:00:33.076382+0000","last_became_peered":"2024-04-25T10:00:33.076382+0000","last_unstale":"2024-04-25T10:00:33.085178+0000","last_undegraded":"2024-04-25T10:00:33.085178+0000","last_fullsized":"2024-04-25T10:00:33.085178+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:23.032578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086029+0000","last_change":"2024-04-25T10:00:33.081953+0000","last_active":"2024-04-25T10:00:33.086029+0000","last_peered":"2024-04-25T10:00:33.086029+0000","last_clean":"2024-04-25T10:00:33.086029+0000","last_became_active":"2024-04-25T10:00:33.081683+0000","last_became_peered":"2024-04-25T10:00:33.081683+0000","last_unstale":"2024-04-25T10:00:33.086029+0000","last_undegraded":"2024-04-25T10:00:33.086029+0000","last_fullsized":"2024-04-25T10:00:33.086029+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:38.220436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.077998+0000","last_change":"2024-04-25T10:00:33.075098+0000","last_active":"2024-04-25T10:00:33.077998+0000","last_peered":"2024-04-25T10:00:33.077998+0000","last_clean":"2024-04-25T10:00:33.077998+0000","last_became_active":"2024-04-25T10:00:33.074811+0000","last_became_peered":"2024-04-25T10:00:33.074811+0000","last_unstale":"2024-04-25T10:00:33.077998+0000","last_undegraded":"2024-04-25T10:00:33.077998+0000","last_fullsized":"2024-04-25T10:00:33.077998+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:49.118519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085160+0000","last_change":"2024-04-25T10:00:33.077341+0000","last_active":"2024-04-25T10:00:33.085160+0000","last_peered":"2024-04-25T10:00:33.085160+0000","last_clean":"2024-04-25T10:00:33.085160+0000","last_became_active":"2024-04-25T10:00:33.077069+0000","last_became_peered":"2024-04-25T10:00:33.077069+0000","last_unstale":"2024-04-25T10:00:33.085160+0000","last_undegraded":"2024-04-25T10:00:33.085160+0000","last_fullsized":"2024-04-25T10:00:33.085160+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.768887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.079752+0000","last_change":"2024-04-25T10:00:33.075175+0000","last_active":"2024-04-25T10:00:33.079752+0000","last_peered":"2024-04-25T10:00:33.079752+0000","last_clean":"2024-04-25T10:00:33.079752+0000","last_became_active":"2024-04-25T10:00:33.074841+0000","last_became_peered":"2024-04-25T10:00:33.074841+0000","last_unstale":"2024-04-25T10:00:33.079752+0000","last_undegraded":"2024-04-25T10:00:33.079752+0000","last_fullsized":"2024-04-25T10:00:33.079752+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:11.208934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093406+0000","last_change":"2024-04-25T10:00:33.089468+0000","last_active":"2024-04-25T10:00:33.093406+0000","last_peered":"2024-04-25T10:00:33.093406+0000","last_clean":"2024-04-25T10:00:33.093406+0000","last_became_active":"2024-04-25T10:00:33.089177+0000","last_became_peered":"2024-04-25T10:00:33.089177+0000","last_unstale":"2024-04-25T10:00:33.093406+0000","last_undegraded":"2024-04-25T10:00:33.093406+0000","last_fullsized":"2024-04-25T10:00:33.093406+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:58.111467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090803+0000","last_change":"2024-04-25T10:00:33.086644+0000","last_active":"2024-04-25T10:00:33.090803+0000","last_peered":"2024-04-25T10:00:33.090803+0000","last_clean":"2024-04-25T10:00:33.090803+0000","last_became_active":"2024-04-25T10:00:33.086379+0000","last_became_peered":"2024-04-25T10:00:33.086379+0000","last_unstale":"2024-04-25T10:00:33.090803+0000","last_undegraded":"2024-04-25T10:00:33.090803+0000","last_fullsized":"2024-04-25T10:00:33.090803+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:35.272876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093032+0000","last_change":"2024-04-25T10:00:33.088002+0000","last_active":"2024-04-25T10:00:33.093032+0000","last_peered":"2024-04-25T10:00:33.093032+0000","last_clean":"2024-04-25T10:00:33.093032+0000","last_became_active":"2024-04-25T10:00:33.087609+0000","last_became_peered":"2024-04-25T10:00:33.087609+0000","last_unstale":"2024-04-25T10:00:33.093032+0000","last_undegraded":"2024-04-25T10:00:33.093032+0000","last_fullsized":"2024-04-25T10:00:33.093032+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:30.045573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.083043+0000","last_change":"2024-04-25T10:00:33.081210+0000","last_active":"2024-04-25T10:00:33.083043+0000","last_peered":"2024-04-25T10:00:33.083043+0000","last_clean":"2024-04-25T10:00:33.083043+0000","last_became_active":"2024-04-25T10:00:33.080936+0000","last_became_peered":"2024-04-25T10:00:33.080936+0000","last_unstale":"2024-04-25T10:00:33.083043+0000","last_undegraded":"2024-04-25T10:00:33.083043+0000","last_fullsized":"2024-04-25T10:00:33.083043+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:23.272207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.084270+0000","last_change":"2024-04-25T10:00:33.078825+0000","last_active":"2024-04-25T10:00:33.084270+0000","last_peered":"2024-04-25T10:00:33.084270+0000","last_clean":"2024-04-25T10:00:33.084270+0000","last_became_active":"2024-04-25T10:00:33.078494+0000","last_became_peered":"2024-04-25T10:00:33.078494+0000","last_unstale":"2024-04-25T10:00:33.084270+0000","last_undegraded":"2024-04-25T10:00:33.084270+0000","last_fullsized":"2024-04-25T10:00:33.084270+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:10.756822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092830+0000","last_change":"2024-04-25T10:00:33.090807+0000","last_active":"2024-04-25T10:00:33.092830+0000","last_peered":"2024-04-25T10:00:33.092830+0000","last_clean":"2024-04-25T10:00:33.092830+0000","last_became_active":"2024-04-25T10:00:33.089480+0000","last_became_peered":"2024-04-25T10:00:33.089480+0000","last_unstale":"2024-04-25T10:00:33.092830+0000","last_undegraded":"2024-04-25T10:00:33.092830+0000","last_fullsized":"2024-04-25T10:00:33.092830+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:35.704465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.078898+0000","last_change":"2024-04-25T10:00:33.075447+0000","last_active":"2024-04-25T10:00:33.078898+0000","last_peered":"2024-04-25T10:00:33.078898+0000","last_clean":"2024-04-25T10:00:33.078898+0000","last_became_active":"2024-04-25T10:00:33.072693+0000","last_became_peered":"2024-04-25T10:00:33.072693+0000","last_unstale":"2024-04-25T10:00:33.078898+0000","last_undegraded":"2024-04-25T10:00:33.078898+0000","last_fullsized":"2024-04-25T10:00:33.078898+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:58.479858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.078271+0000","last_change":"2024-04-25T10:00:33.075096+0000","last_active":"2024-04-25T10:00:33.078271+0000","last_peered":"2024-04-25T10:00:33.078271+0000","last_clean":"2024-04-25T10:00:33.078271+0000","last_became_active":"2024-04-25T10:00:33.074800+0000","last_became_peered":"2024-04-25T10:00:33.074800+0000","last_unstale":"2024-04-25T10:00:33.078271+0000","last_undegraded":"2024-04-25T10:00:33.078271+0000","last_fullsized":"2024-04-25T10:00:33.078271+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:28.307481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.087681+0000","last_change":"2024-04-25T10:00:33.084268+0000","last_active":"2024-04-25T10:00:33.087681+0000","last_peered":"2024-04-25T10:00:33.087681+0000","last_clean":"2024-04-25T10:00:33.087681+0000","last_became_active":"2024-04-25T10:00:33.083973+0000","last_became_peered":"2024-04-25T10:00:33.083973+0000","last_unstale":"2024-04-25T10:00:33.087681+0000","last_undegraded":"2024-04-25T10:00:33.087681+0000","last_fullsized":"2024-04-25T10:00:33.087681+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:10.756822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090823+0000","last_change":"2024-04-25T10:00:33.087052+0000","last_active":"2024-04-25T10:00:33.090823+0000","last_peered":"2024-04-25T10:00:33.090823+0000","last_clean":"2024-04-25T10:00:33.090823+0000","last_became_active":"2024-04-25T10:00:33.086106+0000","last_became_peered":"2024-04-25T10:00:33.086106+0000","last_unstale":"2024-04-25T10:00:33.090823+0000","last_undegraded":"2024-04-25T10:00:33.090823+0000","last_fullsized":"2024-04-25T10:00:33.090823+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:38.220436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.095834+0000","last_change":"2024-04-25T10:00:33.094838+0000","last_active":"2024-04-25T10:00:33.095834+0000","last_peered":"2024-04-25T10:00:33.095834+0000","last_clean":"2024-04-25T10:00:33.095834+0000","last_became_active":"2024-04-25T10:00:33.094553+0000","last_became_peered":"2024-04-25T10:00:33.094553+0000","last_unstale":"2024-04-25T10:00:33.095834+0000","last_undegraded":"2024-04-25T10:00:33.095834+0000","last_fullsized":"2024-04-25T10:00:33.095834+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:38.220436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"116'4","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.098660+0000","last_change":"2024-04-25T10:00:32.058752+0000","last_active":"2024-04-25T10:00:33.098660+0000","last_peered":"2024-04-25T10:00:33.098660+0000","last_clean":"2024-04-25T10:00:33.098660+0000","last_became_active":"2024-04-25T10:00:32.058483+0000","last_became_peered":"2024-04-25T10:00:32.058483+0000","last_unstale":"2024-04-25T10:00:33.098660+0000","last_undegraded":"2024-04-25T10:00:33.098660+0000","last_fullsized":"2024-04-25T10:00:33.098660+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:45.880926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092642+0000","last_change":"2024-04-25T10:00:33.075965+0000","last_active":"2024-04-25T10:00:33.092642+0000","last_peered":"2024-04-25T10:00:33.092642+0000","last_clean":"2024-04-25T10:00:33.092642+0000","last_became_active":"2024-04-25T10:00:33.075664+0000","last_became_peered":"2024-04-25T10:00:33.075664+0000","last_unstale":"2024-04-25T10:00:33.092642+0000","last_undegraded":"2024-04-25T10:00:33.092642+0000","last_fullsized":"2024-04-25T10:00:33.092642+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:05.969814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"116'1","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.084440+0000","last_change":"2024-04-25T10:00:32.072195+0000","last_active":"2024-04-25T10:00:33.084440+0000","last_peered":"2024-04-25T10:00:33.084440+0000","last_clean":"2024-04-25T10:00:33.084440+0000","last_became_active":"2024-04-25T10:00:32.071408+0000","last_became_peered":"2024-04-25T10:00:32.071408+0000","last_unstale":"2024-04-25T10:00:33.084440+0000","last_undegraded":"2024-04-25T10:00:33.084440+0000","last_fullsized":"2024-04-25T10:00:33.084440+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:25.802978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.098474+0000","last_change":"2024-04-25T10:00:33.095042+0000","last_active":"2024-04-25T10:00:33.098474+0000","last_peered":"2024-04-25T10:00:33.098474+0000","last_clean":"2024-04-25T10:00:33.098474+0000","last_became_active":"2024-04-25T10:00:33.094806+0000","last_became_peered":"2024-04-25T10:00:33.094806+0000","last_unstale":"2024-04-25T10:00:33.098474+0000","last_undegraded":"2024-04-25T10:00:33.098474+0000","last_fullsized":"2024-04-25T10:00:33.098474+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:34.575394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.095853+0000","last_change":"2024-04-25T10:00:32.058554+0000","last_active":"2024-04-25T10:00:33.095853+0000","last_peered":"2024-04-25T10:00:33.095853+0000","last_clean":"2024-04-25T10:00:33.095853+0000","last_became_active":"2024-04-25T10:00:32.058248+0000","last_became_peered":"2024-04-25T10:00:32.058248+0000","last_unstale":"2024-04-25T10:00:33.095853+0000","last_undegraded":"2024-04-25T10:00:33.095853+0000","last_fullsized":"2024-04-25T10:00:33.095853+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:33.567737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085097+0000","last_change":"2024-04-25T10:00:33.077798+0000","last_active":"2024-04-25T10:00:33.085097+0000","last_peered":"2024-04-25T10:00:33.085097+0000","last_clean":"2024-04-25T10:00:33.085097+0000","last_became_active":"2024-04-25T10:00:33.077515+0000","last_became_peered":"2024-04-25T10:00:33.077515+0000","last_unstale":"2024-04-25T10:00:33.085097+0000","last_undegraded":"2024-04-25T10:00:33.085097+0000","last_fullsized":"2024-04-25T10:00:33.085097+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:55.527525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092420+0000","last_change":"2024-04-25T10:00:32.062931+0000","last_active":"2024-04-25T10:00:33.092420+0000","last_peered":"2024-04-25T10:00:33.092420+0000","last_clean":"2024-04-25T10:00:33.092420+0000","last_became_active":"2024-04-25T10:00:32.062624+0000","last_became_peered":"2024-04-25T10:00:32.062624+0000","last_unstale":"2024-04-25T10:00:33.092420+0000","last_undegraded":"2024-04-25T10:00:33.092420+0000","last_fullsized":"2024-04-25T10:00:33.092420+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:39.658994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086905+0000","last_change":"2024-04-25T10:00:33.085492+0000","last_active":"2024-04-25T10:00:33.086905+0000","last_peered":"2024-04-25T10:00:33.086905+0000","last_clean":"2024-04-25T10:00:33.086905+0000","last_became_active":"2024-04-25T10:00:33.085176+0000","last_became_peered":"2024-04-25T10:00:33.085176+0000","last_unstale":"2024-04-25T10:00:33.086905+0000","last_undegraded":"2024-04-25T10:00:33.086905+0000","last_fullsized":"2024-04-25T10:00:33.086905+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:46.888583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"116'1","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.096098+0000","last_change":"2024-04-25T10:00:32.062480+0000","last_active":"2024-04-25T10:00:33.096098+0000","last_peered":"2024-04-25T10:00:33.096098+0000","last_clean":"2024-04-25T10:00:33.096098+0000","last_became_active":"2024-04-25T10:00:32.062196+0000","last_became_peered":"2024-04-25T10:00:32.062196+0000","last_unstale":"2024-04-25T10:00:33.096098+0000","last_undegraded":"2024-04-25T10:00:33.096098+0000","last_fullsized":"2024-04-25T10:00:33.096098+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:51.453678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.080731+0000","last_change":"2024-04-25T10:00:33.074895+0000","last_active":"2024-04-25T10:00:33.080731+0000","last_peered":"2024-04-25T10:00:33.080731+0000","last_clean":"2024-04-25T10:00:33.080731+0000","last_became_active":"2024-04-25T10:00:33.074581+0000","last_became_peered":"2024-04-25T10:00:33.074581+0000","last_unstale":"2024-04-25T10:00:33.080731+0000","last_undegraded":"2024-04-25T10:00:33.080731+0000","last_fullsized":"2024-04-25T10:00:33.080731+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:26.810634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"116'1","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093636+0000","last_change":"2024-04-25T10:00:32.068681+0000","last_active":"2024-04-25T10:00:33.093636+0000","last_peered":"2024-04-25T10:00:33.093636+0000","last_clean":"2024-04-25T10:00:33.093636+0000","last_became_active":"2024-04-25T10:00:32.068230+0000","last_became_peered":"2024-04-25T10:00:32.068230+0000","last_unstale":"2024-04-25T10:00:33.093636+0000","last_undegraded":"2024-04-25T10:00:33.093636+0000","last_fullsized":"2024-04-25T10:00:33.093636+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:39.660320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.087653+0000","last_change":"2024-04-25T10:00:33.084125+0000","last_active":"2024-04-25T10:00:33.087653+0000","last_peered":"2024-04-25T10:00:33.087653+0000","last_clean":"2024-04-25T10:00:33.087653+0000","last_became_active":"2024-04-25T10:00:33.083839+0000","last_became_peered":"2024-04-25T10:00:33.083839+0000","last_unstale":"2024-04-25T10:00:33.087653+0000","last_undegraded":"2024-04-25T10:00:33.087653+0000","last_fullsized":"2024-04-25T10:00:33.087653+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:25.995898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.077403+0000","last_change":"2024-04-25T10:00:32.061863+0000","last_active":"2024-04-25T10:00:33.077403+0000","last_peered":"2024-04-25T10:00:33.077403+0000","last_clean":"2024-04-25T10:00:33.077403+0000","last_became_active":"2024-04-25T10:00:32.061414+0000","last_became_peered":"2024-04-25T10:00:32.061414+0000","last_unstale":"2024-04-25T10:00:33.077403+0000","last_undegraded":"2024-04-25T10:00:33.077403+0000","last_fullsized":"2024-04-25T10:00:33.077403+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:45.880926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093414+0000","last_change":"2024-04-25T10:00:33.092069+0000","last_active":"2024-04-25T10:00:33.093414+0000","last_peered":"2024-04-25T10:00:33.093414+0000","last_clean":"2024-04-25T10:00:33.093414+0000","last_became_active":"2024-04-25T10:00:33.089864+0000","last_became_peered":"2024-04-25T10:00:33.089864+0000","last_unstale":"2024-04-25T10:00:33.093414+0000","last_undegraded":"2024-04-25T10:00:33.093414+0000","last_fullsized":"2024-04-25T10:00:33.093414+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:35.272876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.070482+0000","last_change":"2024-04-25T10:00:32.073711+0000","last_active":"2024-04-25T10:00:33.070482+0000","last_peered":"2024-04-25T10:00:33.070482+0000","last_clean":"2024-04-25T10:00:33.070482+0000","last_became_active":"2024-04-25T10:00:32.073416+0000","last_became_peered":"2024-04-25T10:00:32.073416+0000","last_unstale":"2024-04-25T10:00:33.070482+0000","last_undegraded":"2024-04-25T10:00:33.070482+0000","last_fullsized":"2024-04-25T10:00:33.070482+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:21.757151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085349+0000","last_change":"2024-04-25T10:00:33.081273+0000","last_active":"2024-04-25T10:00:33.085349+0000","last_peered":"2024-04-25T10:00:33.085349+0000","last_clean":"2024-04-25T10:00:33.085349+0000","last_became_active":"2024-04-25T10:00:33.081031+0000","last_became_peered":"2024-04-25T10:00:33.081031+0000","last_unstale":"2024-04-25T10:00:33.085349+0000","last_undegraded":"2024-04-25T10:00:33.085349+0000","last_fullsized":"2024-04-25T10:00:33.085349+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:35.272876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.073730+0000","last_change":"2024-04-25T10:00:32.068048+0000","last_active":"2024-04-25T10:00:33.073730+0000","last_peered":"2024-04-25T10:00:33.073730+0000","last_clean":"2024-04-25T10:00:33.073730+0000","last_became_active":"2024-04-25T10:00:32.067736+0000","last_became_peered":"2024-04-25T10:00:32.067736+0000","last_unstale":"2024-04-25T10:00:33.073730+0000","last_undegraded":"2024-04-25T10:00:33.073730+0000","last_fullsized":"2024-04-25T10:00:33.073730+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:40.503212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093357+0000","last_change":"2024-04-25T10:00:33.089782+0000","last_active":"2024-04-25T10:00:33.093357+0000","last_peered":"2024-04-25T10:00:33.093357+0000","last_clean":"2024-04-25T10:00:33.093357+0000","last_became_active":"2024-04-25T10:00:33.089519+0000","last_became_peered":"2024-04-25T10:00:33.089519+0000","last_unstale":"2024-04-25T10:00:33.093357+0000","last_undegraded":"2024-04-25T10:00:33.093357+0000","last_fullsized":"2024-04-25T10:00:33.093357+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:19.488621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.079505+0000","last_change":"2024-04-25T10:00:32.069176+0000","last_active":"2024-04-25T10:00:33.079505+0000","last_peered":"2024-04-25T10:00:33.079505+0000","last_clean":"2024-04-25T10:00:33.079505+0000","last_became_active":"2024-04-25T10:00:32.068865+0000","last_became_peered":"2024-04-25T10:00:32.068865+0000","last_unstale":"2024-04-25T10:00:33.079505+0000","last_undegraded":"2024-04-25T10:00:33.079505+0000","last_fullsized":"2024-04-25T10:00:33.079505+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:05.307362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086252+0000","last_change":"2024-04-25T10:00:33.079690+0000","last_active":"2024-04-25T10:00:33.086252+0000","last_peered":"2024-04-25T10:00:33.086252+0000","last_clean":"2024-04-25T10:00:33.086252+0000","last_became_active":"2024-04-25T10:00:33.079356+0000","last_became_peered":"2024-04-25T10:00:33.079356+0000","last_unstale":"2024-04-25T10:00:33.086252+0000","last_undegraded":"2024-04-25T10:00:33.086252+0000","last_fullsized":"2024-04-25T10:00:33.086252+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.768887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"116'3","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090965+0000","last_change":"2024-04-25T10:00:32.066394+0000","last_active":"2024-04-25T10:00:33.090965+0000","last_peered":"2024-04-25T10:00:33.090965+0000","last_clean":"2024-04-25T10:00:33.090965+0000","last_became_active":"2024-04-25T10:00:32.066056+0000","last_became_peered":"2024-04-25T10:00:32.066056+0000","last_unstale":"2024-04-25T10:00:33.090965+0000","last_undegraded":"2024-04-25T10:00:33.090965+0000","last_fullsized":"2024-04-25T10:00:33.090965+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:40.503212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090750+0000","last_change":"2024-04-25T10:00:33.086129+0000","last_active":"2024-04-25T10:00:33.090750+0000","last_peered":"2024-04-25T10:00:33.090750+0000","last_clean":"2024-04-25T10:00:33.090750+0000","last_became_active":"2024-04-25T10:00:33.085580+0000","last_became_peered":"2024-04-25T10:00:33.085580+0000","last_unstale":"2024-04-25T10:00:33.090750+0000","last_undegraded":"2024-04-25T10:00:33.090750+0000","last_fullsized":"2024-04-25T10:00:33.090750+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:43.973298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"116'6","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.098564+0000","last_change":"2024-04-25T10:00:32.069720+0000","last_active":"2024-04-25T10:00:33.098564+0000","last_peered":"2024-04-25T10:00:33.098564+0000","last_clean":"2024-04-25T10:00:33.098564+0000","last_became_active":"2024-04-25T10:00:32.069484+0000","last_became_peered":"2024-04-25T10:00:32.069484+0000","last_unstale":"2024-04-25T10:00:33.098564+0000","last_undegraded":"2024-04-25T10:00:33.098564+0000","last_fullsized":"2024-04-25T10:00:33.098564+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:44.619876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.084033+0000","last_change":"2024-04-25T10:00:33.079306+0000","last_active":"2024-04-25T10:00:33.084033+0000","last_peered":"2024-04-25T10:00:33.084033+0000","last_clean":"2024-04-25T10:00:33.084033+0000","last_became_active":"2024-04-25T10:00:33.079057+0000","last_became_peered":"2024-04-25T10:00:33.079057+0000","last_unstale":"2024-04-25T10:00:33.084033+0000","last_undegraded":"2024-04-25T10:00:33.084033+0000","last_fullsized":"2024-04-25T10:00:33.084033+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:43.973298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093811+0000","last_change":"2024-04-25T10:00:32.067975+0000","last_active":"2024-04-25T10:00:33.093811+0000","last_peered":"2024-04-25T10:00:33.093811+0000","last_clean":"2024-04-25T10:00:33.093811+0000","last_became_active":"2024-04-25T10:00:32.067655+0000","last_became_peered":"2024-04-25T10:00:32.067655+0000","last_unstale":"2024-04-25T10:00:33.093811+0000","last_undegraded":"2024-04-25T10:00:33.093811+0000","last_fullsized":"2024-04-25T10:00:33.093811+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:27.904957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093228+0000","last_change":"2024-04-25T10:00:33.089117+0000","last_active":"2024-04-25T10:00:33.093228+0000","last_peered":"2024-04-25T10:00:33.093228+0000","last_clean":"2024-04-25T10:00:33.093228+0000","last_became_active":"2024-04-25T10:00:33.088746+0000","last_became_peered":"2024-04-25T10:00:33.088746+0000","last_unstale":"2024-04-25T10:00:33.093228+0000","last_undegraded":"2024-04-25T10:00:33.093228+0000","last_fullsized":"2024-04-25T10:00:33.093228+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:10.756822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.065860+0000","last_change":"2024-04-25T10:00:32.072530+0000","last_active":"2024-04-25T10:00:33.065860+0000","last_peered":"2024-04-25T10:00:33.065860+0000","last_clean":"2024-04-25T10:00:33.065860+0000","last_became_active":"2024-04-25T10:00:32.072234+0000","last_became_peered":"2024-04-25T10:00:32.072234+0000","last_unstale":"2024-04-25T10:00:33.065860+0000","last_undegraded":"2024-04-25T10:00:33.065860+0000","last_fullsized":"2024-04-25T10:00:33.065860+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:53.154087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.075863+0000","last_change":"2024-04-25T10:00:33.071505+0000","last_active":"2024-04-25T10:00:33.075863+0000","last_peered":"2024-04-25T10:00:33.075863+0000","last_clean":"2024-04-25T10:00:33.075863+0000","last_became_active":"2024-04-25T10:00:33.071242+0000","last_became_peered":"2024-04-25T10:00:33.071242+0000","last_unstale":"2024-04-25T10:00:33.075863+0000","last_undegraded":"2024-04-25T10:00:33.075863+0000","last_fullsized":"2024-04-25T10:00:33.075863+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:00.709999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.065151+0000","last_change":"2024-04-25T10:00:32.058306+0000","last_active":"2024-04-25T10:00:33.065151+0000","last_peered":"2024-04-25T10:00:33.065151+0000","last_clean":"2024-04-25T10:00:33.065151+0000","last_became_active":"2024-04-25T10:00:32.057993+0000","last_became_peered":"2024-04-25T10:00:32.057993+0000","last_unstale":"2024-04-25T10:00:33.065151+0000","last_undegraded":"2024-04-25T10:00:33.065151+0000","last_fullsized":"2024-04-25T10:00:33.065151+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:43.632761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.098453+0000","last_change":"2024-04-25T10:00:33.093924+0000","last_active":"2024-04-25T10:00:33.098453+0000","last_peered":"2024-04-25T10:00:33.098453+0000","last_clean":"2024-04-25T10:00:33.098453+0000","last_became_active":"2024-04-25T10:00:33.093644+0000","last_became_peered":"2024-04-25T10:00:33.093644+0000","last_unstale":"2024-04-25T10:00:33.098453+0000","last_undegraded":"2024-04-25T10:00:33.098453+0000","last_fullsized":"2024-04-25T10:00:33.098453+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:44.640418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"116'1","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.094350+0000","last_change":"2024-04-25T10:00:32.070024+0000","last_active":"2024-04-25T10:00:33.094350+0000","last_peered":"2024-04-25T10:00:33.094350+0000","last_clean":"2024-04-25T10:00:33.094350+0000","last_became_active":"2024-04-25T10:00:32.069723+0000","last_became_peered":"2024-04-25T10:00:32.069723+0000","last_unstale":"2024-04-25T10:00:33.094350+0000","last_undegraded":"2024-04-25T10:00:33.094350+0000","last_fullsized":"2024-04-25T10:00:33.094350+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:10.201277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.082487+0000","last_change":"2024-04-25T10:00:33.075187+0000","last_active":"2024-04-25T10:00:33.082487+0000","last_peered":"2024-04-25T10:00:33.082487+0000","last_clean":"2024-04-25T10:00:33.082487+0000","last_became_active":"2024-04-25T10:00:33.074899+0000","last_became_peered":"2024-04-25T10:00:33.074899+0000","last_unstale":"2024-04-25T10:00:33.082487+0000","last_undegraded":"2024-04-25T10:00:33.082487+0000","last_fullsized":"2024-04-25T10:00:33.082487+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:40.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.095546+0000","last_change":"2024-04-25T10:00:32.068913+0000","last_active":"2024-04-25T10:00:33.095546+0000","last_peered":"2024-04-25T10:00:33.095546+0000","last_clean":"2024-04-25T10:00:33.095546+0000","last_became_active":"2024-04-25T10:00:32.068618+0000","last_became_peered":"2024-04-25T10:00:32.068618+0000","last_unstale":"2024-04-25T10:00:33.095546+0000","last_undegraded":"2024-04-25T10:00:33.095546+0000","last_fullsized":"2024-04-25T10:00:33.095546+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:33.434267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092646+0000","last_change":"2024-04-25T10:00:33.082469+0000","last_active":"2024-04-25T10:00:33.092646+0000","last_peered":"2024-04-25T10:00:33.092646+0000","last_clean":"2024-04-25T10:00:33.092646+0000","last_became_active":"2024-04-25T10:00:33.079407+0000","last_became_peered":"2024-04-25T10:00:33.079407+0000","last_unstale":"2024-04-25T10:00:33.092646+0000","last_undegraded":"2024-04-25T10:00:33.092646+0000","last_fullsized":"2024-04-25T10:00:33.092646+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:52.403120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"116'1","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.096083+0000","last_change":"2024-04-25T10:00:32.065456+0000","last_active":"2024-04-25T10:00:33.096083+0000","last_peered":"2024-04-25T10:00:33.096083+0000","last_clean":"2024-04-25T10:00:33.096083+0000","last_became_active":"2024-04-25T10:00:32.065176+0000","last_became_peered":"2024-04-25T10:00:32.065176+0000","last_unstale":"2024-04-25T10:00:33.096083+0000","last_undegraded":"2024-04-25T10:00:33.096083+0000","last_fullsized":"2024-04-25T10:00:33.096083+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:27.299824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.093339+0000","last_change":"2024-04-25T10:00:33.088628+0000","last_active":"2024-04-25T10:00:33.093339+0000","last_peered":"2024-04-25T10:00:33.093339+0000","last_clean":"2024-04-25T10:00:33.093339+0000","last_became_active":"2024-04-25T10:00:33.088330+0000","last_became_peered":"2024-04-25T10:00:33.088330+0000","last_unstale":"2024-04-25T10:00:33.093339+0000","last_undegraded":"2024-04-25T10:00:33.093339+0000","last_fullsized":"2024-04-25T10:00:33.093339+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:28.912614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.067537+0000","last_change":"2024-04-25T10:00:32.066399+0000","last_active":"2024-04-25T10:00:33.067537+0000","last_peered":"2024-04-25T10:00:33.067537+0000","last_clean":"2024-04-25T10:00:33.067537+0000","last_became_active":"2024-04-25T10:00:32.065919+0000","last_became_peered":"2024-04-25T10:00:32.065919+0000","last_unstale":"2024-04-25T10:00:33.067537+0000","last_undegraded":"2024-04-25T10:00:33.067537+0000","last_fullsized":"2024-04-25T10:00:33.067537+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:18.480965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092912+0000","last_change":"2024-04-25T10:00:33.089653+0000","last_active":"2024-04-25T10:00:33.092912+0000","last_peered":"2024-04-25T10:00:33.092912+0000","last_clean":"2024-04-25T10:00:33.092912+0000","last_became_active":"2024-04-25T10:00:33.089322+0000","last_became_peered":"2024-04-25T10:00:33.089322+0000","last_unstale":"2024-04-25T10:00:33.092912+0000","last_undegraded":"2024-04-25T10:00:33.092912+0000","last_fullsized":"2024-04-25T10:00:33.092912+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:48.899768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092983+0000","last_change":"2024-04-25T10:00:32.067452+0000","last_active":"2024-04-25T10:00:33.092983+0000","last_peered":"2024-04-25T10:00:33.092983+0000","last_clean":"2024-04-25T10:00:33.092983+0000","last_became_active":"2024-04-25T10:00:32.067032+0000","last_became_peered":"2024-04-25T10:00:32.067032+0000","last_unstale":"2024-04-25T10:00:33.092983+0000","last_undegraded":"2024-04-25T10:00:33.092983+0000","last_fullsized":"2024-04-25T10:00:33.092983+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:16.507017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.087274+0000","last_change":"2024-04-25T10:00:33.083871+0000","last_active":"2024-04-25T10:00:33.087274+0000","last_peered":"2024-04-25T10:00:33.087274+0000","last_clean":"2024-04-25T10:00:33.087274+0000","last_became_active":"2024-04-25T10:00:33.081695+0000","last_became_peered":"2024-04-25T10:00:33.081695+0000","last_unstale":"2024-04-25T10:00:33.087274+0000","last_undegraded":"2024-04-25T10:00:33.087274+0000","last_fullsized":"2024-04-25T10:00:33.087274+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:14.717224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.068658+0000","last_change":"2024-04-25T10:00:32.058582+0000","last_active":"2024-04-25T10:00:33.068658+0000","last_peered":"2024-04-25T10:00:33.068658+0000","last_clean":"2024-04-25T10:00:33.068658+0000","last_became_active":"2024-04-25T10:00:32.058302+0000","last_became_peered":"2024-04-25T10:00:32.058302+0000","last_unstale":"2024-04-25T10:00:33.068658+0000","last_undegraded":"2024-04-25T10:00:33.068658+0000","last_fullsized":"2024-04-25T10:00:33.068658+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:44.619876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086301+0000","last_change":"2024-04-25T10:00:33.080516+0000","last_active":"2024-04-25T10:00:33.086301+0000","last_peered":"2024-04-25T10:00:33.086301+0000","last_clean":"2024-04-25T10:00:33.086301+0000","last_became_active":"2024-04-25T10:00:33.080135+0000","last_became_peered":"2024-04-25T10:00:33.080135+0000","last_unstale":"2024-04-25T10:00:33.086301+0000","last_undegraded":"2024-04-25T10:00:33.086301+0000","last_fullsized":"2024-04-25T10:00:33.086301+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:55.527525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.065344+0000","last_change":"2024-04-25T10:00:32.069188+0000","last_active":"2024-04-25T10:00:33.065344+0000","last_peered":"2024-04-25T10:00:33.065344+0000","last_clean":"2024-04-25T10:00:33.065344+0000","last_became_active":"2024-04-25T10:00:32.068888+0000","last_became_peered":"2024-04-25T10:00:32.068888+0000","last_unstale":"2024-04-25T10:00:33.065344+0000","last_undegraded":"2024-04-25T10:00:33.065344+0000","last_fullsized":"2024-04-25T10:00:33.065344+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:33.567737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085291+0000","last_change":"2024-04-25T10:00:33.081845+0000","last_active":"2024-04-25T10:00:33.085291+0000","last_peered":"2024-04-25T10:00:33.085291+0000","last_clean":"2024-04-25T10:00:33.085291+0000","last_became_active":"2024-04-25T10:00:33.081595+0000","last_became_peered":"2024-04-25T10:00:33.081595+0000","last_unstale":"2024-04-25T10:00:33.085291+0000","last_undegraded":"2024-04-25T10:00:33.085291+0000","last_fullsized":"2024-04-25T10:00:33.085291+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:11.688215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.067754+0000","last_change":"2024-04-25T10:00:32.059011+0000","last_active":"2024-04-25T10:00:33.067754+0000","last_peered":"2024-04-25T10:00:33.067754+0000","last_clean":"2024-04-25T10:00:33.067754+0000","last_became_active":"2024-04-25T10:00:32.058741+0000","last_became_peered":"2024-04-25T10:00:32.058741+0000","last_unstale":"2024-04-25T10:00:33.067754+0000","last_undegraded":"2024-04-25T10:00:33.067754+0000","last_fullsized":"2024-04-25T10:00:33.067754+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:43.632761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092853+0000","last_change":"2024-04-25T10:00:33.091014+0000","last_active":"2024-04-25T10:00:33.092853+0000","last_peered":"2024-04-25T10:00:33.092853+0000","last_clean":"2024-04-25T10:00:33.092853+0000","last_became_active":"2024-04-25T10:00:33.089860+0000","last_became_peered":"2024-04-25T10:00:33.089860+0000","last_unstale":"2024-04-25T10:00:33.092853+0000","last_undegraded":"2024-04-25T10:00:33.092853+0000","last_fullsized":"2024-04-25T10:00:33.092853+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:23.776136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"116'2","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.092837+0000","last_change":"2024-04-25T10:00:32.065946+0000","last_active":"2024-04-25T10:00:33.092837+0000","last_peered":"2024-04-25T10:00:33.092837+0000","last_clean":"2024-04-25T10:00:33.092837+0000","last_became_active":"2024-04-25T10:00:32.065650+0000","last_became_peered":"2024-04-25T10:00:32.065650+0000","last_unstale":"2024-04-25T10:00:33.092837+0000","last_undegraded":"2024-04-25T10:00:33.092837+0000","last_fullsized":"2024-04-25T10:00:33.092837+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:26.788956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.089278+0000","last_change":"2024-04-25T10:00:33.086393+0000","last_active":"2024-04-25T10:00:33.089278+0000","last_peered":"2024-04-25T10:00:33.089278+0000","last_clean":"2024-04-25T10:00:33.089278+0000","last_became_active":"2024-04-25T10:00:33.086049+0000","last_became_peered":"2024-04-25T10:00:33.086049+0000","last_unstale":"2024-04-25T10:00:33.089278+0000","last_undegraded":"2024-04-25T10:00:33.089278+0000","last_fullsized":"2024-04-25T10:00:33.089278+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:41.510869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.073688+0000","last_change":"2024-04-25T10:00:32.068064+0000","last_active":"2024-04-25T10:00:33.073688+0000","last_peered":"2024-04-25T10:00:33.073688+0000","last_clean":"2024-04-25T10:00:33.073688+0000","last_became_active":"2024-04-25T10:00:32.067687+0000","last_became_peered":"2024-04-25T10:00:32.067687+0000","last_unstale":"2024-04-25T10:00:33.073688+0000","last_undegraded":"2024-04-25T10:00:33.073688+0000","last_fullsized":"2024-04-25T10:00:33.073688+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:05.307362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.074099+0000","last_change":"2024-04-25T10:00:33.072598+0000","last_active":"2024-04-25T10:00:33.074099+0000","last_peered":"2024-04-25T10:00:33.074099+0000","last_clean":"2024-04-25T10:00:33.074099+0000","last_became_active":"2024-04-25T10:00:33.072156+0000","last_became_peered":"2024-04-25T10:00:33.072156+0000","last_unstale":"2024-04-25T10:00:33.074099+0000","last_undegraded":"2024-04-25T10:00:33.074099+0000","last_fullsized":"2024-04-25T10:00:33.074099+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:23.032578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.073909+0000","last_change":"2024-04-25T10:00:32.068129+0000","last_active":"2024-04-25T10:00:33.073909+0000","last_peered":"2024-04-25T10:00:33.073909+0000","last_clean":"2024-04-25T10:00:33.073909+0000","last_became_active":"2024-04-25T10:00:32.067680+0000","last_became_peered":"2024-04-25T10:00:32.067680+0000","last_unstale":"2024-04-25T10:00:33.073909+0000","last_undegraded":"2024-04-25T10:00:33.073909+0000","last_fullsized":"2024-04-25T10:00:33.073909+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.680558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090032+0000","last_change":"2024-04-25T10:00:33.084569+0000","last_active":"2024-04-25T10:00:33.090032+0000","last_peered":"2024-04-25T10:00:33.090032+0000","last_clean":"2024-04-25T10:00:33.090032+0000","last_became_active":"2024-04-25T10:00:33.083734+0000","last_became_peered":"2024-04-25T10:00:33.083734+0000","last_unstale":"2024-04-25T10:00:33.090032+0000","last_undegraded":"2024-04-25T10:00:33.090032+0000","last_fullsized":"2024-04-25T10:00:33.090032+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:22.651062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.072898+0000","last_change":"2024-04-25T10:00:32.074426+0000","last_active":"2024-04-25T10:00:33.072898+0000","last_peered":"2024-04-25T10:00:33.072898+0000","last_clean":"2024-04-25T10:00:33.072898+0000","last_became_active":"2024-04-25T10:00:32.074127+0000","last_became_peered":"2024-04-25T10:00:32.074127+0000","last_unstale":"2024-04-25T10:00:33.072898+0000","last_undegraded":"2024-04-25T10:00:33.072898+0000","last_fullsized":"2024-04-25T10:00:33.072898+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:21.454244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.077417+0000","last_change":"2024-04-25T10:00:33.073354+0000","last_active":"2024-04-25T10:00:33.077417+0000","last_peered":"2024-04-25T10:00:33.077417+0000","last_clean":"2024-04-25T10:00:33.077417+0000","last_became_active":"2024-04-25T10:00:33.072772+0000","last_became_peered":"2024-04-25T10:00:33.072772+0000","last_unstale":"2024-04-25T10:00:33.077417+0000","last_undegraded":"2024-04-25T10:00:33.077417+0000","last_fullsized":"2024-04-25T10:00:33.077417+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:47.544864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.067251+0000","last_change":"2024-04-25T10:00:32.053556+0000","last_active":"2024-04-25T10:00:33.067251+0000","last_peered":"2024-04-25T10:00:33.067251+0000","last_clean":"2024-04-25T10:00:33.067251+0000","last_became_active":"2024-04-25T10:00:32.053281+0000","last_became_peered":"2024-04-25T10:00:32.053281+0000","last_unstale":"2024-04-25T10:00:33.067251+0000","last_undegraded":"2024-04-25T10:00:33.067251+0000","last_fullsized":"2024-04-25T10:00:33.067251+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:33.329170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.086301+0000","last_change":"2024-04-25T10:00:33.079422+0000","last_active":"2024-04-25T10:00:33.086301+0000","last_peered":"2024-04-25T10:00:33.086301+0000","last_clean":"2024-04-25T10:00:33.086301+0000","last_became_active":"2024-04-25T10:00:33.079123+0000","last_became_peered":"2024-04-25T10:00:33.079123+0000","last_unstale":"2024-04-25T10:00:33.086301+0000","last_undegraded":"2024-04-25T10:00:33.086301+0000","last_fullsized":"2024-04-25T10:00:33.086301+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:07.591236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.068149+0000","last_change":"2024-04-25T10:00:32.063538+0000","last_active":"2024-04-25T10:00:33.068149+0000","last_peered":"2024-04-25T10:00:33.068149+0000","last_clean":"2024-04-25T10:00:33.068149+0000","last_became_active":"2024-04-25T10:00:32.063246+0000","last_became_peered":"2024-04-25T10:00:32.063246+0000","last_unstale":"2024-04-25T10:00:33.068149+0000","last_undegraded":"2024-04-25T10:00:33.068149+0000","last_fullsized":"2024-04-25T10:00:33.068149+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:50.024135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.074715+0000","last_change":"2024-04-25T10:00:33.074039+0000","last_active":"2024-04-25T10:00:33.074715+0000","last_peered":"2024-04-25T10:00:33.074715+0000","last_clean":"2024-04-25T10:00:33.074715+0000","last_became_active":"2024-04-25T10:00:33.073730+0000","last_became_peered":"2024-04-25T10:00:33.073730+0000","last_unstale":"2024-04-25T10:00:33.074715+0000","last_undegraded":"2024-04-25T10:00:33.074715+0000","last_fullsized":"2024-04-25T10:00:33.074715+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:07.591236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.069616+0000","last_change":"2024-04-25T10:00:32.073732+0000","last_active":"2024-04-25T10:00:33.069616+0000","last_peered":"2024-04-25T10:00:33.069616+0000","last_clean":"2024-04-25T10:00:33.069616+0000","last_became_active":"2024-04-25T10:00:32.073432+0000","last_became_peered":"2024-04-25T10:00:32.073432+0000","last_unstale":"2024-04-25T10:00:33.069616+0000","last_undegraded":"2024-04-25T10:00:33.069616+0000","last_fullsized":"2024-04-25T10:00:33.069616+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:18.195763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.090389+0000","last_change":"2024-04-25T10:00:33.086386+0000","last_active":"2024-04-25T10:00:33.090389+0000","last_peered":"2024-04-25T10:00:33.090389+0000","last_clean":"2024-04-25T10:00:33.090389+0000","last_became_active":"2024-04-25T10:00:33.086010+0000","last_became_peered":"2024-04-25T10:00:33.086010+0000","last_unstale":"2024-04-25T10:00:33.090389+0000","last_undegraded":"2024-04-25T10:00:33.090389+0000","last_fullsized":"2024-04-25T10:00:33.090389+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:06.315019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.067913+0000","last_change":"2024-04-25T10:00:32.064444+0000","last_active":"2024-04-25T10:00:33.067913+0000","last_peered":"2024-04-25T10:00:33.067913+0000","last_clean":"2024-04-25T10:00:33.067913+0000","last_became_active":"2024-04-25T10:00:32.064160+0000","last_became_peered":"2024-04-25T10:00:32.064160+0000","last_unstale":"2024-04-25T10:00:33.067913+0000","last_undegraded":"2024-04-25T10:00:33.067913+0000","last_fullsized":"2024-04-25T10:00:33.067913+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:53.154087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.085180+0000","last_change":"2024-04-25T10:00:33.078274+0000","last_active":"2024-04-25T10:00:33.085180+0000","last_peered":"2024-04-25T10:00:33.085180+0000","last_clean":"2024-04-25T10:00:33.085180+0000","last_became_active":"2024-04-25T10:00:33.077677+0000","last_became_peered":"2024-04-25T10:00:33.077677+0000","last_unstale":"2024-04-25T10:00:33.085180+0000","last_undegraded":"2024-04-25T10:00:33.085180+0000","last_fullsized":"2024-04-25T10:00:33.085180+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:32.044576+0000","last_clean_scrub_stamp":"2024-04-25T10:00:32.044576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:15.540434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"116'8","reported_seq":20,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.096480+0000","last_change":"2024-04-25T10:00:32.068349+0000","last_active":"2024-04-25T10:00:33.096480+0000","last_peered":"2024-04-25T10:00:33.096480+0000","last_clean":"2024-04-25T10:00:33.096480+0000","last_became_active":"2024-04-25T10:00:32.068075+0000","last_became_peered":"2024-04-25T10:00:32.068075+0000","last_unstale":"2024-04-25T10:00:33.096480+0000","last_undegraded":"2024-04-25T10:00:33.096480+0000","last_fullsized":"2024-04-25T10:00:33.096480+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:31.036919+0000","last_clean_scrub_stamp":"2024-04-25T10:00:31.036919+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:18.480965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":365,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:00:33.068033+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:00:33.068033+0000","last_peered":"2024-04-25T10:00:33.068033+0000","last_clean":"2024-04-25T10:00:33.068033+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:00:33.068033+0000","last_undegraded":"2024-04-25T10:00:33.068033+0000","last_fullsized":"2024-04-25T10:00:33.068033+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41508,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330332,"statfs":{"total":96636764160,"available":96594259968,"internally_reserved":0,"allocated":2056192,"data_stored":1264604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46484,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325356,"statfs":{"total":96636764160,"available":96589164544,"internally_reserved":0,"allocated":1449984,"data_stored":674042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45088,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326752,"statfs":{"total":96636764160,"available":96590594048,"internally_reserved":0,"allocated":1454080,"data_stored":673620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50660,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321180,"statfs":{"total":96636764160,"available":96584888320,"internally_reserved":0,"allocated":1458176,"data_stored":675072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50640,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1445888,"data_stored":673460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50644,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321196,"statfs":{"total":96636764160,"available":96584904704,"internally_reserved":0,"allocated":1445888,"data_stored":673460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50644,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321196,"statfs":{"total":96636764160,"available":96584904704,"internally_reserved":0,"allocated":1449984,"data_stored":673482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51232,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320608,"statfs":{"total":96636764160,"available":96584302592,"internally_reserved":0,"allocated":2043904,"data_stored":1263850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:00:36.430 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:00:36.430 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:00:36.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.570+0000 7f58a2cac640 1 Processor -- start 2024-04-25T10:00:36.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.570+0000 7f58a2cac640 1 -- start start 2024-04-25T10:00:36.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.570+0000 7f58a2cac640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f589c15dc20 0x7f589c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.570+0000 7f58a2cac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f589c0d32f0 0x7f589c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.570+0000 7f58a2cac640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589c15e000 con 0x7f589c161710 2024-04-25T10:00:36.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.570+0000 7f58a2cac640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589c0d1de0 con 0x7f589c15dc20 2024-04-25T10:00:36.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.570+0000 7f58a2cac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589c161c60 con 0x7f589c0d32f0 2024-04-25T10:00:36.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.571+0000 7f58a1249640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f589c0d32f0 0x7f589c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.571+0000 7f58a0a48640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f589c15dc20 0x7f589c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.571+0000 7f58a0a48640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f589c15dc20 0x7f589c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:46016/0 (socket says 172.21.15.19:46016) 2024-04-25T10:00:36.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.571+0000 7f589bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f589c161710 0x7f589c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:52898/0 (socket says 172.21.15.19:52898) 2024-04-25T10:00:36.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.571+0000 7f58a0a48640 1 -- 172.21.15.19:0/1821971496 learned_addr learned my addr 172.21.15.19:0/1821971496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:36.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.571+0000 7f58a0a48640 1 -- 172.21.15.19:0/1821971496 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f589c0d32f0 msgr2=0x7f589c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:36.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.571+0000 7f58a0a48640 1 --2- 172.21.15.19:0/1821971496 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f589c0d32f0 0x7f589c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.571+0000 7f58a0a48640 1 -- 172.21.15.19:0/1821971496 >> v1:172.21.15.112:6789/0 conn(0x7f589c161710 legacy=0x7f589c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.571+0000 7f58a0a48640 1 -- 172.21.15.19:0/1821971496 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c161f40 con 0x7f589c15dc20 2024-04-25T10:00:36.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.572+0000 7f58a0a48640 1 --2- 172.21.15.19:0/1821971496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f589c15dc20 0x7f589c0ce670 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5890009870 tx=0x7f589002a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acc1983d80e038ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:36.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.572+0000 7f589b7fe640 1 -- 172.21.15.19:0/1821971496 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890037070 con 0x7f589c15dc20 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.572+0000 7f589b7fe640 1 -- 172.21.15.19:0/1821971496 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589002adf0 con 0x7f589c15dc20 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.572+0000 7f589b7fe640 1 -- 172.21.15.19:0/1821971496 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890032660 con 0x7f589c15dc20 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.573+0000 7f58a2cac640 1 -- 172.21.15.19:0/1821971496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f589c15dc20 msgr2=0x7f589c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.573+0000 7f58a2cac640 1 --2- 172.21.15.19:0/1821971496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f589c15dc20 0x7f589c0ce670 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5890009870 tx=0x7f589002a160 comp rx=0 tx=0).stop 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.573+0000 7f58a2cac640 1 -- 172.21.15.19:0/1821971496 shutdown_connections 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.573+0000 7f58a2cac640 1 --2- 172.21.15.19:0/1821971496 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f589c0d32f0 0x7f589c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.573+0000 7f58a2cac640 1 --2- 172.21.15.19:0/1821971496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f589c15dc20 0x7f589c0ce670 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.573+0000 7f58a2cac640 1 -- 172.21.15.19:0/1821971496 >> 172.21.15.19:0/1821971496 conn(0x7f589c0d0420 msgr2=0x7f589c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.573+0000 7f58a2cac640 1 -- 172.21.15.19:0/1821971496 shutdown_connections 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.573+0000 7f58a2cac640 1 -- 172.21.15.19:0/1821971496 wait complete. 2024-04-25T10:00:36.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.574+0000 7f58a2cac640 1 Processor -- start 2024-04-25T10:00:36.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.574+0000 7f58a2cac640 1 -- start start 2024-04-25T10:00:36.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a2cac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f589c0d32f0 0x7f589c0cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a2cac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f589c15dc20 0x7f589c0cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a2cac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f589c161710 0x7f589c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a2cac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c0b5e50 con 0x7f589c161710 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a2cac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c162500 con 0x7f589c0d32f0 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a2cac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589c0b5fd0 con 0x7f589c15dc20 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a1249640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f589c161710 0x7f589c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a1249640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f589c161710 0x7f589c0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44562/0 (socket says 172.21.15.19:44562) 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a1249640 1 -- 172.21.15.19:0/570290843 learned_addr learned my addr 172.21.15.19:0/570290843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a0a48640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f589c0d32f0 0x7f589c0cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f589bfff640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f589c15dc20 0x7f589c0cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a1249640 1 -- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f589c15dc20 msgr2=0x7f589c0cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a1249640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f589c15dc20 0x7f589c0cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a1249640 1 -- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f589c0d32f0 msgr2=0x7f589c0cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a1249640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f589c0d32f0 0x7f589c0cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.575+0000 7f58a1249640 1 -- 172.21.15.19:0/570290843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c15d470 con 0x7f589c161710 2024-04-25T10:00:36.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.576+0000 7f58a1249640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f589c161710 0x7f589c0cbc00 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f589400ec40 tx=0x7f589400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:36.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.576+0000 7f58997fa640 1 -- 172.21.15.19:0/570290843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894018020 con 0x7f589c161710 2024-04-25T10:00:36.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.576+0000 7f58997fa640 1 -- 172.21.15.19:0/570290843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589400cc60 con 0x7f589c161710 2024-04-25T10:00:36.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.576+0000 7f58a2cac640 1 -- 172.21.15.19:0/570290843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589c1e87c0 con 0x7f589c161710 2024-04-25T10:00:36.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.576+0000 7f58997fa640 1 -- 172.21.15.19:0/570290843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894014480 con 0x7f589c161710 2024-04-25T10:00:36.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.576+0000 7f58a2cac640 1 -- 172.21.15.19:0/570290843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589c0cc510 con 0x7f589c161710 2024-04-25T10:00:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.577+0000 7f58a2cac640 1 -- 172.21.15.19:0/570290843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f589c073a00 con 0x7f589c161710 2024-04-25T10:00:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.577+0000 7f58997fa640 1 -- 172.21.15.19:0/570290843 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5894005d10 con 0x7f589c161710 2024-04-25T10:00:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.578+0000 7f58997fa640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588006b4d0 0x7f588006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.578+0000 7f58997fa640 1 -- 172.21.15.19:0/570290843 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f589408ae70 con 0x7f589c161710 2024-04-25T10:00:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.578+0000 7f58a0a48640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588006b4d0 0x7f588006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.578+0000 7f58a0a48640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588006b4d0 0x7f588006d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5890034000 tx=0x7f589002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:36.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.582+0000 7f58997fa640 1 -- 172.21.15.19:0/570290843 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5894022070 con 0x7f589c161710 2024-04-25T10:00:36.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.833+0000 7f58a2cac640 1 -- 172.21.15.19:0/570290843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f589c0ceb30 con 0x7f589c161710 2024-04-25T10:00:36.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.835+0000 7f58997fa640 1 -- 172.21.15.19:0/570290843 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5894010040 con 0x7f589c161710 2024-04-25T10:00:36.839 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T10:00:36.839 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T10:00:36.840 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T10:00:36.840 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T10:00:36.840 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:00:36.840 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T10:00:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 -- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588006b4d0 msgr2=0x7f588006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588006b4d0 0x7f588006d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5890034000 tx=0x7f589002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 -- 172.21.15.19:0/570290843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f589c161710 msgr2=0x7f589c0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f589c161710 0x7f589c0cbc00 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f589400ec40 tx=0x7f589400c490 comp rx=0 tx=0).stop 2024-04-25T10:00:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 -- 172.21.15.19:0/570290843 shutdown_connections 2024-04-25T10:00:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588006b4d0 0x7f588006d990 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f589c161710 0x7f589c0cbc00 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f589c15dc20 0x7f589c0cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 --2- 172.21.15.19:0/570290843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f589c0d32f0 0x7f589c0cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 -- 172.21.15.19:0/570290843 >> 172.21.15.19:0/570290843 conn(0x7f589c0d0420 msgr2=0x7f589c1601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 -- 172.21.15.19:0/570290843 shutdown_connections 2024-04-25T10:00:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.838+0000 7f58a2cac640 1 -- 172.21.15.19:0/570290843 wait complete. 2024-04-25T10:00:36.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:36.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.950+0000 7f037bfb0640 1 Processor -- start 2024-04-25T10:00:36.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.951+0000 7f037bfb0640 1 -- start start 2024-04-25T10:00:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.951+0000 7f037bfb0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03740b2cb0 0x7f03740d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.951+0000 7f037bfb0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f03740d1020 0x7f03740d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.951+0000 7f037bfb0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03740b97a0 con 0x7f037415dbe0 2024-04-25T10:00:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.951+0000 7f037bfb0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f037415dfc0 con 0x7f03740b2cb0 2024-04-25T10:00:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.951+0000 7f037bfb0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03740d1a00 con 0x7f03740d1020 2024-04-25T10:00:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.951+0000 7f037954b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03740b2cb0 0x7f03740d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.951+0000 7f0379d4c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f037415dbe0 0x7f03740d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47096/0 (socket says 172.21.15.19:47096) 2024-04-25T10:00:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.951+0000 7f0379d4c640 1 -- 172.21.15.19:0/1401596063 learned_addr learned my addr 172.21.15.19:0/1401596063 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.951+0000 7f037954b640 1 --2- 172.21.15.19:0/1401596063 >> v2:172.21.15.19:3300/0 conn(0x7f03740b2cb0 0x7f03740d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44566/0 (socket says 172.21.15.19:44566) 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.952+0000 7f037a54d640 1 --2- 172.21.15.19:0/1401596063 >> v2:172.21.15.112:3300/0 conn(0x7f03740d1020 0x7f03740d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.952+0000 7f037954b640 1 -- 172.21.15.19:0/1401596063 >> v2:172.21.15.112:3300/0 conn(0x7f03740d1020 msgr2=0x7f03740d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.952+0000 7f037954b640 1 --2- 172.21.15.19:0/1401596063 >> v2:172.21.15.112:3300/0 conn(0x7f03740d1020 0x7f03740d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.952+0000 7f037954b640 1 -- 172.21.15.19:0/1401596063 >> v1:172.21.15.19:6789/0 conn(0x7f037415dbe0 legacy=0x7f03740d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.952+0000 7f0378d4a640 1 -- 172.21.15.19:0/1401596063 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616970101 0 0) 0x7f03740b97a0 con 0x7f037415dbe0 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.952+0000 7f037954b640 1 -- 172.21.15.19:0/1401596063 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03740d1c50 con 0x7f03740b2cb0 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.952+0000 7f037a54d640 1 --2- 172.21.15.19:0/1401596063 >> v2:172.21.15.112:3300/0 conn(0x7f03740d1020 0x7f03740d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.952+0000 7f037954b640 1 --2- 172.21.15.19:0/1401596063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03740b2cb0 0x7f03740d2730 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f03740cf570 tx=0x7f036402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2784f7287a343f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.953+0000 7f0378d4a640 1 -- 172.21.15.19:0/1401596063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364037070 con 0x7f03740b2cb0 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.953+0000 7f0378d4a640 1 -- 172.21.15.19:0/1401596063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0364032440 con 0x7f03740b2cb0 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.953+0000 7f0378d4a640 1 -- 172.21.15.19:0/1401596063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364032810 con 0x7f03740b2cb0 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.953+0000 7f037bfb0640 1 -- 172.21.15.19:0/1401596063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03740b2cb0 msgr2=0x7f03740d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.953+0000 7f037bfb0640 1 --2- 172.21.15.19:0/1401596063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03740b2cb0 0x7f03740d2730 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f03740cf570 tx=0x7f036402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.953+0000 7f037bfb0640 1 -- 172.21.15.19:0/1401596063 shutdown_connections 2024-04-25T10:00:36.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.953+0000 7f037bfb0640 1 --2- 172.21.15.19:0/1401596063 >> v2:172.21.15.112:3300/0 conn(0x7f03740d1020 0x7f03740d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.953+0000 7f037bfb0640 1 --2- 172.21.15.19:0/1401596063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03740b2cb0 0x7f03740d2730 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.953+0000 7f037bfb0640 1 -- 172.21.15.19:0/1401596063 >> 172.21.15.19:0/1401596063 conn(0x7f0374061a20 msgr2=0x7f0374061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:36.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.953+0000 7f037bfb0640 1 -- 172.21.15.19:0/1401596063 shutdown_connections 2024-04-25T10:00:36.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.953+0000 7f037bfb0640 1 -- 172.21.15.19:0/1401596063 wait complete. 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.954+0000 7f037bfb0640 1 Processor -- start 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.955+0000 7f037bfb0640 1 -- start start 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.955+0000 7f037bfb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03740b2cb0 0x7f03741f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.955+0000 7f037bfb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03740d1020 0x7f03741f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.955+0000 7f037bfb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f037415dbe0 0x7f03740b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.955+0000 7f037bfb0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03740b92a0 con 0x7f03740d1020 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.955+0000 7f037bfb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03740d2210 con 0x7f037415dbe0 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.955+0000 7f037bfb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03740b9420 con 0x7f03740b2cb0 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.955+0000 7f037954b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03740d1020 0x7f03741f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.955+0000 7f037954b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03740d1020 0x7f03741f3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44578/0 (socket says 172.21.15.19:44578) 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.955+0000 7f037954b640 1 -- 172.21.15.19:0/989504904 learned_addr learned my addr 172.21.15.19:0/989504904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f037a54d640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f037415dbe0 0x7f03740b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f037954b640 1 -- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03740b2cb0 msgr2=0x7f03741f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f0379d4c640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03740b2cb0 0x7f03741f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f037954b640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03740b2cb0 0x7f03741f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f037954b640 1 -- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f037415dbe0 msgr2=0x7f03740b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f037954b640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f037415dbe0 0x7f03740b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f037954b640 1 -- 172.21.15.19:0/989504904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03740b37b0 con 0x7f03740d1020 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f037a54d640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f037415dbe0 0x7f03740b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f0379d4c640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03740b2cb0 0x7f03741f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f037954b640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03740d1020 0x7f03741f3190 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f0364000c00 tx=0x7f03640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f036a7fc640 1 -- 172.21.15.19:0/989504904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364037040 con 0x7f03740d1020 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f036a7fc640 1 -- 172.21.15.19:0/989504904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0364042070 con 0x7f03740d1020 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f037bfb0640 1 -- 172.21.15.19:0/989504904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03741f3930 con 0x7f03740d1020 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f036a7fc640 1 -- 172.21.15.19:0/989504904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364030a50 con 0x7f03740d1020 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.956+0000 7f037bfb0640 1 -- 172.21.15.19:0/989504904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03741f3ef0 con 0x7f03740d1020 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.957+0000 7f037bfb0640 1 -- 172.21.15.19:0/989504904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0374073990 con 0x7f03740d1020 2024-04-25T10:00:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.958+0000 7f036a7fc640 1 -- 172.21.15.19:0/989504904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0364030bf0 con 0x7f03740d1020 2024-04-25T10:00:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.958+0000 7f036a7fc640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f035806b4d0 0x7f035806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.958+0000 7f036a7fc640 1 -- 172.21.15.19:0/989504904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03640aae10 con 0x7f03740d1020 2024-04-25T10:00:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.959+0000 7f0379d4c640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f035806b4d0 0x7f035806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.959+0000 7f0379d4c640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f035806b4d0 0x7f035806d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0370000f60 tx=0x7f0370009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:36.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:36.962+0000 7f036a7fc640 1 -- 172.21.15.19:0/989504904 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03640ae030 con 0x7f03740d1020 2024-04-25T10:00:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.180+0000 7f037bfb0640 1 -- 172.21.15.19:0/989504904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03740d27d0 con 0x7f03740d1020 2024-04-25T10:00:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.180+0000 7f036a7fc640 1 -- 172.21.15.19:0/989504904 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0364079c10 con 0x7f03740d1020 2024-04-25T10:00:37.185 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:37.185 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:00:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 -- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f035806b4d0 msgr2=0x7f035806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f035806b4d0 0x7f035806d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0370000f60 tx=0x7f0370009040 comp rx=0 tx=0).stop 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 -- 172.21.15.19:0/989504904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03740d1020 msgr2=0x7f03741f3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03740d1020 0x7f03741f3190 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f0364000c00 tx=0x7f03640046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 -- 172.21.15.19:0/989504904 shutdown_connections 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f035806b4d0 0x7f035806d990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f037415dbe0 0x7f03740b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03740d1020 0x7f03741f3190 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 --2- 172.21.15.19:0/989504904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03740b2cb0 0x7f03741f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 -- 172.21.15.19:0/989504904 >> 172.21.15.19:0/989504904 conn(0x7f0374061a20 msgr2=0x7f0374061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 -- 172.21.15.19:0/989504904 shutdown_connections 2024-04-25T10:00:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.182+0000 7f037bfb0640 1 -- 172.21.15.19:0/989504904 wait complete. 2024-04-25T10:00:37.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:00:32.412582+0000', 'modified': '2024-04-25T10:00:34.423175+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8577}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8577': {'gid': 8577, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6834/3442229953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 3442229953}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 3442229953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:00:37.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:00:37.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:00:37.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.703+0000 7f1ea7ff4640 1 Processor -- start 2024-04-25T10:00:37.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.703+0000 7f1ea7ff4640 1 -- start start 2024-04-25T10:00:37.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea7ff4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1ea015dac0 0x7f1ea00d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea7ff4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ea00d1f30 0x7f1ea00d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea7ff4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea00d2420 con 0x7f1ea00d34c0 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea7ff4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea00d3030 con 0x7f1ea00d1f30 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea7ff4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea00d2700 con 0x7f1ea015dac0 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea558f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ea00d1f30 0x7f1ea00d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea6591640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1ea00d34c0 0x7f1ea00b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57438/0 (socket says 172.21.15.19:57438) 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea6591640 1 -- 172.21.15.19:0/1976794779 learned_addr learned my addr 172.21.15.19:0/1976794779 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/1976794779 >> v2:172.21.15.112:3300/0 conn(0x7f1ea015dac0 0x7f1ea00d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea558f640 1 -- 172.21.15.19:0/1976794779 >> v2:172.21.15.112:3300/0 conn(0x7f1ea015dac0 msgr2=0x7f1ea00d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea558f640 1 --2- 172.21.15.19:0/1976794779 >> v2:172.21.15.112:3300/0 conn(0x7f1ea015dac0 0x7f1ea00d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea558f640 1 -- 172.21.15.19:0/1976794779 >> v1:172.21.15.19:6789/0 conn(0x7f1ea00d34c0 legacy=0x7f1ea00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea4d8e640 1 -- 172.21.15.19:0/1976794779 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131028223 0 0) 0x7f1ea00d2420 con 0x7f1ea00d34c0 2024-04-25T10:00:37.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.704+0000 7f1ea558f640 1 -- 172.21.15.19:0/1976794779 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea00d29e0 con 0x7f1ea00d1f30 2024-04-25T10:00:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.705+0000 7f1ea558f640 1 --2- 172.21.15.19:0/1976794779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea00d1f30 0x7f1ea00d03a0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00b6520 tx=0x7f1e9002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8615e7e1f83dcfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.705+0000 7f1ea4d8e640 1 -- 172.21.15.19:0/1976794779 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90037070 con 0x7f1ea00d1f30 2024-04-25T10:00:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.705+0000 7f1ea4d8e640 1 -- 172.21.15.19:0/1976794779 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e90032440 con 0x7f1ea00d1f30 2024-04-25T10:00:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.705+0000 7f1ea4d8e640 1 -- 172.21.15.19:0/1976794779 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90032810 con 0x7f1ea00d1f30 2024-04-25T10:00:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.706+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1976794779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea00d1f30 msgr2=0x7f1ea00d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.706+0000 7f1ea7ff4640 1 --2- 172.21.15.19:0/1976794779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea00d1f30 0x7f1ea00d03a0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00b6520 tx=0x7f1e9002a230 comp rx=0 tx=0).stop 2024-04-25T10:00:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.706+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1976794779 shutdown_connections 2024-04-25T10:00:37.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.706+0000 7f1ea7ff4640 1 --2- 172.21.15.19:0/1976794779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea00d1f30 0x7f1ea00d03a0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.706+0000 7f1ea7ff4640 1 --2- 172.21.15.19:0/1976794779 >> v2:172.21.15.112:3300/0 conn(0x7f1ea015dac0 0x7f1ea00d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.706+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1976794779 >> 172.21.15.19:0/1976794779 conn(0x7f1ea0061c30 msgr2=0x7f1ea006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.706+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1976794779 shutdown_connections 2024-04-25T10:00:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.706+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1976794779 wait complete. 2024-04-25T10:00:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.707+0000 7f1ea7ff4640 1 Processor -- start 2024-04-25T10:00:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.707+0000 7f1ea7ff4640 1 -- start start 2024-04-25T10:00:37.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea7ff4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ea00d1f30 0x7f1ea014bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea7ff4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea00d34c0 0x7f1ea0156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea7ff4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea015dac0 0x7f1ea015e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea7ff4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00b97d0 con 0x7f1ea00d34c0 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea7ff4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea015dea0 con 0x7f1ea015dac0 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea7ff4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00b9950 con 0x7f1ea00d1f30 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea558f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea00d34c0 0x7f1ea0156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea558f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea00d34c0 0x7f1ea0156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48664/0 (socket says 172.21.15.19:48664) 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea558f640 1 -- 172.21.15.19:0/1214777115 learned_addr learned my addr 172.21.15.19:0/1214777115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea6591640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea015dac0 0x7f1ea015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ea00d1f30 0x7f1ea014bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea558f640 1 -- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ea00d1f30 msgr2=0x7f1ea014bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea558f640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ea00d1f30 0x7f1ea014bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea558f640 1 -- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea015dac0 msgr2=0x7f1ea015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea558f640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea015dac0 0x7f1ea015e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea558f640 1 -- 172.21.15.19:0/1214777115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea0157940 con 0x7f1ea00d34c0 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea6591640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea015dac0 0x7f1ea015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1ea558f640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea00d34c0 0x7f1ea0156ef0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1e90000c00 tx=0x7f1e900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.708+0000 7f1e967fc640 1 -- 172.21.15.19:0/1214777115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90037040 con 0x7f1ea00d34c0 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.709+0000 7f1e967fc640 1 -- 172.21.15.19:0/1214777115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e90042070 con 0x7f1ea00d34c0 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.709+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1214777115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea0157c20 con 0x7f1ea00d34c0 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.709+0000 7f1e967fc640 1 -- 172.21.15.19:0/1214777115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90030a50 con 0x7f1ea00d34c0 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.709+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1214777115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea01581e0 con 0x7f1ea00d34c0 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.710+0000 7f1e967fc640 1 -- 172.21.15.19:0/1214777115 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e90030bf0 con 0x7f1ea00d34c0 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.711+0000 7f1e967fc640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e8406b4d0 0x7f1e8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.711+0000 7f1e967fc640 1 -- 172.21.15.19:0/1214777115 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e900aae70 con 0x7f1ea00d34c0 2024-04-25T10:00:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.711+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e8406b4d0 0x7f1e8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.711+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e8406b4d0 0x7f1e8406d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c006fd0 tx=0x7f1e9c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.711+0000 7f1eaadcc740 1 Processor -- start 2024-04-25T10:00:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1eaadcc740 1 -- start start 2024-04-25T10:00:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf401230 0x55a3cf401610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf41ead0 0x55a3cf41f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf3d6f10 0x55a3cf41d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cf339a10 con 0x55a3cf3d6f10 2024-04-25T10:00:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cf2e2b30 con 0x55a3cf41ead0 2024-04-25T10:00:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3cf335c50 con 0x55a3cf401230 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1ea6591640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf3d6f10 0x55a3cf41d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1ea6591640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf3d6f10 0x55a3cf41d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48678/0 (socket says 172.21.15.19:48678) 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1ea6591640 1 -- 172.21.15.19:0/227980573 learned_addr learned my addr 172.21.15.19:0/227980573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1ea558f640 1 --2- 172.21.15.19:0/227980573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf41ead0 0x55a3cf41f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.712+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/227980573 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf401230 0x55a3cf401610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1ea6591640 1 -- 172.21.15.19:0/227980573 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf401230 msgr2=0x55a3cf401610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1ea6591640 1 --2- 172.21.15.19:0/227980573 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf401230 0x55a3cf401610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1ea6591640 1 -- 172.21.15.19:0/227980573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf41ead0 msgr2=0x55a3cf41f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1ea6591640 1 --2- 172.21.15.19:0/227980573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf41ead0 0x55a3cf41f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1ea6591640 1 -- 172.21.15.19:0/227980573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3cf2f7e10 con 0x55a3cf3d6f10 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/227980573 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf401230 0x55a3cf401610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1ea6591640 1 --2- 172.21.15.19:0/227980573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf3d6f10 0x55a3cf41d190 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f1e98007360 tx=0x7f1e98007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07b682c29fd62ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1e837fe640 1 -- 172.21.15.19:0/227980573 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98019070 con 0x55a3cf3d6f10 2024-04-25T10:00:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1e837fe640 1 -- 172.21.15.19:0/227980573 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e98001e20 con 0x55a3cf3d6f10 2024-04-25T10:00:37.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1e837fe640 1 -- 172.21.15.19:0/227980573 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980112e0 con 0x55a3cf3d6f10 2024-04-25T10:00:37.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1eaadcc740 1 -- 172.21.15.19:0/227980573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf3d6f10 msgr2=0x55a3cf41d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.713+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/227980573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf3d6f10 0x55a3cf41d190 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f1e98007360 tx=0x7f1e98007820 comp rx=0 tx=0).stop 2024-04-25T10:00:37.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.714+0000 7f1eaadcc740 1 -- 172.21.15.19:0/227980573 shutdown_connections 2024-04-25T10:00:37.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.714+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/227980573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf3d6f10 0x55a3cf41d190 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.714+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/227980573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf41ead0 0x55a3cf41f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.714+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/227980573 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf401230 0x55a3cf401610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.714+0000 7f1eaadcc740 1 -- 172.21.15.19:0/227980573 >> 172.21.15.19:0/227980573 conn(0x55a3cf3d3840 msgr2=0x55a3cf41c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:37.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.714+0000 7f1eaadcc740 1 -- 172.21.15.19:0/227980573 shutdown_connections 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.714+0000 7f1eaadcc740 1 -- 172.21.15.19:0/227980573 wait complete. 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.714+0000 7f1eaadcc740 1 -- start start 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.714+0000 7f1eaadcc740 1 Processor -- start 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf401230 0x55a3cf415ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf4130f0 0x55a3cf3d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cf3d7cb0 con 0x55a3cf41ead0 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cf2f60c0 con 0x55a3cf4130f0 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3cf3d1770 con 0x55a3cf401230 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1eaadcc740 10 client.0 _ll_register_callbacks cb 0x55a3cf2e9ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1ea558f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1ea558f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48682/0 (socket says 172.21.15.19:48682) 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1ea558f640 1 -- 172.21.15.19:0/1155016967 learned_addr learned my addr 172.21.15.19:0/1155016967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf401230 0x55a3cf415ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.715+0000 7f1ea6591640 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf4130f0 0x55a3cf3d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1ea558f640 1 -- 172.21.15.19:0/1155016967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf401230 msgr2=0x55a3cf415ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1ea558f640 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf401230 0x55a3cf415ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1ea558f640 1 -- 172.21.15.19:0/1155016967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf4130f0 msgr2=0x55a3cf3d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1ea558f640 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf4130f0 0x55a3cf3d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1ea558f640 1 -- 172.21.15.19:0/1155016967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3cf04e2d0 con 0x55a3cf41ead0 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1ea6591640 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf4130f0 0x55a3cf3d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1ea558f640 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d07b0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f1e90079160 tx=0x7f1e90032500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6e99f686c2028b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1e80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1e80ff9640 1 -- 172.21.15.19:0/1155016967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9007d350 con 0x55a3cf41ead0 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1e80ff9640 1 -- 172.21.15.19:0/1155016967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e900796c0 con 0x55a3cf41ead0 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1155016967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e98001e20 con 0x55a3cf41ead0 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1e80ff9640 1 -- 172.21.15.19:0/1155016967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e90098050 con 0x55a3cf41ead0 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1eaadcc740 10 client.8733 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1155016967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55a3cf1186b0 con 0x55a3cf41ead0 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1e80ff9640 1 -- 172.21.15.19:0/1155016967 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e98001e20 con 0x55a3cf41ead0 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1e80ff9640 1 -- 172.21.15.19:0/1155016967 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1e9003a030 con 0x55a3cf41ead0 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1eaadcc740 10 client.8733 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1eaadcc740 10 client.8733 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1eaadcc740 20 client.8733 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55a3cf42d070 0x55a3cf42d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1eaadcc740 4 client.8733 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1155016967 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3cf398920 con 0x55a3cf42d070 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.716+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55a3cf42d070 0x55a3cf42d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.717+0000 7f1e80ff9640 10 client.8733 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.717+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55a3cf42d070 0x55a3cf42d470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e54ea57d8777843 server_cookie=bb06a5f2f6da1e46 in_seq=0 out_seq=0 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.717+0000 7f1e80ff9640 1 -- 172.21.15.19:0/1155016967 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a3cf398920 con 0x55a3cf42d070 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.717+0000 7f1e80ff9640 10 client.8733 handle_command_reply: tid=0 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.717+0000 7f1eaadcc740 1 client.8733 shutdown 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.717+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1155016967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 msgr2=0x55a3cf3d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.717+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d07b0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f1e90079160 tx=0x7f1e90032500 comp rx=0 tx=0).stop 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.717+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1155016967 shutdown_connections 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.717+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55a3cf42d070 0x55a3cf42d470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.717+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf4130f0 0x55a3cf3d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.718+0000 7f1e80ff9640 0 client.8733 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.718+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d07b0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.718+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/1155016967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf401230 0x55a3cf415ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.718+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1155016967 >> 172.21.15.19:0/1155016967 conn(0x55a3cf3d6f10 msgr2=0x55a3cf41f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.718+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1155016967 shutdown_connections 2024-04-25T10:00:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.718+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1155016967 wait complete. 2024-04-25T10:00:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.718+0000 7f1eaadcc740 20 client.8733 trim_cache size 0 max 16384 2024-04-25T10:00:37.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.734+0000 7f1eaadcc740 1 Processor -- start 2024-04-25T10:00:37.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.734+0000 7f1eaadcc740 1 -- start start 2024-04-25T10:00:37.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.734+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf401230 0x55a3cf400330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.734+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf4130f0 0x55a3cf3d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.734+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.734+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea02002e0 con 0x55a3cf41ead0 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.734+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea01ff4e0 con 0x55a3cf401230 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.734+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00b4530 con 0x55a3cf4130f0 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.734+0000 7f1ea6591640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.734+0000 7f1ea6591640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48696/0 (socket says 172.21.15.19:48696) 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.734+0000 7f1ea6591640 1 -- 172.21.15.19:0/1434295473 learned_addr learned my addr 172.21.15.19:0/1434295473 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1ea558f640 1 --2- 172.21.15.19:0/1434295473 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf4130f0 0x55a3cf3d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/1434295473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf401230 0x55a3cf400330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1ea6591640 1 -- 172.21.15.19:0/1434295473 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf4130f0 msgr2=0x55a3cf3d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1ea6591640 1 --2- 172.21.15.19:0/1434295473 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf4130f0 0x55a3cf3d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1ea6591640 1 -- 172.21.15.19:0/1434295473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf401230 msgr2=0x55a3cf400330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1ea6591640 1 --2- 172.21.15.19:0/1434295473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf401230 0x55a3cf400330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1ea6591640 1 -- 172.21.15.19:0/1434295473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3cf403d30 con 0x55a3cf41ead0 2024-04-25T10:00:37.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/1434295473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf401230 0x55a3cf400330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:37.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1ea6591640 1 --2- 172.21.15.19:0/1434295473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d0c30 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f1e98000f80 tx=0x7f1e98011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee6b7b4deb51772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:37.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1e82ffd640 1 -- 172.21.15.19:0/1434295473 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98019070 con 0x55a3cf41ead0 2024-04-25T10:00:37.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1e82ffd640 1 -- 172.21.15.19:0/1434295473 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9800b040 con 0x55a3cf41ead0 2024-04-25T10:00:37.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.735+0000 7f1e82ffd640 1 -- 172.21.15.19:0/1434295473 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98014690 con 0x55a3cf41ead0 2024-04-25T10:00:37.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.736+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1434295473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 msgr2=0x55a3cf3d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.736+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/1434295473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d0c30 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f1e98000f80 tx=0x7f1e98011000 comp rx=0 tx=0).stop 2024-04-25T10:00:37.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.736+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1434295473 shutdown_connections 2024-04-25T10:00:37.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.736+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/1434295473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf41ead0 0x55a3cf3d0c30 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.736+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/1434295473 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf4130f0 0x55a3cf3d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.736+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/1434295473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf401230 0x55a3cf400330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.736+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1434295473 >> 172.21.15.19:0/1434295473 conn(0x55a3cf3d6f10 msgr2=0x55a3cf4169e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:37.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.736+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1434295473 shutdown_connections 2024-04-25T10:00:37.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.736+0000 7f1eaadcc740 1 -- 172.21.15.19:0/1434295473 wait complete. 2024-04-25T10:00:37.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.736+0000 7f1eaadcc740 1 -- start start 2024-04-25T10:00:37.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.736+0000 7f1eaadcc740 1 Processor -- start 2024-04-25T10:00:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf401230 0x55a3cf435da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf4130f0 0x55a3cf4362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1eaadcc740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf41ead0 0x55a3cf401ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cf42dbe0 con 0x55a3cf401230 2024-04-25T10:00:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00b4b30 con 0x55a3cf4130f0 2024-04-25T10:00:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1eaadcc740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3cf403a60 con 0x55a3cf41ead0 2024-04-25T10:00:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1eaadcc740 10 client.0 _ll_register_callbacks cb 0x55a3cf3d1c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1ea5d90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf401230 0x55a3cf435da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1ea5d90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf401230 0x55a3cf435da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48706/0 (socket says 172.21.15.19:48706) 2024-04-25T10:00:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1ea5d90640 1 -- 172.21.15.19:0/3098796057 learned_addr learned my addr 172.21.15.19:0/3098796057 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1ea558f640 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf4130f0 0x55a3cf4362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.737+0000 7f1ea6591640 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf41ead0 0x55a3cf401ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1ea5d90640 1 -- 172.21.15.19:0/3098796057 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf41ead0 msgr2=0x55a3cf401ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf41ead0 0x55a3cf401ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1ea5d90640 1 -- 172.21.15.19:0/3098796057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf4130f0 msgr2=0x55a3cf4362e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf4130f0 0x55a3cf4362e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1ea5d90640 1 -- 172.21.15.19:0/3098796057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3cf48a5c0 con 0x55a3cf401230 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1ea5d90640 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf401230 0x55a3cf435da0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00e5a0 tx=0x7f1e9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4621edf52294765b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1e80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1e80ff9640 1 -- 172.21.15.19:0/3098796057 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c01a050 con 0x55a3cf401230 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1e80ff9640 1 -- 172.21.15.19:0/3098796057 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e9c028b00 con 0x55a3cf401230 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1eaadcc740 1 -- 172.21.15.19:0/3098796057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e9800b040 con 0x55a3cf401230 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1e80ff9640 1 -- 172.21.15.19:0/3098796057 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e9c036050 con 0x55a3cf401230 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1eaadcc740 10 client.8739 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1eaadcc740 1 -- 172.21.15.19:0/3098796057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55a3cf41ffe0 con 0x55a3cf401230 2024-04-25T10:00:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1e80ff9640 1 -- 172.21.15.19:0/3098796057 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9800b040 con 0x55a3cf401230 2024-04-25T10:00:37.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1e80ff9640 1 -- 172.21.15.19:0/3098796057 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1e9c039020 con 0x55a3cf401230 2024-04-25T10:00:37.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1eaadcc740 10 client.8739 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:37.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1eaadcc740 10 client.8739 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:00:37.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1eaadcc740 20 client.8739 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:37.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55a3cf486a70 0x55a3cf486e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1eaadcc740 4 client.8739 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:00:37.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.738+0000 7f1eaadcc740 1 -- 172.21.15.19:0/3098796057 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a3cf346f50 con 0x55a3cf486a70 2024-04-25T10:00:37.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1ea558f640 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55a3cf486a70 0x55a3cf486e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1e80ff9640 10 client.8739 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:37.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1ea558f640 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55a3cf486a70 0x55a3cf486e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=835048b40a6b7245 server_cookie=5106fee60076ba32 in_seq=0 out_seq=0 2024-04-25T10:00:37.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1e80ff9640 1 -- 172.21.15.19:0/3098796057 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55a3cf346f50 con 0x55a3cf486a70 2024-04-25T10:00:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1e80ff9640 10 client.8739 handle_command_reply: tid=0 2024-04-25T10:00:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 1 client.8739 shutdown 2024-04-25T10:00:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 1 -- 172.21.15.19:0/3098796057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf401230 msgr2=0x55a3cf435da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf401230 0x55a3cf435da0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00e5a0 tx=0x7f1e9c004730 comp rx=0 tx=0).stop 2024-04-25T10:00:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 1 -- 172.21.15.19:0/3098796057 shutdown_connections 2024-04-25T10:00:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55a3cf486a70 0x55a3cf486e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55a3cf41ead0 0x55a3cf401ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3cf4130f0 0x55a3cf4362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1e80ff9640 0 client.8739 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 1 --2- 172.21.15.19:0/3098796057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a3cf401230 0x55a3cf435da0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 1 -- 172.21.15.19:0/3098796057 >> 172.21.15.19:0/3098796057 conn(0x55a3cf3d6f10 msgr2=0x55a3cf3d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 1 -- 172.21.15.19:0/3098796057 shutdown_connections 2024-04-25T10:00:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 1 -- 172.21.15.19:0/3098796057 wait complete. 2024-04-25T10:00:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.739+0000 7f1eaadcc740 20 client.8739 trim_cache size 0 max 16384 2024-04-25T10:00:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T10:00:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:00:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 8577, 2024-04-25T10:00:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:00:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:00:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:00:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 4.678347252, 2024-04-25T10:00:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 72, 2024-04-25T10:00:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 117, 2024-04-25T10:00:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 117, 2024-04-25T10:00:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 9.7139450150000002 2024-04-25T10:00:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:37.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e8406b4d0 msgr2=0x7f1e8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e8406b4d0 0x7f1e8406d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c006fd0 tx=0x7f1e9c008040 comp rx=0 tx=0).stop 2024-04-25T10:00:37.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1214777115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea00d34c0 msgr2=0x7f1ea0156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea00d34c0 0x7f1ea0156ef0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1e90000c00 tx=0x7f1e900046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1214777115 shutdown_connections 2024-04-25T10:00:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1e8406b4d0 0x7f1e8406d990 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea015dac0 0x7f1ea015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ea00d34c0 0x7f1ea0156ef0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 --2- 172.21.15.19:0/1214777115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ea00d1f30 0x7f1ea014bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1214777115 >> 172.21.15.19:0/1214777115 conn(0x7f1ea0061c30 msgr2=0x7f1ea00c7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1214777115 shutdown_connections 2024-04-25T10:00:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.740+0000 7f1ea7ff4640 1 -- 172.21.15.19:0/1214777115 wait complete. 2024-04-25T10:00:37.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:00:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.853+0000 7f890329a640 1 Processor -- start 2024-04-25T10:00:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.853+0000 7f890329a640 1 -- start start 2024-04-25T10:00:37.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.854+0000 7f890329a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc15dc70 0x7f88fc0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.854+0000 7f890329a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88fc06c220 con 0x7f88fc09a450 2024-04-25T10:00:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.854+0000 7f890329a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88fc0d0080 con 0x7f88fc06ce10 2024-04-25T10:00:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.854+0000 7f890329a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc09a8b0 con 0x7f88fc15dc70 2024-04-25T10:00:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.854+0000 7f8900835640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f88fc09a450 0x7f88fc06c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57454/0 (socket says 172.21.15.19:57454) 2024-04-25T10:00:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.854+0000 7f8900835640 1 -- 172.21.15.19:0/3538719836 learned_addr learned my addr 172.21.15.19:0/3538719836 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f8901036640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc15dc70 0x7f88fc0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936661462 0 0) 0x7f88fc06c220 con 0x7f88fc09a450 2024-04-25T10:00:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88e4003520 con 0x7f88fc09a450 2024-04-25T10:00:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2716131653 0 0) 0x7f88e4003520 con 0x7f88fc09a450 2024-04-25T10:00:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc15dc70 msgr2=0x7f88fc0d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc15dc70 0x7f88fc0d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 >> v1:172.21.15.112:6789/0 conn(0x7f88fc06ce10 legacy=0x7f88fc0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc0d0360 con 0x7f88fc09a450 2024-04-25T10:00:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f88f0001670 con 0x7f88fc09a450 2024-04-25T10:00:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 >> v1:172.21.15.19:6789/0 conn(0x7f88fc09a450 legacy=0x7f88fc06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc15dc70 0x7f88e40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e4003f20 0x7f88e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e4004d40 0x7f88e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc09a8b0 con 0x7f88e4004d40 2024-04-25T10:00:37.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e400ba20 con 0x7f88e4003f20 2024-04-25T10:00:37.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e400bc20 con 0x7f88fc15dc70 2024-04-25T10:00:37.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f8901837640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e4004d40 0x7f88e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f8900835640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e4003f20 0x7f88e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f8901036640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc15dc70 0x7f88e40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f8901837640 1 -- 172.21.15.19:0/3538719836 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc15dc70 msgr2=0x7f88e40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f8901837640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc15dc70 0x7f88e40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f8901837640 1 -- 172.21.15.19:0/3538719836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e4003f20 msgr2=0x7f88e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f8901837640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e4003f20 0x7f88e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.855+0000 7f8901837640 1 -- 172.21.15.19:0/3538719836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e4003520 con 0x7f88e4004d40 2024-04-25T10:00:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.856+0000 7f8901837640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e4004d40 0x7f88e40051c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f88f4001860 tx=0x7f88e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cb290564b80105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.856+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4037070 con 0x7f88e4004d40 2024-04-25T10:00:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.856+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f402abf0 con 0x7f88e4004d40 2024-04-25T10:00:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.856+0000 7f88fbfff640 1 -- 172.21.15.19:0/3538719836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4001430 con 0x7f88e4004d40 2024-04-25T10:00:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.857+0000 7f890329a640 1 -- 172.21.15.19:0/3538719836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e4004d40 msgr2=0x7f88e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.857+0000 7f890329a640 1 --2- 172.21.15.19:0/3538719836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88e4004d40 0x7f88e40051c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f88f4001860 tx=0x7f88e4005830 comp rx=0 tx=0).stop 2024-04-25T10:00:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.857+0000 7f8901837640 1 -- 172.21.15.19:0/3538719836 reap_dead start 2024-04-25T10:00:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.857+0000 7f890329a640 1 -- 172.21.15.19:0/3538719836 shutdown_connections 2024-04-25T10:00:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.857+0000 7f890329a640 1 -- 172.21.15.19:0/3538719836 >> 172.21.15.19:0/3538719836 conn(0x7f88fc0b3270 msgr2=0x7f88fc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.857+0000 7f890329a640 1 -- 172.21.15.19:0/3538719836 shutdown_connections 2024-04-25T10:00:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.857+0000 7f890329a640 1 -- 172.21.15.19:0/3538719836 wait complete. 2024-04-25T10:00:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.858+0000 7f890329a640 1 Processor -- start 2024-04-25T10:00:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.858+0000 7f890329a640 1 -- start start 2024-04-25T10:00:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f890329a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc15a160 0x7f88fc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f890329a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc0ce2a0 0x7f88fc0ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f890329a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc0c5df0 0x7f88fc0c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f890329a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0d0920 con 0x7f88fc15a160 2024-04-25T10:00:37.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f890329a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0d0c90 con 0x7f88fc0ce2a0 2024-04-25T10:00:37.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f890329a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc15e050 con 0x7f88fc0c5df0 2024-04-25T10:00:37.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f8901036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc15a160 0x7f88fc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f8901036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc15a160 0x7f88fc0b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48720/0 (socket says 172.21.15.19:48720) 2024-04-25T10:00:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f8901036640 1 -- 172.21.15.19:0/1608572603 learned_addr learned my addr 172.21.15.19:0/1608572603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f8900835640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc0ce2a0 0x7f88fc0ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f8901837640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc0c5df0 0x7f88fc0c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f8901036640 1 -- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc0c5df0 msgr2=0x7f88fc0c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f8901036640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc0c5df0 0x7f88fc0c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f8901036640 1 -- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc0ce2a0 msgr2=0x7f88fc0ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f8901036640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc0ce2a0 0x7f88fc0ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.859+0000 7f8901036640 1 -- 172.21.15.19:0/1608572603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc0b49f0 con 0x7f88fc15a160 2024-04-25T10:00:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.860+0000 7f8901036640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc15a160 0x7f88fc0b4d60 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f88ec011e30 tx=0x7f88ec00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.860+0000 7f88f9ffb640 1 -- 172.21.15.19:0/1608572603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00fea0 con 0x7f88fc15a160 2024-04-25T10:00:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.860+0000 7f88f9ffb640 1 -- 172.21.15.19:0/1608572603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec015630 con 0x7f88fc15a160 2024-04-25T10:00:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.860+0000 7f890329a640 1 -- 172.21.15.19:0/1608572603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc0b4710 con 0x7f88fc15a160 2024-04-25T10:00:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.860+0000 7f88f9ffb640 1 -- 172.21.15.19:0/1608572603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec015a00 con 0x7f88fc15a160 2024-04-25T10:00:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.860+0000 7f890329a640 1 -- 172.21.15.19:0/1608572603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc0c55f0 con 0x7f88fc15a160 2024-04-25T10:00:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.861+0000 7f890329a640 1 -- 172.21.15.19:0/1608572603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88fc073990 con 0x7f88fc15a160 2024-04-25T10:00:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.861+0000 7f88f9ffb640 1 -- 172.21.15.19:0/1608572603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88ec015ba0 con 0x7f88fc15a160 2024-04-25T10:00:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.862+0000 7f88f9ffb640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88e806b4d0 0x7f88e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.862+0000 7f88f9ffb640 1 -- 172.21.15.19:0/1608572603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f88ec08b3f0 con 0x7f88fc15a160 2024-04-25T10:00:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.862+0000 7f8900835640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88e806b4d0 0x7f88e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.863+0000 7f8900835640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88e806b4d0 0x7f88e806d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0cf1e0 tx=0x7f88f0001150 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:37.866+0000 7f88f9ffb640 1 -- 172.21.15.19:0/1608572603 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88ec019030 con 0x7f88fc15a160 2024-04-25T10:00:38.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.085+0000 7f890329a640 1 -- 172.21.15.19:0/1608572603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f88fc0b40a0 con 0x7f88fc15a160 2024-04-25T10:00:38.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.087+0000 7f88f9ffb640 1 -- 172.21.15.19:0/1608572603 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7f88ec010040 con 0x7f88fc15a160 2024-04-25T10:00:38.092 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:00:38.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.089+0000 7f890329a640 1 -- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88e806b4d0 msgr2=0x7f88e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:38.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.089+0000 7f890329a640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88e806b4d0 0x7f88e806d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0cf1e0 tx=0x7f88f0001150 comp rx=0 tx=0).stop 2024-04-25T10:00:38.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.089+0000 7f890329a640 1 -- 172.21.15.19:0/1608572603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc15a160 msgr2=0x7f88fc0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:38.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.089+0000 7f890329a640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc15a160 0x7f88fc0b4d60 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f88ec011e30 tx=0x7f88ec00f580 comp rx=0 tx=0).stop 2024-04-25T10:00:38.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.090+0000 7f890329a640 1 -- 172.21.15.19:0/1608572603 shutdown_connections 2024-04-25T10:00:38.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.090+0000 7f890329a640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f88e806b4d0 0x7f88e806d990 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:38.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.090+0000 7f890329a640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88fc0c5df0 0x7f88fc0c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:38.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.090+0000 7f890329a640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88fc0ce2a0 0x7f88fc0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:38.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.090+0000 7f890329a640 1 --2- 172.21.15.19:0/1608572603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc15a160 0x7f88fc0b4d60 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:38.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.090+0000 7f890329a640 1 -- 172.21.15.19:0/1608572603 >> 172.21.15.19:0/1608572603 conn(0x7f88fc0b3270 msgr2=0x7f88fc0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:38.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.090+0000 7f890329a640 1 -- 172.21.15.19:0/1608572603 shutdown_connections 2024-04-25T10:00:38.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:38.090+0000 7f890329a640 1 -- 172.21.15.19:0/1608572603 wait complete. 2024-04-25T10:00:38.105 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:00:38.105 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:00:38.105 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T10:00:38.105 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:00:38.105 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:38.106 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:00:38.106 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:00:38.106 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:00:38.106 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:00:38.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.106 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T10:00:38.122 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:00:38.123 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:38.124 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T10:00:38.124 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:00:38.124 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42308sec preferred_lft 42308sec 2024-04-25T10:00:38.125 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T10:00:38.125 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:38.125 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:00:38.125 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:38.125 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T10:00:38.125 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:00:38.125 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:00:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T10:00:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:38.126 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:00:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:00:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T10:00:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:38.126 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:00:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:00:38.126 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T10:00:38.127 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:38.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.127 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:00:38.187 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T10:00:38.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.189 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T10:00:38.228 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:38.232 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.246 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T10:00:38.246 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:38.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:00:38.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:00:38.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:00:38.246 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:38.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.364 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T10:00:38.379 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:00:38.380 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:00:38.380 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:00:38.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T10:00:38.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:00:38.446 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:38.501 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:00:38.502 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:00:38.502 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:38.557 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:00:38.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:00:38.642 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:00:38.642 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:00:38.650 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:00:38.650 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:00:38.650 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T10:00:38.651 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T10:00:38.651 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T10:00:38.652 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:00:38.652 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:00:38.652 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:00:38.652 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:00:38.652 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:00:38.653 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:00:38.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:38.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.684 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:00:38.747 INFO:teuthology.orchestra.run.smithi019.stdout:client6893 2024-04-25T10:00:38.747 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:00:38.748 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:38.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.804 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:38.804 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:38.804 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:00:38.804 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:38.804 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:38.804 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:38.804 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:38.881 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.881 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.882 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959389 s, 109 MB/s 2024-04-25T10:00:38.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.891 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814867 s, 129 MB/s 2024-04-25T10:00:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852605 s, 123 MB/s 2024-04-25T10:00:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103802 s, 101 MB/s 2024-04-25T10:00:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832748 s, 126 MB/s 2024-04-25T10:00:38.928 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.929 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.929 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722917 s, 145 MB/s 2024-04-25T10:00:38.938 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.938 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.938 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840562 s, 125 MB/s 2024-04-25T10:00:38.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.947 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766206 s, 137 MB/s 2024-04-25T10:00:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754392 s, 139 MB/s 2024-04-25T10:00:38.964 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.964 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.964 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794217 s, 132 MB/s 2024-04-25T10:00:38.972 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.972 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.972 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756813 s, 139 MB/s 2024-04-25T10:00:38.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.982 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861386 s, 122 MB/s 2024-04-25T10:00:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.990 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.991 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763974 s, 137 MB/s 2024-04-25T10:00:38.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:38.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:38.999 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712592 s, 147 MB/s 2024-04-25T10:00:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781376 s, 134 MB/s 2024-04-25T10:00:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773691 s, 136 MB/s 2024-04-25T10:00:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780964 s, 134 MB/s 2024-04-25T10:00:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717247 s, 146 MB/s 2024-04-25T10:00:39.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.041 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772023 s, 136 MB/s 2024-04-25T10:00:39.046 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T10:00:39.042+0000 7f3b0c88f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:00:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633493 s, 166 MB/s 2024-04-25T10:00:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881905 s, 119 MB/s 2024-04-25T10:00:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725651 s, 145 MB/s 2024-04-25T10:00:39.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.075 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740433 s, 142 MB/s 2024-04-25T10:00:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717335 s, 146 MB/s 2024-04-25T10:00:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704393 s, 149 MB/s 2024-04-25T10:00:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.099 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748921 s, 140 MB/s 2024-04-25T10:00:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.108 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704141 s, 149 MB/s 2024-04-25T10:00:39.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.118 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978899 s, 107 MB/s 2024-04-25T10:00:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676401 s, 155 MB/s 2024-04-25T10:00:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643521 s, 163 MB/s 2024-04-25T10:00:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687404 s, 153 MB/s 2024-04-25T10:00:39.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.149 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700921 s, 150 MB/s 2024-04-25T10:00:39.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:39.150 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:39.150 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:39.150 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:00:39.150 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:39.151 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:39.151 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:39.151 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:39.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.214 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654403 s, 160 MB/s 2024-04-25T10:00:39.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.222 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709278 s, 148 MB/s 2024-04-25T10:00:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697427 s, 150 MB/s 2024-04-25T10:00:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745461 s, 141 MB/s 2024-04-25T10:00:39.246 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.246 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.246 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698757 s, 150 MB/s 2024-04-25T10:00:39.254 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.254 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.254 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689592 s, 152 MB/s 2024-04-25T10:00:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709748 s, 148 MB/s 2024-04-25T10:00:39.269 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.269 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.269 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630635 s, 166 MB/s 2024-04-25T10:00:39.277 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.277 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.277 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716089 s, 146 MB/s 2024-04-25T10:00:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637815 s, 164 MB/s 2024-04-25T10:00:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667703 s, 157 MB/s 2024-04-25T10:00:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653515 s, 160 MB/s 2024-04-25T10:00:39.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.307 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069933 s, 150 MB/s 2024-04-25T10:00:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850384 s, 123 MB/s 2024-04-25T10:00:39.325 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.325 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.325 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717955 s, 146 MB/s 2024-04-25T10:00:39.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.334 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793361 s, 132 MB/s 2024-04-25T10:00:39.342 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.342 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.342 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709429 s, 148 MB/s 2024-04-25T10:00:39.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.350 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068963 s, 152 MB/s 2024-04-25T10:00:39.358 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.358 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.358 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732256 s, 143 MB/s 2024-04-25T10:00:39.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.367 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746481 s, 140 MB/s 2024-04-25T10:00:39.374 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.375 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713901 s, 147 MB/s 2024-04-25T10:00:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.383 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715841 s, 146 MB/s 2024-04-25T10:00:39.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.390 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648288 s, 162 MB/s 2024-04-25T10:00:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676925 s, 155 MB/s 2024-04-25T10:00:39.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.405 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663755 s, 158 MB/s 2024-04-25T10:00:39.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.413 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066324 s, 158 MB/s 2024-04-25T10:00:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666083 s, 157 MB/s 2024-04-25T10:00:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068022 s, 154 MB/s 2024-04-25T10:00:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647543 s, 162 MB/s 2024-04-25T10:00:39.443 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.443 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.443 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693024 s, 151 MB/s 2024-04-25T10:00:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688924 s, 152 MB/s 2024-04-25T10:00:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074206 s, 141 MB/s 2024-04-25T10:00:39.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:39.460 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:39.460 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:39.461 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:00:39.461 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:39.461 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:39.461 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:39.461 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835127 s, 126 MB/s 2024-04-25T10:00:39.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.536 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708104 s, 148 MB/s 2024-04-25T10:00:39.542 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623903 s, 168 MB/s 2024-04-25T10:00:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078348 s, 134 MB/s 2024-04-25T10:00:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645175 s, 163 MB/s 2024-04-25T10:00:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638668 s, 164 MB/s 2024-04-25T10:00:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629232 s, 167 MB/s 2024-04-25T10:00:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735233 s, 143 MB/s 2024-04-25T10:00:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.589 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677026 s, 155 MB/s 2024-04-25T10:00:39.598 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.598 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.598 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792283 s, 132 MB/s 2024-04-25T10:00:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743166 s, 141 MB/s 2024-04-25T10:00:39.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.614 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681455 s, 154 MB/s 2024-04-25T10:00:39.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.622 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739301 s, 142 MB/s 2024-04-25T10:00:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636008 s, 165 MB/s 2024-04-25T10:00:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729384 s, 144 MB/s 2024-04-25T10:00:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711196 s, 147 MB/s 2024-04-25T10:00:39.653 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.653 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.653 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687071 s, 153 MB/s 2024-04-25T10:00:39.661 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.661 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.661 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671621 s, 156 MB/s 2024-04-25T10:00:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.669 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711041 s, 147 MB/s 2024-04-25T10:00:39.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.678 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834051 s, 126 MB/s 2024-04-25T10:00:39.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.686 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689884 s, 152 MB/s 2024-04-25T10:00:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.694 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667164 s, 157 MB/s 2024-04-25T10:00:39.701 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.701 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.702 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683336 s, 153 MB/s 2024-04-25T10:00:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.710 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075571 s, 139 MB/s 2024-04-25T10:00:39.717 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.717 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.717 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614749 s, 171 MB/s 2024-04-25T10:00:39.724 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.724 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.724 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662507 s, 158 MB/s 2024-04-25T10:00:39.733 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.733 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.733 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081547 s, 129 MB/s 2024-04-25T10:00:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068684 s, 153 MB/s 2024-04-25T10:00:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644223 s, 163 MB/s 2024-04-25T10:00:39.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.756 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636987 s, 165 MB/s 2024-04-25T10:00:39.763 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.763 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695364 s, 151 MB/s 2024-04-25T10:00:39.771 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.771 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.771 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657768 s, 159 MB/s 2024-04-25T10:00:39.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:39.772 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:39.772 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:39.772 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:00:39.772 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:39.772 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:39.772 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:39.773 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.837 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705412 s, 149 MB/s 2024-04-25T10:00:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701579 s, 149 MB/s 2024-04-25T10:00:39.853 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.853 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.853 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808115 s, 130 MB/s 2024-04-25T10:00:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631957 s, 166 MB/s 2024-04-25T10:00:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.869 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700446 s, 150 MB/s 2024-04-25T10:00:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630246 s, 166 MB/s 2024-04-25T10:00:39.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.886 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882859 s, 119 MB/s 2024-04-25T10:00:39.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.893 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006744 s, 155 MB/s 2024-04-25T10:00:39.901 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758534 s, 138 MB/s 2024-04-25T10:00:39.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.909 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634071 s, 165 MB/s 2024-04-25T10:00:39.917 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.917 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.917 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700668 s, 150 MB/s 2024-04-25T10:00:39.924 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.925 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693104 s, 151 MB/s 2024-04-25T10:00:39.932 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.932 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.932 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062407 s, 168 MB/s 2024-04-25T10:00:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727521 s, 144 MB/s 2024-04-25T10:00:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713924 s, 147 MB/s 2024-04-25T10:00:39.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.956 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676076 s, 155 MB/s 2024-04-25T10:00:39.964 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.964 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.964 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724035 s, 145 MB/s 2024-04-25T10:00:39.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.974 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622392 s, 168 MB/s 2024-04-25T10:00:39.979 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.979 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.979 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710211 s, 148 MB/s 2024-04-25T10:00:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704221 s, 149 MB/s 2024-04-25T10:00:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:39.995 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:39.995 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677232 s, 155 MB/s 2024-04-25T10:00:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720875 s, 145 MB/s 2024-04-25T10:00:40.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.010 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065426 s, 160 MB/s 2024-04-25T10:00:40.018 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.018 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.018 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709245 s, 148 MB/s 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633895 s, 165 MB/s 2024-04-25T10:00:40.034 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.034 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.034 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713961 s, 147 MB/s 2024-04-25T10:00:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677533 s, 155 MB/s 2024-04-25T10:00:40.049 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.049 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.050 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712512 s, 147 MB/s 2024-04-25T10:00:40.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.057 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699977 s, 150 MB/s 2024-04-25T10:00:40.066 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.066 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.066 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775214 s, 135 MB/s 2024-04-25T10:00:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700995 s, 150 MB/s 2024-04-25T10:00:40.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.082 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667488 s, 157 MB/s 2024-04-25T10:00:40.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:40.083 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:40.083 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:40.083 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:00:40.083 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:40.083 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:40.083 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:40.083 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642393 s, 163 MB/s 2024-04-25T10:00:40.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.156 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770288 s, 136 MB/s 2024-04-25T10:00:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624721 s, 168 MB/s 2024-04-25T10:00:40.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.171 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682126 s, 154 MB/s 2024-04-25T10:00:40.179 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.179 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.179 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006786 s, 155 MB/s 2024-04-25T10:00:40.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.186 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629669 s, 167 MB/s 2024-04-25T10:00:40.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.193 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639979 s, 164 MB/s 2024-04-25T10:00:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748544 s, 140 MB/s 2024-04-25T10:00:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712404 s, 147 MB/s 2024-04-25T10:00:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663809 s, 158 MB/s 2024-04-25T10:00:40.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.226 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755416 s, 139 MB/s 2024-04-25T10:00:40.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.234 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713423 s, 147 MB/s 2024-04-25T10:00:40.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.244 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902431 s, 116 MB/s 2024-04-25T10:00:40.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.252 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685912 s, 153 MB/s 2024-04-25T10:00:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.259 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622869 s, 168 MB/s 2024-04-25T10:00:40.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.266 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682909 s, 154 MB/s 2024-04-25T10:00:40.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.274 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615502 s, 170 MB/s 2024-04-25T10:00:40.281 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.281 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.281 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063869 s, 164 MB/s 2024-04-25T10:00:40.288 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.288 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.288 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617807 s, 170 MB/s 2024-04-25T10:00:40.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.295 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607311 s, 173 MB/s 2024-04-25T10:00:40.302 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.302 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695303 s, 151 MB/s 2024-04-25T10:00:40.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.311 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710791 s, 148 MB/s 2024-04-25T10:00:40.319 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.319 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.319 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726092 s, 144 MB/s 2024-04-25T10:00:40.328 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.328 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.328 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822773 s, 127 MB/s 2024-04-25T10:00:40.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.336 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718809 s, 146 MB/s 2024-04-25T10:00:40.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.344 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719481 s, 146 MB/s 2024-04-25T10:00:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746122 s, 141 MB/s 2024-04-25T10:00:40.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.360 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703995 s, 149 MB/s 2024-04-25T10:00:40.368 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.368 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.368 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682529 s, 154 MB/s 2024-04-25T10:00:40.376 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.377 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.377 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739276 s, 142 MB/s 2024-04-25T10:00:40.384 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.384 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.384 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630724 s, 166 MB/s 2024-04-25T10:00:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664485 s, 158 MB/s 2024-04-25T10:00:40.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:40.393 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:40.393 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:40.393 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:00:40.393 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:40.393 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:40.393 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:40.393 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:40.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.458 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643636 s, 163 MB/s 2024-04-25T10:00:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671391 s, 156 MB/s 2024-04-25T10:00:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074664 s, 140 MB/s 2024-04-25T10:00:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702954 s, 149 MB/s 2024-04-25T10:00:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678851 s, 154 MB/s 2024-04-25T10:00:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768914 s, 136 MB/s 2024-04-25T10:00:40.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.505 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617724 s, 170 MB/s 2024-04-25T10:00:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071137 s, 147 MB/s 2024-04-25T10:00:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654301 s, 160 MB/s 2024-04-25T10:00:40.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.528 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672513 s, 156 MB/s 2024-04-25T10:00:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617201 s, 170 MB/s 2024-04-25T10:00:40.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.543 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703423 s, 149 MB/s 2024-04-25T10:00:40.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697156 s, 150 MB/s 2024-04-25T10:00:40.558 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.558 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.558 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616033 s, 170 MB/s 2024-04-25T10:00:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654171 s, 160 MB/s 2024-04-25T10:00:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.573 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639884 s, 164 MB/s 2024-04-25T10:00:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608356 s, 172 MB/s 2024-04-25T10:00:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698342 s, 150 MB/s 2024-04-25T10:00:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692473 s, 151 MB/s 2024-04-25T10:00:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652866 s, 161 MB/s 2024-04-25T10:00:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064089 s, 164 MB/s 2024-04-25T10:00:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066611 s, 157 MB/s 2024-04-25T10:00:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656749 s, 160 MB/s 2024-04-25T10:00:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718362 s, 146 MB/s 2024-04-25T10:00:40.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.641 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719528 s, 146 MB/s 2024-04-25T10:00:40.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.650 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832923 s, 126 MB/s 2024-04-25T10:00:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068858 s, 152 MB/s 2024-04-25T10:00:40.667 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.667 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.667 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789464 s, 133 MB/s 2024-04-25T10:00:40.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.675 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715949 s, 146 MB/s 2024-04-25T10:00:40.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.683 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657594 s, 159 MB/s 2024-04-25T10:00:40.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.690 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628138 s, 167 MB/s 2024-04-25T10:00:40.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.697 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659565 s, 159 MB/s 2024-04-25T10:00:40.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:40.699 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:40.699 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:40.699 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:00:40.699 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:40.699 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:40.699 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:40.699 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612075 s, 171 MB/s 2024-04-25T10:00:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711123 s, 147 MB/s 2024-04-25T10:00:40.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.779 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686821 s, 153 MB/s 2024-04-25T10:00:40.788 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.788 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.788 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859677 s, 122 MB/s 2024-04-25T10:00:40.796 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.796 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.796 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708859 s, 148 MB/s 2024-04-25T10:00:40.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.804 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689955 s, 152 MB/s 2024-04-25T10:00:40.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.812 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674046 s, 156 MB/s 2024-04-25T10:00:40.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.820 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694905 s, 151 MB/s 2024-04-25T10:00:40.827 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.827 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.827 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611598 s, 171 MB/s 2024-04-25T10:00:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.835 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676641 s, 155 MB/s 2024-04-25T10:00:40.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716219 s, 146 MB/s 2024-04-25T10:00:40.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.850 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669148 s, 157 MB/s 2024-04-25T10:00:40.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.858 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642784 s, 163 MB/s 2024-04-25T10:00:40.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710174 s, 148 MB/s 2024-04-25T10:00:40.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.873 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693714 s, 151 MB/s 2024-04-25T10:00:40.880 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.881 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.881 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621823 s, 169 MB/s 2024-04-25T10:00:40.888 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.888 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.888 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709256 s, 148 MB/s 2024-04-25T10:00:40.896 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.897 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.897 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072452 s, 145 MB/s 2024-04-25T10:00:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.904 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685163 s, 153 MB/s 2024-04-25T10:00:40.912 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.912 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.912 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635291 s, 165 MB/s 2024-04-25T10:00:40.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.919 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641098 s, 164 MB/s 2024-04-25T10:00:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697553 s, 150 MB/s 2024-04-25T10:00:40.934 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.934 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.935 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658311 s, 159 MB/s 2024-04-25T10:00:40.942 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.942 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.942 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687115 s, 153 MB/s 2024-04-25T10:00:40.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.950 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737805 s, 142 MB/s 2024-04-25T10:00:40.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.959 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739279 s, 142 MB/s 2024-04-25T10:00:40.966 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.966 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.966 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624232 s, 168 MB/s 2024-04-25T10:00:40.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.974 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070362 s, 149 MB/s 2024-04-25T10:00:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710108 s, 148 MB/s 2024-04-25T10:00:40.990 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.990 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.990 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731533 s, 143 MB/s 2024-04-25T10:00:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691949 s, 152 MB/s 2024-04-25T10:00:41.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.006 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743616 s, 141 MB/s 2024-04-25T10:00:41.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:41.008 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:41.008 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:41.009 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:00:41.009 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:41.009 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:41.009 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:41.009 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:41.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.073 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690019 s, 152 MB/s 2024-04-25T10:00:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071883 s, 146 MB/s 2024-04-25T10:00:41.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.089 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722136 s, 145 MB/s 2024-04-25T10:00:41.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.097 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069863 s, 150 MB/s 2024-04-25T10:00:41.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.105 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709384 s, 148 MB/s 2024-04-25T10:00:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691036 s, 152 MB/s 2024-04-25T10:00:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691818 s, 152 MB/s 2024-04-25T10:00:41.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.129 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702641 s, 149 MB/s 2024-04-25T10:00:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680986 s, 154 MB/s 2024-04-25T10:00:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822187 s, 128 MB/s 2024-04-25T10:00:41.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.153 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657341 s, 160 MB/s 2024-04-25T10:00:41.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.163 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696135 s, 151 MB/s 2024-04-25T10:00:41.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686037 s, 153 MB/s 2024-04-25T10:00:41.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.177 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672702 s, 156 MB/s 2024-04-25T10:00:41.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.184 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651106 s, 161 MB/s 2024-04-25T10:00:41.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.193 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079597 s, 132 MB/s 2024-04-25T10:00:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006839 s, 153 MB/s 2024-04-25T10:00:41.208 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.208 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.208 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639709 s, 164 MB/s 2024-04-25T10:00:41.225 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.225 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.225 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155697 s, 67.3 MB/s 2024-04-25T10:00:41.232 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.232 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.232 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654393 s, 160 MB/s 2024-04-25T10:00:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697692 s, 150 MB/s 2024-04-25T10:00:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696782 s, 150 MB/s 2024-04-25T10:00:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705628 s, 149 MB/s 2024-04-25T10:00:41.264 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.264 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.264 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677898 s, 155 MB/s 2024-04-25T10:00:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706427 s, 148 MB/s 2024-04-25T10:00:41.280 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.280 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.280 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695671 s, 151 MB/s 2024-04-25T10:00:41.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.287 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671705 s, 156 MB/s 2024-04-25T10:00:41.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.295 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688621 s, 152 MB/s 2024-04-25T10:00:41.303 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.303 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.303 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715392 s, 147 MB/s 2024-04-25T10:00:41.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.311 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683586 s, 153 MB/s 2024-04-25T10:00:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642958 s, 163 MB/s 2024-04-25T10:00:41.326 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.326 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.326 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658612 s, 159 MB/s 2024-04-25T10:00:41.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:41.327 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:41.328 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:41.328 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:00:41.328 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:41.328 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:41.328 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:41.328 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.393 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669816 s, 157 MB/s 2024-04-25T10:00:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.400 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677694 s, 155 MB/s 2024-04-25T10:00:41.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.408 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653771 s, 160 MB/s 2024-04-25T10:00:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068029 s, 154 MB/s 2024-04-25T10:00:41.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.423 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700286 s, 150 MB/s 2024-04-25T10:00:41.431 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.431 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.431 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710289 s, 148 MB/s 2024-04-25T10:00:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693705 s, 151 MB/s 2024-04-25T10:00:41.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.447 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692591 s, 151 MB/s 2024-04-25T10:00:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694808 s, 151 MB/s 2024-04-25T10:00:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653944 s, 160 MB/s 2024-04-25T10:00:41.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.470 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653278 s, 161 MB/s 2024-04-25T10:00:41.478 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.478 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.478 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075614 s, 139 MB/s 2024-04-25T10:00:41.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.485 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624865 s, 168 MB/s 2024-04-25T10:00:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071715 s, 146 MB/s 2024-04-25T10:00:41.500 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.500 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.500 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609951 s, 172 MB/s 2024-04-25T10:00:41.508 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.509 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.509 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744335 s, 141 MB/s 2024-04-25T10:00:41.515 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.515 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.516 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606147 s, 173 MB/s 2024-04-25T10:00:41.523 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.524 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719907 s, 146 MB/s 2024-04-25T10:00:41.531 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719745 s, 146 MB/s 2024-04-25T10:00:41.539 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.539 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.539 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672368 s, 156 MB/s 2024-04-25T10:00:41.547 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.547 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.547 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697145 s, 150 MB/s 2024-04-25T10:00:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598512 s, 175 MB/s 2024-04-25T10:00:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673986 s, 156 MB/s 2024-04-25T10:00:41.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.570 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688303 s, 152 MB/s 2024-04-25T10:00:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675716 s, 155 MB/s 2024-04-25T10:00:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711839 s, 147 MB/s 2024-04-25T10:00:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669998 s, 157 MB/s 2024-04-25T10:00:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073993 s, 142 MB/s 2024-04-25T10:00:41.609 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.609 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.609 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687113 s, 153 MB/s 2024-04-25T10:00:41.618 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.618 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.618 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756929 s, 139 MB/s 2024-04-25T10:00:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615478 s, 170 MB/s 2024-04-25T10:00:41.633 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.633 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.633 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703879 s, 149 MB/s 2024-04-25T10:00:41.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:41.634 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:41.634 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:41.634 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:00:41.634 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:41.634 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:41.634 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:41.634 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650999 s, 161 MB/s 2024-04-25T10:00:41.707 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.707 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.707 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723948 s, 145 MB/s 2024-04-25T10:00:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690024 s, 152 MB/s 2024-04-25T10:00:41.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.723 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722538 s, 145 MB/s 2024-04-25T10:00:41.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.731 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712079 s, 147 MB/s 2024-04-25T10:00:41.739 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.739 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.739 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696622 s, 151 MB/s 2024-04-25T10:00:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.748 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761076 s, 138 MB/s 2024-04-25T10:00:41.755 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.755 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.756 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737956 s, 142 MB/s 2024-04-25T10:00:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629789 s, 166 MB/s 2024-04-25T10:00:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714842 s, 147 MB/s 2024-04-25T10:00:41.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.778 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637735 s, 164 MB/s 2024-04-25T10:00:41.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.785 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625742 s, 168 MB/s 2024-04-25T10:00:41.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.793 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.793 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630387 s, 166 MB/s 2024-04-25T10:00:41.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.801 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.801 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717291 s, 146 MB/s 2024-04-25T10:00:41.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.809 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694164 s, 151 MB/s 2024-04-25T10:00:41.817 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.817 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.817 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788796 s, 133 MB/s 2024-04-25T10:00:41.825 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.825 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.825 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071971 s, 146 MB/s 2024-04-25T10:00:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.833 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689678 s, 152 MB/s 2024-04-25T10:00:41.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.842 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743415 s, 141 MB/s 2024-04-25T10:00:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626172 s, 167 MB/s 2024-04-25T10:00:41.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.856 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700708 s, 150 MB/s 2024-04-25T10:00:41.864 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728728 s, 144 MB/s 2024-04-25T10:00:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671306 s, 156 MB/s 2024-04-25T10:00:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710295 s, 148 MB/s 2024-04-25T10:00:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661383 s, 159 MB/s 2024-04-25T10:00:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608362 s, 172 MB/s 2024-04-25T10:00:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666809 s, 157 MB/s 2024-04-25T10:00:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717057 s, 146 MB/s 2024-04-25T10:00:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702475 s, 149 MB/s 2024-04-25T10:00:41.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.926 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694567 s, 151 MB/s 2024-04-25T10:00:41.934 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.934 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.934 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686126 s, 153 MB/s 2024-04-25T10:00:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062139 s, 169 MB/s 2024-04-25T10:00:41.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:41.942 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:41.942 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:41.942 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:00:41.942 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:41.943 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:41.943 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:41.943 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061514 s, 170 MB/s 2024-04-25T10:00:42.013 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.014 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.014 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690651 s, 152 MB/s 2024-04-25T10:00:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675971 s, 155 MB/s 2024-04-25T10:00:42.029 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.029 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.029 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649237 s, 162 MB/s 2024-04-25T10:00:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.036 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.037 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691201 s, 152 MB/s 2024-04-25T10:00:42.044 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713483 s, 147 MB/s 2024-04-25T10:00:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063249 s, 166 MB/s 2024-04-25T10:00:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698745 s, 150 MB/s 2024-04-25T10:00:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615823 s, 170 MB/s 2024-04-25T10:00:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717474 s, 146 MB/s 2024-04-25T10:00:42.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.082 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630822 s, 166 MB/s 2024-04-25T10:00:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687889 s, 152 MB/s 2024-04-25T10:00:42.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.097 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629984 s, 166 MB/s 2024-04-25T10:00:42.104 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.105 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666209 s, 157 MB/s 2024-04-25T10:00:42.112 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705045 s, 149 MB/s 2024-04-25T10:00:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.120 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683043 s, 154 MB/s 2024-04-25T10:00:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688769 s, 152 MB/s 2024-04-25T10:00:42.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.137 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803529 s, 130 MB/s 2024-04-25T10:00:42.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.145 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715997 s, 146 MB/s 2024-04-25T10:00:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711118 s, 147 MB/s 2024-04-25T10:00:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691554 s, 152 MB/s 2024-04-25T10:00:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705595 s, 149 MB/s 2024-04-25T10:00:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.178 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765277 s, 137 MB/s 2024-04-25T10:00:42.185 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.186 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071013 s, 148 MB/s 2024-04-25T10:00:42.195 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.195 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.195 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082394 s, 127 MB/s 2024-04-25T10:00:42.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.202 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657409 s, 160 MB/s 2024-04-25T10:00:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697419 s, 150 MB/s 2024-04-25T10:00:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833784 s, 126 MB/s 2024-04-25T10:00:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617503 s, 170 MB/s 2024-04-25T10:00:42.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.234 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645177 s, 163 MB/s 2024-04-25T10:00:42.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.242 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725353 s, 145 MB/s 2024-04-25T10:00:42.249 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.250 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594591 s, 176 MB/s 2024-04-25T10:00:42.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:42.250 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:42.251 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:42.251 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:00:42.251 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:42.251 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:42.251 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:42.251 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:42.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645885 s, 162 MB/s 2024-04-25T10:00:42.322 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.322 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.322 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653664 s, 160 MB/s 2024-04-25T10:00:42.329 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.330 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.330 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685178 s, 153 MB/s 2024-04-25T10:00:42.337 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.337 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.337 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673973 s, 156 MB/s 2024-04-25T10:00:42.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.345 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.345 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065039 s, 161 MB/s 2024-04-25T10:00:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655587 s, 160 MB/s 2024-04-25T10:00:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704038 s, 149 MB/s 2024-04-25T10:00:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.370 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828626 s, 127 MB/s 2024-04-25T10:00:42.376 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.376 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.376 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642918 s, 163 MB/s 2024-04-25T10:00:42.384 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.384 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.384 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688565 s, 152 MB/s 2024-04-25T10:00:42.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.392 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671963 s, 156 MB/s 2024-04-25T10:00:42.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.399 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623791 s, 168 MB/s 2024-04-25T10:00:42.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.407 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695627 s, 151 MB/s 2024-04-25T10:00:42.414 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.414 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.414 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637586 s, 164 MB/s 2024-04-25T10:00:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697137 s, 150 MB/s 2024-04-25T10:00:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692853 s, 151 MB/s 2024-04-25T10:00:42.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.438 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688503 s, 152 MB/s 2024-04-25T10:00:42.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.447 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788338 s, 133 MB/s 2024-04-25T10:00:42.454 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.454 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.454 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680039 s, 154 MB/s 2024-04-25T10:00:42.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657198 s, 160 MB/s 2024-04-25T10:00:42.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.470 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739493 s, 142 MB/s 2024-04-25T10:00:42.478 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.478 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.478 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719856 s, 146 MB/s 2024-04-25T10:00:42.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.485 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618253 s, 170 MB/s 2024-04-25T10:00:42.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.493 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703304 s, 149 MB/s 2024-04-25T10:00:42.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.501 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694702 s, 151 MB/s 2024-04-25T10:00:42.509 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.510 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.510 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762085 s, 138 MB/s 2024-04-25T10:00:42.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.517 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641323 s, 164 MB/s 2024-04-25T10:00:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635193 s, 165 MB/s 2024-04-25T10:00:42.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.532 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699868 s, 150 MB/s 2024-04-25T10:00:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714844 s, 147 MB/s 2024-04-25T10:00:42.548 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.548 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.548 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065092 s, 161 MB/s 2024-04-25T10:00:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728154 s, 144 MB/s 2024-04-25T10:00:42.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:42.557 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:42.557 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:42.557 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:00:42.557 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:42.557 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:42.557 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:42.557 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071948 s, 146 MB/s 2024-04-25T10:00:42.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.631 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716897 s, 146 MB/s 2024-04-25T10:00:42.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.638 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657985 s, 159 MB/s 2024-04-25T10:00:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633834 s, 165 MB/s 2024-04-25T10:00:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671672 s, 156 MB/s 2024-04-25T10:00:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646508 s, 162 MB/s 2024-04-25T10:00:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708278 s, 148 MB/s 2024-04-25T10:00:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697013 s, 150 MB/s 2024-04-25T10:00:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708567 s, 148 MB/s 2024-04-25T10:00:42.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.692 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677323 s, 155 MB/s 2024-04-25T10:00:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636606 s, 165 MB/s 2024-04-25T10:00:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639148 s, 164 MB/s 2024-04-25T10:00:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614034 s, 171 MB/s 2024-04-25T10:00:42.720 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.720 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.720 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060878 s, 172 MB/s 2024-04-25T10:00:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.728 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705623 s, 149 MB/s 2024-04-25T10:00:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.736 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684859 s, 153 MB/s 2024-04-25T10:00:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064728 s, 162 MB/s 2024-04-25T10:00:42.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.751 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658861 s, 159 MB/s 2024-04-25T10:00:42.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.758 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655549 s, 160 MB/s 2024-04-25T10:00:42.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.766 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683303 s, 153 MB/s 2024-04-25T10:00:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.774 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661441 s, 159 MB/s 2024-04-25T10:00:42.780 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.781 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.781 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624511 s, 168 MB/s 2024-04-25T10:00:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645801 s, 162 MB/s 2024-04-25T10:00:42.795 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.795 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.795 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060438 s, 173 MB/s 2024-04-25T10:00:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688508 s, 152 MB/s 2024-04-25T10:00:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696689 s, 151 MB/s 2024-04-25T10:00:42.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.818 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677942 s, 155 MB/s 2024-04-25T10:00:42.825 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.826 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680849 s, 154 MB/s 2024-04-25T10:00:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680581 s, 154 MB/s 2024-04-25T10:00:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006733 s, 156 MB/s 2024-04-25T10:00:42.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672283 s, 156 MB/s 2024-04-25T10:00:42.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.856 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648618 s, 162 MB/s 2024-04-25T10:00:42.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:42.857 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:42.857 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:42.857 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:00:42.857 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:42.857 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:42.858 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:42.858 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:42.922 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.923 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709639 s, 148 MB/s 2024-04-25T10:00:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.931 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735113 s, 143 MB/s 2024-04-25T10:00:42.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.939 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773878 s, 135 MB/s 2024-04-25T10:00:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062597 s, 168 MB/s 2024-04-25T10:00:42.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.955 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812227 s, 129 MB/s 2024-04-25T10:00:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659465 s, 159 MB/s 2024-04-25T10:00:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714865 s, 147 MB/s 2024-04-25T10:00:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712298 s, 147 MB/s 2024-04-25T10:00:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648685 s, 162 MB/s 2024-04-25T10:00:42.993 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:42.993 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:42.993 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625822 s, 168 MB/s 2024-04-25T10:00:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620262 s, 169 MB/s 2024-04-25T10:00:43.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.008 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731266 s, 143 MB/s 2024-04-25T10:00:43.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.016 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639188 s, 164 MB/s 2024-04-25T10:00:43.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.024 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070965 s, 148 MB/s 2024-04-25T10:00:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709265 s, 148 MB/s 2024-04-25T10:00:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746739 s, 140 MB/s 2024-04-25T10:00:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743426 s, 141 MB/s 2024-04-25T10:00:43.056 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.057 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683849 s, 153 MB/s 2024-04-25T10:00:43.065 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.065 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.065 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752791 s, 139 MB/s 2024-04-25T10:00:43.072 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.073 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669386 s, 157 MB/s 2024-04-25T10:00:43.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.081 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712752 s, 147 MB/s 2024-04-25T10:00:43.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.089 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715019 s, 147 MB/s 2024-04-25T10:00:43.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.098 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824029 s, 127 MB/s 2024-04-25T10:00:43.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.108 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832307 s, 126 MB/s 2024-04-25T10:00:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.115 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068352 s, 153 MB/s 2024-04-25T10:00:43.122 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.122 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.123 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635217 s, 165 MB/s 2024-04-25T10:00:43.131 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.131 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.131 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738104 s, 142 MB/s 2024-04-25T10:00:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856172 s, 122 MB/s 2024-04-25T10:00:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641093 s, 164 MB/s 2024-04-25T10:00:43.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.155 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698573 s, 150 MB/s 2024-04-25T10:00:43.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.163 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691746 s, 152 MB/s 2024-04-25T10:00:43.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.171 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694729 s, 151 MB/s 2024-04-25T10:00:43.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:43.172 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:43.172 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:43.173 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:00:43.173 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:43.173 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:43.173 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:43.173 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.239 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720143 s, 146 MB/s 2024-04-25T10:00:43.248 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.248 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.248 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839799 s, 125 MB/s 2024-04-25T10:00:43.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.256 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711492 s, 147 MB/s 2024-04-25T10:00:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.264 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.264 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648276 s, 162 MB/s 2024-04-25T10:00:43.271 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.272 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.272 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708784 s, 148 MB/s 2024-04-25T10:00:43.279 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.279 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.279 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662434 s, 158 MB/s 2024-04-25T10:00:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662958 s, 158 MB/s 2024-04-25T10:00:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612408 s, 171 MB/s 2024-04-25T10:00:43.301 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.301 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.301 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642999 s, 163 MB/s 2024-04-25T10:00:43.309 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.309 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.309 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698753 s, 150 MB/s 2024-04-25T10:00:43.316 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.316 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.316 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660904 s, 159 MB/s 2024-04-25T10:00:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706552 s, 148 MB/s 2024-04-25T10:00:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.332 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631496 s, 166 MB/s 2024-04-25T10:00:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707265 s, 148 MB/s 2024-04-25T10:00:43.347 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.347 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.347 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718876 s, 146 MB/s 2024-04-25T10:00:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707795 s, 148 MB/s 2024-04-25T10:00:43.363 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.363 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.363 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693252 s, 151 MB/s 2024-04-25T10:00:43.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.371 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650383 s, 161 MB/s 2024-04-25T10:00:43.378 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.378 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.379 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700086 s, 150 MB/s 2024-04-25T10:00:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703726 s, 149 MB/s 2024-04-25T10:00:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077245 s, 136 MB/s 2024-04-25T10:00:43.402 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.402 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.403 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662117 s, 158 MB/s 2024-04-25T10:00:43.410 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.410 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.410 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685686 s, 153 MB/s 2024-04-25T10:00:43.418 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.418 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.418 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707465 s, 148 MB/s 2024-04-25T10:00:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641915 s, 163 MB/s 2024-04-25T10:00:43.434 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.434 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.434 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734173 s, 143 MB/s 2024-04-25T10:00:43.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672528 s, 156 MB/s 2024-04-25T10:00:43.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717938 s, 146 MB/s 2024-04-25T10:00:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.457 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067419 s, 156 MB/s 2024-04-25T10:00:43.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.467 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.467 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833511 s, 126 MB/s 2024-04-25T10:00:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688171 s, 152 MB/s 2024-04-25T10:00:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691769 s, 152 MB/s 2024-04-25T10:00:43.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:43.483 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:43.483 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:43.483 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:00:43.484 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:43.484 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:43.484 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:43.484 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:43.548 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.549 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070994 s, 148 MB/s 2024-04-25T10:00:43.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.556 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714929 s, 147 MB/s 2024-04-25T10:00:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753199 s, 139 MB/s 2024-04-25T10:00:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.573 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691865 s, 152 MB/s 2024-04-25T10:00:43.580 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.580 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.580 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674586 s, 155 MB/s 2024-04-25T10:00:43.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.589 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779287 s, 135 MB/s 2024-04-25T10:00:43.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.596 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638624 s, 164 MB/s 2024-04-25T10:00:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699736 s, 150 MB/s 2024-04-25T10:00:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710374 s, 148 MB/s 2024-04-25T10:00:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669492 s, 157 MB/s 2024-04-25T10:00:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632918 s, 166 MB/s 2024-04-25T10:00:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.635 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693587 s, 151 MB/s 2024-04-25T10:00:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006842 s, 153 MB/s 2024-04-25T10:00:43.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071648 s, 146 MB/s 2024-04-25T10:00:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689763 s, 152 MB/s 2024-04-25T10:00:43.665 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.665 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629116 s, 167 MB/s 2024-04-25T10:00:43.673 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.673 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.673 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664228 s, 158 MB/s 2024-04-25T10:00:43.681 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.681 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.681 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690115 s, 152 MB/s 2024-04-25T10:00:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632053 s, 166 MB/s 2024-04-25T10:00:43.695 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.695 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.695 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061075 s, 172 MB/s 2024-04-25T10:00:43.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.703 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695512 s, 151 MB/s 2024-04-25T10:00:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630906 s, 166 MB/s 2024-04-25T10:00:43.717 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.717 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.717 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636235 s, 165 MB/s 2024-04-25T10:00:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626268 s, 167 MB/s 2024-04-25T10:00:43.732 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.732 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.732 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647599 s, 162 MB/s 2024-04-25T10:00:43.739 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.740 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.740 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667392 s, 157 MB/s 2024-04-25T10:00:43.747 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.747 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.747 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635015 s, 165 MB/s 2024-04-25T10:00:43.754 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.754 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.754 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643214 s, 163 MB/s 2024-04-25T10:00:43.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.762 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705951 s, 149 MB/s 2024-04-25T10:00:43.770 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.770 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.771 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747964 s, 140 MB/s 2024-04-25T10:00:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717368 s, 146 MB/s 2024-04-25T10:00:43.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.787 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741647 s, 141 MB/s 2024-04-25T10:00:43.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:43.789 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:43.789 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:43.789 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:00:43.789 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:43.789 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:43.789 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:43.789 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:43.853 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679772 s, 154 MB/s 2024-04-25T10:00:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654652 s, 160 MB/s 2024-04-25T10:00:43.868 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.868 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.868 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652463 s, 161 MB/s 2024-04-25T10:00:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731726 s, 143 MB/s 2024-04-25T10:00:43.884 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.884 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.884 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700569 s, 150 MB/s 2024-04-25T10:00:43.892 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.892 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.893 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731423 s, 143 MB/s 2024-04-25T10:00:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064 s, 164 MB/s 2024-04-25T10:00:43.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.908 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748448 s, 140 MB/s 2024-04-25T10:00:43.916 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.916 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.916 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695073 s, 151 MB/s 2024-04-25T10:00:43.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.924 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.924 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669231 s, 157 MB/s 2024-04-25T10:00:43.931 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.932 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.932 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719831 s, 146 MB/s 2024-04-25T10:00:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757852 s, 138 MB/s 2024-04-25T10:00:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649697 s, 161 MB/s 2024-04-25T10:00:43.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.955 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713383 s, 147 MB/s 2024-04-25T10:00:43.962 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.962 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.962 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601853 s, 174 MB/s 2024-04-25T10:00:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664229 s, 158 MB/s 2024-04-25T10:00:43.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.978 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705547 s, 149 MB/s 2024-04-25T10:00:43.986 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.986 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.986 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726835 s, 144 MB/s 2024-04-25T10:00:43.994 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:43.994 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:43.994 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732321 s, 143 MB/s 2024-04-25T10:00:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719664 s, 146 MB/s 2024-04-25T10:00:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659869 s, 159 MB/s 2024-04-25T10:00:44.017 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.017 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.017 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702351 s, 149 MB/s 2024-04-25T10:00:44.025 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.025 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.025 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655512 s, 160 MB/s 2024-04-25T10:00:44.032 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.032 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.033 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677149 s, 155 MB/s 2024-04-25T10:00:44.040 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.040 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.040 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065343 s, 160 MB/s 2024-04-25T10:00:44.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.047 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649434 s, 161 MB/s 2024-04-25T10:00:44.055 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.055 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.055 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704045 s, 149 MB/s 2024-04-25T10:00:44.063 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.063 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.063 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677798 s, 155 MB/s 2024-04-25T10:00:44.071 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.071 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.071 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712014 s, 147 MB/s 2024-04-25T10:00:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.079 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702909 s, 149 MB/s 2024-04-25T10:00:44.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.087 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635128 s, 165 MB/s 2024-04-25T10:00:44.094 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.094 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.094 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671131 s, 156 MB/s 2024-04-25T10:00:44.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:44.095 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:44.095 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:44.095 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:00:44.095 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:44.096 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:44.096 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:44.096 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646885 s, 162 MB/s 2024-04-25T10:00:44.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816793 s, 128 MB/s 2024-04-25T10:00:44.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.177 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686323 s, 153 MB/s 2024-04-25T10:00:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685125 s, 153 MB/s 2024-04-25T10:00:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073747 s, 142 MB/s 2024-04-25T10:00:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797704 s, 131 MB/s 2024-04-25T10:00:44.211 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.211 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.211 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777674 s, 135 MB/s 2024-04-25T10:00:44.218 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.218 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.218 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664577 s, 158 MB/s 2024-04-25T10:00:44.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.226 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658197 s, 159 MB/s 2024-04-25T10:00:44.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.234 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736034 s, 142 MB/s 2024-04-25T10:00:44.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.242 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725273 s, 145 MB/s 2024-04-25T10:00:44.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.251 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773279 s, 136 MB/s 2024-04-25T10:00:44.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.259 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715672 s, 147 MB/s 2024-04-25T10:00:44.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.266 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612741 s, 171 MB/s 2024-04-25T10:00:44.274 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.274 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.274 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694082 s, 151 MB/s 2024-04-25T10:00:44.282 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.282 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.282 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689105 s, 152 MB/s 2024-04-25T10:00:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669464 s, 157 MB/s 2024-04-25T10:00:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771221 s, 136 MB/s 2024-04-25T10:00:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629148 s, 167 MB/s 2024-04-25T10:00:44.313 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.313 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.313 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712532 s, 147 MB/s 2024-04-25T10:00:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702076 s, 149 MB/s 2024-04-25T10:00:44.328 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.328 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.328 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638427 s, 164 MB/s 2024-04-25T10:00:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.336 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642143 s, 163 MB/s 2024-04-25T10:00:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743773 s, 141 MB/s 2024-04-25T10:00:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.352 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698376 s, 150 MB/s 2024-04-25T10:00:44.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.360 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069178 s, 152 MB/s 2024-04-25T10:00:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.368 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071957 s, 146 MB/s 2024-04-25T10:00:44.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.376 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.376 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698939 s, 150 MB/s 2024-04-25T10:00:44.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.383 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672772 s, 156 MB/s 2024-04-25T10:00:44.391 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.391 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.392 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700494 s, 150 MB/s 2024-04-25T10:00:44.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.400 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712993 s, 147 MB/s 2024-04-25T10:00:44.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.407 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064628 s, 162 MB/s 2024-04-25T10:00:44.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:44.408 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:44.408 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:44.408 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:00:44.408 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:44.408 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:44.408 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:44.408 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659911 s, 159 MB/s 2024-04-25T10:00:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704918 s, 149 MB/s 2024-04-25T10:00:44.488 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.488 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.488 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680555 s, 154 MB/s 2024-04-25T10:00:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.496 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.496 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679081 s, 154 MB/s 2024-04-25T10:00:44.503 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.503 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.503 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637617 s, 164 MB/s 2024-04-25T10:00:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764108 s, 137 MB/s 2024-04-25T10:00:44.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.519 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642825 s, 163 MB/s 2024-04-25T10:00:44.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.527 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735806 s, 143 MB/s 2024-04-25T10:00:44.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.535 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682857 s, 154 MB/s 2024-04-25T10:00:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.542 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.542 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599084 s, 175 MB/s 2024-04-25T10:00:44.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697573 s, 150 MB/s 2024-04-25T10:00:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613327 s, 171 MB/s 2024-04-25T10:00:44.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.564 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067091 s, 156 MB/s 2024-04-25T10:00:44.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.572 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731643 s, 143 MB/s 2024-04-25T10:00:44.580 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.581 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749826 s, 140 MB/s 2024-04-25T10:00:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615698 s, 170 MB/s 2024-04-25T10:00:44.595 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.595 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.595 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629985 s, 166 MB/s 2024-04-25T10:00:44.602 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.602 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.603 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067402 s, 156 MB/s 2024-04-25T10:00:44.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.611 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708104 s, 148 MB/s 2024-04-25T10:00:44.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.619 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728425 s, 144 MB/s 2024-04-25T10:00:44.626 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.627 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687823 s, 152 MB/s 2024-04-25T10:00:44.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.634 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654575 s, 160 MB/s 2024-04-25T10:00:44.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.642 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071629 s, 146 MB/s 2024-04-25T10:00:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748104 s, 140 MB/s 2024-04-25T10:00:44.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.658 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651497 s, 161 MB/s 2024-04-25T10:00:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072231 s, 145 MB/s 2024-04-25T10:00:44.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.674 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701959 s, 149 MB/s 2024-04-25T10:00:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714226 s, 147 MB/s 2024-04-25T10:00:44.690 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.690 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.690 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703545 s, 149 MB/s 2024-04-25T10:00:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636601 s, 165 MB/s 2024-04-25T10:00:44.704 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647821 s, 162 MB/s 2024-04-25T10:00:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707103 s, 148 MB/s 2024-04-25T10:00:44.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:44.714 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:44.714 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:44.714 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:00:44.714 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:44.714 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:44.714 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:44.714 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671437 s, 156 MB/s 2024-04-25T10:00:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.787 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665048 s, 158 MB/s 2024-04-25T10:00:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659436 s, 159 MB/s 2024-04-25T10:00:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718731 s, 146 MB/s 2024-04-25T10:00:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736229 s, 142 MB/s 2024-04-25T10:00:44.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.819 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701566 s, 149 MB/s 2024-04-25T10:00:44.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.826 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655504 s, 160 MB/s 2024-04-25T10:00:44.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.833 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674548 s, 155 MB/s 2024-04-25T10:00:44.840 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.840 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.840 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622021 s, 169 MB/s 2024-04-25T10:00:44.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.848 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653555 s, 160 MB/s 2024-04-25T10:00:44.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.857 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799531 s, 131 MB/s 2024-04-25T10:00:44.864 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.865 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690089 s, 152 MB/s 2024-04-25T10:00:44.874 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.874 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.874 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008388 s, 125 MB/s 2024-04-25T10:00:44.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.883 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078386 s, 134 MB/s 2024-04-25T10:00:44.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.891 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723017 s, 145 MB/s 2024-04-25T10:00:44.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.898 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623308 s, 168 MB/s 2024-04-25T10:00:44.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773609 s, 136 MB/s 2024-04-25T10:00:44.915 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.915 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.916 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797441 s, 131 MB/s 2024-04-25T10:00:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.923 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687859 s, 152 MB/s 2024-04-25T10:00:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.931 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657927 s, 159 MB/s 2024-04-25T10:00:44.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.939 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704087 s, 149 MB/s 2024-04-25T10:00:44.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.946 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631443 s, 166 MB/s 2024-04-25T10:00:44.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.955 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810298 s, 129 MB/s 2024-04-25T10:00:44.962 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.962 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.962 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632111 s, 166 MB/s 2024-04-25T10:00:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672643 s, 156 MB/s 2024-04-25T10:00:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071354 s, 147 MB/s 2024-04-25T10:00:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702267 s, 149 MB/s 2024-04-25T10:00:44.993 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:44.993 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:44.993 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657594 s, 159 MB/s 2024-04-25T10:00:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637034 s, 165 MB/s 2024-04-25T10:00:45.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.008 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668624 s, 157 MB/s 2024-04-25T10:00:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707974 s, 148 MB/s 2024-04-25T10:00:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649154 s, 162 MB/s 2024-04-25T10:00:45.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:45.025 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:45.025 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:45.025 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:00:45.025 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:45.025 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:45.025 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:45.025 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:45.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.091 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726964 s, 144 MB/s 2024-04-25T10:00:45.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.098 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630459 s, 166 MB/s 2024-04-25T10:00:45.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.107 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748129 s, 140 MB/s 2024-04-25T10:00:45.114 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.114 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.114 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064243 s, 163 MB/s 2024-04-25T10:00:45.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.121 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611469 s, 171 MB/s 2024-04-25T10:00:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.128 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626785 s, 167 MB/s 2024-04-25T10:00:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690797 s, 152 MB/s 2024-04-25T10:00:45.143 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.143 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.143 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666793 s, 157 MB/s 2024-04-25T10:00:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673654 s, 156 MB/s 2024-04-25T10:00:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006868 s, 153 MB/s 2024-04-25T10:00:45.166 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.166 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.166 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678713 s, 154 MB/s 2024-04-25T10:00:45.174 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.174 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.174 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628326 s, 167 MB/s 2024-04-25T10:00:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.182 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666035 s, 157 MB/s 2024-04-25T10:00:45.189 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.189 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.189 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656361 s, 160 MB/s 2024-04-25T10:00:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642206 s, 163 MB/s 2024-04-25T10:00:45.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.204 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712873 s, 147 MB/s 2024-04-25T10:00:45.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.212 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685333 s, 153 MB/s 2024-04-25T10:00:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685515 s, 153 MB/s 2024-04-25T10:00:45.228 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.228 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.228 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737408 s, 142 MB/s 2024-04-25T10:00:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633557 s, 166 MB/s 2024-04-25T10:00:45.243 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.243 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.243 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687465 s, 153 MB/s 2024-04-25T10:00:45.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.251 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641314 s, 164 MB/s 2024-04-25T10:00:45.257 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.257 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.257 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619762 s, 169 MB/s 2024-04-25T10:00:45.264 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.265 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.265 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639598 s, 164 MB/s 2024-04-25T10:00:45.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.273 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711614 s, 147 MB/s 2024-04-25T10:00:45.281 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.281 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.281 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716956 s, 146 MB/s 2024-04-25T10:00:45.289 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.289 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.289 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713097 s, 147 MB/s 2024-04-25T10:00:45.296 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654996 s, 160 MB/s 2024-04-25T10:00:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067214 s, 156 MB/s 2024-04-25T10:00:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632532 s, 166 MB/s 2024-04-25T10:00:45.319 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.319 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.319 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708614 s, 148 MB/s 2024-04-25T10:00:45.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.328 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720803 s, 145 MB/s 2024-04-25T10:00:45.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:45.329 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:45.329 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:45.329 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:00:45.329 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:45.329 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:45.329 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:45.329 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:45.394 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.394 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.394 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652613 s, 161 MB/s 2024-04-25T10:00:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634484 s, 165 MB/s 2024-04-25T10:00:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707558 s, 148 MB/s 2024-04-25T10:00:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703511 s, 149 MB/s 2024-04-25T10:00:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.425 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722186 s, 145 MB/s 2024-04-25T10:00:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656686 s, 160 MB/s 2024-04-25T10:00:45.440 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.440 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.440 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660702 s, 159 MB/s 2024-04-25T10:00:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065066 s, 161 MB/s 2024-04-25T10:00:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751471 s, 140 MB/s 2024-04-25T10:00:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863247 s, 121 MB/s 2024-04-25T10:00:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.473 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677636 s, 155 MB/s 2024-04-25T10:00:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677646 s, 155 MB/s 2024-04-25T10:00:45.489 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835787 s, 125 MB/s 2024-04-25T10:00:45.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.498 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689332 s, 152 MB/s 2024-04-25T10:00:45.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.505 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697151 s, 150 MB/s 2024-04-25T10:00:45.512 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.512 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.513 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061643 s, 170 MB/s 2024-04-25T10:00:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668893 s, 157 MB/s 2024-04-25T10:00:45.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.528 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684975 s, 153 MB/s 2024-04-25T10:00:45.536 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.536 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.536 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717848 s, 146 MB/s 2024-04-25T10:00:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675353 s, 155 MB/s 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074473 s, 141 MB/s 2024-04-25T10:00:45.560 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.560 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.560 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070297 s, 149 MB/s 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719609 s, 146 MB/s 2024-04-25T10:00:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.576 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731457 s, 143 MB/s 2024-04-25T10:00:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624781 s, 168 MB/s 2024-04-25T10:00:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724914 s, 145 MB/s 2024-04-25T10:00:45.599 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.599 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.599 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650676 s, 161 MB/s 2024-04-25T10:00:45.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.606 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670661 s, 156 MB/s 2024-04-25T10:00:45.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.615 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678798 s, 154 MB/s 2024-04-25T10:00:45.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.622 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703747 s, 149 MB/s 2024-04-25T10:00:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681812 s, 154 MB/s 2024-04-25T10:00:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763112 s, 137 MB/s 2024-04-25T10:00:45.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:45.639 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:45.640 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:45.640 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:00:45.640 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:45.640 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:45.640 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:45.640 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:45.705 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.705 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.705 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664327 s, 158 MB/s 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652238 s, 161 MB/s 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658013 s, 159 MB/s 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752237 s, 139 MB/s 2024-04-25T10:00:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070528 s, 149 MB/s 2024-04-25T10:00:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717639 s, 146 MB/s 2024-04-25T10:00:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690935 s, 152 MB/s 2024-04-25T10:00:45.760 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.760 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.760 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703923 s, 149 MB/s 2024-04-25T10:00:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071035 s, 148 MB/s 2024-04-25T10:00:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773298 s, 136 MB/s 2024-04-25T10:00:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751574 s, 140 MB/s 2024-04-25T10:00:45.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.792 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060351 s, 174 MB/s 2024-04-25T10:00:45.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.800 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713944 s, 147 MB/s 2024-04-25T10:00:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066988 s, 157 MB/s 2024-04-25T10:00:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636168 s, 165 MB/s 2024-04-25T10:00:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704904 s, 149 MB/s 2024-04-25T10:00:45.830 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.831 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693007 s, 151 MB/s 2024-04-25T10:00:45.839 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.839 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.839 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720581 s, 146 MB/s 2024-04-25T10:00:45.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.847 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773872 s, 135 MB/s 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.856 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704697 s, 149 MB/s 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640431 s, 164 MB/s 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699593 s, 150 MB/s 2024-04-25T10:00:45.878 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.879 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724012 s, 145 MB/s 2024-04-25T10:00:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743633 s, 141 MB/s 2024-04-25T10:00:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699872 s, 150 MB/s 2024-04-25T10:00:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624843 s, 168 MB/s 2024-04-25T10:00:45.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.910 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711784 s, 147 MB/s 2024-04-25T10:00:45.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.919 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806632 s, 130 MB/s 2024-04-25T10:00:45.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.927 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697269 s, 150 MB/s 2024-04-25T10:00:45.935 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.935 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.935 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711335 s, 147 MB/s 2024-04-25T10:00:45.943 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.943 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.944 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713923 s, 147 MB/s 2024-04-25T10:00:45.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:45.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:45.951 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646837 s, 162 MB/s 2024-04-25T10:00:45.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:45.952 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:45.952 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:45.952 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:00:45.952 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:45.952 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:45.952 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:45.952 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:46.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.016 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609381 s, 172 MB/s 2024-04-25T10:00:46.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.024 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713974 s, 147 MB/s 2024-04-25T10:00:46.031 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.031 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.031 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624154 s, 168 MB/s 2024-04-25T10:00:46.039 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.039 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.039 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720035 s, 146 MB/s 2024-04-25T10:00:46.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.046 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627978 s, 167 MB/s 2024-04-25T10:00:46.054 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.054 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.054 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696994 s, 150 MB/s 2024-04-25T10:00:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713038 s, 147 MB/s 2024-04-25T10:00:46.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.070 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071467 s, 147 MB/s 2024-04-25T10:00:46.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.078 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677217 s, 155 MB/s 2024-04-25T10:00:46.085 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.085 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.085 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648348 s, 162 MB/s 2024-04-25T10:00:46.093 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.093 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.093 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687794 s, 152 MB/s 2024-04-25T10:00:46.100 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.101 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.101 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651074 s, 161 MB/s 2024-04-25T10:00:46.109 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.109 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.109 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732148 s, 143 MB/s 2024-04-25T10:00:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715613 s, 147 MB/s 2024-04-25T10:00:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846387 s, 124 MB/s 2024-04-25T10:00:46.134 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.134 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.134 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701077 s, 150 MB/s 2024-04-25T10:00:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.142 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671696 s, 156 MB/s 2024-04-25T10:00:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.150 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070387 s, 149 MB/s 2024-04-25T10:00:46.157 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.157 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.157 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698371 s, 150 MB/s 2024-04-25T10:00:46.165 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.165 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.165 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668836 s, 157 MB/s 2024-04-25T10:00:46.174 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.175 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.175 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862741 s, 122 MB/s 2024-04-25T10:00:46.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.182 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068573 s, 153 MB/s 2024-04-25T10:00:46.190 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.190 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.190 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653965 s, 160 MB/s 2024-04-25T10:00:46.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.198 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742935 s, 141 MB/s 2024-04-25T10:00:46.205 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.205 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.205 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631575 s, 166 MB/s 2024-04-25T10:00:46.213 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.214 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007368 s, 142 MB/s 2024-04-25T10:00:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675596 s, 155 MB/s 2024-04-25T10:00:46.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.229 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700915 s, 150 MB/s 2024-04-25T10:00:46.236 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.236 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.236 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623353 s, 168 MB/s 2024-04-25T10:00:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.244 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673782 s, 156 MB/s 2024-04-25T10:00:46.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694185 s, 151 MB/s 2024-04-25T10:00:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.260 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720064 s, 146 MB/s 2024-04-25T10:00:46.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:46.261 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:46.261 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:46.261 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:00:46.261 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:46.261 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:46.261 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:46.261 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:46.326 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.326 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.326 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704646 s, 149 MB/s 2024-04-25T10:00:46.333 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.333 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.333 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062704 s, 167 MB/s 2024-04-25T10:00:46.341 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.341 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.341 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710203 s, 148 MB/s 2024-04-25T10:00:46.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.348 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615355 s, 170 MB/s 2024-04-25T10:00:46.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.356 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699759 s, 150 MB/s 2024-04-25T10:00:46.363 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.363 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.363 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631754 s, 166 MB/s 2024-04-25T10:00:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.372 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738163 s, 142 MB/s 2024-04-25T10:00:46.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.379 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697745 s, 150 MB/s 2024-04-25T10:00:46.386 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.386 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.386 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063719 s, 165 MB/s 2024-04-25T10:00:46.394 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.394 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.394 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658746 s, 159 MB/s 2024-04-25T10:00:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631663 s, 166 MB/s 2024-04-25T10:00:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006449 s, 163 MB/s 2024-04-25T10:00:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629674 s, 167 MB/s 2024-04-25T10:00:46.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.424 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.424 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720434 s, 146 MB/s 2024-04-25T10:00:46.431 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711425 s, 147 MB/s 2024-04-25T10:00:46.439 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.439 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.439 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619231 s, 169 MB/s 2024-04-25T10:00:46.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.446 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678374 s, 155 MB/s 2024-04-25T10:00:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673918 s, 156 MB/s 2024-04-25T10:00:46.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696665 s, 151 MB/s 2024-04-25T10:00:46.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.469 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627478 s, 167 MB/s 2024-04-25T10:00:46.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.477 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721956 s, 145 MB/s 2024-04-25T10:00:46.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.485 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702215 s, 149 MB/s 2024-04-25T10:00:46.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.493 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721281 s, 145 MB/s 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659586 s, 159 MB/s 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064747 s, 162 MB/s 2024-04-25T10:00:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.516 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721307 s, 145 MB/s 2024-04-25T10:00:46.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.525 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738005 s, 142 MB/s 2024-04-25T10:00:46.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.532 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648968 s, 162 MB/s 2024-04-25T10:00:46.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.540 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702652 s, 149 MB/s 2024-04-25T10:00:46.547 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.547 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.547 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652209 s, 161 MB/s 2024-04-25T10:00:46.554 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638364 s, 164 MB/s 2024-04-25T10:00:46.563 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.564 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726035 s, 144 MB/s 2024-04-25T10:00:46.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:46.564 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:46.565 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:46.565 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:00:46.565 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:46.565 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:46.565 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:46.565 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:46.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.630 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717149 s, 146 MB/s 2024-04-25T10:00:46.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.638 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695994 s, 151 MB/s 2024-04-25T10:00:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742758 s, 141 MB/s 2024-04-25T10:00:46.653 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.653 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.654 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678405 s, 155 MB/s 2024-04-25T10:00:46.661 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.661 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.661 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682045 s, 154 MB/s 2024-04-25T10:00:46.669 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.669 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.669 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723459 s, 145 MB/s 2024-04-25T10:00:46.677 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.677 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.677 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681848 s, 154 MB/s 2024-04-25T10:00:46.685 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.685 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.685 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716419 s, 146 MB/s 2024-04-25T10:00:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.693 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704433 s, 149 MB/s 2024-04-25T10:00:46.701 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.701 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.702 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730315 s, 144 MB/s 2024-04-25T10:00:46.709 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.709 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.709 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006762 s, 155 MB/s 2024-04-25T10:00:46.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.718 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613599 s, 171 MB/s 2024-04-25T10:00:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743944 s, 141 MB/s 2024-04-25T10:00:46.732 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.732 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.732 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691195 s, 152 MB/s 2024-04-25T10:00:46.740 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.740 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.740 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067676 s, 155 MB/s 2024-04-25T10:00:46.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.748 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.748 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648044 s, 162 MB/s 2024-04-25T10:00:46.754 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.755 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.755 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665682 s, 158 MB/s 2024-04-25T10:00:46.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.762 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645819 s, 162 MB/s 2024-04-25T10:00:46.769 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.769 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.769 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643217 s, 163 MB/s 2024-04-25T10:00:46.777 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.777 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.777 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687882 s, 152 MB/s 2024-04-25T10:00:46.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.785 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699026 s, 150 MB/s 2024-04-25T10:00:46.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.794 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781872 s, 134 MB/s 2024-04-25T10:00:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744797 s, 141 MB/s 2024-04-25T10:00:46.810 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.810 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.810 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702923 s, 149 MB/s 2024-04-25T10:00:46.817 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.817 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.818 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063505 s, 165 MB/s 2024-04-25T10:00:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701994 s, 149 MB/s 2024-04-25T10:00:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687147 s, 153 MB/s 2024-04-25T10:00:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689725 s, 152 MB/s 2024-04-25T10:00:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654185 s, 160 MB/s 2024-04-25T10:00:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.857 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714293 s, 147 MB/s 2024-04-25T10:00:46.864 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.865 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720788 s, 145 MB/s 2024-04-25T10:00:46.872 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.872 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.873 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696404 s, 151 MB/s 2024-04-25T10:00:46.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:46.874 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:46.874 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:46.874 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:00:46.874 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:46.874 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:46.874 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:46.874 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:46.938 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.938 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.939 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612639 s, 171 MB/s 2024-04-25T10:00:46.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.946 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666999 s, 157 MB/s 2024-04-25T10:00:46.953 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.953 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.953 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639906 s, 164 MB/s 2024-04-25T10:00:46.961 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.961 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.961 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702313 s, 149 MB/s 2024-04-25T10:00:46.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.969 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683123 s, 153 MB/s 2024-04-25T10:00:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714088 s, 147 MB/s 2024-04-25T10:00:46.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.984 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650196 s, 161 MB/s 2024-04-25T10:00:46.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.992 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673226 s, 156 MB/s 2024-04-25T10:00:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:46.999 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636599 s, 165 MB/s 2024-04-25T10:00:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.006 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636446 s, 165 MB/s 2024-04-25T10:00:47.014 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.014 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.014 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706435 s, 148 MB/s 2024-04-25T10:00:47.022 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.022 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.022 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683549 s, 153 MB/s 2024-04-25T10:00:47.030 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.030 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.030 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722149 s, 145 MB/s 2024-04-25T10:00:47.038 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.038 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.038 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716592 s, 146 MB/s 2024-04-25T10:00:47.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.046 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700819 s, 150 MB/s 2024-04-25T10:00:47.054 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.054 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.054 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701928 s, 149 MB/s 2024-04-25T10:00:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706627 s, 148 MB/s 2024-04-25T10:00:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696838 s, 150 MB/s 2024-04-25T10:00:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.077 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615065 s, 170 MB/s 2024-04-25T10:00:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689123 s, 152 MB/s 2024-04-25T10:00:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730136 s, 144 MB/s 2024-04-25T10:00:47.101 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.102 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.102 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757286 s, 138 MB/s 2024-04-25T10:00:47.110 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.110 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.110 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722026 s, 145 MB/s 2024-04-25T10:00:47.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.118 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735204 s, 143 MB/s 2024-04-25T10:00:47.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.126 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068747 s, 153 MB/s 2024-04-25T10:00:47.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.133 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.133 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645873 s, 162 MB/s 2024-04-25T10:00:47.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.141 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688592 s, 152 MB/s 2024-04-25T10:00:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695712 s, 151 MB/s 2024-04-25T10:00:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718387 s, 146 MB/s 2024-04-25T10:00:47.165 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.165 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.165 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709345 s, 148 MB/s 2024-04-25T10:00:47.173 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.173 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.173 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712046 s, 147 MB/s 2024-04-25T10:00:47.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.180 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067144 s, 156 MB/s 2024-04-25T10:00:47.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:47.182 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:47.182 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:47.182 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:00:47.182 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:47.182 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:47.182 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:47.182 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:47.247 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.247 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.248 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705629 s, 149 MB/s 2024-04-25T10:00:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.255 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067001 s, 157 MB/s 2024-04-25T10:00:47.263 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.263 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.263 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716342 s, 146 MB/s 2024-04-25T10:00:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.270 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647791 s, 162 MB/s 2024-04-25T10:00:47.277 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.278 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.278 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649245 s, 162 MB/s 2024-04-25T10:00:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.286 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728765 s, 144 MB/s 2024-04-25T10:00:47.293 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.293 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.293 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653961 s, 160 MB/s 2024-04-25T10:00:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.302 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.302 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073284 s, 143 MB/s 2024-04-25T10:00:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068632 s, 153 MB/s 2024-04-25T10:00:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832477 s, 126 MB/s 2024-04-25T10:00:47.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718785 s, 146 MB/s 2024-04-25T10:00:47.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.335 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699793 s, 150 MB/s 2024-04-25T10:00:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704422 s, 149 MB/s 2024-04-25T10:00:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064558 s, 162 MB/s 2024-04-25T10:00:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680049 s, 154 MB/s 2024-04-25T10:00:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672502 s, 156 MB/s 2024-04-25T10:00:47.373 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.373 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.373 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066837 s, 157 MB/s 2024-04-25T10:00:47.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.381 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714498 s, 147 MB/s 2024-04-25T10:00:47.388 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.388 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.388 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632832 s, 166 MB/s 2024-04-25T10:00:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890663 s, 118 MB/s 2024-04-25T10:00:47.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.407 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825802 s, 127 MB/s 2024-04-25T10:00:47.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.415 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722956 s, 145 MB/s 2024-04-25T10:00:47.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.423 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070244 s, 149 MB/s 2024-04-25T10:00:47.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.430 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638867 s, 164 MB/s 2024-04-25T10:00:47.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.438 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070783 s, 148 MB/s 2024-04-25T10:00:47.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.447 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077154 s, 136 MB/s 2024-04-25T10:00:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.455 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707421 s, 148 MB/s 2024-04-25T10:00:47.463 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.463 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.463 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701219 s, 150 MB/s 2024-04-25T10:00:47.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.471 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.471 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700225 s, 150 MB/s 2024-04-25T10:00:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741727 s, 141 MB/s 2024-04-25T10:00:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645345 s, 162 MB/s 2024-04-25T10:00:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.495 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.495 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733771 s, 143 MB/s 2024-04-25T10:00:47.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:47.496 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:47.496 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:47.496 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:00:47.496 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:47.496 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:47.496 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:47.496 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701466 s, 149 MB/s 2024-04-25T10:00:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721854 s, 145 MB/s 2024-04-25T10:00:47.578 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.578 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.578 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767243 s, 137 MB/s 2024-04-25T10:00:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632113 s, 166 MB/s 2024-04-25T10:00:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621405 s, 169 MB/s 2024-04-25T10:00:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.602 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765276 s, 137 MB/s 2024-04-25T10:00:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079017 s, 133 MB/s 2024-04-25T10:00:47.618 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.618 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.618 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703668 s, 149 MB/s 2024-04-25T10:00:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.626 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720859 s, 145 MB/s 2024-04-25T10:00:47.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.634 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.634 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684162 s, 153 MB/s 2024-04-25T10:00:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.642 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675089 s, 155 MB/s 2024-04-25T10:00:47.649 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.649 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.650 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705227 s, 149 MB/s 2024-04-25T10:00:47.657 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.657 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.657 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668914 s, 157 MB/s 2024-04-25T10:00:47.664 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.665 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.665 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645784 s, 162 MB/s 2024-04-25T10:00:47.672 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.672 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.673 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720519 s, 146 MB/s 2024-04-25T10:00:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701341 s, 150 MB/s 2024-04-25T10:00:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.689 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071438 s, 147 MB/s 2024-04-25T10:00:47.696 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.696 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.696 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635515 s, 165 MB/s 2024-04-25T10:00:47.704 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.704 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.704 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754682 s, 139 MB/s 2024-04-25T10:00:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694011 s, 151 MB/s 2024-04-25T10:00:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.720 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637338 s, 165 MB/s 2024-04-25T10:00:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647835 s, 162 MB/s 2024-04-25T10:00:47.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.735 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703859 s, 149 MB/s 2024-04-25T10:00:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.743 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636421 s, 165 MB/s 2024-04-25T10:00:47.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.749 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641692 s, 163 MB/s 2024-04-25T10:00:47.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.757 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717334 s, 146 MB/s 2024-04-25T10:00:47.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.766 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736922 s, 142 MB/s 2024-04-25T10:00:47.775 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.775 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.775 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831267 s, 126 MB/s 2024-04-25T10:00:47.782 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.782 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.782 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066485 s, 158 MB/s 2024-04-25T10:00:47.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.791 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727292 s, 144 MB/s 2024-04-25T10:00:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.798 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626761 s, 167 MB/s 2024-04-25T10:00:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637199 s, 165 MB/s 2024-04-25T10:00:47.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:47.807 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:47.807 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:47.807 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:00:47.807 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:47.807 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:47.807 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:47.807 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:47.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.871 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683136 s, 153 MB/s 2024-04-25T10:00:47.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.879 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667637 s, 157 MB/s 2024-04-25T10:00:47.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.887 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072958 s, 144 MB/s 2024-04-25T10:00:47.896 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.896 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.896 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809014 s, 130 MB/s 2024-04-25T10:00:47.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.903 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660458 s, 159 MB/s 2024-04-25T10:00:47.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.911 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636702 s, 165 MB/s 2024-04-25T10:00:47.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.918 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661572 s, 158 MB/s 2024-04-25T10:00:47.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.926 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069663 s, 151 MB/s 2024-04-25T10:00:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642002 s, 163 MB/s 2024-04-25T10:00:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643817 s, 163 MB/s 2024-04-25T10:00:47.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.948 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654652 s, 160 MB/s 2024-04-25T10:00:47.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675238 s, 155 MB/s 2024-04-25T10:00:47.963 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.963 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.963 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691925 s, 152 MB/s 2024-04-25T10:00:47.971 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.971 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.971 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674703 s, 155 MB/s 2024-04-25T10:00:47.980 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.980 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.980 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835257 s, 126 MB/s 2024-04-25T10:00:47.988 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.988 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.988 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685405 s, 153 MB/s 2024-04-25T10:00:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:47.995 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643588 s, 163 MB/s 2024-04-25T10:00:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.003 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649901 s, 161 MB/s 2024-04-25T10:00:48.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.011 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712083 s, 147 MB/s 2024-04-25T10:00:48.020 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.020 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.020 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849624 s, 123 MB/s 2024-04-25T10:00:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704562 s, 149 MB/s 2024-04-25T10:00:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658947 s, 159 MB/s 2024-04-25T10:00:48.044 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.044 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.044 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734079 s, 143 MB/s 2024-04-25T10:00:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696911 s, 150 MB/s 2024-04-25T10:00:48.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.060 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714054 s, 147 MB/s 2024-04-25T10:00:48.071 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.071 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.071 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102524 s, 102 MB/s 2024-04-25T10:00:48.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.082 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916285 s, 114 MB/s 2024-04-25T10:00:48.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.091 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765731 s, 137 MB/s 2024-04-25T10:00:48.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.099 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.099 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713353 s, 147 MB/s 2024-04-25T10:00:48.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.107 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715885 s, 146 MB/s 2024-04-25T10:00:48.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.115 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070215 s, 149 MB/s 2024-04-25T10:00:48.122 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.122 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.123 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669986 s, 157 MB/s 2024-04-25T10:00:48.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:48.124 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:48.124 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:48.124 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:00:48.124 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:48.124 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:48.124 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:48.124 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:48.189 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.189 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.189 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712819 s, 147 MB/s 2024-04-25T10:00:48.197 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.197 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.197 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700367 s, 150 MB/s 2024-04-25T10:00:48.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.205 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661965 s, 158 MB/s 2024-04-25T10:00:48.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.212 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687653 s, 152 MB/s 2024-04-25T10:00:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716229 s, 146 MB/s 2024-04-25T10:00:48.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.228 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.228 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647703 s, 162 MB/s 2024-04-25T10:00:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631133 s, 166 MB/s 2024-04-25T10:00:48.242 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.243 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.243 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673841 s, 156 MB/s 2024-04-25T10:00:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849665 s, 123 MB/s 2024-04-25T10:00:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007066 s, 148 MB/s 2024-04-25T10:00:48.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.268 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701113 s, 150 MB/s 2024-04-25T10:00:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.275 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662664 s, 158 MB/s 2024-04-25T10:00:48.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.283 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701631 s, 149 MB/s 2024-04-25T10:00:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.291 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642055 s, 163 MB/s 2024-04-25T10:00:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731102 s, 143 MB/s 2024-04-25T10:00:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678608 s, 155 MB/s 2024-04-25T10:00:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070484 s, 149 MB/s 2024-04-25T10:00:48.321 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.322 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.322 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629684 s, 167 MB/s 2024-04-25T10:00:48.329 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.330 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.330 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696306 s, 151 MB/s 2024-04-25T10:00:48.337 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.337 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.337 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656887 s, 160 MB/s 2024-04-25T10:00:48.345 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.345 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.345 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070091 s, 150 MB/s 2024-04-25T10:00:48.354 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.354 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.354 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685025 s, 153 MB/s 2024-04-25T10:00:48.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.360 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705291 s, 149 MB/s 2024-04-25T10:00:48.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.370 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877881 s, 119 MB/s 2024-04-25T10:00:48.377 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.377 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.377 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637338 s, 165 MB/s 2024-04-25T10:00:48.385 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.385 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.385 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668962 s, 157 MB/s 2024-04-25T10:00:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.393 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678387 s, 155 MB/s 2024-04-25T10:00:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.401 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703821 s, 149 MB/s 2024-04-25T10:00:48.409 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.409 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.409 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741905 s, 141 MB/s 2024-04-25T10:00:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652212 s, 161 MB/s 2024-04-25T10:00:48.424 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.424 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.424 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709791 s, 148 MB/s 2024-04-25T10:00:48.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.433 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698191 s, 150 MB/s 2024-04-25T10:00:48.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:48.434 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:48.434 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:00:48.434 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:00:48.434 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:48.434 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:48.434 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:00:48.434 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:00:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714149 s, 147 MB/s 2024-04-25T10:00:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832079 s, 126 MB/s 2024-04-25T10:00:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715766 s, 146 MB/s 2024-04-25T10:00:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670554 s, 156 MB/s 2024-04-25T10:00:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083034 s, 126 MB/s 2024-04-25T10:00:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713723 s, 147 MB/s 2024-04-25T10:00:48.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.549 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696241 s, 151 MB/s 2024-04-25T10:00:48.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656216 s, 160 MB/s 2024-04-25T10:00:48.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.564 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696819 s, 150 MB/s 2024-04-25T10:00:48.571 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.571 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.572 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635935 s, 165 MB/s 2024-04-25T10:00:48.580 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.581 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820824 s, 128 MB/s 2024-04-25T10:00:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694229 s, 151 MB/s 2024-04-25T10:00:48.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725702 s, 144 MB/s 2024-04-25T10:00:48.605 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.605 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.605 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766937 s, 137 MB/s 2024-04-25T10:00:48.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.614 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779802 s, 134 MB/s 2024-04-25T10:00:48.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.621 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647547 s, 162 MB/s 2024-04-25T10:00:48.629 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.629 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.629 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712746 s, 147 MB/s 2024-04-25T10:00:48.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.637 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718478 s, 146 MB/s 2024-04-25T10:00:48.646 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.646 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.646 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746346 s, 140 MB/s 2024-04-25T10:00:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748347 s, 140 MB/s 2024-04-25T10:00:48.663 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.663 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.663 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758169 s, 138 MB/s 2024-04-25T10:00:48.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.671 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715292 s, 147 MB/s 2024-04-25T10:00:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.679 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.679 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695426 s, 151 MB/s 2024-04-25T10:00:48.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.686 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681223 s, 154 MB/s 2024-04-25T10:00:48.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.695 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709493 s, 148 MB/s 2024-04-25T10:00:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669645 s, 157 MB/s 2024-04-25T10:00:48.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.710 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073104 s, 143 MB/s 2024-04-25T10:00:48.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.719 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736058 s, 142 MB/s 2024-04-25T10:00:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752017 s, 139 MB/s 2024-04-25T10:00:48.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.734 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635087 s, 165 MB/s 2024-04-25T10:00:48.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.744 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.744 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869755 s, 121 MB/s 2024-04-25T10:00:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:00:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:00:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066076 s, 159 MB/s 2024-04-25T10:00:48.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:48.752 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:00:48.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.894+0000 7fd34453b640 1 Processor -- start 2024-04-25T10:00:48.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.894+0000 7fd34453b640 1 -- start start 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.895+0000 7fd34453b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd33c0d34c0 0x7fd33c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.895+0000 7fd34453b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33c0d2420 con 0x7fd33c0d08e0 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.895+0000 7fd34453b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33c0d2700 con 0x7fd33c15dac0 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.895+0000 7fd34453b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33c0d0d50 con 0x7fd33c0d34c0 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.895+0000 7fd341ad6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd33c0d34c0 0x7fd33c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.895+0000 7fd341ad6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd33c0d34c0 0x7fd33c0d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41012/0 (socket says 172.21.15.19:41012) 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.895+0000 7fd341ad6640 1 -- 172.21.15.19:0/383344305 learned_addr learned my addr 172.21.15.19:0/383344305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.895+0000 7fd342ad8640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd33c0d08e0 0x7fd33c0b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51090/0 (socket says 172.21.15.19:51090) 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.895+0000 7fd341ad6640 1 -- 172.21.15.19:0/383344305 >> v1:172.21.15.112:6789/0 conn(0x7fd33c15dac0 legacy=0x7fd33c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.895+0000 7fd341ad6640 1 -- 172.21.15.19:0/383344305 >> v1:172.21.15.19:6789/0 conn(0x7fd33c0d08e0 legacy=0x7fd33c0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.895+0000 7fd341ad6640 1 -- 172.21.15.19:0/383344305 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c0d29e0 con 0x7fd33c0d34c0 2024-04-25T10:00:48.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.896+0000 7fd341ad6640 1 --2- 172.21.15.19:0/383344305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd33c0d34c0 0x7fd33c0d03a0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009850 tx=0x7fd32c02a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cccc8c97e2c70b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:48.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.896+0000 7fd3412d5640 1 -- 172.21.15.19:0/383344305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c004230 con 0x7fd33c0d34c0 2024-04-25T10:00:48.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.896+0000 7fd3412d5640 1 -- 172.21.15.19:0/383344305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c0043d0 con 0x7fd33c0d34c0 2024-04-25T10:00:48.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.896+0000 7fd3412d5640 1 -- 172.21.15.19:0/383344305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c003810 con 0x7fd33c0d34c0 2024-04-25T10:00:48.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.897+0000 7fd34453b640 1 -- 172.21.15.19:0/383344305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd33c0d34c0 msgr2=0x7fd33c0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:48.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.897+0000 7fd34453b640 1 --2- 172.21.15.19:0/383344305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd33c0d34c0 0x7fd33c0d03a0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009850 tx=0x7fd32c02a370 comp rx=0 tx=0).stop 2024-04-25T10:00:48.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.897+0000 7fd34453b640 1 -- 172.21.15.19:0/383344305 shutdown_connections 2024-04-25T10:00:48.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.897+0000 7fd34453b640 1 --2- 172.21.15.19:0/383344305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd33c0d34c0 0x7fd33c0d03a0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.897+0000 7fd34453b640 1 -- 172.21.15.19:0/383344305 >> 172.21.15.19:0/383344305 conn(0x7fd33c061c30 msgr2=0x7fd33c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:48.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.897+0000 7fd34453b640 1 -- 172.21.15.19:0/383344305 shutdown_connections 2024-04-25T10:00:48.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.897+0000 7fd34453b640 1 -- 172.21.15.19:0/383344305 wait complete. 2024-04-25T10:00:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.898+0000 7fd34453b640 1 Processor -- start 2024-04-25T10:00:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.898+0000 7fd34453b640 1 -- start start 2024-04-25T10:00:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd34453b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd33c0d08e0 0x7fd33c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd34453b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd33c0d34c0 0x7fd33c15ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd34453b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd33c15dac0 0x7fd33c15f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd34453b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c0d2100 con 0x7fd33c0d34c0 2024-04-25T10:00:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd34453b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c0d3030 con 0x7fd33c0d08e0 2024-04-25T10:00:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd34453b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c15dea0 con 0x7fd33c15dac0 2024-04-25T10:00:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd341ad6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd33c0d34c0 0x7fd33c15ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd341ad6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd33c0d34c0 0x7fd33c15ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41014/0 (socket says 172.21.15.19:41014) 2024-04-25T10:00:48.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd341ad6640 1 -- 172.21.15.19:0/2771774732 learned_addr learned my addr 172.21.15.19:0/2771774732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd3422d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd33c0d08e0 0x7fd33c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd342ad8640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd33c15dac0 0x7fd33c15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd341ad6640 1 -- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd33c15dac0 msgr2=0x7fd33c15f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd341ad6640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd33c15dac0 0x7fd33c15f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd341ad6640 1 -- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd33c0d08e0 msgr2=0x7fd33c0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd341ad6640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd33c0d08e0 0x7fd33c0b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.899+0000 7fd341ad6640 1 -- 172.21.15.19:0/2771774732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c160af0 con 0x7fd33c0d34c0 2024-04-25T10:00:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.900+0000 7fd341ad6640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd33c0d34c0 0x7fd33c15ef00 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd32c034040 tx=0x7fd32c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.900+0000 7fd332ffd640 1 -- 172.21.15.19:0/2771774732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c041030 con 0x7fd33c0d34c0 2024-04-25T10:00:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.900+0000 7fd332ffd640 1 -- 172.21.15.19:0/2771774732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c003c50 con 0x7fd33c0d34c0 2024-04-25T10:00:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.900+0000 7fd332ffd640 1 -- 172.21.15.19:0/2771774732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c003e20 con 0x7fd33c0d34c0 2024-04-25T10:00:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.900+0000 7fd34453b640 1 -- 172.21.15.19:0/2771774732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33c15fa50 con 0x7fd33c0d34c0 2024-04-25T10:00:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.900+0000 7fd34453b640 1 -- 172.21.15.19:0/2771774732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33c160010 con 0x7fd33c0d34c0 2024-04-25T10:00:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.901+0000 7fd332ffd640 1 -- 172.21.15.19:0/2771774732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd32c044070 con 0x7fd33c0d34c0 2024-04-25T10:00:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.902+0000 7fd332ffd640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd32006b4d0 0x7fd32006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.902+0000 7fd332ffd640 1 -- 172.21.15.19:0/2771774732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd32c0aade0 con 0x7fd33c0d34c0 2024-04-25T10:00:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.902+0000 7fd3422d7640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd32006b4d0 0x7fd32006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.903+0000 7fd3422d7640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd32006b4d0 0x7fd32006d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd3380016e0 tx=0x7fd338009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.903+0000 7fd347241740 1 Processor -- start 2024-04-25T10:00:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.903+0000 7fd347241740 1 -- start start 2024-04-25T10:00:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.904+0000 7fd347241740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 0x55ee52c8a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.904+0000 7fd347241740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52cc48d0 0x55ee52c885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.904+0000 7fd347241740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.904+0000 7fd347241740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee52bec000 con 0x55ee52cb2cc0 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.904+0000 7fd347241740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee52b94f20 con 0x55ee52c88200 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.904+0000 7fd347241740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee52be80a0 con 0x55ee52cc48d0 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.904+0000 7fd342ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.904+0000 7fd3422d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 0x55ee52c8a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.904+0000 7fd342ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41018/0 (socket says 172.21.15.19:41018) 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.904+0000 7fd342ad8640 1 -- 172.21.15.19:0/1701271854 learned_addr learned my addr 172.21.15.19:0/1701271854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.904+0000 7fd341ad6640 1 --2- 172.21.15.19:0/1701271854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52cc48d0 0x55ee52c885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.905+0000 7fd342ad8640 1 -- 172.21.15.19:0/1701271854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52cc48d0 msgr2=0x55ee52c885e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.905+0000 7fd342ad8640 1 --2- 172.21.15.19:0/1701271854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52cc48d0 0x55ee52c885e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.905+0000 7fd342ad8640 1 -- 172.21.15.19:0/1701271854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 msgr2=0x55ee52c8a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.905+0000 7fd342ad8640 1 --2- 172.21.15.19:0/1701271854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 0x55ee52c8a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.905+0000 7fd342ad8640 1 -- 172.21.15.19:0/1701271854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee529ccba0 con 0x55ee52cb2cc0 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.905+0000 7fd341ad6640 1 --2- 172.21.15.19:0/1701271854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52cc48d0 0x55ee52c885e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:48.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.905+0000 7fd3422d7640 1 --2- 172.21.15.19:0/1701271854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 0x55ee52c8a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.905+0000 7fd342ad8640 1 --2- 172.21.15.19:0/1701271854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb3140 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd334001b10 tx=0x7fd334001fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c164e8a08a38b6c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.905+0000 7fd31ffff640 1 -- 172.21.15.19:0/1701271854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3340077f0 con 0x55ee52cb2cc0 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.905+0000 7fd31ffff640 1 -- 172.21.15.19:0/1701271854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd334018070 con 0x55ee52cb2cc0 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.905+0000 7fd31ffff640 1 -- 172.21.15.19:0/1701271854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3340133f0 con 0x55ee52cb2cc0 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.906+0000 7fd347241740 1 -- 172.21.15.19:0/1701271854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 msgr2=0x55ee52cb3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.906+0000 7fd347241740 1 --2- 172.21.15.19:0/1701271854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb3140 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd334001b10 tx=0x7fd334001fd0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.906+0000 7fd347241740 1 -- 172.21.15.19:0/1701271854 shutdown_connections 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.906+0000 7fd347241740 1 --2- 172.21.15.19:0/1701271854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb3140 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.906+0000 7fd347241740 1 --2- 172.21.15.19:0/1701271854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52cc48d0 0x55ee52c885e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.906+0000 7fd347241740 1 --2- 172.21.15.19:0/1701271854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 0x55ee52c8a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.906+0000 7fd347241740 1 -- 172.21.15.19:0/1701271854 >> 172.21.15.19:0/1701271854 conn(0x55ee52c50050 msgr2=0x55ee52ccdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.906+0000 7fd347241740 1 -- 172.21.15.19:0/1701271854 shutdown_connections 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.906+0000 7fd347241740 1 -- 172.21.15.19:0/1701271854 wait complete. 2024-04-25T10:00:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.906+0000 7fd347241740 1 -- start start 2024-04-25T10:00:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.906+0000 7fd347241740 1 Processor -- start 2024-04-25T10:00:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd347241740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 0x55ee52cacf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd347241740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52cb2cc0 0x55ee52cb3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd347241740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52ca2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd347241740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee52b8a610 con 0x55ee52cc48d0 2024-04-25T10:00:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd347241740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee52cb0810 con 0x55ee52c88200 2024-04-25T10:00:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd347241740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee52cddde0 con 0x55ee52cb2cc0 2024-04-25T10:00:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd347241740 10 client.0 _ll_register_callbacks cb 0x55ee52ba2f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd342ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52ca2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd3422d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 0x55ee52cacf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd342ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52ca2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41034/0 (socket says 172.21.15.19:41034) 2024-04-25T10:00:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd342ad8640 1 -- 172.21.15.19:0/1727730089 learned_addr learned my addr 172.21.15.19:0/1727730089 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd341ad6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52cb2cc0 0x55ee52cb3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.907+0000 7fd3422d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 0x55ee52cacf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:47520/0 (socket says 172.21.15.19:47520) 2024-04-25T10:00:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd342ad8640 1 -- 172.21.15.19:0/1727730089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52cb2cc0 msgr2=0x55ee52cb3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd342ad8640 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52cb2cc0 0x55ee52cb3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd342ad8640 1 -- 172.21.15.19:0/1727730089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 msgr2=0x55ee52cacf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd342ad8640 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 0x55ee52cacf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd342ad8640 1 -- 172.21.15.19:0/1727730089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee528c0490 con 0x55ee52cc48d0 2024-04-25T10:00:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd3422d7640 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 0x55ee52cacf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd342ad8640 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52ca2610 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd3340012e0 tx=0x7fd334001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a1ffa110a5ffa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd31d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd31d7fa640 1 -- 172.21.15.19:0/1727730089 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334013880 con 0x55ee52cc48d0 2024-04-25T10:00:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd31d7fa640 1 -- 172.21.15.19:0/1727730089 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd334013a20 con 0x55ee52cc48d0 2024-04-25T10:00:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd347241740 1 -- 172.21.15.19:0/1727730089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd334018070 con 0x55ee52cc48d0 2024-04-25T10:00:48.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd31d7fa640 1 -- 172.21.15.19:0/1727730089 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd33402c050 con 0x55ee52cc48d0 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd347241740 10 client.8760 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd347241740 1 -- 172.21.15.19:0/1727730089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55ee52a2d330 con 0x55ee52cc48d0 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd31d7fa640 1 -- 172.21.15.19:0/1727730089 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd334018070 con 0x55ee52cc48d0 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.908+0000 7fd31d7fa640 1 -- 172.21.15.19:0/1727730089 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd33402f020 con 0x55ee52cc48d0 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.909+0000 7fd347241740 10 client.8760 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.909+0000 7fd347241740 10 client.8760 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.909+0000 7fd347241740 20 client.8760 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.909+0000 7fd347241740 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ee52cc7700 0x55ee52cc5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.909+0000 7fd347241740 4 client.8760 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.909+0000 7fd347241740 1 -- 172.21.15.19:0/1727730089 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee52bf9540 con 0x55ee52cc7700 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.909+0000 7fd3422d7640 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ee52cc7700 0x55ee52cc5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.909+0000 7fd3422d7640 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ee52cc7700 0x55ee52cc5b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb91e5ce80c87a7e server_cookie=77eb0f46f2c582c6 in_seq=0 out_seq=0 2024-04-25T10:00:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.909+0000 7fd31d7fa640 10 client.8760 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd31d7fa640 1 -- 172.21.15.19:0/1727730089 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee52bf9540 con 0x55ee52cc7700 2024-04-25T10:00:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd31d7fa640 10 client.8760 handle_command_reply: tid=0 2024-04-25T10:00:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 1 client.8760 shutdown 2024-04-25T10:00:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 1 -- 172.21.15.19:0/1727730089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 msgr2=0x55ee52ca2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52ca2610 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd3340012e0 tx=0x7fd334001250 comp rx=0 tx=0).stop 2024-04-25T10:00:48.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 1 -- 172.21.15.19:0/1727730089 shutdown_connections 2024-04-25T10:00:48.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ee52cc7700 0x55ee52cc5b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52ca2610 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52cb2cc0 0x55ee52cb3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 1 --2- 172.21.15.19:0/1727730089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52c88200 0x55ee52cacf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd31d7fa640 0 client.8760 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:48.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 1 -- 172.21.15.19:0/1727730089 >> 172.21.15.19:0/1727730089 conn(0x55ee52c50050 msgr2=0x55ee52cb0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:48.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 1 -- 172.21.15.19:0/1727730089 shutdown_connections 2024-04-25T10:00:48.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 1 -- 172.21.15.19:0/1727730089 wait complete. 2024-04-25T10:00:48.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.910+0000 7fd347241740 20 client.8760 trim_cache size 0 max 16384 2024-04-25T10:00:48.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.925+0000 7fd347241740 1 Processor -- start 2024-04-25T10:00:48.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.925+0000 7fd347241740 1 -- start start 2024-04-25T10:00:48.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd347241740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52c88200 0x55ee52d2f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd347241740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd347241740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52cb1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd347241740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c200720 con 0x55ee52cc48d0 2024-04-25T10:00:48.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd347241740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c1ff920 con 0x55ee52cb2cc0 2024-04-25T10:00:48.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd347241740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c154700 con 0x55ee52c88200 2024-04-25T10:00:48.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd342ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52cb1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd342ad8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52cb1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41040/0 (socket says 172.21.15.19:41040) 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd342ad8640 1 -- 172.21.15.19:0/3885427379 learned_addr learned my addr 172.21.15.19:0/3885427379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd341ad6640 1 --2- 172.21.15.19:0/3885427379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd3422d7640 1 --2- 172.21.15.19:0/3885427379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52c88200 0x55ee52d2f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd342ad8640 1 -- 172.21.15.19:0/3885427379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52c88200 msgr2=0x55ee52d2f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd342ad8640 1 --2- 172.21.15.19:0/3885427379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52c88200 0x55ee52d2f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd342ad8640 1 -- 172.21.15.19:0/3885427379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52cb2cc0 msgr2=0x55ee52cb1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd342ad8640 1 --2- 172.21.15.19:0/3885427379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd342ad8640 1 -- 172.21.15.19:0/3885427379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee52899610 con 0x55ee52cc48d0 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.926+0000 7fd342ad8640 1 --2- 172.21.15.19:0/3885427379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52cb1c60 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd334008fa0 tx=0x7fd33401f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109c973beea7be09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd31f7fe640 1 -- 172.21.15.19:0/3885427379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334007b20 con 0x55ee52cc48d0 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd31f7fe640 1 -- 172.21.15.19:0/3885427379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd334018020 con 0x55ee52cc48d0 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd31f7fe640 1 -- 172.21.15.19:0/3885427379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334013ae0 con 0x55ee52cc48d0 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd347241740 1 -- 172.21.15.19:0/3885427379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 msgr2=0x55ee52cb1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd347241740 1 --2- 172.21.15.19:0/3885427379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52cb1c60 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd334008fa0 tx=0x7fd33401f070 comp rx=0 tx=0).stop 2024-04-25T10:00:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd347241740 1 -- 172.21.15.19:0/3885427379 shutdown_connections 2024-04-25T10:00:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd347241740 1 --2- 172.21.15.19:0/3885427379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cc48d0 0x55ee52cb1c60 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd347241740 1 --2- 172.21.15.19:0/3885427379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd347241740 1 --2- 172.21.15.19:0/3885427379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52c88200 0x55ee52d2f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd347241740 1 -- 172.21.15.19:0/3885427379 >> 172.21.15.19:0/3885427379 conn(0x55ee52c50050 msgr2=0x55ee52cc6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd347241740 1 -- 172.21.15.19:0/3885427379 shutdown_connections 2024-04-25T10:00:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.927+0000 7fd347241740 1 -- 172.21.15.19:0/3885427379 wait complete. 2024-04-25T10:00:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.928+0000 7fd347241740 1 -- start start 2024-04-25T10:00:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.928+0000 7fd347241740 1 Processor -- start 2024-04-25T10:00:48.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd347241740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52c88200 0x55ee52d27af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd347241740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd347241740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52cc48d0 0x55ee52cb1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd347241740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee52cad040 con 0x55ee52cb2cc0 2024-04-25T10:00:48.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd347241740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c154d00 con 0x55ee52cc48d0 2024-04-25T10:00:48.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd347241740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee52cdd690 con 0x55ee52c88200 2024-04-25T10:00:48.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd347241740 10 client.0 _ll_register_callbacks cb 0x7fd334004c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:48.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd341ad6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd341ad6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41050/0 (socket says 172.21.15.19:41050) 2024-04-25T10:00:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd341ad6640 1 -- 172.21.15.19:0/2712357607 learned_addr learned my addr 172.21.15.19:0/2712357607 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd3422d7640 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52c88200 0x55ee52d27af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd342ad8640 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52cc48d0 0x55ee52cb1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd341ad6640 1 -- 172.21.15.19:0/2712357607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52c88200 msgr2=0x55ee52d27af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd341ad6640 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52c88200 0x55ee52d27af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd341ad6640 1 -- 172.21.15.19:0/2712357607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52cc48d0 msgr2=0x55ee52cb1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd341ad6640 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52cc48d0 0x55ee52cb1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd341ad6640 1 -- 172.21.15.19:0/2712357607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee52c87e40 con 0x55ee52cb2cc0 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd341ad6640 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb16d0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fd32c079530 tx=0x7fd32c07d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be27903f36430119 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd31d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd31d7fa640 1 -- 172.21.15.19:0/2712357607 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c0789f0 con 0x55ee52cb2cc0 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd31d7fa640 1 -- 172.21.15.19:0/2712357607 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd32c078b90 con 0x55ee52cb2cc0 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.929+0000 7fd347241740 1 -- 172.21.15.19:0/2712357607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd334018020 con 0x55ee52cb2cc0 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd31d7fa640 1 -- 172.21.15.19:0/2712357607 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd32c07d0c0 con 0x55ee52cb2cc0 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd347241740 10 client.8766 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd347241740 1 -- 172.21.15.19:0/2712357607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55ee52caea20 con 0x55ee52cb2cc0 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd31d7fa640 1 -- 172.21.15.19:0/2712357607 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd334018020 con 0x55ee52cb2cc0 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd31d7fa640 1 -- 172.21.15.19:0/2712357607 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd32c04e060 con 0x55ee52cb2cc0 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd347241740 10 client.8766 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd347241740 10 client.8766 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd347241740 20 client.8766 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd347241740 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ee52ce7a00 0x55ee52cc3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd347241740 4 client.8766 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd347241740 1 -- 172.21.15.19:0/2712357607 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55ee52c89620 con 0x55ee52ce7a00 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd3422d7640 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ee52ce7a00 0x55ee52cc3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd31d7fa640 10 client.8766 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.930+0000 7fd3422d7640 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ee52ce7a00 0x55ee52cc3650 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c43695bcaca78 server_cookie=6cac36c109ce4e81 in_seq=0 out_seq=0 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd31d7fa640 1 -- 172.21.15.19:0/2712357607 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ee52c89620 con 0x55ee52ce7a00 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd31d7fa640 10 client.8766 handle_command_reply: tid=0 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 1 client.8766 shutdown 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 1 -- 172.21.15.19:0/2712357607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 msgr2=0x55ee52cb16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb16d0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fd32c079530 tx=0x7fd32c07d640 comp rx=0 tx=0).stop 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 1 -- 172.21.15.19:0/2712357607 shutdown_connections 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ee52ce7a00 0x55ee52cc3650 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee52cc48d0 0x55ee52cb1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee52cb2cc0 0x55ee52cb16d0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd31d7fa640 0 client.8766 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 1 --2- 172.21.15.19:0/2712357607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ee52c88200 0x55ee52d27af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 1 -- 172.21.15.19:0/2712357607 >> 172.21.15.19:0/2712357607 conn(0x55ee52c50050 msgr2=0x55ee52cc55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 1 -- 172.21.15.19:0/2712357607 shutdown_connections 2024-04-25T10:00:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 1 -- 172.21.15.19:0/2712357607 wait complete. 2024-04-25T10:00:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.933+0000 7fd347241740 20 client.8766 trim_cache size 0 max 16384 2024-04-25T10:00:48.945 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:48.945 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:00:48.945 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "f3e5034a-13b7-47d5-a470-51dbfdb9fc5c", 2024-04-25T10:00:48.945 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:00:48.945 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:48.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 -- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd32006b4d0 msgr2=0x7fd32006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd32006b4d0 0x7fd32006d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd3380016e0 tx=0x7fd338009040 comp rx=0 tx=0).stop 2024-04-25T10:00:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 -- 172.21.15.19:0/2771774732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd33c0d34c0 msgr2=0x7fd33c15ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd33c0d34c0 0x7fd33c15ef00 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd32c034040 tx=0x7fd32c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 -- 172.21.15.19:0/2771774732 shutdown_connections 2024-04-25T10:00:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd32006b4d0 0x7fd32006d990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd33c15dac0 0x7fd33c15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd33c0d34c0 0x7fd33c15ef00 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 --2- 172.21.15.19:0/2771774732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd33c0d08e0 0x7fd33c0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 -- 172.21.15.19:0/2771774732 >> 172.21.15.19:0/2771774732 conn(0x7fd33c061c30 msgr2=0x7fd33c0b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 -- 172.21.15.19:0/2771774732 shutdown_connections 2024-04-25T10:00:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:48.934+0000 7fd34453b640 1 -- 172.21.15.19:0/2771774732 wait complete. 2024-04-25T10:00:48.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:48.954 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T10:00:49.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.085+0000 7f72d3b89640 1 Processor -- start 2024-04-25T10:00:49.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.085+0000 7f72d3b89640 1 -- start start 2024-04-25T10:00:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d3b89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc15dc20 0x7f72cc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d3b89640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f72cc0d32f0 0x7f72cc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d3b89640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc15e000 con 0x7f72cc161710 2024-04-25T10:00:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d3b89640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72cc0d1de0 con 0x7f72cc0d32f0 2024-04-25T10:00:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d3b89640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc161c60 con 0x7f72cc15dc20 2024-04-25T10:00:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d2126640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f72cc0d32f0 0x7f72cc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d1925640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc15dc20 0x7f72cc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d1124640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f72cc161710 0x7f72cc06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:57182/0 (socket says 172.21.15.19:57182) 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d1124640 1 -- 172.21.15.19:0/2158236279 learned_addr learned my addr 172.21.15.19:0/2158236279 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d0923640 1 -- 172.21.15.19:0/2158236279 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402181762 0 0) 0x7f72cc15e000 con 0x7f72cc161710 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d1925640 1 -- 172.21.15.19:0/2158236279 >> v2:172.21.15.19:3300/0 conn(0x7f72cc0d32f0 msgr2=0x7f72cc0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d1925640 1 --2- 172.21.15.19:0/2158236279 >> v2:172.21.15.19:3300/0 conn(0x7f72cc0d32f0 0x7f72cc0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.086+0000 7f72d1925640 1 -- 172.21.15.19:0/2158236279 >> v1:172.21.15.112:6789/0 conn(0x7f72cc161710 legacy=0x7f72cc06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.087+0000 7f72d1925640 1 -- 172.21.15.19:0/2158236279 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc161f40 con 0x7f72cc15dc20 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.087+0000 7f72d2126640 1 --2- 172.21.15.19:0/2158236279 >> v2:172.21.15.19:3300/0 conn(0x7f72cc0d32f0 0x7f72cc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.087+0000 7f72d1925640 1 --2- 172.21.15.19:0/2158236279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc15dc20 0x7f72cc0ce670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f72c80097a0 tx=0x7f72c802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e15b4916dae5fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.088+0000 7f72d0923640 1 -- 172.21.15.19:0/2158236279 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8037070 con 0x7f72cc15dc20 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.088+0000 7f72d0923640 1 -- 172.21.15.19:0/2158236279 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c8032440 con 0x7f72cc15dc20 2024-04-25T10:00:49.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.088+0000 7f72d0923640 1 -- 172.21.15.19:0/2158236279 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8032810 con 0x7f72cc15dc20 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.088+0000 7f72d3b89640 1 -- 172.21.15.19:0/2158236279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc15dc20 msgr2=0x7f72cc0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.088+0000 7f72d3b89640 1 --2- 172.21.15.19:0/2158236279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc15dc20 0x7f72cc0ce670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f72c80097a0 tx=0x7f72c802a180 comp rx=0 tx=0).stop 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.088+0000 7f72d3b89640 1 -- 172.21.15.19:0/2158236279 shutdown_connections 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.088+0000 7f72d3b89640 1 --2- 172.21.15.19:0/2158236279 >> v2:172.21.15.19:3300/0 conn(0x7f72cc0d32f0 0x7f72cc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.088+0000 7f72d3b89640 1 --2- 172.21.15.19:0/2158236279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc15dc20 0x7f72cc0ce670 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.088+0000 7f72d3b89640 1 -- 172.21.15.19:0/2158236279 >> 172.21.15.19:0/2158236279 conn(0x7f72cc0d0420 msgr2=0x7f72cc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.088+0000 7f72d3b89640 1 -- 172.21.15.19:0/2158236279 shutdown_connections 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.088+0000 7f72d3b89640 1 -- 172.21.15.19:0/2158236279 wait complete. 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.090+0000 7f72d3b89640 1 Processor -- start 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.090+0000 7f72d3b89640 1 -- start start 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.090+0000 7f72d3b89640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72cc0d32f0 0x7f72cc0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.090+0000 7f72d3b89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72cc15dc20 0x7f72cc146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d3b89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc161710 0x7f72cc148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d3b89640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0b5e50 con 0x7f72cc0d32f0 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d3b89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc162500 con 0x7f72cc15dc20 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d3b89640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0b5fd0 con 0x7f72cc161710 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d1925640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72cc0d32f0 0x7f72cc0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d1925640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72cc0d32f0 0x7f72cc0cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41060/0 (socket says 172.21.15.19:41060) 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d1925640 1 -- 172.21.15.19:0/1455064852 learned_addr learned my addr 172.21.15.19:0/1455064852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d2126640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc161710 0x7f72cc148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d1124640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72cc15dc20 0x7f72cc146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d1925640 1 -- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc161710 msgr2=0x7f72cc148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d1925640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc161710 0x7f72cc148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d1925640 1 -- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72cc15dc20 msgr2=0x7f72cc146c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d1925640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72cc15dc20 0x7f72cc146c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d1925640 1 -- 172.21.15.19:0/1455064852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc1595c0 con 0x7f72cc0d32f0 2024-04-25T10:00:49.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d2126640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc161710 0x7f72cc148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:49.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d1925640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72cc0d32f0 0x7f72cc0cb120 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f72c80097a0 tx=0x7f72c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72c27fc640 1 -- 172.21.15.19:0/1455064852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8041070 con 0x7f72cc0d32f0 2024-04-25T10:00:49.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72c27fc640 1 -- 172.21.15.19:0/1455064852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c803c870 con 0x7f72cc0d32f0 2024-04-25T10:00:49.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d3b89640 1 -- 172.21.15.19:0/1455064852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc149400 con 0x7f72cc0d32f0 2024-04-25T10:00:49.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72c27fc640 1 -- 172.21.15.19:0/1455064852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8037040 con 0x7f72cc0d32f0 2024-04-25T10:00:49.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.091+0000 7f72d3b89640 1 -- 172.21.15.19:0/1455064852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc1499c0 con 0x7f72cc0d32f0 2024-04-25T10:00:49.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.093+0000 7f72c27fc640 1 -- 172.21.15.19:0/1455064852 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72c8044070 con 0x7f72cc0d32f0 2024-04-25T10:00:49.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.093+0000 7f72c27fc640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f72b006b4d0 0x7f72b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.093+0000 7f72c27fc640 1 -- 172.21.15.19:0/1455064852 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72c80aac70 con 0x7f72cc0d32f0 2024-04-25T10:00:49.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.093+0000 7f72d1124640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f72b006b4d0 0x7f72b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.094+0000 7f72d1124640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f72b006b4d0 0x7f72b006d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f72cc156ca0 tx=0x7f72bc007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.094+0000 7f72d696e740 1 Processor -- start 2024-04-25T10:00:49.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.094+0000 7f72d696e740 1 -- start start 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d696e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9b5ca0 0x555b7b9b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d696e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9f2370 0x555b7b9b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d696e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9e0760 0x555b7b9e0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d696e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b7b918b90 con 0x555b7b9e0760 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d696e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b7b8c1e40 con 0x555b7b9f2370 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d696e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b7b914c30 con 0x555b7b9b5ca0 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d2126640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9e0760 0x555b7b9e0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d1925640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9b5ca0 0x555b7b9b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d1925640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9b5ca0 0x555b7b9b7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:44298/0 (socket says 172.21.15.19:44298) 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d1925640 1 -- 172.21.15.19:0/779585487 learned_addr learned my addr 172.21.15.19:0/779585487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d1124640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9f2370 0x555b7b9b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d1925640 1 -- 172.21.15.19:0/779585487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9f2370 msgr2=0x555b7b9b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d1925640 1 --2- 172.21.15.19:0/779585487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9f2370 0x555b7b9b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d1925640 1 -- 172.21.15.19:0/779585487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9e0760 msgr2=0x555b7b9e0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d1925640 1 --2- 172.21.15.19:0/779585487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9e0760 0x555b7b9e0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.095+0000 7f72d1925640 1 -- 172.21.15.19:0/779585487 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b7b6f77d0 con 0x555b7b9b5ca0 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.096+0000 7f72d1925640 1 --2- 172.21.15.19:0/779585487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9b5ca0 0x555b7b9b7e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f72c80a9040 tx=0x7f72c8079230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=371929e8cdb9142b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.096+0000 7f72af7fe640 1 -- 172.21.15.19:0/779585487 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c804b070 con 0x555b7b9b5ca0 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.096+0000 7f72af7fe640 1 -- 172.21.15.19:0/779585487 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c807d440 con 0x555b7b9b5ca0 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.096+0000 7f72af7fe640 1 -- 172.21.15.19:0/779585487 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8081a00 con 0x555b7b9b5ca0 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.096+0000 7f72d696e740 1 -- 172.21.15.19:0/779585487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9b5ca0 msgr2=0x555b7b9b7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.096+0000 7f72d696e740 1 --2- 172.21.15.19:0/779585487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9b5ca0 0x555b7b9b7e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f72c80a9040 tx=0x7f72c8079230 comp rx=0 tx=0).stop 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.096+0000 7f72d696e740 1 -- 172.21.15.19:0/779585487 shutdown_connections 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.096+0000 7f72d696e740 1 --2- 172.21.15.19:0/779585487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9e0760 0x555b7b9e0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.096+0000 7f72d696e740 1 --2- 172.21.15.19:0/779585487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9f2370 0x555b7b9b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.096+0000 7f72d696e740 1 --2- 172.21.15.19:0/779585487 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9b5ca0 0x555b7b9b7e00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.096+0000 7f72d696e740 1 -- 172.21.15.19:0/779585487 >> 172.21.15.19:0/779585487 conn(0x555b7b97d870 msgr2=0x555b7b9fb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.097+0000 7f72d696e740 1 -- 172.21.15.19:0/779585487 shutdown_connections 2024-04-25T10:00:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.097+0000 7f72d696e740 1 -- 172.21.15.19:0/779585487 wait complete. 2024-04-25T10:00:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.097+0000 7f72d696e740 1 -- start start 2024-04-25T10:00:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.097+0000 7f72d696e740 1 Processor -- start 2024-04-25T10:00:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d696e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9b5ca0 0x555b7b9e0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d696e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9e0760 0x555b7b9fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d696e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9f2370 0x555b7b9fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d696e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b7b914c30 con 0x555b7b9f2370 2024-04-25T10:00:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d696e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b7b8d53e0 con 0x555b7b9b5ca0 2024-04-25T10:00:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d696e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b7ba0ccf0 con 0x555b7b9e0760 2024-04-25T10:00:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d696e740 10 client.0 _ll_register_callbacks cb 0x555b7b8d4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d2126640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9f2370 0x555b7b9fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d2126640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9f2370 0x555b7b9fece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41078/0 (socket says 172.21.15.19:41078) 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d2126640 1 -- 172.21.15.19:0/1132053885 learned_addr learned my addr 172.21.15.19:0/1132053885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d1124640 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9e0760 0x555b7b9fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.098+0000 7f72d1925640 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9b5ca0 0x555b7b9e0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d2126640 1 -- 172.21.15.19:0/1132053885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9e0760 msgr2=0x555b7b9fe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d2126640 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9e0760 0x555b7b9fe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d2126640 1 -- 172.21.15.19:0/1132053885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9b5ca0 msgr2=0x555b7b9e0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d2126640 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9b5ca0 0x555b7b9e0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d2126640 1 -- 172.21.15.19:0/1132053885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b7b5ed0f0 con 0x555b7b9f2370 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d2126640 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9f2370 0x555b7b9fece0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f72c40043b0 tx=0x7f72c4004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc562f250b479ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72acff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72acff9640 1 -- 172.21.15.19:0/1132053885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c401e400 con 0x555b7b9f2370 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72acff9640 1 -- 172.21.15.19:0/1132053885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72c401fde0 con 0x555b7b9f2370 2024-04-25T10:00:49.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d696e740 1 -- 172.21.15.19:0/1132053885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72c807d440 con 0x555b7b9f2370 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72acff9640 1 -- 172.21.15.19:0/1132053885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72c4016400 con 0x555b7b9f2370 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d696e740 10 client.8778 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d696e740 1 -- 172.21.15.19:0/1132053885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x555b7b757f60 con 0x555b7b9f2370 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72acff9640 1 -- 172.21.15.19:0/1132053885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c807d440 con 0x555b7b9f2370 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72acff9640 1 -- 172.21.15.19:0/1132053885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f72c4026050 con 0x555b7b9f2370 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d696e740 10 client.8778 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d696e740 10 client.8778 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d696e740 20 client.8778 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.099+0000 7f72d696e740 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x555b7b9de200 0x555b7b9e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.100+0000 7f72d1925640 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x555b7b9de200 0x555b7b9e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.100+0000 7f72d696e740 4 client.8778 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.100+0000 7f72d696e740 1 -- 172.21.15.19:0/1132053885 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b7b9260d0 con 0x555b7b9de200 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.100+0000 7f72acff9640 10 client.8778 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:49.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.100+0000 7f72d1925640 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x555b7b9de200 0x555b7b9e35c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e6775521af9a69 server_cookie=9aa35fbbcd394a82 in_seq=0 out_seq=0 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.100+0000 7f72acff9640 1 -- 172.21.15.19:0/1132053885 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555b7b9260d0 con 0x555b7b9de200 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.100+0000 7f72acff9640 10 client.8778 handle_command_reply: tid=0 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 1 client.8778 shutdown 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 1 -- 172.21.15.19:0/1132053885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9f2370 msgr2=0x555b7b9fece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9f2370 0x555b7b9fece0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f72c40043b0 tx=0x7f72c4004870 comp rx=0 tx=0).stop 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 1 -- 172.21.15.19:0/1132053885 shutdown_connections 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x555b7b9de200 0x555b7b9e35c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72acff9640 0 client.8778 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9f2370 0x555b7b9fece0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9e0760 0x555b7b9fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 1 --2- 172.21.15.19:0/1132053885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9b5ca0 0x555b7b9e0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 1 -- 172.21.15.19:0/1132053885 >> 172.21.15.19:0/1132053885 conn(0x555b7b97d870 msgr2=0x555b7b9b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 1 -- 172.21.15.19:0/1132053885 shutdown_connections 2024-04-25T10:00:49.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 1 -- 172.21.15.19:0/1132053885 wait complete. 2024-04-25T10:00:49.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.101+0000 7f72d696e740 20 client.8778 trim_cache size 0 max 16384 2024-04-25T10:00:49.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.116+0000 7f72d696e740 1 Processor -- start 2024-04-25T10:00:49.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.116+0000 7f72d696e740 1 -- start start 2024-04-25T10:00:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.116+0000 7f72d696e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7b9f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.116+0000 7f72d696e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9de200 0x555b7b9f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d696e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9e0760 0x555b7b9e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d696e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc200400 con 0x555b7b9b5ca0 2024-04-25T10:00:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d696e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc1ff5d0 con 0x555b7b9de200 2024-04-25T10:00:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d696e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0d4290 con 0x555b7b9e0760 2024-04-25T10:00:49.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d1925640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7b9f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d1925640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7b9f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41080/0 (socket says 172.21.15.19:41080) 2024-04-25T10:00:49.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d1925640 1 -- 172.21.15.19:0/1539995308 learned_addr learned my addr 172.21.15.19:0/1539995308 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d1124640 1 --2- 172.21.15.19:0/1539995308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9de200 0x555b7b9f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d2126640 1 --2- 172.21.15.19:0/1539995308 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9e0760 0x555b7b9e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d1925640 1 -- 172.21.15.19:0/1539995308 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9e0760 msgr2=0x555b7b9e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d1925640 1 --2- 172.21.15.19:0/1539995308 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9e0760 0x555b7b9e3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d1925640 1 -- 172.21.15.19:0/1539995308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9de200 msgr2=0x555b7b9f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d1925640 1 --2- 172.21.15.19:0/1539995308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9de200 0x555b7b9f63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d1925640 1 -- 172.21.15.19:0/1539995308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b7b5c61b0 con 0x555b7b9b5ca0 2024-04-25T10:00:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d1124640 1 --2- 172.21.15.19:0/1539995308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9de200 0x555b7b9f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72d1925640 1 --2- 172.21.15.19:0/1539995308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7b9f5ea0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f72c80a90f0 tx=0x7f72c807c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2436a2a86fb350 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72aeffd640 1 -- 172.21.15.19:0/1539995308 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8096040 con 0x555b7b9b5ca0 2024-04-25T10:00:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72aeffd640 1 -- 172.21.15.19:0/1539995308 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c8081a00 con 0x555b7b9b5ca0 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.117+0000 7f72aeffd640 1 -- 172.21.15.19:0/1539995308 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c807d440 con 0x555b7b9b5ca0 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.118+0000 7f72d696e740 1 -- 172.21.15.19:0/1539995308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 msgr2=0x555b7b9f5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.118+0000 7f72d696e740 1 --2- 172.21.15.19:0/1539995308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7b9f5ea0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f72c80a90f0 tx=0x7f72c807c000 comp rx=0 tx=0).stop 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.118+0000 7f72d696e740 1 -- 172.21.15.19:0/1539995308 shutdown_connections 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.118+0000 7f72d696e740 1 --2- 172.21.15.19:0/1539995308 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9e0760 0x555b7b9e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.118+0000 7f72d696e740 1 --2- 172.21.15.19:0/1539995308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9de200 0x555b7b9f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.118+0000 7f72d696e740 1 --2- 172.21.15.19:0/1539995308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7b9f5ea0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.118+0000 7f72d696e740 1 -- 172.21.15.19:0/1539995308 >> 172.21.15.19:0/1539995308 conn(0x555b7b97d870 msgr2=0x555b7ba1df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.118+0000 7f72d696e740 1 -- 172.21.15.19:0/1539995308 shutdown_connections 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.118+0000 7f72d696e740 1 -- 172.21.15.19:0/1539995308 wait complete. 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.118+0000 7f72d696e740 1 -- start start 2024-04-25T10:00:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.118+0000 7f72d696e740 1 Processor -- start 2024-04-25T10:00:49.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.119+0000 7f72d696e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7ba4e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.119+0000 7f72d696e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9de200 0x555b7ba692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.119+0000 7f72d696e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9e0760 0x555b7ba665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.119+0000 7f72d696e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc1ff1f0 con 0x555b7b9b5ca0 2024-04-25T10:00:49.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.119+0000 7f72d696e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b7ba69dc0 con 0x555b7b9e0760 2024-04-25T10:00:49.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.119+0000 7f72d696e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b7ba4f020 con 0x555b7b9de200 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.119+0000 7f72d696e740 10 client.0 _ll_register_callbacks cb 0x555b7b9e1fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.119+0000 7f72d1925640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7ba4e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d1925640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7ba4e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41084/0 (socket says 172.21.15.19:41084) 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d1925640 1 -- 172.21.15.19:0/2169866979 learned_addr learned my addr 172.21.15.19:0/2169866979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d1124640 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9de200 0x555b7ba692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d2126640 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9e0760 0x555b7ba665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d1925640 1 -- 172.21.15.19:0/2169866979 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9de200 msgr2=0x555b7ba692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d1925640 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9de200 0x555b7ba692d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d1925640 1 -- 172.21.15.19:0/2169866979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9e0760 msgr2=0x555b7ba665d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d1925640 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9e0760 0x555b7ba665d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d1925640 1 -- 172.21.15.19:0/2169866979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b7b9f7d00 con 0x555b7b9b5ca0 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d1925640 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7ba4e9c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f72c80a90f0 tx=0x7f72c8032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f14c811fe656983 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72acff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72acff9640 1 -- 172.21.15.19:0/2169866979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8096050 con 0x555b7b9b5ca0 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72acff9640 1 -- 172.21.15.19:0/2169866979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72c80776a0 con 0x555b7b9b5ca0 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d696e740 1 -- 172.21.15.19:0/2169866979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72c8081a00 con 0x555b7b9b5ca0 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72acff9640 1 -- 172.21.15.19:0/2169866979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72c8075a10 con 0x555b7b9b5ca0 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d696e740 10 client.8784 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d696e740 1 -- 172.21.15.19:0/2169866979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x555b7ba69810 con 0x555b7b9b5ca0 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72acff9640 1 -- 172.21.15.19:0/2169866979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c8081a00 con 0x555b7b9b5ca0 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72acff9640 1 -- 172.21.15.19:0/2169866979 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f72c803cc60 con 0x555b7b9b5ca0 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.120+0000 7f72d696e740 10 client.8784 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 10 client.8784 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 20 client.8784 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x555b7b9f2370 0x555b7b9f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 4 client.8784 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 -- 172.21.15.19:0/2169866979 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x555b7b920420 con 0x555b7b9f2370 2024-04-25T10:00:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d1124640 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x555b7b9f2370 0x555b7b9f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72acff9640 10 client.8784 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:49.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d1124640 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x555b7b9f2370 0x555b7b9f47f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d368b6e9b23fe3 server_cookie=cf8dc1820902daf7 in_seq=0 out_seq=0 2024-04-25T10:00:49.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72acff9640 1 -- 172.21.15.19:0/2169866979 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x555b7b920420 con 0x555b7b9f2370 2024-04-25T10:00:49.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72acff9640 10 client.8784 handle_command_reply: tid=0 2024-04-25T10:00:49.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 client.8784 shutdown 2024-04-25T10:00:49.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 -- 172.21.15.19:0/2169866979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 msgr2=0x555b7ba4e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7ba4e9c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f72c80a90f0 tx=0x7f72c8032f90 comp rx=0 tx=0).stop 2024-04-25T10:00:49.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 -- 172.21.15.19:0/2169866979 shutdown_connections 2024-04-25T10:00:49.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x555b7b9f2370 0x555b7b9f47f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b7b9e0760 0x555b7ba665d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555b7b9de200 0x555b7ba692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72acff9640 0 client.8784 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:49.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 --2- 172.21.15.19:0/2169866979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555b7b9b5ca0 0x555b7ba4e9c0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 -- 172.21.15.19:0/2169866979 >> 172.21.15.19:0/2169866979 conn(0x555b7b97d870 msgr2=0x555b7b9e11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 -- 172.21.15.19:0/2169866979 shutdown_connections 2024-04-25T10:00:49.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.121+0000 7f72d696e740 1 -- 172.21.15.19:0/2169866979 wait complete. 2024-04-25T10:00:49.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d696e740 20 client.8784 trim_cache size 0 max 16384 2024-04-25T10:00:49.135 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T10:00:49.136 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 -- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f72b006b4d0 msgr2=0x7f72b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f72b006b4d0 0x7f72b006d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f72cc156ca0 tx=0x7f72bc007de0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 -- 172.21.15.19:0/1455064852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72cc0d32f0 msgr2=0x7f72cc0cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72cc0d32f0 0x7f72cc0cb120 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f72c80097a0 tx=0x7f72c80046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 -- 172.21.15.19:0/1455064852 shutdown_connections 2024-04-25T10:00:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f72b006b4d0 0x7f72b006d990 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72cc161710 0x7f72cc148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72cc15dc20 0x7f72cc146c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 --2- 172.21.15.19:0/1455064852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72cc0d32f0 0x7f72cc0cb120 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 -- 172.21.15.19:0/1455064852 >> 172.21.15.19:0/1455064852 conn(0x7f72cc0d0420 msgr2=0x7f72cc0b8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 -- 172.21.15.19:0/1455064852 shutdown_connections 2024-04-25T10:00:49.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.122+0000 7f72d3b89640 1 -- 172.21.15.19:0/1455064852 wait complete. 2024-04-25T10:00:49.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:49.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:00:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.237+0000 7f866ae4d640 1 Processor -- start 2024-04-25T10:00:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.237+0000 7f866ae4d640 1 -- start start 2024-04-25T10:00:49.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.238+0000 7f866ae4d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f866415bb50 0x7f86640c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.238+0000 7f866ae4d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f86640c70d0 0x7f86640cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.238+0000 7f866ae4d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86640c75e0 con 0x7f86640cfb40 2024-04-25T10:00:49.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.238+0000 7f866ae4d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86640c8680 con 0x7f86640c70d0 2024-04-25T10:00:49.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.238+0000 7f866ae4d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86640c78c0 con 0x7f866415bb50 2024-04-25T10:00:49.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.239+0000 7f8663fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f86640c70d0 0x7f86640cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.239+0000 7f86693ea640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f86640cfb40 0x7f8664070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:57188/0 (socket says 172.21.15.19:57188) 2024-04-25T10:00:49.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.239+0000 7f86693ea640 1 -- 172.21.15.19:0/1805816074 learned_addr learned my addr 172.21.15.19:0/1805816074 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:49.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.239+0000 7f8668be9640 1 --2- 172.21.15.19:0/1805816074 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f866415bb50 0x7f86640c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.239+0000 7f8663fff640 1 -- 172.21.15.19:0/1805816074 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f866415bb50 msgr2=0x7f86640c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.239+0000 7f8663fff640 1 --2- 172.21.15.19:0/1805816074 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f866415bb50 0x7f86640c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.239+0000 7f8663fff640 1 -- 172.21.15.19:0/1805816074 >> v1:172.21.15.112:6789/0 conn(0x7f86640cfb40 legacy=0x7f8664070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.239+0000 7f8663fff640 1 -- 172.21.15.19:0/1805816074 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640c7ba0 con 0x7f86640c70d0 2024-04-25T10:00:49.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f8663fff640 1 --2- 172.21.15.19:0/1805816074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86640c70d0 0x7f86640cf600 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f86580097a0 tx=0x7f865802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0ba3ea6c3e801af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f86637fe640 1 -- 172.21.15.19:0/1805816074 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658037070 con 0x7f86640c70d0 2024-04-25T10:00:49.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f86637fe640 1 -- 172.21.15.19:0/1805816074 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8658032440 con 0x7f86640c70d0 2024-04-25T10:00:49.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f86637fe640 1 -- 172.21.15.19:0/1805816074 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658032810 con 0x7f86640c70d0 2024-04-25T10:00:49.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f866ae4d640 1 -- 172.21.15.19:0/1805816074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86640c70d0 msgr2=0x7f86640cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f866ae4d640 1 --2- 172.21.15.19:0/1805816074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86640c70d0 0x7f86640cf600 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f86580097a0 tx=0x7f865802a210 comp rx=0 tx=0).stop 2024-04-25T10:00:49.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f866ae4d640 1 -- 172.21.15.19:0/1805816074 shutdown_connections 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f866ae4d640 1 --2- 172.21.15.19:0/1805816074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86640c70d0 0x7f86640cf600 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f866ae4d640 1 --2- 172.21.15.19:0/1805816074 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f866415bb50 0x7f86640c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f866ae4d640 1 -- 172.21.15.19:0/1805816074 >> 172.21.15.19:0/1805816074 conn(0x7f866406f500 msgr2=0x7f866406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f866ae4d640 1 -- 172.21.15.19:0/1805816074 shutdown_connections 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.240+0000 7f866ae4d640 1 -- 172.21.15.19:0/1805816074 wait complete. 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.241+0000 7f866ae4d640 1 Processor -- start 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.242+0000 7f866ae4d640 1 -- start start 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.242+0000 7f866ae4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86640c70d0 0x7f86641e7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.242+0000 7f866ae4d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86640cfb40 0x7f86641e5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.242+0000 7f866ae4d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f866415bb50 0x7f86641e5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.242+0000 7f866ae4d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866415f6d0 con 0x7f866415bb50 2024-04-25T10:00:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.242+0000 7f866ae4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86640c8160 con 0x7f86640c70d0 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.242+0000 7f866ae4d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866415f850 con 0x7f86640cfb40 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.242+0000 7f86693ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f866415bb50 0x7f86641e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f86693ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f866415bb50 0x7f86641e5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41096/0 (socket says 172.21.15.19:41096) 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f86693ea640 1 -- 172.21.15.19:0/4002236806 learned_addr learned my addr 172.21.15.19:0/4002236806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f8668be9640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86640c70d0 0x7f86641e7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f8663fff640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86640cfb40 0x7f86641e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f86693ea640 1 -- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86640cfb40 msgr2=0x7f86641e5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f86693ea640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86640cfb40 0x7f86641e5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f86693ea640 1 -- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86640c70d0 msgr2=0x7f86641e7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f86693ea640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86640c70d0 0x7f86641e7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f86693ea640 1 -- 172.21.15.19:0/4002236806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8664152570 con 0x7f866415bb50 2024-04-25T10:00:49.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f86693ea640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f866415bb50 0x7f86641e5850 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f865400c420 tx=0x7f865400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f86617fa640 1 -- 172.21.15.19:0/4002236806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654017070 con 0x7f866415bb50 2024-04-25T10:00:49.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f86617fa640 1 -- 172.21.15.19:0/4002236806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8654005ce0 con 0x7f866415bb50 2024-04-25T10:00:49.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f866ae4d640 1 -- 172.21.15.19:0/4002236806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86640ca340 con 0x7f866415bb50 2024-04-25T10:00:49.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f86617fa640 1 -- 172.21.15.19:0/4002236806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540072d0 con 0x7f866415bb50 2024-04-25T10:00:49.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.243+0000 7f866ae4d640 1 -- 172.21.15.19:0/4002236806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86640ca900 con 0x7f866415bb50 2024-04-25T10:00:49.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.245+0000 7f86617fa640 1 -- 172.21.15.19:0/4002236806 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f865401a070 con 0x7f866415bb50 2024-04-25T10:00:49.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.245+0000 7f86617fa640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f865006b4d0 0x7f865006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.245+0000 7f86617fa640 1 -- 172.21.15.19:0/4002236806 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f86540895f0 con 0x7f866415bb50 2024-04-25T10:00:49.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.246+0000 7f8668be9640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f865006b4d0 0x7f865006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.246+0000 7f8668be9640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f865006b4d0 0x7f865006d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f86641f7180 tx=0x7f865c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.246+0000 7f866dc2d740 1 Processor -- start 2024-04-25T10:00:49.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.246+0000 7f866dc2d740 1 -- start start 2024-04-25T10:00:49.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbb10 0x55c1da4be320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbef0 0x55c1da4be860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da48e770 0x55c1da491f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f866dc2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1da3f7da0 con 0x55c1da4bbb10 2024-04-25T10:00:49.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f866dc2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1da3a0b80 con 0x55c1da4bbef0 2024-04-25T10:00:49.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f866dc2d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1da3f3e40 con 0x55c1da48e770 2024-04-25T10:00:49.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f8668be9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbb10 0x55c1da4be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f8668be9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbb10 0x55c1da4be320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41102/0 (socket says 172.21.15.19:41102) 2024-04-25T10:00:49.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f8668be9640 1 -- 172.21.15.19:0/4199268445 learned_addr learned my addr 172.21.15.19:0/4199268445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f8663fff640 1 --2- 172.21.15.19:0/4199268445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbef0 0x55c1da4be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f86693ea640 1 --2- 172.21.15.19:0/4199268445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da48e770 0x55c1da491f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f8668be9640 1 -- 172.21.15.19:0/4199268445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da48e770 msgr2=0x55c1da491f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f8668be9640 1 --2- 172.21.15.19:0/4199268445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da48e770 0x55c1da491f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f8668be9640 1 -- 172.21.15.19:0/4199268445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbef0 msgr2=0x55c1da4be860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f8668be9640 1 --2- 172.21.15.19:0/4199268445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbef0 0x55c1da4be860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f8668be9640 1 -- 172.21.15.19:0/4199268445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1da3aaf10 con 0x55c1da4bbb10 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.247+0000 7f8668be9640 1 --2- 172.21.15.19:0/4199268445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbb10 0x55c1da4be320 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f865c0167f0 tx=0x7f865c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d0f985ea1b7ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f864a7fc640 1 -- 172.21.15.19:0/4199268445 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c01d040 con 0x55c1da4bbb10 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f864a7fc640 1 -- 172.21.15.19:0/4199268445 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c026070 con 0x55c1da4bbb10 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f864a7fc640 1 -- 172.21.15.19:0/4199268445 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c017980 con 0x55c1da4bbb10 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f866dc2d740 1 -- 172.21.15.19:0/4199268445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbb10 msgr2=0x55c1da4be320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f866dc2d740 1 --2- 172.21.15.19:0/4199268445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbb10 0x55c1da4be320 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f865c0167f0 tx=0x7f865c014000 comp rx=0 tx=0).stop 2024-04-25T10:00:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f866dc2d740 1 -- 172.21.15.19:0/4199268445 shutdown_connections 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f866dc2d740 1 --2- 172.21.15.19:0/4199268445 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da48e770 0x55c1da491f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f866dc2d740 1 --2- 172.21.15.19:0/4199268445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbef0 0x55c1da4be860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f866dc2d740 1 --2- 172.21.15.19:0/4199268445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbb10 0x55c1da4be320 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f866dc2d740 1 -- 172.21.15.19:0/4199268445 >> 172.21.15.19:0/4199268445 conn(0x55c1da4909e0 msgr2=0x55c1da4d9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f866dc2d740 1 -- 172.21.15.19:0/4199268445 shutdown_connections 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.248+0000 7f866dc2d740 1 -- 172.21.15.19:0/4199268445 wait complete. 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.249+0000 7f866dc2d740 1 -- start start 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.249+0000 7f866dc2d740 1 Processor -- start 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da4c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4bf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da4bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f866dc2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1da493b90 con 0x55c1da4bbef0 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f866dc2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1da3964b0 con 0x55c1da4bbb10 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f866dc2d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1da493e70 con 0x55c1da4bb150 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f866dc2d740 10 client.0 _ll_register_callbacks cb 0x55c1da3d6180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f8663fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4bf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f8668be9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da4c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f86693ea640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da4bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f8663fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4bf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41106/0 (socket says 172.21.15.19:41106) 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f8663fff640 1 -- 172.21.15.19:0/2364047069 learned_addr learned my addr 172.21.15.19:0/2364047069 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f8663fff640 1 -- 172.21.15.19:0/2364047069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 msgr2=0x55c1da4bfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f8663fff640 1 --2- 172.21.15.19:0/2364047069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da4bfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f8663fff640 1 -- 172.21.15.19:0/2364047069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 msgr2=0x55c1da4c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f8663fff640 1 --2- 172.21.15.19:0/2364047069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da4c24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.250+0000 7f8663fff640 1 -- 172.21.15.19:0/2364047069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1da1d67d0 con 0x55c1da4bbef0 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f8663fff640 1 --2- 172.21.15.19:0/2364047069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4bf8e0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f86580097a0 tx=0x7f8658004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3245b32532d26e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f86437fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f86437fe640 1 -- 172.21.15.19:0/2364047069 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658037040 con 0x55c1da4bbef0 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f86437fe640 1 -- 172.21.15.19:0/2364047069 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8658047c30 con 0x55c1da4bbef0 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f866dc2d740 1 -- 172.21.15.19:0/2364047069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f865c026070 con 0x55c1da4bbef0 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f86437fe640 1 -- 172.21.15.19:0/2364047069 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8658032e20 con 0x55c1da4bbef0 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f866dc2d740 10 client.8793 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f866dc2d740 1 -- 172.21.15.19:0/2364047069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55c1da10c3f0 con 0x55c1da4bbef0 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f86437fe640 1 -- 172.21.15.19:0/2364047069 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c026070 con 0x55c1da4bbef0 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f86437fe640 1 -- 172.21.15.19:0/2364047069 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f865803c410 con 0x55c1da4bbef0 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f866dc2d740 10 client.8793 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f866dc2d740 10 client.8793 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f866dc2d740 20 client.8793 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f866dc2d740 1 --2- 172.21.15.19:0/2364047069 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55c1da4e91f0 0x55c1da4e9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f866dc2d740 4 client.8793 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f866dc2d740 1 -- 172.21.15.19:0/2364047069 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1da4052e0 con 0x55c1da4e91f0 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.251+0000 7f8668be9640 1 --2- 172.21.15.19:0/2364047069 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55c1da4e91f0 0x55c1da4e9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.252+0000 7f8668be9640 1 --2- 172.21.15.19:0/2364047069 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55c1da4e91f0 0x55c1da4e9630 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b851b24318ce3 server_cookie=4184e58471de2df5 in_seq=0 out_seq=0 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.252+0000 7f86437fe640 10 client.8793 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.252+0000 7f86437fe640 1 -- 172.21.15.19:0/2364047069 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c1da4052e0 con 0x55c1da4e91f0 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.252+0000 7f86437fe640 10 client.8793 handle_command_reply: tid=0 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.252+0000 7f866dc2d740 1 client.8793 shutdown 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.252+0000 7f866dc2d740 1 -- 172.21.15.19:0/2364047069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 msgr2=0x55c1da4bf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.252+0000 7f866dc2d740 1 --2- 172.21.15.19:0/2364047069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4bf8e0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f86580097a0 tx=0x7f8658004730 comp rx=0 tx=0).stop 2024-04-25T10:00:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.253+0000 7f866dc2d740 1 -- 172.21.15.19:0/2364047069 shutdown_connections 2024-04-25T10:00:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.253+0000 7f866dc2d740 1 --2- 172.21.15.19:0/2364047069 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55c1da4e91f0 0x55c1da4e9630 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.253+0000 7f866dc2d740 1 --2- 172.21.15.19:0/2364047069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da4bfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.253+0000 7f866dc2d740 1 --2- 172.21.15.19:0/2364047069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4bf8e0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.253+0000 7f866dc2d740 1 --2- 172.21.15.19:0/2364047069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da4c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.253+0000 7f86437fe640 0 client.8793 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.253+0000 7f866dc2d740 1 -- 172.21.15.19:0/2364047069 >> 172.21.15.19:0/2364047069 conn(0x55c1da48e770 msgr2=0x55c1da4be310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.253+0000 7f866dc2d740 1 -- 172.21.15.19:0/2364047069 shutdown_connections 2024-04-25T10:00:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.253+0000 7f866dc2d740 1 -- 172.21.15.19:0/2364047069 wait complete. 2024-04-25T10:00:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.253+0000 7f866dc2d740 20 client.8793 trim_cache size 0 max 16384 2024-04-25T10:00:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.268+0000 7f866dc2d740 1 Processor -- start 2024-04-25T10:00:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.268+0000 7f866dc2d740 1 -- start start 2024-04-25T10:00:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.268+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da489ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.268+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da4f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.268+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da492e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.268+0000 7f866dc2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8664200480 con 0x55c1da4bbef0 2024-04-25T10:00:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.268+0000 7f866dc2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86641ff620 con 0x55c1da4bbb10 2024-04-25T10:00:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.268+0000 7f866dc2d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86640b69b0 con 0x55c1da4bb150 2024-04-25T10:00:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.268+0000 7f86693ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da492e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f86693ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da492e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41120/0 (socket says 172.21.15.19:41120) 2024-04-25T10:00:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f86693ea640 1 -- 172.21.15.19:0/1902471524 learned_addr learned my addr 172.21.15.19:0/1902471524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f8668be9640 1 --2- 172.21.15.19:0/1902471524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da489ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f8663fff640 1 --2- 172.21.15.19:0/1902471524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da4f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f86693ea640 1 -- 172.21.15.19:0/1902471524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 msgr2=0x55c1da489ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f86693ea640 1 --2- 172.21.15.19:0/1902471524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da489ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f86693ea640 1 -- 172.21.15.19:0/1902471524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 msgr2=0x55c1da4f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f86693ea640 1 --2- 172.21.15.19:0/1902471524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da4f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f86693ea640 1 -- 172.21.15.19:0/1902471524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1da0cc0f0 con 0x55c1da4bbef0 2024-04-25T10:00:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f86693ea640 1 --2- 172.21.15.19:0/1902471524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da492e00 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f86540540a0 tx=0x7f8654087860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd34f5100ae3051 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f8649ffb640 1 -- 172.21.15.19:0/1902471524 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654054300 con 0x55c1da4bbef0 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f8649ffb640 1 -- 172.21.15.19:0/1902471524 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865406f070 con 0x55c1da4bbef0 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.269+0000 7f8649ffb640 1 -- 172.21.15.19:0/1902471524 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540544a0 con 0x55c1da4bbef0 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.270+0000 7f866dc2d740 1 -- 172.21.15.19:0/1902471524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 msgr2=0x55c1da492e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.270+0000 7f866dc2d740 1 --2- 172.21.15.19:0/1902471524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da492e00 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f86540540a0 tx=0x7f8654087860 comp rx=0 tx=0).stop 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.270+0000 7f866dc2d740 1 -- 172.21.15.19:0/1902471524 shutdown_connections 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.270+0000 7f866dc2d740 1 --2- 172.21.15.19:0/1902471524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da492e00 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.270+0000 7f866dc2d740 1 --2- 172.21.15.19:0/1902471524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da4f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.270+0000 7f866dc2d740 1 --2- 172.21.15.19:0/1902471524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da489ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.270+0000 7f866dc2d740 1 -- 172.21.15.19:0/1902471524 >> 172.21.15.19:0/1902471524 conn(0x55c1da48e770 msgr2=0x55c1da4e95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.270+0000 7f866dc2d740 1 -- 172.21.15.19:0/1902471524 shutdown_connections 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.270+0000 7f866dc2d740 1 -- 172.21.15.19:0/1902471524 wait complete. 2024-04-25T10:00:49.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.270+0000 7f866dc2d740 1 -- start start 2024-04-25T10:00:49.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f866dc2d740 1 Processor -- start 2024-04-25T10:00:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da4f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da5435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f866dc2d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4d95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f866dc2d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1da494880 con 0x55c1da4bbef0 2024-04-25T10:00:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f866dc2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86641ff020 con 0x55c1da4bbb10 2024-04-25T10:00:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f866dc2d740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1da3fa2b0 con 0x55c1da4bb150 2024-04-25T10:00:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f866dc2d740 10 client.0 _ll_register_callbacks cb 0x7f865c009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f86693ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4d95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f8668be9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da4f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f86693ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4d95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41134/0 (socket says 172.21.15.19:41134) 2024-04-25T10:00:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.271+0000 7f86693ea640 1 -- 172.21.15.19:0/698499040 learned_addr learned my addr 172.21.15.19:0/698499040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f8663fff640 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da5435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86693ea640 1 -- 172.21.15.19:0/698499040 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 msgr2=0x55c1da4f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86693ea640 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da4f2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86693ea640 1 -- 172.21.15.19:0/698499040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 msgr2=0x55c1da5435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86693ea640 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da5435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86693ea640 1 -- 172.21.15.19:0/698499040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1da4f3030 con 0x55c1da4bbef0 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f8668be9640 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da4f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86693ea640 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4d95b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f86540604b0 tx=0x7f8654007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9032c01aa6fb769d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86437fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86437fe640 1 -- 172.21.15.19:0/698499040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654074070 con 0x55c1da4bbef0 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86437fe640 1 -- 172.21.15.19:0/698499040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8654007470 con 0x55c1da4bbef0 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f866dc2d740 1 -- 172.21.15.19:0/698499040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f865406f070 con 0x55c1da4bbef0 2024-04-25T10:00:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86437fe640 1 -- 172.21.15.19:0/698499040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8654007670 con 0x55c1da4bbef0 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f866dc2d740 10 client.8799 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f866dc2d740 1 -- 172.21.15.19:0/698499040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55c1da48ee50 con 0x55c1da4bbef0 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86437fe640 1 -- 172.21.15.19:0/698499040 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865406f070 con 0x55c1da4bbef0 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f86437fe640 1 -- 172.21.15.19:0/698499040 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8654082020 con 0x55c1da4bbef0 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f866dc2d740 10 client.8799 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f866dc2d740 10 client.8799 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f866dc2d740 20 client.8799 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f866dc2d740 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55c1da4d9af0 0x55c1da4cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f866dc2d740 4 client.8799 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.272+0000 7f866dc2d740 1 -- 172.21.15.19:0/698499040 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c1da3ff630 con 0x55c1da4d9af0 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f8668be9640 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55c1da4d9af0 0x55c1da4cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f86437fe640 10 client.8799 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f8668be9640 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55c1da4d9af0 0x55c1da4cecb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87f926cbcd13703 server_cookie=5828473218148b7a in_seq=0 out_seq=0 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f86437fe640 1 -- 172.21.15.19:0/698499040 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55c1da3ff630 con 0x55c1da4d9af0 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f86437fe640 10 client.8799 handle_command_reply: tid=0 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f866dc2d740 1 client.8799 shutdown 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f866dc2d740 1 -- 172.21.15.19:0/698499040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 msgr2=0x55c1da4d95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f866dc2d740 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4d95b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f86540604b0 tx=0x7f8654007f90 comp rx=0 tx=0).stop 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f866dc2d740 1 -- 172.21.15.19:0/698499040 shutdown_connections 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f866dc2d740 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55c1da4d9af0 0x55c1da4cecb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f866dc2d740 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1da4bbef0 0x55c1da4d95b0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f866dc2d740 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1da4bbb10 0x55c1da5435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f866dc2d740 1 --2- 172.21.15.19:0/698499040 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c1da4bb150 0x55c1da4f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f866dc2d740 1 -- 172.21.15.19:0/698499040 >> 172.21.15.19:0/698499040 conn(0x55c1da48e770 msgr2=0x55c1da4c03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f86437fe640 0 client.8799 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f866dc2d740 1 -- 172.21.15.19:0/698499040 shutdown_connections 2024-04-25T10:00:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.273+0000 7f866dc2d740 1 -- 172.21.15.19:0/698499040 wait complete. 2024-04-25T10:00:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866dc2d740 20 client.8799 trim_cache size 0 max 16384 2024-04-25T10:00:49.286 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:49.286 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:00:49.286 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:00:49.286 INFO:teuthology.orchestra.run.smithi019.stdout: "f3e5034a-13b7-47d5-a470-51dbfdb9fc5c": { 2024-04-25T10:00:49.286 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:00:49.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "f3e5034a-13b7-47d5-a470-51dbfdb9fc5c", 2024-04-25T10:00:49.287 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive" 2024-04-25T10:00:49.287 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:49.287 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:00:49.287 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 -- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f865006b4d0 msgr2=0x7f865006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f865006b4d0 0x7f865006d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f86641f7180 tx=0x7f865c00b040 comp rx=0 tx=0).stop 2024-04-25T10:00:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 -- 172.21.15.19:0/4002236806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f866415bb50 msgr2=0x7f86641e5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f866415bb50 0x7f86641e5850 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f865400c420 tx=0x7f865400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 -- 172.21.15.19:0/4002236806 shutdown_connections 2024-04-25T10:00:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f865006b4d0 0x7f865006d990 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f866415bb50 0x7f86641e5850 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86640cfb40 0x7f86641e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 --2- 172.21.15.19:0/4002236806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86640c70d0 0x7f86641e7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 -- 172.21.15.19:0/4002236806 >> 172.21.15.19:0/4002236806 conn(0x7f866406f500 msgr2=0x7f86640b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 -- 172.21.15.19:0/4002236806 shutdown_connections 2024-04-25T10:00:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.274+0000 7f866ae4d640 1 -- 172.21.15.19:0/4002236806 wait complete. 2024-04-25T10:00:49.290 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:49.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.398+0000 7f24040b5640 1 Processor -- start 2024-04-25T10:00:49.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.399+0000 7f24040b5640 1 -- start start 2024-04-25T10:00:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.399+0000 7f24040b5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f23fc15ba00 0x7f23fc09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f24040b5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23fc15f5f0 0x7f23fc15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f24040b5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23fc0cd710 con 0x7f23fc1602e0 2024-04-25T10:00:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f24040b5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23fc0cd4b0 con 0x7f23fc15ba00 2024-04-25T10:00:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f24040b5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc099b70 con 0x7f23fc15f5f0 2024-04-25T10:00:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f2401e51640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f23fc15ba00 0x7f23fc09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f2401650640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f23fc1602e0 0x7f23fc0ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51102/0 (socket says 172.21.15.19:51102) 2024-04-25T10:00:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f2401650640 1 -- 172.21.15.19:0/4119025118 learned_addr learned my addr 172.21.15.19:0/4119025118 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f2402652640 1 --2- 172.21.15.19:0/4119025118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23fc15f5f0 0x7f23fc15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f2401e51640 1 -- 172.21.15.19:0/4119025118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23fc15f5f0 msgr2=0x7f23fc15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f2401e51640 1 --2- 172.21.15.19:0/4119025118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23fc15f5f0 0x7f23fc15fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f2401e51640 1 -- 172.21.15.19:0/4119025118 >> v1:172.21.15.19:6789/0 conn(0x7f23fc1602e0 legacy=0x7f23fc0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f2401e51640 1 -- 172.21.15.19:0/4119025118 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc099e50 con 0x7f23fc15ba00 2024-04-25T10:00:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.400+0000 7f2400e4f640 1 -- 172.21.15.19:0/4119025118 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054497253 0 0) 0x7f23ec001110 con 0x7f23fc1602e0 2024-04-25T10:00:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.401+0000 7f2401e51640 1 --2- 172.21.15.19:0/4119025118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23fc15ba00 0x7f23fc09a930 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f23f80097a0 tx=0x7f23f802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec9e198b4cff63e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.401+0000 7f2400e4f640 1 -- 172.21.15.19:0/4119025118 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8037070 con 0x7f23fc15ba00 2024-04-25T10:00:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.401+0000 7f2400e4f640 1 -- 172.21.15.19:0/4119025118 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f802adb0 con 0x7f23fc15ba00 2024-04-25T10:00:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.401+0000 7f2400e4f640 1 -- 172.21.15.19:0/4119025118 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8032660 con 0x7f23fc15ba00 2024-04-25T10:00:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.402+0000 7f24040b5640 1 -- 172.21.15.19:0/4119025118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23fc15ba00 msgr2=0x7f23fc09a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.402+0000 7f24040b5640 1 --2- 172.21.15.19:0/4119025118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23fc15ba00 0x7f23fc09a930 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f23f80097a0 tx=0x7f23f802a060 comp rx=0 tx=0).stop 2024-04-25T10:00:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.402+0000 7f24040b5640 1 -- 172.21.15.19:0/4119025118 shutdown_connections 2024-04-25T10:00:49.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.402+0000 7f24040b5640 1 --2- 172.21.15.19:0/4119025118 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23fc15f5f0 0x7f23fc15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.402+0000 7f24040b5640 1 --2- 172.21.15.19:0/4119025118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23fc15ba00 0x7f23fc09a930 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.402+0000 7f24040b5640 1 -- 172.21.15.19:0/4119025118 >> 172.21.15.19:0/4119025118 conn(0x7f23fc0b3270 msgr2=0x7f23fc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.402+0000 7f24040b5640 1 -- 172.21.15.19:0/4119025118 shutdown_connections 2024-04-25T10:00:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.402+0000 7f24040b5640 1 -- 172.21.15.19:0/4119025118 wait complete. 2024-04-25T10:00:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.403+0000 7f24040b5640 1 Processor -- start 2024-04-25T10:00:49.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.403+0000 7f24040b5640 1 -- start start 2024-04-25T10:00:49.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f24040b5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23fc15ba00 0x7f23fc0b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f24040b5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23fc15f5f0 0x7f23fc14c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f24040b5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23fc1602e0 0x7f23fc14cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f24040b5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc15bde0 con 0x7f23fc15f5f0 2024-04-25T10:00:49.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f24040b5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc09ae70 con 0x7f23fc1602e0 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f24040b5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc1552d0 con 0x7f23fc15ba00 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f2401650640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23fc15f5f0 0x7f23fc14c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f2401650640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23fc15f5f0 0x7f23fc14c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41154/0 (socket says 172.21.15.19:41154) 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f2401650640 1 -- 172.21.15.19:0/1614029211 learned_addr learned my addr 172.21.15.19:0/1614029211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f2402652640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23fc1602e0 0x7f23fc14cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f2401e51640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23fc15ba00 0x7f23fc0b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f2401650640 1 -- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23fc15ba00 msgr2=0x7f23fc0b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f2401650640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23fc15ba00 0x7f23fc0b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f2401650640 1 -- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23fc1602e0 msgr2=0x7f23fc14cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f2401650640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23fc1602e0 0x7f23fc14cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.404+0000 7f2401650640 1 -- 172.21.15.19:0/1614029211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc06dbf0 con 0x7f23fc15f5f0 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.405+0000 7f2401650640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23fc15f5f0 0x7f23fc14c630 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00c780 tx=0x7f23ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.405+0000 7f23f27fc640 1 -- 172.21.15.19:0/1614029211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec018070 con 0x7f23fc15f5f0 2024-04-25T10:00:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.405+0000 7f23f27fc640 1 -- 172.21.15.19:0/1614029211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ec0072d0 con 0x7f23fc15f5f0 2024-04-25T10:00:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.405+0000 7f24040b5640 1 -- 172.21.15.19:0/1614029211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23fc15db50 con 0x7f23fc15f5f0 2024-04-25T10:00:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.405+0000 7f23f27fc640 1 -- 172.21.15.19:0/1614029211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec0047e0 con 0x7f23fc15f5f0 2024-04-25T10:00:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.405+0000 7f24040b5640 1 -- 172.21.15.19:0/1614029211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc15e110 con 0x7f23fc15f5f0 2024-04-25T10:00:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.406+0000 7f24040b5640 1 -- 172.21.15.19:0/1614029211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23fc073990 con 0x7f23fc15f5f0 2024-04-25T10:00:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.406+0000 7f23f27fc640 1 -- 172.21.15.19:0/1614029211 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23ec01b070 con 0x7f23fc15f5f0 2024-04-25T10:00:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.407+0000 7f23f27fc640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f23e006b4d0 0x7f23e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.407+0000 7f23f27fc640 1 -- 172.21.15.19:0/1614029211 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f23ec089640 con 0x7f23fc15f5f0 2024-04-25T10:00:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.407+0000 7f2401e51640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f23e006b4d0 0x7f23e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.407+0000 7f2401e51640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f23e006b4d0 0x7f23e006d990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f23f8034000 tx=0x7f23f802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.411+0000 7f23f27fc640 1 -- 172.21.15.19:0/1614029211 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23fc0cd710 con 0x7f23fc15f5f0 2024-04-25T10:00:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.629+0000 7f24040b5640 1 -- 172.21.15.19:0/1614029211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23fc0b40a0 con 0x7f23fc15f5f0 2024-04-25T10:00:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.630+0000 7f23f27fc640 1 -- 172.21.15.19:0/1614029211 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f23ec057ce0 con 0x7f23fc15f5f0 2024-04-25T10:00:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:49.634 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:00:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:00:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.631+0000 7f24040b5640 1 -- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f23e006b4d0 msgr2=0x7f23e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.631+0000 7f24040b5640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f23e006b4d0 0x7f23e006d990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f23f8034000 tx=0x7f23f802e000 comp rx=0 tx=0).stop 2024-04-25T10:00:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.631+0000 7f24040b5640 1 -- 172.21.15.19:0/1614029211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23fc15f5f0 msgr2=0x7f23fc14c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.631+0000 7f24040b5640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23fc15f5f0 0x7f23fc14c630 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00c780 tx=0x7f23ec00cc40 comp rx=0 tx=0).stop 2024-04-25T10:00:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.632+0000 7f24040b5640 1 -- 172.21.15.19:0/1614029211 shutdown_connections 2024-04-25T10:00:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.632+0000 7f24040b5640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f23e006b4d0 0x7f23e006d990 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.632+0000 7f24040b5640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23fc1602e0 0x7f23fc14cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.632+0000 7f24040b5640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23fc15f5f0 0x7f23fc14c630 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.632+0000 7f24040b5640 1 --2- 172.21.15.19:0/1614029211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23fc15ba00 0x7f23fc0b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.632+0000 7f24040b5640 1 -- 172.21.15.19:0/1614029211 >> 172.21.15.19:0/1614029211 conn(0x7f23fc0b3270 msgr2=0x7f23fc0cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.632+0000 7f24040b5640 1 -- 172.21.15.19:0/1614029211 shutdown_connections 2024-04-25T10:00:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.632+0000 7f24040b5640 1 -- 172.21.15.19:0/1614029211 wait complete. 2024-04-25T10:00:49.647 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:00:49.647 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:49.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.742+0000 7fc646da9640 1 Processor -- start 2024-04-25T10:00:49.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.742+0000 7fc646da9640 1 -- start start 2024-04-25T10:00:49.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.743+0000 7fc646da9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc64015ec20 0x7fc6400b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.743+0000 7fc646da9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64006e430 0x7fc64006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.743+0000 7fc646da9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6400b3e10 con 0x7fc64006abe0 2024-04-25T10:00:49.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.743+0000 7fc646da9640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6400b3590 con 0x7fc64015ec20 2024-04-25T10:00:49.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.743+0000 7fc646da9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6400b3850 con 0x7fc64006e430 2024-04-25T10:00:49.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.743+0000 7fc63ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc64015ec20 0x7fc6400b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.743+0000 7fc63ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc64015ec20 0x7fc6400b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41156/0 (socket says 172.21.15.19:41156) 2024-04-25T10:00:49.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.743+0000 7fc63ffff640 1 -- 172.21.15.19:0/1062857023 learned_addr learned my addr 172.21.15.19:0/1062857023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.743+0000 7fc645346640 1 --2- 172.21.15.19:0/1062857023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64006e430 0x7fc64006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.744+0000 7fc63ffff640 1 -- 172.21.15.19:0/1062857023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64006e430 msgr2=0x7fc64006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:49.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.744+0000 7fc63ffff640 1 --2- 172.21.15.19:0/1062857023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64006e430 0x7fc64006e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.744+0000 7fc63ffff640 1 -- 172.21.15.19:0/1062857023 >> v1:172.21.15.112:6789/0 conn(0x7fc64006abe0 legacy=0x7fc64006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.744+0000 7fc63ffff640 1 -- 172.21.15.19:0/1062857023 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6400b3b30 con 0x7fc64015ec20 2024-04-25T10:00:49.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.744+0000 7fc63ffff640 1 --2- 172.21.15.19:0/1062857023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64015ec20 0x7fc6400b5270 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fc630009870 tx=0x7fc63002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422c4e0d4cc0813b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.745+0000 7fc63f7fe640 1 -- 172.21.15.19:0/1062857023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc630037070 con 0x7fc64015ec20 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.745+0000 7fc63f7fe640 1 -- 172.21.15.19:0/1062857023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc630032440 con 0x7fc64015ec20 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.745+0000 7fc63f7fe640 1 -- 172.21.15.19:0/1062857023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc630032810 con 0x7fc64015ec20 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.745+0000 7fc646da9640 1 -- 172.21.15.19:0/1062857023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64015ec20 msgr2=0x7fc6400b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.745+0000 7fc646da9640 1 --2- 172.21.15.19:0/1062857023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64015ec20 0x7fc6400b5270 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fc630009870 tx=0x7fc63002a270 comp rx=0 tx=0).stop 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.745+0000 7fc646da9640 1 -- 172.21.15.19:0/1062857023 shutdown_connections 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.745+0000 7fc646da9640 1 --2- 172.21.15.19:0/1062857023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64006e430 0x7fc64006e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.745+0000 7fc646da9640 1 --2- 172.21.15.19:0/1062857023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64015ec20 0x7fc6400b5270 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.745+0000 7fc646da9640 1 -- 172.21.15.19:0/1062857023 >> 172.21.15.19:0/1062857023 conn(0x7fc6400b5af0 msgr2=0x7fc6400b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.745+0000 7fc646da9640 1 -- 172.21.15.19:0/1062857023 shutdown_connections 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.745+0000 7fc646da9640 1 -- 172.21.15.19:0/1062857023 wait complete. 2024-04-25T10:00:49.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.746+0000 7fc646da9640 1 Processor -- start 2024-04-25T10:00:49.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.746+0000 7fc646da9640 1 -- start start 2024-04-25T10:00:49.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.747+0000 7fc646da9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64006abe0 0x7fc640146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.747+0000 7fc646da9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc64006e430 0x7fc6400b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.747+0000 7fc646da9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64015ec20 0x7fc64009d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.747+0000 7fc646da9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64015f0b0 con 0x7fc64006abe0 2024-04-25T10:00:49.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.747+0000 7fc646da9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64015e700 con 0x7fc64006e430 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.747+0000 7fc646da9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64015f230 con 0x7fc64015ec20 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.747+0000 7fc644b45640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64006abe0 0x7fc640146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.747+0000 7fc644b45640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64006abe0 0x7fc640146100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41168/0 (socket says 172.21.15.19:41168) 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.747+0000 7fc644b45640 1 -- 172.21.15.19:0/3340547221 learned_addr learned my addr 172.21.15.19:0/3340547221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc63ffff640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc64006e430 0x7fc6400b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc645346640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64015ec20 0x7fc64009d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc644b45640 1 -- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64015ec20 msgr2=0x7fc64009d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc644b45640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64015ec20 0x7fc64009d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc644b45640 1 -- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc64006e430 msgr2=0x7fc6400b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc644b45640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc64006e430 0x7fc6400b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc644b45640 1 -- 172.21.15.19:0/3340547221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc640146640 con 0x7fc64006abe0 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc645346640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64015ec20 0x7fc64009d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc644b45640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64006abe0 0x7fc640146100 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fc62800c3c0 tx=0x7fc62800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc63d7fa640 1 -- 172.21.15.19:0/3340547221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6280052d0 con 0x7fc64006abe0 2024-04-25T10:00:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc63d7fa640 1 -- 172.21.15.19:0/3340547221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc628005490 con 0x7fc64006abe0 2024-04-25T10:00:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc63d7fa640 1 -- 172.21.15.19:0/3340547221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc628007260 con 0x7fc64006abe0 2024-04-25T10:00:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc646da9640 1 -- 172.21.15.19:0/3340547221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6400b09f0 con 0x7fc64006abe0 2024-04-25T10:00:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.748+0000 7fc646da9640 1 -- 172.21.15.19:0/3340547221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6400b0fb0 con 0x7fc64006abe0 2024-04-25T10:00:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.749+0000 7fc646da9640 1 -- 172.21.15.19:0/3340547221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64006cc80 con 0x7fc64006abe0 2024-04-25T10:00:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.750+0000 7fc63d7fa640 1 -- 172.21.15.19:0/3340547221 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc628007400 con 0x7fc64006abe0 2024-04-25T10:00:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.750+0000 7fc63d7fa640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc62006b500 0x7fc62006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.750+0000 7fc63d7fa640 1 -- 172.21.15.19:0/3340547221 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc628089730 con 0x7fc64006abe0 2024-04-25T10:00:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.750+0000 7fc63ffff640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc62006b500 0x7fc62006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.751+0000 7fc63ffff640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc62006b500 0x7fc62006d9c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc630034000 tx=0x7fc63002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:49.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:49.754+0000 7fc63d7fa640 1 -- 172.21.15.19:0/3340547221 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6400b3e10 con 0x7fc64006abe0 2024-04-25T10:00:50.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.005+0000 7fc646da9640 1 -- 172.21.15.19:0/3340547221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc64006f790 con 0x7fc64006abe0 2024-04-25T10:00:50.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.006+0000 7fc63d7fa640 1 -- 172.21.15.19:0/3340547221 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1446 (secure 0 0 0) 0x7fc628057dd0 con 0x7fc64006abe0 2024-04-25T10:00:50.010 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:50.010 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":487,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076423385,"bytes_used":2466672640,"bytes_avail":770627440640,"bytes_total":773094113280,"write_bytes_sec":89642208,"read_op_per_sec":0,"write_op_per_sec":87},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:00:23.011235+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:50.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 -- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc62006b500 msgr2=0x7fc62006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:50.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc62006b500 0x7fc62006d9c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc630034000 tx=0x7fc63002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:50.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 -- 172.21.15.19:0/3340547221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64006abe0 msgr2=0x7fc640146100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:50.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64006abe0 0x7fc640146100 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fc62800c3c0 tx=0x7fc62800c880 comp rx=0 tx=0).stop 2024-04-25T10:00:50.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 -- 172.21.15.19:0/3340547221 shutdown_connections 2024-04-25T10:00:50.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc62006b500 0x7fc62006d9c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:50.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64015ec20 0x7fc64009d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:50.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc64006e430 0x7fc6400b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:50.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 --2- 172.21.15.19:0/3340547221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64006abe0 0x7fc640146100 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:50.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 -- 172.21.15.19:0/3340547221 >> 172.21.15.19:0/3340547221 conn(0x7fc6400b5af0 msgr2=0x7fc6400b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:50.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 -- 172.21.15.19:0/3340547221 shutdown_connections 2024-04-25T10:00:50.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:50.008+0000 7fc646da9640 1 -- 172.21.15.19:0/3340547221 wait complete. 2024-04-25T10:00:50.024 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:00:51.026 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.135+0000 7fd1c5bed640 1 Processor -- start 2024-04-25T10:00:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.136+0000 7fd1c5bed640 1 -- start start 2024-04-25T10:00:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.136+0000 7fd1c5bed640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c00d0aa0 0x7fd1c00b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.136+0000 7fd1c5bed640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd1c006fea0 0x7fd1c00b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.136+0000 7fd1c5bed640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c00b3710 con 0x7fd1c015a790 2024-04-25T10:00:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.136+0000 7fd1c5bed640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c00cf0e0 con 0x7fd1c006fea0 2024-04-25T10:00:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.136+0000 7fd1c5bed640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00cf3c0 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.137+0000 7fd1beffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c00d0aa0 0x7fd1c00b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.137+0000 7fd1bffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd1c006fea0 0x7fd1c00b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.137+0000 7fd1beffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c00d0aa0 0x7fd1c00b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:44422/0 (socket says 172.21.15.19:44422) 2024-04-25T10:00:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.137+0000 7fd1beffd640 1 -- 172.21.15.19:0/2479565906 learned_addr learned my addr 172.21.15.19:0/2479565906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.137+0000 7fd1bf7fe640 1 --1- 172.21.15.19:0/2479565906 >> v1:172.21.15.112:6789/0 conn(0x7fd1c015a790 0x7fd1c00d0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:57218/0 (socket says 172.21.15.19:57218) 2024-04-25T10:00:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.137+0000 7fd1bffff640 1 -- 172.21.15.19:0/2479565906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c00d0aa0 msgr2=0x7fd1c00b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.137+0000 7fd1bffff640 1 --2- 172.21.15.19:0/2479565906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c00d0aa0 0x7fd1c00b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.137+0000 7fd1bffff640 1 -- 172.21.15.19:0/2479565906 >> v1:172.21.15.112:6789/0 conn(0x7fd1c015a790 legacy=0x7fd1c00d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.137+0000 7fd1bffff640 1 -- 172.21.15.19:0/2479565906 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c00cf6a0 con 0x7fd1c006fea0 2024-04-25T10:00:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.137+0000 7fd1beffd640 1 --2- 172.21.15.19:0/2479565906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c00d0aa0 0x7fd1c00b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.138+0000 7fd1bffff640 1 --2- 172.21.15.19:0/2479565906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c006fea0 0x7fd1c00b43e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009d90 tx=0x7fd1b802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2326b11cf60f3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.138+0000 7fd1be7fc640 1 -- 172.21.15.19:0/2479565906 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8007740 con 0x7fd1c006fea0 2024-04-25T10:00:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.138+0000 7fd1be7fc640 1 -- 172.21.15.19:0/2479565906 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b80078e0 con 0x7fd1c006fea0 2024-04-25T10:00:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.138+0000 7fd1be7fc640 1 -- 172.21.15.19:0/2479565906 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8007cb0 con 0x7fd1c006fea0 2024-04-25T10:00:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.139+0000 7fd1c5bed640 1 -- 172.21.15.19:0/2479565906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c006fea0 msgr2=0x7fd1c00b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.139+0000 7fd1c5bed640 1 --2- 172.21.15.19:0/2479565906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c006fea0 0x7fd1c00b43e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009d90 tx=0x7fd1b802a660 comp rx=0 tx=0).stop 2024-04-25T10:00:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.139+0000 7fd1c5bed640 1 -- 172.21.15.19:0/2479565906 shutdown_connections 2024-04-25T10:00:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.139+0000 7fd1c5bed640 1 --2- 172.21.15.19:0/2479565906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c006fea0 0x7fd1c00b43e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.139+0000 7fd1c5bed640 1 --2- 172.21.15.19:0/2479565906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c00d0aa0 0x7fd1c00b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.139+0000 7fd1c5bed640 1 -- 172.21.15.19:0/2479565906 >> 172.21.15.19:0/2479565906 conn(0x7fd1c00b2370 msgr2=0x7fd1c00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.139+0000 7fd1c5bed640 1 -- 172.21.15.19:0/2479565906 shutdown_connections 2024-04-25T10:00:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.139+0000 7fd1c5bed640 1 -- 172.21.15.19:0/2479565906 wait complete. 2024-04-25T10:00:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.140+0000 7fd1c5bed640 1 Processor -- start 2024-04-25T10:00:51.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.140+0000 7fd1c5bed640 1 -- start start 2024-04-25T10:00:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.141+0000 7fd1c5bed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c006fea0 0x7fd1c014a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.141+0000 7fd1c5bed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1c00d0aa0 0x7fd1c01e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.141+0000 7fd1c5bed640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c015a790 0x7fd1c01e3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.141+0000 7fd1c5bed640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c015ab70 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.141+0000 7fd1c5bed640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00cf980 con 0x7fd1c006fea0 2024-04-25T10:00:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.141+0000 7fd1c5bed640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c015acf0 con 0x7fd1c015a790 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.141+0000 7fd1beffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1c00d0aa0 0x7fd1c01e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.141+0000 7fd1beffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1c00d0aa0 0x7fd1c01e2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41170/0 (socket says 172.21.15.19:41170) 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.141+0000 7fd1beffd640 1 -- 172.21.15.19:0/466571570 learned_addr learned my addr 172.21.15.19:0/466571570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.141+0000 7fd1bf7fe640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c006fea0 0x7fd1c014a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.141+0000 7fd1bffff640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c015a790 0x7fd1c01e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1beffd640 1 -- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c015a790 msgr2=0x7fd1c01e3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1beffd640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c015a790 0x7fd1c01e3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1beffd640 1 -- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c006fea0 msgr2=0x7fd1c014a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1beffd640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c006fea0 0x7fd1c014a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1beffd640 1 -- 172.21.15.19:0/466571570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c01e3850 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1bf7fe640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c006fea0 0x7fd1c014a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1beffd640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1c00d0aa0 0x7fd1c01e2ce0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400ea00 tx=0x7fd1b400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1a7fff640 1 -- 172.21.15.19:0/466571570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4017070 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1a7fff640 1 -- 172.21.15.19:0/466571570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b400ca60 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1c5bed640 1 -- 172.21.15.19:0/466571570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c01e3b30 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1a7fff640 1 -- 172.21.15.19:0/466571570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400ce10 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.142+0000 7fd1c5bed640 1 -- 172.21.15.19:0/466571570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c01e40f0 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.143+0000 7fd1c5bed640 1 -- 172.21.15.19:0/466571570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c0073990 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.144+0000 7fd1a7fff640 1 -- 172.21.15.19:0/466571570 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1b401a070 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.144+0000 7fd1a7fff640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1ac06b4d0 0x7fd1ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.144+0000 7fd1a7fff640 1 -- 172.21.15.19:0/466571570 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd1b408a610 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.144+0000 7fd1bf7fe640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1ac06b4d0 0x7fd1ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.145+0000 7fd1bf7fe640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1ac06b4d0 0x7fd1ac06d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0000f60 tx=0x7fd1b0001130 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.148+0000 7fd1a7fff640 1 -- 172.21.15.19:0/466571570 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1b4058cb0 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.368+0000 7fd1c5bed640 1 -- 172.21.15.19:0/466571570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1c01e44f0 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.369+0000 7fd1a7fff640 1 -- 172.21.15.19:0/466571570 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd1b405cd50 con 0x7fd1c00d0aa0 2024-04-25T10:00:51.431 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:51.431 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 -- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1ac06b4d0 msgr2=0x7fd1ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1ac06b4d0 0x7fd1ac06d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0000f60 tx=0x7fd1b0001130 comp rx=0 tx=0).stop 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 -- 172.21.15.19:0/466571570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1c00d0aa0 msgr2=0x7fd1c01e2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1c00d0aa0 0x7fd1c01e2ce0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400ea00 tx=0x7fd1b400eec0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 -- 172.21.15.19:0/466571570 shutdown_connections 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1ac06b4d0 0x7fd1ac06d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1c015a790 0x7fd1c01e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1c00d0aa0 0x7fd1c01e2ce0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 --2- 172.21.15.19:0/466571570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c006fea0 0x7fd1c014a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 -- 172.21.15.19:0/466571570 >> 172.21.15.19:0/466571570 conn(0x7fd1c00b2370 msgr2=0x7fd1c0072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 -- 172.21.15.19:0/466571570 shutdown_connections 2024-04-25T10:00:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.371+0000 7fd1c5bed640 1 -- 172.21.15.19:0/466571570 wait complete. 2024-04-25T10:00:51.434 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:00:51.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:51.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.530+0000 7f595781b640 1 Processor -- start 2024-04-25T10:00:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.530+0000 7f595781b640 1 -- start start 2024-04-25T10:00:51.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.531+0000 7f595781b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f595015f5f0 0x7f595015fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.531+0000 7f595781b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59500cd710 con 0x7f595015ba00 2024-04-25T10:00:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.531+0000 7f595781b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5950099b70 con 0x7f59501602e0 2024-04-25T10:00:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.531+0000 7f595781b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595015ff80 con 0x7f595015f5f0 2024-04-25T10:00:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.531+0000 7f59555b7640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f595015ba00 0x7f595009a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51110/0 (socket says 172.21.15.19:51110) 2024-04-25T10:00:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.531+0000 7f59555b7640 1 -- 172.21.15.19:0/1036886005 learned_addr learned my addr 172.21.15.19:0/1036886005 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f5955db8640 1 --2- 172.21.15.19:0/1036886005 >> v2:172.21.15.112:3300/0 conn(0x7f595015f5f0 0x7f595015fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469659208 0 0) 0x7f5948001110 con 0x7f595015ba00 2024-04-25T10:00:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5938003520 con 0x7f595015ba00 2024-04-25T10:00:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1468206587 0 0) 0x7f5938003520 con 0x7f595015ba00 2024-04-25T10:00:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 >> v2:172.21.15.112:3300/0 conn(0x7f595015f5f0 msgr2=0x7f595015fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 --2- 172.21.15.19:0/1036886005 >> v2:172.21.15.112:3300/0 conn(0x7f595015f5f0 0x7f595015fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 >> v1:172.21.15.112:6789/0 conn(0x7f59501602e0 legacy=0x7f59500ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950099e50 con 0x7f595015ba00 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f5948001840 con 0x7f595015ba00 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 >> v1:172.21.15.19:6789/0 conn(0x7f595015ba00 legacy=0x7f595009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 --2- 172.21.15.19:0/1036886005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59380039b0 0x7f5938003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 --2- 172.21.15.19:0/1036886005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5938004800 0x7f5938004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 --2- 172.21.15.19:0/1036886005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59380051c0 0x7f593800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595015ff80 con 0x7f59380051c0 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593800c110 con 0x7f5938004800 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593800c310 con 0x7f59380039b0 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f5955db8640 1 --2- 172.21.15.19:0/1036886005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59380051c0 0x7f593800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f5954db6640 1 --2- 172.21.15.19:0/1036886005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5938004800 0x7f5938004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f59555b7640 1 --2- 172.21.15.19:0/1036886005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59380039b0 0x7f5938003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f5955db8640 1 -- 172.21.15.19:0/1036886005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59380039b0 msgr2=0x7f5938003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f5955db8640 1 --2- 172.21.15.19:0/1036886005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59380039b0 0x7f5938003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f5955db8640 1 -- 172.21.15.19:0/1036886005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5938004800 msgr2=0x7f5938004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f5955db8640 1 --2- 172.21.15.19:0/1036886005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5938004800 0x7f5938004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.532+0000 7f5955db8640 1 -- 172.21.15.19:0/1036886005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5938003520 con 0x7f59380051c0 2024-04-25T10:00:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.533+0000 7f5955db8640 1 -- 172.21.15.19:0/1036886005 reap_dead start 2024-04-25T10:00:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.533+0000 7f5955db8640 1 --2- 172.21.15.19:0/1036886005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59380051c0 0x7f593800b9c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f5940009fc0 tx=0x7f5938005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bea35864b1965cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.533+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940035020 con 0x7f59380051c0 2024-04-25T10:00:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.533+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5940003bd0 con 0x7f59380051c0 2024-04-25T10:00:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.533+0000 7f594ffff640 1 -- 172.21.15.19:0/1036886005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940009c10 con 0x7f59380051c0 2024-04-25T10:00:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.534+0000 7f595781b640 1 -- 172.21.15.19:0/1036886005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59380051c0 msgr2=0x7f593800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.534+0000 7f595781b640 1 --2- 172.21.15.19:0/1036886005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59380051c0 0x7f593800b9c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f5940009fc0 tx=0x7f5938005830 comp rx=0 tx=0).stop 2024-04-25T10:00:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.534+0000 7f595781b640 1 -- 172.21.15.19:0/1036886005 shutdown_connections 2024-04-25T10:00:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.534+0000 7f595781b640 1 --2- 172.21.15.19:0/1036886005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59380051c0 0x7f593800b9c0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.534+0000 7f595781b640 1 -- 172.21.15.19:0/1036886005 >> 172.21.15.19:0/1036886005 conn(0x7f59500b3270 msgr2=0x7f59500b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.534+0000 7f595781b640 1 -- 172.21.15.19:0/1036886005 shutdown_connections 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.534+0000 7f595781b640 1 -- 172.21.15.19:0/1036886005 wait complete. 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.535+0000 7f595781b640 1 Processor -- start 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.535+0000 7f595781b640 1 -- start start 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f595781b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380051c0 0x7f5950151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f595781b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59500b6660 0x7f59500b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f595781b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59500c9f10 0x7f59500ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f595781b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595015bde0 con 0x7f59500b6660 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f595781b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595015fa40 con 0x7f59380051c0 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f595781b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59501f3760 con 0x7f59500c9f10 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f5954db6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59500b6660 0x7f59500b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f5954db6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59500b6660 0x7f59500b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41180/0 (socket says 172.21.15.19:41180) 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f5954db6640 1 -- 172.21.15.19:0/2955657320 learned_addr learned my addr 172.21.15.19:0/2955657320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f5955db8640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59500c9f10 0x7f59500ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f59555b7640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380051c0 0x7f5950151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f5954db6640 1 -- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59500c9f10 msgr2=0x7f59500ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f5954db6640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59500c9f10 0x7f59500ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f5954db6640 1 -- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380051c0 msgr2=0x7f5950151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f5954db6640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380051c0 0x7f5950151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f5954db6640 1 -- 172.21.15.19:0/2955657320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500cabd0 con 0x7f59500b6660 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.536+0000 7f5955db8640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59500c9f10 0x7f59500ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.537+0000 7f5954db6640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59500b6660 0x7f59500b6ac0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f594400f7c0 tx=0x7f594400fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.537+0000 7f594dffb640 1 -- 172.21.15.19:0/2955657320 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594400fe70 con 0x7f59500b6660 2024-04-25T10:00:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.537+0000 7f594dffb640 1 -- 172.21.15.19:0/2955657320 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5944018070 con 0x7f59500b6660 2024-04-25T10:00:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.537+0000 7f595781b640 1 -- 172.21.15.19:0/2955657320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59500caeb0 con 0x7f59500b6660 2024-04-25T10:00:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.537+0000 7f594dffb640 1 -- 172.21.15.19:0/2955657320 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594400d730 con 0x7f59500b6660 2024-04-25T10:00:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.537+0000 7f595781b640 1 -- 172.21.15.19:0/2955657320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59500cb470 con 0x7f59500b6660 2024-04-25T10:00:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.538+0000 7f595781b640 1 -- 172.21.15.19:0/2955657320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5950073990 con 0x7f59500b6660 2024-04-25T10:00:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.538+0000 7f594dffb640 1 -- 172.21.15.19:0/2955657320 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f594400d8d0 con 0x7f59500b6660 2024-04-25T10:00:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.539+0000 7f594dffb640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f593c06b4d0 0x7f593c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.539+0000 7f594dffb640 1 -- 172.21.15.19:0/2955657320 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f594408a2b0 con 0x7f59500b6660 2024-04-25T10:00:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.539+0000 7f59555b7640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f593c06b4d0 0x7f593c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.539+0000 7f59555b7640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f593c06b4d0 0x7f593c06d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5948000f90 tx=0x7f5948009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.543+0000 7f594dffb640 1 -- 172.21.15.19:0/2955657320 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f594401a310 con 0x7f59500b6660 2024-04-25T10:00:51.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.794+0000 7f595781b640 1 -- 172.21.15.19:0/2955657320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f59500ca8d0 con 0x7f59500b6660 2024-04-25T10:00:51.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.794+0000 7f594dffb640 1 -- 172.21.15.19:0/2955657320 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1446 (secure 0 0 0) 0x7f5944059030 con 0x7f59500b6660 2024-04-25T10:00:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:51.799 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":489,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076423385,"bytes_used":2466672640,"bytes_avail":770627440640,"bytes_total":773094113280,"write_bytes_sec":88679389,"read_op_per_sec":0,"write_op_per_sec":86},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:00:23.011235+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:51.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 -- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f593c06b4d0 msgr2=0x7f593c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f593c06b4d0 0x7f593c06d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5948000f90 tx=0x7f5948009040 comp rx=0 tx=0).stop 2024-04-25T10:00:51.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 -- 172.21.15.19:0/2955657320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59500b6660 msgr2=0x7f59500b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59500b6660 0x7f59500b6ac0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f594400f7c0 tx=0x7f594400fc80 comp rx=0 tx=0).stop 2024-04-25T10:00:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 -- 172.21.15.19:0/2955657320 shutdown_connections 2024-04-25T10:00:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f593c06b4d0 0x7f593c06d990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59500c9f10 0x7f59500ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59500b6660 0x7f59500b6ac0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 --2- 172.21.15.19:0/2955657320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380051c0 0x7f5950151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 -- 172.21.15.19:0/2955657320 >> 172.21.15.19:0/2955657320 conn(0x7f59500b3270 msgr2=0x7f59500b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 -- 172.21.15.19:0/2955657320 shutdown_connections 2024-04-25T10:00:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:51.797+0000 7f595781b640 1 -- 172.21.15.19:0/2955657320 wait complete. 2024-04-25T10:00:51.812 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:00:52.813 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.923+0000 7f6983088640 1 Processor -- start 2024-04-25T10:00:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.923+0000 7f6983088640 1 -- start start 2024-04-25T10:00:52.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.923+0000 7f6983088640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f697c13d190 0x7f697c0cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.923+0000 7f6983088640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c071ed0 con 0x7f697c15b320 2024-04-25T10:00:52.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.923+0000 7f6983088640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c15a840 con 0x7f697c071940 2024-04-25T10:00:52.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.923+0000 7f6983088640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697c13d570 con 0x7f697c13d190 2024-04-25T10:00:52.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f697c15b320 0x7f697c071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51124/0 (socket says 172.21.15.19:51124) 2024-04-25T10:00:52.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697bfff640 1 -- 172.21.15.19:0/1619458934 learned_addr learned my addr 172.21.15.19:0/1619458934 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:52.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f6980e24640 1 --2- 172.21.15.19:0/1619458934 >> v2:172.21.15.112:3300/0 conn(0x7f697c13d190 0x7f697c0cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585262599 0 0) 0x7f697c071ed0 con 0x7f697c15b320 2024-04-25T10:00:52.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6964003520 con 0x7f697c15b320 2024-04-25T10:00:52.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550166896 0 0) 0x7f697c15a840 con 0x7f697c071940 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f697c071ed0 con 0x7f697c071940 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4267243691 0 0) 0x7f6964003520 con 0x7f697c15b320 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 >> v2:172.21.15.112:3300/0 conn(0x7f697c13d190 msgr2=0x7f697c0cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 --2- 172.21.15.19:0/1619458934 >> v2:172.21.15.112:3300/0 conn(0x7f697c13d190 0x7f697c0cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 >> v1:172.21.15.112:6789/0 conn(0x7f697c071940 legacy=0x7f697c15dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c15ab20 con 0x7f697c15b320 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f6970001670 con 0x7f697c15b320 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f6980e24640 1 --2- 172.21.15.19:0/1619458934 >> v2:172.21.15.112:3300/0 conn(0x7f697c13d190 0x7f697c0cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 >> v1:172.21.15.19:6789/0 conn(0x7f697c15b320 legacy=0x7f697c071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 --2- 172.21.15.19:0/1619458934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6964003b90 0x7f6964003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 --2- 172.21.15.19:0/1619458934 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6964004800 0x7f6964004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 --2- 172.21.15.19:0/1619458934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6964005210 0x7f696400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c13d570 con 0x7f6964003b90 2024-04-25T10:00:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696400c140 con 0x7f6964005210 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696400c340 con 0x7f6964004800 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.924+0000 7f6980e24640 1 --2- 172.21.15.19:0/1619458934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6964003b90 0x7f6964003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.925+0000 7f6981625640 1 --2- 172.21.15.19:0/1619458934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6964005210 0x7f696400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.925+0000 7f697bfff640 1 --2- 172.21.15.19:0/1619458934 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6964004800 0x7f6964004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.925+0000 7f6980e24640 1 -- 172.21.15.19:0/1619458934 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6964004800 msgr2=0x7f6964004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.925+0000 7f6980e24640 1 --2- 172.21.15.19:0/1619458934 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6964004800 0x7f6964004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.925+0000 7f6980e24640 1 -- 172.21.15.19:0/1619458934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6964005210 msgr2=0x7f696400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.925+0000 7f6980e24640 1 --2- 172.21.15.19:0/1619458934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6964005210 0x7f696400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.925+0000 7f6980e24640 1 -- 172.21.15.19:0/1619458934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6964003520 con 0x7f6964003b90 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.925+0000 7f6981625640 1 -- 172.21.15.19:0/1619458934 reap_dead start 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.925+0000 7f6980e24640 1 --2- 172.21.15.19:0/1619458934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6964003b90 0x7f6964003f90 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f697400f4f0 tx=0x7f6964005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b084edde8c286012 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.926+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974033070 con 0x7f6964003b90 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.926+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697400a2d0 con 0x7f6964003b90 2024-04-25T10:00:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.926+0000 7f697b7fe640 1 -- 172.21.15.19:0/1619458934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697400f140 con 0x7f6964003b90 2024-04-25T10:00:52.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.926+0000 7f6983088640 1 -- 172.21.15.19:0/1619458934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6964003b90 msgr2=0x7f6964003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.926+0000 7f6983088640 1 --2- 172.21.15.19:0/1619458934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6964003b90 0x7f6964003f90 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f697400f4f0 tx=0x7f6964005830 comp rx=0 tx=0).stop 2024-04-25T10:00:52.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.926+0000 7f6983088640 1 -- 172.21.15.19:0/1619458934 shutdown_connections 2024-04-25T10:00:52.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.926+0000 7f6983088640 1 --2- 172.21.15.19:0/1619458934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6964003b90 0x7f6964003f90 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.926+0000 7f6983088640 1 -- 172.21.15.19:0/1619458934 >> 172.21.15.19:0/1619458934 conn(0x7f697c06abe0 msgr2=0x7f697c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:52.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.926+0000 7f6983088640 1 -- 172.21.15.19:0/1619458934 shutdown_connections 2024-04-25T10:00:52.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.926+0000 7f6983088640 1 -- 172.21.15.19:0/1619458934 wait complete. 2024-04-25T10:00:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.928+0000 7f6983088640 1 Processor -- start 2024-04-25T10:00:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.928+0000 7f6983088640 1 -- start start 2024-04-25T10:00:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.928+0000 7f6983088640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6964003b90 0x7f697c13c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f6983088640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f697c13bcd0 0x7f697c13a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f6983088640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f697c0afff0 0x7f697c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f6983088640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0713b0 con 0x7f697c13bcd0 2024-04-25T10:00:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f6983088640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c071230 con 0x7f697c0afff0 2024-04-25T10:00:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f6983088640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c071530 con 0x7f6964003b90 2024-04-25T10:00:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f697bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f697c13bcd0 0x7f697c13a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f697bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f697c13bcd0 0x7f697c13a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41210/0 (socket says 172.21.15.19:41210) 2024-04-25T10:00:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f697bfff640 1 -- 172.21.15.19:0/156901863 learned_addr learned my addr 172.21.15.19:0/156901863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f6981625640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f697c0afff0 0x7f697c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f6980e24640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6964003b90 0x7f697c13c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f697bfff640 1 -- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6964003b90 msgr2=0x7f697c13c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f697bfff640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6964003b90 0x7f697c13c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f697bfff640 1 -- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f697c0afff0 msgr2=0x7f697c0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f697bfff640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f697c0afff0 0x7f697c0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f697bfff640 1 -- 172.21.15.19:0/156901863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c13c1e0 con 0x7f697c13bcd0 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f6981625640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f697c0afff0 0x7f697c0b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f697bfff640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f697c13bcd0 0x7f697c13a060 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f6970004e10 tx=0x7f6970007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f69797fa640 1 -- 172.21.15.19:0/156901863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697001a070 con 0x7f697c13bcd0 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f69797fa640 1 -- 172.21.15.19:0/156901863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697000f7b0 con 0x7f697c13bcd0 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f69797fa640 1 -- 172.21.15.19:0/156901863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697000fba0 con 0x7f697c13bcd0 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f6983088640 1 -- 172.21.15.19:0/156901863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c0af510 con 0x7f697c13bcd0 2024-04-25T10:00:52.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.929+0000 7f6983088640 1 -- 172.21.15.19:0/156901863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c13a5a0 con 0x7f697c13bcd0 2024-04-25T10:00:52.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.930+0000 7f6983088640 1 -- 172.21.15.19:0/156901863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c073990 con 0x7f697c13bcd0 2024-04-25T10:00:52.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.931+0000 7f69797fa640 1 -- 172.21.15.19:0/156901863 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f697001d070 con 0x7f697c13bcd0 2024-04-25T10:00:52.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.931+0000 7f69797fa640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f696806b500 0x7f696806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.932+0000 7f69797fa640 1 -- 172.21.15.19:0/156901863 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6970089b70 con 0x7f697c13bcd0 2024-04-25T10:00:52.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.932+0000 7f6980e24640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f696806b500 0x7f696806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.932+0000 7f6980e24640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f696806b500 0x7f696806d9c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6974035000 tx=0x7f6974030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:52.936+0000 7f69797fa640 1 -- 172.21.15.19:0/156901863 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69700588f0 con 0x7f697c13bcd0 2024-04-25T10:00:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.159+0000 7f6983088640 1 -- 172.21.15.19:0/156901863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f697c0cfeb0 con 0x7f697c13bcd0 2024-04-25T10:00:53.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.159+0000 7f69797fa640 1 -- 172.21.15.19:0/156901863 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f69700581f0 con 0x7f697c13bcd0 2024-04-25T10:00:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:53.164 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:00:53.164 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:00:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 -- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f696806b500 msgr2=0x7f696806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f696806b500 0x7f696806d9c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6974035000 tx=0x7f6974030000 comp rx=0 tx=0).stop 2024-04-25T10:00:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 -- 172.21.15.19:0/156901863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f697c13bcd0 msgr2=0x7f697c13a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f697c13bcd0 0x7f697c13a060 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f6970004e10 tx=0x7f6970007430 comp rx=0 tx=0).stop 2024-04-25T10:00:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 -- 172.21.15.19:0/156901863 shutdown_connections 2024-04-25T10:00:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f696806b500 0x7f696806d9c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f697c0afff0 0x7f697c0b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f697c13bcd0 0x7f697c13a060 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 --2- 172.21.15.19:0/156901863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6964003b90 0x7f697c13c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 -- 172.21.15.19:0/156901863 >> 172.21.15.19:0/156901863 conn(0x7f697c06abe0 msgr2=0x7f697c157900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 -- 172.21.15.19:0/156901863 shutdown_connections 2024-04-25T10:00:53.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.161+0000 7f6983088640 1 -- 172.21.15.19:0/156901863 wait complete. 2024-04-25T10:00:53.177 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:00:53.177 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.273+0000 7fed996bd640 1 Processor -- start 2024-04-25T10:00:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.273+0000 7fed996bd640 1 -- start start 2024-04-25T10:00:53.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.274+0000 7fed996bd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fed9415dc00 0x7fed940c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.274+0000 7fed996bd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9415dfe0 con 0x7fed940d1dd0 2024-04-25T10:00:53.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.274+0000 7fed996bd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940d00c0 con 0x7fed940ceb20 2024-04-25T10:00:53.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.274+0000 7fed996bd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed940d03a0 con 0x7fed9415dc00 2024-04-25T10:00:53.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.274+0000 7fed937fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fed940d1dd0 0x7fed940ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51130/0 (socket says 172.21.15.19:51130) 2024-04-25T10:00:53.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.274+0000 7fed937fe640 1 -- 172.21.15.19:0/34662036 learned_addr learned my addr 172.21.15.19:0/34662036 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:53.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.274+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277660295 0 0) 0x7fed84001490 con 0x7fed940d1dd0 2024-04-25T10:00:53.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.274+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed80003520 con 0x7fed940d1dd0 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.274+0000 7fed92ffd640 1 --2- 172.21.15.19:0/34662036 >> v2:172.21.15.112:3300/0 conn(0x7fed9415dc00 0x7fed940c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 712208323 0 0) 0x7fed80003520 con 0x7fed940d1dd0 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 >> v2:172.21.15.112:3300/0 conn(0x7fed9415dc00 msgr2=0x7fed940c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 --2- 172.21.15.19:0/34662036 >> v2:172.21.15.112:3300/0 conn(0x7fed9415dc00 0x7fed940c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 >> v1:172.21.15.112:6789/0 conn(0x7fed940ceb20 legacy=0x7fed940d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed940cc440 con 0x7fed940d1dd0 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fed84001bc0 con 0x7fed940d1dd0 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 >> v1:172.21.15.19:6789/0 conn(0x7fed940d1dd0 legacy=0x7fed940ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed800039b0 0x7fed80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed80004800 0x7fed80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 0x7fed8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940d03a0 con 0x7fed800051c0 2024-04-25T10:00:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8000c110 con 0x7fed800039b0 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8000c310 con 0x7fed80004800 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed937fe640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 0x7fed8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed92ffd640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed800039b0 0x7fed80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed927fc640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed80004800 0x7fed80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed937fe640 1 -- 172.21.15.19:0/34662036 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed80004800 msgr2=0x7fed80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed937fe640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed80004800 0x7fed80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed937fe640 1 -- 172.21.15.19:0/34662036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed800039b0 msgr2=0x7fed80003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed937fe640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed800039b0 0x7fed80003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed937fe640 1 -- 172.21.15.19:0/34662036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed80003520 con 0x7fed800051c0 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.275+0000 7fed92ffd640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed800039b0 0x7fed80003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed937fe640 1 -- 172.21.15.19:0/34662036 reap_dead start 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed937fe640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 0x7fed8000b9c0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fed940d21e0 tx=0x7fed80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4bd5bc664cf658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8400a670 con 0x7fed800051c0 2024-04-25T10:00:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8402f070 con 0x7fed800051c0 2024-04-25T10:00:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed91ffb640 1 -- 172.21.15.19:0/34662036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84034470 con 0x7fed800051c0 2024-04-25T10:00:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed996bd640 1 -- 172.21.15.19:0/34662036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 msgr2=0x7fed8000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed996bd640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 0x7fed8000b9c0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fed940d21e0 tx=0x7fed80005830 comp rx=0 tx=0).stop 2024-04-25T10:00:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed996bd640 1 -- 172.21.15.19:0/34662036 shutdown_connections 2024-04-25T10:00:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed996bd640 1 --2- 172.21.15.19:0/34662036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 0x7fed8000b9c0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed996bd640 1 -- 172.21.15.19:0/34662036 >> 172.21.15.19:0/34662036 conn(0x7fed9406f5d0 msgr2=0x7fed9406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed996bd640 1 -- 172.21.15.19:0/34662036 shutdown_connections 2024-04-25T10:00:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.276+0000 7fed996bd640 1 -- 172.21.15.19:0/34662036 wait complete. 2024-04-25T10:00:53.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.278+0000 7fed996bd640 1 Processor -- start 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.278+0000 7fed996bd640 1 -- start start 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.278+0000 7fed996bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 0x7fed94156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.278+0000 7fed996bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed94142d30 0x7fed941f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.278+0000 7fed996bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed94145820 0x7fed94153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.278+0000 7fed996bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940cfd90 con 0x7fed800051c0 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.278+0000 7fed996bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940cc130 con 0x7fed94145820 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.278+0000 7fed996bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed940cdd70 con 0x7fed94142d30 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed92ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 0x7fed94156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed92ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 0x7fed94156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41230/0 (socket says 172.21.15.19:41230) 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed92ffd640 1 -- 172.21.15.19:0/743209533 learned_addr learned my addr 172.21.15.19:0/743209533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed927fc640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed94142d30 0x7fed941f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed937fe640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed94145820 0x7fed94153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed92ffd640 1 -- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed94142d30 msgr2=0x7fed941f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed92ffd640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed94142d30 0x7fed941f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed92ffd640 1 -- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed94145820 msgr2=0x7fed94153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed92ffd640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed94145820 0x7fed94153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed92ffd640 1 -- 172.21.15.19:0/743209533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed94156bc0 con 0x7fed800051c0 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed92ffd640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 0x7fed94156500 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00f5b0 tx=0x7fed8c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed8b7fe640 1 -- 172.21.15.19:0/743209533 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c013810 con 0x7fed800051c0 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed8b7fe640 1 -- 172.21.15.19:0/743209533 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8c01a070 con 0x7fed800051c0 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed8b7fe640 1 -- 172.21.15.19:0/743209533 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c013bb0 con 0x7fed800051c0 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed996bd640 1 -- 172.21.15.19:0/743209533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed94156ea0 con 0x7fed800051c0 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.279+0000 7fed996bd640 1 -- 172.21.15.19:0/743209533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed940c9cc0 con 0x7fed800051c0 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.280+0000 7fed996bd640 1 -- 172.21.15.19:0/743209533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed94073990 con 0x7fed800051c0 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.281+0000 7fed8b7fe640 1 -- 172.21.15.19:0/743209533 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed8c012070 con 0x7fed800051c0 2024-04-25T10:00:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.281+0000 7fed8b7fe640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed7406b4d0 0x7fed7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.281+0000 7fed8b7fe640 1 -- 172.21.15.19:0/743209533 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed8c08a300 con 0x7fed800051c0 2024-04-25T10:00:53.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.281+0000 7fed927fc640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed7406b4d0 0x7fed7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.282+0000 7fed927fc640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed7406b4d0 0x7fed7406d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fed941f6a70 tx=0x7fed7c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:53.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.285+0000 7fed8b7fe640 1 -- 172.21.15.19:0/743209533 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed8c01c310 con 0x7fed800051c0 2024-04-25T10:00:53.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.539+0000 7fed996bd640 1 -- 172.21.15.19:0/743209533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fed940c66c0 con 0x7fed800051c0 2024-04-25T10:00:53.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.539+0000 7fed8b7fe640 1 -- 172.21.15.19:0/743209533 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1446 (secure 0 0 0) 0x7fed8c059080 con 0x7fed800051c0 2024-04-25T10:00:53.544 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:53.544 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":490,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076423385,"bytes_used":2466672640,"bytes_avail":770627440640,"bytes_total":773094113280,"write_bytes_sec":88679389,"read_op_per_sec":0,"write_op_per_sec":86},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:00:23.011235+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:53.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 -- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed7406b4d0 msgr2=0x7fed7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed7406b4d0 0x7fed7406d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fed941f6a70 tx=0x7fed7c009040 comp rx=0 tx=0).stop 2024-04-25T10:00:53.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 -- 172.21.15.19:0/743209533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 msgr2=0x7fed94156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 0x7fed94156500 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00f5b0 tx=0x7fed8c00fa70 comp rx=0 tx=0).stop 2024-04-25T10:00:53.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 -- 172.21.15.19:0/743209533 shutdown_connections 2024-04-25T10:00:53.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fed7406b4d0 0x7fed7406d990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed94145820 0x7fed94153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed94142d30 0x7fed941f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 --2- 172.21.15.19:0/743209533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed800051c0 0x7fed94156500 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 -- 172.21.15.19:0/743209533 >> 172.21.15.19:0/743209533 conn(0x7fed9406f5d0 msgr2=0x7fed940b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:53.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 -- 172.21.15.19:0/743209533 shutdown_connections 2024-04-25T10:00:53.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:53.541+0000 7fed996bd640 1 -- 172.21.15.19:0/743209533 wait complete. 2024-04-25T10:00:53.557 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:00:54.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:54.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.668+0000 7fafe2ed1640 1 Processor -- start 2024-04-25T10:00:54.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.668+0000 7fafe2ed1640 1 -- start start 2024-04-25T10:00:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.669+0000 7fafe2ed1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fafdc0d2e90 0x7fafdc0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.669+0000 7fafe2ed1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc0b9940 con 0x7fafdc15dd30 2024-04-25T10:00:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.669+0000 7fafe2ed1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc0d1df0 con 0x7fafdc0d0900 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.669+0000 7fafe2ed1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc0d0d40 con 0x7fafdc0d2e90 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.669+0000 7fafe0c6d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fafdc15dd30 0x7fafdc0b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51132/0 (socket says 172.21.15.19:51132) 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.669+0000 7fafe0c6d640 1 -- 172.21.15.19:0/3621046545 learned_addr learned my addr 172.21.15.19:0/3621046545 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.669+0000 7fafdbfff640 1 --2- 172.21.15.19:0/3621046545 >> v2:172.21.15.112:3300/0 conn(0x7fafdc0d2e90 0x7fafdc0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.669+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392130571 0 0) 0x7fafdc0b9940 con 0x7fafdc15dd30 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.669+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafc4003520 con 0x7fafdc15dd30 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3543230126 0 0) 0x7fafc4003520 con 0x7fafdc15dd30 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 >> v2:172.21.15.112:3300/0 conn(0x7fafdc0d2e90 msgr2=0x7fafdc0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 --2- 172.21.15.19:0/3621046545 >> v2:172.21.15.112:3300/0 conn(0x7fafdc0d2e90 0x7fafdc0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 >> v1:172.21.15.112:6789/0 conn(0x7fafdc0d0900 legacy=0x7fafdc0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc0d20d0 con 0x7fafdc15dd30 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fafcc001670 con 0x7fafdc15dd30 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdbfff640 1 --2- 172.21.15.19:0/3621046545 >> v2:172.21.15.112:3300/0 conn(0x7fafdc0d2e90 0x7fafdc0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 >> v1:172.21.15.19:6789/0 conn(0x7fafdc15dd30 legacy=0x7fafdc0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 --2- 172.21.15.19:0/3621046545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc40039b0 0x7fafc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 --2- 172.21.15.19:0/3621046545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafc4004800 0x7fafc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 --2- 172.21.15.19:0/3621046545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40052e0 0x7fafc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0d0d40 con 0x7fafc40052e0 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc400c1b0 con 0x7fafc40039b0 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc400c3b0 con 0x7fafc4004800 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafe146e640 1 --2- 172.21.15.19:0/3621046545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40052e0 0x7fafc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafe0c6d640 1 --2- 172.21.15.19:0/3621046545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc40039b0 0x7fafc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafdbfff640 1 --2- 172.21.15.19:0/3621046545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafc4004800 0x7fafc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafe146e640 1 -- 172.21.15.19:0/3621046545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafc4004800 msgr2=0x7fafc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafe146e640 1 --2- 172.21.15.19:0/3621046545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafc4004800 0x7fafc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafe146e640 1 -- 172.21.15.19:0/3621046545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc40039b0 msgr2=0x7fafc4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafe146e640 1 --2- 172.21.15.19:0/3621046545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc40039b0 0x7fafc4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.670+0000 7fafe146e640 1 -- 172.21.15.19:0/3621046545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc4003520 con 0x7fafc40052e0 2024-04-25T10:00:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.671+0000 7fafe146e640 1 -- 172.21.15.19:0/3621046545 reap_dead start 2024-04-25T10:00:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.671+0000 7fafe146e640 1 --2- 172.21.15.19:0/3621046545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40052e0 0x7fafc400b9c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fafd4001e20 tx=0x7fafc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6f5c890a8c910c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.671+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4001a70 con 0x7fafc40052e0 2024-04-25T10:00:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.671+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd402ae40 con 0x7fafc40052e0 2024-04-25T10:00:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.671+0000 7fafdb7fe640 1 -- 172.21.15.19:0/3621046545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4003bb0 con 0x7fafc40052e0 2024-04-25T10:00:54.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.672+0000 7fafe2ed1640 1 -- 172.21.15.19:0/3621046545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40052e0 msgr2=0x7fafc400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.672+0000 7fafe2ed1640 1 --2- 172.21.15.19:0/3621046545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40052e0 0x7fafc400b9c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fafd4001e20 tx=0x7fafc4005830 comp rx=0 tx=0).stop 2024-04-25T10:00:54.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.672+0000 7fafe2ed1640 1 -- 172.21.15.19:0/3621046545 shutdown_connections 2024-04-25T10:00:54.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.672+0000 7fafe2ed1640 1 --2- 172.21.15.19:0/3621046545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40052e0 0x7fafc400b9c0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.672+0000 7fafe2ed1640 1 -- 172.21.15.19:0/3621046545 >> 172.21.15.19:0/3621046545 conn(0x7fafdc061c30 msgr2=0x7fafdc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:54.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.672+0000 7fafe2ed1640 1 -- 172.21.15.19:0/3621046545 shutdown_connections 2024-04-25T10:00:54.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.672+0000 7fafe2ed1640 1 -- 172.21.15.19:0/3621046545 wait complete. 2024-04-25T10:00:54.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.673+0000 7fafe2ed1640 1 Processor -- start 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.673+0000 7fafe2ed1640 1 -- start start 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafe2ed1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc40052e0 0x7fafdc0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafe2ed1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafdc0b56b0 0x7fafdc155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafe2ed1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafdc155650 0x7fafdc0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafe2ed1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0d2970 con 0x7fafdc0b56b0 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafe2ed1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0d2690 con 0x7fafc40052e0 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafe2ed1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0d1a10 con 0x7fafdc155650 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafdbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafdc0b56b0 0x7fafdc155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafdbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafdc0b56b0 0x7fafdc155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41246/0 (socket says 172.21.15.19:41246) 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafdbfff640 1 -- 172.21.15.19:0/490166856 learned_addr learned my addr 172.21.15.19:0/490166856 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafe0c6d640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc40052e0 0x7fafdc0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafe146e640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafdc155650 0x7fafdc0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafdbfff640 1 -- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafdc155650 msgr2=0x7fafdc0c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafdbfff640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafdc155650 0x7fafdc0c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafdbfff640 1 -- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc40052e0 msgr2=0x7fafdc0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafdbfff640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc40052e0 0x7fafdc0ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.674+0000 7fafdbfff640 1 -- 172.21.15.19:0/490166856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc154070 con 0x7fafdc0b56b0 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.675+0000 7fafdbfff640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafdc0b56b0 0x7fafdc155110 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fafd0011a70 tx=0x7fafd0011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.675+0000 7fafd97fa640 1 -- 172.21.15.19:0/490166856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd000fbb0 con 0x7fafdc0b56b0 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.675+0000 7fafd97fa640 1 -- 172.21.15.19:0/490166856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd000fd50 con 0x7fafdc0b56b0 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.675+0000 7fafd97fa640 1 -- 172.21.15.19:0/490166856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0004260 con 0x7fafdc0b56b0 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.675+0000 7fafe2ed1640 1 -- 172.21.15.19:0/490166856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc154350 con 0x7fafdc0b56b0 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.675+0000 7fafe2ed1640 1 -- 172.21.15.19:0/490166856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc154910 con 0x7fafdc0b56b0 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.676+0000 7fafe2ed1640 1 -- 172.21.15.19:0/490166856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc073990 con 0x7fafdc0b56b0 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.676+0000 7fafd97fa640 1 -- 172.21.15.19:0/490166856 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafd0012040 con 0x7fafdc0b56b0 2024-04-25T10:00:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.677+0000 7fafd97fa640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc806b500 0x7fafc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.677+0000 7fafd97fa640 1 -- 172.21.15.19:0/490166856 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fafd008a860 con 0x7fafdc0b56b0 2024-04-25T10:00:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.677+0000 7fafe0c6d640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc806b500 0x7fafc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.678+0000 7fafe0c6d640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc806b500 0x7fafc806d9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fafcc001730 tx=0x7fafcc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.681+0000 7fafd97fa640 1 -- 172.21.15.19:0/490166856 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fafd0019030 con 0x7fafdc0b56b0 2024-04-25T10:00:54.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.898+0000 7fafe2ed1640 1 -- 172.21.15.19:0/490166856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafdc154d10 con 0x7fafdc0b56b0 2024-04-25T10:00:54.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.899+0000 7fafd97fa640 1 -- 172.21.15.19:0/490166856 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fafd0059550 con 0x7fafdc0b56b0 2024-04-25T10:00:54.904 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:54.904 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:00:54.905 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:00:54.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 -- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc806b500 msgr2=0x7fafc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc806b500 0x7fafc806d9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fafcc001730 tx=0x7fafcc009040 comp rx=0 tx=0).stop 2024-04-25T10:00:54.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 -- 172.21.15.19:0/490166856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafdc0b56b0 msgr2=0x7fafdc155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafdc0b56b0 0x7fafdc155110 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fafd0011a70 tx=0x7fafd0011f30 comp rx=0 tx=0).stop 2024-04-25T10:00:54.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 -- 172.21.15.19:0/490166856 shutdown_connections 2024-04-25T10:00:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc806b500 0x7fafc806d9c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafdc155650 0x7fafdc0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafdc0b56b0 0x7fafdc155110 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 --2- 172.21.15.19:0/490166856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc40052e0 0x7fafdc0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 -- 172.21.15.19:0/490166856 >> 172.21.15.19:0/490166856 conn(0x7fafdc061c30 msgr2=0x7fafdc15faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 -- 172.21.15.19:0/490166856 shutdown_connections 2024-04-25T10:00:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:54.901+0000 7fafe2ed1640 1 -- 172.21.15.19:0/490166856 wait complete. 2024-04-25T10:00:54.917 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:00:54.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.013+0000 7f175f77d640 1 Processor -- start 2024-04-25T10:00:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.014+0000 7f175f77d640 1 -- start start 2024-04-25T10:00:55.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.014+0000 7f175f77d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17581602e0 0x7f17580ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.014+0000 7f175f77d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f175815f5f0 0x7f175815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.014+0000 7f175f77d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17580cd710 con 0x7f175815ba00 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.014+0000 7f175f77d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1758099b70 con 0x7f17581602e0 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.014+0000 7f175f77d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1758099e50 con 0x7f175815f5f0 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.014+0000 7f175cd18640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17581602e0 0x7f17580ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.014+0000 7f175cd18640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17581602e0 0x7f17580ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41258/0 (socket says 172.21.15.19:41258) 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.014+0000 7f175cd18640 1 -- 172.21.15.19:0/2184275330 learned_addr learned my addr 172.21.15.19:0/2184275330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.015+0000 7f175dd1a640 1 --2- 172.21.15.19:0/2184275330 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f175815f5f0 0x7f175815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.015+0000 7f175cd18640 1 -- 172.21.15.19:0/2184275330 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f175815f5f0 msgr2=0x7f175815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.015+0000 7f175cd18640 1 --2- 172.21.15.19:0/2184275330 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f175815f5f0 0x7f175815fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.015+0000 7f175cd18640 1 -- 172.21.15.19:0/2184275330 >> v1:172.21.15.112:6789/0 conn(0x7f175815ba00 legacy=0x7f175809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.015+0000 7f175cd18640 1 -- 172.21.15.19:0/2184275330 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175809a130 con 0x7f17581602e0 2024-04-25T10:00:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.015+0000 7f175cd18640 1 --2- 172.21.15.19:0/2184275330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17581602e0 0x7f17580ccda0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f174c009870 tx=0x7f174c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d03dd7fbdc407 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.016+0000 7f1757fff640 1 -- 172.21.15.19:0/2184275330 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c037070 con 0x7f17581602e0 2024-04-25T10:00:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.016+0000 7f1757fff640 1 -- 172.21.15.19:0/2184275330 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c02ae10 con 0x7f17581602e0 2024-04-25T10:00:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.016+0000 7f1757fff640 1 -- 172.21.15.19:0/2184275330 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c032660 con 0x7f17581602e0 2024-04-25T10:00:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.016+0000 7f175f77d640 1 -- 172.21.15.19:0/2184275330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17581602e0 msgr2=0x7f17580ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.016+0000 7f175f77d640 1 --2- 172.21.15.19:0/2184275330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17581602e0 0x7f17580ccda0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f174c009870 tx=0x7f174c02a100 comp rx=0 tx=0).stop 2024-04-25T10:00:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.016+0000 7f175f77d640 1 -- 172.21.15.19:0/2184275330 shutdown_connections 2024-04-25T10:00:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.016+0000 7f175f77d640 1 --2- 172.21.15.19:0/2184275330 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f175815f5f0 0x7f175815fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.016+0000 7f175f77d640 1 --2- 172.21.15.19:0/2184275330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17581602e0 0x7f17580ccda0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.016+0000 7f175f77d640 1 -- 172.21.15.19:0/2184275330 >> 172.21.15.19:0/2184275330 conn(0x7f17580b3270 msgr2=0x7f17580b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.016+0000 7f175f77d640 1 -- 172.21.15.19:0/2184275330 shutdown_connections 2024-04-25T10:00:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.016+0000 7f175f77d640 1 -- 172.21.15.19:0/2184275330 wait complete. 2024-04-25T10:00:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.018+0000 7f175f77d640 1 Processor -- start 2024-04-25T10:00:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.018+0000 7f175f77d640 1 -- start start 2024-04-25T10:00:55.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.018+0000 7f175f77d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f175815ba00 0x7f1758159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.018+0000 7f175f77d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f175815f5f0 0x7f17580cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.018+0000 7f175f77d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17581602e0 0x7f17580cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.018+0000 7f175f77d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580cd460 con 0x7f175815ba00 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.018+0000 7f175f77d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580cd2e0 con 0x7f17581602e0 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.018+0000 7f175f77d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17580c52d0 con 0x7f175815f5f0 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175d519640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f175815ba00 0x7f1758159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175d519640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f175815ba00 0x7f1758159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41272/0 (socket says 172.21.15.19:41272) 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175d519640 1 -- 172.21.15.19:0/1999864130 learned_addr learned my addr 172.21.15.19:0/1999864130 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175dd1a640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17581602e0 0x7f17580cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175cd18640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f175815f5f0 0x7f17580cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175d519640 1 -- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f175815f5f0 msgr2=0x7f17580cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175d519640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f175815f5f0 0x7f17580cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175d519640 1 -- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17581602e0 msgr2=0x7f17580cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175d519640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17581602e0 0x7f17580cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175d519640 1 -- 172.21.15.19:0/1999864130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580cbb30 con 0x7f175815ba00 2024-04-25T10:00:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175d519640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f175815ba00 0x7f1758159ca0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f175000c430 tx=0x7f175000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f1755ffb640 1 -- 172.21.15.19:0/1999864130 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750011790 con 0x7f175815ba00 2024-04-25T10:00:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f1755ffb640 1 -- 172.21.15.19:0/1999864130 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1750017070 con 0x7f175815ba00 2024-04-25T10:00:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175f77d640 1 -- 172.21.15.19:0/1999864130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17580b54a0 con 0x7f175815ba00 2024-04-25T10:00:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f1755ffb640 1 -- 172.21.15.19:0/1999864130 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750011bb0 con 0x7f175815ba00 2024-04-25T10:00:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.019+0000 7f175f77d640 1 -- 172.21.15.19:0/1999864130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17580b5a60 con 0x7f175815ba00 2024-04-25T10:00:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.020+0000 7f175f77d640 1 -- 172.21.15.19:0/1999864130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17580b2920 con 0x7f175815ba00 2024-04-25T10:00:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.021+0000 7f1755ffb640 1 -- 172.21.15.19:0/1999864130 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f175001c460 con 0x7f175815ba00 2024-04-25T10:00:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.021+0000 7f1755ffb640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f174406b4d0 0x7f174406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.021+0000 7f1755ffb640 1 -- 172.21.15.19:0/1999864130 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f175008a310 con 0x7f175815ba00 2024-04-25T10:00:55.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.022+0000 7f175cd18640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f174406b4d0 0x7f174406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.022+0000 7f175cd18640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f174406b4d0 0x7f174406d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f174c034000 tx=0x7f174c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.026+0000 7f1755ffb640 1 -- 172.21.15.19:0/1999864130 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17580b2920 con 0x7f175815ba00 2024-04-25T10:00:55.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.275+0000 7f175f77d640 1 -- 172.21.15.19:0/1999864130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1758073990 con 0x7f175815ba00 2024-04-25T10:00:55.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.275+0000 7f1755ffb640 1 -- 172.21.15.19:0/1999864130 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1446 (secure 0 0 0) 0x7f17580cbb30 con 0x7f175815ba00 2024-04-25T10:00:55.281 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:55.281 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":492,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076423385,"bytes_used":2466672640,"bytes_avail":770627440640,"bytes_total":773094113280,"write_bytes_sec":88680916,"read_op_per_sec":0,"write_op_per_sec":86},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:00:23.011235+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 -- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f174406b4d0 msgr2=0x7f174406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f174406b4d0 0x7f174406d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f174c034000 tx=0x7f174c02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 -- 172.21.15.19:0/1999864130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f175815ba00 msgr2=0x7f1758159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f175815ba00 0x7f1758159ca0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f175000c430 tx=0x7f175000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 -- 172.21.15.19:0/1999864130 shutdown_connections 2024-04-25T10:00:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f174406b4d0 0x7f174406d990 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17581602e0 0x7f17580cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f175815f5f0 0x7f17580cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 --2- 172.21.15.19:0/1999864130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f175815ba00 0x7f1758159ca0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 -- 172.21.15.19:0/1999864130 >> 172.21.15.19:0/1999864130 conn(0x7f17580b3270 msgr2=0x7f17580b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 -- 172.21.15.19:0/1999864130 shutdown_connections 2024-04-25T10:00:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:55.277+0000 7f175f77d640 1 -- 172.21.15.19:0/1999864130 wait complete. 2024-04-25T10:00:55.294 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:00:56.294 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.403+0000 7fefe0d7e640 1 Processor -- start 2024-04-25T10:00:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.403+0000 7fefe0d7e640 1 -- start start 2024-04-25T10:00:56.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefe0d7e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fefdc15dbe0 0x7fefdc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefe0d7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fefdc0b92a0 0x7fefdc0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefe0d7e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefdc0b97a0 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefe0d7e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefdc15dfc0 con 0x7fefdc15dbe0 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefe0d7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc0d1a00 con 0x7fefdc0b92a0 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefda59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fefdc15dbe0 0x7fefdc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefda59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fefdc15dbe0 0x7fefdc0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41280/0 (socket says 172.21.15.19:41280) 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefda59c640 1 -- 172.21.15.19:0/2909413112 learned_addr learned my addr 172.21.15.19:0/2909413112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefdad9d640 1 --2- 172.21.15.19:0/2909413112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fefdc0b92a0 0x7fefdc0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefda59c640 1 -- 172.21.15.19:0/2909413112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fefdc0b92a0 msgr2=0x7fefdc0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefda59c640 1 --2- 172.21.15.19:0/2909413112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fefdc0b92a0 0x7fefdc0d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefda59c640 1 -- 172.21.15.19:0/2909413112 >> v1:172.21.15.112:6789/0 conn(0x7fefdc0b2cb0 legacy=0x7fefdc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.404+0000 7fefda59c640 1 -- 172.21.15.19:0/2909413112 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc0d1c50 con 0x7fefdc15dbe0 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.405+0000 7fefda59c640 1 --2- 172.21.15.19:0/2909413112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefdc15dbe0 0x7fefdc0d0910 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009870 tx=0x7fefcc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391e0fb2e4bbede server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.405+0000 7fefd959a640 1 -- 172.21.15.19:0/2909413112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc037070 con 0x7fefdc15dbe0 2024-04-25T10:00:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.405+0000 7fefd959a640 1 -- 172.21.15.19:0/2909413112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc032440 con 0x7fefdc15dbe0 2024-04-25T10:00:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.405+0000 7fefd959a640 1 -- 172.21.15.19:0/2909413112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc032810 con 0x7fefdc15dbe0 2024-04-25T10:00:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.406+0000 7fefe0d7e640 1 -- 172.21.15.19:0/2909413112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefdc15dbe0 msgr2=0x7fefdc0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.406+0000 7fefe0d7e640 1 --2- 172.21.15.19:0/2909413112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefdc15dbe0 0x7fefdc0d0910 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009870 tx=0x7fefcc02a210 comp rx=0 tx=0).stop 2024-04-25T10:00:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.406+0000 7fefe0d7e640 1 -- 172.21.15.19:0/2909413112 shutdown_connections 2024-04-25T10:00:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.406+0000 7fefe0d7e640 1 --2- 172.21.15.19:0/2909413112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fefdc0b92a0 0x7fefdc0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.406+0000 7fefe0d7e640 1 --2- 172.21.15.19:0/2909413112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefdc15dbe0 0x7fefdc0d0910 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.406+0000 7fefe0d7e640 1 -- 172.21.15.19:0/2909413112 >> 172.21.15.19:0/2909413112 conn(0x7fefdc061a20 msgr2=0x7fefdc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.406+0000 7fefe0d7e640 1 -- 172.21.15.19:0/2909413112 shutdown_connections 2024-04-25T10:00:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.406+0000 7fefe0d7e640 1 -- 172.21.15.19:0/2909413112 wait complete. 2024-04-25T10:00:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.407+0000 7fefe0d7e640 1 Processor -- start 2024-04-25T10:00:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.407+0000 7fefe0d7e640 1 -- start start 2024-04-25T10:00:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefe0d7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefdc0b2cb0 0x7fefdc0c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefe0d7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fefdc0b92a0 0x7fefdc15f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefe0d7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefdc15dbe0 0x7fefdc15f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefe0d7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc0b6ab0 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefe0d7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc0d2210 con 0x7fefdc15dbe0 2024-04-25T10:00:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefe0d7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc0b6c30 con 0x7fefdc0b92a0 2024-04-25T10:00:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefda59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefdc0b2cb0 0x7fefdc0c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefda59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefdc0b2cb0 0x7fefdc0c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41290/0 (socket says 172.21.15.19:41290) 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefda59c640 1 -- 172.21.15.19:0/1292735846 learned_addr learned my addr 172.21.15.19:0/1292735846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefdad9d640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefdc15dbe0 0x7fefdc15f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefd9d9b640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fefdc0b92a0 0x7fefdc15f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefda59c640 1 -- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fefdc0b92a0 msgr2=0x7fefdc15f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefda59c640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fefdc0b92a0 0x7fefdc15f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefda59c640 1 -- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefdc15dbe0 msgr2=0x7fefdc15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefda59c640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefdc15dbe0 0x7fefdc15f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefda59c640 1 -- 172.21.15.19:0/1292735846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc0b3f80 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.408+0000 7fefdad9d640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefdc15dbe0 0x7fefdc15f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.409+0000 7fefda59c640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefdc0b2cb0 0x7fefdc0c9c10 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fefcc000c00 tx=0x7fefcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.409+0000 7fefcaffd640 1 -- 172.21.15.19:0/1292735846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc041070 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.409+0000 7fefcaffd640 1 -- 172.21.15.19:0/1292735846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc03c870 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.409+0000 7fefe0d7e640 1 -- 172.21.15.19:0/1292735846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc0b4260 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.409+0000 7fefcaffd640 1 -- 172.21.15.19:0/1292735846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc037040 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.409+0000 7fefe0d7e640 1 -- 172.21.15.19:0/1292735846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc0b4820 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.410+0000 7fefe0d7e640 1 -- 172.21.15.19:0/1292735846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefdc073990 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.410+0000 7fefcaffd640 1 -- 172.21.15.19:0/1292735846 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefcc044070 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.411+0000 7fefcaffd640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fefbc06b4d0 0x7fefbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.411+0000 7fefcaffd640 1 -- 172.21.15.19:0/1292735846 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fefcc0aae80 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.411+0000 7fefd9d9b640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fefbc06b4d0 0x7fefbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.412+0000 7fefd9d9b640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fefbc06b4d0 0x7fefbc06d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fefdc15d590 tx=0x7fefd0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.415+0000 7fefcaffd640 1 -- 172.21.15.19:0/1292735846 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fefcc079c00 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.633+0000 7fefe0d7e640 1 -- 172.21.15.19:0/1292735846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefdc155300 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.634+0000 7fefcaffd640 1 -- 172.21.15.19:0/1292735846 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fefcc079500 con 0x7fefdc0b2cb0 2024-04-25T10:00:56.639 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:56.640 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:00:56.640 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:00:56.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 -- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fefbc06b4d0 msgr2=0x7fefbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fefbc06b4d0 0x7fefbc06d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fefdc15d590 tx=0x7fefd0009040 comp rx=0 tx=0).stop 2024-04-25T10:00:56.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 -- 172.21.15.19:0/1292735846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefdc0b2cb0 msgr2=0x7fefdc0c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefdc0b2cb0 0x7fefdc0c9c10 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fefcc000c00 tx=0x7fefcc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 -- 172.21.15.19:0/1292735846 shutdown_connections 2024-04-25T10:00:56.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fefbc06b4d0 0x7fefbc06d990 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefdc15dbe0 0x7fefdc15f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fefdc0b92a0 0x7fefdc15f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 --2- 172.21.15.19:0/1292735846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefdc0b2cb0 0x7fefdc0c9c10 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 -- 172.21.15.19:0/1292735846 >> 172.21.15.19:0/1292735846 conn(0x7fefdc061a20 msgr2=0x7fefdc0d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:56.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 -- 172.21.15.19:0/1292735846 shutdown_connections 2024-04-25T10:00:56.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.636+0000 7fefe0d7e640 1 -- 172.21.15.19:0/1292735846 wait complete. 2024-04-25T10:00:56.652 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:00:56.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:56.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.747+0000 7f33c8cab640 1 Processor -- start 2024-04-25T10:00:56.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.747+0000 7f33c8cab640 1 -- start start 2024-04-25T10:00:56.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c8cab640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33c40d2e90 0x7f33c40d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c8cab640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f33c40d0900 0x7f33c40b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c8cab640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c40b9940 con 0x7f33c415dd30 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c8cab640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c40d0d70 con 0x7f33c40d0900 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c8cab640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40d1df0 con 0x7f33c40d2e90 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c2d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f33c40d0900 0x7f33c40b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c2d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f33c40d0900 0x7f33c40b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41296/0 (socket says 172.21.15.19:41296) 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c2d9d640 1 -- 172.21.15.19:0/3757119266 learned_addr learned my addr 172.21.15.19:0/3757119266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c1d9b640 1 --2- 172.21.15.19:0/3757119266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33c40d2e90 0x7f33c40d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c2d9d640 1 -- 172.21.15.19:0/3757119266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33c40d2e90 msgr2=0x7f33c40d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c2d9d640 1 --2- 172.21.15.19:0/3757119266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33c40d2e90 0x7f33c40d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c2d9d640 1 -- 172.21.15.19:0/3757119266 >> v1:172.21.15.112:6789/0 conn(0x7f33c415dd30 legacy=0x7f33c40b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.748+0000 7f33c2d9d640 1 -- 172.21.15.19:0/3757119266 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c40d20d0 con 0x7f33c40d0900 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.749+0000 7f33c2d9d640 1 --2- 172.21.15.19:0/3757119266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33c40d0900 0x7f33c40b6a40 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009870 tx=0x7f33bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad40a81d2aba2e49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.749+0000 7f33c159a640 1 -- 172.21.15.19:0/3757119266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc037070 con 0x7f33c40d0900 2024-04-25T10:00:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.749+0000 7f33c159a640 1 -- 172.21.15.19:0/3757119266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33bc032440 con 0x7f33c40d0900 2024-04-25T10:00:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.749+0000 7f33c159a640 1 -- 172.21.15.19:0/3757119266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc032810 con 0x7f33c40d0900 2024-04-25T10:00:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.750+0000 7f33c8cab640 1 -- 172.21.15.19:0/3757119266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33c40d0900 msgr2=0x7f33c40b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.750+0000 7f33c8cab640 1 --2- 172.21.15.19:0/3757119266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33c40d0900 0x7f33c40b6a40 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009870 tx=0x7f33bc02a180 comp rx=0 tx=0).stop 2024-04-25T10:00:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.750+0000 7f33c8cab640 1 -- 172.21.15.19:0/3757119266 shutdown_connections 2024-04-25T10:00:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.750+0000 7f33c8cab640 1 --2- 172.21.15.19:0/3757119266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33c40d0900 0x7f33c40b6a40 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.750+0000 7f33c8cab640 1 --2- 172.21.15.19:0/3757119266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33c40d2e90 0x7f33c40d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.750+0000 7f33c8cab640 1 -- 172.21.15.19:0/3757119266 >> 172.21.15.19:0/3757119266 conn(0x7f33c4061c30 msgr2=0x7f33c406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.750+0000 7f33c8cab640 1 -- 172.21.15.19:0/3757119266 shutdown_connections 2024-04-25T10:00:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.750+0000 7f33c8cab640 1 -- 172.21.15.19:0/3757119266 wait complete. 2024-04-25T10:00:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.751+0000 7f33c8cab640 1 Processor -- start 2024-04-25T10:00:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.751+0000 7f33c8cab640 1 -- start start 2024-04-25T10:00:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c8cab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c40d0900 0x7f33c4149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c8cab640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33c40d2e90 0x7f33c40b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c8cab640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33c415dd30 0x7f33c40b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c8cab640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40d2970 con 0x7f33c40d2e90 2024-04-25T10:00:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c8cab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40d2690 con 0x7f33c40d0900 2024-04-25T10:00:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c8cab640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40d1a10 con 0x7f33c415dd30 2024-04-25T10:00:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33c40d2e90 0x7f33c40b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33c40d2e90 0x7f33c40b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41304/0 (socket says 172.21.15.19:41304) 2024-04-25T10:00:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c1d9b640 1 -- 172.21.15.19:0/953123856 learned_addr learned my addr 172.21.15.19:0/953123856 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c2d9d640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33c415dd30 0x7f33c40b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c259c640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c40d0900 0x7f33c4149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c1d9b640 1 -- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33c415dd30 msgr2=0x7f33c40b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c1d9b640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33c415dd30 0x7f33c40b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c1d9b640 1 -- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c40d0900 msgr2=0x7f33c4149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c1d9b640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c40d0900 0x7f33c4149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.752+0000 7f33c1d9b640 1 -- 172.21.15.19:0/953123856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c41f4140 con 0x7f33c40d2e90 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.753+0000 7f33c1d9b640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33c40d2e90 0x7f33c40b5160 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00c7f0 tx=0x7f33ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.753+0000 7f33baffd640 1 -- 172.21.15.19:0/953123856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac015070 con 0x7f33c40d2e90 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.753+0000 7f33baffd640 1 -- 172.21.15.19:0/953123856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac007820 con 0x7f33c40d2e90 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.753+0000 7f33c8cab640 1 -- 172.21.15.19:0/953123856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c40cbda0 con 0x7f33c40d2e90 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.753+0000 7f33baffd640 1 -- 172.21.15.19:0/953123856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac007c10 con 0x7f33c40d2e90 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.753+0000 7f33c8cab640 1 -- 172.21.15.19:0/953123856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c40cc360 con 0x7f33c40d2e90 2024-04-25T10:00:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.754+0000 7f33c8cab640 1 -- 172.21.15.19:0/953123856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c40cd0a0 con 0x7f33c40d2e90 2024-04-25T10:00:56.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.754+0000 7f33baffd640 1 -- 172.21.15.19:0/953123856 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33ac018070 con 0x7f33c40d2e90 2024-04-25T10:00:56.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.755+0000 7f33baffd640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f33a406b4d0 0x7f33a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.755+0000 7f33baffd640 1 -- 172.21.15.19:0/953123856 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f33ac08a4d0 con 0x7f33c40d2e90 2024-04-25T10:00:56.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.755+0000 7f33c259c640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f33a406b4d0 0x7f33a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.755+0000 7f33c259c640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f33a406b4d0 0x7f33a406d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009f20 tx=0x7f33b40094d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:56.759+0000 7f33baffd640 1 -- 172.21.15.19:0/953123856 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33ac0591c0 con 0x7f33c40d2e90 2024-04-25T10:00:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.006+0000 7f33c8cab640 1 -- 172.21.15.19:0/953123856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f33c4073990 con 0x7f33c40d2e90 2024-04-25T10:00:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.007+0000 7f33baffd640 1 -- 172.21.15.19:0/953123856 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f33ac058ac0 con 0x7f33c40d2e90 2024-04-25T10:00:57.012 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:57.012 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":494,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2562392064,"bytes_avail":770531721216,"bytes_total":773094113280,"write_bytes_sec":88694783,"read_op_per_sec":0,"write_op_per_sec":87},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:00:55.018290+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:57.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 -- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f33a406b4d0 msgr2=0x7f33a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f33a406b4d0 0x7f33a406d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009f20 tx=0x7f33b40094d0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 -- 172.21.15.19:0/953123856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33c40d2e90 msgr2=0x7f33c40b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33c40d2e90 0x7f33c40b5160 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00c7f0 tx=0x7f33ac00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 -- 172.21.15.19:0/953123856 shutdown_connections 2024-04-25T10:00:57.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f33a406b4d0 0x7f33a406d990 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33c415dd30 0x7f33c40b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33c40d2e90 0x7f33c40b5160 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 --2- 172.21.15.19:0/953123856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c40d0900 0x7f33c4149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 -- 172.21.15.19:0/953123856 >> 172.21.15.19:0/953123856 conn(0x7f33c4061c30 msgr2=0x7f33c40c8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 -- 172.21.15.19:0/953123856 shutdown_connections 2024-04-25T10:00:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.009+0000 7f33c8cab640 1 -- 172.21.15.19:0/953123856 wait complete. 2024-04-25T10:00:57.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T10:00:57.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:57.025 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T10:00:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.125+0000 7f3f23858640 1 Processor -- start 2024-04-25T10:00:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.125+0000 7f3f23858640 1 -- start start 2024-04-25T10:00:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.126+0000 7f3f23858640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f1c0b9750 0x7f3f1c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.126+0000 7f3f23858640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c0cf060 con 0x7f3f1c15da90 2024-04-25T10:00:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.126+0000 7f3f23858640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c161d70 con 0x7f3f1c0cebb0 2024-04-25T10:00:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.126+0000 7f3f23858640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1c162600 con 0x7f3f1c0b9750 2024-04-25T10:00:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.126+0000 7f3f20df3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f1c0b9750 0x7f3f1c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.126+0000 7f3f215f4640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3f1c15da90 0x7f3f1c161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51148/0 (socket says 172.21.15.19:51148) 2024-04-25T10:00:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.126+0000 7f3f215f4640 1 -- 172.21.15.19:0/2953105863 learned_addr learned my addr 172.21.15.19:0/2953105863 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.126+0000 7f3f20df3640 1 -- 172.21.15.19:0/2953105863 >> v1:172.21.15.112:6789/0 conn(0x7f3f1c0cebb0 legacy=0x7f3f1c0d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.126+0000 7f3f13fff640 1 -- 172.21.15.19:0/2953105863 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871199869 0 0) 0x7f3f1c0cf060 con 0x7f3f1c15da90 2024-04-25T10:00:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.126+0000 7f3f20df3640 1 -- 172.21.15.19:0/2953105863 >> v1:172.21.15.19:6789/0 conn(0x7f3f1c15da90 legacy=0x7f3f1c161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.126+0000 7f3f20df3640 1 -- 172.21.15.19:0/2953105863 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c162040 con 0x7f3f1c0b9750 2024-04-25T10:00:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.127+0000 7f3f20df3640 1 --2- 172.21.15.19:0/2953105863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f1c0b9750 0x7f3f1c0ce670 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0097a0 tx=0x7f3f0c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6765dc350666b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.127+0000 7f3f13fff640 1 -- 172.21.15.19:0/2953105863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c037070 con 0x7f3f1c0b9750 2024-04-25T10:00:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.127+0000 7f3f13fff640 1 -- 172.21.15.19:0/2953105863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c02ae50 con 0x7f3f1c0b9750 2024-04-25T10:00:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.127+0000 7f3f13fff640 1 -- 172.21.15.19:0/2953105863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c032660 con 0x7f3f1c0b9750 2024-04-25T10:00:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.128+0000 7f3f23858640 1 -- 172.21.15.19:0/2953105863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f1c0b9750 msgr2=0x7f3f1c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.128+0000 7f3f23858640 1 --2- 172.21.15.19:0/2953105863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f1c0b9750 0x7f3f1c0ce670 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0097a0 tx=0x7f3f0c02a140 comp rx=0 tx=0).stop 2024-04-25T10:00:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.128+0000 7f3f23858640 1 -- 172.21.15.19:0/2953105863 shutdown_connections 2024-04-25T10:00:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.128+0000 7f3f23858640 1 --2- 172.21.15.19:0/2953105863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f1c0b9750 0x7f3f1c0ce670 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.128+0000 7f3f23858640 1 -- 172.21.15.19:0/2953105863 >> 172.21.15.19:0/2953105863 conn(0x7f3f1c061a20 msgr2=0x7f3f1c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.128+0000 7f3f23858640 1 -- 172.21.15.19:0/2953105863 shutdown_connections 2024-04-25T10:00:57.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.128+0000 7f3f23858640 1 -- 172.21.15.19:0/2953105863 wait complete. 2024-04-25T10:00:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.129+0000 7f3f23858640 1 Processor -- start 2024-04-25T10:00:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.129+0000 7f3f23858640 1 -- start start 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f23858640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f1c0cebb0 0x7f3f1c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f23858640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f1c15da90 0x7f3f1c14ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f23858640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f1c0b8050 0x7f3f1c0b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f23858640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0d1df0 con 0x7f3f1c0b8050 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f23858640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0d1b10 con 0x7f3f1c0cebb0 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f23858640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c15de70 con 0x7f3f1c15da90 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f21df5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f1c0b8050 0x7f3f1c0b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f21df5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f1c0b8050 0x7f3f1c0b84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41328/0 (socket says 172.21.15.19:41328) 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f21df5640 1 -- 172.21.15.19:0/3815000649 learned_addr learned my addr 172.21.15.19:0/3815000649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f215f4640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f1c0cebb0 0x7f3f1c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f20df3640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f1c15da90 0x7f3f1c14ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f21df5640 1 -- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f1c15da90 msgr2=0x7f3f1c14ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f21df5640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f1c15da90 0x7f3f1c14ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f21df5640 1 -- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f1c0cebb0 msgr2=0x7f3f1c15f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f21df5640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f1c0cebb0 0x7f3f1c15f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.130+0000 7f3f21df5640 1 -- 172.21.15.19:0/3815000649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c15e080 con 0x7f3f1c0b8050 2024-04-25T10:00:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.131+0000 7f3f21df5640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f1c0b8050 0x7f3f1c0b84d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800c7b0 tx=0x7f3f1800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.131+0000 7f3f11ffb640 1 -- 172.21.15.19:0/3815000649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f18004b60 con 0x7f3f1c0b8050 2024-04-25T10:00:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.131+0000 7f3f11ffb640 1 -- 172.21.15.19:0/3815000649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f18004d00 con 0x7f3f1c0b8050 2024-04-25T10:00:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.131+0000 7f3f23858640 1 -- 172.21.15.19:0/3815000649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c15e360 con 0x7f3f1c0b8050 2024-04-25T10:00:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.131+0000 7f3f11ffb640 1 -- 172.21.15.19:0/3815000649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1800aa30 con 0x7f3f1c0b8050 2024-04-25T10:00:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.131+0000 7f3f23858640 1 -- 172.21.15.19:0/3815000649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c15e920 con 0x7f3f1c0b8050 2024-04-25T10:00:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.132+0000 7f3f11ffb640 1 -- 172.21.15.19:0/3815000649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f1800abd0 con 0x7f3f1c0b8050 2024-04-25T10:00:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.133+0000 7f3f11ffb640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f0006b4d0 0x7f3f0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.133+0000 7f3f11ffb640 1 -- 172.21.15.19:0/3815000649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3f1808a490 con 0x7f3f1c0b8050 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.133+0000 7f3f215f4640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f0006b4d0 0x7f3f0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.133+0000 7f3f215f4640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f0006b4d0 0x7f3f0006d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000f60 tx=0x7f3f14007d50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.134+0000 7f3f26632740 1 Processor -- start 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.134+0000 7f3f26632740 1 -- start start 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.134+0000 7f3f26632740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be0647ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.134+0000 7f3f26632740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be064b7370 0x55be0647b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.134+0000 7f3f26632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f26632740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be063ddb90 con 0x55be0647aca0 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f26632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be06386e40 con 0x55be064a5760 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f26632740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be063d9c30 con 0x55be064b7370 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f215f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be0647ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f215f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be0647ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41344/0 (socket says 172.21.15.19:41344) 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f215f4640 1 -- 172.21.15.19:0/2255862105 learned_addr learned my addr 172.21.15.19:0/2255862105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f20df3640 1 --2- 172.21.15.19:0/2255862105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be064b7370 0x55be0647b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f21df5640 1 --2- 172.21.15.19:0/2255862105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f215f4640 1 -- 172.21.15.19:0/2255862105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be064b7370 msgr2=0x55be0647b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f215f4640 1 --2- 172.21.15.19:0/2255862105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be064b7370 0x55be0647b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f215f4640 1 -- 172.21.15.19:0/2255862105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 msgr2=0x55be064a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f215f4640 1 --2- 172.21.15.19:0/2255862105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f215f4640 1 -- 172.21.15.19:0/2255862105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be061bc7d0 con 0x55be0647aca0 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f215f4640 1 --2- 172.21.15.19:0/2255862105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be0647ce00 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f3f14004fb0 tx=0x7f3f14013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f06e8d7e162ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f06ffd640 1 -- 172.21.15.19:0/2255862105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14026070 con 0x55be0647aca0 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f06ffd640 1 -- 172.21.15.19:0/2255862105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f14013b80 con 0x55be0647aca0 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.135+0000 7f3f06ffd640 1 -- 172.21.15.19:0/2255862105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14012350 con 0x55be0647aca0 2024-04-25T10:00:57.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.136+0000 7f3f26632740 1 -- 172.21.15.19:0/2255862105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 msgr2=0x55be0647ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.136+0000 7f3f26632740 1 --2- 172.21.15.19:0/2255862105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be0647ce00 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f3f14004fb0 tx=0x7f3f14013000 comp rx=0 tx=0).stop 2024-04-25T10:00:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.136+0000 7f3f26632740 1 -- 172.21.15.19:0/2255862105 shutdown_connections 2024-04-25T10:00:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.136+0000 7f3f26632740 1 --2- 172.21.15.19:0/2255862105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.136+0000 7f3f26632740 1 --2- 172.21.15.19:0/2255862105 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be064b7370 0x55be0647b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.136+0000 7f3f26632740 1 --2- 172.21.15.19:0/2255862105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be0647ce00 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.136+0000 7f3f26632740 1 -- 172.21.15.19:0/2255862105 >> 172.21.15.19:0/2255862105 conn(0x55be06442870 msgr2=0x55be064c0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.136+0000 7f3f26632740 1 -- 172.21.15.19:0/2255862105 shutdown_connections 2024-04-25T10:00:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.136+0000 7f3f26632740 1 -- 172.21.15.19:0/2255862105 wait complete. 2024-04-25T10:00:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.136+0000 7f3f26632740 1 -- start start 2024-04-25T10:00:57.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.136+0000 7f3f26632740 1 Processor -- start 2024-04-25T10:00:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.137+0000 7f3f26632740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be064940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.137+0000 7f3f26632740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be064a5760 0x55be064a7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.137+0000 7f3f26632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064b7370 0x55be064a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.137+0000 7f3f26632740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0639a3e0 con 0x55be0647aca0 2024-04-25T10:00:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.137+0000 7f3f26632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be064a32b0 con 0x55be064b7370 2024-04-25T10:00:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.137+0000 7f3f26632740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be064b99b0 con 0x55be064a5760 2024-04-25T10:00:57.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.137+0000 7f3f26632740 10 client.0 _ll_register_callbacks cb 0x55be06399db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.137+0000 7f3f215f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be064940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.137+0000 7f3f215f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be064940a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41350/0 (socket says 172.21.15.19:41350) 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.137+0000 7f3f215f4640 1 -- 172.21.15.19:0/878911183 learned_addr learned my addr 172.21.15.19:0/878911183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3f21df5640 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064b7370 0x55be064a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3f20df3640 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be064a5760 0x55be064a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3f215f4640 1 -- 172.21.15.19:0/878911183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be064a5760 msgr2=0x55be064a7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3f215f4640 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be064a5760 0x55be064a7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3f215f4640 1 -- 172.21.15.19:0/878911183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064b7370 msgr2=0x55be064a7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3f215f4640 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064b7370 0x55be064a7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3f215f4640 1 -- 172.21.15.19:0/878911183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be060b20f0 con 0x55be0647aca0 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3f215f4640 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be064940a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f3f140137c0 tx=0x7f3f14000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf37dcb2dc1dda6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3efbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3efbfff640 1 -- 172.21.15.19:0/878911183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14030020 con 0x55be0647aca0 2024-04-25T10:00:57.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3efbfff640 1 -- 172.21.15.19:0/878911183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3f14021c70 con 0x55be0647aca0 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3f26632740 1 -- 172.21.15.19:0/878911183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f14013b80 con 0x55be0647aca0 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3efbfff640 1 -- 172.21.15.19:0/878911183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f1403a050 con 0x55be0647aca0 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3f26632740 10 client.8874 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3f26632740 1 -- 172.21.15.19:0/878911183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55be0621cf60 con 0x55be0647aca0 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.138+0000 7f3efbfff640 1 -- 172.21.15.19:0/878911183 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f14013b80 con 0x55be0647aca0 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.139+0000 7f3efbfff640 1 -- 172.21.15.19:0/878911183 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3f1403d020 con 0x55be0647aca0 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.139+0000 7f3f26632740 10 client.8874 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.139+0000 7f3f26632740 10 client.8874 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.139+0000 7f3f26632740 20 client.8874 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.139+0000 7f3f26632740 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55be064ba6e0 0x55be064d3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.139+0000 7f3f26632740 4 client.8874 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.139+0000 7f3f26632740 1 -- 172.21.15.19:0/878911183 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be063eb0d0 con 0x55be064ba6e0 2024-04-25T10:00:57.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.139+0000 7f3f20df3640 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55be064ba6e0 0x55be064d3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.139+0000 7f3efbfff640 10 client.8874 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.139+0000 7f3f20df3640 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55be064ba6e0 0x55be064d3d80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3992d5eaca4d30 server_cookie=f2c40115793a1bde in_seq=0 out_seq=0 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3efbfff640 1 -- 172.21.15.19:0/878911183 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be063eb0d0 con 0x55be064ba6e0 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3efbfff640 10 client.8874 handle_command_reply: tid=0 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 1 client.8874 shutdown 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 1 -- 172.21.15.19:0/878911183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 msgr2=0x55be064940a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be064940a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f3f140137c0 tx=0x7f3f14000ed0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 1 -- 172.21.15.19:0/878911183 shutdown_connections 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55be064ba6e0 0x55be064d3d80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064b7370 0x55be064a7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be064a5760 0x55be064a7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3efbfff640 0 client.8874 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 1 --2- 172.21.15.19:0/878911183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be0647aca0 0x55be064940a0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 1 -- 172.21.15.19:0/878911183 >> 172.21.15.19:0/878911183 conn(0x55be06442870 msgr2=0x55be064a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 1 -- 172.21.15.19:0/878911183 shutdown_connections 2024-04-25T10:00:57.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 1 -- 172.21.15.19:0/878911183 wait complete. 2024-04-25T10:00:57.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.140+0000 7f3f26632740 20 client.8874 trim_cache size 0 max 16384 2024-04-25T10:00:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.155+0000 7f3f26632740 1 Processor -- start 2024-04-25T10:00:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.155+0000 7f3f26632740 1 -- start start 2024-04-25T10:00:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f26632740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be0647aca0 0x55be064c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f26632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f26632740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f26632740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c200440 con 0x55be064b7370 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f26632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c1ff610 con 0x55be064a5760 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f26632740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0cc6d0 con 0x55be0647aca0 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f21df5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f21df5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41354/0 (socket says 172.21.15.19:41354) 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f21df5640 1 -- 172.21.15.19:0/2839608083 learned_addr learned my addr 172.21.15.19:0/2839608083 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f215f4640 1 --2- 172.21.15.19:0/2839608083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be0647aca0 0x55be064c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f20df3640 1 --2- 172.21.15.19:0/2839608083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f21df5640 1 -- 172.21.15.19:0/2839608083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be0647aca0 msgr2=0x55be064c0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f21df5640 1 --2- 172.21.15.19:0/2839608083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be0647aca0 0x55be064c0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f21df5640 1 -- 172.21.15.19:0/2839608083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 msgr2=0x55be064b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f21df5640 1 --2- 172.21.15.19:0/2839608083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064b60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f21df5640 1 -- 172.21.15.19:0/2839608083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be06476390 con 0x55be064b7370 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f21df5640 1 --2- 172.21.15.19:0/2839608083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b6630 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3f18054f80 tx=0x7f3f1801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715048f8d68e3f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f067fc640 1 -- 172.21.15.19:0/2839608083 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f18061180 con 0x55be064b7370 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f067fc640 1 -- 172.21.15.19:0/2839608083 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f18083050 con 0x55be064b7370 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.156+0000 7f3f067fc640 1 -- 172.21.15.19:0/2839608083 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f18055190 con 0x55be064b7370 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.157+0000 7f3f26632740 1 -- 172.21.15.19:0/2839608083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 msgr2=0x55be064b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.157+0000 7f3f26632740 1 --2- 172.21.15.19:0/2839608083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b6630 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3f18054f80 tx=0x7f3f1801d040 comp rx=0 tx=0).stop 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.157+0000 7f3f26632740 1 -- 172.21.15.19:0/2839608083 shutdown_connections 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.157+0000 7f3f26632740 1 --2- 172.21.15.19:0/2839608083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b6630 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.157+0000 7f3f26632740 1 --2- 172.21.15.19:0/2839608083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.157+0000 7f3f26632740 1 --2- 172.21.15.19:0/2839608083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be0647aca0 0x55be064c0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.157+0000 7f3f26632740 1 -- 172.21.15.19:0/2839608083 >> 172.21.15.19:0/2839608083 conn(0x55be06442870 msgr2=0x55be06474ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.157+0000 7f3f26632740 1 -- 172.21.15.19:0/2839608083 shutdown_connections 2024-04-25T10:00:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.157+0000 7f3f26632740 1 -- 172.21.15.19:0/2839608083 wait complete. 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.157+0000 7f3f26632740 1 -- start start 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.158+0000 7f3f26632740 1 Processor -- start 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.158+0000 7f3f26632740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be0647aca0 0x55be064ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.158+0000 7f3f26632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064b89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.158+0000 7f3f26632740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.158+0000 7f3f26632740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be06471030 con 0x55be064b7370 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.158+0000 7f3f26632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0c9280 con 0x55be064a5760 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.158+0000 7f3f26632740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be064c1630 con 0x55be0647aca0 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.158+0000 7f3f26632740 10 client.0 _ll_register_callbacks cb 0x7f3f14024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f21df5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f21df5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41358/0 (socket says 172.21.15.19:41358) 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f21df5640 1 -- 172.21.15.19:0/4191082797 learned_addr learned my addr 172.21.15.19:0/4191082797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f215f4640 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be0647aca0 0x55be064ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f20df3640 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064b89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f21df5640 1 -- 172.21.15.19:0/4191082797 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be0647aca0 msgr2=0x55be064ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f21df5640 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be0647aca0 0x55be064ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f21df5640 1 -- 172.21.15.19:0/4191082797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 msgr2=0x55be064b89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f21df5640 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064b89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f21df5640 1 -- 172.21.15.19:0/4191082797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be0608b1b0 con 0x55be064b7370 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f21df5640 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b8ee0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f3f1808a010 tx=0x7f3f180886f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f593d0dd209f44eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3efbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3efbfff640 1 -- 172.21.15.19:0/4191082797 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f18056ee0 con 0x55be064b7370 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3efbfff640 1 -- 172.21.15.19:0/4191082797 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3f1808a0b0 con 0x55be064b7370 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f26632740 1 -- 172.21.15.19:0/4191082797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f18083050 con 0x55be064b7370 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3efbfff640 1 -- 172.21.15.19:0/4191082797 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f18060cf0 con 0x55be064b7370 2024-04-25T10:00:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f26632740 10 client.8880 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3f26632740 1 -- 172.21.15.19:0/4191082797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55be064a2fe0 con 0x55be064b7370 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.159+0000 7f3efbfff640 1 -- 172.21.15.19:0/4191082797 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f18083050 con 0x55be064b7370 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3efbfff640 1 -- 172.21.15.19:0/4191082797 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3f18077380 con 0x55be064b7370 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 10 client.8880 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 10 client.8880 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 20 client.8880 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55be064b60f0 0x55be064b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 4 client.8880 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 1 -- 172.21.15.19:0/4191082797 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55be063e5420 con 0x55be064b60f0 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f215f4640 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55be064b60f0 0x55be064b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3efbfff640 10 client.8880 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f215f4640 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55be064b60f0 0x55be064b6500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7840c2116608d127 server_cookie=da9f70be48cb834f in_seq=0 out_seq=0 2024-04-25T10:00:57.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3efbfff640 1 -- 172.21.15.19:0/4191082797 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55be063e5420 con 0x55be064b60f0 2024-04-25T10:00:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3efbfff640 10 client.8880 handle_command_reply: tid=0 2024-04-25T10:00:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 1 client.8880 shutdown 2024-04-25T10:00:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 1 -- 172.21.15.19:0/4191082797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 msgr2=0x55be064b8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b8ee0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f3f1808a010 tx=0x7f3f180886f0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 1 -- 172.21.15.19:0/4191082797 shutdown_connections 2024-04-25T10:00:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55be064b60f0 0x55be064b6500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55be064b7370 0x55be064b8ee0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be064a5760 0x55be064b89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 1 --2- 172.21.15.19:0/4191082797 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55be0647aca0 0x55be064ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3efbfff640 0 client.8880 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.160+0000 7f3f26632740 1 -- 172.21.15.19:0/4191082797 >> 172.21.15.19:0/4191082797 conn(0x55be06442870 msgr2=0x55be06474ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f26632740 1 -- 172.21.15.19:0/4191082797 shutdown_connections 2024-04-25T10:00:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f26632740 1 -- 172.21.15.19:0/4191082797 wait complete. 2024-04-25T10:00:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f26632740 20 client.8880 trim_cache size 0 max 16384 2024-04-25T10:00:57.178 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:57.179 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T10:00:57.179 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 -- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f0006b4d0 msgr2=0x7f3f0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f0006b4d0 0x7f3f0006d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000f60 tx=0x7f3f14007d50 comp rx=0 tx=0).stop 2024-04-25T10:00:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 -- 172.21.15.19:0/3815000649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f1c0b8050 msgr2=0x7f3f1c0b84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f1c0b8050 0x7f3f1c0b84d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800c7b0 tx=0x7f3f1800cc70 comp rx=0 tx=0).stop 2024-04-25T10:00:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 -- 172.21.15.19:0/3815000649 shutdown_connections 2024-04-25T10:00:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f0006b4d0 0x7f3f0006d990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f1c0b8050 0x7f3f1c0b84d0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f1c15da90 0x7f3f1c14ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 --2- 172.21.15.19:0/3815000649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f1c0cebb0 0x7f3f1c15f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 -- 172.21.15.19:0/3815000649 >> 172.21.15.19:0/3815000649 conn(0x7f3f1c061a20 msgr2=0x7f3f1c0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 -- 172.21.15.19:0/3815000649 shutdown_connections 2024-04-25T10:00:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.161+0000 7f3f23858640 1 -- 172.21.15.19:0/3815000649 wait complete. 2024-04-25T10:00:57.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:57.182 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:00:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.280+0000 7f71cf04a640 1 Processor -- start 2024-04-25T10:00:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.280+0000 7f71cf04a640 1 -- start start 2024-04-25T10:00:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71cf04a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c815dc20 0x7f71c80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71cf04a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c806cda0 con 0x7f71c80d32f0 2024-04-25T10:00:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71cf04a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c815e000 con 0x7f71c8161710 2024-04-25T10:00:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71cf04a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80d1de0 con 0x7f71c815dc20 2024-04-25T10:00:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71cd5e7640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f71c80d32f0 0x7f71c806c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51164/0 (socket says 172.21.15.19:51164) 2024-04-25T10:00:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71cd5e7640 1 -- 172.21.15.19:0/642588849 learned_addr learned my addr 172.21.15.19:0/642588849 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71ccde6640 1 --2- 172.21.15.19:0/642588849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c815dc20 0x7f71c80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71c77fe640 1 -- 172.21.15.19:0/642588849 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295549973 0 0) 0x7f71c806cda0 con 0x7f71c80d32f0 2024-04-25T10:00:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71c77fe640 1 -- 172.21.15.19:0/642588849 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71b8003520 con 0x7f71c80d32f0 2024-04-25T10:00:57.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71c77fe640 1 -- 172.21.15.19:0/642588849 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497975121 0 0) 0x7f71c815e000 con 0x7f71c8161710 2024-04-25T10:00:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71c77fe640 1 -- 172.21.15.19:0/642588849 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c806cda0 con 0x7f71c8161710 2024-04-25T10:00:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71ccde6640 1 -- 172.21.15.19:0/642588849 >> v1:172.21.15.112:6789/0 conn(0x7f71c8161710 legacy=0x7f71c806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71ccde6640 1 -- 172.21.15.19:0/642588849 >> v1:172.21.15.19:6789/0 conn(0x7f71c80d32f0 legacy=0x7f71c806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.281+0000 7f71ccde6640 1 -- 172.21.15.19:0/642588849 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c8161c60 con 0x7f71c815dc20 2024-04-25T10:00:57.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.282+0000 7f71ccde6640 1 --2- 172.21.15.19:0/642588849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c815dc20 0x7f71c80ce670 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0097d0 tx=0x7f71bc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d5459748430bba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.282+0000 7f71c77fe640 1 -- 172.21.15.19:0/642588849 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc037070 con 0x7f71c815dc20 2024-04-25T10:00:57.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.282+0000 7f71c77fe640 1 -- 172.21.15.19:0/642588849 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc032440 con 0x7f71c815dc20 2024-04-25T10:00:57.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.282+0000 7f71c77fe640 1 -- 172.21.15.19:0/642588849 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc032810 con 0x7f71c815dc20 2024-04-25T10:00:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.283+0000 7f71cf04a640 1 -- 172.21.15.19:0/642588849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c815dc20 msgr2=0x7f71c80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.283+0000 7f71cf04a640 1 --2- 172.21.15.19:0/642588849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c815dc20 0x7f71c80ce670 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0097d0 tx=0x7f71bc02a210 comp rx=0 tx=0).stop 2024-04-25T10:00:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.283+0000 7f71cf04a640 1 -- 172.21.15.19:0/642588849 shutdown_connections 2024-04-25T10:00:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.283+0000 7f71cf04a640 1 --2- 172.21.15.19:0/642588849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c815dc20 0x7f71c80ce670 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.283+0000 7f71cf04a640 1 -- 172.21.15.19:0/642588849 >> 172.21.15.19:0/642588849 conn(0x7f71c80d0420 msgr2=0x7f71c80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.283+0000 7f71cf04a640 1 -- 172.21.15.19:0/642588849 shutdown_connections 2024-04-25T10:00:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.283+0000 7f71cf04a640 1 -- 172.21.15.19:0/642588849 wait complete. 2024-04-25T10:00:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.284+0000 7f71cf04a640 1 Processor -- start 2024-04-25T10:00:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.284+0000 7f71cf04a640 1 -- start start 2024-04-25T10:00:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.285+0000 7f71cf04a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c80d32f0 0x7f71c81495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.285+0000 7f71cf04a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c815dc20 0x7f71c814c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.285+0000 7f71cf04a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c8161710 0x7f71c80b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.285+0000 7f71cf04a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8162500 con 0x7f71c815dc20 2024-04-25T10:00:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.285+0000 7f71cf04a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8162220 con 0x7f71c8161710 2024-04-25T10:00:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.285+0000 7f71cf04a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80b5e50 con 0x7f71c80d32f0 2024-04-25T10:00:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.285+0000 7f71c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c815dc20 0x7f71c814c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.285+0000 7f71c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c815dc20 0x7f71c814c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41372/0 (socket says 172.21.15.19:41372) 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.285+0000 7f71c7fff640 1 -- 172.21.15.19:0/168986399 learned_addr learned my addr 172.21.15.19:0/168986399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.285+0000 7f71ccde6640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c80d32f0 0x7f71c81495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.285+0000 7f71cd5e7640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c8161710 0x7f71c80b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.286+0000 7f71c7fff640 1 -- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c80d32f0 msgr2=0x7f71c81495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.286+0000 7f71c7fff640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c80d32f0 0x7f71c81495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.286+0000 7f71c7fff640 1 -- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c8161710 msgr2=0x7f71c80b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.286+0000 7f71c7fff640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c8161710 0x7f71c80b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.286+0000 7f71c7fff640 1 -- 172.21.15.19:0/168986399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c8149b90 con 0x7f71c815dc20 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.286+0000 7f71c7fff640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c815dc20 0x7f71c814c050 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c530 tx=0x7f71b400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.286+0000 7f71c57fa640 1 -- 172.21.15.19:0/168986399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b40052d0 con 0x7f71c815dc20 2024-04-25T10:00:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.286+0000 7f71c57fa640 1 -- 172.21.15.19:0/168986399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4005490 con 0x7f71c815dc20 2024-04-25T10:00:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.286+0000 7f71cf04a640 1 -- 172.21.15.19:0/168986399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c8149e70 con 0x7f71c815dc20 2024-04-25T10:00:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.286+0000 7f71c57fa640 1 -- 172.21.15.19:0/168986399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4012870 con 0x7f71c815dc20 2024-04-25T10:00:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.286+0000 7f71cf04a640 1 -- 172.21.15.19:0/168986399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c814a430 con 0x7f71c815dc20 2024-04-25T10:00:57.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.288+0000 7f71c57fa640 1 -- 172.21.15.19:0/168986399 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71b4012a10 con 0x7f71c815dc20 2024-04-25T10:00:57.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.288+0000 7f71c57fa640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71ac06b4d0 0x7f71ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.288+0000 7f71c57fa640 1 -- 172.21.15.19:0/168986399 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71b4089f80 con 0x7f71c815dc20 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.288+0000 7f71ccde6640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71ac06b4d0 0x7f71ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.289+0000 7f71ccde6640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71ac06b4d0 0x7f71ac06d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f71bc034000 tx=0x7f71bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.289+0000 7f71d1e25740 1 Processor -- start 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.289+0000 7f71d1e25740 1 -- start start 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x562469438320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469435ef0 0x562469438860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469408770 0x56246940bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71d1e25740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562469371da0 con 0x562469435b10 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71d1e25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246931ab80 con 0x562469408770 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71d1e25740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56246936de40 con 0x562469435ef0 2024-04-25T10:00:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71ccde6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x562469438320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71ccde6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x562469438320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41384/0 (socket says 172.21.15.19:41384) 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71ccde6640 1 -- 172.21.15.19:0/3915934330 learned_addr learned my addr 172.21.15.19:0/3915934330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71c7fff640 1 --2- 172.21.15.19:0/3915934330 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469435ef0 0x562469438860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71cd5e7640 1 --2- 172.21.15.19:0/3915934330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469408770 0x56246940bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71ccde6640 1 -- 172.21.15.19:0/3915934330 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469435ef0 msgr2=0x562469438860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71ccde6640 1 --2- 172.21.15.19:0/3915934330 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469435ef0 0x562469438860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71ccde6640 1 -- 172.21.15.19:0/3915934330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469408770 msgr2=0x56246940bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71ccde6640 1 --2- 172.21.15.19:0/3915934330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469408770 0x56246940bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71ccde6640 1 -- 172.21.15.19:0/3915934330 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562469324f10 con 0x562469435b10 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.290+0000 7f71ccde6640 1 --2- 172.21.15.19:0/3915934330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x562469438320 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0098f0 tx=0x7f71bc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2123b7745b3408d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71b27fc640 1 -- 172.21.15.19:0/3915934330 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc04c070 con 0x562469435b10 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71b27fc640 1 -- 172.21.15.19:0/3915934330 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc032be0 con 0x562469435b10 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71b27fc640 1 -- 172.21.15.19:0/3915934330 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc037040 con 0x562469435b10 2024-04-25T10:00:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71d1e25740 1 -- 172.21.15.19:0/3915934330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 msgr2=0x562469438320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71d1e25740 1 --2- 172.21.15.19:0/3915934330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x562469438320 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0098f0 tx=0x7f71bc02c000 comp rx=0 tx=0).stop 2024-04-25T10:00:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71d1e25740 1 -- 172.21.15.19:0/3915934330 shutdown_connections 2024-04-25T10:00:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71d1e25740 1 --2- 172.21.15.19:0/3915934330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469408770 0x56246940bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71d1e25740 1 --2- 172.21.15.19:0/3915934330 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469435ef0 0x562469438860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71d1e25740 1 --2- 172.21.15.19:0/3915934330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x562469438320 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71d1e25740 1 -- 172.21.15.19:0/3915934330 >> 172.21.15.19:0/3915934330 conn(0x56246940a9e0 msgr2=0x562469453320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71d1e25740 1 -- 172.21.15.19:0/3915934330 shutdown_connections 2024-04-25T10:00:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.291+0000 7f71d1e25740 1 -- 172.21.15.19:0/3915934330 wait complete. 2024-04-25T10:00:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.292+0000 7f71d1e25740 1 -- start start 2024-04-25T10:00:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.292+0000 7f71d1e25740 1 Processor -- start 2024-04-25T10:00:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x56246943c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469435ef0 0x562469457dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469458310 0x5624694349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71d1e25740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246940db90 con 0x562469435b10 2024-04-25T10:00:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71d1e25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624693104b0 con 0x562469458310 2024-04-25T10:00:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71d1e25740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56246940de70 con 0x562469435ef0 2024-04-25T10:00:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71d1e25740 10 client.0 _ll_register_callbacks cb 0x562469350180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71ccde6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x56246943c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71ccde6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x56246943c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41396/0 (socket says 172.21.15.19:41396) 2024-04-25T10:00:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71ccde6640 1 -- 172.21.15.19:0/1819832639 learned_addr learned my addr 172.21.15.19:0/1819832639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71c7fff640 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469435ef0 0x562469457dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71cd5e7640 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469458310 0x5624694349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71ccde6640 1 -- 172.21.15.19:0/1819832639 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469435ef0 msgr2=0x562469457dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71ccde6640 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469435ef0 0x562469457dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71ccde6640 1 -- 172.21.15.19:0/1819832639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469458310 msgr2=0x5624694349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71ccde6640 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469458310 0x5624694349a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71ccde6640 1 -- 172.21.15.19:0/1819832639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624691507d0 con 0x562469435b10 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71cd5e7640 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469458310 0x5624694349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71ccde6640 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x56246943c250 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0097d0 tx=0x7f71bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30b6ecffb052c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.293+0000 7f71a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71a77fe640 1 -- 172.21.15.19:0/1819832639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc037040 con 0x562469435b10 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71a77fe640 1 -- 172.21.15.19:0/1819832639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71bc051b50 con 0x562469435b10 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71d1e25740 1 -- 172.21.15.19:0/1819832639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71bc032be0 con 0x562469435b10 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71a77fe640 1 -- 172.21.15.19:0/1819832639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71bc062050 con 0x562469435b10 2024-04-25T10:00:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71d1e25740 10 client.8892 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71d1e25740 1 -- 172.21.15.19:0/1819832639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5624690863f0 con 0x562469435b10 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71a77fe640 1 -- 172.21.15.19:0/1819832639 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc032be0 con 0x562469435b10 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71a77fe640 1 -- 172.21.15.19:0/1819832639 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f71bc05a480 con 0x562469435b10 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71d1e25740 10 client.8892 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71d1e25740 10 client.8892 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71d1e25740 20 client.8892 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x562469456020 0x562469456400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71d1e25740 4 client.8892 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71d1e25740 1 -- 172.21.15.19:0/1819832639 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56246937f2e0 con 0x562469456020 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.294+0000 7f71c7fff640 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x562469456020 0x562469456400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71a77fe640 10 client.8892 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71c7fff640 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x562469456020 0x562469456400 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f57874c498961c3a server_cookie=1dbbb76fa7258f5 in_seq=0 out_seq=0 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71a77fe640 1 -- 172.21.15.19:0/1819832639 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56246937f2e0 con 0x562469456020 2024-04-25T10:00:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71a77fe640 10 client.8892 handle_command_reply: tid=0 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71d1e25740 1 client.8892 shutdown 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71d1e25740 1 -- 172.21.15.19:0/1819832639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 msgr2=0x56246943c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x56246943c250 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0097d0 tx=0x7f71bc004730 comp rx=0 tx=0).stop 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71d1e25740 1 -- 172.21.15.19:0/1819832639 shutdown_connections 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x562469456020 0x562469456400 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469458310 0x5624694349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469435ef0 0x562469457dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71a77fe640 0 client.8892 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1819832639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435b10 0x56246943c250 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71d1e25740 1 -- 172.21.15.19:0/1819832639 >> 172.21.15.19:0/1819832639 conn(0x562469408770 msgr2=0x562469409400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71d1e25740 1 -- 172.21.15.19:0/1819832639 shutdown_connections 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.295+0000 7f71d1e25740 1 -- 172.21.15.19:0/1819832639 wait complete. 2024-04-25T10:00:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.296+0000 7f71d1e25740 20 client.8892 trim_cache size 0 max 16384 2024-04-25T10:00:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.311+0000 7f71d1e25740 1 Processor -- start 2024-04-25T10:00:57.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.311+0000 7f71d1e25740 1 -- start start 2024-04-25T10:00:57.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.311+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 0x562469438480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.311+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x562469453320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.311+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469458310 0x562469453860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.311+0000 7f71d1e25740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c82003a0 con 0x562469435ef0 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.311+0000 7f71d1e25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c81ff540 con 0x562469435b10 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.311+0000 7f71d1e25740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80cc720 con 0x562469458310 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.311+0000 7f71c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x562469453320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.311+0000 7f71c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x562469453320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41408/0 (socket says 172.21.15.19:41408) 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.311+0000 7f71c7fff640 1 -- 172.21.15.19:0/2049038880 learned_addr learned my addr 172.21.15.19:0/2049038880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71cd5e7640 1 --2- 172.21.15.19:0/2049038880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469458310 0x562469453860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71ccde6640 1 --2- 172.21.15.19:0/2049038880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 0x562469438480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71c7fff640 1 -- 172.21.15.19:0/2049038880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469458310 msgr2=0x562469453860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71c7fff640 1 --2- 172.21.15.19:0/2049038880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469458310 0x562469453860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71c7fff640 1 -- 172.21.15.19:0/2049038880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 msgr2=0x562469438480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71c7fff640 1 --2- 172.21.15.19:0/2049038880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 0x562469438480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71c7fff640 1 -- 172.21.15.19:0/2049038880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624690460f0 con 0x562469435ef0 2024-04-25T10:00:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71cd5e7640 1 --2- 172.21.15.19:0/2049038880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469458310 0x562469453860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71ccde6640 1 --2- 172.21.15.19:0/2049038880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 0x562469438480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71c7fff640 1 --2- 172.21.15.19:0/2049038880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x562469453320 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c500 tx=0x7f71b4088260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240ac091eba6c6d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71b1ffb640 1 -- 172.21.15.19:0/2049038880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4056910 con 0x562469435ef0 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71b1ffb640 1 -- 172.21.15.19:0/2049038880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4070030 con 0x562469435ef0 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.312+0000 7f71b1ffb640 1 -- 172.21.15.19:0/2049038880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4012d00 con 0x562469435ef0 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.313+0000 7f71d1e25740 1 -- 172.21.15.19:0/2049038880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 msgr2=0x562469453320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.313+0000 7f71d1e25740 1 --2- 172.21.15.19:0/2049038880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x562469453320 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c500 tx=0x7f71b4088260 comp rx=0 tx=0).stop 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.313+0000 7f71d1e25740 1 -- 172.21.15.19:0/2049038880 shutdown_connections 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.313+0000 7f71d1e25740 1 --2- 172.21.15.19:0/2049038880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469458310 0x562469453860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.313+0000 7f71d1e25740 1 --2- 172.21.15.19:0/2049038880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x562469453320 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.313+0000 7f71d1e25740 1 --2- 172.21.15.19:0/2049038880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 0x562469438480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.313+0000 7f71d1e25740 1 -- 172.21.15.19:0/2049038880 >> 172.21.15.19:0/2049038880 conn(0x562469408770 msgr2=0x562469459a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.313+0000 7f71d1e25740 1 -- 172.21.15.19:0/2049038880 shutdown_connections 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.313+0000 7f71d1e25740 1 -- 172.21.15.19:0/2049038880 wait complete. 2024-04-25T10:00:57.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.313+0000 7f71d1e25740 1 -- start start 2024-04-25T10:00:57.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.313+0000 7f71d1e25740 1 Processor -- start 2024-04-25T10:00:57.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 0x5624694bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x5624694548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71d1e25740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469458310 0x562469454df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71d1e25740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246940f410 con 0x562469435ef0 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71d1e25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c81fef40 con 0x562469435b10 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71d1e25740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624694031a0 con 0x562469458310 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71d1e25740 10 client.0 _ll_register_callbacks cb 0x7f71bc040050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x5624694548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x5624694548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41416/0 (socket says 172.21.15.19:41416) 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71c7fff640 1 -- 172.21.15.19:0/1768050712 learned_addr learned my addr 172.21.15.19:0/1768050712 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71ccde6640 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 0x5624694bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71cd5e7640 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469458310 0x562469454df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.314+0000 7f71c7fff640 1 -- 172.21.15.19:0/1768050712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469458310 msgr2=0x562469454df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71c7fff640 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469458310 0x562469454df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71c7fff640 1 -- 172.21.15.19:0/1768050712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 msgr2=0x5624694bd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71c7fff640 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 0x5624694bd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71c7fff640 1 -- 172.21.15.19:0/1768050712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624694529b0 con 0x562469435ef0 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71ccde6640 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 0x5624694bd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71c7fff640 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x5624694548b0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f71b4088670 tx=0x7f71b4043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b67baca00ab1d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71a77fe640 1 -- 172.21.15.19:0/1768050712 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4054c80 con 0x562469435ef0 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71a77fe640 1 -- 172.21.15.19:0/1768050712 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71b40430b0 con 0x562469435ef0 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71d1e25740 1 -- 172.21.15.19:0/1768050712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71b4070030 con 0x562469435ef0 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71a77fe640 1 -- 172.21.15.19:0/1768050712 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71b4054e20 con 0x562469435ef0 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71d1e25740 10 client.8898 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71d1e25740 1 -- 172.21.15.19:0/1768050712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x562469461df0 con 0x562469435ef0 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71a77fe640 1 -- 172.21.15.19:0/1768050712 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4070030 con 0x562469435ef0 2024-04-25T10:00:57.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71a77fe640 1 -- 172.21.15.19:0/1768050712 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f71b4071070 con 0x562469435ef0 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71d1e25740 10 client.8898 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71d1e25740 10 client.8898 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71d1e25740 20 client.8898 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x562469448cb0 0x562469449090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71d1e25740 4 client.8898 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71d1e25740 1 -- 172.21.15.19:0/1768050712 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5624694595e0 con 0x562469448cb0 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.315+0000 7f71ccde6640 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x562469448cb0 0x562469449090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71ccde6640 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x562469448cb0 0x562469449090 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e5eb558fe427f9 server_cookie=29ce5744ac4df57b in_seq=0 out_seq=0 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71a77fe640 10 client.8898 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71a77fe640 1 -- 172.21.15.19:0/1768050712 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x7f71bc05f050 con 0x562469448cb0 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71a77fe640 10 client.8898 handle_command_reply: tid=0 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 1 client.8898 shutdown 2024-04-25T10:00:57.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 1 -- 172.21.15.19:0/1768050712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 msgr2=0x5624694548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x5624694548b0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f71b4088670 tx=0x7f71b4043040 comp rx=0 tx=0).stop 2024-04-25T10:00:57.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 1 -- 172.21.15.19:0/1768050712 shutdown_connections 2024-04-25T10:00:57.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x562469448cb0 0x562469449090 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562469458310 0x562469454df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562469435ef0 0x5624694548b0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71a77fe640 0 client.8898 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:57.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 1 --2- 172.21.15.19:0/1768050712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562469435b10 0x5624694bd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 1 -- 172.21.15.19:0/1768050712 >> 172.21.15.19:0/1768050712 conn(0x562469408770 msgr2=0x562469408c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 1 -- 172.21.15.19:0/1768050712 shutdown_connections 2024-04-25T10:00:57.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 1 -- 172.21.15.19:0/1768050712 wait complete. 2024-04-25T10:00:57.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.316+0000 7f71d1e25740 20 client.8898 trim_cache size 0 max 16384 2024-04-25T10:00:57.329 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:57.329 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:00:57.329 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:00:57.329 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 -- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71ac06b4d0 msgr2=0x7f71ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71ac06b4d0 0x7f71ac06d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f71bc034000 tx=0x7f71bc02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 -- 172.21.15.19:0/168986399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c815dc20 msgr2=0x7f71c814c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c815dc20 0x7f71c814c050 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c530 tx=0x7f71b400c9f0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 -- 172.21.15.19:0/168986399 shutdown_connections 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71ac06b4d0 0x7f71ac06d990 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c8161710 0x7f71c80b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c815dc20 0x7f71c814c050 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 --2- 172.21.15.19:0/168986399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c80d32f0 0x7f71c81495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 -- 172.21.15.19:0/168986399 >> 172.21.15.19:0/168986399 conn(0x7f71c80d0420 msgr2=0x7f71c80cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 -- 172.21.15.19:0/168986399 shutdown_connections 2024-04-25T10:00:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.317+0000 7f71cf04a640 1 -- 172.21.15.19:0/168986399 wait complete. 2024-04-25T10:00:57.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:57.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.428+0000 7fd7813e2640 1 Processor -- start 2024-04-25T10:00:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.428+0000 7fd7813e2640 1 -- start start 2024-04-25T10:00:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.429+0000 7fd7813e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c154ee0 0x7fd77c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.429+0000 7fd7813e2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd77c0ce590 0x7fd77c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.429+0000 7fd7813e2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd77c071480 con 0x7fd77c0cfcf0 2024-04-25T10:00:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.429+0000 7fd7813e2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c072090 con 0x7fd77c0ce590 2024-04-25T10:00:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.429+0000 7fd7813e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0d0150 con 0x7fd77c154ee0 2024-04-25T10:00:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.429+0000 7fd77a7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd77c0cfcf0 0x7fd77c0cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51178/0 (socket says 172.21.15.19:51178) 2024-04-25T10:00:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.429+0000 7fd77a7fc640 1 -- 172.21.15.19:0/2015591168 learned_addr learned my addr 172.21.15.19:0/2015591168 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.429+0000 7fd77b7fe640 1 --2- 172.21.15.19:0/2015591168 >> v2:172.21.15.112:3300/0 conn(0x7fd77c0ce590 0x7fd77c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.429+0000 7fd77affd640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c154ee0 0x7fd77c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.429+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336210449 0 0) 0x7fd77c071480 con 0x7fd77c0cfcf0 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.429+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd768003520 con 0x7fd77c0cfcf0 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1167914257 0 0) 0x7fd768003520 con 0x7fd77c0cfcf0 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c154ee0 msgr2=0x7fd77c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c154ee0 0x7fd77c0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 >> v2:172.21.15.112:3300/0 conn(0x7fd77c0ce590 msgr2=0x7fd77c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 --2- 172.21.15.19:0/2015591168 >> v2:172.21.15.112:3300/0 conn(0x7fd77c0ce590 0x7fd77c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c071760 con 0x7fd77c0cfcf0 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fd764001670 con 0x7fd77c0cfcf0 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 >> v1:172.21.15.19:6789/0 conn(0x7fd77c0cfcf0 legacy=0x7fd77c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c154ee0 0x7fd7680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd768003f30 0x7fd768004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768004d40 0x7fd7680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76800b9c0 con 0x7fd768003f30 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76800bb40 con 0x7fd768004d40 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76800bcc0 con 0x7fd77c154ee0 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd77a7fc640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd768003f30 0x7fd768004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd77affd640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c154ee0 0x7fd7680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd77b7fe640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768004d40 0x7fd7680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd77a7fc640 1 -- 172.21.15.19:0/2015591168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c154ee0 msgr2=0x7fd7680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd77a7fc640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c154ee0 0x7fd7680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd77a7fc640 1 -- 172.21.15.19:0/2015591168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768004d40 msgr2=0x7fd7680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd77a7fc640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768004d40 0x7fd7680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.430+0000 7fd77a7fc640 1 -- 172.21.15.19:0/2015591168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd768003520 con 0x7fd768003f30 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.431+0000 7fd77a7fc640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd768003f30 0x7fd768004800 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd764007fd0 tx=0x7fd768005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff17287a966c7c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.431+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764007d50 con 0x7fd768003f30 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.431+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd764005ad0 con 0x7fd768003f30 2024-04-25T10:00:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.431+0000 7fd779ffb640 1 -- 172.21.15.19:0/2015591168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764007070 con 0x7fd768003f30 2024-04-25T10:00:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.431+0000 7fd7813e2640 1 -- 172.21.15.19:0/2015591168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd768003f30 msgr2=0x7fd768004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.431+0000 7fd7813e2640 1 --2- 172.21.15.19:0/2015591168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd768003f30 0x7fd768004800 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd764007fd0 tx=0x7fd768005830 comp rx=0 tx=0).stop 2024-04-25T10:00:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.431+0000 7fd77b7fe640 1 -- 172.21.15.19:0/2015591168 reap_dead start 2024-04-25T10:00:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.432+0000 7fd7813e2640 1 -- 172.21.15.19:0/2015591168 shutdown_connections 2024-04-25T10:00:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.432+0000 7fd7813e2640 1 -- 172.21.15.19:0/2015591168 >> 172.21.15.19:0/2015591168 conn(0x7fd77c06abe0 msgr2=0x7fd77c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.432+0000 7fd7813e2640 1 -- 172.21.15.19:0/2015591168 shutdown_connections 2024-04-25T10:00:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.432+0000 7fd7813e2640 1 -- 172.21.15.19:0/2015591168 wait complete. 2024-04-25T10:00:57.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.433+0000 7fd7813e2640 1 Processor -- start 2024-04-25T10:00:57.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.433+0000 7fd7813e2640 1 -- start start 2024-04-25T10:00:57.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd7813e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768003f30 0x7fd77c14a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd7813e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c0ae950 0x7fd77c14c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd7813e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c15b520 0x7fd77c150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd7813e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c071d20 con 0x7fd77c0ae950 2024-04-25T10:00:57.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd7813e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c071230 con 0x7fd768003f30 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd7813e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c1552c0 con 0x7fd77c15b520 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd77a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c0ae950 0x7fd77c14c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd77a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c0ae950 0x7fd77c14c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41426/0 (socket says 172.21.15.19:41426) 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd77a7fc640 1 -- 172.21.15.19:0/1333299779 learned_addr learned my addr 172.21.15.19:0/1333299779 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd77b7fe640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c15b520 0x7fd77c150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd77affd640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768003f30 0x7fd77c14a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd77a7fc640 1 -- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c15b520 msgr2=0x7fd77c150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd77a7fc640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c15b520 0x7fd77c150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd77a7fc640 1 -- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768003f30 msgr2=0x7fd77c14a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd77a7fc640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768003f30 0x7fd77c14a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd77a7fc640 1 -- 172.21.15.19:0/1333299779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c14a710 con 0x7fd77c0ae950 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd77a7fc640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c0ae950 0x7fd77c14c9b0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fd76402ae20 tx=0x7fd77c071930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd7737fe640 1 -- 172.21.15.19:0/1333299779 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764004bf0 con 0x7fd77c0ae950 2024-04-25T10:00:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd7737fe640 1 -- 172.21.15.19:0/1333299779 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd764043070 con 0x7fd77c0ae950 2024-04-25T10:00:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd7813e2640 1 -- 172.21.15.19:0/1333299779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c15ad60 con 0x7fd77c0ae950 2024-04-25T10:00:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd7737fe640 1 -- 172.21.15.19:0/1333299779 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764007070 con 0x7fd77c0ae950 2024-04-25T10:00:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.434+0000 7fd7813e2640 1 -- 172.21.15.19:0/1333299779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c14d600 con 0x7fd77c0ae950 2024-04-25T10:00:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.436+0000 7fd7813e2640 1 -- 172.21.15.19:0/1333299779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77c073990 con 0x7fd77c0ae950 2024-04-25T10:00:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.436+0000 7fd7737fe640 1 -- 172.21.15.19:0/1333299779 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd764007210 con 0x7fd77c0ae950 2024-04-25T10:00:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.436+0000 7fd7737fe640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd75c06b500 0x7fd75c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.437+0000 7fd7737fe640 1 -- 172.21.15.19:0/1333299779 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7640aaa60 con 0x7fd77c0ae950 2024-04-25T10:00:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.437+0000 7fd77affd640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd75c06b500 0x7fd75c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.437+0000 7fd77affd640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd75c06b500 0x7fd75c06d9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd774009a20 tx=0x7fd774008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.441+0000 7fd7737fe640 1 -- 172.21.15.19:0/1333299779 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd77c14a710 con 0x7fd77c0ae950 2024-04-25T10:00:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.658+0000 7fd7813e2640 1 -- 172.21.15.19:0/1333299779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd77c15a870 con 0x7fd77c0ae950 2024-04-25T10:00:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.659+0000 7fd7737fe640 1 -- 172.21.15.19:0/1333299779 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd764079870 con 0x7fd77c0ae950 2024-04-25T10:00:57.664 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:57.664 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:00:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:00:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 -- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd75c06b500 msgr2=0x7fd75c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd75c06b500 0x7fd75c06d9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd774009a20 tx=0x7fd774008040 comp rx=0 tx=0).stop 2024-04-25T10:00:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 -- 172.21.15.19:0/1333299779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c0ae950 msgr2=0x7fd77c14c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c0ae950 0x7fd77c14c9b0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fd76402ae20 tx=0x7fd77c071930 comp rx=0 tx=0).stop 2024-04-25T10:00:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 -- 172.21.15.19:0/1333299779 shutdown_connections 2024-04-25T10:00:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd75c06b500 0x7fd75c06d9c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd77c15b520 0x7fd77c150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c0ae950 0x7fd77c14c9b0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 --2- 172.21.15.19:0/1333299779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd768003f30 0x7fd77c14a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 -- 172.21.15.19:0/1333299779 >> 172.21.15.19:0/1333299779 conn(0x7fd77c06abe0 msgr2=0x7fd77c0b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 -- 172.21.15.19:0/1333299779 shutdown_connections 2024-04-25T10:00:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.661+0000 7fd7813e2640 1 -- 172.21.15.19:0/1333299779 wait complete. 2024-04-25T10:00:57.677 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:00:57.678 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.772+0000 7f5e342c6640 1 Processor -- start 2024-04-25T10:00:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.772+0000 7f5e342c6640 1 -- start start 2024-04-25T10:00:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.773+0000 7f5e342c6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15dd30 0x7f5e2c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.773+0000 7f5e342c6640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c0d1df0 con 0x7f5e2c0d0ad0 2024-04-25T10:00:57.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.773+0000 7f5e342c6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c0d20d0 con 0x7f5e2c0b9770 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.773+0000 7f5e342c6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0d23b0 con 0x7f5e2c15dd30 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.773+0000 7f5e32863640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5e2c0d0ad0 0x7f5e2c0b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44082/0 (socket says 172.21.15.19:44082) 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.773+0000 7f5e32863640 1 -- 172.21.15.19:0/226487504 learned_addr learned my addr 172.21.15.19:0/226487504 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e32062640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15dd30 0x7f5e2c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509663321 0 0) 0x7f5e2c0d1df0 con 0x7f5e2c0d0ad0 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e14003520 con 0x7f5e2c0d0ad0 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3810051323 0 0) 0x7f5e14003520 con 0x7f5e2c0d0ad0 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15dd30 msgr2=0x7f5e2c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15dd30 0x7f5e2c0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 >> v1:172.21.15.112:6789/0 conn(0x7f5e2c0b9770 legacy=0x7f5e2c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0d2690 con 0x7f5e2c0d0ad0 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f5e1c0019f0 con 0x7f5e2c0d0ad0 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 >> v1:172.21.15.19:6789/0 conn(0x7f5e2c0d0ad0 legacy=0x7f5e2c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 0x7f5e14003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15dd30 0x7f5e14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e14004d40 0x7f5e140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0d23b0 con 0x7f5e140039e0 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1400ba70 con 0x7f5e14004d40 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1400bc70 con 0x7f5e2c15dd30 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e32062640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 0x7f5e14003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e31861640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15dd30 0x7f5e14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e32863640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e14004d40 0x7f5e140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e32062640 1 -- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15dd30 msgr2=0x7f5e14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e32062640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15dd30 0x7f5e14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e32062640 1 -- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e14004d40 msgr2=0x7f5e140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e32062640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e14004d40 0x7f5e140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e32062640 1 -- 172.21.15.19:0/226487504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e14003520 con 0x7f5e140039e0 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.774+0000 7f5e32863640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e14004d40 0x7f5e140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.775+0000 7f5e32062640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 0x7f5e14003dc0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009ac0 tx=0x7f5e14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396916685ed8b1e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.775+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24036070 con 0x7f5e140039e0 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.775+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24031400 con 0x7f5e140039e0 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.775+0000 7f5e31060640 1 -- 172.21.15.19:0/226487504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24009690 con 0x7f5e140039e0 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.776+0000 7f5e342c6640 1 -- 172.21.15.19:0/226487504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 msgr2=0x7f5e14003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.776+0000 7f5e342c6640 1 --2- 172.21.15.19:0/226487504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 0x7f5e14003dc0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009ac0 tx=0x7f5e14005830 comp rx=0 tx=0).stop 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.776+0000 7f5e32863640 1 -- 172.21.15.19:0/226487504 reap_dead start 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.776+0000 7f5e342c6640 1 -- 172.21.15.19:0/226487504 shutdown_connections 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.776+0000 7f5e342c6640 1 -- 172.21.15.19:0/226487504 >> 172.21.15.19:0/226487504 conn(0x7f5e2c061c30 msgr2=0x7f5e2c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.776+0000 7f5e342c6640 1 -- 172.21.15.19:0/226487504 shutdown_connections 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.776+0000 7f5e342c6640 1 -- 172.21.15.19:0/226487504 wait complete. 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.777+0000 7f5e342c6640 1 Processor -- start 2024-04-25T10:00:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.777+0000 7f5e342c6640 1 -- start start 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e342c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 0x7f5e2c0b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e342c6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15bf60 0x7f5e2c0aad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e342c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c0ac640 0x7f5e2c15f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e342c6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0d1b90 con 0x7f5e140039e0 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e342c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0d1a10 con 0x7f5e2c0ac640 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e342c6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0d2e90 con 0x7f5e2c15bf60 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e32062640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 0x7f5e2c0b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e32062640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 0x7f5e2c0b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50442/0 (socket says 172.21.15.19:50442) 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e32062640 1 -- 172.21.15.19:0/3602325560 learned_addr learned my addr 172.21.15.19:0/3602325560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e32863640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c0ac640 0x7f5e2c15f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e31861640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15bf60 0x7f5e2c0aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e32062640 1 -- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15bf60 msgr2=0x7f5e2c0aad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e32062640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15bf60 0x7f5e2c0aad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e32062640 1 -- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c0ac640 msgr2=0x7f5e2c15f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e32062640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c0ac640 0x7f5e2c15f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e32062640 1 -- 172.21.15.19:0/3602325560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0ab5a0 con 0x7f5e140039e0 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.778+0000 7f5e32062640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 0x7f5e2c0b5b40 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f5e24034040 tx=0x7f5e24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.779+0000 7f5e2affd640 1 -- 172.21.15.19:0/3602325560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24040020 con 0x7f5e140039e0 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.779+0000 7f5e2affd640 1 -- 172.21.15.19:0/3602325560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24003b40 con 0x7f5e140039e0 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.779+0000 7f5e342c6640 1 -- 172.21.15.19:0/3602325560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c0ab880 con 0x7f5e140039e0 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.779+0000 7f5e2affd640 1 -- 172.21.15.19:0/3602325560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24036040 con 0x7f5e140039e0 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.779+0000 7f5e342c6640 1 -- 172.21.15.19:0/3602325560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c0abe40 con 0x7f5e140039e0 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.780+0000 7f5e342c6640 1 -- 172.21.15.19:0/3602325560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c073990 con 0x7f5e140039e0 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.780+0000 7f5e2affd640 1 -- 172.21.15.19:0/3602325560 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e24043020 con 0x7f5e140039e0 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.781+0000 7f5e2affd640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e1806b500 0x7f5e1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.781+0000 7f5e2affd640 1 -- 172.21.15.19:0/3602325560 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e240aa220 con 0x7f5e140039e0 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.781+0000 7f5e31861640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e1806b500 0x7f5e1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.781+0000 7f5e31861640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e1806b500 0x7f5e1806d9c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0ca3c0 tx=0x7f5e20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:57.785+0000 7f5e2affd640 1 -- 172.21.15.19:0/3602325560 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e2404a070 con 0x7f5e140039e0 2024-04-25T10:00:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.032+0000 7f5e342c6640 1 -- 172.21.15.19:0/3602325560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5e2c0ab2a0 con 0x7f5e140039e0 2024-04-25T10:00:58.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.033+0000 7f5e2affd640 1 -- 172.21.15.19:0/3602325560 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f5e240044e0 con 0x7f5e140039e0 2024-04-25T10:00:58.038 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:58.038 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":495,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2562392064,"bytes_avail":770531721216,"bytes_total":773094113280,"write_bytes_sec":43359460,"read_op_per_sec":0,"write_op_per_sec":42},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:00:55.018290+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:58.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 -- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e1806b500 msgr2=0x7f5e1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e1806b500 0x7f5e1806d9c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0ca3c0 tx=0x7f5e20009040 comp rx=0 tx=0).stop 2024-04-25T10:00:58.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 -- 172.21.15.19:0/3602325560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 msgr2=0x7f5e2c0b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 0x7f5e2c0b5b40 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f5e24034040 tx=0x7f5e24004730 comp rx=0 tx=0).stop 2024-04-25T10:00:58.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 -- 172.21.15.19:0/3602325560 shutdown_connections 2024-04-25T10:00:58.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e1806b500 0x7f5e1806d9c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e2c0ac640 0x7f5e2c15f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e2c15bf60 0x7f5e2c0aad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 --2- 172.21.15.19:0/3602325560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039e0 0x7f5e2c0b5b40 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 -- 172.21.15.19:0/3602325560 >> 172.21.15.19:0/3602325560 conn(0x7f5e2c061c30 msgr2=0x7f5e2c1606e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 -- 172.21.15.19:0/3602325560 shutdown_connections 2024-04-25T10:00:58.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.035+0000 7f5e342c6640 1 -- 172.21.15.19:0/3602325560 wait complete. 2024-04-25T10:00:58.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T10:00:58.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:58.052 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T10:00:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.149+0000 7f8c4ebd5640 1 Processor -- start 2024-04-25T10:00:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.149+0000 7f8c4ebd5640 1 -- start start 2024-04-25T10:00:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.150+0000 7f8c4ebd5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8c4806c9e0 0x7f8c480b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.150+0000 7f8c4ebd5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4806cf50 con 0x7f8c4815b7d0 2024-04-25T10:00:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.150+0000 7f8c4ebd5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c480ce530 con 0x7f8c480b5620 2024-04-25T10:00:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.150+0000 7f8c4ebd5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c480ce810 con 0x7f8c4806c9e0 2024-04-25T10:00:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.150+0000 7f8c4d172640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8c4806c9e0 0x7f8c480b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.150+0000 7f8c4c971640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8c4815b7d0 0x7f8c480cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44092/0 (socket says 172.21.15.19:44092) 2024-04-25T10:00:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.150+0000 7f8c4c971640 1 -- 172.21.15.19:0/2124600488 learned_addr learned my addr 172.21.15.19:0/2124600488 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.150+0000 7f8c4d172640 1 -- 172.21.15.19:0/2124600488 >> v1:172.21.15.112:6789/0 conn(0x7f8c480b5620 legacy=0x7f8c4806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.150+0000 7f8c4d172640 1 -- 172.21.15.19:0/2124600488 >> v1:172.21.15.19:6789/0 conn(0x7f8c4815b7d0 legacy=0x7f8c480cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.150+0000 7f8c477fe640 1 -- 172.21.15.19:0/2124600488 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804383502 0 0) 0x7f8c4806cf50 con 0x7f8c4815b7d0 2024-04-25T10:00:58.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.151+0000 7f8c4d172640 1 -- 172.21.15.19:0/2124600488 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c480ceaf0 con 0x7f8c4806c9e0 2024-04-25T10:00:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.151+0000 7f8c4d172640 1 --2- 172.21.15.19:0/2124600488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c4806c9e0 0x7f8c480b5ab0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8c300097a0 tx=0x7f8c3002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1a01775a480340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.151+0000 7f8c477fe640 1 -- 172.21.15.19:0/2124600488 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30037070 con 0x7f8c4806c9e0 2024-04-25T10:00:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.151+0000 7f8c477fe640 1 -- 172.21.15.19:0/2124600488 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c30032440 con 0x7f8c4806c9e0 2024-04-25T10:00:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.151+0000 7f8c477fe640 1 -- 172.21.15.19:0/2124600488 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30032810 con 0x7f8c4806c9e0 2024-04-25T10:00:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.152+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/2124600488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c4806c9e0 msgr2=0x7f8c480b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.152+0000 7f8c4ebd5640 1 --2- 172.21.15.19:0/2124600488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c4806c9e0 0x7f8c480b5ab0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8c300097a0 tx=0x7f8c3002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.152+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/2124600488 shutdown_connections 2024-04-25T10:00:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.152+0000 7f8c4ebd5640 1 --2- 172.21.15.19:0/2124600488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c4806c9e0 0x7f8c480b5ab0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.152+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/2124600488 >> 172.21.15.19:0/2124600488 conn(0x7f8c480d0800 msgr2=0x7f8c480d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.152+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/2124600488 shutdown_connections 2024-04-25T10:00:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.152+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/2124600488 wait complete. 2024-04-25T10:00:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.153+0000 7f8c4ebd5640 1 Processor -- start 2024-04-25T10:00:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.153+0000 7f8c4ebd5640 1 -- start start 2024-04-25T10:00:58.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.154+0000 7f8c4ebd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4806c9e0 0x7f8c480c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.154+0000 7f8c4ebd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c480b5620 0x7f8c48148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.154+0000 7f8c4ebd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c4815b7d0 0x7f8c48159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.154+0000 7f8c4ebd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4815bd30 con 0x7f8c480b5620 2024-04-25T10:00:58.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.154+0000 7f8c4ebd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4815bbb0 con 0x7f8c4806c9e0 2024-04-25T10:00:58.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.154+0000 7f8c4ebd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480d1f40 con 0x7f8c4815b7d0 2024-04-25T10:00:58.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.154+0000 7f8c47fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c480b5620 0x7f8c48148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.154+0000 7f8c47fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c480b5620 0x7f8c48148b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50460/0 (socket says 172.21.15.19:50460) 2024-04-25T10:00:58.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.154+0000 7f8c47fff640 1 -- 172.21.15.19:0/8909621 learned_addr learned my addr 172.21.15.19:0/8909621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.154+0000 7f8c4d172640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c4815b7d0 0x7f8c48159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.154+0000 7f8c4c971640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4806c9e0 0x7f8c480c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.155+0000 7f8c47fff640 1 -- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c4815b7d0 msgr2=0x7f8c48159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.155+0000 7f8c47fff640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c4815b7d0 0x7f8c48159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.155+0000 7f8c47fff640 1 -- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4806c9e0 msgr2=0x7f8c480c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.155+0000 7f8c47fff640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4806c9e0 0x7f8c480c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.155+0000 7f8c47fff640 1 -- 172.21.15.19:0/8909621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4814bb20 con 0x7f8c480b5620 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.155+0000 7f8c47fff640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c480b5620 0x7f8c48148b80 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800c430 tx=0x7f8c3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.155+0000 7f8c457fa640 1 -- 172.21.15.19:0/8909621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38011790 con 0x7f8c480b5620 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.155+0000 7f8c457fa640 1 -- 172.21.15.19:0/8909621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c38017070 con 0x7f8c480b5620 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.155+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/8909621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c480ccb10 con 0x7f8c480b5620 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.155+0000 7f8c457fa640 1 -- 172.21.15.19:0/8909621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38011bb0 con 0x7f8c480b5620 2024-04-25T10:00:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.155+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/8909621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c480cd0d0 con 0x7f8c480b5620 2024-04-25T10:00:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.156+0000 7f8c457fa640 1 -- 172.21.15.19:0/8909621 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c3801c460 con 0x7f8c480b5620 2024-04-25T10:00:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.157+0000 7f8c457fa640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c2806b4d0 0x7f8c2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.157+0000 7f8c457fa640 1 -- 172.21.15.19:0/8909621 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c3808a0d0 con 0x7f8c480b5620 2024-04-25T10:00:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.157+0000 7f8c4c971640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c2806b4d0 0x7f8c2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.157+0000 7f8c4c971640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c2806b4d0 0x7f8c2806d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8c400012e0 tx=0x7f8c40001630 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.158+0000 7f8c519b4740 1 Processor -- start 2024-04-25T10:00:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.158+0000 7f8c519b4740 1 -- start start 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d3d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 0x55ff54d3d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d0d770 0x55ff54d10f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c519b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff54c76da0 con 0x55ff54d3ab10 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c519b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff54c1fb80 con 0x55ff54d3aef0 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c519b4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff54c72e40 con 0x55ff54d0d770 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c4c971640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d3d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c4c971640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d3d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50474/0 (socket says 172.21.15.19:50474) 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c4c971640 1 -- 172.21.15.19:0/4162358541 learned_addr learned my addr 172.21.15.19:0/4162358541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c4d172640 1 --2- 172.21.15.19:0/4162358541 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d0d770 0x55ff54d10f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c47fff640 1 --2- 172.21.15.19:0/4162358541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 0x55ff54d3d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c4c971640 1 -- 172.21.15.19:0/4162358541 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d0d770 msgr2=0x55ff54d10f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c4c971640 1 --2- 172.21.15.19:0/4162358541 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d0d770 0x55ff54d10f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c4c971640 1 -- 172.21.15.19:0/4162358541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 msgr2=0x55ff54d3d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c4c971640 1 --2- 172.21.15.19:0/4162358541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 0x55ff54d3d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c4c971640 1 -- 172.21.15.19:0/4162358541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff54c29f10 con 0x55ff54d3ab10 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c4c971640 1 --2- 172.21.15.19:0/4162358541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d3d320 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8c40007850 tx=0x7f8c40013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5a722a3891d622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c3e7fc640 1 -- 172.21.15.19:0/4162358541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40026070 con 0x55ff54d3ab10 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c3e7fc640 1 -- 172.21.15.19:0/4162358541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c40012350 con 0x55ff54d3ab10 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.159+0000 7f8c3e7fc640 1 -- 172.21.15.19:0/4162358541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40012700 con 0x55ff54d3ab10 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.160+0000 7f8c519b4740 1 -- 172.21.15.19:0/4162358541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 msgr2=0x55ff54d3d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.160+0000 7f8c519b4740 1 --2- 172.21.15.19:0/4162358541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d3d320 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8c40007850 tx=0x7f8c40013000 comp rx=0 tx=0).stop 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.160+0000 7f8c519b4740 1 -- 172.21.15.19:0/4162358541 shutdown_connections 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.160+0000 7f8c519b4740 1 --2- 172.21.15.19:0/4162358541 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d0d770 0x55ff54d10f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.160+0000 7f8c519b4740 1 --2- 172.21.15.19:0/4162358541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 0x55ff54d3d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.160+0000 7f8c519b4740 1 --2- 172.21.15.19:0/4162358541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d3d320 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.160+0000 7f8c519b4740 1 -- 172.21.15.19:0/4162358541 >> 172.21.15.19:0/4162358541 conn(0x55ff54d0f9e0 msgr2=0x55ff54d58320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.160+0000 7f8c519b4740 1 -- 172.21.15.19:0/4162358541 shutdown_connections 2024-04-25T10:00:58.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.160+0000 7f8c519b4740 1 -- 172.21.15.19:0/4162358541 wait complete. 2024-04-25T10:00:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.160+0000 7f8c519b4740 1 -- start start 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.161+0000 7f8c519b4740 1 Processor -- start 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.161+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3ab10 0x55ff54d2ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.161+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 0x55ff54d53ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.161+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d3eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.161+0000 7f8c519b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff54d12b90 con 0x55ff54d3ea70 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.161+0000 7f8c519b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff54c154b0 con 0x55ff54d3aef0 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.161+0000 7f8c519b4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff54d12e70 con 0x55ff54d3ab10 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.161+0000 7f8c519b4740 10 client.0 _ll_register_callbacks cb 0x55ff54c55180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c4d172640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d3eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c4d172640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d3eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50480/0 (socket says 172.21.15.19:50480) 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c4d172640 1 -- 172.21.15.19:0/2950359648 learned_addr learned my addr 172.21.15.19:0/2950359648 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c4c971640 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3ab10 0x55ff54d2ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c47fff640 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 0x55ff54d53ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c4d172640 1 -- 172.21.15.19:0/2950359648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3ab10 msgr2=0x55ff54d2ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c4d172640 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3ab10 0x55ff54d2ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c4d172640 1 -- 172.21.15.19:0/2950359648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 msgr2=0x55ff54d53ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c4d172640 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 0x55ff54d53ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c4d172640 1 -- 172.21.15.19:0/2950359648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff54a557d0 con 0x55ff54d3ea70 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c4d172640 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d3eef0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f8c300098c0 tx=0x7f8c30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6995fb9445bd07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c277fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c277fe640 1 -- 172.21.15.19:0/2950359648 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30042070 con 0x55ff54d3ea70 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c277fe640 1 -- 172.21.15.19:0/2950359648 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c30047c40 con 0x55ff54d3ea70 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c519b4740 1 -- 172.21.15.19:0/2950359648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c40012350 con 0x55ff54d3ea70 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c277fe640 1 -- 172.21.15.19:0/2950359648 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c30037040 con 0x55ff54d3ea70 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c519b4740 10 client.8925 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.162+0000 7f8c519b4740 1 -- 172.21.15.19:0/2950359648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55ff5498b3f0 con 0x55ff54d3ea70 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.163+0000 7f8c277fe640 1 -- 172.21.15.19:0/2950359648 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c40012350 con 0x55ff54d3ea70 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.163+0000 7f8c277fe640 1 -- 172.21.15.19:0/2950359648 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8c30032a10 con 0x55ff54d3ea70 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.163+0000 7f8c519b4740 10 client.8925 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.163+0000 7f8c519b4740 10 client.8925 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.163+0000 7f8c519b4740 20 client.8925 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.163+0000 7f8c519b4740 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ff54d6bae0 0x55ff54d6bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.163+0000 7f8c519b4740 4 client.8925 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.163+0000 7f8c519b4740 1 -- 172.21.15.19:0/2950359648 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff54c842e0 con 0x55ff54d6bae0 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.163+0000 7f8c4c971640 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ff54d6bae0 0x55ff54d6bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.163+0000 7f8c4c971640 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ff54d6bae0 0x55ff54d6bef0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24ab332834e0c4b server_cookie=e4cfeacb8943be2a in_seq=0 out_seq=0 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.163+0000 7f8c277fe640 10 client.8925 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c277fe640 1 -- 172.21.15.19:0/2950359648 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff54c842e0 con 0x55ff54d6bae0 2024-04-25T10:00:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c277fe640 10 client.8925 handle_command_reply: tid=0 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 1 client.8925 shutdown 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 1 -- 172.21.15.19:0/2950359648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 msgr2=0x55ff54d3eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d3eef0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f8c300098c0 tx=0x7f8c30004730 comp rx=0 tx=0).stop 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 1 -- 172.21.15.19:0/2950359648 shutdown_connections 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ff54d6bae0 0x55ff54d6bef0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d3eef0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 0x55ff54d53ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 1 --2- 172.21.15.19:0/2950359648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3ab10 0x55ff54d2ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c277fe640 0 client.8925 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 1 -- 172.21.15.19:0/2950359648 >> 172.21.15.19:0/2950359648 conn(0x55ff54d0d770 msgr2=0x55ff54d0e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 1 -- 172.21.15.19:0/2950359648 shutdown_connections 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 1 -- 172.21.15.19:0/2950359648 wait complete. 2024-04-25T10:00:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.164+0000 7f8c519b4740 20 client.8925 trim_cache size 0 max 16384 2024-04-25T10:00:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.179+0000 7f8c519b4740 1 Processor -- start 2024-04-25T10:00:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.179+0000 7f8c519b4740 1 -- start start 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3ab10 0x55ff54d6bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 0x55ff54d391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d55da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c519b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c482002c0 con 0x55ff54d3ea70 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c519b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480b8c50 con 0x55ff54d3aef0 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c519b4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480b8050 con 0x55ff54d3ab10 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c4d172640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d55da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c4d172640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d55da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50496/0 (socket says 172.21.15.19:50496) 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c4d172640 1 -- 172.21.15.19:0/3950919610 learned_addr learned my addr 172.21.15.19:0/3950919610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c4d172640 1 -- 172.21.15.19:0/3950919610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3ab10 msgr2=0x55ff54d6bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c4d172640 1 --2- 172.21.15.19:0/3950919610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3ab10 0x55ff54d6bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c4d172640 1 -- 172.21.15.19:0/3950919610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 msgr2=0x55ff54d391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c4d172640 1 --2- 172.21.15.19:0/3950919610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 0x55ff54d391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c4d172640 1 -- 172.21.15.19:0/3950919610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff5494b0f0 con 0x55ff54d3ea70 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c4d172640 1 --2- 172.21.15.19:0/3950919610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d55da0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f8c30006c90 tx=0x7f8c3003e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645fd0138b2dcf57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c3dffb640 1 -- 172.21.15.19:0/3950919610 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30032a10 con 0x55ff54d3ea70 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c3dffb640 1 -- 172.21.15.19:0/3950919610 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c30005990 con 0x55ff54d3ea70 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.180+0000 7f8c3dffb640 1 -- 172.21.15.19:0/3950919610 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30005d60 con 0x55ff54d3ea70 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.181+0000 7f8c519b4740 1 -- 172.21.15.19:0/3950919610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 msgr2=0x55ff54d55da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.181+0000 7f8c519b4740 1 --2- 172.21.15.19:0/3950919610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d55da0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f8c30006c90 tx=0x7f8c3003e070 comp rx=0 tx=0).stop 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.181+0000 7f8c519b4740 1 -- 172.21.15.19:0/3950919610 shutdown_connections 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.181+0000 7f8c519b4740 1 --2- 172.21.15.19:0/3950919610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ea70 0x55ff54d55da0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.181+0000 7f8c519b4740 1 --2- 172.21.15.19:0/3950919610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3aef0 0x55ff54d391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.181+0000 7f8c519b4740 1 --2- 172.21.15.19:0/3950919610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3ab10 0x55ff54d6bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.181+0000 7f8c519b4740 1 -- 172.21.15.19:0/3950919610 >> 172.21.15.19:0/3950919610 conn(0x55ff54d0d770 msgr2=0x55ff54d12da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.181+0000 7f8c519b4740 1 -- 172.21.15.19:0/3950919610 shutdown_connections 2024-04-25T10:00:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.181+0000 7f8c519b4740 1 -- 172.21.15.19:0/3950919610 wait complete. 2024-04-25T10:00:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.182+0000 7f8c519b4740 1 -- start start 2024-04-25T10:00:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.182+0000 7f8c519b4740 1 Processor -- start 2024-04-25T10:00:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.182+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d56200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.182+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3aef0 0x55ff54d4e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.182+0000 7f8c519b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3ea70 0x55ff54d4e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.182+0000 7f8c519b4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff54d59d50 con 0x55ff54d3ab10 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.182+0000 7f8c519b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480b8650 con 0x55ff54d3ea70 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.182+0000 7f8c519b4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff54d5ab30 con 0x55ff54d3aef0 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c519b4740 10 client.0 _ll_register_callbacks cb 0x7f8c40004da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c4c971640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d56200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c4c971640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d56200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50500/0 (socket says 172.21.15.19:50500) 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c4c971640 1 -- 172.21.15.19:0/3942527543 learned_addr learned my addr 172.21.15.19:0/3942527543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c47fff640 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3aef0 0x55ff54d4e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c4d172640 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3ea70 0x55ff54d4e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c4c971640 1 -- 172.21.15.19:0/3942527543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3aef0 msgr2=0x55ff54d4e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c4c971640 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3aef0 0x55ff54d4e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c4c971640 1 -- 172.21.15.19:0/3942527543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3ea70 msgr2=0x55ff54d4e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c4c971640 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3ea70 0x55ff54d4e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c4c971640 1 -- 172.21.15.19:0/3942527543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff54d10e50 con 0x55ff54d3ab10 2024-04-25T10:00:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c4d172640 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3ea70 0x55ff54d4e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c4c971640 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d56200 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x55ff54d15070 tx=0x7f8c40001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da51e07a018cc1b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c277fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c277fe640 1 -- 172.21.15.19:0/3942527543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40035020 con 0x55ff54d3ab10 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c277fe640 1 -- 172.21.15.19:0/3942527543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c40031170 con 0x55ff54d3ab10 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c519b4740 1 -- 172.21.15.19:0/3942527543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c30005990 con 0x55ff54d3ab10 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c277fe640 1 -- 172.21.15.19:0/3942527543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c4003f050 con 0x55ff54d3ab10 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c519b4740 10 client.8931 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.183+0000 7f8c519b4740 1 -- 172.21.15.19:0/3942527543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55ff54d58f60 con 0x55ff54d3ab10 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c277fe640 1 -- 172.21.15.19:0/3942527543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c30005990 con 0x55ff54d3ab10 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c277fe640 1 -- 172.21.15.19:0/3942527543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8c40042020 con 0x55ff54d3ab10 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 10 client.8931 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 10 client.8931 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 20 client.8931 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ff54d56e30 0x55ff54d57240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 4 client.8931 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 1 -- 172.21.15.19:0/3942527543 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55ff54d59f20 con 0x55ff54d56e30 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c47fff640 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ff54d56e30 0x55ff54d57240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c277fe640 10 client.8931 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c47fff640 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ff54d56e30 0x55ff54d57240 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99093909f51ad0b server_cookie=4258f98135666cf1 in_seq=0 out_seq=0 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c277fe640 1 -- 172.21.15.19:0/3942527543 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55ff54d59f20 con 0x55ff54d56e30 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c277fe640 10 client.8931 handle_command_reply: tid=0 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 1 client.8931 shutdown 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 1 -- 172.21.15.19:0/3942527543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 msgr2=0x55ff54d56200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d56200 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x55ff54d15070 tx=0x7f8c40001400 comp rx=0 tx=0).stop 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 1 -- 172.21.15.19:0/3942527543 shutdown_connections 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x55ff54d56e30 0x55ff54d57240 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff54d3ea70 0x55ff54d4e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c519b4740 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55ff54d3aef0 0x55ff54d4e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.184+0000 7f8c277fe640 0 client.8931 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:58.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c519b4740 1 --2- 172.21.15.19:0/3942527543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ff54d3ab10 0x55ff54d56200 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c519b4740 1 -- 172.21.15.19:0/3942527543 >> 172.21.15.19:0/3942527543 conn(0x55ff54d0d770 msgr2=0x55ff54d3d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c519b4740 1 -- 172.21.15.19:0/3942527543 shutdown_connections 2024-04-25T10:00:58.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c519b4740 1 -- 172.21.15.19:0/3942527543 wait complete. 2024-04-25T10:00:58.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c519b4740 20 client.8931 trim_cache size 0 max 16384 2024-04-25T10:00:58.199 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:58.199 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T10:00:58.199 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:58.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c2806b4d0 msgr2=0x7f8c2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c2806b4d0 0x7f8c2806d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8c400012e0 tx=0x7f8c40001630 comp rx=0 tx=0).stop 2024-04-25T10:00:58.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/8909621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c480b5620 msgr2=0x7f8c48148b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c480b5620 0x7f8c48148b80 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800c430 tx=0x7f8c3800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/8909621 shutdown_connections 2024-04-25T10:00:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c2806b4d0 0x7f8c2806d990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c4815b7d0 0x7f8c48159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c480b5620 0x7f8c48148b80 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 --2- 172.21.15.19:0/8909621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4806c9e0 0x7f8c480c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/8909621 >> 172.21.15.19:0/8909621 conn(0x7f8c480d0800 msgr2=0x7f8c480b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/8909621 shutdown_connections 2024-04-25T10:00:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.185+0000 7f8c4ebd5640 1 -- 172.21.15.19:0/8909621 wait complete. 2024-04-25T10:00:58.202 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:00:58.202 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:00:58.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.300+0000 7ff5ff05a640 1 Processor -- start 2024-04-25T10:00:58.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.300+0000 7ff5ff05a640 1 -- start start 2024-04-25T10:00:58.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5ff05a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5f8158230 0x7ff5f80d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5ff05a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff5f80d1b80 0x7ff5f806c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5ff05a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f80d2020 con 0x7ff5f806c870 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5ff05a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f8158610 con 0x7ff5f80d1b80 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5ff05a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f806bea0 con 0x7ff5f8158230 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5f7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff5f80d1b80 0x7ff5f806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5f7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff5f80d1b80 0x7ff5f806c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50512/0 (socket says 172.21.15.19:50512) 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5f7fff640 1 -- 172.21.15.19:0/1535384402 learned_addr learned my addr 172.21.15.19:0/1535384402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/1535384402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5f8158230 0x7ff5f80d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5f7fff640 1 -- 172.21.15.19:0/1535384402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5f8158230 msgr2=0x7ff5f80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/1535384402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5f8158230 0x7ff5f80d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5f7fff640 1 -- 172.21.15.19:0/1535384402 >> v1:172.21.15.112:6789/0 conn(0x7ff5f806c870 legacy=0x7ff5f80b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.301+0000 7ff5f7fff640 1 -- 172.21.15.19:0/1535384402 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f80d2300 con 0x7ff5f80d1b80 2024-04-25T10:00:58.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.302+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/1535384402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80d1b80 0x7ff5f806c330 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009850 tx=0x7ff5e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c88fcdcb9263cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.302+0000 7ff5f77fe640 1 -- 172.21.15.19:0/1535384402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4004030 con 0x7ff5f80d1b80 2024-04-25T10:00:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.302+0000 7ff5f77fe640 1 -- 172.21.15.19:0/1535384402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e40041d0 con 0x7ff5f80d1b80 2024-04-25T10:00:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.302+0000 7ff5f77fe640 1 -- 172.21.15.19:0/1535384402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4003610 con 0x7ff5f80d1b80 2024-04-25T10:00:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.303+0000 7ff5ff05a640 1 -- 172.21.15.19:0/1535384402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80d1b80 msgr2=0x7ff5f806c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.303+0000 7ff5ff05a640 1 --2- 172.21.15.19:0/1535384402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80d1b80 0x7ff5f806c330 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009850 tx=0x7ff5e402a230 comp rx=0 tx=0).stop 2024-04-25T10:00:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.303+0000 7ff5ff05a640 1 -- 172.21.15.19:0/1535384402 shutdown_connections 2024-04-25T10:00:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.303+0000 7ff5ff05a640 1 --2- 172.21.15.19:0/1535384402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80d1b80 0x7ff5f806c330 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.303+0000 7ff5ff05a640 1 --2- 172.21.15.19:0/1535384402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5f8158230 0x7ff5f80d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.303+0000 7ff5ff05a640 1 -- 172.21.15.19:0/1535384402 >> 172.21.15.19:0/1535384402 conn(0x7ff5f8061600 msgr2=0x7ff5f80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.303+0000 7ff5ff05a640 1 -- 172.21.15.19:0/1535384402 shutdown_connections 2024-04-25T10:00:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.303+0000 7ff5ff05a640 1 -- 172.21.15.19:0/1535384402 wait complete. 2024-04-25T10:00:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.304+0000 7ff5ff05a640 1 Processor -- start 2024-04-25T10:00:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.304+0000 7ff5ff05a640 1 -- start start 2024-04-25T10:00:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5ff05a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f806c870 0x7ff5f80cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5ff05a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5f80d1b80 0x7ff5f81f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5ff05a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8158230 0x7ff5f81e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5ff05a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f815e320 con 0x7ff5f806c870 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5ff05a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f80d25e0 con 0x7ff5f8158230 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5ff05a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f815e4a0 con 0x7ff5f80d1b80 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5fcdf6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f806c870 0x7ff5f80cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5fcdf6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f806c870 0x7ff5f80cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50514/0 (socket says 172.21.15.19:50514) 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5fcdf6640 1 -- 172.21.15.19:0/976632357 learned_addr learned my addr 172.21.15.19:0/976632357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5f80d1b80 0x7ff5f81f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8158230 0x7ff5f81e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5fcdf6640 1 -- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5f80d1b80 msgr2=0x7ff5f81f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5f80d1b80 0x7ff5f81f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5fcdf6640 1 -- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8158230 msgr2=0x7ff5f81e55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8158230 0x7ff5f81e55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.305+0000 7ff5fcdf6640 1 -- 172.21.15.19:0/976632357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f81f3c90 con 0x7ff5f806c870 2024-04-25T10:00:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.306+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f806c870 0x7ff5f80cd770 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00e870 tx=0x7ff5ec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.306+0000 7ff5f57fa640 1 -- 172.21.15.19:0/976632357 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec017070 con 0x7ff5f806c870 2024-04-25T10:00:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.306+0000 7ff5f57fa640 1 -- 172.21.15.19:0/976632357 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec00c940 con 0x7ff5f806c870 2024-04-25T10:00:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.306+0000 7ff5ff05a640 1 -- 172.21.15.19:0/976632357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f8158830 con 0x7ff5f806c870 2024-04-25T10:00:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.306+0000 7ff5f57fa640 1 -- 172.21.15.19:0/976632357 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec00cd30 con 0x7ff5f806c870 2024-04-25T10:00:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.306+0000 7ff5ff05a640 1 -- 172.21.15.19:0/976632357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f8158df0 con 0x7ff5f806c870 2024-04-25T10:00:58.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.307+0000 7ff5f57fa640 1 -- 172.21.15.19:0/976632357 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5ec01a070 con 0x7ff5f806c870 2024-04-25T10:00:58.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.308+0000 7ff5f57fa640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5dc06b4d0 0x7ff5dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.308+0000 7ff5f57fa640 1 -- 172.21.15.19:0/976632357 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5ec08a6b0 con 0x7ff5f806c870 2024-04-25T10:00:58.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.308+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5dc06b4d0 0x7ff5dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.308+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5dc06b4d0 0x7ff5dc06d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4033000 tx=0x7ff5e402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.309+0000 7ff601e30740 1 Processor -- start 2024-04-25T10:00:58.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.309+0000 7ff601e30740 1 -- start start 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff601e30740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b4cb10 0x5571c6b4f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff601e30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 0x5571c6b4f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff601e30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b1f770 0x5571c6b22f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff601e30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c6a88da0 con 0x5571c6b1f770 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff601e30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c6a31b80 con 0x5571c6b4cef0 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff601e30740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571c6a84e40 con 0x5571c6b4cb10 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5fd5f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b1f770 0x5571c6b22f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5fd5f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b1f770 0x5571c6b22f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50526/0 (socket says 172.21.15.19:50526) 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/3354105346 learned_addr learned my addr 172.21.15.19:0/3354105346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/3354105346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b4cb10 0x5571c6b4f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/3354105346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 0x5571c6b4f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/3354105346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b4cb10 msgr2=0x5571c6b4f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/3354105346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b4cb10 0x5571c6b4f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/3354105346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 msgr2=0x5571c6b4f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/3354105346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 0x5571c6b4f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/3354105346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571c6a3bf10 con 0x5571c6b1f770 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/3354105346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 0x5571c6b4f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/3354105346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b1f770 0x5571c6b22f80 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0001d90 tx=0x7ff5f000d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ae7d9230b46434 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5e27fc640 1 -- 172.21.15.19:0/3354105346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0016020 con 0x5571c6b1f770 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5e27fc640 1 -- 172.21.15.19:0/3354105346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f000dd30 con 0x5571c6b1f770 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.310+0000 7ff5e27fc640 1 -- 172.21.15.19:0/3354105346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0012610 con 0x5571c6b1f770 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.311+0000 7ff601e30740 1 -- 172.21.15.19:0/3354105346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b1f770 msgr2=0x5571c6b22f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.311+0000 7ff601e30740 1 --2- 172.21.15.19:0/3354105346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b1f770 0x5571c6b22f80 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0001d90 tx=0x7ff5f000d150 comp rx=0 tx=0).stop 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.311+0000 7ff601e30740 1 -- 172.21.15.19:0/3354105346 shutdown_connections 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.311+0000 7ff601e30740 1 --2- 172.21.15.19:0/3354105346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b1f770 0x5571c6b22f80 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.311+0000 7ff601e30740 1 --2- 172.21.15.19:0/3354105346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 0x5571c6b4f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.311+0000 7ff601e30740 1 --2- 172.21.15.19:0/3354105346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b4cb10 0x5571c6b4f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.311+0000 7ff601e30740 1 -- 172.21.15.19:0/3354105346 >> 172.21.15.19:0/3354105346 conn(0x5571c6b219e0 msgr2=0x5571c6b6a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.311+0000 7ff601e30740 1 -- 172.21.15.19:0/3354105346 shutdown_connections 2024-04-25T10:00:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.311+0000 7ff601e30740 1 -- 172.21.15.19:0/3354105346 wait complete. 2024-04-25T10:00:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.312+0000 7ff601e30740 1 -- start start 2024-04-25T10:00:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.312+0000 7ff601e30740 1 Processor -- start 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff601e30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cb10 0x5571c6b426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff601e30740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b4cef0 0x5571c6b6d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff601e30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b47c10 0x5571c6b4b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff601e30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c6b24b90 con 0x5571c6b47c10 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff601e30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c6a274b0 con 0x5571c6b4cb10 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff601e30740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571c6b24e70 con 0x5571c6b4cef0 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff601e30740 10 client.0 _ll_register_callbacks cb 0x5571c6a67180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5fd5f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b47c10 0x5571c6b4b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5fd5f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b47c10 0x5571c6b4b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50528/0 (socket says 172.21.15.19:50528) 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/1966386545 learned_addr learned my addr 172.21.15.19:0/1966386545 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b4cef0 0x5571c6b6d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cb10 0x5571c6b426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/1966386545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b4cef0 msgr2=0x5571c6b6d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b4cef0 0x5571c6b6d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/1966386545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cb10 msgr2=0x5571c6b426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cb10 0x5571c6b426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/1966386545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571c68677d0 con 0x5571c6b47c10 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b47c10 0x5571c6b4b190 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0012a40 tx=0x7ff5f0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae2722825e01670 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5d77fe640 1 -- 172.21.15.19:0/1966386545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0016040 con 0x5571c6b47c10 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5d77fe640 1 -- 172.21.15.19:0/1966386545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5f0006d60 con 0x5571c6b47c10 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff601e30740 1 -- 172.21.15.19:0/1966386545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5f000dd30 con 0x5571c6b47c10 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.313+0000 7ff5d77fe640 1 -- 172.21.15.19:0/1966386545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5f002a050 con 0x5571c6b47c10 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff601e30740 10 client.8943 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff601e30740 1 -- 172.21.15.19:0/1966386545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5571c679d3f0 con 0x5571c6b47c10 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff5d77fe640 1 -- 172.21.15.19:0/1966386545 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f000dd30 con 0x5571c6b47c10 2024-04-25T10:00:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff5d77fe640 1 -- 172.21.15.19:0/1966386545 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff5f002d020 con 0x5571c6b47c10 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff601e30740 10 client.8943 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff601e30740 10 client.8943 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff601e30740 20 client.8943 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff601e30740 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x5571c6b470c0 0x5571c6b6b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff601e30740 4 client.8943 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff601e30740 1 -- 172.21.15.19:0/1966386545 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571c6a962e0 con 0x5571c6b470c0 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x5571c6b470c0 0x5571c6b6b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x5571c6b470c0 0x5571c6b6b450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e1cf2b624a2ea1 server_cookie=f3d673f2b956a97f in_seq=0 out_seq=0 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.314+0000 7ff5d77fe640 10 client.8943 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff5d77fe640 1 -- 172.21.15.19:0/1966386545 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5571c6a962e0 con 0x5571c6b470c0 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff5d77fe640 10 client.8943 handle_command_reply: tid=0 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 1 client.8943 shutdown 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 1 -- 172.21.15.19:0/1966386545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b47c10 msgr2=0x5571c6b4b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b47c10 0x5571c6b4b190 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0012a40 tx=0x7ff5f0001250 comp rx=0 tx=0).stop 2024-04-25T10:00:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 1 -- 172.21.15.19:0/1966386545 shutdown_connections 2024-04-25T10:00:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x5571c6b470c0 0x5571c6b6b450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b47c10 0x5571c6b4b190 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b4cef0 0x5571c6b6d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff5d77fe640 0 client.8943 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 1 --2- 172.21.15.19:0/1966386545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cb10 0x5571c6b426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 1 -- 172.21.15.19:0/1966386545 >> 172.21.15.19:0/1966386545 conn(0x5571c6b1f770 msgr2=0x5571c6b242b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 1 -- 172.21.15.19:0/1966386545 shutdown_connections 2024-04-25T10:00:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 1 -- 172.21.15.19:0/1966386545 wait complete. 2024-04-25T10:00:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.315+0000 7ff601e30740 20 client.8943 trim_cache size 0 max 16384 2024-04-25T10:00:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.330+0000 7ff601e30740 1 Processor -- start 2024-04-25T10:00:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.330+0000 7ff601e30740 1 -- start start 2024-04-25T10:00:58.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff601e30740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b47c10 0x5571c6b1ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff601e30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cb10 0x5571c6b20480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff601e30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 0x5571c6b48520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff601e30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f8200610 con 0x5571c6b4cb10 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff601e30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f81ff800 con 0x5571c6b4cef0 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff601e30740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f815a500 con 0x5571c6b47c10 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cb10 0x5571c6b20480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cb10 0x5571c6b20480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50540/0 (socket says 172.21.15.19:50540) 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5f7fff640 1 -- 172.21.15.19:0/1348377589 learned_addr learned my addr 172.21.15.19:0/1348377589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/1348377589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b47c10 0x5571c6b1ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/1348377589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 0x5571c6b48520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5f7fff640 1 -- 172.21.15.19:0/1348377589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b47c10 msgr2=0x5571c6b1ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/1348377589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b47c10 0x5571c6b1ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5f7fff640 1 -- 172.21.15.19:0/1348377589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 msgr2=0x5571c6b48520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/1348377589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 0x5571c6b48520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5f7fff640 1 -- 172.21.15.19:0/1348377589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571c675d0f0 con 0x5571c6b4cb10 2024-04-25T10:00:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/1348377589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 0x5571c6b48520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.331+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/1348377589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cb10 0x5571c6b20480 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7ff5e402a820 tx=0x7ff5e4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8aa01de603072b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff5e1ffb640 1 -- 172.21.15.19:0/1348377589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4050070 con 0x5571c6b4cb10 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff5e1ffb640 1 -- 172.21.15.19:0/1348377589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e4003d60 con 0x5571c6b4cb10 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff5e1ffb640 1 -- 172.21.15.19:0/1348377589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e40060e0 con 0x5571c6b4cb10 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff601e30740 1 -- 172.21.15.19:0/1348377589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cb10 msgr2=0x5571c6b20480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff601e30740 1 --2- 172.21.15.19:0/1348377589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cb10 0x5571c6b20480 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7ff5e402a820 tx=0x7ff5e4008030 comp rx=0 tx=0).stop 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff601e30740 1 -- 172.21.15.19:0/1348377589 shutdown_connections 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff601e30740 1 --2- 172.21.15.19:0/1348377589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cef0 0x5571c6b48520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff601e30740 1 --2- 172.21.15.19:0/1348377589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cb10 0x5571c6b20480 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff601e30740 1 --2- 172.21.15.19:0/1348377589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b47c10 0x5571c6b1ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff601e30740 1 -- 172.21.15.19:0/1348377589 >> 172.21.15.19:0/1348377589 conn(0x5571c6b1f770 msgr2=0x5571c6b1ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff601e30740 1 -- 172.21.15.19:0/1348377589 shutdown_connections 2024-04-25T10:00:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.332+0000 7ff601e30740 1 -- 172.21.15.19:0/1348377589 wait complete. 2024-04-25T10:00:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.333+0000 7ff601e30740 1 -- start start 2024-04-25T10:00:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.333+0000 7ff601e30740 1 Processor -- start 2024-04-25T10:00:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff601e30740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b47c10 0x5571c6bdc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff601e30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cb10 0x5571c6b69bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff601e30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cef0 0x5571c6b6a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff601e30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c6b4b110 con 0x5571c6b4cef0 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff601e30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f815ab00 con 0x5571c6b4cb10 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff601e30740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571c6b49510 con 0x5571c6b47c10 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff601e30740 10 client.0 _ll_register_callbacks cb 0x7ff5f00015e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5fd5f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cef0 0x5571c6b6a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5fd5f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cef0 0x5571c6b6a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50556/0 (socket says 172.21.15.19:50556) 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/521467575 learned_addr learned my addr 172.21.15.19:0/521467575 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b47c10 0x5571c6bdc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5f7fff640 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cb10 0x5571c6b69bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/521467575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b47c10 msgr2=0x5571c6bdc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b47c10 0x5571c6bdc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/521467575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cb10 msgr2=0x5571c6b69bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cb10 0x5571c6b69bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5fd5f7640 1 -- 172.21.15.19:0/521467575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571c6b48540 con 0x5571c6b4cef0 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5fd5f7640 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cef0 0x5571c6b6a100 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0004c30 tx=0x7ff5f0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f9138fe5036353 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5d77fe640 1 -- 172.21.15.19:0/521467575 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0016030 con 0x5571c6b4cef0 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff5d77fe640 1 -- 172.21.15.19:0/521467575 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5f0005310 con 0x5571c6b4cef0 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.334+0000 7ff601e30740 1 -- 172.21.15.19:0/521467575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5e4003d60 con 0x5571c6b4cef0 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff5d77fe640 1 -- 172.21.15.19:0/521467575 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5f0039050 con 0x5571c6b4cef0 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff601e30740 10 client.8949 fetch_fsmap learned FSMap version 72 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff601e30740 1 -- 172.21.15.19:0/521467575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5571c6b48b00 con 0x5571c6b4cef0 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff5d77fe640 1 -- 172.21.15.19:0/521467575 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e4003d60 con 0x5571c6b4cef0 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff5d77fe640 1 -- 172.21.15.19:0/521467575 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff5f003c020 con 0x5571c6b4cef0 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff601e30740 10 client.8949 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff601e30740 10 client.8949 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:00:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff601e30740 20 client.8949 populate_metadata read hostname 'smithi019' 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff601e30740 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x5571c6b4c3d0 0x5571c6be02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff601e30740 4 client.8949 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff601e30740 1 -- 172.21.15.19:0/521467575 --> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5571c6a90630 con 0x5571c6b4c3d0 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x5571c6b4c3d0 0x5571c6be02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff5d77fe640 10 client.8949 ms_handle_connect on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff5fcdf6640 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x5571c6b4c3d0 0x5571c6be02a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e948445369de3c10 server_cookie=1646aafa19ebb2c4 in_seq=0 out_seq=0 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff5d77fe640 1 -- 172.21.15.19:0/521467575 <== mds.0 v2:172.21.15.19:6833/3442229953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5571c6a90630 con 0x5571c6b4c3d0 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff5d77fe640 10 client.8949 handle_command_reply: tid=0 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.335+0000 7ff601e30740 1 client.8949 shutdown 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff601e30740 1 -- 172.21.15.19:0/521467575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cef0 msgr2=0x5571c6b6a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff601e30740 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cef0 0x5571c6b6a100 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0004c30 tx=0x7ff5f0009040 comp rx=0 tx=0).stop 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff601e30740 1 -- 172.21.15.19:0/521467575 shutdown_connections 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff601e30740 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.19:6833/3442229953,v1:172.21.15.19:6834/3442229953] conn(0x5571c6b4c3d0 0x5571c6be02a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff601e30740 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c6b4cef0 0x5571c6b6a100 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff601e30740 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571c6b4cb10 0x5571c6b69bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5d77fe640 0 client.8949 ms_handle_reset on v2:172.21.15.19:6833/3442229953 2024-04-25T10:00:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff601e30740 1 --2- 172.21.15.19:0/521467575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5571c6b47c10 0x5571c6bdc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff601e30740 1 -- 172.21.15.19:0/521467575 >> 172.21.15.19:0/521467575 conn(0x5571c6b1f770 msgr2=0x5571c6b26600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff601e30740 1 -- 172.21.15.19:0/521467575 shutdown_connections 2024-04-25T10:00:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff601e30740 1 -- 172.21.15.19:0/521467575 wait complete. 2024-04-25T10:00:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff601e30740 20 client.8949 trim_cache size 0 max 16384 2024-04-25T10:00:58.347 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:00:58.347 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:00:58.347 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:00:58.348 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:00:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 -- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5dc06b4d0 msgr2=0x7ff5dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5dc06b4d0 0x7ff5dc06d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4033000 tx=0x7ff5e402e000 comp rx=0 tx=0).stop 2024-04-25T10:00:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 -- 172.21.15.19:0/976632357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f806c870 msgr2=0x7ff5f80cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f806c870 0x7ff5f80cd770 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00e870 tx=0x7ff5ec00ed30 comp rx=0 tx=0).stop 2024-04-25T10:00:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 -- 172.21.15.19:0/976632357 shutdown_connections 2024-04-25T10:00:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5dc06b4d0 0x7ff5dc06d990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5f8158230 0x7ff5f81e55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5f80d1b80 0x7ff5f81f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 --2- 172.21.15.19:0/976632357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f806c870 0x7ff5f80cd770 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 -- 172.21.15.19:0/976632357 >> 172.21.15.19:0/976632357 conn(0x7ff5f8061600 msgr2=0x7ff5f80d0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 -- 172.21.15.19:0/976632357 shutdown_connections 2024-04-25T10:00:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.336+0000 7ff5ff05a640 1 -- 172.21.15.19:0/976632357 wait complete. 2024-04-25T10:00:58.352 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:00:58.352 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:58.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.448+0000 7fb1f4b63640 1 Processor -- start 2024-04-25T10:00:58.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.448+0000 7fb1f4b63640 1 -- start start 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1f4b63640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb1f015dd30 0x7fb1f00b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1f4b63640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb1f00b9770 0x7fb1f00d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1f4b63640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1f00d2e90 0x7fb1f00d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1f4b63640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f00d1df0 con 0x7fb1f00b9770 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1f4b63640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f00d20d0 con 0x7fb1f015dd30 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1f4b63640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00d23b0 con 0x7fb1f00d2e90 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1edd9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb1f00b9770 0x7fb1f00d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1edd9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb1f00b9770 0x7fb1f00d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50562/0 (socket says 172.21.15.19:50562) 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1edd9b640 1 -- 172.21.15.19:0/2813326939 learned_addr learned my addr 172.21.15.19:0/2813326939 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1ee59c640 1 --2- 172.21.15.19:0/2813326939 >> v2:172.21.15.112:3300/0 conn(0x7fb1f015dd30 0x7fb1f00b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1eed9d640 1 --2- 172.21.15.19:0/2813326939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1f00d2e90 0x7fb1f00d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1edd9b640 1 -- 172.21.15.19:0/2813326939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1f00d2e90 msgr2=0x7fb1f00d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1edd9b640 1 --2- 172.21.15.19:0/2813326939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1f00d2e90 0x7fb1f00d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1edd9b640 1 -- 172.21.15.19:0/2813326939 >> v2:172.21.15.112:3300/0 conn(0x7fb1f015dd30 msgr2=0x7fb1f00b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1edd9b640 1 --2- 172.21.15.19:0/2813326939 >> v2:172.21.15.112:3300/0 conn(0x7fb1f015dd30 0x7fb1f00b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.449+0000 7fb1edd9b640 1 -- 172.21.15.19:0/2813326939 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f00d2690 con 0x7fb1f00b9770 2024-04-25T10:00:58.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.450+0000 7fb1edd9b640 1 --2- 172.21.15.19:0/2813326939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f00b9770 0x7fb1f00d03c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fb1f006d000 tx=0x7fb1e402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552fe0f51ff5c3cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.450+0000 7fb1ed59a640 1 -- 172.21.15.19:0/2813326939 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4004030 con 0x7fb1f00b9770 2024-04-25T10:00:58.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.450+0000 7fb1ed59a640 1 -- 172.21.15.19:0/2813326939 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e40041d0 con 0x7fb1f00b9770 2024-04-25T10:00:58.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.450+0000 7fb1ed59a640 1 -- 172.21.15.19:0/2813326939 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4003610 con 0x7fb1f00b9770 2024-04-25T10:00:58.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.451+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2813326939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f00b9770 msgr2=0x7fb1f00d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.451+0000 7fb1f4b63640 1 --2- 172.21.15.19:0/2813326939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f00b9770 0x7fb1f00d03c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fb1f006d000 tx=0x7fb1e402a130 comp rx=0 tx=0).stop 2024-04-25T10:00:58.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.451+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2813326939 shutdown_connections 2024-04-25T10:00:58.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.451+0000 7fb1f4b63640 1 --2- 172.21.15.19:0/2813326939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1f00d2e90 0x7fb1f00d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.451+0000 7fb1f4b63640 1 --2- 172.21.15.19:0/2813326939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f00b9770 0x7fb1f00d03c0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.451+0000 7fb1f4b63640 1 --2- 172.21.15.19:0/2813326939 >> v2:172.21.15.112:3300/0 conn(0x7fb1f015dd30 0x7fb1f00b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.451+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2813326939 >> 172.21.15.19:0/2813326939 conn(0x7fb1f0061c30 msgr2=0x7fb1f006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.451+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2813326939 shutdown_connections 2024-04-25T10:00:58.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.451+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2813326939 wait complete. 2024-04-25T10:00:58.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.452+0000 7fb1f4b63640 1 Processor -- start 2024-04-25T10:00:58.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.452+0000 7fb1f4b63640 1 -- start start 2024-04-25T10:00:58.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.453+0000 7fb1f4b63640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1f00b9770 0x7fb1f01601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.453+0000 7fb1f4b63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f00d2e90 0x7fb1f0160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.453+0000 7fb1f4b63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f015dd30 0x7fb1f0159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.453+0000 7fb1f4b63640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00d1b90 con 0x7fb1f015dd30 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.453+0000 7fb1f4b63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00d1a10 con 0x7fb1f00d2e90 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.453+0000 7fb1f4b63640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00b5430 con 0x7fb1f00b9770 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.453+0000 7fb1eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f015dd30 0x7fb1f0159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.453+0000 7fb1eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f015dd30 0x7fb1f0159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50574/0 (socket says 172.21.15.19:50574) 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.453+0000 7fb1eed9d640 1 -- 172.21.15.19:0/2215474901 learned_addr learned my addr 172.21.15.19:0/2215474901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.453+0000 7fb1ee59c640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1f00b9770 0x7fb1f01601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.453+0000 7fb1edd9b640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f00d2e90 0x7fb1f0160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.454+0000 7fb1eed9d640 1 -- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1f00b9770 msgr2=0x7fb1f01601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.454+0000 7fb1eed9d640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1f00b9770 0x7fb1f01601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.454+0000 7fb1eed9d640 1 -- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f00d2e90 msgr2=0x7fb1f0160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.454+0000 7fb1eed9d640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f00d2e90 0x7fb1f0160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.454+0000 7fb1eed9d640 1 -- 172.21.15.19:0/2215474901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f00c8e30 con 0x7fb1f015dd30 2024-04-25T10:00:58.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.454+0000 7fb1eed9d640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f015dd30 0x7fb1f0159480 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0007e40 tx=0x7fb1e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.454+0000 7fb1deffd640 1 -- 172.21.15.19:0/2215474901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0014620 con 0x7fb1f015dd30 2024-04-25T10:00:58.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.454+0000 7fb1deffd640 1 -- 172.21.15.19:0/2215474901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e0019070 con 0x7fb1f015dd30 2024-04-25T10:00:58.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.454+0000 7fb1deffd640 1 -- 172.21.15.19:0/2215474901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0014a20 con 0x7fb1f015dd30 2024-04-25T10:00:58.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.454+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2215474901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f00c8070 con 0x7fb1f015dd30 2024-04-25T10:00:58.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.454+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2215474901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f00c8630 con 0x7fb1f015dd30 2024-04-25T10:00:58.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.455+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2215474901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f0073990 con 0x7fb1f015dd30 2024-04-25T10:00:58.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.456+0000 7fb1deffd640 1 -- 172.21.15.19:0/2215474901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1e0014bc0 con 0x7fb1f015dd30 2024-04-25T10:00:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.456+0000 7fb1deffd640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1d006b4d0 0x7fb1d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.456+0000 7fb1deffd640 1 -- 172.21.15.19:0/2215474901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb1e008ab70 con 0x7fb1f015dd30 2024-04-25T10:00:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.456+0000 7fb1ee59c640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1d006b4d0 0x7fb1d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.457+0000 7fb1ee59c640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1d006b4d0 0x7fb1d006d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00ce1c0 tx=0x7fb1e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.460+0000 7fb1deffd640 1 -- 172.21.15.19:0/2215474901 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1e0059210 con 0x7fb1f015dd30 2024-04-25T10:00:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.680+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2215474901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1f00d4630 con 0x7fb1f015dd30 2024-04-25T10:00:58.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.680+0000 7fb1deffd640 1 -- 172.21.15.19:0/2215474901 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb1e005d2b0 con 0x7fb1f015dd30 2024-04-25T10:00:58.685 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:58.686 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:00:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:00:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1d006b4d0 msgr2=0x7fb1d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1d006b4d0 0x7fb1d006d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00ce1c0 tx=0x7fb1e8008040 comp rx=0 tx=0).stop 2024-04-25T10:00:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2215474901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f015dd30 msgr2=0x7fb1f0159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f015dd30 0x7fb1f0159480 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0007e40 tx=0x7fb1e000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2215474901 shutdown_connections 2024-04-25T10:00:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1d006b4d0 0x7fb1d006d990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f015dd30 0x7fb1f0159480 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f00d2e90 0x7fb1f0160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 --2- 172.21.15.19:0/2215474901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1f00b9770 0x7fb1f01601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2215474901 >> 172.21.15.19:0/2215474901 conn(0x7fb1f0061c30 msgr2=0x7fb1f00b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2215474901 shutdown_connections 2024-04-25T10:00:58.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.682+0000 7fb1f4b63640 1 -- 172.21.15.19:0/2215474901 wait complete. 2024-04-25T10:00:58.699 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:00:58.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.794+0000 7fc8c90ac640 1 Processor -- start 2024-04-25T10:00:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.794+0000 7fc8c90ac640 1 -- start start 2024-04-25T10:00:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.794+0000 7fc8c90ac640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc8c40d0aa0 0x7fc8c40b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.794+0000 7fc8c90ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8c406fea0 0x7fc8c40b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.794+0000 7fc8c90ac640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c40b3710 con 0x7fc8c415a790 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.794+0000 7fc8c90ac640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c40cf0e0 con 0x7fc8c40d0aa0 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.794+0000 7fc8c90ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c40cf3c0 con 0x7fc8c406fea0 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.795+0000 7fc8bbfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc8c40d0aa0 0x7fc8c40b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.795+0000 7fc8c359e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8c406fea0 0x7fc8c40b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.795+0000 7fc8c2d9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc8c415a790 0x7fc8c40d0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:53450/0 (socket says 172.21.15.19:53450) 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.795+0000 7fc8c2d9d640 1 -- 172.21.15.19:0/1588751025 learned_addr learned my addr 172.21.15.19:0/1588751025 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.795+0000 7fc8c259c640 1 -- 172.21.15.19:0/1588751025 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782353634 0 0) 0x7fc8c40b3710 con 0x7fc8c415a790 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.795+0000 7fc8c259c640 1 -- 172.21.15.19:0/1588751025 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8b0005730 con 0x7fc8c415a790 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.795+0000 7fc8c359e640 1 -- 172.21.15.19:0/1588751025 >> v2:172.21.15.112:3300/0 conn(0x7fc8c40d0aa0 msgr2=0x7fc8c40b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.795+0000 7fc8c359e640 1 --2- 172.21.15.19:0/1588751025 >> v2:172.21.15.112:3300/0 conn(0x7fc8c40d0aa0 0x7fc8c40b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.795+0000 7fc8c359e640 1 -- 172.21.15.19:0/1588751025 >> v1:172.21.15.112:6789/0 conn(0x7fc8c415a790 legacy=0x7fc8c40d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.795+0000 7fc8c359e640 1 -- 172.21.15.19:0/1588751025 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c40cf6a0 con 0x7fc8c406fea0 2024-04-25T10:00:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.796+0000 7fc8c359e640 1 --2- 172.21.15.19:0/1588751025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8c406fea0 0x7fc8c40b43e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009d90 tx=0x7fc8bc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=726559296dedc015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.796+0000 7fc8c259c640 1 -- 172.21.15.19:0/1588751025 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc007740 con 0x7fc8c406fea0 2024-04-25T10:00:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.796+0000 7fc8c259c640 1 -- 172.21.15.19:0/1588751025 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8bc0078e0 con 0x7fc8c406fea0 2024-04-25T10:00:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.796+0000 7fc8c259c640 1 -- 172.21.15.19:0/1588751025 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc007c90 con 0x7fc8c406fea0 2024-04-25T10:00:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.797+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1588751025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8c406fea0 msgr2=0x7fc8c40b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.797+0000 7fc8c90ac640 1 --2- 172.21.15.19:0/1588751025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8c406fea0 0x7fc8c40b43e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009d90 tx=0x7fc8bc02a660 comp rx=0 tx=0).stop 2024-04-25T10:00:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.797+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1588751025 shutdown_connections 2024-04-25T10:00:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.797+0000 7fc8c90ac640 1 --2- 172.21.15.19:0/1588751025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8c406fea0 0x7fc8c40b43e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.797+0000 7fc8c90ac640 1 --2- 172.21.15.19:0/1588751025 >> v2:172.21.15.112:3300/0 conn(0x7fc8c40d0aa0 0x7fc8c40b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.797+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1588751025 >> 172.21.15.19:0/1588751025 conn(0x7fc8c40b2370 msgr2=0x7fc8c40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.797+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1588751025 shutdown_connections 2024-04-25T10:00:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.797+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1588751025 wait complete. 2024-04-25T10:00:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.798+0000 7fc8c90ac640 1 Processor -- start 2024-04-25T10:00:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.798+0000 7fc8c90ac640 1 -- start start 2024-04-25T10:00:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c90ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8c406fea0 0x7fc8c40aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c90ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c40d0aa0 0x7fc8c4156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c90ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8c415a790 0x7fc8c415cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c90ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c415ab70 con 0x7fc8c406fea0 2024-04-25T10:00:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c90ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c40cf980 con 0x7fc8c40d0aa0 2024-04-25T10:00:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c90ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c415acf0 con 0x7fc8c415a790 2024-04-25T10:00:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8c406fea0 0x7fc8c40aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8c406fea0 0x7fc8c40aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50580/0 (socket says 172.21.15.19:50580) 2024-04-25T10:00:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c2d9d640 1 -- 172.21.15.19:0/1238584971 learned_addr learned my addr 172.21.15.19:0/1238584971 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:00:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c359e640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8c415a790 0x7fc8c415cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8bbfff640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c40d0aa0 0x7fc8c4156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c2d9d640 1 -- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8c415a790 msgr2=0x7fc8c415cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c2d9d640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8c415a790 0x7fc8c415cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c2d9d640 1 -- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c40d0aa0 msgr2=0x7fc8c4156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c2d9d640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c40d0aa0 0x7fc8c4156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c2d9d640 1 -- 172.21.15.19:0/1238584971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c415bf60 con 0x7fc8c406fea0 2024-04-25T10:00:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8bbfff640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c40d0aa0 0x7fc8c4156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.799+0000 7fc8c2d9d640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8c406fea0 0x7fc8c40aec50 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400c430 tx=0x7fc8b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.800+0000 7fc8bb7fe640 1 -- 172.21.15.19:0/1238584971 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4017070 con 0x7fc8c406fea0 2024-04-25T10:00:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.800+0000 7fc8bb7fe640 1 -- 172.21.15.19:0/1238584971 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b4005ce0 con 0x7fc8c406fea0 2024-04-25T10:00:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.800+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1238584971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c415c200 con 0x7fc8c406fea0 2024-04-25T10:00:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.800+0000 7fc8bb7fe640 1 -- 172.21.15.19:0/1238584971 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40072f0 con 0x7fc8c406fea0 2024-04-25T10:00:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.800+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1238584971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c415c7c0 con 0x7fc8c406fea0 2024-04-25T10:00:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.801+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1238584971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c4073990 con 0x7fc8c406fea0 2024-04-25T10:00:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.801+0000 7fc8bb7fe640 1 -- 172.21.15.19:0/1238584971 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8b401a070 con 0x7fc8c406fea0 2024-04-25T10:00:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.802+0000 7fc8bb7fe640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8a406b4d0 0x7fc8a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.802+0000 7fc8bb7fe640 1 -- 172.21.15.19:0/1238584971 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8b4089820 con 0x7fc8c406fea0 2024-04-25T10:00:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.802+0000 7fc8bbfff640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8a406b4d0 0x7fc8a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.802+0000 7fc8bbfff640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8a406b4d0 0x7fc8a406d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40ccb30 tx=0x7fc8ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:58.806+0000 7fc8bb7fe640 1 -- 172.21.15.19:0/1238584971 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8b4057e30 con 0x7fc8c406fea0 2024-04-25T10:00:59.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.056+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1238584971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc8c406d330 con 0x7fc8c406fea0 2024-04-25T10:00:59.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.056+0000 7fc8bb7fe640 1 -- 172.21.15.19:0/1238584971 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fc8b405bed0 con 0x7fc8c406fea0 2024-04-25T10:00:59.061 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:00:59.062 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":496,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2562392064,"bytes_avail":770531721216,"bytes_total":773094113280,"write_bytes_sec":43359460,"read_op_per_sec":0,"write_op_per_sec":42},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:00:55.018290+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:00:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8a406b4d0 msgr2=0x7fc8a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8a406b4d0 0x7fc8a406d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40ccb30 tx=0x7fc8ac008040 comp rx=0 tx=0).stop 2024-04-25T10:00:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1238584971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8c406fea0 msgr2=0x7fc8c40aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8c406fea0 0x7fc8c40aec50 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400c430 tx=0x7fc8b400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1238584971 shutdown_connections 2024-04-25T10:00:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc8a406b4d0 0x7fc8a406d990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8c415a790 0x7fc8c415cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c40d0aa0 0x7fc8c4156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 --2- 172.21.15.19:0/1238584971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8c406fea0 0x7fc8c40aec50 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1238584971 >> 172.21.15.19:0/1238584971 conn(0x7fc8c40b2370 msgr2=0x7fc8c40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1238584971 shutdown_connections 2024-04-25T10:00:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:00:59.058+0000 7fc8c90ac640 1 -- 172.21.15.19:0/1238584971 wait complete. 2024-04-25T10:00:59.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T10:01:00.076 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:00.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.184+0000 7f4d3d92b640 1 Processor -- start 2024-04-25T10:01:00.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.184+0000 7f4d3d92b640 1 -- start start 2024-04-25T10:01:00.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.185+0000 7f4d3d92b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815dd20 0x7f4d380b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.185+0000 7f4d3d92b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4d380d11b0 0x7f4d380d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.185+0000 7f4d3d92b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3815eac0 con 0x7f4d3815e100 2024-04-25T10:01:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.185+0000 7f4d3d92b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d380d1c00 con 0x7f4d380d11b0 2024-04-25T10:01:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.185+0000 7f4d3d92b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815eda0 con 0x7f4d3815dd20 2024-04-25T10:01:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.186+0000 7f4d36ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815dd20 0x7f4d380b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.186+0000 7f4d377fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4d380d11b0 0x7f4d380d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.186+0000 7f4d36ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815dd20 0x7f4d380b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:50182/0 (socket says 172.21.15.19:50182) 2024-04-25T10:01:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.186+0000 7f4d36ffd640 1 -- 172.21.15.19:0/1851825047 learned_addr learned my addr 172.21.15.19:0/1851825047 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.186+0000 7f4d367fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4d3815e100 0x7f4d380d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:53460/0 (socket says 172.21.15.19:53460) 2024-04-25T10:01:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.186+0000 7f4d36ffd640 1 -- 172.21.15.19:0/1851825047 >> v2:172.21.15.112:3300/0 conn(0x7f4d380d11b0 msgr2=0x7f4d380d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.186+0000 7f4d36ffd640 1 --2- 172.21.15.19:0/1851825047 >> v2:172.21.15.112:3300/0 conn(0x7f4d380d11b0 0x7f4d380d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.186+0000 7f4d36ffd640 1 -- 172.21.15.19:0/1851825047 >> v1:172.21.15.112:6789/0 conn(0x7f4d3815e100 legacy=0x7f4d380d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.186+0000 7f4d36ffd640 1 -- 172.21.15.19:0/1851825047 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3815f080 con 0x7f4d3815dd20 2024-04-25T10:01:00.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.187+0000 7f4d36ffd640 1 --2- 172.21.15.19:0/1851825047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815dd20 0x7f4d380b96d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009e50 tx=0x7f4d2802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ec1f77405c4ab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.187+0000 7f4d35ffb640 1 -- 172.21.15.19:0/1851825047 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28036070 con 0x7f4d3815dd20 2024-04-25T10:01:00.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.187+0000 7f4d35ffb640 1 -- 172.21.15.19:0/1851825047 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d28031600 con 0x7f4d3815dd20 2024-04-25T10:01:00.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.187+0000 7f4d35ffb640 1 -- 172.21.15.19:0/1851825047 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d280319d0 con 0x7f4d3815dd20 2024-04-25T10:01:00.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.187+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1851825047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815dd20 msgr2=0x7f4d380b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.188+0000 7f4d3d92b640 1 --2- 172.21.15.19:0/1851825047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815dd20 0x7f4d380b96d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009e50 tx=0x7f4d2802a680 comp rx=0 tx=0).stop 2024-04-25T10:01:00.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.188+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1851825047 shutdown_connections 2024-04-25T10:01:00.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.188+0000 7f4d3d92b640 1 --2- 172.21.15.19:0/1851825047 >> v2:172.21.15.112:3300/0 conn(0x7f4d380d11b0 0x7f4d380d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.188+0000 7f4d3d92b640 1 --2- 172.21.15.19:0/1851825047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815dd20 0x7f4d380b96d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.188+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1851825047 >> 172.21.15.19:0/1851825047 conn(0x7f4d38061a20 msgr2=0x7f4d38061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.188+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1851825047 shutdown_connections 2024-04-25T10:01:00.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.188+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1851825047 wait complete. 2024-04-25T10:01:00.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.189+0000 7f4d3d92b640 1 Processor -- start 2024-04-25T10:01:00.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.189+0000 7f4d3d92b640 1 -- start start 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d3d92b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d380d11b0 0x7f4d380ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d3d92b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3815dd20 0x7f4d380cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d3d92b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815e100 0x7f4d381f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d3d92b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815e6e0 con 0x7f4d3815dd20 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d3d92b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815e560 con 0x7f4d380d11b0 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d3d92b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815e860 con 0x7f4d3815e100 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3815dd20 0x7f4d380cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d367fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3815dd20 0x7f4d380cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50586/0 (socket says 172.21.15.19:50586) 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d36ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d380d11b0 0x7f4d380ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d367fc640 1 -- 172.21.15.19:0/1666569515 learned_addr learned my addr 172.21.15.19:0/1666569515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d377fe640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815e100 0x7f4d381f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d36ffd640 1 -- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815e100 msgr2=0x7f4d381f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d36ffd640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815e100 0x7f4d381f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d36ffd640 1 -- 172.21.15.19:0/1666569515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3815dd20 msgr2=0x7f4d380cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d36ffd640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3815dd20 0x7f4d380cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.190+0000 7f4d36ffd640 1 -- 172.21.15.19:0/1666569515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d38158210 con 0x7f4d380d11b0 2024-04-25T10:01:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.191+0000 7f4d36ffd640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d380d11b0 0x7f4d380ca000 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4d28030000 tx=0x7f4d28004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.191+0000 7f4d2ffff640 1 -- 172.21.15.19:0/1666569515 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28036050 con 0x7f4d380d11b0 2024-04-25T10:01:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.191+0000 7f4d2ffff640 1 -- 172.21.15.19:0/1666569515 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d28037040 con 0x7f4d380d11b0 2024-04-25T10:01:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.191+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1666569515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d380cb5f0 con 0x7f4d380d11b0 2024-04-25T10:01:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.191+0000 7f4d2ffff640 1 -- 172.21.15.19:0/1666569515 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28009510 con 0x7f4d380d11b0 2024-04-25T10:01:00.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.191+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1666569515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d380cbbb0 con 0x7f4d380d11b0 2024-04-25T10:01:00.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.192+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1666569515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d38073990 con 0x7f4d380d11b0 2024-04-25T10:01:00.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.193+0000 7f4d2ffff640 1 -- 172.21.15.19:0/1666569515 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d28044020 con 0x7f4d380d11b0 2024-04-25T10:01:00.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.193+0000 7f4d2ffff640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d1806b4d0 0x7f4d1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.193+0000 7f4d2ffff640 1 -- 172.21.15.19:0/1666569515 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d280aa650 con 0x7f4d380d11b0 2024-04-25T10:01:00.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.193+0000 7f4d367fc640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d1806b4d0 0x7f4d1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.194+0000 7f4d367fc640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d1806b4d0 0x7f4d1806d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4d3815c9e0 tx=0x7f4d20007f10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.198+0000 7f4d2ffff640 1 -- 172.21.15.19:0/1666569515 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d2804b020 con 0x7f4d380d11b0 2024-04-25T10:01:00.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.415+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1666569515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d380b9010 con 0x7f4d380d11b0 2024-04-25T10:01:00.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.416+0000 7f4d2ffff640 1 -- 172.21.15.19:0/1666569515 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4d28078ce0 con 0x7f4d380d11b0 2024-04-25T10:01:00.422 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:00.422 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:01:00.422 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d1806b4d0 msgr2=0x7f4d1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d1806b4d0 0x7f4d1806d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4d3815c9e0 tx=0x7f4d20007f10 comp rx=0 tx=0).stop 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d380d11b0 msgr2=0x7f4d380ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d380d11b0 0x7f4d380ca000 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4d28030000 tx=0x7f4d28004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1666569515 shutdown_connections 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4d1806b4d0 0x7f4d1806d990 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d3815e100 0x7f4d381f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3815dd20 0x7f4d380cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 --2- 172.21.15.19:0/1666569515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d380d11b0 0x7f4d380ca000 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1666569515 >> 172.21.15.19:0/1666569515 conn(0x7f4d38061a20 msgr2=0x7f4d380d3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1666569515 shutdown_connections 2024-04-25T10:01:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.418+0000 7f4d3d92b640 1 -- 172.21.15.19:0/1666569515 wait complete. 2024-04-25T10:01:00.435 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:01:00.435 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.529+0000 7f6609f42640 1 Processor -- start 2024-04-25T10:01:00.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.529+0000 7f6609f42640 1 -- start start 2024-04-25T10:01:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.530+0000 7f6609f42640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66040b6cb0 0x7f66040b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.530+0000 7f6609f42640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66040b5a60 0x7f66040d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.530+0000 7f6609f42640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66040ceee0 con 0x7f660415db70 2024-04-25T10:01:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.530+0000 7f6609f42640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66040d1d10 con 0x7f66040b6cb0 2024-04-25T10:01:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.530+0000 7f6609f42640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66040d1ff0 con 0x7f66040b5a60 2024-04-25T10:01:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.530+0000 7f66037fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f660415db70 0x7f6604162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44096/0 (socket says 172.21.15.19:44096) 2024-04-25T10:01:00.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.530+0000 7f66037fe640 1 -- 172.21.15.19:0/1148551691 learned_addr learned my addr 172.21.15.19:0/1148551691 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.530+0000 7f6602ffd640 1 --2- 172.21.15.19:0/1148551691 >> v2:172.21.15.112:3300/0 conn(0x7f66040b6cb0 0x7f66040b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.530+0000 7f6603fff640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66040b5a60 0x7f66040d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.530+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410854171 0 0) 0x7f66040ceee0 con 0x7f660415db70 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.530+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65ec003520 con 0x7f660415db70 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2100924721 0 0) 0x7f65ec003520 con 0x7f660415db70 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66040b5a60 msgr2=0x7f66040d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66040b5a60 0x7f66040d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 >> v2:172.21.15.112:3300/0 conn(0x7f66040b6cb0 msgr2=0x7f66040b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 --2- 172.21.15.19:0/1148551691 >> v2:172.21.15.112:3300/0 conn(0x7f66040b6cb0 0x7f66040b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040d22d0 con 0x7f660415db70 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f65f4001670 con 0x7f660415db70 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 >> v1:172.21.15.19:6789/0 conn(0x7f660415db70 legacy=0x7f6604162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66040b5a60 0x7f65ec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec003f30 0x7f65ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec004d40 0x7f65ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040d1d10 con 0x7f65ec003f30 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040d1ff0 con 0x7f65ec004d40 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec00b9c0 con 0x7f66040b5a60 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f6602ffd640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec003f30 0x7f65ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f66037fe640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66040b5a60 0x7f65ec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f6603fff640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec004d40 0x7f65ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f6602ffd640 1 -- 172.21.15.19:0/1148551691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66040b5a60 msgr2=0x7f65ec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f6602ffd640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66040b5a60 0x7f65ec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f6602ffd640 1 -- 172.21.15.19:0/1148551691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec004d40 msgr2=0x7f65ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f6602ffd640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65ec004d40 0x7f65ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.531+0000 7f6602ffd640 1 -- 172.21.15.19:0/1148551691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec003520 con 0x7f65ec003f30 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.532+0000 7f6602ffd640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec003f30 0x7f65ec004800 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009a90 tx=0x7f65ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d45ac099d46dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.532+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8036070 con 0x7f65ec003f30 2024-04-25T10:01:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.532+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f8031400 con 0x7f65ec003f30 2024-04-25T10:01:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.532+0000 7f66027fc640 1 -- 172.21.15.19:0/1148551691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8009690 con 0x7f65ec003f30 2024-04-25T10:01:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.532+0000 7f6609f42640 1 -- 172.21.15.19:0/1148551691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec003f30 msgr2=0x7f65ec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.533+0000 7f6609f42640 1 --2- 172.21.15.19:0/1148551691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65ec003f30 0x7f65ec004800 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009a90 tx=0x7f65ec005830 comp rx=0 tx=0).stop 2024-04-25T10:01:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.533+0000 7f6603fff640 1 -- 172.21.15.19:0/1148551691 reap_dead start 2024-04-25T10:01:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.533+0000 7f6609f42640 1 -- 172.21.15.19:0/1148551691 shutdown_connections 2024-04-25T10:01:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.533+0000 7f6609f42640 1 -- 172.21.15.19:0/1148551691 >> 172.21.15.19:0/1148551691 conn(0x7f66040d0520 msgr2=0x7f66040d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.533+0000 7f6609f42640 1 -- 172.21.15.19:0/1148551691 shutdown_connections 2024-04-25T10:01:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.533+0000 7f6609f42640 1 -- 172.21.15.19:0/1148551691 wait complete. 2024-04-25T10:01:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.534+0000 7f6609f42640 1 Processor -- start 2024-04-25T10:01:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.534+0000 7f6609f42640 1 -- start start 2024-04-25T10:01:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6609f42640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65ec003f30 0x7f660414c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6609f42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66040cab70 0x7f66040cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6609f42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66040d26b0 0x7f66040d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6609f42640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040cf1c0 con 0x7f66040cab70 2024-04-25T10:01:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6609f42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040ce530 con 0x7f66040d26b0 2024-04-25T10:01:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6609f42640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66040cf340 con 0x7f65ec003f30 2024-04-25T10:01:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6602ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66040cab70 0x7f66040cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6602ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66040cab70 0x7f66040cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50600/0 (socket says 172.21.15.19:50600) 2024-04-25T10:01:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6602ffd640 1 -- 172.21.15.19:0/2200085407 learned_addr learned my addr 172.21.15.19:0/2200085407 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6603fff640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66040d26b0 0x7f66040d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6602ffd640 1 -- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65ec003f30 msgr2=0x7f660414c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f66037fe640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65ec003f30 0x7f660414c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6602ffd640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65ec003f30 0x7f660414c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6602ffd640 1 -- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66040d26b0 msgr2=0x7f66040d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6602ffd640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66040d26b0 0x7f66040d2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6602ffd640 1 -- 172.21.15.19:0/2200085407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040cb630 con 0x7f66040cab70 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6602ffd640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66040cab70 0x7f66040cafd0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f65f8034040 tx=0x7f65f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f65ebfff640 1 -- 172.21.15.19:0/2200085407 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8040020 con 0x7f66040cab70 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f65ebfff640 1 -- 172.21.15.19:0/2200085407 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f8003c60 con 0x7f66040cab70 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6609f42640 1 -- 172.21.15.19:0/2200085407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66040cb910 con 0x7f66040cab70 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f65ebfff640 1 -- 172.21.15.19:0/2200085407 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8036040 con 0x7f66040cab70 2024-04-25T10:01:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.535+0000 7f6609f42640 1 -- 172.21.15.19:0/2200085407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66040cbed0 con 0x7f66040cab70 2024-04-25T10:01:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.537+0000 7f6609f42640 1 -- 172.21.15.19:0/2200085407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6604073990 con 0x7f66040cab70 2024-04-25T10:01:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.537+0000 7f65ebfff640 1 -- 172.21.15.19:0/2200085407 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65f8043020 con 0x7f66040cab70 2024-04-25T10:01:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.537+0000 7f65ebfff640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f65f006b500 0x7f65f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.537+0000 7f65ebfff640 1 -- 172.21.15.19:0/2200085407 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65f80aa8a0 con 0x7f66040cab70 2024-04-25T10:01:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.537+0000 7f66037fe640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f65f006b500 0x7f65f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.538+0000 7f66037fe640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f65f006b500 0x7f65f006d9c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f65f40014f0 tx=0x7f65f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.541+0000 7f65ebfff640 1 -- 172.21.15.19:0/2200085407 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65f804a030 con 0x7f66040cab70 2024-04-25T10:01:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.796+0000 7f6609f42640 1 -- 172.21.15.19:0/2200085407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f66040d1820 con 0x7f66040cab70 2024-04-25T10:01:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.796+0000 7f65ebfff640 1 -- 172.21.15.19:0/2200085407 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f65f80044e0 con 0x7f66040cab70 2024-04-25T10:01:00.802 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:00.802 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":498,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076566130,"bytes_used":2562400256,"bytes_avail":770531713024,"bytes_total":773094113280,"write_bytes_sec":43360477,"read_op_per_sec":0,"write_op_per_sec":43},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:00:55.018290+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:01:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 -- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f65f006b500 msgr2=0x7f65f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f65f006b500 0x7f65f006d9c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f65f40014f0 tx=0x7f65f4009040 comp rx=0 tx=0).stop 2024-04-25T10:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 -- 172.21.15.19:0/2200085407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66040cab70 msgr2=0x7f66040cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66040cab70 0x7f66040cafd0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f65f8034040 tx=0x7f65f8004730 comp rx=0 tx=0).stop 2024-04-25T10:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 -- 172.21.15.19:0/2200085407 shutdown_connections 2024-04-25T10:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f65f006b500 0x7f65f006d9c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66040d26b0 0x7f66040d2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66040cab70 0x7f66040cafd0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 --2- 172.21.15.19:0/2200085407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f65ec003f30 0x7f660414c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 -- 172.21.15.19:0/2200085407 >> 172.21.15.19:0/2200085407 conn(0x7f66040d0520 msgr2=0x7f66040d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 -- 172.21.15.19:0/2200085407 shutdown_connections 2024-04-25T10:01:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:00.798+0000 7f6609f42640 1 -- 172.21.15.19:0/2200085407 wait complete. 2024-04-25T10:01:00.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T10:01:01.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:01.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.925+0000 7f837515c640 1 Processor -- start 2024-04-25T10:01:01.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.925+0000 7f837515c640 1 -- start start 2024-04-25T10:01:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.925+0000 7f837515c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f837015dc20 0x7f83701616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.925+0000 7f837515c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f83700cedf0 0x7f83700d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.925+0000 7f837515c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837015e000 con 0x7f83700b93a0 2024-04-25T10:01:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.925+0000 7f837515c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83700b6bb0 con 0x7f83700cedf0 2024-04-25T10:01:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.925+0000 7f837515c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83700b6030 con 0x7f837015dc20 2024-04-25T10:01:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.926+0000 7f836f59e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f83700cedf0 0x7f83700d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.926+0000 7f836f59e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f83700cedf0 0x7f83700d2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50602/0 (socket says 172.21.15.19:50602) 2024-04-25T10:01:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.926+0000 7f836f59e640 1 -- 172.21.15.19:0/3670567341 learned_addr learned my addr 172.21.15.19:0/3670567341 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.926+0000 7f836ed9d640 1 --2- 172.21.15.19:0/3670567341 >> v2:172.21.15.112:3300/0 conn(0x7f837015dc20 0x7f83701616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.926+0000 7f836f59e640 1 -- 172.21.15.19:0/3670567341 >> v2:172.21.15.112:3300/0 conn(0x7f837015dc20 msgr2=0x7f83701616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.926+0000 7f836f59e640 1 --2- 172.21.15.19:0/3670567341 >> v2:172.21.15.112:3300/0 conn(0x7f837015dc20 0x7f83701616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.926+0000 7f836f59e640 1 -- 172.21.15.19:0/3670567341 >> v1:172.21.15.112:6789/0 conn(0x7f83700b93a0 legacy=0x7f83700ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.926+0000 7f836f59e640 1 -- 172.21.15.19:0/3670567341 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8370161de0 con 0x7f83700cedf0 2024-04-25T10:01:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.927+0000 7f836f59e640 1 --2- 172.21.15.19:0/3670567341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83700cedf0 0x7f83700d2510 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f837008be60 tx=0x7f836802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63639f81b56d0e20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.927+0000 7f836e59c640 1 -- 172.21.15.19:0/3670567341 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368036070 con 0x7f83700cedf0 2024-04-25T10:01:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.927+0000 7f836e59c640 1 -- 172.21.15.19:0/3670567341 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8368031600 con 0x7f83700cedf0 2024-04-25T10:01:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.927+0000 7f836e59c640 1 -- 172.21.15.19:0/3670567341 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83680319d0 con 0x7f83700cedf0 2024-04-25T10:01:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.927+0000 7f837515c640 1 -- 172.21.15.19:0/3670567341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83700cedf0 msgr2=0x7f83700d2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.927+0000 7f837515c640 1 --2- 172.21.15.19:0/3670567341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83700cedf0 0x7f83700d2510 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f837008be60 tx=0x7f836802a730 comp rx=0 tx=0).stop 2024-04-25T10:01:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.928+0000 7f837515c640 1 -- 172.21.15.19:0/3670567341 shutdown_connections 2024-04-25T10:01:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.928+0000 7f837515c640 1 --2- 172.21.15.19:0/3670567341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83700cedf0 0x7f83700d2510 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.928+0000 7f837515c640 1 --2- 172.21.15.19:0/3670567341 >> v2:172.21.15.112:3300/0 conn(0x7f837015dc20 0x7f83701616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.928+0000 7f837515c640 1 -- 172.21.15.19:0/3670567341 >> 172.21.15.19:0/3670567341 conn(0x7f83700d0490 msgr2=0x7f83700d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.928+0000 7f837515c640 1 -- 172.21.15.19:0/3670567341 shutdown_connections 2024-04-25T10:01:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.928+0000 7f837515c640 1 -- 172.21.15.19:0/3670567341 wait complete. 2024-04-25T10:01:01.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.929+0000 7f837515c640 1 Processor -- start 2024-04-25T10:01:01.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.929+0000 7f837515c640 1 -- start start 2024-04-25T10:01:01.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.930+0000 7f837515c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83700b93a0 0x7f837015e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.930+0000 7f837515c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83700cedf0 0x7f837014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.930+0000 7f837515c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f837015dc20 0x7f83701609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.930+0000 7f837515c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83700d2a50 con 0x7f837015dc20 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.930+0000 7f837515c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83701623a0 con 0x7f83700cedf0 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.930+0000 7f837515c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83700d2bd0 con 0x7f83700b93a0 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.930+0000 7f836f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f837015dc20 0x7f83701609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.930+0000 7f836f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f837015dc20 0x7f83701609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50610/0 (socket says 172.21.15.19:50610) 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.930+0000 7f836f59e640 1 -- 172.21.15.19:0/2722166455 learned_addr learned my addr 172.21.15.19:0/2722166455 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.930+0000 7f836ed9d640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83700b93a0 0x7f837015e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f8367fff640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83700cedf0 0x7f837014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f836f59e640 1 -- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83700b93a0 msgr2=0x7f837015e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f836f59e640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83700b93a0 0x7f837015e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f836f59e640 1 -- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83700cedf0 msgr2=0x7f837014ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f836f59e640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83700cedf0 0x7f837014ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f836f59e640 1 -- 172.21.15.19:0/2722166455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837015e950 con 0x7f837015dc20 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f836f59e640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f837015dc20 0x7f83701609b0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8368009f40 tx=0x7f8368004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f83677fe640 1 -- 172.21.15.19:0/2722166455 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368036050 con 0x7f837015dc20 2024-04-25T10:01:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f83677fe640 1 -- 172.21.15.19:0/2722166455 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8368042070 con 0x7f837015dc20 2024-04-25T10:01:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f837515c640 1 -- 172.21.15.19:0/2722166455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837015ec30 con 0x7f837015dc20 2024-04-25T10:01:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f83677fe640 1 -- 172.21.15.19:0/2722166455 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368009510 con 0x7f837015dc20 2024-04-25T10:01:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.931+0000 7f837515c640 1 -- 172.21.15.19:0/2722166455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837015f1f0 con 0x7f837015dc20 2024-04-25T10:01:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.932+0000 7f837515c640 1 -- 172.21.15.19:0/2722166455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8370073990 con 0x7f837015dc20 2024-04-25T10:01:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.933+0000 7f83677fe640 1 -- 172.21.15.19:0/2722166455 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8368007bf0 con 0x7f837015dc20 2024-04-25T10:01:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.933+0000 7f83677fe640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f835006b4d0 0x7f835006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.933+0000 7f83677fe640 1 -- 172.21.15.19:0/2722166455 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83680ab090 con 0x7f837015dc20 2024-04-25T10:01:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.933+0000 7f836ed9d640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f835006b4d0 0x7f835006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.934+0000 7f836ed9d640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f835006b4d0 0x7f835006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8360009a30 tx=0x7f8360008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:01.937+0000 7f83677fe640 1 -- 172.21.15.19:0/2722166455 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8368045360 con 0x7f837015dc20 2024-04-25T10:01:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.154+0000 7f837515c640 1 -- 172.21.15.19:0/2722166455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83700d1720 con 0x7f837015dc20 2024-04-25T10:01:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.155+0000 7f83677fe640 1 -- 172.21.15.19:0/2722166455 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f8368079e10 con 0x7f837015dc20 2024-04-25T10:01:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:02.160 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:01:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:01:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 -- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f835006b4d0 msgr2=0x7f835006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f835006b4d0 0x7f835006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8360009a30 tx=0x7f8360008040 comp rx=0 tx=0).stop 2024-04-25T10:01:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 -- 172.21.15.19:0/2722166455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f837015dc20 msgr2=0x7f83701609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f837015dc20 0x7f83701609b0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8368009f40 tx=0x7f8368004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 -- 172.21.15.19:0/2722166455 shutdown_connections 2024-04-25T10:01:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f835006b4d0 0x7f835006d990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f837015dc20 0x7f83701609b0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83700cedf0 0x7f837014ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 --2- 172.21.15.19:0/2722166455 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83700b93a0 0x7f837015e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 -- 172.21.15.19:0/2722166455 >> 172.21.15.19:0/2722166455 conn(0x7f83700d0490 msgr2=0x7f83700b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 -- 172.21.15.19:0/2722166455 shutdown_connections 2024-04-25T10:01:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.157+0000 7f837515c640 1 -- 172.21.15.19:0/2722166455 wait complete. 2024-04-25T10:01:02.173 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:01:02.173 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.268+0000 7f40b494c640 1 Processor -- start 2024-04-25T10:01:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.268+0000 7f40b494c640 1 -- start start 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.268+0000 7f40b494c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40b015dbe0 0x7f40b00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.268+0000 7f40b494c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b00b2cb0 0x7f40b00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.268+0000 7f40b494c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b00b97a0 con 0x7f40b00b92a0 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.268+0000 7f40b494c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b015dfc0 con 0x7f40b015dbe0 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.268+0000 7f40b494c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00d1a00 con 0x7f40b00b2cb0 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40aed9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f40b00b92a0 0x7f40b00b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44100/0 (socket says 172.21.15.19:44100) 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40aed9d640 1 -- 172.21.15.19:0/3608514435 learned_addr learned my addr 172.21.15.19:0/3608514435 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925310496 0 0) 0x7f40b00b97a0 con 0x7f40b00b92a0 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f409c003520 con 0x7f40b00b92a0 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40ae59c640 1 --2- 172.21.15.19:0/3608514435 >> v2:172.21.15.112:3300/0 conn(0x7f40b015dbe0 0x7f40b00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40a7fff640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b00b2cb0 0x7f40b00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2124616841 0 0) 0x7f409c003520 con 0x7f40b00b92a0 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b00b2cb0 msgr2=0x7f40b00d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b00b2cb0 0x7f40b00d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 >> v2:172.21.15.112:3300/0 conn(0x7f40b015dbe0 msgr2=0x7f40b00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 --2- 172.21.15.19:0/3608514435 >> v2:172.21.15.112:3300/0 conn(0x7f40b015dbe0 0x7f40b00d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00d1c50 con 0x7f40b00b92a0 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f40a80019f0 con 0x7f40b00b92a0 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 >> v1:172.21.15.19:6789/0 conn(0x7f40b00b92a0 legacy=0x7f40b00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f409c0039b0 0x7f409c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f409c004800 0x7f409c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b00b2cb0 0x7f409c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00d1a00 con 0x7f409c0039b0 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c00b9c0 con 0x7f409c004800 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c00bb40 con 0x7f40b00b2cb0 2024-04-25T10:01:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40ae59c640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f409c0039b0 0x7f409c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40a7fff640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f409c004800 0x7f409c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.269+0000 7f40aed9d640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b00b2cb0 0x7f409c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.270+0000 7f40ae59c640 1 -- 172.21.15.19:0/3608514435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b00b2cb0 msgr2=0x7f409c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.270+0000 7f40ae59c640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b00b2cb0 0x7f409c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.270+0000 7f40ae59c640 1 -- 172.21.15.19:0/3608514435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f409c004800 msgr2=0x7f409c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.270+0000 7f40ae59c640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f409c004800 0x7f409c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.270+0000 7f40ae59c640 1 -- 172.21.15.19:0/3608514435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c003520 con 0x7f409c0039b0 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.270+0000 7f40ae59c640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f409c0039b0 0x7f409c003dd0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009ac0 tx=0x7f409c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c97ce2572613382 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.270+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0036070 con 0x7f409c0039b0 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.270+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a0031400 con 0x7f409c0039b0 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.271+0000 7f40add9b640 1 -- 172.21.15.19:0/3608514435 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0009690 con 0x7f409c0039b0 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.271+0000 7f40b494c640 1 -- 172.21.15.19:0/3608514435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f409c0039b0 msgr2=0x7f409c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.271+0000 7f40b494c640 1 --2- 172.21.15.19:0/3608514435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f409c0039b0 0x7f409c003dd0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009ac0 tx=0x7f409c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.271+0000 7f40aed9d640 1 -- 172.21.15.19:0/3608514435 reap_dead start 2024-04-25T10:01:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.271+0000 7f40b494c640 1 -- 172.21.15.19:0/3608514435 shutdown_connections 2024-04-25T10:01:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.271+0000 7f40b494c640 1 -- 172.21.15.19:0/3608514435 >> 172.21.15.19:0/3608514435 conn(0x7f40b0061a20 msgr2=0x7f40b0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.271+0000 7f40b494c640 1 -- 172.21.15.19:0/3608514435 shutdown_connections 2024-04-25T10:01:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.271+0000 7f40b494c640 1 -- 172.21.15.19:0/3608514435 wait complete. 2024-04-25T10:01:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.272+0000 7f40b494c640 1 Processor -- start 2024-04-25T10:01:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.273+0000 7f40b494c640 1 -- start start 2024-04-25T10:01:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.273+0000 7f40b494c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f409c0039b0 0x7f40b01ed5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.273+0000 7f40b494c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b01f3f00 0x7f40b00b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.273+0000 7f40b494c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40b00b8a30 0x7f40b00b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.273+0000 7f40b494c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00b6ab0 con 0x7f40b00b8a30 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.273+0000 7f40b494c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00d2210 con 0x7f409c0039b0 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.273+0000 7f40b494c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00b6c30 con 0x7f40b01f3f00 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.273+0000 7f40aed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40b00b8a30 0x7f40b00b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.273+0000 7f40aed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40b00b8a30 0x7f40b00b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50640/0 (socket says 172.21.15.19:50640) 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.273+0000 7f40aed9d640 1 -- 172.21.15.19:0/3081095245 learned_addr learned my addr 172.21.15.19:0/3081095245 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40ae59c640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f409c0039b0 0x7f40b01ed5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40a7fff640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b01f3f00 0x7f40b00b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40aed9d640 1 -- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b01f3f00 msgr2=0x7f40b00b84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40aed9d640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b01f3f00 0x7f40b00b84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40aed9d640 1 -- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f409c0039b0 msgr2=0x7f40b01ed5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40aed9d640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f409c0039b0 0x7f40b01ed5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40aed9d640 1 -- 172.21.15.19:0/3081095245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00b5230 con 0x7f40b00b8a30 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40aed9d640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40b00b8a30 0x7f40b00b3090 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f40b00b96d0 tx=0x7f40a8007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40a6ffd640 1 -- 172.21.15.19:0/3081095245 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a800ab80 con 0x7f40b00b8a30 2024-04-25T10:01:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40a6ffd640 1 -- 172.21.15.19:0/3081095245 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a8018070 con 0x7f40b00b8a30 2024-04-25T10:01:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40b494c640 1 -- 172.21.15.19:0/3081095245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b00b5510 con 0x7f40b00b8a30 2024-04-25T10:01:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40a6ffd640 1 -- 172.21.15.19:0/3081095245 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a80133f0 con 0x7f40b00b8a30 2024-04-25T10:01:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.274+0000 7f40b494c640 1 -- 172.21.15.19:0/3081095245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b00b5ad0 con 0x7f40b00b8a30 2024-04-25T10:01:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.275+0000 7f40b494c640 1 -- 172.21.15.19:0/3081095245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b0073990 con 0x7f40b00b8a30 2024-04-25T10:01:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.276+0000 7f40a6ffd640 1 -- 172.21.15.19:0/3081095245 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40a8013590 con 0x7f40b00b8a30 2024-04-25T10:01:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.276+0000 7f40a6ffd640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f409006b500 0x7f409006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.276+0000 7f40a6ffd640 1 -- 172.21.15.19:0/3081095245 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f40a808a8f0 con 0x7f40b00b8a30 2024-04-25T10:01:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.276+0000 7f40ae59c640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f409006b500 0x7f409006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.277+0000 7f40ae59c640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f409006b500 0x7f409006d9c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f40a0033000 tx=0x7f40a0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.280+0000 7f40a6ffd640 1 -- 172.21.15.19:0/3081095245 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40a80595e0 con 0x7f40b00b8a30 2024-04-25T10:01:02.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.530+0000 7f40b494c640 1 -- 172.21.15.19:0/3081095245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f40b015c020 con 0x7f40b00b8a30 2024-04-25T10:01:02.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.530+0000 7f40a6ffd640 1 -- 172.21.15.19:0/3081095245 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1594 (secure 0 0 0) 0x7f40a8058ee0 con 0x7f40b00b8a30 2024-04-25T10:01:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:02.536 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":499,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076566130,"bytes_used":2562400256,"bytes_avail":770531713024,"bytes_total":773094113280,"write_bytes_sec":15015,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:00:55.018290+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:01:02.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.532+0000 7f40b494c640 1 -- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f409006b500 msgr2=0x7f409006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.532+0000 7f40b494c640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f409006b500 0x7f409006d9c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f40a0033000 tx=0x7f40a0008030 comp rx=0 tx=0).stop 2024-04-25T10:01:02.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.532+0000 7f40b494c640 1 -- 172.21.15.19:0/3081095245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40b00b8a30 msgr2=0x7f40b00b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.532+0000 7f40b494c640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40b00b8a30 0x7f40b00b3090 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f40b00b96d0 tx=0x7f40a8007a90 comp rx=0 tx=0).stop 2024-04-25T10:01:02.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.532+0000 7f40b494c640 1 -- 172.21.15.19:0/3081095245 shutdown_connections 2024-04-25T10:01:02.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.532+0000 7f40b494c640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f409006b500 0x7f409006d9c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.532+0000 7f40b494c640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40b00b8a30 0x7f40b00b3090 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.532+0000 7f40b494c640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40b01f3f00 0x7f40b00b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.532+0000 7f40b494c640 1 --2- 172.21.15.19:0/3081095245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f409c0039b0 0x7f40b01ed5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.532+0000 7f40b494c640 1 -- 172.21.15.19:0/3081095245 >> 172.21.15.19:0/3081095245 conn(0x7f40b0061a20 msgr2=0x7f40b015b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.533+0000 7f40b494c640 1 -- 172.21.15.19:0/3081095245 shutdown_connections 2024-04-25T10:01:02.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:02.533+0000 7f40b494c640 1 -- 172.21.15.19:0/3081095245 wait complete. 2024-04-25T10:01:02.549 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T10:01:03.550 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:03.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.659+0000 7fd2b11e2640 1 Processor -- start 2024-04-25T10:01:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.659+0000 7fd2b11e2640 1 -- start start 2024-04-25T10:01:03.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.660+0000 7fd2b11e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2ac154ee0 0x7fd2ac0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.660+0000 7fd2b11e2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2ac0cfcf0 0x7fd2ac0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.660+0000 7fd2b11e2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac071480 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.660+0000 7fd2b11e2640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ac0ce830 con 0x7fd2ac0cfcf0 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.660+0000 7fd2b11e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac072090 con 0x7fd2ac154ee0 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.660+0000 7fd2aa59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2ac0cfcf0 0x7fd2ac0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.660+0000 7fd2ab59e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd2ac0ce3c0 0x7fd2ac15e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44114/0 (socket says 172.21.15.19:44114) 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.660+0000 7fd2aa59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2ac0cfcf0 0x7fd2ac0cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50652/0 (socket says 172.21.15.19:50652) 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.660+0000 7fd2ab59e640 1 -- 172.21.15.19:0/956962849 learned_addr learned my addr 172.21.15.19:0/956962849 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.660+0000 7fd2aad9d640 1 --2- 172.21.15.19:0/956962849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2ac154ee0 0x7fd2ac0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.661+0000 7fd2aa59c640 1 -- 172.21.15.19:0/956962849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2ac154ee0 msgr2=0x7fd2ac0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.661+0000 7fd2aa59c640 1 --2- 172.21.15.19:0/956962849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2ac154ee0 0x7fd2ac0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.661+0000 7fd2aa59c640 1 -- 172.21.15.19:0/956962849 >> v1:172.21.15.19:6789/0 conn(0x7fd2ac0ce3c0 legacy=0x7fd2ac15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.661+0000 7fd2a9d9b640 1 -- 172.21.15.19:0/956962849 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538910123 0 0) 0x7fd2ac071480 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.661+0000 7fd2aa59c640 1 -- 172.21.15.19:0/956962849 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac071760 con 0x7fd2ac0cfcf0 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.661+0000 7fd2aa59c640 1 --2- 172.21.15.19:0/956962849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2ac0cfcf0 0x7fd2ac0cde80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000ba40 tx=0x7fd2a002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b80656947173f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.661+0000 7fd2a9d9b640 1 -- 172.21.15.19:0/956962849 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0032750 con 0x7fd2ac0cfcf0 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.661+0000 7fd2a9d9b640 1 -- 172.21.15.19:0/956962849 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a0030050 con 0x7fd2ac0cfcf0 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.662+0000 7fd2a9d9b640 1 -- 172.21.15.19:0/956962849 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0032b10 con 0x7fd2ac0cfcf0 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.662+0000 7fd2b11e2640 1 -- 172.21.15.19:0/956962849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2ac0cfcf0 msgr2=0x7fd2ac0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.662+0000 7fd2b11e2640 1 --2- 172.21.15.19:0/956962849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2ac0cfcf0 0x7fd2ac0cde80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000ba40 tx=0x7fd2a002c850 comp rx=0 tx=0).stop 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.662+0000 7fd2b11e2640 1 -- 172.21.15.19:0/956962849 shutdown_connections 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.662+0000 7fd2b11e2640 1 --2- 172.21.15.19:0/956962849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2ac0cfcf0 0x7fd2ac0cde80 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.662+0000 7fd2b11e2640 1 --2- 172.21.15.19:0/956962849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2ac154ee0 0x7fd2ac0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.662+0000 7fd2b11e2640 1 -- 172.21.15.19:0/956962849 >> 172.21.15.19:0/956962849 conn(0x7fd2ac06abe0 msgr2=0x7fd2ac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.662+0000 7fd2b11e2640 1 -- 172.21.15.19:0/956962849 shutdown_connections 2024-04-25T10:01:03.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.662+0000 7fd2b11e2640 1 -- 172.21.15.19:0/956962849 wait complete. 2024-04-25T10:01:03.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.664+0000 7fd2b11e2640 1 Processor -- start 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.664+0000 7fd2b11e2640 1 -- start start 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.664+0000 7fd2b11e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2ac0ce3c0 0x7fd2ac0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.664+0000 7fd2b11e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac0cfcf0 0x7fd2ac143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.664+0000 7fd2b11e2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2ac154ee0 0x7fd2ac1f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.664+0000 7fd2b11e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac071230 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.664+0000 7fd2b11e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac071d20 con 0x7fd2ac0cfcf0 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.664+0000 7fd2b11e2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac1552c0 con 0x7fd2ac154ee0 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.664+0000 7fd2aad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2ac0ce3c0 0x7fd2ac0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2aad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2ac0ce3c0 0x7fd2ac0a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50656/0 (socket says 172.21.15.19:50656) 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2aad9d640 1 -- 172.21.15.19:0/52511869 learned_addr learned my addr 172.21.15.19:0/52511869 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2ab59e640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2ac154ee0 0x7fd2ac1f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2aa59c640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac0cfcf0 0x7fd2ac143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2aad9d640 1 -- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2ac154ee0 msgr2=0x7fd2ac1f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2aad9d640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2ac154ee0 0x7fd2ac1f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2aad9d640 1 -- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac0cfcf0 msgr2=0x7fd2ac143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2aad9d640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac0cfcf0 0x7fd2ac143fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2aad9d640 1 -- 172.21.15.19:0/52511869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac15ac90 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2aa59c640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac0cfcf0 0x7fd2ac143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2aad9d640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2ac0ce3c0 0x7fd2ac0a84a0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400d5f0 tx=0x7fd2a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2937fe640 1 -- 172.21.15.19:0/52511869 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40125f0 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2937fe640 1 -- 172.21.15.19:0/52511869 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a4019070 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2b11e2640 1 -- 172.21.15.19:0/52511869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac157f50 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2937fe640 1 -- 172.21.15.19:0/52511869 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40129d0 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.665+0000 7fd2b11e2640 1 -- 172.21.15.19:0/52511869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac15a9b0 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.666+0000 7fd2b11e2640 1 -- 172.21.15.19:0/52511869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ac073990 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.667+0000 7fd2937fe640 1 -- 172.21.15.19:0/52511869 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2a4012b70 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.667+0000 7fd2937fe640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd29806b4d0 0x7fd29806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.667+0000 7fd2937fe640 1 -- 172.21.15.19:0/52511869 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2a408a710 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.668+0000 7fd2aa59c640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd29806b4d0 0x7fd29806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.668+0000 7fd2aa59c640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd29806b4d0 0x7fd29806d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0037000 tx=0x7fd2a0032000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.672+0000 7fd2937fe640 1 -- 172.21.15.19:0/52511869 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2a4059490 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.889+0000 7fd2b11e2640 1 -- 172.21.15.19:0/52511869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2ac0cedd0 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.890+0000 7fd2937fe640 1 -- 172.21.15.19:0/52511869 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd2a4058d90 con 0x7fd2ac0ce3c0 2024-04-25T10:01:03.896 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:03.896 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:01:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:01:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 -- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd29806b4d0 msgr2=0x7fd29806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd29806b4d0 0x7fd29806d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0037000 tx=0x7fd2a0032000 comp rx=0 tx=0).stop 2024-04-25T10:01:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 -- 172.21.15.19:0/52511869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2ac0ce3c0 msgr2=0x7fd2ac0a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2ac0ce3c0 0x7fd2ac0a84a0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400d5f0 tx=0x7fd2a400dab0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 -- 172.21.15.19:0/52511869 shutdown_connections 2024-04-25T10:01:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd29806b4d0 0x7fd29806d990 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2ac154ee0 0x7fd2ac1f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac0cfcf0 0x7fd2ac143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 --2- 172.21.15.19:0/52511869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2ac0ce3c0 0x7fd2ac0a84a0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 -- 172.21.15.19:0/52511869 >> 172.21.15.19:0/52511869 conn(0x7fd2ac06abe0 msgr2=0x7fd2ac0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 -- 172.21.15.19:0/52511869 shutdown_connections 2024-04-25T10:01:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:03.892+0000 7fd2b11e2640 1 -- 172.21.15.19:0/52511869 wait complete. 2024-04-25T10:01:03.908 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:01:03.909 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:04.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.003+0000 7feea398b640 1 Processor -- start 2024-04-25T10:01:04.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.004+0000 7feea398b640 1 -- start start 2024-04-25T10:01:04.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.004+0000 7feea398b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fee9c15dbe0 0x7fee9c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.004+0000 7feea398b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fee9c0b92a0 0x7fee9c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.004+0000 7feea398b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9c0b97a0 con 0x7fee9c0b2cb0 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.004+0000 7feea398b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9c15dfc0 con 0x7fee9c15dbe0 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.004+0000 7feea398b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9c0d1a00 con 0x7fee9c0b92a0 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.004+0000 7feea1727640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fee9c15dbe0 0x7fee9c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.004+0000 7feea1727640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fee9c15dbe0 0x7fee9c0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50670/0 (socket says 172.21.15.19:50670) 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.004+0000 7feea1727640 1 -- 172.21.15.19:0/1884856561 learned_addr learned my addr 172.21.15.19:0/1884856561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.005+0000 7feea1f28640 1 --2- 172.21.15.19:0/1884856561 >> v2:172.21.15.112:3300/0 conn(0x7fee9c0b92a0 0x7fee9c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.005+0000 7feea1727640 1 -- 172.21.15.19:0/1884856561 >> v2:172.21.15.112:3300/0 conn(0x7fee9c0b92a0 msgr2=0x7fee9c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.005+0000 7feea1727640 1 --2- 172.21.15.19:0/1884856561 >> v2:172.21.15.112:3300/0 conn(0x7fee9c0b92a0 0x7fee9c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.005+0000 7feea1727640 1 -- 172.21.15.19:0/1884856561 >> v1:172.21.15.112:6789/0 conn(0x7fee9c0b2cb0 legacy=0x7fee9c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.005+0000 7feea1727640 1 -- 172.21.15.19:0/1884856561 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c0d1c50 con 0x7fee9c15dbe0 2024-04-25T10:01:04.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.005+0000 7feea1f28640 1 --2- 172.21.15.19:0/1884856561 >> v2:172.21.15.112:3300/0 conn(0x7fee9c0b92a0 0x7fee9c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:04.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.005+0000 7feea1727640 1 --2- 172.21.15.19:0/1884856561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c15dbe0 0x7fee9c0d0910 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0cfb90 tx=0x7fee9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dec665e7eb94958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.006+0000 7fee9bfff640 1 -- 172.21.15.19:0/1884856561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94037070 con 0x7fee9c15dbe0 2024-04-25T10:01:04.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.006+0000 7fee9bfff640 1 -- 172.21.15.19:0/1884856561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee94032440 con 0x7fee9c15dbe0 2024-04-25T10:01:04.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.006+0000 7fee9bfff640 1 -- 172.21.15.19:0/1884856561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94032810 con 0x7fee9c15dbe0 2024-04-25T10:01:04.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.006+0000 7feea398b640 1 -- 172.21.15.19:0/1884856561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c15dbe0 msgr2=0x7fee9c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.006+0000 7feea398b640 1 --2- 172.21.15.19:0/1884856561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c15dbe0 0x7fee9c0d0910 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0cfb90 tx=0x7fee9402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.006+0000 7feea398b640 1 -- 172.21.15.19:0/1884856561 shutdown_connections 2024-04-25T10:01:04.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.006+0000 7feea398b640 1 --2- 172.21.15.19:0/1884856561 >> v2:172.21.15.112:3300/0 conn(0x7fee9c0b92a0 0x7fee9c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.006+0000 7feea398b640 1 --2- 172.21.15.19:0/1884856561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c15dbe0 0x7fee9c0d0910 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.006+0000 7feea398b640 1 -- 172.21.15.19:0/1884856561 >> 172.21.15.19:0/1884856561 conn(0x7fee9c061a20 msgr2=0x7fee9c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:04.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.006+0000 7feea398b640 1 -- 172.21.15.19:0/1884856561 shutdown_connections 2024-04-25T10:01:04.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.006+0000 7feea398b640 1 -- 172.21.15.19:0/1884856561 wait complete. 2024-04-25T10:01:04.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.007+0000 7feea398b640 1 Processor -- start 2024-04-25T10:01:04.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.008+0000 7feea398b640 1 -- start start 2024-04-25T10:01:04.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.008+0000 7feea398b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee9c0b2cb0 0x7fee9c1f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.008+0000 7feea398b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0b92a0 0x7fee9c1f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.008+0000 7feea398b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee9c15dbe0 0x7fee9c0c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.008+0000 7feea398b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0b6ab0 con 0x7fee9c0b92a0 2024-04-25T10:01:04.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.008+0000 7feea398b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0d2210 con 0x7fee9c0b2cb0 2024-04-25T10:01:04.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.008+0000 7feea398b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0b6c30 con 0x7fee9c15dbe0 2024-04-25T10:01:04.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.008+0000 7feea0f26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0b92a0 0x7fee9c1f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.008+0000 7feea0f26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0b92a0 0x7fee9c1f6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50680/0 (socket says 172.21.15.19:50680) 2024-04-25T10:01:04.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.008+0000 7feea0f26640 1 -- 172.21.15.19:0/3290364504 learned_addr learned my addr 172.21.15.19:0/3290364504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:04.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7feea1727640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee9c0b2cb0 0x7fee9c1f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7feea1f28640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee9c15dbe0 0x7fee9c0c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7feea0f26640 1 -- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee9c15dbe0 msgr2=0x7fee9c0c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7feea0f26640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee9c15dbe0 0x7fee9c0c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7feea0f26640 1 -- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee9c0b2cb0 msgr2=0x7fee9c1f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7feea0f26640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee9c0b2cb0 0x7fee9c1f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7feea0f26640 1 -- 172.21.15.19:0/3290364504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c1f7730 con 0x7fee9c0b92a0 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7feea1727640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee9c0b2cb0 0x7fee9c1f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7feea0f26640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0b92a0 0x7fee9c1f6ee0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fee9000c440 tx=0x7fee90001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7fee99ffb640 1 -- 172.21.15.19:0/3290364504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee900052d0 con 0x7fee9c0b92a0 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7fee99ffb640 1 -- 172.21.15.19:0/3290364504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee90005470 con 0x7fee9c0b92a0 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7fee99ffb640 1 -- 172.21.15.19:0/3290364504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90007260 con 0x7fee9c0b92a0 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7feea398b640 1 -- 172.21.15.19:0/3290364504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c1f7a10 con 0x7fee9c0b92a0 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.009+0000 7feea398b640 1 -- 172.21.15.19:0/3290364504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c144650 con 0x7fee9c0b92a0 2024-04-25T10:01:04.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.010+0000 7feea398b640 1 -- 172.21.15.19:0/3290364504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9c073990 con 0x7fee9c0b92a0 2024-04-25T10:01:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.011+0000 7fee99ffb640 1 -- 172.21.15.19:0/3290364504 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee90007400 con 0x7fee9c0b92a0 2024-04-25T10:01:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.011+0000 7fee99ffb640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee8806b4d0 0x7fee8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.011+0000 7fee99ffb640 1 -- 172.21.15.19:0/3290364504 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fee900898d0 con 0x7fee9c0b92a0 2024-04-25T10:01:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.011+0000 7feea1727640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee8806b4d0 0x7fee8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.012+0000 7feea1727640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee8806b4d0 0x7fee8806d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fee94034000 tx=0x7fee9402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.015+0000 7fee99ffb640 1 -- 172.21.15.19:0/3290364504 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee900585c0 con 0x7fee9c0b92a0 2024-04-25T10:01:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.271+0000 7feea398b640 1 -- 172.21.15.19:0/3290364504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fee9c0b28f0 con 0x7fee9c0b92a0 2024-04-25T10:01:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.271+0000 7fee99ffb640 1 -- 172.21.15.19:0/3290364504 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1594 (secure 0 0 0) 0x7fee90057ec0 con 0x7fee9c0b92a0 2024-04-25T10:01:04.277 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:04.277 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":501,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076566130,"bytes_used":2562400256,"bytes_avail":770531713024,"bytes_total":773094113280,"write_bytes_sec":15014,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:00:55.018290+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:01:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.273+0000 7feea398b640 1 -- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee8806b4d0 msgr2=0x7fee8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.273+0000 7feea398b640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee8806b4d0 0x7fee8806d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fee94034000 tx=0x7fee9402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.273+0000 7feea398b640 1 -- 172.21.15.19:0/3290364504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0b92a0 msgr2=0x7fee9c1f6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.273+0000 7feea398b640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0b92a0 0x7fee9c1f6ee0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fee9000c440 tx=0x7fee90001e70 comp rx=0 tx=0).stop 2024-04-25T10:01:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.273+0000 7feea398b640 1 -- 172.21.15.19:0/3290364504 shutdown_connections 2024-04-25T10:01:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.273+0000 7feea398b640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee8806b4d0 0x7fee8806d990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.273+0000 7feea398b640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee9c15dbe0 0x7fee9c0c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.273+0000 7feea398b640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9c0b92a0 0x7fee9c1f6ee0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.273+0000 7feea398b640 1 --2- 172.21.15.19:0/3290364504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee9c0b2cb0 0x7fee9c1f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.273+0000 7feea398b640 1 -- 172.21.15.19:0/3290364504 >> 172.21.15.19:0/3290364504 conn(0x7fee9c061a20 msgr2=0x7fee9c15b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.274+0000 7feea398b640 1 -- 172.21.15.19:0/3290364504 shutdown_connections 2024-04-25T10:01:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:04.274+0000 7feea398b640 1 -- 172.21.15.19:0/3290364504 wait complete. 2024-04-25T10:01:04.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T10:01:05.291 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.400+0000 7fc5cc9f5640 1 Processor -- start 2024-04-25T10:01:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.400+0000 7fc5cc9f5640 1 -- start start 2024-04-25T10:01:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5cc9f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5c806abe0 0x7fc5c806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5cc9f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc5c80ce330 0x7fc5c806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5cc9f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c81626e0 con 0x7fc5c80700a0 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5cc9f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c806b500 con 0x7fc5c80ce330 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5cc9f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c80cd270 con 0x7fc5c806abe0 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5bffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc5c80ce330 0x7fc5c806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5bffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc5c80ce330 0x7fc5c806fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50694/0 (socket says 172.21.15.19:50694) 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5bffff640 1 -- 172.21.15.19:0/1531567912 learned_addr learned my addr 172.21.15.19:0/1531567912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5c659c640 1 --2- 172.21.15.19:0/1531567912 >> v2:172.21.15.112:3300/0 conn(0x7fc5c806abe0 0x7fc5c806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5bffff640 1 -- 172.21.15.19:0/1531567912 >> v2:172.21.15.112:3300/0 conn(0x7fc5c806abe0 msgr2=0x7fc5c806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5bffff640 1 --2- 172.21.15.19:0/1531567912 >> v2:172.21.15.112:3300/0 conn(0x7fc5c806abe0 0x7fc5c806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5bffff640 1 -- 172.21.15.19:0/1531567912 >> v1:172.21.15.112:6789/0 conn(0x7fc5c80700a0 legacy=0x7fc5c806e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5bffff640 1 -- 172.21.15.19:0/1531567912 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c80cd550 con 0x7fc5c80ce330 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.401+0000 7fc5c659c640 1 --2- 172.21.15.19:0/1531567912 >> v2:172.21.15.112:3300/0 conn(0x7fc5c806abe0 0x7fc5c806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.402+0000 7fc5bffff640 1 --2- 172.21.15.19:0/1531567912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c80ce330 0x7fc5c806fb60 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fc5c806e200 tx=0x7fc5b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ecff243113d56f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.402+0000 7fc5c5d9b640 1 -- 172.21.15.19:0/1531567912 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0037070 con 0x7fc5c80ce330 2024-04-25T10:01:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.402+0000 7fc5c5d9b640 1 -- 172.21.15.19:0/1531567912 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b0032440 con 0x7fc5c80ce330 2024-04-25T10:01:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.402+0000 7fc5c5d9b640 1 -- 172.21.15.19:0/1531567912 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0032810 con 0x7fc5c80ce330 2024-04-25T10:01:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.403+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/1531567912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c80ce330 msgr2=0x7fc5c806fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.403+0000 7fc5cc9f5640 1 --2- 172.21.15.19:0/1531567912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c80ce330 0x7fc5c806fb60 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fc5c806e200 tx=0x7fc5b002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.403+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/1531567912 shutdown_connections 2024-04-25T10:01:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.403+0000 7fc5cc9f5640 1 --2- 172.21.15.19:0/1531567912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c80ce330 0x7fc5c806fb60 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.403+0000 7fc5cc9f5640 1 --2- 172.21.15.19:0/1531567912 >> v2:172.21.15.112:3300/0 conn(0x7fc5c806abe0 0x7fc5c806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.403+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/1531567912 >> 172.21.15.19:0/1531567912 conn(0x7fc5c80b2220 msgr2=0x7fc5c80b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.403+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/1531567912 shutdown_connections 2024-04-25T10:01:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.403+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/1531567912 wait complete. 2024-04-25T10:01:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.404+0000 7fc5cc9f5640 1 Processor -- start 2024-04-25T10:01:05.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.404+0000 7fc5cc9f5640 1 -- start start 2024-04-25T10:01:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.405+0000 7fc5cc9f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5c806abe0 0x7fc5c815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.405+0000 7fc5cc9f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c80700a0 0x7fc5c80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.405+0000 7fc5cc9f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c80ce330 0x7fc5c80b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.405+0000 7fc5cc9f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c8161780 con 0x7fc5c80700a0 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.405+0000 7fc5cc9f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c80cdb10 con 0x7fc5c806abe0 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.405+0000 7fc5cc9f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c806eb00 con 0x7fc5c80ce330 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.405+0000 7fc5bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c80700a0 0x7fc5c80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.405+0000 7fc5bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c80700a0 0x7fc5c80b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50710/0 (socket says 172.21.15.19:50710) 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.405+0000 7fc5bffff640 1 -- 172.21.15.19:0/2215902019 learned_addr learned my addr 172.21.15.19:0/2215902019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.405+0000 7fc5c659c640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5c806abe0 0x7fc5c815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5c6d9d640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c80ce330 0x7fc5c80b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5bffff640 1 -- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c80ce330 msgr2=0x7fc5c80b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5bffff640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c80ce330 0x7fc5c80b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5bffff640 1 -- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5c806abe0 msgr2=0x7fc5c815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5bffff640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5c806abe0 0x7fc5c815cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5bffff640 1 -- 172.21.15.19:0/2215902019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c80ada70 con 0x7fc5c80700a0 2024-04-25T10:01:05.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5c659c640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5c806abe0 0x7fc5c815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:05.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5bffff640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c80700a0 0x7fc5c80b4320 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0000c00 tx=0x7fc5b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5beffd640 1 -- 172.21.15.19:0/2215902019 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0041070 con 0x7fc5c80700a0 2024-04-25T10:01:05.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5beffd640 1 -- 172.21.15.19:0/2215902019 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b003c870 con 0x7fc5c80700a0 2024-04-25T10:01:05.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/2215902019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c809d280 con 0x7fc5c80700a0 2024-04-25T10:01:05.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5beffd640 1 -- 172.21.15.19:0/2215902019 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0037040 con 0x7fc5c80700a0 2024-04-25T10:01:05.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.406+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/2215902019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c809d840 con 0x7fc5c80700a0 2024-04-25T10:01:05.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.407+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/2215902019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c8073990 con 0x7fc5c80700a0 2024-04-25T10:01:05.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.408+0000 7fc5beffd640 1 -- 172.21.15.19:0/2215902019 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5b0044070 con 0x7fc5c80700a0 2024-04-25T10:01:05.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.408+0000 7fc5beffd640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 0x7fc5a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.408+0000 7fc5beffd640 1 -- 172.21.15.19:0/2215902019 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc5b00ab0b0 con 0x7fc5c80700a0 2024-04-25T10:01:05.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.409+0000 7fc5c659c640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 0x7fc5a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.409+0000 7fc5c659c640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 0x7fc5a806d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8072630 tx=0x7fc5b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.413+0000 7fc5beffd640 1 -- 172.21.15.19:0/2215902019 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5b0079f50 con 0x7fc5c80700a0 2024-04-25T10:01:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.630+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/2215902019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5c80afe50 con 0x7fc5c80700a0 2024-04-25T10:01:05.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.630+0000 7fc5beffd640 1 -- 172.21.15.19:0/2215902019 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc5b0079850 con 0x7fc5c80700a0 2024-04-25T10:01:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:05.636 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:01:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:01:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 msgr2=0x7fc5a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 0x7fc5a806d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8072630 tx=0x7fc5b8008040 comp rx=0 tx=0).stop 2024-04-25T10:01:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/2215902019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c80700a0 msgr2=0x7fc5c80b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c80700a0 0x7fc5c80b4320 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0000c00 tx=0x7fc5b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/2215902019 shutdown_connections 2024-04-25T10:01:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 0x7fc5a806d990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c80ce330 0x7fc5c80b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c80700a0 0x7fc5c80b4320 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 --2- 172.21.15.19:0/2215902019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5c806abe0 0x7fc5c815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/2215902019 >> 172.21.15.19:0/2215902019 conn(0x7fc5c80b2220 msgr2=0x7fc5c815fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/2215902019 shutdown_connections 2024-04-25T10:01:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.632+0000 7fc5cc9f5640 1 -- 172.21.15.19:0/2215902019 wait complete. 2024-04-25T10:01:05.648 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:01:05.649 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:05.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.743+0000 7fd1ed71d640 1 Processor -- start 2024-04-25T10:01:05.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.743+0000 7fd1ed71d640 1 -- start start 2024-04-25T10:01:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.744+0000 7fd1ed71d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd1e80b2cb0 0x7fd1e80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.744+0000 7fd1ed71d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e80b97a0 con 0x7fd1e815dbe0 2024-04-25T10:01:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.744+0000 7fd1ed71d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e80d1c50 con 0x7fd1e80d1020 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.744+0000 7fd1ed71d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e815dfc0 con 0x7fd1e80b2cb0 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.744+0000 7fd1e6ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd1e815dbe0 0x7fd1e80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44120/0 (socket says 172.21.15.19:44120) 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.744+0000 7fd1e6ffd640 1 -- 172.21.15.19:0/3921304525 learned_addr learned my addr 172.21.15.19:0/3921304525 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.744+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213704015 0 0) 0x7fd1e80b97a0 con 0x7fd1e815dbe0 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.744+0000 7fd1e67fc640 1 --2- 172.21.15.19:0/3921304525 >> v2:172.21.15.112:3300/0 conn(0x7fd1e80b2cb0 0x7fd1e80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.744+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1d0003520 con 0x7fd1e815dbe0 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 293751945 0 0) 0x7fd1d0003520 con 0x7fd1e815dbe0 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 >> v2:172.21.15.112:3300/0 conn(0x7fd1e80b2cb0 msgr2=0x7fd1e80d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 --2- 172.21.15.19:0/3921304525 >> v2:172.21.15.112:3300/0 conn(0x7fd1e80b2cb0 0x7fd1e80d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 >> v1:172.21.15.112:6789/0 conn(0x7fd1e80d1020 legacy=0x7fd1e80d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80d1f30 con 0x7fd1e815dbe0 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fd1e0001670 con 0x7fd1e815dbe0 2024-04-25T10:01:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 >> v1:172.21.15.19:6789/0 conn(0x7fd1e815dbe0 legacy=0x7fd1e80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 --2- 172.21.15.19:0/3921304525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d00039b0 0x7fd1d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 --2- 172.21.15.19:0/3921304525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1d0004800 0x7fd1d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 --2- 172.21.15.19:0/3921304525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1d00051c0 0x7fd1d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d000c030 con 0x7fd1d00039b0 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e6ffd640 1 --2- 172.21.15.19:0/3921304525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d00039b0 0x7fd1d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d000c1e0 con 0x7fd1d0004800 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d000c3e0 con 0x7fd1d00051c0 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e67fc640 1 --2- 172.21.15.19:0/3921304525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1d0004800 0x7fd1d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e77fe640 1 --2- 172.21.15.19:0/3921304525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1d00051c0 0x7fd1d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e6ffd640 1 -- 172.21.15.19:0/3921304525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1d00051c0 msgr2=0x7fd1d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e6ffd640 1 --2- 172.21.15.19:0/3921304525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1d00051c0 0x7fd1d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e6ffd640 1 -- 172.21.15.19:0/3921304525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1d0004800 msgr2=0x7fd1d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e6ffd640 1 --2- 172.21.15.19:0/3921304525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1d0004800 0x7fd1d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e77fe640 1 -- 172.21.15.19:0/3921304525 reap_dead start 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.745+0000 7fd1e6ffd640 1 -- 172.21.15.19:0/3921304525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d0003520 con 0x7fd1d00039b0 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.746+0000 7fd1e6ffd640 1 --2- 172.21.15.19:0/3921304525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d00039b0 0x7fd1d0003dd0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c240 tx=0x7fd1d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73c66289fbfae79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.746+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0007d10 con 0x7fd1d00039b0 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.746+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e00058d0 con 0x7fd1d00039b0 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.746+0000 7fd1e5ffb640 1 -- 172.21.15.19:0/3921304525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0005d40 con 0x7fd1d00039b0 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.747+0000 7fd1ed71d640 1 -- 172.21.15.19:0/3921304525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d00039b0 msgr2=0x7fd1d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.747+0000 7fd1ed71d640 1 --2- 172.21.15.19:0/3921304525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d00039b0 0x7fd1d0003dd0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c240 tx=0x7fd1d0005830 comp rx=0 tx=0).stop 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.747+0000 7fd1ed71d640 1 -- 172.21.15.19:0/3921304525 shutdown_connections 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.747+0000 7fd1ed71d640 1 --2- 172.21.15.19:0/3921304525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d00039b0 0x7fd1d0003dd0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.747+0000 7fd1ed71d640 1 -- 172.21.15.19:0/3921304525 >> 172.21.15.19:0/3921304525 conn(0x7fd1e8061a20 msgr2=0x7fd1e8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.747+0000 7fd1ed71d640 1 -- 172.21.15.19:0/3921304525 shutdown_connections 2024-04-25T10:01:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.747+0000 7fd1ed71d640 1 -- 172.21.15.19:0/3921304525 wait complete. 2024-04-25T10:01:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.748+0000 7fd1ed71d640 1 Processor -- start 2024-04-25T10:01:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.748+0000 7fd1ed71d640 1 -- start start 2024-04-25T10:01:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1ed71d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1d00039b0 0x7fd1e81f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1ed71d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e8155750 0x7fd1e815f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1ed71d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e81ed380 0x7fd1e815ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1ed71d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80b92a0 con 0x7fd1e81ed380 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1ed71d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80d1a00 con 0x7fd1e8155750 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1ed71d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80b9420 con 0x7fd1d00039b0 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1e77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e81ed380 0x7fd1e815ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1e77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e81ed380 0x7fd1e815ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50734/0 (socket says 172.21.15.19:50734) 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1e77fe640 1 -- 172.21.15.19:0/1514017861 learned_addr learned my addr 172.21.15.19:0/1514017861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1e67fc640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e8155750 0x7fd1e815f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1e6ffd640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1d00039b0 0x7fd1e81f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1e77fe640 1 -- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1d00039b0 msgr2=0x7fd1e81f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1e77fe640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1d00039b0 0x7fd1e81f3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1e77fe640 1 -- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e8155750 msgr2=0x7fd1e815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1e77fe640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e8155750 0x7fd1e815f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1e77fe640 1 -- 172.21.15.19:0/1514017861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e81f44f0 con 0x7fd1e81ed380 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.749+0000 7fd1e77fe640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e81ed380 0x7fd1e815ffa0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80b64a0 tx=0x7fd1d800fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.750+0000 7fd1cf7fe640 1 -- 172.21.15.19:0/1514017861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8003b80 con 0x7fd1e81ed380 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.750+0000 7fd1cf7fe640 1 -- 172.21.15.19:0/1514017861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d8018070 con 0x7fd1e81ed380 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.750+0000 7fd1ed71d640 1 -- 172.21.15.19:0/1514017861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e815e4c0 con 0x7fd1e81ed380 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.750+0000 7fd1cf7fe640 1 -- 172.21.15.19:0/1514017861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d80050d0 con 0x7fd1e81ed380 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.750+0000 7fd1ed71d640 1 -- 172.21.15.19:0/1514017861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e815ea80 con 0x7fd1e81ed380 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.751+0000 7fd1ed71d640 1 -- 172.21.15.19:0/1514017861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e8073990 con 0x7fd1e81ed380 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.751+0000 7fd1cf7fe640 1 -- 172.21.15.19:0/1514017861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1d8005270 con 0x7fd1e81ed380 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.751+0000 7fd1cf7fe640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1d406b500 0x7fd1d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.752+0000 7fd1cf7fe640 1 -- 172.21.15.19:0/1514017861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd1d808a1d0 con 0x7fd1e81ed380 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.752+0000 7fd1e6ffd640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1d406b500 0x7fd1d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.752+0000 7fd1e6ffd640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1d406b500 0x7fd1d406d9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0033000 tx=0x7fd1e002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:05.756+0000 7fd1cf7fe640 1 -- 172.21.15.19:0/1514017861 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1e81f44f0 con 0x7fd1e81ed380 2024-04-25T10:01:06.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.004+0000 7fd1ed71d640 1 -- 172.21.15.19:0/1514017861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd1e8160b40 con 0x7fd1e81ed380 2024-04-25T10:01:06.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.004+0000 7fd1cf7fe640 1 -- 172.21.15.19:0/1514017861 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1571 (secure 0 0 0) 0x7fd1d801a310 con 0x7fd1e81ed380 2024-04-25T10:01:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:06.010 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":503,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076566130,"bytes_used":2562408448,"bytes_avail":770531704832,"bytes_total":773094113280,"write_bytes_sec":15014,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8577}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:01:05.021044+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:01:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 -- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1d406b500 msgr2=0x7fd1d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1d406b500 0x7fd1d406d9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0033000 tx=0x7fd1e002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 -- 172.21.15.19:0/1514017861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e81ed380 msgr2=0x7fd1e815ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e81ed380 0x7fd1e815ffa0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80b64a0 tx=0x7fd1d800fc70 comp rx=0 tx=0).stop 2024-04-25T10:01:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 -- 172.21.15.19:0/1514017861 shutdown_connections 2024-04-25T10:01:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1d406b500 0x7fd1d406d9c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e81ed380 0x7fd1e815ffa0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e8155750 0x7fd1e815f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 --2- 172.21.15.19:0/1514017861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1d00039b0 0x7fd1e81f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 -- 172.21.15.19:0/1514017861 >> 172.21.15.19:0/1514017861 conn(0x7fd1e8061a20 msgr2=0x7fd1e815cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:06.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 -- 172.21.15.19:0/1514017861 shutdown_connections 2024-04-25T10:01:06.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.006+0000 7fd1ed71d640 1 -- 172.21.15.19:0/1514017861 wait complete. 2024-04-25T10:01:06.023 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:01:06.023 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:01:06.024 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:06.061 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:06.104 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:01:06.104 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:01:06.104 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:06.120 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:01:06.120 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:06.176 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:06.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.311+0000 7f86118b3640 1 Processor -- start 2024-04-25T10:01:06.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.311+0000 7f86118b3640 1 -- start start 2024-04-25T10:01:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.312+0000 7f86118b3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f860c15dbe0 0x7f860c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.312+0000 7f86118b3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860c0b6220 con 0x7f860c0b2cb0 2024-04-25T10:01:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.312+0000 7f86118b3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860c0b97a0 con 0x7f860c0b92a0 2024-04-25T10:01:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.312+0000 7f86118b3640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f860c15dfc0 con 0x7f860c15dbe0 2024-04-25T10:01:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.312+0000 7f860affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f860c15dbe0 0x7f860c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.312+0000 7f860a7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f860c0b2cb0 0x7f860c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44134/0 (socket says 172.21.15.19:44134) 2024-04-25T10:01:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.312+0000 7f860a7fc640 1 -- 172.21.15.19:0/1769640063 learned_addr learned my addr 172.21.15.19:0/1769640063 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:06.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.312+0000 7f860affd640 1 -- 172.21.15.19:0/1769640063 >> v1:172.21.15.112:6789/0 conn(0x7f860c0b92a0 legacy=0x7f860c0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.312+0000 7f860affd640 1 -- 172.21.15.19:0/1769640063 >> v1:172.21.15.19:6789/0 conn(0x7f860c0b2cb0 legacy=0x7f860c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.312+0000 7f860affd640 1 -- 172.21.15.19:0/1769640063 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c0d1c50 con 0x7f860c15dbe0 2024-04-25T10:01:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.313+0000 7f860affd640 1 --2- 172.21.15.19:0/1769640063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f860c15dbe0 0x7f860c0d0910 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f86040097a0 tx=0x7f860402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6798936505c0a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.313+0000 7f8609ffb640 1 -- 172.21.15.19:0/1769640063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604037070 con 0x7f860c15dbe0 2024-04-25T10:01:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.313+0000 7f8609ffb640 1 -- 172.21.15.19:0/1769640063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8604032440 con 0x7f860c15dbe0 2024-04-25T10:01:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.313+0000 7f8609ffb640 1 -- 172.21.15.19:0/1769640063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604032810 con 0x7f860c15dbe0 2024-04-25T10:01:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.314+0000 7f86118b3640 1 -- 172.21.15.19:0/1769640063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f860c15dbe0 msgr2=0x7f860c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.314+0000 7f86118b3640 1 --2- 172.21.15.19:0/1769640063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f860c15dbe0 0x7f860c0d0910 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f86040097a0 tx=0x7f860402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.314+0000 7f86118b3640 1 -- 172.21.15.19:0/1769640063 shutdown_connections 2024-04-25T10:01:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.314+0000 7f86118b3640 1 --2- 172.21.15.19:0/1769640063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f860c15dbe0 0x7f860c0d0910 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.314+0000 7f86118b3640 1 -- 172.21.15.19:0/1769640063 >> 172.21.15.19:0/1769640063 conn(0x7f860c061a20 msgr2=0x7f860c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.314+0000 7f86118b3640 1 -- 172.21.15.19:0/1769640063 shutdown_connections 2024-04-25T10:01:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.314+0000 7f86118b3640 1 -- 172.21.15.19:0/1769640063 wait complete. 2024-04-25T10:01:06.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.315+0000 7f86118b3640 1 Processor -- start 2024-04-25T10:01:06.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.315+0000 7f86118b3640 1 -- start start 2024-04-25T10:01:06.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f86118b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f860c0b2cb0 0x7f860c0c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f86118b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f860c0b92a0 0x7f860c144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f86118b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f860c15dbe0 0x7f860c144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f86118b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0d2210 con 0x7f860c0b92a0 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f86118b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0d1a00 con 0x7f860c0b2cb0 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f86118b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0b6ab0 con 0x7f860c15dbe0 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f860a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f860c0b92a0 0x7f860c144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f860a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f860c0b92a0 0x7f860c144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50742/0 (socket says 172.21.15.19:50742) 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f860a7fc640 1 -- 172.21.15.19:0/256980299 learned_addr learned my addr 172.21.15.19:0/256980299 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f860affd640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f860c0b2cb0 0x7f860c0c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f860b7fe640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f860c15dbe0 0x7f860c144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f860a7fc640 1 -- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f860c15dbe0 msgr2=0x7f860c144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f860a7fc640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f860c15dbe0 0x7f860c144e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.316+0000 7f860a7fc640 1 -- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f860c0b2cb0 msgr2=0x7f860c0c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.317+0000 7f860a7fc640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f860c0b2cb0 0x7f860c0c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.317+0000 7f860a7fc640 1 -- 172.21.15.19:0/256980299 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c1e63b0 con 0x7f860c0b92a0 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.317+0000 7f860b7fe640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f860c15dbe0 0x7f860c144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.317+0000 7f860a7fc640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f860c0b92a0 0x7f860c144900 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f860000c500 tx=0x7f860000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.317+0000 7f85f3fff640 1 -- 172.21.15.19:0/256980299 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600017070 con 0x7f860c0b92a0 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.317+0000 7f85f3fff640 1 -- 172.21.15.19:0/256980299 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8600005ce0 con 0x7f860c0b92a0 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.317+0000 7f85f3fff640 1 -- 172.21.15.19:0/256980299 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86000072f0 con 0x7f860c0b92a0 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.317+0000 7f86118b3640 1 -- 172.21.15.19:0/256980299 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860c145470 con 0x7f860c0b92a0 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.317+0000 7f86118b3640 1 -- 172.21.15.19:0/256980299 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860c145a30 con 0x7f860c0b92a0 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.318+0000 7f86118b3640 1 -- 172.21.15.19:0/256980299 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860c073990 con 0x7f860c0b92a0 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.319+0000 7f85f3fff640 1 -- 172.21.15.19:0/256980299 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f860001a070 con 0x7f860c0b92a0 2024-04-25T10:01:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.319+0000 7f85f3fff640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85f806b4d0 0x7f85f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.319+0000 7f85f3fff640 1 -- 172.21.15.19:0/256980299 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8600089560 con 0x7f860c0b92a0 2024-04-25T10:01:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.319+0000 7f860affd640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85f806b4d0 0x7f85f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.320+0000 7f860affd640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85f806b4d0 0x7f85f806d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8604034000 tx=0x7f860402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.323+0000 7f85f3fff640 1 -- 172.21.15.19:0/256980299 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8600057c90 con 0x7f860c0b92a0 2024-04-25T10:01:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.506+0000 7f86118b3640 1 -- 172.21.15.19:0/256980299 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f860c15c720 con 0x7f860c0b92a0 2024-04-25T10:01:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.507+0000 7f85f3fff640 1 -- 172.21.15.19:0/256980299 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f860005bd30 con 0x7f860c0b92a0 2024-04-25T10:01:06.512 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:06.512 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":117,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:00:34.061124+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:30.702453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:31.396192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 -- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85f806b4d0 msgr2=0x7f85f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85f806b4d0 0x7f85f806d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8604034000 tx=0x7f860402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 -- 172.21.15.19:0/256980299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f860c0b92a0 msgr2=0x7f860c144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f860c0b92a0 0x7f860c144900 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f860000c500 tx=0x7f860000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 -- 172.21.15.19:0/256980299 shutdown_connections 2024-04-25T10:01:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85f806b4d0 0x7f85f806d990 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f860c15dbe0 0x7f860c144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f860c0b92a0 0x7f860c144900 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 --2- 172.21.15.19:0/256980299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f860c0b2cb0 0x7f860c0c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 -- 172.21.15.19:0/256980299 >> 172.21.15.19:0/256980299 conn(0x7f860c061a20 msgr2=0x7f860c0d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 -- 172.21.15.19:0/256980299 shutdown_connections 2024-04-25T10:01:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.509+0000 7f86118b3640 1 -- 172.21.15.19:0/256980299 wait complete. 2024-04-25T10:01:06.526 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:00:30.702453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:00:31.396192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:06.527 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.625+0000 7f92c6e93640 1 Processor -- start 2024-04-25T10:01:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.625+0000 7f92c6e93640 1 -- start start 2024-04-25T10:01:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.626+0000 7f92c6e93640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f92c015b7d0 0x7f92c00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.626+0000 7f92c6e93640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92c00b5620 0x7f92c00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.626+0000 7f92c6e93640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c006cf50 con 0x7f92c006c2d0 2024-04-25T10:01:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.626+0000 7f92c6e93640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c00ce530 con 0x7f92c00b5620 2024-04-25T10:01:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.626+0000 7f92c6e93640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c00ce810 con 0x7f92c015b7d0 2024-04-25T10:01:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.626+0000 7f92bffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92c00b5620 0x7f92c00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.626+0000 7f92bffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92c00b5620 0x7f92c00b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50756/0 (socket says 172.21.15.19:50756) 2024-04-25T10:01:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.626+0000 7f92bffff640 1 -- 172.21.15.19:0/3834504866 learned_addr learned my addr 172.21.15.19:0/3834504866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.627+0000 7f92c4c2f640 1 --2- 172.21.15.19:0/3834504866 >> v2:172.21.15.112:3300/0 conn(0x7f92c015b7d0 0x7f92c00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.627+0000 7f92bffff640 1 -- 172.21.15.19:0/3834504866 >> v2:172.21.15.112:3300/0 conn(0x7f92c015b7d0 msgr2=0x7f92c00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.627+0000 7f92bffff640 1 --2- 172.21.15.19:0/3834504866 >> v2:172.21.15.112:3300/0 conn(0x7f92c015b7d0 0x7f92c00cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.627+0000 7f92bffff640 1 -- 172.21.15.19:0/3834504866 >> v1:172.21.15.112:6789/0 conn(0x7f92c006c2d0 legacy=0x7f92c006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.627+0000 7f92bffff640 1 -- 172.21.15.19:0/3834504866 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c00ceaf0 con 0x7f92c00b5620 2024-04-25T10:01:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.627+0000 7f92bffff640 1 --2- 172.21.15.19:0/3834504866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c00b5620 0x7f92c00b5ab0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f92c008c0d0 tx=0x7f92b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7688f6d459e7d8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.628+0000 7f92bf7fe640 1 -- 172.21.15.19:0/3834504866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4037070 con 0x7f92c00b5620 2024-04-25T10:01:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.628+0000 7f92bf7fe640 1 -- 172.21.15.19:0/3834504866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b4032440 con 0x7f92c00b5620 2024-04-25T10:01:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.628+0000 7f92bf7fe640 1 -- 172.21.15.19:0/3834504866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4032810 con 0x7f92c00b5620 2024-04-25T10:01:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.628+0000 7f92c6e93640 1 -- 172.21.15.19:0/3834504866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c00b5620 msgr2=0x7f92c00b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.628+0000 7f92c6e93640 1 --2- 172.21.15.19:0/3834504866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c00b5620 0x7f92c00b5ab0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f92c008c0d0 tx=0x7f92b402a270 comp rx=0 tx=0).stop 2024-04-25T10:01:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.628+0000 7f92c6e93640 1 -- 172.21.15.19:0/3834504866 shutdown_connections 2024-04-25T10:01:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.628+0000 7f92c6e93640 1 --2- 172.21.15.19:0/3834504866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c00b5620 0x7f92c00b5ab0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.628+0000 7f92c6e93640 1 --2- 172.21.15.19:0/3834504866 >> v2:172.21.15.112:3300/0 conn(0x7f92c015b7d0 0x7f92c00cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.628+0000 7f92c6e93640 1 -- 172.21.15.19:0/3834504866 >> 172.21.15.19:0/3834504866 conn(0x7f92c00d0800 msgr2=0x7f92c00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.628+0000 7f92c6e93640 1 -- 172.21.15.19:0/3834504866 shutdown_connections 2024-04-25T10:01:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.628+0000 7f92c6e93640 1 -- 172.21.15.19:0/3834504866 wait complete. 2024-04-25T10:01:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.629+0000 7f92c6e93640 1 Processor -- start 2024-04-25T10:01:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.629+0000 7f92c6e93640 1 -- start start 2024-04-25T10:01:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.630+0000 7f92c6e93640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c006c2d0 0x7f92c01551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.630+0000 7f92c6e93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c00b5620 0x7f92c0155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.630+0000 7f92c6e93640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c015b7d0 0x7f92c0155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.630+0000 7f92c6e93640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00cf6d0 con 0x7f92c00b5620 2024-04-25T10:01:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.630+0000 7f92c6e93640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00cf550 con 0x7f92c006c2d0 2024-04-25T10:01:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.630+0000 7f92c6e93640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c015bbb0 con 0x7f92c015b7d0 2024-04-25T10:01:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.630+0000 7f92bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c00b5620 0x7f92c0155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.630+0000 7f92bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c00b5620 0x7f92c0155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50760/0 (socket says 172.21.15.19:50760) 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.630+0000 7f92bffff640 1 -- 172.21.15.19:0/1524566121 learned_addr learned my addr 172.21.15.19:0/1524566121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.630+0000 7f92c4c2f640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c006c2d0 0x7f92c01551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92c5430640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c015b7d0 0x7f92c0155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92bffff640 1 -- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c015b7d0 msgr2=0x7f92c0155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92bffff640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c015b7d0 0x7f92c0155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92bffff640 1 -- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c006c2d0 msgr2=0x7f92c01551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92bffff640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c006c2d0 0x7f92c01551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92bffff640 1 -- 172.21.15.19:0/1524566121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c0158700 con 0x7f92c00b5620 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92bffff640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c00b5620 0x7f92c0155a70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f92b402a2b0 tx=0x7f92b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92bd7fa640 1 -- 172.21.15.19:0/1524566121 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4037040 con 0x7f92c00b5620 2024-04-25T10:01:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92bd7fa640 1 -- 172.21.15.19:0/1524566121 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b4042070 con 0x7f92c00b5620 2024-04-25T10:01:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92c6e93640 1 -- 172.21.15.19:0/1524566121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c01589e0 con 0x7f92c00b5620 2024-04-25T10:01:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92bd7fa640 1 -- 172.21.15.19:0/1524566121 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4030880 con 0x7f92c00b5620 2024-04-25T10:01:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.631+0000 7f92c6e93640 1 -- 172.21.15.19:0/1524566121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c01f7010 con 0x7f92c00b5620 2024-04-25T10:01:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.632+0000 7f92c6e93640 1 -- 172.21.15.19:0/1524566121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c0073a00 con 0x7f92c00b5620 2024-04-25T10:01:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.633+0000 7f92bd7fa640 1 -- 172.21.15.19:0/1524566121 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92b4030a20 con 0x7f92c00b5620 2024-04-25T10:01:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.633+0000 7f92bd7fa640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92ac06b500 0x7f92ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.633+0000 7f92bd7fa640 1 -- 172.21.15.19:0/1524566121 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f92b40aabd0 con 0x7f92c00b5620 2024-04-25T10:01:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.633+0000 7f92c4c2f640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92ac06b500 0x7f92ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.634+0000 7f92c4c2f640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92ac06b500 0x7f92ac06d9c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f92b8011340 tx=0x7f92b800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.637+0000 7f92bd7fa640 1 -- 172.21.15.19:0/1524566121 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92b40798c0 con 0x7f92c00b5620 2024-04-25T10:01:06.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.818+0000 7f92c6e93640 1 -- 172.21.15.19:0/1524566121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f92c015eaa0 con 0x7f92c00b5620 2024-04-25T10:01:06.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.818+0000 7f92bd7fa640 1 -- 172.21.15.19:0/1524566121 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7f92b40791c0 con 0x7f92c00b5620 2024-04-25T10:01:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:01:06.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 -- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92ac06b500 msgr2=0x7f92ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92ac06b500 0x7f92ac06d9c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f92b8011340 tx=0x7f92b800b040 comp rx=0 tx=0).stop 2024-04-25T10:01:06.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 -- 172.21.15.19:0/1524566121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c00b5620 msgr2=0x7f92c0155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c00b5620 0x7f92c0155a70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f92b402a2b0 tx=0x7f92b40046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 -- 172.21.15.19:0/1524566121 shutdown_connections 2024-04-25T10:01:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92ac06b500 0x7f92ac06d9c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c015b7d0 0x7f92c0155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c00b5620 0x7f92c0155a70 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 --2- 172.21.15.19:0/1524566121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c006c2d0 0x7f92c01551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 -- 172.21.15.19:0/1524566121 >> 172.21.15.19:0/1524566121 conn(0x7f92c00d0800 msgr2=0x7f92c015f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 -- 172.21.15.19:0/1524566121 shutdown_connections 2024-04-25T10:01:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.820+0000 7f92c6e93640 1 -- 172.21.15.19:0/1524566121 wait complete. 2024-04-25T10:01:06.836 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:01:06.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.935+0000 7f0b1275c640 1 Processor -- start 2024-04-25T10:01:06.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.935+0000 7f0b1275c640 1 -- start start 2024-04-25T10:01:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.935+0000 7f0b1275c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b0c0cf600 0x7f0b0c0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.935+0000 7f0b1275c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0b0c0cbf20 0x7f0b0c0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.935+0000 7f0b1275c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0c0c75e0 con 0x7f0b0c15bb50 2024-04-25T10:01:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.935+0000 7f0b1275c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0c0c8680 con 0x7f0b0c0cf600 2024-04-25T10:01:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.935+0000 7f0b1275c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0c0cff80 con 0x7f0b0c0cbf20 2024-04-25T10:01:06.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.936+0000 7f0b0b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b0c0cf600 0x7f0b0c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.936+0000 7f0b0bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0b0c15bb50 0x7f0b0c0c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44146/0 (socket says 172.21.15.19:44146) 2024-04-25T10:01:06.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.936+0000 7f0b0bfff640 1 -- 172.21.15.19:0/2180987380 learned_addr learned my addr 172.21.15.19:0/2180987380 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:06.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.936+0000 7f0b10cf9640 1 --2- 172.21.15.19:0/2180987380 >> v2:172.21.15.112:3300/0 conn(0x7f0b0c0cbf20 0x7f0b0c0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.936+0000 7f0b0b7fe640 1 -- 172.21.15.19:0/2180987380 >> v2:172.21.15.112:3300/0 conn(0x7f0b0c0cbf20 msgr2=0x7f0b0c0cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:06.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.936+0000 7f0b0b7fe640 1 --2- 172.21.15.19:0/2180987380 >> v2:172.21.15.112:3300/0 conn(0x7f0b0c0cbf20 0x7f0b0c0cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.936+0000 7f0b0b7fe640 1 -- 172.21.15.19:0/2180987380 >> v1:172.21.15.19:6789/0 conn(0x7f0b0c15bb50 legacy=0x7f0b0c0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.936+0000 7f0b0affd640 1 -- 172.21.15.19:0/2180987380 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311119589 0 0) 0x7f0b0c0c75e0 con 0x7f0b0c15bb50 2024-04-25T10:01:06.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.936+0000 7f0b0b7fe640 1 -- 172.21.15.19:0/2180987380 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c0c78c0 con 0x7f0b0c0cf600 2024-04-25T10:01:06.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.936+0000 7f0b10cf9640 1 --2- 172.21.15.19:0/2180987380 >> v2:172.21.15.112:3300/0 conn(0x7f0b0c0cbf20 0x7f0b0c0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:06.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b0b7fe640 1 --2- 172.21.15.19:0/2180987380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c0cf600 0x7f0b0c0cfa40 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c06f370 tx=0x7f0b0002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bf809e3c2d57dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b0affd640 1 -- 172.21.15.19:0/2180987380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00037070 con 0x7f0b0c0cf600 2024-04-25T10:01:06.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b0affd640 1 -- 172.21.15.19:0/2180987380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b00032440 con 0x7f0b0c0cf600 2024-04-25T10:01:06.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b0affd640 1 -- 172.21.15.19:0/2180987380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00032810 con 0x7f0b0c0cf600 2024-04-25T10:01:06.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b1275c640 1 -- 172.21.15.19:0/2180987380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c0cf600 msgr2=0x7f0b0c0cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b1275c640 1 --2- 172.21.15.19:0/2180987380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c0cf600 0x7f0b0c0cfa40 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c06f370 tx=0x7f0b0002a230 comp rx=0 tx=0).stop 2024-04-25T10:01:06.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b1275c640 1 -- 172.21.15.19:0/2180987380 shutdown_connections 2024-04-25T10:01:06.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b1275c640 1 --2- 172.21.15.19:0/2180987380 >> v2:172.21.15.112:3300/0 conn(0x7f0b0c0cbf20 0x7f0b0c0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b1275c640 1 --2- 172.21.15.19:0/2180987380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c0cf600 0x7f0b0c0cfa40 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b1275c640 1 -- 172.21.15.19:0/2180987380 >> 172.21.15.19:0/2180987380 conn(0x7f0b0c06f500 msgr2=0x7f0b0c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:06.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b1275c640 1 -- 172.21.15.19:0/2180987380 shutdown_connections 2024-04-25T10:01:06.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.937+0000 7f0b1275c640 1 -- 172.21.15.19:0/2180987380 wait complete. 2024-04-25T10:01:06.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.938+0000 7f0b1275c640 1 Processor -- start 2024-04-25T10:01:06.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.939+0000 7f0b1275c640 1 -- start start 2024-04-25T10:01:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.939+0000 7f0b1275c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b0c0cbf20 0x7f0b0c0af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.939+0000 7f0b1275c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0c0cf600 0x7f0b0c0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.939+0000 7f0b1275c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c15bb50 0x7f0b0c0c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.939+0000 7f0b1275c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0c8160 con 0x7f0b0c15bb50 2024-04-25T10:01:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.939+0000 7f0b1275c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0c7e80 con 0x7f0b0c0cf600 2024-04-25T10:01:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.939+0000 7f0b1275c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0c72a0 con 0x7f0b0c0cbf20 2024-04-25T10:01:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b10cf9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c15bb50 0x7f0b0c0c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b10cf9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c15bb50 0x7f0b0c0c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50774/0 (socket says 172.21.15.19:50774) 2024-04-25T10:01:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b10cf9640 1 -- 172.21.15.19:0/2030945206 learned_addr learned my addr 172.21.15.19:0/2030945206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:06.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b0bfff640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b0c0cbf20 0x7f0b0c0af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b0b7fe640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0c0cf600 0x7f0b0c0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b10cf9640 1 -- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b0c0cbf20 msgr2=0x7f0b0c0af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b10cf9640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b0c0cbf20 0x7f0b0c0af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b10cf9640 1 -- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0c0cf600 msgr2=0x7f0b0c0c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b10cf9640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0c0cf600 0x7f0b0c0c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b10cf9640 1 -- 172.21.15.19:0/2030945206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c0c9ef0 con 0x7f0b0c15bb50 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b10cf9640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c15bb50 0x7f0b0c0c9870 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400cf20 tx=0x7f0b0400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b08ff9640 1 -- 172.21.15.19:0/2030945206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0401a070 con 0x7f0b0c15bb50 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b08ff9640 1 -- 172.21.15.19:0/2030945206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0400d840 con 0x7f0b0c15bb50 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b08ff9640 1 -- 172.21.15.19:0/2030945206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0400dc30 con 0x7f0b0c15bb50 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b1275c640 1 -- 172.21.15.19:0/2030945206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c0ca1d0 con 0x7f0b0c15bb50 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.940+0000 7f0b1275c640 1 -- 172.21.15.19:0/2030945206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c0ca790 con 0x7f0b0c15bb50 2024-04-25T10:01:06.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.941+0000 7f0b1275c640 1 -- 172.21.15.19:0/2030945206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c159120 con 0x7f0b0c15bb50 2024-04-25T10:01:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.942+0000 7f0b08ff9640 1 -- 172.21.15.19:0/2030945206 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b0401d070 con 0x7f0b0c15bb50 2024-04-25T10:01:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.942+0000 7f0b08ff9640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0af806b4d0 0x7f0af806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.942+0000 7f0b08ff9640 1 -- 172.21.15.19:0/2030945206 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b0408ac50 con 0x7f0b0c15bb50 2024-04-25T10:01:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.942+0000 7f0b0bfff640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0af806b4d0 0x7f0af806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.943+0000 7f0b0bfff640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0af806b4d0 0x7f0af806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0afc000f60 tx=0x7f0afc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:06.947+0000 7f0b08ff9640 1 -- 172.21.15.19:0/2030945206 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b0c159120 con 0x7f0b0c15bb50 2024-04-25T10:01:07.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.128+0000 7f0b1275c640 1 -- 172.21.15.19:0/2030945206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b0c0cf150 con 0x7f0b0c15bb50 2024-04-25T10:01:07.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.129+0000 7f0b08ff9640 1 -- 172.21.15.19:0/2030945206 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b04090020 con 0x7f0b0c15bb50 2024-04-25T10:01:07.134 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:01:07.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.130+0000 7f0b1275c640 1 -- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0af806b4d0 msgr2=0x7f0af806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.130+0000 7f0b1275c640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0af806b4d0 0x7f0af806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0afc000f60 tx=0x7f0afc009040 comp rx=0 tx=0).stop 2024-04-25T10:01:07.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.130+0000 7f0b1275c640 1 -- 172.21.15.19:0/2030945206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c15bb50 msgr2=0x7f0b0c0c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.130+0000 7f0b1275c640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c15bb50 0x7f0b0c0c9870 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400cf20 tx=0x7f0b0400a7d0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.130+0000 7f0b1275c640 1 -- 172.21.15.19:0/2030945206 shutdown_connections 2024-04-25T10:01:07.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.131+0000 7f0b1275c640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0af806b4d0 0x7f0af806d990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.131+0000 7f0b1275c640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c15bb50 0x7f0b0c0c9870 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.131+0000 7f0b1275c640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b0c0cf600 0x7f0b0c0c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.131+0000 7f0b1275c640 1 --2- 172.21.15.19:0/2030945206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b0c0cbf20 0x7f0b0c0af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.131+0000 7f0b1275c640 1 -- 172.21.15.19:0/2030945206 >> 172.21.15.19:0/2030945206 conn(0x7f0b0c06f500 msgr2=0x7f0b0c0d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:07.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.131+0000 7f0b1275c640 1 -- 172.21.15.19:0/2030945206 shutdown_connections 2024-04-25T10:01:07.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.131+0000 7f0b1275c640 1 -- 172.21.15.19:0/2030945206 wait complete. 2024-04-25T10:01:07.147 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:01:07.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.245+0000 7f29bebf5640 1 Processor -- start 2024-04-25T10:01:07.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.245+0000 7f29bebf5640 1 -- start start 2024-04-25T10:01:07.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.246+0000 7f29bebf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b80cf600 0x7f29b80cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.246+0000 7f29bebf5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b80c75e0 con 0x7f29b815bb50 2024-04-25T10:01:07.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.246+0000 7f29bebf5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b80c78c0 con 0x7f29b80cbf20 2024-04-25T10:01:07.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.246+0000 7f29bebf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80c8680 con 0x7f29b80cf600 2024-04-25T10:01:07.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.246+0000 7f29bc991640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f29b815bb50 0x7f29b80c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44158/0 (socket says 172.21.15.19:44158) 2024-04-25T10:01:07.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.246+0000 7f29bc991640 1 -- 172.21.15.19:0/2945064413 learned_addr learned my addr 172.21.15.19:0/2945064413 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:07.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b7fff640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b80cf600 0x7f29b80cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226240251 0 0) 0x7f29b80c75e0 con 0x7f29b815bb50 2024-04-25T10:01:07.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29a0003520 con 0x7f29b815bb50 2024-04-25T10:01:07.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304813581 0 0) 0x7f29b80c78c0 con 0x7f29b80cbf20 2024-04-25T10:01:07.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b80c75e0 con 0x7f29b80cbf20 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1978505147 0 0) 0x7f29a0003520 con 0x7f29b815bb50 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b80cf600 msgr2=0x7f29b80cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b80cf600 0x7f29b80cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 >> v1:172.21.15.112:6789/0 conn(0x7f29b80cbf20 legacy=0x7f29b80cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b80c7ba0 con 0x7f29b815bb50 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f29b00019f0 con 0x7f29b815bb50 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 >> v1:172.21.15.19:6789/0 conn(0x7f29b815bb50 legacy=0x7f29b80c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a0003b70 0x7f29a0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b80cf600 0x7f29a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29a0004d40 0x7f29a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80c8680 con 0x7f29a0004d40 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a000ba70 con 0x7f29a0003b70 2024-04-25T10:01:07.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a000bc20 con 0x7f29b80cf600 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29bd192640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29a0004d40 0x7f29a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29b7fff640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b80cf600 0x7f29a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29bc991640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a0003b70 0x7f29a0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29bd192640 1 -- 172.21.15.19:0/2945064413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b80cf600 msgr2=0x7f29a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29bd192640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b80cf600 0x7f29a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29bd192640 1 -- 172.21.15.19:0/2945064413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a0003b70 msgr2=0x7f29a0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.247+0000 7f29bd192640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a0003b70 0x7f29a0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.248+0000 7f29bd192640 1 -- 172.21.15.19:0/2945064413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0003520 con 0x7f29a0004d40 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.248+0000 7f29bd192640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29a0004d40 0x7f29a00051c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f29a80018d0 tx=0x7f29a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22c277c78fbd99f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.248+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80014f0 con 0x7f29a0004d40 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.248+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a80052d0 con 0x7f29a0004d40 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.248+0000 7f29b77fe640 1 -- 172.21.15.19:0/2945064413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a802aca0 con 0x7f29a0004d40 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.249+0000 7f29bebf5640 1 -- 172.21.15.19:0/2945064413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29a0004d40 msgr2=0x7f29a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.249+0000 7f29bebf5640 1 --2- 172.21.15.19:0/2945064413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29a0004d40 0x7f29a00051c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f29a80018d0 tx=0x7f29a0005830 comp rx=0 tx=0).stop 2024-04-25T10:01:07.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.249+0000 7f29bd192640 1 -- 172.21.15.19:0/2945064413 reap_dead start 2024-04-25T10:01:07.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.249+0000 7f29bebf5640 1 -- 172.21.15.19:0/2945064413 shutdown_connections 2024-04-25T10:01:07.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.249+0000 7f29bebf5640 1 -- 172.21.15.19:0/2945064413 >> 172.21.15.19:0/2945064413 conn(0x7f29b806f500 msgr2=0x7f29b806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:07.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.249+0000 7f29bebf5640 1 -- 172.21.15.19:0/2945064413 shutdown_connections 2024-04-25T10:01:07.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.249+0000 7f29bebf5640 1 -- 172.21.15.19:0/2945064413 wait complete. 2024-04-25T10:01:07.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.250+0000 7f29bebf5640 1 Processor -- start 2024-04-25T10:01:07.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.250+0000 7f29bebf5640 1 -- start start 2024-04-25T10:01:07.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29bebf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b815e6a0 0x7f29b815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29bebf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b815c240 0x7f29b8145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29bebf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b815ea80 0x7f29b8146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29bebf5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80c72a0 con 0x7f29b815ea80 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29bebf5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80c8160 con 0x7f29b815c240 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29bebf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b815f6d0 con 0x7f29b815e6a0 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29bd192640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b815ea80 0x7f29b8146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29bd192640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b815ea80 0x7f29b8146330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50784/0 (socket says 172.21.15.19:50784) 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29bd192640 1 -- 172.21.15.19:0/2868848264 learned_addr learned my addr 172.21.15.19:0/2868848264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29bc991640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b815e6a0 0x7f29b815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29b7fff640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b815c240 0x7f29b8145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.251+0000 7f29bd192640 1 -- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b815e6a0 msgr2=0x7f29b815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.252+0000 7f29bd192640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b815e6a0 0x7f29b815c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.252+0000 7f29bd192640 1 -- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b815c240 msgr2=0x7f29b8145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.252+0000 7f29bd192640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b815c240 0x7f29b8145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.252+0000 7f29bd192640 1 -- 172.21.15.19:0/2868848264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b80ca330 con 0x7f29b815ea80 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.252+0000 7f29bd192640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b815ea80 0x7f29b8146330 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f29a802a730 tx=0x7f29a80012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.252+0000 7f29b57fa640 1 -- 172.21.15.19:0/2868848264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80014f0 con 0x7f29b815ea80 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.252+0000 7f29b57fa640 1 -- 172.21.15.19:0/2868848264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a8041070 con 0x7f29b815ea80 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.252+0000 7f29bebf5640 1 -- 172.21.15.19:0/2868848264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b80ca610 con 0x7f29b815ea80 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.252+0000 7f29b57fa640 1 -- 172.21.15.19:0/2868848264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8005480 con 0x7f29b815ea80 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.252+0000 7f29bebf5640 1 -- 172.21.15.19:0/2868848264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b80cabd0 con 0x7f29b815ea80 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.253+0000 7f29bebf5640 1 -- 172.21.15.19:0/2868848264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b8073a00 con 0x7f29b815ea80 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.253+0000 7f29b57fa640 1 -- 172.21.15.19:0/2868848264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29a803a070 con 0x7f29b815ea80 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.254+0000 7f29b57fa640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a406b4d0 0x7f29a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.254+0000 7f29b57fa640 1 -- 172.21.15.19:0/2868848264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29a80a9930 con 0x7f29b815ea80 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.254+0000 7f29bc991640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a406b4d0 0x7f29a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.255+0000 7f29bc991640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a406b4d0 0x7f29a406d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f29b0004bc0 tx=0x7f29b0006040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.258+0000 7f29b57fa640 1 -- 172.21.15.19:0/2868848264 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29a8078620 con 0x7f29b815ea80 2024-04-25T10:01:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.436+0000 7f29bebf5640 1 -- 172.21.15.19:0/2868848264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29b80cf150 con 0x7f29b815ea80 2024-04-25T10:01:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.437+0000 7f29b57fa640 1 -- 172.21.15.19:0/2868848264 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7f29a8077f20 con 0x7f29b815ea80 2024-04-25T10:01:07.442 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:01:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 -- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a406b4d0 msgr2=0x7f29a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a406b4d0 0x7f29a406d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f29b0004bc0 tx=0x7f29b0006040 comp rx=0 tx=0).stop 2024-04-25T10:01:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 -- 172.21.15.19:0/2868848264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b815ea80 msgr2=0x7f29b8146330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b815ea80 0x7f29b8146330 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f29a802a730 tx=0x7f29a80012d0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 -- 172.21.15.19:0/2868848264 shutdown_connections 2024-04-25T10:01:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a406b4d0 0x7f29a406d990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b815ea80 0x7f29b8146330 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b815c240 0x7f29b8145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 --2- 172.21.15.19:0/2868848264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29b815e6a0 0x7f29b815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 -- 172.21.15.19:0/2868848264 >> 172.21.15.19:0/2868848264 conn(0x7f29b806f500 msgr2=0x7f29b80cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 -- 172.21.15.19:0/2868848264 shutdown_connections 2024-04-25T10:01:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.439+0000 7f29bebf5640 1 -- 172.21.15.19:0/2868848264 wait complete. 2024-04-25T10:01:07.455 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:01:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.551+0000 7f5d4f2f7640 1 Processor -- start 2024-04-25T10:01:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.551+0000 7f5d4f2f7640 1 -- start start 2024-04-25T10:01:07.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.552+0000 7f5d4f2f7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d4815b5a0 0x7f5d480b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.552+0000 7f5d4f2f7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d480cf320 con 0x7f5d4815b980 2024-04-25T10:01:07.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.552+0000 7f5d4f2f7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d480cf040 con 0x7f5d48162450 2024-04-25T10:01:07.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.552+0000 7f5d4f2f7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d48161970 con 0x7f5d4815b5a0 2024-04-25T10:01:07.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.552+0000 7f5d4d093640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d4815b5a0 0x7f5d480b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.552+0000 7f5d4c892640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5d4815b980 0x7f5d480ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44166/0 (socket says 172.21.15.19:44166) 2024-04-25T10:01:07.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.552+0000 7f5d4c892640 1 -- 172.21.15.19:0/1107966960 learned_addr learned my addr 172.21.15.19:0/1107966960 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:07.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.553+0000 7f5d4d093640 1 -- 172.21.15.19:0/1107966960 >> v1:172.21.15.112:6789/0 conn(0x7f5d48162450 legacy=0x7f5d480d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:07.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.553+0000 7f5d4d093640 1 -- 172.21.15.19:0/1107966960 >> v1:172.21.15.19:6789/0 conn(0x7f5d4815b980 legacy=0x7f5d480ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.553+0000 7f5d4d093640 1 -- 172.21.15.19:0/1107966960 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48161c50 con 0x7f5d4815b5a0 2024-04-25T10:01:07.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.553+0000 7f5d47fff640 1 -- 172.21.15.19:0/1107966960 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966778883 0 0) 0x7f5d3c001290 con 0x7f5d4815b980 2024-04-25T10:01:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.553+0000 7f5d4d093640 1 --2- 172.21.15.19:0/1107966960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4815b5a0 0x7f5d480b5530 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f5d400097d0 tx=0x7f5d4002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb551a249317a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.553+0000 7f5d47fff640 1 -- 172.21.15.19:0/1107966960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40037070 con 0x7f5d4815b5a0 2024-04-25T10:01:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.553+0000 7f5d47fff640 1 -- 172.21.15.19:0/1107966960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40032440 con 0x7f5d4815b5a0 2024-04-25T10:01:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.553+0000 7f5d47fff640 1 -- 172.21.15.19:0/1107966960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40032810 con 0x7f5d4815b5a0 2024-04-25T10:01:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.554+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1107966960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4815b5a0 msgr2=0x7f5d480b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.554+0000 7f5d4f2f7640 1 --2- 172.21.15.19:0/1107966960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4815b5a0 0x7f5d480b5530 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f5d400097d0 tx=0x7f5d4002a210 comp rx=0 tx=0).stop 2024-04-25T10:01:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.554+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1107966960 shutdown_connections 2024-04-25T10:01:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.554+0000 7f5d4f2f7640 1 --2- 172.21.15.19:0/1107966960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4815b5a0 0x7f5d480b5530 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.554+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1107966960 >> 172.21.15.19:0/1107966960 conn(0x7f5d480d0530 msgr2=0x7f5d480d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.554+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1107966960 shutdown_connections 2024-04-25T10:01:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.554+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1107966960 wait complete. 2024-04-25T10:01:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.555+0000 7f5d4f2f7640 1 Processor -- start 2024-04-25T10:01:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.555+0000 7f5d4f2f7640 1 -- start start 2024-04-25T10:01:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4f2f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4815b5a0 0x7f5d4815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4f2f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d4815b980 0x7f5d480c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4f2f7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d48162450 0x7f5d480c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4f2f7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480ceba0 con 0x7f5d4815b5a0 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4f2f7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480cea20 con 0x7f5d4815b980 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4f2f7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480ced20 con 0x7f5d48162450 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4d093640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4815b5a0 0x7f5d4815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4d093640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4815b5a0 0x7f5d4815fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50804/0 (socket says 172.21.15.19:50804) 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4d093640 1 -- 172.21.15.19:0/1717881469 learned_addr learned my addr 172.21.15.19:0/1717881469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4c892640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d4815b980 0x7f5d480c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4d093640 1 -- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d48162450 msgr2=0x7f5d480c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4d894640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d48162450 0x7f5d480c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4d093640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d48162450 0x7f5d480c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4d093640 1 -- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d4815b980 msgr2=0x7f5d480c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4d093640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d4815b980 0x7f5d480c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4d093640 1 -- 172.21.15.19:0/1717881469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48160380 con 0x7f5d4815b5a0 2024-04-25T10:01:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4c892640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d4815b980 0x7f5d480c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.556+0000 7f5d4d894640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d48162450 0x7f5d480c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.557+0000 7f5d4d093640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4815b5a0 0x7f5d4815fe40 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f5d40035040 tx=0x7f5d400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.557+0000 7f5d45ffb640 1 -- 172.21.15.19:0/1717881469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40037040 con 0x7f5d4815b5a0 2024-04-25T10:01:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.557+0000 7f5d45ffb640 1 -- 172.21.15.19:0/1717881469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40004230 con 0x7f5d4815b5a0 2024-04-25T10:01:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.557+0000 7f5d45ffb640 1 -- 172.21.15.19:0/1717881469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4003c670 con 0x7f5d4815b5a0 2024-04-25T10:01:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.557+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1717881469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4815edc0 con 0x7f5d4815b5a0 2024-04-25T10:01:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.557+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1717881469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d480cad20 con 0x7f5d4815b5a0 2024-04-25T10:01:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.558+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1717881469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4815f2e0 con 0x7f5d4815b5a0 2024-04-25T10:01:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.558+0000 7f5d45ffb640 1 -- 172.21.15.19:0/1717881469 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d4003a070 con 0x7f5d4815b5a0 2024-04-25T10:01:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.559+0000 7f5d45ffb640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d3406b500 0x7f5d3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.559+0000 7f5d45ffb640 1 -- 172.21.15.19:0/1717881469 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5d400aacc0 con 0x7f5d4815b5a0 2024-04-25T10:01:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.559+0000 7f5d4c892640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d3406b500 0x7f5d3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.559+0000 7f5d4c892640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d3406b500 0x7f5d3406d9c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5d4815d6b0 tx=0x7f5d3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.563+0000 7f5d45ffb640 1 -- 172.21.15.19:0/1717881469 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d4815f2e0 con 0x7f5d4815b5a0 2024-04-25T10:01:07.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.756+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1717881469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5d48073a00 con 0x7f5d4815b5a0 2024-04-25T10:01:07.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.756+0000 7f5d45ffb640 1 -- 172.21.15.19:0/1717881469 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v117) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5d40041020 con 0x7f5d4815b5a0 2024-04-25T10:01:07.762 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T10:01:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.758+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d3406b500 msgr2=0x7f5d3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.758+0000 7f5d4f2f7640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d3406b500 0x7f5d3406d9c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5d4815d6b0 tx=0x7f5d3c009040 comp rx=0 tx=0).stop 2024-04-25T10:01:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.758+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1717881469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4815b5a0 msgr2=0x7f5d4815fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.758+0000 7f5d4f2f7640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4815b5a0 0x7f5d4815fe40 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f5d40035040 tx=0x7f5d400046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.758+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1717881469 shutdown_connections 2024-04-25T10:01:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.758+0000 7f5d4f2f7640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d3406b500 0x7f5d3406d9c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.758+0000 7f5d4f2f7640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d48162450 0x7f5d480c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.758+0000 7f5d4f2f7640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d4815b980 0x7f5d480c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.758+0000 7f5d4f2f7640 1 --2- 172.21.15.19:0/1717881469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4815b5a0 0x7f5d4815fe40 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.758+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1717881469 >> 172.21.15.19:0/1717881469 conn(0x7f5d480d0530 msgr2=0x7f5d480d29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.758+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1717881469 shutdown_connections 2024-04-25T10:01:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:07.759+0000 7f5d4f2f7640 1 -- 172.21.15.19:0/1717881469 wait complete. 2024-04-25T10:01:07.775 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:01:07.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:07.775 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:08.285 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:08.287 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:08.413 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:01:08.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:08.413 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:01:08.429 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:01:08.429 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:08.447 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:08.447 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:01:08.447 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:08.484 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:08.485 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:01:08.485 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:08.523 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:08.524 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.620+0000 7fc5de75c640 1 Processor -- start 2024-04-25T10:01:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.620+0000 7fc5de75c640 1 -- start start 2024-04-25T10:01:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.621+0000 7fc5de75c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5d815db70 0x7fc5d8162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.621+0000 7fc5de75c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d8162690 con 0x7fc5d80b5c30 2024-04-25T10:01:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.621+0000 7fc5de75c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d80ceee0 con 0x7fc5d80b6cb0 2024-04-25T10:01:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.621+0000 7fc5de75c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d80d1d10 con 0x7fc5d815db70 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.621+0000 7fc5dccf9640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc5d80b5c30 0x7fc5d80d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49846/0 (socket says 172.21.15.19:49846) 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.621+0000 7fc5dccf9640 1 -- 172.21.15.19:0/4230826267 learned_addr learned my addr 172.21.15.19:0/4230826267 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.621+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248304716 0 0) 0x7fc5d8162690 con 0x7fc5d80b5c30 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.621+0000 7fc5d7fff640 1 --2- 172.21.15.19:0/4230826267 >> v2:172.21.15.112:3300/0 conn(0x7fc5d815db70 0x7fc5d8162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c4003520 con 0x7fc5d80b5c30 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3994423670 0 0) 0x7fc5c4003520 con 0x7fc5d80b5c30 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 >> v2:172.21.15.112:3300/0 conn(0x7fc5d815db70 msgr2=0x7fc5d8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 --2- 172.21.15.19:0/4230826267 >> v2:172.21.15.112:3300/0 conn(0x7fc5d815db70 0x7fc5d8162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 >> v1:172.21.15.112:6789/0 conn(0x7fc5d80b6cb0 legacy=0x7fc5d80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d80d1ff0 con 0x7fc5d80b5c30 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fc5d00019f0 con 0x7fc5d80b5c30 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 >> v1:172.21.15.19:6789/0 conn(0x7fc5d80b5c30 legacy=0x7fc5d80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 --2- 172.21.15.19:0/4230826267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c40039b0 0x7fc5c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 --2- 172.21.15.19:0/4230826267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5c4004800 0x7fc5c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 --2- 172.21.15.19:0/4230826267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c40051c0 0x7fc5c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80d1d10 con 0x7fc5c40039b0 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c400c060 con 0x7fc5c4004800 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c400c260 con 0x7fc5c40051c0 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d7fff640 1 --2- 172.21.15.19:0/4230826267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c40039b0 0x7fc5c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d77fe640 1 --2- 172.21.15.19:0/4230826267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5c4004800 0x7fc5c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5dccf9640 1 --2- 172.21.15.19:0/4230826267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c40051c0 0x7fc5c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d7fff640 1 -- 172.21.15.19:0/4230826267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c40051c0 msgr2=0x7fc5c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d7fff640 1 --2- 172.21.15.19:0/4230826267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c40051c0 0x7fc5c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d7fff640 1 -- 172.21.15.19:0/4230826267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5c4004800 msgr2=0x7fc5c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d7fff640 1 --2- 172.21.15.19:0/4230826267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5c4004800 0x7fc5c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5d7fff640 1 -- 172.21.15.19:0/4230826267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c4003520 con 0x7fc5c40039b0 2024-04-25T10:01:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.622+0000 7fc5dccf9640 1 -- 172.21.15.19:0/4230826267 reap_dead start 2024-04-25T10:01:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.623+0000 7fc5d7fff640 1 --2- 172.21.15.19:0/4230826267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c40039b0 0x7fc5c4003dd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009ac0 tx=0x7fc5c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1b17d9690a9465 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.623+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0036070 con 0x7fc5c40039b0 2024-04-25T10:01:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.623+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c0031400 con 0x7fc5c40039b0 2024-04-25T10:01:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.623+0000 7fc5d6ffd640 1 -- 172.21.15.19:0/4230826267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0009690 con 0x7fc5c40039b0 2024-04-25T10:01:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.624+0000 7fc5de75c640 1 -- 172.21.15.19:0/4230826267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c40039b0 msgr2=0x7fc5c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.624+0000 7fc5de75c640 1 --2- 172.21.15.19:0/4230826267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c40039b0 0x7fc5c4003dd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009ac0 tx=0x7fc5c4005830 comp rx=0 tx=0).stop 2024-04-25T10:01:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.624+0000 7fc5de75c640 1 -- 172.21.15.19:0/4230826267 shutdown_connections 2024-04-25T10:01:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.624+0000 7fc5de75c640 1 --2- 172.21.15.19:0/4230826267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5c40039b0 0x7fc5c4003dd0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.624+0000 7fc5de75c640 1 -- 172.21.15.19:0/4230826267 >> 172.21.15.19:0/4230826267 conn(0x7fc5d80d0520 msgr2=0x7fc5d80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.624+0000 7fc5de75c640 1 -- 172.21.15.19:0/4230826267 shutdown_connections 2024-04-25T10:01:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.624+0000 7fc5de75c640 1 -- 172.21.15.19:0/4230826267 wait complete. 2024-04-25T10:01:08.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.625+0000 7fc5de75c640 1 Processor -- start 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.625+0000 7fc5de75c640 1 -- start start 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5de75c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c40039b0 0x7fc5d81e8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5de75c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d8158e60 0x7fc5d8160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5de75c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81592e0 0x7fc5d81f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5de75c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80ce530 con 0x7fc5d8158e60 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5de75c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d815df50 con 0x7fc5d81592e0 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5de75c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80cf1c0 con 0x7fc5c40039b0 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d8158e60 0x7fc5d8160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d8158e60 0x7fc5d8160410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45812/0 (socket says 172.21.15.19:45812) 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5d77fe640 1 -- 172.21.15.19:0/726939153 learned_addr learned my addr 172.21.15.19:0/726939153 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5d7fff640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c40039b0 0x7fc5d81e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5dccf9640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81592e0 0x7fc5d81f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5d77fe640 1 -- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c40039b0 msgr2=0x7fc5d81e8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5d77fe640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c40039b0 0x7fc5d81e8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5d77fe640 1 -- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81592e0 msgr2=0x7fc5d81f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5d77fe640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81592e0 0x7fc5d81f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5d77fe640 1 -- 172.21.15.19:0/726939153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d81e86b0 con 0x7fc5d8158e60 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.626+0000 7fc5d77fe640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d8158e60 0x7fc5d8160410 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800f7f0 tx=0x7fc5c800fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.627+0000 7fc5d4ff9640 1 -- 172.21.15.19:0/726939153 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80050d0 con 0x7fc5d8158e60 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.627+0000 7fc5d4ff9640 1 -- 172.21.15.19:0/726939153 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c8018070 con 0x7fc5d8158e60 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.627+0000 7fc5de75c640 1 -- 172.21.15.19:0/726939153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d81e6df0 con 0x7fc5d8158e60 2024-04-25T10:01:08.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.627+0000 7fc5d4ff9640 1 -- 172.21.15.19:0/726939153 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8005490 con 0x7fc5d8158e60 2024-04-25T10:01:08.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.627+0000 7fc5de75c640 1 -- 172.21.15.19:0/726939153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d81e73b0 con 0x7fc5d8158e60 2024-04-25T10:01:08.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.628+0000 7fc5de75c640 1 -- 172.21.15.19:0/726939153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d81e7db0 con 0x7fc5d8158e60 2024-04-25T10:01:08.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.628+0000 7fc5d4ff9640 1 -- 172.21.15.19:0/726939153 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5c800d730 con 0x7fc5d8158e60 2024-04-25T10:01:08.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.629+0000 7fc5d4ff9640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b806b4d0 0x7fc5b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.629+0000 7fc5d4ff9640 1 -- 172.21.15.19:0/726939153 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc5c808a0a0 con 0x7fc5d8158e60 2024-04-25T10:01:08.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.629+0000 7fc5d7fff640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b806b4d0 0x7fc5b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.630+0000 7fc5d7fff640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b806b4d0 0x7fc5b806d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0033000 tx=0x7fc5c0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.633+0000 7fc5d4ff9640 1 -- 172.21.15.19:0/726939153 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5c8058730 con 0x7fc5d8158e60 2024-04-25T10:01:08.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.851+0000 7fc5de75c640 1 -- 172.21.15.19:0/726939153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5d8073990 con 0x7fc5d8158e60 2024-04-25T10:01:08.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.852+0000 7fc5d4ff9640 1 -- 172.21.15.19:0/726939153 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc5c805c7d0 con 0x7fc5d8158e60 2024-04-25T10:01:08.857 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:08.857 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:01:08.858 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:01:08.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 -- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b806b4d0 msgr2=0x7fc5b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b806b4d0 0x7fc5b806d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0033000 tx=0x7fc5c0008030 comp rx=0 tx=0).stop 2024-04-25T10:01:08.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 -- 172.21.15.19:0/726939153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d8158e60 msgr2=0x7fc5d8160410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d8158e60 0x7fc5d8160410 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800f7f0 tx=0x7fc5c800fcb0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 -- 172.21.15.19:0/726939153 shutdown_connections 2024-04-25T10:01:08.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b806b4d0 0x7fc5b806d990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d81592e0 0x7fc5d81f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d8158e60 0x7fc5d8160410 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 --2- 172.21.15.19:0/726939153 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5c40039b0 0x7fc5d81e8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 -- 172.21.15.19:0/726939153 >> 172.21.15.19:0/726939153 conn(0x7fc5d80d0520 msgr2=0x7fc5d80c8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:08.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 -- 172.21.15.19:0/726939153 shutdown_connections 2024-04-25T10:01:08.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.854+0000 7fc5de75c640 1 -- 172.21.15.19:0/726939153 wait complete. 2024-04-25T10:01:08.870 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.966+0000 7f71cda41640 1 Processor -- start 2024-04-25T10:01:08.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.966+0000 7f71cda41640 1 -- start start 2024-04-25T10:01:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71cda41640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71c80b6cb0 0x7f71c80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71cda41640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c80ceee0 con 0x7f71c815db70 2024-04-25T10:01:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71cda41640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c80d1d10 con 0x7f71c80b5a60 2024-04-25T10:01:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71cda41640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c80d1ff0 con 0x7f71c80b6cb0 2024-04-25T10:01:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71c6ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f71c815db70 0x7f71c8162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49852/0 (socket says 172.21.15.19:49852) 2024-04-25T10:01:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71c6ffd640 1 -- 172.21.15.19:0/3603331368 learned_addr learned my addr 172.21.15.19:0/3603331368 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71bffff640 1 --2- 172.21.15.19:0/3603331368 >> v2:172.21.15.112:3300/0 conn(0x7f71c80b6cb0 0x7f71c80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71c67fc640 1 -- 172.21.15.19:0/3603331368 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080957443 0 0) 0x7f71c80ceee0 con 0x7f71c815db70 2024-04-25T10:01:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71c67fc640 1 -- 172.21.15.19:0/3603331368 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71b4003520 con 0x7f71c815db70 2024-04-25T10:01:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71c67fc640 1 -- 172.21.15.19:0/3603331368 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1239238903 0 0) 0x7f71b4003520 con 0x7f71c815db70 2024-04-25T10:01:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71bffff640 1 -- 172.21.15.19:0/3603331368 >> v1:172.21.15.112:6789/0 conn(0x7f71c80b5a60 legacy=0x7f71c80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71bffff640 1 -- 172.21.15.19:0/3603331368 >> v1:172.21.15.19:6789/0 conn(0x7f71c815db70 legacy=0x7f71c8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.967+0000 7f71bffff640 1 -- 172.21.15.19:0/3603331368 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c80d22d0 con 0x7f71c80b6cb0 2024-04-25T10:01:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.968+0000 7f71bffff640 1 --2- 172.21.15.19:0/3603331368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c80b6cb0 0x7f71c80b5520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f71b00097a0 tx=0x7f71b002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42ada0d81f80516e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.968+0000 7f71c67fc640 1 -- 172.21.15.19:0/3603331368 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0037070 con 0x7f71c80b6cb0 2024-04-25T10:01:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.968+0000 7f71c67fc640 1 -- 172.21.15.19:0/3603331368 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b002abc0 con 0x7f71c80b6cb0 2024-04-25T10:01:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.968+0000 7f71c67fc640 1 -- 172.21.15.19:0/3603331368 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8001670 con 0x7f71c80b6cb0 2024-04-25T10:01:08.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.969+0000 7f71cda41640 1 -- 172.21.15.19:0/3603331368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c80b6cb0 msgr2=0x7f71c80b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.969+0000 7f71cda41640 1 --2- 172.21.15.19:0/3603331368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c80b6cb0 0x7f71c80b5520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f71b00097a0 tx=0x7f71b002a130 comp rx=0 tx=0).stop 2024-04-25T10:01:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.969+0000 7f71cda41640 1 -- 172.21.15.19:0/3603331368 shutdown_connections 2024-04-25T10:01:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.969+0000 7f71cda41640 1 --2- 172.21.15.19:0/3603331368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c80b6cb0 0x7f71c80b5520 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.969+0000 7f71cda41640 1 -- 172.21.15.19:0/3603331368 >> 172.21.15.19:0/3603331368 conn(0x7f71c80d0520 msgr2=0x7f71c80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.969+0000 7f71cda41640 1 -- 172.21.15.19:0/3603331368 shutdown_connections 2024-04-25T10:01:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.969+0000 7f71cda41640 1 -- 172.21.15.19:0/3603331368 wait complete. 2024-04-25T10:01:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.970+0000 7f71cda41640 1 Processor -- start 2024-04-25T10:01:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.970+0000 7f71cda41640 1 -- start start 2024-04-25T10:01:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71cda41640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c80b5a60 0x7f71c8150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71cda41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c80b6cb0 0x7f71c80c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71cda41640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c815db70 0x7f71c80d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71cda41640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80cf1c0 con 0x7f71c80b5a60 2024-04-25T10:01:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71cda41640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80ce530 con 0x7f71c80b6cb0 2024-04-25T10:01:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71cda41640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80cf340 con 0x7f71c815db70 2024-04-25T10:01:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c80b5a60 0x7f71c8150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c80b5a60 0x7f71c8150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45826/0 (socket says 172.21.15.19:45826) 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71c6ffd640 1 -- 172.21.15.19:0/2728782893 learned_addr learned my addr 172.21.15.19:0/2728782893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71bffff640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c80b6cb0 0x7f71c80c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71c77fe640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c815db70 0x7f71c80d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71c6ffd640 1 -- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c815db70 msgr2=0x7f71c80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71c6ffd640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c815db70 0x7f71c80d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71c6ffd640 1 -- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c80b6cb0 msgr2=0x7f71c80c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71c6ffd640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c80b6cb0 0x7f71c80c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71c6ffd640 1 -- 172.21.15.19:0/2728782893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c8150dd0 con 0x7f71c80b5a60 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.971+0000 7f71bffff640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c80b6cb0 0x7f71c80c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.972+0000 7f71c6ffd640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c80b5a60 0x7f71c8150890 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f71b800c5b0 tx=0x7f71b800ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.972+0000 7f71c4ff9640 1 -- 172.21.15.19:0/2728782893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8017070 con 0x7f71c80b5a60 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.972+0000 7f71c4ff9640 1 -- 172.21.15.19:0/2728782893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b8005ce0 con 0x7f71c80b5a60 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.972+0000 7f71c4ff9640 1 -- 172.21.15.19:0/2728782893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8007330 con 0x7f71c80b5a60 2024-04-25T10:01:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.972+0000 7f71cda41640 1 -- 172.21.15.19:0/2728782893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c80ca1b0 con 0x7f71c80b5a60 2024-04-25T10:01:08.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.972+0000 7f71cda41640 1 -- 172.21.15.19:0/2728782893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c81e7c50 con 0x7f71c80b5a60 2024-04-25T10:01:08.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.973+0000 7f71cda41640 1 -- 172.21.15.19:0/2728782893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c8073990 con 0x7f71c80b5a60 2024-04-25T10:01:08.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.973+0000 7f71c4ff9640 1 -- 172.21.15.19:0/2728782893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71b801a070 con 0x7f71c80b5a60 2024-04-25T10:01:08.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.974+0000 7f71c4ff9640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a806b500 0x7f71a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.974+0000 7f71c4ff9640 1 -- 172.21.15.19:0/2728782893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71b8089890 con 0x7f71c80b5a60 2024-04-25T10:01:08.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.974+0000 7f71bffff640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a806b500 0x7f71a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.975+0000 7f71bffff640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a806b500 0x7f71a806d9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f71b0034000 tx=0x7f71b002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:08.978+0000 7f71c4ff9640 1 -- 172.21.15.19:0/2728782893 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71b8058590 con 0x7f71c80b5a60 2024-04-25T10:01:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.159+0000 7f71cda41640 1 -- 172.21.15.19:0/2728782893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71c80d1820 con 0x7f71c80b5a60 2024-04-25T10:01:09.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.161+0000 7f71c4ff9640 1 -- 172.21.15.19:0/2728782893 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f71b8057e90 con 0x7f71c80b5a60 2024-04-25T10:01:09.166 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:09.166 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":117,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:00:34.061124+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:30.702453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:31.396192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.162+0000 7f71cda41640 1 -- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a806b500 msgr2=0x7f71a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.162+0000 7f71cda41640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a806b500 0x7f71a806d9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f71b0034000 tx=0x7f71b002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.162+0000 7f71cda41640 1 -- 172.21.15.19:0/2728782893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c80b5a60 msgr2=0x7f71c8150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.162+0000 7f71cda41640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c80b5a60 0x7f71c8150890 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f71b800c5b0 tx=0x7f71b800ca70 comp rx=0 tx=0).stop 2024-04-25T10:01:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.162+0000 7f71cda41640 1 -- 172.21.15.19:0/2728782893 shutdown_connections 2024-04-25T10:01:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.162+0000 7f71cda41640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a806b500 0x7f71a806d9c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.162+0000 7f71cda41640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71c815db70 0x7f71c80d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.162+0000 7f71cda41640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71c80b6cb0 0x7f71c80c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.162+0000 7f71cda41640 1 --2- 172.21.15.19:0/2728782893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71c80b5a60 0x7f71c8150890 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.162+0000 7f71cda41640 1 -- 172.21.15.19:0/2728782893 >> 172.21.15.19:0/2728782893 conn(0x7f71c80d0520 msgr2=0x7f71c8157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.163+0000 7f71cda41640 1 -- 172.21.15.19:0/2728782893 shutdown_connections 2024-04-25T10:01:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.163+0000 7f71cda41640 1 -- 172.21.15.19:0/2728782893 wait complete. 2024-04-25T10:01:09.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:00:30.702453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:00:31.396192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:09.180 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:09.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.278+0000 7fcaae8ee640 1 Processor -- start 2024-04-25T10:01:09.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.278+0000 7fcaae8ee640 1 -- start start 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.278+0000 7fcaae8ee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcaa815dc20 0x7fcaa80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.278+0000 7fcaae8ee640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa806cda0 con 0x7fcaa80d32f0 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaae8ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa815e000 con 0x7fcaa8161710 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaae8ee640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa80d1de0 con 0x7fcaa815dc20 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaace8b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcaa80d32f0 0x7fcaa806c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49860/0 (socket says 172.21.15.19:49860) 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaace8b640 1 -- 172.21.15.19:0/3846444249 learned_addr learned my addr 172.21.15.19:0/3846444249 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa7fff640 1 --2- 172.21.15.19:0/3846444249 >> v2:172.21.15.112:3300/0 conn(0x7fcaa815dc20 0x7fcaa80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016734518 0 0) 0x7fcaa806cda0 con 0x7fcaa80d32f0 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca90003520 con 0x7fcaa80d32f0 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2646055259 0 0) 0x7fca90003520 con 0x7fcaa80d32f0 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 >> v2:172.21.15.112:3300/0 conn(0x7fcaa815dc20 msgr2=0x7fcaa80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 --2- 172.21.15.19:0/3846444249 >> v2:172.21.15.112:3300/0 conn(0x7fcaa815dc20 0x7fcaa80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 >> v1:172.21.15.112:6789/0 conn(0x7fcaa8161710 legacy=0x7fcaa806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa8161c60 con 0x7fcaa80d32f0 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fcaa00019f0 con 0x7fcaa80d32f0 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 >> v1:172.21.15.19:6789/0 conn(0x7fcaa80d32f0 legacy=0x7fcaa806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 --2- 172.21.15.19:0/3846444249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca900039b0 0x7fca90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 --2- 172.21.15.19:0/3846444249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca90004800 0x7fca90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 --2- 172.21.15.19:0/3846444249 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca900051c0 0x7fca9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa80d1de0 con 0x7fca900039b0 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9000c060 con 0x7fca90004800 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9000c260 con 0x7fca900051c0 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.279+0000 7fcaa7fff640 1 --2- 172.21.15.19:0/3846444249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca900039b0 0x7fca90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.280+0000 7fcaace8b640 1 --2- 172.21.15.19:0/3846444249 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca900051c0 0x7fca9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.280+0000 7fcaa77fe640 1 --2- 172.21.15.19:0/3846444249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca90004800 0x7fca90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.280+0000 7fcaa7fff640 1 -- 172.21.15.19:0/3846444249 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca900051c0 msgr2=0x7fca9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.280+0000 7fcaa7fff640 1 --2- 172.21.15.19:0/3846444249 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca900051c0 0x7fca9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.280+0000 7fcaa7fff640 1 -- 172.21.15.19:0/3846444249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca90004800 msgr2=0x7fca90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.280+0000 7fcaa7fff640 1 --2- 172.21.15.19:0/3846444249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca90004800 0x7fca90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.280+0000 7fcaa7fff640 1 -- 172.21.15.19:0/3846444249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca90003520 con 0x7fca900039b0 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.280+0000 7fcaace8b640 1 -- 172.21.15.19:0/3846444249 reap_dead start 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.280+0000 7fcaa7fff640 1 --2- 172.21.15.19:0/3846444249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca900039b0 0x7fca90003dd0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fca98009ac0 tx=0x7fca90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e367d90d601a2027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.281+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98036070 con 0x7fca900039b0 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.281+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca98031400 con 0x7fca900039b0 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.281+0000 7fcaa6ffd640 1 -- 172.21.15.19:0/3846444249 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98009690 con 0x7fca900039b0 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.281+0000 7fcaae8ee640 1 -- 172.21.15.19:0/3846444249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca900039b0 msgr2=0x7fca90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.281+0000 7fcaae8ee640 1 --2- 172.21.15.19:0/3846444249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca900039b0 0x7fca90003dd0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fca98009ac0 tx=0x7fca90005830 comp rx=0 tx=0).stop 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.281+0000 7fcaae8ee640 1 -- 172.21.15.19:0/3846444249 shutdown_connections 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.281+0000 7fcaae8ee640 1 --2- 172.21.15.19:0/3846444249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca900039b0 0x7fca90003dd0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.281+0000 7fcaae8ee640 1 -- 172.21.15.19:0/3846444249 >> 172.21.15.19:0/3846444249 conn(0x7fcaa80d0420 msgr2=0x7fcaa80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.281+0000 7fcaae8ee640 1 -- 172.21.15.19:0/3846444249 shutdown_connections 2024-04-25T10:01:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.281+0000 7fcaae8ee640 1 -- 172.21.15.19:0/3846444249 wait complete. 2024-04-25T10:01:09.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.282+0000 7fcaae8ee640 1 Processor -- start 2024-04-25T10:01:09.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.283+0000 7fcaae8ee640 1 -- start start 2024-04-25T10:01:09.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.283+0000 7fcaae8ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca900039b0 0x7fcaa80c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.283+0000 7fcaae8ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa81586a0 0x7fcaa80b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.283+0000 7fcaae8ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcaa81579e0 0x7fcaa8157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.283+0000 7fcaae8ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa8162500 con 0x7fcaa81586a0 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.283+0000 7fcaae8ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa8162220 con 0x7fca900039b0 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.283+0000 7fcaae8ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa80b5e50 con 0x7fcaa81579e0 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.283+0000 7fcaa77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa81586a0 0x7fcaa80b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.283+0000 7fcaa77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa81586a0 0x7fcaa80b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45842/0 (socket says 172.21.15.19:45842) 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.283+0000 7fcaa77fe640 1 -- 172.21.15.19:0/1726709239 learned_addr learned my addr 172.21.15.19:0/1726709239 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaa7fff640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca900039b0 0x7fcaa80c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaace8b640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcaa81579e0 0x7fcaa8157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaa77fe640 1 -- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcaa81579e0 msgr2=0x7fcaa8157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaa77fe640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcaa81579e0 0x7fcaa8157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaa77fe640 1 -- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca900039b0 msgr2=0x7fcaa80c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaa77fe640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca900039b0 0x7fcaa80c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaa77fe640 1 -- 172.21.15.19:0/1726709239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa81f71f0 con 0x7fcaa81586a0 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaace8b640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcaa81579e0 0x7fcaa8157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaa77fe640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa81586a0 0x7fcaa80b50e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fca9c007fd0 tx=0x7fca9c00e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaa4ff9640 1 -- 172.21.15.19:0/1726709239 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c019070 con 0x7fcaa81586a0 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaa4ff9640 1 -- 172.21.15.19:0/1726709239 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c0052e0 con 0x7fcaa81586a0 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaae8ee640 1 -- 172.21.15.19:0/1726709239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa81f74d0 con 0x7fcaa81586a0 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaa4ff9640 1 -- 172.21.15.19:0/1726709239 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c003810 con 0x7fcaa81586a0 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.284+0000 7fcaae8ee640 1 -- 172.21.15.19:0/1726709239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa81f7a90 con 0x7fcaa81586a0 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.285+0000 7fcaae8ee640 1 -- 172.21.15.19:0/1726709239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa80b95d0 con 0x7fcaa81586a0 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.286+0000 7fcaa4ff9640 1 -- 172.21.15.19:0/1726709239 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca9c01c070 con 0x7fcaa81586a0 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.286+0000 7fcaa4ff9640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca9406b500 0x7fca9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.286+0000 7fcaa4ff9640 1 -- 172.21.15.19:0/1726709239 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fca9c089960 con 0x7fcaa81586a0 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.286+0000 7fcaa7fff640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca9406b500 0x7fca9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.287+0000 7fcaa7fff640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca9406b500 0x7fca9406d9c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fca98033000 tx=0x7fca98008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.290+0000 7fcaa4ff9640 1 -- 172.21.15.19:0/1726709239 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcaa80b95d0 con 0x7fcaa81586a0 2024-04-25T10:01:09.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.469+0000 7fcaae8ee640 1 -- 172.21.15.19:0/1726709239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcaa8073a00 con 0x7fcaa81586a0 2024-04-25T10:01:09.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.469+0000 7fcaa4ff9640 1 -- 172.21.15.19:0/1726709239 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca9c058000 con 0x7fcaa81586a0 2024-04-25T10:01:09.475 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:01:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 -- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca9406b500 msgr2=0x7fca9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca9406b500 0x7fca9406d9c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fca98033000 tx=0x7fca98008030 comp rx=0 tx=0).stop 2024-04-25T10:01:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 -- 172.21.15.19:0/1726709239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa81586a0 msgr2=0x7fcaa80b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa81586a0 0x7fcaa80b50e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fca9c007fd0 tx=0x7fca9c00e8a0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 -- 172.21.15.19:0/1726709239 shutdown_connections 2024-04-25T10:01:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca9406b500 0x7fca9406d9c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcaa81579e0 0x7fcaa8157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa81586a0 0x7fcaa80b50e0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 --2- 172.21.15.19:0/1726709239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca900039b0 0x7fcaa80c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 -- 172.21.15.19:0/1726709239 >> 172.21.15.19:0/1726709239 conn(0x7fcaa80d0420 msgr2=0x7fcaa80d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 -- 172.21.15.19:0/1726709239 shutdown_connections 2024-04-25T10:01:09.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.471+0000 7fcaae8ee640 1 -- 172.21.15.19:0/1726709239 wait complete. 2024-04-25T10:01:09.488 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:01:09.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.586+0000 7f161c209640 1 Processor -- start 2024-04-25T10:01:09.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.587+0000 7f161c209640 1 -- start start 2024-04-25T10:01:09.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.587+0000 7f161c209640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f161406c330 0x7f16140b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.587+0000 7f161c209640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f161406c7e0 0x7f16140b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.587+0000 7f161c209640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16140d1d50 con 0x7f1614158230 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f161c209640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1614158610 con 0x7f161406c7e0 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f161c209640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161406bea0 con 0x7f161406c330 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1619fa5640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1614158230 0x7f16140d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49864/0 (socket says 172.21.15.19:49864) 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1619fa5640 1 -- 172.21.15.19:0/2719987134 learned_addr learned my addr 172.21.15.19:0/2719987134 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f161a7a6640 1 --2- 172.21.15.19:0/2719987134 >> v2:172.21.15.112:3300/0 conn(0x7f161406c7e0 0x7f16140b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477112409 0 0) 0x7f16140d1d50 con 0x7f1614158230 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f16197a4640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f161406c330 0x7f16140b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1600003520 con 0x7f1614158230 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2766861751 0 0) 0x7f1600003520 con 0x7f1614158230 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f161406c330 msgr2=0x7f16140b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f161406c330 0x7f16140b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 >> v2:172.21.15.112:3300/0 conn(0x7f161406c7e0 msgr2=0x7f16140b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 --2- 172.21.15.19:0/2719987134 >> v2:172.21.15.112:3300/0 conn(0x7f161406c7e0 0x7f16140b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140d2020 con 0x7f1614158230 2024-04-25T10:01:09.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f1610001670 con 0x7f1614158230 2024-04-25T10:01:09.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 >> v1:172.21.15.19:6789/0 conn(0x7f1614158230 legacy=0x7f16140d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f161406c330 0x7f16000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600003f30 0x7f1600004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1600004d40 0x7f16000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1614158610 con 0x7f1600003f30 2024-04-25T10:01:09.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161406bea0 con 0x7f1600004d40 2024-04-25T10:01:09.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160000b9c0 con 0x7f161406c330 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f161a7a6640 1 --2- 172.21.15.19:0/2719987134 >> v2:172.21.15.112:3300/0 conn(0x7f161406c7e0 0x7f16140b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.588+0000 7f16197a4640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600003f30 0x7f1600004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.589+0000 7f1619fa5640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f161406c330 0x7f16000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.589+0000 7f161a7a6640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1600004d40 0x7f16000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.589+0000 7f16197a4640 1 -- 172.21.15.19:0/2719987134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f161406c330 msgr2=0x7f16000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.589+0000 7f16197a4640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f161406c330 0x7f16000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.589+0000 7f16197a4640 1 -- 172.21.15.19:0/2719987134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1600004d40 msgr2=0x7f16000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.589+0000 7f16197a4640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1600004d40 0x7f16000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.589+0000 7f16197a4640 1 -- 172.21.15.19:0/2719987134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600003520 con 0x7f1600003f30 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.589+0000 7f16197a4640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600003f30 0x7f1600004800 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f1604009ab0 tx=0x7f1600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea98749a53c04549 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.590+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604036070 con 0x7f1600003f30 2024-04-25T10:01:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.590+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1604005990 con 0x7f1600003f30 2024-04-25T10:01:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.590+0000 7f1618fa3640 1 -- 172.21.15.19:0/2719987134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604009680 con 0x7f1600003f30 2024-04-25T10:01:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.590+0000 7f161c209640 1 -- 172.21.15.19:0/2719987134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600003f30 msgr2=0x7f1600004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.590+0000 7f161c209640 1 --2- 172.21.15.19:0/2719987134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1600003f30 0x7f1600004800 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f1604009ab0 tx=0x7f1600005830 comp rx=0 tx=0).stop 2024-04-25T10:01:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.590+0000 7f161a7a6640 1 -- 172.21.15.19:0/2719987134 reap_dead start 2024-04-25T10:01:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.590+0000 7f161c209640 1 -- 172.21.15.19:0/2719987134 shutdown_connections 2024-04-25T10:01:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.590+0000 7f161c209640 1 -- 172.21.15.19:0/2719987134 >> 172.21.15.19:0/2719987134 conn(0x7f1614061600 msgr2=0x7f16140619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.590+0000 7f161c209640 1 -- 172.21.15.19:0/2719987134 shutdown_connections 2024-04-25T10:01:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.590+0000 7f161c209640 1 -- 172.21.15.19:0/2719987134 wait complete. 2024-04-25T10:01:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.591+0000 7f161c209640 1 Processor -- start 2024-04-25T10:01:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.591+0000 7f161c209640 1 -- start start 2024-04-25T10:01:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.592+0000 7f161c209640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1600003f30 0x7f1614153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.592+0000 7f161c209640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16140cd640 0x7f1614148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.592+0000 7f161c209640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1614148b80 0x7f16141451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.592+0000 7f161c209640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16140d25e0 con 0x7f16140cd640 2024-04-25T10:01:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.592+0000 7f161c209640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161406bb30 con 0x7f1614148b80 2024-04-25T10:01:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.592+0000 7f161c209640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161415e320 con 0x7f1600003f30 2024-04-25T10:01:09.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.592+0000 7f16197a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16140cd640 0x7f1614148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.592+0000 7f16197a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16140cd640 0x7f1614148640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45870/0 (socket says 172.21.15.19:45870) 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.592+0000 7f16197a4640 1 -- 172.21.15.19:0/3142150318 learned_addr learned my addr 172.21.15.19:0/3142150318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f1619fa5640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1600003f30 0x7f1614153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f161a7a6640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1614148b80 0x7f16141451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f16197a4640 1 -- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1600003f30 msgr2=0x7f1614153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f16197a4640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1600003f30 0x7f1614153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f16197a4640 1 -- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1614148b80 msgr2=0x7f16141451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f16197a4640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1614148b80 0x7f16141451c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f16197a4640 1 -- 172.21.15.19:0/3142150318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614155f70 con 0x7f16140cd640 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f1619fa5640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1600003f30 0x7f1614153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f16197a4640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16140cd640 0x7f1614148640 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f1604009bd0 tx=0x7f1604004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f160a7fc640 1 -- 172.21.15.19:0/3142150318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604040020 con 0x7f16140cd640 2024-04-25T10:01:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f160a7fc640 1 -- 172.21.15.19:0/3142150318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1604037040 con 0x7f16140cd640 2024-04-25T10:01:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f160a7fc640 1 -- 172.21.15.19:0/3142150318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604036030 con 0x7f16140cd640 2024-04-25T10:01:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f161c209640 1 -- 172.21.15.19:0/3142150318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1614156250 con 0x7f16140cd640 2024-04-25T10:01:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.593+0000 7f161c209640 1 -- 172.21.15.19:0/3142150318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1614156810 con 0x7f16140cd640 2024-04-25T10:01:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.594+0000 7f161c209640 1 -- 172.21.15.19:0/3142150318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1614156bf0 con 0x7f16140cd640 2024-04-25T10:01:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.595+0000 7f160a7fc640 1 -- 172.21.15.19:0/3142150318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1604043020 con 0x7f16140cd640 2024-04-25T10:01:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.595+0000 7f160a7fc640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15f806b500 0x7f15f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.595+0000 7f160a7fc640 1 -- 172.21.15.19:0/3142150318 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16040aa500 con 0x7f16140cd640 2024-04-25T10:01:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.595+0000 7f1619fa5640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15f806b500 0x7f15f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.596+0000 7f1619fa5640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15f806b500 0x7f15f806d9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1610007fd0 tx=0x7f1610009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.599+0000 7f160a7fc640 1 -- 172.21.15.19:0/3142150318 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1614156bf0 con 0x7f16140cd640 2024-04-25T10:01:09.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.778+0000 7f161c209640 1 -- 172.21.15.19:0/3142150318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f161406de50 con 0x7f16140cd640 2024-04-25T10:01:09.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.779+0000 7f160a7fc640 1 -- 172.21.15.19:0/3142150318 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f160404a030 con 0x7f16140cd640 2024-04-25T10:01:09.784 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:01:09.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 -- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15f806b500 msgr2=0x7f15f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15f806b500 0x7f15f806d9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1610007fd0 tx=0x7f1610009040 comp rx=0 tx=0).stop 2024-04-25T10:01:09.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 -- 172.21.15.19:0/3142150318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16140cd640 msgr2=0x7f1614148640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16140cd640 0x7f1614148640 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f1604009bd0 tx=0x7f1604004730 comp rx=0 tx=0).stop 2024-04-25T10:01:09.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 -- 172.21.15.19:0/3142150318 shutdown_connections 2024-04-25T10:01:09.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f15f806b500 0x7f15f806d9c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1614148b80 0x7f16141451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16140cd640 0x7f1614148640 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 --2- 172.21.15.19:0/3142150318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1600003f30 0x7f1614153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 -- 172.21.15.19:0/3142150318 >> 172.21.15.19:0/3142150318 conn(0x7f1614061600 msgr2=0x7f16140cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 -- 172.21.15.19:0/3142150318 shutdown_connections 2024-04-25T10:01:09.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.781+0000 7f161c209640 1 -- 172.21.15.19:0/3142150318 wait complete. 2024-04-25T10:01:09.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:01:09.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.896+0000 7ff29d72b640 1 Processor -- start 2024-04-25T10:01:09.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.896+0000 7ff29d72b640 1 -- start start 2024-04-25T10:01:09.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff29d72b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff298161710 0x7ff2980d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff29d72b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29815e000 con 0x7ff2980ced80 2024-04-25T10:01:09.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff29d72b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2980d1de0 con 0x7ff29815dc20 2024-04-25T10:01:09.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff29d72b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff298161c60 con 0x7ff298161710 2024-04-25T10:01:09.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff2967fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff2980ced80 0x7ff2980d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49872/0 (socket says 172.21.15.19:49872) 2024-04-25T10:01:09.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff2967fc640 1 -- 172.21.15.19:0/2313048667 learned_addr learned my addr 172.21.15.19:0/2313048667 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:09.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff2977fe640 1 --2- 172.21.15.19:0/2313048667 >> v2:172.21.15.112:3300/0 conn(0x7ff298161710 0x7ff2980d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234527654 0 0) 0x7ff29815e000 con 0x7ff2980ced80 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff280003520 con 0x7ff2980ced80 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532679241 0 0) 0x7ff2980d1de0 con 0x7ff29815dc20 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff29815e000 con 0x7ff29815dc20 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3660941419 0 0) 0x7ff280003520 con 0x7ff2980ced80 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 >> v2:172.21.15.112:3300/0 conn(0x7ff298161710 msgr2=0x7ff2980d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff295ffb640 1 --2- 172.21.15.19:0/2313048667 >> v2:172.21.15.112:3300/0 conn(0x7ff298161710 0x7ff2980d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 >> v1:172.21.15.112:6789/0 conn(0x7ff29815dc20 legacy=0x7ff2980ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.897+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298161f40 con 0x7ff2980ced80 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7ff28c001670 con 0x7ff2980ced80 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 >> v1:172.21.15.19:6789/0 conn(0x7ff2980ced80 legacy=0x7ff2980d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff295ffb640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff280003b40 0x7ff280003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff295ffb640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 0x7ff280004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff295ffb640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2800051c0 0x7ff28000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff298161c60 con 0x7ff280004800 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28000c040 con 0x7ff2800051c0 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28000c240 con 0x7ff280003b40 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff2967fc640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 0x7ff280004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff296ffd640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff280003b40 0x7ff280003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff2977fe640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2800051c0 0x7ff28000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff2967fc640 1 -- 172.21.15.19:0/2313048667 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff280003b40 msgr2=0x7ff280003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff2967fc640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff280003b40 0x7ff280003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff2967fc640 1 -- 172.21.15.19:0/2313048667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2800051c0 msgr2=0x7ff28000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff2967fc640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2800051c0 0x7ff28000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff2967fc640 1 -- 172.21.15.19:0/2313048667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280003520 con 0x7ff280004800 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff2977fe640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2800051c0 0x7ff28000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.898+0000 7ff2977fe640 1 -- 172.21.15.19:0/2313048667 reap_dead start 2024-04-25T10:01:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.899+0000 7ff2967fc640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 0x7ff280004c80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c190 tx=0x7ff280005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1f00b9fe56c8db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.899+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c007cc0 con 0x7ff280004800 2024-04-25T10:01:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.899+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c005af0 con 0x7ff280004800 2024-04-25T10:01:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.899+0000 7ff295ffb640 1 -- 172.21.15.19:0/2313048667 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c007070 con 0x7ff280004800 2024-04-25T10:01:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.899+0000 7ff29d72b640 1 -- 172.21.15.19:0/2313048667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 msgr2=0x7ff280004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.899+0000 7ff29d72b640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 0x7ff280004c80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c190 tx=0x7ff280005830 comp rx=0 tx=0).stop 2024-04-25T10:01:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.899+0000 7ff29d72b640 1 -- 172.21.15.19:0/2313048667 shutdown_connections 2024-04-25T10:01:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.899+0000 7ff29d72b640 1 --2- 172.21.15.19:0/2313048667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 0x7ff280004c80 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.899+0000 7ff29d72b640 1 -- 172.21.15.19:0/2313048667 >> 172.21.15.19:0/2313048667 conn(0x7ff2980d0420 msgr2=0x7ff2980d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.899+0000 7ff29d72b640 1 -- 172.21.15.19:0/2313048667 shutdown_connections 2024-04-25T10:01:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.899+0000 7ff29d72b640 1 -- 172.21.15.19:0/2313048667 wait complete. 2024-04-25T10:01:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.901+0000 7ff29d72b640 1 Processor -- start 2024-04-25T10:01:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.901+0000 7ff29d72b640 1 -- start start 2024-04-25T10:01:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.901+0000 7ff29d72b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 0x7ff29815ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.901+0000 7ff29d72b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2980cc850 0x7ff2980cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.901+0000 7ff29d72b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2981e7ef0 0x7ff2981e8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.901+0000 7ff29d72b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980b5e50 con 0x7ff280004800 2024-04-25T10:01:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.901+0000 7ff29d72b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff298162500 con 0x7ff2981e7ef0 2024-04-25T10:01:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.901+0000 7ff29d72b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980b5fd0 con 0x7ff2980cc850 2024-04-25T10:01:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff296ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 0x7ff29815ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff296ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 0x7ff29815ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45876/0 (socket says 172.21.15.19:45876) 2024-04-25T10:01:09.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff296ffd640 1 -- 172.21.15.19:0/981509407 learned_addr learned my addr 172.21.15.19:0/981509407 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:09.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff2977fe640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2981e7ef0 0x7ff2981e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff296ffd640 1 -- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2980cc850 msgr2=0x7ff2980cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff296ffd640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2980cc850 0x7ff2980cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff296ffd640 1 -- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2981e7ef0 msgr2=0x7ff2981e8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff296ffd640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2981e7ef0 0x7ff2981e8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff296ffd640 1 -- 172.21.15.19:0/981509407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2981e8c70 con 0x7ff280004800 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff296ffd640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 0x7ff29815ce40 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7ff28800f7d0 tx=0x7ff28800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff27f7fe640 1 -- 172.21.15.19:0/981509407 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28800fe80 con 0x7ff280004800 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff27f7fe640 1 -- 172.21.15.19:0/981509407 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff288018070 con 0x7ff280004800 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff29d72b640 1 -- 172.21.15.19:0/981509407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2981e8f50 con 0x7ff280004800 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff27f7fe640 1 -- 172.21.15.19:0/981509407 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28800d6d0 con 0x7ff280004800 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.902+0000 7ff29d72b640 1 -- 172.21.15.19:0/981509407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2981f3fb0 con 0x7ff280004800 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.903+0000 7ff29d72b640 1 -- 172.21.15.19:0/981509407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff298073a00 con 0x7ff280004800 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.904+0000 7ff27f7fe640 1 -- 172.21.15.19:0/981509407 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff28800d870 con 0x7ff280004800 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.904+0000 7ff27f7fe640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28406b4d0 0x7ff28406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.904+0000 7ff27f7fe640 1 -- 172.21.15.19:0/981509407 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff28808a2f0 con 0x7ff280004800 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.904+0000 7ff2967fc640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28406b4d0 0x7ff28406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.905+0000 7ff2967fc640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28406b4d0 0x7ff28406d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff28c032000 tx=0x7ff28c007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:09.908+0000 7ff27f7fe640 1 -- 172.21.15.19:0/981509407 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff288058fe0 con 0x7ff280004800 2024-04-25T10:01:10.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.086+0000 7ff29d72b640 1 -- 172.21.15.19:0/981509407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2981f4180 con 0x7ff280004800 2024-04-25T10:01:10.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.087+0000 7ff27f7fe640 1 -- 172.21.15.19:0/981509407 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2880588e0 con 0x7ff280004800 2024-04-25T10:01:10.092 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:01:10.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.088+0000 7ff29d72b640 1 -- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28406b4d0 msgr2=0x7ff28406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.088+0000 7ff29d72b640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28406b4d0 0x7ff28406d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff28c032000 tx=0x7ff28c007000 comp rx=0 tx=0).stop 2024-04-25T10:01:10.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.088+0000 7ff29d72b640 1 -- 172.21.15.19:0/981509407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 msgr2=0x7ff29815ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.088+0000 7ff29d72b640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 0x7ff29815ce40 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7ff28800f7d0 tx=0x7ff28800fc90 comp rx=0 tx=0).stop 2024-04-25T10:01:10.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.088+0000 7ff29d72b640 1 -- 172.21.15.19:0/981509407 shutdown_connections 2024-04-25T10:01:10.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.088+0000 7ff29d72b640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28406b4d0 0x7ff28406d990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.088+0000 7ff29d72b640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2981e7ef0 0x7ff2981e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.088+0000 7ff29d72b640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2980cc850 0x7ff2980cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.088+0000 7ff29d72b640 1 --2- 172.21.15.19:0/981509407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff280004800 0x7ff29815ce40 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.088+0000 7ff29d72b640 1 -- 172.21.15.19:0/981509407 >> 172.21.15.19:0/981509407 conn(0x7ff2980d0420 msgr2=0x7ff2980b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.089+0000 7ff29d72b640 1 -- 172.21.15.19:0/981509407 shutdown_connections 2024-04-25T10:01:10.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.089+0000 7ff29d72b640 1 -- 172.21.15.19:0/981509407 wait complete. 2024-04-25T10:01:10.105 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:10.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.201+0000 7fb8902bc640 1 Processor -- start 2024-04-25T10:01:10.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.201+0000 7fb8902bc640 1 -- start start 2024-04-25T10:01:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.202+0000 7fb8902bc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb88815db70 0x7fb888162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.202+0000 7fb8902bc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb888162690 con 0x7fb8880b5c30 2024-04-25T10:01:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.202+0000 7fb8902bc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8880ceee0 con 0x7fb8880b6cb0 2024-04-25T10:01:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.202+0000 7fb8902bc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880d1d10 con 0x7fb88815db70 2024-04-25T10:01:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.202+0000 7fb88e859640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb8880b5c30 0x7fb8880d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49880/0 (socket says 172.21.15.19:49880) 2024-04-25T10:01:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.202+0000 7fb88e859640 1 -- 172.21.15.19:0/1609355830 learned_addr learned my addr 172.21.15.19:0/1609355830 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.202+0000 7fb88e058640 1 --2- 172.21.15.19:0/1609355830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb88815db70 0x7fb888162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.202+0000 7fb88d056640 1 -- 172.21.15.19:0/1609355830 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727791188 0 0) 0x7fb888162690 con 0x7fb8880b5c30 2024-04-25T10:01:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.202+0000 7fb88d056640 1 -- 172.21.15.19:0/1609355830 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb874003520 con 0x7fb8880b5c30 2024-04-25T10:01:10.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.202+0000 7fb88d056640 1 -- 172.21.15.19:0/1609355830 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801099375 0 0) 0x7fb8880ceee0 con 0x7fb8880b6cb0 2024-04-25T10:01:10.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.203+0000 7fb88d056640 1 -- 172.21.15.19:0/1609355830 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb888162690 con 0x7fb8880b6cb0 2024-04-25T10:01:10.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.203+0000 7fb88e058640 1 -- 172.21.15.19:0/1609355830 >> v1:172.21.15.112:6789/0 conn(0x7fb8880b6cb0 legacy=0x7fb8880b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.203+0000 7fb88e058640 1 -- 172.21.15.19:0/1609355830 >> v1:172.21.15.19:6789/0 conn(0x7fb8880b5c30 legacy=0x7fb8880d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.203+0000 7fb88d056640 1 -- 172.21.15.19:0/1609355830 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 708709318 0 0) 0x7fb874003520 con 0x7fb8880b5c30 2024-04-25T10:01:10.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.203+0000 7fb88e058640 1 -- 172.21.15.19:0/1609355830 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8880d1ff0 con 0x7fb88815db70 2024-04-25T10:01:10.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.203+0000 7fb88e058640 1 --2- 172.21.15.19:0/1609355830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb88815db70 0x7fb888162150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb8840097d0 tx=0x7fb88402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=453e2632d1842fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.204+0000 7fb88d056640 1 -- 172.21.15.19:0/1609355830 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884037070 con 0x7fb88815db70 2024-04-25T10:01:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.204+0000 7fb88d056640 1 -- 172.21.15.19:0/1609355830 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88402ab90 con 0x7fb88815db70 2024-04-25T10:01:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.204+0000 7fb88d056640 1 -- 172.21.15.19:0/1609355830 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8800019f0 con 0x7fb88815db70 2024-04-25T10:01:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.204+0000 7fb8902bc640 1 -- 172.21.15.19:0/1609355830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb88815db70 msgr2=0x7fb888162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.204+0000 7fb8902bc640 1 --2- 172.21.15.19:0/1609355830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb88815db70 0x7fb888162150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb8840097d0 tx=0x7fb88402a100 comp rx=0 tx=0).stop 2024-04-25T10:01:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.204+0000 7fb8902bc640 1 -- 172.21.15.19:0/1609355830 shutdown_connections 2024-04-25T10:01:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.204+0000 7fb8902bc640 1 --2- 172.21.15.19:0/1609355830 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb88815db70 0x7fb888162150 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.204+0000 7fb8902bc640 1 -- 172.21.15.19:0/1609355830 >> 172.21.15.19:0/1609355830 conn(0x7fb8880d0520 msgr2=0x7fb8880d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.204+0000 7fb8902bc640 1 -- 172.21.15.19:0/1609355830 shutdown_connections 2024-04-25T10:01:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.204+0000 7fb8902bc640 1 -- 172.21.15.19:0/1609355830 wait complete. 2024-04-25T10:01:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.205+0000 7fb8902bc640 1 Processor -- start 2024-04-25T10:01:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.205+0000 7fb8902bc640 1 -- start start 2024-04-25T10:01:10.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.206+0000 7fb8902bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8880b5c30 0x7fb8880c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.206+0000 7fb8902bc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8880b6cb0 0x7fb88815c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.206+0000 7fb8902bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb88815db70 0x7fb88815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.206+0000 7fb8902bc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880ce530 con 0x7fb88815db70 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.206+0000 7fb8902bc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88815df50 con 0x7fb8880b5c30 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.206+0000 7fb8902bc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880cf1c0 con 0x7fb8880b6cb0 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.206+0000 7fb88e859640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb88815db70 0x7fb88815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.206+0000 7fb88e859640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb88815db70 0x7fb88815c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45884/0 (socket says 172.21.15.19:45884) 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.206+0000 7fb88e859640 1 -- 172.21.15.19:0/1975627380 learned_addr learned my addr 172.21.15.19:0/1975627380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb88e058640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8880b5c30 0x7fb8880c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb88d857640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8880b6cb0 0x7fb88815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb88e859640 1 -- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8880b6cb0 msgr2=0x7fb88815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb88e859640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8880b6cb0 0x7fb88815c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb88e859640 1 -- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8880b5c30 msgr2=0x7fb8880c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb88e859640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8880b5c30 0x7fb8880c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb88e859640 1 -- 172.21.15.19:0/1975627380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88815ee90 con 0x7fb88815db70 2024-04-25T10:01:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb88d857640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8880b6cb0 0x7fb88815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:10.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb88e859640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb88815db70 0x7fb88815c870 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fb880005c00 tx=0x7fb88000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb87effd640 1 -- 172.21.15.19:0/1975627380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880017070 con 0x7fb88815db70 2024-04-25T10:01:10.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb87effd640 1 -- 172.21.15.19:0/1975627380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88000ad60 con 0x7fb88815db70 2024-04-25T10:01:10.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb8902bc640 1 -- 172.21.15.19:0/1975627380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb888152a90 con 0x7fb88815db70 2024-04-25T10:01:10.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb87effd640 1 -- 172.21.15.19:0/1975627380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880012640 con 0x7fb88815db70 2024-04-25T10:01:10.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.207+0000 7fb8902bc640 1 -- 172.21.15.19:0/1975627380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb888153050 con 0x7fb88815db70 2024-04-25T10:01:10.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.208+0000 7fb8902bc640 1 -- 172.21.15.19:0/1975627380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb888073990 con 0x7fb88815db70 2024-04-25T10:01:10.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.209+0000 7fb87effd640 1 -- 172.21.15.19:0/1975627380 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb88001a070 con 0x7fb88815db70 2024-04-25T10:01:10.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.209+0000 7fb87effd640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb86c06b4d0 0x7fb86c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.209+0000 7fb87effd640 1 -- 172.21.15.19:0/1975627380 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb88008a130 con 0x7fb88815db70 2024-04-25T10:01:10.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.209+0000 7fb88e058640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb86c06b4d0 0x7fb86c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.210+0000 7fb88e058640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb86c06b4d0 0x7fb86c06d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb884034000 tx=0x7fb88402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.213+0000 7fb87effd640 1 -- 172.21.15.19:0/1975627380 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb880058860 con 0x7fb88815db70 2024-04-25T10:01:10.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.434+0000 7fb8902bc640 1 -- 172.21.15.19:0/1975627380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8880b59e0 con 0x7fb88815db70 2024-04-25T10:01:10.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.434+0000 7fb87effd640 1 -- 172.21.15.19:0/1975627380 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb88005c900 con 0x7fb88815db70 2024-04-25T10:01:10.440 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:10.440 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:01:10.440 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:01:10.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.436+0000 7fb8902bc640 1 -- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb86c06b4d0 msgr2=0x7fb86c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.436+0000 7fb8902bc640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb86c06b4d0 0x7fb86c06d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb884034000 tx=0x7fb88402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:10.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.436+0000 7fb8902bc640 1 -- 172.21.15.19:0/1975627380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb88815db70 msgr2=0x7fb88815c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.436+0000 7fb8902bc640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb88815db70 0x7fb88815c870 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fb880005c00 tx=0x7fb88000a430 comp rx=0 tx=0).stop 2024-04-25T10:01:10.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.436+0000 7fb8902bc640 1 -- 172.21.15.19:0/1975627380 shutdown_connections 2024-04-25T10:01:10.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.436+0000 7fb8902bc640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb86c06b4d0 0x7fb86c06d990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.436+0000 7fb8902bc640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb88815db70 0x7fb88815c870 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.436+0000 7fb8902bc640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8880b6cb0 0x7fb88815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.436+0000 7fb8902bc640 1 --2- 172.21.15.19:0/1975627380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8880b5c30 0x7fb8880c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.436+0000 7fb8902bc640 1 -- 172.21.15.19:0/1975627380 >> 172.21.15.19:0/1975627380 conn(0x7fb8880d0520 msgr2=0x7fb88815a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.437+0000 7fb8902bc640 1 -- 172.21.15.19:0/1975627380 shutdown_connections 2024-04-25T10:01:10.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.437+0000 7fb8902bc640 1 -- 172.21.15.19:0/1975627380 wait complete. 2024-04-25T10:01:10.453 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:10.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.549+0000 7f8f2d26b640 1 Processor -- start 2024-04-25T10:01:10.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.549+0000 7f8f2d26b640 1 -- start start 2024-04-25T10:01:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.549+0000 7f8f2d26b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f2815dbe0 0x7f8f280d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.549+0000 7f8f2d26b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8f280b92a0 0x7f8f280d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.549+0000 7f8f2d26b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f280b97a0 con 0x7f8f280b2cb0 2024-04-25T10:01:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.549+0000 7f8f2d26b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2815dfc0 con 0x7f8f2815dbe0 2024-04-25T10:01:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.549+0000 7f8f2d26b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f280d1a00 con 0x7f8f280b92a0 2024-04-25T10:01:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.550+0000 7f8f26ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f2815dbe0 0x7f8f280d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.550+0000 7f8f1ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8f280b2cb0 0x7f8f280b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49888/0 (socket says 172.21.15.19:49888) 2024-04-25T10:01:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.550+0000 7f8f1ffff640 1 -- 172.21.15.19:0/85482477 learned_addr learned my addr 172.21.15.19:0/85482477 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.550+0000 7f8f277fe640 1 --2- 172.21.15.19:0/85482477 >> v2:172.21.15.112:3300/0 conn(0x7f8f280b92a0 0x7f8f280d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.550+0000 7f8f26ffd640 1 -- 172.21.15.19:0/85482477 >> v2:172.21.15.112:3300/0 conn(0x7f8f280b92a0 msgr2=0x7f8f280d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.550+0000 7f8f26ffd640 1 --2- 172.21.15.19:0/85482477 >> v2:172.21.15.112:3300/0 conn(0x7f8f280b92a0 0x7f8f280d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.550+0000 7f8f26ffd640 1 -- 172.21.15.19:0/85482477 >> v1:172.21.15.19:6789/0 conn(0x7f8f280b2cb0 legacy=0x7f8f280b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.550+0000 7f8f26ffd640 1 -- 172.21.15.19:0/85482477 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f280d1c50 con 0x7f8f2815dbe0 2024-04-25T10:01:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.550+0000 7f8f267fc640 1 -- 172.21.15.19:0/85482477 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777693246 0 0) 0x7f8f280b97a0 con 0x7f8f280b2cb0 2024-04-25T10:01:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.551+0000 7f8f26ffd640 1 --2- 172.21.15.19:0/85482477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f2815dbe0 0x7f8f280d0910 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f8f280cfb90 tx=0x7f8f1802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ebee413121dfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.551+0000 7f8f267fc640 1 -- 172.21.15.19:0/85482477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18037070 con 0x7f8f2815dbe0 2024-04-25T10:01:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.551+0000 7f8f267fc640 1 -- 172.21.15.19:0/85482477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f18032440 con 0x7f8f2815dbe0 2024-04-25T10:01:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.551+0000 7f8f267fc640 1 -- 172.21.15.19:0/85482477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1802ae60 con 0x7f8f2815dbe0 2024-04-25T10:01:10.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.551+0000 7f8f2d26b640 1 -- 172.21.15.19:0/85482477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f2815dbe0 msgr2=0x7f8f280d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.551+0000 7f8f2d26b640 1 --2- 172.21.15.19:0/85482477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f2815dbe0 0x7f8f280d0910 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f8f280cfb90 tx=0x7f8f1802a110 comp rx=0 tx=0).stop 2024-04-25T10:01:10.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.551+0000 7f8f2d26b640 1 -- 172.21.15.19:0/85482477 shutdown_connections 2024-04-25T10:01:10.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.551+0000 7f8f2d26b640 1 --2- 172.21.15.19:0/85482477 >> v2:172.21.15.112:3300/0 conn(0x7f8f280b92a0 0x7f8f280d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.552+0000 7f8f2d26b640 1 --2- 172.21.15.19:0/85482477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f2815dbe0 0x7f8f280d0910 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.552+0000 7f8f2d26b640 1 -- 172.21.15.19:0/85482477 >> 172.21.15.19:0/85482477 conn(0x7f8f28061a20 msgr2=0x7f8f28061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.552+0000 7f8f2d26b640 1 -- 172.21.15.19:0/85482477 shutdown_connections 2024-04-25T10:01:10.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.552+0000 7f8f2d26b640 1 -- 172.21.15.19:0/85482477 wait complete. 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.553+0000 7f8f2d26b640 1 Processor -- start 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.553+0000 7f8f2d26b640 1 -- start start 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.553+0000 7f8f2d26b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f280b2cb0 0x7f8f280b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.553+0000 7f8f2d26b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f280b92a0 0x7f8f280b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.553+0000 7f8f2d26b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f2815dbe0 0x7f8f2814cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.553+0000 7f8f2d26b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280b6ab0 con 0x7f8f280b92a0 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.553+0000 7f8f2d26b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280d2210 con 0x7f8f280b2cb0 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.553+0000 7f8f2d26b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280b6c30 con 0x7f8f2815dbe0 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f280b92a0 0x7f8f280b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f280b92a0 0x7f8f280b5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45904/0 (socket says 172.21.15.19:45904) 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1ffff640 1 -- 172.21.15.19:0/2237640562 learned_addr learned my addr 172.21.15.19:0/2237640562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1ffff640 1 -- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f2815dbe0 msgr2=0x7f8f2814cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1ffff640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f2815dbe0 0x7f8f2814cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1ffff640 1 -- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f280b2cb0 msgr2=0x7f8f280b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1ffff640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f280b2cb0 0x7f8f280b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1ffff640 1 -- 172.21.15.19:0/2237640562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f280b3930 con 0x7f8f280b92a0 2024-04-25T10:01:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1ffff640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f280b92a0 0x7f8f280b5500 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000c4b0 tx=0x7f8f1000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1f7fe640 1 -- 172.21.15.19:0/2237640562 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f10018070 con 0x7f8f280b92a0 2024-04-25T10:01:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1f7fe640 1 -- 172.21.15.19:0/2237640562 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f10007500 con 0x7f8f280b92a0 2024-04-25T10:01:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f2d26b640 1 -- 172.21.15.19:0/2237640562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f280b3c10 con 0x7f8f280b92a0 2024-04-25T10:01:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f1f7fe640 1 -- 172.21.15.19:0/2237640562 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f10004a20 con 0x7f8f280b92a0 2024-04-25T10:01:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.554+0000 7f8f2d26b640 1 -- 172.21.15.19:0/2237640562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f280b41d0 con 0x7f8f280b92a0 2024-04-25T10:01:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.555+0000 7f8f2d26b640 1 -- 172.21.15.19:0/2237640562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f28073990 con 0x7f8f280b92a0 2024-04-25T10:01:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.556+0000 7f8f1f7fe640 1 -- 172.21.15.19:0/2237640562 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f1001b070 con 0x7f8f280b92a0 2024-04-25T10:01:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.556+0000 7f8f1f7fe640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f0806b4d0 0x7f8f0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.556+0000 7f8f1f7fe640 1 -- 172.21.15.19:0/2237640562 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f100896c0 con 0x7f8f280b92a0 2024-04-25T10:01:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.556+0000 7f8f26ffd640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f0806b4d0 0x7f8f0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.557+0000 7f8f26ffd640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f0806b4d0 0x7f8f0806d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8f18034000 tx=0x7f8f1802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.560+0000 7f8f1f7fe640 1 -- 172.21.15.19:0/2237640562 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f10058470 con 0x7f8f280b92a0 2024-04-25T10:01:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.745+0000 7f8f2d26b640 1 -- 172.21.15.19:0/2237640562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f2806b080 con 0x7f8f280b92a0 2024-04-25T10:01:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.746+0000 7f8f1f7fe640 1 -- 172.21.15.19:0/2237640562 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f8f10057d70 con 0x7f8f280b92a0 2024-04-25T10:01:10.752 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:10.752 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":117,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:00:34.061124+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:30.702453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:31.396192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 -- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f0806b4d0 msgr2=0x7f8f0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f0806b4d0 0x7f8f0806d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8f18034000 tx=0x7f8f1802c000 comp rx=0 tx=0).stop 2024-04-25T10:01:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 -- 172.21.15.19:0/2237640562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f280b92a0 msgr2=0x7f8f280b5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f280b92a0 0x7f8f280b5500 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000c4b0 tx=0x7f8f1000c970 comp rx=0 tx=0).stop 2024-04-25T10:01:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 -- 172.21.15.19:0/2237640562 shutdown_connections 2024-04-25T10:01:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f0806b4d0 0x7f8f0806d990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f2815dbe0 0x7f8f2814cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f280b92a0 0x7f8f280b5500 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 --2- 172.21.15.19:0/2237640562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f280b2cb0 0x7f8f280b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 -- 172.21.15.19:0/2237640562 >> 172.21.15.19:0/2237640562 conn(0x7f8f28061a20 msgr2=0x7f8f2806c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 -- 172.21.15.19:0/2237640562 shutdown_connections 2024-04-25T10:01:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.748+0000 7f8f2d26b640 1 -- 172.21.15.19:0/2237640562 wait complete. 2024-04-25T10:01:10.765 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:01:10.765 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.860+0000 7f78ffc1f640 1 Processor -- start 2024-04-25T10:01:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.861+0000 7f78ffc1f640 1 -- start start 2024-04-25T10:01:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.861+0000 7f78ffc1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78f80b2cb0 0x7f78f80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.861+0000 7f78ffc1f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f78f80d1020 0x7f78f80d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.861+0000 7f78ffc1f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f80b97a0 con 0x7f78f815dbe0 2024-04-25T10:01:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.861+0000 7f78ffc1f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f815dfc0 con 0x7f78f80d1020 2024-04-25T10:01:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.861+0000 7f78ffc1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80d1a00 con 0x7f78f80b2cb0 2024-04-25T10:01:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.861+0000 7f78fe1bc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f78f80d1020 0x7f78f80d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.861+0000 7f78fd9bb640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f78f815dbe0 0x7f78f80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49892/0 (socket says 172.21.15.19:49892) 2024-04-25T10:01:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.861+0000 7f78fd9bb640 1 -- 172.21.15.19:0/2490753437 learned_addr learned my addr 172.21.15.19:0/2490753437 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.862+0000 7f78fd1ba640 1 --2- 172.21.15.19:0/2490753437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78f80b2cb0 0x7f78f80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.862+0000 7f78fe1bc640 1 -- 172.21.15.19:0/2490753437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78f80b2cb0 msgr2=0x7f78f80d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.862+0000 7f78fe1bc640 1 --2- 172.21.15.19:0/2490753437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78f80b2cb0 0x7f78f80d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.862+0000 7f78fe1bc640 1 -- 172.21.15.19:0/2490753437 >> v1:172.21.15.19:6789/0 conn(0x7f78f815dbe0 legacy=0x7f78f80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.862+0000 7f78fc9b9640 1 -- 172.21.15.19:0/2490753437 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953329160 0 0) 0x7f78f80b97a0 con 0x7f78f815dbe0 2024-04-25T10:01:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.862+0000 7f78fe1bc640 1 -- 172.21.15.19:0/2490753437 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f80d1c50 con 0x7f78f80d1020 2024-04-25T10:01:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.862+0000 7f78fe1bc640 1 --2- 172.21.15.19:0/2490753437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78f80d1020 0x7f78f80d2c70 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009d90 tx=0x7f78e802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa12de0fd118da9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.863+0000 7f78fc9b9640 1 -- 172.21.15.19:0/2490753437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8007740 con 0x7f78f80d1020 2024-04-25T10:01:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.863+0000 7f78fc9b9640 1 -- 172.21.15.19:0/2490753437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e80078e0 con 0x7f78f80d1020 2024-04-25T10:01:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.863+0000 7f78fc9b9640 1 -- 172.21.15.19:0/2490753437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8007cb0 con 0x7f78f80d1020 2024-04-25T10:01:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.863+0000 7f78ffc1f640 1 -- 172.21.15.19:0/2490753437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78f80d1020 msgr2=0x7f78f80d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.863+0000 7f78ffc1f640 1 --2- 172.21.15.19:0/2490753437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78f80d1020 0x7f78f80d2c70 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009d90 tx=0x7f78e802a660 comp rx=0 tx=0).stop 2024-04-25T10:01:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.863+0000 7f78ffc1f640 1 -- 172.21.15.19:0/2490753437 shutdown_connections 2024-04-25T10:01:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.863+0000 7f78ffc1f640 1 --2- 172.21.15.19:0/2490753437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78f80d1020 0x7f78f80d2c70 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.863+0000 7f78ffc1f640 1 --2- 172.21.15.19:0/2490753437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78f80b2cb0 0x7f78f80d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.863+0000 7f78ffc1f640 1 -- 172.21.15.19:0/2490753437 >> 172.21.15.19:0/2490753437 conn(0x7f78f8061a20 msgr2=0x7f78f8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.863+0000 7f78ffc1f640 1 -- 172.21.15.19:0/2490753437 shutdown_connections 2024-04-25T10:01:10.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.863+0000 7f78ffc1f640 1 -- 172.21.15.19:0/2490753437 wait complete. 2024-04-25T10:01:10.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.864+0000 7f78ffc1f640 1 Processor -- start 2024-04-25T10:01:10.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.865+0000 7f78ffc1f640 1 -- start start 2024-04-25T10:01:10.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.865+0000 7f78ffc1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78f80b2cb0 0x7f78f81556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.865+0000 7f78ffc1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78f80d1020 0x7f78f80b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.865+0000 7f78ffc1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78f815dbe0 0x7f78f8153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.865+0000 7f78ffc1f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80b92a0 con 0x7f78f815dbe0 2024-04-25T10:01:10.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.865+0000 7f78ffc1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80d2210 con 0x7f78f80b2cb0 2024-04-25T10:01:10.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.865+0000 7f78ffc1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80b9420 con 0x7f78f80d1020 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.865+0000 7f78fe1bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78f815dbe0 0x7f78f8153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.865+0000 7f78fe1bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78f815dbe0 0x7f78f8153bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45910/0 (socket says 172.21.15.19:45910) 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.865+0000 7f78fe1bc640 1 -- 172.21.15.19:0/4163308438 learned_addr learned my addr 172.21.15.19:0/4163308438 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78fd9bb640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78f80b2cb0 0x7f78f81556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78fd1ba640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78f80d1020 0x7f78f80b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78fe1bc640 1 -- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78f80d1020 msgr2=0x7f78f80b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78fe1bc640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78f80d1020 0x7f78f80b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78fe1bc640 1 -- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78f80b2cb0 msgr2=0x7f78f81556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78fe1bc640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78f80b2cb0 0x7f78f81556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78fe1bc640 1 -- 172.21.15.19:0/4163308438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f8155cc0 con 0x7f78f815dbe0 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78fd9bb640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78f80b2cb0 0x7f78f81556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78fe1bc640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78f815dbe0 0x7f78f8153bf0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009eb0 tx=0x7f78e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78f67fc640 1 -- 172.21.15.19:0/4163308438 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8040070 con 0x7f78f815dbe0 2024-04-25T10:01:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78f67fc640 1 -- 172.21.15.19:0/4163308438 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e8031bc0 con 0x7f78f815dbe0 2024-04-25T10:01:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78ffc1f640 1 -- 172.21.15.19:0/4163308438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f8155fa0 con 0x7f78f815dbe0 2024-04-25T10:01:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78f67fc640 1 -- 172.21.15.19:0/4163308438 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8045460 con 0x7f78f815dbe0 2024-04-25T10:01:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.866+0000 7f78ffc1f640 1 -- 172.21.15.19:0/4163308438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f8156560 con 0x7f78f815dbe0 2024-04-25T10:01:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.867+0000 7f78ffc1f640 1 -- 172.21.15.19:0/4163308438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f80b5930 con 0x7f78f815dbe0 2024-04-25T10:01:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.868+0000 7f78f67fc640 1 -- 172.21.15.19:0/4163308438 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78e8039040 con 0x7f78f815dbe0 2024-04-25T10:01:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.868+0000 7f78f67fc640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78e406b4d0 0x7f78e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.868+0000 7f78f67fc640 1 -- 172.21.15.19:0/4163308438 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78e80abd50 con 0x7f78f815dbe0 2024-04-25T10:01:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.868+0000 7f78fd9bb640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78e406b4d0 0x7f78e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.869+0000 7f78fd9bb640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78e406b4d0 0x7f78e406d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f78f0000f60 tx=0x7f78f0007cc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:10.872+0000 7f78f67fc640 1 -- 172.21.15.19:0/4163308438 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78e80af030 con 0x7f78f815dbe0 2024-04-25T10:01:11.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.090+0000 7f78ffc1f640 1 -- 172.21.15.19:0/4163308438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78f815cc50 con 0x7f78f815dbe0 2024-04-25T10:01:11.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.090+0000 7f78f67fc640 1 -- 172.21.15.19:0/4163308438 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f78e807aad0 con 0x7f78f815dbe0 2024-04-25T10:01:11.096 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:11.096 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:01:11.096 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 -- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78e406b4d0 msgr2=0x7f78e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78e406b4d0 0x7f78e406d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f78f0000f60 tx=0x7f78f0007cc0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 -- 172.21.15.19:0/4163308438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78f815dbe0 msgr2=0x7f78f8153bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78f815dbe0 0x7f78f8153bf0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009eb0 tx=0x7f78e8004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 -- 172.21.15.19:0/4163308438 shutdown_connections 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78e406b4d0 0x7f78e406d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78f815dbe0 0x7f78f8153bf0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78f80d1020 0x7f78f80b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 --2- 172.21.15.19:0/4163308438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78f80b2cb0 0x7f78f81556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 -- 172.21.15.19:0/4163308438 >> 172.21.15.19:0/4163308438 conn(0x7f78f8061a20 msgr2=0x7f78f80d2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 -- 172.21.15.19:0/4163308438 shutdown_connections 2024-04-25T10:01:11.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.092+0000 7f78ffc1f640 1 -- 172.21.15.19:0/4163308438 wait complete. 2024-04-25T10:01:11.109 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.205+0000 7f93604dd640 1 Processor -- start 2024-04-25T10:01:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.205+0000 7f93604dd640 1 -- start start 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f93604dd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93580cc6c0 0x7f9358071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f93604dd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f935815b320 0x7f9358071770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f93604dd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9358071ed0 con 0x7f935813d190 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f93604dd640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9358071cb0 con 0x7f935815b320 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f93604dd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935813d570 con 0x7f93580cc6c0 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f935ea7a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f935815b320 0x7f9358071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f935ea7a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f935815b320 0x7f9358071770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45914/0 (socket says 172.21.15.19:45914) 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f935ea7a640 1 -- 172.21.15.19:0/2495598880 learned_addr learned my addr 172.21.15.19:0/2495598880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f935da78640 1 --2- 172.21.15.19:0/2495598880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93580cc6c0 0x7f9358071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f935ea7a640 1 -- 172.21.15.19:0/2495598880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93580cc6c0 msgr2=0x7f9358071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f935ea7a640 1 --2- 172.21.15.19:0/2495598880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93580cc6c0 0x7f9358071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f935ea7a640 1 -- 172.21.15.19:0/2495598880 >> v1:172.21.15.112:6789/0 conn(0x7f935813d190 legacy=0x7f93580cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.206+0000 7f935ea7a640 1 -- 172.21.15.19:0/2495598880 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935815a840 con 0x7f935815b320 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.207+0000 7f935ea7a640 1 --2- 172.21.15.19:0/2495598880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935815b320 0x7f9358071770 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f9350009e50 tx=0x7f935002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd6ca4a9d9e50c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.207+0000 7f935d277640 1 -- 172.21.15.19:0/2495598880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350036070 con 0x7f935815b320 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.207+0000 7f935d277640 1 -- 172.21.15.19:0/2495598880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9350031600 con 0x7f935815b320 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.207+0000 7f935d277640 1 -- 172.21.15.19:0/2495598880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500319d0 con 0x7f935815b320 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.207+0000 7f93604dd640 1 -- 172.21.15.19:0/2495598880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935815b320 msgr2=0x7f9358071770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.207+0000 7f93604dd640 1 --2- 172.21.15.19:0/2495598880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935815b320 0x7f9358071770 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f9350009e50 tx=0x7f935002a680 comp rx=0 tx=0).stop 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.208+0000 7f93604dd640 1 -- 172.21.15.19:0/2495598880 shutdown_connections 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.208+0000 7f93604dd640 1 --2- 172.21.15.19:0/2495598880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935815b320 0x7f9358071770 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.208+0000 7f93604dd640 1 --2- 172.21.15.19:0/2495598880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93580cc6c0 0x7f9358071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.208+0000 7f93604dd640 1 -- 172.21.15.19:0/2495598880 >> 172.21.15.19:0/2495598880 conn(0x7f935806abe0 msgr2=0x7f935806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.208+0000 7f93604dd640 1 -- 172.21.15.19:0/2495598880 shutdown_connections 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.208+0000 7f93604dd640 1 -- 172.21.15.19:0/2495598880 wait complete. 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.209+0000 7f93604dd640 1 Processor -- start 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.209+0000 7f93604dd640 1 -- start start 2024-04-25T10:01:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.209+0000 7f93604dd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93580cc6c0 0x7f935815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.209+0000 7f93604dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935813d190 0x7f93581396e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.209+0000 7f93604dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f935815b320 0x7f9358167a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.209+0000 7f93604dd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93580cf620 con 0x7f935813d190 2024-04-25T10:01:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.209+0000 7f93604dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935815ae00 con 0x7f935815b320 2024-04-25T10:01:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.209+0000 7f93604dd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93580cf7a0 con 0x7f93580cc6c0 2024-04-25T10:01:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935da78640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935813d190 0x7f93581396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935da78640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935813d190 0x7f93581396e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45916/0 (socket says 172.21.15.19:45916) 2024-04-25T10:01:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935da78640 1 -- 172.21.15.19:0/1127746964 learned_addr learned my addr 172.21.15.19:0/1127746964 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935e279640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93580cc6c0 0x7f935815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935ea7a640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f935815b320 0x7f9358167a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935da78640 1 -- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93580cc6c0 msgr2=0x7f935815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935da78640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93580cc6c0 0x7f935815d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935da78640 1 -- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f935815b320 msgr2=0x7f9358167a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935da78640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f935815b320 0x7f9358167a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935da78640 1 -- 172.21.15.19:0/1127746964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935815c130 con 0x7f935813d190 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935e279640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93580cc6c0 0x7f935815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935ea7a640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f935815b320 0x7f9358167a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f935da78640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935813d190 0x7f93581396e0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f934800e9e0 tx=0x7f934800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f934effd640 1 -- 172.21.15.19:0/1127746964 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348017070 con 0x7f935813d190 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f934effd640 1 -- 172.21.15.19:0/1127746964 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934800c9d0 con 0x7f935813d190 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f93604dd640 1 -- 172.21.15.19:0/1127746964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935815c410 con 0x7f935813d190 2024-04-25T10:01:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f934effd640 1 -- 172.21.15.19:0/1127746964 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934800cdc0 con 0x7f935813d190 2024-04-25T10:01:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.210+0000 7f93604dd640 1 -- 172.21.15.19:0/1127746964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935815ccb0 con 0x7f935813d190 2024-04-25T10:01:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.211+0000 7f93604dd640 1 -- 172.21.15.19:0/1127746964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935815c5e0 con 0x7f935813d190 2024-04-25T10:01:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.212+0000 7f934effd640 1 -- 172.21.15.19:0/1127746964 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f934801a070 con 0x7f935813d190 2024-04-25T10:01:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.212+0000 7f934effd640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933c06b4d0 0x7f933c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.212+0000 7f934effd640 1 -- 172.21.15.19:0/1127746964 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f934808a7e0 con 0x7f935813d190 2024-04-25T10:01:11.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.212+0000 7f935e279640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933c06b4d0 0x7f933c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.213+0000 7f935e279640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933c06b4d0 0x7f933c06d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9354001700 tx=0x7f9354009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.216+0000 7f934effd640 1 -- 172.21.15.19:0/1127746964 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9348059450 con 0x7f935813d190 2024-04-25T10:01:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.434+0000 7f93604dd640 1 -- 172.21.15.19:0/1127746964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93580685e0 con 0x7f935813d190 2024-04-25T10:01:11.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.435+0000 7f934effd640 1 -- 172.21.15.19:0/1127746964 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f93580685e0 con 0x7f935813d190 2024-04-25T10:01:11.440 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:11.440 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:32.412582+0000","modified":"2024-04-25T10:00:34.423175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3442229953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3442229953},{"type":"v1","addr":"172.21.15.19:6834","nonce":3442229953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:01:11.441 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 72 2024-04-25T10:01:11.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 -- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933c06b4d0 msgr2=0x7f933c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933c06b4d0 0x7f933c06d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9354001700 tx=0x7f9354009040 comp rx=0 tx=0).stop 2024-04-25T10:01:11.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 -- 172.21.15.19:0/1127746964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935813d190 msgr2=0x7f93581396e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935813d190 0x7f93581396e0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f934800e9e0 tx=0x7f934800eea0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 -- 172.21.15.19:0/1127746964 shutdown_connections 2024-04-25T10:01:11.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933c06b4d0 0x7f933c06d990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f935815b320 0x7f9358167a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935813d190 0x7f93581396e0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 --2- 172.21.15.19:0/1127746964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93580cc6c0 0x7f935815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 -- 172.21.15.19:0/1127746964 >> 172.21.15.19:0/1127746964 conn(0x7f935806abe0 msgr2=0x7f93581579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 -- 172.21.15.19:0/1127746964 shutdown_connections 2024-04-25T10:01:11.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.438+0000 7f93604dd640 1 -- 172.21.15.19:0/1127746964 wait complete. 2024-04-25T10:01:11.453 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:11.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.549+0000 7fc5d44cb640 1 Processor -- start 2024-04-25T10:01:11.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.549+0000 7fc5d44cb640 1 -- start start 2024-04-25T10:01:11.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.549+0000 7fc5d44cb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5cc1602e0 0x7fc5cc0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.549+0000 7fc5d44cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc15f5f0 0x7fc5cc15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.549+0000 7fc5d44cb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5cc0cd710 con 0x7fc5cc15ba00 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.549+0000 7fc5d44cb640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc099b70 con 0x7fc5cc1602e0 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.549+0000 7fc5d44cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc099e50 con 0x7fc5cc15f5f0 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.550+0000 7fc5d1a66640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5cc1602e0 0x7fc5cc0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.550+0000 7fc5d2a68640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc15f5f0 0x7fc5cc15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.550+0000 7fc5d2267640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc5cc15ba00 0x7fc5cc09a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:49640/0 (socket says 172.21.15.19:49640) 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.550+0000 7fc5d2267640 1 -- 172.21.15.19:0/1404623732 learned_addr learned my addr 172.21.15.19:0/1404623732 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.550+0000 7fc5d2a68640 1 -- 172.21.15.19:0/1404623732 >> v2:172.21.15.112:3300/0 conn(0x7fc5cc1602e0 msgr2=0x7fc5cc0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.550+0000 7fc5d2a68640 1 --2- 172.21.15.19:0/1404623732 >> v2:172.21.15.112:3300/0 conn(0x7fc5cc1602e0 0x7fc5cc0ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.550+0000 7fc5d2a68640 1 -- 172.21.15.19:0/1404623732 >> v1:172.21.15.112:6789/0 conn(0x7fc5cc15ba00 legacy=0x7fc5cc09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.550+0000 7fc5d1265640 1 -- 172.21.15.19:0/1404623732 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546950132 0 0) 0x7fc5c8001110 con 0x7fc5cc15ba00 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.550+0000 7fc5d2a68640 1 -- 172.21.15.19:0/1404623732 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc09a130 con 0x7fc5cc15f5f0 2024-04-25T10:01:11.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.550+0000 7fc5d1a66640 1 --2- 172.21.15.19:0/1404623732 >> v2:172.21.15.112:3300/0 conn(0x7fc5cc1602e0 0x7fc5cc0ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:11.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.551+0000 7fc5d2a68640 1 --2- 172.21.15.19:0/1404623732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc15f5f0 0x7fc5cc15fa80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009d90 tx=0x7fc5c402a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6de0f701616ee3e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.551+0000 7fc5d1265640 1 -- 172.21.15.19:0/1404623732 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4007740 con 0x7fc5cc15f5f0 2024-04-25T10:01:11.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.551+0000 7fc5d1265640 1 -- 172.21.15.19:0/1404623732 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c40078e0 con 0x7fc5cc15f5f0 2024-04-25T10:01:11.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.551+0000 7fc5d1265640 1 -- 172.21.15.19:0/1404623732 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4007cb0 con 0x7fc5cc15f5f0 2024-04-25T10:01:11.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.552+0000 7fc5d44cb640 1 -- 172.21.15.19:0/1404623732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc15f5f0 msgr2=0x7fc5cc15fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.552+0000 7fc5d44cb640 1 --2- 172.21.15.19:0/1404623732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc15f5f0 0x7fc5cc15fa80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009d90 tx=0x7fc5c402a530 comp rx=0 tx=0).stop 2024-04-25T10:01:11.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.552+0000 7fc5d44cb640 1 -- 172.21.15.19:0/1404623732 shutdown_connections 2024-04-25T10:01:11.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.552+0000 7fc5d44cb640 1 --2- 172.21.15.19:0/1404623732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc15f5f0 0x7fc5cc15fa80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.552+0000 7fc5d44cb640 1 --2- 172.21.15.19:0/1404623732 >> v2:172.21.15.112:3300/0 conn(0x7fc5cc1602e0 0x7fc5cc0ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.552+0000 7fc5d44cb640 1 -- 172.21.15.19:0/1404623732 >> 172.21.15.19:0/1404623732 conn(0x7fc5cc0b3270 msgr2=0x7fc5cc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.552+0000 7fc5d44cb640 1 -- 172.21.15.19:0/1404623732 shutdown_connections 2024-04-25T10:01:11.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.552+0000 7fc5d44cb640 1 -- 172.21.15.19:0/1404623732 wait complete. 2024-04-25T10:01:11.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.553+0000 7fc5d44cb640 1 Processor -- start 2024-04-25T10:01:11.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.553+0000 7fc5d44cb640 1 -- start start 2024-04-25T10:01:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d44cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc15ba00 0x7fc5cc159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d44cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc15f5f0 0x7fc5cc1f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d44cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc1602e0 0x7fc5cc0cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d44cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0cd460 con 0x7fc5cc15ba00 2024-04-25T10:01:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d44cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0cd2e0 con 0x7fc5cc15f5f0 2024-04-25T10:01:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d44cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0c52d0 con 0x7fc5cc1602e0 2024-04-25T10:01:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d2267640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc15ba00 0x7fc5cc159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d2267640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc15ba00 0x7fc5cc159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45922/0 (socket says 172.21.15.19:45922) 2024-04-25T10:01:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d2267640 1 -- 172.21.15.19:0/673953475 learned_addr learned my addr 172.21.15.19:0/673953475 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d1a66640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc15f5f0 0x7fc5cc1f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d2a68640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc1602e0 0x7fc5cc0cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d2267640 1 -- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc1602e0 msgr2=0x7fc5cc0cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d2267640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc1602e0 0x7fc5cc0cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d2267640 1 -- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc15f5f0 msgr2=0x7fc5cc1f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d2267640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc15f5f0 0x7fc5cc1f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d2267640 1 -- 172.21.15.19:0/673953475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc06bd90 con 0x7fc5cc15ba00 2024-04-25T10:01:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d2a68640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc1602e0 0x7fc5cc0cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5d2267640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc15ba00 0x7fc5cc159da0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c780 tx=0x7fc5c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5c2ffd640 1 -- 172.21.15.19:0/673953475 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8018070 con 0x7fc5cc15ba00 2024-04-25T10:01:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.554+0000 7fc5c2ffd640 1 -- 172.21.15.19:0/673953475 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c80072d0 con 0x7fc5cc15ba00 2024-04-25T10:01:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.555+0000 7fc5d44cb640 1 -- 172.21.15.19:0/673953475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc06c000 con 0x7fc5cc15ba00 2024-04-25T10:01:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.555+0000 7fc5c2ffd640 1 -- 172.21.15.19:0/673953475 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80047e0 con 0x7fc5cc15ba00 2024-04-25T10:01:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.555+0000 7fc5d44cb640 1 -- 172.21.15.19:0/673953475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc1f4050 con 0x7fc5cc15ba00 2024-04-25T10:01:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.555+0000 7fc5d44cb640 1 -- 172.21.15.19:0/673953475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc073990 con 0x7fc5cc15ba00 2024-04-25T10:01:11.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.556+0000 7fc5c2ffd640 1 -- 172.21.15.19:0/673953475 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5c801b070 con 0x7fc5cc15ba00 2024-04-25T10:01:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.556+0000 7fc5c2ffd640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b006b4d0 0x7fc5b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.557+0000 7fc5c2ffd640 1 -- 172.21.15.19:0/673953475 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc5c8089530 con 0x7fc5cc15ba00 2024-04-25T10:01:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.557+0000 7fc5d1a66640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b006b4d0 0x7fc5b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.557+0000 7fc5d1a66640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b006b4d0 0x7fc5b006d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009560 tx=0x7fc5bc009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.561+0000 7fc5c2ffd640 1 -- 172.21.15.19:0/673953475 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5cc0cd710 con 0x7fc5cc15ba00 2024-04-25T10:01:11.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.743+0000 7fc5d44cb640 1 -- 172.21.15.19:0/673953475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5cc0b40a0 con 0x7fc5cc15ba00 2024-04-25T10:01:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.744+0000 7fc5c2ffd640 1 -- 172.21.15.19:0/673953475 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16940 (secure 0 0 0) 0x7fc5c8057c60 con 0x7fc5cc15ba00 2024-04-25T10:01:11.774 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":117,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:00:34.061124+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:30.702453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:31.396192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 -- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b006b4d0 msgr2=0x7fc5b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b006b4d0 0x7fc5b006d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009560 tx=0x7fc5bc009190 comp rx=0 tx=0).stop 2024-04-25T10:01:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 -- 172.21.15.19:0/673953475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc15ba00 msgr2=0x7fc5cc159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc15ba00 0x7fc5cc159da0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c780 tx=0x7fc5c800cc40 comp rx=0 tx=0).stop 2024-04-25T10:01:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 -- 172.21.15.19:0/673953475 shutdown_connections 2024-04-25T10:01:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5b006b4d0 0x7fc5b006d990 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc1602e0 0x7fc5cc0cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc15f5f0 0x7fc5cc1f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 --2- 172.21.15.19:0/673953475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5cc15ba00 0x7fc5cc159da0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 -- 172.21.15.19:0/673953475 >> 172.21.15.19:0/673953475 conn(0x7fc5cc0b3270 msgr2=0x7fc5cc0ceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 -- 172.21.15.19:0/673953475 shutdown_connections 2024-04-25T10:01:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.746+0000 7fc5d44cb640 1 -- 172.21.15.19:0/673953475 wait complete. 2024-04-25T10:01:11.777 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:01:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda61c4c640 1 Processor -- start 2024-04-25T10:01:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda61c4c640 1 -- start start 2024-04-25T10:01:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda61c4c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fda5c0d34c0 0x7fda5c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda61c4c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda5c0d08e0 0x7fda5c0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda61c4c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5c0d2420 con 0x7fda5c15dac0 2024-04-25T10:01:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda61c4c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0d0d50 con 0x7fda5c0d08e0 2024-04-25T10:01:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda61c4c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0d3030 con 0x7fda5c0d34c0 2024-04-25T10:01:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda5bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda5c0d08e0 0x7fda5c0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda5b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fda5c15dac0 0x7fda5c0d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49894/0 (socket says 172.21.15.19:49894) 2024-04-25T10:01:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda5b7fe640 1 -- 172.21.15.19:0/2700149608 learned_addr learned my addr 172.21.15.19:0/2700149608 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda5affd640 1 --2- 172.21.15.19:0/2700149608 >> v2:172.21.15.112:3300/0 conn(0x7fda5c0d34c0 0x7fda5c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda5bfff640 1 -- 172.21.15.19:0/2700149608 >> v2:172.21.15.112:3300/0 conn(0x7fda5c0d34c0 msgr2=0x7fda5c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda5bfff640 1 --2- 172.21.15.19:0/2700149608 >> v2:172.21.15.112:3300/0 conn(0x7fda5c0d34c0 0x7fda5c0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda5bfff640 1 -- 172.21.15.19:0/2700149608 >> v1:172.21.15.19:6789/0 conn(0x7fda5c15dac0 legacy=0x7fda5c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda5a7fc640 1 -- 172.21.15.19:0/2700149608 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687620967 0 0) 0x7fda5c0d2420 con 0x7fda5c15dac0 2024-04-25T10:01:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.875+0000 7fda5bfff640 1 -- 172.21.15.19:0/2700149608 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0d2700 con 0x7fda5c0d08e0 2024-04-25T10:01:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.876+0000 7fda5bfff640 1 --2- 172.21.15.19:0/2700149608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda5c0d08e0 0x7fda5c0b6990 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fda5c161590 tx=0x7fda5402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3d528680f56e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.876+0000 7fda5a7fc640 1 -- 172.21.15.19:0/2700149608 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54007740 con 0x7fda5c0d08e0 2024-04-25T10:01:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.876+0000 7fda5a7fc640 1 -- 172.21.15.19:0/2700149608 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda540078e0 con 0x7fda5c0d08e0 2024-04-25T10:01:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.876+0000 7fda5a7fc640 1 -- 172.21.15.19:0/2700149608 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54007cb0 con 0x7fda5c0d08e0 2024-04-25T10:01:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.877+0000 7fda61c4c640 1 -- 172.21.15.19:0/2700149608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda5c0d08e0 msgr2=0x7fda5c0b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.877+0000 7fda61c4c640 1 --2- 172.21.15.19:0/2700149608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda5c0d08e0 0x7fda5c0b6990 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fda5c161590 tx=0x7fda5402a710 comp rx=0 tx=0).stop 2024-04-25T10:01:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.877+0000 7fda61c4c640 1 -- 172.21.15.19:0/2700149608 shutdown_connections 2024-04-25T10:01:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.877+0000 7fda61c4c640 1 --2- 172.21.15.19:0/2700149608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda5c0d08e0 0x7fda5c0b6990 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.877+0000 7fda61c4c640 1 --2- 172.21.15.19:0/2700149608 >> v2:172.21.15.112:3300/0 conn(0x7fda5c0d34c0 0x7fda5c0d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.877+0000 7fda61c4c640 1 -- 172.21.15.19:0/2700149608 >> 172.21.15.19:0/2700149608 conn(0x7fda5c061c30 msgr2=0x7fda5c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.877+0000 7fda61c4c640 1 -- 172.21.15.19:0/2700149608 shutdown_connections 2024-04-25T10:01:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.877+0000 7fda61c4c640 1 -- 172.21.15.19:0/2700149608 wait complete. 2024-04-25T10:01:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.878+0000 7fda61c4c640 1 Processor -- start 2024-04-25T10:01:11.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.878+0000 7fda61c4c640 1 -- start start 2024-04-25T10:01:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda61c4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda5c0d08e0 0x7fda5c1f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda61c4c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda5c0d34c0 0x7fda5c152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda61c4c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda5c15dac0 0x7fda5c0cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda61c4c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0d2100 con 0x7fda5c0d34c0 2024-04-25T10:01:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda61c4c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0d2cc0 con 0x7fda5c0d08e0 2024-04-25T10:01:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda61c4c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c15dea0 con 0x7fda5c15dac0 2024-04-25T10:01:11.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda5c0d34c0 0x7fda5c152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda5c0d34c0 0x7fda5c152e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45936/0 (socket says 172.21.15.19:45936) 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda5affd640 1 -- 172.21.15.19:0/2659433900 learned_addr learned my addr 172.21.15.19:0/2659433900 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda5affd640 1 -- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda5c15dac0 msgr2=0x7fda5c0cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda5affd640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda5c15dac0 0x7fda5c0cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda5affd640 1 -- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda5c0d08e0 msgr2=0x7fda5c1f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda5affd640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda5c0d08e0 0x7fda5c1f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda5affd640 1 -- 172.21.15.19:0/2659433900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c1533a0 con 0x7fda5c0d34c0 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda5affd640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda5c0d34c0 0x7fda5c152e60 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fda5000d830 tx=0x7fda5000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda43fff640 1 -- 172.21.15.19:0/2659433900 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5000b6c0 con 0x7fda5c0d34c0 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda43fff640 1 -- 172.21.15.19:0/2659433900 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda50019070 con 0x7fda5c0d34c0 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda61c4c640 1 -- 172.21.15.19:0/2659433900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c0cade0 con 0x7fda5c0d34c0 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda43fff640 1 -- 172.21.15.19:0/2659433900 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5000baa0 con 0x7fda5c0d34c0 2024-04-25T10:01:11.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.880+0000 7fda61c4c640 1 -- 172.21.15.19:0/2659433900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c0cb3a0 con 0x7fda5c0d34c0 2024-04-25T10:01:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.881+0000 7fda61c4c640 1 -- 172.21.15.19:0/2659433900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c073a00 con 0x7fda5c0d34c0 2024-04-25T10:01:11.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.881+0000 7fda43fff640 1 -- 172.21.15.19:0/2659433900 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda5000bc40 con 0x7fda5c0d34c0 2024-04-25T10:01:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.882+0000 7fda43fff640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4806b4d0 0x7fda4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.882+0000 7fda43fff640 1 -- 172.21.15.19:0/2659433900 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda5008a850 con 0x7fda5c0d34c0 2024-04-25T10:01:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.882+0000 7fda5b7fe640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4806b4d0 0x7fda4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.882+0000 7fda5b7fe640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4806b4d0 0x7fda4806d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fda4c007fd0 tx=0x7fda4c007960 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:11.886+0000 7fda43fff640 1 -- 172.21.15.19:0/2659433900 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda5c1533a0 con 0x7fda5c0d34c0 2024-04-25T10:01:12.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.107+0000 7fda61c4c640 1 -- 172.21.15.19:0/2659433900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fda5c06b210 con 0x7fda5c0d34c0 2024-04-25T10:01:12.113 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T10:01:12.108+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:01:12.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.109+0000 7fda43fff640 1 -- 172.21.15.19:0/2659433900 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) v1 ==== 161+0+0 (secure 0 0 0) 0x7fda50059540 con 0x7fda5c0d34c0 2024-04-25T10:01:12.114 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:01:12.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 -- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4806b4d0 msgr2=0x7fda4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4806b4d0 0x7fda4806d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fda4c007fd0 tx=0x7fda4c007960 comp rx=0 tx=0).stop 2024-04-25T10:01:12.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 -- 172.21.15.19:0/2659433900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda5c0d34c0 msgr2=0x7fda5c152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda5c0d34c0 0x7fda5c152e60 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fda5000d830 tx=0x7fda5000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 -- 172.21.15.19:0/2659433900 shutdown_connections 2024-04-25T10:01:12.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fda4806b4d0 0x7fda4806d990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda5c15dac0 0x7fda5c0cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda5c0d34c0 0x7fda5c152e60 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 --2- 172.21.15.19:0/2659433900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda5c0d08e0 0x7fda5c1f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 -- 172.21.15.19:0/2659433900 >> 172.21.15.19:0/2659433900 conn(0x7fda5c061c30 msgr2=0x7fda5c0b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 -- 172.21.15.19:0/2659433900 shutdown_connections 2024-04-25T10:01:12.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.112+0000 7fda61c4c640 1 -- 172.21.15.19:0/2659433900 wait complete. 2024-04-25T10:01:12.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:01:12.155 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:01:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.230+0000 7ff47ca9e640 1 Processor -- start 2024-04-25T10:01:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.230+0000 7ff47ca9e640 1 -- start start 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff47ca9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff4780b9750 0x7ff4780ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff47ca9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4780cf060 con 0x7ff47815da90 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff47ca9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff478161d70 con 0x7ff4780cebb0 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff47ca9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff478162600 con 0x7ff4780b9750 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff47659c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff47815da90 0x7ff478161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49900/0 (socket says 172.21.15.19:49900) 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff47659c640 1 -- 172.21.15.19:0/3972903178 learned_addr learned my addr 172.21.15.19:0/3972903178 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff46ffff640 1 --2- 172.21.15.19:0/3972903178 >> v2:172.21.15.112:3300/0 conn(0x7ff4780b9750 0x7ff4780ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff475d9b640 1 -- 172.21.15.19:0/3972903178 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284496313 0 0) 0x7ff4780cf060 con 0x7ff47815da90 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff475d9b640 1 -- 172.21.15.19:0/3972903178 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff464003520 con 0x7ff47815da90 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff475d9b640 1 -- 172.21.15.19:0/3972903178 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959617201 0 0) 0x7ff478161d70 con 0x7ff4780cebb0 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff475d9b640 1 -- 172.21.15.19:0/3972903178 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4780cf060 con 0x7ff4780cebb0 2024-04-25T10:01:12.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff475d9b640 1 -- 172.21.15.19:0/3972903178 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2561402625 0 0) 0x7ff464003520 con 0x7ff47815da90 2024-04-25T10:01:12.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff46ffff640 1 -- 172.21.15.19:0/3972903178 >> v1:172.21.15.112:6789/0 conn(0x7ff4780cebb0 legacy=0x7ff4780d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff46ffff640 1 -- 172.21.15.19:0/3972903178 >> v1:172.21.15.19:6789/0 conn(0x7ff47815da90 legacy=0x7ff478161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff46ffff640 1 -- 172.21.15.19:0/3972903178 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff478162040 con 0x7ff4780b9750 2024-04-25T10:01:12.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff46ffff640 1 --2- 172.21.15.19:0/3972903178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4780b9750 0x7ff4780ce670 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff4600097a0 tx=0x7ff46002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2319cecc5e293e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff475d9b640 1 -- 172.21.15.19:0/3972903178 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460037070 con 0x7ff4780b9750 2024-04-25T10:01:12.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff475d9b640 1 -- 172.21.15.19:0/3972903178 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46002abc0 con 0x7ff4780b9750 2024-04-25T10:01:12.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.231+0000 7ff475d9b640 1 -- 172.21.15.19:0/3972903178 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468001670 con 0x7ff4780b9750 2024-04-25T10:01:12.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.232+0000 7ff47ca9e640 1 -- 172.21.15.19:0/3972903178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4780b9750 msgr2=0x7ff4780ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.232+0000 7ff47ca9e640 1 --2- 172.21.15.19:0/3972903178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4780b9750 0x7ff4780ce670 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff4600097a0 tx=0x7ff46002a130 comp rx=0 tx=0).stop 2024-04-25T10:01:12.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.232+0000 7ff47ca9e640 1 -- 172.21.15.19:0/3972903178 shutdown_connections 2024-04-25T10:01:12.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.232+0000 7ff47ca9e640 1 --2- 172.21.15.19:0/3972903178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4780b9750 0x7ff4780ce670 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.232+0000 7ff47ca9e640 1 -- 172.21.15.19:0/3972903178 >> 172.21.15.19:0/3972903178 conn(0x7ff478061a20 msgr2=0x7ff478061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.232+0000 7ff47ca9e640 1 -- 172.21.15.19:0/3972903178 shutdown_connections 2024-04-25T10:01:12.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.232+0000 7ff47ca9e640 1 -- 172.21.15.19:0/3972903178 wait complete. 2024-04-25T10:01:12.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.233+0000 7ff47ca9e640 1 Processor -- start 2024-04-25T10:01:12.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.233+0000 7ff47ca9e640 1 -- start start 2024-04-25T10:01:12.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff47ca9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4780cebb0 0x7ff4781f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff47ca9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff47815da90 0x7ff4780b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff47ca9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4781f3580 0x7ff4780cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff47ca9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4780d1df0 con 0x7ff47815da90 2024-04-25T10:01:12.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff47ca9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4780d1b10 con 0x7ff4781f3580 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff47ca9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47815de70 con 0x7ff4780cebb0 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff47815da90 0x7ff4780b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff47815da90 0x7ff4780b83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45974/0 (socket says 172.21.15.19:45974) 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46ffff640 1 -- 172.21.15.19:0/1659309710 learned_addr learned my addr 172.21.15.19:0/1659309710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff47659c640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4780cebb0 0x7ff4781f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff476d9d640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4781f3580 0x7ff4780cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46ffff640 1 -- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4780cebb0 msgr2=0x7ff4781f75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46ffff640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4780cebb0 0x7ff4781f75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46ffff640 1 -- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4781f3580 msgr2=0x7ff4780cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46ffff640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4781f3580 0x7ff4780cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46ffff640 1 -- 172.21.15.19:0/1659309710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47815f200 con 0x7ff47815da90 2024-04-25T10:01:12.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46ffff640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff47815da90 0x7ff4780b83e0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ff460035040 tx=0x7ff4600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46effd640 1 -- 172.21.15.19:0/1659309710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460037050 con 0x7ff47815da90 2024-04-25T10:01:12.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46effd640 1 -- 172.21.15.19:0/1659309710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff460043070 con 0x7ff47815da90 2024-04-25T10:01:12.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff47ca9e640 1 -- 172.21.15.19:0/1659309710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47815f4e0 con 0x7ff47815da90 2024-04-25T10:01:12.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff46effd640 1 -- 172.21.15.19:0/1659309710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460006e10 con 0x7ff47815da90 2024-04-25T10:01:12.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.235+0000 7ff47ca9e640 1 -- 172.21.15.19:0/1659309710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47815faa0 con 0x7ff47815da90 2024-04-25T10:01:12.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.236+0000 7ff47ca9e640 1 -- 172.21.15.19:0/1659309710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff478073a00 con 0x7ff47815da90 2024-04-25T10:01:12.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.236+0000 7ff46effd640 1 -- 172.21.15.19:0/1659309710 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff46003ca10 con 0x7ff47815da90 2024-04-25T10:01:12.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.237+0000 7ff46effd640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff45806b4d0 0x7ff45806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.237+0000 7ff46effd640 1 -- 172.21.15.19:0/1659309710 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff4600ab060 con 0x7ff47815da90 2024-04-25T10:01:12.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.237+0000 7ff47659c640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff45806b4d0 0x7ff45806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.237+0000 7ff47659c640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff45806b4d0 0x7ff45806d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff4680011e0 tx=0x7ff468009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.241+0000 7ff46effd640 1 -- 172.21.15.19:0/1659309710 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff460030e20 con 0x7ff47815da90 2024-04-25T10:01:12.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:12.463+0000 7ff47ca9e640 1 -- 172.21.15.19:0/1659309710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff4780d1590 con 0x7ff47815da90 2024-04-25T10:01:13.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.112+0000 7ff46effd640 1 -- 172.21.15.19:0/1659309710 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff460079eb0 con 0x7ff47815da90 2024-04-25T10:01:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.115+0000 7ff47ca9e640 1 -- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff45806b4d0 msgr2=0x7ff45806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.115+0000 7ff47ca9e640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff45806b4d0 0x7ff45806d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff4680011e0 tx=0x7ff468009040 comp rx=0 tx=0).stop 2024-04-25T10:01:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.115+0000 7ff47ca9e640 1 -- 172.21.15.19:0/1659309710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff47815da90 msgr2=0x7ff4780b83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.115+0000 7ff47ca9e640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff47815da90 0x7ff4780b83e0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ff460035040 tx=0x7ff4600046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.115+0000 7ff47ca9e640 1 -- 172.21.15.19:0/1659309710 shutdown_connections 2024-04-25T10:01:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.115+0000 7ff47ca9e640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff45806b4d0 0x7ff45806d990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.115+0000 7ff47ca9e640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4781f3580 0x7ff4780cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.115+0000 7ff47ca9e640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff47815da90 0x7ff4780b83e0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.115+0000 7ff47ca9e640 1 --2- 172.21.15.19:0/1659309710 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4780cebb0 0x7ff4781f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.115+0000 7ff47ca9e640 1 -- 172.21.15.19:0/1659309710 >> 172.21.15.19:0/1659309710 conn(0x7ff478061a20 msgr2=0x7ff47815bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.116+0000 7ff47ca9e640 1 -- 172.21.15.19:0/1659309710 shutdown_connections 2024-04-25T10:01:13.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.116+0000 7ff47ca9e640 1 -- 172.21.15.19:0/1659309710 wait complete. 2024-04-25T10:01:13.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:01:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.282+0000 7f0fa7007640 1 Processor -- start 2024-04-25T10:01:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.282+0000 7f0fa7007640 1 -- start start 2024-04-25T10:01:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.282+0000 7f0fa7007640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0fa015ba20 0x7f0fa006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.282+0000 7f0fa7007640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fa00af260 0x7f0fa006c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.282+0000 7f0fa7007640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa0072aa0 con 0x7f0fa0072610 2024-04-25T10:01:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.282+0000 7f0fa7007640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa006f300 con 0x7f0fa015ba20 2024-04-25T10:01:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.282+0000 7f0fa7007640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa0072d80 con 0x7f0fa00af260 2024-04-25T10:01:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.283+0000 7f0fa4da3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0fa015ba20 0x7f0fa006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.283+0000 7f0f9ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0fa0072610 0x7f0fa00acd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49902/0 (socket says 172.21.15.19:49902) 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.283+0000 7f0f9ffff640 1 -- 172.21.15.19:0/967564385 learned_addr learned my addr 172.21.15.19:0/967564385 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.283+0000 7f0fa55a4640 1 --2- 172.21.15.19:0/967564385 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fa00af260 0x7f0fa006c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.283+0000 7f0fa4da3640 1 -- 172.21.15.19:0/967564385 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fa00af260 msgr2=0x7f0fa006c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.283+0000 7f0fa4da3640 1 --2- 172.21.15.19:0/967564385 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fa00af260 0x7f0fa006c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.283+0000 7f0fa4da3640 1 -- 172.21.15.19:0/967564385 >> v1:172.21.15.19:6789/0 conn(0x7f0fa0072610 legacy=0x7f0fa00acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.283+0000 7f0fa4da3640 1 -- 172.21.15.19:0/967564385 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa006f530 con 0x7f0fa015ba20 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.283+0000 7f0f9f7fe640 1 -- 172.21.15.19:0/967564385 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126690517 0 0) 0x7f0fa0072aa0 con 0x7f0fa0072610 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.283+0000 7f0fa55a4640 1 --2- 172.21.15.19:0/967564385 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fa00af260 0x7f0fa006c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0fa4da3640 1 --2- 172.21.15.19:0/967564385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fa015ba20 0x7f0fa006bbf0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009dc0 tx=0x7f0f9802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe73232f784d515e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0f9f7fe640 1 -- 172.21.15.19:0/967564385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9802ae60 con 0x7f0fa015ba20 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0f9f7fe640 1 -- 172.21.15.19:0/967564385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f98007740 con 0x7f0fa015ba20 2024-04-25T10:01:13.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0f9f7fe640 1 -- 172.21.15.19:0/967564385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98007b10 con 0x7f0fa015ba20 2024-04-25T10:01:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0fa7007640 1 -- 172.21.15.19:0/967564385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fa015ba20 msgr2=0x7f0fa006bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0fa7007640 1 --2- 172.21.15.19:0/967564385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fa015ba20 0x7f0fa006bbf0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009dc0 tx=0x7f0f9802a680 comp rx=0 tx=0).stop 2024-04-25T10:01:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0fa7007640 1 -- 172.21.15.19:0/967564385 shutdown_connections 2024-04-25T10:01:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0fa7007640 1 --2- 172.21.15.19:0/967564385 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fa00af260 0x7f0fa006c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0fa7007640 1 --2- 172.21.15.19:0/967564385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fa015ba20 0x7f0fa006bbf0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0fa7007640 1 -- 172.21.15.19:0/967564385 >> 172.21.15.19:0/967564385 conn(0x7f0fa0071af0 msgr2=0x7f0fa0071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0fa7007640 1 -- 172.21.15.19:0/967564385 shutdown_connections 2024-04-25T10:01:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.284+0000 7f0fa7007640 1 -- 172.21.15.19:0/967564385 wait complete. 2024-04-25T10:01:13.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.286+0000 7f0fa7007640 1 Processor -- start 2024-04-25T10:01:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.286+0000 7f0fa7007640 1 -- start start 2024-04-25T10:01:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa7007640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fa00af260 0x7f0fa00b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa7007640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa015ba20 0x7f0fa01f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa7007640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fa0159ed0 0x7f0fa00b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa7007640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa0070010 con 0x7f0fa0159ed0 2024-04-25T10:01:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa7007640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa006fb30 con 0x7f0fa015ba20 2024-04-25T10:01:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa7007640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa01f7ef0 con 0x7f0fa00af260 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa55a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fa0159ed0 0x7f0fa00b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa55a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fa0159ed0 0x7f0fa00b4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45988/0 (socket says 172.21.15.19:45988) 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa55a4640 1 -- 172.21.15.19:0/631123249 learned_addr learned my addr 172.21.15.19:0/631123249 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0f9ffff640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa015ba20 0x7f0fa01f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa55a4640 1 -- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fa00af260 msgr2=0x7f0fa00b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa55a4640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fa00af260 0x7f0fa00b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa55a4640 1 -- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa015ba20 msgr2=0x7f0fa01f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa55a4640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa015ba20 0x7f0fa01f79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa55a4640 1 -- 172.21.15.19:0/631123249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa00b57b0 con 0x7f0fa0159ed0 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa55a4640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fa0159ed0 0x7f0fa00b4a70 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800c7f0 tx=0x7f0f8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0f9d7fa640 1 -- 172.21.15.19:0/631123249 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88015070 con 0x7f0fa0159ed0 2024-04-25T10:01:13.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0f9d7fa640 1 -- 172.21.15.19:0/631123249 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f88007820 con 0x7f0fa0159ed0 2024-04-25T10:01:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa7007640 1 -- 172.21.15.19:0/631123249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa00b5a90 con 0x7f0fa0159ed0 2024-04-25T10:01:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0f9d7fa640 1 -- 172.21.15.19:0/631123249 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88007c50 con 0x7f0fa0159ed0 2024-04-25T10:01:13.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.287+0000 7f0fa7007640 1 -- 172.21.15.19:0/631123249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa00b4fb0 con 0x7f0fa0159ed0 2024-04-25T10:01:13.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.288+0000 7f0fa7007640 1 -- 172.21.15.19:0/631123249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa0073a00 con 0x7f0fa0159ed0 2024-04-25T10:01:13.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.289+0000 7f0f9d7fa640 1 -- 172.21.15.19:0/631123249 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f88018070 con 0x7f0fa0159ed0 2024-04-25T10:01:13.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.289+0000 7f0f9d7fa640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f8006b4d0 0x7f0f8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.289+0000 7f0f9d7fa640 1 -- 172.21.15.19:0/631123249 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0f8808a520 con 0x7f0fa0159ed0 2024-04-25T10:01:13.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.290+0000 7f0fa4da3640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f8006b4d0 0x7f0f8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.290+0000 7f0fa4da3640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f8006b4d0 0x7f0f8006d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0f98034000 tx=0x7f0f9802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.294+0000 7f0f9d7fa640 1 -- 172.21.15.19:0/631123249 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f880591b0 con 0x7f0fa0159ed0 2024-04-25T10:01:13.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:13.475+0000 7f0fa7007640 1 -- 172.21.15.19:0/631123249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0fa00af7d0 con 0x7f0fa0159ed0 2024-04-25T10:01:14.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.113+0000 7f0f9d7fa640 1 -- 172.21.15.19:0/631123249 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v119) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0f88058ab0 con 0x7f0fa0159ed0 2024-04-25T10:01:14.118 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.117+0000 7f0fa7007640 1 -- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f8006b4d0 msgr2=0x7f0f8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.118+0000 7f0fa7007640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f8006b4d0 0x7f0f8006d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0f98034000 tx=0x7f0f9802e000 comp rx=0 tx=0).stop 2024-04-25T10:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.118+0000 7f0fa7007640 1 -- 172.21.15.19:0/631123249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fa0159ed0 msgr2=0x7f0fa00b4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.118+0000 7f0fa7007640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fa0159ed0 0x7f0fa00b4a70 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800c7f0 tx=0x7f0f8800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.118+0000 7f0fa7007640 1 -- 172.21.15.19:0/631123249 shutdown_connections 2024-04-25T10:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.118+0000 7f0fa7007640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f8006b4d0 0x7f0f8006d990 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.118+0000 7f0fa7007640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fa0159ed0 0x7f0fa00b4a70 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.118+0000 7f0fa7007640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fa015ba20 0x7f0fa01f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.118+0000 7f0fa7007640 1 --2- 172.21.15.19:0/631123249 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fa00af260 0x7f0fa00b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.118+0000 7f0fa7007640 1 -- 172.21.15.19:0/631123249 >> 172.21.15.19:0/631123249 conn(0x7f0fa0071af0 msgr2=0x7f0fa0071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.118+0000 7f0fa7007640 1 -- 172.21.15.19:0/631123249 shutdown_connections 2024-04-25T10:01:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.118+0000 7f0fa7007640 1 -- 172.21.15.19:0/631123249 wait complete. 2024-04-25T10:01:14.136 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:01:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.243+0000 7fbfcb038640 1 Processor -- start 2024-04-25T10:01:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.243+0000 7fbfcb038640 1 -- start start 2024-04-25T10:01:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfcb038640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfc4144d50 0x7fbfc406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfcb038640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc40cccc0 con 0x7fbfc41446d0 2024-04-25T10:01:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfcb038640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc40cc140 con 0x7fbfc41452a0 2024-04-25T10:01:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfcb038640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc40b2b20 con 0x7fbfc4144d50 2024-04-25T10:01:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc8dd4640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbfc41446d0 0x7fbfc40b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49910/0 (socket says 172.21.15.19:49910) 2024-04-25T10:01:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc8dd4640 1 -- 172.21.15.19:0/1187899739 learned_addr learned my addr 172.21.15.19:0/1187899739 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866364340 0 0) 0x7fbfc40cccc0 con 0x7fbfc41446d0 2024-04-25T10:01:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb4003520 con 0x7fbfc41446d0 2024-04-25T10:01:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc3fff640 1 --2- 172.21.15.19:0/1187899739 >> v2:172.21.15.112:3300/0 conn(0x7fbfc4144d50 0x7fbfc406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2241789483 0 0) 0x7fbfb4003520 con 0x7fbfc41446d0 2024-04-25T10:01:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 >> v2:172.21.15.112:3300/0 conn(0x7fbfc4144d50 msgr2=0x7fbfc406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc37fe640 1 --2- 172.21.15.19:0/1187899739 >> v2:172.21.15.112:3300/0 conn(0x7fbfc4144d50 0x7fbfc406d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 >> v1:172.21.15.112:6789/0 conn(0x7fbfc41452a0 legacy=0x7fbfc406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc40cc420 con 0x7fbfc41446d0 2024-04-25T10:01:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fbfb8001670 con 0x7fbfc41446d0 2024-04-25T10:01:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 >> v1:172.21.15.19:6789/0 conn(0x7fbfc41446d0 legacy=0x7fbfc40b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc37fe640 1 --2- 172.21.15.19:0/1187899739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfb40039b0 0x7fbfb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.244+0000 7fbfc37fe640 1 --2- 172.21.15.19:0/1187899739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfb4004800 0x7fbfb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc37fe640 1 --2- 172.21.15.19:0/1187899739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 0x7fbfb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc40b2b20 con 0x7fbfb40051c0 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb400c110 con 0x7fbfb40039b0 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb400c310 con 0x7fbfb4004800 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc95d5640 1 --2- 172.21.15.19:0/1187899739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 0x7fbfb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc8dd4640 1 --2- 172.21.15.19:0/1187899739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfb40039b0 0x7fbfb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc3fff640 1 --2- 172.21.15.19:0/1187899739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfb4004800 0x7fbfb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc95d5640 1 -- 172.21.15.19:0/1187899739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfb4004800 msgr2=0x7fbfb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc95d5640 1 --2- 172.21.15.19:0/1187899739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfb4004800 0x7fbfb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc95d5640 1 -- 172.21.15.19:0/1187899739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfb40039b0 msgr2=0x7fbfb4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc95d5640 1 --2- 172.21.15.19:0/1187899739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfb40039b0 0x7fbfb4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.245+0000 7fbfc95d5640 1 -- 172.21.15.19:0/1187899739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4003520 con 0x7fbfb40051c0 2024-04-25T10:01:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfc95d5640 1 -- 172.21.15.19:0/1187899739 reap_dead start 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfc95d5640 1 --2- 172.21.15.19:0/1187899739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 0x7fbfb400b9c0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc001e50 tx=0x7fbfb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116f38fd74508550 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc001a70 con 0x7fbfb40051c0 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc003970 con 0x7fbfb40051c0 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfc37fe640 1 -- 172.21.15.19:0/1187899739 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc003d60 con 0x7fbfb40051c0 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfcb038640 1 -- 172.21.15.19:0/1187899739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 msgr2=0x7fbfb400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfcb038640 1 --2- 172.21.15.19:0/1187899739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 0x7fbfb400b9c0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc001e50 tx=0x7fbfb4005830 comp rx=0 tx=0).stop 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfcb038640 1 -- 172.21.15.19:0/1187899739 shutdown_connections 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfcb038640 1 --2- 172.21.15.19:0/1187899739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 0x7fbfb400b9c0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfcb038640 1 -- 172.21.15.19:0/1187899739 >> 172.21.15.19:0/1187899739 conn(0x7fbfc406abe0 msgr2=0x7fbfc406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfcb038640 1 -- 172.21.15.19:0/1187899739 shutdown_connections 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.246+0000 7fbfcb038640 1 -- 172.21.15.19:0/1187899739 wait complete. 2024-04-25T10:01:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.248+0000 7fbfcb038640 1 Processor -- start 2024-04-25T10:01:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.248+0000 7fbfcb038640 1 -- start start 2024-04-25T10:01:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.248+0000 7fbfcb038640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 0x7fbfc41565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.248+0000 7fbfcb038640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfc4156c80 0x7fbfc4157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.248+0000 7fbfcb038640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfc40b0ac0 0x7fbfc40b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.248+0000 7fbfcb038640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc40cc9e0 con 0x7fbfb40051c0 2024-04-25T10:01:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.248+0000 7fbfcb038640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc406d400 con 0x7fbfc4156c80 2024-04-25T10:01:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.248+0000 7fbfcb038640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc4144ab0 con 0x7fbfc40b0ac0 2024-04-25T10:01:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc8dd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 0x7fbfc41565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc8dd4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 0x7fbfc41565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46004/0 (socket says 172.21.15.19:46004) 2024-04-25T10:01:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc8dd4640 1 -- 172.21.15.19:0/1049816604 learned_addr learned my addr 172.21.15.19:0/1049816604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc8dd4640 1 -- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfc40b0ac0 msgr2=0x7fbfc40b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc8dd4640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfc40b0ac0 0x7fbfc40b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc8dd4640 1 -- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfc4156c80 msgr2=0x7fbfc4157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc8dd4640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfc4156c80 0x7fbfc4157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc8dd4640 1 -- 172.21.15.19:0/1049816604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc4157720 con 0x7fbfb40051c0 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc8dd4640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 0x7fbfc41565a0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8007340 tx=0x7fbfb8007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc17fa640 1 -- 172.21.15.19:0/1049816604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800f8e0 con 0x7fbfb40051c0 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc17fa640 1 -- 172.21.15.19:0/1049816604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb8018070 con 0x7fbfb40051c0 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfcb038640 1 -- 172.21.15.19:0/1049816604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc4157a00 con 0x7fbfb40051c0 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfc17fa640 1 -- 172.21.15.19:0/1049816604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800fce0 con 0x7fbfb40051c0 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.249+0000 7fbfcb038640 1 -- 172.21.15.19:0/1049816604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc4157fc0 con 0x7fbfb40051c0 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.250+0000 7fbfcb038640 1 -- 172.21.15.19:0/1049816604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc40aebb0 con 0x7fbfb40051c0 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.251+0000 7fbfc17fa640 1 -- 172.21.15.19:0/1049816604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfb801d460 con 0x7fbfb40051c0 2024-04-25T10:01:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.251+0000 7fbfc17fa640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfa806b4d0 0x7fbfa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.251+0000 7fbfc17fa640 1 -- 172.21.15.19:0/1049816604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fbfb808a1e0 con 0x7fbfb40051c0 2024-04-25T10:01:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.252+0000 7fbfc3fff640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfa806b4d0 0x7fbfa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.252+0000 7fbfc3fff640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfa806b4d0 0x7fbfa806d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0006fd0 tx=0x7fbfb00040a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.255+0000 7fbfc17fa640 1 -- 172.21.15.19:0/1049816604 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfc40aebb0 con 0x7fbfb40051c0 2024-04-25T10:01:14.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:14.442+0000 7fbfcb038640 1 -- 172.21.15.19:0/1049816604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbfc4073a00 con 0x7fbfb40051c0 2024-04-25T10:01:15.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.115+0000 7fbfc17fa640 1 -- 172.21.15.19:0/1049816604 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v120) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbfb8059180 con 0x7fbfb40051c0 2024-04-25T10:01:15.120 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T10:01:15.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 -- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfa806b4d0 msgr2=0x7fbfa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfa806b4d0 0x7fbfa806d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0006fd0 tx=0x7fbfb00040a0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 -- 172.21.15.19:0/1049816604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 msgr2=0x7fbfc41565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 0x7fbfc41565a0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8007340 tx=0x7fbfb8007800 comp rx=0 tx=0).stop 2024-04-25T10:01:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 -- 172.21.15.19:0/1049816604 shutdown_connections 2024-04-25T10:01:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbfa806b4d0 0x7fbfa806d990 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbfc40b0ac0 0x7fbfc40b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfc4156c80 0x7fbfc4157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 --2- 172.21.15.19:0/1049816604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfb40051c0 0x7fbfc41565a0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 -- 172.21.15.19:0/1049816604 >> 172.21.15.19:0/1049816604 conn(0x7fbfc406abe0 msgr2=0x7fbfc4141970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 -- 172.21.15.19:0/1049816604 shutdown_connections 2024-04-25T10:01:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.119+0000 7fbfcb038640 1 -- 172.21.15.19:0/1049816604 wait complete. 2024-04-25T10:01:15.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:01:15.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.286+0000 7fc2e0f42640 1 Processor -- start 2024-04-25T10:01:15.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.286+0000 7fc2e0f42640 1 -- start start 2024-04-25T10:01:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.287+0000 7fc2e0f42640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2dc15b7d0 0x7fc2dc0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.287+0000 7fc2e0f42640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc2dc06c9e0 0x7fc2dc0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.287+0000 7fc2e0f42640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2dc06cf50 con 0x7fc2dc0b5620 2024-04-25T10:01:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.287+0000 7fc2e0f42640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc0b5fc0 con 0x7fc2dc06c9e0 2024-04-25T10:01:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.287+0000 7fc2e0f42640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0ce530 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.287+0000 7fc2dad9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc2dc06c9e0 0x7fc2dc0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.287+0000 7fc2dad9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc2dc06c9e0 0x7fc2dc0b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46018/0 (socket says 172.21.15.19:46018) 2024-04-25T10:01:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.287+0000 7fc2dad9d640 1 -- 172.21.15.19:0/319319361 learned_addr learned my addr 172.21.15.19:0/319319361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:15.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.287+0000 7fc2da59c640 1 --2- 172.21.15.19:0/319319361 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2dc15b7d0 0x7fc2dc0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.288+0000 7fc2dad9d640 1 -- 172.21.15.19:0/319319361 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2dc15b7d0 msgr2=0x7fc2dc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:15.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.288+0000 7fc2dad9d640 1 --2- 172.21.15.19:0/319319361 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2dc15b7d0 0x7fc2dc0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.288+0000 7fc2dad9d640 1 -- 172.21.15.19:0/319319361 >> v1:172.21.15.112:6789/0 conn(0x7fc2dc0b5620 legacy=0x7fc2dc06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.288+0000 7fc2dad9d640 1 -- 172.21.15.19:0/319319361 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc0ce810 con 0x7fc2dc06c9e0 2024-04-25T10:01:15.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.288+0000 7fc2dad9d640 1 --2- 172.21.15.19:0/319319361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2dc06c9e0 0x7fc2dc0b5a80 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009e50 tx=0x7fc2d402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde5637be63a9988 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:15.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.288+0000 7fc2d959a640 1 -- 172.21.15.19:0/319319361 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4036070 con 0x7fc2dc06c9e0 2024-04-25T10:01:15.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.288+0000 7fc2d959a640 1 -- 172.21.15.19:0/319319361 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d4031600 con 0x7fc2dc06c9e0 2024-04-25T10:01:15.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.288+0000 7fc2d959a640 1 -- 172.21.15.19:0/319319361 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40319d0 con 0x7fc2dc06c9e0 2024-04-25T10:01:15.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.289+0000 7fc2e0f42640 1 -- 172.21.15.19:0/319319361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2dc06c9e0 msgr2=0x7fc2dc0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.289+0000 7fc2e0f42640 1 --2- 172.21.15.19:0/319319361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2dc06c9e0 0x7fc2dc0b5a80 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009e50 tx=0x7fc2d402a710 comp rx=0 tx=0).stop 2024-04-25T10:01:15.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.289+0000 7fc2e0f42640 1 -- 172.21.15.19:0/319319361 shutdown_connections 2024-04-25T10:01:15.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.289+0000 7fc2e0f42640 1 --2- 172.21.15.19:0/319319361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2dc06c9e0 0x7fc2dc0b5a80 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.289+0000 7fc2e0f42640 1 --2- 172.21.15.19:0/319319361 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2dc15b7d0 0x7fc2dc0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.289+0000 7fc2e0f42640 1 -- 172.21.15.19:0/319319361 >> 172.21.15.19:0/319319361 conn(0x7fc2dc0d0800 msgr2=0x7fc2dc0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:15.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.289+0000 7fc2e0f42640 1 -- 172.21.15.19:0/319319361 shutdown_connections 2024-04-25T10:01:15.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.289+0000 7fc2e0f42640 1 -- 172.21.15.19:0/319319361 wait complete. 2024-04-25T10:01:15.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.290+0000 7fc2e0f42640 1 Processor -- start 2024-04-25T10:01:15.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.290+0000 7fc2e0f42640 1 -- start start 2024-04-25T10:01:15.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.291+0000 7fc2e0f42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2dc06c9e0 0x7fc2dc0cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.291+0000 7fc2e0f42640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2dc0b5620 0x7fc2dc1f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.291+0000 7fc2e0f42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2dc15b7d0 0x7fc2dc14bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.291+0000 7fc2e0f42640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0cf550 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.291+0000 7fc2e0f42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc06d230 con 0x7fc2dc06c9e0 2024-04-25T10:01:15.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.291+0000 7fc2e0f42640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0cf6d0 con 0x7fc2dc0b5620 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.291+0000 7fc2dad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2dc15b7d0 0x7fc2dc14bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.291+0000 7fc2dad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2dc15b7d0 0x7fc2dc14bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46030/0 (socket says 172.21.15.19:46030) 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.291+0000 7fc2dad9d640 1 -- 172.21.15.19:0/1923955966 learned_addr learned my addr 172.21.15.19:0/1923955966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2da59c640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2dc06c9e0 0x7fc2dc0cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2d9d9b640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2dc0b5620 0x7fc2dc1f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2dad9d640 1 -- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2dc0b5620 msgr2=0x7fc2dc1f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2dad9d640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2dc0b5620 0x7fc2dc1f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2dad9d640 1 -- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2dc06c9e0 msgr2=0x7fc2dc0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2dad9d640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2dc06c9e0 0x7fc2dc0cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2dad9d640 1 -- 172.21.15.19:0/1923955966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc0cdfa0 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2d9d9b640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2dc0b5620 0x7fc2dc1f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2da59c640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2dc06c9e0 0x7fc2dc0cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2dad9d640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2dc15b7d0 0x7fc2dc14bf30 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009f70 tx=0x7fc2d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:15.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2caffd640 1 -- 172.21.15.19:0/1923955966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d403f020 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2caffd640 1 -- 172.21.15.19:0/1923955966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d4037040 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2e0f42640 1 -- 172.21.15.19:0/1923955966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc0cd420 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2caffd640 1 -- 172.21.15.19:0/1923955966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4036070 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.292+0000 7fc2e0f42640 1 -- 172.21.15.19:0/1923955966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc0cd9e0 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.293+0000 7fc2e0f42640 1 -- 172.21.15.19:0/1923955966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2dc073a00 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.294+0000 7fc2caffd640 1 -- 172.21.15.19:0/1923955966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2d4042020 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.294+0000 7fc2caffd640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc2bc06b4d0 0x7fc2bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.294+0000 7fc2caffd640 1 -- 172.21.15.19:0/1923955966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc2d40aa480 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.294+0000 7fc2da59c640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc2bc06b4d0 0x7fc2bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.295+0000 7fc2da59c640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc2bc06b4d0 0x7fc2bc06d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc006fd0 tx=0x7fc2cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:15.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.298+0000 7fc2caffd640 1 -- 172.21.15.19:0/1923955966 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2d4049030 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.478+0000 7fc2e0f42640 1 -- 172.21.15.19:0/1923955966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc2dc0cdd80 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.481+0000 7fc2caffd640 1 -- 172.21.15.19:0/1923955966 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc2d40794e0 con 0x7fc2dc15b7d0 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 -- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc2bc06b4d0 msgr2=0x7fc2bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc2bc06b4d0 0x7fc2bc06d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc006fd0 tx=0x7fc2cc008040 comp rx=0 tx=0).stop 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 -- 172.21.15.19:0/1923955966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2dc15b7d0 msgr2=0x7fc2dc14bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2dc15b7d0 0x7fc2dc14bf30 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009f70 tx=0x7fc2d4004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 -- 172.21.15.19:0/1923955966 shutdown_connections 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc2bc06b4d0 0x7fc2bc06d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2dc15b7d0 0x7fc2dc14bf30 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2dc0b5620 0x7fc2dc1f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 --2- 172.21.15.19:0/1923955966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2dc06c9e0 0x7fc2dc0cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 -- 172.21.15.19:0/1923955966 >> 172.21.15.19:0/1923955966 conn(0x7fc2dc0d0800 msgr2=0x7fc2dc0d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 -- 172.21.15.19:0/1923955966 shutdown_connections 2024-04-25T10:01:15.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.483+0000 7fc2e0f42640 1 -- 172.21.15.19:0/1923955966 wait complete. 2024-04-25T10:01:15.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:01:15.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.595+0000 7f274a39d640 1 Processor -- start 2024-04-25T10:01:15.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.595+0000 7f274a39d640 1 -- start start 2024-04-25T10:01:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.596+0000 7f274a39d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f274415dd30 0x7f27440b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.596+0000 7f274a39d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27440d1df0 con 0x7f27440b9770 2024-04-25T10:01:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.596+0000 7f274a39d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27440d20d0 con 0x7f27440d0ad0 2024-04-25T10:01:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.596+0000 7f274a39d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27440d23b0 con 0x7f274415dd30 2024-04-25T10:01:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.596+0000 7f2743fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f274415dd30 0x7f27440b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.596+0000 7f27437fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f27440b9770 0x7f27440d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49912/0 (socket says 172.21.15.19:49912) 2024-04-25T10:01:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.596+0000 7f27437fe640 1 -- 172.21.15.19:0/3301034903 learned_addr learned my addr 172.21.15.19:0/3301034903 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.596+0000 7f2743fff640 1 -- 172.21.15.19:0/3301034903 >> v1:172.21.15.112:6789/0 conn(0x7f27440d0ad0 legacy=0x7f27440b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.596+0000 7f2743fff640 1 -- 172.21.15.19:0/3301034903 >> v1:172.21.15.19:6789/0 conn(0x7f27440b9770 legacy=0x7f27440d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.596+0000 7f2743fff640 1 -- 172.21.15.19:0/3301034903 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27440d2690 con 0x7f274415dd30 2024-04-25T10:01:15.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.596+0000 7f2742ffd640 1 -- 172.21.15.19:0/3301034903 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957502014 0 0) 0x7f27440d1df0 con 0x7f27440b9770 2024-04-25T10:01:15.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.597+0000 7f2743fff640 1 --2- 172.21.15.19:0/3301034903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f274415dd30 0x7f27440b9230 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f27380097a0 tx=0x7f273802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265521f624fd5dba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:15.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.597+0000 7f2742ffd640 1 -- 172.21.15.19:0/3301034903 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738037070 con 0x7f274415dd30 2024-04-25T10:01:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.597+0000 7f2742ffd640 1 -- 172.21.15.19:0/3301034903 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273802adc0 con 0x7f274415dd30 2024-04-25T10:01:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.597+0000 7f2742ffd640 1 -- 172.21.15.19:0/3301034903 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738032660 con 0x7f274415dd30 2024-04-25T10:01:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.597+0000 7f274a39d640 1 -- 172.21.15.19:0/3301034903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f274415dd30 msgr2=0x7f27440b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.597+0000 7f274a39d640 1 --2- 172.21.15.19:0/3301034903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f274415dd30 0x7f27440b9230 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f27380097a0 tx=0x7f273802a0b0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.597+0000 7f274a39d640 1 -- 172.21.15.19:0/3301034903 shutdown_connections 2024-04-25T10:01:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.597+0000 7f274a39d640 1 --2- 172.21.15.19:0/3301034903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f274415dd30 0x7f27440b9230 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.597+0000 7f274a39d640 1 -- 172.21.15.19:0/3301034903 >> 172.21.15.19:0/3301034903 conn(0x7f2744061c30 msgr2=0x7f274406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.598+0000 7f274a39d640 1 -- 172.21.15.19:0/3301034903 shutdown_connections 2024-04-25T10:01:15.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.598+0000 7f274a39d640 1 -- 172.21.15.19:0/3301034903 wait complete. 2024-04-25T10:01:15.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.599+0000 7f274a39d640 1 Processor -- start 2024-04-25T10:01:15.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.599+0000 7f274a39d640 1 -- start start 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f274a39d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27440b9770 0x7f27441529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f274a39d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0ad0 0x7f27440b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f274a39d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f274415dd30 0x7f274415f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f274a39d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27440d1b90 con 0x7f27440d0ad0 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f274a39d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27440d1a10 con 0x7f27440b9770 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f274a39d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27440d2e90 con 0x7f274415dd30 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f27437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0ad0 0x7f27440b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f27437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0ad0 0x7f27440b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46048/0 (socket says 172.21.15.19:46048) 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f27437fe640 1 -- 172.21.15.19:0/1310652425 learned_addr learned my addr 172.21.15.19:0/1310652425 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f2743fff640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27440b9770 0x7f27441529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f274893a640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f274415dd30 0x7f274415f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f27437fe640 1 -- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f274415dd30 msgr2=0x7f274415f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f27437fe640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f274415dd30 0x7f274415f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f27437fe640 1 -- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27440b9770 msgr2=0x7f27441529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f27437fe640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27440b9770 0x7f27441529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f27437fe640 1 -- 172.21.15.19:0/1310652425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744152f30 con 0x7f27440d0ad0 2024-04-25T10:01:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f274893a640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f274415dd30 0x7f274415f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f27437fe640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0ad0 0x7f27440b3960 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f273400c4d0 tx=0x7f273400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f2740ff9640 1 -- 172.21.15.19:0/1310652425 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734007480 con 0x7f27440d0ad0 2024-04-25T10:01:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f2740ff9640 1 -- 172.21.15.19:0/1310652425 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27340057a0 con 0x7f27440d0ad0 2024-04-25T10:01:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f2740ff9640 1 -- 172.21.15.19:0/1310652425 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734007620 con 0x7f27440d0ad0 2024-04-25T10:01:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f274a39d640 1 -- 172.21.15.19:0/1310652425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27440b5920 con 0x7f27440d0ad0 2024-04-25T10:01:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.600+0000 7f274a39d640 1 -- 172.21.15.19:0/1310652425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27440b61c0 con 0x7f27440d0ad0 2024-04-25T10:01:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.601+0000 7f274a39d640 1 -- 172.21.15.19:0/1310652425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2744073990 con 0x7f27440d0ad0 2024-04-25T10:01:15.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.602+0000 7f2740ff9640 1 -- 172.21.15.19:0/1310652425 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2734005b50 con 0x7f27440d0ad0 2024-04-25T10:01:15.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.602+0000 7f2740ff9640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f273006b500 0x7f273006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.602+0000 7f2740ff9640 1 -- 172.21.15.19:0/1310652425 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2734089d60 con 0x7f27440d0ad0 2024-04-25T10:01:15.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.602+0000 7f2743fff640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f273006b500 0x7f273006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.603+0000 7f2743fff640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f273006b500 0x7f273006d9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2738034000 tx=0x7f273802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:15.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.606+0000 7f2740ff9640 1 -- 172.21.15.19:0/1310652425 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27340182b0 con 0x7f27440d0ad0 2024-04-25T10:01:15.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:15.855+0000 7f274a39d640 1 -- 172.21.15.19:0/1310652425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f2744153450 con 0x7f27440d0ad0 2024-04-25T10:01:16.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.117+0000 7f2740ff9640 1 -- 172.21.15.19:0/1310652425 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v408) v1 ==== 166+0+0 (secure 0 0 0) 0x7f2734058cb0 con 0x7f27440d0ad0 2024-04-25T10:01:16.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 -- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f273006b500 msgr2=0x7f273006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f273006b500 0x7f273006d9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2738034000 tx=0x7f273802e000 comp rx=0 tx=0).stop 2024-04-25T10:01:16.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 -- 172.21.15.19:0/1310652425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0ad0 msgr2=0x7f27440b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0ad0 0x7f27440b3960 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f273400c4d0 tx=0x7f273400c990 comp rx=0 tx=0).stop 2024-04-25T10:01:16.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 -- 172.21.15.19:0/1310652425 shutdown_connections 2024-04-25T10:01:16.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f273006b500 0x7f273006d9c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f274415dd30 0x7f274415f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0ad0 0x7f27440b3960 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 --2- 172.21.15.19:0/1310652425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27440b9770 0x7f27441529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 -- 172.21.15.19:0/1310652425 >> 172.21.15.19:0/1310652425 conn(0x7f2744061c30 msgr2=0x7f27440d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 -- 172.21.15.19:0/1310652425 shutdown_connections 2024-04-25T10:01:16.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.119+0000 7f274a39d640 1 -- 172.21.15.19:0/1310652425 wait complete. 2024-04-25T10:01:16.135 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:01:16.135 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:01:16.135 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.232+0000 7ff6e3a2a640 1 Processor -- start 2024-04-25T10:01:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.233+0000 7ff6e3a2a640 1 -- start start 2024-04-25T10:01:16.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.233+0000 7ff6e3a2a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6dc15db60 0x7ff6dc0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.233+0000 7ff6e3a2a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6dc0b5f00 0x7ff6dc0d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.233+0000 7ff6e3a2a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6dc15df40 0x7ff6dc0d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.233+0000 7ff6e3a2a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6dc0ce930 con 0x7ff6dc15db60 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.233+0000 7ff6e3a2a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6dc0b6450 con 0x7ff6dc15df40 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.233+0000 7ff6e3a2a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0cec00 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.234+0000 7ff6e17c6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6dc15db60 0x7ff6dc0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.234+0000 7ff6e17c6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6dc15db60 0x7ff6dc0ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46054/0 (socket says 172.21.15.19:46054) 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.234+0000 7ff6e17c6640 1 -- 172.21.15.19:0/2671538028 learned_addr learned my addr 172.21.15.19:0/2671538028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.234+0000 7ff6e0fc5640 1 --2- 172.21.15.19:0/2671538028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6dc0b5f00 0x7ff6dc0d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.234+0000 7ff6e17c6640 1 -- 172.21.15.19:0/2671538028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6dc0b5f00 msgr2=0x7ff6dc0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.234+0000 7ff6e1fc7640 1 --2- 172.21.15.19:0/2671538028 >> v2:172.21.15.112:3300/0 conn(0x7ff6dc15df40 0x7ff6dc0d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.234+0000 7ff6e17c6640 1 --2- 172.21.15.19:0/2671538028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6dc0b5f00 0x7ff6dc0d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.234+0000 7ff6e17c6640 1 -- 172.21.15.19:0/2671538028 >> v2:172.21.15.112:3300/0 conn(0x7ff6dc15df40 msgr2=0x7ff6dc0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.234+0000 7ff6e17c6640 1 --2- 172.21.15.19:0/2671538028 >> v2:172.21.15.112:3300/0 conn(0x7ff6dc15df40 0x7ff6dc0d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.234+0000 7ff6e17c6640 1 -- 172.21.15.19:0/2671538028 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc0ceee0 con 0x7ff6dc15db60 2024-04-25T10:01:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6e17c6640 1 --2- 172.21.15.19:0/2671538028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6dc15db60 0x7ff6dc0ce3f0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc161890 tx=0x7ff6d802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1021e1c480532280 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6d3fff640 1 -- 172.21.15.19:0/2671538028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8037070 con 0x7ff6dc15db60 2024-04-25T10:01:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6d3fff640 1 -- 172.21.15.19:0/2671538028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d8032440 con 0x7ff6dc15db60 2024-04-25T10:01:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6d3fff640 1 -- 172.21.15.19:0/2671538028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8032810 con 0x7ff6dc15db60 2024-04-25T10:01:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/2671538028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6dc15db60 msgr2=0x7ff6dc0ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6e3a2a640 1 --2- 172.21.15.19:0/2671538028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6dc15db60 0x7ff6dc0ce3f0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc161890 tx=0x7ff6d802a180 comp rx=0 tx=0).stop 2024-04-25T10:01:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/2671538028 shutdown_connections 2024-04-25T10:01:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6e3a2a640 1 --2- 172.21.15.19:0/2671538028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6dc0b5f00 0x7ff6dc0d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6e3a2a640 1 --2- 172.21.15.19:0/2671538028 >> v2:172.21.15.112:3300/0 conn(0x7ff6dc15df40 0x7ff6dc0d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6e3a2a640 1 --2- 172.21.15.19:0/2671538028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6dc15db60 0x7ff6dc0ce3f0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/2671538028 >> 172.21.15.19:0/2671538028 conn(0x7ff6dc061600 msgr2=0x7ff6dc061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/2671538028 shutdown_connections 2024-04-25T10:01:16.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.235+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/2671538028 wait complete. 2024-04-25T10:01:16.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.237+0000 7ff6e3a2a640 1 Processor -- start 2024-04-25T10:01:16.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.237+0000 7ff6e3a2a640 1 -- start start 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.237+0000 7ff6e3a2a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6dc0b5f00 0x7ff6dc154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.237+0000 7ff6e3a2a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6dc15db60 0x7ff6dc15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.237+0000 7ff6e3a2a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6dc15df40 0x7ff6dc0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.237+0000 7ff6e3a2a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc161950 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.237+0000 7ff6e3a2a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0d1d50 con 0x7ff6dc15df40 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.237+0000 7ff6e3a2a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc161ad0 con 0x7ff6dc15db60 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.237+0000 7ff6e17c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6dc0b5f00 0x7ff6dc154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.237+0000 7ff6e17c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6dc0b5f00 0x7ff6dc154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46066/0 (socket says 172.21.15.19:46066) 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.237+0000 7ff6e17c6640 1 -- 172.21.15.19:0/190119399 learned_addr learned my addr 172.21.15.19:0/190119399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6e1fc7640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6dc15df40 0x7ff6dc0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6e17c6640 1 -- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6dc15db60 msgr2=0x7ff6dc15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6e17c6640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6dc15db60 0x7ff6dc15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6e17c6640 1 -- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6dc15df40 msgr2=0x7ff6dc0cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6e17c6640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6dc15df40 0x7ff6dc0cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6e17c6640 1 -- 172.21.15.19:0/190119399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc15ece0 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6e17c6640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6dc0b5f00 0x7ff6dc154b60 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8000c00 tx=0x7ff6d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6d1ffb640 1 -- 172.21.15.19:0/190119399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8037040 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6d1ffb640 1 -- 172.21.15.19:0/190119399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d8042070 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/190119399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6dc15efc0 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6d1ffb640 1 -- 172.21.15.19:0/190119399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8030bd0 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.238+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/190119399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6dc1f4010 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.239+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/190119399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6dc073270 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.240+0000 7ff6d1ffb640 1 -- 172.21.15.19:0/190119399 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6d8047460 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.240+0000 7ff6d1ffb640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6c006b4d0 0x7ff6c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.241+0000 7ff6d1ffb640 1 -- 172.21.15.19:0/190119399 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff6d80aac50 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.241+0000 7ff6e0fc5640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6c006b4d0 0x7ff6c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.241+0000 7ff6e0fc5640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6c006b4d0 0x7ff6c006d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc006b10 tx=0x7ff6cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.245+0000 7ff6d1ffb640 1 -- 172.21.15.19:0/190119399 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6d8045360 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.425+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/190119399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6dc073990 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.426+0000 7ff6d1ffb640 1 -- 172.21.15.19:0/190119399 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+13291 (secure 0 0 0) 0x7ff6d8079c30 con 0x7ff6dc0b5f00 2024-04-25T10:01:16.431 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:16.431 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":120,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:01:15.114691+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/3442229953":"2024-04-26T10:01:12.109953+0000","172.21.15.19:6833/3442229953":"2024-04-26T10:01:12.109953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6c006b4d0 msgr2=0x7ff6c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6c006b4d0 0x7ff6c006d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc006b10 tx=0x7ff6cc008040 comp rx=0 tx=0).stop 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/190119399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6dc0b5f00 msgr2=0x7ff6dc154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6dc0b5f00 0x7ff6dc154b60 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8000c00 tx=0x7ff6d80046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/190119399 shutdown_connections 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff6c006b4d0 0x7ff6c006d990 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6dc15df40 0x7ff6dc0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6dc15db60 0x7ff6dc15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 --2- 172.21.15.19:0/190119399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6dc0b5f00 0x7ff6dc154b60 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/190119399 >> 172.21.15.19:0/190119399 conn(0x7ff6dc061600 msgr2=0x7ff6dc15d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/190119399 shutdown_connections 2024-04-25T10:01:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.428+0000 7ff6e3a2a640 1 -- 172.21.15.19:0/190119399 wait complete. 2024-04-25T10:01:16.443 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:16.444 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:16.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.543+0000 7ff068c12640 1 Processor -- start 2024-04-25T10:01:16.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.543+0000 7ff068c12640 1 -- start start 2024-04-25T10:01:16.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.543+0000 7ff068c12640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 0x7ff0640cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.543+0000 7ff068c12640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff0640cbf20 0x7ff0640cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.543+0000 7ff068c12640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0640c75e0 con 0x7ff06415bb50 2024-04-25T10:01:16.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.543+0000 7ff068c12640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0640c8680 con 0x7ff0640cbf20 2024-04-25T10:01:16.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.543+0000 7ff068c12640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640cff80 con 0x7ff0640cf600 2024-04-25T10:01:16.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.544+0000 7ff062d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff0640cbf20 0x7ff0640cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.544+0000 7ff06259c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff06415bb50 0x7ff0640c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49914/0 (socket says 172.21.15.19:49914) 2024-04-25T10:01:16.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.544+0000 7ff06259c640 1 -- 172.21.15.19:0/1481265114 learned_addr learned my addr 172.21.15.19:0/1481265114 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:16.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.544+0000 7ff05bfff640 1 --2- 172.21.15.19:0/1481265114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 0x7ff0640cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.544+0000 7ff061d9b640 1 -- 172.21.15.19:0/1481265114 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426128634 0 0) 0x7ff0640c75e0 con 0x7ff06415bb50 2024-04-25T10:01:16.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.544+0000 7ff061d9b640 1 -- 172.21.15.19:0/1481265114 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff050003520 con 0x7ff06415bb50 2024-04-25T10:01:16.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.544+0000 7ff062d9d640 1 -- 172.21.15.19:0/1481265114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 msgr2=0x7ff0640cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.544+0000 7ff062d9d640 1 --2- 172.21.15.19:0/1481265114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 0x7ff0640cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.544+0000 7ff062d9d640 1 -- 172.21.15.19:0/1481265114 >> v1:172.21.15.19:6789/0 conn(0x7ff06415bb50 legacy=0x7ff0640c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.544+0000 7ff062d9d640 1 -- 172.21.15.19:0/1481265114 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640c78c0 con 0x7ff0640cbf20 2024-04-25T10:01:16.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.544+0000 7ff05bfff640 1 --2- 172.21.15.19:0/1481265114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 0x7ff0640cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:16.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff062d9d640 1 --2- 172.21.15.19:0/1481265114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0640cbf20 0x7ff0640cc3b0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009d90 tx=0x7ff05c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c325a156aa733f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff061d9b640 1 -- 172.21.15.19:0/1481265114 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c007740 con 0x7ff0640cbf20 2024-04-25T10:01:16.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff061d9b640 1 -- 172.21.15.19:0/1481265114 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c0078e0 con 0x7ff0640cbf20 2024-04-25T10:01:16.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff061d9b640 1 -- 172.21.15.19:0/1481265114 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c007cb0 con 0x7ff0640cbf20 2024-04-25T10:01:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff068c12640 1 -- 172.21.15.19:0/1481265114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0640cbf20 msgr2=0x7ff0640cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff068c12640 1 --2- 172.21.15.19:0/1481265114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0640cbf20 0x7ff0640cc3b0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009d90 tx=0x7ff05c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff068c12640 1 -- 172.21.15.19:0/1481265114 shutdown_connections 2024-04-25T10:01:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff068c12640 1 --2- 172.21.15.19:0/1481265114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0640cbf20 0x7ff0640cc3b0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff068c12640 1 --2- 172.21.15.19:0/1481265114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 0x7ff0640cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff068c12640 1 -- 172.21.15.19:0/1481265114 >> 172.21.15.19:0/1481265114 conn(0x7ff06406f500 msgr2=0x7ff06406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff068c12640 1 -- 172.21.15.19:0/1481265114 shutdown_connections 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.545+0000 7ff068c12640 1 -- 172.21.15.19:0/1481265114 wait complete. 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.547+0000 7ff068c12640 1 Processor -- start 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.547+0000 7ff068c12640 1 -- start start 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.547+0000 7ff068c12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0640cbf20 0x7ff06414bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.547+0000 7ff068c12640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 0x7ff06415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff068c12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06415bb50 0x7ff0640afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff068c12640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640c8160 con 0x7ff0640cbf20 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff068c12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640c7e80 con 0x7ff06415bb50 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff068c12640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640c72a0 con 0x7ff0640cf600 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff06259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0640cbf20 0x7ff06414bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff06259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0640cbf20 0x7ff06414bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46096/0 (socket says 172.21.15.19:46096) 2024-04-25T10:01:16.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff06259c640 1 -- 172.21.15.19:0/448871291 learned_addr learned my addr 172.21.15.19:0/448871291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff06259c640 1 -- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 msgr2=0x7ff06415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff062d9d640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06415bb50 0x7ff0640afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff05bfff640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 0x7ff06415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff06259c640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 0x7ff06415e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff06259c640 1 -- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06415bb50 msgr2=0x7ff0640afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff06259c640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06415bb50 0x7ff0640afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff06259c640 1 -- 172.21.15.19:0/448871291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06414c1b0 con 0x7ff0640cbf20 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff062d9d640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06415bb50 0x7ff0640afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff05bfff640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 0x7ff06415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff06259c640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0640cbf20 0x7ff06414bc70 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7ff05400c390 tx=0x7ff05400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff05affd640 1 -- 172.21.15.19:0/448871291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054017070 con 0x7ff0640cbf20 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff05affd640 1 -- 172.21.15.19:0/448871291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054005ce0 con 0x7ff0640cbf20 2024-04-25T10:01:16.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff068c12640 1 -- 172.21.15.19:0/448871291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06415ec70 con 0x7ff0640cbf20 2024-04-25T10:01:16.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff05affd640 1 -- 172.21.15.19:0/448871291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0540072f0 con 0x7ff0640cbf20 2024-04-25T10:01:16.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.548+0000 7ff068c12640 1 -- 172.21.15.19:0/448871291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06415bfd0 con 0x7ff0640cbf20 2024-04-25T10:01:16.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.549+0000 7ff068c12640 1 -- 172.21.15.19:0/448871291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff064073a00 con 0x7ff0640cbf20 2024-04-25T10:01:16.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.550+0000 7ff05affd640 1 -- 172.21.15.19:0/448871291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff05401a070 con 0x7ff0640cbf20 2024-04-25T10:01:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.550+0000 7ff05affd640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff04406b4d0 0x7ff04406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.550+0000 7ff05affd640 1 -- 172.21.15.19:0/448871291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff0540894b0 con 0x7ff0640cbf20 2024-04-25T10:01:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.550+0000 7ff05bfff640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff04406b4d0 0x7ff04406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.551+0000 7ff05bfff640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff04406b4d0 0x7ff04406d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff04c006fd0 tx=0x7ff04c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.554+0000 7ff05affd640 1 -- 172.21.15.19:0/448871291 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0540584a0 con 0x7ff0640cbf20 2024-04-25T10:01:16.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.732+0000 7ff068c12640 1 -- 172.21.15.19:0/448871291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0640ce5d0 con 0x7ff0640cbf20 2024-04-25T10:01:16.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.733+0000 7ff05affd640 1 -- 172.21.15.19:0/448871291 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff054057da0 con 0x7ff0640cbf20 2024-04-25T10:01:16.737 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:01:16.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 -- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff04406b4d0 msgr2=0x7ff04406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff04406b4d0 0x7ff04406d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff04c006fd0 tx=0x7ff04c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:16.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 -- 172.21.15.19:0/448871291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0640cbf20 msgr2=0x7ff06414bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0640cbf20 0x7ff06414bc70 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7ff05400c390 tx=0x7ff05400c850 comp rx=0 tx=0).stop 2024-04-25T10:01:16.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 -- 172.21.15.19:0/448871291 shutdown_connections 2024-04-25T10:01:16.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff04406b4d0 0x7ff04406d990 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06415bb50 0x7ff0640afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0640cf600 0x7ff06415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 --2- 172.21.15.19:0/448871291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0640cbf20 0x7ff06414bc70 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 -- 172.21.15.19:0/448871291 >> 172.21.15.19:0/448871291 conn(0x7ff06406f500 msgr2=0x7ff0640cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 -- 172.21.15.19:0/448871291 shutdown_connections 2024-04-25T10:01:16.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.735+0000 7ff068c12640 1 -- 172.21.15.19:0/448871291 wait complete. 2024-04-25T10:01:16.750 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:01:16.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.847+0000 7f049b170640 1 Processor -- start 2024-04-25T10:01:16.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.847+0000 7f049b170640 1 -- start start 2024-04-25T10:01:16.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.848+0000 7f049b170640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049406ce10 0x7f04940d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.848+0000 7f049b170640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f049409a450 0x7f04940cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.848+0000 7f049b170640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049406c220 con 0x7f049415dc70 2024-04-25T10:01:16.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.848+0000 7f049b170640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04940d0c90 con 0x7f049409a450 2024-04-25T10:01:16.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.848+0000 7f049b170640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04940d0080 con 0x7f049406ce10 2024-04-25T10:01:16.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.848+0000 7f0498f0c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f049415dc70 0x7f049406c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49916/0 (socket says 172.21.15.19:49916) 2024-04-25T10:01:16.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.848+0000 7f0498f0c640 1 -- 172.21.15.19:0/364430375 learned_addr learned my addr 172.21.15.19:0/364430375 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:16.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.848+0000 7f0493fff640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049406ce10 0x7f04940d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.848+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597911229 0 0) 0x7f049406c220 con 0x7f049415dc70 2024-04-25T10:01:16.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.848+0000 7f049970d640 1 --2- 172.21.15.19:0/364430375 >> v2:172.21.15.112:3300/0 conn(0x7f049409a450 0x7f04940cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.848+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0484003520 con 0x7f049415dc70 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2826721444 0 0) 0x7f0484003520 con 0x7f049415dc70 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049406ce10 msgr2=0x7f04940d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049406ce10 0x7f04940d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 >> v2:172.21.15.112:3300/0 conn(0x7f049409a450 msgr2=0x7f04940cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 --2- 172.21.15.19:0/364430375 >> v2:172.21.15.112:3300/0 conn(0x7f049409a450 0x7f04940cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940d0360 con 0x7f049415dc70 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0488001670 con 0x7f049415dc70 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 >> v1:172.21.15.19:6789/0 conn(0x7f049415dc70 legacy=0x7f049406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 0x7f0484003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0484004800 0x7f0484004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049406ce10 0x7f04840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940d0c90 con 0x7f04840039b0 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940d0080 con 0x7f0484004800 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048400b9c0 con 0x7f049406ce10 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f0498f0c640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 0x7f0484003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f0493fff640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0484004800 0x7f0484004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f049970d640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049406ce10 0x7f04840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f0498f0c640 1 -- 172.21.15.19:0/364430375 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049406ce10 msgr2=0x7f04840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f0498f0c640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049406ce10 0x7f04840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f0498f0c640 1 -- 172.21.15.19:0/364430375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0484004800 msgr2=0x7f0484004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f0498f0c640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0484004800 0x7f0484004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.849+0000 7f0498f0c640 1 -- 172.21.15.19:0/364430375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0484003520 con 0x7f04840039b0 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.850+0000 7f0498f0c640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 0x7f0484003dd0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f048800c290 tx=0x7f0484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a97094ff9715a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.850+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488007d50 con 0x7f04840039b0 2024-04-25T10:01:16.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.850+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0488005af0 con 0x7f04840039b0 2024-04-25T10:01:16.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.850+0000 7f04937fe640 1 -- 172.21.15.19:0/364430375 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488007070 con 0x7f04840039b0 2024-04-25T10:01:16.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.850+0000 7f049b170640 1 -- 172.21.15.19:0/364430375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 msgr2=0x7f0484003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.850+0000 7f049b170640 1 --2- 172.21.15.19:0/364430375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 0x7f0484003dd0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f048800c290 tx=0x7f0484005830 comp rx=0 tx=0).stop 2024-04-25T10:01:16.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.851+0000 7f049970d640 1 -- 172.21.15.19:0/364430375 reap_dead start 2024-04-25T10:01:16.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.851+0000 7f049b170640 1 -- 172.21.15.19:0/364430375 shutdown_connections 2024-04-25T10:01:16.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.851+0000 7f049b170640 1 -- 172.21.15.19:0/364430375 >> 172.21.15.19:0/364430375 conn(0x7f04940b3270 msgr2=0x7f04940b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.851+0000 7f049b170640 1 -- 172.21.15.19:0/364430375 shutdown_connections 2024-04-25T10:01:16.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.851+0000 7f049b170640 1 -- 172.21.15.19:0/364430375 wait complete. 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.852+0000 7f049b170640 1 Processor -- start 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.852+0000 7f049b170640 1 -- start start 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f049b170640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 0x7f049409c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f049b170640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0494156d70 0x7f049409c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f049b170640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049409cca0 0x7f04940c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f049b170640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049415e050 con 0x7f04840039b0 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f049b170640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940d0920 con 0x7f0494156d70 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f049b170640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04940d1120 con 0x7f049409cca0 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f0498f0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 0x7f049409c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f0498f0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 0x7f049409c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46124/0 (socket says 172.21.15.19:46124) 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f0498f0c640 1 -- 172.21.15.19:0/2355064988 learned_addr learned my addr 172.21.15.19:0/2355064988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f0493fff640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0494156d70 0x7f049409c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f049970d640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049409cca0 0x7f04940c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f0498f0c640 1 -- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049409cca0 msgr2=0x7f04940c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f0498f0c640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049409cca0 0x7f04940c73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f0498f0c640 1 -- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0494156d70 msgr2=0x7f049409c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f0498f0c640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0494156d70 0x7f049409c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f0498f0c640 1 -- 172.21.15.19:0/2355064988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049409d120 con 0x7f04840039b0 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f0498f0c640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 0x7f049409c1b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f0488000c00 tx=0x7f04940d0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f04917fa640 1 -- 172.21.15.19:0/2355064988 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488042070 con 0x7f04840039b0 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f04917fa640 1 -- 172.21.15.19:0/2355064988 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0488007630 con 0x7f04840039b0 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f04917fa640 1 -- 172.21.15.19:0/2355064988 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488007d50 con 0x7f04840039b0 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.853+0000 7f049b170640 1 -- 172.21.15.19:0/2355064988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049409b6d0 con 0x7f04840039b0 2024-04-25T10:01:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.854+0000 7f049b170640 1 -- 172.21.15.19:0/2355064988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049409bc90 con 0x7f04840039b0 2024-04-25T10:01:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.854+0000 7f049b170640 1 -- 172.21.15.19:0/2355064988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0494073990 con 0x7f04840039b0 2024-04-25T10:01:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.855+0000 7f04917fa640 1 -- 172.21.15.19:0/2355064988 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0488045070 con 0x7f04840039b0 2024-04-25T10:01:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.855+0000 7f04917fa640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f047806b4d0 0x7f047806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.855+0000 7f04917fa640 1 -- 172.21.15.19:0/2355064988 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f04880aaaf0 con 0x7f04840039b0 2024-04-25T10:01:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.855+0000 7f0493fff640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f047806b4d0 0x7f047806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.856+0000 7f0493fff640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f047806b4d0 0x7f047806d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f04940c9d40 tx=0x7f0480008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:16.859+0000 7f04917fa640 1 -- 172.21.15.19:0/2355064988 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f048804c030 con 0x7f04840039b0 2024-04-25T10:01:17.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.111+0000 7f049b170640 1 -- 172.21.15.19:0/2355064988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f04940b5f20 con 0x7f04840039b0 2024-04-25T10:01:17.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.123+0000 7f04917fa640 1 -- 172.21.15.19:0/2355064988 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v409) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0488079c50 con 0x7f04840039b0 2024-04-25T10:01:17.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 -- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f047806b4d0 msgr2=0x7f047806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f047806b4d0 0x7f047806d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f04940c9d40 tx=0x7f0480008040 comp rx=0 tx=0).stop 2024-04-25T10:01:17.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 -- 172.21.15.19:0/2355064988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 msgr2=0x7f049409c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 0x7f049409c1b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f0488000c00 tx=0x7f04940d0c00 comp rx=0 tx=0).stop 2024-04-25T10:01:17.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 -- 172.21.15.19:0/2355064988 shutdown_connections 2024-04-25T10:01:17.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f047806b4d0 0x7f047806d990 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f049409cca0 0x7f04940c73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0494156d70 0x7f049409c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 --2- 172.21.15.19:0/2355064988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04840039b0 0x7f049409c1b0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 -- 172.21.15.19:0/2355064988 >> 172.21.15.19:0/2355064988 conn(0x7f04940b3270 msgr2=0x7f04940ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 -- 172.21.15.19:0/2355064988 shutdown_connections 2024-04-25T10:01:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.125+0000 7f049b170640 1 -- 172.21.15.19:0/2355064988 wait complete. 2024-04-25T10:01:17.141 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:01:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.241+0000 7fc14e15c640 1 Processor -- start 2024-04-25T10:01:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.241+0000 7fc14e15c640 1 -- start start 2024-04-25T10:01:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc14e15c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1480ced80 0x7fc1480d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc14e15c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc148161710 0x7fc1480d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc14e15c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc14815e000 con 0x7fc14815dc20 2024-04-25T10:01:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc14e15c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1480d1de0 con 0x7fc1480ced80 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc14e15c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc148161c60 con 0x7fc148161710 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc146ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1480ced80 0x7fc1480d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc1477fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc14815dc20 0x7fc1480ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49926/0 (socket says 172.21.15.19:49926) 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc1477fe640 1 -- 172.21.15.19:0/3458948584 learned_addr learned my addr 172.21.15.19:0/3458948584 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc147fff640 1 --2- 172.21.15.19:0/3458948584 >> v2:172.21.15.112:3300/0 conn(0x7fc148161710 0x7fc1480d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc146ffd640 1 -- 172.21.15.19:0/3458948584 >> v2:172.21.15.112:3300/0 conn(0x7fc148161710 msgr2=0x7fc1480d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc146ffd640 1 --2- 172.21.15.19:0/3458948584 >> v2:172.21.15.112:3300/0 conn(0x7fc148161710 0x7fc1480d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc146ffd640 1 -- 172.21.15.19:0/3458948584 >> v1:172.21.15.19:6789/0 conn(0x7fc14815dc20 legacy=0x7fc1480ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc146ffd640 1 -- 172.21.15.19:0/3458948584 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148161f40 con 0x7fc1480ced80 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.242+0000 7fc1467fc640 1 -- 172.21.15.19:0/3458948584 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828185983 0 0) 0x7fc14815e000 con 0x7fc14815dc20 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.243+0000 7fc146ffd640 1 --2- 172.21.15.19:0/3458948584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1480ced80 0x7fc1480d22e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fc1480b6570 tx=0x7fc13c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1555f2a4a602a896 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.243+0000 7fc1467fc640 1 -- 172.21.15.19:0/3458948584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c037070 con 0x7fc1480ced80 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.243+0000 7fc1467fc640 1 -- 172.21.15.19:0/3458948584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c032440 con 0x7fc1480ced80 2024-04-25T10:01:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.243+0000 7fc1467fc640 1 -- 172.21.15.19:0/3458948584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c032810 con 0x7fc1480ced80 2024-04-25T10:01:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.244+0000 7fc14e15c640 1 -- 172.21.15.19:0/3458948584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1480ced80 msgr2=0x7fc1480d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.244+0000 7fc14e15c640 1 --2- 172.21.15.19:0/3458948584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1480ced80 0x7fc1480d22e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fc1480b6570 tx=0x7fc13c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.244+0000 7fc14e15c640 1 -- 172.21.15.19:0/3458948584 shutdown_connections 2024-04-25T10:01:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.244+0000 7fc14e15c640 1 --2- 172.21.15.19:0/3458948584 >> v2:172.21.15.112:3300/0 conn(0x7fc148161710 0x7fc1480d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.244+0000 7fc14e15c640 1 --2- 172.21.15.19:0/3458948584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1480ced80 0x7fc1480d22e0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.244+0000 7fc14e15c640 1 -- 172.21.15.19:0/3458948584 >> 172.21.15.19:0/3458948584 conn(0x7fc1480d0420 msgr2=0x7fc1480d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.244+0000 7fc14e15c640 1 -- 172.21.15.19:0/3458948584 shutdown_connections 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.244+0000 7fc14e15c640 1 -- 172.21.15.19:0/3458948584 wait complete. 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.245+0000 7fc14e15c640 1 Processor -- start 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.245+0000 7fc14e15c640 1 -- start start 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc14e15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1480ced80 0x7fc148160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc14e15c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc14815dc20 0x7fc148158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc14e15c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc148161710 0x7fc14815e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc14e15c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480b5e50 con 0x7fc1480ced80 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc14e15c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc148162500 con 0x7fc148161710 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc14e15c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480b5fd0 con 0x7fc14815dc20 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc1477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1480ced80 0x7fc148160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc1477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1480ced80 0x7fc148160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46146/0 (socket says 172.21.15.19:46146) 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc1477fe640 1 -- 172.21.15.19:0/3691618641 learned_addr learned my addr 172.21.15.19:0/3691618641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc147fff640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc148161710 0x7fc14815e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc1477fe640 1 -- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc14815dc20 msgr2=0x7fc148158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc146ffd640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc14815dc20 0x7fc148158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc1477fe640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc14815dc20 0x7fc148158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc1477fe640 1 -- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc148161710 msgr2=0x7fc14815e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc1477fe640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc148161710 0x7fc14815e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc1477fe640 1 -- 172.21.15.19:0/3691618641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1480c97f0 con 0x7fc1480ced80 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc147fff640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc148161710 0x7fc14815e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.246+0000 7fc1477fe640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1480ced80 0x7fc148160cb0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c430 tx=0x7fc13800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.247+0000 7fc12ffff640 1 -- 172.21.15.19:0/3691618641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1380074e0 con 0x7fc1480ced80 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.247+0000 7fc12ffff640 1 -- 172.21.15.19:0/3691618641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1380047e0 con 0x7fc1480ced80 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.247+0000 7fc14e15c640 1 -- 172.21.15.19:0/3691618641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1480c9ad0 con 0x7fc1480ced80 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.247+0000 7fc12ffff640 1 -- 172.21.15.19:0/3691618641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138004bd0 con 0x7fc1480ced80 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.247+0000 7fc14e15c640 1 -- 172.21.15.19:0/3691618641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1481589a0 con 0x7fc1480ced80 2024-04-25T10:01:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.248+0000 7fc14e15c640 1 -- 172.21.15.19:0/3691618641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1480c9ff0 con 0x7fc1480ced80 2024-04-25T10:01:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.248+0000 7fc12ffff640 1 -- 172.21.15.19:0/3691618641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc138011070 con 0x7fc1480ced80 2024-04-25T10:01:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.249+0000 7fc12ffff640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc13406b4d0 0x7fc13406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.249+0000 7fc12ffff640 1 -- 172.21.15.19:0/3691618641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc138089d30 con 0x7fc1480ced80 2024-04-25T10:01:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.249+0000 7fc146ffd640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc13406b4d0 0x7fc13406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.249+0000 7fc146ffd640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc13406b4d0 0x7fc13406d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc13c034000 tx=0x7fc13c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:17.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.253+0000 7fc12ffff640 1 -- 172.21.15.19:0/3691618641 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc13800e280 con 0x7fc1480ced80 2024-04-25T10:01:17.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.431+0000 7fc14e15c640 1 -- 172.21.15.19:0/3691618641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc148073a00 con 0x7fc1480ced80 2024-04-25T10:01:17.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.437+0000 7fc12ffff640 1 -- 172.21.15.19:0/3691618641 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc138058d30 con 0x7fc1480ced80 2024-04-25T10:01:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 -- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc13406b4d0 msgr2=0x7fc13406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc13406b4d0 0x7fc13406d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc13c034000 tx=0x7fc13c02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 -- 172.21.15.19:0/3691618641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1480ced80 msgr2=0x7fc148160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1480ced80 0x7fc148160cb0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c430 tx=0x7fc13800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 -- 172.21.15.19:0/3691618641 shutdown_connections 2024-04-25T10:01:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc13406b4d0 0x7fc13406d990 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc148161710 0x7fc14815e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc14815dc20 0x7fc148158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 --2- 172.21.15.19:0/3691618641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1480ced80 0x7fc148160cb0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 -- 172.21.15.19:0/3691618641 >> 172.21.15.19:0/3691618641 conn(0x7fc1480d0420 msgr2=0x7fc1480d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 -- 172.21.15.19:0/3691618641 shutdown_connections 2024-04-25T10:01:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.439+0000 7fc14e15c640 1 -- 172.21.15.19:0/3691618641 wait complete. 2024-04-25T10:01:17.454 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:01:17.455 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:17.497 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:17.540 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:01:17.540 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:01:17.540 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:17.556 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:17.556 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:01:17.556 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:17.597 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:17.598 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:01:17.598 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:17.613 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:17.613 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:01:17.613 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:17.657 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:17.657 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:01:17.657 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:17.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.753+0000 7fb3c88d1640 1 Processor -- start 2024-04-25T10:01:17.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.753+0000 7fb3c88d1640 1 -- start start 2024-04-25T10:01:17.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.754+0000 7fb3c88d1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3c4144750 0x7fb3c406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.754+0000 7fb3c88d1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3c40cf030 0x7fb3c406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.754+0000 7fb3c88d1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c4144b30 con 0x7fb3c406eb10 2024-04-25T10:01:17.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.754+0000 7fb3c88d1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c40712d0 con 0x7fb3c40cf030 2024-04-25T10:01:17.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.754+0000 7fb3c88d1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c40715b0 con 0x7fb3c4144750 2024-04-25T10:01:17.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.754+0000 7fb3c2d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3c40cf030 0x7fb3c406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.754+0000 7fb3c1d9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb3c406eb10 0x7fb3c40ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50144/0 (socket says 172.21.15.19:50144) 2024-04-25T10:01:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.755+0000 7fb3c1d9b640 1 -- 172.21.15.19:0/180196394 learned_addr learned my addr 172.21.15.19:0/180196394 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.755+0000 7fb3c2d9d640 1 -- 172.21.15.19:0/180196394 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3c4144750 msgr2=0x7fb3c406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.755+0000 7fb3c259c640 1 --2- 172.21.15.19:0/180196394 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3c4144750 0x7fb3c406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.755+0000 7fb3c2d9d640 1 --2- 172.21.15.19:0/180196394 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3c4144750 0x7fb3c406e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.755+0000 7fb3c159a640 1 -- 172.21.15.19:0/180196394 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071267387 0 0) 0x7fb3c4144b30 con 0x7fb3c406eb10 2024-04-25T10:01:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.755+0000 7fb3c2d9d640 1 -- 172.21.15.19:0/180196394 >> v1:172.21.15.19:6789/0 conn(0x7fb3c406eb10 legacy=0x7fb3c40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.755+0000 7fb3c2d9d640 1 -- 172.21.15.19:0/180196394 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c4071890 con 0x7fb3c40cf030 2024-04-25T10:01:17.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.755+0000 7fb3c259c640 1 --2- 172.21.15.19:0/180196394 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3c4144750 0x7fb3c406e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c2d9d640 1 --2- 172.21.15.19:0/180196394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3c40cf030 0x7fb3c406d290 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009d90 tx=0x7fb3bc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2803b2db770289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:17.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c159a640 1 -- 172.21.15.19:0/180196394 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc031600 con 0x7fb3c40cf030 2024-04-25T10:01:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c159a640 1 -- 172.21.15.19:0/180196394 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc0317a0 con 0x7fb3c40cf030 2024-04-25T10:01:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c159a640 1 -- 172.21.15.19:0/180196394 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc031b70 con 0x7fb3c40cf030 2024-04-25T10:01:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c88d1640 1 -- 172.21.15.19:0/180196394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3c40cf030 msgr2=0x7fb3c406d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c88d1640 1 --2- 172.21.15.19:0/180196394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3c40cf030 0x7fb3c406d290 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009d90 tx=0x7fb3bc02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c88d1640 1 -- 172.21.15.19:0/180196394 shutdown_connections 2024-04-25T10:01:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c88d1640 1 --2- 172.21.15.19:0/180196394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3c40cf030 0x7fb3c406d290 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c88d1640 1 --2- 172.21.15.19:0/180196394 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3c4144750 0x7fb3c406e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c88d1640 1 -- 172.21.15.19:0/180196394 >> 172.21.15.19:0/180196394 conn(0x7fb3c406abe0 msgr2=0x7fb3c406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c88d1640 1 -- 172.21.15.19:0/180196394 shutdown_connections 2024-04-25T10:01:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.756+0000 7fb3c88d1640 1 -- 172.21.15.19:0/180196394 wait complete. 2024-04-25T10:01:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.758+0000 7fb3c88d1640 1 Processor -- start 2024-04-25T10:01:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.758+0000 7fb3c88d1640 1 -- start start 2024-04-25T10:01:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.758+0000 7fb3c88d1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3c406eb10 0x7fb3c412f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.758+0000 7fb3c88d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3c40cf030 0x7fb3c415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c88d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3c4144750 0x7fb3c415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c88d1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c41455d0 con 0x7fb3c4144750 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c88d1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c4145450 con 0x7fb3c40cf030 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c88d1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c415c000 con 0x7fb3c406eb10 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3c4144750 0x7fb3c415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3c4144750 0x7fb3c415a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59360/0 (socket says 172.21.15.19:59360) 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c2d9d640 1 -- 172.21.15.19:0/2056173316 learned_addr learned my addr 172.21.15.19:0/2056173316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c1d9b640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3c40cf030 0x7fb3c415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c259c640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3c406eb10 0x7fb3c412f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c2d9d640 1 -- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3c406eb10 msgr2=0x7fb3c412f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c2d9d640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3c406eb10 0x7fb3c412f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c2d9d640 1 -- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3c40cf030 msgr2=0x7fb3c415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c2d9d640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3c40cf030 0x7fb3c415ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c2d9d640 1 -- 172.21.15.19:0/2056173316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c41308f0 con 0x7fb3c4144750 2024-04-25T10:01:17.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c2d9d640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3c4144750 0x7fb3c415a720 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009eb0 tx=0x7fb3bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:17.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3b2ffd640 1 -- 172.21.15.19:0/2056173316 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc03f070 con 0x7fb3c4144750 2024-04-25T10:01:17.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3b2ffd640 1 -- 172.21.15.19:0/2056173316 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc030be0 con 0x7fb3c4144750 2024-04-25T10:01:17.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c88d1640 1 -- 172.21.15.19:0/2056173316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c4130330 con 0x7fb3c4144750 2024-04-25T10:01:17.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3b2ffd640 1 -- 172.21.15.19:0/2056173316 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc009450 con 0x7fb3c4144750 2024-04-25T10:01:17.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.759+0000 7fb3c88d1640 1 -- 172.21.15.19:0/2056173316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c412d9f0 con 0x7fb3c4144750 2024-04-25T10:01:17.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.760+0000 7fb3c88d1640 1 -- 172.21.15.19:0/2056173316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c4073990 con 0x7fb3c4144750 2024-04-25T10:01:17.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.761+0000 7fb3b2ffd640 1 -- 172.21.15.19:0/2056173316 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3bc003990 con 0x7fb3c4144750 2024-04-25T10:01:17.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.761+0000 7fb3b2ffd640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3a406b4d0 0x7fb3a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.761+0000 7fb3b2ffd640 1 -- 172.21.15.19:0/2056173316 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb3bc0ab0c0 con 0x7fb3c4144750 2024-04-25T10:01:17.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.761+0000 7fb3c259c640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3a406b4d0 0x7fb3a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.762+0000 7fb3c259c640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3a406b4d0 0x7fb3a406d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400a990 tx=0x7fb3b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:17.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.766+0000 7fb3b2ffd640 1 -- 172.21.15.19:0/2056173316 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3bc07a0a0 con 0x7fb3c4144750 2024-04-25T10:01:17.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.982+0000 7fb3c88d1640 1 -- 172.21.15.19:0/2056173316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3c40d0510 con 0x7fb3c4144750 2024-04-25T10:01:17.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.982+0000 7fb3b2ffd640 1 -- 172.21.15.19:0/2056173316 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fb3bc0799a0 con 0x7fb3c4144750 2024-04-25T10:01:17.987 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:17.987 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/164064689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":164064689},{"type":"v1","addr":"172.21.15.112:6836","nonce":164064689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7600,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4121275569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4121275569},{"type":"v1","addr":"172.21.15.19:6835","nonce":4121275569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1324136290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1324136290},{"type":"v1","addr":"172.21.15.112:6837","nonce":1324136290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":9123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/484918755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":484918755},{"type":"v1","addr":"172.21.15.19:6834","nonce":484918755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[]} 2024-04-25T10:01:17.987 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 74 2024-04-25T10:01:17.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 -- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3a406b4d0 msgr2=0x7fb3a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3a406b4d0 0x7fb3a406d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400a990 tx=0x7fb3b4008040 comp rx=0 tx=0).stop 2024-04-25T10:01:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 -- 172.21.15.19:0/2056173316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3c4144750 msgr2=0x7fb3c415a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3c4144750 0x7fb3c415a720 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009eb0 tx=0x7fb3bc004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 -- 172.21.15.19:0/2056173316 shutdown_connections 2024-04-25T10:01:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3a406b4d0 0x7fb3a406d990 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3c4144750 0x7fb3c415a720 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3c40cf030 0x7fb3c415ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 --2- 172.21.15.19:0/2056173316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3c406eb10 0x7fb3c412f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 -- 172.21.15.19:0/2056173316 >> 172.21.15.19:0/2056173316 conn(0x7fb3c406abe0 msgr2=0x7fb3c4143430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 -- 172.21.15.19:0/2056173316 shutdown_connections 2024-04-25T10:01:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:17.984+0000 7fb3c88d1640 1 -- 172.21.15.19:0/2056173316 wait complete. 2024-04-25T10:01:17.999 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:01:18.000 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:01:18.000 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:01:18.000 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:18.000 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:01:18.000 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:01:18.000 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:01:18.000 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:18.001 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:01:18.001 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:01:18.001 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:01:18.001 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:18.001 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:01:18.001 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:01:18.001 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:01:18.001 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:18.009 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:01:18.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:01:18.010 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:01:18.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:01:18.013 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:01:18.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:01:18.051 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:01:18.052 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:18.052 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:01:18.052 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:18.055 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:01:18.056 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:18.125 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T10:01:18.125 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:01:18.136 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T10:01:23.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:01:24.013 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:01:24.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:01:24.016 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:01:24.016 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:24.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:01:24.099 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T10:01:24.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.127+0000 7f9cbe142640 1 Processor -- start 2024-04-25T10:01:24.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.127+0000 7f9cbe142640 1 -- start start 2024-04-25T10:01:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.128+0000 7f9cbe142640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9cb815dc20 0x7f9cb80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.128+0000 7f9cbe142640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9cb8161710 0x7f9cb80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.128+0000 7f9cbe142640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cb80b9330 0x7f9cb806bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.128+0000 7f9cbe142640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb815e000 con 0x7f9cb8161710 2024-04-25T10:01:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.128+0000 7f9cbe142640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb80d1de0 con 0x7f9cb815dc20 2024-04-25T10:01:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.128+0000 7f9cbe142640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb8161c60 con 0x7f9cb80b9330 2024-04-25T10:01:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.128+0000 7f9cb6ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9cb8161710 0x7f9cb80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.128+0000 7f9cb6ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9cb8161710 0x7f9cb80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59410/0 (socket says 172.21.15.19:59410) 2024-04-25T10:01:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.128+0000 7f9cb6ffd640 1 -- 172.21.15.19:0/3092397544 learned_addr learned my addr 172.21.15.19:0/3092397544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.129+0000 7f9cb7fff640 1 --2- 172.21.15.19:0/3092397544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cb80b9330 0x7f9cb806bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.129+0000 7f9cb77fe640 1 --2- 172.21.15.19:0/3092397544 >> v2:172.21.15.112:3300/0 conn(0x7f9cb815dc20 0x7f9cb80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.129+0000 7f9cb6ffd640 1 -- 172.21.15.19:0/3092397544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cb80b9330 msgr2=0x7f9cb806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.129+0000 7f9cb6ffd640 1 --2- 172.21.15.19:0/3092397544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cb80b9330 0x7f9cb806bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.129+0000 7f9cb6ffd640 1 -- 172.21.15.19:0/3092397544 >> v2:172.21.15.112:3300/0 conn(0x7f9cb815dc20 msgr2=0x7f9cb80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.129+0000 7f9cb6ffd640 1 --2- 172.21.15.19:0/3092397544 >> v2:172.21.15.112:3300/0 conn(0x7f9cb815dc20 0x7f9cb80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.129+0000 7f9cb6ffd640 1 -- 172.21.15.19:0/3092397544 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb8161f40 con 0x7f9cb8161710 2024-04-25T10:01:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.129+0000 7f9cb77fe640 1 --2- 172.21.15.19:0/3092397544 >> v2:172.21.15.112:3300/0 conn(0x7f9cb815dc20 0x7f9cb80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.129+0000 7f9cb6ffd640 1 --2- 172.21.15.19:0/3092397544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cb8161710 0x7f9cb80cebb0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80d1520 tx=0x7f9cac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae7f0edb757351d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cb67fc640 1 -- 172.21.15.19:0/3092397544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac037070 con 0x7f9cb8161710 2024-04-25T10:01:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cb67fc640 1 -- 172.21.15.19:0/3092397544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cac032440 con 0x7f9cb8161710 2024-04-25T10:01:24.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cb67fc640 1 -- 172.21.15.19:0/3092397544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac032810 con 0x7f9cb8161710 2024-04-25T10:01:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cbe142640 1 -- 172.21.15.19:0/3092397544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cb8161710 msgr2=0x7f9cb80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cbe142640 1 --2- 172.21.15.19:0/3092397544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cb8161710 0x7f9cb80cebb0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80d1520 tx=0x7f9cac02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cbe142640 1 -- 172.21.15.19:0/3092397544 shutdown_connections 2024-04-25T10:01:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cbe142640 1 --2- 172.21.15.19:0/3092397544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cb80b9330 0x7f9cb806bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cbe142640 1 --2- 172.21.15.19:0/3092397544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cb8161710 0x7f9cb80cebb0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cbe142640 1 --2- 172.21.15.19:0/3092397544 >> v2:172.21.15.112:3300/0 conn(0x7f9cb815dc20 0x7f9cb80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cbe142640 1 -- 172.21.15.19:0/3092397544 >> 172.21.15.19:0/3092397544 conn(0x7f9cb80d0420 msgr2=0x7f9cb80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cbe142640 1 -- 172.21.15.19:0/3092397544 shutdown_connections 2024-04-25T10:01:24.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.130+0000 7f9cbe142640 1 -- 172.21.15.19:0/3092397544 wait complete. 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.131+0000 7f9cbe142640 1 Processor -- start 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.132+0000 7f9cbe142640 1 -- start start 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.132+0000 7f9cbe142640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cb80b9330 0x7f9cb81e9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.132+0000 7f9cbe142640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cb815dc20 0x7f9cb80c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.132+0000 7f9cbe142640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8161710 0x7f9cb81475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.132+0000 7f9cbe142640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb80b5e50 con 0x7f9cb80b9330 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.132+0000 7f9cbe142640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb8162500 con 0x7f9cb8161710 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.132+0000 7f9cbe142640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb80b5fd0 con 0x7f9cb815dc20 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.132+0000 7f9cb77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cb80b9330 0x7f9cb81e9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cb77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cb80b9330 0x7f9cb81e9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59420/0 (socket says 172.21.15.19:59420) 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cb77fe640 1 -- 172.21.15.19:0/547972566 learned_addr learned my addr 172.21.15.19:0/547972566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cb6ffd640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cb815dc20 0x7f9cb80c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cb7fff640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8161710 0x7f9cb81475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cb77fe640 1 -- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cb815dc20 msgr2=0x7f9cb80c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cb77fe640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cb815dc20 0x7f9cb80c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cb77fe640 1 -- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8161710 msgr2=0x7f9cb81475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cb77fe640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8161710 0x7f9cb81475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cb77fe640 1 -- 172.21.15.19:0/547972566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb80caec0 con 0x7f9cb80b9330 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cb7fff640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8161710 0x7f9cb81475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cb77fe640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cb80b9330 0x7f9cb81e9af0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800d7d0 tx=0x7f9ca800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9c9ffff640 1 -- 172.21.15.19:0/547972566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8004030 con 0x7f9cb80b9330 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9c9ffff640 1 -- 172.21.15.19:0/547972566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca80041d0 con 0x7f9cb80b9330 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cbe142640 1 -- 172.21.15.19:0/547972566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb80cb1a0 con 0x7f9cb80b9330 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9c9ffff640 1 -- 172.21.15.19:0/547972566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca80096a0 con 0x7f9cb80b9330 2024-04-25T10:01:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.133+0000 7f9cbe142640 1 -- 172.21.15.19:0/547972566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb81e92f0 con 0x7f9cb80b9330 2024-04-25T10:01:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.134+0000 7f9cbe142640 1 -- 172.21.15.19:0/547972566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb8073a00 con 0x7f9cb80b9330 2024-04-25T10:01:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.135+0000 7f9c9ffff640 1 -- 172.21.15.19:0/547972566 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ca8009840 con 0x7f9cb80b9330 2024-04-25T10:01:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.135+0000 7f9c9ffff640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ca406b4d0 0x7f9ca406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.136+0000 7f9c9ffff640 1 -- 172.21.15.19:0/547972566 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9ca808a2c0 con 0x7f9cb80b9330 2024-04-25T10:01:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.136+0000 7f9cb6ffd640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ca406b4d0 0x7f9ca406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.136+0000 7f9cb6ffd640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ca406b4d0 0x7f9ca406d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9cac034000 tx=0x7f9cac02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.140+0000 7f9c9ffff640 1 -- 172.21.15.19:0/547972566 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ca80592b0 con 0x7f9cb80b9330 2024-04-25T10:01:24.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:24.332+0000 7f9cbe142640 1 -- 172.21.15.19:0/547972566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9cb80d16b0 con 0x7f9cb80b9330 2024-04-25T10:01:25.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.607+0000 7f9c9ffff640 1 -- 172.21.15.19:0/547972566 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v125) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9ca8058bb0 con 0x7f9cb80b9330 2024-04-25T10:01:25.612 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T10:01:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.610+0000 7f9cbe142640 1 -- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ca406b4d0 msgr2=0x7f9ca406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.610+0000 7f9cbe142640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ca406b4d0 0x7f9ca406d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9cac034000 tx=0x7f9cac02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.610+0000 7f9cbe142640 1 -- 172.21.15.19:0/547972566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cb80b9330 msgr2=0x7f9cb81e9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.610+0000 7f9cbe142640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cb80b9330 0x7f9cb81e9af0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800d7d0 tx=0x7f9ca800dc90 comp rx=0 tx=0).stop 2024-04-25T10:01:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.610+0000 7f9cbe142640 1 -- 172.21.15.19:0/547972566 shutdown_connections 2024-04-25T10:01:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.610+0000 7f9cbe142640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ca406b4d0 0x7f9ca406d990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.610+0000 7f9cbe142640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cb8161710 0x7f9cb81475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.610+0000 7f9cbe142640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cb815dc20 0x7f9cb80c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.610+0000 7f9cbe142640 1 --2- 172.21.15.19:0/547972566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cb80b9330 0x7f9cb81e9af0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.610+0000 7f9cbe142640 1 -- 172.21.15.19:0/547972566 >> 172.21.15.19:0/547972566 conn(0x7f9cb80d0420 msgr2=0x7f9cb815ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:25.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.611+0000 7f9cbe142640 1 -- 172.21.15.19:0/547972566 shutdown_connections 2024-04-25T10:01:25.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.611+0000 7f9cbe142640 1 -- 172.21.15.19:0/547972566 wait complete. 2024-04-25T10:01:25.631 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:01:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.735+0000 7fc8ac953640 1 Processor -- start 2024-04-25T10:01:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.735+0000 7fc8ac953640 1 -- start start 2024-04-25T10:01:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.735+0000 7fc8ac953640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc8a815ba00 0x7fc8a809a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8ac953640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a80cd710 con 0x7fc8a81602e0 2024-04-25T10:01:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8ac953640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a8099b70 con 0x7fc8a815f5f0 2024-04-25T10:01:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8ac953640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a80cd4b0 con 0x7fc8a815ba00 2024-04-25T10:01:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc89ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc8a81602e0 0x7fc8a80ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50184/0 (socket says 172.21.15.19:50184) 2024-04-25T10:01:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc89ffff640 1 -- 172.21.15.19:0/1683385486 learned_addr learned my addr 172.21.15.19:0/1683385486 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638943414 0 0) 0x7fc890001110 con 0x7fc8a81602e0 2024-04-25T10:01:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a659c640 1 --2- 172.21.15.19:0/1683385486 >> v2:172.21.15.112:3300/0 conn(0x7fc8a815ba00 0x7fc8a809a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc894003520 con 0x7fc8a81602e0 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2271128548 0 0) 0x7fc894003520 con 0x7fc8a81602e0 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 >> v2:172.21.15.112:3300/0 conn(0x7fc8a815ba00 msgr2=0x7fc8a809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 --2- 172.21.15.19:0/1683385486 >> v2:172.21.15.112:3300/0 conn(0x7fc8a815ba00 0x7fc8a809a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 >> v1:172.21.15.112:6789/0 conn(0x7fc8a815f5f0 legacy=0x7fc8a80c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a8099e50 con 0x7fc8a81602e0 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fc890001840 con 0x7fc8a81602e0 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 >> v1:172.21.15.19:6789/0 conn(0x7fc8a81602e0 legacy=0x7fc8a80ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 --2- 172.21.15.19:0/1683385486 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8940039b0 0x7fc894003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 --2- 172.21.15.19:0/1683385486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc894004800 0x7fc894004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 --2- 172.21.15.19:0/1683385486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8940052e0 0x7fc89400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80cd4b0 con 0x7fc894004800 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89400c1b0 con 0x7fc8940052e0 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.736+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89400c3b0 con 0x7fc8940039b0 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.737+0000 7fc89ffff640 1 --2- 172.21.15.19:0/1683385486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc894004800 0x7fc894004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.737+0000 7fc8a659c640 1 --2- 172.21.15.19:0/1683385486 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8940039b0 0x7fc894003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.737+0000 7fc8a6d9d640 1 --2- 172.21.15.19:0/1683385486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8940052e0 0x7fc89400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.737+0000 7fc89ffff640 1 -- 172.21.15.19:0/1683385486 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8940039b0 msgr2=0x7fc894003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.737+0000 7fc89ffff640 1 --2- 172.21.15.19:0/1683385486 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8940039b0 0x7fc894003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.737+0000 7fc89ffff640 1 -- 172.21.15.19:0/1683385486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8940052e0 msgr2=0x7fc89400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.737+0000 7fc89ffff640 1 --2- 172.21.15.19:0/1683385486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8940052e0 0x7fc89400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.737+0000 7fc89ffff640 1 -- 172.21.15.19:0/1683385486 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc894003520 con 0x7fc894004800 2024-04-25T10:01:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.737+0000 7fc8a6d9d640 1 -- 172.21.15.19:0/1683385486 reap_dead start 2024-04-25T10:01:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.738+0000 7fc89ffff640 1 --2- 172.21.15.19:0/1683385486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc894004800 0x7fc894004c80 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fc89000c490 tx=0x7fc894005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7280ba54f8911944 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.738+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89000c0e0 con 0x7fc894004800 2024-04-25T10:01:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.738+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc890034070 con 0x7fc894004800 2024-04-25T10:01:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.738+0000 7fc8a5d9b640 1 -- 172.21.15.19:0/1683385486 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890005d30 con 0x7fc894004800 2024-04-25T10:01:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.738+0000 7fc8ac953640 1 -- 172.21.15.19:0/1683385486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc894004800 msgr2=0x7fc894004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.738+0000 7fc8ac953640 1 --2- 172.21.15.19:0/1683385486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc894004800 0x7fc894004c80 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fc89000c490 tx=0x7fc894005830 comp rx=0 tx=0).stop 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.738+0000 7fc8ac953640 1 -- 172.21.15.19:0/1683385486 shutdown_connections 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.738+0000 7fc8ac953640 1 --2- 172.21.15.19:0/1683385486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc894004800 0x7fc894004c80 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.738+0000 7fc8ac953640 1 -- 172.21.15.19:0/1683385486 >> 172.21.15.19:0/1683385486 conn(0x7fc8a80b3270 msgr2=0x7fc8a80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.738+0000 7fc8ac953640 1 -- 172.21.15.19:0/1683385486 shutdown_connections 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.738+0000 7fc8ac953640 1 -- 172.21.15.19:0/1683385486 wait complete. 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.740+0000 7fc8ac953640 1 Processor -- start 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.740+0000 7fc8ac953640 1 -- start start 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.740+0000 7fc8ac953640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc894004800 0x7fc8a80b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.740+0000 7fc8ac953640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a8158ca0 0x7fc8a814da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.740+0000 7fc8ac953640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a80cae50 0x7fc8a80cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.740+0000 7fc8ac953640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a815bde0 con 0x7fc8a80cae50 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.740+0000 7fc8ac953640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a815fa40 con 0x7fc8a8158ca0 2024-04-25T10:01:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.740+0000 7fc8ac953640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80c9db0 con 0x7fc894004800 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8a6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a80cae50 0x7fc8a80cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8a6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a80cae50 0x7fc8a80cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59442/0 (socket says 172.21.15.19:59442) 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8a6d9d640 1 -- 172.21.15.19:0/3606394794 learned_addr learned my addr 172.21.15.19:0/3606394794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8a659c640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc894004800 0x7fc8a80b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc89ffff640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a8158ca0 0x7fc8a814da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8a6d9d640 1 -- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc894004800 msgr2=0x7fc8a80b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8a6d9d640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc894004800 0x7fc8a80b57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8a6d9d640 1 -- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a8158ca0 msgr2=0x7fc8a814da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8a6d9d640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a8158ca0 0x7fc8a814da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8a6d9d640 1 -- 172.21.15.19:0/3606394794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80ca090 con 0x7fc8a80cae50 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8a6d9d640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a80cae50 0x7fc8a80cb300 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80c82d0 tx=0x7fc8a000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc89effd640 1 -- 172.21.15.19:0/3606394794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a001a070 con 0x7fc8a80cae50 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc89effd640 1 -- 172.21.15.19:0/3606394794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a0006d60 con 0x7fc8a80cae50 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8ac953640 1 -- 172.21.15.19:0/3606394794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a80ca370 con 0x7fc8a80cae50 2024-04-25T10:01:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc89effd640 1 -- 172.21.15.19:0/3606394794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0003bb0 con 0x7fc8a80cae50 2024-04-25T10:01:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.741+0000 7fc8ac953640 1 -- 172.21.15.19:0/3606394794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a80ca930 con 0x7fc8a80cae50 2024-04-25T10:01:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.742+0000 7fc8ac953640 1 -- 172.21.15.19:0/3606394794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a80cab30 con 0x7fc8a80cae50 2024-04-25T10:01:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.743+0000 7fc89effd640 1 -- 172.21.15.19:0/3606394794 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8a0005070 con 0x7fc8a80cae50 2024-04-25T10:01:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.743+0000 7fc89effd640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc88806b500 0x7fc88806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.743+0000 7fc89effd640 1 -- 172.21.15.19:0/3606394794 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc8a00894a0 con 0x7fc8a80cae50 2024-04-25T10:01:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.744+0000 7fc8a659c640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc88806b500 0x7fc88806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.744+0000 7fc8a659c640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc88806b500 0x7fc88806d9c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc898009a60 tx=0x7fc898008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:25.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.748+0000 7fc89effd640 1 -- 172.21.15.19:0/3606394794 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8a80cab30 con 0x7fc8a80cae50 2024-04-25T10:01:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.995+0000 7fc8ac953640 1 -- 172.21.15.19:0/3606394794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc8a8073990 con 0x7fc8a80cae50 2024-04-25T10:01:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.995+0000 7fc89effd640 1 -- 172.21.15.19:0/3606394794 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc8a0058440 con 0x7fc8a80cae50 2024-04-25T10:01:26.000 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:26.000 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:01:26.000 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T10:01:26.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.000 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T10:01:26.000 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T10:01:26.000 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.000 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:01:26.000 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:01:26.000 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T10:01:26.002 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:26.004 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.005 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:01:26.006 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T10:01:26.007 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:01:26.008 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T10:01:26.009 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:01:26.010 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:01:26.012 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:01:26.013 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:01:26.013 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:01:26.013 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.013 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.013 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T10:01:26.013 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:01:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 -- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc88806b500 msgr2=0x7fc88806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc88806b500 0x7fc88806d9c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc898009a60 tx=0x7fc898008040 comp rx=0 tx=0).stop 2024-04-25T10:01:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 -- 172.21.15.19:0/3606394794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a80cae50 msgr2=0x7fc8a80cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a80cae50 0x7fc8a80cb300 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80c82d0 tx=0x7fc8a000fbe0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 -- 172.21.15.19:0/3606394794 shutdown_connections 2024-04-25T10:01:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc88806b500 0x7fc88806d9c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8a80cae50 0x7fc8a80cb300 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a8158ca0 0x7fc8a814da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 --2- 172.21.15.19:0/3606394794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc894004800 0x7fc8a80b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 -- 172.21.15.19:0/3606394794 >> 172.21.15.19:0/3606394794 conn(0x7fc8a80b3270 msgr2=0x7fc8a80c58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 -- 172.21.15.19:0/3606394794 shutdown_connections 2024-04-25T10:01:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:25.997+0000 7fc8ac953640 1 -- 172.21.15.19:0/3606394794 wait complete. 2024-04-25T10:01:26.016 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.112+0000 7f6fa56c4640 1 Processor -- start 2024-04-25T10:01:26.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.112+0000 7f6fa56c4640 1 -- start start 2024-04-25T10:01:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6fa56c4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6fa015ba00 0x7f6fa009a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6fa56c4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6fa01602e0 0x7f6fa00ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6fa56c4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa00cd710 con 0x7f6fa015f5f0 2024-04-25T10:01:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6fa56c4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa0099b70 con 0x7f6fa015ba00 2024-04-25T10:01:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6fa56c4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa0099e50 con 0x7f6fa01602e0 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6f9effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6fa015ba00 0x7f6fa009a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6f9effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6fa015ba00 0x7f6fa009a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59452/0 (socket says 172.21.15.19:59452) 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6f9effd640 1 -- 172.21.15.19:0/4235557298 learned_addr learned my addr 172.21.15.19:0/4235557298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6f97fff640 1 --2- 172.21.15.19:0/4235557298 >> v2:172.21.15.112:3300/0 conn(0x7f6fa01602e0 0x7f6fa00ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6f9effd640 1 -- 172.21.15.19:0/4235557298 >> v2:172.21.15.112:3300/0 conn(0x7f6fa01602e0 msgr2=0x7f6fa00ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6f9effd640 1 --2- 172.21.15.19:0/4235557298 >> v2:172.21.15.112:3300/0 conn(0x7f6fa01602e0 0x7f6fa00ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6f9effd640 1 -- 172.21.15.19:0/4235557298 >> v1:172.21.15.112:6789/0 conn(0x7f6fa015f5f0 legacy=0x7f6fa00c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.113+0000 7f6f9effd640 1 -- 172.21.15.19:0/4235557298 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa009a130 con 0x7f6fa015ba00 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.114+0000 7f6f9effd640 1 --2- 172.21.15.19:0/4235557298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa015ba00 0x7f6fa009a930 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00b7830 tx=0x7f6f9002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15028d6f611d47ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.114+0000 7f6f9e7fc640 1 -- 172.21.15.19:0/4235557298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90037070 con 0x7f6fa015ba00 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.114+0000 7f6f9e7fc640 1 -- 172.21.15.19:0/4235557298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f90032440 con 0x7f6fa015ba00 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.114+0000 7f6f9e7fc640 1 -- 172.21.15.19:0/4235557298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90032810 con 0x7f6fa015ba00 2024-04-25T10:01:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.115+0000 7f6fa56c4640 1 -- 172.21.15.19:0/4235557298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa015ba00 msgr2=0x7f6fa009a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.115+0000 7f6fa56c4640 1 --2- 172.21.15.19:0/4235557298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa015ba00 0x7f6fa009a930 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00b7830 tx=0x7f6f9002a150 comp rx=0 tx=0).stop 2024-04-25T10:01:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.115+0000 7f6fa56c4640 1 -- 172.21.15.19:0/4235557298 shutdown_connections 2024-04-25T10:01:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.115+0000 7f6fa56c4640 1 --2- 172.21.15.19:0/4235557298 >> v2:172.21.15.112:3300/0 conn(0x7f6fa01602e0 0x7f6fa00ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.115+0000 7f6fa56c4640 1 --2- 172.21.15.19:0/4235557298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa015ba00 0x7f6fa009a930 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.115+0000 7f6fa56c4640 1 -- 172.21.15.19:0/4235557298 >> 172.21.15.19:0/4235557298 conn(0x7f6fa00b3270 msgr2=0x7f6fa00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.115+0000 7f6fa56c4640 1 -- 172.21.15.19:0/4235557298 shutdown_connections 2024-04-25T10:01:26.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.115+0000 7f6fa56c4640 1 -- 172.21.15.19:0/4235557298 wait complete. 2024-04-25T10:01:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.116+0000 7f6fa56c4640 1 Processor -- start 2024-04-25T10:01:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.116+0000 7f6fa56c4640 1 -- start start 2024-04-25T10:01:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6fa56c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa015ba00 0x7f6fa006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6fa56c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa015f5f0 0x7f6fa01e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6fa56c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fa01602e0 0x7f6fa01e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6fa56c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa015bde0 con 0x7f6fa015f5f0 2024-04-25T10:01:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6fa56c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa00cd2e0 con 0x7f6fa015ba00 2024-04-25T10:01:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6fa56c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa01e7fa0 con 0x7f6fa01602e0 2024-04-25T10:01:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa015f5f0 0x7f6fa01e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa015f5f0 0x7f6fa01e53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59458/0 (socket says 172.21.15.19:59458) 2024-04-25T10:01:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f97fff640 1 -- 172.21.15.19:0/1211177347 learned_addr learned my addr 172.21.15.19:0/1211177347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f9effd640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa015ba00 0x7f6fa006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f9f7fe640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fa01602e0 0x7f6fa01e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f97fff640 1 -- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fa01602e0 msgr2=0x7f6fa01e70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f97fff640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fa01602e0 0x7f6fa01e70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f97fff640 1 -- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa015ba00 msgr2=0x7f6fa006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f97fff640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa015ba00 0x7f6fa006d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f97fff640 1 -- 172.21.15.19:0/1211177347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa00c89f0 con 0x7f6fa015f5f0 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f9effd640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa015ba00 0x7f6fa006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f9f7fe640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fa01602e0 0x7f6fa01e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.117+0000 7f6f97fff640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa015f5f0 0x7f6fa01e53d0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800d800 tx=0x7f6f8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.118+0000 7f6f977fe640 1 -- 172.21.15.19:0/1211177347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800b6c0 con 0x7f6fa015f5f0 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.118+0000 7f6f977fe640 1 -- 172.21.15.19:0/1211177347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f88019070 con 0x7f6fa015f5f0 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.118+0000 7f6fa56c4640 1 -- 172.21.15.19:0/1211177347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa00c8cd0 con 0x7f6fa015f5f0 2024-04-25T10:01:26.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.118+0000 7f6f977fe640 1 -- 172.21.15.19:0/1211177347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800ba80 con 0x7f6fa015f5f0 2024-04-25T10:01:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.118+0000 7f6fa56c4640 1 -- 172.21.15.19:0/1211177347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa00c9290 con 0x7f6fa015f5f0 2024-04-25T10:01:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.119+0000 7f6fa56c4640 1 -- 172.21.15.19:0/1211177347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa0073990 con 0x7f6fa015f5f0 2024-04-25T10:01:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.119+0000 7f6f977fe640 1 -- 172.21.15.19:0/1211177347 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f8800bc20 con 0x7f6fa015f5f0 2024-04-25T10:01:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.119+0000 7f6f977fe640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f8006b4d0 0x7f6f8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.119+0000 7f6f977fe640 1 -- 172.21.15.19:0/1211177347 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6f8808a520 con 0x7f6fa015f5f0 2024-04-25T10:01:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.120+0000 7f6f9effd640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f8006b4d0 0x7f6f8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.120+0000 7f6f9effd640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f8006b4d0 0x7f6f8006d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6f90034000 tx=0x7f6f9002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.124+0000 7f6f977fe640 1 -- 172.21.15.19:0/1211177347 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f8801c360 con 0x7f6fa015f5f0 2024-04-25T10:01:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.308+0000 7f6fa56c4640 1 -- 172.21.15.19:0/1211177347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fa009a9c0 con 0x7f6fa015f5f0 2024-04-25T10:01:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.308+0000 7f6f977fe640 1 -- 172.21.15.19:0/1211177347 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+13162 (secure 0 0 0) 0x7f6f88059560 con 0x7f6fa015f5f0 2024-04-25T10:01:26.313 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:26.313 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":125,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:01:25.606587+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 -- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f8006b4d0 msgr2=0x7f6f8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f8006b4d0 0x7f6f8006d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6f90034000 tx=0x7f6f9002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 -- 172.21.15.19:0/1211177347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa015f5f0 msgr2=0x7f6fa01e53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa015f5f0 0x7f6fa01e53d0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800d800 tx=0x7f6f8800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 -- 172.21.15.19:0/1211177347 shutdown_connections 2024-04-25T10:01:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6f8006b4d0 0x7f6f8006d990 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fa01602e0 0x7f6fa01e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa015f5f0 0x7f6fa01e53d0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 --2- 172.21.15.19:0/1211177347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa015ba00 0x7f6fa006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 -- 172.21.15.19:0/1211177347 >> 172.21.15.19:0/1211177347 conn(0x7f6fa00b3270 msgr2=0x7f6fa00d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 -- 172.21.15.19:0/1211177347 shutdown_connections 2024-04-25T10:01:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.310+0000 7f6fa56c4640 1 -- 172.21.15.19:0/1211177347 wait complete. 2024-04-25T10:01:26.326 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:26.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:26.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.425+0000 7f1b06a1f640 1 Processor -- start 2024-04-25T10:01:26.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.425+0000 7f1b06a1f640 1 -- start start 2024-04-25T10:01:26.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1b06a1f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b0015b7d0 0x7f1b000cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1b06a1f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1b000b5620 0x7f1b000b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1b06a1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b0006c2d0 0x7f1b0006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1b06a1f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0006cc80 con 0x7f1b0015b7d0 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1b06a1f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0006cf50 con 0x7f1b000b5620 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1b06a1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b000ce530 con 0x7f1b0006c2d0 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1afffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b0015b7d0 0x7f1b000cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1afffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b0015b7d0 0x7f1b000cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59468/0 (socket says 172.21.15.19:59468) 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1afffff640 1 -- 172.21.15.19:0/840304898 learned_addr learned my addr 172.21.15.19:0/840304898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1b04fbc640 1 --2- 172.21.15.19:0/840304898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b0006c2d0 0x7f1b0006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1aff7fe640 1 --2- 172.21.15.19:0/840304898 >> v2:172.21.15.112:3300/0 conn(0x7f1b000b5620 0x7f1b000b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1afffff640 1 -- 172.21.15.19:0/840304898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b0006c2d0 msgr2=0x7f1b0006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1afffff640 1 --2- 172.21.15.19:0/840304898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b0006c2d0 0x7f1b0006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1afffff640 1 -- 172.21.15.19:0/840304898 >> v2:172.21.15.112:3300/0 conn(0x7f1b000b5620 msgr2=0x7f1b000b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1afffff640 1 --2- 172.21.15.19:0/840304898 >> v2:172.21.15.112:3300/0 conn(0x7f1b000b5620 0x7f1b000b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.426+0000 7f1afffff640 1 -- 172.21.15.19:0/840304898 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b000ce810 con 0x7f1b0015b7d0 2024-04-25T10:01:26.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.427+0000 7f1afffff640 1 --2- 172.21.15.19:0/840304898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0015b7d0 0x7f1b000cf010 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f1b000ce180 tx=0x7f1af002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301c34aad71b6f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.427+0000 7f1afeffd640 1 -- 172.21.15.19:0/840304898 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0037070 con 0x7f1b0015b7d0 2024-04-25T10:01:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.427+0000 7f1afeffd640 1 -- 172.21.15.19:0/840304898 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af0032440 con 0x7f1b0015b7d0 2024-04-25T10:01:26.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.427+0000 7f1afeffd640 1 -- 172.21.15.19:0/840304898 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0032810 con 0x7f1b0015b7d0 2024-04-25T10:01:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.428+0000 7f1b06a1f640 1 -- 172.21.15.19:0/840304898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0015b7d0 msgr2=0x7f1b000cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.428+0000 7f1b06a1f640 1 --2- 172.21.15.19:0/840304898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0015b7d0 0x7f1b000cf010 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f1b000ce180 tx=0x7f1af002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.428+0000 7f1b06a1f640 1 -- 172.21.15.19:0/840304898 shutdown_connections 2024-04-25T10:01:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.428+0000 7f1b06a1f640 1 --2- 172.21.15.19:0/840304898 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b0006c2d0 0x7f1b0006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.428+0000 7f1b06a1f640 1 --2- 172.21.15.19:0/840304898 >> v2:172.21.15.112:3300/0 conn(0x7f1b000b5620 0x7f1b000b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.428+0000 7f1b06a1f640 1 --2- 172.21.15.19:0/840304898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b0015b7d0 0x7f1b000cf010 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.428+0000 7f1b06a1f640 1 -- 172.21.15.19:0/840304898 >> 172.21.15.19:0/840304898 conn(0x7f1b000d0800 msgr2=0x7f1b000d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.428+0000 7f1b06a1f640 1 -- 172.21.15.19:0/840304898 shutdown_connections 2024-04-25T10:01:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.428+0000 7f1b06a1f640 1 -- 172.21.15.19:0/840304898 wait complete. 2024-04-25T10:01:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.429+0000 7f1b06a1f640 1 Processor -- start 2024-04-25T10:01:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.429+0000 7f1b06a1f640 1 -- start start 2024-04-25T10:01:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1b06a1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b0006c2d0 0x7f1b001f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1b06a1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b000b5620 0x7f1b000c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1b06a1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b0015b7d0 0x7f1b00159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1b06a1f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b000cf550 con 0x7f1b000b5620 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1b06a1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0006d230 con 0x7f1b0015b7d0 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1b06a1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b000cf6d0 con 0x7f1b0006c2d0 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1aff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b000b5620 0x7f1b000c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1aff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b000b5620 0x7f1b000c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59482/0 (socket says 172.21.15.19:59482) 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1aff7fe640 1 -- 172.21.15.19:0/3616671461 learned_addr learned my addr 172.21.15.19:0/3616671461 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1afffff640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b0006c2d0 0x7f1b001f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1b04fbc640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b0015b7d0 0x7f1b00159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.430+0000 7f1aff7fe640 1 -- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b0006c2d0 msgr2=0x7f1b001f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.431+0000 7f1aff7fe640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b0006c2d0 0x7f1b001f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.431+0000 7f1aff7fe640 1 -- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b0015b7d0 msgr2=0x7f1b00159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.431+0000 7f1aff7fe640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b0015b7d0 0x7f1b00159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.431+0000 7f1aff7fe640 1 -- 172.21.15.19:0/3616671461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b000c99d0 con 0x7f1b000b5620 2024-04-25T10:01:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.431+0000 7f1afffff640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b0006c2d0 0x7f1b001f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.431+0000 7f1aff7fe640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b000b5620 0x7f1b000c9060 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800e9b0 tx=0x7f1ae800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.431+0000 7f1afcff9640 1 -- 172.21.15.19:0/3616671461 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8017070 con 0x7f1b000b5620 2024-04-25T10:01:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.431+0000 7f1afcff9640 1 -- 172.21.15.19:0/3616671461 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae800c9a0 con 0x7f1b000b5620 2024-04-25T10:01:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.431+0000 7f1afcff9640 1 -- 172.21.15.19:0/3616671461 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800cd90 con 0x7f1b000b5620 2024-04-25T10:01:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.431+0000 7f1b06a1f640 1 -- 172.21.15.19:0/3616671461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b001f6960 con 0x7f1b000b5620 2024-04-25T10:01:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.431+0000 7f1b06a1f640 1 -- 172.21.15.19:0/3616671461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b001f2f80 con 0x7f1b000b5620 2024-04-25T10:01:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.432+0000 7f1b06a1f640 1 -- 172.21.15.19:0/3616671461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b00073a00 con 0x7f1b000b5620 2024-04-25T10:01:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.432+0000 7f1afcff9640 1 -- 172.21.15.19:0/3616671461 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ae801a070 con 0x7f1b000b5620 2024-04-25T10:01:26.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.433+0000 7f1afcff9640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ae006b500 0x7f1ae006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.433+0000 7f1afcff9640 1 -- 172.21.15.19:0/3616671461 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1ae808a340 con 0x7f1b000b5620 2024-04-25T10:01:26.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.433+0000 7f1afffff640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ae006b500 0x7f1ae006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.434+0000 7f1afffff640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ae006b500 0x7f1ae006d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1af0034000 tx=0x7f1af002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.437+0000 7f1afcff9640 1 -- 172.21.15.19:0/3616671461 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ae8059410 con 0x7f1b000b5620 2024-04-25T10:01:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.616+0000 7f1b06a1f640 1 -- 172.21.15.19:0/3616671461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b0006b210 con 0x7f1b000b5620 2024-04-25T10:01:26.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.616+0000 7f1afcff9640 1 -- 172.21.15.19:0/3616671461 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ae8058d10 con 0x7f1b000b5620 2024-04-25T10:01:26.621 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:01:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.618+0000 7f1b06a1f640 1 -- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ae006b500 msgr2=0x7f1ae006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.618+0000 7f1b06a1f640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ae006b500 0x7f1ae006d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1af0034000 tx=0x7f1af002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.618+0000 7f1b06a1f640 1 -- 172.21.15.19:0/3616671461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b000b5620 msgr2=0x7f1b000c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.618+0000 7f1b06a1f640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b000b5620 0x7f1b000c9060 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800e9b0 tx=0x7f1ae800ee70 comp rx=0 tx=0).stop 2024-04-25T10:01:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.619+0000 7f1b06a1f640 1 -- 172.21.15.19:0/3616671461 shutdown_connections 2024-04-25T10:01:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.619+0000 7f1b06a1f640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ae006b500 0x7f1ae006d9c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.619+0000 7f1b06a1f640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b0015b7d0 0x7f1b00159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.619+0000 7f1b06a1f640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b000b5620 0x7f1b000c9060 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.619+0000 7f1b06a1f640 1 --2- 172.21.15.19:0/3616671461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b0006c2d0 0x7f1b001f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.619+0000 7f1b06a1f640 1 -- 172.21.15.19:0/3616671461 >> 172.21.15.19:0/3616671461 conn(0x7f1b000d0800 msgr2=0x7f1b000b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.619+0000 7f1b06a1f640 1 -- 172.21.15.19:0/3616671461 shutdown_connections 2024-04-25T10:01:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.619+0000 7f1b06a1f640 1 -- 172.21.15.19:0/3616671461 wait complete. 2024-04-25T10:01:26.634 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:01:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.730+0000 7f58f1207640 1 Processor -- start 2024-04-25T10:01:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.731+0000 7f58f1207640 1 -- start start 2024-04-25T10:01:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.731+0000 7f58f1207640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58ec15dc00 0x7f58ec0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.731+0000 7f58f1207640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f58ec0ceb20 0x7f58ec0d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.731+0000 7f58f1207640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58ec0d1c00 0x7f58ec0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.731+0000 7f58f1207640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ec0d2060 con 0x7f58ec0d1c00 2024-04-25T10:01:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.731+0000 7f58f1207640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ec15dfe0 con 0x7f58ec0ceb20 2024-04-25T10:01:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.731+0000 7f58f1207640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0d00c0 con 0x7f58ec15dc00 2024-04-25T10:01:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.731+0000 7f58eb59e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58ec0d1c00 0x7f58ec0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.732+0000 7f58eb59e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58ec0d1c00 0x7f58ec0ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59494/0 (socket says 172.21.15.19:59494) 2024-04-25T10:01:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.732+0000 7f58eb59e640 1 -- 172.21.15.19:0/1118733172 learned_addr learned my addr 172.21.15.19:0/1118733172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.732+0000 7f58e3fff640 1 --2- 172.21.15.19:0/1118733172 >> v2:172.21.15.112:3300/0 conn(0x7f58ec0ceb20 0x7f58ec0d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.732+0000 7f58ead9d640 1 --2- 172.21.15.19:0/1118733172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58ec15dc00 0x7f58ec0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.732+0000 7f58eb59e640 1 -- 172.21.15.19:0/1118733172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58ec15dc00 msgr2=0x7f58ec0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.732+0000 7f58eb59e640 1 --2- 172.21.15.19:0/1118733172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58ec15dc00 0x7f58ec0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.732+0000 7f58eb59e640 1 -- 172.21.15.19:0/1118733172 >> v2:172.21.15.112:3300/0 conn(0x7f58ec0ceb20 msgr2=0x7f58ec0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.732+0000 7f58eb59e640 1 --2- 172.21.15.19:0/1118733172 >> v2:172.21.15.112:3300/0 conn(0x7f58ec0ceb20 0x7f58ec0d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.732+0000 7f58eb59e640 1 -- 172.21.15.19:0/1118733172 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0d03a0 con 0x7f58ec0d1c00 2024-04-25T10:01:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.732+0000 7f58e3fff640 1 --2- 172.21.15.19:0/1118733172 >> v2:172.21.15.112:3300/0 conn(0x7f58ec0ceb20 0x7f58ec0d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.732+0000 7f58eb59e640 1 --2- 172.21.15.19:0/1118733172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58ec0d1c00 0x7f58ec0ccc40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f58ec06e170 tx=0x7f58e402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43853f96b40fb0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58ea59c640 1 -- 172.21.15.19:0/1118733172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4036070 con 0x7f58ec0d1c00 2024-04-25T10:01:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58ea59c640 1 -- 172.21.15.19:0/1118733172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e4031600 con 0x7f58ec0d1c00 2024-04-25T10:01:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58ea59c640 1 -- 172.21.15.19:0/1118733172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e40319d0 con 0x7f58ec0d1c00 2024-04-25T10:01:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58f1207640 1 -- 172.21.15.19:0/1118733172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58ec0d1c00 msgr2=0x7f58ec0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58f1207640 1 --2- 172.21.15.19:0/1118733172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58ec0d1c00 0x7f58ec0ccc40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f58ec06e170 tx=0x7f58e402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58f1207640 1 -- 172.21.15.19:0/1118733172 shutdown_connections 2024-04-25T10:01:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58f1207640 1 --2- 172.21.15.19:0/1118733172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58ec0d1c00 0x7f58ec0ccc40 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58f1207640 1 --2- 172.21.15.19:0/1118733172 >> v2:172.21.15.112:3300/0 conn(0x7f58ec0ceb20 0x7f58ec0d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58f1207640 1 --2- 172.21.15.19:0/1118733172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58ec15dc00 0x7f58ec0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58f1207640 1 -- 172.21.15.19:0/1118733172 >> 172.21.15.19:0/1118733172 conn(0x7f58ec06f5d0 msgr2=0x7f58ec06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58f1207640 1 -- 172.21.15.19:0/1118733172 shutdown_connections 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.733+0000 7f58f1207640 1 -- 172.21.15.19:0/1118733172 wait complete. 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.735+0000 7f58f1207640 1 Processor -- start 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.735+0000 7f58f1207640 1 -- start start 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.735+0000 7f58f1207640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58ec0ceb20 0x7f58ec157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.735+0000 7f58f1207640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58ec0d1c00 0x7f58ec06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.735+0000 7f58f1207640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58ec15dc00 0x7f58ec15c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.735+0000 7f58f1207640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0cc130 con 0x7f58ec0ceb20 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.735+0000 7f58f1207640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0cc720 con 0x7f58ec15dc00 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.735+0000 7f58f1207640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0cfd90 con 0x7f58ec0d1c00 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.735+0000 7f58ead9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58ec0ceb20 0x7f58ec157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.735+0000 7f58ead9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58ec0ceb20 0x7f58ec157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59496/0 (socket says 172.21.15.19:59496) 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.735+0000 7f58ead9d640 1 -- 172.21.15.19:0/1267217795 learned_addr learned my addr 172.21.15.19:0/1267217795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58e3fff640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58ec0d1c00 0x7f58ec06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58eb59e640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58ec15dc00 0x7f58ec15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58ead9d640 1 -- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58ec0d1c00 msgr2=0x7f58ec06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58ead9d640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58ec0d1c00 0x7f58ec06bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58ead9d640 1 -- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58ec15dc00 msgr2=0x7f58ec15c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58ead9d640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58ec15dc00 0x7f58ec15c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58ead9d640 1 -- 172.21.15.19:0/1267217795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec1f3b70 con 0x7f58ec0ceb20 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58ead9d640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58ec0ceb20 0x7f58ec157330 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00d800 tx=0x7f58dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58e37fe640 1 -- 172.21.15.19:0/1267217795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00b6c0 con 0x7f58ec0ceb20 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58e37fe640 1 -- 172.21.15.19:0/1267217795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc019070 con 0x7f58ec0ceb20 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58e37fe640 1 -- 172.21.15.19:0/1267217795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00ba80 con 0x7f58ec0ceb20 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58f1207640 1 -- 172.21.15.19:0/1267217795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ec1f3e50 con 0x7f58ec0ceb20 2024-04-25T10:01:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.736+0000 7f58f1207640 1 -- 172.21.15.19:0/1267217795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec1f4410 con 0x7f58ec0ceb20 2024-04-25T10:01:26.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.737+0000 7f58f1207640 1 -- 172.21.15.19:0/1267217795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ec073990 con 0x7f58ec0ceb20 2024-04-25T10:01:26.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.738+0000 7f58e37fe640 1 -- 172.21.15.19:0/1267217795 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58dc00bc20 con 0x7f58ec0ceb20 2024-04-25T10:01:26.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.738+0000 7f58e37fe640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58cc06b4d0 0x7f58cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.738+0000 7f58e37fe640 1 -- 172.21.15.19:0/1267217795 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f58dc08a4f0 con 0x7f58ec0ceb20 2024-04-25T10:01:26.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.739+0000 7f58e3fff640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58cc06b4d0 0x7f58cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.739+0000 7f58e3fff640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58cc06b4d0 0x7f58cc06d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f58d4006fd0 tx=0x7f58d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.743+0000 7f58e37fe640 1 -- 172.21.15.19:0/1267217795 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58dc059530 con 0x7f58ec0ceb20 2024-04-25T10:01:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.925+0000 7f58f1207640 1 -- 172.21.15.19:0/1267217795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f58ec0c66c0 con 0x7f58ec0ceb20 2024-04-25T10:01:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.925+0000 7f58e37fe640 1 -- 172.21.15.19:0/1267217795 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v125) v1 ==== 84+0+65 (secure 0 0 0) 0x7f58dc058e30 con 0x7f58ec0ceb20 2024-04-25T10:01:26.930 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:26.930 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T10:01:26.930 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:01:26.930 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T10:01:26.930 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T10:01:26.930 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:26.931 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T10:01:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.927+0000 7f58f1207640 1 -- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58cc06b4d0 msgr2=0x7f58cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.927+0000 7f58f1207640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58cc06b4d0 0x7f58cc06d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f58d4006fd0 tx=0x7f58d4008040 comp rx=0 tx=0).stop 2024-04-25T10:01:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.927+0000 7f58f1207640 1 -- 172.21.15.19:0/1267217795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58ec0ceb20 msgr2=0x7f58ec157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.927+0000 7f58f1207640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58ec0ceb20 0x7f58ec157330 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00d800 tx=0x7f58dc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.927+0000 7f58f1207640 1 -- 172.21.15.19:0/1267217795 shutdown_connections 2024-04-25T10:01:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.927+0000 7f58f1207640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58cc06b4d0 0x7f58cc06d990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.927+0000 7f58f1207640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58ec15dc00 0x7f58ec15c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.927+0000 7f58f1207640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58ec0d1c00 0x7f58ec06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.927+0000 7f58f1207640 1 --2- 172.21.15.19:0/1267217795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58ec0ceb20 0x7f58ec157330 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.927+0000 7f58f1207640 1 -- 172.21.15.19:0/1267217795 >> 172.21.15.19:0/1267217795 conn(0x7f58ec06f5d0 msgr2=0x7f58ec0c7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.928+0000 7f58f1207640 1 -- 172.21.15.19:0/1267217795 shutdown_connections 2024-04-25T10:01:26.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:26.928+0000 7f58f1207640 1 -- 172.21.15.19:0/1267217795 wait complete. 2024-04-25T10:01:26.943 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:01:26.943 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:01:27.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.044+0000 7f31aa3bd640 1 Processor -- start 2024-04-25T10:01:27.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.044+0000 7f31aa3bd640 1 -- start start 2024-04-25T10:01:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.045+0000 7f31aa3bd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f31a415b7d0 0x7f31a40cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.045+0000 7f31aa3bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31a406c9e0 0x7f31a40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.045+0000 7f31aa3bd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a406cf50 con 0x7f31a40b5620 2024-04-25T10:01:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.045+0000 7f31aa3bd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a40ce530 con 0x7f31a415b7d0 2024-04-25T10:01:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.045+0000 7f31aa3bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a40ce810 con 0x7f31a406c9e0 2024-04-25T10:01:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.045+0000 7f31a3fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f31a415b7d0 0x7f31a40cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.045+0000 7f31a895a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31a406c9e0 0x7f31a40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.045+0000 7f31a37fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f31a40b5620 0x7f31a406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:59874/0 (socket says 172.21.15.19:59874) 2024-04-25T10:01:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.045+0000 7f31a37fe640 1 -- 172.21.15.19:0/3892951548 learned_addr learned my addr 172.21.15.19:0/3892951548 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.046+0000 7f31a3fff640 1 -- 172.21.15.19:0/3892951548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31a406c9e0 msgr2=0x7f31a40b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.046+0000 7f31a3fff640 1 --2- 172.21.15.19:0/3892951548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31a406c9e0 0x7f31a40b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.046+0000 7f31a3fff640 1 -- 172.21.15.19:0/3892951548 >> v1:172.21.15.112:6789/0 conn(0x7f31a40b5620 legacy=0x7f31a406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.046+0000 7f31a3fff640 1 -- 172.21.15.19:0/3892951548 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a40ceaf0 con 0x7f31a415b7d0 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31a3fff640 1 --2- 172.21.15.19:0/3892951548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31a415b7d0 0x7f31a40cf010 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f318c0097a0 tx=0x7f318c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7b8915d2dc30d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31a2ffd640 1 -- 172.21.15.19:0/3892951548 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c037070 con 0x7f31a415b7d0 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31a2ffd640 1 -- 172.21.15.19:0/3892951548 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c032440 con 0x7f31a415b7d0 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31a2ffd640 1 -- 172.21.15.19:0/3892951548 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c032810 con 0x7f31a415b7d0 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3892951548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31a415b7d0 msgr2=0x7f31a40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31aa3bd640 1 --2- 172.21.15.19:0/3892951548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31a415b7d0 0x7f31a40cf010 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f318c0097a0 tx=0x7f318c02a180 comp rx=0 tx=0).stop 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3892951548 shutdown_connections 2024-04-25T10:01:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31aa3bd640 1 --2- 172.21.15.19:0/3892951548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31a406c9e0 0x7f31a40b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31aa3bd640 1 --2- 172.21.15.19:0/3892951548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31a415b7d0 0x7f31a40cf010 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3892951548 >> 172.21.15.19:0/3892951548 conn(0x7f31a40d0800 msgr2=0x7f31a40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3892951548 shutdown_connections 2024-04-25T10:01:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.047+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3892951548 wait complete. 2024-04-25T10:01:27.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.049+0000 7f31aa3bd640 1 Processor -- start 2024-04-25T10:01:27.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.049+0000 7f31aa3bd640 1 -- start start 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.049+0000 7f31aa3bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a406c9e0 0x7f31a40d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.049+0000 7f31aa3bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31a40b5620 0x7f31a40c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.049+0000 7f31aa3bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31a415b7d0 0x7f31a40b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.049+0000 7f31aa3bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a40cf6d0 con 0x7f31a406c9e0 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.049+0000 7f31aa3bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a40cf550 con 0x7f31a40b5620 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.049+0000 7f31aa3bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a415bbb0 con 0x7f31a415b7d0 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a406c9e0 0x7f31a40d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a406c9e0 0x7f31a40d27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59500/0 (socket says 172.21.15.19:59500) 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a3fff640 1 -- 172.21.15.19:0/3470073997 learned_addr learned my addr 172.21.15.19:0/3470073997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a895a640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31a415b7d0 0x7f31a40b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a37fe640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31a40b5620 0x7f31a40c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a3fff640 1 -- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31a415b7d0 msgr2=0x7f31a40b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a3fff640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31a415b7d0 0x7f31a40b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a3fff640 1 -- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31a40b5620 msgr2=0x7f31a40c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a3fff640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31a40b5620 0x7f31a40c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a3fff640 1 -- 172.21.15.19:0/3470073997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a41486a0 con 0x7f31a406c9e0 2024-04-25T10:01:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a37fe640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31a40b5620 0x7f31a40c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a3fff640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a406c9e0 0x7f31a40d27e0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f318c035040 tx=0x7f318c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a0ff9640 1 -- 172.21.15.19:0/3470073997 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c037040 con 0x7f31a406c9e0 2024-04-25T10:01:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a0ff9640 1 -- 172.21.15.19:0/3470073997 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c030500 con 0x7f31a406c9e0 2024-04-25T10:01:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3470073997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a4148980 con 0x7f31a406c9e0 2024-04-25T10:01:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31a0ff9640 1 -- 172.21.15.19:0/3470073997 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c030910 con 0x7f31a406c9e0 2024-04-25T10:01:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.050+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3470073997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a4148f40 con 0x7f31a406c9e0 2024-04-25T10:01:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.051+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3470073997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a41461e0 con 0x7f31a406c9e0 2024-04-25T10:01:27.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.052+0000 7f31a0ff9640 1 -- 172.21.15.19:0/3470073997 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f318c044020 con 0x7f31a406c9e0 2024-04-25T10:01:27.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.052+0000 7f31a0ff9640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f318406b4d0 0x7f318406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.052+0000 7f31a0ff9640 1 -- 172.21.15.19:0/3470073997 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f318c0aa270 con 0x7f31a406c9e0 2024-04-25T10:01:27.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.053+0000 7f31a37fe640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f318406b4d0 0x7f318406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.053+0000 7f31a37fe640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f318406b4d0 0x7f318406d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f31a40d02f0 tx=0x7f3194009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.057+0000 7f31a0ff9640 1 -- 172.21.15.19:0/3470073997 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f318c0792b0 con 0x7f31a406c9e0 2024-04-25T10:01:27.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.238+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3470073997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f31a4073a00 con 0x7f31a406c9e0 2024-04-25T10:01:27.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.621+0000 7f31a0ff9640 1 -- 172.21.15.19:0/3470073997 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v126) v1 ==== 138+0+0 (secure 0 0 0) 0x7f318c078bb0 con 0x7f31a406c9e0 2024-04-25T10:01:27.625 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T10:01:27.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f318406b4d0 msgr2=0x7f318406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f318406b4d0 0x7f318406d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f31a40d02f0 tx=0x7f3194009040 comp rx=0 tx=0).stop 2024-04-25T10:01:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3470073997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a406c9e0 msgr2=0x7f31a40d27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a406c9e0 0x7f31a40d27e0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f318c035040 tx=0x7f318c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3470073997 shutdown_connections 2024-04-25T10:01:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f318406b4d0 0x7f318406d990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31a415b7d0 0x7f31a40b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31a40b5620 0x7f31a40c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 --2- 172.21.15.19:0/3470073997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a406c9e0 0x7f31a40d27e0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3470073997 >> 172.21.15.19:0/3470073997 conn(0x7f31a40d0800 msgr2=0x7f31a415eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3470073997 shutdown_connections 2024-04-25T10:01:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.623+0000 7f31aa3bd640 1 -- 172.21.15.19:0/3470073997 wait complete. 2024-04-25T10:01:27.643 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:01:27.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.749+0000 7f54907db640 1 Processor -- start 2024-04-25T10:01:27.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.749+0000 7f54907db640 1 -- start start 2024-04-25T10:01:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f54907db640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f548815b7d0 0x7f54880cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f54907db640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548806c9e0 0x7f54880b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f54907db640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548806cf50 con 0x7f54880b5620 2024-04-25T10:01:27.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f54907db640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54880ce530 con 0x7f548815b7d0 2024-04-25T10:01:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f54907db640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54880ce810 con 0x7f548806c9e0 2024-04-25T10:01:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f548e577640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f548815b7d0 0x7f54880cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f548dd76640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f54880b5620 0x7f548806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40428/0 (socket says 172.21.15.19:40428) 2024-04-25T10:01:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f548dd76640 1 -- 172.21.15.19:0/1510515344 learned_addr learned my addr 172.21.15.19:0/1510515344 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f548ed78640 1 --2- 172.21.15.19:0/1510515344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548806c9e0 0x7f54880b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f548e577640 1 -- 172.21.15.19:0/1510515344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548806c9e0 msgr2=0x7f54880b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f548e577640 1 --2- 172.21.15.19:0/1510515344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548806c9e0 0x7f54880b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f548e577640 1 -- 172.21.15.19:0/1510515344 >> v1:172.21.15.19:6789/0 conn(0x7f54880b5620 legacy=0x7f548806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f548e577640 1 -- 172.21.15.19:0/1510515344 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54880ceaf0 con 0x7f548815b7d0 2024-04-25T10:01:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.750+0000 7f548ed78640 1 --2- 172.21.15.19:0/1510515344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548806c9e0 0x7f54880b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.751+0000 7f548e577640 1 --2- 172.21.15.19:0/1510515344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548815b7d0 0x7f54880cf010 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f54840097a0 tx=0x7f548402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1f49d62dd4dd80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.751+0000 7f548d575640 1 -- 172.21.15.19:0/1510515344 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484037070 con 0x7f548815b7d0 2024-04-25T10:01:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.751+0000 7f548d575640 1 -- 172.21.15.19:0/1510515344 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5484032440 con 0x7f548815b7d0 2024-04-25T10:01:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.751+0000 7f548d575640 1 -- 172.21.15.19:0/1510515344 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484032810 con 0x7f548815b7d0 2024-04-25T10:01:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.752+0000 7f54907db640 1 -- 172.21.15.19:0/1510515344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548815b7d0 msgr2=0x7f54880cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.752+0000 7f54907db640 1 --2- 172.21.15.19:0/1510515344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548815b7d0 0x7f54880cf010 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f54840097a0 tx=0x7f548402a180 comp rx=0 tx=0).stop 2024-04-25T10:01:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.752+0000 7f54907db640 1 -- 172.21.15.19:0/1510515344 shutdown_connections 2024-04-25T10:01:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.752+0000 7f54907db640 1 --2- 172.21.15.19:0/1510515344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548806c9e0 0x7f54880b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.752+0000 7f54907db640 1 --2- 172.21.15.19:0/1510515344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548815b7d0 0x7f54880cf010 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.752+0000 7f54907db640 1 -- 172.21.15.19:0/1510515344 >> 172.21.15.19:0/1510515344 conn(0x7f54880d0800 msgr2=0x7f54880d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.752+0000 7f54907db640 1 -- 172.21.15.19:0/1510515344 shutdown_connections 2024-04-25T10:01:27.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.752+0000 7f54907db640 1 -- 172.21.15.19:0/1510515344 wait complete. 2024-04-25T10:01:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.753+0000 7f54907db640 1 Processor -- start 2024-04-25T10:01:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.753+0000 7f54907db640 1 -- start start 2024-04-25T10:01:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f54907db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548806c9e0 0x7f5488148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f54907db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54880b5620 0x7f54880b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f54907db640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548815b7d0 0x7f54881f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f54907db640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54880cf6d0 con 0x7f548806c9e0 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f54907db640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54880cf550 con 0x7f54880b5620 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f54907db640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548815bbb0 con 0x7f548815b7d0 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f548e577640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548806c9e0 0x7f5488148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f548e577640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548806c9e0 0x7f5488148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33422/0 (socket says 172.21.15.19:33422) 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f548e577640 1 -- 172.21.15.19:0/1920761335 learned_addr learned my addr 172.21.15.19:0/1920761335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f548ed78640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548815b7d0 0x7f54881f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f548e577640 1 -- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548815b7d0 msgr2=0x7f54881f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f548e577640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548815b7d0 0x7f54881f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f548e577640 1 -- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54880b5620 msgr2=0x7f54880b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f548e577640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54880b5620 0x7f54880b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f548e577640 1 -- 172.21.15.19:0/1920761335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54880b7540 con 0x7f548806c9e0 2024-04-25T10:01:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f548e577640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548806c9e0 0x7f5488148af0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f5484035040 tx=0x7f54840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f547effd640 1 -- 172.21.15.19:0/1920761335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484037040 con 0x7f548806c9e0 2024-04-25T10:01:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f547effd640 1 -- 172.21.15.19:0/1920761335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5484030770 con 0x7f548806c9e0 2024-04-25T10:01:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f54907db640 1 -- 172.21.15.19:0/1920761335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54880b7810 con 0x7f548806c9e0 2024-04-25T10:01:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f547effd640 1 -- 172.21.15.19:0/1920761335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484030b80 con 0x7f548806c9e0 2024-04-25T10:01:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.754+0000 7f54907db640 1 -- 172.21.15.19:0/1920761335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54880b80b0 con 0x7f548806c9e0 2024-04-25T10:01:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.756+0000 7f54907db640 1 -- 172.21.15.19:0/1920761335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54880739e0 con 0x7f548806c9e0 2024-04-25T10:01:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.756+0000 7f547effd640 1 -- 172.21.15.19:0/1920761335 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5484044020 con 0x7f548806c9e0 2024-04-25T10:01:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.756+0000 7f547effd640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f546c06b4d0 0x7f546c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.757+0000 7f547effd640 1 -- 172.21.15.19:0/1920761335 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f54840aa690 con 0x7f548806c9e0 2024-04-25T10:01:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.757+0000 7f548dd76640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f546c06b4d0 0x7f546c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.757+0000 7f548dd76640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f546c06b4d0 0x7f546c06d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5478005880 tx=0x7f5478007a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.761+0000 7f547effd640 1 -- 172.21.15.19:0/1920761335 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5484032e20 con 0x7f548806c9e0 2024-04-25T10:01:27.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:27.945+0000 7f54907db640 1 -- 172.21.15.19:0/1920761335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f548815c270 con 0x7f548806c9e0 2024-04-25T10:01:28.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.623+0000 7f547effd640 1 -- 172.21.15.19:0/1920761335 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v127) v1 ==== 170+0+0 (secure 0 0 0) 0x7f54840795f0 con 0x7f548806c9e0 2024-04-25T10:01:28.628 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T10:01:28.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 -- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f546c06b4d0 msgr2=0x7f546c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f546c06b4d0 0x7f546c06d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5478005880 tx=0x7f5478007a20 comp rx=0 tx=0).stop 2024-04-25T10:01:28.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 -- 172.21.15.19:0/1920761335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548806c9e0 msgr2=0x7f5488148af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548806c9e0 0x7f5488148af0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f5484035040 tx=0x7f54840046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 -- 172.21.15.19:0/1920761335 shutdown_connections 2024-04-25T10:01:28.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f546c06b4d0 0x7f546c06d990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548815b7d0 0x7f54881f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54880b5620 0x7f54880b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 --2- 172.21.15.19:0/1920761335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f548806c9e0 0x7f5488148af0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 -- 172.21.15.19:0/1920761335 >> 172.21.15.19:0/1920761335 conn(0x7f54880d0800 msgr2=0x7f54880d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 -- 172.21.15.19:0/1920761335 shutdown_connections 2024-04-25T10:01:28.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.627+0000 7f54907db640 1 -- 172.21.15.19:0/1920761335 wait complete. 2024-04-25T10:01:28.659 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:01:28.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.763+0000 7f4d0f39e640 1 Processor -- start 2024-04-25T10:01:28.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.764+0000 7f4d0f39e640 1 -- start start 2024-04-25T10:01:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.764+0000 7f4d0f39e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d08070dc0 0x7f4d0806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.764+0000 7f4d0f39e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4d080711a0 0x7f4d080b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.764+0000 7f4d0f39e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d080d1770 con 0x7f4d080b4830 2024-04-25T10:01:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.764+0000 7f4d0f39e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d081622b0 con 0x7f4d080711a0 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.764+0000 7f4d0f39e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d080b10f0 con 0x7f4d08070dc0 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.765+0000 7f4d0c939640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4d080711a0 0x7f4d080b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.765+0000 7f4d0c939640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4d080711a0 0x7f4d080b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33436/0 (socket says 172.21.15.19:33436) 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.765+0000 7f4d0c939640 1 -- 172.21.15.19:0/3542053211 learned_addr learned my addr 172.21.15.19:0/3542053211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.765+0000 7f4d0d13a640 1 --2- 172.21.15.19:0/3542053211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d08070dc0 0x7f4d0806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.765+0000 7f4d0c939640 1 -- 172.21.15.19:0/3542053211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d08070dc0 msgr2=0x7f4d0806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.765+0000 7f4d0c939640 1 --2- 172.21.15.19:0/3542053211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d08070dc0 0x7f4d0806ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.765+0000 7f4d0c939640 1 -- 172.21.15.19:0/3542053211 >> v1:172.21.15.112:6789/0 conn(0x7f4d080b4830 legacy=0x7f4d080b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.765+0000 7f4d0c939640 1 -- 172.21.15.19:0/3542053211 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d08162740 con 0x7f4d080711a0 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.766+0000 7f4d0c939640 1 --2- 172.21.15.19:0/3542053211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d080711a0 0x7f4d080b07a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009870 tx=0x7f4cf802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb5be7e0306fc47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.766+0000 7f4cfffff640 1 -- 172.21.15.19:0/3542053211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8037070 con 0x7f4d080711a0 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.766+0000 7f4cfffff640 1 -- 172.21.15.19:0/3542053211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf8032440 con 0x7f4d080711a0 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.766+0000 7f4cfffff640 1 -- 172.21.15.19:0/3542053211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8032810 con 0x7f4d080711a0 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.766+0000 7f4d0f39e640 1 -- 172.21.15.19:0/3542053211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d080711a0 msgr2=0x7f4d080b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.766+0000 7f4d0f39e640 1 --2- 172.21.15.19:0/3542053211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d080711a0 0x7f4d080b07a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009870 tx=0x7f4cf802a130 comp rx=0 tx=0).stop 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.766+0000 7f4d0f39e640 1 -- 172.21.15.19:0/3542053211 shutdown_connections 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.766+0000 7f4d0f39e640 1 --2- 172.21.15.19:0/3542053211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d080711a0 0x7f4d080b07a0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.766+0000 7f4d0f39e640 1 --2- 172.21.15.19:0/3542053211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d08070dc0 0x7f4d0806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.766+0000 7f4d0f39e640 1 -- 172.21.15.19:0/3542053211 >> 172.21.15.19:0/3542053211 conn(0x7f4d080b3530 msgr2=0x7f4d080b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.767+0000 7f4d0f39e640 1 -- 172.21.15.19:0/3542053211 shutdown_connections 2024-04-25T10:01:28.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.767+0000 7f4d0f39e640 1 -- 172.21.15.19:0/3542053211 wait complete. 2024-04-25T10:01:28.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.768+0000 7f4d0f39e640 1 Processor -- start 2024-04-25T10:01:28.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.768+0000 7f4d0f39e640 1 -- start start 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.768+0000 7f4d0f39e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d08070dc0 0x7f4d081555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.768+0000 7f4d0f39e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d080711a0 0x7f4d08152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.768+0000 7f4d0f39e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d080b4830 0x7f4d0815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.768+0000 7f4d0f39e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08161f40 con 0x7f4d080711a0 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.768+0000 7f4d0f39e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08161c60 con 0x7f4d08070dc0 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.768+0000 7f4d0f39e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08071600 con 0x7f4d080b4830 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0c939640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d080711a0 0x7f4d08152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0c939640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d080711a0 0x7f4d08152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33438/0 (socket says 172.21.15.19:33438) 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0c939640 1 -- 172.21.15.19:0/2710130070 learned_addr learned my addr 172.21.15.19:0/2710130070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0d93b640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d080b4830 0x7f4d0815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0d13a640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d08070dc0 0x7f4d081555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0c939640 1 -- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d080b4830 msgr2=0x7f4d0815c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0c939640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d080b4830 0x7f4d0815c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0c939640 1 -- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d08070dc0 msgr2=0x7f4d081555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0c939640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d08070dc0 0x7f4d081555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0c939640 1 -- 172.21.15.19:0/2710130070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d08153770 con 0x7f4d080711a0 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0d13a640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d08070dc0 0x7f4d081555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0c939640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d080711a0 0x7f4d08152d20 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f4cf802a170 tx=0x7f4cf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4cfdffb640 1 -- 172.21.15.19:0/2710130070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8041020 con 0x7f4d080711a0 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4cfdffb640 1 -- 172.21.15.19:0/2710130070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf8030660 con 0x7f4d080711a0 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0f39e640 1 -- 172.21.15.19:0/2710130070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d08153a50 con 0x7f4d080711a0 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4cfdffb640 1 -- 172.21.15.19:0/2710130070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8037040 con 0x7f4d080711a0 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.769+0000 7f4d0f39e640 1 -- 172.21.15.19:0/2710130070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d08154010 con 0x7f4d080711a0 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.771+0000 7f4d0f39e640 1 -- 172.21.15.19:0/2710130070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d081543f0 con 0x7f4d080711a0 2024-04-25T10:01:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.771+0000 7f4cfdffb640 1 -- 172.21.15.19:0/2710130070 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cf8044020 con 0x7f4d080711a0 2024-04-25T10:01:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.772+0000 7f4cfdffb640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cec06b4d0 0x7f4cec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.772+0000 7f4cfdffb640 1 -- 172.21.15.19:0/2710130070 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f4cf80aab40 con 0x7f4d080711a0 2024-04-25T10:01:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.772+0000 7f4d0d13a640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cec06b4d0 0x7f4cec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.772+0000 7f4d0d13a640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cec06b4d0 0x7f4cec06d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4d081610f0 tx=0x7f4d04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.776+0000 7f4cfdffb640 1 -- 172.21.15.19:0/2710130070 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d081543f0 con 0x7f4d080711a0 2024-04-25T10:01:29.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:28.998+0000 7f4d0f39e640 1 -- 172.21.15.19:0/2710130070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4d08073a00 con 0x7f4d080711a0 2024-04-25T10:01:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.649+0000 7f4cfdffb640 1 -- 172.21.15.19:0/2710130070 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v77) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4cf804b030 con 0x7f4d080711a0 2024-04-25T10:01:29.660 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 19 and data pool 20 2024-04-25T10:01:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.669+0000 7f4d0f39e640 1 -- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cec06b4d0 msgr2=0x7f4cec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.669+0000 7f4d0f39e640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cec06b4d0 0x7f4cec06d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4d081610f0 tx=0x7f4d04008040 comp rx=0 tx=0).stop 2024-04-25T10:01:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.669+0000 7f4d0f39e640 1 -- 172.21.15.19:0/2710130070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d080711a0 msgr2=0x7f4d08152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.669+0000 7f4d0f39e640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d080711a0 0x7f4d08152d20 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f4cf802a170 tx=0x7f4cf80046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.669+0000 7f4d0f39e640 1 -- 172.21.15.19:0/2710130070 shutdown_connections 2024-04-25T10:01:29.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.669+0000 7f4d0f39e640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cec06b4d0 0x7f4cec06d990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.669+0000 7f4d0f39e640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d080b4830 0x7f4d0815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.669+0000 7f4d0f39e640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d080711a0 0x7f4d08152d20 secure :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f4cf802a170 tx=0x7f4cf80046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.669+0000 7f4d0f39e640 1 --2- 172.21.15.19:0/2710130070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d08070dc0 0x7f4d081555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.669+0000 7f4d0f39e640 1 -- 172.21.15.19:0/2710130070 >> 172.21.15.19:0/2710130070 conn(0x7f4d080b3530 msgr2=0x7f4d0806ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.671+0000 7f4d0f39e640 1 -- 172.21.15.19:0/2710130070 shutdown_connections 2024-04-25T10:01:29.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.671+0000 7f4d0f39e640 1 -- 172.21.15.19:0/2710130070 wait complete. 2024-04-25T10:01:29.690 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.790+0000 7fe95ca4d640 1 Processor -- start 2024-04-25T10:01:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.790+0000 7fe95ca4d640 1 -- start start 2024-04-25T10:01:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.791+0000 7fe95ca4d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe95815dbe0 0x7fe9580d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.791+0000 7fe95ca4d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9580b2cb0 0x7fe9580d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.791+0000 7fe95ca4d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9580b97a0 con 0x7fe9580b92a0 2024-04-25T10:01:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.791+0000 7fe95ca4d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95815dfc0 con 0x7fe9580b2cb0 2024-04-25T10:01:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.791+0000 7fe95ca4d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580d1a00 con 0x7fe95815dbe0 2024-04-25T10:01:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.791+0000 7fe955d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9580b2cb0 0x7fe9580d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.791+0000 7fe955d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9580b2cb0 0x7fe9580d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33444/0 (socket says 172.21.15.19:33444) 2024-04-25T10:01:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.791+0000 7fe955d9b640 1 -- 172.21.15.19:0/2615333114 learned_addr learned my addr 172.21.15.19:0/2615333114 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.792+0000 7fe95659c640 1 --2- 172.21.15.19:0/2615333114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe95815dbe0 0x7fe9580d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.792+0000 7fe955d9b640 1 -- 172.21.15.19:0/2615333114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe95815dbe0 msgr2=0x7fe9580d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.792+0000 7fe955d9b640 1 --2- 172.21.15.19:0/2615333114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe95815dbe0 0x7fe9580d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.792+0000 7fe955d9b640 1 -- 172.21.15.19:0/2615333114 >> v1:172.21.15.112:6789/0 conn(0x7fe9580b92a0 legacy=0x7fe9580b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.792+0000 7fe955d9b640 1 -- 172.21.15.19:0/2615333114 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580d1c50 con 0x7fe9580b2cb0 2024-04-25T10:01:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.792+0000 7fe955d9b640 1 --2- 172.21.15.19:0/2615333114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9580b2cb0 0x7fe9580d0e50 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009870 tx=0x7fe94c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16eac42ec26d78f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.793+0000 7fe95559a640 1 -- 172.21.15.19:0/2615333114 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c037070 con 0x7fe9580b2cb0 2024-04-25T10:01:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.793+0000 7fe95559a640 1 -- 172.21.15.19:0/2615333114 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c032440 con 0x7fe9580b2cb0 2024-04-25T10:01:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.793+0000 7fe95559a640 1 -- 172.21.15.19:0/2615333114 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c032810 con 0x7fe9580b2cb0 2024-04-25T10:01:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.793+0000 7fe95ca4d640 1 -- 172.21.15.19:0/2615333114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9580b2cb0 msgr2=0x7fe9580d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.793+0000 7fe95ca4d640 1 --2- 172.21.15.19:0/2615333114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9580b2cb0 0x7fe9580d0e50 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009870 tx=0x7fe94c02a270 comp rx=0 tx=0).stop 2024-04-25T10:01:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.793+0000 7fe95ca4d640 1 -- 172.21.15.19:0/2615333114 shutdown_connections 2024-04-25T10:01:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.793+0000 7fe95ca4d640 1 --2- 172.21.15.19:0/2615333114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9580b2cb0 0x7fe9580d0e50 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.793+0000 7fe95ca4d640 1 --2- 172.21.15.19:0/2615333114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe95815dbe0 0x7fe9580d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.793+0000 7fe95ca4d640 1 -- 172.21.15.19:0/2615333114 >> 172.21.15.19:0/2615333114 conn(0x7fe958061a20 msgr2=0x7fe958061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.793+0000 7fe95ca4d640 1 -- 172.21.15.19:0/2615333114 shutdown_connections 2024-04-25T10:01:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.793+0000 7fe95ca4d640 1 -- 172.21.15.19:0/2615333114 wait complete. 2024-04-25T10:01:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.795+0000 7fe95ca4d640 1 Processor -- start 2024-04-25T10:01:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.795+0000 7fe95ca4d640 1 -- start start 2024-04-25T10:01:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.795+0000 7fe95ca4d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9580b2cb0 0x7fe95815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.795+0000 7fe95ca4d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9580b92a0 0x7fe9580c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.795+0000 7fe95ca4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95815dbe0 0x7fe9580b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.795+0000 7fe95ca4d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580b6ab0 con 0x7fe9580b92a0 2024-04-25T10:01:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.795+0000 7fe95ca4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580d2210 con 0x7fe95815dbe0 2024-04-25T10:01:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.795+0000 7fe95ca4d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580b6c30 con 0x7fe9580b2cb0 2024-04-25T10:01:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.795+0000 7fe955d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9580b92a0 0x7fe9580c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.795+0000 7fe955d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9580b92a0 0x7fe9580c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33448/0 (socket says 172.21.15.19:33448) 2024-04-25T10:01:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.795+0000 7fe955d9b640 1 -- 172.21.15.19:0/3716592946 learned_addr learned my addr 172.21.15.19:0/3716592946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe95659c640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9580b2cb0 0x7fe95815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe956d9d640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95815dbe0 0x7fe9580b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe955d9b640 1 -- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9580b2cb0 msgr2=0x7fe95815f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe955d9b640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9580b2cb0 0x7fe95815f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe955d9b640 1 -- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95815dbe0 msgr2=0x7fe9580b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe955d9b640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95815dbe0 0x7fe9580b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe955d9b640 1 -- 172.21.15.19:0/3716592946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95815e710 con 0x7fe9580b92a0 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe956d9d640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95815dbe0 0x7fe9580b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe955d9b640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9580b92a0 0x7fe9580c9450 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fe94c000c00 tx=0x7fe94c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe946ffd640 1 -- 172.21.15.19:0/3716592946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c037040 con 0x7fe9580b92a0 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe946ffd640 1 -- 172.21.15.19:0/3716592946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c042070 con 0x7fe9580b92a0 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe95ca4d640 1 -- 172.21.15.19:0/3716592946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95815e9f0 con 0x7fe9580b92a0 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe946ffd640 1 -- 172.21.15.19:0/3716592946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c030880 con 0x7fe9580b92a0 2024-04-25T10:01:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.796+0000 7fe95ca4d640 1 -- 172.21.15.19:0/3716592946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95815efb0 con 0x7fe9580b92a0 2024-04-25T10:01:29.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.797+0000 7fe95ca4d640 1 -- 172.21.15.19:0/3716592946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe958073990 con 0x7fe9580b92a0 2024-04-25T10:01:29.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.798+0000 7fe946ffd640 1 -- 172.21.15.19:0/3716592946 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe94c030a20 con 0x7fe9580b92a0 2024-04-25T10:01:29.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.798+0000 7fe946ffd640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.798+0000 7fe946ffd640 1 -- 172.21.15.19:0/3716592946 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe94c0aabf0 con 0x7fe9580b92a0 2024-04-25T10:01:29.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.798+0000 7fe95659c640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.799+0000 7fe95659c640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe93806b4d0 0x7fe93806d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe9480073f0 tx=0x7fe94800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.802+0000 7fe946ffd640 1 -- 172.21.15.19:0/3716592946 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe94c0798e0 con 0x7fe9580b92a0 2024-04-25T10:01:29.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:01:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.982+0000 7fe95ca4d640 1 -- 172.21.15.19:0/3716592946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9580b48b0 con 0x7fe9580b92a0 2024-04-25T10:01:29.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.983+0000 7fe946ffd640 1 -- 172.21.15.19:0/3716592946 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16901 (secure 0 0 0) 0x7fe94c0791e0 con 0x7fe9580b92a0 2024-04-25T10:01:29.988 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:29.988 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":128,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:01:28.999821+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:27.270153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:27.977083+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 -- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe93806b4d0 msgr2=0x7fe93806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe93806b4d0 0x7fe93806d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe9480073f0 tx=0x7fe94800b040 comp rx=0 tx=0).stop 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 -- 172.21.15.19:0/3716592946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9580b92a0 msgr2=0x7fe9580c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9580b92a0 0x7fe9580c9450 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fe94c000c00 tx=0x7fe94c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 -- 172.21.15.19:0/3716592946 shutdown_connections 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe95815dbe0 0x7fe9580b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9580b92a0 0x7fe9580c9450 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 --2- 172.21.15.19:0/3716592946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9580b2cb0 0x7fe95815f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 -- 172.21.15.19:0/3716592946 >> 172.21.15.19:0/3716592946 conn(0x7fe958061a20 msgr2=0x7fe9580b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 -- 172.21.15.19:0/3716592946 shutdown_connections 2024-04-25T10:01:29.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:29.985+0000 7fe95ca4d640 1 -- 172.21.15.19:0/3716592946 wait complete. 2024-04-25T10:01:30.001 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:30.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.097+0000 7fcfe03cf640 1 Processor -- start 2024-04-25T10:01:30.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.097+0000 7fcfe03cf640 1 -- start start 2024-04-25T10:01:30.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.098+0000 7fcfe03cf640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcfd815db60 0x7fcfd80ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.098+0000 7fcfe03cf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcfd80d0b10 0x7fcfd80d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.098+0000 7fcfe03cf640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd80cec00 con 0x7fcfd80b5f00 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.098+0000 7fcfe03cf640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd80ce930 con 0x7fcfd80d0b10 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.098+0000 7fcfe03cf640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd80b6450 con 0x7fcfd815db60 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.098+0000 7fcfde96c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcfd80d0b10 0x7fcfd80d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.098+0000 7fcfde96c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcfd80d0b10 0x7fcfd80d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33460/0 (socket says 172.21.15.19:33460) 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.098+0000 7fcfdd96a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcfd80b5f00 0x7fcfd80d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40436/0 (socket says 172.21.15.19:40436) 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.098+0000 7fcfde96c640 1 -- 172.21.15.19:0/3093500483 learned_addr learned my addr 172.21.15.19:0/3093500483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.099+0000 7fcfde16b640 1 --2- 172.21.15.19:0/3093500483 >> v2:172.21.15.112:3300/0 conn(0x7fcfd815db60 0x7fcfd80ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.099+0000 7fcfde96c640 1 -- 172.21.15.19:0/3093500483 >> v2:172.21.15.112:3300/0 conn(0x7fcfd815db60 msgr2=0x7fcfd80ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.099+0000 7fcfde96c640 1 --2- 172.21.15.19:0/3093500483 >> v2:172.21.15.112:3300/0 conn(0x7fcfd815db60 0x7fcfd80ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.099+0000 7fcfde96c640 1 -- 172.21.15.19:0/3093500483 >> v1:172.21.15.19:6789/0 conn(0x7fcfd80b5f00 legacy=0x7fcfd80d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.099+0000 7fcfde96c640 1 -- 172.21.15.19:0/3093500483 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80ceee0 con 0x7fcfd80d0b10 2024-04-25T10:01:30.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.099+0000 7fcfde96c640 1 --2- 172.21.15.19:0/3093500483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfd80d0b10 0x7fcfd80d2270 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009e50 tx=0x7fcfd002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4c37f88eadac31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.100+0000 7fcfdd169640 1 -- 172.21.15.19:0/3093500483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0036070 con 0x7fcfd80d0b10 2024-04-25T10:01:30.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.100+0000 7fcfdd169640 1 -- 172.21.15.19:0/3093500483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd0031600 con 0x7fcfd80d0b10 2024-04-25T10:01:30.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.100+0000 7fcfdd169640 1 -- 172.21.15.19:0/3093500483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00319d0 con 0x7fcfd80d0b10 2024-04-25T10:01:30.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.100+0000 7fcfe03cf640 1 -- 172.21.15.19:0/3093500483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfd80d0b10 msgr2=0x7fcfd80d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.100+0000 7fcfe03cf640 1 --2- 172.21.15.19:0/3093500483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfd80d0b10 0x7fcfd80d2270 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009e50 tx=0x7fcfd002a6d0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.100+0000 7fcfe03cf640 1 -- 172.21.15.19:0/3093500483 shutdown_connections 2024-04-25T10:01:30.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.100+0000 7fcfe03cf640 1 --2- 172.21.15.19:0/3093500483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfd80d0b10 0x7fcfd80d2270 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.100+0000 7fcfe03cf640 1 --2- 172.21.15.19:0/3093500483 >> v2:172.21.15.112:3300/0 conn(0x7fcfd815db60 0x7fcfd80ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.100+0000 7fcfe03cf640 1 -- 172.21.15.19:0/3093500483 >> 172.21.15.19:0/3093500483 conn(0x7fcfd8061600 msgr2=0x7fcfd8061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.100+0000 7fcfe03cf640 1 -- 172.21.15.19:0/3093500483 shutdown_connections 2024-04-25T10:01:30.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.100+0000 7fcfe03cf640 1 -- 172.21.15.19:0/3093500483 wait complete. 2024-04-25T10:01:30.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.101+0000 7fcfe03cf640 1 Processor -- start 2024-04-25T10:01:30.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.102+0000 7fcfe03cf640 1 -- start start 2024-04-25T10:01:30.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.102+0000 7fcfe03cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd80b5f00 0x7fcfd81f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.102+0000 7fcfe03cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfd80d0b10 0x7fcfd80b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.102+0000 7fcfe03cf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfd815db60 0x7fcfd8151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.102+0000 7fcfe03cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd815df40 con 0x7fcfd80d0b10 2024-04-25T10:01:30.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.102+0000 7fcfe03cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80d1d50 con 0x7fcfd80b5f00 2024-04-25T10:01:30.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.102+0000 7fcfe03cf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd815e0c0 con 0x7fcfd815db60 2024-04-25T10:01:30.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.102+0000 7fcfdd96a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfd80d0b10 0x7fcfd80b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.102+0000 7fcfdd96a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfd80d0b10 0x7fcfd80b90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33470/0 (socket says 172.21.15.19:33470) 2024-04-25T10:01:30.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.102+0000 7fcfdd96a640 1 -- 172.21.15.19:0/1886497069 learned_addr learned my addr 172.21.15.19:0/1886497069 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:30.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfde16b640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd80b5f00 0x7fcfd81f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfde96c640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfd815db60 0x7fcfd8151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfdd96a640 1 -- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfd815db60 msgr2=0x7fcfd8151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfdd96a640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfd815db60 0x7fcfd8151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfdd96a640 1 -- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd80b5f00 msgr2=0x7fcfd81f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfdd96a640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd80b5f00 0x7fcfd81f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfdd96a640 1 -- 172.21.15.19:0/1886497069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd81f5240 con 0x7fcfd80d0b10 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfdd96a640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfd80d0b10 0x7fcfd80b90f0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800c400 tx=0x7fcfc800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfceffd640 1 -- 172.21.15.19:0/1886497069 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80074e0 con 0x7fcfd80d0b10 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfceffd640 1 -- 172.21.15.19:0/1886497069 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc8007680 con 0x7fcfd80d0b10 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfceffd640 1 -- 172.21.15.19:0/1886497069 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8004a00 con 0x7fcfd80d0b10 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfe03cf640 1 -- 172.21.15.19:0/1886497069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd815e7e0 con 0x7fcfd80d0b10 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.103+0000 7fcfe03cf640 1 -- 172.21.15.19:0/1886497069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd815f020 con 0x7fcfd80d0b10 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.104+0000 7fcfe03cf640 1 -- 172.21.15.19:0/1886497069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd8073270 con 0x7fcfd80d0b10 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.105+0000 7fcfceffd640 1 -- 172.21.15.19:0/1886497069 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfc801a020 con 0x7fcfd80d0b10 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.105+0000 7fcfceffd640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfbc06b4d0 0x7fcfbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.105+0000 7fcfceffd640 1 -- 172.21.15.19:0/1886497069 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcfc8089bb0 con 0x7fcfd80d0b10 2024-04-25T10:01:30.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.105+0000 7fcfde16b640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfbc06b4d0 0x7fcfbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.106+0000 7fcfde16b640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfbc06b4d0 0x7fcfbc06d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4006fd0 tx=0x7fcfd4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.109+0000 7fcfceffd640 1 -- 172.21.15.19:0/1886497069 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcfd815e7e0 con 0x7fcfd80d0b10 2024-04-25T10:01:30.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.291+0000 7fcfe03cf640 1 -- 172.21.15.19:0/1886497069 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfd8073990 con 0x7fcfd80d0b10 2024-04-25T10:01:30.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.292+0000 7fcfceffd640 1 -- 172.21.15.19:0/1886497069 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16901 (secure 0 0 0) 0x7fcfd81f5240 con 0x7fcfd80d0b10 2024-04-25T10:01:30.297 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:30.297 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":128,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:01:28.999821+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:27.270153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:27.977083+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 -- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfbc06b4d0 msgr2=0x7fcfbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfbc06b4d0 0x7fcfbc06d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4006fd0 tx=0x7fcfd4008040 comp rx=0 tx=0).stop 2024-04-25T10:01:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 -- 172.21.15.19:0/1886497069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfd80d0b10 msgr2=0x7fcfd80b90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfd80d0b10 0x7fcfd80b90f0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800c400 tx=0x7fcfc800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 -- 172.21.15.19:0/1886497069 shutdown_connections 2024-04-25T10:01:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfbc06b4d0 0x7fcfbc06d990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfd815db60 0x7fcfd8151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfd80d0b10 0x7fcfd80b90f0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 --2- 172.21.15.19:0/1886497069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd80b5f00 0x7fcfd81f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 -- 172.21.15.19:0/1886497069 >> 172.21.15.19:0/1886497069 conn(0x7fcfd8061600 msgr2=0x7fcfd815d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 -- 172.21.15.19:0/1886497069 shutdown_connections 2024-04-25T10:01:30.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.294+0000 7fcfe03cf640 1 -- 172.21.15.19:0/1886497069 wait complete. 2024-04-25T10:01:30.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:01:30.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.410+0000 7f8f9f5c4640 1 Processor -- start 2024-04-25T10:01:30.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.410+0000 7f8f9f5c4640 1 -- start start 2024-04-25T10:01:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.411+0000 7f8f9f5c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f9815bb50 0x7f8f980c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.411+0000 7f8f9f5c4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f980c70d0 0x7f8f980cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.411+0000 7f8f9f5c4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f980c75e0 con 0x7f8f980cfb40 2024-04-25T10:01:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.411+0000 7f8f9f5c4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f980c8680 con 0x7f8f980c70d0 2024-04-25T10:01:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.411+0000 7f8f9f5c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980c78c0 con 0x7f8f9815bb50 2024-04-25T10:01:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.411+0000 7f8f9cb5f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f980c70d0 0x7f8f980cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.411+0000 7f8f9db61640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8f980cfb40 0x7f8f98070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40452/0 (socket says 172.21.15.19:40452) 2024-04-25T10:01:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.411+0000 7f8f9cb5f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f980c70d0 0x7f8f980cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33480/0 (socket says 172.21.15.19:33480) 2024-04-25T10:01:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.411+0000 7f8f9db61640 1 -- 172.21.15.19:0/577712742 learned_addr learned my addr 172.21.15.19:0/577712742 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.412+0000 7f8f9d360640 1 --2- 172.21.15.19:0/577712742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f9815bb50 0x7f8f980c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.412+0000 7f8f9cb5f640 1 -- 172.21.15.19:0/577712742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f9815bb50 msgr2=0x7f8f980c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.412+0000 7f8f9cb5f640 1 --2- 172.21.15.19:0/577712742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f9815bb50 0x7f8f980c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.412+0000 7f8f9cb5f640 1 -- 172.21.15.19:0/577712742 >> v1:172.21.15.19:6789/0 conn(0x7f8f980cfb40 legacy=0x7f8f98070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.412+0000 7f8f97fff640 1 -- 172.21.15.19:0/577712742 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880373817 0 0) 0x7f8f980c75e0 con 0x7f8f980cfb40 2024-04-25T10:01:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.412+0000 7f8f9cb5f640 1 -- 172.21.15.19:0/577712742 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980c7ba0 con 0x7f8f980c70d0 2024-04-25T10:01:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.412+0000 7f8f9cb5f640 1 --2- 172.21.15.19:0/577712742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f980c70d0 0x7f8f980cf600 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0097a0 tx=0x7f8f8c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fdb85046aa8b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.413+0000 7f8f97fff640 1 -- 172.21.15.19:0/577712742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c037070 con 0x7f8f980c70d0 2024-04-25T10:01:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.413+0000 7f8f97fff640 1 -- 172.21.15.19:0/577712742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c032440 con 0x7f8f980c70d0 2024-04-25T10:01:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.413+0000 7f8f97fff640 1 -- 172.21.15.19:0/577712742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c032810 con 0x7f8f980c70d0 2024-04-25T10:01:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.413+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/577712742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f980c70d0 msgr2=0x7f8f980cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.413+0000 7f8f9f5c4640 1 --2- 172.21.15.19:0/577712742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f980c70d0 0x7f8f980cf600 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0097a0 tx=0x7f8f8c02a210 comp rx=0 tx=0).stop 2024-04-25T10:01:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.413+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/577712742 shutdown_connections 2024-04-25T10:01:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.413+0000 7f8f9f5c4640 1 --2- 172.21.15.19:0/577712742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f980c70d0 0x7f8f980cf600 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.413+0000 7f8f9f5c4640 1 --2- 172.21.15.19:0/577712742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f9815bb50 0x7f8f980c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.413+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/577712742 >> 172.21.15.19:0/577712742 conn(0x7f8f9806f500 msgr2=0x7f8f9806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.413+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/577712742 shutdown_connections 2024-04-25T10:01:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.413+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/577712742 wait complete. 2024-04-25T10:01:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.414+0000 7f8f9f5c4640 1 Processor -- start 2024-04-25T10:01:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.414+0000 7f8f9f5c4640 1 -- start start 2024-04-25T10:01:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.415+0000 7f8f9f5c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f980c70d0 0x7f8f9815e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.415+0000 7f8f9f5c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f980cfb40 0x7f8f98150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.415+0000 7f8f9f5c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f9815bb50 0x7f8f9806d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.415+0000 7f8f9f5c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9815f6d0 con 0x7f8f9815bb50 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.415+0000 7f8f9f5c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980c8160 con 0x7f8f980cfb40 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.415+0000 7f8f9f5c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9815f850 con 0x7f8f980c70d0 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.415+0000 7f8f9db61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f9815bb50 0x7f8f9806d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.415+0000 7f8f9db61640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f9815bb50 0x7f8f9806d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33482/0 (socket says 172.21.15.19:33482) 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.415+0000 7f8f9db61640 1 -- 172.21.15.19:0/3816206209 learned_addr learned my addr 172.21.15.19:0/3816206209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9cb5f640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f980cfb40 0x7f8f98150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9d360640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f980c70d0 0x7f8f9815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9db61640 1 -- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f980c70d0 msgr2=0x7f8f9815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9db61640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f980c70d0 0x7f8f9815e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9db61640 1 -- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f980cfb40 msgr2=0x7f8f98150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9db61640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f980cfb40 0x7f8f98150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9db61640 1 -- 172.21.15.19:0/3816206209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9806bd90 con 0x7f8f9815bb50 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9cb5f640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f980cfb40 0x7f8f98150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9d360640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f980c70d0 0x7f8f9815e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9db61640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f9815bb50 0x7f8f9806d670 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c7a0 tx=0x7f8f8800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f95ffb640 1 -- 172.21.15.19:0/3816206209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88018070 con 0x7f8f9815bb50 2024-04-25T10:01:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f95ffb640 1 -- 172.21.15.19:0/3816206209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8800a640 con 0x7f8f9815bb50 2024-04-25T10:01:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f95ffb640 1 -- 172.21.15.19:0/3816206209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8800aa70 con 0x7f8f9815bb50 2024-04-25T10:01:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/3816206209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9806c070 con 0x7f8f9815bb50 2024-04-25T10:01:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.416+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/3816206209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9806c630 con 0x7f8f9815bb50 2024-04-25T10:01:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.417+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/3816206209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f98073a00 con 0x7f8f9815bb50 2024-04-25T10:01:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.418+0000 7f8f95ffb640 1 -- 172.21.15.19:0/3816206209 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f8800ac10 con 0x7f8f9815bb50 2024-04-25T10:01:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.418+0000 7f8f95ffb640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f8406b4d0 0x7f8f8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.418+0000 7f8f95ffb640 1 -- 172.21.15.19:0/3816206209 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f88089a10 con 0x7f8f9815bb50 2024-04-25T10:01:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.418+0000 7f8f9d360640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f8406b4d0 0x7f8f8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.419+0000 7f8f9d360640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f8406b4d0 0x7f8f8406d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009a20 tx=0x7f8f90008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.422+0000 7f8f95ffb640 1 -- 172.21.15.19:0/3816206209 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f88058700 con 0x7f8f9815bb50 2024-04-25T10:01:30.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.644+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/3816206209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8f980cf150 con 0x7f8f9815bb50 2024-04-25T10:01:30.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.649+0000 7f8f95ffb640 1 -- 172.21.15.19:0/3816206209 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v78) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8f88058000 con 0x7f8f9815bb50 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f8406b4d0 msgr2=0x7f8f8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f8406b4d0 0x7f8f8406d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009a20 tx=0x7f8f90008040 comp rx=0 tx=0).stop 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/3816206209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f9815bb50 msgr2=0x7f8f9806d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f9815bb50 0x7f8f9806d670 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c7a0 tx=0x7f8f8800cc60 comp rx=0 tx=0).stop 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/3816206209 shutdown_connections 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8f8406b4d0 0x7f8f8406d990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f9815bb50 0x7f8f9806d670 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f980cfb40 0x7f8f98150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 --2- 172.21.15.19:0/3816206209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f980c70d0 0x7f8f9815e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/3816206209 >> 172.21.15.19:0/3816206209 conn(0x7f8f9806f500 msgr2=0x7f8f981567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/3816206209 shutdown_connections 2024-04-25T10:01:30.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.651+0000 7f8f9f5c4640 1 -- 172.21.15.19:0/3816206209 wait complete. 2024-04-25T10:01:30.667 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:01:30.667 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:01:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.766+0000 7f8cd31ce640 1 Processor -- start 2024-04-25T10:01:30.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.766+0000 7f8cd31ce640 1 -- start start 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8cd31ce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8ccc06c9e0 0x7f8ccc0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8cd31ce640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ccc06cf50 con 0x7f8ccc15b7d0 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8cd31ce640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ccc0ce530 con 0x7f8ccc0b5620 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8cd31ce640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc0ce810 con 0x7f8ccc06c9e0 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8cd0f6a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8ccc15b7d0 0x7f8ccc0cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40454/0 (socket says 172.21.15.19:40454) 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8cd0f6a640 1 -- 172.21.15.19:0/3394649777 learned_addr learned my addr 172.21.15.19:0/3394649777 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8cd176b640 1 --2- 172.21.15.19:0/3394649777 >> v2:172.21.15.112:3300/0 conn(0x7f8ccc06c9e0 0x7f8ccc0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8ccb7fe640 1 -- 172.21.15.19:0/3394649777 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326910023 0 0) 0x7f8ccc06cf50 con 0x7f8ccc15b7d0 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8ccb7fe640 1 -- 172.21.15.19:0/3394649777 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cbc003520 con 0x7f8ccc15b7d0 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8ccb7fe640 1 -- 172.21.15.19:0/3394649777 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598367427 0 0) 0x7f8ccc0ce530 con 0x7f8ccc0b5620 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8ccb7fe640 1 -- 172.21.15.19:0/3394649777 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ccc06cf50 con 0x7f8ccc0b5620 2024-04-25T10:01:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8cd176b640 1 -- 172.21.15.19:0/3394649777 >> v1:172.21.15.112:6789/0 conn(0x7f8ccc0b5620 legacy=0x7f8ccc06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8cd176b640 1 -- 172.21.15.19:0/3394649777 >> v1:172.21.15.19:6789/0 conn(0x7f8ccc15b7d0 legacy=0x7f8ccc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8ccb7fe640 1 -- 172.21.15.19:0/3394649777 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1488049226 0 0) 0x7f8cbc003520 con 0x7f8ccc15b7d0 2024-04-25T10:01:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.767+0000 7f8cd176b640 1 -- 172.21.15.19:0/3394649777 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc0ceaf0 con 0x7f8ccc06c9e0 2024-04-25T10:01:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.768+0000 7f8cd176b640 1 --2- 172.21.15.19:0/3394649777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ccc06c9e0 0x7f8ccc0b5ab0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009d90 tx=0x7f8cc402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be6eeff3d0de090f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.768+0000 7f8ccb7fe640 1 -- 172.21.15.19:0/3394649777 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0001670 con 0x7f8ccc06c9e0 2024-04-25T10:01:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.768+0000 7f8ccb7fe640 1 -- 172.21.15.19:0/3394649777 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc4007540 con 0x7f8ccc06c9e0 2024-04-25T10:01:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.768+0000 7f8ccb7fe640 1 -- 172.21.15.19:0/3394649777 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4007910 con 0x7f8ccc06c9e0 2024-04-25T10:01:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.769+0000 7f8cd31ce640 1 -- 172.21.15.19:0/3394649777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ccc06c9e0 msgr2=0x7f8ccc0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.769+0000 7f8cd31ce640 1 --2- 172.21.15.19:0/3394649777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ccc06c9e0 0x7f8ccc0b5ab0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009d90 tx=0x7f8cc402a680 comp rx=0 tx=0).stop 2024-04-25T10:01:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.769+0000 7f8cd31ce640 1 -- 172.21.15.19:0/3394649777 shutdown_connections 2024-04-25T10:01:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.769+0000 7f8cd31ce640 1 --2- 172.21.15.19:0/3394649777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ccc06c9e0 0x7f8ccc0b5ab0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.769+0000 7f8cd31ce640 1 -- 172.21.15.19:0/3394649777 >> 172.21.15.19:0/3394649777 conn(0x7f8ccc0d0800 msgr2=0x7f8ccc0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.769+0000 7f8cd31ce640 1 -- 172.21.15.19:0/3394649777 shutdown_connections 2024-04-25T10:01:30.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.769+0000 7f8cd31ce640 1 -- 172.21.15.19:0/3394649777 wait complete. 2024-04-25T10:01:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.770+0000 7f8cd31ce640 1 Processor -- start 2024-04-25T10:01:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.770+0000 7f8cd31ce640 1 -- start start 2024-04-25T10:01:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8cd31ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ccc06c9e0 0x7f8ccc150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8cd31ce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ccc0b5620 0x7f8ccc15d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8cd31ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ccc15b7d0 0x7f8ccc15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8cd31ce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc15bd30 con 0x7f8ccc0b5620 2024-04-25T10:01:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8cd31ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc15bbb0 con 0x7f8ccc06c9e0 2024-04-25T10:01:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8cd31ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0d1f40 con 0x7f8ccc15b7d0 2024-04-25T10:01:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ccc0b5620 0x7f8ccc15d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ccc0b5620 0x7f8ccc15d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33486/0 (socket says 172.21.15.19:33486) 2024-04-25T10:01:30.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8ccbfff640 1 -- 172.21.15.19:0/429809061 learned_addr learned my addr 172.21.15.19:0/429809061 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8cd0f6a640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ccc06c9e0 0x7f8ccc150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8cd176b640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ccc15b7d0 0x7f8ccc15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8ccbfff640 1 -- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ccc15b7d0 msgr2=0x7f8ccc15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.771+0000 7f8ccbfff640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ccc15b7d0 0x7f8ccc15d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.772+0000 7f8ccbfff640 1 -- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ccc06c9e0 msgr2=0x7f8ccc150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.772+0000 7f8ccbfff640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ccc06c9e0 0x7f8ccc150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.772+0000 7f8ccbfff640 1 -- 172.21.15.19:0/429809061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc1f3a60 con 0x7f8ccc0b5620 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.772+0000 7f8cd176b640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ccc15b7d0 0x7f8ccc15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.772+0000 7f8ccbfff640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ccc0b5620 0x7f8ccc15d160 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800c4c0 tx=0x7f8cb800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.772+0000 7f8cc97fa640 1 -- 172.21.15.19:0/429809061 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb80052d0 con 0x7f8ccc0b5620 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.772+0000 7f8cc97fa640 1 -- 172.21.15.19:0/429809061 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb8005470 con 0x7f8ccc0b5620 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.772+0000 7f8cd31ce640 1 -- 172.21.15.19:0/429809061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc1f3d40 con 0x7f8ccc0b5620 2024-04-25T10:01:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.772+0000 7f8cc97fa640 1 -- 172.21.15.19:0/429809061 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8012870 con 0x7f8ccc0b5620 2024-04-25T10:01:30.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.772+0000 7f8cd31ce640 1 -- 172.21.15.19:0/429809061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc1f4300 con 0x7f8ccc0b5620 2024-04-25T10:01:30.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.773+0000 7f8cd31ce640 1 -- 172.21.15.19:0/429809061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc073a00 con 0x7f8ccc0b5620 2024-04-25T10:01:30.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.773+0000 7f8cc97fa640 1 -- 172.21.15.19:0/429809061 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cb8012a10 con 0x7f8ccc0b5620 2024-04-25T10:01:30.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.774+0000 7f8cc97fa640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cb006b4d0 0x7f8cb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.774+0000 7f8cc97fa640 1 -- 172.21.15.19:0/429809061 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8cb808a860 con 0x7f8ccc0b5620 2024-04-25T10:01:30.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.774+0000 7f8cd0f6a640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cb006b4d0 0x7f8cb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.774+0000 7f8cd0f6a640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cb006b4d0 0x7f8cb006d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00017e0 tx=0x7f8cc0001610 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.778+0000 7f8cc97fa640 1 -- 172.21.15.19:0/429809061 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8cb8058c90 con 0x7f8ccc0b5620 2024-04-25T10:01:31.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:30.998+0000 7f8cd31ce640 1 -- 172.21.15.19:0/429809061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8ccc15b420 con 0x7f8ccc0b5620 2024-04-25T10:01:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.662+0000 7f8cc97fa640 1 -- 172.21.15.19:0/429809061 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v79) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8cb8058590 con 0x7f8ccc0b5620 2024-04-25T10:01:31.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.664+0000 7f8cd31ce640 1 -- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cb006b4d0 msgr2=0x7f8cb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:31.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.664+0000 7f8cd31ce640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cb006b4d0 0x7f8cb006d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00017e0 tx=0x7f8cc0001610 comp rx=0 tx=0).stop 2024-04-25T10:01:31.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.664+0000 7f8cd31ce640 1 -- 172.21.15.19:0/429809061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ccc0b5620 msgr2=0x7f8ccc15d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:31.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.664+0000 7f8cd31ce640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ccc0b5620 0x7f8ccc15d160 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800c4c0 tx=0x7f8cb800c980 comp rx=0 tx=0).stop 2024-04-25T10:01:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.665+0000 7f8cd31ce640 1 -- 172.21.15.19:0/429809061 shutdown_connections 2024-04-25T10:01:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.665+0000 7f8cd31ce640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cb006b4d0 0x7f8cb006d990 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.665+0000 7f8cd31ce640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ccc15b7d0 0x7f8ccc15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.665+0000 7f8cd31ce640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ccc0b5620 0x7f8ccc15d160 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.665+0000 7f8cd31ce640 1 --2- 172.21.15.19:0/429809061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ccc06c9e0 0x7f8ccc150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.665+0000 7f8cd31ce640 1 -- 172.21.15.19:0/429809061 >> 172.21.15.19:0/429809061 conn(0x7f8ccc0d0800 msgr2=0x7f8ccc0b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.665+0000 7f8cd31ce640 1 -- 172.21.15.19:0/429809061 shutdown_connections 2024-04-25T10:01:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.665+0000 7f8cd31ce640 1 -- 172.21.15.19:0/429809061 wait complete. 2024-04-25T10:01:31.683 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:01:31.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:31.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.785+0000 7f833d4d2640 1 Processor -- start 2024-04-25T10:01:31.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.785+0000 7f833d4d2640 1 -- start start 2024-04-25T10:01:31.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.786+0000 7f833d4d2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f83380d3300 0x7f83381689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:31.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.786+0000 7f833d4d2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8338161d50 con 0x7f833815da70 2024-04-25T10:01:31.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.786+0000 7f833d4d2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8338162020 con 0x7f83380b9730 2024-04-25T10:01:31.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.786+0000 7f833d4d2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83381625e0 con 0x7f83380d3300 2024-04-25T10:01:31.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.786+0000 7f8336ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f833815da70 0x7f8338161640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40456/0 (socket says 172.21.15.19:40456) 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.786+0000 7f8336ffd640 1 -- 172.21.15.19:0/3650926099 learned_addr learned my addr 172.21.15.19:0/3650926099 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.786+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306319498 0 0) 0x7f8338161d50 con 0x7f833815da70 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.786+0000 7f83377fe640 1 --2- 172.21.15.19:0/3650926099 >> v2:172.21.15.112:3300/0 conn(0x7f83380d3300 0x7f83381689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.786+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8324003520 con 0x7f833815da70 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.786+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 308067515 0 0) 0x7f8324003520 con 0x7f833815da70 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 >> v2:172.21.15.112:3300/0 conn(0x7f83380d3300 msgr2=0x7f83381689d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 --2- 172.21.15.19:0/3650926099 >> v2:172.21.15.112:3300/0 conn(0x7f83380d3300 0x7f83381689d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 >> v1:172.21.15.112:6789/0 conn(0x7f83380b9730 legacy=0x7f83380ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8338162300 con 0x7f833815da70 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8328001670 con 0x7f833815da70 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 >> v1:172.21.15.19:6789/0 conn(0x7f833815da70 legacy=0x7f8338161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 --2- 172.21.15.19:0/3650926099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83240039b0 0x7f8324003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 --2- 172.21.15.19:0/3650926099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8324004800 0x7f8324004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 --2- 172.21.15.19:0/3650926099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83240051c0 0x7f832400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83381625e0 con 0x7f83240051c0 2024-04-25T10:01:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832400c160 con 0x7f83240039b0 2024-04-25T10:01:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832400c360 con 0x7f8324004800 2024-04-25T10:01:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83377fe640 1 --2- 172.21.15.19:0/3650926099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83240051c0 0x7f832400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f8336ffd640 1 --2- 172.21.15.19:0/3650926099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83240039b0 0x7f8324003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f832ffff640 1 --2- 172.21.15.19:0/3650926099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8324004800 0x7f8324004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83377fe640 1 -- 172.21.15.19:0/3650926099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8324004800 msgr2=0x7f8324004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83377fe640 1 --2- 172.21.15.19:0/3650926099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8324004800 0x7f8324004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83377fe640 1 -- 172.21.15.19:0/3650926099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83240039b0 msgr2=0x7f8324003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83377fe640 1 --2- 172.21.15.19:0/3650926099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83240039b0 0x7f8324003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.787+0000 7f83377fe640 1 -- 172.21.15.19:0/3650926099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8324003520 con 0x7f83240051c0 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.788+0000 7f83377fe640 1 -- 172.21.15.19:0/3650926099 reap_dead start 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.788+0000 7f83377fe640 1 --2- 172.21.15.19:0/3650926099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83240051c0 0x7f832400b9c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f8330009f90 tx=0x7f8324005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9eb242a317d51bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.788+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330035020 con 0x7f83240051c0 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.788+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8330003b90 con 0x7f83240051c0 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.788+0000 7f83367fc640 1 -- 172.21.15.19:0/3650926099 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330009c10 con 0x7f83240051c0 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.788+0000 7f833d4d2640 1 -- 172.21.15.19:0/3650926099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83240051c0 msgr2=0x7f832400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.789+0000 7f833d4d2640 1 --2- 172.21.15.19:0/3650926099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83240051c0 0x7f832400b9c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f8330009f90 tx=0x7f8324005830 comp rx=0 tx=0).stop 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.789+0000 7f833d4d2640 1 -- 172.21.15.19:0/3650926099 shutdown_connections 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.789+0000 7f833d4d2640 1 --2- 172.21.15.19:0/3650926099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83240051c0 0x7f832400b9c0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.789+0000 7f833d4d2640 1 -- 172.21.15.19:0/3650926099 >> 172.21.15.19:0/3650926099 conn(0x7f8338061a20 msgr2=0x7f8338061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.789+0000 7f833d4d2640 1 -- 172.21.15.19:0/3650926099 shutdown_connections 2024-04-25T10:01:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.789+0000 7f833d4d2640 1 -- 172.21.15.19:0/3650926099 wait complete. 2024-04-25T10:01:31.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.790+0000 7f833d4d2640 1 Processor -- start 2024-04-25T10:01:31.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.790+0000 7f833d4d2640 1 -- start start 2024-04-25T10:01:31.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f833d4d2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83240051c0 0x7f83381f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:31.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f833d4d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83380c74b0 0x7f83380b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f833d4d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83380c8170 0x7f83380c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f833d4d2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833815de50 con 0x7f83380c8170 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f833d4d2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83380d1dd0 con 0x7f83380c74b0 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f833d4d2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8338161640 con 0x7f83240051c0 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f83377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83380c8170 0x7f83380c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f83377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83380c8170 0x7f83380c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33506/0 (socket says 172.21.15.19:33506) 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f83377fe640 1 -- 172.21.15.19:0/2696091904 learned_addr learned my addr 172.21.15.19:0/2696091904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f832ffff640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83380c74b0 0x7f83380b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f8336ffd640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83240051c0 0x7f83381f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f83377fe640 1 -- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83240051c0 msgr2=0x7f83381f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f83377fe640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83240051c0 0x7f83381f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f83377fe640 1 -- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83380c74b0 msgr2=0x7f83380b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:31.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f83377fe640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83380c74b0 0x7f83380b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f83377fe640 1 -- 172.21.15.19:0/2696091904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83381f3f20 con 0x7f83380c8170 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f83377fe640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83380c8170 0x7f83380c85f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f833000e230 tx=0x7f8330009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f832f7fe640 1 -- 172.21.15.19:0/2696091904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330035070 con 0x7f83380c8170 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f832f7fe640 1 -- 172.21.15.19:0/2696091904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8330041070 con 0x7f83380c8170 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f833d4d2640 1 -- 172.21.15.19:0/2696091904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83381f4200 con 0x7f83380c8170 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.791+0000 7f832f7fe640 1 -- 172.21.15.19:0/2696091904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330009c10 con 0x7f83380c8170 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.792+0000 7f833d4d2640 1 -- 172.21.15.19:0/2696091904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83381f3c10 con 0x7f83380c8170 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.793+0000 7f833d4d2640 1 -- 172.21.15.19:0/2696091904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8338073990 con 0x7f83380c8170 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.793+0000 7f832f7fe640 1 -- 172.21.15.19:0/2696091904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8330005d10 con 0x7f83380c8170 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.793+0000 7f832f7fe640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831806b500 0x7f831806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.794+0000 7f832f7fe640 1 -- 172.21.15.19:0/2696091904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83300aaed0 con 0x7f83380c8170 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.794+0000 7f8336ffd640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831806b500 0x7f831806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.794+0000 7f8336ffd640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831806b500 0x7f831806d9c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8328001cd0 tx=0x7f8328000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:31.798+0000 7f832f7fe640 1 -- 172.21.15.19:0/2696091904 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8330079560 con 0x7f83380c8170 2024-04-25T10:01:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.015+0000 7f833d4d2640 1 -- 172.21.15.19:0/2696091904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83380d1570 con 0x7f83380c8170 2024-04-25T10:01:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.015+0000 7f832f7fe640 1 -- 172.21.15.19:0/2696091904 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f833007d600 con 0x7f83380c8170 2024-04-25T10:01:32.020 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:32.020 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:01:30.999245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/250977615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":250977615},{"type":"v1","addr":"172.21.15.19:6835","nonce":250977615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:01:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 79 2024-04-25T10:01:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 -- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831806b500 msgr2=0x7f831806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831806b500 0x7f831806d9c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8328001cd0 tx=0x7f8328000ed0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 -- 172.21.15.19:0/2696091904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83380c8170 msgr2=0x7f83380c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83380c8170 0x7f83380c85f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f833000e230 tx=0x7f8330009750 comp rx=0 tx=0).stop 2024-04-25T10:01:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 -- 172.21.15.19:0/2696091904 shutdown_connections 2024-04-25T10:01:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f831806b500 0x7f831806d9c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83380c8170 0x7f83380c85f0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83380c74b0 0x7f83380b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 --2- 172.21.15.19:0/2696091904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83240051c0 0x7f83381f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 -- 172.21.15.19:0/2696091904 >> 172.21.15.19:0/2696091904 conn(0x7f8338061a20 msgr2=0x7f83380b84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 -- 172.21.15.19:0/2696091904 shutdown_connections 2024-04-25T10:01:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.017+0000 7f833d4d2640 1 -- 172.21.15.19:0/2696091904 wait complete. 2024-04-25T10:01:32.033 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:32.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.130+0000 7ff4212f5640 1 Processor -- start 2024-04-25T10:01:32.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.130+0000 7ff4212f5640 1 -- start start 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.130+0000 7ff4212f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff41c15dd30 0x7ff41c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.130+0000 7ff4212f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff41c0b9770 0x7ff41c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.130+0000 7ff4212f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff41c0d2e90 0x7ff41c0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.130+0000 7ff4212f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41c0d1df0 con 0x7ff41c0b9770 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.130+0000 7ff4212f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41c0d20d0 con 0x7ff41c15dd30 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.130+0000 7ff4212f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0d23b0 con 0x7ff41c0d2e90 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41a7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff41c0b9770 0x7ff41c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41a7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff41c0b9770 0x7ff41c0d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33516/0 (socket says 172.21.15.19:33516) 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41a7fc640 1 -- 172.21.15.19:0/1848022312 learned_addr learned my addr 172.21.15.19:0/1848022312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41a7fc640 1 -- 172.21.15.19:0/1848022312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff41c0d2e90 msgr2=0x7ff41c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41b7fe640 1 --2- 172.21.15.19:0/1848022312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff41c0d2e90 0x7ff41c0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41affd640 1 --2- 172.21.15.19:0/1848022312 >> v2:172.21.15.112:3300/0 conn(0x7ff41c15dd30 0x7ff41c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41a7fc640 1 --2- 172.21.15.19:0/1848022312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff41c0d2e90 0x7ff41c0d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41a7fc640 1 -- 172.21.15.19:0/1848022312 >> v2:172.21.15.112:3300/0 conn(0x7ff41c15dd30 msgr2=0x7ff41c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41a7fc640 1 --2- 172.21.15.19:0/1848022312 >> v2:172.21.15.112:3300/0 conn(0x7ff41c15dd30 0x7ff41c0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41a7fc640 1 -- 172.21.15.19:0/1848022312 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c0d2690 con 0x7ff41c0b9770 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41affd640 1 --2- 172.21.15.19:0/1848022312 >> v2:172.21.15.112:3300/0 conn(0x7ff41c15dd30 0x7ff41c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.131+0000 7ff41b7fe640 1 --2- 172.21.15.19:0/1848022312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff41c0d2e90 0x7ff41c0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff41a7fc640 1 --2- 172.21.15.19:0/1848022312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff41c0b9770 0x7ff41c0d03c0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff41c06d000 tx=0x7ff40402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e788216416739289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff419ffb640 1 -- 172.21.15.19:0/1848022312 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404004030 con 0x7ff41c0b9770 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff419ffb640 1 -- 172.21.15.19:0/1848022312 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4040041d0 con 0x7ff41c0b9770 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff419ffb640 1 -- 172.21.15.19:0/1848022312 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404003610 con 0x7ff41c0b9770 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff4212f5640 1 -- 172.21.15.19:0/1848022312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff41c0b9770 msgr2=0x7ff41c0d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff4212f5640 1 --2- 172.21.15.19:0/1848022312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff41c0b9770 0x7ff41c0d03c0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff41c06d000 tx=0x7ff40402a130 comp rx=0 tx=0).stop 2024-04-25T10:01:32.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff4212f5640 1 -- 172.21.15.19:0/1848022312 shutdown_connections 2024-04-25T10:01:32.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff4212f5640 1 --2- 172.21.15.19:0/1848022312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff41c0d2e90 0x7ff41c0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff4212f5640 1 --2- 172.21.15.19:0/1848022312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff41c0b9770 0x7ff41c0d03c0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff4212f5640 1 --2- 172.21.15.19:0/1848022312 >> v2:172.21.15.112:3300/0 conn(0x7ff41c15dd30 0x7ff41c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff4212f5640 1 -- 172.21.15.19:0/1848022312 >> 172.21.15.19:0/1848022312 conn(0x7ff41c061c30 msgr2=0x7ff41c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:32.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff4212f5640 1 -- 172.21.15.19:0/1848022312 shutdown_connections 2024-04-25T10:01:32.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.132+0000 7ff4212f5640 1 -- 172.21.15.19:0/1848022312 wait complete. 2024-04-25T10:01:32.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.134+0000 7ff4212f5640 1 Processor -- start 2024-04-25T10:01:32.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.134+0000 7ff4212f5640 1 -- start start 2024-04-25T10:01:32.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.134+0000 7ff4212f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff41c0b9770 0x7ff41c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.134+0000 7ff4212f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff41c0d2e90 0x7ff41c0cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.134+0000 7ff4212f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff41c15dd30 0x7ff41c15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.134+0000 7ff4212f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0d1b90 con 0x7ff41c0b9770 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.134+0000 7ff4212f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0d1a10 con 0x7ff41c0d2e90 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.134+0000 7ff4212f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0b5430 con 0x7ff41c15dd30 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff41c0b9770 0x7ff41c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff41c0b9770 0x7ff41c0cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33520/0 (socket says 172.21.15.19:33520) 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41affd640 1 -- 172.21.15.19:0/2514084189 learned_addr learned my addr 172.21.15.19:0/2514084189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41b7fe640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff41c15dd30 0x7ff41c15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41a7fc640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff41c0d2e90 0x7ff41c0cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41affd640 1 -- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff41c15dd30 msgr2=0x7ff41c15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41affd640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff41c15dd30 0x7ff41c15ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41affd640 1 -- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff41c0d2e90 msgr2=0x7ff41c0cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41affd640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff41c0d2e90 0x7ff41c0cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41affd640 1 -- 172.21.15.19:0/2514084189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c0aa520 con 0x7ff41c0b9770 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41a7fc640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff41c0d2e90 0x7ff41c0cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff41affd640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff41c0b9770 0x7ff41c0cd670 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c820 tx=0x7ff41400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff4137fe640 1 -- 172.21.15.19:0/2514084189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414015070 con 0x7ff41c0b9770 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff4137fe640 1 -- 172.21.15.19:0/2514084189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff414007a30 con 0x7ff41c0b9770 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff4137fe640 1 -- 172.21.15.19:0/2514084189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414007de0 con 0x7ff41c0b9770 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff4212f5640 1 -- 172.21.15.19:0/2514084189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c0aa800 con 0x7ff41c0b9770 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.135+0000 7ff4212f5640 1 -- 172.21.15.19:0/2514084189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c160210 con 0x7ff41c0b9770 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.136+0000 7ff4212f5640 1 -- 172.21.15.19:0/2514084189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c073990 con 0x7ff41c0b9770 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.137+0000 7ff4137fe640 1 -- 172.21.15.19:0/2514084189 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff414018070 con 0x7ff41c0b9770 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.137+0000 7ff4137fe640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3fc06b4d0 0x7ff3fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.137+0000 7ff4137fe640 1 -- 172.21.15.19:0/2514084189 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff41408a100 con 0x7ff41c0b9770 2024-04-25T10:01:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.137+0000 7ff41a7fc640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3fc06b4d0 0x7ff3fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.138+0000 7ff41a7fc640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3fc06b4d0 0x7ff3fc06d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff404033000 tx=0x7ff40402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.141+0000 7ff4137fe640 1 -- 172.21.15.19:0/2514084189 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff414058830 con 0x7ff41c0b9770 2024-04-25T10:01:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.325+0000 7ff4212f5640 1 -- 172.21.15.19:0/2514084189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff41c0b51f0 con 0x7ff41c0b9770 2024-04-25T10:01:32.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.326+0000 7ff4137fe640 1 -- 172.21.15.19:0/2514084189 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16888 (secure 0 0 0) 0x7ff41405c8d0 con 0x7ff41c0b9770 2024-04-25T10:01:32.331 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:32.331 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":129,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:01:30.649078+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:27.270153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:27.977083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 -- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3fc06b4d0 msgr2=0x7ff3fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3fc06b4d0 0x7ff3fc06d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff404033000 tx=0x7ff40402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 -- 172.21.15.19:0/2514084189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff41c0b9770 msgr2=0x7ff41c0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff41c0b9770 0x7ff41c0cd670 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c820 tx=0x7ff41400cce0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 -- 172.21.15.19:0/2514084189 shutdown_connections 2024-04-25T10:01:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3fc06b4d0 0x7ff3fc06d990 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff41c15dd30 0x7ff41c15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff41c0d2e90 0x7ff41c0cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 --2- 172.21.15.19:0/2514084189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff41c0b9770 0x7ff41c0cd670 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 -- 172.21.15.19:0/2514084189 >> 172.21.15.19:0/2514084189 conn(0x7ff41c061c30 msgr2=0x7ff41c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 -- 172.21.15.19:0/2514084189 shutdown_connections 2024-04-25T10:01:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.328+0000 7ff4212f5640 1 -- 172.21.15.19:0/2514084189 wait complete. 2024-04-25T10:01:32.344 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:01:32.344 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:32.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.440+0000 7f4aae7b7640 1 Processor -- start 2024-04-25T10:01:32.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.440+0000 7f4aae7b7640 1 -- start start 2024-04-25T10:01:32.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aae7b7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4aa815dc20 0x7f4aa81616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aae7b7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4aa80b93a0 0x7f4aa80ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aae7b7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa815e000 con 0x7f4aa80cec20 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aae7b7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa80cf090 con 0x7f4aa80b93a0 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aae7b7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa80b6bb0 con 0x7f4aa815dc20 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aa77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4aa80b93a0 0x7f4aa80ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aa77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4aa80b93a0 0x7f4aa80ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33522/0 (socket says 172.21.15.19:33522) 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aa77fe640 1 -- 172.21.15.19:0/3904250063 learned_addr learned my addr 172.21.15.19:0/3904250063 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aa7fff640 1 --2- 172.21.15.19:0/3904250063 >> v2:172.21.15.112:3300/0 conn(0x7f4aa815dc20 0x7f4aa81616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aa77fe640 1 -- 172.21.15.19:0/3904250063 >> v2:172.21.15.112:3300/0 conn(0x7f4aa815dc20 msgr2=0x7f4aa81616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aa77fe640 1 --2- 172.21.15.19:0/3904250063 >> v2:172.21.15.112:3300/0 conn(0x7f4aa815dc20 0x7f4aa81616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aa77fe640 1 -- 172.21.15.19:0/3904250063 >> v1:172.21.15.112:6789/0 conn(0x7f4aa80cec20 legacy=0x7f4aa806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.441+0000 7f4aa77fe640 1 -- 172.21.15.19:0/3904250063 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80b6030 con 0x7f4aa80b93a0 2024-04-25T10:01:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.442+0000 7f4aa77fe640 1 --2- 172.21.15.19:0/3904250063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa80b93a0 0x7f4aa80ce6e0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f4aa808be60 tx=0x7f4a9802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca947b4c0a06e841 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.442+0000 7f4aa6ffd640 1 -- 172.21.15.19:0/3904250063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98037070 con 0x7f4aa80b93a0 2024-04-25T10:01:32.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.442+0000 7f4aa6ffd640 1 -- 172.21.15.19:0/3904250063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a98032440 con 0x7f4aa80b93a0 2024-04-25T10:01:32.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.442+0000 7f4aa6ffd640 1 -- 172.21.15.19:0/3904250063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98032810 con 0x7f4aa80b93a0 2024-04-25T10:01:32.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.443+0000 7f4aae7b7640 1 -- 172.21.15.19:0/3904250063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa80b93a0 msgr2=0x7f4aa80ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.443+0000 7f4aae7b7640 1 --2- 172.21.15.19:0/3904250063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa80b93a0 0x7f4aa80ce6e0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f4aa808be60 tx=0x7f4a9802a230 comp rx=0 tx=0).stop 2024-04-25T10:01:32.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.443+0000 7f4aae7b7640 1 -- 172.21.15.19:0/3904250063 shutdown_connections 2024-04-25T10:01:32.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.443+0000 7f4aae7b7640 1 --2- 172.21.15.19:0/3904250063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa80b93a0 0x7f4aa80ce6e0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.443+0000 7f4aae7b7640 1 --2- 172.21.15.19:0/3904250063 >> v2:172.21.15.112:3300/0 conn(0x7f4aa815dc20 0x7f4aa81616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.443+0000 7f4aae7b7640 1 -- 172.21.15.19:0/3904250063 >> 172.21.15.19:0/3904250063 conn(0x7f4aa80d0490 msgr2=0x7f4aa80d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:32.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.443+0000 7f4aae7b7640 1 -- 172.21.15.19:0/3904250063 shutdown_connections 2024-04-25T10:01:32.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.443+0000 7f4aae7b7640 1 -- 172.21.15.19:0/3904250063 wait complete. 2024-04-25T10:01:32.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.444+0000 7f4aae7b7640 1 Processor -- start 2024-04-25T10:01:32.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.444+0000 7f4aae7b7640 1 -- start start 2024-04-25T10:01:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aae7b7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4aa80b93a0 0x7f4aa8146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aae7b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aa80cec20 0x7f4aa80c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aae7b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa815dc20 0x7f4aa8143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aae7b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa81623a0 con 0x7f4aa815dc20 2024-04-25T10:01:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aae7b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa81620c0 con 0x7f4aa80cec20 2024-04-25T10:01:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aae7b7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80d3520 con 0x7f4aa80b93a0 2024-04-25T10:01:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aacd54640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa815dc20 0x7f4aa8143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aacd54640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa815dc20 0x7f4aa8143dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33524/0 (socket says 172.21.15.19:33524) 2024-04-25T10:01:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aacd54640 1 -- 172.21.15.19:0/2661353029 learned_addr learned my addr 172.21.15.19:0/2661353029 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aa7fff640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4aa80b93a0 0x7f4aa8146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aa77fe640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aa80cec20 0x7f4aa80c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aacd54640 1 -- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4aa80b93a0 msgr2=0x7f4aa8146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aacd54640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4aa80b93a0 0x7f4aa8146810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aacd54640 1 -- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aa80cec20 msgr2=0x7f4aa80c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aacd54640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aa80cec20 0x7f4aa80c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aacd54640 1 -- 172.21.15.19:0/2661353029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80cbb30 con 0x7f4aa815dc20 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.445+0000 7f4aa77fe640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aa80cec20 0x7f4aa80c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.446+0000 7f4aacd54640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa815dc20 0x7f4aa8143dd0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000c780 tx=0x7f4aa000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.446+0000 7f4aa4ff9640 1 -- 172.21.15.19:0/2661353029 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0004b60 con 0x7f4aa815dc20 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.446+0000 7f4aa4ff9640 1 -- 172.21.15.19:0/2661353029 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa0004d00 con 0x7f4aa815dc20 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.446+0000 7f4aae7b7640 1 -- 172.21.15.19:0/2661353029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa80cbdd0 con 0x7f4aa815dc20 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.446+0000 7f4aa4ff9640 1 -- 172.21.15.19:0/2661353029 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000aa90 con 0x7f4aa815dc20 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.446+0000 7f4aae7b7640 1 -- 172.21.15.19:0/2661353029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa80cc390 con 0x7f4aa815dc20 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.447+0000 7f4aae7b7640 1 -- 172.21.15.19:0/2661353029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa8073990 con 0x7f4aa815dc20 2024-04-25T10:01:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.447+0000 7f4aa4ff9640 1 -- 172.21.15.19:0/2661353029 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4aa000ac30 con 0x7f4aa815dc20 2024-04-25T10:01:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.448+0000 7f4aa4ff9640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a8806b500 0x7f4a8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.448+0000 7f4aa4ff9640 1 -- 172.21.15.19:0/2661353029 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4aa008a8a0 con 0x7f4aa815dc20 2024-04-25T10:01:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.448+0000 7f4aa7fff640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a8806b500 0x7f4a8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.449+0000 7f4aa7fff640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a8806b500 0x7f4a8806d9c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4a90006fd0 tx=0x7f4a90008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.452+0000 7f4aa4ff9640 1 -- 172.21.15.19:0/2661353029 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4aa0059580 con 0x7f4aa815dc20 2024-04-25T10:01:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.629+0000 7f4aae7b7640 1 -- 172.21.15.19:0/2661353029 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4aa80d1720 con 0x7f4a8806b500 2024-04-25T10:01:32.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.636+0000 7f4aa4ff9640 1 -- 172.21.15.19:0/2661353029 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247073 (secure 0 0 0) 0x7f4aa80d1720 con 0x7f4a8806b500 2024-04-25T10:01:32.646 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:32.648 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:01:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 -- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a8806b500 msgr2=0x7f4a8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a8806b500 0x7f4a8806d9c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4a90006fd0 tx=0x7f4a90008040 comp rx=0 tx=0).stop 2024-04-25T10:01:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 -- 172.21.15.19:0/2661353029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa815dc20 msgr2=0x7f4aa8143dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa815dc20 0x7f4aa8143dd0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000c780 tx=0x7f4aa000cc40 comp rx=0 tx=0).stop 2024-04-25T10:01:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 -- 172.21.15.19:0/2661353029 shutdown_connections 2024-04-25T10:01:32.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a8806b500 0x7f4a8806d9c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa815dc20 0x7f4aa8143dd0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aa80cec20 0x7f4aa80c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 --2- 172.21.15.19:0/2661353029 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4aa80b93a0 0x7f4aa8146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 -- 172.21.15.19:0/2661353029 >> 172.21.15.19:0/2661353029 conn(0x7f4aa80d0490 msgr2=0x7f4aa80d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 -- 172.21.15.19:0/2661353029 shutdown_connections 2024-04-25T10:01:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.639+0000 7f4aae7b7640 1 -- 172.21.15.19:0/2661353029 wait complete. 2024-04-25T10:01:32.655 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-04-25T10:01:31.027424+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385164,"kb_used_data":13528,"kb_used_omap":12,"kb_used_meta":371507,"kb_avail":754589556,"statfs":{"total":773094113280,"available":772699705344,"internally_reserved":0,"allocated":13852672,"data_stored":7070132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":380423730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001255"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.639822+0000","last_change":"2024-04-25T10:01:28.639822+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.639822+0000","last_undegraded":"2024-04-25T10:01:28.639822+0000","last_fullsized":"2024-04-25T10:01:28.639822+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633777+0000","last_change":"2024-04-25T10:01:28.633777+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633777+0000","last_undegraded":"2024-04-25T10:01:28.633777+0000","last_fullsized":"2024-04-25T10:01:28.633777+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631663+0000","last_change":"2024-04-25T10:01:28.631663+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631663+0000","last_undegraded":"2024-04-25T10:01:28.631663+0000","last_fullsized":"2024-04-25T10:01:28.631663+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633655+0000","last_change":"2024-04-25T10:01:28.633655+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633655+0000","last_undegraded":"2024-04-25T10:01:28.633655+0000","last_fullsized":"2024-04-25T10:01:28.633655+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631604+0000","last_change":"2024-04-25T10:01:28.631604+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631604+0000","last_undegraded":"2024-04-25T10:01:28.631604+0000","last_fullsized":"2024-04-25T10:01:28.631604+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632464+0000","last_change":"2024-04-25T10:01:28.632464+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632464+0000","last_undegraded":"2024-04-25T10:01:28.632464+0000","last_fullsized":"2024-04-25T10:01:28.632464+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630901+0000","last_change":"2024-04-25T10:01:28.630901+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630901+0000","last_undegraded":"2024-04-25T10:01:28.630901+0000","last_fullsized":"2024-04-25T10:01:28.630901+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633293+0000","last_change":"2024-04-25T10:01:28.633293+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633293+0000","last_undegraded":"2024-04-25T10:01:28.633293+0000","last_fullsized":"2024-04-25T10:01:28.633293+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632483+0000","last_change":"2024-04-25T10:01:28.632483+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632483+0000","last_undegraded":"2024-04-25T10:01:28.632483+0000","last_fullsized":"2024-04-25T10:01:28.632483+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634762+0000","last_change":"2024-04-25T10:01:28.634762+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634762+0000","last_undegraded":"2024-04-25T10:01:28.634762+0000","last_fullsized":"2024-04-25T10:01:28.634762+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634423+0000","last_change":"2024-04-25T10:01:28.634423+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634423+0000","last_undegraded":"2024-04-25T10:01:28.634423+0000","last_fullsized":"2024-04-25T10:01:28.634423+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633781+0000","last_change":"2024-04-25T10:01:28.633781+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633781+0000","last_undegraded":"2024-04-25T10:01:28.633781+0000","last_fullsized":"2024-04-25T10:01:28.633781+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631273+0000","last_change":"2024-04-25T10:01:28.631273+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631273+0000","last_undegraded":"2024-04-25T10:01:28.631273+0000","last_fullsized":"2024-04-25T10:01:28.631273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633250+0000","last_change":"2024-04-25T10:01:28.633250+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633250+0000","last_undegraded":"2024-04-25T10:01:28.633250+0000","last_fullsized":"2024-04-25T10:01:28.633250+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632131+0000","last_change":"2024-04-25T10:01:28.632131+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632131+0000","last_undegraded":"2024-04-25T10:01:28.632131+0000","last_fullsized":"2024-04-25T10:01:28.632131+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631442+0000","last_change":"2024-04-25T10:01:28.631442+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631442+0000","last_undegraded":"2024-04-25T10:01:28.631442+0000","last_fullsized":"2024-04-25T10:01:28.631442+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631831+0000","last_change":"2024-04-25T10:01:28.631831+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631831+0000","last_undegraded":"2024-04-25T10:01:28.631831+0000","last_fullsized":"2024-04-25T10:01:28.631831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635356+0000","last_change":"2024-04-25T10:01:28.635356+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635356+0000","last_undegraded":"2024-04-25T10:01:28.635356+0000","last_fullsized":"2024-04-25T10:01:28.635356+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629531+0000","last_change":"2024-04-25T10:01:28.629531+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629531+0000","last_undegraded":"2024-04-25T10:01:28.629531+0000","last_fullsized":"2024-04-25T10:01:28.629531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633354+0000","last_change":"2024-04-25T10:01:28.633354+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633354+0000","last_undegraded":"2024-04-25T10:01:28.633354+0000","last_fullsized":"2024-04-25T10:01:28.633354+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634748+0000","last_change":"2024-04-25T10:01:28.634748+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634748+0000","last_undegraded":"2024-04-25T10:01:28.634748+0000","last_fullsized":"2024-04-25T10:01:28.634748+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631665+0000","last_change":"2024-04-25T10:01:28.631665+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631665+0000","last_undegraded":"2024-04-25T10:01:28.631665+0000","last_fullsized":"2024-04-25T10:01:28.631665+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634367+0000","last_change":"2024-04-25T10:01:28.634367+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634367+0000","last_undegraded":"2024-04-25T10:01:28.634367+0000","last_fullsized":"2024-04-25T10:01:28.634367+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631023+0000","last_change":"2024-04-25T10:01:28.631023+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631023+0000","last_undegraded":"2024-04-25T10:01:28.631023+0000","last_fullsized":"2024-04-25T10:01:28.631023+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631167+0000","last_change":"2024-04-25T10:01:28.631167+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631167+0000","last_undegraded":"2024-04-25T10:01:28.631167+0000","last_fullsized":"2024-04-25T10:01:28.631167+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629895+0000","last_change":"2024-04-25T10:01:28.629895+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629895+0000","last_undegraded":"2024-04-25T10:01:28.629895+0000","last_fullsized":"2024-04-25T10:01:28.629895+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628825+0000","last_change":"2024-04-25T10:01:28.628825+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628825+0000","last_undegraded":"2024-04-25T10:01:28.628825+0000","last_fullsized":"2024-04-25T10:01:28.628825+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.637120+0000","last_change":"2024-04-25T10:01:28.637120+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.637120+0000","last_undegraded":"2024-04-25T10:01:28.637120+0000","last_fullsized":"2024-04-25T10:01:28.637120+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632165+0000","last_change":"2024-04-25T10:01:28.632165+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632165+0000","last_undegraded":"2024-04-25T10:01:28.632165+0000","last_fullsized":"2024-04-25T10:01:28.632165+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629573+0000","last_change":"2024-04-25T10:01:28.629573+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629573+0000","last_undegraded":"2024-04-25T10:01:28.629573+0000","last_fullsized":"2024-04-25T10:01:28.629573+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632296+0000","last_change":"2024-04-25T10:01:28.632296+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632296+0000","last_undegraded":"2024-04-25T10:01:28.632296+0000","last_fullsized":"2024-04-25T10:01:28.632296+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634707+0000","last_change":"2024-04-25T10:01:28.634707+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634707+0000","last_undegraded":"2024-04-25T10:01:28.634707+0000","last_fullsized":"2024-04-25T10:01:28.634707+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632907+0000","last_change":"2024-04-25T10:01:28.632907+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632907+0000","last_undegraded":"2024-04-25T10:01:28.632907+0000","last_fullsized":"2024-04-25T10:01:28.632907+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645733+0000","last_change":"2024-04-25T10:01:28.644943+0000","last_active":"2024-04-25T10:01:28.645733+0000","last_peered":"2024-04-25T10:01:28.645733+0000","last_clean":"2024-04-25T10:01:28.645733+0000","last_became_active":"2024-04-25T10:01:28.644689+0000","last_became_peered":"2024-04-25T10:01:28.644689+0000","last_unstale":"2024-04-25T10:01:28.645733+0000","last_undegraded":"2024-04-25T10:01:28.645733+0000","last_fullsized":"2024-04-25T10:01:28.645733+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:48.121372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631082+0000","last_change":"2024-04-25T10:01:28.631082+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631082+0000","last_undegraded":"2024-04-25T10:01:28.631082+0000","last_fullsized":"2024-04-25T10:01:28.631082+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.647323+0000","last_change":"2024-04-25T10:01:28.645571+0000","last_active":"2024-04-25T10:01:28.647323+0000","last_peered":"2024-04-25T10:01:28.647323+0000","last_clean":"2024-04-25T10:01:28.647323+0000","last_became_active":"2024-04-25T10:01:28.645283+0000","last_became_peered":"2024-04-25T10:01:28.645283+0000","last_unstale":"2024-04-25T10:01:28.647323+0000","last_undegraded":"2024-04-25T10:01:28.647323+0000","last_fullsized":"2024-04-25T10:01:28.647323+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:11.425437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633628+0000","last_change":"2024-04-25T10:01:28.633628+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633628+0000","last_undegraded":"2024-04-25T10:01:28.633628+0000","last_fullsized":"2024-04-25T10:01:28.633628+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643814+0000","last_change":"2024-04-25T10:01:28.640421+0000","last_active":"2024-04-25T10:01:28.643814+0000","last_peered":"2024-04-25T10:01:28.643814+0000","last_clean":"2024-04-25T10:01:28.643814+0000","last_became_active":"2024-04-25T10:01:28.639536+0000","last_became_peered":"2024-04-25T10:01:28.639536+0000","last_unstale":"2024-04-25T10:01:28.643814+0000","last_undegraded":"2024-04-25T10:01:28.643814+0000","last_fullsized":"2024-04-25T10:01:28.643814+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.056666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.638331+0000","last_change":"2024-04-25T10:01:28.638331+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.638331+0000","last_undegraded":"2024-04-25T10:01:28.638331+0000","last_fullsized":"2024-04-25T10:01:28.638331+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643382+0000","last_change":"2024-04-25T10:01:28.642064+0000","last_active":"2024-04-25T10:01:28.643382+0000","last_peered":"2024-04-25T10:01:28.643382+0000","last_clean":"2024-04-25T10:01:28.643382+0000","last_became_active":"2024-04-25T10:01:28.641767+0000","last_became_peered":"2024-04-25T10:01:28.641767+0000","last_unstale":"2024-04-25T10:01:28.643382+0000","last_undegraded":"2024-04-25T10:01:28.643382+0000","last_fullsized":"2024-04-25T10:01:28.643382+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:28.715123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632603+0000","last_change":"2024-04-25T10:01:28.632603+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632603+0000","last_undegraded":"2024-04-25T10:01:28.632603+0000","last_fullsized":"2024-04-25T10:01:28.632603+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645590+0000","last_change":"2024-04-25T10:01:28.643613+0000","last_active":"2024-04-25T10:01:28.645590+0000","last_peered":"2024-04-25T10:01:28.645590+0000","last_clean":"2024-04-25T10:01:28.645590+0000","last_became_active":"2024-04-25T10:01:28.643326+0000","last_became_peered":"2024-04-25T10:01:28.643326+0000","last_unstale":"2024-04-25T10:01:28.645590+0000","last_undegraded":"2024-04-25T10:01:28.645590+0000","last_fullsized":"2024-04-25T10:01:28.645590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:47.363454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632282+0000","last_change":"2024-04-25T10:01:28.632282+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632282+0000","last_undegraded":"2024-04-25T10:01:28.632282+0000","last_fullsized":"2024-04-25T10:01:28.632282+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648286+0000","last_change":"2024-04-25T10:01:28.647013+0000","last_active":"2024-04-25T10:01:28.648286+0000","last_peered":"2024-04-25T10:01:28.648286+0000","last_clean":"2024-04-25T10:01:28.648286+0000","last_became_active":"2024-04-25T10:01:28.646667+0000","last_became_peered":"2024-04-25T10:01:28.646667+0000","last_unstale":"2024-04-25T10:01:28.648286+0000","last_undegraded":"2024-04-25T10:01:28.648286+0000","last_fullsized":"2024-04-25T10:01:28.648286+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:28.715123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634967+0000","last_change":"2024-04-25T10:01:28.634967+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634967+0000","last_undegraded":"2024-04-25T10:01:28.634967+0000","last_fullsized":"2024-04-25T10:01:28.634967+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643539+0000","last_change":"2024-04-25T10:01:28.639325+0000","last_active":"2024-04-25T10:01:28.643539+0000","last_peered":"2024-04-25T10:01:28.643539+0000","last_clean":"2024-04-25T10:01:28.643539+0000","last_became_active":"2024-04-25T10:01:28.639055+0000","last_became_peered":"2024-04-25T10:01:28.639055+0000","last_unstale":"2024-04-25T10:01:28.643539+0000","last_undegraded":"2024-04-25T10:01:28.643539+0000","last_fullsized":"2024-04-25T10:01:28.643539+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:09.888995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629531+0000","last_change":"2024-04-25T10:01:28.629531+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629531+0000","last_undegraded":"2024-04-25T10:01:28.629531+0000","last_fullsized":"2024-04-25T10:01:28.629531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.640085+0000","last_change":"2024-04-25T10:01:28.639418+0000","last_active":"2024-04-25T10:01:28.640085+0000","last_peered":"2024-04-25T10:01:28.640085+0000","last_clean":"2024-04-25T10:01:28.640085+0000","last_became_active":"2024-04-25T10:01:28.639134+0000","last_became_peered":"2024-04-25T10:01:28.639134+0000","last_unstale":"2024-04-25T10:01:28.640085+0000","last_undegraded":"2024-04-25T10:01:28.640085+0000","last_fullsized":"2024-04-25T10:01:28.640085+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:09.409126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631028+0000","last_change":"2024-04-25T10:01:28.631028+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631028+0000","last_undegraded":"2024-04-25T10:01:28.631028+0000","last_fullsized":"2024-04-25T10:01:28.631028+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.641745+0000","last_change":"2024-04-25T10:01:28.640498+0000","last_active":"2024-04-25T10:01:28.641745+0000","last_peered":"2024-04-25T10:01:28.641745+0000","last_clean":"2024-04-25T10:01:28.641745+0000","last_became_active":"2024-04-25T10:01:28.640186+0000","last_became_peered":"2024-04-25T10:01:28.640186+0000","last_unstale":"2024-04-25T10:01:28.641745+0000","last_undegraded":"2024-04-25T10:01:28.641745+0000","last_fullsized":"2024-04-25T10:01:28.641745+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:23.471932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632233+0000","last_change":"2024-04-25T10:01:28.632233+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632233+0000","last_undegraded":"2024-04-25T10:01:28.632233+0000","last_fullsized":"2024-04-25T10:01:28.632233+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648909+0000","last_change":"2024-04-25T10:01:28.647926+0000","last_active":"2024-04-25T10:01:28.648909+0000","last_peered":"2024-04-25T10:01:28.648909+0000","last_clean":"2024-04-25T10:01:28.648909+0000","last_became_active":"2024-04-25T10:01:28.646963+0000","last_became_peered":"2024-04-25T10:01:28.646963+0000","last_unstale":"2024-04-25T10:01:28.648909+0000","last_undegraded":"2024-04-25T10:01:28.648909+0000","last_fullsized":"2024-04-25T10:01:28.648909+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:56.486037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629097+0000","last_change":"2024-04-25T10:01:28.629097+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629097+0000","last_undegraded":"2024-04-25T10:01:28.629097+0000","last_fullsized":"2024-04-25T10:01:28.629097+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643792+0000","last_change":"2024-04-25T10:01:28.638314+0000","last_active":"2024-04-25T10:01:28.643792+0000","last_peered":"2024-04-25T10:01:28.643792+0000","last_clean":"2024-04-25T10:01:28.643792+0000","last_became_active":"2024-04-25T10:01:28.637960+0000","last_became_peered":"2024-04-25T10:01:28.637960+0000","last_unstale":"2024-04-25T10:01:28.643792+0000","last_undegraded":"2024-04-25T10:01:28.643792+0000","last_fullsized":"2024-04-25T10:01:28.643792+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.356021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628748+0000","last_change":"2024-04-25T10:01:28.628748+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628748+0000","last_undegraded":"2024-04-25T10:01:28.628748+0000","last_fullsized":"2024-04-25T10:01:28.628748+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643363+0000","last_change":"2024-04-25T10:01:28.639720+0000","last_active":"2024-04-25T10:01:28.643363+0000","last_peered":"2024-04-25T10:01:28.643363+0000","last_clean":"2024-04-25T10:01:28.643363+0000","last_became_active":"2024-04-25T10:01:28.638956+0000","last_became_peered":"2024-04-25T10:01:28.638956+0000","last_unstale":"2024-04-25T10:01:28.643363+0000","last_undegraded":"2024-04-25T10:01:28.643363+0000","last_fullsized":"2024-04-25T10:01:28.643363+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:11.425437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635339+0000","last_change":"2024-04-25T10:01:28.635339+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635339+0000","last_undegraded":"2024-04-25T10:01:28.635339+0000","last_fullsized":"2024-04-25T10:01:28.635339+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648870+0000","last_change":"2024-04-25T10:01:28.644548+0000","last_active":"2024-04-25T10:01:28.648870+0000","last_peered":"2024-04-25T10:01:28.648870+0000","last_clean":"2024-04-25T10:01:28.648870+0000","last_became_active":"2024-04-25T10:01:28.643068+0000","last_became_peered":"2024-04-25T10:01:28.643068+0000","last_unstale":"2024-04-25T10:01:28.648870+0000","last_undegraded":"2024-04-25T10:01:28.648870+0000","last_fullsized":"2024-04-25T10:01:28.648870+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:28.602317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628719+0000","last_change":"2024-04-25T10:01:28.628719+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628719+0000","last_undegraded":"2024-04-25T10:01:28.628719+0000","last_fullsized":"2024-04-25T10:01:28.628719+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.642739+0000","last_change":"2024-04-25T10:01:28.641313+0000","last_active":"2024-04-25T10:01:28.642739+0000","last_peered":"2024-04-25T10:01:28.642739+0000","last_clean":"2024-04-25T10:01:28.642739+0000","last_became_active":"2024-04-25T10:01:28.641020+0000","last_became_peered":"2024-04-25T10:01:28.641020+0000","last_unstale":"2024-04-25T10:01:28.642739+0000","last_undegraded":"2024-04-25T10:01:28.642739+0000","last_fullsized":"2024-04-25T10:01:28.642739+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:47.363454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632145+0000","last_change":"2024-04-25T10:01:28.632145+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632145+0000","last_undegraded":"2024-04-25T10:01:28.632145+0000","last_fullsized":"2024-04-25T10:01:28.632145+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643497+0000","last_change":"2024-04-25T10:01:28.639344+0000","last_active":"2024-04-25T10:01:28.643497+0000","last_peered":"2024-04-25T10:01:28.643497+0000","last_clean":"2024-04-25T10:01:28.643497+0000","last_became_active":"2024-04-25T10:01:28.639079+0000","last_became_peered":"2024-04-25T10:01:28.639079+0000","last_unstale":"2024-04-25T10:01:28.643497+0000","last_undegraded":"2024-04-25T10:01:28.643497+0000","last_fullsized":"2024-04-25T10:01:28.643497+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:13.447365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632144+0000","last_change":"2024-04-25T10:01:28.632144+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632144+0000","last_undegraded":"2024-04-25T10:01:28.632144+0000","last_fullsized":"2024-04-25T10:01:28.632144+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.644323+0000","last_change":"2024-04-25T10:01:28.642872+0000","last_active":"2024-04-25T10:01:28.644323+0000","last_peered":"2024-04-25T10:01:28.644323+0000","last_clean":"2024-04-25T10:01:28.644323+0000","last_became_active":"2024-04-25T10:01:28.642250+0000","last_became_peered":"2024-04-25T10:01:28.642250+0000","last_unstale":"2024-04-25T10:01:28.644323+0000","last_undegraded":"2024-04-25T10:01:28.644323+0000","last_fullsized":"2024-04-25T10:01:28.644323+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:09.710270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630415+0000","last_change":"2024-04-25T10:01:28.630415+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630415+0000","last_undegraded":"2024-04-25T10:01:28.630415+0000","last_fullsized":"2024-04-25T10:01:28.630415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.644964+0000","last_change":"2024-04-25T10:01:28.643788+0000","last_active":"2024-04-25T10:01:28.644964+0000","last_peered":"2024-04-25T10:01:28.644964+0000","last_clean":"2024-04-25T10:01:28.644964+0000","last_became_active":"2024-04-25T10:01:28.643521+0000","last_became_peered":"2024-04-25T10:01:28.643521+0000","last_unstale":"2024-04-25T10:01:28.644964+0000","last_undegraded":"2024-04-25T10:01:28.644964+0000","last_fullsized":"2024-04-25T10:01:28.644964+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:10.820229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628642+0000","last_change":"2024-04-25T10:01:28.628642+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628642+0000","last_undegraded":"2024-04-25T10:01:28.628642+0000","last_fullsized":"2024-04-25T10:01:28.628642+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643193+0000","last_change":"2024-04-25T10:01:28.640275+0000","last_active":"2024-04-25T10:01:28.643193+0000","last_peered":"2024-04-25T10:01:28.643193+0000","last_clean":"2024-04-25T10:01:28.643193+0000","last_became_active":"2024-04-25T10:01:28.640015+0000","last_became_peered":"2024-04-25T10:01:28.640015+0000","last_unstale":"2024-04-25T10:01:28.643193+0000","last_undegraded":"2024-04-25T10:01:28.643193+0000","last_fullsized":"2024-04-25T10:01:28.643193+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:03.858446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635398+0000","last_change":"2024-04-25T10:01:28.635398+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635398+0000","last_undegraded":"2024-04-25T10:01:28.635398+0000","last_fullsized":"2024-04-25T10:01:28.635398+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645355+0000","last_change":"2024-04-25T10:01:28.644201+0000","last_active":"2024-04-25T10:01:28.645355+0000","last_peered":"2024-04-25T10:01:28.645355+0000","last_clean":"2024-04-25T10:01:28.645355+0000","last_became_active":"2024-04-25T10:01:28.643895+0000","last_became_peered":"2024-04-25T10:01:28.643895+0000","last_unstale":"2024-04-25T10:01:28.645355+0000","last_undegraded":"2024-04-25T10:01:28.645355+0000","last_fullsized":"2024-04-25T10:01:28.645355+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.356021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632929+0000","last_change":"2024-04-25T10:01:28.632929+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632929+0000","last_undegraded":"2024-04-25T10:01:28.632929+0000","last_fullsized":"2024-04-25T10:01:28.632929+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643746+0000","last_change":"2024-04-25T10:01:28.640556+0000","last_active":"2024-04-25T10:01:28.643746+0000","last_peered":"2024-04-25T10:01:28.643746+0000","last_clean":"2024-04-25T10:01:28.643746+0000","last_became_active":"2024-04-25T10:01:28.640292+0000","last_became_peered":"2024-04-25T10:01:28.640292+0000","last_unstale":"2024-04-25T10:01:28.643746+0000","last_undegraded":"2024-04-25T10:01:28.643746+0000","last_fullsized":"2024-04-25T10:01:28.643746+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:31.176590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634251+0000","last_change":"2024-04-25T10:01:28.634251+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634251+0000","last_undegraded":"2024-04-25T10:01:28.634251+0000","last_fullsized":"2024-04-25T10:01:28.634251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648139+0000","last_change":"2024-04-25T10:01:28.641737+0000","last_active":"2024-04-25T10:01:28.648139+0000","last_peered":"2024-04-25T10:01:28.648139+0000","last_clean":"2024-04-25T10:01:28.648139+0000","last_became_active":"2024-04-25T10:01:28.641451+0000","last_became_peered":"2024-04-25T10:01:28.641451+0000","last_unstale":"2024-04-25T10:01:28.648139+0000","last_undegraded":"2024-04-25T10:01:28.648139+0000","last_fullsized":"2024-04-25T10:01:28.648139+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:02.288325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634539+0000","last_change":"2024-04-25T10:01:28.634539+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634539+0000","last_undegraded":"2024-04-25T10:01:28.634539+0000","last_fullsized":"2024-04-25T10:01:28.634539+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645115+0000","last_change":"2024-04-25T10:01:28.644516+0000","last_active":"2024-04-25T10:01:28.645115+0000","last_peered":"2024-04-25T10:01:28.645115+0000","last_clean":"2024-04-25T10:01:28.645115+0000","last_became_active":"2024-04-25T10:01:28.644155+0000","last_became_peered":"2024-04-25T10:01:28.644155+0000","last_unstale":"2024-04-25T10:01:28.645115+0000","last_undegraded":"2024-04-25T10:01:28.645115+0000","last_fullsized":"2024-04-25T10:01:28.645115+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:13.085301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.636198+0000","last_change":"2024-04-25T10:01:28.636198+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.636198+0000","last_undegraded":"2024-04-25T10:01:28.636198+0000","last_fullsized":"2024-04-25T10:01:28.636198+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.641997+0000","last_change":"2024-04-25T10:01:28.638744+0000","last_active":"2024-04-25T10:01:28.641997+0000","last_peered":"2024-04-25T10:01:28.641997+0000","last_clean":"2024-04-25T10:01:28.641997+0000","last_became_active":"2024-04-25T10:01:28.638312+0000","last_became_peered":"2024-04-25T10:01:28.638312+0000","last_unstale":"2024-04-25T10:01:28.641997+0000","last_undegraded":"2024-04-25T10:01:28.641997+0000","last_fullsized":"2024-04-25T10:01:28.641997+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:41.741278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632131+0000","last_change":"2024-04-25T10:01:28.632131+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632131+0000","last_undegraded":"2024-04-25T10:01:28.632131+0000","last_fullsized":"2024-04-25T10:01:28.632131+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648947+0000","last_change":"2024-04-25T10:01:28.647785+0000","last_active":"2024-04-25T10:01:28.648947+0000","last_peered":"2024-04-25T10:01:28.648947+0000","last_clean":"2024-04-25T10:01:28.648947+0000","last_became_active":"2024-04-25T10:01:28.647474+0000","last_became_peered":"2024-04-25T10:01:28.647474+0000","last_unstale":"2024-04-25T10:01:28.648947+0000","last_undegraded":"2024-04-25T10:01:28.648947+0000","last_fullsized":"2024-04-25T10:01:28.648947+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:21.347290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629400+0000","last_change":"2024-04-25T10:01:28.629400+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629400+0000","last_undegraded":"2024-04-25T10:01:28.629400+0000","last_fullsized":"2024-04-25T10:01:28.629400+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645364+0000","last_change":"2024-04-25T10:01:28.642073+0000","last_active":"2024-04-25T10:01:28.645364+0000","last_peered":"2024-04-25T10:01:28.645364+0000","last_clean":"2024-04-25T10:01:28.645364+0000","last_became_active":"2024-04-25T10:01:28.641446+0000","last_became_peered":"2024-04-25T10:01:28.641446+0000","last_unstale":"2024-04-25T10:01:28.645364+0000","last_undegraded":"2024-04-25T10:01:28.645364+0000","last_fullsized":"2024-04-25T10:01:28.645364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:09.409126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629182+0000","last_change":"2024-04-25T10:01:28.629182+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629182+0000","last_undegraded":"2024-04-25T10:01:28.629182+0000","last_fullsized":"2024-04-25T10:01:28.629182+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645618+0000","last_change":"2024-04-25T10:01:28.643918+0000","last_active":"2024-04-25T10:01:28.645618+0000","last_peered":"2024-04-25T10:01:28.645618+0000","last_clean":"2024-04-25T10:01:28.645618+0000","last_became_active":"2024-04-25T10:01:28.643575+0000","last_became_peered":"2024-04-25T10:01:28.643575+0000","last_unstale":"2024-04-25T10:01:28.645618+0000","last_undegraded":"2024-04-25T10:01:28.645618+0000","last_fullsized":"2024-04-25T10:01:28.645618+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:10.820229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634919+0000","last_change":"2024-04-25T10:01:28.634919+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634919+0000","last_undegraded":"2024-04-25T10:01:28.634919+0000","last_fullsized":"2024-04-25T10:01:28.634919+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.637809+0000","last_change":"2024-04-25T10:01:28.633829+0000","last_active":"2024-04-25T10:01:28.637809+0000","last_peered":"2024-04-25T10:01:28.637809+0000","last_clean":"2024-04-25T10:01:28.637809+0000","last_became_active":"2024-04-25T10:01:28.632994+0000","last_became_peered":"2024-04-25T10:01:28.632994+0000","last_unstale":"2024-04-25T10:01:28.637809+0000","last_undegraded":"2024-04-25T10:01:28.637809+0000","last_fullsized":"2024-04-25T10:01:28.637809+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:57.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630716+0000","last_change":"2024-04-25T10:01:28.630716+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630716+0000","last_undegraded":"2024-04-25T10:01:28.630716+0000","last_fullsized":"2024-04-25T10:01:28.630716+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.649655+0000","last_change":"2024-04-25T10:01:28.649336+0000","last_active":"2024-04-25T10:01:28.649655+0000","last_peered":"2024-04-25T10:01:28.649655+0000","last_clean":"2024-04-25T10:01:28.649655+0000","last_became_active":"2024-04-25T10:01:28.649067+0000","last_became_peered":"2024-04-25T10:01:28.649067+0000","last_unstale":"2024-04-25T10:01:28.649655+0000","last_undegraded":"2024-04-25T10:01:28.649655+0000","last_fullsized":"2024-04-25T10:01:28.649655+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:54.846448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631752+0000","last_change":"2024-04-25T10:01:28.631752+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631752+0000","last_undegraded":"2024-04-25T10:01:28.631752+0000","last_fullsized":"2024-04-25T10:01:28.631752+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643332+0000","last_change":"2024-04-25T10:01:28.637991+0000","last_active":"2024-04-25T10:01:28.643332+0000","last_peered":"2024-04-25T10:01:28.643332+0000","last_clean":"2024-04-25T10:01:28.643332+0000","last_became_active":"2024-04-25T10:01:28.637710+0000","last_became_peered":"2024-04-25T10:01:28.637710+0000","last_unstale":"2024-04-25T10:01:28.643332+0000","last_undegraded":"2024-04-25T10:01:28.643332+0000","last_fullsized":"2024-04-25T10:01:28.643332+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:32.105344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630788+0000","last_change":"2024-04-25T10:01:28.630788+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630788+0000","last_undegraded":"2024-04-25T10:01:28.630788+0000","last_fullsized":"2024-04-25T10:01:28.630788+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.639587+0000","last_change":"2024-04-25T10:01:28.637799+0000","last_active":"2024-04-25T10:01:28.639587+0000","last_peered":"2024-04-25T10:01:28.639587+0000","last_clean":"2024-04-25T10:01:28.639587+0000","last_became_active":"2024-04-25T10:01:28.637491+0000","last_became_peered":"2024-04-25T10:01:28.637491+0000","last_unstale":"2024-04-25T10:01:28.639587+0000","last_undegraded":"2024-04-25T10:01:28.639587+0000","last_fullsized":"2024-04-25T10:01:28.639587+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:10.916547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628958+0000","last_change":"2024-04-25T10:01:28.628958+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628958+0000","last_undegraded":"2024-04-25T10:01:28.628958+0000","last_fullsized":"2024-04-25T10:01:28.628958+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.647431+0000","last_change":"2024-04-25T10:01:28.645758+0000","last_active":"2024-04-25T10:01:28.647431+0000","last_peered":"2024-04-25T10:01:28.647431+0000","last_clean":"2024-04-25T10:01:28.647431+0000","last_became_active":"2024-04-25T10:01:28.645393+0000","last_became_peered":"2024-04-25T10:01:28.645393+0000","last_unstale":"2024-04-25T10:01:28.647431+0000","last_undegraded":"2024-04-25T10:01:28.647431+0000","last_fullsized":"2024-04-25T10:01:28.647431+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.056666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631251+0000","last_change":"2024-04-25T10:01:28.631251+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631251+0000","last_undegraded":"2024-04-25T10:01:28.631251+0000","last_fullsized":"2024-04-25T10:01:28.631251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.638771+0000","last_change":"2024-04-25T10:01:28.636229+0000","last_active":"2024-04-25T10:01:28.638771+0000","last_peered":"2024-04-25T10:01:28.638771+0000","last_clean":"2024-04-25T10:01:28.638771+0000","last_became_active":"2024-04-25T10:01:28.635741+0000","last_became_peered":"2024-04-25T10:01:28.635741+0000","last_unstale":"2024-04-25T10:01:28.638771+0000","last_undegraded":"2024-04-25T10:01:28.638771+0000","last_fullsized":"2024-04-25T10:01:28.638771+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:28.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":388,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.631752+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:01:28.631752+0000","last_peered":"2024-04-25T10:01:28.631752+0000","last_clean":"2024-04-25T10:01:28.631752+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:01:28.631752+0000","last_undegraded":"2024-04-25T10:01:28.631752+0000","last_fullsized":"2024-04-25T10:01:28.631752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29156,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342684,"statfs":{"total":96636764160,"available":96606908416,"internally_reserved":0,"allocated":2183168,"data_stored":1329872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46624,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325216,"statfs":{"total":96636764160,"available":96589021184,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54928,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316912,"statfs":{"total":96636764160,"available":96580517888,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54944,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316896,"statfs":{"total":96636764160,"available":96580501504,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50772,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321068,"statfs":{"total":96636764160,"available":96584773632,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50784,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321056,"statfs":{"total":96636764160,"available":96584761344,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51348,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320492,"statfs":{"total":96636764160,"available":96584183808,"internally_reserved":0,"allocated":2170880,"data_stored":1323213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:32.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.792+0000 7fa7b125d640 1 Processor -- start 2024-04-25T10:01:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.792+0000 7fa7b125d640 1 -- start start 2024-04-25T10:01:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7b125d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa7ac15da70 0x7fa7ac161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7b125d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa7ac0b9730 0x7fa7ac0ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7b125d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac161b80 0x7fa7ac0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7b125d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac1625e0 con 0x7fa7ac15da70 2024-04-25T10:01:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7b125d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac162020 con 0x7fa7ac0b9730 2024-04-25T10:01:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7b125d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac162300 con 0x7fa7ac161b80 2024-04-25T10:01:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7aad9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa7ac15da70 0x7fa7ac161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7aad9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa7ac15da70 0x7fa7ac161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33534/0 (socket says 172.21.15.19:33534) 2024-04-25T10:01:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7aad9d640 1 -- 172.21.15.19:0/3701049665 learned_addr learned my addr 172.21.15.19:0/3701049665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7a3fff640 1 --2- 172.21.15.19:0/3701049665 >> v2:172.21.15.112:3300/0 conn(0x7fa7ac0b9730 0x7fa7ac0ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7ab59e640 1 --2- 172.21.15.19:0/3701049665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac161b80 0x7fa7ac0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7aad9d640 1 -- 172.21.15.19:0/3701049665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac161b80 msgr2=0x7fa7ac0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7aad9d640 1 --2- 172.21.15.19:0/3701049665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac161b80 0x7fa7ac0ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7aad9d640 1 -- 172.21.15.19:0/3701049665 >> v2:172.21.15.112:3300/0 conn(0x7fa7ac0b9730 msgr2=0x7fa7ac0ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7aad9d640 1 --2- 172.21.15.19:0/3701049665 >> v2:172.21.15.112:3300/0 conn(0x7fa7ac0b9730 0x7fa7ac0ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.793+0000 7fa7aad9d640 1 -- 172.21.15.19:0/3701049665 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac0d1af0 con 0x7fa7ac15da70 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.794+0000 7fa7aad9d640 1 --2- 172.21.15.19:0/3701049665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac15da70 0x7fa7ac161640 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0c72a0 tx=0x7fa79c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae821827040caa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.794+0000 7fa7aa59c640 1 -- 172.21.15.19:0/3701049665 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c037070 con 0x7fa7ac15da70 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.794+0000 7fa7aa59c640 1 -- 172.21.15.19:0/3701049665 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c032440 con 0x7fa7ac15da70 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.794+0000 7fa7aa59c640 1 -- 172.21.15.19:0/3701049665 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c032810 con 0x7fa7ac15da70 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.795+0000 7fa7b125d640 1 -- 172.21.15.19:0/3701049665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac15da70 msgr2=0x7fa7ac161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.795+0000 7fa7b125d640 1 --2- 172.21.15.19:0/3701049665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac15da70 0x7fa7ac161640 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0c72a0 tx=0x7fa79c02a140 comp rx=0 tx=0).stop 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.795+0000 7fa7b125d640 1 -- 172.21.15.19:0/3701049665 shutdown_connections 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.795+0000 7fa7b125d640 1 --2- 172.21.15.19:0/3701049665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac161b80 0x7fa7ac0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.795+0000 7fa7b125d640 1 --2- 172.21.15.19:0/3701049665 >> v2:172.21.15.112:3300/0 conn(0x7fa7ac0b9730 0x7fa7ac0ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.795+0000 7fa7b125d640 1 --2- 172.21.15.19:0/3701049665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac15da70 0x7fa7ac161640 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.795+0000 7fa7b125d640 1 -- 172.21.15.19:0/3701049665 >> 172.21.15.19:0/3701049665 conn(0x7fa7ac061a20 msgr2=0x7fa7ac061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.795+0000 7fa7b125d640 1 -- 172.21.15.19:0/3701049665 shutdown_connections 2024-04-25T10:01:32.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.795+0000 7fa7b125d640 1 -- 172.21.15.19:0/3701049665 wait complete. 2024-04-25T10:01:32.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.796+0000 7fa7b125d640 1 Processor -- start 2024-04-25T10:01:32.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.796+0000 7fa7b125d640 1 -- start start 2024-04-25T10:01:32.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7b125d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac15da70 0x7fa7ac15dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7b125d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7ac161b80 0x7fa7ac148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7b125d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac1587d0 0x7fa7ac0b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7b125d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac06c120 con 0x7fa7ac1587d0 2024-04-25T10:01:32.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7b125d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac15de50 con 0x7fa7ac161b80 2024-04-25T10:01:32.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7b125d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac06c2a0 con 0x7fa7ac15da70 2024-04-25T10:01:32.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7ab59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac1587d0 0x7fa7ac0b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7ab59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac1587d0 0x7fa7ac0b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33538/0 (socket says 172.21.15.19:33538) 2024-04-25T10:01:32.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7ab59e640 1 -- 172.21.15.19:0/2680199090 learned_addr learned my addr 172.21.15.19:0/2680199090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:32.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7aad9d640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac15da70 0x7fa7ac15dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7a3fff640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7ac161b80 0x7fa7ac148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7ab59e640 1 -- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac15da70 msgr2=0x7fa7ac15dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7ab59e640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac15da70 0x7fa7ac15dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7ab59e640 1 -- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7ac161b80 msgr2=0x7fa7ac148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7ab59e640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7ac161b80 0x7fa7ac148c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.797+0000 7fa7ab59e640 1 -- 172.21.15.19:0/2680199090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac149190 con 0x7fa7ac1587d0 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.798+0000 7fa7ab59e640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac1587d0 0x7fa7ac0b7800 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4007d40 tx=0x7fa7a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.798+0000 7fa7a37fe640 1 -- 172.21.15.19:0/2680199090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4014420 con 0x7fa7ac1587d0 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.798+0000 7fa7a37fe640 1 -- 172.21.15.19:0/2680199090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a4019070 con 0x7fa7ac1587d0 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.798+0000 7fa7b125d640 1 -- 172.21.15.19:0/2680199090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ac15e660 con 0x7fa7ac1587d0 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.798+0000 7fa7a37fe640 1 -- 172.21.15.19:0/2680199090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4014820 con 0x7fa7ac1587d0 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.798+0000 7fa7b125d640 1 -- 172.21.15.19:0/2680199090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac15ec20 con 0x7fa7ac1587d0 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.799+0000 7fa7b125d640 1 -- 172.21.15.19:0/2680199090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ac073990 con 0x7fa7ac1587d0 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.799+0000 7fa7a37fe640 1 -- 172.21.15.19:0/2680199090 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7a40149c0 con 0x7fa7ac1587d0 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.800+0000 7fa7a37fe640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa78c06b4d0 0x7fa78c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.800+0000 7fa7a37fe640 1 -- 172.21.15.19:0/2680199090 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa7a408a9d0 con 0x7fa7ac1587d0 2024-04-25T10:01:32.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.800+0000 7fa7aad9d640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa78c06b4d0 0x7fa78c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.800+0000 7fa7aad9d640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa78c06b4d0 0x7fa78c06d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa79c034000 tx=0x7fa79c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.804+0000 7fa7a37fe640 1 -- 172.21.15.19:0/2680199090 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7a4059070 con 0x7fa7ac1587d0 2024-04-25T10:01:32.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.982+0000 7fa7b125d640 1 -- 172.21.15.19:0/2680199090 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7ac0d3970 con 0x7fa78c06b4d0 2024-04-25T10:01:32.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.988+0000 7fa7a37fe640 1 -- 172.21.15.19:0/2680199090 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247073 (secure 0 0 0) 0x7fa7ac0d3970 con 0x7fa78c06b4d0 2024-04-25T10:01:32.998 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:33.000 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:01:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 -- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa78c06b4d0 msgr2=0x7fa78c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa78c06b4d0 0x7fa78c06d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa79c034000 tx=0x7fa79c02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 -- 172.21.15.19:0/2680199090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac1587d0 msgr2=0x7fa7ac0b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac1587d0 0x7fa7ac0b7800 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4007d40 tx=0x7fa7a400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 -- 172.21.15.19:0/2680199090 shutdown_connections 2024-04-25T10:01:33.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa78c06b4d0 0x7fa78c06d990 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac1587d0 0x7fa7ac0b7800 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7ac161b80 0x7fa7ac148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 --2- 172.21.15.19:0/2680199090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac15da70 0x7fa7ac15dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 -- 172.21.15.19:0/2680199090 >> 172.21.15.19:0/2680199090 conn(0x7fa7ac061a20 msgr2=0x7fa7ac0b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 -- 172.21.15.19:0/2680199090 shutdown_connections 2024-04-25T10:01:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:32.991+0000 7fa7b125d640 1 -- 172.21.15.19:0/2680199090 wait complete. 2024-04-25T10:01:33.008 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-04-25T10:01:31.027424+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385164,"kb_used_data":13528,"kb_used_omap":12,"kb_used_meta":371507,"kb_avail":754589556,"statfs":{"total":773094113280,"available":772699705344,"internally_reserved":0,"allocated":13852672,"data_stored":7070132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":380423730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001255"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.639822+0000","last_change":"2024-04-25T10:01:28.639822+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.639822+0000","last_undegraded":"2024-04-25T10:01:28.639822+0000","last_fullsized":"2024-04-25T10:01:28.639822+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633777+0000","last_change":"2024-04-25T10:01:28.633777+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633777+0000","last_undegraded":"2024-04-25T10:01:28.633777+0000","last_fullsized":"2024-04-25T10:01:28.633777+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631663+0000","last_change":"2024-04-25T10:01:28.631663+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631663+0000","last_undegraded":"2024-04-25T10:01:28.631663+0000","last_fullsized":"2024-04-25T10:01:28.631663+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633655+0000","last_change":"2024-04-25T10:01:28.633655+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633655+0000","last_undegraded":"2024-04-25T10:01:28.633655+0000","last_fullsized":"2024-04-25T10:01:28.633655+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631604+0000","last_change":"2024-04-25T10:01:28.631604+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631604+0000","last_undegraded":"2024-04-25T10:01:28.631604+0000","last_fullsized":"2024-04-25T10:01:28.631604+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632464+0000","last_change":"2024-04-25T10:01:28.632464+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632464+0000","last_undegraded":"2024-04-25T10:01:28.632464+0000","last_fullsized":"2024-04-25T10:01:28.632464+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630901+0000","last_change":"2024-04-25T10:01:28.630901+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630901+0000","last_undegraded":"2024-04-25T10:01:28.630901+0000","last_fullsized":"2024-04-25T10:01:28.630901+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633293+0000","last_change":"2024-04-25T10:01:28.633293+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633293+0000","last_undegraded":"2024-04-25T10:01:28.633293+0000","last_fullsized":"2024-04-25T10:01:28.633293+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632483+0000","last_change":"2024-04-25T10:01:28.632483+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632483+0000","last_undegraded":"2024-04-25T10:01:28.632483+0000","last_fullsized":"2024-04-25T10:01:28.632483+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634762+0000","last_change":"2024-04-25T10:01:28.634762+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634762+0000","last_undegraded":"2024-04-25T10:01:28.634762+0000","last_fullsized":"2024-04-25T10:01:28.634762+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634423+0000","last_change":"2024-04-25T10:01:28.634423+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634423+0000","last_undegraded":"2024-04-25T10:01:28.634423+0000","last_fullsized":"2024-04-25T10:01:28.634423+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633781+0000","last_change":"2024-04-25T10:01:28.633781+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633781+0000","last_undegraded":"2024-04-25T10:01:28.633781+0000","last_fullsized":"2024-04-25T10:01:28.633781+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631273+0000","last_change":"2024-04-25T10:01:28.631273+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631273+0000","last_undegraded":"2024-04-25T10:01:28.631273+0000","last_fullsized":"2024-04-25T10:01:28.631273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633250+0000","last_change":"2024-04-25T10:01:28.633250+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633250+0000","last_undegraded":"2024-04-25T10:01:28.633250+0000","last_fullsized":"2024-04-25T10:01:28.633250+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632131+0000","last_change":"2024-04-25T10:01:28.632131+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632131+0000","last_undegraded":"2024-04-25T10:01:28.632131+0000","last_fullsized":"2024-04-25T10:01:28.632131+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631442+0000","last_change":"2024-04-25T10:01:28.631442+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631442+0000","last_undegraded":"2024-04-25T10:01:28.631442+0000","last_fullsized":"2024-04-25T10:01:28.631442+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631831+0000","last_change":"2024-04-25T10:01:28.631831+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631831+0000","last_undegraded":"2024-04-25T10:01:28.631831+0000","last_fullsized":"2024-04-25T10:01:28.631831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635356+0000","last_change":"2024-04-25T10:01:28.635356+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635356+0000","last_undegraded":"2024-04-25T10:01:28.635356+0000","last_fullsized":"2024-04-25T10:01:28.635356+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629531+0000","last_change":"2024-04-25T10:01:28.629531+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629531+0000","last_undegraded":"2024-04-25T10:01:28.629531+0000","last_fullsized":"2024-04-25T10:01:28.629531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633354+0000","last_change":"2024-04-25T10:01:28.633354+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633354+0000","last_undegraded":"2024-04-25T10:01:28.633354+0000","last_fullsized":"2024-04-25T10:01:28.633354+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634748+0000","last_change":"2024-04-25T10:01:28.634748+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634748+0000","last_undegraded":"2024-04-25T10:01:28.634748+0000","last_fullsized":"2024-04-25T10:01:28.634748+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631665+0000","last_change":"2024-04-25T10:01:28.631665+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631665+0000","last_undegraded":"2024-04-25T10:01:28.631665+0000","last_fullsized":"2024-04-25T10:01:28.631665+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634367+0000","last_change":"2024-04-25T10:01:28.634367+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634367+0000","last_undegraded":"2024-04-25T10:01:28.634367+0000","last_fullsized":"2024-04-25T10:01:28.634367+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631023+0000","last_change":"2024-04-25T10:01:28.631023+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631023+0000","last_undegraded":"2024-04-25T10:01:28.631023+0000","last_fullsized":"2024-04-25T10:01:28.631023+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631167+0000","last_change":"2024-04-25T10:01:28.631167+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631167+0000","last_undegraded":"2024-04-25T10:01:28.631167+0000","last_fullsized":"2024-04-25T10:01:28.631167+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629895+0000","last_change":"2024-04-25T10:01:28.629895+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629895+0000","last_undegraded":"2024-04-25T10:01:28.629895+0000","last_fullsized":"2024-04-25T10:01:28.629895+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628825+0000","last_change":"2024-04-25T10:01:28.628825+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628825+0000","last_undegraded":"2024-04-25T10:01:28.628825+0000","last_fullsized":"2024-04-25T10:01:28.628825+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.637120+0000","last_change":"2024-04-25T10:01:28.637120+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.637120+0000","last_undegraded":"2024-04-25T10:01:28.637120+0000","last_fullsized":"2024-04-25T10:01:28.637120+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632165+0000","last_change":"2024-04-25T10:01:28.632165+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632165+0000","last_undegraded":"2024-04-25T10:01:28.632165+0000","last_fullsized":"2024-04-25T10:01:28.632165+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629573+0000","last_change":"2024-04-25T10:01:28.629573+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629573+0000","last_undegraded":"2024-04-25T10:01:28.629573+0000","last_fullsized":"2024-04-25T10:01:28.629573+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632296+0000","last_change":"2024-04-25T10:01:28.632296+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632296+0000","last_undegraded":"2024-04-25T10:01:28.632296+0000","last_fullsized":"2024-04-25T10:01:28.632296+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634707+0000","last_change":"2024-04-25T10:01:28.634707+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634707+0000","last_undegraded":"2024-04-25T10:01:28.634707+0000","last_fullsized":"2024-04-25T10:01:28.634707+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632907+0000","last_change":"2024-04-25T10:01:28.632907+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632907+0000","last_undegraded":"2024-04-25T10:01:28.632907+0000","last_fullsized":"2024-04-25T10:01:28.632907+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645733+0000","last_change":"2024-04-25T10:01:28.644943+0000","last_active":"2024-04-25T10:01:28.645733+0000","last_peered":"2024-04-25T10:01:28.645733+0000","last_clean":"2024-04-25T10:01:28.645733+0000","last_became_active":"2024-04-25T10:01:28.644689+0000","last_became_peered":"2024-04-25T10:01:28.644689+0000","last_unstale":"2024-04-25T10:01:28.645733+0000","last_undegraded":"2024-04-25T10:01:28.645733+0000","last_fullsized":"2024-04-25T10:01:28.645733+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:48.121372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631082+0000","last_change":"2024-04-25T10:01:28.631082+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631082+0000","last_undegraded":"2024-04-25T10:01:28.631082+0000","last_fullsized":"2024-04-25T10:01:28.631082+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.647323+0000","last_change":"2024-04-25T10:01:28.645571+0000","last_active":"2024-04-25T10:01:28.647323+0000","last_peered":"2024-04-25T10:01:28.647323+0000","last_clean":"2024-04-25T10:01:28.647323+0000","last_became_active":"2024-04-25T10:01:28.645283+0000","last_became_peered":"2024-04-25T10:01:28.645283+0000","last_unstale":"2024-04-25T10:01:28.647323+0000","last_undegraded":"2024-04-25T10:01:28.647323+0000","last_fullsized":"2024-04-25T10:01:28.647323+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:11.425437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633628+0000","last_change":"2024-04-25T10:01:28.633628+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633628+0000","last_undegraded":"2024-04-25T10:01:28.633628+0000","last_fullsized":"2024-04-25T10:01:28.633628+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643814+0000","last_change":"2024-04-25T10:01:28.640421+0000","last_active":"2024-04-25T10:01:28.643814+0000","last_peered":"2024-04-25T10:01:28.643814+0000","last_clean":"2024-04-25T10:01:28.643814+0000","last_became_active":"2024-04-25T10:01:28.639536+0000","last_became_peered":"2024-04-25T10:01:28.639536+0000","last_unstale":"2024-04-25T10:01:28.643814+0000","last_undegraded":"2024-04-25T10:01:28.643814+0000","last_fullsized":"2024-04-25T10:01:28.643814+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.056666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.638331+0000","last_change":"2024-04-25T10:01:28.638331+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.638331+0000","last_undegraded":"2024-04-25T10:01:28.638331+0000","last_fullsized":"2024-04-25T10:01:28.638331+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643382+0000","last_change":"2024-04-25T10:01:28.642064+0000","last_active":"2024-04-25T10:01:28.643382+0000","last_peered":"2024-04-25T10:01:28.643382+0000","last_clean":"2024-04-25T10:01:28.643382+0000","last_became_active":"2024-04-25T10:01:28.641767+0000","last_became_peered":"2024-04-25T10:01:28.641767+0000","last_unstale":"2024-04-25T10:01:28.643382+0000","last_undegraded":"2024-04-25T10:01:28.643382+0000","last_fullsized":"2024-04-25T10:01:28.643382+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:28.715123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632603+0000","last_change":"2024-04-25T10:01:28.632603+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632603+0000","last_undegraded":"2024-04-25T10:01:28.632603+0000","last_fullsized":"2024-04-25T10:01:28.632603+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645590+0000","last_change":"2024-04-25T10:01:28.643613+0000","last_active":"2024-04-25T10:01:28.645590+0000","last_peered":"2024-04-25T10:01:28.645590+0000","last_clean":"2024-04-25T10:01:28.645590+0000","last_became_active":"2024-04-25T10:01:28.643326+0000","last_became_peered":"2024-04-25T10:01:28.643326+0000","last_unstale":"2024-04-25T10:01:28.645590+0000","last_undegraded":"2024-04-25T10:01:28.645590+0000","last_fullsized":"2024-04-25T10:01:28.645590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:47.363454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632282+0000","last_change":"2024-04-25T10:01:28.632282+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632282+0000","last_undegraded":"2024-04-25T10:01:28.632282+0000","last_fullsized":"2024-04-25T10:01:28.632282+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648286+0000","last_change":"2024-04-25T10:01:28.647013+0000","last_active":"2024-04-25T10:01:28.648286+0000","last_peered":"2024-04-25T10:01:28.648286+0000","last_clean":"2024-04-25T10:01:28.648286+0000","last_became_active":"2024-04-25T10:01:28.646667+0000","last_became_peered":"2024-04-25T10:01:28.646667+0000","last_unstale":"2024-04-25T10:01:28.648286+0000","last_undegraded":"2024-04-25T10:01:28.648286+0000","last_fullsized":"2024-04-25T10:01:28.648286+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:28.715123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634967+0000","last_change":"2024-04-25T10:01:28.634967+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634967+0000","last_undegraded":"2024-04-25T10:01:28.634967+0000","last_fullsized":"2024-04-25T10:01:28.634967+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643539+0000","last_change":"2024-04-25T10:01:28.639325+0000","last_active":"2024-04-25T10:01:28.643539+0000","last_peered":"2024-04-25T10:01:28.643539+0000","last_clean":"2024-04-25T10:01:28.643539+0000","last_became_active":"2024-04-25T10:01:28.639055+0000","last_became_peered":"2024-04-25T10:01:28.639055+0000","last_unstale":"2024-04-25T10:01:28.643539+0000","last_undegraded":"2024-04-25T10:01:28.643539+0000","last_fullsized":"2024-04-25T10:01:28.643539+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:09.888995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629531+0000","last_change":"2024-04-25T10:01:28.629531+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629531+0000","last_undegraded":"2024-04-25T10:01:28.629531+0000","last_fullsized":"2024-04-25T10:01:28.629531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.640085+0000","last_change":"2024-04-25T10:01:28.639418+0000","last_active":"2024-04-25T10:01:28.640085+0000","last_peered":"2024-04-25T10:01:28.640085+0000","last_clean":"2024-04-25T10:01:28.640085+0000","last_became_active":"2024-04-25T10:01:28.639134+0000","last_became_peered":"2024-04-25T10:01:28.639134+0000","last_unstale":"2024-04-25T10:01:28.640085+0000","last_undegraded":"2024-04-25T10:01:28.640085+0000","last_fullsized":"2024-04-25T10:01:28.640085+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:09.409126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631028+0000","last_change":"2024-04-25T10:01:28.631028+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631028+0000","last_undegraded":"2024-04-25T10:01:28.631028+0000","last_fullsized":"2024-04-25T10:01:28.631028+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.641745+0000","last_change":"2024-04-25T10:01:28.640498+0000","last_active":"2024-04-25T10:01:28.641745+0000","last_peered":"2024-04-25T10:01:28.641745+0000","last_clean":"2024-04-25T10:01:28.641745+0000","last_became_active":"2024-04-25T10:01:28.640186+0000","last_became_peered":"2024-04-25T10:01:28.640186+0000","last_unstale":"2024-04-25T10:01:28.641745+0000","last_undegraded":"2024-04-25T10:01:28.641745+0000","last_fullsized":"2024-04-25T10:01:28.641745+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:23.471932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632233+0000","last_change":"2024-04-25T10:01:28.632233+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632233+0000","last_undegraded":"2024-04-25T10:01:28.632233+0000","last_fullsized":"2024-04-25T10:01:28.632233+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648909+0000","last_change":"2024-04-25T10:01:28.647926+0000","last_active":"2024-04-25T10:01:28.648909+0000","last_peered":"2024-04-25T10:01:28.648909+0000","last_clean":"2024-04-25T10:01:28.648909+0000","last_became_active":"2024-04-25T10:01:28.646963+0000","last_became_peered":"2024-04-25T10:01:28.646963+0000","last_unstale":"2024-04-25T10:01:28.648909+0000","last_undegraded":"2024-04-25T10:01:28.648909+0000","last_fullsized":"2024-04-25T10:01:28.648909+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:56.486037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629097+0000","last_change":"2024-04-25T10:01:28.629097+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629097+0000","last_undegraded":"2024-04-25T10:01:28.629097+0000","last_fullsized":"2024-04-25T10:01:28.629097+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643792+0000","last_change":"2024-04-25T10:01:28.638314+0000","last_active":"2024-04-25T10:01:28.643792+0000","last_peered":"2024-04-25T10:01:28.643792+0000","last_clean":"2024-04-25T10:01:28.643792+0000","last_became_active":"2024-04-25T10:01:28.637960+0000","last_became_peered":"2024-04-25T10:01:28.637960+0000","last_unstale":"2024-04-25T10:01:28.643792+0000","last_undegraded":"2024-04-25T10:01:28.643792+0000","last_fullsized":"2024-04-25T10:01:28.643792+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.356021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628748+0000","last_change":"2024-04-25T10:01:28.628748+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628748+0000","last_undegraded":"2024-04-25T10:01:28.628748+0000","last_fullsized":"2024-04-25T10:01:28.628748+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643363+0000","last_change":"2024-04-25T10:01:28.639720+0000","last_active":"2024-04-25T10:01:28.643363+0000","last_peered":"2024-04-25T10:01:28.643363+0000","last_clean":"2024-04-25T10:01:28.643363+0000","last_became_active":"2024-04-25T10:01:28.638956+0000","last_became_peered":"2024-04-25T10:01:28.638956+0000","last_unstale":"2024-04-25T10:01:28.643363+0000","last_undegraded":"2024-04-25T10:01:28.643363+0000","last_fullsized":"2024-04-25T10:01:28.643363+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:11.425437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635339+0000","last_change":"2024-04-25T10:01:28.635339+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635339+0000","last_undegraded":"2024-04-25T10:01:28.635339+0000","last_fullsized":"2024-04-25T10:01:28.635339+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648870+0000","last_change":"2024-04-25T10:01:28.644548+0000","last_active":"2024-04-25T10:01:28.648870+0000","last_peered":"2024-04-25T10:01:28.648870+0000","last_clean":"2024-04-25T10:01:28.648870+0000","last_became_active":"2024-04-25T10:01:28.643068+0000","last_became_peered":"2024-04-25T10:01:28.643068+0000","last_unstale":"2024-04-25T10:01:28.648870+0000","last_undegraded":"2024-04-25T10:01:28.648870+0000","last_fullsized":"2024-04-25T10:01:28.648870+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:28.602317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628719+0000","last_change":"2024-04-25T10:01:28.628719+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628719+0000","last_undegraded":"2024-04-25T10:01:28.628719+0000","last_fullsized":"2024-04-25T10:01:28.628719+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.642739+0000","last_change":"2024-04-25T10:01:28.641313+0000","last_active":"2024-04-25T10:01:28.642739+0000","last_peered":"2024-04-25T10:01:28.642739+0000","last_clean":"2024-04-25T10:01:28.642739+0000","last_became_active":"2024-04-25T10:01:28.641020+0000","last_became_peered":"2024-04-25T10:01:28.641020+0000","last_unstale":"2024-04-25T10:01:28.642739+0000","last_undegraded":"2024-04-25T10:01:28.642739+0000","last_fullsized":"2024-04-25T10:01:28.642739+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:47.363454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632145+0000","last_change":"2024-04-25T10:01:28.632145+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632145+0000","last_undegraded":"2024-04-25T10:01:28.632145+0000","last_fullsized":"2024-04-25T10:01:28.632145+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643497+0000","last_change":"2024-04-25T10:01:28.639344+0000","last_active":"2024-04-25T10:01:28.643497+0000","last_peered":"2024-04-25T10:01:28.643497+0000","last_clean":"2024-04-25T10:01:28.643497+0000","last_became_active":"2024-04-25T10:01:28.639079+0000","last_became_peered":"2024-04-25T10:01:28.639079+0000","last_unstale":"2024-04-25T10:01:28.643497+0000","last_undegraded":"2024-04-25T10:01:28.643497+0000","last_fullsized":"2024-04-25T10:01:28.643497+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:13.447365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632144+0000","last_change":"2024-04-25T10:01:28.632144+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632144+0000","last_undegraded":"2024-04-25T10:01:28.632144+0000","last_fullsized":"2024-04-25T10:01:28.632144+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.644323+0000","last_change":"2024-04-25T10:01:28.642872+0000","last_active":"2024-04-25T10:01:28.644323+0000","last_peered":"2024-04-25T10:01:28.644323+0000","last_clean":"2024-04-25T10:01:28.644323+0000","last_became_active":"2024-04-25T10:01:28.642250+0000","last_became_peered":"2024-04-25T10:01:28.642250+0000","last_unstale":"2024-04-25T10:01:28.644323+0000","last_undegraded":"2024-04-25T10:01:28.644323+0000","last_fullsized":"2024-04-25T10:01:28.644323+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:09.710270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630415+0000","last_change":"2024-04-25T10:01:28.630415+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630415+0000","last_undegraded":"2024-04-25T10:01:28.630415+0000","last_fullsized":"2024-04-25T10:01:28.630415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.644964+0000","last_change":"2024-04-25T10:01:28.643788+0000","last_active":"2024-04-25T10:01:28.644964+0000","last_peered":"2024-04-25T10:01:28.644964+0000","last_clean":"2024-04-25T10:01:28.644964+0000","last_became_active":"2024-04-25T10:01:28.643521+0000","last_became_peered":"2024-04-25T10:01:28.643521+0000","last_unstale":"2024-04-25T10:01:28.644964+0000","last_undegraded":"2024-04-25T10:01:28.644964+0000","last_fullsized":"2024-04-25T10:01:28.644964+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:10.820229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628642+0000","last_change":"2024-04-25T10:01:28.628642+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628642+0000","last_undegraded":"2024-04-25T10:01:28.628642+0000","last_fullsized":"2024-04-25T10:01:28.628642+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643193+0000","last_change":"2024-04-25T10:01:28.640275+0000","last_active":"2024-04-25T10:01:28.643193+0000","last_peered":"2024-04-25T10:01:28.643193+0000","last_clean":"2024-04-25T10:01:28.643193+0000","last_became_active":"2024-04-25T10:01:28.640015+0000","last_became_peered":"2024-04-25T10:01:28.640015+0000","last_unstale":"2024-04-25T10:01:28.643193+0000","last_undegraded":"2024-04-25T10:01:28.643193+0000","last_fullsized":"2024-04-25T10:01:28.643193+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:03.858446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635398+0000","last_change":"2024-04-25T10:01:28.635398+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635398+0000","last_undegraded":"2024-04-25T10:01:28.635398+0000","last_fullsized":"2024-04-25T10:01:28.635398+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645355+0000","last_change":"2024-04-25T10:01:28.644201+0000","last_active":"2024-04-25T10:01:28.645355+0000","last_peered":"2024-04-25T10:01:28.645355+0000","last_clean":"2024-04-25T10:01:28.645355+0000","last_became_active":"2024-04-25T10:01:28.643895+0000","last_became_peered":"2024-04-25T10:01:28.643895+0000","last_unstale":"2024-04-25T10:01:28.645355+0000","last_undegraded":"2024-04-25T10:01:28.645355+0000","last_fullsized":"2024-04-25T10:01:28.645355+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.356021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632929+0000","last_change":"2024-04-25T10:01:28.632929+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632929+0000","last_undegraded":"2024-04-25T10:01:28.632929+0000","last_fullsized":"2024-04-25T10:01:28.632929+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643746+0000","last_change":"2024-04-25T10:01:28.640556+0000","last_active":"2024-04-25T10:01:28.643746+0000","last_peered":"2024-04-25T10:01:28.643746+0000","last_clean":"2024-04-25T10:01:28.643746+0000","last_became_active":"2024-04-25T10:01:28.640292+0000","last_became_peered":"2024-04-25T10:01:28.640292+0000","last_unstale":"2024-04-25T10:01:28.643746+0000","last_undegraded":"2024-04-25T10:01:28.643746+0000","last_fullsized":"2024-04-25T10:01:28.643746+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:31.176590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634251+0000","last_change":"2024-04-25T10:01:28.634251+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634251+0000","last_undegraded":"2024-04-25T10:01:28.634251+0000","last_fullsized":"2024-04-25T10:01:28.634251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648139+0000","last_change":"2024-04-25T10:01:28.641737+0000","last_active":"2024-04-25T10:01:28.648139+0000","last_peered":"2024-04-25T10:01:28.648139+0000","last_clean":"2024-04-25T10:01:28.648139+0000","last_became_active":"2024-04-25T10:01:28.641451+0000","last_became_peered":"2024-04-25T10:01:28.641451+0000","last_unstale":"2024-04-25T10:01:28.648139+0000","last_undegraded":"2024-04-25T10:01:28.648139+0000","last_fullsized":"2024-04-25T10:01:28.648139+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:02.288325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634539+0000","last_change":"2024-04-25T10:01:28.634539+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634539+0000","last_undegraded":"2024-04-25T10:01:28.634539+0000","last_fullsized":"2024-04-25T10:01:28.634539+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645115+0000","last_change":"2024-04-25T10:01:28.644516+0000","last_active":"2024-04-25T10:01:28.645115+0000","last_peered":"2024-04-25T10:01:28.645115+0000","last_clean":"2024-04-25T10:01:28.645115+0000","last_became_active":"2024-04-25T10:01:28.644155+0000","last_became_peered":"2024-04-25T10:01:28.644155+0000","last_unstale":"2024-04-25T10:01:28.645115+0000","last_undegraded":"2024-04-25T10:01:28.645115+0000","last_fullsized":"2024-04-25T10:01:28.645115+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:13.085301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.636198+0000","last_change":"2024-04-25T10:01:28.636198+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.636198+0000","last_undegraded":"2024-04-25T10:01:28.636198+0000","last_fullsized":"2024-04-25T10:01:28.636198+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.641997+0000","last_change":"2024-04-25T10:01:28.638744+0000","last_active":"2024-04-25T10:01:28.641997+0000","last_peered":"2024-04-25T10:01:28.641997+0000","last_clean":"2024-04-25T10:01:28.641997+0000","last_became_active":"2024-04-25T10:01:28.638312+0000","last_became_peered":"2024-04-25T10:01:28.638312+0000","last_unstale":"2024-04-25T10:01:28.641997+0000","last_undegraded":"2024-04-25T10:01:28.641997+0000","last_fullsized":"2024-04-25T10:01:28.641997+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:41.741278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632131+0000","last_change":"2024-04-25T10:01:28.632131+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632131+0000","last_undegraded":"2024-04-25T10:01:28.632131+0000","last_fullsized":"2024-04-25T10:01:28.632131+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648947+0000","last_change":"2024-04-25T10:01:28.647785+0000","last_active":"2024-04-25T10:01:28.648947+0000","last_peered":"2024-04-25T10:01:28.648947+0000","last_clean":"2024-04-25T10:01:28.648947+0000","last_became_active":"2024-04-25T10:01:28.647474+0000","last_became_peered":"2024-04-25T10:01:28.647474+0000","last_unstale":"2024-04-25T10:01:28.648947+0000","last_undegraded":"2024-04-25T10:01:28.648947+0000","last_fullsized":"2024-04-25T10:01:28.648947+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:21.347290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629400+0000","last_change":"2024-04-25T10:01:28.629400+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629400+0000","last_undegraded":"2024-04-25T10:01:28.629400+0000","last_fullsized":"2024-04-25T10:01:28.629400+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645364+0000","last_change":"2024-04-25T10:01:28.642073+0000","last_active":"2024-04-25T10:01:28.645364+0000","last_peered":"2024-04-25T10:01:28.645364+0000","last_clean":"2024-04-25T10:01:28.645364+0000","last_became_active":"2024-04-25T10:01:28.641446+0000","last_became_peered":"2024-04-25T10:01:28.641446+0000","last_unstale":"2024-04-25T10:01:28.645364+0000","last_undegraded":"2024-04-25T10:01:28.645364+0000","last_fullsized":"2024-04-25T10:01:28.645364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:09.409126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629182+0000","last_change":"2024-04-25T10:01:28.629182+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629182+0000","last_undegraded":"2024-04-25T10:01:28.629182+0000","last_fullsized":"2024-04-25T10:01:28.629182+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645618+0000","last_change":"2024-04-25T10:01:28.643918+0000","last_active":"2024-04-25T10:01:28.645618+0000","last_peered":"2024-04-25T10:01:28.645618+0000","last_clean":"2024-04-25T10:01:28.645618+0000","last_became_active":"2024-04-25T10:01:28.643575+0000","last_became_peered":"2024-04-25T10:01:28.643575+0000","last_unstale":"2024-04-25T10:01:28.645618+0000","last_undegraded":"2024-04-25T10:01:28.645618+0000","last_fullsized":"2024-04-25T10:01:28.645618+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:10.820229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634919+0000","last_change":"2024-04-25T10:01:28.634919+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634919+0000","last_undegraded":"2024-04-25T10:01:28.634919+0000","last_fullsized":"2024-04-25T10:01:28.634919+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.637809+0000","last_change":"2024-04-25T10:01:28.633829+0000","last_active":"2024-04-25T10:01:28.637809+0000","last_peered":"2024-04-25T10:01:28.637809+0000","last_clean":"2024-04-25T10:01:28.637809+0000","last_became_active":"2024-04-25T10:01:28.632994+0000","last_became_peered":"2024-04-25T10:01:28.632994+0000","last_unstale":"2024-04-25T10:01:28.637809+0000","last_undegraded":"2024-04-25T10:01:28.637809+0000","last_fullsized":"2024-04-25T10:01:28.637809+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:57.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630716+0000","last_change":"2024-04-25T10:01:28.630716+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630716+0000","last_undegraded":"2024-04-25T10:01:28.630716+0000","last_fullsized":"2024-04-25T10:01:28.630716+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.649655+0000","last_change":"2024-04-25T10:01:28.649336+0000","last_active":"2024-04-25T10:01:28.649655+0000","last_peered":"2024-04-25T10:01:28.649655+0000","last_clean":"2024-04-25T10:01:28.649655+0000","last_became_active":"2024-04-25T10:01:28.649067+0000","last_became_peered":"2024-04-25T10:01:28.649067+0000","last_unstale":"2024-04-25T10:01:28.649655+0000","last_undegraded":"2024-04-25T10:01:28.649655+0000","last_fullsized":"2024-04-25T10:01:28.649655+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:54.846448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631752+0000","last_change":"2024-04-25T10:01:28.631752+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631752+0000","last_undegraded":"2024-04-25T10:01:28.631752+0000","last_fullsized":"2024-04-25T10:01:28.631752+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643332+0000","last_change":"2024-04-25T10:01:28.637991+0000","last_active":"2024-04-25T10:01:28.643332+0000","last_peered":"2024-04-25T10:01:28.643332+0000","last_clean":"2024-04-25T10:01:28.643332+0000","last_became_active":"2024-04-25T10:01:28.637710+0000","last_became_peered":"2024-04-25T10:01:28.637710+0000","last_unstale":"2024-04-25T10:01:28.643332+0000","last_undegraded":"2024-04-25T10:01:28.643332+0000","last_fullsized":"2024-04-25T10:01:28.643332+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:32.105344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630788+0000","last_change":"2024-04-25T10:01:28.630788+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630788+0000","last_undegraded":"2024-04-25T10:01:28.630788+0000","last_fullsized":"2024-04-25T10:01:28.630788+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.639587+0000","last_change":"2024-04-25T10:01:28.637799+0000","last_active":"2024-04-25T10:01:28.639587+0000","last_peered":"2024-04-25T10:01:28.639587+0000","last_clean":"2024-04-25T10:01:28.639587+0000","last_became_active":"2024-04-25T10:01:28.637491+0000","last_became_peered":"2024-04-25T10:01:28.637491+0000","last_unstale":"2024-04-25T10:01:28.639587+0000","last_undegraded":"2024-04-25T10:01:28.639587+0000","last_fullsized":"2024-04-25T10:01:28.639587+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:10.916547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628958+0000","last_change":"2024-04-25T10:01:28.628958+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628958+0000","last_undegraded":"2024-04-25T10:01:28.628958+0000","last_fullsized":"2024-04-25T10:01:28.628958+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.647431+0000","last_change":"2024-04-25T10:01:28.645758+0000","last_active":"2024-04-25T10:01:28.647431+0000","last_peered":"2024-04-25T10:01:28.647431+0000","last_clean":"2024-04-25T10:01:28.647431+0000","last_became_active":"2024-04-25T10:01:28.645393+0000","last_became_peered":"2024-04-25T10:01:28.645393+0000","last_unstale":"2024-04-25T10:01:28.647431+0000","last_undegraded":"2024-04-25T10:01:28.647431+0000","last_fullsized":"2024-04-25T10:01:28.647431+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.056666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631251+0000","last_change":"2024-04-25T10:01:28.631251+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631251+0000","last_undegraded":"2024-04-25T10:01:28.631251+0000","last_fullsized":"2024-04-25T10:01:28.631251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.638771+0000","last_change":"2024-04-25T10:01:28.636229+0000","last_active":"2024-04-25T10:01:28.638771+0000","last_peered":"2024-04-25T10:01:28.638771+0000","last_clean":"2024-04-25T10:01:28.638771+0000","last_became_active":"2024-04-25T10:01:28.635741+0000","last_became_peered":"2024-04-25T10:01:28.635741+0000","last_unstale":"2024-04-25T10:01:28.638771+0000","last_undegraded":"2024-04-25T10:01:28.638771+0000","last_fullsized":"2024-04-25T10:01:28.638771+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:28.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":388,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.631752+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:01:28.631752+0000","last_peered":"2024-04-25T10:01:28.631752+0000","last_clean":"2024-04-25T10:01:28.631752+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:01:28.631752+0000","last_undegraded":"2024-04-25T10:01:28.631752+0000","last_fullsized":"2024-04-25T10:01:28.631752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29156,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342684,"statfs":{"total":96636764160,"available":96606908416,"internally_reserved":0,"allocated":2183168,"data_stored":1329872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46624,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325216,"statfs":{"total":96636764160,"available":96589021184,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54928,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316912,"statfs":{"total":96636764160,"available":96580517888,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54944,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316896,"statfs":{"total":96636764160,"available":96580501504,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50772,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321068,"statfs":{"total":96636764160,"available":96584773632,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50784,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321056,"statfs":{"total":96636764160,"available":96584761344,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51348,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320492,"statfs":{"total":96636764160,"available":96584183808,"internally_reserved":0,"allocated":2170880,"data_stored":1323213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:33.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.145+0000 7f0894007640 1 Processor -- start 2024-04-25T10:01:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.145+0000 7f0894007640 1 -- start start 2024-04-25T10:01:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f0894007640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f088c15db70 0x7f088c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f0894007640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c162690 con 0x7f088c0b5c30 2024-04-25T10:01:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f0894007640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c0ceee0 con 0x7f088c0b6cb0 2024-04-25T10:01:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f0894007640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c0d1d10 con 0x7f088c15db70 2024-04-25T10:01:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f0891da3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f088c15db70 0x7f088c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f08925a4640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f088c0b5c30 0x7f088c0d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40472/0 (socket says 172.21.15.19:40472) 2024-04-25T10:01:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f08925a4640 1 -- 172.21.15.19:0/3422518727 learned_addr learned my addr 172.21.15.19:0/3422518727 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f0891da3640 1 --2- 172.21.15.19:0/3422518727 >> v2:172.21.15.19:3300/0 conn(0x7f088c15db70 0x7f088c162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33544/0 (socket says 172.21.15.19:33544) 2024-04-25T10:01:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f0891da3640 1 -- 172.21.15.19:0/3422518727 >> v1:172.21.15.112:6789/0 conn(0x7f088c0b6cb0 legacy=0x7f088c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f0891da3640 1 -- 172.21.15.19:0/3422518727 >> v1:172.21.15.19:6789/0 conn(0x7f088c0b5c30 legacy=0x7f088c0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f0890da1640 1 -- 172.21.15.19:0/3422518727 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839907145 0 0) 0x7f088c162690 con 0x7f088c0b5c30 2024-04-25T10:01:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.146+0000 7f0891da3640 1 -- 172.21.15.19:0/3422518727 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0d1ff0 con 0x7f088c15db70 2024-04-25T10:01:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.147+0000 7f0891da3640 1 --2- 172.21.15.19:0/3422518727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f088c15db70 0x7f088c162150 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f08880097a0 tx=0x7f088802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37e07bf4068b5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.147+0000 7f0890da1640 1 -- 172.21.15.19:0/3422518727 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888037070 con 0x7f088c15db70 2024-04-25T10:01:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.147+0000 7f0890da1640 1 -- 172.21.15.19:0/3422518727 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0888032440 con 0x7f088c15db70 2024-04-25T10:01:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.147+0000 7f0890da1640 1 -- 172.21.15.19:0/3422518727 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888032810 con 0x7f088c15db70 2024-04-25T10:01:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.148+0000 7f0894007640 1 -- 172.21.15.19:0/3422518727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f088c15db70 msgr2=0x7f088c162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.148+0000 7f0894007640 1 --2- 172.21.15.19:0/3422518727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f088c15db70 0x7f088c162150 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f08880097a0 tx=0x7f088802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.148+0000 7f0894007640 1 -- 172.21.15.19:0/3422518727 shutdown_connections 2024-04-25T10:01:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.148+0000 7f0894007640 1 --2- 172.21.15.19:0/3422518727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f088c15db70 0x7f088c162150 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.148+0000 7f0894007640 1 -- 172.21.15.19:0/3422518727 >> 172.21.15.19:0/3422518727 conn(0x7f088c0d0520 msgr2=0x7f088c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.148+0000 7f0894007640 1 -- 172.21.15.19:0/3422518727 shutdown_connections 2024-04-25T10:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.148+0000 7f0894007640 1 -- 172.21.15.19:0/3422518727 wait complete. 2024-04-25T10:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.149+0000 7f0894007640 1 Processor -- start 2024-04-25T10:01:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.149+0000 7f0894007640 1 -- start start 2024-04-25T10:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0894007640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f088c0b5c30 0x7f088c1510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0894007640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f088c0b6cb0 0x7f088c1f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0894007640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f088c15db70 0x7f088c15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0894007640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0ce530 con 0x7f088c0b5c30 2024-04-25T10:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0894007640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c15df50 con 0x7f088c0b6cb0 2024-04-25T10:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0894007640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0cf1c0 con 0x7f088c15db70 2024-04-25T10:01:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0891da3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f088c0b5c30 0x7f088c1510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0891da3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f088c0b5c30 0x7f088c1510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33546/0 (socket says 172.21.15.19:33546) 2024-04-25T10:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0891da3640 1 -- 172.21.15.19:0/2505155652 learned_addr learned my addr 172.21.15.19:0/2505155652 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f08925a4640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f088c15db70 0x7f088c15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f08915a2640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f088c0b6cb0 0x7f088c1f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0891da3640 1 -- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f088c15db70 msgr2=0x7f088c15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0891da3640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f088c15db70 0x7f088c15c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0891da3640 1 -- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f088c0b6cb0 msgr2=0x7f088c1f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0891da3640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f088c0b6cb0 0x7f088c1f3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f0891da3640 1 -- 172.21.15.19:0/2505155652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c1515f0 con 0x7f088c0b5c30 2024-04-25T10:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.150+0000 7f08915a2640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f088c0b6cb0 0x7f088c1f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.151+0000 7f0891da3640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f088c0b5c30 0x7f088c1510b0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f08880097a0 tx=0x7f08880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.151+0000 7f08827fc640 1 -- 172.21.15.19:0/2505155652 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888041070 con 0x7f088c0b5c30 2024-04-25T10:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.151+0000 7f08827fc640 1 -- 172.21.15.19:0/2505155652 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088803c870 con 0x7f088c0b5c30 2024-04-25T10:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.151+0000 7f0894007640 1 -- 172.21.15.19:0/2505155652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c160270 con 0x7f088c0b5c30 2024-04-25T10:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.151+0000 7f08827fc640 1 -- 172.21.15.19:0/2505155652 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888037040 con 0x7f088c0b5c30 2024-04-25T10:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.151+0000 7f0894007640 1 -- 172.21.15.19:0/2505155652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c160830 con 0x7f088c0b5c30 2024-04-25T10:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.152+0000 7f0894007640 1 -- 172.21.15.19:0/2505155652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c073990 con 0x7f088c0b5c30 2024-04-25T10:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.152+0000 7f08827fc640 1 -- 172.21.15.19:0/2505155652 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0888044070 con 0x7f088c0b5c30 2024-04-25T10:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.153+0000 7f08827fc640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f087006b4d0 0x7f087006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.153+0000 7f08827fc640 1 -- 172.21.15.19:0/2505155652 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f08880aada0 con 0x7f088c0b5c30 2024-04-25T10:01:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.153+0000 7f08915a2640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f087006b4d0 0x7f087006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.154+0000 7f08915a2640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f087006b4d0 0x7f087006d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f088c0d3650 tx=0x7f087c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.157+0000 7f08827fc640 1 -- 172.21.15.19:0/2505155652 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08880795f0 con 0x7f088c0b5c30 2024-04-25T10:01:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.335+0000 7f0894007640 1 -- 172.21.15.19:0/2505155652 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f088c0d1820 con 0x7f087006b4d0 2024-04-25T10:01:33.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.342+0000 7f08827fc640 1 -- 172.21.15.19:0/2505155652 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247073 (secure 0 0 0) 0x7f088c0d1820 con 0x7f087006b4d0 2024-04-25T10:01:33.349 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:01:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 -- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f087006b4d0 msgr2=0x7f087006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f087006b4d0 0x7f087006d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f088c0d3650 tx=0x7f087c009040 comp rx=0 tx=0).stop 2024-04-25T10:01:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 -- 172.21.15.19:0/2505155652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f088c0b5c30 msgr2=0x7f088c1510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f088c0b5c30 0x7f088c1510b0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f08880097a0 tx=0x7f08880046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 -- 172.21.15.19:0/2505155652 shutdown_connections 2024-04-25T10:01:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f087006b4d0 0x7f087006d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f088c15db70 0x7f088c15c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f088c0b6cb0 0x7f088c1f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 --2- 172.21.15.19:0/2505155652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f088c0b5c30 0x7f088c1510b0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 -- 172.21.15.19:0/2505155652 >> 172.21.15.19:0/2505155652 conn(0x7f088c0d0520 msgr2=0x7f088c15a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 -- 172.21.15.19:0/2505155652 shutdown_connections 2024-04-25T10:01:33.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.345+0000 7f0894007640 1 -- 172.21.15.19:0/2505155652 wait complete. 2024-04-25T10:01:33.360 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-04-25T10:01:33.027851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385164,"kb_used_data":13528,"kb_used_omap":12,"kb_used_meta":371507,"kb_avail":754589556,"statfs":{"total":773094113280,"available":772699705344,"internally_reserved":0,"allocated":13852672,"data_stored":7070132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":380423730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.404787"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.639822+0000","last_change":"2024-04-25T10:01:28.639822+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.639822+0000","last_undegraded":"2024-04-25T10:01:28.639822+0000","last_fullsized":"2024-04-25T10:01:28.639822+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633777+0000","last_change":"2024-04-25T10:01:28.633777+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633777+0000","last_undegraded":"2024-04-25T10:01:28.633777+0000","last_fullsized":"2024-04-25T10:01:28.633777+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631663+0000","last_change":"2024-04-25T10:01:28.631663+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631663+0000","last_undegraded":"2024-04-25T10:01:28.631663+0000","last_fullsized":"2024-04-25T10:01:28.631663+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633655+0000","last_change":"2024-04-25T10:01:28.633655+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633655+0000","last_undegraded":"2024-04-25T10:01:28.633655+0000","last_fullsized":"2024-04-25T10:01:28.633655+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631604+0000","last_change":"2024-04-25T10:01:28.631604+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631604+0000","last_undegraded":"2024-04-25T10:01:28.631604+0000","last_fullsized":"2024-04-25T10:01:28.631604+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632464+0000","last_change":"2024-04-25T10:01:28.632464+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632464+0000","last_undegraded":"2024-04-25T10:01:28.632464+0000","last_fullsized":"2024-04-25T10:01:28.632464+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630901+0000","last_change":"2024-04-25T10:01:28.630901+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630901+0000","last_undegraded":"2024-04-25T10:01:28.630901+0000","last_fullsized":"2024-04-25T10:01:28.630901+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633293+0000","last_change":"2024-04-25T10:01:28.633293+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633293+0000","last_undegraded":"2024-04-25T10:01:28.633293+0000","last_fullsized":"2024-04-25T10:01:28.633293+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632483+0000","last_change":"2024-04-25T10:01:28.632483+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632483+0000","last_undegraded":"2024-04-25T10:01:28.632483+0000","last_fullsized":"2024-04-25T10:01:28.632483+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634762+0000","last_change":"2024-04-25T10:01:28.634762+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634762+0000","last_undegraded":"2024-04-25T10:01:28.634762+0000","last_fullsized":"2024-04-25T10:01:28.634762+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634423+0000","last_change":"2024-04-25T10:01:28.634423+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634423+0000","last_undegraded":"2024-04-25T10:01:28.634423+0000","last_fullsized":"2024-04-25T10:01:28.634423+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633781+0000","last_change":"2024-04-25T10:01:28.633781+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633781+0000","last_undegraded":"2024-04-25T10:01:28.633781+0000","last_fullsized":"2024-04-25T10:01:28.633781+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631273+0000","last_change":"2024-04-25T10:01:28.631273+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631273+0000","last_undegraded":"2024-04-25T10:01:28.631273+0000","last_fullsized":"2024-04-25T10:01:28.631273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633250+0000","last_change":"2024-04-25T10:01:28.633250+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633250+0000","last_undegraded":"2024-04-25T10:01:28.633250+0000","last_fullsized":"2024-04-25T10:01:28.633250+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632131+0000","last_change":"2024-04-25T10:01:28.632131+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632131+0000","last_undegraded":"2024-04-25T10:01:28.632131+0000","last_fullsized":"2024-04-25T10:01:28.632131+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631442+0000","last_change":"2024-04-25T10:01:28.631442+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631442+0000","last_undegraded":"2024-04-25T10:01:28.631442+0000","last_fullsized":"2024-04-25T10:01:28.631442+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631831+0000","last_change":"2024-04-25T10:01:28.631831+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631831+0000","last_undegraded":"2024-04-25T10:01:28.631831+0000","last_fullsized":"2024-04-25T10:01:28.631831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635356+0000","last_change":"2024-04-25T10:01:28.635356+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635356+0000","last_undegraded":"2024-04-25T10:01:28.635356+0000","last_fullsized":"2024-04-25T10:01:28.635356+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629531+0000","last_change":"2024-04-25T10:01:28.629531+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629531+0000","last_undegraded":"2024-04-25T10:01:28.629531+0000","last_fullsized":"2024-04-25T10:01:28.629531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633354+0000","last_change":"2024-04-25T10:01:28.633354+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633354+0000","last_undegraded":"2024-04-25T10:01:28.633354+0000","last_fullsized":"2024-04-25T10:01:28.633354+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634748+0000","last_change":"2024-04-25T10:01:28.634748+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634748+0000","last_undegraded":"2024-04-25T10:01:28.634748+0000","last_fullsized":"2024-04-25T10:01:28.634748+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631665+0000","last_change":"2024-04-25T10:01:28.631665+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631665+0000","last_undegraded":"2024-04-25T10:01:28.631665+0000","last_fullsized":"2024-04-25T10:01:28.631665+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634367+0000","last_change":"2024-04-25T10:01:28.634367+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634367+0000","last_undegraded":"2024-04-25T10:01:28.634367+0000","last_fullsized":"2024-04-25T10:01:28.634367+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631023+0000","last_change":"2024-04-25T10:01:28.631023+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631023+0000","last_undegraded":"2024-04-25T10:01:28.631023+0000","last_fullsized":"2024-04-25T10:01:28.631023+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631167+0000","last_change":"2024-04-25T10:01:28.631167+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631167+0000","last_undegraded":"2024-04-25T10:01:28.631167+0000","last_fullsized":"2024-04-25T10:01:28.631167+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629895+0000","last_change":"2024-04-25T10:01:28.629895+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629895+0000","last_undegraded":"2024-04-25T10:01:28.629895+0000","last_fullsized":"2024-04-25T10:01:28.629895+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628825+0000","last_change":"2024-04-25T10:01:28.628825+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628825+0000","last_undegraded":"2024-04-25T10:01:28.628825+0000","last_fullsized":"2024-04-25T10:01:28.628825+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.637120+0000","last_change":"2024-04-25T10:01:28.637120+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.637120+0000","last_undegraded":"2024-04-25T10:01:28.637120+0000","last_fullsized":"2024-04-25T10:01:28.637120+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632165+0000","last_change":"2024-04-25T10:01:28.632165+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632165+0000","last_undegraded":"2024-04-25T10:01:28.632165+0000","last_fullsized":"2024-04-25T10:01:28.632165+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629573+0000","last_change":"2024-04-25T10:01:28.629573+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629573+0000","last_undegraded":"2024-04-25T10:01:28.629573+0000","last_fullsized":"2024-04-25T10:01:28.629573+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632296+0000","last_change":"2024-04-25T10:01:28.632296+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632296+0000","last_undegraded":"2024-04-25T10:01:28.632296+0000","last_fullsized":"2024-04-25T10:01:28.632296+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634707+0000","last_change":"2024-04-25T10:01:28.634707+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634707+0000","last_undegraded":"2024-04-25T10:01:28.634707+0000","last_fullsized":"2024-04-25T10:01:28.634707+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632907+0000","last_change":"2024-04-25T10:01:28.632907+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632907+0000","last_undegraded":"2024-04-25T10:01:28.632907+0000","last_fullsized":"2024-04-25T10:01:28.632907+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645733+0000","last_change":"2024-04-25T10:01:28.644943+0000","last_active":"2024-04-25T10:01:28.645733+0000","last_peered":"2024-04-25T10:01:28.645733+0000","last_clean":"2024-04-25T10:01:28.645733+0000","last_became_active":"2024-04-25T10:01:28.644689+0000","last_became_peered":"2024-04-25T10:01:28.644689+0000","last_unstale":"2024-04-25T10:01:28.645733+0000","last_undegraded":"2024-04-25T10:01:28.645733+0000","last_fullsized":"2024-04-25T10:01:28.645733+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:48.121372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631082+0000","last_change":"2024-04-25T10:01:28.631082+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631082+0000","last_undegraded":"2024-04-25T10:01:28.631082+0000","last_fullsized":"2024-04-25T10:01:28.631082+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.647323+0000","last_change":"2024-04-25T10:01:28.645571+0000","last_active":"2024-04-25T10:01:28.647323+0000","last_peered":"2024-04-25T10:01:28.647323+0000","last_clean":"2024-04-25T10:01:28.647323+0000","last_became_active":"2024-04-25T10:01:28.645283+0000","last_became_peered":"2024-04-25T10:01:28.645283+0000","last_unstale":"2024-04-25T10:01:28.647323+0000","last_undegraded":"2024-04-25T10:01:28.647323+0000","last_fullsized":"2024-04-25T10:01:28.647323+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:11.425437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633628+0000","last_change":"2024-04-25T10:01:28.633628+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633628+0000","last_undegraded":"2024-04-25T10:01:28.633628+0000","last_fullsized":"2024-04-25T10:01:28.633628+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643814+0000","last_change":"2024-04-25T10:01:28.640421+0000","last_active":"2024-04-25T10:01:28.643814+0000","last_peered":"2024-04-25T10:01:28.643814+0000","last_clean":"2024-04-25T10:01:28.643814+0000","last_became_active":"2024-04-25T10:01:28.639536+0000","last_became_peered":"2024-04-25T10:01:28.639536+0000","last_unstale":"2024-04-25T10:01:28.643814+0000","last_undegraded":"2024-04-25T10:01:28.643814+0000","last_fullsized":"2024-04-25T10:01:28.643814+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.056666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.638331+0000","last_change":"2024-04-25T10:01:28.638331+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.638331+0000","last_undegraded":"2024-04-25T10:01:28.638331+0000","last_fullsized":"2024-04-25T10:01:28.638331+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643382+0000","last_change":"2024-04-25T10:01:28.642064+0000","last_active":"2024-04-25T10:01:28.643382+0000","last_peered":"2024-04-25T10:01:28.643382+0000","last_clean":"2024-04-25T10:01:28.643382+0000","last_became_active":"2024-04-25T10:01:28.641767+0000","last_became_peered":"2024-04-25T10:01:28.641767+0000","last_unstale":"2024-04-25T10:01:28.643382+0000","last_undegraded":"2024-04-25T10:01:28.643382+0000","last_fullsized":"2024-04-25T10:01:28.643382+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:28.715123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632603+0000","last_change":"2024-04-25T10:01:28.632603+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632603+0000","last_undegraded":"2024-04-25T10:01:28.632603+0000","last_fullsized":"2024-04-25T10:01:28.632603+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645590+0000","last_change":"2024-04-25T10:01:28.643613+0000","last_active":"2024-04-25T10:01:28.645590+0000","last_peered":"2024-04-25T10:01:28.645590+0000","last_clean":"2024-04-25T10:01:28.645590+0000","last_became_active":"2024-04-25T10:01:28.643326+0000","last_became_peered":"2024-04-25T10:01:28.643326+0000","last_unstale":"2024-04-25T10:01:28.645590+0000","last_undegraded":"2024-04-25T10:01:28.645590+0000","last_fullsized":"2024-04-25T10:01:28.645590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:47.363454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632282+0000","last_change":"2024-04-25T10:01:28.632282+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632282+0000","last_undegraded":"2024-04-25T10:01:28.632282+0000","last_fullsized":"2024-04-25T10:01:28.632282+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648286+0000","last_change":"2024-04-25T10:01:28.647013+0000","last_active":"2024-04-25T10:01:28.648286+0000","last_peered":"2024-04-25T10:01:28.648286+0000","last_clean":"2024-04-25T10:01:28.648286+0000","last_became_active":"2024-04-25T10:01:28.646667+0000","last_became_peered":"2024-04-25T10:01:28.646667+0000","last_unstale":"2024-04-25T10:01:28.648286+0000","last_undegraded":"2024-04-25T10:01:28.648286+0000","last_fullsized":"2024-04-25T10:01:28.648286+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:28.715123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634967+0000","last_change":"2024-04-25T10:01:28.634967+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634967+0000","last_undegraded":"2024-04-25T10:01:28.634967+0000","last_fullsized":"2024-04-25T10:01:28.634967+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643539+0000","last_change":"2024-04-25T10:01:28.639325+0000","last_active":"2024-04-25T10:01:28.643539+0000","last_peered":"2024-04-25T10:01:28.643539+0000","last_clean":"2024-04-25T10:01:28.643539+0000","last_became_active":"2024-04-25T10:01:28.639055+0000","last_became_peered":"2024-04-25T10:01:28.639055+0000","last_unstale":"2024-04-25T10:01:28.643539+0000","last_undegraded":"2024-04-25T10:01:28.643539+0000","last_fullsized":"2024-04-25T10:01:28.643539+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:09.888995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629531+0000","last_change":"2024-04-25T10:01:28.629531+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629531+0000","last_undegraded":"2024-04-25T10:01:28.629531+0000","last_fullsized":"2024-04-25T10:01:28.629531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.640085+0000","last_change":"2024-04-25T10:01:28.639418+0000","last_active":"2024-04-25T10:01:28.640085+0000","last_peered":"2024-04-25T10:01:28.640085+0000","last_clean":"2024-04-25T10:01:28.640085+0000","last_became_active":"2024-04-25T10:01:28.639134+0000","last_became_peered":"2024-04-25T10:01:28.639134+0000","last_unstale":"2024-04-25T10:01:28.640085+0000","last_undegraded":"2024-04-25T10:01:28.640085+0000","last_fullsized":"2024-04-25T10:01:28.640085+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:09.409126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631028+0000","last_change":"2024-04-25T10:01:28.631028+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631028+0000","last_undegraded":"2024-04-25T10:01:28.631028+0000","last_fullsized":"2024-04-25T10:01:28.631028+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.641745+0000","last_change":"2024-04-25T10:01:28.640498+0000","last_active":"2024-04-25T10:01:28.641745+0000","last_peered":"2024-04-25T10:01:28.641745+0000","last_clean":"2024-04-25T10:01:28.641745+0000","last_became_active":"2024-04-25T10:01:28.640186+0000","last_became_peered":"2024-04-25T10:01:28.640186+0000","last_unstale":"2024-04-25T10:01:28.641745+0000","last_undegraded":"2024-04-25T10:01:28.641745+0000","last_fullsized":"2024-04-25T10:01:28.641745+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:23.471932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632233+0000","last_change":"2024-04-25T10:01:28.632233+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632233+0000","last_undegraded":"2024-04-25T10:01:28.632233+0000","last_fullsized":"2024-04-25T10:01:28.632233+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648909+0000","last_change":"2024-04-25T10:01:28.647926+0000","last_active":"2024-04-25T10:01:28.648909+0000","last_peered":"2024-04-25T10:01:28.648909+0000","last_clean":"2024-04-25T10:01:28.648909+0000","last_became_active":"2024-04-25T10:01:28.646963+0000","last_became_peered":"2024-04-25T10:01:28.646963+0000","last_unstale":"2024-04-25T10:01:28.648909+0000","last_undegraded":"2024-04-25T10:01:28.648909+0000","last_fullsized":"2024-04-25T10:01:28.648909+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:56.486037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629097+0000","last_change":"2024-04-25T10:01:28.629097+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629097+0000","last_undegraded":"2024-04-25T10:01:28.629097+0000","last_fullsized":"2024-04-25T10:01:28.629097+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643792+0000","last_change":"2024-04-25T10:01:28.638314+0000","last_active":"2024-04-25T10:01:28.643792+0000","last_peered":"2024-04-25T10:01:28.643792+0000","last_clean":"2024-04-25T10:01:28.643792+0000","last_became_active":"2024-04-25T10:01:28.637960+0000","last_became_peered":"2024-04-25T10:01:28.637960+0000","last_unstale":"2024-04-25T10:01:28.643792+0000","last_undegraded":"2024-04-25T10:01:28.643792+0000","last_fullsized":"2024-04-25T10:01:28.643792+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.356021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628748+0000","last_change":"2024-04-25T10:01:28.628748+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628748+0000","last_undegraded":"2024-04-25T10:01:28.628748+0000","last_fullsized":"2024-04-25T10:01:28.628748+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643363+0000","last_change":"2024-04-25T10:01:28.639720+0000","last_active":"2024-04-25T10:01:28.643363+0000","last_peered":"2024-04-25T10:01:28.643363+0000","last_clean":"2024-04-25T10:01:28.643363+0000","last_became_active":"2024-04-25T10:01:28.638956+0000","last_became_peered":"2024-04-25T10:01:28.638956+0000","last_unstale":"2024-04-25T10:01:28.643363+0000","last_undegraded":"2024-04-25T10:01:28.643363+0000","last_fullsized":"2024-04-25T10:01:28.643363+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:11.425437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635339+0000","last_change":"2024-04-25T10:01:28.635339+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635339+0000","last_undegraded":"2024-04-25T10:01:28.635339+0000","last_fullsized":"2024-04-25T10:01:28.635339+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648870+0000","last_change":"2024-04-25T10:01:28.644548+0000","last_active":"2024-04-25T10:01:28.648870+0000","last_peered":"2024-04-25T10:01:28.648870+0000","last_clean":"2024-04-25T10:01:28.648870+0000","last_became_active":"2024-04-25T10:01:28.643068+0000","last_became_peered":"2024-04-25T10:01:28.643068+0000","last_unstale":"2024-04-25T10:01:28.648870+0000","last_undegraded":"2024-04-25T10:01:28.648870+0000","last_fullsized":"2024-04-25T10:01:28.648870+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:28.602317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628719+0000","last_change":"2024-04-25T10:01:28.628719+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628719+0000","last_undegraded":"2024-04-25T10:01:28.628719+0000","last_fullsized":"2024-04-25T10:01:28.628719+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.642739+0000","last_change":"2024-04-25T10:01:28.641313+0000","last_active":"2024-04-25T10:01:28.642739+0000","last_peered":"2024-04-25T10:01:28.642739+0000","last_clean":"2024-04-25T10:01:28.642739+0000","last_became_active":"2024-04-25T10:01:28.641020+0000","last_became_peered":"2024-04-25T10:01:28.641020+0000","last_unstale":"2024-04-25T10:01:28.642739+0000","last_undegraded":"2024-04-25T10:01:28.642739+0000","last_fullsized":"2024-04-25T10:01:28.642739+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:47.363454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632145+0000","last_change":"2024-04-25T10:01:28.632145+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632145+0000","last_undegraded":"2024-04-25T10:01:28.632145+0000","last_fullsized":"2024-04-25T10:01:28.632145+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643497+0000","last_change":"2024-04-25T10:01:28.639344+0000","last_active":"2024-04-25T10:01:28.643497+0000","last_peered":"2024-04-25T10:01:28.643497+0000","last_clean":"2024-04-25T10:01:28.643497+0000","last_became_active":"2024-04-25T10:01:28.639079+0000","last_became_peered":"2024-04-25T10:01:28.639079+0000","last_unstale":"2024-04-25T10:01:28.643497+0000","last_undegraded":"2024-04-25T10:01:28.643497+0000","last_fullsized":"2024-04-25T10:01:28.643497+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:13.447365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632144+0000","last_change":"2024-04-25T10:01:28.632144+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632144+0000","last_undegraded":"2024-04-25T10:01:28.632144+0000","last_fullsized":"2024-04-25T10:01:28.632144+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.644323+0000","last_change":"2024-04-25T10:01:28.642872+0000","last_active":"2024-04-25T10:01:28.644323+0000","last_peered":"2024-04-25T10:01:28.644323+0000","last_clean":"2024-04-25T10:01:28.644323+0000","last_became_active":"2024-04-25T10:01:28.642250+0000","last_became_peered":"2024-04-25T10:01:28.642250+0000","last_unstale":"2024-04-25T10:01:28.644323+0000","last_undegraded":"2024-04-25T10:01:28.644323+0000","last_fullsized":"2024-04-25T10:01:28.644323+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:09.710270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630415+0000","last_change":"2024-04-25T10:01:28.630415+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630415+0000","last_undegraded":"2024-04-25T10:01:28.630415+0000","last_fullsized":"2024-04-25T10:01:28.630415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.644964+0000","last_change":"2024-04-25T10:01:28.643788+0000","last_active":"2024-04-25T10:01:28.644964+0000","last_peered":"2024-04-25T10:01:28.644964+0000","last_clean":"2024-04-25T10:01:28.644964+0000","last_became_active":"2024-04-25T10:01:28.643521+0000","last_became_peered":"2024-04-25T10:01:28.643521+0000","last_unstale":"2024-04-25T10:01:28.644964+0000","last_undegraded":"2024-04-25T10:01:28.644964+0000","last_fullsized":"2024-04-25T10:01:28.644964+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:10.820229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628642+0000","last_change":"2024-04-25T10:01:28.628642+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628642+0000","last_undegraded":"2024-04-25T10:01:28.628642+0000","last_fullsized":"2024-04-25T10:01:28.628642+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643193+0000","last_change":"2024-04-25T10:01:28.640275+0000","last_active":"2024-04-25T10:01:28.643193+0000","last_peered":"2024-04-25T10:01:28.643193+0000","last_clean":"2024-04-25T10:01:28.643193+0000","last_became_active":"2024-04-25T10:01:28.640015+0000","last_became_peered":"2024-04-25T10:01:28.640015+0000","last_unstale":"2024-04-25T10:01:28.643193+0000","last_undegraded":"2024-04-25T10:01:28.643193+0000","last_fullsized":"2024-04-25T10:01:28.643193+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:03.858446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635398+0000","last_change":"2024-04-25T10:01:28.635398+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635398+0000","last_undegraded":"2024-04-25T10:01:28.635398+0000","last_fullsized":"2024-04-25T10:01:28.635398+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645355+0000","last_change":"2024-04-25T10:01:28.644201+0000","last_active":"2024-04-25T10:01:28.645355+0000","last_peered":"2024-04-25T10:01:28.645355+0000","last_clean":"2024-04-25T10:01:28.645355+0000","last_became_active":"2024-04-25T10:01:28.643895+0000","last_became_peered":"2024-04-25T10:01:28.643895+0000","last_unstale":"2024-04-25T10:01:28.645355+0000","last_undegraded":"2024-04-25T10:01:28.645355+0000","last_fullsized":"2024-04-25T10:01:28.645355+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.356021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632929+0000","last_change":"2024-04-25T10:01:28.632929+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632929+0000","last_undegraded":"2024-04-25T10:01:28.632929+0000","last_fullsized":"2024-04-25T10:01:28.632929+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643746+0000","last_change":"2024-04-25T10:01:28.640556+0000","last_active":"2024-04-25T10:01:28.643746+0000","last_peered":"2024-04-25T10:01:28.643746+0000","last_clean":"2024-04-25T10:01:28.643746+0000","last_became_active":"2024-04-25T10:01:28.640292+0000","last_became_peered":"2024-04-25T10:01:28.640292+0000","last_unstale":"2024-04-25T10:01:28.643746+0000","last_undegraded":"2024-04-25T10:01:28.643746+0000","last_fullsized":"2024-04-25T10:01:28.643746+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:31.176590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634251+0000","last_change":"2024-04-25T10:01:28.634251+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634251+0000","last_undegraded":"2024-04-25T10:01:28.634251+0000","last_fullsized":"2024-04-25T10:01:28.634251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648139+0000","last_change":"2024-04-25T10:01:28.641737+0000","last_active":"2024-04-25T10:01:28.648139+0000","last_peered":"2024-04-25T10:01:28.648139+0000","last_clean":"2024-04-25T10:01:28.648139+0000","last_became_active":"2024-04-25T10:01:28.641451+0000","last_became_peered":"2024-04-25T10:01:28.641451+0000","last_unstale":"2024-04-25T10:01:28.648139+0000","last_undegraded":"2024-04-25T10:01:28.648139+0000","last_fullsized":"2024-04-25T10:01:28.648139+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:02.288325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634539+0000","last_change":"2024-04-25T10:01:28.634539+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634539+0000","last_undegraded":"2024-04-25T10:01:28.634539+0000","last_fullsized":"2024-04-25T10:01:28.634539+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645115+0000","last_change":"2024-04-25T10:01:28.644516+0000","last_active":"2024-04-25T10:01:28.645115+0000","last_peered":"2024-04-25T10:01:28.645115+0000","last_clean":"2024-04-25T10:01:28.645115+0000","last_became_active":"2024-04-25T10:01:28.644155+0000","last_became_peered":"2024-04-25T10:01:28.644155+0000","last_unstale":"2024-04-25T10:01:28.645115+0000","last_undegraded":"2024-04-25T10:01:28.645115+0000","last_fullsized":"2024-04-25T10:01:28.645115+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:13.085301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.636198+0000","last_change":"2024-04-25T10:01:28.636198+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.636198+0000","last_undegraded":"2024-04-25T10:01:28.636198+0000","last_fullsized":"2024-04-25T10:01:28.636198+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.641997+0000","last_change":"2024-04-25T10:01:28.638744+0000","last_active":"2024-04-25T10:01:28.641997+0000","last_peered":"2024-04-25T10:01:28.641997+0000","last_clean":"2024-04-25T10:01:28.641997+0000","last_became_active":"2024-04-25T10:01:28.638312+0000","last_became_peered":"2024-04-25T10:01:28.638312+0000","last_unstale":"2024-04-25T10:01:28.641997+0000","last_undegraded":"2024-04-25T10:01:28.641997+0000","last_fullsized":"2024-04-25T10:01:28.641997+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:41.741278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632131+0000","last_change":"2024-04-25T10:01:28.632131+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632131+0000","last_undegraded":"2024-04-25T10:01:28.632131+0000","last_fullsized":"2024-04-25T10:01:28.632131+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648947+0000","last_change":"2024-04-25T10:01:28.647785+0000","last_active":"2024-04-25T10:01:28.648947+0000","last_peered":"2024-04-25T10:01:28.648947+0000","last_clean":"2024-04-25T10:01:28.648947+0000","last_became_active":"2024-04-25T10:01:28.647474+0000","last_became_peered":"2024-04-25T10:01:28.647474+0000","last_unstale":"2024-04-25T10:01:28.648947+0000","last_undegraded":"2024-04-25T10:01:28.648947+0000","last_fullsized":"2024-04-25T10:01:28.648947+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:21.347290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629400+0000","last_change":"2024-04-25T10:01:28.629400+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629400+0000","last_undegraded":"2024-04-25T10:01:28.629400+0000","last_fullsized":"2024-04-25T10:01:28.629400+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645364+0000","last_change":"2024-04-25T10:01:28.642073+0000","last_active":"2024-04-25T10:01:28.645364+0000","last_peered":"2024-04-25T10:01:28.645364+0000","last_clean":"2024-04-25T10:01:28.645364+0000","last_became_active":"2024-04-25T10:01:28.641446+0000","last_became_peered":"2024-04-25T10:01:28.641446+0000","last_unstale":"2024-04-25T10:01:28.645364+0000","last_undegraded":"2024-04-25T10:01:28.645364+0000","last_fullsized":"2024-04-25T10:01:28.645364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:09.409126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629182+0000","last_change":"2024-04-25T10:01:28.629182+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629182+0000","last_undegraded":"2024-04-25T10:01:28.629182+0000","last_fullsized":"2024-04-25T10:01:28.629182+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645618+0000","last_change":"2024-04-25T10:01:28.643918+0000","last_active":"2024-04-25T10:01:28.645618+0000","last_peered":"2024-04-25T10:01:28.645618+0000","last_clean":"2024-04-25T10:01:28.645618+0000","last_became_active":"2024-04-25T10:01:28.643575+0000","last_became_peered":"2024-04-25T10:01:28.643575+0000","last_unstale":"2024-04-25T10:01:28.645618+0000","last_undegraded":"2024-04-25T10:01:28.645618+0000","last_fullsized":"2024-04-25T10:01:28.645618+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:10.820229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634919+0000","last_change":"2024-04-25T10:01:28.634919+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634919+0000","last_undegraded":"2024-04-25T10:01:28.634919+0000","last_fullsized":"2024-04-25T10:01:28.634919+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.637809+0000","last_change":"2024-04-25T10:01:28.633829+0000","last_active":"2024-04-25T10:01:28.637809+0000","last_peered":"2024-04-25T10:01:28.637809+0000","last_clean":"2024-04-25T10:01:28.637809+0000","last_became_active":"2024-04-25T10:01:28.632994+0000","last_became_peered":"2024-04-25T10:01:28.632994+0000","last_unstale":"2024-04-25T10:01:28.637809+0000","last_undegraded":"2024-04-25T10:01:28.637809+0000","last_fullsized":"2024-04-25T10:01:28.637809+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:57.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630716+0000","last_change":"2024-04-25T10:01:28.630716+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630716+0000","last_undegraded":"2024-04-25T10:01:28.630716+0000","last_fullsized":"2024-04-25T10:01:28.630716+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.649655+0000","last_change":"2024-04-25T10:01:28.649336+0000","last_active":"2024-04-25T10:01:28.649655+0000","last_peered":"2024-04-25T10:01:28.649655+0000","last_clean":"2024-04-25T10:01:28.649655+0000","last_became_active":"2024-04-25T10:01:28.649067+0000","last_became_peered":"2024-04-25T10:01:28.649067+0000","last_unstale":"2024-04-25T10:01:28.649655+0000","last_undegraded":"2024-04-25T10:01:28.649655+0000","last_fullsized":"2024-04-25T10:01:28.649655+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:54.846448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631752+0000","last_change":"2024-04-25T10:01:28.631752+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631752+0000","last_undegraded":"2024-04-25T10:01:28.631752+0000","last_fullsized":"2024-04-25T10:01:28.631752+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643332+0000","last_change":"2024-04-25T10:01:28.637991+0000","last_active":"2024-04-25T10:01:28.643332+0000","last_peered":"2024-04-25T10:01:28.643332+0000","last_clean":"2024-04-25T10:01:28.643332+0000","last_became_active":"2024-04-25T10:01:28.637710+0000","last_became_peered":"2024-04-25T10:01:28.637710+0000","last_unstale":"2024-04-25T10:01:28.643332+0000","last_undegraded":"2024-04-25T10:01:28.643332+0000","last_fullsized":"2024-04-25T10:01:28.643332+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:32.105344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630788+0000","last_change":"2024-04-25T10:01:28.630788+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630788+0000","last_undegraded":"2024-04-25T10:01:28.630788+0000","last_fullsized":"2024-04-25T10:01:28.630788+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.639587+0000","last_change":"2024-04-25T10:01:28.637799+0000","last_active":"2024-04-25T10:01:28.639587+0000","last_peered":"2024-04-25T10:01:28.639587+0000","last_clean":"2024-04-25T10:01:28.639587+0000","last_became_active":"2024-04-25T10:01:28.637491+0000","last_became_peered":"2024-04-25T10:01:28.637491+0000","last_unstale":"2024-04-25T10:01:28.639587+0000","last_undegraded":"2024-04-25T10:01:28.639587+0000","last_fullsized":"2024-04-25T10:01:28.639587+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:10.916547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628958+0000","last_change":"2024-04-25T10:01:28.628958+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628958+0000","last_undegraded":"2024-04-25T10:01:28.628958+0000","last_fullsized":"2024-04-25T10:01:28.628958+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.647431+0000","last_change":"2024-04-25T10:01:28.645758+0000","last_active":"2024-04-25T10:01:28.647431+0000","last_peered":"2024-04-25T10:01:28.647431+0000","last_clean":"2024-04-25T10:01:28.647431+0000","last_became_active":"2024-04-25T10:01:28.645393+0000","last_became_peered":"2024-04-25T10:01:28.645393+0000","last_unstale":"2024-04-25T10:01:28.647431+0000","last_undegraded":"2024-04-25T10:01:28.647431+0000","last_fullsized":"2024-04-25T10:01:28.647431+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.056666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631251+0000","last_change":"2024-04-25T10:01:28.631251+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631251+0000","last_undegraded":"2024-04-25T10:01:28.631251+0000","last_fullsized":"2024-04-25T10:01:28.631251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.638771+0000","last_change":"2024-04-25T10:01:28.636229+0000","last_active":"2024-04-25T10:01:28.638771+0000","last_peered":"2024-04-25T10:01:28.638771+0000","last_clean":"2024-04-25T10:01:28.638771+0000","last_became_active":"2024-04-25T10:01:28.635741+0000","last_became_peered":"2024-04-25T10:01:28.635741+0000","last_unstale":"2024-04-25T10:01:28.638771+0000","last_undegraded":"2024-04-25T10:01:28.638771+0000","last_fullsized":"2024-04-25T10:01:28.638771+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:28.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":388,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.631752+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:01:28.631752+0000","last_peered":"2024-04-25T10:01:28.631752+0000","last_clean":"2024-04-25T10:01:28.631752+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:01:28.631752+0000","last_undegraded":"2024-04-25T10:01:28.631752+0000","last_fullsized":"2024-04-25T10:01:28.631752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29156,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342684,"statfs":{"total":96636764160,"available":96606908416,"internally_reserved":0,"allocated":2183168,"data_stored":1329872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46624,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325216,"statfs":{"total":96636764160,"available":96589021184,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54928,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316912,"statfs":{"total":96636764160,"available":96580517888,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54944,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316896,"statfs":{"total":96636764160,"available":96580501504,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50772,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321068,"statfs":{"total":96636764160,"available":96584773632,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50784,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321056,"statfs":{"total":96636764160,"available":96584761344,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51348,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320492,"statfs":{"total":96636764160,"available":96584183808,"internally_reserved":0,"allocated":2170880,"data_stored":1323213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:33.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3f is not active+clean 2024-04-25T10:01:33.365 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.639822+0000', 'last_change': '2024-04-25T10:01:28.639822+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.639822+0000', 'last_undegraded': '2024-04-25T10:01:28.639822+0000', 'last_fullsized': '2024-04-25T10:01:28.639822+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:01:33.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3e is not active+clean 2024-04-25T10:01:33.365 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.633777+0000', 'last_change': '2024-04-25T10:01:28.633777+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.633777+0000', 'last_undegraded': '2024-04-25T10:01:28.633777+0000', 'last_fullsized': '2024-04-25T10:01:28.633777+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:01:33.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3d is not active+clean 2024-04-25T10:01:33.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631663+0000', 'last_change': '2024-04-25T10:01:28.631663+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631663+0000', 'last_undegraded': '2024-04-25T10:01:28.631663+0000', 'last_fullsized': '2024-04-25T10:01:28.631663+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:01:33.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3c is not active+clean 2024-04-25T10:01:33.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.633655+0000', 'last_change': '2024-04-25T10:01:28.633655+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.633655+0000', 'last_undegraded': '2024-04-25T10:01:28.633655+0000', 'last_fullsized': '2024-04-25T10:01:28.633655+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:01:33.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3b is not active+clean 2024-04-25T10:01:33.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631604+0000', 'last_change': '2024-04-25T10:01:28.631604+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631604+0000', 'last_undegraded': '2024-04-25T10:01:28.631604+0000', 'last_fullsized': '2024-04-25T10:01:28.631604+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:01:33.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3a is not active+clean 2024-04-25T10:01:33.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632464+0000', 'last_change': '2024-04-25T10:01:28.632464+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632464+0000', 'last_undegraded': '2024-04-25T10:01:28.632464+0000', 'last_fullsized': '2024-04-25T10:01:28.632464+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:01:33.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.39 is not active+clean 2024-04-25T10:01:33.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.630901+0000', 'last_change': '2024-04-25T10:01:28.630901+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.630901+0000', 'last_undegraded': '2024-04-25T10:01:28.630901+0000', 'last_fullsized': '2024-04-25T10:01:28.630901+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:01:33.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.38 is not active+clean 2024-04-25T10:01:33.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.633293+0000', 'last_change': '2024-04-25T10:01:28.633293+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.633293+0000', 'last_undegraded': '2024-04-25T10:01:28.633293+0000', 'last_fullsized': '2024-04-25T10:01:28.633293+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:01:33.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.37 is not active+clean 2024-04-25T10:01:33.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632483+0000', 'last_change': '2024-04-25T10:01:28.632483+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632483+0000', 'last_undegraded': '2024-04-25T10:01:28.632483+0000', 'last_fullsized': '2024-04-25T10:01:28.632483+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:01:33.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.36 is not active+clean 2024-04-25T10:01:33.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.634762+0000', 'last_change': '2024-04-25T10:01:28.634762+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.634762+0000', 'last_undegraded': '2024-04-25T10:01:28.634762+0000', 'last_fullsized': '2024-04-25T10:01:28.634762+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:01:33.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.35 is not active+clean 2024-04-25T10:01:33.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.634423+0000', 'last_change': '2024-04-25T10:01:28.634423+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.634423+0000', 'last_undegraded': '2024-04-25T10:01:28.634423+0000', 'last_fullsized': '2024-04-25T10:01:28.634423+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:01:33.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.34 is not active+clean 2024-04-25T10:01:33.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.633781+0000', 'last_change': '2024-04-25T10:01:28.633781+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.633781+0000', 'last_undegraded': '2024-04-25T10:01:28.633781+0000', 'last_fullsized': '2024-04-25T10:01:28.633781+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:01:33.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.33 is not active+clean 2024-04-25T10:01:33.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631273+0000', 'last_change': '2024-04-25T10:01:28.631273+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631273+0000', 'last_undegraded': '2024-04-25T10:01:28.631273+0000', 'last_fullsized': '2024-04-25T10:01:28.631273+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:01:33.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.32 is not active+clean 2024-04-25T10:01:33.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.633250+0000', 'last_change': '2024-04-25T10:01:28.633250+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.633250+0000', 'last_undegraded': '2024-04-25T10:01:28.633250+0000', 'last_fullsized': '2024-04-25T10:01:28.633250+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:01:33.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.31 is not active+clean 2024-04-25T10:01:33.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632131+0000', 'last_change': '2024-04-25T10:01:28.632131+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632131+0000', 'last_undegraded': '2024-04-25T10:01:28.632131+0000', 'last_fullsized': '2024-04-25T10:01:28.632131+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:01:33.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.30 is not active+clean 2024-04-25T10:01:33.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631442+0000', 'last_change': '2024-04-25T10:01:28.631442+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631442+0000', 'last_undegraded': '2024-04-25T10:01:28.631442+0000', 'last_fullsized': '2024-04-25T10:01:28.631442+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:01:33.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2f is not active+clean 2024-04-25T10:01:33.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631831+0000', 'last_change': '2024-04-25T10:01:28.631831+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631831+0000', 'last_undegraded': '2024-04-25T10:01:28.631831+0000', 'last_fullsized': '2024-04-25T10:01:28.631831+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:01:33.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2e is not active+clean 2024-04-25T10:01:33.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.635356+0000', 'last_change': '2024-04-25T10:01:28.635356+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.635356+0000', 'last_undegraded': '2024-04-25T10:01:28.635356+0000', 'last_fullsized': '2024-04-25T10:01:28.635356+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:01:33.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2d is not active+clean 2024-04-25T10:01:33.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.629531+0000', 'last_change': '2024-04-25T10:01:28.629531+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.629531+0000', 'last_undegraded': '2024-04-25T10:01:28.629531+0000', 'last_fullsized': '2024-04-25T10:01:28.629531+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:01:33.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2c is not active+clean 2024-04-25T10:01:33.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.633354+0000', 'last_change': '2024-04-25T10:01:28.633354+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.633354+0000', 'last_undegraded': '2024-04-25T10:01:28.633354+0000', 'last_fullsized': '2024-04-25T10:01:28.633354+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:01:33.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2b is not active+clean 2024-04-25T10:01:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.634748+0000', 'last_change': '2024-04-25T10:01:28.634748+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.634748+0000', 'last_undegraded': '2024-04-25T10:01:28.634748+0000', 'last_fullsized': '2024-04-25T10:01:28.634748+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:01:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2a is not active+clean 2024-04-25T10:01:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631665+0000', 'last_change': '2024-04-25T10:01:28.631665+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631665+0000', 'last_undegraded': '2024-04-25T10:01:28.631665+0000', 'last_fullsized': '2024-04-25T10:01:28.631665+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:01:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.29 is not active+clean 2024-04-25T10:01:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.634367+0000', 'last_change': '2024-04-25T10:01:28.634367+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.634367+0000', 'last_undegraded': '2024-04-25T10:01:28.634367+0000', 'last_fullsized': '2024-04-25T10:01:28.634367+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:01:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.28 is not active+clean 2024-04-25T10:01:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631023+0000', 'last_change': '2024-04-25T10:01:28.631023+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631023+0000', 'last_undegraded': '2024-04-25T10:01:28.631023+0000', 'last_fullsized': '2024-04-25T10:01:28.631023+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:01:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.27 is not active+clean 2024-04-25T10:01:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631167+0000', 'last_change': '2024-04-25T10:01:28.631167+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631167+0000', 'last_undegraded': '2024-04-25T10:01:28.631167+0000', 'last_fullsized': '2024-04-25T10:01:28.631167+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:01:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.26 is not active+clean 2024-04-25T10:01:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.629895+0000', 'last_change': '2024-04-25T10:01:28.629895+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.629895+0000', 'last_undegraded': '2024-04-25T10:01:28.629895+0000', 'last_fullsized': '2024-04-25T10:01:28.629895+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:01:33.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.25 is not active+clean 2024-04-25T10:01:33.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.628825+0000', 'last_change': '2024-04-25T10:01:28.628825+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.628825+0000', 'last_undegraded': '2024-04-25T10:01:28.628825+0000', 'last_fullsized': '2024-04-25T10:01:28.628825+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:01:33.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.24 is not active+clean 2024-04-25T10:01:33.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.637120+0000', 'last_change': '2024-04-25T10:01:28.637120+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.637120+0000', 'last_undegraded': '2024-04-25T10:01:28.637120+0000', 'last_fullsized': '2024-04-25T10:01:28.637120+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:01:33.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.23 is not active+clean 2024-04-25T10:01:33.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632165+0000', 'last_change': '2024-04-25T10:01:28.632165+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632165+0000', 'last_undegraded': '2024-04-25T10:01:28.632165+0000', 'last_fullsized': '2024-04-25T10:01:28.632165+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:01:33.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.22 is not active+clean 2024-04-25T10:01:33.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.629573+0000', 'last_change': '2024-04-25T10:01:28.629573+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.629573+0000', 'last_undegraded': '2024-04-25T10:01:28.629573+0000', 'last_fullsized': '2024-04-25T10:01:28.629573+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:01:33.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.21 is not active+clean 2024-04-25T10:01:33.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632296+0000', 'last_change': '2024-04-25T10:01:28.632296+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632296+0000', 'last_undegraded': '2024-04-25T10:01:28.632296+0000', 'last_fullsized': '2024-04-25T10:01:28.632296+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:01:33.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.20 is not active+clean 2024-04-25T10:01:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.634707+0000', 'last_change': '2024-04-25T10:01:28.634707+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.634707+0000', 'last_undegraded': '2024-04-25T10:01:28.634707+0000', 'last_fullsized': '2024-04-25T10:01:28.634707+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:01:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.18 is not active+clean 2024-04-25T10:01:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632907+0000', 'last_change': '2024-04-25T10:01:28.632907+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632907+0000', 'last_undegraded': '2024-04-25T10:01:28.632907+0000', 'last_fullsized': '2024-04-25T10:01:28.632907+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:01:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.19 is not active+clean 2024-04-25T10:01:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631082+0000', 'last_change': '2024-04-25T10:01:28.631082+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631082+0000', 'last_undegraded': '2024-04-25T10:01:28.631082+0000', 'last_fullsized': '2024-04-25T10:01:28.631082+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:01:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1a is not active+clean 2024-04-25T10:01:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.633628+0000', 'last_change': '2024-04-25T10:01:28.633628+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.633628+0000', 'last_undegraded': '2024-04-25T10:01:28.633628+0000', 'last_fullsized': '2024-04-25T10:01:28.633628+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:01:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1b is not active+clean 2024-04-25T10:01:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.638331+0000', 'last_change': '2024-04-25T10:01:28.638331+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.638331+0000', 'last_undegraded': '2024-04-25T10:01:28.638331+0000', 'last_fullsized': '2024-04-25T10:01:28.638331+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:01:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1c is not active+clean 2024-04-25T10:01:33.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632603+0000', 'last_change': '2024-04-25T10:01:28.632603+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632603+0000', 'last_undegraded': '2024-04-25T10:01:28.632603+0000', 'last_fullsized': '2024-04-25T10:01:28.632603+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:01:33.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1d is not active+clean 2024-04-25T10:01:33.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632282+0000', 'last_change': '2024-04-25T10:01:28.632282+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632282+0000', 'last_undegraded': '2024-04-25T10:01:28.632282+0000', 'last_fullsized': '2024-04-25T10:01:28.632282+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:01:33.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1e is not active+clean 2024-04-25T10:01:33.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.634967+0000', 'last_change': '2024-04-25T10:01:28.634967+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.634967+0000', 'last_undegraded': '2024-04-25T10:01:28.634967+0000', 'last_fullsized': '2024-04-25T10:01:28.634967+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:01:33.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1f is not active+clean 2024-04-25T10:01:33.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.629531+0000', 'last_change': '2024-04-25T10:01:28.629531+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.629531+0000', 'last_undegraded': '2024-04-25T10:01:28.629531+0000', 'last_fullsized': '2024-04-25T10:01:28.629531+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:01:33.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.10 is not active+clean 2024-04-25T10:01:33.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631028+0000', 'last_change': '2024-04-25T10:01:28.631028+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631028+0000', 'last_undegraded': '2024-04-25T10:01:28.631028+0000', 'last_fullsized': '2024-04-25T10:01:28.631028+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:01:33.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.11 is not active+clean 2024-04-25T10:01:33.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632233+0000', 'last_change': '2024-04-25T10:01:28.632233+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632233+0000', 'last_undegraded': '2024-04-25T10:01:28.632233+0000', 'last_fullsized': '2024-04-25T10:01:28.632233+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:01:33.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.12 is not active+clean 2024-04-25T10:01:33.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.629097+0000', 'last_change': '2024-04-25T10:01:28.629097+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.629097+0000', 'last_undegraded': '2024-04-25T10:01:28.629097+0000', 'last_fullsized': '2024-04-25T10:01:28.629097+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:01:33.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.13 is not active+clean 2024-04-25T10:01:33.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.628748+0000', 'last_change': '2024-04-25T10:01:28.628748+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.628748+0000', 'last_undegraded': '2024-04-25T10:01:28.628748+0000', 'last_fullsized': '2024-04-25T10:01:28.628748+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:01:33.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.14 is not active+clean 2024-04-25T10:01:33.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.635339+0000', 'last_change': '2024-04-25T10:01:28.635339+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.635339+0000', 'last_undegraded': '2024-04-25T10:01:28.635339+0000', 'last_fullsized': '2024-04-25T10:01:28.635339+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:01:33.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.16 is not active+clean 2024-04-25T10:01:33.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.628719+0000', 'last_change': '2024-04-25T10:01:28.628719+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.628719+0000', 'last_undegraded': '2024-04-25T10:01:28.628719+0000', 'last_fullsized': '2024-04-25T10:01:28.628719+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:01:33.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.17 is not active+clean 2024-04-25T10:01:33.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632145+0000', 'last_change': '2024-04-25T10:01:28.632145+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632145+0000', 'last_undegraded': '2024-04-25T10:01:28.632145+0000', 'last_fullsized': '2024-04-25T10:01:28.632145+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:01:33.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.8 is not active+clean 2024-04-25T10:01:33.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632144+0000', 'last_change': '2024-04-25T10:01:28.632144+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632144+0000', 'last_undegraded': '2024-04-25T10:01:28.632144+0000', 'last_fullsized': '2024-04-25T10:01:28.632144+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:01:33.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.9 is not active+clean 2024-04-25T10:01:33.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.630415+0000', 'last_change': '2024-04-25T10:01:28.630415+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.630415+0000', 'last_undegraded': '2024-04-25T10:01:28.630415+0000', 'last_fullsized': '2024-04-25T10:01:28.630415+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:01:33.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.a is not active+clean 2024-04-25T10:01:33.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.628642+0000', 'last_change': '2024-04-25T10:01:28.628642+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.628642+0000', 'last_undegraded': '2024-04-25T10:01:28.628642+0000', 'last_fullsized': '2024-04-25T10:01:28.628642+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:01:33.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.b is not active+clean 2024-04-25T10:01:33.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.635398+0000', 'last_change': '2024-04-25T10:01:28.635398+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.635398+0000', 'last_undegraded': '2024-04-25T10:01:28.635398+0000', 'last_fullsized': '2024-04-25T10:01:28.635398+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:01:33.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.c is not active+clean 2024-04-25T10:01:33.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632929+0000', 'last_change': '2024-04-25T10:01:28.632929+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632929+0000', 'last_undegraded': '2024-04-25T10:01:28.632929+0000', 'last_fullsized': '2024-04-25T10:01:28.632929+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:01:33.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.d is not active+clean 2024-04-25T10:01:33.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.634251+0000', 'last_change': '2024-04-25T10:01:28.634251+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.634251+0000', 'last_undegraded': '2024-04-25T10:01:28.634251+0000', 'last_fullsized': '2024-04-25T10:01:28.634251+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:01:33.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.e is not active+clean 2024-04-25T10:01:33.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.634539+0000', 'last_change': '2024-04-25T10:01:28.634539+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.634539+0000', 'last_undegraded': '2024-04-25T10:01:28.634539+0000', 'last_fullsized': '2024-04-25T10:01:28.634539+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:01:33.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.f is not active+clean 2024-04-25T10:01:33.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.636198+0000', 'last_change': '2024-04-25T10:01:28.636198+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.636198+0000', 'last_undegraded': '2024-04-25T10:01:28.636198+0000', 'last_fullsized': '2024-04-25T10:01:28.636198+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:01:33.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.0 is not active+clean 2024-04-25T10:01:33.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.632131+0000', 'last_change': '2024-04-25T10:01:28.632131+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.632131+0000', 'last_undegraded': '2024-04-25T10:01:28.632131+0000', 'last_fullsized': '2024-04-25T10:01:28.632131+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:01:33.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1 is not active+clean 2024-04-25T10:01:33.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.629400+0000', 'last_change': '2024-04-25T10:01:28.629400+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.629400+0000', 'last_undegraded': '2024-04-25T10:01:28.629400+0000', 'last_fullsized': '2024-04-25T10:01:28.629400+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:01:33.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2 is not active+clean 2024-04-25T10:01:33.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.629182+0000', 'last_change': '2024-04-25T10:01:28.629182+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.629182+0000', 'last_undegraded': '2024-04-25T10:01:28.629182+0000', 'last_fullsized': '2024-04-25T10:01:28.629182+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:01:33.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3 is not active+clean 2024-04-25T10:01:33.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.634919+0000', 'last_change': '2024-04-25T10:01:28.634919+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.634919+0000', 'last_undegraded': '2024-04-25T10:01:28.634919+0000', 'last_fullsized': '2024-04-25T10:01:28.634919+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:01:33.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.4 is not active+clean 2024-04-25T10:01:33.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.630716+0000', 'last_change': '2024-04-25T10:01:28.630716+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.630716+0000', 'last_undegraded': '2024-04-25T10:01:28.630716+0000', 'last_fullsized': '2024-04-25T10:01:28.630716+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:01:33.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.5 is not active+clean 2024-04-25T10:01:33.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631752+0000', 'last_change': '2024-04-25T10:01:28.631752+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631752+0000', 'last_undegraded': '2024-04-25T10:01:28.631752+0000', 'last_fullsized': '2024-04-25T10:01:28.631752+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:01:33.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.6 is not active+clean 2024-04-25T10:01:33.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.630788+0000', 'last_change': '2024-04-25T10:01:28.630788+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.630788+0000', 'last_undegraded': '2024-04-25T10:01:28.630788+0000', 'last_fullsized': '2024-04-25T10:01:28.630788+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:01:33.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.7 is not active+clean 2024-04-25T10:01:33.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.628958+0000', 'last_change': '2024-04-25T10:01:28.628958+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.628958+0000', 'last_undegraded': '2024-04-25T10:01:28.628958+0000', 'last_fullsized': '2024-04-25T10:01:28.628958+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:01:33.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.15 is not active+clean 2024-04-25T10:01:33.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:01:28.631251+0000', 'last_change': '2024-04-25T10:01:28.631251+0000', 'last_active': '2024-04-25T10:01:28.621726+0000', 'last_peered': '2024-04-25T10:01:28.621726+0000', 'last_clean': '2024-04-25T10:01:28.621726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:01:28.631251+0000', 'last_undegraded': '2024-04-25T10:01:28.631251+0000', 'last_fullsized': '2024-04-25T10:01:28.631251+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'last_clean_scrub_stamp': '2024-04-25T10:01:28.621726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:01:33.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:33.446 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T10:01:33.442+0000 7f9b2e0b8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:01:33.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.495+0000 7f03e1929640 1 Processor -- start 2024-04-25T10:01:33.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.496+0000 7f03e1929640 1 -- start start 2024-04-25T10:01:33.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.496+0000 7f03e1929640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03dc15ba00 0x7f03dc09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.496+0000 7f03e1929640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f03dc1602e0 0x7f03dc0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.496+0000 7f03e1929640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03dc0cd710 con 0x7f03dc15f5f0 2024-04-25T10:01:33.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.496+0000 7f03e1929640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03dc099b70 con 0x7f03dc15ba00 2024-04-25T10:01:33.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.496+0000 7f03e1929640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03dc099e50 con 0x7f03dc1602e0 2024-04-25T10:01:33.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.496+0000 7f03daffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03dc15ba00 0x7f03dc09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.497+0000 7f03daffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03dc15ba00 0x7f03dc09a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33560/0 (socket says 172.21.15.19:33560) 2024-04-25T10:01:33.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.497+0000 7f03daffd640 1 -- 172.21.15.19:0/546926941 learned_addr learned my addr 172.21.15.19:0/546926941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:33.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.497+0000 7f03da7fc640 1 --2- 172.21.15.19:0/546926941 >> v2:172.21.15.112:3300/0 conn(0x7f03dc1602e0 0x7f03dc0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.497+0000 7f03daffd640 1 -- 172.21.15.19:0/546926941 >> v2:172.21.15.112:3300/0 conn(0x7f03dc1602e0 msgr2=0x7f03dc0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:33.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.497+0000 7f03daffd640 1 --2- 172.21.15.19:0/546926941 >> v2:172.21.15.112:3300/0 conn(0x7f03dc1602e0 0x7f03dc0ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.497+0000 7f03daffd640 1 -- 172.21.15.19:0/546926941 >> v1:172.21.15.112:6789/0 conn(0x7f03dc15f5f0 legacy=0x7f03dc0c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.497+0000 7f03daffd640 1 -- 172.21.15.19:0/546926941 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc09a130 con 0x7f03dc15ba00 2024-04-25T10:01:33.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.497+0000 7f03daffd640 1 --2- 172.21.15.19:0/546926941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03dc15ba00 0x7f03dc09a930 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0b7830 tx=0x7f03d402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fc0e2d6a1dbbe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:33.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.498+0000 7f03d9ffb640 1 -- 172.21.15.19:0/546926941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4037070 con 0x7f03dc15ba00 2024-04-25T10:01:33.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.498+0000 7f03d9ffb640 1 -- 172.21.15.19:0/546926941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d4032440 con 0x7f03dc15ba00 2024-04-25T10:01:33.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.498+0000 7f03d9ffb640 1 -- 172.21.15.19:0/546926941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4032810 con 0x7f03dc15ba00 2024-04-25T10:01:33.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.498+0000 7f03e1929640 1 -- 172.21.15.19:0/546926941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03dc15ba00 msgr2=0x7f03dc09a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.498+0000 7f03e1929640 1 --2- 172.21.15.19:0/546926941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03dc15ba00 0x7f03dc09a930 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0b7830 tx=0x7f03d402a160 comp rx=0 tx=0).stop 2024-04-25T10:01:33.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.498+0000 7f03e1929640 1 -- 172.21.15.19:0/546926941 shutdown_connections 2024-04-25T10:01:33.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.498+0000 7f03e1929640 1 --2- 172.21.15.19:0/546926941 >> v2:172.21.15.112:3300/0 conn(0x7f03dc1602e0 0x7f03dc0ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.498+0000 7f03e1929640 1 --2- 172.21.15.19:0/546926941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03dc15ba00 0x7f03dc09a930 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.498+0000 7f03e1929640 1 -- 172.21.15.19:0/546926941 >> 172.21.15.19:0/546926941 conn(0x7f03dc0b3270 msgr2=0x7f03dc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:33.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.498+0000 7f03e1929640 1 -- 172.21.15.19:0/546926941 shutdown_connections 2024-04-25T10:01:33.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.498+0000 7f03e1929640 1 -- 172.21.15.19:0/546926941 wait complete. 2024-04-25T10:01:33.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.499+0000 7f03e1929640 1 Processor -- start 2024-04-25T10:01:33.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.500+0000 7f03e1929640 1 -- start start 2024-04-25T10:01:33.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.500+0000 7f03e1929640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03dc15ba00 0x7f03dc1547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.500+0000 7f03e1929640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03dc15f5f0 0x7f03dc154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.500+0000 7f03e1929640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03dc1602e0 0x7f03dc15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.500+0000 7f03e1929640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc15bde0 con 0x7f03dc15ba00 2024-04-25T10:01:33.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.500+0000 7f03e1929640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0cd2e0 con 0x7f03dc1602e0 2024-04-25T10:01:33.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.500+0000 7f03e1929640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0cb1a0 con 0x7f03dc15f5f0 2024-04-25T10:01:33.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.500+0000 7f03daffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03dc15ba00 0x7f03dc1547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.500+0000 7f03daffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03dc15ba00 0x7f03dc1547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33570/0 (socket says 172.21.15.19:33570) 2024-04-25T10:01:33.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.500+0000 7f03daffd640 1 -- 172.21.15.19:0/767347447 learned_addr learned my addr 172.21.15.19:0/767347447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:33.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03da7fc640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03dc15f5f0 0x7f03dc154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03db7fe640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03dc1602e0 0x7f03dc15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03daffd640 1 -- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03dc15f5f0 msgr2=0x7f03dc154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03daffd640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03dc15f5f0 0x7f03dc154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03daffd640 1 -- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03dc1602e0 msgr2=0x7f03dc15c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03daffd640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03dc1602e0 0x7f03dc15c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03daffd640 1 -- 172.21.15.19:0/767347447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc0cb480 con 0x7f03dc15ba00 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03db7fe640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03dc1602e0 0x7f03dc15c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03daffd640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03dc15ba00 0x7f03dc1547e0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f03d402a1a0 tx=0x7f03d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03d3fff640 1 -- 172.21.15.19:0/767347447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4041020 con 0x7f03dc15ba00 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03d3fff640 1 -- 172.21.15.19:0/767347447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d40306c0 con 0x7f03dc15ba00 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03e1929640 1 -- 172.21.15.19:0/767347447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc1e5550 con 0x7f03dc15ba00 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03d3fff640 1 -- 172.21.15.19:0/767347447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4037040 con 0x7f03dc15ba00 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.501+0000 7f03e1929640 1 -- 172.21.15.19:0/767347447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc15c630 con 0x7f03dc15ba00 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.502+0000 7f03e1929640 1 -- 172.21.15.19:0/767347447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03dc073990 con 0x7f03dc15ba00 2024-04-25T10:01:33.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.503+0000 7f03d3fff640 1 -- 172.21.15.19:0/767347447 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03d4044020 con 0x7f03dc15ba00 2024-04-25T10:01:33.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.503+0000 7f03d3fff640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03bc06b4d0 0x7f03bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.503+0000 7f03d3fff640 1 -- 172.21.15.19:0/767347447 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03d40aac10 con 0x7f03dc15ba00 2024-04-25T10:01:33.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.503+0000 7f03da7fc640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03bc06b4d0 0x7f03bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.504+0000 7f03da7fc640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03bc06b4d0 0x7f03bc06d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f03c4006fd0 tx=0x7f03c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:33.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.507+0000 7f03d3fff640 1 -- 172.21.15.19:0/767347447 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03d404b030 con 0x7f03dc15ba00 2024-04-25T10:01:33.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.765+0000 7f03e1929640 1 -- 172.21.15.19:0/767347447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f03dc09a9c0 con 0x7f03dc15ba00 2024-04-25T10:01:33.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.766+0000 7f03d3fff640 1 -- 172.21.15.19:0/767347447 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1455 (secure 0 0 0) 0x7f03d4079880 con 0x7f03dc15ba00 2024-04-25T10:01:33.770 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:33.770 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":531,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":129,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":394407936,"bytes_avail":772699705344,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":79,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9213}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:01:13.023211+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:01:33.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 -- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03bc06b4d0 msgr2=0x7f03bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03bc06b4d0 0x7f03bc06d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f03c4006fd0 tx=0x7f03c4008040 comp rx=0 tx=0).stop 2024-04-25T10:01:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 -- 172.21.15.19:0/767347447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03dc15ba00 msgr2=0x7f03dc1547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03dc15ba00 0x7f03dc1547e0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f03d402a1a0 tx=0x7f03d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 -- 172.21.15.19:0/767347447 shutdown_connections 2024-04-25T10:01:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f03bc06b4d0 0x7f03bc06d990 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03dc1602e0 0x7f03dc15c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03dc15f5f0 0x7f03dc154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 --2- 172.21.15.19:0/767347447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03dc15ba00 0x7f03dc1547e0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 -- 172.21.15.19:0/767347447 >> 172.21.15.19:0/767347447 conn(0x7f03dc0b3270 msgr2=0x7f03dc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 -- 172.21.15.19:0/767347447 shutdown_connections 2024-04-25T10:01:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.768+0000 7f03e1929640 1 -- 172.21.15.19:0/767347447 wait complete. 2024-04-25T10:01:33.784 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:01:33.784 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.880+0000 7f3b647b7640 1 Processor -- start 2024-04-25T10:01:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.880+0000 7f3b647b7640 1 -- start start 2024-04-25T10:01:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.880+0000 7f3b647b7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3b5c15dd30 0x7f3b5c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.880+0000 7f3b647b7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3b5c0d0ad0 0x7f3b5c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.880+0000 7f3b647b7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5c0d1df0 con 0x7f3b5c0b9770 2024-04-25T10:01:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.880+0000 7f3b647b7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5c0d20d0 con 0x7f3b5c15dd30 2024-04-25T10:01:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.881+0000 7f3b647b7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5c0d23b0 con 0x7f3b5c0d0ad0 2024-04-25T10:01:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.881+0000 7f3b62553640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3b5c15dd30 0x7f3b5c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.881+0000 7f3b62553640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3b5c15dd30 0x7f3b5c0b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33586/0 (socket says 172.21.15.19:33586) 2024-04-25T10:01:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.881+0000 7f3b62553640 1 -- 172.21.15.19:0/3895288949 learned_addr learned my addr 172.21.15.19:0/3895288949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.881+0000 7f3b62553640 1 -- 172.21.15.19:0/3895288949 >> v2:172.21.15.112:3300/0 conn(0x7f3b5c0d0ad0 msgr2=0x7f3b5c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.881+0000 7f3b62d54640 1 --2- 172.21.15.19:0/3895288949 >> v2:172.21.15.112:3300/0 conn(0x7f3b5c0d0ad0 0x7f3b5c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.881+0000 7f3b62553640 1 --2- 172.21.15.19:0/3895288949 >> v2:172.21.15.112:3300/0 conn(0x7f3b5c0d0ad0 0x7f3b5c0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.881+0000 7f3b62553640 1 -- 172.21.15.19:0/3895288949 >> v1:172.21.15.112:6789/0 conn(0x7f3b5c0b9770 legacy=0x7f3b5c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.881+0000 7f3b62553640 1 -- 172.21.15.19:0/3895288949 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c0d2690 con 0x7f3b5c15dd30 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.881+0000 7f3b62d54640 1 --2- 172.21.15.19:0/3895288949 >> v2:172.21.15.112:3300/0 conn(0x7f3b5c0d0ad0 0x7f3b5c0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.882+0000 7f3b62553640 1 --2- 172.21.15.19:0/3895288949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b5c15dd30 0x7f3b5c0b9230 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0ce200 tx=0x7f3b5802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b394af9cfedadfc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.882+0000 7f3b61551640 1 -- 172.21.15.19:0/3895288949 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58037070 con 0x7f3b5c15dd30 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.882+0000 7f3b61551640 1 -- 172.21.15.19:0/3895288949 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5802ae20 con 0x7f3b5c15dd30 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.882+0000 7f3b61551640 1 -- 172.21.15.19:0/3895288949 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58032660 con 0x7f3b5c15dd30 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.882+0000 7f3b647b7640 1 -- 172.21.15.19:0/3895288949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b5c15dd30 msgr2=0x7f3b5c0b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.882+0000 7f3b647b7640 1 --2- 172.21.15.19:0/3895288949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b5c15dd30 0x7f3b5c0b9230 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0ce200 tx=0x7f3b5802a110 comp rx=0 tx=0).stop 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.882+0000 7f3b647b7640 1 -- 172.21.15.19:0/3895288949 shutdown_connections 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.882+0000 7f3b647b7640 1 --2- 172.21.15.19:0/3895288949 >> v2:172.21.15.112:3300/0 conn(0x7f3b5c0d0ad0 0x7f3b5c0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.882+0000 7f3b647b7640 1 --2- 172.21.15.19:0/3895288949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b5c15dd30 0x7f3b5c0b9230 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.882+0000 7f3b647b7640 1 -- 172.21.15.19:0/3895288949 >> 172.21.15.19:0/3895288949 conn(0x7f3b5c061c30 msgr2=0x7f3b5c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.883+0000 7f3b647b7640 1 -- 172.21.15.19:0/3895288949 shutdown_connections 2024-04-25T10:01:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.883+0000 7f3b647b7640 1 -- 172.21.15.19:0/3895288949 wait complete. 2024-04-25T10:01:33.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.884+0000 7f3b647b7640 1 Processor -- start 2024-04-25T10:01:33.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.884+0000 7f3b647b7640 1 -- start start 2024-04-25T10:01:33.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.884+0000 7f3b647b7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3b5c0b9770 0x7f3b5c14e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.884+0000 7f3b647b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b5c0d0ad0 0x7f3b5c14f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.884+0000 7f3b647b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b5c15dd30 0x7f3b5c1579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.884+0000 7f3b647b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0d1b90 con 0x7f3b5c0d0ad0 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.884+0000 7f3b647b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0d1a10 con 0x7f3b5c15dd30 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.884+0000 7f3b647b7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0d2e90 con 0x7f3b5c0b9770 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.884+0000 7f3b61d52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b5c0d0ad0 0x7f3b5c14f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.884+0000 7f3b61d52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b5c0d0ad0 0x7f3b5c14f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33594/0 (socket says 172.21.15.19:33594) 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b61d52640 1 -- 172.21.15.19:0/659395110 learned_addr learned my addr 172.21.15.19:0/659395110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b62d54640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b5c15dd30 0x7f3b5c1579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b62553640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3b5c0b9770 0x7f3b5c14e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b61d52640 1 -- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3b5c0b9770 msgr2=0x7f3b5c14e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b61d52640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3b5c0b9770 0x7f3b5c14e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b61d52640 1 -- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b5c15dd30 msgr2=0x7f3b5c1579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b61d52640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b5c15dd30 0x7f3b5c1579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b61d52640 1 -- 172.21.15.19:0/659395110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c153ac0 con 0x7f3b5c0d0ad0 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b62d54640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b5c15dd30 0x7f3b5c1579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b61d52640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b5c0d0ad0 0x7f3b5c14f450 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c004640 tx=0x7f3b4c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b52ffd640 1 -- 172.21.15.19:0/659395110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c011790 con 0x7f3b5c0d0ad0 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b52ffd640 1 -- 172.21.15.19:0/659395110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c017070 con 0x7f3b5c0d0ad0 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b647b7640 1 -- 172.21.15.19:0/659395110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c153d60 con 0x7f3b5c0d0ad0 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b52ffd640 1 -- 172.21.15.19:0/659395110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c011bb0 con 0x7f3b5c0d0ad0 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.885+0000 7f3b647b7640 1 -- 172.21.15.19:0/659395110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c14e650 con 0x7f3b5c0d0ad0 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.886+0000 7f3b647b7640 1 -- 172.21.15.19:0/659395110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5c153f30 con 0x7f3b5c0d0ad0 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.887+0000 7f3b52ffd640 1 -- 172.21.15.19:0/659395110 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b4c011d50 con 0x7f3b5c0d0ad0 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.887+0000 7f3b52ffd640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3b4006b4d0 0x7f3b4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.887+0000 7f3b52ffd640 1 -- 172.21.15.19:0/659395110 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b4c089f20 con 0x7f3b5c0d0ad0 2024-04-25T10:01:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.887+0000 7f3b62553640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3b4006b4d0 0x7f3b4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.888+0000 7f3b62553640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3b4006b4d0 0x7f3b4006d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3b58034000 tx=0x7f3b5802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:33.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:33.891+0000 7f3b52ffd640 1 -- 172.21.15.19:0/659395110 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b4c058c10 con 0x7f3b5c0d0ad0 2024-04-25T10:01:34.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.067+0000 7f3b647b7640 1 -- 172.21.15.19:0/659395110 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b5c073990 con 0x7f3b4006b4d0 2024-04-25T10:01:34.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.073+0000 7f3b52ffd640 1 -- 172.21.15.19:0/659395110 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247073 (secure 0 0 0) 0x7f3b5c073990 con 0x7f3b4006b4d0 2024-04-25T10:01:34.080 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:34.082 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:01:34.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.076+0000 7f3b647b7640 1 -- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3b4006b4d0 msgr2=0x7f3b4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.076+0000 7f3b647b7640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3b4006b4d0 0x7f3b4006d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3b58034000 tx=0x7f3b5802e000 comp rx=0 tx=0).stop 2024-04-25T10:01:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.076+0000 7f3b647b7640 1 -- 172.21.15.19:0/659395110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b5c0d0ad0 msgr2=0x7f3b5c14f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.076+0000 7f3b647b7640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b5c0d0ad0 0x7f3b5c14f450 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c004640 tx=0x7f3b4c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.076+0000 7f3b647b7640 1 -- 172.21.15.19:0/659395110 shutdown_connections 2024-04-25T10:01:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.076+0000 7f3b647b7640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3b4006b4d0 0x7f3b4006d990 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.076+0000 7f3b647b7640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b5c15dd30 0x7f3b5c1579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.076+0000 7f3b647b7640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b5c0d0ad0 0x7f3b5c14f450 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.076+0000 7f3b647b7640 1 --2- 172.21.15.19:0/659395110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3b5c0b9770 0x7f3b5c14e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.076+0000 7f3b647b7640 1 -- 172.21.15.19:0/659395110 >> 172.21.15.19:0/659395110 conn(0x7f3b5c061c30 msgr2=0x7f3b5c0d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.077+0000 7f3b647b7640 1 -- 172.21.15.19:0/659395110 shutdown_connections 2024-04-25T10:01:34.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:34.077+0000 7f3b647b7640 1 -- 172.21.15.19:0/659395110 wait complete. 2024-04-25T10:01:34.092 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-04-25T10:01:33.027851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385164,"kb_used_data":13528,"kb_used_omap":12,"kb_used_meta":371507,"kb_avail":754589556,"statfs":{"total":773094113280,"available":772699705344,"internally_reserved":0,"allocated":13852672,"data_stored":7070132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":380423730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.404787"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.639822+0000","last_change":"2024-04-25T10:01:28.639822+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.639822+0000","last_undegraded":"2024-04-25T10:01:28.639822+0000","last_fullsized":"2024-04-25T10:01:28.639822+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633777+0000","last_change":"2024-04-25T10:01:28.633777+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633777+0000","last_undegraded":"2024-04-25T10:01:28.633777+0000","last_fullsized":"2024-04-25T10:01:28.633777+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631663+0000","last_change":"2024-04-25T10:01:28.631663+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631663+0000","last_undegraded":"2024-04-25T10:01:28.631663+0000","last_fullsized":"2024-04-25T10:01:28.631663+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633655+0000","last_change":"2024-04-25T10:01:28.633655+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633655+0000","last_undegraded":"2024-04-25T10:01:28.633655+0000","last_fullsized":"2024-04-25T10:01:28.633655+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631604+0000","last_change":"2024-04-25T10:01:28.631604+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631604+0000","last_undegraded":"2024-04-25T10:01:28.631604+0000","last_fullsized":"2024-04-25T10:01:28.631604+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632464+0000","last_change":"2024-04-25T10:01:28.632464+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632464+0000","last_undegraded":"2024-04-25T10:01:28.632464+0000","last_fullsized":"2024-04-25T10:01:28.632464+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630901+0000","last_change":"2024-04-25T10:01:28.630901+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630901+0000","last_undegraded":"2024-04-25T10:01:28.630901+0000","last_fullsized":"2024-04-25T10:01:28.630901+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633293+0000","last_change":"2024-04-25T10:01:28.633293+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633293+0000","last_undegraded":"2024-04-25T10:01:28.633293+0000","last_fullsized":"2024-04-25T10:01:28.633293+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632483+0000","last_change":"2024-04-25T10:01:28.632483+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632483+0000","last_undegraded":"2024-04-25T10:01:28.632483+0000","last_fullsized":"2024-04-25T10:01:28.632483+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634762+0000","last_change":"2024-04-25T10:01:28.634762+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634762+0000","last_undegraded":"2024-04-25T10:01:28.634762+0000","last_fullsized":"2024-04-25T10:01:28.634762+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634423+0000","last_change":"2024-04-25T10:01:28.634423+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634423+0000","last_undegraded":"2024-04-25T10:01:28.634423+0000","last_fullsized":"2024-04-25T10:01:28.634423+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633781+0000","last_change":"2024-04-25T10:01:28.633781+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633781+0000","last_undegraded":"2024-04-25T10:01:28.633781+0000","last_fullsized":"2024-04-25T10:01:28.633781+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631273+0000","last_change":"2024-04-25T10:01:28.631273+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631273+0000","last_undegraded":"2024-04-25T10:01:28.631273+0000","last_fullsized":"2024-04-25T10:01:28.631273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633250+0000","last_change":"2024-04-25T10:01:28.633250+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633250+0000","last_undegraded":"2024-04-25T10:01:28.633250+0000","last_fullsized":"2024-04-25T10:01:28.633250+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632131+0000","last_change":"2024-04-25T10:01:28.632131+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632131+0000","last_undegraded":"2024-04-25T10:01:28.632131+0000","last_fullsized":"2024-04-25T10:01:28.632131+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631442+0000","last_change":"2024-04-25T10:01:28.631442+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631442+0000","last_undegraded":"2024-04-25T10:01:28.631442+0000","last_fullsized":"2024-04-25T10:01:28.631442+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631831+0000","last_change":"2024-04-25T10:01:28.631831+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631831+0000","last_undegraded":"2024-04-25T10:01:28.631831+0000","last_fullsized":"2024-04-25T10:01:28.631831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635356+0000","last_change":"2024-04-25T10:01:28.635356+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635356+0000","last_undegraded":"2024-04-25T10:01:28.635356+0000","last_fullsized":"2024-04-25T10:01:28.635356+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629531+0000","last_change":"2024-04-25T10:01:28.629531+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629531+0000","last_undegraded":"2024-04-25T10:01:28.629531+0000","last_fullsized":"2024-04-25T10:01:28.629531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633354+0000","last_change":"2024-04-25T10:01:28.633354+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633354+0000","last_undegraded":"2024-04-25T10:01:28.633354+0000","last_fullsized":"2024-04-25T10:01:28.633354+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634748+0000","last_change":"2024-04-25T10:01:28.634748+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634748+0000","last_undegraded":"2024-04-25T10:01:28.634748+0000","last_fullsized":"2024-04-25T10:01:28.634748+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631665+0000","last_change":"2024-04-25T10:01:28.631665+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631665+0000","last_undegraded":"2024-04-25T10:01:28.631665+0000","last_fullsized":"2024-04-25T10:01:28.631665+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634367+0000","last_change":"2024-04-25T10:01:28.634367+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634367+0000","last_undegraded":"2024-04-25T10:01:28.634367+0000","last_fullsized":"2024-04-25T10:01:28.634367+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631023+0000","last_change":"2024-04-25T10:01:28.631023+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631023+0000","last_undegraded":"2024-04-25T10:01:28.631023+0000","last_fullsized":"2024-04-25T10:01:28.631023+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631167+0000","last_change":"2024-04-25T10:01:28.631167+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631167+0000","last_undegraded":"2024-04-25T10:01:28.631167+0000","last_fullsized":"2024-04-25T10:01:28.631167+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629895+0000","last_change":"2024-04-25T10:01:28.629895+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629895+0000","last_undegraded":"2024-04-25T10:01:28.629895+0000","last_fullsized":"2024-04-25T10:01:28.629895+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628825+0000","last_change":"2024-04-25T10:01:28.628825+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628825+0000","last_undegraded":"2024-04-25T10:01:28.628825+0000","last_fullsized":"2024-04-25T10:01:28.628825+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.637120+0000","last_change":"2024-04-25T10:01:28.637120+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.637120+0000","last_undegraded":"2024-04-25T10:01:28.637120+0000","last_fullsized":"2024-04-25T10:01:28.637120+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632165+0000","last_change":"2024-04-25T10:01:28.632165+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632165+0000","last_undegraded":"2024-04-25T10:01:28.632165+0000","last_fullsized":"2024-04-25T10:01:28.632165+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629573+0000","last_change":"2024-04-25T10:01:28.629573+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629573+0000","last_undegraded":"2024-04-25T10:01:28.629573+0000","last_fullsized":"2024-04-25T10:01:28.629573+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632296+0000","last_change":"2024-04-25T10:01:28.632296+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632296+0000","last_undegraded":"2024-04-25T10:01:28.632296+0000","last_fullsized":"2024-04-25T10:01:28.632296+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634707+0000","last_change":"2024-04-25T10:01:28.634707+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634707+0000","last_undegraded":"2024-04-25T10:01:28.634707+0000","last_fullsized":"2024-04-25T10:01:28.634707+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632907+0000","last_change":"2024-04-25T10:01:28.632907+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632907+0000","last_undegraded":"2024-04-25T10:01:28.632907+0000","last_fullsized":"2024-04-25T10:01:28.632907+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645733+0000","last_change":"2024-04-25T10:01:28.644943+0000","last_active":"2024-04-25T10:01:28.645733+0000","last_peered":"2024-04-25T10:01:28.645733+0000","last_clean":"2024-04-25T10:01:28.645733+0000","last_became_active":"2024-04-25T10:01:28.644689+0000","last_became_peered":"2024-04-25T10:01:28.644689+0000","last_unstale":"2024-04-25T10:01:28.645733+0000","last_undegraded":"2024-04-25T10:01:28.645733+0000","last_fullsized":"2024-04-25T10:01:28.645733+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:48.121372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631082+0000","last_change":"2024-04-25T10:01:28.631082+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631082+0000","last_undegraded":"2024-04-25T10:01:28.631082+0000","last_fullsized":"2024-04-25T10:01:28.631082+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.647323+0000","last_change":"2024-04-25T10:01:28.645571+0000","last_active":"2024-04-25T10:01:28.647323+0000","last_peered":"2024-04-25T10:01:28.647323+0000","last_clean":"2024-04-25T10:01:28.647323+0000","last_became_active":"2024-04-25T10:01:28.645283+0000","last_became_peered":"2024-04-25T10:01:28.645283+0000","last_unstale":"2024-04-25T10:01:28.647323+0000","last_undegraded":"2024-04-25T10:01:28.647323+0000","last_fullsized":"2024-04-25T10:01:28.647323+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:11.425437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.633628+0000","last_change":"2024-04-25T10:01:28.633628+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.633628+0000","last_undegraded":"2024-04-25T10:01:28.633628+0000","last_fullsized":"2024-04-25T10:01:28.633628+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643814+0000","last_change":"2024-04-25T10:01:28.640421+0000","last_active":"2024-04-25T10:01:28.643814+0000","last_peered":"2024-04-25T10:01:28.643814+0000","last_clean":"2024-04-25T10:01:28.643814+0000","last_became_active":"2024-04-25T10:01:28.639536+0000","last_became_peered":"2024-04-25T10:01:28.639536+0000","last_unstale":"2024-04-25T10:01:28.643814+0000","last_undegraded":"2024-04-25T10:01:28.643814+0000","last_fullsized":"2024-04-25T10:01:28.643814+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.056666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.638331+0000","last_change":"2024-04-25T10:01:28.638331+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.638331+0000","last_undegraded":"2024-04-25T10:01:28.638331+0000","last_fullsized":"2024-04-25T10:01:28.638331+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643382+0000","last_change":"2024-04-25T10:01:28.642064+0000","last_active":"2024-04-25T10:01:28.643382+0000","last_peered":"2024-04-25T10:01:28.643382+0000","last_clean":"2024-04-25T10:01:28.643382+0000","last_became_active":"2024-04-25T10:01:28.641767+0000","last_became_peered":"2024-04-25T10:01:28.641767+0000","last_unstale":"2024-04-25T10:01:28.643382+0000","last_undegraded":"2024-04-25T10:01:28.643382+0000","last_fullsized":"2024-04-25T10:01:28.643382+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:28.715123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632603+0000","last_change":"2024-04-25T10:01:28.632603+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632603+0000","last_undegraded":"2024-04-25T10:01:28.632603+0000","last_fullsized":"2024-04-25T10:01:28.632603+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645590+0000","last_change":"2024-04-25T10:01:28.643613+0000","last_active":"2024-04-25T10:01:28.645590+0000","last_peered":"2024-04-25T10:01:28.645590+0000","last_clean":"2024-04-25T10:01:28.645590+0000","last_became_active":"2024-04-25T10:01:28.643326+0000","last_became_peered":"2024-04-25T10:01:28.643326+0000","last_unstale":"2024-04-25T10:01:28.645590+0000","last_undegraded":"2024-04-25T10:01:28.645590+0000","last_fullsized":"2024-04-25T10:01:28.645590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:47.363454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632282+0000","last_change":"2024-04-25T10:01:28.632282+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632282+0000","last_undegraded":"2024-04-25T10:01:28.632282+0000","last_fullsized":"2024-04-25T10:01:28.632282+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648286+0000","last_change":"2024-04-25T10:01:28.647013+0000","last_active":"2024-04-25T10:01:28.648286+0000","last_peered":"2024-04-25T10:01:28.648286+0000","last_clean":"2024-04-25T10:01:28.648286+0000","last_became_active":"2024-04-25T10:01:28.646667+0000","last_became_peered":"2024-04-25T10:01:28.646667+0000","last_unstale":"2024-04-25T10:01:28.648286+0000","last_undegraded":"2024-04-25T10:01:28.648286+0000","last_fullsized":"2024-04-25T10:01:28.648286+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:28.715123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634967+0000","last_change":"2024-04-25T10:01:28.634967+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634967+0000","last_undegraded":"2024-04-25T10:01:28.634967+0000","last_fullsized":"2024-04-25T10:01:28.634967+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643539+0000","last_change":"2024-04-25T10:01:28.639325+0000","last_active":"2024-04-25T10:01:28.643539+0000","last_peered":"2024-04-25T10:01:28.643539+0000","last_clean":"2024-04-25T10:01:28.643539+0000","last_became_active":"2024-04-25T10:01:28.639055+0000","last_became_peered":"2024-04-25T10:01:28.639055+0000","last_unstale":"2024-04-25T10:01:28.643539+0000","last_undegraded":"2024-04-25T10:01:28.643539+0000","last_fullsized":"2024-04-25T10:01:28.643539+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:09.888995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629531+0000","last_change":"2024-04-25T10:01:28.629531+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629531+0000","last_undegraded":"2024-04-25T10:01:28.629531+0000","last_fullsized":"2024-04-25T10:01:28.629531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.640085+0000","last_change":"2024-04-25T10:01:28.639418+0000","last_active":"2024-04-25T10:01:28.640085+0000","last_peered":"2024-04-25T10:01:28.640085+0000","last_clean":"2024-04-25T10:01:28.640085+0000","last_became_active":"2024-04-25T10:01:28.639134+0000","last_became_peered":"2024-04-25T10:01:28.639134+0000","last_unstale":"2024-04-25T10:01:28.640085+0000","last_undegraded":"2024-04-25T10:01:28.640085+0000","last_fullsized":"2024-04-25T10:01:28.640085+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:09.409126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631028+0000","last_change":"2024-04-25T10:01:28.631028+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631028+0000","last_undegraded":"2024-04-25T10:01:28.631028+0000","last_fullsized":"2024-04-25T10:01:28.631028+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.641745+0000","last_change":"2024-04-25T10:01:28.640498+0000","last_active":"2024-04-25T10:01:28.641745+0000","last_peered":"2024-04-25T10:01:28.641745+0000","last_clean":"2024-04-25T10:01:28.641745+0000","last_became_active":"2024-04-25T10:01:28.640186+0000","last_became_peered":"2024-04-25T10:01:28.640186+0000","last_unstale":"2024-04-25T10:01:28.641745+0000","last_undegraded":"2024-04-25T10:01:28.641745+0000","last_fullsized":"2024-04-25T10:01:28.641745+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:23.471932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632233+0000","last_change":"2024-04-25T10:01:28.632233+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632233+0000","last_undegraded":"2024-04-25T10:01:28.632233+0000","last_fullsized":"2024-04-25T10:01:28.632233+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648909+0000","last_change":"2024-04-25T10:01:28.647926+0000","last_active":"2024-04-25T10:01:28.648909+0000","last_peered":"2024-04-25T10:01:28.648909+0000","last_clean":"2024-04-25T10:01:28.648909+0000","last_became_active":"2024-04-25T10:01:28.646963+0000","last_became_peered":"2024-04-25T10:01:28.646963+0000","last_unstale":"2024-04-25T10:01:28.648909+0000","last_undegraded":"2024-04-25T10:01:28.648909+0000","last_fullsized":"2024-04-25T10:01:28.648909+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:56.486037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629097+0000","last_change":"2024-04-25T10:01:28.629097+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629097+0000","last_undegraded":"2024-04-25T10:01:28.629097+0000","last_fullsized":"2024-04-25T10:01:28.629097+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643792+0000","last_change":"2024-04-25T10:01:28.638314+0000","last_active":"2024-04-25T10:01:28.643792+0000","last_peered":"2024-04-25T10:01:28.643792+0000","last_clean":"2024-04-25T10:01:28.643792+0000","last_became_active":"2024-04-25T10:01:28.637960+0000","last_became_peered":"2024-04-25T10:01:28.637960+0000","last_unstale":"2024-04-25T10:01:28.643792+0000","last_undegraded":"2024-04-25T10:01:28.643792+0000","last_fullsized":"2024-04-25T10:01:28.643792+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.356021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628748+0000","last_change":"2024-04-25T10:01:28.628748+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628748+0000","last_undegraded":"2024-04-25T10:01:28.628748+0000","last_fullsized":"2024-04-25T10:01:28.628748+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643363+0000","last_change":"2024-04-25T10:01:28.639720+0000","last_active":"2024-04-25T10:01:28.643363+0000","last_peered":"2024-04-25T10:01:28.643363+0000","last_clean":"2024-04-25T10:01:28.643363+0000","last_became_active":"2024-04-25T10:01:28.638956+0000","last_became_peered":"2024-04-25T10:01:28.638956+0000","last_unstale":"2024-04-25T10:01:28.643363+0000","last_undegraded":"2024-04-25T10:01:28.643363+0000","last_fullsized":"2024-04-25T10:01:28.643363+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:11.425437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635339+0000","last_change":"2024-04-25T10:01:28.635339+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635339+0000","last_undegraded":"2024-04-25T10:01:28.635339+0000","last_fullsized":"2024-04-25T10:01:28.635339+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648870+0000","last_change":"2024-04-25T10:01:28.644548+0000","last_active":"2024-04-25T10:01:28.648870+0000","last_peered":"2024-04-25T10:01:28.648870+0000","last_clean":"2024-04-25T10:01:28.648870+0000","last_became_active":"2024-04-25T10:01:28.643068+0000","last_became_peered":"2024-04-25T10:01:28.643068+0000","last_unstale":"2024-04-25T10:01:28.648870+0000","last_undegraded":"2024-04-25T10:01:28.648870+0000","last_fullsized":"2024-04-25T10:01:28.648870+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:28.602317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628719+0000","last_change":"2024-04-25T10:01:28.628719+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628719+0000","last_undegraded":"2024-04-25T10:01:28.628719+0000","last_fullsized":"2024-04-25T10:01:28.628719+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.642739+0000","last_change":"2024-04-25T10:01:28.641313+0000","last_active":"2024-04-25T10:01:28.642739+0000","last_peered":"2024-04-25T10:01:28.642739+0000","last_clean":"2024-04-25T10:01:28.642739+0000","last_became_active":"2024-04-25T10:01:28.641020+0000","last_became_peered":"2024-04-25T10:01:28.641020+0000","last_unstale":"2024-04-25T10:01:28.642739+0000","last_undegraded":"2024-04-25T10:01:28.642739+0000","last_fullsized":"2024-04-25T10:01:28.642739+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:47.363454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632145+0000","last_change":"2024-04-25T10:01:28.632145+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632145+0000","last_undegraded":"2024-04-25T10:01:28.632145+0000","last_fullsized":"2024-04-25T10:01:28.632145+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643497+0000","last_change":"2024-04-25T10:01:28.639344+0000","last_active":"2024-04-25T10:01:28.643497+0000","last_peered":"2024-04-25T10:01:28.643497+0000","last_clean":"2024-04-25T10:01:28.643497+0000","last_became_active":"2024-04-25T10:01:28.639079+0000","last_became_peered":"2024-04-25T10:01:28.639079+0000","last_unstale":"2024-04-25T10:01:28.643497+0000","last_undegraded":"2024-04-25T10:01:28.643497+0000","last_fullsized":"2024-04-25T10:01:28.643497+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:13.447365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632144+0000","last_change":"2024-04-25T10:01:28.632144+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632144+0000","last_undegraded":"2024-04-25T10:01:28.632144+0000","last_fullsized":"2024-04-25T10:01:28.632144+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.644323+0000","last_change":"2024-04-25T10:01:28.642872+0000","last_active":"2024-04-25T10:01:28.644323+0000","last_peered":"2024-04-25T10:01:28.644323+0000","last_clean":"2024-04-25T10:01:28.644323+0000","last_became_active":"2024-04-25T10:01:28.642250+0000","last_became_peered":"2024-04-25T10:01:28.642250+0000","last_unstale":"2024-04-25T10:01:28.644323+0000","last_undegraded":"2024-04-25T10:01:28.644323+0000","last_fullsized":"2024-04-25T10:01:28.644323+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:09.710270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630415+0000","last_change":"2024-04-25T10:01:28.630415+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630415+0000","last_undegraded":"2024-04-25T10:01:28.630415+0000","last_fullsized":"2024-04-25T10:01:28.630415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.644964+0000","last_change":"2024-04-25T10:01:28.643788+0000","last_active":"2024-04-25T10:01:28.644964+0000","last_peered":"2024-04-25T10:01:28.644964+0000","last_clean":"2024-04-25T10:01:28.644964+0000","last_became_active":"2024-04-25T10:01:28.643521+0000","last_became_peered":"2024-04-25T10:01:28.643521+0000","last_unstale":"2024-04-25T10:01:28.644964+0000","last_undegraded":"2024-04-25T10:01:28.644964+0000","last_fullsized":"2024-04-25T10:01:28.644964+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:10.820229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628642+0000","last_change":"2024-04-25T10:01:28.628642+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628642+0000","last_undegraded":"2024-04-25T10:01:28.628642+0000","last_fullsized":"2024-04-25T10:01:28.628642+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643193+0000","last_change":"2024-04-25T10:01:28.640275+0000","last_active":"2024-04-25T10:01:28.643193+0000","last_peered":"2024-04-25T10:01:28.643193+0000","last_clean":"2024-04-25T10:01:28.643193+0000","last_became_active":"2024-04-25T10:01:28.640015+0000","last_became_peered":"2024-04-25T10:01:28.640015+0000","last_unstale":"2024-04-25T10:01:28.643193+0000","last_undegraded":"2024-04-25T10:01:28.643193+0000","last_fullsized":"2024-04-25T10:01:28.643193+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:03.858446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.635398+0000","last_change":"2024-04-25T10:01:28.635398+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.635398+0000","last_undegraded":"2024-04-25T10:01:28.635398+0000","last_fullsized":"2024-04-25T10:01:28.635398+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645355+0000","last_change":"2024-04-25T10:01:28.644201+0000","last_active":"2024-04-25T10:01:28.645355+0000","last_peered":"2024-04-25T10:01:28.645355+0000","last_clean":"2024-04-25T10:01:28.645355+0000","last_became_active":"2024-04-25T10:01:28.643895+0000","last_became_peered":"2024-04-25T10:01:28.643895+0000","last_unstale":"2024-04-25T10:01:28.645355+0000","last_undegraded":"2024-04-25T10:01:28.645355+0000","last_fullsized":"2024-04-25T10:01:28.645355+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:00.356021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632929+0000","last_change":"2024-04-25T10:01:28.632929+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632929+0000","last_undegraded":"2024-04-25T10:01:28.632929+0000","last_fullsized":"2024-04-25T10:01:28.632929+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643746+0000","last_change":"2024-04-25T10:01:28.640556+0000","last_active":"2024-04-25T10:01:28.643746+0000","last_peered":"2024-04-25T10:01:28.643746+0000","last_clean":"2024-04-25T10:01:28.643746+0000","last_became_active":"2024-04-25T10:01:28.640292+0000","last_became_peered":"2024-04-25T10:01:28.640292+0000","last_unstale":"2024-04-25T10:01:28.643746+0000","last_undegraded":"2024-04-25T10:01:28.643746+0000","last_fullsized":"2024-04-25T10:01:28.643746+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:31.176590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634251+0000","last_change":"2024-04-25T10:01:28.634251+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634251+0000","last_undegraded":"2024-04-25T10:01:28.634251+0000","last_fullsized":"2024-04-25T10:01:28.634251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648139+0000","last_change":"2024-04-25T10:01:28.641737+0000","last_active":"2024-04-25T10:01:28.648139+0000","last_peered":"2024-04-25T10:01:28.648139+0000","last_clean":"2024-04-25T10:01:28.648139+0000","last_became_active":"2024-04-25T10:01:28.641451+0000","last_became_peered":"2024-04-25T10:01:28.641451+0000","last_unstale":"2024-04-25T10:01:28.648139+0000","last_undegraded":"2024-04-25T10:01:28.648139+0000","last_fullsized":"2024-04-25T10:01:28.648139+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:02.288325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634539+0000","last_change":"2024-04-25T10:01:28.634539+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634539+0000","last_undegraded":"2024-04-25T10:01:28.634539+0000","last_fullsized":"2024-04-25T10:01:28.634539+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645115+0000","last_change":"2024-04-25T10:01:28.644516+0000","last_active":"2024-04-25T10:01:28.645115+0000","last_peered":"2024-04-25T10:01:28.645115+0000","last_clean":"2024-04-25T10:01:28.645115+0000","last_became_active":"2024-04-25T10:01:28.644155+0000","last_became_peered":"2024-04-25T10:01:28.644155+0000","last_unstale":"2024-04-25T10:01:28.645115+0000","last_undegraded":"2024-04-25T10:01:28.645115+0000","last_fullsized":"2024-04-25T10:01:28.645115+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:13.085301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.636198+0000","last_change":"2024-04-25T10:01:28.636198+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.636198+0000","last_undegraded":"2024-04-25T10:01:28.636198+0000","last_fullsized":"2024-04-25T10:01:28.636198+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.641997+0000","last_change":"2024-04-25T10:01:28.638744+0000","last_active":"2024-04-25T10:01:28.641997+0000","last_peered":"2024-04-25T10:01:28.641997+0000","last_clean":"2024-04-25T10:01:28.641997+0000","last_became_active":"2024-04-25T10:01:28.638312+0000","last_became_peered":"2024-04-25T10:01:28.638312+0000","last_unstale":"2024-04-25T10:01:28.641997+0000","last_undegraded":"2024-04-25T10:01:28.641997+0000","last_fullsized":"2024-04-25T10:01:28.641997+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:41.741278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.632131+0000","last_change":"2024-04-25T10:01:28.632131+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.632131+0000","last_undegraded":"2024-04-25T10:01:28.632131+0000","last_fullsized":"2024-04-25T10:01:28.632131+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.648947+0000","last_change":"2024-04-25T10:01:28.647785+0000","last_active":"2024-04-25T10:01:28.648947+0000","last_peered":"2024-04-25T10:01:28.648947+0000","last_clean":"2024-04-25T10:01:28.648947+0000","last_became_active":"2024-04-25T10:01:28.647474+0000","last_became_peered":"2024-04-25T10:01:28.647474+0000","last_unstale":"2024-04-25T10:01:28.648947+0000","last_undegraded":"2024-04-25T10:01:28.648947+0000","last_fullsized":"2024-04-25T10:01:28.648947+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:21.347290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629400+0000","last_change":"2024-04-25T10:01:28.629400+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629400+0000","last_undegraded":"2024-04-25T10:01:28.629400+0000","last_fullsized":"2024-04-25T10:01:28.629400+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645364+0000","last_change":"2024-04-25T10:01:28.642073+0000","last_active":"2024-04-25T10:01:28.645364+0000","last_peered":"2024-04-25T10:01:28.645364+0000","last_clean":"2024-04-25T10:01:28.645364+0000","last_became_active":"2024-04-25T10:01:28.641446+0000","last_became_peered":"2024-04-25T10:01:28.641446+0000","last_unstale":"2024-04-25T10:01:28.645364+0000","last_undegraded":"2024-04-25T10:01:28.645364+0000","last_fullsized":"2024-04-25T10:01:28.645364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:09.409126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.629182+0000","last_change":"2024-04-25T10:01:28.629182+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.629182+0000","last_undegraded":"2024-04-25T10:01:28.629182+0000","last_fullsized":"2024-04-25T10:01:28.629182+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.645618+0000","last_change":"2024-04-25T10:01:28.643918+0000","last_active":"2024-04-25T10:01:28.645618+0000","last_peered":"2024-04-25T10:01:28.645618+0000","last_clean":"2024-04-25T10:01:28.645618+0000","last_became_active":"2024-04-25T10:01:28.643575+0000","last_became_peered":"2024-04-25T10:01:28.643575+0000","last_unstale":"2024-04-25T10:01:28.645618+0000","last_undegraded":"2024-04-25T10:01:28.645618+0000","last_fullsized":"2024-04-25T10:01:28.645618+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:10.820229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.634919+0000","last_change":"2024-04-25T10:01:28.634919+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.634919+0000","last_undegraded":"2024-04-25T10:01:28.634919+0000","last_fullsized":"2024-04-25T10:01:28.634919+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.637809+0000","last_change":"2024-04-25T10:01:28.633829+0000","last_active":"2024-04-25T10:01:28.637809+0000","last_peered":"2024-04-25T10:01:28.637809+0000","last_clean":"2024-04-25T10:01:28.637809+0000","last_became_active":"2024-04-25T10:01:28.632994+0000","last_became_peered":"2024-04-25T10:01:28.632994+0000","last_unstale":"2024-04-25T10:01:28.637809+0000","last_undegraded":"2024-04-25T10:01:28.637809+0000","last_fullsized":"2024-04-25T10:01:28.637809+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:57.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630716+0000","last_change":"2024-04-25T10:01:28.630716+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630716+0000","last_undegraded":"2024-04-25T10:01:28.630716+0000","last_fullsized":"2024-04-25T10:01:28.630716+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.649655+0000","last_change":"2024-04-25T10:01:28.649336+0000","last_active":"2024-04-25T10:01:28.649655+0000","last_peered":"2024-04-25T10:01:28.649655+0000","last_clean":"2024-04-25T10:01:28.649655+0000","last_became_active":"2024-04-25T10:01:28.649067+0000","last_became_peered":"2024-04-25T10:01:28.649067+0000","last_unstale":"2024-04-25T10:01:28.649655+0000","last_undegraded":"2024-04-25T10:01:28.649655+0000","last_fullsized":"2024-04-25T10:01:28.649655+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:54.846448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631752+0000","last_change":"2024-04-25T10:01:28.631752+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631752+0000","last_undegraded":"2024-04-25T10:01:28.631752+0000","last_fullsized":"2024-04-25T10:01:28.631752+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.643332+0000","last_change":"2024-04-25T10:01:28.637991+0000","last_active":"2024-04-25T10:01:28.643332+0000","last_peered":"2024-04-25T10:01:28.643332+0000","last_clean":"2024-04-25T10:01:28.643332+0000","last_became_active":"2024-04-25T10:01:28.637710+0000","last_became_peered":"2024-04-25T10:01:28.637710+0000","last_unstale":"2024-04-25T10:01:28.643332+0000","last_undegraded":"2024-04-25T10:01:28.643332+0000","last_fullsized":"2024-04-25T10:01:28.643332+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:32.105344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.630788+0000","last_change":"2024-04-25T10:01:28.630788+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.630788+0000","last_undegraded":"2024-04-25T10:01:28.630788+0000","last_fullsized":"2024-04-25T10:01:28.630788+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.639587+0000","last_change":"2024-04-25T10:01:28.637799+0000","last_active":"2024-04-25T10:01:28.639587+0000","last_peered":"2024-04-25T10:01:28.639587+0000","last_clean":"2024-04-25T10:01:28.639587+0000","last_became_active":"2024-04-25T10:01:28.637491+0000","last_became_peered":"2024-04-25T10:01:28.637491+0000","last_unstale":"2024-04-25T10:01:28.639587+0000","last_undegraded":"2024-04-25T10:01:28.639587+0000","last_fullsized":"2024-04-25T10:01:28.639587+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:10.916547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.628958+0000","last_change":"2024-04-25T10:01:28.628958+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.628958+0000","last_undegraded":"2024-04-25T10:01:28.628958+0000","last_fullsized":"2024-04-25T10:01:28.628958+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.647431+0000","last_change":"2024-04-25T10:01:28.645758+0000","last_active":"2024-04-25T10:01:28.647431+0000","last_peered":"2024-04-25T10:01:28.647431+0000","last_clean":"2024-04-25T10:01:28.647431+0000","last_became_active":"2024-04-25T10:01:28.645393+0000","last_became_peered":"2024-04-25T10:01:28.645393+0000","last_unstale":"2024-04-25T10:01:28.647431+0000","last_undegraded":"2024-04-25T10:01:28.647431+0000","last_fullsized":"2024-04-25T10:01:28.647431+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.056666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-04-25T10:01:28.631251+0000","last_change":"2024-04-25T10:01:28.631251+0000","last_active":"2024-04-25T10:01:28.621726+0000","last_peered":"2024-04-25T10:01:28.621726+0000","last_clean":"2024-04-25T10:01:28.621726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:28.631251+0000","last_undegraded":"2024-04-25T10:01:28.631251+0000","last_fullsized":"2024-04-25T10:01:28.631251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.638771+0000","last_change":"2024-04-25T10:01:28.636229+0000","last_active":"2024-04-25T10:01:28.638771+0000","last_peered":"2024-04-25T10:01:28.638771+0000","last_clean":"2024-04-25T10:01:28.638771+0000","last_became_active":"2024-04-25T10:01:28.635741+0000","last_became_peered":"2024-04-25T10:01:28.635741+0000","last_unstale":"2024-04-25T10:01:28.638771+0000","last_undegraded":"2024-04-25T10:01:28.638771+0000","last_fullsized":"2024-04-25T10:01:28.638771+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:28.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":388,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T10:01:28.631752+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:01:28.631752+0000","last_peered":"2024-04-25T10:01:28.631752+0000","last_clean":"2024-04-25T10:01:28.631752+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:01:28.631752+0000","last_undegraded":"2024-04-25T10:01:28.631752+0000","last_fullsized":"2024-04-25T10:01:28.631752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29156,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342684,"statfs":{"total":96636764160,"available":96606908416,"internally_reserved":0,"allocated":2183168,"data_stored":1329872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46624,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325216,"statfs":{"total":96636764160,"available":96589021184,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54928,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316912,"statfs":{"total":96636764160,"available":96580517888,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54944,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316896,"statfs":{"total":96636764160,"available":96580501504,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50772,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321068,"statfs":{"total":96636764160,"available":96584773632,"internally_reserved":0,"allocated":1576960,"data_stored":732845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50784,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321056,"statfs":{"total":96636764160,"available":96584761344,"internally_reserved":0,"allocated":1589248,"data_stored":739504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51348,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320492,"statfs":{"total":96636764160,"available":96584183808,"internally_reserved":0,"allocated":2170880,"data_stored":1323213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:37.097 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:37.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.206+0000 7fe494d7e640 1 Processor -- start 2024-04-25T10:01:37.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.206+0000 7fe494d7e640 1 -- start start 2024-04-25T10:01:37.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.207+0000 7fe494d7e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4900d1020 0x7fe4900d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.207+0000 7fe494d7e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4900b97a0 con 0x7fe4900b2cb0 2024-04-25T10:01:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.207+0000 7fe494d7e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4900d1c50 con 0x7fe49015dbe0 2024-04-25T10:01:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.208+0000 7fe494d7e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49015dfc0 con 0x7fe4900d1020 2024-04-25T10:01:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.208+0000 7fe48ed9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4900d1020 0x7fe4900d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.208+0000 7fe48dd9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe4900b2cb0 0x7fe4900d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40478/0 (socket says 172.21.15.19:40478) 2024-04-25T10:01:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.208+0000 7fe48dd9b640 1 -- 172.21.15.19:0/2692466809 learned_addr learned my addr 172.21.15.19:0/2692466809 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.208+0000 7fe48d59a640 1 -- 172.21.15.19:0/2692466809 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328676728 0 0) 0x7fe4900b97a0 con 0x7fe4900b2cb0 2024-04-25T10:01:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.208+0000 7fe48d59a640 1 -- 172.21.15.19:0/2692466809 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe47c003520 con 0x7fe4900b2cb0 2024-04-25T10:01:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.208+0000 7fe48ed9d640 1 -- 172.21.15.19:0/2692466809 >> v1:172.21.15.112:6789/0 conn(0x7fe49015dbe0 legacy=0x7fe4900d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.208+0000 7fe48ed9d640 1 -- 172.21.15.19:0/2692466809 >> v1:172.21.15.19:6789/0 conn(0x7fe4900b2cb0 legacy=0x7fe4900d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.208+0000 7fe48ed9d640 1 -- 172.21.15.19:0/2692466809 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4900d1f30 con 0x7fe4900d1020 2024-04-25T10:01:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.209+0000 7fe48ed9d640 1 --2- 172.21.15.19:0/2692466809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4900d1020 0x7fe4900d2e40 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fe480009d90 tx=0x7fe48002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4ffbb332ecae97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.209+0000 7fe48d59a640 1 -- 172.21.15.19:0/2692466809 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480007740 con 0x7fe4900d1020 2024-04-25T10:01:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.209+0000 7fe48d59a640 1 -- 172.21.15.19:0/2692466809 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4800078e0 con 0x7fe4900d1020 2024-04-25T10:01:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.209+0000 7fe48d59a640 1 -- 172.21.15.19:0/2692466809 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480007cb0 con 0x7fe4900d1020 2024-04-25T10:01:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.209+0000 7fe494d7e640 1 -- 172.21.15.19:0/2692466809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4900d1020 msgr2=0x7fe4900d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.209+0000 7fe494d7e640 1 --2- 172.21.15.19:0/2692466809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4900d1020 0x7fe4900d2e40 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fe480009d90 tx=0x7fe48002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.210+0000 7fe494d7e640 1 -- 172.21.15.19:0/2692466809 shutdown_connections 2024-04-25T10:01:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.210+0000 7fe494d7e640 1 --2- 172.21.15.19:0/2692466809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4900d1020 0x7fe4900d2e40 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.210+0000 7fe494d7e640 1 -- 172.21.15.19:0/2692466809 >> 172.21.15.19:0/2692466809 conn(0x7fe490061a20 msgr2=0x7fe490061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.210+0000 7fe494d7e640 1 -- 172.21.15.19:0/2692466809 shutdown_connections 2024-04-25T10:01:37.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.210+0000 7fe494d7e640 1 -- 172.21.15.19:0/2692466809 wait complete. 2024-04-25T10:01:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.211+0000 7fe494d7e640 1 Processor -- start 2024-04-25T10:01:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.211+0000 7fe494d7e640 1 -- start start 2024-04-25T10:01:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe494d7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4900b2cb0 0x7fe49014be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe494d7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4900d1020 0x7fe490157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe494d7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49015dbe0 0x7fe490157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe494d7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900b92a0 con 0x7fe49015dbe0 2024-04-25T10:01:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe494d7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900d1a00 con 0x7fe4900b2cb0 2024-04-25T10:01:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe494d7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900b9420 con 0x7fe4900d1020 2024-04-25T10:01:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe48ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49015dbe0 0x7fe490157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe48ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49015dbe0 0x7fe490157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33610/0 (socket says 172.21.15.19:33610) 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe48ed9d640 1 -- 172.21.15.19:0/2486409768 learned_addr learned my addr 172.21.15.19:0/2486409768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe48dd9b640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4900d1020 0x7fe490157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe48e59c640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4900b2cb0 0x7fe49014be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe48ed9d640 1 -- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4900d1020 msgr2=0x7fe490157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe48ed9d640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4900d1020 0x7fe490157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe48ed9d640 1 -- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4900b2cb0 msgr2=0x7fe49014be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe48ed9d640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4900b2cb0 0x7fe49014be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe48ed9d640 1 -- 172.21.15.19:0/2486409768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4900b3f60 con 0x7fe49015dbe0 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.212+0000 7fe48ed9d640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49015dbe0 0x7fe490157fc0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fe48002ace0 tx=0x7fe480004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.213+0000 7fe486ffd640 1 -- 172.21.15.19:0/2486409768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480040070 con 0x7fe49015dbe0 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.213+0000 7fe486ffd640 1 -- 172.21.15.19:0/2486409768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe480031bc0 con 0x7fe49015dbe0 2024-04-25T10:01:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.213+0000 7fe486ffd640 1 -- 172.21.15.19:0/2486409768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480009450 con 0x7fe49015dbe0 2024-04-25T10:01:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.213+0000 7fe494d7e640 1 -- 172.21.15.19:0/2486409768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4900b4240 con 0x7fe49015dbe0 2024-04-25T10:01:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.213+0000 7fe494d7e640 1 -- 172.21.15.19:0/2486409768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4900b4520 con 0x7fe49015dbe0 2024-04-25T10:01:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.214+0000 7fe494d7e640 1 -- 172.21.15.19:0/2486409768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe490073990 con 0x7fe49015dbe0 2024-04-25T10:01:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.214+0000 7fe486ffd640 1 -- 172.21.15.19:0/2486409768 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe480039040 con 0x7fe49015dbe0 2024-04-25T10:01:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.214+0000 7fe486ffd640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe47006b500 0x7fe47006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.215+0000 7fe486ffd640 1 -- 172.21.15.19:0/2486409768 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe4800aba30 con 0x7fe49015dbe0 2024-04-25T10:01:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.215+0000 7fe48e59c640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe47006b500 0x7fe47006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.215+0000 7fe48e59c640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe47006b500 0x7fe47006d9c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe4880016e0 tx=0x7fe488009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.219+0000 7fe486ffd640 1 -- 172.21.15.19:0/2486409768 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe48007a0d0 con 0x7fe49015dbe0 2024-04-25T10:01:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.398+0000 7fe494d7e640 1 -- 172.21.15.19:0/2486409768 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4900cb5d0 con 0x7fe47006b500 2024-04-25T10:01:37.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.404+0000 7fe486ffd640 1 -- 172.21.15.19:0/2486409768 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7fe488001cc0 con 0x7fe47006b500 2024-04-25T10:01:37.412 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:01:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 -- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe47006b500 msgr2=0x7fe47006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe47006b500 0x7fe47006d9c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe4880016e0 tx=0x7fe488009040 comp rx=0 tx=0).stop 2024-04-25T10:01:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 -- 172.21.15.19:0/2486409768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49015dbe0 msgr2=0x7fe490157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49015dbe0 0x7fe490157fc0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fe48002ace0 tx=0x7fe480004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 -- 172.21.15.19:0/2486409768 shutdown_connections 2024-04-25T10:01:37.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe47006b500 0x7fe47006d9c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe49015dbe0 0x7fe490157fc0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4900d1020 0x7fe490157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 --2- 172.21.15.19:0/2486409768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4900b2cb0 0x7fe49014be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 -- 172.21.15.19:0/2486409768 >> 172.21.15.19:0/2486409768 conn(0x7fe490061a20 msgr2=0x7fe490158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 -- 172.21.15.19:0/2486409768 shutdown_connections 2024-04-25T10:01:37.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.407+0000 7fe494d7e640 1 -- 172.21.15.19:0/2486409768 wait complete. 2024-04-25T10:01:37.423 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-25T10:01:37.028874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385472,"kb_used_data":13832,"kb_used_omap":12,"kb_used_meta":371507,"kb_avail":754589248,"statfs":{"total":773094113280,"available":772699389952,"internally_reserved":0,"allocated":14163968,"data_stored":7212460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":380423730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002166"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.080184+0000","last_change":"2024-04-25T10:01:29.656780+0000","last_active":"2024-04-25T10:01:31.080184+0000","last_peered":"2024-04-25T10:01:31.080184+0000","last_clean":"2024-04-25T10:01:31.080184+0000","last_became_active":"2024-04-25T10:01:29.656478+0000","last_became_peered":"2024-04-25T10:01:29.656478+0000","last_unstale":"2024-04-25T10:01:31.080184+0000","last_undegraded":"2024-04-25T10:01:31.080184+0000","last_fullsized":"2024-04-25T10:01:31.080184+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:25.489764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.181221+0000","last_change":"2024-04-25T10:01:29.653638+0000","last_active":"2024-04-25T10:01:32.181221+0000","last_peered":"2024-04-25T10:01:32.181221+0000","last_clean":"2024-04-25T10:01:32.181221+0000","last_became_active":"2024-04-25T10:01:29.653281+0000","last_became_peered":"2024-04-25T10:01:29.653281+0000","last_unstale":"2024-04-25T10:01:32.181221+0000","last_undegraded":"2024-04-25T10:01:32.181221+0000","last_fullsized":"2024-04-25T10:01:32.181221+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:44.122015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659896+0000","last_change":"2024-04-25T10:01:29.665774+0000","last_active":"2024-04-25T10:01:30.659896+0000","last_peered":"2024-04-25T10:01:30.659896+0000","last_clean":"2024-04-25T10:01:30.659896+0000","last_became_active":"2024-04-25T10:01:29.665217+0000","last_became_peered":"2024-04-25T10:01:29.665217+0000","last_unstale":"2024-04-25T10:01:30.659896+0000","last_undegraded":"2024-04-25T10:01:30.659896+0000","last_fullsized":"2024-04-25T10:01:30.659896+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:12.431961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180982+0000","last_change":"2024-04-25T10:01:29.655286+0000","last_active":"2024-04-25T10:01:32.180982+0000","last_peered":"2024-04-25T10:01:32.180982+0000","last_clean":"2024-04-25T10:01:32.180982+0000","last_became_active":"2024-04-25T10:01:29.655013+0000","last_became_peered":"2024-04-25T10:01:29.655013+0000","last_unstale":"2024-04-25T10:01:32.180982+0000","last_undegraded":"2024-04-25T10:01:32.180982+0000","last_fullsized":"2024-04-25T10:01:32.180982+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:37.245128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180583+0000","last_change":"2024-04-25T10:01:29.651478+0000","last_active":"2024-04-25T10:01:32.180583+0000","last_peered":"2024-04-25T10:01:32.180583+0000","last_clean":"2024-04-25T10:01:32.180583+0000","last_became_active":"2024-04-25T10:01:29.651195+0000","last_became_peered":"2024-04-25T10:01:29.651195+0000","last_unstale":"2024-04-25T10:01:32.180583+0000","last_undegraded":"2024-04-25T10:01:32.180583+0000","last_fullsized":"2024-04-25T10:01:32.180583+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:20.353286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660061+0000","last_change":"2024-04-25T10:01:29.667511+0000","last_active":"2024-04-25T10:01:30.660061+0000","last_peered":"2024-04-25T10:01:30.660061+0000","last_clean":"2024-04-25T10:01:30.660061+0000","last_became_active":"2024-04-25T10:01:29.667186+0000","last_became_peered":"2024-04-25T10:01:29.667186+0000","last_unstale":"2024-04-25T10:01:30.660061+0000","last_undegraded":"2024-04-25T10:01:30.660061+0000","last_fullsized":"2024-04-25T10:01:30.660061+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:19.039051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660609+0000","last_change":"2024-04-25T10:01:29.670987+0000","last_active":"2024-04-25T10:01:30.660609+0000","last_peered":"2024-04-25T10:01:30.660609+0000","last_clean":"2024-04-25T10:01:30.660609+0000","last_became_active":"2024-04-25T10:01:29.670695+0000","last_became_peered":"2024-04-25T10:01:29.670695+0000","last_unstale":"2024-04-25T10:01:30.660609+0000","last_undegraded":"2024-04-25T10:01:30.660609+0000","last_fullsized":"2024-04-25T10:01:30.660609+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:13.472346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.912470+0000","last_change":"2024-04-25T10:01:29.655093+0000","last_active":"2024-04-25T10:01:31.912470+0000","last_peered":"2024-04-25T10:01:31.912470+0000","last_clean":"2024-04-25T10:01:31.912470+0000","last_became_active":"2024-04-25T10:01:29.654795+0000","last_became_peered":"2024-04-25T10:01:29.654795+0000","last_unstale":"2024-04-25T10:01:31.912470+0000","last_undegraded":"2024-04-25T10:01:31.912470+0000","last_fullsized":"2024-04-25T10:01:31.912470+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:10.716794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660419+0000","last_change":"2024-04-25T10:01:29.659466+0000","last_active":"2024-04-25T10:01:30.660419+0000","last_peered":"2024-04-25T10:01:30.660419+0000","last_clean":"2024-04-25T10:01:30.660419+0000","last_became_active":"2024-04-25T10:01:29.658703+0000","last_became_peered":"2024-04-25T10:01:29.658703+0000","last_unstale":"2024-04-25T10:01:30.660419+0000","last_undegraded":"2024-04-25T10:01:30.660419+0000","last_fullsized":"2024-04-25T10:01:30.660419+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:06.227231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180495+0000","last_change":"2024-04-25T10:01:29.657570+0000","last_active":"2024-04-25T10:01:32.180495+0000","last_peered":"2024-04-25T10:01:32.180495+0000","last_clean":"2024-04-25T10:01:32.180495+0000","last_became_active":"2024-04-25T10:01:29.657234+0000","last_became_peered":"2024-04-25T10:01:29.657234+0000","last_unstale":"2024-04-25T10:01:32.180495+0000","last_undegraded":"2024-04-25T10:01:32.180495+0000","last_fullsized":"2024-04-25T10:01:32.180495+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:43.845601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.181063+0000","last_change":"2024-04-25T10:01:29.662390+0000","last_active":"2024-04-25T10:01:32.181063+0000","last_peered":"2024-04-25T10:01:32.181063+0000","last_clean":"2024-04-25T10:01:32.181063+0000","last_became_active":"2024-04-25T10:01:29.662112+0000","last_became_peered":"2024-04-25T10:01:29.662112+0000","last_unstale":"2024-04-25T10:01:32.181063+0000","last_undegraded":"2024-04-25T10:01:32.181063+0000","last_fullsized":"2024-04-25T10:01:32.181063+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:19.341958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.080131+0000","last_change":"2024-04-25T10:01:29.656492+0000","last_active":"2024-04-25T10:01:31.080131+0000","last_peered":"2024-04-25T10:01:31.080131+0000","last_clean":"2024-04-25T10:01:31.080131+0000","last_became_active":"2024-04-25T10:01:29.656087+0000","last_became_peered":"2024-04-25T10:01:29.656087+0000","last_unstale":"2024-04-25T10:01:31.080131+0000","last_undegraded":"2024-04-25T10:01:31.080131+0000","last_fullsized":"2024-04-25T10:01:31.080131+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:34.797587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660278+0000","last_change":"2024-04-25T10:01:29.667000+0000","last_active":"2024-04-25T10:01:30.660278+0000","last_peered":"2024-04-25T10:01:30.660278+0000","last_clean":"2024-04-25T10:01:30.660278+0000","last_became_active":"2024-04-25T10:01:29.666690+0000","last_became_peered":"2024-04-25T10:01:29.666690+0000","last_unstale":"2024-04-25T10:01:30.660278+0000","last_undegraded":"2024-04-25T10:01:30.660278+0000","last_fullsized":"2024-04-25T10:01:30.660278+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:15.798988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.079858+0000","last_change":"2024-04-25T10:01:29.663292+0000","last_active":"2024-04-25T10:01:31.079858+0000","last_peered":"2024-04-25T10:01:31.079858+0000","last_clean":"2024-04-25T10:01:31.079858+0000","last_became_active":"2024-04-25T10:01:29.663004+0000","last_became_peered":"2024-04-25T10:01:29.663004+0000","last_unstale":"2024-04-25T10:01:31.079858+0000","last_undegraded":"2024-04-25T10:01:31.079858+0000","last_fullsized":"2024-04-25T10:01:31.079858+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:07.333973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.079768+0000","last_change":"2024-04-25T10:01:29.661162+0000","last_active":"2024-04-25T10:01:31.079768+0000","last_peered":"2024-04-25T10:01:31.079768+0000","last_clean":"2024-04-25T10:01:31.079768+0000","last_became_active":"2024-04-25T10:01:29.660857+0000","last_became_peered":"2024-04-25T10:01:29.660857+0000","last_unstale":"2024-04-25T10:01:31.079768+0000","last_undegraded":"2024-04-25T10:01:31.079768+0000","last_fullsized":"2024-04-25T10:01:31.079768+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:38.088019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659877+0000","last_change":"2024-04-25T10:01:29.659227+0000","last_active":"2024-04-25T10:01:30.659877+0000","last_peered":"2024-04-25T10:01:30.659877+0000","last_clean":"2024-04-25T10:01:30.659877+0000","last_became_active":"2024-04-25T10:01:29.658937+0000","last_became_peered":"2024-04-25T10:01:29.658937+0000","last_unstale":"2024-04-25T10:01:30.659877+0000","last_undegraded":"2024-04-25T10:01:30.659877+0000","last_fullsized":"2024-04-25T10:01:30.659877+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:24.373763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660451+0000","last_change":"2024-04-25T10:01:29.659751+0000","last_active":"2024-04-25T10:01:30.660451+0000","last_peered":"2024-04-25T10:01:30.660451+0000","last_clean":"2024-04-25T10:01:30.660451+0000","last_became_active":"2024-04-25T10:01:29.659075+0000","last_became_peered":"2024-04-25T10:01:29.659075+0000","last_unstale":"2024-04-25T10:01:30.660451+0000","last_undegraded":"2024-04-25T10:01:30.660451+0000","last_fullsized":"2024-04-25T10:01:30.660451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:45.476919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180450+0000","last_change":"2024-04-25T10:01:29.656626+0000","last_active":"2024-04-25T10:01:32.180450+0000","last_peered":"2024-04-25T10:01:32.180450+0000","last_clean":"2024-04-25T10:01:32.180450+0000","last_became_active":"2024-04-25T10:01:29.656326+0000","last_became_peered":"2024-04-25T10:01:29.656326+0000","last_unstale":"2024-04-25T10:01:32.180450+0000","last_undegraded":"2024-04-25T10:01:32.180450+0000","last_fullsized":"2024-04-25T10:01:32.180450+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:32.281616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659721+0000","last_change":"2024-04-25T10:01:29.655980+0000","last_active":"2024-04-25T10:01:30.659721+0000","last_peered":"2024-04-25T10:01:30.659721+0000","last_clean":"2024-04-25T10:01:30.659721+0000","last_became_active":"2024-04-25T10:01:29.655718+0000","last_became_peered":"2024-04-25T10:01:29.655718+0000","last_unstale":"2024-04-25T10:01:30.659721+0000","last_undegraded":"2024-04-25T10:01:30.659721+0000","last_fullsized":"2024-04-25T10:01:30.659721+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:23.607420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180726+0000","last_change":"2024-04-25T10:01:29.664735+0000","last_active":"2024-04-25T10:01:32.180726+0000","last_peered":"2024-04-25T10:01:32.180726+0000","last_clean":"2024-04-25T10:01:32.180726+0000","last_became_active":"2024-04-25T10:01:29.664475+0000","last_became_peered":"2024-04-25T10:01:29.664475+0000","last_unstale":"2024-04-25T10:01:32.180726+0000","last_undegraded":"2024-04-25T10:01:32.180726+0000","last_fullsized":"2024-04-25T10:01:32.180726+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:56.596509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.181042+0000","last_change":"2024-04-25T10:01:29.653006+0000","last_active":"2024-04-25T10:01:32.181042+0000","last_peered":"2024-04-25T10:01:32.181042+0000","last_clean":"2024-04-25T10:01:32.181042+0000","last_became_active":"2024-04-25T10:01:29.652677+0000","last_became_peered":"2024-04-25T10:01:29.652677+0000","last_unstale":"2024-04-25T10:01:32.181042+0000","last_undegraded":"2024-04-25T10:01:32.181042+0000","last_fullsized":"2024-04-25T10:01:32.181042+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:24.373763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.079759+0000","last_change":"2024-04-25T10:01:29.658258+0000","last_active":"2024-04-25T10:01:31.079759+0000","last_peered":"2024-04-25T10:01:31.079759+0000","last_clean":"2024-04-25T10:01:31.079759+0000","last_became_active":"2024-04-25T10:01:29.657977+0000","last_became_peered":"2024-04-25T10:01:29.657977+0000","last_unstale":"2024-04-25T10:01:31.079759+0000","last_undegraded":"2024-04-25T10:01:31.079759+0000","last_fullsized":"2024-04-25T10:01:31.079759+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:02.892169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180843+0000","last_change":"2024-04-25T10:01:29.657759+0000","last_active":"2024-04-25T10:01:32.180843+0000","last_peered":"2024-04-25T10:01:32.180843+0000","last_clean":"2024-04-25T10:01:32.180843+0000","last_became_active":"2024-04-25T10:01:29.657263+0000","last_became_peered":"2024-04-25T10:01:29.657263+0000","last_unstale":"2024-04-25T10:01:32.180843+0000","last_undegraded":"2024-04-25T10:01:32.180843+0000","last_fullsized":"2024-04-25T10:01:32.180843+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:14.091825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659969+0000","last_change":"2024-04-25T10:01:29.659641+0000","last_active":"2024-04-25T10:01:30.659969+0000","last_peered":"2024-04-25T10:01:30.659969+0000","last_clean":"2024-04-25T10:01:30.659969+0000","last_became_active":"2024-04-25T10:01:29.657947+0000","last_became_peered":"2024-04-25T10:01:29.657947+0000","last_unstale":"2024-04-25T10:01:30.659969+0000","last_undegraded":"2024-04-25T10:01:30.659969+0000","last_fullsized":"2024-04-25T10:01:30.659969+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:19.341958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659637+0000","last_change":"2024-04-25T10:01:29.662877+0000","last_active":"2024-04-25T10:01:30.659637+0000","last_peered":"2024-04-25T10:01:30.659637+0000","last_clean":"2024-04-25T10:01:30.659637+0000","last_became_active":"2024-04-25T10:01:29.662288+0000","last_became_peered":"2024-04-25T10:01:29.662288+0000","last_unstale":"2024-04-25T10:01:30.659637+0000","last_undegraded":"2024-04-25T10:01:30.659637+0000","last_fullsized":"2024-04-25T10:01:30.659637+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:01.362545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659916+0000","last_change":"2024-04-25T10:01:29.667530+0000","last_active":"2024-04-25T10:01:30.659916+0000","last_peered":"2024-04-25T10:01:30.659916+0000","last_clean":"2024-04-25T10:01:30.659916+0000","last_became_active":"2024-04-25T10:01:29.666001+0000","last_became_peered":"2024-04-25T10:01:29.666001+0000","last_unstale":"2024-04-25T10:01:30.659916+0000","last_undegraded":"2024-04-25T10:01:30.659916+0000","last_fullsized":"2024-04-25T10:01:30.659916+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:27.063190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660107+0000","last_change":"2024-04-25T10:01:29.662990+0000","last_active":"2024-04-25T10:01:30.660107+0000","last_peered":"2024-04-25T10:01:30.660107+0000","last_clean":"2024-04-25T10:01:30.660107+0000","last_became_active":"2024-04-25T10:01:29.662659+0000","last_became_peered":"2024-04-25T10:01:29.662659+0000","last_unstale":"2024-04-25T10:01:30.660107+0000","last_undegraded":"2024-04-25T10:01:30.660107+0000","last_fullsized":"2024-04-25T10:01:30.660107+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:00.208825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180562+0000","last_change":"2024-04-25T10:01:29.657262+0000","last_active":"2024-04-25T10:01:32.180562+0000","last_peered":"2024-04-25T10:01:32.180562+0000","last_clean":"2024-04-25T10:01:32.180562+0000","last_became_active":"2024-04-25T10:01:29.656964+0000","last_became_peered":"2024-04-25T10:01:29.656964+0000","last_unstale":"2024-04-25T10:01:32.180562+0000","last_undegraded":"2024-04-25T10:01:32.180562+0000","last_fullsized":"2024-04-25T10:01:32.180562+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:15.956728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660974+0000","last_change":"2024-04-25T10:01:29.667879+0000","last_active":"2024-04-25T10:01:30.660974+0000","last_peered":"2024-04-25T10:01:30.660974+0000","last_clean":"2024-04-25T10:01:30.660974+0000","last_became_active":"2024-04-25T10:01:29.667539+0000","last_became_peered":"2024-04-25T10:01:29.667539+0000","last_unstale":"2024-04-25T10:01:30.660974+0000","last_undegraded":"2024-04-25T10:01:30.660974+0000","last_fullsized":"2024-04-25T10:01:30.660974+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:37.245128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659424+0000","last_change":"2024-04-25T10:01:29.666428+0000","last_active":"2024-04-25T10:01:30.659424+0000","last_peered":"2024-04-25T10:01:30.659424+0000","last_clean":"2024-04-25T10:01:30.659424+0000","last_became_active":"2024-04-25T10:01:29.665500+0000","last_became_peered":"2024-04-25T10:01:29.665500+0000","last_unstale":"2024-04-25T10:01:30.659424+0000","last_undegraded":"2024-04-25T10:01:30.659424+0000","last_fullsized":"2024-04-25T10:01:30.659424+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:29.759292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.912454+0000","last_change":"2024-04-25T10:01:29.655147+0000","last_active":"2024-04-25T10:01:31.912454+0000","last_peered":"2024-04-25T10:01:31.912454+0000","last_clean":"2024-04-25T10:01:31.912454+0000","last_became_active":"2024-04-25T10:01:29.654852+0000","last_became_peered":"2024-04-25T10:01:29.654852+0000","last_unstale":"2024-04-25T10:01:31.912454+0000","last_undegraded":"2024-04-25T10:01:31.912454+0000","last_fullsized":"2024-04-25T10:01:31.912454+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:24.478456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180305+0000","last_change":"2024-04-25T10:01:29.662787+0000","last_active":"2024-04-25T10:01:32.180305+0000","last_peered":"2024-04-25T10:01:32.180305+0000","last_clean":"2024-04-25T10:01:32.180305+0000","last_became_active":"2024-04-25T10:01:29.662488+0000","last_became_peered":"2024-04-25T10:01:29.662488+0000","last_unstale":"2024-04-25T10:01:32.180305+0000","last_undegraded":"2024-04-25T10:01:32.180305+0000","last_fullsized":"2024-04-25T10:01:32.180305+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:35.844560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.912495+0000","last_change":"2024-04-25T10:01:29.656866+0000","last_active":"2024-04-25T10:01:31.912495+0000","last_peered":"2024-04-25T10:01:31.912495+0000","last_clean":"2024-04-25T10:01:31.912495+0000","last_became_active":"2024-04-25T10:01:29.656596+0000","last_became_peered":"2024-04-25T10:01:29.656596+0000","last_unstale":"2024-04-25T10:01:31.912495+0000","last_undegraded":"2024-04-25T10:01:31.912495+0000","last_fullsized":"2024-04-25T10:01:31.912495+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:55.852972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"128'4","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659739+0000","last_change":"2024-04-25T10:01:28.644943+0000","last_active":"2024-04-25T10:01:30.659739+0000","last_peered":"2024-04-25T10:01:30.659739+0000","last_clean":"2024-04-25T10:01:30.659739+0000","last_became_active":"2024-04-25T10:01:28.644689+0000","last_became_peered":"2024-04-25T10:01:28.644689+0000","last_unstale":"2024-04-25T10:01:30.659739+0000","last_undegraded":"2024-04-25T10:01:30.659739+0000","last_fullsized":"2024-04-25T10:01:30.659739+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:57.293153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.079857+0000","last_change":"2024-04-25T10:01:29.657694+0000","last_active":"2024-04-25T10:01:31.079857+0000","last_peered":"2024-04-25T10:01:31.079857+0000","last_clean":"2024-04-25T10:01:31.079857+0000","last_became_active":"2024-04-25T10:01:29.657407+0000","last_became_peered":"2024-04-25T10:01:29.657407+0000","last_unstale":"2024-04-25T10:01:31.079857+0000","last_undegraded":"2024-04-25T10:01:31.079857+0000","last_fullsized":"2024-04-25T10:01:31.079857+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:40.550448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"128'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180881+0000","last_change":"2024-04-25T10:01:28.645571+0000","last_active":"2024-04-25T10:01:32.180881+0000","last_peered":"2024-04-25T10:01:32.180881+0000","last_clean":"2024-04-25T10:01:32.180881+0000","last_became_active":"2024-04-25T10:01:28.645283+0000","last_became_peered":"2024-04-25T10:01:28.645283+0000","last_unstale":"2024-04-25T10:01:32.180881+0000","last_undegraded":"2024-04-25T10:01:32.180881+0000","last_fullsized":"2024-04-25T10:01:32.180881+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:23.367239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180964+0000","last_change":"2024-04-25T10:01:29.660410+0000","last_active":"2024-04-25T10:01:32.180964+0000","last_peered":"2024-04-25T10:01:32.180964+0000","last_clean":"2024-04-25T10:01:32.180964+0000","last_became_active":"2024-04-25T10:01:29.659948+0000","last_became_peered":"2024-04-25T10:01:29.659948+0000","last_unstale":"2024-04-25T10:01:32.180964+0000","last_undegraded":"2024-04-25T10:01:32.180964+0000","last_fullsized":"2024-04-25T10:01:32.180964+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:15.780571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660187+0000","last_change":"2024-04-25T10:01:28.640421+0000","last_active":"2024-04-25T10:01:30.660187+0000","last_peered":"2024-04-25T10:01:30.660187+0000","last_clean":"2024-04-25T10:01:30.660187+0000","last_became_active":"2024-04-25T10:01:28.639536+0000","last_became_peered":"2024-04-25T10:01:28.639536+0000","last_unstale":"2024-04-25T10:01:30.660187+0000","last_undegraded":"2024-04-25T10:01:30.660187+0000","last_fullsized":"2024-04-25T10:01:30.660187+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:36.238604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.079885+0000","last_change":"2024-04-25T10:01:29.663435+0000","last_active":"2024-04-25T10:01:31.079885+0000","last_peered":"2024-04-25T10:01:31.079885+0000","last_clean":"2024-04-25T10:01:31.079885+0000","last_became_active":"2024-04-25T10:01:29.663075+0000","last_became_peered":"2024-04-25T10:01:29.663075+0000","last_unstale":"2024-04-25T10:01:31.079885+0000","last_undegraded":"2024-04-25T10:01:31.079885+0000","last_fullsized":"2024-04-25T10:01:31.079885+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:31.850027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660063+0000","last_change":"2024-04-25T10:01:28.642064+0000","last_active":"2024-04-25T10:01:30.660063+0000","last_peered":"2024-04-25T10:01:30.660063+0000","last_clean":"2024-04-25T10:01:30.660063+0000","last_became_active":"2024-04-25T10:01:28.641767+0000","last_became_peered":"2024-04-25T10:01:28.641767+0000","last_unstale":"2024-04-25T10:01:30.660063+0000","last_undegraded":"2024-04-25T10:01:30.660063+0000","last_fullsized":"2024-04-25T10:01:30.660063+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:43.115491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.912738+0000","last_change":"2024-04-25T10:01:29.646543+0000","last_active":"2024-04-25T10:01:31.912738+0000","last_peered":"2024-04-25T10:01:31.912738+0000","last_clean":"2024-04-25T10:01:31.912738+0000","last_became_active":"2024-04-25T10:01:29.646266+0000","last_became_peered":"2024-04-25T10:01:29.646266+0000","last_unstale":"2024-04-25T10:01:31.912738+0000","last_undegraded":"2024-04-25T10:01:31.912738+0000","last_fullsized":"2024-04-25T10:01:31.912738+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:11.812724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"128'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.912626+0000","last_change":"2024-04-25T10:01:28.643613+0000","last_active":"2024-04-25T10:01:31.912626+0000","last_peered":"2024-04-25T10:01:31.912626+0000","last_clean":"2024-04-25T10:01:31.912626+0000","last_became_active":"2024-04-25T10:01:28.643326+0000","last_became_peered":"2024-04-25T10:01:28.643326+0000","last_unstale":"2024-04-25T10:01:31.912626+0000","last_undegraded":"2024-04-25T10:01:31.912626+0000","last_fullsized":"2024-04-25T10:01:31.912626+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:48.121372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.912865+0000","last_change":"2024-04-25T10:01:29.653831+0000","last_active":"2024-04-25T10:01:31.912865+0000","last_peered":"2024-04-25T10:01:31.912865+0000","last_clean":"2024-04-25T10:01:31.912865+0000","last_became_active":"2024-04-25T10:01:29.653562+0000","last_became_peered":"2024-04-25T10:01:29.653562+0000","last_unstale":"2024-04-25T10:01:31.912865+0000","last_undegraded":"2024-04-25T10:01:31.912865+0000","last_fullsized":"2024-04-25T10:01:31.912865+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:46.263454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"128'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180948+0000","last_change":"2024-04-25T10:01:28.647013+0000","last_active":"2024-04-25T10:01:32.180948+0000","last_peered":"2024-04-25T10:01:32.180948+0000","last_clean":"2024-04-25T10:01:32.180948+0000","last_became_active":"2024-04-25T10:01:28.646667+0000","last_became_peered":"2024-04-25T10:01:28.646667+0000","last_unstale":"2024-04-25T10:01:32.180948+0000","last_undegraded":"2024-04-25T10:01:32.180948+0000","last_fullsized":"2024-04-25T10:01:32.180948+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:36.238604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180618+0000","last_change":"2024-04-25T10:01:29.667107+0000","last_active":"2024-04-25T10:01:32.180618+0000","last_peered":"2024-04-25T10:01:32.180618+0000","last_clean":"2024-04-25T10:01:32.180618+0000","last_became_active":"2024-04-25T10:01:29.665126+0000","last_became_peered":"2024-04-25T10:01:29.665126+0000","last_unstale":"2024-04-25T10:01:32.180618+0000","last_undegraded":"2024-04-25T10:01:32.180618+0000","last_fullsized":"2024-04-25T10:01:32.180618+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:18.113778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660476+0000","last_change":"2024-04-25T10:01:28.639325+0000","last_active":"2024-04-25T10:01:30.660476+0000","last_peered":"2024-04-25T10:01:30.660476+0000","last_clean":"2024-04-25T10:01:30.660476+0000","last_became_active":"2024-04-25T10:01:28.639055+0000","last_became_peered":"2024-04-25T10:01:28.639055+0000","last_unstale":"2024-04-25T10:01:30.660476+0000","last_undegraded":"2024-04-25T10:01:30.660476+0000","last_fullsized":"2024-04-25T10:01:30.660476+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:45.256930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660161+0000","last_change":"2024-04-25T10:01:29.669106+0000","last_active":"2024-04-25T10:01:30.660161+0000","last_peered":"2024-04-25T10:01:30.660161+0000","last_clean":"2024-04-25T10:01:30.660161+0000","last_became_active":"2024-04-25T10:01:29.668796+0000","last_became_peered":"2024-04-25T10:01:29.668796+0000","last_unstale":"2024-04-25T10:01:30.660161+0000","last_undegraded":"2024-04-25T10:01:30.660161+0000","last_fullsized":"2024-04-25T10:01:30.660161+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:23.827775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659294+0000","last_change":"2024-04-25T10:01:28.639418+0000","last_active":"2024-04-25T10:01:30.659294+0000","last_peered":"2024-04-25T10:01:30.659294+0000","last_clean":"2024-04-25T10:01:30.659294+0000","last_became_active":"2024-04-25T10:01:28.639134+0000","last_became_peered":"2024-04-25T10:01:28.639134+0000","last_unstale":"2024-04-25T10:01:30.659294+0000","last_undegraded":"2024-04-25T10:01:30.659294+0000","last_fullsized":"2024-04-25T10:01:30.659294+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:54.297212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.079774+0000","last_change":"2024-04-25T10:01:29.664062+0000","last_active":"2024-04-25T10:01:31.079774+0000","last_peered":"2024-04-25T10:01:31.079774+0000","last_clean":"2024-04-25T10:01:31.079774+0000","last_became_active":"2024-04-25T10:01:29.663741+0000","last_became_peered":"2024-04-25T10:01:29.663741+0000","last_unstale":"2024-04-25T10:01:31.079774+0000","last_undegraded":"2024-04-25T10:01:31.079774+0000","last_fullsized":"2024-04-25T10:01:31.079774+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:08.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.181186+0000","last_change":"2024-04-25T10:01:28.640498+0000","last_active":"2024-04-25T10:01:32.181186+0000","last_peered":"2024-04-25T10:01:32.181186+0000","last_clean":"2024-04-25T10:01:32.181186+0000","last_became_active":"2024-04-25T10:01:28.640186+0000","last_became_peered":"2024-04-25T10:01:28.640186+0000","last_unstale":"2024-04-25T10:01:32.181186+0000","last_undegraded":"2024-04-25T10:01:32.181186+0000","last_fullsized":"2024-04-25T10:01:32.181186+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:45.256930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659987+0000","last_change":"2024-04-25T10:01:29.655040+0000","last_active":"2024-04-25T10:01:30.659987+0000","last_peered":"2024-04-25T10:01:30.659987+0000","last_clean":"2024-04-25T10:01:30.659987+0000","last_became_active":"2024-04-25T10:01:29.654673+0000","last_became_peered":"2024-04-25T10:01:29.654673+0000","last_unstale":"2024-04-25T10:01:30.659987+0000","last_undegraded":"2024-04-25T10:01:30.659987+0000","last_fullsized":"2024-04-25T10:01:30.659987+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:02.546964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.080096+0000","last_change":"2024-04-25T10:01:28.647926+0000","last_active":"2024-04-25T10:01:31.080096+0000","last_peered":"2024-04-25T10:01:31.080096+0000","last_clean":"2024-04-25T10:01:31.080096+0000","last_became_active":"2024-04-25T10:01:28.646963+0000","last_became_peered":"2024-04-25T10:01:28.646963+0000","last_unstale":"2024-04-25T10:01:31.080096+0000","last_undegraded":"2024-04-25T10:01:31.080096+0000","last_fullsized":"2024-04-25T10:01:31.080096+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:35.564381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659570+0000","last_change":"2024-04-25T10:01:29.665672+0000","last_active":"2024-04-25T10:01:30.659570+0000","last_peered":"2024-04-25T10:01:30.659570+0000","last_clean":"2024-04-25T10:01:30.659570+0000","last_became_active":"2024-04-25T10:01:29.664768+0000","last_became_peered":"2024-04-25T10:01:29.664768+0000","last_unstale":"2024-04-25T10:01:30.659570+0000","last_undegraded":"2024-04-25T10:01:30.659570+0000","last_fullsized":"2024-04-25T10:01:30.659570+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:33.111868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"128'3","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660281+0000","last_change":"2024-04-25T10:01:28.638314+0000","last_active":"2024-04-25T10:01:30.660281+0000","last_peered":"2024-04-25T10:01:30.660281+0000","last_clean":"2024-04-25T10:01:30.660281+0000","last_became_active":"2024-04-25T10:01:28.637960+0000","last_became_peered":"2024-04-25T10:01:28.637960+0000","last_unstale":"2024-04-25T10:01:30.660281+0000","last_undegraded":"2024-04-25T10:01:30.660281+0000","last_fullsized":"2024-04-25T10:01:30.660281+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:23.367239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660946+0000","last_change":"2024-04-25T10:01:29.663164+0000","last_active":"2024-04-25T10:01:30.660946+0000","last_peered":"2024-04-25T10:01:30.660946+0000","last_clean":"2024-04-25T10:01:30.660946+0000","last_became_active":"2024-04-25T10:01:29.662517+0000","last_became_peered":"2024-04-25T10:01:29.662517+0000","last_unstale":"2024-04-25T10:01:30.660946+0000","last_undegraded":"2024-04-25T10:01:30.660946+0000","last_fullsized":"2024-04-25T10:01:30.660946+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:15.780571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"128'6","reported_seq":20,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660158+0000","last_change":"2024-04-25T10:01:28.639720+0000","last_active":"2024-04-25T10:01:30.660158+0000","last_peered":"2024-04-25T10:01:30.660158+0000","last_clean":"2024-04-25T10:01:30.660158+0000","last_became_active":"2024-04-25T10:01:28.638956+0000","last_became_peered":"2024-04-25T10:01:28.638956+0000","last_unstale":"2024-04-25T10:01:30.660158+0000","last_undegraded":"2024-04-25T10:01:30.660158+0000","last_fullsized":"2024-04-25T10:01:30.660158+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:01.540440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180702+0000","last_change":"2024-04-25T10:01:29.670458+0000","last_active":"2024-04-25T10:01:32.180702+0000","last_peered":"2024-04-25T10:01:32.180702+0000","last_clean":"2024-04-25T10:01:32.180702+0000","last_became_active":"2024-04-25T10:01:29.666004+0000","last_became_peered":"2024-04-25T10:01:29.666004+0000","last_unstale":"2024-04-25T10:01:32.180702+0000","last_undegraded":"2024-04-25T10:01:32.180702+0000","last_fullsized":"2024-04-25T10:01:32.180702+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:20.695200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.079899+0000","last_change":"2024-04-25T10:01:28.644548+0000","last_active":"2024-04-25T10:01:31.079899+0000","last_peered":"2024-04-25T10:01:31.079899+0000","last_clean":"2024-04-25T10:01:31.079899+0000","last_became_active":"2024-04-25T10:01:28.643068+0000","last_became_peered":"2024-04-25T10:01:28.643068+0000","last_unstale":"2024-04-25T10:01:31.079899+0000","last_undegraded":"2024-04-25T10:01:31.079899+0000","last_fullsized":"2024-04-25T10:01:31.079899+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:40.351532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660407+0000","last_change":"2024-04-25T10:01:29.657107+0000","last_active":"2024-04-25T10:01:30.660407+0000","last_peered":"2024-04-25T10:01:30.660407+0000","last_clean":"2024-04-25T10:01:30.660407+0000","last_became_active":"2024-04-25T10:01:29.656829+0000","last_became_peered":"2024-04-25T10:01:29.656829+0000","last_unstale":"2024-04-25T10:01:30.660407+0000","last_undegraded":"2024-04-25T10:01:30.660407+0000","last_fullsized":"2024-04-25T10:01:30.660407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.798438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659161+0000","last_change":"2024-04-25T10:01:28.641313+0000","last_active":"2024-04-25T10:01:30.659161+0000","last_peered":"2024-04-25T10:01:30.659161+0000","last_clean":"2024-04-25T10:01:30.659161+0000","last_became_active":"2024-04-25T10:01:28.641020+0000","last_became_peered":"2024-04-25T10:01:28.641020+0000","last_unstale":"2024-04-25T10:01:30.659161+0000","last_undegraded":"2024-04-25T10:01:30.659161+0000","last_fullsized":"2024-04-25T10:01:30.659161+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:14.048509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.181086+0000","last_change":"2024-04-25T10:01:29.661355+0000","last_active":"2024-04-25T10:01:32.181086+0000","last_peered":"2024-04-25T10:01:32.181086+0000","last_clean":"2024-04-25T10:01:32.181086+0000","last_became_active":"2024-04-25T10:01:29.661092+0000","last_became_peered":"2024-04-25T10:01:29.661092+0000","last_unstale":"2024-04-25T10:01:32.181086+0000","last_undegraded":"2024-04-25T10:01:32.181086+0000","last_fullsized":"2024-04-25T10:01:32.181086+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:23.827775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"128'8","reported_seq":22,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660140+0000","last_change":"2024-04-25T10:01:28.639344+0000","last_active":"2024-04-25T10:01:30.660140+0000","last_peered":"2024-04-25T10:01:30.660140+0000","last_clean":"2024-04-25T10:01:30.660140+0000","last_became_active":"2024-04-25T10:01:28.639079+0000","last_became_peered":"2024-04-25T10:01:28.639079+0000","last_unstale":"2024-04-25T10:01:30.660140+0000","last_undegraded":"2024-04-25T10:01:30.660140+0000","last_fullsized":"2024-04-25T10:01:30.660140+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:54.846448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660601+0000","last_change":"2024-04-25T10:01:29.661035+0000","last_active":"2024-04-25T10:01:30.660601+0000","last_peered":"2024-04-25T10:01:30.660601+0000","last_clean":"2024-04-25T10:01:30.660601+0000","last_became_active":"2024-04-25T10:01:29.660496+0000","last_became_peered":"2024-04-25T10:01:29.660496+0000","last_unstale":"2024-04-25T10:01:30.660601+0000","last_undegraded":"2024-04-25T10:01:30.660601+0000","last_fullsized":"2024-04-25T10:01:30.660601+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:14.091825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"128'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.181275+0000","last_change":"2024-04-25T10:01:28.642872+0000","last_active":"2024-04-25T10:01:32.181275+0000","last_peered":"2024-04-25T10:01:32.181275+0000","last_clean":"2024-04-25T10:01:32.181275+0000","last_became_active":"2024-04-25T10:01:28.642250+0000","last_became_peered":"2024-04-25T10:01:28.642250+0000","last_unstale":"2024-04-25T10:01:32.181275+0000","last_undegraded":"2024-04-25T10:01:32.181275+0000","last_fullsized":"2024-04-25T10:01:32.181275+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:58.570780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660771+0000","last_change":"2024-04-25T10:01:29.656242+0000","last_active":"2024-04-25T10:01:30.660771+0000","last_peered":"2024-04-25T10:01:30.660771+0000","last_clean":"2024-04-25T10:01:30.660771+0000","last_became_active":"2024-04-25T10:01:29.655987+0000","last_became_peered":"2024-04-25T10:01:29.655987+0000","last_unstale":"2024-04-25T10:01:30.660771+0000","last_undegraded":"2024-04-25T10:01:30.660771+0000","last_fullsized":"2024-04-25T10:01:30.660771+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:59.273474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660020+0000","last_change":"2024-04-25T10:01:28.643788+0000","last_active":"2024-04-25T10:01:30.660020+0000","last_peered":"2024-04-25T10:01:30.660020+0000","last_clean":"2024-04-25T10:01:30.660020+0000","last_became_active":"2024-04-25T10:01:28.643521+0000","last_became_peered":"2024-04-25T10:01:28.643521+0000","last_unstale":"2024-04-25T10:01:30.660020+0000","last_undegraded":"2024-04-25T10:01:30.660020+0000","last_fullsized":"2024-04-25T10:01:30.660020+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:09.710270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660632+0000","last_change":"2024-04-25T10:01:29.661956+0000","last_active":"2024-04-25T10:01:30.660632+0000","last_peered":"2024-04-25T10:01:30.660632+0000","last_clean":"2024-04-25T10:01:30.660632+0000","last_became_active":"2024-04-25T10:01:29.661690+0000","last_became_peered":"2024-04-25T10:01:29.661690+0000","last_unstale":"2024-04-25T10:01:30.660632+0000","last_undegraded":"2024-04-25T10:01:30.660632+0000","last_fullsized":"2024-04-25T10:01:30.660632+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.988119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"128'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660374+0000","last_change":"2024-04-25T10:01:28.640275+0000","last_active":"2024-04-25T10:01:30.660374+0000","last_peered":"2024-04-25T10:01:30.660374+0000","last_clean":"2024-04-25T10:01:30.660374+0000","last_became_active":"2024-04-25T10:01:28.640015+0000","last_became_peered":"2024-04-25T10:01:28.640015+0000","last_unstale":"2024-04-25T10:01:30.660374+0000","last_undegraded":"2024-04-25T10:01:30.660374+0000","last_fullsized":"2024-04-25T10:01:30.660374+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:23.471932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.181043+0000","last_change":"2024-04-25T10:01:29.671551+0000","last_active":"2024-04-25T10:01:32.181043+0000","last_peered":"2024-04-25T10:01:32.181043+0000","last_clean":"2024-04-25T10:01:32.181043+0000","last_became_active":"2024-04-25T10:01:29.671218+0000","last_became_peered":"2024-04-25T10:01:29.671218+0000","last_unstale":"2024-04-25T10:01:32.181043+0000","last_undegraded":"2024-04-25T10:01:32.181043+0000","last_fullsized":"2024-04-25T10:01:32.181043+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:34.211993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180833+0000","last_change":"2024-04-25T10:01:28.644201+0000","last_active":"2024-04-25T10:01:32.180833+0000","last_peered":"2024-04-25T10:01:32.180833+0000","last_clean":"2024-04-25T10:01:32.180833+0000","last_became_active":"2024-04-25T10:01:28.643895+0000","last_became_peered":"2024-04-25T10:01:28.643895+0000","last_unstale":"2024-04-25T10:01:32.180833+0000","last_undegraded":"2024-04-25T10:01:32.180833+0000","last_fullsized":"2024-04-25T10:01:32.180833+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:14.774047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180749+0000","last_change":"2024-04-25T10:01:29.658230+0000","last_active":"2024-04-25T10:01:32.180749+0000","last_peered":"2024-04-25T10:01:32.180749+0000","last_clean":"2024-04-25T10:01:32.180749+0000","last_became_active":"2024-04-25T10:01:29.657910+0000","last_became_peered":"2024-04-25T10:01:29.657910+0000","last_unstale":"2024-04-25T10:01:32.180749+0000","last_undegraded":"2024-04-25T10:01:32.180749+0000","last_fullsized":"2024-04-25T10:01:32.180749+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:23.607420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.912731+0000","last_change":"2024-04-25T10:01:28.640556+0000","last_active":"2024-04-25T10:01:31.912731+0000","last_peered":"2024-04-25T10:01:31.912731+0000","last_clean":"2024-04-25T10:01:31.912731+0000","last_became_active":"2024-04-25T10:01:28.640292+0000","last_became_peered":"2024-04-25T10:01:28.640292+0000","last_unstale":"2024-04-25T10:01:31.912731+0000","last_undegraded":"2024-04-25T10:01:31.912731+0000","last_fullsized":"2024-04-25T10:01:31.912731+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:14.048509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.080163+0000","last_change":"2024-04-25T10:01:29.661918+0000","last_active":"2024-04-25T10:01:31.080163+0000","last_peered":"2024-04-25T10:01:31.080163+0000","last_clean":"2024-04-25T10:01:31.080163+0000","last_became_active":"2024-04-25T10:01:29.661599+0000","last_became_peered":"2024-04-25T10:01:29.661599+0000","last_unstale":"2024-04-25T10:01:31.080163+0000","last_undegraded":"2024-04-25T10:01:31.080163+0000","last_fullsized":"2024-04-25T10:01:31.080163+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:26.622724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.079895+0000","last_change":"2024-04-25T10:01:28.641737+0000","last_active":"2024-04-25T10:01:31.079895+0000","last_peered":"2024-04-25T10:01:31.079895+0000","last_clean":"2024-04-25T10:01:31.079895+0000","last_became_active":"2024-04-25T10:01:28.641451+0000","last_became_peered":"2024-04-25T10:01:28.641451+0000","last_unstale":"2024-04-25T10:01:31.079895+0000","last_undegraded":"2024-04-25T10:01:31.079895+0000","last_fullsized":"2024-04-25T10:01:31.079895+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:06.898958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180776+0000","last_change":"2024-04-25T10:01:29.663101+0000","last_active":"2024-04-25T10:01:32.180776+0000","last_peered":"2024-04-25T10:01:32.180776+0000","last_clean":"2024-04-25T10:01:32.180776+0000","last_became_active":"2024-04-25T10:01:29.662707+0000","last_became_peered":"2024-04-25T10:01:29.662707+0000","last_unstale":"2024-04-25T10:01:32.180776+0000","last_undegraded":"2024-04-25T10:01:32.180776+0000","last_fullsized":"2024-04-25T10:01:32.180776+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:00.208825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660858+0000","last_change":"2024-04-25T10:01:28.644516+0000","last_active":"2024-04-25T10:01:30.660858+0000","last_peered":"2024-04-25T10:01:30.660858+0000","last_clean":"2024-04-25T10:01:30.660858+0000","last_became_active":"2024-04-25T10:01:28.644155+0000","last_became_peered":"2024-04-25T10:01:28.644155+0000","last_unstale":"2024-04-25T10:01:30.660858+0000","last_undegraded":"2024-04-25T10:01:30.660858+0000","last_fullsized":"2024-04-25T10:01:30.660858+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:18.335434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.181209+0000","last_change":"2024-04-25T10:01:29.666393+0000","last_active":"2024-04-25T10:01:32.181209+0000","last_peered":"2024-04-25T10:01:32.181209+0000","last_clean":"2024-04-25T10:01:32.181209+0000","last_became_active":"2024-04-25T10:01:29.666071+0000","last_became_peered":"2024-04-25T10:01:29.666071+0000","last_unstale":"2024-04-25T10:01:32.181209+0000","last_undegraded":"2024-04-25T10:01:32.181209+0000","last_fullsized":"2024-04-25T10:01:32.181209+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:19.039051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660068+0000","last_change":"2024-04-25T10:01:28.638744+0000","last_active":"2024-04-25T10:01:30.660068+0000","last_peered":"2024-04-25T10:01:30.660068+0000","last_clean":"2024-04-25T10:01:30.660068+0000","last_became_active":"2024-04-25T10:01:28.638312+0000","last_became_peered":"2024-04-25T10:01:28.638312+0000","last_unstale":"2024-04-25T10:01:30.660068+0000","last_undegraded":"2024-04-25T10:01:30.660068+0000","last_fullsized":"2024-04-25T10:01:30.660068+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:58.570780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180583+0000","last_change":"2024-04-25T10:01:29.664301+0000","last_active":"2024-04-25T10:01:32.180583+0000","last_peered":"2024-04-25T10:01:32.180583+0000","last_clean":"2024-04-25T10:01:32.180583+0000","last_became_active":"2024-04-25T10:01:29.663898+0000","last_became_peered":"2024-04-25T10:01:29.663898+0000","last_unstale":"2024-04-25T10:01:32.180583+0000","last_undegraded":"2024-04-25T10:01:32.180583+0000","last_fullsized":"2024-04-25T10:01:32.180583+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:27.750691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.080127+0000","last_change":"2024-04-25T10:01:28.647785+0000","last_active":"2024-04-25T10:01:31.080127+0000","last_peered":"2024-04-25T10:01:31.080127+0000","last_clean":"2024-04-25T10:01:31.080127+0000","last_became_active":"2024-04-25T10:01:28.647474+0000","last_became_peered":"2024-04-25T10:01:28.647474+0000","last_unstale":"2024-04-25T10:01:31.080127+0000","last_undegraded":"2024-04-25T10:01:31.080127+0000","last_fullsized":"2024-04-25T10:01:31.080127+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:03.763863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659354+0000","last_change":"2024-04-25T10:01:29.662951+0000","last_active":"2024-04-25T10:01:30.659354+0000","last_peered":"2024-04-25T10:01:30.659354+0000","last_clean":"2024-04-25T10:01:30.659354+0000","last_became_active":"2024-04-25T10:01:29.662672+0000","last_became_peered":"2024-04-25T10:01:29.662672+0000","last_unstale":"2024-04-25T10:01:30.659354+0000","last_undegraded":"2024-04-25T10:01:30.659354+0000","last_fullsized":"2024-04-25T10:01:30.659354+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:35.295654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"128'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.912805+0000","last_change":"2024-04-25T10:01:28.642073+0000","last_active":"2024-04-25T10:01:31.912805+0000","last_peered":"2024-04-25T10:01:31.912805+0000","last_clean":"2024-04-25T10:01:31.912805+0000","last_became_active":"2024-04-25T10:01:28.641446+0000","last_became_peered":"2024-04-25T10:01:28.641446+0000","last_unstale":"2024-04-25T10:01:31.912805+0000","last_undegraded":"2024-04-25T10:01:31.912805+0000","last_fullsized":"2024-04-25T10:01:31.912805+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:59.292156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660401+0000","last_change":"2024-04-25T10:01:29.663358+0000","last_active":"2024-04-25T10:01:30.660401+0000","last_peered":"2024-04-25T10:01:30.660401+0000","last_clean":"2024-04-25T10:01:30.660401+0000","last_became_active":"2024-04-25T10:01:29.663093+0000","last_became_peered":"2024-04-25T10:01:29.663093+0000","last_unstale":"2024-04-25T10:01:30.660401+0000","last_undegraded":"2024-04-25T10:01:30.660401+0000","last_fullsized":"2024-04-25T10:01:30.660401+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:17.850553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:31.912831+0000","last_change":"2024-04-25T10:01:28.643918+0000","last_active":"2024-04-25T10:01:31.912831+0000","last_peered":"2024-04-25T10:01:31.912831+0000","last_clean":"2024-04-25T10:01:31.912831+0000","last_became_active":"2024-04-25T10:01:28.643575+0000","last_became_peered":"2024-04-25T10:01:28.643575+0000","last_unstale":"2024-04-25T10:01:31.912831+0000","last_undegraded":"2024-04-25T10:01:31.912831+0000","last_fullsized":"2024-04-25T10:01:31.912831+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:54.297212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.181123+0000","last_change":"2024-04-25T10:01:29.653658+0000","last_active":"2024-04-25T10:01:32.181123+0000","last_peered":"2024-04-25T10:01:32.181123+0000","last_clean":"2024-04-25T10:01:32.181123+0000","last_became_active":"2024-04-25T10:01:29.653337+0000","last_became_peered":"2024-04-25T10:01:29.653337+0000","last_unstale":"2024-04-25T10:01:32.181123+0000","last_undegraded":"2024-04-25T10:01:32.181123+0000","last_fullsized":"2024-04-25T10:01:32.181123+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:46.608455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660299+0000","last_change":"2024-04-25T10:01:28.633829+0000","last_active":"2024-04-25T10:01:30.660299+0000","last_peered":"2024-04-25T10:01:30.660299+0000","last_clean":"2024-04-25T10:01:30.660299+0000","last_became_active":"2024-04-25T10:01:28.632994+0000","last_became_peered":"2024-04-25T10:01:28.632994+0000","last_unstale":"2024-04-25T10:01:30.660299+0000","last_undegraded":"2024-04-25T10:01:30.660299+0000","last_fullsized":"2024-04-25T10:01:30.660299+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:09.710270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660210+0000","last_change":"2024-04-25T10:01:29.661333+0000","last_active":"2024-04-25T10:01:30.660210+0000","last_peered":"2024-04-25T10:01:30.660210+0000","last_clean":"2024-04-25T10:01:30.660210+0000","last_became_active":"2024-04-25T10:01:29.660927+0000","last_became_peered":"2024-04-25T10:01:29.660927+0000","last_unstale":"2024-04-25T10:01:30.660210+0000","last_undegraded":"2024-04-25T10:01:30.660210+0000","last_fullsized":"2024-04-25T10:01:30.660210+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:39.545201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180898+0000","last_change":"2024-04-25T10:01:28.649336+0000","last_active":"2024-04-25T10:01:32.180898+0000","last_peered":"2024-04-25T10:01:32.180898+0000","last_clean":"2024-04-25T10:01:32.180898+0000","last_became_active":"2024-04-25T10:01:28.649067+0000","last_became_peered":"2024-04-25T10:01:28.649067+0000","last_unstale":"2024-04-25T10:01:32.180898+0000","last_undegraded":"2024-04-25T10:01:32.180898+0000","last_fullsized":"2024-04-25T10:01:32.180898+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:10.806201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660315+0000","last_change":"2024-04-25T10:01:29.659716+0000","last_active":"2024-04-25T10:01:30.660315+0000","last_peered":"2024-04-25T10:01:30.660315+0000","last_clean":"2024-04-25T10:01:30.660315+0000","last_became_active":"2024-04-25T10:01:29.659366+0000","last_became_peered":"2024-04-25T10:01:29.659366+0000","last_unstale":"2024-04-25T10:01:30.660315+0000","last_undegraded":"2024-04-25T10:01:30.660315+0000","last_fullsized":"2024-04-25T10:01:30.660315+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:48.980271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660622+0000","last_change":"2024-04-25T10:01:28.637991+0000","last_active":"2024-04-25T10:01:30.660622+0000","last_peered":"2024-04-25T10:01:30.660622+0000","last_clean":"2024-04-25T10:01:30.660622+0000","last_became_active":"2024-04-25T10:01:28.637710+0000","last_became_peered":"2024-04-25T10:01:28.637710+0000","last_unstale":"2024-04-25T10:01:30.660622+0000","last_undegraded":"2024-04-25T10:01:30.660622+0000","last_fullsized":"2024-04-25T10:01:30.660622+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:18.032527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660092+0000","last_change":"2024-04-25T10:01:29.661804+0000","last_active":"2024-04-25T10:01:30.660092+0000","last_peered":"2024-04-25T10:01:30.660092+0000","last_clean":"2024-04-25T10:01:30.660092+0000","last_became_active":"2024-04-25T10:01:29.661535+0000","last_became_peered":"2024-04-25T10:01:29.661535+0000","last_unstale":"2024-04-25T10:01:30.660092+0000","last_undegraded":"2024-04-25T10:01:30.660092+0000","last_fullsized":"2024-04-25T10:01:30.660092+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:44.122015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660235+0000","last_change":"2024-04-25T10:01:28.637799+0000","last_active":"2024-04-25T10:01:30.660235+0000","last_peered":"2024-04-25T10:01:30.660235+0000","last_clean":"2024-04-25T10:01:30.660235+0000","last_became_active":"2024-04-25T10:01:28.637491+0000","last_became_peered":"2024-04-25T10:01:28.637491+0000","last_unstale":"2024-04-25T10:01:30.660235+0000","last_undegraded":"2024-04-25T10:01:30.660235+0000","last_fullsized":"2024-04-25T10:01:30.660235+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:10.806201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660548+0000","last_change":"2024-04-25T10:01:29.664026+0000","last_active":"2024-04-25T10:01:30.660548+0000","last_peered":"2024-04-25T10:01:30.660548+0000","last_clean":"2024-04-25T10:01:30.660548+0000","last_became_active":"2024-04-25T10:01:29.663752+0000","last_became_peered":"2024-04-25T10:01:29.663752+0000","last_unstale":"2024-04-25T10:01:30.660548+0000","last_undegraded":"2024-04-25T10:01:30.660548+0000","last_fullsized":"2024-04-25T10:01:30.660548+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:54.783333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:32.180696+0000","last_change":"2024-04-25T10:01:28.645758+0000","last_active":"2024-04-25T10:01:32.180696+0000","last_peered":"2024-04-25T10:01:32.180696+0000","last_clean":"2024-04-25T10:01:32.180696+0000","last_became_active":"2024-04-25T10:01:28.645393+0000","last_became_peered":"2024-04-25T10:01:28.645393+0000","last_unstale":"2024-04-25T10:01:32.180696+0000","last_undegraded":"2024-04-25T10:01:32.180696+0000","last_fullsized":"2024-04-25T10:01:32.180696+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:13.085301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660000+0000","last_change":"2024-04-25T10:01:29.663370+0000","last_active":"2024-04-25T10:01:30.660000+0000","last_peered":"2024-04-25T10:01:30.660000+0000","last_clean":"2024-04-25T10:01:30.660000+0000","last_became_active":"2024-04-25T10:01:29.661109+0000","last_became_peered":"2024-04-25T10:01:29.661109+0000","last_unstale":"2024-04-25T10:01:30.660000+0000","last_undegraded":"2024-04-25T10:01:30.660000+0000","last_fullsized":"2024-04-25T10:01:30.660000+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:28.621726+0000","last_clean_scrub_stamp":"2024-04-25T10:01:28.621726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:29.721647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.660507+0000","last_change":"2024-04-25T10:01:28.636229+0000","last_active":"2024-04-25T10:01:30.660507+0000","last_peered":"2024-04-25T10:01:30.660507+0000","last_clean":"2024-04-25T10:01:30.660507+0000","last_became_active":"2024-04-25T10:01:28.635741+0000","last_became_peered":"2024-04-25T10:01:28.635741+0000","last_unstale":"2024-04-25T10:01:30.660507+0000","last_undegraded":"2024-04-25T10:01:30.660507+0000","last_fullsized":"2024-04-25T10:01:30.660507+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:27.615203+0000","last_clean_scrub_stamp":"2024-04-25T10:01:27.615203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:14.774047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":392,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:01:30.659701+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:01:30.659701+0000","last_peered":"2024-04-25T10:01:30.659701+0000","last_clean":"2024-04-25T10:01:30.659701+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:01:30.659701+0000","last_undegraded":"2024-04-25T10:01:30.659701+0000","last_fullsized":"2024-04-25T10:01:30.659701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29196,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342644,"statfs":{"total":96636764160,"available":96606867456,"internally_reserved":0,"allocated":2224128,"data_stored":1345433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574956,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46656,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325184,"statfs":{"total":96636764160,"available":96588988416,"internally_reserved":0,"allocated":1622016,"data_stored":754035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46648,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325192,"statfs":{"total":96636764160,"available":96588996608,"internally_reserved":0,"allocated":1617920,"data_stored":753385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54964,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316876,"statfs":{"total":96636764160,"available":96580481024,"internally_reserved":0,"allocated":1613824,"data_stored":753363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54984,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316856,"statfs":{"total":96636764160,"available":96580460544,"internally_reserved":0,"allocated":1630208,"data_stored":755045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574956,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50820,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321020,"statfs":{"total":96636764160,"available":96584724480,"internally_reserved":0,"allocated":1622016,"data_stored":753979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50812,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321028,"statfs":{"total":96636764160,"available":96584732672,"internally_reserved":0,"allocated":1617920,"data_stored":753433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51392,"kb_used_data":2164,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320448,"statfs":{"total":96636764160,"available":96584138752,"internally_reserved":0,"allocated":2215936,"data_stored":1343787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:37.427 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:01:37.427 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:01:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.567+0000 7f3822893640 1 Processor -- start 2024-04-25T10:01:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.567+0000 7f3822893640 1 -- start start 2024-04-25T10:01:37.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.567+0000 7f3822893640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f381c0ced80 0x7f381c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.567+0000 7f3822893640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f381c161710 0x7f381c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.567+0000 7f3822893640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381c15e000 con 0x7f381c15dc20 2024-04-25T10:01:37.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.567+0000 7f3822893640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381c0d1de0 con 0x7f381c161710 2024-04-25T10:01:37.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.568+0000 7f3822893640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c161c60 con 0x7f381c0ced80 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.568+0000 7f3820e30640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f381c161710 0x7f381c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.568+0000 7f3820e30640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f381c161710 0x7f381c0d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33618/0 (socket says 172.21.15.19:33618) 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.568+0000 7f3820e30640 1 -- 172.21.15.19:0/1545248254 learned_addr learned my addr 172.21.15.19:0/1545248254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.568+0000 7f381b7fe640 1 --2- 172.21.15.19:0/1545248254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f381c0ced80 0x7f381c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.568+0000 7f3820e30640 1 -- 172.21.15.19:0/1545248254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f381c0ced80 msgr2=0x7f381c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.568+0000 7f3820e30640 1 --2- 172.21.15.19:0/1545248254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f381c0ced80 0x7f381c0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.568+0000 7f3820e30640 1 -- 172.21.15.19:0/1545248254 >> v1:172.21.15.112:6789/0 conn(0x7f381c15dc20 legacy=0x7f381c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.568+0000 7f3820e30640 1 -- 172.21.15.19:0/1545248254 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c161f40 con 0x7f381c161710 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.569+0000 7f3820e30640 1 --2- 172.21.15.19:0/1545248254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f381c161710 0x7f381c0d2820 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f3814009e50 tx=0x7f381402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a295133f4815df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.569+0000 7f381affd640 1 -- 172.21.15.19:0/1545248254 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3814036070 con 0x7f381c161710 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.569+0000 7f381affd640 1 -- 172.21.15.19:0/1545248254 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3814031600 con 0x7f381c161710 2024-04-25T10:01:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.569+0000 7f381affd640 1 -- 172.21.15.19:0/1545248254 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38140319d0 con 0x7f381c161710 2024-04-25T10:01:37.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.569+0000 7f3822893640 1 -- 172.21.15.19:0/1545248254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f381c161710 msgr2=0x7f381c0d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.569+0000 7f3822893640 1 --2- 172.21.15.19:0/1545248254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f381c161710 0x7f381c0d2820 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f3814009e50 tx=0x7f381402a710 comp rx=0 tx=0).stop 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.570+0000 7f3822893640 1 -- 172.21.15.19:0/1545248254 shutdown_connections 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.570+0000 7f3822893640 1 --2- 172.21.15.19:0/1545248254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f381c161710 0x7f381c0d2820 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.570+0000 7f3822893640 1 --2- 172.21.15.19:0/1545248254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f381c0ced80 0x7f381c0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.570+0000 7f3822893640 1 -- 172.21.15.19:0/1545248254 >> 172.21.15.19:0/1545248254 conn(0x7f381c0d0420 msgr2=0x7f381c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.570+0000 7f3822893640 1 -- 172.21.15.19:0/1545248254 shutdown_connections 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.570+0000 7f3822893640 1 -- 172.21.15.19:0/1545248254 wait complete. 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.571+0000 7f3822893640 1 Processor -- start 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.571+0000 7f3822893640 1 -- start start 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f3822893640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f381c0ced80 0x7f381c0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f3822893640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f381c15dc20 0x7f381c0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f3822893640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f381c161710 0x7f381c0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f3822893640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0b5e50 con 0x7f381c15dc20 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f3822893640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c162500 con 0x7f381c0ced80 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f3822893640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0b5fd0 con 0x7f381c161710 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f381b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f381c15dc20 0x7f381c0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f381b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f381c15dc20 0x7f381c0cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33622/0 (socket says 172.21.15.19:33622) 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f381b7fe640 1 -- 172.21.15.19:0/2794629162 learned_addr learned my addr 172.21.15.19:0/2794629162 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f3820e30640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f381c161710 0x7f381c0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f381bfff640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f381c0ced80 0x7f381c0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f381b7fe640 1 -- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f381c161710 msgr2=0x7f381c0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f381b7fe640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f381c161710 0x7f381c0ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f381b7fe640 1 -- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f381c0ced80 msgr2=0x7f381c0c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f381b7fe640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f381c0ced80 0x7f381c0c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f381b7fe640 1 -- 172.21.15.19:0/2794629162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c1e9160 con 0x7f381c15dc20 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f381b7fe640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f381c15dc20 0x7f381c0cc6c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f381000d5f0 tx=0x7f381000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f3818ff9640 1 -- 172.21.15.19:0/2794629162 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38100125f0 con 0x7f381c15dc20 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f3818ff9640 1 -- 172.21.15.19:0/2794629162 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3810019070 con 0x7f381c15dc20 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f3822893640 1 -- 172.21.15.19:0/2794629162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c1e9440 con 0x7f381c15dc20 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.572+0000 7f3818ff9640 1 -- 172.21.15.19:0/2794629162 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38100129d0 con 0x7f381c15dc20 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.573+0000 7f3822893640 1 -- 172.21.15.19:0/2794629162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c1e9a00 con 0x7f381c15dc20 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.573+0000 7f3822893640 1 -- 172.21.15.19:0/2794629162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c073a00 con 0x7f381c15dc20 2024-04-25T10:01:37.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.574+0000 7f3818ff9640 1 -- 172.21.15.19:0/2794629162 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3810012b70 con 0x7f381c15dc20 2024-04-25T10:01:37.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.574+0000 7f3818ff9640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f380806b4d0 0x7f380806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.574+0000 7f3818ff9640 1 -- 172.21.15.19:0/2794629162 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f381008a6f0 con 0x7f381c15dc20 2024-04-25T10:01:37.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.575+0000 7f381bfff640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f380806b4d0 0x7f380806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.575+0000 7f381bfff640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f380806b4d0 0x7f380806d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f381c0b8d10 tx=0x7f380c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.578+0000 7f3818ff9640 1 -- 172.21.15.19:0/2794629162 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3810058cf0 con 0x7f381c15dc20 2024-04-25T10:01:37.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.829+0000 7f3822893640 1 -- 172.21.15.19:0/2794629162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f381c0d16b0 con 0x7f381c15dc20 2024-04-25T10:01:37.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.833+0000 7f3818ff9640 1 -- 172.21.15.19:0/2794629162 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f381005cd90 con 0x7f381c15dc20 2024-04-25T10:01:37.838 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T10:01:37.838 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T10:01:37.838 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T10:01:37.838 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T10:01:37.838 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:01:37.838 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T10:01:37.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.835+0000 7f3822893640 1 -- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f380806b4d0 msgr2=0x7f380806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.835+0000 7f3822893640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f380806b4d0 0x7f380806d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f381c0b8d10 tx=0x7f380c009040 comp rx=0 tx=0).stop 2024-04-25T10:01:37.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.835+0000 7f3822893640 1 -- 172.21.15.19:0/2794629162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f381c15dc20 msgr2=0x7f381c0cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.835+0000 7f3822893640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f381c15dc20 0x7f381c0cc6c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f381000d5f0 tx=0x7f381000dab0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.835+0000 7f3822893640 1 -- 172.21.15.19:0/2794629162 shutdown_connections 2024-04-25T10:01:37.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.835+0000 7f3822893640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f380806b4d0 0x7f380806d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.835+0000 7f3822893640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f381c161710 0x7f381c0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.835+0000 7f3822893640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f381c15dc20 0x7f381c0cc6c0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.835+0000 7f3822893640 1 --2- 172.21.15.19:0/2794629162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f381c0ced80 0x7f381c0c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.835+0000 7f3822893640 1 -- 172.21.15.19:0/2794629162 >> 172.21.15.19:0/2794629162 conn(0x7f381c0d0420 msgr2=0x7f381c0b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.835+0000 7f3822893640 1 -- 172.21.15.19:0/2794629162 shutdown_connections 2024-04-25T10:01:37.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.836+0000 7f3822893640 1 -- 172.21.15.19:0/2794629162 wait complete. 2024-04-25T10:01:37.851 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:37.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.947+0000 7f3f4a076640 1 Processor -- start 2024-04-25T10:01:37.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.947+0000 7f3f4a076640 1 -- start start 2024-04-25T10:01:37.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.948+0000 7f3f4a076640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f440d0ad0 0x7f3f44168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.948+0000 7f3f4a076640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f440b9940 con 0x7f3f440d2e90 2024-04-25T10:01:37.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.948+0000 7f3f4a076640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f440d1df0 con 0x7f3f4415dd30 2024-04-25T10:01:37.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.948+0000 7f3f4a076640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f440b7a50 con 0x7f3f440d0ad0 2024-04-25T10:01:37.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.948+0000 7f3f42ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3f440d2e90 0x7f3f440d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56546/0 (socket says 172.21.15.19:56546) 2024-04-25T10:01:37.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.948+0000 7f3f42ffd640 1 -- 172.21.15.19:0/4277991305 learned_addr learned my addr 172.21.15.19:0/4277991305 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:37.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.948+0000 7f3f43fff640 1 --2- 172.21.15.19:0/4277991305 >> v2:172.21.15.112:3300/0 conn(0x7f3f440d0ad0 0x7f3f44168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.948+0000 7f3f427fc640 1 -- 172.21.15.19:0/4277991305 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216706875 0 0) 0x7f3f440b9940 con 0x7f3f440d2e90 2024-04-25T10:01:37.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.948+0000 7f3f427fc640 1 -- 172.21.15.19:0/4277991305 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c003520 con 0x7f3f440d2e90 2024-04-25T10:01:37.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.948+0000 7f3f427fc640 1 -- 172.21.15.19:0/4277991305 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2563737839 0 0) 0x7f3f2c003520 con 0x7f3f440d2e90 2024-04-25T10:01:37.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.949+0000 7f3f43fff640 1 -- 172.21.15.19:0/4277991305 >> v1:172.21.15.112:6789/0 conn(0x7f3f4415dd30 legacy=0x7f3f440b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.949+0000 7f3f43fff640 1 -- 172.21.15.19:0/4277991305 >> v1:172.21.15.19:6789/0 conn(0x7f3f440d2e90 legacy=0x7f3f440d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.949+0000 7f3f43fff640 1 -- 172.21.15.19:0/4277991305 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f440d20d0 con 0x7f3f440d0ad0 2024-04-25T10:01:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.949+0000 7f3f43fff640 1 --2- 172.21.15.19:0/4277991305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f440d0ad0 0x7f3f44168760 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009d90 tx=0x7f3f3c02a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95e92ae5a65c3c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.949+0000 7f3f427fc640 1 -- 172.21.15.19:0/4277991305 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f380017f0 con 0x7f3f440d0ad0 2024-04-25T10:01:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.950+0000 7f3f427fc640 1 -- 172.21.15.19:0/4277991305 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c007540 con 0x7f3f440d0ad0 2024-04-25T10:01:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.950+0000 7f3f427fc640 1 -- 172.21.15.19:0/4277991305 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007910 con 0x7f3f440d0ad0 2024-04-25T10:01:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.950+0000 7f3f4a076640 1 -- 172.21.15.19:0/4277991305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f440d0ad0 msgr2=0x7f3f44168760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.950+0000 7f3f4a076640 1 --2- 172.21.15.19:0/4277991305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f440d0ad0 0x7f3f44168760 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009d90 tx=0x7f3f3c02a560 comp rx=0 tx=0).stop 2024-04-25T10:01:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.950+0000 7f3f4a076640 1 -- 172.21.15.19:0/4277991305 shutdown_connections 2024-04-25T10:01:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.950+0000 7f3f4a076640 1 --2- 172.21.15.19:0/4277991305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f440d0ad0 0x7f3f44168760 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.950+0000 7f3f4a076640 1 -- 172.21.15.19:0/4277991305 >> 172.21.15.19:0/4277991305 conn(0x7f3f44061c30 msgr2=0x7f3f4406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.950+0000 7f3f4a076640 1 -- 172.21.15.19:0/4277991305 shutdown_connections 2024-04-25T10:01:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.950+0000 7f3f4a076640 1 -- 172.21.15.19:0/4277991305 wait complete. 2024-04-25T10:01:37.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.951+0000 7f3f4a076640 1 Processor -- start 2024-04-25T10:01:37.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.951+0000 7f3f4a076640 1 -- start start 2024-04-25T10:01:37.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.952+0000 7f3f4a076640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f440d0ad0 0x7f3f44149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.952+0000 7f3f4a076640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f440d2e90 0x7f3f44159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.952+0000 7f3f4a076640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f4415dd30 0x7f3f440b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.952+0000 7f3f4a076640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440d2970 con 0x7f3f440d2e90 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.952+0000 7f3f4a076640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440d2690 con 0x7f3f440d0ad0 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.952+0000 7f3f4a076640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440d1a10 con 0x7f3f4415dd30 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.952+0000 7f3f42ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f440d2e90 0x7f3f44159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.952+0000 7f3f42ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f440d2e90 0x7f3f44159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56390/0 (socket says 172.21.15.19:56390) 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.952+0000 7f3f42ffd640 1 -- 172.21.15.19:0/2828736516 learned_addr learned my addr 172.21.15.19:0/2828736516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.952+0000 7f3f437fe640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f440d0ad0 0x7f3f44149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.952+0000 7f3f43fff640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f4415dd30 0x7f3f440b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.953+0000 7f3f42ffd640 1 -- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f4415dd30 msgr2=0x7f3f440b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.953+0000 7f3f42ffd640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f4415dd30 0x7f3f440b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.953+0000 7f3f42ffd640 1 -- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f440d0ad0 msgr2=0x7f3f44149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.953+0000 7f3f42ffd640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f440d0ad0 0x7f3f44149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.953+0000 7f3f42ffd640 1 -- 172.21.15.19:0/2828736516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4414a150 con 0x7f3f440d2e90 2024-04-25T10:01:37.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.953+0000 7f3f42ffd640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f440d2e90 0x7f3f44159d10 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c780 tx=0x7f3f3800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.953+0000 7f3f2bfff640 1 -- 172.21.15.19:0/2828736516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38017070 con 0x7f3f440d2e90 2024-04-25T10:01:37.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.953+0000 7f3f2bfff640 1 -- 172.21.15.19:0/2828736516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f38007270 con 0x7f3f440d2e90 2024-04-25T10:01:37.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.953+0000 7f3f4a076640 1 -- 172.21.15.19:0/2828736516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f440b4710 con 0x7f3f440d2e90 2024-04-25T10:01:37.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.953+0000 7f3f2bfff640 1 -- 172.21.15.19:0/2828736516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f380057a0 con 0x7f3f440d2e90 2024-04-25T10:01:37.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.953+0000 7f3f4a076640 1 -- 172.21.15.19:0/2828736516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f440b4cd0 con 0x7f3f440d2e90 2024-04-25T10:01:37.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.954+0000 7f3f4a076640 1 -- 172.21.15.19:0/2828736516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f44073990 con 0x7f3f440d2e90 2024-04-25T10:01:37.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.955+0000 7f3f2bfff640 1 -- 172.21.15.19:0/2828736516 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f3801a070 con 0x7f3f440d2e90 2024-04-25T10:01:37.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.955+0000 7f3f2bfff640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f3006b4d0 0x7f3f3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.955+0000 7f3f2bfff640 1 -- 172.21.15.19:0/2828736516 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3f38089870 con 0x7f3f440d2e90 2024-04-25T10:01:37.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.955+0000 7f3f437fe640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f3006b4d0 0x7f3f3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.956+0000 7f3f437fe640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f3006b4d0 0x7f3f3006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3f34000f60 tx=0x7f3f34009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:37.959+0000 7f3f2bfff640 1 -- 172.21.15.19:0/2828736516 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f38057e80 con 0x7f3f440d2e90 2024-04-25T10:01:38.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.175+0000 7f3f4a076640 1 -- 172.21.15.19:0/2828736516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f440d4250 con 0x7f3f440d2e90 2024-04-25T10:01:38.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.176+0000 7f3f2bfff640 1 -- 172.21.15.19:0/2828736516 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3f3805bf20 con 0x7f3f440d2e90 2024-04-25T10:01:38.180 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:38.181 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:01:30.999245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/250977615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":250977615},{"type":"v1","addr":"172.21.15.19:6835","nonce":250977615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:01:38.181 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 79 2024-04-25T10:01:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 -- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f3006b4d0 msgr2=0x7f3f3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f3006b4d0 0x7f3f3006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3f34000f60 tx=0x7f3f34009040 comp rx=0 tx=0).stop 2024-04-25T10:01:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 -- 172.21.15.19:0/2828736516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f440d2e90 msgr2=0x7f3f44159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f440d2e90 0x7f3f44159d10 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c780 tx=0x7f3f3800cc40 comp rx=0 tx=0).stop 2024-04-25T10:01:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 -- 172.21.15.19:0/2828736516 shutdown_connections 2024-04-25T10:01:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3f3006b4d0 0x7f3f3006d990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f4415dd30 0x7f3f440b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f440d2e90 0x7f3f44159d10 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 --2- 172.21.15.19:0/2828736516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f440d0ad0 0x7f3f44149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 -- 172.21.15.19:0/2828736516 >> 172.21.15.19:0/2828736516 conn(0x7f3f44061c30 msgr2=0x7f3f4406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 -- 172.21.15.19:0/2828736516 shutdown_connections 2024-04-25T10:01:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.178+0000 7f3f4a076640 1 -- 172.21.15.19:0/2828736516 wait complete. 2024-04-25T10:01:38.467 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:38.469 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:38.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:01:29.000131+0000', 'modified': '2024-04-25T10:01:30.999245+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9213}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9213': {'gid': 9213, 'name': 'c', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6835/250977615', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 250977615}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 250977615}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:01:38.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:01:38.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:01:38.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.696+0000 7f97ccc93640 1 Processor -- start 2024-04-25T10:01:38.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.696+0000 7f97ccc93640 1 -- start start 2024-04-25T10:01:38.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97ccc93640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f97c815dc20 0x7f97c80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97ccc93640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97c80d32f0 0x7f97c80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97ccc93640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c815e000 con 0x7f97c8161710 2024-04-25T10:01:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97ccc93640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c80d1de0 con 0x7f97c815dc20 2024-04-25T10:01:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97ccc93640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8161c60 con 0x7f97c80d32f0 2024-04-25T10:01:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97c6d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97c80d32f0 0x7f97c80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97c659c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f97c815dc20 0x7f97c80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97bffff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f97c8161710 0x7f97c806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:52320/0 (socket says 172.21.15.19:52320) 2024-04-25T10:01:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97bffff640 1 -- 172.21.15.19:0/71727773 learned_addr learned my addr 172.21.15.19:0/71727773 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97c659c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f97c815dc20 0x7f97c80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:36328/0 (socket says 172.21.15.19:36328) 2024-04-25T10:01:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97c5d9b640 1 -- 172.21.15.19:0/71727773 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509266140 0 0) 0x7f97c815e000 con 0x7f97c8161710 2024-04-25T10:01:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97c6d9d640 1 -- 172.21.15.19:0/71727773 >> v2:172.21.15.112:3300/0 conn(0x7f97c815dc20 msgr2=0x7f97c80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97c6d9d640 1 --2- 172.21.15.19:0/71727773 >> v2:172.21.15.112:3300/0 conn(0x7f97c815dc20 0x7f97c80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97c6d9d640 1 -- 172.21.15.19:0/71727773 >> v1:172.21.15.112:6789/0 conn(0x7f97c8161710 legacy=0x7f97c806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.697+0000 7f97c6d9d640 1 -- 172.21.15.19:0/71727773 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c8161f40 con 0x7f97c80d32f0 2024-04-25T10:01:38.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.698+0000 7f97c6d9d640 1 --2- 172.21.15.19:0/71727773 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97c80d32f0 0x7f97c80cebb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009d90 tx=0x7f97c002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=324125a7dd1a9169 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.698+0000 7f97c5d9b640 1 -- 172.21.15.19:0/71727773 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0007740 con 0x7f97c80d32f0 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.698+0000 7f97c5d9b640 1 -- 172.21.15.19:0/71727773 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c00078e0 con 0x7f97c80d32f0 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.698+0000 7f97c5d9b640 1 -- 172.21.15.19:0/71727773 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0007cb0 con 0x7f97c80d32f0 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.699+0000 7f97ccc93640 1 -- 172.21.15.19:0/71727773 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97c80d32f0 msgr2=0x7f97c80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.699+0000 7f97ccc93640 1 --2- 172.21.15.19:0/71727773 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97c80d32f0 0x7f97c80cebb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009d90 tx=0x7f97c002a660 comp rx=0 tx=0).stop 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.699+0000 7f97ccc93640 1 -- 172.21.15.19:0/71727773 shutdown_connections 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.699+0000 7f97ccc93640 1 --2- 172.21.15.19:0/71727773 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97c80d32f0 0x7f97c80cebb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.699+0000 7f97ccc93640 1 --2- 172.21.15.19:0/71727773 >> v2:172.21.15.112:3300/0 conn(0x7f97c815dc20 0x7f97c80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.699+0000 7f97ccc93640 1 -- 172.21.15.19:0/71727773 >> 172.21.15.19:0/71727773 conn(0x7f97c80d0420 msgr2=0x7f97c80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.699+0000 7f97ccc93640 1 -- 172.21.15.19:0/71727773 shutdown_connections 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.699+0000 7f97ccc93640 1 -- 172.21.15.19:0/71727773 wait complete. 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.700+0000 7f97ccc93640 1 Processor -- start 2024-04-25T10:01:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.700+0000 7f97ccc93640 1 -- start start 2024-04-25T10:01:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.701+0000 7f97ccc93640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c80d32f0 0x7f97c81605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.701+0000 7f97ccc93640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97c815dc20 0x7f97c80aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.701+0000 7f97ccc93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97c8161710 0x7f97c81f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.701+0000 7f97ccc93640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80b5e50 con 0x7f97c8161710 2024-04-25T10:01:38.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.701+0000 7f97ccc93640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8162500 con 0x7f97c80d32f0 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.701+0000 7f97ccc93640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80b5fd0 con 0x7f97c815dc20 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.701+0000 7f97c6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97c8161710 0x7f97c81f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.701+0000 7f97c6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97c8161710 0x7f97c81f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56398/0 (socket says 172.21.15.19:56398) 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.701+0000 7f97c6d9d640 1 -- 172.21.15.19:0/2587992722 learned_addr learned my addr 172.21.15.19:0/2587992722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.701+0000 7f97bffff640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97c815dc20 0x7f97c80aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.701+0000 7f97c659c640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c80d32f0 0x7f97c81605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.702+0000 7f97c6d9d640 1 -- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97c815dc20 msgr2=0x7f97c80aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.702+0000 7f97c6d9d640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97c815dc20 0x7f97c80aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.702+0000 7f97c6d9d640 1 -- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c80d32f0 msgr2=0x7f97c81605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.702+0000 7f97c6d9d640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c80d32f0 0x7f97c81605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.702+0000 7f97c6d9d640 1 -- 172.21.15.19:0/2587992722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c80ccd40 con 0x7f97c8161710 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.702+0000 7f97c6d9d640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97c8161710 0x7f97c81f6570 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009eb0 tx=0x7f97c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.702+0000 7f97beffd640 1 -- 172.21.15.19:0/2587992722 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c003f070 con 0x7f97c8161710 2024-04-25T10:01:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.702+0000 7f97beffd640 1 -- 172.21.15.19:0/2587992722 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c0031c40 con 0x7f97c8161710 2024-04-25T10:01:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.702+0000 7f97ccc93640 1 -- 172.21.15.19:0/2587992722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c80cd020 con 0x7f97c8161710 2024-04-25T10:01:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.702+0000 7f97beffd640 1 -- 172.21.15.19:0/2587992722 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0009450 con 0x7f97c8161710 2024-04-25T10:01:38.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.702+0000 7f97ccc93640 1 -- 172.21.15.19:0/2587992722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c80cd5e0 con 0x7f97c8161710 2024-04-25T10:01:38.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.704+0000 7f97beffd640 1 -- 172.21.15.19:0/2587992722 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97c0003990 con 0x7f97c8161710 2024-04-25T10:01:38.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.704+0000 7f97beffd640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f97a806b4d0 0x7f97a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.704+0000 7f97beffd640 1 -- 172.21.15.19:0/2587992722 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97c00ab040 con 0x7f97c8161710 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.704+0000 7f97c659c640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f97a806b4d0 0x7f97a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.705+0000 7f97c659c640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f97a806b4d0 0x7f97a806d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f97c8161210 tx=0x7f97b8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.705+0000 7f97cf991740 1 Processor -- start 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.705+0000 7f97cf991740 1 -- start start 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97cf991740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e7230 0x560f028e7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97cf991740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f02905020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97cf991740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028bcf10 0x560f02903190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97cf991740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0281fa10 con 0x560f02904ad0 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97cf991740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f027c8b30 con 0x560f028e7230 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97cf991740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f0281bc50 con 0x560f028bcf10 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f02905020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f02905020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56412/0 (socket says 172.21.15.19:56412) 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97bffff640 1 -- 172.21.15.19:0/2394963492 learned_addr learned my addr 172.21.15.19:0/2394963492 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97c659c640 1 --2- 172.21.15.19:0/2394963492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e7230 0x560f028e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97c6d9d640 1 --2- 172.21.15.19:0/2394963492 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028bcf10 0x560f02903190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97bffff640 1 -- 172.21.15.19:0/2394963492 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028bcf10 msgr2=0x560f02903190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97bffff640 1 --2- 172.21.15.19:0/2394963492 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028bcf10 0x560f02903190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97bffff640 1 -- 172.21.15.19:0/2394963492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e7230 msgr2=0x560f028e7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97bffff640 1 --2- 172.21.15.19:0/2394963492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e7230 0x560f028e7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97bffff640 1 -- 172.21.15.19:0/2394963492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f027dde10 con 0x560f02904ad0 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97bffff640 1 --2- 172.21.15.19:0/2394963492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f02905020 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f97b00076b0 tx=0x7f97b00077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73b1c0dd6746140 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97a7fff640 1 -- 172.21.15.19:0/2394963492 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b001a070 con 0x560f02904ad0 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97a7fff640 1 -- 172.21.15.19:0/2394963492 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b00017f0 con 0x560f02904ad0 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.706+0000 7f97a7fff640 1 -- 172.21.15.19:0/2394963492 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0001bc0 con 0x560f02904ad0 2024-04-25T10:01:38.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.707+0000 7f97cf991740 1 -- 172.21.15.19:0/2394963492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 msgr2=0x560f02905020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.707+0000 7f97cf991740 1 --2- 172.21.15.19:0/2394963492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f02905020 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f97b00076b0 tx=0x7f97b00077a0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.707+0000 7f97cf991740 1 -- 172.21.15.19:0/2394963492 shutdown_connections 2024-04-25T10:01:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.707+0000 7f97cf991740 1 --2- 172.21.15.19:0/2394963492 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028bcf10 0x560f02903190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.707+0000 7f97cf991740 1 --2- 172.21.15.19:0/2394963492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f02905020 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.707+0000 7f97cf991740 1 --2- 172.21.15.19:0/2394963492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e7230 0x560f028e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.707+0000 7f97cf991740 1 -- 172.21.15.19:0/2394963492 >> 172.21.15.19:0/2394963492 conn(0x560f028b9840 msgr2=0x560f02902040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.707+0000 7f97cf991740 1 -- 172.21.15.19:0/2394963492 shutdown_connections 2024-04-25T10:01:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.707+0000 7f97cf991740 1 -- 172.21.15.19:0/2394963492 wait complete. 2024-04-25T10:01:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.708+0000 7f97cf991740 1 -- start start 2024-04-25T10:01:38.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.708+0000 7f97cf991740 1 Processor -- start 2024-04-25T10:01:38.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97cf991740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e7230 0x560f028d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97cf991740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97cf991740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e9360 0x560f028e5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97cf991740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f028bdcb0 con 0x560f02904ad0 2024-04-25T10:01:38.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97cf991740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f027dc0c0 con 0x560f028e9360 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97cf991740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f028b7770 con 0x560f028e7230 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97cf991740 10 client.0 _ll_register_callbacks cb 0x560f027cfec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56418/0 (socket says 172.21.15.19:56418) 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97bffff640 1 -- 172.21.15.19:0/3760643125 learned_addr learned my addr 172.21.15.19:0/3760643125 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97c659c640 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e7230 0x560f028d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97c6d9d640 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e9360 0x560f028e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97bffff640 1 -- 172.21.15.19:0/3760643125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e7230 msgr2=0x560f028d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97bffff640 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e7230 0x560f028d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97bffff640 1 -- 172.21.15.19:0/3760643125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e9360 msgr2=0x560f028e5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97bffff640 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e9360 0x560f028e5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97bffff640 1 -- 172.21.15.19:0/3760643125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f025342d0 con 0x560f02904ad0 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97bffff640 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028d0bf0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f97b0013000 tx=0x7f97b0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962448e0f7ca9997 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.709+0000 7f97a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97a57fa640 1 -- 172.21.15.19:0/3760643125 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0025060 con 0x560f02904ad0 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97a57fa640 1 -- 172.21.15.19:0/3760643125 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97b0014860 con 0x560f02904ad0 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97cf991740 1 -- 172.21.15.19:0/3760643125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97b00017f0 con 0x560f02904ad0 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97a57fa640 1 -- 172.21.15.19:0/3760643125 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97b002b050 con 0x560f02904ad0 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97cf991740 10 client.9381 fetch_fsmap learned FSMap version 79 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97cf991740 1 -- 172.21.15.19:0/3760643125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x560f025fe6b0 con 0x560f02904ad0 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97a57fa640 1 -- 172.21.15.19:0/3760643125 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b00017f0 con 0x560f02904ad0 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97a57fa640 1 -- 172.21.15.19:0/3760643125 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f97b0014a60 con 0x560f02904ad0 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97cf991740 10 client.9381 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97cf991740 10 client.9381 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97cf991740 20 client.9381 populate_metadata read hostname 'smithi019' 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97cf991740 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x560f028e80a0 0x560f028e8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97cf991740 4 client.9381 mds_command: new command op to 9213 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:38.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97cf991740 1 -- 172.21.15.19:0/3760643125 --> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f0287e920 con 0x560f028e80a0 2024-04-25T10:01:38.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97c659c640 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x560f028e80a0 0x560f028e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97c659c640 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x560f028e80a0 0x560f028e8480 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56b15b81d1674f server_cookie=4c6fb32040f47c40 in_seq=0 out_seq=0 2024-04-25T10:01:38.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.710+0000 7f97a57fa640 10 client.9381 ms_handle_connect on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:38.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97a57fa640 1 -- 172.21.15.19:0/3760643125 <== mds.0 v2:172.21.15.19:6833/250977615 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f0287e920 con 0x560f028e80a0 2024-04-25T10:01:38.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97a57fa640 10 client.9381 handle_command_reply: tid=0 2024-04-25T10:01:38.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97cf991740 1 client.9381 shutdown 2024-04-25T10:01:38.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97cf991740 1 -- 172.21.15.19:0/3760643125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 msgr2=0x560f028d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97cf991740 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028d0bf0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f97b0013000 tx=0x7f97b0000ed0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97cf991740 1 -- 172.21.15.19:0/3760643125 shutdown_connections 2024-04-25T10:01:38.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97cf991740 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x560f028e80a0 0x560f028e8480 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97cf991740 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e9360 0x560f028e5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97cf991740 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028d0bf0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97cf991740 1 --2- 172.21.15.19:0/3760643125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e7230 0x560f028d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97a57fa640 0 client.9381 ms_handle_reset on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:38.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97cf991740 1 -- 172.21.15.19:0/3760643125 >> 172.21.15.19:0/3760643125 conn(0x560f028bcf10 msgr2=0x560f029055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97cf991740 1 -- 172.21.15.19:0/3760643125 shutdown_connections 2024-04-25T10:01:38.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.711+0000 7f97cf991740 1 -- 172.21.15.19:0/3760643125 wait complete. 2024-04-25T10:01:38.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.712+0000 7f97cf991740 20 client.9381 trim_cache size 0 max 16384 2024-04-25T10:01:38.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.727+0000 7f97cf991740 1 Processor -- start 2024-04-25T10:01:38.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.727+0000 7f97cf991740 1 -- start start 2024-04-25T10:01:38.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.728+0000 7f97cf991740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e7230 0x560f028ba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.728+0000 7f97cf991740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e9360 0x560f028e5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.728+0000 7f97cf991740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.728+0000 7f97cf991740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c81ff6c0 con 0x560f02904ad0 2024-04-25T10:01:38.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.728+0000 7f97cf991740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c81f4880 con 0x560f028e7230 2024-04-25T10:01:38.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.728+0000 7f97cf991740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c81f4e80 con 0x560f028e9360 2024-04-25T10:01:38.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.728+0000 7f97c6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.728+0000 7f97c6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56424/0 (socket says 172.21.15.19:56424) 2024-04-25T10:01:38.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.728+0000 7f97c6d9d640 1 -- 172.21.15.19:0/1868195474 learned_addr learned my addr 172.21.15.19:0/1868195474 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:38.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.728+0000 7f97bffff640 1 --2- 172.21.15.19:0/1868195474 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e9360 0x560f028e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.728+0000 7f97c6d9d640 1 -- 172.21.15.19:0/1868195474 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e9360 msgr2=0x560f028e5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97c6d9d640 1 --2- 172.21.15.19:0/1868195474 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e9360 0x560f028e5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97c6d9d640 1 -- 172.21.15.19:0/1868195474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e7230 msgr2=0x560f028ba3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97c659c640 1 --2- 172.21.15.19:0/1868195474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e7230 0x560f028ba3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97c6d9d640 1 --2- 172.21.15.19:0/1868195474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e7230 0x560f028ba3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97c6d9d640 1 -- 172.21.15.19:0/1868195474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f028e7ae0 con 0x560f02904ad0 2024-04-25T10:01:38.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97c6d9d640 1 --2- 172.21.15.19:0/1868195474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028b2bc0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f97c00795f0 tx=0x7f97c0009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51f72a8d2c71b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97a77fe640 1 -- 172.21.15.19:0/1868195474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0048040 con 0x560f02904ad0 2024-04-25T10:01:38.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97a77fe640 1 -- 172.21.15.19:0/1868195474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c0007840 con 0x560f02904ad0 2024-04-25T10:01:38.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97a77fe640 1 -- 172.21.15.19:0/1868195474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00980c0 con 0x560f02904ad0 2024-04-25T10:01:38.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97cf991740 1 -- 172.21.15.19:0/1868195474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 msgr2=0x560f028b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97cf991740 1 --2- 172.21.15.19:0/1868195474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028b2bc0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f97c00795f0 tx=0x7f97c0009a70 comp rx=0 tx=0).stop 2024-04-25T10:01:38.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97cf991740 1 -- 172.21.15.19:0/1868195474 shutdown_connections 2024-04-25T10:01:38.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97cf991740 1 --2- 172.21.15.19:0/1868195474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f02904ad0 0x560f028b2bc0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97cf991740 1 --2- 172.21.15.19:0/1868195474 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e9360 0x560f028e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97cf991740 1 --2- 172.21.15.19:0/1868195474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f028e7230 0x560f028ba3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97cf991740 1 -- 172.21.15.19:0/1868195474 >> 172.21.15.19:0/1868195474 conn(0x560f028bcf10 msgr2=0x560f028f8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97cf991740 1 -- 172.21.15.19:0/1868195474 shutdown_connections 2024-04-25T10:01:38.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.729+0000 7f97cf991740 1 -- 172.21.15.19:0/1868195474 wait complete. 2024-04-25T10:01:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.730+0000 7f97cf991740 1 -- start start 2024-04-25T10:01:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.730+0000 7f97cf991740 1 Processor -- start 2024-04-25T10:01:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97cf991740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f028e7230 0x560f0296c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97cf991740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e9360 0x560f02902f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97cf991740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f02904ad0 0x560f02903440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97cf991740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f028ea300 con 0x560f028e7230 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97cf991740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f028e9d40 con 0x560f02904ad0 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97cf991740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f028ead30 con 0x560f028e9360 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97cf991740 10 client.0 _ll_register_callbacks cb 0x560f028b7c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97c659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f028e7230 0x560f0296c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97c659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f028e7230 0x560f0296c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56426/0 (socket says 172.21.15.19:56426) 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97c659c640 1 -- 172.21.15.19:0/1167935131 learned_addr learned my addr 172.21.15.19:0/1167935131 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97bffff640 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e9360 0x560f02902f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97c6d9d640 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f02904ad0 0x560f02903440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97c659c640 1 -- 172.21.15.19:0/1167935131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e9360 msgr2=0x560f02902f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97c659c640 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e9360 0x560f02902f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97c659c640 1 -- 172.21.15.19:0/1167935131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f02904ad0 msgr2=0x560f02903440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97c659c640 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f02904ad0 0x560f02903440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.731+0000 7f97c659c640 1 -- 172.21.15.19:0/1167935131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f02970ba0 con 0x560f028e7230 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97c659c640 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f028e7230 0x560f0296c500 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f97b8016040 tx=0x7f97b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20ea33815c42dbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97a57fa640 1 -- 172.21.15.19:0/1167935131 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8004490 con 0x560f028e7230 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97a57fa640 1 -- 172.21.15.19:0/1167935131 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97b802a2c0 con 0x560f028e7230 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97cf991740 1 -- 172.21.15.19:0/1167935131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97c0007840 con 0x560f028e7230 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97a57fa640 1 -- 172.21.15.19:0/1167935131 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97b8039050 con 0x560f028e7230 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97cf991740 10 client.9387 fetch_fsmap learned FSMap version 79 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97cf991740 1 -- 172.21.15.19:0/1167935131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x560f028b2bc0 con 0x560f028e7230 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97a57fa640 1 -- 172.21.15.19:0/1167935131 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c0007840 con 0x560f028e7230 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97a57fa640 1 -- 172.21.15.19:0/1167935131 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f97b803c020 con 0x560f028e7230 2024-04-25T10:01:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97cf991740 10 client.9387 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97cf991740 10 client.9387 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97cf991740 20 client.9387 populate_metadata read hostname 'smithi019' 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97cf991740 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x560f0291c5e0 0x560f028e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97cf991740 4 client.9387 mds_command: new command op to 9213 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97cf991740 1 -- 172.21.15.19:0/1167935131 --> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560f028fa180 con 0x560f0291c5e0 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.732+0000 7f97bffff640 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x560f0291c5e0 0x560f028e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97bffff640 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x560f0291c5e0 0x560f028e3d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a28578f7477e104b server_cookie=de19250e913ed7a5 in_seq=0 out_seq=0 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97a57fa640 10 client.9387 ms_handle_connect on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97a57fa640 1 -- 172.21.15.19:0/1167935131 <== mds.0 v2:172.21.15.19:6833/250977615 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x560f028fa180 con 0x560f0291c5e0 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97a57fa640 10 client.9387 handle_command_reply: tid=0 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 1 client.9387 shutdown 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 1 -- 172.21.15.19:0/1167935131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f028e7230 msgr2=0x560f0296c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f028e7230 0x560f0296c500 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f97b8016040 tx=0x7f97b8004730 comp rx=0 tx=0).stop 2024-04-25T10:01:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 1 -- 172.21.15.19:0/1167935131 shutdown_connections 2024-04-25T10:01:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x560f0291c5e0 0x560f028e3d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f02904ad0 0x560f02903440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f028e9360 0x560f02902f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 1 --2- 172.21.15.19:0/1167935131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f028e7230 0x560f0296c500 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97a57fa640 0 client.9387 ms_handle_reset on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 1 -- 172.21.15.19:0/1167935131 >> 172.21.15.19:0/1167935131 conn(0x560f028bcf10 msgr2=0x560f028f9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 1 -- 172.21.15.19:0/1167935131 shutdown_connections 2024-04-25T10:01:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 1 -- 172.21.15.19:0/1167935131 wait complete. 2024-04-25T10:01:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.733+0000 7f97cf991740 20 client.9387 trim_cache size 0 max 16384 2024-04-25T10:01:38.746 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 9213, 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 9.0837022429999994, 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 79, 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 129, 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 129, 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 20.610181133000001 2024-04-25T10:01:38.747 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:01:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 -- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f97a806b4d0 msgr2=0x7f97a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f97a806b4d0 0x7f97a806d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f97c8161210 tx=0x7f97b8009190 comp rx=0 tx=0).stop 2024-04-25T10:01:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 -- 172.21.15.19:0/2587992722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97c8161710 msgr2=0x7f97c81f6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97c8161710 0x7f97c81f6570 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009eb0 tx=0x7f97c0004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 -- 172.21.15.19:0/2587992722 shutdown_connections 2024-04-25T10:01:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f97a806b4d0 0x7f97a806d990 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97c8161710 0x7f97c81f6570 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97c815dc20 0x7f97c80aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 --2- 172.21.15.19:0/2587992722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97c80d32f0 0x7f97c81605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 -- 172.21.15.19:0/2587992722 >> 172.21.15.19:0/2587992722 conn(0x7f97c80d0420 msgr2=0x7f97c815b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 -- 172.21.15.19:0/2587992722 shutdown_connections 2024-04-25T10:01:38.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.734+0000 7f97ccc93640 1 -- 172.21.15.19:0/2587992722 wait complete. 2024-04-25T10:01:38.750 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:01:38.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.847+0000 7f1905e3f640 1 Processor -- start 2024-04-25T10:01:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.847+0000 7f1905e3f640 1 -- start start 2024-04-25T10:01:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f1905e3f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f190006c1d0 0x7f1900168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f1905e3f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19000adcf0 con 0x7f1900071ac0 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f1905e3f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190015ec60 con 0x7f1900154ed0 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f1905e3f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190015ef40 con 0x7f190006c1d0 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f18feffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1900071ac0 0x7f19000ad5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56550/0 (socket says 172.21.15.19:56550) 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f18feffd640 1 -- 172.21.15.19:0/2580058875 learned_addr learned my addr 172.21.15.19:0/2580058875 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f18fffff640 1 --2- 172.21.15.19:0/2580058875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f190006c1d0 0x7f1900168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f18fe7fc640 1 -- 172.21.15.19:0/2580058875 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978048092 0 0) 0x7f19000adcf0 con 0x7f1900071ac0 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f18fe7fc640 1 -- 172.21.15.19:0/2580058875 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e8003520 con 0x7f1900071ac0 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f18fffff640 1 -- 172.21.15.19:0/2580058875 >> v1:172.21.15.112:6789/0 conn(0x7f1900154ed0 legacy=0x7f19000aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f18fffff640 1 -- 172.21.15.19:0/2580058875 >> v1:172.21.15.19:6789/0 conn(0x7f1900071ac0 legacy=0x7f19000ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f18fe7fc640 1 -- 172.21.15.19:0/2580058875 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1833431635 0 0) 0x7f18e8003520 con 0x7f1900071ac0 2024-04-25T10:01:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.848+0000 7f18fffff640 1 -- 172.21.15.19:0/2580058875 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190006db90 con 0x7f190006c1d0 2024-04-25T10:01:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.849+0000 7f18fffff640 1 --2- 172.21.15.19:0/2580058875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f190006c1d0 0x7f1900168930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009dc0 tx=0x7f18f802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce5ac10f1709df7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.849+0000 7f18fe7fc640 1 -- 172.21.15.19:0/2580058875 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0001670 con 0x7f190006c1d0 2024-04-25T10:01:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.849+0000 7f18fe7fc640 1 -- 172.21.15.19:0/2580058875 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f8007540 con 0x7f190006c1d0 2024-04-25T10:01:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.849+0000 7f18fe7fc640 1 -- 172.21.15.19:0/2580058875 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8007910 con 0x7f190006c1d0 2024-04-25T10:01:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.850+0000 7f1905e3f640 1 -- 172.21.15.19:0/2580058875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f190006c1d0 msgr2=0x7f1900168930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.850+0000 7f1905e3f640 1 --2- 172.21.15.19:0/2580058875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f190006c1d0 0x7f1900168930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009dc0 tx=0x7f18f802a660 comp rx=0 tx=0).stop 2024-04-25T10:01:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.850+0000 7f1905e3f640 1 -- 172.21.15.19:0/2580058875 shutdown_connections 2024-04-25T10:01:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.850+0000 7f1905e3f640 1 --2- 172.21.15.19:0/2580058875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f190006c1d0 0x7f1900168930 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.850+0000 7f1905e3f640 1 -- 172.21.15.19:0/2580058875 >> 172.21.15.19:0/2580058875 conn(0x7f19000af670 msgr2=0x7f19000b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.850+0000 7f1905e3f640 1 -- 172.21.15.19:0/2580058875 shutdown_connections 2024-04-25T10:01:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.850+0000 7f1905e3f640 1 -- 172.21.15.19:0/2580058875 wait complete. 2024-04-25T10:01:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.851+0000 7f1905e3f640 1 Processor -- start 2024-04-25T10:01:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.851+0000 7f1905e3f640 1 -- start start 2024-04-25T10:01:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f1905e3f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1900071ac0 0x7f1900148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f1905e3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1900154ed0 0x7f1900148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f1905e3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19001492f0 0x7f190014a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f1905e3f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1900168ff0 con 0x7f19001492f0 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f1905e3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1900168e70 con 0x7f1900154ed0 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f1905e3f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1900169170 con 0x7f1900071ac0 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f18fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19001492f0 0x7f190014a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f18fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19001492f0 0x7f190014a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56438/0 (socket says 172.21.15.19:56438) 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f18fffff640 1 -- 172.21.15.19:0/986266437 learned_addr learned my addr 172.21.15.19:0/986266437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f18ff7fe640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1900071ac0 0x7f1900148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f18feffd640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1900154ed0 0x7f1900148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f18fffff640 1 -- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1900071ac0 msgr2=0x7f1900148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f18fffff640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1900071ac0 0x7f1900148220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f18fffff640 1 -- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1900154ed0 msgr2=0x7f1900148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f18fffff640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1900154ed0 0x7f1900148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f18fffff640 1 -- 172.21.15.19:0/986266437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190014b590 con 0x7f19001492f0 2024-04-25T10:01:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.852+0000 7f18feffd640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1900154ed0 0x7f1900148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.853+0000 7f18fffff640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19001492f0 0x7f190014a820 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009ee0 tx=0x7f18f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.853+0000 7f18e7fff640 1 -- 172.21.15.19:0/986266437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8040070 con 0x7f19001492f0 2024-04-25T10:01:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.853+0000 7f18e7fff640 1 -- 172.21.15.19:0/986266437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f8031bc0 con 0x7f19001492f0 2024-04-25T10:01:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.853+0000 7f1905e3f640 1 -- 172.21.15.19:0/986266437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190014b870 con 0x7f19001492f0 2024-04-25T10:01:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.853+0000 7f18e7fff640 1 -- 172.21.15.19:0/986266437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8009450 con 0x7f19001492f0 2024-04-25T10:01:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.853+0000 7f1905e3f640 1 -- 172.21.15.19:0/986266437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190014be30 con 0x7f19001492f0 2024-04-25T10:01:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.854+0000 7f1905e3f640 1 -- 172.21.15.19:0/986266437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19000b2b10 con 0x7f19001492f0 2024-04-25T10:01:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.854+0000 7f18e7fff640 1 -- 172.21.15.19:0/986266437 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18f8039040 con 0x7f19001492f0 2024-04-25T10:01:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.855+0000 7f18e7fff640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f18ec06b500 0x7f18ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.855+0000 7f18e7fff640 1 -- 172.21.15.19:0/986266437 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f18f80aba90 con 0x7f19001492f0 2024-04-25T10:01:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.855+0000 7f18ff7fe640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f18ec06b500 0x7f18ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.855+0000 7f18ff7fe640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f18ec06b500 0x7f18ec06d9c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f18f4000f60 tx=0x7f18f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:38.859+0000 7f18e7fff640 1 -- 172.21.15.19:0/986266437 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18f807a8b0 con 0x7f19001492f0 2024-04-25T10:01:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.076+0000 7f1905e3f640 1 -- 172.21.15.19:0/986266437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1900073990 con 0x7f19001492f0 2024-04-25T10:01:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.076+0000 7f18e7fff640 1 -- 172.21.15.19:0/986266437 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) v1 ==== 53+0+73 (secure 0 0 0) 0x7f18f807a1b0 con 0x7f19001492f0 2024-04-25T10:01:39.081 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:01:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.078+0000 7f1905e3f640 1 -- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f18ec06b500 msgr2=0x7f18ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.078+0000 7f1905e3f640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f18ec06b500 0x7f18ec06d9c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f18f4000f60 tx=0x7f18f4009040 comp rx=0 tx=0).stop 2024-04-25T10:01:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.078+0000 7f1905e3f640 1 -- 172.21.15.19:0/986266437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19001492f0 msgr2=0x7f190014a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.078+0000 7f1905e3f640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19001492f0 0x7f190014a820 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009ee0 tx=0x7f18f8004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.078+0000 7f1905e3f640 1 -- 172.21.15.19:0/986266437 shutdown_connections 2024-04-25T10:01:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.078+0000 7f1905e3f640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f18ec06b500 0x7f18ec06d9c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.078+0000 7f1905e3f640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19001492f0 0x7f190014a820 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.078+0000 7f1905e3f640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1900154ed0 0x7f1900148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.078+0000 7f1905e3f640 1 --2- 172.21.15.19:0/986266437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1900071ac0 0x7f1900148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.078+0000 7f1905e3f640 1 -- 172.21.15.19:0/986266437 >> 172.21.15.19:0/986266437 conn(0x7f190006c1d0 msgr2=0x7f190015e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.079+0000 7f1905e3f640 1 -- 172.21.15.19:0/986266437 shutdown_connections 2024-04-25T10:01:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.079+0000 7f1905e3f640 1 -- 172.21.15.19:0/986266437 wait complete. 2024-04-25T10:01:39.094 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:01:39.094 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:01:39.094 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T10:01:39.094 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:01:39.094 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:39.094 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:01:39.094 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:01:39.094 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:01:39.094 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:39.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:39.095 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:39.111 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42247sec preferred_lft 42247sec 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:39.112 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:39.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:39.114 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:01:39.174 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T10:01:39.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:39.176 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T10:01:39.233 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T10:01:39.233 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:01:39.233 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:01:39.233 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:01:39.233 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:01:39.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:39.234 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T10:01:39.234 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:39.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:01:39.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:01:39.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:01:39.234 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:39.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:39.319 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T10:01:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:01:39.334 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:01:39.335 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:39.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:39.335 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T10:01:39.399 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:01:39.399 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:39.455 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:39.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:01:39.455 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:39.510 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:01:39.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:39.510 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:01:39.592 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:01:39.592 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:01:39.606 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:01:39.606 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:01:39.606 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T10:01:39.606 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T10:01:39.607 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T10:01:39.607 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:01:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:01:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:01:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:01:39.607 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:01:39.609 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:01:39.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:39.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:39.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:39.675 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:01:39.736 INFO:teuthology.orchestra.run.smithi019.stdout:client7265 2024-04-25T10:01:39.736 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:01:39.736 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:39.792 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:39.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.928+0000 7f44e685d640 1 Processor -- start 2024-04-25T10:01:39.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.928+0000 7f44e685d640 1 -- start start 2024-04-25T10:01:39.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44e685d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f44e0154ee0 0x7f44e00cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44e685d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f44e00cfcf0 0x7f44e00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44e685d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44e00ce3c0 0x7f44e015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44e685d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e00ce830 con 0x7f44e00cfcf0 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44e685d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e0072090 con 0x7f44e0154ee0 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44e685d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e0071230 con 0x7f44e00ce3c0 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44df7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f44e00cfcf0 0x7f44e00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44df7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f44e00cfcf0 0x7f44e00cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56446/0 (socket says 172.21.15.19:56446) 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44df7fe640 1 -- 172.21.15.19:0/4082688732 learned_addr learned my addr 172.21.15.19:0/4082688732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44dffff640 1 --2- 172.21.15.19:0/4082688732 >> v2:172.21.15.112:3300/0 conn(0x7f44e0154ee0 0x7f44e00cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44e4dfa640 1 --2- 172.21.15.19:0/4082688732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44e00ce3c0 0x7f44e015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44df7fe640 1 -- 172.21.15.19:0/4082688732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44e00ce3c0 msgr2=0x7f44e015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44df7fe640 1 --2- 172.21.15.19:0/4082688732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44e00ce3c0 0x7f44e015e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44df7fe640 1 -- 172.21.15.19:0/4082688732 >> v2:172.21.15.112:3300/0 conn(0x7f44e0154ee0 msgr2=0x7f44e00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44df7fe640 1 --2- 172.21.15.19:0/4082688732 >> v2:172.21.15.112:3300/0 conn(0x7f44e0154ee0 0x7f44e00cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44df7fe640 1 -- 172.21.15.19:0/4082688732 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e0071480 con 0x7f44e00cfcf0 2024-04-25T10:01:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.929+0000 7f44e4dfa640 1 --2- 172.21.15.19:0/4082688732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44e00ce3c0 0x7f44e015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.930+0000 7f44df7fe640 1 --2- 172.21.15.19:0/4082688732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e00cfcf0 0x7f44e00cde80 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f44e00d02e0 tx=0x7f44d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a5ac0aaaf0418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.930+0000 7f44deffd640 1 -- 172.21.15.19:0/4082688732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d0037070 con 0x7f44e00cfcf0 2024-04-25T10:01:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.930+0000 7f44deffd640 1 -- 172.21.15.19:0/4082688732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d0032440 con 0x7f44e00cfcf0 2024-04-25T10:01:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.930+0000 7f44deffd640 1 -- 172.21.15.19:0/4082688732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d0032810 con 0x7f44e00cfcf0 2024-04-25T10:01:39.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.931+0000 7f44e685d640 1 -- 172.21.15.19:0/4082688732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e00cfcf0 msgr2=0x7f44e00cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.931+0000 7f44e685d640 1 --2- 172.21.15.19:0/4082688732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e00cfcf0 0x7f44e00cde80 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f44e00d02e0 tx=0x7f44d002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.931+0000 7f44e685d640 1 -- 172.21.15.19:0/4082688732 shutdown_connections 2024-04-25T10:01:39.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.931+0000 7f44e685d640 1 --2- 172.21.15.19:0/4082688732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44e00ce3c0 0x7f44e015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.931+0000 7f44e685d640 1 --2- 172.21.15.19:0/4082688732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e00cfcf0 0x7f44e00cde80 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.931+0000 7f44e685d640 1 --2- 172.21.15.19:0/4082688732 >> v2:172.21.15.112:3300/0 conn(0x7f44e0154ee0 0x7f44e00cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.931+0000 7f44e685d640 1 -- 172.21.15.19:0/4082688732 >> 172.21.15.19:0/4082688732 conn(0x7f44e006abe0 msgr2=0x7f44e006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.931+0000 7f44e685d640 1 -- 172.21.15.19:0/4082688732 shutdown_connections 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.931+0000 7f44e685d640 1 -- 172.21.15.19:0/4082688732 wait complete. 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.932+0000 7f44e685d640 1 Processor -- start 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.932+0000 7f44e685d640 1 -- start start 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44e685d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e00ce3c0 0x7f44e00cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44e685d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44e00cfcf0 0x7f44e0146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44e685d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e0154ee0 0x7f44e0146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44e685d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e0071d20 con 0x7f44e00ce3c0 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44e685d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e0071a40 con 0x7f44e0154ee0 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44e685d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e01552c0 con 0x7f44e00cfcf0 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e00ce3c0 0x7f44e00cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e00ce3c0 0x7f44e00cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56452/0 (socket says 172.21.15.19:56452) 2024-04-25T10:01:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44dffff640 1 -- 172.21.15.19:0/1594442998 learned_addr learned my addr 172.21.15.19:0/1594442998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44e4dfa640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e0154ee0 0x7f44e0146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44df7fe640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44e00cfcf0 0x7f44e0146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44dffff640 1 -- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44e00cfcf0 msgr2=0x7f44e0146000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44dffff640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44e00cfcf0 0x7f44e0146000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44dffff640 1 -- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e0154ee0 msgr2=0x7f44e0146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44dffff640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e0154ee0 0x7f44e0146540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44dffff640 1 -- 172.21.15.19:0/1594442998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e00aaee0 con 0x7f44e00ce3c0 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.933+0000 7f44e4dfa640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e0154ee0 0x7f44e0146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.934+0000 7f44dffff640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e00ce3c0 0x7f44e00cc970 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f44d400d5f0 tx=0x7f44d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.934+0000 7f44dcff9640 1 -- 172.21.15.19:0/1594442998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40125f0 con 0x7f44e00ce3c0 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.934+0000 7f44dcff9640 1 -- 172.21.15.19:0/1594442998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d4019070 con 0x7f44e00ce3c0 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.934+0000 7f44e685d640 1 -- 172.21.15.19:0/1594442998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e0144030 con 0x7f44e00ce3c0 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.934+0000 7f44dcff9640 1 -- 172.21.15.19:0/1594442998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40129d0 con 0x7f44e00ce3c0 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.934+0000 7f44e685d640 1 -- 172.21.15.19:0/1594442998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e01445f0 con 0x7f44e00ce3c0 2024-04-25T10:01:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.935+0000 7f44e685d640 1 -- 172.21.15.19:0/1594442998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e0073990 con 0x7f44e00ce3c0 2024-04-25T10:01:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.935+0000 7f44dcff9640 1 -- 172.21.15.19:0/1594442998 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44d4012b70 con 0x7f44e00ce3c0 2024-04-25T10:01:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.935+0000 7f44dcff9640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f44cc06b4d0 0x7f44cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.936+0000 7f44dcff9640 1 -- 172.21.15.19:0/1594442998 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f44d408a880 con 0x7f44e00ce3c0 2024-04-25T10:01:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.936+0000 7f44df7fe640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f44cc06b4d0 0x7f44cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.936+0000 7f44df7fe640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f44cc06b4d0 0x7f44cc06d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f44d0034000 tx=0x7f44d002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:39.940+0000 7f44dcff9640 1 -- 172.21.15.19:0/1594442998 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44d4059440 con 0x7f44e00ce3c0 2024-04-25T10:01:40.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.156+0000 7f44e685d640 1 -- 172.21.15.19:0/1594442998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44e006d360 con 0x7f44e00ce3c0 2024-04-25T10:01:40.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.157+0000 7f44dcff9640 1 -- 172.21.15.19:0/1594442998 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f44d4058d40 con 0x7f44e00ce3c0 2024-04-25T10:01:40.161 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:40.161 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:01:30.999245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/250977615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":250977615},{"type":"v1","addr":"172.21.15.19:6835","nonce":250977615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:01:40.162 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 79 2024-04-25T10:01:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 -- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f44cc06b4d0 msgr2=0x7f44cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f44cc06b4d0 0x7f44cc06d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f44d0034000 tx=0x7f44d002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 -- 172.21.15.19:0/1594442998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e00ce3c0 msgr2=0x7f44e00cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e00ce3c0 0x7f44e00cc970 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f44d400d5f0 tx=0x7f44d400dab0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 -- 172.21.15.19:0/1594442998 shutdown_connections 2024-04-25T10:01:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f44cc06b4d0 0x7f44cc06d990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e0154ee0 0x7f44e0146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44e00cfcf0 0x7f44e0146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 --2- 172.21.15.19:0/1594442998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e00ce3c0 0x7f44e00cc970 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 -- 172.21.15.19:0/1594442998 >> 172.21.15.19:0/1594442998 conn(0x7f44e006abe0 msgr2=0x7f44e015c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 -- 172.21.15.19:0/1594442998 shutdown_connections 2024-04-25T10:01:40.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.159+0000 7f44e685d640 1 -- 172.21.15.19:0/1594442998 wait complete. 2024-04-25T10:01:40.174 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:40.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.271+0000 7f2af584a640 1 Processor -- start 2024-04-25T10:01:40.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.271+0000 7f2af584a640 1 -- start start 2024-04-25T10:01:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.271+0000 7f2af584a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2af015dd30 0x7f2af00b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.272+0000 7f2aee7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2af00b9770 0x7f2af00d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56558/0 (socket says 172.21.15.19:56558) 2024-04-25T10:01:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.272+0000 7f2aeeffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2af015dd30 0x7f2af00b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.272+0000 7f2af584a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af00d1df0 con 0x7f2af00b9770 2024-04-25T10:01:40.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.272+0000 7f2aee7fc640 1 -- 172.21.15.19:0/4013595126 learned_addr learned my addr 172.21.15.19:0/4013595126 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:40.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.272+0000 7f2af584a640 1 -- 172.21.15.19:0/4013595126 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af00d20d0 con 0x7f2af00d0ad0 2024-04-25T10:01:40.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.272+0000 7f2af584a640 1 -- 172.21.15.19:0/4013595126 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af00d23b0 con 0x7f2af015dd30 2024-04-25T10:01:40.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.272+0000 7f2aeeffd640 1 -- 172.21.15.19:0/4013595126 >> v1:172.21.15.112:6789/0 conn(0x7f2af00d0ad0 legacy=0x7f2af00b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:40.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.272+0000 7f2aeeffd640 1 -- 172.21.15.19:0/4013595126 >> v1:172.21.15.19:6789/0 conn(0x7f2af00b9770 legacy=0x7f2af00d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.272+0000 7f2aedffb640 1 -- 172.21.15.19:0/4013595126 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338457555 0 0) 0x7f2af00d1df0 con 0x7f2af00b9770 2024-04-25T10:01:40.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.272+0000 7f2aeeffd640 1 -- 172.21.15.19:0/4013595126 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00d2690 con 0x7f2af015dd30 2024-04-25T10:01:40.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.273+0000 7f2aeeffd640 1 --2- 172.21.15.19:0/4013595126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af015dd30 0x7f2af00b9230 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80097a0 tx=0x7f2ae802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56607abd56b3d27a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.273+0000 7f2aedffb640 1 -- 172.21.15.19:0/4013595126 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8037070 con 0x7f2af015dd30 2024-04-25T10:01:40.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.273+0000 7f2aedffb640 1 -- 172.21.15.19:0/4013595126 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae802adc0 con 0x7f2af015dd30 2024-04-25T10:01:40.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.273+0000 7f2aedffb640 1 -- 172.21.15.19:0/4013595126 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8032660 con 0x7f2af015dd30 2024-04-25T10:01:40.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.273+0000 7f2af584a640 1 -- 172.21.15.19:0/4013595126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af015dd30 msgr2=0x7f2af00b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.273+0000 7f2af584a640 1 --2- 172.21.15.19:0/4013595126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af015dd30 0x7f2af00b9230 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80097a0 tx=0x7f2ae802a0b0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.273+0000 7f2af584a640 1 -- 172.21.15.19:0/4013595126 shutdown_connections 2024-04-25T10:01:40.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.273+0000 7f2af584a640 1 --2- 172.21.15.19:0/4013595126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af015dd30 0x7f2af00b9230 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.273+0000 7f2af584a640 1 -- 172.21.15.19:0/4013595126 >> 172.21.15.19:0/4013595126 conn(0x7f2af0061c30 msgr2=0x7f2af006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.273+0000 7f2af584a640 1 -- 172.21.15.19:0/4013595126 shutdown_connections 2024-04-25T10:01:40.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.273+0000 7f2af584a640 1 -- 172.21.15.19:0/4013595126 wait complete. 2024-04-25T10:01:40.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.275+0000 7f2af584a640 1 Processor -- start 2024-04-25T10:01:40.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.275+0000 7f2af584a640 1 -- start start 2024-04-25T10:01:40.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.275+0000 7f2af584a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af00b9770 0x7f2af01f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2af584a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af00d0ad0 0x7f2af00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2af584a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2af015dd30 0x7f2af00b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2af584a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00d1b90 con 0x7f2af00d0ad0 2024-04-25T10:01:40.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2af584a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00d1a10 con 0x7f2af00b9770 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2af584a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00d2e90 con 0x7f2af015dd30 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aee7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af00d0ad0 0x7f2af00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aee7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af00d0ad0 0x7f2af00b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56470/0 (socket says 172.21.15.19:56470) 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aee7fc640 1 -- 172.21.15.19:0/3523154323 learned_addr learned my addr 172.21.15.19:0/3523154323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aeeffd640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af00b9770 0x7f2af01f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aef7fe640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2af015dd30 0x7f2af00b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aee7fc640 1 -- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2af015dd30 msgr2=0x7f2af00b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aee7fc640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2af015dd30 0x7f2af00b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aee7fc640 1 -- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af00b9770 msgr2=0x7f2af01f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aee7fc640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af00b9770 0x7f2af01f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aee7fc640 1 -- 172.21.15.19:0/3523154323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af015f810 con 0x7f2af00d0ad0 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aeeffd640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af00b9770 0x7f2af01f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aef7fe640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2af015dd30 0x7f2af00b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2aee7fc640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af00d0ad0 0x7f2af00b5ab0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800b9d0 tx=0x7f2ad800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2ae7fff640 1 -- 172.21.15.19:0/3523154323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad800cda0 con 0x7f2af00d0ad0 2024-04-25T10:01:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2ae7fff640 1 -- 172.21.15.19:0/3523154323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad8010070 con 0x7f2af00d0ad0 2024-04-25T10:01:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2ae7fff640 1 -- 172.21.15.19:0/3523154323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad80156b0 con 0x7f2af00d0ad0 2024-04-25T10:01:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2af584a640 1 -- 172.21.15.19:0/3523154323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af00b5ff0 con 0x7f2af00d0ad0 2024-04-25T10:01:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.276+0000 7f2af584a640 1 -- 172.21.15.19:0/3523154323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af015fab0 con 0x7f2af00d0ad0 2024-04-25T10:01:40.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.278+0000 7f2af584a640 1 -- 172.21.15.19:0/3523154323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af0073990 con 0x7f2af00d0ad0 2024-04-25T10:01:40.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.278+0000 7f2ae7fff640 1 -- 172.21.15.19:0/3523154323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ad801e3f0 con 0x7f2af00d0ad0 2024-04-25T10:01:40.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.278+0000 7f2ae7fff640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ad006b4d0 0x7f2ad006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.278+0000 7f2ae7fff640 1 -- 172.21.15.19:0/3523154323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ad808a530 con 0x7f2af00d0ad0 2024-04-25T10:01:40.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.279+0000 7f2aeeffd640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ad006b4d0 0x7f2ad006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.279+0000 7f2aeeffd640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ad006b4d0 0x7f2ad006d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8034000 tx=0x7f2ae802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.283+0000 7f2ae7fff640 1 -- 172.21.15.19:0/3523154323 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ad80592b0 con 0x7f2af00d0ad0 2024-04-25T10:01:40.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.503+0000 7f2af584a640 1 -- 172.21.15.19:0/3523154323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2af00c9f60 con 0x7f2af00d0ad0 2024-04-25T10:01:40.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.503+0000 7f2ae7fff640 1 -- 172.21.15.19:0/3523154323 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2ad8058bb0 con 0x7f2af00d0ad0 2024-04-25T10:01:40.508 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:40.508 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:01:30.999245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/250977615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":250977615},{"type":"v1","addr":"172.21.15.19:6835","nonce":250977615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:01:40.509 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 79 2024-04-25T10:01:40.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.505+0000 7f2af584a640 1 -- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ad006b4d0 msgr2=0x7f2ad006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.505+0000 7f2af584a640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ad006b4d0 0x7f2ad006d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8034000 tx=0x7f2ae802e000 comp rx=0 tx=0).stop 2024-04-25T10:01:40.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.505+0000 7f2af584a640 1 -- 172.21.15.19:0/3523154323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af00d0ad0 msgr2=0x7f2af00b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.505+0000 7f2af584a640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af00d0ad0 0x7f2af00b5ab0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800b9d0 tx=0x7f2ad800be90 comp rx=0 tx=0).stop 2024-04-25T10:01:40.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.505+0000 7f2af584a640 1 -- 172.21.15.19:0/3523154323 shutdown_connections 2024-04-25T10:01:40.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.505+0000 7f2af584a640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ad006b4d0 0x7f2ad006d990 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.505+0000 7f2af584a640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2af015dd30 0x7f2af00b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.505+0000 7f2af584a640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af00d0ad0 0x7f2af00b5ab0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.505+0000 7f2af584a640 1 --2- 172.21.15.19:0/3523154323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2af00b9770 0x7f2af01f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.505+0000 7f2af584a640 1 -- 172.21.15.19:0/3523154323 >> 172.21.15.19:0/3523154323 conn(0x7f2af0061c30 msgr2=0x7f2af00c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.506+0000 7f2af584a640 1 -- 172.21.15.19:0/3523154323 shutdown_connections 2024-04-25T10:01:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:40.506+0000 7f2af584a640 1 -- 172.21.15.19:0/3523154323 wait complete. 2024-04-25T10:01:40.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:40.521 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:40.521 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:40.521 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:01:40.521 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:40.521 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:40.521 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:40.521 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:40.555 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.555 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.555 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978668 s, 107 MB/s 2024-04-25T10:01:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826421 s, 127 MB/s 2024-04-25T10:01:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698201 s, 150 MB/s 2024-04-25T10:01:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850097 s, 123 MB/s 2024-04-25T10:01:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769977 s, 136 MB/s 2024-04-25T10:01:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866101 s, 121 MB/s 2024-04-25T10:01:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748264 s, 140 MB/s 2024-04-25T10:01:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643252 s, 163 MB/s 2024-04-25T10:01:40.623 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.623 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.623 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068023 s, 154 MB/s 2024-04-25T10:01:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.633 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854089 s, 123 MB/s 2024-04-25T10:01:40.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.641 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716245 s, 146 MB/s 2024-04-25T10:01:40.649 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.649 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.649 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727106 s, 144 MB/s 2024-04-25T10:01:40.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.659 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881514 s, 119 MB/s 2024-04-25T10:01:40.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684913 s, 153 MB/s 2024-04-25T10:01:40.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.675 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750781 s, 140 MB/s 2024-04-25T10:01:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.684 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793543 s, 132 MB/s 2024-04-25T10:01:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796883 s, 132 MB/s 2024-04-25T10:01:40.700 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.700 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.700 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655534 s, 160 MB/s 2024-04-25T10:01:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750213 s, 140 MB/s 2024-04-25T10:01:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074577 s, 141 MB/s 2024-04-25T10:01:40.725 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.725 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.725 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716367 s, 146 MB/s 2024-04-25T10:01:40.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.736 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.736 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981591 s, 107 MB/s 2024-04-25T10:01:40.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.743 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647855 s, 162 MB/s 2024-04-25T10:01:40.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.751 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753956 s, 139 MB/s 2024-04-25T10:01:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.759 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624378 s, 168 MB/s 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689947 s, 152 MB/s 2024-04-25T10:01:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711668 s, 147 MB/s 2024-04-25T10:01:40.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.783 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771178 s, 136 MB/s 2024-04-25T10:01:40.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.791 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720692 s, 145 MB/s 2024-04-25T10:01:40.799 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.799 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.799 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712274 s, 147 MB/s 2024-04-25T10:01:40.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.807 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643412 s, 163 MB/s 2024-04-25T10:01:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736274 s, 142 MB/s 2024-04-25T10:01:40.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:40.816 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:40.816 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:40.816 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:01:40.816 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:40.816 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:40.816 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:40.816 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:40.881 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.881 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.881 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708835 s, 148 MB/s 2024-04-25T10:01:40.889 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.889 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.889 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738478 s, 142 MB/s 2024-04-25T10:01:40.897 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.897 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.897 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680038 s, 154 MB/s 2024-04-25T10:01:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.905 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672469 s, 156 MB/s 2024-04-25T10:01:40.912 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.912 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.912 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067979 s, 154 MB/s 2024-04-25T10:01:40.920 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.920 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.920 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695953 s, 151 MB/s 2024-04-25T10:01:40.929 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.929 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.929 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769227 s, 136 MB/s 2024-04-25T10:01:40.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.938 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.938 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759567 s, 138 MB/s 2024-04-25T10:01:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794388 s, 132 MB/s 2024-04-25T10:01:40.954 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.954 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.954 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721428 s, 145 MB/s 2024-04-25T10:01:40.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.965 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963163 s, 109 MB/s 2024-04-25T10:01:40.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.973 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731884 s, 143 MB/s 2024-04-25T10:01:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851801 s, 123 MB/s 2024-04-25T10:01:40.990 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.990 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.990 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634182 s, 165 MB/s 2024-04-25T10:01:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702539 s, 149 MB/s 2024-04-25T10:01:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068559 s, 153 MB/s 2024-04-25T10:01:41.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.016 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985441 s, 106 MB/s 2024-04-25T10:01:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725105 s, 145 MB/s 2024-04-25T10:01:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624694 s, 168 MB/s 2024-04-25T10:01:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712399 s, 147 MB/s 2024-04-25T10:01:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726971 s, 144 MB/s 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729859 s, 144 MB/s 2024-04-25T10:01:41.064 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.064 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.064 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075366 s, 139 MB/s 2024-04-25T10:01:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970425 s, 108 MB/s 2024-04-25T10:01:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631631 s, 166 MB/s 2024-04-25T10:01:41.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.090 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066517 s, 158 MB/s 2024-04-25T10:01:41.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.097 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698521 s, 150 MB/s 2024-04-25T10:01:41.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.105 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702212 s, 149 MB/s 2024-04-25T10:01:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708518 s, 148 MB/s 2024-04-25T10:01:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723436 s, 145 MB/s 2024-04-25T10:01:41.132 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.132 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.132 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979212 s, 107 MB/s 2024-04-25T10:01:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693332 s, 151 MB/s 2024-04-25T10:01:41.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:41.141 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:41.141 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:41.141 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:01:41.141 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:41.141 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:41.142 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:41.142 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:41.206 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.206 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.206 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065208 s, 161 MB/s 2024-04-25T10:01:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736748 s, 142 MB/s 2024-04-25T10:01:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072827 s, 144 MB/s 2024-04-25T10:01:41.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.230 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674186 s, 156 MB/s 2024-04-25T10:01:41.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.237 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666669 s, 157 MB/s 2024-04-25T10:01:41.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.245 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691313 s, 152 MB/s 2024-04-25T10:01:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007084 s, 148 MB/s 2024-04-25T10:01:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699213 s, 150 MB/s 2024-04-25T10:01:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644805 s, 163 MB/s 2024-04-25T10:01:41.278 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.278 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.278 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854995 s, 123 MB/s 2024-04-25T10:01:41.285 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.285 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.285 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615712 s, 170 MB/s 2024-04-25T10:01:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676134 s, 155 MB/s 2024-04-25T10:01:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703572 s, 149 MB/s 2024-04-25T10:01:41.308 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.308 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.308 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684611 s, 153 MB/s 2024-04-25T10:01:41.316 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.316 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.316 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733181 s, 143 MB/s 2024-04-25T10:01:41.324 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.324 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.324 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069543 s, 151 MB/s 2024-04-25T10:01:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906018 s, 116 MB/s 2024-04-25T10:01:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701243 s, 150 MB/s 2024-04-25T10:01:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709128 s, 148 MB/s 2024-04-25T10:01:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626508 s, 167 MB/s 2024-04-25T10:01:41.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.367 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904175 s, 116 MB/s 2024-04-25T10:01:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698252 s, 150 MB/s 2024-04-25T10:01:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.383 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689257 s, 152 MB/s 2024-04-25T10:01:41.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.390 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647255 s, 162 MB/s 2024-04-25T10:01:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655773 s, 160 MB/s 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723502 s, 145 MB/s 2024-04-25T10:01:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862799 s, 122 MB/s 2024-04-25T10:01:41.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.423 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649951 s, 161 MB/s 2024-04-25T10:01:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.430 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604567 s, 173 MB/s 2024-04-25T10:01:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669127 s, 157 MB/s 2024-04-25T10:01:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688515 s, 152 MB/s 2024-04-25T10:01:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706173 s, 148 MB/s 2024-04-25T10:01:41.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:41.454 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:41.454 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:41.454 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:01:41.454 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:41.454 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:41.454 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:41.454 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:41.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.519 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650556 s, 161 MB/s 2024-04-25T10:01:41.526 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.526 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.526 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651761 s, 161 MB/s 2024-04-25T10:01:41.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.534 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677784 s, 155 MB/s 2024-04-25T10:01:41.541 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.541 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.541 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641103 s, 164 MB/s 2024-04-25T10:01:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664536 s, 158 MB/s 2024-04-25T10:01:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638488 s, 164 MB/s 2024-04-25T10:01:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067301 s, 156 MB/s 2024-04-25T10:01:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727906 s, 144 MB/s 2024-04-25T10:01:41.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.581 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749018 s, 140 MB/s 2024-04-25T10:01:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703878 s, 149 MB/s 2024-04-25T10:01:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684726 s, 153 MB/s 2024-04-25T10:01:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624771 s, 168 MB/s 2024-04-25T10:01:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720083 s, 146 MB/s 2024-04-25T10:01:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.620 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685623 s, 153 MB/s 2024-04-25T10:01:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720933 s, 145 MB/s 2024-04-25T10:01:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.635 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668012 s, 157 MB/s 2024-04-25T10:01:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693816 s, 151 MB/s 2024-04-25T10:01:41.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683938 s, 153 MB/s 2024-04-25T10:01:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723303 s, 145 MB/s 2024-04-25T10:01:41.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662901 s, 158 MB/s 2024-04-25T10:01:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743998 s, 141 MB/s 2024-04-25T10:01:41.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.682 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648051 s, 162 MB/s 2024-04-25T10:01:41.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.689 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636252 s, 165 MB/s 2024-04-25T10:01:41.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.697 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709206 s, 148 MB/s 2024-04-25T10:01:41.705 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.705 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.705 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690897 s, 152 MB/s 2024-04-25T10:01:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686024 s, 153 MB/s 2024-04-25T10:01:41.720 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673059 s, 156 MB/s 2024-04-25T10:01:41.730 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.730 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.730 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893543 s, 117 MB/s 2024-04-25T10:01:41.738 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.738 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.738 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703427 s, 149 MB/s 2024-04-25T10:01:41.746 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.746 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.746 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701627 s, 149 MB/s 2024-04-25T10:01:41.754 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.754 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.754 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697429 s, 150 MB/s 2024-04-25T10:01:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716254 s, 146 MB/s 2024-04-25T10:01:41.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:41.763 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:41.764 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:41.764 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:01:41.764 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:41.764 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:41.764 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:41.764 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848278 s, 124 MB/s 2024-04-25T10:01:41.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.838 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640108 s, 164 MB/s 2024-04-25T10:01:41.845 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.845 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.845 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620709 s, 169 MB/s 2024-04-25T10:01:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753272 s, 139 MB/s 2024-04-25T10:01:41.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670684 s, 156 MB/s 2024-04-25T10:01:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626831 s, 167 MB/s 2024-04-25T10:01:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747403 s, 140 MB/s 2024-04-25T10:01:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692352 s, 151 MB/s 2024-04-25T10:01:41.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.893 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721798 s, 145 MB/s 2024-04-25T10:01:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690631 s, 152 MB/s 2024-04-25T10:01:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635694 s, 165 MB/s 2024-04-25T10:01:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064567 s, 162 MB/s 2024-04-25T10:01:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668401 s, 157 MB/s 2024-04-25T10:01:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686083 s, 153 MB/s 2024-04-25T10:01:41.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.939 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070706 s, 148 MB/s 2024-04-25T10:01:41.947 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.947 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.947 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690763 s, 152 MB/s 2024-04-25T10:01:41.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.955 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737166 s, 142 MB/s 2024-04-25T10:01:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.962 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629676 s, 167 MB/s 2024-04-25T10:01:41.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.970 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716216 s, 146 MB/s 2024-04-25T10:01:41.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.978 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672568 s, 156 MB/s 2024-04-25T10:01:41.986 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788408 s, 133 MB/s 2024-04-25T10:01:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669998 s, 157 MB/s 2024-04-25T10:01:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749269 s, 140 MB/s 2024-04-25T10:01:42.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.010 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610843 s, 172 MB/s 2024-04-25T10:01:42.021 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877448 s, 120 MB/s 2024-04-25T10:01:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.027 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638866 s, 164 MB/s 2024-04-25T10:01:42.035 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.035 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.035 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710145 s, 148 MB/s 2024-04-25T10:01:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984248 s, 107 MB/s 2024-04-25T10:01:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704147 s, 149 MB/s 2024-04-25T10:01:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703614 s, 149 MB/s 2024-04-25T10:01:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.070 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714808 s, 147 MB/s 2024-04-25T10:01:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.078 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714825 s, 147 MB/s 2024-04-25T10:01:42.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:42.079 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:42.079 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:42.079 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:01:42.079 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:42.079 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:42.079 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:42.079 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:42.144 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.145 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707725 s, 148 MB/s 2024-04-25T10:01:42.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.154 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865621 s, 121 MB/s 2024-04-25T10:01:42.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.162 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705209 s, 149 MB/s 2024-04-25T10:01:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724468 s, 145 MB/s 2024-04-25T10:01:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.178 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653906 s, 160 MB/s 2024-04-25T10:01:42.187 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.187 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.187 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864053 s, 121 MB/s 2024-04-25T10:01:42.194 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.195 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.195 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649543 s, 161 MB/s 2024-04-25T10:01:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808863 s, 130 MB/s 2024-04-25T10:01:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752799 s, 139 MB/s 2024-04-25T10:01:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719426 s, 146 MB/s 2024-04-25T10:01:42.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645899 s, 162 MB/s 2024-04-25T10:01:42.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.237 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828794 s, 127 MB/s 2024-04-25T10:01:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.244 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621493 s, 169 MB/s 2024-04-25T10:01:42.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.252 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722707 s, 145 MB/s 2024-04-25T10:01:42.261 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.261 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.261 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847655 s, 124 MB/s 2024-04-25T10:01:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673137 s, 156 MB/s 2024-04-25T10:01:42.277 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.277 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.277 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728386 s, 144 MB/s 2024-04-25T10:01:42.285 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.285 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.285 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717902 s, 146 MB/s 2024-04-25T10:01:42.293 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.293 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.293 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706641 s, 148 MB/s 2024-04-25T10:01:42.301 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.301 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.301 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694542 s, 151 MB/s 2024-04-25T10:01:42.308 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.309 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.309 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706066 s, 149 MB/s 2024-04-25T10:01:42.316 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.316 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.316 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658407 s, 159 MB/s 2024-04-25T10:01:42.324 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.325 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.325 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074565 s, 141 MB/s 2024-04-25T10:01:42.332 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.332 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.332 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672323 s, 156 MB/s 2024-04-25T10:01:42.340 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.340 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.340 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726156 s, 144 MB/s 2024-04-25T10:01:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697709 s, 150 MB/s 2024-04-25T10:01:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073271 s, 143 MB/s 2024-04-25T10:01:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695799 s, 151 MB/s 2024-04-25T10:01:42.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.372 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707634 s, 148 MB/s 2024-04-25T10:01:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704931 s, 149 MB/s 2024-04-25T10:01:42.388 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.388 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.388 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067027 s, 156 MB/s 2024-04-25T10:01:42.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.396 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708343 s, 148 MB/s 2024-04-25T10:01:42.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:42.397 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:42.397 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:42.397 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:01:42.397 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:42.397 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:42.398 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:42.398 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:42.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687456 s, 153 MB/s 2024-04-25T10:01:42.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.470 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680432 s, 154 MB/s 2024-04-25T10:01:42.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.477 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641348 s, 163 MB/s 2024-04-25T10:01:42.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.484 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659018 s, 159 MB/s 2024-04-25T10:01:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.492 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.492 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652803 s, 161 MB/s 2024-04-25T10:01:42.502 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.502 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.502 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860734 s, 122 MB/s 2024-04-25T10:01:42.509 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.509 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.509 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680381 s, 154 MB/s 2024-04-25T10:01:42.516 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.517 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687161 s, 153 MB/s 2024-04-25T10:01:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726761 s, 144 MB/s 2024-04-25T10:01:42.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.532 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615586 s, 170 MB/s 2024-04-25T10:01:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080106 s, 131 MB/s 2024-04-25T10:01:42.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.549 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735586 s, 143 MB/s 2024-04-25T10:01:42.557 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.557 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.557 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672404 s, 156 MB/s 2024-04-25T10:01:42.563 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.563 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621344 s, 169 MB/s 2024-04-25T10:01:42.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.573 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086204 s, 122 MB/s 2024-04-25T10:01:42.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.581 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706469 s, 148 MB/s 2024-04-25T10:01:42.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.589 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656111 s, 160 MB/s 2024-04-25T10:01:42.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698575 s, 150 MB/s 2024-04-25T10:01:42.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.606 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662019 s, 158 MB/s 2024-04-25T10:01:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736034 s, 142 MB/s 2024-04-25T10:01:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771314 s, 136 MB/s 2024-04-25T10:01:42.629 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.629 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.629 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696284 s, 151 MB/s 2024-04-25T10:01:42.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.637 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744312 s, 141 MB/s 2024-04-25T10:01:42.644 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.644 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.644 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599227 s, 175 MB/s 2024-04-25T10:01:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.652 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754251 s, 139 MB/s 2024-04-25T10:01:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664241 s, 158 MB/s 2024-04-25T10:01:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712048 s, 147 MB/s 2024-04-25T10:01:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927289 s, 113 MB/s 2024-04-25T10:01:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.687 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824666 s, 127 MB/s 2024-04-25T10:01:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766421 s, 137 MB/s 2024-04-25T10:01:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.705 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751744 s, 139 MB/s 2024-04-25T10:01:42.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.712 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700587 s, 150 MB/s 2024-04-25T10:01:42.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:42.713 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:42.713 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:42.713 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:01:42.714 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:42.714 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:42.714 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:42.714 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:42.777 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.778 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629082 s, 167 MB/s 2024-04-25T10:01:42.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.786 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727256 s, 144 MB/s 2024-04-25T10:01:42.793 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.793 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.793 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065575 s, 160 MB/s 2024-04-25T10:01:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673125 s, 156 MB/s 2024-04-25T10:01:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643819 s, 163 MB/s 2024-04-25T10:01:42.816 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.816 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.816 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716993 s, 146 MB/s 2024-04-25T10:01:42.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.823 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613534 s, 171 MB/s 2024-04-25T10:01:42.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.831 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676258 s, 155 MB/s 2024-04-25T10:01:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746487 s, 140 MB/s 2024-04-25T10:01:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676328 s, 155 MB/s 2024-04-25T10:01:42.855 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.855 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.855 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697511 s, 150 MB/s 2024-04-25T10:01:42.862 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.863 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.863 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647006 s, 162 MB/s 2024-04-25T10:01:42.870 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.870 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.870 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694702 s, 151 MB/s 2024-04-25T10:01:42.880 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.880 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.880 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889712 s, 118 MB/s 2024-04-25T10:01:42.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.887 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649177 s, 162 MB/s 2024-04-25T10:01:42.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.895 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681258 s, 154 MB/s 2024-04-25T10:01:42.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.902 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637851 s, 164 MB/s 2024-04-25T10:01:42.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.910 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645967 s, 162 MB/s 2024-04-25T10:01:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069933 s, 150 MB/s 2024-04-25T10:01:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.925 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619381 s, 169 MB/s 2024-04-25T10:01:42.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.933 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706773 s, 148 MB/s 2024-04-25T10:01:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676179 s, 155 MB/s 2024-04-25T10:01:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622085 s, 169 MB/s 2024-04-25T10:01:42.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.956 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690617 s, 152 MB/s 2024-04-25T10:01:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696173 s, 151 MB/s 2024-04-25T10:01:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.971 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698429 s, 150 MB/s 2024-04-25T10:01:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.979 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703799 s, 149 MB/s 2024-04-25T10:01:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676687 s, 155 MB/s 2024-04-25T10:01:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:42.994 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623983 s, 168 MB/s 2024-04-25T10:01:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863234 s, 121 MB/s 2024-04-25T10:01:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648117 s, 162 MB/s 2024-04-25T10:01:43.019 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.019 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.019 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701871 s, 149 MB/s 2024-04-25T10:01:43.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:43.020 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:43.020 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:43.020 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:01:43.020 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:43.021 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:43.021 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:43.021 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.086 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071085 s, 148 MB/s 2024-04-25T10:01:43.093 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.093 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.093 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694429 s, 151 MB/s 2024-04-25T10:01:43.101 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.101 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.101 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675409 s, 155 MB/s 2024-04-25T10:01:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085986 s, 122 MB/s 2024-04-25T10:01:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.118 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704777 s, 149 MB/s 2024-04-25T10:01:43.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.126 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.126 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698042 s, 150 MB/s 2024-04-25T10:01:43.134 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.134 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.135 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734401 s, 143 MB/s 2024-04-25T10:01:43.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.142 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655502 s, 160 MB/s 2024-04-25T10:01:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735397 s, 143 MB/s 2024-04-25T10:01:43.158 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.159 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.159 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719607 s, 146 MB/s 2024-04-25T10:01:43.166 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.166 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.166 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694942 s, 151 MB/s 2024-04-25T10:01:43.175 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.175 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.175 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721754 s, 145 MB/s 2024-04-25T10:01:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.183 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689483 s, 152 MB/s 2024-04-25T10:01:43.190 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.190 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.190 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681849 s, 154 MB/s 2024-04-25T10:01:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662788 s, 158 MB/s 2024-04-25T10:01:43.205 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.206 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.206 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660792 s, 159 MB/s 2024-04-25T10:01:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684167 s, 153 MB/s 2024-04-25T10:01:43.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.221 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653642 s, 160 MB/s 2024-04-25T10:01:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072242 s, 145 MB/s 2024-04-25T10:01:43.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.237 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661881 s, 158 MB/s 2024-04-25T10:01:43.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.245 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714475 s, 147 MB/s 2024-04-25T10:01:43.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.252 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643011 s, 163 MB/s 2024-04-25T10:01:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625213 s, 168 MB/s 2024-04-25T10:01:43.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.266 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644725 s, 163 MB/s 2024-04-25T10:01:43.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.275 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739932 s, 142 MB/s 2024-04-25T10:01:43.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.283 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730646 s, 144 MB/s 2024-04-25T10:01:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771305 s, 136 MB/s 2024-04-25T10:01:43.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.299 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670446 s, 156 MB/s 2024-04-25T10:01:43.306 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.306 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.306 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062019 s, 169 MB/s 2024-04-25T10:01:43.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.314 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697647 s, 150 MB/s 2024-04-25T10:01:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732216 s, 143 MB/s 2024-04-25T10:01:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722492 s, 145 MB/s 2024-04-25T10:01:43.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:43.332 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:43.332 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:43.332 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:01:43.332 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:43.332 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:43.332 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:43.332 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:43.403 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.404 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.404 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133595 s, 78.5 MB/s 2024-04-25T10:01:43.412 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.413 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848213 s, 124 MB/s 2024-04-25T10:01:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694461 s, 151 MB/s 2024-04-25T10:01:43.428 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.428 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.429 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717209 s, 146 MB/s 2024-04-25T10:01:43.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.437 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722734 s, 145 MB/s 2024-04-25T10:01:43.444 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.444 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660498 s, 159 MB/s 2024-04-25T10:01:43.452 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.452 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.452 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070134 s, 150 MB/s 2024-04-25T10:01:43.460 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.460 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.460 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696786 s, 150 MB/s 2024-04-25T10:01:43.467 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.467 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.467 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659863 s, 159 MB/s 2024-04-25T10:01:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687278 s, 153 MB/s 2024-04-25T10:01:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068276 s, 154 MB/s 2024-04-25T10:01:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.491 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065067 s, 161 MB/s 2024-04-25T10:01:43.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.497 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006128 s, 171 MB/s 2024-04-25T10:01:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646915 s, 162 MB/s 2024-04-25T10:01:43.512 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.513 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684565 s, 153 MB/s 2024-04-25T10:01:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624946 s, 168 MB/s 2024-04-25T10:01:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.528 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705288 s, 149 MB/s 2024-04-25T10:01:43.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.536 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701414 s, 149 MB/s 2024-04-25T10:01:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688559 s, 152 MB/s 2024-04-25T10:01:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651768 s, 161 MB/s 2024-04-25T10:01:43.560 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.560 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.560 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817735 s, 128 MB/s 2024-04-25T10:01:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856015 s, 122 MB/s 2024-04-25T10:01:43.578 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.578 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.578 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811525 s, 129 MB/s 2024-04-25T10:01:43.586 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.586 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.586 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708134 s, 148 MB/s 2024-04-25T10:01:43.594 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.594 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.594 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686839 s, 153 MB/s 2024-04-25T10:01:43.602 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.602 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.602 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688977 s, 152 MB/s 2024-04-25T10:01:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709789 s, 148 MB/s 2024-04-25T10:01:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644751 s, 163 MB/s 2024-04-25T10:01:43.624 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.624 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.624 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633105 s, 166 MB/s 2024-04-25T10:01:43.632 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.632 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.632 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070569 s, 149 MB/s 2024-04-25T10:01:43.639 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.639 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.640 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627682 s, 167 MB/s 2024-04-25T10:01:43.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.648 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768029 s, 137 MB/s 2024-04-25T10:01:43.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:43.649 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:43.649 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:43.649 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:01:43.649 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:43.650 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:43.650 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:43.650 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:43.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.714 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634533 s, 165 MB/s 2024-04-25T10:01:43.722 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.722 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.722 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736274 s, 142 MB/s 2024-04-25T10:01:43.730 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.730 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.730 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067386 s, 156 MB/s 2024-04-25T10:01:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.738 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707618 s, 148 MB/s 2024-04-25T10:01:43.745 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.745 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.745 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699655 s, 150 MB/s 2024-04-25T10:01:43.753 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.753 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.753 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682272 s, 154 MB/s 2024-04-25T10:01:43.760 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.760 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.760 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064155 s, 163 MB/s 2024-04-25T10:01:43.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.768 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685457 s, 153 MB/s 2024-04-25T10:01:43.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.776 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706622 s, 148 MB/s 2024-04-25T10:01:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926285 s, 113 MB/s 2024-04-25T10:01:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066265 s, 158 MB/s 2024-04-25T10:01:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694307 s, 151 MB/s 2024-04-25T10:01:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688785 s, 152 MB/s 2024-04-25T10:01:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085918 s, 122 MB/s 2024-04-25T10:01:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624242 s, 168 MB/s 2024-04-25T10:01:43.836 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.836 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.836 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867245 s, 121 MB/s 2024-04-25T10:01:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688801 s, 152 MB/s 2024-04-25T10:01:43.851 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.851 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.851 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649436 s, 161 MB/s 2024-04-25T10:01:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602319 s, 174 MB/s 2024-04-25T10:01:43.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.865 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657458 s, 159 MB/s 2024-04-25T10:01:43.874 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.874 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.874 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722352 s, 145 MB/s 2024-04-25T10:01:43.881 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.881 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.881 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654014 s, 160 MB/s 2024-04-25T10:01:43.889 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.890 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748362 s, 140 MB/s 2024-04-25T10:01:43.897 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.897 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.897 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660213 s, 159 MB/s 2024-04-25T10:01:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645885 s, 162 MB/s 2024-04-25T10:01:43.912 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.912 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.912 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721236 s, 145 MB/s 2024-04-25T10:01:43.920 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.920 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.921 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695775 s, 151 MB/s 2024-04-25T10:01:43.928 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.928 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.928 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070874 s, 148 MB/s 2024-04-25T10:01:43.935 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.936 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.936 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650517 s, 161 MB/s 2024-04-25T10:01:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684376 s, 153 MB/s 2024-04-25T10:01:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656693 s, 160 MB/s 2024-04-25T10:01:43.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:43.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:43.959 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722783 s, 145 MB/s 2024-04-25T10:01:43.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:43.960 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:43.960 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:43.960 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:01:43.960 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:43.960 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:43.960 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:43.960 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684515 s, 153 MB/s 2024-04-25T10:01:44.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.033 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663695 s, 158 MB/s 2024-04-25T10:01:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.041 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659564 s, 159 MB/s 2024-04-25T10:01:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.048 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633532 s, 166 MB/s 2024-04-25T10:01:44.055 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.056 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.056 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628076 s, 167 MB/s 2024-04-25T10:01:44.062 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.063 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.063 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643579 s, 163 MB/s 2024-04-25T10:01:44.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.070 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654726 s, 160 MB/s 2024-04-25T10:01:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715625 s, 147 MB/s 2024-04-25T10:01:44.085 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.085 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.085 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654787 s, 160 MB/s 2024-04-25T10:01:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689574 s, 152 MB/s 2024-04-25T10:01:44.101 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.101 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.101 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672102 s, 156 MB/s 2024-04-25T10:01:44.108 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.108 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.108 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650641 s, 161 MB/s 2024-04-25T10:01:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729244 s, 144 MB/s 2024-04-25T10:01:44.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.123 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612124 s, 171 MB/s 2024-04-25T10:01:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652028 s, 161 MB/s 2024-04-25T10:01:44.138 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.138 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.138 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654762 s, 160 MB/s 2024-04-25T10:01:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616718 s, 170 MB/s 2024-04-25T10:01:44.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.153 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696237 s, 151 MB/s 2024-04-25T10:01:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723435 s, 145 MB/s 2024-04-25T10:01:44.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691959 s, 152 MB/s 2024-04-25T10:01:44.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.177 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690678 s, 152 MB/s 2024-04-25T10:01:44.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696808 s, 150 MB/s 2024-04-25T10:01:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718241 s, 146 MB/s 2024-04-25T10:01:44.200 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.200 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.200 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665022 s, 158 MB/s 2024-04-25T10:01:44.208 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.208 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.208 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692499 s, 151 MB/s 2024-04-25T10:01:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713167 s, 147 MB/s 2024-04-25T10:01:44.223 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.223 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.223 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632942 s, 166 MB/s 2024-04-25T10:01:44.231 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.231 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.231 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677689 s, 155 MB/s 2024-04-25T10:01:44.238 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.238 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657623 s, 159 MB/s 2024-04-25T10:01:44.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.245 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598422 s, 175 MB/s 2024-04-25T10:01:44.253 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.253 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.253 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685696 s, 153 MB/s 2024-04-25T10:01:44.261 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.261 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.261 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717206 s, 146 MB/s 2024-04-25T10:01:44.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:44.262 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:44.262 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:44.262 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:01:44.263 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:44.263 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:44.263 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:44.263 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:44.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.327 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623598 s, 168 MB/s 2024-04-25T10:01:44.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.335 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698331 s, 150 MB/s 2024-04-25T10:01:44.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.343 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071706 s, 146 MB/s 2024-04-25T10:01:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.351 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696317 s, 151 MB/s 2024-04-25T10:01:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.358 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688109 s, 152 MB/s 2024-04-25T10:01:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.367 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716554 s, 146 MB/s 2024-04-25T10:01:44.374 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.374 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.374 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649035 s, 162 MB/s 2024-04-25T10:01:44.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.382 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704209 s, 149 MB/s 2024-04-25T10:01:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679082 s, 154 MB/s 2024-04-25T10:01:44.397 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.397 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.397 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066191 s, 158 MB/s 2024-04-25T10:01:44.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.405 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713521 s, 147 MB/s 2024-04-25T10:01:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635518 s, 165 MB/s 2024-04-25T10:01:44.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.421 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.421 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739657 s, 142 MB/s 2024-04-25T10:01:44.428 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.429 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071807 s, 146 MB/s 2024-04-25T10:01:44.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.437 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737082 s, 142 MB/s 2024-04-25T10:01:44.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.445 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721109 s, 145 MB/s 2024-04-25T10:01:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686692 s, 153 MB/s 2024-04-25T10:01:44.463 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.463 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.463 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933836 s, 112 MB/s 2024-04-25T10:01:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703454 s, 149 MB/s 2024-04-25T10:01:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.479 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653383 s, 160 MB/s 2024-04-25T10:01:44.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.486 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695628 s, 151 MB/s 2024-04-25T10:01:44.494 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.494 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.494 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688758 s, 152 MB/s 2024-04-25T10:01:44.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.501 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628573 s, 167 MB/s 2024-04-25T10:01:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067705 s, 155 MB/s 2024-04-25T10:01:44.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.517 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071455 s, 147 MB/s 2024-04-25T10:01:44.525 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.525 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.525 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720325 s, 146 MB/s 2024-04-25T10:01:44.533 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.533 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.533 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666373 s, 157 MB/s 2024-04-25T10:01:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688795 s, 152 MB/s 2024-04-25T10:01:44.548 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.548 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.548 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681674 s, 154 MB/s 2024-04-25T10:01:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761501 s, 138 MB/s 2024-04-25T10:01:44.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.564 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610978 s, 172 MB/s 2024-04-25T10:01:44.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.572 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690513 s, 152 MB/s 2024-04-25T10:01:44.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:44.573 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:44.573 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:44.573 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:01:44.573 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:44.573 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:44.573 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:44.573 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:44.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.639 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.639 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705898 s, 149 MB/s 2024-04-25T10:01:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.647 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736234 s, 142 MB/s 2024-04-25T10:01:44.655 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.655 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.655 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697776 s, 150 MB/s 2024-04-25T10:01:44.663 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.663 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.663 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736678 s, 142 MB/s 2024-04-25T10:01:44.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.671 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733344 s, 143 MB/s 2024-04-25T10:01:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599774 s, 175 MB/s 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701684 s, 149 MB/s 2024-04-25T10:01:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917125 s, 114 MB/s 2024-04-25T10:01:44.704 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.704 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.704 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726556 s, 144 MB/s 2024-04-25T10:01:44.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614909 s, 171 MB/s 2024-04-25T10:01:44.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.719 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067455 s, 155 MB/s 2024-04-25T10:01:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696133 s, 151 MB/s 2024-04-25T10:01:44.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.735 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705487 s, 149 MB/s 2024-04-25T10:01:44.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.742 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647642 s, 162 MB/s 2024-04-25T10:01:44.750 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.750 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.750 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688297 s, 152 MB/s 2024-04-25T10:01:44.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.758 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693045 s, 151 MB/s 2024-04-25T10:01:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724287 s, 145 MB/s 2024-04-25T10:01:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.774 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700658 s, 150 MB/s 2024-04-25T10:01:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829368 s, 126 MB/s 2024-04-25T10:01:44.790 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.791 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633259 s, 166 MB/s 2024-04-25T10:01:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705935 s, 149 MB/s 2024-04-25T10:01:44.806 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.806 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.806 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704306 s, 149 MB/s 2024-04-25T10:01:44.814 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.814 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.814 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671486 s, 156 MB/s 2024-04-25T10:01:44.822 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.822 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.822 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672177 s, 156 MB/s 2024-04-25T10:01:44.830 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.830 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.830 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713102 s, 147 MB/s 2024-04-25T10:01:44.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.837 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065119 s, 161 MB/s 2024-04-25T10:01:44.845 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.845 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.845 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689153 s, 152 MB/s 2024-04-25T10:01:44.853 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.853 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.853 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746465 s, 140 MB/s 2024-04-25T10:01:44.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.861 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708536 s, 148 MB/s 2024-04-25T10:01:44.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.869 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675474 s, 155 MB/s 2024-04-25T10:01:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.877 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683315 s, 153 MB/s 2024-04-25T10:01:44.884 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.884 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.884 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661786 s, 158 MB/s 2024-04-25T10:01:44.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:44.885 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:44.885 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:44.886 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:01:44.886 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:44.886 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:44.886 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:44.886 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690785 s, 152 MB/s 2024-04-25T10:01:44.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.958 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703884 s, 149 MB/s 2024-04-25T10:01:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772523 s, 136 MB/s 2024-04-25T10:01:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708945 s, 148 MB/s 2024-04-25T10:01:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716425 s, 146 MB/s 2024-04-25T10:01:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.990 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665633 s, 158 MB/s 2024-04-25T10:01:44.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:44.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:44.998 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066602 s, 157 MB/s 2024-04-25T10:01:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660212 s, 159 MB/s 2024-04-25T10:01:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735986 s, 142 MB/s 2024-04-25T10:01:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661048 s, 159 MB/s 2024-04-25T10:01:45.028 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.029 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648113 s, 162 MB/s 2024-04-25T10:01:45.036 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.036 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.037 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694527 s, 151 MB/s 2024-04-25T10:01:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681107 s, 154 MB/s 2024-04-25T10:01:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713455 s, 147 MB/s 2024-04-25T10:01:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706029 s, 149 MB/s 2024-04-25T10:01:45.068 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.068 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.068 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691282 s, 152 MB/s 2024-04-25T10:01:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672286 s, 156 MB/s 2024-04-25T10:01:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.086 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633779 s, 165 MB/s 2024-04-25T10:01:45.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.091 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693661 s, 151 MB/s 2024-04-25T10:01:45.099 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.099 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.099 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791238 s, 133 MB/s 2024-04-25T10:01:45.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.107 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693774 s, 151 MB/s 2024-04-25T10:01:45.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.116 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730102 s, 144 MB/s 2024-04-25T10:01:45.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.123 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640855 s, 164 MB/s 2024-04-25T10:01:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741738 s, 141 MB/s 2024-04-25T10:01:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624814 s, 168 MB/s 2024-04-25T10:01:45.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.146 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688062 s, 152 MB/s 2024-04-25T10:01:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632335 s, 166 MB/s 2024-04-25T10:01:45.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.162 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753886 s, 139 MB/s 2024-04-25T10:01:45.170 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.170 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.170 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745672 s, 141 MB/s 2024-04-25T10:01:45.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.178 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717635 s, 146 MB/s 2024-04-25T10:01:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068236 s, 154 MB/s 2024-04-25T10:01:45.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.193 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626423 s, 167 MB/s 2024-04-25T10:01:45.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:45.194 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:45.194 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:45.195 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:01:45.195 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:45.195 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:45.195 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:45.195 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:45.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.260 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741714 s, 141 MB/s 2024-04-25T10:01:45.267 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.267 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.267 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661307 s, 159 MB/s 2024-04-25T10:01:45.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.275 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716037 s, 146 MB/s 2024-04-25T10:01:45.282 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.283 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656422 s, 160 MB/s 2024-04-25T10:01:45.290 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.290 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.290 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635765 s, 165 MB/s 2024-04-25T10:01:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697896 s, 150 MB/s 2024-04-25T10:01:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746911 s, 140 MB/s 2024-04-25T10:01:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835948 s, 125 MB/s 2024-04-25T10:01:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874583 s, 120 MB/s 2024-04-25T10:01:45.333 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.333 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.333 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715009 s, 147 MB/s 2024-04-25T10:01:45.341 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.341 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.341 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675085 s, 155 MB/s 2024-04-25T10:01:45.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681464 s, 154 MB/s 2024-04-25T10:01:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644055 s, 163 MB/s 2024-04-25T10:01:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630374 s, 166 MB/s 2024-04-25T10:01:45.371 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.371 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.371 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688648 s, 152 MB/s 2024-04-25T10:01:45.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.379 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007169 s, 146 MB/s 2024-04-25T10:01:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.388 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074579 s, 141 MB/s 2024-04-25T10:01:45.395 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.395 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.395 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714984 s, 147 MB/s 2024-04-25T10:01:45.403 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.403 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.403 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659176 s, 159 MB/s 2024-04-25T10:01:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.411 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669542 s, 157 MB/s 2024-04-25T10:01:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700213 s, 150 MB/s 2024-04-25T10:01:45.426 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.426 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.427 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713083 s, 147 MB/s 2024-04-25T10:01:45.434 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.434 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.434 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622781 s, 168 MB/s 2024-04-25T10:01:45.442 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.442 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.442 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717078 s, 146 MB/s 2024-04-25T10:01:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660062 s, 159 MB/s 2024-04-25T10:01:45.457 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.457 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.457 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723299 s, 145 MB/s 2024-04-25T10:01:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.465 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667392 s, 157 MB/s 2024-04-25T10:01:45.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.474 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.474 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830084 s, 126 MB/s 2024-04-25T10:01:45.482 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.482 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.482 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682893 s, 154 MB/s 2024-04-25T10:01:45.490 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.493 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072093 s, 145 MB/s 2024-04-25T10:01:45.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.498 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719959 s, 146 MB/s 2024-04-25T10:01:45.506 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.507 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.507 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727226 s, 144 MB/s 2024-04-25T10:01:45.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:45.508 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:45.508 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:45.508 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:01:45.508 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:45.508 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:45.508 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:45.508 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634008 s, 165 MB/s 2024-04-25T10:01:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725286 s, 145 MB/s 2024-04-25T10:01:45.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.588 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619523 s, 169 MB/s 2024-04-25T10:01:45.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.596 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684526 s, 153 MB/s 2024-04-25T10:01:45.604 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.604 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.604 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707663 s, 148 MB/s 2024-04-25T10:01:45.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.614 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00935318 s, 112 MB/s 2024-04-25T10:01:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645099 s, 163 MB/s 2024-04-25T10:01:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.631 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.631 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689932 s, 152 MB/s 2024-04-25T10:01:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815334 s, 129 MB/s 2024-04-25T10:01:45.645 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.645 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.646 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641596 s, 163 MB/s 2024-04-25T10:01:45.654 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.654 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.654 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773399 s, 136 MB/s 2024-04-25T10:01:45.662 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.662 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.663 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735098 s, 143 MB/s 2024-04-25T10:01:45.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.672 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813449 s, 129 MB/s 2024-04-25T10:01:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.680 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714529 s, 147 MB/s 2024-04-25T10:01:45.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.687 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625895 s, 168 MB/s 2024-04-25T10:01:45.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.694 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669826 s, 157 MB/s 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701979 s, 149 MB/s 2024-04-25T10:01:45.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.710 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704168 s, 149 MB/s 2024-04-25T10:01:45.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.718 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705223 s, 149 MB/s 2024-04-25T10:01:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651284 s, 161 MB/s 2024-04-25T10:01:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.734 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743612 s, 141 MB/s 2024-04-25T10:01:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703159 s, 149 MB/s 2024-04-25T10:01:45.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858365 s, 122 MB/s 2024-04-25T10:01:45.759 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.759 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.759 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698102 s, 150 MB/s 2024-04-25T10:01:45.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.766 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620168 s, 169 MB/s 2024-04-25T10:01:45.774 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.774 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.774 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640553 s, 164 MB/s 2024-04-25T10:01:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678957 s, 154 MB/s 2024-04-25T10:01:45.789 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.789 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.789 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065925 s, 159 MB/s 2024-04-25T10:01:45.798 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.798 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.798 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825268 s, 127 MB/s 2024-04-25T10:01:45.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658369 s, 159 MB/s 2024-04-25T10:01:45.813 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.813 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.813 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670997 s, 156 MB/s 2024-04-25T10:01:45.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.820 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.820 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635313 s, 165 MB/s 2024-04-25T10:01:45.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:45.821 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:45.822 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:45.822 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:01:45.822 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:45.822 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:45.822 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:45.822 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:45.886 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640474 s, 164 MB/s 2024-04-25T10:01:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.894 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006221 s, 169 MB/s 2024-04-25T10:01:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716284 s, 146 MB/s 2024-04-25T10:01:45.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.909 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702878 s, 149 MB/s 2024-04-25T10:01:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732361 s, 143 MB/s 2024-04-25T10:01:45.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.925 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650835 s, 161 MB/s 2024-04-25T10:01:45.932 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.932 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.932 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644875 s, 163 MB/s 2024-04-25T10:01:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751952 s, 139 MB/s 2024-04-25T10:01:45.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.948 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629704 s, 167 MB/s 2024-04-25T10:01:45.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.958 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876828 s, 120 MB/s 2024-04-25T10:01:45.966 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.966 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.966 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732192 s, 143 MB/s 2024-04-25T10:01:45.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671843 s, 156 MB/s 2024-04-25T10:01:45.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.982 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761861 s, 138 MB/s 2024-04-25T10:01:45.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.989 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641597 s, 163 MB/s 2024-04-25T10:01:45.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:45.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:45.999 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873338 s, 120 MB/s 2024-04-25T10:01:46.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.007 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719518 s, 146 MB/s 2024-04-25T10:01:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703983 s, 149 MB/s 2024-04-25T10:01:46.023 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.024 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764441 s, 137 MB/s 2024-04-25T10:01:46.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.033 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830542 s, 126 MB/s 2024-04-25T10:01:46.040 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.041 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690302 s, 152 MB/s 2024-04-25T10:01:46.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.048 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641449 s, 163 MB/s 2024-04-25T10:01:46.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.057 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827162 s, 127 MB/s 2024-04-25T10:01:46.065 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.065 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.065 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706667 s, 148 MB/s 2024-04-25T10:01:46.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.073 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723403 s, 145 MB/s 2024-04-25T10:01:46.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.081 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683834 s, 153 MB/s 2024-04-25T10:01:46.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.089 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684872 s, 153 MB/s 2024-04-25T10:01:46.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.097 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740601 s, 142 MB/s 2024-04-25T10:01:46.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.106 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692619 s, 151 MB/s 2024-04-25T10:01:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063937 s, 164 MB/s 2024-04-25T10:01:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819878 s, 128 MB/s 2024-04-25T10:01:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697082 s, 150 MB/s 2024-04-25T10:01:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.137 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633773 s, 165 MB/s 2024-04-25T10:01:46.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:46.138 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:46.138 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:46.138 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:01:46.138 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:46.138 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:46.138 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:46.138 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.204 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.204 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836553 s, 125 MB/s 2024-04-25T10:01:46.211 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.211 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.211 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702855 s, 149 MB/s 2024-04-25T10:01:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850416 s, 123 MB/s 2024-04-25T10:01:46.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.229 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724669 s, 145 MB/s 2024-04-25T10:01:46.236 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.236 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.237 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669134 s, 157 MB/s 2024-04-25T10:01:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083257 s, 126 MB/s 2024-04-25T10:01:46.254 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.254 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.254 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757182 s, 138 MB/s 2024-04-25T10:01:46.262 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.262 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.262 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654285 s, 160 MB/s 2024-04-25T10:01:46.269 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.269 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.270 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696555 s, 151 MB/s 2024-04-25T10:01:46.277 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685193 s, 153 MB/s 2024-04-25T10:01:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643663 s, 163 MB/s 2024-04-25T10:01:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701969 s, 149 MB/s 2024-04-25T10:01:46.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.299 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061825 s, 170 MB/s 2024-04-25T10:01:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703544 s, 149 MB/s 2024-04-25T10:01:46.316 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.316 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.316 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825643 s, 127 MB/s 2024-04-25T10:01:46.324 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.324 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.324 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641362 s, 163 MB/s 2024-04-25T10:01:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664961 s, 158 MB/s 2024-04-25T10:01:46.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.339 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713504 s, 147 MB/s 2024-04-25T10:01:46.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.348 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.348 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790274 s, 133 MB/s 2024-04-25T10:01:46.355 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.355 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.355 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062744 s, 167 MB/s 2024-04-25T10:01:46.362 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.362 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.362 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620787 s, 169 MB/s 2024-04-25T10:01:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.372 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804187 s, 130 MB/s 2024-04-25T10:01:46.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.379 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066555 s, 158 MB/s 2024-04-25T10:01:46.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.388 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761685 s, 138 MB/s 2024-04-25T10:01:46.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.398 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926585 s, 113 MB/s 2024-04-25T10:01:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706796 s, 148 MB/s 2024-04-25T10:01:46.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.413 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687614 s, 152 MB/s 2024-04-25T10:01:46.422 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.422 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.422 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737748 s, 142 MB/s 2024-04-25T10:01:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674026 s, 156 MB/s 2024-04-25T10:01:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066214 s, 158 MB/s 2024-04-25T10:01:46.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.447 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896343 s, 117 MB/s 2024-04-25T10:01:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.454 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643908 s, 163 MB/s 2024-04-25T10:01:46.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:46.455 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:46.455 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:46.455 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:01:46.455 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:46.455 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:46.455 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:46.455 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:46.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.519 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604448 s, 173 MB/s 2024-04-25T10:01:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668142 s, 157 MB/s 2024-04-25T10:01:46.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.535 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787962 s, 133 MB/s 2024-04-25T10:01:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693279 s, 151 MB/s 2024-04-25T10:01:46.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065748 s, 159 MB/s 2024-04-25T10:01:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.559 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747272 s, 140 MB/s 2024-04-25T10:01:46.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.567 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719411 s, 146 MB/s 2024-04-25T10:01:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.575 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657599 s, 159 MB/s 2024-04-25T10:01:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691964 s, 152 MB/s 2024-04-25T10:01:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638823 s, 164 MB/s 2024-04-25T10:01:46.597 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.597 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.597 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622074 s, 169 MB/s 2024-04-25T10:01:46.608 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.608 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.608 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978368 s, 107 MB/s 2024-04-25T10:01:46.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695844 s, 151 MB/s 2024-04-25T10:01:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.625 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079245 s, 132 MB/s 2024-04-25T10:01:46.632 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.632 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.632 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708208 s, 148 MB/s 2024-04-25T10:01:46.640 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.640 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.641 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655634 s, 160 MB/s 2024-04-25T10:01:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072183 s, 145 MB/s 2024-04-25T10:01:46.655 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.655 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.656 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654627 s, 160 MB/s 2024-04-25T10:01:46.663 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.663 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.663 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658099 s, 159 MB/s 2024-04-25T10:01:46.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.671 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.671 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724033 s, 145 MB/s 2024-04-25T10:01:46.679 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.679 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.679 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714513 s, 147 MB/s 2024-04-25T10:01:46.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.687 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.687 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692481 s, 151 MB/s 2024-04-25T10:01:46.695 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.695 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.695 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728219 s, 144 MB/s 2024-04-25T10:01:46.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.703 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724551 s, 145 MB/s 2024-04-25T10:01:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715247 s, 147 MB/s 2024-04-25T10:01:46.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.719 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066346 s, 158 MB/s 2024-04-25T10:01:46.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.727 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711373 s, 147 MB/s 2024-04-25T10:01:46.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.735 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.735 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697313 s, 150 MB/s 2024-04-25T10:01:46.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.743 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679828 s, 154 MB/s 2024-04-25T10:01:46.750 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.750 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.750 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642928 s, 163 MB/s 2024-04-25T10:01:46.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.758 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702672 s, 149 MB/s 2024-04-25T10:01:46.765 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.765 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.765 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681264 s, 154 MB/s 2024-04-25T10:01:46.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:46.767 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:46.767 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:46.767 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:01:46.767 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:46.767 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:46.767 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:46.767 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.832 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664315 s, 158 MB/s 2024-04-25T10:01:46.840 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.840 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.840 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673293 s, 156 MB/s 2024-04-25T10:01:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106258 s, 98.7 MB/s 2024-04-25T10:01:46.860 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.860 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.860 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082921 s, 126 MB/s 2024-04-25T10:01:46.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.867 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628362 s, 167 MB/s 2024-04-25T10:01:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631767 s, 166 MB/s 2024-04-25T10:01:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.883 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712858 s, 147 MB/s 2024-04-25T10:01:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681755 s, 154 MB/s 2024-04-25T10:01:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684299 s, 153 MB/s 2024-04-25T10:01:46.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.907 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.907 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722502 s, 145 MB/s 2024-04-25T10:01:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.915 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778934 s, 135 MB/s 2024-04-25T10:01:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651734 s, 161 MB/s 2024-04-25T10:01:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.932 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855156 s, 123 MB/s 2024-04-25T10:01:46.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.940 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.940 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671796 s, 156 MB/s 2024-04-25T10:01:46.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.948 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721384 s, 145 MB/s 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707476 s, 148 MB/s 2024-04-25T10:01:46.963 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.964 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.964 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645157 s, 163 MB/s 2024-04-25T10:01:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699484 s, 150 MB/s 2024-04-25T10:01:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620502 s, 169 MB/s 2024-04-25T10:01:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813591 s, 129 MB/s 2024-04-25T10:01:46.994 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:46.995 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:46.995 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068336 s, 153 MB/s 2024-04-25T10:01:47.002 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.002 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.002 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062071 s, 169 MB/s 2024-04-25T10:01:47.009 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.009 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.009 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665188 s, 158 MB/s 2024-04-25T10:01:47.017 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.017 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.017 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702797 s, 149 MB/s 2024-04-25T10:01:47.025 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.025 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.025 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723878 s, 145 MB/s 2024-04-25T10:01:47.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.033 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695254 s, 151 MB/s 2024-04-25T10:01:47.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.041 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.041 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070055 s, 150 MB/s 2024-04-25T10:01:47.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.048 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634175 s, 165 MB/s 2024-04-25T10:01:47.056 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.056 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.056 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695699 s, 151 MB/s 2024-04-25T10:01:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715206 s, 147 MB/s 2024-04-25T10:01:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724915 s, 145 MB/s 2024-04-25T10:01:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.080 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681488 s, 154 MB/s 2024-04-25T10:01:47.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:47.081 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:47.081 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:47.081 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:01:47.081 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:47.081 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:47.082 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:47.082 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:47.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.146 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647179 s, 162 MB/s 2024-04-25T10:01:47.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.154 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006818 s, 154 MB/s 2024-04-25T10:01:47.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641793 s, 163 MB/s 2024-04-25T10:01:47.168 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656245 s, 160 MB/s 2024-04-25T10:01:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.177 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764861 s, 137 MB/s 2024-04-25T10:01:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.186 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775399 s, 135 MB/s 2024-04-25T10:01:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678006 s, 155 MB/s 2024-04-25T10:01:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727111 s, 144 MB/s 2024-04-25T10:01:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621862 s, 169 MB/s 2024-04-25T10:01:47.216 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.216 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.216 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649774 s, 161 MB/s 2024-04-25T10:01:47.224 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.224 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.225 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071063 s, 148 MB/s 2024-04-25T10:01:47.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.233 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808711 s, 130 MB/s 2024-04-25T10:01:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663601 s, 158 MB/s 2024-04-25T10:01:47.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.251 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930339 s, 113 MB/s 2024-04-25T10:01:47.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.258 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627353 s, 167 MB/s 2024-04-25T10:01:47.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.266 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701096 s, 150 MB/s 2024-04-25T10:01:47.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.273 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.273 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637873 s, 164 MB/s 2024-04-25T10:01:47.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.283 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841631 s, 125 MB/s 2024-04-25T10:01:47.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.292 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803655 s, 130 MB/s 2024-04-25T10:01:47.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696924 s, 150 MB/s 2024-04-25T10:01:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724164 s, 145 MB/s 2024-04-25T10:01:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652603 s, 161 MB/s 2024-04-25T10:01:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724937 s, 145 MB/s 2024-04-25T10:01:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639163 s, 164 MB/s 2024-04-25T10:01:47.338 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.338 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.339 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071764 s, 146 MB/s 2024-04-25T10:01:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010051 s, 104 MB/s 2024-04-25T10:01:47.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.356 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617917 s, 170 MB/s 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684927 s, 153 MB/s 2024-04-25T10:01:47.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.372 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691957 s, 152 MB/s 2024-04-25T10:01:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.380 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652389 s, 161 MB/s 2024-04-25T10:01:47.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.387 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614878 s, 171 MB/s 2024-04-25T10:01:47.395 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.395 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.395 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736818 s, 142 MB/s 2024-04-25T10:01:47.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:47.396 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:47.396 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:47.396 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:01:47.396 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:47.397 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:47.397 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:47.397 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:47.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709798 s, 148 MB/s 2024-04-25T10:01:47.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.470 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731051 s, 143 MB/s 2024-04-25T10:01:47.476 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623514 s, 168 MB/s 2024-04-25T10:01:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636126 s, 165 MB/s 2024-04-25T10:01:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064691 s, 162 MB/s 2024-04-25T10:01:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658532 s, 159 MB/s 2024-04-25T10:01:47.506 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.506 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.506 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637052 s, 165 MB/s 2024-04-25T10:01:47.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.513 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609495 s, 172 MB/s 2024-04-25T10:01:47.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.520 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652642 s, 161 MB/s 2024-04-25T10:01:47.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.528 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625787 s, 168 MB/s 2024-04-25T10:01:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837784 s, 125 MB/s 2024-04-25T10:01:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709895 s, 148 MB/s 2024-04-25T10:01:47.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.553 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.553 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700923 s, 150 MB/s 2024-04-25T10:01:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704246 s, 149 MB/s 2024-04-25T10:01:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644796 s, 163 MB/s 2024-04-25T10:01:47.576 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.576 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.576 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718453 s, 146 MB/s 2024-04-25T10:01:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721012 s, 145 MB/s 2024-04-25T10:01:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.593 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766266 s, 137 MB/s 2024-04-25T10:01:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.601 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738555 s, 142 MB/s 2024-04-25T10:01:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648816 s, 162 MB/s 2024-04-25T10:01:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675293 s, 155 MB/s 2024-04-25T10:01:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728923 s, 144 MB/s 2024-04-25T10:01:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778594 s, 135 MB/s 2024-04-25T10:01:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.642 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731691 s, 143 MB/s 2024-04-25T10:01:47.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.651 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753258 s, 139 MB/s 2024-04-25T10:01:47.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.658 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716178 s, 146 MB/s 2024-04-25T10:01:47.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684536 s, 153 MB/s 2024-04-25T10:01:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701457 s, 149 MB/s 2024-04-25T10:01:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670335 s, 156 MB/s 2024-04-25T10:01:47.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.689 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689188 s, 152 MB/s 2024-04-25T10:01:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656741 s, 160 MB/s 2024-04-25T10:01:47.704 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.704 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.704 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649199 s, 162 MB/s 2024-04-25T10:01:47.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:47.705 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:47.705 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:47.705 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:01:47.706 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:47.706 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:47.706 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:47.706 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:47.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.772 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683606 s, 153 MB/s 2024-04-25T10:01:47.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.779 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656164 s, 160 MB/s 2024-04-25T10:01:47.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.788 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.788 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756022 s, 139 MB/s 2024-04-25T10:01:47.796 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.796 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.796 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729129 s, 144 MB/s 2024-04-25T10:01:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717888 s, 146 MB/s 2024-04-25T10:01:47.811 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.811 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.811 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630228 s, 166 MB/s 2024-04-25T10:01:47.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.818 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640844 s, 164 MB/s 2024-04-25T10:01:47.825 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.825 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.825 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612816 s, 171 MB/s 2024-04-25T10:01:47.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.833 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735167 s, 143 MB/s 2024-04-25T10:01:47.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.842 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747094 s, 140 MB/s 2024-04-25T10:01:47.849 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.849 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.849 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648522 s, 162 MB/s 2024-04-25T10:01:47.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.856 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643829 s, 163 MB/s 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062828 s, 167 MB/s 2024-04-25T10:01:47.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.871 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647355 s, 162 MB/s 2024-04-25T10:01:47.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.879 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697648 s, 150 MB/s 2024-04-25T10:01:47.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.887 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651637 s, 161 MB/s 2024-04-25T10:01:47.894 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.894 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.894 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633462 s, 166 MB/s 2024-04-25T10:01:47.901 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.901 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.901 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701452 s, 149 MB/s 2024-04-25T10:01:47.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.909 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704203 s, 149 MB/s 2024-04-25T10:01:47.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.919 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00908308 s, 115 MB/s 2024-04-25T10:01:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705585 s, 149 MB/s 2024-04-25T10:01:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677768 s, 155 MB/s 2024-04-25T10:01:47.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.946 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.946 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693065 s, 151 MB/s 2024-04-25T10:01:47.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.950 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061132 s, 172 MB/s 2024-04-25T10:01:47.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.959 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859771 s, 122 MB/s 2024-04-25T10:01:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699256 s, 150 MB/s 2024-04-25T10:01:47.975 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.975 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.975 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716429 s, 146 MB/s 2024-04-25T10:01:47.983 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.983 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.983 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687639 s, 152 MB/s 2024-04-25T10:01:47.991 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.991 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.991 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741929 s, 141 MB/s 2024-04-25T10:01:47.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:47.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:47.999 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063742 s, 165 MB/s 2024-04-25T10:01:48.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.008 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864951 s, 121 MB/s 2024-04-25T10:01:48.015 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.015 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.015 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637836 s, 164 MB/s 2024-04-25T10:01:48.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:48.017 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:48.017 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:48.017 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:01:48.017 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:48.017 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:48.017 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:48.017 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:48.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.083 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675456 s, 155 MB/s 2024-04-25T10:01:48.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.090 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667705 s, 157 MB/s 2024-04-25T10:01:48.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.098 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718506 s, 146 MB/s 2024-04-25T10:01:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.106 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680021 s, 154 MB/s 2024-04-25T10:01:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706696 s, 148 MB/s 2024-04-25T10:01:48.120 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.121 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630873 s, 166 MB/s 2024-04-25T10:01:48.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.129 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743915 s, 141 MB/s 2024-04-25T10:01:48.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.137 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705141 s, 149 MB/s 2024-04-25T10:01:48.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.145 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711403 s, 147 MB/s 2024-04-25T10:01:48.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.155 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847306 s, 124 MB/s 2024-04-25T10:01:48.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.163 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730874 s, 143 MB/s 2024-04-25T10:01:48.170 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.170 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.170 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685535 s, 153 MB/s 2024-04-25T10:01:48.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.178 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689722 s, 152 MB/s 2024-04-25T10:01:48.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719188 s, 146 MB/s 2024-04-25T10:01:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854544 s, 123 MB/s 2024-04-25T10:01:48.203 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.203 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.203 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639479 s, 164 MB/s 2024-04-25T10:01:48.211 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.211 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.211 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695547 s, 151 MB/s 2024-04-25T10:01:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762521 s, 138 MB/s 2024-04-25T10:01:48.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.228 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.228 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656457 s, 160 MB/s 2024-04-25T10:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.235 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742146 s, 141 MB/s 2024-04-25T10:01:48.243 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.243 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.243 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661013 s, 159 MB/s 2024-04-25T10:01:48.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.251 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675332 s, 155 MB/s 2024-04-25T10:01:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724496 s, 145 MB/s 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656946 s, 160 MB/s 2024-04-25T10:01:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855014 s, 123 MB/s 2024-04-25T10:01:48.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.283 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062117 s, 169 MB/s 2024-04-25T10:01:48.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.291 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.291 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662737 s, 158 MB/s 2024-04-25T10:01:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779698 s, 134 MB/s 2024-04-25T10:01:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759179 s, 138 MB/s 2024-04-25T10:01:48.316 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.316 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.316 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735254 s, 143 MB/s 2024-04-25T10:01:48.324 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.324 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.324 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649521 s, 161 MB/s 2024-04-25T10:01:48.332 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703317 s, 149 MB/s 2024-04-25T10:01:48.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:48.333 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:48.333 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:48.333 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:01:48.333 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:48.333 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:48.334 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:48.334 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.400 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730684 s, 144 MB/s 2024-04-25T10:01:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.408 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.408 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697391 s, 150 MB/s 2024-04-25T10:01:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715397 s, 147 MB/s 2024-04-25T10:01:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635832 s, 165 MB/s 2024-04-25T10:01:48.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.430 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.431 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676948 s, 155 MB/s 2024-04-25T10:01:48.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.438 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.438 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690826 s, 152 MB/s 2024-04-25T10:01:48.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.446 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.446 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650004 s, 161 MB/s 2024-04-25T10:01:48.454 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.454 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.454 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722576 s, 145 MB/s 2024-04-25T10:01:48.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723262 s, 145 MB/s 2024-04-25T10:01:48.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.470 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703803 s, 149 MB/s 2024-04-25T10:01:48.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.477 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622566 s, 168 MB/s 2024-04-25T10:01:48.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.485 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.485 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678755 s, 154 MB/s 2024-04-25T10:01:48.492 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.493 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715975 s, 146 MB/s 2024-04-25T10:01:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.501 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698571 s, 150 MB/s 2024-04-25T10:01:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693703 s, 151 MB/s 2024-04-25T10:01:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100558 s, 104 MB/s 2024-04-25T10:01:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662255 s, 158 MB/s 2024-04-25T10:01:48.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.534 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621776 s, 169 MB/s 2024-04-25T10:01:48.544 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.544 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.544 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870589 s, 120 MB/s 2024-04-25T10:01:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070806 s, 148 MB/s 2024-04-25T10:01:48.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.559 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693653 s, 151 MB/s 2024-04-25T10:01:48.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.568 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731903 s, 143 MB/s 2024-04-25T10:01:48.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.575 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616754 s, 170 MB/s 2024-04-25T10:01:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680236 s, 154 MB/s 2024-04-25T10:01:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.593 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.593 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921369 s, 114 MB/s 2024-04-25T10:01:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664782 s, 158 MB/s 2024-04-25T10:01:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688963 s, 152 MB/s 2024-04-25T10:01:48.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.615 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666095 s, 157 MB/s 2024-04-25T10:01:48.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.622 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634404 s, 165 MB/s 2024-04-25T10:01:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731533 s, 143 MB/s 2024-04-25T10:01:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696331 s, 151 MB/s 2024-04-25T10:01:48.646 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.646 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.646 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666312 s, 157 MB/s 2024-04-25T10:01:48.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:48.647 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:48.648 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:48.648 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:01:48.648 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:48.648 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:48.648 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:48.648 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:48.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.715 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.715 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746358 s, 140 MB/s 2024-04-25T10:01:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102032 s, 103 MB/s 2024-04-25T10:01:48.733 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.733 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.733 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692335 s, 151 MB/s 2024-04-25T10:01:48.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.742 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071792 s, 146 MB/s 2024-04-25T10:01:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710818 s, 148 MB/s 2024-04-25T10:01:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.758 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733611 s, 143 MB/s 2024-04-25T10:01:48.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.766 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721432 s, 145 MB/s 2024-04-25T10:01:48.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.773 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648226 s, 162 MB/s 2024-04-25T10:01:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937054 s, 112 MB/s 2024-04-25T10:01:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739043 s, 142 MB/s 2024-04-25T10:01:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815488 s, 129 MB/s 2024-04-25T10:01:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891331 s, 118 MB/s 2024-04-25T10:01:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.818 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662019 s, 158 MB/s 2024-04-25T10:01:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649418 s, 161 MB/s 2024-04-25T10:01:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695532 s, 151 MB/s 2024-04-25T10:01:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936983 s, 112 MB/s 2024-04-25T10:01:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.852 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723207 s, 145 MB/s 2024-04-25T10:01:48.859 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.859 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.859 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653808 s, 160 MB/s 2024-04-25T10:01:48.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.867 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748576 s, 140 MB/s 2024-04-25T10:01:48.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.875 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710869 s, 148 MB/s 2024-04-25T10:01:48.882 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.882 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.882 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062382 s, 168 MB/s 2024-04-25T10:01:48.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.891 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706455 s, 148 MB/s 2024-04-25T10:01:48.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.899 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714086 s, 147 MB/s 2024-04-25T10:01:48.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.906 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642388 s, 163 MB/s 2024-04-25T10:01:48.913 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.914 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676691 s, 155 MB/s 2024-04-25T10:01:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.921 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632598 s, 166 MB/s 2024-04-25T10:01:48.929 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.929 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.929 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726058 s, 144 MB/s 2024-04-25T10:01:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701283 s, 150 MB/s 2024-04-25T10:01:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638465 s, 164 MB/s 2024-04-25T10:01:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725753 s, 144 MB/s 2024-04-25T10:01:48.961 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.961 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.962 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832685 s, 126 MB/s 2024-04-25T10:01:48.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708363 s, 148 MB/s 2024-04-25T10:01:48.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:48.971 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:48.971 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:48.971 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:01:48.971 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:48.971 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:48.971 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:48.971 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070797 s, 148 MB/s 2024-04-25T10:01:49.045 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.046 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775173 s, 135 MB/s 2024-04-25T10:01:49.053 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.053 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.053 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651227 s, 161 MB/s 2024-04-25T10:01:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671405 s, 156 MB/s 2024-04-25T10:01:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631816 s, 166 MB/s 2024-04-25T10:01:49.076 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.076 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.076 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716982 s, 146 MB/s 2024-04-25T10:01:49.084 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.084 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.084 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709606 s, 148 MB/s 2024-04-25T10:01:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.092 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.092 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651459 s, 161 MB/s 2024-04-25T10:01:49.100 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.100 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.100 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758143 s, 138 MB/s 2024-04-25T10:01:49.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.107 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064035 s, 164 MB/s 2024-04-25T10:01:49.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.115 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717001 s, 146 MB/s 2024-04-25T10:01:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695657 s, 151 MB/s 2024-04-25T10:01:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755136 s, 139 MB/s 2024-04-25T10:01:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872122 s, 120 MB/s 2024-04-25T10:01:49.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.149 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.149 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070931 s, 148 MB/s 2024-04-25T10:01:49.157 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.157 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.158 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724126 s, 145 MB/s 2024-04-25T10:01:49.166 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.166 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.166 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741217 s, 141 MB/s 2024-04-25T10:01:49.174 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.174 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.174 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746383 s, 140 MB/s 2024-04-25T10:01:49.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.182 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674502 s, 155 MB/s 2024-04-25T10:01:49.191 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.191 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.191 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830841 s, 126 MB/s 2024-04-25T10:01:49.199 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.199 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.199 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720395 s, 146 MB/s 2024-04-25T10:01:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861263 s, 122 MB/s 2024-04-25T10:01:49.218 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.218 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.218 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829804 s, 126 MB/s 2024-04-25T10:01:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.229 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100092 s, 105 MB/s 2024-04-25T10:01:49.238 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.238 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.238 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820701 s, 128 MB/s 2024-04-25T10:01:49.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.245 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626103 s, 167 MB/s 2024-04-25T10:01:49.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.253 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.253 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666996 s, 157 MB/s 2024-04-25T10:01:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.260 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704429 s, 149 MB/s 2024-04-25T10:01:49.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.268 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640823 s, 164 MB/s 2024-04-25T10:01:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699899 s, 150 MB/s 2024-04-25T10:01:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669644 s, 157 MB/s 2024-04-25T10:01:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.294 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103078 s, 102 MB/s 2024-04-25T10:01:49.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:49.296 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:49.296 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:49.296 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:01:49.296 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:49.296 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:49.296 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:49.296 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:49.361 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.361 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.361 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698002 s, 150 MB/s 2024-04-25T10:01:49.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.370 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809718 s, 129 MB/s 2024-04-25T10:01:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692234 s, 151 MB/s 2024-04-25T10:01:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736147 s, 142 MB/s 2024-04-25T10:01:49.397 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.397 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.397 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953327 s, 110 MB/s 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630035 s, 166 MB/s 2024-04-25T10:01:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.411 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063264 s, 166 MB/s 2024-04-25T10:01:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612542 s, 171 MB/s 2024-04-25T10:01:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656998 s, 160 MB/s 2024-04-25T10:01:49.433 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.433 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.433 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626232 s, 167 MB/s 2024-04-25T10:01:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686361 s, 153 MB/s 2024-04-25T10:01:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714461 s, 147 MB/s 2024-04-25T10:01:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670586 s, 156 MB/s 2024-04-25T10:01:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624403 s, 168 MB/s 2024-04-25T10:01:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706032 s, 149 MB/s 2024-04-25T10:01:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662692 s, 158 MB/s 2024-04-25T10:01:49.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.486 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064701 s, 162 MB/s 2024-04-25T10:01:49.494 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.494 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.494 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729366 s, 144 MB/s 2024-04-25T10:01:49.503 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.503 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.503 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743882 s, 141 MB/s 2024-04-25T10:01:49.510 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.510 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.510 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691952 s, 152 MB/s 2024-04-25T10:01:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713157 s, 147 MB/s 2024-04-25T10:01:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723216 s, 145 MB/s 2024-04-25T10:01:49.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.535 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710107 s, 148 MB/s 2024-04-25T10:01:49.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.543 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759616 s, 138 MB/s 2024-04-25T10:01:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742223 s, 141 MB/s 2024-04-25T10:01:49.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.559 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652272 s, 161 MB/s 2024-04-25T10:01:49.566 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.566 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.567 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655121 s, 160 MB/s 2024-04-25T10:01:49.574 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.574 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.574 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679783 s, 154 MB/s 2024-04-25T10:01:49.582 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.582 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.582 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670488 s, 156 MB/s 2024-04-25T10:01:49.590 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.590 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.590 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069598 s, 151 MB/s 2024-04-25T10:01:49.598 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.598 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.598 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715939 s, 146 MB/s 2024-04-25T10:01:49.605 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.605 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.605 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659965 s, 159 MB/s 2024-04-25T10:01:49.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:49.606 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:49.607 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:49.607 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:01:49.607 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:49.607 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:49.607 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:49.607 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:49.672 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.672 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.672 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687445 s, 153 MB/s 2024-04-25T10:01:49.681 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.682 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852468 s, 123 MB/s 2024-04-25T10:01:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.689 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688773 s, 152 MB/s 2024-04-25T10:01:49.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.697 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648887 s, 162 MB/s 2024-04-25T10:01:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076009 s, 138 MB/s 2024-04-25T10:01:49.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634144 s, 165 MB/s 2024-04-25T10:01:49.721 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.721 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.721 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716824 s, 146 MB/s 2024-04-25T10:01:49.729 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.729 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.729 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734979 s, 143 MB/s 2024-04-25T10:01:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.737 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065206 s, 161 MB/s 2024-04-25T10:01:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.745 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695394 s, 151 MB/s 2024-04-25T10:01:49.752 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.753 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.753 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072424 s, 145 MB/s 2024-04-25T10:01:49.761 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.761 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.761 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071595 s, 146 MB/s 2024-04-25T10:01:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655383 s, 160 MB/s 2024-04-25T10:01:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650143 s, 161 MB/s 2024-04-25T10:01:49.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.785 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818713 s, 128 MB/s 2024-04-25T10:01:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728213 s, 144 MB/s 2024-04-25T10:01:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689809 s, 152 MB/s 2024-04-25T10:01:49.809 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.809 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.809 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707228 s, 148 MB/s 2024-04-25T10:01:49.816 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.817 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.817 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695072 s, 151 MB/s 2024-04-25T10:01:49.824 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.825 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.825 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708018 s, 148 MB/s 2024-04-25T10:01:49.832 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.832 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.832 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686654 s, 153 MB/s 2024-04-25T10:01:49.840 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.840 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.840 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663594 s, 158 MB/s 2024-04-25T10:01:49.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.848 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074205 s, 141 MB/s 2024-04-25T10:01:49.858 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.858 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.858 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853899 s, 123 MB/s 2024-04-25T10:01:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071824 s, 146 MB/s 2024-04-25T10:01:49.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.874 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.874 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692156 s, 151 MB/s 2024-04-25T10:01:49.882 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.882 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.882 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648615 s, 162 MB/s 2024-04-25T10:01:49.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.890 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798767 s, 131 MB/s 2024-04-25T10:01:49.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.899 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.899 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826683 s, 127 MB/s 2024-04-25T10:01:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856233 s, 122 MB/s 2024-04-25T10:01:49.916 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.916 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.916 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691637 s, 152 MB/s 2024-04-25T10:01:49.924 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728749 s, 144 MB/s 2024-04-25T10:01:49.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:49.926 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:49.926 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:49.926 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:01:49.926 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:49.926 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:49.926 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:49.926 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750023 s, 140 MB/s 2024-04-25T10:01:50.001 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.001 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.001 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811923 s, 129 MB/s 2024-04-25T10:01:50.009 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.009 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.009 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073413 s, 143 MB/s 2024-04-25T10:01:50.017 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.017 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.017 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072819 s, 144 MB/s 2024-04-25T10:01:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.026 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754494 s, 139 MB/s 2024-04-25T10:01:50.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.033 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.033 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617665 s, 170 MB/s 2024-04-25T10:01:50.040 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.040 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.041 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671663 s, 156 MB/s 2024-04-25T10:01:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677962 s, 155 MB/s 2024-04-25T10:01:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828598 s, 127 MB/s 2024-04-25T10:01:50.065 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.065 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.065 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693134 s, 151 MB/s 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703303 s, 149 MB/s 2024-04-25T10:01:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714988 s, 147 MB/s 2024-04-25T10:01:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741143 s, 141 MB/s 2024-04-25T10:01:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733756 s, 143 MB/s 2024-04-25T10:01:50.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.106 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071089 s, 148 MB/s 2024-04-25T10:01:50.114 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.114 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.114 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713088 s, 147 MB/s 2024-04-25T10:01:50.122 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.122 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.122 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712204 s, 147 MB/s 2024-04-25T10:01:50.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.129 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601466 s, 174 MB/s 2024-04-25T10:01:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722488 s, 145 MB/s 2024-04-25T10:01:50.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.146 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745878 s, 141 MB/s 2024-04-25T10:01:50.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.154 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725201 s, 145 MB/s 2024-04-25T10:01:50.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659144 s, 159 MB/s 2024-04-25T10:01:50.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717257 s, 146 MB/s 2024-04-25T10:01:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.177 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634052 s, 165 MB/s 2024-04-25T10:01:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748188 s, 140 MB/s 2024-04-25T10:01:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743372 s, 141 MB/s 2024-04-25T10:01:50.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.201 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697779 s, 150 MB/s 2024-04-25T10:01:50.211 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.211 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.211 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899449 s, 117 MB/s 2024-04-25T10:01:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676406 s, 155 MB/s 2024-04-25T10:01:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.227 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701568 s, 149 MB/s 2024-04-25T10:01:50.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.234 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687321 s, 153 MB/s 2024-04-25T10:01:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862309 s, 122 MB/s 2024-04-25T10:01:50.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:50.245 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:50.245 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:01:50.245 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:01:50.245 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:50.245 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:50.245 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:01:50.246 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:01:50.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696421 s, 151 MB/s 2024-04-25T10:01:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675325 s, 155 MB/s 2024-04-25T10:01:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658718 s, 159 MB/s 2024-04-25T10:01:50.333 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.333 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.333 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683622 s, 153 MB/s 2024-04-25T10:01:50.342 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.342 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.342 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781082 s, 134 MB/s 2024-04-25T10:01:50.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.350 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.350 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699909 s, 150 MB/s 2024-04-25T10:01:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792377 s, 132 MB/s 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704074 s, 149 MB/s 2024-04-25T10:01:50.374 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.374 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.374 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642857 s, 163 MB/s 2024-04-25T10:01:50.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.381 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651474 s, 161 MB/s 2024-04-25T10:01:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877681 s, 119 MB/s 2024-04-25T10:01:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703796 s, 149 MB/s 2024-04-25T10:01:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067624 s, 155 MB/s 2024-04-25T10:01:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784823 s, 134 MB/s 2024-04-25T10:01:50.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.423 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652339 s, 161 MB/s 2024-04-25T10:01:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.432 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773775 s, 136 MB/s 2024-04-25T10:01:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633191 s, 166 MB/s 2024-04-25T10:01:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730942 s, 143 MB/s 2024-04-25T10:01:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662505 s, 158 MB/s 2024-04-25T10:01:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678034 s, 155 MB/s 2024-04-25T10:01:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670603 s, 156 MB/s 2024-04-25T10:01:50.479 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.479 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.479 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767307 s, 137 MB/s 2024-04-25T10:01:50.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.486 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615501 s, 170 MB/s 2024-04-25T10:01:50.495 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.495 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.495 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827518 s, 127 MB/s 2024-04-25T10:01:50.503 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.503 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.503 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733175 s, 143 MB/s 2024-04-25T10:01:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704714 s, 149 MB/s 2024-04-25T10:01:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685238 s, 153 MB/s 2024-04-25T10:01:50.526 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.527 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670694 s, 156 MB/s 2024-04-25T10:01:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.535 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705179 s, 149 MB/s 2024-04-25T10:01:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678554 s, 155 MB/s 2024-04-25T10:01:50.550 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.550 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.550 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705828 s, 149 MB/s 2024-04-25T10:01:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:01:50.558 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:01:50.559 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731302 s, 143 MB/s 2024-04-25T10:01:50.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:50.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:01:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.701+0000 7f29c5e58640 1 Processor -- start 2024-04-25T10:01:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.701+0000 7f29c5e58640 1 -- start start 2024-04-25T10:01:50.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29c5e58640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29c0161240 0x7f29c00b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29c5e58640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f29c0161890 0x7f29c00ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29c5e58640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c0161620 con 0x7f29c00b54a0 2024-04-25T10:01:50.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29c5e58640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00b7120 con 0x7f29c0161890 2024-04-25T10:01:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29c5e58640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c0161e00 con 0x7f29c0161240 2024-04-25T10:01:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29b7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f29c0161890 0x7f29c00ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29bffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f29c00b54a0 0x7f29c00d1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39804/0 (socket says 172.21.15.19:39804) 2024-04-25T10:01:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29bffff640 1 -- 172.21.15.19:0/1844219704 learned_addr learned my addr 172.21.15.19:0/1844219704 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/1844219704 >> v2:172.21.15.112:3300/0 conn(0x7f29c0161240 0x7f29c00b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29b7fff640 1 -- 172.21.15.19:0/1844219704 >> v2:172.21.15.112:3300/0 conn(0x7f29c0161240 msgr2=0x7f29c00b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29b7fff640 1 --2- 172.21.15.19:0/1844219704 >> v2:172.21.15.112:3300/0 conn(0x7f29c0161240 0x7f29c00b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29b7fff640 1 -- 172.21.15.19:0/1844219704 >> v1:172.21.15.19:6789/0 conn(0x7f29c00b54a0 legacy=0x7f29c00d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.702+0000 7f29b7fff640 1 -- 172.21.15.19:0/1844219704 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c01620e0 con 0x7f29c0161890 2024-04-25T10:01:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.703+0000 7f29b7fff640 1 --2- 172.21.15.19:0/1844219704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c0161890 0x7f29c00ceac0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f29c00d0400 tx=0x7f29a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30566bdd322df9c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.703+0000 7f29beffd640 1 -- 172.21.15.19:0/1844219704 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8037070 con 0x7f29c0161890 2024-04-25T10:01:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.703+0000 7f29beffd640 1 -- 172.21.15.19:0/1844219704 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a8032440 con 0x7f29c0161890 2024-04-25T10:01:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.703+0000 7f29beffd640 1 -- 172.21.15.19:0/1844219704 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8032810 con 0x7f29c0161890 2024-04-25T10:01:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.704+0000 7f29c5e58640 1 -- 172.21.15.19:0/1844219704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c0161890 msgr2=0x7f29c00ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.704+0000 7f29c5e58640 1 --2- 172.21.15.19:0/1844219704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c0161890 0x7f29c00ceac0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f29c00d0400 tx=0x7f29a802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.704+0000 7f29c5e58640 1 -- 172.21.15.19:0/1844219704 shutdown_connections 2024-04-25T10:01:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.704+0000 7f29c5e58640 1 --2- 172.21.15.19:0/1844219704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c0161890 0x7f29c00ceac0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.704+0000 7f29c5e58640 1 --2- 172.21.15.19:0/1844219704 >> v2:172.21.15.112:3300/0 conn(0x7f29c0161240 0x7f29c00b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.704+0000 7f29c5e58640 1 -- 172.21.15.19:0/1844219704 >> 172.21.15.19:0/1844219704 conn(0x7f29c00d04a0 msgr2=0x7f29c00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.704+0000 7f29c5e58640 1 -- 172.21.15.19:0/1844219704 shutdown_connections 2024-04-25T10:01:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.704+0000 7f29c5e58640 1 -- 172.21.15.19:0/1844219704 wait complete. 2024-04-25T10:01:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.705+0000 7f29c5e58640 1 Processor -- start 2024-04-25T10:01:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.705+0000 7f29c5e58640 1 -- start start 2024-04-25T10:01:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29c5e58640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29c00b54a0 0x7f29c01f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29c5e58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c0161240 0x7f29c01f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29c5e58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c0161890 0x7f29c00b8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29c5e58640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00d15f0 con 0x7f29c0161890 2024-04-25T10:01:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29c5e58640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c01626a0 con 0x7f29c0161240 2024-04-25T10:01:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29c5e58640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00cf000 con 0x7f29c00b54a0 2024-04-25T10:01:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c0161890 0x7f29c00b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29bffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c0161890 0x7f29c00b8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38860/0 (socket says 172.21.15.19:38860) 2024-04-25T10:01:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29bffff640 1 -- 172.21.15.19:0/1849675345 learned_addr learned my addr 172.21.15.19:0/1849675345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29b7fff640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c0161240 0x7f29c01f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29c00b54a0 0x7f29c01f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29bffff640 1 -- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29c00b54a0 msgr2=0x7f29c01f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29bffff640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29c00b54a0 0x7f29c01f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29bffff640 1 -- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c0161240 msgr2=0x7f29c01f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29bffff640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c0161240 0x7f29c01f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29bffff640 1 -- 172.21.15.19:0/1849675345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c014d740 con 0x7f29c0161890 2024-04-25T10:01:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29c00b54a0 0x7f29c01f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:50.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.706+0000 7f29bffff640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c0161890 0x7f29c00b8430 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f29b800cbc0 tx=0x7f29b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.707+0000 7f29bcff9640 1 -- 172.21.15.19:0/1849675345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8017070 con 0x7f29c0161890 2024-04-25T10:01:50.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.707+0000 7f29bcff9640 1 -- 172.21.15.19:0/1849675345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b800ad40 con 0x7f29c0161890 2024-04-25T10:01:50.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.707+0000 7f29bcff9640 1 -- 172.21.15.19:0/1849675345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8012600 con 0x7f29c0161890 2024-04-25T10:01:50.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.707+0000 7f29c5e58640 1 -- 172.21.15.19:0/1849675345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c01f3d00 con 0x7f29c0161890 2024-04-25T10:01:50.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.707+0000 7f29c5e58640 1 -- 172.21.15.19:0/1849675345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c01f42c0 con 0x7f29c0161890 2024-04-25T10:01:50.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.708+0000 7f29bcff9640 1 -- 172.21.15.19:0/1849675345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29b801a070 con 0x7f29c0161890 2024-04-25T10:01:50.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.708+0000 7f29bcff9640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a006b4d0 0x7f29a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.709+0000 7f29bcff9640 1 -- 172.21.15.19:0/1849675345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29b8089f20 con 0x7f29c0161890 2024-04-25T10:01:50.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.709+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a006b4d0 0x7f29a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.709+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a006b4d0 0x7f29a006d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009ac0 tx=0x7f29b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.709+0000 7f29c8c36740 1 Processor -- start 2024-04-25T10:01:50.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.709+0000 7f29c8c36740 1 -- start start 2024-04-25T10:01:50.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.710+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.710+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.710+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd18a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.710+0000 7f29c8c36740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bd0c3000 con 0x5591bd15f200 2024-04-25T10:01:50.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.710+0000 7f29c8c36740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bd06bf20 con 0x5591bd19b8d0 2024-04-25T10:01:50.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.710+0000 7f29c8c36740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591bd0bf0a0 con 0x5591bd189cc0 2024-04-25T10:01:50.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.710+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38876/0 (socket says 172.21.15.19:38876) 2024-04-25T10:01:50.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29bf7fe640 1 -- 172.21.15.19:0/3847657535 learned_addr learned my addr 172.21.15.19:0/3847657535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:50.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29b7fff640 1 --2- 172.21.15.19:0/3847657535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29bffff640 1 --2- 172.21.15.19:0/3847657535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd18a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29bf7fe640 1 -- 172.21.15.19:0/3847657535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 msgr2=0x5591bd18a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/3847657535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd18a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29bf7fe640 1 -- 172.21.15.19:0/3847657535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 msgr2=0x5591bd15f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/3847657535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd15f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29bf7fe640 1 -- 172.21.15.19:0/3847657535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591bcea3ba0 con 0x5591bd15f200 2024-04-25T10:01:50.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/3847657535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd161360 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f29b0010b90 tx=0x7f29b0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978d248d88836066 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29b57fa640 1 -- 172.21.15.19:0/3847657535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b001c050 con 0x5591bd15f200 2024-04-25T10:01:50.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29b57fa640 1 -- 172.21.15.19:0/3847657535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0026070 con 0x5591bd15f200 2024-04-25T10:01:50.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.711+0000 7f29b57fa640 1 -- 172.21.15.19:0/3847657535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000f040 con 0x5591bd15f200 2024-04-25T10:01:50.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.712+0000 7f29c8c36740 1 -- 172.21.15.19:0/3847657535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 msgr2=0x5591bd161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.712+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3847657535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd161360 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f29b0010b90 tx=0x7f29b0013000 comp rx=0 tx=0).stop 2024-04-25T10:01:50.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.712+0000 7f29c8c36740 1 -- 172.21.15.19:0/3847657535 shutdown_connections 2024-04-25T10:01:50.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.712+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3847657535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd18a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.712+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3847657535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd15f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.712+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3847657535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd161360 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.712+0000 7f29c8c36740 1 -- 172.21.15.19:0/3847657535 >> 172.21.15.19:0/3847657535 conn(0x5591bd127050 msgr2=0x5591bd1a4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.712+0000 7f29c8c36740 1 -- 172.21.15.19:0/3847657535 shutdown_connections 2024-04-25T10:01:50.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.712+0000 7f29c8c36740 1 -- 172.21.15.19:0/3847657535 wait complete. 2024-04-25T10:01:50.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.712+0000 7f29c8c36740 1 -- start start 2024-04-25T10:01:50.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29c8c36740 1 Processor -- start 2024-04-25T10:01:50.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd15f200 0x5591bd18c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd189cc0 0x5591bd18a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd18d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29c8c36740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bd061610 con 0x5591bd189cc0 2024-04-25T10:01:50.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29c8c36740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bd187810 con 0x5591bd19b8d0 2024-04-25T10:01:50.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29c8c36740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591bd1b58f0 con 0x5591bd15f200 2024-04-25T10:01:50.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29c8c36740 10 client.0 _ll_register_callbacks cb 0x5591bd079f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:50.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd189cc0 0x5591bd18a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd189cc0 0x5591bd18a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38884/0 (socket says 172.21.15.19:38884) 2024-04-25T10:01:50.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29b7fff640 1 -- 172.21.15.19:0/3022370742 learned_addr learned my addr 172.21.15.19:0/3022370742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:50.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd15f200 0x5591bd18c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.713+0000 7f29bffff640 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd18d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f29b7fff640 1 -- 172.21.15.19:0/3022370742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd15f200 msgr2=0x5591bd18c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f29b7fff640 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd15f200 0x5591bd18c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f29b7fff640 1 -- 172.21.15.19:0/3022370742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 msgr2=0x5591bd18d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f29b7fff640 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd18d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f29b7fff640 1 -- 172.21.15.19:0/3022370742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591bcd97490 con 0x5591bd189cc0 2024-04-25T10:01:50.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f29bffff640 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd18d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:50.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd15f200 0x5591bd18c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:50.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f29b7fff640 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd189cc0 0x5591bd18a7f0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f29a802a790 tx=0x7f29a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfbe3764638d6eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f299e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:01:50.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f299e7fc640 1 -- 172.21.15.19:0/3022370742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8037040 con 0x5591bd189cc0 2024-04-25T10:01:50.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f299e7fc640 1 -- 172.21.15.19:0/3022370742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29a8032a90 con 0x5591bd189cc0 2024-04-25T10:01:50.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f29c8c36740 1 -- 172.21.15.19:0/3022370742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29b0026070 con 0x5591bd189cc0 2024-04-25T10:01:50.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f299e7fc640 1 -- 172.21.15.19:0/3022370742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a804e050 con 0x5591bd189cc0 2024-04-25T10:01:50.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f29c8c36740 10 client.9423 fetch_fsmap learned FSMap version 79 2024-04-25T10:01:50.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f29c8c36740 1 -- 172.21.15.19:0/3022370742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5591bcf04330 con 0x5591bd189cc0 2024-04-25T10:01:50.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.714+0000 7f299e7fc640 1 -- 172.21.15.19:0/3022370742 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0026070 con 0x5591bd189cc0 2024-04-25T10:01:50.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f299e7fc640 1 -- 172.21.15.19:0/3022370742 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29a8051020 con 0x5591bd189cc0 2024-04-25T10:01:50.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f29c8c36740 10 client.9423 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:01:50.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f29c8c36740 10 client.9423 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:01:50.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f29c8c36740 20 client.9423 populate_metadata read hostname 'smithi019' 2024-04-25T10:01:50.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5591bd18d6b0 0x5591bd1b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f29c8c36740 4 client.9423 mds_command: new command op to 9213 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:50.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f29c8c36740 1 -- 172.21.15.19:0/3022370742 --> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591bd0d0540 con 0x5591bd18d6b0 2024-04-25T10:01:50.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5591bd18d6b0 0x5591bd1b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5591bd18d6b0 0x5591bd1b4a80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=effd08e75ab6d1b9 server_cookie=1ed72e2386d023b2 in_seq=0 out_seq=0 2024-04-25T10:01:50.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f299e7fc640 10 client.9423 ms_handle_connect on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:50.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f299e7fc640 1 -- 172.21.15.19:0/3022370742 <== mds.0 v2:172.21.15.19:6833/250977615 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5591bd0d0540 con 0x5591bd18d6b0 2024-04-25T10:01:50.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.715+0000 7f299e7fc640 10 client.9423 handle_command_reply: tid=0 2024-04-25T10:01:50.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 1 client.9423 shutdown 2024-04-25T10:01:50.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 1 -- 172.21.15.19:0/3022370742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd189cc0 msgr2=0x5591bd18a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd189cc0 0x5591bd18a7f0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f29a802a790 tx=0x7f29a8004730 comp rx=0 tx=0).stop 2024-04-25T10:01:50.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 1 -- 172.21.15.19:0/3022370742 shutdown_connections 2024-04-25T10:01:50.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5591bd18d6b0 0x5591bd1b4a80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd18d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd189cc0 0x5591bd18a7f0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f299e7fc640 0 client.9423 ms_handle_reset on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3022370742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd15f200 0x5591bd18c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 1 -- 172.21.15.19:0/3022370742 >> 172.21.15.19:0/3022370742 conn(0x5591bd127050 msgr2=0x5591bd19dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 1 -- 172.21.15.19:0/3022370742 shutdown_connections 2024-04-25T10:01:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 1 -- 172.21.15.19:0/3022370742 wait complete. 2024-04-25T10:01:50.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.716+0000 7f29c8c36740 20 client.9423 trim_cache size 0 max 16384 2024-04-25T10:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.731+0000 7f29c8c36740 1 Processor -- start 2024-04-25T10:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.731+0000 7f29c8c36740 1 -- start start 2024-04-25T10:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd19fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd18a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29c8c36740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0200540 con 0x5591bd15f200 2024-04-25T10:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29c8c36740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c014f990 con 0x5591bd19b8d0 2024-04-25T10:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29c8c36740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c01f0e60 con 0x5591bd189cc0 2024-04-25T10:01:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38892/0 (socket says 172.21.15.19:38892) 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29bf7fe640 1 -- 172.21.15.19:0/551708025 learned_addr learned my addr 172.21.15.19:0/551708025 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29b7fff640 1 --2- 172.21.15.19:0/551708025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd19fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29bffff640 1 --2- 172.21.15.19:0/551708025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd18a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29bf7fe640 1 -- 172.21.15.19:0/551708025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 msgr2=0x5591bd19fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/551708025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd19fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29bf7fe640 1 -- 172.21.15.19:0/551708025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 msgr2=0x5591bd18a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/551708025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd18a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29bf7fe640 1 -- 172.21.15.19:0/551708025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591bd1a0360 con 0x5591bd15f200 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29bffff640 1 --2- 172.21.15.19:0/551708025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd18a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.732+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/551708025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd19eee0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f29b00105c0 tx=0x7f29b0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd62ff2407308f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29b4ff9640 1 -- 172.21.15.19:0/551708025 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b001c040 con 0x5591bd15f200 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29b4ff9640 1 -- 172.21.15.19:0/551708025 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0026040 con 0x5591bd15f200 2024-04-25T10:01:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29b4ff9640 1 -- 172.21.15.19:0/551708025 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0004480 con 0x5591bd15f200 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29c8c36740 1 -- 172.21.15.19:0/551708025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 msgr2=0x5591bd19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29c8c36740 1 --2- 172.21.15.19:0/551708025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd19eee0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f29b00105c0 tx=0x7f29b0015000 comp rx=0 tx=0).stop 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29c8c36740 1 -- 172.21.15.19:0/551708025 shutdown_connections 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29c8c36740 1 --2- 172.21.15.19:0/551708025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd18a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29c8c36740 1 --2- 172.21.15.19:0/551708025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29c8c36740 1 --2- 172.21.15.19:0/551708025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd19eee0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29c8c36740 1 -- 172.21.15.19:0/551708025 >> 172.21.15.19:0/551708025 conn(0x5591bd127050 msgr2=0x5591bd1a6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29c8c36740 1 -- 172.21.15.19:0/551708025 shutdown_connections 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.733+0000 7f29c8c36740 1 -- 172.21.15.19:0/551708025 wait complete. 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.734+0000 7f29c8c36740 1 -- start start 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.734+0000 7f29c8c36740 1 Processor -- start 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd178270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd19a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29c8c36740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd186f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29c8c36740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c014f390 con 0x5591bd15f200 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29c8c36740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bd184670 con 0x5591bd19b8d0 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29c8c36740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591bd19b130 con 0x5591bd189cc0 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29c8c36740 10 client.0 _ll_register_callbacks cb 0x7f29b0009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd178270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd178270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38900/0 (socket says 172.21.15.19:38900) 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29bf7fe640 1 -- 172.21.15.19:0/3647443407 learned_addr learned my addr 172.21.15.19:0/3647443407 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29b7fff640 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29bffff640 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd186f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29bf7fe640 1 -- 172.21.15.19:0/3647443407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 msgr2=0x5591bd19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29bf7fe640 1 -- 172.21.15.19:0/3647443407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 msgr2=0x5591bd186f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd186f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29bf7fe640 1 -- 172.21.15.19:0/3647443407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591bcd70610 con 0x5591bd15f200 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29bf7fe640 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd178270 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f29b0038040 tx=0x7f29b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44ca23d152f86e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f299e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f299e7fc640 1 -- 172.21.15.19:0/3647443407 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b001c040 con 0x5591bd15f200 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f299e7fc640 1 -- 172.21.15.19:0/3647443407 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29b0036810 con 0x5591bd15f200 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.735+0000 7f29c8c36740 1 -- 172.21.15.19:0/3647443407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29b0026040 con 0x5591bd15f200 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f299e7fc640 1 -- 172.21.15.19:0/3647443407 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29b0021750 con 0x5591bd15f200 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f29c8c36740 10 client.9429 fetch_fsmap learned FSMap version 79 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f29c8c36740 1 -- 172.21.15.19:0/3647443407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5591bd20fb20 con 0x5591bd15f200 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f299e7fc640 1 -- 172.21.15.19:0/3647443407 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0026040 con 0x5591bd15f200 2024-04-25T10:01:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f299e7fc640 1 -- 172.21.15.19:0/3647443407 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29b003f070 con 0x5591bd15f200 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f29c8c36740 10 client.9429 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f29c8c36740 10 client.9429 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f29c8c36740 20 client.9429 populate_metadata read hostname 'smithi019' 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5591bd19ab90 0x5591bd187440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f29c8c36740 4 client.9429 mds_command: new command op to 9213 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f29c8c36740 1 -- 172.21.15.19:0/3647443407 --> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5591bd0ca890 con 0x5591bd19ab90 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f29b7fff640 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5591bd19ab90 0x5591bd187440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f299e7fc640 10 client.9429 ms_handle_connect on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.736+0000 7f29b7fff640 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5591bd19ab90 0x5591bd187440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc8d2c0ebe76598 server_cookie=facbc6cdc5a6b4f0 in_seq=0 out_seq=0 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.740+0000 7f299e7fc640 1 -- 172.21.15.19:0/3647443407 <== mds.0 v2:172.21.15.19:6833/250977615 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5591bd0ca890 con 0x5591bd19ab90 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.740+0000 7f299e7fc640 10 client.9429 handle_command_reply: tid=0 2024-04-25T10:01:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.740+0000 7f29c8c36740 1 client.9429 shutdown 2024-04-25T10:01:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.740+0000 7f29c8c36740 1 -- 172.21.15.19:0/3647443407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 msgr2=0x5591bd178270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.740+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd178270 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f29b0038040 tx=0x7f29b0004730 comp rx=0 tx=0).stop 2024-04-25T10:01:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.740+0000 7f29c8c36740 1 -- 172.21.15.19:0/3647443407 shutdown_connections 2024-04-25T10:01:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.740+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5591bd19ab90 0x5591bd187440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.740+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591bd19b8d0 0x5591bd186f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.740+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5591bd189cc0 0x5591bd19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.740+0000 7f29c8c36740 1 --2- 172.21.15.19:0/3647443407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5591bd15f200 0x5591bd178270 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.740+0000 7f29c8c36740 1 -- 172.21.15.19:0/3647443407 >> 172.21.15.19:0/3647443407 conn(0x5591bd127050 msgr2=0x5591bd18a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f299e7fc640 0 client.9429 ms_handle_reset on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c8c36740 1 -- 172.21.15.19:0/3647443407 shutdown_connections 2024-04-25T10:01:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c8c36740 1 -- 172.21.15.19:0/3647443407 wait complete. 2024-04-25T10:01:50.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c8c36740 20 client.9429 trim_cache size 0 max 16384 2024-04-25T10:01:50.766 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:01:50.766 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:01:50.766 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "2e7d34d9-08f9-4095-95bb-ebac9cd01916", 2024-04-25T10:01:50.766 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:01:50.766 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 -- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a006b4d0 msgr2=0x7f29a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a006b4d0 0x7f29a006d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009ac0 tx=0x7f29b0008040 comp rx=0 tx=0).stop 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 -- 172.21.15.19:0/1849675345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c0161890 msgr2=0x7f29c00b8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c0161890 0x7f29c00b8430 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f29b800cbc0 tx=0x7f29b800a430 comp rx=0 tx=0).stop 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 -- 172.21.15.19:0/1849675345 shutdown_connections 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29a006b4d0 0x7f29a006d990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c0161890 0x7f29c00b8430 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c0161240 0x7f29c01f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 --2- 172.21.15.19:0/1849675345 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f29c00b54a0 0x7f29c01f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 -- 172.21.15.19:0/1849675345 >> 172.21.15.19:0/1849675345 conn(0x7f29c00d04a0 msgr2=0x7f29c00b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 -- 172.21.15.19:0/1849675345 shutdown_connections 2024-04-25T10:01:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.741+0000 7f29c5e58640 1 -- 172.21.15.19:0/1849675345 wait complete. 2024-04-25T10:01:50.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:50.769 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T10:01:50.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.888+0000 7f5cdc69e640 1 Processor -- start 2024-04-25T10:01:50.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.888+0000 7f5cdc69e640 1 -- start start 2024-04-25T10:01:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.889+0000 7f5cdc69e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5cd40cd4c0 0x7f5cd40cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.889+0000 7f5cdc69e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5cd415b1c0 0x7f5cd415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.889+0000 7f5cdc69e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415e1a0 0x7f5cd4150b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.889+0000 7f5cdc69e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd415e6e0 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.889+0000 7f5cdc69e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd415e9c0 con 0x7f5cd415b1c0 2024-04-25T10:01:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.889+0000 7f5cdc69e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd415a9c0 con 0x7f5cd415e1a0 2024-04-25T10:01:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.889+0000 7f5cda43a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5cd40cd4c0 0x7f5cd40cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.890+0000 7f5cda43a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5cd40cd4c0 0x7f5cd40cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38910/0 (socket says 172.21.15.19:38910) 2024-04-25T10:01:50.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.890+0000 7f5cda43a640 1 -- 172.21.15.19:0/1027493585 learned_addr learned my addr 172.21.15.19:0/1027493585 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.890+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/1027493585 >> v2:172.21.15.112:3300/0 conn(0x7f5cd415b1c0 0x7f5cd415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.890+0000 7f5cdac3b640 1 --2- 172.21.15.19:0/1027493585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415e1a0 0x7f5cd4150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.890+0000 7f5cda43a640 1 -- 172.21.15.19:0/1027493585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415e1a0 msgr2=0x7f5cd4150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.890+0000 7f5cda43a640 1 --2- 172.21.15.19:0/1027493585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415e1a0 0x7f5cd4150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.890+0000 7f5cda43a640 1 -- 172.21.15.19:0/1027493585 >> v2:172.21.15.112:3300/0 conn(0x7f5cd415b1c0 msgr2=0x7f5cd415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.890+0000 7f5cda43a640 1 --2- 172.21.15.19:0/1027493585 >> v2:172.21.15.112:3300/0 conn(0x7f5cd415b1c0 0x7f5cd415dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.890+0000 7f5cda43a640 1 -- 172.21.15.19:0/1027493585 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd415aca0 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.890+0000 7f5cdac3b640 1 --2- 172.21.15.19:0/1027493585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415e1a0 0x7f5cd4150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cda43a640 1 --2- 172.21.15.19:0/1027493585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd40cd4c0 0x7f5cd40cd8a0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40cdde0 tx=0x7f5cd002b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f6b82d65939d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cd9438640 1 -- 172.21.15.19:0/1027493585 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0038070 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cd9438640 1 -- 172.21.15.19:0/1027493585 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd0033640 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cd9438640 1 -- 172.21.15.19:0/1027493585 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0033a10 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cdc69e640 1 -- 172.21.15.19:0/1027493585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd40cd4c0 msgr2=0x7f5cd40cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cdc69e640 1 --2- 172.21.15.19:0/1027493585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd40cd4c0 0x7f5cd40cd8a0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40cdde0 tx=0x7f5cd002b2f0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cdc69e640 1 -- 172.21.15.19:0/1027493585 shutdown_connections 2024-04-25T10:01:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cdc69e640 1 --2- 172.21.15.19:0/1027493585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415e1a0 0x7f5cd4150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cdc69e640 1 --2- 172.21.15.19:0/1027493585 >> v2:172.21.15.112:3300/0 conn(0x7f5cd415b1c0 0x7f5cd415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cdc69e640 1 --2- 172.21.15.19:0/1027493585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd40cd4c0 0x7f5cd40cd8a0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cdc69e640 1 -- 172.21.15.19:0/1027493585 >> 172.21.15.19:0/1027493585 conn(0x7f5cd406abe0 msgr2=0x7f5cd406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cdc69e640 1 -- 172.21.15.19:0/1027493585 shutdown_connections 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.891+0000 7f5cdc69e640 1 -- 172.21.15.19:0/1027493585 wait complete. 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.893+0000 7f5cdc69e640 1 Processor -- start 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.893+0000 7f5cdc69e640 1 -- start start 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.893+0000 7f5cdc69e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd40cd4c0 0x7f5cd414e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.893+0000 7f5cdc69e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415b1c0 0x7f5cd40c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.893+0000 7f5cdc69e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415e1a0 0x7f5cd40adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.893+0000 7f5cdc69e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406ee20 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.893+0000 7f5cdc69e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406eca0 con 0x7f5cd415b1c0 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.893+0000 7f5cdc69e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41510d0 con 0x7f5cd415e1a0 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.893+0000 7f5cda43a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd40cd4c0 0x7f5cd414e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.893+0000 7f5cda43a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd40cd4c0 0x7f5cd414e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38926/0 (socket says 172.21.15.19:38926) 2024-04-25T10:01:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.893+0000 7f5cda43a640 1 -- 172.21.15.19:0/883697049 learned_addr learned my addr 172.21.15.19:0/883697049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cdac3b640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415e1a0 0x7f5cd40adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415b1c0 0x7f5cd40c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cda43a640 1 -- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415e1a0 msgr2=0x7f5cd40adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cda43a640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415e1a0 0x7f5cd40adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cda43a640 1 -- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415b1c0 msgr2=0x7f5cd40c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cda43a640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415b1c0 0x7f5cd40c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cda43a640 1 -- 172.21.15.19:0/883697049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd40c4b40 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415b1c0 0x7f5cd40c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cda43a640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd40cd4c0 0x7f5cd414e7e0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5cd002b330 tx=0x7f5cd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cc6ffd640 1 -- 172.21.15.19:0/883697049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0043070 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cc6ffd640 1 -- 172.21.15.19:0/883697049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd0031770 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cdc69e640 1 -- 172.21.15.19:0/883697049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd40c4e20 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cc6ffd640 1 -- 172.21.15.19:0/883697049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0038040 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.894+0000 7f5cdc69e640 1 -- 172.21.15.19:0/883697049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd40c53e0 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.896+0000 7f5cc6ffd640 1 -- 172.21.15.19:0/883697049 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5cd0046070 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.896+0000 7f5cc6ffd640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5cb806b500 0x7f5cb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.896+0000 7f5cc6ffd640 1 -- 172.21.15.19:0/883697049 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5cd00aa0a0 con 0x7f5cd40cd4c0 2024-04-25T10:01:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.896+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5cb806b500 0x7f5cb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.897+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5cb806b500 0x7f5cb806d9c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5cd414ed20 tx=0x7f5cc800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.897+0000 7f5cdf3a6740 1 Processor -- start 2024-04-25T10:01:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.897+0000 7f5cdf3a6740 1 -- start start 2024-04-25T10:01:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a10180 0x5633609e9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a2de70 0x5633609e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336094ab90 con 0x5633609e69a0 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633608f3e40 con 0x563360a10180 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x563360946c30 con 0x563360a2de70 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdac3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdac3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38928/0 (socket says 172.21.15.19:38928) 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdac3b640 1 -- 172.21.15.19:0/2762415655 learned_addr learned my addr 172.21.15.19:0/2762415655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/2762415655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a2de70 0x5633609e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cda43a640 1 --2- 172.21.15.19:0/2762415655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a10180 0x5633609e9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdac3b640 1 -- 172.21.15.19:0/2762415655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a2de70 msgr2=0x5633609e6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdac3b640 1 --2- 172.21.15.19:0/2762415655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a2de70 0x5633609e6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdac3b640 1 -- 172.21.15.19:0/2762415655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a10180 msgr2=0x5633609e9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdac3b640 1 --2- 172.21.15.19:0/2762415655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a10180 0x5633609e9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.898+0000 7f5cdac3b640 1 -- 172.21.15.19:0/2762415655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633607297d0 con 0x5633609e69a0 2024-04-25T10:01:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.899+0000 7f5cdac3b640 1 --2- 172.21.15.19:0/2762415655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e3d90 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00ca00 tx=0x7f5ccc00cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e7fdeea003e065 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.899+0000 7f5cb7fff640 1 -- 172.21.15.19:0/2762415655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc018020 con 0x5633609e69a0 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.899+0000 7f5cb7fff640 1 -- 172.21.15.19:0/2762415655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc00ea20 con 0x5633609e69a0 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.899+0000 7f5cb7fff640 1 -- 172.21.15.19:0/2762415655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc00ee10 con 0x5633609e69a0 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.899+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/2762415655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 msgr2=0x5633609e3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.899+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/2762415655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e3d90 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00ca00 tx=0x7f5ccc00cec0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.899+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/2762415655 shutdown_connections 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.899+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/2762415655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e3d90 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.899+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/2762415655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a2de70 0x5633609e6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.899+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/2762415655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a10180 0x5633609e9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.899+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/2762415655 >> 172.21.15.19:0/2762415655 conn(0x5633609af870 msgr2=0x563360a2d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.900+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/2762415655 shutdown_connections 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.900+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/2762415655 wait complete. 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.900+0000 7f5cdf3a6740 1 -- start start 2024-04-25T10:01:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.900+0000 7f5cdf3a6740 1 Processor -- start 2024-04-25T10:01:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5633609e69a0 0x563360a12e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563360a10180 0x5633609e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a2de70 0x5633609e2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633609e7c50 con 0x563360a10180 2024-04-25T10:01:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633609073e0 con 0x563360a2de70 2024-04-25T10:01:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633609e7f30 con 0x5633609e69a0 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cdf3a6740 10 client.0 _ll_register_callbacks cb 0x563360906db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cd9c39640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563360a10180 0x5633609e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cd9c39640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563360a10180 0x5633609e1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38932/0 (socket says 172.21.15.19:38932) 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cd9c39640 1 -- 172.21.15.19:0/3123309335 learned_addr learned my addr 172.21.15.19:0/3123309335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cda43a640 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5633609e69a0 0x563360a12e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cdac3b640 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a2de70 0x5633609e2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.901+0000 7f5cd9c39640 1 -- 172.21.15.19:0/3123309335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5633609e69a0 msgr2=0x563360a12e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5633609e69a0 0x563360a12e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cd9c39640 1 -- 172.21.15.19:0/3123309335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a2de70 msgr2=0x5633609e2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a2de70 0x5633609e2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cd9c39640 1 -- 172.21.15.19:0/3123309335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56336061f0f0 con 0x563360a10180 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563360a10180 0x5633609e1ae0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f5cd406b3f0 tx=0x7f5cc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e914e85c92f64175 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cb57fa640 1 -- 172.21.15.19:0/3123309335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8028060 con 0x563360a10180 2024-04-25T10:01:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cb57fa640 1 -- 172.21.15.19:0/3123309335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5cc8014b10 con 0x563360a10180 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/3123309335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ccc00ea20 con 0x563360a10180 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cb57fa640 1 -- 172.21.15.19:0/3123309335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cc802e050 con 0x563360a10180 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cdf3a6740 10 client.9441 fetch_fsmap learned FSMap version 79 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/3123309335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x563360789f60 con 0x563360a10180 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cb57fa640 1 -- 172.21.15.19:0/3123309335 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc00ea20 con 0x563360a10180 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cb57fa640 1 -- 172.21.15.19:0/3123309335 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5cc8031020 con 0x563360a10180 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cdf3a6740 10 client.9441 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cdf3a6740 10 client.9441 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cdf3a6740 20 client.9441 populate_metadata read hostname 'smithi019' 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x563360a11890 0x563360a11ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cdf3a6740 4 client.9441 mds_command: new command op to 9213 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/3123309335 --> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633609580d0 con 0x563360a11890 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.902+0000 7f5cda43a640 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x563360a11890 0x563360a11ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.903+0000 7f5cb57fa640 10 client.9441 ms_handle_connect on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.903+0000 7f5cda43a640 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x563360a11890 0x563360a11ca0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf7758d8ab37bb1 server_cookie=d3d37e8974e65a04 in_seq=0 out_seq=0 2024-04-25T10:01:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.903+0000 7f5cb57fa640 1 -- 172.21.15.19:0/3123309335 <== mds.0 v2:172.21.15.19:6833/250977615 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5633609580d0 con 0x563360a11890 2024-04-25T10:01:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.903+0000 7f5cb57fa640 10 client.9441 handle_command_reply: tid=0 2024-04-25T10:01:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.903+0000 7f5cdf3a6740 1 client.9441 shutdown 2024-04-25T10:01:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.903+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/3123309335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563360a10180 msgr2=0x5633609e1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.903+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563360a10180 0x5633609e1ae0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f5cd406b3f0 tx=0x7f5cc8004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.904+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/3123309335 shutdown_connections 2024-04-25T10:01:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.904+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x563360a11890 0x563360a11ca0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.904+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a2de70 0x5633609e2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.904+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563360a10180 0x5633609e1ae0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.904+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/3123309335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5633609e69a0 0x563360a12e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.904+0000 7f5cb57fa640 0 client.9441 ms_handle_reset on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:50.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.904+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/3123309335 >> 172.21.15.19:0/3123309335 conn(0x5633609af870 msgr2=0x563360a315b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.904+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/3123309335 shutdown_connections 2024-04-25T10:01:50.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.904+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/3123309335 wait complete. 2024-04-25T10:01:50.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.904+0000 7f5cdf3a6740 20 client.9441 trim_cache size 0 max 16384 2024-04-25T10:01:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.918+0000 7f5cdf3a6740 1 Processor -- start 2024-04-25T10:01:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cdf3a6740 1 -- start start 2024-04-25T10:01:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a10180 0x563360a9aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a2de70 0x563360a9af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41fc1d0 con 0x5633609e69a0 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41fb3a0 con 0x563360a2de70 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40ab090 con 0x563360a10180 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cda43a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cda43a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38940/0 (socket says 172.21.15.19:38940) 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cda43a640 1 -- 172.21.15.19:0/114568558 learned_addr learned my addr 172.21.15.19:0/114568558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/114568558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a10180 0x563360a9aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.919+0000 7f5cdac3b640 1 --2- 172.21.15.19:0/114568558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a2de70 0x563360a9af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.920+0000 7f5cda43a640 1 -- 172.21.15.19:0/114568558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a10180 msgr2=0x563360a9aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.920+0000 7f5cda43a640 1 --2- 172.21.15.19:0/114568558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a10180 0x563360a9aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.920+0000 7f5cda43a640 1 -- 172.21.15.19:0/114568558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a2de70 msgr2=0x563360a9af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.920+0000 7f5cda43a640 1 --2- 172.21.15.19:0/114568558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a2de70 0x563360a9af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.920+0000 7f5cda43a640 1 -- 172.21.15.19:0/114568558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633605f81b0 con 0x5633609e69a0 2024-04-25T10:01:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.920+0000 7f5cda43a640 1 --2- 172.21.15.19:0/114568558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e1cd0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00a84d0 tx=0x7f5cd0040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8431a0b79d858f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.920+0000 7f5cb77fe640 1 -- 172.21.15.19:0/114568558 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0080e30 con 0x5633609e69a0 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.920+0000 7f5cb77fe640 1 -- 172.21.15.19:0/114568558 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd00767c0 con 0x5633609e69a0 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.920+0000 7f5cb77fe640 1 -- 172.21.15.19:0/114568558 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0074d90 con 0x5633609e69a0 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.920+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/114568558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 msgr2=0x5633609e1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.920+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/114568558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e1cd0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00a84d0 tx=0x7f5cd0040040 comp rx=0 tx=0).stop 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.921+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/114568558 shutdown_connections 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.921+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/114568558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a2de70 0x563360a9af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.921+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/114568558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a10180 0x563360a9aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.921+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/114568558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x5633609e1cd0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.921+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/114568558 >> 172.21.15.19:0/114568558 conn(0x5633609af870 msgr2=0x563360a23cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.921+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/114568558 shutdown_connections 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.921+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/114568558 wait complete. 2024-04-25T10:01:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.921+0000 7f5cdf3a6740 1 -- start start 2024-04-25T10:01:50.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.921+0000 7f5cdf3a6740 1 Processor -- start 2024-04-25T10:01:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x563360a4e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a10180 0x563360a9f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cdf3a6740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a2de70 0x563360a45f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633609559b0 con 0x5633609e69a0 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40b4420 con 0x563360a10180 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cdf3a6740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56336094d030 con 0x563360a2de70 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cdf3a6740 10 client.0 _ll_register_callbacks cb 0x7f5cc8009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cda43a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x563360a4e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cda43a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x563360a4e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38946/0 (socket says 172.21.15.19:38946) 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cda43a640 1 -- 172.21.15.19:0/769102297 learned_addr learned my addr 172.21.15.19:0/769102297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a10180 0x563360a9f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.922+0000 7f5cdac3b640 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a2de70 0x563360a45f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cda43a640 1 -- 172.21.15.19:0/769102297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a2de70 msgr2=0x563360a45f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cda43a640 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a2de70 0x563360a45f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cda43a640 1 -- 172.21.15.19:0/769102297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a10180 msgr2=0x563360a9f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cda43a640 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a10180 0x563360a9f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cda43a640 1 -- 172.21.15.19:0/769102297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563360a2f420 con 0x5633609e69a0 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a10180 0x563360a9f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cda43a640 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x563360a4e150 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00786b0 tx=0x7f5cd0039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d114b402881cb72d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cb57fa640 1 -- 172.21.15.19:0/769102297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00a6060 con 0x5633609e69a0 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cb57fa640 1 -- 172.21.15.19:0/769102297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5cd0090100 con 0x5633609e69a0 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/769102297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5cd00767c0 con 0x5633609e69a0 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cb57fa640 1 -- 172.21.15.19:0/769102297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cd0095050 con 0x5633609e69a0 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cdf3a6740 10 client.9447 fetch_fsmap learned FSMap version 79 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/769102297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x563360a123f0 con 0x5633609e69a0 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cb57fa640 1 -- 172.21.15.19:0/769102297 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd00767c0 con 0x5633609e69a0 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cb57fa640 1 -- 172.21.15.19:0/769102297 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5cd00a80b0 con 0x5633609e69a0 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cdf3a6740 10 client.9447 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:01:50.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cdf3a6740 10 client.9447 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cdf3a6740 20 client.9447 populate_metadata read hostname 'smithi019' 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x563360a0f3b0 0x563360a46480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cdf3a6740 4 client.9447 mds_command: new command op to 9213 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/769102297 --> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x563360952420 con 0x563360a0f3b0 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.923+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x563360a0f3b0 0x563360a46480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cb57fa640 10 client.9447 ms_handle_connect on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cd9c39640 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x563360a0f3b0 0x563360a46480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f67daf8291285232 server_cookie=af3a1a1b622a5f61 in_seq=0 out_seq=0 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cb57fa640 1 -- 172.21.15.19:0/769102297 <== mds.0 v2:172.21.15.19:6833/250977615 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x563360952420 con 0x563360a0f3b0 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cb57fa640 10 client.9447 handle_command_reply: tid=0 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 1 client.9447 shutdown 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/769102297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 msgr2=0x563360a4e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x563360a4e150 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00786b0 tx=0x7f5cd0039000 comp rx=0 tx=0).stop 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/769102297 shutdown_connections 2024-04-25T10:01:50.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x563360a0f3b0 0x563360a46480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563360a2de70 0x563360a45f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563360a10180 0x563360a9f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 1 --2- 172.21.15.19:0/769102297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633609e69a0 0x563360a4e150 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cb57fa640 0 client.9447 ms_handle_reset on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/769102297 >> 172.21.15.19:0/769102297 conn(0x5633609af870 msgr2=0x563360a11c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/769102297 shutdown_connections 2024-04-25T10:01:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 1 -- 172.21.15.19:0/769102297 wait complete. 2024-04-25T10:01:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.924+0000 7f5cdf3a6740 20 client.9447 trim_cache size 0 max 16384 2024-04-25T10:01:50.935 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:01:50.935 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T10:01:50.935 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:01:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 -- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5cb806b500 msgr2=0x7f5cb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5cb806b500 0x7f5cb806d9c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5cd414ed20 tx=0x7f5cc800b040 comp rx=0 tx=0).stop 2024-04-25T10:01:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 -- 172.21.15.19:0/883697049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd40cd4c0 msgr2=0x7f5cd414e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd40cd4c0 0x7f5cd414e7e0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5cd002b330 tx=0x7f5cd00046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 -- 172.21.15.19:0/883697049 shutdown_connections 2024-04-25T10:01:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5cb806b500 0x7f5cb806d9c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415e1a0 0x7f5cd40adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415b1c0 0x7f5cd40c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 --2- 172.21.15.19:0/883697049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd40cd4c0 0x7f5cd414e7e0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 -- 172.21.15.19:0/883697049 >> 172.21.15.19:0/883697049 conn(0x7f5cd406abe0 msgr2=0x7f5cd415d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 -- 172.21.15.19:0/883697049 shutdown_connections 2024-04-25T10:01:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:50.925+0000 7f5cdc69e640 1 -- 172.21.15.19:0/883697049 wait complete. 2024-04-25T10:01:50.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:50.940 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T10:01:51.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.038+0000 7f51a2328640 1 Processor -- start 2024-04-25T10:01:51.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.038+0000 7f51a2328640 1 -- start start 2024-04-25T10:01:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a2328640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f519c15b5a0 0x7f519c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a2328640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f519c15b980 0x7f519c0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a2328640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f519c162450 0x7f519c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a2328640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c0cf320 con 0x7f519c162450 2024-04-25T10:01:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a2328640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c0cf040 con 0x7f519c15b980 2024-04-25T10:01:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a2328640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c161970 con 0x7f519c15b5a0 2024-04-25T10:01:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a08c5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f519c162450 0x7f519c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a08c5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f519c162450 0x7f519c0ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38950/0 (socket says 172.21.15.19:38950) 2024-04-25T10:01:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a08c5640 1 -- 172.21.15.19:0/967832238 learned_addr learned my addr 172.21.15.19:0/967832238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f519bfff640 1 --2- 172.21.15.19:0/967832238 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f519c15b5a0 0x7f519c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f519b7fe640 1 --2- 172.21.15.19:0/967832238 >> v2:172.21.15.112:3300/0 conn(0x7f519c15b980 0x7f519c0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a08c5640 1 -- 172.21.15.19:0/967832238 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f519c15b5a0 msgr2=0x7f519c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a08c5640 1 --2- 172.21.15.19:0/967832238 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f519c15b5a0 0x7f519c0b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a08c5640 1 -- 172.21.15.19:0/967832238 >> v2:172.21.15.112:3300/0 conn(0x7f519c15b980 msgr2=0x7f519c0b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a08c5640 1 --2- 172.21.15.19:0/967832238 >> v2:172.21.15.112:3300/0 conn(0x7f519c15b980 0x7f519c0b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f51a08c5640 1 -- 172.21.15.19:0/967832238 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c161c50 con 0x7f519c162450 2024-04-25T10:01:51.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.039+0000 7f519b7fe640 1 --2- 172.21.15.19:0/967832238 >> v2:172.21.15.112:3300/0 conn(0x7f519c15b980 0x7f519c0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:51.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.040+0000 7f51a08c5640 1 --2- 172.21.15.19:0/967832238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f519c162450 0x7f519c0ce310 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f519c06d330 tx=0x7f519402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bc409ce77c63f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.040+0000 7f519affd640 1 -- 172.21.15.19:0/967832238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194036070 con 0x7f519c162450 2024-04-25T10:01:51.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.040+0000 7f519affd640 1 -- 172.21.15.19:0/967832238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5194031600 con 0x7f519c162450 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.040+0000 7f519affd640 1 -- 172.21.15.19:0/967832238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940319d0 con 0x7f519c162450 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.041+0000 7f51a2328640 1 -- 172.21.15.19:0/967832238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f519c162450 msgr2=0x7f519c0ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.041+0000 7f51a2328640 1 --2- 172.21.15.19:0/967832238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f519c162450 0x7f519c0ce310 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f519c06d330 tx=0x7f519402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.041+0000 7f51a2328640 1 -- 172.21.15.19:0/967832238 shutdown_connections 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.041+0000 7f51a2328640 1 --2- 172.21.15.19:0/967832238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f519c162450 0x7f519c0ce310 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.041+0000 7f51a2328640 1 --2- 172.21.15.19:0/967832238 >> v2:172.21.15.112:3300/0 conn(0x7f519c15b980 0x7f519c0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.041+0000 7f51a2328640 1 --2- 172.21.15.19:0/967832238 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f519c15b5a0 0x7f519c0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.041+0000 7f51a2328640 1 -- 172.21.15.19:0/967832238 >> 172.21.15.19:0/967832238 conn(0x7f519c0d0530 msgr2=0x7f519c0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.041+0000 7f51a2328640 1 -- 172.21.15.19:0/967832238 shutdown_connections 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.041+0000 7f51a2328640 1 -- 172.21.15.19:0/967832238 wait complete. 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.042+0000 7f51a2328640 1 Processor -- start 2024-04-25T10:01:51.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.042+0000 7f51a2328640 1 -- start start 2024-04-25T10:01:51.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f51a2328640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f519c15b5a0 0x7f519c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f51a2328640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f519c15b980 0x7f519c146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f51a2328640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f519c162450 0x7f519c15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f51a2328640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0ce9d0 con 0x7f519c15b5a0 2024-04-25T10:01:51.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f51a2328640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0ce850 con 0x7f519c162450 2024-04-25T10:01:51.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f51a2328640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0ceb50 con 0x7f519c15b980 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f519bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f519c15b5a0 0x7f519c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f519bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f519c15b5a0 0x7f519c1579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38962/0 (socket says 172.21.15.19:38962) 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f519bfff640 1 -- 172.21.15.19:0/1483160124 learned_addr learned my addr 172.21.15.19:0/1483160124 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f51a08c5640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f519c162450 0x7f519c15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f519b7fe640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f519c15b980 0x7f519c146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f519bfff640 1 -- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f519c15b980 msgr2=0x7f519c146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f519bfff640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f519c15b980 0x7f519c146b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f519bfff640 1 -- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f519c162450 msgr2=0x7f519c15a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f519bfff640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f519c162450 0x7f519c15a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f519bfff640 1 -- 172.21.15.19:0/1483160124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c0c8900 con 0x7f519c15b5a0 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.043+0000 7f519bfff640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f519c15b5a0 0x7f519c1579a0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f518400d830 tx=0x7f518400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.044+0000 7f5198ff9640 1 -- 172.21.15.19:0/1483160124 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184018030 con 0x7f519c15b5a0 2024-04-25T10:01:51.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.044+0000 7f5198ff9640 1 -- 172.21.15.19:0/1483160124 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518400be10 con 0x7f519c15b5a0 2024-04-25T10:01:51.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.044+0000 7f51a2328640 1 -- 172.21.15.19:0/1483160124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c0c8be0 con 0x7f519c15b5a0 2024-04-25T10:01:51.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.044+0000 7f5198ff9640 1 -- 172.21.15.19:0/1483160124 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51840098b0 con 0x7f519c15b5a0 2024-04-25T10:01:51.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.044+0000 7f51a2328640 1 -- 172.21.15.19:0/1483160124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c157340 con 0x7f519c15b5a0 2024-04-25T10:01:51.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.045+0000 7f5198ff9640 1 -- 172.21.15.19:0/1483160124 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f518401b070 con 0x7f519c15b5a0 2024-04-25T10:01:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.046+0000 7f5198ff9640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f517c06b4d0 0x7f517c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.046+0000 7f519b7fe640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f517c06b4d0 0x7f517c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.046+0000 7f5198ff9640 1 -- 172.21.15.19:0/1483160124 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f518408a340 con 0x7f519c15b5a0 2024-04-25T10:01:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.046+0000 7f519b7fe640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f517c06b4d0 0x7f517c06d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f519c1f6f40 tx=0x7f518c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.047+0000 7f51a5020740 1 Processor -- start 2024-04-25T10:01:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.047+0000 7f51a5020740 1 -- start start 2024-04-25T10:01:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.047+0000 7f51a5020740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 0x562360407320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.047+0000 7f51a5020740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360407860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.047+0000 7f51a5020740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603d7770 0x5623603daf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.047+0000 7f51a5020740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x562360340da0 con 0x562360404ef0 2024-04-25T10:01:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.047+0000 7f51a5020740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623602e9b80 con 0x5623603d7770 2024-04-25T10:01:51.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.047+0000 7f51a5020740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56236033ce40 con 0x562360404b10 2024-04-25T10:01:51.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360407860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360407860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38966/0 (socket says 172.21.15.19:38966) 2024-04-25T10:01:51.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f519b7fe640 1 -- 172.21.15.19:0/3620497996 learned_addr learned my addr 172.21.15.19:0/3620497996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:51.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f519bfff640 1 --2- 172.21.15.19:0/3620497996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 0x562360407320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f51a08c5640 1 --2- 172.21.15.19:0/3620497996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603d7770 0x5623603daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f519b7fe640 1 -- 172.21.15.19:0/3620497996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 msgr2=0x562360407320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f519b7fe640 1 --2- 172.21.15.19:0/3620497996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 0x562360407320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f519b7fe640 1 -- 172.21.15.19:0/3620497996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603d7770 msgr2=0x5623603daf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f519b7fe640 1 --2- 172.21.15.19:0/3620497996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603d7770 0x5623603daf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f519b7fe640 1 -- 172.21.15.19:0/3620497996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623602f3f10 con 0x562360404ef0 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f519b7fe640 1 --2- 172.21.15.19:0/3620497996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360407860 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f518c005b40 tx=0x7f518c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35366edd3c781e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f5191ffb640 1 -- 172.21.15.19:0/3620497996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c00f040 con 0x562360404ef0 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f5191ffb640 1 -- 172.21.15.19:0/3620497996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c026070 con 0x562360404ef0 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.048+0000 7f5191ffb640 1 -- 172.21.15.19:0/3620497996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c006450 con 0x562360404ef0 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.049+0000 7f51a5020740 1 -- 172.21.15.19:0/3620497996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 msgr2=0x562360407860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.049+0000 7f51a5020740 1 --2- 172.21.15.19:0/3620497996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360407860 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f518c005b40 tx=0x7f518c014000 comp rx=0 tx=0).stop 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.049+0000 7f51a5020740 1 -- 172.21.15.19:0/3620497996 shutdown_connections 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.049+0000 7f51a5020740 1 --2- 172.21.15.19:0/3620497996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603d7770 0x5623603daf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.049+0000 7f51a5020740 1 --2- 172.21.15.19:0/3620497996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360407860 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.049+0000 7f51a5020740 1 --2- 172.21.15.19:0/3620497996 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 0x562360407320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.049+0000 7f51a5020740 1 -- 172.21.15.19:0/3620497996 >> 172.21.15.19:0/3620497996 conn(0x5623603d99e0 msgr2=0x562360422320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.049+0000 7f51a5020740 1 -- 172.21.15.19:0/3620497996 shutdown_connections 2024-04-25T10:01:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.049+0000 7f51a5020740 1 -- 172.21.15.19:0/3620497996 wait complete. 2024-04-25T10:01:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.049+0000 7f51a5020740 1 -- start start 2024-04-25T10:01:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f51a5020740 1 Processor -- start 2024-04-25T10:01:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f51a5020740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 0x562360425dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f51a5020740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f51a5020740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603fbde0 0x5623603fa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f51a5020740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623603dcb90 con 0x562360404ef0 2024-04-25T10:01:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f51a5020740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623602df4b0 con 0x5623603fbde0 2024-04-25T10:01:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f51a5020740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623603dce70 con 0x562360404b10 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f51a5020740 10 client.0 _ll_register_callbacks cb 0x56236031f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38974/0 (socket says 172.21.15.19:38974) 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f519b7fe640 1 -- 172.21.15.19:0/349182411 learned_addr learned my addr 172.21.15.19:0/349182411 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f51a08c5640 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603fbde0 0x5623603fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.050+0000 7f519bfff640 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 0x562360425dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f519b7fe640 1 -- 172.21.15.19:0/349182411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 msgr2=0x562360425dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f519b7fe640 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 0x562360425dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f519b7fe640 1 -- 172.21.15.19:0/349182411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603fbde0 msgr2=0x5623603fa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f519b7fe640 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603fbde0 0x5623603fa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f519b7fe640 1 -- 172.21.15.19:0/349182411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56236011f7d0 con 0x562360404ef0 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f519b7fe640 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603f9d30 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x5623603df070 tx=0x7f518c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cdd890d7554495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f517affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f517affd640 1 -- 172.21.15.19:0/349182411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c003a20 con 0x562360404ef0 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f517affd640 1 -- 172.21.15.19:0/349182411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f518c003bc0 con 0x562360404ef0 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f51a5020740 1 -- 172.21.15.19:0/349182411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f518c026070 con 0x562360404ef0 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f517affd640 1 -- 172.21.15.19:0/349182411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f518c003dc0 con 0x562360404ef0 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f51a5020740 10 client.9459 fetch_fsmap learned FSMap version 79 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f51a5020740 1 -- 172.21.15.19:0/349182411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5623600553f0 con 0x562360404ef0 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f517affd640 1 -- 172.21.15.19:0/349182411 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c026070 con 0x562360404ef0 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f517affd640 1 -- 172.21.15.19:0/349182411 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f518c03f080 con 0x562360404ef0 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f51a5020740 10 client.9459 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.051+0000 7f51a5020740 10 client.9459 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.052+0000 7f51a5020740 20 client.9459 populate_metadata read hostname 'smithi019' 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.052+0000 7f51a5020740 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5623603fad50 0x5623603fb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.052+0000 7f51a5020740 4 client.9459 mds_command: new command op to 9213 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.052+0000 7f51a5020740 1 -- 172.21.15.19:0/349182411 --> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56236034e2e0 con 0x5623603fad50 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.052+0000 7f519bfff640 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5623603fad50 0x5623603fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.052+0000 7f517affd640 10 client.9459 ms_handle_connect on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.052+0000 7f519bfff640 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5623603fad50 0x5623603fb130 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bcfe2c02b21d464 server_cookie=12b8d3fbfde78fb3 in_seq=0 out_seq=0 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.052+0000 7f517affd640 1 -- 172.21.15.19:0/349182411 <== mds.0 v2:172.21.15.19:6833/250977615 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56236034e2e0 con 0x5623603fad50 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.052+0000 7f517affd640 10 client.9459 handle_command_reply: tid=0 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.052+0000 7f51a5020740 1 client.9459 shutdown 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f51a5020740 1 -- 172.21.15.19:0/349182411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 msgr2=0x5623603f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f51a5020740 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603f9d30 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x5623603df070 tx=0x7f518c004730 comp rx=0 tx=0).stop 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f51a5020740 1 -- 172.21.15.19:0/349182411 shutdown_connections 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f51a5020740 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x5623603fad50 0x5623603fb130 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f51a5020740 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603fbde0 0x5623603fa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f51a5020740 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603f9d30 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f51a5020740 1 --2- 172.21.15.19:0/349182411 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 0x562360425dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f517affd640 0 client.9459 ms_handle_reset on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f51a5020740 1 -- 172.21.15.19:0/349182411 >> 172.21.15.19:0/349182411 conn(0x5623603d7770 msgr2=0x562360403d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f51a5020740 1 -- 172.21.15.19:0/349182411 shutdown_connections 2024-04-25T10:01:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f51a5020740 1 -- 172.21.15.19:0/349182411 wait complete. 2024-04-25T10:01:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.053+0000 7f51a5020740 20 client.9459 trim_cache size 0 max 16384 2024-04-25T10:01:51.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.068+0000 7f51a5020740 1 Processor -- start 2024-04-25T10:01:51.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.068+0000 7f51a5020740 1 -- start start 2024-04-25T10:01:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a5020740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5623603fbde0 0x562360403dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a5020740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562360404b10 0x5623603d6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a5020740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a5020740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c2005e0 con 0x562360404ef0 2024-04-25T10:01:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a5020740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c1ff7e0 con 0x562360404b10 2024-04-25T10:01:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a5020740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c149550 con 0x5623603fbde0 2024-04-25T10:01:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a08c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603d6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a08c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603d6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38976/0 (socket says 172.21.15.19:38976) 2024-04-25T10:01:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a08c5640 1 -- 172.21.15.19:0/886670647 learned_addr learned my addr 172.21.15.19:0/886670647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f519b7fe640 1 --2- 172.21.15.19:0/886670647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562360404b10 0x5623603d6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f519bfff640 1 --2- 172.21.15.19:0/886670647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5623603fbde0 0x562360403dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a08c5640 1 -- 172.21.15.19:0/886670647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5623603fbde0 msgr2=0x562360403dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a08c5640 1 --2- 172.21.15.19:0/886670647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5623603fbde0 0x562360403dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a08c5640 1 -- 172.21.15.19:0/886670647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562360404b10 msgr2=0x5623603d6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a08c5640 1 --2- 172.21.15.19:0/886670647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562360404b10 0x5623603d6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a08c5640 1 -- 172.21.15.19:0/886670647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623600150f0 con 0x562360404ef0 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f519b7fe640 1 --2- 172.21.15.19:0/886670647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562360404b10 0x5623603d6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51a08c5640 1 --2- 172.21.15.19:0/886670647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603d6b40 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f519402ac90 tx=0x7f519402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f50ee855775a1a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51917fa640 1 -- 172.21.15.19:0/886670647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194036040 con 0x562360404ef0 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.069+0000 7f51917fa640 1 -- 172.21.15.19:0/886670647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5194037040 con 0x562360404ef0 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.070+0000 7f51917fa640 1 -- 172.21.15.19:0/886670647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194031a90 con 0x562360404ef0 2024-04-25T10:01:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.070+0000 7f51a5020740 1 -- 172.21.15.19:0/886670647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 msgr2=0x5623603d6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.070+0000 7f51a5020740 1 --2- 172.21.15.19:0/886670647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603d6b40 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f519402ac90 tx=0x7f519402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.070+0000 7f51a5020740 1 -- 172.21.15.19:0/886670647 shutdown_connections 2024-04-25T10:01:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.070+0000 7f51a5020740 1 --2- 172.21.15.19:0/886670647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x5623603d6b40 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.070+0000 7f51a5020740 1 --2- 172.21.15.19:0/886670647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562360404b10 0x5623603d6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.070+0000 7f51a5020740 1 --2- 172.21.15.19:0/886670647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5623603fbde0 0x562360403dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.070+0000 7f51a5020740 1 -- 172.21.15.19:0/886670647 >> 172.21.15.19:0/886670647 conn(0x5623603d7770 msgr2=0x562360422320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.070+0000 7f51a5020740 1 -- 172.21.15.19:0/886670647 shutdown_connections 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.070+0000 7f51a5020740 1 -- 172.21.15.19:0/886670647 wait complete. 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.071+0000 7f51a5020740 1 -- start start 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.071+0000 7f51a5020740 1 Processor -- start 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.071+0000 7f51a5020740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603fbde0 0x5623603d6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.071+0000 7f51a5020740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 0x5623603d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.071+0000 7f51a5020740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360417cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.071+0000 7f51a5020740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623603d8e90 con 0x562360404ef0 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.071+0000 7f51a5020740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c1ff230 con 0x5623603fbde0 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.071+0000 7f51a5020740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x562360423710 con 0x562360404b10 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a5020740 10 client.0 _ll_register_callbacks cb 0x7f518c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a08c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360417cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a08c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360417cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38992/0 (socket says 172.21.15.19:38992) 2024-04-25T10:01:51.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a08c5640 1 -- 172.21.15.19:0/1932836796 learned_addr learned my addr 172.21.15.19:0/1932836796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:51.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a08c5640 1 -- 172.21.15.19:0/1932836796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 msgr2=0x5623603d6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a08c5640 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 0x5623603d6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a08c5640 1 -- 172.21.15.19:0/1932836796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603fbde0 msgr2=0x5623603d6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f519bfff640 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603fbde0 0x5623603d6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a08c5640 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603fbde0 0x5623603d6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a08c5640 1 -- 172.21.15.19:0/1932836796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623604076a0 con 0x562360404ef0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f519bfff640 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603fbde0 0x5623603d6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a08c5640 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360417cb0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f5194045000 tx=0x7f5194004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cac5749bab6cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f517affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f517affd640 1 -- 172.21.15.19:0/1932836796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194036070 con 0x562360404ef0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f517affd640 1 -- 172.21.15.19:0/1932836796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5194031650 con 0x562360404ef0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a5020740 1 -- 172.21.15.19:0/1932836796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5194037040 con 0x562360404ef0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f517affd640 1 -- 172.21.15.19:0/1932836796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5194031850 con 0x562360404ef0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a5020740 10 client.9465 fetch_fsmap learned FSMap version 79 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.072+0000 7f51a5020740 1 -- 172.21.15.19:0/1932836796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5623603f83f0 con 0x562360404ef0 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f517affd640 1 -- 172.21.15.19:0/1932836796 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5194037040 con 0x562360404ef0 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f517affd640 1 -- 172.21.15.19:0/1932836796 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f519403c040 con 0x562360404ef0 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 10 client.9465 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 10 client.9465 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 20 client.9465 populate_metadata read hostname 'smithi019' 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x56236048c340 0x5623604181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 4 client.9465 mds_command: new command op to 9213 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 1 -- 172.21.15.19:0/1932836796 --> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562360348630 con 0x56236048c340 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f519bfff640 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x56236048c340 0x5623604181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f517affd640 10 client.9465 ms_handle_connect on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f519bfff640 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x56236048c340 0x5623604181f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff4139242925456 server_cookie=90b31beafb11b2e6 in_seq=0 out_seq=0 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f517affd640 1 -- 172.21.15.19:0/1932836796 <== mds.0 v2:172.21.15.19:6833/250977615 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x562360348630 con 0x56236048c340 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f517affd640 10 client.9465 handle_command_reply: tid=0 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 1 client.9465 shutdown 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 1 -- 172.21.15.19:0/1932836796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 msgr2=0x562360417cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360417cb0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f5194045000 tx=0x7f5194004ab0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 1 -- 172.21.15.19:0/1932836796 shutdown_connections 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.19:6833/250977615,v1:172.21.15.19:6835/250977615] conn(0x56236048c340 0x5623604181f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562360404ef0 0x562360417cb0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x562360404b10 0x5623603d6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f517affd640 0 client.9465 ms_handle_reset on v2:172.21.15.19:6833/250977615 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 1 --2- 172.21.15.19:0/1932836796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5623603fbde0 0x5623603d6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.073+0000 7f51a5020740 1 -- 172.21.15.19:0/1932836796 >> 172.21.15.19:0/1932836796 conn(0x5623603d7770 msgr2=0x5623603d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a5020740 1 -- 172.21.15.19:0/1932836796 shutdown_connections 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a5020740 1 -- 172.21.15.19:0/1932836796 wait complete. 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a5020740 20 client.9465 trim_cache size 0 max 16384 2024-04-25T10:01:51.086 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:01:51.086 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:01:51.086 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:01:51.086 INFO:teuthology.orchestra.run.smithi019.stdout: "2e7d34d9-08f9-4095-95bb-ebac9cd01916": { 2024-04-25T10:01:51.086 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:01:51.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "2e7d34d9-08f9-4095-95bb-ebac9cd01916", 2024-04-25T10:01:51.086 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive" 2024-04-25T10:01:51.086 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:51.086 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:01:51.086 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:01:51.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 -- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f517c06b4d0 msgr2=0x7f517c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f517c06b4d0 0x7f517c06d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f519c1f6f40 tx=0x7f518c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:51.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 -- 172.21.15.19:0/1483160124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f519c15b5a0 msgr2=0x7f519c1579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f519c15b5a0 0x7f519c1579a0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f518400d830 tx=0x7f518400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 -- 172.21.15.19:0/1483160124 shutdown_connections 2024-04-25T10:01:51.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f517c06b4d0 0x7f517c06d990 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f519c162450 0x7f519c15a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f519c15b980 0x7f519c146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 --2- 172.21.15.19:0/1483160124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f519c15b5a0 0x7f519c1579a0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 -- 172.21.15.19:0/1483160124 >> 172.21.15.19:0/1483160124 conn(0x7f519c0d0530 msgr2=0x7f519c0b7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 -- 172.21.15.19:0/1483160124 shutdown_connections 2024-04-25T10:01:51.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.074+0000 7f51a2328640 1 -- 172.21.15.19:0/1483160124 wait complete. 2024-04-25T10:01:51.089 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:51.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.185+0000 7f3a4c968640 1 Processor -- start 2024-04-25T10:01:51.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.185+0000 7f3a4c968640 1 -- start start 2024-04-25T10:01:51.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a4c968640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3a4815dd30 0x7f3a480b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a4c968640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3a480b9770 0x7f3a480d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a4c968640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a480d1df0 con 0x7f3a480d2e90 2024-04-25T10:01:51.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a4c968640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a480b7a50 con 0x7f3a480b9770 2024-04-25T10:01:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a4c968640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a480d20d0 con 0x7f3a4815dd30 2024-04-25T10:01:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a45d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3a480b9770 0x7f3a480d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a45d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3a480b9770 0x7f3a480d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39002/0 (socket says 172.21.15.19:39002) 2024-04-25T10:01:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a45d9b640 1 -- 172.21.15.19:0/1406110923 learned_addr learned my addr 172.21.15.19:0/1406110923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a46d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3a480d2e90 0x7f3a480b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39812/0 (socket says 172.21.15.19:39812) 2024-04-25T10:01:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a4659c640 1 --2- 172.21.15.19:0/1406110923 >> v2:172.21.15.112:3300/0 conn(0x7f3a4815dd30 0x7f3a480b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a45d9b640 1 -- 172.21.15.19:0/1406110923 >> v2:172.21.15.112:3300/0 conn(0x7f3a4815dd30 msgr2=0x7f3a480b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a45d9b640 1 --2- 172.21.15.19:0/1406110923 >> v2:172.21.15.112:3300/0 conn(0x7f3a4815dd30 0x7f3a480b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a45d9b640 1 -- 172.21.15.19:0/1406110923 >> v1:172.21.15.19:6789/0 conn(0x7f3a480d2e90 legacy=0x7f3a480b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.186+0000 7f3a45d9b640 1 -- 172.21.15.19:0/1406110923 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a480d23b0 con 0x7f3a480b9770 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.187+0000 7f3a45d9b640 1 --2- 172.21.15.19:0/1406110923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9770 0x7f3a480d03c0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f3a4806d000 tx=0x7f3a3c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5175352286135da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.187+0000 7f3a4559a640 1 -- 172.21.15.19:0/1406110923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c004030 con 0x7f3a480b9770 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.187+0000 7f3a4559a640 1 -- 172.21.15.19:0/1406110923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c0041d0 con 0x7f3a480b9770 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.187+0000 7f3a4559a640 1 -- 172.21.15.19:0/1406110923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c003610 con 0x7f3a480b9770 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.188+0000 7f3a4c968640 1 -- 172.21.15.19:0/1406110923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9770 msgr2=0x7f3a480d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.188+0000 7f3a4c968640 1 --2- 172.21.15.19:0/1406110923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9770 0x7f3a480d03c0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f3a4806d000 tx=0x7f3a3c02a130 comp rx=0 tx=0).stop 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.188+0000 7f3a4c968640 1 -- 172.21.15.19:0/1406110923 shutdown_connections 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.188+0000 7f3a4c968640 1 --2- 172.21.15.19:0/1406110923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9770 0x7f3a480d03c0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.188+0000 7f3a4c968640 1 --2- 172.21.15.19:0/1406110923 >> v2:172.21.15.112:3300/0 conn(0x7f3a4815dd30 0x7f3a480b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.188+0000 7f3a4c968640 1 -- 172.21.15.19:0/1406110923 >> 172.21.15.19:0/1406110923 conn(0x7f3a48061c30 msgr2=0x7f3a4806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.188+0000 7f3a4c968640 1 -- 172.21.15.19:0/1406110923 shutdown_connections 2024-04-25T10:01:51.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.188+0000 7f3a4c968640 1 -- 172.21.15.19:0/1406110923 wait complete. 2024-04-25T10:01:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.189+0000 7f3a4c968640 1 Processor -- start 2024-04-25T10:01:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.189+0000 7f3a4c968640 1 -- start start 2024-04-25T10:01:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4c968640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9770 0x7f3a48160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4c968640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a480d2e90 0x7f3a4814fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4c968640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3a4815dd30 0x7f3a480cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4c968640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480d1a10 con 0x7f3a480b9770 2024-04-25T10:01:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4c968640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480d2970 con 0x7f3a480d2e90 2024-04-25T10:01:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4c968640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480d1b90 con 0x7f3a4815dd30 2024-04-25T10:01:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9770 0x7f3a48160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9770 0x7f3a48160c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39016/0 (socket says 172.21.15.19:39016) 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4659c640 1 -- 172.21.15.19:0/3085710764 learned_addr learned my addr 172.21.15.19:0/3085710764 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a45d9b640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a480d2e90 0x7f3a4814fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a46d9d640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3a4815dd30 0x7f3a480cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4659c640 1 -- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3a4815dd30 msgr2=0x7f3a480cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4659c640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3a4815dd30 0x7f3a480cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4659c640 1 -- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a480d2e90 msgr2=0x7f3a4814fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.190+0000 7f3a4659c640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a480d2e90 0x7f3a4814fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.191+0000 7f3a4659c640 1 -- 172.21.15.19:0/3085710764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a481f6f90 con 0x7f3a480b9770 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.191+0000 7f3a45d9b640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a480d2e90 0x7f3a4814fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.191+0000 7f3a4659c640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9770 0x7f3a48160c50 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800d5f0 tx=0x7f3a3800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.191+0000 7f3a36ffd640 1 -- 172.21.15.19:0/3085710764 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a380125f0 con 0x7f3a480b9770 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.191+0000 7f3a36ffd640 1 -- 172.21.15.19:0/3085710764 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a38019070 con 0x7f3a480b9770 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.191+0000 7f3a36ffd640 1 -- 172.21.15.19:0/3085710764 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a380129d0 con 0x7f3a480b9770 2024-04-25T10:01:51.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.191+0000 7f3a4c968640 1 -- 172.21.15.19:0/3085710764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a481f7270 con 0x7f3a480b9770 2024-04-25T10:01:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.191+0000 7f3a4c968640 1 -- 172.21.15.19:0/3085710764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a481f7830 con 0x7f3a480b9770 2024-04-25T10:01:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.192+0000 7f3a4c968640 1 -- 172.21.15.19:0/3085710764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a48073990 con 0x7f3a480b9770 2024-04-25T10:01:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.192+0000 7f3a36ffd640 1 -- 172.21.15.19:0/3085710764 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a38012b70 con 0x7f3a480b9770 2024-04-25T10:01:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.193+0000 7f3a36ffd640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a2806b4d0 0x7f3a2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.193+0000 7f3a36ffd640 1 -- 172.21.15.19:0/3085710764 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a3808a840 con 0x7f3a480b9770 2024-04-25T10:01:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.193+0000 7f3a45d9b640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a2806b4d0 0x7f3a2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.194+0000 7f3a45d9b640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a2806b4d0 0x7f3a2806d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c033000 tx=0x7f3a3c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.197+0000 7f3a36ffd640 1 -- 172.21.15.19:0/3085710764 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a38059490 con 0x7f3a480b9770 2024-04-25T10:01:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.412+0000 7f3a4c968640 1 -- 172.21.15.19:0/3085710764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a480c7b60 con 0x7f3a480b9770 2024-04-25T10:01:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.413+0000 7f3a36ffd640 1 -- 172.21.15.19:0/3085710764 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3a38058d90 con 0x7f3a480b9770 2024-04-25T10:01:51.417 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:51.418 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:01:30.999245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/250977615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":250977615},{"type":"v1","addr":"172.21.15.19:6835","nonce":250977615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:01:51.418 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 79 2024-04-25T10:01:51.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 -- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a2806b4d0 msgr2=0x7f3a2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a2806b4d0 0x7f3a2806d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c033000 tx=0x7f3a3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 -- 172.21.15.19:0/3085710764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9770 msgr2=0x7f3a48160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9770 0x7f3a48160c50 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800d5f0 tx=0x7f3a3800dab0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 -- 172.21.15.19:0/3085710764 shutdown_connections 2024-04-25T10:01:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3a2806b4d0 0x7f3a2806d990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3a4815dd30 0x7f3a480cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a480d2e90 0x7f3a4814fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 --2- 172.21.15.19:0/3085710764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9770 0x7f3a48160c50 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 -- 172.21.15.19:0/3085710764 >> 172.21.15.19:0/3085710764 conn(0x7f3a48061c30 msgr2=0x7f3a480c8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 -- 172.21.15.19:0/3085710764 shutdown_connections 2024-04-25T10:01:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.415+0000 7f3a4c968640 1 -- 172.21.15.19:0/3085710764 wait complete. 2024-04-25T10:01:51.430 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:01:51.430 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.526+0000 7fbcfabb0640 1 Processor -- start 2024-04-25T10:01:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.526+0000 7fbcfabb0640 1 -- start start 2024-04-25T10:01:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcfabb0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbcf415dd30 0x7fbcf40b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcfabb0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbcf40d0ad0 0x7fbcf40b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcfabb0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf40d1df0 con 0x7fbcf40b9770 2024-04-25T10:01:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcfabb0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf40d20d0 con 0x7fbcf415dd30 2024-04-25T10:01:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcfabb0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf40d23b0 con 0x7fbcf40d0ad0 2024-04-25T10:01:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcf894c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbcf415dd30 0x7fbcf40b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcebfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbcf40b9770 0x7fbcf40d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39828/0 (socket says 172.21.15.19:39828) 2024-04-25T10:01:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcebfff640 1 -- 172.21.15.19:0/1611969756 learned_addr learned my addr 172.21.15.19:0/1611969756 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcf914d640 1 --2- 172.21.15.19:0/1611969756 >> v2:172.21.15.112:3300/0 conn(0x7fbcf40d0ad0 0x7fbcf40b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcf894c640 1 -- 172.21.15.19:0/1611969756 >> v2:172.21.15.112:3300/0 conn(0x7fbcf40d0ad0 msgr2=0x7fbcf40b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcf894c640 1 --2- 172.21.15.19:0/1611969756 >> v2:172.21.15.112:3300/0 conn(0x7fbcf40d0ad0 0x7fbcf40b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcf894c640 1 -- 172.21.15.19:0/1611969756 >> v1:172.21.15.19:6789/0 conn(0x7fbcf40b9770 legacy=0x7fbcf40d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcf894c640 1 -- 172.21.15.19:0/1611969756 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf40d2690 con 0x7fbcf415dd30 2024-04-25T10:01:51.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbceb7fe640 1 -- 172.21.15.19:0/1611969756 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399948104 0 0) 0x7fbcf40d1df0 con 0x7fbcf40b9770 2024-04-25T10:01:51.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.527+0000 7fbcf914d640 1 --2- 172.21.15.19:0/1611969756 >> v2:172.21.15.112:3300/0 conn(0x7fbcf40d0ad0 0x7fbcf40b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:51.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.528+0000 7fbcf894c640 1 --2- 172.21.15.19:0/1611969756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcf415dd30 0x7fbcf40b9230 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40ce200 tx=0x7fbcf002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30a0f7226d335ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.528+0000 7fbceb7fe640 1 -- 172.21.15.19:0/1611969756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0037070 con 0x7fbcf415dd30 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.528+0000 7fbceb7fe640 1 -- 172.21.15.19:0/1611969756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf002add0 con 0x7fbcf415dd30 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.528+0000 7fbceb7fe640 1 -- 172.21.15.19:0/1611969756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0032660 con 0x7fbcf415dd30 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.529+0000 7fbcfabb0640 1 -- 172.21.15.19:0/1611969756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcf415dd30 msgr2=0x7fbcf40b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.529+0000 7fbcfabb0640 1 --2- 172.21.15.19:0/1611969756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcf415dd30 0x7fbcf40b9230 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40ce200 tx=0x7fbcf002a080 comp rx=0 tx=0).stop 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.529+0000 7fbcfabb0640 1 -- 172.21.15.19:0/1611969756 shutdown_connections 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.529+0000 7fbcfabb0640 1 --2- 172.21.15.19:0/1611969756 >> v2:172.21.15.112:3300/0 conn(0x7fbcf40d0ad0 0x7fbcf40b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.529+0000 7fbcfabb0640 1 --2- 172.21.15.19:0/1611969756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcf415dd30 0x7fbcf40b9230 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.529+0000 7fbcfabb0640 1 -- 172.21.15.19:0/1611969756 >> 172.21.15.19:0/1611969756 conn(0x7fbcf4061c30 msgr2=0x7fbcf406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.529+0000 7fbcfabb0640 1 -- 172.21.15.19:0/1611969756 shutdown_connections 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.529+0000 7fbcfabb0640 1 -- 172.21.15.19:0/1611969756 wait complete. 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.530+0000 7fbcfabb0640 1 Processor -- start 2024-04-25T10:01:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.530+0000 7fbcfabb0640 1 -- start start 2024-04-25T10:01:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcfabb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcf40b9770 0x7fbcf41f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcfabb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcf40d0ad0 0x7fbcf4159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcfabb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf415dd30 0x7fbcf4159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcfabb0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf40d1b90 con 0x7fbcf40b9770 2024-04-25T10:01:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcfabb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf40d1a10 con 0x7fbcf415dd30 2024-04-25T10:01:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcfabb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf40d2e90 con 0x7fbcf40d0ad0 2024-04-25T10:01:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcf894c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcf40b9770 0x7fbcf41f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcf894c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcf40b9770 0x7fbcf41f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39034/0 (socket says 172.21.15.19:39034) 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcf894c640 1 -- 172.21.15.19:0/585306516 learned_addr learned my addr 172.21.15.19:0/585306516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcf914d640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf415dd30 0x7fbcf4159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcebfff640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcf40d0ad0 0x7fbcf4159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcf894c640 1 -- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcf40d0ad0 msgr2=0x7fbcf4159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcf894c640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcf40d0ad0 0x7fbcf4159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcf894c640 1 -- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf415dd30 msgr2=0x7fbcf4159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcf894c640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf415dd30 0x7fbcf4159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcf894c640 1 -- 172.21.15.19:0/585306516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf40b3140 con 0x7fbcf40b9770 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.531+0000 7fbcf894c640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcf40b9770 0x7fbcf41f7490 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0000c00 tx=0x7fbcf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.532+0000 7fbce97fa640 1 -- 172.21.15.19:0/585306516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0037050 con 0x7fbcf40b9770 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.532+0000 7fbce97fa640 1 -- 172.21.15.19:0/585306516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf0038040 con 0x7fbcf40b9770 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.532+0000 7fbcfabb0640 1 -- 172.21.15.19:0/585306516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf40b3420 con 0x7fbcf40b9770 2024-04-25T10:01:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.532+0000 7fbce97fa640 1 -- 172.21.15.19:0/585306516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0004270 con 0x7fbcf40b9770 2024-04-25T10:01:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.532+0000 7fbcfabb0640 1 -- 172.21.15.19:0/585306516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf41595a0 con 0x7fbcf40b9770 2024-04-25T10:01:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.533+0000 7fbcfabb0640 1 -- 172.21.15.19:0/585306516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf4073990 con 0x7fbcf40b9770 2024-04-25T10:01:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.533+0000 7fbce97fa640 1 -- 172.21.15.19:0/585306516 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcf0003610 con 0x7fbcf40b9770 2024-04-25T10:01:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.534+0000 7fbce97fa640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbce006b4d0 0x7fbce006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.534+0000 7fbce97fa640 1 -- 172.21.15.19:0/585306516 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbcf00aa9d0 con 0x7fbcf40b9770 2024-04-25T10:01:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.534+0000 7fbcebfff640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbce006b4d0 0x7fbce006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.534+0000 7fbcebfff640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbce006b4d0 0x7fbce006d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40b7a30 tx=0x7fbce4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.538+0000 7fbce97fa640 1 -- 172.21.15.19:0/585306516 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbcf0079190 con 0x7fbcf40b9770 2024-04-25T10:01:51.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.794+0000 7fbcfabb0640 1 -- 172.21.15.19:0/585306516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbcf40b35f0 con 0x7fbcf40b9770 2024-04-25T10:01:51.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.795+0000 7fbce97fa640 1 -- 172.21.15.19:0/585306516 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7fbcf007d230 con 0x7fbcf40b9770 2024-04-25T10:01:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:51.800 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":549,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":129,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":888,"data_bytes":908298504,"bytes_used":2103189504,"bytes_avail":770990923776,"bytes_total":773094113280,"write_bytes_sec":75630058,"read_op_per_sec":0,"write_op_per_sec":74},"fsmap":{"epoch":79,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9213}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:01:13.023211+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:01:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 -- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbce006b4d0 msgr2=0x7fbce006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbce006b4d0 0x7fbce006d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40b7a30 tx=0x7fbce4009040 comp rx=0 tx=0).stop 2024-04-25T10:01:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 -- 172.21.15.19:0/585306516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcf40b9770 msgr2=0x7fbcf41f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcf40b9770 0x7fbcf41f7490 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0000c00 tx=0x7fbcf00046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 -- 172.21.15.19:0/585306516 shutdown_connections 2024-04-25T10:01:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbce006b4d0 0x7fbce006d990 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf415dd30 0x7fbcf4159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcf40d0ad0 0x7fbcf4159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 --2- 172.21.15.19:0/585306516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcf40b9770 0x7fbcf41f7490 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 -- 172.21.15.19:0/585306516 >> 172.21.15.19:0/585306516 conn(0x7fbcf4061c30 msgr2=0x7fbcf40b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 -- 172.21.15.19:0/585306516 shutdown_connections 2024-04-25T10:01:51.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:51.797+0000 7fbcfabb0640 1 -- 172.21.15.19:0/585306516 wait complete. 2024-04-25T10:01:51.812 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:01:52.813 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.923+0000 7f0f74b22640 1 Processor -- start 2024-04-25T10:01:52.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.923+0000 7f0f74b22640 1 -- start start 2024-04-25T10:01:52.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.924+0000 7f0f74b22640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0f700b6cb0 0x7f0f700b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.924+0000 7f0f74b22640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f700ceee0 con 0x7f0f700b5a60 2024-04-25T10:01:52.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.924+0000 7f0f74b22640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f700d1d10 con 0x7f0f7015db70 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.924+0000 7f0f74b22640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f700d1ff0 con 0x7f0f700b6cb0 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.924+0000 7f0f6ed9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0f700b5a60 0x7f0f700d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39838/0 (socket says 172.21.15.19:39838) 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.924+0000 7f0f6ed9d640 1 -- 172.21.15.19:0/2870126944 learned_addr learned my addr 172.21.15.19:0/2870126944 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.924+0000 7f0f6dd9b640 1 --2- 172.21.15.19:0/2870126944 >> v2:172.21.15.112:3300/0 conn(0x7f0f700b6cb0 0x7f0f700b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.924+0000 7f0f6d59a640 1 -- 172.21.15.19:0/2870126944 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519468271 0 0) 0x7f0f700ceee0 con 0x7f0f700b5a60 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.924+0000 7f0f6d59a640 1 -- 172.21.15.19:0/2870126944 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f58003520 con 0x7f0f700b5a60 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.925+0000 7f0f6d59a640 1 -- 172.21.15.19:0/2870126944 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690087718 0 0) 0x7f0f700d1d10 con 0x7f0f7015db70 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.925+0000 7f0f6d59a640 1 -- 172.21.15.19:0/2870126944 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f700ceee0 con 0x7f0f7015db70 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.925+0000 7f0f6dd9b640 1 -- 172.21.15.19:0/2870126944 >> v1:172.21.15.112:6789/0 conn(0x7f0f7015db70 legacy=0x7f0f70162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.925+0000 7f0f6dd9b640 1 -- 172.21.15.19:0/2870126944 >> v1:172.21.15.19:6789/0 conn(0x7f0f700b5a60 legacy=0x7f0f700d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.925+0000 7f0f6d59a640 1 -- 172.21.15.19:0/2870126944 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4208069420 0 0) 0x7f0f58003520 con 0x7f0f700b5a60 2024-04-25T10:01:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.925+0000 7f0f6dd9b640 1 -- 172.21.15.19:0/2870126944 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f700d22d0 con 0x7f0f700b6cb0 2024-04-25T10:01:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.926+0000 7f0f6dd9b640 1 --2- 172.21.15.19:0/2870126944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f700b6cb0 0x7f0f700b5520 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0f640097a0 tx=0x7f0f6402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31c8cc89a4b6329c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.926+0000 7f0f6d59a640 1 -- 172.21.15.19:0/2870126944 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64037070 con 0x7f0f700b6cb0 2024-04-25T10:01:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.926+0000 7f0f6d59a640 1 -- 172.21.15.19:0/2870126944 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6402abc0 con 0x7f0f700b6cb0 2024-04-25T10:01:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.926+0000 7f0f6d59a640 1 -- 172.21.15.19:0/2870126944 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680019f0 con 0x7f0f700b6cb0 2024-04-25T10:01:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.926+0000 7f0f74b22640 1 -- 172.21.15.19:0/2870126944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f700b6cb0 msgr2=0x7f0f700b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.926+0000 7f0f74b22640 1 --2- 172.21.15.19:0/2870126944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f700b6cb0 0x7f0f700b5520 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0f640097a0 tx=0x7f0f6402a130 comp rx=0 tx=0).stop 2024-04-25T10:01:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.926+0000 7f0f74b22640 1 -- 172.21.15.19:0/2870126944 shutdown_connections 2024-04-25T10:01:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.926+0000 7f0f74b22640 1 --2- 172.21.15.19:0/2870126944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f700b6cb0 0x7f0f700b5520 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.926+0000 7f0f74b22640 1 -- 172.21.15.19:0/2870126944 >> 172.21.15.19:0/2870126944 conn(0x7f0f700d0520 msgr2=0x7f0f700d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.927+0000 7f0f74b22640 1 -- 172.21.15.19:0/2870126944 shutdown_connections 2024-04-25T10:01:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.927+0000 7f0f74b22640 1 -- 172.21.15.19:0/2870126944 wait complete. 2024-04-25T10:01:52.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.928+0000 7f0f74b22640 1 Processor -- start 2024-04-25T10:01:52.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.928+0000 7f0f74b22640 1 -- start start 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f74b22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f700b5a60 0x7f0f7015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f74b22640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f700b6cb0 0x7f0f701f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f74b22640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f7015db70 0x7f0f70156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f74b22640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700cf1c0 con 0x7f0f7015db70 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f74b22640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700ce530 con 0x7f0f700b5a60 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f74b22640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700cf340 con 0x7f0f700b6cb0 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f7015db70 0x7f0f70156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f7015db70 0x7f0f70156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39048/0 (socket says 172.21.15.19:39048) 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6ed9d640 1 -- 172.21.15.19:0/2040340417 learned_addr learned my addr 172.21.15.19:0/2040340417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6dd9b640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f700b6cb0 0x7f0f701f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6e59c640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f700b5a60 0x7f0f7015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6ed9d640 1 -- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f700b6cb0 msgr2=0x7f0f701f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6ed9d640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f700b6cb0 0x7f0f701f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6ed9d640 1 -- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f700b5a60 msgr2=0x7f0f7015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6ed9d640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f700b5a60 0x7f0f7015e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6ed9d640 1 -- 172.21.15.19:0/2040340417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f701606c0 con 0x7f0f7015db70 2024-04-25T10:01:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6dd9b640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f700b6cb0 0x7f0f701f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f6ed9d640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f7015db70 0x7f0f70156bd0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800cc20 tx=0x7f0f6800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.929+0000 7f0f5effd640 1 -- 172.21.15.19:0/2040340417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68017070 con 0x7f0f7015db70 2024-04-25T10:01:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.930+0000 7f0f5effd640 1 -- 172.21.15.19:0/2040340417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6800ce30 con 0x7f0f7015db70 2024-04-25T10:01:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.930+0000 7f0f74b22640 1 -- 172.21.15.19:0/2040340417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f701609a0 con 0x7f0f7015db70 2024-04-25T10:01:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.930+0000 7f0f5effd640 1 -- 172.21.15.19:0/2040340417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68012620 con 0x7f0f7015db70 2024-04-25T10:01:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.930+0000 7f0f74b22640 1 -- 172.21.15.19:0/2040340417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f70160f60 con 0x7f0f7015db70 2024-04-25T10:01:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.931+0000 7f0f74b22640 1 -- 172.21.15.19:0/2040340417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f70073990 con 0x7f0f7015db70 2024-04-25T10:01:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.931+0000 7f0f5effd640 1 -- 172.21.15.19:0/2040340417 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f6801a070 con 0x7f0f7015db70 2024-04-25T10:01:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.932+0000 7f0f5effd640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f5006b4d0 0x7f0f5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.932+0000 7f0f5effd640 1 -- 172.21.15.19:0/2040340417 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f6808a2a0 con 0x7f0f7015db70 2024-04-25T10:01:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.932+0000 7f0f6e59c640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f5006b4d0 0x7f0f5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.933+0000 7f0f6e59c640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f5006b4d0 0x7f0f5006d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0f600011e0 tx=0x7f0f60009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:52.936+0000 7f0f5effd640 1 -- 172.21.15.19:0/2040340417 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f68058f90 con 0x7f0f7015db70 2024-04-25T10:01:53.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.192+0000 7f0f74b22640 1 -- 172.21.15.19:0/2040340417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f700d0c80 con 0x7f0f7015db70 2024-04-25T10:01:53.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.192+0000 7f0f5effd640 1 -- 172.21.15.19:0/2040340417 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0f68058890 con 0x7f0f7015db70 2024-04-25T10:01:53.197 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:53.197 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:01:30.999245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/250977615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":250977615},{"type":"v1","addr":"172.21.15.19:6835","nonce":250977615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:01:53.197 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 79 2024-04-25T10:01:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 -- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f5006b4d0 msgr2=0x7f0f5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f5006b4d0 0x7f0f5006d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0f600011e0 tx=0x7f0f60009040 comp rx=0 tx=0).stop 2024-04-25T10:01:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 -- 172.21.15.19:0/2040340417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f7015db70 msgr2=0x7f0f70156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f7015db70 0x7f0f70156bd0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800cc20 tx=0x7f0f6800a430 comp rx=0 tx=0).stop 2024-04-25T10:01:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 -- 172.21.15.19:0/2040340417 shutdown_connections 2024-04-25T10:01:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f5006b4d0 0x7f0f5006d990 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f7015db70 0x7f0f70156bd0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f700b6cb0 0x7f0f701f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 --2- 172.21.15.19:0/2040340417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f700b5a60 0x7f0f7015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 -- 172.21.15.19:0/2040340417 >> 172.21.15.19:0/2040340417 conn(0x7f0f700d0520 msgr2=0x7f0f7015c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 -- 172.21.15.19:0/2040340417 shutdown_connections 2024-04-25T10:01:53.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.195+0000 7f0f74b22640 1 -- 172.21.15.19:0/2040340417 wait complete. 2024-04-25T10:01:53.210 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:01:53.211 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.306+0000 7f2a2095c640 1 Processor -- start 2024-04-25T10:01:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.306+0000 7f2a2095c640 1 -- start start 2024-04-25T10:01:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.307+0000 7f2a2095c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a1c071d50 0x7f2a1c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.307+0000 7f2a2095c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1c06ee10 con 0x7f2a1c16aa50 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.307+0000 7f2a2095c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1c16ae30 con 0x7f2a1c06e9f0 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.307+0000 7f2a2095c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c0cd850 con 0x7f2a1c071d50 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.307+0000 7f2a1a59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2a1c16aa50 0x7f2a1c0cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39844/0 (socket says 172.21.15.19:39844) 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.307+0000 7f2a1a59c640 1 -- 172.21.15.19:0/3630966628 learned_addr learned my addr 172.21.15.19:0/3630966628 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.307+0000 7f2a19d9b640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a1c071d50 0x7f2a1c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.307+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657779655 0 0) 0x7f2a1c06ee10 con 0x7f2a1c16aa50 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a04003500 con 0x7f2a1c16aa50 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118285603 0 0) 0x7f2a1c16ae30 con 0x7f2a1c06e9f0 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a1c06ee10 con 0x7f2a1c06e9f0 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1773835822 0 0) 0x7f2a04003500 con 0x7f2a1c16aa50 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a1c071d50 msgr2=0x7f2a1c06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a1c071d50 0x7f2a1c06e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 >> v1:172.21.15.112:6789/0 conn(0x7f2a1c06e9f0 legacy=0x7f2a1c0b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c0cf050 con 0x7f2a1c16aa50 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f2a0c001670 con 0x7f2a1c16aa50 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 >> v1:172.21.15.19:6789/0 conn(0x7f2a1c16aa50 legacy=0x7f2a1c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a04003b80 0x7f2a04003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0400afc0 0x7f2a0400d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a1c071d50 0x7f2a0400f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c0cd850 con 0x7f2a04003b80 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0400ff20 con 0x7f2a0400afc0 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.318+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04010120 con 0x7f2a1c071d50 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.319+0000 7f2a1a59c640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a04003b80 0x7f2a04003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.319+0000 7f2a19d9b640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0400afc0 0x7f2a0400d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.319+0000 7f2a1ad9d640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a1c071d50 0x7f2a0400f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.319+0000 7f2a1a59c640 1 -- 172.21.15.19:0/3630966628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a1c071d50 msgr2=0x7f2a0400f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.319+0000 7f2a1a59c640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a1c071d50 0x7f2a0400f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.319+0000 7f2a1a59c640 1 -- 172.21.15.19:0/3630966628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0400afc0 msgr2=0x7f2a0400d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.319+0000 7f2a1a59c640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0400afc0 0x7f2a0400d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.319+0000 7f2a1a59c640 1 -- 172.21.15.19:0/3630966628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a04003500 con 0x7f2a04003b80 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.319+0000 7f2a1a59c640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a04003b80 0x7f2a04003f80 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c007fd0 tx=0x7f2a04006aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442510992a3663b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.320+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c007ce0 con 0x7f2a04003b80 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.320+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c005ad0 con 0x7f2a04003b80 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.320+0000 7f2a1959a640 1 -- 172.21.15.19:0/3630966628 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c007070 con 0x7f2a04003b80 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.320+0000 7f2a2095c640 1 -- 172.21.15.19:0/3630966628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a04003b80 msgr2=0x7f2a04003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.320+0000 7f2a2095c640 1 --2- 172.21.15.19:0/3630966628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a04003b80 0x7f2a04003f80 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c007fd0 tx=0x7f2a04006aa0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.320+0000 7f2a1ad9d640 1 -- 172.21.15.19:0/3630966628 reap_dead start 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.320+0000 7f2a2095c640 1 -- 172.21.15.19:0/3630966628 shutdown_connections 2024-04-25T10:01:53.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.320+0000 7f2a2095c640 1 -- 172.21.15.19:0/3630966628 >> 172.21.15.19:0/3630966628 conn(0x7f2a1c061a20 msgr2=0x7f2a1c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.320+0000 7f2a2095c640 1 -- 172.21.15.19:0/3630966628 shutdown_connections 2024-04-25T10:01:53.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.320+0000 7f2a2095c640 1 -- 172.21.15.19:0/3630966628 wait complete. 2024-04-25T10:01:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.322+0000 7f2a2095c640 1 Processor -- start 2024-04-25T10:01:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.322+0000 7f2a2095c640 1 -- start start 2024-04-25T10:01:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.322+0000 7f2a2095c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a04003b80 0x7f2a1c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.322+0000 7f2a2095c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a1c0ab290 0x7f2a1c15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.322+0000 7f2a2095c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a1c0b0d80 0x7f2a1c0b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.322+0000 7f2a2095c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c0cf610 con 0x7f2a1c0ab290 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.322+0000 7f2a2095c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c08ad90 con 0x7f2a1c0b0d80 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.322+0000 7f2a2095c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c0cf8f0 con 0x7f2a04003b80 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a19d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a1c0ab290 0x7f2a1c15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a19d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a1c0ab290 0x7f2a1c15a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39054/0 (socket says 172.21.15.19:39054) 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a19d9b640 1 -- 172.21.15.19:0/3156382316 learned_addr learned my addr 172.21.15.19:0/3156382316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a1ad9d640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a1c0b0d80 0x7f2a1c0b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a1a59c640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a04003b80 0x7f2a1c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a19d9b640 1 -- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a04003b80 msgr2=0x7f2a1c0ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a19d9b640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a04003b80 0x7f2a1c0ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a19d9b640 1 -- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a1c0b0d80 msgr2=0x7f2a1c0b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a19d9b640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a1c0b0d80 0x7f2a1c0b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a19d9b640 1 -- 172.21.15.19:0/3156382316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c15ac30 con 0x7f2a1c0ab290 2024-04-25T10:01:53.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a19d9b640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a1c0ab290 0x7f2a1c15a4d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2a100107e0 tx=0x7f2a10010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a0affd640 1 -- 172.21.15.19:0/3156382316 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1001b020 con 0x7f2a1c0ab290 2024-04-25T10:01:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a0affd640 1 -- 172.21.15.19:0/3156382316 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1000ead0 con 0x7f2a1c0ab290 2024-04-25T10:01:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a2095c640 1 -- 172.21.15.19:0/3156382316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c15af10 con 0x7f2a1c0ab290 2024-04-25T10:01:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a0affd640 1 -- 172.21.15.19:0/3156382316 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10017470 con 0x7f2a1c0ab290 2024-04-25T10:01:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.323+0000 7f2a2095c640 1 -- 172.21.15.19:0/3156382316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c158640 con 0x7f2a1c0ab290 2024-04-25T10:01:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.324+0000 7f2a2095c640 1 -- 172.21.15.19:0/3156382316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1c15b430 con 0x7f2a1c0ab290 2024-04-25T10:01:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.325+0000 7f2a0affd640 1 -- 172.21.15.19:0/3156382316 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a1001e020 con 0x7f2a1c0ab290 2024-04-25T10:01:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.325+0000 7f2a0affd640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29fc06b4d0 0x7f29fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.325+0000 7f2a0affd640 1 -- 172.21.15.19:0/3156382316 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2a1008abb0 con 0x7f2a1c0ab290 2024-04-25T10:01:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.325+0000 7f2a1a59c640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29fc06b4d0 0x7f29fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.326+0000 7f2a1a59c640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29fc06b4d0 0x7f29fc06d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c032000 tx=0x7f2a0c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.329+0000 7f2a0affd640 1 -- 172.21.15.19:0/3156382316 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a1c15b430 con 0x7f2a1c0ab290 2024-04-25T10:01:53.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.582+0000 7f2a2095c640 1 -- 172.21.15.19:0/3156382316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2a1c0ce290 con 0x7f2a1c0ab290 2024-04-25T10:01:53.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.582+0000 7f2a0affd640 1 -- 172.21.15.19:0/3156382316 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7f2a1c0ce290 con 0x7f2a1c0ab290 2024-04-25T10:01:53.587 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:53.587 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":550,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":129,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":888,"data_bytes":908298504,"bytes_used":2103189504,"bytes_avail":770990923776,"bytes_total":773094113280,"write_bytes_sec":75630058,"read_op_per_sec":0,"write_op_per_sec":74},"fsmap":{"epoch":79,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9213}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:01:13.023211+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:01:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 -- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29fc06b4d0 msgr2=0x7f29fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29fc06b4d0 0x7f29fc06d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c032000 tx=0x7f2a0c009030 comp rx=0 tx=0).stop 2024-04-25T10:01:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 -- 172.21.15.19:0/3156382316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a1c0ab290 msgr2=0x7f2a1c15a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a1c0ab290 0x7f2a1c15a4d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2a100107e0 tx=0x7f2a10010ca0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 -- 172.21.15.19:0/3156382316 shutdown_connections 2024-04-25T10:01:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f29fc06b4d0 0x7f29fc06d990 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a1c0b0d80 0x7f2a1c0b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a1c0ab290 0x7f2a1c15a4d0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 --2- 172.21.15.19:0/3156382316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a04003b80 0x7f2a1c0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 -- 172.21.15.19:0/3156382316 >> 172.21.15.19:0/3156382316 conn(0x7f2a1c061a20 msgr2=0x7f2a1c16b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 -- 172.21.15.19:0/3156382316 shutdown_connections 2024-04-25T10:01:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:53.584+0000 7f2a2095c640 1 -- 172.21.15.19:0/3156382316 wait complete. 2024-04-25T10:01:53.600 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:01:54.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:54.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.711+0000 7f8cddcd1640 1 Processor -- start 2024-04-25T10:01:54.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.711+0000 7f8cddcd1640 1 -- start start 2024-04-25T10:01:54.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cddcd1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cd815dc20 0x7f8cd81616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cddcd1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cd80b93a0 0x7f8cd80ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cddcd1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8cd80cec20 0x7f8cd806c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cddcd1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd815e000 con 0x7f8cd80b93a0 2024-04-25T10:01:54.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cddcd1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd80b6bb0 con 0x7f8cd80cec20 2024-04-25T10:01:54.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cddcd1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80b6030 con 0x7f8cd815dc20 2024-04-25T10:01:54.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cd6ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cd80b93a0 0x7f8cd80ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cd6ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cd80b93a0 0x7f8cd80ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39062/0 (socket says 172.21.15.19:39062) 2024-04-25T10:01:54.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cd6ffd640 1 -- 172.21.15.19:0/872752239 learned_addr learned my addr 172.21.15.19:0/872752239 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:54.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cd7fff640 1 --2- 172.21.15.19:0/872752239 >> v2:172.21.15.112:3300/0 conn(0x7f8cd80cec20 0x7f8cd806c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cd77fe640 1 --2- 172.21.15.19:0/872752239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cd815dc20 0x7f8cd81616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cd6ffd640 1 -- 172.21.15.19:0/872752239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cd815dc20 msgr2=0x7f8cd81616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cd6ffd640 1 --2- 172.21.15.19:0/872752239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cd815dc20 0x7f8cd81616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cd6ffd640 1 -- 172.21.15.19:0/872752239 >> v2:172.21.15.112:3300/0 conn(0x7f8cd80cec20 msgr2=0x7f8cd806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cd6ffd640 1 --2- 172.21.15.19:0/872752239 >> v2:172.21.15.112:3300/0 conn(0x7f8cd80cec20 0x7f8cd806c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.712+0000 7f8cd6ffd640 1 -- 172.21.15.19:0/872752239 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd8161de0 con 0x7f8cd80b93a0 2024-04-25T10:01:54.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.713+0000 7f8cd6ffd640 1 --2- 172.21.15.19:0/872752239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd80b93a0 0x7f8cd80ce6e0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80d1ff0 tx=0x7f8ccc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2f6e14b50e0471 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.713+0000 7f8cd67fc640 1 -- 172.21.15.19:0/872752239 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc037070 con 0x7f8cd80b93a0 2024-04-25T10:01:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.714+0000 7f8cd67fc640 1 -- 172.21.15.19:0/872752239 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc032440 con 0x7f8cd80b93a0 2024-04-25T10:01:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.714+0000 7f8cd67fc640 1 -- 172.21.15.19:0/872752239 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc032810 con 0x7f8cd80b93a0 2024-04-25T10:01:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.714+0000 7f8cddcd1640 1 -- 172.21.15.19:0/872752239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd80b93a0 msgr2=0x7f8cd80ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.714+0000 7f8cddcd1640 1 --2- 172.21.15.19:0/872752239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd80b93a0 0x7f8cd80ce6e0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80d1ff0 tx=0x7f8ccc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.714+0000 7f8cddcd1640 1 -- 172.21.15.19:0/872752239 shutdown_connections 2024-04-25T10:01:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.714+0000 7f8cddcd1640 1 --2- 172.21.15.19:0/872752239 >> v2:172.21.15.112:3300/0 conn(0x7f8cd80cec20 0x7f8cd806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.714+0000 7f8cddcd1640 1 --2- 172.21.15.19:0/872752239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd80b93a0 0x7f8cd80ce6e0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.714+0000 7f8cddcd1640 1 --2- 172.21.15.19:0/872752239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cd815dc20 0x7f8cd81616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.714+0000 7f8cddcd1640 1 -- 172.21.15.19:0/872752239 >> 172.21.15.19:0/872752239 conn(0x7f8cd80d0490 msgr2=0x7f8cd80d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.714+0000 7f8cddcd1640 1 -- 172.21.15.19:0/872752239 shutdown_connections 2024-04-25T10:01:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.714+0000 7f8cddcd1640 1 -- 172.21.15.19:0/872752239 wait complete. 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.715+0000 7f8cddcd1640 1 Processor -- start 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.716+0000 7f8cddcd1640 1 -- start start 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.716+0000 7f8cddcd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd80b93a0 0x7f8cd8149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.716+0000 7f8cddcd1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd80cec20 0x7f8cd815f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.716+0000 7f8cddcd1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cd815dc20 0x7f8cd80caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cddcd1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd806c610 con 0x7f8cd80cec20 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cddcd1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd81623a0 con 0x7f8cd80b93a0 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cddcd1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd806c790 con 0x7f8cd815dc20 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cd6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd80cec20 0x7f8cd815f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cd6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd80cec20 0x7f8cd815f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39072/0 (socket says 172.21.15.19:39072) 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cd6ffd640 1 -- 172.21.15.19:0/2555459215 learned_addr learned my addr 172.21.15.19:0/2555459215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:54.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cd77fe640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd80b93a0 0x7f8cd8149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cd7fff640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cd815dc20 0x7f8cd80caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cd6ffd640 1 -- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cd815dc20 msgr2=0x7f8cd80caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cd6ffd640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cd815dc20 0x7f8cd80caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cd6ffd640 1 -- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd80b93a0 msgr2=0x7f8cd8149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cd6ffd640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd80b93a0 0x7f8cd8149b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cd6ffd640 1 -- 172.21.15.19:0/2555459215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd814a090 con 0x7f8cd80cec20 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cd6ffd640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd80cec20 0x7f8cd815f230 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc000c00 tx=0x7f8ccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.717+0000 7f8cbffff640 1 -- 172.21.15.19:0/2555459215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc041070 con 0x7f8cd80cec20 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.718+0000 7f8cbffff640 1 -- 172.21.15.19:0/2555459215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc03c870 con 0x7f8cd80cec20 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.718+0000 7f8cbffff640 1 -- 172.21.15.19:0/2555459215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc037040 con 0x7f8cd80cec20 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.718+0000 7f8cddcd1640 1 -- 172.21.15.19:0/2555459215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd815fcf0 con 0x7f8cd80cec20 2024-04-25T10:01:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.718+0000 7f8cddcd1640 1 -- 172.21.15.19:0/2555459215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd81602b0 con 0x7f8cd80cec20 2024-04-25T10:01:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.719+0000 7f8cddcd1640 1 -- 172.21.15.19:0/2555459215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd8073990 con 0x7f8cd80cec20 2024-04-25T10:01:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.719+0000 7f8cbffff640 1 -- 172.21.15.19:0/2555459215 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ccc044070 con 0x7f8cd80cec20 2024-04-25T10:01:54.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.719+0000 7f8cbffff640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cc406b520 0x7f8cc406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.720+0000 7f8cbffff640 1 -- 172.21.15.19:0/2555459215 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ccc0ab0d0 con 0x7f8cd80cec20 2024-04-25T10:01:54.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.720+0000 7f8cd77fe640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cc406b520 0x7f8cc406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.720+0000 7f8cd77fe640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cc406b520 0x7f8cc406d9e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80d3810 tx=0x7f8cc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.724+0000 7f8cbffff640 1 -- 172.21.15.19:0/2555459215 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ccc079970 con 0x7f8cd80cec20 2024-04-25T10:01:54.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.940+0000 7f8cddcd1640 1 -- 172.21.15.19:0/2555459215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cd80d1720 con 0x7f8cd80cec20 2024-04-25T10:01:54.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.940+0000 7f8cbffff640 1 -- 172.21.15.19:0/2555459215 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f8ccc07da10 con 0x7f8cd80cec20 2024-04-25T10:01:54.945 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:54.945 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:01:30.999245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/250977615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":250977615},{"type":"v1","addr":"172.21.15.19:6835","nonce":250977615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:01:54.945 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 79 2024-04-25T10:01:54.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.942+0000 7f8cddcd1640 1 -- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cc406b520 msgr2=0x7f8cc406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.942+0000 7f8cddcd1640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cc406b520 0x7f8cc406d9e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80d3810 tx=0x7f8cc8008040 comp rx=0 tx=0).stop 2024-04-25T10:01:54.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.942+0000 7f8cddcd1640 1 -- 172.21.15.19:0/2555459215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd80cec20 msgr2=0x7f8cd815f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.942+0000 7f8cddcd1640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd80cec20 0x7f8cd815f230 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc000c00 tx=0x7f8ccc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.942+0000 7f8cddcd1640 1 -- 172.21.15.19:0/2555459215 shutdown_connections 2024-04-25T10:01:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.942+0000 7f8cddcd1640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cc406b520 0x7f8cc406d9e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.942+0000 7f8cddcd1640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cd815dc20 0x7f8cd80caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.942+0000 7f8cddcd1640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd80cec20 0x7f8cd815f230 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.942+0000 7f8cddcd1640 1 --2- 172.21.15.19:0/2555459215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cd80b93a0 0x7f8cd8149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.942+0000 7f8cddcd1640 1 -- 172.21.15.19:0/2555459215 >> 172.21.15.19:0/2555459215 conn(0x7f8cd80d0490 msgr2=0x7f8cd80b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.943+0000 7f8cddcd1640 1 -- 172.21.15.19:0/2555459215 shutdown_connections 2024-04-25T10:01:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:54.943+0000 7f8cddcd1640 1 -- 172.21.15.19:0/2555459215 wait complete. 2024-04-25T10:01:54.958 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:01:54.958 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:55.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.053+0000 7fe10c729640 1 Processor -- start 2024-04-25T10:01:55.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.053+0000 7fe10c729640 1 -- start start 2024-04-25T10:01:55.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.054+0000 7fe10c729640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe10415dc70 0x7fe1040d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.054+0000 7fe10c729640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe10409a450 0x7fe1040cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.054+0000 7fe10c729640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10406c220 con 0x7fe10406c700 2024-04-25T10:01:55.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.054+0000 7fe10c729640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1040d0c90 con 0x7fe10415dc70 2024-04-25T10:01:55.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.054+0000 7fe10c729640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1040d0080 con 0x7fe10409a450 2024-04-25T10:01:55.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.055+0000 7fe10a4c5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe10415dc70 0x7fe1040d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.055+0000 7fe10acc6640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe10406c700 0x7fe1040cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39858/0 (socket says 172.21.15.19:39858) 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.055+0000 7fe10acc6640 1 -- 172.21.15.19:0/1402950646 learned_addr learned my addr 172.21.15.19:0/1402950646 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.055+0000 7fe109cc4640 1 --2- 172.21.15.19:0/1402950646 >> v2:172.21.15.112:3300/0 conn(0x7fe10409a450 0x7fe1040cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.055+0000 7fe10a4c5640 1 -- 172.21.15.19:0/1402950646 >> v2:172.21.15.112:3300/0 conn(0x7fe10409a450 msgr2=0x7fe1040cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.055+0000 7fe10a4c5640 1 --2- 172.21.15.19:0/1402950646 >> v2:172.21.15.112:3300/0 conn(0x7fe10409a450 0x7fe1040cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.055+0000 7fe10a4c5640 1 -- 172.21.15.19:0/1402950646 >> v1:172.21.15.19:6789/0 conn(0x7fe10406c700 legacy=0x7fe1040cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.055+0000 7fe10a4c5640 1 -- 172.21.15.19:0/1402950646 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1040d0360 con 0x7fe10415dc70 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.055+0000 7fe1094c3640 1 -- 172.21.15.19:0/1402950646 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210774135 0 0) 0x7fe10406c220 con 0x7fe10406c700 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.055+0000 7fe109cc4640 1 --2- 172.21.15.19:0/1402950646 >> v2:172.21.15.112:3300/0 conn(0x7fe10409a450 0x7fe1040cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe10a4c5640 1 --2- 172.21.15.19:0/1402950646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe10415dc70 0x7fe1040d1120 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fe1040733b0 tx=0x7fe10002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7993c754c3b9e286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe1094c3640 1 -- 172.21.15.19:0/1402950646 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100037070 con 0x7fe10415dc70 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe1094c3640 1 -- 172.21.15.19:0/1402950646 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100032440 con 0x7fe10415dc70 2024-04-25T10:01:55.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe1094c3640 1 -- 172.21.15.19:0/1402950646 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100032810 con 0x7fe10415dc70 2024-04-25T10:01:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe10c729640 1 -- 172.21.15.19:0/1402950646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe10415dc70 msgr2=0x7fe1040d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe10c729640 1 --2- 172.21.15.19:0/1402950646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe10415dc70 0x7fe1040d1120 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fe1040733b0 tx=0x7fe10002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe10c729640 1 -- 172.21.15.19:0/1402950646 shutdown_connections 2024-04-25T10:01:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe10c729640 1 --2- 172.21.15.19:0/1402950646 >> v2:172.21.15.112:3300/0 conn(0x7fe10409a450 0x7fe1040cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe10c729640 1 --2- 172.21.15.19:0/1402950646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe10415dc70 0x7fe1040d1120 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe10c729640 1 -- 172.21.15.19:0/1402950646 >> 172.21.15.19:0/1402950646 conn(0x7fe1040b3270 msgr2=0x7fe1040b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe10c729640 1 -- 172.21.15.19:0/1402950646 shutdown_connections 2024-04-25T10:01:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.056+0000 7fe10c729640 1 -- 172.21.15.19:0/1402950646 wait complete. 2024-04-25T10:01:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.058+0000 7fe10c729640 1 Processor -- start 2024-04-25T10:01:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.058+0000 7fe10c729640 1 -- start start 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.058+0000 7fe10c729640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe10406c700 0x7fe10409d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.058+0000 7fe10c729640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10409a450 0x7fe1040c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.058+0000 7fe10c729640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe10415dc70 0x7fe104153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.058+0000 7fe10c729640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10415e050 con 0x7fe10415dc70 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.058+0000 7fe10c729640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040d0920 con 0x7fe10409a450 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.058+0000 7fe10c729640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10406cb90 con 0x7fe10406c700 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.058+0000 7fe10acc6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe10415dc70 0x7fe104153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.058+0000 7fe10acc6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe10415dc70 0x7fe104153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39086/0 (socket says 172.21.15.19:39086) 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe10acc6640 1 -- 172.21.15.19:0/936940358 learned_addr learned my addr 172.21.15.19:0/936940358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe10a4c5640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe10406c700 0x7fe10409d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe109cc4640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10409a450 0x7fe1040c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe10acc6640 1 -- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe10406c700 msgr2=0x7fe10409d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe10acc6640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe10406c700 0x7fe10409d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe10acc6640 1 -- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10409a450 msgr2=0x7fe1040c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe10acc6640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10409a450 0x7fe1040c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe10acc6640 1 -- 172.21.15.19:0/936940358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10409c290 con 0x7fe10415dc70 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe10acc6640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe10415dc70 0x7fe104153780 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c7a0 tx=0x7fe0fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe0faffd640 1 -- 172.21.15.19:0/936940358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc018070 con 0x7fe10415dc70 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe0faffd640 1 -- 172.21.15.19:0/936940358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc00a640 con 0x7fe10415dc70 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe10c729640 1 -- 172.21.15.19:0/936940358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10409c570 con 0x7fe10415dc70 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe0faffd640 1 -- 172.21.15.19:0/936940358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc00aab0 con 0x7fe10415dc70 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.059+0000 7fe10c729640 1 -- 172.21.15.19:0/936940358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10409cb30 con 0x7fe10415dc70 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.060+0000 7fe10c729640 1 -- 172.21.15.19:0/936940358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe104073990 con 0x7fe10415dc70 2024-04-25T10:01:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.061+0000 7fe0faffd640 1 -- 172.21.15.19:0/936940358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0fc01b070 con 0x7fe10415dc70 2024-04-25T10:01:55.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.061+0000 7fe0faffd640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe0e806b4d0 0x7fe0e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.061+0000 7fe0faffd640 1 -- 172.21.15.19:0/936940358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0fc0899c0 con 0x7fe10415dc70 2024-04-25T10:01:55.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.061+0000 7fe10a4c5640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe0e806b4d0 0x7fe0e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.062+0000 7fe10a4c5640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe0e806b4d0 0x7fe0e806d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe100034000 tx=0x7fe10002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.065+0000 7fe0faffd640 1 -- 172.21.15.19:0/936940358 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0fc08be20 con 0x7fe10415dc70 2024-04-25T10:01:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.321+0000 7fe10c729640 1 -- 172.21.15.19:0/936940358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe1040cfd00 con 0x7fe10415dc70 2024-04-25T10:01:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.322+0000 7fe0faffd640 1 -- 172.21.15.19:0/936940358 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7fe0fc0586b0 con 0x7fe10415dc70 2024-04-25T10:01:55.326 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:55.326 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":552,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":129,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":888,"data_bytes":908298504,"bytes_used":2103189504,"bytes_avail":770990923776,"bytes_total":773094113280,"write_bytes_sec":75631930,"read_op_per_sec":0,"write_op_per_sec":74},"fsmap":{"epoch":79,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9213}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:01:13.023211+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:01:55.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 -- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe0e806b4d0 msgr2=0x7fe0e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe0e806b4d0 0x7fe0e806d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe100034000 tx=0x7fe10002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:55.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 -- 172.21.15.19:0/936940358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe10415dc70 msgr2=0x7fe104153780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe10415dc70 0x7fe104153780 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c7a0 tx=0x7fe0fc00cc60 comp rx=0 tx=0).stop 2024-04-25T10:01:55.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 -- 172.21.15.19:0/936940358 shutdown_connections 2024-04-25T10:01:55.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe0e806b4d0 0x7fe0e806d990 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe10415dc70 0x7fe104153780 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe10409a450 0x7fe1040c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 --2- 172.21.15.19:0/936940358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe10406c700 0x7fe10409d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 -- 172.21.15.19:0/936940358 >> 172.21.15.19:0/936940358 conn(0x7fe1040b3270 msgr2=0x7fe1040ce050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:55.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 -- 172.21.15.19:0/936940358 shutdown_connections 2024-04-25T10:01:55.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:55.324+0000 7fe10c729640 1 -- 172.21.15.19:0/936940358 wait complete. 2024-04-25T10:01:55.339 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:01:56.341 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.450+0000 7fa773a4a640 1 Processor -- start 2024-04-25T10:01:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.451+0000 7fa773a4a640 1 -- start start 2024-04-25T10:01:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.451+0000 7fa773a4a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa76c0b2cb0 0x7fa76c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.452+0000 7fa773a4a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76c0b97a0 con 0x7fa76c15dbe0 2024-04-25T10:01:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.452+0000 7fa773a4a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76c0d1c50 con 0x7fa76c0d1020 2024-04-25T10:01:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.452+0000 7fa7717e6640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa76c15dbe0 0x7fa76c0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39874/0 (socket says 172.21.15.19:39874) 2024-04-25T10:01:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.452+0000 7fa773a4a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c15dfc0 con 0x7fa76c0b2cb0 2024-04-25T10:01:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.452+0000 7fa7717e6640 1 -- 172.21.15.19:0/2002292985 learned_addr learned my addr 172.21.15.19:0/2002292985 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:01:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.452+0000 7fa770fe5640 1 --2- 172.21.15.19:0/2002292985 >> v2:172.21.15.112:3300/0 conn(0x7fa76c0b2cb0 0x7fa76c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.452+0000 7fa76bfff640 1 -- 172.21.15.19:0/2002292985 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374335035 0 0) 0x7fa76c0b97a0 con 0x7fa76c15dbe0 2024-04-25T10:01:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.452+0000 7fa76bfff640 1 -- 172.21.15.19:0/2002292985 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa754003520 con 0x7fa76c15dbe0 2024-04-25T10:01:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.452+0000 7fa770fe5640 1 -- 172.21.15.19:0/2002292985 >> v1:172.21.15.112:6789/0 conn(0x7fa76c0d1020 legacy=0x7fa76c0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.452+0000 7fa770fe5640 1 -- 172.21.15.19:0/2002292985 >> v1:172.21.15.19:6789/0 conn(0x7fa76c15dbe0 legacy=0x7fa76c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.452+0000 7fa770fe5640 1 -- 172.21.15.19:0/2002292985 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c0d1f30 con 0x7fa76c0b2cb0 2024-04-25T10:01:56.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.453+0000 7fa770fe5640 1 --2- 172.21.15.19:0/2002292985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c0b2cb0 0x7fa76c0d2730 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa7600097a0 tx=0x7fa76002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c93c5e6e8fc93659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.453+0000 7fa76bfff640 1 -- 172.21.15.19:0/2002292985 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760037070 con 0x7fa76c0b2cb0 2024-04-25T10:01:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.453+0000 7fa76bfff640 1 -- 172.21.15.19:0/2002292985 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa760032440 con 0x7fa76c0b2cb0 2024-04-25T10:01:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.453+0000 7fa76bfff640 1 -- 172.21.15.19:0/2002292985 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760032810 con 0x7fa76c0b2cb0 2024-04-25T10:01:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.454+0000 7fa773a4a640 1 -- 172.21.15.19:0/2002292985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c0b2cb0 msgr2=0x7fa76c0d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.454+0000 7fa773a4a640 1 --2- 172.21.15.19:0/2002292985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c0b2cb0 0x7fa76c0d2730 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa7600097a0 tx=0x7fa76002a230 comp rx=0 tx=0).stop 2024-04-25T10:01:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.454+0000 7fa773a4a640 1 -- 172.21.15.19:0/2002292985 shutdown_connections 2024-04-25T10:01:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.454+0000 7fa773a4a640 1 --2- 172.21.15.19:0/2002292985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c0b2cb0 0x7fa76c0d2730 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.454+0000 7fa773a4a640 1 -- 172.21.15.19:0/2002292985 >> 172.21.15.19:0/2002292985 conn(0x7fa76c061a20 msgr2=0x7fa76c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.454+0000 7fa773a4a640 1 -- 172.21.15.19:0/2002292985 shutdown_connections 2024-04-25T10:01:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.454+0000 7fa773a4a640 1 -- 172.21.15.19:0/2002292985 wait complete. 2024-04-25T10:01:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.455+0000 7fa773a4a640 1 Processor -- start 2024-04-25T10:01:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.455+0000 7fa773a4a640 1 -- start start 2024-04-25T10:01:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa773a4a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c0b2cb0 0x7fa76c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa773a4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76c0d1020 0x7fa76c14c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa773a4a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa76c15dbe0 0x7fa76c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa773a4a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c0b92a0 con 0x7fa76c0d1020 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa773a4a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c0d1a00 con 0x7fa76c0b2cb0 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa773a4a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c0b9420 con 0x7fa76c15dbe0 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa770fe5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76c0d1020 0x7fa76c14c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa770fe5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76c0d1020 0x7fa76c14c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39098/0 (socket says 172.21.15.19:39098) 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa770fe5640 1 -- 172.21.15.19:0/1654727594 learned_addr learned my addr 172.21.15.19:0/1654727594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa7717e6640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c0b2cb0 0x7fa76c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa771fe7640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa76c15dbe0 0x7fa76c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa770fe5640 1 -- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa76c15dbe0 msgr2=0x7fa76c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa770fe5640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa76c15dbe0 0x7fa76c0b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa770fe5640 1 -- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c0b2cb0 msgr2=0x7fa76c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa770fe5640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c0b2cb0 0x7fa76c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa770fe5640 1 -- 172.21.15.19:0/1654727594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c15f470 con 0x7fa76c0d1020 2024-04-25T10:01:56.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.456+0000 7fa770fe5640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76c0d1020 0x7fa76c14c960 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fa760035040 tx=0x7fa7600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.457+0000 7fa769ffb640 1 -- 172.21.15.19:0/1654727594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760037040 con 0x7fa76c0d1020 2024-04-25T10:01:56.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.457+0000 7fa769ffb640 1 -- 172.21.15.19:0/1654727594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa760030500 con 0x7fa76c0d1020 2024-04-25T10:01:56.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.457+0000 7fa773a4a640 1 -- 172.21.15.19:0/1654727594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76c15f730 con 0x7fa76c0d1020 2024-04-25T10:01:56.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.457+0000 7fa769ffb640 1 -- 172.21.15.19:0/1654727594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760030910 con 0x7fa76c0d1020 2024-04-25T10:01:56.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.457+0000 7fa773a4a640 1 -- 172.21.15.19:0/1654727594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c15fcf0 con 0x7fa76c0d1020 2024-04-25T10:01:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.458+0000 7fa773a4a640 1 -- 172.21.15.19:0/1654727594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c073990 con 0x7fa76c0d1020 2024-04-25T10:01:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.458+0000 7fa769ffb640 1 -- 172.21.15.19:0/1654727594 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa760044020 con 0x7fa76c0d1020 2024-04-25T10:01:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.459+0000 7fa769ffb640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa75806b500 0x7fa75806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.459+0000 7fa769ffb640 1 -- 172.21.15.19:0/1654727594 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa7600aa520 con 0x7fa76c0d1020 2024-04-25T10:01:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.459+0000 7fa7717e6640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa75806b500 0x7fa75806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.460+0000 7fa7717e6640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa75806b500 0x7fa75806d9c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa7640011e0 tx=0x7fa764007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.463+0000 7fa769ffb640 1 -- 172.21.15.19:0/1654727594 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa760078b40 con 0x7fa76c0d1020 2024-04-25T10:01:56.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.685+0000 7fa773a4a640 1 -- 172.21.15.19:0/1654727594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa76c0d3440 con 0x7fa76c0d1020 2024-04-25T10:01:56.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.686+0000 7fa769ffb640 1 -- 172.21.15.19:0/1654727594 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa76007cbe0 con 0x7fa76c0d1020 2024-04-25T10:01:56.691 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:56.691 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:01:30.999245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/250977615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":250977615},{"type":"v1","addr":"172.21.15.19:6835","nonce":250977615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:01:56.691 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 79 2024-04-25T10:01:56.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.687+0000 7fa773a4a640 1 -- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa75806b500 msgr2=0x7fa75806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.687+0000 7fa773a4a640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa75806b500 0x7fa75806d9c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa7640011e0 tx=0x7fa764007d30 comp rx=0 tx=0).stop 2024-04-25T10:01:56.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.687+0000 7fa773a4a640 1 -- 172.21.15.19:0/1654727594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76c0d1020 msgr2=0x7fa76c14c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.687+0000 7fa773a4a640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76c0d1020 0x7fa76c14c960 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fa760035040 tx=0x7fa7600046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.688+0000 7fa773a4a640 1 -- 172.21.15.19:0/1654727594 shutdown_connections 2024-04-25T10:01:56.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.688+0000 7fa773a4a640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa75806b500 0x7fa75806d9c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.688+0000 7fa773a4a640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa76c15dbe0 0x7fa76c0b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.688+0000 7fa773a4a640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76c0d1020 0x7fa76c14c960 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.688+0000 7fa773a4a640 1 --2- 172.21.15.19:0/1654727594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa76c0b2cb0 0x7fa76c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.688+0000 7fa773a4a640 1 -- 172.21.15.19:0/1654727594 >> 172.21.15.19:0/1654727594 conn(0x7fa76c061a20 msgr2=0x7fa76c0b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.688+0000 7fa773a4a640 1 -- 172.21.15.19:0/1654727594 shutdown_connections 2024-04-25T10:01:56.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.688+0000 7fa773a4a640 1 -- 172.21.15.19:0/1654727594 wait complete. 2024-04-25T10:01:56.703 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:01:56.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.798+0000 7efd39549640 1 Processor -- start 2024-04-25T10:01:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.798+0000 7efd39549640 1 -- start start 2024-04-25T10:01:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.799+0000 7efd39549640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efd340b2cb0 0x7efd340d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.799+0000 7efd39549640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efd340d1020 0x7efd340d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.799+0000 7efd39549640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd340b97a0 con 0x7efd3415dbe0 2024-04-25T10:01:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.799+0000 7efd39549640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3415dfc0 con 0x7efd340b2cb0 2024-04-25T10:01:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.799+0000 7efd39549640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd340d1a00 con 0x7efd340d1020 2024-04-25T10:01:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.799+0000 7efd2bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efd340b2cb0 0x7efd340d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.799+0000 7efd2bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efd340b2cb0 0x7efd340d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39114/0 (socket says 172.21.15.19:39114) 2024-04-25T10:01:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.799+0000 7efd2bfff640 1 -- 172.21.15.19:0/2542205757 learned_addr learned my addr 172.21.15.19:0/2542205757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.799+0000 7efd337fe640 1 --2- 172.21.15.19:0/2542205757 >> v2:172.21.15.112:3300/0 conn(0x7efd340d1020 0x7efd340d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.800+0000 7efd2bfff640 1 -- 172.21.15.19:0/2542205757 >> v2:172.21.15.112:3300/0 conn(0x7efd340d1020 msgr2=0x7efd340d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.800+0000 7efd2bfff640 1 --2- 172.21.15.19:0/2542205757 >> v2:172.21.15.112:3300/0 conn(0x7efd340d1020 0x7efd340d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.800+0000 7efd2bfff640 1 -- 172.21.15.19:0/2542205757 >> v1:172.21.15.112:6789/0 conn(0x7efd3415dbe0 legacy=0x7efd340d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.800+0000 7efd2bfff640 1 -- 172.21.15.19:0/2542205757 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340d1c50 con 0x7efd340b2cb0 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.800+0000 7efd2bfff640 1 --2- 172.21.15.19:0/2542205757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd340b2cb0 0x7efd340d2730 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7efd340cf570 tx=0x7efd1c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e122f7bdfb67f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.800+0000 7efd327fc640 1 -- 172.21.15.19:0/2542205757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c037070 con 0x7efd340b2cb0 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.800+0000 7efd327fc640 1 -- 172.21.15.19:0/2542205757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1c032440 con 0x7efd340b2cb0 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.800+0000 7efd327fc640 1 -- 172.21.15.19:0/2542205757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c032810 con 0x7efd340b2cb0 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.801+0000 7efd39549640 1 -- 172.21.15.19:0/2542205757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd340b2cb0 msgr2=0x7efd340d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.801+0000 7efd39549640 1 --2- 172.21.15.19:0/2542205757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd340b2cb0 0x7efd340d2730 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7efd340cf570 tx=0x7efd1c02a230 comp rx=0 tx=0).stop 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.801+0000 7efd39549640 1 -- 172.21.15.19:0/2542205757 shutdown_connections 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.801+0000 7efd39549640 1 --2- 172.21.15.19:0/2542205757 >> v2:172.21.15.112:3300/0 conn(0x7efd340d1020 0x7efd340d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.801+0000 7efd39549640 1 --2- 172.21.15.19:0/2542205757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd340b2cb0 0x7efd340d2730 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.801+0000 7efd39549640 1 -- 172.21.15.19:0/2542205757 >> 172.21.15.19:0/2542205757 conn(0x7efd34061a20 msgr2=0x7efd34061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.801+0000 7efd39549640 1 -- 172.21.15.19:0/2542205757 shutdown_connections 2024-04-25T10:01:56.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.801+0000 7efd39549640 1 -- 172.21.15.19:0/2542205757 wait complete. 2024-04-25T10:01:56.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.802+0000 7efd39549640 1 Processor -- start 2024-04-25T10:01:56.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.802+0000 7efd39549640 1 -- start start 2024-04-25T10:01:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.803+0000 7efd39549640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd340b2cb0 0x7efd340cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.803+0000 7efd39549640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd340d1020 0x7efd340b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.803+0000 7efd39549640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd3415dbe0 0x7efd340b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.803+0000 7efd39549640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd340b92a0 con 0x7efd340d1020 2024-04-25T10:01:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.803+0000 7efd39549640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd340d2210 con 0x7efd340b2cb0 2024-04-25T10:01:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.803+0000 7efd39549640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd340b9420 con 0x7efd3415dbe0 2024-04-25T10:01:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.803+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd340d1020 0x7efd340b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.803+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd340d1020 0x7efd340b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39130/0 (socket says 172.21.15.19:39130) 2024-04-25T10:01:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.803+0000 7efd2bfff640 1 -- 172.21.15.19:0/3420603659 learned_addr learned my addr 172.21.15.19:0/3420603659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.803+0000 7efd32ffd640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd340b2cb0 0x7efd340cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.803+0000 7efd337fe640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd3415dbe0 0x7efd340b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.804+0000 7efd2bfff640 1 -- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd3415dbe0 msgr2=0x7efd340b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.804+0000 7efd2bfff640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd3415dbe0 0x7efd340b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.804+0000 7efd2bfff640 1 -- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd340b2cb0 msgr2=0x7efd340cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.804+0000 7efd2bfff640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd340b2cb0 0x7efd340cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.804+0000 7efd2bfff640 1 -- 172.21.15.19:0/3420603659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340ccfa0 con 0x7efd340d1020 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.804+0000 7efd2bfff640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd340d1020 0x7efd340b4de0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7efd1c000c00 tx=0x7efd1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.804+0000 7efd2b7fe640 1 -- 172.21.15.19:0/3420603659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c037040 con 0x7efd340d1020 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.804+0000 7efd2b7fe640 1 -- 172.21.15.19:0/3420603659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1c042070 con 0x7efd340d1020 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.804+0000 7efd39549640 1 -- 172.21.15.19:0/3420603659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd340cd280 con 0x7efd340d1020 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.804+0000 7efd2b7fe640 1 -- 172.21.15.19:0/3420603659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c030bb0 con 0x7efd340d1020 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.804+0000 7efd39549640 1 -- 172.21.15.19:0/3420603659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd340cd840 con 0x7efd340d1020 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.805+0000 7efd39549640 1 -- 172.21.15.19:0/3420603659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd34073990 con 0x7efd340d1020 2024-04-25T10:01:56.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.805+0000 7efd2b7fe640 1 -- 172.21.15.19:0/3420603659 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd1c047460 con 0x7efd340d1020 2024-04-25T10:01:56.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.806+0000 7efd2b7fe640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.806+0000 7efd2b7fe640 1 -- 172.21.15.19:0/3420603659 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd1c0aae90 con 0x7efd340d1020 2024-04-25T10:01:56.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.806+0000 7efd32ffd640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.807+0000 7efd32ffd640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 0x7efd1406d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efd341f70e0 tx=0x7efd24009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:56.810+0000 7efd2b7fe640 1 -- 172.21.15.19:0/3420603659 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd1c079530 con 0x7efd340d1020 2024-04-25T10:01:57.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.061+0000 7efd39549640 1 -- 172.21.15.19:0/3420603659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efd3415ae50 con 0x7efd340d1020 2024-04-25T10:01:57.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.061+0000 7efd2b7fe640 1 -- 172.21.15.19:0/3420603659 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7efd1c07d5d0 con 0x7efd340d1020 2024-04-25T10:01:57.066 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:57.066 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":554,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":129,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2569338880,"bytes_avail":770524774400,"bytes_total":773094113280,"write_bytes_sec":89651697,"read_op_per_sec":0,"write_op_per_sec":88},"fsmap":{"epoch":79,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9213}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T10:01:55.033579+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:01:57.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 -- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 msgr2=0x7efd1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 0x7efd1406d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efd341f70e0 tx=0x7efd24009060 comp rx=0 tx=0).stop 2024-04-25T10:01:57.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 -- 172.21.15.19:0/3420603659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd340d1020 msgr2=0x7efd340b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd340d1020 0x7efd340b4de0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7efd1c000c00 tx=0x7efd1c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 -- 172.21.15.19:0/3420603659 shutdown_connections 2024-04-25T10:01:57.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd3415dbe0 0x7efd340b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd340d1020 0x7efd340b4de0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 --2- 172.21.15.19:0/3420603659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd340b2cb0 0x7efd340cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 -- 172.21.15.19:0/3420603659 >> 172.21.15.19:0/3420603659 conn(0x7efd34061a20 msgr2=0x7efd3415a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 -- 172.21.15.19:0/3420603659 shutdown_connections 2024-04-25T10:01:57.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.063+0000 7efd39549640 1 -- 172.21.15.19:0/3420603659 wait complete. 2024-04-25T10:01:57.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:01:57.079 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:01:57.079 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:57.089 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:01:57.089 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'a', 'd', 'b'} 2024-04-25T10:01:57.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T10:01:57.237 INFO:teuthology.orchestra.run.smithi019.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T10:01:57.237 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T10:01:57.237 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:57.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.334+0000 7ff411b2b640 1 Processor -- start 2024-04-25T10:01:57.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.334+0000 7ff411b2b640 1 -- start start 2024-04-25T10:01:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.335+0000 7ff411b2b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff40c0d2e90 0x7ff40c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.335+0000 7ff411b2b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 0x7ff40c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.335+0000 7ff411b2b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40c0b9940 con 0x7ff40c15dd30 2024-04-25T10:01:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.335+0000 7ff411b2b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40c0d0d40 con 0x7ff40c0d2e90 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.335+0000 7ff411b2b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c0d1df0 con 0x7ff40c0d0900 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.335+0000 7ff40bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 0x7ff40c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.335+0000 7ff40affd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff40c0d2e90 0x7ff40c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.336+0000 7ff40bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 0x7ff40c0b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:46256/0 (socket says 172.21.15.19:46256) 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.336+0000 7ff40affd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff40c0d2e90 0x7ff40c0d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:40872/0 (socket says 172.21.15.19:40872) 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.336+0000 7ff40bfff640 1 -- 172.21.15.19:0/2567608212 learned_addr learned my addr 172.21.15.19:0/2567608212 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.336+0000 7ff40b7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ff40c15dd30 0x7ff40c0b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:47986/0 (socket says 172.21.15.19:47986) 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.336+0000 7ff40bfff640 1 -- 172.21.15.19:0/2567608212 >> v2:172.21.15.112:3300/0 conn(0x7ff40c0d2e90 msgr2=0x7ff40c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.336+0000 7ff40bfff640 1 --2- 172.21.15.19:0/2567608212 >> v2:172.21.15.112:3300/0 conn(0x7ff40c0d2e90 0x7ff40c0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.336+0000 7ff40bfff640 1 -- 172.21.15.19:0/2567608212 >> v1:172.21.15.112:6789/0 conn(0x7ff40c15dd30 legacy=0x7ff40c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.336+0000 7ff40bfff640 1 -- 172.21.15.19:0/2567608212 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0d20d0 con 0x7ff40c0d0900 2024-04-25T10:01:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.336+0000 7ff40affd640 1 --2- 172.21.15.19:0/2567608212 >> v2:172.21.15.112:3300/0 conn(0x7ff40c0d2e90 0x7ff40c0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.337+0000 7ff40bfff640 1 --2- 172.21.15.19:0/2567608212 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 0x7ff40c0b6a40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff404009e50 tx=0x7ff40402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3658820f00553cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.337+0000 7ff40a7fc640 1 -- 172.21.15.19:0/2567608212 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404036070 con 0x7ff40c0d0900 2024-04-25T10:01:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.337+0000 7ff40a7fc640 1 -- 172.21.15.19:0/2567608212 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff404031600 con 0x7ff40c0d0900 2024-04-25T10:01:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.337+0000 7ff40a7fc640 1 -- 172.21.15.19:0/2567608212 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4040319d0 con 0x7ff40c0d0900 2024-04-25T10:01:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.337+0000 7ff411b2b640 1 -- 172.21.15.19:0/2567608212 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 msgr2=0x7ff40c0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.337+0000 7ff411b2b640 1 --2- 172.21.15.19:0/2567608212 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 0x7ff40c0b6a40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff404009e50 tx=0x7ff40402a680 comp rx=0 tx=0).stop 2024-04-25T10:01:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.337+0000 7ff411b2b640 1 -- 172.21.15.19:0/2567608212 shutdown_connections 2024-04-25T10:01:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.337+0000 7ff411b2b640 1 --2- 172.21.15.19:0/2567608212 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 0x7ff40c0b6a40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.337+0000 7ff411b2b640 1 --2- 172.21.15.19:0/2567608212 >> v2:172.21.15.112:3300/0 conn(0x7ff40c0d2e90 0x7ff40c0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.337+0000 7ff411b2b640 1 -- 172.21.15.19:0/2567608212 >> 172.21.15.19:0/2567608212 conn(0x7ff40c061c30 msgr2=0x7ff40c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.338+0000 7ff411b2b640 1 -- 172.21.15.19:0/2567608212 shutdown_connections 2024-04-25T10:01:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.338+0000 7ff411b2b640 1 -- 172.21.15.19:0/2567608212 wait complete. 2024-04-25T10:01:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.339+0000 7ff411b2b640 1 Processor -- start 2024-04-25T10:01:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.339+0000 7ff411b2b640 1 -- start start 2024-04-25T10:01:57.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff411b2b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 0x7ff40c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff411b2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff40c0d2e90 0x7ff40c0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff411b2b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c15dd30 0x7ff40c0b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff411b2b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c0d2970 con 0x7ff40c15dd30 2024-04-25T10:01:57.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff411b2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c0d2690 con 0x7ff40c0d2e90 2024-04-25T10:01:57.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff411b2b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c0d1a10 con 0x7ff40c0d0900 2024-04-25T10:01:57.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c15dd30 0x7ff40c0b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c15dd30 0x7ff40c0b7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39138/0 (socket says 172.21.15.19:39138) 2024-04-25T10:01:57.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40bfff640 1 -- 172.21.15.19:0/567103848 learned_addr learned my addr 172.21.15.19:0/567103848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:01:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40affd640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff40c0d2e90 0x7ff40c0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40b7fe640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 0x7ff40c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40bfff640 1 -- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 msgr2=0x7ff40c0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40bfff640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 0x7ff40c0b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40bfff640 1 -- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff40c0d2e90 msgr2=0x7ff40c0b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40bfff640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff40c0d2e90 0x7ff40c0b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40bfff640 1 -- 172.21.15.19:0/567103848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0b3d50 con 0x7ff40c15dd30 2024-04-25T10:01:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40affd640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff40c0d2e90 0x7ff40c0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff40bfff640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c15dd30 0x7ff40c0b7f90 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff404034040 tx=0x7ff404004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff3f3fff640 1 -- 172.21.15.19:0/567103848 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404036050 con 0x7ff40c15dd30 2024-04-25T10:01:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff3f3fff640 1 -- 172.21.15.19:0/567103848 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff404042070 con 0x7ff40c15dd30 2024-04-25T10:01:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff411b2b640 1 -- 172.21.15.19:0/567103848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c1f6c20 con 0x7ff40c15dd30 2024-04-25T10:01:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.340+0000 7ff3f3fff640 1 -- 172.21.15.19:0/567103848 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404009510 con 0x7ff40c15dd30 2024-04-25T10:01:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.341+0000 7ff411b2b640 1 -- 172.21.15.19:0/567103848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c1f71e0 con 0x7ff40c15dd30 2024-04-25T10:01:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.341+0000 7ff411b2b640 1 -- 172.21.15.19:0/567103848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40c073990 con 0x7ff40c15dd30 2024-04-25T10:01:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.342+0000 7ff3f3fff640 1 -- 172.21.15.19:0/567103848 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff404007c70 con 0x7ff40c15dd30 2024-04-25T10:01:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.342+0000 7ff3f3fff640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3f806b4d0 0x7ff3f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.342+0000 7ff3f3fff640 1 -- 172.21.15.19:0/567103848 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4040ab220 con 0x7ff40c15dd30 2024-04-25T10:01:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.343+0000 7ff40b7fe640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3f806b4d0 0x7ff3f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.343+0000 7ff40b7fe640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3f806b4d0 0x7ff3f806d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc000f60 tx=0x7ff3fc007ce0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.347+0000 7ff3f3fff640 1 -- 172.21.15.19:0/567103848 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff404079f00 con 0x7ff40c15dd30 2024-04-25T10:01:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.563+0000 7ff411b2b640 1 -- 172.21.15.19:0/567103848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff40c1608b0 con 0x7ff40c15dd30 2024-04-25T10:01:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.563+0000 7ff3f3fff640 1 -- 172.21.15.19:0/567103848 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ff404079800 con 0x7ff40c15dd30 2024-04-25T10:01:57.568 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:01:57.568 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:01:30.999245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/250977615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":250977615},{"type":"v1","addr":"172.21.15.19:6835","nonce":250977615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:01:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 79 2024-04-25T10:01:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 -- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3f806b4d0 msgr2=0x7ff3f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3f806b4d0 0x7ff3f806d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc000f60 tx=0x7ff3fc007ce0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 -- 172.21.15.19:0/567103848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c15dd30 msgr2=0x7ff40c0b7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c15dd30 0x7ff40c0b7f90 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff404034040 tx=0x7ff404004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 -- 172.21.15.19:0/567103848 shutdown_connections 2024-04-25T10:01:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff3f806b4d0 0x7ff3f806d990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c15dd30 0x7ff40c0b7f90 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff40c0d2e90 0x7ff40c0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 --2- 172.21.15.19:0/567103848 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff40c0d0900 0x7ff40c0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 -- 172.21.15.19:0/567103848 >> 172.21.15.19:0/567103848 conn(0x7ff40c061c30 msgr2=0x7ff40c0c7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 -- 172.21.15.19:0/567103848 shutdown_connections 2024-04-25T10:01:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:01:57.565+0000 7ff411b2b640 1 -- 172.21.15.19:0/567103848 wait complete. 2024-04-25T10:01:57.581 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:02:02.582 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.691+0000 7f0bbf21f640 1 Processor -- start 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.691+0000 7f0bbf21f640 1 -- start start 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.692+0000 7f0bbf21f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bb80cfec0 0x7f0bb80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.692+0000 7f0bbf21f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb8071480 con 0x7f0bb8154ee0 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.692+0000 7f0bbf21f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb8071760 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.692+0000 7f0bbf21f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb80ce830 con 0x7f0bb80cfec0 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.692+0000 7f0bb7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bb80cfec0 0x7f0bb80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.692+0000 7f0bbcfbb640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0bb8154ee0 0x7f0bb80cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54092/0 (socket says 172.21.15.19:54092) 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.692+0000 7f0bbcfbb640 1 -- 172.21.15.19:0/915359322 learned_addr learned my addr 172.21.15.19:0/915359322 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.693+0000 7f0bb7fff640 1 -- 172.21.15.19:0/915359322 >> v1:172.21.15.112:6789/0 conn(0x7f0bb80ce3c0 legacy=0x7f0bb806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.693+0000 7f0bb7fff640 1 -- 172.21.15.19:0/915359322 >> v1:172.21.15.19:6789/0 conn(0x7f0bb8154ee0 legacy=0x7f0bb80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.693+0000 7f0bb7fff640 1 -- 172.21.15.19:0/915359322 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8071a40 con 0x7f0bb80cfec0 2024-04-25T10:02:02.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.693+0000 7f0bb77fe640 1 -- 172.21.15.19:0/915359322 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087953712 0 0) 0x7f0bb8071480 con 0x7f0bb8154ee0 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.693+0000 7f0bb7fff640 1 --2- 172.21.15.19:0/915359322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bb80cfec0 0x7f0bb80cde80 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0097a0 tx=0x7f0bac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499edd31931fe5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.693+0000 7f0bb77fe640 1 -- 172.21.15.19:0/915359322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac037070 con 0x7f0bb80cfec0 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.694+0000 7f0bb77fe640 1 -- 172.21.15.19:0/915359322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bac032440 con 0x7f0bb80cfec0 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.694+0000 7f0bb77fe640 1 -- 172.21.15.19:0/915359322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac032810 con 0x7f0bb80cfec0 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.694+0000 7f0bbf21f640 1 -- 172.21.15.19:0/915359322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bb80cfec0 msgr2=0x7f0bb80cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.694+0000 7f0bbf21f640 1 --2- 172.21.15.19:0/915359322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bb80cfec0 0x7f0bb80cde80 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0097a0 tx=0x7f0bac02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.694+0000 7f0bbf21f640 1 -- 172.21.15.19:0/915359322 shutdown_connections 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.694+0000 7f0bbf21f640 1 --2- 172.21.15.19:0/915359322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bb80cfec0 0x7f0bb80cde80 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.694+0000 7f0bbf21f640 1 -- 172.21.15.19:0/915359322 >> 172.21.15.19:0/915359322 conn(0x7f0bb806abe0 msgr2=0x7f0bb806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.694+0000 7f0bbf21f640 1 -- 172.21.15.19:0/915359322 shutdown_connections 2024-04-25T10:02:02.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.694+0000 7f0bbf21f640 1 -- 172.21.15.19:0/915359322 wait complete. 2024-04-25T10:02:02.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.695+0000 7f0bbf21f640 1 Processor -- start 2024-04-25T10:02:02.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.695+0000 7f0bbf21f640 1 -- start start 2024-04-25T10:02:02.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.696+0000 7f0bbf21f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bb80ce3c0 0x7f0bb81e3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.696+0000 7f0bbf21f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bb80cfec0 0x7f0bb8149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.696+0000 7f0bbf21f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bb8154ee0 0x7f0bb81f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.696+0000 7f0bbf21f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb8071230 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.696+0000 7f0bbf21f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb81552c0 con 0x7f0bb8154ee0 2024-04-25T10:02:02.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.696+0000 7f0bbf21f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb8155440 con 0x7f0bb80cfec0 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.696+0000 7f0bbcfbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bb80ce3c0 0x7f0bb81e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.696+0000 7f0bbcfbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bb80ce3c0 0x7f0bb81e3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44524/0 (socket says 172.21.15.19:44524) 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.696+0000 7f0bbcfbb640 1 -- 172.21.15.19:0/2485113613 learned_addr learned my addr 172.21.15.19:0/2485113613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bbd7bc640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bb8154ee0 0x7f0bb81f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bb7fff640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bb80cfec0 0x7f0bb8149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bbcfbb640 1 -- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bb80cfec0 msgr2=0x7f0bb8149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bbcfbb640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bb80cfec0 0x7f0bb8149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bbcfbb640 1 -- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bb8154ee0 msgr2=0x7f0bb81f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bbcfbb640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bb8154ee0 0x7f0bb81f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bbcfbb640 1 -- 172.21.15.19:0/2485113613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb80adab0 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bbd7bc640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bb8154ee0 0x7f0bb81f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bbcfbb640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bb80ce3c0 0x7f0bb81e3180 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000c7c0 tx=0x7f0bb000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bb57fa640 1 -- 172.21.15.19:0/2485113613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0018070 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bb57fa640 1 -- 172.21.15.19:0/2485113613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb000a640 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bbf21f640 1 -- 172.21.15.19:0/2485113613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb80add90 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bb57fa640 1 -- 172.21.15.19:0/2485113613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb000aa30 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.697+0000 7f0bbf21f640 1 -- 172.21.15.19:0/2485113613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb80ae350 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.698+0000 7f0bbf21f640 1 -- 172.21.15.19:0/2485113613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb8073990 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.699+0000 7f0bb57fa640 1 -- 172.21.15.19:0/2485113613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bb001b070 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.699+0000 7f0bb57fa640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ba406b4d0 0x7f0ba406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.699+0000 7f0bb57fa640 1 -- 172.21.15.19:0/2485113613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bb00899c0 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.700+0000 7f0bb7fff640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ba406b4d0 0x7f0ba406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.700+0000 7f0bb7fff640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ba406b4d0 0x7f0ba406d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0bac034000 tx=0x7f0bac02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.704+0000 7f0bb57fa640 1 -- 172.21.15.19:0/2485113613 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0bb008be20 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.922+0000 7f0bbf21f640 1 -- 172.21.15.19:0/2485113613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bb806d360 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.923+0000 7f0bb57fa640 1 -- 172.21.15.19:0/2485113613 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0bb00586b0 con 0x7f0bb80ce3c0 2024-04-25T10:02:02.927 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:02.928 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:01:30.999245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/250977615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":250977615},{"type":"v1","addr":"172.21.15.19:6835","nonce":250977615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:02.928 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 79 2024-04-25T10:02:02.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 -- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ba406b4d0 msgr2=0x7f0ba406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ba406b4d0 0x7f0ba406d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0bac034000 tx=0x7f0bac02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 -- 172.21.15.19:0/2485113613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bb80ce3c0 msgr2=0x7f0bb81e3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bb80ce3c0 0x7f0bb81e3180 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000c7c0 tx=0x7f0bb000cc80 comp rx=0 tx=0).stop 2024-04-25T10:02:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 -- 172.21.15.19:0/2485113613 shutdown_connections 2024-04-25T10:02:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0ba406b4d0 0x7f0ba406d990 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bb8154ee0 0x7f0bb81f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bb80cfec0 0x7f0bb8149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 --2- 172.21.15.19:0/2485113613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bb80ce3c0 0x7f0bb81e3180 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 -- 172.21.15.19:0/2485113613 >> 172.21.15.19:0/2485113613 conn(0x7f0bb806abe0 msgr2=0x7f0bb806de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 -- 172.21.15.19:0/2485113613 shutdown_connections 2024-04-25T10:02:02.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:02.925+0000 7f0bbf21f640 1 -- 172.21.15.19:0/2485113613 wait complete. 2024-04-25T10:02:02.941 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:02:07.942 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:08.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.052+0000 7f31374cb640 1 Processor -- start 2024-04-25T10:02:08.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.052+0000 7f31374cb640 1 -- start start 2024-04-25T10:02:08.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f31374cb640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f31300c6df0 0x7f31300d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f31374cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31300d1c00 0x7f31300ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f31374cb640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313015dfe0 con 0x7f313015dc00 2024-04-25T10:02:08.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f31374cb640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31300d2090 con 0x7f31300c6df0 2024-04-25T10:02:08.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f31374cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31300d00c0 con 0x7f31300d1c00 2024-04-25T10:02:08.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f3134a66640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f31300c6df0 0x7f31300d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f3135267640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f313015dc00 0x7f31300ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60024/0 (socket says 172.21.15.19:60024) 2024-04-25T10:02:08.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f3135267640 1 -- 172.21.15.19:0/65026313 learned_addr learned my addr 172.21.15.19:0/65026313 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f3135a68640 1 --2- 172.21.15.19:0/65026313 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31300d1c00 0x7f31300ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f3134a66640 1 -- 172.21.15.19:0/65026313 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31300d1c00 msgr2=0x7f31300ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f3134a66640 1 --2- 172.21.15.19:0/65026313 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31300d1c00 0x7f31300ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f3134a66640 1 -- 172.21.15.19:0/65026313 >> v1:172.21.15.19:6789/0 conn(0x7f313015dc00 legacy=0x7f31300ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f3127fff640 1 -- 172.21.15.19:0/65026313 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137339375 0 0) 0x7f312c001110 con 0x7f313015dc00 2024-04-25T10:02:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.053+0000 7f3134a66640 1 -- 172.21.15.19:0/65026313 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31300d03a0 con 0x7f31300c6df0 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.054+0000 7f3134a66640 1 --2- 172.21.15.19:0/65026313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31300c6df0 0x7f31300d16c0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f31200097a0 tx=0x7f312002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504f01f9a4da9798 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.054+0000 7f3127fff640 1 -- 172.21.15.19:0/65026313 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120037070 con 0x7f31300c6df0 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.054+0000 7f3127fff640 1 -- 172.21.15.19:0/65026313 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3120032440 con 0x7f31300c6df0 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.054+0000 7f3127fff640 1 -- 172.21.15.19:0/65026313 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120032810 con 0x7f31300c6df0 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.055+0000 7f31374cb640 1 -- 172.21.15.19:0/65026313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31300c6df0 msgr2=0x7f31300d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.055+0000 7f31374cb640 1 --2- 172.21.15.19:0/65026313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31300c6df0 0x7f31300d16c0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f31200097a0 tx=0x7f312002a180 comp rx=0 tx=0).stop 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.055+0000 7f31374cb640 1 -- 172.21.15.19:0/65026313 shutdown_connections 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.055+0000 7f31374cb640 1 --2- 172.21.15.19:0/65026313 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31300d1c00 0x7f31300ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.055+0000 7f31374cb640 1 --2- 172.21.15.19:0/65026313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31300c6df0 0x7f31300d16c0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.055+0000 7f31374cb640 1 -- 172.21.15.19:0/65026313 >> 172.21.15.19:0/65026313 conn(0x7f313006f5d0 msgr2=0x7f313006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.055+0000 7f31374cb640 1 -- 172.21.15.19:0/65026313 shutdown_connections 2024-04-25T10:02:08.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.055+0000 7f31374cb640 1 -- 172.21.15.19:0/65026313 wait complete. 2024-04-25T10:02:08.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.056+0000 7f31374cb640 1 Processor -- start 2024-04-25T10:02:08.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.056+0000 7f31374cb640 1 -- start start 2024-04-25T10:02:08.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.057+0000 7f31374cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31300c6df0 0x7f313006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.057+0000 7f31374cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31300d1c00 0x7f31301e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.057+0000 7f31374cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f313015dc00 0x7f3130156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.057+0000 7f31374cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31300cc130 con 0x7f313015dc00 2024-04-25T10:02:08.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.057+0000 7f31374cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31300cc720 con 0x7f31300c6df0 2024-04-25T10:02:08.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.057+0000 7f31374cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31300cfd90 con 0x7f31300d1c00 2024-04-25T10:02:08.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.057+0000 7f3135a68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f313015dc00 0x7f3130156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.057+0000 7f3135a68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f313015dc00 0x7f3130156ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46434/0 (socket says 172.21.15.19:46434) 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.057+0000 7f3135a68640 1 -- 172.21.15.19:0/1736104491 learned_addr learned my addr 172.21.15.19:0/1736104491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.057+0000 7f3135267640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31300c6df0 0x7f313006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.057+0000 7f3134a66640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31300d1c00 0x7f31301e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.058+0000 7f3135a68640 1 -- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31300d1c00 msgr2=0x7f31301e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.058+0000 7f3135a68640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31300d1c00 0x7f31301e8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.058+0000 7f3135a68640 1 -- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31300c6df0 msgr2=0x7f313006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.058+0000 7f3135a68640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31300c6df0 0x7f313006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.058+0000 7f3135a68640 1 -- 172.21.15.19:0/1736104491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31301e9600 con 0x7f313015dc00 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.058+0000 7f3135a68640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f313015dc00 0x7f3130156ce0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f3128009dd0 tx=0x7f312800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.058+0000 7f3125ffb640 1 -- 172.21.15.19:0/1736104491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128018020 con 0x7f313015dc00 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.058+0000 7f3125ffb640 1 -- 172.21.15.19:0/1736104491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3128014480 con 0x7f313015dc00 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.058+0000 7f31374cb640 1 -- 172.21.15.19:0/1736104491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31301f7940 con 0x7f313015dc00 2024-04-25T10:02:08.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.058+0000 7f3125ffb640 1 -- 172.21.15.19:0/1736104491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128014870 con 0x7f313015dc00 2024-04-25T10:02:08.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.058+0000 7f31374cb640 1 -- 172.21.15.19:0/1736104491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31301f7f00 con 0x7f313015dc00 2024-04-25T10:02:08.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.059+0000 7f31374cb640 1 -- 172.21.15.19:0/1736104491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3130073990 con 0x7f313015dc00 2024-04-25T10:02:08.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.060+0000 7f3125ffb640 1 -- 172.21.15.19:0/1736104491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f312801b020 con 0x7f313015dc00 2024-04-25T10:02:08.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.060+0000 7f3125ffb640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f311406b4d0 0x7f311406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.060+0000 7f3125ffb640 1 -- 172.21.15.19:0/1736104491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f312808ad50 con 0x7f313015dc00 2024-04-25T10:02:08.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.061+0000 7f3135267640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f311406b4d0 0x7f311406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.061+0000 7f3135267640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f311406b4d0 0x7f311406d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f312c007fd0 tx=0x7f312c007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:08.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.065+0000 7f3125ffb640 1 -- 172.21.15.19:0/1736104491 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f312808e030 con 0x7f313015dc00 2024-04-25T10:02:08.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.282+0000 7f31374cb640 1 -- 172.21.15.19:0/1736104491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31300c66c0 con 0x7f313015dc00 2024-04-25T10:02:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.283+0000 7f3125ffb640 1 -- 172.21.15.19:0/1736104491 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3128022030 con 0x7f313015dc00 2024-04-25T10:02:08.288 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:08.288 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:01:30.999245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/250977615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":250977615},{"type":"v1","addr":"172.21.15.19:6835","nonce":250977615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 79 2024-04-25T10:02:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 -- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f311406b4d0 msgr2=0x7f311406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f311406b4d0 0x7f311406d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f312c007fd0 tx=0x7f312c007de0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 -- 172.21.15.19:0/1736104491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f313015dc00 msgr2=0x7f3130156ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f313015dc00 0x7f3130156ce0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f3128009dd0 tx=0x7f312800c490 comp rx=0 tx=0).stop 2024-04-25T10:02:08.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 -- 172.21.15.19:0/1736104491 shutdown_connections 2024-04-25T10:02:08.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f311406b4d0 0x7f311406d990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f313015dc00 0x7f3130156ce0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f31300d1c00 0x7f31301e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 --2- 172.21.15.19:0/1736104491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31300c6df0 0x7f313006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 -- 172.21.15.19:0/1736104491 >> 172.21.15.19:0/1736104491 conn(0x7f313006f5d0 msgr2=0x7f313015b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:08.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 -- 172.21.15.19:0/1736104491 shutdown_connections 2024-04-25T10:02:08.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:08.285+0000 7f31374cb640 1 -- 172.21.15.19:0/1736104491 wait complete. 2024-04-25T10:02:08.301 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:02:08.614 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:08.617 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:13.302 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:13.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.411+0000 7f14d3ed1640 1 Processor -- start 2024-04-25T10:02:13.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.412+0000 7f14d3ed1640 1 -- start start 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.412+0000 7f14d3ed1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f14cc06abe0 0x7f14cc06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.412+0000 7f14d3ed1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14cc073150 con 0x7f14cc15b9c0 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.412+0000 7f14d3ed1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14cc0cfa60 con 0x7f14cc072620 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.412+0000 7f14d3ed1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14cc0cfd40 con 0x7f14cc06abe0 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d1c6d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f14cc15b9c0 0x7f14cc15bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60032/0 (socket says 172.21.15.19:60032) 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d1c6d640 1 -- 172.21.15.19:0/2506483985 learned_addr learned my addr 172.21.15.19:0/2506483985 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d146c640 1 --2- 172.21.15.19:0/2506483985 >> v2:172.21.15.112:3300/0 conn(0x7f14cc06abe0 0x7f14cc06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991410831 0 0) 0x7f14cc073150 con 0x7f14cc15b9c0 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14b4003520 con 0x7f14cc15b9c0 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658717398 0 0) 0x7f14cc0cfa60 con 0x7f14cc072620 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14cc073150 con 0x7f14cc072620 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4160458270 0 0) 0x7f14b4003520 con 0x7f14cc15b9c0 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 >> v2:172.21.15.112:3300/0 conn(0x7f14cc06abe0 msgr2=0x7f14cc06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 --2- 172.21.15.19:0/2506483985 >> v2:172.21.15.112:3300/0 conn(0x7f14cc06abe0 0x7f14cc06b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 >> v1:172.21.15.112:6789/0 conn(0x7f14cc072620 legacy=0x7f14cc072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc0d0020 con 0x7f14cc15b9c0 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f14bc0019f0 con 0x7f14cc15b9c0 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 >> v1:172.21.15.19:6789/0 conn(0x7f14cc15b9c0 legacy=0x7f14cc15bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 --2- 172.21.15.19:0/2506483985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b4003b40 0x7f14b4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 --2- 172.21.15.19:0/2506483985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14b4004800 0x7f14b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 --2- 172.21.15.19:0/2506483985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14b40051c0 0x7f14b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0cfd40 con 0x7f14b4003b40 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b400c040 con 0x7f14b40051c0 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b400c240 con 0x7f14b4004800 2024-04-25T10:02:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.413+0000 7f14d1c6d640 1 --2- 172.21.15.19:0/2506483985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b4003b40 0x7f14b4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.414+0000 7f14d246e640 1 --2- 172.21.15.19:0/2506483985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14b40051c0 0x7f14b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.414+0000 7f14d146c640 1 --2- 172.21.15.19:0/2506483985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14b4004800 0x7f14b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.414+0000 7f14d1c6d640 1 -- 172.21.15.19:0/2506483985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14b4004800 msgr2=0x7f14b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.414+0000 7f14d1c6d640 1 --2- 172.21.15.19:0/2506483985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14b4004800 0x7f14b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.414+0000 7f14d1c6d640 1 -- 172.21.15.19:0/2506483985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14b40051c0 msgr2=0x7f14b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.414+0000 7f14d1c6d640 1 --2- 172.21.15.19:0/2506483985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14b40051c0 0x7f14b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.414+0000 7f14d246e640 1 -- 172.21.15.19:0/2506483985 reap_dead start 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.414+0000 7f14d1c6d640 1 -- 172.21.15.19:0/2506483985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b4003520 con 0x7f14b4003b40 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.415+0000 7f14d1c6d640 1 --2- 172.21.15.19:0/2506483985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b4003b40 0x7f14b4003f40 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00a820 tx=0x7f14b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c315141d2d65fffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.415+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc00a450 con 0x7f14b4003b40 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.415+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14bc02f070 con 0x7f14b4003b40 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.415+0000 7f14d0c6b640 1 -- 172.21.15.19:0/2506483985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc02bcd0 con 0x7f14b4003b40 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.415+0000 7f14d3ed1640 1 -- 172.21.15.19:0/2506483985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b4003b40 msgr2=0x7f14b4003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.415+0000 7f14d3ed1640 1 --2- 172.21.15.19:0/2506483985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b4003b40 0x7f14b4003f40 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00a820 tx=0x7f14b4005830 comp rx=0 tx=0).stop 2024-04-25T10:02:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.415+0000 7f14d3ed1640 1 -- 172.21.15.19:0/2506483985 shutdown_connections 2024-04-25T10:02:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.415+0000 7f14d3ed1640 1 --2- 172.21.15.19:0/2506483985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b4003b40 0x7f14b4003f40 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.415+0000 7f14d3ed1640 1 -- 172.21.15.19:0/2506483985 >> 172.21.15.19:0/2506483985 conn(0x7f14cc0b2370 msgr2=0x7f14cc0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.415+0000 7f14d3ed1640 1 -- 172.21.15.19:0/2506483985 shutdown_connections 2024-04-25T10:02:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.415+0000 7f14d3ed1640 1 -- 172.21.15.19:0/2506483985 wait complete. 2024-04-25T10:02:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.417+0000 7f14d3ed1640 1 Processor -- start 2024-04-25T10:02:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.417+0000 7f14d3ed1640 1 -- start start 2024-04-25T10:02:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.417+0000 7f14d3ed1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14b4003b40 0x7f14cc0b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.417+0000 7f14d3ed1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14cc1f7230 0x7f14cc0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.417+0000 7f14d3ed1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14cc1f7690 0x7f14cc0cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.417+0000 7f14d3ed1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc06b070 con 0x7f14cc1f7690 2024-04-25T10:02:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.417+0000 7f14d3ed1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc0cf710 con 0x7f14cc1f7230 2024-04-25T10:02:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.417+0000 7f14d3ed1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc06b1f0 con 0x7f14b4003b40 2024-04-25T10:02:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.417+0000 7f14d246e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14cc1f7690 0x7f14cc0cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d246e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14cc1f7690 0x7f14cc0cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46454/0 (socket says 172.21.15.19:46454) 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d246e640 1 -- 172.21.15.19:0/1184133626 learned_addr learned my addr 172.21.15.19:0/1184133626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d146c640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14cc1f7230 0x7f14cc0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d1c6d640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14b4003b40 0x7f14cc0b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d246e640 1 -- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14b4003b40 msgr2=0x7f14cc0b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d246e640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14b4003b40 0x7f14cc0b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d246e640 1 -- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14cc1f7230 msgr2=0x7f14cc0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d246e640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14cc1f7230 0x7f14cc0caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d246e640 1 -- 172.21.15.19:0/1184133626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc0b5790 con 0x7f14cc1f7690 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d246e640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14cc1f7690 0x7f14cc0cb000 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f14b4004780 tx=0x7f14c400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14ca7fc640 1 -- 172.21.15.19:0/1184133626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4019070 con 0x7f14cc1f7690 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14ca7fc640 1 -- 172.21.15.19:0/1184133626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c4005030 con 0x7f14cc1f7690 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14ca7fc640 1 -- 172.21.15.19:0/1184133626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c40053e0 con 0x7f14cc1f7690 2024-04-25T10:02:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d3ed1640 1 -- 172.21.15.19:0/1184133626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14cc0b3ed0 con 0x7f14cc1f7690 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.418+0000 7f14d3ed1640 1 -- 172.21.15.19:0/1184133626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14cc0b4490 con 0x7f14cc1f7690 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.419+0000 7f14d3ed1640 1 -- 172.21.15.19:0/1184133626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14cc073990 con 0x7f14cc1f7690 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.420+0000 7f14ca7fc640 1 -- 172.21.15.19:0/1184133626 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14c40067d0 con 0x7f14cc1f7690 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.420+0000 7f14ca7fc640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14b806b500 0x7f14b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.420+0000 7f14ca7fc640 1 -- 172.21.15.19:0/1184133626 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f14c4089520 con 0x7f14cc1f7690 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.420+0000 7f14d1c6d640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14b806b500 0x7f14b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.421+0000 7f14d1c6d640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14b806b500 0x7f14b806d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f14bc033000 tx=0x7f14bc02d000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.425+0000 7f14ca7fc640 1 -- 172.21.15.19:0/1184133626 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14c4058120 con 0x7f14cc1f7690 2024-04-25T10:02:13.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.648+0000 7f14d3ed1640 1 -- 172.21.15.19:0/1184133626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14cc155d80 con 0x7f14cc1f7690 2024-04-25T10:02:13.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.649+0000 7f14ca7fc640 1 -- 172.21.15.19:0/1184133626 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4161 (secure 0 0 0) 0x7f14c4057a20 con 0x7f14cc1f7690 2024-04-25T10:02:13.654 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:13.654 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:02:12.709067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9210":{"gid":9210,"name":"a","rank":0,"incarnation":80,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:13.654 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 81 2024-04-25T10:02:13.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 -- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14b806b500 msgr2=0x7f14b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14b806b500 0x7f14b806d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f14bc033000 tx=0x7f14bc02d000 comp rx=0 tx=0).stop 2024-04-25T10:02:13.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 -- 172.21.15.19:0/1184133626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14cc1f7690 msgr2=0x7f14cc0cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14cc1f7690 0x7f14cc0cb000 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f14b4004780 tx=0x7f14c400fc00 comp rx=0 tx=0).stop 2024-04-25T10:02:13.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 -- 172.21.15.19:0/1184133626 shutdown_connections 2024-04-25T10:02:13.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f14b806b500 0x7f14b806d9c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14cc1f7690 0x7f14cc0cb000 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14cc1f7230 0x7f14cc0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 --2- 172.21.15.19:0/1184133626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14b4003b40 0x7f14cc0b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 -- 172.21.15.19:0/1184133626 >> 172.21.15.19:0/1184133626 conn(0x7f14cc0b2370 msgr2=0x7f14cc0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 -- 172.21.15.19:0/1184133626 shutdown_connections 2024-04-25T10:02:13.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:13.651+0000 7f14d3ed1640 1 -- 172.21.15.19:0/1184133626 wait complete. 2024-04-25T10:02:13.666 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:02:18.448 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T10:02:18.444+0000 7f195ebfa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:02:18.668 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:18.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.776+0000 7f95dbff9640 1 Processor -- start 2024-04-25T10:02:18.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.777+0000 7f95dbff9640 1 -- start start 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.777+0000 7f95dbff9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95d40b5f00 0x7f95d40ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.777+0000 7f95dbff9640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d40cec00 con 0x7f95d40d0b10 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.777+0000 7f95dbff9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d40ceee0 con 0x7f95d415db60 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.777+0000 7f95dbff9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d40ce930 con 0x7f95d40b5f00 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.778+0000 7f95d9594640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f95d40d0b10 0x7f95d40d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50282/0 (socket says 172.21.15.19:50282) 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.778+0000 7f95d9594640 1 -- 172.21.15.19:0/2456067071 learned_addr learned my addr 172.21.15.19:0/2456067071 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.778+0000 7f95da596640 1 --2- 172.21.15.19:0/2456067071 >> v2:172.21.15.112:3300/0 conn(0x7f95d40b5f00 0x7f95d40ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.778+0000 7f95d8d93640 1 -- 172.21.15.19:0/2456067071 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686192168 0 0) 0x7f95d40cec00 con 0x7f95d40d0b10 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.778+0000 7f95d8d93640 1 -- 172.21.15.19:0/2456067071 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c0003520 con 0x7f95d40d0b10 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.778+0000 7f95da596640 1 -- 172.21.15.19:0/2456067071 >> v1:172.21.15.112:6789/0 conn(0x7f95d415db60 legacy=0x7f95d40d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.778+0000 7f95da596640 1 -- 172.21.15.19:0/2456067071 >> v1:172.21.15.19:6789/0 conn(0x7f95d40d0b10 legacy=0x7f95d40d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.778+0000 7f95d8d93640 1 -- 172.21.15.19:0/2456067071 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3716422292 0 0) 0x7f95c0003520 con 0x7f95d40d0b10 2024-04-25T10:02:18.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.778+0000 7f95da596640 1 -- 172.21.15.19:0/2456067071 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40cf1c0 con 0x7f95d40b5f00 2024-04-25T10:02:18.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.779+0000 7f95da596640 1 --2- 172.21.15.19:0/2456067071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d40b5f00 0x7f95d40ce3f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009dc0 tx=0x7f95cc02a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=950b1d4dc537c6f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.779+0000 7f95d8d93640 1 -- 172.21.15.19:0/2456067071 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4001670 con 0x7f95d40b5f00 2024-04-25T10:02:18.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.779+0000 7f95d8d93640 1 -- 172.21.15.19:0/2456067071 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95cc007540 con 0x7f95d40b5f00 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.779+0000 7f95d8d93640 1 -- 172.21.15.19:0/2456067071 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc007910 con 0x7f95d40b5f00 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.780+0000 7f95dbff9640 1 -- 172.21.15.19:0/2456067071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d40b5f00 msgr2=0x7f95d40ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.780+0000 7f95dbff9640 1 --2- 172.21.15.19:0/2456067071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d40b5f00 0x7f95d40ce3f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009dc0 tx=0x7f95cc02a630 comp rx=0 tx=0).stop 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.780+0000 7f95dbff9640 1 -- 172.21.15.19:0/2456067071 shutdown_connections 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.780+0000 7f95dbff9640 1 --2- 172.21.15.19:0/2456067071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d40b5f00 0x7f95d40ce3f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.780+0000 7f95dbff9640 1 -- 172.21.15.19:0/2456067071 >> 172.21.15.19:0/2456067071 conn(0x7f95d4061600 msgr2=0x7f95d4061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.780+0000 7f95dbff9640 1 -- 172.21.15.19:0/2456067071 shutdown_connections 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.780+0000 7f95dbff9640 1 -- 172.21.15.19:0/2456067071 wait complete. 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.781+0000 7f95dbff9640 1 Processor -- start 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.781+0000 7f95dbff9640 1 -- start start 2024-04-25T10:02:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95dbff9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d40b5f00 0x7f95d4151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95dbff9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95d40d0b10 0x7f95d40c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95dbff9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95d415db60 0x7f95d40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95dbff9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d415df40 con 0x7f95d415db60 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95dbff9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d40b6450 con 0x7f95d40b5f00 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95dbff9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d415e0c0 con 0x7f95d40d0b10 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95da596640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95d415db60 0x7f95d40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95da596640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95d415db60 0x7f95d40cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53674/0 (socket says 172.21.15.19:53674) 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95da596640 1 -- 172.21.15.19:0/2403173045 learned_addr learned my addr 172.21.15.19:0/2403173045 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95d9594640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95d40d0b10 0x7f95d40c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95d9d95640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d40b5f00 0x7f95d4151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95da596640 1 -- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95d40d0b10 msgr2=0x7f95d40c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95da596640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95d40d0b10 0x7f95d40c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95da596640 1 -- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d40b5f00 msgr2=0x7f95d4151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95da596640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d40b5f00 0x7f95d4151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.782+0000 7f95da596640 1 -- 172.21.15.19:0/2403173045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40c91d0 con 0x7f95d415db60 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.783+0000 7f95da596640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95d415db60 0x7f95d40cd200 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009dc0 tx=0x7f95cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.783+0000 7f95ca7fc640 1 -- 172.21.15.19:0/2403173045 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc040070 con 0x7f95d415db60 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.783+0000 7f95ca7fc640 1 -- 172.21.15.19:0/2403173045 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95cc031bc0 con 0x7f95d415db60 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.783+0000 7f95dbff9640 1 -- 172.21.15.19:0/2403173045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d40cc720 con 0x7f95d415db60 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.783+0000 7f95ca7fc640 1 -- 172.21.15.19:0/2403173045 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc009450 con 0x7f95d415db60 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.783+0000 7f95dbff9640 1 -- 172.21.15.19:0/2403173045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d40ccce0 con 0x7f95d415db60 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.784+0000 7f95dbff9640 1 -- 172.21.15.19:0/2403173045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d4073270 con 0x7f95d415db60 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.784+0000 7f95ca7fc640 1 -- 172.21.15.19:0/2403173045 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95cc039040 con 0x7f95d415db60 2024-04-25T10:02:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.785+0000 7f95ca7fc640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95b806b4d0 0x7f95b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.785+0000 7f95ca7fc640 1 -- 172.21.15.19:0/2403173045 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f95cc0abd00 con 0x7f95d415db60 2024-04-25T10:02:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.785+0000 7f95d9d95640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95b806b4d0 0x7f95b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.785+0000 7f95d9d95640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95b806b4d0 0x7f95b806d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f95d41f71b0 tx=0x7f95d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:18.789+0000 7f95ca7fc640 1 -- 172.21.15.19:0/2403173045 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f95cc07a900 con 0x7f95d415db60 2024-04-25T10:02:19.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.008+0000 7f95dbff9640 1 -- 172.21.15.19:0/2403173045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95d40cc530 con 0x7f95d415db60 2024-04-25T10:02:19.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.009+0000 7f95ca7fc640 1 -- 172.21.15.19:0/2403173045 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f95cc07a200 con 0x7f95d415db60 2024-04-25T10:02:19.014 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:19.015 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:02:14.631627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9210":{"gid":9210,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":17,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:19.015 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-04-25T10:02:19.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 -- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95b806b4d0 msgr2=0x7f95b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95b806b4d0 0x7f95b806d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f95d41f71b0 tx=0x7f95d0009040 comp rx=0 tx=0).stop 2024-04-25T10:02:19.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 -- 172.21.15.19:0/2403173045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95d415db60 msgr2=0x7f95d40cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95d415db60 0x7f95d40cd200 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009dc0 tx=0x7f95cc004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:19.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 -- 172.21.15.19:0/2403173045 shutdown_connections 2024-04-25T10:02:19.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f95b806b4d0 0x7f95b806d990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95d415db60 0x7f95d40cd200 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95d40d0b10 0x7f95d40c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 --2- 172.21.15.19:0/2403173045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d40b5f00 0x7f95d4151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 -- 172.21.15.19:0/2403173045 >> 172.21.15.19:0/2403173045 conn(0x7f95d4061600 msgr2=0x7f95d40b52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 -- 172.21.15.19:0/2403173045 shutdown_connections 2024-04-25T10:02:19.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.011+0000 7f95dbff9640 1 -- 172.21.15.19:0/2403173045 wait complete. 2024-04-25T10:02:19.028 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-25T10:02:19.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.126+0000 7f62e6190640 1 Processor -- start 2024-04-25T10:02:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.126+0000 7f62e6190640 1 -- start start 2024-04-25T10:02:19.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.126+0000 7f62e6190640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f62e015dbe0 0x7f62e00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.126+0000 7f62e6190640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e00b92a0 0x7f62e00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.126+0000 7f62e6190640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e00b97a0 con 0x7f62e00b2cb0 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.126+0000 7f62e6190640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e015dfc0 con 0x7f62e015dbe0 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.126+0000 7f62e6190640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00d1a00 con 0x7f62e00b92a0 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.127+0000 7f62deffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f62e00b2cb0 0x7f62e00b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50294/0 (socket says 172.21.15.19:50294) 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.127+0000 7f62deffd640 1 -- 172.21.15.19:0/2419414292 learned_addr learned my addr 172.21.15.19:0/2419414292 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.127+0000 7f62dffff640 1 --2- 172.21.15.19:0/2419414292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e00b92a0 0x7f62e00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.127+0000 7f62df7fe640 1 --2- 172.21.15.19:0/2419414292 >> v2:172.21.15.112:3300/0 conn(0x7f62e015dbe0 0x7f62e00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.127+0000 7f62de7fc640 1 -- 172.21.15.19:0/2419414292 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083464919 0 0) 0x7f62e00b97a0 con 0x7f62e00b2cb0 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.127+0000 7f62de7fc640 1 -- 172.21.15.19:0/2419414292 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c8003520 con 0x7f62e00b2cb0 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.127+0000 7f62dffff640 1 -- 172.21.15.19:0/2419414292 >> v2:172.21.15.112:3300/0 conn(0x7f62e015dbe0 msgr2=0x7f62e00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.127+0000 7f62dffff640 1 --2- 172.21.15.19:0/2419414292 >> v2:172.21.15.112:3300/0 conn(0x7f62e015dbe0 0x7f62e00d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.127+0000 7f62dffff640 1 -- 172.21.15.19:0/2419414292 >> v1:172.21.15.19:6789/0 conn(0x7f62e00b2cb0 legacy=0x7f62e00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.127+0000 7f62de7fc640 1 -- 172.21.15.19:0/2419414292 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4117586184 0 0) 0x7f62c8003520 con 0x7f62e00b2cb0 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.127+0000 7f62dffff640 1 -- 172.21.15.19:0/2419414292 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e00d1c50 con 0x7f62e00b92a0 2024-04-25T10:02:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.128+0000 7f62dffff640 1 --2- 172.21.15.19:0/2419414292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e00b92a0 0x7f62e00d0e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009d90 tx=0x7f62d802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f63771a38b6106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.128+0000 7f62de7fc640 1 -- 172.21.15.19:0/2419414292 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4001670 con 0x7f62e00b92a0 2024-04-25T10:02:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.128+0000 7f62de7fc640 1 -- 172.21.15.19:0/2419414292 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d8007540 con 0x7f62e00b92a0 2024-04-25T10:02:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.128+0000 7f62de7fc640 1 -- 172.21.15.19:0/2419414292 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d80078f0 con 0x7f62e00b92a0 2024-04-25T10:02:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.129+0000 7f62e6190640 1 -- 172.21.15.19:0/2419414292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e00b92a0 msgr2=0x7f62e00d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.129+0000 7f62e6190640 1 --2- 172.21.15.19:0/2419414292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e00b92a0 0x7f62e00d0e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009d90 tx=0x7f62d802a5f0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.129+0000 7f62e6190640 1 -- 172.21.15.19:0/2419414292 shutdown_connections 2024-04-25T10:02:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.129+0000 7f62e6190640 1 --2- 172.21.15.19:0/2419414292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e00b92a0 0x7f62e00d0e50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.129+0000 7f62e6190640 1 --2- 172.21.15.19:0/2419414292 >> v2:172.21.15.112:3300/0 conn(0x7f62e015dbe0 0x7f62e00d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.129+0000 7f62e6190640 1 -- 172.21.15.19:0/2419414292 >> 172.21.15.19:0/2419414292 conn(0x7f62e0061a20 msgr2=0x7f62e0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.129+0000 7f62e6190640 1 -- 172.21.15.19:0/2419414292 shutdown_connections 2024-04-25T10:02:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.129+0000 7f62e6190640 1 -- 172.21.15.19:0/2419414292 wait complete. 2024-04-25T10:02:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.130+0000 7f62e6190640 1 Processor -- start 2024-04-25T10:02:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.130+0000 7f62e6190640 1 -- start start 2024-04-25T10:02:19.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62e6190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e00b2cb0 0x7f62e01f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62e6190640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e00b92a0 0x7f62e00b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62e6190640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e015dbe0 0x7f62e0160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62e6190640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00b6ab0 con 0x7f62e00b2cb0 2024-04-25T10:02:19.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62e6190640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00d2210 con 0x7f62e00b92a0 2024-04-25T10:02:19.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62e6190640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00b6c30 con 0x7f62e015dbe0 2024-04-25T10:02:19.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62df7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e00b2cb0 0x7f62e01f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62df7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e00b2cb0 0x7f62e01f2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53684/0 (socket says 172.21.15.19:53684) 2024-04-25T10:02:19.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62df7fe640 1 -- 172.21.15.19:0/110495960 learned_addr learned my addr 172.21.15.19:0/110495960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62deffd640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e00b92a0 0x7f62e00b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62dffff640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e015dbe0 0x7f62e0160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62df7fe640 1 -- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e015dbe0 msgr2=0x7f62e0160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62df7fe640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e015dbe0 0x7f62e0160480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62df7fe640 1 -- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e00b92a0 msgr2=0x7f62e00b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62df7fe640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e00b92a0 0x7f62e00b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62df7fe640 1 -- 172.21.15.19:0/110495960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e00b5450 con 0x7f62e00b2cb0 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62dffff640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e015dbe0 0x7f62e0160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.131+0000 7f62df7fe640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e00b2cb0 0x7f62e01f2740 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f62d000e9b0 tx=0x7f62d000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.132+0000 7f62c7fff640 1 -- 172.21.15.19:0/110495960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0017070 con 0x7f62e00b2cb0 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.132+0000 7f62c7fff640 1 -- 172.21.15.19:0/110495960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d000c9a0 con 0x7f62e00b2cb0 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.132+0000 7f62e6190640 1 -- 172.21.15.19:0/110495960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e00b5730 con 0x7f62e00b2cb0 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.132+0000 7f62c7fff640 1 -- 172.21.15.19:0/110495960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000cd90 con 0x7f62e00b2cb0 2024-04-25T10:02:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.132+0000 7f62e6190640 1 -- 172.21.15.19:0/110495960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e015e5d0 con 0x7f62e00b2cb0 2024-04-25T10:02:19.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.133+0000 7f62e6190640 1 -- 172.21.15.19:0/110495960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e0073990 con 0x7f62e00b2cb0 2024-04-25T10:02:19.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.133+0000 7f62c7fff640 1 -- 172.21.15.19:0/110495960 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62d001a070 con 0x7f62e00b2cb0 2024-04-25T10:02:19.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.134+0000 7f62c7fff640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62cc06b4d0 0x7f62cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.134+0000 7f62c7fff640 1 -- 172.21.15.19:0/110495960 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f62d008a680 con 0x7f62e00b2cb0 2024-04-25T10:02:19.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.134+0000 7f62deffd640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62cc06b4d0 0x7f62cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.134+0000 7f62deffd640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62cc06b4d0 0x7f62cc06d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f62e01520f0 tx=0x7f62d4001d40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.138+0000 7f62c7fff640 1 -- 172.21.15.19:0/110495960 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62d0058d60 con 0x7f62e00b2cb0 2024-04-25T10:02:19.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.356+0000 7f62e6190640 1 -- 172.21.15.19:0/110495960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62e014fbd0 con 0x7f62e00b2cb0 2024-04-25T10:02:19.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.357+0000 7f62c7fff640 1 -- 172.21.15.19:0/110495960 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f62d005ce00 con 0x7f62e00b2cb0 2024-04-25T10:02:19.361 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:19.362 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:02:14.631627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9210":{"gid":9210,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":17,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-04-25T10:02:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.358+0000 7f62e6190640 1 -- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62cc06b4d0 msgr2=0x7f62cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.358+0000 7f62e6190640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62cc06b4d0 0x7f62cc06d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f62e01520f0 tx=0x7f62d4001d40 comp rx=0 tx=0).stop 2024-04-25T10:02:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.358+0000 7f62e6190640 1 -- 172.21.15.19:0/110495960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e00b2cb0 msgr2=0x7f62e01f2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.358+0000 7f62e6190640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e00b2cb0 0x7f62e01f2740 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f62d000e9b0 tx=0x7f62d000ee70 comp rx=0 tx=0).stop 2024-04-25T10:02:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.358+0000 7f62e6190640 1 -- 172.21.15.19:0/110495960 shutdown_connections 2024-04-25T10:02:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.358+0000 7f62e6190640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f62cc06b4d0 0x7f62cc06d990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.358+0000 7f62e6190640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62e015dbe0 0x7f62e0160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.358+0000 7f62e6190640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e00b92a0 0x7f62e00b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.358+0000 7f62e6190640 1 --2- 172.21.15.19:0/110495960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62e00b2cb0 0x7f62e01f2740 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.358+0000 7f62e6190640 1 -- 172.21.15.19:0/110495960 >> 172.21.15.19:0/110495960 conn(0x7f62e0061a20 msgr2=0x7f62e00ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.359+0000 7f62e6190640 1 -- 172.21.15.19:0/110495960 shutdown_connections 2024-04-25T10:02:19.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.359+0000 7f62e6190640 1 -- 172.21.15.19:0/110495960 wait complete. 2024-04-25T10:02:19.374 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:02:19.374 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:02:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.469+0000 7f85cb54c640 1 Processor -- start 2024-04-25T10:02:19.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.469+0000 7f85cb54c640 1 -- start start 2024-04-25T10:02:19.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.470+0000 7f85cb54c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c40b2cb0 0x7f85c40d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.470+0000 7f85cb54c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f85c40d1020 0x7f85c40d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.470+0000 7f85cb54c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c40b97a0 con 0x7f85c415dbe0 2024-04-25T10:02:19.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.470+0000 7f85cb54c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c415dfc0 con 0x7f85c40d1020 2024-04-25T10:02:19.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.470+0000 7f85cb54c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40d1a00 con 0x7f85c40b2cb0 2024-04-25T10:02:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.470+0000 7f85c92e8640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f85c415dbe0 0x7f85c40d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50298/0 (socket says 172.21.15.19:50298) 2024-04-25T10:02:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.470+0000 7f85c92e8640 1 -- 172.21.15.19:0/610102265 learned_addr learned my addr 172.21.15.19:0/610102265 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.470+0000 7f85c8ae7640 1 --2- 172.21.15.19:0/610102265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c40b2cb0 0x7f85c40d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.470+0000 7f85c9ae9640 1 --2- 172.21.15.19:0/610102265 >> v2:172.21.15.112:3300/0 conn(0x7f85c40d1020 0x7f85c40d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.470+0000 7f85bbfff640 1 -- 172.21.15.19:0/610102265 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441640515 0 0) 0x7f85c40b97a0 con 0x7f85c415dbe0 2024-04-25T10:02:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.471+0000 7f85bbfff640 1 -- 172.21.15.19:0/610102265 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b0003520 con 0x7f85c415dbe0 2024-04-25T10:02:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.471+0000 7f85c8ae7640 1 -- 172.21.15.19:0/610102265 >> v2:172.21.15.112:3300/0 conn(0x7f85c40d1020 msgr2=0x7f85c40d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.471+0000 7f85c8ae7640 1 --2- 172.21.15.19:0/610102265 >> v2:172.21.15.112:3300/0 conn(0x7f85c40d1020 0x7f85c40d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.471+0000 7f85bbfff640 1 -- 172.21.15.19:0/610102265 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 160737384 0 0) 0x7f85b0003520 con 0x7f85c415dbe0 2024-04-25T10:02:19.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.471+0000 7f85c8ae7640 1 -- 172.21.15.19:0/610102265 >> v1:172.21.15.19:6789/0 conn(0x7f85c415dbe0 legacy=0x7f85c40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.471+0000 7f85c8ae7640 1 -- 172.21.15.19:0/610102265 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40d1c50 con 0x7f85c40b2cb0 2024-04-25T10:02:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85c8ae7640 1 --2- 172.21.15.19:0/610102265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c40b2cb0 0x7f85c40d2730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f85b40097a0 tx=0x7f85b402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20c32e1659013943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85bbfff640 1 -- 172.21.15.19:0/610102265 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4037070 con 0x7f85c40b2cb0 2024-04-25T10:02:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85bbfff640 1 -- 172.21.15.19:0/610102265 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b402aba0 con 0x7f85c40b2cb0 2024-04-25T10:02:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85bbfff640 1 -- 172.21.15.19:0/610102265 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0001670 con 0x7f85c40b2cb0 2024-04-25T10:02:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85cb54c640 1 -- 172.21.15.19:0/610102265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c40b2cb0 msgr2=0x7f85c40d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85cb54c640 1 --2- 172.21.15.19:0/610102265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c40b2cb0 0x7f85c40d2730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f85b40097a0 tx=0x7f85b402a110 comp rx=0 tx=0).stop 2024-04-25T10:02:19.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85cb54c640 1 -- 172.21.15.19:0/610102265 shutdown_connections 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85cb54c640 1 --2- 172.21.15.19:0/610102265 >> v2:172.21.15.112:3300/0 conn(0x7f85c40d1020 0x7f85c40d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85cb54c640 1 --2- 172.21.15.19:0/610102265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c40b2cb0 0x7f85c40d2730 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85cb54c640 1 -- 172.21.15.19:0/610102265 >> 172.21.15.19:0/610102265 conn(0x7f85c4061a20 msgr2=0x7f85c4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85cb54c640 1 -- 172.21.15.19:0/610102265 shutdown_connections 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.472+0000 7f85cb54c640 1 -- 172.21.15.19:0/610102265 wait complete. 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.473+0000 7f85cb54c640 1 Processor -- start 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.474+0000 7f85cb54c640 1 -- start start 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.474+0000 7f85cb54c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c40b2cb0 0x7f85c4155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.474+0000 7f85cb54c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c40d1020 0x7f85c41f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.474+0000 7f85cb54c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c415dbe0 0x7f85c41f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.474+0000 7f85cb54c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40b92a0 con 0x7f85c40b2cb0 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.474+0000 7f85cb54c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40d2210 con 0x7f85c415dbe0 2024-04-25T10:02:19.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.474+0000 7f85cb54c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40b9420 con 0x7f85c40d1020 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.474+0000 7f85c92e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c40b2cb0 0x7f85c4155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.474+0000 7f85c92e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c40b2cb0 0x7f85c4155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53692/0 (socket says 172.21.15.19:53692) 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.474+0000 7f85c92e8640 1 -- 172.21.15.19:0/1539309078 learned_addr learned my addr 172.21.15.19:0/1539309078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85c92e8640 1 -- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c40d1020 msgr2=0x7f85c41f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85c9ae9640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c415dbe0 0x7f85c41f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85c8ae7640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c40d1020 0x7f85c41f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85c92e8640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c40d1020 0x7f85c41f4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85c92e8640 1 -- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c415dbe0 msgr2=0x7f85c41f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85c92e8640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c415dbe0 0x7f85c41f4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85c92e8640 1 -- 172.21.15.19:0/1539309078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c415a060 con 0x7f85c40b2cb0 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85c92e8640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c40b2cb0 0x7f85c4155800 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f85c000c730 tx=0x7f85c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85b9ffb640 1 -- 172.21.15.19:0/1539309078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0017070 con 0x7f85c40b2cb0 2024-04-25T10:02:19.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85b9ffb640 1 -- 172.21.15.19:0/1539309078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c0005ad0 con 0x7f85c40b2cb0 2024-04-25T10:02:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85cb54c640 1 -- 172.21.15.19:0/1539309078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c415a340 con 0x7f85c40b2cb0 2024-04-25T10:02:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85b9ffb640 1 -- 172.21.15.19:0/1539309078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0007070 con 0x7f85c40b2cb0 2024-04-25T10:02:19.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.475+0000 7f85cb54c640 1 -- 172.21.15.19:0/1539309078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c415a900 con 0x7f85c40b2cb0 2024-04-25T10:02:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.476+0000 7f85cb54c640 1 -- 172.21.15.19:0/1539309078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c4073990 con 0x7f85c40b2cb0 2024-04-25T10:02:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.477+0000 7f85b9ffb640 1 -- 172.21.15.19:0/1539309078 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85c001a070 con 0x7f85c40b2cb0 2024-04-25T10:02:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.477+0000 7f85b9ffb640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85a806b4d0 0x7f85a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.477+0000 7f85b9ffb640 1 -- 172.21.15.19:0/1539309078 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f85c00897a0 con 0x7f85c40b2cb0 2024-04-25T10:02:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.477+0000 7f85c8ae7640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85a806b4d0 0x7f85a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.478+0000 7f85c8ae7640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85a806b4d0 0x7f85a806d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f85b4034000 tx=0x7f85b402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.481+0000 7f85b9ffb640 1 -- 172.21.15.19:0/1539309078 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85c0057df0 con 0x7f85c40b2cb0 2024-04-25T10:02:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.732+0000 7f85cb54c640 1 -- 172.21.15.19:0/1539309078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f85c40d3300 con 0x7f85c40b2cb0 2024-04-25T10:02:19.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.733+0000 7f85b9ffb640 1 -- 172.21.15.19:0/1539309078 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1340 (secure 0 0 0) 0x7f85c005be90 con 0x7f85c40b2cb0 2024-04-25T10:02:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:19.738 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":576,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1049,"data_bytes":1079550683,"bytes_used":2583949312,"bytes_avail":770510163968,"bytes_total":773094113280,"read_bytes_sec":237915,"write_bytes_sec":352368,"read_op_per_sec":7,"write_op_per_sec":103},"fsmap":{"epoch":83,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9210}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:02:19.039949+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:02:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 -- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85a806b4d0 msgr2=0x7f85a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85a806b4d0 0x7f85a806d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f85b4034000 tx=0x7f85b402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 -- 172.21.15.19:0/1539309078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c40b2cb0 msgr2=0x7f85c4155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c40b2cb0 0x7f85c4155800 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f85c000c730 tx=0x7f85c000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 -- 172.21.15.19:0/1539309078 shutdown_connections 2024-04-25T10:02:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85a806b4d0 0x7f85a806d990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c415dbe0 0x7f85c41f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c40d1020 0x7f85c41f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 --2- 172.21.15.19:0/1539309078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85c40b2cb0 0x7f85c4155800 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 -- 172.21.15.19:0/1539309078 >> 172.21.15.19:0/1539309078 conn(0x7f85c4061a20 msgr2=0x7f85c40b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 -- 172.21.15.19:0/1539309078 shutdown_connections 2024-04-25T10:02:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:19.735+0000 7f85cb54c640 1 -- 172.21.15.19:0/1539309078 wait complete. 2024-04-25T10:02:19.750 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:02:19.751 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:02:19.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:19.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:19.833 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:02:19.834 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:02:19.834 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:19.850 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:02:19.850 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:19.906 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.042+0000 7f28f224c640 1 Processor -- start 2024-04-25T10:02:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.042+0000 7f28f224c640 1 -- start start 2024-04-25T10:02:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28f224c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f28ec154ee0 0x7f28ec0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28f224c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f28ec0cfcf0 0x7f28ec0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28f224c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28ec0ce3c0 0x7f28ec15e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28f224c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ec0ce830 con 0x7f28ec154ee0 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28f224c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ec072090 con 0x7f28ec0cfcf0 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28f224c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec071230 con 0x7f28ec0ce3c0 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28eb7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f28ec154ee0 0x7f28ec0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28eb7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f28ec154ee0 0x7f28ec0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53704/0 (socket says 172.21.15.19:53704) 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28eb7fe640 1 -- 172.21.15.19:0/3499662449 learned_addr learned my addr 172.21.15.19:0/3499662449 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28ebfff640 1 --2- 172.21.15.19:0/3499662449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28ec0ce3c0 0x7f28ec15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28eaffd640 1 --2- 172.21.15.19:0/3499662449 >> v2:172.21.15.112:3300/0 conn(0x7f28ec0cfcf0 0x7f28ec0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28eb7fe640 1 -- 172.21.15.19:0/3499662449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28ec0ce3c0 msgr2=0x7f28ec15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28eb7fe640 1 --2- 172.21.15.19:0/3499662449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28ec0ce3c0 0x7f28ec15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28eb7fe640 1 -- 172.21.15.19:0/3499662449 >> v2:172.21.15.112:3300/0 conn(0x7f28ec0cfcf0 msgr2=0x7f28ec0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28eb7fe640 1 --2- 172.21.15.19:0/3499662449 >> v2:172.21.15.112:3300/0 conn(0x7f28ec0cfcf0 0x7f28ec0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28eb7fe640 1 -- 172.21.15.19:0/3499662449 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec071480 con 0x7f28ec154ee0 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28ebfff640 1 --2- 172.21.15.19:0/3499662449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28ec0ce3c0 0x7f28ec15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.043+0000 7f28eaffd640 1 --2- 172.21.15.19:0/3499662449 >> v2:172.21.15.112:3300/0 conn(0x7f28ec0cfcf0 0x7f28ec0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.044+0000 7f28eb7fe640 1 --2- 172.21.15.19:0/3499662449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec154ee0 0x7f28ec0cf7b0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0cf560 tx=0x7f28dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b0101125201329 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.044+0000 7f28ea7fc640 1 -- 172.21.15.19:0/3499662449 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc037070 con 0x7f28ec154ee0 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.044+0000 7f28ea7fc640 1 -- 172.21.15.19:0/3499662449 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc032440 con 0x7f28ec154ee0 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.044+0000 7f28ea7fc640 1 -- 172.21.15.19:0/3499662449 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc032810 con 0x7f28ec154ee0 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.045+0000 7f28f224c640 1 -- 172.21.15.19:0/3499662449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec154ee0 msgr2=0x7f28ec0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.045+0000 7f28f224c640 1 --2- 172.21.15.19:0/3499662449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec154ee0 0x7f28ec0cf7b0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0cf560 tx=0x7f28dc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.045+0000 7f28f224c640 1 -- 172.21.15.19:0/3499662449 shutdown_connections 2024-04-25T10:02:20.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.045+0000 7f28f224c640 1 --2- 172.21.15.19:0/3499662449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28ec0ce3c0 0x7f28ec15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.045+0000 7f28f224c640 1 --2- 172.21.15.19:0/3499662449 >> v2:172.21.15.112:3300/0 conn(0x7f28ec0cfcf0 0x7f28ec0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.045+0000 7f28f224c640 1 --2- 172.21.15.19:0/3499662449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec154ee0 0x7f28ec0cf7b0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.045+0000 7f28f224c640 1 -- 172.21.15.19:0/3499662449 >> 172.21.15.19:0/3499662449 conn(0x7f28ec06abe0 msgr2=0x7f28ec06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.045+0000 7f28f224c640 1 -- 172.21.15.19:0/3499662449 shutdown_connections 2024-04-25T10:02:20.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.045+0000 7f28f224c640 1 -- 172.21.15.19:0/3499662449 wait complete. 2024-04-25T10:02:20.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.046+0000 7f28f224c640 1 Processor -- start 2024-04-25T10:02:20.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.046+0000 7f28f224c640 1 -- start start 2024-04-25T10:02:20.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28f224c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec0ce3c0 0x7f28ec1431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28f224c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28ec0cfcf0 0x7f28ec152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28f224c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec154ee0 0x7f28ec155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28f224c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec071d20 con 0x7f28ec154ee0 2024-04-25T10:02:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28f224c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec071a40 con 0x7f28ec0ce3c0 2024-04-25T10:02:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28f224c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec1552c0 con 0x7f28ec0cfcf0 2024-04-25T10:02:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28ebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec154ee0 0x7f28ec155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28ebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec154ee0 0x7f28ec155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53712/0 (socket says 172.21.15.19:53712) 2024-04-25T10:02:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28ebfff640 1 -- 172.21.15.19:0/35270728 learned_addr learned my addr 172.21.15.19:0/35270728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28eb7fe640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec0ce3c0 0x7f28ec1431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28eaffd640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28ec0cfcf0 0x7f28ec152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28ebfff640 1 -- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28ec0cfcf0 msgr2=0x7f28ec152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28ebfff640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28ec0cfcf0 0x7f28ec152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28ebfff640 1 -- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec0ce3c0 msgr2=0x7f28ec1431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28ebfff640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec0ce3c0 0x7f28ec1431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.047+0000 7f28ebfff640 1 -- 172.21.15.19:0/35270728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec152840 con 0x7f28ec154ee0 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.048+0000 7f28ebfff640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec154ee0 0x7f28ec155600 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f28e400cef0 tx=0x7f28e400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.048+0000 7f28d3fff640 1 -- 172.21.15.19:0/35270728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e401a070 con 0x7f28ec154ee0 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.048+0000 7f28d3fff640 1 -- 172.21.15.19:0/35270728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e400d840 con 0x7f28ec154ee0 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.048+0000 7f28f224c640 1 -- 172.21.15.19:0/35270728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ec151260 con 0x7f28ec154ee0 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.048+0000 7f28d3fff640 1 -- 172.21.15.19:0/35270728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e400dc30 con 0x7f28ec154ee0 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.048+0000 7f28f224c640 1 -- 172.21.15.19:0/35270728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ec151820 con 0x7f28ec154ee0 2024-04-25T10:02:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.049+0000 7f28f224c640 1 -- 172.21.15.19:0/35270728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ec073990 con 0x7f28ec154ee0 2024-04-25T10:02:20.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.049+0000 7f28d3fff640 1 -- 172.21.15.19:0/35270728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28e401d070 con 0x7f28ec154ee0 2024-04-25T10:02:20.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.050+0000 7f28d3fff640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d806b500 0x7f28d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.050+0000 7f28d3fff640 1 -- 172.21.15.19:0/35270728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f28e408ab40 con 0x7f28ec154ee0 2024-04-25T10:02:20.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.050+0000 7f28eb7fe640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d806b500 0x7f28d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.050+0000 7f28eb7fe640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d806b500 0x7f28d806d9c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f28dc034000 tx=0x7f28dc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.054+0000 7f28d3fff640 1 -- 172.21.15.19:0/35270728 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28e4059190 con 0x7f28ec154ee0 2024-04-25T10:02:20.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.236+0000 7f28f224c640 1 -- 172.21.15.19:0/35270728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28ec06d360 con 0x7f28ec154ee0 2024-04-25T10:02:20.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.238+0000 7f28d3fff640 1 -- 172.21.15.19:0/35270728 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+17015 (secure 0 0 0) 0x7f28e405d230 con 0x7f28ec154ee0 2024-04-25T10:02:20.243 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:20.243 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":130,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:12.609472+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:27.270153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:27.977083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/250977615":"2024-04-26T10:02:12.609390+0000","172.21.15.19:6833/250977615":"2024-04-26T10:02:12.609390+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 -- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d806b500 msgr2=0x7f28d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d806b500 0x7f28d806d9c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f28dc034000 tx=0x7f28dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 -- 172.21.15.19:0/35270728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec154ee0 msgr2=0x7f28ec155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec154ee0 0x7f28ec155600 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f28e400cef0 tx=0x7f28e400a710 comp rx=0 tx=0).stop 2024-04-25T10:02:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 -- 172.21.15.19:0/35270728 shutdown_connections 2024-04-25T10:02:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f28d806b500 0x7f28d806d9c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec154ee0 0x7f28ec155600 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28ec0cfcf0 0x7f28ec152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 --2- 172.21.15.19:0/35270728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec0ce3c0 0x7f28ec1431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 -- 172.21.15.19:0/35270728 >> 172.21.15.19:0/35270728 conn(0x7f28ec06abe0 msgr2=0x7f28ec15c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 -- 172.21.15.19:0/35270728 shutdown_connections 2024-04-25T10:02:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.240+0000 7f28f224c640 1 -- 172.21.15.19:0/35270728 wait complete. 2024-04-25T10:02:20.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:27.270153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:27.977083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:20.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.356+0000 7fa562d89640 1 Processor -- start 2024-04-25T10:02:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.356+0000 7fa562d89640 1 -- start start 2024-04-25T10:02:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa562d89640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa55c0b5620 0x7fa55c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa562d89640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa55c06c2d0 0x7fa55c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa562d89640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55c06cc80 con 0x7fa55c15b7d0 2024-04-25T10:02:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa562d89640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55c06cf50 con 0x7fa55c0b5620 2024-04-25T10:02:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa562d89640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55c0ce530 con 0x7fa55c06c2d0 2024-04-25T10:02:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa55bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa55c0b5620 0x7fa55c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa55bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa55c0b5620 0x7fa55c0b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53726/0 (socket says 172.21.15.19:53726) 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa55bfff640 1 -- 172.21.15.19:0/1286387874 learned_addr learned my addr 172.21.15.19:0/1286387874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa561326640 1 --2- 172.21.15.19:0/1286387874 >> v2:172.21.15.112:3300/0 conn(0x7fa55c06c2d0 0x7fa55c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa55bfff640 1 -- 172.21.15.19:0/1286387874 >> v2:172.21.15.112:3300/0 conn(0x7fa55c06c2d0 msgr2=0x7fa55c06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa55bfff640 1 --2- 172.21.15.19:0/1286387874 >> v2:172.21.15.112:3300/0 conn(0x7fa55c06c2d0 0x7fa55c06c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa55bfff640 1 -- 172.21.15.19:0/1286387874 >> v1:172.21.15.112:6789/0 conn(0x7fa55c15b7d0 legacy=0x7fa55c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa55bfff640 1 -- 172.21.15.19:0/1286387874 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0ce810 con 0x7fa55c0b5620 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.357+0000 7fa561326640 1 --2- 172.21.15.19:0/1286387874 >> v2:172.21.15.112:3300/0 conn(0x7fa55c06c2d0 0x7fa55c06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.358+0000 7fa55bfff640 1 --2- 172.21.15.19:0/1286387874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa55c0b5620 0x7fa55c0b5ab0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fa55c161890 tx=0x7fa54802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca8ae0019cef9e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.358+0000 7fa55b7fe640 1 -- 172.21.15.19:0/1286387874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548037070 con 0x7fa55c0b5620 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.358+0000 7fa55b7fe640 1 -- 172.21.15.19:0/1286387874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa548032440 con 0x7fa55c0b5620 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.358+0000 7fa55b7fe640 1 -- 172.21.15.19:0/1286387874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548032810 con 0x7fa55c0b5620 2024-04-25T10:02:20.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.358+0000 7fa562d89640 1 -- 172.21.15.19:0/1286387874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa55c0b5620 msgr2=0x7fa55c0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.358+0000 7fa562d89640 1 --2- 172.21.15.19:0/1286387874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa55c0b5620 0x7fa55c0b5ab0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fa55c161890 tx=0x7fa54802a270 comp rx=0 tx=0).stop 2024-04-25T10:02:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.359+0000 7fa562d89640 1 -- 172.21.15.19:0/1286387874 shutdown_connections 2024-04-25T10:02:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.359+0000 7fa562d89640 1 --2- 172.21.15.19:0/1286387874 >> v2:172.21.15.112:3300/0 conn(0x7fa55c06c2d0 0x7fa55c06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.359+0000 7fa562d89640 1 --2- 172.21.15.19:0/1286387874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa55c0b5620 0x7fa55c0b5ab0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.359+0000 7fa562d89640 1 -- 172.21.15.19:0/1286387874 >> 172.21.15.19:0/1286387874 conn(0x7fa55c0d0800 msgr2=0x7fa55c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.359+0000 7fa562d89640 1 -- 172.21.15.19:0/1286387874 shutdown_connections 2024-04-25T10:02:20.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.359+0000 7fa562d89640 1 -- 172.21.15.19:0/1286387874 wait complete. 2024-04-25T10:02:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.360+0000 7fa562d89640 1 Processor -- start 2024-04-25T10:02:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.360+0000 7fa562d89640 1 -- start start 2024-04-25T10:02:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa562d89640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa55c06c2d0 0x7fa55c0c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa562d89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c0b5620 0x7fa55c0cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa562d89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa55c15b7d0 0x7fa55c0cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa562d89640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c15bbb0 con 0x7fa55c06c2d0 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa562d89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c06d230 con 0x7fa55c0b5620 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa562d89640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c15bd30 con 0x7fa55c15b7d0 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa560b25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa55c06c2d0 0x7fa55c0c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa560b25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa55c06c2d0 0x7fa55c0c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53734/0 (socket says 172.21.15.19:53734) 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa560b25640 1 -- 172.21.15.19:0/956705235 learned_addr learned my addr 172.21.15.19:0/956705235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa55bfff640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c0b5620 0x7fa55c0cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa561326640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa55c15b7d0 0x7fa55c0cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa560b25640 1 -- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa55c15b7d0 msgr2=0x7fa55c0cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa560b25640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa55c15b7d0 0x7fa55c0cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa560b25640 1 -- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c0b5620 msgr2=0x7fa55c0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa560b25640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c0b5620 0x7fa55c0cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa560b25640 1 -- 172.21.15.19:0/956705235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0c7af0 con 0x7fa55c06c2d0 2024-04-25T10:02:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa560b25640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa55c06c2d0 0x7fa55c0c8b90 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c3c0 tx=0x7fa55000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa5597fa640 1 -- 172.21.15.19:0/956705235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5500052d0 con 0x7fa55c06c2d0 2024-04-25T10:02:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa5597fa640 1 -- 172.21.15.19:0/956705235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa550005490 con 0x7fa55c06c2d0 2024-04-25T10:02:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa562d89640 1 -- 172.21.15.19:0/956705235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c0c7dd0 con 0x7fa55c06c2d0 2024-04-25T10:02:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa5597fa640 1 -- 172.21.15.19:0/956705235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550007260 con 0x7fa55c06c2d0 2024-04-25T10:02:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.361+0000 7fa562d89640 1 -- 172.21.15.19:0/956705235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c0c8390 con 0x7fa55c06c2d0 2024-04-25T10:02:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.363+0000 7fa562d89640 1 -- 172.21.15.19:0/956705235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c073a00 con 0x7fa55c06c2d0 2024-04-25T10:02:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.363+0000 7fa5597fa640 1 -- 172.21.15.19:0/956705235 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa550007400 con 0x7fa55c06c2d0 2024-04-25T10:02:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.363+0000 7fa5597fa640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa54006b4d0 0x7fa54006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.363+0000 7fa5597fa640 1 -- 172.21.15.19:0/956705235 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa550089950 con 0x7fa55c06c2d0 2024-04-25T10:02:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.363+0000 7fa55bfff640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa54006b4d0 0x7fa54006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.364+0000 7fa55bfff640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa54006b4d0 0x7fa54006d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa548034000 tx=0x7fa54802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.367+0000 7fa5597fa640 1 -- 172.21.15.19:0/956705235 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa550058560 con 0x7fa55c06c2d0 2024-04-25T10:02:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.547+0000 7fa562d89640 1 -- 172.21.15.19:0/956705235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa55c06c7d0 con 0x7fa55c06c2d0 2024-04-25T10:02:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.547+0000 7fa5597fa640 1 -- 172.21.15.19:0/956705235 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa550057e60 con 0x7fa55c06c2d0 2024-04-25T10:02:20.552 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:02:20.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.549+0000 7fa562d89640 1 -- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa54006b4d0 msgr2=0x7fa54006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.549+0000 7fa562d89640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa54006b4d0 0x7fa54006d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa548034000 tx=0x7fa54802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:20.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.549+0000 7fa562d89640 1 -- 172.21.15.19:0/956705235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa55c06c2d0 msgr2=0x7fa55c0c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.549+0000 7fa562d89640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa55c06c2d0 0x7fa55c0c8b90 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c3c0 tx=0x7fa55000c880 comp rx=0 tx=0).stop 2024-04-25T10:02:20.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.550+0000 7fa562d89640 1 -- 172.21.15.19:0/956705235 shutdown_connections 2024-04-25T10:02:20.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.550+0000 7fa562d89640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa54006b4d0 0x7fa54006d990 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.550+0000 7fa562d89640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa55c15b7d0 0x7fa55c0cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.550+0000 7fa562d89640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c0b5620 0x7fa55c0cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.550+0000 7fa562d89640 1 --2- 172.21.15.19:0/956705235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa55c06c2d0 0x7fa55c0c8b90 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.550+0000 7fa562d89640 1 -- 172.21.15.19:0/956705235 >> 172.21.15.19:0/956705235 conn(0x7fa55c0d0800 msgr2=0x7fa55c15fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.550+0000 7fa562d89640 1 -- 172.21.15.19:0/956705235 shutdown_connections 2024-04-25T10:02:20.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.550+0000 7fa562d89640 1 -- 172.21.15.19:0/956705235 wait complete. 2024-04-25T10:02:20.566 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.666+0000 7fe30d5b6640 1 Processor -- start 2024-04-25T10:02:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.666+0000 7fe30d5b6640 1 -- start start 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.666+0000 7fe30d5b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe308161710 0x7fe3080d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.666+0000 7fe30d5b6640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30815e000 con 0x7fe30815dc20 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.666+0000 7fe30d5b6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3080d1de0 con 0x7fe3080ced80 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.666+0000 7fe30d5b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe308161c60 con 0x7fe308161710 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe306ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe30815dc20 0x7fe3080ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50304/0 (socket says 172.21.15.19:50304) 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe306ffd640 1 -- 172.21.15.19:0/3745228197 learned_addr learned my addr 172.21.15.19:0/3745228197 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe3077fe640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe308161710 0x7fe3080d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386419660 0 0) 0x7fe30815e000 con 0x7fe30815dc20 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2f0003520 con 0x7fe30815dc20 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163715186 0 0) 0x7fe3080d1de0 con 0x7fe3080ced80 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe30815e000 con 0x7fe3080ced80 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3777813899 0 0) 0x7fe2f0003520 con 0x7fe30815dc20 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe308161710 msgr2=0x7fe3080d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe308161710 0x7fe3080d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 >> v1:172.21.15.112:6789/0 conn(0x7fe3080ced80 legacy=0x7fe3080d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308161f40 con 0x7fe30815dc20 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fe2f8001670 con 0x7fe30815dc20 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 >> v1:172.21.15.19:6789/0 conn(0x7fe30815dc20 legacy=0x7fe3080ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f0003b70 0x7fe2f0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f0004800 0x7fe2f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe308161710 0x7fe2f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe308161c60 con 0x7fe2f0003b70 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f000ba70 con 0x7fe2f0004800 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f000bc70 con 0x7fe308161710 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe306ffd640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f0003b70 0x7fe2f0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.667+0000 7fe3067fc640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f0004800 0x7fe2f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.668+0000 7fe3077fe640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe308161710 0x7fe2f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.668+0000 7fe306ffd640 1 -- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe308161710 msgr2=0x7fe2f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.668+0000 7fe306ffd640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe308161710 0x7fe2f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.668+0000 7fe306ffd640 1 -- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f0004800 msgr2=0x7fe2f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.668+0000 7fe306ffd640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f0004800 0x7fe2f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.668+0000 7fe306ffd640 1 -- 172.21.15.19:0/3745228197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f0003520 con 0x7fe2f0003b70 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.668+0000 7fe3067fc640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f0004800 0x7fe2f0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.668+0000 7fe306ffd640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f0003b70 0x7fe2f0003f70 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800c0e0 tx=0x7fe2f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd969db1803ecd2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.668+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8007c00 con 0x7fe2f0003b70 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.668+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f8005ad0 con 0x7fe2f0003b70 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.669+0000 7fe305ffb640 1 -- 172.21.15.19:0/3745228197 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8007070 con 0x7fe2f0003b70 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.669+0000 7fe30d5b6640 1 -- 172.21.15.19:0/3745228197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f0003b70 msgr2=0x7fe2f0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.669+0000 7fe30d5b6640 1 --2- 172.21.15.19:0/3745228197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f0003b70 0x7fe2f0003f70 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800c0e0 tx=0x7fe2f0005830 comp rx=0 tx=0).stop 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.669+0000 7fe3077fe640 1 -- 172.21.15.19:0/3745228197 reap_dead start 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.669+0000 7fe30d5b6640 1 -- 172.21.15.19:0/3745228197 shutdown_connections 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.669+0000 7fe30d5b6640 1 -- 172.21.15.19:0/3745228197 >> 172.21.15.19:0/3745228197 conn(0x7fe3080d0420 msgr2=0x7fe3080d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.669+0000 7fe30d5b6640 1 -- 172.21.15.19:0/3745228197 shutdown_connections 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.669+0000 7fe30d5b6640 1 -- 172.21.15.19:0/3745228197 wait complete. 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.670+0000 7fe30d5b6640 1 Processor -- start 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.671+0000 7fe30d5b6640 1 -- start start 2024-04-25T10:02:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.671+0000 7fe30d5b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f0003b70 0x7fe3080cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.671+0000 7fe30d5b6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080ac7c0 0x7fe308152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.671+0000 7fe30d5b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3080acc20 0x7fe308152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.671+0000 7fe30d5b6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080b5e50 con 0x7fe3080ac7c0 2024-04-25T10:02:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.671+0000 7fe30d5b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe308162500 con 0x7fe2f0003b70 2024-04-25T10:02:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.671+0000 7fe30d5b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080b5fd0 con 0x7fe3080acc20 2024-04-25T10:02:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.671+0000 7fe3067fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080ac7c0 0x7fe308152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.671+0000 7fe3067fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080ac7c0 0x7fe308152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53760/0 (socket says 172.21.15.19:53760) 2024-04-25T10:02:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.671+0000 7fe3067fc640 1 -- 172.21.15.19:0/855607360 learned_addr learned my addr 172.21.15.19:0/855607360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe306ffd640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f0003b70 0x7fe3080cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe3077fe640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3080acc20 0x7fe308152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe3067fc640 1 -- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3080acc20 msgr2=0x7fe308152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe3067fc640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3080acc20 0x7fe308152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe3067fc640 1 -- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f0003b70 msgr2=0x7fe3080cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe3067fc640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f0003b70 0x7fe3080cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe3067fc640 1 -- 172.21.15.19:0/855607360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3080cb6d0 con 0x7fe3080ac7c0 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe3077fe640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3080acc20 0x7fe308152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe306ffd640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f0003b70 0x7fe3080cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe3067fc640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080ac7c0 0x7fe308152920 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00e7e0 tx=0x7fe2fc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe2ef7fe640 1 -- 172.21.15.19:0/855607360 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0050d0 con 0x7fe3080ac7c0 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe2ef7fe640 1 -- 172.21.15.19:0/855607360 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2fc005270 con 0x7fe3080ac7c0 2024-04-25T10:02:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe30d5b6640 1 -- 172.21.15.19:0/855607360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3080cb9b0 con 0x7fe3080ac7c0 2024-04-25T10:02:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe2ef7fe640 1 -- 172.21.15.19:0/855607360 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc005440 con 0x7fe3080ac7c0 2024-04-25T10:02:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.672+0000 7fe30d5b6640 1 -- 172.21.15.19:0/855607360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3080cbf70 con 0x7fe3080ac7c0 2024-04-25T10:02:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.673+0000 7fe30d5b6640 1 -- 172.21.15.19:0/855607360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe308073a00 con 0x7fe3080ac7c0 2024-04-25T10:02:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.673+0000 7fe2ef7fe640 1 -- 172.21.15.19:0/855607360 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2fc01a020 con 0x7fe3080ac7c0 2024-04-25T10:02:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.674+0000 7fe2ef7fe640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2f406b500 0x7fe2f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.674+0000 7fe2ef7fe640 1 -- 172.21.15.19:0/855607360 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe2fc089bc0 con 0x7fe3080ac7c0 2024-04-25T10:02:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.674+0000 7fe306ffd640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2f406b500 0x7fe2f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.675+0000 7fe306ffd640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2f406b500 0x7fe2f406d9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8032000 tx=0x7fe2f802d000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.678+0000 7fe2ef7fe640 1 -- 172.21.15.19:0/855607360 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2fc058890 con 0x7fe3080ac7c0 2024-04-25T10:02:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.856+0000 7fe30d5b6640 1 -- 172.21.15.19:0/855607360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3080d16b0 con 0x7fe3080ac7c0 2024-04-25T10:02:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.856+0000 7fe2ef7fe640 1 -- 172.21.15.19:0/855607360 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2fc058190 con 0x7fe3080ac7c0 2024-04-25T10:02:20.862 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:02:20.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 -- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2f406b500 msgr2=0x7fe2f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2f406b500 0x7fe2f406d9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8032000 tx=0x7fe2f802d000 comp rx=0 tx=0).stop 2024-04-25T10:02:20.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 -- 172.21.15.19:0/855607360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080ac7c0 msgr2=0x7fe308152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080ac7c0 0x7fe308152920 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00e7e0 tx=0x7fe2fc00eca0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 -- 172.21.15.19:0/855607360 shutdown_connections 2024-04-25T10:02:20.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe2f406b500 0x7fe2f406d9c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3080acc20 0x7fe308152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080ac7c0 0x7fe308152920 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 --2- 172.21.15.19:0/855607360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f0003b70 0x7fe3080cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 -- 172.21.15.19:0/855607360 >> 172.21.15.19:0/855607360 conn(0x7fe3080d0420 msgr2=0x7fe3080b8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 -- 172.21.15.19:0/855607360 shutdown_connections 2024-04-25T10:02:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.858+0000 7fe30d5b6640 1 -- 172.21.15.19:0/855607360 wait complete. 2024-04-25T10:02:20.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:20.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.973+0000 7f2b1954c640 1 Processor -- start 2024-04-25T10:02:20.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.973+0000 7f2b1954c640 1 -- start start 2024-04-25T10:02:20.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b1954c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1415a9f0 0x7f2b140c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b1954c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2b140cd6f0 0x7f2b140cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b1954c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b140c6940 con 0x7f2b140cdbf0 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b1954c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b140b0020 con 0x7f2b140cd6f0 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b1954c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b140cb5a0 con 0x7f2b1415a9f0 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b137fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2b140cd6f0 0x7f2b140cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b127fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2b140cdbf0 0x7f2b140ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50320/0 (socket says 172.21.15.19:50320) 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b127fc640 1 -- 172.21.15.19:0/3666221259 learned_addr learned my addr 172.21.15.19:0/3666221259 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b137fe640 1 -- 172.21.15.19:0/3666221259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1415a9f0 msgr2=0x7f2b140c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b12ffd640 1 --2- 172.21.15.19:0/3666221259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1415a9f0 0x7f2b140c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b137fe640 1 --2- 172.21.15.19:0/3666221259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1415a9f0 0x7f2b140c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b137fe640 1 -- 172.21.15.19:0/3666221259 >> v1:172.21.15.19:6789/0 conn(0x7f2b140cdbf0 legacy=0x7f2b140ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b11ffb640 1 -- 172.21.15.19:0/3666221259 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266398778 0 0) 0x7f2afc001110 con 0x7f2b140cdbf0 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b137fe640 1 -- 172.21.15.19:0/3666221259 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1415add0 con 0x7f2b140cd6f0 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.974+0000 7f2b12ffd640 1 --2- 172.21.15.19:0/3666221259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1415a9f0 0x7f2b140c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.975+0000 7f2b137fe640 1 --2- 172.21.15.19:0/3666221259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b140cd6f0 0x7f2b140cba30 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0097a0 tx=0x7f2b0c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5a299fe15e9d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.975+0000 7f2b11ffb640 1 -- 172.21.15.19:0/3666221259 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c037070 con 0x7f2b140cd6f0 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.975+0000 7f2b11ffb640 1 -- 172.21.15.19:0/3666221259 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c032440 con 0x7f2b140cd6f0 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.975+0000 7f2b11ffb640 1 -- 172.21.15.19:0/3666221259 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c032810 con 0x7f2b140cd6f0 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.976+0000 7f2b1954c640 1 -- 172.21.15.19:0/3666221259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b140cd6f0 msgr2=0x7f2b140cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.976+0000 7f2b1954c640 1 --2- 172.21.15.19:0/3666221259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b140cd6f0 0x7f2b140cba30 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0097a0 tx=0x7f2b0c02a180 comp rx=0 tx=0).stop 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.976+0000 7f2b1954c640 1 -- 172.21.15.19:0/3666221259 shutdown_connections 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.976+0000 7f2b1954c640 1 --2- 172.21.15.19:0/3666221259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b140cd6f0 0x7f2b140cba30 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.976+0000 7f2b1954c640 1 --2- 172.21.15.19:0/3666221259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1415a9f0 0x7f2b140c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.976+0000 7f2b1954c640 1 -- 172.21.15.19:0/3666221259 >> 172.21.15.19:0/3666221259 conn(0x7f2b140b2390 msgr2=0x7f2b140b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.976+0000 7f2b1954c640 1 -- 172.21.15.19:0/3666221259 shutdown_connections 2024-04-25T10:02:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.976+0000 7f2b1954c640 1 -- 172.21.15.19:0/3666221259 wait complete. 2024-04-25T10:02:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.977+0000 7f2b1954c640 1 Processor -- start 2024-04-25T10:02:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.977+0000 7f2b1954c640 1 -- start start 2024-04-25T10:02:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b1954c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b140cd6f0 0x7f2b1414de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b1954c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b140cdbf0 0x7f2b14154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b1954c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b1415a9f0 0x7f2b1414f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b1954c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b140cb230 con 0x7f2b1415a9f0 2024-04-25T10:02:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b1954c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b140caf50 con 0x7f2b140cdbf0 2024-04-25T10:02:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b1954c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b140ce030 con 0x7f2b140cd6f0 2024-04-25T10:02:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b1415a9f0 0x7f2b1414f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b1415a9f0 0x7f2b1414f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53780/0 (socket says 172.21.15.19:53780) 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b137fe640 1 -- 172.21.15.19:0/1949241666 learned_addr learned my addr 172.21.15.19:0/1949241666 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b127fc640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b140cdbf0 0x7f2b14154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b12ffd640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b140cd6f0 0x7f2b1414de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b137fe640 1 -- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b140cd6f0 msgr2=0x7f2b1414de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b137fe640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b140cd6f0 0x7f2b1414de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b137fe640 1 -- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b140cdbf0 msgr2=0x7f2b14154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b137fe640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b140cdbf0 0x7f2b14154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.978+0000 7f2b137fe640 1 -- 172.21.15.19:0/1949241666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b14154990 con 0x7f2b1415a9f0 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.979+0000 7f2b137fe640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b1415a9f0 0x7f2b1414f710 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0097a0 tx=0x7f2b0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.979+0000 7f2b0b7fe640 1 -- 172.21.15.19:0/1949241666 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c037050 con 0x7f2b1415a9f0 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.979+0000 7f2b0b7fe640 1 -- 172.21.15.19:0/1949241666 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c038040 con 0x7f2b1415a9f0 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.979+0000 7f2b0b7fe640 1 -- 172.21.15.19:0/1949241666 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c004490 con 0x7f2b1415a9f0 2024-04-25T10:02:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.979+0000 7f2b1954c640 1 -- 172.21.15.19:0/1949241666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b141533b0 con 0x7f2b1415a9f0 2024-04-25T10:02:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.979+0000 7f2b1954c640 1 -- 172.21.15.19:0/1949241666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b14153970 con 0x7f2b1415a9f0 2024-04-25T10:02:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.980+0000 7f2b1954c640 1 -- 172.21.15.19:0/1949241666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b140ca400 con 0x7f2b1415a9f0 2024-04-25T10:02:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.980+0000 7f2b0b7fe640 1 -- 172.21.15.19:0/1949241666 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b0c030980 con 0x7f2b1415a9f0 2024-04-25T10:02:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.981+0000 7f2b0b7fe640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2af406b4d0 0x7f2af406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.981+0000 7f2b0b7fe640 1 -- 172.21.15.19:0/1949241666 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2b0c0aaef0 con 0x7f2b1415a9f0 2024-04-25T10:02:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.981+0000 7f2b12ffd640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2af406b4d0 0x7f2af406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.981+0000 7f2b12ffd640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2af406b4d0 0x7f2af406d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2b04007510 tx=0x7f2b040074a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:20.985+0000 7f2b0b7fe640 1 -- 172.21.15.19:0/1949241666 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b0c032e20 con 0x7f2b1415a9f0 2024-04-25T10:02:21.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.165+0000 7f2b1954c640 1 -- 172.21.15.19:0/1949241666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2b1406cc80 con 0x7f2b1415a9f0 2024-04-25T10:02:21.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.166+0000 7f2b0b7fe640 1 -- 172.21.15.19:0/1949241666 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2b0c079cb0 con 0x7f2b1415a9f0 2024-04-25T10:02:21.170 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:02:21.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 -- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2af406b4d0 msgr2=0x7f2af406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2af406b4d0 0x7f2af406d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2b04007510 tx=0x7f2b040074a0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 -- 172.21.15.19:0/1949241666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b1415a9f0 msgr2=0x7f2b1414f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b1415a9f0 0x7f2b1414f710 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0097a0 tx=0x7f2b0c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 -- 172.21.15.19:0/1949241666 shutdown_connections 2024-04-25T10:02:21.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2af406b4d0 0x7f2af406d990 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b1415a9f0 0x7f2b1414f710 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b140cdbf0 0x7f2b14154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 --2- 172.21.15.19:0/1949241666 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b140cd6f0 0x7f2b1414de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 -- 172.21.15.19:0/1949241666 >> 172.21.15.19:0/1949241666 conn(0x7f2b140b2390 msgr2=0x7f2b140ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 -- 172.21.15.19:0/1949241666 shutdown_connections 2024-04-25T10:02:21.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.168+0000 7f2b1954c640 1 -- 172.21.15.19:0/1949241666 wait complete. 2024-04-25T10:02:21.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:21.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.281+0000 7f5fe544a640 1 Processor -- start 2024-04-25T10:02:21.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.281+0000 7f5fe544a640 1 -- start start 2024-04-25T10:02:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fe544a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5fe00ced80 0x7f5fe00d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fe544a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5fe00d1ce0 0x7f5fe00b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fe544a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe00d2120 con 0x7f5fe015dc20 2024-04-25T10:02:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fe544a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe00d23a0 con 0x7f5fe00d1ce0 2024-04-25T10:02:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fe544a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe00d2680 con 0x7f5fe00ced80 2024-04-25T10:02:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fdf7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5fe00d1ce0 0x7f5fe00b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fdf7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5fe00d1ce0 0x7f5fe00b9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53786/0 (socket says 172.21.15.19:53786) 2024-04-25T10:02:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fdf7fe640 1 -- 172.21.15.19:0/295979957 learned_addr learned my addr 172.21.15.19:0/295979957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fde7fc640 1 --2- 172.21.15.19:0/295979957 >> v2:172.21.15.112:3300/0 conn(0x7f5fe00ced80 0x7f5fe00d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fdf7fe640 1 -- 172.21.15.19:0/295979957 >> v2:172.21.15.112:3300/0 conn(0x7f5fe00ced80 msgr2=0x7f5fe00d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fdf7fe640 1 --2- 172.21.15.19:0/295979957 >> v2:172.21.15.112:3300/0 conn(0x7f5fe00ced80 0x7f5fe00d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fdf7fe640 1 -- 172.21.15.19:0/295979957 >> v1:172.21.15.112:6789/0 conn(0x7f5fe015dc20 legacy=0x7f5fe00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fdf7fe640 1 -- 172.21.15.19:0/295979957 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe00d2960 con 0x7f5fe00d1ce0 2024-04-25T10:02:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.282+0000 7f5fde7fc640 1 --2- 172.21.15.19:0/295979957 >> v2:172.21.15.112:3300/0 conn(0x7f5fe00ced80 0x7f5fe00d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.283+0000 7f5fdf7fe640 1 --2- 172.21.15.19:0/295979957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe00d1ce0 0x7f5fe00b9490 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00b61c0 tx=0x7f5fd802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb543e9aa266ebaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.283+0000 7f5fddffb640 1 -- 172.21.15.19:0/295979957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8036070 con 0x7f5fe00d1ce0 2024-04-25T10:02:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.283+0000 7f5fddffb640 1 -- 172.21.15.19:0/295979957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd8031600 con 0x7f5fe00d1ce0 2024-04-25T10:02:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.283+0000 7f5fddffb640 1 -- 172.21.15.19:0/295979957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80319d0 con 0x7f5fe00d1ce0 2024-04-25T10:02:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.284+0000 7f5fe544a640 1 -- 172.21.15.19:0/295979957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe00d1ce0 msgr2=0x7f5fe00b9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.284+0000 7f5fe544a640 1 --2- 172.21.15.19:0/295979957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe00d1ce0 0x7f5fe00b9490 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00b61c0 tx=0x7f5fd802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.284+0000 7f5fe544a640 1 -- 172.21.15.19:0/295979957 shutdown_connections 2024-04-25T10:02:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.284+0000 7f5fe544a640 1 --2- 172.21.15.19:0/295979957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe00d1ce0 0x7f5fe00b9490 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.284+0000 7f5fe544a640 1 --2- 172.21.15.19:0/295979957 >> v2:172.21.15.112:3300/0 conn(0x7f5fe00ced80 0x7f5fe00d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.284+0000 7f5fe544a640 1 -- 172.21.15.19:0/295979957 >> 172.21.15.19:0/295979957 conn(0x7f5fe0061a20 msgr2=0x7f5fe0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.284+0000 7f5fe544a640 1 -- 172.21.15.19:0/295979957 shutdown_connections 2024-04-25T10:02:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.284+0000 7f5fe544a640 1 -- 172.21.15.19:0/295979957 wait complete. 2024-04-25T10:02:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.285+0000 7f5fe544a640 1 Processor -- start 2024-04-25T10:02:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.285+0000 7f5fe544a640 1 -- start start 2024-04-25T10:02:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fe544a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe00ced80 0x7f5fe0160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fe544a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fe00d1ce0 0x7f5fe00c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fe544a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe015dc20 0x7f5fe01f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fe544a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe00b6a30 con 0x7f5fe015dc20 2024-04-25T10:02:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fe544a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe00d2f20 con 0x7f5fe00ced80 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fe544a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe00b6bb0 con 0x7f5fe00d1ce0 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fdf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe015dc20 0x7f5fe01f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fdf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe015dc20 0x7f5fe01f6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53792/0 (socket says 172.21.15.19:53792) 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fdf7fe640 1 -- 172.21.15.19:0/464294255 learned_addr learned my addr 172.21.15.19:0/464294255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fdeffd640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe00ced80 0x7f5fe0160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fde7fc640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fe00d1ce0 0x7f5fe00c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fdf7fe640 1 -- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fe00d1ce0 msgr2=0x7f5fe00c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fdf7fe640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fe00d1ce0 0x7f5fe00c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fdf7fe640 1 -- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe00ced80 msgr2=0x7f5fe0160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fdf7fe640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe00ced80 0x7f5fe0160550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.286+0000 7f5fdf7fe640 1 -- 172.21.15.19:0/464294255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe00c7e90 con 0x7f5fe015dc20 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.287+0000 7f5fdf7fe640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe015dc20 0x7f5fe01f6990 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009f40 tx=0x7f5fd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.287+0000 7f5fc77fe640 1 -- 172.21.15.19:0/464294255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8036050 con 0x7f5fe015dc20 2024-04-25T10:02:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.287+0000 7f5fc77fe640 1 -- 172.21.15.19:0/464294255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd8042070 con 0x7f5fe015dc20 2024-04-25T10:02:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.287+0000 7f5fc77fe640 1 -- 172.21.15.19:0/464294255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8009510 con 0x7f5fe015dc20 2024-04-25T10:02:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.287+0000 7f5fe544a640 1 -- 172.21.15.19:0/464294255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe00c8170 con 0x7f5fe015dc20 2024-04-25T10:02:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.287+0000 7f5fe544a640 1 -- 172.21.15.19:0/464294255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe00c8730 con 0x7f5fe015dc20 2024-04-25T10:02:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.288+0000 7f5fe544a640 1 -- 172.21.15.19:0/464294255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe0073a00 con 0x7f5fe015dc20 2024-04-25T10:02:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.288+0000 7f5fc77fe640 1 -- 172.21.15.19:0/464294255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fd8007c50 con 0x7f5fe015dc20 2024-04-25T10:02:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.289+0000 7f5fc77fe640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5fcc06b4d0 0x7f5fcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.289+0000 7f5fc77fe640 1 -- 172.21.15.19:0/464294255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5fd80ab1f0 con 0x7f5fe015dc20 2024-04-25T10:02:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.289+0000 7f5fdeffd640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5fcc06b4d0 0x7f5fcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.290+0000 7f5fdeffd640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5fcc06b4d0 0x7f5fcc06d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00016e0 tx=0x7f5fd0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.293+0000 7f5fc77fe640 1 -- 172.21.15.19:0/464294255 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5fd8079e90 con 0x7f5fe015dc20 2024-04-25T10:02:21.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.486+0000 7f5fe544a640 1 -- 172.21.15.19:0/464294255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5fe00008d0 con 0x7f5fe015dc20 2024-04-25T10:02:21.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.486+0000 7f5fc77fe640 1 -- 172.21.15.19:0/464294255 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v130) v1 ==== 80+0+120 (secure 0 0 0) 0x7f5fd8079790 con 0x7f5fe015dc20 2024-04-25T10:02:21.492 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/250977615 2024-04-26T10:02:12.609390+0000 2024-04-25T10:02:21.492 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6833/250977615 2024-04-26T10:02:12.609390+0000 2024-04-25T10:02:21.492 INFO:teuthology.orchestra.run.smithi019.stderr:listed 2 entries 2024-04-25T10:02:21.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.488+0000 7f5fe544a640 1 -- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5fcc06b4d0 msgr2=0x7f5fcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.488+0000 7f5fe544a640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5fcc06b4d0 0x7f5fcc06d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00016e0 tx=0x7f5fd0009040 comp rx=0 tx=0).stop 2024-04-25T10:02:21.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.488+0000 7f5fe544a640 1 -- 172.21.15.19:0/464294255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe015dc20 msgr2=0x7f5fe01f6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.488+0000 7f5fe544a640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe015dc20 0x7f5fe01f6990 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009f40 tx=0x7f5fd8004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.488+0000 7f5fe544a640 1 -- 172.21.15.19:0/464294255 shutdown_connections 2024-04-25T10:02:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.488+0000 7f5fe544a640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5fcc06b4d0 0x7f5fcc06d990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.488+0000 7f5fe544a640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe015dc20 0x7f5fe01f6990 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.488+0000 7f5fe544a640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fe00d1ce0 0x7f5fe00c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.488+0000 7f5fe544a640 1 --2- 172.21.15.19:0/464294255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe00ced80 0x7f5fe0160550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.488+0000 7f5fe544a640 1 -- 172.21.15.19:0/464294255 >> 172.21.15.19:0/464294255 conn(0x7f5fe0061a20 msgr2=0x7f5fe0061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.489+0000 7f5fe544a640 1 -- 172.21.15.19:0/464294255 shutdown_connections 2024-04-25T10:02:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.489+0000 7f5fe544a640 1 -- 172.21.15.19:0/464294255 wait complete. 2024-04-25T10:02:21.504 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:02:21.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:21.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:21.532 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:02:21.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:21.533 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:02:21.589 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:02:21.589 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:21.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:21.610 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:02:21.610 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:21.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:02:21.646 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:02:21.646 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:21.667 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:21.667 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.782+0000 7fe14710a640 1 Processor -- start 2024-04-25T10:02:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.782+0000 7fe14710a640 1 -- start start 2024-04-25T10:02:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.783+0000 7fe14710a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1400cfec0 0x7fe1400cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.783+0000 7fe14710a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe1400ce3c0 0x7fe14006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.783+0000 7fe14710a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe140071480 con 0x7fe140154ee0 2024-04-25T10:02:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.783+0000 7fe14710a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe140072090 con 0x7fe1400ce3c0 2024-04-25T10:02:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.783+0000 7fe14710a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe140071230 con 0x7fe1400cfec0 2024-04-25T10:02:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.783+0000 7fe1456a7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe1400ce3c0 0x7fe14006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.783+0000 7fe144ea6640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe140154ee0 0x7fe1400cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50336/0 (socket says 172.21.15.19:50336) 2024-04-25T10:02:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.783+0000 7fe144ea6640 1 -- 172.21.15.19:0/810563346 learned_addr learned my addr 172.21.15.19:0/810563346 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.784+0000 7fe13ffff640 1 --2- 172.21.15.19:0/810563346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1400cfec0 0x7fe1400cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.784+0000 7fe1456a7640 1 -- 172.21.15.19:0/810563346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1400cfec0 msgr2=0x7fe1400cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.784+0000 7fe1456a7640 1 --2- 172.21.15.19:0/810563346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1400cfec0 0x7fe1400cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.784+0000 7fe1456a7640 1 -- 172.21.15.19:0/810563346 >> v1:172.21.15.19:6789/0 conn(0x7fe140154ee0 legacy=0x7fe1400cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.784+0000 7fe1456a7640 1 -- 172.21.15.19:0/810563346 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe140071760 con 0x7fe1400ce3c0 2024-04-25T10:02:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.784+0000 7fe1456a7640 1 --2- 172.21.15.19:0/810563346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1400ce3c0 0x7fe14006d6f0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe1280097a0 tx=0x7fe12802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6675aea3d6f05f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.784+0000 7fe13f7fe640 1 -- 172.21.15.19:0/810563346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128037070 con 0x7fe1400ce3c0 2024-04-25T10:02:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.785+0000 7fe13f7fe640 1 -- 172.21.15.19:0/810563346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe128032440 con 0x7fe1400ce3c0 2024-04-25T10:02:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.785+0000 7fe13f7fe640 1 -- 172.21.15.19:0/810563346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128032810 con 0x7fe1400ce3c0 2024-04-25T10:02:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.785+0000 7fe14710a640 1 -- 172.21.15.19:0/810563346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1400ce3c0 msgr2=0x7fe14006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.785+0000 7fe14710a640 1 --2- 172.21.15.19:0/810563346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1400ce3c0 0x7fe14006d6f0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe1280097a0 tx=0x7fe12802a180 comp rx=0 tx=0).stop 2024-04-25T10:02:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.785+0000 7fe14710a640 1 -- 172.21.15.19:0/810563346 shutdown_connections 2024-04-25T10:02:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.785+0000 7fe14710a640 1 --2- 172.21.15.19:0/810563346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1400ce3c0 0x7fe14006d6f0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.785+0000 7fe14710a640 1 --2- 172.21.15.19:0/810563346 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1400cfec0 0x7fe1400cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.785+0000 7fe14710a640 1 -- 172.21.15.19:0/810563346 >> 172.21.15.19:0/810563346 conn(0x7fe14006abe0 msgr2=0x7fe14006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.785+0000 7fe14710a640 1 -- 172.21.15.19:0/810563346 shutdown_connections 2024-04-25T10:02:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.785+0000 7fe14710a640 1 -- 172.21.15.19:0/810563346 wait complete. 2024-04-25T10:02:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.786+0000 7fe14710a640 1 Processor -- start 2024-04-25T10:02:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.786+0000 7fe14710a640 1 -- start start 2024-04-25T10:02:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.787+0000 7fe14710a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1400ce3c0 0x7fe14015b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.787+0000 7fe14710a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1400cfec0 0x7fe140152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.787+0000 7fe14710a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140154ee0 0x7fe1401597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.787+0000 7fe14710a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe140071d20 con 0x7fe140154ee0 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.787+0000 7fe14710a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1400ce800 con 0x7fe1400ce3c0 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.787+0000 7fe14710a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1401552c0 con 0x7fe1400cfec0 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.787+0000 7fe1456a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140154ee0 0x7fe1401597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.787+0000 7fe1456a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140154ee0 0x7fe1401597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53810/0 (socket says 172.21.15.19:53810) 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.787+0000 7fe1456a7640 1 -- 172.21.15.19:0/1508829650 learned_addr learned my addr 172.21.15.19:0/1508829650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe13ffff640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1400cfec0 0x7fe140152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe1456a7640 1 -- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1400cfec0 msgr2=0x7fe140152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe1456a7640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1400cfec0 0x7fe140152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe1456a7640 1 -- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1400ce3c0 msgr2=0x7fe14015b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe1456a7640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1400ce3c0 0x7fe14015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe1456a7640 1 -- 172.21.15.19:0/1508829650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14015a4e0 con 0x7fe140154ee0 2024-04-25T10:02:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe13ffff640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1400cfec0 0x7fe140152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe1456a7640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140154ee0 0x7fe1401597b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe128035040 tx=0x7fe1280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe13d7fa640 1 -- 172.21.15.19:0/1508829650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128037040 con 0x7fe140154ee0 2024-04-25T10:02:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe13d7fa640 1 -- 172.21.15.19:0/1508829650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe128042070 con 0x7fe140154ee0 2024-04-25T10:02:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe14710a640 1 -- 172.21.15.19:0/1508829650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe140159f20 con 0x7fe140154ee0 2024-04-25T10:02:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe13d7fa640 1 -- 172.21.15.19:0/1508829650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128030b50 con 0x7fe140154ee0 2024-04-25T10:02:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.788+0000 7fe14710a640 1 -- 172.21.15.19:0/1508829650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1401f35d0 con 0x7fe140154ee0 2024-04-25T10:02:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.789+0000 7fe14710a640 1 -- 172.21.15.19:0/1508829650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe140073990 con 0x7fe140154ee0 2024-04-25T10:02:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.790+0000 7fe13d7fa640 1 -- 172.21.15.19:0/1508829650 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe128030cf0 con 0x7fe140154ee0 2024-04-25T10:02:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.790+0000 7fe13d7fa640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe12006b4d0 0x7fe12006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.790+0000 7fe13d7fa640 1 -- 172.21.15.19:0/1508829650 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe1280aaeb0 con 0x7fe140154ee0 2024-04-25T10:02:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.790+0000 7fe144ea6640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe12006b4d0 0x7fe12006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.791+0000 7fe144ea6640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe12006b4d0 0x7fe12006d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe1380012e0 tx=0x7fe138001590 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:21.795+0000 7fe13d7fa640 1 -- 172.21.15.19:0/1508829650 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe128032e20 con 0x7fe140154ee0 2024-04-25T10:02:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.010+0000 7fe14710a640 1 -- 172.21.15.19:0/1508829650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe14006d360 con 0x7fe140154ee0 2024-04-25T10:02:22.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.011+0000 7fe13d7fa640 1 -- 172.21.15.19:0/1508829650 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4158 (secure 0 0 0) 0x7fe128045360 con 0x7fe140154ee0 2024-04-25T10:02:22.016 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:22.016 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:02:14.631627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9210":{"gid":9210,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":17,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-04-25T10:02:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 -- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe12006b4d0 msgr2=0x7fe12006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe12006b4d0 0x7fe12006d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe1380012e0 tx=0x7fe138001590 comp rx=0 tx=0).stop 2024-04-25T10:02:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 -- 172.21.15.19:0/1508829650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140154ee0 msgr2=0x7fe1401597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140154ee0 0x7fe1401597b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe128035040 tx=0x7fe1280046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 -- 172.21.15.19:0/1508829650 shutdown_connections 2024-04-25T10:02:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe12006b4d0 0x7fe12006d990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140154ee0 0x7fe1401597b0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1400cfec0 0x7fe140152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 --2- 172.21.15.19:0/1508829650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1400ce3c0 0x7fe14015b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 -- 172.21.15.19:0/1508829650 >> 172.21.15.19:0/1508829650 conn(0x7fe14006abe0 msgr2=0x7fe1400b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 -- 172.21.15.19:0/1508829650 shutdown_connections 2024-04-25T10:02:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.013+0000 7fe14710a640 1 -- 172.21.15.19:0/1508829650 wait complete. 2024-04-25T10:02:22.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:22.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.126+0000 7fe4b681c640 1 Processor -- start 2024-04-25T10:02:22.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.126+0000 7fe4b681c640 1 -- start start 2024-04-25T10:02:22.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.127+0000 7fe4b681c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4b006abe0 0x7fe4b006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.127+0000 7fe4b681c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4b00ce330 0x7fe4b006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.127+0000 7fe4b681c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b01626e0 con 0x7fe4b00700a0 2024-04-25T10:02:22.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.127+0000 7fe4b681c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b006b500 con 0x7fe4b00ce330 2024-04-25T10:02:22.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.127+0000 7fe4b681c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b00cd270 con 0x7fe4b006abe0 2024-04-25T10:02:22.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.127+0000 7fe4af7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4b00ce330 0x7fe4b006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.127+0000 7fe4af7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4b00ce330 0x7fe4b006fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53814/0 (socket says 172.21.15.19:53814) 2024-04-25T10:02:22.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.127+0000 7fe4af7fe640 1 -- 172.21.15.19:0/3233132796 learned_addr learned my addr 172.21.15.19:0/3233132796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:22.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.127+0000 7fe4affff640 1 --2- 172.21.15.19:0/3233132796 >> v2:172.21.15.112:3300/0 conn(0x7fe4b006abe0 0x7fe4b006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.128+0000 7fe4af7fe640 1 -- 172.21.15.19:0/3233132796 >> v2:172.21.15.112:3300/0 conn(0x7fe4b006abe0 msgr2=0x7fe4b006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.128+0000 7fe4af7fe640 1 --2- 172.21.15.19:0/3233132796 >> v2:172.21.15.112:3300/0 conn(0x7fe4b006abe0 0x7fe4b006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.128+0000 7fe4af7fe640 1 -- 172.21.15.19:0/3233132796 >> v1:172.21.15.112:6789/0 conn(0x7fe4b00700a0 legacy=0x7fe4b006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.128+0000 7fe4af7fe640 1 -- 172.21.15.19:0/3233132796 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b00cd550 con 0x7fe4b00ce330 2024-04-25T10:02:22.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.128+0000 7fe4af7fe640 1 --2- 172.21.15.19:0/3233132796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b00ce330 0x7fe4b006fb60 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fe4b006e200 tx=0x7fe4a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7713cc4f8d82fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.128+0000 7fe4aeffd640 1 -- 172.21.15.19:0/3233132796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0037070 con 0x7fe4b00ce330 2024-04-25T10:02:22.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.129+0000 7fe4aeffd640 1 -- 172.21.15.19:0/3233132796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a0032440 con 0x7fe4b00ce330 2024-04-25T10:02:22.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.129+0000 7fe4aeffd640 1 -- 172.21.15.19:0/3233132796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0032810 con 0x7fe4b00ce330 2024-04-25T10:02:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.129+0000 7fe4b681c640 1 -- 172.21.15.19:0/3233132796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b00ce330 msgr2=0x7fe4b006fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.129+0000 7fe4b681c640 1 --2- 172.21.15.19:0/3233132796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b00ce330 0x7fe4b006fb60 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fe4b006e200 tx=0x7fe4a002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.129+0000 7fe4b681c640 1 -- 172.21.15.19:0/3233132796 shutdown_connections 2024-04-25T10:02:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.129+0000 7fe4b681c640 1 --2- 172.21.15.19:0/3233132796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b00ce330 0x7fe4b006fb60 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.129+0000 7fe4b681c640 1 --2- 172.21.15.19:0/3233132796 >> v2:172.21.15.112:3300/0 conn(0x7fe4b006abe0 0x7fe4b006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.129+0000 7fe4b681c640 1 -- 172.21.15.19:0/3233132796 >> 172.21.15.19:0/3233132796 conn(0x7fe4b00b2220 msgr2=0x7fe4b00b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.129+0000 7fe4b681c640 1 -- 172.21.15.19:0/3233132796 shutdown_connections 2024-04-25T10:02:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.129+0000 7fe4b681c640 1 -- 172.21.15.19:0/3233132796 wait complete. 2024-04-25T10:02:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.130+0000 7fe4b681c640 1 Processor -- start 2024-04-25T10:02:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.131+0000 7fe4b681c640 1 -- start start 2024-04-25T10:02:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.131+0000 7fe4b681c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b006abe0 0x7fe4b015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.131+0000 7fe4b681c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b00700a0 0x7fe4b0159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.131+0000 7fe4b681c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4b00ce330 0x7fe4b0150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.131+0000 7fe4b681c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b0161780 con 0x7fe4b006abe0 2024-04-25T10:02:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.131+0000 7fe4b681c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00cdb10 con 0x7fe4b00700a0 2024-04-25T10:02:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.131+0000 7fe4b681c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b006eb00 con 0x7fe4b00ce330 2024-04-25T10:02:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.131+0000 7fe4affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b006abe0 0x7fe4b015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.131+0000 7fe4affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b006abe0 0x7fe4b015cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53820/0 (socket says 172.21.15.19:53820) 2024-04-25T10:02:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.131+0000 7fe4affff640 1 -- 172.21.15.19:0/3119995574 learned_addr learned my addr 172.21.15.19:0/3119995574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4b4db9640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4b00ce330 0x7fe4b0150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4af7fe640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b00700a0 0x7fe4b0159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4affff640 1 -- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4b00ce330 msgr2=0x7fe4b0150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4affff640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4b00ce330 0x7fe4b0150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4affff640 1 -- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b00700a0 msgr2=0x7fe4b0159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4affff640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b00700a0 0x7fe4b0159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4affff640 1 -- 172.21.15.19:0/3119995574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b015b5b0 con 0x7fe4b006abe0 2024-04-25T10:02:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4b4db9640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4b00ce330 0x7fe4b0150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4affff640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b006abe0 0x7fe4b015cf00 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fe49800e9b0 tx=0x7fe49800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4acff9640 1 -- 172.21.15.19:0/3119995574 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498017070 con 0x7fe4b006abe0 2024-04-25T10:02:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4acff9640 1 -- 172.21.15.19:0/3119995574 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49800c9a0 con 0x7fe4b006abe0 2024-04-25T10:02:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4b681c640 1 -- 172.21.15.19:0/3119995574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b015b890 con 0x7fe4b006abe0 2024-04-25T10:02:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4acff9640 1 -- 172.21.15.19:0/3119995574 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49800cd90 con 0x7fe4b006abe0 2024-04-25T10:02:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.132+0000 7fe4b681c640 1 -- 172.21.15.19:0/3119995574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b015be50 con 0x7fe4b006abe0 2024-04-25T10:02:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.133+0000 7fe4b681c640 1 -- 172.21.15.19:0/3119995574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b0073990 con 0x7fe4b006abe0 2024-04-25T10:02:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.133+0000 7fe4acff9640 1 -- 172.21.15.19:0/3119995574 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe49801a070 con 0x7fe4b006abe0 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.134+0000 7fe4acff9640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe49006b4d0 0x7fe49006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.134+0000 7fe4acff9640 1 -- 172.21.15.19:0/3119995574 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe49808a730 con 0x7fe4b006abe0 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.134+0000 7fe4af7fe640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe49006b4d0 0x7fe49006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.134+0000 7fe4af7fe640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe49006b4d0 0x7fe49006d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0034000 tx=0x7fe4a002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.138+0000 7fe4acff9640 1 -- 172.21.15.19:0/3119995574 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe498059460 con 0x7fe4b006abe0 2024-04-25T10:02:22.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.324+0000 7fe4b681c640 1 -- 172.21.15.19:0/3119995574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4b00b6670 con 0x7fe4b006abe0 2024-04-25T10:02:22.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.325+0000 7fe4acff9640 1 -- 172.21.15.19:0/3119995574 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+17015 (secure 0 0 0) 0x7fe498058d60 con 0x7fe4b006abe0 2024-04-25T10:02:22.330 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":130,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:12.609472+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:27.270153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:27.977083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/250977615":"2024-04-26T10:02:12.609390+0000","172.21.15.19:6833/250977615":"2024-04-26T10:02:12.609390+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.327+0000 7fe4b681c640 1 -- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe49006b4d0 msgr2=0x7fe49006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.327+0000 7fe4b681c640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe49006b4d0 0x7fe49006d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0034000 tx=0x7fe4a002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.327+0000 7fe4b681c640 1 -- 172.21.15.19:0/3119995574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b006abe0 msgr2=0x7fe4b015cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.327+0000 7fe4b681c640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b006abe0 0x7fe4b015cf00 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fe49800e9b0 tx=0x7fe49800ee70 comp rx=0 tx=0).stop 2024-04-25T10:02:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.327+0000 7fe4b681c640 1 -- 172.21.15.19:0/3119995574 shutdown_connections 2024-04-25T10:02:22.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.327+0000 7fe4b681c640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe49006b4d0 0x7fe49006d990 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.327+0000 7fe4b681c640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4b00ce330 0x7fe4b0150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.327+0000 7fe4b681c640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b00700a0 0x7fe4b0159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.327+0000 7fe4b681c640 1 --2- 172.21.15.19:0/3119995574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b006abe0 0x7fe4b015cf00 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.327+0000 7fe4b681c640 1 -- 172.21.15.19:0/3119995574 >> 172.21.15.19:0/3119995574 conn(0x7fe4b00b2220 msgr2=0x7fe4b00ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.327+0000 7fe4b681c640 1 -- 172.21.15.19:0/3119995574 shutdown_connections 2024-04-25T10:02:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.328+0000 7fe4b681c640 1 -- 172.21.15.19:0/3119995574 wait complete. 2024-04-25T10:02:22.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:27.270153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:27.977083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:22.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:22.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.444+0000 7ff9aff9e640 1 Processor -- start 2024-04-25T10:02:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.444+0000 7ff9aff9e640 1 -- start start 2024-04-25T10:02:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9aff9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff9a80b5620 0x7ff9a80b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9aff9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a806cf50 con 0x7ff9a806c2d0 2024-04-25T10:02:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9aff9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a80ce530 con 0x7ff9a815b7d0 2024-04-25T10:02:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9aff9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a80ce810 con 0x7ff9a80b5620 2024-04-25T10:02:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9ae53b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff9a806c2d0 0x7ff9a806c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50344/0 (socket says 172.21.15.19:50344) 2024-04-25T10:02:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9ae53b640 1 -- 172.21.15.19:0/1116297336 learned_addr learned my addr 172.21.15.19:0/1116297336 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9ad539640 1 --2- 172.21.15.19:0/1116297336 >> v2:172.21.15.112:3300/0 conn(0x7ff9a80b5620 0x7ff9a80b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665448414 0 0) 0x7ff9a806cf50 con 0x7ff9a806c2d0 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff994003520 con 0x7ff9a806c2d0 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3088475253 0 0) 0x7ff994003520 con 0x7ff9a806c2d0 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 >> v2:172.21.15.112:3300/0 conn(0x7ff9a80b5620 msgr2=0x7ff9a80b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 --2- 172.21.15.19:0/1116297336 >> v2:172.21.15.112:3300/0 conn(0x7ff9a80b5620 0x7ff9a80b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 >> v1:172.21.15.112:6789/0 conn(0x7ff9a815b7d0 legacy=0x7ff9a80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a80ceaf0 con 0x7ff9a806c2d0 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7ff9a00019f0 con 0x7ff9a806c2d0 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9ad539640 1 --2- 172.21.15.19:0/1116297336 >> v2:172.21.15.112:3300/0 conn(0x7ff9a80b5620 0x7ff9a80b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 >> v1:172.21.15.19:6789/0 conn(0x7ff9a806c2d0 legacy=0x7ff9a806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 --2- 172.21.15.19:0/1116297336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9940039b0 0x7ff994003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 --2- 172.21.15.19:0/1116297336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9940043b0 0x7ff994004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 --2- 172.21.15.19:0/1116297336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff994005080 0x7ff99400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80ce810 con 0x7ff994005080 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99400bfe0 con 0x7ff9940043b0 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.445+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99400c1e0 con 0x7ff9940039b0 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.446+0000 7ff9ad539640 1 --2- 172.21.15.19:0/1116297336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9940043b0 0x7ff994004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.446+0000 7ff9ae53b640 1 --2- 172.21.15.19:0/1116297336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff994005080 0x7ff99400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.446+0000 7ff9add3a640 1 --2- 172.21.15.19:0/1116297336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9940039b0 0x7ff994003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.446+0000 7ff9ae53b640 1 -- 172.21.15.19:0/1116297336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9940039b0 msgr2=0x7ff994003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.446+0000 7ff9ae53b640 1 --2- 172.21.15.19:0/1116297336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9940039b0 0x7ff994003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.446+0000 7ff9ae53b640 1 -- 172.21.15.19:0/1116297336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9940043b0 msgr2=0x7ff994004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.446+0000 7ff9ae53b640 1 --2- 172.21.15.19:0/1116297336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9940043b0 0x7ff994004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.446+0000 7ff9ae53b640 1 -- 172.21.15.19:0/1116297336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff994003520 con 0x7ff994005080 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.446+0000 7ff9ae53b640 1 -- 172.21.15.19:0/1116297336 reap_dead start 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.447+0000 7ff9ae53b640 1 --2- 172.21.15.19:0/1116297336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff994005080 0x7ff99400b9c0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000a7c0 tx=0x7ff994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233d88c22cc57586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.447+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000a410 con 0x7ff994005080 2024-04-25T10:02:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.447+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a002f070 con 0x7ff994005080 2024-04-25T10:02:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.447+0000 7ff9acd38640 1 -- 172.21.15.19:0/1116297336 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a002bc00 con 0x7ff994005080 2024-04-25T10:02:22.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.447+0000 7ff9aff9e640 1 -- 172.21.15.19:0/1116297336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff994005080 msgr2=0x7ff99400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.447+0000 7ff9aff9e640 1 --2- 172.21.15.19:0/1116297336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff994005080 0x7ff99400b9c0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000a7c0 tx=0x7ff994005830 comp rx=0 tx=0).stop 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.447+0000 7ff9aff9e640 1 -- 172.21.15.19:0/1116297336 shutdown_connections 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.447+0000 7ff9aff9e640 1 --2- 172.21.15.19:0/1116297336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff994005080 0x7ff99400b9c0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.447+0000 7ff9aff9e640 1 -- 172.21.15.19:0/1116297336 >> 172.21.15.19:0/1116297336 conn(0x7ff9a80d0800 msgr2=0x7ff9a80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.447+0000 7ff9aff9e640 1 -- 172.21.15.19:0/1116297336 shutdown_connections 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.447+0000 7ff9aff9e640 1 -- 172.21.15.19:0/1116297336 wait complete. 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.449+0000 7ff9aff9e640 1 Processor -- start 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.449+0000 7ff9aff9e640 1 -- start start 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.449+0000 7ff9aff9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff994005080 0x7ff9a80c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.449+0000 7ff9aff9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9a81514e0 0x7ff9a81f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.449+0000 7ff9aff9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9a80cc450 0x7ff9a80cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.449+0000 7ff9aff9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a815bd30 con 0x7ff9a81514e0 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.449+0000 7ff9aff9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a815bbb0 con 0x7ff9a80cc450 2024-04-25T10:02:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.449+0000 7ff9aff9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80b5ab0 con 0x7ff994005080 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.449+0000 7ff9ad539640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9a81514e0 0x7ff9a81f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.449+0000 7ff9ad539640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9a81514e0 0x7ff9a81f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53850/0 (socket says 172.21.15.19:53850) 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.449+0000 7ff9ad539640 1 -- 172.21.15.19:0/4223931337 learned_addr learned my addr 172.21.15.19:0/4223931337 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff9ae53b640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9a80cc450 0x7ff9a80cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff9ad539640 1 -- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff994005080 msgr2=0x7ff9a80c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff9ad539640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff994005080 0x7ff9a80c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff9ad539640 1 -- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9a80cc450 msgr2=0x7ff9a80cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff9ad539640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9a80cc450 0x7ff9a80cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff9ad539640 1 -- 172.21.15.19:0/4223931337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a80cd1b0 con 0x7ff9a81514e0 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff9ad539640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9a81514e0 0x7ff9a81f3070 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7ff998011ac0 tx=0x7ff998011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff99e7fc640 1 -- 172.21.15.19:0/4223931337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99800fc10 con 0x7ff9a81514e0 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff99e7fc640 1 -- 172.21.15.19:0/4223931337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99800fdb0 con 0x7ff9a81514e0 2024-04-25T10:02:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff9aff9e640 1 -- 172.21.15.19:0/4223931337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a80cd490 con 0x7ff9a81514e0 2024-04-25T10:02:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff99e7fc640 1 -- 172.21.15.19:0/4223931337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998004260 con 0x7ff9a81514e0 2024-04-25T10:02:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.450+0000 7ff9aff9e640 1 -- 172.21.15.19:0/4223931337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a8151c50 con 0x7ff9a81514e0 2024-04-25T10:02:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.451+0000 7ff9aff9e640 1 -- 172.21.15.19:0/4223931337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a8073a00 con 0x7ff9a81514e0 2024-04-25T10:02:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.452+0000 7ff99e7fc640 1 -- 172.21.15.19:0/4223931337 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff998012040 con 0x7ff9a81514e0 2024-04-25T10:02:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.452+0000 7ff99e7fc640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff98c06b4d0 0x7ff98c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.452+0000 7ff99e7fc640 1 -- 172.21.15.19:0/4223931337 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff99808a8d0 con 0x7ff9a81514e0 2024-04-25T10:02:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.452+0000 7ff9add3a640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff98c06b4d0 0x7ff98c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.453+0000 7ff9add3a640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff98c06b4d0 0x7ff98c06d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4001880 tx=0x7ff9a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.456+0000 7ff99e7fc640 1 -- 172.21.15.19:0/4223931337 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff998019030 con 0x7ff9a81514e0 2024-04-25T10:02:22.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.637+0000 7ff9aff9e640 1 -- 172.21.15.19:0/4223931337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9a8160ae0 con 0x7ff9a81514e0 2024-04-25T10:02:22.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.637+0000 7ff99e7fc640 1 -- 172.21.15.19:0/4223931337 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff998059570 con 0x7ff9a81514e0 2024-04-25T10:02:22.642 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:02:22.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 -- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff98c06b4d0 msgr2=0x7ff98c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff98c06b4d0 0x7ff98c06d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4001880 tx=0x7ff9a4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:22.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 -- 172.21.15.19:0/4223931337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9a81514e0 msgr2=0x7ff9a81f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9a81514e0 0x7ff9a81f3070 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7ff998011ac0 tx=0x7ff998011f80 comp rx=0 tx=0).stop 2024-04-25T10:02:22.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 -- 172.21.15.19:0/4223931337 shutdown_connections 2024-04-25T10:02:22.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff98c06b4d0 0x7ff98c06d990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9a80cc450 0x7ff9a80cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9a81514e0 0x7ff9a81f3070 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 --2- 172.21.15.19:0/4223931337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff994005080 0x7ff9a80c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 -- 172.21.15.19:0/4223931337 >> 172.21.15.19:0/4223931337 conn(0x7ff9a80d0800 msgr2=0x7ff9a80b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 -- 172.21.15.19:0/4223931337 shutdown_connections 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.639+0000 7ff9aff9e640 1 -- 172.21.15.19:0/4223931337 wait complete. 2024-04-25T10:02:22.656 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:22.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.755+0000 7fb4d525d640 1 Processor -- start 2024-04-25T10:02:22.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.755+0000 7fb4d525d640 1 -- start start 2024-04-25T10:02:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4d525d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4d015dc20 0x7fb4d00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4d525d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d006cda0 con 0x7fb4d00d32f0 2024-04-25T10:02:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4d525d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d015e000 con 0x7fb4d0161710 2024-04-25T10:02:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4d525d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00d1de0 con 0x7fb4d015dc20 2024-04-25T10:02:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4cf59e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb4d00d32f0 0x7fb4d006c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50352/0 (socket says 172.21.15.19:50352) 2024-04-25T10:02:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4cf59e640 1 -- 172.21.15.19:0/584035305 learned_addr learned my addr 172.21.15.19:0/584035305 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4ced9d640 1 --2- 172.21.15.19:0/584035305 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4d015dc20 0x7fb4d00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4ce59c640 1 -- 172.21.15.19:0/584035305 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190951705 0 0) 0x7fb4d006cda0 con 0x7fb4d00d32f0 2024-04-25T10:02:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4ce59c640 1 -- 172.21.15.19:0/584035305 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4bc003520 con 0x7fb4d00d32f0 2024-04-25T10:02:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4ce59c640 1 -- 172.21.15.19:0/584035305 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473189660 0 0) 0x7fb4d015e000 con 0x7fb4d0161710 2024-04-25T10:02:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4ce59c640 1 -- 172.21.15.19:0/584035305 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4d006cda0 con 0x7fb4d0161710 2024-04-25T10:02:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4ced9d640 1 -- 172.21.15.19:0/584035305 >> v1:172.21.15.112:6789/0 conn(0x7fb4d0161710 legacy=0x7fb4d006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4ced9d640 1 -- 172.21.15.19:0/584035305 >> v1:172.21.15.19:6789/0 conn(0x7fb4d00d32f0 legacy=0x7fb4d006c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.756+0000 7fb4ced9d640 1 -- 172.21.15.19:0/584035305 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d0161c60 con 0x7fb4d015dc20 2024-04-25T10:02:22.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.757+0000 7fb4ced9d640 1 --2- 172.21.15.19:0/584035305 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4d015dc20 0x7fb4d00ce670 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00097d0 tx=0x7fb4c002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7318dca2fe519ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.757+0000 7fb4ce59c640 1 -- 172.21.15.19:0/584035305 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0037070 con 0x7fb4d015dc20 2024-04-25T10:02:22.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.757+0000 7fb4ce59c640 1 -- 172.21.15.19:0/584035305 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c0032440 con 0x7fb4d015dc20 2024-04-25T10:02:22.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.757+0000 7fb4ce59c640 1 -- 172.21.15.19:0/584035305 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0032810 con 0x7fb4d015dc20 2024-04-25T10:02:22.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.758+0000 7fb4d525d640 1 -- 172.21.15.19:0/584035305 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4d015dc20 msgr2=0x7fb4d00ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.758+0000 7fb4d525d640 1 --2- 172.21.15.19:0/584035305 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4d015dc20 0x7fb4d00ce670 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00097d0 tx=0x7fb4c002a210 comp rx=0 tx=0).stop 2024-04-25T10:02:22.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.758+0000 7fb4d525d640 1 -- 172.21.15.19:0/584035305 shutdown_connections 2024-04-25T10:02:22.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.758+0000 7fb4d525d640 1 --2- 172.21.15.19:0/584035305 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4d015dc20 0x7fb4d00ce670 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.758+0000 7fb4d525d640 1 -- 172.21.15.19:0/584035305 >> 172.21.15.19:0/584035305 conn(0x7fb4d00d0420 msgr2=0x7fb4d00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.758+0000 7fb4d525d640 1 -- 172.21.15.19:0/584035305 shutdown_connections 2024-04-25T10:02:22.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.758+0000 7fb4d525d640 1 -- 172.21.15.19:0/584035305 wait complete. 2024-04-25T10:02:22.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.759+0000 7fb4d525d640 1 Processor -- start 2024-04-25T10:02:22.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.759+0000 7fb4d525d640 1 -- start start 2024-04-25T10:02:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4d525d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4d00d32f0 0x7fb4d015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4d525d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d015dc20 0x7fb4d0160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4d525d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4d0161710 0x7fb4d01f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4d525d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0162500 con 0x7fb4d015dc20 2024-04-25T10:02:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4d525d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0162220 con 0x7fb4d00d32f0 2024-04-25T10:02:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4d525d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00b5e50 con 0x7fb4d0161710 2024-04-25T10:02:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d015dc20 0x7fb4d0160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d015dc20 0x7fb4d0160950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53854/0 (socket says 172.21.15.19:53854) 2024-04-25T10:02:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4c7fff640 1 -- 172.21.15.19:0/2688104675 learned_addr learned my addr 172.21.15.19:0/2688104675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4ced9d640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4d00d32f0 0x7fb4d015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4cf59e640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4d0161710 0x7fb4d01f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4c7fff640 1 -- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4d0161710 msgr2=0x7fb4d01f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.760+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4d0161710 0x7fb4d01f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.761+0000 7fb4c7fff640 1 -- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4d00d32f0 msgr2=0x7fb4d015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.761+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4d00d32f0 0x7fb4d015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.761+0000 7fb4c7fff640 1 -- 172.21.15.19:0/2688104675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d015ea00 con 0x7fb4d015dc20 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.761+0000 7fb4c7fff640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d015dc20 0x7fb4d0160950 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800c490 tx=0x7fb4b800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.761+0000 7fb4c77fe640 1 -- 172.21.15.19:0/2688104675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80052d0 con 0x7fb4d015dc20 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.761+0000 7fb4c77fe640 1 -- 172.21.15.19:0/2688104675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b8005470 con 0x7fb4d015dc20 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.761+0000 7fb4d525d640 1 -- 172.21.15.19:0/2688104675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d015ece0 con 0x7fb4d015dc20 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.761+0000 7fb4c77fe640 1 -- 172.21.15.19:0/2688104675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8012850 con 0x7fb4d015dc20 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.761+0000 7fb4d525d640 1 -- 172.21.15.19:0/2688104675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d015f2a0 con 0x7fb4d015dc20 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.762+0000 7fb4d525d640 1 -- 172.21.15.19:0/2688104675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d0073a00 con 0x7fb4d015dc20 2024-04-25T10:02:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.762+0000 7fb4c77fe640 1 -- 172.21.15.19:0/2688104675 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4b80129f0 con 0x7fb4d015dc20 2024-04-25T10:02:22.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.763+0000 7fb4c77fe640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4b006b4d0 0x7fb4b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.763+0000 7fb4c77fe640 1 -- 172.21.15.19:0/2688104675 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb4b8089ec0 con 0x7fb4d015dc20 2024-04-25T10:02:22.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.763+0000 7fb4ced9d640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4b006b4d0 0x7fb4b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.764+0000 7fb4ced9d640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4b006b4d0 0x7fb4b006d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0034000 tx=0x7fb4c002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.767+0000 7fb4c77fe640 1 -- 172.21.15.19:0/2688104675 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4b80585a0 con 0x7fb4d015dc20 2024-04-25T10:02:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.947+0000 7fb4d525d640 1 -- 172.21.15.19:0/2688104675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4d015e6f0 con 0x7fb4d015dc20 2024-04-25T10:02:22.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.947+0000 7fb4c77fe640 1 -- 172.21.15.19:0/2688104675 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4b805c640 con 0x7fb4d015dc20 2024-04-25T10:02:22.952 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:02:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 -- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4b006b4d0 msgr2=0x7fb4b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4b006b4d0 0x7fb4b006d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0034000 tx=0x7fb4c002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 -- 172.21.15.19:0/2688104675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d015dc20 msgr2=0x7fb4d0160950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d015dc20 0x7fb4d0160950 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800c490 tx=0x7fb4b800c950 comp rx=0 tx=0).stop 2024-04-25T10:02:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 -- 172.21.15.19:0/2688104675 shutdown_connections 2024-04-25T10:02:22.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4b006b4d0 0x7fb4b006d990 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4d0161710 0x7fb4d01f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d015dc20 0x7fb4d0160950 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 --2- 172.21.15.19:0/2688104675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4d00d32f0 0x7fb4d015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 -- 172.21.15.19:0/2688104675 >> 172.21.15.19:0/2688104675 conn(0x7fb4d00d0420 msgr2=0x7fb4d00cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 -- 172.21.15.19:0/2688104675 shutdown_connections 2024-04-25T10:02:22.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:22.949+0000 7fb4d525d640 1 -- 172.21.15.19:0/2688104675 wait complete. 2024-04-25T10:02:22.965 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:23.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.064+0000 7f64bfb10640 1 Processor -- start 2024-04-25T10:02:23.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.064+0000 7f64bfb10640 1 -- start start 2024-04-25T10:02:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.065+0000 7f64bfb10640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64b815dd80 0x7f64b80b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.065+0000 7f64bfb10640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b80b3930 con 0x7f64b80ad570 2024-04-25T10:02:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.065+0000 7f64bfb10640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b80b3c10 con 0x7f64b806abe0 2024-04-25T10:02:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.065+0000 7f64bfb10640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b806e410 con 0x7f64b815dd80 2024-04-25T10:02:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.065+0000 7f64be0ad640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f64b80ad570 0x7f64b806ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50356/0 (socket says 172.21.15.19:50356) 2024-04-25T10:02:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.065+0000 7f64be0ad640 1 -- 172.21.15.19:0/3883341481 learned_addr learned my addr 172.21.15.19:0/3883341481 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.065+0000 7f64bd0ab640 1 --2- 172.21.15.19:0/3883341481 >> v2:172.21.15.112:3300/0 conn(0x7f64b815dd80 0x7f64b80b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.065+0000 7f64bc8aa640 1 -- 172.21.15.19:0/3883341481 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750397930 0 0) 0x7f64b80b3930 con 0x7f64b80ad570 2024-04-25T10:02:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.065+0000 7f64bc8aa640 1 -- 172.21.15.19:0/3883341481 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64a4003520 con 0x7f64b80ad570 2024-04-25T10:02:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.066+0000 7f64bc8aa640 1 -- 172.21.15.19:0/3883341481 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939337319 0 0) 0x7f64b80b3c10 con 0x7f64b806abe0 2024-04-25T10:02:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.066+0000 7f64bc8aa640 1 -- 172.21.15.19:0/3883341481 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b80b3930 con 0x7f64b806abe0 2024-04-25T10:02:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.066+0000 7f64bd0ab640 1 -- 172.21.15.19:0/3883341481 >> v1:172.21.15.112:6789/0 conn(0x7f64b806abe0 legacy=0x7f64b806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.066+0000 7f64bd0ab640 1 -- 172.21.15.19:0/3883341481 >> v1:172.21.15.19:6789/0 conn(0x7f64b80ad570 legacy=0x7f64b806ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.066+0000 7f64bd0ab640 1 -- 172.21.15.19:0/3883341481 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b806e660 con 0x7f64b815dd80 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.066+0000 7f64bd0ab640 1 --2- 172.21.15.19:0/3883341481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b815dd80 0x7f64b80b5290 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f64a80097a0 tx=0x7f64a802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6643bab43a5ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.067+0000 7f64bc8aa640 1 -- 172.21.15.19:0/3883341481 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8037070 con 0x7f64b815dd80 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.067+0000 7f64bc8aa640 1 -- 172.21.15.19:0/3883341481 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a8032440 con 0x7f64b815dd80 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.067+0000 7f64bc8aa640 1 -- 172.21.15.19:0/3883341481 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8032810 con 0x7f64b815dd80 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.067+0000 7f64bfb10640 1 -- 172.21.15.19:0/3883341481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b815dd80 msgr2=0x7f64b80b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.067+0000 7f64bfb10640 1 --2- 172.21.15.19:0/3883341481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b815dd80 0x7f64b80b5290 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f64a80097a0 tx=0x7f64a802a230 comp rx=0 tx=0).stop 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.067+0000 7f64bfb10640 1 -- 172.21.15.19:0/3883341481 shutdown_connections 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.067+0000 7f64bfb10640 1 --2- 172.21.15.19:0/3883341481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b815dd80 0x7f64b80b5290 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.067+0000 7f64bfb10640 1 -- 172.21.15.19:0/3883341481 >> 172.21.15.19:0/3883341481 conn(0x7f64b80b5af0 msgr2=0x7f64b80b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.067+0000 7f64bfb10640 1 -- 172.21.15.19:0/3883341481 shutdown_connections 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.067+0000 7f64bfb10640 1 -- 172.21.15.19:0/3883341481 wait complete. 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.068+0000 7f64bfb10640 1 Processor -- start 2024-04-25T10:02:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.068+0000 7f64bfb10640 1 -- start start 2024-04-25T10:02:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64bfb10640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64b806abe0 0x7f64b815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64bfb10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b80ad570 0x7f64b815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64bfb10640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b815dd80 0x7f64b8157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64bfb10640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8070280 con 0x7f64b815dd80 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64bfb10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80b57d0 con 0x7f64b80ad570 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64bfb10640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8070400 con 0x7f64b806abe0 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64be0ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b815dd80 0x7f64b8157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64be0ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b815dd80 0x7f64b8157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53864/0 (socket says 172.21.15.19:53864) 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64be0ad640 1 -- 172.21.15.19:0/3458012755 learned_addr learned my addr 172.21.15.19:0/3458012755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64bd8ac640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64b806abe0 0x7f64b815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64bd0ab640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b80ad570 0x7f64b815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64be0ad640 1 -- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64b806abe0 msgr2=0x7f64b815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64be0ad640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64b806abe0 0x7f64b815aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64be0ad640 1 -- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b80ad570 msgr2=0x7f64b815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64be0ad640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b80ad570 0x7f64b815c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.069+0000 7f64be0ad640 1 -- 172.21.15.19:0/3458012755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8159c00 con 0x7f64b815dd80 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.070+0000 7f64be0ad640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b815dd80 0x7f64b8157df0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f64b000cbc0 tx=0x7f64b000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.070+0000 7f64ae7fc640 1 -- 172.21.15.19:0/3458012755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0017070 con 0x7f64b815dd80 2024-04-25T10:02:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.070+0000 7f64ae7fc640 1 -- 172.21.15.19:0/3458012755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b000ace0 con 0x7f64b815dd80 2024-04-25T10:02:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.070+0000 7f64bfb10640 1 -- 172.21.15.19:0/3458012755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b8159ee0 con 0x7f64b815dd80 2024-04-25T10:02:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.070+0000 7f64ae7fc640 1 -- 172.21.15.19:0/3458012755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0012600 con 0x7f64b815dd80 2024-04-25T10:02:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.070+0000 7f64bfb10640 1 -- 172.21.15.19:0/3458012755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b815a4a0 con 0x7f64b815dd80 2024-04-25T10:02:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.071+0000 7f64bfb10640 1 -- 172.21.15.19:0/3458012755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b8073a00 con 0x7f64b815dd80 2024-04-25T10:02:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.071+0000 7f64ae7fc640 1 -- 172.21.15.19:0/3458012755 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f64b001a070 con 0x7f64b815dd80 2024-04-25T10:02:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.072+0000 7f64ae7fc640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f649c06b4d0 0x7f649c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.072+0000 7f64ae7fc640 1 -- 172.21.15.19:0/3458012755 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f64b008a070 con 0x7f64b815dd80 2024-04-25T10:02:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.072+0000 7f64bd8ac640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f649c06b4d0 0x7f649c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.072+0000 7f64bd8ac640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f649c06b4d0 0x7f649c06d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f64b4007fd0 tx=0x7f64b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.076+0000 7f64ae7fc640 1 -- 172.21.15.19:0/3458012755 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f64b00586c0 con 0x7f64b815dd80 2024-04-25T10:02:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.275+0000 7f64bfb10640 1 -- 172.21.15.19:0/3458012755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64b815a810 con 0x7f64b815dd80 2024-04-25T10:02:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.276+0000 7f64ae7fc640 1 -- 172.21.15.19:0/3458012755 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64b005c760 con 0x7f64b815dd80 2024-04-25T10:02:23.280 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:02:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.277+0000 7f64bfb10640 1 -- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f649c06b4d0 msgr2=0x7f649c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.277+0000 7f64bfb10640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f649c06b4d0 0x7f649c06d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f64b4007fd0 tx=0x7f64b4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.277+0000 7f64bfb10640 1 -- 172.21.15.19:0/3458012755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b815dd80 msgr2=0x7f64b8157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.277+0000 7f64bfb10640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b815dd80 0x7f64b8157df0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f64b000cbc0 tx=0x7f64b000a430 comp rx=0 tx=0).stop 2024-04-25T10:02:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.277+0000 7f64bfb10640 1 -- 172.21.15.19:0/3458012755 shutdown_connections 2024-04-25T10:02:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.277+0000 7f64bfb10640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f649c06b4d0 0x7f649c06d990 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.277+0000 7f64bfb10640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b815dd80 0x7f64b8157df0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.277+0000 7f64bfb10640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64b80ad570 0x7f64b815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.277+0000 7f64bfb10640 1 --2- 172.21.15.19:0/3458012755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64b806abe0 0x7f64b815aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.277+0000 7f64bfb10640 1 -- 172.21.15.19:0/3458012755 >> 172.21.15.19:0/3458012755 conn(0x7f64b80b5af0 msgr2=0x7f64b80b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.278+0000 7f64bfb10640 1 -- 172.21.15.19:0/3458012755 shutdown_connections 2024-04-25T10:02:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.278+0000 7f64bfb10640 1 -- 172.21.15.19:0/3458012755 wait complete. 2024-04-25T10:02:23.294 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:23.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.390+0000 7f010a01f640 1 Processor -- start 2024-04-25T10:02:23.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.390+0000 7f010a01f640 1 -- start start 2024-04-25T10:02:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.391+0000 7f010a01f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0104071ac0 0x7f01040ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.391+0000 7f010a01f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010406c1d0 0x7f010406afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.391+0000 7f010a01f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010415ec60 con 0x7f0104154ed0 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.391+0000 7f010a01f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010415ef40 con 0x7f0104071ac0 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.391+0000 7f010a01f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010406db90 con 0x7f010406c1d0 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.391+0000 7f01037fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0104154ed0 0x7f01040aced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50362/0 (socket says 172.21.15.19:50362) 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.391+0000 7f01037fe640 1 -- 172.21.15.19:0/3880920494 learned_addr learned my addr 172.21.15.19:0/3880920494 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.391+0000 7f0103fff640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010406c1d0 0x7f010406afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.391+0000 7f0102ffd640 1 --2- 172.21.15.19:0/3880920494 >> v2:172.21.15.112:3300/0 conn(0x7f0104071ac0 0x7f01040ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.391+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603751403 0 0) 0x7f010415ec60 con 0x7f0104154ed0 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.391+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00ec003520 con 0x7f0104154ed0 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1811083413 0 0) 0x7f00ec003520 con 0x7f0104154ed0 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010406c1d0 msgr2=0x7f010406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010406c1d0 0x7f010406afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 >> v2:172.21.15.112:3300/0 conn(0x7f0104071ac0 msgr2=0x7f01040ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 --2- 172.21.15.19:0/3880920494 >> v2:172.21.15.112:3300/0 conn(0x7f0104071ac0 0x7f01040ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010406de70 con 0x7f0104154ed0 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f00f4001670 con 0x7f0104154ed0 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f0103fff640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010406c1d0 0x7f010406afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f0102ffd640 1 --2- 172.21.15.19:0/3880920494 >> v2:172.21.15.112:3300/0 conn(0x7f0104071ac0 0x7f01040ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 >> v1:172.21.15.19:6789/0 conn(0x7f0104154ed0 legacy=0x7f01040aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010406c1d0 0x7f00ec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00ec003ef0 0x7f00ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00ec004d40 0x7f00ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010415ef40 con 0x7f00ec004d40 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010406db90 con 0x7f00ec003ef0 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec00b9c0 con 0x7f010406c1d0 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f0103fff640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00ec004d40 0x7f00ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f01037fe640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010406c1d0 0x7f00ec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f0102ffd640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00ec003ef0 0x7f00ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f0103fff640 1 -- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010406c1d0 msgr2=0x7f00ec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f0103fff640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010406c1d0 0x7f00ec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f0103fff640 1 -- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00ec003ef0 msgr2=0x7f00ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f0103fff640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00ec003ef0 0x7f00ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.392+0000 7f0103fff640 1 -- 172.21.15.19:0/3880920494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec003520 con 0x7f00ec004d40 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.393+0000 7f0103fff640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00ec004d40 0x7f00ec0051c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00e350 tx=0x7f00ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310cc8a2385d9a70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.393+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc033070 con 0x7f00ec004d40 2024-04-25T10:02:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.393+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc02f660 con 0x7f00ec004d40 2024-04-25T10:02:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.393+0000 7f01027fc640 1 -- 172.21.15.19:0/3880920494 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc009de0 con 0x7f00ec004d40 2024-04-25T10:02:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.394+0000 7f010a01f640 1 -- 172.21.15.19:0/3880920494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00ec004d40 msgr2=0x7f00ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.394+0000 7f010a01f640 1 --2- 172.21.15.19:0/3880920494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00ec004d40 0x7f00ec0051c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00e350 tx=0x7f00ec005830 comp rx=0 tx=0).stop 2024-04-25T10:02:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.394+0000 7f0103fff640 1 -- 172.21.15.19:0/3880920494 reap_dead start 2024-04-25T10:02:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.394+0000 7f010a01f640 1 -- 172.21.15.19:0/3880920494 shutdown_connections 2024-04-25T10:02:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.394+0000 7f010a01f640 1 -- 172.21.15.19:0/3880920494 >> 172.21.15.19:0/3880920494 conn(0x7f01040af670 msgr2=0x7f01040b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.394+0000 7f010a01f640 1 -- 172.21.15.19:0/3880920494 shutdown_connections 2024-04-25T10:02:23.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.394+0000 7f010a01f640 1 -- 172.21.15.19:0/3880920494 wait complete. 2024-04-25T10:02:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.395+0000 7f010a01f640 1 Processor -- start 2024-04-25T10:02:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.395+0000 7f010a01f640 1 -- start start 2024-04-25T10:02:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f010a01f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010414e4d0 0x7f0104152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f010a01f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01041526b0 0x7f010414f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f010a01f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010414fec0 0x7f0104156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f010a01f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010406b120 con 0x7f010414e4d0 2024-04-25T10:02:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f010a01f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010406afa0 con 0x7f01041526b0 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f010a01f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010406b2a0 con 0x7f010414fec0 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f01037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010414e4d0 0x7f0104152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f01037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010414e4d0 0x7f0104152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53880/0 (socket says 172.21.15.19:53880) 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f01037fe640 1 -- 172.21.15.19:0/3754973049 learned_addr learned my addr 172.21.15.19:0/3754973049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f0103fff640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010414fec0 0x7f0104156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f0102ffd640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01041526b0 0x7f010414f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f01037fe640 1 -- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010414fec0 msgr2=0x7f0104156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f01037fe640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010414fec0 0x7f0104156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f01037fe640 1 -- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01041526b0 msgr2=0x7f010414f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f01037fe640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01041526b0 0x7f010414f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f01037fe640 1 -- 172.21.15.19:0/3754973049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01041518b0 con 0x7f010414e4d0 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f0103fff640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010414fec0 0x7f0104156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:23.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f01037fe640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010414e4d0 0x7f0104152170 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f00f4007310 tx=0x7f00f40077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f00ebfff640 1 -- 172.21.15.19:0/3754973049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f400fa80 con 0x7f010414e4d0 2024-04-25T10:02:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.396+0000 7f00ebfff640 1 -- 172.21.15.19:0/3754973049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f4018070 con 0x7f010414e4d0 2024-04-25T10:02:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.397+0000 7f00ebfff640 1 -- 172.21.15.19:0/3754973049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f400fe80 con 0x7f010414e4d0 2024-04-25T10:02:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.397+0000 7f010a01f640 1 -- 172.21.15.19:0/3754973049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0104155770 con 0x7f010414e4d0 2024-04-25T10:02:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.397+0000 7f010a01f640 1 -- 172.21.15.19:0/3754973049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010414e8b0 con 0x7f010414e4d0 2024-04-25T10:02:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.397+0000 7f010a01f640 1 -- 172.21.15.19:0/3754973049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01040b2b10 con 0x7f010414e4d0 2024-04-25T10:02:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.398+0000 7f00ebfff640 1 -- 172.21.15.19:0/3754973049 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f00f401d460 con 0x7f010414e4d0 2024-04-25T10:02:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.398+0000 7f00ebfff640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f00f006b4d0 0x7f00f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.398+0000 7f00ebfff640 1 -- 172.21.15.19:0/3754973049 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f00f408a690 con 0x7f010414e4d0 2024-04-25T10:02:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.398+0000 7f0102ffd640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f00f006b4d0 0x7f00f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.399+0000 7f0102ffd640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f00f006b4d0 0x7f00f006d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0104154290 tx=0x7f00f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.403+0000 7f00ebfff640 1 -- 172.21.15.19:0/3754973049 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00f4059290 con 0x7f010414e4d0 2024-04-25T10:02:23.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.622+0000 7f010a01f640 1 -- 172.21.15.19:0/3754973049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0104073990 con 0x7f010414e4d0 2024-04-25T10:02:23.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.623+0000 7f00ebfff640 1 -- 172.21.15.19:0/3754973049 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f00f4058b90 con 0x7f010414e4d0 2024-04-25T10:02:23.628 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:23.628 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:02:14.631627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9210":{"gid":9210,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":17,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:23.628 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-04-25T10:02:23.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 -- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f00f006b4d0 msgr2=0x7f00f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f00f006b4d0 0x7f00f006d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0104154290 tx=0x7f00f8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:23.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 -- 172.21.15.19:0/3754973049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010414e4d0 msgr2=0x7f0104152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010414e4d0 0x7f0104152170 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f00f4007310 tx=0x7f00f40077d0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 -- 172.21.15.19:0/3754973049 shutdown_connections 2024-04-25T10:02:23.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f00f006b4d0 0x7f00f006d990 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f010414fec0 0x7f0104156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01041526b0 0x7f010414f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 --2- 172.21.15.19:0/3754973049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010414e4d0 0x7f0104152170 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 -- 172.21.15.19:0/3754973049 >> 172.21.15.19:0/3754973049 conn(0x7f0104160bd0 msgr2=0x7f01040aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 -- 172.21.15.19:0/3754973049 shutdown_connections 2024-04-25T10:02:23.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.625+0000 7f010a01f640 1 -- 172.21.15.19:0/3754973049 wait complete. 2024-04-25T10:02:23.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.737+0000 7f5075ce1640 1 Processor -- start 2024-04-25T10:02:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.738+0000 7f5075ce1640 1 -- start start 2024-04-25T10:02:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.738+0000 7f5075ce1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f50700b2cb0 0x7f50700d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.738+0000 7f5075ce1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50700d1020 0x7f50700d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.738+0000 7f5075ce1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50700b97a0 con 0x7f507015dbe0 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.738+0000 7f5075ce1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507015dfc0 con 0x7f50700b2cb0 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.738+0000 7f5075ce1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50700d1a00 con 0x7f50700d1020 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.738+0000 7f506f7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f507015dbe0 0x7f50700d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50376/0 (socket says 172.21.15.19:50376) 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.738+0000 7f506f7fe640 1 -- 172.21.15.19:0/868951300 learned_addr learned my addr 172.21.15.19:0/868951300 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506ffff640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50700d1020 0x7f50700d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506effd640 1 --2- 172.21.15.19:0/868951300 >> v2:172.21.15.112:3300/0 conn(0x7f50700b2cb0 0x7f50700d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564764420 0 0) 0x7f50700b97a0 con 0x7f507015dbe0 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5058003520 con 0x7f507015dbe0 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 444853658 0 0) 0x7f5058003520 con 0x7f507015dbe0 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50700d1020 msgr2=0x7f50700d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50700d1020 0x7f50700d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 >> v2:172.21.15.112:3300/0 conn(0x7f50700b2cb0 msgr2=0x7f50700d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 --2- 172.21.15.19:0/868951300 >> v2:172.21.15.112:3300/0 conn(0x7f50700b2cb0 0x7f50700d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700d1c50 con 0x7f507015dbe0 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f5060001670 con 0x7f507015dbe0 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 >> v1:172.21.15.19:6789/0 conn(0x7f507015dbe0 legacy=0x7f50700d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50580039b0 0x7f5058003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50580043a0 0x7f5058004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5058004ef0 0x7f505800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700d1a00 con 0x7f50580043a0 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505800bf00 con 0x7f5058004ef0 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505800c080 con 0x7f50580039b0 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506ffff640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50700d1020 0x7f50700d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:23.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506effd640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50580043a0 0x7f5058004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506f7fe640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50580039b0 0x7f5058003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.739+0000 7f506ffff640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5058004ef0 0x7f505800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.740+0000 7f506effd640 1 -- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50580039b0 msgr2=0x7f5058003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.740+0000 7f506effd640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50580039b0 0x7f5058003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.740+0000 7f506effd640 1 -- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5058004ef0 msgr2=0x7f505800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.740+0000 7f506effd640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5058004ef0 0x7f505800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.740+0000 7f506effd640 1 -- 172.21.15.19:0/868951300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5058003520 con 0x7f50580043a0 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.740+0000 7f506effd640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50580043a0 0x7f5058004880 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f5064009a90 tx=0x7f5058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e576d5b423c3232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.740+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064036070 con 0x7f50580043a0 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.740+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5064031400 con 0x7f50580043a0 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.740+0000 7f506e7fc640 1 -- 172.21.15.19:0/868951300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064009690 con 0x7f50580043a0 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.741+0000 7f5075ce1640 1 -- 172.21.15.19:0/868951300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50580043a0 msgr2=0x7f5058004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.741+0000 7f5075ce1640 1 --2- 172.21.15.19:0/868951300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50580043a0 0x7f5058004880 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f5064009a90 tx=0x7f5058005830 comp rx=0 tx=0).stop 2024-04-25T10:02:23.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.741+0000 7f506ffff640 1 -- 172.21.15.19:0/868951300 reap_dead start 2024-04-25T10:02:23.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.741+0000 7f5075ce1640 1 -- 172.21.15.19:0/868951300 shutdown_connections 2024-04-25T10:02:23.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.741+0000 7f5075ce1640 1 -- 172.21.15.19:0/868951300 >> 172.21.15.19:0/868951300 conn(0x7f5070061a20 msgr2=0x7f5070061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.741+0000 7f5075ce1640 1 -- 172.21.15.19:0/868951300 shutdown_connections 2024-04-25T10:02:23.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.741+0000 7f5075ce1640 1 -- 172.21.15.19:0/868951300 wait complete. 2024-04-25T10:02:23.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.742+0000 7f5075ce1640 1 Processor -- start 2024-04-25T10:02:23.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.742+0000 7f5075ce1640 1 -- start start 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.743+0000 7f5075ce1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50580043a0 0x7f50701f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.743+0000 7f5075ce1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5070153170 0x7f50701f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.743+0000 7f5075ce1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5070152b90 0x7f50700cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.743+0000 7f5075ce1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700b92a0 con 0x7f5070152b90 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.743+0000 7f5075ce1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700d2210 con 0x7f50580043a0 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.743+0000 7f5075ce1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50700b9420 con 0x7f5070153170 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.743+0000 7f506ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5070152b90 0x7f50700cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.743+0000 7f506ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5070152b90 0x7f50700cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53904/0 (socket says 172.21.15.19:53904) 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.743+0000 7f506ffff640 1 -- 172.21.15.19:0/494493259 learned_addr learned my addr 172.21.15.19:0/494493259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.743+0000 7f506f7fe640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50580043a0 0x7f50701f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.743+0000 7f506effd640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5070153170 0x7f50701f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f506ffff640 1 -- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5070153170 msgr2=0x7f50701f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f506ffff640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5070153170 0x7f50701f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f506ffff640 1 -- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50580043a0 msgr2=0x7f50701f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f506ffff640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50580043a0 0x7f50701f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f506ffff640 1 -- 172.21.15.19:0/494493259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700cd430 con 0x7f5070152b90 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f506effd640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5070153170 0x7f50701f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f506f7fe640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50580043a0 0x7f50701f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f506ffff640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5070152b90 0x7f50700cc760 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f50700b9ac0 tx=0x7f506800f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f5057fff640 1 -- 172.21.15.19:0/494493259 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068015630 con 0x7f5070152b90 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f5057fff640 1 -- 172.21.15.19:0/494493259 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506801d070 con 0x7f5070152b90 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f5057fff640 1 -- 172.21.15.19:0/494493259 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068015a10 con 0x7f5070152b90 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f5075ce1640 1 -- 172.21.15.19:0/494493259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50700cbe40 con 0x7f5070152b90 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.744+0000 7f5075ce1640 1 -- 172.21.15.19:0/494493259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50700cd9b0 con 0x7f5070152b90 2024-04-25T10:02:23.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.745+0000 7f5075ce1640 1 -- 172.21.15.19:0/494493259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5070073990 con 0x7f5070152b90 2024-04-25T10:02:23.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.746+0000 7f5057fff640 1 -- 172.21.15.19:0/494493259 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5068015bb0 con 0x7f5070152b90 2024-04-25T10:02:23.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.746+0000 7f5057fff640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f505c06b500 0x7f505c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.746+0000 7f5057fff640 1 -- 172.21.15.19:0/494493259 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f506808add0 con 0x7f5070152b90 2024-04-25T10:02:23.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.746+0000 7f506f7fe640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f505c06b500 0x7f505c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.747+0000 7f506f7fe640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f505c06b500 0x7f505c06d9c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5060001630 tx=0x7f5060009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.750+0000 7f5057fff640 1 -- 172.21.15.19:0/494493259 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5068010040 con 0x7f5070152b90 2024-04-25T10:02:23.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.930+0000 7f5075ce1640 1 -- 172.21.15.19:0/494493259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5070068350 con 0x7f5070152b90 2024-04-25T10:02:23.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.931+0000 7f5057fff640 1 -- 172.21.15.19:0/494493259 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+17015 (secure 0 0 0) 0x7f5068059b00 con 0x7f5070152b90 2024-04-25T10:02:23.939 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:23.939 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":130,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:12.609472+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:27.270153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:27.977083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/250977615":"2024-04-26T10:02:12.609390+0000","172.21.15.19:6833/250977615":"2024-04-26T10:02:12.609390+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 -- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f505c06b500 msgr2=0x7f505c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f505c06b500 0x7f505c06d9c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5060001630 tx=0x7f5060009040 comp rx=0 tx=0).stop 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 -- 172.21.15.19:0/494493259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5070152b90 msgr2=0x7f50700cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5070152b90 0x7f50700cc760 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f50700b9ac0 tx=0x7f506800f870 comp rx=0 tx=0).stop 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 -- 172.21.15.19:0/494493259 shutdown_connections 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f505c06b500 0x7f505c06d9c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5070152b90 0x7f50700cc760 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5070153170 0x7f50701f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 --2- 172.21.15.19:0/494493259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50580043a0 0x7f50701f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 -- 172.21.15.19:0/494493259 >> 172.21.15.19:0/494493259 conn(0x7f5070061a20 msgr2=0x7f50700b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 -- 172.21.15.19:0/494493259 shutdown_connections 2024-04-25T10:02:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:23.933+0000 7f5075ce1640 1 -- 172.21.15.19:0/494493259 wait complete. 2024-04-25T10:02:23.950 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:02:23.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.046+0000 7f3fe7561640 1 Processor -- start 2024-04-25T10:02:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.046+0000 7f3fe7561640 1 -- start start 2024-04-25T10:02:24.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe7561640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00b6cb0 0x7f3fe00b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe7561640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3fe00b5a60 0x7f3fe00d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe7561640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00ceee0 con 0x7f3fe015db70 2024-04-25T10:02:24.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe7561640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe00d1d10 con 0x7f3fe00b5a60 2024-04-25T10:02:24.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe7561640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00d1ff0 con 0x7f3fe00b6cb0 2024-04-25T10:02:24.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe5afe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3fe00b5a60 0x7f3fe00d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe5afe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3fe00b5a60 0x7f3fe00d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53920/0 (socket says 172.21.15.19:53920) 2024-04-25T10:02:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe5afe640 1 -- 172.21.15.19:0/2930236285 learned_addr learned my addr 172.21.15.19:0/2930236285 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe4afc640 1 --2- 172.21.15.19:0/2930236285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00b6cb0 0x7f3fe00b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe5afe640 1 -- 172.21.15.19:0/2930236285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00b6cb0 msgr2=0x7f3fe00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe5afe640 1 --2- 172.21.15.19:0/2930236285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00b6cb0 0x7f3fe00b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.047+0000 7f3fe5afe640 1 -- 172.21.15.19:0/2930236285 >> v1:172.21.15.112:6789/0 conn(0x7f3fe015db70 legacy=0x7f3fe0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.048+0000 7f3fe5afe640 1 -- 172.21.15.19:0/2930236285 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00d22d0 con 0x7f3fe00b5a60 2024-04-25T10:02:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.048+0000 7f3fe5afe640 1 --2- 172.21.15.19:0/2930236285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe00b5a60 0x7f3fe00d27f0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009e50 tx=0x7f3fd802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14b447fe3d2621d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.048+0000 7f3fd7fff640 1 -- 172.21.15.19:0/2930236285 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8036070 con 0x7f3fe00b5a60 2024-04-25T10:02:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.048+0000 7f3fd7fff640 1 -- 172.21.15.19:0/2930236285 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd8031600 con 0x7f3fe00b5a60 2024-04-25T10:02:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.048+0000 7f3fd7fff640 1 -- 172.21.15.19:0/2930236285 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80319d0 con 0x7f3fe00b5a60 2024-04-25T10:02:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.049+0000 7f3fe7561640 1 -- 172.21.15.19:0/2930236285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe00b5a60 msgr2=0x7f3fe00d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.049+0000 7f3fe7561640 1 --2- 172.21.15.19:0/2930236285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe00b5a60 0x7f3fe00d27f0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009e50 tx=0x7f3fd802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.049+0000 7f3fe7561640 1 -- 172.21.15.19:0/2930236285 shutdown_connections 2024-04-25T10:02:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.049+0000 7f3fe7561640 1 --2- 172.21.15.19:0/2930236285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe00b5a60 0x7f3fe00d27f0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.049+0000 7f3fe7561640 1 --2- 172.21.15.19:0/2930236285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00b6cb0 0x7f3fe00b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.049+0000 7f3fe7561640 1 -- 172.21.15.19:0/2930236285 >> 172.21.15.19:0/2930236285 conn(0x7f3fe00d0520 msgr2=0x7f3fe00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.049+0000 7f3fe7561640 1 -- 172.21.15.19:0/2930236285 shutdown_connections 2024-04-25T10:02:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.049+0000 7f3fe7561640 1 -- 172.21.15.19:0/2930236285 wait complete. 2024-04-25T10:02:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.050+0000 7f3fe7561640 1 Processor -- start 2024-04-25T10:02:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.050+0000 7f3fe7561640 1 -- start start 2024-04-25T10:02:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe7561640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe00b5a60 0x7f3fe00c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe7561640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00b6cb0 0x7f3fe00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe7561640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe015db70 0x7f3fe01489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe7561640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00cf1c0 con 0x7f3fe00b5a60 2024-04-25T10:02:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe7561640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00ce530 con 0x7f3fe015db70 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe7561640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00cf340 con 0x7f3fe00b6cb0 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe52fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe00b5a60 0x7f3fe00c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe52fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe00b5a60 0x7f3fe00c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53928/0 (socket says 172.21.15.19:53928) 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe52fd640 1 -- 172.21.15.19:0/422357161 learned_addr learned my addr 172.21.15.19:0/422357161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe4afc640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00b6cb0 0x7f3fe00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe52fd640 1 -- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00b6cb0 msgr2=0x7f3fe00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe5afe640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe015db70 0x7f3fe01489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe52fd640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00b6cb0 0x7f3fe00cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe52fd640 1 -- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe015db70 msgr2=0x7f3fe01489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe52fd640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe015db70 0x7f3fe01489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.051+0000 7f3fe52fd640 1 -- 172.21.15.19:0/422357161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00cd360 con 0x7f3fe00b5a60 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.052+0000 7f3fe52fd640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe00b5a60 0x7f3fe00c6f40 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c430 tx=0x7f3fdc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.052+0000 7f3fd5ffb640 1 -- 172.21.15.19:0/422357161 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc011790 con 0x7f3fe00b5a60 2024-04-25T10:02:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.052+0000 7f3fd5ffb640 1 -- 172.21.15.19:0/422357161 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc017070 con 0x7f3fe00b5a60 2024-04-25T10:02:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.052+0000 7f3fe7561640 1 -- 172.21.15.19:0/422357161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe00cd640 con 0x7f3fe00b5a60 2024-04-25T10:02:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.052+0000 7f3fd5ffb640 1 -- 172.21.15.19:0/422357161 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc011bb0 con 0x7f3fe00b5a60 2024-04-25T10:02:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.052+0000 7f3fe7561640 1 -- 172.21.15.19:0/422357161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe00cdc00 con 0x7f3fe00b5a60 2024-04-25T10:02:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.053+0000 7f3fe7561640 1 -- 172.21.15.19:0/422357161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe0073990 con 0x7f3fe00b5a60 2024-04-25T10:02:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.053+0000 7f3fd5ffb640 1 -- 172.21.15.19:0/422357161 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fdc01c460 con 0x7f3fe00b5a60 2024-04-25T10:02:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.054+0000 7f3fd5ffb640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3fc406b4d0 0x7f3fc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.054+0000 7f3fd5ffb640 1 -- 172.21.15.19:0/422357161 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3fdc08a210 con 0x7f3fe00b5a60 2024-04-25T10:02:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.054+0000 7f3fe4afc640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3fc406b4d0 0x7f3fc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.054+0000 7f3fe4afc640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3fc406b4d0 0x7f3fc406d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0006fd0 tx=0x7f3fd0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.058+0000 7f3fd5ffb640 1 -- 172.21.15.19:0/422357161 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fdc058f40 con 0x7f3fe00b5a60 2024-04-25T10:02:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.277+0000 7f3fe7561640 1 -- 172.21.15.19:0/422357161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fe015c410 con 0x7f3fe00b5a60 2024-04-25T10:02:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.278+0000 7f3fd5ffb640 1 -- 172.21.15.19:0/422357161 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f3fdc058840 con 0x7f3fe00b5a60 2024-04-25T10:02:24.283 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:24.283 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:02:14.631627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9210":{"gid":9210,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":17,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-04-25T10:02:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 -- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3fc406b4d0 msgr2=0x7f3fc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3fc406b4d0 0x7f3fc406d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0006fd0 tx=0x7f3fd0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 -- 172.21.15.19:0/422357161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe00b5a60 msgr2=0x7f3fe00c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe00b5a60 0x7f3fe00c6f40 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c430 tx=0x7f3fdc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 -- 172.21.15.19:0/422357161 shutdown_connections 2024-04-25T10:02:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3fc406b4d0 0x7f3fc406d990 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe015db70 0x7f3fe01489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe00b6cb0 0x7f3fe00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 --2- 172.21.15.19:0/422357161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fe00b5a60 0x7f3fe00c6f40 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 -- 172.21.15.19:0/422357161 >> 172.21.15.19:0/422357161 conn(0x7f3fe00d0520 msgr2=0x7f3fe0156fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 -- 172.21.15.19:0/422357161 shutdown_connections 2024-04-25T10:02:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.280+0000 7f3fe7561640 1 -- 172.21.15.19:0/422357161 wait complete. 2024-04-25T10:02:24.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:24.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.392+0000 7f4a2e102640 1 Processor -- start 2024-04-25T10:02:24.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.392+0000 7f4a2e102640 1 -- start start 2024-04-25T10:02:24.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.393+0000 7f4a2e102640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4a2815dd20 0x7f4a280b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.393+0000 7f4a2e102640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4a280d11b0 0x7f4a280d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.393+0000 7f4a2e102640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2815eac0 con 0x7f4a2815e100 2024-04-25T10:02:24.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.393+0000 7f4a2e102640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a280d1c00 con 0x7f4a2815dd20 2024-04-25T10:02:24.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.393+0000 7f4a2e102640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2815eda0 con 0x7f4a280d11b0 2024-04-25T10:02:24.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.393+0000 7f4a277fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4a2815dd20 0x7f4a280b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.393+0000 7f4a277fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4a2815dd20 0x7f4a280b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53944/0 (socket says 172.21.15.19:53944) 2024-04-25T10:02:24.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.393+0000 7f4a277fe640 1 -- 172.21.15.19:0/2622139511 learned_addr learned my addr 172.21.15.19:0/2622139511 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:24.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.394+0000 7f4a27fff640 1 --2- 172.21.15.19:0/2622139511 >> v2:172.21.15.112:3300/0 conn(0x7f4a280d11b0 0x7f4a280d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.394+0000 7f4a277fe640 1 -- 172.21.15.19:0/2622139511 >> v2:172.21.15.112:3300/0 conn(0x7f4a280d11b0 msgr2=0x7f4a280d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.394+0000 7f4a277fe640 1 --2- 172.21.15.19:0/2622139511 >> v2:172.21.15.112:3300/0 conn(0x7f4a280d11b0 0x7f4a280d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.394+0000 7f4a277fe640 1 -- 172.21.15.19:0/2622139511 >> v1:172.21.15.112:6789/0 conn(0x7f4a2815e100 legacy=0x7f4a280d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.394+0000 7f4a277fe640 1 -- 172.21.15.19:0/2622139511 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2815f080 con 0x7f4a2815dd20 2024-04-25T10:02:24.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.394+0000 7f4a277fe640 1 --2- 172.21.15.19:0/2622139511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2815dd20 0x7f4a280b96d0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f4a280d1990 tx=0x7f4a1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2496fbc0d678eff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.394+0000 7f4a267fc640 1 -- 172.21.15.19:0/2622139511 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18037070 con 0x7f4a2815dd20 2024-04-25T10:02:24.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.395+0000 7f4a267fc640 1 -- 172.21.15.19:0/2622139511 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a18032440 con 0x7f4a2815dd20 2024-04-25T10:02:24.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.395+0000 7f4a267fc640 1 -- 172.21.15.19:0/2622139511 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18032810 con 0x7f4a2815dd20 2024-04-25T10:02:24.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.395+0000 7f4a2e102640 1 -- 172.21.15.19:0/2622139511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2815dd20 msgr2=0x7f4a280b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.395+0000 7f4a2e102640 1 --2- 172.21.15.19:0/2622139511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2815dd20 0x7f4a280b96d0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f4a280d1990 tx=0x7f4a1802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.395+0000 7f4a2e102640 1 -- 172.21.15.19:0/2622139511 shutdown_connections 2024-04-25T10:02:24.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.395+0000 7f4a2e102640 1 --2- 172.21.15.19:0/2622139511 >> v2:172.21.15.112:3300/0 conn(0x7f4a280d11b0 0x7f4a280d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.395+0000 7f4a2e102640 1 --2- 172.21.15.19:0/2622139511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2815dd20 0x7f4a280b96d0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.395+0000 7f4a2e102640 1 -- 172.21.15.19:0/2622139511 >> 172.21.15.19:0/2622139511 conn(0x7f4a28061a20 msgr2=0x7f4a28061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.395+0000 7f4a2e102640 1 -- 172.21.15.19:0/2622139511 shutdown_connections 2024-04-25T10:02:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.395+0000 7f4a2e102640 1 -- 172.21.15.19:0/2622139511 wait complete. 2024-04-25T10:02:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.396+0000 7f4a2e102640 1 Processor -- start 2024-04-25T10:02:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.396+0000 7f4a2e102640 1 -- start start 2024-04-25T10:02:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.397+0000 7f4a2e102640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a280d11b0 0x7f4a280ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.397+0000 7f4a2e102640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2815dd20 0x7f4a280b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.397+0000 7f4a2e102640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a2815e100 0x7f4a280b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.397+0000 7f4a2e102640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2815e710 con 0x7f4a2815dd20 2024-04-25T10:02:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.397+0000 7f4a2e102640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2815e590 con 0x7f4a2815e100 2024-04-25T10:02:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.397+0000 7f4a2e102640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a281f7080 con 0x7f4a280d11b0 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.397+0000 7f4a26ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2815dd20 0x7f4a280b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.397+0000 7f4a26ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2815dd20 0x7f4a280b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53960/0 (socket says 172.21.15.19:53960) 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.397+0000 7f4a26ffd640 1 -- 172.21.15.19:0/1632612899 learned_addr learned my addr 172.21.15.19:0/1632612899 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a26ffd640 1 -- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a280d11b0 msgr2=0x7f4a280ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a26ffd640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a280d11b0 0x7f4a280ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a26ffd640 1 -- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a2815e100 msgr2=0x7f4a280b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a27fff640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a2815e100 0x7f4a280b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a26ffd640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a2815e100 0x7f4a280b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a26ffd640 1 -- 172.21.15.19:0/1632612899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28159580 con 0x7f4a2815dd20 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a26ffd640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2815dd20 0x7f4a280b2350 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00c400 tx=0x7f4a1c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a0ffff640 1 -- 172.21.15.19:0/1632612899 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c017070 con 0x7f4a2815dd20 2024-04-25T10:02:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a2e102640 1 -- 172.21.15.19:0/1632612899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a28159860 con 0x7f4a2815dd20 2024-04-25T10:02:24.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a0ffff640 1 -- 172.21.15.19:0/1632612899 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c0052d0 con 0x7f4a2815dd20 2024-04-25T10:02:24.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a0ffff640 1 -- 172.21.15.19:0/1632612899 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c003830 con 0x7f4a2815dd20 2024-04-25T10:02:24.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.398+0000 7f4a2e102640 1 -- 172.21.15.19:0/1632612899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a28159e20 con 0x7f4a2815dd20 2024-04-25T10:02:24.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.399+0000 7f4a2e102640 1 -- 172.21.15.19:0/1632612899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a28073990 con 0x7f4a2815dd20 2024-04-25T10:02:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.399+0000 7f4a0ffff640 1 -- 172.21.15.19:0/1632612899 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a1c01a070 con 0x7f4a2815dd20 2024-04-25T10:02:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.400+0000 7f4a0ffff640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a1406b4d0 0x7f4a1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.400+0000 7f4a0ffff640 1 -- 172.21.15.19:0/1632612899 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4a1c0897c0 con 0x7f4a2815dd20 2024-04-25T10:02:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.400+0000 7f4a277fe640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a1406b4d0 0x7f4a1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.401+0000 7f4a277fe640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a1406b4d0 0x7f4a1406d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4a18034000 tx=0x7f4a1802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.404+0000 7f4a0ffff640 1 -- 172.21.15.19:0/1632612899 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a1c0583e0 con 0x7f4a2815dd20 2024-04-25T10:02:24.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.622+0000 7f4a2e102640 1 -- 172.21.15.19:0/1632612899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a280b5570 con 0x7f4a2815dd20 2024-04-25T10:02:24.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.623+0000 7f4a0ffff640 1 -- 172.21.15.19:0/1632612899 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f4a1c057ce0 con 0x7f4a2815dd20 2024-04-25T10:02:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:24.628 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:29.000131+0000","modified":"2024-04-25T10:02:14.631627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9210":{"gid":9210,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":17,"addr":"172.21.15.19:6834/1696858180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1696858180},{"type":"v1","addr":"172.21.15.19:6834","nonce":1696858180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:24.628 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-04-25T10:02:24.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 -- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a1406b4d0 msgr2=0x7f4a1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a1406b4d0 0x7f4a1406d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4a18034000 tx=0x7f4a1802c000 comp rx=0 tx=0).stop 2024-04-25T10:02:24.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 -- 172.21.15.19:0/1632612899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2815dd20 msgr2=0x7f4a280b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2815dd20 0x7f4a280b2350 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00c400 tx=0x7f4a1c00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 -- 172.21.15.19:0/1632612899 shutdown_connections 2024-04-25T10:02:24.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4a1406b4d0 0x7f4a1406d990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a2815e100 0x7f4a280b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2815dd20 0x7f4a280b2350 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 --2- 172.21.15.19:0/1632612899 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a280d11b0 0x7f4a280ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 -- 172.21.15.19:0/1632612899 >> 172.21.15.19:0/1632612899 conn(0x7f4a28061a20 msgr2=0x7f4a280d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 -- 172.21.15.19:0/1632612899 shutdown_connections 2024-04-25T10:02:24.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.625+0000 7f4a2e102640 1 -- 172.21.15.19:0/1632612899 wait complete. 2024-04-25T10:02:24.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.738+0000 7fd251a09640 1 Processor -- start 2024-04-25T10:02:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.738+0000 7fd251a09640 1 -- start start 2024-04-25T10:02:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd251a09640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd24c0d11b0 0x7fd24c168880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd251a09640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c15e810 con 0x7fd24c0b96d0 2024-04-25T10:02:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd251a09640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c15eac0 con 0x7fd24c15dd20 2024-04-25T10:02:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd251a09640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c0d1c00 con 0x7fd24c0d11b0 2024-04-25T10:02:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd24a7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd24c0b96d0 0x7fd24c0d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50382/0 (socket says 172.21.15.19:50382) 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd24a7fc640 1 -- 172.21.15.19:0/251364049 learned_addr learned my addr 172.21.15.19:0/251364049 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536682402 0 0) 0x7fd24c15e810 con 0x7fd24c0b96d0 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd24b7fe640 1 --2- 172.21.15.19:0/251364049 >> v2:172.21.15.112:3300/0 conn(0x7fd24c0d11b0 0x7fd24c168880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd238003520 con 0x7fd24c0b96d0 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1321643600 0 0) 0x7fd238003520 con 0x7fd24c0b96d0 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 >> v2:172.21.15.112:3300/0 conn(0x7fd24c0d11b0 msgr2=0x7fd24c168880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd249ffb640 1 --2- 172.21.15.19:0/251364049 >> v2:172.21.15.112:3300/0 conn(0x7fd24c0d11b0 0x7fd24c168880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 >> v1:172.21.15.112:6789/0 conn(0x7fd24c15dd20 legacy=0x7fd24c15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c15eda0 con 0x7fd24c0b96d0 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.739+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fd234001670 con 0x7fd24c0b96d0 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 >> v1:172.21.15.19:6789/0 conn(0x7fd24c0b96d0 legacy=0x7fd24c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd249ffb640 1 --2- 172.21.15.19:0/251364049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2380039b0 0x7fd238003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd249ffb640 1 --2- 172.21.15.19:0/251364049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd238004800 0x7fd238004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd249ffb640 1 --2- 172.21.15.19:0/251364049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2380051c0 0x7fd23800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0d1c00 con 0x7fd2380039b0 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23800c160 con 0x7fd2380051c0 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23800c360 con 0x7fd238004800 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd24affd640 1 --2- 172.21.15.19:0/251364049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2380039b0 0x7fd238003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd24b7fe640 1 --2- 172.21.15.19:0/251364049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2380051c0 0x7fd23800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd24a7fc640 1 --2- 172.21.15.19:0/251364049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd238004800 0x7fd238004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd24affd640 1 -- 172.21.15.19:0/251364049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd238004800 msgr2=0x7fd238004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd24affd640 1 --2- 172.21.15.19:0/251364049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd238004800 0x7fd238004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd24affd640 1 -- 172.21.15.19:0/251364049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2380051c0 msgr2=0x7fd23800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd24affd640 1 --2- 172.21.15.19:0/251364049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2380051c0 0x7fd23800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd24affd640 1 -- 172.21.15.19:0/251364049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd238003520 con 0x7fd2380039b0 2024-04-25T10:02:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.740+0000 7fd24b7fe640 1 -- 172.21.15.19:0/251364049 reap_dead start 2024-04-25T10:02:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.741+0000 7fd24affd640 1 --2- 172.21.15.19:0/251364049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2380039b0 0x7fd238003dd0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fd244001750 tx=0x7fd238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1eac9b2934f91d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.741+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244001370 con 0x7fd2380039b0 2024-04-25T10:02:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.741+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2440052d0 con 0x7fd2380039b0 2024-04-25T10:02:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.741+0000 7fd249ffb640 1 -- 172.21.15.19:0/251364049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24402abc0 con 0x7fd2380039b0 2024-04-25T10:02:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.741+0000 7fd251a09640 1 -- 172.21.15.19:0/251364049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2380039b0 msgr2=0x7fd238003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.741+0000 7fd251a09640 1 --2- 172.21.15.19:0/251364049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2380039b0 0x7fd238003dd0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fd244001750 tx=0x7fd238005830 comp rx=0 tx=0).stop 2024-04-25T10:02:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.741+0000 7fd251a09640 1 -- 172.21.15.19:0/251364049 shutdown_connections 2024-04-25T10:02:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.741+0000 7fd251a09640 1 --2- 172.21.15.19:0/251364049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2380039b0 0x7fd238003dd0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.741+0000 7fd251a09640 1 -- 172.21.15.19:0/251364049 >> 172.21.15.19:0/251364049 conn(0x7fd24c061a20 msgr2=0x7fd24c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.742+0000 7fd251a09640 1 -- 172.21.15.19:0/251364049 shutdown_connections 2024-04-25T10:02:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.742+0000 7fd251a09640 1 -- 172.21.15.19:0/251364049 wait complete. 2024-04-25T10:02:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.743+0000 7fd251a09640 1 Processor -- start 2024-04-25T10:02:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.743+0000 7fd251a09640 1 -- start start 2024-04-25T10:02:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd251a09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2380039b0 0x7fd24c0cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd251a09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd24c15b250 0x7fd24c151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd251a09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd24c0ccea0 0x7fd24c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd251a09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0d0aa0 con 0x7fd24c15b250 2024-04-25T10:02:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd251a09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0d1e50 con 0x7fd2380039b0 2024-04-25T10:02:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd251a09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0d0c20 con 0x7fd24c0ccea0 2024-04-25T10:02:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd24c15b250 0x7fd24c151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd24c15b250 0x7fd24c151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53972/0 (socket says 172.21.15.19:53972) 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24a7fc640 1 -- 172.21.15.19:0/3600986266 learned_addr learned my addr 172.21.15.19:0/3600986266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24b7fe640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd24c0ccea0 0x7fd24c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24affd640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2380039b0 0x7fd24c0cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24a7fc640 1 -- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd24c0ccea0 msgr2=0x7fd24c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24a7fc640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd24c0ccea0 0x7fd24c0cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24a7fc640 1 -- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2380039b0 msgr2=0x7fd24c0cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24a7fc640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2380039b0 0x7fd24c0cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24a7fc640 1 -- 172.21.15.19:0/3600986266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0caf50 con 0x7fd24c15b250 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24affd640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2380039b0 0x7fd24c0cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd24a7fc640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd24c15b250 0x7fd24c151f20 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fd234007370 tx=0x7fd234007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd243fff640 1 -- 172.21.15.19:0/3600986266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23401a070 con 0x7fd24c15b250 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd243fff640 1 -- 172.21.15.19:0/3600986266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23400f940 con 0x7fd24c15b250 2024-04-25T10:02:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd251a09640 1 -- 172.21.15.19:0/3600986266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c0cb230 con 0x7fd24c15b250 2024-04-25T10:02:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.744+0000 7fd243fff640 1 -- 172.21.15.19:0/3600986266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400fd30 con 0x7fd24c15b250 2024-04-25T10:02:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.745+0000 7fd251a09640 1 -- 172.21.15.19:0/3600986266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c0cb7f0 con 0x7fd24c15b250 2024-04-25T10:02:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.745+0000 7fd251a09640 1 -- 172.21.15.19:0/3600986266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24c073990 con 0x7fd24c15b250 2024-04-25T10:02:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.746+0000 7fd243fff640 1 -- 172.21.15.19:0/3600986266 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd23401d070 con 0x7fd24c15b250 2024-04-25T10:02:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.746+0000 7fd243fff640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd22c06b4d0 0x7fd22c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.746+0000 7fd243fff640 1 -- 172.21.15.19:0/3600986266 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd234089e00 con 0x7fd24c15b250 2024-04-25T10:02:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.746+0000 7fd24affd640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd22c06b4d0 0x7fd22c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.747+0000 7fd24affd640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd22c06b4d0 0x7fd22c06d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd244032000 tx=0x7fd24402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.750+0000 7fd243fff640 1 -- 172.21.15.19:0/3600986266 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd234058a10 con 0x7fd24c15b250 2024-04-25T10:02:24.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.930+0000 7fd251a09640 1 -- 172.21.15.19:0/3600986266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd24c0b4b20 con 0x7fd24c15b250 2024-04-25T10:02:24.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.931+0000 7fd243fff640 1 -- 172.21.15.19:0/3600986266 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+17015 (secure 0 0 0) 0x7fd234058310 con 0x7fd24c15b250 2024-04-25T10:02:24.936 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:24.936 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":130,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:12.609472+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:27.270153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:27.977083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/250977615":"2024-04-26T10:02:12.609390+0000","172.21.15.19:6833/250977615":"2024-04-26T10:02:12.609390+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 -- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd22c06b4d0 msgr2=0x7fd22c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd22c06b4d0 0x7fd22c06d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd244032000 tx=0x7fd24402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 -- 172.21.15.19:0/3600986266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd24c15b250 msgr2=0x7fd24c151f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd24c15b250 0x7fd24c151f20 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fd234007370 tx=0x7fd234007830 comp rx=0 tx=0).stop 2024-04-25T10:02:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 -- 172.21.15.19:0/3600986266 shutdown_connections 2024-04-25T10:02:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd22c06b4d0 0x7fd22c06d990 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd24c0ccea0 0x7fd24c0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd24c15b250 0x7fd24c151f20 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 --2- 172.21.15.19:0/3600986266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2380039b0 0x7fd24c0cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 -- 172.21.15.19:0/3600986266 >> 172.21.15.19:0/3600986266 conn(0x7fd24c061a20 msgr2=0x7fd24c15fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 -- 172.21.15.19:0/3600986266 shutdown_connections 2024-04-25T10:02:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:24.933+0000 7fd251a09640 1 -- 172.21.15.19:0/3600986266 wait complete. 2024-04-25T10:02:24.949 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:02:25.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.046+0000 7fd863fc8640 1 Processor -- start 2024-04-25T10:02:25.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.046+0000 7fd863fc8640 1 -- start start 2024-04-25T10:02:25.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.047+0000 7fd863fc8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd85c15dc20 0x7fd85c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.047+0000 7fd863fc8640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd85c161710 0x7fd85c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.047+0000 7fd863fc8640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c15e000 con 0x7fd85c0b9330 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.047+0000 7fd863fc8640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c0d1de0 con 0x7fd85c161710 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.047+0000 7fd863fc8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c161c60 con 0x7fd85c15dc20 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.047+0000 7fd861563640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd85c161710 0x7fd85c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.047+0000 7fd862565640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd85c0b9330 0x7fd85c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50398/0 (socket says 172.21.15.19:50398) 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.047+0000 7fd862565640 1 -- 172.21.15.19:0/2110294465 learned_addr learned my addr 172.21.15.19:0/2110294465 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.048+0000 7fd861d64640 1 --2- 172.21.15.19:0/2110294465 >> v2:172.21.15.112:3300/0 conn(0x7fd85c15dc20 0x7fd85c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.048+0000 7fd861563640 1 -- 172.21.15.19:0/2110294465 >> v2:172.21.15.112:3300/0 conn(0x7fd85c15dc20 msgr2=0x7fd85c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.048+0000 7fd861563640 1 --2- 172.21.15.19:0/2110294465 >> v2:172.21.15.112:3300/0 conn(0x7fd85c15dc20 0x7fd85c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.048+0000 7fd861563640 1 -- 172.21.15.19:0/2110294465 >> v1:172.21.15.19:6789/0 conn(0x7fd85c0b9330 legacy=0x7fd85c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.048+0000 7fd861563640 1 -- 172.21.15.19:0/2110294465 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c161f40 con 0x7fd85c161710 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.048+0000 7fd860d62640 1 -- 172.21.15.19:0/2110294465 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888974614 0 0) 0x7fd85c15e000 con 0x7fd85c0b9330 2024-04-25T10:02:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.048+0000 7fd861563640 1 --2- 172.21.15.19:0/2110294465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85c161710 0x7fd85c0cebb0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0d1520 tx=0x7fd85002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9457b77fc46d382 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.049+0000 7fd860d62640 1 -- 172.21.15.19:0/2110294465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850037070 con 0x7fd85c161710 2024-04-25T10:02:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.049+0000 7fd860d62640 1 -- 172.21.15.19:0/2110294465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd850032440 con 0x7fd85c161710 2024-04-25T10:02:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.049+0000 7fd860d62640 1 -- 172.21.15.19:0/2110294465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850032810 con 0x7fd85c161710 2024-04-25T10:02:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.049+0000 7fd863fc8640 1 -- 172.21.15.19:0/2110294465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85c161710 msgr2=0x7fd85c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.049+0000 7fd863fc8640 1 --2- 172.21.15.19:0/2110294465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85c161710 0x7fd85c0cebb0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0d1520 tx=0x7fd85002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.049+0000 7fd863fc8640 1 -- 172.21.15.19:0/2110294465 shutdown_connections 2024-04-25T10:02:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.049+0000 7fd863fc8640 1 --2- 172.21.15.19:0/2110294465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85c161710 0x7fd85c0cebb0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.049+0000 7fd863fc8640 1 --2- 172.21.15.19:0/2110294465 >> v2:172.21.15.112:3300/0 conn(0x7fd85c15dc20 0x7fd85c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.049+0000 7fd863fc8640 1 -- 172.21.15.19:0/2110294465 >> 172.21.15.19:0/2110294465 conn(0x7fd85c0d0420 msgr2=0x7fd85c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.049+0000 7fd863fc8640 1 -- 172.21.15.19:0/2110294465 shutdown_connections 2024-04-25T10:02:25.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.049+0000 7fd863fc8640 1 -- 172.21.15.19:0/2110294465 wait complete. 2024-04-25T10:02:25.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.050+0000 7fd863fc8640 1 Processor -- start 2024-04-25T10:02:25.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.050+0000 7fd863fc8640 1 -- start start 2024-04-25T10:02:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd863fc8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd85c0b9330 0x7fd85c1f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd863fc8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85c15dc20 0x7fd85c160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd863fc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd85c161710 0x7fd85c0b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd863fc8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0b5e50 con 0x7fd85c15dc20 2024-04-25T10:02:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd863fc8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c162500 con 0x7fd85c161710 2024-04-25T10:02:25.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd863fc8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0b5fd0 con 0x7fd85c0b9330 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd861563640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85c15dc20 0x7fd85c160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd861563640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85c15dc20 0x7fd85c160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53996/0 (socket says 172.21.15.19:53996) 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd861563640 1 -- 172.21.15.19:0/3428816387 learned_addr learned my addr 172.21.15.19:0/3428816387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd862565640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd85c161710 0x7fd85c0b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd861d64640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd85c0b9330 0x7fd85c1f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd861563640 1 -- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd85c0b9330 msgr2=0x7fd85c1f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd861563640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd85c0b9330 0x7fd85c1f4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.051+0000 7fd861563640 1 -- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd85c161710 msgr2=0x7fd85c0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.052+0000 7fd861563640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd85c161710 0x7fd85c0b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.052+0000 7fd861563640 1 -- 172.21.15.19:0/3428816387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c15fba0 con 0x7fd85c15dc20 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.052+0000 7fd862565640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd85c161710 0x7fd85c0b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.052+0000 7fd861563640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85c15dc20 0x7fd85c160940 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fd850000c00 tx=0x7fd8500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.052+0000 7fd85a7fc640 1 -- 172.21.15.19:0/3428816387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850037040 con 0x7fd85c15dc20 2024-04-25T10:02:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.052+0000 7fd85a7fc640 1 -- 172.21.15.19:0/3428816387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd850042070 con 0x7fd85c15dc20 2024-04-25T10:02:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.052+0000 7fd863fc8640 1 -- 172.21.15.19:0/3428816387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c15fe80 con 0x7fd85c15dc20 2024-04-25T10:02:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.052+0000 7fd85a7fc640 1 -- 172.21.15.19:0/3428816387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850030bd0 con 0x7fd85c15dc20 2024-04-25T10:02:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.052+0000 7fd863fc8640 1 -- 172.21.15.19:0/3428816387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c160440 con 0x7fd85c15dc20 2024-04-25T10:02:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.053+0000 7fd863fc8640 1 -- 172.21.15.19:0/3428816387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c073a00 con 0x7fd85c15dc20 2024-04-25T10:02:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.053+0000 7fd85a7fc640 1 -- 172.21.15.19:0/3428816387 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd850047460 con 0x7fd85c15dc20 2024-04-25T10:02:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.054+0000 7fd85a7fc640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd84806b4d0 0x7fd84806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.054+0000 7fd85a7fc640 1 -- 172.21.15.19:0/3428816387 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd8500aaf30 con 0x7fd85c15dc20 2024-04-25T10:02:25.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.054+0000 7fd861d64640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd84806b4d0 0x7fd84806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.054+0000 7fd861d64640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd84806b4d0 0x7fd84806d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd85c15fa00 tx=0x7fd84c0039b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.058+0000 7fd85a7fc640 1 -- 172.21.15.19:0/3428816387 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd850032e20 con 0x7fd85c15dc20 2024-04-25T10:02:25.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.274+0000 7fd863fc8640 1 -- 172.21.15.19:0/3428816387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd85c0d16b0 con 0x7fd85c15dc20 2024-04-25T10:02:25.279 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T10:02:25.275+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:02:25.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.276+0000 7fd85a7fc640 1 -- 172.21.15.19:0/3428816387 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd850045360 con 0x7fd85c15dc20 2024-04-25T10:02:25.281 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:02:25.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.279+0000 7fd863fc8640 1 -- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd84806b4d0 msgr2=0x7fd84806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.279+0000 7fd863fc8640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd84806b4d0 0x7fd84806d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd85c15fa00 tx=0x7fd84c0039b0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.279+0000 7fd863fc8640 1 -- 172.21.15.19:0/3428816387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85c15dc20 msgr2=0x7fd85c160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.279+0000 7fd863fc8640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85c15dc20 0x7fd85c160940 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fd850000c00 tx=0x7fd8500046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.279+0000 7fd863fc8640 1 -- 172.21.15.19:0/3428816387 shutdown_connections 2024-04-25T10:02:25.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.279+0000 7fd863fc8640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd84806b4d0 0x7fd84806d990 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.279+0000 7fd863fc8640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd85c161710 0x7fd85c0b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.279+0000 7fd863fc8640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd85c15dc20 0x7fd85c160940 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.279+0000 7fd863fc8640 1 --2- 172.21.15.19:0/3428816387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd85c0b9330 0x7fd85c1f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.279+0000 7fd863fc8640 1 -- 172.21.15.19:0/3428816387 >> 172.21.15.19:0/3428816387 conn(0x7fd85c0d0420 msgr2=0x7fd85c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.280+0000 7fd863fc8640 1 -- 172.21.15.19:0/3428816387 shutdown_connections 2024-04-25T10:02:25.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.280+0000 7fd863fc8640 1 -- 172.21.15.19:0/3428816387 wait complete. 2024-04-25T10:02:25.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:02:25.321 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T10:02:25.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.396+0000 7f4acfd09640 1 Processor -- start 2024-04-25T10:02:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.396+0000 7f4acfd09640 1 -- start start 2024-04-25T10:02:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4acfd09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac815dc20 0x7f4ac80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4acfd09640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4ac80d32f0 0x7f4ac80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4acfd09640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac815e000 con 0x7f4ac8161710 2024-04-25T10:02:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4acfd09640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac80d1de0 con 0x7f4ac80d32f0 2024-04-25T10:02:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4acfd09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8161c60 con 0x7f4ac815dc20 2024-04-25T10:02:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4ace2a6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4ac80d32f0 0x7f4ac80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4ace2a6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4ac80d32f0 0x7f4ac80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54016/0 (socket says 172.21.15.19:54016) 2024-04-25T10:02:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4ace2a6640 1 -- 172.21.15.19:0/2109949648 learned_addr learned my addr 172.21.15.19:0/2109949648 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4acdaa5640 1 --2- 172.21.15.19:0/2109949648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac815dc20 0x7f4ac80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4ace2a6640 1 -- 172.21.15.19:0/2109949648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac815dc20 msgr2=0x7f4ac80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4ace2a6640 1 --2- 172.21.15.19:0/2109949648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac815dc20 0x7f4ac80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4ace2a6640 1 -- 172.21.15.19:0/2109949648 >> v1:172.21.15.112:6789/0 conn(0x7f4ac8161710 legacy=0x7f4ac806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.397+0000 7f4ace2a6640 1 -- 172.21.15.19:0/2109949648 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac8161f40 con 0x7f4ac80d32f0 2024-04-25T10:02:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.398+0000 7f4ace2a6640 1 --2- 172.21.15.19:0/2109949648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac80d32f0 0x7f4ac80cebb0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009e50 tx=0x7f4ac002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e383c1182ffb8cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.398+0000 7f4accaa3640 1 -- 172.21.15.19:0/2109949648 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0036070 con 0x7f4ac80d32f0 2024-04-25T10:02:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.398+0000 7f4accaa3640 1 -- 172.21.15.19:0/2109949648 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac0031600 con 0x7f4ac80d32f0 2024-04-25T10:02:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.398+0000 7f4accaa3640 1 -- 172.21.15.19:0/2109949648 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00319d0 con 0x7f4ac80d32f0 2024-04-25T10:02:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.399+0000 7f4acfd09640 1 -- 172.21.15.19:0/2109949648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac80d32f0 msgr2=0x7f4ac80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.399+0000 7f4acfd09640 1 --2- 172.21.15.19:0/2109949648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac80d32f0 0x7f4ac80cebb0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009e50 tx=0x7f4ac002a710 comp rx=0 tx=0).stop 2024-04-25T10:02:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.399+0000 7f4acfd09640 1 -- 172.21.15.19:0/2109949648 shutdown_connections 2024-04-25T10:02:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.399+0000 7f4acfd09640 1 --2- 172.21.15.19:0/2109949648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac80d32f0 0x7f4ac80cebb0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.399+0000 7f4acfd09640 1 --2- 172.21.15.19:0/2109949648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac815dc20 0x7f4ac80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.399+0000 7f4acfd09640 1 -- 172.21.15.19:0/2109949648 >> 172.21.15.19:0/2109949648 conn(0x7f4ac80d0420 msgr2=0x7f4ac80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.399+0000 7f4acfd09640 1 -- 172.21.15.19:0/2109949648 shutdown_connections 2024-04-25T10:02:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.399+0000 7f4acfd09640 1 -- 172.21.15.19:0/2109949648 wait complete. 2024-04-25T10:02:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.400+0000 7f4acfd09640 1 Processor -- start 2024-04-25T10:02:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.400+0000 7f4acfd09640 1 -- start start 2024-04-25T10:02:25.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acfd09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ac80d32f0 0x7f4ac814c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acfd09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac815dc20 0x7f4ac80acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acfd09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac8161710 0x7f4ac80ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acfd09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80b5e50 con 0x7f4ac815dc20 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acfd09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8162500 con 0x7f4ac80d32f0 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acfd09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80b5fd0 con 0x7f4ac8161710 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acd2a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac815dc20 0x7f4ac80acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acd2a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac815dc20 0x7f4ac80acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54032/0 (socket says 172.21.15.19:54032) 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acd2a4640 1 -- 172.21.15.19:0/77918704 learned_addr learned my addr 172.21.15.19:0/77918704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4ace2a6640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac8161710 0x7f4ac80ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acdaa5640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ac80d32f0 0x7f4ac814c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acd2a4640 1 -- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac8161710 msgr2=0x7f4ac80ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acd2a4640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac8161710 0x7f4ac80ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acd2a4640 1 -- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ac80d32f0 msgr2=0x7f4ac814c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acd2a4640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ac80d32f0 0x7f4ac814c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acd2a4640 1 -- 172.21.15.19:0/77918704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80cd350 con 0x7f4ac815dc20 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4acd2a4640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac815dc20 0x7f4ac80acf20 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800c3c0 tx=0x7f4ab800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.401+0000 7f4abe7fc640 1 -- 172.21.15.19:0/77918704 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80052d0 con 0x7f4ac815dc20 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.402+0000 7f4abe7fc640 1 -- 172.21.15.19:0/77918704 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab8005490 con 0x7f4ac815dc20 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.402+0000 7f4acfd09640 1 -- 172.21.15.19:0/77918704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac814cc30 con 0x7f4ac815dc20 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.402+0000 7f4abe7fc640 1 -- 172.21.15.19:0/77918704 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8007260 con 0x7f4ac815dc20 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.402+0000 7f4acfd09640 1 -- 172.21.15.19:0/77918704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac80cc5d0 con 0x7f4ac815dc20 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.402+0000 7f4acfd09640 1 -- 172.21.15.19:0/77918704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac814ca50 con 0x7f4ac815dc20 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.403+0000 7f4abe7fc640 1 -- 172.21.15.19:0/77918704 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ab8007400 con 0x7f4ac815dc20 2024-04-25T10:02:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.403+0000 7f4abe7fc640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4aac06b500 0x7f4aac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.404+0000 7f4abe7fc640 1 -- 172.21.15.19:0/77918704 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f4ab8089830 con 0x7f4ac815dc20 2024-04-25T10:02:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.404+0000 7f4acdaa5640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4aac06b500 0x7f4aac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.404+0000 7f4acdaa5640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4aac06b500 0x7f4aac06d9c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80aa590 tx=0x7f4ac4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.408+0000 7f4abe7fc640 1 -- 172.21.15.19:0/77918704 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ac814ca50 con 0x7f4ac815dc20 2024-04-25T10:02:25.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:25.633+0000 7f4acfd09640 1 -- 172.21.15.19:0/77918704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4ac8073a00 con 0x7f4ac815dc20 2024-04-25T10:02:26.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.278+0000 7f4abe7fc640 1 -- 172.21.15.19:0/77918704 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4ab8057e10 con 0x7f4ac815dc20 2024-04-25T10:02:26.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 -- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4aac06b500 msgr2=0x7f4aac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4aac06b500 0x7f4aac06d9c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80aa590 tx=0x7f4ac4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:26.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 -- 172.21.15.19:0/77918704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac815dc20 msgr2=0x7f4ac80acf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac815dc20 0x7f4ac80acf20 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800c3c0 tx=0x7f4ab800c880 comp rx=0 tx=0).stop 2024-04-25T10:02:26.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 -- 172.21.15.19:0/77918704 shutdown_connections 2024-04-25T10:02:26.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4aac06b500 0x7f4aac06d9c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac8161710 0x7f4ac80ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ac815dc20 0x7f4ac80acf20 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 --2- 172.21.15.19:0/77918704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ac80d32f0 0x7f4ac814c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 -- 172.21.15.19:0/77918704 >> 172.21.15.19:0/77918704 conn(0x7f4ac80d0420 msgr2=0x7f4ac80b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 -- 172.21.15.19:0/77918704 shutdown_connections 2024-04-25T10:02:26.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.282+0000 7f4acfd09640 1 -- 172.21.15.19:0/77918704 wait complete. 2024-04-25T10:02:26.301 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:02:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.451+0000 7f6efb322640 1 Processor -- start 2024-04-25T10:02:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.451+0000 7f6efb322640 1 -- start start 2024-04-25T10:02:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.452+0000 7f6efb322640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ef415b980 0x7f6ef40cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.452+0000 7f6efb322640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef40cf320 con 0x7f6ef40b5c40 2024-04-25T10:02:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.452+0000 7f6efb322640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef40cf040 con 0x7f6ef415b5a0 2024-04-25T10:02:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.452+0000 7f6efb322640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef4161970 con 0x7f6ef415b980 2024-04-25T10:02:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.452+0000 7f6ef98bf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ef415b980 0x7f6ef40cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.452+0000 7f6ef88bd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6ef40b5c40 0x7f6ef40ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50422/0 (socket says 172.21.15.19:50422) 2024-04-25T10:02:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.452+0000 7f6ef88bd640 1 -- 172.21.15.19:0/55744750 learned_addr learned my addr 172.21.15.19:0/55744750 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.452+0000 7f6ef98bf640 1 -- 172.21.15.19:0/55744750 >> v1:172.21.15.112:6789/0 conn(0x7f6ef415b5a0 legacy=0x7f6ef40b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.452+0000 7f6ef98bf640 1 -- 172.21.15.19:0/55744750 >> v1:172.21.15.19:6789/0 conn(0x7f6ef40b5c40 legacy=0x7f6ef40ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.452+0000 7f6ef98bf640 1 -- 172.21.15.19:0/55744750 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef4161c50 con 0x7f6ef415b980 2024-04-25T10:02:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.452+0000 7f6eebfff640 1 -- 172.21.15.19:0/55744750 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3699510041 0 0) 0x7f6ee4001290 con 0x7f6ef40b5c40 2024-04-25T10:02:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.453+0000 7f6ef98bf640 1 --2- 172.21.15.19:0/55744750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ef415b980 0x7f6ef40cea20 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009d90 tx=0x7f6eec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620e00b1826ac1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.453+0000 7f6eebfff640 1 -- 172.21.15.19:0/55744750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec007740 con 0x7f6ef415b980 2024-04-25T10:02:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.453+0000 7f6eebfff640 1 -- 172.21.15.19:0/55744750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eec0078e0 con 0x7f6ef415b980 2024-04-25T10:02:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.453+0000 7f6eebfff640 1 -- 172.21.15.19:0/55744750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec007cb0 con 0x7f6ef415b980 2024-04-25T10:02:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.454+0000 7f6efb322640 1 -- 172.21.15.19:0/55744750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ef415b980 msgr2=0x7f6ef40cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.454+0000 7f6efb322640 1 --2- 172.21.15.19:0/55744750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ef415b980 0x7f6ef40cea20 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009d90 tx=0x7f6eec02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.454+0000 7f6efb322640 1 -- 172.21.15.19:0/55744750 shutdown_connections 2024-04-25T10:02:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.454+0000 7f6efb322640 1 --2- 172.21.15.19:0/55744750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ef415b980 0x7f6ef40cea20 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.454+0000 7f6efb322640 1 -- 172.21.15.19:0/55744750 >> 172.21.15.19:0/55744750 conn(0x7f6ef40d0530 msgr2=0x7f6ef40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.454+0000 7f6efb322640 1 -- 172.21.15.19:0/55744750 shutdown_connections 2024-04-25T10:02:26.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.454+0000 7f6efb322640 1 -- 172.21.15.19:0/55744750 wait complete. 2024-04-25T10:02:26.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.455+0000 7f6efb322640 1 Processor -- start 2024-04-25T10:02:26.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.455+0000 7f6efb322640 1 -- start start 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6efb322640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ef40b5c40 0x7f6ef41531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6efb322640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ef415b5a0 0x7f6ef41f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6efb322640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef415b980 0x7f6ef40c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6efb322640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41625d0 con 0x7f6ef40b5c40 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6efb322640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4162450 con 0x7f6ef415b980 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6efb322640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4162750 con 0x7f6ef415b5a0 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef90be640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ef40b5c40 0x7f6ef41531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef90be640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ef40b5c40 0x7f6ef41531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54050/0 (socket says 172.21.15.19:54050) 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef90be640 1 -- 172.21.15.19:0/13754577 learned_addr learned my addr 172.21.15.19:0/13754577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef98bf640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef415b980 0x7f6ef40c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef88bd640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ef415b5a0 0x7f6ef41f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef90be640 1 -- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ef415b5a0 msgr2=0x7f6ef41f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef90be640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ef415b5a0 0x7f6ef41f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef90be640 1 -- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef415b980 msgr2=0x7f6ef40c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef90be640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef415b980 0x7f6ef40c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef90be640 1 -- 172.21.15.19:0/13754577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef4153990 con 0x7f6ef40b5c40 2024-04-25T10:02:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef88bd640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ef415b5a0 0x7f6ef41f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.456+0000 7f6ef90be640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ef40b5c40 0x7f6ef41531f0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000c430 tx=0x7f6ef000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.457+0000 7f6ee9ffb640 1 -- 172.21.15.19:0/13754577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0011790 con 0x7f6ef40b5c40 2024-04-25T10:02:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.457+0000 7f6ee9ffb640 1 -- 172.21.15.19:0/13754577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef0017070 con 0x7f6ef40b5c40 2024-04-25T10:02:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.457+0000 7f6efb322640 1 -- 172.21.15.19:0/13754577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef4153c70 con 0x7f6ef40b5c40 2024-04-25T10:02:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.457+0000 7f6ee9ffb640 1 -- 172.21.15.19:0/13754577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0011b70 con 0x7f6ef40b5c40 2024-04-25T10:02:26.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.457+0000 7f6efb322640 1 -- 172.21.15.19:0/13754577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef4154230 con 0x7f6ef40b5c40 2024-04-25T10:02:26.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.458+0000 7f6efb322640 1 -- 172.21.15.19:0/13754577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef4073a00 con 0x7f6ef40b5c40 2024-04-25T10:02:26.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.458+0000 7f6ee9ffb640 1 -- 172.21.15.19:0/13754577 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ef0011d10 con 0x7f6ef40b5c40 2024-04-25T10:02:26.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.459+0000 7f6ee9ffb640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6ed806b4d0 0x7f6ed806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.459+0000 7f6ee9ffb640 1 -- 172.21.15.19:0/13754577 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f6ef008a270 con 0x7f6ef40b5c40 2024-04-25T10:02:26.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.459+0000 7f6ef88bd640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6ed806b4d0 0x7f6ed806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.459+0000 7f6ef88bd640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6ed806b4d0 0x7f6ed806d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4001660 tx=0x7f6ee4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.463+0000 7f6ee9ffb640 1 -- 172.21.15.19:0/13754577 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ef0058ec0 con 0x7f6ef40b5c40 2024-04-25T10:02:26.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:26.643+0000 7f6efb322640 1 -- 172.21.15.19:0/13754577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6ef4157360 con 0x7f6ef40b5c40 2024-04-25T10:02:27.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.279+0000 7f6ee9ffb640 1 -- 172.21.15.19:0/13754577 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v132) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6ef00587c0 con 0x7f6ef40b5c40 2024-04-25T10:02:27.284 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:02:27.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.283+0000 7f6efb322640 1 -- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6ed806b4d0 msgr2=0x7f6ed806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.283+0000 7f6efb322640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6ed806b4d0 0x7f6ed806d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4001660 tx=0x7f6ee4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.283+0000 7f6efb322640 1 -- 172.21.15.19:0/13754577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ef40b5c40 msgr2=0x7f6ef41531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.283+0000 7f6efb322640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ef40b5c40 0x7f6ef41531f0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000c430 tx=0x7f6ef000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.283+0000 7f6efb322640 1 -- 172.21.15.19:0/13754577 shutdown_connections 2024-04-25T10:02:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.283+0000 7f6efb322640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6ed806b4d0 0x7f6ed806d990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.283+0000 7f6efb322640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef415b980 0x7f6ef40c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.283+0000 7f6efb322640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ef415b5a0 0x7f6ef41f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.283+0000 7f6efb322640 1 --2- 172.21.15.19:0/13754577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ef40b5c40 0x7f6ef41531f0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.284+0000 7f6efb322640 1 -- 172.21.15.19:0/13754577 >> 172.21.15.19:0/13754577 conn(0x7f6ef40d0530 msgr2=0x7f6ef40b8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.284+0000 7f6efb322640 1 -- 172.21.15.19:0/13754577 shutdown_connections 2024-04-25T10:02:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.284+0000 7f6efb322640 1 -- 172.21.15.19:0/13754577 wait complete. 2024-04-25T10:02:27.302 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:02:27.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.408+0000 7fd3877f5640 1 Processor -- start 2024-04-25T10:02:27.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.408+0000 7fd3877f5640 1 -- start start 2024-04-25T10:02:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.409+0000 7fd3877f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd3800cc4f0 0x7fd3800cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.409+0000 7fd3877f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd38015b980 0x7fd3800cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.409+0000 7fd3877f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd380072280 con 0x7fd3800cdb20 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.409+0000 7fd3877f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd380071270 con 0x7fd38015b980 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.409+0000 7fd3877f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38015bd80 con 0x7fd3800cc4f0 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.409+0000 7fd385d92640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd38015b980 0x7fd3800cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.409+0000 7fd385d92640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd38015b980 0x7fd3800cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54052/0 (socket says 172.21.15.19:54052) 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.409+0000 7fd385d92640 1 -- 172.21.15.19:0/2102505342 learned_addr learned my addr 172.21.15.19:0/2102505342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.410+0000 7fd385591640 1 --2- 172.21.15.19:0/2102505342 >> v2:172.21.15.112:3300/0 conn(0x7fd3800cc4f0 0x7fd3800cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.410+0000 7fd385d92640 1 -- 172.21.15.19:0/2102505342 >> v2:172.21.15.112:3300/0 conn(0x7fd3800cc4f0 msgr2=0x7fd3800cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.410+0000 7fd385d92640 1 --2- 172.21.15.19:0/2102505342 >> v2:172.21.15.112:3300/0 conn(0x7fd3800cc4f0 0x7fd3800cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.410+0000 7fd385d92640 1 -- 172.21.15.19:0/2102505342 >> v1:172.21.15.112:6789/0 conn(0x7fd3800cdb20 legacy=0x7fd38015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.410+0000 7fd385d92640 1 -- 172.21.15.19:0/2102505342 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3800714c0 con 0x7fd38015b980 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.410+0000 7fd385591640 1 --2- 172.21.15.19:0/2102505342 >> v2:172.21.15.112:3300/0 conn(0x7fd3800cc4f0 0x7fd3800cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.410+0000 7fd385d92640 1 --2- 172.21.15.19:0/2102505342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38015b980 0x7fd3800cdfb0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fd3800cd290 tx=0x7fd37002c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd59dafdf70cbaf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.411+0000 7fd37ffff640 1 -- 172.21.15.19:0/2102505342 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370033610 con 0x7fd38015b980 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.411+0000 7fd37ffff640 1 -- 172.21.15.19:0/2102505342 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3700337b0 con 0x7fd38015b980 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.411+0000 7fd37ffff640 1 -- 172.21.15.19:0/2102505342 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370033b80 con 0x7fd38015b980 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.411+0000 7fd3877f5640 1 -- 172.21.15.19:0/2102505342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38015b980 msgr2=0x7fd3800cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.411+0000 7fd3877f5640 1 --2- 172.21.15.19:0/2102505342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38015b980 0x7fd3800cdfb0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fd3800cd290 tx=0x7fd37002c900 comp rx=0 tx=0).stop 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.411+0000 7fd3877f5640 1 -- 172.21.15.19:0/2102505342 shutdown_connections 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.411+0000 7fd3877f5640 1 --2- 172.21.15.19:0/2102505342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd38015b980 0x7fd3800cdfb0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.411+0000 7fd3877f5640 1 --2- 172.21.15.19:0/2102505342 >> v2:172.21.15.112:3300/0 conn(0x7fd3800cc4f0 0x7fd3800cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.411+0000 7fd3877f5640 1 -- 172.21.15.19:0/2102505342 >> 172.21.15.19:0/2102505342 conn(0x7fd38006abe0 msgr2=0x7fd38006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.411+0000 7fd3877f5640 1 -- 172.21.15.19:0/2102505342 shutdown_connections 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.411+0000 7fd3877f5640 1 -- 172.21.15.19:0/2102505342 wait complete. 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.413+0000 7fd3877f5640 1 Processor -- start 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.413+0000 7fd3877f5640 1 -- start start 2024-04-25T10:02:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.413+0000 7fd3877f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3800cc4f0 0x7fd38014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.413+0000 7fd3877f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3800cdb20 0x7fd380151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.413+0000 7fd3877f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd38015b980 0x7fd380157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.413+0000 7fd3877f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380071d60 con 0x7fd3800cdb20 2024-04-25T10:02:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.413+0000 7fd3877f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380071a80 con 0x7fd3800cc4f0 2024-04-25T10:02:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.413+0000 7fd3877f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800ae500 con 0x7fd38015b980 2024-04-25T10:02:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.413+0000 7fd384d90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3800cdb20 0x7fd380151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd384d90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3800cdb20 0x7fd380151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54064/0 (socket says 172.21.15.19:54064) 2024-04-25T10:02:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd384d90640 1 -- 172.21.15.19:0/1806524304 learned_addr learned my addr 172.21.15.19:0/1806524304 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd385591640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3800cc4f0 0x7fd38014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd385d92640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd38015b980 0x7fd380157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd384d90640 1 -- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd38015b980 msgr2=0x7fd380157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd384d90640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd38015b980 0x7fd380157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd384d90640 1 -- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3800cc4f0 msgr2=0x7fd38014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd384d90640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3800cc4f0 0x7fd38014e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd384d90640 1 -- 172.21.15.19:0/1806524304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd380152a20 con 0x7fd3800cdb20 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd385591640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3800cc4f0 0x7fd38014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd385d92640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd38015b980 0x7fd380157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd384d90640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3800cdb20 0x7fd380151fd0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fd37400c460 tx=0x7fd37400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd37dffb640 1 -- 172.21.15.19:0/1806524304 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374016020 con 0x7fd3800cdb20 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd37dffb640 1 -- 172.21.15.19:0/1806524304 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd374012870 con 0x7fd3800cdb20 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd3877f5640 1 -- 172.21.15.19:0/1806524304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd380152d00 con 0x7fd3800cdb20 2024-04-25T10:02:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd37dffb640 1 -- 172.21.15.19:0/1806524304 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374012c60 con 0x7fd3800cdb20 2024-04-25T10:02:27.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.414+0000 7fd3877f5640 1 -- 172.21.15.19:0/1806524304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3801532c0 con 0x7fd3800cdb20 2024-04-25T10:02:27.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.416+0000 7fd3877f5640 1 -- 172.21.15.19:0/1806524304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd380073a00 con 0x7fd3800cdb20 2024-04-25T10:02:27.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.416+0000 7fd37dffb640 1 -- 172.21.15.19:0/1806524304 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd374019020 con 0x7fd3800cdb20 2024-04-25T10:02:27.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.416+0000 7fd37dffb640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd36c06b4d0 0x7fd36c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.416+0000 7fd37dffb640 1 -- 172.21.15.19:0/1806524304 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5835+0+0 (secure 0 0 0) 0x7fd37408a2b0 con 0x7fd3800cdb20 2024-04-25T10:02:27.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.417+0000 7fd385591640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd36c06b4d0 0x7fd36c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.417+0000 7fd385591640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd36c06b4d0 0x7fd36c06d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd37800a990 tx=0x7fd378008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.421+0000 7fd37dffb640 1 -- 172.21.15.19:0/1806524304 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd374020020 con 0x7fd3800cdb20 2024-04-25T10:02:27.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:27.600+0000 7fd3877f5640 1 -- 172.21.15.19:0/1806524304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd38006d3a0 con 0x7fd3800cdb20 2024-04-25T10:02:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.281+0000 7fd37dffb640 1 -- 172.21.15.19:0/1806524304 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v133) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd3740182c0 con 0x7fd3800cdb20 2024-04-25T10:02:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T10:02:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.285+0000 7fd3877f5640 1 -- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd36c06b4d0 msgr2=0x7fd36c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.285+0000 7fd3877f5640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd36c06b4d0 0x7fd36c06d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd37800a990 tx=0x7fd378008040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.285+0000 7fd3877f5640 1 -- 172.21.15.19:0/1806524304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3800cdb20 msgr2=0x7fd380151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.285+0000 7fd3877f5640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3800cdb20 0x7fd380151fd0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fd37400c460 tx=0x7fd37400c920 comp rx=0 tx=0).stop 2024-04-25T10:02:28.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.287+0000 7fd3877f5640 1 -- 172.21.15.19:0/1806524304 shutdown_connections 2024-04-25T10:02:28.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.287+0000 7fd3877f5640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd36c06b4d0 0x7fd36c06d990 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.287+0000 7fd3877f5640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd38015b980 0x7fd380157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.287+0000 7fd3877f5640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3800cdb20 0x7fd380151fd0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.287+0000 7fd3877f5640 1 --2- 172.21.15.19:0/1806524304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3800cc4f0 0x7fd38014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.287+0000 7fd3877f5640 1 -- 172.21.15.19:0/1806524304 >> 172.21.15.19:0/1806524304 conn(0x7fd38006abe0 msgr2=0x7fd380159df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.288+0000 7fd3877f5640 1 -- 172.21.15.19:0/1806524304 shutdown_connections 2024-04-25T10:02:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.288+0000 7fd3877f5640 1 -- 172.21.15.19:0/1806524304 wait complete. 2024-04-25T10:02:28.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:02:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.422+0000 7f409579e640 1 Processor -- start 2024-04-25T10:02:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.422+0000 7f409579e640 1 -- start start 2024-04-25T10:02:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.423+0000 7f409579e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4090161890 0x7f40900b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.423+0000 7f409579e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40900b54a0 0x7f40900b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.423+0000 7f409579e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40900cf1d0 con 0x7f4090161240 2024-04-25T10:02:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.423+0000 7f409579e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40900b7120 con 0x7f4090161890 2024-04-25T10:02:28.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.423+0000 7f409579e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40900b5e30 con 0x7f40900b54a0 2024-04-25T10:02:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.424+0000 7f408effd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4090161240 0x7f40900ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:45330/0 (socket says 172.21.15.19:45330) 2024-04-25T10:02:28.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.424+0000 7f408effd640 1 -- 172.21.15.19:0/1073832060 learned_addr learned my addr 172.21.15.19:0/1073832060 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.425+0000 7f408f7fe640 1 --2- 172.21.15.19:0/1073832060 >> v2:172.21.15.112:3300/0 conn(0x7f40900b54a0 0x7f40900b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.425+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420496499 0 0) 0x7f40900cf1d0 con 0x7f4090161240 2024-04-25T10:02:28.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.425+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4078003520 con 0x7f4090161240 2024-04-25T10:02:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 <== mon.1 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1834092393 0 0) 0x7f4078003520 con 0x7f4090161240 2024-04-25T10:02:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 >> v2:172.21.15.112:3300/0 conn(0x7f40900b54a0 msgr2=0x7f40900b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 --2- 172.21.15.19:0/1073832060 >> v2:172.21.15.112:3300/0 conn(0x7f40900b54a0 0x7f40900b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 >> v2:172.21.15.19:3300/0 conn(0x7f4090161890 msgr2=0x7f40900b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 --2- 172.21.15.19:0/1073832060 >> v2:172.21.15.19:3300/0 conn(0x7f4090161890 0x7f40900b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4090161620 con 0x7f4090161240 2024-04-25T10:02:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 <== mon.1 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f4080001670 con 0x7f4090161240 2024-04-25T10:02:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 >> v1:172.21.15.112:6789/0 conn(0x7f4090161240 legacy=0x7f40900ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 --2- 172.21.15.19:0/1073832060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40780039d0 0x7f4078003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 --2- 172.21.15.19:0/1073832060 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4078004800 0x7f4078004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 --2- 172.21.15.19:0/1073832060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40780051c0 0x7f407800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40900b5e30 con 0x7f40780051c0 2024-04-25T10:02:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407800c040 con 0x7f40780039d0 2024-04-25T10:02:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.426+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407800c240 con 0x7f4078004800 2024-04-25T10:02:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.427+0000 7f408effd640 1 --2- 172.21.15.19:0/1073832060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40780039d0 0x7f4078003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.428+0000 7f408effd640 1 -- 172.21.15.19:0/1073832060 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4078004800 msgr2=0x7f4078004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:02:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.428+0000 7f408effd640 1 --2- 172.21.15.19:0/1073832060 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4078004800 0x7f4078004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.428+0000 7f408effd640 1 -- 172.21.15.19:0/1073832060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40780051c0 msgr2=0x7f407800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.428+0000 7f408effd640 1 --2- 172.21.15.19:0/1073832060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40780051c0 0x7f407800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.428+0000 7f408effd640 1 -- 172.21.15.19:0/1073832060 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4090161620 con 0x7f40780039d0 2024-04-25T10:02:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.428+0000 7f408f7fe640 1 -- 172.21.15.19:0/1073832060 reap_dead start 2024-04-25T10:02:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.429+0000 7f408effd640 1 --2- 172.21.15.19:0/1073832060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40780039d0 0x7f4078003e10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f408000c510 tx=0x7f4078005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c74c209393fc34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.430+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080036070 con 0x7f40780039d0 2024-04-25T10:02:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.430+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4080004aa0 con 0x7f40780039d0 2024-04-25T10:02:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.430+0000 7f408dffb640 1 -- 172.21.15.19:0/1073832060 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408000c0e0 con 0x7f40780039d0 2024-04-25T10:02:28.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.431+0000 7f409579e640 1 -- 172.21.15.19:0/1073832060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40780039d0 msgr2=0x7f4078003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.431+0000 7f409579e640 1 --2- 172.21.15.19:0/1073832060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40780039d0 0x7f4078003e10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f408000c510 tx=0x7f4078005830 comp rx=0 tx=0).stop 2024-04-25T10:02:28.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.432+0000 7f409579e640 1 -- 172.21.15.19:0/1073832060 shutdown_connections 2024-04-25T10:02:28.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.432+0000 7f409579e640 1 --2- 172.21.15.19:0/1073832060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40780039d0 0x7f4078003e10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.432+0000 7f409579e640 1 -- 172.21.15.19:0/1073832060 >> 172.21.15.19:0/1073832060 conn(0x7f40900d04a0 msgr2=0x7f40900d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.434+0000 7f409579e640 1 -- 172.21.15.19:0/1073832060 shutdown_connections 2024-04-25T10:02:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.435+0000 7f409579e640 1 -- 172.21.15.19:0/1073832060 wait complete. 2024-04-25T10:02:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.437+0000 7f409579e640 1 Processor -- start 2024-04-25T10:02:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.437+0000 7f409579e640 1 -- start start 2024-04-25T10:02:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.438+0000 7f409579e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40780039d0 0x7f40900abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.439+0000 7f408effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40780039d0 0x7f40900abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.439+0000 7f408effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40780039d0 0x7f40900abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36620/0 (socket says 172.21.15.19:36620) 2024-04-25T10:02:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.439+0000 7f408effd640 1 -- 172.21.15.19:0/3987419871 learned_addr learned my addr 172.21.15.19:0/3987419871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.439+0000 7f409579e640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4090155050 0x7f40900aa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.439+0000 7f409579e640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900b8600 0x7f40900b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.439+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40901623c0 con 0x7f40780039d0 2024-04-25T10:02:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.439+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40901620e0 con 0x7f40900b8600 2024-04-25T10:02:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.439+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40901626a0 con 0x7f4090155050 2024-04-25T10:02:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.440+0000 7f408f7fe640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900b8600 0x7f40900b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.440+0000 7f408e7fc640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4090155050 0x7f40900aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.441+0000 7f408effd640 1 -- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4090155050 msgr2=0x7f40900aa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.441+0000 7f408effd640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4090155050 0x7f40900aa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.441+0000 7f408effd640 1 -- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900b8600 msgr2=0x7f40900b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.441+0000 7f408effd640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900b8600 0x7f40900b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.441+0000 7f408effd640 1 -- 172.21.15.19:0/3987419871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900aac60 con 0x7f40780039d0 2024-04-25T10:02:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.441+0000 7f408f7fe640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900b8600 0x7f40900b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.441+0000 7f408e7fc640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4090155050 0x7f40900aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.441+0000 7f408effd640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40780039d0 0x7f40900abd00 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f4080000c00 tx=0x7f4080001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.442+0000 7f40777fe640 1 -- 172.21.15.19:0/3987419871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080036040 con 0x7f40780039d0 2024-04-25T10:02:28.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.442+0000 7f40777fe640 1 -- 172.21.15.19:0/3987419871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4080007270 con 0x7f40780039d0 2024-04-25T10:02:28.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.442+0000 7f40777fe640 1 -- 172.21.15.19:0/3987419871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080007660 con 0x7f40780039d0 2024-04-25T10:02:28.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.443+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40900aaf40 con 0x7f40780039d0 2024-04-25T10:02:28.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.443+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40900ab500 con 0x7f40780039d0 2024-04-25T10:02:28.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.444+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40900b8fc0 con 0x7f40780039d0 2024-04-25T10:02:28.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.445+0000 7f40777fe640 1 -- 172.21.15.19:0/3987419871 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4080030480 con 0x7f40780039d0 2024-04-25T10:02:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.445+0000 7f40777fe640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f407c06b4d0 0x7f407c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.446+0000 7f40777fe640 1 -- 172.21.15.19:0/3987419871 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f40800aa580 con 0x7f40780039d0 2024-04-25T10:02:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.446+0000 7f408e7fc640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f407c06b4d0 0x7f407c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.448+0000 7f408e7fc640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f407c06b4d0 0x7f407c06d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f40900b4a20 tx=0x7f4084009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.454+0000 7f40777fe640 1 -- 172.21.15.19:0/3987419871 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4080078e40 con 0x7f40780039d0 2024-04-25T10:02:28.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.652+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4090073a00 con 0x7f40780039d0 2024-04-25T10:02:28.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.655+0000 7f40777fe640 1 -- 172.21.15.19:0/3987419871 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f408007cee0 con 0x7f40780039d0 2024-04-25T10:02:28.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f407c06b4d0 msgr2=0x7f407c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f407c06b4d0 0x7f407c06d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f40900b4a20 tx=0x7f4084009110 comp rx=0 tx=0).stop 2024-04-25T10:02:28.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40780039d0 msgr2=0x7f40900abd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40780039d0 0x7f40900abd00 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f4080000c00 tx=0x7f4080001a40 comp rx=0 tx=0).stop 2024-04-25T10:02:28.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 shutdown_connections 2024-04-25T10:02:28.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f407c06b4d0 0x7f407c06d990 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900b8600 0x7f40900b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4090155050 0x7f40900aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 --2- 172.21.15.19:0/3987419871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40780039d0 0x7f40900abd00 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 >> 172.21.15.19:0/3987419871 conn(0x7f40900d04a0 msgr2=0x7f40900b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 shutdown_connections 2024-04-25T10:02:28.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.657+0000 7f409579e640 1 -- 172.21.15.19:0/3987419871 wait complete. 2024-04-25T10:02:28.673 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:02:28.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.770+0000 7f43fd9d5640 1 Processor -- start 2024-04-25T10:02:28.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.770+0000 7f43fd9d5640 1 -- start start 2024-04-25T10:02:28.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43fd9d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f43f8154ee0 0x7f43f80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43fd9d5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f43f80ce590 0x7f43f806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43fd9d5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f8071480 con 0x7f43f80cfcf0 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43fd9d5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f8072090 con 0x7f43f8154ee0 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43fd9d5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f8071230 con 0x7f43f80ce590 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43f6ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f43f8154ee0 0x7f43f80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43f6ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f43f8154ee0 0x7f43f80cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36626/0 (socket says 172.21.15.19:36626) 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43f6ffd640 1 -- 172.21.15.19:0/2941239127 learned_addr learned my addr 172.21.15.19:0/2941239127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43f77fe640 1 --2- 172.21.15.19:0/2941239127 >> v2:172.21.15.112:3300/0 conn(0x7f43f80ce590 0x7f43f806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43f6ffd640 1 -- 172.21.15.19:0/2941239127 >> v2:172.21.15.112:3300/0 conn(0x7f43f80ce590 msgr2=0x7f43f806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43f6ffd640 1 --2- 172.21.15.19:0/2941239127 >> v2:172.21.15.112:3300/0 conn(0x7f43f80ce590 0x7f43f806d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43f6ffd640 1 -- 172.21.15.19:0/2941239127 >> v1:172.21.15.112:6789/0 conn(0x7f43f80cfcf0 legacy=0x7f43f80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43f6ffd640 1 -- 172.21.15.19:0/2941239127 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f8071760 con 0x7f43f8154ee0 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.771+0000 7f43f77fe640 1 --2- 172.21.15.19:0/2941239127 >> v2:172.21.15.112:3300/0 conn(0x7f43f80ce590 0x7f43f806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.772+0000 7f43f6ffd640 1 --2- 172.21.15.19:0/2941239127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f8154ee0 0x7f43f80cf7b0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f43f80d0700 tx=0x7f43f002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2968106c387cb0fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.772+0000 7f43f5ffb640 1 -- 172.21.15.19:0/2941239127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0037070 con 0x7f43f8154ee0 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.772+0000 7f43f5ffb640 1 -- 172.21.15.19:0/2941239127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f0032440 con 0x7f43f8154ee0 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.772+0000 7f43f5ffb640 1 -- 172.21.15.19:0/2941239127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0032810 con 0x7f43f8154ee0 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.773+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2941239127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f8154ee0 msgr2=0x7f43f80cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.773+0000 7f43fd9d5640 1 --2- 172.21.15.19:0/2941239127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f8154ee0 0x7f43f80cf7b0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f43f80d0700 tx=0x7f43f002a230 comp rx=0 tx=0).stop 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.773+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2941239127 shutdown_connections 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.773+0000 7f43fd9d5640 1 --2- 172.21.15.19:0/2941239127 >> v2:172.21.15.112:3300/0 conn(0x7f43f80ce590 0x7f43f806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.773+0000 7f43fd9d5640 1 --2- 172.21.15.19:0/2941239127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f8154ee0 0x7f43f80cf7b0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.773+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2941239127 >> 172.21.15.19:0/2941239127 conn(0x7f43f806abe0 msgr2=0x7f43f806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.773+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2941239127 shutdown_connections 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.773+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2941239127 wait complete. 2024-04-25T10:02:28.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.774+0000 7f43fd9d5640 1 Processor -- start 2024-04-25T10:02:28.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.774+0000 7f43fd9d5640 1 -- start start 2024-04-25T10:02:28.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43fd9d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43f80ce590 0x7f43f81524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43fd9d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f80cfcf0 0x7f43f81f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43fd9d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f8154ee0 0x7f43f814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43fd9d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f81552c0 con 0x7f43f8154ee0 2024-04-25T10:02:28.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43fd9d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f8071d20 con 0x7f43f80cfcf0 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43fd9d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f8155440 con 0x7f43f80ce590 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f8154ee0 0x7f43f814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f8154ee0 0x7f43f814ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36628/0 (socket says 172.21.15.19:36628) 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43f77fe640 1 -- 172.21.15.19:0/2724859929 learned_addr learned my addr 172.21.15.19:0/2724859929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43f6ffd640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43f80ce590 0x7f43f81524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43f67fc640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f80cfcf0 0x7f43f81f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43f77fe640 1 -- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43f80ce590 msgr2=0x7f43f81524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43f77fe640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43f80ce590 0x7f43f81524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43f77fe640 1 -- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f80cfcf0 msgr2=0x7f43f81f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43f77fe640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f80cfcf0 0x7f43f81f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43f77fe640 1 -- 172.21.15.19:0/2724859929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f81f5650 con 0x7f43f8154ee0 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.775+0000 7f43f77fe640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f8154ee0 0x7f43f814ff50 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f43e800cef0 tx=0x7f43e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.776+0000 7f43effff640 1 -- 172.21.15.19:0/2724859929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e801a070 con 0x7f43f8154ee0 2024-04-25T10:02:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.776+0000 7f43effff640 1 -- 172.21.15.19:0/2724859929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e800d860 con 0x7f43f8154ee0 2024-04-25T10:02:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.776+0000 7f43effff640 1 -- 172.21.15.19:0/2724859929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e800dc50 con 0x7f43f8154ee0 2024-04-25T10:02:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.776+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2724859929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f81f5930 con 0x7f43f8154ee0 2024-04-25T10:02:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.776+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2724859929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f814f750 con 0x7f43f8154ee0 2024-04-25T10:02:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.777+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2724859929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f8073990 con 0x7f43f8154ee0 2024-04-25T10:02:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.777+0000 7f43effff640 1 -- 172.21.15.19:0/2724859929 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43e801d070 con 0x7f43f8154ee0 2024-04-25T10:02:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.777+0000 7f43effff640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43d806b4d0 0x7f43d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.778+0000 7f43effff640 1 -- 172.21.15.19:0/2724859929 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f43e808a870 con 0x7f43f8154ee0 2024-04-25T10:02:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.778+0000 7f43f6ffd640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43d806b4d0 0x7f43d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.778+0000 7f43f6ffd640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43d806b4d0 0x7f43d806d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f43f0034000 tx=0x7f43f002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:28.782+0000 7f43effff640 1 -- 172.21.15.19:0/2724859929 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43e8059310 con 0x7f43f8154ee0 2024-04-25T10:02:29.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.030+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2724859929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f43f806d360 con 0x7f43f8154ee0 2024-04-25T10:02:29.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.290+0000 7f43effff640 1 -- 172.21.15.19:0/2724859929 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v464) v1 ==== 166+0+0 (secure 0 0 0) 0x7f43e805d3b0 con 0x7f43f8154ee0 2024-04-25T10:02:29.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43d806b4d0 msgr2=0x7f43d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43d806b4d0 0x7f43d806d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f43f0034000 tx=0x7f43f002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:29.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2724859929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f8154ee0 msgr2=0x7f43f814ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f8154ee0 0x7f43f814ff50 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f43e800cef0 tx=0x7f43e800a710 comp rx=0 tx=0).stop 2024-04-25T10:02:29.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2724859929 shutdown_connections 2024-04-25T10:02:29.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f43d806b4d0 0x7f43d806d990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43f8154ee0 0x7f43f814ff50 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f80cfcf0 0x7f43f81f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 --2- 172.21.15.19:0/2724859929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43f80ce590 0x7f43f81524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2724859929 >> 172.21.15.19:0/2724859929 conn(0x7f43f806abe0 msgr2=0x7f43f80b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2724859929 shutdown_connections 2024-04-25T10:02:29.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.292+0000 7f43fd9d5640 1 -- 172.21.15.19:0/2724859929 wait complete. 2024-04-25T10:02:29.309 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:02:29.310 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:02:29.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:29.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.407+0000 7fc630e38640 1 Processor -- start 2024-04-25T10:02:29.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.407+0000 7fc630e38640 1 -- start start 2024-04-25T10:02:29.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc630e38640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc62c154ee0 0x7fc62c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc630e38640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc62c0cfcf0 0x7fc62c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc630e38640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc62c071480 con 0x7fc62c0ce3c0 2024-04-25T10:02:29.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc630e38640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62c0ce830 con 0x7fc62c154ee0 2024-04-25T10:02:29.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc630e38640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62c072090 con 0x7fc62c0cfcf0 2024-04-25T10:02:29.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc62a59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc62c154ee0 0x7fc62c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc62ad9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc62c0ce3c0 0x7fc62c15e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60994/0 (socket says 172.21.15.19:60994) 2024-04-25T10:02:29.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc62ad9d640 1 -- 172.21.15.19:0/3046271432 learned_addr learned my addr 172.21.15.19:0/3046271432 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:29.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc62a59c640 1 -- 172.21.15.19:0/3046271432 >> v2:172.21.15.112:3300/0 conn(0x7fc62c0cfcf0 msgr2=0x7fc62c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc62a59c640 1 --2- 172.21.15.19:0/3046271432 >> v2:172.21.15.112:3300/0 conn(0x7fc62c0cfcf0 0x7fc62c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc62a59c640 1 -- 172.21.15.19:0/3046271432 >> v1:172.21.15.19:6789/0 conn(0x7fc62c0ce3c0 legacy=0x7fc62c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc62a59c640 1 -- 172.21.15.19:0/3046271432 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c071760 con 0x7fc62c154ee0 2024-04-25T10:02:29.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.408+0000 7fc62959a640 1 -- 172.21.15.19:0/3046271432 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381049673 0 0) 0x7fc62c071480 con 0x7fc62c0ce3c0 2024-04-25T10:02:29.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.409+0000 7fc62a59c640 1 --2- 172.21.15.19:0/3046271432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62c154ee0 0x7fc62c0cf7b0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0cf560 tx=0x7fc62402c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2550abf491507bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.409+0000 7fc62959a640 1 -- 172.21.15.19:0/3046271432 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc624032770 con 0x7fc62c154ee0 2024-04-25T10:02:29.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.409+0000 7fc62959a640 1 -- 172.21.15.19:0/3046271432 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc624038070 con 0x7fc62c154ee0 2024-04-25T10:02:29.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.409+0000 7fc62959a640 1 -- 172.21.15.19:0/3046271432 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc624032b30 con 0x7fc62c154ee0 2024-04-25T10:02:29.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.410+0000 7fc630e38640 1 -- 172.21.15.19:0/3046271432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62c154ee0 msgr2=0x7fc62c0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.410+0000 7fc630e38640 1 --2- 172.21.15.19:0/3046271432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62c154ee0 0x7fc62c0cf7b0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0cf560 tx=0x7fc62402c870 comp rx=0 tx=0).stop 2024-04-25T10:02:29.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.410+0000 7fc630e38640 1 -- 172.21.15.19:0/3046271432 shutdown_connections 2024-04-25T10:02:29.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.410+0000 7fc630e38640 1 --2- 172.21.15.19:0/3046271432 >> v2:172.21.15.112:3300/0 conn(0x7fc62c0cfcf0 0x7fc62c0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.410+0000 7fc630e38640 1 --2- 172.21.15.19:0/3046271432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62c154ee0 0x7fc62c0cf7b0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.410+0000 7fc630e38640 1 -- 172.21.15.19:0/3046271432 >> 172.21.15.19:0/3046271432 conn(0x7fc62c06abe0 msgr2=0x7fc62c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.410+0000 7fc630e38640 1 -- 172.21.15.19:0/3046271432 shutdown_connections 2024-04-25T10:02:29.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.410+0000 7fc630e38640 1 -- 172.21.15.19:0/3046271432 wait complete. 2024-04-25T10:02:29.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.411+0000 7fc630e38640 1 Processor -- start 2024-04-25T10:02:29.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.411+0000 7fc630e38640 1 -- start start 2024-04-25T10:02:29.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.412+0000 7fc630e38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc62c0ce3c0 0x7fc62c144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.412+0000 7fc630e38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc62c0cfcf0 0x7fc62c148060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.412+0000 7fc630e38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62c154ee0 0x7fc62c1f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.412+0000 7fc630e38640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c071d20 con 0x7fc62c154ee0 2024-04-25T10:02:29.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.412+0000 7fc630e38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c071230 con 0x7fc62c0ce3c0 2024-04-25T10:02:29.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.412+0000 7fc630e38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c1552c0 con 0x7fc62c0cfcf0 2024-04-25T10:02:29.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.412+0000 7fc62ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62c154ee0 0x7fc62c1f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.412+0000 7fc62ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62c154ee0 0x7fc62c1f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36638/0 (socket says 172.21.15.19:36638) 2024-04-25T10:02:29.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.412+0000 7fc62ad9d640 1 -- 172.21.15.19:0/1150633383 learned_addr learned my addr 172.21.15.19:0/1150633383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.412+0000 7fc62a59c640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc62c0ce3c0 0x7fc62c144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.412+0000 7fc629d9b640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc62c0cfcf0 0x7fc62c148060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.413+0000 7fc62ad9d640 1 -- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc62c0cfcf0 msgr2=0x7fc62c148060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.413+0000 7fc62ad9d640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc62c0cfcf0 0x7fc62c148060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.413+0000 7fc62ad9d640 1 -- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc62c0ce3c0 msgr2=0x7fc62c144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.413+0000 7fc62ad9d640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc62c0ce3c0 0x7fc62c144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.413+0000 7fc62ad9d640 1 -- 172.21.15.19:0/1150633383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c1f7840 con 0x7fc62c154ee0 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.413+0000 7fc62ad9d640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62c154ee0 0x7fc62c1f6f60 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00c7e0 tx=0x7fc61c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.413+0000 7fc61affd640 1 -- 172.21.15.19:0/1150633383 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c018070 con 0x7fc62c154ee0 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.413+0000 7fc61affd640 1 -- 172.21.15.19:0/1150633383 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c00a640 con 0x7fc62c154ee0 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.413+0000 7fc61affd640 1 -- 172.21.15.19:0/1150633383 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00aa30 con 0x7fc62c154ee0 2024-04-25T10:02:29.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.413+0000 7fc630e38640 1 -- 172.21.15.19:0/1150633383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62c1f7b20 con 0x7fc62c154ee0 2024-04-25T10:02:29.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.413+0000 7fc630e38640 1 -- 172.21.15.19:0/1150633383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c147b40 con 0x7fc62c154ee0 2024-04-25T10:02:29.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.414+0000 7fc630e38640 1 -- 172.21.15.19:0/1150633383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c073990 con 0x7fc62c154ee0 2024-04-25T10:02:29.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.414+0000 7fc61affd640 1 -- 172.21.15.19:0/1150633383 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc61c00abd0 con 0x7fc62c154ee0 2024-04-25T10:02:29.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.415+0000 7fc61affd640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc60c06b4d0 0x7fc60c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.415+0000 7fc61affd640 1 -- 172.21.15.19:0/1150633383 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc61c0897c0 con 0x7fc62c154ee0 2024-04-25T10:02:29.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.415+0000 7fc62a59c640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc60c06b4d0 0x7fc60c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.416+0000 7fc62a59c640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc60c06b4d0 0x7fc60c06d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc624037000 tx=0x7fc624032000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.419+0000 7fc61affd640 1 -- 172.21.15.19:0/1150633383 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc61c058770 con 0x7fc62c154ee0 2024-04-25T10:02:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.601+0000 7fc630e38640 1 -- 172.21.15.19:0/1150633383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc62c06d360 con 0x7fc62c154ee0 2024-04-25T10:02:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.602+0000 7fc61affd640 1 -- 172.21.15.19:0/1150633383 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+13419 (secure 0 0 0) 0x7fc61c058070 con 0x7fc62c154ee0 2024-04-25T10:02:29.607 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:29.607 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":133,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:28.280534+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/1696858180":"2024-04-26T10:02:25.276061+0000","172.21.15.19:6832/1696858180":"2024-04-26T10:02:25.276061+0000","172.21.15.19:6835/250977615":"2024-04-26T10:02:12.609390+0000","172.21.15.19:6833/250977615":"2024-04-26T10:02:12.609390+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.604+0000 7fc630e38640 1 -- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc60c06b4d0 msgr2=0x7fc60c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.604+0000 7fc630e38640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc60c06b4d0 0x7fc60c06d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc624037000 tx=0x7fc624032000 comp rx=0 tx=0).stop 2024-04-25T10:02:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.604+0000 7fc630e38640 1 -- 172.21.15.19:0/1150633383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62c154ee0 msgr2=0x7fc62c1f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.604+0000 7fc630e38640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62c154ee0 0x7fc62c1f6f60 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00c7e0 tx=0x7fc61c00cca0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.604+0000 7fc630e38640 1 -- 172.21.15.19:0/1150633383 shutdown_connections 2024-04-25T10:02:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.604+0000 7fc630e38640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc60c06b4d0 0x7fc60c06d990 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.604+0000 7fc630e38640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62c154ee0 0x7fc62c1f6f60 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.604+0000 7fc630e38640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc62c0cfcf0 0x7fc62c148060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.604+0000 7fc630e38640 1 --2- 172.21.15.19:0/1150633383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc62c0ce3c0 0x7fc62c144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.604+0000 7fc630e38640 1 -- 172.21.15.19:0/1150633383 >> 172.21.15.19:0/1150633383 conn(0x7fc62c06abe0 msgr2=0x7fc62c06e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.605+0000 7fc630e38640 1 -- 172.21.15.19:0/1150633383 shutdown_connections 2024-04-25T10:02:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.605+0000 7fc630e38640 1 -- 172.21.15.19:0/1150633383 wait complete. 2024-04-25T10:02:29.620 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:29.620 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:29.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.720+0000 7f610a438640 1 Processor -- start 2024-04-25T10:02:29.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.720+0000 7f610a438640 1 -- start start 2024-04-25T10:02:29.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f610a438640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f610415ba20 0x7f610406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f610a438640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61040af260 0x7f610406c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f610a438640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6104072aa0 con 0x7f6104072610 2024-04-25T10:02:29.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f610a438640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f610406f300 con 0x7f61040af260 2024-04-25T10:02:29.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f610a438640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6104072d80 con 0x7f610415ba20 2024-04-25T10:02:29.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f61089d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61040af260 0x7f610406c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f61089d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61040af260 0x7f610406c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36642/0 (socket says 172.21.15.19:36642) 2024-04-25T10:02:29.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f61089d5640 1 -- 172.21.15.19:0/1417797287 learned_addr learned my addr 172.21.15.19:0/1417797287 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:29.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f6103fff640 1 --2- 172.21.15.19:0/1417797287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f610415ba20 0x7f610406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f61089d5640 1 -- 172.21.15.19:0/1417797287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f610415ba20 msgr2=0x7f610406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f61089d5640 1 --2- 172.21.15.19:0/1417797287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f610415ba20 0x7f610406bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f61089d5640 1 -- 172.21.15.19:0/1417797287 >> v1:172.21.15.112:6789/0 conn(0x7f6104072610 legacy=0x7f61040acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.721+0000 7f61089d5640 1 -- 172.21.15.19:0/1417797287 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610406f530 con 0x7f61040af260 2024-04-25T10:02:29.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.722+0000 7f61089d5640 1 --2- 172.21.15.19:0/1417797287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61040af260 0x7f610406c130 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009e50 tx=0x7f60fc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3039e767929409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.722+0000 7f6102ffd640 1 -- 172.21.15.19:0/1417797287 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc036070 con 0x7f61040af260 2024-04-25T10:02:29.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.722+0000 7f6102ffd640 1 -- 172.21.15.19:0/1417797287 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60fc031600 con 0x7f61040af260 2024-04-25T10:02:29.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.722+0000 7f6102ffd640 1 -- 172.21.15.19:0/1417797287 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc0319d0 con 0x7f61040af260 2024-04-25T10:02:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.723+0000 7f610a438640 1 -- 172.21.15.19:0/1417797287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61040af260 msgr2=0x7f610406c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.723+0000 7f610a438640 1 --2- 172.21.15.19:0/1417797287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61040af260 0x7f610406c130 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009e50 tx=0x7f60fc02a710 comp rx=0 tx=0).stop 2024-04-25T10:02:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.723+0000 7f610a438640 1 -- 172.21.15.19:0/1417797287 shutdown_connections 2024-04-25T10:02:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.723+0000 7f610a438640 1 --2- 172.21.15.19:0/1417797287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61040af260 0x7f610406c130 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.723+0000 7f610a438640 1 --2- 172.21.15.19:0/1417797287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f610415ba20 0x7f610406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.723+0000 7f610a438640 1 -- 172.21.15.19:0/1417797287 >> 172.21.15.19:0/1417797287 conn(0x7f6104071af0 msgr2=0x7f6104071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.723+0000 7f610a438640 1 -- 172.21.15.19:0/1417797287 shutdown_connections 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.723+0000 7f610a438640 1 -- 172.21.15.19:0/1417797287 wait complete. 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.724+0000 7f610a438640 1 Processor -- start 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.724+0000 7f610a438640 1 -- start start 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f610a438640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61040af260 0x7f6104153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f610a438640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f610415ba20 0x7f610415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f610a438640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61040b3730 0x7f61040ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f610a438640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6104070010 con 0x7f61040b3730 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f610a438640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610406faf0 con 0x7f610415ba20 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f610a438640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6104154300 con 0x7f61040af260 2024-04-25T10:02:29.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f61089d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61040b3730 0x7f61040ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f61089d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61040b3730 0x7f61040ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36652/0 (socket says 172.21.15.19:36652) 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f61089d5640 1 -- 172.21.15.19:0/681282886 learned_addr learned my addr 172.21.15.19:0/681282886 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f6103fff640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61040af260 0x7f6104153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f61037fe640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f610415ba20 0x7f610415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f61089d5640 1 -- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61040af260 msgr2=0x7f6104153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f61089d5640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61040af260 0x7f6104153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f61089d5640 1 -- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f610415ba20 msgr2=0x7f610415cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f61089d5640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f610415ba20 0x7f610415cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.725+0000 7f61089d5640 1 -- 172.21.15.19:0/681282886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61040cdba0 con 0x7f61040b3730 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.726+0000 7f61089d5640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61040b3730 0x7f61040ce960 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f60fc034040 tx=0x7f60fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.726+0000 7f6100ff9640 1 -- 172.21.15.19:0/681282886 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc036050 con 0x7f61040b3730 2024-04-25T10:02:29.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.726+0000 7f6100ff9640 1 -- 172.21.15.19:0/681282886 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60fc042070 con 0x7f61040b3730 2024-04-25T10:02:29.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.726+0000 7f610a438640 1 -- 172.21.15.19:0/681282886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61040cde80 con 0x7f61040b3730 2024-04-25T10:02:29.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.726+0000 7f6100ff9640 1 -- 172.21.15.19:0/681282886 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc009510 con 0x7f61040b3730 2024-04-25T10:02:29.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.726+0000 7f610a438640 1 -- 172.21.15.19:0/681282886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61040ce440 con 0x7f61040b3730 2024-04-25T10:02:29.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.727+0000 7f610a438640 1 -- 172.21.15.19:0/681282886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61040ce640 con 0x7f61040b3730 2024-04-25T10:02:29.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.727+0000 7f6100ff9640 1 -- 172.21.15.19:0/681282886 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60fc007bd0 con 0x7f61040b3730 2024-04-25T10:02:29.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.728+0000 7f6100ff9640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f60e406b4d0 0x7f60e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.728+0000 7f6100ff9640 1 -- 172.21.15.19:0/681282886 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f60fc0aae90 con 0x7f61040b3730 2024-04-25T10:02:29.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.728+0000 7f6103fff640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f60e406b4d0 0x7f60e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.728+0000 7f6103fff640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f60e406b4d0 0x7f60e406d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0079a0 tx=0x7f60ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.732+0000 7f6100ff9640 1 -- 172.21.15.19:0/681282886 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61040ce640 con 0x7f61040b3730 2024-04-25T10:02:29.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.910+0000 7f610a438640 1 -- 172.21.15.19:0/681282886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61040af7d0 con 0x7f61040b3730 2024-04-25T10:02:29.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.911+0000 7f6100ff9640 1 -- 172.21.15.19:0/681282886 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60fc0ac0a0 con 0x7f61040b3730 2024-04-25T10:02:29.915 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:02:29.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.912+0000 7f610a438640 1 -- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f60e406b4d0 msgr2=0x7f60e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.912+0000 7f610a438640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f60e406b4d0 0x7f60e406d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0079a0 tx=0x7f60ec008040 comp rx=0 tx=0).stop 2024-04-25T10:02:29.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.912+0000 7f610a438640 1 -- 172.21.15.19:0/681282886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61040b3730 msgr2=0x7f61040ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.912+0000 7f610a438640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61040b3730 0x7f61040ce960 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f60fc034040 tx=0x7f60fc004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.912+0000 7f610a438640 1 -- 172.21.15.19:0/681282886 shutdown_connections 2024-04-25T10:02:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.912+0000 7f610a438640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f60e406b4d0 0x7f60e406d990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.912+0000 7f610a438640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61040b3730 0x7f61040ce960 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.912+0000 7f610a438640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f610415ba20 0x7f610415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.912+0000 7f610a438640 1 --2- 172.21.15.19:0/681282886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61040af260 0x7f6104153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.912+0000 7f610a438640 1 -- 172.21.15.19:0/681282886 >> 172.21.15.19:0/681282886 conn(0x7f6104071af0 msgr2=0x7f61040b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.913+0000 7f610a438640 1 -- 172.21.15.19:0/681282886 shutdown_connections 2024-04-25T10:02:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:29.913+0000 7f610a438640 1 -- 172.21.15.19:0/681282886 wait complete. 2024-04-25T10:02:29.928 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:02:30.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.025+0000 7fa6ce6b6640 1 Processor -- start 2024-04-25T10:02:30.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.025+0000 7fa6ce6b6640 1 -- start start 2024-04-25T10:02:30.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6ce6b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c815db70 0x7fa6c8162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6ce6b6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa6c80b6cb0 0x7fa6c80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6ce6b6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c8162690 con 0x7fa6c80b5a60 2024-04-25T10:02:30.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6ce6b6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c80d3800 con 0x7fa6c80b6cb0 2024-04-25T10:02:30.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6ce6b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80ceee0 con 0x7fa6c815db70 2024-04-25T10:02:30.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6c7fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c815db70 0x7fa6c8162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6c77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa6c80b6cb0 0x7fa6c80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6ccc53640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa6c80b5a60 0x7fa6c806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:45356/0 (socket says 172.21.15.19:45356) 2024-04-25T10:02:30.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6ccc53640 1 -- 172.21.15.19:0/1774266944 learned_addr learned my addr 172.21.15.19:0/1774266944 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6c7fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c815db70 0x7fa6c8162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55712/0 (socket says 172.21.15.19:55712) 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6c6ffd640 1 -- 172.21.15.19:0/1774266944 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825832280 0 0) 0x7fa6c8162690 con 0x7fa6c80b5a60 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6c6ffd640 1 -- 172.21.15.19:0/1774266944 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6b0003520 con 0x7fa6c80b5a60 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6c7fff640 1 -- 172.21.15.19:0/1774266944 >> v2:172.21.15.112:3300/0 conn(0x7fa6c80b6cb0 msgr2=0x7fa6c80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6c7fff640 1 --2- 172.21.15.19:0/1774266944 >> v2:172.21.15.112:3300/0 conn(0x7fa6c80b6cb0 0x7fa6c80b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6c7fff640 1 -- 172.21.15.19:0/1774266944 >> v1:172.21.15.112:6789/0 conn(0x7fa6c80b5a60 legacy=0x7fa6c806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6c7fff640 1 -- 172.21.15.19:0/1774266944 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c80d1d10 con 0x7fa6c815db70 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.026+0000 7fa6c77fe640 1 --2- 172.21.15.19:0/1774266944 >> v2:172.21.15.112:3300/0 conn(0x7fa6c80b6cb0 0x7fa6c80b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.027+0000 7fa6c7fff640 1 --2- 172.21.15.19:0/1774266944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c815db70 0x7fa6c8162150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80097a0 tx=0x7fa6b802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e84d03648f0328f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.027+0000 7fa6c6ffd640 1 -- 172.21.15.19:0/1774266944 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8037070 con 0x7fa6c815db70 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.027+0000 7fa6c6ffd640 1 -- 172.21.15.19:0/1774266944 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b8032440 con 0x7fa6c815db70 2024-04-25T10:02:30.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.027+0000 7fa6c6ffd640 1 -- 172.21.15.19:0/1774266944 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8032810 con 0x7fa6c815db70 2024-04-25T10:02:30.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.028+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1774266944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c815db70 msgr2=0x7fa6c8162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.028+0000 7fa6ce6b6640 1 --2- 172.21.15.19:0/1774266944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c815db70 0x7fa6c8162150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80097a0 tx=0x7fa6b802a180 comp rx=0 tx=0).stop 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.028+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1774266944 shutdown_connections 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.028+0000 7fa6ce6b6640 1 --2- 172.21.15.19:0/1774266944 >> v2:172.21.15.112:3300/0 conn(0x7fa6c80b6cb0 0x7fa6c80b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.028+0000 7fa6ce6b6640 1 --2- 172.21.15.19:0/1774266944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c815db70 0x7fa6c8162150 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.028+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1774266944 >> 172.21.15.19:0/1774266944 conn(0x7fa6c80d0520 msgr2=0x7fa6c80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.028+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1774266944 shutdown_connections 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.028+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1774266944 wait complete. 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.029+0000 7fa6ce6b6640 1 Processor -- start 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.029+0000 7fa6ce6b6640 1 -- start start 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6ce6b6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c80b5a60 0x7fa6c8160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6ce6b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c80b6cb0 0x7fa6c814be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6ce6b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6c815db70 0x7fa6c80cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6ce6b6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c815df50 con 0x7fa6c80b5a60 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6ce6b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80d22d0 con 0x7fa6c815db70 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6ce6b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80ce530 con 0x7fa6c80b6cb0 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c80b5a60 0x7fa6c8160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c80b5a60 0x7fa6c8160e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36668/0 (socket says 172.21.15.19:36668) 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6c7fff640 1 -- 172.21.15.19:0/1271052031 learned_addr learned my addr 172.21.15.19:0/1271052031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6c77fe640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c80b6cb0 0x7fa6c814be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6ccc53640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6c815db70 0x7fa6c80cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6c7fff640 1 -- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c80b6cb0 msgr2=0x7fa6c814be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6c7fff640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c80b6cb0 0x7fa6c814be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6c7fff640 1 -- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6c815db70 msgr2=0x7fa6c80cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6c7fff640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6c815db70 0x7fa6c80cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6c7fff640 1 -- 172.21.15.19:0/1271052031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c8161390 con 0x7fa6c80b5a60 2024-04-25T10:02:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.030+0000 7fa6c7fff640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c80b5a60 0x7fa6c8160e50 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8035040 tx=0x7fa6b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.031+0000 7fa6c4ff9640 1 -- 172.21.15.19:0/1271052031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8041030 con 0x7fa6c80b5a60 2024-04-25T10:02:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.031+0000 7fa6c4ff9640 1 -- 172.21.15.19:0/1271052031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b803d4e0 con 0x7fa6c80b5a60 2024-04-25T10:02:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.031+0000 7fa6c4ff9640 1 -- 172.21.15.19:0/1271052031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8037040 con 0x7fa6c80b5a60 2024-04-25T10:02:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.031+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1271052031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c80cab70 con 0x7fa6c80b5a60 2024-04-25T10:02:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.031+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1271052031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c80cb130 con 0x7fa6c80b5a60 2024-04-25T10:02:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.032+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1271052031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c8073990 con 0x7fa6c80b5a60 2024-04-25T10:02:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.032+0000 7fa6c4ff9640 1 -- 172.21.15.19:0/1271052031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6b8044070 con 0x7fa6c80b5a60 2024-04-25T10:02:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.033+0000 7fa6c4ff9640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6b406b4d0 0x7fa6b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.033+0000 7fa6c4ff9640 1 -- 172.21.15.19:0/1271052031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa6b80aa5c0 con 0x7fa6c80b5a60 2024-04-25T10:02:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.033+0000 7fa6c77fe640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6b406b4d0 0x7fa6b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.033+0000 7fa6c77fe640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6b406b4d0 0x7fa6b406d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80b93e0 tx=0x7fa6bc009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.037+0000 7fa6c4ff9640 1 -- 172.21.15.19:0/1271052031 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6b8079040 con 0x7fa6c80b5a60 2024-04-25T10:02:30.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.285+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1271052031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7fa6c80b8610 con 0x7fa6c80b5a60 2024-04-25T10:02:30.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.291+0000 7fa6c4ff9640 1 -- 172.21.15.19:0/1271052031 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v465) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa6b807d0e0 con 0x7fa6c80b5a60 2024-04-25T10:02:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6b406b4d0 msgr2=0x7fa6b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6b406b4d0 0x7fa6b406d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80b93e0 tx=0x7fa6bc009190 comp rx=0 tx=0).stop 2024-04-25T10:02:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1271052031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c80b5a60 msgr2=0x7fa6c8160e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c80b5a60 0x7fa6c8160e50 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8035040 tx=0x7fa6b80046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1271052031 shutdown_connections 2024-04-25T10:02:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa6b406b4d0 0x7fa6b406d990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6c815db70 0x7fa6c80cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6c80b6cb0 0x7fa6c814be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 --2- 172.21.15.19:0/1271052031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c80b5a60 0x7fa6c8160e50 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1271052031 >> 172.21.15.19:0/1271052031 conn(0x7fa6c80d0520 msgr2=0x7fa6c8162150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1271052031 shutdown_connections 2024-04-25T10:02:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.293+0000 7fa6ce6b6640 1 -- 172.21.15.19:0/1271052031 wait complete. 2024-04-25T10:02:30.309 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:02:30.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.408+0000 7f54b1fb0640 1 Processor -- start 2024-04-25T10:02:30.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.408+0000 7f54b1fb0640 1 -- start start 2024-04-25T10:02:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54b1fb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1452a0 0x7f54ac06dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54b1fb0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac0cccc0 con 0x7f54ac144d50 2024-04-25T10:02:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54b1fb0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac0cc140 con 0x7f54ac1446d0 2024-04-25T10:02:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54b1fb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0b2b20 con 0x7f54ac1452a0 2024-04-25T10:02:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54aaffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f54ac144d50 0x7f54ac06d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:32778/0 (socket says 172.21.15.19:32778) 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54aaffd640 1 -- 172.21.15.19:0/2690390897 learned_addr learned my addr 172.21.15.19:0/2690390897 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54abfff640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1452a0 0x7f54ac06dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176650121 0 0) 0x7f54ac0cccc0 con 0x7f54ac144d50 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5494003520 con 0x7f54ac144d50 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359344760 0 0) 0x7f54ac0cc140 con 0x7f54ac1446d0 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54ac0cccc0 con 0x7f54ac1446d0 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1921291236 0 0) 0x7f5494003520 con 0x7f54ac144d50 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1452a0 msgr2=0x7f54ac06dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54aa7fc640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1452a0 0x7f54ac06dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.409+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 >> v1:172.21.15.112:6789/0 conn(0x7f54ac1446d0 legacy=0x7f54ac0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac0cc420 con 0x7f54ac144d50 2024-04-25T10:02:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f54a0001670 con 0x7f54ac144d50 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 >> v1:172.21.15.19:6789/0 conn(0x7f54ac144d50 legacy=0x7f54ac06d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54aa7fc640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1452a0 0x7f5494003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54aa7fc640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5494004120 0x7f5494004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54aa7fc640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 0x7f54940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0b2b20 con 0x7f5494004d40 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549400bb00 con 0x7f5494004120 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549400bd00 con 0x7f54ac1452a0 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54abfff640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 0x7f54940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54aaffd640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5494004120 0x7f5494004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54ab7fe640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1452a0 0x7f5494003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54abfff640 1 -- 172.21.15.19:0/2690390897 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1452a0 msgr2=0x7f5494003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54abfff640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1452a0 0x7f5494003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54abfff640 1 -- 172.21.15.19:0/2690390897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5494004120 msgr2=0x7f5494004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54abfff640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5494004120 0x7f5494004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.410+0000 7f54abfff640 1 -- 172.21.15.19:0/2690390897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5494003520 con 0x7f5494004d40 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.411+0000 7f54abfff640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 0x7f54940051c0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009fc0 tx=0x7f5494005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa86c9904425e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.411+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4009c10 con 0x7f5494004d40 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.411+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a4036070 con 0x7f5494004d40 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.411+0000 7f54aa7fc640 1 -- 172.21.15.19:0/2690390897 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4031880 con 0x7f5494004d40 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.411+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2690390897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 msgr2=0x7f54940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.411+0000 7f54b1fb0640 1 --2- 172.21.15.19:0/2690390897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 0x7f54940051c0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009fc0 tx=0x7f5494005830 comp rx=0 tx=0).stop 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.411+0000 7f54abfff640 1 -- 172.21.15.19:0/2690390897 reap_dead start 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.411+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2690390897 shutdown_connections 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.411+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2690390897 >> 172.21.15.19:0/2690390897 conn(0x7f54ac06abe0 msgr2=0x7f54ac06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.411+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2690390897 shutdown_connections 2024-04-25T10:02:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.411+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2690390897 wait complete. 2024-04-25T10:02:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.413+0000 7f54b1fb0640 1 Processor -- start 2024-04-25T10:02:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.413+0000 7f54b1fb0640 1 -- start start 2024-04-25T10:02:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.413+0000 7f54b1fb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 0x7f54ac0ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.413+0000 7f54b1fb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1f2230 0x7f54ac0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54b1fb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54ac0c6d70 0x7f54ac0c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54b1fb0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0cc9e0 con 0x7f5494004d40 2024-04-25T10:02:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54b1fb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac06d400 con 0x7f54ac0c6d70 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54b1fb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac144ab0 con 0x7f54ac1f2230 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54ab7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 0x7f54ac0ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54ab7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 0x7f54ac0ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36692/0 (socket says 172.21.15.19:36692) 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54ab7fe640 1 -- 172.21.15.19:0/2677076552 learned_addr learned my addr 172.21.15.19:0/2677076552 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54ab7fe640 1 -- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1f2230 msgr2=0x7f54ac0c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54ab7fe640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1f2230 0x7f54ac0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54ab7fe640 1 -- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54ac0c6d70 msgr2=0x7f54ac0c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54ab7fe640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54ac0c6d70 0x7f54ac0c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54ab7fe640 1 -- 172.21.15.19:0/2677076552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac1f26b0 con 0x7f5494004d40 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54ab7fe640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 0x7f54ac0ab500 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f549c00f810 tx=0x7f549c00fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f5493fff640 1 -- 172.21.15.19:0/2677076552 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0052e0 con 0x7f5494004d40 2024-04-25T10:02:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f5493fff640 1 -- 172.21.15.19:0/2677076552 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c018070 con 0x7f5494004d40 2024-04-25T10:02:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f5493fff640 1 -- 172.21.15.19:0/2677076552 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c00d8f0 con 0x7f5494004d40 2024-04-25T10:02:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2677076552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac0abc30 con 0x7f5494004d40 2024-04-25T10:02:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.414+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2677076552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac0ac1f0 con 0x7f5494004d40 2024-04-25T10:02:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.415+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2677076552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ac073a00 con 0x7f5494004d40 2024-04-25T10:02:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.416+0000 7f5493fff640 1 -- 172.21.15.19:0/2677076552 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f549c00da90 con 0x7f5494004d40 2024-04-25T10:02:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.416+0000 7f5493fff640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f549806b4d0 0x7f549806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.416+0000 7f5493fff640 1 -- 172.21.15.19:0/2677076552 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f549c089fe0 con 0x7f5494004d40 2024-04-25T10:02:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.416+0000 7f54aaffd640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f549806b4d0 0x7f549806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.417+0000 7f54aaffd640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f549806b4d0 0x7f549806d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f54a0001050 tx=0x7f54a0001ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.420+0000 7f5493fff640 1 -- 172.21.15.19:0/2677076552 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f549c01a310 con 0x7f5494004d40 2024-04-25T10:02:30.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.603+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2677076552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f54ac143190 con 0x7f5494004d40 2024-04-25T10:02:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.606+0000 7f5493fff640 1 -- 172.21.15.19:0/2677076552 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f549c059010 con 0x7f5494004d40 2024-04-25T10:02:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f549806b4d0 msgr2=0x7f549806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f549806b4d0 0x7f549806d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f54a0001050 tx=0x7f54a0001ac0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2677076552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 msgr2=0x7f54ac0ab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 0x7f54ac0ab500 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f549c00f810 tx=0x7f549c00fcd0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2677076552 shutdown_connections 2024-04-25T10:02:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f549806b4d0 0x7f549806d990 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54ac0c6d70 0x7f54ac0c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54ac1f2230 0x7f54ac0c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 --2- 172.21.15.19:0/2677076552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5494004d40 0x7f54ac0ab500 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2677076552 >> 172.21.15.19:0/2677076552 conn(0x7f54ac06abe0 msgr2=0x7f54ac143470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2677076552 shutdown_connections 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.608+0000 7f54b1fb0640 1 -- 172.21.15.19:0/2677076552 wait complete. 2024-04-25T10:02:30.624 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:02:30.624 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:30.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:30.721 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:02:30.721 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:02:30.721 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:30.737 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:02:30.737 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:02:30.737 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:30.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:30.779 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:02:30.779 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:30.794 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:02:30.794 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:02:30.794 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:30.836 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:30.836 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:02:30.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:30.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.932+0000 7f67510fc640 1 Processor -- start 2024-04-25T10:02:30.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.932+0000 7f67510fc640 1 -- start start 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.933+0000 7f67510fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f674c15a790 0x7f674c0d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.933+0000 7f67510fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f674c0b4eb0 0x7f674c0d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.933+0000 7f67510fc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674c0b3710 con 0x7f674c06fea0 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.933+0000 7f67510fc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674c0cf0e0 con 0x7f674c15a790 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.933+0000 7f67510fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0cf3c0 con 0x7f674c0b4eb0 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.933+0000 7f674a59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f674c06fea0 0x7f674c072300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:32780/0 (socket says 172.21.15.19:32780) 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.933+0000 7f674a59c640 1 -- 172.21.15.19:0/2967790936 learned_addr learned my addr 172.21.15.19:0/2967790936 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844819191 0 0) 0x7f674c0b3710 con 0x7f674c06fea0 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f674b59e640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f674c0b4eb0 0x7f674c0d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f674ad9d640 1 --2- 172.21.15.19:0/2967790936 >> v2:172.21.15.112:3300/0 conn(0x7f674c15a790 0x7f674c0d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6738003520 con 0x7f674c06fea0 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2173232564 0 0) 0x7f6738003520 con 0x7f674c06fea0 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f674c0b4eb0 msgr2=0x7f674c0d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f674c0b4eb0 0x7f674c0d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 >> v2:172.21.15.112:3300/0 conn(0x7f674c15a790 msgr2=0x7f674c0d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 --2- 172.21.15.19:0/2967790936 >> v2:172.21.15.112:3300/0 conn(0x7f674c15a790 0x7f674c0d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c0cf6a0 con 0x7f674c06fea0 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f6734001670 con 0x7f674c06fea0 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 >> v1:172.21.15.19:6789/0 conn(0x7f674c06fea0 legacy=0x7f674c072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67380039b0 0x7f6738003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f674c0b4eb0 0x7f6738004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6738004d40 0x7f67380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0cf0e0 con 0x7f67380039b0 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0cf3c0 con 0x7f6738004d40 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673800b9c0 con 0x7f674c0b4eb0 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f674ad9d640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67380039b0 0x7f6738003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f674a59c640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f674c0b4eb0 0x7f6738004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f674b59e640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6738004d40 0x7f67380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f674ad9d640 1 -- 172.21.15.19:0/2967790936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f674c0b4eb0 msgr2=0x7f6738004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f674ad9d640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f674c0b4eb0 0x7f6738004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.934+0000 7f674ad9d640 1 -- 172.21.15.19:0/2967790936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6738004d40 msgr2=0x7f67380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.935+0000 7f674ad9d640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6738004d40 0x7f67380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.935+0000 7f674ad9d640 1 -- 172.21.15.19:0/2967790936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6738003520 con 0x7f67380039b0 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.935+0000 7f674ad9d640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67380039b0 0x7f6738003dd0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f6744009ac0 tx=0x7f6738005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cda4e666d60669 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.935+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744036070 con 0x7f67380039b0 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.935+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6744031400 con 0x7f67380039b0 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.935+0000 7f6749d9b640 1 -- 172.21.15.19:0/2967790936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744009690 con 0x7f67380039b0 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.936+0000 7f67510fc640 1 -- 172.21.15.19:0/2967790936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67380039b0 msgr2=0x7f6738003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.936+0000 7f67510fc640 1 --2- 172.21.15.19:0/2967790936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67380039b0 0x7f6738003dd0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f6744009ac0 tx=0x7f6738005830 comp rx=0 tx=0).stop 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.936+0000 7f674b59e640 1 -- 172.21.15.19:0/2967790936 reap_dead start 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.936+0000 7f67510fc640 1 -- 172.21.15.19:0/2967790936 shutdown_connections 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.936+0000 7f67510fc640 1 -- 172.21.15.19:0/2967790936 >> 172.21.15.19:0/2967790936 conn(0x7f674c0b2370 msgr2=0x7f674c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.936+0000 7f67510fc640 1 -- 172.21.15.19:0/2967790936 shutdown_connections 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.936+0000 7f67510fc640 1 -- 172.21.15.19:0/2967790936 wait complete. 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.937+0000 7f67510fc640 1 Processor -- start 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.937+0000 7f67510fc640 1 -- start start 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.938+0000 7f67510fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67380039b0 0x7f674c153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.938+0000 7f67510fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674c15c8d0 0x7f674c1eff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.938+0000 7f67510fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f674c15cd50 0x7f674c0b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.938+0000 7f67510fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c15ab70 con 0x7f674c15c8d0 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.938+0000 7f67510fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0cf980 con 0x7f674c15cd50 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.938+0000 7f67510fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c15acf0 con 0x7f67380039b0 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.938+0000 7f674a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674c15c8d0 0x7f674c1eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.938+0000 7f674a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674c15c8d0 0x7f674c1eff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36720/0 (socket says 172.21.15.19:36720) 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.938+0000 7f674a59c640 1 -- 172.21.15.19:0/294126391 learned_addr learned my addr 172.21.15.19:0/294126391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f674ad9d640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67380039b0 0x7f674c153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f674b59e640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f674c15cd50 0x7f674c0b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f674a59c640 1 -- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67380039b0 msgr2=0x7f674c153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f674a59c640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67380039b0 0x7f674c153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f674a59c640 1 -- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f674c15cd50 msgr2=0x7f674c0b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f674a59c640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f674c15cd50 0x7f674c0b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f674a59c640 1 -- 172.21.15.19:0/294126391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c1f0a00 con 0x7f674c15c8d0 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f674ad9d640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67380039b0 0x7f674c153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f674b59e640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f674c15cd50 0x7f674c0b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f674a59c640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674c15c8d0 0x7f674c1eff80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f6734007340 tx=0x7f6734007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f67437fe640 1 -- 172.21.15.19:0/294126391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400f9c0 con 0x7f674c15c8d0 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f67437fe640 1 -- 172.21.15.19:0/294126391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6734018070 con 0x7f674c15c8d0 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f67437fe640 1 -- 172.21.15.19:0/294126391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400fda0 con 0x7f674c15c8d0 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f67510fc640 1 -- 172.21.15.19:0/294126391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674c1f0ce0 con 0x7f674c15c8d0 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.939+0000 7f67510fc640 1 -- 172.21.15.19:0/294126391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674c15d840 con 0x7f674c15c8d0 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.940+0000 7f67510fc640 1 -- 172.21.15.19:0/294126391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674c1f0eb0 con 0x7f674c15c8d0 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.941+0000 7f67437fe640 1 -- 172.21.15.19:0/294126391 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f673401d460 con 0x7f674c15c8d0 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.941+0000 7f67437fe640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f672c06b4d0 0x7f672c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.941+0000 7f67437fe640 1 -- 172.21.15.19:0/294126391 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f673408a1f0 con 0x7f674c15c8d0 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.941+0000 7f674ad9d640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f672c06b4d0 0x7f672c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.942+0000 7f674ad9d640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f672c06b4d0 0x7f672c06d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6744033000 tx=0x7f6744008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:30.945+0000 7f67437fe640 1 -- 172.21.15.19:0/294126391 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6734059190 con 0x7f674c15c8d0 2024-04-25T10:02:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.160+0000 7f67510fc640 1 -- 172.21.15.19:0/294126391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f674c073990 con 0x7f674c15c8d0 2024-04-25T10:02:31.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.160+0000 7f67437fe640 1 -- 172.21.15.19:0/294126391 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+2914 (secure 0 0 0) 0x7f6734058a90 con 0x7f674c15c8d0 2024-04-25T10:02:31.165 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:31.165 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8074,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3381560592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3381560592},{"type":"v1","addr":"172.21.15.112:6835","nonce":3381560592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":8080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2600014936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2600014936},{"type":"v1","addr":"172.21.15.112:6837","nonce":2600014936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":9678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1408586258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1408586258},{"type":"v1","addr":"172.21.15.19:6833","nonce":1408586258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[]} 2024-04-25T10:02:31.166 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 85 2024-04-25T10:02:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 -- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f672c06b4d0 msgr2=0x7f672c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f672c06b4d0 0x7f672c06d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6744033000 tx=0x7f6744008030 comp rx=0 tx=0).stop 2024-04-25T10:02:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 -- 172.21.15.19:0/294126391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674c15c8d0 msgr2=0x7f674c1eff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674c15c8d0 0x7f674c1eff80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f6734007340 tx=0x7f6734007800 comp rx=0 tx=0).stop 2024-04-25T10:02:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 -- 172.21.15.19:0/294126391 shutdown_connections 2024-04-25T10:02:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f672c06b4d0 0x7f672c06d990 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f674c15cd50 0x7f674c0b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f674c15c8d0 0x7f674c1eff80 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 --2- 172.21.15.19:0/294126391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67380039b0 0x7f674c153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 -- 172.21.15.19:0/294126391 >> 172.21.15.19:0/294126391 conn(0x7f674c0b2370 msgr2=0x7f674c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 -- 172.21.15.19:0/294126391 shutdown_connections 2024-04-25T10:02:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.162+0000 7f67510fc640 1 -- 172.21.15.19:0/294126391 wait complete. 2024-04-25T10:02:31.178 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:02:31.179 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:02:31.179 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:02:31.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:31.179 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:02:31.179 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:02:31.180 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:02:31.180 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:02:31.180 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:02:31.180 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:31.180 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:02:31.180 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:02:31.181 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:02:31.181 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:31.183 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:02:31.183 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.188 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:02:31.188 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:02:31.192 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:02:31.192 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:02:31.192 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:02:31.193 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:02:31.230 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:02:31.231 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.234 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:02:31.234 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.238 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:02:31.238 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.238 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:02:31.257 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:02:31.308 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T10:02:31.315 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T10:02:31.316 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T10:02:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.341+0000 7f348c212640 1 Processor -- start 2024-04-25T10:02:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.341+0000 7f348c212640 1 -- start start 2024-04-25T10:02:31.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.341+0000 7f348c212640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348415b7d0 0x7f34840cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.341+0000 7f348c212640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34840b5620 0x7f34840b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f348c212640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f348406c2d0 0x7f348406c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f348c212640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34840b5fc0 con 0x7f34840b5620 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f348c212640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348406cca0 con 0x7f348406c2d0 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f348c212640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348406cf50 con 0x7f348415b7d0 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f34897ad640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34840b5620 0x7f34840b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f34897ad640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34840b5620 0x7f34840b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36764/0 (socket says 172.21.15.19:36764) 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f34897ad640 1 -- 172.21.15.19:0/1719235544 learned_addr learned my addr 172.21.15.19:0/1719235544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f348a7af640 1 --2- 172.21.15.19:0/1719235544 >> v2:172.21.15.112:3300/0 conn(0x7f348406c2d0 0x7f348406c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f3489fae640 1 --2- 172.21.15.19:0/1719235544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348415b7d0 0x7f34840cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f34897ad640 1 -- 172.21.15.19:0/1719235544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348415b7d0 msgr2=0x7f34840cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f34897ad640 1 --2- 172.21.15.19:0/1719235544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348415b7d0 0x7f34840cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f34897ad640 1 -- 172.21.15.19:0/1719235544 >> v2:172.21.15.112:3300/0 conn(0x7f348406c2d0 msgr2=0x7f348406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f34897ad640 1 --2- 172.21.15.19:0/1719235544 >> v2:172.21.15.112:3300/0 conn(0x7f348406c2d0 0x7f348406c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.342+0000 7f34897ad640 1 -- 172.21.15.19:0/1719235544 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34840ce530 con 0x7f34840b5620 2024-04-25T10:02:31.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.343+0000 7f34897ad640 1 --2- 172.21.15.19:0/1719235544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34840b5620 0x7f34840b5a80 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f348415e340 tx=0x7f347802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d796338a3197cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.343+0000 7f3488fac640 1 -- 172.21.15.19:0/1719235544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478037070 con 0x7f34840b5620 2024-04-25T10:02:31.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.343+0000 7f3488fac640 1 -- 172.21.15.19:0/1719235544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3478032440 con 0x7f34840b5620 2024-04-25T10:02:31.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.343+0000 7f3488fac640 1 -- 172.21.15.19:0/1719235544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478032810 con 0x7f34840b5620 2024-04-25T10:02:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.343+0000 7f348c212640 1 -- 172.21.15.19:0/1719235544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34840b5620 msgr2=0x7f34840b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.343+0000 7f348c212640 1 --2- 172.21.15.19:0/1719235544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34840b5620 0x7f34840b5a80 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f348415e340 tx=0x7f347802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.343+0000 7f348c212640 1 -- 172.21.15.19:0/1719235544 shutdown_connections 2024-04-25T10:02:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.344+0000 7f348c212640 1 --2- 172.21.15.19:0/1719235544 >> v2:172.21.15.112:3300/0 conn(0x7f348406c2d0 0x7f348406c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.344+0000 7f348c212640 1 --2- 172.21.15.19:0/1719235544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34840b5620 0x7f34840b5a80 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.344+0000 7f348c212640 1 --2- 172.21.15.19:0/1719235544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348415b7d0 0x7f34840cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.344+0000 7f348c212640 1 -- 172.21.15.19:0/1719235544 >> 172.21.15.19:0/1719235544 conn(0x7f34840d0800 msgr2=0x7f34840d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.344+0000 7f348c212640 1 -- 172.21.15.19:0/1719235544 shutdown_connections 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.344+0000 7f348c212640 1 -- 172.21.15.19:0/1719235544 wait complete. 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.345+0000 7f348c212640 1 Processor -- start 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.345+0000 7f348c212640 1 -- start start 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348c212640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348406c2d0 0x7f3484157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348c212640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34840b5620 0x7f34840cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348c212640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f348415b7d0 0x7f34841603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348c212640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348406d230 con 0x7f348415b7d0 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348c212640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34840ceaf0 con 0x7f34840b5620 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348c212640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34840cf550 con 0x7f348406c2d0 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348a7af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f348415b7d0 0x7f34841603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348a7af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f348415b7d0 0x7f34841603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36774/0 (socket says 172.21.15.19:36774) 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348a7af640 1 -- 172.21.15.19:0/853985323 learned_addr learned my addr 172.21.15.19:0/853985323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f34897ad640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34840b5620 0x7f34840cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f3489fae640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348406c2d0 0x7f3484157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348a7af640 1 -- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348406c2d0 msgr2=0x7f3484157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348a7af640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348406c2d0 0x7f3484157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348a7af640 1 -- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34840b5620 msgr2=0x7f34840cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348a7af640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34840b5620 0x7f34840cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348a7af640 1 -- 172.21.15.19:0/853985323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34841586e0 con 0x7f348415b7d0 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f34897ad640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34840b5620 0x7f34840cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348a7af640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f348415b7d0 0x7f34841603a0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f347400ef00 tx=0x7f347400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f34827fc640 1 -- 172.21.15.19:0/853985323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474018020 con 0x7f348415b7d0 2024-04-25T10:02:31.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f34827fc640 1 -- 172.21.15.19:0/853985323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347400bd10 con 0x7f348415b7d0 2024-04-25T10:02:31.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348c212640 1 -- 172.21.15.19:0/853985323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34840cd580 con 0x7f348415b7d0 2024-04-25T10:02:31.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f34827fc640 1 -- 172.21.15.19:0/853985323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474014640 con 0x7f348415b7d0 2024-04-25T10:02:31.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.346+0000 7f348c212640 1 -- 172.21.15.19:0/853985323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34840cdb40 con 0x7f348415b7d0 2024-04-25T10:02:31.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.347+0000 7f348c212640 1 -- 172.21.15.19:0/853985323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3484073a00 con 0x7f348415b7d0 2024-04-25T10:02:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.348+0000 7f34827fc640 1 -- 172.21.15.19:0/853985323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3474005d10 con 0x7f348415b7d0 2024-04-25T10:02:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.348+0000 7f34827fc640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f347006b500 0x7f347006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.349+0000 7f34827fc640 1 -- 172.21.15.19:0/853985323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f347408a500 con 0x7f348415b7d0 2024-04-25T10:02:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.349+0000 7f3489fae640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f347006b500 0x7f347006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.349+0000 7f3489fae640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f347006b500 0x7f347006d9c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f34840b2790 tx=0x7f347c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.353+0000 7f34827fc640 1 -- 172.21.15.19:0/853985323 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3474022070 con 0x7f348415b7d0 2024-04-25T10:02:31.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.545+0000 7f348c212640 1 -- 172.21.15.19:0/853985323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f348415dbc0 con 0x7f348415b7d0 2024-04-25T10:02:31.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.616+0000 7f34827fc640 1 -- 172.21.15.19:0/853985323 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v135) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3474058e40 con 0x7f348415b7d0 2024-04-25T10:02:31.620 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T10:02:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 -- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f347006b500 msgr2=0x7f347006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f347006b500 0x7f347006d9c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f34840b2790 tx=0x7f347c009190 comp rx=0 tx=0).stop 2024-04-25T10:02:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 -- 172.21.15.19:0/853985323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f348415b7d0 msgr2=0x7f34841603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f348415b7d0 0x7f34841603a0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f347400ef00 tx=0x7f347400b480 comp rx=0 tx=0).stop 2024-04-25T10:02:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 -- 172.21.15.19:0/853985323 shutdown_connections 2024-04-25T10:02:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f347006b500 0x7f347006d9c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f348415b7d0 0x7f34841603a0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34840b5620 0x7f34840cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 --2- 172.21.15.19:0/853985323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348406c2d0 0x7f3484157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 -- 172.21.15.19:0/853985323 >> 172.21.15.19:0/853985323 conn(0x7f34840d0800 msgr2=0x7f34840b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 -- 172.21.15.19:0/853985323 shutdown_connections 2024-04-25T10:02:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.618+0000 7f348c212640 1 -- 172.21.15.19:0/853985323 wait complete. 2024-04-25T10:02:31.634 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:02:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.771+0000 7f4b6ab42640 1 Processor -- start 2024-04-25T10:02:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.771+0000 7f4b6ab42640 1 -- start start 2024-04-25T10:02:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b6ab42640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b6415b9c0 0x7f4b6415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b6ab42640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 0x7f4b6406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b6ab42640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b640cfa60 con 0x7f4b6406abe0 2024-04-25T10:02:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b6ab42640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b640731a0 con 0x7f4b6415b9c0 2024-04-25T10:02:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b6ab42640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b640cfd40 con 0x7f4b64072d30 2024-04-25T10:02:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b690df640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 0x7f4b6406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b688de640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4b6415b9c0 0x7f4b6415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b690df640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 0x7f4b6406b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55816/0 (socket says 172.21.15.19:55816) 2024-04-25T10:02:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b690df640 1 -- 172.21.15.19:0/1423780338 learned_addr learned my addr 172.21.15.19:0/1423780338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b5bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4b6406abe0 0x7f4b64072620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:45374/0 (socket says 172.21.15.19:45374) 2024-04-25T10:02:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b690df640 1 -- 172.21.15.19:0/1423780338 >> v2:172.21.15.112:3300/0 conn(0x7f4b6415b9c0 msgr2=0x7f4b6415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b690df640 1 --2- 172.21.15.19:0/1423780338 >> v2:172.21.15.112:3300/0 conn(0x7f4b6415b9c0 0x7f4b6415bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b690df640 1 -- 172.21.15.19:0/1423780338 >> v1:172.21.15.112:6789/0 conn(0x7f4b6406abe0 legacy=0x7f4b64072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b690df640 1 -- 172.21.15.19:0/1423780338 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b640d0020 con 0x7f4b64072d30 2024-04-25T10:02:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.772+0000 7f4b688de640 1 --2- 172.21.15.19:0/1423780338 >> v2:172.21.15.112:3300/0 conn(0x7f4b6415b9c0 0x7f4b6415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.773+0000 7f4b690df640 1 --2- 172.21.15.19:0/1423780338 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 0x7f4b6406b070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009e50 tx=0x7f4b5c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb4361a37c74a738 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.773+0000 7f4b5b7fe640 1 -- 172.21.15.19:0/1423780338 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c036070 con 0x7f4b64072d30 2024-04-25T10:02:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.773+0000 7f4b5b7fe640 1 -- 172.21.15.19:0/1423780338 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c031600 con 0x7f4b64072d30 2024-04-25T10:02:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.773+0000 7f4b5b7fe640 1 -- 172.21.15.19:0/1423780338 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0319d0 con 0x7f4b64072d30 2024-04-25T10:02:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.774+0000 7f4b6ab42640 1 -- 172.21.15.19:0/1423780338 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 msgr2=0x7f4b6406b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.774+0000 7f4b6ab42640 1 --2- 172.21.15.19:0/1423780338 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 0x7f4b6406b070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009e50 tx=0x7f4b5c02a710 comp rx=0 tx=0).stop 2024-04-25T10:02:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.774+0000 7f4b6ab42640 1 -- 172.21.15.19:0/1423780338 shutdown_connections 2024-04-25T10:02:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.774+0000 7f4b6ab42640 1 --2- 172.21.15.19:0/1423780338 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 0x7f4b6406b070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.774+0000 7f4b6ab42640 1 --2- 172.21.15.19:0/1423780338 >> v2:172.21.15.112:3300/0 conn(0x7f4b6415b9c0 0x7f4b6415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.774+0000 7f4b6ab42640 1 -- 172.21.15.19:0/1423780338 >> 172.21.15.19:0/1423780338 conn(0x7f4b640b2370 msgr2=0x7f4b640b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.774+0000 7f4b6ab42640 1 -- 172.21.15.19:0/1423780338 shutdown_connections 2024-04-25T10:02:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.774+0000 7f4b6ab42640 1 -- 172.21.15.19:0/1423780338 wait complete. 2024-04-25T10:02:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.775+0000 7f4b6ab42640 1 Processor -- start 2024-04-25T10:02:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.775+0000 7f4b6ab42640 1 -- start start 2024-04-25T10:02:31.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.776+0000 7f4b6ab42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b6406abe0 0x7f4b640ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.776+0000 7f4b6ab42640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 0x7f4b64158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.776+0000 7f4b6ab42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6415b9c0 0x7f4b640cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.776+0000 7f4b6ab42640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6415c2e0 con 0x7f4b6415b9c0 2024-04-25T10:02:31.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.776+0000 7f4b6ab42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b640cf710 con 0x7f4b6406abe0 2024-04-25T10:02:31.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.776+0000 7f4b6ab42640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b640d0820 con 0x7f4b64072d30 2024-04-25T10:02:31.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.776+0000 7f4b690df640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6415b9c0 0x7f4b640cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.776+0000 7f4b690df640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6415b9c0 0x7f4b640cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36778/0 (socket says 172.21.15.19:36778) 2024-04-25T10:02:31.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.776+0000 7f4b690df640 1 -- 172.21.15.19:0/3497507580 learned_addr learned my addr 172.21.15.19:0/3497507580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:31.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.776+0000 7f4b688de640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b6406abe0 0x7f4b640ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.776+0000 7f4b5bfff640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 0x7f4b64158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b690df640 1 -- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 msgr2=0x7f4b64158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b690df640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 0x7f4b64158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b690df640 1 -- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b6406abe0 msgr2=0x7f4b640ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b690df640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b6406abe0 0x7f4b640ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b690df640 1 -- 172.21.15.19:0/3497507580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b640cacd0 con 0x7f4b6415b9c0 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b688de640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b6406abe0 0x7f4b640ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b690df640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6415b9c0 0x7f4b640cbd70 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009e50 tx=0x7f4b5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b597fa640 1 -- 172.21.15.19:0/3497507580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c036050 con 0x7f4b6415b9c0 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b597fa640 1 -- 172.21.15.19:0/3497507580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c042070 con 0x7f4b6415b9c0 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b6ab42640 1 -- 172.21.15.19:0/3497507580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b640cafb0 con 0x7f4b6415b9c0 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b597fa640 1 -- 172.21.15.19:0/3497507580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c009510 con 0x7f4b6415b9c0 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.777+0000 7f4b6ab42640 1 -- 172.21.15.19:0/3497507580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b640cb570 con 0x7f4b6415b9c0 2024-04-25T10:02:31.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.778+0000 7f4b6ab42640 1 -- 172.21.15.19:0/3497507580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b64145a10 con 0x7f4b6415b9c0 2024-04-25T10:02:31.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.778+0000 7f4b597fa640 1 -- 172.21.15.19:0/3497507580 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b5c007bf0 con 0x7f4b6415b9c0 2024-04-25T10:02:31.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.779+0000 7f4b597fa640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5006b500 0x7f4b5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.779+0000 7f4b597fa640 1 -- 172.21.15.19:0/3497507580 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4b5c0aad60 con 0x7f4b6415b9c0 2024-04-25T10:02:31.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.779+0000 7f4b688de640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5006b500 0x7f4b5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.780+0000 7f4b688de640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5006b500 0x7f4b5006d9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009af0 tx=0x7f4b60008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:31.783+0000 7f4b597fa640 1 -- 172.21.15.19:0/3497507580 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b64145a10 con 0x7f4b6415b9c0 2024-04-25T10:02:32.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.032+0000 7f4b6ab42640 1 -- 172.21.15.19:0/3497507580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4b64072910 con 0x7f4b6415b9c0 2024-04-25T10:02:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.032+0000 7f4b597fa640 1 -- 172.21.15.19:0/3497507580 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4b5c079f50 con 0x7f4b6415b9c0 2024-04-25T10:02:32.037 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:32.037 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:02:32.037 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T10:02:32.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.037 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T10:02:32.037 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T10:02:32.037 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.037 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:02:32.038 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:02:32.038 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:02:32.038 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.038 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.038 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T10:02:32.038 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T10:02:32.038 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.038 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:02:32.038 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:02:32.039 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T10:02:32.041 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:02:32.044 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:02:32.047 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.049 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T10:02:32.051 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:02:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 -- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5006b500 msgr2=0x7f4b5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5006b500 0x7f4b5006d9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009af0 tx=0x7f4b60008040 comp rx=0 tx=0).stop 2024-04-25T10:02:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 -- 172.21.15.19:0/3497507580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6415b9c0 msgr2=0x7f4b640cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6415b9c0 0x7f4b640cbd70 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009e50 tx=0x7f4b5c004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 -- 172.21.15.19:0/3497507580 shutdown_connections 2024-04-25T10:02:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b5006b500 0x7f4b5006d9c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6415b9c0 0x7f4b640cbd70 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b64072d30 0x7f4b64158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 --2- 172.21.15.19:0/3497507580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b6406abe0 0x7f4b640ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 -- 172.21.15.19:0/3497507580 >> 172.21.15.19:0/3497507580 conn(0x7f4b640b2370 msgr2=0x7f4b64159450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 -- 172.21.15.19:0/3497507580 shutdown_connections 2024-04-25T10:02:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.034+0000 7f4b6ab42640 1 -- 172.21.15.19:0/3497507580 wait complete. 2024-04-25T10:02:32.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:32.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.150+0000 7f4684922640 1 Processor -- start 2024-04-25T10:02:32.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.150+0000 7f4684922640 1 -- start start 2024-04-25T10:02:32.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.151+0000 7f4684922640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f468015dd30 0x7f46800b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.151+0000 7f4684922640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46800b9770 0x7f46800d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.151+0000 7f4684922640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46800d2e90 0x7f46800d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.151+0000 7f4684922640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46800d1df0 con 0x7f468015dd30 2024-04-25T10:02:32.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.151+0000 7f4684922640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46800d20d0 con 0x7f46800b9770 2024-04-25T10:02:32.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.151+0000 7f4684922640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800d23b0 con 0x7f46800d2e90 2024-04-25T10:02:32.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.151+0000 7f467e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f468015dd30 0x7f46800b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.151+0000 7f467e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f468015dd30 0x7f46800b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36788/0 (socket says 172.21.15.19:36788) 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.151+0000 7f467e59c640 1 -- 172.21.15.19:0/1526200347 learned_addr learned my addr 172.21.15.19:0/1526200347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.151+0000 7f467ed9d640 1 --2- 172.21.15.19:0/1526200347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46800d2e90 0x7f46800d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.151+0000 7f467dd9b640 1 --2- 172.21.15.19:0/1526200347 >> v2:172.21.15.112:3300/0 conn(0x7f46800b9770 0x7f46800d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.152+0000 7f467e59c640 1 -- 172.21.15.19:0/1526200347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46800d2e90 msgr2=0x7f46800d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.152+0000 7f467e59c640 1 --2- 172.21.15.19:0/1526200347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46800d2e90 0x7f46800d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.152+0000 7f467e59c640 1 -- 172.21.15.19:0/1526200347 >> v2:172.21.15.112:3300/0 conn(0x7f46800b9770 msgr2=0x7f46800d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.152+0000 7f467e59c640 1 --2- 172.21.15.19:0/1526200347 >> v2:172.21.15.112:3300/0 conn(0x7f46800b9770 0x7f46800d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.152+0000 7f467e59c640 1 -- 172.21.15.19:0/1526200347 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800d2690 con 0x7f468015dd30 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.152+0000 7f467e59c640 1 --2- 172.21.15.19:0/1526200347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468015dd30 0x7f46800b9230 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f46800b6540 tx=0x7f467002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07175ec5027822a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.152+0000 7f467d59a640 1 -- 172.21.15.19:0/1526200347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670036070 con 0x7f468015dd30 2024-04-25T10:02:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.152+0000 7f467d59a640 1 -- 172.21.15.19:0/1526200347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4670031600 con 0x7f468015dd30 2024-04-25T10:02:32.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.152+0000 7f467d59a640 1 -- 172.21.15.19:0/1526200347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46700319d0 con 0x7f468015dd30 2024-04-25T10:02:32.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.153+0000 7f4684922640 1 -- 172.21.15.19:0/1526200347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468015dd30 msgr2=0x7f46800b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.153+0000 7f4684922640 1 --2- 172.21.15.19:0/1526200347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468015dd30 0x7f46800b9230 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f46800b6540 tx=0x7f467002a610 comp rx=0 tx=0).stop 2024-04-25T10:02:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.153+0000 7f4684922640 1 -- 172.21.15.19:0/1526200347 shutdown_connections 2024-04-25T10:02:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.153+0000 7f4684922640 1 --2- 172.21.15.19:0/1526200347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46800d2e90 0x7f46800d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.153+0000 7f4684922640 1 --2- 172.21.15.19:0/1526200347 >> v2:172.21.15.112:3300/0 conn(0x7f46800b9770 0x7f46800d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.153+0000 7f4684922640 1 --2- 172.21.15.19:0/1526200347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468015dd30 0x7f46800b9230 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.153+0000 7f4684922640 1 -- 172.21.15.19:0/1526200347 >> 172.21.15.19:0/1526200347 conn(0x7f4680061c30 msgr2=0x7f468006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.153+0000 7f4684922640 1 -- 172.21.15.19:0/1526200347 shutdown_connections 2024-04-25T10:02:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.153+0000 7f4684922640 1 -- 172.21.15.19:0/1526200347 wait complete. 2024-04-25T10:02:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.154+0000 7f4684922640 1 Processor -- start 2024-04-25T10:02:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.154+0000 7f4684922640 1 -- start start 2024-04-25T10:02:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.155+0000 7f4684922640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46800b9770 0x7f46800b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.155+0000 7f4684922640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46800d2e90 0x7f4680154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.155+0000 7f4684922640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f468015dd30 0x7f46801600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.155+0000 7f4684922640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800d1b90 con 0x7f46800d2e90 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.155+0000 7f4684922640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800d1a10 con 0x7f46800b9770 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.155+0000 7f4684922640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800b5430 con 0x7f468015dd30 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.155+0000 7f467dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46800d2e90 0x7f4680154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.155+0000 7f467dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46800d2e90 0x7f4680154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36800/0 (socket says 172.21.15.19:36800) 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.155+0000 7f467dd9b640 1 -- 172.21.15.19:0/849275195 learned_addr learned my addr 172.21.15.19:0/849275195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f467e59c640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46800b9770 0x7f46800b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f467ed9d640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f468015dd30 0x7f46801600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f467dd9b640 1 -- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f468015dd30 msgr2=0x7f46801600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f467dd9b640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f468015dd30 0x7f46801600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f467dd9b640 1 -- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46800b9770 msgr2=0x7f46800b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f467dd9b640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46800b9770 0x7f46800b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f467dd9b640 1 -- 172.21.15.19:0/849275195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4680160f40 con 0x7f46800d2e90 2024-04-25T10:02:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f467e59c640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46800b9770 0x7f46800b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f467ed9d640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f468015dd30 0x7f46801600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f467dd9b640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46800d2e90 0x7f4680154700 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f466800c7f0 tx=0x7f466800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f4676ffd640 1 -- 172.21.15.19:0/849275195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668015070 con 0x7f46800d2e90 2024-04-25T10:02:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f4676ffd640 1 -- 172.21.15.19:0/849275195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4668007820 con 0x7f46800d2e90 2024-04-25T10:02:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f4684922640 1 -- 172.21.15.19:0/849275195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46800b2300 con 0x7f46800d2e90 2024-04-25T10:02:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f4676ffd640 1 -- 172.21.15.19:0/849275195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668007c10 con 0x7f46800d2e90 2024-04-25T10:02:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.156+0000 7f4684922640 1 -- 172.21.15.19:0/849275195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46800c7800 con 0x7f46800d2e90 2024-04-25T10:02:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.157+0000 7f4684922640 1 -- 172.21.15.19:0/849275195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4680073990 con 0x7f46800d2e90 2024-04-25T10:02:32.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.158+0000 7f4676ffd640 1 -- 172.21.15.19:0/849275195 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4668018070 con 0x7f46800d2e90 2024-04-25T10:02:32.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.158+0000 7f4676ffd640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f466006b4d0 0x7f466006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.158+0000 7f4676ffd640 1 -- 172.21.15.19:0/849275195 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f466808a0a0 con 0x7f46800d2e90 2024-04-25T10:02:32.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.158+0000 7f467e59c640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f466006b4d0 0x7f466006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.159+0000 7f467e59c640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f466006b4d0 0x7f466006d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4670033000 tx=0x7f4670009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.162+0000 7f4676ffd640 1 -- 172.21.15.19:0/849275195 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4680160f40 con 0x7f46800d2e90 2024-04-25T10:02:32.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.344+0000 7f4684922640 1 -- 172.21.15.19:0/849275195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46800cf660 con 0x7f46800d2e90 2024-04-25T10:02:32.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.345+0000 7f4676ffd640 1 -- 172.21.15.19:0/849275195 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+13162 (secure 0 0 0) 0x7f4668059170 con 0x7f46800d2e90 2024-04-25T10:02:32.350 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:32.350 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":136,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:31.621647+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:32.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 -- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f466006b4d0 msgr2=0x7f466006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f466006b4d0 0x7f466006d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4670033000 tx=0x7f4670009000 comp rx=0 tx=0).stop 2024-04-25T10:02:32.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 -- 172.21.15.19:0/849275195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46800d2e90 msgr2=0x7f4680154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46800d2e90 0x7f4680154700 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f466800c7f0 tx=0x7f466800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 -- 172.21.15.19:0/849275195 shutdown_connections 2024-04-25T10:02:32.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f466006b4d0 0x7f466006d990 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f468015dd30 0x7f46801600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46800d2e90 0x7f4680154700 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 --2- 172.21.15.19:0/849275195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46800b9770 0x7f46800b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 -- 172.21.15.19:0/849275195 >> 172.21.15.19:0/849275195 conn(0x7f4680061c30 msgr2=0x7f46800b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 -- 172.21.15.19:0/849275195 shutdown_connections 2024-04-25T10:02:32.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.347+0000 7f4684922640 1 -- 172.21.15.19:0/849275195 wait complete. 2024-04-25T10:02:32.363 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:32.363 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.462+0000 7fc49851d640 1 Processor -- start 2024-04-25T10:02:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.462+0000 7fc49851d640 1 -- start start 2024-04-25T10:02:32.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.463+0000 7fc49851d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc49015bb50 0x7fc4900c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.463+0000 7fc49851d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4900c70d0 0x7fc4900cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.463+0000 7fc49851d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc4900cfb40 0x7fc490070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.463+0000 7fc49851d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4900c8680 con 0x7fc4900cfb40 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.463+0000 7fc49851d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4900c75e0 con 0x7fc49015bb50 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.463+0000 7fc49851d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900c78c0 con 0x7fc4900c70d0 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.463+0000 7fc496aba640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc4900cfb40 0x7fc490070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.463+0000 7fc496aba640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc4900cfb40 0x7fc490070cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36812/0 (socket says 172.21.15.19:36812) 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.463+0000 7fc496aba640 1 -- 172.21.15.19:0/3963870623 learned_addr learned my addr 172.21.15.19:0/3963870623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.463+0000 7fc4962b9640 1 --2- 172.21.15.19:0/3963870623 >> v2:172.21.15.112:3300/0 conn(0x7fc49015bb50 0x7fc4900c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.463+0000 7fc495ab8640 1 --2- 172.21.15.19:0/3963870623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4900c70d0 0x7fc4900cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.464+0000 7fc496aba640 1 -- 172.21.15.19:0/3963870623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4900c70d0 msgr2=0x7fc4900cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.464+0000 7fc496aba640 1 --2- 172.21.15.19:0/3963870623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4900c70d0 0x7fc4900cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.464+0000 7fc496aba640 1 -- 172.21.15.19:0/3963870623 >> v2:172.21.15.112:3300/0 conn(0x7fc49015bb50 msgr2=0x7fc4900c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.464+0000 7fc496aba640 1 --2- 172.21.15.19:0/3963870623 >> v2:172.21.15.112:3300/0 conn(0x7fc49015bb50 0x7fc4900c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.464+0000 7fc496aba640 1 -- 172.21.15.19:0/3963870623 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4900c7ba0 con 0x7fc4900cfb40 2024-04-25T10:02:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.464+0000 7fc496aba640 1 --2- 172.21.15.19:0/3963870623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4900cfb40 0x7fc490070cf0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc490070ab0 tx=0x7fc48002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc52e65c1bdb5eba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.464+0000 7fc4952b7640 1 -- 172.21.15.19:0/3963870623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480036070 con 0x7fc4900cfb40 2024-04-25T10:02:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.465+0000 7fc4952b7640 1 -- 172.21.15.19:0/3963870623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc480031600 con 0x7fc4900cfb40 2024-04-25T10:02:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.465+0000 7fc4952b7640 1 -- 172.21.15.19:0/3963870623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800319d0 con 0x7fc4900cfb40 2024-04-25T10:02:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.465+0000 7fc49851d640 1 -- 172.21.15.19:0/3963870623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4900cfb40 msgr2=0x7fc490070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.465+0000 7fc49851d640 1 --2- 172.21.15.19:0/3963870623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4900cfb40 0x7fc490070cf0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc490070ab0 tx=0x7fc48002a730 comp rx=0 tx=0).stop 2024-04-25T10:02:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.465+0000 7fc49851d640 1 -- 172.21.15.19:0/3963870623 shutdown_connections 2024-04-25T10:02:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.465+0000 7fc49851d640 1 --2- 172.21.15.19:0/3963870623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4900c70d0 0x7fc4900cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.465+0000 7fc49851d640 1 --2- 172.21.15.19:0/3963870623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4900cfb40 0x7fc490070cf0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.465+0000 7fc49851d640 1 --2- 172.21.15.19:0/3963870623 >> v2:172.21.15.112:3300/0 conn(0x7fc49015bb50 0x7fc4900c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.465+0000 7fc49851d640 1 -- 172.21.15.19:0/3963870623 >> 172.21.15.19:0/3963870623 conn(0x7fc49006f500 msgr2=0x7fc49006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.465+0000 7fc49851d640 1 -- 172.21.15.19:0/3963870623 shutdown_connections 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.465+0000 7fc49851d640 1 -- 172.21.15.19:0/3963870623 wait complete. 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.466+0000 7fc49851d640 1 Processor -- start 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.466+0000 7fc49851d640 1 -- start start 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc49851d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4900c70d0 0x7fc49014ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc49851d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4900cfb40 0x7fc49015e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc49851d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49015bb50 0x7fc4900ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc49851d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49015f6d0 con 0x7fc4900c70d0 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc49851d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900c8160 con 0x7fc4900cfb40 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc49851d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49015f850 con 0x7fc49015bb50 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc4962b9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4900c70d0 0x7fc49014ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc4962b9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4900c70d0 0x7fc49014ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36828/0 (socket says 172.21.15.19:36828) 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc4962b9640 1 -- 172.21.15.19:0/1975005377 learned_addr learned my addr 172.21.15.19:0/1975005377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc496aba640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49015bb50 0x7fc4900ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc495ab8640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4900cfb40 0x7fc49015e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc4962b9640 1 -- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49015bb50 msgr2=0x7fc4900ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.467+0000 7fc4962b9640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49015bb50 0x7fc4900ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.468+0000 7fc4962b9640 1 -- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4900cfb40 msgr2=0x7fc49015e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.468+0000 7fc4962b9640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4900cfb40 0x7fc49015e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.468+0000 7fc4962b9640 1 -- 172.21.15.19:0/1975005377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49014d340 con 0x7fc4900c70d0 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.468+0000 7fc4962b9640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4900c70d0 0x7fc49014ce00 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fc48800d830 tx=0x7fc48800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.468+0000 7fc48effd640 1 -- 172.21.15.19:0/1975005377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48800b6c0 con 0x7fc4900c70d0 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.468+0000 7fc48effd640 1 -- 172.21.15.19:0/1975005377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc488019070 con 0x7fc4900c70d0 2024-04-25T10:02:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.468+0000 7fc49851d640 1 -- 172.21.15.19:0/1975005377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49015d4e0 con 0x7fc4900c70d0 2024-04-25T10:02:32.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.468+0000 7fc48effd640 1 -- 172.21.15.19:0/1975005377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48800ba80 con 0x7fc4900c70d0 2024-04-25T10:02:32.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.468+0000 7fc49851d640 1 -- 172.21.15.19:0/1975005377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49015dd80 con 0x7fc4900c70d0 2024-04-25T10:02:32.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.469+0000 7fc49851d640 1 -- 172.21.15.19:0/1975005377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc490073a00 con 0x7fc4900c70d0 2024-04-25T10:02:32.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.469+0000 7fc48effd640 1 -- 172.21.15.19:0/1975005377 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc48801e460 con 0x7fc4900c70d0 2024-04-25T10:02:32.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.470+0000 7fc48effd640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 0x7fc47c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.470+0000 7fc48effd640 1 -- 172.21.15.19:0/1975005377 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc48808a580 con 0x7fc4900c70d0 2024-04-25T10:02:32.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.470+0000 7fc495ab8640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 0x7fc47c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.470+0000 7fc495ab8640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 0x7fc47c06d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc4900c7510 tx=0x7fc484008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.474+0000 7fc48effd640 1 -- 172.21.15.19:0/1975005377 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc49014d340 con 0x7fc4900c70d0 2024-04-25T10:02:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.654+0000 7fc49851d640 1 -- 172.21.15.19:0/1975005377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4900cf150 con 0x7fc4900c70d0 2024-04-25T10:02:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.654+0000 7fc48effd640 1 -- 172.21.15.19:0/1975005377 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc488058f70 con 0x7fc4900c70d0 2024-04-25T10:02:32.659 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:02:32.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 -- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 msgr2=0x7fc47c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 0x7fc47c06d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc4900c7510 tx=0x7fc484008040 comp rx=0 tx=0).stop 2024-04-25T10:02:32.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 -- 172.21.15.19:0/1975005377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4900c70d0 msgr2=0x7fc49014ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4900c70d0 0x7fc49014ce00 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fc48800d830 tx=0x7fc48800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 -- 172.21.15.19:0/1975005377 shutdown_connections 2024-04-25T10:02:32.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc47c06b4d0 0x7fc47c06d990 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc49015bb50 0x7fc4900ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4900cfb40 0x7fc49015e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 --2- 172.21.15.19:0/1975005377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4900c70d0 0x7fc49014ce00 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 -- 172.21.15.19:0/1975005377 >> 172.21.15.19:0/1975005377 conn(0x7fc49006f500 msgr2=0x7fc490070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 -- 172.21.15.19:0/1975005377 shutdown_connections 2024-04-25T10:02:32.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.656+0000 7fc49851d640 1 -- 172.21.15.19:0/1975005377 wait complete. 2024-04-25T10:02:32.671 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:02:32.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.768+0000 7faf6781f640 1 Processor -- start 2024-04-25T10:02:32.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.768+0000 7faf6781f640 1 -- start start 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf6781f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faf600d2e90 0x7faf600d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf6781f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf600d0900 0x7faf600b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf6781f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf600b9940 con 0x7faf6015dd30 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf6781f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf600d0d40 con 0x7faf600d2e90 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf6781f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf600d1df0 con 0x7faf600d0900 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf65dbc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf600d0900 0x7faf600b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf64dba640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faf600d2e90 0x7faf600d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf65dbc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf600d0900 0x7faf600b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55900/0 (socket says 172.21.15.19:55900) 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf65dbc640 1 -- 172.21.15.19:0/206068575 learned_addr learned my addr 172.21.15.19:0/206068575 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf655bb640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faf6015dd30 0x7faf600b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:45378/0 (socket says 172.21.15.19:45378) 2024-04-25T10:02:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf64dba640 1 -- 172.21.15.19:0/206068575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf600d0900 msgr2=0x7faf600b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf64dba640 1 --2- 172.21.15.19:0/206068575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf600d0900 0x7faf600b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf64dba640 1 -- 172.21.15.19:0/206068575 >> v1:172.21.15.112:6789/0 conn(0x7faf6015dd30 legacy=0x7faf600b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf64dba640 1 -- 172.21.15.19:0/206068575 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600d20d0 con 0x7faf600d2e90 2024-04-25T10:02:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.769+0000 7faf65dbc640 1 --2- 172.21.15.19:0/206068575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf600d0900 0x7faf600b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.770+0000 7faf64dba640 1 --2- 172.21.15.19:0/206068575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf600d2e90 0x7faf600d03c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faf50009790 tx=0x7faf5002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6608d93285b55101 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.770+0000 7faf57fff640 1 -- 172.21.15.19:0/206068575 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50037070 con 0x7faf600d2e90 2024-04-25T10:02:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.770+0000 7faf57fff640 1 -- 172.21.15.19:0/206068575 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5002ae50 con 0x7faf600d2e90 2024-04-25T10:02:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.770+0000 7faf57fff640 1 -- 172.21.15.19:0/206068575 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50032660 con 0x7faf600d2e90 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.771+0000 7faf6781f640 1 -- 172.21.15.19:0/206068575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf600d2e90 msgr2=0x7faf600d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.771+0000 7faf6781f640 1 --2- 172.21.15.19:0/206068575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf600d2e90 0x7faf600d03c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faf50009790 tx=0x7faf5002a100 comp rx=0 tx=0).stop 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.771+0000 7faf6781f640 1 -- 172.21.15.19:0/206068575 shutdown_connections 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.771+0000 7faf6781f640 1 --2- 172.21.15.19:0/206068575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf600d0900 0x7faf600b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.771+0000 7faf6781f640 1 --2- 172.21.15.19:0/206068575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf600d2e90 0x7faf600d03c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.771+0000 7faf6781f640 1 -- 172.21.15.19:0/206068575 >> 172.21.15.19:0/206068575 conn(0x7faf60061c30 msgr2=0x7faf6006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.771+0000 7faf6781f640 1 -- 172.21.15.19:0/206068575 shutdown_connections 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.771+0000 7faf6781f640 1 -- 172.21.15.19:0/206068575 wait complete. 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.772+0000 7faf6781f640 1 Processor -- start 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.772+0000 7faf6781f640 1 -- start start 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf6781f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf600d0900 0x7faf60149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf6781f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf600d2e90 0x7faf600caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf6781f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf6015dd30 0x7faf6015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf6781f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf600d2970 con 0x7faf600d2e90 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf6781f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf600d2690 con 0x7faf600d0900 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf6781f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf600d1a10 con 0x7faf6015dd30 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf64dba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf600d2e90 0x7faf600caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf64dba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf600d2e90 0x7faf600caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36834/0 (socket says 172.21.15.19:36834) 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf64dba640 1 -- 172.21.15.19:0/480196631 learned_addr learned my addr 172.21.15.19:0/480196631 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf655bb640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf600d0900 0x7faf60149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf65dbc640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf6015dd30 0x7faf6015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf64dba640 1 -- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf6015dd30 msgr2=0x7faf6015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf64dba640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf6015dd30 0x7faf6015f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf64dba640 1 -- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf600d0900 msgr2=0x7faf60149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf64dba640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf600d0900 0x7faf60149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.773+0000 7faf64dba640 1 -- 172.21.15.19:0/480196631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600b2fc0 con 0x7faf600d2e90 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.774+0000 7faf64dba640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf600d2e90 0x7faf600caa80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7faf50035040 tx=0x7faf500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.774+0000 7faf55ffb640 1 -- 172.21.15.19:0/480196631 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50041020 con 0x7faf600d2e90 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.774+0000 7faf6781f640 1 -- 172.21.15.19:0/480196631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf600b32a0 con 0x7faf600d2e90 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.774+0000 7faf55ffb640 1 -- 172.21.15.19:0/480196631 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf50005990 con 0x7faf600d2e90 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.774+0000 7faf55ffb640 1 -- 172.21.15.19:0/480196631 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50037040 con 0x7faf600d2e90 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.774+0000 7faf6781f640 1 -- 172.21.15.19:0/480196631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf600b3860 con 0x7faf600d2e90 2024-04-25T10:02:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.775+0000 7faf6781f640 1 -- 172.21.15.19:0/480196631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf60073990 con 0x7faf600d2e90 2024-04-25T10:02:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.775+0000 7faf55ffb640 1 -- 172.21.15.19:0/480196631 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf50044020 con 0x7faf600d2e90 2024-04-25T10:02:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.776+0000 7faf55ffb640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faf4406b500 0x7faf4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.776+0000 7faf55ffb640 1 -- 172.21.15.19:0/480196631 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5249+0+0 (secure 0 0 0) 0x7faf500a9fb0 con 0x7faf600d2e90 2024-04-25T10:02:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.776+0000 7faf655bb640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faf4406b500 0x7faf4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.776+0000 7faf655bb640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faf4406b500 0x7faf4406d9c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7faf5c000f60 tx=0x7faf5c007ce0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.780+0000 7faf55ffb640 1 -- 172.21.15.19:0/480196631 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf5004b030 con 0x7faf600d2e90 2024-04-25T10:02:32.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.963+0000 7faf6781f640 1 -- 172.21.15.19:0/480196631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faf600d4610 con 0x7faf600d2e90 2024-04-25T10:02:32.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.963+0000 7faf55ffb640 1 -- 172.21.15.19:0/480196631 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v136) v1 ==== 84+0+65 (secure 0 0 0) 0x7faf50078f60 con 0x7faf600d2e90 2024-04-25T10:02:32.968 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:32.968 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T10:02:32.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:02:32.969 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T10:02:32.969 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T10:02:32.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:02:32.969 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T10:02:32.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 -- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faf4406b500 msgr2=0x7faf4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faf4406b500 0x7faf4406d9c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7faf5c000f60 tx=0x7faf5c007ce0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 -- 172.21.15.19:0/480196631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf600d2e90 msgr2=0x7faf600caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf600d2e90 0x7faf600caa80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7faf50035040 tx=0x7faf500046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 -- 172.21.15.19:0/480196631 shutdown_connections 2024-04-25T10:02:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faf4406b500 0x7faf4406d9c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf6015dd30 0x7faf6015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf600d2e90 0x7faf600caa80 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 --2- 172.21.15.19:0/480196631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf600d0900 0x7faf60149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 -- 172.21.15.19:0/480196631 >> 172.21.15.19:0/480196631 conn(0x7faf60061c30 msgr2=0x7faf600ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 -- 172.21.15.19:0/480196631 shutdown_connections 2024-04-25T10:02:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:32.965+0000 7faf6781f640 1 -- 172.21.15.19:0/480196631 wait complete. 2024-04-25T10:02:32.981 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:02:32.981 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:02:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.081+0000 7f5b06e07640 1 Processor -- start 2024-04-25T10:02:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.081+0000 7f5b06e07640 1 -- start start 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.082+0000 7f5b06e07640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b0015b5a0 0x7f5b000b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.082+0000 7f5b06e07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015b980 0x7f5b000b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.082+0000 7f5b06e07640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b00162450 0x7f5b000ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.082+0000 7f5b06e07640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b000cf320 con 0x7f5b00162450 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.082+0000 7f5b06e07640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b000cf040 con 0x7f5b0015b5a0 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.082+0000 7f5b06e07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b00161970 con 0x7f5b0015b980 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.082+0000 7f5b053a4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b00162450 0x7f5b000ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.082+0000 7f5b053a4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b00162450 0x7f5b000ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36838/0 (socket says 172.21.15.19:36838) 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.082+0000 7f5b053a4640 1 -- 172.21.15.19:0/2674691237 learned_addr learned my addr 172.21.15.19:0/2674691237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.083+0000 7f5afffff640 1 --2- 172.21.15.19:0/2674691237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015b980 0x7f5b000b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.083+0000 7f5b04ba3640 1 --2- 172.21.15.19:0/2674691237 >> v2:172.21.15.112:3300/0 conn(0x7f5b0015b5a0 0x7f5b000b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.083+0000 7f5b053a4640 1 -- 172.21.15.19:0/2674691237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015b980 msgr2=0x7f5b000b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.083+0000 7f5b053a4640 1 --2- 172.21.15.19:0/2674691237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015b980 0x7f5b000b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.083+0000 7f5b053a4640 1 -- 172.21.15.19:0/2674691237 >> v2:172.21.15.112:3300/0 conn(0x7f5b0015b5a0 msgr2=0x7f5b000b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.083+0000 7f5b053a4640 1 --2- 172.21.15.19:0/2674691237 >> v2:172.21.15.112:3300/0 conn(0x7f5b0015b5a0 0x7f5b000b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.083+0000 7f5b053a4640 1 -- 172.21.15.19:0/2674691237 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b00161c50 con 0x7f5b00162450 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.083+0000 7f5afffff640 1 --2- 172.21.15.19:0/2674691237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015b980 0x7f5b000b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.083+0000 7f5b04ba3640 1 --2- 172.21.15.19:0/2674691237 >> v2:172.21.15.112:3300/0 conn(0x7f5b0015b5a0 0x7f5b000b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.083+0000 7f5b053a4640 1 --2- 172.21.15.19:0/2674691237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b00162450 0x7f5b000ce310 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f5b0008c0d0 tx=0x7f5af002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877c96fb6b48adb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5aff7fe640 1 -- 172.21.15.19:0/2674691237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0037070 con 0x7f5b00162450 2024-04-25T10:02:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5aff7fe640 1 -- 172.21.15.19:0/2674691237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af0032440 con 0x7f5b00162450 2024-04-25T10:02:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5aff7fe640 1 -- 172.21.15.19:0/2674691237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0032810 con 0x7f5b00162450 2024-04-25T10:02:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5b06e07640 1 -- 172.21.15.19:0/2674691237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b00162450 msgr2=0x7f5b000ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5b06e07640 1 --2- 172.21.15.19:0/2674691237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b00162450 0x7f5b000ce310 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f5b0008c0d0 tx=0x7f5af002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5b06e07640 1 -- 172.21.15.19:0/2674691237 shutdown_connections 2024-04-25T10:02:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5b06e07640 1 --2- 172.21.15.19:0/2674691237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015b980 0x7f5b000b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5b06e07640 1 --2- 172.21.15.19:0/2674691237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b00162450 0x7f5b000ce310 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5b06e07640 1 --2- 172.21.15.19:0/2674691237 >> v2:172.21.15.112:3300/0 conn(0x7f5b0015b5a0 0x7f5b000b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5b06e07640 1 -- 172.21.15.19:0/2674691237 >> 172.21.15.19:0/2674691237 conn(0x7f5b000d0530 msgr2=0x7f5b000d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5b06e07640 1 -- 172.21.15.19:0/2674691237 shutdown_connections 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.084+0000 7f5b06e07640 1 -- 172.21.15.19:0/2674691237 wait complete. 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.085+0000 7f5b06e07640 1 Processor -- start 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.085+0000 7f5b06e07640 1 -- start start 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.086+0000 7f5b06e07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b0015b5a0 0x7f5b001520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.086+0000 7f5b06e07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b0015b980 0x7f5b000b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.086+0000 7f5b06e07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b00162450 0x7f5b00159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.086+0000 7f5b06e07640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b000ce9d0 con 0x7f5b0015b5a0 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.086+0000 7f5b06e07640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b000ce850 con 0x7f5b0015b980 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.086+0000 7f5b06e07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b000ceb50 con 0x7f5b00162450 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.086+0000 7f5b04ba3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b0015b5a0 0x7f5b001520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.086+0000 7f5b04ba3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b0015b5a0 0x7f5b001520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36852/0 (socket says 172.21.15.19:36852) 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.086+0000 7f5b04ba3640 1 -- 172.21.15.19:0/750682463 learned_addr learned my addr 172.21.15.19:0/750682463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5b053a4640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b00162450 0x7f5b00159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5afffff640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b0015b980 0x7f5b000b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5b04ba3640 1 -- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b00162450 msgr2=0x7f5b00159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5b04ba3640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b00162450 0x7f5b00159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5b04ba3640 1 -- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b0015b980 msgr2=0x7f5b000b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5b04ba3640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b0015b980 0x7f5b000b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5b04ba3640 1 -- 172.21.15.19:0/750682463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b00158b90 con 0x7f5b0015b5a0 2024-04-25T10:02:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5b04ba3640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b0015b5a0 0x7f5b001520c0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5af8007e40 tx=0x7f5af800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5afd7fa640 1 -- 172.21.15.19:0/750682463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af801a070 con 0x7f5b0015b5a0 2024-04-25T10:02:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5afd7fa640 1 -- 172.21.15.19:0/750682463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af8003b90 con 0x7f5b0015b5a0 2024-04-25T10:02:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5b06e07640 1 -- 172.21.15.19:0/750682463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b00158e30 con 0x7f5b0015b5a0 2024-04-25T10:02:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5afd7fa640 1 -- 172.21.15.19:0/750682463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af800de80 con 0x7f5b0015b5a0 2024-04-25T10:02:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.087+0000 7f5b06e07640 1 -- 172.21.15.19:0/750682463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b001593f0 con 0x7f5b0015b5a0 2024-04-25T10:02:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.088+0000 7f5b06e07640 1 -- 172.21.15.19:0/750682463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b00158670 con 0x7f5b0015b5a0 2024-04-25T10:02:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.089+0000 7f5afd7fa640 1 -- 172.21.15.19:0/750682463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5af801d070 con 0x7f5b0015b5a0 2024-04-25T10:02:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.089+0000 7f5afd7fa640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5aec06b4d0 0x7f5aec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.089+0000 7f5afd7fa640 1 -- 172.21.15.19:0/750682463 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5af808a100 con 0x7f5b0015b5a0 2024-04-25T10:02:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.089+0000 7f5afffff640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5aec06b4d0 0x7f5aec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.090+0000 7f5afffff640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5aec06b4d0 0x7f5aec06d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5b000b5880 tx=0x7f5af4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.093+0000 7f5afd7fa640 1 -- 172.21.15.19:0/750682463 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b00158670 con 0x7f5b0015b5a0 2024-04-25T10:02:33.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.280+0000 7f5b06e07640 1 -- 172.21.15.19:0/750682463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5b00073a00 con 0x7f5b0015b5a0 2024-04-25T10:02:33.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.619+0000 7f5afd7fa640 1 -- 172.21.15.19:0/750682463 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v137) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5af8010070 con 0x7f5b0015b5a0 2024-04-25T10:02:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T10:02:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.621+0000 7f5b06e07640 1 -- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5aec06b4d0 msgr2=0x7f5aec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.621+0000 7f5b06e07640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5aec06b4d0 0x7f5aec06d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5b000b5880 tx=0x7f5af4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.621+0000 7f5b06e07640 1 -- 172.21.15.19:0/750682463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b0015b5a0 msgr2=0x7f5b001520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.621+0000 7f5b06e07640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b0015b5a0 0x7f5b001520c0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5af8007e40 tx=0x7f5af800d4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.621+0000 7f5b06e07640 1 -- 172.21.15.19:0/750682463 shutdown_connections 2024-04-25T10:02:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.621+0000 7f5b06e07640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5aec06b4d0 0x7f5aec06d990 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.621+0000 7f5b06e07640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b00162450 0x7f5b00159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.621+0000 7f5b06e07640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b0015b980 0x7f5b000b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.621+0000 7f5b06e07640 1 --2- 172.21.15.19:0/750682463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b0015b5a0 0x7f5b001520c0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.621+0000 7f5b06e07640 1 -- 172.21.15.19:0/750682463 >> 172.21.15.19:0/750682463 conn(0x7f5b000d0530 msgr2=0x7f5b000d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.622+0000 7f5b06e07640 1 -- 172.21.15.19:0/750682463 shutdown_connections 2024-04-25T10:02:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.622+0000 7f5b06e07640 1 -- 172.21.15.19:0/750682463 wait complete. 2024-04-25T10:02:33.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:02:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.745+0000 7f717dfb0640 1 Processor -- start 2024-04-25T10:02:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.745+0000 7f717dfb0640 1 -- start start 2024-04-25T10:02:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f717dfb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f717815b5a0 0x7f71780b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f717dfb0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f717815b980 0x7f71780b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f717dfb0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7178162450 0x7f71780ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f717dfb0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71780cf320 con 0x7f7178162450 2024-04-25T10:02:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f717dfb0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71780cf040 con 0x7f717815b980 2024-04-25T10:02:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f717dfb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7178161970 con 0x7f717815b5a0 2024-04-25T10:02:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f7177fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7178162450 0x7f71780ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f7177fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7178162450 0x7f71780ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36864/0 (socket says 172.21.15.19:36864) 2024-04-25T10:02:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f7177fff640 1 -- 172.21.15.19:0/2406072719 learned_addr learned my addr 172.21.15.19:0/2406072719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f7176ffd640 1 --2- 172.21.15.19:0/2406072719 >> v2:172.21.15.112:3300/0 conn(0x7f717815b980 0x7f71780b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f71777fe640 1 --2- 172.21.15.19:0/2406072719 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f717815b5a0 0x7f71780b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f7177fff640 1 -- 172.21.15.19:0/2406072719 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f717815b5a0 msgr2=0x7f71780b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f7177fff640 1 --2- 172.21.15.19:0/2406072719 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f717815b5a0 0x7f71780b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f7177fff640 1 -- 172.21.15.19:0/2406072719 >> v2:172.21.15.112:3300/0 conn(0x7f717815b980 msgr2=0x7f71780b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.746+0000 7f7177fff640 1 --2- 172.21.15.19:0/2406072719 >> v2:172.21.15.112:3300/0 conn(0x7f717815b980 0x7f71780b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.747+0000 7f7177fff640 1 -- 172.21.15.19:0/2406072719 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7178161c50 con 0x7f7178162450 2024-04-25T10:02:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.747+0000 7f7177fff640 1 --2- 172.21.15.19:0/2406072719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7178162450 0x7f71780ce310 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f717806d330 tx=0x7f717002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91af45864d404d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.747+0000 7f71767fc640 1 -- 172.21.15.19:0/2406072719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170036070 con 0x7f7178162450 2024-04-25T10:02:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.747+0000 7f71767fc640 1 -- 172.21.15.19:0/2406072719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7170031600 con 0x7f7178162450 2024-04-25T10:02:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.747+0000 7f71767fc640 1 -- 172.21.15.19:0/2406072719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71700319d0 con 0x7f7178162450 2024-04-25T10:02:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.748+0000 7f717dfb0640 1 -- 172.21.15.19:0/2406072719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7178162450 msgr2=0x7f71780ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.748+0000 7f717dfb0640 1 --2- 172.21.15.19:0/2406072719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7178162450 0x7f71780ce310 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f717806d330 tx=0x7f717002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.748+0000 7f717dfb0640 1 -- 172.21.15.19:0/2406072719 shutdown_connections 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.748+0000 7f717dfb0640 1 --2- 172.21.15.19:0/2406072719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7178162450 0x7f71780ce310 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.748+0000 7f717dfb0640 1 --2- 172.21.15.19:0/2406072719 >> v2:172.21.15.112:3300/0 conn(0x7f717815b980 0x7f71780b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.748+0000 7f717dfb0640 1 --2- 172.21.15.19:0/2406072719 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f717815b5a0 0x7f71780b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.748+0000 7f717dfb0640 1 -- 172.21.15.19:0/2406072719 >> 172.21.15.19:0/2406072719 conn(0x7f71780d0530 msgr2=0x7f71780d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.748+0000 7f717dfb0640 1 -- 172.21.15.19:0/2406072719 shutdown_connections 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.748+0000 7f717dfb0640 1 -- 172.21.15.19:0/2406072719 wait complete. 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.749+0000 7f717dfb0640 1 Processor -- start 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.749+0000 7f717dfb0640 1 -- start start 2024-04-25T10:02:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.750+0000 7f717dfb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717815b5a0 0x7f71781579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.750+0000 7f717dfb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f717815b980 0x7f7178146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.750+0000 7f717dfb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7178162450 0x7f717815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.750+0000 7f717dfb0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71780ce9d0 con 0x7f717815b5a0 2024-04-25T10:02:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.750+0000 7f717dfb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71780ce850 con 0x7f7178162450 2024-04-25T10:02:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.750+0000 7f717dfb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71780ceb50 con 0x7f717815b980 2024-04-25T10:02:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.750+0000 7f71777fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717815b5a0 0x7f71781579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.750+0000 7f71777fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717815b5a0 0x7f71781579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36872/0 (socket says 172.21.15.19:36872) 2024-04-25T10:02:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.750+0000 7f71777fe640 1 -- 172.21.15.19:0/3239739839 learned_addr learned my addr 172.21.15.19:0/3239739839 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.750+0000 7f7177fff640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7178162450 0x7f717815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f7176ffd640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f717815b980 0x7f7178146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f71777fe640 1 -- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f717815b980 msgr2=0x7f7178146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f71777fe640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f717815b980 0x7f7178146b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f71777fe640 1 -- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7178162450 msgr2=0x7f717815a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f71777fe640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7178162450 0x7f717815a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f71777fe640 1 -- 172.21.15.19:0/3239739839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71780c8900 con 0x7f717815b5a0 2024-04-25T10:02:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f71777fe640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717815b5a0 0x7f71781579a0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f716800e8a0 tx=0x7f716800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f715ffff640 1 -- 172.21.15.19:0/3239739839 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7168017070 con 0x7f717815b5a0 2024-04-25T10:02:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f715ffff640 1 -- 172.21.15.19:0/3239739839 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716800ee90 con 0x7f717815b5a0 2024-04-25T10:02:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f717dfb0640 1 -- 172.21.15.19:0/3239739839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71780c8be0 con 0x7f717815b5a0 2024-04-25T10:02:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f715ffff640 1 -- 172.21.15.19:0/3239739839 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716800cb80 con 0x7f717815b5a0 2024-04-25T10:02:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.751+0000 7f717dfb0640 1 -- 172.21.15.19:0/3239739839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7178157340 con 0x7f717815b5a0 2024-04-25T10:02:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.752+0000 7f717dfb0640 1 -- 172.21.15.19:0/3239739839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7178073a00 con 0x7f717815b5a0 2024-04-25T10:02:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.753+0000 7f715ffff640 1 -- 172.21.15.19:0/3239739839 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f716801a070 con 0x7f717815b5a0 2024-04-25T10:02:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.753+0000 7f715ffff640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f716406b4d0 0x7f716406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.753+0000 7f715ffff640 1 -- 172.21.15.19:0/3239739839 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f7168089f80 con 0x7f717815b5a0 2024-04-25T10:02:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.753+0000 7f7176ffd640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f716406b4d0 0x7f716406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.754+0000 7f7176ffd640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f716406b4d0 0x7f716406d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f71781f6f40 tx=0x7f716c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.757+0000 7f715ffff640 1 -- 172.21.15.19:0/3239739839 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7168058ee0 con 0x7f717815b5a0 2024-04-25T10:02:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:33.939+0000 7f717dfb0640 1 -- 172.21.15.19:0/3239739839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f71780d17c0 con 0x7f717815b5a0 2024-04-25T10:02:34.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.620+0000 7f715ffff640 1 -- 172.21.15.19:0/3239739839 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v138) v1 ==== 170+0+0 (secure 0 0 0) 0x7f71680587e0 con 0x7f717815b5a0 2024-04-25T10:02:34.626 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T10:02:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.623+0000 7f717dfb0640 1 -- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f716406b4d0 msgr2=0x7f716406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.623+0000 7f717dfb0640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f716406b4d0 0x7f716406d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f71781f6f40 tx=0x7f716c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.623+0000 7f717dfb0640 1 -- 172.21.15.19:0/3239739839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717815b5a0 msgr2=0x7f71781579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.623+0000 7f717dfb0640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717815b5a0 0x7f71781579a0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f716800e8a0 tx=0x7f716800ed60 comp rx=0 tx=0).stop 2024-04-25T10:02:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.624+0000 7f717dfb0640 1 -- 172.21.15.19:0/3239739839 shutdown_connections 2024-04-25T10:02:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.624+0000 7f717dfb0640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f716406b4d0 0x7f716406d990 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.624+0000 7f717dfb0640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7178162450 0x7f717815a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.624+0000 7f717dfb0640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f717815b980 0x7f7178146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.624+0000 7f717dfb0640 1 --2- 172.21.15.19:0/3239739839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f717815b5a0 0x7f71781579a0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.624+0000 7f717dfb0640 1 -- 172.21.15.19:0/3239739839 >> 172.21.15.19:0/3239739839 conn(0x7f71780d0530 msgr2=0x7f71780b7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.624+0000 7f717dfb0640 1 -- 172.21.15.19:0/3239739839 shutdown_connections 2024-04-25T10:02:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.624+0000 7f717dfb0640 1 -- 172.21.15.19:0/3239739839 wait complete. 2024-04-25T10:02:34.658 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:02:34.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.761+0000 7fd4da1b0640 1 Processor -- start 2024-04-25T10:02:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.761+0000 7fd4da1b0640 1 -- start start 2024-04-25T10:02:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.761+0000 7fd4da1b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4d415dd80 0x7fd4d40b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.761+0000 7fd4da1b0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd4d40ad570 0x7fd4d406ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.761+0000 7fd4da1b0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d40b3930 con 0x7fd4d406abe0 2024-04-25T10:02:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.761+0000 7fd4da1b0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d406e410 con 0x7fd4d40ad570 2024-04-25T10:02:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.761+0000 7fd4da1b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40b3c10 con 0x7fd4d415dd80 2024-04-25T10:02:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.762+0000 7fd4d3fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd4d40ad570 0x7fd4d406ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.762+0000 7fd4d3fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd4d40ad570 0x7fd4d406ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36882/0 (socket says 172.21.15.19:36882) 2024-04-25T10:02:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.762+0000 7fd4d3fff640 1 -- 172.21.15.19:0/4046319778 learned_addr learned my addr 172.21.15.19:0/4046319778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.762+0000 7fd4d2ffd640 1 --2- 172.21.15.19:0/4046319778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4d415dd80 0x7fd4d40b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.762+0000 7fd4d3fff640 1 -- 172.21.15.19:0/4046319778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4d415dd80 msgr2=0x7fd4d40b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.762+0000 7fd4d3fff640 1 --2- 172.21.15.19:0/4046319778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4d415dd80 0x7fd4d40b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.762+0000 7fd4d3fff640 1 -- 172.21.15.19:0/4046319778 >> v1:172.21.15.112:6789/0 conn(0x7fd4d406abe0 legacy=0x7fd4d406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.762+0000 7fd4d3fff640 1 -- 172.21.15.19:0/4046319778 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d406e660 con 0x7fd4d40ad570 2024-04-25T10:02:34.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.763+0000 7fd4d3fff640 1 --2- 172.21.15.19:0/4046319778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d40ad570 0x7fd4d406ee60 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009e50 tx=0x7fd4cc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbc7252b2fa4591 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.763+0000 7fd4d27fc640 1 -- 172.21.15.19:0/4046319778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc036070 con 0x7fd4d40ad570 2024-04-25T10:02:34.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.763+0000 7fd4d27fc640 1 -- 172.21.15.19:0/4046319778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc031600 con 0x7fd4d40ad570 2024-04-25T10:02:34.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.763+0000 7fd4d27fc640 1 -- 172.21.15.19:0/4046319778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0319d0 con 0x7fd4d40ad570 2024-04-25T10:02:34.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.763+0000 7fd4da1b0640 1 -- 172.21.15.19:0/4046319778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d40ad570 msgr2=0x7fd4d406ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.763+0000 7fd4da1b0640 1 --2- 172.21.15.19:0/4046319778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d40ad570 0x7fd4d406ee60 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009e50 tx=0x7fd4cc02a710 comp rx=0 tx=0).stop 2024-04-25T10:02:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.764+0000 7fd4da1b0640 1 -- 172.21.15.19:0/4046319778 shutdown_connections 2024-04-25T10:02:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.764+0000 7fd4da1b0640 1 --2- 172.21.15.19:0/4046319778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d40ad570 0x7fd4d406ee60 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.764+0000 7fd4da1b0640 1 --2- 172.21.15.19:0/4046319778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4d415dd80 0x7fd4d40b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.764+0000 7fd4da1b0640 1 -- 172.21.15.19:0/4046319778 >> 172.21.15.19:0/4046319778 conn(0x7fd4d40b5af0 msgr2=0x7fd4d40b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.764+0000 7fd4da1b0640 1 -- 172.21.15.19:0/4046319778 shutdown_connections 2024-04-25T10:02:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.764+0000 7fd4da1b0640 1 -- 172.21.15.19:0/4046319778 wait complete. 2024-04-25T10:02:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.765+0000 7fd4da1b0640 1 Processor -- start 2024-04-25T10:02:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.765+0000 7fd4da1b0640 1 -- start start 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4da1b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d406abe0 0x7fd4d4156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4da1b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d40ad570 0x7fd4d4158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4da1b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4d415dd80 0x7fd4d41f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4da1b0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4070280 con 0x7fd4d40ad570 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4da1b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40b57d0 con 0x7fd4d406abe0 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4da1b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4070400 con 0x7fd4d415dd80 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d40ad570 0x7fd4d4158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4d2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d40ad570 0x7fd4d4158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36884/0 (socket says 172.21.15.19:36884) 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4d2ffd640 1 -- 172.21.15.19:0/2728567104 learned_addr learned my addr 172.21.15.19:0/2728567104 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4d37fe640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d406abe0 0x7fd4d4156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4d2ffd640 1 -- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4d415dd80 msgr2=0x7fd4d41f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4d2ffd640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4d415dd80 0x7fd4d41f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4d2ffd640 1 -- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d406abe0 msgr2=0x7fd4d4156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4d2ffd640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d406abe0 0x7fd4d4156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4d2ffd640 1 -- 172.21.15.19:0/2728567104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d41e9eb0 con 0x7fd4d40ad570 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4d37fe640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d406abe0 0x7fd4d4156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4d2ffd640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d40ad570 0x7fd4d4158f70 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800d5c0 tx=0x7fd4c800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4bbfff640 1 -- 172.21.15.19:0/2728567104 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80125f0 con 0x7fd4d40ad570 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4bbfff640 1 -- 172.21.15.19:0/2728567104 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c8019070 con 0x7fd4d40ad570 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4bbfff640 1 -- 172.21.15.19:0/2728567104 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80129d0 con 0x7fd4d40ad570 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4da1b0640 1 -- 172.21.15.19:0/2728567104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d41e6b70 con 0x7fd4d40ad570 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.766+0000 7fd4da1b0640 1 -- 172.21.15.19:0/2728567104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d41e7130 con 0x7fd4d40ad570 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.767+0000 7fd4da1b0640 1 -- 172.21.15.19:0/2728567104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d4073a00 con 0x7fd4d40ad570 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.768+0000 7fd4bbfff640 1 -- 172.21.15.19:0/2728567104 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4c8012b70 con 0x7fd4d40ad570 2024-04-25T10:02:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.768+0000 7fd4bbfff640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4c006b4d0 0x7fd4c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.768+0000 7fd4bbfff640 1 -- 172.21.15.19:0/2728567104 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd4c808a640 con 0x7fd4d40ad570 2024-04-25T10:02:34.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.769+0000 7fd4d37fe640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4c006b4d0 0x7fd4c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.769+0000 7fd4d37fe640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4c006b4d0 0x7fd4c006d990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4001680 tx=0x7fd4c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.773+0000 7fd4bbfff640 1 -- 172.21.15.19:0/2728567104 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4c8059420 con 0x7fd4d40ad570 2024-04-25T10:02:34.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:34.989+0000 7fd4da1b0640 1 -- 172.21.15.19:0/2728567104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd4d40ae3e0 con 0x7fd4d40ad570 2024-04-25T10:02:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.650+0000 7fd4bbfff640 1 -- 172.21.15.19:0/2728567104 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v88) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd4c8058d20 con 0x7fd4d40ad570 2024-04-25T10:02:35.657 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 21 and data pool 22 2024-04-25T10:02:35.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.661+0000 7fd4da1b0640 1 -- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4c006b4d0 msgr2=0x7fd4c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.661+0000 7fd4da1b0640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4c006b4d0 0x7fd4c006d990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4001680 tx=0x7fd4c4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:35.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.661+0000 7fd4da1b0640 1 -- 172.21.15.19:0/2728567104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d40ad570 msgr2=0x7fd4d4158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.661+0000 7fd4da1b0640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d40ad570 0x7fd4d4158f70 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800d5c0 tx=0x7fd4c800da80 comp rx=0 tx=0).stop 2024-04-25T10:02:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.662+0000 7fd4da1b0640 1 -- 172.21.15.19:0/2728567104 shutdown_connections 2024-04-25T10:02:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.662+0000 7fd4da1b0640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4c006b4d0 0x7fd4c006d990 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.662+0000 7fd4da1b0640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4d415dd80 0x7fd4d41f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.662+0000 7fd4da1b0640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d40ad570 0x7fd4d4158f70 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.662+0000 7fd4da1b0640 1 --2- 172.21.15.19:0/2728567104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d406abe0 0x7fd4d4156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.662+0000 7fd4da1b0640 1 -- 172.21.15.19:0/2728567104 >> 172.21.15.19:0/2728567104 conn(0x7fd4d40b5af0 msgr2=0x7fd4d40af950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.663+0000 7fd4da1b0640 1 -- 172.21.15.19:0/2728567104 shutdown_connections 2024-04-25T10:02:35.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.663+0000 7fd4da1b0640 1 -- 172.21.15.19:0/2728567104 wait complete. 2024-04-25T10:02:35.687 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:35.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.787+0000 7f71a6cc4640 1 Processor -- start 2024-04-25T10:02:35.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.787+0000 7f71a6cc4640 1 -- start start 2024-04-25T10:02:35.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.788+0000 7f71a6cc4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71a00cfc90 0x7f71a00cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.788+0000 7f71a6cc4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a00cf330 con 0x7f71a00aecf0 2024-04-25T10:02:35.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.788+0000 7f71a6cc4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a0071d80 con 0x7f71a0165680 2024-04-25T10:02:35.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.788+0000 7f71a6cc4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a0072340 con 0x7f71a00cfc90 2024-04-25T10:02:35.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.788+0000 7f719ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f71a00aecf0 0x7f71a00cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:32816/0 (socket says 172.21.15.19:32816) 2024-04-25T10:02:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.788+0000 7f719ffff640 1 -- 172.21.15.19:0/1203612919 learned_addr learned my addr 172.21.15.19:0/1203612919 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.788+0000 7f71a5261640 1 --2- 172.21.15.19:0/1203612919 >> v2:172.21.15.112:3300/0 conn(0x7f71a00cfc90 0x7f71a00cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.788+0000 7f719f7fe640 1 -- 172.21.15.19:0/1203612919 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379827809 0 0) 0x7f71a00cf330 con 0x7f71a00aecf0 2024-04-25T10:02:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.788+0000 7f719f7fe640 1 -- 172.21.15.19:0/1203612919 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f718c003520 con 0x7f71a00aecf0 2024-04-25T10:02:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.789+0000 7f71a5261640 1 -- 172.21.15.19:0/1203612919 >> v1:172.21.15.112:6789/0 conn(0x7f71a0165680 legacy=0x7f71a00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.789+0000 7f719f7fe640 1 -- 172.21.15.19:0/1203612919 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 541072591 0 0) 0x7f718c003520 con 0x7f71a00aecf0 2024-04-25T10:02:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.789+0000 7f71a5261640 1 -- 172.21.15.19:0/1203612919 >> v1:172.21.15.19:6789/0 conn(0x7f71a00aecf0 legacy=0x7f71a00cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.789+0000 7f71a5261640 1 -- 172.21.15.19:0/1203612919 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0072060 con 0x7f71a00cfc90 2024-04-25T10:02:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.789+0000 7f71a5261640 1 --2- 172.21.15.19:0/1203612919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71a00cfc90 0x7f71a00cec70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f71880097a0 tx=0x7f718802a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1649f2d248a240dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.790+0000 7f719f7fe640 1 -- 172.21.15.19:0/1203612919 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188037070 con 0x7f71a00cfc90 2024-04-25T10:02:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.790+0000 7f719f7fe640 1 -- 172.21.15.19:0/1203612919 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718802aad0 con 0x7f71a00cfc90 2024-04-25T10:02:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.790+0000 7f719f7fe640 1 -- 172.21.15.19:0/1203612919 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7190001670 con 0x7f71a00cfc90 2024-04-25T10:02:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.790+0000 7f71a6cc4640 1 -- 172.21.15.19:0/1203612919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71a00cfc90 msgr2=0x7f71a00cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.790+0000 7f71a6cc4640 1 --2- 172.21.15.19:0/1203612919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71a00cfc90 0x7f71a00cec70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f71880097a0 tx=0x7f718802a040 comp rx=0 tx=0).stop 2024-04-25T10:02:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.790+0000 7f71a6cc4640 1 -- 172.21.15.19:0/1203612919 shutdown_connections 2024-04-25T10:02:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.790+0000 7f71a6cc4640 1 --2- 172.21.15.19:0/1203612919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71a00cfc90 0x7f71a00cec70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.790+0000 7f71a6cc4640 1 -- 172.21.15.19:0/1203612919 >> 172.21.15.19:0/1203612919 conn(0x7f71a006abe0 msgr2=0x7f71a006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.790+0000 7f71a6cc4640 1 -- 172.21.15.19:0/1203612919 shutdown_connections 2024-04-25T10:02:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.790+0000 7f71a6cc4640 1 -- 172.21.15.19:0/1203612919 wait complete. 2024-04-25T10:02:35.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.791+0000 7f71a6cc4640 1 Processor -- start 2024-04-25T10:02:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.791+0000 7f71a6cc4640 1 -- start start 2024-04-25T10:02:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.792+0000 7f71a6cc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a00aecf0 0x7f71a01565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.792+0000 7f71a6cc4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71a00cfc90 0x7f71a01f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.792+0000 7f71a6cc4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71a0165680 0x7f71a01f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.792+0000 7f71a6cc4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a0165a60 con 0x7f71a00aecf0 2024-04-25T10:02:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.792+0000 7f71a6cc4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a0071670 con 0x7f71a00cfc90 2024-04-25T10:02:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.792+0000 7f71a6cc4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00ccdd0 con 0x7f71a0165680 2024-04-25T10:02:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.792+0000 7f71a4a60640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a00aecf0 0x7f71a01565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.792+0000 7f71a4a60640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a00aecf0 0x7f71a01565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36892/0 (socket says 172.21.15.19:36892) 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.792+0000 7f71a4a60640 1 -- 172.21.15.19:0/316626905 learned_addr learned my addr 172.21.15.19:0/316626905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f719ffff640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71a00cfc90 0x7f71a01f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f71a4a60640 1 -- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71a0165680 msgr2=0x7f71a01f8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f71a5261640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71a0165680 0x7f71a01f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f71a4a60640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71a0165680 0x7f71a01f8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f71a4a60640 1 -- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71a00cfc90 msgr2=0x7f71a01f7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f71a4a60640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71a00cfc90 0x7f71a01f7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f71a4a60640 1 -- 172.21.15.19:0/316626905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a01f7da0 con 0x7f71a00aecf0 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f71a5261640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71a0165680 0x7f71a01f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f719ffff640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71a00cfc90 0x7f71a01f7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f71a4a60640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a00aecf0 0x7f71a01565d0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f719800cb20 tx=0x7f7198002110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f719d7fa640 1 -- 172.21.15.19:0/316626905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198018070 con 0x7f71a00aecf0 2024-04-25T10:02:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f719d7fa640 1 -- 172.21.15.19:0/316626905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7198003da0 con 0x7f71a00aecf0 2024-04-25T10:02:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f71a6cc4640 1 -- 172.21.15.19:0/316626905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a01f8020 con 0x7f71a00aecf0 2024-04-25T10:02:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f719d7fa640 1 -- 172.21.15.19:0/316626905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198006d50 con 0x7f71a00aecf0 2024-04-25T10:02:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.793+0000 7f71a6cc4640 1 -- 172.21.15.19:0/316626905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a01f85e0 con 0x7f71a00aecf0 2024-04-25T10:02:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.794+0000 7f71a6cc4640 1 -- 172.21.15.19:0/316626905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a0073990 con 0x7f71a00aecf0 2024-04-25T10:02:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.795+0000 7f719d7fa640 1 -- 172.21.15.19:0/316626905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f719801b070 con 0x7f71a00aecf0 2024-04-25T10:02:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.795+0000 7f719d7fa640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f718006b4d0 0x7f718006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.795+0000 7f719d7fa640 1 -- 172.21.15.19:0/316626905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7198089b80 con 0x7f71a00aecf0 2024-04-25T10:02:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.795+0000 7f719ffff640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f718006b4d0 0x7f718006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.796+0000 7f719ffff640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f718006b4d0 0x7f718006d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f71a01fadb0 tx=0x7f7190001bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.799+0000 7f719d7fa640 1 -- 172.21.15.19:0/316626905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71a01f7da0 con 0x7f71a00aecf0 2024-04-25T10:02:35.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.987+0000 7f71a6cc4640 1 -- 172.21.15.19:0/316626905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71a01f87e0 con 0x7f71a00aecf0 2024-04-25T10:02:35.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.988+0000 7f719d7fa640 1 -- 172.21.15.19:0/316626905 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f71980587d0 con 0x7f71a00aecf0 2024-04-25T10:02:35.994 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:35.994 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":139,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:34.991765+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:33.312366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:33.971764+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:35.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 -- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f718006b4d0 msgr2=0x7f718006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f718006b4d0 0x7f718006d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f71a01fadb0 tx=0x7f7190001bc0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 -- 172.21.15.19:0/316626905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a00aecf0 msgr2=0x7f71a01565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a00aecf0 0x7f71a01565d0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f719800cb20 tx=0x7f7198002110 comp rx=0 tx=0).stop 2024-04-25T10:02:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 -- 172.21.15.19:0/316626905 shutdown_connections 2024-04-25T10:02:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f718006b4d0 0x7f718006d990 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71a0165680 0x7f71a01f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71a00cfc90 0x7f71a01f7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 --2- 172.21.15.19:0/316626905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a00aecf0 0x7f71a01565d0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 -- 172.21.15.19:0/316626905 >> 172.21.15.19:0/316626905 conn(0x7f71a006abe0 msgr2=0x7f71a00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 -- 172.21.15.19:0/316626905 shutdown_connections 2024-04-25T10:02:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:35.990+0000 7f71a6cc4640 1 -- 172.21.15.19:0/316626905 wait complete. 2024-04-25T10:02:36.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:36.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.102+0000 7f46df6dd640 1 Processor -- start 2024-04-25T10:02:36.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.103+0000 7f46df6dd640 1 -- start start 2024-04-25T10:02:36.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.103+0000 7f46df6dd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46d815bf50 0x7f46d80d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.103+0000 7f46df6dd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815c400 0x7f46d80d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.103+0000 7f46df6dd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d815c930 con 0x7f46d81594f0 2024-04-25T10:02:36.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.103+0000 7f46df6dd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d80d12d0 con 0x7f46d815bf50 2024-04-25T10:02:36.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.103+0000 7f46df6dd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d81598d0 con 0x7f46d815c400 2024-04-25T10:02:36.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.103+0000 7f46dd479640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f46d81594f0 0x7f46d80b50f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:32830/0 (socket says 172.21.15.19:32830) 2024-04-25T10:02:36.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46dd479640 1 -- 172.21.15.19:0/627094613 learned_addr learned my addr 172.21.15.19:0/627094613 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46dcc78640 1 --2- 172.21.15.19:0/627094613 >> v2:172.21.15.112:3300/0 conn(0x7f46d815bf50 0x7f46d80d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46ddc7a640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815c400 0x7f46d80d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136062464 0 0) 0x7f46d815c930 con 0x7f46d81594f0 2024-04-25T10:02:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c4003520 con 0x7f46d81594f0 2024-04-25T10:02:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1556670453 0 0) 0x7f46c4003520 con 0x7f46d81594f0 2024-04-25T10:02:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815c400 msgr2=0x7f46d80d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815c400 0x7f46d80d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 >> v2:172.21.15.112:3300/0 conn(0x7f46d815bf50 msgr2=0x7f46d80d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 --2- 172.21.15.19:0/627094613 >> v2:172.21.15.112:3300/0 conn(0x7f46d815bf50 0x7f46d80d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d815cc10 con 0x7f46d81594f0 2024-04-25T10:02:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f46d4001670 con 0x7f46d81594f0 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 >> v1:172.21.15.19:6789/0 conn(0x7f46d81594f0 legacy=0x7f46d80b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c40039b0 0x7f46c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c4004800 0x7f46c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815c400 0x7f46c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d81598d0 con 0x7f46c4004800 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c400b9c0 con 0x7f46c40039b0 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c400bb40 con 0x7f46d815c400 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46dcc78640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c4004800 0x7f46c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46dd479640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c40039b0 0x7f46c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.104+0000 7f46ddc7a640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815c400 0x7f46c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.105+0000 7f46dcc78640 1 -- 172.21.15.19:0/627094613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815c400 msgr2=0x7f46c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.105+0000 7f46dcc78640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d815c400 0x7f46c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.105+0000 7f46dcc78640 1 -- 172.21.15.19:0/627094613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c40039b0 msgr2=0x7f46c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.105+0000 7f46dcc78640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c40039b0 0x7f46c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.105+0000 7f46dcc78640 1 -- 172.21.15.19:0/627094613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c4003520 con 0x7f46c4004800 2024-04-25T10:02:36.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.105+0000 7f46dcc78640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c4004800 0x7f46c4004c80 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009af0 tx=0x7f46c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d99d79187c8870b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.105+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8036070 con 0x7f46c4004800 2024-04-25T10:02:36.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.105+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c8031400 con 0x7f46c4004800 2024-04-25T10:02:36.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.105+0000 7f46cffff640 1 -- 172.21.15.19:0/627094613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8009690 con 0x7f46c4004800 2024-04-25T10:02:36.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.106+0000 7f46df6dd640 1 -- 172.21.15.19:0/627094613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c4004800 msgr2=0x7f46c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.106+0000 7f46df6dd640 1 --2- 172.21.15.19:0/627094613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c4004800 0x7f46c4004c80 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009af0 tx=0x7f46c4005830 comp rx=0 tx=0).stop 2024-04-25T10:02:36.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.106+0000 7f46ddc7a640 1 -- 172.21.15.19:0/627094613 reap_dead start 2024-04-25T10:02:36.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.106+0000 7f46df6dd640 1 -- 172.21.15.19:0/627094613 shutdown_connections 2024-04-25T10:02:36.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.106+0000 7f46df6dd640 1 -- 172.21.15.19:0/627094613 >> 172.21.15.19:0/627094613 conn(0x7f46d806f520 msgr2=0x7f46d806f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.106+0000 7f46df6dd640 1 -- 172.21.15.19:0/627094613 shutdown_connections 2024-04-25T10:02:36.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.106+0000 7f46df6dd640 1 -- 172.21.15.19:0/627094613 wait complete. 2024-04-25T10:02:36.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.107+0000 7f46df6dd640 1 Processor -- start 2024-04-25T10:02:36.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.107+0000 7f46df6dd640 1 -- start start 2024-04-25T10:02:36.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46df6dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c4004800 0x7f46d8155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46df6dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8157680 0x7f46d81f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46df6dd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d81f17e0 0x7f46d81f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46df6dd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d8070e10 con 0x7f46d8157680 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46df6dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80b4c60 con 0x7f46c4004800 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46df6dd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d8070f90 con 0x7f46d81f17e0 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46dcc78640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8157680 0x7f46d81f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46dcc78640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8157680 0x7f46d81f45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36908/0 (socket says 172.21.15.19:36908) 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46dcc78640 1 -- 172.21.15.19:0/1070582678 learned_addr learned my addr 172.21.15.19:0/1070582678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46dd479640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c4004800 0x7f46d8155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46ddc7a640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d81f17e0 0x7f46d81f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46dcc78640 1 -- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d81f17e0 msgr2=0x7f46d81f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46dcc78640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d81f17e0 0x7f46d81f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46dcc78640 1 -- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c4004800 msgr2=0x7f46d8155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46dcc78640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c4004800 0x7f46d8155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46dcc78640 1 -- 172.21.15.19:0/1070582678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d81f4b00 con 0x7f46d8157680 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.108+0000 7f46ddc7a640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d81f17e0 0x7f46d81f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.109+0000 7f46dcc78640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8157680 0x7f46d81f45c0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009af0 tx=0x7f46c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.109+0000 7f46cdffb640 1 -- 172.21.15.19:0/1070582678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8041020 con 0x7f46d8157680 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.109+0000 7f46cdffb640 1 -- 172.21.15.19:0/1070582678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c802ade0 con 0x7f46d8157680 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.109+0000 7f46df6dd640 1 -- 172.21.15.19:0/1070582678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d8155fc0 con 0x7f46d8157680 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.109+0000 7f46cdffb640 1 -- 172.21.15.19:0/1070582678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8036040 con 0x7f46d8157680 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.109+0000 7f46df6dd640 1 -- 172.21.15.19:0/1070582678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d8156580 con 0x7f46d8157680 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.110+0000 7f46df6dd640 1 -- 172.21.15.19:0/1070582678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d8073990 con 0x7f46d8157680 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.110+0000 7f46cdffb640 1 -- 172.21.15.19:0/1070582678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46c8044020 con 0x7f46d8157680 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.111+0000 7f46cdffb640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b500 0x7f46bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.111+0000 7f46cdffb640 1 -- 172.21.15.19:0/1070582678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f46c80aa8d0 con 0x7f46d8157680 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.111+0000 7f46dd479640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b500 0x7f46bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.112+0000 7f46dd479640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b500 0x7f46bc06d9c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f46d40017e0 tx=0x7f46d40015b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.115+0000 7f46cdffb640 1 -- 172.21.15.19:0/1070582678 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46c804b030 con 0x7f46d8157680 2024-04-25T10:02:36.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.298+0000 7f46df6dd640 1 -- 172.21.15.19:0/1070582678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46d80700e0 con 0x7f46d8157680 2024-04-25T10:02:36.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.299+0000 7f46cdffb640 1 -- 172.21.15.19:0/1070582678 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f46c8079520 con 0x7f46d8157680 2024-04-25T10:02:36.305 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:36.305 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":139,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:34.991765+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:33.312366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:33.971764+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:36.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 -- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b500 msgr2=0x7f46bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b500 0x7f46bc06d9c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f46d40017e0 tx=0x7f46d40015b0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 -- 172.21.15.19:0/1070582678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8157680 msgr2=0x7f46d81f45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8157680 0x7f46d81f45c0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009af0 tx=0x7f46c8004730 comp rx=0 tx=0).stop 2024-04-25T10:02:36.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 -- 172.21.15.19:0/1070582678 shutdown_connections 2024-04-25T10:02:36.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f46bc06b500 0x7f46bc06d9c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46d81f17e0 0x7f46d81f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d8157680 0x7f46d81f45c0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 --2- 172.21.15.19:0/1070582678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46c4004800 0x7f46d8155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 -- 172.21.15.19:0/1070582678 >> 172.21.15.19:0/1070582678 conn(0x7f46d806f520 msgr2=0x7f46d80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 -- 172.21.15.19:0/1070582678 shutdown_connections 2024-04-25T10:02:36.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.301+0000 7f46df6dd640 1 -- 172.21.15.19:0/1070582678 wait complete. 2024-04-25T10:02:36.318 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:02:36.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.417+0000 7fb3f871d640 1 Processor -- start 2024-04-25T10:02:36.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.417+0000 7fb3f871d640 1 -- start start 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f871d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01610f0 0x7fb3f006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f871d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb3f01616f0 0x7fb3f0161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f871d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f00c6960 con 0x7fb3f00ce0a0 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f871d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f01614d0 con 0x7fb3f01616f0 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f871d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0162260 con 0x7fb3f01610f0 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f64b9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01610f0 0x7fb3f006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f5cb8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb3f01616f0 0x7fb3f0161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f64b9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01610f0 0x7fb3f006c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55982/0 (socket says 172.21.15.19:55982) 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f64b9640 1 -- 172.21.15.19:0/4043800184 learned_addr learned my addr 172.21.15.19:0/4043800184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f5cb8640 1 --2- 172.21.15.19:0/4043800184 >> v2:172.21.15.112:3300/0 conn(0x7fb3f01616f0 0x7fb3f0161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:33488/0 (socket says 172.21.15.19:33488) 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f64b9640 1 -- 172.21.15.19:0/4043800184 >> v2:172.21.15.112:3300/0 conn(0x7fb3f01616f0 msgr2=0x7fb3f0161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f64b9640 1 --2- 172.21.15.19:0/4043800184 >> v2:172.21.15.112:3300/0 conn(0x7fb3f01616f0 0x7fb3f0161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f64b9640 1 -- 172.21.15.19:0/4043800184 >> v1:172.21.15.112:6789/0 conn(0x7fb3f00ce0a0 legacy=0x7fb3f00d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.418+0000 7fb3f64b9640 1 -- 172.21.15.19:0/4043800184 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f0162540 con 0x7fb3f01610f0 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.419+0000 7fb3f64b9640 1 --2- 172.21.15.19:0/4043800184 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01610f0 0x7fb3f006c1c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009870 tx=0x7fb3ec02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a0204850ced6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.419+0000 7fb3f54b7640 1 -- 172.21.15.19:0/4043800184 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec037070 con 0x7fb3f01610f0 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.419+0000 7fb3f54b7640 1 -- 172.21.15.19:0/4043800184 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ec032440 con 0x7fb3f01610f0 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.419+0000 7fb3f54b7640 1 -- 172.21.15.19:0/4043800184 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec032810 con 0x7fb3f01610f0 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.420+0000 7fb3f871d640 1 -- 172.21.15.19:0/4043800184 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01610f0 msgr2=0x7fb3f006c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.420+0000 7fb3f871d640 1 --2- 172.21.15.19:0/4043800184 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01610f0 0x7fb3f006c1c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009870 tx=0x7fb3ec02a270 comp rx=0 tx=0).stop 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.420+0000 7fb3f871d640 1 -- 172.21.15.19:0/4043800184 shutdown_connections 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.420+0000 7fb3f871d640 1 --2- 172.21.15.19:0/4043800184 >> v2:172.21.15.112:3300/0 conn(0x7fb3f01616f0 0x7fb3f0161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.420+0000 7fb3f871d640 1 --2- 172.21.15.19:0/4043800184 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01610f0 0x7fb3f006c1c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.420+0000 7fb3f871d640 1 -- 172.21.15.19:0/4043800184 >> 172.21.15.19:0/4043800184 conn(0x7fb3f006f500 msgr2=0x7fb3f006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.420+0000 7fb3f871d640 1 -- 172.21.15.19:0/4043800184 shutdown_connections 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.420+0000 7fb3f871d640 1 -- 172.21.15.19:0/4043800184 wait complete. 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.421+0000 7fb3f871d640 1 Processor -- start 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.421+0000 7fb3f871d640 1 -- start start 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.422+0000 7fb3f871d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f00ce0a0 0x7fb3f015d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.422+0000 7fb3f871d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3f01610f0 0x7fb3f00b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.422+0000 7fb3f871d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01616f0 0x7fb3f0146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.422+0000 7fb3f871d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00cf690 con 0x7fb3f01610f0 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.422+0000 7fb3f871d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00cfce0 con 0x7fb3f00ce0a0 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.422+0000 7fb3f871d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f006c700 con 0x7fb3f01616f0 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.422+0000 7fb3f5cb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3f01610f0 0x7fb3f00b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.422+0000 7fb3f5cb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3f01610f0 0x7fb3f00b7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36910/0 (socket says 172.21.15.19:36910) 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.422+0000 7fb3f5cb8640 1 -- 172.21.15.19:0/3938419507 learned_addr learned my addr 172.21.15.19:0/3938419507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.422+0000 7fb3f64b9640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f00ce0a0 0x7fb3f015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.422+0000 7fb3f6cba640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01616f0 0x7fb3f0146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3f5cb8640 1 -- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01616f0 msgr2=0x7fb3f0146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3f5cb8640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01616f0 0x7fb3f0146060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3f5cb8640 1 -- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f00ce0a0 msgr2=0x7fb3f015d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3f5cb8640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f00ce0a0 0x7fb3f015d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3f5cb8640 1 -- 172.21.15.19:0/3938419507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f015f670 con 0x7fb3f01610f0 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3f64b9640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f00ce0a0 0x7fb3f015d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3f5cb8640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3f01610f0 0x7fb3f00b7210 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000e9b0 tx=0x7fb3e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3e6ffd640 1 -- 172.21.15.19:0/3938419507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0017070 con 0x7fb3f01610f0 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3f871d640 1 -- 172.21.15.19:0/3938419507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f015f950 con 0x7fb3f01610f0 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3e6ffd640 1 -- 172.21.15.19:0/3938419507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e000c9a0 con 0x7fb3f01610f0 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3e6ffd640 1 -- 172.21.15.19:0/3938419507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e000cd90 con 0x7fb3f01610f0 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.423+0000 7fb3f871d640 1 -- 172.21.15.19:0/3938419507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f01601f0 con 0x7fb3f01610f0 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.424+0000 7fb3f871d640 1 -- 172.21.15.19:0/3938419507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f0073a00 con 0x7fb3f01610f0 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.425+0000 7fb3e6ffd640 1 -- 172.21.15.19:0/3938419507 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3e001a070 con 0x7fb3f01610f0 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.425+0000 7fb3e6ffd640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3d406b4d0 0x7fb3d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.425+0000 7fb3e6ffd640 1 -- 172.21.15.19:0/3938419507 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3e008a420 con 0x7fb3f01610f0 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.425+0000 7fb3f64b9640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3d406b4d0 0x7fb3d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.426+0000 7fb3f64b9640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3d406b4d0 0x7fb3d406d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec034000 tx=0x7fb3ec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.430+0000 7fb3e6ffd640 1 -- 172.21.15.19:0/3938419507 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3e00589b0 con 0x7fb3f01610f0 2024-04-25T10:02:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:36.650+0000 7fb3f871d640 1 -- 172.21.15.19:0/3938419507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb3f00b7a30 con 0x7fb3f01610f0 2024-04-25T10:02:37.625 INFO:tasks.ceph.mds.d.smithi112.stderr:2024-04-25T10:02:37.621+0000 7fb2f3cf1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:02:37.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.652+0000 7fb3e6ffd640 1 -- 172.21.15.19:0/3938419507 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb3e005ca50 con 0x7fb3f01610f0 2024-04-25T10:02:37.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 -- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3d406b4d0 msgr2=0x7fb3d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3d406b4d0 0x7fb3d406d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec034000 tx=0x7fb3ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:37.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 -- 172.21.15.19:0/3938419507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3f01610f0 msgr2=0x7fb3f00b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3f01610f0 0x7fb3f00b7210 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000e9b0 tx=0x7fb3e000ee70 comp rx=0 tx=0).stop 2024-04-25T10:02:37.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 -- 172.21.15.19:0/3938419507 shutdown_connections 2024-04-25T10:02:37.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb3d406b4d0 0x7fb3d406d990 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb3f01616f0 0x7fb3f0146060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3f01610f0 0x7fb3f00b7210 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 --2- 172.21.15.19:0/3938419507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f00ce0a0 0x7fb3f015d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 -- 172.21.15.19:0/3938419507 >> 172.21.15.19:0/3938419507 conn(0x7fb3f006f500 msgr2=0x7fb3f00c6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 -- 172.21.15.19:0/3938419507 shutdown_connections 2024-04-25T10:02:37.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.655+0000 7fb3f871d640 1 -- 172.21.15.19:0/3938419507 wait complete. 2024-04-25T10:02:37.675 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:02:37.675 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:02:37.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.780+0000 7f059e03a640 1 Processor -- start 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.780+0000 7f059e03a640 1 -- start start 2024-04-25T10:02:37.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f059e03a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05981616f0 0x7f0598161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f059e03a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059806c8d0 con 0x7f05980ce0a0 2024-04-25T10:02:37.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f059e03a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05980c6960 con 0x7f05981610f0 2024-04-25T10:02:37.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f059e03a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05981614d0 con 0x7f05981616f0 2024-04-25T10:02:37.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f0597fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f05980ce0a0 0x7f05980d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54844/0 (socket says 172.21.15.19:54844) 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f0597fff640 1 -- 172.21.15.19:0/2792027209 learned_addr learned my addr 172.21.15.19:0/2792027209 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f0596ffd640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05981616f0 0x7f0598161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261396562 0 0) 0x7f059806c8d0 con 0x7f05980ce0a0 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0580003520 con 0x7f05980ce0a0 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 758095366 0 0) 0x7f0580003520 con 0x7f05980ce0a0 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05981616f0 msgr2=0x7f0598161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f05967fc640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05981616f0 0x7f0598161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 >> v1:172.21.15.112:6789/0 conn(0x7f05981610f0 legacy=0x7f059806c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598162260 con 0x7f05980ce0a0 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f05900019f0 con 0x7f05980ce0a0 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 >> v1:172.21.15.19:6789/0 conn(0x7f05980ce0a0 legacy=0x7f05980d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f05967fc640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05981616f0 0x7f05800039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.781+0000 7f05967fc640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 0x7f0580004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f05967fc640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0580004d40 0x7f05800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05981614d0 con 0x7f0580003f60 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058000baa0 con 0x7f0580004d40 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058000bca0 con 0x7f05981616f0 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f0596ffd640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 0x7f0580004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f05977fe640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05981616f0 0x7f05800039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f0597fff640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0580004d40 0x7f05800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f0596ffd640 1 -- 172.21.15.19:0/2792027209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05981616f0 msgr2=0x7f05800039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f0596ffd640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05981616f0 0x7f05800039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f0596ffd640 1 -- 172.21.15.19:0/2792027209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0580004d40 msgr2=0x7f05800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f0596ffd640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0580004d40 0x7f05800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.782+0000 7f0596ffd640 1 -- 172.21.15.19:0/2792027209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0580003520 con 0x7f0580003f60 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.783+0000 7f0596ffd640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 0x7f0580004800 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f058c009ac0 tx=0x7f0580005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f9a5f07017c607 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.783+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c036070 con 0x7f0580003f60 2024-04-25T10:02:37.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.783+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c031400 con 0x7f0580003f60 2024-04-25T10:02:37.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.783+0000 7f05967fc640 1 -- 172.21.15.19:0/2792027209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c009690 con 0x7f0580003f60 2024-04-25T10:02:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.783+0000 7f059e03a640 1 -- 172.21.15.19:0/2792027209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 msgr2=0x7f0580004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.783+0000 7f059e03a640 1 --2- 172.21.15.19:0/2792027209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 0x7f0580004800 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f058c009ac0 tx=0x7f0580005830 comp rx=0 tx=0).stop 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.783+0000 7f0597fff640 1 -- 172.21.15.19:0/2792027209 reap_dead start 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.783+0000 7f059e03a640 1 -- 172.21.15.19:0/2792027209 shutdown_connections 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.783+0000 7f059e03a640 1 -- 172.21.15.19:0/2792027209 >> 172.21.15.19:0/2792027209 conn(0x7f059806f500 msgr2=0x7f059806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.783+0000 7f059e03a640 1 -- 172.21.15.19:0/2792027209 shutdown_connections 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.783+0000 7f059e03a640 1 -- 172.21.15.19:0/2792027209 wait complete. 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.785+0000 7f059e03a640 1 Processor -- start 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.785+0000 7f059e03a640 1 -- start start 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.785+0000 7f059e03a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 0x7f05980b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f059e03a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0598154b70 0x7f05980ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f059e03a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0598154ff0 0x7f05980ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f059e03a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05980cfce0 con 0x7f0580003f60 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f059e03a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05980cfa00 con 0x7f0598154b70 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f059e03a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05980cf690 con 0x7f0598154ff0 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f05977fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 0x7f05980b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f05977fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 0x7f05980b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40932/0 (socket says 172.21.15.19:40932) 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f05977fe640 1 -- 172.21.15.19:0/1073021436 learned_addr learned my addr 172.21.15.19:0/1073021436 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f0597fff640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0598154ff0 0x7f05980ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f0596ffd640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0598154b70 0x7f05980ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f05977fe640 1 -- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0598154ff0 msgr2=0x7f05980ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f05977fe640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0598154ff0 0x7f05980ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f05977fe640 1 -- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0598154b70 msgr2=0x7f05980ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f05977fe640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0598154b70 0x7f05980ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f05977fe640 1 -- 172.21.15.19:0/1073021436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980addc0 con 0x7f0580003f60 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f0596ffd640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0598154b70 0x7f05980ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f05977fe640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 0x7f05980b02c0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f058800f820 tx=0x7f058800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f057ffff640 1 -- 172.21.15.19:0/1073021436 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05880050d0 con 0x7f0580003f60 2024-04-25T10:02:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f057ffff640 1 -- 172.21.15.19:0/1073021436 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0588018070 con 0x7f0580003f60 2024-04-25T10:02:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f059e03a640 1 -- 172.21.15.19:0/1073021436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05981e8700 con 0x7f0580003f60 2024-04-25T10:02:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f057ffff640 1 -- 172.21.15.19:0/1073021436 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588005490 con 0x7f0580003f60 2024-04-25T10:02:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.786+0000 7f059e03a640 1 -- 172.21.15.19:0/1073021436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05981e8cc0 con 0x7f0580003f60 2024-04-25T10:02:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.787+0000 7f059e03a640 1 -- 172.21.15.19:0/1073021436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0598073a00 con 0x7f0580003f60 2024-04-25T10:02:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.788+0000 7f057ffff640 1 -- 172.21.15.19:0/1073021436 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f058800d730 con 0x7f0580003f60 2024-04-25T10:02:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.788+0000 7f057ffff640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f058406b4d0 0x7f058406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.788+0000 7f057ffff640 1 -- 172.21.15.19:0/1073021436 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f058808a1d0 con 0x7f0580003f60 2024-04-25T10:02:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.789+0000 7f0596ffd640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f058406b4d0 0x7f058406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.789+0000 7f0596ffd640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f058406b4d0 0x7f058406d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f058c033000 tx=0x7f058c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:37.793+0000 7f057ffff640 1 -- 172.21.15.19:0/1073021436 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0588058f50 con 0x7f0580003f60 2024-04-25T10:02:38.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.009+0000 7f059e03a640 1 -- 172.21.15.19:0/1073021436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f05980b0f50 con 0x7f0580003f60 2024-04-25T10:02:38.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.667+0000 7f057ffff640 1 -- 172.21.15.19:0/1073021436 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0588058850 con 0x7f0580003f60 2024-04-25T10:02:38.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.669+0000 7f059e03a640 1 -- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f058406b4d0 msgr2=0x7f058406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.669+0000 7f059e03a640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f058406b4d0 0x7f058406d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f058c033000 tx=0x7f058c008030 comp rx=0 tx=0).stop 2024-04-25T10:02:38.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.669+0000 7f059e03a640 1 -- 172.21.15.19:0/1073021436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 msgr2=0x7f05980b02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.669+0000 7f059e03a640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 0x7f05980b02c0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f058800f820 tx=0x7f058800fce0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.669+0000 7f059e03a640 1 -- 172.21.15.19:0/1073021436 shutdown_connections 2024-04-25T10:02:38.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.669+0000 7f059e03a640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f058406b4d0 0x7f058406d990 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.669+0000 7f059e03a640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0598154ff0 0x7f05980ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.669+0000 7f059e03a640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0598154b70 0x7f05980ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.669+0000 7f059e03a640 1 --2- 172.21.15.19:0/1073021436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580003f60 0x7f05980b02c0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.669+0000 7f059e03a640 1 -- 172.21.15.19:0/1073021436 >> 172.21.15.19:0/1073021436 conn(0x7f059806f500 msgr2=0x7f05980b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.670+0000 7f059e03a640 1 -- 172.21.15.19:0/1073021436 shutdown_connections 2024-04-25T10:02:38.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.670+0000 7f059e03a640 1 -- 172.21.15.19:0/1073021436 wait complete. 2024-04-25T10:02:38.686 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:38.689 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:38.690 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:02:38.690 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.832+0000 7f4b91309640 1 Processor -- start 2024-04-25T10:02:38.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.832+0000 7f4b91309640 1 -- start start 2024-04-25T10:02:38.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b91309640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b8c0d1020 0x7f4b8c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b91309640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8c0b97a0 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b91309640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8c0d1c50 con 0x7f4b8c15dbe0 2024-04-25T10:02:38.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b91309640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c15dfc0 con 0x7f4b8c0d1020 2024-04-25T10:02:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b83fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4b8c0b2cb0 0x7f4b8c0d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54852/0 (socket says 172.21.15.19:54852) 2024-04-25T10:02:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b83fff640 1 -- 172.21.15.19:0/4203489262 learned_addr learned my addr 172.21.15.19:0/4203489262 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b8b7fe640 1 --2- 172.21.15.19:0/4203489262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b8c0d1020 0x7f4b8c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b8a7fc640 1 -- 172.21.15.19:0/4203489262 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790595900 0 0) 0x7f4b8c0b97a0 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b8a7fc640 1 -- 172.21.15.19:0/4203489262 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b78003520 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b8a7fc640 1 -- 172.21.15.19:0/4203489262 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361997763 0 0) 0x7f4b8c0d1c50 con 0x7f4b8c15dbe0 2024-04-25T10:02:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b8a7fc640 1 -- 172.21.15.19:0/4203489262 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b8c0b97a0 con 0x7f4b8c15dbe0 2024-04-25T10:02:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b8b7fe640 1 -- 172.21.15.19:0/4203489262 >> v1:172.21.15.112:6789/0 conn(0x7f4b8c15dbe0 legacy=0x7f4b8c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b8b7fe640 1 -- 172.21.15.19:0/4203489262 >> v1:172.21.15.19:6789/0 conn(0x7f4b8c0b2cb0 legacy=0x7f4b8c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.833+0000 7f4b8b7fe640 1 -- 172.21.15.19:0/4203489262 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c0d1f30 con 0x7f4b8c0d1020 2024-04-25T10:02:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.834+0000 7f4b8b7fe640 1 --2- 172.21.15.19:0/4203489262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b8c0d1020 0x7f4b8c0d2e40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009dc0 tx=0x7f4b8402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40cb8178de155f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.834+0000 7f4b8a7fc640 1 -- 172.21.15.19:0/4203489262 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b84007740 con 0x7f4b8c0d1020 2024-04-25T10:02:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.834+0000 7f4b8a7fc640 1 -- 172.21.15.19:0/4203489262 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b840078e0 con 0x7f4b8c0d1020 2024-04-25T10:02:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.834+0000 7f4b8a7fc640 1 -- 172.21.15.19:0/4203489262 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b84007cb0 con 0x7f4b8c0d1020 2024-04-25T10:02:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.835+0000 7f4b91309640 1 -- 172.21.15.19:0/4203489262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b8c0d1020 msgr2=0x7f4b8c0d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.835+0000 7f4b91309640 1 --2- 172.21.15.19:0/4203489262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b8c0d1020 0x7f4b8c0d2e40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009dc0 tx=0x7f4b8402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.835+0000 7f4b91309640 1 -- 172.21.15.19:0/4203489262 shutdown_connections 2024-04-25T10:02:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.835+0000 7f4b91309640 1 --2- 172.21.15.19:0/4203489262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b8c0d1020 0x7f4b8c0d2e40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.835+0000 7f4b91309640 1 -- 172.21.15.19:0/4203489262 >> 172.21.15.19:0/4203489262 conn(0x7f4b8c061a20 msgr2=0x7f4b8c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.835+0000 7f4b91309640 1 -- 172.21.15.19:0/4203489262 shutdown_connections 2024-04-25T10:02:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.835+0000 7f4b91309640 1 -- 172.21.15.19:0/4203489262 wait complete. 2024-04-25T10:02:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.836+0000 7f4b91309640 1 Processor -- start 2024-04-25T10:02:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.836+0000 7f4b91309640 1 -- start start 2024-04-25T10:02:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.837+0000 7f4b91309640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b8c0b2cb0 0x7f4b8c160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.837+0000 7f4b91309640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b8c0d1020 0x7f4b8c14fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.837+0000 7f4b91309640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b8c15dbe0 0x7f4b8c154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.837+0000 7f4b91309640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c0b9420 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.837+0000 7f4b91309640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c0b92a0 con 0x7f4b8c0d1020 2024-04-25T10:02:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.837+0000 7f4b91309640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c0b6ab0 con 0x7f4b8c15dbe0 2024-04-25T10:02:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.837+0000 7f4b8affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b8c0b2cb0 0x7f4b8c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.837+0000 7f4b8affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b8c0b2cb0 0x7f4b8c160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40948/0 (socket says 172.21.15.19:40948) 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.837+0000 7f4b8affd640 1 -- 172.21.15.19:0/192432956 learned_addr learned my addr 172.21.15.19:0/192432956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.837+0000 7f4b8b7fe640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b8c15dbe0 0x7f4b8c154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.837+0000 7f4b83fff640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b8c0d1020 0x7f4b8c14fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.838+0000 7f4b8affd640 1 -- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b8c15dbe0 msgr2=0x7f4b8c154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.838+0000 7f4b8affd640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b8c15dbe0 0x7f4b8c154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.838+0000 7f4b8affd640 1 -- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b8c0d1020 msgr2=0x7f4b8c14fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.838+0000 7f4b8affd640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b8c0d1020 0x7f4b8c14fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.838+0000 7f4b8affd640 1 -- 172.21.15.19:0/192432956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c14fff0 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.838+0000 7f4b8affd640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b8c0b2cb0 0x7f4b8c160890 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c430 tx=0x7f4b7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.838+0000 7f4b837fe640 1 -- 172.21.15.19:0/192432956 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c017070 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.838+0000 7f4b837fe640 1 -- 172.21.15.19:0/192432956 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c0052f0 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.838+0000 7f4b91309640 1 -- 172.21.15.19:0/192432956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8c0b31c0 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.838+0000 7f4b837fe640 1 -- 172.21.15.19:0/192432956 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c003830 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.838+0000 7f4b91309640 1 -- 172.21.15.19:0/192432956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c0b3780 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.839+0000 7f4b91309640 1 -- 172.21.15.19:0/192432956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c073990 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.840+0000 7f4b837fe640 1 -- 172.21.15.19:0/192432956 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b7c01a070 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.840+0000 7f4b837fe640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b6c06b4d0 0x7f4b6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.840+0000 7f4b837fe640 1 -- 172.21.15.19:0/192432956 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b7c089670 con 0x7f4b8c0b2cb0 2024-04-25T10:02:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.840+0000 7f4b83fff640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b6c06b4d0 0x7f4b6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.841+0000 7f4b83fff640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b6c06b4d0 0x7f4b6c06d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c158240 tx=0x7f4b74007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:38.844+0000 7f4b837fe640 1 -- 172.21.15.19:0/192432956 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b7c057da0 con 0x7f4b8c0b2cb0 2024-04-25T10:02:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.062+0000 7f4b91309640 1 -- 172.21.15.19:0/192432956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b8c0ceec0 con 0x7f4b8c0b2cb0 2024-04-25T10:02:39.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.062+0000 7f4b837fe640 1 -- 172.21.15.19:0/192432956 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f4b7c05be40 con 0x7f4b8c0b2cb0 2024-04-25T10:02:39.068 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:39.068 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7466,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/590882657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":590882657},{"type":"v1","addr":"172.21.15.112:6836","nonce":590882657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":9744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1740426538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1740426538},{"type":"v1","addr":"172.21.15.19:6833","nonce":1740426538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/525024117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":525024117},{"type":"v1","addr":"172.21.15.19:6835","nonce":525024117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:34.992088+0000","modified":"2024-04-25T10:02:38.011991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2648279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2648279035},{"type":"v1","addr":"172.21.15.112:6837","nonce":2648279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:02:39.069 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 91 2024-04-25T10:02:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 -- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b6c06b4d0 msgr2=0x7f4b6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b6c06b4d0 0x7f4b6c06d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c158240 tx=0x7f4b74007d30 comp rx=0 tx=0).stop 2024-04-25T10:02:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 -- 172.21.15.19:0/192432956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b8c0b2cb0 msgr2=0x7f4b8c160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b8c0b2cb0 0x7f4b8c160890 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c430 tx=0x7f4b7c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 -- 172.21.15.19:0/192432956 shutdown_connections 2024-04-25T10:02:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b6c06b4d0 0x7f4b6c06d990 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4b8c15dbe0 0x7f4b8c154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b8c0d1020 0x7f4b8c14fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 --2- 172.21.15.19:0/192432956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b8c0b2cb0 0x7f4b8c160890 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 -- 172.21.15.19:0/192432956 >> 172.21.15.19:0/192432956 conn(0x7f4b8c061a20 msgr2=0x7f4b8c0d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 -- 172.21.15.19:0/192432956 shutdown_connections 2024-04-25T10:02:39.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.064+0000 7f4b91309640 1 -- 172.21.15.19:0/192432956 wait complete. 2024-04-25T10:02:39.081 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:39.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.176+0000 7fa336969640 1 Processor -- start 2024-04-25T10:02:39.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.176+0000 7fa336969640 1 -- start start 2024-04-25T10:02:39.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa336969640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa33015dd30 0x7fa3300b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa336969640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa3300d0ad0 0x7fa3300b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa336969640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3300d1df0 con 0x7fa3300b9770 2024-04-25T10:02:39.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa336969640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3300d20d0 con 0x7fa3300d0ad0 2024-04-25T10:02:39.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa336969640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3300d23b0 con 0x7fa33015dd30 2024-04-25T10:02:39.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa334f06640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa3300d0ad0 0x7fa3300b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa32f7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa3300b9770 0x7fa3300d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54862/0 (socket says 172.21.15.19:54862) 2024-04-25T10:02:39.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa334f06640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa3300d0ad0 0x7fa3300b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40962/0 (socket says 172.21.15.19:40962) 2024-04-25T10:02:39.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa32f7fe640 1 -- 172.21.15.19:0/3849302950 learned_addr learned my addr 172.21.15.19:0/3849302950 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:39.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa32ffff640 1 --2- 172.21.15.19:0/3849302950 >> v2:172.21.15.112:3300/0 conn(0x7fa33015dd30 0x7fa3300b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa334f06640 1 -- 172.21.15.19:0/3849302950 >> v2:172.21.15.112:3300/0 conn(0x7fa33015dd30 msgr2=0x7fa3300b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa334f06640 1 --2- 172.21.15.19:0/3849302950 >> v2:172.21.15.112:3300/0 conn(0x7fa33015dd30 0x7fa3300b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa334f06640 1 -- 172.21.15.19:0/3849302950 >> v1:172.21.15.19:6789/0 conn(0x7fa3300b9770 legacy=0x7fa3300d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa334f06640 1 -- 172.21.15.19:0/3849302950 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3300d2690 con 0x7fa3300d0ad0 2024-04-25T10:02:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.177+0000 7fa32effd640 1 -- 172.21.15.19:0/3849302950 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490421503 0 0) 0x7fa3300d1df0 con 0x7fa3300b9770 2024-04-25T10:02:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.178+0000 7fa334f06640 1 --2- 172.21.15.19:0/3849302950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3300d0ad0 0x7fa3300b6a40 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fa33006d000 tx=0x7fa32802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf6ca615ee9d6cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.178+0000 7fa32effd640 1 -- 172.21.15.19:0/3849302950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328007740 con 0x7fa3300d0ad0 2024-04-25T10:02:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.178+0000 7fa32effd640 1 -- 172.21.15.19:0/3849302950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3280078e0 con 0x7fa3300d0ad0 2024-04-25T10:02:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.178+0000 7fa32effd640 1 -- 172.21.15.19:0/3849302950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328007cb0 con 0x7fa3300d0ad0 2024-04-25T10:02:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.179+0000 7fa336969640 1 -- 172.21.15.19:0/3849302950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3300d0ad0 msgr2=0x7fa3300b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.179+0000 7fa336969640 1 --2- 172.21.15.19:0/3849302950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3300d0ad0 0x7fa3300b6a40 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fa33006d000 tx=0x7fa32802a560 comp rx=0 tx=0).stop 2024-04-25T10:02:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.179+0000 7fa336969640 1 -- 172.21.15.19:0/3849302950 shutdown_connections 2024-04-25T10:02:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.179+0000 7fa336969640 1 --2- 172.21.15.19:0/3849302950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3300d0ad0 0x7fa3300b6a40 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.179+0000 7fa336969640 1 --2- 172.21.15.19:0/3849302950 >> v2:172.21.15.112:3300/0 conn(0x7fa33015dd30 0x7fa3300b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.179+0000 7fa336969640 1 -- 172.21.15.19:0/3849302950 >> 172.21.15.19:0/3849302950 conn(0x7fa330061c30 msgr2=0x7fa33006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.179+0000 7fa336969640 1 -- 172.21.15.19:0/3849302950 shutdown_connections 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.179+0000 7fa336969640 1 -- 172.21.15.19:0/3849302950 wait complete. 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.180+0000 7fa336969640 1 Processor -- start 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.180+0000 7fa336969640 1 -- start start 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa336969640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3300b9770 0x7fa3300cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa336969640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3300d0ad0 0x7fa3301565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa336969640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa33015dd30 0x7fa330148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa336969640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300d1b90 con 0x7fa3300b9770 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa336969640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300d1a10 con 0x7fa3300d0ad0 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa336969640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300d2e90 con 0x7fa33015dd30 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa32ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3300b9770 0x7fa3300cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa32ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3300b9770 0x7fa3300cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40976/0 (socket says 172.21.15.19:40976) 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa32ffff640 1 -- 172.21.15.19:0/1661236960 learned_addr learned my addr 172.21.15.19:0/1661236960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa32f7fe640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3300d0ad0 0x7fa3301565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa334f06640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa33015dd30 0x7fa330148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa32ffff640 1 -- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa33015dd30 msgr2=0x7fa330148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa32ffff640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa33015dd30 0x7fa330148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa32ffff640 1 -- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3300d0ad0 msgr2=0x7fa3301565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa32ffff640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3300d0ad0 0x7fa3301565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.181+0000 7fa32ffff640 1 -- 172.21.15.19:0/1661236960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa330149960 con 0x7fa3300b9770 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.182+0000 7fa32ffff640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3300b9770 0x7fa3300cccf0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fa31800d5f0 tx=0x7fa31800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.182+0000 7fa32cff9640 1 -- 172.21.15.19:0/1661236960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180125f0 con 0x7fa3300b9770 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.182+0000 7fa32cff9640 1 -- 172.21.15.19:0/1661236960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa318019070 con 0x7fa3300b9770 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.182+0000 7fa32cff9640 1 -- 172.21.15.19:0/1661236960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180129d0 con 0x7fa3300b9770 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.182+0000 7fa336969640 1 -- 172.21.15.19:0/1661236960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3300cbc50 con 0x7fa3300b9770 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.182+0000 7fa336969640 1 -- 172.21.15.19:0/1661236960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3300cc210 con 0x7fa3300b9770 2024-04-25T10:02:39.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.183+0000 7fa336969640 1 -- 172.21.15.19:0/1661236960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa330073990 con 0x7fa3300b9770 2024-04-25T10:02:39.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.183+0000 7fa32cff9640 1 -- 172.21.15.19:0/1661236960 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa318012b70 con 0x7fa3300b9770 2024-04-25T10:02:39.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.184+0000 7fa32cff9640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa31006b4d0 0x7fa31006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.184+0000 7fa32cff9640 1 -- 172.21.15.19:0/1661236960 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa31808a6c0 con 0x7fa3300b9770 2024-04-25T10:02:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.184+0000 7fa32f7fe640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa31006b4d0 0x7fa31006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.184+0000 7fa32f7fe640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa31006b4d0 0x7fa31006d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa3301f6750 tx=0x7fa320009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.188+0000 7fa32cff9640 1 -- 172.21.15.19:0/1661236960 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3180593a0 con 0x7fa3300b9770 2024-04-25T10:02:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.368+0000 7fa336969640 1 -- 172.21.15.19:0/1661236960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa33015d0a0 con 0x7fa3300b9770 2024-04-25T10:02:39.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.369+0000 7fa32cff9640 1 -- 172.21.15.19:0/1661236960 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16880 (secure 0 0 0) 0x7fa318058ca0 con 0x7fa3300b9770 2024-04-25T10:02:39.375 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:39.375 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":140,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:36.636851+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:33.312366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:33.971764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 -- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa31006b4d0 msgr2=0x7fa31006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa31006b4d0 0x7fa31006d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa3301f6750 tx=0x7fa320009040 comp rx=0 tx=0).stop 2024-04-25T10:02:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 -- 172.21.15.19:0/1661236960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3300b9770 msgr2=0x7fa3300cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3300b9770 0x7fa3300cccf0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fa31800d5f0 tx=0x7fa31800dab0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 -- 172.21.15.19:0/1661236960 shutdown_connections 2024-04-25T10:02:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa31006b4d0 0x7fa31006d990 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa33015dd30 0x7fa330148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3300d0ad0 0x7fa3301565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 --2- 172.21.15.19:0/1661236960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3300b9770 0x7fa3300cccf0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 -- 172.21.15.19:0/1661236960 >> 172.21.15.19:0/1661236960 conn(0x7fa330061c30 msgr2=0x7fa3300ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 -- 172.21.15.19:0/1661236960 shutdown_connections 2024-04-25T10:02:39.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.371+0000 7fa336969640 1 -- 172.21.15.19:0/1661236960 wait complete. 2024-04-25T10:02:39.388 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:02:39.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:39.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.484+0000 7f4f614bd640 1 Processor -- start 2024-04-25T10:02:39.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.484+0000 7f4f614bd640 1 -- start start 2024-04-25T10:02:39.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.484+0000 7f4f614bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5c15dd30 0x7f4f5c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.484+0000 7f4f614bd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4f5c0d0ad0 0x7f4f5c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.485+0000 7f4f614bd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f5c0d1df0 con 0x7f4f5c0b9770 2024-04-25T10:02:39.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.485+0000 7f4f614bd640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f5c0d20d0 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.485+0000 7f4f614bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c0d23b0 con 0x7f4f5c15dd30 2024-04-25T10:02:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.485+0000 7f4f5b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4f5c0d0ad0 0x7f4f5c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.485+0000 7f4f5b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4f5c0d0ad0 0x7f4f5c0b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40984/0 (socket says 172.21.15.19:40984) 2024-04-25T10:02:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.485+0000 7f4f5b7fe640 1 -- 172.21.15.19:0/4084376266 learned_addr learned my addr 172.21.15.19:0/4084376266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.485+0000 7f4f5affd640 1 --2- 172.21.15.19:0/4084376266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5c15dd30 0x7f4f5c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.485+0000 7f4f5b7fe640 1 -- 172.21.15.19:0/4084376266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5c15dd30 msgr2=0x7f4f5c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.485+0000 7f4f5b7fe640 1 --2- 172.21.15.19:0/4084376266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5c15dd30 0x7f4f5c0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.485+0000 7f4f5b7fe640 1 -- 172.21.15.19:0/4084376266 >> v1:172.21.15.112:6789/0 conn(0x7f4f5c0b9770 legacy=0x7f4f5c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.485+0000 7f4f5b7fe640 1 -- 172.21.15.19:0/4084376266 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c0d2690 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.486+0000 7f4f5b7fe640 1 --2- 172.21.15.19:0/4084376266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f5c0d0ad0 0x7f4f5c0b6a40 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009e50 tx=0x7f4f5402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c33576d0cecf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.486+0000 7f4f5a7fc640 1 -- 172.21.15.19:0/4084376266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54036070 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.486+0000 7f4f5a7fc640 1 -- 172.21.15.19:0/4084376266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f54031600 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.486+0000 7f4f5a7fc640 1 -- 172.21.15.19:0/4084376266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f540319d0 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.486+0000 7f4f614bd640 1 -- 172.21.15.19:0/4084376266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f5c0d0ad0 msgr2=0x7f4f5c0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.486+0000 7f4f614bd640 1 --2- 172.21.15.19:0/4084376266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f5c0d0ad0 0x7f4f5c0b6a40 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009e50 tx=0x7f4f5402a680 comp rx=0 tx=0).stop 2024-04-25T10:02:39.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.486+0000 7f4f614bd640 1 -- 172.21.15.19:0/4084376266 shutdown_connections 2024-04-25T10:02:39.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.486+0000 7f4f614bd640 1 --2- 172.21.15.19:0/4084376266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f5c0d0ad0 0x7f4f5c0b6a40 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.486+0000 7f4f614bd640 1 --2- 172.21.15.19:0/4084376266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5c15dd30 0x7f4f5c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.486+0000 7f4f614bd640 1 -- 172.21.15.19:0/4084376266 >> 172.21.15.19:0/4084376266 conn(0x7f4f5c061c30 msgr2=0x7f4f5c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.486+0000 7f4f614bd640 1 -- 172.21.15.19:0/4084376266 shutdown_connections 2024-04-25T10:02:39.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.487+0000 7f4f614bd640 1 -- 172.21.15.19:0/4084376266 wait complete. 2024-04-25T10:02:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.488+0000 7f4f614bd640 1 Processor -- start 2024-04-25T10:02:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.488+0000 7f4f614bd640 1 -- start start 2024-04-25T10:02:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.488+0000 7f4f614bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b9770 0x7f4f5c1f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.488+0000 7f4f614bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f5c0d0ad0 0x7f4f5c14dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.488+0000 7f4f614bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5c15dd30 0x7f4f5c0cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.488+0000 7f4f614bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c0d1b90 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.488+0000 7f4f614bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c0d1a10 con 0x7f4f5c0b9770 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.488+0000 7f4f614bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c0d2e90 con 0x7f4f5c15dd30 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f5c0d0ad0 0x7f4f5c14dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f5c0d0ad0 0x7f4f5c14dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41000/0 (socket says 172.21.15.19:41000) 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f53fff640 1 -- 172.21.15.19:0/2512256876 learned_addr learned my addr 172.21.15.19:0/2512256876 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f5affd640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b9770 0x7f4f5c1f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f5b7fe640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5c15dd30 0x7f4f5c0cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f53fff640 1 -- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5c15dd30 msgr2=0x7f4f5c0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f53fff640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5c15dd30 0x7f4f5c0cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f53fff640 1 -- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b9770 msgr2=0x7f4f5c1f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f53fff640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b9770 0x7f4f5c1f3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f53fff640 1 -- 172.21.15.19:0/2512256876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c0b44a0 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f53fff640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f5c0d0ad0 0x7f4f5c14dcc0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400c400 tx=0x7f4f4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f537fe640 1 -- 172.21.15.19:0/2512256876 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440052d0 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f537fe640 1 -- 172.21.15.19:0/2512256876 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f440054b0 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f537fe640 1 -- 172.21.15.19:0/2512256876 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44006a40 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f614bd640 1 -- 172.21.15.19:0/2512256876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5c0b4780 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.489+0000 7f4f614bd640 1 -- 172.21.15.19:0/2512256876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5c14e200 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.490+0000 7f4f614bd640 1 -- 172.21.15.19:0/2512256876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5c073990 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.491+0000 7f4f537fe640 1 -- 172.21.15.19:0/2512256876 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f44006be0 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.491+0000 7f4f537fe640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f3c06b500 0x7f4f3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.491+0000 7f4f537fe640 1 -- 172.21.15.19:0/2512256876 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4f440897f0 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.492+0000 7f4f5affd640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f3c06b500 0x7f4f3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.492+0000 7f4f5affd640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f3c06b500 0x7f4f3c06d9c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009aa0 tx=0x7f4f4c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.495+0000 7f4f537fe640 1 -- 172.21.15.19:0/2512256876 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f44057df0 con 0x7f4f5c0d0ad0 2024-04-25T10:02:39.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.672+0000 7f4f614bd640 1 -- 172.21.15.19:0/2512256876 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f5c15d2a0 con 0x7f4f3c06b500 2024-04-25T10:02:39.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.680+0000 7f4f537fe640 1 -- 172.21.15.19:0/2512256876 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7f4f5c15d2a0 con 0x7f4f3c06b500 2024-04-25T10:02:39.689 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:39.690 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:02:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 -- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f3c06b500 msgr2=0x7f4f3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f3c06b500 0x7f4f3c06d9c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009aa0 tx=0x7f4f4c009190 comp rx=0 tx=0).stop 2024-04-25T10:02:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 -- 172.21.15.19:0/2512256876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f5c0d0ad0 msgr2=0x7f4f5c14dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f5c0d0ad0 0x7f4f5c14dcc0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400c400 tx=0x7f4f4400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 -- 172.21.15.19:0/2512256876 shutdown_connections 2024-04-25T10:02:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4f3c06b500 0x7f4f3c06d9c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5c15dd30 0x7f4f5c0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f5c0d0ad0 0x7f4f5c14dcc0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 --2- 172.21.15.19:0/2512256876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f5c0b9770 0x7f4f5c1f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 -- 172.21.15.19:0/2512256876 >> 172.21.15.19:0/2512256876 conn(0x7f4f5c061c30 msgr2=0x7f4f5c0c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 -- 172.21.15.19:0/2512256876 shutdown_connections 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.683+0000 7f4f614bd640 1 -- 172.21.15.19:0/2512256876 wait complete. 2024-04-25T10:02:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-04-25T10:02:39.044496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421296,"kb_used_data":14952,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754553424,"statfs":{"total":773094113280,"available":772662706176,"internally_reserved":0,"allocated":15310848,"data_stored":7764060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":415944242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.423213"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657252+0000","last_change":"2024-04-25T10:02:35.659995+0000","last_active":"2024-04-25T10:02:36.657252+0000","last_peered":"2024-04-25T10:02:36.657252+0000","last_clean":"2024-04-25T10:02:36.657252+0000","last_became_active":"2024-04-25T10:02:35.659715+0000","last_became_peered":"2024-04-25T10:02:35.659715+0000","last_unstale":"2024-04-25T10:02:36.657252+0000","last_undegraded":"2024-04-25T10:02:36.657252+0000","last_fullsized":"2024-04-25T10:02:36.657252+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:49.843714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642903+0000","last_change":"2024-04-25T10:02:35.657684+0000","last_active":"2024-04-25T10:02:36.642903+0000","last_peered":"2024-04-25T10:02:36.642903+0000","last_clean":"2024-04-25T10:02:36.642903+0000","last_became_active":"2024-04-25T10:02:35.657389+0000","last_became_peered":"2024-04-25T10:02:35.657389+0000","last_unstale":"2024-04-25T10:02:36.642903+0000","last_undegraded":"2024-04-25T10:02:36.642903+0000","last_fullsized":"2024-04-25T10:02:36.642903+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:02.976090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642586+0000","last_change":"2024-04-25T10:02:35.661627+0000","last_active":"2024-04-25T10:02:36.642586+0000","last_peered":"2024-04-25T10:02:36.642586+0000","last_clean":"2024-04-25T10:02:36.642586+0000","last_became_active":"2024-04-25T10:02:35.661363+0000","last_became_peered":"2024-04-25T10:02:35.661363+0000","last_unstale":"2024-04-25T10:02:36.642586+0000","last_undegraded":"2024-04-25T10:02:36.642586+0000","last_fullsized":"2024-04-25T10:02:36.642586+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:21.954841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737723+0000","last_change":"2024-04-25T10:02:35.656418+0000","last_active":"2024-04-25T10:02:36.737723+0000","last_peered":"2024-04-25T10:02:36.737723+0000","last_clean":"2024-04-25T10:02:36.737723+0000","last_became_active":"2024-04-25T10:02:35.656089+0000","last_became_peered":"2024-04-25T10:02:35.656089+0000","last_unstale":"2024-04-25T10:02:36.737723+0000","last_undegraded":"2024-04-25T10:02:36.737723+0000","last_fullsized":"2024-04-25T10:02:36.737723+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:09.551303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123400+0000","last_change":"2024-04-25T10:02:35.657356+0000","last_active":"2024-04-25T10:02:37.123400+0000","last_peered":"2024-04-25T10:02:37.123400+0000","last_clean":"2024-04-25T10:02:37.123400+0000","last_became_active":"2024-04-25T10:02:35.657060+0000","last_became_peered":"2024-04-25T10:02:35.657060+0000","last_unstale":"2024-04-25T10:02:37.123400+0000","last_undegraded":"2024-04-25T10:02:37.123400+0000","last_fullsized":"2024-04-25T10:02:37.123400+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:13.338984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123100+0000","last_change":"2024-04-25T10:02:35.661116+0000","last_active":"2024-04-25T10:02:37.123100+0000","last_peered":"2024-04-25T10:02:37.123100+0000","last_clean":"2024-04-25T10:02:37.123100+0000","last_became_active":"2024-04-25T10:02:35.658907+0000","last_became_peered":"2024-04-25T10:02:35.658907+0000","last_unstale":"2024-04-25T10:02:37.123100+0000","last_undegraded":"2024-04-25T10:02:37.123100+0000","last_fullsized":"2024-04-25T10:02:37.123100+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:49.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657165+0000","last_change":"2024-04-25T10:02:35.648401+0000","last_active":"2024-04-25T10:02:36.657165+0000","last_peered":"2024-04-25T10:02:36.657165+0000","last_clean":"2024-04-25T10:02:36.657165+0000","last_became_active":"2024-04-25T10:02:35.647639+0000","last_became_peered":"2024-04-25T10:02:35.647639+0000","last_unstale":"2024-04-25T10:02:36.657165+0000","last_undegraded":"2024-04-25T10:02:36.657165+0000","last_fullsized":"2024-04-25T10:02:36.657165+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:55.793255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641443+0000","last_change":"2024-04-25T10:02:35.662351+0000","last_active":"2024-04-25T10:02:36.641443+0000","last_peered":"2024-04-25T10:02:36.641443+0000","last_clean":"2024-04-25T10:02:36.641443+0000","last_became_active":"2024-04-25T10:02:35.662093+0000","last_became_peered":"2024-04-25T10:02:35.662093+0000","last_unstale":"2024-04-25T10:02:36.641443+0000","last_undegraded":"2024-04-25T10:02:36.641443+0000","last_fullsized":"2024-04-25T10:02:36.641443+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:52.211278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123243+0000","last_change":"2024-04-25T10:02:35.647699+0000","last_active":"2024-04-25T10:02:37.123243+0000","last_peered":"2024-04-25T10:02:37.123243+0000","last_clean":"2024-04-25T10:02:37.123243+0000","last_became_active":"2024-04-25T10:02:35.647393+0000","last_became_peered":"2024-04-25T10:02:35.647393+0000","last_unstale":"2024-04-25T10:02:37.123243+0000","last_undegraded":"2024-04-25T10:02:37.123243+0000","last_fullsized":"2024-04-25T10:02:37.123243+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:38.407520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123193+0000","last_change":"2024-04-25T10:02:35.652213+0000","last_active":"2024-04-25T10:02:37.123193+0000","last_peered":"2024-04-25T10:02:37.123193+0000","last_clean":"2024-04-25T10:02:37.123193+0000","last_became_active":"2024-04-25T10:02:35.651790+0000","last_became_peered":"2024-04-25T10:02:35.651790+0000","last_unstale":"2024-04-25T10:02:37.123193+0000","last_undegraded":"2024-04-25T10:02:37.123193+0000","last_fullsized":"2024-04-25T10:02:37.123193+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.678474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122281+0000","last_change":"2024-04-25T10:02:35.663399+0000","last_active":"2024-04-25T10:02:37.122281+0000","last_peered":"2024-04-25T10:02:37.122281+0000","last_clean":"2024-04-25T10:02:37.122281+0000","last_became_active":"2024-04-25T10:02:35.662169+0000","last_became_peered":"2024-04-25T10:02:35.662169+0000","last_unstale":"2024-04-25T10:02:37.122281+0000","last_undegraded":"2024-04-25T10:02:37.122281+0000","last_fullsized":"2024-04-25T10:02:37.122281+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:52.261567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737909+0000","last_change":"2024-04-25T10:02:35.663866+0000","last_active":"2024-04-25T10:02:36.737909+0000","last_peered":"2024-04-25T10:02:36.737909+0000","last_clean":"2024-04-25T10:02:36.737909+0000","last_became_active":"2024-04-25T10:02:35.663594+0000","last_became_peered":"2024-04-25T10:02:35.663594+0000","last_unstale":"2024-04-25T10:02:36.737909+0000","last_undegraded":"2024-04-25T10:02:36.737909+0000","last_fullsized":"2024-04-25T10:02:36.737909+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:32.612423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123087+0000","last_change":"2024-04-25T10:02:35.647593+0000","last_active":"2024-04-25T10:02:37.123087+0000","last_peered":"2024-04-25T10:02:37.123087+0000","last_clean":"2024-04-25T10:02:37.123087+0000","last_became_active":"2024-04-25T10:02:35.647002+0000","last_became_peered":"2024-04-25T10:02:35.647002+0000","last_unstale":"2024-04-25T10:02:37.123087+0000","last_undegraded":"2024-04-25T10:02:37.123087+0000","last_fullsized":"2024-04-25T10:02:37.123087+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:44.924061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737721+0000","last_change":"2024-04-25T10:02:35.667948+0000","last_active":"2024-04-25T10:02:36.737721+0000","last_peered":"2024-04-25T10:02:36.737721+0000","last_clean":"2024-04-25T10:02:36.737721+0000","last_became_active":"2024-04-25T10:02:35.666903+0000","last_became_peered":"2024-04-25T10:02:35.666903+0000","last_unstale":"2024-04-25T10:02:36.737721+0000","last_undegraded":"2024-04-25T10:02:36.737721+0000","last_fullsized":"2024-04-25T10:02:36.737721+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:39.535778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642621+0000","last_change":"2024-04-25T10:02:35.657089+0000","last_active":"2024-04-25T10:02:36.642621+0000","last_peered":"2024-04-25T10:02:36.642621+0000","last_clean":"2024-04-25T10:02:36.642621+0000","last_became_active":"2024-04-25T10:02:35.656757+0000","last_became_peered":"2024-04-25T10:02:35.656757+0000","last_unstale":"2024-04-25T10:02:36.642621+0000","last_undegraded":"2024-04-25T10:02:36.642621+0000","last_fullsized":"2024-04-25T10:02:36.642621+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:40.210106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.121824+0000","last_change":"2024-04-25T10:02:35.663002+0000","last_active":"2024-04-25T10:02:37.121824+0000","last_peered":"2024-04-25T10:02:37.121824+0000","last_clean":"2024-04-25T10:02:37.121824+0000","last_became_active":"2024-04-25T10:02:35.662713+0000","last_became_peered":"2024-04-25T10:02:35.662713+0000","last_unstale":"2024-04-25T10:02:37.121824+0000","last_undegraded":"2024-04-25T10:02:37.121824+0000","last_fullsized":"2024-04-25T10:02:37.121824+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:37.951795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.738064+0000","last_change":"2024-04-25T10:02:35.651304+0000","last_active":"2024-04-25T10:02:36.738064+0000","last_peered":"2024-04-25T10:02:36.738064+0000","last_clean":"2024-04-25T10:02:36.738064+0000","last_became_active":"2024-04-25T10:02:35.651042+0000","last_became_peered":"2024-04-25T10:02:35.651042+0000","last_unstale":"2024-04-25T10:02:36.738064+0000","last_undegraded":"2024-04-25T10:02:36.738064+0000","last_fullsized":"2024-04-25T10:02:36.738064+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:29.027829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122436+0000","last_change":"2024-04-25T10:02:35.647873+0000","last_active":"2024-04-25T10:02:37.122436+0000","last_peered":"2024-04-25T10:02:37.122436+0000","last_clean":"2024-04-25T10:02:37.122436+0000","last_became_active":"2024-04-25T10:02:35.647586+0000","last_became_peered":"2024-04-25T10:02:35.647586+0000","last_unstale":"2024-04-25T10:02:37.122436+0000","last_undegraded":"2024-04-25T10:02:37.122436+0000","last_fullsized":"2024-04-25T10:02:37.122436+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:35.757406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642192+0000","last_change":"2024-04-25T10:02:35.658850+0000","last_active":"2024-04-25T10:02:36.642192+0000","last_peered":"2024-04-25T10:02:36.642192+0000","last_clean":"2024-04-25T10:02:36.642192+0000","last_became_active":"2024-04-25T10:02:35.658604+0000","last_became_peered":"2024-04-25T10:02:35.658604+0000","last_unstale":"2024-04-25T10:02:36.642192+0000","last_undegraded":"2024-04-25T10:02:36.642192+0000","last_fullsized":"2024-04-25T10:02:36.642192+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:27.077595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122121+0000","last_change":"2024-04-25T10:02:35.654890+0000","last_active":"2024-04-25T10:02:37.122121+0000","last_peered":"2024-04-25T10:02:37.122121+0000","last_clean":"2024-04-25T10:02:37.122121+0000","last_became_active":"2024-04-25T10:02:35.654356+0000","last_became_peered":"2024-04-25T10:02:35.654356+0000","last_unstale":"2024-04-25T10:02:37.122121+0000","last_undegraded":"2024-04-25T10:02:37.122121+0000","last_fullsized":"2024-04-25T10:02:37.122121+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.210710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123309+0000","last_change":"2024-04-25T10:02:35.646990+0000","last_active":"2024-04-25T10:02:37.123309+0000","last_peered":"2024-04-25T10:02:37.123309+0000","last_clean":"2024-04-25T10:02:37.123309+0000","last_became_active":"2024-04-25T10:02:35.646569+0000","last_became_peered":"2024-04-25T10:02:35.646569+0000","last_unstale":"2024-04-25T10:02:37.123309+0000","last_undegraded":"2024-04-25T10:02:37.123309+0000","last_fullsized":"2024-04-25T10:02:37.123309+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:22.939497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657075+0000","last_change":"2024-04-25T10:02:35.648552+0000","last_active":"2024-04-25T10:02:36.657075+0000","last_peered":"2024-04-25T10:02:36.657075+0000","last_clean":"2024-04-25T10:02:36.657075+0000","last_became_active":"2024-04-25T10:02:35.648087+0000","last_became_peered":"2024-04-25T10:02:35.648087+0000","last_unstale":"2024-04-25T10:02:36.657075+0000","last_undegraded":"2024-04-25T10:02:36.657075+0000","last_fullsized":"2024-04-25T10:02:36.657075+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:00.614908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642003+0000","last_change":"2024-04-25T10:02:35.657929+0000","last_active":"2024-04-25T10:02:36.642003+0000","last_peered":"2024-04-25T10:02:36.642003+0000","last_clean":"2024-04-25T10:02:36.642003+0000","last_became_active":"2024-04-25T10:02:35.657559+0000","last_became_peered":"2024-04-25T10:02:35.657559+0000","last_unstale":"2024-04-25T10:02:36.642003+0000","last_undegraded":"2024-04-25T10:02:36.642003+0000","last_fullsized":"2024-04-25T10:02:36.642003+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:03.713031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122423+0000","last_change":"2024-04-25T10:02:35.655100+0000","last_active":"2024-04-25T10:02:37.122423+0000","last_peered":"2024-04-25T10:02:37.122423+0000","last_clean":"2024-04-25T10:02:37.122423+0000","last_became_active":"2024-04-25T10:02:35.654736+0000","last_became_peered":"2024-04-25T10:02:35.654736+0000","last_unstale":"2024-04-25T10:02:37.122423+0000","last_undegraded":"2024-04-25T10:02:37.122423+0000","last_fullsized":"2024-04-25T10:02:37.122423+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:05.575417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657300+0000","last_change":"2024-04-25T10:02:35.647105+0000","last_active":"2024-04-25T10:02:36.657300+0000","last_peered":"2024-04-25T10:02:36.657300+0000","last_clean":"2024-04-25T10:02:36.657300+0000","last_became_active":"2024-04-25T10:02:35.646832+0000","last_became_peered":"2024-04-25T10:02:35.646832+0000","last_unstale":"2024-04-25T10:02:36.657300+0000","last_undegraded":"2024-04-25T10:02:36.657300+0000","last_fullsized":"2024-04-25T10:02:36.657300+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:21.954841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657426+0000","last_change":"2024-04-25T10:02:35.647152+0000","last_active":"2024-04-25T10:02:36.657426+0000","last_peered":"2024-04-25T10:02:36.657426+0000","last_clean":"2024-04-25T10:02:36.657426+0000","last_became_active":"2024-04-25T10:02:35.646658+0000","last_became_peered":"2024-04-25T10:02:35.646658+0000","last_unstale":"2024-04-25T10:02:36.657426+0000","last_undegraded":"2024-04-25T10:02:36.657426+0000","last_fullsized":"2024-04-25T10:02:36.657426+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:40.210106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123176+0000","last_change":"2024-04-25T10:02:35.648468+0000","last_active":"2024-04-25T10:02:37.123176+0000","last_peered":"2024-04-25T10:02:37.123176+0000","last_clean":"2024-04-25T10:02:37.123176+0000","last_became_active":"2024-04-25T10:02:35.648175+0000","last_became_peered":"2024-04-25T10:02:35.648175+0000","last_unstale":"2024-04-25T10:02:37.123176+0000","last_undegraded":"2024-04-25T10:02:37.123176+0000","last_fullsized":"2024-04-25T10:02:37.123176+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:25.821546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122626+0000","last_change":"2024-04-25T10:02:35.661263+0000","last_active":"2024-04-25T10:02:37.122626+0000","last_peered":"2024-04-25T10:02:37.122626+0000","last_clean":"2024-04-25T10:02:37.122626+0000","last_became_active":"2024-04-25T10:02:35.660965+0000","last_became_peered":"2024-04-25T10:02:35.660965+0000","last_unstale":"2024-04-25T10:02:37.122626+0000","last_undegraded":"2024-04-25T10:02:37.122626+0000","last_fullsized":"2024-04-25T10:02:37.122626+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:07.360659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642652+0000","last_change":"2024-04-25T10:02:35.660659+0000","last_active":"2024-04-25T10:02:36.642652+0000","last_peered":"2024-04-25T10:02:36.642652+0000","last_clean":"2024-04-25T10:02:36.642652+0000","last_became_active":"2024-04-25T10:02:35.660328+0000","last_became_peered":"2024-04-25T10:02:35.660328+0000","last_unstale":"2024-04-25T10:02:36.642652+0000","last_undegraded":"2024-04-25T10:02:36.642652+0000","last_fullsized":"2024-04-25T10:02:36.642652+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:50.018601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122415+0000","last_change":"2024-04-25T10:02:35.651836+0000","last_active":"2024-04-25T10:02:37.122415+0000","last_peered":"2024-04-25T10:02:37.122415+0000","last_clean":"2024-04-25T10:02:37.122415+0000","last_became_active":"2024-04-25T10:02:35.651505+0000","last_became_peered":"2024-04-25T10:02:35.651505+0000","last_unstale":"2024-04-25T10:02:37.122415+0000","last_undegraded":"2024-04-25T10:02:37.122415+0000","last_fullsized":"2024-04-25T10:02:37.122415+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:41.293767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122767+0000","last_change":"2024-04-25T10:02:35.665285+0000","last_active":"2024-04-25T10:02:37.122767+0000","last_peered":"2024-04-25T10:02:37.122767+0000","last_clean":"2024-04-25T10:02:37.122767+0000","last_became_active":"2024-04-25T10:02:35.664866+0000","last_became_peered":"2024-04-25T10:02:35.664866+0000","last_unstale":"2024-04-25T10:02:37.122767+0000","last_undegraded":"2024-04-25T10:02:37.122767+0000","last_fullsized":"2024-04-25T10:02:37.122767+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:30.290870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642532+0000","last_change":"2024-04-25T10:02:35.661577+0000","last_active":"2024-04-25T10:02:36.642532+0000","last_peered":"2024-04-25T10:02:36.642532+0000","last_clean":"2024-04-25T10:02:36.642532+0000","last_became_active":"2024-04-25T10:02:35.661269+0000","last_became_peered":"2024-04-25T10:02:35.661269+0000","last_unstale":"2024-04-25T10:02:36.642532+0000","last_undegraded":"2024-04-25T10:02:36.642532+0000","last_fullsized":"2024-04-25T10:02:36.642532+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:54.266564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642730+0000","last_change":"2024-04-25T10:02:35.660836+0000","last_active":"2024-04-25T10:02:36.642730+0000","last_peered":"2024-04-25T10:02:36.642730+0000","last_clean":"2024-04-25T10:02:36.642730+0000","last_became_active":"2024-04-25T10:02:35.660507+0000","last_became_peered":"2024-04-25T10:02:35.660507+0000","last_unstale":"2024-04-25T10:02:36.642730+0000","last_undegraded":"2024-04-25T10:02:36.642730+0000","last_fullsized":"2024-04-25T10:02:36.642730+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:44.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"139'4","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123314+0000","last_change":"2024-04-25T10:02:34.643503+0000","last_active":"2024-04-25T10:02:37.123314+0000","last_peered":"2024-04-25T10:02:37.123314+0000","last_clean":"2024-04-25T10:02:37.123314+0000","last_became_active":"2024-04-25T10:02:34.643220+0000","last_became_peered":"2024-04-25T10:02:34.643220+0000","last_unstale":"2024-04-25T10:02:37.123314+0000","last_undegraded":"2024-04-25T10:02:37.123314+0000","last_fullsized":"2024-04-25T10:02:37.123314+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:28.023310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642745+0000","last_change":"2024-04-25T10:02:35.656969+0000","last_active":"2024-04-25T10:02:36.642745+0000","last_peered":"2024-04-25T10:02:36.642745+0000","last_clean":"2024-04-25T10:02:36.642745+0000","last_became_active":"2024-04-25T10:02:35.656678+0000","last_became_peered":"2024-04-25T10:02:35.656678+0000","last_unstale":"2024-04-25T10:02:36.642745+0000","last_undegraded":"2024-04-25T10:02:36.642745+0000","last_fullsized":"2024-04-25T10:02:36.642745+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:14.386733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641363+0000","last_change":"2024-04-25T10:02:34.640369+0000","last_active":"2024-04-25T10:02:36.641363+0000","last_peered":"2024-04-25T10:02:36.641363+0000","last_clean":"2024-04-25T10:02:36.641363+0000","last_became_active":"2024-04-25T10:02:34.638506+0000","last_became_peered":"2024-04-25T10:02:34.638506+0000","last_unstale":"2024-04-25T10:02:36.641363+0000","last_undegraded":"2024-04-25T10:02:36.641363+0000","last_fullsized":"2024-04-25T10:02:36.641363+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:43.280232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641933+0000","last_change":"2024-04-25T10:02:35.657040+0000","last_active":"2024-04-25T10:02:36.641933+0000","last_peered":"2024-04-25T10:02:36.641933+0000","last_clean":"2024-04-25T10:02:36.641933+0000","last_became_active":"2024-04-25T10:02:35.656671+0000","last_became_peered":"2024-04-25T10:02:35.656671+0000","last_unstale":"2024-04-25T10:02:36.641933+0000","last_undegraded":"2024-04-25T10:02:36.641933+0000","last_fullsized":"2024-04-25T10:02:36.641933+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:08.796466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642254+0000","last_change":"2024-04-25T10:02:34.633749+0000","last_active":"2024-04-25T10:02:36.642254+0000","last_peered":"2024-04-25T10:02:36.642254+0000","last_clean":"2024-04-25T10:02:36.642254+0000","last_became_active":"2024-04-25T10:02:34.633207+0000","last_became_peered":"2024-04-25T10:02:34.633207+0000","last_unstale":"2024-04-25T10:02:36.642254+0000","last_undegraded":"2024-04-25T10:02:36.642254+0000","last_fullsized":"2024-04-25T10:02:36.642254+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:39.205587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641890+0000","last_change":"2024-04-25T10:02:35.646474+0000","last_active":"2024-04-25T10:02:36.641890+0000","last_peered":"2024-04-25T10:02:36.641890+0000","last_clean":"2024-04-25T10:02:36.641890+0000","last_became_active":"2024-04-25T10:02:35.646192+0000","last_became_peered":"2024-04-25T10:02:35.646192+0000","last_unstale":"2024-04-25T10:02:36.641890+0000","last_undegraded":"2024-04-25T10:02:36.641890+0000","last_fullsized":"2024-04-25T10:02:36.641890+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.678474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642482+0000","last_change":"2024-04-25T10:02:34.640535+0000","last_active":"2024-04-25T10:02:36.642482+0000","last_peered":"2024-04-25T10:02:36.642482+0000","last_clean":"2024-04-25T10:02:36.642482+0000","last_became_active":"2024-04-25T10:02:34.640221+0000","last_became_peered":"2024-04-25T10:02:34.640221+0000","last_unstale":"2024-04-25T10:02:36.642482+0000","last_undegraded":"2024-04-25T10:02:36.642482+0000","last_fullsized":"2024-04-25T10:02:36.642482+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:51.347385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.121937+0000","last_change":"2024-04-25T10:02:35.655202+0000","last_active":"2024-04-25T10:02:37.121937+0000","last_peered":"2024-04-25T10:02:37.121937+0000","last_clean":"2024-04-25T10:02:37.121937+0000","last_became_active":"2024-04-25T10:02:35.654898+0000","last_became_peered":"2024-04-25T10:02:35.654898+0000","last_unstale":"2024-04-25T10:02:37.121937+0000","last_undegraded":"2024-04-25T10:02:37.121937+0000","last_fullsized":"2024-04-25T10:02:37.121937+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:15.741862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642697+0000","last_change":"2024-04-25T10:02:34.640849+0000","last_active":"2024-04-25T10:02:36.642697+0000","last_peered":"2024-04-25T10:02:36.642697+0000","last_clean":"2024-04-25T10:02:36.642697+0000","last_became_active":"2024-04-25T10:02:34.640263+0000","last_became_peered":"2024-04-25T10:02:34.640263+0000","last_unstale":"2024-04-25T10:02:36.642697+0000","last_undegraded":"2024-04-25T10:02:36.642697+0000","last_fullsized":"2024-04-25T10:02:36.642697+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:11.792032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737667+0000","last_change":"2024-04-25T10:02:35.654715+0000","last_active":"2024-04-25T10:02:36.737667+0000","last_peered":"2024-04-25T10:02:36.737667+0000","last_clean":"2024-04-25T10:02:36.737667+0000","last_became_active":"2024-04-25T10:02:35.654444+0000","last_became_peered":"2024-04-25T10:02:35.654444+0000","last_unstale":"2024-04-25T10:02:36.737667+0000","last_undegraded":"2024-04-25T10:02:36.737667+0000","last_fullsized":"2024-04-25T10:02:36.737667+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:40.880883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123163+0000","last_change":"2024-04-25T10:02:34.645621+0000","last_active":"2024-04-25T10:02:37.123163+0000","last_peered":"2024-04-25T10:02:37.123163+0000","last_clean":"2024-04-25T10:02:37.123163+0000","last_became_active":"2024-04-25T10:02:34.645059+0000","last_became_peered":"2024-04-25T10:02:34.645059+0000","last_unstale":"2024-04-25T10:02:37.123163+0000","last_undegraded":"2024-04-25T10:02:37.123163+0000","last_fullsized":"2024-04-25T10:02:37.123163+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:07.675769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642471+0000","last_change":"2024-04-25T10:02:35.656049+0000","last_active":"2024-04-25T10:02:36.642471+0000","last_peered":"2024-04-25T10:02:36.642471+0000","last_clean":"2024-04-25T10:02:36.642471+0000","last_became_active":"2024-04-25T10:02:35.655744+0000","last_became_peered":"2024-04-25T10:02:35.655744+0000","last_unstale":"2024-04-25T10:02:36.642471+0000","last_undegraded":"2024-04-25T10:02:36.642471+0000","last_fullsized":"2024-04-25T10:02:36.642471+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:55.793255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.656954+0000","last_change":"2024-04-25T10:02:34.635361+0000","last_active":"2024-04-25T10:02:36.656954+0000","last_peered":"2024-04-25T10:02:36.656954+0000","last_clean":"2024-04-25T10:02:36.656954+0000","last_became_active":"2024-04-25T10:02:34.634925+0000","last_became_peered":"2024-04-25T10:02:34.634925+0000","last_unstale":"2024-04-25T10:02:36.656954+0000","last_undegraded":"2024-04-25T10:02:36.656954+0000","last_fullsized":"2024-04-25T10:02:36.656954+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:51.347385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657177+0000","last_change":"2024-04-25T10:02:35.661977+0000","last_active":"2024-04-25T10:02:36.657177+0000","last_peered":"2024-04-25T10:02:36.657177+0000","last_clean":"2024-04-25T10:02:36.657177+0000","last_became_active":"2024-04-25T10:02:35.661711+0000","last_became_peered":"2024-04-25T10:02:35.661711+0000","last_unstale":"2024-04-25T10:02:36.657177+0000","last_undegraded":"2024-04-25T10:02:36.657177+0000","last_fullsized":"2024-04-25T10:02:36.657177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:41.842674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122609+0000","last_change":"2024-04-25T10:02:34.641294+0000","last_active":"2024-04-25T10:02:37.122609+0000","last_peered":"2024-04-25T10:02:37.122609+0000","last_clean":"2024-04-25T10:02:37.122609+0000","last_became_active":"2024-04-25T10:02:34.641008+0000","last_became_peered":"2024-04-25T10:02:34.641008+0000","last_unstale":"2024-04-25T10:02:37.122609+0000","last_undegraded":"2024-04-25T10:02:37.122609+0000","last_fullsized":"2024-04-25T10:02:37.122609+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:31.607904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642630+0000","last_change":"2024-04-25T10:02:35.657238+0000","last_active":"2024-04-25T10:02:36.642630+0000","last_peered":"2024-04-25T10:02:36.642630+0000","last_clean":"2024-04-25T10:02:36.642630+0000","last_became_active":"2024-04-25T10:02:35.656925+0000","last_became_peered":"2024-04-25T10:02:35.656925+0000","last_unstale":"2024-04-25T10:02:36.642630+0000","last_undegraded":"2024-04-25T10:02:36.642630+0000","last_fullsized":"2024-04-25T10:02:36.642630+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:50.871875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641948+0000","last_change":"2024-04-25T10:02:34.631919+0000","last_active":"2024-04-25T10:02:36.641948+0000","last_peered":"2024-04-25T10:02:36.641948+0000","last_clean":"2024-04-25T10:02:36.641948+0000","last_became_active":"2024-04-25T10:02:34.631620+0000","last_became_peered":"2024-04-25T10:02:34.631620+0000","last_unstale":"2024-04-25T10:02:36.641948+0000","last_undegraded":"2024-04-25T10:02:36.641948+0000","last_fullsized":"2024-04-25T10:02:36.641948+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:51.602050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122706+0000","last_change":"2024-04-25T10:02:35.646769+0000","last_active":"2024-04-25T10:02:37.122706+0000","last_peered":"2024-04-25T10:02:37.122706+0000","last_clean":"2024-04-25T10:02:37.122706+0000","last_became_active":"2024-04-25T10:02:35.646488+0000","last_became_peered":"2024-04-25T10:02:35.646488+0000","last_unstale":"2024-04-25T10:02:37.122706+0000","last_undegraded":"2024-04-25T10:02:37.122706+0000","last_fullsized":"2024-04-25T10:02:37.122706+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:18.430074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657552+0000","last_change":"2024-04-25T10:02:34.642849+0000","last_active":"2024-04-25T10:02:36.657552+0000","last_peered":"2024-04-25T10:02:36.657552+0000","last_clean":"2024-04-25T10:02:36.657552+0000","last_became_active":"2024-04-25T10:02:34.641713+0000","last_became_peered":"2024-04-25T10:02:34.641713+0000","last_unstale":"2024-04-25T10:02:36.657552+0000","last_undegraded":"2024-04-25T10:02:36.657552+0000","last_fullsized":"2024-04-25T10:02:36.657552+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:02.270880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122615+0000","last_change":"2024-04-25T10:02:35.646711+0000","last_active":"2024-04-25T10:02:37.122615+0000","last_peered":"2024-04-25T10:02:37.122615+0000","last_clean":"2024-04-25T10:02:37.122615+0000","last_became_active":"2024-04-25T10:02:35.646369+0000","last_became_peered":"2024-04-25T10:02:35.646369+0000","last_unstale":"2024-04-25T10:02:37.122615+0000","last_undegraded":"2024-04-25T10:02:37.122615+0000","last_fullsized":"2024-04-25T10:02:37.122615+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:39.109981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122562+0000","last_change":"2024-04-25T10:02:34.634059+0000","last_active":"2024-04-25T10:02:37.122562+0000","last_peered":"2024-04-25T10:02:37.122562+0000","last_clean":"2024-04-25T10:02:37.122562+0000","last_became_active":"2024-04-25T10:02:34.633764+0000","last_became_peered":"2024-04-25T10:02:34.633764+0000","last_unstale":"2024-04-25T10:02:37.122562+0000","last_undegraded":"2024-04-25T10:02:37.122562+0000","last_fullsized":"2024-04-25T10:02:37.122562+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:03.293271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642669+0000","last_change":"2024-04-25T10:02:35.663689+0000","last_active":"2024-04-25T10:02:36.642669+0000","last_peered":"2024-04-25T10:02:36.642669+0000","last_clean":"2024-04-25T10:02:36.642669+0000","last_became_active":"2024-04-25T10:02:35.663394+0000","last_became_peered":"2024-04-25T10:02:35.663394+0000","last_unstale":"2024-04-25T10:02:36.642669+0000","last_undegraded":"2024-04-25T10:02:36.642669+0000","last_fullsized":"2024-04-25T10:02:36.642669+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:41.592106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123369+0000","last_change":"2024-04-25T10:02:34.643724+0000","last_active":"2024-04-25T10:02:37.123369+0000","last_peered":"2024-04-25T10:02:37.123369+0000","last_clean":"2024-04-25T10:02:37.123369+0000","last_became_active":"2024-04-25T10:02:34.643418+0000","last_became_peered":"2024-04-25T10:02:34.643418+0000","last_unstale":"2024-04-25T10:02:37.123369+0000","last_undegraded":"2024-04-25T10:02:37.123369+0000","last_fullsized":"2024-04-25T10:02:37.123369+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:59.582687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642476+0000","last_change":"2024-04-25T10:02:35.656049+0000","last_active":"2024-04-25T10:02:36.642476+0000","last_peered":"2024-04-25T10:02:36.642476+0000","last_clean":"2024-04-25T10:02:36.642476+0000","last_became_active":"2024-04-25T10:02:35.655641+0000","last_became_peered":"2024-04-25T10:02:35.655641+0000","last_unstale":"2024-04-25T10:02:36.642476+0000","last_undegraded":"2024-04-25T10:02:36.642476+0000","last_fullsized":"2024-04-25T10:02:36.642476+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:41.842674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737651+0000","last_change":"2024-04-25T10:02:34.643903+0000","last_active":"2024-04-25T10:02:36.737651+0000","last_peered":"2024-04-25T10:02:36.737651+0000","last_clean":"2024-04-25T10:02:36.737651+0000","last_became_active":"2024-04-25T10:02:34.639682+0000","last_became_peered":"2024-04-25T10:02:34.639682+0000","last_unstale":"2024-04-25T10:02:36.737651+0000","last_undegraded":"2024-04-25T10:02:36.737651+0000","last_fullsized":"2024-04-25T10:02:36.737651+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:52.306679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123153+0000","last_change":"2024-04-25T10:02:35.649224+0000","last_active":"2024-04-25T10:02:37.123153+0000","last_peered":"2024-04-25T10:02:37.123153+0000","last_clean":"2024-04-25T10:02:37.123153+0000","last_became_active":"2024-04-25T10:02:35.648925+0000","last_became_peered":"2024-04-25T10:02:35.648925+0000","last_unstale":"2024-04-25T10:02:37.123153+0000","last_undegraded":"2024-04-25T10:02:37.123153+0000","last_fullsized":"2024-04-25T10:02:37.123153+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:20.331229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737701+0000","last_change":"2024-04-25T10:02:34.638730+0000","last_active":"2024-04-25T10:02:36.737701+0000","last_peered":"2024-04-25T10:02:36.737701+0000","last_clean":"2024-04-25T10:02:36.737701+0000","last_became_active":"2024-04-25T10:02:34.638315+0000","last_became_peered":"2024-04-25T10:02:34.638315+0000","last_unstale":"2024-04-25T10:02:36.737701+0000","last_undegraded":"2024-04-25T10:02:36.737701+0000","last_fullsized":"2024-04-25T10:02:36.737701+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:53.262045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642636+0000","last_change":"2024-04-25T10:02:35.657296+0000","last_active":"2024-04-25T10:02:36.642636+0000","last_peered":"2024-04-25T10:02:36.642636+0000","last_clean":"2024-04-25T10:02:36.642636+0000","last_became_active":"2024-04-25T10:02:35.655489+0000","last_became_peered":"2024-04-25T10:02:35.655489+0000","last_unstale":"2024-04-25T10:02:36.642636+0000","last_undegraded":"2024-04-25T10:02:36.642636+0000","last_fullsized":"2024-04-25T10:02:36.642636+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.606569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.656836+0000","last_change":"2024-04-25T10:02:34.637023+0000","last_active":"2024-04-25T10:02:36.656836+0000","last_peered":"2024-04-25T10:02:36.656836+0000","last_clean":"2024-04-25T10:02:36.656836+0000","last_became_active":"2024-04-25T10:02:34.636234+0000","last_became_peered":"2024-04-25T10:02:34.636234+0000","last_unstale":"2024-04-25T10:02:36.656836+0000","last_undegraded":"2024-04-25T10:02:36.656836+0000","last_fullsized":"2024-04-25T10:02:36.656836+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:40.587587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641621+0000","last_change":"2024-04-25T10:02:35.653013+0000","last_active":"2024-04-25T10:02:36.641621+0000","last_peered":"2024-04-25T10:02:36.641621+0000","last_clean":"2024-04-25T10:02:36.641621+0000","last_became_active":"2024-04-25T10:02:35.651592+0000","last_became_peered":"2024-04-25T10:02:35.651592+0000","last_unstale":"2024-04-25T10:02:36.641621+0000","last_undegraded":"2024-04-25T10:02:36.641621+0000","last_fullsized":"2024-04-25T10:02:36.641621+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:25.821546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122485+0000","last_change":"2024-04-25T10:02:34.636464+0000","last_active":"2024-04-25T10:02:37.122485+0000","last_peered":"2024-04-25T10:02:37.122485+0000","last_clean":"2024-04-25T10:02:37.122485+0000","last_became_active":"2024-04-25T10:02:34.636178+0000","last_became_peered":"2024-04-25T10:02:34.636178+0000","last_unstale":"2024-04-25T10:02:37.122485+0000","last_undegraded":"2024-04-25T10:02:37.122485+0000","last_fullsized":"2024-04-25T10:02:37.122485+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:00.297330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642302+0000","last_change":"2024-04-25T10:02:35.659218+0000","last_active":"2024-04-25T10:02:36.642302+0000","last_peered":"2024-04-25T10:02:36.642302+0000","last_clean":"2024-04-25T10:02:36.642302+0000","last_became_active":"2024-04-25T10:02:35.658935+0000","last_became_peered":"2024-04-25T10:02:35.658935+0000","last_unstale":"2024-04-25T10:02:36.642302+0000","last_undegraded":"2024-04-25T10:02:36.642302+0000","last_fullsized":"2024-04-25T10:02:36.642302+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:22.939497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641481+0000","last_change":"2024-04-25T10:02:34.640607+0000","last_active":"2024-04-25T10:02:36.641481+0000","last_peered":"2024-04-25T10:02:36.641481+0000","last_clean":"2024-04-25T10:02:36.641481+0000","last_became_active":"2024-04-25T10:02:34.639879+0000","last_became_peered":"2024-04-25T10:02:34.639879+0000","last_unstale":"2024-04-25T10:02:36.641481+0000","last_undegraded":"2024-04-25T10:02:36.641481+0000","last_fullsized":"2024-04-25T10:02:36.641481+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:13.382214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657446+0000","last_change":"2024-04-25T10:02:35.666616+0000","last_active":"2024-04-25T10:02:36.657446+0000","last_peered":"2024-04-25T10:02:36.657446+0000","last_clean":"2024-04-25T10:02:36.657446+0000","last_became_active":"2024-04-25T10:02:35.666259+0000","last_became_peered":"2024-04-25T10:02:35.666259+0000","last_unstale":"2024-04-25T10:02:36.657446+0000","last_undegraded":"2024-04-25T10:02:36.657446+0000","last_fullsized":"2024-04-25T10:02:36.657446+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.606569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642739+0000","last_change":"2024-04-25T10:02:34.643130+0000","last_active":"2024-04-25T10:02:36.642739+0000","last_peered":"2024-04-25T10:02:36.642739+0000","last_clean":"2024-04-25T10:02:36.642739+0000","last_became_active":"2024-04-25T10:02:34.642723+0000","last_became_peered":"2024-04-25T10:02:34.642723+0000","last_unstale":"2024-04-25T10:02:36.642739+0000","last_undegraded":"2024-04-25T10:02:36.642739+0000","last_fullsized":"2024-04-25T10:02:36.642739+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:04.514370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657398+0000","last_change":"2024-04-25T10:02:35.651697+0000","last_active":"2024-04-25T10:02:36.657398+0000","last_peered":"2024-04-25T10:02:36.657398+0000","last_clean":"2024-04-25T10:02:36.657398+0000","last_became_active":"2024-04-25T10:02:35.651448+0000","last_became_peered":"2024-04-25T10:02:35.651448+0000","last_unstale":"2024-04-25T10:02:36.657398+0000","last_undegraded":"2024-04-25T10:02:36.657398+0000","last_fullsized":"2024-04-25T10:02:36.657398+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:26.693313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737801+0000","last_change":"2024-04-25T10:02:34.643261+0000","last_active":"2024-04-25T10:02:36.737801+0000","last_peered":"2024-04-25T10:02:36.737801+0000","last_clean":"2024-04-25T10:02:36.737801+0000","last_became_active":"2024-04-25T10:02:34.640276+0000","last_became_peered":"2024-04-25T10:02:34.640276+0000","last_unstale":"2024-04-25T10:02:36.737801+0000","last_undegraded":"2024-04-25T10:02:36.737801+0000","last_fullsized":"2024-04-25T10:02:36.737801+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:02.708512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641558+0000","last_change":"2024-04-25T10:02:35.662112+0000","last_active":"2024-04-25T10:02:36.641558+0000","last_peered":"2024-04-25T10:02:36.641558+0000","last_clean":"2024-04-25T10:02:36.641558+0000","last_became_active":"2024-04-25T10:02:35.661263+0000","last_became_peered":"2024-04-25T10:02:35.661263+0000","last_unstale":"2024-04-25T10:02:36.641558+0000","last_undegraded":"2024-04-25T10:02:36.641558+0000","last_fullsized":"2024-04-25T10:02:36.641558+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:20.331229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122029+0000","last_change":"2024-04-25T10:02:34.641857+0000","last_active":"2024-04-25T10:02:37.122029+0000","last_peered":"2024-04-25T10:02:37.122029+0000","last_clean":"2024-04-25T10:02:37.122029+0000","last_became_active":"2024-04-25T10:02:34.641529+0000","last_became_peered":"2024-04-25T10:02:34.641529+0000","last_unstale":"2024-04-25T10:02:37.122029+0000","last_undegraded":"2024-04-25T10:02:37.122029+0000","last_fullsized":"2024-04-25T10:02:37.122029+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:22.663870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642611+0000","last_change":"2024-04-25T10:02:35.661000+0000","last_active":"2024-04-25T10:02:36.642611+0000","last_peered":"2024-04-25T10:02:36.642611+0000","last_clean":"2024-04-25T10:02:36.642611+0000","last_became_active":"2024-04-25T10:02:35.659838+0000","last_became_peered":"2024-04-25T10:02:35.659838+0000","last_unstale":"2024-04-25T10:02:36.642611+0000","last_undegraded":"2024-04-25T10:02:36.642611+0000","last_fullsized":"2024-04-25T10:02:36.642611+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:26.693313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641563+0000","last_change":"2024-04-25T10:02:34.636630+0000","last_active":"2024-04-25T10:02:36.641563+0000","last_peered":"2024-04-25T10:02:36.641563+0000","last_clean":"2024-04-25T10:02:36.641563+0000","last_became_active":"2024-04-25T10:02:34.635854+0000","last_became_peered":"2024-04-25T10:02:34.635854+0000","last_unstale":"2024-04-25T10:02:36.641563+0000","last_undegraded":"2024-04-25T10:02:36.641563+0000","last_fullsized":"2024-04-25T10:02:36.641563+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:28.821370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642019+0000","last_change":"2024-04-25T10:02:35.654507+0000","last_active":"2024-04-25T10:02:36.642019+0000","last_peered":"2024-04-25T10:02:36.642019+0000","last_clean":"2024-04-25T10:02:36.642019+0000","last_became_active":"2024-04-25T10:02:35.652135+0000","last_became_peered":"2024-04-25T10:02:35.652135+0000","last_unstale":"2024-04-25T10:02:36.642019+0000","last_undegraded":"2024-04-25T10:02:36.642019+0000","last_fullsized":"2024-04-25T10:02:36.642019+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:38.407520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.121809+0000","last_change":"2024-04-25T10:02:34.641092+0000","last_active":"2024-04-25T10:02:37.121809+0000","last_peered":"2024-04-25T10:02:37.121809+0000","last_clean":"2024-04-25T10:02:37.121809+0000","last_became_active":"2024-04-25T10:02:34.640753+0000","last_became_peered":"2024-04-25T10:02:34.640753+0000","last_unstale":"2024-04-25T10:02:37.121809+0000","last_undegraded":"2024-04-25T10:02:37.121809+0000","last_fullsized":"2024-04-25T10:02:37.121809+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:51.392051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642759+0000","last_change":"2024-04-25T10:02:35.658419+0000","last_active":"2024-04-25T10:02:36.642759+0000","last_peered":"2024-04-25T10:02:36.642759+0000","last_clean":"2024-04-25T10:02:36.642759+0000","last_became_active":"2024-04-25T10:02:35.658109+0000","last_became_peered":"2024-04-25T10:02:35.658109+0000","last_unstale":"2024-04-25T10:02:36.642759+0000","last_undegraded":"2024-04-25T10:02:36.642759+0000","last_fullsized":"2024-04-25T10:02:36.642759+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.699760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642737+0000","last_change":"2024-04-25T10:02:34.635245+0000","last_active":"2024-04-25T10:02:36.642737+0000","last_peered":"2024-04-25T10:02:36.642737+0000","last_clean":"2024-04-25T10:02:36.642737+0000","last_became_active":"2024-04-25T10:02:34.634933+0000","last_became_peered":"2024-04-25T10:02:34.634933+0000","last_unstale":"2024-04-25T10:02:36.642737+0000","last_undegraded":"2024-04-25T10:02:36.642737+0000","last_fullsized":"2024-04-25T10:02:36.642737+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:01.590104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123150+0000","last_change":"2024-04-25T10:02:35.661913+0000","last_active":"2024-04-25T10:02:37.123150+0000","last_peered":"2024-04-25T10:02:37.123150+0000","last_clean":"2024-04-25T10:02:37.123150+0000","last_became_active":"2024-04-25T10:02:35.661180+0000","last_became_peered":"2024-04-25T10:02:35.661180+0000","last_unstale":"2024-04-25T10:02:37.123150+0000","last_undegraded":"2024-04-25T10:02:37.123150+0000","last_fullsized":"2024-04-25T10:02:37.123150+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:48.022935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.738046+0000","last_change":"2024-04-25T10:02:34.639264+0000","last_active":"2024-04-25T10:02:36.738046+0000","last_peered":"2024-04-25T10:02:36.738046+0000","last_clean":"2024-04-25T10:02:36.738046+0000","last_became_active":"2024-04-25T10:02:34.638959+0000","last_became_peered":"2024-04-25T10:02:34.638959+0000","last_unstale":"2024-04-25T10:02:36.738046+0000","last_undegraded":"2024-04-25T10:02:36.738046+0000","last_fullsized":"2024-04-25T10:02:36.738046+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:49.867356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737946+0000","last_change":"2024-04-25T10:02:35.664607+0000","last_active":"2024-04-25T10:02:36.737946+0000","last_peered":"2024-04-25T10:02:36.737946+0000","last_clean":"2024-04-25T10:02:36.737946+0000","last_became_active":"2024-04-25T10:02:35.663577+0000","last_became_peered":"2024-04-25T10:02:35.663577+0000","last_unstale":"2024-04-25T10:02:36.737946+0000","last_undegraded":"2024-04-25T10:02:36.737946+0000","last_fullsized":"2024-04-25T10:02:36.737946+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:08.680288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657645+0000","last_change":"2024-04-25T10:02:34.642629+0000","last_active":"2024-04-25T10:02:36.657645+0000","last_peered":"2024-04-25T10:02:36.657645+0000","last_clean":"2024-04-25T10:02:36.657645+0000","last_became_active":"2024-04-25T10:02:34.642336+0000","last_became_peered":"2024-04-25T10:02:34.642336+0000","last_unstale":"2024-04-25T10:02:36.657645+0000","last_undegraded":"2024-04-25T10:02:36.657645+0000","last_fullsized":"2024-04-25T10:02:36.657645+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:51.691612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657580+0000","last_change":"2024-04-25T10:02:35.658702+0000","last_active":"2024-04-25T10:02:36.657580+0000","last_peered":"2024-04-25T10:02:36.657580+0000","last_clean":"2024-04-25T10:02:36.657580+0000","last_became_active":"2024-04-25T10:02:35.658418+0000","last_became_peered":"2024-04-25T10:02:35.658418+0000","last_unstale":"2024-04-25T10:02:36.657580+0000","last_undegraded":"2024-04-25T10:02:36.657580+0000","last_fullsized":"2024-04-25T10:02:36.657580+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:44.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642795+0000","last_change":"2024-04-25T10:02:34.633345+0000","last_active":"2024-04-25T10:02:36.642795+0000","last_peered":"2024-04-25T10:02:36.642795+0000","last_clean":"2024-04-25T10:02:36.642795+0000","last_became_active":"2024-04-25T10:02:34.632906+0000","last_became_peered":"2024-04-25T10:02:34.632906+0000","last_unstale":"2024-04-25T10:02:36.642795+0000","last_undegraded":"2024-04-25T10:02:36.642795+0000","last_fullsized":"2024-04-25T10:02:36.642795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:14.512825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123256+0000","last_change":"2024-04-25T10:02:35.655580+0000","last_active":"2024-04-25T10:02:37.123256+0000","last_peered":"2024-04-25T10:02:37.123256+0000","last_clean":"2024-04-25T10:02:37.123256+0000","last_became_active":"2024-04-25T10:02:35.655335+0000","last_became_peered":"2024-04-25T10:02:35.655335+0000","last_unstale":"2024-04-25T10:02:37.123256+0000","last_undegraded":"2024-04-25T10:02:37.123256+0000","last_fullsized":"2024-04-25T10:02:37.123256+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:27.077595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122983+0000","last_change":"2024-04-25T10:02:34.644163+0000","last_active":"2024-04-25T10:02:37.122983+0000","last_peered":"2024-04-25T10:02:37.122983+0000","last_clean":"2024-04-25T10:02:37.122983+0000","last_became_active":"2024-04-25T10:02:34.643892+0000","last_became_peered":"2024-04-25T10:02:34.643892+0000","last_unstale":"2024-04-25T10:02:37.122983+0000","last_undegraded":"2024-04-25T10:02:37.122983+0000","last_fullsized":"2024-04-25T10:02:37.122983+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:02.661463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642721+0000","last_change":"2024-04-25T10:02:35.656319+0000","last_active":"2024-04-25T10:02:36.642721+0000","last_peered":"2024-04-25T10:02:36.642721+0000","last_clean":"2024-04-25T10:02:36.642721+0000","last_became_active":"2024-04-25T10:02:35.656040+0000","last_became_peered":"2024-04-25T10:02:35.656040+0000","last_unstale":"2024-04-25T10:02:36.642721+0000","last_undegraded":"2024-04-25T10:02:36.642721+0000","last_fullsized":"2024-04-25T10:02:36.642721+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:52.351904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657055+0000","last_change":"2024-04-25T10:02:34.642345+0000","last_active":"2024-04-25T10:02:36.657055+0000","last_peered":"2024-04-25T10:02:36.657055+0000","last_clean":"2024-04-25T10:02:36.657055+0000","last_became_active":"2024-04-25T10:02:34.641888+0000","last_became_peered":"2024-04-25T10:02:34.641888+0000","last_unstale":"2024-04-25T10:02:36.657055+0000","last_undegraded":"2024-04-25T10:02:36.657055+0000","last_fullsized":"2024-04-25T10:02:36.657055+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:14.512825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737663+0000","last_change":"2024-04-25T10:02:35.667016+0000","last_active":"2024-04-25T10:02:36.737663+0000","last_peered":"2024-04-25T10:02:36.737663+0000","last_clean":"2024-04-25T10:02:36.737663+0000","last_became_active":"2024-04-25T10:02:35.666736+0000","last_became_peered":"2024-04-25T10:02:35.666736+0000","last_unstale":"2024-04-25T10:02:36.737663+0000","last_undegraded":"2024-04-25T10:02:36.737663+0000","last_fullsized":"2024-04-25T10:02:36.737663+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:35.875909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122416+0000","last_change":"2024-04-25T10:02:34.642788+0000","last_active":"2024-04-25T10:02:37.122416+0000","last_peered":"2024-04-25T10:02:37.122416+0000","last_clean":"2024-04-25T10:02:37.122416+0000","last_became_active":"2024-04-25T10:02:34.642492+0000","last_became_peered":"2024-04-25T10:02:34.642492+0000","last_unstale":"2024-04-25T10:02:37.122416+0000","last_undegraded":"2024-04-25T10:02:37.122416+0000","last_fullsized":"2024-04-25T10:02:37.122416+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:15.889113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642559+0000","last_change":"2024-04-25T10:02:35.660534+0000","last_active":"2024-04-25T10:02:36.642559+0000","last_peered":"2024-04-25T10:02:36.642559+0000","last_clean":"2024-04-25T10:02:36.642559+0000","last_became_active":"2024-04-25T10:02:35.660192+0000","last_became_peered":"2024-04-25T10:02:35.660192+0000","last_unstale":"2024-04-25T10:02:36.642559+0000","last_undegraded":"2024-04-25T10:02:36.642559+0000","last_fullsized":"2024-04-25T10:02:36.642559+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:49.843714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657076+0000","last_change":"2024-04-25T10:02:34.642329+0000","last_active":"2024-04-25T10:02:36.657076+0000","last_peered":"2024-04-25T10:02:36.657076+0000","last_clean":"2024-04-25T10:02:36.657076+0000","last_became_active":"2024-04-25T10:02:34.641905+0000","last_became_peered":"2024-04-25T10:02:34.641905+0000","last_unstale":"2024-04-25T10:02:36.657076+0000","last_undegraded":"2024-04-25T10:02:36.657076+0000","last_fullsized":"2024-04-25T10:02:36.657076+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:04.514370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642634+0000","last_change":"2024-04-25T10:02:35.660001+0000","last_active":"2024-04-25T10:02:36.642634+0000","last_peered":"2024-04-25T10:02:36.642634+0000","last_clean":"2024-04-25T10:02:36.642634+0000","last_became_active":"2024-04-25T10:02:35.659323+0000","last_became_peered":"2024-04-25T10:02:35.659323+0000","last_unstale":"2024-04-25T10:02:36.642634+0000","last_undegraded":"2024-04-25T10:02:36.642634+0000","last_fullsized":"2024-04-25T10:02:36.642634+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:01.934600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641613+0000","last_change":"2024-04-25T10:02:34.634113+0000","last_active":"2024-04-25T10:02:36.641613+0000","last_peered":"2024-04-25T10:02:36.641613+0000","last_clean":"2024-04-25T10:02:36.641613+0000","last_became_active":"2024-04-25T10:02:34.633573+0000","last_became_peered":"2024-04-25T10:02:34.633573+0000","last_unstale":"2024-04-25T10:02:36.641613+0000","last_undegraded":"2024-04-25T10:02:36.641613+0000","last_fullsized":"2024-04-25T10:02:36.641613+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:25.688794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642087+0000","last_change":"2024-04-25T10:02:35.657180+0000","last_active":"2024-04-25T10:02:36.642087+0000","last_peered":"2024-04-25T10:02:36.642087+0000","last_clean":"2024-04-25T10:02:36.642087+0000","last_became_active":"2024-04-25T10:02:35.656445+0000","last_became_peered":"2024-04-25T10:02:35.656445+0000","last_unstale":"2024-04-25T10:02:36.642087+0000","last_undegraded":"2024-04-25T10:02:36.642087+0000","last_fullsized":"2024-04-25T10:02:36.642087+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:53.311199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"139'6","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122039+0000","last_change":"2024-04-25T10:02:34.641165+0000","last_active":"2024-04-25T10:02:37.122039+0000","last_peered":"2024-04-25T10:02:37.122039+0000","last_clean":"2024-04-25T10:02:37.122039+0000","last_became_active":"2024-04-25T10:02:34.640851+0000","last_became_peered":"2024-04-25T10:02:34.640851+0000","last_unstale":"2024-04-25T10:02:37.122039+0000","last_undegraded":"2024-04-25T10:02:37.122039+0000","last_fullsized":"2024-04-25T10:02:37.122039+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:07.068342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":414,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641721+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:02:36.641721+0000","last_peered":"2024-04-25T10:02:36.641721+0000","last_clean":"2024-04-25T10:02:36.641721+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:02:36.641721+0000","last_undegraded":"2024-04-25T10:02:36.641721+0000","last_fullsized":"2024-04-25T10:02:36.641721+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51532,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2355200,"data_stored":1412703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50948,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320892,"statfs":{"total":96636764160,"available":96584593408,"internally_reserved":0,"allocated":1761280,"data_stored":822383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46792,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325048,"statfs":{"total":96636764160,"available":96588849152,"internally_reserved":0,"allocated":1757184,"data_stored":822313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55120,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316720,"statfs":{"total":96636764160,"available":96580321280,"internally_reserved":0,"allocated":1769472,"data_stored":823925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59280,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312560,"statfs":{"total":96636764160,"available":96576061440,"internally_reserved":0,"allocated":1765376,"data_stored":822929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50960,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320880,"statfs":{"total":96636764160,"available":96584581120,"internally_reserved":0,"allocated":1765376,"data_stored":822425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50952,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320888,"statfs":{"total":96636764160,"available":96584589312,"internally_reserved":0,"allocated":1761280,"data_stored":822383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55712,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316128,"statfs":{"total":96636764160,"available":96579715072,"internally_reserved":0,"allocated":2375680,"data_stored":1414999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:39.706 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:39.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.836+0000 7f2625846640 1 Processor -- start 2024-04-25T10:02:39.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.837+0000 7f2625846640 1 -- start start 2024-04-25T10:02:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.837+0000 7f2625846640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f26200af720 0x7f26200afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.837+0000 7f2625846640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f26200d02f0 0x7f26200d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.837+0000 7f2625846640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262006fe00 con 0x7f2620158220 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.837+0000 7f2625846640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26200b3670 con 0x7f26200af720 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.837+0000 7f2625846640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2620070a10 con 0x7f26200d02f0 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.837+0000 7f2617fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f26200af720 0x7f26200afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.837+0000 7f261effd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2620158220 0x7f262015ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54866/0 (socket says 172.21.15.19:54866) 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.838+0000 7f261effd640 1 -- 172.21.15.19:0/596835206 learned_addr learned my addr 172.21.15.19:0/596835206 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.838+0000 7f261f7fe640 1 --2- 172.21.15.19:0/596835206 >> v2:172.21.15.112:3300/0 conn(0x7f26200d02f0 0x7f26200d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.838+0000 7f2617fff640 1 -- 172.21.15.19:0/596835206 >> v2:172.21.15.112:3300/0 conn(0x7f26200d02f0 msgr2=0x7f26200d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.838+0000 7f2617fff640 1 --2- 172.21.15.19:0/596835206 >> v2:172.21.15.112:3300/0 conn(0x7f26200d02f0 0x7f26200d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.838+0000 7f2617fff640 1 -- 172.21.15.19:0/596835206 >> v1:172.21.15.19:6789/0 conn(0x7f2620158220 legacy=0x7f262015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.838+0000 7f2617fff640 1 -- 172.21.15.19:0/596835206 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26200700e0 con 0x7f26200af720 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.838+0000 7f261f7fe640 1 --2- 172.21.15.19:0/596835206 >> v2:172.21.15.112:3300/0 conn(0x7f26200d02f0 0x7f26200d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.839+0000 7f2617fff640 1 --2- 172.21.15.19:0/596835206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26200af720 0x7f26200afba0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f26200726d0 tx=0x7f260802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4de0451c466b0f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.839+0000 7f261e7fc640 1 -- 172.21.15.19:0/596835206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608037070 con 0x7f26200af720 2024-04-25T10:02:39.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.839+0000 7f261e7fc640 1 -- 172.21.15.19:0/596835206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608032440 con 0x7f26200af720 2024-04-25T10:02:39.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.839+0000 7f261e7fc640 1 -- 172.21.15.19:0/596835206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608032810 con 0x7f26200af720 2024-04-25T10:02:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.839+0000 7f2625846640 1 -- 172.21.15.19:0/596835206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26200af720 msgr2=0x7f26200afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.839+0000 7f2625846640 1 --2- 172.21.15.19:0/596835206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26200af720 0x7f26200afba0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f26200726d0 tx=0x7f260802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.839+0000 7f2625846640 1 -- 172.21.15.19:0/596835206 shutdown_connections 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.839+0000 7f2625846640 1 --2- 172.21.15.19:0/596835206 >> v2:172.21.15.112:3300/0 conn(0x7f26200d02f0 0x7f26200d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.839+0000 7f2625846640 1 --2- 172.21.15.19:0/596835206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26200af720 0x7f26200afba0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.839+0000 7f2625846640 1 -- 172.21.15.19:0/596835206 >> 172.21.15.19:0/596835206 conn(0x7f26200b2370 msgr2=0x7f26200b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.840+0000 7f2625846640 1 -- 172.21.15.19:0/596835206 shutdown_connections 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.840+0000 7f2625846640 1 -- 172.21.15.19:0/596835206 wait complete. 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.841+0000 7f2625846640 1 Processor -- start 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.841+0000 7f2625846640 1 -- start start 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.841+0000 7f2625846640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200af720 0x7f262014f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.841+0000 7f2625846640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26200d02f0 0x7f2620159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.841+0000 7f2625846640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2620158220 0x7f2620159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.841+0000 7f2625846640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2620158600 con 0x7f26200d02f0 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2625846640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26200706a0 con 0x7f26200af720 2024-04-25T10:02:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2625846640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2620072960 con 0x7f2620158220 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2617fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26200d02f0 0x7f2620159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2617fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26200d02f0 0x7f2620159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41010/0 (socket says 172.21.15.19:41010) 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2617fff640 1 -- 172.21.15.19:0/1274181715 learned_addr learned my addr 172.21.15.19:0/1274181715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f261effd640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200af720 0x7f262014f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f261f7fe640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2620158220 0x7f2620159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2617fff640 1 -- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2620158220 msgr2=0x7f2620159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2617fff640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2620158220 0x7f2620159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2617fff640 1 -- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200af720 msgr2=0x7f262014f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2617fff640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200af720 0x7f262014f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2617fff640 1 -- 172.21.15.19:0/1274181715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262015f900 con 0x7f26200d02f0 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f261f7fe640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2620158220 0x7f2620159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2617fff640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26200d02f0 0x7f2620159200 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f2608000c00 tx=0x7f26080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f261cff9640 1 -- 172.21.15.19:0/1274181715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608037040 con 0x7f26200d02f0 2024-04-25T10:02:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f261cff9640 1 -- 172.21.15.19:0/1274181715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608042070 con 0x7f26200d02f0 2024-04-25T10:02:39.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2625846640 1 -- 172.21.15.19:0/1274181715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262015fbe0 con 0x7f26200d02f0 2024-04-25T10:02:39.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f261cff9640 1 -- 172.21.15.19:0/1274181715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608030bd0 con 0x7f26200d02f0 2024-04-25T10:02:39.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.842+0000 7f2625846640 1 -- 172.21.15.19:0/1274181715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26201601a0 con 0x7f26200d02f0 2024-04-25T10:02:39.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.843+0000 7f2625846640 1 -- 172.21.15.19:0/1274181715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2620073990 con 0x7f26200d02f0 2024-04-25T10:02:39.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.844+0000 7f261cff9640 1 -- 172.21.15.19:0/1274181715 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2608047460 con 0x7f26200d02f0 2024-04-25T10:02:39.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.844+0000 7f261cff9640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f260006b500 0x7f260006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.844+0000 7f261cff9640 1 -- 172.21.15.19:0/1274181715 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f26080aae70 con 0x7f26200d02f0 2024-04-25T10:02:39.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.845+0000 7f261effd640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f260006b500 0x7f260006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.845+0000 7f261effd640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f260006b500 0x7f260006d9c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2610007fd0 tx=0x7f2610007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:39.848+0000 7f261cff9640 1 -- 172.21.15.19:0/1274181715 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2608032e20 con 0x7f26200d02f0 2024-04-25T10:02:40.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.027+0000 7f2625846640 1 -- 172.21.15.19:0/1274181715 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2620161a40 con 0x7f260006b500 2024-04-25T10:02:40.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.033+0000 7f261cff9640 1 -- 172.21.15.19:0/1274181715 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7f2620161a40 con 0x7f260006b500 2024-04-25T10:02:40.042 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:02:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 -- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f260006b500 msgr2=0x7f260006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f260006b500 0x7f260006d9c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2610007fd0 tx=0x7f2610007e80 comp rx=0 tx=0).stop 2024-04-25T10:02:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 -- 172.21.15.19:0/1274181715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26200d02f0 msgr2=0x7f2620159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26200d02f0 0x7f2620159200 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f2608000c00 tx=0x7f26080046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 -- 172.21.15.19:0/1274181715 shutdown_connections 2024-04-25T10:02:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f260006b500 0x7f260006d9c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2620158220 0x7f2620159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26200d02f0 0x7f2620159200 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 --2- 172.21.15.19:0/1274181715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26200af720 0x7f262014f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 -- 172.21.15.19:0/1274181715 >> 172.21.15.19:0/1274181715 conn(0x7f26200b2370 msgr2=0x7f2620155cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 -- 172.21.15.19:0/1274181715 shutdown_connections 2024-04-25T10:02:40.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.036+0000 7f2625846640 1 -- 172.21.15.19:0/1274181715 wait complete. 2024-04-25T10:02:40.053 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-04-25T10:02:39.044496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421296,"kb_used_data":14952,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754553424,"statfs":{"total":773094113280,"available":772662706176,"internally_reserved":0,"allocated":15310848,"data_stored":7764060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":415944242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.423213"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657252+0000","last_change":"2024-04-25T10:02:35.659995+0000","last_active":"2024-04-25T10:02:36.657252+0000","last_peered":"2024-04-25T10:02:36.657252+0000","last_clean":"2024-04-25T10:02:36.657252+0000","last_became_active":"2024-04-25T10:02:35.659715+0000","last_became_peered":"2024-04-25T10:02:35.659715+0000","last_unstale":"2024-04-25T10:02:36.657252+0000","last_undegraded":"2024-04-25T10:02:36.657252+0000","last_fullsized":"2024-04-25T10:02:36.657252+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:49.843714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642903+0000","last_change":"2024-04-25T10:02:35.657684+0000","last_active":"2024-04-25T10:02:36.642903+0000","last_peered":"2024-04-25T10:02:36.642903+0000","last_clean":"2024-04-25T10:02:36.642903+0000","last_became_active":"2024-04-25T10:02:35.657389+0000","last_became_peered":"2024-04-25T10:02:35.657389+0000","last_unstale":"2024-04-25T10:02:36.642903+0000","last_undegraded":"2024-04-25T10:02:36.642903+0000","last_fullsized":"2024-04-25T10:02:36.642903+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:02.976090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642586+0000","last_change":"2024-04-25T10:02:35.661627+0000","last_active":"2024-04-25T10:02:36.642586+0000","last_peered":"2024-04-25T10:02:36.642586+0000","last_clean":"2024-04-25T10:02:36.642586+0000","last_became_active":"2024-04-25T10:02:35.661363+0000","last_became_peered":"2024-04-25T10:02:35.661363+0000","last_unstale":"2024-04-25T10:02:36.642586+0000","last_undegraded":"2024-04-25T10:02:36.642586+0000","last_fullsized":"2024-04-25T10:02:36.642586+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:21.954841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737723+0000","last_change":"2024-04-25T10:02:35.656418+0000","last_active":"2024-04-25T10:02:36.737723+0000","last_peered":"2024-04-25T10:02:36.737723+0000","last_clean":"2024-04-25T10:02:36.737723+0000","last_became_active":"2024-04-25T10:02:35.656089+0000","last_became_peered":"2024-04-25T10:02:35.656089+0000","last_unstale":"2024-04-25T10:02:36.737723+0000","last_undegraded":"2024-04-25T10:02:36.737723+0000","last_fullsized":"2024-04-25T10:02:36.737723+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:09.551303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123400+0000","last_change":"2024-04-25T10:02:35.657356+0000","last_active":"2024-04-25T10:02:37.123400+0000","last_peered":"2024-04-25T10:02:37.123400+0000","last_clean":"2024-04-25T10:02:37.123400+0000","last_became_active":"2024-04-25T10:02:35.657060+0000","last_became_peered":"2024-04-25T10:02:35.657060+0000","last_unstale":"2024-04-25T10:02:37.123400+0000","last_undegraded":"2024-04-25T10:02:37.123400+0000","last_fullsized":"2024-04-25T10:02:37.123400+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:13.338984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123100+0000","last_change":"2024-04-25T10:02:35.661116+0000","last_active":"2024-04-25T10:02:37.123100+0000","last_peered":"2024-04-25T10:02:37.123100+0000","last_clean":"2024-04-25T10:02:37.123100+0000","last_became_active":"2024-04-25T10:02:35.658907+0000","last_became_peered":"2024-04-25T10:02:35.658907+0000","last_unstale":"2024-04-25T10:02:37.123100+0000","last_undegraded":"2024-04-25T10:02:37.123100+0000","last_fullsized":"2024-04-25T10:02:37.123100+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:49.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657165+0000","last_change":"2024-04-25T10:02:35.648401+0000","last_active":"2024-04-25T10:02:36.657165+0000","last_peered":"2024-04-25T10:02:36.657165+0000","last_clean":"2024-04-25T10:02:36.657165+0000","last_became_active":"2024-04-25T10:02:35.647639+0000","last_became_peered":"2024-04-25T10:02:35.647639+0000","last_unstale":"2024-04-25T10:02:36.657165+0000","last_undegraded":"2024-04-25T10:02:36.657165+0000","last_fullsized":"2024-04-25T10:02:36.657165+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:55.793255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641443+0000","last_change":"2024-04-25T10:02:35.662351+0000","last_active":"2024-04-25T10:02:36.641443+0000","last_peered":"2024-04-25T10:02:36.641443+0000","last_clean":"2024-04-25T10:02:36.641443+0000","last_became_active":"2024-04-25T10:02:35.662093+0000","last_became_peered":"2024-04-25T10:02:35.662093+0000","last_unstale":"2024-04-25T10:02:36.641443+0000","last_undegraded":"2024-04-25T10:02:36.641443+0000","last_fullsized":"2024-04-25T10:02:36.641443+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:52.211278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123243+0000","last_change":"2024-04-25T10:02:35.647699+0000","last_active":"2024-04-25T10:02:37.123243+0000","last_peered":"2024-04-25T10:02:37.123243+0000","last_clean":"2024-04-25T10:02:37.123243+0000","last_became_active":"2024-04-25T10:02:35.647393+0000","last_became_peered":"2024-04-25T10:02:35.647393+0000","last_unstale":"2024-04-25T10:02:37.123243+0000","last_undegraded":"2024-04-25T10:02:37.123243+0000","last_fullsized":"2024-04-25T10:02:37.123243+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:38.407520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123193+0000","last_change":"2024-04-25T10:02:35.652213+0000","last_active":"2024-04-25T10:02:37.123193+0000","last_peered":"2024-04-25T10:02:37.123193+0000","last_clean":"2024-04-25T10:02:37.123193+0000","last_became_active":"2024-04-25T10:02:35.651790+0000","last_became_peered":"2024-04-25T10:02:35.651790+0000","last_unstale":"2024-04-25T10:02:37.123193+0000","last_undegraded":"2024-04-25T10:02:37.123193+0000","last_fullsized":"2024-04-25T10:02:37.123193+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.678474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122281+0000","last_change":"2024-04-25T10:02:35.663399+0000","last_active":"2024-04-25T10:02:37.122281+0000","last_peered":"2024-04-25T10:02:37.122281+0000","last_clean":"2024-04-25T10:02:37.122281+0000","last_became_active":"2024-04-25T10:02:35.662169+0000","last_became_peered":"2024-04-25T10:02:35.662169+0000","last_unstale":"2024-04-25T10:02:37.122281+0000","last_undegraded":"2024-04-25T10:02:37.122281+0000","last_fullsized":"2024-04-25T10:02:37.122281+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:52.261567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737909+0000","last_change":"2024-04-25T10:02:35.663866+0000","last_active":"2024-04-25T10:02:36.737909+0000","last_peered":"2024-04-25T10:02:36.737909+0000","last_clean":"2024-04-25T10:02:36.737909+0000","last_became_active":"2024-04-25T10:02:35.663594+0000","last_became_peered":"2024-04-25T10:02:35.663594+0000","last_unstale":"2024-04-25T10:02:36.737909+0000","last_undegraded":"2024-04-25T10:02:36.737909+0000","last_fullsized":"2024-04-25T10:02:36.737909+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:32.612423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123087+0000","last_change":"2024-04-25T10:02:35.647593+0000","last_active":"2024-04-25T10:02:37.123087+0000","last_peered":"2024-04-25T10:02:37.123087+0000","last_clean":"2024-04-25T10:02:37.123087+0000","last_became_active":"2024-04-25T10:02:35.647002+0000","last_became_peered":"2024-04-25T10:02:35.647002+0000","last_unstale":"2024-04-25T10:02:37.123087+0000","last_undegraded":"2024-04-25T10:02:37.123087+0000","last_fullsized":"2024-04-25T10:02:37.123087+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:44.924061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737721+0000","last_change":"2024-04-25T10:02:35.667948+0000","last_active":"2024-04-25T10:02:36.737721+0000","last_peered":"2024-04-25T10:02:36.737721+0000","last_clean":"2024-04-25T10:02:36.737721+0000","last_became_active":"2024-04-25T10:02:35.666903+0000","last_became_peered":"2024-04-25T10:02:35.666903+0000","last_unstale":"2024-04-25T10:02:36.737721+0000","last_undegraded":"2024-04-25T10:02:36.737721+0000","last_fullsized":"2024-04-25T10:02:36.737721+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:39.535778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642621+0000","last_change":"2024-04-25T10:02:35.657089+0000","last_active":"2024-04-25T10:02:36.642621+0000","last_peered":"2024-04-25T10:02:36.642621+0000","last_clean":"2024-04-25T10:02:36.642621+0000","last_became_active":"2024-04-25T10:02:35.656757+0000","last_became_peered":"2024-04-25T10:02:35.656757+0000","last_unstale":"2024-04-25T10:02:36.642621+0000","last_undegraded":"2024-04-25T10:02:36.642621+0000","last_fullsized":"2024-04-25T10:02:36.642621+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:40.210106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.121824+0000","last_change":"2024-04-25T10:02:35.663002+0000","last_active":"2024-04-25T10:02:37.121824+0000","last_peered":"2024-04-25T10:02:37.121824+0000","last_clean":"2024-04-25T10:02:37.121824+0000","last_became_active":"2024-04-25T10:02:35.662713+0000","last_became_peered":"2024-04-25T10:02:35.662713+0000","last_unstale":"2024-04-25T10:02:37.121824+0000","last_undegraded":"2024-04-25T10:02:37.121824+0000","last_fullsized":"2024-04-25T10:02:37.121824+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:37.951795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.738064+0000","last_change":"2024-04-25T10:02:35.651304+0000","last_active":"2024-04-25T10:02:36.738064+0000","last_peered":"2024-04-25T10:02:36.738064+0000","last_clean":"2024-04-25T10:02:36.738064+0000","last_became_active":"2024-04-25T10:02:35.651042+0000","last_became_peered":"2024-04-25T10:02:35.651042+0000","last_unstale":"2024-04-25T10:02:36.738064+0000","last_undegraded":"2024-04-25T10:02:36.738064+0000","last_fullsized":"2024-04-25T10:02:36.738064+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:29.027829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122436+0000","last_change":"2024-04-25T10:02:35.647873+0000","last_active":"2024-04-25T10:02:37.122436+0000","last_peered":"2024-04-25T10:02:37.122436+0000","last_clean":"2024-04-25T10:02:37.122436+0000","last_became_active":"2024-04-25T10:02:35.647586+0000","last_became_peered":"2024-04-25T10:02:35.647586+0000","last_unstale":"2024-04-25T10:02:37.122436+0000","last_undegraded":"2024-04-25T10:02:37.122436+0000","last_fullsized":"2024-04-25T10:02:37.122436+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:35.757406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642192+0000","last_change":"2024-04-25T10:02:35.658850+0000","last_active":"2024-04-25T10:02:36.642192+0000","last_peered":"2024-04-25T10:02:36.642192+0000","last_clean":"2024-04-25T10:02:36.642192+0000","last_became_active":"2024-04-25T10:02:35.658604+0000","last_became_peered":"2024-04-25T10:02:35.658604+0000","last_unstale":"2024-04-25T10:02:36.642192+0000","last_undegraded":"2024-04-25T10:02:36.642192+0000","last_fullsized":"2024-04-25T10:02:36.642192+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:27.077595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122121+0000","last_change":"2024-04-25T10:02:35.654890+0000","last_active":"2024-04-25T10:02:37.122121+0000","last_peered":"2024-04-25T10:02:37.122121+0000","last_clean":"2024-04-25T10:02:37.122121+0000","last_became_active":"2024-04-25T10:02:35.654356+0000","last_became_peered":"2024-04-25T10:02:35.654356+0000","last_unstale":"2024-04-25T10:02:37.122121+0000","last_undegraded":"2024-04-25T10:02:37.122121+0000","last_fullsized":"2024-04-25T10:02:37.122121+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:13.210710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123309+0000","last_change":"2024-04-25T10:02:35.646990+0000","last_active":"2024-04-25T10:02:37.123309+0000","last_peered":"2024-04-25T10:02:37.123309+0000","last_clean":"2024-04-25T10:02:37.123309+0000","last_became_active":"2024-04-25T10:02:35.646569+0000","last_became_peered":"2024-04-25T10:02:35.646569+0000","last_unstale":"2024-04-25T10:02:37.123309+0000","last_undegraded":"2024-04-25T10:02:37.123309+0000","last_fullsized":"2024-04-25T10:02:37.123309+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:22.939497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657075+0000","last_change":"2024-04-25T10:02:35.648552+0000","last_active":"2024-04-25T10:02:36.657075+0000","last_peered":"2024-04-25T10:02:36.657075+0000","last_clean":"2024-04-25T10:02:36.657075+0000","last_became_active":"2024-04-25T10:02:35.648087+0000","last_became_peered":"2024-04-25T10:02:35.648087+0000","last_unstale":"2024-04-25T10:02:36.657075+0000","last_undegraded":"2024-04-25T10:02:36.657075+0000","last_fullsized":"2024-04-25T10:02:36.657075+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:00.614908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642003+0000","last_change":"2024-04-25T10:02:35.657929+0000","last_active":"2024-04-25T10:02:36.642003+0000","last_peered":"2024-04-25T10:02:36.642003+0000","last_clean":"2024-04-25T10:02:36.642003+0000","last_became_active":"2024-04-25T10:02:35.657559+0000","last_became_peered":"2024-04-25T10:02:35.657559+0000","last_unstale":"2024-04-25T10:02:36.642003+0000","last_undegraded":"2024-04-25T10:02:36.642003+0000","last_fullsized":"2024-04-25T10:02:36.642003+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:03.713031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122423+0000","last_change":"2024-04-25T10:02:35.655100+0000","last_active":"2024-04-25T10:02:37.122423+0000","last_peered":"2024-04-25T10:02:37.122423+0000","last_clean":"2024-04-25T10:02:37.122423+0000","last_became_active":"2024-04-25T10:02:35.654736+0000","last_became_peered":"2024-04-25T10:02:35.654736+0000","last_unstale":"2024-04-25T10:02:37.122423+0000","last_undegraded":"2024-04-25T10:02:37.122423+0000","last_fullsized":"2024-04-25T10:02:37.122423+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:05.575417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657300+0000","last_change":"2024-04-25T10:02:35.647105+0000","last_active":"2024-04-25T10:02:36.657300+0000","last_peered":"2024-04-25T10:02:36.657300+0000","last_clean":"2024-04-25T10:02:36.657300+0000","last_became_active":"2024-04-25T10:02:35.646832+0000","last_became_peered":"2024-04-25T10:02:35.646832+0000","last_unstale":"2024-04-25T10:02:36.657300+0000","last_undegraded":"2024-04-25T10:02:36.657300+0000","last_fullsized":"2024-04-25T10:02:36.657300+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:21.954841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657426+0000","last_change":"2024-04-25T10:02:35.647152+0000","last_active":"2024-04-25T10:02:36.657426+0000","last_peered":"2024-04-25T10:02:36.657426+0000","last_clean":"2024-04-25T10:02:36.657426+0000","last_became_active":"2024-04-25T10:02:35.646658+0000","last_became_peered":"2024-04-25T10:02:35.646658+0000","last_unstale":"2024-04-25T10:02:36.657426+0000","last_undegraded":"2024-04-25T10:02:36.657426+0000","last_fullsized":"2024-04-25T10:02:36.657426+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:40.210106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123176+0000","last_change":"2024-04-25T10:02:35.648468+0000","last_active":"2024-04-25T10:02:37.123176+0000","last_peered":"2024-04-25T10:02:37.123176+0000","last_clean":"2024-04-25T10:02:37.123176+0000","last_became_active":"2024-04-25T10:02:35.648175+0000","last_became_peered":"2024-04-25T10:02:35.648175+0000","last_unstale":"2024-04-25T10:02:37.123176+0000","last_undegraded":"2024-04-25T10:02:37.123176+0000","last_fullsized":"2024-04-25T10:02:37.123176+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:25.821546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122626+0000","last_change":"2024-04-25T10:02:35.661263+0000","last_active":"2024-04-25T10:02:37.122626+0000","last_peered":"2024-04-25T10:02:37.122626+0000","last_clean":"2024-04-25T10:02:37.122626+0000","last_became_active":"2024-04-25T10:02:35.660965+0000","last_became_peered":"2024-04-25T10:02:35.660965+0000","last_unstale":"2024-04-25T10:02:37.122626+0000","last_undegraded":"2024-04-25T10:02:37.122626+0000","last_fullsized":"2024-04-25T10:02:37.122626+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:07.360659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642652+0000","last_change":"2024-04-25T10:02:35.660659+0000","last_active":"2024-04-25T10:02:36.642652+0000","last_peered":"2024-04-25T10:02:36.642652+0000","last_clean":"2024-04-25T10:02:36.642652+0000","last_became_active":"2024-04-25T10:02:35.660328+0000","last_became_peered":"2024-04-25T10:02:35.660328+0000","last_unstale":"2024-04-25T10:02:36.642652+0000","last_undegraded":"2024-04-25T10:02:36.642652+0000","last_fullsized":"2024-04-25T10:02:36.642652+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:50.018601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122415+0000","last_change":"2024-04-25T10:02:35.651836+0000","last_active":"2024-04-25T10:02:37.122415+0000","last_peered":"2024-04-25T10:02:37.122415+0000","last_clean":"2024-04-25T10:02:37.122415+0000","last_became_active":"2024-04-25T10:02:35.651505+0000","last_became_peered":"2024-04-25T10:02:35.651505+0000","last_unstale":"2024-04-25T10:02:37.122415+0000","last_undegraded":"2024-04-25T10:02:37.122415+0000","last_fullsized":"2024-04-25T10:02:37.122415+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:41.293767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122767+0000","last_change":"2024-04-25T10:02:35.665285+0000","last_active":"2024-04-25T10:02:37.122767+0000","last_peered":"2024-04-25T10:02:37.122767+0000","last_clean":"2024-04-25T10:02:37.122767+0000","last_became_active":"2024-04-25T10:02:35.664866+0000","last_became_peered":"2024-04-25T10:02:35.664866+0000","last_unstale":"2024-04-25T10:02:37.122767+0000","last_undegraded":"2024-04-25T10:02:37.122767+0000","last_fullsized":"2024-04-25T10:02:37.122767+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:30.290870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642532+0000","last_change":"2024-04-25T10:02:35.661577+0000","last_active":"2024-04-25T10:02:36.642532+0000","last_peered":"2024-04-25T10:02:36.642532+0000","last_clean":"2024-04-25T10:02:36.642532+0000","last_became_active":"2024-04-25T10:02:35.661269+0000","last_became_peered":"2024-04-25T10:02:35.661269+0000","last_unstale":"2024-04-25T10:02:36.642532+0000","last_undegraded":"2024-04-25T10:02:36.642532+0000","last_fullsized":"2024-04-25T10:02:36.642532+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:54.266564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642730+0000","last_change":"2024-04-25T10:02:35.660836+0000","last_active":"2024-04-25T10:02:36.642730+0000","last_peered":"2024-04-25T10:02:36.642730+0000","last_clean":"2024-04-25T10:02:36.642730+0000","last_became_active":"2024-04-25T10:02:35.660507+0000","last_became_peered":"2024-04-25T10:02:35.660507+0000","last_unstale":"2024-04-25T10:02:36.642730+0000","last_undegraded":"2024-04-25T10:02:36.642730+0000","last_fullsized":"2024-04-25T10:02:36.642730+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:44.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"139'4","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123314+0000","last_change":"2024-04-25T10:02:34.643503+0000","last_active":"2024-04-25T10:02:37.123314+0000","last_peered":"2024-04-25T10:02:37.123314+0000","last_clean":"2024-04-25T10:02:37.123314+0000","last_became_active":"2024-04-25T10:02:34.643220+0000","last_became_peered":"2024-04-25T10:02:34.643220+0000","last_unstale":"2024-04-25T10:02:37.123314+0000","last_undegraded":"2024-04-25T10:02:37.123314+0000","last_fullsized":"2024-04-25T10:02:37.123314+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:28.023310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642745+0000","last_change":"2024-04-25T10:02:35.656969+0000","last_active":"2024-04-25T10:02:36.642745+0000","last_peered":"2024-04-25T10:02:36.642745+0000","last_clean":"2024-04-25T10:02:36.642745+0000","last_became_active":"2024-04-25T10:02:35.656678+0000","last_became_peered":"2024-04-25T10:02:35.656678+0000","last_unstale":"2024-04-25T10:02:36.642745+0000","last_undegraded":"2024-04-25T10:02:36.642745+0000","last_fullsized":"2024-04-25T10:02:36.642745+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:14.386733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641363+0000","last_change":"2024-04-25T10:02:34.640369+0000","last_active":"2024-04-25T10:02:36.641363+0000","last_peered":"2024-04-25T10:02:36.641363+0000","last_clean":"2024-04-25T10:02:36.641363+0000","last_became_active":"2024-04-25T10:02:34.638506+0000","last_became_peered":"2024-04-25T10:02:34.638506+0000","last_unstale":"2024-04-25T10:02:36.641363+0000","last_undegraded":"2024-04-25T10:02:36.641363+0000","last_fullsized":"2024-04-25T10:02:36.641363+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:43.280232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641933+0000","last_change":"2024-04-25T10:02:35.657040+0000","last_active":"2024-04-25T10:02:36.641933+0000","last_peered":"2024-04-25T10:02:36.641933+0000","last_clean":"2024-04-25T10:02:36.641933+0000","last_became_active":"2024-04-25T10:02:35.656671+0000","last_became_peered":"2024-04-25T10:02:35.656671+0000","last_unstale":"2024-04-25T10:02:36.641933+0000","last_undegraded":"2024-04-25T10:02:36.641933+0000","last_fullsized":"2024-04-25T10:02:36.641933+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:08.796466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642254+0000","last_change":"2024-04-25T10:02:34.633749+0000","last_active":"2024-04-25T10:02:36.642254+0000","last_peered":"2024-04-25T10:02:36.642254+0000","last_clean":"2024-04-25T10:02:36.642254+0000","last_became_active":"2024-04-25T10:02:34.633207+0000","last_became_peered":"2024-04-25T10:02:34.633207+0000","last_unstale":"2024-04-25T10:02:36.642254+0000","last_undegraded":"2024-04-25T10:02:36.642254+0000","last_fullsized":"2024-04-25T10:02:36.642254+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:39.205587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641890+0000","last_change":"2024-04-25T10:02:35.646474+0000","last_active":"2024-04-25T10:02:36.641890+0000","last_peered":"2024-04-25T10:02:36.641890+0000","last_clean":"2024-04-25T10:02:36.641890+0000","last_became_active":"2024-04-25T10:02:35.646192+0000","last_became_peered":"2024-04-25T10:02:35.646192+0000","last_unstale":"2024-04-25T10:02:36.641890+0000","last_undegraded":"2024-04-25T10:02:36.641890+0000","last_fullsized":"2024-04-25T10:02:36.641890+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:16.678474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642482+0000","last_change":"2024-04-25T10:02:34.640535+0000","last_active":"2024-04-25T10:02:36.642482+0000","last_peered":"2024-04-25T10:02:36.642482+0000","last_clean":"2024-04-25T10:02:36.642482+0000","last_became_active":"2024-04-25T10:02:34.640221+0000","last_became_peered":"2024-04-25T10:02:34.640221+0000","last_unstale":"2024-04-25T10:02:36.642482+0000","last_undegraded":"2024-04-25T10:02:36.642482+0000","last_fullsized":"2024-04-25T10:02:36.642482+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:51.347385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.121937+0000","last_change":"2024-04-25T10:02:35.655202+0000","last_active":"2024-04-25T10:02:37.121937+0000","last_peered":"2024-04-25T10:02:37.121937+0000","last_clean":"2024-04-25T10:02:37.121937+0000","last_became_active":"2024-04-25T10:02:35.654898+0000","last_became_peered":"2024-04-25T10:02:35.654898+0000","last_unstale":"2024-04-25T10:02:37.121937+0000","last_undegraded":"2024-04-25T10:02:37.121937+0000","last_fullsized":"2024-04-25T10:02:37.121937+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:15.741862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642697+0000","last_change":"2024-04-25T10:02:34.640849+0000","last_active":"2024-04-25T10:02:36.642697+0000","last_peered":"2024-04-25T10:02:36.642697+0000","last_clean":"2024-04-25T10:02:36.642697+0000","last_became_active":"2024-04-25T10:02:34.640263+0000","last_became_peered":"2024-04-25T10:02:34.640263+0000","last_unstale":"2024-04-25T10:02:36.642697+0000","last_undegraded":"2024-04-25T10:02:36.642697+0000","last_fullsized":"2024-04-25T10:02:36.642697+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:11.792032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737667+0000","last_change":"2024-04-25T10:02:35.654715+0000","last_active":"2024-04-25T10:02:36.737667+0000","last_peered":"2024-04-25T10:02:36.737667+0000","last_clean":"2024-04-25T10:02:36.737667+0000","last_became_active":"2024-04-25T10:02:35.654444+0000","last_became_peered":"2024-04-25T10:02:35.654444+0000","last_unstale":"2024-04-25T10:02:36.737667+0000","last_undegraded":"2024-04-25T10:02:36.737667+0000","last_fullsized":"2024-04-25T10:02:36.737667+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:40.880883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123163+0000","last_change":"2024-04-25T10:02:34.645621+0000","last_active":"2024-04-25T10:02:37.123163+0000","last_peered":"2024-04-25T10:02:37.123163+0000","last_clean":"2024-04-25T10:02:37.123163+0000","last_became_active":"2024-04-25T10:02:34.645059+0000","last_became_peered":"2024-04-25T10:02:34.645059+0000","last_unstale":"2024-04-25T10:02:37.123163+0000","last_undegraded":"2024-04-25T10:02:37.123163+0000","last_fullsized":"2024-04-25T10:02:37.123163+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:07.675769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642471+0000","last_change":"2024-04-25T10:02:35.656049+0000","last_active":"2024-04-25T10:02:36.642471+0000","last_peered":"2024-04-25T10:02:36.642471+0000","last_clean":"2024-04-25T10:02:36.642471+0000","last_became_active":"2024-04-25T10:02:35.655744+0000","last_became_peered":"2024-04-25T10:02:35.655744+0000","last_unstale":"2024-04-25T10:02:36.642471+0000","last_undegraded":"2024-04-25T10:02:36.642471+0000","last_fullsized":"2024-04-25T10:02:36.642471+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:55.793255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.656954+0000","last_change":"2024-04-25T10:02:34.635361+0000","last_active":"2024-04-25T10:02:36.656954+0000","last_peered":"2024-04-25T10:02:36.656954+0000","last_clean":"2024-04-25T10:02:36.656954+0000","last_became_active":"2024-04-25T10:02:34.634925+0000","last_became_peered":"2024-04-25T10:02:34.634925+0000","last_unstale":"2024-04-25T10:02:36.656954+0000","last_undegraded":"2024-04-25T10:02:36.656954+0000","last_fullsized":"2024-04-25T10:02:36.656954+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:51.347385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657177+0000","last_change":"2024-04-25T10:02:35.661977+0000","last_active":"2024-04-25T10:02:36.657177+0000","last_peered":"2024-04-25T10:02:36.657177+0000","last_clean":"2024-04-25T10:02:36.657177+0000","last_became_active":"2024-04-25T10:02:35.661711+0000","last_became_peered":"2024-04-25T10:02:35.661711+0000","last_unstale":"2024-04-25T10:02:36.657177+0000","last_undegraded":"2024-04-25T10:02:36.657177+0000","last_fullsized":"2024-04-25T10:02:36.657177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:41.842674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122609+0000","last_change":"2024-04-25T10:02:34.641294+0000","last_active":"2024-04-25T10:02:37.122609+0000","last_peered":"2024-04-25T10:02:37.122609+0000","last_clean":"2024-04-25T10:02:37.122609+0000","last_became_active":"2024-04-25T10:02:34.641008+0000","last_became_peered":"2024-04-25T10:02:34.641008+0000","last_unstale":"2024-04-25T10:02:37.122609+0000","last_undegraded":"2024-04-25T10:02:37.122609+0000","last_fullsized":"2024-04-25T10:02:37.122609+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:31.607904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642630+0000","last_change":"2024-04-25T10:02:35.657238+0000","last_active":"2024-04-25T10:02:36.642630+0000","last_peered":"2024-04-25T10:02:36.642630+0000","last_clean":"2024-04-25T10:02:36.642630+0000","last_became_active":"2024-04-25T10:02:35.656925+0000","last_became_peered":"2024-04-25T10:02:35.656925+0000","last_unstale":"2024-04-25T10:02:36.642630+0000","last_undegraded":"2024-04-25T10:02:36.642630+0000","last_fullsized":"2024-04-25T10:02:36.642630+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:50.871875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641948+0000","last_change":"2024-04-25T10:02:34.631919+0000","last_active":"2024-04-25T10:02:36.641948+0000","last_peered":"2024-04-25T10:02:36.641948+0000","last_clean":"2024-04-25T10:02:36.641948+0000","last_became_active":"2024-04-25T10:02:34.631620+0000","last_became_peered":"2024-04-25T10:02:34.631620+0000","last_unstale":"2024-04-25T10:02:36.641948+0000","last_undegraded":"2024-04-25T10:02:36.641948+0000","last_fullsized":"2024-04-25T10:02:36.641948+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:51.602050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122706+0000","last_change":"2024-04-25T10:02:35.646769+0000","last_active":"2024-04-25T10:02:37.122706+0000","last_peered":"2024-04-25T10:02:37.122706+0000","last_clean":"2024-04-25T10:02:37.122706+0000","last_became_active":"2024-04-25T10:02:35.646488+0000","last_became_peered":"2024-04-25T10:02:35.646488+0000","last_unstale":"2024-04-25T10:02:37.122706+0000","last_undegraded":"2024-04-25T10:02:37.122706+0000","last_fullsized":"2024-04-25T10:02:37.122706+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:18.430074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657552+0000","last_change":"2024-04-25T10:02:34.642849+0000","last_active":"2024-04-25T10:02:36.657552+0000","last_peered":"2024-04-25T10:02:36.657552+0000","last_clean":"2024-04-25T10:02:36.657552+0000","last_became_active":"2024-04-25T10:02:34.641713+0000","last_became_peered":"2024-04-25T10:02:34.641713+0000","last_unstale":"2024-04-25T10:02:36.657552+0000","last_undegraded":"2024-04-25T10:02:36.657552+0000","last_fullsized":"2024-04-25T10:02:36.657552+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:02.270880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122615+0000","last_change":"2024-04-25T10:02:35.646711+0000","last_active":"2024-04-25T10:02:37.122615+0000","last_peered":"2024-04-25T10:02:37.122615+0000","last_clean":"2024-04-25T10:02:37.122615+0000","last_became_active":"2024-04-25T10:02:35.646369+0000","last_became_peered":"2024-04-25T10:02:35.646369+0000","last_unstale":"2024-04-25T10:02:37.122615+0000","last_undegraded":"2024-04-25T10:02:37.122615+0000","last_fullsized":"2024-04-25T10:02:37.122615+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:39.109981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122562+0000","last_change":"2024-04-25T10:02:34.634059+0000","last_active":"2024-04-25T10:02:37.122562+0000","last_peered":"2024-04-25T10:02:37.122562+0000","last_clean":"2024-04-25T10:02:37.122562+0000","last_became_active":"2024-04-25T10:02:34.633764+0000","last_became_peered":"2024-04-25T10:02:34.633764+0000","last_unstale":"2024-04-25T10:02:37.122562+0000","last_undegraded":"2024-04-25T10:02:37.122562+0000","last_fullsized":"2024-04-25T10:02:37.122562+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:03.293271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642669+0000","last_change":"2024-04-25T10:02:35.663689+0000","last_active":"2024-04-25T10:02:36.642669+0000","last_peered":"2024-04-25T10:02:36.642669+0000","last_clean":"2024-04-25T10:02:36.642669+0000","last_became_active":"2024-04-25T10:02:35.663394+0000","last_became_peered":"2024-04-25T10:02:35.663394+0000","last_unstale":"2024-04-25T10:02:36.642669+0000","last_undegraded":"2024-04-25T10:02:36.642669+0000","last_fullsized":"2024-04-25T10:02:36.642669+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:41.592106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123369+0000","last_change":"2024-04-25T10:02:34.643724+0000","last_active":"2024-04-25T10:02:37.123369+0000","last_peered":"2024-04-25T10:02:37.123369+0000","last_clean":"2024-04-25T10:02:37.123369+0000","last_became_active":"2024-04-25T10:02:34.643418+0000","last_became_peered":"2024-04-25T10:02:34.643418+0000","last_unstale":"2024-04-25T10:02:37.123369+0000","last_undegraded":"2024-04-25T10:02:37.123369+0000","last_fullsized":"2024-04-25T10:02:37.123369+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:59.582687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642476+0000","last_change":"2024-04-25T10:02:35.656049+0000","last_active":"2024-04-25T10:02:36.642476+0000","last_peered":"2024-04-25T10:02:36.642476+0000","last_clean":"2024-04-25T10:02:36.642476+0000","last_became_active":"2024-04-25T10:02:35.655641+0000","last_became_peered":"2024-04-25T10:02:35.655641+0000","last_unstale":"2024-04-25T10:02:36.642476+0000","last_undegraded":"2024-04-25T10:02:36.642476+0000","last_fullsized":"2024-04-25T10:02:36.642476+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:41.842674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737651+0000","last_change":"2024-04-25T10:02:34.643903+0000","last_active":"2024-04-25T10:02:36.737651+0000","last_peered":"2024-04-25T10:02:36.737651+0000","last_clean":"2024-04-25T10:02:36.737651+0000","last_became_active":"2024-04-25T10:02:34.639682+0000","last_became_peered":"2024-04-25T10:02:34.639682+0000","last_unstale":"2024-04-25T10:02:36.737651+0000","last_undegraded":"2024-04-25T10:02:36.737651+0000","last_fullsized":"2024-04-25T10:02:36.737651+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:52.306679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123153+0000","last_change":"2024-04-25T10:02:35.649224+0000","last_active":"2024-04-25T10:02:37.123153+0000","last_peered":"2024-04-25T10:02:37.123153+0000","last_clean":"2024-04-25T10:02:37.123153+0000","last_became_active":"2024-04-25T10:02:35.648925+0000","last_became_peered":"2024-04-25T10:02:35.648925+0000","last_unstale":"2024-04-25T10:02:37.123153+0000","last_undegraded":"2024-04-25T10:02:37.123153+0000","last_fullsized":"2024-04-25T10:02:37.123153+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:20.331229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737701+0000","last_change":"2024-04-25T10:02:34.638730+0000","last_active":"2024-04-25T10:02:36.737701+0000","last_peered":"2024-04-25T10:02:36.737701+0000","last_clean":"2024-04-25T10:02:36.737701+0000","last_became_active":"2024-04-25T10:02:34.638315+0000","last_became_peered":"2024-04-25T10:02:34.638315+0000","last_unstale":"2024-04-25T10:02:36.737701+0000","last_undegraded":"2024-04-25T10:02:36.737701+0000","last_fullsized":"2024-04-25T10:02:36.737701+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:53.262045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642636+0000","last_change":"2024-04-25T10:02:35.657296+0000","last_active":"2024-04-25T10:02:36.642636+0000","last_peered":"2024-04-25T10:02:36.642636+0000","last_clean":"2024-04-25T10:02:36.642636+0000","last_became_active":"2024-04-25T10:02:35.655489+0000","last_became_peered":"2024-04-25T10:02:35.655489+0000","last_unstale":"2024-04-25T10:02:36.642636+0000","last_undegraded":"2024-04-25T10:02:36.642636+0000","last_fullsized":"2024-04-25T10:02:36.642636+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.606569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.656836+0000","last_change":"2024-04-25T10:02:34.637023+0000","last_active":"2024-04-25T10:02:36.656836+0000","last_peered":"2024-04-25T10:02:36.656836+0000","last_clean":"2024-04-25T10:02:36.656836+0000","last_became_active":"2024-04-25T10:02:34.636234+0000","last_became_peered":"2024-04-25T10:02:34.636234+0000","last_unstale":"2024-04-25T10:02:36.656836+0000","last_undegraded":"2024-04-25T10:02:36.656836+0000","last_fullsized":"2024-04-25T10:02:36.656836+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:40.587587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641621+0000","last_change":"2024-04-25T10:02:35.653013+0000","last_active":"2024-04-25T10:02:36.641621+0000","last_peered":"2024-04-25T10:02:36.641621+0000","last_clean":"2024-04-25T10:02:36.641621+0000","last_became_active":"2024-04-25T10:02:35.651592+0000","last_became_peered":"2024-04-25T10:02:35.651592+0000","last_unstale":"2024-04-25T10:02:36.641621+0000","last_undegraded":"2024-04-25T10:02:36.641621+0000","last_fullsized":"2024-04-25T10:02:36.641621+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:25.821546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122485+0000","last_change":"2024-04-25T10:02:34.636464+0000","last_active":"2024-04-25T10:02:37.122485+0000","last_peered":"2024-04-25T10:02:37.122485+0000","last_clean":"2024-04-25T10:02:37.122485+0000","last_became_active":"2024-04-25T10:02:34.636178+0000","last_became_peered":"2024-04-25T10:02:34.636178+0000","last_unstale":"2024-04-25T10:02:37.122485+0000","last_undegraded":"2024-04-25T10:02:37.122485+0000","last_fullsized":"2024-04-25T10:02:37.122485+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:00.297330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642302+0000","last_change":"2024-04-25T10:02:35.659218+0000","last_active":"2024-04-25T10:02:36.642302+0000","last_peered":"2024-04-25T10:02:36.642302+0000","last_clean":"2024-04-25T10:02:36.642302+0000","last_became_active":"2024-04-25T10:02:35.658935+0000","last_became_peered":"2024-04-25T10:02:35.658935+0000","last_unstale":"2024-04-25T10:02:36.642302+0000","last_undegraded":"2024-04-25T10:02:36.642302+0000","last_fullsized":"2024-04-25T10:02:36.642302+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:22.939497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641481+0000","last_change":"2024-04-25T10:02:34.640607+0000","last_active":"2024-04-25T10:02:36.641481+0000","last_peered":"2024-04-25T10:02:36.641481+0000","last_clean":"2024-04-25T10:02:36.641481+0000","last_became_active":"2024-04-25T10:02:34.639879+0000","last_became_peered":"2024-04-25T10:02:34.639879+0000","last_unstale":"2024-04-25T10:02:36.641481+0000","last_undegraded":"2024-04-25T10:02:36.641481+0000","last_fullsized":"2024-04-25T10:02:36.641481+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:13.382214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657446+0000","last_change":"2024-04-25T10:02:35.666616+0000","last_active":"2024-04-25T10:02:36.657446+0000","last_peered":"2024-04-25T10:02:36.657446+0000","last_clean":"2024-04-25T10:02:36.657446+0000","last_became_active":"2024-04-25T10:02:35.666259+0000","last_became_peered":"2024-04-25T10:02:35.666259+0000","last_unstale":"2024-04-25T10:02:36.657446+0000","last_undegraded":"2024-04-25T10:02:36.657446+0000","last_fullsized":"2024-04-25T10:02:36.657446+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.606569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642739+0000","last_change":"2024-04-25T10:02:34.643130+0000","last_active":"2024-04-25T10:02:36.642739+0000","last_peered":"2024-04-25T10:02:36.642739+0000","last_clean":"2024-04-25T10:02:36.642739+0000","last_became_active":"2024-04-25T10:02:34.642723+0000","last_became_peered":"2024-04-25T10:02:34.642723+0000","last_unstale":"2024-04-25T10:02:36.642739+0000","last_undegraded":"2024-04-25T10:02:36.642739+0000","last_fullsized":"2024-04-25T10:02:36.642739+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:04.514370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657398+0000","last_change":"2024-04-25T10:02:35.651697+0000","last_active":"2024-04-25T10:02:36.657398+0000","last_peered":"2024-04-25T10:02:36.657398+0000","last_clean":"2024-04-25T10:02:36.657398+0000","last_became_active":"2024-04-25T10:02:35.651448+0000","last_became_peered":"2024-04-25T10:02:35.651448+0000","last_unstale":"2024-04-25T10:02:36.657398+0000","last_undegraded":"2024-04-25T10:02:36.657398+0000","last_fullsized":"2024-04-25T10:02:36.657398+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:26.693313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737801+0000","last_change":"2024-04-25T10:02:34.643261+0000","last_active":"2024-04-25T10:02:36.737801+0000","last_peered":"2024-04-25T10:02:36.737801+0000","last_clean":"2024-04-25T10:02:36.737801+0000","last_became_active":"2024-04-25T10:02:34.640276+0000","last_became_peered":"2024-04-25T10:02:34.640276+0000","last_unstale":"2024-04-25T10:02:36.737801+0000","last_undegraded":"2024-04-25T10:02:36.737801+0000","last_fullsized":"2024-04-25T10:02:36.737801+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:02.708512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641558+0000","last_change":"2024-04-25T10:02:35.662112+0000","last_active":"2024-04-25T10:02:36.641558+0000","last_peered":"2024-04-25T10:02:36.641558+0000","last_clean":"2024-04-25T10:02:36.641558+0000","last_became_active":"2024-04-25T10:02:35.661263+0000","last_became_peered":"2024-04-25T10:02:35.661263+0000","last_unstale":"2024-04-25T10:02:36.641558+0000","last_undegraded":"2024-04-25T10:02:36.641558+0000","last_fullsized":"2024-04-25T10:02:36.641558+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:20.331229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122029+0000","last_change":"2024-04-25T10:02:34.641857+0000","last_active":"2024-04-25T10:02:37.122029+0000","last_peered":"2024-04-25T10:02:37.122029+0000","last_clean":"2024-04-25T10:02:37.122029+0000","last_became_active":"2024-04-25T10:02:34.641529+0000","last_became_peered":"2024-04-25T10:02:34.641529+0000","last_unstale":"2024-04-25T10:02:37.122029+0000","last_undegraded":"2024-04-25T10:02:37.122029+0000","last_fullsized":"2024-04-25T10:02:37.122029+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:22.663870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642611+0000","last_change":"2024-04-25T10:02:35.661000+0000","last_active":"2024-04-25T10:02:36.642611+0000","last_peered":"2024-04-25T10:02:36.642611+0000","last_clean":"2024-04-25T10:02:36.642611+0000","last_became_active":"2024-04-25T10:02:35.659838+0000","last_became_peered":"2024-04-25T10:02:35.659838+0000","last_unstale":"2024-04-25T10:02:36.642611+0000","last_undegraded":"2024-04-25T10:02:36.642611+0000","last_fullsized":"2024-04-25T10:02:36.642611+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:26.693313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641563+0000","last_change":"2024-04-25T10:02:34.636630+0000","last_active":"2024-04-25T10:02:36.641563+0000","last_peered":"2024-04-25T10:02:36.641563+0000","last_clean":"2024-04-25T10:02:36.641563+0000","last_became_active":"2024-04-25T10:02:34.635854+0000","last_became_peered":"2024-04-25T10:02:34.635854+0000","last_unstale":"2024-04-25T10:02:36.641563+0000","last_undegraded":"2024-04-25T10:02:36.641563+0000","last_fullsized":"2024-04-25T10:02:36.641563+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:28.821370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642019+0000","last_change":"2024-04-25T10:02:35.654507+0000","last_active":"2024-04-25T10:02:36.642019+0000","last_peered":"2024-04-25T10:02:36.642019+0000","last_clean":"2024-04-25T10:02:36.642019+0000","last_became_active":"2024-04-25T10:02:35.652135+0000","last_became_peered":"2024-04-25T10:02:35.652135+0000","last_unstale":"2024-04-25T10:02:36.642019+0000","last_undegraded":"2024-04-25T10:02:36.642019+0000","last_fullsized":"2024-04-25T10:02:36.642019+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:38.407520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.121809+0000","last_change":"2024-04-25T10:02:34.641092+0000","last_active":"2024-04-25T10:02:37.121809+0000","last_peered":"2024-04-25T10:02:37.121809+0000","last_clean":"2024-04-25T10:02:37.121809+0000","last_became_active":"2024-04-25T10:02:34.640753+0000","last_became_peered":"2024-04-25T10:02:34.640753+0000","last_unstale":"2024-04-25T10:02:37.121809+0000","last_undegraded":"2024-04-25T10:02:37.121809+0000","last_fullsized":"2024-04-25T10:02:37.121809+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:51.392051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642759+0000","last_change":"2024-04-25T10:02:35.658419+0000","last_active":"2024-04-25T10:02:36.642759+0000","last_peered":"2024-04-25T10:02:36.642759+0000","last_clean":"2024-04-25T10:02:36.642759+0000","last_became_active":"2024-04-25T10:02:35.658109+0000","last_became_peered":"2024-04-25T10:02:35.658109+0000","last_unstale":"2024-04-25T10:02:36.642759+0000","last_undegraded":"2024-04-25T10:02:36.642759+0000","last_fullsized":"2024-04-25T10:02:36.642759+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.699760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642737+0000","last_change":"2024-04-25T10:02:34.635245+0000","last_active":"2024-04-25T10:02:36.642737+0000","last_peered":"2024-04-25T10:02:36.642737+0000","last_clean":"2024-04-25T10:02:36.642737+0000","last_became_active":"2024-04-25T10:02:34.634933+0000","last_became_peered":"2024-04-25T10:02:34.634933+0000","last_unstale":"2024-04-25T10:02:36.642737+0000","last_undegraded":"2024-04-25T10:02:36.642737+0000","last_fullsized":"2024-04-25T10:02:36.642737+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:01.590104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123150+0000","last_change":"2024-04-25T10:02:35.661913+0000","last_active":"2024-04-25T10:02:37.123150+0000","last_peered":"2024-04-25T10:02:37.123150+0000","last_clean":"2024-04-25T10:02:37.123150+0000","last_became_active":"2024-04-25T10:02:35.661180+0000","last_became_peered":"2024-04-25T10:02:35.661180+0000","last_unstale":"2024-04-25T10:02:37.123150+0000","last_undegraded":"2024-04-25T10:02:37.123150+0000","last_fullsized":"2024-04-25T10:02:37.123150+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:48.022935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.738046+0000","last_change":"2024-04-25T10:02:34.639264+0000","last_active":"2024-04-25T10:02:36.738046+0000","last_peered":"2024-04-25T10:02:36.738046+0000","last_clean":"2024-04-25T10:02:36.738046+0000","last_became_active":"2024-04-25T10:02:34.638959+0000","last_became_peered":"2024-04-25T10:02:34.638959+0000","last_unstale":"2024-04-25T10:02:36.738046+0000","last_undegraded":"2024-04-25T10:02:36.738046+0000","last_fullsized":"2024-04-25T10:02:36.738046+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:49.867356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737946+0000","last_change":"2024-04-25T10:02:35.664607+0000","last_active":"2024-04-25T10:02:36.737946+0000","last_peered":"2024-04-25T10:02:36.737946+0000","last_clean":"2024-04-25T10:02:36.737946+0000","last_became_active":"2024-04-25T10:02:35.663577+0000","last_became_peered":"2024-04-25T10:02:35.663577+0000","last_unstale":"2024-04-25T10:02:36.737946+0000","last_undegraded":"2024-04-25T10:02:36.737946+0000","last_fullsized":"2024-04-25T10:02:36.737946+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:08.680288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657645+0000","last_change":"2024-04-25T10:02:34.642629+0000","last_active":"2024-04-25T10:02:36.657645+0000","last_peered":"2024-04-25T10:02:36.657645+0000","last_clean":"2024-04-25T10:02:36.657645+0000","last_became_active":"2024-04-25T10:02:34.642336+0000","last_became_peered":"2024-04-25T10:02:34.642336+0000","last_unstale":"2024-04-25T10:02:36.657645+0000","last_undegraded":"2024-04-25T10:02:36.657645+0000","last_fullsized":"2024-04-25T10:02:36.657645+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:51.691612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657580+0000","last_change":"2024-04-25T10:02:35.658702+0000","last_active":"2024-04-25T10:02:36.657580+0000","last_peered":"2024-04-25T10:02:36.657580+0000","last_clean":"2024-04-25T10:02:36.657580+0000","last_became_active":"2024-04-25T10:02:35.658418+0000","last_became_peered":"2024-04-25T10:02:35.658418+0000","last_unstale":"2024-04-25T10:02:36.657580+0000","last_undegraded":"2024-04-25T10:02:36.657580+0000","last_fullsized":"2024-04-25T10:02:36.657580+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:44.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642795+0000","last_change":"2024-04-25T10:02:34.633345+0000","last_active":"2024-04-25T10:02:36.642795+0000","last_peered":"2024-04-25T10:02:36.642795+0000","last_clean":"2024-04-25T10:02:36.642795+0000","last_became_active":"2024-04-25T10:02:34.632906+0000","last_became_peered":"2024-04-25T10:02:34.632906+0000","last_unstale":"2024-04-25T10:02:36.642795+0000","last_undegraded":"2024-04-25T10:02:36.642795+0000","last_fullsized":"2024-04-25T10:02:36.642795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:14.512825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.123256+0000","last_change":"2024-04-25T10:02:35.655580+0000","last_active":"2024-04-25T10:02:37.123256+0000","last_peered":"2024-04-25T10:02:37.123256+0000","last_clean":"2024-04-25T10:02:37.123256+0000","last_became_active":"2024-04-25T10:02:35.655335+0000","last_became_peered":"2024-04-25T10:02:35.655335+0000","last_unstale":"2024-04-25T10:02:37.123256+0000","last_undegraded":"2024-04-25T10:02:37.123256+0000","last_fullsized":"2024-04-25T10:02:37.123256+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:27.077595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122983+0000","last_change":"2024-04-25T10:02:34.644163+0000","last_active":"2024-04-25T10:02:37.122983+0000","last_peered":"2024-04-25T10:02:37.122983+0000","last_clean":"2024-04-25T10:02:37.122983+0000","last_became_active":"2024-04-25T10:02:34.643892+0000","last_became_peered":"2024-04-25T10:02:34.643892+0000","last_unstale":"2024-04-25T10:02:37.122983+0000","last_undegraded":"2024-04-25T10:02:37.122983+0000","last_fullsized":"2024-04-25T10:02:37.122983+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:02.661463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642721+0000","last_change":"2024-04-25T10:02:35.656319+0000","last_active":"2024-04-25T10:02:36.642721+0000","last_peered":"2024-04-25T10:02:36.642721+0000","last_clean":"2024-04-25T10:02:36.642721+0000","last_became_active":"2024-04-25T10:02:35.656040+0000","last_became_peered":"2024-04-25T10:02:35.656040+0000","last_unstale":"2024-04-25T10:02:36.642721+0000","last_undegraded":"2024-04-25T10:02:36.642721+0000","last_fullsized":"2024-04-25T10:02:36.642721+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:52.351904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657055+0000","last_change":"2024-04-25T10:02:34.642345+0000","last_active":"2024-04-25T10:02:36.657055+0000","last_peered":"2024-04-25T10:02:36.657055+0000","last_clean":"2024-04-25T10:02:36.657055+0000","last_became_active":"2024-04-25T10:02:34.641888+0000","last_became_peered":"2024-04-25T10:02:34.641888+0000","last_unstale":"2024-04-25T10:02:36.657055+0000","last_undegraded":"2024-04-25T10:02:36.657055+0000","last_fullsized":"2024-04-25T10:02:36.657055+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:14.512825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.737663+0000","last_change":"2024-04-25T10:02:35.667016+0000","last_active":"2024-04-25T10:02:36.737663+0000","last_peered":"2024-04-25T10:02:36.737663+0000","last_clean":"2024-04-25T10:02:36.737663+0000","last_became_active":"2024-04-25T10:02:35.666736+0000","last_became_peered":"2024-04-25T10:02:35.666736+0000","last_unstale":"2024-04-25T10:02:36.737663+0000","last_undegraded":"2024-04-25T10:02:36.737663+0000","last_fullsized":"2024-04-25T10:02:36.737663+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:35.875909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122416+0000","last_change":"2024-04-25T10:02:34.642788+0000","last_active":"2024-04-25T10:02:37.122416+0000","last_peered":"2024-04-25T10:02:37.122416+0000","last_clean":"2024-04-25T10:02:37.122416+0000","last_became_active":"2024-04-25T10:02:34.642492+0000","last_became_peered":"2024-04-25T10:02:34.642492+0000","last_unstale":"2024-04-25T10:02:37.122416+0000","last_undegraded":"2024-04-25T10:02:37.122416+0000","last_fullsized":"2024-04-25T10:02:37.122416+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:15.889113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642559+0000","last_change":"2024-04-25T10:02:35.660534+0000","last_active":"2024-04-25T10:02:36.642559+0000","last_peered":"2024-04-25T10:02:36.642559+0000","last_clean":"2024-04-25T10:02:36.642559+0000","last_became_active":"2024-04-25T10:02:35.660192+0000","last_became_peered":"2024-04-25T10:02:35.660192+0000","last_unstale":"2024-04-25T10:02:36.642559+0000","last_undegraded":"2024-04-25T10:02:36.642559+0000","last_fullsized":"2024-04-25T10:02:36.642559+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:49.843714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.657076+0000","last_change":"2024-04-25T10:02:34.642329+0000","last_active":"2024-04-25T10:02:36.657076+0000","last_peered":"2024-04-25T10:02:36.657076+0000","last_clean":"2024-04-25T10:02:36.657076+0000","last_became_active":"2024-04-25T10:02:34.641905+0000","last_became_peered":"2024-04-25T10:02:34.641905+0000","last_unstale":"2024-04-25T10:02:36.657076+0000","last_undegraded":"2024-04-25T10:02:36.657076+0000","last_fullsized":"2024-04-25T10:02:36.657076+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:04.514370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642634+0000","last_change":"2024-04-25T10:02:35.660001+0000","last_active":"2024-04-25T10:02:36.642634+0000","last_peered":"2024-04-25T10:02:36.642634+0000","last_clean":"2024-04-25T10:02:36.642634+0000","last_became_active":"2024-04-25T10:02:35.659323+0000","last_became_peered":"2024-04-25T10:02:35.659323+0000","last_unstale":"2024-04-25T10:02:36.642634+0000","last_undegraded":"2024-04-25T10:02:36.642634+0000","last_fullsized":"2024-04-25T10:02:36.642634+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:01.934600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641613+0000","last_change":"2024-04-25T10:02:34.634113+0000","last_active":"2024-04-25T10:02:36.641613+0000","last_peered":"2024-04-25T10:02:36.641613+0000","last_clean":"2024-04-25T10:02:36.641613+0000","last_became_active":"2024-04-25T10:02:34.633573+0000","last_became_peered":"2024-04-25T10:02:34.633573+0000","last_unstale":"2024-04-25T10:02:36.641613+0000","last_undegraded":"2024-04-25T10:02:36.641613+0000","last_fullsized":"2024-04-25T10:02:36.641613+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:25.688794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.642087+0000","last_change":"2024-04-25T10:02:35.657180+0000","last_active":"2024-04-25T10:02:36.642087+0000","last_peered":"2024-04-25T10:02:36.642087+0000","last_clean":"2024-04-25T10:02:36.642087+0000","last_became_active":"2024-04-25T10:02:35.656445+0000","last_became_peered":"2024-04-25T10:02:35.656445+0000","last_unstale":"2024-04-25T10:02:36.642087+0000","last_undegraded":"2024-04-25T10:02:36.642087+0000","last_fullsized":"2024-04-25T10:02:36.642087+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:34.619840+0000","last_clean_scrub_stamp":"2024-04-25T10:02:34.619840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:53.311199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"139'6","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:37.122039+0000","last_change":"2024-04-25T10:02:34.641165+0000","last_active":"2024-04-25T10:02:37.122039+0000","last_peered":"2024-04-25T10:02:37.122039+0000","last_clean":"2024-04-25T10:02:37.122039+0000","last_became_active":"2024-04-25T10:02:34.640851+0000","last_became_peered":"2024-04-25T10:02:34.640851+0000","last_unstale":"2024-04-25T10:02:37.122039+0000","last_undegraded":"2024-04-25T10:02:37.122039+0000","last_fullsized":"2024-04-25T10:02:37.122039+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:33.615321+0000","last_clean_scrub_stamp":"2024-04-25T10:02:33.615321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:07.068342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":414,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:02:36.641721+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:02:36.641721+0000","last_peered":"2024-04-25T10:02:36.641721+0000","last_clean":"2024-04-25T10:02:36.641721+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:02:36.641721+0000","last_undegraded":"2024-04-25T10:02:36.641721+0000","last_fullsized":"2024-04-25T10:02:36.641721+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51532,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2355200,"data_stored":1412703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50948,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320892,"statfs":{"total":96636764160,"available":96584593408,"internally_reserved":0,"allocated":1761280,"data_stored":822383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46792,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325048,"statfs":{"total":96636764160,"available":96588849152,"internally_reserved":0,"allocated":1757184,"data_stored":822313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55120,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316720,"statfs":{"total":96636764160,"available":96580321280,"internally_reserved":0,"allocated":1769472,"data_stored":823925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59280,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312560,"statfs":{"total":96636764160,"available":96576061440,"internally_reserved":0,"allocated":1765376,"data_stored":822929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50960,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320880,"statfs":{"total":96636764160,"available":96584581120,"internally_reserved":0,"allocated":1765376,"data_stored":822425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50952,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320888,"statfs":{"total":96636764160,"available":96584589312,"internally_reserved":0,"allocated":1761280,"data_stored":822383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55712,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316128,"statfs":{"total":96636764160,"available":96579715072,"internally_reserved":0,"allocated":2375680,"data_stored":1414999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:40.094 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:02:40.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:02:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.196+0000 7f1c2ce9e640 1 Processor -- start 2024-04-25T10:02:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.196+0000 7f1c2ce9e640 1 -- start start 2024-04-25T10:02:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.197+0000 7f1c2ce9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1c2815dc20 0x7f1c280ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.197+0000 7f1c2ce9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c280d32f0 0x7f1c280cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.197+0000 7f1c2ce9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2815e000 con 0x7f1c28161710 2024-04-25T10:02:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.197+0000 7f1c2ce9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c280d1de0 con 0x7f1c280d32f0 2024-04-25T10:02:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.197+0000 7f1c2ce9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c28161c60 con 0x7f1c2815dc20 2024-04-25T10:02:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.197+0000 7f1c26d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c280d32f0 0x7f1c280cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.197+0000 7f1c25d9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1c28161710 0x7f1c2806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54876/0 (socket says 172.21.15.19:54876) 2024-04-25T10:02:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.197+0000 7f1c25d9b640 1 -- 172.21.15.19:0/3669962011 learned_addr learned my addr 172.21.15.19:0/3669962011 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.197+0000 7f1c2659c640 1 --2- 172.21.15.19:0/3669962011 >> v2:172.21.15.112:3300/0 conn(0x7f1c2815dc20 0x7f1c280ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.197+0000 7f1c26d9d640 1 -- 172.21.15.19:0/3669962011 >> v2:172.21.15.112:3300/0 conn(0x7f1c2815dc20 msgr2=0x7f1c280ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.197+0000 7f1c26d9d640 1 --2- 172.21.15.19:0/3669962011 >> v2:172.21.15.112:3300/0 conn(0x7f1c2815dc20 0x7f1c280ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.198+0000 7f1c26d9d640 1 -- 172.21.15.19:0/3669962011 >> v1:172.21.15.19:6789/0 conn(0x7f1c28161710 legacy=0x7f1c2806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.198+0000 7f1c26d9d640 1 -- 172.21.15.19:0/3669962011 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c28161f40 con 0x7f1c280d32f0 2024-04-25T10:02:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.198+0000 7f1c2559a640 1 -- 172.21.15.19:0/3669962011 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923049930 0 0) 0x7f1c2815e000 con 0x7f1c28161710 2024-04-25T10:02:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.198+0000 7f1c26d9d640 1 --2- 172.21.15.19:0/3669962011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c280d32f0 0x7f1c280cebb0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f1c280d1520 tx=0x7f1c2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c893a7d589b7c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.198+0000 7f1c2559a640 1 -- 172.21.15.19:0/3669962011 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20037070 con 0x7f1c280d32f0 2024-04-25T10:02:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.198+0000 7f1c2559a640 1 -- 172.21.15.19:0/3669962011 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20032440 con 0x7f1c280d32f0 2024-04-25T10:02:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.198+0000 7f1c2559a640 1 -- 172.21.15.19:0/3669962011 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20032810 con 0x7f1c280d32f0 2024-04-25T10:02:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.199+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/3669962011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c280d32f0 msgr2=0x7f1c280cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.199+0000 7f1c2ce9e640 1 --2- 172.21.15.19:0/3669962011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c280d32f0 0x7f1c280cebb0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f1c280d1520 tx=0x7f1c2002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.199+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/3669962011 shutdown_connections 2024-04-25T10:02:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.199+0000 7f1c2ce9e640 1 --2- 172.21.15.19:0/3669962011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c280d32f0 0x7f1c280cebb0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.199+0000 7f1c2ce9e640 1 --2- 172.21.15.19:0/3669962011 >> v2:172.21.15.112:3300/0 conn(0x7f1c2815dc20 0x7f1c280ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.199+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/3669962011 >> 172.21.15.19:0/3669962011 conn(0x7f1c280d0420 msgr2=0x7f1c280d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.199+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/3669962011 shutdown_connections 2024-04-25T10:02:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.199+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/3669962011 wait complete. 2024-04-25T10:02:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.200+0000 7f1c2ce9e640 1 Processor -- start 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.201+0000 7f1c2ce9e640 1 -- start start 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.201+0000 7f1c2ce9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c280d32f0 0x7f1c280aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.201+0000 7f1c2ce9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c2815dc20 0x7f1c280aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.201+0000 7f1c2ce9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28161710 0x7f1c281f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.201+0000 7f1c2ce9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c280b5e50 con 0x7f1c28161710 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.201+0000 7f1c2ce9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28162500 con 0x7f1c280d32f0 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.201+0000 7f1c2ce9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c280b5fd0 con 0x7f1c2815dc20 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.201+0000 7f1c26d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28161710 0x7f1c281f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.201+0000 7f1c26d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28161710 0x7f1c281f3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41022/0 (socket says 172.21.15.19:41022) 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.201+0000 7f1c26d9d640 1 -- 172.21.15.19:0/2746005224 learned_addr learned my addr 172.21.15.19:0/2746005224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c2659c640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c280d32f0 0x7f1c280aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c25d9b640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c2815dc20 0x7f1c280aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c26d9d640 1 -- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c2815dc20 msgr2=0x7f1c280aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c26d9d640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c2815dc20 0x7f1c280aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c26d9d640 1 -- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c280d32f0 msgr2=0x7f1c280aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c26d9d640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c280d32f0 0x7f1c280aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c26d9d640 1 -- 172.21.15.19:0/2746005224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c280cc600 con 0x7f1c28161710 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c2659c640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c280d32f0 0x7f1c280aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c25d9b640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c2815dc20 0x7f1c280aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c26d9d640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28161710 0x7f1c281f3ba0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f1c20035040 tx=0x7f1c200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c1effd640 1 -- 172.21.15.19:0/2746005224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20037050 con 0x7f1c28161710 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c1effd640 1 -- 172.21.15.19:0/2746005224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20038040 con 0x7f1c28161710 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/2746005224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c280cc8a0 con 0x7f1c28161710 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c1effd640 1 -- 172.21.15.19:0/2746005224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20004450 con 0x7f1c28161710 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.202+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/2746005224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c280cce60 con 0x7f1c28161710 2024-04-25T10:02:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.203+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/2746005224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c280d16b0 con 0x7f1c28161710 2024-04-25T10:02:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.204+0000 7f1c1effd640 1 -- 172.21.15.19:0/2746005224 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c2003c6d0 con 0x7f1c28161710 2024-04-25T10:02:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.204+0000 7f1c1effd640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0806b4d0 0x7f1c0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.204+0000 7f1c1effd640 1 -- 172.21.15.19:0/2746005224 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c200aad80 con 0x7f1c28161710 2024-04-25T10:02:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.204+0000 7f1c2659c640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0806b4d0 0x7f1c0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.205+0000 7f1c2659c640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0806b4d0 0x7f1c0806d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1c280d3260 tx=0x7f1c1800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.208+0000 7f1c1effd640 1 -- 172.21.15.19:0/2746005224 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c20079b90 con 0x7f1c28161710 2024-04-25T10:02:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.460+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/2746005224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1c280739e0 con 0x7f1c28161710 2024-04-25T10:02:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.462+0000 7f1c1effd640 1 -- 172.21.15.19:0/2746005224 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1c20079490 con 0x7f1c28161710 2024-04-25T10:02:40.468 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T10:02:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T10:02:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T10:02:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T10:02:40.468 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:02:40.468 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T10:02:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0806b4d0 msgr2=0x7f1c0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0806b4d0 0x7f1c0806d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1c280d3260 tx=0x7f1c1800b040 comp rx=0 tx=0).stop 2024-04-25T10:02:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/2746005224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28161710 msgr2=0x7f1c281f3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28161710 0x7f1c281f3ba0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f1c20035040 tx=0x7f1c200046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/2746005224 shutdown_connections 2024-04-25T10:02:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c0806b4d0 0x7f1c0806d990 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c28161710 0x7f1c281f3ba0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c2815dc20 0x7f1c280aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 --2- 172.21.15.19:0/2746005224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c280d32f0 0x7f1c280aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/2746005224 >> 172.21.15.19:0/2746005224 conn(0x7f1c280d0420 msgr2=0x7f1c2815cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/2746005224 shutdown_connections 2024-04-25T10:02:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.464+0000 7f1c2ce9e640 1 -- 172.21.15.19:0/2746005224 wait complete. 2024-04-25T10:02:40.481 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.576+0000 7f0c5ecb6640 1 Processor -- start 2024-04-25T10:02:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.576+0000 7f0c5ecb6640 1 -- start start 2024-04-25T10:02:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c5ecb6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c580d2e90 0x7f0c580d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c5ecb6640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c580b9940 con 0x7f0c5815dd30 2024-04-25T10:02:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c5ecb6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c580d1df0 con 0x7f0c580d0900 2024-04-25T10:02:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c5ecb6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580d0d70 con 0x7f0c580d2e90 2024-04-25T10:02:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c5ca52640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0c5815dd30 0x7f0c580b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54888/0 (socket says 172.21.15.19:54888) 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c5ca52640 1 -- 172.21.15.19:0/4246926113 learned_addr learned my addr 172.21.15.19:0/4246926113 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575546617 0 0) 0x7f0c580b9940 con 0x7f0c5815dd30 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c40003520 con 0x7f0c5815dd30 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4ffff640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c580d2e90 0x7f0c580d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3274220679 0 0) 0x7f0c40003520 con 0x7f0c5815dd30 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c580d2e90 msgr2=0x7f0c580d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c580d2e90 0x7f0c580d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 >> v1:172.21.15.112:6789/0 conn(0x7f0c580d0900 legacy=0x7f0c580b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c580d20d0 con 0x7f0c5815dd30 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0c54001670 con 0x7f0c5815dd30 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 >> v1:172.21.15.19:6789/0 conn(0x7f0c5815dd30 legacy=0x7f0c580b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c580d2e90 0x7f0c400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40003f60 0x7f0c40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.577+0000 7f0c4f7fe640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c40004d40 0x7f0c400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.578+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580d0d70 con 0x7f0c40004d40 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.578+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4000bb50 con 0x7f0c40003f60 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.578+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4000bd50 con 0x7f0c580d2e90 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.578+0000 7f0c5d253640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c40004d40 0x7f0c400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.578+0000 7f0c4ffff640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40003f60 0x7f0c40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.578+0000 7f0c5ca52640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c580d2e90 0x7f0c400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.578+0000 7f0c5d253640 1 -- 172.21.15.19:0/4246926113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c580d2e90 msgr2=0x7f0c400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.578+0000 7f0c5d253640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c580d2e90 0x7f0c400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.578+0000 7f0c5d253640 1 -- 172.21.15.19:0/4246926113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40003f60 msgr2=0x7f0c40004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.578+0000 7f0c5d253640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40003f60 0x7f0c40004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.578+0000 7f0c5d253640 1 -- 172.21.15.19:0/4246926113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c40003520 con 0x7f0c40004d40 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.579+0000 7f0c5d253640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c40004d40 0x7f0c400051c0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f0c50001e50 tx=0x7f0c40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd9f477f51c6daf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.579+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50001a70 con 0x7f0c40004d40 2024-04-25T10:02:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.579+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c50003970 con 0x7f0c40004d40 2024-04-25T10:02:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.579+0000 7f0c4f7fe640 1 -- 172.21.15.19:0/4246926113 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50003d60 con 0x7f0c40004d40 2024-04-25T10:02:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.579+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/4246926113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c40004d40 msgr2=0x7f0c400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.579+0000 7f0c5ecb6640 1 --2- 172.21.15.19:0/4246926113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c40004d40 0x7f0c400051c0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f0c50001e50 tx=0x7f0c40005830 comp rx=0 tx=0).stop 2024-04-25T10:02:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.579+0000 7f0c5d253640 1 -- 172.21.15.19:0/4246926113 reap_dead start 2024-04-25T10:02:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.579+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/4246926113 shutdown_connections 2024-04-25T10:02:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.579+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/4246926113 >> 172.21.15.19:0/4246926113 conn(0x7f0c58061c30 msgr2=0x7f0c5806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.579+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/4246926113 shutdown_connections 2024-04-25T10:02:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.579+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/4246926113 wait complete. 2024-04-25T10:02:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.581+0000 7f0c5ecb6640 1 Processor -- start 2024-04-25T10:02:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.581+0000 7f0c5ecb6640 1 -- start start 2024-04-25T10:02:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.581+0000 7f0c5ecb6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c58154410 0x7f0c5815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.581+0000 7f0c5ecb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c581f3e10 0x7f0c5815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.581+0000 7f0c5ecb6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c5815e110 0x7f0c5815e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.581+0000 7f0c5ecb6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580d2970 con 0x7f0c58154410 2024-04-25T10:02:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.581+0000 7f0c5ecb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580d2690 con 0x7f0c581f3e10 2024-04-25T10:02:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.581+0000 7f0c5ecb6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580d1a10 con 0x7f0c5815e110 2024-04-25T10:02:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5ca52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c58154410 0x7f0c5815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5ca52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c58154410 0x7f0c5815bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41044/0 (socket says 172.21.15.19:41044) 2024-04-25T10:02:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5ca52640 1 -- 172.21.15.19:0/3570466097 learned_addr learned my addr 172.21.15.19:0/3570466097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c4ffff640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c581f3e10 0x7f0c5815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5d253640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c5815e110 0x7f0c5815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5ca52640 1 -- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c5815e110 msgr2=0x7f0c5815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5ca52640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c5815e110 0x7f0c5815e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5ca52640 1 -- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c581f3e10 msgr2=0x7f0c5815c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5ca52640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c581f3e10 0x7f0c5815c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5ca52640 1 -- 172.21.15.19:0/3570466097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c58160260 con 0x7f0c58154410 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5d253640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c5815e110 0x7f0c5815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c4ffff640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c581f3e10 0x7f0c5815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5ca52640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c58154410 0x7f0c5815bba0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f0c54001b80 tx=0x7f0c54007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c4d7fa640 1 -- 172.21.15.19:0/3570466097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400f960 con 0x7f0c58154410 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c4d7fa640 1 -- 172.21.15.19:0/3570466097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54018070 con 0x7f0c58154410 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c4d7fa640 1 -- 172.21.15.19:0/3570466097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400fd80 con 0x7f0c58154410 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/3570466097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c58160540 con 0x7f0c58154410 2024-04-25T10:02:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.582+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/3570466097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c58160b00 con 0x7f0c58154410 2024-04-25T10:02:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.583+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/3570466097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c580d47a0 con 0x7f0c58154410 2024-04-25T10:02:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.584+0000 7f0c4d7fa640 1 -- 172.21.15.19:0/3570466097 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c5401d460 con 0x7f0c58154410 2024-04-25T10:02:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.584+0000 7f0c4d7fa640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c4406b4d0 0x7f0c4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.584+0000 7f0c4d7fa640 1 -- 172.21.15.19:0/3570466097 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c5408a580 con 0x7f0c58154410 2024-04-25T10:02:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.584+0000 7f0c4ffff640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c4406b4d0 0x7f0c4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.585+0000 7f0c4ffff640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c4406b4d0 0x7f0c4406d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0c580cf510 tx=0x7f0c48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.588+0000 7f0c4d7fa640 1 -- 172.21.15.19:0/3570466097 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c54059270 con 0x7f0c58154410 2024-04-25T10:02:40.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.806+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/3570466097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c58073990 con 0x7f0c58154410 2024-04-25T10:02:40.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.806+0000 7f0c4d7fa640 1 -- 172.21.15.19:0/3570466097 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f0c54058b70 con 0x7f0c58154410 2024-04-25T10:02:40.812 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:40.812 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7466,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/590882657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":590882657},{"type":"v1","addr":"172.21.15.112:6836","nonce":590882657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":9744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1740426538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1740426538},{"type":"v1","addr":"172.21.15.19:6833","nonce":1740426538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/525024117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":525024117},{"type":"v1","addr":"172.21.15.19:6835","nonce":525024117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:34.992088+0000","modified":"2024-04-25T10:02:38.011991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2648279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2648279035},{"type":"v1","addr":"172.21.15.112:6837","nonce":2648279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:02:40.813 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 91 2024-04-25T10:02:40.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.808+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c4406b4d0 msgr2=0x7f0c4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.808+0000 7f0c5ecb6640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c4406b4d0 0x7f0c4406d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0c580cf510 tx=0x7f0c48008040 comp rx=0 tx=0).stop 2024-04-25T10:02:40.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.808+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/3570466097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c58154410 msgr2=0x7f0c5815bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.808+0000 7f0c5ecb6640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c58154410 0x7f0c5815bba0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f0c54001b80 tx=0x7f0c54007720 comp rx=0 tx=0).stop 2024-04-25T10:02:40.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.808+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/3570466097 shutdown_connections 2024-04-25T10:02:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.808+0000 7f0c5ecb6640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c4406b4d0 0x7f0c4406d990 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.808+0000 7f0c5ecb6640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c5815e110 0x7f0c5815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.808+0000 7f0c5ecb6640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c581f3e10 0x7f0c5815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.808+0000 7f0c5ecb6640 1 --2- 172.21.15.19:0/3570466097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c58154410 0x7f0c5815bba0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.808+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/3570466097 >> 172.21.15.19:0/3570466097 conn(0x7f0c58061c30 msgr2=0x7f0c580b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.809+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/3570466097 shutdown_connections 2024-04-25T10:02:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:40.809+0000 7f0c5ecb6640 1 -- 172.21.15.19:0/3570466097 wait complete. 2024-04-25T10:02:41.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:34.992088+0000', 'modified': '2024-04-25T10:02:38.011991+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9762}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9762': {'gid': 9762, 'name': 'd', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6837/2648279035', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6835', 'nonce': 2648279035}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 2648279035}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:41.228 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:41.228 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:02:41.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.329+0000 7fd5f0cc4640 1 Processor -- start 2024-04-25T10:02:41.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.329+0000 7fd5f0cc4640 1 -- start start 2024-04-25T10:02:41.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.330+0000 7fd5f0cc4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5ec15dc20 0x7fd5ec0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.330+0000 7fd5f0cc4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5ec161710 0x7fd5ec0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.330+0000 7fd5f0cc4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ec15e000 con 0x7fd5ec0b9330 2024-04-25T10:02:41.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.330+0000 7fd5f0cc4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ec0d1de0 con 0x7fd5ec15dc20 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.330+0000 7fd5f0cc4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec161c60 con 0x7fd5ec161710 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.330+0000 7fd5ea59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5ec15dc20 0x7fd5ec0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.330+0000 7fd5ead9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd5ec0b9330 0x7fd5ec06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54896/0 (socket says 172.21.15.19:54896) 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.330+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/877055160 >> v2:172.21.15.19:3300/0 conn(0x7fd5ec15dc20 0x7fd5ec0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41058/0 (socket says 172.21.15.19:41058) 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.330+0000 7fd5ead9d640 1 -- 172.21.15.19:0/877055160 learned_addr learned my addr 172.21.15.19:0/877055160 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.331+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/877055160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5ec161710 0x7fd5ec0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.331+0000 7fd5ea59c640 1 -- 172.21.15.19:0/877055160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5ec161710 msgr2=0x7fd5ec0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.331+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/877055160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5ec161710 0x7fd5ec0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.331+0000 7fd5ea59c640 1 -- 172.21.15.19:0/877055160 >> v1:172.21.15.19:6789/0 conn(0x7fd5ec0b9330 legacy=0x7fd5ec06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.331+0000 7fd5e959a640 1 -- 172.21.15.19:0/877055160 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451693047 0 0) 0x7fd5ec15e000 con 0x7fd5ec0b9330 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.331+0000 7fd5ea59c640 1 -- 172.21.15.19:0/877055160 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec161f40 con 0x7fd5ec15dc20 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.331+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/877055160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5ec161710 0x7fd5ec0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.331+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/877055160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5ec15dc20 0x7fd5ec0ce670 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40097a0 tx=0x7fd5e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfebf30c77a1a0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.332+0000 7fd5e959a640 1 -- 172.21.15.19:0/877055160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4037070 con 0x7fd5ec15dc20 2024-04-25T10:02:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.332+0000 7fd5e959a640 1 -- 172.21.15.19:0/877055160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e4032440 con 0x7fd5ec15dc20 2024-04-25T10:02:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.332+0000 7fd5e959a640 1 -- 172.21.15.19:0/877055160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4032810 con 0x7fd5ec15dc20 2024-04-25T10:02:41.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.332+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/877055160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5ec15dc20 msgr2=0x7fd5ec0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.332+0000 7fd5f0cc4640 1 --2- 172.21.15.19:0/877055160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5ec15dc20 0x7fd5ec0ce670 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40097a0 tx=0x7fd5e402a180 comp rx=0 tx=0).stop 2024-04-25T10:02:41.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.332+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/877055160 shutdown_connections 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.332+0000 7fd5f0cc4640 1 --2- 172.21.15.19:0/877055160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5ec161710 0x7fd5ec0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.332+0000 7fd5f0cc4640 1 --2- 172.21.15.19:0/877055160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5ec15dc20 0x7fd5ec0ce670 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.332+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/877055160 >> 172.21.15.19:0/877055160 conn(0x7fd5ec0d0420 msgr2=0x7fd5ec0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.332+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/877055160 shutdown_connections 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.332+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/877055160 wait complete. 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.334+0000 7fd5f0cc4640 1 Processor -- start 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.334+0000 7fd5f0cc4640 1 -- start start 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.334+0000 7fd5f0cc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5ec0b9330 0x7fd5ec0aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.334+0000 7fd5f0cc4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5ec15dc20 0x7fd5ec0d3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.334+0000 7fd5f0cc4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5ec161710 0x7fd5ec0d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.334+0000 7fd5f0cc4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0b5e50 con 0x7fd5ec0b9330 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.334+0000 7fd5f0cc4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec162500 con 0x7fd5ec161710 2024-04-25T10:02:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.334+0000 7fd5f0cc4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0b5fd0 con 0x7fd5ec15dc20 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.334+0000 7fd5ea59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5ec0b9330 0x7fd5ec0aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.334+0000 7fd5ea59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5ec0b9330 0x7fd5ec0aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41068/0 (socket says 172.21.15.19:41068) 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.334+0000 7fd5ea59c640 1 -- 172.21.15.19:0/2145360007 learned_addr learned my addr 172.21.15.19:0/2145360007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5ec15dc20 0x7fd5ec0d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5ea59c640 1 -- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5ec15dc20 msgr2=0x7fd5ec0d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5ec15dc20 0x7fd5ec0d3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5ead9d640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5ec161710 0x7fd5ec0d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5ea59c640 1 -- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5ec161710 msgr2=0x7fd5ec0d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5ec161710 0x7fd5ec0d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5ea59c640 1 -- 172.21.15.19:0/2145360007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec0aacd0 con 0x7fd5ec0b9330 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5ec0b9330 0x7fd5ec0aa380 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40097a0 tx=0x7fd5e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5e2ffd640 1 -- 172.21.15.19:0/2145360007 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4037050 con 0x7fd5ec0b9330 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5e2ffd640 1 -- 172.21.15.19:0/2145360007 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e4038040 con 0x7fd5ec0b9330 2024-04-25T10:02:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/2145360007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec0aafb0 con 0x7fd5ec0b9330 2024-04-25T10:02:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5e2ffd640 1 -- 172.21.15.19:0/2145360007 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4004490 con 0x7fd5ec0b9330 2024-04-25T10:02:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.335+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/2145360007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec0aa8c0 con 0x7fd5ec0b9330 2024-04-25T10:02:41.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.337+0000 7fd5e2ffd640 1 -- 172.21.15.19:0/2145360007 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5e4030980 con 0x7fd5ec0b9330 2024-04-25T10:02:41.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.337+0000 7fd5e2ffd640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5cc06b500 0x7fd5cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.337+0000 7fd5e2ffd640 1 -- 172.21.15.19:0/2145360007 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd5e40aae90 con 0x7fd5ec0b9330 2024-04-25T10:02:41.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.337+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5cc06b500 0x7fd5cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.338+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5cc06b500 0x7fd5cc06d9c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0ab800 tx=0x7fd5d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.338+0000 7fd5f39ca740 1 Processor -- start 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.338+0000 7fd5f39ca740 1 -- start start 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f82d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f84aad0 0x563e2f84b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f802f10 0x563e2f849190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2f765a10 con 0x563e2f82d230 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2f70eb30 con 0x563e2f802f10 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e2f761c50 con 0x563e2f84aad0 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5ea59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f82d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5ea59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f82d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41084/0 (socket says 172.21.15.19:41084) 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5ea59c640 1 -- 172.21.15.19:0/1870317447 learned_addr learned my addr 172.21.15.19:0/1870317447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5ead9d640 1 --2- 172.21.15.19:0/1870317447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f802f10 0x563e2f849190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/1870317447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f84aad0 0x563e2f84b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5ea59c640 1 -- 172.21.15.19:0/1870317447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f84aad0 msgr2=0x563e2f84b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/1870317447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f84aad0 0x563e2f84b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5ea59c640 1 -- 172.21.15.19:0/1870317447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f802f10 msgr2=0x563e2f849190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/1870317447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f802f10 0x563e2f849190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.339+0000 7fd5ea59c640 1 -- 172.21.15.19:0/1870317447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e2f723e10 con 0x563e2f82d230 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.340+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/1870317447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f82d610 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4035040 tx=0x7fd5e4032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d9ac8f323ac221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.340+0000 7fd5cbfff640 1 -- 172.21.15.19:0/1870317447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4091030 con 0x563e2f82d230 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.340+0000 7fd5cbfff640 1 -- 172.21.15.19:0/1870317447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e4079d30 con 0x563e2f82d230 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.340+0000 7fd5cbfff640 1 -- 172.21.15.19:0/1870317447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4079630 con 0x563e2f82d230 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.340+0000 7fd5f39ca740 1 -- 172.21.15.19:0/1870317447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 msgr2=0x563e2f82d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.340+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/1870317447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f82d610 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4035040 tx=0x7fd5e4032440 comp rx=0 tx=0).stop 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.340+0000 7fd5f39ca740 1 -- 172.21.15.19:0/1870317447 shutdown_connections 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.341+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/1870317447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f802f10 0x563e2f849190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.341+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/1870317447 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f84aad0 0x563e2f84b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.341+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/1870317447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f82d610 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.341+0000 7fd5f39ca740 1 -- 172.21.15.19:0/1870317447 >> 172.21.15.19:0/1870317447 conn(0x563e2f7ff840 msgr2=0x563e2f848040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.341+0000 7fd5f39ca740 1 -- 172.21.15.19:0/1870317447 shutdown_connections 2024-04-25T10:02:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.341+0000 7fd5f39ca740 1 -- 172.21.15.19:0/1870317447 wait complete. 2024-04-25T10:02:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.341+0000 7fd5f39ca740 1 -- start start 2024-04-25T10:02:41.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.341+0000 7fd5f39ca740 1 Processor -- start 2024-04-25T10:02:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f82d230 0x563e2f81fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f830300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f81c440 0x563e2f81c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2f765a10 con 0x563e2f84aad0 2024-04-25T10:02:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2f7220c0 con 0x563e2f81c440 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e2f803cb0 con 0x563e2f82d230 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5f39ca740 10 client.0 _ll_register_callbacks cb 0x563e2f715ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5e9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f830300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5e9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f830300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41092/0 (socket says 172.21.15.19:41092) 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5e9d9b640 1 -- 172.21.15.19:0/3417449137 learned_addr learned my addr 172.21.15.19:0/3417449137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5ead9d640 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f81c440 0x563e2f81c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.342+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f82d230 0x563e2f81fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5e9d9b640 1 -- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f82d230 msgr2=0x563e2f81fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f82d230 0x563e2f81fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5e9d9b640 1 -- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f81c440 msgr2=0x563e2f81c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f81c440 0x563e2f81c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5e9d9b640 1 -- 172.21.15.19:0/3417449137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e2f47a2d0 con 0x563e2f84aad0 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f830300 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4012700 tx=0x7fd5d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8152d8c446e144da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:41.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3417449137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4023070 con 0x563e2f84aad0 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3417449137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd5d401f800 con 0x563e2f84aad0 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3417449137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5e4079d30 con 0x563e2f84aad0 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3417449137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5d402e050 con 0x563e2f84aad0 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5f39ca740 10 client.9885 fetch_fsmap learned FSMap version 91 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3417449137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x563e2f5446b0 con 0x563e2f84aad0 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3417449137 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e4079d30 con 0x563e2f84aad0 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3417449137 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd5d4031020 con 0x563e2f84aad0 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5f39ca740 10 client.9885 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5f39ca740 10 client.9885 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5f39ca740 20 client.9885 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x563e2f863a60 0x563e2f865e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5f39ca740 4 client.9885 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:41.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.343+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3417449137 --> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e2f7c4920 con 0x563e2f863a60 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.344+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x563e2f863a60 0x563e2f865e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.344+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x563e2f863a60 0x563e2f865e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=355fb36be67adb44 server_cookie=71873efa8654209b in_seq=0 out_seq=0 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.344+0000 7fd5c97fa640 10 client.9885 ms_handle_connect on v2:172.21.15.112:6835/2648279035 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3417449137 <== mds.0 v2:172.21.15.112:6835/2648279035 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563e2f7c4920 con 0x563e2f863a60 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5c97fa640 10 client.9885 handle_command_reply: tid=0 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 1 client.9885 shutdown 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3417449137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 msgr2=0x563e2f830300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f830300 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4012700 tx=0x7fd5d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3417449137 shutdown_connections 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x563e2f863a60 0x563e2f865e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f81c440 0x563e2f81c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f830300 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3417449137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f82d230 0x563e2f81fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5c97fa640 0 client.9885 ms_handle_reset on v2:172.21.15.112:6835/2648279035 2024-04-25T10:02:41.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3417449137 >> 172.21.15.19:0/3417449137 conn(0x563e2f802f10 msgr2=0x563e2f8024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3417449137 shutdown_connections 2024-04-25T10:02:41.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3417449137 wait complete. 2024-04-25T10:02:41.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.345+0000 7fd5f39ca740 20 client.9885 trim_cache size 0 max 16384 2024-04-25T10:02:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.361+0000 7fd5f39ca740 1 Processor -- start 2024-04-25T10:02:41.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.361+0000 7fd5f39ca740 1 -- start start 2024-04-25T10:02:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f81c440 0x563e2f8042e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f800de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f84aad0 0x563e2f801320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1ff590 con 0x563e2f82d230 2024-04-25T10:02:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1f4e90 con 0x563e2f84aad0 2024-04-25T10:02:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1fefc0 con 0x563e2f81c440 2024-04-25T10:02:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5e9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f800de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5e9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f800de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41096/0 (socket says 172.21.15.19:41096) 2024-04-25T10:02:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5e9d9b640 1 -- 172.21.15.19:0/2840873836 learned_addr learned my addr 172.21.15.19:0/2840873836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/2840873836 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f81c440 0x563e2f8042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5ead9d640 1 --2- 172.21.15.19:0/2840873836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f84aad0 0x563e2f801320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5e9d9b640 1 -- 172.21.15.19:0/2840873836 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f81c440 msgr2=0x563e2f8042e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/2840873836 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f81c440 0x563e2f8042e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5e9d9b640 1 -- 172.21.15.19:0/2840873836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f84aad0 msgr2=0x563e2f801320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/2840873836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f84aad0 0x563e2f801320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5e9d9b640 1 -- 172.21.15.19:0/2840873836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e2f830670 con 0x563e2f82d230 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/2840873836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f800de0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400b560 tx=0x7fd5d4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98514b3020fbf54e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5cb7fe640 1 -- 172.21.15.19:0/2840873836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4023040 con 0x563e2f82d230 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5cb7fe640 1 -- 172.21.15.19:0/2840873836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d4037070 con 0x563e2f82d230 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.362+0000 7fd5cb7fe640 1 -- 172.21.15.19:0/2840873836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4019040 con 0x563e2f82d230 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.363+0000 7fd5f39ca740 1 -- 172.21.15.19:0/2840873836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 msgr2=0x563e2f800de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.363+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/2840873836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f800de0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400b560 tx=0x7fd5d4020000 comp rx=0 tx=0).stop 2024-04-25T10:02:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.363+0000 7fd5f39ca740 1 -- 172.21.15.19:0/2840873836 shutdown_connections 2024-04-25T10:02:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.363+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/2840873836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f84aad0 0x563e2f801320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.363+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/2840873836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f82d230 0x563e2f800de0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.363+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/2840873836 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f81c440 0x563e2f8042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.363+0000 7fd5f39ca740 1 -- 172.21.15.19:0/2840873836 >> 172.21.15.19:0/2840873836 conn(0x563e2f802f10 msgr2=0x563e2f802670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.363+0000 7fd5f39ca740 1 -- 172.21.15.19:0/2840873836 shutdown_connections 2024-04-25T10:02:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.363+0000 7fd5f39ca740 1 -- 172.21.15.19:0/2840873836 wait complete. 2024-04-25T10:02:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.364+0000 7fd5f39ca740 1 -- start start 2024-04-25T10:02:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.364+0000 7fd5f39ca740 1 Processor -- start 2024-04-25T10:02:41.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.364+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f81c440 0x563e2f8b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.364+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f82d230 0x563e2f846c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.364+0000 7fd5f39ca740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f8666f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.364+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2f81e620 con 0x563e2f84aad0 2024-04-25T10:02:41.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.364+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2f81f530 con 0x563e2f81c440 2024-04-25T10:02:41.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.364+0000 7fd5f39ca740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e2f83f7b0 con 0x563e2f82d230 2024-04-25T10:02:41.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.364+0000 7fd5f39ca740 10 client.0 _ll_register_callbacks cb 0x563e2f7fdc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:41.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5ead9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f8666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5ead9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f8666f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41110/0 (socket says 172.21.15.19:41110) 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5ead9d640 1 -- 172.21.15.19:0/3554105504 learned_addr learned my addr 172.21.15.19:0/3554105504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f81c440 0x563e2f8b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5e9d9b640 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f82d230 0x563e2f846c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5ead9d640 1 -- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f82d230 msgr2=0x563e2f846c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5ead9d640 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f82d230 0x563e2f846c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5ead9d640 1 -- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f81c440 msgr2=0x563e2f8b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5ead9d640 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f81c440 0x563e2f8b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5ead9d640 1 -- 172.21.15.19:0/3554105504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e2f847160 con 0x563e2f84aad0 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5ead9d640 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f8666f0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0012e0 tx=0x7fd5dc007380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0b04364857dd2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3554105504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc015820 con 0x563e2f84aad0 2024-04-25T10:02:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3554105504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd5dc01aa20 con 0x563e2f84aad0 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3554105504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5d4037070 con 0x563e2f84aad0 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3554105504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5dc025050 con 0x563e2f84aad0 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5f39ca740 10 client.9891 fetch_fsmap learned FSMap version 91 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.365+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3554105504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x563e2f8443f0 con 0x563e2f84aad0 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3554105504 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d4037070 con 0x563e2f84aad0 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3554105504 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd5dc028020 con 0x563e2f84aad0 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5f39ca740 10 client.9891 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5f39ca740 10 client.9891 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5f39ca740 20 client.9891 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x563e2f801450 0x563e2f81b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5f39ca740 4 client.9891 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3554105504 --> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563e2f772f50 con 0x563e2f801450 2024-04-25T10:02:41.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x563e2f801450 0x563e2f81b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5ea59c640 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x563e2f801450 0x563e2f81b140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddccca17e8a026f0 server_cookie=1699b85cb1c73ab8 in_seq=0 out_seq=0 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5c97fa640 10 client.9891 ms_handle_connect on v2:172.21.15.112:6835/2648279035 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5c97fa640 1 -- 172.21.15.19:0/3554105504 <== mds.0 v2:172.21.15.112:6835/2648279035 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x563e2f772f50 con 0x563e2f801450 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5c97fa640 10 client.9891 handle_command_reply: tid=0 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.366+0000 7fd5f39ca740 1 client.9891 shutdown 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3554105504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 msgr2=0x563e2f8666f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f8666f0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0012e0 tx=0x7fd5dc007380 comp rx=0 tx=0).stop 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3554105504 shutdown_connections 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x563e2f801450 0x563e2f81b140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e2f84aad0 0x563e2f8666f0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5c97fa640 0 client.9891 ms_handle_reset on v2:172.21.15.112:6835/2648279035 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x563e2f82d230 0x563e2f846c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f39ca740 1 --2- 172.21.15.19:0/3554105504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e2f81c440 0x563e2f8b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3554105504 >> 172.21.15.19:0/3554105504 conn(0x563e2f802f10 msgr2=0x563e2f8032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3554105504 shutdown_connections 2024-04-25T10:02:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f39ca740 1 -- 172.21.15.19:0/3554105504 wait complete. 2024-04-25T10:02:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f39ca740 20 client.9891 trim_cache size 0 max 16384 2024-04-25T10:02:41.380 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:02:41.380 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T10:02:41.380 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:02:41.380 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 9762, 2024-04-25T10:02:41.380 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:02:41.380 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:02:41.380 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:02:41.381 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.7177430549999997, 2024-04-25T10:02:41.381 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 91, 2024-04-25T10:02:41.381 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 140, 2024-04-25T10:02:41.381 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 140, 2024-04-25T10:02:41.381 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 10.054958653 2024-04-25T10:02:41.381 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:02:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5cc06b500 msgr2=0x7fd5cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5cc06b500 0x7fd5cc06d9c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0ab800 tx=0x7fd5d4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/2145360007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5ec0b9330 msgr2=0x7fd5ec0aa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5ec0b9330 0x7fd5ec0aa380 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40097a0 tx=0x7fd5e40046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/2145360007 shutdown_connections 2024-04-25T10:02:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd5cc06b500 0x7fd5cc06d9c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5ec161710 0x7fd5ec0d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5ec15dc20 0x7fd5ec0d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 --2- 172.21.15.19:0/2145360007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5ec0b9330 0x7fd5ec0aa380 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/2145360007 >> 172.21.15.19:0/2145360007 conn(0x7fd5ec0d0420 msgr2=0x7fd5ec0c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/2145360007 shutdown_connections 2024-04-25T10:02:41.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.367+0000 7fd5f0cc4640 1 -- 172.21.15.19:0/2145360007 wait complete. 2024-04-25T10:02:41.384 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:41.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.479+0000 7f0db866f640 1 Processor -- start 2024-04-25T10:02:41.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.479+0000 7f0db866f640 1 -- start start 2024-04-25T10:02:41.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.480+0000 7f0db866f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0db015b980 0x7f0db00cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.480+0000 7f0db866f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0db006b5f0 0x7f0db00cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.480+0000 7f0db866f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db015f710 con 0x7f0db00b2d40 2024-04-25T10:02:41.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.480+0000 7f0db866f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db015f9b0 con 0x7f0db006b5f0 2024-04-25T10:02:41.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.480+0000 7f0db866f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db015fc90 con 0x7f0db015b980 2024-04-25T10:02:41.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.480+0000 7f0db6c0c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0db00b2d40 0x7f0db00ce2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54900/0 (socket says 172.21.15.19:54900) 2024-04-25T10:02:41.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.480+0000 7f0db6c0c640 1 -- 172.21.15.19:0/3640404275 learned_addr learned my addr 172.21.15.19:0/3640404275 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:41.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.480+0000 7f0db5c0a640 1 --2- 172.21.15.19:0/3640404275 >> v2:172.21.15.112:3300/0 conn(0x7f0db006b5f0 0x7f0db00cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.480+0000 7f0db640b640 1 --2- 172.21.15.19:0/3640404275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0db015b980 0x7f0db00cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.480+0000 7f0db5409640 1 -- 172.21.15.19:0/3640404275 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091968749 0 0) 0x7f0db015f710 con 0x7f0db00b2d40 2024-04-25T10:02:41.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.480+0000 7f0db5409640 1 -- 172.21.15.19:0/3640404275 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d9c003520 con 0x7f0db00b2d40 2024-04-25T10:02:41.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.481+0000 7f0db5c0a640 1 -- 172.21.15.19:0/3640404275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0db015b980 msgr2=0x7f0db00cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.481+0000 7f0db5c0a640 1 --2- 172.21.15.19:0/3640404275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0db015b980 0x7f0db00cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.481+0000 7f0db5409640 1 -- 172.21.15.19:0/3640404275 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3900734203 0 0) 0x7f0d9c003520 con 0x7f0db00b2d40 2024-04-25T10:02:41.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.481+0000 7f0db5c0a640 1 -- 172.21.15.19:0/3640404275 >> v1:172.21.15.19:6789/0 conn(0x7f0db00b2d40 legacy=0x7f0db00ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.481+0000 7f0db5c0a640 1 -- 172.21.15.19:0/3640404275 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db015ff70 con 0x7f0db006b5f0 2024-04-25T10:02:41.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.481+0000 7f0db640b640 1 --2- 172.21.15.19:0/3640404275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0db015b980 0x7f0db00cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db5c0a640 1 --2- 172.21.15.19:0/3640404275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db006b5f0 0x7f0db00cf8a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0da00097a0 tx=0x7f0da002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=594fa49a9242065e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db5409640 1 -- 172.21.15.19:0/3640404275 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da0037070 con 0x7f0db006b5f0 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db5409640 1 -- 172.21.15.19:0/3640404275 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da002aba0 con 0x7f0db006b5f0 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db5409640 1 -- 172.21.15.19:0/3640404275 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da80019f0 con 0x7f0db006b5f0 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db866f640 1 -- 172.21.15.19:0/3640404275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db006b5f0 msgr2=0x7f0db00cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db866f640 1 --2- 172.21.15.19:0/3640404275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db006b5f0 0x7f0db00cf8a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0da00097a0 tx=0x7f0da002a110 comp rx=0 tx=0).stop 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db866f640 1 -- 172.21.15.19:0/3640404275 shutdown_connections 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db866f640 1 --2- 172.21.15.19:0/3640404275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db006b5f0 0x7f0db00cf8a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db866f640 1 --2- 172.21.15.19:0/3640404275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0db015b980 0x7f0db00cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db866f640 1 -- 172.21.15.19:0/3640404275 >> 172.21.15.19:0/3640404275 conn(0x7f0db00b2220 msgr2=0x7f0db00b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db866f640 1 -- 172.21.15.19:0/3640404275 shutdown_connections 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.482+0000 7f0db866f640 1 -- 172.21.15.19:0/3640404275 wait complete. 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.484+0000 7f0db866f640 1 Processor -- start 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.484+0000 7f0db866f640 1 -- start start 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.484+0000 7f0db866f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db006b5f0 0x7f0db00b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.484+0000 7f0db866f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db00b2d40 0x7f0db01e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.484+0000 7f0db866f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0db015b980 0x7f0db0152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.484+0000 7f0db866f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db00cf550 con 0x7f0db006b5f0 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.484+0000 7f0db866f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db015bd60 con 0x7f0db00b2d40 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.484+0000 7f0db866f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db00cfde0 con 0x7f0db015b980 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.484+0000 7f0db640b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db006b5f0 0x7f0db00b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.484+0000 7f0db640b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db006b5f0 0x7f0db00b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41122/0 (socket says 172.21.15.19:41122) 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.484+0000 7f0db640b640 1 -- 172.21.15.19:0/1284022172 learned_addr learned my addr 172.21.15.19:0/1284022172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0db6c0c640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0db015b980 0x7f0db0152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0db5c0a640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db00b2d40 0x7f0db01e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0db640b640 1 -- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0db015b980 msgr2=0x7f0db0152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0db640b640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0db015b980 0x7f0db0152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0db640b640 1 -- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db00b2d40 msgr2=0x7f0db01e5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0db640b640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db00b2d40 0x7f0db01e5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0db640b640 1 -- 172.21.15.19:0/1284022172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db01e6650 con 0x7f0db006b5f0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0db640b640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db006b5f0 0x7f0db00b54f0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00e990 tx=0x7f0dac00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0da6ffd640 1 -- 172.21.15.19:0/1284022172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac017070 con 0x7f0db006b5f0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0da6ffd640 1 -- 172.21.15.19:0/1284022172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac00c9e0 con 0x7f0db006b5f0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0da6ffd640 1 -- 172.21.15.19:0/1284022172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac00cdb0 con 0x7f0db006b5f0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0db866f640 1 -- 172.21.15.19:0/1284022172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db00b4730 con 0x7f0db006b5f0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.485+0000 7f0db866f640 1 -- 172.21.15.19:0/1284022172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db00b4cf0 con 0x7f0db006b5f0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.486+0000 7f0db866f640 1 -- 172.21.15.19:0/1284022172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db0073990 con 0x7f0db006b5f0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.487+0000 7f0da6ffd640 1 -- 172.21.15.19:0/1284022172 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0dac01a070 con 0x7f0db006b5f0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.487+0000 7f0da6ffd640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9406b4d0 0x7f0d9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.487+0000 7f0da6ffd640 1 -- 172.21.15.19:0/1284022172 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0dac08a370 con 0x7f0db006b5f0 2024-04-25T10:02:41.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.488+0000 7f0db5c0a640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9406b4d0 0x7f0d9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.488+0000 7f0db5c0a640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9406b4d0 0x7f0d9406d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0da0034000 tx=0x7f0da002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.492+0000 7f0da6ffd640 1 -- 172.21.15.19:0/1284022172 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0dac058f30 con 0x7f0db006b5f0 2024-04-25T10:02:41.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.712+0000 7f0db866f640 1 -- 172.21.15.19:0/1284022172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0db00b6670 con 0x7f0db006b5f0 2024-04-25T10:02:41.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.713+0000 7f0da6ffd640 1 -- 172.21.15.19:0/1284022172 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0dac058830 con 0x7f0db006b5f0 2024-04-25T10:02:41.719 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:02:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 -- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9406b4d0 msgr2=0x7f0d9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9406b4d0 0x7f0d9406d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0da0034000 tx=0x7f0da002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 -- 172.21.15.19:0/1284022172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db006b5f0 msgr2=0x7f0db00b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db006b5f0 0x7f0db00b54f0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00e990 tx=0x7f0dac00ee50 comp rx=0 tx=0).stop 2024-04-25T10:02:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 -- 172.21.15.19:0/1284022172 shutdown_connections 2024-04-25T10:02:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d9406b4d0 0x7f0d9406d990 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0db015b980 0x7f0db0152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db00b2d40 0x7f0db01e5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 --2- 172.21.15.19:0/1284022172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db006b5f0 0x7f0db00b54f0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 -- 172.21.15.19:0/1284022172 >> 172.21.15.19:0/1284022172 conn(0x7f0db00b2220 msgr2=0x7f0db00add90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 -- 172.21.15.19:0/1284022172 shutdown_connections 2024-04-25T10:02:41.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:41.715+0000 7f0db866f640 1 -- 172.21.15.19:0/1284022172 wait complete. 2024-04-25T10:02:41.731 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:02:41.731 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:41.731 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T10:02:41.731 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:02:41.731 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:41.732 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:02:41.732 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:02:41.732 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:41.732 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:41.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.732 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T10:02:41.748 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:41.748 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:41.748 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:41.748 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:41.748 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:41.748 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:41.748 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:41.748 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:41.748 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42184sec preferred_lft 42184sec 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T10:02:41.749 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T10:02:41.750 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:41.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.751 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:41.811 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.812 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T10:02:41.868 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T10:02:41.868 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:41.868 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:02:41.869 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:02:41.869 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:41.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.869 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T10:02:41.869 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:41.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:02:41.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:02:41.870 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:02:41.870 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:41.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.954 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T10:02:41.969 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:02:41.969 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:02:41.970 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:41.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.970 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T10:02:42.034 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:42.034 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:42.089 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:42.090 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:42.090 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:42.145 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:02:42.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:42.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:02:42.228 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:02:42.228 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:02:42.235 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:02:42.235 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:02:42.235 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T10:02:42.235 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T10:02:42.235 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T10:02:42.236 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:02:42.236 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:42.236 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:42.236 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:42.236 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:42.238 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:42.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:42.238 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:42.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:42.304 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:42.367 INFO:teuthology.orchestra.run.smithi019.stdout:client7562 2024-04-25T10:02:42.367 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:02:42.367 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:42.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:42.423 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:42.423 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:42.423 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:02:42.423 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:42.424 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:42.424 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:42.424 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792649 s, 132 MB/s 2024-04-25T10:02:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805889 s, 130 MB/s 2024-04-25T10:02:42.514 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.514 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.514 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702646 s, 149 MB/s 2024-04-25T10:02:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791284 s, 133 MB/s 2024-04-25T10:02:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773933 s, 135 MB/s 2024-04-25T10:02:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753252 s, 139 MB/s 2024-04-25T10:02:42.547 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.547 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.547 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658016 s, 159 MB/s 2024-04-25T10:02:42.555 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.555 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.555 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696088 s, 151 MB/s 2024-04-25T10:02:42.563 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076019 s, 138 MB/s 2024-04-25T10:02:42.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.572 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.572 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073618 s, 142 MB/s 2024-04-25T10:02:42.580 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.580 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.580 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074921 s, 140 MB/s 2024-04-25T10:02:42.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.589 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779217 s, 135 MB/s 2024-04-25T10:02:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698547 s, 150 MB/s 2024-04-25T10:02:42.604 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.605 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.605 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697019 s, 150 MB/s 2024-04-25T10:02:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798116 s, 131 MB/s 2024-04-25T10:02:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074782 s, 140 MB/s 2024-04-25T10:02:42.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.630 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746204 s, 141 MB/s 2024-04-25T10:02:42.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.639 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.639 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740387 s, 142 MB/s 2024-04-25T10:02:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.647 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.647 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704175 s, 149 MB/s 2024-04-25T10:02:42.654 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.654 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.654 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652978 s, 161 MB/s 2024-04-25T10:02:42.662 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.662 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749445 s, 140 MB/s 2024-04-25T10:02:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694335 s, 151 MB/s 2024-04-25T10:02:42.677 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621836 s, 169 MB/s 2024-04-25T10:02:42.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.686 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751291 s, 140 MB/s 2024-04-25T10:02:42.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737521 s, 142 MB/s 2024-04-25T10:02:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667106 s, 157 MB/s 2024-04-25T10:02:42.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.710 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726529 s, 144 MB/s 2024-04-25T10:02:42.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.719 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805252 s, 130 MB/s 2024-04-25T10:02:42.726 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.727 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693473 s, 151 MB/s 2024-04-25T10:02:42.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.734 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691204 s, 152 MB/s 2024-04-25T10:02:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718535 s, 146 MB/s 2024-04-25T10:02:42.750 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.751 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708893 s, 148 MB/s 2024-04-25T10:02:42.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:42.752 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:42.752 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:42.752 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:02:42.752 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:42.752 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:42.753 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:42.753 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:42.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.818 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.819 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765988 s, 137 MB/s 2024-04-25T10:02:42.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.826 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700239 s, 150 MB/s 2024-04-25T10:02:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767151 s, 137 MB/s 2024-04-25T10:02:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075222 s, 139 MB/s 2024-04-25T10:02:42.851 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.851 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.851 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660447 s, 159 MB/s 2024-04-25T10:02:42.858 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.858 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.859 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680781 s, 154 MB/s 2024-04-25T10:02:42.866 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.866 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.866 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637347 s, 165 MB/s 2024-04-25T10:02:42.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693452 s, 151 MB/s 2024-04-25T10:02:42.881 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.881 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.881 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703057 s, 149 MB/s 2024-04-25T10:02:42.889 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.889 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.890 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729402 s, 144 MB/s 2024-04-25T10:02:42.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.898 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802211 s, 131 MB/s 2024-04-25T10:02:42.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.906 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688351 s, 152 MB/s 2024-04-25T10:02:42.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.914 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069577 s, 151 MB/s 2024-04-25T10:02:42.921 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.921 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.921 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637251 s, 165 MB/s 2024-04-25T10:02:42.929 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.930 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715353 s, 147 MB/s 2024-04-25T10:02:42.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.937 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665754 s, 158 MB/s 2024-04-25T10:02:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629033 s, 167 MB/s 2024-04-25T10:02:42.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.952 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714871 s, 147 MB/s 2024-04-25T10:02:42.960 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.961 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.961 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074094 s, 142 MB/s 2024-04-25T10:02:42.968 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.968 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.968 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689661 s, 152 MB/s 2024-04-25T10:02:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.977 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747894 s, 140 MB/s 2024-04-25T10:02:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679703 s, 154 MB/s 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631471 s, 166 MB/s 2024-04-25T10:02:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707155 s, 148 MB/s 2024-04-25T10:02:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878635 s, 119 MB/s 2024-04-25T10:02:43.017 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.017 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.017 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665752 s, 158 MB/s 2024-04-25T10:02:43.025 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.025 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.025 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733689 s, 143 MB/s 2024-04-25T10:02:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661412 s, 159 MB/s 2024-04-25T10:02:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643403 s, 163 MB/s 2024-04-25T10:02:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647985 s, 162 MB/s 2024-04-25T10:02:43.055 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.055 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.055 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692586 s, 151 MB/s 2024-04-25T10:02:43.063 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.063 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.063 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690857 s, 152 MB/s 2024-04-25T10:02:43.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.064 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:43.064 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:43.064 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:02:43.064 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:43.064 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:43.065 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:43.065 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:43.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.130 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.130 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654332 s, 160 MB/s 2024-04-25T10:02:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.137 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671993 s, 156 MB/s 2024-04-25T10:02:43.144 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.144 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.144 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625228 s, 168 MB/s 2024-04-25T10:02:43.152 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.152 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.152 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685796 s, 153 MB/s 2024-04-25T10:02:43.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637474 s, 164 MB/s 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662919 s, 158 MB/s 2024-04-25T10:02:43.174 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.175 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.175 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704639 s, 149 MB/s 2024-04-25T10:02:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647569 s, 162 MB/s 2024-04-25T10:02:43.189 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.189 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.189 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645702 s, 162 MB/s 2024-04-25T10:02:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757913 s, 138 MB/s 2024-04-25T10:02:43.205 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.206 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.206 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700833 s, 150 MB/s 2024-04-25T10:02:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.213 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674174 s, 156 MB/s 2024-04-25T10:02:43.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.221 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708727 s, 148 MB/s 2024-04-25T10:02:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.229 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684745 s, 153 MB/s 2024-04-25T10:02:43.236 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.236 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.236 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061986 s, 169 MB/s 2024-04-25T10:02:43.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.244 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070752 s, 148 MB/s 2024-04-25T10:02:43.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.252 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685519 s, 153 MB/s 2024-04-25T10:02:43.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.260 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707111 s, 148 MB/s 2024-04-25T10:02:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708158 s, 148 MB/s 2024-04-25T10:02:43.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.276 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675835 s, 155 MB/s 2024-04-25T10:02:43.284 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.284 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.284 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772286 s, 136 MB/s 2024-04-25T10:02:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676913 s, 155 MB/s 2024-04-25T10:02:43.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.299 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663357 s, 158 MB/s 2024-04-25T10:02:43.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.307 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696362 s, 151 MB/s 2024-04-25T10:02:43.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.315 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652622 s, 161 MB/s 2024-04-25T10:02:43.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.323 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721742 s, 145 MB/s 2024-04-25T10:02:43.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.334 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067349 s, 156 MB/s 2024-04-25T10:02:43.338 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.338 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.338 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665927 s, 157 MB/s 2024-04-25T10:02:43.347 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.347 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.347 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830168 s, 126 MB/s 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631909 s, 166 MB/s 2024-04-25T10:02:43.362 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.362 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.363 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072744 s, 144 MB/s 2024-04-25T10:02:43.371 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.372 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.372 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809687 s, 130 MB/s 2024-04-25T10:02:43.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.373 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:43.373 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:43.373 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:02:43.373 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:43.373 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:43.373 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:43.373 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:43.440 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.440 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.440 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770952 s, 136 MB/s 2024-04-25T10:02:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701406 s, 149 MB/s 2024-04-25T10:02:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715708 s, 147 MB/s 2024-04-25T10:02:43.464 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.464 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.464 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690224 s, 152 MB/s 2024-04-25T10:02:43.472 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.472 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.472 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711342 s, 147 MB/s 2024-04-25T10:02:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722424 s, 145 MB/s 2024-04-25T10:02:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635237 s, 165 MB/s 2024-04-25T10:02:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722103 s, 145 MB/s 2024-04-25T10:02:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670022 s, 156 MB/s 2024-04-25T10:02:43.512 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.512 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.512 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816529 s, 128 MB/s 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720844 s, 145 MB/s 2024-04-25T10:02:43.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.528 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686296 s, 153 MB/s 2024-04-25T10:02:43.536 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.536 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.536 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740314 s, 142 MB/s 2024-04-25T10:02:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.544 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.544 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667517 s, 157 MB/s 2024-04-25T10:02:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694136 s, 151 MB/s 2024-04-25T10:02:43.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.560 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711476 s, 147 MB/s 2024-04-25T10:02:43.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.568 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721286 s, 145 MB/s 2024-04-25T10:02:43.576 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.576 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.576 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713219 s, 147 MB/s 2024-04-25T10:02:43.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.583 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.583 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624083 s, 168 MB/s 2024-04-25T10:02:43.591 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.591 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.591 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709941 s, 148 MB/s 2024-04-25T10:02:43.599 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.599 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.599 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720553 s, 146 MB/s 2024-04-25T10:02:43.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680432 s, 154 MB/s 2024-04-25T10:02:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716877 s, 146 MB/s 2024-04-25T10:02:43.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.622 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.622 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649066 s, 162 MB/s 2024-04-25T10:02:43.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.630 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.630 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717261 s, 146 MB/s 2024-04-25T10:02:43.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.638 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.638 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717261 s, 146 MB/s 2024-04-25T10:02:43.646 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.646 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725635 s, 145 MB/s 2024-04-25T10:02:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758462 s, 138 MB/s 2024-04-25T10:02:43.664 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.664 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.664 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773483 s, 136 MB/s 2024-04-25T10:02:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728082 s, 144 MB/s 2024-04-25T10:02:43.679 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.679 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.680 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640669 s, 164 MB/s 2024-04-25T10:02:43.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.689 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831723 s, 126 MB/s 2024-04-25T10:02:43.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.690 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:43.690 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:43.690 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:02:43.690 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:43.690 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:43.690 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:43.690 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:43.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.756 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691663 s, 152 MB/s 2024-04-25T10:02:43.765 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.765 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.765 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836696 s, 125 MB/s 2024-04-25T10:02:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708691 s, 148 MB/s 2024-04-25T10:02:43.780 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676206 s, 155 MB/s 2024-04-25T10:02:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651591 s, 161 MB/s 2024-04-25T10:02:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711602 s, 147 MB/s 2024-04-25T10:02:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713136 s, 147 MB/s 2024-04-25T10:02:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645881 s, 162 MB/s 2024-04-25T10:02:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.819 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712611 s, 147 MB/s 2024-04-25T10:02:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.826 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063884 s, 164 MB/s 2024-04-25T10:02:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753833 s, 139 MB/s 2024-04-25T10:02:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691381 s, 152 MB/s 2024-04-25T10:02:43.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.851 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712466 s, 147 MB/s 2024-04-25T10:02:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608907 s, 172 MB/s 2024-04-25T10:02:43.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.866 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708678 s, 148 MB/s 2024-04-25T10:02:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:43.873 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713079 s, 147 MB/s 2024-04-25T10:02:43.882 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076747 s, 137 MB/s 2024-04-25T10:02:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622617 s, 168 MB/s 2024-04-25T10:02:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689789 s, 152 MB/s 2024-04-25T10:02:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621688 s, 169 MB/s 2024-04-25T10:02:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706304 s, 148 MB/s 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680611 s, 154 MB/s 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649482 s, 161 MB/s 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627258 s, 167 MB/s 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065398 s, 160 MB/s 2024-04-25T10:02:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627535 s, 167 MB/s 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688875 s, 152 MB/s 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692709 s, 151 MB/s 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719683 s, 146 MB/s 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.013 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685705 s, 153 MB/s 2024-04-25T10:02:44.013 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.013 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.013 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656421 s, 160 MB/s 2024-04-25T10:02:44.013 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.013 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.013 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670258 s, 156 MB/s 2024-04-25T10:02:44.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.013 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:44.013 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:44.013 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:02:44.014 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:44.014 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:44.014 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:44.014 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710614 s, 148 MB/s 2024-04-25T10:02:44.084 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.084 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.084 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717868 s, 146 MB/s 2024-04-25T10:02:44.092 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.092 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.092 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643539 s, 163 MB/s 2024-04-25T10:02:44.101 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.101 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.101 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818367 s, 128 MB/s 2024-04-25T10:02:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822957 s, 127 MB/s 2024-04-25T10:02:44.117 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.117 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.117 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635022 s, 165 MB/s 2024-04-25T10:02:44.124 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.124 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.124 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605052 s, 173 MB/s 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679322 s, 154 MB/s 2024-04-25T10:02:44.139 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.139 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.139 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666221 s, 157 MB/s 2024-04-25T10:02:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687264 s, 153 MB/s 2024-04-25T10:02:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641473 s, 163 MB/s 2024-04-25T10:02:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.162 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.162 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638224 s, 164 MB/s 2024-04-25T10:02:44.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675457 s, 155 MB/s 2024-04-25T10:02:44.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.178 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075727 s, 138 MB/s 2024-04-25T10:02:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.185 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627169 s, 167 MB/s 2024-04-25T10:02:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.193 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719255 s, 146 MB/s 2024-04-25T10:02:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701308 s, 150 MB/s 2024-04-25T10:02:44.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.209 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067348 s, 156 MB/s 2024-04-25T10:02:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625438 s, 168 MB/s 2024-04-25T10:02:44.223 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.224 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.224 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698468 s, 150 MB/s 2024-04-25T10:02:44.231 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.231 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.231 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693025 s, 151 MB/s 2024-04-25T10:02:44.239 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729621 s, 144 MB/s 2024-04-25T10:02:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684585 s, 153 MB/s 2024-04-25T10:02:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.255 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.255 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637041 s, 165 MB/s 2024-04-25T10:02:44.262 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.262 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.262 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676382 s, 155 MB/s 2024-04-25T10:02:44.269 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.270 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.270 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668291 s, 157 MB/s 2024-04-25T10:02:44.277 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.277 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.277 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637919 s, 164 MB/s 2024-04-25T10:02:44.285 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.285 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.285 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682554 s, 154 MB/s 2024-04-25T10:02:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639815 s, 164 MB/s 2024-04-25T10:02:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652075 s, 161 MB/s 2024-04-25T10:02:44.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.307 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687596 s, 152 MB/s 2024-04-25T10:02:44.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.315 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689569 s, 152 MB/s 2024-04-25T10:02:44.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.316 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:44.316 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:44.316 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:02:44.316 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:44.316 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:44.316 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:44.316 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:44.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.381 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063222 s, 166 MB/s 2024-04-25T10:02:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.389 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719611 s, 146 MB/s 2024-04-25T10:02:44.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.396 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604477 s, 173 MB/s 2024-04-25T10:02:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721905 s, 145 MB/s 2024-04-25T10:02:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713327 s, 147 MB/s 2024-04-25T10:02:44.419 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.419 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.420 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620693 s, 169 MB/s 2024-04-25T10:02:44.426 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.426 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.427 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625953 s, 168 MB/s 2024-04-25T10:02:44.434 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.434 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.434 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633914 s, 165 MB/s 2024-04-25T10:02:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681575 s, 154 MB/s 2024-04-25T10:02:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.450 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.450 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722773 s, 145 MB/s 2024-04-25T10:02:44.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.458 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.458 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727348 s, 144 MB/s 2024-04-25T10:02:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657696 s, 159 MB/s 2024-04-25T10:02:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634684 s, 165 MB/s 2024-04-25T10:02:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676215 s, 155 MB/s 2024-04-25T10:02:44.488 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.488 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.488 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073905 s, 142 MB/s 2024-04-25T10:02:44.496 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.496 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.496 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684528 s, 153 MB/s 2024-04-25T10:02:44.504 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.504 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.504 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713961 s, 147 MB/s 2024-04-25T10:02:44.511 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.512 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635408 s, 165 MB/s 2024-04-25T10:02:44.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.519 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.519 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672242 s, 156 MB/s 2024-04-25T10:02:44.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.527 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698042 s, 150 MB/s 2024-04-25T10:02:44.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.534 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.535 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647838 s, 162 MB/s 2024-04-25T10:02:44.542 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.543 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701606 s, 149 MB/s 2024-04-25T10:02:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719881 s, 146 MB/s 2024-04-25T10:02:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701222 s, 150 MB/s 2024-04-25T10:02:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692385 s, 151 MB/s 2024-04-25T10:02:44.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.574 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.574 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643167 s, 163 MB/s 2024-04-25T10:02:44.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.581 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668293 s, 157 MB/s 2024-04-25T10:02:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.589 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652528 s, 161 MB/s 2024-04-25T10:02:44.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.596 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.596 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654475 s, 160 MB/s 2024-04-25T10:02:44.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.603 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660092 s, 159 MB/s 2024-04-25T10:02:44.611 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.611 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.611 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681122 s, 154 MB/s 2024-04-25T10:02:44.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.619 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693861 s, 151 MB/s 2024-04-25T10:02:44.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.621 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:44.621 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:44.621 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:02:44.621 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:44.622 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:44.622 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:44.622 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:44.688 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.688 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.688 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749712 s, 140 MB/s 2024-04-25T10:02:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670676 s, 156 MB/s 2024-04-25T10:02:44.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.704 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728371 s, 144 MB/s 2024-04-25T10:02:44.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.711 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063454 s, 165 MB/s 2024-04-25T10:02:44.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.719 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706749 s, 148 MB/s 2024-04-25T10:02:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723374 s, 145 MB/s 2024-04-25T10:02:44.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.735 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691905 s, 152 MB/s 2024-04-25T10:02:44.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.742 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.742 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620538 s, 169 MB/s 2024-04-25T10:02:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658491 s, 159 MB/s 2024-04-25T10:02:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678719 s, 154 MB/s 2024-04-25T10:02:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708325 s, 148 MB/s 2024-04-25T10:02:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630432 s, 166 MB/s 2024-04-25T10:02:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666759 s, 157 MB/s 2024-04-25T10:02:44.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.787 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653899 s, 160 MB/s 2024-04-25T10:02:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069919 s, 150 MB/s 2024-04-25T10:02:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628045 s, 167 MB/s 2024-04-25T10:02:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070025 s, 150 MB/s 2024-04-25T10:02:44.817 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.817 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.817 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691965 s, 152 MB/s 2024-04-25T10:02:44.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.827 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833898 s, 126 MB/s 2024-04-25T10:02:44.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.835 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702138 s, 149 MB/s 2024-04-25T10:02:44.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.842 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.842 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629378 s, 167 MB/s 2024-04-25T10:02:44.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.850 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.850 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702955 s, 149 MB/s 2024-04-25T10:02:44.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.857 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632299 s, 166 MB/s 2024-04-25T10:02:44.864 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.865 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064504 s, 163 MB/s 2024-04-25T10:02:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.872 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621038 s, 169 MB/s 2024-04-25T10:02:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.878 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.879 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631656 s, 166 MB/s 2024-04-25T10:02:44.886 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.886 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.886 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656784 s, 160 MB/s 2024-04-25T10:02:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.894 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697984 s, 150 MB/s 2024-04-25T10:02:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.901 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.902 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666076 s, 157 MB/s 2024-04-25T10:02:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.909 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616313 s, 170 MB/s 2024-04-25T10:02:44.916 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.916 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.916 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630871 s, 166 MB/s 2024-04-25T10:02:44.924 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.925 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732756 s, 143 MB/s 2024-04-25T10:02:44.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.925 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:44.925 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:44.925 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:02:44.925 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:44.925 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:44.925 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:44.925 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:44.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:44.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:44.992 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758295 s, 138 MB/s 2024-04-25T10:02:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696011 s, 151 MB/s 2024-04-25T10:02:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066145 s, 159 MB/s 2024-04-25T10:02:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642267 s, 163 MB/s 2024-04-25T10:02:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676883 s, 155 MB/s 2024-04-25T10:02:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.030 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651532 s, 161 MB/s 2024-04-25T10:02:45.038 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.038 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.038 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071663 s, 146 MB/s 2024-04-25T10:02:45.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.046 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709862 s, 148 MB/s 2024-04-25T10:02:45.053 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.053 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.053 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649652 s, 161 MB/s 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700646 s, 150 MB/s 2024-04-25T10:02:45.068 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.068 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.068 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637984 s, 164 MB/s 2024-04-25T10:02:45.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613753 s, 171 MB/s 2024-04-25T10:02:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.082 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060206 s, 174 MB/s 2024-04-25T10:02:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.090 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649953 s, 161 MB/s 2024-04-25T10:02:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674144 s, 156 MB/s 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692612 s, 151 MB/s 2024-04-25T10:02:45.112 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.112 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.112 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617918 s, 170 MB/s 2024-04-25T10:02:45.120 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.120 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.120 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065898 s, 159 MB/s 2024-04-25T10:02:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653318 s, 161 MB/s 2024-04-25T10:02:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730557 s, 144 MB/s 2024-04-25T10:02:45.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.142 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611092 s, 172 MB/s 2024-04-25T10:02:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720201 s, 146 MB/s 2024-04-25T10:02:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721548 s, 145 MB/s 2024-04-25T10:02:45.168 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.168 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.168 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856935 s, 122 MB/s 2024-04-25T10:02:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708157 s, 148 MB/s 2024-04-25T10:02:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660402 s, 159 MB/s 2024-04-25T10:02:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720659 s, 146 MB/s 2024-04-25T10:02:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639644 s, 164 MB/s 2024-04-25T10:02:45.207 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.207 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.207 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733897 s, 143 MB/s 2024-04-25T10:02:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.214 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685825 s, 153 MB/s 2024-04-25T10:02:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680941 s, 154 MB/s 2024-04-25T10:02:45.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.230 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681753 s, 154 MB/s 2024-04-25T10:02:45.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.231 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:45.231 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:45.231 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:02:45.231 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:45.231 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:45.231 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:45.232 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730387 s, 144 MB/s 2024-04-25T10:02:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621969 s, 169 MB/s 2024-04-25T10:02:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671022 s, 156 MB/s 2024-04-25T10:02:45.320 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.320 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.320 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075232 s, 139 MB/s 2024-04-25T10:02:45.328 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.328 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.328 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717465 s, 146 MB/s 2024-04-25T10:02:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645164 s, 163 MB/s 2024-04-25T10:02:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694721 s, 151 MB/s 2024-04-25T10:02:45.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.351 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685572 s, 153 MB/s 2024-04-25T10:02:45.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.359 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698198 s, 150 MB/s 2024-04-25T10:02:45.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.366 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.366 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641477 s, 163 MB/s 2024-04-25T10:02:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653229 s, 161 MB/s 2024-04-25T10:02:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.381 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.382 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720811 s, 145 MB/s 2024-04-25T10:02:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.389 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065686 s, 160 MB/s 2024-04-25T10:02:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.397 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717218 s, 146 MB/s 2024-04-25T10:02:45.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.405 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709064 s, 148 MB/s 2024-04-25T10:02:45.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.413 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688374 s, 152 MB/s 2024-04-25T10:02:45.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.421 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687466 s, 153 MB/s 2024-04-25T10:02:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.429 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723489 s, 145 MB/s 2024-04-25T10:02:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752594 s, 139 MB/s 2024-04-25T10:02:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.445 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633575 s, 166 MB/s 2024-04-25T10:02:45.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.453 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814406 s, 129 MB/s 2024-04-25T10:02:45.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.461 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695745 s, 151 MB/s 2024-04-25T10:02:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710134 s, 148 MB/s 2024-04-25T10:02:45.479 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.479 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.479 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855585 s, 123 MB/s 2024-04-25T10:02:45.486 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.487 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.487 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699892 s, 150 MB/s 2024-04-25T10:02:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.494 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651896 s, 161 MB/s 2024-04-25T10:02:45.502 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.502 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.502 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707297 s, 148 MB/s 2024-04-25T10:02:45.510 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.510 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.510 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726617 s, 144 MB/s 2024-04-25T10:02:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616553 s, 170 MB/s 2024-04-25T10:02:45.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.524 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570632 s, 184 MB/s 2024-04-25T10:02:45.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.532 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070102 s, 150 MB/s 2024-04-25T10:02:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620663 s, 169 MB/s 2024-04-25T10:02:45.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.540 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:45.540 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:45.540 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:02:45.540 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:45.540 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:45.541 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:45.541 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:45.605 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.606 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658107 s, 159 MB/s 2024-04-25T10:02:45.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.614 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657692 s, 159 MB/s 2024-04-25T10:02:45.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.620 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647825 s, 162 MB/s 2024-04-25T10:02:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708533 s, 148 MB/s 2024-04-25T10:02:45.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.636 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063131 s, 166 MB/s 2024-04-25T10:02:45.643 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.643 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.643 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063921 s, 164 MB/s 2024-04-25T10:02:45.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.651 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703497 s, 149 MB/s 2024-04-25T10:02:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656353 s, 160 MB/s 2024-04-25T10:02:45.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709115 s, 148 MB/s 2024-04-25T10:02:45.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.674 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.674 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681422 s, 154 MB/s 2024-04-25T10:02:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.681 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622503 s, 168 MB/s 2024-04-25T10:02:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912287 s, 115 MB/s 2024-04-25T10:02:45.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.699 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733311 s, 143 MB/s 2024-04-25T10:02:45.706 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.706 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.706 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622715 s, 168 MB/s 2024-04-25T10:02:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653895 s, 160 MB/s 2024-04-25T10:02:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649955 s, 161 MB/s 2024-04-25T10:02:45.729 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.729 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.729 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706524 s, 148 MB/s 2024-04-25T10:02:45.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.737 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688506 s, 152 MB/s 2024-04-25T10:02:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711049 s, 147 MB/s 2024-04-25T10:02:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625052 s, 168 MB/s 2024-04-25T10:02:45.760 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.760 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.760 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688425 s, 152 MB/s 2024-04-25T10:02:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.768 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745544 s, 141 MB/s 2024-04-25T10:02:45.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.776 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699133 s, 150 MB/s 2024-04-25T10:02:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687693 s, 152 MB/s 2024-04-25T10:02:45.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.792 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699336 s, 150 MB/s 2024-04-25T10:02:45.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.800 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715015 s, 147 MB/s 2024-04-25T10:02:45.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.807 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662792 s, 158 MB/s 2024-04-25T10:02:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728826 s, 144 MB/s 2024-04-25T10:02:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700742 s, 150 MB/s 2024-04-25T10:02:45.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.831 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.831 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692642 s, 151 MB/s 2024-04-25T10:02:45.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.839 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643917 s, 163 MB/s 2024-04-25T10:02:45.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.847 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.847 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753394 s, 139 MB/s 2024-04-25T10:02:45.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.848 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:45.848 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:45.848 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:02:45.848 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:45.848 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:45.848 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:45.849 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:45.913 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.913 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.913 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673114 s, 156 MB/s 2024-04-25T10:02:45.921 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.921 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.921 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684696 s, 153 MB/s 2024-04-25T10:02:45.929 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.929 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.929 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680784 s, 154 MB/s 2024-04-25T10:02:45.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.937 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007121 s, 147 MB/s 2024-04-25T10:02:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716235 s, 146 MB/s 2024-04-25T10:02:45.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.952 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646971 s, 162 MB/s 2024-04-25T10:02:45.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.960 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644395 s, 163 MB/s 2024-04-25T10:02:45.966 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.966 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.967 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063067 s, 166 MB/s 2024-04-25T10:02:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649023 s, 162 MB/s 2024-04-25T10:02:45.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.982 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.982 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731661 s, 143 MB/s 2024-04-25T10:02:45.989 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.990 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.990 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695557 s, 151 MB/s 2024-04-25T10:02:45.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:45.998 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:45.999 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793111 s, 132 MB/s 2024-04-25T10:02:46.006 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.007 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.007 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722183 s, 145 MB/s 2024-04-25T10:02:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.015 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733813 s, 143 MB/s 2024-04-25T10:02:46.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.026 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729609 s, 144 MB/s 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681208 s, 154 MB/s 2024-04-25T10:02:46.038 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.039 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.039 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697611 s, 150 MB/s 2024-04-25T10:02:46.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.046 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625625 s, 168 MB/s 2024-04-25T10:02:46.053 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.053 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.054 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678786 s, 154 MB/s 2024-04-25T10:02:46.061 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.061 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.061 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688967 s, 152 MB/s 2024-04-25T10:02:46.069 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.069 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.069 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662332 s, 158 MB/s 2024-04-25T10:02:46.077 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.077 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.077 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679074 s, 154 MB/s 2024-04-25T10:02:46.085 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.085 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.085 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724882 s, 145 MB/s 2024-04-25T10:02:46.092 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.092 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.092 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634906 s, 165 MB/s 2024-04-25T10:02:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637587 s, 164 MB/s 2024-04-25T10:02:46.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.107 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.107 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714681 s, 147 MB/s 2024-04-25T10:02:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.116 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720716 s, 145 MB/s 2024-04-25T10:02:46.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.123 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688781 s, 152 MB/s 2024-04-25T10:02:46.132 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.132 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.132 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742667 s, 141 MB/s 2024-04-25T10:02:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007054 s, 149 MB/s 2024-04-25T10:02:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.147 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640541 s, 164 MB/s 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676219 s, 155 MB/s 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.156 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:46.156 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:46.156 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:02:46.156 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:46.156 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:46.156 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:46.156 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.222 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720298 s, 146 MB/s 2024-04-25T10:02:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.231 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701059 s, 150 MB/s 2024-04-25T10:02:46.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.238 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643028 s, 163 MB/s 2024-04-25T10:02:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689923 s, 152 MB/s 2024-04-25T10:02:46.255 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.255 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.255 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863081 s, 121 MB/s 2024-04-25T10:02:46.262 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.262 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.263 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690753 s, 152 MB/s 2024-04-25T10:02:46.270 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.270 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.270 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652473 s, 161 MB/s 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724023 s, 145 MB/s 2024-04-25T10:02:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082441 s, 127 MB/s 2024-04-25T10:02:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656489 s, 160 MB/s 2024-04-25T10:02:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635957 s, 165 MB/s 2024-04-25T10:02:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708913 s, 148 MB/s 2024-04-25T10:02:46.318 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.318 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.318 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071953 s, 146 MB/s 2024-04-25T10:02:46.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.328 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851968 s, 123 MB/s 2024-04-25T10:02:46.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.336 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759497 s, 138 MB/s 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721843 s, 145 MB/s 2024-04-25T10:02:46.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.352 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669786 s, 157 MB/s 2024-04-25T10:02:46.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.360 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692895 s, 151 MB/s 2024-04-25T10:02:46.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.367 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064512 s, 163 MB/s 2024-04-25T10:02:46.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.375 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066085 s, 159 MB/s 2024-04-25T10:02:46.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.383 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.383 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725943 s, 144 MB/s 2024-04-25T10:02:46.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.392 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733541 s, 143 MB/s 2024-04-25T10:02:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073976 s, 142 MB/s 2024-04-25T10:02:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785379 s, 134 MB/s 2024-04-25T10:02:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640974 s, 164 MB/s 2024-04-25T10:02:46.424 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.425 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736853 s, 142 MB/s 2024-04-25T10:02:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684059 s, 153 MB/s 2024-04-25T10:02:46.440 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.440 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.440 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714873 s, 147 MB/s 2024-04-25T10:02:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.448 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714607 s, 147 MB/s 2024-04-25T10:02:46.456 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.457 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723821 s, 145 MB/s 2024-04-25T10:02:46.464 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.464 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.465 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070305 s, 149 MB/s 2024-04-25T10:02:46.472 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.472 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.472 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647061 s, 162 MB/s 2024-04-25T10:02:46.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.473 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:46.473 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:46.473 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:02:46.473 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:46.474 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:46.474 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:46.474 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:46.539 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.539 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.539 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726248 s, 144 MB/s 2024-04-25T10:02:46.547 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.547 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662413 s, 158 MB/s 2024-04-25T10:02:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.555 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066387 s, 158 MB/s 2024-04-25T10:02:46.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.564 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.564 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724129 s, 145 MB/s 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681201 s, 154 MB/s 2024-04-25T10:02:46.578 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.578 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.578 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732128 s, 143 MB/s 2024-04-25T10:02:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.586 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614712 s, 171 MB/s 2024-04-25T10:02:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785411 s, 134 MB/s 2024-04-25T10:02:46.602 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.602 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.602 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694126 s, 151 MB/s 2024-04-25T10:02:46.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.610 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.610 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705859 s, 149 MB/s 2024-04-25T10:02:46.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.617 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.617 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575763 s, 182 MB/s 2024-04-25T10:02:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.625 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.625 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677381 s, 155 MB/s 2024-04-25T10:02:46.632 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.632 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.633 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709592 s, 148 MB/s 2024-04-25T10:02:46.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.642 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737487 s, 142 MB/s 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667973 s, 157 MB/s 2024-04-25T10:02:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.657 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803185 s, 131 MB/s 2024-04-25T10:02:46.665 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.665 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.665 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687032 s, 153 MB/s 2024-04-25T10:02:46.673 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.673 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.673 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733089 s, 143 MB/s 2024-04-25T10:02:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.681 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069589 s, 151 MB/s 2024-04-25T10:02:46.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.689 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.689 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674342 s, 155 MB/s 2024-04-25T10:02:46.696 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.697 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.697 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672658 s, 156 MB/s 2024-04-25T10:02:46.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.704 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.704 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612958 s, 171 MB/s 2024-04-25T10:02:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.711 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678184 s, 155 MB/s 2024-04-25T10:02:46.719 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.720 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.720 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713052 s, 147 MB/s 2024-04-25T10:02:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622709 s, 168 MB/s 2024-04-25T10:02:46.733 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.734 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619631 s, 169 MB/s 2024-04-25T10:02:46.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.741 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647767 s, 162 MB/s 2024-04-25T10:02:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.749 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687807 s, 152 MB/s 2024-04-25T10:02:46.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.756 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.756 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681698 s, 154 MB/s 2024-04-25T10:02:46.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.766 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.766 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828395 s, 127 MB/s 2024-04-25T10:02:46.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.773 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639672 s, 164 MB/s 2024-04-25T10:02:46.781 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.781 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.781 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692521 s, 151 MB/s 2024-04-25T10:02:46.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.782 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:46.782 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:46.782 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:02:46.782 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:46.782 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:46.782 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:46.782 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747108 s, 140 MB/s 2024-04-25T10:02:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.857 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725189 s, 145 MB/s 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697334 s, 150 MB/s 2024-04-25T10:02:46.872 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.873 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741722 s, 141 MB/s 2024-04-25T10:02:46.880 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.880 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.880 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681102 s, 154 MB/s 2024-04-25T10:02:46.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.887 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.887 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629409 s, 167 MB/s 2024-04-25T10:02:46.894 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642045 s, 163 MB/s 2024-04-25T10:02:46.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.903 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736309 s, 142 MB/s 2024-04-25T10:02:46.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.911 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705442 s, 149 MB/s 2024-04-25T10:02:46.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.919 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.919 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685015 s, 153 MB/s 2024-04-25T10:02:46.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.927 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.927 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727729 s, 144 MB/s 2024-04-25T10:02:46.935 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.935 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.935 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069571 s, 151 MB/s 2024-04-25T10:02:46.943 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.943 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.943 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728753 s, 144 MB/s 2024-04-25T10:02:46.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.951 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.951 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067754 s, 155 MB/s 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676265 s, 155 MB/s 2024-04-25T10:02:46.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.965 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.966 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635832 s, 165 MB/s 2024-04-25T10:02:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651411 s, 161 MB/s 2024-04-25T10:02:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710978 s, 147 MB/s 2024-04-25T10:02:46.988 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.988 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.988 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646254 s, 162 MB/s 2024-04-25T10:02:46.996 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:46.996 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:46.996 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692786 s, 151 MB/s 2024-04-25T10:02:47.003 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.004 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.004 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677977 s, 155 MB/s 2024-04-25T10:02:47.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.011 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066088 s, 159 MB/s 2024-04-25T10:02:47.019 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.019 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.019 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676909 s, 155 MB/s 2024-04-25T10:02:47.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.026 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.027 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671868 s, 156 MB/s 2024-04-25T10:02:47.034 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.035 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.035 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704866 s, 149 MB/s 2024-04-25T10:02:47.042 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.042 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.042 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696755 s, 150 MB/s 2024-04-25T10:02:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.049 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612072 s, 171 MB/s 2024-04-25T10:02:47.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.058 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728545 s, 144 MB/s 2024-04-25T10:02:47.065 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.065 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.066 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678292 s, 155 MB/s 2024-04-25T10:02:47.073 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737068 s, 142 MB/s 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688161 s, 152 MB/s 2024-04-25T10:02:47.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.089 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.090 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072608 s, 144 MB/s 2024-04-25T10:02:47.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.091 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:47.091 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:47.091 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:02:47.091 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:47.091 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:47.091 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:47.091 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658795 s, 159 MB/s 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723654 s, 145 MB/s 2024-04-25T10:02:47.172 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.172 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.172 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624576 s, 168 MB/s 2024-04-25T10:02:47.181 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.182 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854614 s, 123 MB/s 2024-04-25T10:02:47.189 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.189 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.189 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673454 s, 156 MB/s 2024-04-25T10:02:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713372 s, 147 MB/s 2024-04-25T10:02:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636463 s, 165 MB/s 2024-04-25T10:02:47.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.212 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.212 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686898 s, 153 MB/s 2024-04-25T10:02:47.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.220 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.220 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672416 s, 156 MB/s 2024-04-25T10:02:47.228 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.229 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774829 s, 135 MB/s 2024-04-25T10:02:47.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.237 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729873 s, 144 MB/s 2024-04-25T10:02:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.245 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.245 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691743 s, 152 MB/s 2024-04-25T10:02:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691426 s, 152 MB/s 2024-04-25T10:02:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.260 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681294 s, 154 MB/s 2024-04-25T10:02:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.269 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800644 s, 131 MB/s 2024-04-25T10:02:47.277 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.277 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.277 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071666 s, 146 MB/s 2024-04-25T10:02:47.285 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.285 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.285 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668717 s, 157 MB/s 2024-04-25T10:02:47.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.292 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.292 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608804 s, 172 MB/s 2024-04-25T10:02:47.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.299 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677764 s, 155 MB/s 2024-04-25T10:02:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626802 s, 167 MB/s 2024-04-25T10:02:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791533 s, 132 MB/s 2024-04-25T10:02:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657564 s, 159 MB/s 2024-04-25T10:02:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734404 s, 143 MB/s 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677157 s, 155 MB/s 2024-04-25T10:02:47.346 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.346 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.346 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663491 s, 158 MB/s 2024-04-25T10:02:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.355 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.355 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717669 s, 146 MB/s 2024-04-25T10:02:47.362 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.362 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.362 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621615 s, 169 MB/s 2024-04-25T10:02:47.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.370 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718756 s, 146 MB/s 2024-04-25T10:02:47.378 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.378 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.378 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719411 s, 146 MB/s 2024-04-25T10:02:47.385 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.385 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.385 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632434 s, 166 MB/s 2024-04-25T10:02:47.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.392 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626677 s, 167 MB/s 2024-04-25T10:02:47.400 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.400 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.400 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702625 s, 149 MB/s 2024-04-25T10:02:47.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.401 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:47.402 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:47.402 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:02:47.402 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:47.402 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:47.402 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:47.402 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:47.468 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.468 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.468 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788922 s, 133 MB/s 2024-04-25T10:02:47.476 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.476 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.476 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739961 s, 142 MB/s 2024-04-25T10:02:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613367 s, 171 MB/s 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666142 s, 157 MB/s 2024-04-25T10:02:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647989 s, 162 MB/s 2024-04-25T10:02:47.507 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.507 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.507 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081786 s, 128 MB/s 2024-04-25T10:02:47.515 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.515 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.515 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723364 s, 145 MB/s 2024-04-25T10:02:47.523 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.523 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.523 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675348 s, 155 MB/s 2024-04-25T10:02:47.530 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.530 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.530 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639631 s, 164 MB/s 2024-04-25T10:02:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721059 s, 145 MB/s 2024-04-25T10:02:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.546 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624297 s, 168 MB/s 2024-04-25T10:02:47.554 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.554 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.554 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792141 s, 132 MB/s 2024-04-25T10:02:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616629 s, 170 MB/s 2024-04-25T10:02:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734395 s, 143 MB/s 2024-04-25T10:02:47.577 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.577 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.578 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727812 s, 144 MB/s 2024-04-25T10:02:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.585 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627749 s, 167 MB/s 2024-04-25T10:02:47.592 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.592 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.592 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064707 s, 162 MB/s 2024-04-25T10:02:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.599 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640954 s, 164 MB/s 2024-04-25T10:02:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.607 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658064 s, 159 MB/s 2024-04-25T10:02:47.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.615 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.615 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740577 s, 142 MB/s 2024-04-25T10:02:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714422 s, 147 MB/s 2024-04-25T10:02:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728771 s, 144 MB/s 2024-04-25T10:02:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.639 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673663 s, 156 MB/s 2024-04-25T10:02:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.647 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718066 s, 146 MB/s 2024-04-25T10:02:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.655 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727262 s, 144 MB/s 2024-04-25T10:02:47.665 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.665 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.665 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699486 s, 150 MB/s 2024-04-25T10:02:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.671 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680887 s, 154 MB/s 2024-04-25T10:02:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679128 s, 154 MB/s 2024-04-25T10:02:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694633 s, 151 MB/s 2024-04-25T10:02:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721069 s, 145 MB/s 2024-04-25T10:02:47.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.703 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.704 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840301 s, 125 MB/s 2024-04-25T10:02:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.712 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762608 s, 137 MB/s 2024-04-25T10:02:47.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.713 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:47.713 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:47.713 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:02:47.713 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:47.714 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:47.714 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:47.714 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:47.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.778 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.778 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628525 s, 167 MB/s 2024-04-25T10:02:47.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.785 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.785 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636279 s, 165 MB/s 2024-04-25T10:02:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591505 s, 177 MB/s 2024-04-25T10:02:48.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.710 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.916835 s, 1.1 MB/s 2024-04-25T10:02:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.723 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012028 s, 87.2 MB/s 2024-04-25T10:02:48.733 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.734 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950662 s, 110 MB/s 2024-04-25T10:02:48.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.743 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.743 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796507 s, 132 MB/s 2024-04-25T10:02:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679745 s, 154 MB/s 2024-04-25T10:02:48.760 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.760 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.760 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770234 s, 136 MB/s 2024-04-25T10:02:48.767 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.767 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.767 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674014 s, 156 MB/s 2024-04-25T10:02:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730367 s, 144 MB/s 2024-04-25T10:02:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.783 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068438 s, 153 MB/s 2024-04-25T10:02:48.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.791 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736711 s, 142 MB/s 2024-04-25T10:02:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.799 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710484 s, 148 MB/s 2024-04-25T10:02:48.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733858 s, 143 MB/s 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072452 s, 145 MB/s 2024-04-25T10:02:48.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.823 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.823 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617301 s, 170 MB/s 2024-04-25T10:02:48.830 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.830 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657478 s, 159 MB/s 2024-04-25T10:02:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647339 s, 162 MB/s 2024-04-25T10:02:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707535 s, 148 MB/s 2024-04-25T10:02:48.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.854 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753134 s, 139 MB/s 2024-04-25T10:02:48.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.867 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.867 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785829 s, 133 MB/s 2024-04-25T10:02:48.872 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.873 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.873 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851151 s, 123 MB/s 2024-04-25T10:02:48.880 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700821 s, 150 MB/s 2024-04-25T10:02:48.889 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.890 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065259 s, 161 MB/s 2024-04-25T10:02:48.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.896 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659441 s, 159 MB/s 2024-04-25T10:02:48.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.904 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.904 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072323 s, 145 MB/s 2024-04-25T10:02:48.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.911 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.912 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621458 s, 169 MB/s 2024-04-25T10:02:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.920 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855814 s, 123 MB/s 2024-04-25T10:02:48.928 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.928 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.928 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685785 s, 153 MB/s 2024-04-25T10:02:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709713 s, 148 MB/s 2024-04-25T10:02:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732392 s, 143 MB/s 2024-04-25T10:02:48.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:48.945 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:48.945 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:48.946 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:02:48.946 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:48.946 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:48.946 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:48.946 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:49.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.011 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.011 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072189 s, 145 MB/s 2024-04-25T10:02:49.018 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.018 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.018 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064189 s, 163 MB/s 2024-04-25T10:02:49.027 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.027 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.027 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734562 s, 143 MB/s 2024-04-25T10:02:49.035 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.035 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.035 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071798 s, 146 MB/s 2024-04-25T10:02:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728579 s, 144 MB/s 2024-04-25T10:02:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667487 s, 157 MB/s 2024-04-25T10:02:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841021 s, 125 MB/s 2024-04-25T10:02:49.067 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.067 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.067 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637833 s, 164 MB/s 2024-04-25T10:02:49.074 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.075 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662046 s, 158 MB/s 2024-04-25T10:02:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742858 s, 141 MB/s 2024-04-25T10:02:49.090 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.093 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.093 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636655 s, 165 MB/s 2024-04-25T10:02:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682896 s, 154 MB/s 2024-04-25T10:02:49.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.106 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.106 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706881 s, 148 MB/s 2024-04-25T10:02:49.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.115 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.115 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830191 s, 126 MB/s 2024-04-25T10:02:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678981 s, 154 MB/s 2024-04-25T10:02:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712812 s, 147 MB/s 2024-04-25T10:02:49.139 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.139 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.139 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700382 s, 150 MB/s 2024-04-25T10:02:49.146 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.147 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.147 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691284 s, 152 MB/s 2024-04-25T10:02:49.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.155 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751125 s, 140 MB/s 2024-04-25T10:02:49.163 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.164 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.164 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766811 s, 137 MB/s 2024-04-25T10:02:49.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.171 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.171 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685024 s, 153 MB/s 2024-04-25T10:02:49.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.178 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.179 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629047 s, 167 MB/s 2024-04-25T10:02:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650714 s, 161 MB/s 2024-04-25T10:02:49.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.193 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633961 s, 165 MB/s 2024-04-25T10:02:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707744 s, 148 MB/s 2024-04-25T10:02:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692509 s, 151 MB/s 2024-04-25T10:02:49.216 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.216 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670345 s, 156 MB/s 2024-04-25T10:02:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730344 s, 144 MB/s 2024-04-25T10:02:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834297 s, 126 MB/s 2024-04-25T10:02:49.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.241 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063065 s, 166 MB/s 2024-04-25T10:02:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.249 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750201 s, 140 MB/s 2024-04-25T10:02:49.257 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.257 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.258 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070947 s, 148 MB/s 2024-04-25T10:02:49.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.259 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:49.259 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:49.259 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:02:49.259 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:49.259 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:49.259 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:49.259 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:49.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.328 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701621 s, 149 MB/s 2024-04-25T10:02:49.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.336 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745231 s, 141 MB/s 2024-04-25T10:02:49.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.344 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703345 s, 149 MB/s 2024-04-25T10:02:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703962 s, 149 MB/s 2024-04-25T10:02:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693876 s, 151 MB/s 2024-04-25T10:02:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663248 s, 158 MB/s 2024-04-25T10:02:49.374 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667845 s, 157 MB/s 2024-04-25T10:02:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690558 s, 152 MB/s 2024-04-25T10:02:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.391 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.391 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721041 s, 145 MB/s 2024-04-25T10:02:49.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.398 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.398 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062941 s, 167 MB/s 2024-04-25T10:02:49.405 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.406 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.406 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699149 s, 150 MB/s 2024-04-25T10:02:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064448 s, 163 MB/s 2024-04-25T10:02:49.421 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.421 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.421 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707006 s, 148 MB/s 2024-04-25T10:02:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742041 s, 141 MB/s 2024-04-25T10:02:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070485 s, 149 MB/s 2024-04-25T10:02:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720151 s, 146 MB/s 2024-04-25T10:02:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655887 s, 160 MB/s 2024-04-25T10:02:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784567 s, 134 MB/s 2024-04-25T10:02:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068979 s, 152 MB/s 2024-04-25T10:02:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653078 s, 161 MB/s 2024-04-25T10:02:49.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.485 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711087 s, 147 MB/s 2024-04-25T10:02:49.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.493 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.493 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772517 s, 136 MB/s 2024-04-25T10:02:49.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.501 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.501 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690859 s, 152 MB/s 2024-04-25T10:02:49.509 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.509 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.509 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694756 s, 151 MB/s 2024-04-25T10:02:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070749 s, 148 MB/s 2024-04-25T10:02:49.524 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685674 s, 153 MB/s 2024-04-25T10:02:49.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.532 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.532 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690459 s, 152 MB/s 2024-04-25T10:02:49.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.540 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.540 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711429 s, 147 MB/s 2024-04-25T10:02:49.548 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.549 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.549 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072061 s, 146 MB/s 2024-04-25T10:02:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.556 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655117 s, 160 MB/s 2024-04-25T10:02:49.563 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.563 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.563 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633329 s, 166 MB/s 2024-04-25T10:02:49.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.570 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.570 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620241 s, 169 MB/s 2024-04-25T10:02:49.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.572 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:49.572 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:49.572 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:02:49.572 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:49.572 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:49.572 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:49.572 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070019 s, 150 MB/s 2024-04-25T10:02:49.644 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.645 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.645 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715222 s, 147 MB/s 2024-04-25T10:02:49.652 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.652 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.652 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676084 s, 155 MB/s 2024-04-25T10:02:49.660 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.660 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.661 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704531 s, 149 MB/s 2024-04-25T10:02:49.668 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.668 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.668 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725995 s, 144 MB/s 2024-04-25T10:02:49.676 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.676 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.676 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707655 s, 148 MB/s 2024-04-25T10:02:49.684 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.684 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.684 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070493 s, 149 MB/s 2024-04-25T10:02:49.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.692 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687072 s, 153 MB/s 2024-04-25T10:02:49.701 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.701 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.701 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083269 s, 126 MB/s 2024-04-25T10:02:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834713 s, 126 MB/s 2024-04-25T10:02:49.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.718 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.718 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071249 s, 147 MB/s 2024-04-25T10:02:49.726 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.726 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.726 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703139 s, 149 MB/s 2024-04-25T10:02:49.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.734 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.734 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668855 s, 157 MB/s 2024-04-25T10:02:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649586 s, 161 MB/s 2024-04-25T10:02:49.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.750 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.750 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732833 s, 143 MB/s 2024-04-25T10:02:49.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.757 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620696 s, 169 MB/s 2024-04-25T10:02:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709076 s, 148 MB/s 2024-04-25T10:02:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737572 s, 142 MB/s 2024-04-25T10:02:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719066 s, 146 MB/s 2024-04-25T10:02:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713859 s, 147 MB/s 2024-04-25T10:02:49.797 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.797 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.797 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741118 s, 141 MB/s 2024-04-25T10:02:49.804 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.805 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063272 s, 166 MB/s 2024-04-25T10:02:49.812 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.813 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.813 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687306 s, 153 MB/s 2024-04-25T10:02:49.822 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.822 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.822 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841442 s, 125 MB/s 2024-04-25T10:02:49.830 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.830 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.830 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711831 s, 147 MB/s 2024-04-25T10:02:49.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.837 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.837 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610157 s, 172 MB/s 2024-04-25T10:02:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.845 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.845 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675117 s, 155 MB/s 2024-04-25T10:02:49.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.854 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818973 s, 128 MB/s 2024-04-25T10:02:49.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.861 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676029 s, 155 MB/s 2024-04-25T10:02:49.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.869 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.869 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714894 s, 147 MB/s 2024-04-25T10:02:49.877 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.877 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.877 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639871 s, 164 MB/s 2024-04-25T10:02:49.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.885 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.885 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698886 s, 150 MB/s 2024-04-25T10:02:49.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.887 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:49.887 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:49.888 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:02:49.888 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:49.888 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:49.889 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:49.889 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:49.953 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.953 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.953 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746321 s, 140 MB/s 2024-04-25T10:02:49.961 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.961 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.961 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070923 s, 148 MB/s 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692499 s, 151 MB/s 2024-04-25T10:02:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.977 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006496 s, 161 MB/s 2024-04-25T10:02:49.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.985 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718884 s, 146 MB/s 2024-04-25T10:02:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071479 s, 147 MB/s 2024-04-25T10:02:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658244 s, 159 MB/s 2024-04-25T10:02:50.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.008 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713134 s, 147 MB/s 2024-04-25T10:02:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715098 s, 147 MB/s 2024-04-25T10:02:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.024 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634169 s, 165 MB/s 2024-04-25T10:02:50.032 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.032 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.032 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736923 s, 142 MB/s 2024-04-25T10:02:50.040 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.040 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.040 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696359 s, 151 MB/s 2024-04-25T10:02:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065471 s, 160 MB/s 2024-04-25T10:02:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.055 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.055 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657779 s, 159 MB/s 2024-04-25T10:02:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.062 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634438 s, 165 MB/s 2024-04-25T10:02:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704222 s, 149 MB/s 2024-04-25T10:02:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.078 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711338 s, 147 MB/s 2024-04-25T10:02:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710777 s, 148 MB/s 2024-04-25T10:02:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.094 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.094 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690036 s, 152 MB/s 2024-04-25T10:02:50.100 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.101 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.101 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625484 s, 168 MB/s 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643635 s, 163 MB/s 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720291 s, 146 MB/s 2024-04-25T10:02:50.124 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.124 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.124 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696931 s, 150 MB/s 2024-04-25T10:02:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711887 s, 147 MB/s 2024-04-25T10:02:50.139 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.139 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687254 s, 153 MB/s 2024-04-25T10:02:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.147 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063749 s, 164 MB/s 2024-04-25T10:02:50.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.154 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.154 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649708 s, 161 MB/s 2024-04-25T10:02:50.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.161 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638983 s, 164 MB/s 2024-04-25T10:02:50.168 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644406 s, 163 MB/s 2024-04-25T10:02:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693412 s, 151 MB/s 2024-04-25T10:02:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713838 s, 147 MB/s 2024-04-25T10:02:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658555 s, 159 MB/s 2024-04-25T10:02:50.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.193 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:50.193 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:50.193 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:02:50.193 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:50.194 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:50.194 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:50.194 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:50.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.260 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758923 s, 138 MB/s 2024-04-25T10:02:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645967 s, 162 MB/s 2024-04-25T10:02:50.274 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.275 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679324 s, 154 MB/s 2024-04-25T10:02:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062938 s, 167 MB/s 2024-04-25T10:02:50.289 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.290 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.290 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678305 s, 155 MB/s 2024-04-25T10:02:50.297 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.297 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.297 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695716 s, 151 MB/s 2024-04-25T10:02:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068405 s, 153 MB/s 2024-04-25T10:02:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705087 s, 149 MB/s 2024-04-25T10:02:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697731 s, 150 MB/s 2024-04-25T10:02:50.328 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.329 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.329 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654646 s, 160 MB/s 2024-04-25T10:02:50.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.336 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.336 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674267 s, 156 MB/s 2024-04-25T10:02:50.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.344 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.344 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704781 s, 149 MB/s 2024-04-25T10:02:50.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.351 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065146 s, 161 MB/s 2024-04-25T10:02:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819758 s, 128 MB/s 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705048 s, 149 MB/s 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691816 s, 152 MB/s 2024-04-25T10:02:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694861 s, 151 MB/s 2024-04-25T10:02:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705833 s, 149 MB/s 2024-04-25T10:02:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.399 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634247 s, 165 MB/s 2024-04-25T10:02:50.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.407 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.407 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675004 s, 155 MB/s 2024-04-25T10:02:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700644 s, 150 MB/s 2024-04-25T10:02:50.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.423 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.423 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735455 s, 143 MB/s 2024-04-25T10:02:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.431 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691437 s, 152 MB/s 2024-04-25T10:02:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739753 s, 142 MB/s 2024-04-25T10:02:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684617 s, 153 MB/s 2024-04-25T10:02:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688457 s, 152 MB/s 2024-04-25T10:02:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.462 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632577 s, 166 MB/s 2024-04-25T10:02:50.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.469 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.469 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062391 s, 168 MB/s 2024-04-25T10:02:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685408 s, 153 MB/s 2024-04-25T10:02:50.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.484 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.484 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661435 s, 159 MB/s 2024-04-25T10:02:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683628 s, 153 MB/s 2024-04-25T10:02:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745596 s, 141 MB/s 2024-04-25T10:02:50.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.501 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:50.502 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:50.502 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:02:50.502 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:50.502 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:50.502 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:50.502 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.567 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663054 s, 158 MB/s 2024-04-25T10:02:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629179 s, 167 MB/s 2024-04-25T10:02:50.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.581 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.581 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674345 s, 155 MB/s 2024-04-25T10:02:50.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.588 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.588 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606859 s, 173 MB/s 2024-04-25T10:02:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062239 s, 168 MB/s 2024-04-25T10:02:50.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.603 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.603 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069669 s, 151 MB/s 2024-04-25T10:02:50.611 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.611 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.611 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736479 s, 142 MB/s 2024-04-25T10:02:50.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.619 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.619 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710985 s, 147 MB/s 2024-04-25T10:02:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689648 s, 152 MB/s 2024-04-25T10:02:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702992 s, 149 MB/s 2024-04-25T10:02:50.642 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.643 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.643 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667711 s, 157 MB/s 2024-04-25T10:02:50.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.651 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.651 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751416 s, 140 MB/s 2024-04-25T10:02:50.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.659 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704292 s, 149 MB/s 2024-04-25T10:02:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072025 s, 146 MB/s 2024-04-25T10:02:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703659 s, 149 MB/s 2024-04-25T10:02:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735089 s, 143 MB/s 2024-04-25T10:02:50.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.692 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.692 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724036 s, 145 MB/s 2024-04-25T10:02:50.698 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601191 s, 174 MB/s 2024-04-25T10:02:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690216 s, 152 MB/s 2024-04-25T10:02:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643198 s, 163 MB/s 2024-04-25T10:02:50.722 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.722 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.722 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737884 s, 142 MB/s 2024-04-25T10:02:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647069 s, 162 MB/s 2024-04-25T10:02:50.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.737 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.737 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650057 s, 161 MB/s 2024-04-25T10:02:50.745 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.745 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.745 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686698 s, 153 MB/s 2024-04-25T10:02:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664539 s, 158 MB/s 2024-04-25T10:02:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691569 s, 152 MB/s 2024-04-25T10:02:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071301 s, 147 MB/s 2024-04-25T10:02:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735236 s, 143 MB/s 2024-04-25T10:02:50.783 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650171 s, 161 MB/s 2024-04-25T10:02:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.792 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.792 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709334 s, 148 MB/s 2024-04-25T10:02:50.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.800 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.800 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711311 s, 147 MB/s 2024-04-25T10:02:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704804 s, 149 MB/s 2024-04-25T10:02:50.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.809 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:50.809 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:50.809 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:02:50.809 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:50.809 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:50.809 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:50.809 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.876 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679787 s, 154 MB/s 2024-04-25T10:02:50.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.883 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.883 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656898 s, 160 MB/s 2024-04-25T10:02:50.890 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.891 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.891 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657139 s, 160 MB/s 2024-04-25T10:02:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.898 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653715 s, 160 MB/s 2024-04-25T10:02:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716481 s, 146 MB/s 2024-04-25T10:02:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676439 s, 155 MB/s 2024-04-25T10:02:50.921 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.921 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.921 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658427 s, 159 MB/s 2024-04-25T10:02:50.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.930 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.930 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718451 s, 146 MB/s 2024-04-25T10:02:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628477 s, 167 MB/s 2024-04-25T10:02:50.945 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.945 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.945 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745712 s, 141 MB/s 2024-04-25T10:02:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675315 s, 155 MB/s 2024-04-25T10:02:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.960 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.960 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632562 s, 166 MB/s 2024-04-25T10:02:50.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.969 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.969 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846948 s, 124 MB/s 2024-04-25T10:02:50.977 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.977 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.977 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712811 s, 147 MB/s 2024-04-25T10:02:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625163 s, 168 MB/s 2024-04-25T10:02:50.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:50.992 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:50.992 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716905 s, 146 MB/s 2024-04-25T10:02:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735709 s, 143 MB/s 2024-04-25T10:02:51.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.008 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705201 s, 149 MB/s 2024-04-25T10:02:51.015 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.015 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.015 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616696 s, 170 MB/s 2024-04-25T10:02:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627334 s, 167 MB/s 2024-04-25T10:02:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.030 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625256 s, 168 MB/s 2024-04-25T10:02:51.037 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.037 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.037 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068839 s, 152 MB/s 2024-04-25T10:02:51.045 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.046 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.046 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743137 s, 141 MB/s 2024-04-25T10:02:51.054 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.054 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.054 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716738 s, 146 MB/s 2024-04-25T10:02:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653062 s, 161 MB/s 2024-04-25T10:02:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653487 s, 160 MB/s 2024-04-25T10:02:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061262 s, 171 MB/s 2024-04-25T10:02:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716195 s, 146 MB/s 2024-04-25T10:02:51.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.091 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.091 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634243 s, 165 MB/s 2024-04-25T10:02:51.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.098 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.098 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635168 s, 165 MB/s 2024-04-25T10:02:51.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.105 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.106 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655677 s, 160 MB/s 2024-04-25T10:02:51.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.114 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710416 s, 148 MB/s 2024-04-25T10:02:51.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.115 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:51.115 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:51.115 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:02:51.115 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:51.115 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:51.115 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:51.115 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:51.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.180 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.180 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727852 s, 144 MB/s 2024-04-25T10:02:51.188 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.188 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.189 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728875 s, 144 MB/s 2024-04-25T10:02:51.196 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.196 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.196 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646678 s, 162 MB/s 2024-04-25T10:02:51.203 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.203 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.203 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683532 s, 153 MB/s 2024-04-25T10:02:51.211 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.211 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.211 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712655 s, 147 MB/s 2024-04-25T10:02:51.218 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.218 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.218 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609649 s, 172 MB/s 2024-04-25T10:02:51.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.227 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.227 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767515 s, 137 MB/s 2024-04-25T10:02:51.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.234 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634445 s, 165 MB/s 2024-04-25T10:02:51.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.241 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063529 s, 165 MB/s 2024-04-25T10:02:51.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.250 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.250 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811137 s, 129 MB/s 2024-04-25T10:02:51.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.258 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.258 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735689 s, 143 MB/s 2024-04-25T10:02:51.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.266 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.266 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064488 s, 163 MB/s 2024-04-25T10:02:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804506 s, 130 MB/s 2024-04-25T10:02:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663152 s, 158 MB/s 2024-04-25T10:02:51.290 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.290 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.290 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721334 s, 145 MB/s 2024-04-25T10:02:51.298 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.298 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.298 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649367 s, 161 MB/s 2024-04-25T10:02:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745624 s, 141 MB/s 2024-04-25T10:02:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689798 s, 152 MB/s 2024-04-25T10:02:51.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.323 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.323 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769454 s, 136 MB/s 2024-04-25T10:02:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717713 s, 146 MB/s 2024-04-25T10:02:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648224 s, 162 MB/s 2024-04-25T10:02:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685339 s, 153 MB/s 2024-04-25T10:02:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705163 s, 149 MB/s 2024-04-25T10:02:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746835 s, 140 MB/s 2024-04-25T10:02:51.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.370 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.370 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661854 s, 158 MB/s 2024-04-25T10:02:51.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.379 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745671 s, 141 MB/s 2024-04-25T10:02:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714415 s, 147 MB/s 2024-04-25T10:02:51.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.396 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.396 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817681 s, 128 MB/s 2024-04-25T10:02:51.404 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.404 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.405 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733659 s, 143 MB/s 2024-04-25T10:02:51.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.413 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.413 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752412 s, 139 MB/s 2024-04-25T10:02:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643396 s, 163 MB/s 2024-04-25T10:02:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748388 s, 140 MB/s 2024-04-25T10:02:51.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.430 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:51.430 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:51.430 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:02:51.430 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:51.430 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:51.430 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:51.430 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:51.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.497 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082078 s, 128 MB/s 2024-04-25T10:02:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.504 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665338 s, 158 MB/s 2024-04-25T10:02:51.511 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.511 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.512 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630062 s, 166 MB/s 2024-04-25T10:02:51.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.520 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.520 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717447 s, 146 MB/s 2024-04-25T10:02:51.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.527 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.528 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697037 s, 150 MB/s 2024-04-25T10:02:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.536 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.536 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704301 s, 149 MB/s 2024-04-25T10:02:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068212 s, 154 MB/s 2024-04-25T10:02:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685304 s, 153 MB/s 2024-04-25T10:02:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747801 s, 140 MB/s 2024-04-25T10:02:51.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.568 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.568 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729618 s, 144 MB/s 2024-04-25T10:02:51.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.575 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.575 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676094 s, 155 MB/s 2024-04-25T10:02:51.584 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.584 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.584 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007805 s, 134 MB/s 2024-04-25T10:02:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713066 s, 147 MB/s 2024-04-25T10:02:51.599 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.600 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669863 s, 157 MB/s 2024-04-25T10:02:51.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.607 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061334 s, 171 MB/s 2024-04-25T10:02:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006198 s, 169 MB/s 2024-04-25T10:02:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063798 s, 164 MB/s 2024-04-25T10:02:51.628 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.628 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.628 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677491 s, 155 MB/s 2024-04-25T10:02:51.636 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.636 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.636 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676823 s, 155 MB/s 2024-04-25T10:02:51.644 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.644 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.644 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707287 s, 148 MB/s 2024-04-25T10:02:51.652 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.652 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.652 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690878 s, 152 MB/s 2024-04-25T10:02:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678137 s, 155 MB/s 2024-04-25T10:02:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.667 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669334 s, 157 MB/s 2024-04-25T10:02:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693657 s, 151 MB/s 2024-04-25T10:02:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638277 s, 164 MB/s 2024-04-25T10:02:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.691 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811635 s, 129 MB/s 2024-04-25T10:02:51.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.699 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.699 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692299 s, 151 MB/s 2024-04-25T10:02:51.707 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.707 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.707 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656039 s, 160 MB/s 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699211 s, 150 MB/s 2024-04-25T10:02:51.722 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.722 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.722 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681122 s, 154 MB/s 2024-04-25T10:02:51.730 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.730 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.731 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728975 s, 144 MB/s 2024-04-25T10:02:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727215 s, 144 MB/s 2024-04-25T10:02:51.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.740 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:51.740 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:51.740 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:02:51.740 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:51.740 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:51.740 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:51.740 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:51.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.805 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.805 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682974 s, 154 MB/s 2024-04-25T10:02:51.813 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.813 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.813 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072631 s, 144 MB/s 2024-04-25T10:02:51.821 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.821 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.821 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722013 s, 145 MB/s 2024-04-25T10:02:51.829 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.829 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.829 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702287 s, 149 MB/s 2024-04-25T10:02:51.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.838 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.839 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858273 s, 122 MB/s 2024-04-25T10:02:51.846 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.846 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.847 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693223 s, 151 MB/s 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689474 s, 152 MB/s 2024-04-25T10:02:51.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.861 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630602 s, 166 MB/s 2024-04-25T10:02:51.870 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.870 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.870 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735179 s, 143 MB/s 2024-04-25T10:02:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070633 s, 148 MB/s 2024-04-25T10:02:51.886 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.886 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.886 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699135 s, 150 MB/s 2024-04-25T10:02:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.893 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661794 s, 158 MB/s 2024-04-25T10:02:51.900 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.901 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.901 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654964 s, 160 MB/s 2024-04-25T10:02:51.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.908 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.908 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006474 s, 162 MB/s 2024-04-25T10:02:51.915 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669278 s, 157 MB/s 2024-04-25T10:02:51.923 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.924 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.924 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707706 s, 148 MB/s 2024-04-25T10:02:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683678 s, 153 MB/s 2024-04-25T10:02:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689971 s, 152 MB/s 2024-04-25T10:02:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653154 s, 161 MB/s 2024-04-25T10:02:51.954 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.955 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707515 s, 148 MB/s 2024-04-25T10:02:51.963 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.963 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.963 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734164 s, 143 MB/s 2024-04-25T10:02:51.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.970 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656585 s, 160 MB/s 2024-04-25T10:02:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.978 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682966 s, 154 MB/s 2024-04-25T10:02:51.986 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.986 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.986 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693596 s, 151 MB/s 2024-04-25T10:02:51.993 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:51.993 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:51.993 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651783 s, 161 MB/s 2024-04-25T10:02:52.001 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.001 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.001 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699075 s, 150 MB/s 2024-04-25T10:02:52.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.008 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.008 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682414 s, 154 MB/s 2024-04-25T10:02:52.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.016 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.016 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638367 s, 164 MB/s 2024-04-25T10:02:52.023 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.023 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.023 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063818 s, 164 MB/s 2024-04-25T10:02:52.030 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.030 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.030 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614055 s, 171 MB/s 2024-04-25T10:02:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738482 s, 142 MB/s 2024-04-25T10:02:52.045 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.045 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.046 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062662 s, 167 MB/s 2024-04-25T10:02:52.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:52.047 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:52.047 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:52.047 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:02:52.047 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:52.047 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:52.047 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:52.047 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.113 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.113 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723065 s, 145 MB/s 2024-04-25T10:02:52.120 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.121 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.121 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732331 s, 143 MB/s 2024-04-25T10:02:52.128 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.129 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.129 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715527 s, 147 MB/s 2024-04-25T10:02:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709412 s, 148 MB/s 2024-04-25T10:02:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715548 s, 147 MB/s 2024-04-25T10:02:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688588 s, 152 MB/s 2024-04-25T10:02:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713113 s, 147 MB/s 2024-04-25T10:02:52.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.169 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.169 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732334 s, 143 MB/s 2024-04-25T10:02:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719738 s, 146 MB/s 2024-04-25T10:02:52.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.184 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.184 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638325 s, 164 MB/s 2024-04-25T10:02:52.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.193 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.193 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826743 s, 127 MB/s 2024-04-25T10:02:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.201 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710596 s, 148 MB/s 2024-04-25T10:02:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704685 s, 149 MB/s 2024-04-25T10:02:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712967 s, 147 MB/s 2024-04-25T10:02:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653226 s, 161 MB/s 2024-04-25T10:02:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635349 s, 165 MB/s 2024-04-25T10:02:52.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.241 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.241 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859386 s, 122 MB/s 2024-04-25T10:02:52.248 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.248 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.249 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664638 s, 158 MB/s 2024-04-25T10:02:52.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.256 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.256 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644671 s, 163 MB/s 2024-04-25T10:02:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681648 s, 154 MB/s 2024-04-25T10:02:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668262 s, 157 MB/s 2024-04-25T10:02:52.278 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.278 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.278 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638022 s, 164 MB/s 2024-04-25T10:02:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.286 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701329 s, 150 MB/s 2024-04-25T10:02:52.294 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.294 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.294 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732714 s, 143 MB/s 2024-04-25T10:02:52.302 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.302 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.302 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677132 s, 155 MB/s 2024-04-25T10:02:52.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.310 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.310 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703772 s, 149 MB/s 2024-04-25T10:02:52.317 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.318 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.318 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691559 s, 152 MB/s 2024-04-25T10:02:52.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.327 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.327 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827867 s, 127 MB/s 2024-04-25T10:02:52.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.334 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.334 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665516 s, 158 MB/s 2024-04-25T10:02:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.344 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816834 s, 128 MB/s 2024-04-25T10:02:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701255 s, 150 MB/s 2024-04-25T10:02:52.359 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.360 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.360 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630206 s, 166 MB/s 2024-04-25T10:02:52.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:52.360 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:52.360 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:52.360 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:02:52.360 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:52.360 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:52.361 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:52.361 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:52.426 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.426 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.426 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703193 s, 149 MB/s 2024-04-25T10:02:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644268 s, 163 MB/s 2024-04-25T10:02:52.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.441 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.441 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692754 s, 151 MB/s 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720858 s, 145 MB/s 2024-04-25T10:02:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694628 s, 151 MB/s 2024-04-25T10:02:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743081 s, 141 MB/s 2024-04-25T10:02:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684281 s, 153 MB/s 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703553 s, 149 MB/s 2024-04-25T10:02:52.489 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.489 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.489 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720894 s, 145 MB/s 2024-04-25T10:02:52.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.497 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.497 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712563 s, 147 MB/s 2024-04-25T10:02:52.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.505 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.505 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699752 s, 150 MB/s 2024-04-25T10:02:52.513 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.514 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.514 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690406 s, 152 MB/s 2024-04-25T10:02:52.521 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.521 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.521 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699956 s, 150 MB/s 2024-04-25T10:02:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632889 s, 166 MB/s 2024-04-25T10:02:52.536 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.536 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.536 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692952 s, 151 MB/s 2024-04-25T10:02:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064767 s, 162 MB/s 2024-04-25T10:02:52.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.551 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.551 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718659 s, 146 MB/s 2024-04-25T10:02:52.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.562 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.562 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065672 s, 160 MB/s 2024-04-25T10:02:52.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.567 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.567 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709921 s, 148 MB/s 2024-04-25T10:02:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620032 s, 169 MB/s 2024-04-25T10:02:52.582 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.582 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.582 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677882 s, 155 MB/s 2024-04-25T10:02:52.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.589 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.590 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694415 s, 151 MB/s 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731306 s, 143 MB/s 2024-04-25T10:02:52.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.606 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.606 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710127 s, 148 MB/s 2024-04-25T10:02:52.613 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.613 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.613 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614853 s, 171 MB/s 2024-04-25T10:02:52.620 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.621 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.621 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676117 s, 155 MB/s 2024-04-25T10:02:52.628 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.628 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.628 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649652 s, 161 MB/s 2024-04-25T10:02:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641055 s, 164 MB/s 2024-04-25T10:02:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672131 s, 156 MB/s 2024-04-25T10:02:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651376 s, 161 MB/s 2024-04-25T10:02:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699019 s, 150 MB/s 2024-04-25T10:02:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696094 s, 151 MB/s 2024-04-25T10:02:52.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:52.667 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:52.667 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:52.667 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:02:52.668 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:52.668 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:52.668 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:52.668 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683045 s, 154 MB/s 2024-04-25T10:02:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731819 s, 143 MB/s 2024-04-25T10:02:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663922 s, 158 MB/s 2024-04-25T10:02:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.757 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702815 s, 149 MB/s 2024-04-25T10:02:52.765 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.765 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.765 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715187 s, 147 MB/s 2024-04-25T10:02:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677469 s, 155 MB/s 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652634 s, 161 MB/s 2024-04-25T10:02:52.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.787 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.787 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643957 s, 163 MB/s 2024-04-25T10:02:52.795 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.795 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.795 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685528 s, 153 MB/s 2024-04-25T10:02:52.803 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.803 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.803 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068095 s, 154 MB/s 2024-04-25T10:02:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707369 s, 148 MB/s 2024-04-25T10:02:52.819 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.819 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.819 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700598 s, 150 MB/s 2024-04-25T10:02:52.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.826 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.826 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635782 s, 165 MB/s 2024-04-25T10:02:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.834 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660644 s, 159 MB/s 2024-04-25T10:02:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682895 s, 154 MB/s 2024-04-25T10:02:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.849 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707208 s, 148 MB/s 2024-04-25T10:02:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.857 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676992 s, 155 MB/s 2024-04-25T10:02:52.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.865 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.865 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717531 s, 146 MB/s 2024-04-25T10:02:52.872 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.872 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.872 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655512 s, 160 MB/s 2024-04-25T10:02:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.880 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638226 s, 164 MB/s 2024-04-25T10:02:52.888 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.888 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.888 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734003 s, 143 MB/s 2024-04-25T10:02:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654708 s, 160 MB/s 2024-04-25T10:02:52.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.903 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007241 s, 145 MB/s 2024-04-25T10:02:52.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.910 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.910 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642206 s, 163 MB/s 2024-04-25T10:02:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644327 s, 163 MB/s 2024-04-25T10:02:52.925 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.926 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699509 s, 150 MB/s 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662798 s, 158 MB/s 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621852 s, 169 MB/s 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707062 s, 148 MB/s 2024-04-25T10:02:52.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.956 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.956 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692825 s, 151 MB/s 2024-04-25T10:02:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645727 s, 162 MB/s 2024-04-25T10:02:52.971 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:52.971 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:52.971 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065722 s, 160 MB/s 2024-04-25T10:02:52.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:52.972 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:52.972 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:02:52.972 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:02:52.972 DEBUG:teuthology.orchestra.run.smithi019:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:52.972 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:52.972 DEBUG:teuthology.orchestra.run.smithi019:> done 2024-04-25T10:02:52.973 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:02:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726204 s, 144 MB/s 2024-04-25T10:02:53.047 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.049 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.049 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824701 s, 127 MB/s 2024-04-25T10:02:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723911 s, 145 MB/s 2024-04-25T10:02:53.063 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.064 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.064 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703433 s, 149 MB/s 2024-04-25T10:02:53.072 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.072 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.072 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750324 s, 140 MB/s 2024-04-25T10:02:53.080 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.080 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.080 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672163 s, 156 MB/s 2024-04-25T10:02:53.087 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.087 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.087 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661153 s, 159 MB/s 2024-04-25T10:02:53.096 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.096 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.097 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801785 s, 131 MB/s 2024-04-25T10:02:53.104 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.104 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.104 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007272 s, 144 MB/s 2024-04-25T10:02:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690054 s, 152 MB/s 2024-04-25T10:02:53.120 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.120 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.120 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687598 s, 152 MB/s 2024-04-25T10:02:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656101 s, 160 MB/s 2024-04-25T10:02:53.135 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.135 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.135 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649236 s, 162 MB/s 2024-04-25T10:02:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695931 s, 151 MB/s 2024-04-25T10:02:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702962 s, 149 MB/s 2024-04-25T10:02:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718672 s, 146 MB/s 2024-04-25T10:02:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679143 s, 154 MB/s 2024-04-25T10:02:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689522 s, 152 MB/s 2024-04-25T10:02:53.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.182 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.182 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071445 s, 147 MB/s 2024-04-25T10:02:53.190 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.190 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.190 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068679 s, 153 MB/s 2024-04-25T10:02:53.197 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.197 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.197 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685282 s, 153 MB/s 2024-04-25T10:02:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.205 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699227 s, 150 MB/s 2024-04-25T10:02:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742565 s, 141 MB/s 2024-04-25T10:02:53.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.221 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.221 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640847 s, 164 MB/s 2024-04-25T10:02:53.228 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.229 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.229 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066796 s, 157 MB/s 2024-04-25T10:02:53.236 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.237 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.237 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701336 s, 150 MB/s 2024-04-25T10:02:53.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.244 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.244 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656003 s, 160 MB/s 2024-04-25T10:02:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.251 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628989 s, 167 MB/s 2024-04-25T10:02:53.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.259 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.259 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659805 s, 159 MB/s 2024-04-25T10:02:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711542 s, 147 MB/s 2024-04-25T10:02:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705608 s, 149 MB/s 2024-04-25T10:02:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T10:02:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T10:02:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633766 s, 165 MB/s 2024-04-25T10:02:53.283 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T10:02:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.422+0000 7f2c026c4640 1 Processor -- start 2024-04-25T10:02:53.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.422+0000 7f2c026c4640 1 -- start start 2024-04-25T10:02:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2c026c4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2bfc15b7d0 0x7f2bfc0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2c026c4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2bfc0b5620 0x7f2bfc0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2c026c4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bfc06cf50 con 0x7f2bfc06c2d0 2024-04-25T10:02:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2c026c4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc0ce530 con 0x7f2bfc15b7d0 2024-04-25T10:02:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2c026c4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc0ce810 con 0x7f2bfc0b5620 2024-04-25T10:02:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2bfbfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2bfc15b7d0 0x7f2bfc0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2bfbfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2bfc15b7d0 0x7f2bfc0cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43558/0 (socket says 172.21.15.19:43558) 2024-04-25T10:02:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2bfbfff640 1 -- 172.21.15.19:0/2419618362 learned_addr learned my addr 172.21.15.19:0/2419618362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2bfb7fe640 1 --2- 172.21.15.19:0/2419618362 >> v2:172.21.15.112:3300/0 conn(0x7f2bfc0b5620 0x7f2bfc0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2bfbfff640 1 -- 172.21.15.19:0/2419618362 >> v2:172.21.15.112:3300/0 conn(0x7f2bfc0b5620 msgr2=0x7f2bfc0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2bfbfff640 1 --2- 172.21.15.19:0/2419618362 >> v2:172.21.15.112:3300/0 conn(0x7f2bfc0b5620 0x7f2bfc0b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2bfbfff640 1 -- 172.21.15.19:0/2419618362 >> v1:172.21.15.112:6789/0 conn(0x7f2bfc06c2d0 legacy=0x7f2bfc06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.423+0000 7f2bfbfff640 1 -- 172.21.15.19:0/2419618362 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0ceaf0 con 0x7f2bfc15b7d0 2024-04-25T10:02:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.424+0000 7f2bfbfff640 1 --2- 172.21.15.19:0/2419618362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc15b7d0 0x7f2bfc0cf010 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0ce180 tx=0x7f2bec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276ed4909cd00f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.424+0000 7f2bfaffd640 1 -- 172.21.15.19:0/2419618362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec037070 con 0x7f2bfc15b7d0 2024-04-25T10:02:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.424+0000 7f2bfaffd640 1 -- 172.21.15.19:0/2419618362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec032440 con 0x7f2bfc15b7d0 2024-04-25T10:02:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.424+0000 7f2bfaffd640 1 -- 172.21.15.19:0/2419618362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec032810 con 0x7f2bfc15b7d0 2024-04-25T10:02:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.425+0000 7f2c026c4640 1 -- 172.21.15.19:0/2419618362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc15b7d0 msgr2=0x7f2bfc0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.425+0000 7f2c026c4640 1 --2- 172.21.15.19:0/2419618362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc15b7d0 0x7f2bfc0cf010 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0ce180 tx=0x7f2bec02a270 comp rx=0 tx=0).stop 2024-04-25T10:02:53.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.425+0000 7f2c026c4640 1 -- 172.21.15.19:0/2419618362 shutdown_connections 2024-04-25T10:02:53.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.425+0000 7f2c026c4640 1 --2- 172.21.15.19:0/2419618362 >> v2:172.21.15.112:3300/0 conn(0x7f2bfc0b5620 0x7f2bfc0b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.425+0000 7f2c026c4640 1 --2- 172.21.15.19:0/2419618362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc15b7d0 0x7f2bfc0cf010 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.425+0000 7f2c026c4640 1 -- 172.21.15.19:0/2419618362 >> 172.21.15.19:0/2419618362 conn(0x7f2bfc0d0800 msgr2=0x7f2bfc0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.425+0000 7f2c026c4640 1 -- 172.21.15.19:0/2419618362 shutdown_connections 2024-04-25T10:02:53.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.425+0000 7f2c026c4640 1 -- 172.21.15.19:0/2419618362 wait complete. 2024-04-25T10:02:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.426+0000 7f2c026c4640 1 Processor -- start 2024-04-25T10:02:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.426+0000 7f2c026c4640 1 -- start start 2024-04-25T10:02:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2c026c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2bfc06c2d0 0x7f2bfc0aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2c026c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc0b5620 0x7f2bfc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2c026c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bfc15b7d0 0x7f2bfc0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2c026c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0cf6d0 con 0x7f2bfc0b5620 2024-04-25T10:02:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2c026c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0cf550 con 0x7f2bfc15b7d0 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2c026c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc15bbb0 con 0x7f2bfc06c2d0 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2bfb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc0b5620 0x7f2bfc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2bfb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc0b5620 0x7f2bfc0cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43562/0 (socket says 172.21.15.19:43562) 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2bfb7fe640 1 -- 172.21.15.19:0/3594777482 learned_addr learned my addr 172.21.15.19:0/3594777482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2bfbfff640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2bfc06c2d0 0x7f2bfc0aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2c00c61640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bfc15b7d0 0x7f2bfc0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2bfb7fe640 1 -- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2bfc06c2d0 msgr2=0x7f2bfc0aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2bfb7fe640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2bfc06c2d0 0x7f2bfc0aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2bfb7fe640 1 -- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bfc15b7d0 msgr2=0x7f2bfc0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2bfb7fe640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bfc15b7d0 0x7f2bfc0cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2bfb7fe640 1 -- 172.21.15.19:0/3594777482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0c8720 con 0x7f2bfc0b5620 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.427+0000 7f2c00c61640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bfc15b7d0 0x7f2bfc0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.428+0000 7f2bfb7fe640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc0b5620 0x7f2bfc0cc3c0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f2be400d800 tx=0x7f2be400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.428+0000 7f2bf8ff9640 1 -- 172.21.15.19:0/3594777482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be400b6c0 con 0x7f2bfc0b5620 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.428+0000 7f2bf8ff9640 1 -- 172.21.15.19:0/3594777482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be4019070 con 0x7f2bfc0b5620 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.428+0000 7f2c026c4640 1 -- 172.21.15.19:0/3594777482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bfc0ccfa0 con 0x7f2bfc0b5620 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.428+0000 7f2bf8ff9640 1 -- 172.21.15.19:0/3594777482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be400ba80 con 0x7f2bfc0b5620 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.428+0000 7f2c026c4640 1 -- 172.21.15.19:0/3594777482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bfc0c8cc0 con 0x7f2bfc0b5620 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.429+0000 7f2c026c4640 1 -- 172.21.15.19:0/3594777482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bfc073a00 con 0x7f2bfc0b5620 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.429+0000 7f2bf8ff9640 1 -- 172.21.15.19:0/3594777482 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2be400bc20 con 0x7f2bfc0b5620 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.430+0000 7f2bf8ff9640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bdc06b4d0 0x7f2bdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.430+0000 7f2bf8ff9640 1 -- 172.21.15.19:0/3594777482 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2be408a740 con 0x7f2bfc0b5620 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.430+0000 7f2bfbfff640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bdc06b4d0 0x7f2bdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.431+0000 7f2bfbfff640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bdc06b4d0 0x7f2bdc06d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2bec034000 tx=0x7f2bec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.434+0000 7f2bf8ff9640 1 -- 172.21.15.19:0/3594777482 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2be4059490 con 0x7f2bfc0b5620 2024-04-25T10:02:53.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.650+0000 7f2c026c4640 1 -- 172.21.15.19:0/3594777482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f2bfc160250 con 0x7f2bfc0b5620 2024-04-25T10:02:53.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.714+0000 7f2bf8ff9640 1 -- 172.21.15.19:0/3594777482 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v92) v1 ==== 105+0+0 (secure 0 0 0) 0x7f2be4058d90 con 0x7f2bfc0b5620 2024-04-25T10:02:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.715+0000 7f2c026c4640 1 -- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bdc06b4d0 msgr2=0x7f2bdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.716+0000 7f2c026c4640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bdc06b4d0 0x7f2bdc06d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2bec034000 tx=0x7f2bec02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.716+0000 7f2c026c4640 1 -- 172.21.15.19:0/3594777482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc0b5620 msgr2=0x7f2bfc0cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.716+0000 7f2c026c4640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc0b5620 0x7f2bfc0cc3c0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f2be400d800 tx=0x7f2be400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.716+0000 7f2c026c4640 1 -- 172.21.15.19:0/3594777482 shutdown_connections 2024-04-25T10:02:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.716+0000 7f2c026c4640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2bdc06b4d0 0x7f2bdc06d990 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.716+0000 7f2c026c4640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bfc15b7d0 0x7f2bfc0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.716+0000 7f2c026c4640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bfc0b5620 0x7f2bfc0cc3c0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.716+0000 7f2c026c4640 1 --2- 172.21.15.19:0/3594777482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2bfc06c2d0 0x7f2bfc0aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.716+0000 7f2c026c4640 1 -- 172.21.15.19:0/3594777482 >> 172.21.15.19:0/3594777482 conn(0x7f2bfc0d0800 msgr2=0x7f2bfc15ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.716+0000 7f2c026c4640 1 -- 172.21.15.19:0/3594777482 shutdown_connections 2024-04-25T10:02:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.716+0000 7f2c026c4640 1 -- 172.21.15.19:0/3594777482 wait complete. 2024-04-25T10:02:53.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T10:02:53.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.830+0000 7f90e0015640 1 Processor -- start 2024-04-25T10:02:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.830+0000 7f90e0015640 1 -- start start 2024-04-25T10:02:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90e0015640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f90d815dc20 0x7f90d80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90e0015640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90d80d32f0 0x7f90d80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90e0015640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d815e000 con 0x7f90d8161710 2024-04-25T10:02:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90e0015640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d80d1de0 con 0x7f90d815dc20 2024-04-25T10:02:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90e0015640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8161c60 con 0x7f90d80d32f0 2024-04-25T10:02:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90dddb1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f90d815dc20 0x7f90d80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90dddb1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f90d815dc20 0x7f90d80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43570/0 (socket says 172.21.15.19:43570) 2024-04-25T10:02:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90dddb1640 1 -- 172.21.15.19:0/3339330066 learned_addr learned my addr 172.21.15.19:0/3339330066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90de5b2640 1 --2- 172.21.15.19:0/3339330066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90d80d32f0 0x7f90d80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90dddb1640 1 -- 172.21.15.19:0/3339330066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90d80d32f0 msgr2=0x7f90d80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90dddb1640 1 --2- 172.21.15.19:0/3339330066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90d80d32f0 0x7f90d80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90dddb1640 1 -- 172.21.15.19:0/3339330066 >> v1:172.21.15.112:6789/0 conn(0x7f90d8161710 legacy=0x7f90d806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90dddb1640 1 -- 172.21.15.19:0/3339330066 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d8161f40 con 0x7f90d815dc20 2024-04-25T10:02:53.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.831+0000 7f90dcdaf640 1 -- 172.21.15.19:0/3339330066 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045479165 0 0) 0x7f90d815e000 con 0x7f90d8161710 2024-04-25T10:02:53.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.832+0000 7f90dddb1640 1 --2- 172.21.15.19:0/3339330066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90d815dc20 0x7f90d80ce670 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009870 tx=0x7f90d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c758d2f193a1eb47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.832+0000 7f90dcdaf640 1 -- 172.21.15.19:0/3339330066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4037070 con 0x7f90d815dc20 2024-04-25T10:02:53.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.832+0000 7f90dcdaf640 1 -- 172.21.15.19:0/3339330066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d4032440 con 0x7f90d815dc20 2024-04-25T10:02:53.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.832+0000 7f90dcdaf640 1 -- 172.21.15.19:0/3339330066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4032810 con 0x7f90d815dc20 2024-04-25T10:02:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.833+0000 7f90e0015640 1 -- 172.21.15.19:0/3339330066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90d815dc20 msgr2=0x7f90d80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.833+0000 7f90e0015640 1 --2- 172.21.15.19:0/3339330066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90d815dc20 0x7f90d80ce670 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009870 tx=0x7f90d402a270 comp rx=0 tx=0).stop 2024-04-25T10:02:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.833+0000 7f90e0015640 1 -- 172.21.15.19:0/3339330066 shutdown_connections 2024-04-25T10:02:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.833+0000 7f90e0015640 1 --2- 172.21.15.19:0/3339330066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90d80d32f0 0x7f90d80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.833+0000 7f90e0015640 1 --2- 172.21.15.19:0/3339330066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90d815dc20 0x7f90d80ce670 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.833+0000 7f90e0015640 1 -- 172.21.15.19:0/3339330066 >> 172.21.15.19:0/3339330066 conn(0x7f90d80d0420 msgr2=0x7f90d80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.833+0000 7f90e0015640 1 -- 172.21.15.19:0/3339330066 shutdown_connections 2024-04-25T10:02:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.833+0000 7f90e0015640 1 -- 172.21.15.19:0/3339330066 wait complete. 2024-04-25T10:02:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.834+0000 7f90e0015640 1 Processor -- start 2024-04-25T10:02:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.834+0000 7f90e0015640 1 -- start start 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90e0015640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90d80d32f0 0x7f90d80aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90e0015640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90d815dc20 0x7f90d80b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90e0015640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90d8161710 0x7f90d80acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90e0015640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80b5e50 con 0x7f90d815dc20 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90e0015640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8162500 con 0x7f90d8161710 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90e0015640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80b5fd0 con 0x7f90d80d32f0 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90dd5b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90d815dc20 0x7f90d80b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90dd5b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90d815dc20 0x7f90d80b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43572/0 (socket says 172.21.15.19:43572) 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90dd5b0640 1 -- 172.21.15.19:0/4132361827 learned_addr learned my addr 172.21.15.19:0/4132361827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90dddb1640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90d80d32f0 0x7f90d80aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90de5b2640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90d8161710 0x7f90d80acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90dd5b0640 1 -- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90d80d32f0 msgr2=0x7f90d80aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90dd5b0640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90d80d32f0 0x7f90d80aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90dd5b0640 1 -- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90d8161710 msgr2=0x7f90d80acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90dd5b0640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90d8161710 0x7f90d80acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.835+0000 7f90dd5b0640 1 -- 172.21.15.19:0/4132361827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d814ce00 con 0x7f90d815dc20 2024-04-25T10:02:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.836+0000 7f90dd5b0640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90d815dc20 0x7f90d80b2460 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f90c800c440 tx=0x7f90c800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.836+0000 7f90ce7fc640 1 -- 172.21.15.19:0/4132361827 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8017070 con 0x7f90d815dc20 2024-04-25T10:02:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.836+0000 7f90ce7fc640 1 -- 172.21.15.19:0/4132361827 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c8005480 con 0x7f90d815dc20 2024-04-25T10:02:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.836+0000 7f90e0015640 1 -- 172.21.15.19:0/4132361827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d814d0e0 con 0x7f90d815dc20 2024-04-25T10:02:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.836+0000 7f90ce7fc640 1 -- 172.21.15.19:0/4132361827 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8003830 con 0x7f90d815dc20 2024-04-25T10:02:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.836+0000 7f90e0015640 1 -- 172.21.15.19:0/4132361827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d814d6a0 con 0x7f90d815dc20 2024-04-25T10:02:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.837+0000 7f90e0015640 1 -- 172.21.15.19:0/4132361827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d8073a00 con 0x7f90d815dc20 2024-04-25T10:02:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.837+0000 7f90ce7fc640 1 -- 172.21.15.19:0/4132361827 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90c801a070 con 0x7f90d815dc20 2024-04-25T10:02:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.837+0000 7f90ce7fc640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90bc06b500 0x7f90bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.838+0000 7f90ce7fc640 1 -- 172.21.15.19:0/4132361827 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f90c80895b0 con 0x7f90d815dc20 2024-04-25T10:02:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.838+0000 7f90dddb1640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90bc06b500 0x7f90bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.838+0000 7f90dddb1640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90bc06b500 0x7f90bc06d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f90d4034000 tx=0x7f90d402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:53.842+0000 7f90ce7fc640 1 -- 172.21.15.19:0/4132361827 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90c8057c20 con 0x7f90d815dc20 2024-04-25T10:02:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.062+0000 7f90e0015640 1 -- 172.21.15.19:0/4132361827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f90d80d16b0 con 0x7f90d815dc20 2024-04-25T10:02:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.719+0000 7f90ce7fc640 1 -- 172.21.15.19:0/4132361827 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v93) v1 ==== 121+0+0 (secure 0 0 0) 0x7f90c805bcc0 con 0x7f90d815dc20 2024-04-25T10:02:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 -- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90bc06b500 msgr2=0x7f90bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90bc06b500 0x7f90bc06d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f90d4034000 tx=0x7f90d402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 -- 172.21.15.19:0/4132361827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90d815dc20 msgr2=0x7f90d80b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90d815dc20 0x7f90d80b2460 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f90c800c440 tx=0x7f90c800c900 comp rx=0 tx=0).stop 2024-04-25T10:02:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 -- 172.21.15.19:0/4132361827 shutdown_connections 2024-04-25T10:02:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f90bc06b500 0x7f90bc06d9c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90d8161710 0x7f90d80acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90d815dc20 0x7f90d80b2460 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 --2- 172.21.15.19:0/4132361827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90d80d32f0 0x7f90d80aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 -- 172.21.15.19:0/4132361827 >> 172.21.15.19:0/4132361827 conn(0x7f90d80d0420 msgr2=0x7f90d80b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 -- 172.21.15.19:0/4132361827 shutdown_connections 2024-04-25T10:02:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.722+0000 7f90e0015640 1 -- 172.21.15.19:0/4132361827 wait complete. 2024-04-25T10:02:54.742 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:54.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.845+0000 7f613257e640 1 Processor -- start 2024-04-25T10:02:54.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.845+0000 7f613257e640 1 -- start start 2024-04-25T10:02:54.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.845+0000 7f613257e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f612c15dbe0 0x7f612c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.845+0000 7f613257e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c0b2cb0 0x7f612c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.845+0000 7f613257e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f612c0b92a0 0x7f612c0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.845+0000 7f613257e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612c15dfc0 con 0x7f612c15dbe0 2024-04-25T10:02:54.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.845+0000 7f613257e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612c0d1a00 con 0x7f612c0b92a0 2024-04-25T10:02:54.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.845+0000 7f613257e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0b97a0 con 0x7f612c0b2cb0 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.846+0000 7f612bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f612c15dbe0 0x7f612c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.846+0000 7f612bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f612c15dbe0 0x7f612c0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43578/0 (socket says 172.21.15.19:43578) 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.846+0000 7f612bfff640 1 -- 172.21.15.19:0/1264517743 learned_addr learned my addr 172.21.15.19:0/1264517743 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.846+0000 7f6130b1b640 1 --2- 172.21.15.19:0/1264517743 >> v2:172.21.15.112:3300/0 conn(0x7f612c0b92a0 0x7f612c0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.846+0000 7f612b7fe640 1 --2- 172.21.15.19:0/1264517743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c0b2cb0 0x7f612c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.846+0000 7f612bfff640 1 -- 172.21.15.19:0/1264517743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c0b2cb0 msgr2=0x7f612c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.846+0000 7f612bfff640 1 --2- 172.21.15.19:0/1264517743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c0b2cb0 0x7f612c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.846+0000 7f612bfff640 1 -- 172.21.15.19:0/1264517743 >> v2:172.21.15.112:3300/0 conn(0x7f612c0b92a0 msgr2=0x7f612c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.846+0000 7f612bfff640 1 --2- 172.21.15.19:0/1264517743 >> v2:172.21.15.112:3300/0 conn(0x7f612c0b92a0 0x7f612c0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.846+0000 7f612bfff640 1 -- 172.21.15.19:0/1264517743 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c0d1c50 con 0x7f612c15dbe0 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.847+0000 7f612bfff640 1 --2- 172.21.15.19:0/1264517743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f612c15dbe0 0x7f612c0d0910 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f612c06c640 tx=0x7f611c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def48f1217eb4809 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.847+0000 7f612affd640 1 -- 172.21.15.19:0/1264517743 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c037070 con 0x7f612c15dbe0 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.847+0000 7f612affd640 1 -- 172.21.15.19:0/1264517743 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c032440 con 0x7f612c15dbe0 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.847+0000 7f612affd640 1 -- 172.21.15.19:0/1264517743 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c032810 con 0x7f612c15dbe0 2024-04-25T10:02:54.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.847+0000 7f613257e640 1 -- 172.21.15.19:0/1264517743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f612c15dbe0 msgr2=0x7f612c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.847+0000 7f613257e640 1 --2- 172.21.15.19:0/1264517743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f612c15dbe0 0x7f612c0d0910 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f612c06c640 tx=0x7f611c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.847+0000 7f613257e640 1 -- 172.21.15.19:0/1264517743 shutdown_connections 2024-04-25T10:02:54.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.847+0000 7f613257e640 1 --2- 172.21.15.19:0/1264517743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c0b2cb0 0x7f612c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.847+0000 7f613257e640 1 --2- 172.21.15.19:0/1264517743 >> v2:172.21.15.112:3300/0 conn(0x7f612c0b92a0 0x7f612c0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.847+0000 7f613257e640 1 --2- 172.21.15.19:0/1264517743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f612c15dbe0 0x7f612c0d0910 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.848+0000 7f613257e640 1 -- 172.21.15.19:0/1264517743 >> 172.21.15.19:0/1264517743 conn(0x7f612c061a20 msgr2=0x7f612c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.848+0000 7f613257e640 1 -- 172.21.15.19:0/1264517743 shutdown_connections 2024-04-25T10:02:54.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.848+0000 7f613257e640 1 -- 172.21.15.19:0/1264517743 wait complete. 2024-04-25T10:02:54.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.849+0000 7f613257e640 1 Processor -- start 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.849+0000 7f613257e640 1 -- start start 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f613257e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f612c0b2cb0 0x7f612c0b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f613257e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c0b92a0 0x7f612c14bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f613257e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c15dbe0 0x7f612c15ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f613257e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0b6ab0 con 0x7f612c0b2cb0 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f613257e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0d2210 con 0x7f612c0b92a0 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f613257e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0b6c30 con 0x7f612c15dbe0 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f612bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f612c0b2cb0 0x7f612c0b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f612bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f612c0b2cb0 0x7f612c0b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43586/0 (socket says 172.21.15.19:43586) 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f612bfff640 1 -- 172.21.15.19:0/4005585075 learned_addr learned my addr 172.21.15.19:0/4005585075 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f612b7fe640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c0b92a0 0x7f612c14bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f6130b1b640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c15dbe0 0x7f612c15ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f612bfff640 1 -- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c15dbe0 msgr2=0x7f612c15ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f612bfff640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c15dbe0 0x7f612c15ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f612bfff640 1 -- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c0b92a0 msgr2=0x7f612c14bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f612bfff640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c0b92a0 0x7f612c14bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f612bfff640 1 -- 172.21.15.19:0/4005585075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c0b39c0 con 0x7f612c0b2cb0 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f612b7fe640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c0b92a0 0x7f612c14bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f612bfff640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f612c0b2cb0 0x7f612c0b32e0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f611c02a220 tx=0x7f611c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f6128ff9640 1 -- 172.21.15.19:0/4005585075 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c041020 con 0x7f612c0b2cb0 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f6128ff9640 1 -- 172.21.15.19:0/4005585075 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c0306c0 con 0x7f612c0b2cb0 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f6128ff9640 1 -- 172.21.15.19:0/4005585075 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c037040 con 0x7f612c0b2cb0 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.850+0000 7f613257e640 1 -- 172.21.15.19:0/4005585075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612c0b3ca0 con 0x7f612c0b2cb0 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.851+0000 7f613257e640 1 -- 172.21.15.19:0/4005585075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c0b4da0 con 0x7f612c0b2cb0 2024-04-25T10:02:54.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.851+0000 7f613257e640 1 -- 172.21.15.19:0/4005585075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612c073990 con 0x7f612c0b2cb0 2024-04-25T10:02:54.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.852+0000 7f6128ff9640 1 -- 172.21.15.19:0/4005585075 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f611c044020 con 0x7f612c0b2cb0 2024-04-25T10:02:54.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.852+0000 7f6128ff9640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f611806b4d0 0x7f611806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.852+0000 7f6128ff9640 1 -- 172.21.15.19:0/4005585075 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f611c0aab40 con 0x7f612c0b2cb0 2024-04-25T10:02:54.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.853+0000 7f612b7fe640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f611806b4d0 0x7f611806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.853+0000 7f612b7fe640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f611806b4d0 0x7f611806d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f612c1f6b10 tx=0x7f6120008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:54.857+0000 7f6128ff9640 1 -- 172.21.15.19:0/4005585075 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f611c04b030 con 0x7f612c0b2cb0 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.072+0000 7f613257e640 1 -- 172.21.15.19:0/4005585075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f612c062210 con 0x7f612c0b2cb0 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.073+0000 7f6128ff9640 1 -- 172.21.15.19:0/4005585075 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f611c079890 con 0x7f612c0b2cb0 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":94,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7466,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/590882657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":590882657},{"type":"v1","addr":"172.21.15.112:6836","nonce":590882657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":9744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1740426538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1740426538},{"type":"v1","addr":"172.21.15.19:6833","nonce":1740426538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":94,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:02:34.992088+0000","modified":"2024-04-25T10:02:54.722246+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9750":{"gid":9750,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/525024117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":525024117},{"type":"v1","addr":"172.21.15.19:6835","nonce":525024117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9762":{"gid":9762,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2648279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2648279035},{"type":"v1","addr":"172.21.15.112:6837","nonce":2648279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 94 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 -- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f611806b4d0 msgr2=0x7f611806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f611806b4d0 0x7f611806d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f612c1f6b10 tx=0x7f6120008040 comp rx=0 tx=0).stop 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 -- 172.21.15.19:0/4005585075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f612c0b2cb0 msgr2=0x7f612c0b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f612c0b2cb0 0x7f612c0b32e0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f611c02a220 tx=0x7f611c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 -- 172.21.15.19:0/4005585075 shutdown_connections 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f611806b4d0 0x7f611806d990 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c15dbe0 0x7f612c15ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c0b92a0 0x7f612c14bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 --2- 172.21.15.19:0/4005585075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f612c0b2cb0 0x7f612c0b32e0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 -- 172.21.15.19:0/4005585075 >> 172.21.15.19:0/4005585075 conn(0x7f612c061a20 msgr2=0x7f612c15b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 -- 172.21.15.19:0/4005585075 shutdown_connections 2024-04-25T10:02:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.075+0000 7f613257e640 1 -- 172.21.15.19:0/4005585075 wait complete. 2024-04-25T10:02:55.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T10:02:34.992088+0000', 'modified': '2024-04-25T10:02:54.722246+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9762}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9750': {'gid': 9750, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/525024117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 525024117}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 525024117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9762': {'gid': 9762, 'name': 'd', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6837/2648279035', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6835', 'nonce': 2648279035}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 2648279035}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:55.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:55.494 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:02:55.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.594+0000 7fe75c29e640 1 Processor -- start 2024-04-25T10:02:55.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.594+0000 7fe75c29e640 1 -- start start 2024-04-25T10:02:55.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.595+0000 7fe75c29e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe75415bb50 0x7fe7540c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.595+0000 7fe75c29e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe7540c70d0 0x7fe7540cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.595+0000 7fe75c29e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7540c75e0 con 0x7fe7540cfb40 2024-04-25T10:02:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.595+0000 7fe75c29e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7540c8680 con 0x7fe75415bb50 2024-04-25T10:02:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.595+0000 7fe75c29e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7540c78c0 con 0x7fe7540c70d0 2024-04-25T10:02:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.595+0000 7fe75a03a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe75415bb50 0x7fe7540c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.595+0000 7fe75a03a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe75415bb50 0x7fe7540c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43602/0 (socket says 172.21.15.19:43602) 2024-04-25T10:02:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.595+0000 7fe75a03a640 1 -- 172.21.15.19:0/795308589 learned_addr learned my addr 172.21.15.19:0/795308589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.596+0000 7fe759839640 1 --2- 172.21.15.19:0/795308589 >> v2:172.21.15.112:3300/0 conn(0x7fe7540c70d0 0x7fe7540cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.596+0000 7fe75a03a640 1 -- 172.21.15.19:0/795308589 >> v2:172.21.15.112:3300/0 conn(0x7fe7540c70d0 msgr2=0x7fe7540cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.596+0000 7fe75a03a640 1 --2- 172.21.15.19:0/795308589 >> v2:172.21.15.112:3300/0 conn(0x7fe7540c70d0 0x7fe7540cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.596+0000 7fe75a03a640 1 -- 172.21.15.19:0/795308589 >> v1:172.21.15.112:6789/0 conn(0x7fe7540cfb40 legacy=0x7fe754070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.596+0000 7fe75a03a640 1 -- 172.21.15.19:0/795308589 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7540c7ba0 con 0x7fe75415bb50 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.596+0000 7fe75a03a640 1 --2- 172.21.15.19:0/795308589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75415bb50 0x7fe7540c6b90 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fe75406f220 tx=0x7fe74402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f3bb9f8f7787e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.597+0000 7fe759038640 1 -- 172.21.15.19:0/795308589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744036070 con 0x7fe75415bb50 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.597+0000 7fe759038640 1 -- 172.21.15.19:0/795308589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe744031600 con 0x7fe75415bb50 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.597+0000 7fe759038640 1 -- 172.21.15.19:0/795308589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7440319d0 con 0x7fe75415bb50 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.597+0000 7fe75c29e640 1 -- 172.21.15.19:0/795308589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75415bb50 msgr2=0x7fe7540c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.597+0000 7fe75c29e640 1 --2- 172.21.15.19:0/795308589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75415bb50 0x7fe7540c6b90 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fe75406f220 tx=0x7fe74402a780 comp rx=0 tx=0).stop 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.597+0000 7fe75c29e640 1 -- 172.21.15.19:0/795308589 shutdown_connections 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.597+0000 7fe75c29e640 1 --2- 172.21.15.19:0/795308589 >> v2:172.21.15.112:3300/0 conn(0x7fe7540c70d0 0x7fe7540cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.597+0000 7fe75c29e640 1 --2- 172.21.15.19:0/795308589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe75415bb50 0x7fe7540c6b90 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.597+0000 7fe75c29e640 1 -- 172.21.15.19:0/795308589 >> 172.21.15.19:0/795308589 conn(0x7fe75406f500 msgr2=0x7fe75406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.597+0000 7fe75c29e640 1 -- 172.21.15.19:0/795308589 shutdown_connections 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.597+0000 7fe75c29e640 1 -- 172.21.15.19:0/795308589 wait complete. 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.598+0000 7fe75c29e640 1 Processor -- start 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.598+0000 7fe75c29e640 1 -- start start 2024-04-25T10:02:55.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.599+0000 7fe75c29e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7540c70d0 0x7fe7540b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.599+0000 7fe75c29e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7540cfb40 0x7fe7541587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.599+0000 7fe75c29e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe75415bb50 0x7fe75415c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.599+0000 7fe75c29e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75415f6d0 con 0x7fe7540cfb40 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.599+0000 7fe75c29e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540c8160 con 0x7fe7540c70d0 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.599+0000 7fe75c29e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75415f850 con 0x7fe75415bb50 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.599+0000 7fe759839640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7540cfb40 0x7fe7541587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.599+0000 7fe759839640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7540cfb40 0x7fe7541587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43616/0 (socket says 172.21.15.19:43616) 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.599+0000 7fe759839640 1 -- 172.21.15.19:0/1745803435 learned_addr learned my addr 172.21.15.19:0/1745803435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.599+0000 7fe75a03a640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7540c70d0 0x7fe7540b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.599+0000 7fe75a83b640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe75415bb50 0x7fe75415c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.600+0000 7fe759839640 1 -- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe75415bb50 msgr2=0x7fe75415c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.600+0000 7fe759839640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe75415bb50 0x7fe75415c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.600+0000 7fe759839640 1 -- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7540c70d0 msgr2=0x7fe7540b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.600+0000 7fe759839640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7540c70d0 0x7fe7540b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.600+0000 7fe759839640 1 -- 172.21.15.19:0/1745803435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75415cdf0 con 0x7fe7540cfb40 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.600+0000 7fe759839640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7540cfb40 0x7fe7541587e0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fe74800d7d0 tx=0x7fe74800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.600+0000 7fe752ffd640 1 -- 172.21.15.19:0/1745803435 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe748004020 con 0x7fe7540cfb40 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.600+0000 7fe752ffd640 1 -- 172.21.15.19:0/1745803435 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7480041e0 con 0x7fe7540cfb40 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.600+0000 7fe752ffd640 1 -- 172.21.15.19:0/1745803435 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7480043d0 con 0x7fe7540cfb40 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.600+0000 7fe75c29e640 1 -- 172.21.15.19:0/1745803435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe754158e50 con 0x7fe7540cfb40 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.600+0000 7fe75c29e640 1 -- 172.21.15.19:0/1745803435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe754159410 con 0x7fe7540cfb40 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.601+0000 7fe752ffd640 1 -- 172.21.15.19:0/1745803435 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe74801b020 con 0x7fe7540cfb40 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.602+0000 7fe752ffd640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe74006b4d0 0x7fe74006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.602+0000 7fe752ffd640 1 -- 172.21.15.19:0/1745803435 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe748089fc0 con 0x7fe7540cfb40 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.602+0000 7fe75a03a640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe74006b4d0 0x7fe74006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.603+0000 7fe75a03a640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe74006b4d0 0x7fe74006d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe744033000 tx=0x7fe744009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.603+0000 7fe75efa2740 1 Processor -- start 2024-04-25T10:02:55.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.603+0000 7fe75efa2740 1 -- start start 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df447e0 0x55dd2df1ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df44bc0 0x55dd2df47040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df1ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75efa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2de7fa10 con 0x55dd2df47580 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75efa2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2de28b30 con 0x55dd2df44bc0 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75efa2740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd2de7bc50 con 0x55dd2df447e0 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75a83b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df1ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75a83b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df1ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43626/0 (socket says 172.21.15.19:43626) 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75a83b640 1 -- 172.21.15.19:0/3044713496 learned_addr learned my addr 172.21.15.19:0/3044713496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75a03a640 1 --2- 172.21.15.19:0/3044713496 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df447e0 0x55dd2df1ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe759839640 1 --2- 172.21.15.19:0/3044713496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df44bc0 0x55dd2df47040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75a83b640 1 -- 172.21.15.19:0/3044713496 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df447e0 msgr2=0x55dd2df1ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75a83b640 1 --2- 172.21.15.19:0/3044713496 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df447e0 0x55dd2df1ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75a83b640 1 -- 172.21.15.19:0/3044713496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df44bc0 msgr2=0x55dd2df47040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75a83b640 1 --2- 172.21.15.19:0/3044713496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df44bc0 0x55dd2df47040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.604+0000 7fe75a83b640 1 -- 172.21.15.19:0/3044713496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd2de3de10 con 0x55dd2df47580 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.605+0000 7fe75a83b640 1 --2- 172.21.15.19:0/3044713496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df1ade0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fe74c005020 tx=0x7fe74c0054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecebee4d374171a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.605+0000 7fe73bfff640 1 -- 172.21.15.19:0/3044713496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c018030 con 0x55dd2df47580 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.605+0000 7fe73bfff640 1 -- 172.21.15.19:0/3044713496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c00fae0 con 0x55dd2df47580 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.605+0000 7fe73bfff640 1 -- 172.21.15.19:0/3044713496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00fcb0 con 0x55dd2df47580 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.605+0000 7fe75efa2740 1 -- 172.21.15.19:0/3044713496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 msgr2=0x55dd2df1ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.605+0000 7fe75efa2740 1 --2- 172.21.15.19:0/3044713496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df1ade0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fe74c005020 tx=0x7fe74c0054e0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.606+0000 7fe75efa2740 1 -- 172.21.15.19:0/3044713496 shutdown_connections 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.606+0000 7fe75efa2740 1 --2- 172.21.15.19:0/3044713496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df1ade0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.606+0000 7fe75efa2740 1 --2- 172.21.15.19:0/3044713496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df44bc0 0x55dd2df47040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.606+0000 7fe75efa2740 1 --2- 172.21.15.19:0/3044713496 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df447e0 0x55dd2df1ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.606+0000 7fe75efa2740 1 -- 172.21.15.19:0/3044713496 >> 172.21.15.19:0/3044713496 conn(0x55dd2df19840 msgr2=0x55dd2df62040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.606+0000 7fe75efa2740 1 -- 172.21.15.19:0/3044713496 shutdown_connections 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.606+0000 7fe75efa2740 1 -- 172.21.15.19:0/3044713496 wait complete. 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.606+0000 7fe75efa2740 1 -- start start 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.606+0000 7fe75efa2740 1 Processor -- start 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.607+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df44bc0 0x55dd2df590b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.607+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df47580 0x55dd2df42d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.607+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df434a0 0x55dd2df4a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.607+0000 7fe75efa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2de3c0c0 con 0x55dd2df434a0 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.607+0000 7fe75efa2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2df5fd40 con 0x55dd2df47580 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.607+0000 7fe75efa2740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd2df76e50 con 0x55dd2df44bc0 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.607+0000 7fe75efa2740 10 client.0 _ll_register_callbacks cb 0x55dd2de2fec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.607+0000 7fe75a83b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df434a0 0x55dd2df4a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.607+0000 7fe75a83b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df434a0 0x55dd2df4a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43638/0 (socket says 172.21.15.19:43638) 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.607+0000 7fe75a83b640 1 -- 172.21.15.19:0/1723497746 learned_addr learned my addr 172.21.15.19:0/1723497746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.607+0000 7fe759839640 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df47580 0x55dd2df42d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe75a03a640 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df44bc0 0x55dd2df590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe75a83b640 1 -- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df44bc0 msgr2=0x55dd2df590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe75a83b640 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df44bc0 0x55dd2df590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe75a83b640 1 -- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df47580 msgr2=0x55dd2df42d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe75a83b640 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df47580 0x55dd2df42d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe75a83b640 1 -- 172.21.15.19:0/1723497746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd2db942d0 con 0x55dd2df434a0 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe75a03a640 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df44bc0 0x55dd2df590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe75a83b640 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df434a0 0x55dd2df4a360 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fe74c017040 tx=0x7fe74c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c670b2a3093737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe7397fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe7397fa640 1 -- 172.21.15.19:0/1723497746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c007230 con 0x55dd2df434a0 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe7397fa640 1 -- 172.21.15.19:0/1723497746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe74c0073d0 con 0x55dd2df434a0 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe75efa2740 1 -- 172.21.15.19:0/1723497746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe74c00fae0 con 0x55dd2df434a0 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe7397fa640 1 -- 172.21.15.19:0/1723497746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe74c02b050 con 0x55dd2df434a0 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe75efa2740 10 client.9927 fetch_fsmap learned FSMap version 94 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe75efa2740 1 -- 172.21.15.19:0/1723497746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x55dd2dc5e6b0 con 0x55dd2df434a0 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.608+0000 7fe7397fa640 1 -- 172.21.15.19:0/1723497746 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c00fae0 con 0x55dd2df434a0 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.609+0000 7fe7397fa640 1 -- 172.21.15.19:0/1723497746 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 94) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fe74c02e020 con 0x55dd2df434a0 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.609+0000 7fe75efa2740 10 client.9927 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.609+0000 7fe75efa2740 10 client.9927 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.609+0000 7fe75efa2740 20 client.9927 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.609+0000 7fe75efa2740 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x55dd2df82ba0 0x55dd2df84f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.609+0000 7fe75efa2740 4 client.9927 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.609+0000 7fe75efa2740 1 -- 172.21.15.19:0/1723497746 --> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd2dede920 con 0x55dd2df82ba0 2024-04-25T10:02:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.609+0000 7fe75a03a640 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x55dd2df82ba0 0x55dd2df84f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.609+0000 7fe7397fa640 10 client.9927 ms_handle_connect on v2:172.21.15.112:6835/2648279035 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.609+0000 7fe75a03a640 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x55dd2df82ba0 0x55dd2df84f90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5767a263cc0f7d29 server_cookie=4a5dc327f9f93471 in_seq=0 out_seq=0 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe7397fa640 1 -- 172.21.15.19:0/1723497746 <== mds.0 v2:172.21.15.112:6835/2648279035 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd2dede920 con 0x55dd2df82ba0 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe7397fa640 10 client.9927 handle_command_reply: tid=0 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe75efa2740 1 client.9927 shutdown 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe75efa2740 1 -- 172.21.15.19:0/1723497746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df434a0 msgr2=0x55dd2df4a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe75efa2740 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df434a0 0x55dd2df4a360 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fe74c017040 tx=0x7fe74c0036f0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe75efa2740 1 -- 172.21.15.19:0/1723497746 shutdown_connections 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe75efa2740 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x55dd2df82ba0 0x55dd2df84f90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe75efa2740 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df434a0 0x55dd2df4a360 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe75efa2740 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df47580 0x55dd2df42d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe75efa2740 1 --2- 172.21.15.19:0/1723497746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df44bc0 0x55dd2df590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe7397fa640 0 client.9927 ms_handle_reset on v2:172.21.15.112:6835/2648279035 2024-04-25T10:02:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe75efa2740 1 -- 172.21.15.19:0/1723497746 >> 172.21.15.19:0/1723497746 conn(0x55dd2df447e0 msgr2=0x55dd2df1ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe75efa2740 1 -- 172.21.15.19:0/1723497746 shutdown_connections 2024-04-25T10:02:55.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.610+0000 7fe75efa2740 1 -- 172.21.15.19:0/1723497746 wait complete. 2024-04-25T10:02:55.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.611+0000 7fe75efa2740 20 client.9927 trim_cache size 0 max 16384 2024-04-25T10:02:55.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.627+0000 7fe75efa2740 1 Processor -- start 2024-04-25T10:02:55.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.627+0000 7fe75efa2740 1 -- start start 2024-04-25T10:02:55.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df434a0 0x55dd2df1ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df44bc0 0x55dd2df1b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df5e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75efa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7541fc210 con 0x55dd2df47580 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75efa2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540ca250 con 0x55dd2df434a0 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75efa2740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75406cf00 con 0x55dd2df44bc0 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75a83b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df5e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75a83b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df5e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43654/0 (socket says 172.21.15.19:43654) 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75a83b640 1 -- 172.21.15.19:0/3010616552 learned_addr learned my addr 172.21.15.19:0/3010616552 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe759839640 1 --2- 172.21.15.19:0/3010616552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df44bc0 0x55dd2df1b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75a03a640 1 --2- 172.21.15.19:0/3010616552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df434a0 0x55dd2df1ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75a83b640 1 -- 172.21.15.19:0/3010616552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df44bc0 msgr2=0x55dd2df1b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75a83b640 1 --2- 172.21.15.19:0/3010616552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df44bc0 0x55dd2df1b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75a83b640 1 -- 172.21.15.19:0/3010616552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df434a0 msgr2=0x55dd2df1ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75a83b640 1 --2- 172.21.15.19:0/3010616552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df434a0 0x55dd2df1ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75a83b640 1 -- 172.21.15.19:0/3010616552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd2df60740 con 0x55dd2df47580 2024-04-25T10:02:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe75a83b640 1 --2- 172.21.15.19:0/3010616552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df5e980 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0076c0 tx=0x7fe74c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e013cbc9e704bcba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe73b7fe640 1 -- 172.21.15.19:0/3010616552 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c018070 con 0x55dd2df47580 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe73b7fe640 1 -- 172.21.15.19:0/3010616552 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c00fce0 con 0x55dd2df47580 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.628+0000 7fe73b7fe640 1 -- 172.21.15.19:0/3010616552 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c003ad0 con 0x55dd2df47580 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.629+0000 7fe75efa2740 1 -- 172.21.15.19:0/3010616552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 msgr2=0x55dd2df5e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.629+0000 7fe75efa2740 1 --2- 172.21.15.19:0/3010616552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df5e980 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0076c0 tx=0x7fe74c01e000 comp rx=0 tx=0).stop 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.629+0000 7fe75efa2740 1 -- 172.21.15.19:0/3010616552 shutdown_connections 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.629+0000 7fe75efa2740 1 --2- 172.21.15.19:0/3010616552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df47580 0x55dd2df5e980 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.629+0000 7fe75efa2740 1 --2- 172.21.15.19:0/3010616552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df44bc0 0x55dd2df1b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.629+0000 7fe75efa2740 1 --2- 172.21.15.19:0/3010616552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df434a0 0x55dd2df1ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.629+0000 7fe75efa2740 1 -- 172.21.15.19:0/3010616552 >> 172.21.15.19:0/3010616552 conn(0x55dd2df447e0 msgr2=0x55dd2df42b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.629+0000 7fe75efa2740 1 -- 172.21.15.19:0/3010616552 shutdown_connections 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.629+0000 7fe75efa2740 1 -- 172.21.15.19:0/3010616552 wait complete. 2024-04-25T10:02:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.630+0000 7fe75efa2740 1 -- start start 2024-04-25T10:02:55.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.630+0000 7fe75efa2740 1 Processor -- start 2024-04-25T10:02:55.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.630+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df434a0 0x55dd2df1ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.630+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df44bc0 0x55dd2df1b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe75efa2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df47580 0x55dd2df40f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe75efa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2df5a020 con 0x55dd2df44bc0 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe75efa2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2df59a60 con 0x55dd2df47580 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe75efa2740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd2df620a0 con 0x55dd2df434a0 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe75efa2740 10 client.0 _ll_register_callbacks cb 0x7fe74c004a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe759839640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df44bc0 0x55dd2df1b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe759839640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df44bc0 0x55dd2df1b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43660/0 (socket says 172.21.15.19:43660) 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe759839640 1 -- 172.21.15.19:0/2845161884 learned_addr learned my addr 172.21.15.19:0/2845161884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe75a03a640 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df434a0 0x55dd2df1ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe75a83b640 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df47580 0x55dd2df40f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe759839640 1 -- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df434a0 msgr2=0x55dd2df1ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe759839640 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df434a0 0x55dd2df1ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe759839640 1 -- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df47580 msgr2=0x55dd2df40f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe759839640 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df47580 0x55dd2df40f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe759839640 1 -- 172.21.15.19:0/2845161884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd2dfd19c0 con 0x55dd2df44bc0 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe759839640 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df44bc0 0x55dd2df1b320 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe7480607f0 tx=0x7fe7480549f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776ec016b2510e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe7397fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe7397fa640 1 -- 172.21.15.19:0/2845161884 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe748070150 con 0x55dd2df44bc0 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe7397fa640 1 -- 172.21.15.19:0/2845161884 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe748057b10 con 0x55dd2df44bc0 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe75efa2740 1 -- 172.21.15.19:0/2845161884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe74c00fce0 con 0x55dd2df44bc0 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe7397fa640 1 -- 172.21.15.19:0/2845161884 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe748057d10 con 0x55dd2df44bc0 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe75efa2740 10 client.9933 fetch_fsmap learned FSMap version 94 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe75efa2740 1 -- 172.21.15.19:0/2845161884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x55dd2dfd0fc0 con 0x55dd2df44bc0 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.631+0000 7fe7397fa640 1 -- 172.21.15.19:0/2845161884 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c00fce0 con 0x55dd2df44bc0 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe7397fa640 1 -- 172.21.15.19:0/2845161884 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 94) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fe748013070 con 0x55dd2df44bc0 2024-04-25T10:02:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe75efa2740 10 client.9933 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe75efa2740 10 client.9933 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe75efa2740 20 client.9933 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe75efa2740 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x55dd2df41b50 0x55dd2df57ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe75efa2740 4 client.9933 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe75efa2740 1 -- 172.21.15.19:0/2845161884 --> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dd2de8cf50 con 0x55dd2df41b50 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe75a03a640 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x55dd2df41b50 0x55dd2df57ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe7397fa640 10 client.9933 ms_handle_connect on v2:172.21.15.112:6835/2648279035 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe75a03a640 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x55dd2df41b50 0x55dd2df57ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c91574541c2cb1 server_cookie=43d1af70e764942e in_seq=0 out_seq=0 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe7397fa640 1 -- 172.21.15.19:0/2845161884 <== mds.0 v2:172.21.15.112:6835/2648279035 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55dd2de8cf50 con 0x55dd2df41b50 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe7397fa640 10 client.9933 handle_command_reply: tid=0 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.632+0000 7fe75efa2740 1 client.9933 shutdown 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75efa2740 1 -- 172.21.15.19:0/2845161884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df44bc0 msgr2=0x55dd2df1b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75efa2740 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df44bc0 0x55dd2df1b320 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe7480607f0 tx=0x7fe7480549f0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75efa2740 1 -- 172.21.15.19:0/2845161884 shutdown_connections 2024-04-25T10:02:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75efa2740 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:6835/2648279035,v1:172.21.15.112:6837/2648279035] conn(0x55dd2df41b50 0x55dd2df57ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75efa2740 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd2df47580 0x55dd2df40f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75efa2740 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dd2df44bc0 0x55dd2df1b320 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75efa2740 1 --2- 172.21.15.19:0/2845161884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55dd2df434a0 0x55dd2df1ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe7397fa640 0 client.9933 ms_handle_reset on v2:172.21.15.112:6835/2648279035 2024-04-25T10:02:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75efa2740 1 -- 172.21.15.19:0/2845161884 >> 172.21.15.19:0/2845161884 conn(0x55dd2df447e0 msgr2=0x55dd2df4ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75efa2740 1 -- 172.21.15.19:0/2845161884 shutdown_connections 2024-04-25T10:02:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75efa2740 1 -- 172.21.15.19:0/2845161884 wait complete. 2024-04-25T10:02:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75efa2740 20 client.9933 trim_cache size 0 max 16384 2024-04-25T10:02:55.644 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 9762, 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 19.982736558999999, 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 94, 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 142, 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 142, 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 24.319953701999999 2024-04-25T10:02:55.645 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:02:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75c29e640 1 -- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe74006b4d0 msgr2=0x7fe74006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75c29e640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe74006b4d0 0x7fe74006d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe744033000 tx=0x7fe744009000 comp rx=0 tx=0).stop 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75c29e640 1 -- 172.21.15.19:0/1745803435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7540cfb40 msgr2=0x7fe7541587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75c29e640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7540cfb40 0x7fe7541587e0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fe74800d7d0 tx=0x7fe74800dc90 comp rx=0 tx=0).stop 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75c29e640 1 -- 172.21.15.19:0/1745803435 shutdown_connections 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75c29e640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe74006b4d0 0x7fe74006d990 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75c29e640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe75415bb50 0x7fe75415c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75c29e640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7540cfb40 0x7fe7541587e0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75c29e640 1 --2- 172.21.15.19:0/1745803435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7540c70d0 0x7fe7540b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75c29e640 1 -- 172.21.15.19:0/1745803435 >> 172.21.15.19:0/1745803435 conn(0x7fe75406f500 msgr2=0x7fe75406f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.633+0000 7fe75c29e640 1 -- 172.21.15.19:0/1745803435 shutdown_connections 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.634+0000 7fe75c29e640 1 -- 172.21.15.19:0/1745803435 wait complete. 2024-04-25T10:02:55.649 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub start abs_test_path recursive 2024-04-25T10:02:55.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.749+0000 7fa9dea58640 1 Processor -- start 2024-04-25T10:02:55.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.749+0000 7fa9dea58640 1 -- start start 2024-04-25T10:02:55.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.750+0000 7fa9dea58640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d815b7d0 0x7fa9d80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.750+0000 7fa9dea58640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9d806c9e0 0x7fa9d80b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.750+0000 7fa9dea58640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d806cf50 con 0x7fa9d80b5620 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.750+0000 7fa9dea58640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d80b5fc0 con 0x7fa9d806c9e0 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.750+0000 7fa9dea58640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80ce530 con 0x7fa9d815b7d0 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.750+0000 7fa9d7fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d815b7d0 0x7fa9d80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.750+0000 7fa9dcff5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa9d806c9e0 0x7fa9d80b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.750+0000 7fa9d77fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa9d80b5620 0x7fa9d806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:43798/0 (socket says 172.21.15.19:43798) 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.750+0000 7fa9d77fe640 1 -- 172.21.15.19:0/1110525565 learned_addr learned my addr 172.21.15.19:0/1110525565 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.750+0000 7fa9d7fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d815b7d0 0x7fa9d80cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:41382/0 (socket says 172.21.15.19:41382) 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.751+0000 7fa9d6ffd640 1 -- 172.21.15.19:0/1110525565 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714110219 0 0) 0x7fa9d806cf50 con 0x7fa9d80b5620 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.751+0000 7fa9d6ffd640 1 -- 172.21.15.19:0/1110525565 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c0003520 con 0x7fa9d80b5620 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.751+0000 7fa9d7fff640 1 -- 172.21.15.19:0/1110525565 >> v2:172.21.15.112:3300/0 conn(0x7fa9d806c9e0 msgr2=0x7fa9d80b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.751+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/1110525565 >> v2:172.21.15.112:3300/0 conn(0x7fa9d806c9e0 0x7fa9d80b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.751+0000 7fa9d7fff640 1 -- 172.21.15.19:0/1110525565 >> v1:172.21.15.112:6789/0 conn(0x7fa9d80b5620 legacy=0x7fa9d806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.751+0000 7fa9d7fff640 1 -- 172.21.15.19:0/1110525565 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d80ce810 con 0x7fa9d815b7d0 2024-04-25T10:02:55.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.751+0000 7fa9dcff5640 1 --2- 172.21.15.19:0/1110525565 >> v2:172.21.15.112:3300/0 conn(0x7fa9d806c9e0 0x7fa9d80b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:55.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.751+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/1110525565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d815b7d0 0x7fa9d80cf010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0097a0 tx=0x7fa9cc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a9363891abecf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.752+0000 7fa9d6ffd640 1 -- 172.21.15.19:0/1110525565 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc037070 con 0x7fa9d815b7d0 2024-04-25T10:02:55.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.752+0000 7fa9d6ffd640 1 -- 172.21.15.19:0/1110525565 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc032440 con 0x7fa9d815b7d0 2024-04-25T10:02:55.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.752+0000 7fa9d6ffd640 1 -- 172.21.15.19:0/1110525565 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc032810 con 0x7fa9d815b7d0 2024-04-25T10:02:55.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.752+0000 7fa9dea58640 1 -- 172.21.15.19:0/1110525565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d815b7d0 msgr2=0x7fa9d80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.752+0000 7fa9dea58640 1 --2- 172.21.15.19:0/1110525565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d815b7d0 0x7fa9d80cf010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0097a0 tx=0x7fa9cc02a180 comp rx=0 tx=0).stop 2024-04-25T10:02:55.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.752+0000 7fa9dea58640 1 -- 172.21.15.19:0/1110525565 shutdown_connections 2024-04-25T10:02:55.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.752+0000 7fa9dea58640 1 --2- 172.21.15.19:0/1110525565 >> v2:172.21.15.112:3300/0 conn(0x7fa9d806c9e0 0x7fa9d80b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.752+0000 7fa9dea58640 1 --2- 172.21.15.19:0/1110525565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d815b7d0 0x7fa9d80cf010 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.752+0000 7fa9dea58640 1 -- 172.21.15.19:0/1110525565 >> 172.21.15.19:0/1110525565 conn(0x7fa9d80d0800 msgr2=0x7fa9d80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.752+0000 7fa9dea58640 1 -- 172.21.15.19:0/1110525565 shutdown_connections 2024-04-25T10:02:55.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.752+0000 7fa9dea58640 1 -- 172.21.15.19:0/1110525565 wait complete. 2024-04-25T10:02:55.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.753+0000 7fa9dea58640 1 Processor -- start 2024-04-25T10:02:55.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.754+0000 7fa9dea58640 1 -- start start 2024-04-25T10:02:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.754+0000 7fa9dea58640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d806c9e0 0x7fa9d80cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.754+0000 7fa9dea58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d80b5620 0x7fa9d80cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.754+0000 7fa9dea58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d815b7d0 0x7fa9d80cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.754+0000 7fa9dea58640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80cf550 con 0x7fa9d815b7d0 2024-04-25T10:02:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.754+0000 7fa9dea58640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d806d230 con 0x7fa9d80b5620 2024-04-25T10:02:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.754+0000 7fa9dea58640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80cf6d0 con 0x7fa9d806c9e0 2024-04-25T10:02:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.754+0000 7fa9dcff5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d815b7d0 0x7fa9d80cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9dcff5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d815b7d0 0x7fa9d80cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43664/0 (socket says 172.21.15.19:43664) 2024-04-25T10:02:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9dcff5640 1 -- 172.21.15.19:0/1271398352 learned_addr learned my addr 172.21.15.19:0/1271398352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/1271398352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d806c9e0 0x7fa9d80cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/1271398352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d80b5620 0x7fa9d80cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9dcff5640 1 -- 172.21.15.19:0/1271398352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d806c9e0 msgr2=0x7fa9d80cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9dcff5640 1 --2- 172.21.15.19:0/1271398352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d806c9e0 0x7fa9d80cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9dcff5640 1 -- 172.21.15.19:0/1271398352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d80b5620 msgr2=0x7fa9d80cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9dcff5640 1 --2- 172.21.15.19:0/1271398352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d80b5620 0x7fa9d80cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9dcff5640 1 -- 172.21.15.19:0/1271398352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d80b7e50 con 0x7fa9d815b7d0 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/1271398352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d80b5620 0x7fa9d80cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/1271398352 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9d806c9e0 0x7fa9d80cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9dcff5640 1 --2- 172.21.15.19:0/1271398352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9d815b7d0 0x7fa9d80cb7f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000efc0 tx=0x7fa9d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9d4ff9640 1 -- 172.21.15.19:0/1271398352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0018030 con 0x7fa9d815b7d0 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9d4ff9640 1 -- 172.21.15.19:0/1271398352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d000cd30 con 0x7fa9d815b7d0 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9dea58640 1 -- 172.21.15.19:0/1271398352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d80b8130 con 0x7fa9d815b7d0 2024-04-25T10:02:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9d4ff9640 1 -- 172.21.15.19:0/1271398352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0003bb0 con 0x7fa9d815b7d0 2024-04-25T10:02:55.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.755+0000 7fa9dea58640 1 -- 172.21.15.19:0/1271398352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d80b86f0 con 0x7fa9d815b7d0 2024-04-25T10:02:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.757+0000 7fa9d4ff9640 1 -- 172.21.15.19:0/1271398352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9d001b070 con 0x7fa9d815b7d0 2024-04-25T10:02:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.757+0000 7fa9d4ff9640 1 --2- 172.21.15.19:0/1271398352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa9c406b4d0 0x7fa9c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.757+0000 7fa9d4ff9640 1 -- 172.21.15.19:0/1271398352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa9d008ad50 con 0x7fa9d815b7d0 2024-04-25T10:02:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.758+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/1271398352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa9c406b4d0 0x7fa9c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.758+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/1271398352 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa9c406b4d0 0x7fa9c406d990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc034000 tx=0x7fa9cc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.758+0000 7fa9e1836740 1 Processor -- start 2024-04-25T10:02:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.758+0000 7fa9e1836740 1 -- start start 2024-04-25T10:02:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfffe0 0x56302ae027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae02d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302add2c40 0x56302add6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9e1836740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302ad3bb20 con 0x56302add2c40 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9e1836740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302ace4e10 con 0x56302adfffe0 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9e1836740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56302ad37bc0 con 0x56302ae003c0 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9dcff5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302add2c40 0x56302add6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9dcff5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302add2c40 0x56302add6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43670/0 (socket says 172.21.15.19:43670) 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9dcff5640 1 -- 172.21.15.19:0/2999087907 learned_addr learned my addr 172.21.15.19:0/2999087907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/2999087907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfffe0 0x56302ae027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/2999087907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae02d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9dcff5640 1 -- 172.21.15.19:0/2999087907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 msgr2=0x56302ae02d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9dcff5640 1 --2- 172.21.15.19:0/2999087907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae02d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9dcff5640 1 -- 172.21.15.19:0/2999087907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfffe0 msgr2=0x56302ae027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9dcff5640 1 --2- 172.21.15.19:0/2999087907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfffe0 0x56302ae027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9dcff5640 1 -- 172.21.15.19:0/2999087907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56302ab1c7e0 con 0x56302add2c40 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.759+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/2999087907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae02d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9dcff5640 1 --2- 172.21.15.19:0/2999087907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302add2c40 0x56302add6450 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00614f0 tx=0x7fa9d0058f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d97478f3759e075 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9bdffb640 1 -- 172.21.15.19:0/2999087907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0057610 con 0x56302add2c40 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9bdffb640 1 -- 172.21.15.19:0/2999087907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d005d150 con 0x56302add2c40 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9bdffb640 1 -- 172.21.15.19:0/2999087907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0058760 con 0x56302add2c40 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9e1836740 1 -- 172.21.15.19:0/2999087907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302add2c40 msgr2=0x56302add6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9e1836740 1 --2- 172.21.15.19:0/2999087907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302add2c40 0x56302add6450 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00614f0 tx=0x7fa9d0058f80 comp rx=0 tx=0).stop 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9e1836740 1 -- 172.21.15.19:0/2999087907 shutdown_connections 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9e1836740 1 --2- 172.21.15.19:0/2999087907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302add2c40 0x56302add6450 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9e1836740 1 --2- 172.21.15.19:0/2999087907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae02d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9e1836740 1 --2- 172.21.15.19:0/2999087907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfffe0 0x56302ae027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9e1836740 1 -- 172.21.15.19:0/2999087907 >> 172.21.15.19:0/2999087907 conn(0x56302add4eb0 msgr2=0x56302ae1d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9e1836740 1 -- 172.21.15.19:0/2999087907 shutdown_connections 2024-04-25T10:02:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.760+0000 7fa9e1836740 1 -- 172.21.15.19:0/2999087907 wait complete. 2024-04-25T10:02:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.761+0000 7fa9e1836740 1 -- start start 2024-04-25T10:02:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.761+0000 7fa9e1836740 1 Processor -- start 2024-04-25T10:02:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302adfffe0 0x56302ae314c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302ae003c0 0x56302ae06890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfc7f0 0x56302adf8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9e1836740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302add8060 con 0x56302ae003c0 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9e1836740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302acda540 con 0x56302adfc7f0 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9e1836740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56302add8340 con 0x56302adfffe0 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9e1836740 10 client.0 _ll_register_callbacks cb 0x56302aa4fbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302ae003c0 0x56302ae06890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302ae003c0 0x56302ae06890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43682/0 (socket says 172.21.15.19:43682) 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9d77fe640 1 -- 172.21.15.19:0/510116614 learned_addr learned my addr 172.21.15.19:0/510116614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9dcff5640 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfc7f0 0x56302adf8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302adfffe0 0x56302ae314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9d77fe640 1 -- 172.21.15.19:0/510116614 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302adfffe0 msgr2=0x56302ae314c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302adfffe0 0x56302ae314c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9d77fe640 1 -- 172.21.15.19:0/510116614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfc7f0 msgr2=0x56302adf8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfc7f0 0x56302adf8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9d77fe640 1 -- 172.21.15.19:0/510116614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56302ab7cf70 con 0x56302ae003c0 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302ae003c0 0x56302ae06890 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8005880 tx=0x7fa9c8007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ff776b41f853fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.762+0000 7fa9b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/510116614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8001bb0 con 0x56302ae003c0 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/510116614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa9c8014400 con 0x56302ae003c0 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9e1836740 1 -- 172.21.15.19:0/510116614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9d005d150 con 0x56302ae003c0 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/510116614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9c8021050 con 0x56302ae003c0 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9e1836740 10 client.9942 fetch_fsmap learned FSMap version 94 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9e1836740 1 -- 172.21.15.19:0/510116614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x56302aa100d0 con 0x56302ae003c0 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/510116614 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d005d150 con 0x56302ae003c0 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/510116614 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 94) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fa9c8024020 con 0x56302ae003c0 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9e1836740 10 client.9942 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T10:02:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9e1836740 10 client.9942 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9e1836740 20 client.9942 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9e1836740 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x56302ae1ea70 0x56302ae1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9e1836740 4 client.9942 mds_command: new command op to 9750 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9e1836740 1 -- 172.21.15.19:0/510116614 --> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56302ad49060 con 0x56302ae1ea70 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.763+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x56302ae1ea70 0x56302ae1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x56302ae1ea70 0x56302ae1ee50 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9750 client_cookie=292c9bc83a9d0d58 server_cookie=8777638140701cc9 in_seq=0 out_seq=0 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9b6ffd640 10 client.9942 ms_handle_connect on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/510116614 <== mds.9750 v2:172.21.15.19:6834/525024117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56302ad49060 con 0x56302ae1ea70 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9b6ffd640 10 client.9942 handle_command_reply: tid=0 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9e1836740 1 client.9942 shutdown 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9e1836740 1 -- 172.21.15.19:0/510116614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302ae003c0 msgr2=0x56302ae06890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9e1836740 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302ae003c0 0x56302ae06890 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8005880 tx=0x7fa9c8007800 comp rx=0 tx=0).stop 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9e1836740 1 -- 172.21.15.19:0/510116614 shutdown_connections 2024-04-25T10:02:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9e1836740 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x56302ae1ea70 0x56302ae1ee50 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9e1836740 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfc7f0 0x56302adf8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9e1836740 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302ae003c0 0x56302ae06890 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9b6ffd640 0 client.9942 ms_handle_reset on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9e1836740 1 --2- 172.21.15.19:0/510116614 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302adfffe0 0x56302ae314c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9e1836740 1 -- 172.21.15.19:0/510116614 >> 172.21.15.19:0/510116614 conn(0x56302add2c40 msgr2=0x56302adff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.764+0000 7fa9e1836740 1 -- 172.21.15.19:0/510116614 shutdown_connections 2024-04-25T10:02:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.765+0000 7fa9e1836740 1 -- 172.21.15.19:0/510116614 wait complete. 2024-04-25T10:02:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.765+0000 7fa9e1836740 20 client.9942 trim_cache size 0 max 16384 2024-04-25T10:02:55.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.780+0000 7fa9e1836740 1 Processor -- start 2024-04-25T10:02:55.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.780+0000 7fa9e1836740 1 -- start start 2024-04-25T10:02:55.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.780+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfc7f0 0x56302ae21250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.780+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfffe0 0x56302ae21790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.780+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.780+0000 7fa9e1836740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d81ffc70 con 0x56302adfffe0 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.780+0000 7fa9e1836740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d82002d0 con 0x56302adfc7f0 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.780+0000 7fa9e1836740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d81fe8b0 con 0x56302ae003c0 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfffe0 0x56302ae21790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfffe0 0x56302ae21790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43694/0 (socket says 172.21.15.19:43694) 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9d77fe640 1 -- 172.21.15.19:0/2433733371 learned_addr learned my addr 172.21.15.19:0/2433733371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9dcff5640 1 --2- 172.21.15.19:0/2433733371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9d77fe640 1 -- 172.21.15.19:0/2433733371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 msgr2=0x56302ae1ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/2433733371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae1ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9d77fe640 1 -- 172.21.15.19:0/2433733371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfc7f0 msgr2=0x56302ae21250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/2433733371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfc7f0 0x56302ae21250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9d77fe640 1 -- 172.21.15.19:0/2433733371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56302ae03ab0 con 0x56302adfffe0 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/2433733371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfffe0 0x56302ae21790 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8007de0 tx=0x7fa9c800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95e7d688f8018c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9bd7fa640 1 -- 172.21.15.19:0/2433733371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8029020 con 0x56302adfffe0 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9bd7fa640 1 -- 172.21.15.19:0/2433733371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c8019070 con 0x56302adfffe0 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.781+0000 7fa9bd7fa640 1 -- 172.21.15.19:0/2433733371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8001bb0 con 0x56302adfffe0 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.782+0000 7fa9e1836740 1 -- 172.21.15.19:0/2433733371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfffe0 msgr2=0x56302ae21790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.782+0000 7fa9e1836740 1 --2- 172.21.15.19:0/2433733371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfffe0 0x56302ae21790 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8007de0 tx=0x7fa9c800e000 comp rx=0 tx=0).stop 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.782+0000 7fa9e1836740 1 -- 172.21.15.19:0/2433733371 shutdown_connections 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.782+0000 7fa9e1836740 1 --2- 172.21.15.19:0/2433733371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae1ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.782+0000 7fa9e1836740 1 --2- 172.21.15.19:0/2433733371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfffe0 0x56302ae21790 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.782+0000 7fa9e1836740 1 --2- 172.21.15.19:0/2433733371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfc7f0 0x56302ae21250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.782+0000 7fa9e1836740 1 -- 172.21.15.19:0/2433733371 >> 172.21.15.19:0/2433733371 conn(0x56302add2c40 msgr2=0x56302add79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.782+0000 7fa9e1836740 1 -- 172.21.15.19:0/2433733371 shutdown_connections 2024-04-25T10:02:55.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.782+0000 7fa9e1836740 1 -- 172.21.15.19:0/2433733371 wait complete. 2024-04-25T10:02:55.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.782+0000 7fa9e1836740 1 -- start start 2024-04-25T10:02:55.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.783+0000 7fa9e1836740 1 Processor -- start 2024-04-25T10:02:55.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.783+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfc7f0 0x56302ae885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.783+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfffe0 0x56302ae2f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.783+0000 7fa9e1836740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae2fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.783+0000 7fa9e1836740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302adf9b50 con 0x56302adfc7f0 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.783+0000 7fa9e1836740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d81feeb0 con 0x56302adfffe0 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.783+0000 7fa9e1836740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56302adfe730 con 0x56302ae003c0 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.783+0000 7fa9e1836740 10 client.0 _ll_register_callbacks cb 0x7fa9d00555b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.783+0000 7fa9d7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfc7f0 0x56302ae885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.783+0000 7fa9d7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfc7f0 0x56302ae885e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43710/0 (socket says 172.21.15.19:43710) 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.783+0000 7fa9d7fff640 1 -- 172.21.15.19:0/805749002 learned_addr learned my addr 172.21.15.19:0/805749002 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfffe0 0x56302ae2f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9dcff5640 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae2fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9d7fff640 1 -- 172.21.15.19:0/805749002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 msgr2=0x56302ae2fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae2fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9d7fff640 1 -- 172.21.15.19:0/805749002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfffe0 msgr2=0x56302ae2f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfffe0 0x56302ae2f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9d7fff640 1 -- 172.21.15.19:0/805749002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56302ae37840 con 0x56302adfc7f0 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9dcff5640 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae2fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9d7fff640 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfc7f0 0x56302ae885e0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc035040 tx=0x7fa9cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533cd59130e04ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/805749002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc037040 con 0x56302adfc7f0 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/805749002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa9cc04e250 con 0x56302adfc7f0 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9e1836740 1 -- 172.21.15.19:0/805749002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9c8019070 con 0x56302adfc7f0 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/805749002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9cc05d050 con 0x56302adfc7f0 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9e1836740 10 client.9948 fetch_fsmap learned FSMap version 94 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9e1836740 1 -- 172.21.15.19:0/805749002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x56302ae2cd30 con 0x56302adfc7f0 2024-04-25T10:02:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/805749002 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c8019070 con 0x56302adfc7f0 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/805749002 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 94) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fa9cc060020 con 0x56302adfc7f0 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9e1836740 10 client.9948 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9e1836740 10 client.9948 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.784+0000 7fa9e1836740 20 client.9948 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x56302ae2c7b0 0x56302ae8c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 4 client.9948 mds_command: new command op to 9750 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 -- 172.21.15.19:0/805749002 --> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] -- command(tid 0: {"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}) v1 -- 0x56302ad433b0 con 0x56302ae2c7b0 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x56302ae2c7b0 0x56302ae8c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9d77fe640 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x56302ae2c7b0 0x56302ae8c180 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9750 client_cookie=5f49b096c0abb8fd server_cookie=8523cf8ecf923ecd in_seq=0 out_seq=0 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9b6ffd640 10 client.9948 ms_handle_connect on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9b6ffd640 1 -- 172.21.15.19:0/805749002 <== mds.9750 v2:172.21.15.19:6834/525024117 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x56302ad433b0 con 0x56302ae2c7b0 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9b6ffd640 10 client.9948 handle_command_reply: tid=0 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 client.9948 shutdown 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 -- 172.21.15.19:0/805749002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfc7f0 msgr2=0x56302ae885e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfc7f0 0x56302ae885e0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc035040 tx=0x7fa9cc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 -- 172.21.15.19:0/805749002 shutdown_connections 2024-04-25T10:02:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x56302ae2c7b0 0x56302ae8c180 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56302ae003c0 0x56302ae2fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9b6ffd640 0 client.9948 ms_handle_reset on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56302adfffe0 0x56302ae2f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 --2- 172.21.15.19:0/805749002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56302adfc7f0 0x56302ae885e0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 -- 172.21.15.19:0/805749002 >> 172.21.15.19:0/805749002 conn(0x56302add2c40 msgr2=0x56302ae1ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.785+0000 7fa9e1836740 1 -- 172.21.15.19:0/805749002 shutdown_connections 2024-04-25T10:02:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.786+0000 7fa9e1836740 1 -- 172.21.15.19:0/805749002 wait complete. 2024-04-25T10:02:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.786+0000 7fa9e1836740 20 client.9948 trim_cache size 0 max 16384 2024-04-25T10:02:55.796 INFO:teuthology.orchestra.run.smithi019.stderr:Error EINVAL: MDS is not active 2024-04-25T10:02:55.797 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:02:55.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub pause 2024-04-25T10:02:55.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.895+0000 7fcbba03a640 1 Processor -- start 2024-04-25T10:02:55.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.896+0000 7fcbba03a640 1 -- start start 2024-04-25T10:02:55.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.896+0000 7fcbba03a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbb40cd4c0 0x7fcbb40cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.896+0000 7fcbba03a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcbb415b1c0 0x7fcbb415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.896+0000 7fcbba03a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcbb415e1a0 0x7fcbb4150b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.896+0000 7fcbba03a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb415b620 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.896+0000 7fcbba03a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb415e6e0 con 0x7fcbb415e1a0 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.896+0000 7fcbba03a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb415e9c0 con 0x7fcbb415b1c0 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.896+0000 7fcbb37fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbb40cd4c0 0x7fcbb40cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.897+0000 7fcbb37fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbb40cd4c0 0x7fcbb40cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43722/0 (socket says 172.21.15.19:43722) 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.897+0000 7fcbb37fe640 1 -- 172.21.15.19:0/3770561369 learned_addr learned my addr 172.21.15.19:0/3770561369 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.897+0000 7fcbb3fff640 1 --2- 172.21.15.19:0/3770561369 >> v2:172.21.15.112:3300/0 conn(0x7fcbb415e1a0 0x7fcbb4150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.897+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/3770561369 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcbb415b1c0 0x7fcbb415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.897+0000 7fcbb37fe640 1 -- 172.21.15.19:0/3770561369 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcbb415b1c0 msgr2=0x7fcbb415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.897+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/3770561369 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcbb415b1c0 0x7fcbb415dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.897+0000 7fcbb37fe640 1 -- 172.21.15.19:0/3770561369 >> v2:172.21.15.112:3300/0 conn(0x7fcbb415e1a0 msgr2=0x7fcbb4150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.897+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/3770561369 >> v2:172.21.15.112:3300/0 conn(0x7fcbb415e1a0 0x7fcbb4150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.897+0000 7fcbb37fe640 1 -- 172.21.15.19:0/3770561369 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb415a9c0 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.897+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/3770561369 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcbb415b1c0 0x7fcbb415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.897+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/3770561369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40cd4c0 0x7fcbb40cd8a0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40af3a0 tx=0x7fcba402b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca5bc2717340cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbb27fc640 1 -- 172.21.15.19:0/3770561369 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba400b040 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbb27fc640 1 -- 172.21.15.19:0/3770561369 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba4030710 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbb27fc640 1 -- 172.21.15.19:0/3770561369 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4030ae0 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbba03a640 1 -- 172.21.15.19:0/3770561369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40cd4c0 msgr2=0x7fcbb40cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbba03a640 1 --2- 172.21.15.19:0/3770561369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40cd4c0 0x7fcbb40cd8a0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40af3a0 tx=0x7fcba402b5a0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbba03a640 1 -- 172.21.15.19:0/3770561369 shutdown_connections 2024-04-25T10:02:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbba03a640 1 --2- 172.21.15.19:0/3770561369 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcbb415b1c0 0x7fcbb415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbba03a640 1 --2- 172.21.15.19:0/3770561369 >> v2:172.21.15.112:3300/0 conn(0x7fcbb415e1a0 0x7fcbb4150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbba03a640 1 --2- 172.21.15.19:0/3770561369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40cd4c0 0x7fcbb40cd8a0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbba03a640 1 -- 172.21.15.19:0/3770561369 >> 172.21.15.19:0/3770561369 conn(0x7fcbb406abe0 msgr2=0x7fcbb406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbba03a640 1 -- 172.21.15.19:0/3770561369 shutdown_connections 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.898+0000 7fcbba03a640 1 -- 172.21.15.19:0/3770561369 wait complete. 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.899+0000 7fcbba03a640 1 Processor -- start 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.900+0000 7fcbba03a640 1 -- start start 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.900+0000 7fcbba03a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40cd4c0 0x7fcbb4141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.900+0000 7fcbba03a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcbb415b1c0 0x7fcbb4152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.900+0000 7fcbba03a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb415e1a0 0x7fcbb415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.900+0000 7fcbba03a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb406eca0 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.900+0000 7fcbba03a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb415a670 con 0x7fcbb415e1a0 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.900+0000 7fcbba03a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb406ee20 con 0x7fcbb415b1c0 2024-04-25T10:02:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.900+0000 7fcbb37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40cd4c0 0x7fcbb4141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.900+0000 7fcbb37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40cd4c0 0x7fcbb4141ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43724/0 (socket says 172.21.15.19:43724) 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.900+0000 7fcbb37fe640 1 -- 172.21.15.19:0/418194923 learned_addr learned my addr 172.21.15.19:0/418194923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcbb3fff640 1 --2- 172.21.15.19:0/418194923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb415e1a0 0x7fcbb415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/418194923 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcbb415b1c0 0x7fcbb4152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcbb37fe640 1 -- 172.21.15.19:0/418194923 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcbb415b1c0 msgr2=0x7fcbb4152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/418194923 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcbb415b1c0 0x7fcbb4152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcbb37fe640 1 -- 172.21.15.19:0/418194923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb415e1a0 msgr2=0x7fcbb415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/418194923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb415e1a0 0x7fcbb415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcbb37fe640 1 -- 172.21.15.19:0/418194923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb4154090 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/418194923 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcbb415b1c0 0x7fcbb4152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/418194923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb40cd4c0 0x7fcbb4141ef0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fcba400a1c0 tx=0x7fcba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcb9bfff640 1 -- 172.21.15.19:0/418194923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba403f020 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcb9bfff640 1 -- 172.21.15.19:0/418194923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba4030e30 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcbba03a640 1 -- 172.21.15.19:0/418194923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb4154370 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcb9bfff640 1 -- 172.21.15.19:0/418194923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba403b430 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.901+0000 7fcbba03a640 1 -- 172.21.15.19:0/418194923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb4160490 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.903+0000 7fcb9bfff640 1 -- 172.21.15.19:0/418194923 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcba4042020 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.903+0000 7fcb9bfff640 1 --2- 172.21.15.19:0/418194923 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcba006b4d0 0x7fcba006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.903+0000 7fcb9bfff640 1 -- 172.21.15.19:0/418194923 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcba40aac30 con 0x7fcbb40cd4c0 2024-04-25T10:02:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.904+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/418194923 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcba006b4d0 0x7fcba006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.904+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/418194923 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcba006b4d0 0x7fcba006d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4144fa0 tx=0x7fcba8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.904+0000 7fcbbce0e740 1 Processor -- start 2024-04-25T10:02:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.904+0000 7fcbbce0e740 1 -- start start 2024-04-25T10:02:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.905+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d12672200 0x558d126725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.905+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1268dc40 0x558d12647220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.905+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d126448c0 0x558d126465e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.905+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d125aa9b0 con 0x558d126448c0 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.905+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d12553c60 con 0x558d1268dc40 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.905+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d125a6a50 con 0x558d12672200 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.905+0000 7fcbb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d126448c0 0x558d126465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.905+0000 7fcbb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d126448c0 0x558d126465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43730/0 (socket says 172.21.15.19:43730) 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.905+0000 7fcbb2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1268dc40 0x558d12647220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.905+0000 7fcbb3fff640 1 -- 172.21.15.19:0/1215124855 learned_addr learned my addr 172.21.15.19:0/1215124855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.905+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/1215124855 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d12672200 0x558d126725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.906+0000 7fcbb3fff640 1 -- 172.21.15.19:0/1215124855 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d12672200 msgr2=0x558d126725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.906+0000 7fcbb3fff640 1 --2- 172.21.15.19:0/1215124855 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d12672200 0x558d126725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.906+0000 7fcbb3fff640 1 -- 172.21.15.19:0/1215124855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1268dc40 msgr2=0x558d12647220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.906+0000 7fcbb3fff640 1 --2- 172.21.15.19:0/1215124855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1268dc40 0x558d12647220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.906+0000 7fcbb3fff640 1 -- 172.21.15.19:0/1215124855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d12547b30 con 0x558d126448c0 2024-04-25T10:02:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.906+0000 7fcbb3fff640 1 --2- 172.21.15.19:0/1215124855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d126448c0 0x558d126465e0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009ee0 tx=0x7fcbac003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c935b99ca78b7e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.906+0000 7fcb997fa640 1 -- 172.21.15.19:0/1215124855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0047a0 con 0x558d126448c0 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.906+0000 7fcb997fa640 1 -- 172.21.15.19:0/1215124855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac01a070 con 0x558d126448c0 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.906+0000 7fcb997fa640 1 -- 172.21.15.19:0/1215124855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac015620 con 0x558d126448c0 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.906+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1215124855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d126448c0 msgr2=0x558d126465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.906+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/1215124855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d126448c0 0x558d126465e0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009ee0 tx=0x7fcbac003d60 comp rx=0 tx=0).stop 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.907+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1215124855 shutdown_connections 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.907+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/1215124855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d126448c0 0x558d126465e0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.907+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/1215124855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1268dc40 0x558d12647220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.907+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/1215124855 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d12672200 0x558d126725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.907+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1215124855 >> 172.21.15.19:0/1215124855 conn(0x558d1260f270 msgr2=0x558d1268d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.907+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1215124855 shutdown_connections 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.907+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1215124855 wait complete. 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.907+0000 7fcbbce0e740 1 -- start start 2024-04-25T10:02:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.907+0000 7fcbbce0e740 1 Processor -- start 2024-04-25T10:02:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d12672200 0x558d1265e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ce60 0x558d1266d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ae50 0x558d1266b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d12647760 con 0x558d1266ce60 2024-04-25T10:02:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d12567200 con 0x558d12672200 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d12643ca0 con 0x558d1266ae50 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbbce0e740 10 client.0 _ll_register_callbacks cb 0x558d12566bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbb2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ce60 0x558d1266d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbb2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ce60 0x558d1266d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43742/0 (socket says 172.21.15.19:43742) 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbb2ffd640 1 -- 172.21.15.19:0/2749274664 learned_addr learned my addr 172.21.15.19:0/2749274664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbb3fff640 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ae50 0x558d1266b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.908+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d12672200 0x558d1265e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbb2ffd640 1 -- 172.21.15.19:0/2749274664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ae50 msgr2=0x558d1266b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ae50 0x558d1266b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbb2ffd640 1 -- 172.21.15.19:0/2749274664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d12672200 msgr2=0x558d1265e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d12672200 0x558d1265e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbb2ffd640 1 -- 172.21.15.19:0/2749274664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d12389500 con 0x558d1266ce60 2024-04-25T10:02:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ce60 0x558d1266d240 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fcba8014740 tx=0x7fcba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24a29dc5cdf67d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcb927fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcb927fc640 1 -- 172.21.15.19:0/2749274664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8025070 con 0x558d1266ce60 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcb927fc640 1 -- 172.21.15.19:0/2749274664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcba8015a60 con 0x558d1266ce60 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbbce0e740 1 -- 172.21.15.19:0/2749274664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbac01a070 con 0x558d1266ce60 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcb927fc640 1 -- 172.21.15.19:0/2749274664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba8030050 con 0x558d1266ce60 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbbce0e740 10 client.9960 fetch_fsmap learned FSMap version 94 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbbce0e740 1 -- 172.21.15.19:0/2749274664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x558d123e9c90 con 0x558d1266ce60 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcb927fc640 1 -- 172.21.15.19:0/2749274664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac01a070 con 0x558d1266ce60 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcb927fc640 1 -- 172.21.15.19:0/2749274664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 94) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fcba8033020 con 0x558d1266ce60 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbbce0e740 10 client.9960 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbbce0e740 10 client.9960 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbbce0e740 20 client.9960 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x558d1268fb60 0x558d1268ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbbce0e740 4 client.9960 mds_command: new command op to 9750 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.909+0000 7fcbbce0e740 1 -- 172.21.15.19:0/2749274664 --> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d125b7ef0 con 0x558d1268fb60 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.910+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x558d1268fb60 0x558d1268ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.910+0000 7fcb927fc640 10 client.9960 ms_handle_connect on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.910+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x558d1268fb60 0x558d1268ff40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9750 client_cookie=e7e2bef18f44864d server_cookie=f0c8f3656ea714f6 in_seq=0 out_seq=0 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.910+0000 7fcb927fc640 1 -- 172.21.15.19:0/2749274664 <== mds.9750 v2:172.21.15.19:6834/525024117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d125b7ef0 con 0x558d1268fb60 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.910+0000 7fcb927fc640 10 client.9960 handle_command_reply: tid=0 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.910+0000 7fcbbce0e740 1 client.9960 shutdown 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.910+0000 7fcbbce0e740 1 -- 172.21.15.19:0/2749274664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ce60 msgr2=0x558d1266d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.911+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ce60 0x558d1266d240 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fcba8014740 tx=0x7fcba80046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.911+0000 7fcbbce0e740 1 -- 172.21.15.19:0/2749274664 shutdown_connections 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.911+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x558d1268fb60 0x558d1268ff40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.911+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ae50 0x558d1266b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.911+0000 7fcb927fc640 0 client.9960 ms_handle_reset on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.911+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ce60 0x558d1266d240 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.911+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/2749274664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d12672200 0x558d1265e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.911+0000 7fcbbce0e740 1 -- 172.21.15.19:0/2749274664 >> 172.21.15.19:0/2749274664 conn(0x558d1260f270 msgr2=0x558d12642cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.911+0000 7fcbbce0e740 1 -- 172.21.15.19:0/2749274664 shutdown_connections 2024-04-25T10:02:55.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.911+0000 7fcbbce0e740 1 -- 172.21.15.19:0/2749274664 wait complete. 2024-04-25T10:02:55.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.911+0000 7fcbbce0e740 20 client.9960 trim_cache size 0 max 16384 2024-04-25T10:02:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.925+0000 7fcbbce0e740 1 Processor -- start 2024-04-25T10:02:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.925+0000 7fcbbce0e740 1 -- start start 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.926+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ae50 0x558d1269f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.926+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ce60 0x558d12669480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.926+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d12672200 0x558d126699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.926+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb4200630 con 0x558d1266ae50 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.926+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb415f6d0 con 0x558d12672200 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.926+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb41ff320 con 0x558d1266ce60 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.926+0000 7fcbb37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ae50 0x558d1269f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.926+0000 7fcbb37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ae50 0x558d1269f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43758/0 (socket says 172.21.15.19:43758) 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.926+0000 7fcbb37fe640 1 -- 172.21.15.19:0/11004702 learned_addr learned my addr 172.21.15.19:0/11004702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/11004702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ce60 0x558d12669480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcbb3fff640 1 --2- 172.21.15.19:0/11004702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d12672200 0x558d126699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcbb37fe640 1 -- 172.21.15.19:0/11004702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ce60 msgr2=0x558d12669480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/11004702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ce60 0x558d12669480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcbb37fe640 1 -- 172.21.15.19:0/11004702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d12672200 msgr2=0x558d126699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/11004702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d12672200 0x558d126699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcbb37fe640 1 -- 172.21.15.19:0/11004702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d1227ee20 con 0x558d1266ae50 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcbb3fff640 1 --2- 172.21.15.19:0/11004702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d12672200 0x558d126699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/11004702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ae50 0x558d1269f9d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fcba4078ef0 tx=0x7fcba400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca293c1f09b29dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcb98ff9640 1 -- 172.21.15.19:0/11004702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba407d0b0 con 0x558d1266ae50 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcb98ff9640 1 -- 172.21.15.19:0/11004702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba4077290 con 0x558d1266ae50 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcb98ff9640 1 -- 172.21.15.19:0/11004702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4075860 con 0x558d1266ae50 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcbbce0e740 1 -- 172.21.15.19:0/11004702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ae50 msgr2=0x558d1269f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.927+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/11004702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ae50 0x558d1269f9d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fcba4078ef0 tx=0x7fcba400c040 comp rx=0 tx=0).stop 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.928+0000 7fcbbce0e740 1 -- 172.21.15.19:0/11004702 shutdown_connections 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.928+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/11004702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d12672200 0x558d126699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.928+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/11004702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ce60 0x558d12669480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.928+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/11004702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d1266ae50 0x558d1269f9d0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.928+0000 7fcbbce0e740 1 -- 172.21.15.19:0/11004702 >> 172.21.15.19:0/11004702 conn(0x558d1260f270 msgr2=0x558d1263d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.928+0000 7fcbbce0e740 1 -- 172.21.15.19:0/11004702 shutdown_connections 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.928+0000 7fcbbce0e740 1 -- 172.21.15.19:0/11004702 wait complete. 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.928+0000 7fcbbce0e740 1 -- start start 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.928+0000 7fcbbce0e740 1 Processor -- start 2024-04-25T10:02:55.939 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:02:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ae50 0x558d1268ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1266ce60 0x558d126f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbbce0e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d12672200 0x558d126f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d126739c0 con 0x558d12672200 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1266e490 con 0x558d1266ce60 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbbce0e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d1266f310 con 0x558d1266ae50 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbbce0e740 10 client.0 _ll_register_callbacks cb 0x7fcba800ff00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d12672200 0x558d126f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbb3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d12672200 0x558d126f80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43762/0 (socket says 172.21.15.19:43762) 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbb3fff640 1 -- 172.21.15.19:0/1909455883 learned_addr learned my addr 172.21.15.19:0/1909455883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1266ce60 0x558d126f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ae50 0x558d1268ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbb3fff640 1 -- 172.21.15.19:0/1909455883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ae50 msgr2=0x558d1268ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbb3fff640 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ae50 0x558d1268ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbb3fff640 1 -- 172.21.15.19:0/1909455883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1266ce60 msgr2=0x558d126f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbb3fff640 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1266ce60 0x558d126f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbb3fff640 1 -- 172.21.15.19:0/1909455883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d12565c40 con 0x558d12672200 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.929+0000 7fcbb2ffd640 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1266ce60 0x558d126f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcbb3fff640 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d12672200 0x558d126f80d0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009e80 tx=0x7fcbac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a482c0d17679a6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcb927fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcb927fc640 1 -- 172.21.15.19:0/1909455883 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac028430 con 0x558d12672200 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcb927fc640 1 -- 172.21.15.19:0/1909455883 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcbac015420 con 0x558d12672200 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1909455883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcba4077290 con 0x558d12672200 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcb927fc640 1 -- 172.21.15.19:0/1909455883 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbac030050 con 0x558d12672200 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcbbce0e740 10 client.9966 fetch_fsmap learned FSMap version 94 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1909455883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x558d12571980 con 0x558d12672200 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcb927fc640 1 -- 172.21.15.19:0/1909455883 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba4077290 con 0x558d12672200 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcb927fc640 1 -- 172.21.15.19:0/1909455883 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 94) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fcbac033020 con 0x558d12672200 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcbbce0e740 10 client.9966 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcbbce0e740 10 client.9966 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcbbce0e740 20 client.9966 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x558d12672ea0 0x558d12673280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcbbce0e740 4 client.9966 mds_command: new command op to 9750 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1909455883 --> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x558d125b2240 con 0x558d12672ea0 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.930+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x558d12672ea0 0x558d12673280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcb927fc640 10 client.9966 ms_handle_connect on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbb37fe640 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x558d12672ea0 0x558d12673280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9750 client_cookie=27d92f91281f9f16 server_cookie=6efd9c3c5c43d915 in_seq=0 out_seq=0 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcb927fc640 1 -- 172.21.15.19:0/1909455883 <== mds.9750 v2:172.21.15.19:6834/525024117 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x558d125b2240 con 0x558d12672ea0 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcb927fc640 10 client.9966 handle_command_reply: tid=0 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 1 client.9966 shutdown 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1909455883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d12672200 msgr2=0x558d126f80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d12672200 0x558d126f80d0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009e80 tx=0x7fcbac004ab0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1909455883 shutdown_connections 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x558d12672ea0 0x558d12673280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558d12672200 0x558d126f80d0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1266ce60 0x558d126f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcb927fc640 0 client.9966 ms_handle_reset on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 1 --2- 172.21.15.19:0/1909455883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d1266ae50 0x558d1268ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1909455883 >> 172.21.15.19:0/1909455883 conn(0x558d1260f270 msgr2=0x558d1265e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1909455883 shutdown_connections 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 1 -- 172.21.15.19:0/1909455883 wait complete. 2024-04-25T10:02:55.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:55.931+0000 7fcbbce0e740 20 client.9966 trim_cache size 0 max 16384 2024-04-25T10:02:55.944 INFO:teuthology.orchestra.run.smithi019.stderr:Error EINVAL: MDS is not active 2024-04-25T10:02:55.944 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub abort 2024-04-25T10:02:56.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.042+0000 7f5d9b310640 1 Processor -- start 2024-04-25T10:02:56.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.042+0000 7f5d9b310640 1 -- start start 2024-04-25T10:02:56.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d9b310640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d940ce230 0x7f5d940cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d9b310640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d9415b270 0x7f5d9415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d9b310640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d94072280 con 0x7f5d940cc4f0 2024-04-25T10:02:56.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d9b310640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d94071270 con 0x7f5d940ce230 2024-04-25T10:02:56.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d9b310640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940714c0 con 0x7f5d9415b270 2024-04-25T10:02:56.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d988ab640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d940ce230 0x7f5d940cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d988ab640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d940ce230 0x7f5d940cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43778/0 (socket says 172.21.15.19:43778) 2024-04-25T10:02:56.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d988ab640 1 -- 172.21.15.19:0/1116435357 learned_addr learned my addr 172.21.15.19:0/1116435357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d998ad640 1 --2- 172.21.15.19:0/1116435357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d9415b270 0x7f5d9415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d988ab640 1 -- 172.21.15.19:0/1116435357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d9415b270 msgr2=0x7f5d9415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d988ab640 1 --2- 172.21.15.19:0/1116435357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d9415b270 0x7f5d9415b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d988ab640 1 -- 172.21.15.19:0/1116435357 >> v1:172.21.15.112:6789/0 conn(0x7f5d940cc4f0 legacy=0x7f5d940cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.043+0000 7f5d988ab640 1 -- 172.21.15.19:0/1116435357 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940717a0 con 0x7f5d940ce230 2024-04-25T10:02:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.044+0000 7f5d988ab640 1 --2- 172.21.15.19:0/1116435357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d940ce230 0x7f5d940cc8d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009870 tx=0x7f5d8802c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14746c257cffd0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.044+0000 7f5d93fff640 1 -- 172.21.15.19:0/1116435357 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88038070 con 0x7f5d940ce230 2024-04-25T10:02:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.044+0000 7f5d93fff640 1 -- 172.21.15.19:0/1116435357 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d88031710 con 0x7f5d940ce230 2024-04-25T10:02:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.044+0000 7f5d93fff640 1 -- 172.21.15.19:0/1116435357 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88031ae0 con 0x7f5d940ce230 2024-04-25T10:02:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.045+0000 7f5d9b310640 1 -- 172.21.15.19:0/1116435357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d940ce230 msgr2=0x7f5d940cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.045+0000 7f5d9b310640 1 --2- 172.21.15.19:0/1116435357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d940ce230 0x7f5d940cc8d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009870 tx=0x7f5d8802c420 comp rx=0 tx=0).stop 2024-04-25T10:02:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.045+0000 7f5d9b310640 1 -- 172.21.15.19:0/1116435357 shutdown_connections 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.045+0000 7f5d9b310640 1 --2- 172.21.15.19:0/1116435357 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d9415b270 0x7f5d9415b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.045+0000 7f5d9b310640 1 --2- 172.21.15.19:0/1116435357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d940ce230 0x7f5d940cc8d0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.045+0000 7f5d9b310640 1 -- 172.21.15.19:0/1116435357 >> 172.21.15.19:0/1116435357 conn(0x7f5d9406abe0 msgr2=0x7f5d9406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.045+0000 7f5d9b310640 1 -- 172.21.15.19:0/1116435357 shutdown_connections 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.045+0000 7f5d9b310640 1 -- 172.21.15.19:0/1116435357 wait complete. 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.046+0000 7f5d9b310640 1 Processor -- start 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.046+0000 7f5d9b310640 1 -- start start 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d9b310640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d940cc4f0 0x7f5d9413bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d9b310640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d940ce230 0x7f5d941599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d9b310640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d9415b270 0x7f5d94159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d9b310640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9415bbf0 con 0x7f5d9415b270 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d9b310640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94071d60 con 0x7f5d940ce230 2024-04-25T10:02:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d9b310640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9415bd70 con 0x7f5d940cc4f0 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d998ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d9415b270 0x7f5d94159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d998ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d9415b270 0x7f5d94159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43788/0 (socket says 172.21.15.19:43788) 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d998ad640 1 -- 172.21.15.19:0/2795663174 learned_addr learned my addr 172.21.15.19:0/2795663174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d988ab640 1 --2- 172.21.15.19:0/2795663174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d940ce230 0x7f5d941599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d990ac640 1 --2- 172.21.15.19:0/2795663174 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d940cc4f0 0x7f5d9413bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d998ad640 1 -- 172.21.15.19:0/2795663174 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d940cc4f0 msgr2=0x7f5d9413bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d998ad640 1 --2- 172.21.15.19:0/2795663174 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d940cc4f0 0x7f5d9413bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d998ad640 1 -- 172.21.15.19:0/2795663174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d940ce230 msgr2=0x7f5d941599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d998ad640 1 --2- 172.21.15.19:0/2795663174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d940ce230 0x7f5d941599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d998ad640 1 -- 172.21.15.19:0/2795663174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9414c090 con 0x7f5d9415b270 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d988ab640 1 --2- 172.21.15.19:0/2795663174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d940ce230 0x7f5d941599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d990ac640 1 --2- 172.21.15.19:0/2795663174 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d940cc4f0 0x7f5d9413bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.047+0000 7f5d998ad640 1 --2- 172.21.15.19:0/2795663174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d9415b270 0x7f5d94159ee0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f5d84007e70 tx=0x7f5d8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.048+0000 7f5d91ffb640 1 -- 172.21.15.19:0/2795663174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84014620 con 0x7f5d9415b270 2024-04-25T10:02:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.048+0000 7f5d91ffb640 1 -- 172.21.15.19:0/2795663174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84019070 con 0x7f5d9415b270 2024-04-25T10:02:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.048+0000 7f5d9b310640 1 -- 172.21.15.19:0/2795663174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9414e3a0 con 0x7f5d9415b270 2024-04-25T10:02:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.048+0000 7f5d91ffb640 1 -- 172.21.15.19:0/2795663174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84014a20 con 0x7f5d9415b270 2024-04-25T10:02:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.048+0000 7f5d9b310640 1 -- 172.21.15.19:0/2795663174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9414e960 con 0x7f5d9415b270 2024-04-25T10:02:56.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.049+0000 7f5d91ffb640 1 -- 172.21.15.19:0/2795663174 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d84014bc0 con 0x7f5d9415b270 2024-04-25T10:02:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.050+0000 7f5d91ffb640 1 --2- 172.21.15.19:0/2795663174 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d8006b4d0 0x7f5d8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.050+0000 7f5d91ffb640 1 -- 172.21.15.19:0/2795663174 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5d8408ad00 con 0x7f5d9415b270 2024-04-25T10:02:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.050+0000 7f5d990ac640 1 --2- 172.21.15.19:0/2795663174 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d8006b4d0 0x7f5d8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.050+0000 7f5d990ac640 1 --2- 172.21.15.19:0/2795663174 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d8006b4d0 0x7f5d8006d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00a830 tx=0x7f5d8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.051+0000 7f5d9e00e740 1 Processor -- start 2024-04-25T10:02:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.051+0000 7f5d9e00e740 1 -- start start 2024-04-25T10:02:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4d6200 0x5579be4d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4f1c40 0x5579be4ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4a88c0 0x5579be4aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579be40e9b0 con 0x5579be4a88c0 2024-04-25T10:02:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579be3b7c60 con 0x5579be4d6200 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579be40aa50 con 0x5579be4f1c40 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d998ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4a88c0 0x5579be4aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d998ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4a88c0 0x5579be4aa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43804/0 (socket says 172.21.15.19:43804) 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d998ad640 1 -- 172.21.15.19:0/2139916066 learned_addr learned my addr 172.21.15.19:0/2139916066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d990ac640 1 --2- 172.21.15.19:0/2139916066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4d6200 0x5579be4d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d988ab640 1 --2- 172.21.15.19:0/2139916066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4f1c40 0x5579be4ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d998ad640 1 -- 172.21.15.19:0/2139916066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4f1c40 msgr2=0x5579be4ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d998ad640 1 --2- 172.21.15.19:0/2139916066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4f1c40 0x5579be4ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d998ad640 1 -- 172.21.15.19:0/2139916066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4d6200 msgr2=0x5579be4d65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d998ad640 1 --2- 172.21.15.19:0/2139916066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4d6200 0x5579be4d65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d998ad640 1 -- 172.21.15.19:0/2139916066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579be3abb30 con 0x5579be4a88c0 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d998ad640 1 --2- 172.21.15.19:0/2139916066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4a88c0 0x5579be4aa5e0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f5d840614a0 tx=0x7f5d84059230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695e26baa30a55bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d7affd640 1 -- 172.21.15.19:0/2139916066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8405d170 con 0x5579be4a88c0 2024-04-25T10:02:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d7affd640 1 -- 172.21.15.19:0/2139916066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d840619d0 con 0x5579be4a88c0 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.052+0000 7f5d7affd640 1 -- 172.21.15.19:0/2139916066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840558a0 con 0x5579be4a88c0 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.053+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2139916066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4a88c0 msgr2=0x5579be4aa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.053+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2139916066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4a88c0 0x5579be4aa5e0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f5d840614a0 tx=0x7f5d84059230 comp rx=0 tx=0).stop 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.053+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2139916066 shutdown_connections 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.053+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2139916066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4a88c0 0x5579be4aa5e0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.053+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2139916066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4f1c40 0x5579be4ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.053+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2139916066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4d6200 0x5579be4d65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.053+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2139916066 >> 172.21.15.19:0/2139916066 conn(0x5579be473270 msgr2=0x5579be4f1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.053+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2139916066 shutdown_connections 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.053+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2139916066 wait complete. 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.054+0000 7f5d9e00e740 1 -- start start 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.054+0000 7f5d9e00e740 1 Processor -- start 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4d6200 0x5579be4f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4ccb20 0x5579be4cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4f3d50 0x5579be4f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579be4ab760 con 0x5579be4d6200 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579be3cb200 con 0x5579be4ccb20 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579be4a7ca0 con 0x5579be4f3d50 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d9e00e740 10 client.0 _ll_register_callbacks cb 0x5579be3cabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d990ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4d6200 0x5579be4f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d990ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4d6200 0x5579be4f0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43814/0 (socket says 172.21.15.19:43814) 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d990ac640 1 -- 172.21.15.19:0/2118763326 learned_addr learned my addr 172.21.15.19:0/2118763326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d998ad640 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4f3d50 0x5579be4f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d988ab640 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4ccb20 0x5579be4cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d990ac640 1 -- 172.21.15.19:0/2118763326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4f3d50 msgr2=0x5579be4f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d990ac640 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4f3d50 0x5579be4f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d990ac640 1 -- 172.21.15.19:0/2118763326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4ccb20 msgr2=0x5579be4cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d990ac640 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4ccb20 0x5579be4cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d990ac640 1 -- 172.21.15.19:0/2118763326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579be1ed500 con 0x5579be4d6200 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d990ac640 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4d6200 0x5579be4f0310 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00cfc0 tx=0x7f5d8c0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884e1f9ae0c3cb2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d73fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d73fff640 1 -- 172.21.15.19:0/2118763326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c023020 con 0x5579be4d6200 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d73fff640 1 -- 172.21.15.19:0/2118763326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5d8c01f600 con 0x5579be4d6200 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2118763326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d840619d0 con 0x5579be4d6200 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.055+0000 7f5d73fff640 1 -- 172.21.15.19:0/2118763326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d8c005910 con 0x5579be4d6200 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d9e00e740 10 client.9978 fetch_fsmap learned FSMap version 94 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2118763326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x5579be24dc90 con 0x5579be4d6200 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d73fff640 1 -- 172.21.15.19:0/2118763326 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d840619d0 con 0x5579be4d6200 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d73fff640 1 -- 172.21.15.19:0/2118763326 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 94) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f5d8c030080 con 0x5579be4d6200 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d9e00e740 10 client.9978 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d9e00e740 10 client.9978 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d9e00e740 20 client.9978 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5579be4d9310 0x5579be4d71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d9e00e740 4 client.9978 mds_command: new command op to 9750 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2118763326 --> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579be41bef0 con 0x5579be4d9310 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d988ab640 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5579be4d9310 0x5579be4d71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d73fff640 10 client.9978 ms_handle_connect on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.056+0000 7f5d988ab640 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5579be4d9310 0x5579be4d71d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9750 client_cookie=60c791f21960b178 server_cookie=5eda3f0a195a006e in_seq=0 out_seq=0 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d73fff640 1 -- 172.21.15.19:0/2118763326 <== mds.9750 v2:172.21.15.19:6834/525024117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5579be41bef0 con 0x5579be4d9310 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d73fff640 10 client.9978 handle_command_reply: tid=0 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 1 client.9978 shutdown 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2118763326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4d6200 msgr2=0x5579be4f0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4d6200 0x5579be4f0310 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00cfc0 tx=0x7f5d8c0046f0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2118763326 shutdown_connections 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5579be4d9310 0x5579be4d71d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4f3d50 0x5579be4f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d73fff640 0 client.9978 ms_handle_reset on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4ccb20 0x5579be4cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2118763326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4d6200 0x5579be4f0310 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2118763326 >> 172.21.15.19:0/2118763326 conn(0x5579be473270 msgr2=0x5579be4aa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2118763326 shutdown_connections 2024-04-25T10:02:56.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2118763326 wait complete. 2024-04-25T10:02:56.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.057+0000 7f5d9e00e740 20 client.9978 trim_cache size 0 max 16384 2024-04-25T10:02:56.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.072+0000 7f5d9e00e740 1 Processor -- start 2024-04-25T10:02:56.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.072+0000 7f5d9e00e740 1 -- start start 2024-04-25T10:02:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.072+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4ccb20 0x5579be4f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.072+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4d6200 0x5579be4f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4f3d50 0x5579be4cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94200610 con 0x5579be4f3d50 2024-04-25T10:02:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9413d590 con 0x5579be4d6200 2024-04-25T10:02:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941ff2d0 con 0x5579be4ccb20 2024-04-25T10:02:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d998ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4f3d50 0x5579be4cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d998ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4f3d50 0x5579be4cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43816/0 (socket says 172.21.15.19:43816) 2024-04-25T10:02:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d998ad640 1 -- 172.21.15.19:0/2808936457 learned_addr learned my addr 172.21.15.19:0/2808936457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d998ad640 1 -- 172.21.15.19:0/2808936457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4ccb20 msgr2=0x5579be4f0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d998ad640 1 --2- 172.21.15.19:0/2808936457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4ccb20 0x5579be4f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d998ad640 1 -- 172.21.15.19:0/2808936457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4d6200 msgr2=0x5579be4f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d988ab640 1 --2- 172.21.15.19:0/2808936457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4d6200 0x5579be4f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d998ad640 1 --2- 172.21.15.19:0/2808936457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4d6200 0x5579be4f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d998ad640 1 -- 172.21.15.19:0/2808936457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579be0e2e20 con 0x5579be4f3d50 2024-04-25T10:02:56.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d988ab640 1 --2- 172.21.15.19:0/2808936457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4d6200 0x5579be4f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:56.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d998ad640 1 --2- 172.21.15.19:0/2808936457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4f3d50 0x5579be4cd060 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f5d840890c0 tx=0x7f5d84020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0b729f02f93a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d7a7fc640 1 -- 172.21.15.19:0/2808936457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84088070 con 0x5579be4f3d50 2024-04-25T10:02:56.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d7a7fc640 1 -- 172.21.15.19:0/2808936457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8405d170 con 0x5579be4f3d50 2024-04-25T10:02:56.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.073+0000 7f5d7a7fc640 1 -- 172.21.15.19:0/2808936457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840558a0 con 0x5579be4f3d50 2024-04-25T10:02:56.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.074+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2808936457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4f3d50 msgr2=0x5579be4cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.074+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2808936457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4f3d50 0x5579be4cd060 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f5d840890c0 tx=0x7f5d84020000 comp rx=0 tx=0).stop 2024-04-25T10:02:56.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.074+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2808936457 shutdown_connections 2024-04-25T10:02:56.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.074+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2808936457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4f3d50 0x5579be4cd060 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.074+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2808936457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4d6200 0x5579be4f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.074+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/2808936457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4ccb20 0x5579be4f0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.074+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2808936457 >> 172.21.15.19:0/2808936457 conn(0x5579be473270 msgr2=0x5579be4abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.074+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2808936457 shutdown_connections 2024-04-25T10:02:56.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.074+0000 7f5d9e00e740 1 -- 172.21.15.19:0/2808936457 wait complete. 2024-04-25T10:02:56.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.074+0000 7f5d9e00e740 1 -- start start 2024-04-25T10:02:56.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.075+0000 7f5d9e00e740 1 Processor -- start 2024-04-25T10:02:56.087 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:02:56.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.075+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4ccb20 0x5579be55b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.075+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4d6200 0x5579be55bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.075+0000 7f5d9e00e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4f3d50 0x5579be4d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.075+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579be4d9de0 con 0x5579be4ccb20 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.075+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579be5016f0 con 0x5579be4f3d50 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.075+0000 7f5d9e00e740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579be4d9710 con 0x5579be4d6200 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.075+0000 7f5d9e00e740 10 client.0 _ll_register_callbacks cb 0x7f5d8c00af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.075+0000 7f5d990ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4ccb20 0x5579be55b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d990ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4ccb20 0x5579be55b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43818/0 (socket says 172.21.15.19:43818) 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d990ac640 1 -- 172.21.15.19:0/1425746072 learned_addr learned my addr 172.21.15.19:0/1425746072 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d988ab640 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4d6200 0x5579be55bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d998ad640 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4f3d50 0x5579be4d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d990ac640 1 -- 172.21.15.19:0/1425746072 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4d6200 msgr2=0x5579be55bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d990ac640 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4d6200 0x5579be55bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d990ac640 1 -- 172.21.15.19:0/1425746072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4f3d50 msgr2=0x5579be4d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d990ac640 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4f3d50 0x5579be4d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d990ac640 1 -- 172.21.15.19:0/1425746072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579be3c9c40 con 0x5579be4ccb20 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d990ac640 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4ccb20 0x5579be55b750 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c005d60 tx=0x7f5d8c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358510610fd9193c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d73fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d73fff640 1 -- 172.21.15.19:0/1425746072 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c023050 con 0x5579be4ccb20 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d73fff640 1 -- 172.21.15.19:0/1425746072 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5d8c028800 con 0x5579be4ccb20 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d9e00e740 1 -- 172.21.15.19:0/1425746072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d8405d170 con 0x5579be4ccb20 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d73fff640 1 -- 172.21.15.19:0/1425746072 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d8c040050 con 0x5579be4ccb20 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d9e00e740 10 client.9984 fetch_fsmap learned FSMap version 94 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.076+0000 7f5d9e00e740 1 -- 172.21.15.19:0/1425746072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x5579be3d5980 con 0x5579be4ccb20 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d73fff640 1 -- 172.21.15.19:0/1425746072 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8405d170 con 0x5579be4ccb20 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d73fff640 1 -- 172.21.15.19:0/1425746072 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 94) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f5d8c043020 con 0x5579be4ccb20 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 10 client.9984 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 10 client.9984 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 20 client.9984 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5579be4f1280 0x5579be50a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 4 client.9984 mds_command: new command op to 9750 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 1 -- 172.21.15.19:0/1425746072 --> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x5579be4a7350 con 0x5579be4f1280 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d988ab640 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5579be4f1280 0x5579be50a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d73fff640 10 client.9984 ms_handle_connect on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d988ab640 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5579be4f1280 0x5579be50a480 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9750 client_cookie=ee230bb44522446f server_cookie=12f722fdf2586c87 in_seq=0 out_seq=0 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d73fff640 1 -- 172.21.15.19:0/1425746072 <== mds.9750 v2:172.21.15.19:6834/525024117 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x5579be4a7350 con 0x5579be4f1280 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d73fff640 10 client.9984 handle_command_reply: tid=0 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 1 client.9984 shutdown 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 1 -- 172.21.15.19:0/1425746072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4ccb20 msgr2=0x5579be55b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4ccb20 0x5579be55b750 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c005d60 tx=0x7f5d8c011000 comp rx=0 tx=0).stop 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 1 -- 172.21.15.19:0/1425746072 shutdown_connections 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5579be4f1280 0x5579be50a480 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579be4f3d50 0x5579be4d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5579be4d6200 0x5579be55bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d73fff640 0 client.9984 ms_handle_reset on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 1 --2- 172.21.15.19:0/1425746072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579be4ccb20 0x5579be55b750 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.077+0000 7f5d9e00e740 1 -- 172.21.15.19:0/1425746072 >> 172.21.15.19:0/1425746072 conn(0x5579be473270 msgr2=0x5579be4abbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.078+0000 7f5d9e00e740 1 -- 172.21.15.19:0/1425746072 shutdown_connections 2024-04-25T10:02:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.078+0000 7f5d9e00e740 1 -- 172.21.15.19:0/1425746072 wait complete. 2024-04-25T10:02:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.078+0000 7f5d9e00e740 20 client.9984 trim_cache size 0 max 16384 2024-04-25T10:02:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:Error EINVAL: MDS is not active 2024-04-25T10:02:56.091 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub resume 2024-04-25T10:02:56.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.190+0000 7f3ed9cf7640 1 Processor -- start 2024-04-25T10:02:56.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.190+0000 7f3ed9cf7640 1 -- start start 2024-04-25T10:02:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.190+0000 7f3ed9cf7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3ed40d0ab0 0x7f3ed4168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.190+0000 7f3ed9cf7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed40d2420 con 0x7f3ed40d34c0 2024-04-25T10:02:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.190+0000 7f3ed9cf7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed40d2700 con 0x7f3ed415dac0 2024-04-25T10:02:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.190+0000 7f3ed9cf7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed40d3030 con 0x7f3ed40d0ab0 2024-04-25T10:02:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed2ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3ed40d34c0 0x7f3ed40d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47784/0 (socket says 172.21.15.19:47784) 2024-04-25T10:02:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed2ffd640 1 -- 172.21.15.19:0/3348085103 learned_addr learned my addr 172.21.15.19:0/3348085103 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793805260 0 0) 0x7f3ed40d2420 con 0x7f3ed40d34c0 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed3fff640 1 --2- 172.21.15.19:0/3348085103 >> v2:172.21.15.112:3300/0 conn(0x7f3ed40d0ab0 0x7f3ed4168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ebc003520 con 0x7f3ed40d34c0 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3521647221 0 0) 0x7f3ebc003520 con 0x7f3ed40d34c0 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 >> v2:172.21.15.112:3300/0 conn(0x7f3ed40d0ab0 msgr2=0x7f3ed4168670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 --2- 172.21.15.19:0/3348085103 >> v2:172.21.15.112:3300/0 conn(0x7f3ed40d0ab0 0x7f3ed4168670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 >> v1:172.21.15.112:6789/0 conn(0x7f3ed415dac0 legacy=0x7f3ed40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed40d29e0 con 0x7f3ed40d34c0 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f3ec40017f0 con 0x7f3ed40d34c0 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 >> v1:172.21.15.19:6789/0 conn(0x7f3ed40d34c0 legacy=0x7f3ed40d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc0039b0 0x7f3ebc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc004800 0x7f3ebc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ebc0051c0 0x7f3ebc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40d3030 con 0x7f3ebc0039b0 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc00c070 con 0x7f3ebc0051c0 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc00c270 con 0x7f3ebc004800 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.191+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc0039b0 0x7f3ebc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.192+0000 7f3ed3fff640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ebc0051c0 0x7f3ebc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.192+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc004800 0x7f3ebc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.192+0000 7f3ed37fe640 1 -- 172.21.15.19:0/3348085103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc004800 msgr2=0x7f3ebc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.192+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc004800 0x7f3ebc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.192+0000 7f3ed37fe640 1 -- 172.21.15.19:0/3348085103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ebc0051c0 msgr2=0x7f3ebc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.192+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ebc0051c0 0x7f3ebc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.192+0000 7f3ed37fe640 1 -- 172.21.15.19:0/3348085103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc003520 con 0x7f3ebc0039b0 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.192+0000 7f3ed3fff640 1 -- 172.21.15.19:0/3348085103 reap_dead start 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.192+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc004800 0x7f3ebc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.192+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc0039b0 0x7f3ebc003df0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8001750 tx=0x7f3ebc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f925ee0e6dcc0d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.193+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8001370 con 0x7f3ebc0039b0 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.193+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec80052d0 con 0x7f3ebc0039b0 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.193+0000 7f3ed27fc640 1 -- 172.21.15.19:0/3348085103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec802abc0 con 0x7f3ebc0039b0 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.193+0000 7f3ed9cf7640 1 -- 172.21.15.19:0/3348085103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc0039b0 msgr2=0x7f3ebc003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.193+0000 7f3ed9cf7640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc0039b0 0x7f3ebc003df0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8001750 tx=0x7f3ebc005830 comp rx=0 tx=0).stop 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.193+0000 7f3ed9cf7640 1 -- 172.21.15.19:0/3348085103 shutdown_connections 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.193+0000 7f3ed9cf7640 1 --2- 172.21.15.19:0/3348085103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ebc0039b0 0x7f3ebc003df0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.193+0000 7f3ed9cf7640 1 -- 172.21.15.19:0/3348085103 >> 172.21.15.19:0/3348085103 conn(0x7f3ed4061c30 msgr2=0x7f3ed406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.193+0000 7f3ed9cf7640 1 -- 172.21.15.19:0/3348085103 shutdown_connections 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.193+0000 7f3ed9cf7640 1 -- 172.21.15.19:0/3348085103 wait complete. 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.194+0000 7f3ed9cf7640 1 Processor -- start 2024-04-25T10:02:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.195+0000 7f3ed9cf7640 1 -- start start 2024-04-25T10:02:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.195+0000 7f3ed9cf7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc0039b0 0x7f3ed41517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.195+0000 7f3ed9cf7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ed40cd280 0x7f3ed4160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.195+0000 7f3ed9cf7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed4151dd0 0x7f3ed40b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.195+0000 7f3ed9cf7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40b97d0 con 0x7f3ed40cd280 2024-04-25T10:02:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.195+0000 7f3ed9cf7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed415dea0 con 0x7f3ed4151dd0 2024-04-25T10:02:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.195+0000 7f3ed9cf7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40b9950 con 0x7f3ebc0039b0 2024-04-25T10:02:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.195+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ed40cd280 0x7f3ed4160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.195+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ed40cd280 0x7f3ed4160a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43830/0 (socket says 172.21.15.19:43830) 2024-04-25T10:02:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.195+0000 7f3ed2ffd640 1 -- 172.21.15.19:0/1405799021 learned_addr learned my addr 172.21.15.19:0/1405799021 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ed3fff640 1 --2- 172.21.15.19:0/1405799021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed4151dd0 0x7f3ed40b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/1405799021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc0039b0 0x7f3ed41517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ed2ffd640 1 -- 172.21.15.19:0/1405799021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc0039b0 msgr2=0x7f3ed41517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/1405799021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ebc0039b0 0x7f3ed41517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ed2ffd640 1 -- 172.21.15.19:0/1405799021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed4151dd0 msgr2=0x7f3ed40b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/1405799021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ed4151dd0 0x7f3ed40b4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ed2ffd640 1 -- 172.21.15.19:0/1405799021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed40b5a70 con 0x7f3ed40cd280 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/1405799021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ed40cd280 0x7f3ed4160a50 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e780 tx=0x7f3ec400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ebbfff640 1 -- 172.21.15.19:0/1405799021 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40070d0 con 0x7f3ed40cd280 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ebbfff640 1 -- 172.21.15.19:0/1405799021 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4018070 con 0x7f3ed40cd280 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ebbfff640 1 -- 172.21.15.19:0/1405799021 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4007470 con 0x7f3ed40cd280 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ed9cf7640 1 -- 172.21.15.19:0/1405799021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed40b5d50 con 0x7f3ed40cd280 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.196+0000 7f3ed9cf7640 1 -- 172.21.15.19:0/1405799021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed40ccd60 con 0x7f3ed40cd280 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.198+0000 7f3ebbfff640 1 -- 172.21.15.19:0/1405799021 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ec40057a0 con 0x7f3ed40cd280 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.198+0000 7f3ebbfff640 1 --2- 172.21.15.19:0/1405799021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ec006b4d0 0x7f3ec006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.198+0000 7f3ebbfff640 1 -- 172.21.15.19:0/1405799021 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3ec408a120 con 0x7f3ed40cd280 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.198+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/1405799021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ec006b4d0 0x7f3ec006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.199+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/1405799021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3ec006b4d0 0x7f3ec006d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8032000 tx=0x7f3ec802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.199+0000 7f3edc9f1740 1 Processor -- start 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.199+0000 7f3edc9f1740 1 -- start start 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a30c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a302bb90 con 0x5616a30f3760 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a2fd4e40 con 0x5616a3105370 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616a3027c30 con 0x5616a30c8ca0 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43836/0 (socket says 172.21.15.19:43836) 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed3fff640 1 -- 172.21.15.19:0/923374240 learned_addr learned my addr 172.21.15.19:0/923374240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/923374240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a30c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/923374240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed3fff640 1 -- 172.21.15.19:0/923374240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 msgr2=0x5616a30cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed3fff640 1 --2- 172.21.15.19:0/923374240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed3fff640 1 -- 172.21.15.19:0/923374240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 msgr2=0x5616a30c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed3fff640 1 --2- 172.21.15.19:0/923374240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a30c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed3fff640 1 -- 172.21.15.19:0/923374240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616a2e0a7d0 con 0x5616a30f3760 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/923374240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.200+0000 7f3ed3fff640 1 --2- 172.21.15.19:0/923374240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30f3be0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f3ed406cc00 tx=0x7f3ecc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcaadac5375d993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3eb97fa640 1 -- 172.21.15.19:0/923374240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc003b90 con 0x5616a30f3760 2024-04-25T10:02:56.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3eb97fa640 1 -- 172.21.15.19:0/923374240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc01d070 con 0x5616a30f3760 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3eb97fa640 1 -- 172.21.15.19:0/923374240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc012e70 con 0x5616a30f3760 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3edc9f1740 1 -- 172.21.15.19:0/923374240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 msgr2=0x5616a30f3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/923374240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30f3be0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f3ed406cc00 tx=0x7f3ecc009750 comp rx=0 tx=0).stop 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3edc9f1740 1 -- 172.21.15.19:0/923374240 shutdown_connections 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/923374240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30f3be0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/923374240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a30c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/923374240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3edc9f1740 1 -- 172.21.15.19:0/923374240 >> 172.21.15.19:0/923374240 conn(0x5616a3090870 msgr2=0x5616a310e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3edc9f1740 1 -- 172.21.15.19:0/923374240 shutdown_connections 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.201+0000 7f3edc9f1740 1 -- 172.21.15.19:0/923374240 wait complete. 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.202+0000 7f3edc9f1740 1 -- start start 2024-04-25T10:02:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.202+0000 7f3edc9f1740 1 Processor -- start 2024-04-25T10:02:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30dcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30df590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a30dfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a2fe83e0 con 0x5616a30f3760 2024-04-25T10:02:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a30f12b0 con 0x5616a3105370 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616a3122e80 con 0x5616a30c8ca0 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3edc9f1740 10 client.0 _ll_register_callbacks cb 0x5616a2fe7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30df590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30df590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43842/0 (socket says 172.21.15.19:43842) 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3ed2ffd640 1 -- 172.21.15.19:0/1263068292 learned_addr learned my addr 172.21.15.19:0/1263068292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30dcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3ed3fff640 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a30dfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3ed2ffd640 1 -- 172.21.15.19:0/1263068292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 msgr2=0x5616a30dcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30dcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3ed2ffd640 1 -- 172.21.15.19:0/1263068292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 msgr2=0x5616a30dfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a30dfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3ed2ffd640 1 -- 172.21.15.19:0/1263068292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616a2d000f0 con 0x5616a30f3760 2024-04-25T10:02:56.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30df590 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40583e0 tx=0x7f3ec4058f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc399c2be3ab081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.203+0000 7f3eb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3eb27fc640 1 -- 172.21.15.19:0/1263068292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40569e0 con 0x5616a30f3760 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3eb27fc640 1 -- 172.21.15.19:0/1263068292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3ec4054d50 con 0x5616a30f3760 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3edc9f1740 1 -- 172.21.15.19:0/1263068292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ecc01d070 con 0x5616a30f3760 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3eb27fc640 1 -- 172.21.15.19:0/1263068292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ec4075050 con 0x5616a30f3760 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3edc9f1740 10 client.9996 fetch_fsmap learned FSMap version 94 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3edc9f1740 1 -- 172.21.15.19:0/1263068292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x5616a2e6af60 con 0x5616a30f3760 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3eb27fc640 1 -- 172.21.15.19:0/1263068292 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc01d070 con 0x5616a30f3760 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3eb27fc640 1 -- 172.21.15.19:0/1263068292 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 94) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f3ec4057b30 con 0x5616a30f3760 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3edc9f1740 10 client.9996 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3edc9f1740 10 client.9996 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3edc9f1740 20 client.9996 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5616a30f08e0 0x5616a30f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3edc9f1740 4 client.9996 mds_command: new command op to 9750 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3edc9f1740 1 -- 172.21.15.19:0/1263068292 --> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616a30390d0 con 0x5616a30f08e0 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.204+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5616a30f08e0 0x5616a30f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3eb27fc640 10 client.9996 ms_handle_connect on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5616a30f08e0 0x5616a30f3b60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9750 client_cookie=2d64ab54a2c675c9 server_cookie=2141b8e8be9f121e in_seq=0 out_seq=0 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3eb27fc640 1 -- 172.21.15.19:0/1263068292 <== mds.9750 v2:172.21.15.19:6834/525024117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616a30390d0 con 0x5616a30f08e0 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3eb27fc640 10 client.9996 handle_command_reply: tid=0 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 1 client.9996 shutdown 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 1 -- 172.21.15.19:0/1263068292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 msgr2=0x5616a30df590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30df590 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40583e0 tx=0x7f3ec4058f20 comp rx=0 tx=0).stop 2024-04-25T10:02:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 1 -- 172.21.15.19:0/1263068292 shutdown_connections 2024-04-25T10:02:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5616a30f08e0 0x5616a30f3b60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a30dfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30f3760 0x5616a30df590 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3eb27fc640 0 client.9996 ms_handle_reset on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/1263068292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30dcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 1 -- 172.21.15.19:0/1263068292 >> 172.21.15.19:0/1263068292 conn(0x5616a3090870 msgr2=0x5616a30f16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 1 -- 172.21.15.19:0/1263068292 shutdown_connections 2024-04-25T10:02:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 1 -- 172.21.15.19:0/1263068292 wait complete. 2024-04-25T10:02:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.205+0000 7f3edc9f1740 20 client.9996 trim_cache size 0 max 16384 2024-04-25T10:02:56.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.220+0000 7f3edc9f1740 1 Processor -- start 2024-04-25T10:02:56.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.220+0000 7f3edc9f1740 1 -- start start 2024-04-25T10:02:56.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30dbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a30f3760 0x5616a30f5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a3105370 0x5616a30f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed4200c70 con 0x5616a3105370 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed415edd0 con 0x5616a30f3760 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed41ff850 con 0x5616a30c8ca0 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a3105370 0x5616a30f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a3105370 0x5616a30f6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43856/0 (socket says 172.21.15.19:43856) 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3ed3fff640 1 -- 172.21.15.19:0/4193957617 learned_addr learned my addr 172.21.15.19:0/4193957617 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3ed3fff640 1 -- 172.21.15.19:0/4193957617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 msgr2=0x5616a30dbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3ed3fff640 1 --2- 172.21.15.19:0/4193957617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30dbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3ed3fff640 1 -- 172.21.15.19:0/4193957617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a30f3760 msgr2=0x5616a30f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.221+0000 7f3ed3fff640 1 --2- 172.21.15.19:0/4193957617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a30f3760 0x5616a30f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3ed3fff640 1 -- 172.21.15.19:0/4193957617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616a2cd91b0 con 0x5616a3105370 2024-04-25T10:02:56.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3ed3fff640 1 --2- 172.21.15.19:0/4193957617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a3105370 0x5616a30f6a40 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc000f80 tx=0x7f3ecc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc0af8769888698 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3eb8ff9640 1 -- 172.21.15.19:0/4193957617 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc009410 con 0x5616a3105370 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3eb8ff9640 1 -- 172.21.15.19:0/4193957617 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc01d040 con 0x5616a3105370 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3eb8ff9640 1 -- 172.21.15.19:0/4193957617 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc003bb0 con 0x5616a3105370 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3edc9f1740 1 -- 172.21.15.19:0/4193957617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a3105370 msgr2=0x5616a30f6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/4193957617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a3105370 0x5616a30f6a40 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc000f80 tx=0x7f3ecc018000 comp rx=0 tx=0).stop 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3edc9f1740 1 -- 172.21.15.19:0/4193957617 shutdown_connections 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/4193957617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a3105370 0x5616a30f6a40 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/4193957617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a30f3760 0x5616a30f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/4193957617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30c8ca0 0x5616a30dbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.222+0000 7f3edc9f1740 1 -- 172.21.15.19:0/4193957617 >> 172.21.15.19:0/4193957617 conn(0x5616a3090870 msgr2=0x5616a3121b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.223+0000 7f3edc9f1740 1 -- 172.21.15.19:0/4193957617 shutdown_connections 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.223+0000 7f3edc9f1740 1 -- 172.21.15.19:0/4193957617 wait complete. 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.223+0000 7f3edc9f1740 1 -- start start 2024-04-25T10:02:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.223+0000 7f3edc9f1740 1 Processor -- start 2024-04-25T10:02:56.233 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:02:56.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30c8ca0 0x5616a3121b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30f3760 0x5616a3122050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3edc9f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a3178da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a30f0760 con 0x5616a30c8ca0 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a30f6610 con 0x5616a3105370 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3edc9f1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616a30f5830 con 0x5616a30f3760 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3edc9f1740 10 client.0 _ll_register_callbacks cb 0x7f3ec4058240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3ed37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30c8ca0 0x5616a3121b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3ed37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30c8ca0 0x5616a3121b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43860/0 (socket says 172.21.15.19:43860) 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3ed37fe640 1 -- 172.21.15.19:0/3002901325 learned_addr learned my addr 172.21.15.19:0/3002901325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30f3760 0x5616a3122050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3ed3fff640 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a3178da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3ed37fe640 1 -- 172.21.15.19:0/3002901325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30f3760 msgr2=0x5616a3122050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30f3760 0x5616a3122050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3ed37fe640 1 -- 172.21.15.19:0/3002901325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 msgr2=0x5616a3178da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a3178da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3ed37fe640 1 -- 172.21.15.19:0/3002901325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616a3176790 con 0x5616a30c8ca0 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3ed37fe640 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30c8ca0 0x5616a3121b10 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80052e0 tx=0x7f3ec8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987e1d909d97e53b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.224+0000 7f3eb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3eb27fc640 1 -- 172.21.15.19:0/3002901325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec804fe60 con 0x5616a30c8ca0 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3eb27fc640 1 -- 172.21.15.19:0/3002901325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3ec8005410 con 0x5616a30c8ca0 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3edc9f1740 1 -- 172.21.15.19:0/3002901325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ecc01d040 con 0x5616a30c8ca0 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3eb27fc640 1 -- 172.21.15.19:0/3002901325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ec8007040 con 0x5616a30c8ca0 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3edc9f1740 10 client.10002 fetch_fsmap learned FSMap version 94 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3edc9f1740 1 -- 172.21.15.19:0/3002901325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x5616a30db9f0 con 0x5616a30c8ca0 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3eb27fc640 1 -- 172.21.15.19:0/3002901325 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc01d040 con 0x5616a30c8ca0 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3eb27fc640 1 -- 172.21.15.19:0/3002901325 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 94) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f3ec80071e0 con 0x5616a30c8ca0 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3edc9f1740 10 client.10002 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3edc9f1740 10 client.10002 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3edc9f1740 20 client.10002 populate_metadata read hostname 'smithi019' 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5616a31798e0 0x5616a30dec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3edc9f1740 4 client.10002 mds_command: new command op to 9750 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3edc9f1740 1 -- 172.21.15.19:0/3002901325 --> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x5616a3033420 con 0x5616a31798e0 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5616a31798e0 0x5616a30dec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3eb27fc640 10 client.10002 ms_handle_connect on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.225+0000 7f3ed2ffd640 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5616a31798e0 0x5616a30dec80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9750 client_cookie=e15e0a75c48cdfc5 server_cookie=b0fe2376b59ea67e in_seq=0 out_seq=0 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3eb27fc640 1 -- 172.21.15.19:0/3002901325 <== mds.9750 v2:172.21.15.19:6834/525024117 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x5616a3033420 con 0x5616a31798e0 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3eb27fc640 10 client.10002 handle_command_reply: tid=0 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 1 client.10002 shutdown 2024-04-25T10:02:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 1 -- 172.21.15.19:0/3002901325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30c8ca0 msgr2=0x5616a3121b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30c8ca0 0x5616a3121b10 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80052e0 tx=0x7f3ec8001120 comp rx=0 tx=0).stop 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 1 -- 172.21.15.19:0/3002901325 shutdown_connections 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.19:6834/525024117,v1:172.21.15.19:6835/525024117] conn(0x5616a31798e0 0x5616a30dec80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616a3105370 0x5616a3178da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5616a30f3760 0x5616a3122050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3eb27fc640 0 client.10002 ms_handle_reset on v2:172.21.15.19:6834/525024117 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 1 --2- 172.21.15.19:0/3002901325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616a30c8ca0 0x5616a3121b10 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 1 -- 172.21.15.19:0/3002901325 >> 172.21.15.19:0/3002901325 conn(0x5616a3090870 msgr2=0x5616a30dc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 1 -- 172.21.15.19:0/3002901325 shutdown_connections 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 1 -- 172.21.15.19:0/3002901325 wait complete. 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.226+0000 7f3edc9f1740 20 client.10002 trim_cache size 0 max 16384 2024-04-25T10:02:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:Error EINVAL: MDS is not active 2024-04-25T10:02:56.238 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:02:56.238 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:56.275 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:56.315 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:02:56.315 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:02:56.316 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:56.332 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:02:56.332 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:56.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:56.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.523+0000 7f872c84d640 1 Processor -- start 2024-04-25T10:02:56.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.523+0000 7f872c84d640 1 -- start start 2024-04-25T10:02:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f872c84d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f872815dbe0 0x7f87280d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f872c84d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f87280b92a0 0x7f87280d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f872c84d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87280b97a0 con 0x7f87280b2cb0 2024-04-25T10:02:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f872c84d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f872815dfc0 con 0x7f87280b92a0 2024-04-25T10:02:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f872c84d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87280d1a00 con 0x7f872815dbe0 2024-04-25T10:02:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f8726d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f87280b92a0 0x7f87280d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f8726d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f87280b92a0 0x7f87280d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43864/0 (socket says 172.21.15.19:43864) 2024-04-25T10:02:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f8726d9d640 1 -- 172.21.15.19:0/2761500778 learned_addr learned my addr 172.21.15.19:0/2761500778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f872659c640 1 --2- 172.21.15.19:0/2761500778 >> v2:172.21.15.112:3300/0 conn(0x7f872815dbe0 0x7f87280d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f8726d9d640 1 -- 172.21.15.19:0/2761500778 >> v2:172.21.15.112:3300/0 conn(0x7f872815dbe0 msgr2=0x7f87280d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f8726d9d640 1 --2- 172.21.15.19:0/2761500778 >> v2:172.21.15.112:3300/0 conn(0x7f872815dbe0 0x7f87280d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f8726d9d640 1 -- 172.21.15.19:0/2761500778 >> v1:172.21.15.112:6789/0 conn(0x7f87280b2cb0 legacy=0x7f87280b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.524+0000 7f8726d9d640 1 -- 172.21.15.19:0/2761500778 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87280d1c50 con 0x7f87280b92a0 2024-04-25T10:02:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.525+0000 7f8726d9d640 1 --2- 172.21.15.19:0/2761500778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87280b92a0 0x7f87280d0e50 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f872806c140 tx=0x7f872002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9585165bd2ebef82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.525+0000 7f8725d9b640 1 -- 172.21.15.19:0/2761500778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720036070 con 0x7f87280b92a0 2024-04-25T10:02:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.525+0000 7f8725d9b640 1 -- 172.21.15.19:0/2761500778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8720031600 con 0x7f87280b92a0 2024-04-25T10:02:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.525+0000 7f8725d9b640 1 -- 172.21.15.19:0/2761500778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87200319d0 con 0x7f87280b92a0 2024-04-25T10:02:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.526+0000 7f872c84d640 1 -- 172.21.15.19:0/2761500778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87280b92a0 msgr2=0x7f87280d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.526+0000 7f872c84d640 1 --2- 172.21.15.19:0/2761500778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87280b92a0 0x7f87280d0e50 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f872806c140 tx=0x7f872002a660 comp rx=0 tx=0).stop 2024-04-25T10:02:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.526+0000 7f872c84d640 1 -- 172.21.15.19:0/2761500778 shutdown_connections 2024-04-25T10:02:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.526+0000 7f872c84d640 1 --2- 172.21.15.19:0/2761500778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87280b92a0 0x7f87280d0e50 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.526+0000 7f872c84d640 1 --2- 172.21.15.19:0/2761500778 >> v2:172.21.15.112:3300/0 conn(0x7f872815dbe0 0x7f87280d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.526+0000 7f872c84d640 1 -- 172.21.15.19:0/2761500778 >> 172.21.15.19:0/2761500778 conn(0x7f8728061a20 msgr2=0x7f8728061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.526+0000 7f872c84d640 1 -- 172.21.15.19:0/2761500778 shutdown_connections 2024-04-25T10:02:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.526+0000 7f872c84d640 1 -- 172.21.15.19:0/2761500778 wait complete. 2024-04-25T10:02:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.527+0000 7f872c84d640 1 Processor -- start 2024-04-25T10:02:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.527+0000 7f872c84d640 1 -- start start 2024-04-25T10:02:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f872c84d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87280b2cb0 0x7f87280c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f872c84d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87280b92a0 0x7f87281598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f872c84d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f872815dbe0 0x7f8728159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f872c84d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87280b6ab0 con 0x7f87280b92a0 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f872c84d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87280d2210 con 0x7f872815dbe0 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f872c84d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87280b6c30 con 0x7f87280b2cb0 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f871ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87280b92a0 0x7f87281598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f871ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87280b92a0 0x7f87281598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43870/0 (socket says 172.21.15.19:43870) 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f871ffff640 1 -- 172.21.15.19:0/172021261 learned_addr learned my addr 172.21.15.19:0/172021261 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f8726d9d640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f872815dbe0 0x7f8728159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f872659c640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87280b2cb0 0x7f87280c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f871ffff640 1 -- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87280b2cb0 msgr2=0x7f87280c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f871ffff640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87280b2cb0 0x7f87280c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f871ffff640 1 -- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f872815dbe0 msgr2=0x7f8728159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f871ffff640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f872815dbe0 0x7f8728159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f871ffff640 1 -- 172.21.15.19:0/172021261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87280b33f0 con 0x7f87280b92a0 2024-04-25T10:02:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f872659c640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87280b2cb0 0x7f87280c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.528+0000 7f871ffff640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87280b92a0 0x7f87281598b0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f871000c400 tx=0x7f871000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.529+0000 7f871effd640 1 -- 172.21.15.19:0/172021261 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710011790 con 0x7f87280b92a0 2024-04-25T10:02:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.529+0000 7f871effd640 1 -- 172.21.15.19:0/172021261 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8710017070 con 0x7f87280b92a0 2024-04-25T10:02:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.529+0000 7f872c84d640 1 -- 172.21.15.19:0/172021261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87280b36d0 con 0x7f87280b92a0 2024-04-25T10:02:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.529+0000 7f871effd640 1 -- 172.21.15.19:0/172021261 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710011bb0 con 0x7f87280b92a0 2024-04-25T10:02:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.529+0000 7f872c84d640 1 -- 172.21.15.19:0/172021261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87280b3c90 con 0x7f87280b92a0 2024-04-25T10:02:56.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.530+0000 7f872c84d640 1 -- 172.21.15.19:0/172021261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87280b3e90 con 0x7f87280b92a0 2024-04-25T10:02:56.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.530+0000 7f871effd640 1 -- 172.21.15.19:0/172021261 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f871001c460 con 0x7f87280b92a0 2024-04-25T10:02:56.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.531+0000 7f871effd640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f870806b4d0 0x7f870806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.531+0000 7f871effd640 1 -- 172.21.15.19:0/172021261 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f871008a170 con 0x7f87280b92a0 2024-04-25T10:02:56.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.531+0000 7f872659c640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f870806b4d0 0x7f870806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.532+0000 7f872659c640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f870806b4d0 0x7f870806d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8718006fd0 tx=0x7f8718008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.535+0000 7f871effd640 1 -- 172.21.15.19:0/172021261 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87280b3e90 con 0x7f87280b92a0 2024-04-25T10:02:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.719+0000 7f872c84d640 1 -- 172.21.15.19:0/172021261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8728073990 con 0x7f87280b92a0 2024-04-25T10:02:56.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.720+0000 7f871effd640 1 -- 172.21.15.19:0/172021261 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+17037 (secure 0 0 0) 0x7f8710058e20 con 0x7f87280b92a0 2024-04-25T10:02:56.725 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:56.725 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":142,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:48.692096+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:33.312366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:33.971764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.1","mappings":[{"from":3,"to":0}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 -- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f870806b4d0 msgr2=0x7f870806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f870806b4d0 0x7f870806d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8718006fd0 tx=0x7f8718008040 comp rx=0 tx=0).stop 2024-04-25T10:02:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 -- 172.21.15.19:0/172021261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87280b92a0 msgr2=0x7f87281598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87280b92a0 0x7f87281598b0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f871000c400 tx=0x7f871000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 -- 172.21.15.19:0/172021261 shutdown_connections 2024-04-25T10:02:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f870806b4d0 0x7f870806d990 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f872815dbe0 0x7f8728159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87280b92a0 0x7f87281598b0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 --2- 172.21.15.19:0/172021261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87280b2cb0 0x7f87280c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 -- 172.21.15.19:0/172021261 >> 172.21.15.19:0/172021261 conn(0x7f8728061a20 msgr2=0x7f87280b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 -- 172.21.15.19:0/172021261 shutdown_connections 2024-04-25T10:02:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.723+0000 7f872c84d640 1 -- 172.21.15.19:0/172021261 wait complete. 2024-04-25T10:02:56.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:33.312366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:33.971764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:56.739 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:56.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.838+0000 7f20e5510640 1 Processor -- start 2024-04-25T10:02:56.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.838+0000 7f20e5510640 1 -- start start 2024-04-25T10:02:56.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.839+0000 7f20e5510640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20e015a9f0 0x7f20e00c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.839+0000 7f20e5510640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20e00cdbf0 0x7f20e00ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.839+0000 7f20e5510640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f20e00cd520 0x7f20e00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.839+0000 7f20e5510640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e00b0020 con 0x7f20e015a9f0 2024-04-25T10:02:56.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.839+0000 7f20e5510640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e00cb5a0 con 0x7f20e00cd520 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.839+0000 7f20e5510640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00c6940 con 0x7f20e00cdbf0 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.839+0000 7f20deffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20e015a9f0 0x7f20e00c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.839+0000 7f20deffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20e015a9f0 0x7f20e00c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43880/0 (socket says 172.21.15.19:43880) 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.839+0000 7f20deffd640 1 -- 172.21.15.19:0/4130338504 learned_addr learned my addr 172.21.15.19:0/4130338504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.839+0000 7f20de7fc640 1 --2- 172.21.15.19:0/4130338504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20e00cdbf0 0x7f20e00ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.839+0000 7f20df7fe640 1 --2- 172.21.15.19:0/4130338504 >> v2:172.21.15.112:3300/0 conn(0x7f20e00cd520 0x7f20e00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.840+0000 7f20deffd640 1 -- 172.21.15.19:0/4130338504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20e00cdbf0 msgr2=0x7f20e00ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.840+0000 7f20deffd640 1 --2- 172.21.15.19:0/4130338504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20e00cdbf0 0x7f20e00ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.840+0000 7f20deffd640 1 -- 172.21.15.19:0/4130338504 >> v2:172.21.15.112:3300/0 conn(0x7f20e00cd520 msgr2=0x7f20e00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.840+0000 7f20deffd640 1 --2- 172.21.15.19:0/4130338504 >> v2:172.21.15.112:3300/0 conn(0x7f20e00cd520 0x7f20e00ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.840+0000 7f20deffd640 1 -- 172.21.15.19:0/4130338504 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e015add0 con 0x7f20e015a9f0 2024-04-25T10:02:56.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.840+0000 7f20df7fe640 1 --2- 172.21.15.19:0/4130338504 >> v2:172.21.15.112:3300/0 conn(0x7f20e00cd520 0x7f20e00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.840+0000 7f20deffd640 1 --2- 172.21.15.19:0/4130338504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20e015a9f0 0x7f20e00c6010 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f20e00b6950 tx=0x7f20d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ee08971d356d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.840+0000 7f20ddffb640 1 -- 172.21.15.19:0/4130338504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8037070 con 0x7f20e015a9f0 2024-04-25T10:02:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.841+0000 7f20ddffb640 1 -- 172.21.15.19:0/4130338504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8032440 con 0x7f20e015a9f0 2024-04-25T10:02:56.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.841+0000 7f20ddffb640 1 -- 172.21.15.19:0/4130338504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8032810 con 0x7f20e015a9f0 2024-04-25T10:02:56.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.841+0000 7f20e5510640 1 -- 172.21.15.19:0/4130338504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20e015a9f0 msgr2=0x7f20e00c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.841+0000 7f20e5510640 1 --2- 172.21.15.19:0/4130338504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20e015a9f0 0x7f20e00c6010 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f20e00b6950 tx=0x7f20d802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.841+0000 7f20e5510640 1 -- 172.21.15.19:0/4130338504 shutdown_connections 2024-04-25T10:02:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.841+0000 7f20e5510640 1 --2- 172.21.15.19:0/4130338504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20e00cdbf0 0x7f20e00ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.841+0000 7f20e5510640 1 --2- 172.21.15.19:0/4130338504 >> v2:172.21.15.112:3300/0 conn(0x7f20e00cd520 0x7f20e00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.841+0000 7f20e5510640 1 --2- 172.21.15.19:0/4130338504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20e015a9f0 0x7f20e00c6010 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.841+0000 7f20e5510640 1 -- 172.21.15.19:0/4130338504 >> 172.21.15.19:0/4130338504 conn(0x7f20e00b2390 msgr2=0x7f20e00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.841+0000 7f20e5510640 1 -- 172.21.15.19:0/4130338504 shutdown_connections 2024-04-25T10:02:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.841+0000 7f20e5510640 1 -- 172.21.15.19:0/4130338504 wait complete. 2024-04-25T10:02:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.842+0000 7f20e5510640 1 Processor -- start 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.842+0000 7f20e5510640 1 -- start start 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.843+0000 7f20e5510640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20e00cd520 0x7f20e00c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.843+0000 7f20e5510640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e00cdbf0 0x7f20e00ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.843+0000 7f20e5510640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20e015a9f0 0x7f20e006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.843+0000 7f20e5510640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00cb230 con 0x7f20e015a9f0 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.843+0000 7f20e5510640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00caf50 con 0x7f20e00cdbf0 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.843+0000 7f20e5510640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00ce070 con 0x7f20e00cd520 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.843+0000 7f20df7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20e015a9f0 0x7f20e006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.843+0000 7f20df7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20e015a9f0 0x7f20e006abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43886/0 (socket says 172.21.15.19:43886) 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.843+0000 7f20df7fe640 1 -- 172.21.15.19:0/3769674984 learned_addr learned my addr 172.21.15.19:0/3769674984 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20de7fc640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e00cdbf0 0x7f20e00ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20deffd640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20e00cd520 0x7f20e00c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20df7fe640 1 -- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20e00cd520 msgr2=0x7f20e00c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20df7fe640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20e00cd520 0x7f20e00c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20df7fe640 1 -- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e00cdbf0 msgr2=0x7f20e00ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20df7fe640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e00cdbf0 0x7f20e00ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20df7fe640 1 -- 172.21.15.19:0/3769674984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e00c8750 con 0x7f20e015a9f0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20df7fe640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20e015a9f0 0x7f20e006abe0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f20d0007e10 tx=0x7f20d000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20c77fe640 1 -- 172.21.15.19:0/3769674984 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d001a070 con 0x7f20e015a9f0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20c77fe640 1 -- 172.21.15.19:0/3769674984 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d0003b90 con 0x7f20e015a9f0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20e5510640 1 -- 172.21.15.19:0/3769674984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e00ae5a0 con 0x7f20e015a9f0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20c77fe640 1 -- 172.21.15.19:0/3769674984 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0004030 con 0x7f20e015a9f0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.844+0000 7f20e5510640 1 -- 172.21.15.19:0/3769674984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e00ae2c0 con 0x7f20e015a9f0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.845+0000 7f20e5510640 1 -- 172.21.15.19:0/3769674984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e00b2ff0 con 0x7f20e015a9f0 2024-04-25T10:02:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.845+0000 7f20c77fe640 1 -- 172.21.15.19:0/3769674984 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20d001d070 con 0x7f20e015a9f0 2024-04-25T10:02:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.846+0000 7f20c77fe640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20cc06b4d0 0x7f20cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.846+0000 7f20c77fe640 1 -- 172.21.15.19:0/3769674984 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f20d008a750 con 0x7f20e015a9f0 2024-04-25T10:02:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.846+0000 7f20deffd640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20cc06b4d0 0x7f20cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.847+0000 7f20deffd640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20cc06b4d0 0x7f20cc06d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f20d8034000 tx=0x7f20d802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:56.850+0000 7f20c77fe640 1 -- 172.21.15.19:0/3769674984 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20d0010070 con 0x7f20e015a9f0 2024-04-25T10:02:57.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.029+0000 7f20e5510640 1 -- 172.21.15.19:0/3769674984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f20e00af8c0 con 0x7f20e015a9f0 2024-04-25T10:02:57.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.030+0000 7f20c77fe640 1 -- 172.21.15.19:0/3769674984 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v142) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20d0059370 con 0x7f20e015a9f0 2024-04-25T10:02:57.035 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 -- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20cc06b4d0 msgr2=0x7f20cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20cc06b4d0 0x7f20cc06d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f20d8034000 tx=0x7f20d802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 -- 172.21.15.19:0/3769674984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20e015a9f0 msgr2=0x7f20e006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20e015a9f0 0x7f20e006abe0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f20d0007e10 tx=0x7f20d000d4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 -- 172.21.15.19:0/3769674984 shutdown_connections 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20cc06b4d0 0x7f20cc06d990 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20e015a9f0 0x7f20e006abe0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e00cdbf0 0x7f20e00ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 --2- 172.21.15.19:0/3769674984 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20e00cd520 0x7f20e00c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 -- 172.21.15.19:0/3769674984 >> 172.21.15.19:0/3769674984 conn(0x7f20e00b2390 msgr2=0x7f20e00cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 -- 172.21.15.19:0/3769674984 shutdown_connections 2024-04-25T10:02:57.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.032+0000 7f20e5510640 1 -- 172.21.15.19:0/3769674984 wait complete. 2024-04-25T10:02:57.047 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.146+0000 7f520d08f640 1 Processor -- start 2024-04-25T10:02:57.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.146+0000 7f520d08f640 1 -- start start 2024-04-25T10:02:57.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f520d08f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52080cc4f0 0x7f52080cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f520d08f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080cdb20 0x7f52080cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f520d08f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f520815b270 0x7f520815b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f520d08f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52080ce4c0 con 0x7f52080cc4f0 2024-04-25T10:02:57.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f520d08f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5208071270 con 0x7f520815b270 2024-04-25T10:02:57.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f520d08f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5208072280 con 0x7f52080cdb20 2024-04-25T10:02:57.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f5206d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52080cc4f0 0x7f52080cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f5206d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52080cc4f0 0x7f52080cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43892/0 (socket says 172.21.15.19:43892) 2024-04-25T10:02:57.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f5206d9d640 1 -- 172.21.15.19:0/3681005700 learned_addr learned my addr 172.21.15.19:0/3681005700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:57.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f520759e640 1 --2- 172.21.15.19:0/3681005700 >> v2:172.21.15.112:3300/0 conn(0x7f520815b270 0x7f520815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f51fffff640 1 --2- 172.21.15.19:0/3681005700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080cdb20 0x7f52080cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f5206d9d640 1 -- 172.21.15.19:0/3681005700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080cdb20 msgr2=0x7f52080cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f5206d9d640 1 --2- 172.21.15.19:0/3681005700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080cdb20 0x7f52080cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f5206d9d640 1 -- 172.21.15.19:0/3681005700 >> v2:172.21.15.112:3300/0 conn(0x7f520815b270 msgr2=0x7f520815b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f5206d9d640 1 --2- 172.21.15.19:0/3681005700 >> v2:172.21.15.112:3300/0 conn(0x7f520815b270 0x7f520815b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.147+0000 7f5206d9d640 1 -- 172.21.15.19:0/3681005700 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52080714c0 con 0x7f52080cc4f0 2024-04-25T10:02:57.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.148+0000 7f5206d9d640 1 --2- 172.21.15.19:0/3681005700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52080cc4f0 0x7f52080cc8d0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f5208158aa0 tx=0x7f51f802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca5729f117c36a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.148+0000 7f520659c640 1 -- 172.21.15.19:0/3681005700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8032790 con 0x7f52080cc4f0 2024-04-25T10:02:57.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.148+0000 7f520659c640 1 -- 172.21.15.19:0/3681005700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f8038070 con 0x7f52080cc4f0 2024-04-25T10:02:57.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.148+0000 7f520659c640 1 -- 172.21.15.19:0/3681005700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8032b50 con 0x7f52080cc4f0 2024-04-25T10:02:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.149+0000 7f520d08f640 1 -- 172.21.15.19:0/3681005700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52080cc4f0 msgr2=0x7f52080cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.149+0000 7f520d08f640 1 --2- 172.21.15.19:0/3681005700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52080cc4f0 0x7f52080cc8d0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f5208158aa0 tx=0x7f51f802c890 comp rx=0 tx=0).stop 2024-04-25T10:02:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.149+0000 7f520d08f640 1 -- 172.21.15.19:0/3681005700 shutdown_connections 2024-04-25T10:02:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.149+0000 7f520d08f640 1 --2- 172.21.15.19:0/3681005700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080cdb20 0x7f52080cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.149+0000 7f520d08f640 1 --2- 172.21.15.19:0/3681005700 >> v2:172.21.15.112:3300/0 conn(0x7f520815b270 0x7f520815b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.149+0000 7f520d08f640 1 --2- 172.21.15.19:0/3681005700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52080cc4f0 0x7f52080cc8d0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.149+0000 7f520d08f640 1 -- 172.21.15.19:0/3681005700 >> 172.21.15.19:0/3681005700 conn(0x7f520806abe0 msgr2=0x7f520806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.149+0000 7f520d08f640 1 -- 172.21.15.19:0/3681005700 shutdown_connections 2024-04-25T10:02:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.149+0000 7f520d08f640 1 -- 172.21.15.19:0/3681005700 wait complete. 2024-04-25T10:02:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.150+0000 7f520d08f640 1 Processor -- start 2024-04-25T10:02:57.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.150+0000 7f520d08f640 1 -- start start 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520d08f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52080cc4f0 0x7f52081452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520d08f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080cdb20 0x7f5208157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520d08f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815b270 0x7f52081579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520d08f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5208071d60 con 0x7f520815b270 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520d08f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5208071a80 con 0x7f52080cc4f0 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520d08f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520815bbc0 con 0x7f52080cdb20 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815b270 0x7f52081579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815b270 0x7f52081579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43898/0 (socket says 172.21.15.19:43898) 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520759e640 1 -- 172.21.15.19:0/1467671618 learned_addr learned my addr 172.21.15.19:0/1467671618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f5206d9d640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52080cc4f0 0x7f52081452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f51fffff640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080cdb20 0x7f5208157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520759e640 1 -- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080cdb20 msgr2=0x7f5208157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520759e640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080cdb20 0x7f5208157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520759e640 1 -- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52080cc4f0 msgr2=0x7f52081452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520759e640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52080cc4f0 0x7f52081452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.151+0000 7f520759e640 1 -- 172.21.15.19:0/1467671618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52080c4780 con 0x7f520815b270 2024-04-25T10:02:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.152+0000 7f520759e640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815b270 0x7f52081579a0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f5200007e70 tx=0x7f520000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.152+0000 7f51ff7fe640 1 -- 172.21.15.19:0/1467671618 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200014620 con 0x7f520815b270 2024-04-25T10:02:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.152+0000 7f51ff7fe640 1 -- 172.21.15.19:0/1467671618 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5200019070 con 0x7f520815b270 2024-04-25T10:02:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.152+0000 7f520d08f640 1 -- 172.21.15.19:0/1467671618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52080c4a60 con 0x7f520815b270 2024-04-25T10:02:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.152+0000 7f51ff7fe640 1 -- 172.21.15.19:0/1467671618 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200014a20 con 0x7f520815b270 2024-04-25T10:02:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.152+0000 7f520d08f640 1 -- 172.21.15.19:0/1467671618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52080c5020 con 0x7f520815b270 2024-04-25T10:02:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.153+0000 7f520d08f640 1 -- 172.21.15.19:0/1467671618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5208073a00 con 0x7f520815b270 2024-04-25T10:02:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.153+0000 7f51ff7fe640 1 -- 172.21.15.19:0/1467671618 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5200014bc0 con 0x7f520815b270 2024-04-25T10:02:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.154+0000 7f51ff7fe640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51e806b4d0 0x7f51e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.154+0000 7f51ff7fe640 1 -- 172.21.15.19:0/1467671618 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f520008ac40 con 0x7f520815b270 2024-04-25T10:02:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.154+0000 7f5206d9d640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51e806b4d0 0x7f51e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.154+0000 7f5206d9d640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51e806b4d0 0x7f51e806d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f51f8037000 tx=0x7f51f8032000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.158+0000 7f51ff7fe640 1 -- 172.21.15.19:0/1467671618 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5200059230 con 0x7f520815b270 2024-04-25T10:02:57.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.337+0000 7f520d08f640 1 -- 172.21.15.19:0/1467671618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f520815a400 con 0x7f520815b270 2024-04-25T10:02:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.338+0000 7f51ff7fe640 1 -- 172.21.15.19:0/1467671618 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v142) v1 ==== 104+0+11 (secure 0 0 0) 0x7f520005d2d0 con 0x7f520815b270 2024-04-25T10:02:57.343 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:02:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 -- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51e806b4d0 msgr2=0x7f51e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51e806b4d0 0x7f51e806d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f51f8037000 tx=0x7f51f8032000 comp rx=0 tx=0).stop 2024-04-25T10:02:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 -- 172.21.15.19:0/1467671618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815b270 msgr2=0x7f52081579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815b270 0x7f52081579a0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f5200007e70 tx=0x7f520000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 -- 172.21.15.19:0/1467671618 shutdown_connections 2024-04-25T10:02:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f51e806b4d0 0x7f51e806d990 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815b270 0x7f52081579a0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080cdb20 0x7f5208157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 --2- 172.21.15.19:0/1467671618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52080cc4f0 0x7f52081452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 -- 172.21.15.19:0/1467671618 >> 172.21.15.19:0/1467671618 conn(0x7f520806abe0 msgr2=0x7f520806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 -- 172.21.15.19:0/1467671618 shutdown_connections 2024-04-25T10:02:57.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.340+0000 7f520d08f640 1 -- 172.21.15.19:0/1467671618 wait complete. 2024-04-25T10:02:57.355 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.454+0000 7fcdd5e9e640 1 Processor -- start 2024-04-25T10:02:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.454+0000 7fcdd5e9e640 1 -- start start 2024-04-25T10:02:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdd5e9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcdd015dc20 0x7fcdd00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdd5e9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdd0161710 0x7fcdd00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdd5e9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdd00b9330 0x7fcdd006bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdd5e9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd015e000 con 0x7fcdd0161710 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdd5e9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd00d1de0 con 0x7fcdd015dc20 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdd5e9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0161c60 con 0x7fcdd00b9330 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdceffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdd0161710 0x7fcdd00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdceffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdd0161710 0x7fcdd00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43904/0 (socket says 172.21.15.19:43904) 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdceffd640 1 -- 172.21.15.19:0/434725859 learned_addr learned my addr 172.21.15.19:0/434725859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdcf7fe640 1 --2- 172.21.15.19:0/434725859 >> v2:172.21.15.112:3300/0 conn(0x7fcdd015dc20 0x7fcdd00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdcffff640 1 --2- 172.21.15.19:0/434725859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdd00b9330 0x7fcdd006bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdceffd640 1 -- 172.21.15.19:0/434725859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdd00b9330 msgr2=0x7fcdd006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdceffd640 1 --2- 172.21.15.19:0/434725859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdd00b9330 0x7fcdd006bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdceffd640 1 -- 172.21.15.19:0/434725859 >> v2:172.21.15.112:3300/0 conn(0x7fcdd015dc20 msgr2=0x7fcdd00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.455+0000 7fcdceffd640 1 --2- 172.21.15.19:0/434725859 >> v2:172.21.15.112:3300/0 conn(0x7fcdd015dc20 0x7fcdd00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.456+0000 7fcdceffd640 1 -- 172.21.15.19:0/434725859 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd0161f40 con 0x7fcdd0161710 2024-04-25T10:02:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.456+0000 7fcdcffff640 1 --2- 172.21.15.19:0/434725859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdd00b9330 0x7fcdd006bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.456+0000 7fcdceffd640 1 --2- 172.21.15.19:0/434725859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0161710 0x7fcdd00cebb0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00d1520 tx=0x7fcdc402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc686ae884eeb2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.456+0000 7fcdce7fc640 1 -- 172.21.15.19:0/434725859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4037070 con 0x7fcdd0161710 2024-04-25T10:02:57.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.456+0000 7fcdce7fc640 1 -- 172.21.15.19:0/434725859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc4032440 con 0x7fcdd0161710 2024-04-25T10:02:57.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.456+0000 7fcdce7fc640 1 -- 172.21.15.19:0/434725859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4032810 con 0x7fcdd0161710 2024-04-25T10:02:57.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.457+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/434725859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0161710 msgr2=0x7fcdd00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.457+0000 7fcdd5e9e640 1 --2- 172.21.15.19:0/434725859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0161710 0x7fcdd00cebb0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00d1520 tx=0x7fcdc402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.457+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/434725859 shutdown_connections 2024-04-25T10:02:57.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.457+0000 7fcdd5e9e640 1 --2- 172.21.15.19:0/434725859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdd00b9330 0x7fcdd006bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.457+0000 7fcdd5e9e640 1 --2- 172.21.15.19:0/434725859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0161710 0x7fcdd00cebb0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.457+0000 7fcdd5e9e640 1 --2- 172.21.15.19:0/434725859 >> v2:172.21.15.112:3300/0 conn(0x7fcdd015dc20 0x7fcdd00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.457+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/434725859 >> 172.21.15.19:0/434725859 conn(0x7fcdd00d0420 msgr2=0x7fcdd00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.457+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/434725859 shutdown_connections 2024-04-25T10:02:57.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.457+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/434725859 wait complete. 2024-04-25T10:02:57.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.458+0000 7fcdd5e9e640 1 Processor -- start 2024-04-25T10:02:57.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.458+0000 7fcdd5e9e640 1 -- start start 2024-04-25T10:02:57.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdd5e9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdd00b9330 0x7fcdd01e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdd5e9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdd015dc20 0x7fcdd0154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdd5e9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0161710 0x7fcdd00d2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdd5e9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00b5e50 con 0x7fcdd0161710 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdd5e9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0162500 con 0x7fcdd00b9330 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdd5e9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00b5fd0 con 0x7fcdd015dc20 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdcffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0161710 0x7fcdd00d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdcffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0161710 0x7fcdd00d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43906/0 (socket says 172.21.15.19:43906) 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdcffff640 1 -- 172.21.15.19:0/477977579 learned_addr learned my addr 172.21.15.19:0/477977579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdceffd640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdd015dc20 0x7fcdd0154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdcf7fe640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdd00b9330 0x7fcdd01e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdcffff640 1 -- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdd015dc20 msgr2=0x7fcdd0154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdcffff640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdd015dc20 0x7fcdd0154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdcffff640 1 -- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdd00b9330 msgr2=0x7fcdd01e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.459+0000 7fcdcffff640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdd00b9330 0x7fcdd01e7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.460+0000 7fcdcffff640 1 -- 172.21.15.19:0/477977579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd01e8460 con 0x7fcdd0161710 2024-04-25T10:02:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.460+0000 7fcdcffff640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0161710 0x7fcdd00d2910 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8007d40 tx=0x7fcdc800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.460+0000 7fcdb7fff640 1 -- 172.21.15.19:0/477977579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc801a070 con 0x7fcdd0161710 2024-04-25T10:02:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.460+0000 7fcdb7fff640 1 -- 172.21.15.19:0/477977579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc8015400 con 0x7fcdd0161710 2024-04-25T10:02:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.460+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/477977579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd01e8700 con 0x7fcdd0161710 2024-04-25T10:02:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.460+0000 7fcdb7fff640 1 -- 172.21.15.19:0/477977579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80157f0 con 0x7fcdd0161710 2024-04-25T10:02:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.460+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/477977579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd01e8cc0 con 0x7fcdd0161710 2024-04-25T10:02:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.461+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/477977579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd0073a00 con 0x7fcdd0161710 2024-04-25T10:02:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.461+0000 7fcdb7fff640 1 -- 172.21.15.19:0/477977579 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdc801d070 con 0x7fcdd0161710 2024-04-25T10:02:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.462+0000 7fcdb7fff640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcdbc06b500 0x7fcdbc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.462+0000 7fcdb7fff640 1 -- 172.21.15.19:0/477977579 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcdc808a460 con 0x7fcdd0161710 2024-04-25T10:02:57.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.462+0000 7fcdcf7fe640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcdbc06b500 0x7fcdbc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.462+0000 7fcdcf7fe640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcdbc06b500 0x7fcdbc06d9c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009aa0 tx=0x7fcdc0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.466+0000 7fcdb7fff640 1 -- 172.21.15.19:0/477977579 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdc8010070 con 0x7fcdd0161710 2024-04-25T10:02:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.649+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/477977579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcdd00d16b0 con 0x7fcdd0161710 2024-04-25T10:02:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.649+0000 7fcdb7fff640 1 -- 172.21.15.19:0/477977579 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v142) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcdc8058ad0 con 0x7fcdd0161710 2024-04-25T10:02:57.654 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:02:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcdbc06b500 msgr2=0x7fcdbc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcdbc06b500 0x7fcdbc06d9c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009aa0 tx=0x7fcdc0009190 comp rx=0 tx=0).stop 2024-04-25T10:02:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/477977579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0161710 msgr2=0x7fcdd00d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0161710 0x7fcdd00d2910 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8007d40 tx=0x7fcdc800d4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/477977579 shutdown_connections 2024-04-25T10:02:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcdbc06b500 0x7fcdbc06d9c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdd0161710 0x7fcdd00d2910 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcdd015dc20 0x7fcdd0154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 --2- 172.21.15.19:0/477977579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdd00b9330 0x7fcdd01e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/477977579 >> 172.21.15.19:0/477977579 conn(0x7fcdd00d0420 msgr2=0x7fcdd00d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/477977579 shutdown_connections 2024-04-25T10:02:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.651+0000 7fcdd5e9e640 1 -- 172.21.15.19:0/477977579 wait complete. 2024-04-25T10:02:57.667 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.764+0000 7f5ae1dd5640 1 Processor -- start 2024-04-25T10:02:57.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.764+0000 7f5ae1dd5640 1 -- start start 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5ae1dd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5adc15b7d0 0x7f5adc0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5ae1dd5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5adc06c9e0 0x7f5adc0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5ae1dd5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5adc06cf50 con 0x7f5adc0b5620 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5ae1dd5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5adc0b5fc0 con 0x7f5adc06c9e0 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5ae1dd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc0ce530 con 0x7f5adc15b7d0 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5adbfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5adc06c9e0 0x7f5adc0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5adaffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5adc0b5620 0x7f5adc06c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47588/0 (socket says 172.21.15.19:47588) 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5adbfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5adc06c9e0 0x7f5adc0b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60020/0 (socket says 172.21.15.19:60020) 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5adaffd640 1 -- 172.21.15.19:0/3711542420 learned_addr learned my addr 172.21.15.19:0/3711542420 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5adb7fe640 1 --2- 172.21.15.19:0/3711542420 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5adc15b7d0 0x7f5adc0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5adbfff640 1 -- 172.21.15.19:0/3711542420 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5adc15b7d0 msgr2=0x7f5adc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5adbfff640 1 --2- 172.21.15.19:0/3711542420 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5adc15b7d0 0x7f5adc0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5adbfff640 1 -- 172.21.15.19:0/3711542420 >> v1:172.21.15.19:6789/0 conn(0x7f5adc0b5620 legacy=0x7f5adc06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5ada7fc640 1 -- 172.21.15.19:0/3711542420 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246702493 0 0) 0x7f5adc06cf50 con 0x7f5adc0b5620 2024-04-25T10:02:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5adbfff640 1 -- 172.21.15.19:0/3711542420 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc0ce810 con 0x7f5adc06c9e0 2024-04-25T10:02:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.765+0000 7f5adb7fe640 1 --2- 172.21.15.19:0/3711542420 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5adc15b7d0 0x7f5adc0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.766+0000 7f5adbfff640 1 --2- 172.21.15.19:0/3711542420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5adc06c9e0 0x7f5adc0b5a80 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009d90 tx=0x7f5ad402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f124417d45cdd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.766+0000 7f5ada7fc640 1 -- 172.21.15.19:0/3711542420 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4007740 con 0x7f5adc06c9e0 2024-04-25T10:02:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.766+0000 7f5ada7fc640 1 -- 172.21.15.19:0/3711542420 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad40078e0 con 0x7f5adc06c9e0 2024-04-25T10:02:57.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.766+0000 7f5ada7fc640 1 -- 172.21.15.19:0/3711542420 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4007cb0 con 0x7f5adc06c9e0 2024-04-25T10:02:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.767+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/3711542420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5adc06c9e0 msgr2=0x7f5adc0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.767+0000 7f5ae1dd5640 1 --2- 172.21.15.19:0/3711542420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5adc06c9e0 0x7f5adc0b5a80 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009d90 tx=0x7f5ad402a660 comp rx=0 tx=0).stop 2024-04-25T10:02:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.767+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/3711542420 shutdown_connections 2024-04-25T10:02:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.767+0000 7f5ae1dd5640 1 --2- 172.21.15.19:0/3711542420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5adc06c9e0 0x7f5adc0b5a80 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.767+0000 7f5ae1dd5640 1 --2- 172.21.15.19:0/3711542420 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5adc15b7d0 0x7f5adc0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.767+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/3711542420 >> 172.21.15.19:0/3711542420 conn(0x7f5adc0d0800 msgr2=0x7f5adc0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.767+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/3711542420 shutdown_connections 2024-04-25T10:02:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.767+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/3711542420 wait complete. 2024-04-25T10:02:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.768+0000 7f5ae1dd5640 1 Processor -- start 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.768+0000 7f5ae1dd5640 1 -- start start 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5ae1dd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5adc06c9e0 0x7f5adc1593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5ae1dd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5adc0b5620 0x7f5adc1f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5ae1dd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5adc15b7d0 0x7f5adc0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5ae1dd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc0cf550 con 0x7f5adc06c9e0 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5ae1dd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc06d230 con 0x7f5adc15b7d0 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5ae1dd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc0cf6d0 con 0x7f5adc0b5620 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5adb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5adc06c9e0 0x7f5adc1593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5adb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5adc06c9e0 0x7f5adc1593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60024/0 (socket says 172.21.15.19:60024) 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5adb7fe640 1 -- 172.21.15.19:0/80250326 learned_addr learned my addr 172.21.15.19:0/80250326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5adaffd640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5adc0b5620 0x7f5adc1f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5adbfff640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5adc15b7d0 0x7f5adc0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5adb7fe640 1 -- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5adc0b5620 msgr2=0x7f5adc1f3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5adb7fe640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5adc0b5620 0x7f5adc1f3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5adb7fe640 1 -- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5adc15b7d0 msgr2=0x7f5adc0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5adb7fe640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5adc15b7d0 0x7f5adc0c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.769+0000 7f5adb7fe640 1 -- 172.21.15.19:0/80250326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc0c88c0 con 0x7f5adc06c9e0 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.770+0000 7f5adb7fe640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5adc06c9e0 0x7f5adc1593b0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00c7f0 tx=0x7f5acc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.770+0000 7f5ac3fff640 1 -- 172.21.15.19:0/80250326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc015070 con 0x7f5adc06c9e0 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.770+0000 7f5ac3fff640 1 -- 172.21.15.19:0/80250326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc007820 con 0x7f5adc06c9e0 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.770+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/80250326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5adc0c8ba0 con 0x7f5adc06c9e0 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.770+0000 7f5ac3fff640 1 -- 172.21.15.19:0/80250326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc007bd0 con 0x7f5adc06c9e0 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.770+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/80250326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc0c9160 con 0x7f5adc06c9e0 2024-04-25T10:02:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.771+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/80250326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5adc073a00 con 0x7f5adc06c9e0 2024-04-25T10:02:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.771+0000 7f5ac3fff640 1 -- 172.21.15.19:0/80250326 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5acc018070 con 0x7f5adc06c9e0 2024-04-25T10:02:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.771+0000 7f5ac3fff640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac806b4d0 0x7f5ac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.772+0000 7f5ac3fff640 1 -- 172.21.15.19:0/80250326 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5acc08a3b0 con 0x7f5adc06c9e0 2024-04-25T10:02:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.772+0000 7f5adaffd640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac806b4d0 0x7f5ac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.772+0000 7f5adaffd640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac806b4d0 0x7f5ac806d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0c73a0 tx=0x7f5ad0007d50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.776+0000 7f5ac3fff640 1 -- 172.21.15.19:0/80250326 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5acc058ab0 con 0x7f5adc06c9e0 2024-04-25T10:02:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.968+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/80250326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5adc160240 con 0x7f5adc06c9e0 2024-04-25T10:02:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.968+0000 7f5ac3fff640 1 -- 172.21.15.19:0/80250326 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v142) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5acc05cb50 con 0x7f5adc06c9e0 2024-04-25T10:02:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T10:02:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac806b4d0 msgr2=0x7f5ac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac806b4d0 0x7f5ac806d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0c73a0 tx=0x7f5ad0007d50 comp rx=0 tx=0).stop 2024-04-25T10:02:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/80250326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5adc06c9e0 msgr2=0x7f5adc1593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5adc06c9e0 0x7f5adc1593b0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00c7f0 tx=0x7f5acc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/80250326 shutdown_connections 2024-04-25T10:02:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac806b4d0 0x7f5ac806d990 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5adc15b7d0 0x7f5adc0c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5adc0b5620 0x7f5adc1f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 --2- 172.21.15.19:0/80250326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5adc06c9e0 0x7f5adc1593b0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/80250326 >> 172.21.15.19:0/80250326 conn(0x7f5adc0d0800 msgr2=0x7f5adc160ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/80250326 shutdown_connections 2024-04-25T10:02:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:57.970+0000 7f5ae1dd5640 1 -- 172.21.15.19:0/80250326 wait complete. 2024-04-25T10:02:57.986 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:02:57.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:57.986 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:58.595 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:02:58.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:58.596 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:02:58.611 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:02:58.611 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:58.627 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:58.627 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:02:58.627 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:58.667 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:02:58.668 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:02:58.668 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:58.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:58.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.804+0000 7ffba3688640 1 Processor -- start 2024-04-25T10:02:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.804+0000 7ffba3688640 1 -- start start 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.805+0000 7ffba3688640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffb9c15dd30 0x7ffb9c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.805+0000 7ffba3688640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb9c0d0ad0 0x7ffb9c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.805+0000 7ffba3688640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9c0d1df0 con 0x7ffb9c0b9770 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.805+0000 7ffba3688640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9c0d20d0 con 0x7ffb9c15dd30 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.805+0000 7ffba3688640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9c0d23b0 con 0x7ffb9c0d0ad0 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.805+0000 7ffba1424640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffb9c15dd30 0x7ffb9c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.805+0000 7ffba0c23640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ffb9c0b9770 0x7ffb9c0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47604/0 (socket says 172.21.15.19:47604) 2024-04-25T10:02:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.805+0000 7ffba0c23640 1 -- 172.21.15.19:0/3633854357 learned_addr learned my addr 172.21.15.19:0/3633854357 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.806+0000 7ffba1c25640 1 --2- 172.21.15.19:0/3633854357 >> v2:172.21.15.112:3300/0 conn(0x7ffb9c0d0ad0 0x7ffb9c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.806+0000 7ffba1424640 1 -- 172.21.15.19:0/3633854357 >> v2:172.21.15.112:3300/0 conn(0x7ffb9c0d0ad0 msgr2=0x7ffb9c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.806+0000 7ffba1424640 1 --2- 172.21.15.19:0/3633854357 >> v2:172.21.15.112:3300/0 conn(0x7ffb9c0d0ad0 0x7ffb9c0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.806+0000 7ffba1424640 1 -- 172.21.15.19:0/3633854357 >> v1:172.21.15.19:6789/0 conn(0x7ffb9c0b9770 legacy=0x7ffb9c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.806+0000 7ffb93fff640 1 -- 172.21.15.19:0/3633854357 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411279182 0 0) 0x7ffb9c0d1df0 con 0x7ffb9c0b9770 2024-04-25T10:02:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.806+0000 7ffba1424640 1 -- 172.21.15.19:0/3633854357 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c0d2690 con 0x7ffb9c15dd30 2024-04-25T10:02:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.806+0000 7ffba1424640 1 --2- 172.21.15.19:0/3633854357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb9c15dd30 0x7ffb9c0b9230 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0ce200 tx=0x7ffb9402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7125997ef2b949 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.806+0000 7ffb93fff640 1 -- 172.21.15.19:0/3633854357 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94037070 con 0x7ffb9c15dd30 2024-04-25T10:02:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.807+0000 7ffb93fff640 1 -- 172.21.15.19:0/3633854357 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9402add0 con 0x7ffb9c15dd30 2024-04-25T10:02:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.807+0000 7ffb93fff640 1 -- 172.21.15.19:0/3633854357 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94032660 con 0x7ffb9c15dd30 2024-04-25T10:02:58.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.807+0000 7ffba3688640 1 -- 172.21.15.19:0/3633854357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb9c15dd30 msgr2=0x7ffb9c0b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.807+0000 7ffba3688640 1 --2- 172.21.15.19:0/3633854357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb9c15dd30 0x7ffb9c0b9230 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0ce200 tx=0x7ffb9402a080 comp rx=0 tx=0).stop 2024-04-25T10:02:58.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.807+0000 7ffba3688640 1 -- 172.21.15.19:0/3633854357 shutdown_connections 2024-04-25T10:02:58.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.807+0000 7ffba3688640 1 --2- 172.21.15.19:0/3633854357 >> v2:172.21.15.112:3300/0 conn(0x7ffb9c0d0ad0 0x7ffb9c0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.807+0000 7ffba3688640 1 --2- 172.21.15.19:0/3633854357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb9c15dd30 0x7ffb9c0b9230 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.807+0000 7ffba3688640 1 -- 172.21.15.19:0/3633854357 >> 172.21.15.19:0/3633854357 conn(0x7ffb9c061c30 msgr2=0x7ffb9c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.807+0000 7ffba3688640 1 -- 172.21.15.19:0/3633854357 shutdown_connections 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.807+0000 7ffba3688640 1 -- 172.21.15.19:0/3633854357 wait complete. 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.808+0000 7ffba3688640 1 Processor -- start 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.808+0000 7ffba3688640 1 -- start start 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba3688640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9c0b9770 0x7ffb9c0b8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba3688640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb9c0d0ad0 0x7ffb9c1f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba3688640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb9c15dd30 0x7ffb9c156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba3688640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c0d1b90 con 0x7ffb9c15dd30 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba3688640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c0d1a10 con 0x7ffb9c0b9770 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba3688640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c0d2e90 con 0x7ffb9c0d0ad0 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba1c25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb9c15dd30 0x7ffb9c156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba1c25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb9c15dd30 0x7ffb9c156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60036/0 (socket says 172.21.15.19:60036) 2024-04-25T10:02:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba1c25640 1 -- 172.21.15.19:0/4217645714 learned_addr learned my addr 172.21.15.19:0/4217645714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba1424640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9c0b9770 0x7ffb9c0b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba0c23640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb9c0d0ad0 0x7ffb9c1f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba1c25640 1 -- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb9c0d0ad0 msgr2=0x7ffb9c1f7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba1c25640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb9c0d0ad0 0x7ffb9c1f7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba1c25640 1 -- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9c0b9770 msgr2=0x7ffb9c0b8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba1c25640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9c0b9770 0x7ffb9c0b8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.809+0000 7ffba1c25640 1 -- 172.21.15.19:0/4217645714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c158440 con 0x7ffb9c15dd30 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.810+0000 7ffba1c25640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb9c15dd30 0x7ffb9c156460 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7ffb98007e40 tx=0x7ffb9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.810+0000 7ffb91ffb640 1 -- 172.21.15.19:0/4217645714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98014620 con 0x7ffb9c15dd30 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.810+0000 7ffb91ffb640 1 -- 172.21.15.19:0/4217645714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb98019070 con 0x7ffb9c15dd30 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.810+0000 7ffb91ffb640 1 -- 172.21.15.19:0/4217645714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98014a20 con 0x7ffb9c15dd30 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.810+0000 7ffba3688640 1 -- 172.21.15.19:0/4217645714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c158720 con 0x7ffb9c15dd30 2024-04-25T10:02:58.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.810+0000 7ffba3688640 1 -- 172.21.15.19:0/4217645714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c158110 con 0x7ffb9c15dd30 2024-04-25T10:02:58.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.811+0000 7ffba3688640 1 -- 172.21.15.19:0/4217645714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9c073990 con 0x7ffb9c15dd30 2024-04-25T10:02:58.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.811+0000 7ffb91ffb640 1 -- 172.21.15.19:0/4217645714 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb98014bc0 con 0x7ffb9c15dd30 2024-04-25T10:02:58.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.812+0000 7ffb91ffb640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8006b4d0 0x7ffb8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.812+0000 7ffb91ffb640 1 -- 172.21.15.19:0/4217645714 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ffb9808aca0 con 0x7ffb9c15dd30 2024-04-25T10:02:58.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.812+0000 7ffba1424640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8006b4d0 0x7ffb8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.812+0000 7ffba1424640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8006b4d0 0x7ffb8006d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ffb94034000 tx=0x7ffb9402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:58.816+0000 7ffb91ffb640 1 -- 172.21.15.19:0/4217645714 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb98059970 con 0x7ffb9c15dd30 2024-04-25T10:02:59.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.039+0000 7ffba3688640 1 -- 172.21.15.19:0/4217645714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb9c0ca310 con 0x7ffb9c15dd30 2024-04-25T10:02:59.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.040+0000 7ffb91ffb640 1 -- 172.21.15.19:0/4217645714 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4949 (secure 0 0 0) 0x7ffb98059270 con 0x7ffb9c15dd30 2024-04-25T10:02:59.045 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:59.045 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":94,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7466,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/590882657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":590882657},{"type":"v1","addr":"172.21.15.112:6836","nonce":590882657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":9744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1740426538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1740426538},{"type":"v1","addr":"172.21.15.19:6833","nonce":1740426538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":94,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:02:34.992088+0000","modified":"2024-04-25T10:02:54.722246+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9750":{"gid":9750,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/525024117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":525024117},{"type":"v1","addr":"172.21.15.19:6835","nonce":525024117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9762":{"gid":9762,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2648279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2648279035},{"type":"v1","addr":"172.21.15.112:6837","nonce":2648279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:02:59.045 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 94 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 -- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8006b4d0 msgr2=0x7ffb8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8006b4d0 0x7ffb8006d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ffb94034000 tx=0x7ffb9402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 -- 172.21.15.19:0/4217645714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb9c15dd30 msgr2=0x7ffb9c156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb9c15dd30 0x7ffb9c156460 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7ffb98007e40 tx=0x7ffb9800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 -- 172.21.15.19:0/4217645714 shutdown_connections 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8006b4d0 0x7ffb8006d990 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb9c15dd30 0x7ffb9c156460 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb9c0d0ad0 0x7ffb9c1f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 --2- 172.21.15.19:0/4217645714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb9c0b9770 0x7ffb9c0b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 -- 172.21.15.19:0/4217645714 >> 172.21.15.19:0/4217645714 conn(0x7ffb9c061c30 msgr2=0x7ffb9c0d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 -- 172.21.15.19:0/4217645714 shutdown_connections 2024-04-25T10:02:59.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.042+0000 7ffba3688640 1 -- 172.21.15.19:0/4217645714 wait complete. 2024-04-25T10:02:59.058 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.154+0000 7f81146b3640 1 Processor -- start 2024-04-25T10:02:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.154+0000 7f81146b3640 1 -- start start 2024-04-25T10:02:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.155+0000 7f81146b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f810c15dd30 0x7f810c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.155+0000 7f81146b3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f810c0d0ad0 0x7f810c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.155+0000 7f81146b3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f810c0d1df0 con 0x7f810c0b9770 2024-04-25T10:02:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.155+0000 7f81146b3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810c0d20d0 con 0x7f810c0d0ad0 2024-04-25T10:02:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.155+0000 7f81146b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0d23b0 con 0x7f810c15dd30 2024-04-25T10:02:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.155+0000 7f8111c4e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f810c0b9770 0x7f810c0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47614/0 (socket says 172.21.15.19:47614) 2024-04-25T10:02:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.155+0000 7f8111c4e640 1 -- 172.21.15.19:0/3202554020 learned_addr learned my addr 172.21.15.19:0/3202554020 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:02:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.155+0000 7f8112c50640 1 --2- 172.21.15.19:0/3202554020 >> v2:172.21.15.112:3300/0 conn(0x7f810c0d0ad0 0x7f810c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.155+0000 7f811244f640 1 --2- 172.21.15.19:0/3202554020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f810c15dd30 0x7f810c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.156+0000 7f811144d640 1 -- 172.21.15.19:0/3202554020 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156271911 0 0) 0x7f810c0d1df0 con 0x7f810c0b9770 2024-04-25T10:02:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.156+0000 7f811144d640 1 -- 172.21.15.19:0/3202554020 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80f4003520 con 0x7f810c0b9770 2024-04-25T10:02:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.156+0000 7f8112c50640 1 -- 172.21.15.19:0/3202554020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f810c15dd30 msgr2=0x7f810c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.156+0000 7f8112c50640 1 --2- 172.21.15.19:0/3202554020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f810c15dd30 0x7f810c0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.156+0000 7f8112c50640 1 -- 172.21.15.19:0/3202554020 >> v1:172.21.15.19:6789/0 conn(0x7f810c0b9770 legacy=0x7f810c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.156+0000 7f8112c50640 1 -- 172.21.15.19:0/3202554020 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c0d2690 con 0x7f810c0d0ad0 2024-04-25T10:02:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.156+0000 7f8112c50640 1 --2- 172.21.15.19:0/3202554020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f810c0d0ad0 0x7f810c0b6a40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009d90 tx=0x7f80fc02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826d371916c6098e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.157+0000 7f811144d640 1 -- 172.21.15.19:0/3202554020 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc031600 con 0x7f810c0d0ad0 2024-04-25T10:02:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.157+0000 7f811144d640 1 -- 172.21.15.19:0/3202554020 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80fc0317a0 con 0x7f810c0d0ad0 2024-04-25T10:02:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.157+0000 7f811144d640 1 -- 172.21.15.19:0/3202554020 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc031b70 con 0x7f810c0d0ad0 2024-04-25T10:02:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.157+0000 7f81146b3640 1 -- 172.21.15.19:0/3202554020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f810c0d0ad0 msgr2=0x7f810c0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.157+0000 7f81146b3640 1 --2- 172.21.15.19:0/3202554020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f810c0d0ad0 0x7f810c0b6a40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009d90 tx=0x7f80fc02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.157+0000 7f81146b3640 1 -- 172.21.15.19:0/3202554020 shutdown_connections 2024-04-25T10:02:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.157+0000 7f81146b3640 1 --2- 172.21.15.19:0/3202554020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f810c0d0ad0 0x7f810c0b6a40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.157+0000 7f81146b3640 1 --2- 172.21.15.19:0/3202554020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f810c15dd30 0x7f810c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.157+0000 7f81146b3640 1 -- 172.21.15.19:0/3202554020 >> 172.21.15.19:0/3202554020 conn(0x7f810c061c30 msgr2=0x7f810c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.157+0000 7f81146b3640 1 -- 172.21.15.19:0/3202554020 shutdown_connections 2024-04-25T10:02:59.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.157+0000 7f81146b3640 1 -- 172.21.15.19:0/3202554020 wait complete. 2024-04-25T10:02:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.158+0000 7f81146b3640 1 Processor -- start 2024-04-25T10:02:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.158+0000 7f81146b3640 1 -- start start 2024-04-25T10:02:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.159+0000 7f81146b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f810c0b9770 0x7f810c0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.159+0000 7f81146b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f810c0d0ad0 0x7f810c0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.159+0000 7f81146b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f810c15dd30 0x7f810c1f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.159+0000 7f81146b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0d1b90 con 0x7f810c0b9770 2024-04-25T10:02:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.159+0000 7f81146b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0d1a10 con 0x7f810c15dd30 2024-04-25T10:02:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.159+0000 7f81146b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0d2e90 con 0x7f810c0d0ad0 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.159+0000 7f811244f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f810c0b9770 0x7f810c0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.159+0000 7f811244f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f810c0b9770 0x7f810c0b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60038/0 (socket says 172.21.15.19:60038) 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.159+0000 7f811244f640 1 -- 172.21.15.19:0/2328422698 learned_addr learned my addr 172.21.15.19:0/2328422698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.160+0000 7f811244f640 1 -- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f810c0d0ad0 msgr2=0x7f810c0cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.160+0000 7f811244f640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f810c0d0ad0 0x7f810c0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.160+0000 7f811244f640 1 -- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f810c15dd30 msgr2=0x7f810c1f6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.160+0000 7f811244f640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f810c15dd30 0x7f810c1f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.160+0000 7f811244f640 1 -- 172.21.15.19:0/2328422698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c0cbd90 con 0x7f810c0b9770 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.160+0000 7f811244f640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f810c0b9770 0x7f810c0b5d60 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f810400e870 tx=0x7f810400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.160+0000 7f810affd640 1 -- 172.21.15.19:0/2328422698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104017070 con 0x7f810c0b9770 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.160+0000 7f810affd640 1 -- 172.21.15.19:0/2328422698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810400c940 con 0x7f810c0b9770 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.160+0000 7f81146b3640 1 -- 172.21.15.19:0/2328422698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810c0cc070 con 0x7f810c0b9770 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.160+0000 7f810affd640 1 -- 172.21.15.19:0/2328422698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810400cd30 con 0x7f810c0b9770 2024-04-25T10:02:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.160+0000 7f81146b3640 1 -- 172.21.15.19:0/2328422698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810c0cb9d0 con 0x7f810c0b9770 2024-04-25T10:02:59.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.161+0000 7f81146b3640 1 -- 172.21.15.19:0/2328422698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f810c073990 con 0x7f810c0b9770 2024-04-25T10:02:59.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.161+0000 7f810affd640 1 -- 172.21.15.19:0/2328422698 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f810401a070 con 0x7f810c0b9770 2024-04-25T10:02:59.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.162+0000 7f810affd640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f80f806b4d0 0x7f80f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.162+0000 7f810affd640 1 -- 172.21.15.19:0/2328422698 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f810408a320 con 0x7f810c0b9770 2024-04-25T10:02:59.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.162+0000 7f8111c4e640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f80f806b4d0 0x7f80f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.163+0000 7f8111c4e640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f80f806b4d0 0x7f80f806d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8100007fd0 tx=0x7f81000077c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.166+0000 7f810affd640 1 -- 172.21.15.19:0/2328422698 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8104058f40 con 0x7f810c0b9770 2024-04-25T10:02:59.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.346+0000 7f81146b3640 1 -- 172.21.15.19:0/2328422698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f810c15d2c0 con 0x7f810c0b9770 2024-04-25T10:02:59.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.347+0000 7f810affd640 1 -- 172.21.15.19:0/2328422698 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+17037 (secure 0 0 0) 0x7f810408a0e0 con 0x7f810c0b9770 2024-04-25T10:02:59.352 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:02:59.353 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":142,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:48.692096+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:33.312366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:33.971764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.1","mappings":[{"from":3,"to":0}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 -- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f80f806b4d0 msgr2=0x7f80f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f80f806b4d0 0x7f80f806d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8100007fd0 tx=0x7f81000077c0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 -- 172.21.15.19:0/2328422698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f810c0b9770 msgr2=0x7f810c0b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f810c0b9770 0x7f810c0b5d60 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f810400e870 tx=0x7f810400ed30 comp rx=0 tx=0).stop 2024-04-25T10:02:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 -- 172.21.15.19:0/2328422698 shutdown_connections 2024-04-25T10:02:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f80f806b4d0 0x7f80f806d990 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f810c15dd30 0x7f810c1f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f810c0d0ad0 0x7f810c0cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 --2- 172.21.15.19:0/2328422698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f810c0b9770 0x7f810c0b5d60 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 -- 172.21.15.19:0/2328422698 >> 172.21.15.19:0/2328422698 conn(0x7f810c061c30 msgr2=0x7f810c0ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 -- 172.21.15.19:0/2328422698 shutdown_connections 2024-04-25T10:02:59.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.350+0000 7f81146b3640 1 -- 172.21.15.19:0/2328422698 wait complete. 2024-04-25T10:02:59.366 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:33.312366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:33.971764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:59.367 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:59.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.465+0000 7fc652a9a640 1 Processor -- start 2024-04-25T10:02:59.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.465+0000 7fc652a9a640 1 -- start start 2024-04-25T10:02:59.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.466+0000 7fc652a9a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc64c0ccdd0 0x7fc64c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.466+0000 7fc652a9a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64c0cdcd0 0x7fc64c0cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.466+0000 7fc652a9a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64c0cb190 con 0x7fc64c15bb20 2024-04-25T10:02:59.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.466+0000 7fc652a9a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64c0d0450 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.466+0000 7fc652a9a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c0cb470 con 0x7fc64c0cdcd0 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.466+0000 7fc64bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc64c0ccdd0 0x7fc64c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.466+0000 7fc64bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc64c0ccdd0 0x7fc64c0cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60046/0 (socket says 172.21.15.19:60046) 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.466+0000 7fc64bfff640 1 -- 172.21.15.19:0/2445839888 learned_addr learned my addr 172.21.15.19:0/2445839888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.467+0000 7fc651037640 1 --2- 172.21.15.19:0/2445839888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64c0cdcd0 0x7fc64c0cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.467+0000 7fc64bfff640 1 -- 172.21.15.19:0/2445839888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64c0cdcd0 msgr2=0x7fc64c0cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.467+0000 7fc64bfff640 1 --2- 172.21.15.19:0/2445839888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64c0cdcd0 0x7fc64c0cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.467+0000 7fc64bfff640 1 -- 172.21.15.19:0/2445839888 >> v1:172.21.15.112:6789/0 conn(0x7fc64c15bb20 legacy=0x7fc64c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.467+0000 7fc64bfff640 1 -- 172.21.15.19:0/2445839888 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c0cb750 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.467+0000 7fc651037640 1 --2- 172.21.15.19:0/2445839888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64c0cdcd0 0x7fc64c0cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.467+0000 7fc64bfff640 1 --2- 172.21.15.19:0/2445839888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64c0ccdd0 0x7fc64c0cf9d0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fc640009870 tx=0x7fc64002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73368b393704698c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.468+0000 7fc64b7fe640 1 -- 172.21.15.19:0/2445839888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640037070 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.468+0000 7fc64b7fe640 1 -- 172.21.15.19:0/2445839888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc640032440 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.468+0000 7fc64b7fe640 1 -- 172.21.15.19:0/2445839888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640032810 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.468+0000 7fc652a9a640 1 -- 172.21.15.19:0/2445839888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64c0ccdd0 msgr2=0x7fc64c0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.468+0000 7fc652a9a640 1 --2- 172.21.15.19:0/2445839888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64c0ccdd0 0x7fc64c0cf9d0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fc640009870 tx=0x7fc64002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.468+0000 7fc652a9a640 1 -- 172.21.15.19:0/2445839888 shutdown_connections 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.468+0000 7fc652a9a640 1 --2- 172.21.15.19:0/2445839888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64c0cdcd0 0x7fc64c0cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.468+0000 7fc652a9a640 1 --2- 172.21.15.19:0/2445839888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64c0ccdd0 0x7fc64c0cf9d0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.468+0000 7fc652a9a640 1 -- 172.21.15.19:0/2445839888 >> 172.21.15.19:0/2445839888 conn(0x7fc64c0b2390 msgr2=0x7fc64c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.468+0000 7fc652a9a640 1 -- 172.21.15.19:0/2445839888 shutdown_connections 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.468+0000 7fc652a9a640 1 -- 172.21.15.19:0/2445839888 wait complete. 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.469+0000 7fc652a9a640 1 Processor -- start 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.469+0000 7fc652a9a640 1 -- start start 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.470+0000 7fc652a9a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64c0ccdd0 0x7fc64c0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.470+0000 7fc652a9a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64c0cdcd0 0x7fc64c153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.470+0000 7fc652a9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc64c15bb20 0x7fc64c0c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.470+0000 7fc652a9a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c15bf00 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.470+0000 7fc652a9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c0cbd10 con 0x7fc64c15bb20 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.470+0000 7fc652a9a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c15c080 con 0x7fc64c0cdcd0 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.470+0000 7fc650836640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64c0ccdd0 0x7fc64c0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.470+0000 7fc650836640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64c0ccdd0 0x7fc64c0c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60058/0 (socket says 172.21.15.19:60058) 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.470+0000 7fc650836640 1 -- 172.21.15.19:0/4197422156 learned_addr learned my addr 172.21.15.19:0/4197422156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.470+0000 7fc651037640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc64c15bb20 0x7fc64c0c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.470+0000 7fc64bfff640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64c0cdcd0 0x7fc64c153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.471+0000 7fc650836640 1 -- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64c0cdcd0 msgr2=0x7fc64c153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.471+0000 7fc650836640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64c0cdcd0 0x7fc64c153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.471+0000 7fc650836640 1 -- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc64c15bb20 msgr2=0x7fc64c0c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.471+0000 7fc650836640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc64c15bb20 0x7fc64c0c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.471+0000 7fc650836640 1 -- 172.21.15.19:0/4197422156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c0b3760 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.471+0000 7fc650836640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64c0ccdd0 0x7fc64c0c9490 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fc64400c760 tx=0x7fc64400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.471+0000 7fc6497fa640 1 -- 172.21.15.19:0/4197422156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644016020 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.471+0000 7fc6497fa640 1 -- 172.21.15.19:0/4197422156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644006d50 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.471+0000 7fc652a9a640 1 -- 172.21.15.19:0/4197422156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c0ca5d0 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.471+0000 7fc6497fa640 1 -- 172.21.15.19:0/4197422156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644003bb0 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.471+0000 7fc652a9a640 1 -- 172.21.15.19:0/4197422156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c0c8c90 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.472+0000 7fc652a9a640 1 -- 172.21.15.19:0/4197422156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c073a00 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.473+0000 7fc6497fa640 1 -- 172.21.15.19:0/4197422156 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc64400a850 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.473+0000 7fc6497fa640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.473+0000 7fc6497fa640 1 -- 172.21.15.19:0/4197422156 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc644089e40 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.473+0000 7fc64bfff640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.474+0000 7fc64bfff640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc63806b4d0 0x7fc63806d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc640034000 tx=0x7fc64002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.477+0000 7fc6497fa640 1 -- 172.21.15.19:0/4197422156 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc644020020 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.658+0000 7fc652a9a640 1 -- 172.21.15.19:0/4197422156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc64c0ae990 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.659+0000 7fc6497fa640 1 -- 172.21.15.19:0/4197422156 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v142) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc644058540 con 0x7fc64c0ccdd0 2024-04-25T10:02:59.664 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:02:59.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 -- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc63806b4d0 msgr2=0x7fc63806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc63806b4d0 0x7fc63806d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc640034000 tx=0x7fc64002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:59.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 -- 172.21.15.19:0/4197422156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64c0ccdd0 msgr2=0x7fc64c0c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64c0ccdd0 0x7fc64c0c9490 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fc64400c760 tx=0x7fc64400cc20 comp rx=0 tx=0).stop 2024-04-25T10:02:59.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 -- 172.21.15.19:0/4197422156 shutdown_connections 2024-04-25T10:02:59.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc64c15bb20 0x7fc64c0c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc64c0cdcd0 0x7fc64c153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 --2- 172.21.15.19:0/4197422156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64c0ccdd0 0x7fc64c0c9490 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 -- 172.21.15.19:0/4197422156 >> 172.21.15.19:0/4197422156 conn(0x7fc64c0b2390 msgr2=0x7fc64c0aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 -- 172.21.15.19:0/4197422156 shutdown_connections 2024-04-25T10:02:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.661+0000 7fc652a9a640 1 -- 172.21.15.19:0/4197422156 wait complete. 2024-04-25T10:02:59.677 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:59.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.777+0000 7feae9ba5640 1 Processor -- start 2024-04-25T10:02:59.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.777+0000 7feae9ba5640 1 -- start start 2024-04-25T10:02:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.778+0000 7feae9ba5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feae40d34c0 0x7feae40d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.778+0000 7feae9ba5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae40d2420 con 0x7feae40d08e0 2024-04-25T10:02:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.778+0000 7feae9ba5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae40d2700 con 0x7feae415dac0 2024-04-25T10:02:59.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.778+0000 7feae9ba5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae40d0d50 con 0x7feae40d34c0 2024-04-25T10:02:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.778+0000 7feae2ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feae40d34c0 0x7feae40d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.778+0000 7feae2ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feae40d34c0 0x7feae40d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60066/0 (socket says 172.21.15.19:60066) 2024-04-25T10:02:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.778+0000 7feae3fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feae40d08e0 0x7feae40b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47618/0 (socket says 172.21.15.19:47618) 2024-04-25T10:02:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.778+0000 7feae2ffd640 1 -- 172.21.15.19:0/2137061491 learned_addr learned my addr 172.21.15.19:0/2137061491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.779+0000 7feae2ffd640 1 -- 172.21.15.19:0/2137061491 >> v1:172.21.15.112:6789/0 conn(0x7feae415dac0 legacy=0x7feae40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.779+0000 7feae2ffd640 1 -- 172.21.15.19:0/2137061491 >> v1:172.21.15.19:6789/0 conn(0x7feae40d08e0 legacy=0x7feae40b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.779+0000 7feae2ffd640 1 -- 172.21.15.19:0/2137061491 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae40d29e0 con 0x7feae40d34c0 2024-04-25T10:02:59.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.779+0000 7feae27fc640 1 -- 172.21.15.19:0/2137061491 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413778807 0 0) 0x7feae40d2420 con 0x7feae40d08e0 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.779+0000 7feae2ffd640 1 --2- 172.21.15.19:0/2137061491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feae40d34c0 0x7feae40d03a0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fead8009850 tx=0x7fead802a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2ca7be32bf6995 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.779+0000 7feae27fc640 1 -- 172.21.15.19:0/2137061491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8004230 con 0x7feae40d34c0 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.779+0000 7feae27fc640 1 -- 172.21.15.19:0/2137061491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead80043d0 con 0x7feae40d34c0 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.779+0000 7feae27fc640 1 -- 172.21.15.19:0/2137061491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8003810 con 0x7feae40d34c0 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.780+0000 7feae9ba5640 1 -- 172.21.15.19:0/2137061491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feae40d34c0 msgr2=0x7feae40d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.780+0000 7feae9ba5640 1 --2- 172.21.15.19:0/2137061491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feae40d34c0 0x7feae40d03a0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fead8009850 tx=0x7fead802a370 comp rx=0 tx=0).stop 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.780+0000 7feae9ba5640 1 -- 172.21.15.19:0/2137061491 shutdown_connections 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.780+0000 7feae9ba5640 1 --2- 172.21.15.19:0/2137061491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feae40d34c0 0x7feae40d03a0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.780+0000 7feae9ba5640 1 -- 172.21.15.19:0/2137061491 >> 172.21.15.19:0/2137061491 conn(0x7feae4061c30 msgr2=0x7feae406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.780+0000 7feae9ba5640 1 -- 172.21.15.19:0/2137061491 shutdown_connections 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.780+0000 7feae9ba5640 1 -- 172.21.15.19:0/2137061491 wait complete. 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.781+0000 7feae9ba5640 1 Processor -- start 2024-04-25T10:02:59.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.781+0000 7feae9ba5640 1 -- start start 2024-04-25T10:02:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae9ba5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae40d08e0 0x7feae40b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae9ba5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feae40d34c0 0x7feae415ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae9ba5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feae415dac0 0x7feae40ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae9ba5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae40d2100 con 0x7feae415dac0 2024-04-25T10:02:59.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae9ba5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae40d3030 con 0x7feae40d08e0 2024-04-25T10:02:59.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae9ba5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae415dea0 con 0x7feae40d34c0 2024-04-25T10:02:59.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feae415dac0 0x7feae40ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feae415dac0 0x7feae40ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60078/0 (socket says 172.21.15.19:60078) 2024-04-25T10:02:59.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae3fff640 1 -- 172.21.15.19:0/2509153933 learned_addr learned my addr 172.21.15.19:0/2509153933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:02:59.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae3fff640 1 -- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feae40d34c0 msgr2=0x7feae415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae3fff640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feae40d34c0 0x7feae415ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae3fff640 1 -- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae40d08e0 msgr2=0x7feae40b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.782+0000 7feae37fe640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae40d08e0 0x7feae40b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.783+0000 7feae3fff640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae40d08e0 0x7feae40b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.783+0000 7feae3fff640 1 -- 172.21.15.19:0/2509153933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae40b3c60 con 0x7feae415dac0 2024-04-25T10:02:59.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.783+0000 7feae3fff640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feae415dac0 0x7feae40ccc70 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c7d0 tx=0x7feadc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.783+0000 7feacbfff640 1 -- 172.21.15.19:0/2509153933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc018070 con 0x7feae415dac0 2024-04-25T10:02:59.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.783+0000 7feacbfff640 1 -- 172.21.15.19:0/2509153933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc00a640 con 0x7feae415dac0 2024-04-25T10:02:59.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.783+0000 7feae9ba5640 1 -- 172.21.15.19:0/2509153933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae40b3f40 con 0x7feae415dac0 2024-04-25T10:02:59.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.783+0000 7feacbfff640 1 -- 172.21.15.19:0/2509153933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00aa30 con 0x7feae415dac0 2024-04-25T10:02:59.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.783+0000 7feae9ba5640 1 -- 172.21.15.19:0/2509153933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae40b4500 con 0x7feae415dac0 2024-04-25T10:02:59.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.784+0000 7feae9ba5640 1 -- 172.21.15.19:0/2509153933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae4073a00 con 0x7feae415dac0 2024-04-25T10:02:59.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.784+0000 7feacbfff640 1 -- 172.21.15.19:0/2509153933 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feadc01b070 con 0x7feae415dac0 2024-04-25T10:02:59.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.785+0000 7feacbfff640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fead006b4d0 0x7fead006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.785+0000 7feacbfff640 1 -- 172.21.15.19:0/2509153933 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7feadc089a80 con 0x7feae415dac0 2024-04-25T10:02:59.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.785+0000 7feae37fe640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fead006b4d0 0x7fead006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.786+0000 7feae37fe640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fead006b4d0 0x7fead006d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fead4000f60 tx=0x7fead4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.789+0000 7feacbfff640 1 -- 172.21.15.19:0/2509153933 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feadc0586b0 con 0x7feae415dac0 2024-04-25T10:02:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.969+0000 7feae9ba5640 1 -- 172.21.15.19:0/2509153933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feae4160f40 con 0x7feae415dac0 2024-04-25T10:02:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.969+0000 7feacbfff640 1 -- 172.21.15.19:0/2509153933 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v142) v1 ==== 104+0+11 (secure 0 0 0) 0x7feadc057fb0 con 0x7feae415dac0 2024-04-25T10:02:59.974 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.971+0000 7feae9ba5640 1 -- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fead006b4d0 msgr2=0x7fead006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.971+0000 7feae9ba5640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fead006b4d0 0x7fead006d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fead4000f60 tx=0x7fead4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.971+0000 7feae9ba5640 1 -- 172.21.15.19:0/2509153933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feae415dac0 msgr2=0x7feae40ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.971+0000 7feae9ba5640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feae415dac0 0x7feae40ccc70 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c7d0 tx=0x7feadc00cc90 comp rx=0 tx=0).stop 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.971+0000 7feae9ba5640 1 -- 172.21.15.19:0/2509153933 shutdown_connections 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.971+0000 7feae9ba5640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fead006b4d0 0x7fead006d990 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.971+0000 7feae9ba5640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feae415dac0 0x7feae40ccc70 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.971+0000 7feae9ba5640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feae40d34c0 0x7feae415ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.971+0000 7feae9ba5640 1 --2- 172.21.15.19:0/2509153933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae40d08e0 0x7feae40b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.971+0000 7feae9ba5640 1 -- 172.21.15.19:0/2509153933 >> 172.21.15.19:0/2509153933 conn(0x7feae4061c30 msgr2=0x7feae40c8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.972+0000 7feae9ba5640 1 -- 172.21.15.19:0/2509153933 shutdown_connections 2024-04-25T10:02:59.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:02:59.972+0000 7feae9ba5640 1 -- 172.21.15.19:0/2509153933 wait complete. 2024-04-25T10:02:59.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.086+0000 7f74360fc640 1 Processor -- start 2024-04-25T10:03:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.086+0000 7f74360fc640 1 -- start start 2024-04-25T10:03:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.087+0000 7f74360fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f743015b980 0x7f74300cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.087+0000 7f74360fc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7430072280 con 0x7f74300cc4f0 2024-04-25T10:03:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.087+0000 7f74360fc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74300714c0 con 0x7f74300ce230 2024-04-25T10:03:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.087+0000 7f74360fc640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7430071270 con 0x7f743015b980 2024-04-25T10:03:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.087+0000 7f742f7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f74300cc4f0 0x7f74300cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47624/0 (socket says 172.21.15.19:47624) 2024-04-25T10:03:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.087+0000 7f742ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f743015b980 0x7f74300cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.087+0000 7f742f7fe640 1 -- 172.21.15.19:0/1604337462 learned_addr learned my addr 172.21.15.19:0/1604337462 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.087+0000 7f742ffff640 1 -- 172.21.15.19:0/1604337462 >> v1:172.21.15.112:6789/0 conn(0x7f74300ce230 legacy=0x7f743015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.087+0000 7f742ffff640 1 -- 172.21.15.19:0/1604337462 >> v1:172.21.15.19:6789/0 conn(0x7f74300cc4f0 legacy=0x7f74300cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.087+0000 7f742ffff640 1 -- 172.21.15.19:0/1604337462 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74300717a0 con 0x7f743015b980 2024-04-25T10:03:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.087+0000 7f742e7fc640 1 -- 172.21.15.19:0/1604337462 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427168718 0 0) 0x7f7430072280 con 0x7f74300cc4f0 2024-04-25T10:03:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.088+0000 7f742ffff640 1 --2- 172.21.15.19:0/1604337462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f743015b980 0x7f74300cc8d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f7428009d90 tx=0x7f742802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcb90cf3bf427e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.088+0000 7f742e7fc640 1 -- 172.21.15.19:0/1604337462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428007740 con 0x7f743015b980 2024-04-25T10:03:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.088+0000 7f742e7fc640 1 -- 172.21.15.19:0/1604337462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74280078e0 con 0x7f743015b980 2024-04-25T10:03:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.088+0000 7f742e7fc640 1 -- 172.21.15.19:0/1604337462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428007cb0 con 0x7f743015b980 2024-04-25T10:03:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.089+0000 7f74360fc640 1 -- 172.21.15.19:0/1604337462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f743015b980 msgr2=0x7f74300cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.089+0000 7f74360fc640 1 --2- 172.21.15.19:0/1604337462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f743015b980 0x7f74300cc8d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f7428009d90 tx=0x7f742802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.089+0000 7f74360fc640 1 -- 172.21.15.19:0/1604337462 shutdown_connections 2024-04-25T10:03:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.089+0000 7f74360fc640 1 --2- 172.21.15.19:0/1604337462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f743015b980 0x7f74300cc8d0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.089+0000 7f74360fc640 1 -- 172.21.15.19:0/1604337462 >> 172.21.15.19:0/1604337462 conn(0x7f743006abe0 msgr2=0x7f743006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.089+0000 7f74360fc640 1 -- 172.21.15.19:0/1604337462 shutdown_connections 2024-04-25T10:03:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.089+0000 7f74360fc640 1 -- 172.21.15.19:0/1604337462 wait complete. 2024-04-25T10:03:00.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.090+0000 7f74360fc640 1 Processor -- start 2024-04-25T10:03:00.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.090+0000 7f74360fc640 1 -- start start 2024-04-25T10:03:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f74360fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74300cc4f0 0x7f74300ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f74360fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74300ce230 0x7f7430147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f74360fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f743015b980 0x7f7430145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f74360fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743006e8a0 con 0x7f74300cc4f0 2024-04-25T10:03:00.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f74360fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7430071d60 con 0x7f743015b980 2024-04-25T10:03:00.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f74360fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74300ae500 con 0x7f74300ce230 2024-04-25T10:03:00.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f742f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74300cc4f0 0x7f74300ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f742f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74300cc4f0 0x7f74300ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60110/0 (socket says 172.21.15.19:60110) 2024-04-25T10:03:00.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f742f7fe640 1 -- 172.21.15.19:0/1181098929 learned_addr learned my addr 172.21.15.19:0/1181098929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:00.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f742ffff640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f743015b980 0x7f7430145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f742effd640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74300ce230 0x7f7430147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f742f7fe640 1 -- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74300ce230 msgr2=0x7f7430147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f742f7fe640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74300ce230 0x7f7430147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f742f7fe640 1 -- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f743015b980 msgr2=0x7f7430145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f742f7fe640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f743015b980 0x7f7430145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f742f7fe640 1 -- 172.21.15.19:0/1181098929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74300b0f10 con 0x7f74300cc4f0 2024-04-25T10:03:00.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.091+0000 7f742ffff640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f743015b980 0x7f7430145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:00.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.092+0000 7f742f7fe640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74300cc4f0 0x7f74300ae8c0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f742000c420 tx=0x7f742000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.092+0000 7f7417fff640 1 -- 172.21.15.19:0/1181098929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200074e0 con 0x7f74300cc4f0 2024-04-25T10:03:00.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.092+0000 7f7417fff640 1 -- 172.21.15.19:0/1181098929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74200047e0 con 0x7f74300cc4f0 2024-04-25T10:03:00.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.092+0000 7f74360fc640 1 -- 172.21.15.19:0/1181098929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743015e000 con 0x7f74300cc4f0 2024-04-25T10:03:00.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.092+0000 7f7417fff640 1 -- 172.21.15.19:0/1181098929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420004c10 con 0x7f74300cc4f0 2024-04-25T10:03:00.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.092+0000 7f74360fc640 1 -- 172.21.15.19:0/1181098929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74301543c0 con 0x7f74300cc4f0 2024-04-25T10:03:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.093+0000 7f74360fc640 1 -- 172.21.15.19:0/1181098929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7430073a00 con 0x7f74300cc4f0 2024-04-25T10:03:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.093+0000 7f7417fff640 1 -- 172.21.15.19:0/1181098929 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7420011070 con 0x7f74300cc4f0 2024-04-25T10:03:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.094+0000 7f7417fff640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f741c06b4d0 0x7f741c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.094+0000 7f7417fff640 1 -- 172.21.15.19:0/1181098929 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7420089ff0 con 0x7f74300cc4f0 2024-04-25T10:03:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.094+0000 7f742effd640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f741c06b4d0 0x7f741c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.094+0000 7f742effd640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f741c06b4d0 0x7f741c06d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7430144ac0 tx=0x7f7424009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.098+0000 7f7417fff640 1 -- 172.21.15.19:0/1181098929 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f742000e280 con 0x7f74300cc4f0 2024-04-25T10:03:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.281+0000 7f74360fc640 1 -- 172.21.15.19:0/1181098929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f743006d3a0 con 0x7f74300cc4f0 2024-04-25T10:03:00.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.282+0000 7f7417fff640 1 -- 172.21.15.19:0/1181098929 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v142) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7420058ca0 con 0x7f74300cc4f0 2024-04-25T10:03:00.287 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:03:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 -- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f741c06b4d0 msgr2=0x7f741c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f741c06b4d0 0x7f741c06d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7430144ac0 tx=0x7f7424009040 comp rx=0 tx=0).stop 2024-04-25T10:03:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 -- 172.21.15.19:0/1181098929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74300cc4f0 msgr2=0x7f74300ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74300cc4f0 0x7f74300ae8c0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f742000c420 tx=0x7f742000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 -- 172.21.15.19:0/1181098929 shutdown_connections 2024-04-25T10:03:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f741c06b4d0 0x7f741c06d990 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f743015b980 0x7f7430145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74300ce230 0x7f7430147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 --2- 172.21.15.19:0/1181098929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74300cc4f0 0x7f74300ae8c0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 -- 172.21.15.19:0/1181098929 >> 172.21.15.19:0/1181098929 conn(0x7f743006abe0 msgr2=0x7f74300b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 -- 172.21.15.19:0/1181098929 shutdown_connections 2024-04-25T10:03:00.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.284+0000 7f74360fc640 1 -- 172.21.15.19:0/1181098929 wait complete. 2024-04-25T10:03:00.300 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.395+0000 7f20db2d2640 1 Processor -- start 2024-04-25T10:03:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.395+0000 7f20db2d2640 1 -- start start 2024-04-25T10:03:00.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.396+0000 7f20db2d2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20d415db70 0x7f20d4162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.396+0000 7f20db2d2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d40b6cb0 0x7f20d40b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.396+0000 7f20db2d2640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d4162690 con 0x7f20d40b5a60 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.396+0000 7f20db2d2640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d40d3800 con 0x7f20d415db70 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.396+0000 7f20db2d2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40ceee0 con 0x7f20d40b6cb0 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.396+0000 7f20d906e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20d415db70 0x7f20d4162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.396+0000 7f20d986f640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f20d40b5a60 0x7f20d406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47630/0 (socket says 172.21.15.19:47630) 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.396+0000 7f20d986f640 1 -- 172.21.15.19:0/2094405438 learned_addr learned my addr 172.21.15.19:0/2094405438 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.397+0000 7f20d906e640 1 -- 172.21.15.19:0/2094405438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d40b6cb0 msgr2=0x7f20d40b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.397+0000 7f20d886d640 1 --2- 172.21.15.19:0/2094405438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d40b6cb0 0x7f20d40b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.397+0000 7f20d3fff640 1 -- 172.21.15.19:0/2094405438 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128870513 0 0) 0x7f20d4162690 con 0x7f20d40b5a60 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.397+0000 7f20d906e640 1 --2- 172.21.15.19:0/2094405438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d40b6cb0 0x7f20d40b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.397+0000 7f20d906e640 1 -- 172.21.15.19:0/2094405438 >> v1:172.21.15.19:6789/0 conn(0x7f20d40b5a60 legacy=0x7f20d406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.397+0000 7f20d906e640 1 -- 172.21.15.19:0/2094405438 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d40d1d10 con 0x7f20d415db70 2024-04-25T10:03:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.397+0000 7f20d906e640 1 --2- 172.21.15.19:0/2094405438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d415db70 0x7f20d4162150 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0097a0 tx=0x7f20cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5485f08efad585 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.398+0000 7f20d3fff640 1 -- 172.21.15.19:0/2094405438 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc037070 con 0x7f20d415db70 2024-04-25T10:03:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.398+0000 7f20d3fff640 1 -- 172.21.15.19:0/2094405438 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc032440 con 0x7f20d415db70 2024-04-25T10:03:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.398+0000 7f20d3fff640 1 -- 172.21.15.19:0/2094405438 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc032810 con 0x7f20d415db70 2024-04-25T10:03:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.398+0000 7f20db2d2640 1 -- 172.21.15.19:0/2094405438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d415db70 msgr2=0x7f20d4162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.398+0000 7f20db2d2640 1 --2- 172.21.15.19:0/2094405438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d415db70 0x7f20d4162150 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0097a0 tx=0x7f20cc02a180 comp rx=0 tx=0).stop 2024-04-25T10:03:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.398+0000 7f20db2d2640 1 -- 172.21.15.19:0/2094405438 shutdown_connections 2024-04-25T10:03:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.398+0000 7f20db2d2640 1 --2- 172.21.15.19:0/2094405438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d40b6cb0 0x7f20d40b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.398+0000 7f20db2d2640 1 --2- 172.21.15.19:0/2094405438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d415db70 0x7f20d4162150 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.398+0000 7f20db2d2640 1 -- 172.21.15.19:0/2094405438 >> 172.21.15.19:0/2094405438 conn(0x7f20d40d0520 msgr2=0x7f20d40d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.398+0000 7f20db2d2640 1 -- 172.21.15.19:0/2094405438 shutdown_connections 2024-04-25T10:03:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.398+0000 7f20db2d2640 1 -- 172.21.15.19:0/2094405438 wait complete. 2024-04-25T10:03:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.399+0000 7f20db2d2640 1 Processor -- start 2024-04-25T10:03:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.399+0000 7f20db2d2640 1 -- start start 2024-04-25T10:03:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.400+0000 7f20db2d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20d40b5a60 0x7f20d40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.400+0000 7f20db2d2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d40b6cb0 0x7f20d40cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.400+0000 7f20db2d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d415db70 0x7f20d40cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.400+0000 7f20db2d2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d415df50 con 0x7f20d415db70 2024-04-25T10:03:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.400+0000 7f20db2d2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40d22d0 con 0x7f20d40b5a60 2024-04-25T10:03:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.400+0000 7f20db2d2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40ce530 con 0x7f20d40b6cb0 2024-04-25T10:03:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.400+0000 7f20d986f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d415db70 0x7f20d40cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.400+0000 7f20d986f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d415db70 0x7f20d40cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60120/0 (socket says 172.21.15.19:60120) 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.400+0000 7f20d986f640 1 -- 172.21.15.19:0/2739779068 learned_addr learned my addr 172.21.15.19:0/2739779068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.400+0000 7f20d886d640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d40b6cb0 0x7f20d40cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.400+0000 7f20d906e640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20d40b5a60 0x7f20d40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.401+0000 7f20d986f640 1 -- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d40b6cb0 msgr2=0x7f20d40cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.401+0000 7f20d986f640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d40b6cb0 0x7f20d40cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.401+0000 7f20d986f640 1 -- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20d40b5a60 msgr2=0x7f20d40c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.401+0000 7f20d986f640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20d40b5a60 0x7f20d40c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.401+0000 7f20d986f640 1 -- 172.21.15.19:0/2739779068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d4147370 con 0x7f20d415db70 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.401+0000 7f20d986f640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d415db70 0x7f20d40cda50 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c780 tx=0x7f20c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.401+0000 7f20d1ffb640 1 -- 172.21.15.19:0/2739779068 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4018070 con 0x7f20d415db70 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.401+0000 7f20d1ffb640 1 -- 172.21.15.19:0/2739779068 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c400a640 con 0x7f20d415db70 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.401+0000 7f20db2d2640 1 -- 172.21.15.19:0/2739779068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d4147650 con 0x7f20d415db70 2024-04-25T10:03:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.401+0000 7f20d1ffb640 1 -- 172.21.15.19:0/2739779068 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c400aa30 con 0x7f20d415db70 2024-04-25T10:03:00.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.401+0000 7f20db2d2640 1 -- 172.21.15.19:0/2739779068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d4147c10 con 0x7f20d415db70 2024-04-25T10:03:00.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.402+0000 7f20db2d2640 1 -- 172.21.15.19:0/2739779068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d4073990 con 0x7f20d415db70 2024-04-25T10:03:00.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.402+0000 7f20d1ffb640 1 -- 172.21.15.19:0/2739779068 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20c401b070 con 0x7f20d415db70 2024-04-25T10:03:00.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.403+0000 7f20d1ffb640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20c006b4d0 0x7f20c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.403+0000 7f20d1ffb640 1 -- 172.21.15.19:0/2739779068 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f20c40898a0 con 0x7f20d415db70 2024-04-25T10:03:00.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.403+0000 7f20d906e640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20c006b4d0 0x7f20c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.404+0000 7f20d906e640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20c006b4d0 0x7f20c006d990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f20cc034000 tx=0x7f20cc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.407+0000 7f20d1ffb640 1 -- 172.21.15.19:0/2739779068 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20c4057f10 con 0x7f20d415db70 2024-04-25T10:03:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.622+0000 7f20db2d2640 1 -- 172.21.15.19:0/2739779068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20d40ce310 con 0x7f20d415db70 2024-04-25T10:03:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.623+0000 7f20d1ffb640 1 -- 172.21.15.19:0/2739779068 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f20c405bfb0 con 0x7f20d415db70 2024-04-25T10:03:00.628 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:00.628 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":94,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7466,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/590882657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":590882657},{"type":"v1","addr":"172.21.15.112:6836","nonce":590882657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":9744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1740426538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1740426538},{"type":"v1","addr":"172.21.15.19:6833","nonce":1740426538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":94,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:02:34.992088+0000","modified":"2024-04-25T10:02:54.722246+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9750":{"gid":9750,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/525024117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":525024117},{"type":"v1","addr":"172.21.15.19:6835","nonce":525024117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9762":{"gid":9762,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2648279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2648279035},{"type":"v1","addr":"172.21.15.112:6837","nonce":2648279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:03:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 94 2024-04-25T10:03:00.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 -- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20c006b4d0 msgr2=0x7f20c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20c006b4d0 0x7f20c006d990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f20cc034000 tx=0x7f20cc02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:00.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 -- 172.21.15.19:0/2739779068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d415db70 msgr2=0x7f20d40cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d415db70 0x7f20d40cda50 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c780 tx=0x7f20c400cc40 comp rx=0 tx=0).stop 2024-04-25T10:03:00.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 -- 172.21.15.19:0/2739779068 shutdown_connections 2024-04-25T10:03:00.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f20c006b4d0 0x7f20c006d990 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d415db70 0x7f20d40cda50 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20d40b6cb0 0x7f20d40cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 --2- 172.21.15.19:0/2739779068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20d40b5a60 0x7f20d40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 -- 172.21.15.19:0/2739779068 >> 172.21.15.19:0/2739779068 conn(0x7f20d40d0520 msgr2=0x7f20d4158880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 -- 172.21.15.19:0/2739779068 shutdown_connections 2024-04-25T10:03:00.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.625+0000 7f20db2d2640 1 -- 172.21.15.19:0/2739779068 wait complete. 2024-04-25T10:03:00.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.738+0000 7fcf97142640 1 Processor -- start 2024-04-25T10:03:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.738+0000 7fcf97142640 1 -- start start 2024-04-25T10:03:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf97142640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf9015dbe0 0x7fcf900d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf97142640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf900b92a0 0x7fcf900d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf97142640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf900b97a0 con 0x7fcf900b2cb0 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf97142640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9015dfc0 con 0x7fcf9015dbe0 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf97142640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf900d1a00 con 0x7fcf900b92a0 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf94ede640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf9015dbe0 0x7fcf900d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf8ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcf900b2cb0 0x7fcf900b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47646/0 (socket says 172.21.15.19:47646) 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf8ffff640 1 -- 172.21.15.19:0/2351188886 learned_addr learned my addr 172.21.15.19:0/2351188886 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf956df640 1 --2- 172.21.15.19:0/2351188886 >> v2:172.21.15.112:3300/0 conn(0x7fcf900b92a0 0x7fcf900d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf94ede640 1 -- 172.21.15.19:0/2351188886 >> v2:172.21.15.112:3300/0 conn(0x7fcf900b92a0 msgr2=0x7fcf900d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf94ede640 1 --2- 172.21.15.19:0/2351188886 >> v2:172.21.15.112:3300/0 conn(0x7fcf900b92a0 0x7fcf900d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf94ede640 1 -- 172.21.15.19:0/2351188886 >> v1:172.21.15.19:6789/0 conn(0x7fcf900b2cb0 legacy=0x7fcf900b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf94ede640 1 -- 172.21.15.19:0/2351188886 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf900d1c50 con 0x7fcf9015dbe0 2024-04-25T10:03:00.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.739+0000 7fcf8f7fe640 1 -- 172.21.15.19:0/2351188886 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293680790 0 0) 0x7fcf900b97a0 con 0x7fcf900b2cb0 2024-04-25T10:03:00.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.740+0000 7fcf94ede640 1 --2- 172.21.15.19:0/2351188886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf9015dbe0 0x7fcf900d0910 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fcf900cfb90 tx=0x7fcf8402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a0eb5ae6f69d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.740+0000 7fcf8f7fe640 1 -- 172.21.15.19:0/2351188886 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84037070 con 0x7fcf9015dbe0 2024-04-25T10:03:00.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.740+0000 7fcf8f7fe640 1 -- 172.21.15.19:0/2351188886 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf84032440 con 0x7fcf9015dbe0 2024-04-25T10:03:00.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.740+0000 7fcf8f7fe640 1 -- 172.21.15.19:0/2351188886 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8402ae60 con 0x7fcf9015dbe0 2024-04-25T10:03:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.740+0000 7fcf97142640 1 -- 172.21.15.19:0/2351188886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf9015dbe0 msgr2=0x7fcf900d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.740+0000 7fcf97142640 1 --2- 172.21.15.19:0/2351188886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf9015dbe0 0x7fcf900d0910 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fcf900cfb90 tx=0x7fcf8402a110 comp rx=0 tx=0).stop 2024-04-25T10:03:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.741+0000 7fcf97142640 1 -- 172.21.15.19:0/2351188886 shutdown_connections 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.741+0000 7fcf97142640 1 --2- 172.21.15.19:0/2351188886 >> v2:172.21.15.112:3300/0 conn(0x7fcf900b92a0 0x7fcf900d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.741+0000 7fcf97142640 1 --2- 172.21.15.19:0/2351188886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf9015dbe0 0x7fcf900d0910 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.741+0000 7fcf97142640 1 -- 172.21.15.19:0/2351188886 >> 172.21.15.19:0/2351188886 conn(0x7fcf90061a20 msgr2=0x7fcf90061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.741+0000 7fcf97142640 1 -- 172.21.15.19:0/2351188886 shutdown_connections 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.741+0000 7fcf97142640 1 -- 172.21.15.19:0/2351188886 wait complete. 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.742+0000 7fcf97142640 1 Processor -- start 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.742+0000 7fcf97142640 1 -- start start 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf97142640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf900b2cb0 0x7fcf900cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf97142640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf900b92a0 0x7fcf90151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf97142640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf9015dbe0 0x7fcf900b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf97142640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900b6ab0 con 0x7fcf900b92a0 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf97142640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900d2210 con 0x7fcf9015dbe0 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf97142640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900b6c30 con 0x7fcf900b2cb0 2024-04-25T10:03:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf900b92a0 0x7fcf90151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf900b92a0 0x7fcf90151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60138/0 (socket says 172.21.15.19:60138) 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8ffff640 1 -- 172.21.15.19:0/2606571244 learned_addr learned my addr 172.21.15.19:0/2606571244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf956df640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf9015dbe0 0x7fcf900b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf94ede640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf900b2cb0 0x7fcf900cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8ffff640 1 -- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf900b2cb0 msgr2=0x7fcf900cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8ffff640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf900b2cb0 0x7fcf900cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8ffff640 1 -- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf9015dbe0 msgr2=0x7fcf900b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8ffff640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf9015dbe0 0x7fcf900b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8ffff640 1 -- 172.21.15.19:0/2606571244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf901f4320 con 0x7fcf900b92a0 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf94ede640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf900b2cb0 0x7fcf900cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8ffff640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf900b92a0 0x7fcf90151cd0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00c4b0 tx=0x7fcf7c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8d7fa640 1 -- 172.21.15.19:0/2606571244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c018070 con 0x7fcf900b92a0 2024-04-25T10:03:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8d7fa640 1 -- 172.21.15.19:0/2606571244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c007500 con 0x7fcf900b92a0 2024-04-25T10:03:00.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf97142640 1 -- 172.21.15.19:0/2606571244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf900b3e30 con 0x7fcf900b92a0 2024-04-25T10:03:00.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.743+0000 7fcf8d7fa640 1 -- 172.21.15.19:0/2606571244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c004a20 con 0x7fcf900b92a0 2024-04-25T10:03:00.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.744+0000 7fcf97142640 1 -- 172.21.15.19:0/2606571244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf900b7390 con 0x7fcf900b92a0 2024-04-25T10:03:00.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.745+0000 7fcf97142640 1 -- 172.21.15.19:0/2606571244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf900b6ea0 con 0x7fcf900b92a0 2024-04-25T10:03:00.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.745+0000 7fcf8d7fa640 1 -- 172.21.15.19:0/2606571244 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf7c004bc0 con 0x7fcf900b92a0 2024-04-25T10:03:00.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.746+0000 7fcf8d7fa640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf7406b500 0x7fcf7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.746+0000 7fcf8d7fa640 1 -- 172.21.15.19:0/2606571244 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcf7c0895e0 con 0x7fcf900b92a0 2024-04-25T10:03:00.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.746+0000 7fcf94ede640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf7406b500 0x7fcf7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.747+0000 7fcf94ede640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf7406b500 0x7fcf7406d9c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcf84034000 tx=0x7fcf8402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.752+0000 7fcf8d7fa640 1 -- 172.21.15.19:0/2606571244 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf7c057d20 con 0x7fcf900b92a0 2024-04-25T10:03:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.935+0000 7fcf97142640 1 -- 172.21.15.19:0/2606571244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf90073990 con 0x7fcf900b92a0 2024-04-25T10:03:00.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.937+0000 7fcf8d7fa640 1 -- 172.21.15.19:0/2606571244 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+17037 (secure 0 0 0) 0x7fcf7c05bdc0 con 0x7fcf900b92a0 2024-04-25T10:03:00.941 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:00.942 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":142,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:48.692096+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:33.312366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:33.971764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.1","mappings":[{"from":3,"to":0}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 -- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf7406b500 msgr2=0x7fcf7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf7406b500 0x7fcf7406d9c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcf84034000 tx=0x7fcf8402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 -- 172.21.15.19:0/2606571244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf900b92a0 msgr2=0x7fcf90151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf900b92a0 0x7fcf90151cd0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00c4b0 tx=0x7fcf7c00c970 comp rx=0 tx=0).stop 2024-04-25T10:03:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 -- 172.21.15.19:0/2606571244 shutdown_connections 2024-04-25T10:03:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf7406b500 0x7fcf7406d9c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf9015dbe0 0x7fcf900b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf900b92a0 0x7fcf90151cd0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 --2- 172.21.15.19:0/2606571244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf900b2cb0 0x7fcf900cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 -- 172.21.15.19:0/2606571244 >> 172.21.15.19:0/2606571244 conn(0x7fcf90061a20 msgr2=0x7fcf9006c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 -- 172.21.15.19:0/2606571244 shutdown_connections 2024-04-25T10:03:00.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:00.939+0000 7fcf97142640 1 -- 172.21.15.19:0/2606571244 wait complete. 2024-04-25T10:03:00.955 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:03:00.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:01.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.052+0000 7f6b730bd640 1 Processor -- start 2024-04-25T10:03:01.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.052+0000 7f6b730bd640 1 -- start start 2024-04-25T10:03:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.053+0000 7f6b730bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b6c15dc70 0x7f6b6c0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.053+0000 7f6b730bd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b6c06ce10 0x7f6b6c0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.053+0000 7f6b730bd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c06c220 con 0x7f6b6c09a450 2024-04-25T10:03:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.053+0000 7f6b730bd640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c09a8b0 con 0x7f6b6c06ce10 2024-04-25T10:03:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.053+0000 7f6b730bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0d0c90 con 0x7f6b6c15dc70 2024-04-25T10:03:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.053+0000 7f6b6bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6b6c09a450 0x7f6b6c06c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47650/0 (socket says 172.21.15.19:47650) 2024-04-25T10:03:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.053+0000 7f6b7165a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b6c06ce10 0x7f6b6c0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.053+0000 7f6b6bfff640 1 -- 172.21.15.19:0/656066095 learned_addr learned my addr 172.21.15.19:0/656066095 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.053+0000 7f6b70e59640 1 --2- 172.21.15.19:0/656066095 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b6c15dc70 0x7f6b6c0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.054+0000 7f6b7165a640 1 -- 172.21.15.19:0/656066095 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b6c15dc70 msgr2=0x7f6b6c0d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.054+0000 7f6b7165a640 1 --2- 172.21.15.19:0/656066095 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b6c15dc70 0x7f6b6c0d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.054+0000 7f6b7165a640 1 -- 172.21.15.19:0/656066095 >> v1:172.21.15.19:6789/0 conn(0x7f6b6c09a450 legacy=0x7f6b6c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.054+0000 7f6b6b7fe640 1 -- 172.21.15.19:0/656066095 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789117648 0 0) 0x7f6b6c06c220 con 0x7f6b6c09a450 2024-04-25T10:03:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.054+0000 7f6b7165a640 1 -- 172.21.15.19:0/656066095 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0d0080 con 0x7f6b6c06ce10 2024-04-25T10:03:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.054+0000 7f6b7165a640 1 --2- 172.21.15.19:0/656066095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6c06ce10 0x7f6b6c0cf9c0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009d90 tx=0x7f6b6402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd96fab65017788 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.054+0000 7f6b6b7fe640 1 -- 172.21.15.19:0/656066095 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64007740 con 0x7f6b6c06ce10 2024-04-25T10:03:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.054+0000 7f6b6b7fe640 1 -- 172.21.15.19:0/656066095 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b640078e0 con 0x7f6b6c06ce10 2024-04-25T10:03:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.054+0000 7f6b6b7fe640 1 -- 172.21.15.19:0/656066095 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64007cb0 con 0x7f6b6c06ce10 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.055+0000 7f6b730bd640 1 -- 172.21.15.19:0/656066095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6c06ce10 msgr2=0x7f6b6c0cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.055+0000 7f6b730bd640 1 --2- 172.21.15.19:0/656066095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6c06ce10 0x7f6b6c0cf9c0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009d90 tx=0x7f6b6402a660 comp rx=0 tx=0).stop 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.055+0000 7f6b730bd640 1 -- 172.21.15.19:0/656066095 shutdown_connections 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.055+0000 7f6b730bd640 1 --2- 172.21.15.19:0/656066095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6c06ce10 0x7f6b6c0cf9c0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.055+0000 7f6b730bd640 1 --2- 172.21.15.19:0/656066095 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b6c15dc70 0x7f6b6c0d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.055+0000 7f6b730bd640 1 -- 172.21.15.19:0/656066095 >> 172.21.15.19:0/656066095 conn(0x7f6b6c0b3270 msgr2=0x7f6b6c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.055+0000 7f6b730bd640 1 -- 172.21.15.19:0/656066095 shutdown_connections 2024-04-25T10:03:01.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.055+0000 7f6b730bd640 1 -- 172.21.15.19:0/656066095 wait complete. 2024-04-25T10:03:01.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.056+0000 7f6b730bd640 1 Processor -- start 2024-04-25T10:03:01.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.056+0000 7f6b730bd640 1 -- start start 2024-04-25T10:03:01.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.057+0000 7f6b730bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b6c06ce10 0x7f6b6c09d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.057+0000 7f6b730bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6c09a450 0x7f6b6c1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.057+0000 7f6b730bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6c15dc70 0x7f6b6c153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.057+0000 7f6b730bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0d0920 con 0x7f6b6c09a450 2024-04-25T10:03:01.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.057+0000 7f6b730bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0d0640 con 0x7f6b6c15dc70 2024-04-25T10:03:01.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.057+0000 7f6b730bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c15e050 con 0x7f6b6c06ce10 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.057+0000 7f6b6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6c09a450 0x7f6b6c1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.057+0000 7f6b6bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6c09a450 0x7f6b6c1539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60156/0 (socket says 172.21.15.19:60156) 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.057+0000 7f6b6bfff640 1 -- 172.21.15.19:0/2825413465 learned_addr learned my addr 172.21.15.19:0/2825413465 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b70e59640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b6c06ce10 0x7f6b6c09d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b6bfff640 1 -- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b6c06ce10 msgr2=0x7f6b6c09d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b7165a640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6c15dc70 0x7f6b6c153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b6bfff640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b6c06ce10 0x7f6b6c09d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b6bfff640 1 -- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6c15dc70 msgr2=0x7f6b6c153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b6bfff640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6c15dc70 0x7f6b6c153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b6bfff640 1 -- 172.21.15.19:0/2825413465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c09c290 con 0x7f6b6c09a450 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b6bfff640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6c09a450 0x7f6b6c1539b0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000c4d0 tx=0x7f6b6000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b697fa640 1 -- 172.21.15.19:0/2825413465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60018070 con 0x7f6b6c09a450 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b697fa640 1 -- 172.21.15.19:0/2825413465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b60007500 con 0x7f6b6c09a450 2024-04-25T10:03:01.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b730bd640 1 -- 172.21.15.19:0/2825413465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c09c570 con 0x7f6b6c09a450 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b697fa640 1 -- 172.21.15.19:0/2825413465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60004a20 con 0x7f6b6c09a450 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.058+0000 7f6b730bd640 1 -- 172.21.15.19:0/2825413465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c09cb30 con 0x7f6b6c09a450 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.059+0000 7f6b730bd640 1 -- 172.21.15.19:0/2825413465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6c073990 con 0x7f6b6c09a450 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.060+0000 7f6b697fa640 1 -- 172.21.15.19:0/2825413465 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b60004bc0 con 0x7f6b6c09a450 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.060+0000 7f6b697fa640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b5806b4d0 0x7f6b5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.060+0000 7f6b697fa640 1 -- 172.21.15.19:0/2825413465 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f6b600897c0 con 0x7f6b6c09a450 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.060+0000 7f6b70e59640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b5806b4d0 0x7f6b5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.061+0000 7f6b70e59640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b5806b4d0 0x7f6b5806d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c006fd0 tx=0x7f6b5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.064+0000 7f6b697fa640 1 -- 172.21.15.19:0/2825413465 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b60058400 con 0x7f6b6c09a450 2024-04-25T10:03:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.282+0000 7f6b730bd640 1 -- 172.21.15.19:0/2825413465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b6c0cb870 con 0x7f6b6c09a450 2024-04-25T10:03:01.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.283+0000 7f6b697fa640 1 -- 172.21.15.19:0/2825413465 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f6b60057d00 con 0x7f6b6c09a450 2024-04-25T10:03:01.288 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:01.288 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":94,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7466,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/590882657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":590882657},{"type":"v1","addr":"172.21.15.112:6836","nonce":590882657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":9744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1740426538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1740426538},{"type":"v1","addr":"172.21.15.19:6833","nonce":1740426538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":94,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:02:34.992088+0000","modified":"2024-04-25T10:02:54.722246+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9750":{"gid":9750,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/525024117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":525024117},{"type":"v1","addr":"172.21.15.19:6835","nonce":525024117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9762":{"gid":9762,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2648279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2648279035},{"type":"v1","addr":"172.21.15.112:6837","nonce":2648279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:03:01.288 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 94 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 -- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b5806b4d0 msgr2=0x7f6b5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b5806b4d0 0x7f6b5806d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c006fd0 tx=0x7f6b5c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 -- 172.21.15.19:0/2825413465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6c09a450 msgr2=0x7f6b6c1539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6c09a450 0x7f6b6c1539b0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000c4d0 tx=0x7f6b6000c990 comp rx=0 tx=0).stop 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 -- 172.21.15.19:0/2825413465 shutdown_connections 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b5806b4d0 0x7f6b5806d990 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6c15dc70 0x7f6b6c153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6c09a450 0x7f6b6c1539b0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 --2- 172.21.15.19:0/2825413465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b6c06ce10 0x7f6b6c09d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 -- 172.21.15.19:0/2825413465 >> 172.21.15.19:0/2825413465 conn(0x7f6b6c0b3270 msgr2=0x7f6b6c0cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 -- 172.21.15.19:0/2825413465 shutdown_connections 2024-04-25T10:03:01.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.285+0000 7f6b730bd640 1 -- 172.21.15.19:0/2825413465 wait complete. 2024-04-25T10:03:01.301 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:01.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.396+0000 7f1571390640 1 Processor -- start 2024-04-25T10:03:01.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.396+0000 7f1571390640 1 -- start start 2024-04-25T10:03:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.397+0000 7f1571390640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f156c15ba00 0x7f156c09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.397+0000 7f1571390640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f156c1602e0 0x7f156c0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.397+0000 7f1571390640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0cd710 con 0x7f156c15f5f0 2024-04-25T10:03:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.397+0000 7f1571390640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c099b70 con 0x7f156c15ba00 2024-04-25T10:03:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.397+0000 7f1571390640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c099e50 con 0x7f156c1602e0 2024-04-25T10:03:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.397+0000 7f156affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f156c15ba00 0x7f156c09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.397+0000 7f156b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f156c15f5f0 0x7f156c0c52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47660/0 (socket says 172.21.15.19:47660) 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.397+0000 7f156b7fe640 1 -- 172.21.15.19:0/2917564444 learned_addr learned my addr 172.21.15.19:0/2917564444 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.397+0000 7f156a7fc640 1 --2- 172.21.15.19:0/2917564444 >> v2:172.21.15.112:3300/0 conn(0x7f156c1602e0 0x7f156c0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.397+0000 7f1569ffb640 1 -- 172.21.15.19:0/2917564444 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751264424 0 0) 0x7f155c001490 con 0x7f156c15f5f0 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.398+0000 7f1569ffb640 1 -- 172.21.15.19:0/2917564444 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1554003520 con 0x7f156c15f5f0 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.398+0000 7f156affd640 1 -- 172.21.15.19:0/2917564444 >> v2:172.21.15.112:3300/0 conn(0x7f156c1602e0 msgr2=0x7f156c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.398+0000 7f156affd640 1 --2- 172.21.15.19:0/2917564444 >> v2:172.21.15.112:3300/0 conn(0x7f156c1602e0 0x7f156c0ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.398+0000 7f156affd640 1 -- 172.21.15.19:0/2917564444 >> v1:172.21.15.19:6789/0 conn(0x7f156c15f5f0 legacy=0x7f156c0c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.398+0000 7f156affd640 1 -- 172.21.15.19:0/2917564444 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c09a130 con 0x7f156c15ba00 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.398+0000 7f156affd640 1 --2- 172.21.15.19:0/2917564444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f156c15ba00 0x7f156c09a930 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f156c0b7830 tx=0x7f156402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2addd28866db643a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.398+0000 7f1569ffb640 1 -- 172.21.15.19:0/2917564444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564037070 con 0x7f156c15ba00 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.398+0000 7f1569ffb640 1 -- 172.21.15.19:0/2917564444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156402add0 con 0x7f156c15ba00 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.398+0000 7f1569ffb640 1 -- 172.21.15.19:0/2917564444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564032660 con 0x7f156c15ba00 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.399+0000 7f1571390640 1 -- 172.21.15.19:0/2917564444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f156c15ba00 msgr2=0x7f156c09a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.399+0000 7f1571390640 1 --2- 172.21.15.19:0/2917564444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f156c15ba00 0x7f156c09a930 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f156c0b7830 tx=0x7f156402a080 comp rx=0 tx=0).stop 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.399+0000 7f1571390640 1 -- 172.21.15.19:0/2917564444 shutdown_connections 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.399+0000 7f1571390640 1 --2- 172.21.15.19:0/2917564444 >> v2:172.21.15.112:3300/0 conn(0x7f156c1602e0 0x7f156c0ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.399+0000 7f1571390640 1 --2- 172.21.15.19:0/2917564444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f156c15ba00 0x7f156c09a930 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.399+0000 7f1571390640 1 -- 172.21.15.19:0/2917564444 >> 172.21.15.19:0/2917564444 conn(0x7f156c0b3270 msgr2=0x7f156c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.399+0000 7f1571390640 1 -- 172.21.15.19:0/2917564444 shutdown_connections 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.399+0000 7f1571390640 1 -- 172.21.15.19:0/2917564444 wait complete. 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.400+0000 7f1571390640 1 Processor -- start 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.400+0000 7f1571390640 1 -- start start 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f1571390640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f156c15ba00 0x7f156c1e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f1571390640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f156c15f5f0 0x7f156c149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f1571390640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f156c1602e0 0x7f156c149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f1571390640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c15bde0 con 0x7f156c15ba00 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f1571390640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0cd2e0 con 0x7f156c1602e0 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f1571390640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c147650 con 0x7f156c15f5f0 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f156affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f156c15ba00 0x7f156c1e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f156affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f156c15ba00 0x7f156c1e53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60174/0 (socket says 172.21.15.19:60174) 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f156affd640 1 -- 172.21.15.19:0/3101300798 learned_addr learned my addr 172.21.15.19:0/3101300798 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f156b7fe640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f156c1602e0 0x7f156c149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f156a7fc640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f156c15f5f0 0x7f156c149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f156affd640 1 -- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f156c15f5f0 msgr2=0x7f156c149950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f156affd640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f156c15f5f0 0x7f156c149950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f156affd640 1 -- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f156c1602e0 msgr2=0x7f156c149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f156affd640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f156c1602e0 0x7f156c149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f156affd640 1 -- 172.21.15.19:0/3101300798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c1e5a00 con 0x7f156c15ba00 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.401+0000 7f156b7fe640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f156c1602e0 0x7f156c149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.402+0000 7f156affd640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f156c15ba00 0x7f156c1e53d0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f1564000c00 tx=0x7f15640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.402+0000 7f15537fe640 1 -- 172.21.15.19:0/3101300798 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564037040 con 0x7f156c15ba00 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.402+0000 7f15537fe640 1 -- 172.21.15.19:0/3101300798 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1564042070 con 0x7f156c15ba00 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.402+0000 7f1571390640 1 -- 172.21.15.19:0/3101300798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c1478f0 con 0x7f156c15ba00 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.402+0000 7f15537fe640 1 -- 172.21.15.19:0/3101300798 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156403d660 con 0x7f156c15ba00 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.402+0000 7f1571390640 1 -- 172.21.15.19:0/3101300798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c147eb0 con 0x7f156c15ba00 2024-04-25T10:03:01.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.403+0000 7f1571390640 1 -- 172.21.15.19:0/3101300798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c073990 con 0x7f156c15ba00 2024-04-25T10:03:01.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.403+0000 7f15537fe640 1 -- 172.21.15.19:0/3101300798 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f156403d800 con 0x7f156c15ba00 2024-04-25T10:03:01.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.404+0000 7f15537fe640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f155806b4d0 0x7f155806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.404+0000 7f15537fe640 1 -- 172.21.15.19:0/3101300798 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f15640aa8d0 con 0x7f156c15ba00 2024-04-25T10:03:01.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.404+0000 7f156a7fc640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f155806b4d0 0x7f155806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.405+0000 7f156a7fc640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f155806b4d0 0x7f155806d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1560009ad0 tx=0x7f1560008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.408+0000 7f15537fe640 1 -- 172.21.15.19:0/3101300798 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1564079620 con 0x7f156c15ba00 2024-04-25T10:03:01.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.624+0000 7f1571390640 1 -- 172.21.15.19:0/3101300798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f156c0b40a0 con 0x7f156c15ba00 2024-04-25T10:03:01.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.624+0000 7f15537fe640 1 -- 172.21.15.19:0/3101300798 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f1564078f20 con 0x7f156c15ba00 2024-04-25T10:03:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:01.630 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":94,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7466,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/590882657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":590882657},{"type":"v1","addr":"172.21.15.112:6836","nonce":590882657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":9744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1740426538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1740426538},{"type":"v1","addr":"172.21.15.19:6833","nonce":1740426538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":94,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:02:34.992088+0000","modified":"2024-04-25T10:02:54.722246+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9750":{"gid":9750,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/525024117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":525024117},{"type":"v1","addr":"172.21.15.19:6835","nonce":525024117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9762":{"gid":9762,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/2648279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2648279035},{"type":"v1","addr":"172.21.15.112:6837","nonce":2648279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:03:01.630 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 94 2024-04-25T10:03:01.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.626+0000 7f1571390640 1 -- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f155806b4d0 msgr2=0x7f155806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.626+0000 7f1571390640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f155806b4d0 0x7f155806d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1560009ad0 tx=0x7f1560008040 comp rx=0 tx=0).stop 2024-04-25T10:03:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.626+0000 7f1571390640 1 -- 172.21.15.19:0/3101300798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f156c15ba00 msgr2=0x7f156c1e53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.626+0000 7f1571390640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f156c15ba00 0x7f156c1e53d0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f1564000c00 tx=0x7f15640046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.626+0000 7f1571390640 1 -- 172.21.15.19:0/3101300798 shutdown_connections 2024-04-25T10:03:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.626+0000 7f1571390640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f155806b4d0 0x7f155806d990 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.626+0000 7f1571390640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f156c1602e0 0x7f156c149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.626+0000 7f1571390640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f156c15f5f0 0x7f156c149950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.626+0000 7f1571390640 1 --2- 172.21.15.19:0/3101300798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f156c15ba00 0x7f156c1e53d0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.626+0000 7f1571390640 1 -- 172.21.15.19:0/3101300798 >> 172.21.15.19:0/3101300798 conn(0x7f156c0b3270 msgr2=0x7f156c0cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.627+0000 7f1571390640 1 -- 172.21.15.19:0/3101300798 shutdown_connections 2024-04-25T10:03:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.627+0000 7f1571390640 1 -- 172.21.15.19:0/3101300798 wait complete. 2024-04-25T10:03:01.642 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.738+0000 7f0907031640 1 Processor -- start 2024-04-25T10:03:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.738+0000 7f0907031640 1 -- start start 2024-04-25T10:03:01.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.739+0000 7f0907031640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f090015dc70 0x7f09000d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.739+0000 7f0907031640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090006ce10 0x7f09000cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.739+0000 7f0907031640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090006c220 con 0x7f090009a450 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.739+0000 7f0907031640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09000d0c90 con 0x7f090015dc70 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.739+0000 7f0907031640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09000d0080 con 0x7f090006ce10 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.739+0000 7f0904dcd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f090015dc70 0x7f09000d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.739+0000 7f08fffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f090009a450 0x7f090006c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47672/0 (socket says 172.21.15.19:47672) 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.739+0000 7f08fffff640 1 -- 172.21.15.19:0/1449674286 learned_addr learned my addr 172.21.15.19:0/1449674286 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.740+0000 7f09055ce640 1 --2- 172.21.15.19:0/1449674286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090006ce10 0x7f09000cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.740+0000 7f0904dcd640 1 -- 172.21.15.19:0/1449674286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090006ce10 msgr2=0x7f09000cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.740+0000 7f0904dcd640 1 --2- 172.21.15.19:0/1449674286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090006ce10 0x7f09000cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.740+0000 7f0904dcd640 1 -- 172.21.15.19:0/1449674286 >> v1:172.21.15.19:6789/0 conn(0x7f090009a450 legacy=0x7f090006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.740+0000 7f0904dcd640 1 -- 172.21.15.19:0/1449674286 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09000d0360 con 0x7f090015dc70 2024-04-25T10:03:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.740+0000 7f0904dcd640 1 --2- 172.21.15.19:0/1449674286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f090015dc70 0x7f09000d1120 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f08f40097a0 tx=0x7f08f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b0992dc1709557 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.741+0000 7f08ff7fe640 1 -- 172.21.15.19:0/1449674286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4037070 con 0x7f090015dc70 2024-04-25T10:03:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.741+0000 7f08ff7fe640 1 -- 172.21.15.19:0/1449674286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f4032440 con 0x7f090015dc70 2024-04-25T10:03:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.741+0000 7f08ff7fe640 1 -- 172.21.15.19:0/1449674286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4032810 con 0x7f090015dc70 2024-04-25T10:03:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.741+0000 7f0907031640 1 -- 172.21.15.19:0/1449674286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f090015dc70 msgr2=0x7f09000d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.741+0000 7f0907031640 1 --2- 172.21.15.19:0/1449674286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f090015dc70 0x7f09000d1120 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f08f40097a0 tx=0x7f08f402a180 comp rx=0 tx=0).stop 2024-04-25T10:03:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.741+0000 7f0907031640 1 -- 172.21.15.19:0/1449674286 shutdown_connections 2024-04-25T10:03:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.741+0000 7f0907031640 1 --2- 172.21.15.19:0/1449674286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090006ce10 0x7f09000cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.741+0000 7f0907031640 1 --2- 172.21.15.19:0/1449674286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f090015dc70 0x7f09000d1120 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.741+0000 7f0907031640 1 -- 172.21.15.19:0/1449674286 >> 172.21.15.19:0/1449674286 conn(0x7f09000b3270 msgr2=0x7f09000b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.741+0000 7f0907031640 1 -- 172.21.15.19:0/1449674286 shutdown_connections 2024-04-25T10:03:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.741+0000 7f0907031640 1 -- 172.21.15.19:0/1449674286 wait complete. 2024-04-25T10:03:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.742+0000 7f0907031640 1 Processor -- start 2024-04-25T10:03:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.742+0000 7f0907031640 1 -- start start 2024-04-25T10:03:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.743+0000 7f0907031640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090006ce10 0x7f0900150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.743+0000 7f0907031640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f090009a450 0x7f09000b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.743+0000 7f0907031640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f090015dc70 0x7f09000b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.743+0000 7f0907031640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090015e050 con 0x7f090009a450 2024-04-25T10:03:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.743+0000 7f0907031640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09000d0920 con 0x7f090015dc70 2024-04-25T10:03:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.743+0000 7f0907031640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09000cbdd0 con 0x7f090006ce10 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.743+0000 7f08fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f090009a450 0x7f09000b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.743+0000 7f08fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f090009a450 0x7f09000b5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60194/0 (socket says 172.21.15.19:60194) 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.743+0000 7f08fffff640 1 -- 172.21.15.19:0/2422394979 learned_addr learned my addr 172.21.15.19:0/2422394979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f09055ce640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f090015dc70 0x7f09000b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f0904dcd640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090006ce10 0x7f0900150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f08fffff640 1 -- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090006ce10 msgr2=0x7f0900150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f08fffff640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090006ce10 0x7f0900150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f08fffff640 1 -- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f090015dc70 msgr2=0x7f09000b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f08fffff640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f090015dc70 0x7f09000b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f08fffff640 1 -- 172.21.15.19:0/2422394979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0900158d10 con 0x7f090009a450 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f08fffff640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f090009a450 0x7f09000b5730 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00c430 tx=0x7f08ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f08fd7fa640 1 -- 172.21.15.19:0/2422394979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec017070 con 0x7f090009a450 2024-04-25T10:03:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f08fd7fa640 1 -- 172.21.15.19:0/2422394979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec005ce0 con 0x7f090009a450 2024-04-25T10:03:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f0907031640 1 -- 172.21.15.19:0/2422394979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09000c92b0 con 0x7f090009a450 2024-04-25T10:03:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f08fd7fa640 1 -- 172.21.15.19:0/2422394979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0072f0 con 0x7f090009a450 2024-04-25T10:03:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.744+0000 7f0907031640 1 -- 172.21.15.19:0/2422394979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09001579f0 con 0x7f090009a450 2024-04-25T10:03:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.745+0000 7f0907031640 1 -- 172.21.15.19:0/2422394979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0900073990 con 0x7f090009a450 2024-04-25T10:03:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.746+0000 7f08fd7fa640 1 -- 172.21.15.19:0/2422394979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08ec01a070 con 0x7f090009a450 2024-04-25T10:03:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.746+0000 7f08fd7fa640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b4d0 0x7f08e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.746+0000 7f08fd7fa640 1 -- 172.21.15.19:0/2422394979 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f08ec0896a0 con 0x7f090009a450 2024-04-25T10:03:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.746+0000 7f0904dcd640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b4d0 0x7f08e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.747+0000 7f0904dcd640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b4d0 0x7f08e406d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f08f4034000 tx=0x7f08f402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.750+0000 7f08fd7fa640 1 -- 172.21.15.19:0/2422394979 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08ec057da0 con 0x7f090009a450 2024-04-25T10:03:01.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.932+0000 7f0907031640 1 -- 172.21.15.19:0/2422394979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09000b40a0 con 0x7f090009a450 2024-04-25T10:03:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.933+0000 7f08fd7fa640 1 -- 172.21.15.19:0/2422394979 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+17037 (secure 0 0 0) 0x7f08ec05be40 con 0x7f090009a450 2024-04-25T10:03:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:01.938 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":142,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:02:48.692096+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:33.312366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:33.971764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.1","mappings":[{"from":3,"to":0}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 -- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b4d0 msgr2=0x7f08e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b4d0 0x7f08e406d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f08f4034000 tx=0x7f08f402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 -- 172.21.15.19:0/2422394979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f090009a450 msgr2=0x7f09000b5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f090009a450 0x7f09000b5730 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00c430 tx=0x7f08ec00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 -- 172.21.15.19:0/2422394979 shutdown_connections 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f08e406b4d0 0x7f08e406d990 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f090015dc70 0x7f09000b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f090009a450 0x7f09000b5730 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 --2- 172.21.15.19:0/2422394979 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f090006ce10 0x7f0900150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 -- 172.21.15.19:0/2422394979 >> 172.21.15.19:0/2422394979 conn(0x7f09000b3270 msgr2=0x7f090015ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 -- 172.21.15.19:0/2422394979 shutdown_connections 2024-04-25T10:03:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:01.935+0000 7f0907031640 1 -- 172.21.15.19:0/2422394979 wait complete. 2024-04-25T10:03:01.951 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:03:02.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.048+0000 7feb8d134640 1 Processor -- start 2024-04-25T10:03:02.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.048+0000 7feb8d134640 1 -- start start 2024-04-25T10:03:02.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb8d134640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb880b9750 0x7feb880ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb8d134640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb880cebb0 0x7feb880d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb8d134640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb880cf060 con 0x7feb8815da90 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb8d134640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb88162600 con 0x7feb880b9750 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb8d134640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb88161d70 con 0x7feb880cebb0 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb7ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb880b9750 0x7feb880ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb7ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb880b9750 0x7feb880ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60198/0 (socket says 172.21.15.19:60198) 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb7ffff640 1 -- 172.21.15.19:0/1274669524 learned_addr learned my addr 172.21.15.19:0/1274669524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb8759e640 1 --2- 172.21.15.19:0/1274669524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb880cebb0 0x7feb880d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb7ffff640 1 -- 172.21.15.19:0/1274669524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb880cebb0 msgr2=0x7feb880d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb7ffff640 1 --2- 172.21.15.19:0/1274669524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb880cebb0 0x7feb880d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb7ffff640 1 -- 172.21.15.19:0/1274669524 >> v1:172.21.15.112:6789/0 conn(0x7feb8815da90 legacy=0x7feb88161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.049+0000 7feb7ffff640 1 -- 172.21.15.19:0/1274669524 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb88162040 con 0x7feb880b9750 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.050+0000 7feb7ffff640 1 --2- 172.21.15.19:0/1274669524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb880b9750 0x7feb880ce670 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7feb70009870 tx=0x7feb7002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910ffe1e9dab70a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.050+0000 7feb8659c640 1 -- 172.21.15.19:0/1274669524 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70037070 con 0x7feb880b9750 2024-04-25T10:03:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.050+0000 7feb8659c640 1 -- 172.21.15.19:0/1274669524 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb70032440 con 0x7feb880b9750 2024-04-25T10:03:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.050+0000 7feb8659c640 1 -- 172.21.15.19:0/1274669524 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70032810 con 0x7feb880b9750 2024-04-25T10:03:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.051+0000 7feb8d134640 1 -- 172.21.15.19:0/1274669524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb880b9750 msgr2=0x7feb880ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.051+0000 7feb8d134640 1 --2- 172.21.15.19:0/1274669524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb880b9750 0x7feb880ce670 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7feb70009870 tx=0x7feb7002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.051+0000 7feb8d134640 1 -- 172.21.15.19:0/1274669524 shutdown_connections 2024-04-25T10:03:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.051+0000 7feb8d134640 1 --2- 172.21.15.19:0/1274669524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb880cebb0 0x7feb880d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.051+0000 7feb8d134640 1 --2- 172.21.15.19:0/1274669524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb880b9750 0x7feb880ce670 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.051+0000 7feb8d134640 1 -- 172.21.15.19:0/1274669524 >> 172.21.15.19:0/1274669524 conn(0x7feb88061a20 msgr2=0x7feb88061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.051+0000 7feb8d134640 1 -- 172.21.15.19:0/1274669524 shutdown_connections 2024-04-25T10:03:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.051+0000 7feb8d134640 1 -- 172.21.15.19:0/1274669524 wait complete. 2024-04-25T10:03:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.052+0000 7feb8d134640 1 Processor -- start 2024-04-25T10:03:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.052+0000 7feb8d134640 1 -- start start 2024-04-25T10:03:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb8d134640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb880cebb0 0x7feb881601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb8d134640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8815da90 0x7feb88158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb8d134640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb881f43e0 0x7feb8815c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb8d134640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb880d1df0 con 0x7feb8815da90 2024-04-25T10:03:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb8d134640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb880d1b10 con 0x7feb880cebb0 2024-04-25T10:03:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb8d134640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8815de70 con 0x7feb881f43e0 2024-04-25T10:03:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8815da90 0x7feb88158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8815da90 0x7feb88158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60212/0 (socket says 172.21.15.19:60212) 2024-04-25T10:03:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb7ffff640 1 -- 172.21.15.19:0/1853276231 learned_addr learned my addr 172.21.15.19:0/1853276231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb86d9d640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb880cebb0 0x7feb881601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb8759e640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb881f43e0 0x7feb8815c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb7ffff640 1 -- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb881f43e0 msgr2=0x7feb8815c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb7ffff640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb881f43e0 0x7feb8815c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb7ffff640 1 -- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb880cebb0 msgr2=0x7feb881601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb7ffff640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb880cebb0 0x7feb881601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb7ffff640 1 -- 172.21.15.19:0/1853276231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb881f6d00 con 0x7feb8815da90 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb86d9d640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb880cebb0 0x7feb881601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.053+0000 7feb7ffff640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8815da90 0x7feb88158f90 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7feb70000c00 tx=0x7feb700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.054+0000 7feb7f7fe640 1 -- 172.21.15.19:0/1853276231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70037040 con 0x7feb8815da90 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.054+0000 7feb7f7fe640 1 -- 172.21.15.19:0/1853276231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb70042070 con 0x7feb8815da90 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.054+0000 7feb8d134640 1 -- 172.21.15.19:0/1853276231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb881f6fe0 con 0x7feb8815da90 2024-04-25T10:03:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.054+0000 7feb7f7fe640 1 -- 172.21.15.19:0/1853276231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70030880 con 0x7feb8815da90 2024-04-25T10:03:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.054+0000 7feb8d134640 1 -- 172.21.15.19:0/1853276231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb881f75a0 con 0x7feb8815da90 2024-04-25T10:03:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.055+0000 7feb8d134640 1 -- 172.21.15.19:0/1853276231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb88073a00 con 0x7feb8815da90 2024-04-25T10:03:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.055+0000 7feb7f7fe640 1 -- 172.21.15.19:0/1853276231 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb7002ad10 con 0x7feb8815da90 2024-04-25T10:03:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.055+0000 7feb7f7fe640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6806b500 0x7feb6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.056+0000 7feb7f7fe640 1 -- 172.21.15.19:0/1853276231 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6159+0+0 (secure 0 0 0) 0x7feb700aa920 con 0x7feb8815da90 2024-04-25T10:03:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.056+0000 7feb86d9d640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6806b500 0x7feb6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.056+0000 7feb86d9d640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6806b500 0x7feb6806d9c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7feb880ca090 tx=0x7feb78009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.060+0000 7feb7f7fe640 1 -- 172.21.15.19:0/1853276231 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb70079700 con 0x7feb8815da90 2024-04-25T10:03:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.276+0000 7feb8d134640 1 -- 172.21.15.19:0/1853276231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feb880d1590 con 0x7feb8815da90 2024-04-25T10:03:02.281 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T10:03:02.276+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:03:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.278+0000 7feb7f7fe640 1 -- 172.21.15.19:0/1853276231 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) v1 ==== 161+0+0 (secure 0 0 0) 0x7feb70079000 con 0x7feb8815da90 2024-04-25T10:03:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:03:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 -- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6806b500 msgr2=0x7feb6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6806b500 0x7feb6806d9c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7feb880ca090 tx=0x7feb78009040 comp rx=0 tx=0).stop 2024-04-25T10:03:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 -- 172.21.15.19:0/1853276231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8815da90 msgr2=0x7feb88158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8815da90 0x7feb88158f90 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7feb70000c00 tx=0x7feb700046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 -- 172.21.15.19:0/1853276231 shutdown_connections 2024-04-25T10:03:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feb6806b500 0x7feb6806d9c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb881f43e0 0x7feb8815c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8815da90 0x7feb88158f90 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 --2- 172.21.15.19:0/1853276231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb880cebb0 0x7feb881601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 -- 172.21.15.19:0/1853276231 >> 172.21.15.19:0/1853276231 conn(0x7feb88061a20 msgr2=0x7feb880b57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 -- 172.21.15.19:0/1853276231 shutdown_connections 2024-04-25T10:03:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.281+0000 7feb8d134640 1 -- 172.21.15.19:0/1853276231 wait complete. 2024-04-25T10:03:02.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:03:02.326 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T10:03:02.327 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T10:03:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.399+0000 7fb0a7437640 1 Processor -- start 2024-04-25T10:03:02.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.400+0000 7fb0a7437640 1 -- start start 2024-04-25T10:03:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.400+0000 7fb0a7437640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb0a0158230 0x7fb0a00d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.400+0000 7fb0a7437640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a00d2020 con 0x7fb0a00d1b80 2024-04-25T10:03:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.400+0000 7fb0a7437640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a00d2300 con 0x7fb0a00b4e60 2024-04-25T10:03:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.400+0000 7fb0a7437640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a0158610 con 0x7fb0a0158230 2024-04-25T10:03:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.401+0000 7fb0a49d2640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb0a00d1b80 0x7fb0a006c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47698/0 (socket says 172.21.15.19:47698) 2024-04-25T10:03:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.401+0000 7fb0a49d2640 1 -- 172.21.15.19:0/2168307900 learned_addr learned my addr 172.21.15.19:0/2168307900 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:02.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.401+0000 7fb0a51d3640 1 --2- 172.21.15.19:0/2168307900 >> v2:172.21.15.112:3300/0 conn(0x7fb0a0158230 0x7fb0a00d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.401+0000 7fb093fff640 1 -- 172.21.15.19:0/2168307900 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010702738 0 0) 0x7fb0a00d2020 con 0x7fb0a00d1b80 2024-04-25T10:03:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.401+0000 7fb093fff640 1 -- 172.21.15.19:0/2168307900 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb08c003520 con 0x7fb0a00d1b80 2024-04-25T10:03:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.401+0000 7fb0a51d3640 1 -- 172.21.15.19:0/2168307900 >> v1:172.21.15.112:6789/0 conn(0x7fb0a00b4e60 legacy=0x7fb0a006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.401+0000 7fb0a51d3640 1 -- 172.21.15.19:0/2168307900 >> v1:172.21.15.19:6789/0 conn(0x7fb0a00d1b80 legacy=0x7fb0a006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.401+0000 7fb0a51d3640 1 -- 172.21.15.19:0/2168307900 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a006bb30 con 0x7fb0a0158230 2024-04-25T10:03:02.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.402+0000 7fb0a51d3640 1 --2- 172.21.15.19:0/2168307900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0158230 0x7fb0a00d1640 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0097a0 tx=0x7fb09c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff2651c85d1e377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.402+0000 7fb093fff640 1 -- 172.21.15.19:0/2168307900 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c037070 con 0x7fb0a0158230 2024-04-25T10:03:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.402+0000 7fb093fff640 1 -- 172.21.15.19:0/2168307900 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c032440 con 0x7fb0a0158230 2024-04-25T10:03:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.402+0000 7fb093fff640 1 -- 172.21.15.19:0/2168307900 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0327f0 con 0x7fb0a0158230 2024-04-25T10:03:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.402+0000 7fb0a7437640 1 -- 172.21.15.19:0/2168307900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0158230 msgr2=0x7fb0a00d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.402+0000 7fb0a7437640 1 --2- 172.21.15.19:0/2168307900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0158230 0x7fb0a00d1640 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0097a0 tx=0x7fb09c02a230 comp rx=0 tx=0).stop 2024-04-25T10:03:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.402+0000 7fb0a7437640 1 -- 172.21.15.19:0/2168307900 shutdown_connections 2024-04-25T10:03:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.402+0000 7fb0a7437640 1 --2- 172.21.15.19:0/2168307900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0158230 0x7fb0a00d1640 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.402+0000 7fb0a7437640 1 -- 172.21.15.19:0/2168307900 >> 172.21.15.19:0/2168307900 conn(0x7fb0a0061600 msgr2=0x7fb0a00619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.403+0000 7fb0a7437640 1 -- 172.21.15.19:0/2168307900 shutdown_connections 2024-04-25T10:03:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.403+0000 7fb0a7437640 1 -- 172.21.15.19:0/2168307900 wait complete. 2024-04-25T10:03:02.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.404+0000 7fb0a7437640 1 Processor -- start 2024-04-25T10:03:02.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.404+0000 7fb0a7437640 1 -- start start 2024-04-25T10:03:02.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.404+0000 7fb0a7437640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b4e60 0x7fb0a01e4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.404+0000 7fb0a7437640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a00d1b80 0x7fb0a00cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.404+0000 7fb0a7437640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0158230 0x7fb0a01f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.404+0000 7fb0a7437640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a006ca40 con 0x7fb0a00d1b80 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.404+0000 7fb0a7437640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a006bea0 con 0x7fb0a0158230 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.404+0000 7fb0a7437640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a015e320 con 0x7fb0a00b4e60 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a49d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a00d1b80 0x7fb0a00cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a49d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a00d1b80 0x7fb0a00cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60246/0 (socket says 172.21.15.19:60246) 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a49d2640 1 -- 172.21.15.19:0/1758920922 learned_addr learned my addr 172.21.15.19:0/1758920922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a51d3640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b4e60 0x7fb0a01e4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a59d4640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0158230 0x7fb0a01f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a49d2640 1 -- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b4e60 msgr2=0x7fb0a01e4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a49d2640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b4e60 0x7fb0a01e4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a49d2640 1 -- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0158230 msgr2=0x7fb0a01f60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a49d2640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0158230 0x7fb0a01f60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a49d2640 1 -- 172.21.15.19:0/1758920922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a01e5840 con 0x7fb0a00d1b80 2024-04-25T10:03:02.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a49d2640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a00d1b80 0x7fb0a00cac40 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fb094001260 tx=0x7fb09400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb091ffb640 1 -- 172.21.15.19:0/1758920922 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb094017070 con 0x7fb0a00d1b80 2024-04-25T10:03:02.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb091ffb640 1 -- 172.21.15.19:0/1758920922 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0940074e0 con 0x7fb0a00d1b80 2024-04-25T10:03:02.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a7437640 1 -- 172.21.15.19:0/1758920922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a01e5b20 con 0x7fb0a00d1b80 2024-04-25T10:03:02.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb091ffb640 1 -- 172.21.15.19:0/1758920922 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0940059a0 con 0x7fb0a00d1b80 2024-04-25T10:03:02.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.405+0000 7fb0a7437640 1 -- 172.21.15.19:0/1758920922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a01e60e0 con 0x7fb0a00d1b80 2024-04-25T10:03:02.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.406+0000 7fb0a7437640 1 -- 172.21.15.19:0/1758920922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a0073a00 con 0x7fb0a00d1b80 2024-04-25T10:03:02.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.407+0000 7fb091ffb640 1 -- 172.21.15.19:0/1758920922 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb09401a070 con 0x7fb0a00d1b80 2024-04-25T10:03:02.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.407+0000 7fb091ffb640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 0x7fb08406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.407+0000 7fb091ffb640 1 -- 172.21.15.19:0/1758920922 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fb0940896e0 con 0x7fb0a00d1b80 2024-04-25T10:03:02.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.407+0000 7fb0a51d3640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 0x7fb08406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.408+0000 7fb0a51d3640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 0x7fb08406d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb09c034000 tx=0x7fb09c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.411+0000 7fb091ffb640 1 -- 172.21.15.19:0/1758920922 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb094057ce0 con 0x7fb0a00d1b80 2024-04-25T10:03:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:02.633+0000 7fb0a7437640 1 -- 172.21.15.19:0/1758920922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb0a00d0900 con 0x7fb0a00d1b80 2024-04-25T10:03:03.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.287+0000 7fb091ffb640 1 -- 172.21.15.19:0/1758920922 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb09405bd80 con 0x7fb0a00d1b80 2024-04-25T10:03:03.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 -- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 msgr2=0x7fb08406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 0x7fb08406d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb09c034000 tx=0x7fb09c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:03.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 -- 172.21.15.19:0/1758920922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a00d1b80 msgr2=0x7fb0a00cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a00d1b80 0x7fb0a00cac40 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fb094001260 tx=0x7fb09400c870 comp rx=0 tx=0).stop 2024-04-25T10:03:03.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 -- 172.21.15.19:0/1758920922 shutdown_connections 2024-04-25T10:03:03.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb08406b4d0 0x7fb08406d990 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a0158230 0x7fb0a01f60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a00d1b80 0x7fb0a00cac40 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 --2- 172.21.15.19:0/1758920922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb0a00b4e60 0x7fb0a01e4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 -- 172.21.15.19:0/1758920922 >> 172.21.15.19:0/1758920922 conn(0x7fb0a0061600 msgr2=0x7fb0a00b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 -- 172.21.15.19:0/1758920922 shutdown_connections 2024-04-25T10:03:03.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.291+0000 7fb0a7437640 1 -- 172.21.15.19:0/1758920922 wait complete. 2024-04-25T10:03:03.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:03:03.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.459+0000 7f8af9d7e640 1 Processor -- start 2024-04-25T10:03:03.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.459+0000 7f8af9d7e640 1 -- start start 2024-04-25T10:03:03.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.459+0000 7f8af9d7e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8af415dc20 0x7f8af40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.459+0000 7f8af9d7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8af40d32f0 0x7f8af40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af9d7e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af415e000 con 0x7f8af4161710 2024-04-25T10:03:03.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af9d7e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af40d1de0 con 0x7f8af415dc20 2024-04-25T10:03:03.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af9d7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4161c60 con 0x7f8af40d32f0 2024-04-25T10:03:03.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8af415dc20 0x7f8af40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8af40d32f0 0x7f8af40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8af415dc20 0x7f8af40ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:52508/0 (socket says 172.21.15.19:52508) 2024-04-25T10:03:03.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af37fe640 1 -- 172.21.15.19:0/287903110 learned_addr learned my addr 172.21.15.19:0/287903110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:03.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af2ffd640 1 --1- 172.21.15.19:0/287903110 >> v1:172.21.15.112:6789/0 conn(0x7f8af4161710 0x7f8af406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:45208/0 (socket says 172.21.15.19:45208) 2024-04-25T10:03:03.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af37fe640 1 -- 172.21.15.19:0/287903110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8af40d32f0 msgr2=0x7f8af40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af37fe640 1 --2- 172.21.15.19:0/287903110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8af40d32f0 0x7f8af40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af37fe640 1 -- 172.21.15.19:0/287903110 >> v1:172.21.15.112:6789/0 conn(0x7f8af4161710 legacy=0x7f8af406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af37fe640 1 -- 172.21.15.19:0/287903110 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4161f40 con 0x7f8af415dc20 2024-04-25T10:03:03.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.460+0000 7f8af3fff640 1 --2- 172.21.15.19:0/287903110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8af40d32f0 0x7f8af40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.461+0000 7f8af37fe640 1 --2- 172.21.15.19:0/287903110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415dc20 0x7f8af40ce670 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009870 tx=0x7f8ae402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bcee10526f39cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.461+0000 7f8af27fc640 1 -- 172.21.15.19:0/287903110 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4037070 con 0x7f8af415dc20 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.461+0000 7f8af27fc640 1 -- 172.21.15.19:0/287903110 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae4032440 con 0x7f8af415dc20 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.461+0000 7f8af27fc640 1 -- 172.21.15.19:0/287903110 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4032810 con 0x7f8af415dc20 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.462+0000 7f8af9d7e640 1 -- 172.21.15.19:0/287903110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415dc20 msgr2=0x7f8af40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.462+0000 7f8af9d7e640 1 --2- 172.21.15.19:0/287903110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415dc20 0x7f8af40ce670 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009870 tx=0x7f8ae402a270 comp rx=0 tx=0).stop 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.462+0000 7f8af9d7e640 1 -- 172.21.15.19:0/287903110 shutdown_connections 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.462+0000 7f8af9d7e640 1 --2- 172.21.15.19:0/287903110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8af40d32f0 0x7f8af40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.462+0000 7f8af9d7e640 1 --2- 172.21.15.19:0/287903110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af415dc20 0x7f8af40ce670 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.462+0000 7f8af9d7e640 1 -- 172.21.15.19:0/287903110 >> 172.21.15.19:0/287903110 conn(0x7f8af40d0420 msgr2=0x7f8af40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.462+0000 7f8af9d7e640 1 -- 172.21.15.19:0/287903110 shutdown_connections 2024-04-25T10:03:03.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.462+0000 7f8af9d7e640 1 -- 172.21.15.19:0/287903110 wait complete. 2024-04-25T10:03:03.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.463+0000 7f8af9d7e640 1 Processor -- start 2024-04-25T10:03:03.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.463+0000 7f8af9d7e640 1 -- start start 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af9d7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8af40d32f0 0x7f8af4156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af9d7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af415dc20 0x7f8af40cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af9d7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af4161710 0x7f8af4157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af9d7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40b5e50 con 0x7f8af415dc20 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af9d7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4162500 con 0x7f8af4161710 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af9d7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40b5fd0 con 0x7f8af40d32f0 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af415dc20 0x7f8af40cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af415dc20 0x7f8af40cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60262/0 (socket says 172.21.15.19:60262) 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af2ffd640 1 -- 172.21.15.19:0/938228684 learned_addr learned my addr 172.21.15.19:0/938228684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af37fe640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8af40d32f0 0x7f8af4156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af3fff640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af4161710 0x7f8af4157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af2ffd640 1 -- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8af40d32f0 msgr2=0x7f8af4156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af2ffd640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8af40d32f0 0x7f8af4156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af2ffd640 1 -- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af4161710 msgr2=0x7f8af4157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af2ffd640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af4161710 0x7f8af4157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.464+0000 7f8af2ffd640 1 -- 172.21.15.19:0/938228684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4157200 con 0x7f8af415dc20 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.465+0000 7f8af2ffd640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af415dc20 0x7f8af40cc9b0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800c3c0 tx=0x7f8ae800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.465+0000 7f8adbfff640 1 -- 172.21.15.19:0/938228684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80074e0 con 0x7f8af415dc20 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.465+0000 7f8adbfff640 1 -- 172.21.15.19:0/938228684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae8007680 con 0x7f8af415dc20 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.465+0000 7f8af9d7e640 1 -- 172.21.15.19:0/938228684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af40ccef0 con 0x7f8af415dc20 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.465+0000 7f8adbfff640 1 -- 172.21.15.19:0/938228684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8004a00 con 0x7f8af415dc20 2024-04-25T10:03:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.465+0000 7f8af9d7e640 1 -- 172.21.15.19:0/938228684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af415ead0 con 0x7f8af415dc20 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.466+0000 7f8af9d7e640 1 -- 172.21.15.19:0/938228684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af4073a00 con 0x7f8af415dc20 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.466+0000 7f8adbfff640 1 -- 172.21.15.19:0/938228684 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ae801a020 con 0x7f8af415dc20 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.467+0000 7f8adbfff640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ae006b4d0 0x7f8ae006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.467+0000 7f8adbfff640 1 -- 172.21.15.19:0/938228684 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f8ae8089ba0 con 0x7f8af415dc20 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.467+0000 7f8af37fe640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ae006b4d0 0x7f8ae006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.467+0000 7f8af37fe640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ae006b4d0 0x7f8ae006d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4034000 tx=0x7f8ae402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.471+0000 7f8adbfff640 1 -- 172.21.15.19:0/938228684 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ae8058800 con 0x7f8af415dc20 2024-04-25T10:03:03.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:03.652+0000 7f8af9d7e640 1 -- 172.21.15.19:0/938228684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8af40d16b0 con 0x7f8af415dc20 2024-04-25T10:03:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.289+0000 7f8adbfff640 1 -- 172.21.15.19:0/938228684 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v144) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8ae8058100 con 0x7f8af415dc20 2024-04-25T10:03:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:03:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 -- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ae006b4d0 msgr2=0x7f8ae006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ae006b4d0 0x7f8ae006d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4034000 tx=0x7f8ae402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 -- 172.21.15.19:0/938228684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af415dc20 msgr2=0x7f8af40cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af415dc20 0x7f8af40cc9b0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800c3c0 tx=0x7f8ae800c880 comp rx=0 tx=0).stop 2024-04-25T10:03:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 -- 172.21.15.19:0/938228684 shutdown_connections 2024-04-25T10:03:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ae006b4d0 0x7f8ae006d990 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8af4161710 0x7f8af4157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af415dc20 0x7f8af40cc9b0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 --2- 172.21.15.19:0/938228684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8af40d32f0 0x7f8af4156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 -- 172.21.15.19:0/938228684 >> 172.21.15.19:0/938228684 conn(0x7f8af40d0420 msgr2=0x7f8af40d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 -- 172.21.15.19:0/938228684 shutdown_connections 2024-04-25T10:03:04.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.293+0000 7f8af9d7e640 1 -- 172.21.15.19:0/938228684 wait complete. 2024-04-25T10:03:04.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:03:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.416+0000 7f5d8877e640 1 Processor -- start 2024-04-25T10:03:04.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.416+0000 7f5d8877e640 1 -- start start 2024-04-25T10:03:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d8877e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d800b5620 0x7f5d800b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d8877e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8006cf50 con 0x7f5d8015b7d0 2024-04-25T10:03:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d8877e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d800ce530 con 0x7f5d8006c2d0 2024-04-25T10:03:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d8877e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d800ce810 con 0x7f5d800b5620 2024-04-25T10:03:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d85d19640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d800b5620 0x7f5d800b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d8651a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5d8015b7d0 0x7f5d800cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47708/0 (socket says 172.21.15.19:47708) 2024-04-25T10:03:04.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d85d19640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d800b5620 0x7f5d800b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60272/0 (socket says 172.21.15.19:60272) 2024-04-25T10:03:04.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d8651a640 1 -- 172.21.15.19:0/1343480803 learned_addr learned my addr 172.21.15.19:0/1343480803 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:04.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d85d19640 1 -- 172.21.15.19:0/1343480803 >> v1:172.21.15.112:6789/0 conn(0x7f5d8006c2d0 legacy=0x7f5d8006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d85d19640 1 -- 172.21.15.19:0/1343480803 >> v1:172.21.15.19:6789/0 conn(0x7f5d8015b7d0 legacy=0x7f5d800cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d85518640 1 -- 172.21.15.19:0/1343480803 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766541415 0 0) 0x7f5d8006cf50 con 0x7f5d8015b7d0 2024-04-25T10:03:04.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.417+0000 7f5d85d19640 1 -- 172.21.15.19:0/1343480803 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d800ceaf0 con 0x7f5d800b5620 2024-04-25T10:03:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.418+0000 7f5d85d19640 1 --2- 172.21.15.19:0/1343480803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d800b5620 0x7f5d800b5ab0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f5d740097a0 tx=0x7f5d7402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44b96a3ebf28f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.418+0000 7f5d85518640 1 -- 172.21.15.19:0/1343480803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74037070 con 0x7f5d800b5620 2024-04-25T10:03:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.418+0000 7f5d85518640 1 -- 172.21.15.19:0/1343480803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d74032440 con 0x7f5d800b5620 2024-04-25T10:03:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.418+0000 7f5d85518640 1 -- 172.21.15.19:0/1343480803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74032810 con 0x7f5d800b5620 2024-04-25T10:03:04.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.419+0000 7f5d8877e640 1 -- 172.21.15.19:0/1343480803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d800b5620 msgr2=0x7f5d800b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.419+0000 7f5d8877e640 1 --2- 172.21.15.19:0/1343480803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d800b5620 0x7f5d800b5ab0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f5d740097a0 tx=0x7f5d7402a230 comp rx=0 tx=0).stop 2024-04-25T10:03:04.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.419+0000 7f5d8877e640 1 -- 172.21.15.19:0/1343480803 shutdown_connections 2024-04-25T10:03:04.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.419+0000 7f5d8877e640 1 --2- 172.21.15.19:0/1343480803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d800b5620 0x7f5d800b5ab0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.419+0000 7f5d8877e640 1 -- 172.21.15.19:0/1343480803 >> 172.21.15.19:0/1343480803 conn(0x7f5d800d0800 msgr2=0x7f5d800d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.419+0000 7f5d8877e640 1 -- 172.21.15.19:0/1343480803 shutdown_connections 2024-04-25T10:03:04.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.419+0000 7f5d8877e640 1 -- 172.21.15.19:0/1343480803 wait complete. 2024-04-25T10:03:04.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.420+0000 7f5d8877e640 1 Processor -- start 2024-04-25T10:03:04.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.420+0000 7f5d8877e640 1 -- start start 2024-04-25T10:03:04.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8877e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8006c2d0 0x7f5d800c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8877e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d800b5620 0x7f5d801571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8877e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d8015b7d0 0x7f5d801f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8877e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8015bd30 con 0x7f5d8006c2d0 2024-04-25T10:03:04.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8877e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8015bbb0 con 0x7f5d8015b7d0 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8877e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d800d1f40 con 0x7f5d800b5620 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8651a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8006c2d0 0x7f5d800c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8651a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8006c2d0 0x7f5d800c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60278/0 (socket says 172.21.15.19:60278) 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8651a640 1 -- 172.21.15.19:0/3178329564 learned_addr learned my addr 172.21.15.19:0/3178329564 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d85d19640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d800b5620 0x7f5d801571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d86d1b640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d8015b7d0 0x7f5d801f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8651a640 1 -- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d800b5620 msgr2=0x7f5d801571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8651a640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d800b5620 0x7f5d801571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8651a640 1 -- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d8015b7d0 msgr2=0x7f5d801f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8651a640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d8015b7d0 0x7f5d801f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8651a640 1 -- 172.21.15.19:0/3178329564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d801f3d70 con 0x7f5d8006c2d0 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d86d1b640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d8015b7d0 0x7f5d801f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:04.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.421+0000 7f5d8651a640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8006c2d0 0x7f5d800c8990 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800c3f0 tx=0x7f5d7800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.422+0000 7f5d7effd640 1 -- 172.21.15.19:0/3178329564 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d780074e0 con 0x7f5d8006c2d0 2024-04-25T10:03:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.422+0000 7f5d7effd640 1 -- 172.21.15.19:0/3178329564 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d780047e0 con 0x7f5d8006c2d0 2024-04-25T10:03:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.422+0000 7f5d8877e640 1 -- 172.21.15.19:0/3178329564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d801f4050 con 0x7f5d8006c2d0 2024-04-25T10:03:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.422+0000 7f5d7effd640 1 -- 172.21.15.19:0/3178329564 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d78004c10 con 0x7f5d8006c2d0 2024-04-25T10:03:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.422+0000 7f5d8877e640 1 -- 172.21.15.19:0/3178329564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d80157a30 con 0x7f5d8006c2d0 2024-04-25T10:03:04.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.423+0000 7f5d8877e640 1 -- 172.21.15.19:0/3178329564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d800b6970 con 0x7f5d8006c2d0 2024-04-25T10:03:04.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.423+0000 7f5d7effd640 1 -- 172.21.15.19:0/3178329564 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d78011070 con 0x7f5d8006c2d0 2024-04-25T10:03:04.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.423+0000 7f5d7effd640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d6c06b4d0 0x7f5d6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.424+0000 7f5d7effd640 1 -- 172.21.15.19:0/3178329564 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5807+0+0 (secure 0 0 0) 0x7f5d78089d50 con 0x7f5d8006c2d0 2024-04-25T10:03:04.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.424+0000 7f5d85d19640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d6c06b4d0 0x7f5d6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.424+0000 7f5d85d19640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d6c06b4d0 0x7f5d6c06d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5d74034000 tx=0x7f5d7402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.428+0000 7f5d7effd640 1 -- 172.21.15.19:0/3178329564 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d800b6970 con 0x7f5d8006c2d0 2024-04-25T10:03:04.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:04.616+0000 7f5d8877e640 1 -- 172.21.15.19:0/3178329564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d80157820 con 0x7f5d8006c2d0 2024-04-25T10:03:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.291+0000 7f5d7effd640 1 -- 172.21.15.19:0/3178329564 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v145) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5d7808d030 con 0x7f5d8006c2d0 2024-04-25T10:03:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.295+0000 7f5d8877e640 1 -- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d6c06b4d0 msgr2=0x7f5d6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.295+0000 7f5d8877e640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d6c06b4d0 0x7f5d6c06d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5d74034000 tx=0x7f5d7402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.295+0000 7f5d8877e640 1 -- 172.21.15.19:0/3178329564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8006c2d0 msgr2=0x7f5d800c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.295+0000 7f5d8877e640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8006c2d0 0x7f5d800c8990 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800c3f0 tx=0x7f5d7800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.296+0000 7f5d8877e640 1 -- 172.21.15.19:0/3178329564 shutdown_connections 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.296+0000 7f5d8877e640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d6c06b4d0 0x7f5d6c06d990 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.296+0000 7f5d8877e640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d8015b7d0 0x7f5d801f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.296+0000 7f5d8877e640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d800b5620 0x7f5d801571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.296+0000 7f5d8877e640 1 --2- 172.21.15.19:0/3178329564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8006c2d0 0x7f5d800c8990 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.296+0000 7f5d8877e640 1 -- 172.21.15.19:0/3178329564 >> 172.21.15.19:0/3178329564 conn(0x7f5d800d0800 msgr2=0x7f5d80160af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.296+0000 7f5d8877e640 1 -- 172.21.15.19:0/3178329564 shutdown_connections 2024-04-25T10:03:05.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.296+0000 7f5d8877e640 1 -- 172.21.15.19:0/3178329564 wait complete. 2024-04-25T10:03:05.319 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:03:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.426+0000 7f6b4085d640 1 Processor -- start 2024-04-25T10:03:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.427+0000 7f6b4085d640 1 -- start start 2024-04-25T10:03:05.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.427+0000 7f6b4085d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6b3c0af520 0x7f6b3c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b4085d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3c0adf30 con 0x7f6b3c0b4040 2024-04-25T10:03:05.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b4085d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3c0ad970 con 0x7f6b3c15b8a0 2024-04-25T10:03:05.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b4085d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c0adc50 con 0x7f6b3c0af520 2024-04-25T10:03:05.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3ad9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6b3c0b4040 0x7f6b3c0b44b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47712/0 (socket says 172.21.15.19:47712) 2024-04-25T10:03:05.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3ad9d640 1 -- 172.21.15.19:0/1890147396 learned_addr learned my addr 172.21.15.19:0/1890147396 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126781640 0 0) 0x7f6b34001490 con 0x7f6b3c0b4040 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b39d9b640 1 --2- 172.21.15.19:0/1890147396 >> v2:172.21.15.112:3300/0 conn(0x7f6b3c0af520 0x7f6b3c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b24003520 con 0x7f6b3c0b4040 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2684931543 0 0) 0x7f6b24003520 con 0x7f6b3c0b4040 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 >> v2:172.21.15.112:3300/0 conn(0x7f6b3c0af520 msgr2=0x7f6b3c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 --2- 172.21.15.19:0/1890147396 >> v2:172.21.15.112:3300/0 conn(0x7f6b3c0af520 0x7f6b3c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 >> v1:172.21.15.112:6789/0 conn(0x7f6b3c15b8a0 legacy=0x7f6b3c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c0b3840 con 0x7f6b3c0b4040 2024-04-25T10:03:05.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f6b34001bc0 con 0x7f6b3c0b4040 2024-04-25T10:03:05.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 >> v1:172.21.15.19:6789/0 conn(0x7f6b3c0b4040 legacy=0x7f6b3c0b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b240039b0 0x7f6b24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b24004800 0x7f6b24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240051c0 0x7f6b2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c0adc50 con 0x7f6b240051c0 2024-04-25T10:03:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2400c060 con 0x7f6b240039b0 2024-04-25T10:03:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2400c260 con 0x7f6b24004800 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.428+0000 7f6b3a59c640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b240039b0 0x7f6b24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.429+0000 7f6b3ad9d640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240051c0 0x7f6b2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.429+0000 7f6b39d9b640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b24004800 0x7f6b24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.429+0000 7f6b3a59c640 1 -- 172.21.15.19:0/1890147396 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b24004800 msgr2=0x7f6b24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.429+0000 7f6b3a59c640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b24004800 0x7f6b24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.429+0000 7f6b3a59c640 1 -- 172.21.15.19:0/1890147396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240051c0 msgr2=0x7f6b2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.429+0000 7f6b3a59c640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240051c0 0x7f6b2400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.429+0000 7f6b3a59c640 1 -- 172.21.15.19:0/1890147396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b24003520 con 0x7f6b240039b0 2024-04-25T10:03:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.429+0000 7f6b3ad9d640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240051c0 0x7f6b2400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.429+0000 7f6b3ad9d640 1 -- 172.21.15.19:0/1890147396 reap_dead start 2024-04-25T10:03:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.430+0000 7f6b3a59c640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b240039b0 0x7f6b24003dd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c001750 tx=0x7f6b24005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=974dd674113fc046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.430+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c001370 con 0x7f6b240039b0 2024-04-25T10:03:05.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.430+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c0052d0 con 0x7f6b240039b0 2024-04-25T10:03:05.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.430+0000 7f6b3959a640 1 -- 172.21.15.19:0/1890147396 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c02abc0 con 0x7f6b240039b0 2024-04-25T10:03:05.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.430+0000 7f6b4085d640 1 -- 172.21.15.19:0/1890147396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b240039b0 msgr2=0x7f6b24003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.430+0000 7f6b4085d640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b240039b0 0x7f6b24003dd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c001750 tx=0x7f6b24005830 comp rx=0 tx=0).stop 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.430+0000 7f6b4085d640 1 -- 172.21.15.19:0/1890147396 shutdown_connections 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.430+0000 7f6b4085d640 1 --2- 172.21.15.19:0/1890147396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b240039b0 0x7f6b24003dd0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.430+0000 7f6b4085d640 1 -- 172.21.15.19:0/1890147396 >> 172.21.15.19:0/1890147396 conn(0x7f6b3c0cfec0 msgr2=0x7f6b3c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.430+0000 7f6b4085d640 1 -- 172.21.15.19:0/1890147396 shutdown_connections 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.430+0000 7f6b4085d640 1 -- 172.21.15.19:0/1890147396 wait complete. 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.432+0000 7f6b4085d640 1 Processor -- start 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.432+0000 7f6b4085d640 1 -- start start 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b4085d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240039b0 0x7f6b3c149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b4085d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b3c148b80 0x7f6b3c159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b4085d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b3c15a3e0 0x7f6b3c09b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b4085d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c15be00 con 0x7f6b240039b0 2024-04-25T10:03:05.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b4085d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c15bc80 con 0x7f6b3c148b80 2024-04-25T10:03:05.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b4085d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c06abe0 con 0x7f6b3c15a3e0 2024-04-25T10:03:05.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b3a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240039b0 0x7f6b3c149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b3a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240039b0 0x7f6b3c149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60290/0 (socket says 172.21.15.19:60290) 2024-04-25T10:03:05.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b3a59c640 1 -- 172.21.15.19:0/1325107215 learned_addr learned my addr 172.21.15.19:0/1325107215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:05.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b39d9b640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b3c148b80 0x7f6b3c159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b3ad9d640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b3c15a3e0 0x7f6b3c09b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b3a59c640 1 -- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b3c15a3e0 msgr2=0x7f6b3c09b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b3a59c640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b3c15a3e0 0x7f6b3c09b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b3a59c640 1 -- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b3c148b80 msgr2=0x7f6b3c159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b3a59c640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b3c148b80 0x7f6b3c159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b3a59c640 1 -- 172.21.15.19:0/1325107215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c15a9a0 con 0x7f6b240039b0 2024-04-25T10:03:05.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b3ad9d640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b3c15a3e0 0x7f6b3c09b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:05.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b3a59c640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240039b0 0x7f6b3c149700 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c000c00 tx=0x7f6b2c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b2affd640 1 -- 172.21.15.19:0/1325107215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c03f030 con 0x7f6b240039b0 2024-04-25T10:03:05.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b2affd640 1 -- 172.21.15.19:0/1325107215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c005480 con 0x7f6b240039b0 2024-04-25T10:03:05.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b4085d640 1 -- 172.21.15.19:0/1325107215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3c157b70 con 0x7f6b240039b0 2024-04-25T10:03:05.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.433+0000 7f6b2affd640 1 -- 172.21.15.19:0/1325107215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c037040 con 0x7f6b240039b0 2024-04-25T10:03:05.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.434+0000 7f6b4085d640 1 -- 172.21.15.19:0/1325107215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c158130 con 0x7f6b240039b0 2024-04-25T10:03:05.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.434+0000 7f6b4085d640 1 -- 172.21.15.19:0/1325107215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3c073a00 con 0x7f6b240039b0 2024-04-25T10:03:05.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.435+0000 7f6b2affd640 1 -- 172.21.15.19:0/1325107215 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b2c042070 con 0x7f6b240039b0 2024-04-25T10:03:05.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.435+0000 7f6b2affd640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1c06b4d0 0x7f6b1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.435+0000 7f6b2affd640 1 -- 172.21.15.19:0/1325107215 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6b2c0a99d0 con 0x7f6b240039b0 2024-04-25T10:03:05.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.436+0000 7f6b39d9b640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1c06b4d0 0x7f6b1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.436+0000 7f6b39d9b640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1c06b4d0 0x7f6b1c06d990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c14c260 tx=0x7f6b30009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.440+0000 7f6b2affd640 1 -- 172.21.15.19:0/1325107215 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b2c0789c0 con 0x7f6b240039b0 2024-04-25T10:03:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.618+0000 7f6b4085d640 1 -- 172.21.15.19:0/1325107215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6b3c073a00 con 0x7f6b240039b0 2024-04-25T10:03:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.621+0000 7f6b2affd640 1 -- 172.21.15.19:0/1325107215 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6b2c0782c0 con 0x7f6b240039b0 2024-04-25T10:03:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.623+0000 7f6b4085d640 1 -- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1c06b4d0 msgr2=0x7f6b1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.623+0000 7f6b4085d640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1c06b4d0 0x7f6b1c06d990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c14c260 tx=0x7f6b30009190 comp rx=0 tx=0).stop 2024-04-25T10:03:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.623+0000 7f6b4085d640 1 -- 172.21.15.19:0/1325107215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240039b0 msgr2=0x7f6b3c149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.623+0000 7f6b4085d640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240039b0 0x7f6b3c149700 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c000c00 tx=0x7f6b2c001120 comp rx=0 tx=0).stop 2024-04-25T10:03:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.623+0000 7f6b4085d640 1 -- 172.21.15.19:0/1325107215 shutdown_connections 2024-04-25T10:03:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.624+0000 7f6b4085d640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6b1c06b4d0 0x7f6b1c06d990 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.624+0000 7f6b4085d640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b3c15a3e0 0x7f6b3c09b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.624+0000 7f6b4085d640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b3c148b80 0x7f6b3c159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.624+0000 7f6b4085d640 1 --2- 172.21.15.19:0/1325107215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b240039b0 0x7f6b3c149700 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.624+0000 7f6b4085d640 1 -- 172.21.15.19:0/1325107215 >> 172.21.15.19:0/1325107215 conn(0x7f6b3c0cfec0 msgr2=0x7f6b3c0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.624+0000 7f6b4085d640 1 -- 172.21.15.19:0/1325107215 shutdown_connections 2024-04-25T10:03:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.624+0000 7f6b4085d640 1 -- 172.21.15.19:0/1325107215 wait complete. 2024-04-25T10:03:05.639 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:03:05.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.736+0000 7fea87f14640 1 Processor -- start 2024-04-25T10:03:05.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.736+0000 7fea87f14640 1 -- start start 2024-04-25T10:03:05.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.737+0000 7fea87f14640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea800b9730 0x7fea800ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.737+0000 7fea87f14640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fea800ceb90 0x7fea800d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.737+0000 7fea87f14640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea80161d50 con 0x7fea8015da70 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.737+0000 7fea87f14640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea801625e0 con 0x7fea800ceb90 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.737+0000 7fea87f14640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea800cf000 con 0x7fea800b9730 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.737+0000 7fea85cb0640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fea8015da70 0x7fea80161640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47720/0 (socket says 172.21.15.19:47720) 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.737+0000 7fea85cb0640 1 -- 172.21.15.19:0/1454169094 learned_addr learned my addr 172.21.15.19:0/1454169094 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.737+0000 7fea864b1640 1 --2- 172.21.15.19:0/1454169094 >> v2:172.21.15.112:3300/0 conn(0x7fea800ceb90 0x7fea800d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.737+0000 7fea854af640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea800b9730 0x7fea800ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.737+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292963128 0 0) 0x7fea80161d50 con 0x7fea8015da70 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.737+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea6c003520 con 0x7fea8015da70 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 147149630 0 0) 0x7fea6c003520 con 0x7fea8015da70 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea800b9730 msgr2=0x7fea800ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea800b9730 0x7fea800ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 >> v2:172.21.15.112:3300/0 conn(0x7fea800ceb90 msgr2=0x7fea800d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 --2- 172.21.15.19:0/1454169094 >> v2:172.21.15.112:3300/0 conn(0x7fea800ceb90 0x7fea800d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea80162020 con 0x7fea8015da70 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fea7c001670 con 0x7fea8015da70 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 >> v1:172.21.15.19:6789/0 conn(0x7fea8015da70 legacy=0x7fea80161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea6c0039b0 0x7fea6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea6c004310 0x7fea6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea6c004e50 0x7fea6c0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea801625e0 con 0x7fea6c004e50 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800cf000 con 0x7fea6c004310 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c00bde0 con 0x7fea6c0039b0 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea864b1640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea6c004e50 0x7fea6c0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea854af640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea6c004310 0x7fea6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea85cb0640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea6c0039b0 0x7fea6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea864b1640 1 -- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea6c0039b0 msgr2=0x7fea6c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea864b1640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea6c0039b0 0x7fea6c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea864b1640 1 -- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea6c004310 msgr2=0x7fea6c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea864b1640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea6c004310 0x7fea6c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea864b1640 1 -- 172.21.15.19:0/1454169094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c003520 con 0x7fea6c004e50 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.738+0000 7fea854af640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea6c004310 0x7fea6c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.739+0000 7fea864b1640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea6c004e50 0x7fea6c0052d0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fea78009fc0 tx=0x7fea6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98618732f4629597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.739+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78009c10 con 0x7fea6c004e50 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.739+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea78036070 con 0x7fea6c004e50 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.739+0000 7fea84cae640 1 -- 172.21.15.19:0/1454169094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7802d070 con 0x7fea6c004e50 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.739+0000 7fea87f14640 1 -- 172.21.15.19:0/1454169094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea6c004e50 msgr2=0x7fea6c0052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.739+0000 7fea87f14640 1 --2- 172.21.15.19:0/1454169094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea6c004e50 0x7fea6c0052d0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fea78009fc0 tx=0x7fea6c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.739+0000 7fea864b1640 1 -- 172.21.15.19:0/1454169094 reap_dead start 2024-04-25T10:03:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.740+0000 7fea87f14640 1 -- 172.21.15.19:0/1454169094 shutdown_connections 2024-04-25T10:03:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.740+0000 7fea87f14640 1 -- 172.21.15.19:0/1454169094 >> 172.21.15.19:0/1454169094 conn(0x7fea80061a20 msgr2=0x7fea80061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.740+0000 7fea87f14640 1 -- 172.21.15.19:0/1454169094 shutdown_connections 2024-04-25T10:03:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.740+0000 7fea87f14640 1 -- 172.21.15.19:0/1454169094 wait complete. 2024-04-25T10:03:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.741+0000 7fea87f14640 1 Processor -- start 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.741+0000 7fea87f14640 1 -- start start 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea87f14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea6c004e50 0x7fea8014ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea87f14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea800ccba0 0x7fea800c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea87f14640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea800b4620 0x7fea800b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea87f14640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800d1dd0 con 0x7fea800ccba0 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea87f14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800d1af0 con 0x7fea6c004e50 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea87f14640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8015de50 con 0x7fea800b4620 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea854af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea800ccba0 0x7fea800c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea854af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea800ccba0 0x7fea800c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60304/0 (socket says 172.21.15.19:60304) 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea854af640 1 -- 172.21.15.19:0/1092408406 learned_addr learned my addr 172.21.15.19:0/1092408406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea854af640 1 -- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea800b4620 msgr2=0x7fea800b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea864b1640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea800b4620 0x7fea800b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea85cb0640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea6c004e50 0x7fea8014ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea854af640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea800b4620 0x7fea800b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea854af640 1 -- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea6c004e50 msgr2=0x7fea8014ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea854af640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea6c004e50 0x7fea8014ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea854af640 1 -- 172.21.15.19:0/1092408406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea800c8710 con 0x7fea800ccba0 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea854af640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea800ccba0 0x7fea800c7e90 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fea70011aa0 tx=0x7fea70011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea767fc640 1 -- 172.21.15.19:0/1092408406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7000fbe0 con 0x7fea800ccba0 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea767fc640 1 -- 172.21.15.19:0/1092408406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7000fd80 con 0x7fea800ccba0 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.742+0000 7fea87f14640 1 -- 172.21.15.19:0/1092408406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea800c89f0 con 0x7fea800ccba0 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.743+0000 7fea767fc640 1 -- 172.21.15.19:0/1092408406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70014660 con 0x7fea800ccba0 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.743+0000 7fea87f14640 1 -- 172.21.15.19:0/1092408406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea800c83d0 con 0x7fea800ccba0 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.743+0000 7fea87f14640 1 -- 172.21.15.19:0/1092408406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea800b4fe0 con 0x7fea800ccba0 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.744+0000 7fea767fc640 1 -- 172.21.15.19:0/1092408406 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea70012040 con 0x7fea800ccba0 2024-04-25T10:03:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.744+0000 7fea767fc640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fea6406b500 0x7fea6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.744+0000 7fea767fc640 1 -- 172.21.15.19:0/1092408406 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fea7008a8a0 con 0x7fea800ccba0 2024-04-25T10:03:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.745+0000 7fea85cb0640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fea6406b500 0x7fea6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.745+0000 7fea85cb0640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fea6406b500 0x7fea6406d9c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fea7c001050 tx=0x7fea7c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:05.749+0000 7fea767fc640 1 -- 172.21.15.19:0/1092408406 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea70019030 con 0x7fea800ccba0 2024-04-25T10:03:06.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.014+0000 7fea87f14640 1 -- 172.21.15.19:0/1092408406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7fea80073990 con 0x7fea800ccba0 2024-04-25T10:03:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.292+0000 7fea767fc640 1 -- 172.21.15.19:0/1092408406 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v497) v1 ==== 159+0+0 (secure 0 0 0) 0x7fea70059920 con 0x7fea800ccba0 2024-04-25T10:03:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.295+0000 7fea87f14640 1 -- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fea6406b500 msgr2=0x7fea6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.295+0000 7fea87f14640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fea6406b500 0x7fea6406d9c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fea7c001050 tx=0x7fea7c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.295+0000 7fea87f14640 1 -- 172.21.15.19:0/1092408406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea800ccba0 msgr2=0x7fea800c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.295+0000 7fea87f14640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea800ccba0 0x7fea800c7e90 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fea70011aa0 tx=0x7fea70011f60 comp rx=0 tx=0).stop 2024-04-25T10:03:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.295+0000 7fea87f14640 1 -- 172.21.15.19:0/1092408406 shutdown_connections 2024-04-25T10:03:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.295+0000 7fea87f14640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fea6406b500 0x7fea6406d9c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.295+0000 7fea87f14640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fea800b4620 0x7fea800b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.296+0000 7fea87f14640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea800ccba0 0x7fea800c7e90 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.296+0000 7fea87f14640 1 --2- 172.21.15.19:0/1092408406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea6c004e50 0x7fea8014ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.296+0000 7fea87f14640 1 -- 172.21.15.19:0/1092408406 >> 172.21.15.19:0/1092408406 conn(0x7fea80061a20 msgr2=0x7fea8015cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.296+0000 7fea87f14640 1 -- 172.21.15.19:0/1092408406 shutdown_connections 2024-04-25T10:03:06.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.296+0000 7fea87f14640 1 -- 172.21.15.19:0/1092408406 wait complete. 2024-04-25T10:03:06.312 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:03:06.312 INFO:tasks.cephfs_test_runner:Starting test: test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:03:06.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:06.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.409+0000 7f7726af5640 1 Processor -- start 2024-04-25T10:03:06.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.409+0000 7f7726af5640 1 -- start start 2024-04-25T10:03:06.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7726af5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f772015db70 0x7f7720162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7726af5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f77200b5c30 0x7f77200d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7726af5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7720162690 con 0x7f77200b6cb0 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7726af5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77200ceee0 con 0x7f772015db70 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7726af5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77200d1d10 con 0x7f77200b5c30 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7724891640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f772015db70 0x7f7720162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f771ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f77200b6cb0 0x7f77200b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47728/0 (socket says 172.21.15.19:47728) 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f771ffff640 1 -- 172.21.15.19:0/3720009080 learned_addr learned my addr 172.21.15.19:0/3720009080 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7725092640 1 --2- 172.21.15.19:0/3720009080 >> v2:172.21.15.112:3300/0 conn(0x7f77200b5c30 0x7f77200d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7724891640 1 -- 172.21.15.19:0/3720009080 >> v2:172.21.15.112:3300/0 conn(0x7f77200b5c30 msgr2=0x7f77200d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7724891640 1 --2- 172.21.15.19:0/3720009080 >> v2:172.21.15.112:3300/0 conn(0x7f77200b5c30 0x7f77200d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7724891640 1 -- 172.21.15.19:0/3720009080 >> v1:172.21.15.19:6789/0 conn(0x7f77200b6cb0 legacy=0x7f77200b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7724891640 1 -- 172.21.15.19:0/3720009080 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77200d1ff0 con 0x7f772015db70 2024-04-25T10:03:06.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.410+0000 7f7725092640 1 --2- 172.21.15.19:0/3720009080 >> v2:172.21.15.112:3300/0 conn(0x7f77200b5c30 0x7f77200d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.411+0000 7f7724891640 1 --2- 172.21.15.19:0/3720009080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772015db70 0x7f7720162150 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f772008ccc0 tx=0x7f771802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c100f9e54a2fdc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.411+0000 7f771f7fe640 1 -- 172.21.15.19:0/3720009080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718037070 con 0x7f772015db70 2024-04-25T10:03:06.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.411+0000 7f771f7fe640 1 -- 172.21.15.19:0/3720009080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7718032440 con 0x7f772015db70 2024-04-25T10:03:06.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.411+0000 7f771f7fe640 1 -- 172.21.15.19:0/3720009080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718032810 con 0x7f772015db70 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.412+0000 7f7726af5640 1 -- 172.21.15.19:0/3720009080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772015db70 msgr2=0x7f7720162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.412+0000 7f7726af5640 1 --2- 172.21.15.19:0/3720009080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772015db70 0x7f7720162150 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f772008ccc0 tx=0x7f771802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.412+0000 7f7726af5640 1 -- 172.21.15.19:0/3720009080 shutdown_connections 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.412+0000 7f7726af5640 1 --2- 172.21.15.19:0/3720009080 >> v2:172.21.15.112:3300/0 conn(0x7f77200b5c30 0x7f77200d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.412+0000 7f7726af5640 1 --2- 172.21.15.19:0/3720009080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772015db70 0x7f7720162150 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.412+0000 7f7726af5640 1 -- 172.21.15.19:0/3720009080 >> 172.21.15.19:0/3720009080 conn(0x7f77200d0520 msgr2=0x7f77200d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.412+0000 7f7726af5640 1 -- 172.21.15.19:0/3720009080 shutdown_connections 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.412+0000 7f7726af5640 1 -- 172.21.15.19:0/3720009080 wait complete. 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.413+0000 7f7726af5640 1 Processor -- start 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.413+0000 7f7726af5640 1 -- start start 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7726af5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77200b5c30 0x7f77200cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7726af5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77200b6cb0 0x7f77201f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7726af5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772015db70 0x7f772015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7726af5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77200ce530 con 0x7f772015db70 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7726af5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772015df50 con 0x7f77200b6cb0 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7726af5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77200cf1c0 con 0x7f77200b5c30 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7725092640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772015db70 0x7f772015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7725092640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772015db70 0x7f772015b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60316/0 (socket says 172.21.15.19:60316) 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7725092640 1 -- 172.21.15.19:0/4188964277 learned_addr learned my addr 172.21.15.19:0/4188964277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f771ffff640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77200b6cb0 0x7f77201f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7724891640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77200b5c30 0x7f77200cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7725092640 1 -- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77200b5c30 msgr2=0x7f77200cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7725092640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77200b5c30 0x7f77200cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7725092640 1 -- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77200b6cb0 msgr2=0x7f77201f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7725092640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77200b6cb0 0x7f77201f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.414+0000 7f7725092640 1 -- 172.21.15.19:0/4188964277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77200ab570 con 0x7f772015db70 2024-04-25T10:03:06.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.415+0000 7f7725092640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772015db70 0x7f772015b170 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f770800c7f0 tx=0x7f770800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.415+0000 7f771d7fa640 1 -- 172.21.15.19:0/4188964277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708015070 con 0x7f772015db70 2024-04-25T10:03:06.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.415+0000 7f771d7fa640 1 -- 172.21.15.19:0/4188964277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77080077c0 con 0x7f772015db70 2024-04-25T10:03:06.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.415+0000 7f771d7fa640 1 -- 172.21.15.19:0/4188964277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708007bb0 con 0x7f772015db70 2024-04-25T10:03:06.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.415+0000 7f7726af5640 1 -- 172.21.15.19:0/4188964277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77200aacd0 con 0x7f772015db70 2024-04-25T10:03:06.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.415+0000 7f7726af5640 1 -- 172.21.15.19:0/4188964277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77200ab290 con 0x7f772015db70 2024-04-25T10:03:06.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.416+0000 7f7726af5640 1 -- 172.21.15.19:0/4188964277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7720073990 con 0x7f772015db70 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.416+0000 7f771d7fa640 1 -- 172.21.15.19:0/4188964277 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7708018070 con 0x7f772015db70 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.417+0000 7f771d7fa640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f770006b4d0 0x7f770006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.417+0000 7f771d7fa640 1 -- 172.21.15.19:0/4188964277 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7708089f90 con 0x7f772015db70 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.417+0000 7f7724891640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f770006b4d0 0x7f770006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.417+0000 7f7724891640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f770006b4d0 0x7f770006d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7718034000 tx=0x7f771802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.421+0000 7f771d7fa640 1 -- 172.21.15.19:0/4188964277 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f77200ab570 con 0x7f772015db70 2024-04-25T10:03:06.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.601+0000 7f7726af5640 1 -- 172.21.15.19:0/4188964277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77200d1820 con 0x7f772015db70 2024-04-25T10:03:06.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.602+0000 7f771d7fa640 1 -- 172.21.15.19:0/4188964277 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+13293 (secure 0 0 0) 0x7f77080589c0 con 0x7f772015db70 2024-04-25T10:03:06.606 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":145,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:03:05.290248+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6837/2648279035":"2024-04-26T10:03:02.277138+0000","172.21.15.112:6835/2648279035":"2024-04-26T10:03:02.277138+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 -- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f770006b4d0 msgr2=0x7f770006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f770006b4d0 0x7f770006d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7718034000 tx=0x7f771802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 -- 172.21.15.19:0/4188964277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772015db70 msgr2=0x7f772015b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772015db70 0x7f772015b170 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f770800c7f0 tx=0x7f770800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 -- 172.21.15.19:0/4188964277 shutdown_connections 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f770006b4d0 0x7f770006d990 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f772015db70 0x7f772015b170 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77200b6cb0 0x7f77201f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 --2- 172.21.15.19:0/4188964277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77200b5c30 0x7f77200cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 -- 172.21.15.19:0/4188964277 >> 172.21.15.19:0/4188964277 conn(0x7f77200d0520 msgr2=0x7f772015a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 -- 172.21.15.19:0/4188964277 shutdown_connections 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.604+0000 7f7726af5640 1 -- 172.21.15.19:0/4188964277 wait complete. 2024-04-25T10:03:06.620 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:06.620 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:06.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.719+0000 7f9d138e1640 1 Processor -- start 2024-04-25T10:03:06.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.720+0000 7f9d138e1640 1 -- start start 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.720+0000 7f9d138e1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d0c15dbd0 0x7f9d0c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.720+0000 7f9d138e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d0c06bbf0 0x7f9d0c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.720+0000 7f9d138e1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0cb0e0 con 0x7f9d0c06ff30 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.720+0000 7f9d138e1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c0cb3c0 con 0x7f9d0c15dbd0 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.720+0000 7f9d138e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0cb6a0 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.720+0000 7f9d1167d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d0c15dbd0 0x7f9d0c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.721+0000 7f9d1167d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d0c15dbd0 0x7f9d0c0caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60328/0 (socket says 172.21.15.19:60328) 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.721+0000 7f9d10e7c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9d0c06ff30 0x7f9d0c0cc180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47736/0 (socket says 172.21.15.19:47736) 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.721+0000 7f9d1167d640 1 -- 172.21.15.19:0/1760402486 learned_addr learned my addr 172.21.15.19:0/1760402486 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.721+0000 7f9d11e7e640 1 --2- 172.21.15.19:0/1760402486 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d0c06bbf0 0x7f9d0c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.721+0000 7f9d1167d640 1 -- 172.21.15.19:0/1760402486 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d0c06bbf0 msgr2=0x7f9d0c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.721+0000 7f9d1167d640 1 --2- 172.21.15.19:0/1760402486 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d0c06bbf0 0x7f9d0c06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.721+0000 7f9d1167d640 1 -- 172.21.15.19:0/1760402486 >> v1:172.21.15.19:6789/0 conn(0x7f9d0c06ff30 legacy=0x7f9d0c0cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.721+0000 7f9d0bfff640 1 -- 172.21.15.19:0/1760402486 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201484082 0 0) 0x7f9d0c0cb0e0 con 0x7f9d0c06ff30 2024-04-25T10:03:06.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.721+0000 7f9d1167d640 1 -- 172.21.15.19:0/1760402486 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0cb980 con 0x7f9d0c15dbd0 2024-04-25T10:03:06.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d1167d640 1 --2- 172.21.15.19:0/1760402486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d0c15dbd0 0x7f9d0c0caa60 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009870 tx=0x7f9d0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ab94fe3ed9bd6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d0bfff640 1 -- 172.21.15.19:0/1760402486 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04037070 con 0x7f9d0c15dbd0 2024-04-25T10:03:06.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d0bfff640 1 -- 172.21.15.19:0/1760402486 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d04032440 con 0x7f9d0c15dbd0 2024-04-25T10:03:06.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d0bfff640 1 -- 172.21.15.19:0/1760402486 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04032810 con 0x7f9d0c15dbd0 2024-04-25T10:03:06.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d138e1640 1 -- 172.21.15.19:0/1760402486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d0c15dbd0 msgr2=0x7f9d0c0caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d138e1640 1 --2- 172.21.15.19:0/1760402486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d0c15dbd0 0x7f9d0c0caa60 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009870 tx=0x7f9d0402a270 comp rx=0 tx=0).stop 2024-04-25T10:03:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d138e1640 1 -- 172.21.15.19:0/1760402486 shutdown_connections 2024-04-25T10:03:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d138e1640 1 --2- 172.21.15.19:0/1760402486 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d0c06bbf0 0x7f9d0c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d138e1640 1 --2- 172.21.15.19:0/1760402486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d0c15dbd0 0x7f9d0c0caa60 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d138e1640 1 -- 172.21.15.19:0/1760402486 >> 172.21.15.19:0/1760402486 conn(0x7f9d0c071af0 msgr2=0x7f9d0c071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d138e1640 1 -- 172.21.15.19:0/1760402486 shutdown_connections 2024-04-25T10:03:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.722+0000 7f9d138e1640 1 -- 172.21.15.19:0/1760402486 wait complete. 2024-04-25T10:03:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.724+0000 7f9d138e1640 1 Processor -- start 2024-04-25T10:03:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.724+0000 7f9d138e1640 1 -- start start 2024-04-25T10:03:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.724+0000 7f9d138e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d0c06bbf0 0x7f9d0c0c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.724+0000 7f9d138e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d0c06ff30 0x7f9d0c1f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.724+0000 7f9d138e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d0c15dbd0 0x7f9d0c1f0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.724+0000 7f9d138e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c15e130 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.724+0000 7f9d138e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c15dfb0 con 0x7f9d0c06ff30 2024-04-25T10:03:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.724+0000 7f9d138e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c070360 con 0x7f9d0c15dbd0 2024-04-25T10:03:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.724+0000 7f9d1167d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d0c06bbf0 0x7f9d0c0c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d1167d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d0c06bbf0 0x7f9d0c0c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60330/0 (socket says 172.21.15.19:60330) 2024-04-25T10:03:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d1167d640 1 -- 172.21.15.19:0/2299770838 learned_addr learned my addr 172.21.15.19:0/2299770838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d11e7e640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d0c15dbd0 0x7f9d0c1f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d10e7c640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d0c06ff30 0x7f9d0c1f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d1167d640 1 -- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d0c15dbd0 msgr2=0x7f9d0c1f0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d1167d640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d0c15dbd0 0x7f9d0c1f0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d1167d640 1 -- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d0c06ff30 msgr2=0x7f9d0c1f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d1167d640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d0c06ff30 0x7f9d0c1f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d1167d640 1 -- 172.21.15.19:0/2299770838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0c7620 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d10e7c640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d0c06ff30 0x7f9d0c1f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d1167d640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d0c06bbf0 0x7f9d0c0c6f20 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f9d0402a2b0 tx=0x7f9d040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d09ffb640 1 -- 172.21.15.19:0/2299770838 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04041020 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d09ffb640 1 -- 172.21.15.19:0/2299770838 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d040306c0 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d138e1640 1 -- 172.21.15.19:0/2299770838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c0c7900 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d09ffb640 1 -- 172.21.15.19:0/2299770838 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04037040 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.725+0000 7f9d138e1640 1 -- 172.21.15.19:0/2299770838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c0c7ec0 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.726+0000 7f9d138e1640 1 -- 172.21.15.19:0/2299770838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c073a00 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.727+0000 7f9d09ffb640 1 -- 172.21.15.19:0/2299770838 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d04044020 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.727+0000 7f9d09ffb640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cf806b4d0 0x7f9cf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.727+0000 7f9d09ffb640 1 -- 172.21.15.19:0/2299770838 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9d040aa890 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.727+0000 7f9d10e7c640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cf806b4d0 0x7f9cf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.728+0000 7f9d10e7c640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cf806b4d0 0x7f9cf806d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c068620 tx=0x7f9d00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.731+0000 7f9d09ffb640 1 -- 172.21.15.19:0/2299770838 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d0404b030 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.911+0000 7f9d138e1640 1 -- 172.21.15.19:0/2299770838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d0c061b30 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.911+0000 7f9d09ffb640 1 -- 172.21.15.19:0/2299770838 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d04079880 con 0x7f9d0c06bbf0 2024-04-25T10:03:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:03:06.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 -- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cf806b4d0 msgr2=0x7f9cf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cf806b4d0 0x7f9cf806d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c068620 tx=0x7f9d00009040 comp rx=0 tx=0).stop 2024-04-25T10:03:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 -- 172.21.15.19:0/2299770838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d0c06bbf0 msgr2=0x7f9d0c0c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d0c06bbf0 0x7f9d0c0c6f20 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f9d0402a2b0 tx=0x7f9d040046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 -- 172.21.15.19:0/2299770838 shutdown_connections 2024-04-25T10:03:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9cf806b4d0 0x7f9cf806d990 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d0c15dbd0 0x7f9d0c1f0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d0c06ff30 0x7f9d0c1f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 --2- 172.21.15.19:0/2299770838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d0c06bbf0 0x7f9d0c0c6f20 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 -- 172.21.15.19:0/2299770838 >> 172.21.15.19:0/2299770838 conn(0x7f9d0c071af0 msgr2=0x7f9d0c0c6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 -- 172.21.15.19:0/2299770838 shutdown_connections 2024-04-25T10:03:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:06.913+0000 7f9d138e1640 1 -- 172.21.15.19:0/2299770838 wait complete. 2024-04-25T10:03:06.929 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:03:07.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.025+0000 7fcf7f693640 1 Processor -- start 2024-04-25T10:03:07.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.026+0000 7fcf7f693640 1 -- start start 2024-04-25T10:03:07.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.026+0000 7fcf7f693640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf780d2e90 0x7fcf780d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.026+0000 7fcf7f693640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d0900 0x7fcf780b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.026+0000 7fcf7f693640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf780b9940 con 0x7fcf7815dd30 2024-04-25T10:03:07.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.026+0000 7fcf7f693640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf780d0d40 con 0x7fcf780d2e90 2024-04-25T10:03:07.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.026+0000 7fcf7f693640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf780d1df0 con 0x7fcf780d0900 2024-04-25T10:03:07.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.027+0000 7fcf7cc2e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf780d2e90 0x7fcf780d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.027+0000 7fcf7cc2e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf780d2e90 0x7fcf780d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60338/0 (socket says 172.21.15.19:60338) 2024-04-25T10:03:07.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.027+0000 7fcf7cc2e640 1 -- 172.21.15.19:0/2502491395 learned_addr learned my addr 172.21.15.19:0/2502491395 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:07.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.027+0000 7fcf7dc30640 1 --2- 172.21.15.19:0/2502491395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d0900 0x7fcf780b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.027+0000 7fcf7cc2e640 1 -- 172.21.15.19:0/2502491395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d0900 msgr2=0x7fcf780b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.027+0000 7fcf7cc2e640 1 --2- 172.21.15.19:0/2502491395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d0900 0x7fcf780b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.027+0000 7fcf7cc2e640 1 -- 172.21.15.19:0/2502491395 >> v1:172.21.15.112:6789/0 conn(0x7fcf7815dd30 legacy=0x7fcf780b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.027+0000 7fcf7cc2e640 1 -- 172.21.15.19:0/2502491395 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf780d20d0 con 0x7fcf780d2e90 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.027+0000 7fcf7dc30640 1 --2- 172.21.15.19:0/2502491395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d0900 0x7fcf780b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.028+0000 7fcf7cc2e640 1 --2- 172.21.15.19:0/2502491395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf780d2e90 0x7fcf780d03c0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009850 tx=0x7fcf6802a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0179b10db2ffc44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.028+0000 7fcf6ffff640 1 -- 172.21.15.19:0/2502491395 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6802ae60 con 0x7fcf780d2e90 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.028+0000 7fcf6ffff640 1 -- 172.21.15.19:0/2502491395 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf68004030 con 0x7fcf780d2e90 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.028+0000 7fcf6ffff640 1 -- 172.21.15.19:0/2502491395 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68004460 con 0x7fcf780d2e90 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.028+0000 7fcf7f693640 1 -- 172.21.15.19:0/2502491395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf780d2e90 msgr2=0x7fcf780d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.028+0000 7fcf7f693640 1 --2- 172.21.15.19:0/2502491395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf780d2e90 0x7fcf780d03c0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009850 tx=0x7fcf6802a150 comp rx=0 tx=0).stop 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.028+0000 7fcf7f693640 1 -- 172.21.15.19:0/2502491395 shutdown_connections 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.028+0000 7fcf7f693640 1 --2- 172.21.15.19:0/2502491395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d0900 0x7fcf780b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.028+0000 7fcf7f693640 1 --2- 172.21.15.19:0/2502491395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf780d2e90 0x7fcf780d03c0 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.028+0000 7fcf7f693640 1 -- 172.21.15.19:0/2502491395 >> 172.21.15.19:0/2502491395 conn(0x7fcf78061c30 msgr2=0x7fcf7806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.029+0000 7fcf7f693640 1 -- 172.21.15.19:0/2502491395 shutdown_connections 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.029+0000 7fcf7f693640 1 -- 172.21.15.19:0/2502491395 wait complete. 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.030+0000 7fcf7f693640 1 Processor -- start 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.030+0000 7fcf7f693640 1 -- start start 2024-04-25T10:03:07.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.030+0000 7fcf7f693640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf780d0900 0x7fcf7814ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.030+0000 7fcf7f693640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d2e90 0x7fcf781f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.030+0000 7fcf7f693640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf7815dd30 0x7fcf7815f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.030+0000 7fcf7f693640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf780d2970 con 0x7fcf7815dd30 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.030+0000 7fcf7f693640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf780d2690 con 0x7fcf780d0900 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.030+0000 7fcf7f693640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf780d1a10 con 0x7fcf780d2e90 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7dc30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf7815dd30 0x7fcf7815f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7dc30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf7815dd30 0x7fcf7815f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60344/0 (socket says 172.21.15.19:60344) 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7dc30640 1 -- 172.21.15.19:0/2643119980 learned_addr learned my addr 172.21.15.19:0/2643119980 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7d42f640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf780d0900 0x7fcf7814ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7cc2e640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d2e90 0x7fcf781f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7dc30640 1 -- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d2e90 msgr2=0x7fcf781f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7dc30640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d2e90 0x7fcf781f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7dc30640 1 -- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf780d0900 msgr2=0x7fcf7814ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7dc30640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf780d0900 0x7fcf7814ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7dc30640 1 -- 172.21.15.19:0/2643119980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7814b320 con 0x7fcf7815dd30 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7cc2e640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d2e90 0x7fcf781f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7dc30640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf7815dd30 0x7fcf7815f700 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000cef0 tx=0x7fcf7000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf6dffb640 1 -- 172.21.15.19:0/2643119980 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7001a070 con 0x7fcf7815dd30 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf6dffb640 1 -- 172.21.15.19:0/2643119980 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7000d860 con 0x7fcf7815dd30 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf7f693640 1 -- 172.21.15.19:0/2643119980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf781579b0 con 0x7fcf7815dd30 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.031+0000 7fcf6dffb640 1 -- 172.21.15.19:0/2643119980 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7000dc50 con 0x7fcf7815dd30 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.032+0000 7fcf7f693640 1 -- 172.21.15.19:0/2643119980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf78157f70 con 0x7fcf7815dd30 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.032+0000 7fcf7f693640 1 -- 172.21.15.19:0/2643119980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf78073990 con 0x7fcf7815dd30 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.033+0000 7fcf6dffb640 1 -- 172.21.15.19:0/2643119980 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf7001d070 con 0x7fcf7815dd30 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.033+0000 7fcf6dffb640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf5c06b4d0 0x7fcf5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.033+0000 7fcf6dffb640 1 -- 172.21.15.19:0/2643119980 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcf7008a8b0 con 0x7fcf7815dd30 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.033+0000 7fcf7d42f640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf5c06b4d0 0x7fcf5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.034+0000 7fcf7d42f640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf5c06b4d0 0x7fcf5c06d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcf74006fd0 tx=0x7fcf74009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.037+0000 7fcf6dffb640 1 -- 172.21.15.19:0/2643119980 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf70059310 con 0x7fcf7815dd30 2024-04-25T10:03:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:07.292+0000 7fcf7f693640 1 -- 172.21.15.19:0/2643119980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7fcf781f3b10 con 0x7fcf7815dd30 2024-04-25T10:03:08.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.298+0000 7fcf6dffb640 1 -- 172.21.15.19:0/2643119980 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v499) v1 ==== 143+0+0 (secure 0 0 0) 0x7fcf7005d3b0 con 0x7fcf7815dd30 2024-04-25T10:03:08.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 -- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf5c06b4d0 msgr2=0x7fcf5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf5c06b4d0 0x7fcf5c06d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcf74006fd0 tx=0x7fcf74009060 comp rx=0 tx=0).stop 2024-04-25T10:03:08.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 -- 172.21.15.19:0/2643119980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf7815dd30 msgr2=0x7fcf7815f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf7815dd30 0x7fcf7815f700 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000cef0 tx=0x7fcf7000a710 comp rx=0 tx=0).stop 2024-04-25T10:03:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 -- 172.21.15.19:0/2643119980 shutdown_connections 2024-04-25T10:03:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf5c06b4d0 0x7fcf5c06d990 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf7815dd30 0x7fcf7815f700 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf780d2e90 0x7fcf781f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 --2- 172.21.15.19:0/2643119980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf780d0900 0x7fcf7814ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 -- 172.21.15.19:0/2643119980 >> 172.21.15.19:0/2643119980 conn(0x7fcf78061c30 msgr2=0x7fcf780c8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 -- 172.21.15.19:0/2643119980 shutdown_connections 2024-04-25T10:03:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.301+0000 7fcf7f693640 1 -- 172.21.15.19:0/2643119980 wait complete. 2024-04-25T10:03:08.320 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:03:08.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.425+0000 7ff0d27e0640 1 Processor -- start 2024-04-25T10:03:08.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.426+0000 7ff0d27e0640 1 -- start start 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.426+0000 7ff0d27e0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff0cc0cf600 0x7ff0cc0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.426+0000 7ff0d27e0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0cc0cbf20 0x7ff0cc0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.426+0000 7ff0d27e0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0cc0c75e0 con 0x7ff0cc15bb50 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.426+0000 7ff0d27e0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0cc0c8680 con 0x7ff0cc0cf600 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.426+0000 7ff0d27e0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0cff80 con 0x7ff0cc0cbf20 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.426+0000 7ff0cbfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff0cc15bb50 0x7ff0cc0c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45114/0 (socket says 172.21.15.19:45114) 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0cbfff640 1 -- 172.21.15.19:0/2739535675 learned_addr learned my addr 172.21.15.19:0/2739535675 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576260972 0 0) 0x7ff0cc0c75e0 con 0x7ff0cc15bb50 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0b4003520 con 0x7ff0cc15bb50 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0cb7fe640 1 --2- 172.21.15.19:0/2739535675 >> v2:172.21.15.112:3300/0 conn(0x7ff0cc0cf600 0x7ff0cc0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0d0d7d640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0cc0cbf20 0x7ff0cc0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3881289531 0 0) 0x7ff0b4003520 con 0x7ff0cc15bb50 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0cc0cbf20 msgr2=0x7ff0cc0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0cc0cbf20 0x7ff0cc0cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 >> v2:172.21.15.112:3300/0 conn(0x7ff0cc0cf600 msgr2=0x7ff0cc0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 --2- 172.21.15.19:0/2739535675 >> v2:172.21.15.112:3300/0 conn(0x7ff0cc0cf600 0x7ff0cc0cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc0c78c0 con 0x7ff0cc15bb50 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7ff0bc001670 con 0x7ff0cc15bb50 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 >> v1:172.21.15.19:6789/0 conn(0x7ff0cc15bb50 legacy=0x7ff0cc0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0cc0cbf20 0x7ff0b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b4003f30 0x7ff0b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4004d40 0x7ff0b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0c8680 con 0x7ff0b4003f30 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0cff80 con 0x7ff0b4004d40 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b400b9f0 con 0x7ff0cc0cbf20 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0cb7fe640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b4003f30 0x7ff0b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0cbfff640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0cc0cbf20 0x7ff0b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.427+0000 7ff0d0d7d640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4004d40 0x7ff0b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.428+0000 7ff0cb7fe640 1 -- 172.21.15.19:0/2739535675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0cc0cbf20 msgr2=0x7ff0b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.428+0000 7ff0cb7fe640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0cc0cbf20 0x7ff0b40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.428+0000 7ff0cb7fe640 1 -- 172.21.15.19:0/2739535675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4004d40 msgr2=0x7ff0b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.428+0000 7ff0cb7fe640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4004d40 0x7ff0b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.428+0000 7ff0cb7fe640 1 -- 172.21.15.19:0/2739535675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4003520 con 0x7ff0b4003f30 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.428+0000 7ff0cb7fe640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b4003f30 0x7ff0b4004800 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009ac0 tx=0x7ff0b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982342f385765c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.428+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0036070 con 0x7ff0b4003f30 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.428+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0031400 con 0x7ff0b4003f30 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.429+0000 7ff0caffd640 1 -- 172.21.15.19:0/2739535675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0009690 con 0x7ff0b4003f30 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.429+0000 7ff0d27e0640 1 -- 172.21.15.19:0/2739535675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b4003f30 msgr2=0x7ff0b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.429+0000 7ff0d27e0640 1 --2- 172.21.15.19:0/2739535675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b4003f30 0x7ff0b4004800 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009ac0 tx=0x7ff0b4005830 comp rx=0 tx=0).stop 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.429+0000 7ff0d0d7d640 1 -- 172.21.15.19:0/2739535675 reap_dead start 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.429+0000 7ff0d27e0640 1 -- 172.21.15.19:0/2739535675 shutdown_connections 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.429+0000 7ff0d27e0640 1 -- 172.21.15.19:0/2739535675 >> 172.21.15.19:0/2739535675 conn(0x7ff0cc06f500 msgr2=0x7ff0cc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.429+0000 7ff0d27e0640 1 -- 172.21.15.19:0/2739535675 shutdown_connections 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.429+0000 7ff0d27e0640 1 -- 172.21.15.19:0/2739535675 wait complete. 2024-04-25T10:03:08.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.430+0000 7ff0d27e0640 1 Processor -- start 2024-04-25T10:03:08.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.431+0000 7ff0d27e0640 1 -- start start 2024-04-25T10:03:08.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.431+0000 7ff0d27e0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0b4003f30 0x7ff0cc1473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.431+0000 7ff0d27e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc0cd380 0x7ff0cc0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.431+0000 7ff0d27e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0cc0c88f0 0x7ff0cc0c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.431+0000 7ff0d27e0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0c8160 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.431+0000 7ff0d27e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0c7e80 con 0x7ff0cc0cd380 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.431+0000 7ff0d27e0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0c72a0 con 0x7ff0b4003f30 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.431+0000 7ff0d0d7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0cc0c88f0 0x7ff0cc0c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.431+0000 7ff0d0d7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0cc0c88f0 0x7ff0cc0c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39430/0 (socket says 172.21.15.19:39430) 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.431+0000 7ff0d0d7d640 1 -- 172.21.15.19:0/753211745 learned_addr learned my addr 172.21.15.19:0/753211745 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0cb7fe640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc0cd380 0x7ff0cc0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0cbfff640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0b4003f30 0x7ff0cc1473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0d0d7d640 1 -- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0b4003f30 msgr2=0x7ff0cc1473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0d0d7d640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0b4003f30 0x7ff0cc1473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0d0d7d640 1 -- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc0cd380 msgr2=0x7ff0cc0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0d0d7d640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc0cd380 0x7ff0cc0cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0d0d7d640 1 -- 172.21.15.19:0/753211745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc0c92b0 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0cbfff640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0b4003f30 0x7ff0cc1473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0cb7fe640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc0cd380 0x7ff0cc0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0d0d7d640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0cc0c88f0 0x7ff0cc0c8d70 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc06f270 tx=0x7ff0c400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0c8ff9640 1 -- 172.21.15.19:0/753211745 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c400fde0 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0c8ff9640 1 -- 172.21.15.19:0/753211745 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c401d070 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0d27e0640 1 -- 172.21.15.19:0/753211745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0cc0c9570 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0c8ff9640 1 -- 172.21.15.19:0/753211745 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4015670 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.432+0000 7ff0d27e0640 1 -- 172.21.15.19:0/753211745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0cc0c9b30 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.433+0000 7ff0d27e0640 1 -- 172.21.15.19:0/753211745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0cc15ac80 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.434+0000 7ff0c8ff9640 1 -- 172.21.15.19:0/753211745 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0c4015810 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.434+0000 7ff0c8ff9640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0b806b500 0x7ff0b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.434+0000 7ff0c8ff9640 1 -- 172.21.15.19:0/753211745 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff0c408a9a0 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.434+0000 7ff0cbfff640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0b806b500 0x7ff0b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.435+0000 7ff0cbfff640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0b806b500 0x7ff0b806d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc001510 tx=0x7ff0bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.438+0000 7ff0c8ff9640 1 -- 172.21.15.19:0/753211745 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0cc15ac80 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.617+0000 7ff0d27e0640 1 -- 172.21.15.19:0/753211745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff0cc073a00 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.620+0000 7ff0c8ff9640 1 -- 172.21.15.19:0/753211745 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff0c4010040 con 0x7ff0cc0c88f0 2024-04-25T10:03:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 -- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0b806b500 msgr2=0x7ff0b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0b806b500 0x7ff0b806d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc001510 tx=0x7ff0bc009040 comp rx=0 tx=0).stop 2024-04-25T10:03:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 -- 172.21.15.19:0/753211745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0cc0c88f0 msgr2=0x7ff0cc0c8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0cc0c88f0 0x7ff0cc0c8d70 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc06f270 tx=0x7ff0c400f580 comp rx=0 tx=0).stop 2024-04-25T10:03:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 -- 172.21.15.19:0/753211745 shutdown_connections 2024-04-25T10:03:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff0b806b500 0x7ff0b806d9c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0cc0c88f0 0x7ff0cc0c8d70 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc0cd380 0x7ff0cc0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 --2- 172.21.15.19:0/753211745 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0b4003f30 0x7ff0cc1473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 -- 172.21.15.19:0/753211745 >> 172.21.15.19:0/753211745 conn(0x7ff0cc06f500 msgr2=0x7ff0cc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 -- 172.21.15.19:0/753211745 shutdown_connections 2024-04-25T10:03:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.622+0000 7ff0d27e0640 1 -- 172.21.15.19:0/753211745 wait complete. 2024-04-25T10:03:08.638 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:03:08.638 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:08.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:08.715 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:03:08.715 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:03:08.716 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:08.731 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:03:08.731 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:03:08.731 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:08.773 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:08.773 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:03:08.773 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:08.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:03:08.789 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:03:08.789 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:08.827 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:08.830 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:08.830 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:03:08.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:08.832 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:08.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.927+0000 7f15a937e640 1 Processor -- start 2024-04-25T10:03:08.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.927+0000 7f15a937e640 1 -- start start 2024-04-25T10:03:08.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a937e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15a415dbe0 0x7f15a40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a937e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f15a40b92a0 0x7f15a40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a937e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a40b97a0 con 0x7f15a40b2cb0 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a937e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a415dfc0 con 0x7f15a415dbe0 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a937e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a40d1a00 con 0x7f15a40b92a0 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a2ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15a415dbe0 0x7f15a40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a2ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15a415dbe0 0x7f15a40d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39434/0 (socket says 172.21.15.19:39434) 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a2ffd640 1 -- 172.21.15.19:0/576802409 learned_addr learned my addr 172.21.15.19:0/576802409 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a37fe640 1 --2- 172.21.15.19:0/576802409 >> v2:172.21.15.112:3300/0 conn(0x7f15a40b92a0 0x7f15a40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a2ffd640 1 -- 172.21.15.19:0/576802409 >> v2:172.21.15.112:3300/0 conn(0x7f15a40b92a0 msgr2=0x7f15a40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a2ffd640 1 --2- 172.21.15.19:0/576802409 >> v2:172.21.15.112:3300/0 conn(0x7f15a40b92a0 0x7f15a40d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a2ffd640 1 -- 172.21.15.19:0/576802409 >> v1:172.21.15.19:6789/0 conn(0x7f15a40b2cb0 legacy=0x7f15a40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a2ffd640 1 -- 172.21.15.19:0/576802409 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a40d1c50 con 0x7f15a415dbe0 2024-04-25T10:03:08.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.928+0000 7f15a27fc640 1 -- 172.21.15.19:0/576802409 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542717648 0 0) 0x7f15a40b97a0 con 0x7f15a40b2cb0 2024-04-25T10:03:08.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.929+0000 7f15a2ffd640 1 --2- 172.21.15.19:0/576802409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a415dbe0 0x7f15a40d0910 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f15a40cfb90 tx=0x7f159402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8856f9bd6af280ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.929+0000 7f15a27fc640 1 -- 172.21.15.19:0/576802409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594037070 con 0x7f15a415dbe0 2024-04-25T10:03:08.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.929+0000 7f15a27fc640 1 -- 172.21.15.19:0/576802409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1594032440 con 0x7f15a415dbe0 2024-04-25T10:03:08.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.929+0000 7f15a27fc640 1 -- 172.21.15.19:0/576802409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594032810 con 0x7f15a415dbe0 2024-04-25T10:03:08.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.930+0000 7f15a937e640 1 -- 172.21.15.19:0/576802409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a415dbe0 msgr2=0x7f15a40d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.930+0000 7f15a937e640 1 --2- 172.21.15.19:0/576802409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a415dbe0 0x7f15a40d0910 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f15a40cfb90 tx=0x7f159402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.930+0000 7f15a937e640 1 -- 172.21.15.19:0/576802409 shutdown_connections 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.930+0000 7f15a937e640 1 --2- 172.21.15.19:0/576802409 >> v2:172.21.15.112:3300/0 conn(0x7f15a40b92a0 0x7f15a40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.930+0000 7f15a937e640 1 --2- 172.21.15.19:0/576802409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a415dbe0 0x7f15a40d0910 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.930+0000 7f15a937e640 1 -- 172.21.15.19:0/576802409 >> 172.21.15.19:0/576802409 conn(0x7f15a4061a20 msgr2=0x7f15a4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.930+0000 7f15a937e640 1 -- 172.21.15.19:0/576802409 shutdown_connections 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.930+0000 7f15a937e640 1 -- 172.21.15.19:0/576802409 wait complete. 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.931+0000 7f15a937e640 1 Processor -- start 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.931+0000 7f15a937e640 1 -- start start 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f15a937e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15a40b2cb0 0x7f15a40cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f15a937e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40b92a0 0x7f15a40caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f15a937e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a415dbe0 0x7f15a41f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f15a937e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40b6ab0 con 0x7f15a40b92a0 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f15a937e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40d2210 con 0x7f15a415dbe0 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f15a937e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40b6c30 con 0x7f15a40b2cb0 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40b92a0 0x7f15a40caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40b92a0 0x7f15a40caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39448/0 (socket says 172.21.15.19:39448) 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159bfff640 1 -- 172.21.15.19:0/3878525300 learned_addr learned my addr 172.21.15.19:0/3878525300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f15a2ffd640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15a40b2cb0 0x7f15a40cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f15a37fe640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a415dbe0 0x7f15a41f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159bfff640 1 -- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15a40b2cb0 msgr2=0x7f15a40cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159bfff640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15a40b2cb0 0x7f15a40cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159bfff640 1 -- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a415dbe0 msgr2=0x7f15a41f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159bfff640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a415dbe0 0x7f15a41f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159bfff640 1 -- 172.21.15.19:0/3878525300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a40cb1f0 con 0x7f15a40b92a0 2024-04-25T10:03:08.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159bfff640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40b92a0 0x7f15a40caa50 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c520 tx=0x7f158c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159b7fe640 1 -- 172.21.15.19:0/3878525300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c017070 con 0x7f15a40b92a0 2024-04-25T10:03:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159b7fe640 1 -- 172.21.15.19:0/3878525300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c0035f0 con 0x7f15a40b92a0 2024-04-25T10:03:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f15a937e640 1 -- 172.21.15.19:0/3878525300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a40cb4d0 con 0x7f15a40b92a0 2024-04-25T10:03:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.932+0000 7f159b7fe640 1 -- 172.21.15.19:0/3878525300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c0039e0 con 0x7f15a40b92a0 2024-04-25T10:03:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.933+0000 7f15a937e640 1 -- 172.21.15.19:0/3878525300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a40cba90 con 0x7f15a40b92a0 2024-04-25T10:03:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.933+0000 7f15a937e640 1 -- 172.21.15.19:0/3878525300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a4073990 con 0x7f15a40b92a0 2024-04-25T10:03:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.934+0000 7f159b7fe640 1 -- 172.21.15.19:0/3878525300 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f158c01a070 con 0x7f15a40b92a0 2024-04-25T10:03:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.934+0000 7f159b7fe640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f158406b4d0 0x7f158406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.934+0000 7f159b7fe640 1 -- 172.21.15.19:0/3878525300 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f158c089350 con 0x7f15a40b92a0 2024-04-25T10:03:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.935+0000 7f15a2ffd640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f158406b4d0 0x7f158406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.935+0000 7f15a2ffd640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f158406b4d0 0x7f158406d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1594034000 tx=0x7f159402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:08.938+0000 7f159b7fe640 1 -- 172.21.15.19:0/3878525300 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f158c057d80 con 0x7f15a40b92a0 2024-04-25T10:03:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.156+0000 7f15a937e640 1 -- 172.21.15.19:0/3878525300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15a40620e0 con 0x7f15a40b92a0 2024-04-25T10:03:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.157+0000 7f159b7fe640 1 -- 172.21.15.19:0/3878525300 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+3703 (secure 0 0 0) 0x7f158c05be20 con 0x7f15a40b92a0 2024-04-25T10:03:09.162 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:09.162 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":96,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7466,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/590882657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":590882657},{"type":"v1","addr":"172.21.15.112:6836","nonce":590882657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":8788,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2346375241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":2346375241},{"type":"v1","addr":"172.21.15.112:6837","nonce":2346375241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":9744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1740426538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1740426538},{"type":"v1","addr":"172.21.15.19:6833","nonce":1740426538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":10140,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2020317100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2020317100},{"type":"v1","addr":"172.21.15.19:6835","nonce":2020317100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[]} 2024-04-25T10:03:09.162 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 96 2024-04-25T10:03:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.158+0000 7f15a937e640 1 -- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f158406b4d0 msgr2=0x7f158406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.158+0000 7f15a937e640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f158406b4d0 0x7f158406d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1594034000 tx=0x7f159402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.158+0000 7f15a937e640 1 -- 172.21.15.19:0/3878525300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40b92a0 msgr2=0x7f15a40caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.158+0000 7f15a937e640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40b92a0 0x7f15a40caa50 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c520 tx=0x7f158c00c9e0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.158+0000 7f15a937e640 1 -- 172.21.15.19:0/3878525300 shutdown_connections 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.158+0000 7f15a937e640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f158406b4d0 0x7f158406d990 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.158+0000 7f15a937e640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a415dbe0 0x7f15a41f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.158+0000 7f15a937e640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40b92a0 0x7f15a40caa50 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.159+0000 7f15a937e640 1 --2- 172.21.15.19:0/3878525300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15a40b2cb0 0x7f15a40cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.159+0000 7f15a937e640 1 -- 172.21.15.19:0/3878525300 >> 172.21.15.19:0/3878525300 conn(0x7f15a4061a20 msgr2=0x7f15a415c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.159+0000 7f15a937e640 1 -- 172.21.15.19:0/3878525300 shutdown_connections 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.159+0000 7f15a937e640 1 -- 172.21.15.19:0/3878525300 wait complete. 2024-04-25T10:03:09.175 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:03:09.175 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:03:09.175 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:03:09.175 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:09.175 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:03:09.176 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:03:09.176 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:03:09.176 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:09.176 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:03:09.176 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:03:09.176 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:03:09.176 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:09.177 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:03:09.177 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:03:09.177 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:03:09.177 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:09.184 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:03:09.185 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:03:09.185 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:03:09.186 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:03:09.187 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:03:09.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:03:09.188 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:03:09.189 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:03:09.227 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:03:09.227 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:09.228 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:03:09.228 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:09.230 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:03:09.230 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:09.231 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:03:09.231 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:09.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:03:09.302 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:03:09.303 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T10:03:09.310 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T10:03:09.311 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T10:03:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.332+0000 7f657b84c640 1 Processor -- start 2024-04-25T10:03:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.332+0000 7f657b84c640 1 -- start start 2024-04-25T10:03:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f657b84c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f657415ba80 0x7f65740cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f657b84c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657415ce20 con 0x7f65740cf250 2024-04-25T10:03:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f657b84c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657406fe50 con 0x7f657415c990 2024-04-25T10:03:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f657b84c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6574070130 con 0x7f657415ba80 2024-04-25T10:03:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f6579de9640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f65740cf250 0x7f65740b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45156/0 (socket says 172.21.15.19:45156) 2024-04-25T10:03:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f6579de9640 1 -- 172.21.15.19:0/248264360 learned_addr learned my addr 172.21.15.19:0/248264360 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:09.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f65795e8640 1 --2- 172.21.15.19:0/248264360 >> v2:172.21.15.112:3300/0 conn(0x7f657415ba80 0x7f65740cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f656bfff640 1 -- 172.21.15.19:0/248264360 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810564429 0 0) 0x7f657415ce20 con 0x7f65740cf250 2024-04-25T10:03:09.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f656bfff640 1 -- 172.21.15.19:0/248264360 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6560003520 con 0x7f65740cf250 2024-04-25T10:03:09.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f65795e8640 1 -- 172.21.15.19:0/248264360 >> v1:172.21.15.112:6789/0 conn(0x7f657415c990 legacy=0x7f65740ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f65795e8640 1 -- 172.21.15.19:0/248264360 >> v1:172.21.15.19:6789/0 conn(0x7f65740cf250 legacy=0x7f65740b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.333+0000 7f65795e8640 1 -- 172.21.15.19:0/248264360 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65740cc8b0 con 0x7f657415ba80 2024-04-25T10:03:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.334+0000 7f65795e8640 1 --2- 172.21.15.19:0/248264360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657415ba80 0x7f65740cd390 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f65700097a0 tx=0x7f657002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f3d825f81b54d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.334+0000 7f656bfff640 1 -- 172.21.15.19:0/248264360 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570037070 con 0x7f657415ba80 2024-04-25T10:03:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.334+0000 7f656bfff640 1 -- 172.21.15.19:0/248264360 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6570032440 con 0x7f657415ba80 2024-04-25T10:03:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.334+0000 7f656bfff640 1 -- 172.21.15.19:0/248264360 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570037070 con 0x7f657415ba80 2024-04-25T10:03:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.335+0000 7f657b84c640 1 -- 172.21.15.19:0/248264360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657415ba80 msgr2=0x7f65740cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.335+0000 7f657b84c640 1 --2- 172.21.15.19:0/248264360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657415ba80 0x7f65740cd390 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f65700097a0 tx=0x7f657002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.335+0000 7f657b84c640 1 -- 172.21.15.19:0/248264360 shutdown_connections 2024-04-25T10:03:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.335+0000 7f657b84c640 1 --2- 172.21.15.19:0/248264360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657415ba80 0x7f65740cd390 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.335+0000 7f657b84c640 1 -- 172.21.15.19:0/248264360 >> 172.21.15.19:0/248264360 conn(0x7f657406abe0 msgr2=0x7f657406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.335+0000 7f657b84c640 1 -- 172.21.15.19:0/248264360 shutdown_connections 2024-04-25T10:03:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.335+0000 7f657b84c640 1 -- 172.21.15.19:0/248264360 wait complete. 2024-04-25T10:03:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.336+0000 7f657b84c640 1 Processor -- start 2024-04-25T10:03:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.336+0000 7f657b84c640 1 -- start start 2024-04-25T10:03:09.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.337+0000 7f657b84c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65740cf250 0x7f657414d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.337+0000 7f657b84c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f657415ba80 0x7f657414e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.337+0000 7f657b84c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657415c990 0x7f657414e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.337+0000 7f657b84c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657406fb00 con 0x7f657415c990 2024-04-25T10:03:09.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.337+0000 7f657b84c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65740cce70 con 0x7f65740cf250 2024-04-25T10:03:09.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.337+0000 7f657b84c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657406b850 con 0x7f657415ba80 2024-04-25T10:03:09.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.337+0000 7f6579de9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657415c990 0x7f657414e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.337+0000 7f6579de9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657415c990 0x7f657414e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39474/0 (socket says 172.21.15.19:39474) 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.337+0000 7f6579de9640 1 -- 172.21.15.19:0/2038642980 learned_addr learned my addr 172.21.15.19:0/2038642980 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.337+0000 7f6578de7640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f657415ba80 0x7f657414e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.337+0000 7f65795e8640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65740cf250 0x7f657414d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f6578de7640 1 -- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65740cf250 msgr2=0x7f657414d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f6578de7640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65740cf250 0x7f657414d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f6578de7640 1 -- 172.21.15.19:0/2038642980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657415c990 msgr2=0x7f657414e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f6578de7640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657415c990 0x7f657414e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f6578de7640 1 -- 172.21.15.19:0/2038642980 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657414db30 con 0x7f657415ba80 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f6579de9640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657415c990 0x7f657414e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f6578de7640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f657415ba80 0x7f657414e3b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f656400c3c0 tx=0x7f656400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f6569ffb640 1 -- 172.21.15.19:0/2038642980 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564017070 con 0x7f657415ba80 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f6569ffb640 1 -- 172.21.15.19:0/2038642980 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65640052d0 con 0x7f657415ba80 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f6569ffb640 1 -- 172.21.15.19:0/2038642980 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564003830 con 0x7f657415ba80 2024-04-25T10:03:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f657b84c640 1 -- 172.21.15.19:0/2038642980 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6574151770 con 0x7f657415ba80 2024-04-25T10:03:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.338+0000 7f657b84c640 1 -- 172.21.15.19:0/2038642980 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6574151d30 con 0x7f657415ba80 2024-04-25T10:03:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.339+0000 7f657b84c640 1 -- 172.21.15.19:0/2038642980 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6574073a00 con 0x7f657415ba80 2024-04-25T10:03:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.340+0000 7f6569ffb640 1 -- 172.21.15.19:0/2038642980 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f656401a070 con 0x7f657415ba80 2024-04-25T10:03:09.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.340+0000 7f6569ffb640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f655806b4d0 0x7f655806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.340+0000 7f6569ffb640 1 -- 172.21.15.19:0/2038642980 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6564089390 con 0x7f657415ba80 2024-04-25T10:03:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.340+0000 7f65795e8640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f655806b4d0 0x7f655806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.341+0000 7f65795e8640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f655806b4d0 0x7f655806d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6570034000 tx=0x7f657002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.345+0000 7f6569ffb640 1 -- 172.21.15.19:0/2038642980 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f657414db30 con 0x7f657415ba80 2024-04-25T10:03:09.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:09.536+0000 7f657b84c640 1 -- 172.21.15.19:0/2038642980 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f657406c6c0 con 0x7f657415ba80 2024-04-25T10:03:10.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.651+0000 7f6569ffb640 1 -- 172.21.15.19:0/2038642980 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v150) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6564057c90 con 0x7f657415ba80 2024-04-25T10:03:10.656 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T10:03:10.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 -- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f655806b4d0 msgr2=0x7f655806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f655806b4d0 0x7f655806d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6570034000 tx=0x7f657002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:10.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 -- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f657415ba80 msgr2=0x7f657414e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f657415ba80 0x7f657414e3b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f656400c3c0 tx=0x7f656400c880 comp rx=0 tx=0).stop 2024-04-25T10:03:10.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 -- 172.21.15.19:0/2038642980 shutdown_connections 2024-04-25T10:03:10.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f655806b4d0 0x7f655806d990 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f657415c990 0x7f657414e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f657415ba80 0x7f657414e3b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 --2- 172.21.15.19:0/2038642980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65740cf250 0x7f657414d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 -- 172.21.15.19:0/2038642980 >> 172.21.15.19:0/2038642980 conn(0x7f657406abe0 msgr2=0x7f65740b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 -- 172.21.15.19:0/2038642980 shutdown_connections 2024-04-25T10:03:10.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.654+0000 7f657b84c640 1 -- 172.21.15.19:0/2038642980 wait complete. 2024-04-25T10:03:10.673 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:03:10.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.775+0000 7f6ed0a12640 1 Processor -- start 2024-04-25T10:03:10.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.775+0000 7f6ed0a12640 1 -- start start 2024-04-25T10:03:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.776+0000 7f6ed0a12640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ecc0b6cb0 0x7f6ecc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.776+0000 7f6ed0a12640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ecc0ceee0 con 0x7f6ecc15db70 2024-04-25T10:03:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.776+0000 7f6ed0a12640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ecc0d1d10 con 0x7f6ecc0b5a60 2024-04-25T10:03:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.776+0000 7f6ed0a12640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ecc0d1ff0 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.776+0000 7f6ec9d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ecc0b6cb0 0x7f6ecc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.776+0000 7f6ec9d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ecc0b6cb0 0x7f6ecc0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39488/0 (socket says 172.21.15.19:39488) 2024-04-25T10:03:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.776+0000 7f6eca59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6ecc15db70 0x7f6ecc162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45172/0 (socket says 172.21.15.19:45172) 2024-04-25T10:03:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.776+0000 7f6ec9d9b640 1 -- 172.21.15.19:0/2994696849 learned_addr learned my addr 172.21.15.19:0/2994696849 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.777+0000 7f6ec9d9b640 1 -- 172.21.15.19:0/2994696849 >> v1:172.21.15.112:6789/0 conn(0x7f6ecc0b5a60 legacy=0x7f6ecc0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.777+0000 7f6ec9d9b640 1 -- 172.21.15.19:0/2994696849 >> v1:172.21.15.19:6789/0 conn(0x7f6ecc15db70 legacy=0x7f6ecc162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.777+0000 7f6ec9d9b640 1 -- 172.21.15.19:0/2994696849 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc0d22d0 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.777+0000 7f6ec959a640 1 -- 172.21.15.19:0/2994696849 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114243037 0 0) 0x7f6ecc0ceee0 con 0x7f6ecc15db70 2024-04-25T10:03:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.777+0000 7f6ec9d9b640 1 --2- 172.21.15.19:0/2994696849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc0b6cb0 0x7f6ecc0b5520 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009870 tx=0x7f6eb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab7e2c09231d344 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.777+0000 7f6ec959a640 1 -- 172.21.15.19:0/2994696849 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4037070 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.777+0000 7f6ec959a640 1 -- 172.21.15.19:0/2994696849 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4032440 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.778+0000 7f6ec959a640 1 -- 172.21.15.19:0/2994696849 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4032810 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.778+0000 7f6ed0a12640 1 -- 172.21.15.19:0/2994696849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc0b6cb0 msgr2=0x7f6ecc0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.778+0000 7f6ed0a12640 1 --2- 172.21.15.19:0/2994696849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc0b6cb0 0x7f6ecc0b5520 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009870 tx=0x7f6eb402a270 comp rx=0 tx=0).stop 2024-04-25T10:03:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.778+0000 7f6ed0a12640 1 -- 172.21.15.19:0/2994696849 shutdown_connections 2024-04-25T10:03:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.778+0000 7f6ed0a12640 1 --2- 172.21.15.19:0/2994696849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc0b6cb0 0x7f6ecc0b5520 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.778+0000 7f6ed0a12640 1 -- 172.21.15.19:0/2994696849 >> 172.21.15.19:0/2994696849 conn(0x7f6ecc0d0520 msgr2=0x7f6ecc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.778+0000 7f6ed0a12640 1 -- 172.21.15.19:0/2994696849 shutdown_connections 2024-04-25T10:03:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.778+0000 7f6ed0a12640 1 -- 172.21.15.19:0/2994696849 wait complete. 2024-04-25T10:03:10.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.779+0000 7f6ed0a12640 1 Processor -- start 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.780+0000 7f6ed0a12640 1 -- start start 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.780+0000 7f6ed0a12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc0b5a60 0x7f6ecc0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.780+0000 7f6ed0a12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc0b6cb0 0x7f6ecc156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.780+0000 7f6ed0a12640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ecc15db70 0x7f6ecc0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.780+0000 7f6ed0a12640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0cf1c0 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.780+0000 7f6ed0a12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0ce530 con 0x7f6ecc0b5a60 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.780+0000 7f6ed0a12640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0cf340 con 0x7f6ecc15db70 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.780+0000 7f6ec9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc0b6cb0 0x7f6ecc156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.780+0000 7f6ec9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc0b6cb0 0x7f6ecc156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39494/0 (socket says 172.21.15.19:39494) 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.780+0000 7f6ec9d9b640 1 -- 172.21.15.19:0/377653287 learned_addr learned my addr 172.21.15.19:0/377653287 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6eca59c640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc0b5a60 0x7f6ecc0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ec9d9b640 1 -- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ecc15db70 msgr2=0x7f6ecc0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ecad9d640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ecc15db70 0x7f6ecc0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ec9d9b640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ecc15db70 0x7f6ecc0cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ec9d9b640 1 -- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc0b5a60 msgr2=0x7f6ecc0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ec9d9b640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc0b5a60 0x7f6ecc0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ec9d9b640 1 -- 172.21.15.19:0/377653287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc1574b0 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6eca59c640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc0b5a60 0x7f6ecc0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ec9d9b640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc0b6cb0 0x7f6ecc156f70 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009870 tx=0x7f6eb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ec2ffd640 1 -- 172.21.15.19:0/377653287 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4041020 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ec2ffd640 1 -- 172.21.15.19:0/377653287 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4030970 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ed0a12640 1 -- 172.21.15.19:0/377653287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc0cbb60 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ec2ffd640 1 -- 172.21.15.19:0/377653287 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4037040 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.781+0000 7f6ed0a12640 1 -- 172.21.15.19:0/377653287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc0cc400 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.782+0000 7f6ed0a12640 1 -- 172.21.15.19:0/377653287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc073990 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.783+0000 7f6ec2ffd640 1 -- 172.21.15.19:0/377653287 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6eb4044020 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.783+0000 7f6ec2ffd640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eac06b4d0 0x7f6eac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.783+0000 7f6ec2ffd640 1 -- 172.21.15.19:0/377653287 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6eb40aa9f0 con 0x7f6ecc0b6cb0 2024-04-25T10:03:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.783+0000 7f6eca59c640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eac06b4d0 0x7f6eac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.784+0000 7f6eca59c640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eac06b4d0 0x7f6eac06d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4000f60 tx=0x7f6ec4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:10.787+0000 7f6ec2ffd640 1 -- 172.21.15.19:0/377653287 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6eb404b030 con 0x7f6ecc0b6cb0 2024-04-25T10:03:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.034+0000 7f6ed0a12640 1 -- 172.21.15.19:0/377653287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6ecc0ce310 con 0x7f6ecc0b6cb0 2024-04-25T10:03:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.035+0000 7f6ec2ffd640 1 -- 172.21.15.19:0/377653287 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6ecc1574b0 con 0x7f6ecc0b6cb0 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.040 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T10:03:11.041 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:03:11.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:11.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.045 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:03:11.053 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:03:11.053 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:11.053 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:03:11.053 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.053 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.053 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T10:03:11.053 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:03:11.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.036+0000 7f6ed0a12640 1 -- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eac06b4d0 msgr2=0x7f6eac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.036+0000 7f6ed0a12640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eac06b4d0 0x7f6eac06d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4000f60 tx=0x7f6ec4009040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.036+0000 7f6ed0a12640 1 -- 172.21.15.19:0/377653287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc0b6cb0 msgr2=0x7f6ecc156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.036+0000 7f6ed0a12640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc0b6cb0 0x7f6ecc156f70 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009870 tx=0x7f6eb40046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.036+0000 7f6ed0a12640 1 -- 172.21.15.19:0/377653287 shutdown_connections 2024-04-25T10:03:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.036+0000 7f6ed0a12640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6eac06b4d0 0x7f6eac06d990 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.036+0000 7f6ed0a12640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6ecc15db70 0x7f6ecc0cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.036+0000 7f6ed0a12640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc0b6cb0 0x7f6ecc156f70 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.036+0000 7f6ed0a12640 1 --2- 172.21.15.19:0/377653287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ecc0b5a60 0x7f6ecc0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.037+0000 7f6ed0a12640 1 -- 172.21.15.19:0/377653287 >> 172.21.15.19:0/377653287 conn(0x7f6ecc0d0520 msgr2=0x7f6ecc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.037+0000 7f6ed0a12640 1 -- 172.21.15.19:0/377653287 shutdown_connections 2024-04-25T10:03:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.037+0000 7f6ed0a12640 1 -- 172.21.15.19:0/377653287 wait complete. 2024-04-25T10:03:11.056 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:11.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.152+0000 7f26ef05d640 1 Processor -- start 2024-04-25T10:03:11.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.152+0000 7f26ef05d640 1 -- start start 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26ef05d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f26e80b2cb0 0x7f26e80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26ef05d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f26e80d1020 0x7f26e80d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26ef05d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e80b97a0 con 0x7f26e815dbe0 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26ef05d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e815dfc0 con 0x7f26e80b2cb0 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26ef05d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e80d1a00 con 0x7f26e80d1020 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26e7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f26e80b2cb0 0x7f26e80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26ecdf9640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f26e815dbe0 0x7f26e80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45182/0 (socket says 172.21.15.19:45182) 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26e7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f26e80b2cb0 0x7f26e80d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39506/0 (socket says 172.21.15.19:39506) 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26ecdf9640 1 -- 172.21.15.19:0/4222593273 learned_addr learned my addr 172.21.15.19:0/4222593273 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26ed5fa640 1 --2- 172.21.15.19:0/4222593273 >> v2:172.21.15.112:3300/0 conn(0x7f26e80d1020 0x7f26e80d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26e7fff640 1 -- 172.21.15.19:0/4222593273 >> v2:172.21.15.112:3300/0 conn(0x7f26e80d1020 msgr2=0x7f26e80d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26e7fff640 1 --2- 172.21.15.19:0/4222593273 >> v2:172.21.15.112:3300/0 conn(0x7f26e80d1020 0x7f26e80d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26e7fff640 1 -- 172.21.15.19:0/4222593273 >> v1:172.21.15.19:6789/0 conn(0x7f26e815dbe0 legacy=0x7f26e80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.153+0000 7f26e7fff640 1 -- 172.21.15.19:0/4222593273 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e80d1c50 con 0x7f26e80b2cb0 2024-04-25T10:03:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.154+0000 7f26e7fff640 1 --2- 172.21.15.19:0/4222593273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e80b2cb0 0x7f26e80d2730 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f26e80cf570 tx=0x7f26dc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66545449bb83aaf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.154+0000 7f26e77fe640 1 -- 172.21.15.19:0/4222593273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc037070 con 0x7f26e80b2cb0 2024-04-25T10:03:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.154+0000 7f26e77fe640 1 -- 172.21.15.19:0/4222593273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc032440 con 0x7f26e80b2cb0 2024-04-25T10:03:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.154+0000 7f26e77fe640 1 -- 172.21.15.19:0/4222593273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc032810 con 0x7f26e80b2cb0 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.155+0000 7f26ef05d640 1 -- 172.21.15.19:0/4222593273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e80b2cb0 msgr2=0x7f26e80d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.155+0000 7f26ef05d640 1 --2- 172.21.15.19:0/4222593273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e80b2cb0 0x7f26e80d2730 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f26e80cf570 tx=0x7f26dc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.155+0000 7f26ef05d640 1 -- 172.21.15.19:0/4222593273 shutdown_connections 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.155+0000 7f26ef05d640 1 --2- 172.21.15.19:0/4222593273 >> v2:172.21.15.112:3300/0 conn(0x7f26e80d1020 0x7f26e80d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.155+0000 7f26ef05d640 1 --2- 172.21.15.19:0/4222593273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e80b2cb0 0x7f26e80d2730 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.155+0000 7f26ef05d640 1 -- 172.21.15.19:0/4222593273 >> 172.21.15.19:0/4222593273 conn(0x7f26e8061a20 msgr2=0x7f26e8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.155+0000 7f26ef05d640 1 -- 172.21.15.19:0/4222593273 shutdown_connections 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.155+0000 7f26ef05d640 1 -- 172.21.15.19:0/4222593273 wait complete. 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.156+0000 7f26ef05d640 1 Processor -- start 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.156+0000 7f26ef05d640 1 -- start start 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ef05d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e80b2cb0 0x7f26e8152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ef05d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26e80d1020 0x7f26e80cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ef05d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e815dbe0 0x7f26e80cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ef05d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80b92a0 con 0x7f26e80b2cb0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ef05d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80d2210 con 0x7f26e815dbe0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ef05d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80b9420 con 0x7f26e80d1020 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ecdf9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e80b2cb0 0x7f26e8152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ecdf9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e80b2cb0 0x7f26e8152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39512/0 (socket says 172.21.15.19:39512) 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ecdf9640 1 -- 172.21.15.19:0/3122569537 learned_addr learned my addr 172.21.15.19:0/3122569537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ed5fa640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e815dbe0 0x7f26e80cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26e7fff640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26e80d1020 0x7f26e80cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ecdf9640 1 -- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26e80d1020 msgr2=0x7f26e80cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ecdf9640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26e80d1020 0x7f26e80cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ecdf9640 1 -- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e815dbe0 msgr2=0x7f26e80cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ecdf9640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e815dbe0 0x7f26e80cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ecdf9640 1 -- 172.21.15.19:0/3122569537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e80cb150 con 0x7f26e80b2cb0 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26e7fff640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26e80d1020 0x7f26e80cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.157+0000 7f26ed5fa640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e815dbe0 0x7f26e80cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.158+0000 7f26ecdf9640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e80b2cb0 0x7f26e8152e50 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c3c0 tx=0x7f26d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.158+0000 7f26e57fa640 1 -- 172.21.15.19:0/3122569537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8017070 con 0x7f26e80b2cb0 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.158+0000 7f26e57fa640 1 -- 172.21.15.19:0/3122569537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d8005ce0 con 0x7f26e80b2cb0 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.158+0000 7f26ef05d640 1 -- 172.21.15.19:0/3122569537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e81534c0 con 0x7f26e80b2cb0 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.158+0000 7f26e57fa640 1 -- 172.21.15.19:0/3122569537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80072f0 con 0x7f26e80b2cb0 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.158+0000 7f26ef05d640 1 -- 172.21.15.19:0/3122569537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e8153a80 con 0x7f26e80b2cb0 2024-04-25T10:03:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.159+0000 7f26ef05d640 1 -- 172.21.15.19:0/3122569537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e8073990 con 0x7f26e80b2cb0 2024-04-25T10:03:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.159+0000 7f26e57fa640 1 -- 172.21.15.19:0/3122569537 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26d801a070 con 0x7f26e80b2cb0 2024-04-25T10:03:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.160+0000 7f26e57fa640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26d406b4d0 0x7f26d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.160+0000 7f26e57fa640 1 -- 172.21.15.19:0/3122569537 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f26d8089360 con 0x7f26e80b2cb0 2024-04-25T10:03:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.160+0000 7f26e7fff640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26d406b4d0 0x7f26d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.160+0000 7f26e7fff640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26d406b4d0 0x7f26d406d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f26dc034000 tx=0x7f26dc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.164+0000 7f26e57fa640 1 -- 172.21.15.19:0/3122569537 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26d8058430 con 0x7f26e80b2cb0 2024-04-25T10:03:11.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.345+0000 7f26ef05d640 1 -- 172.21.15.19:0/3122569537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26e80c9ad0 con 0x7f26e80b2cb0 2024-04-25T10:03:11.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.346+0000 7f26e57fa640 1 -- 172.21.15.19:0/3122569537 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+13162 (secure 0 0 0) 0x7f26d8057d30 con 0x7f26e80b2cb0 2024-04-25T10:03:11.351 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:11.351 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":150,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:03:10.645962+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:11.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 -- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26d406b4d0 msgr2=0x7f26d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26d406b4d0 0x7f26d406d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f26dc034000 tx=0x7f26dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:11.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 -- 172.21.15.19:0/3122569537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e80b2cb0 msgr2=0x7f26e8152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e80b2cb0 0x7f26e8152e50 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c3c0 tx=0x7f26d800c880 comp rx=0 tx=0).stop 2024-04-25T10:03:11.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 -- 172.21.15.19:0/3122569537 shutdown_connections 2024-04-25T10:03:11.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26d406b4d0 0x7f26d406d990 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e815dbe0 0x7f26e80cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26e80d1020 0x7f26e80cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 --2- 172.21.15.19:0/3122569537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e80b2cb0 0x7f26e8152e50 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 -- 172.21.15.19:0/3122569537 >> 172.21.15.19:0/3122569537 conn(0x7f26e8061a20 msgr2=0x7f26e8159600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 -- 172.21.15.19:0/3122569537 shutdown_connections 2024-04-25T10:03:11.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.348+0000 7f26ef05d640 1 -- 172.21.15.19:0/3122569537 wait complete. 2024-04-25T10:03:11.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:11.364 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:11.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.463+0000 7f7005046640 1 Processor -- start 2024-04-25T10:03:11.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.463+0000 7f7005046640 1 -- start start 2024-04-25T10:03:11.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.464+0000 7f7005046640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7000161710 0x7f70000d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.464+0000 7f7005046640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700015e000 con 0x7f700015dc20 2024-04-25T10:03:11.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.464+0000 7f7005046640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70000d1de0 con 0x7f70000ced80 2024-04-25T10:03:11.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.464+0000 7f7005046640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7000161c60 con 0x7f7000161710 2024-04-25T10:03:11.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.464+0000 7f6fff59e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7000161710 0x7f70000d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.464+0000 7f6ffed9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f700015dc20 0x7f70000ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45186/0 (socket says 172.21.15.19:45186) 2024-04-25T10:03:11.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.464+0000 7f6ffed9d640 1 -- 172.21.15.19:0/1307595199 learned_addr learned my addr 172.21.15.19:0/1307595199 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.464+0000 7f6fff59e640 1 -- 172.21.15.19:0/1307595199 >> v1:172.21.15.112:6789/0 conn(0x7f70000ced80 legacy=0x7f70000d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.464+0000 7f6fff59e640 1 -- 172.21.15.19:0/1307595199 >> v1:172.21.15.19:6789/0 conn(0x7f700015dc20 legacy=0x7f70000ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.464+0000 7f6fff59e640 1 -- 172.21.15.19:0/1307595199 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000161f40 con 0x7f7000161710 2024-04-25T10:03:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.465+0000 7f6fff59e640 1 --2- 172.21.15.19:0/1307595199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7000161710 0x7f70000d29f0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009d90 tx=0x7f6ff802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f92868b1b34a8dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.465+0000 7f6ffdd9b640 1 -- 172.21.15.19:0/1307595199 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8007740 con 0x7f7000161710 2024-04-25T10:03:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.465+0000 7f6ffdd9b640 1 -- 172.21.15.19:0/1307595199 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff80078e0 con 0x7f7000161710 2024-04-25T10:03:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.465+0000 7f6ffdd9b640 1 -- 172.21.15.19:0/1307595199 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8007cb0 con 0x7f7000161710 2024-04-25T10:03:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.466+0000 7f7005046640 1 -- 172.21.15.19:0/1307595199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7000161710 msgr2=0x7f70000d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.466+0000 7f7005046640 1 --2- 172.21.15.19:0/1307595199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7000161710 0x7f70000d29f0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009d90 tx=0x7f6ff802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.466+0000 7f7005046640 1 -- 172.21.15.19:0/1307595199 shutdown_connections 2024-04-25T10:03:11.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.466+0000 7f7005046640 1 --2- 172.21.15.19:0/1307595199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7000161710 0x7f70000d29f0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.466+0000 7f7005046640 1 -- 172.21.15.19:0/1307595199 >> 172.21.15.19:0/1307595199 conn(0x7f70000d0420 msgr2=0x7f70000d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.466+0000 7f7005046640 1 -- 172.21.15.19:0/1307595199 shutdown_connections 2024-04-25T10:03:11.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.466+0000 7f7005046640 1 -- 172.21.15.19:0/1307595199 wait complete. 2024-04-25T10:03:11.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.467+0000 7f7005046640 1 Processor -- start 2024-04-25T10:03:11.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.467+0000 7f7005046640 1 -- start start 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f7005046640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70000ced80 0x7f7000149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f7005046640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f700015dc20 0x7f70001e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f7005046640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7000161710 0x7f70001e9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f7005046640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70000b5e50 con 0x7f70000ced80 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f7005046640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7000162500 con 0x7f700015dc20 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f7005046640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70000b5fd0 con 0x7f7000161710 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f6ffed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70000ced80 0x7f7000149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f6ffed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70000ced80 0x7f7000149ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39528/0 (socket says 172.21.15.19:39528) 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f6ffed9d640 1 -- 172.21.15.19:0/4049012279 learned_addr learned my addr 172.21.15.19:0/4049012279 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f6ffe59c640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f700015dc20 0x7f70001e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f6fff59e640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7000161710 0x7f70001e9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f6ffed9d640 1 -- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7000161710 msgr2=0x7f70001e9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f6ffed9d640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7000161710 0x7f70001e9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f6ffed9d640 1 -- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f700015dc20 msgr2=0x7f70001e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f6ffed9d640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f700015dc20 0x7f70001e96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f6ffed9d640 1 -- 172.21.15.19:0/4049012279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70000cc8e0 con 0x7f70000ced80 2024-04-25T10:03:11.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.468+0000 7f6ffe59c640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f700015dc20 0x7f70001e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:11.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.469+0000 7f6ffed9d640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70000ced80 0x7f7000149ce0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000c400 tx=0x7f6ff000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.469+0000 7f6fe77fe640 1 -- 172.21.15.19:0/4049012279 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0017070 con 0x7f70000ced80 2024-04-25T10:03:11.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.469+0000 7f6fe77fe640 1 -- 172.21.15.19:0/4049012279 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff0005ce0 con 0x7f70000ced80 2024-04-25T10:03:11.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.469+0000 7f7005046640 1 -- 172.21.15.19:0/4049012279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700014caa0 con 0x7f70000ced80 2024-04-25T10:03:11.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.469+0000 7f6fe77fe640 1 -- 172.21.15.19:0/4049012279 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00072f0 con 0x7f70000ced80 2024-04-25T10:03:11.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.469+0000 7f7005046640 1 -- 172.21.15.19:0/4049012279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70000cce40 con 0x7f70000ced80 2024-04-25T10:03:11.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.470+0000 7f7005046640 1 -- 172.21.15.19:0/4049012279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7000073a00 con 0x7f70000ced80 2024-04-25T10:03:11.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.470+0000 7f6fe77fe640 1 -- 172.21.15.19:0/4049012279 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ff001a070 con 0x7f70000ced80 2024-04-25T10:03:11.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.471+0000 7f6fe77fe640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fec06b4d0 0x7f6fec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.471+0000 7f6fe77fe640 1 -- 172.21.15.19:0/4049012279 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6ff00893f0 con 0x7f70000ced80 2024-04-25T10:03:11.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.471+0000 7f6ffe59c640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fec06b4d0 0x7f6fec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.471+0000 7f6ffe59c640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fec06b4d0 0x7f6fec06d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f70000ca4b0 tx=0x7f6ff4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.475+0000 7f6fe77fe640 1 -- 172.21.15.19:0/4049012279 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70000cc8e0 con 0x7f70000ced80 2024-04-25T10:03:11.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.654+0000 7f7005046640 1 -- 172.21.15.19:0/4049012279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70000d16b0 con 0x7f70000ced80 2024-04-25T10:03:11.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.654+0000 7f6fe77fe640 1 -- 172.21.15.19:0/4049012279 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ff00584c0 con 0x7f70000ced80 2024-04-25T10:03:11.659 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:03:11.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 -- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fec06b4d0 msgr2=0x7f6fec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fec06b4d0 0x7f6fec06d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f70000ca4b0 tx=0x7f6ff4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 -- 172.21.15.19:0/4049012279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70000ced80 msgr2=0x7f7000149ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70000ced80 0x7f7000149ce0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000c400 tx=0x7f6ff000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 -- 172.21.15.19:0/4049012279 shutdown_connections 2024-04-25T10:03:11.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6fec06b4d0 0x7f6fec06d990 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7000161710 0x7f70001e9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f700015dc20 0x7f70001e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 --2- 172.21.15.19:0/4049012279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70000ced80 0x7f7000149ce0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 -- 172.21.15.19:0/4049012279 >> 172.21.15.19:0/4049012279 conn(0x7f70000d0420 msgr2=0x7f70000d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 -- 172.21.15.19:0/4049012279 shutdown_connections 2024-04-25T10:03:11.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.656+0000 7f7005046640 1 -- 172.21.15.19:0/4049012279 wait complete. 2024-04-25T10:03:11.672 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:03:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.768+0000 7f45b8bf9640 1 Processor -- start 2024-04-25T10:03:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.768+0000 7f45b8bf9640 1 -- start start 2024-04-25T10:03:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b8bf9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f45b415dd30 0x7f45b40b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b8bf9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45b40d0ad0 0x7f45b40b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b8bf9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b40d1df0 con 0x7f45b40b9770 2024-04-25T10:03:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b8bf9640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b40d20d0 con 0x7f45b40d0ad0 2024-04-25T10:03:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b8bf9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b40d23b0 con 0x7f45b415dd30 2024-04-25T10:03:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b2d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45b40d0ad0 0x7f45b40b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b2d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45b40d0ad0 0x7f45b40b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39530/0 (socket says 172.21.15.19:39530) 2024-04-25T10:03:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b2d9d640 1 -- 172.21.15.19:0/3484169586 learned_addr learned my addr 172.21.15.19:0/3484169586 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b259c640 1 --2- 172.21.15.19:0/3484169586 >> v2:172.21.15.112:3300/0 conn(0x7f45b415dd30 0x7f45b40b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b2d9d640 1 -- 172.21.15.19:0/3484169586 >> v2:172.21.15.112:3300/0 conn(0x7f45b415dd30 msgr2=0x7f45b40b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b2d9d640 1 --2- 172.21.15.19:0/3484169586 >> v2:172.21.15.112:3300/0 conn(0x7f45b415dd30 0x7f45b40b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b2d9d640 1 -- 172.21.15.19:0/3484169586 >> v1:172.21.15.112:6789/0 conn(0x7f45b40b9770 legacy=0x7f45b40d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b2d9d640 1 -- 172.21.15.19:0/3484169586 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b40d2690 con 0x7f45b40d0ad0 2024-04-25T10:03:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.769+0000 7f45b259c640 1 --2- 172.21.15.19:0/3484169586 >> v2:172.21.15.112:3300/0 conn(0x7f45b415dd30 0x7f45b40b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.770+0000 7f45b2d9d640 1 --2- 172.21.15.19:0/3484169586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b40d0ad0 0x7f45b40b6a40 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f45b406d000 tx=0x7f45ac02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d258eee44ffcc67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.770+0000 7f45b1d9b640 1 -- 172.21.15.19:0/3484169586 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac036070 con 0x7f45b40d0ad0 2024-04-25T10:03:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.770+0000 7f45b1d9b640 1 -- 172.21.15.19:0/3484169586 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac031600 con 0x7f45b40d0ad0 2024-04-25T10:03:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.770+0000 7f45b1d9b640 1 -- 172.21.15.19:0/3484169586 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0319d0 con 0x7f45b40d0ad0 2024-04-25T10:03:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.770+0000 7f45b8bf9640 1 -- 172.21.15.19:0/3484169586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b40d0ad0 msgr2=0x7f45b40b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.770+0000 7f45b8bf9640 1 --2- 172.21.15.19:0/3484169586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b40d0ad0 0x7f45b40b6a40 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f45b406d000 tx=0x7f45ac02a610 comp rx=0 tx=0).stop 2024-04-25T10:03:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.770+0000 7f45b8bf9640 1 -- 172.21.15.19:0/3484169586 shutdown_connections 2024-04-25T10:03:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.771+0000 7f45b8bf9640 1 --2- 172.21.15.19:0/3484169586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b40d0ad0 0x7f45b40b6a40 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.771+0000 7f45b8bf9640 1 --2- 172.21.15.19:0/3484169586 >> v2:172.21.15.112:3300/0 conn(0x7f45b415dd30 0x7f45b40b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.771+0000 7f45b8bf9640 1 -- 172.21.15.19:0/3484169586 >> 172.21.15.19:0/3484169586 conn(0x7f45b4061c30 msgr2=0x7f45b406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.771+0000 7f45b8bf9640 1 -- 172.21.15.19:0/3484169586 shutdown_connections 2024-04-25T10:03:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.771+0000 7f45b8bf9640 1 -- 172.21.15.19:0/3484169586 wait complete. 2024-04-25T10:03:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.772+0000 7f45b8bf9640 1 Processor -- start 2024-04-25T10:03:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.772+0000 7f45b8bf9640 1 -- start start 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b8bf9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45b40b9770 0x7f45b40c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b8bf9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45b40d0ad0 0x7f45b40b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b8bf9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b415dd30 0x7f45b415ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b8bf9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40d1b90 con 0x7f45b415dd30 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b8bf9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40d1a10 con 0x7f45b40d0ad0 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b8bf9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40d2e90 con 0x7f45b40b9770 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b415dd30 0x7f45b415ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b415dd30 0x7f45b415ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39534/0 (socket says 172.21.15.19:39534) 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b2d9d640 1 -- 172.21.15.19:0/2315059523 learned_addr learned my addr 172.21.15.19:0/2315059523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45abfff640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45b40d0ad0 0x7f45b40b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b259c640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45b40b9770 0x7f45b40c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b2d9d640 1 -- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45b40b9770 msgr2=0x7f45b40c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b2d9d640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45b40b9770 0x7f45b40c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b2d9d640 1 -- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45b40d0ad0 msgr2=0x7f45b40b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b2d9d640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45b40d0ad0 0x7f45b40b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b2d9d640 1 -- 172.21.15.19:0/2315059523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b414ad70 con 0x7f45b415dd30 2024-04-25T10:03:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45abfff640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45b40d0ad0 0x7f45b40b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b2d9d640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b415dd30 0x7f45b415ff80 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009f40 tx=0x7f45ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45aaffd640 1 -- 172.21.15.19:0/2315059523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac036050 con 0x7f45b415dd30 2024-04-25T10:03:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45aaffd640 1 -- 172.21.15.19:0/2315059523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac042070 con 0x7f45b415dd30 2024-04-25T10:03:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b8bf9640 1 -- 172.21.15.19:0/2315059523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b40b5860 con 0x7f45b415dd30 2024-04-25T10:03:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45aaffd640 1 -- 172.21.15.19:0/2315059523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac007bd0 con 0x7f45b415dd30 2024-04-25T10:03:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.773+0000 7f45b8bf9640 1 -- 172.21.15.19:0/2315059523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b40b5e20 con 0x7f45b415dd30 2024-04-25T10:03:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.774+0000 7f45b8bf9640 1 -- 172.21.15.19:0/2315059523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b4073990 con 0x7f45b415dd30 2024-04-25T10:03:11.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.775+0000 7f45aaffd640 1 -- 172.21.15.19:0/2315059523 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45ac047460 con 0x7f45b415dd30 2024-04-25T10:03:11.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.775+0000 7f45aaffd640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f459406b4d0 0x7f459406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.775+0000 7f45aaffd640 1 -- 172.21.15.19:0/2315059523 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f45ac0ab010 con 0x7f45b415dd30 2024-04-25T10:03:11.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.775+0000 7f45b259c640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f459406b4d0 0x7f459406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.776+0000 7f45b259c640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f459406b4d0 0x7f459406d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f45a4006fd0 tx=0x7f45a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.779+0000 7f45aaffd640 1 -- 172.21.15.19:0/2315059523 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45ac07a040 con 0x7f45b415dd30 2024-04-25T10:03:11.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.966+0000 7f45b8bf9640 1 -- 172.21.15.19:0/2315059523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f45b415e1b0 con 0x7f45b415dd30 2024-04-25T10:03:11.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.966+0000 7f45aaffd640 1 -- 172.21.15.19:0/2315059523 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v150) v1 ==== 84+0+65 (secure 0 0 0) 0x7f45ac079940 con 0x7f45b415dd30 2024-04-25T10:03:11.971 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:11.971 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T10:03:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:03:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T10:03:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T10:03:11.972 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:11.972 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T10:03:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.968+0000 7f45b8bf9640 1 -- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f459406b4d0 msgr2=0x7f459406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.968+0000 7f45b8bf9640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f459406b4d0 0x7f459406d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f45a4006fd0 tx=0x7f45a4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.968+0000 7f45b8bf9640 1 -- 172.21.15.19:0/2315059523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b415dd30 msgr2=0x7f45b415ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.968+0000 7f45b8bf9640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b415dd30 0x7f45b415ff80 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009f40 tx=0x7f45ac004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.969+0000 7f45b8bf9640 1 -- 172.21.15.19:0/2315059523 shutdown_connections 2024-04-25T10:03:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.969+0000 7f45b8bf9640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f459406b4d0 0x7f459406d990 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.969+0000 7f45b8bf9640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b415dd30 0x7f45b415ff80 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.969+0000 7f45b8bf9640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45b40d0ad0 0x7f45b40b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.969+0000 7f45b8bf9640 1 --2- 172.21.15.19:0/2315059523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45b40b9770 0x7f45b40c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.969+0000 7f45b8bf9640 1 -- 172.21.15.19:0/2315059523 >> 172.21.15.19:0/2315059523 conn(0x7f45b4061c30 msgr2=0x7f45b415cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.969+0000 7f45b8bf9640 1 -- 172.21.15.19:0/2315059523 shutdown_connections 2024-04-25T10:03:11.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:11.969+0000 7f45b8bf9640 1 -- 172.21.15.19:0/2315059523 wait complete. 2024-04-25T10:03:11.984 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:03:11.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:03:12.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.084+0000 7f062eaf5640 1 Processor -- start 2024-04-25T10:03:12.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.084+0000 7f062eaf5640 1 -- start start 2024-04-25T10:03:12.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.085+0000 7f062eaf5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f062815dbd0 0x7f06280caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.085+0000 7f062eaf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f062806ff30 0x7f06280cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.085+0000 7f062eaf5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f062806bbf0 0x7f062806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.085+0000 7f062eaf5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06280cc6c0 con 0x7f062815dbd0 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.085+0000 7f062eaf5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06280cb0e0 con 0x7f062806bbf0 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.085+0000 7f062eaf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06280cb3c0 con 0x7f062806ff30 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.085+0000 7f062c891640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f062815dbd0 0x7f06280caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.085+0000 7f062c891640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f062815dbd0 0x7f06280caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39538/0 (socket says 172.21.15.19:39538) 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.085+0000 7f062c891640 1 -- 172.21.15.19:0/3237931018 learned_addr learned my addr 172.21.15.19:0/3237931018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.086+0000 7f062c891640 1 -- 172.21.15.19:0/3237931018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f062806ff30 msgr2=0x7f06280cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.086+0000 7f062d092640 1 --2- 172.21.15.19:0/3237931018 >> v2:172.21.15.112:3300/0 conn(0x7f062806bbf0 0x7f062806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.086+0000 7f0627fff640 1 --2- 172.21.15.19:0/3237931018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f062806ff30 0x7f06280cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.086+0000 7f062c891640 1 --2- 172.21.15.19:0/3237931018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f062806ff30 0x7f06280cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.086+0000 7f062c891640 1 -- 172.21.15.19:0/3237931018 >> v2:172.21.15.112:3300/0 conn(0x7f062806bbf0 msgr2=0x7f062806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.086+0000 7f062c891640 1 --2- 172.21.15.19:0/3237931018 >> v2:172.21.15.112:3300/0 conn(0x7f062806bbf0 0x7f062806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.086+0000 7f062c891640 1 -- 172.21.15.19:0/3237931018 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06280cb6a0 con 0x7f062815dbd0 2024-04-25T10:03:12.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.086+0000 7f062c891640 1 --2- 172.21.15.19:0/3237931018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f062815dbd0 0x7f06280caa60 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f0620009e50 tx=0x7f062002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16d5e6780f1735 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f06277fe640 1 -- 172.21.15.19:0/3237931018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620036070 con 0x7f062815dbd0 2024-04-25T10:03:12.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f06277fe640 1 -- 172.21.15.19:0/3237931018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0620031600 con 0x7f062815dbd0 2024-04-25T10:03:12.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f06277fe640 1 -- 172.21.15.19:0/3237931018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06200319d0 con 0x7f062815dbd0 2024-04-25T10:03:12.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f062eaf5640 1 -- 172.21.15.19:0/3237931018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f062815dbd0 msgr2=0x7f06280caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f062eaf5640 1 --2- 172.21.15.19:0/3237931018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f062815dbd0 0x7f06280caa60 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f0620009e50 tx=0x7f062002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f062eaf5640 1 -- 172.21.15.19:0/3237931018 shutdown_connections 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f062eaf5640 1 --2- 172.21.15.19:0/3237931018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f062806ff30 0x7f06280cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f062eaf5640 1 --2- 172.21.15.19:0/3237931018 >> v2:172.21.15.112:3300/0 conn(0x7f062806bbf0 0x7f062806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f062eaf5640 1 --2- 172.21.15.19:0/3237931018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f062815dbd0 0x7f06280caa60 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f062eaf5640 1 -- 172.21.15.19:0/3237931018 >> 172.21.15.19:0/3237931018 conn(0x7f0628071af0 msgr2=0x7f0628071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f062eaf5640 1 -- 172.21.15.19:0/3237931018 shutdown_connections 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.087+0000 7f062eaf5640 1 -- 172.21.15.19:0/3237931018 wait complete. 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.088+0000 7f062eaf5640 1 Processor -- start 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.088+0000 7f062eaf5640 1 -- start start 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.089+0000 7f062eaf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f062806bbf0 0x7f06280b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.089+0000 7f062eaf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f062806ff30 0x7f06280b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.089+0000 7f062eaf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f062815dbd0 0x7f062814b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.089+0000 7f062eaf5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062815dfb0 con 0x7f062806bbf0 2024-04-25T10:03:12.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.089+0000 7f062eaf5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06280cbc60 con 0x7f062815dbd0 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.089+0000 7f062eaf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062815e130 con 0x7f062806ff30 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.089+0000 7f062c891640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f062806bbf0 0x7f06280b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.089+0000 7f062c891640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f062806bbf0 0x7f06280b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39540/0 (socket says 172.21.15.19:39540) 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.089+0000 7f062c891640 1 -- 172.21.15.19:0/3842972223 learned_addr learned my addr 172.21.15.19:0/3842972223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f062d092640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f062815dbd0 0x7f062814b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f0627fff640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f062806ff30 0x7f06280b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f062c891640 1 -- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f062806ff30 msgr2=0x7f06280b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f062c891640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f062806ff30 0x7f06280b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f062c891640 1 -- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f062815dbd0 msgr2=0x7f062814b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f062c891640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f062815dbd0 0x7f062814b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f062c891640 1 -- 172.21.15.19:0/3842972223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06280c8550 con 0x7f062806bbf0 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f0627fff640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f062806ff30 0x7f06280b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:12.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f062c891640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f062806bbf0 0x7f06280b2b90 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f0620009f70 tx=0x7f0620004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f06257fa640 1 -- 172.21.15.19:0/3842972223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062003f020 con 0x7f062806bbf0 2024-04-25T10:03:12.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f06257fa640 1 -- 172.21.15.19:0/3842972223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0620037040 con 0x7f062806bbf0 2024-04-25T10:03:12.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f062eaf5640 1 -- 172.21.15.19:0/3842972223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06280c8830 con 0x7f062806bbf0 2024-04-25T10:03:12.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f06257fa640 1 -- 172.21.15.19:0/3842972223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620036070 con 0x7f062806bbf0 2024-04-25T10:03:12.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.090+0000 7f062eaf5640 1 -- 172.21.15.19:0/3842972223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06280c8df0 con 0x7f062806bbf0 2024-04-25T10:03:12.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.091+0000 7f062eaf5640 1 -- 172.21.15.19:0/3842972223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0628073a00 con 0x7f062806bbf0 2024-04-25T10:03:12.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.092+0000 7f06257fa640 1 -- 172.21.15.19:0/3842972223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0620042020 con 0x7f062806bbf0 2024-04-25T10:03:12.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.092+0000 7f06257fa640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061406b4d0 0x7f061406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.092+0000 7f06257fa640 1 -- 172.21.15.19:0/3842972223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f06200aa5c0 con 0x7f062806bbf0 2024-04-25T10:03:12.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.092+0000 7f0627fff640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061406b4d0 0x7f061406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.093+0000 7f0627fff640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061406b4d0 0x7f061406d990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f06281f6fa0 tx=0x7f061c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.096+0000 7f06257fa640 1 -- 172.21.15.19:0/3842972223 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0620049030 con 0x7f062806bbf0 2024-04-25T10:03:12.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.275+0000 7f062eaf5640 1 -- 172.21.15.19:0/3842972223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0628061b30 con 0x7f062806bbf0 2024-04-25T10:03:12.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.649+0000 7f06257fa640 1 -- 172.21.15.19:0/3842972223 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v151) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0620079690 con 0x7f062806bbf0 2024-04-25T10:03:12.654 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T10:03:12.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.652+0000 7f062eaf5640 1 -- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061406b4d0 msgr2=0x7f061406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.652+0000 7f062eaf5640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061406b4d0 0x7f061406d990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f06281f6fa0 tx=0x7f061c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:12.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.652+0000 7f062eaf5640 1 -- 172.21.15.19:0/3842972223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f062806bbf0 msgr2=0x7f06280b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.652+0000 7f062eaf5640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f062806bbf0 0x7f06280b2b90 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f0620009f70 tx=0x7f0620004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.653+0000 7f062eaf5640 1 -- 172.21.15.19:0/3842972223 shutdown_connections 2024-04-25T10:03:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.653+0000 7f062eaf5640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f061406b4d0 0x7f061406d990 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.653+0000 7f062eaf5640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f062815dbd0 0x7f062814b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.653+0000 7f062eaf5640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f062806ff30 0x7f06280b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.653+0000 7f062eaf5640 1 --2- 172.21.15.19:0/3842972223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f062806bbf0 0x7f06280b2b90 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.653+0000 7f062eaf5640 1 -- 172.21.15.19:0/3842972223 >> 172.21.15.19:0/3842972223 conn(0x7f0628071af0 msgr2=0x7f06280b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.653+0000 7f062eaf5640 1 -- 172.21.15.19:0/3842972223 shutdown_connections 2024-04-25T10:03:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.653+0000 7f062eaf5640 1 -- 172.21.15.19:0/3842972223 wait complete. 2024-04-25T10:03:12.675 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:03:12.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.780+0000 7f321caf5640 1 Processor -- start 2024-04-25T10:03:12.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.780+0000 7f321caf5640 1 -- start start 2024-04-25T10:03:12.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f321caf5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f321815dc20 0x7f32180ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f321caf5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3218161710 0x7f32180cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f321caf5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321815e000 con 0x7f32180b9330 2024-04-25T10:03:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f321caf5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32180d1de0 con 0x7f3218161710 2024-04-25T10:03:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f321caf5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3218161c60 con 0x7f321815dc20 2024-04-25T10:03:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f320ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3218161710 0x7f32180cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f320ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3218161710 0x7f32180cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39544/0 (socket says 172.21.15.19:39544) 2024-04-25T10:03:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f320ffff640 1 -- 172.21.15.19:0/1790451983 learned_addr learned my addr 172.21.15.19:0/1790451983 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f320ffff640 1 -- 172.21.15.19:0/1790451983 >> v2:172.21.15.112:3300/0 conn(0x7f321815dc20 msgr2=0x7f32180ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f320ffff640 1 --2- 172.21.15.19:0/1790451983 >> v2:172.21.15.112:3300/0 conn(0x7f321815dc20 0x7f32180ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f320ffff640 1 -- 172.21.15.19:0/1790451983 >> v1:172.21.15.112:6789/0 conn(0x7f32180b9330 legacy=0x7f321806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.781+0000 7f320ffff640 1 -- 172.21.15.19:0/1790451983 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3218161f40 con 0x7f3218161710 2024-04-25T10:03:12.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.782+0000 7f320ffff640 1 --2- 172.21.15.19:0/1790451983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3218161710 0x7f32180cebb0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f32180d1520 tx=0x7f320002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57fbbe5f4165d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.782+0000 7f3215d9b640 1 -- 172.21.15.19:0/1790451983 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3200037070 con 0x7f3218161710 2024-04-25T10:03:12.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.782+0000 7f3215d9b640 1 -- 172.21.15.19:0/1790451983 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3200032440 con 0x7f3218161710 2024-04-25T10:03:12.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.782+0000 7f3215d9b640 1 -- 172.21.15.19:0/1790451983 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3200032810 con 0x7f3218161710 2024-04-25T10:03:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.782+0000 7f321caf5640 1 -- 172.21.15.19:0/1790451983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3218161710 msgr2=0x7f32180cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.782+0000 7f321caf5640 1 --2- 172.21.15.19:0/1790451983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3218161710 0x7f32180cebb0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f32180d1520 tx=0x7f320002a230 comp rx=0 tx=0).stop 2024-04-25T10:03:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.783+0000 7f321caf5640 1 -- 172.21.15.19:0/1790451983 shutdown_connections 2024-04-25T10:03:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.783+0000 7f321caf5640 1 --2- 172.21.15.19:0/1790451983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3218161710 0x7f32180cebb0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.783+0000 7f321caf5640 1 --2- 172.21.15.19:0/1790451983 >> v2:172.21.15.112:3300/0 conn(0x7f321815dc20 0x7f32180ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.783+0000 7f321caf5640 1 -- 172.21.15.19:0/1790451983 >> 172.21.15.19:0/1790451983 conn(0x7f32180d0420 msgr2=0x7f32180d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.783+0000 7f321caf5640 1 -- 172.21.15.19:0/1790451983 shutdown_connections 2024-04-25T10:03:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.783+0000 7f321caf5640 1 -- 172.21.15.19:0/1790451983 wait complete. 2024-04-25T10:03:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.784+0000 7f321caf5640 1 Processor -- start 2024-04-25T10:03:12.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.784+0000 7f321caf5640 1 -- start start 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f321caf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32180b9330 0x7f3218150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f321caf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f321815dc20 0x7f32180ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f321caf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3218161710 0x7f32180cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f321caf5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32180b5e50 con 0x7f3218161710 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f321caf5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3218162500 con 0x7f32180b9330 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f321caf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32180b5fd0 con 0x7f321815dc20 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f3216d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3218161710 0x7f32180cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f3216d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3218161710 0x7f32180cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39556/0 (socket says 172.21.15.19:39556) 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f3216d9d640 1 -- 172.21.15.19:0/3627106880 learned_addr learned my addr 172.21.15.19:0/3627106880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f321659c640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32180b9330 0x7f3218150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f320ffff640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f321815dc20 0x7f32180ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f3216d9d640 1 -- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f321815dc20 msgr2=0x7f32180ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f3216d9d640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f321815dc20 0x7f32180ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f3216d9d640 1 -- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32180b9330 msgr2=0x7f3218150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f3216d9d640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32180b9330 0x7f3218150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f3216d9d640 1 -- 172.21.15.19:0/3627106880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32180cd690 con 0x7f3218161710 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f321659c640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32180b9330 0x7f3218150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f320ffff640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f321815dc20 0x7f32180ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f3216d9d640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3218161710 0x7f32180cd150 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f321000ba10 tx=0x7f321000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f320effd640 1 -- 172.21.15.19:0/3627106880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210019070 con 0x7f3218161710 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f320effd640 1 -- 172.21.15.19:0/3627106880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321000cb20 con 0x7f3218161710 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f321caf5640 1 -- 172.21.15.19:0/3627106880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321814cd40 con 0x7f3218161710 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f320effd640 1 -- 172.21.15.19:0/3627106880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210014440 con 0x7f3218161710 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.785+0000 7f321caf5640 1 -- 172.21.15.19:0/3627106880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321814d300 con 0x7f3218161710 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.787+0000 7f321caf5640 1 -- 172.21.15.19:0/3627106880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32180cfc20 con 0x7f3218161710 2024-04-25T10:03:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.787+0000 7f320effd640 1 -- 172.21.15.19:0/3627106880 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f321001c070 con 0x7f3218161710 2024-04-25T10:03:12.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.788+0000 7f320effd640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f31f806b4d0 0x7f31f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.788+0000 7f320effd640 1 -- 172.21.15.19:0/3627106880 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f3210089db0 con 0x7f3218161710 2024-04-25T10:03:12.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.788+0000 7f321659c640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f31f806b4d0 0x7f31f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.788+0000 7f321659c640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f31f806b4d0 0x7f31f806d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f32180b8dd0 tx=0x7f3208008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.792+0000 7f320effd640 1 -- 172.21.15.19:0/3627106880 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32180cfc20 con 0x7f3218161710 2024-04-25T10:03:12.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:12.973+0000 7f321caf5640 1 -- 172.21.15.19:0/3627106880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3218073a00 con 0x7f3218161710 2024-04-25T10:03:13.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.655+0000 7f320effd640 1 -- 172.21.15.19:0/3627106880 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v152) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3210058d10 con 0x7f3218161710 2024-04-25T10:03:13.660 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T10:03:13.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.658+0000 7f321caf5640 1 -- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f31f806b4d0 msgr2=0x7f31f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.658+0000 7f321caf5640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f31f806b4d0 0x7f31f806d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f32180b8dd0 tx=0x7f3208008040 comp rx=0 tx=0).stop 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.659+0000 7f321caf5640 1 -- 172.21.15.19:0/3627106880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3218161710 msgr2=0x7f32180cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.659+0000 7f321caf5640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3218161710 0x7f32180cd150 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f321000ba10 tx=0x7f321000bed0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.659+0000 7f321caf5640 1 -- 172.21.15.19:0/3627106880 shutdown_connections 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.659+0000 7f321caf5640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f31f806b4d0 0x7f31f806d990 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.659+0000 7f321caf5640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3218161710 0x7f32180cd150 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.659+0000 7f321caf5640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f321815dc20 0x7f32180ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.659+0000 7f321caf5640 1 --2- 172.21.15.19:0/3627106880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32180b9330 0x7f3218150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.659+0000 7f321caf5640 1 -- 172.21.15.19:0/3627106880 >> 172.21.15.19:0/3627106880 conn(0x7f32180d0420 msgr2=0x7f32180b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.659+0000 7f321caf5640 1 -- 172.21.15.19:0/3627106880 shutdown_connections 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.659+0000 7f321caf5640 1 -- 172.21.15.19:0/3627106880 wait complete. 2024-04-25T10:03:13.694 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.798+0000 7fbc92b9e640 1 Processor -- start 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.798+0000 7fbc92b9e640 1 -- start start 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc92b9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbc8c0ced80 0x7fbc8c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc92b9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbc8c161710 0x7fbc8c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc92b9e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8c15e000 con 0x7fbc8c15dc20 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc92b9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8c0d1de0 con 0x7fbc8c0ced80 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc92b9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8c161c60 con 0x7fbc8c161710 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc8bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbc8c0ced80 0x7fbc8c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc9093a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbc8c15dc20 0x7fbc8c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45198/0 (socket says 172.21.15.19:45198) 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc8bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbc8c0ced80 0x7fbc8c0d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39564/0 (socket says 172.21.15.19:39564) 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc9093a640 1 -- 172.21.15.19:0/2193757407 learned_addr learned my addr 172.21.15.19:0/2193757407 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc9113b640 1 --2- 172.21.15.19:0/2193757407 >> v2:172.21.15.112:3300/0 conn(0x7fbc8c161710 0x7fbc8c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc8bfff640 1 -- 172.21.15.19:0/2193757407 >> v2:172.21.15.112:3300/0 conn(0x7fbc8c161710 msgr2=0x7fbc8c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc8bfff640 1 --2- 172.21.15.19:0/2193757407 >> v2:172.21.15.112:3300/0 conn(0x7fbc8c161710 0x7fbc8c0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc8bfff640 1 -- 172.21.15.19:0/2193757407 >> v1:172.21.15.19:6789/0 conn(0x7fbc8c15dc20 legacy=0x7fbc8c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc8bfff640 1 -- 172.21.15.19:0/2193757407 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c161f40 con 0x7fbc8c0ced80 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.799+0000 7fbc8b7fe640 1 -- 172.21.15.19:0/2193757407 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329512306 0 0) 0x7fbc8c15e000 con 0x7fbc8c15dc20 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.800+0000 7fbc8bfff640 1 --2- 172.21.15.19:0/2193757407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc8c0ced80 0x7fbc8c0d22e0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0b6570 tx=0x7fbc7802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7157282d5cc2a228 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.800+0000 7fbc8b7fe640 1 -- 172.21.15.19:0/2193757407 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78037070 con 0x7fbc8c0ced80 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.800+0000 7fbc8b7fe640 1 -- 172.21.15.19:0/2193757407 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc78032440 con 0x7fbc8c0ced80 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.800+0000 7fbc8b7fe640 1 -- 172.21.15.19:0/2193757407 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78032810 con 0x7fbc8c0ced80 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.801+0000 7fbc92b9e640 1 -- 172.21.15.19:0/2193757407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc8c0ced80 msgr2=0x7fbc8c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.801+0000 7fbc92b9e640 1 --2- 172.21.15.19:0/2193757407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc8c0ced80 0x7fbc8c0d22e0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0b6570 tx=0x7fbc7802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.801+0000 7fbc92b9e640 1 -- 172.21.15.19:0/2193757407 shutdown_connections 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.801+0000 7fbc92b9e640 1 --2- 172.21.15.19:0/2193757407 >> v2:172.21.15.112:3300/0 conn(0x7fbc8c161710 0x7fbc8c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.801+0000 7fbc92b9e640 1 --2- 172.21.15.19:0/2193757407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc8c0ced80 0x7fbc8c0d22e0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.801+0000 7fbc92b9e640 1 -- 172.21.15.19:0/2193757407 >> 172.21.15.19:0/2193757407 conn(0x7fbc8c0d0420 msgr2=0x7fbc8c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.801+0000 7fbc92b9e640 1 -- 172.21.15.19:0/2193757407 shutdown_connections 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.801+0000 7fbc92b9e640 1 -- 172.21.15.19:0/2193757407 wait complete. 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.802+0000 7fbc92b9e640 1 Processor -- start 2024-04-25T10:03:13.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.802+0000 7fbc92b9e640 1 -- start start 2024-04-25T10:03:13.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc92b9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc8c0ced80 0x7fbc8c152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc92b9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc8c15dc20 0x7fbc8c1e7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc92b9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc8c161710 0x7fbc8c1e7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc92b9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0b5e50 con 0x7fbc8c161710 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc92b9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c162500 con 0x7fbc8c15dc20 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc92b9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0b5fd0 con 0x7fbc8c0ced80 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc9113b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc8c161710 0x7fbc8c1e7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc9113b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc8c161710 0x7fbc8c1e7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39580/0 (socket says 172.21.15.19:39580) 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc9113b640 1 -- 172.21.15.19:0/96074957 learned_addr learned my addr 172.21.15.19:0/96074957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc8bfff640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc8c15dc20 0x7fbc8c1e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc9093a640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc8c0ced80 0x7fbc8c152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc9113b640 1 -- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc8c0ced80 msgr2=0x7fbc8c152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc9113b640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc8c0ced80 0x7fbc8c152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc9113b640 1 -- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc8c15dc20 msgr2=0x7fbc8c1e7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc9113b640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc8c15dc20 0x7fbc8c1e7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc9113b640 1 -- 172.21.15.19:0/96074957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c15f570 con 0x7fbc8c161710 2024-04-25T10:03:13.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.803+0000 7fbc9113b640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc8c161710 0x7fbc8c1e7a50 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fbc84007d20 tx=0x7fbc8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.804+0000 7fbc897fa640 1 -- 172.21.15.19:0/96074957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84014420 con 0x7fbc8c161710 2024-04-25T10:03:13.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.804+0000 7fbc897fa640 1 -- 172.21.15.19:0/96074957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc84019070 con 0x7fbc8c161710 2024-04-25T10:03:13.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.804+0000 7fbc897fa640 1 -- 172.21.15.19:0/96074957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84014820 con 0x7fbc8c161710 2024-04-25T10:03:13.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.804+0000 7fbc92b9e640 1 -- 172.21.15.19:0/96074957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8c0c7db0 con 0x7fbc8c161710 2024-04-25T10:03:13.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.804+0000 7fbc92b9e640 1 -- 172.21.15.19:0/96074957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8c0c8330 con 0x7fbc8c161710 2024-04-25T10:03:13.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.805+0000 7fbc92b9e640 1 -- 172.21.15.19:0/96074957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8c073a00 con 0x7fbc8c161710 2024-04-25T10:03:13.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.805+0000 7fbc897fa640 1 -- 172.21.15.19:0/96074957 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc840149c0 con 0x7fbc8c161710 2024-04-25T10:03:13.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.806+0000 7fbc897fa640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc7006b4d0 0x7fbc7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.806+0000 7fbc897fa640 1 -- 172.21.15.19:0/96074957 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fbc8408aab0 con 0x7fbc8c161710 2024-04-25T10:03:13.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.806+0000 7fbc9093a640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc7006b4d0 0x7fbc7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.806+0000 7fbc9093a640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc7006b4d0 0x7fbc7006d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fbc80007fd0 tx=0x7fbc80007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:13.810+0000 7fbc897fa640 1 -- 172.21.15.19:0/96074957 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc8c15f570 con 0x7fbc8c161710 2024-04-25T10:03:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.028+0000 7fbc92b9e640 1 -- 172.21.15.19:0/96074957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbc8c0d16b0 con 0x7fbc8c161710 2024-04-25T10:03:14.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.657+0000 7fbc897fa640 1 -- 172.21.15.19:0/96074957 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v98) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbc84059110 con 0x7fbc8c161710 2024-04-25T10:03:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 23 and data pool 24 2024-04-25T10:03:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 -- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc7006b4d0 msgr2=0x7fbc7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc7006b4d0 0x7fbc7006d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fbc80007fd0 tx=0x7fbc80007de0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 -- 172.21.15.19:0/96074957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc8c161710 msgr2=0x7fbc8c1e7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc8c161710 0x7fbc8c1e7a50 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fbc84007d20 tx=0x7fbc8400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 -- 172.21.15.19:0/96074957 shutdown_connections 2024-04-25T10:03:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbc7006b4d0 0x7fbc7006d990 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc8c161710 0x7fbc8c1e7a50 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc8c15dc20 0x7fbc8c1e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 --2- 172.21.15.19:0/96074957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc8c0ced80 0x7fbc8c152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 -- 172.21.15.19:0/96074957 >> 172.21.15.19:0/96074957 conn(0x7fbc8c0d0420 msgr2=0x7fbc8c0b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 -- 172.21.15.19:0/96074957 shutdown_connections 2024-04-25T10:03:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.660+0000 7fbc92b9e640 1 -- 172.21.15.19:0/96074957 wait complete. 2024-04-25T10:03:14.730 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:14.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.833+0000 7f208e8ac640 1 Processor -- start 2024-04-25T10:03:14.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.833+0000 7f208e8ac640 1 -- start start 2024-04-25T10:03:14.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.833+0000 7f208e8ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20880b5f00 0x7f20880ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.833+0000 7f208e8ac640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20880cec00 con 0x7f20880d0b10 2024-04-25T10:03:14.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.833+0000 7f208e8ac640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20880ceee0 con 0x7f208815db60 2024-04-25T10:03:14.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.833+0000 7f208e8ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20880ce930 con 0x7f20880b5f00 2024-04-25T10:03:14.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.833+0000 7f20877fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f20880d0b10 0x7f20880d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45210/0 (socket says 172.21.15.19:45210) 2024-04-25T10:03:14.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.834+0000 7f20877fe640 1 -- 172.21.15.19:0/1665413306 learned_addr learned my addr 172.21.15.19:0/1665413306 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:14.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.834+0000 7f208ce49640 1 --2- 172.21.15.19:0/1665413306 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20880b5f00 0x7f20880ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.834+0000 7f2086ffd640 1 -- 172.21.15.19:0/1665413306 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491106672 0 0) 0x7f20880cec00 con 0x7f20880d0b10 2024-04-25T10:03:14.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.834+0000 7f2086ffd640 1 -- 172.21.15.19:0/1665413306 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2070003520 con 0x7f20880d0b10 2024-04-25T10:03:14.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.834+0000 7f2086ffd640 1 -- 172.21.15.19:0/1665413306 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719959709 0 0) 0x7f20880ceee0 con 0x7f208815db60 2024-04-25T10:03:14.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.834+0000 7f2086ffd640 1 -- 172.21.15.19:0/1665413306 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20880cec00 con 0x7f208815db60 2024-04-25T10:03:14.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.834+0000 7f208ce49640 1 -- 172.21.15.19:0/1665413306 >> v1:172.21.15.112:6789/0 conn(0x7f208815db60 legacy=0x7f20880d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.834+0000 7f208ce49640 1 -- 172.21.15.19:0/1665413306 >> v1:172.21.15.19:6789/0 conn(0x7f20880d0b10 legacy=0x7f20880d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.834+0000 7f208ce49640 1 -- 172.21.15.19:0/1665413306 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20880cf1c0 con 0x7f20880b5f00 2024-04-25T10:03:14.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.835+0000 7f208ce49640 1 --2- 172.21.15.19:0/1665413306 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20880b5f00 0x7f20880ce3f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2080009df0 tx=0x7f208002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c469cc486a213ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.835+0000 7f2086ffd640 1 -- 172.21.15.19:0/1665413306 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208002ae60 con 0x7f20880b5f00 2024-04-25T10:03:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.835+0000 7f2086ffd640 1 -- 172.21.15.19:0/1665413306 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080007740 con 0x7f20880b5f00 2024-04-25T10:03:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.835+0000 7f2086ffd640 1 -- 172.21.15.19:0/1665413306 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080007af0 con 0x7f20880b5f00 2024-04-25T10:03:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.835+0000 7f208e8ac640 1 -- 172.21.15.19:0/1665413306 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20880b5f00 msgr2=0x7f20880ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.835+0000 7f208e8ac640 1 --2- 172.21.15.19:0/1665413306 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20880b5f00 0x7f20880ce3f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2080009df0 tx=0x7f208002a680 comp rx=0 tx=0).stop 2024-04-25T10:03:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.836+0000 7f208e8ac640 1 -- 172.21.15.19:0/1665413306 shutdown_connections 2024-04-25T10:03:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.836+0000 7f208e8ac640 1 --2- 172.21.15.19:0/1665413306 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20880b5f00 0x7f20880ce3f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.836+0000 7f208e8ac640 1 -- 172.21.15.19:0/1665413306 >> 172.21.15.19:0/1665413306 conn(0x7f2088061600 msgr2=0x7f2088061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.836+0000 7f208e8ac640 1 -- 172.21.15.19:0/1665413306 shutdown_connections 2024-04-25T10:03:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.836+0000 7f208e8ac640 1 -- 172.21.15.19:0/1665413306 wait complete. 2024-04-25T10:03:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.837+0000 7f208e8ac640 1 Processor -- start 2024-04-25T10:03:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.837+0000 7f208e8ac640 1 -- start start 2024-04-25T10:03:14.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f208e8ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20880b5f00 0x7f20880c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f208e8ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20880d0b10 0x7f20880c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f208e8ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208815db60 0x7f208815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f208e8ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208815df40 con 0x7f20880b5f00 2024-04-25T10:03:14.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f208e8ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20880b6450 con 0x7f208815db60 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f208e8ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208815e0c0 con 0x7f20880d0b10 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2087fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20880b5f00 0x7f20880c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2087fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20880b5f00 0x7f20880c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39588/0 (socket says 172.21.15.19:39588) 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2087fff640 1 -- 172.21.15.19:0/288607919 learned_addr learned my addr 172.21.15.19:0/288607919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f20877fe640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20880d0b10 0x7f20880c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f208ce49640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208815db60 0x7f208815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2087fff640 1 -- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20880d0b10 msgr2=0x7f20880c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2087fff640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20880d0b10 0x7f20880c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2087fff640 1 -- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208815db60 msgr2=0x7f208815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2087fff640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208815db60 0x7f208815e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2087fff640 1 -- 172.21.15.19:0/288607919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20880cb100 con 0x7f20880b5f00 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2087fff640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20880b5f00 0x7f20880c9e60 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f207800c530 tx=0x7f207800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2084ff9640 1 -- 172.21.15.19:0/288607919 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20780052d0 con 0x7f20880b5f00 2024-04-25T10:03:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2084ff9640 1 -- 172.21.15.19:0/288607919 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2078005490 con 0x7f20880b5f00 2024-04-25T10:03:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.838+0000 7f2084ff9640 1 -- 172.21.15.19:0/288607919 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078012870 con 0x7f20880b5f00 2024-04-25T10:03:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.839+0000 7f208e8ac640 1 -- 172.21.15.19:0/288607919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20880cb3e0 con 0x7f20880b5f00 2024-04-25T10:03:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.839+0000 7f208e8ac640 1 -- 172.21.15.19:0/288607919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20880cb9a0 con 0x7f20880b5f00 2024-04-25T10:03:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.840+0000 7f208e8ac640 1 -- 172.21.15.19:0/288607919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2088073270 con 0x7f20880b5f00 2024-04-25T10:03:14.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.840+0000 7f2084ff9640 1 -- 172.21.15.19:0/288607919 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2078012a10 con 0x7f20880b5f00 2024-04-25T10:03:14.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.841+0000 7f2084ff9640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f207406b4d0 0x7f207406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.841+0000 7f2084ff9640 1 -- 172.21.15.19:0/288607919 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2078089e90 con 0x7f20880b5f00 2024-04-25T10:03:14.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.841+0000 7f20877fe640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f207406b4d0 0x7f207406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.841+0000 7f20877fe640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f207406b4d0 0x7f207406d990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f208815e4a0 tx=0x7f207c007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:14.845+0000 7f2084ff9640 1 -- 172.21.15.19:0/288607919 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2078058540 con 0x7f20880b5f00 2024-04-25T10:03:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.025+0000 7f208e8ac640 1 -- 172.21.15.19:0/288607919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2088073990 con 0x7f20880b5f00 2024-04-25T10:03:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.027+0000 7f2084ff9640 1 -- 172.21.15.19:0/288607919 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f207805c5e0 con 0x7f20880b5f00 2024-04-25T10:03:15.032 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:15.032 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":153,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:03:14.029745+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:12.307052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:13.005066+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.028+0000 7f208e8ac640 1 -- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f207406b4d0 msgr2=0x7f207406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.028+0000 7f208e8ac640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f207406b4d0 0x7f207406d990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f208815e4a0 tx=0x7f207c007d30 comp rx=0 tx=0).stop 2024-04-25T10:03:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.028+0000 7f208e8ac640 1 -- 172.21.15.19:0/288607919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20880b5f00 msgr2=0x7f20880c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.028+0000 7f208e8ac640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20880b5f00 0x7f20880c9e60 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f207800c530 tx=0x7f207800c9f0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.028+0000 7f208e8ac640 1 -- 172.21.15.19:0/288607919 shutdown_connections 2024-04-25T10:03:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.028+0000 7f208e8ac640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f207406b4d0 0x7f207406d990 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.028+0000 7f208e8ac640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208815db60 0x7f208815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.028+0000 7f208e8ac640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20880d0b10 0x7f20880c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.028+0000 7f208e8ac640 1 --2- 172.21.15.19:0/288607919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20880b5f00 0x7f20880c9e60 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.028+0000 7f208e8ac640 1 -- 172.21.15.19:0/288607919 >> 172.21.15.19:0/288607919 conn(0x7f2088061600 msgr2=0x7f20880b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.029+0000 7f208e8ac640 1 -- 172.21.15.19:0/288607919 shutdown_connections 2024-04-25T10:03:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.029+0000 7f208e8ac640 1 -- 172.21.15.19:0/288607919 wait complete. 2024-04-25T10:03:15.045 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.141+0000 7f22cdf4f640 1 Processor -- start 2024-04-25T10:03:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.141+0000 7f22cdf4f640 1 -- start start 2024-04-25T10:03:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22cdf4f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22c815dd20 0x7f22c80b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22cdf4f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22c815e100 0x7f22c80d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22cdf4f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f22c815e560 0x7f22c80d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22cdf4f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c80d1c00 con 0x7f22c815dd20 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22cdf4f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c815eac0 con 0x7f22c815e560 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22cdf4f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c815eda0 con 0x7f22c815e100 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22c815dd20 0x7f22c80b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22c815dd20 0x7f22c80b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39598/0 (socket says 172.21.15.19:39598) 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c77fe640 1 -- 172.21.15.19:0/171038744 learned_addr learned my addr 172.21.15.19:0/171038744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c7fff640 1 --2- 172.21.15.19:0/171038744 >> v2:172.21.15.112:3300/0 conn(0x7f22c815e560 0x7f22c80d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c6ffd640 1 --2- 172.21.15.19:0/171038744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22c815e100 0x7f22c80d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c77fe640 1 -- 172.21.15.19:0/171038744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22c815e100 msgr2=0x7f22c80d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c77fe640 1 --2- 172.21.15.19:0/171038744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22c815e100 0x7f22c80d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c77fe640 1 -- 172.21.15.19:0/171038744 >> v2:172.21.15.112:3300/0 conn(0x7f22c815e560 msgr2=0x7f22c80d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c77fe640 1 --2- 172.21.15.19:0/171038744 >> v2:172.21.15.112:3300/0 conn(0x7f22c815e560 0x7f22c80d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c77fe640 1 -- 172.21.15.19:0/171038744 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c815f080 con 0x7f22c815dd20 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c7fff640 1 --2- 172.21.15.19:0/171038744 >> v2:172.21.15.112:3300/0 conn(0x7f22c815e560 0x7f22c80d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.142+0000 7f22c6ffd640 1 --2- 172.21.15.19:0/171038744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22c815e100 0x7f22c80d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.143+0000 7f22c77fe640 1 --2- 172.21.15.19:0/171038744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22c815dd20 0x7f22c80b96d0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f22c80cab60 tx=0x7f22b802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f818783c3cc16e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.143+0000 7f22c67fc640 1 -- 172.21.15.19:0/171038744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8037070 con 0x7f22c815dd20 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.143+0000 7f22c67fc640 1 -- 172.21.15.19:0/171038744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b8032440 con 0x7f22c815dd20 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.143+0000 7f22c67fc640 1 -- 172.21.15.19:0/171038744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8032810 con 0x7f22c815dd20 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.144+0000 7f22cdf4f640 1 -- 172.21.15.19:0/171038744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22c815dd20 msgr2=0x7f22c80b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.144+0000 7f22cdf4f640 1 --2- 172.21.15.19:0/171038744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22c815dd20 0x7f22c80b96d0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f22c80cab60 tx=0x7f22b802a140 comp rx=0 tx=0).stop 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.144+0000 7f22cdf4f640 1 -- 172.21.15.19:0/171038744 shutdown_connections 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.144+0000 7f22cdf4f640 1 --2- 172.21.15.19:0/171038744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22c815e100 0x7f22c80d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.144+0000 7f22cdf4f640 1 --2- 172.21.15.19:0/171038744 >> v2:172.21.15.112:3300/0 conn(0x7f22c815e560 0x7f22c80d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.144+0000 7f22cdf4f640 1 --2- 172.21.15.19:0/171038744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22c815dd20 0x7f22c80b96d0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.144+0000 7f22cdf4f640 1 -- 172.21.15.19:0/171038744 >> 172.21.15.19:0/171038744 conn(0x7f22c8061a20 msgr2=0x7f22c8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.144+0000 7f22cdf4f640 1 -- 172.21.15.19:0/171038744 shutdown_connections 2024-04-25T10:03:15.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.144+0000 7f22cdf4f640 1 -- 172.21.15.19:0/171038744 wait complete. 2024-04-25T10:03:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.145+0000 7f22cdf4f640 1 Processor -- start 2024-04-25T10:03:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.145+0000 7f22cdf4f640 1 -- start start 2024-04-25T10:03:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22cdf4f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22c815dd20 0x7f22c814e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22cdf4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c815e100 0x7f22c80cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22cdf4f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22c815e560 0x7f22c8156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22cdf4f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80d3c20 con 0x7f22c815e560 2024-04-25T10:03:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22cdf4f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80d3aa0 con 0x7f22c815e100 2024-04-25T10:03:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22cdf4f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80d3da0 con 0x7f22c815dd20 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22c815e560 0x7f22c8156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22c815e560 0x7f22c8156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39602/0 (socket says 172.21.15.19:39602) 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22c7fff640 1 -- 172.21.15.19:0/1100260005 learned_addr learned my addr 172.21.15.19:0/1100260005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22c6ffd640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c815e100 0x7f22c80cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22c77fe640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22c815dd20 0x7f22c814e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22c7fff640 1 -- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22c815dd20 msgr2=0x7f22c814e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22c7fff640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22c815dd20 0x7f22c814e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22c7fff640 1 -- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c815e100 msgr2=0x7f22c80cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22c7fff640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c815e100 0x7f22c80cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22c7fff640 1 -- 172.21.15.19:0/1100260005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c80cc510 con 0x7f22c815e560 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.146+0000 7f22c7fff640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22c815e560 0x7f22c8156ff0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f22c0007e90 tx=0x7f22c000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.147+0000 7f22affff640 1 -- 172.21.15.19:0/1100260005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c001a070 con 0x7f22c815e560 2024-04-25T10:03:15.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.147+0000 7f22affff640 1 -- 172.21.15.19:0/1100260005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c0015400 con 0x7f22c815e560 2024-04-25T10:03:15.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.147+0000 7f22cdf4f640 1 -- 172.21.15.19:0/1100260005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c80caf30 con 0x7f22c815e560 2024-04-25T10:03:15.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.147+0000 7f22affff640 1 -- 172.21.15.19:0/1100260005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00157f0 con 0x7f22c815e560 2024-04-25T10:03:15.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.147+0000 7f22cdf4f640 1 -- 172.21.15.19:0/1100260005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c80cb4f0 con 0x7f22c815e560 2024-04-25T10:03:15.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.148+0000 7f22cdf4f640 1 -- 172.21.15.19:0/1100260005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c8073990 con 0x7f22c815e560 2024-04-25T10:03:15.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.148+0000 7f22affff640 1 -- 172.21.15.19:0/1100260005 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22c001d070 con 0x7f22c815e560 2024-04-25T10:03:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.149+0000 7f22affff640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f22b406b4d0 0x7f22b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.149+0000 7f22affff640 1 -- 172.21.15.19:0/1100260005 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f22c008a510 con 0x7f22c815e560 2024-04-25T10:03:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.149+0000 7f22c77fe640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f22b406b4d0 0x7f22b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.150+0000 7f22c77fe640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f22b406b4d0 0x7f22b406d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f22b8034000 tx=0x7f22b802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.153+0000 7f22affff640 1 -- 172.21.15.19:0/1100260005 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22c0010070 con 0x7f22c815e560 2024-04-25T10:03:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.332+0000 7f22cdf4f640 1 -- 172.21.15.19:0/1100260005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22c80b4ef0 con 0x7f22c815e560 2024-04-25T10:03:15.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.334+0000 7f22affff640 1 -- 172.21.15.19:0/1100260005 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f22c0058bb0 con 0x7f22c815e560 2024-04-25T10:03:15.339 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:15.339 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":153,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:03:14.029745+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:12.307052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:13.005066+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.335+0000 7f22cdf4f640 1 -- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f22b406b4d0 msgr2=0x7f22b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.335+0000 7f22cdf4f640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f22b406b4d0 0x7f22b406d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f22b8034000 tx=0x7f22b802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.335+0000 7f22cdf4f640 1 -- 172.21.15.19:0/1100260005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22c815e560 msgr2=0x7f22c8156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.335+0000 7f22cdf4f640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22c815e560 0x7f22c8156ff0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f22c0007e90 tx=0x7f22c000d4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.335+0000 7f22cdf4f640 1 -- 172.21.15.19:0/1100260005 shutdown_connections 2024-04-25T10:03:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.335+0000 7f22cdf4f640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f22b406b4d0 0x7f22b406d990 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.335+0000 7f22cdf4f640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22c815e560 0x7f22c8156ff0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.335+0000 7f22cdf4f640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22c815e100 0x7f22c80cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.335+0000 7f22cdf4f640 1 --2- 172.21.15.19:0/1100260005 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22c815dd20 0x7f22c814e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.335+0000 7f22cdf4f640 1 -- 172.21.15.19:0/1100260005 >> 172.21.15.19:0/1100260005 conn(0x7f22c8061a20 msgr2=0x7f22c80b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.336+0000 7f22cdf4f640 1 -- 172.21.15.19:0/1100260005 shutdown_connections 2024-04-25T10:03:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.336+0000 7f22cdf4f640 1 -- 172.21.15.19:0/1100260005 wait complete. 2024-04-25T10:03:15.352 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:03:15.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.450+0000 7feac9215640 1 Processor -- start 2024-04-25T10:03:15.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.450+0000 7feac9215640 1 -- start start 2024-04-25T10:03:15.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.451+0000 7feac9215640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cf600 0x7feac40cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.451+0000 7feac9215640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feac40cbf20 0x7feac40cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.451+0000 7feac9215640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac40c75e0 con 0x7feac415bb50 2024-04-25T10:03:15.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.451+0000 7feac9215640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac40c8680 con 0x7feac40cbf20 2024-04-25T10:03:15.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.451+0000 7feac9215640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac40cff80 con 0x7feac40cf600 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.451+0000 7feac359e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feac40cbf20 0x7feac40cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.451+0000 7feac359e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feac40cbf20 0x7feac40cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39606/0 (socket says 172.21.15.19:39606) 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.452+0000 7feac359e640 1 -- 172.21.15.19:0/563715470 learned_addr learned my addr 172.21.15.19:0/563715470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.452+0000 7feac259c640 1 --2- 172.21.15.19:0/563715470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cf600 0x7feac40cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.452+0000 7feac359e640 1 -- 172.21.15.19:0/563715470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cf600 msgr2=0x7feac40cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.452+0000 7feac359e640 1 --2- 172.21.15.19:0/563715470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cf600 0x7feac40cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.452+0000 7feac359e640 1 -- 172.21.15.19:0/563715470 >> v1:172.21.15.112:6789/0 conn(0x7feac415bb50 legacy=0x7feac40c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.452+0000 7feac359e640 1 -- 172.21.15.19:0/563715470 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac40c78c0 con 0x7feac40cbf20 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.452+0000 7feac259c640 1 --2- 172.21.15.19:0/563715470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cf600 0x7feac40cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.452+0000 7feac359e640 1 --2- 172.21.15.19:0/563715470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac40cbf20 0x7feac40cc3b0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7feab4009e50 tx=0x7feab402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f66cc04ae45269 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.453+0000 7feac1d9b640 1 -- 172.21.15.19:0/563715470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4036070 con 0x7feac40cbf20 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.453+0000 7feac1d9b640 1 -- 172.21.15.19:0/563715470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab4031600 con 0x7feac40cbf20 2024-04-25T10:03:15.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.453+0000 7feac1d9b640 1 -- 172.21.15.19:0/563715470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40319d0 con 0x7feac40cbf20 2024-04-25T10:03:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.453+0000 7feac9215640 1 -- 172.21.15.19:0/563715470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac40cbf20 msgr2=0x7feac40cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.453+0000 7feac9215640 1 --2- 172.21.15.19:0/563715470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac40cbf20 0x7feac40cc3b0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7feab4009e50 tx=0x7feab402a7a0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.453+0000 7feac9215640 1 -- 172.21.15.19:0/563715470 shutdown_connections 2024-04-25T10:03:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.453+0000 7feac9215640 1 --2- 172.21.15.19:0/563715470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac40cbf20 0x7feac40cc3b0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.453+0000 7feac9215640 1 --2- 172.21.15.19:0/563715470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cf600 0x7feac40cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.453+0000 7feac9215640 1 -- 172.21.15.19:0/563715470 >> 172.21.15.19:0/563715470 conn(0x7feac406f500 msgr2=0x7feac406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.453+0000 7feac9215640 1 -- 172.21.15.19:0/563715470 shutdown_connections 2024-04-25T10:03:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.453+0000 7feac9215640 1 -- 172.21.15.19:0/563715470 wait complete. 2024-04-25T10:03:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.455+0000 7feac9215640 1 Processor -- start 2024-04-25T10:03:15.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.455+0000 7feac9215640 1 -- start start 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.455+0000 7feac9215640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cbf20 0x7feac415e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.455+0000 7feac9215640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac40cf600 0x7feac414ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.455+0000 7feac9215640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac415bb50 0x7feac414b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.455+0000 7feac9215640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac40c8160 con 0x7feac415bb50 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.455+0000 7feac9215640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac40c7e80 con 0x7feac40cf600 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.455+0000 7feac9215640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac40c72a0 con 0x7feac40cbf20 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.455+0000 7feac359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac415bb50 0x7feac414b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.455+0000 7feac359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac415bb50 0x7feac414b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39620/0 (socket says 172.21.15.19:39620) 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.455+0000 7feac359e640 1 -- 172.21.15.19:0/1009043523 learned_addr learned my addr 172.21.15.19:0/1009043523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac2d9d640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cbf20 0x7feac415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac259c640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac40cf600 0x7feac414ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac359e640 1 -- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cbf20 msgr2=0x7feac415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac359e640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cbf20 0x7feac415e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac359e640 1 -- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac40cf600 msgr2=0x7feac414ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac359e640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac40cf600 0x7feac414ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac359e640 1 -- 172.21.15.19:0/1009043523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac4157c60 con 0x7feac415bb50 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac2d9d640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cbf20 0x7feac415e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac259c640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac40cf600 0x7feac414ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac359e640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac415bb50 0x7feac414b3d0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7feab4034040 tx=0x7feab4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feabb7fe640 1 -- 172.21.15.19:0/1009043523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4041020 con 0x7feac415bb50 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feabb7fe640 1 -- 172.21.15.19:0/1009043523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab4007780 con 0x7feac415bb50 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feabb7fe640 1 -- 172.21.15.19:0/1009043523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4036040 con 0x7feac415bb50 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac9215640 1 -- 172.21.15.19:0/1009043523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac4157f40 con 0x7feac415bb50 2024-04-25T10:03:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.456+0000 7feac9215640 1 -- 172.21.15.19:0/1009043523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac4158500 con 0x7feac415bb50 2024-04-25T10:03:15.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.457+0000 7feac9215640 1 -- 172.21.15.19:0/1009043523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac4073a00 con 0x7feac415bb50 2024-04-25T10:03:15.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.458+0000 7feabb7fe640 1 -- 172.21.15.19:0/1009043523 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feab4044020 con 0x7feac415bb50 2024-04-25T10:03:15.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.458+0000 7feabb7fe640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feaa406b4d0 0x7feaa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.458+0000 7feabb7fe640 1 -- 172.21.15.19:0/1009043523 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feab40aafb0 con 0x7feac415bb50 2024-04-25T10:03:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.458+0000 7feac2d9d640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feaa406b4d0 0x7feaa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.459+0000 7feac2d9d640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feaa406b4d0 0x7feaa406d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7feabc0015e0 tx=0x7feabc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.462+0000 7feabb7fe640 1 -- 172.21.15.19:0/1009043523 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feab404b030 con 0x7feac415bb50 2024-04-25T10:03:15.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:15.684+0000 7feac9215640 1 -- 172.21.15.19:0/1009043523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feac40cf150 con 0x7feac415bb50 2024-04-25T10:03:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.685+0000 7feabb7fe640 1 -- 172.21.15.19:0/1009043523 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v101) v1 ==== 118+0+0 (secure 0 0 0) 0x7feab4079d30 con 0x7feac415bb50 2024-04-25T10:03:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.688+0000 7feac9215640 1 -- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feaa406b4d0 msgr2=0x7feaa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.688+0000 7feac9215640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feaa406b4d0 0x7feaa406d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7feabc0015e0 tx=0x7feabc009040 comp rx=0 tx=0).stop 2024-04-25T10:03:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.688+0000 7feac9215640 1 -- 172.21.15.19:0/1009043523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac415bb50 msgr2=0x7feac414b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.688+0000 7feac9215640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac415bb50 0x7feac414b3d0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7feab4034040 tx=0x7feab4004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.688+0000 7feac9215640 1 -- 172.21.15.19:0/1009043523 shutdown_connections 2024-04-25T10:03:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.688+0000 7feac9215640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7feaa406b4d0 0x7feaa406d990 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.688+0000 7feac9215640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac415bb50 0x7feac414b3d0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.688+0000 7feac9215640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feac40cf600 0x7feac414ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.688+0000 7feac9215640 1 --2- 172.21.15.19:0/1009043523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feac40cbf20 0x7feac415e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.688+0000 7feac9215640 1 -- 172.21.15.19:0/1009043523 >> 172.21.15.19:0/1009043523 conn(0x7feac406f500 msgr2=0x7feac40cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.689+0000 7feac9215640 1 -- 172.21.15.19:0/1009043523 shutdown_connections 2024-04-25T10:03:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.689+0000 7feac9215640 1 -- 172.21.15.19:0/1009043523 wait complete. 2024-04-25T10:03:16.707 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:03:16.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:03:16.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.813+0000 7ff90f2cb640 1 Processor -- start 2024-04-25T10:03:16.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.813+0000 7ff90f2cb640 1 -- start start 2024-04-25T10:03:16.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.814+0000 7ff90f2cb640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff90815b7d0 0x7ff9080cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.814+0000 7ff90f2cb640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90806cf50 con 0x7ff9080b5620 2024-04-25T10:03:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.814+0000 7ff90f2cb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9080ce530 con 0x7ff90806c9e0 2024-04-25T10:03:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.814+0000 7ff90f2cb640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9080ce810 con 0x7ff90815b7d0 2024-04-25T10:03:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.814+0000 7ff90d067640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff90815b7d0 0x7ff9080cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.814+0000 7ff90c866640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff9080b5620 0x7ff90806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45222/0 (socket says 172.21.15.19:45222) 2024-04-25T10:03:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.814+0000 7ff90c866640 1 -- 172.21.15.19:0/1507672892 learned_addr learned my addr 172.21.15.19:0/1507672892 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.814+0000 7ff90d067640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff90815b7d0 0x7ff9080cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39622/0 (socket says 172.21.15.19:39622) 2024-04-25T10:03:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.815+0000 7ff90d067640 1 -- 172.21.15.19:0/1507672892 >> v1:172.21.15.112:6789/0 conn(0x7ff90806c9e0 legacy=0x7ff9080d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.815+0000 7ff90d067640 1 -- 172.21.15.19:0/1507672892 >> v1:172.21.15.19:6789/0 conn(0x7ff9080b5620 legacy=0x7ff90806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.815+0000 7ff90d067640 1 -- 172.21.15.19:0/1507672892 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9080ceaf0 con 0x7ff90815b7d0 2024-04-25T10:03:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.815+0000 7ff90d067640 1 --2- 172.21.15.19:0/1507672892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff90815b7d0 0x7ff9080cf010 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7ff9000097a0 tx=0x7ff90002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb648d773f7a37c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.815+0000 7ff907fff640 1 -- 172.21.15.19:0/1507672892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900037070 con 0x7ff90815b7d0 2024-04-25T10:03:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.816+0000 7ff907fff640 1 -- 172.21.15.19:0/1507672892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff900032440 con 0x7ff90815b7d0 2024-04-25T10:03:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.816+0000 7ff907fff640 1 -- 172.21.15.19:0/1507672892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900032810 con 0x7ff90815b7d0 2024-04-25T10:03:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.816+0000 7ff90f2cb640 1 -- 172.21.15.19:0/1507672892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff90815b7d0 msgr2=0x7ff9080cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.816+0000 7ff90f2cb640 1 --2- 172.21.15.19:0/1507672892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff90815b7d0 0x7ff9080cf010 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7ff9000097a0 tx=0x7ff90002a230 comp rx=0 tx=0).stop 2024-04-25T10:03:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.816+0000 7ff90f2cb640 1 -- 172.21.15.19:0/1507672892 shutdown_connections 2024-04-25T10:03:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.816+0000 7ff90f2cb640 1 --2- 172.21.15.19:0/1507672892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff90815b7d0 0x7ff9080cf010 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.816+0000 7ff90f2cb640 1 -- 172.21.15.19:0/1507672892 >> 172.21.15.19:0/1507672892 conn(0x7ff9080d0800 msgr2=0x7ff9080d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.816+0000 7ff90f2cb640 1 -- 172.21.15.19:0/1507672892 shutdown_connections 2024-04-25T10:03:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.816+0000 7ff90f2cb640 1 -- 172.21.15.19:0/1507672892 wait complete. 2024-04-25T10:03:16.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.817+0000 7ff90f2cb640 1 Processor -- start 2024-04-25T10:03:16.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.817+0000 7ff90f2cb640 1 -- start start 2024-04-25T10:03:16.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90f2cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff90806c9e0 0x7ff9080b8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90f2cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9080b5620 0x7ff9080cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90f2cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff90815b7d0 0x7ff9081f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90f2cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080cf6d0 con 0x7ff90806c9e0 2024-04-25T10:03:16.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90f2cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080cf550 con 0x7ff90815b7d0 2024-04-25T10:03:16.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90f2cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90815bbb0 con 0x7ff9080b5620 2024-04-25T10:03:16.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90d067640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff90806c9e0 0x7ff9080b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90d067640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff90806c9e0 0x7ff9080b8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39634/0 (socket says 172.21.15.19:39634) 2024-04-25T10:03:16.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90d067640 1 -- 172.21.15.19:0/3224464628 learned_addr learned my addr 172.21.15.19:0/3224464628 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:16.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90d868640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff90815b7d0 0x7ff9081f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90d067640 1 -- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9080b5620 msgr2=0x7ff9080cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.818+0000 7ff90c866640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9080b5620 0x7ff9080cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff90d067640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9080b5620 0x7ff9080cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff90d067640 1 -- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff90815b7d0 msgr2=0x7ff9081f7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff90d067640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff90815b7d0 0x7ff9081f7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff90d067640 1 -- 172.21.15.19:0/3224464628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9080ca120 con 0x7ff90806c9e0 2024-04-25T10:03:16.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff90d868640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff90815b7d0 0x7ff9081f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff90c866640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9080b5620 0x7ff9080cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff90d067640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff90806c9e0 0x7ff9080b8390 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7ff900035040 tx=0x7ff9000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff905ffb640 1 -- 172.21.15.19:0/3224464628 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900037040 con 0x7ff90806c9e0 2024-04-25T10:03:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff905ffb640 1 -- 172.21.15.19:0/3224464628 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff900030770 con 0x7ff90806c9e0 2024-04-25T10:03:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff905ffb640 1 -- 172.21.15.19:0/3224464628 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900030b80 con 0x7ff90806c9e0 2024-04-25T10:03:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff90f2cb640 1 -- 172.21.15.19:0/3224464628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9080ca400 con 0x7ff90806c9e0 2024-04-25T10:03:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.819+0000 7ff90f2cb640 1 -- 172.21.15.19:0/3224464628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9080b88d0 con 0x7ff90806c9e0 2024-04-25T10:03:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.820+0000 7ff90f2cb640 1 -- 172.21.15.19:0/3224464628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff908073a00 con 0x7ff90806c9e0 2024-04-25T10:03:16.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.821+0000 7ff905ffb640 1 -- 172.21.15.19:0/3224464628 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff900044020 con 0x7ff90806c9e0 2024-04-25T10:03:16.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.821+0000 7ff905ffb640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff8f406b500 0x7ff8f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.821+0000 7ff905ffb640 1 -- 172.21.15.19:0/3224464628 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff9000aa8a0 con 0x7ff90806c9e0 2024-04-25T10:03:16.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.821+0000 7ff90c866640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff8f406b500 0x7ff8f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.822+0000 7ff90c866640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff8f406b500 0x7ff8f406d9c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff9080b8d60 tx=0x7ff8fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:16.825+0000 7ff905ffb640 1 -- 172.21.15.19:0/3224464628 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff900032e20 con 0x7ff90806c9e0 2024-04-25T10:03:17.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.043+0000 7ff90f2cb640 1 -- 172.21.15.19:0/3224464628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff9080ca5d0 con 0x7ff90806c9e0 2024-04-25T10:03:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.690+0000 7ff905ffb640 1 -- 172.21.15.19:0/3224464628 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v102) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff9000313a0 con 0x7ff90806c9e0 2024-04-25T10:03:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 -- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff8f406b500 msgr2=0x7ff8f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff8f406b500 0x7ff8f406d9c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff9080b8d60 tx=0x7ff8fc009040 comp rx=0 tx=0).stop 2024-04-25T10:03:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 -- 172.21.15.19:0/3224464628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff90806c9e0 msgr2=0x7ff9080b8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff90806c9e0 0x7ff9080b8390 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7ff900035040 tx=0x7ff9000046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 -- 172.21.15.19:0/3224464628 shutdown_connections 2024-04-25T10:03:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff8f406b500 0x7ff8f406d9c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff90815b7d0 0x7ff9081f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9080b5620 0x7ff9080cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 --2- 172.21.15.19:0/3224464628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff90806c9e0 0x7ff9080b8390 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 -- 172.21.15.19:0/3224464628 >> 172.21.15.19:0/3224464628 conn(0x7ff9080d0800 msgr2=0x7ff9080d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 -- 172.21.15.19:0/3224464628 shutdown_connections 2024-04-25T10:03:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.693+0000 7ff90f2cb640 1 -- 172.21.15.19:0/3224464628 wait complete. 2024-04-25T10:03:17.711 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:03:17.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.814+0000 7fa0cacf5640 1 Processor -- start 2024-04-25T10:03:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.815+0000 7fa0cacf5640 1 -- start start 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.815+0000 7fa0cacf5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0c415dbe0 0x7fa0c40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.815+0000 7fa0cacf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.815+0000 7fa0cacf5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0c40b92a0 0x7fa0c40b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.815+0000 7fa0cacf5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c415dfc0 con 0x7fa0c40b92a0 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.815+0000 7fa0cacf5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40d1a00 con 0x7fa0c415dbe0 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.815+0000 7fa0cacf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b97a0 con 0x7fa0c40b2cb0 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.815+0000 7fa0c9292640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0c40b92a0 0x7fa0c40b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.816+0000 7fa0c9292640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0c40b92a0 0x7fa0c40b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49574/0 (socket says 172.21.15.19:49574) 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.816+0000 7fa0c9292640 1 -- 172.21.15.19:0/1261895055 learned_addr learned my addr 172.21.15.19:0/1261895055 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.816+0000 7fa0c8a91640 1 --2- 172.21.15.19:0/1261895055 >> v2:172.21.15.112:3300/0 conn(0x7fa0c415dbe0 0x7fa0c40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.816+0000 7fa0c3fff640 1 --2- 172.21.15.19:0/1261895055 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.816+0000 7fa0c9292640 1 -- 172.21.15.19:0/1261895055 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0c40b2cb0 msgr2=0x7fa0c40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.816+0000 7fa0c9292640 1 --2- 172.21.15.19:0/1261895055 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c40d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.816+0000 7fa0c9292640 1 -- 172.21.15.19:0/1261895055 >> v2:172.21.15.112:3300/0 conn(0x7fa0c415dbe0 msgr2=0x7fa0c40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.816+0000 7fa0c9292640 1 --2- 172.21.15.19:0/1261895055 >> v2:172.21.15.112:3300/0 conn(0x7fa0c415dbe0 0x7fa0c40d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.816+0000 7fa0c9292640 1 -- 172.21.15.19:0/1261895055 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40d1c50 con 0x7fa0c40b92a0 2024-04-25T10:03:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.816+0000 7fa0c3fff640 1 --2- 172.21.15.19:0/1261895055 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0c9292640 1 --2- 172.21.15.19:0/1261895055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40b92a0 0x7fa0c40b5400 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fa0c406c140 tx=0x7fa0ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea29fa20ab24b139 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0c37fe640 1 -- 172.21.15.19:0/1261895055 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac037070 con 0x7fa0c40b92a0 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0c37fe640 1 -- 172.21.15.19:0/1261895055 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac032440 con 0x7fa0c40b92a0 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0c37fe640 1 -- 172.21.15.19:0/1261895055 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac032810 con 0x7fa0c40b92a0 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0cacf5640 1 -- 172.21.15.19:0/1261895055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40b92a0 msgr2=0x7fa0c40b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0cacf5640 1 --2- 172.21.15.19:0/1261895055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40b92a0 0x7fa0c40b5400 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fa0c406c140 tx=0x7fa0ac02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0cacf5640 1 -- 172.21.15.19:0/1261895055 shutdown_connections 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0cacf5640 1 --2- 172.21.15.19:0/1261895055 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0cacf5640 1 --2- 172.21.15.19:0/1261895055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40b92a0 0x7fa0c40b5400 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0cacf5640 1 --2- 172.21.15.19:0/1261895055 >> v2:172.21.15.112:3300/0 conn(0x7fa0c415dbe0 0x7fa0c40d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0cacf5640 1 -- 172.21.15.19:0/1261895055 >> 172.21.15.19:0/1261895055 conn(0x7fa0c4061a20 msgr2=0x7fa0c4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0cacf5640 1 -- 172.21.15.19:0/1261895055 shutdown_connections 2024-04-25T10:03:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.817+0000 7fa0cacf5640 1 -- 172.21.15.19:0/1261895055 wait complete. 2024-04-25T10:03:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.818+0000 7fa0cacf5640 1 Processor -- start 2024-04-25T10:03:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.819+0000 7fa0cacf5640 1 -- start start 2024-04-25T10:03:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.819+0000 7fa0cacf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c41503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.819+0000 7fa0cacf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40b92a0 0x7fa0c415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.819+0000 7fa0cacf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.819+0000 7fa0cacf5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b6ab0 con 0x7fa0c40b92a0 2024-04-25T10:03:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.819+0000 7fa0cacf5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40d2210 con 0x7fa0c415dbe0 2024-04-25T10:03:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.819+0000 7fa0cacf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b6c30 con 0x7fa0c40b2cb0 2024-04-25T10:03:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.819+0000 7fa0c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40b92a0 0x7fa0c415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40b92a0 0x7fa0c415af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49590/0 (socket says 172.21.15.19:49590) 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c3fff640 1 -- 172.21.15.19:0/2214312623 learned_addr learned my addr 172.21.15.19:0/2214312623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c9292640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c8a91640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c41503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c3fff640 1 -- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0c40b2cb0 msgr2=0x7fa0c41503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c3fff640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c41503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c3fff640 1 -- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0c415dbe0 msgr2=0x7fa0c40b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c3fff640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c3fff640 1 -- 172.21.15.19:0/2214312623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40b40c0 con 0x7fa0c40b92a0 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c3fff640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40b92a0 0x7fa0c415af20 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00cf60 tx=0x7fa0bc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c17fa640 1 -- 172.21.15.19:0/2214312623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc01a070 con 0x7fa0c40b92a0 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c17fa640 1 -- 172.21.15.19:0/2214312623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc00d840 con 0x7fa0c40b92a0 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0cacf5640 1 -- 172.21.15.19:0/2214312623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c40b43a0 con 0x7fa0c40b92a0 2024-04-25T10:03:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0c17fa640 1 -- 172.21.15.19:0/2214312623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00dc10 con 0x7fa0c40b92a0 2024-04-25T10:03:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.820+0000 7fa0cacf5640 1 -- 172.21.15.19:0/2214312623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40b5fe0 con 0x7fa0c40b92a0 2024-04-25T10:03:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.821+0000 7fa0cacf5640 1 -- 172.21.15.19:0/2214312623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c4073990 con 0x7fa0c40b92a0 2024-04-25T10:03:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.822+0000 7fa0c17fa640 1 -- 172.21.15.19:0/2214312623 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0bc01d070 con 0x7fa0c40b92a0 2024-04-25T10:03:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.822+0000 7fa0c17fa640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0a406b4d0 0x7fa0a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.823+0000 7fa0c17fa640 1 -- 172.21.15.19:0/2214312623 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0bc08ac50 con 0x7fa0c40b92a0 2024-04-25T10:03:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.823+0000 7fa0c8a91640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0a406b4d0 0x7fa0a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.823+0000 7fa0c8a91640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0a406b4d0 0x7fa0a406d990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006fd0 tx=0x7fa0b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:17.827+0000 7fa0c17fa640 1 -- 172.21.15.19:0/2214312623 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0bc059250 con 0x7fa0c40b92a0 2024-04-25T10:03:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.044+0000 7fa0cacf5640 1 -- 172.21.15.19:0/2214312623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0c40622d0 con 0x7fa0c40b92a0 2024-04-25T10:03:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.045+0000 7fa0c17fa640 1 -- 172.21.15.19:0/2214312623 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4951 (secure 0 0 0) 0x7fa0bc05d2f0 con 0x7fa0c40b92a0 2024-04-25T10:03:18.050 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:18.050 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":102,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3725393673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3725393673},{"type":"v1","addr":"172.21.15.112:6837","nonce":3725393673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8866,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2539453196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2539453196},{"type":"v1","addr":"172.21.15.112:6836","nonce":2539453196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10221,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3896097924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3896097924},{"type":"v1","addr":"172.21.15.19:6834","nonce":3896097924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:14.030067+0000","modified":"2024-04-25T10:03:17.045113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10224":{"gid":10224,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/717842662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":717842662},{"type":"v1","addr":"172.21.15.19:6835","nonce":717842662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:03:18.050 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 102 2024-04-25T10:03:18.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 -- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0a406b4d0 msgr2=0x7fa0a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0a406b4d0 0x7fa0a406d990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006fd0 tx=0x7fa0b4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:18.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 -- 172.21.15.19:0/2214312623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40b92a0 msgr2=0x7fa0c415af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40b92a0 0x7fa0c415af20 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00cf60 tx=0x7fa0bc00cf90 comp rx=0 tx=0).stop 2024-04-25T10:03:18.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 -- 172.21.15.19:0/2214312623 shutdown_connections 2024-04-25T10:03:18.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa0a406b4d0 0x7fa0a406d990 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40b92a0 0x7fa0c415af20 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 --2- 172.21.15.19:0/2214312623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c41503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 -- 172.21.15.19:0/2214312623 >> 172.21.15.19:0/2214312623 conn(0x7fa0c4061a20 msgr2=0x7fa0c40d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 -- 172.21.15.19:0/2214312623 shutdown_connections 2024-04-25T10:03:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.047+0000 7fa0cacf5640 1 -- 172.21.15.19:0/2214312623 wait complete. 2024-04-25T10:03:18.063 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.159+0000 7fd57d3b6640 1 Processor -- start 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.159+0000 7fd57d3b6640 1 -- start start 2024-04-25T10:03:18.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd57d3b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780b9730 0x7fd5780ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd57d3b6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5780ceb90 0x7fd5780d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd57d3b6640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd578161d50 con 0x7fd57815da70 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd57d3b6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5781625e0 con 0x7fd5780ceb90 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd57d3b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780cf000 con 0x7fd5780b9730 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd576ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd57815da70 0x7fd578161640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46122/0 (socket says 172.21.15.19:46122) 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd576ffd640 1 -- 172.21.15.19:0/3524480372 learned_addr learned my addr 172.21.15.19:0/3524480372 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414041591 0 0) 0x7fd578161d50 con 0x7fd57815da70 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd560003520 con 0x7fd57815da70 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd5777fe640 1 --2- 172.21.15.19:0/3524480372 >> v2:172.21.15.112:3300/0 conn(0x7fd5780ceb90 0x7fd5780d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd5767fc640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780b9730 0x7fd5780ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 24433114 0 0) 0x7fd560003520 con 0x7fd57815da70 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780b9730 msgr2=0x7fd5780ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780b9730 0x7fd5780ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 >> v2:172.21.15.112:3300/0 conn(0x7fd5780ceb90 msgr2=0x7fd5780d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 --2- 172.21.15.19:0/3524480372 >> v2:172.21.15.112:3300/0 conn(0x7fd5780ceb90 0x7fd5780d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578162020 con 0x7fd57815da70 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fd568001670 con 0x7fd57815da70 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 >> v1:172.21.15.19:6789/0 conn(0x7fd57815da70 legacy=0x7fd578161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5600039b0 0x7fd560003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd560004800 0x7fd560004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.160+0000 7fd575ffb640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd560005180 0x7fd56000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.161+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781625e0 con 0x7fd560005180 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.161+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780cf000 con 0x7fd5600039b0 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.161+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56000bf30 con 0x7fd560004800 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.161+0000 7fd5777fe640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd560005180 0x7fd56000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.161+0000 7fd576ffd640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5600039b0 0x7fd560003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.161+0000 7fd5767fc640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd560004800 0x7fd560004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.161+0000 7fd5777fe640 1 -- 172.21.15.19:0/3524480372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd560004800 msgr2=0x7fd560004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.161+0000 7fd5777fe640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd560004800 0x7fd560004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.161+0000 7fd5777fe640 1 -- 172.21.15.19:0/3524480372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5600039b0 msgr2=0x7fd560003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.161+0000 7fd5777fe640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5600039b0 0x7fd560003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.161+0000 7fd5777fe640 1 -- 172.21.15.19:0/3524480372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd560003520 con 0x7fd560005180 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.162+0000 7fd5777fe640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd560005180 0x7fd56000b9c0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fd570007360 tx=0x7fd560005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6585944c33817259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.162+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570009be0 con 0x7fd560005180 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.162+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57002e620 con 0x7fd560005180 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.162+0000 7fd575ffb640 1 -- 172.21.15.19:0/3524480372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57002e9f0 con 0x7fd560005180 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.162+0000 7fd57d3b6640 1 -- 172.21.15.19:0/3524480372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd560005180 msgr2=0x7fd56000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.162+0000 7fd57d3b6640 1 --2- 172.21.15.19:0/3524480372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd560005180 0x7fd56000b9c0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fd570007360 tx=0x7fd560005830 comp rx=0 tx=0).stop 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.162+0000 7fd5777fe640 1 -- 172.21.15.19:0/3524480372 reap_dead start 2024-04-25T10:03:18.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.162+0000 7fd57d3b6640 1 -- 172.21.15.19:0/3524480372 shutdown_connections 2024-04-25T10:03:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.162+0000 7fd57d3b6640 1 -- 172.21.15.19:0/3524480372 >> 172.21.15.19:0/3524480372 conn(0x7fd578061a20 msgr2=0x7fd578061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.162+0000 7fd57d3b6640 1 -- 172.21.15.19:0/3524480372 shutdown_connections 2024-04-25T10:03:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.162+0000 7fd57d3b6640 1 -- 172.21.15.19:0/3524480372 wait complete. 2024-04-25T10:03:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.164+0000 7fd57d3b6640 1 Processor -- start 2024-04-25T10:03:18.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.164+0000 7fd57d3b6640 1 -- start start 2024-04-25T10:03:18.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.164+0000 7fd57d3b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780c8bb0 0x7fd57814d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.164+0000 7fd57d3b6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5780c8f90 0x7fd5780cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.164+0000 7fd57d3b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780cdc80 0x7fd5780cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.164+0000 7fd57d3b6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780d1dd0 con 0x7fd5780c8f90 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.164+0000 7fd57d3b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780d1af0 con 0x7fd5780cdc80 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.164+0000 7fd57d3b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57815de50 con 0x7fd5780c8bb0 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd5767fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5780c8f90 0x7fd5780cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd5767fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5780c8f90 0x7fd5780cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49606/0 (socket says 172.21.15.19:49606) 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd5767fc640 1 -- 172.21.15.19:0/2217726286 learned_addr learned my addr 172.21.15.19:0/2217726286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd576ffd640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780c8bb0 0x7fd57814d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd5767fc640 1 -- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780c8bb0 msgr2=0x7fd57814d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd5777fe640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780cdc80 0x7fd5780cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd5767fc640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780c8bb0 0x7fd57814d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd5767fc640 1 -- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780cdc80 msgr2=0x7fd5780cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd5767fc640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780cdc80 0x7fd5780cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd5767fc640 1 -- 172.21.15.19:0/2217726286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57815f3e0 con 0x7fd5780c8f90 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd576ffd640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780c8bb0 0x7fd57814d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd5777fe640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780cdc80 0x7fd5780cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd5767fc640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5780c8f90 0x7fd5780cd740 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fd56c010870 tx=0x7fd56c010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd55f7fe640 1 -- 172.21.15.19:0/2217726286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c01a020 con 0x7fd5780c8f90 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd55f7fe640 1 -- 172.21.15.19:0/2217726286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c00e9c0 con 0x7fd5780c8f90 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd57d3b6640 1 -- 172.21.15.19:0/2217726286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57815f6c0 con 0x7fd5780c8f90 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd55f7fe640 1 -- 172.21.15.19:0/2217726286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c00ed70 con 0x7fd5780c8f90 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.165+0000 7fd57d3b6640 1 -- 172.21.15.19:0/2217726286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57815fc80 con 0x7fd5780c8f90 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.166+0000 7fd57d3b6640 1 -- 172.21.15.19:0/2217726286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd578073990 con 0x7fd5780c8f90 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.167+0000 7fd55f7fe640 1 -- 172.21.15.19:0/2217726286 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd56c01d020 con 0x7fd5780c8f90 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.167+0000 7fd55f7fe640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd56406b500 0x7fd56406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.167+0000 7fd55f7fe640 1 -- 172.21.15.19:0/2217726286 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd56c08ae10 con 0x7fd5780c8f90 2024-04-25T10:03:18.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.168+0000 7fd576ffd640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd56406b500 0x7fd56406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.168+0000 7fd576ffd640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd56406b500 0x7fd56406d9c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd568004840 tx=0x7fd568009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.172+0000 7fd55f7fe640 1 -- 172.21.15.19:0/2217726286 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd56c024020 con 0x7fd5780c8f90 2024-04-25T10:03:18.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.351+0000 7fd57d3b6640 1 -- 172.21.15.19:0/2217726286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5780d1570 con 0x7fd5780c8f90 2024-04-25T10:03:18.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.352+0000 7fd55f7fe640 1 -- 172.21.15.19:0/2217726286 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+16872 (secure 0 0 0) 0x7fd56c059b00 con 0x7fd5780c8f90 2024-04-25T10:03:18.357 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:18.357 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":154,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:03:15.658204+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:12.307052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:13.005066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:18.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.354+0000 7fd57d3b6640 1 -- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd56406b500 msgr2=0x7fd56406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.354+0000 7fd57d3b6640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd56406b500 0x7fd56406d9c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd568004840 tx=0x7fd568009040 comp rx=0 tx=0).stop 2024-04-25T10:03:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.354+0000 7fd57d3b6640 1 -- 172.21.15.19:0/2217726286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5780c8f90 msgr2=0x7fd5780cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.354+0000 7fd57d3b6640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5780c8f90 0x7fd5780cd740 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fd56c010870 tx=0x7fd56c010d30 comp rx=0 tx=0).stop 2024-04-25T10:03:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.354+0000 7fd57d3b6640 1 -- 172.21.15.19:0/2217726286 shutdown_connections 2024-04-25T10:03:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.354+0000 7fd57d3b6640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd56406b500 0x7fd56406d9c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.354+0000 7fd57d3b6640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780cdc80 0x7fd5780cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.354+0000 7fd57d3b6640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5780c8f90 0x7fd5780cd740 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.354+0000 7fd57d3b6640 1 --2- 172.21.15.19:0/2217726286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780c8bb0 0x7fd57814d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.354+0000 7fd57d3b6640 1 -- 172.21.15.19:0/2217726286 >> 172.21.15.19:0/2217726286 conn(0x7fd578061a20 msgr2=0x7fd5780b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.355+0000 7fd57d3b6640 1 -- 172.21.15.19:0/2217726286 shutdown_connections 2024-04-25T10:03:18.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.355+0000 7fd57d3b6640 1 -- 172.21.15.19:0/2217726286 wait complete. 2024-04-25T10:03:18.371 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:03:18.371 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.467+0000 7f7adac7c640 1 Processor -- start 2024-04-25T10:03:18.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.467+0000 7f7adac7c640 1 -- start start 2024-04-25T10:03:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.468+0000 7f7adac7c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7ad40cfec0 0x7f7ad40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.468+0000 7f7adac7c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ad40ce3c0 0x7f7ad406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.468+0000 7f7adac7c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad4071480 con 0x7f7ad4154ee0 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.468+0000 7f7adac7c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad40ce830 con 0x7f7ad40cfec0 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.468+0000 7f7adac7c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad4072090 con 0x7f7ad40ce3c0 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.468+0000 7f7ad9219640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ad40ce3c0 0x7f7ad406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.468+0000 7f7ad3fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7ad40cfec0 0x7f7ad40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.468+0000 7f7ad8a18640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7ad4154ee0 0x7f7ad40cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:39494/0 (socket says 172.21.15.19:39494) 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.468+0000 7f7ad3fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7ad40cfec0 0x7f7ad40cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:58454/0 (socket says 172.21.15.19:58454) 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.468+0000 7f7ad9219640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ad40ce3c0 0x7f7ad406d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55290/0 (socket says 172.21.15.19:55290) 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.468+0000 7f7ad8a18640 1 -- 172.21.15.19:0/3581727626 learned_addr learned my addr 172.21.15.19:0/3581727626 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.469+0000 7f7ad3fff640 1 -- 172.21.15.19:0/3581727626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ad40ce3c0 msgr2=0x7f7ad406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.469+0000 7f7ad3fff640 1 --2- 172.21.15.19:0/3581727626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ad40ce3c0 0x7f7ad406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.469+0000 7f7ad3fff640 1 -- 172.21.15.19:0/3581727626 >> v1:172.21.15.112:6789/0 conn(0x7f7ad4154ee0 legacy=0x7f7ad40cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.469+0000 7f7ad3fff640 1 -- 172.21.15.19:0/3581727626 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad4071760 con 0x7f7ad40cfec0 2024-04-25T10:03:18.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.469+0000 7f7ad3fff640 1 --2- 172.21.15.19:0/3581727626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ad40cfec0 0x7f7ad40cde80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80097a0 tx=0x7f7ac802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e1a3354bbd7c683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.470+0000 7f7ad37fe640 1 -- 172.21.15.19:0/3581727626 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8037070 con 0x7f7ad40cfec0 2024-04-25T10:03:18.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.470+0000 7f7ad37fe640 1 -- 172.21.15.19:0/3581727626 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac8032440 con 0x7f7ad40cfec0 2024-04-25T10:03:18.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.470+0000 7f7ad37fe640 1 -- 172.21.15.19:0/3581727626 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8032810 con 0x7f7ad40cfec0 2024-04-25T10:03:18.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.470+0000 7f7adac7c640 1 -- 172.21.15.19:0/3581727626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ad40cfec0 msgr2=0x7f7ad40cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.470+0000 7f7adac7c640 1 --2- 172.21.15.19:0/3581727626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ad40cfec0 0x7f7ad40cde80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80097a0 tx=0x7f7ac802a180 comp rx=0 tx=0).stop 2024-04-25T10:03:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.470+0000 7f7adac7c640 1 -- 172.21.15.19:0/3581727626 shutdown_connections 2024-04-25T10:03:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.470+0000 7f7adac7c640 1 --2- 172.21.15.19:0/3581727626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ad40ce3c0 0x7f7ad406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.470+0000 7f7adac7c640 1 --2- 172.21.15.19:0/3581727626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ad40cfec0 0x7f7ad40cde80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.470+0000 7f7adac7c640 1 -- 172.21.15.19:0/3581727626 >> 172.21.15.19:0/3581727626 conn(0x7f7ad406abe0 msgr2=0x7f7ad406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.470+0000 7f7adac7c640 1 -- 172.21.15.19:0/3581727626 shutdown_connections 2024-04-25T10:03:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.470+0000 7f7adac7c640 1 -- 172.21.15.19:0/3581727626 wait complete. 2024-04-25T10:03:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.471+0000 7f7adac7c640 1 Processor -- start 2024-04-25T10:03:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.471+0000 7f7adac7c640 1 -- start start 2024-04-25T10:03:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.472+0000 7f7adac7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ad40ce3c0 0x7f7ad41f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.472+0000 7f7adac7c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ad40cfec0 0x7f7ad4152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.472+0000 7f7adac7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ad4154ee0 0x7f7ad4155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.472+0000 7f7adac7c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad4071d20 con 0x7f7ad4154ee0 2024-04-25T10:03:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.472+0000 7f7adac7c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad4071230 con 0x7f7ad40ce3c0 2024-04-25T10:03:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.472+0000 7f7adac7c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad41552c0 con 0x7f7ad40cfec0 2024-04-25T10:03:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.472+0000 7f7ad9219640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ad4154ee0 0x7f7ad4155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.472+0000 7f7ad9219640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ad4154ee0 0x7f7ad4155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49610/0 (socket says 172.21.15.19:49610) 2024-04-25T10:03:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.472+0000 7f7ad9219640 1 -- 172.21.15.19:0/3321991696 learned_addr learned my addr 172.21.15.19:0/3321991696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7ad3fff640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ad40cfec0 0x7f7ad4152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7ad8a18640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ad40ce3c0 0x7f7ad41f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7ad9219640 1 -- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ad40cfec0 msgr2=0x7f7ad4152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7ad9219640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ad40cfec0 0x7f7ad4152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7ad9219640 1 -- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ad40ce3c0 msgr2=0x7f7ad41f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7ad9219640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ad40ce3c0 0x7f7ad41f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7ad9219640 1 -- 172.21.15.19:0/3321991696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad41527d0 con 0x7f7ad4154ee0 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7ad9219640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ad4154ee0 0x7f7ad4155a10 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00efc0 tx=0x7f7acc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7ad17fa640 1 -- 172.21.15.19:0/3321991696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc018020 con 0x7f7ad4154ee0 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7ad17fa640 1 -- 172.21.15.19:0/3321991696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc014480 con 0x7f7ad4154ee0 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7adac7c640 1 -- 172.21.15.19:0/3321991696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad41f2230 con 0x7f7ad4154ee0 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7ad17fa640 1 -- 172.21.15.19:0/3321991696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc014870 con 0x7f7ad4154ee0 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.473+0000 7f7adac7c640 1 -- 172.21.15.19:0/3321991696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad41f27f0 con 0x7f7ad4154ee0 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.474+0000 7f7adac7c640 1 -- 172.21.15.19:0/3321991696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad4073990 con 0x7f7ad4154ee0 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.475+0000 7f7ad17fa640 1 -- 172.21.15.19:0/3321991696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7acc005d10 con 0x7f7ad4154ee0 2024-04-25T10:03:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.475+0000 7f7ad17fa640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7ac006b4d0 0x7f7ac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.475+0000 7f7ad17fa640 1 -- 172.21.15.19:0/3321991696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7acc08b050 con 0x7f7ad4154ee0 2024-04-25T10:03:18.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.475+0000 7f7ad8a18640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7ac006b4d0 0x7f7ac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.476+0000 7f7ad8a18640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7ac006b4d0 0x7f7ac006d990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4007fd0 tx=0x7f7ac4007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.479+0000 7f7ad17fa640 1 -- 172.21.15.19:0/3321991696 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7acc022030 con 0x7f7ad4154ee0 2024-04-25T10:03:18.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.655+0000 7f7adac7c640 1 -- 172.21.15.19:0/3321991696 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ad406d360 con 0x7f7ac006b4d0 2024-04-25T10:03:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.661+0000 7f7ad17fa640 1 -- 172.21.15.19:0/3321991696 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7f7ad406d360 con 0x7f7ac006b4d0 2024-04-25T10:03:18.669 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:18.670 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 -- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7ac006b4d0 msgr2=0x7f7ac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7ac006b4d0 0x7f7ac006d990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4007fd0 tx=0x7f7ac4007e80 comp rx=0 tx=0).stop 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 -- 172.21.15.19:0/3321991696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ad4154ee0 msgr2=0x7f7ad4155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ad4154ee0 0x7f7ad4155a10 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00efc0 tx=0x7f7acc00c490 comp rx=0 tx=0).stop 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 -- 172.21.15.19:0/3321991696 shutdown_connections 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7ac006b4d0 0x7f7ac006d990 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ad4154ee0 0x7f7ad4155a10 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ad40cfec0 0x7f7ad4152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 --2- 172.21.15.19:0/3321991696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ad40ce3c0 0x7f7ad41f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 -- 172.21.15.19:0/3321991696 >> 172.21.15.19:0/3321991696 conn(0x7f7ad406abe0 msgr2=0x7f7ad40cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 -- 172.21.15.19:0/3321991696 shutdown_connections 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.664+0000 7f7adac7c640 1 -- 172.21.15.19:0/3321991696 wait complete. 2024-04-25T10:03:18.680 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-04-25T10:03:17.053104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":438980,"kb_used_data":16020,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754535740,"statfs":{"total":773094113280,"available":772644597760,"internally_reserved":0,"allocated":16404480,"data_stored":8290979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.400866"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.661263+0000","last_change":"2024-04-25T10:03:13.661263+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.661263+0000","last_undegraded":"2024-04-25T10:03:13.661263+0000","last_fullsized":"2024-04-25T10:03:13.661263+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.660945+0000","last_change":"2024-04-25T10:03:13.660945+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.660945+0000","last_undegraded":"2024-04-25T10:03:13.660945+0000","last_fullsized":"2024-04-25T10:03:13.660945+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.660032+0000","last_change":"2024-04-25T10:03:13.660032+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.660032+0000","last_undegraded":"2024-04-25T10:03:13.660032+0000","last_fullsized":"2024-04-25T10:03:13.660032+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.659869+0000","last_change":"2024-04-25T10:03:13.659869+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.659869+0000","last_undegraded":"2024-04-25T10:03:13.659869+0000","last_fullsized":"2024-04-25T10:03:13.659869+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.668842+0000","last_change":"2024-04-25T10:03:13.668842+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.668842+0000","last_undegraded":"2024-04-25T10:03:13.668842+0000","last_fullsized":"2024-04-25T10:03:13.668842+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.660476+0000","last_change":"2024-04-25T10:03:13.660476+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.660476+0000","last_undegraded":"2024-04-25T10:03:13.660476+0000","last_fullsized":"2024-04-25T10:03:13.660476+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663869+0000","last_change":"2024-04-25T10:03:13.663869+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663869+0000","last_undegraded":"2024-04-25T10:03:13.663869+0000","last_fullsized":"2024-04-25T10:03:13.663869+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664682+0000","last_change":"2024-04-25T10:03:13.664682+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664682+0000","last_undegraded":"2024-04-25T10:03:13.664682+0000","last_fullsized":"2024-04-25T10:03:13.664682+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665331+0000","last_change":"2024-04-25T10:03:13.665331+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665331+0000","last_undegraded":"2024-04-25T10:03:13.665331+0000","last_fullsized":"2024-04-25T10:03:13.665331+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.667802+0000","last_change":"2024-04-25T10:03:13.667802+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.667802+0000","last_undegraded":"2024-04-25T10:03:13.667802+0000","last_fullsized":"2024-04-25T10:03:13.667802+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666865+0000","last_change":"2024-04-25T10:03:13.666865+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666865+0000","last_undegraded":"2024-04-25T10:03:13.666865+0000","last_fullsized":"2024-04-25T10:03:13.666865+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.661269+0000","last_change":"2024-04-25T10:03:13.661269+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.661269+0000","last_undegraded":"2024-04-25T10:03:13.661269+0000","last_fullsized":"2024-04-25T10:03:13.661269+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666713+0000","last_change":"2024-04-25T10:03:13.666713+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666713+0000","last_undegraded":"2024-04-25T10:03:13.666713+0000","last_fullsized":"2024-04-25T10:03:13.666713+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664154+0000","last_change":"2024-04-25T10:03:13.664154+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664154+0000","last_undegraded":"2024-04-25T10:03:13.664154+0000","last_fullsized":"2024-04-25T10:03:13.664154+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666030+0000","last_change":"2024-04-25T10:03:13.666030+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666030+0000","last_undegraded":"2024-04-25T10:03:13.666030+0000","last_fullsized":"2024-04-25T10:03:13.666030+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.662079+0000","last_change":"2024-04-25T10:03:13.662079+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.662079+0000","last_undegraded":"2024-04-25T10:03:13.662079+0000","last_fullsized":"2024-04-25T10:03:13.662079+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663659+0000","last_change":"2024-04-25T10:03:13.663659+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663659+0000","last_undegraded":"2024-04-25T10:03:13.663659+0000","last_fullsized":"2024-04-25T10:03:13.663659+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664864+0000","last_change":"2024-04-25T10:03:13.664864+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664864+0000","last_undegraded":"2024-04-25T10:03:13.664864+0000","last_fullsized":"2024-04-25T10:03:13.664864+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666492+0000","last_change":"2024-04-25T10:03:13.666492+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666492+0000","last_undegraded":"2024-04-25T10:03:13.666492+0000","last_fullsized":"2024-04-25T10:03:13.666492+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.667065+0000","last_change":"2024-04-25T10:03:13.667065+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.667065+0000","last_undegraded":"2024-04-25T10:03:13.667065+0000","last_fullsized":"2024-04-25T10:03:13.667065+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666293+0000","last_change":"2024-04-25T10:03:13.666293+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666293+0000","last_undegraded":"2024-04-25T10:03:13.666293+0000","last_fullsized":"2024-04-25T10:03:13.666293+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.660053+0000","last_change":"2024-04-25T10:03:13.660053+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.660053+0000","last_undegraded":"2024-04-25T10:03:13.660053+0000","last_fullsized":"2024-04-25T10:03:13.660053+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663657+0000","last_change":"2024-04-25T10:03:13.663657+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663657+0000","last_undegraded":"2024-04-25T10:03:13.663657+0000","last_fullsized":"2024-04-25T10:03:13.663657+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.659711+0000","last_change":"2024-04-25T10:03:13.659711+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.659711+0000","last_undegraded":"2024-04-25T10:03:13.659711+0000","last_fullsized":"2024-04-25T10:03:13.659711+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666990+0000","last_change":"2024-04-25T10:03:13.666990+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666990+0000","last_undegraded":"2024-04-25T10:03:13.666990+0000","last_fullsized":"2024-04-25T10:03:13.666990+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.676405+0000","last_change":"2024-04-25T10:03:13.676405+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.676405+0000","last_undegraded":"2024-04-25T10:03:13.676405+0000","last_fullsized":"2024-04-25T10:03:13.676405+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663037+0000","last_change":"2024-04-25T10:03:13.663037+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663037+0000","last_undegraded":"2024-04-25T10:03:13.663037+0000","last_fullsized":"2024-04-25T10:03:13.663037+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664731+0000","last_change":"2024-04-25T10:03:13.664731+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664731+0000","last_undegraded":"2024-04-25T10:03:13.664731+0000","last_fullsized":"2024-04-25T10:03:13.664731+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.662013+0000","last_change":"2024-04-25T10:03:13.662013+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.662013+0000","last_undegraded":"2024-04-25T10:03:13.662013+0000","last_fullsized":"2024-04-25T10:03:13.662013+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664612+0000","last_change":"2024-04-25T10:03:13.664612+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664612+0000","last_undegraded":"2024-04-25T10:03:13.664612+0000","last_fullsized":"2024-04-25T10:03:13.664612+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665575+0000","last_change":"2024-04-25T10:03:13.665575+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665575+0000","last_undegraded":"2024-04-25T10:03:13.665575+0000","last_fullsized":"2024-04-25T10:03:13.665575+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664441+0000","last_change":"2024-04-25T10:03:13.664441+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664441+0000","last_undegraded":"2024-04-25T10:03:13.664441+0000","last_fullsized":"2024-04-25T10:03:13.664441+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.661572+0000","last_change":"2024-04-25T10:03:13.661572+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.661572+0000","last_undegraded":"2024-04-25T10:03:13.661572+0000","last_fullsized":"2024-04-25T10:03:13.661572+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.675430+0000","last_change":"2024-04-25T10:03:13.674884+0000","last_active":"2024-04-25T10:03:13.675430+0000","last_peered":"2024-04-25T10:03:13.675430+0000","last_clean":"2024-04-25T10:03:13.675430+0000","last_became_active":"2024-04-25T10:03:13.674524+0000","last_became_peered":"2024-04-25T10:03:13.674524+0000","last_unstale":"2024-04-25T10:03:13.675430+0000","last_undegraded":"2024-04-25T10:03:13.675430+0000","last_fullsized":"2024-04-25T10:03:13.675430+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:59.983936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.662893+0000","last_change":"2024-04-25T10:03:13.662893+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.662893+0000","last_undegraded":"2024-04-25T10:03:13.662893+0000","last_fullsized":"2024-04-25T10:03:13.662893+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679971+0000","last_change":"2024-04-25T10:03:13.679019+0000","last_active":"2024-04-25T10:03:13.679971+0000","last_peered":"2024-04-25T10:03:13.679971+0000","last_clean":"2024-04-25T10:03:13.679971+0000","last_became_active":"2024-04-25T10:03:13.678461+0000","last_became_peered":"2024-04-25T10:03:13.678461+0000","last_unstale":"2024-04-25T10:03:13.679971+0000","last_undegraded":"2024-04-25T10:03:13.679971+0000","last_fullsized":"2024-04-25T10:03:13.679971+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:33.172897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663137+0000","last_change":"2024-04-25T10:03:13.663137+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663137+0000","last_undegraded":"2024-04-25T10:03:13.663137+0000","last_fullsized":"2024-04-25T10:03:13.663137+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.675509+0000","last_change":"2024-04-25T10:03:13.674905+0000","last_active":"2024-04-25T10:03:13.675509+0000","last_peered":"2024-04-25T10:03:13.675509+0000","last_clean":"2024-04-25T10:03:13.675509+0000","last_became_active":"2024-04-25T10:03:13.674590+0000","last_became_peered":"2024-04-25T10:03:13.674590+0000","last_unstale":"2024-04-25T10:03:13.675509+0000","last_undegraded":"2024-04-25T10:03:13.675509+0000","last_fullsized":"2024-04-25T10:03:13.675509+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:16.436615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.661145+0000","last_change":"2024-04-25T10:03:13.661145+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.661145+0000","last_undegraded":"2024-04-25T10:03:13.661145+0000","last_fullsized":"2024-04-25T10:03:13.661145+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.677237+0000","last_change":"2024-04-25T10:03:13.673989+0000","last_active":"2024-04-25T10:03:13.677237+0000","last_peered":"2024-04-25T10:03:13.677237+0000","last_clean":"2024-04-25T10:03:13.677237+0000","last_became_active":"2024-04-25T10:03:13.673670+0000","last_became_peered":"2024-04-25T10:03:13.673670+0000","last_unstale":"2024-04-25T10:03:13.677237+0000","last_undegraded":"2024-04-25T10:03:13.677237+0000","last_fullsized":"2024-04-25T10:03:13.677237+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:29.504127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664563+0000","last_change":"2024-04-25T10:03:13.664563+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664563+0000","last_undegraded":"2024-04-25T10:03:13.664563+0000","last_fullsized":"2024-04-25T10:03:13.664563+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.680052+0000","last_change":"2024-04-25T10:03:13.677056+0000","last_active":"2024-04-25T10:03:13.680052+0000","last_peered":"2024-04-25T10:03:13.680052+0000","last_clean":"2024-04-25T10:03:13.680052+0000","last_became_active":"2024-04-25T10:03:13.676751+0000","last_became_peered":"2024-04-25T10:03:13.676751+0000","last_unstale":"2024-04-25T10:03:13.680052+0000","last_undegraded":"2024-04-25T10:03:13.680052+0000","last_fullsized":"2024-04-25T10:03:13.680052+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:25.428507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.668039+0000","last_change":"2024-04-25T10:03:13.668039+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.668039+0000","last_undegraded":"2024-04-25T10:03:13.668039+0000","last_fullsized":"2024-04-25T10:03:13.668039+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679611+0000","last_change":"2024-04-25T10:03:13.674339+0000","last_active":"2024-04-25T10:03:13.679611+0000","last_peered":"2024-04-25T10:03:13.679611+0000","last_clean":"2024-04-25T10:03:13.679611+0000","last_became_active":"2024-04-25T10:03:13.674074+0000","last_became_peered":"2024-04-25T10:03:13.674074+0000","last_unstale":"2024-04-25T10:03:13.679611+0000","last_undegraded":"2024-04-25T10:03:13.679611+0000","last_fullsized":"2024-04-25T10:03:13.679611+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:13.450850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.673268+0000","last_change":"2024-04-25T10:03:13.673268+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.673268+0000","last_undegraded":"2024-04-25T10:03:13.673268+0000","last_fullsized":"2024-04-25T10:03:13.673268+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.682495+0000","last_change":"2024-04-25T10:03:13.681030+0000","last_active":"2024-04-25T10:03:13.682495+0000","last_peered":"2024-04-25T10:03:13.682495+0000","last_clean":"2024-04-25T10:03:13.682495+0000","last_became_active":"2024-04-25T10:03:13.680669+0000","last_became_peered":"2024-04-25T10:03:13.680669+0000","last_unstale":"2024-04-25T10:03:13.682495+0000","last_undegraded":"2024-04-25T10:03:13.682495+0000","last_fullsized":"2024-04-25T10:03:13.682495+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:04.380494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.660188+0000","last_change":"2024-04-25T10:03:13.660188+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.660188+0000","last_undegraded":"2024-04-25T10:03:13.660188+0000","last_fullsized":"2024-04-25T10:03:13.660188+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679388+0000","last_change":"2024-04-25T10:03:13.670929+0000","last_active":"2024-04-25T10:03:13.679388+0000","last_peered":"2024-04-25T10:03:13.679388+0000","last_clean":"2024-04-25T10:03:13.679388+0000","last_became_active":"2024-04-25T10:03:13.670647+0000","last_became_peered":"2024-04-25T10:03:13.670647+0000","last_unstale":"2024-04-25T10:03:13.679388+0000","last_undegraded":"2024-04-25T10:03:13.679388+0000","last_fullsized":"2024-04-25T10:03:13.679388+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:10.022627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.662063+0000","last_change":"2024-04-25T10:03:13.662063+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.662063+0000","last_undegraded":"2024-04-25T10:03:13.662063+0000","last_fullsized":"2024-04-25T10:03:13.662063+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.681610+0000","last_change":"2024-04-25T10:03:13.680361+0000","last_active":"2024-04-25T10:03:13.681610+0000","last_peered":"2024-04-25T10:03:13.681610+0000","last_clean":"2024-04-25T10:03:13.681610+0000","last_became_active":"2024-04-25T10:03:13.680075+0000","last_became_peered":"2024-04-25T10:03:13.680075+0000","last_unstale":"2024-04-25T10:03:13.681610+0000","last_undegraded":"2024-04-25T10:03:13.681610+0000","last_fullsized":"2024-04-25T10:03:13.681610+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:03.435847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664441+0000","last_change":"2024-04-25T10:03:13.664441+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664441+0000","last_undegraded":"2024-04-25T10:03:13.664441+0000","last_fullsized":"2024-04-25T10:03:13.664441+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.671673+0000","last_change":"2024-04-25T10:03:13.669655+0000","last_active":"2024-04-25T10:03:13.671673+0000","last_peered":"2024-04-25T10:03:13.671673+0000","last_clean":"2024-04-25T10:03:13.671673+0000","last_became_active":"2024-04-25T10:03:13.669366+0000","last_became_peered":"2024-04-25T10:03:13.669366+0000","last_unstale":"2024-04-25T10:03:13.671673+0000","last_undegraded":"2024-04-25T10:03:13.671673+0000","last_fullsized":"2024-04-25T10:03:13.671673+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:19.871768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665865+0000","last_change":"2024-04-25T10:03:13.665865+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665865+0000","last_undegraded":"2024-04-25T10:03:13.665865+0000","last_fullsized":"2024-04-25T10:03:13.665865+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.682150+0000","last_change":"2024-04-25T10:03:13.681202+0000","last_active":"2024-04-25T10:03:13.682150+0000","last_peered":"2024-04-25T10:03:13.682150+0000","last_clean":"2024-04-25T10:03:13.682150+0000","last_became_active":"2024-04-25T10:03:13.680736+0000","last_became_peered":"2024-04-25T10:03:13.680736+0000","last_unstale":"2024-04-25T10:03:13.682150+0000","last_undegraded":"2024-04-25T10:03:13.682150+0000","last_fullsized":"2024-04-25T10:03:13.682150+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:31.006719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664258+0000","last_change":"2024-04-25T10:03:13.664258+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664258+0000","last_undegraded":"2024-04-25T10:03:13.664258+0000","last_fullsized":"2024-04-25T10:03:13.664258+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679919+0000","last_change":"2024-04-25T10:03:13.671104+0000","last_active":"2024-04-25T10:03:13.679919+0000","last_peered":"2024-04-25T10:03:13.679919+0000","last_clean":"2024-04-25T10:03:13.679919+0000","last_became_active":"2024-04-25T10:03:13.670810+0000","last_became_peered":"2024-04-25T10:03:13.670810+0000","last_unstale":"2024-04-25T10:03:13.679919+0000","last_undegraded":"2024-04-25T10:03:13.679919+0000","last_fullsized":"2024-04-25T10:03:13.679919+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:11.244494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665310+0000","last_change":"2024-04-25T10:03:13.665310+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665310+0000","last_undegraded":"2024-04-25T10:03:13.665310+0000","last_fullsized":"2024-04-25T10:03:13.665310+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.674265+0000","last_change":"2024-04-25T10:03:13.672839+0000","last_active":"2024-04-25T10:03:13.674265+0000","last_peered":"2024-04-25T10:03:13.674265+0000","last_clean":"2024-04-25T10:03:13.674265+0000","last_became_active":"2024-04-25T10:03:13.671105+0000","last_became_peered":"2024-04-25T10:03:13.671105+0000","last_unstale":"2024-04-25T10:03:13.674265+0000","last_undegraded":"2024-04-25T10:03:13.674265+0000","last_fullsized":"2024-04-25T10:03:13.674265+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:51.239805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663836+0000","last_change":"2024-04-25T10:03:13.663836+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663836+0000","last_undegraded":"2024-04-25T10:03:13.663836+0000","last_fullsized":"2024-04-25T10:03:13.663836+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.680896+0000","last_change":"2024-04-25T10:03:13.680518+0000","last_active":"2024-04-25T10:03:13.680896+0000","last_peered":"2024-04-25T10:03:13.680896+0000","last_clean":"2024-04-25T10:03:13.680896+0000","last_became_active":"2024-04-25T10:03:13.680260+0000","last_became_peered":"2024-04-25T10:03:13.680260+0000","last_unstale":"2024-04-25T10:03:13.680896+0000","last_undegraded":"2024-04-25T10:03:13.680896+0000","last_fullsized":"2024-04-25T10:03:13.680896+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:51.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666559+0000","last_change":"2024-04-25T10:03:13.666559+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666559+0000","last_undegraded":"2024-04-25T10:03:13.666559+0000","last_fullsized":"2024-04-25T10:03:13.666559+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679409+0000","last_change":"2024-04-25T10:03:13.675759+0000","last_active":"2024-04-25T10:03:13.679409+0000","last_peered":"2024-04-25T10:03:13.679409+0000","last_clean":"2024-04-25T10:03:13.679409+0000","last_became_active":"2024-04-25T10:03:13.675488+0000","last_became_peered":"2024-04-25T10:03:13.675488+0000","last_unstale":"2024-04-25T10:03:13.679409+0000","last_undegraded":"2024-04-25T10:03:13.679409+0000","last_fullsized":"2024-04-25T10:03:13.679409+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:58.961175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663840+0000","last_change":"2024-04-25T10:03:13.663840+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663840+0000","last_undegraded":"2024-04-25T10:03:13.663840+0000","last_fullsized":"2024-04-25T10:03:13.663840+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.676273+0000","last_change":"2024-04-25T10:03:13.673662+0000","last_active":"2024-04-25T10:03:13.676273+0000","last_peered":"2024-04-25T10:03:13.676273+0000","last_clean":"2024-04-25T10:03:13.676273+0000","last_became_active":"2024-04-25T10:03:13.673355+0000","last_became_peered":"2024-04-25T10:03:13.673355+0000","last_unstale":"2024-04-25T10:03:13.676273+0000","last_undegraded":"2024-04-25T10:03:13.676273+0000","last_fullsized":"2024-04-25T10:03:13.676273+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:51.423687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665866+0000","last_change":"2024-04-25T10:03:13.665866+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665866+0000","last_undegraded":"2024-04-25T10:03:13.665866+0000","last_fullsized":"2024-04-25T10:03:13.665866+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679969+0000","last_change":"2024-04-25T10:03:13.678163+0000","last_active":"2024-04-25T10:03:13.679969+0000","last_peered":"2024-04-25T10:03:13.679969+0000","last_clean":"2024-04-25T10:03:13.679969+0000","last_became_active":"2024-04-25T10:03:13.677884+0000","last_became_peered":"2024-04-25T10:03:13.677884+0000","last_unstale":"2024-04-25T10:03:13.679969+0000","last_undegraded":"2024-04-25T10:03:13.679969+0000","last_fullsized":"2024-04-25T10:03:13.679969+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:04.430596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664148+0000","last_change":"2024-04-25T10:03:13.664148+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664148+0000","last_undegraded":"2024-04-25T10:03:13.664148+0000","last_fullsized":"2024-04-25T10:03:13.664148+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.672242+0000","last_change":"2024-04-25T10:03:13.667893+0000","last_active":"2024-04-25T10:03:13.672242+0000","last_peered":"2024-04-25T10:03:13.672242+0000","last_clean":"2024-04-25T10:03:13.672242+0000","last_became_active":"2024-04-25T10:03:13.667483+0000","last_became_peered":"2024-04-25T10:03:13.667483+0000","last_unstale":"2024-04-25T10:03:13.672242+0000","last_undegraded":"2024-04-25T10:03:13.672242+0000","last_fullsized":"2024-04-25T10:03:13.672242+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.341450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665173+0000","last_change":"2024-04-25T10:03:13.665173+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665173+0000","last_undegraded":"2024-04-25T10:03:13.665173+0000","last_fullsized":"2024-04-25T10:03:13.665173+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.680068+0000","last_change":"2024-04-25T10:03:13.678499+0000","last_active":"2024-04-25T10:03:13.680068+0000","last_peered":"2024-04-25T10:03:13.680068+0000","last_clean":"2024-04-25T10:03:13.680068+0000","last_became_active":"2024-04-25T10:03:13.678219+0000","last_became_peered":"2024-04-25T10:03:13.678219+0000","last_unstale":"2024-04-25T10:03:13.680068+0000","last_undegraded":"2024-04-25T10:03:13.680068+0000","last_fullsized":"2024-04-25T10:03:13.680068+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:24.928561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664289+0000","last_change":"2024-04-25T10:03:13.664289+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664289+0000","last_undegraded":"2024-04-25T10:03:13.664289+0000","last_fullsized":"2024-04-25T10:03:13.664289+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679898+0000","last_change":"2024-04-25T10:03:13.670893+0000","last_active":"2024-04-25T10:03:13.679898+0000","last_peered":"2024-04-25T10:03:13.679898+0000","last_clean":"2024-04-25T10:03:13.679898+0000","last_became_active":"2024-04-25T10:03:13.670599+0000","last_became_peered":"2024-04-25T10:03:13.670599+0000","last_unstale":"2024-04-25T10:03:13.679898+0000","last_undegraded":"2024-04-25T10:03:13.679898+0000","last_fullsized":"2024-04-25T10:03:13.679898+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:35.799225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665101+0000","last_change":"2024-04-25T10:03:13.665101+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665101+0000","last_undegraded":"2024-04-25T10:03:13.665101+0000","last_fullsized":"2024-04-25T10:03:13.665101+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.677138+0000","last_change":"2024-04-25T10:03:13.676633+0000","last_active":"2024-04-25T10:03:13.677138+0000","last_peered":"2024-04-25T10:03:13.677138+0000","last_clean":"2024-04-25T10:03:13.677138+0000","last_became_active":"2024-04-25T10:03:13.676353+0000","last_became_peered":"2024-04-25T10:03:13.676353+0000","last_unstale":"2024-04-25T10:03:13.677138+0000","last_undegraded":"2024-04-25T10:03:13.677138+0000","last_fullsized":"2024-04-25T10:03:13.677138+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:00.968592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.662544+0000","last_change":"2024-04-25T10:03:13.662544+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.662544+0000","last_undegraded":"2024-04-25T10:03:13.662544+0000","last_fullsized":"2024-04-25T10:03:13.662544+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.677253+0000","last_change":"2024-04-25T10:03:13.673943+0000","last_active":"2024-04-25T10:03:13.677253+0000","last_peered":"2024-04-25T10:03:13.677253+0000","last_clean":"2024-04-25T10:03:13.677253+0000","last_became_active":"2024-04-25T10:03:13.673646+0000","last_became_peered":"2024-04-25T10:03:13.673646+0000","last_unstale":"2024-04-25T10:03:13.677253+0000","last_undegraded":"2024-04-25T10:03:13.677253+0000","last_fullsized":"2024-04-25T10:03:13.677253+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:50.254439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.659603+0000","last_change":"2024-04-25T10:03:13.659603+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.659603+0000","last_undegraded":"2024-04-25T10:03:13.659603+0000","last_fullsized":"2024-04-25T10:03:13.659603+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.672425+0000","last_change":"2024-04-25T10:03:13.671428+0000","last_active":"2024-04-25T10:03:13.672425+0000","last_peered":"2024-04-25T10:03:13.672425+0000","last_clean":"2024-04-25T10:03:13.672425+0000","last_became_active":"2024-04-25T10:03:13.671080+0000","last_became_peered":"2024-04-25T10:03:13.671080+0000","last_unstale":"2024-04-25T10:03:13.672425+0000","last_undegraded":"2024-04-25T10:03:13.672425+0000","last_fullsized":"2024-04-25T10:03:13.672425+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:33.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.667127+0000","last_change":"2024-04-25T10:03:13.667127+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.667127+0000","last_undegraded":"2024-04-25T10:03:13.667127+0000","last_fullsized":"2024-04-25T10:03:13.667127+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679834+0000","last_change":"2024-04-25T10:03:13.669724+0000","last_active":"2024-04-25T10:03:13.679834+0000","last_peered":"2024-04-25T10:03:13.679834+0000","last_clean":"2024-04-25T10:03:13.679834+0000","last_became_active":"2024-04-25T10:03:13.669446+0000","last_became_peered":"2024-04-25T10:03:13.669446+0000","last_unstale":"2024-04-25T10:03:13.679834+0000","last_undegraded":"2024-04-25T10:03:13.679834+0000","last_fullsized":"2024-04-25T10:03:13.679834+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:22.924857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663231+0000","last_change":"2024-04-25T10:03:13.663231+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663231+0000","last_undegraded":"2024-04-25T10:03:13.663231+0000","last_fullsized":"2024-04-25T10:03:13.663231+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.682132+0000","last_change":"2024-04-25T10:03:13.681265+0000","last_active":"2024-04-25T10:03:13.682132+0000","last_peered":"2024-04-25T10:03:13.682132+0000","last_clean":"2024-04-25T10:03:13.682132+0000","last_became_active":"2024-04-25T10:03:13.680824+0000","last_became_peered":"2024-04-25T10:03:13.680824+0000","last_unstale":"2024-04-25T10:03:13.682132+0000","last_undegraded":"2024-04-25T10:03:13.682132+0000","last_fullsized":"2024-04-25T10:03:13.682132+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:38.503607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.668201+0000","last_change":"2024-04-25T10:03:13.668201+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.668201+0000","last_undegraded":"2024-04-25T10:03:13.668201+0000","last_fullsized":"2024-04-25T10:03:13.668201+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.673387+0000","last_change":"2024-04-25T10:03:13.670694+0000","last_active":"2024-04-25T10:03:13.673387+0000","last_peered":"2024-04-25T10:03:13.673387+0000","last_clean":"2024-04-25T10:03:13.673387+0000","last_became_active":"2024-04-25T10:03:13.670408+0000","last_became_peered":"2024-04-25T10:03:13.670408+0000","last_unstale":"2024-04-25T10:03:13.673387+0000","last_undegraded":"2024-04-25T10:03:13.673387+0000","last_fullsized":"2024-04-25T10:03:13.673387+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:33.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664171+0000","last_change":"2024-04-25T10:03:13.664171+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664171+0000","last_undegraded":"2024-04-25T10:03:13.664171+0000","last_fullsized":"2024-04-25T10:03:13.664171+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.682344+0000","last_change":"2024-04-25T10:03:13.679719+0000","last_active":"2024-04-25T10:03:13.682344+0000","last_peered":"2024-04-25T10:03:13.682344+0000","last_clean":"2024-04-25T10:03:13.682344+0000","last_became_active":"2024-04-25T10:03:13.679453+0000","last_became_peered":"2024-04-25T10:03:13.679453+0000","last_unstale":"2024-04-25T10:03:13.682344+0000","last_undegraded":"2024-04-25T10:03:13.682344+0000","last_fullsized":"2024-04-25T10:03:13.682344+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:16.308823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.659279+0000","last_change":"2024-04-25T10:03:13.659279+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.659279+0000","last_undegraded":"2024-04-25T10:03:13.659279+0000","last_fullsized":"2024-04-25T10:03:13.659279+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.672296+0000","last_change":"2024-04-25T10:03:13.670937+0000","last_active":"2024-04-25T10:03:13.672296+0000","last_peered":"2024-04-25T10:03:13.672296+0000","last_clean":"2024-04-25T10:03:13.672296+0000","last_became_active":"2024-04-25T10:03:13.670561+0000","last_became_peered":"2024-04-25T10:03:13.670561+0000","last_unstale":"2024-04-25T10:03:13.672296+0000","last_undegraded":"2024-04-25T10:03:13.672296+0000","last_fullsized":"2024-04-25T10:03:13.672296+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:27.872809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.661551+0000","last_change":"2024-04-25T10:03:13.661551+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.661551+0000","last_undegraded":"2024-04-25T10:03:13.661551+0000","last_fullsized":"2024-04-25T10:03:13.661551+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.674244+0000","last_change":"2024-04-25T10:03:13.671727+0000","last_active":"2024-04-25T10:03:13.674244+0000","last_peered":"2024-04-25T10:03:13.674244+0000","last_clean":"2024-04-25T10:03:13.674244+0000","last_became_active":"2024-04-25T10:03:13.671439+0000","last_became_peered":"2024-04-25T10:03:13.671439+0000","last_unstale":"2024-04-25T10:03:13.674244+0000","last_undegraded":"2024-04-25T10:03:13.674244+0000","last_fullsized":"2024-04-25T10:03:13.674244+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:53.770956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664094+0000","last_change":"2024-04-25T10:03:13.664094+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664094+0000","last_undegraded":"2024-04-25T10:03:13.664094+0000","last_fullsized":"2024-04-25T10:03:13.664094+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.675461+0000","last_change":"2024-04-25T10:03:13.674966+0000","last_active":"2024-04-25T10:03:13.675461+0000","last_peered":"2024-04-25T10:03:13.675461+0000","last_clean":"2024-04-25T10:03:13.675461+0000","last_became_active":"2024-04-25T10:03:13.674633+0000","last_became_peered":"2024-04-25T10:03:13.674633+0000","last_unstale":"2024-04-25T10:03:13.675461+0000","last_undegraded":"2024-04-25T10:03:13.675461+0000","last_fullsized":"2024-04-25T10:03:13.675461+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:05.106689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666020+0000","last_change":"2024-04-25T10:03:13.666020+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666020+0000","last_undegraded":"2024-04-25T10:03:13.666020+0000","last_fullsized":"2024-04-25T10:03:13.666020+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679869+0000","last_change":"2024-04-25T10:03:13.670435+0000","last_active":"2024-04-25T10:03:13.679869+0000","last_peered":"2024-04-25T10:03:13.679869+0000","last_clean":"2024-04-25T10:03:13.679869+0000","last_became_active":"2024-04-25T10:03:13.670146+0000","last_became_peered":"2024-04-25T10:03:13.670146+0000","last_unstale":"2024-04-25T10:03:13.679869+0000","last_undegraded":"2024-04-25T10:03:13.679869+0000","last_fullsized":"2024-04-25T10:03:13.679869+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:34.731705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.659572+0000","last_change":"2024-04-25T10:03:13.659572+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.659572+0000","last_undegraded":"2024-04-25T10:03:13.659572+0000","last_fullsized":"2024-04-25T10:03:13.659572+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.682440+0000","last_change":"2024-04-25T10:03:13.681369+0000","last_active":"2024-04-25T10:03:13.682440+0000","last_peered":"2024-04-25T10:03:13.682440+0000","last_clean":"2024-04-25T10:03:13.682440+0000","last_became_active":"2024-04-25T10:03:13.681075+0000","last_became_peered":"2024-04-25T10:03:13.681075+0000","last_unstale":"2024-04-25T10:03:13.682440+0000","last_undegraded":"2024-04-25T10:03:13.682440+0000","last_fullsized":"2024-04-25T10:03:13.682440+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:31.792118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":439,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.668539+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:03:13.668539+0000","last_peered":"2024-04-25T10:03:13.668539+0000","last_clean":"2024-04-25T10:03:13.668539+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:03:13.668539+0000","last_undegraded":"2024-04-25T10:03:13.668539+0000","last_fullsized":"2024-04-25T10:03:13.668539+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51664,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320176,"statfs":{"total":96636764160,"available":96583860224,"internally_reserved":0,"allocated":2494464,"data_stored":1478316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51084,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320756,"statfs":{"total":96636764160,"available":96584454144,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59404,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312436,"statfs":{"total":96636764160,"available":96575934464,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59404,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312436,"statfs":{"total":96636764160,"available":96575934464,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55248,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316592,"statfs":{"total":96636764160,"available":96580190208,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55244,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316596,"statfs":{"total":96636764160,"available":96580194304,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55840,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316000,"statfs":{"total":96636764160,"available":96579584000,"internally_reserved":0,"allocated":2506752,"data_stored":1484975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:18.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:18.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.817+0000 7eff33fa5640 1 Processor -- start 2024-04-25T10:03:18.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.817+0000 7eff33fa5640 1 -- start start 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff33fa5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7eff2c0d2e90 0x7eff2c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff33fa5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7eff2c0d0900 0x7eff2c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff33fa5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2c0b9940 con 0x7eff2c15dd30 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff33fa5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2c0d0d40 con 0x7eff2c0d0900 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff33fa5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2c0d1df0 con 0x7eff2c0d2e90 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff32542640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7eff2c0d0900 0x7eff2c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff31d41640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7eff2c15dd30 0x7eff2c0b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46126/0 (socket says 172.21.15.19:46126) 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff31d41640 1 -- 172.21.15.19:0/1443669953 learned_addr learned my addr 172.21.15.19:0/1443669953 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff31540640 1 --2- 172.21.15.19:0/1443669953 >> v2:172.21.15.112:3300/0 conn(0x7eff2c0d2e90 0x7eff2c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff32542640 1 -- 172.21.15.19:0/1443669953 >> v2:172.21.15.112:3300/0 conn(0x7eff2c0d2e90 msgr2=0x7eff2c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff32542640 1 --2- 172.21.15.19:0/1443669953 >> v2:172.21.15.112:3300/0 conn(0x7eff2c0d2e90 0x7eff2c0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff32542640 1 -- 172.21.15.19:0/1443669953 >> v1:172.21.15.19:6789/0 conn(0x7eff2c15dd30 legacy=0x7eff2c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff30d3f640 1 -- 172.21.15.19:0/1443669953 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905704280 0 0) 0x7eff2c0b9940 con 0x7eff2c15dd30 2024-04-25T10:03:18.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff32542640 1 -- 172.21.15.19:0/1443669953 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0d20d0 con 0x7eff2c0d0900 2024-04-25T10:03:18.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.818+0000 7eff31540640 1 --2- 172.21.15.19:0/1443669953 >> v2:172.21.15.112:3300/0 conn(0x7eff2c0d2e90 0x7eff2c0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:18.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.819+0000 7eff32542640 1 --2- 172.21.15.19:0/1443669953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff2c0d0900 0x7eff2c0b6a40 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0b6540 tx=0x7eff2402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85b69cbb1931554 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.819+0000 7eff30d3f640 1 -- 172.21.15.19:0/1443669953 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24007740 con 0x7eff2c0d0900 2024-04-25T10:03:18.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.819+0000 7eff30d3f640 1 -- 172.21.15.19:0/1443669953 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff240078e0 con 0x7eff2c0d0900 2024-04-25T10:03:18.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.819+0000 7eff30d3f640 1 -- 172.21.15.19:0/1443669953 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24007cb0 con 0x7eff2c0d0900 2024-04-25T10:03:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.820+0000 7eff33fa5640 1 -- 172.21.15.19:0/1443669953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff2c0d0900 msgr2=0x7eff2c0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.820+0000 7eff33fa5640 1 --2- 172.21.15.19:0/1443669953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff2c0d0900 0x7eff2c0b6a40 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0b6540 tx=0x7eff2402a560 comp rx=0 tx=0).stop 2024-04-25T10:03:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.820+0000 7eff33fa5640 1 -- 172.21.15.19:0/1443669953 shutdown_connections 2024-04-25T10:03:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.820+0000 7eff33fa5640 1 --2- 172.21.15.19:0/1443669953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff2c0d0900 0x7eff2c0b6a40 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.820+0000 7eff33fa5640 1 --2- 172.21.15.19:0/1443669953 >> v2:172.21.15.112:3300/0 conn(0x7eff2c0d2e90 0x7eff2c0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.820+0000 7eff33fa5640 1 -- 172.21.15.19:0/1443669953 >> 172.21.15.19:0/1443669953 conn(0x7eff2c061c30 msgr2=0x7eff2c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.820+0000 7eff33fa5640 1 -- 172.21.15.19:0/1443669953 shutdown_connections 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.820+0000 7eff33fa5640 1 -- 172.21.15.19:0/1443669953 wait complete. 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.821+0000 7eff33fa5640 1 Processor -- start 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.821+0000 7eff33fa5640 1 -- start start 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff33fa5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c0d0900 0x7eff2c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff33fa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff2c0d2e90 0x7eff2c0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff33fa5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff2c15dd30 0x7eff2c0b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff33fa5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0d2970 con 0x7eff2c0d2e90 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff33fa5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0d2690 con 0x7eff2c0d0900 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff33fa5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0d1a10 con 0x7eff2c15dd30 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff31540640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff2c0d2e90 0x7eff2c0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff31540640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff2c0d2e90 0x7eff2c0b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49630/0 (socket says 172.21.15.19:49630) 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff31540640 1 -- 172.21.15.19:0/1483087050 learned_addr learned my addr 172.21.15.19:0/1483087050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff32542640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff2c15dd30 0x7eff2c0b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff31d41640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c0d0900 0x7eff2c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff31540640 1 -- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff2c15dd30 msgr2=0x7eff2c0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff31540640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff2c15dd30 0x7eff2c0b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff31540640 1 -- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c0d0900 msgr2=0x7eff2c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff31540640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c0d0900 0x7eff2c0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff31540640 1 -- 172.21.15.19:0/1483087050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0b4ad0 con 0x7eff2c0d2e90 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff31d41640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c0d0900 0x7eff2c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff32542640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff2c15dd30 0x7eff2c0b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff31540640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff2c0d2e90 0x7eff2c0b53f0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00c7f0 tx=0x7eff1c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff227fc640 1 -- 172.21.15.19:0/1483087050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c005b90 con 0x7eff2c0d2e90 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.822+0000 7eff227fc640 1 -- 172.21.15.19:0/1483087050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c005d30 con 0x7eff2c0d2e90 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.823+0000 7eff33fa5640 1 -- 172.21.15.19:0/1483087050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c0b3a50 con 0x7eff2c0d2e90 2024-04-25T10:03:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.823+0000 7eff227fc640 1 -- 172.21.15.19:0/1483087050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c007a30 con 0x7eff2c0d2e90 2024-04-25T10:03:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.823+0000 7eff33fa5640 1 -- 172.21.15.19:0/1483087050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c0b4010 con 0x7eff2c0d2e90 2024-04-25T10:03:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.823+0000 7eff33fa5640 1 -- 172.21.15.19:0/1483087050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c073990 con 0x7eff2c0d2e90 2024-04-25T10:03:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.824+0000 7eff227fc640 1 -- 172.21.15.19:0/1483087050 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff1c007bd0 con 0x7eff2c0d2e90 2024-04-25T10:03:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.824+0000 7eff227fc640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff1006b4d0 0x7eff1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.824+0000 7eff227fc640 1 -- 172.21.15.19:0/1483087050 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7eff1c08a520 con 0x7eff2c0d2e90 2024-04-25T10:03:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.825+0000 7eff31d41640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff1006b4d0 0x7eff1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.825+0000 7eff31d41640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff1006b4d0 0x7eff1006d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7eff28000f60 tx=0x7eff28007ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:18.828+0000 7eff227fc640 1 -- 172.21.15.19:0/1483087050 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff1c015020 con 0x7eff2c0d2e90 2024-04-25T10:03:19.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.005+0000 7eff33fa5640 1 -- 172.21.15.19:0/1483087050 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff2c0b74b0 con 0x7eff1006b4d0 2024-04-25T10:03:19.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.012+0000 7eff227fc640 1 -- 172.21.15.19:0/1483087050 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7eff2c0b74b0 con 0x7eff1006b4d0 2024-04-25T10:03:19.019 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:03:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.014+0000 7eff33fa5640 1 -- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff1006b4d0 msgr2=0x7eff1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.014+0000 7eff33fa5640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff1006b4d0 0x7eff1006d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7eff28000f60 tx=0x7eff28007ca0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.014+0000 7eff33fa5640 1 -- 172.21.15.19:0/1483087050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff2c0d2e90 msgr2=0x7eff2c0b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.014+0000 7eff33fa5640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff2c0d2e90 0x7eff2c0b53f0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00c7f0 tx=0x7eff1c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.014+0000 7eff33fa5640 1 -- 172.21.15.19:0/1483087050 shutdown_connections 2024-04-25T10:03:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.014+0000 7eff33fa5640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff1006b4d0 0x7eff1006d990 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.014+0000 7eff33fa5640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff2c15dd30 0x7eff2c0b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.014+0000 7eff33fa5640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff2c0d2e90 0x7eff2c0b53f0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.014+0000 7eff33fa5640 1 --2- 172.21.15.19:0/1483087050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff2c0d0900 0x7eff2c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.014+0000 7eff33fa5640 1 -- 172.21.15.19:0/1483087050 >> 172.21.15.19:0/1483087050 conn(0x7eff2c061c30 msgr2=0x7eff2c160120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.015+0000 7eff33fa5640 1 -- 172.21.15.19:0/1483087050 shutdown_connections 2024-04-25T10:03:19.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.015+0000 7eff33fa5640 1 -- 172.21.15.19:0/1483087050 wait complete. 2024-04-25T10:03:19.031 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-04-25T10:03:17.053104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":438980,"kb_used_data":16020,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754535740,"statfs":{"total":773094113280,"available":772644597760,"internally_reserved":0,"allocated":16404480,"data_stored":8290979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.400866"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.661263+0000","last_change":"2024-04-25T10:03:13.661263+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.661263+0000","last_undegraded":"2024-04-25T10:03:13.661263+0000","last_fullsized":"2024-04-25T10:03:13.661263+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.660945+0000","last_change":"2024-04-25T10:03:13.660945+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.660945+0000","last_undegraded":"2024-04-25T10:03:13.660945+0000","last_fullsized":"2024-04-25T10:03:13.660945+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.660032+0000","last_change":"2024-04-25T10:03:13.660032+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.660032+0000","last_undegraded":"2024-04-25T10:03:13.660032+0000","last_fullsized":"2024-04-25T10:03:13.660032+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.659869+0000","last_change":"2024-04-25T10:03:13.659869+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.659869+0000","last_undegraded":"2024-04-25T10:03:13.659869+0000","last_fullsized":"2024-04-25T10:03:13.659869+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.668842+0000","last_change":"2024-04-25T10:03:13.668842+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.668842+0000","last_undegraded":"2024-04-25T10:03:13.668842+0000","last_fullsized":"2024-04-25T10:03:13.668842+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.660476+0000","last_change":"2024-04-25T10:03:13.660476+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.660476+0000","last_undegraded":"2024-04-25T10:03:13.660476+0000","last_fullsized":"2024-04-25T10:03:13.660476+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663869+0000","last_change":"2024-04-25T10:03:13.663869+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663869+0000","last_undegraded":"2024-04-25T10:03:13.663869+0000","last_fullsized":"2024-04-25T10:03:13.663869+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664682+0000","last_change":"2024-04-25T10:03:13.664682+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664682+0000","last_undegraded":"2024-04-25T10:03:13.664682+0000","last_fullsized":"2024-04-25T10:03:13.664682+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665331+0000","last_change":"2024-04-25T10:03:13.665331+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665331+0000","last_undegraded":"2024-04-25T10:03:13.665331+0000","last_fullsized":"2024-04-25T10:03:13.665331+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.667802+0000","last_change":"2024-04-25T10:03:13.667802+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.667802+0000","last_undegraded":"2024-04-25T10:03:13.667802+0000","last_fullsized":"2024-04-25T10:03:13.667802+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666865+0000","last_change":"2024-04-25T10:03:13.666865+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666865+0000","last_undegraded":"2024-04-25T10:03:13.666865+0000","last_fullsized":"2024-04-25T10:03:13.666865+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.661269+0000","last_change":"2024-04-25T10:03:13.661269+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.661269+0000","last_undegraded":"2024-04-25T10:03:13.661269+0000","last_fullsized":"2024-04-25T10:03:13.661269+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666713+0000","last_change":"2024-04-25T10:03:13.666713+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666713+0000","last_undegraded":"2024-04-25T10:03:13.666713+0000","last_fullsized":"2024-04-25T10:03:13.666713+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664154+0000","last_change":"2024-04-25T10:03:13.664154+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664154+0000","last_undegraded":"2024-04-25T10:03:13.664154+0000","last_fullsized":"2024-04-25T10:03:13.664154+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666030+0000","last_change":"2024-04-25T10:03:13.666030+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666030+0000","last_undegraded":"2024-04-25T10:03:13.666030+0000","last_fullsized":"2024-04-25T10:03:13.666030+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.662079+0000","last_change":"2024-04-25T10:03:13.662079+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.662079+0000","last_undegraded":"2024-04-25T10:03:13.662079+0000","last_fullsized":"2024-04-25T10:03:13.662079+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663659+0000","last_change":"2024-04-25T10:03:13.663659+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663659+0000","last_undegraded":"2024-04-25T10:03:13.663659+0000","last_fullsized":"2024-04-25T10:03:13.663659+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664864+0000","last_change":"2024-04-25T10:03:13.664864+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664864+0000","last_undegraded":"2024-04-25T10:03:13.664864+0000","last_fullsized":"2024-04-25T10:03:13.664864+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666492+0000","last_change":"2024-04-25T10:03:13.666492+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666492+0000","last_undegraded":"2024-04-25T10:03:13.666492+0000","last_fullsized":"2024-04-25T10:03:13.666492+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.667065+0000","last_change":"2024-04-25T10:03:13.667065+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.667065+0000","last_undegraded":"2024-04-25T10:03:13.667065+0000","last_fullsized":"2024-04-25T10:03:13.667065+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666293+0000","last_change":"2024-04-25T10:03:13.666293+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666293+0000","last_undegraded":"2024-04-25T10:03:13.666293+0000","last_fullsized":"2024-04-25T10:03:13.666293+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.660053+0000","last_change":"2024-04-25T10:03:13.660053+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.660053+0000","last_undegraded":"2024-04-25T10:03:13.660053+0000","last_fullsized":"2024-04-25T10:03:13.660053+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663657+0000","last_change":"2024-04-25T10:03:13.663657+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663657+0000","last_undegraded":"2024-04-25T10:03:13.663657+0000","last_fullsized":"2024-04-25T10:03:13.663657+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.659711+0000","last_change":"2024-04-25T10:03:13.659711+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.659711+0000","last_undegraded":"2024-04-25T10:03:13.659711+0000","last_fullsized":"2024-04-25T10:03:13.659711+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666990+0000","last_change":"2024-04-25T10:03:13.666990+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666990+0000","last_undegraded":"2024-04-25T10:03:13.666990+0000","last_fullsized":"2024-04-25T10:03:13.666990+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.676405+0000","last_change":"2024-04-25T10:03:13.676405+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.676405+0000","last_undegraded":"2024-04-25T10:03:13.676405+0000","last_fullsized":"2024-04-25T10:03:13.676405+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663037+0000","last_change":"2024-04-25T10:03:13.663037+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663037+0000","last_undegraded":"2024-04-25T10:03:13.663037+0000","last_fullsized":"2024-04-25T10:03:13.663037+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664731+0000","last_change":"2024-04-25T10:03:13.664731+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664731+0000","last_undegraded":"2024-04-25T10:03:13.664731+0000","last_fullsized":"2024-04-25T10:03:13.664731+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.662013+0000","last_change":"2024-04-25T10:03:13.662013+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.662013+0000","last_undegraded":"2024-04-25T10:03:13.662013+0000","last_fullsized":"2024-04-25T10:03:13.662013+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664612+0000","last_change":"2024-04-25T10:03:13.664612+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664612+0000","last_undegraded":"2024-04-25T10:03:13.664612+0000","last_fullsized":"2024-04-25T10:03:13.664612+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665575+0000","last_change":"2024-04-25T10:03:13.665575+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665575+0000","last_undegraded":"2024-04-25T10:03:13.665575+0000","last_fullsized":"2024-04-25T10:03:13.665575+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664441+0000","last_change":"2024-04-25T10:03:13.664441+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664441+0000","last_undegraded":"2024-04-25T10:03:13.664441+0000","last_fullsized":"2024-04-25T10:03:13.664441+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.661572+0000","last_change":"2024-04-25T10:03:13.661572+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.661572+0000","last_undegraded":"2024-04-25T10:03:13.661572+0000","last_fullsized":"2024-04-25T10:03:13.661572+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.675430+0000","last_change":"2024-04-25T10:03:13.674884+0000","last_active":"2024-04-25T10:03:13.675430+0000","last_peered":"2024-04-25T10:03:13.675430+0000","last_clean":"2024-04-25T10:03:13.675430+0000","last_became_active":"2024-04-25T10:03:13.674524+0000","last_became_peered":"2024-04-25T10:03:13.674524+0000","last_unstale":"2024-04-25T10:03:13.675430+0000","last_undegraded":"2024-04-25T10:03:13.675430+0000","last_fullsized":"2024-04-25T10:03:13.675430+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:59.983936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.662893+0000","last_change":"2024-04-25T10:03:13.662893+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.662893+0000","last_undegraded":"2024-04-25T10:03:13.662893+0000","last_fullsized":"2024-04-25T10:03:13.662893+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679971+0000","last_change":"2024-04-25T10:03:13.679019+0000","last_active":"2024-04-25T10:03:13.679971+0000","last_peered":"2024-04-25T10:03:13.679971+0000","last_clean":"2024-04-25T10:03:13.679971+0000","last_became_active":"2024-04-25T10:03:13.678461+0000","last_became_peered":"2024-04-25T10:03:13.678461+0000","last_unstale":"2024-04-25T10:03:13.679971+0000","last_undegraded":"2024-04-25T10:03:13.679971+0000","last_fullsized":"2024-04-25T10:03:13.679971+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:33.172897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663137+0000","last_change":"2024-04-25T10:03:13.663137+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663137+0000","last_undegraded":"2024-04-25T10:03:13.663137+0000","last_fullsized":"2024-04-25T10:03:13.663137+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.675509+0000","last_change":"2024-04-25T10:03:13.674905+0000","last_active":"2024-04-25T10:03:13.675509+0000","last_peered":"2024-04-25T10:03:13.675509+0000","last_clean":"2024-04-25T10:03:13.675509+0000","last_became_active":"2024-04-25T10:03:13.674590+0000","last_became_peered":"2024-04-25T10:03:13.674590+0000","last_unstale":"2024-04-25T10:03:13.675509+0000","last_undegraded":"2024-04-25T10:03:13.675509+0000","last_fullsized":"2024-04-25T10:03:13.675509+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:16.436615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.661145+0000","last_change":"2024-04-25T10:03:13.661145+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.661145+0000","last_undegraded":"2024-04-25T10:03:13.661145+0000","last_fullsized":"2024-04-25T10:03:13.661145+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.677237+0000","last_change":"2024-04-25T10:03:13.673989+0000","last_active":"2024-04-25T10:03:13.677237+0000","last_peered":"2024-04-25T10:03:13.677237+0000","last_clean":"2024-04-25T10:03:13.677237+0000","last_became_active":"2024-04-25T10:03:13.673670+0000","last_became_peered":"2024-04-25T10:03:13.673670+0000","last_unstale":"2024-04-25T10:03:13.677237+0000","last_undegraded":"2024-04-25T10:03:13.677237+0000","last_fullsized":"2024-04-25T10:03:13.677237+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:29.504127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664563+0000","last_change":"2024-04-25T10:03:13.664563+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664563+0000","last_undegraded":"2024-04-25T10:03:13.664563+0000","last_fullsized":"2024-04-25T10:03:13.664563+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.680052+0000","last_change":"2024-04-25T10:03:13.677056+0000","last_active":"2024-04-25T10:03:13.680052+0000","last_peered":"2024-04-25T10:03:13.680052+0000","last_clean":"2024-04-25T10:03:13.680052+0000","last_became_active":"2024-04-25T10:03:13.676751+0000","last_became_peered":"2024-04-25T10:03:13.676751+0000","last_unstale":"2024-04-25T10:03:13.680052+0000","last_undegraded":"2024-04-25T10:03:13.680052+0000","last_fullsized":"2024-04-25T10:03:13.680052+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:25.428507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.668039+0000","last_change":"2024-04-25T10:03:13.668039+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.668039+0000","last_undegraded":"2024-04-25T10:03:13.668039+0000","last_fullsized":"2024-04-25T10:03:13.668039+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679611+0000","last_change":"2024-04-25T10:03:13.674339+0000","last_active":"2024-04-25T10:03:13.679611+0000","last_peered":"2024-04-25T10:03:13.679611+0000","last_clean":"2024-04-25T10:03:13.679611+0000","last_became_active":"2024-04-25T10:03:13.674074+0000","last_became_peered":"2024-04-25T10:03:13.674074+0000","last_unstale":"2024-04-25T10:03:13.679611+0000","last_undegraded":"2024-04-25T10:03:13.679611+0000","last_fullsized":"2024-04-25T10:03:13.679611+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:13.450850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.673268+0000","last_change":"2024-04-25T10:03:13.673268+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.673268+0000","last_undegraded":"2024-04-25T10:03:13.673268+0000","last_fullsized":"2024-04-25T10:03:13.673268+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.682495+0000","last_change":"2024-04-25T10:03:13.681030+0000","last_active":"2024-04-25T10:03:13.682495+0000","last_peered":"2024-04-25T10:03:13.682495+0000","last_clean":"2024-04-25T10:03:13.682495+0000","last_became_active":"2024-04-25T10:03:13.680669+0000","last_became_peered":"2024-04-25T10:03:13.680669+0000","last_unstale":"2024-04-25T10:03:13.682495+0000","last_undegraded":"2024-04-25T10:03:13.682495+0000","last_fullsized":"2024-04-25T10:03:13.682495+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:04.380494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.660188+0000","last_change":"2024-04-25T10:03:13.660188+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.660188+0000","last_undegraded":"2024-04-25T10:03:13.660188+0000","last_fullsized":"2024-04-25T10:03:13.660188+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679388+0000","last_change":"2024-04-25T10:03:13.670929+0000","last_active":"2024-04-25T10:03:13.679388+0000","last_peered":"2024-04-25T10:03:13.679388+0000","last_clean":"2024-04-25T10:03:13.679388+0000","last_became_active":"2024-04-25T10:03:13.670647+0000","last_became_peered":"2024-04-25T10:03:13.670647+0000","last_unstale":"2024-04-25T10:03:13.679388+0000","last_undegraded":"2024-04-25T10:03:13.679388+0000","last_fullsized":"2024-04-25T10:03:13.679388+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:10.022627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.662063+0000","last_change":"2024-04-25T10:03:13.662063+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.662063+0000","last_undegraded":"2024-04-25T10:03:13.662063+0000","last_fullsized":"2024-04-25T10:03:13.662063+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.681610+0000","last_change":"2024-04-25T10:03:13.680361+0000","last_active":"2024-04-25T10:03:13.681610+0000","last_peered":"2024-04-25T10:03:13.681610+0000","last_clean":"2024-04-25T10:03:13.681610+0000","last_became_active":"2024-04-25T10:03:13.680075+0000","last_became_peered":"2024-04-25T10:03:13.680075+0000","last_unstale":"2024-04-25T10:03:13.681610+0000","last_undegraded":"2024-04-25T10:03:13.681610+0000","last_fullsized":"2024-04-25T10:03:13.681610+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:03.435847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664441+0000","last_change":"2024-04-25T10:03:13.664441+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664441+0000","last_undegraded":"2024-04-25T10:03:13.664441+0000","last_fullsized":"2024-04-25T10:03:13.664441+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.671673+0000","last_change":"2024-04-25T10:03:13.669655+0000","last_active":"2024-04-25T10:03:13.671673+0000","last_peered":"2024-04-25T10:03:13.671673+0000","last_clean":"2024-04-25T10:03:13.671673+0000","last_became_active":"2024-04-25T10:03:13.669366+0000","last_became_peered":"2024-04-25T10:03:13.669366+0000","last_unstale":"2024-04-25T10:03:13.671673+0000","last_undegraded":"2024-04-25T10:03:13.671673+0000","last_fullsized":"2024-04-25T10:03:13.671673+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:19.871768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665865+0000","last_change":"2024-04-25T10:03:13.665865+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665865+0000","last_undegraded":"2024-04-25T10:03:13.665865+0000","last_fullsized":"2024-04-25T10:03:13.665865+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.682150+0000","last_change":"2024-04-25T10:03:13.681202+0000","last_active":"2024-04-25T10:03:13.682150+0000","last_peered":"2024-04-25T10:03:13.682150+0000","last_clean":"2024-04-25T10:03:13.682150+0000","last_became_active":"2024-04-25T10:03:13.680736+0000","last_became_peered":"2024-04-25T10:03:13.680736+0000","last_unstale":"2024-04-25T10:03:13.682150+0000","last_undegraded":"2024-04-25T10:03:13.682150+0000","last_fullsized":"2024-04-25T10:03:13.682150+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:31.006719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664258+0000","last_change":"2024-04-25T10:03:13.664258+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664258+0000","last_undegraded":"2024-04-25T10:03:13.664258+0000","last_fullsized":"2024-04-25T10:03:13.664258+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679919+0000","last_change":"2024-04-25T10:03:13.671104+0000","last_active":"2024-04-25T10:03:13.679919+0000","last_peered":"2024-04-25T10:03:13.679919+0000","last_clean":"2024-04-25T10:03:13.679919+0000","last_became_active":"2024-04-25T10:03:13.670810+0000","last_became_peered":"2024-04-25T10:03:13.670810+0000","last_unstale":"2024-04-25T10:03:13.679919+0000","last_undegraded":"2024-04-25T10:03:13.679919+0000","last_fullsized":"2024-04-25T10:03:13.679919+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:11.244494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665310+0000","last_change":"2024-04-25T10:03:13.665310+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665310+0000","last_undegraded":"2024-04-25T10:03:13.665310+0000","last_fullsized":"2024-04-25T10:03:13.665310+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.674265+0000","last_change":"2024-04-25T10:03:13.672839+0000","last_active":"2024-04-25T10:03:13.674265+0000","last_peered":"2024-04-25T10:03:13.674265+0000","last_clean":"2024-04-25T10:03:13.674265+0000","last_became_active":"2024-04-25T10:03:13.671105+0000","last_became_peered":"2024-04-25T10:03:13.671105+0000","last_unstale":"2024-04-25T10:03:13.674265+0000","last_undegraded":"2024-04-25T10:03:13.674265+0000","last_fullsized":"2024-04-25T10:03:13.674265+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:51.239805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663836+0000","last_change":"2024-04-25T10:03:13.663836+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663836+0000","last_undegraded":"2024-04-25T10:03:13.663836+0000","last_fullsized":"2024-04-25T10:03:13.663836+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.680896+0000","last_change":"2024-04-25T10:03:13.680518+0000","last_active":"2024-04-25T10:03:13.680896+0000","last_peered":"2024-04-25T10:03:13.680896+0000","last_clean":"2024-04-25T10:03:13.680896+0000","last_became_active":"2024-04-25T10:03:13.680260+0000","last_became_peered":"2024-04-25T10:03:13.680260+0000","last_unstale":"2024-04-25T10:03:13.680896+0000","last_undegraded":"2024-04-25T10:03:13.680896+0000","last_fullsized":"2024-04-25T10:03:13.680896+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:51.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666559+0000","last_change":"2024-04-25T10:03:13.666559+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666559+0000","last_undegraded":"2024-04-25T10:03:13.666559+0000","last_fullsized":"2024-04-25T10:03:13.666559+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679409+0000","last_change":"2024-04-25T10:03:13.675759+0000","last_active":"2024-04-25T10:03:13.679409+0000","last_peered":"2024-04-25T10:03:13.679409+0000","last_clean":"2024-04-25T10:03:13.679409+0000","last_became_active":"2024-04-25T10:03:13.675488+0000","last_became_peered":"2024-04-25T10:03:13.675488+0000","last_unstale":"2024-04-25T10:03:13.679409+0000","last_undegraded":"2024-04-25T10:03:13.679409+0000","last_fullsized":"2024-04-25T10:03:13.679409+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:58.961175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663840+0000","last_change":"2024-04-25T10:03:13.663840+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663840+0000","last_undegraded":"2024-04-25T10:03:13.663840+0000","last_fullsized":"2024-04-25T10:03:13.663840+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.676273+0000","last_change":"2024-04-25T10:03:13.673662+0000","last_active":"2024-04-25T10:03:13.676273+0000","last_peered":"2024-04-25T10:03:13.676273+0000","last_clean":"2024-04-25T10:03:13.676273+0000","last_became_active":"2024-04-25T10:03:13.673355+0000","last_became_peered":"2024-04-25T10:03:13.673355+0000","last_unstale":"2024-04-25T10:03:13.676273+0000","last_undegraded":"2024-04-25T10:03:13.676273+0000","last_fullsized":"2024-04-25T10:03:13.676273+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:51.423687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665866+0000","last_change":"2024-04-25T10:03:13.665866+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665866+0000","last_undegraded":"2024-04-25T10:03:13.665866+0000","last_fullsized":"2024-04-25T10:03:13.665866+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679969+0000","last_change":"2024-04-25T10:03:13.678163+0000","last_active":"2024-04-25T10:03:13.679969+0000","last_peered":"2024-04-25T10:03:13.679969+0000","last_clean":"2024-04-25T10:03:13.679969+0000","last_became_active":"2024-04-25T10:03:13.677884+0000","last_became_peered":"2024-04-25T10:03:13.677884+0000","last_unstale":"2024-04-25T10:03:13.679969+0000","last_undegraded":"2024-04-25T10:03:13.679969+0000","last_fullsized":"2024-04-25T10:03:13.679969+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:04.430596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664148+0000","last_change":"2024-04-25T10:03:13.664148+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664148+0000","last_undegraded":"2024-04-25T10:03:13.664148+0000","last_fullsized":"2024-04-25T10:03:13.664148+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.672242+0000","last_change":"2024-04-25T10:03:13.667893+0000","last_active":"2024-04-25T10:03:13.672242+0000","last_peered":"2024-04-25T10:03:13.672242+0000","last_clean":"2024-04-25T10:03:13.672242+0000","last_became_active":"2024-04-25T10:03:13.667483+0000","last_became_peered":"2024-04-25T10:03:13.667483+0000","last_unstale":"2024-04-25T10:03:13.672242+0000","last_undegraded":"2024-04-25T10:03:13.672242+0000","last_fullsized":"2024-04-25T10:03:13.672242+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.341450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665173+0000","last_change":"2024-04-25T10:03:13.665173+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665173+0000","last_undegraded":"2024-04-25T10:03:13.665173+0000","last_fullsized":"2024-04-25T10:03:13.665173+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.680068+0000","last_change":"2024-04-25T10:03:13.678499+0000","last_active":"2024-04-25T10:03:13.680068+0000","last_peered":"2024-04-25T10:03:13.680068+0000","last_clean":"2024-04-25T10:03:13.680068+0000","last_became_active":"2024-04-25T10:03:13.678219+0000","last_became_peered":"2024-04-25T10:03:13.678219+0000","last_unstale":"2024-04-25T10:03:13.680068+0000","last_undegraded":"2024-04-25T10:03:13.680068+0000","last_fullsized":"2024-04-25T10:03:13.680068+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:24.928561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664289+0000","last_change":"2024-04-25T10:03:13.664289+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664289+0000","last_undegraded":"2024-04-25T10:03:13.664289+0000","last_fullsized":"2024-04-25T10:03:13.664289+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679898+0000","last_change":"2024-04-25T10:03:13.670893+0000","last_active":"2024-04-25T10:03:13.679898+0000","last_peered":"2024-04-25T10:03:13.679898+0000","last_clean":"2024-04-25T10:03:13.679898+0000","last_became_active":"2024-04-25T10:03:13.670599+0000","last_became_peered":"2024-04-25T10:03:13.670599+0000","last_unstale":"2024-04-25T10:03:13.679898+0000","last_undegraded":"2024-04-25T10:03:13.679898+0000","last_fullsized":"2024-04-25T10:03:13.679898+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:35.799225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.665101+0000","last_change":"2024-04-25T10:03:13.665101+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.665101+0000","last_undegraded":"2024-04-25T10:03:13.665101+0000","last_fullsized":"2024-04-25T10:03:13.665101+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.677138+0000","last_change":"2024-04-25T10:03:13.676633+0000","last_active":"2024-04-25T10:03:13.677138+0000","last_peered":"2024-04-25T10:03:13.677138+0000","last_clean":"2024-04-25T10:03:13.677138+0000","last_became_active":"2024-04-25T10:03:13.676353+0000","last_became_peered":"2024-04-25T10:03:13.676353+0000","last_unstale":"2024-04-25T10:03:13.677138+0000","last_undegraded":"2024-04-25T10:03:13.677138+0000","last_fullsized":"2024-04-25T10:03:13.677138+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:00.968592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.662544+0000","last_change":"2024-04-25T10:03:13.662544+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.662544+0000","last_undegraded":"2024-04-25T10:03:13.662544+0000","last_fullsized":"2024-04-25T10:03:13.662544+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.677253+0000","last_change":"2024-04-25T10:03:13.673943+0000","last_active":"2024-04-25T10:03:13.677253+0000","last_peered":"2024-04-25T10:03:13.677253+0000","last_clean":"2024-04-25T10:03:13.677253+0000","last_became_active":"2024-04-25T10:03:13.673646+0000","last_became_peered":"2024-04-25T10:03:13.673646+0000","last_unstale":"2024-04-25T10:03:13.677253+0000","last_undegraded":"2024-04-25T10:03:13.677253+0000","last_fullsized":"2024-04-25T10:03:13.677253+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:50.254439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.659603+0000","last_change":"2024-04-25T10:03:13.659603+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.659603+0000","last_undegraded":"2024-04-25T10:03:13.659603+0000","last_fullsized":"2024-04-25T10:03:13.659603+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.672425+0000","last_change":"2024-04-25T10:03:13.671428+0000","last_active":"2024-04-25T10:03:13.672425+0000","last_peered":"2024-04-25T10:03:13.672425+0000","last_clean":"2024-04-25T10:03:13.672425+0000","last_became_active":"2024-04-25T10:03:13.671080+0000","last_became_peered":"2024-04-25T10:03:13.671080+0000","last_unstale":"2024-04-25T10:03:13.672425+0000","last_undegraded":"2024-04-25T10:03:13.672425+0000","last_fullsized":"2024-04-25T10:03:13.672425+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:33.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.667127+0000","last_change":"2024-04-25T10:03:13.667127+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.667127+0000","last_undegraded":"2024-04-25T10:03:13.667127+0000","last_fullsized":"2024-04-25T10:03:13.667127+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679834+0000","last_change":"2024-04-25T10:03:13.669724+0000","last_active":"2024-04-25T10:03:13.679834+0000","last_peered":"2024-04-25T10:03:13.679834+0000","last_clean":"2024-04-25T10:03:13.679834+0000","last_became_active":"2024-04-25T10:03:13.669446+0000","last_became_peered":"2024-04-25T10:03:13.669446+0000","last_unstale":"2024-04-25T10:03:13.679834+0000","last_undegraded":"2024-04-25T10:03:13.679834+0000","last_fullsized":"2024-04-25T10:03:13.679834+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:22.924857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.663231+0000","last_change":"2024-04-25T10:03:13.663231+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.663231+0000","last_undegraded":"2024-04-25T10:03:13.663231+0000","last_fullsized":"2024-04-25T10:03:13.663231+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.682132+0000","last_change":"2024-04-25T10:03:13.681265+0000","last_active":"2024-04-25T10:03:13.682132+0000","last_peered":"2024-04-25T10:03:13.682132+0000","last_clean":"2024-04-25T10:03:13.682132+0000","last_became_active":"2024-04-25T10:03:13.680824+0000","last_became_peered":"2024-04-25T10:03:13.680824+0000","last_unstale":"2024-04-25T10:03:13.682132+0000","last_undegraded":"2024-04-25T10:03:13.682132+0000","last_fullsized":"2024-04-25T10:03:13.682132+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:38.503607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.668201+0000","last_change":"2024-04-25T10:03:13.668201+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.668201+0000","last_undegraded":"2024-04-25T10:03:13.668201+0000","last_fullsized":"2024-04-25T10:03:13.668201+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.673387+0000","last_change":"2024-04-25T10:03:13.670694+0000","last_active":"2024-04-25T10:03:13.673387+0000","last_peered":"2024-04-25T10:03:13.673387+0000","last_clean":"2024-04-25T10:03:13.673387+0000","last_became_active":"2024-04-25T10:03:13.670408+0000","last_became_peered":"2024-04-25T10:03:13.670408+0000","last_unstale":"2024-04-25T10:03:13.673387+0000","last_undegraded":"2024-04-25T10:03:13.673387+0000","last_fullsized":"2024-04-25T10:03:13.673387+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:33.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664171+0000","last_change":"2024-04-25T10:03:13.664171+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664171+0000","last_undegraded":"2024-04-25T10:03:13.664171+0000","last_fullsized":"2024-04-25T10:03:13.664171+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.682344+0000","last_change":"2024-04-25T10:03:13.679719+0000","last_active":"2024-04-25T10:03:13.682344+0000","last_peered":"2024-04-25T10:03:13.682344+0000","last_clean":"2024-04-25T10:03:13.682344+0000","last_became_active":"2024-04-25T10:03:13.679453+0000","last_became_peered":"2024-04-25T10:03:13.679453+0000","last_unstale":"2024-04-25T10:03:13.682344+0000","last_undegraded":"2024-04-25T10:03:13.682344+0000","last_fullsized":"2024-04-25T10:03:13.682344+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:16.308823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.659279+0000","last_change":"2024-04-25T10:03:13.659279+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.659279+0000","last_undegraded":"2024-04-25T10:03:13.659279+0000","last_fullsized":"2024-04-25T10:03:13.659279+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.672296+0000","last_change":"2024-04-25T10:03:13.670937+0000","last_active":"2024-04-25T10:03:13.672296+0000","last_peered":"2024-04-25T10:03:13.672296+0000","last_clean":"2024-04-25T10:03:13.672296+0000","last_became_active":"2024-04-25T10:03:13.670561+0000","last_became_peered":"2024-04-25T10:03:13.670561+0000","last_unstale":"2024-04-25T10:03:13.672296+0000","last_undegraded":"2024-04-25T10:03:13.672296+0000","last_fullsized":"2024-04-25T10:03:13.672296+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:27.872809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.661551+0000","last_change":"2024-04-25T10:03:13.661551+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.661551+0000","last_undegraded":"2024-04-25T10:03:13.661551+0000","last_fullsized":"2024-04-25T10:03:13.661551+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.674244+0000","last_change":"2024-04-25T10:03:13.671727+0000","last_active":"2024-04-25T10:03:13.674244+0000","last_peered":"2024-04-25T10:03:13.674244+0000","last_clean":"2024-04-25T10:03:13.674244+0000","last_became_active":"2024-04-25T10:03:13.671439+0000","last_became_peered":"2024-04-25T10:03:13.671439+0000","last_unstale":"2024-04-25T10:03:13.674244+0000","last_undegraded":"2024-04-25T10:03:13.674244+0000","last_fullsized":"2024-04-25T10:03:13.674244+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:53.770956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.664094+0000","last_change":"2024-04-25T10:03:13.664094+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.664094+0000","last_undegraded":"2024-04-25T10:03:13.664094+0000","last_fullsized":"2024-04-25T10:03:13.664094+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.675461+0000","last_change":"2024-04-25T10:03:13.674966+0000","last_active":"2024-04-25T10:03:13.675461+0000","last_peered":"2024-04-25T10:03:13.675461+0000","last_clean":"2024-04-25T10:03:13.675461+0000","last_became_active":"2024-04-25T10:03:13.674633+0000","last_became_peered":"2024-04-25T10:03:13.674633+0000","last_unstale":"2024-04-25T10:03:13.675461+0000","last_undegraded":"2024-04-25T10:03:13.675461+0000","last_fullsized":"2024-04-25T10:03:13.675461+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:05.106689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.666020+0000","last_change":"2024-04-25T10:03:13.666020+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.666020+0000","last_undegraded":"2024-04-25T10:03:13.666020+0000","last_fullsized":"2024-04-25T10:03:13.666020+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.679869+0000","last_change":"2024-04-25T10:03:13.670435+0000","last_active":"2024-04-25T10:03:13.679869+0000","last_peered":"2024-04-25T10:03:13.679869+0000","last_clean":"2024-04-25T10:03:13.679869+0000","last_became_active":"2024-04-25T10:03:13.670146+0000","last_became_peered":"2024-04-25T10:03:13.670146+0000","last_unstale":"2024-04-25T10:03:13.679869+0000","last_undegraded":"2024-04-25T10:03:13.679869+0000","last_fullsized":"2024-04-25T10:03:13.679869+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:34.731705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":152,"state":"creating+peering","last_fresh":"2024-04-25T10:03:13.659572+0000","last_change":"2024-04-25T10:03:13.659572+0000","last_active":"2024-04-25T10:03:13.651178+0000","last_peered":"2024-04-25T10:03:13.651178+0000","last_clean":"2024-04-25T10:03:13.651178+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:13.659572+0000","last_undegraded":"2024-04-25T10:03:13.659572+0000","last_fullsized":"2024-04-25T10:03:13.659572+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.682440+0000","last_change":"2024-04-25T10:03:13.681369+0000","last_active":"2024-04-25T10:03:13.682440+0000","last_peered":"2024-04-25T10:03:13.682440+0000","last_clean":"2024-04-25T10:03:13.682440+0000","last_became_active":"2024-04-25T10:03:13.681075+0000","last_became_peered":"2024-04-25T10:03:13.681075+0000","last_unstale":"2024-04-25T10:03:13.682440+0000","last_undegraded":"2024-04-25T10:03:13.682440+0000","last_fullsized":"2024-04-25T10:03:13.682440+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:31.792118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":439,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.668539+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:03:13.668539+0000","last_peered":"2024-04-25T10:03:13.668539+0000","last_clean":"2024-04-25T10:03:13.668539+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:03:13.668539+0000","last_undegraded":"2024-04-25T10:03:13.668539+0000","last_fullsized":"2024-04-25T10:03:13.668539+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51664,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320176,"statfs":{"total":96636764160,"available":96583860224,"internally_reserved":0,"allocated":2494464,"data_stored":1478316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51084,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320756,"statfs":{"total":96636764160,"available":96584454144,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59404,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312436,"statfs":{"total":96636764160,"available":96575934464,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59404,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312436,"statfs":{"total":96636764160,"available":96575934464,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55248,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316592,"statfs":{"total":96636764160,"available":96580190208,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55244,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316596,"statfs":{"total":96636764160,"available":96580194304,"internally_reserved":0,"allocated":1900544,"data_stored":887948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55840,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316000,"statfs":{"total":96636764160,"available":96579584000,"internally_reserved":0,"allocated":2506752,"data_stored":1484975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:19.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.167+0000 7f270068f640 1 Processor -- start 2024-04-25T10:03:19.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.168+0000 7f270068f640 1 -- start start 2024-04-25T10:03:19.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.168+0000 7f270068f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26f815ba00 0x7f26f809a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.168+0000 7f270068f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f26f815f5f0 0x7f26f815faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.168+0000 7f270068f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f80cd710 con 0x7f26f81602e0 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.168+0000 7f270068f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f80cd4b0 con 0x7f26f815f5f0 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.168+0000 7f270068f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f8099b70 con 0x7f26f815ba00 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fdc2a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f26f81602e0 0x7f26f80ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46132/0 (socket says 172.21.15.19:46132) 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fdc2a640 1 -- 172.21.15.19:0/3861555782 learned_addr learned my addr 172.21.15.19:0/3861555782 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fe42b640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26f815ba00 0x7f26f809a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fec2c640 1 --2- 172.21.15.19:0/3861555782 >> v2:172.21.15.112:3300/0 conn(0x7f26f815f5f0 0x7f26f815faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816442390 0 0) 0x7f26ec001110 con 0x7f26f81602e0 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e0003520 con 0x7f26f81602e0 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 960119126 0 0) 0x7f26e0003520 con 0x7f26f81602e0 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26f815ba00 msgr2=0x7f26f809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26f815ba00 0x7f26f809a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 >> v2:172.21.15.112:3300/0 conn(0x7f26f815f5f0 msgr2=0x7f26f815faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 --2- 172.21.15.19:0/3861555782 >> v2:172.21.15.112:3300/0 conn(0x7f26f815f5f0 0x7f26f815faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f8099e50 con 0x7f26f81602e0 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f26ec001840 con 0x7f26f81602e0 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fec2c640 1 --2- 172.21.15.19:0/3861555782 >> v2:172.21.15.112:3300/0 conn(0x7f26f815f5f0 0x7f26f815faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 >> v1:172.21.15.19:6789/0 conn(0x7f26f81602e0 legacy=0x7f26f80ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26f815ba00 0x7f26e00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0003f30 0x7f26e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e0004d40 0x7f26e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80cd4b0 con 0x7f26e0004d40 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f8099b70 con 0x7f26e0003f30 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.169+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e000baa0 con 0x7f26f815ba00 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.170+0000 7f26fec2c640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e0004d40 0x7f26e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.170+0000 7f26fe42b640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26f815ba00 0x7f26e00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.170+0000 7f26fdc2a640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0003f30 0x7f26e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.170+0000 7f26fec2c640 1 -- 172.21.15.19:0/3861555782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26f815ba00 msgr2=0x7f26e00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.170+0000 7f26fec2c640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26f815ba00 0x7f26e00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.170+0000 7f26fec2c640 1 -- 172.21.15.19:0/3861555782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0003f30 msgr2=0x7f26e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.170+0000 7f26fec2c640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0003f30 0x7f26e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.170+0000 7f26fec2c640 1 -- 172.21.15.19:0/3861555782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e0003520 con 0x7f26e0004d40 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.170+0000 7f26fec2c640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e0004d40 0x7f26e00051c0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f26f000e350 tx=0x7f26e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e102956b5e900b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.171+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0009de0 con 0x7f26e0004d40 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.171+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f002e660 con 0x7f26e0004d40 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.171+0000 7f26fd429640 1 -- 172.21.15.19:0/3861555782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f002ea10 con 0x7f26e0004d40 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.171+0000 7f270068f640 1 -- 172.21.15.19:0/3861555782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e0004d40 msgr2=0x7f26e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.171+0000 7f270068f640 1 --2- 172.21.15.19:0/3861555782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e0004d40 0x7f26e00051c0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f26f000e350 tx=0x7f26e0005830 comp rx=0 tx=0).stop 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.171+0000 7f26fec2c640 1 -- 172.21.15.19:0/3861555782 reap_dead start 2024-04-25T10:03:19.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.171+0000 7f270068f640 1 -- 172.21.15.19:0/3861555782 shutdown_connections 2024-04-25T10:03:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.171+0000 7f270068f640 1 -- 172.21.15.19:0/3861555782 >> 172.21.15.19:0/3861555782 conn(0x7f26f80b3270 msgr2=0x7f26f80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.171+0000 7f270068f640 1 -- 172.21.15.19:0/3861555782 shutdown_connections 2024-04-25T10:03:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.171+0000 7f270068f640 1 -- 172.21.15.19:0/3861555782 wait complete. 2024-04-25T10:03:19.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.172+0000 7f270068f640 1 Processor -- start 2024-04-25T10:03:19.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.172+0000 7f270068f640 1 -- start start 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.173+0000 7f270068f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26e0004d40 0x7f26f806f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.173+0000 7f270068f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f806fc40 0x7f26f80b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.173+0000 7f270068f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26f80b0db0 0x7f26f80b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.173+0000 7f270068f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f815bde0 con 0x7f26f80b0db0 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.173+0000 7f270068f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f809a930 con 0x7f26f806fc40 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.173+0000 7f270068f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80700c0 con 0x7f26e0004d40 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.173+0000 7f26fec2c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26f80b0db0 0x7f26f80b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.173+0000 7f26fec2c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26f80b0db0 0x7f26f80b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49648/0 (socket says 172.21.15.19:49648) 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.173+0000 7f26fec2c640 1 -- 172.21.15.19:0/2057417806 learned_addr learned my addr 172.21.15.19:0/2057417806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.173+0000 7f26fe42b640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26e0004d40 0x7f26f806f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.173+0000 7f26fdc2a640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f806fc40 0x7f26f80b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.174+0000 7f26fec2c640 1 -- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26e0004d40 msgr2=0x7f26f806f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.174+0000 7f26fec2c640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26e0004d40 0x7f26f806f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.174+0000 7f26fec2c640 1 -- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f806fc40 msgr2=0x7f26f80b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.174+0000 7f26fec2c640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f806fc40 0x7f26f80b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.174+0000 7f26fec2c640 1 -- 172.21.15.19:0/2057417806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f80b17a0 con 0x7f26f80b0db0 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.174+0000 7f26fec2c640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26f80b0db0 0x7f26f80b1260 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f26f0002230 tx=0x7f26f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.174+0000 7f26f6ffd640 1 -- 172.21.15.19:0/2057417806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0009de0 con 0x7f26f80b0db0 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.174+0000 7f26f6ffd640 1 -- 172.21.15.19:0/2057417806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f0003990 con 0x7f26f80b0db0 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.174+0000 7f270068f640 1 -- 172.21.15.19:0/2057417806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f814f0b0 con 0x7f26f80b0db0 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.174+0000 7f26f6ffd640 1 -- 172.21.15.19:0/2057417806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0003d60 con 0x7f26f80b0db0 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.174+0000 7f270068f640 1 -- 172.21.15.19:0/2057417806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f80702e0 con 0x7f26f80b0db0 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.175+0000 7f270068f640 1 -- 172.21.15.19:0/2057417806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f8073990 con 0x7f26f80b0db0 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.175+0000 7f26f6ffd640 1 -- 172.21.15.19:0/2057417806 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26f00368c0 con 0x7f26f80b0db0 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.176+0000 7f26f6ffd640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26e406b4d0 0x7f26e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.176+0000 7f26f6ffd640 1 -- 172.21.15.19:0/2057417806 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f26f00ab3c0 con 0x7f26f80b0db0 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.176+0000 7f26fe42b640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26e406b4d0 0x7f26e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.176+0000 7f26fe42b640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26e406b4d0 0x7f26e406d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009b00 tx=0x7f26e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.180+0000 7f26f6ffd640 1 -- 172.21.15.19:0/2057417806 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26f80b17a0 con 0x7f26f80b0db0 2024-04-25T10:03:19.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.356+0000 7f270068f640 1 -- 172.21.15.19:0/2057417806 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26f80b24b0 con 0x7f26e406b4d0 2024-04-25T10:03:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.362+0000 7f26f6ffd640 1 -- 172.21.15.19:0/2057417806 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252480 (secure 0 0 0) 0x7f26f80b24b0 con 0x7f26e406b4d0 2024-04-25T10:03:19.369 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:03:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 -- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26e406b4d0 msgr2=0x7f26e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26e406b4d0 0x7f26e406d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009b00 tx=0x7f26e8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 -- 172.21.15.19:0/2057417806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26f80b0db0 msgr2=0x7f26f80b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26f80b0db0 0x7f26f80b1260 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f26f0002230 tx=0x7f26f0004ab0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 -- 172.21.15.19:0/2057417806 shutdown_connections 2024-04-25T10:03:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f26e406b4d0 0x7f26e406d990 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26f80b0db0 0x7f26f80b1260 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f806fc40 0x7f26f80b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 --2- 172.21.15.19:0/2057417806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26e0004d40 0x7f26f806f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 -- 172.21.15.19:0/2057417806 >> 172.21.15.19:0/2057417806 conn(0x7f26f80b3270 msgr2=0x7f26f80cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 -- 172.21.15.19:0/2057417806 shutdown_connections 2024-04-25T10:03:19.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.365+0000 7f270068f640 1 -- 172.21.15.19:0/2057417806 wait complete. 2024-04-25T10:03:19.381 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-04-25T10:03:19.053588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439324,"kb_used_data":16360,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754535396,"statfs":{"total":773094113280,"available":772644245504,"internally_reserved":0,"allocated":16752640,"data_stored":8453284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001432"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663538+0000","last_change":"2024-04-25T10:03:14.674988+0000","last_active":"2024-04-25T10:03:15.663538+0000","last_peered":"2024-04-25T10:03:15.663538+0000","last_clean":"2024-04-25T10:03:15.663538+0000","last_became_active":"2024-04-25T10:03:14.674705+0000","last_became_peered":"2024-04-25T10:03:14.674705+0000","last_unstale":"2024-04-25T10:03:15.663538+0000","last_undegraded":"2024-04-25T10:03:15.663538+0000","last_fullsized":"2024-04-25T10:03:15.663538+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:55.709812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663091+0000","last_change":"2024-04-25T10:03:14.677891+0000","last_active":"2024-04-25T10:03:15.663091+0000","last_peered":"2024-04-25T10:03:15.663091+0000","last_clean":"2024-04-25T10:03:15.663091+0000","last_became_active":"2024-04-25T10:03:14.676509+0000","last_became_peered":"2024-04-25T10:03:14.676509+0000","last_unstale":"2024-04-25T10:03:15.663091+0000","last_undegraded":"2024-04-25T10:03:15.663091+0000","last_fullsized":"2024-04-25T10:03:15.663091+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:25.956823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663156+0000","last_change":"2024-04-25T10:03:14.693467+0000","last_active":"2024-04-25T10:03:15.663156+0000","last_peered":"2024-04-25T10:03:15.663156+0000","last_clean":"2024-04-25T10:03:15.663156+0000","last_became_active":"2024-04-25T10:03:14.693229+0000","last_became_peered":"2024-04-25T10:03:14.693229+0000","last_unstale":"2024-04-25T10:03:15.663156+0000","last_undegraded":"2024-04-25T10:03:15.663156+0000","last_fullsized":"2024-04-25T10:03:15.663156+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:04.852884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662858+0000","last_change":"2024-04-25T10:03:14.668329+0000","last_active":"2024-04-25T10:03:15.662858+0000","last_peered":"2024-04-25T10:03:15.662858+0000","last_clean":"2024-04-25T10:03:15.662858+0000","last_became_active":"2024-04-25T10:03:14.668042+0000","last_became_peered":"2024-04-25T10:03:14.668042+0000","last_unstale":"2024-04-25T10:03:15.662858+0000","last_undegraded":"2024-04-25T10:03:15.662858+0000","last_fullsized":"2024-04-25T10:03:15.662858+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:23.955400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661748+0000","last_change":"2024-04-25T10:03:14.681919+0000","last_active":"2024-04-25T10:03:15.661748+0000","last_peered":"2024-04-25T10:03:15.661748+0000","last_clean":"2024-04-25T10:03:15.661748+0000","last_became_active":"2024-04-25T10:03:14.681625+0000","last_became_peered":"2024-04-25T10:03:14.681625+0000","last_unstale":"2024-04-25T10:03:15.661748+0000","last_undegraded":"2024-04-25T10:03:15.661748+0000","last_fullsized":"2024-04-25T10:03:15.661748+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.895682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662459+0000","last_change":"2024-04-25T10:03:14.664822+0000","last_active":"2024-04-25T10:03:15.662459+0000","last_peered":"2024-04-25T10:03:15.662459+0000","last_clean":"2024-04-25T10:03:15.662459+0000","last_became_active":"2024-04-25T10:03:14.664548+0000","last_became_peered":"2024-04-25T10:03:14.664548+0000","last_unstale":"2024-04-25T10:03:15.662459+0000","last_undegraded":"2024-04-25T10:03:15.662459+0000","last_fullsized":"2024-04-25T10:03:15.662459+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:35.733949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032783+0000","last_change":"2024-04-25T10:03:14.696283+0000","last_active":"2024-04-25T10:03:16.032783+0000","last_peered":"2024-04-25T10:03:16.032783+0000","last_clean":"2024-04-25T10:03:16.032783+0000","last_became_active":"2024-04-25T10:03:14.696049+0000","last_became_peered":"2024-04-25T10:03:14.696049+0000","last_unstale":"2024-04-25T10:03:16.032783+0000","last_undegraded":"2024-04-25T10:03:16.032783+0000","last_fullsized":"2024-04-25T10:03:16.032783+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:36.801468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.031940+0000","last_change":"2024-04-25T10:03:14.677271+0000","last_active":"2024-04-25T10:03:16.031940+0000","last_peered":"2024-04-25T10:03:16.031940+0000","last_clean":"2024-04-25T10:03:16.031940+0000","last_became_active":"2024-04-25T10:03:14.676987+0000","last_became_peered":"2024-04-25T10:03:14.676987+0000","last_unstale":"2024-04-25T10:03:16.031940+0000","last_undegraded":"2024-04-25T10:03:16.031940+0000","last_fullsized":"2024-04-25T10:03:16.031940+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:28.465371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032612+0000","last_change":"2024-04-25T10:03:14.681985+0000","last_active":"2024-04-25T10:03:16.032612+0000","last_peered":"2024-04-25T10:03:16.032612+0000","last_clean":"2024-04-25T10:03:16.032612+0000","last_became_active":"2024-04-25T10:03:14.681702+0000","last_became_peered":"2024-04-25T10:03:14.681702+0000","last_unstale":"2024-04-25T10:03:16.032612+0000","last_undegraded":"2024-04-25T10:03:16.032612+0000","last_fullsized":"2024-04-25T10:03:16.032612+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:44.343693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032758+0000","last_change":"2024-04-25T10:03:14.683567+0000","last_active":"2024-04-25T10:03:16.032758+0000","last_peered":"2024-04-25T10:03:16.032758+0000","last_clean":"2024-04-25T10:03:16.032758+0000","last_became_active":"2024-04-25T10:03:14.681589+0000","last_became_peered":"2024-04-25T10:03:14.681589+0000","last_unstale":"2024-04-25T10:03:16.032758+0000","last_undegraded":"2024-04-25T10:03:16.032758+0000","last_fullsized":"2024-04-25T10:03:16.032758+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:23.927100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032162+0000","last_change":"2024-04-25T10:03:14.681356+0000","last_active":"2024-04-25T10:03:16.032162+0000","last_peered":"2024-04-25T10:03:16.032162+0000","last_clean":"2024-04-25T10:03:16.032162+0000","last_became_active":"2024-04-25T10:03:14.680990+0000","last_became_peered":"2024-04-25T10:03:14.680990+0000","last_unstale":"2024-04-25T10:03:16.032162+0000","last_undegraded":"2024-04-25T10:03:16.032162+0000","last_fullsized":"2024-04-25T10:03:16.032162+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:52.425930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663219+0000","last_change":"2024-04-25T10:03:14.665666+0000","last_active":"2024-04-25T10:03:15.663219+0000","last_peered":"2024-04-25T10:03:15.663219+0000","last_clean":"2024-04-25T10:03:15.663219+0000","last_became_active":"2024-04-25T10:03:14.665328+0000","last_became_peered":"2024-04-25T10:03:14.665328+0000","last_unstale":"2024-04-25T10:03:15.663219+0000","last_undegraded":"2024-04-25T10:03:15.663219+0000","last_fullsized":"2024-04-25T10:03:15.663219+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:10.838625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032537+0000","last_change":"2024-04-25T10:03:14.674760+0000","last_active":"2024-04-25T10:03:16.032537+0000","last_peered":"2024-04-25T10:03:16.032537+0000","last_clean":"2024-04-25T10:03:16.032537+0000","last_became_active":"2024-04-25T10:03:14.674456+0000","last_became_peered":"2024-04-25T10:03:14.674456+0000","last_unstale":"2024-04-25T10:03:16.032537+0000","last_undegraded":"2024-04-25T10:03:16.032537+0000","last_fullsized":"2024-04-25T10:03:16.032537+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:44.291626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661326+0000","last_change":"2024-04-25T10:03:14.694517+0000","last_active":"2024-04-25T10:03:15.661326+0000","last_peered":"2024-04-25T10:03:15.661326+0000","last_clean":"2024-04-25T10:03:15.661326+0000","last_became_active":"2024-04-25T10:03:14.694069+0000","last_became_peered":"2024-04-25T10:03:14.694069+0000","last_unstale":"2024-04-25T10:03:15.661326+0000","last_undegraded":"2024-04-25T10:03:15.661326+0000","last_fullsized":"2024-04-25T10:03:15.661326+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:13.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032343+0000","last_change":"2024-04-25T10:03:14.683731+0000","last_active":"2024-04-25T10:03:16.032343+0000","last_peered":"2024-04-25T10:03:16.032343+0000","last_clean":"2024-04-25T10:03:16.032343+0000","last_became_active":"2024-04-25T10:03:14.683402+0000","last_became_peered":"2024-04-25T10:03:14.683402+0000","last_unstale":"2024-04-25T10:03:16.032343+0000","last_undegraded":"2024-04-25T10:03:16.032343+0000","last_fullsized":"2024-04-25T10:03:16.032343+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:39.505850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662812+0000","last_change":"2024-04-25T10:03:14.669397+0000","last_active":"2024-04-25T10:03:15.662812+0000","last_peered":"2024-04-25T10:03:15.662812+0000","last_clean":"2024-04-25T10:03:15.662812+0000","last_became_active":"2024-04-25T10:03:14.668368+0000","last_became_peered":"2024-04-25T10:03:14.668368+0000","last_unstale":"2024-04-25T10:03:15.662812+0000","last_undegraded":"2024-04-25T10:03:15.662812+0000","last_fullsized":"2024-04-25T10:03:15.662812+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:28.463537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032771+0000","last_change":"2024-04-25T10:03:14.681114+0000","last_active":"2024-04-25T10:03:16.032771+0000","last_peered":"2024-04-25T10:03:16.032771+0000","last_clean":"2024-04-25T10:03:16.032771+0000","last_became_active":"2024-04-25T10:03:14.680775+0000","last_became_peered":"2024-04-25T10:03:14.680775+0000","last_unstale":"2024-04-25T10:03:16.032771+0000","last_undegraded":"2024-04-25T10:03:16.032771+0000","last_fullsized":"2024-04-25T10:03:16.032771+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:35.733949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032399+0000","last_change":"2024-04-25T10:03:14.674046+0000","last_active":"2024-04-25T10:03:16.032399+0000","last_peered":"2024-04-25T10:03:16.032399+0000","last_clean":"2024-04-25T10:03:16.032399+0000","last_became_active":"2024-04-25T10:03:14.673789+0000","last_became_peered":"2024-04-25T10:03:14.673789+0000","last_unstale":"2024-04-25T10:03:16.032399+0000","last_undegraded":"2024-04-25T10:03:16.032399+0000","last_fullsized":"2024-04-25T10:03:16.032399+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:04.852884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032669+0000","last_change":"2024-04-25T10:03:14.675106+0000","last_active":"2024-04-25T10:03:16.032669+0000","last_peered":"2024-04-25T10:03:16.032669+0000","last_clean":"2024-04-25T10:03:16.032669+0000","last_became_active":"2024-04-25T10:03:14.674810+0000","last_became_peered":"2024-04-25T10:03:14.674810+0000","last_unstale":"2024-04-25T10:03:16.032669+0000","last_undegraded":"2024-04-25T10:03:16.032669+0000","last_fullsized":"2024-04-25T10:03:16.032669+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:31.422540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695708+0000","last_change":"2024-04-25T10:03:14.691234+0000","last_active":"2024-04-25T10:03:15.695708+0000","last_peered":"2024-04-25T10:03:15.695708+0000","last_clean":"2024-04-25T10:03:15.695708+0000","last_became_active":"2024-04-25T10:03:14.690950+0000","last_became_peered":"2024-04-25T10:03:14.690950+0000","last_unstale":"2024-04-25T10:03:15.695708+0000","last_undegraded":"2024-04-25T10:03:15.695708+0000","last_fullsized":"2024-04-25T10:03:15.695708+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:20.623444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032667+0000","last_change":"2024-04-25T10:03:14.689443+0000","last_active":"2024-04-25T10:03:16.032667+0000","last_peered":"2024-04-25T10:03:16.032667+0000","last_clean":"2024-04-25T10:03:16.032667+0000","last_became_active":"2024-04-25T10:03:14.689151+0000","last_became_peered":"2024-04-25T10:03:14.689151+0000","last_unstale":"2024-04-25T10:03:16.032667+0000","last_undegraded":"2024-04-25T10:03:16.032667+0000","last_fullsized":"2024-04-25T10:03:16.032667+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:16.734137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662133+0000","last_change":"2024-04-25T10:03:14.677533+0000","last_active":"2024-04-25T10:03:15.662133+0000","last_peered":"2024-04-25T10:03:15.662133+0000","last_clean":"2024-04-25T10:03:15.662133+0000","last_became_active":"2024-04-25T10:03:14.677261+0000","last_became_peered":"2024-04-25T10:03:14.677261+0000","last_unstale":"2024-04-25T10:03:15.662133+0000","last_undegraded":"2024-04-25T10:03:15.662133+0000","last_fullsized":"2024-04-25T10:03:15.662133+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:31.946288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661340+0000","last_change":"2024-04-25T10:03:14.680014+0000","last_active":"2024-04-25T10:03:15.661340+0000","last_peered":"2024-04-25T10:03:15.661340+0000","last_clean":"2024-04-25T10:03:15.661340+0000","last_became_active":"2024-04-25T10:03:14.679525+0000","last_became_peered":"2024-04-25T10:03:14.679525+0000","last_unstale":"2024-04-25T10:03:15.661340+0000","last_undegraded":"2024-04-25T10:03:15.661340+0000","last_fullsized":"2024-04-25T10:03:15.661340+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:09.709075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663186+0000","last_change":"2024-04-25T10:03:14.668828+0000","last_active":"2024-04-25T10:03:15.663186+0000","last_peered":"2024-04-25T10:03:15.663186+0000","last_clean":"2024-04-25T10:03:15.663186+0000","last_became_active":"2024-04-25T10:03:14.668000+0000","last_became_peered":"2024-04-25T10:03:14.668000+0000","last_unstale":"2024-04-25T10:03:15.663186+0000","last_undegraded":"2024-04-25T10:03:15.663186+0000","last_fullsized":"2024-04-25T10:03:15.663186+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:59.362567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032734+0000","last_change":"2024-04-25T10:03:14.673302+0000","last_active":"2024-04-25T10:03:16.032734+0000","last_peered":"2024-04-25T10:03:16.032734+0000","last_clean":"2024-04-25T10:03:16.032734+0000","last_became_active":"2024-04-25T10:03:14.673003+0000","last_became_peered":"2024-04-25T10:03:14.673003+0000","last_unstale":"2024-04-25T10:03:16.032734+0000","last_undegraded":"2024-04-25T10:03:16.032734+0000","last_fullsized":"2024-04-25T10:03:16.032734+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:05.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695652+0000","last_change":"2024-04-25T10:03:14.693330+0000","last_active":"2024-04-25T10:03:15.695652+0000","last_peered":"2024-04-25T10:03:15.695652+0000","last_clean":"2024-04-25T10:03:15.695652+0000","last_became_active":"2024-04-25T10:03:14.692664+0000","last_became_peered":"2024-04-25T10:03:14.692664+0000","last_unstale":"2024-04-25T10:03:15.695652+0000","last_undegraded":"2024-04-25T10:03:15.695652+0000","last_fullsized":"2024-04-25T10:03:15.695652+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:33.297902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662549+0000","last_change":"2024-04-25T10:03:14.680270+0000","last_active":"2024-04-25T10:03:15.662549+0000","last_peered":"2024-04-25T10:03:15.662549+0000","last_clean":"2024-04-25T10:03:15.662549+0000","last_became_active":"2024-04-25T10:03:14.680005+0000","last_became_peered":"2024-04-25T10:03:14.680005+0000","last_unstale":"2024-04-25T10:03:15.662549+0000","last_undegraded":"2024-04-25T10:03:15.662549+0000","last_fullsized":"2024-04-25T10:03:15.662549+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:36.801468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032504+0000","last_change":"2024-04-25T10:03:14.666917+0000","last_active":"2024-04-25T10:03:16.032504+0000","last_peered":"2024-04-25T10:03:16.032504+0000","last_clean":"2024-04-25T10:03:16.032504+0000","last_became_active":"2024-04-25T10:03:14.666580+0000","last_became_peered":"2024-04-25T10:03:14.666580+0000","last_unstale":"2024-04-25T10:03:16.032504+0000","last_undegraded":"2024-04-25T10:03:16.032504+0000","last_fullsized":"2024-04-25T10:03:16.032504+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:43.733533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662990+0000","last_change":"2024-04-25T10:03:14.675411+0000","last_active":"2024-04-25T10:03:15.662990+0000","last_peered":"2024-04-25T10:03:15.662990+0000","last_clean":"2024-04-25T10:03:15.662990+0000","last_became_active":"2024-04-25T10:03:14.675071+0000","last_became_peered":"2024-04-25T10:03:14.675071+0000","last_unstale":"2024-04-25T10:03:15.662990+0000","last_undegraded":"2024-04-25T10:03:15.662990+0000","last_fullsized":"2024-04-25T10:03:15.662990+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:34.175141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032478+0000","last_change":"2024-04-25T10:03:14.680054+0000","last_active":"2024-04-25T10:03:16.032478+0000","last_peered":"2024-04-25T10:03:16.032478+0000","last_clean":"2024-04-25T10:03:16.032478+0000","last_became_active":"2024-04-25T10:03:14.679780+0000","last_became_peered":"2024-04-25T10:03:14.679780+0000","last_unstale":"2024-04-25T10:03:16.032478+0000","last_undegraded":"2024-04-25T10:03:16.032478+0000","last_fullsized":"2024-04-25T10:03:16.032478+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:55.709812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032215+0000","last_change":"2024-04-25T10:03:14.678473+0000","last_active":"2024-04-25T10:03:16.032215+0000","last_peered":"2024-04-25T10:03:16.032215+0000","last_clean":"2024-04-25T10:03:16.032215+0000","last_became_active":"2024-04-25T10:03:14.677596+0000","last_became_peered":"2024-04-25T10:03:14.677596+0000","last_unstale":"2024-04-25T10:03:16.032215+0000","last_undegraded":"2024-04-25T10:03:16.032215+0000","last_fullsized":"2024-04-25T10:03:16.032215+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:04.438091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032098+0000","last_change":"2024-04-25T10:03:14.672679+0000","last_active":"2024-04-25T10:03:16.032098+0000","last_peered":"2024-04-25T10:03:16.032098+0000","last_clean":"2024-04-25T10:03:16.032098+0000","last_became_active":"2024-04-25T10:03:14.672415+0000","last_became_peered":"2024-04-25T10:03:14.672415+0000","last_unstale":"2024-04-25T10:03:16.032098+0000","last_undegraded":"2024-04-25T10:03:16.032098+0000","last_fullsized":"2024-04-25T10:03:16.032098+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:59.362567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662222+0000","last_change":"2024-04-25T10:03:14.668281+0000","last_active":"2024-04-25T10:03:15.662222+0000","last_peered":"2024-04-25T10:03:15.662222+0000","last_clean":"2024-04-25T10:03:15.662222+0000","last_became_active":"2024-04-25T10:03:14.668002+0000","last_became_peered":"2024-04-25T10:03:14.668002+0000","last_unstale":"2024-04-25T10:03:15.662222+0000","last_undegraded":"2024-04-25T10:03:15.662222+0000","last_fullsized":"2024-04-25T10:03:15.662222+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:44.343693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"153'4","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663624+0000","last_change":"2024-04-25T10:03:13.674884+0000","last_active":"2024-04-25T10:03:15.663624+0000","last_peered":"2024-04-25T10:03:15.663624+0000","last_clean":"2024-04-25T10:03:15.663624+0000","last_became_active":"2024-04-25T10:03:13.674524+0000","last_became_peered":"2024-04-25T10:03:13.674524+0000","last_unstale":"2024-04-25T10:03:15.663624+0000","last_undegraded":"2024-04-25T10:03:15.663624+0000","last_fullsized":"2024-04-25T10:03:15.663624+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:30.635663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032047+0000","last_change":"2024-04-25T10:03:14.676233+0000","last_active":"2024-04-25T10:03:16.032047+0000","last_peered":"2024-04-25T10:03:16.032047+0000","last_clean":"2024-04-25T10:03:16.032047+0000","last_became_active":"2024-04-25T10:03:14.675959+0000","last_became_peered":"2024-04-25T10:03:14.675959+0000","last_unstale":"2024-04-25T10:03:16.032047+0000","last_undegraded":"2024-04-25T10:03:16.032047+0000","last_fullsized":"2024-04-25T10:03:16.032047+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:10.838625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661524+0000","last_change":"2024-04-25T10:03:13.679019+0000","last_active":"2024-04-25T10:03:15.661524+0000","last_peered":"2024-04-25T10:03:15.661524+0000","last_clean":"2024-04-25T10:03:15.661524+0000","last_became_active":"2024-04-25T10:03:13.678461+0000","last_became_peered":"2024-04-25T10:03:13.678461+0000","last_unstale":"2024-04-25T10:03:15.661524+0000","last_undegraded":"2024-04-25T10:03:15.661524+0000","last_fullsized":"2024-04-25T10:03:15.661524+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:08.056138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661367+0000","last_change":"2024-04-25T10:03:14.694677+0000","last_active":"2024-04-25T10:03:15.661367+0000","last_peered":"2024-04-25T10:03:15.661367+0000","last_clean":"2024-04-25T10:03:15.661367+0000","last_became_active":"2024-04-25T10:03:14.694420+0000","last_became_peered":"2024-04-25T10:03:14.694420+0000","last_unstale":"2024-04-25T10:03:15.661367+0000","last_undegraded":"2024-04-25T10:03:15.661367+0000","last_fullsized":"2024-04-25T10:03:15.661367+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:49.159049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032736+0000","last_change":"2024-04-25T10:03:13.674905+0000","last_active":"2024-04-25T10:03:16.032736+0000","last_peered":"2024-04-25T10:03:16.032736+0000","last_clean":"2024-04-25T10:03:16.032736+0000","last_became_active":"2024-04-25T10:03:13.674590+0000","last_became_peered":"2024-04-25T10:03:13.674590+0000","last_unstale":"2024-04-25T10:03:16.032736+0000","last_undegraded":"2024-04-25T10:03:16.032736+0000","last_fullsized":"2024-04-25T10:03:16.032736+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:10.022627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663029+0000","last_change":"2024-04-25T10:03:14.667446+0000","last_active":"2024-04-25T10:03:15.663029+0000","last_peered":"2024-04-25T10:03:15.663029+0000","last_clean":"2024-04-25T10:03:15.663029+0000","last_became_active":"2024-04-25T10:03:14.667170+0000","last_became_peered":"2024-04-25T10:03:14.667170+0000","last_unstale":"2024-04-25T10:03:15.663029+0000","last_undegraded":"2024-04-25T10:03:15.663029+0000","last_fullsized":"2024-04-25T10:03:15.663029+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:58.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695558+0000","last_change":"2024-04-25T10:03:13.673989+0000","last_active":"2024-04-25T10:03:15.695558+0000","last_peered":"2024-04-25T10:03:15.695558+0000","last_clean":"2024-04-25T10:03:15.695558+0000","last_became_active":"2024-04-25T10:03:13.673670+0000","last_became_peered":"2024-04-25T10:03:13.673670+0000","last_unstale":"2024-04-25T10:03:15.695558+0000","last_undegraded":"2024-04-25T10:03:15.695558+0000","last_fullsized":"2024-04-25T10:03:15.695558+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:40.623717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.033073+0000","last_change":"2024-04-25T10:03:14.684249+0000","last_active":"2024-04-25T10:03:16.033073+0000","last_peered":"2024-04-25T10:03:16.033073+0000","last_clean":"2024-04-25T10:03:16.033073+0000","last_became_active":"2024-04-25T10:03:14.683952+0000","last_became_peered":"2024-04-25T10:03:14.683952+0000","last_unstale":"2024-04-25T10:03:16.033073+0000","last_undegraded":"2024-04-25T10:03:16.033073+0000","last_fullsized":"2024-04-25T10:03:16.033073+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:12.246737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032435+0000","last_change":"2024-04-25T10:03:13.677056+0000","last_active":"2024-04-25T10:03:16.032435+0000","last_peered":"2024-04-25T10:03:16.032435+0000","last_clean":"2024-04-25T10:03:16.032435+0000","last_became_active":"2024-04-25T10:03:13.676751+0000","last_became_peered":"2024-04-25T10:03:13.676751+0000","last_unstale":"2024-04-25T10:03:16.032435+0000","last_undegraded":"2024-04-25T10:03:16.032435+0000","last_fullsized":"2024-04-25T10:03:16.032435+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:30.425664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695669+0000","last_change":"2024-04-25T10:03:14.692243+0000","last_active":"2024-04-25T10:03:15.695669+0000","last_peered":"2024-04-25T10:03:15.695669+0000","last_clean":"2024-04-25T10:03:15.695669+0000","last_became_active":"2024-04-25T10:03:14.691969+0000","last_became_peered":"2024-04-25T10:03:14.691969+0000","last_unstale":"2024-04-25T10:03:15.695669+0000","last_undegraded":"2024-04-25T10:03:15.695669+0000","last_fullsized":"2024-04-25T10:03:15.695669+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:40.965938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661463+0000","last_change":"2024-04-25T10:03:13.674339+0000","last_active":"2024-04-25T10:03:15.661463+0000","last_peered":"2024-04-25T10:03:15.661463+0000","last_clean":"2024-04-25T10:03:15.661463+0000","last_became_active":"2024-04-25T10:03:13.674074+0000","last_became_peered":"2024-04-25T10:03:13.674074+0000","last_unstale":"2024-04-25T10:03:15.661463+0000","last_undegraded":"2024-04-25T10:03:15.661463+0000","last_fullsized":"2024-04-25T10:03:15.661463+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:42.731289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695673+0000","last_change":"2024-04-25T10:03:14.693007+0000","last_active":"2024-04-25T10:03:15.695673+0000","last_peered":"2024-04-25T10:03:15.695673+0000","last_clean":"2024-04-25T10:03:15.695673+0000","last_became_active":"2024-04-25T10:03:14.692321+0000","last_became_peered":"2024-04-25T10:03:14.692321+0000","last_unstale":"2024-04-25T10:03:15.695673+0000","last_undegraded":"2024-04-25T10:03:15.695673+0000","last_fullsized":"2024-04-25T10:03:15.695673+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:29.049939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695399+0000","last_change":"2024-04-25T10:03:13.681030+0000","last_active":"2024-04-25T10:03:15.695399+0000","last_peered":"2024-04-25T10:03:15.695399+0000","last_clean":"2024-04-25T10:03:15.695399+0000","last_became_active":"2024-04-25T10:03:13.680669+0000","last_became_peered":"2024-04-25T10:03:13.680669+0000","last_unstale":"2024-04-25T10:03:15.695399+0000","last_undegraded":"2024-04-25T10:03:15.695399+0000","last_fullsized":"2024-04-25T10:03:15.695399+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:11.777899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663547+0000","last_change":"2024-04-25T10:03:14.671062+0000","last_active":"2024-04-25T10:03:15.663547+0000","last_peered":"2024-04-25T10:03:15.663547+0000","last_clean":"2024-04-25T10:03:15.663547+0000","last_became_active":"2024-04-25T10:03:14.670731+0000","last_became_peered":"2024-04-25T10:03:14.670731+0000","last_unstale":"2024-04-25T10:03:15.663547+0000","last_undegraded":"2024-04-25T10:03:15.663547+0000","last_fullsized":"2024-04-25T10:03:15.663547+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:17.438859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032058+0000","last_change":"2024-04-25T10:03:13.670929+0000","last_active":"2024-04-25T10:03:16.032058+0000","last_peered":"2024-04-25T10:03:16.032058+0000","last_clean":"2024-04-25T10:03:16.032058+0000","last_became_active":"2024-04-25T10:03:13.670647+0000","last_became_peered":"2024-04-25T10:03:13.670647+0000","last_unstale":"2024-04-25T10:03:16.032058+0000","last_undegraded":"2024-04-25T10:03:16.032058+0000","last_fullsized":"2024-04-25T10:03:16.032058+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:54.474670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662254+0000","last_change":"2024-04-25T10:03:14.667661+0000","last_active":"2024-04-25T10:03:15.662254+0000","last_peered":"2024-04-25T10:03:15.662254+0000","last_clean":"2024-04-25T10:03:15.662254+0000","last_became_active":"2024-04-25T10:03:14.667384+0000","last_became_peered":"2024-04-25T10:03:14.667384+0000","last_unstale":"2024-04-25T10:03:15.662254+0000","last_undegraded":"2024-04-25T10:03:15.662254+0000","last_fullsized":"2024-04-25T10:03:15.662254+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:23.927100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032503+0000","last_change":"2024-04-25T10:03:13.680361+0000","last_active":"2024-04-25T10:03:16.032503+0000","last_peered":"2024-04-25T10:03:16.032503+0000","last_clean":"2024-04-25T10:03:16.032503+0000","last_became_active":"2024-04-25T10:03:13.680075+0000","last_became_peered":"2024-04-25T10:03:13.680075+0000","last_unstale":"2024-04-25T10:03:16.032503+0000","last_undegraded":"2024-04-25T10:03:16.032503+0000","last_fullsized":"2024-04-25T10:03:16.032503+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:41.835542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032121+0000","last_change":"2024-04-25T10:03:14.679029+0000","last_active":"2024-04-25T10:03:16.032121+0000","last_peered":"2024-04-25T10:03:16.032121+0000","last_clean":"2024-04-25T10:03:16.032121+0000","last_became_active":"2024-04-25T10:03:14.678623+0000","last_became_peered":"2024-04-25T10:03:14.678623+0000","last_unstale":"2024-04-25T10:03:16.032121+0000","last_undegraded":"2024-04-25T10:03:16.032121+0000","last_fullsized":"2024-04-25T10:03:16.032121+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:09.058381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"153'3","reported_seq":17,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663312+0000","last_change":"2024-04-25T10:03:13.669655+0000","last_active":"2024-04-25T10:03:15.663312+0000","last_peered":"2024-04-25T10:03:15.663312+0000","last_clean":"2024-04-25T10:03:15.663312+0000","last_became_active":"2024-04-25T10:03:13.669366+0000","last_became_peered":"2024-04-25T10:03:13.669366+0000","last_unstale":"2024-04-25T10:03:15.663312+0000","last_undegraded":"2024-04-25T10:03:15.663312+0000","last_fullsized":"2024-04-25T10:03:15.663312+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:18.239201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695492+0000","last_change":"2024-04-25T10:03:14.688513+0000","last_active":"2024-04-25T10:03:15.695492+0000","last_peered":"2024-04-25T10:03:15.695492+0000","last_clean":"2024-04-25T10:03:15.695492+0000","last_became_active":"2024-04-25T10:03:14.687663+0000","last_became_peered":"2024-04-25T10:03:14.687663+0000","last_unstale":"2024-04-25T10:03:15.695492+0000","last_undegraded":"2024-04-25T10:03:15.695492+0000","last_fullsized":"2024-04-25T10:03:15.695492+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.827804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"153'6","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032704+0000","last_change":"2024-04-25T10:03:13.681202+0000","last_active":"2024-04-25T10:03:16.032704+0000","last_peered":"2024-04-25T10:03:16.032704+0000","last_clean":"2024-04-25T10:03:16.032704+0000","last_became_active":"2024-04-25T10:03:13.680736+0000","last_became_peered":"2024-04-25T10:03:13.680736+0000","last_unstale":"2024-04-25T10:03:16.032704+0000","last_undegraded":"2024-04-25T10:03:16.032704+0000","last_fullsized":"2024-04-25T10:03:16.032704+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:22.924857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032567+0000","last_change":"2024-04-25T10:03:14.672705+0000","last_active":"2024-04-25T10:03:16.032567+0000","last_peered":"2024-04-25T10:03:16.032567+0000","last_clean":"2024-04-25T10:03:16.032567+0000","last_became_active":"2024-04-25T10:03:14.672098+0000","last_became_peered":"2024-04-25T10:03:14.672098+0000","last_unstale":"2024-04-25T10:03:16.032567+0000","last_undegraded":"2024-04-25T10:03:16.032567+0000","last_fullsized":"2024-04-25T10:03:16.032567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:06.108933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661873+0000","last_change":"2024-04-25T10:03:13.671104+0000","last_active":"2024-04-25T10:03:15.661873+0000","last_peered":"2024-04-25T10:03:15.661873+0000","last_clean":"2024-04-25T10:03:15.661873+0000","last_became_active":"2024-04-25T10:03:13.670810+0000","last_became_peered":"2024-04-25T10:03:13.670810+0000","last_unstale":"2024-04-25T10:03:15.661873+0000","last_undegraded":"2024-04-25T10:03:15.661873+0000","last_fullsized":"2024-04-25T10:03:15.661873+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:26.230423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032411+0000","last_change":"2024-04-25T10:03:14.683180+0000","last_active":"2024-04-25T10:03:16.032411+0000","last_peered":"2024-04-25T10:03:16.032411+0000","last_clean":"2024-04-25T10:03:16.032411+0000","last_became_active":"2024-04-25T10:03:14.681022+0000","last_became_peered":"2024-04-25T10:03:14.681022+0000","last_unstale":"2024-04-25T10:03:16.032411+0000","last_undegraded":"2024-04-25T10:03:16.032411+0000","last_fullsized":"2024-04-25T10:03:16.032411+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:31.946288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032321+0000","last_change":"2024-04-25T10:03:13.672839+0000","last_active":"2024-04-25T10:03:16.032321+0000","last_peered":"2024-04-25T10:03:16.032321+0000","last_clean":"2024-04-25T10:03:16.032321+0000","last_became_active":"2024-04-25T10:03:13.671105+0000","last_became_peered":"2024-04-25T10:03:13.671105+0000","last_unstale":"2024-04-25T10:03:16.032321+0000","last_undegraded":"2024-04-25T10:03:16.032321+0000","last_fullsized":"2024-04-25T10:03:16.032321+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:30.944045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032890+0000","last_change":"2024-04-25T10:03:14.678441+0000","last_active":"2024-04-25T10:03:16.032890+0000","last_peered":"2024-04-25T10:03:16.032890+0000","last_clean":"2024-04-25T10:03:16.032890+0000","last_became_active":"2024-04-25T10:03:14.678057+0000","last_became_peered":"2024-04-25T10:03:14.678057+0000","last_unstale":"2024-04-25T10:03:16.032890+0000","last_undegraded":"2024-04-25T10:03:16.032890+0000","last_fullsized":"2024-04-25T10:03:16.032890+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:42.837785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032322+0000","last_change":"2024-04-25T10:03:13.680518+0000","last_active":"2024-04-25T10:03:16.032322+0000","last_peered":"2024-04-25T10:03:16.032322+0000","last_clean":"2024-04-25T10:03:16.032322+0000","last_became_active":"2024-04-25T10:03:13.680260+0000","last_became_peered":"2024-04-25T10:03:13.680260+0000","last_unstale":"2024-04-25T10:03:16.032322+0000","last_undegraded":"2024-04-25T10:03:16.032322+0000","last_fullsized":"2024-04-25T10:03:16.032322+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:04.430596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695620+0000","last_change":"2024-04-25T10:03:14.690620+0000","last_active":"2024-04-25T10:03:15.695620+0000","last_peered":"2024-04-25T10:03:15.695620+0000","last_clean":"2024-04-25T10:03:15.695620+0000","last_became_active":"2024-04-25T10:03:14.690062+0000","last_became_peered":"2024-04-25T10:03:14.690062+0000","last_unstale":"2024-04-25T10:03:15.695620+0000","last_undegraded":"2024-04-25T10:03:15.695620+0000","last_fullsized":"2024-04-25T10:03:15.695620+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:32.342537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"153'8","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032174+0000","last_change":"2024-04-25T10:03:13.675759+0000","last_active":"2024-04-25T10:03:16.032174+0000","last_peered":"2024-04-25T10:03:16.032174+0000","last_clean":"2024-04-25T10:03:16.032174+0000","last_became_active":"2024-04-25T10:03:13.675488+0000","last_became_peered":"2024-04-25T10:03:13.675488+0000","last_unstale":"2024-04-25T10:03:16.032174+0000","last_undegraded":"2024-04-25T10:03:16.032174+0000","last_fullsized":"2024-04-25T10:03:16.032174+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:46.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.031966+0000","last_change":"2024-04-25T10:03:14.693995+0000","last_active":"2024-04-25T10:03:16.031966+0000","last_peered":"2024-04-25T10:03:16.031966+0000","last_clean":"2024-04-25T10:03:16.031966+0000","last_became_active":"2024-04-25T10:03:14.693721+0000","last_became_peered":"2024-04-25T10:03:14.693721+0000","last_unstale":"2024-04-25T10:03:16.031966+0000","last_undegraded":"2024-04-25T10:03:16.031966+0000","last_fullsized":"2024-04-25T10:03:16.031966+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:46.234899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032693+0000","last_change":"2024-04-25T10:03:13.673662+0000","last_active":"2024-04-25T10:03:16.032693+0000","last_peered":"2024-04-25T10:03:16.032693+0000","last_clean":"2024-04-25T10:03:16.032693+0000","last_became_active":"2024-04-25T10:03:13.673355+0000","last_became_peered":"2024-04-25T10:03:13.673355+0000","last_unstale":"2024-04-25T10:03:16.032693+0000","last_undegraded":"2024-04-25T10:03:16.032693+0000","last_fullsized":"2024-04-25T10:03:16.032693+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:27.463128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032537+0000","last_change":"2024-04-25T10:03:14.677433+0000","last_active":"2024-04-25T10:03:16.032537+0000","last_peered":"2024-04-25T10:03:16.032537+0000","last_clean":"2024-04-25T10:03:16.032537+0000","last_became_active":"2024-04-25T10:03:14.676902+0000","last_became_peered":"2024-04-25T10:03:14.676902+0000","last_unstale":"2024-04-25T10:03:16.032537+0000","last_undegraded":"2024-04-25T10:03:16.032537+0000","last_fullsized":"2024-04-25T10:03:16.032537+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:17.438859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.033020+0000","last_change":"2024-04-25T10:03:13.678163+0000","last_active":"2024-04-25T10:03:16.033020+0000","last_peered":"2024-04-25T10:03:16.033020+0000","last_clean":"2024-04-25T10:03:16.033020+0000","last_became_active":"2024-04-25T10:03:13.677884+0000","last_became_peered":"2024-04-25T10:03:13.677884+0000","last_unstale":"2024-04-25T10:03:16.033020+0000","last_undegraded":"2024-04-25T10:03:16.033020+0000","last_fullsized":"2024-04-25T10:03:16.033020+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:01.697483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032567+0000","last_change":"2024-04-25T10:03:14.673455+0000","last_active":"2024-04-25T10:03:16.032567+0000","last_peered":"2024-04-25T10:03:16.032567+0000","last_clean":"2024-04-25T10:03:16.032567+0000","last_became_active":"2024-04-25T10:03:14.673174+0000","last_became_peered":"2024-04-25T10:03:14.673174+0000","last_unstale":"2024-04-25T10:03:16.032567+0000","last_undegraded":"2024-04-25T10:03:16.032567+0000","last_fullsized":"2024-04-25T10:03:16.032567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:32.794361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661808+0000","last_change":"2024-04-25T10:03:13.667893+0000","last_active":"2024-04-25T10:03:15.661808+0000","last_peered":"2024-04-25T10:03:15.661808+0000","last_clean":"2024-04-25T10:03:15.661808+0000","last_became_active":"2024-04-25T10:03:13.667483+0000","last_became_peered":"2024-04-25T10:03:13.667483+0000","last_unstale":"2024-04-25T10:03:15.661808+0000","last_undegraded":"2024-04-25T10:03:15.661808+0000","last_fullsized":"2024-04-25T10:03:15.661808+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:43.289383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032270+0000","last_change":"2024-04-25T10:03:14.680270+0000","last_active":"2024-04-25T10:03:16.032270+0000","last_peered":"2024-04-25T10:03:16.032270+0000","last_clean":"2024-04-25T10:03:16.032270+0000","last_became_active":"2024-04-25T10:03:14.679992+0000","last_became_peered":"2024-04-25T10:03:14.679992+0000","last_unstale":"2024-04-25T10:03:16.032270+0000","last_undegraded":"2024-04-25T10:03:16.032270+0000","last_fullsized":"2024-04-25T10:03:16.032270+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:32.008962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.033043+0000","last_change":"2024-04-25T10:03:13.678499+0000","last_active":"2024-04-25T10:03:16.033043+0000","last_peered":"2024-04-25T10:03:16.033043+0000","last_clean":"2024-04-25T10:03:16.033043+0000","last_became_active":"2024-04-25T10:03:13.678219+0000","last_became_peered":"2024-04-25T10:03:13.678219+0000","last_unstale":"2024-04-25T10:03:16.033043+0000","last_undegraded":"2024-04-25T10:03:16.033043+0000","last_fullsized":"2024-04-25T10:03:16.033043+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:45.649559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032593+0000","last_change":"2024-04-25T10:03:14.690253+0000","last_active":"2024-04-25T10:03:16.032593+0000","last_peered":"2024-04-25T10:03:16.032593+0000","last_clean":"2024-04-25T10:03:16.032593+0000","last_became_active":"2024-04-25T10:03:14.689918+0000","last_became_peered":"2024-04-25T10:03:14.689918+0000","last_unstale":"2024-04-25T10:03:16.032593+0000","last_undegraded":"2024-04-25T10:03:16.032593+0000","last_fullsized":"2024-04-25T10:03:16.032593+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:58.106950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661895+0000","last_change":"2024-04-25T10:03:13.670893+0000","last_active":"2024-04-25T10:03:15.661895+0000","last_peered":"2024-04-25T10:03:15.661895+0000","last_clean":"2024-04-25T10:03:15.661895+0000","last_became_active":"2024-04-25T10:03:13.670599+0000","last_became_peered":"2024-04-25T10:03:13.670599+0000","last_unstale":"2024-04-25T10:03:15.661895+0000","last_undegraded":"2024-04-25T10:03:15.661895+0000","last_fullsized":"2024-04-25T10:03:15.661895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:24.954579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661809+0000","last_change":"2024-04-25T10:03:14.687032+0000","last_active":"2024-04-25T10:03:15.661809+0000","last_peered":"2024-04-25T10:03:15.661809+0000","last_clean":"2024-04-25T10:03:15.661809+0000","last_became_active":"2024-04-25T10:03:14.686714+0000","last_became_peered":"2024-04-25T10:03:14.686714+0000","last_unstale":"2024-04-25T10:03:15.661809+0000","last_undegraded":"2024-04-25T10:03:15.661809+0000","last_fullsized":"2024-04-25T10:03:15.661809+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:03.973242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032641+0000","last_change":"2024-04-25T10:03:13.676633+0000","last_active":"2024-04-25T10:03:16.032641+0000","last_peered":"2024-04-25T10:03:16.032641+0000","last_clean":"2024-04-25T10:03:16.032641+0000","last_became_active":"2024-04-25T10:03:13.676353+0000","last_became_peered":"2024-04-25T10:03:13.676353+0000","last_unstale":"2024-04-25T10:03:16.032641+0000","last_undegraded":"2024-04-25T10:03:16.032641+0000","last_fullsized":"2024-04-25T10:03:16.032641+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:25.428507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661506+0000","last_change":"2024-04-25T10:03:14.675835+0000","last_active":"2024-04-25T10:03:15.661506+0000","last_peered":"2024-04-25T10:03:15.661506+0000","last_clean":"2024-04-25T10:03:15.661506+0000","last_became_active":"2024-04-25T10:03:14.675561+0000","last_became_peered":"2024-04-25T10:03:14.675561+0000","last_unstale":"2024-04-25T10:03:15.661506+0000","last_undegraded":"2024-04-25T10:03:15.661506+0000","last_fullsized":"2024-04-25T10:03:15.661506+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:31.454154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695547+0000","last_change":"2024-04-25T10:03:13.673943+0000","last_active":"2024-04-25T10:03:15.695547+0000","last_peered":"2024-04-25T10:03:15.695547+0000","last_clean":"2024-04-25T10:03:15.695547+0000","last_became_active":"2024-04-25T10:03:13.673646+0000","last_became_peered":"2024-04-25T10:03:13.673646+0000","last_unstale":"2024-04-25T10:03:15.695547+0000","last_undegraded":"2024-04-25T10:03:15.695547+0000","last_fullsized":"2024-04-25T10:03:15.695547+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:02.140986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663038+0000","last_change":"2024-04-25T10:03:14.676320+0000","last_active":"2024-04-25T10:03:15.663038+0000","last_peered":"2024-04-25T10:03:15.663038+0000","last_clean":"2024-04-25T10:03:15.663038+0000","last_became_active":"2024-04-25T10:03:14.675981+0000","last_became_peered":"2024-04-25T10:03:14.675981+0000","last_unstale":"2024-04-25T10:03:15.663038+0000","last_undegraded":"2024-04-25T10:03:15.663038+0000","last_fullsized":"2024-04-25T10:03:15.663038+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:27.232666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662332+0000","last_change":"2024-04-25T10:03:13.671428+0000","last_active":"2024-04-25T10:03:15.662332+0000","last_peered":"2024-04-25T10:03:15.662332+0000","last_clean":"2024-04-25T10:03:15.662332+0000","last_became_active":"2024-04-25T10:03:13.671080+0000","last_became_peered":"2024-04-25T10:03:13.671080+0000","last_unstale":"2024-04-25T10:03:15.662332+0000","last_undegraded":"2024-04-25T10:03:15.662332+0000","last_fullsized":"2024-04-25T10:03:15.662332+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:53.770956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032066+0000","last_change":"2024-04-25T10:03:14.674783+0000","last_active":"2024-04-25T10:03:16.032066+0000","last_peered":"2024-04-25T10:03:16.032066+0000","last_clean":"2024-04-25T10:03:16.032066+0000","last_became_active":"2024-04-25T10:03:14.674513+0000","last_became_peered":"2024-04-25T10:03:14.674513+0000","last_unstale":"2024-04-25T10:03:16.032066+0000","last_undegraded":"2024-04-25T10:03:16.032066+0000","last_fullsized":"2024-04-25T10:03:16.032066+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:52.242049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661552+0000","last_change":"2024-04-25T10:03:13.669724+0000","last_active":"2024-04-25T10:03:15.661552+0000","last_peered":"2024-04-25T10:03:15.661552+0000","last_clean":"2024-04-25T10:03:15.661552+0000","last_became_active":"2024-04-25T10:03:13.669446+0000","last_became_peered":"2024-04-25T10:03:13.669446+0000","last_unstale":"2024-04-25T10:03:15.661552+0000","last_undegraded":"2024-04-25T10:03:15.661552+0000","last_fullsized":"2024-04-25T10:03:15.661552+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:09.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032122+0000","last_change":"2024-04-25T10:03:14.696579+0000","last_active":"2024-04-25T10:03:16.032122+0000","last_peered":"2024-04-25T10:03:16.032122+0000","last_clean":"2024-04-25T10:03:16.032122+0000","last_became_active":"2024-04-25T10:03:14.696291+0000","last_became_peered":"2024-04-25T10:03:14.696291+0000","last_unstale":"2024-04-25T10:03:16.032122+0000","last_undegraded":"2024-04-25T10:03:16.032122+0000","last_fullsized":"2024-04-25T10:03:16.032122+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:53.141987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"154'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032973+0000","last_change":"2024-04-25T10:03:13.681265+0000","last_active":"2024-04-25T10:03:16.032973+0000","last_peered":"2024-04-25T10:03:16.032973+0000","last_clean":"2024-04-25T10:03:16.032973+0000","last_became_active":"2024-04-25T10:03:13.680824+0000","last_became_peered":"2024-04-25T10:03:13.680824+0000","last_unstale":"2024-04-25T10:03:16.032973+0000","last_undegraded":"2024-04-25T10:03:16.032973+0000","last_fullsized":"2024-04-25T10:03:16.032973+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:40.866203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695539+0000","last_change":"2024-04-25T10:03:14.692915+0000","last_active":"2024-04-25T10:03:15.695539+0000","last_peered":"2024-04-25T10:03:15.695539+0000","last_clean":"2024-04-25T10:03:15.695539+0000","last_became_active":"2024-04-25T10:03:14.692006+0000","last_became_peered":"2024-04-25T10:03:14.692006+0000","last_unstale":"2024-04-25T10:03:15.695539+0000","last_undegraded":"2024-04-25T10:03:15.695539+0000","last_fullsized":"2024-04-25T10:03:15.695539+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:42.744369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032353+0000","last_change":"2024-04-25T10:03:13.670694+0000","last_active":"2024-04-25T10:03:16.032353+0000","last_peered":"2024-04-25T10:03:16.032353+0000","last_clean":"2024-04-25T10:03:16.032353+0000","last_became_active":"2024-04-25T10:03:13.670408+0000","last_became_peered":"2024-04-25T10:03:13.670408+0000","last_unstale":"2024-04-25T10:03:16.032353+0000","last_undegraded":"2024-04-25T10:03:16.032353+0000","last_fullsized":"2024-04-25T10:03:16.032353+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:52.139744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032337+0000","last_change":"2024-04-25T10:03:14.693604+0000","last_active":"2024-04-25T10:03:16.032337+0000","last_peered":"2024-04-25T10:03:16.032337+0000","last_clean":"2024-04-25T10:03:16.032337+0000","last_became_active":"2024-04-25T10:03:14.693315+0000","last_became_peered":"2024-04-25T10:03:14.693315+0000","last_unstale":"2024-04-25T10:03:16.032337+0000","last_undegraded":"2024-04-25T10:03:16.032337+0000","last_fullsized":"2024-04-25T10:03:16.032337+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:41.868446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695686+0000","last_change":"2024-04-25T10:03:13.679719+0000","last_active":"2024-04-25T10:03:15.695686+0000","last_peered":"2024-04-25T10:03:15.695686+0000","last_clean":"2024-04-25T10:03:15.695686+0000","last_became_active":"2024-04-25T10:03:13.679453+0000","last_became_peered":"2024-04-25T10:03:13.679453+0000","last_unstale":"2024-04-25T10:03:15.695686+0000","last_undegraded":"2024-04-25T10:03:15.695686+0000","last_fullsized":"2024-04-25T10:03:15.695686+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:07.634628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663006+0000","last_change":"2024-04-25T10:03:14.676318+0000","last_active":"2024-04-25T10:03:15.663006+0000","last_peered":"2024-04-25T10:03:15.663006+0000","last_clean":"2024-04-25T10:03:15.663006+0000","last_became_active":"2024-04-25T10:03:14.676037+0000","last_became_peered":"2024-04-25T10:03:14.676037+0000","last_unstale":"2024-04-25T10:03:15.663006+0000","last_undegraded":"2024-04-25T10:03:15.663006+0000","last_fullsized":"2024-04-25T10:03:15.663006+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:27.054273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663488+0000","last_change":"2024-04-25T10:03:13.670937+0000","last_active":"2024-04-25T10:03:15.663488+0000","last_peered":"2024-04-25T10:03:15.663488+0000","last_clean":"2024-04-25T10:03:15.663488+0000","last_became_active":"2024-04-25T10:03:13.670561+0000","last_became_peered":"2024-04-25T10:03:13.670561+0000","last_unstale":"2024-04-25T10:03:15.663488+0000","last_undegraded":"2024-04-25T10:03:15.663488+0000","last_fullsized":"2024-04-25T10:03:15.663488+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:04.722407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662724+0000","last_change":"2024-04-25T10:03:14.668203+0000","last_active":"2024-04-25T10:03:15.662724+0000","last_peered":"2024-04-25T10:03:15.662724+0000","last_clean":"2024-04-25T10:03:15.662724+0000","last_became_active":"2024-04-25T10:03:14.667905+0000","last_became_peered":"2024-04-25T10:03:14.667905+0000","last_unstale":"2024-04-25T10:03:15.662724+0000","last_undegraded":"2024-04-25T10:03:15.662724+0000","last_fullsized":"2024-04-25T10:03:15.662724+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:14.453093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032426+0000","last_change":"2024-04-25T10:03:13.671727+0000","last_active":"2024-04-25T10:03:16.032426+0000","last_peered":"2024-04-25T10:03:16.032426+0000","last_clean":"2024-04-25T10:03:16.032426+0000","last_became_active":"2024-04-25T10:03:13.671439+0000","last_became_peered":"2024-04-25T10:03:13.671439+0000","last_unstale":"2024-04-25T10:03:16.032426+0000","last_undegraded":"2024-04-25T10:03:16.032426+0000","last_fullsized":"2024-04-25T10:03:16.032426+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.341450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032829+0000","last_change":"2024-04-25T10:03:14.689063+0000","last_active":"2024-04-25T10:03:16.032829+0000","last_peered":"2024-04-25T10:03:16.032829+0000","last_clean":"2024-04-25T10:03:16.032829+0000","last_became_active":"2024-04-25T10:03:14.688747+0000","last_became_peered":"2024-04-25T10:03:14.688747+0000","last_unstale":"2024-04-25T10:03:16.032829+0000","last_undegraded":"2024-04-25T10:03:16.032829+0000","last_fullsized":"2024-04-25T10:03:16.032829+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:39.228952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032435+0000","last_change":"2024-04-25T10:03:13.674966+0000","last_active":"2024-04-25T10:03:16.032435+0000","last_peered":"2024-04-25T10:03:16.032435+0000","last_clean":"2024-04-25T10:03:16.032435+0000","last_became_active":"2024-04-25T10:03:13.674633+0000","last_became_peered":"2024-04-25T10:03:13.674633+0000","last_unstale":"2024-04-25T10:03:16.032435+0000","last_undegraded":"2024-04-25T10:03:16.032435+0000","last_fullsized":"2024-04-25T10:03:16.032435+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:58.961175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695657+0000","last_change":"2024-04-25T10:03:14.690356+0000","last_active":"2024-04-25T10:03:15.695657+0000","last_peered":"2024-04-25T10:03:15.695657+0000","last_clean":"2024-04-25T10:03:15.695657+0000","last_became_active":"2024-04-25T10:03:14.689751+0000","last_became_peered":"2024-04-25T10:03:14.689751+0000","last_unstale":"2024-04-25T10:03:15.695657+0000","last_undegraded":"2024-04-25T10:03:15.695657+0000","last_fullsized":"2024-04-25T10:03:15.695657+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:29.903213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661164+0000","last_change":"2024-04-25T10:03:13.670435+0000","last_active":"2024-04-25T10:03:15.661164+0000","last_peered":"2024-04-25T10:03:15.661164+0000","last_clean":"2024-04-25T10:03:15.661164+0000","last_became_active":"2024-04-25T10:03:13.670146+0000","last_became_peered":"2024-04-25T10:03:13.670146+0000","last_unstale":"2024-04-25T10:03:15.661164+0000","last_undegraded":"2024-04-25T10:03:15.661164+0000","last_fullsized":"2024-04-25T10:03:15.661164+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:57.556755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662567+0000","last_change":"2024-04-25T10:03:14.676525+0000","last_active":"2024-04-25T10:03:15.662567+0000","last_peered":"2024-04-25T10:03:15.662567+0000","last_clean":"2024-04-25T10:03:15.662567+0000","last_became_active":"2024-04-25T10:03:14.675930+0000","last_became_peered":"2024-04-25T10:03:14.675930+0000","last_unstale":"2024-04-25T10:03:15.662567+0000","last_undegraded":"2024-04-25T10:03:15.662567+0000","last_fullsized":"2024-04-25T10:03:15.662567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:12.246737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032895+0000","last_change":"2024-04-25T10:03:13.681369+0000","last_active":"2024-04-25T10:03:16.032895+0000","last_peered":"2024-04-25T10:03:16.032895+0000","last_clean":"2024-04-25T10:03:16.032895+0000","last_became_active":"2024-04-25T10:03:13.681075+0000","last_became_peered":"2024-04-25T10:03:13.681075+0000","last_unstale":"2024-04-25T10:03:16.032895+0000","last_undegraded":"2024-04-25T10:03:16.032895+0000","last_fullsized":"2024-04-25T10:03:16.032895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:45.232656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":443,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662442+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:03:15.662442+0000","last_peered":"2024-04-25T10:03:15.662442+0000","last_clean":"2024-04-25T10:03:15.662442+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:03:15.662442+0000","last_undegraded":"2024-04-25T10:03:15.662442+0000","last_fullsized":"2024-04-25T10:03:15.662442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51704,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320136,"statfs":{"total":96636764160,"available":96583819264,"internally_reserved":0,"allocated":2531328,"data_stored":1498834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51120,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320720,"statfs":{"total":96636764160,"available":96584417280,"internally_reserved":0,"allocated":1937408,"data_stored":908466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51128,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320712,"statfs":{"total":96636764160,"available":96584409088,"internally_reserved":0,"allocated":1945600,"data_stored":908590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59456,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312384,"statfs":{"total":96636764160,"available":96575881216,"internally_reserved":0,"allocated":1953792,"data_stored":909230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59464,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312376,"statfs":{"total":96636764160,"available":96575873024,"internally_reserved":0,"allocated":1957888,"data_stored":910682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574977,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55284,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316556,"statfs":{"total":96636764160,"available":96580153344,"internally_reserved":0,"allocated":1937408,"data_stored":908466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55296,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316544,"statfs":{"total":96636764160,"available":96580141056,"internally_reserved":0,"allocated":1949696,"data_stored":910078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55872,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315968,"statfs":{"total":96636764160,"available":96579551232,"internally_reserved":0,"allocated":2539520,"data_stored":1498938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:19.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:03:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.517+0000 7fd014f5c640 1 Processor -- start 2024-04-25T10:03:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.517+0000 7fd014f5c640 1 -- start start 2024-04-25T10:03:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.517+0000 7fd014f5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01015dc70 0x7fd0100d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.517+0000 7fd014f5c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01006c220 con 0x7fd01006ce10 2024-04-25T10:03:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.517+0000 7fd014f5c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0100d0080 con 0x7fd01009a450 2024-04-25T10:03:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.517+0000 7fd014f5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01009a8b0 con 0x7fd01015dc70 2024-04-25T10:03:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.518+0000 7fd00ed9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd01006ce10 0x7fd0100cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46142/0 (socket says 172.21.15.19:46142) 2024-04-25T10:03:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.518+0000 7fd00ed9d640 1 -- 172.21.15.19:0/1507748418 learned_addr learned my addr 172.21.15.19:0/1507748418 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.518+0000 7fd00e59c640 1 --2- 172.21.15.19:0/1507748418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01015dc70 0x7fd0100d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.518+0000 7fd00dd9b640 1 -- 172.21.15.19:0/1507748418 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928756979 0 0) 0x7fd01006c220 con 0x7fd01006ce10 2024-04-25T10:03:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.518+0000 7fd00dd9b640 1 -- 172.21.15.19:0/1507748418 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcffc003520 con 0x7fd01006ce10 2024-04-25T10:03:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.518+0000 7fd00dd9b640 1 -- 172.21.15.19:0/1507748418 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004314743 0 0) 0x7fd0100d0080 con 0x7fd01009a450 2024-04-25T10:03:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.518+0000 7fd00dd9b640 1 -- 172.21.15.19:0/1507748418 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd01006c220 con 0x7fd01009a450 2024-04-25T10:03:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.518+0000 7fd00dd9b640 1 -- 172.21.15.19:0/1507748418 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3258723071 0 0) 0x7fcffc003520 con 0x7fd01006ce10 2024-04-25T10:03:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.518+0000 7fd00e59c640 1 -- 172.21.15.19:0/1507748418 >> v1:172.21.15.112:6789/0 conn(0x7fd01009a450 legacy=0x7fd01006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.518+0000 7fd00e59c640 1 -- 172.21.15.19:0/1507748418 >> v1:172.21.15.19:6789/0 conn(0x7fd01006ce10 legacy=0x7fd0100cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.518+0000 7fd00e59c640 1 -- 172.21.15.19:0/1507748418 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0100d0360 con 0x7fd01015dc70 2024-04-25T10:03:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.519+0000 7fd00e59c640 1 --2- 172.21.15.19:0/1507748418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01015dc70 0x7fd0100d1120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd0000097d0 tx=0x7fd00002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ca8353bfbdecf0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.519+0000 7fd00dd9b640 1 -- 172.21.15.19:0/1507748418 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000037070 con 0x7fd01015dc70 2024-04-25T10:03:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.519+0000 7fd00dd9b640 1 -- 172.21.15.19:0/1507748418 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00002ac30 con 0x7fd01015dc70 2024-04-25T10:03:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.519+0000 7fd00dd9b640 1 -- 172.21.15.19:0/1507748418 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0080019f0 con 0x7fd01015dc70 2024-04-25T10:03:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.520+0000 7fd014f5c640 1 -- 172.21.15.19:0/1507748418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01015dc70 msgr2=0x7fd0100d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.520+0000 7fd014f5c640 1 --2- 172.21.15.19:0/1507748418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01015dc70 0x7fd0100d1120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd0000097d0 tx=0x7fd00002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.520+0000 7fd014f5c640 1 -- 172.21.15.19:0/1507748418 shutdown_connections 2024-04-25T10:03:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.520+0000 7fd014f5c640 1 --2- 172.21.15.19:0/1507748418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01015dc70 0x7fd0100d1120 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.520+0000 7fd014f5c640 1 -- 172.21.15.19:0/1507748418 >> 172.21.15.19:0/1507748418 conn(0x7fd0100b3270 msgr2=0x7fd0100b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.520+0000 7fd014f5c640 1 -- 172.21.15.19:0/1507748418 shutdown_connections 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.520+0000 7fd014f5c640 1 -- 172.21.15.19:0/1507748418 wait complete. 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.521+0000 7fd014f5c640 1 Processor -- start 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.521+0000 7fd014f5c640 1 -- start start 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd014f5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd01006ce10 0x7fd01014eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd014f5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01009a450 0x7fd01009ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd014f5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd01015dc70 0x7fd0100c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd014f5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0100d0920 con 0x7fd01006ce10 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd014f5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0100d0c90 con 0x7fd01015dc70 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd014f5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01015e050 con 0x7fd01009a450 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd00e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd01006ce10 0x7fd01014eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd00e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd01006ce10 0x7fd01014eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49654/0 (socket says 172.21.15.19:49654) 2024-04-25T10:03:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd00e59c640 1 -- 172.21.15.19:0/1376661358 learned_addr learned my addr 172.21.15.19:0/1376661358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd007fff640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01009a450 0x7fd01009ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd00ed9d640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd01015dc70 0x7fd0100c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd00e59c640 1 -- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01009a450 msgr2=0x7fd01009ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd00e59c640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01009a450 0x7fd01009ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd00e59c640 1 -- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd01015dc70 msgr2=0x7fd0100c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd00e59c640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd01015dc70 0x7fd0100c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd00e59c640 1 -- 172.21.15.19:0/1376661358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01009ba10 con 0x7fd01006ce10 2024-04-25T10:03:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.522+0000 7fd007fff640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01009a450 0x7fd01009ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.523+0000 7fd00e59c640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd01006ce10 0x7fd01014eb80 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fd000035040 tx=0x7fd0000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.523+0000 7fd006ffd640 1 -- 172.21.15.19:0/1376661358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000042020 con 0x7fd01006ce10 2024-04-25T10:03:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.523+0000 7fd006ffd640 1 -- 172.21.15.19:0/1376661358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00003d690 con 0x7fd01006ce10 2024-04-25T10:03:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.523+0000 7fd014f5c640 1 -- 172.21.15.19:0/1376661358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01009bcf0 con 0x7fd01006ce10 2024-04-25T10:03:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.523+0000 7fd006ffd640 1 -- 172.21.15.19:0/1376661358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000037040 con 0x7fd01006ce10 2024-04-25T10:03:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.523+0000 7fd014f5c640 1 -- 172.21.15.19:0/1376661358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01009c2b0 con 0x7fd01006ce10 2024-04-25T10:03:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.524+0000 7fd014f5c640 1 -- 172.21.15.19:0/1376661358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0100caa90 con 0x7fd01006ce10 2024-04-25T10:03:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.524+0000 7fd006ffd640 1 -- 172.21.15.19:0/1376661358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd000045020 con 0x7fd01006ce10 2024-04-25T10:03:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.525+0000 7fd006ffd640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 0x7fcff006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.525+0000 7fd006ffd640 1 -- 172.21.15.19:0/1376661358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd0000aaca0 con 0x7fd01006ce10 2024-04-25T10:03:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.525+0000 7fd007fff640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 0x7fcff006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.525+0000 7fd007fff640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 0x7fcff006d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd01015cb80 tx=0x7fcff8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.529+0000 7fd006ffd640 1 -- 172.21.15.19:0/1376661358 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd00004c030 con 0x7fd01006ce10 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.778+0000 7fd014f5c640 1 -- 172.21.15.19:0/1376661358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd010073990 con 0x7fd01006ce10 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.778+0000 7fd006ffd640 1 -- 172.21.15.19:0/1376661358 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1309 (secure 0 0 0) 0x7fd000079990 con 0x7fd01006ce10 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":637,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":154,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":449867776,"bytes_avail":772644245504,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":102,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"a","status":"up:active","gid":10224}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T10:03:03.049999+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 -- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 msgr2=0x7fcff006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 0x7fcff006d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd01015cb80 tx=0x7fcff8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 -- 172.21.15.19:0/1376661358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd01006ce10 msgr2=0x7fd01014eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd01006ce10 0x7fd01014eb80 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fd000035040 tx=0x7fd0000046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 -- 172.21.15.19:0/1376661358 shutdown_connections 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 0x7fcff006d990 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd01015dc70 0x7fd0100c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd01009a450 0x7fd01009ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 --2- 172.21.15.19:0/1376661358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd01006ce10 0x7fd01014eb80 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 -- 172.21.15.19:0/1376661358 >> 172.21.15.19:0/1376661358 conn(0x7fd0100b3270 msgr2=0x7fd0100b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 -- 172.21.15.19:0/1376661358 shutdown_connections 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.780+0000 7fd014f5c640 1 -- 172.21.15.19:0/1376661358 wait complete. 2024-04-25T10:03:19.797 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:03:19.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.894+0000 7f40f8838640 1 Processor -- start 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.894+0000 7f40f8838640 1 -- start start 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.895+0000 7f40f8838640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40f40d2e90 0x7f40f40d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.895+0000 7f40f8838640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f40b9940 con 0x7f40f40d0900 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.895+0000 7f40f8838640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f40d1df0 con 0x7f40f415dd30 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.895+0000 7f40f8838640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40d0d70 con 0x7f40f40d2e90 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.895+0000 7f40f2d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f40f40d0900 0x7f40f40b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46152/0 (socket says 172.21.15.19:46152) 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.895+0000 7f40f2d9d640 1 -- 172.21.15.19:0/4165593729 learned_addr learned my addr 172.21.15.19:0/4165593729 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.895+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026116592 0 0) 0x7f40f40b9940 con 0x7f40f40d0900 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.895+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40dc003520 con 0x7f40f40d0900 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.895+0000 7f40f1d9b640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40f40d2e90 0x7f40f40d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2507671949 0 0) 0x7f40dc003520 con 0x7f40f40d0900 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40f40d2e90 msgr2=0x7f40f40d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40f40d2e90 0x7f40f40d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 >> v1:172.21.15.112:6789/0 conn(0x7f40f415dd30 legacy=0x7f40f40b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f40d20d0 con 0x7f40f40d0900 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f40ec001670 con 0x7f40f40d0900 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 >> v1:172.21.15.19:6789/0 conn(0x7f40f40d0900 legacy=0x7f40f40b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc0039e0 0x7f40dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40f40d2e90 0x7f40dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc004d40 0x7f40dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40d0d70 con 0x7f40dc004d40 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc00baf0 con 0x7f40dc0039e0 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc00bcf0 con 0x7f40f40d2e90 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f2d9d640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc004d40 0x7f40dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f259c640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc0039e0 0x7f40dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f1d9b640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40f40d2e90 0x7f40dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f2d9d640 1 -- 172.21.15.19:0/4165593729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40f40d2e90 msgr2=0x7f40dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f2d9d640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40f40d2e90 0x7f40dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f2d9d640 1 -- 172.21.15.19:0/4165593729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc0039e0 msgr2=0x7f40dc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f2d9d640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc0039e0 0x7f40dc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.896+0000 7f40f2d9d640 1 -- 172.21.15.19:0/4165593729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc003520 con 0x7f40dc004d40 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.897+0000 7f40f2d9d640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc004d40 0x7f40dc0051c0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00c210 tx=0x7f40dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50618530a359aa91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.897+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec007d10 con 0x7f40dc004d40 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.897+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec0058d0 con 0x7f40dc004d40 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.897+0000 7f40f159a640 1 -- 172.21.15.19:0/4165593729 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec005ca0 con 0x7f40dc004d40 2024-04-25T10:03:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.898+0000 7f40f8838640 1 -- 172.21.15.19:0/4165593729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc004d40 msgr2=0x7f40dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.898+0000 7f40f8838640 1 --2- 172.21.15.19:0/4165593729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40dc004d40 0x7f40dc0051c0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00c210 tx=0x7f40dc005830 comp rx=0 tx=0).stop 2024-04-25T10:03:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.898+0000 7f40f2d9d640 1 -- 172.21.15.19:0/4165593729 reap_dead start 2024-04-25T10:03:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.898+0000 7f40f8838640 1 -- 172.21.15.19:0/4165593729 shutdown_connections 2024-04-25T10:03:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.898+0000 7f40f8838640 1 -- 172.21.15.19:0/4165593729 >> 172.21.15.19:0/4165593729 conn(0x7f40f4061c30 msgr2=0x7f40f406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.898+0000 7f40f8838640 1 -- 172.21.15.19:0/4165593729 shutdown_connections 2024-04-25T10:03:19.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.898+0000 7f40f8838640 1 -- 172.21.15.19:0/4165593729 wait complete. 2024-04-25T10:03:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.899+0000 7f40f8838640 1 Processor -- start 2024-04-25T10:03:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.899+0000 7f40f8838640 1 -- start start 2024-04-25T10:03:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f8838640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40dc004d40 0x7f40f4152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f8838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f40ccb60 0x7f40f40cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f8838640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f40ccfc0 0x7f40f4150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f8838640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40d2970 con 0x7f40f40ccfc0 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f8838640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40d2690 con 0x7f40f40ccb60 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f8838640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40d1a10 con 0x7f40dc004d40 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f40ccfc0 0x7f40f4150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f40ccfc0 0x7f40f4150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49666/0 (socket says 172.21.15.19:49666) 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f2d9d640 1 -- 172.21.15.19:0/1385062038 learned_addr learned my addr 172.21.15.19:0/1385062038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f259c640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40dc004d40 0x7f40f4152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f1d9b640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f40ccb60 0x7f40f40cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f2d9d640 1 -- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40dc004d40 msgr2=0x7f40f4152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f2d9d640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40dc004d40 0x7f40f4152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f2d9d640 1 -- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f40ccb60 msgr2=0x7f40f40cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f2d9d640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f40ccb60 0x7f40f40cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f2d9d640 1 -- 172.21.15.19:0/1385062038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f40cbac0 con 0x7f40f40ccfc0 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f1d9b640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f40ccb60 0x7f40f40cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.900+0000 7f40f2d9d640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f40ccfc0 0x7f40f4150630 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f40ec001970 tx=0x7f40f40d1fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.901+0000 7f40e2ffd640 1 -- 172.21.15.19:0/1385062038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec007d10 con 0x7f40f40ccfc0 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.901+0000 7f40e2ffd640 1 -- 172.21.15.19:0/1385062038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec005030 con 0x7f40f40ccfc0 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.901+0000 7f40f8838640 1 -- 172.21.15.19:0/1385062038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f40cbda0 con 0x7f40f40ccfc0 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.901+0000 7f40e2ffd640 1 -- 172.21.15.19:0/1385062038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec001c70 con 0x7f40f40ccfc0 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.901+0000 7f40f8838640 1 -- 172.21.15.19:0/1385062038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f40cc360 con 0x7f40f40ccfc0 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.902+0000 7f40f8838640 1 -- 172.21.15.19:0/1385062038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f4073990 con 0x7f40f40ccfc0 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.902+0000 7f40e2ffd640 1 -- 172.21.15.19:0/1385062038 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40ec044020 con 0x7f40f40ccfc0 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.903+0000 7f40e2ffd640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f40d406b4d0 0x7f40d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.903+0000 7f40e2ffd640 1 -- 172.21.15.19:0/1385062038 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f40ec02f050 con 0x7f40f40ccfc0 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.903+0000 7f40f259c640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f40d406b4d0 0x7f40d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.903+0000 7f40f259c640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f40d406b4d0 0x7f40d406d990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f40e4001ee0 tx=0x7f40e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:19.907+0000 7f40e2ffd640 1 -- 172.21.15.19:0/1385062038 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40ec0780d0 con 0x7f40f40ccfc0 2024-04-25T10:03:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.083+0000 7f40f8838640 1 -- 172.21.15.19:0/1385062038 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40f415d150 con 0x7f40d406b4d0 2024-04-25T10:03:20.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.089+0000 7f40e2ffd640 1 -- 172.21.15.19:0/1385062038 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252480 (secure 0 0 0) 0x7f40f415d150 con 0x7f40d406b4d0 2024-04-25T10:03:20.097 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:20.098 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:03:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 -- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f40d406b4d0 msgr2=0x7f40d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f40d406b4d0 0x7f40d406d990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f40e4001ee0 tx=0x7f40e4009040 comp rx=0 tx=0).stop 2024-04-25T10:03:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 -- 172.21.15.19:0/1385062038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f40ccfc0 msgr2=0x7f40f4150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f40ccfc0 0x7f40f4150630 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f40ec001970 tx=0x7f40f40d1fc0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 -- 172.21.15.19:0/1385062038 shutdown_connections 2024-04-25T10:03:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f40d406b4d0 0x7f40d406d990 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f40ccfc0 0x7f40f4150630 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40f40ccb60 0x7f40f40cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 --2- 172.21.15.19:0/1385062038 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f40dc004d40 0x7f40f4152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 -- 172.21.15.19:0/1385062038 >> 172.21.15.19:0/1385062038 conn(0x7f40f4061c30 msgr2=0x7f40f40b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 -- 172.21.15.19:0/1385062038 shutdown_connections 2024-04-25T10:03:20.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:20.092+0000 7f40f8838640 1 -- 172.21.15.19:0/1385062038 wait complete. 2024-04-25T10:03:20.108 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-04-25T10:03:19.053588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439324,"kb_used_data":16360,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754535396,"statfs":{"total":773094113280,"available":772644245504,"internally_reserved":0,"allocated":16752640,"data_stored":8453284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001432"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663538+0000","last_change":"2024-04-25T10:03:14.674988+0000","last_active":"2024-04-25T10:03:15.663538+0000","last_peered":"2024-04-25T10:03:15.663538+0000","last_clean":"2024-04-25T10:03:15.663538+0000","last_became_active":"2024-04-25T10:03:14.674705+0000","last_became_peered":"2024-04-25T10:03:14.674705+0000","last_unstale":"2024-04-25T10:03:15.663538+0000","last_undegraded":"2024-04-25T10:03:15.663538+0000","last_fullsized":"2024-04-25T10:03:15.663538+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:55.709812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663091+0000","last_change":"2024-04-25T10:03:14.677891+0000","last_active":"2024-04-25T10:03:15.663091+0000","last_peered":"2024-04-25T10:03:15.663091+0000","last_clean":"2024-04-25T10:03:15.663091+0000","last_became_active":"2024-04-25T10:03:14.676509+0000","last_became_peered":"2024-04-25T10:03:14.676509+0000","last_unstale":"2024-04-25T10:03:15.663091+0000","last_undegraded":"2024-04-25T10:03:15.663091+0000","last_fullsized":"2024-04-25T10:03:15.663091+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:25.956823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663156+0000","last_change":"2024-04-25T10:03:14.693467+0000","last_active":"2024-04-25T10:03:15.663156+0000","last_peered":"2024-04-25T10:03:15.663156+0000","last_clean":"2024-04-25T10:03:15.663156+0000","last_became_active":"2024-04-25T10:03:14.693229+0000","last_became_peered":"2024-04-25T10:03:14.693229+0000","last_unstale":"2024-04-25T10:03:15.663156+0000","last_undegraded":"2024-04-25T10:03:15.663156+0000","last_fullsized":"2024-04-25T10:03:15.663156+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:04.852884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662858+0000","last_change":"2024-04-25T10:03:14.668329+0000","last_active":"2024-04-25T10:03:15.662858+0000","last_peered":"2024-04-25T10:03:15.662858+0000","last_clean":"2024-04-25T10:03:15.662858+0000","last_became_active":"2024-04-25T10:03:14.668042+0000","last_became_peered":"2024-04-25T10:03:14.668042+0000","last_unstale":"2024-04-25T10:03:15.662858+0000","last_undegraded":"2024-04-25T10:03:15.662858+0000","last_fullsized":"2024-04-25T10:03:15.662858+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:23.955400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661748+0000","last_change":"2024-04-25T10:03:14.681919+0000","last_active":"2024-04-25T10:03:15.661748+0000","last_peered":"2024-04-25T10:03:15.661748+0000","last_clean":"2024-04-25T10:03:15.661748+0000","last_became_active":"2024-04-25T10:03:14.681625+0000","last_became_peered":"2024-04-25T10:03:14.681625+0000","last_unstale":"2024-04-25T10:03:15.661748+0000","last_undegraded":"2024-04-25T10:03:15.661748+0000","last_fullsized":"2024-04-25T10:03:15.661748+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.895682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662459+0000","last_change":"2024-04-25T10:03:14.664822+0000","last_active":"2024-04-25T10:03:15.662459+0000","last_peered":"2024-04-25T10:03:15.662459+0000","last_clean":"2024-04-25T10:03:15.662459+0000","last_became_active":"2024-04-25T10:03:14.664548+0000","last_became_peered":"2024-04-25T10:03:14.664548+0000","last_unstale":"2024-04-25T10:03:15.662459+0000","last_undegraded":"2024-04-25T10:03:15.662459+0000","last_fullsized":"2024-04-25T10:03:15.662459+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:35.733949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032783+0000","last_change":"2024-04-25T10:03:14.696283+0000","last_active":"2024-04-25T10:03:16.032783+0000","last_peered":"2024-04-25T10:03:16.032783+0000","last_clean":"2024-04-25T10:03:16.032783+0000","last_became_active":"2024-04-25T10:03:14.696049+0000","last_became_peered":"2024-04-25T10:03:14.696049+0000","last_unstale":"2024-04-25T10:03:16.032783+0000","last_undegraded":"2024-04-25T10:03:16.032783+0000","last_fullsized":"2024-04-25T10:03:16.032783+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:36.801468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.031940+0000","last_change":"2024-04-25T10:03:14.677271+0000","last_active":"2024-04-25T10:03:16.031940+0000","last_peered":"2024-04-25T10:03:16.031940+0000","last_clean":"2024-04-25T10:03:16.031940+0000","last_became_active":"2024-04-25T10:03:14.676987+0000","last_became_peered":"2024-04-25T10:03:14.676987+0000","last_unstale":"2024-04-25T10:03:16.031940+0000","last_undegraded":"2024-04-25T10:03:16.031940+0000","last_fullsized":"2024-04-25T10:03:16.031940+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:28.465371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032612+0000","last_change":"2024-04-25T10:03:14.681985+0000","last_active":"2024-04-25T10:03:16.032612+0000","last_peered":"2024-04-25T10:03:16.032612+0000","last_clean":"2024-04-25T10:03:16.032612+0000","last_became_active":"2024-04-25T10:03:14.681702+0000","last_became_peered":"2024-04-25T10:03:14.681702+0000","last_unstale":"2024-04-25T10:03:16.032612+0000","last_undegraded":"2024-04-25T10:03:16.032612+0000","last_fullsized":"2024-04-25T10:03:16.032612+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:44.343693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032758+0000","last_change":"2024-04-25T10:03:14.683567+0000","last_active":"2024-04-25T10:03:16.032758+0000","last_peered":"2024-04-25T10:03:16.032758+0000","last_clean":"2024-04-25T10:03:16.032758+0000","last_became_active":"2024-04-25T10:03:14.681589+0000","last_became_peered":"2024-04-25T10:03:14.681589+0000","last_unstale":"2024-04-25T10:03:16.032758+0000","last_undegraded":"2024-04-25T10:03:16.032758+0000","last_fullsized":"2024-04-25T10:03:16.032758+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:23.927100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032162+0000","last_change":"2024-04-25T10:03:14.681356+0000","last_active":"2024-04-25T10:03:16.032162+0000","last_peered":"2024-04-25T10:03:16.032162+0000","last_clean":"2024-04-25T10:03:16.032162+0000","last_became_active":"2024-04-25T10:03:14.680990+0000","last_became_peered":"2024-04-25T10:03:14.680990+0000","last_unstale":"2024-04-25T10:03:16.032162+0000","last_undegraded":"2024-04-25T10:03:16.032162+0000","last_fullsized":"2024-04-25T10:03:16.032162+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:52.425930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663219+0000","last_change":"2024-04-25T10:03:14.665666+0000","last_active":"2024-04-25T10:03:15.663219+0000","last_peered":"2024-04-25T10:03:15.663219+0000","last_clean":"2024-04-25T10:03:15.663219+0000","last_became_active":"2024-04-25T10:03:14.665328+0000","last_became_peered":"2024-04-25T10:03:14.665328+0000","last_unstale":"2024-04-25T10:03:15.663219+0000","last_undegraded":"2024-04-25T10:03:15.663219+0000","last_fullsized":"2024-04-25T10:03:15.663219+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:10.838625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032537+0000","last_change":"2024-04-25T10:03:14.674760+0000","last_active":"2024-04-25T10:03:16.032537+0000","last_peered":"2024-04-25T10:03:16.032537+0000","last_clean":"2024-04-25T10:03:16.032537+0000","last_became_active":"2024-04-25T10:03:14.674456+0000","last_became_peered":"2024-04-25T10:03:14.674456+0000","last_unstale":"2024-04-25T10:03:16.032537+0000","last_undegraded":"2024-04-25T10:03:16.032537+0000","last_fullsized":"2024-04-25T10:03:16.032537+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:44.291626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661326+0000","last_change":"2024-04-25T10:03:14.694517+0000","last_active":"2024-04-25T10:03:15.661326+0000","last_peered":"2024-04-25T10:03:15.661326+0000","last_clean":"2024-04-25T10:03:15.661326+0000","last_became_active":"2024-04-25T10:03:14.694069+0000","last_became_peered":"2024-04-25T10:03:14.694069+0000","last_unstale":"2024-04-25T10:03:15.661326+0000","last_undegraded":"2024-04-25T10:03:15.661326+0000","last_fullsized":"2024-04-25T10:03:15.661326+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:13.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032343+0000","last_change":"2024-04-25T10:03:14.683731+0000","last_active":"2024-04-25T10:03:16.032343+0000","last_peered":"2024-04-25T10:03:16.032343+0000","last_clean":"2024-04-25T10:03:16.032343+0000","last_became_active":"2024-04-25T10:03:14.683402+0000","last_became_peered":"2024-04-25T10:03:14.683402+0000","last_unstale":"2024-04-25T10:03:16.032343+0000","last_undegraded":"2024-04-25T10:03:16.032343+0000","last_fullsized":"2024-04-25T10:03:16.032343+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:39.505850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662812+0000","last_change":"2024-04-25T10:03:14.669397+0000","last_active":"2024-04-25T10:03:15.662812+0000","last_peered":"2024-04-25T10:03:15.662812+0000","last_clean":"2024-04-25T10:03:15.662812+0000","last_became_active":"2024-04-25T10:03:14.668368+0000","last_became_peered":"2024-04-25T10:03:14.668368+0000","last_unstale":"2024-04-25T10:03:15.662812+0000","last_undegraded":"2024-04-25T10:03:15.662812+0000","last_fullsized":"2024-04-25T10:03:15.662812+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:28.463537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032771+0000","last_change":"2024-04-25T10:03:14.681114+0000","last_active":"2024-04-25T10:03:16.032771+0000","last_peered":"2024-04-25T10:03:16.032771+0000","last_clean":"2024-04-25T10:03:16.032771+0000","last_became_active":"2024-04-25T10:03:14.680775+0000","last_became_peered":"2024-04-25T10:03:14.680775+0000","last_unstale":"2024-04-25T10:03:16.032771+0000","last_undegraded":"2024-04-25T10:03:16.032771+0000","last_fullsized":"2024-04-25T10:03:16.032771+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:35.733949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032399+0000","last_change":"2024-04-25T10:03:14.674046+0000","last_active":"2024-04-25T10:03:16.032399+0000","last_peered":"2024-04-25T10:03:16.032399+0000","last_clean":"2024-04-25T10:03:16.032399+0000","last_became_active":"2024-04-25T10:03:14.673789+0000","last_became_peered":"2024-04-25T10:03:14.673789+0000","last_unstale":"2024-04-25T10:03:16.032399+0000","last_undegraded":"2024-04-25T10:03:16.032399+0000","last_fullsized":"2024-04-25T10:03:16.032399+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:04.852884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032669+0000","last_change":"2024-04-25T10:03:14.675106+0000","last_active":"2024-04-25T10:03:16.032669+0000","last_peered":"2024-04-25T10:03:16.032669+0000","last_clean":"2024-04-25T10:03:16.032669+0000","last_became_active":"2024-04-25T10:03:14.674810+0000","last_became_peered":"2024-04-25T10:03:14.674810+0000","last_unstale":"2024-04-25T10:03:16.032669+0000","last_undegraded":"2024-04-25T10:03:16.032669+0000","last_fullsized":"2024-04-25T10:03:16.032669+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:31.422540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695708+0000","last_change":"2024-04-25T10:03:14.691234+0000","last_active":"2024-04-25T10:03:15.695708+0000","last_peered":"2024-04-25T10:03:15.695708+0000","last_clean":"2024-04-25T10:03:15.695708+0000","last_became_active":"2024-04-25T10:03:14.690950+0000","last_became_peered":"2024-04-25T10:03:14.690950+0000","last_unstale":"2024-04-25T10:03:15.695708+0000","last_undegraded":"2024-04-25T10:03:15.695708+0000","last_fullsized":"2024-04-25T10:03:15.695708+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:20.623444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032667+0000","last_change":"2024-04-25T10:03:14.689443+0000","last_active":"2024-04-25T10:03:16.032667+0000","last_peered":"2024-04-25T10:03:16.032667+0000","last_clean":"2024-04-25T10:03:16.032667+0000","last_became_active":"2024-04-25T10:03:14.689151+0000","last_became_peered":"2024-04-25T10:03:14.689151+0000","last_unstale":"2024-04-25T10:03:16.032667+0000","last_undegraded":"2024-04-25T10:03:16.032667+0000","last_fullsized":"2024-04-25T10:03:16.032667+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:16.734137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662133+0000","last_change":"2024-04-25T10:03:14.677533+0000","last_active":"2024-04-25T10:03:15.662133+0000","last_peered":"2024-04-25T10:03:15.662133+0000","last_clean":"2024-04-25T10:03:15.662133+0000","last_became_active":"2024-04-25T10:03:14.677261+0000","last_became_peered":"2024-04-25T10:03:14.677261+0000","last_unstale":"2024-04-25T10:03:15.662133+0000","last_undegraded":"2024-04-25T10:03:15.662133+0000","last_fullsized":"2024-04-25T10:03:15.662133+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:31.946288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661340+0000","last_change":"2024-04-25T10:03:14.680014+0000","last_active":"2024-04-25T10:03:15.661340+0000","last_peered":"2024-04-25T10:03:15.661340+0000","last_clean":"2024-04-25T10:03:15.661340+0000","last_became_active":"2024-04-25T10:03:14.679525+0000","last_became_peered":"2024-04-25T10:03:14.679525+0000","last_unstale":"2024-04-25T10:03:15.661340+0000","last_undegraded":"2024-04-25T10:03:15.661340+0000","last_fullsized":"2024-04-25T10:03:15.661340+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:09.709075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663186+0000","last_change":"2024-04-25T10:03:14.668828+0000","last_active":"2024-04-25T10:03:15.663186+0000","last_peered":"2024-04-25T10:03:15.663186+0000","last_clean":"2024-04-25T10:03:15.663186+0000","last_became_active":"2024-04-25T10:03:14.668000+0000","last_became_peered":"2024-04-25T10:03:14.668000+0000","last_unstale":"2024-04-25T10:03:15.663186+0000","last_undegraded":"2024-04-25T10:03:15.663186+0000","last_fullsized":"2024-04-25T10:03:15.663186+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:59.362567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032734+0000","last_change":"2024-04-25T10:03:14.673302+0000","last_active":"2024-04-25T10:03:16.032734+0000","last_peered":"2024-04-25T10:03:16.032734+0000","last_clean":"2024-04-25T10:03:16.032734+0000","last_became_active":"2024-04-25T10:03:14.673003+0000","last_became_peered":"2024-04-25T10:03:14.673003+0000","last_unstale":"2024-04-25T10:03:16.032734+0000","last_undegraded":"2024-04-25T10:03:16.032734+0000","last_fullsized":"2024-04-25T10:03:16.032734+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:05.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695652+0000","last_change":"2024-04-25T10:03:14.693330+0000","last_active":"2024-04-25T10:03:15.695652+0000","last_peered":"2024-04-25T10:03:15.695652+0000","last_clean":"2024-04-25T10:03:15.695652+0000","last_became_active":"2024-04-25T10:03:14.692664+0000","last_became_peered":"2024-04-25T10:03:14.692664+0000","last_unstale":"2024-04-25T10:03:15.695652+0000","last_undegraded":"2024-04-25T10:03:15.695652+0000","last_fullsized":"2024-04-25T10:03:15.695652+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:33.297902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662549+0000","last_change":"2024-04-25T10:03:14.680270+0000","last_active":"2024-04-25T10:03:15.662549+0000","last_peered":"2024-04-25T10:03:15.662549+0000","last_clean":"2024-04-25T10:03:15.662549+0000","last_became_active":"2024-04-25T10:03:14.680005+0000","last_became_peered":"2024-04-25T10:03:14.680005+0000","last_unstale":"2024-04-25T10:03:15.662549+0000","last_undegraded":"2024-04-25T10:03:15.662549+0000","last_fullsized":"2024-04-25T10:03:15.662549+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:36.801468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032504+0000","last_change":"2024-04-25T10:03:14.666917+0000","last_active":"2024-04-25T10:03:16.032504+0000","last_peered":"2024-04-25T10:03:16.032504+0000","last_clean":"2024-04-25T10:03:16.032504+0000","last_became_active":"2024-04-25T10:03:14.666580+0000","last_became_peered":"2024-04-25T10:03:14.666580+0000","last_unstale":"2024-04-25T10:03:16.032504+0000","last_undegraded":"2024-04-25T10:03:16.032504+0000","last_fullsized":"2024-04-25T10:03:16.032504+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:43.733533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662990+0000","last_change":"2024-04-25T10:03:14.675411+0000","last_active":"2024-04-25T10:03:15.662990+0000","last_peered":"2024-04-25T10:03:15.662990+0000","last_clean":"2024-04-25T10:03:15.662990+0000","last_became_active":"2024-04-25T10:03:14.675071+0000","last_became_peered":"2024-04-25T10:03:14.675071+0000","last_unstale":"2024-04-25T10:03:15.662990+0000","last_undegraded":"2024-04-25T10:03:15.662990+0000","last_fullsized":"2024-04-25T10:03:15.662990+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:34.175141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032478+0000","last_change":"2024-04-25T10:03:14.680054+0000","last_active":"2024-04-25T10:03:16.032478+0000","last_peered":"2024-04-25T10:03:16.032478+0000","last_clean":"2024-04-25T10:03:16.032478+0000","last_became_active":"2024-04-25T10:03:14.679780+0000","last_became_peered":"2024-04-25T10:03:14.679780+0000","last_unstale":"2024-04-25T10:03:16.032478+0000","last_undegraded":"2024-04-25T10:03:16.032478+0000","last_fullsized":"2024-04-25T10:03:16.032478+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:55.709812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032215+0000","last_change":"2024-04-25T10:03:14.678473+0000","last_active":"2024-04-25T10:03:16.032215+0000","last_peered":"2024-04-25T10:03:16.032215+0000","last_clean":"2024-04-25T10:03:16.032215+0000","last_became_active":"2024-04-25T10:03:14.677596+0000","last_became_peered":"2024-04-25T10:03:14.677596+0000","last_unstale":"2024-04-25T10:03:16.032215+0000","last_undegraded":"2024-04-25T10:03:16.032215+0000","last_fullsized":"2024-04-25T10:03:16.032215+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:04.438091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032098+0000","last_change":"2024-04-25T10:03:14.672679+0000","last_active":"2024-04-25T10:03:16.032098+0000","last_peered":"2024-04-25T10:03:16.032098+0000","last_clean":"2024-04-25T10:03:16.032098+0000","last_became_active":"2024-04-25T10:03:14.672415+0000","last_became_peered":"2024-04-25T10:03:14.672415+0000","last_unstale":"2024-04-25T10:03:16.032098+0000","last_undegraded":"2024-04-25T10:03:16.032098+0000","last_fullsized":"2024-04-25T10:03:16.032098+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:59.362567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662222+0000","last_change":"2024-04-25T10:03:14.668281+0000","last_active":"2024-04-25T10:03:15.662222+0000","last_peered":"2024-04-25T10:03:15.662222+0000","last_clean":"2024-04-25T10:03:15.662222+0000","last_became_active":"2024-04-25T10:03:14.668002+0000","last_became_peered":"2024-04-25T10:03:14.668002+0000","last_unstale":"2024-04-25T10:03:15.662222+0000","last_undegraded":"2024-04-25T10:03:15.662222+0000","last_fullsized":"2024-04-25T10:03:15.662222+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:44.343693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"153'4","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663624+0000","last_change":"2024-04-25T10:03:13.674884+0000","last_active":"2024-04-25T10:03:15.663624+0000","last_peered":"2024-04-25T10:03:15.663624+0000","last_clean":"2024-04-25T10:03:15.663624+0000","last_became_active":"2024-04-25T10:03:13.674524+0000","last_became_peered":"2024-04-25T10:03:13.674524+0000","last_unstale":"2024-04-25T10:03:15.663624+0000","last_undegraded":"2024-04-25T10:03:15.663624+0000","last_fullsized":"2024-04-25T10:03:15.663624+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:30.635663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032047+0000","last_change":"2024-04-25T10:03:14.676233+0000","last_active":"2024-04-25T10:03:16.032047+0000","last_peered":"2024-04-25T10:03:16.032047+0000","last_clean":"2024-04-25T10:03:16.032047+0000","last_became_active":"2024-04-25T10:03:14.675959+0000","last_became_peered":"2024-04-25T10:03:14.675959+0000","last_unstale":"2024-04-25T10:03:16.032047+0000","last_undegraded":"2024-04-25T10:03:16.032047+0000","last_fullsized":"2024-04-25T10:03:16.032047+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:10.838625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661524+0000","last_change":"2024-04-25T10:03:13.679019+0000","last_active":"2024-04-25T10:03:15.661524+0000","last_peered":"2024-04-25T10:03:15.661524+0000","last_clean":"2024-04-25T10:03:15.661524+0000","last_became_active":"2024-04-25T10:03:13.678461+0000","last_became_peered":"2024-04-25T10:03:13.678461+0000","last_unstale":"2024-04-25T10:03:15.661524+0000","last_undegraded":"2024-04-25T10:03:15.661524+0000","last_fullsized":"2024-04-25T10:03:15.661524+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:08.056138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661367+0000","last_change":"2024-04-25T10:03:14.694677+0000","last_active":"2024-04-25T10:03:15.661367+0000","last_peered":"2024-04-25T10:03:15.661367+0000","last_clean":"2024-04-25T10:03:15.661367+0000","last_became_active":"2024-04-25T10:03:14.694420+0000","last_became_peered":"2024-04-25T10:03:14.694420+0000","last_unstale":"2024-04-25T10:03:15.661367+0000","last_undegraded":"2024-04-25T10:03:15.661367+0000","last_fullsized":"2024-04-25T10:03:15.661367+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:49.159049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032736+0000","last_change":"2024-04-25T10:03:13.674905+0000","last_active":"2024-04-25T10:03:16.032736+0000","last_peered":"2024-04-25T10:03:16.032736+0000","last_clean":"2024-04-25T10:03:16.032736+0000","last_became_active":"2024-04-25T10:03:13.674590+0000","last_became_peered":"2024-04-25T10:03:13.674590+0000","last_unstale":"2024-04-25T10:03:16.032736+0000","last_undegraded":"2024-04-25T10:03:16.032736+0000","last_fullsized":"2024-04-25T10:03:16.032736+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:10.022627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663029+0000","last_change":"2024-04-25T10:03:14.667446+0000","last_active":"2024-04-25T10:03:15.663029+0000","last_peered":"2024-04-25T10:03:15.663029+0000","last_clean":"2024-04-25T10:03:15.663029+0000","last_became_active":"2024-04-25T10:03:14.667170+0000","last_became_peered":"2024-04-25T10:03:14.667170+0000","last_unstale":"2024-04-25T10:03:15.663029+0000","last_undegraded":"2024-04-25T10:03:15.663029+0000","last_fullsized":"2024-04-25T10:03:15.663029+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:58.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695558+0000","last_change":"2024-04-25T10:03:13.673989+0000","last_active":"2024-04-25T10:03:15.695558+0000","last_peered":"2024-04-25T10:03:15.695558+0000","last_clean":"2024-04-25T10:03:15.695558+0000","last_became_active":"2024-04-25T10:03:13.673670+0000","last_became_peered":"2024-04-25T10:03:13.673670+0000","last_unstale":"2024-04-25T10:03:15.695558+0000","last_undegraded":"2024-04-25T10:03:15.695558+0000","last_fullsized":"2024-04-25T10:03:15.695558+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:40.623717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.033073+0000","last_change":"2024-04-25T10:03:14.684249+0000","last_active":"2024-04-25T10:03:16.033073+0000","last_peered":"2024-04-25T10:03:16.033073+0000","last_clean":"2024-04-25T10:03:16.033073+0000","last_became_active":"2024-04-25T10:03:14.683952+0000","last_became_peered":"2024-04-25T10:03:14.683952+0000","last_unstale":"2024-04-25T10:03:16.033073+0000","last_undegraded":"2024-04-25T10:03:16.033073+0000","last_fullsized":"2024-04-25T10:03:16.033073+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:12.246737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032435+0000","last_change":"2024-04-25T10:03:13.677056+0000","last_active":"2024-04-25T10:03:16.032435+0000","last_peered":"2024-04-25T10:03:16.032435+0000","last_clean":"2024-04-25T10:03:16.032435+0000","last_became_active":"2024-04-25T10:03:13.676751+0000","last_became_peered":"2024-04-25T10:03:13.676751+0000","last_unstale":"2024-04-25T10:03:16.032435+0000","last_undegraded":"2024-04-25T10:03:16.032435+0000","last_fullsized":"2024-04-25T10:03:16.032435+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:30.425664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695669+0000","last_change":"2024-04-25T10:03:14.692243+0000","last_active":"2024-04-25T10:03:15.695669+0000","last_peered":"2024-04-25T10:03:15.695669+0000","last_clean":"2024-04-25T10:03:15.695669+0000","last_became_active":"2024-04-25T10:03:14.691969+0000","last_became_peered":"2024-04-25T10:03:14.691969+0000","last_unstale":"2024-04-25T10:03:15.695669+0000","last_undegraded":"2024-04-25T10:03:15.695669+0000","last_fullsized":"2024-04-25T10:03:15.695669+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:40.965938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661463+0000","last_change":"2024-04-25T10:03:13.674339+0000","last_active":"2024-04-25T10:03:15.661463+0000","last_peered":"2024-04-25T10:03:15.661463+0000","last_clean":"2024-04-25T10:03:15.661463+0000","last_became_active":"2024-04-25T10:03:13.674074+0000","last_became_peered":"2024-04-25T10:03:13.674074+0000","last_unstale":"2024-04-25T10:03:15.661463+0000","last_undegraded":"2024-04-25T10:03:15.661463+0000","last_fullsized":"2024-04-25T10:03:15.661463+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:42.731289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695673+0000","last_change":"2024-04-25T10:03:14.693007+0000","last_active":"2024-04-25T10:03:15.695673+0000","last_peered":"2024-04-25T10:03:15.695673+0000","last_clean":"2024-04-25T10:03:15.695673+0000","last_became_active":"2024-04-25T10:03:14.692321+0000","last_became_peered":"2024-04-25T10:03:14.692321+0000","last_unstale":"2024-04-25T10:03:15.695673+0000","last_undegraded":"2024-04-25T10:03:15.695673+0000","last_fullsized":"2024-04-25T10:03:15.695673+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:29.049939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695399+0000","last_change":"2024-04-25T10:03:13.681030+0000","last_active":"2024-04-25T10:03:15.695399+0000","last_peered":"2024-04-25T10:03:15.695399+0000","last_clean":"2024-04-25T10:03:15.695399+0000","last_became_active":"2024-04-25T10:03:13.680669+0000","last_became_peered":"2024-04-25T10:03:13.680669+0000","last_unstale":"2024-04-25T10:03:15.695399+0000","last_undegraded":"2024-04-25T10:03:15.695399+0000","last_fullsized":"2024-04-25T10:03:15.695399+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:11.777899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663547+0000","last_change":"2024-04-25T10:03:14.671062+0000","last_active":"2024-04-25T10:03:15.663547+0000","last_peered":"2024-04-25T10:03:15.663547+0000","last_clean":"2024-04-25T10:03:15.663547+0000","last_became_active":"2024-04-25T10:03:14.670731+0000","last_became_peered":"2024-04-25T10:03:14.670731+0000","last_unstale":"2024-04-25T10:03:15.663547+0000","last_undegraded":"2024-04-25T10:03:15.663547+0000","last_fullsized":"2024-04-25T10:03:15.663547+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:17.438859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032058+0000","last_change":"2024-04-25T10:03:13.670929+0000","last_active":"2024-04-25T10:03:16.032058+0000","last_peered":"2024-04-25T10:03:16.032058+0000","last_clean":"2024-04-25T10:03:16.032058+0000","last_became_active":"2024-04-25T10:03:13.670647+0000","last_became_peered":"2024-04-25T10:03:13.670647+0000","last_unstale":"2024-04-25T10:03:16.032058+0000","last_undegraded":"2024-04-25T10:03:16.032058+0000","last_fullsized":"2024-04-25T10:03:16.032058+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:54.474670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662254+0000","last_change":"2024-04-25T10:03:14.667661+0000","last_active":"2024-04-25T10:03:15.662254+0000","last_peered":"2024-04-25T10:03:15.662254+0000","last_clean":"2024-04-25T10:03:15.662254+0000","last_became_active":"2024-04-25T10:03:14.667384+0000","last_became_peered":"2024-04-25T10:03:14.667384+0000","last_unstale":"2024-04-25T10:03:15.662254+0000","last_undegraded":"2024-04-25T10:03:15.662254+0000","last_fullsized":"2024-04-25T10:03:15.662254+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:23.927100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032503+0000","last_change":"2024-04-25T10:03:13.680361+0000","last_active":"2024-04-25T10:03:16.032503+0000","last_peered":"2024-04-25T10:03:16.032503+0000","last_clean":"2024-04-25T10:03:16.032503+0000","last_became_active":"2024-04-25T10:03:13.680075+0000","last_became_peered":"2024-04-25T10:03:13.680075+0000","last_unstale":"2024-04-25T10:03:16.032503+0000","last_undegraded":"2024-04-25T10:03:16.032503+0000","last_fullsized":"2024-04-25T10:03:16.032503+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:41.835542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032121+0000","last_change":"2024-04-25T10:03:14.679029+0000","last_active":"2024-04-25T10:03:16.032121+0000","last_peered":"2024-04-25T10:03:16.032121+0000","last_clean":"2024-04-25T10:03:16.032121+0000","last_became_active":"2024-04-25T10:03:14.678623+0000","last_became_peered":"2024-04-25T10:03:14.678623+0000","last_unstale":"2024-04-25T10:03:16.032121+0000","last_undegraded":"2024-04-25T10:03:16.032121+0000","last_fullsized":"2024-04-25T10:03:16.032121+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:09.058381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"153'3","reported_seq":17,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663312+0000","last_change":"2024-04-25T10:03:13.669655+0000","last_active":"2024-04-25T10:03:15.663312+0000","last_peered":"2024-04-25T10:03:15.663312+0000","last_clean":"2024-04-25T10:03:15.663312+0000","last_became_active":"2024-04-25T10:03:13.669366+0000","last_became_peered":"2024-04-25T10:03:13.669366+0000","last_unstale":"2024-04-25T10:03:15.663312+0000","last_undegraded":"2024-04-25T10:03:15.663312+0000","last_fullsized":"2024-04-25T10:03:15.663312+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:18.239201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695492+0000","last_change":"2024-04-25T10:03:14.688513+0000","last_active":"2024-04-25T10:03:15.695492+0000","last_peered":"2024-04-25T10:03:15.695492+0000","last_clean":"2024-04-25T10:03:15.695492+0000","last_became_active":"2024-04-25T10:03:14.687663+0000","last_became_peered":"2024-04-25T10:03:14.687663+0000","last_unstale":"2024-04-25T10:03:15.695492+0000","last_undegraded":"2024-04-25T10:03:15.695492+0000","last_fullsized":"2024-04-25T10:03:15.695492+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.827804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"153'6","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032704+0000","last_change":"2024-04-25T10:03:13.681202+0000","last_active":"2024-04-25T10:03:16.032704+0000","last_peered":"2024-04-25T10:03:16.032704+0000","last_clean":"2024-04-25T10:03:16.032704+0000","last_became_active":"2024-04-25T10:03:13.680736+0000","last_became_peered":"2024-04-25T10:03:13.680736+0000","last_unstale":"2024-04-25T10:03:16.032704+0000","last_undegraded":"2024-04-25T10:03:16.032704+0000","last_fullsized":"2024-04-25T10:03:16.032704+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:22.924857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032567+0000","last_change":"2024-04-25T10:03:14.672705+0000","last_active":"2024-04-25T10:03:16.032567+0000","last_peered":"2024-04-25T10:03:16.032567+0000","last_clean":"2024-04-25T10:03:16.032567+0000","last_became_active":"2024-04-25T10:03:14.672098+0000","last_became_peered":"2024-04-25T10:03:14.672098+0000","last_unstale":"2024-04-25T10:03:16.032567+0000","last_undegraded":"2024-04-25T10:03:16.032567+0000","last_fullsized":"2024-04-25T10:03:16.032567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:06.108933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661873+0000","last_change":"2024-04-25T10:03:13.671104+0000","last_active":"2024-04-25T10:03:15.661873+0000","last_peered":"2024-04-25T10:03:15.661873+0000","last_clean":"2024-04-25T10:03:15.661873+0000","last_became_active":"2024-04-25T10:03:13.670810+0000","last_became_peered":"2024-04-25T10:03:13.670810+0000","last_unstale":"2024-04-25T10:03:15.661873+0000","last_undegraded":"2024-04-25T10:03:15.661873+0000","last_fullsized":"2024-04-25T10:03:15.661873+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:26.230423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032411+0000","last_change":"2024-04-25T10:03:14.683180+0000","last_active":"2024-04-25T10:03:16.032411+0000","last_peered":"2024-04-25T10:03:16.032411+0000","last_clean":"2024-04-25T10:03:16.032411+0000","last_became_active":"2024-04-25T10:03:14.681022+0000","last_became_peered":"2024-04-25T10:03:14.681022+0000","last_unstale":"2024-04-25T10:03:16.032411+0000","last_undegraded":"2024-04-25T10:03:16.032411+0000","last_fullsized":"2024-04-25T10:03:16.032411+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:31.946288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032321+0000","last_change":"2024-04-25T10:03:13.672839+0000","last_active":"2024-04-25T10:03:16.032321+0000","last_peered":"2024-04-25T10:03:16.032321+0000","last_clean":"2024-04-25T10:03:16.032321+0000","last_became_active":"2024-04-25T10:03:13.671105+0000","last_became_peered":"2024-04-25T10:03:13.671105+0000","last_unstale":"2024-04-25T10:03:16.032321+0000","last_undegraded":"2024-04-25T10:03:16.032321+0000","last_fullsized":"2024-04-25T10:03:16.032321+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:30.944045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032890+0000","last_change":"2024-04-25T10:03:14.678441+0000","last_active":"2024-04-25T10:03:16.032890+0000","last_peered":"2024-04-25T10:03:16.032890+0000","last_clean":"2024-04-25T10:03:16.032890+0000","last_became_active":"2024-04-25T10:03:14.678057+0000","last_became_peered":"2024-04-25T10:03:14.678057+0000","last_unstale":"2024-04-25T10:03:16.032890+0000","last_undegraded":"2024-04-25T10:03:16.032890+0000","last_fullsized":"2024-04-25T10:03:16.032890+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:42.837785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032322+0000","last_change":"2024-04-25T10:03:13.680518+0000","last_active":"2024-04-25T10:03:16.032322+0000","last_peered":"2024-04-25T10:03:16.032322+0000","last_clean":"2024-04-25T10:03:16.032322+0000","last_became_active":"2024-04-25T10:03:13.680260+0000","last_became_peered":"2024-04-25T10:03:13.680260+0000","last_unstale":"2024-04-25T10:03:16.032322+0000","last_undegraded":"2024-04-25T10:03:16.032322+0000","last_fullsized":"2024-04-25T10:03:16.032322+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:04.430596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695620+0000","last_change":"2024-04-25T10:03:14.690620+0000","last_active":"2024-04-25T10:03:15.695620+0000","last_peered":"2024-04-25T10:03:15.695620+0000","last_clean":"2024-04-25T10:03:15.695620+0000","last_became_active":"2024-04-25T10:03:14.690062+0000","last_became_peered":"2024-04-25T10:03:14.690062+0000","last_unstale":"2024-04-25T10:03:15.695620+0000","last_undegraded":"2024-04-25T10:03:15.695620+0000","last_fullsized":"2024-04-25T10:03:15.695620+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:32.342537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"153'8","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032174+0000","last_change":"2024-04-25T10:03:13.675759+0000","last_active":"2024-04-25T10:03:16.032174+0000","last_peered":"2024-04-25T10:03:16.032174+0000","last_clean":"2024-04-25T10:03:16.032174+0000","last_became_active":"2024-04-25T10:03:13.675488+0000","last_became_peered":"2024-04-25T10:03:13.675488+0000","last_unstale":"2024-04-25T10:03:16.032174+0000","last_undegraded":"2024-04-25T10:03:16.032174+0000","last_fullsized":"2024-04-25T10:03:16.032174+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:46.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.031966+0000","last_change":"2024-04-25T10:03:14.693995+0000","last_active":"2024-04-25T10:03:16.031966+0000","last_peered":"2024-04-25T10:03:16.031966+0000","last_clean":"2024-04-25T10:03:16.031966+0000","last_became_active":"2024-04-25T10:03:14.693721+0000","last_became_peered":"2024-04-25T10:03:14.693721+0000","last_unstale":"2024-04-25T10:03:16.031966+0000","last_undegraded":"2024-04-25T10:03:16.031966+0000","last_fullsized":"2024-04-25T10:03:16.031966+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:46.234899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032693+0000","last_change":"2024-04-25T10:03:13.673662+0000","last_active":"2024-04-25T10:03:16.032693+0000","last_peered":"2024-04-25T10:03:16.032693+0000","last_clean":"2024-04-25T10:03:16.032693+0000","last_became_active":"2024-04-25T10:03:13.673355+0000","last_became_peered":"2024-04-25T10:03:13.673355+0000","last_unstale":"2024-04-25T10:03:16.032693+0000","last_undegraded":"2024-04-25T10:03:16.032693+0000","last_fullsized":"2024-04-25T10:03:16.032693+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:27.463128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032537+0000","last_change":"2024-04-25T10:03:14.677433+0000","last_active":"2024-04-25T10:03:16.032537+0000","last_peered":"2024-04-25T10:03:16.032537+0000","last_clean":"2024-04-25T10:03:16.032537+0000","last_became_active":"2024-04-25T10:03:14.676902+0000","last_became_peered":"2024-04-25T10:03:14.676902+0000","last_unstale":"2024-04-25T10:03:16.032537+0000","last_undegraded":"2024-04-25T10:03:16.032537+0000","last_fullsized":"2024-04-25T10:03:16.032537+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:17.438859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.033020+0000","last_change":"2024-04-25T10:03:13.678163+0000","last_active":"2024-04-25T10:03:16.033020+0000","last_peered":"2024-04-25T10:03:16.033020+0000","last_clean":"2024-04-25T10:03:16.033020+0000","last_became_active":"2024-04-25T10:03:13.677884+0000","last_became_peered":"2024-04-25T10:03:13.677884+0000","last_unstale":"2024-04-25T10:03:16.033020+0000","last_undegraded":"2024-04-25T10:03:16.033020+0000","last_fullsized":"2024-04-25T10:03:16.033020+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:01.697483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032567+0000","last_change":"2024-04-25T10:03:14.673455+0000","last_active":"2024-04-25T10:03:16.032567+0000","last_peered":"2024-04-25T10:03:16.032567+0000","last_clean":"2024-04-25T10:03:16.032567+0000","last_became_active":"2024-04-25T10:03:14.673174+0000","last_became_peered":"2024-04-25T10:03:14.673174+0000","last_unstale":"2024-04-25T10:03:16.032567+0000","last_undegraded":"2024-04-25T10:03:16.032567+0000","last_fullsized":"2024-04-25T10:03:16.032567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:32.794361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661808+0000","last_change":"2024-04-25T10:03:13.667893+0000","last_active":"2024-04-25T10:03:15.661808+0000","last_peered":"2024-04-25T10:03:15.661808+0000","last_clean":"2024-04-25T10:03:15.661808+0000","last_became_active":"2024-04-25T10:03:13.667483+0000","last_became_peered":"2024-04-25T10:03:13.667483+0000","last_unstale":"2024-04-25T10:03:15.661808+0000","last_undegraded":"2024-04-25T10:03:15.661808+0000","last_fullsized":"2024-04-25T10:03:15.661808+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:43.289383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032270+0000","last_change":"2024-04-25T10:03:14.680270+0000","last_active":"2024-04-25T10:03:16.032270+0000","last_peered":"2024-04-25T10:03:16.032270+0000","last_clean":"2024-04-25T10:03:16.032270+0000","last_became_active":"2024-04-25T10:03:14.679992+0000","last_became_peered":"2024-04-25T10:03:14.679992+0000","last_unstale":"2024-04-25T10:03:16.032270+0000","last_undegraded":"2024-04-25T10:03:16.032270+0000","last_fullsized":"2024-04-25T10:03:16.032270+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:32.008962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.033043+0000","last_change":"2024-04-25T10:03:13.678499+0000","last_active":"2024-04-25T10:03:16.033043+0000","last_peered":"2024-04-25T10:03:16.033043+0000","last_clean":"2024-04-25T10:03:16.033043+0000","last_became_active":"2024-04-25T10:03:13.678219+0000","last_became_peered":"2024-04-25T10:03:13.678219+0000","last_unstale":"2024-04-25T10:03:16.033043+0000","last_undegraded":"2024-04-25T10:03:16.033043+0000","last_fullsized":"2024-04-25T10:03:16.033043+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:45.649559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032593+0000","last_change":"2024-04-25T10:03:14.690253+0000","last_active":"2024-04-25T10:03:16.032593+0000","last_peered":"2024-04-25T10:03:16.032593+0000","last_clean":"2024-04-25T10:03:16.032593+0000","last_became_active":"2024-04-25T10:03:14.689918+0000","last_became_peered":"2024-04-25T10:03:14.689918+0000","last_unstale":"2024-04-25T10:03:16.032593+0000","last_undegraded":"2024-04-25T10:03:16.032593+0000","last_fullsized":"2024-04-25T10:03:16.032593+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:58.106950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661895+0000","last_change":"2024-04-25T10:03:13.670893+0000","last_active":"2024-04-25T10:03:15.661895+0000","last_peered":"2024-04-25T10:03:15.661895+0000","last_clean":"2024-04-25T10:03:15.661895+0000","last_became_active":"2024-04-25T10:03:13.670599+0000","last_became_peered":"2024-04-25T10:03:13.670599+0000","last_unstale":"2024-04-25T10:03:15.661895+0000","last_undegraded":"2024-04-25T10:03:15.661895+0000","last_fullsized":"2024-04-25T10:03:15.661895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:24.954579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661809+0000","last_change":"2024-04-25T10:03:14.687032+0000","last_active":"2024-04-25T10:03:15.661809+0000","last_peered":"2024-04-25T10:03:15.661809+0000","last_clean":"2024-04-25T10:03:15.661809+0000","last_became_active":"2024-04-25T10:03:14.686714+0000","last_became_peered":"2024-04-25T10:03:14.686714+0000","last_unstale":"2024-04-25T10:03:15.661809+0000","last_undegraded":"2024-04-25T10:03:15.661809+0000","last_fullsized":"2024-04-25T10:03:15.661809+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:03.973242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032641+0000","last_change":"2024-04-25T10:03:13.676633+0000","last_active":"2024-04-25T10:03:16.032641+0000","last_peered":"2024-04-25T10:03:16.032641+0000","last_clean":"2024-04-25T10:03:16.032641+0000","last_became_active":"2024-04-25T10:03:13.676353+0000","last_became_peered":"2024-04-25T10:03:13.676353+0000","last_unstale":"2024-04-25T10:03:16.032641+0000","last_undegraded":"2024-04-25T10:03:16.032641+0000","last_fullsized":"2024-04-25T10:03:16.032641+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:25.428507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661506+0000","last_change":"2024-04-25T10:03:14.675835+0000","last_active":"2024-04-25T10:03:15.661506+0000","last_peered":"2024-04-25T10:03:15.661506+0000","last_clean":"2024-04-25T10:03:15.661506+0000","last_became_active":"2024-04-25T10:03:14.675561+0000","last_became_peered":"2024-04-25T10:03:14.675561+0000","last_unstale":"2024-04-25T10:03:15.661506+0000","last_undegraded":"2024-04-25T10:03:15.661506+0000","last_fullsized":"2024-04-25T10:03:15.661506+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:31.454154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695547+0000","last_change":"2024-04-25T10:03:13.673943+0000","last_active":"2024-04-25T10:03:15.695547+0000","last_peered":"2024-04-25T10:03:15.695547+0000","last_clean":"2024-04-25T10:03:15.695547+0000","last_became_active":"2024-04-25T10:03:13.673646+0000","last_became_peered":"2024-04-25T10:03:13.673646+0000","last_unstale":"2024-04-25T10:03:15.695547+0000","last_undegraded":"2024-04-25T10:03:15.695547+0000","last_fullsized":"2024-04-25T10:03:15.695547+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:02.140986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663038+0000","last_change":"2024-04-25T10:03:14.676320+0000","last_active":"2024-04-25T10:03:15.663038+0000","last_peered":"2024-04-25T10:03:15.663038+0000","last_clean":"2024-04-25T10:03:15.663038+0000","last_became_active":"2024-04-25T10:03:14.675981+0000","last_became_peered":"2024-04-25T10:03:14.675981+0000","last_unstale":"2024-04-25T10:03:15.663038+0000","last_undegraded":"2024-04-25T10:03:15.663038+0000","last_fullsized":"2024-04-25T10:03:15.663038+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:27.232666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662332+0000","last_change":"2024-04-25T10:03:13.671428+0000","last_active":"2024-04-25T10:03:15.662332+0000","last_peered":"2024-04-25T10:03:15.662332+0000","last_clean":"2024-04-25T10:03:15.662332+0000","last_became_active":"2024-04-25T10:03:13.671080+0000","last_became_peered":"2024-04-25T10:03:13.671080+0000","last_unstale":"2024-04-25T10:03:15.662332+0000","last_undegraded":"2024-04-25T10:03:15.662332+0000","last_fullsized":"2024-04-25T10:03:15.662332+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:53.770956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032066+0000","last_change":"2024-04-25T10:03:14.674783+0000","last_active":"2024-04-25T10:03:16.032066+0000","last_peered":"2024-04-25T10:03:16.032066+0000","last_clean":"2024-04-25T10:03:16.032066+0000","last_became_active":"2024-04-25T10:03:14.674513+0000","last_became_peered":"2024-04-25T10:03:14.674513+0000","last_unstale":"2024-04-25T10:03:16.032066+0000","last_undegraded":"2024-04-25T10:03:16.032066+0000","last_fullsized":"2024-04-25T10:03:16.032066+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:52.242049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661552+0000","last_change":"2024-04-25T10:03:13.669724+0000","last_active":"2024-04-25T10:03:15.661552+0000","last_peered":"2024-04-25T10:03:15.661552+0000","last_clean":"2024-04-25T10:03:15.661552+0000","last_became_active":"2024-04-25T10:03:13.669446+0000","last_became_peered":"2024-04-25T10:03:13.669446+0000","last_unstale":"2024-04-25T10:03:15.661552+0000","last_undegraded":"2024-04-25T10:03:15.661552+0000","last_fullsized":"2024-04-25T10:03:15.661552+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:09.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032122+0000","last_change":"2024-04-25T10:03:14.696579+0000","last_active":"2024-04-25T10:03:16.032122+0000","last_peered":"2024-04-25T10:03:16.032122+0000","last_clean":"2024-04-25T10:03:16.032122+0000","last_became_active":"2024-04-25T10:03:14.696291+0000","last_became_peered":"2024-04-25T10:03:14.696291+0000","last_unstale":"2024-04-25T10:03:16.032122+0000","last_undegraded":"2024-04-25T10:03:16.032122+0000","last_fullsized":"2024-04-25T10:03:16.032122+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:53.141987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"154'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032973+0000","last_change":"2024-04-25T10:03:13.681265+0000","last_active":"2024-04-25T10:03:16.032973+0000","last_peered":"2024-04-25T10:03:16.032973+0000","last_clean":"2024-04-25T10:03:16.032973+0000","last_became_active":"2024-04-25T10:03:13.680824+0000","last_became_peered":"2024-04-25T10:03:13.680824+0000","last_unstale":"2024-04-25T10:03:16.032973+0000","last_undegraded":"2024-04-25T10:03:16.032973+0000","last_fullsized":"2024-04-25T10:03:16.032973+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:40.866203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695539+0000","last_change":"2024-04-25T10:03:14.692915+0000","last_active":"2024-04-25T10:03:15.695539+0000","last_peered":"2024-04-25T10:03:15.695539+0000","last_clean":"2024-04-25T10:03:15.695539+0000","last_became_active":"2024-04-25T10:03:14.692006+0000","last_became_peered":"2024-04-25T10:03:14.692006+0000","last_unstale":"2024-04-25T10:03:15.695539+0000","last_undegraded":"2024-04-25T10:03:15.695539+0000","last_fullsized":"2024-04-25T10:03:15.695539+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:42.744369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032353+0000","last_change":"2024-04-25T10:03:13.670694+0000","last_active":"2024-04-25T10:03:16.032353+0000","last_peered":"2024-04-25T10:03:16.032353+0000","last_clean":"2024-04-25T10:03:16.032353+0000","last_became_active":"2024-04-25T10:03:13.670408+0000","last_became_peered":"2024-04-25T10:03:13.670408+0000","last_unstale":"2024-04-25T10:03:16.032353+0000","last_undegraded":"2024-04-25T10:03:16.032353+0000","last_fullsized":"2024-04-25T10:03:16.032353+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:52.139744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032337+0000","last_change":"2024-04-25T10:03:14.693604+0000","last_active":"2024-04-25T10:03:16.032337+0000","last_peered":"2024-04-25T10:03:16.032337+0000","last_clean":"2024-04-25T10:03:16.032337+0000","last_became_active":"2024-04-25T10:03:14.693315+0000","last_became_peered":"2024-04-25T10:03:14.693315+0000","last_unstale":"2024-04-25T10:03:16.032337+0000","last_undegraded":"2024-04-25T10:03:16.032337+0000","last_fullsized":"2024-04-25T10:03:16.032337+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:41.868446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695686+0000","last_change":"2024-04-25T10:03:13.679719+0000","last_active":"2024-04-25T10:03:15.695686+0000","last_peered":"2024-04-25T10:03:15.695686+0000","last_clean":"2024-04-25T10:03:15.695686+0000","last_became_active":"2024-04-25T10:03:13.679453+0000","last_became_peered":"2024-04-25T10:03:13.679453+0000","last_unstale":"2024-04-25T10:03:15.695686+0000","last_undegraded":"2024-04-25T10:03:15.695686+0000","last_fullsized":"2024-04-25T10:03:15.695686+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:07.634628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663006+0000","last_change":"2024-04-25T10:03:14.676318+0000","last_active":"2024-04-25T10:03:15.663006+0000","last_peered":"2024-04-25T10:03:15.663006+0000","last_clean":"2024-04-25T10:03:15.663006+0000","last_became_active":"2024-04-25T10:03:14.676037+0000","last_became_peered":"2024-04-25T10:03:14.676037+0000","last_unstale":"2024-04-25T10:03:15.663006+0000","last_undegraded":"2024-04-25T10:03:15.663006+0000","last_fullsized":"2024-04-25T10:03:15.663006+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:27.054273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663488+0000","last_change":"2024-04-25T10:03:13.670937+0000","last_active":"2024-04-25T10:03:15.663488+0000","last_peered":"2024-04-25T10:03:15.663488+0000","last_clean":"2024-04-25T10:03:15.663488+0000","last_became_active":"2024-04-25T10:03:13.670561+0000","last_became_peered":"2024-04-25T10:03:13.670561+0000","last_unstale":"2024-04-25T10:03:15.663488+0000","last_undegraded":"2024-04-25T10:03:15.663488+0000","last_fullsized":"2024-04-25T10:03:15.663488+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:04.722407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662724+0000","last_change":"2024-04-25T10:03:14.668203+0000","last_active":"2024-04-25T10:03:15.662724+0000","last_peered":"2024-04-25T10:03:15.662724+0000","last_clean":"2024-04-25T10:03:15.662724+0000","last_became_active":"2024-04-25T10:03:14.667905+0000","last_became_peered":"2024-04-25T10:03:14.667905+0000","last_unstale":"2024-04-25T10:03:15.662724+0000","last_undegraded":"2024-04-25T10:03:15.662724+0000","last_fullsized":"2024-04-25T10:03:15.662724+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:14.453093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032426+0000","last_change":"2024-04-25T10:03:13.671727+0000","last_active":"2024-04-25T10:03:16.032426+0000","last_peered":"2024-04-25T10:03:16.032426+0000","last_clean":"2024-04-25T10:03:16.032426+0000","last_became_active":"2024-04-25T10:03:13.671439+0000","last_became_peered":"2024-04-25T10:03:13.671439+0000","last_unstale":"2024-04-25T10:03:16.032426+0000","last_undegraded":"2024-04-25T10:03:16.032426+0000","last_fullsized":"2024-04-25T10:03:16.032426+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.341450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032829+0000","last_change":"2024-04-25T10:03:14.689063+0000","last_active":"2024-04-25T10:03:16.032829+0000","last_peered":"2024-04-25T10:03:16.032829+0000","last_clean":"2024-04-25T10:03:16.032829+0000","last_became_active":"2024-04-25T10:03:14.688747+0000","last_became_peered":"2024-04-25T10:03:14.688747+0000","last_unstale":"2024-04-25T10:03:16.032829+0000","last_undegraded":"2024-04-25T10:03:16.032829+0000","last_fullsized":"2024-04-25T10:03:16.032829+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:39.228952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032435+0000","last_change":"2024-04-25T10:03:13.674966+0000","last_active":"2024-04-25T10:03:16.032435+0000","last_peered":"2024-04-25T10:03:16.032435+0000","last_clean":"2024-04-25T10:03:16.032435+0000","last_became_active":"2024-04-25T10:03:13.674633+0000","last_became_peered":"2024-04-25T10:03:13.674633+0000","last_unstale":"2024-04-25T10:03:16.032435+0000","last_undegraded":"2024-04-25T10:03:16.032435+0000","last_fullsized":"2024-04-25T10:03:16.032435+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:58.961175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695657+0000","last_change":"2024-04-25T10:03:14.690356+0000","last_active":"2024-04-25T10:03:15.695657+0000","last_peered":"2024-04-25T10:03:15.695657+0000","last_clean":"2024-04-25T10:03:15.695657+0000","last_became_active":"2024-04-25T10:03:14.689751+0000","last_became_peered":"2024-04-25T10:03:14.689751+0000","last_unstale":"2024-04-25T10:03:15.695657+0000","last_undegraded":"2024-04-25T10:03:15.695657+0000","last_fullsized":"2024-04-25T10:03:15.695657+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:29.903213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661164+0000","last_change":"2024-04-25T10:03:13.670435+0000","last_active":"2024-04-25T10:03:15.661164+0000","last_peered":"2024-04-25T10:03:15.661164+0000","last_clean":"2024-04-25T10:03:15.661164+0000","last_became_active":"2024-04-25T10:03:13.670146+0000","last_became_peered":"2024-04-25T10:03:13.670146+0000","last_unstale":"2024-04-25T10:03:15.661164+0000","last_undegraded":"2024-04-25T10:03:15.661164+0000","last_fullsized":"2024-04-25T10:03:15.661164+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:57.556755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662567+0000","last_change":"2024-04-25T10:03:14.676525+0000","last_active":"2024-04-25T10:03:15.662567+0000","last_peered":"2024-04-25T10:03:15.662567+0000","last_clean":"2024-04-25T10:03:15.662567+0000","last_became_active":"2024-04-25T10:03:14.675930+0000","last_became_peered":"2024-04-25T10:03:14.675930+0000","last_unstale":"2024-04-25T10:03:15.662567+0000","last_undegraded":"2024-04-25T10:03:15.662567+0000","last_fullsized":"2024-04-25T10:03:15.662567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:12.246737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032895+0000","last_change":"2024-04-25T10:03:13.681369+0000","last_active":"2024-04-25T10:03:16.032895+0000","last_peered":"2024-04-25T10:03:16.032895+0000","last_clean":"2024-04-25T10:03:16.032895+0000","last_became_active":"2024-04-25T10:03:13.681075+0000","last_became_peered":"2024-04-25T10:03:13.681075+0000","last_unstale":"2024-04-25T10:03:16.032895+0000","last_undegraded":"2024-04-25T10:03:16.032895+0000","last_fullsized":"2024-04-25T10:03:16.032895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:45.232656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":443,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662442+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:03:15.662442+0000","last_peered":"2024-04-25T10:03:15.662442+0000","last_clean":"2024-04-25T10:03:15.662442+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:03:15.662442+0000","last_undegraded":"2024-04-25T10:03:15.662442+0000","last_fullsized":"2024-04-25T10:03:15.662442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51704,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320136,"statfs":{"total":96636764160,"available":96583819264,"internally_reserved":0,"allocated":2531328,"data_stored":1498834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51120,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320720,"statfs":{"total":96636764160,"available":96584417280,"internally_reserved":0,"allocated":1937408,"data_stored":908466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51128,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320712,"statfs":{"total":96636764160,"available":96584409088,"internally_reserved":0,"allocated":1945600,"data_stored":908590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59456,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312384,"statfs":{"total":96636764160,"available":96575881216,"internally_reserved":0,"allocated":1953792,"data_stored":909230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59464,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312376,"statfs":{"total":96636764160,"available":96575873024,"internally_reserved":0,"allocated":1957888,"data_stored":910682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574977,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55284,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316556,"statfs":{"total":96636764160,"available":96580153344,"internally_reserved":0,"allocated":1937408,"data_stored":908466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55296,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316544,"statfs":{"total":96636764160,"available":96580141056,"internally_reserved":0,"allocated":1949696,"data_stored":910078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55872,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315968,"statfs":{"total":96636764160,"available":96579551232,"internally_reserved":0,"allocated":2539520,"data_stored":1498938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:20.317 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T10:03:20.312+0000 7f79d1af1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:23.112 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:23.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.223+0000 7f1391276640 1 Processor -- start 2024-04-25T10:03:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.223+0000 7f1391276640 1 -- start start 2024-04-25T10:03:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.223+0000 7f1391276640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f138c0d1e70 0x7f138c0c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f1391276640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f138c0c7140 0x7f138c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f1391276640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138c0c7970 con 0x7f138c15ef00 2024-04-25T10:03:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f1391276640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138c15f2e0 con 0x7f138c0c7140 2024-04-25T10:03:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f1391276640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0c7c50 con 0x7f138c0d1e70 2024-04-25T10:03:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f138b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f138c0c7140 0x7f138c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f138b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f138c0c7140 0x7f138c0cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49680/0 (socket says 172.21.15.19:49680) 2024-04-25T10:03:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f138b7fe640 1 -- 172.21.15.19:0/1941350293 learned_addr learned my addr 172.21.15.19:0/1941350293 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f138a7fc640 1 --2- 172.21.15.19:0/1941350293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f138c0d1e70 0x7f138c0c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f138b7fe640 1 -- 172.21.15.19:0/1941350293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f138c0d1e70 msgr2=0x7f138c0c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f138b7fe640 1 --2- 172.21.15.19:0/1941350293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f138c0d1e70 0x7f138c0c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f138b7fe640 1 -- 172.21.15.19:0/1941350293 >> v1:172.21.15.112:6789/0 conn(0x7f138c15ef00 legacy=0x7f138c0d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f138b7fe640 1 -- 172.21.15.19:0/1941350293 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c0c7f30 con 0x7f138c0c7140 2024-04-25T10:03:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.224+0000 7f138a7fc640 1 --2- 172.21.15.19:0/1941350293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f138c0d1e70 0x7f138c0c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.225+0000 7f138b7fe640 1 --2- 172.21.15.19:0/1941350293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138c0c7140 0x7f138c0cc270 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f137c009e50 tx=0x7f137c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0de9dba0b138ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.225+0000 7f1389ffb640 1 -- 172.21.15.19:0/1941350293 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c036070 con 0x7f138c0c7140 2024-04-25T10:03:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.225+0000 7f1389ffb640 1 -- 172.21.15.19:0/1941350293 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c031600 con 0x7f138c0c7140 2024-04-25T10:03:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.225+0000 7f1389ffb640 1 -- 172.21.15.19:0/1941350293 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c0319d0 con 0x7f138c0c7140 2024-04-25T10:03:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.225+0000 7f1391276640 1 -- 172.21.15.19:0/1941350293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138c0c7140 msgr2=0x7f138c0cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.225+0000 7f1391276640 1 --2- 172.21.15.19:0/1941350293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138c0c7140 0x7f138c0cc270 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f137c009e50 tx=0x7f137c02a750 comp rx=0 tx=0).stop 2024-04-25T10:03:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.226+0000 7f1391276640 1 -- 172.21.15.19:0/1941350293 shutdown_connections 2024-04-25T10:03:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.226+0000 7f1391276640 1 --2- 172.21.15.19:0/1941350293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138c0c7140 0x7f138c0cc270 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.226+0000 7f1391276640 1 --2- 172.21.15.19:0/1941350293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f138c0d1e70 0x7f138c0c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.226+0000 7f1391276640 1 -- 172.21.15.19:0/1941350293 >> 172.21.15.19:0/1941350293 conn(0x7f138c06f5d0 msgr2=0x7f138c06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.226+0000 7f1391276640 1 -- 172.21.15.19:0/1941350293 shutdown_connections 2024-04-25T10:03:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.226+0000 7f1391276640 1 -- 172.21.15.19:0/1941350293 wait complete. 2024-04-25T10:03:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.227+0000 7f1391276640 1 Processor -- start 2024-04-25T10:03:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.227+0000 7f1391276640 1 -- start start 2024-04-25T10:03:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f1391276640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f138c0c7140 0x7f138c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f1391276640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138c0d1e70 0x7f138c14afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f1391276640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138c15ef00 0x7f138c0b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f1391276640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0c7560 con 0x7f138c0d1e70 2024-04-25T10:03:23.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f1391276640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0c84f0 con 0x7f138c15ef00 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f1391276640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0c76e0 con 0x7f138c0c7140 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f138a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138c0d1e70 0x7f138c14afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f138a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138c0d1e70 0x7f138c14afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49682/0 (socket says 172.21.15.19:49682) 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f138a7fc640 1 -- 172.21.15.19:0/2693614197 learned_addr learned my addr 172.21.15.19:0/2693614197 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f138affd640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f138c0c7140 0x7f138c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f138b7fe640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138c15ef00 0x7f138c0b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f138a7fc640 1 -- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f138c0c7140 msgr2=0x7f138c154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f138a7fc640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f138c0c7140 0x7f138c154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f138a7fc640 1 -- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138c15ef00 msgr2=0x7f138c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f138a7fc640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138c15ef00 0x7f138c0b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.228+0000 7f138a7fc640 1 -- 172.21.15.19:0/2693614197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c0b3750 con 0x7f138c0d1e70 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.229+0000 7f138a7fc640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138c0d1e70 0x7f138c14afb0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f137400c7f0 tx=0x7f137400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.229+0000 7f13837fe640 1 -- 172.21.15.19:0/2693614197 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374015070 con 0x7f138c0d1e70 2024-04-25T10:03:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.229+0000 7f13837fe640 1 -- 172.21.15.19:0/2693614197 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1374007a30 con 0x7f138c0d1e70 2024-04-25T10:03:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.229+0000 7f1391276640 1 -- 172.21.15.19:0/2693614197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138c0b39c0 con 0x7f138c0d1e70 2024-04-25T10:03:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.229+0000 7f13837fe640 1 -- 172.21.15.19:0/2693614197 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374007e00 con 0x7f138c0d1e70 2024-04-25T10:03:23.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.229+0000 7f1391276640 1 -- 172.21.15.19:0/2693614197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138c0cb5a0 con 0x7f138c0d1e70 2024-04-25T10:03:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.230+0000 7f1391276640 1 -- 172.21.15.19:0/2693614197 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138c0cabc0 con 0x7f138c0d1e70 2024-04-25T10:03:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.230+0000 7f13837fe640 1 -- 172.21.15.19:0/2693614197 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1374018070 con 0x7f138c0d1e70 2024-04-25T10:03:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.231+0000 7f13837fe640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136c06b500 0x7f136c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.231+0000 7f13837fe640 1 -- 172.21.15.19:0/2693614197 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f137408a230 con 0x7f138c0d1e70 2024-04-25T10:03:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.231+0000 7f138affd640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136c06b500 0x7f136c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.232+0000 7f138affd640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136c06b500 0x7f136c06d9c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1384006fd0 tx=0x7f1384009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.236+0000 7f13837fe640 1 -- 172.21.15.19:0/2693614197 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f138c0cabc0 con 0x7f138c0d1e70 2024-04-25T10:03:23.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.410+0000 7f1391276640 1 -- 172.21.15.19:0/2693614197 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f138c073990 con 0x7f136c06b500 2024-04-25T10:03:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.417+0000 7f13837fe640 1 -- 172.21.15.19:0/2693614197 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252480 (secure 0 0 0) 0x7f138c073990 con 0x7f136c06b500 2024-04-25T10:03:23.424 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 -- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136c06b500 msgr2=0x7f136c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136c06b500 0x7f136c06d9c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1384006fd0 tx=0x7f1384009060 comp rx=0 tx=0).stop 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 -- 172.21.15.19:0/2693614197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138c0d1e70 msgr2=0x7f138c14afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138c0d1e70 0x7f138c14afb0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f137400c7f0 tx=0x7f137400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 -- 172.21.15.19:0/2693614197 shutdown_connections 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f136c06b500 0x7f136c06d9c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f138c15ef00 0x7f138c0b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138c0d1e70 0x7f138c14afb0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 --2- 172.21.15.19:0/2693614197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f138c0c7140 0x7f138c154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 -- 172.21.15.19:0/2693614197 >> 172.21.15.19:0/2693614197 conn(0x7f138c06f5d0 msgr2=0x7f138c0ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 -- 172.21.15.19:0/2693614197 shutdown_connections 2024-04-25T10:03:23.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.420+0000 7f1391276640 1 -- 172.21.15.19:0/2693614197 wait complete. 2024-04-25T10:03:23.436 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-04-25T10:03:23.054708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439324,"kb_used_data":16360,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754535396,"statfs":{"total":773094113280,"available":772644245504,"internally_reserved":0,"allocated":16752640,"data_stored":8453284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.395138"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663538+0000","last_change":"2024-04-25T10:03:14.674988+0000","last_active":"2024-04-25T10:03:15.663538+0000","last_peered":"2024-04-25T10:03:15.663538+0000","last_clean":"2024-04-25T10:03:15.663538+0000","last_became_active":"2024-04-25T10:03:14.674705+0000","last_became_peered":"2024-04-25T10:03:14.674705+0000","last_unstale":"2024-04-25T10:03:15.663538+0000","last_undegraded":"2024-04-25T10:03:15.663538+0000","last_fullsized":"2024-04-25T10:03:15.663538+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:55.709812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663091+0000","last_change":"2024-04-25T10:03:14.677891+0000","last_active":"2024-04-25T10:03:15.663091+0000","last_peered":"2024-04-25T10:03:15.663091+0000","last_clean":"2024-04-25T10:03:15.663091+0000","last_became_active":"2024-04-25T10:03:14.676509+0000","last_became_peered":"2024-04-25T10:03:14.676509+0000","last_unstale":"2024-04-25T10:03:15.663091+0000","last_undegraded":"2024-04-25T10:03:15.663091+0000","last_fullsized":"2024-04-25T10:03:15.663091+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:25.956823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663156+0000","last_change":"2024-04-25T10:03:14.693467+0000","last_active":"2024-04-25T10:03:15.663156+0000","last_peered":"2024-04-25T10:03:15.663156+0000","last_clean":"2024-04-25T10:03:15.663156+0000","last_became_active":"2024-04-25T10:03:14.693229+0000","last_became_peered":"2024-04-25T10:03:14.693229+0000","last_unstale":"2024-04-25T10:03:15.663156+0000","last_undegraded":"2024-04-25T10:03:15.663156+0000","last_fullsized":"2024-04-25T10:03:15.663156+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:04.852884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662858+0000","last_change":"2024-04-25T10:03:14.668329+0000","last_active":"2024-04-25T10:03:15.662858+0000","last_peered":"2024-04-25T10:03:15.662858+0000","last_clean":"2024-04-25T10:03:15.662858+0000","last_became_active":"2024-04-25T10:03:14.668042+0000","last_became_peered":"2024-04-25T10:03:14.668042+0000","last_unstale":"2024-04-25T10:03:15.662858+0000","last_undegraded":"2024-04-25T10:03:15.662858+0000","last_fullsized":"2024-04-25T10:03:15.662858+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:23.955400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661748+0000","last_change":"2024-04-25T10:03:14.681919+0000","last_active":"2024-04-25T10:03:15.661748+0000","last_peered":"2024-04-25T10:03:15.661748+0000","last_clean":"2024-04-25T10:03:15.661748+0000","last_became_active":"2024-04-25T10:03:14.681625+0000","last_became_peered":"2024-04-25T10:03:14.681625+0000","last_unstale":"2024-04-25T10:03:15.661748+0000","last_undegraded":"2024-04-25T10:03:15.661748+0000","last_fullsized":"2024-04-25T10:03:15.661748+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.895682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662459+0000","last_change":"2024-04-25T10:03:14.664822+0000","last_active":"2024-04-25T10:03:15.662459+0000","last_peered":"2024-04-25T10:03:15.662459+0000","last_clean":"2024-04-25T10:03:15.662459+0000","last_became_active":"2024-04-25T10:03:14.664548+0000","last_became_peered":"2024-04-25T10:03:14.664548+0000","last_unstale":"2024-04-25T10:03:15.662459+0000","last_undegraded":"2024-04-25T10:03:15.662459+0000","last_fullsized":"2024-04-25T10:03:15.662459+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:35.733949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032783+0000","last_change":"2024-04-25T10:03:14.696283+0000","last_active":"2024-04-25T10:03:16.032783+0000","last_peered":"2024-04-25T10:03:16.032783+0000","last_clean":"2024-04-25T10:03:16.032783+0000","last_became_active":"2024-04-25T10:03:14.696049+0000","last_became_peered":"2024-04-25T10:03:14.696049+0000","last_unstale":"2024-04-25T10:03:16.032783+0000","last_undegraded":"2024-04-25T10:03:16.032783+0000","last_fullsized":"2024-04-25T10:03:16.032783+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:36.801468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.031940+0000","last_change":"2024-04-25T10:03:14.677271+0000","last_active":"2024-04-25T10:03:16.031940+0000","last_peered":"2024-04-25T10:03:16.031940+0000","last_clean":"2024-04-25T10:03:16.031940+0000","last_became_active":"2024-04-25T10:03:14.676987+0000","last_became_peered":"2024-04-25T10:03:14.676987+0000","last_unstale":"2024-04-25T10:03:16.031940+0000","last_undegraded":"2024-04-25T10:03:16.031940+0000","last_fullsized":"2024-04-25T10:03:16.031940+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:28.465371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032612+0000","last_change":"2024-04-25T10:03:14.681985+0000","last_active":"2024-04-25T10:03:16.032612+0000","last_peered":"2024-04-25T10:03:16.032612+0000","last_clean":"2024-04-25T10:03:16.032612+0000","last_became_active":"2024-04-25T10:03:14.681702+0000","last_became_peered":"2024-04-25T10:03:14.681702+0000","last_unstale":"2024-04-25T10:03:16.032612+0000","last_undegraded":"2024-04-25T10:03:16.032612+0000","last_fullsized":"2024-04-25T10:03:16.032612+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:44.343693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032758+0000","last_change":"2024-04-25T10:03:14.683567+0000","last_active":"2024-04-25T10:03:16.032758+0000","last_peered":"2024-04-25T10:03:16.032758+0000","last_clean":"2024-04-25T10:03:16.032758+0000","last_became_active":"2024-04-25T10:03:14.681589+0000","last_became_peered":"2024-04-25T10:03:14.681589+0000","last_unstale":"2024-04-25T10:03:16.032758+0000","last_undegraded":"2024-04-25T10:03:16.032758+0000","last_fullsized":"2024-04-25T10:03:16.032758+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:23.927100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032162+0000","last_change":"2024-04-25T10:03:14.681356+0000","last_active":"2024-04-25T10:03:16.032162+0000","last_peered":"2024-04-25T10:03:16.032162+0000","last_clean":"2024-04-25T10:03:16.032162+0000","last_became_active":"2024-04-25T10:03:14.680990+0000","last_became_peered":"2024-04-25T10:03:14.680990+0000","last_unstale":"2024-04-25T10:03:16.032162+0000","last_undegraded":"2024-04-25T10:03:16.032162+0000","last_fullsized":"2024-04-25T10:03:16.032162+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:52.425930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663219+0000","last_change":"2024-04-25T10:03:14.665666+0000","last_active":"2024-04-25T10:03:15.663219+0000","last_peered":"2024-04-25T10:03:15.663219+0000","last_clean":"2024-04-25T10:03:15.663219+0000","last_became_active":"2024-04-25T10:03:14.665328+0000","last_became_peered":"2024-04-25T10:03:14.665328+0000","last_unstale":"2024-04-25T10:03:15.663219+0000","last_undegraded":"2024-04-25T10:03:15.663219+0000","last_fullsized":"2024-04-25T10:03:15.663219+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:10.838625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032537+0000","last_change":"2024-04-25T10:03:14.674760+0000","last_active":"2024-04-25T10:03:16.032537+0000","last_peered":"2024-04-25T10:03:16.032537+0000","last_clean":"2024-04-25T10:03:16.032537+0000","last_became_active":"2024-04-25T10:03:14.674456+0000","last_became_peered":"2024-04-25T10:03:14.674456+0000","last_unstale":"2024-04-25T10:03:16.032537+0000","last_undegraded":"2024-04-25T10:03:16.032537+0000","last_fullsized":"2024-04-25T10:03:16.032537+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:44.291626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661326+0000","last_change":"2024-04-25T10:03:14.694517+0000","last_active":"2024-04-25T10:03:15.661326+0000","last_peered":"2024-04-25T10:03:15.661326+0000","last_clean":"2024-04-25T10:03:15.661326+0000","last_became_active":"2024-04-25T10:03:14.694069+0000","last_became_peered":"2024-04-25T10:03:14.694069+0000","last_unstale":"2024-04-25T10:03:15.661326+0000","last_undegraded":"2024-04-25T10:03:15.661326+0000","last_fullsized":"2024-04-25T10:03:15.661326+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:13.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032343+0000","last_change":"2024-04-25T10:03:14.683731+0000","last_active":"2024-04-25T10:03:16.032343+0000","last_peered":"2024-04-25T10:03:16.032343+0000","last_clean":"2024-04-25T10:03:16.032343+0000","last_became_active":"2024-04-25T10:03:14.683402+0000","last_became_peered":"2024-04-25T10:03:14.683402+0000","last_unstale":"2024-04-25T10:03:16.032343+0000","last_undegraded":"2024-04-25T10:03:16.032343+0000","last_fullsized":"2024-04-25T10:03:16.032343+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:39.505850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662812+0000","last_change":"2024-04-25T10:03:14.669397+0000","last_active":"2024-04-25T10:03:15.662812+0000","last_peered":"2024-04-25T10:03:15.662812+0000","last_clean":"2024-04-25T10:03:15.662812+0000","last_became_active":"2024-04-25T10:03:14.668368+0000","last_became_peered":"2024-04-25T10:03:14.668368+0000","last_unstale":"2024-04-25T10:03:15.662812+0000","last_undegraded":"2024-04-25T10:03:15.662812+0000","last_fullsized":"2024-04-25T10:03:15.662812+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:28.463537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032771+0000","last_change":"2024-04-25T10:03:14.681114+0000","last_active":"2024-04-25T10:03:16.032771+0000","last_peered":"2024-04-25T10:03:16.032771+0000","last_clean":"2024-04-25T10:03:16.032771+0000","last_became_active":"2024-04-25T10:03:14.680775+0000","last_became_peered":"2024-04-25T10:03:14.680775+0000","last_unstale":"2024-04-25T10:03:16.032771+0000","last_undegraded":"2024-04-25T10:03:16.032771+0000","last_fullsized":"2024-04-25T10:03:16.032771+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:35.733949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032399+0000","last_change":"2024-04-25T10:03:14.674046+0000","last_active":"2024-04-25T10:03:16.032399+0000","last_peered":"2024-04-25T10:03:16.032399+0000","last_clean":"2024-04-25T10:03:16.032399+0000","last_became_active":"2024-04-25T10:03:14.673789+0000","last_became_peered":"2024-04-25T10:03:14.673789+0000","last_unstale":"2024-04-25T10:03:16.032399+0000","last_undegraded":"2024-04-25T10:03:16.032399+0000","last_fullsized":"2024-04-25T10:03:16.032399+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:04.852884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032669+0000","last_change":"2024-04-25T10:03:14.675106+0000","last_active":"2024-04-25T10:03:16.032669+0000","last_peered":"2024-04-25T10:03:16.032669+0000","last_clean":"2024-04-25T10:03:16.032669+0000","last_became_active":"2024-04-25T10:03:14.674810+0000","last_became_peered":"2024-04-25T10:03:14.674810+0000","last_unstale":"2024-04-25T10:03:16.032669+0000","last_undegraded":"2024-04-25T10:03:16.032669+0000","last_fullsized":"2024-04-25T10:03:16.032669+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:31.422540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695708+0000","last_change":"2024-04-25T10:03:14.691234+0000","last_active":"2024-04-25T10:03:15.695708+0000","last_peered":"2024-04-25T10:03:15.695708+0000","last_clean":"2024-04-25T10:03:15.695708+0000","last_became_active":"2024-04-25T10:03:14.690950+0000","last_became_peered":"2024-04-25T10:03:14.690950+0000","last_unstale":"2024-04-25T10:03:15.695708+0000","last_undegraded":"2024-04-25T10:03:15.695708+0000","last_fullsized":"2024-04-25T10:03:15.695708+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:20.623444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032667+0000","last_change":"2024-04-25T10:03:14.689443+0000","last_active":"2024-04-25T10:03:16.032667+0000","last_peered":"2024-04-25T10:03:16.032667+0000","last_clean":"2024-04-25T10:03:16.032667+0000","last_became_active":"2024-04-25T10:03:14.689151+0000","last_became_peered":"2024-04-25T10:03:14.689151+0000","last_unstale":"2024-04-25T10:03:16.032667+0000","last_undegraded":"2024-04-25T10:03:16.032667+0000","last_fullsized":"2024-04-25T10:03:16.032667+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:16.734137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662133+0000","last_change":"2024-04-25T10:03:14.677533+0000","last_active":"2024-04-25T10:03:15.662133+0000","last_peered":"2024-04-25T10:03:15.662133+0000","last_clean":"2024-04-25T10:03:15.662133+0000","last_became_active":"2024-04-25T10:03:14.677261+0000","last_became_peered":"2024-04-25T10:03:14.677261+0000","last_unstale":"2024-04-25T10:03:15.662133+0000","last_undegraded":"2024-04-25T10:03:15.662133+0000","last_fullsized":"2024-04-25T10:03:15.662133+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:31.946288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661340+0000","last_change":"2024-04-25T10:03:14.680014+0000","last_active":"2024-04-25T10:03:15.661340+0000","last_peered":"2024-04-25T10:03:15.661340+0000","last_clean":"2024-04-25T10:03:15.661340+0000","last_became_active":"2024-04-25T10:03:14.679525+0000","last_became_peered":"2024-04-25T10:03:14.679525+0000","last_unstale":"2024-04-25T10:03:15.661340+0000","last_undegraded":"2024-04-25T10:03:15.661340+0000","last_fullsized":"2024-04-25T10:03:15.661340+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:09.709075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663186+0000","last_change":"2024-04-25T10:03:14.668828+0000","last_active":"2024-04-25T10:03:15.663186+0000","last_peered":"2024-04-25T10:03:15.663186+0000","last_clean":"2024-04-25T10:03:15.663186+0000","last_became_active":"2024-04-25T10:03:14.668000+0000","last_became_peered":"2024-04-25T10:03:14.668000+0000","last_unstale":"2024-04-25T10:03:15.663186+0000","last_undegraded":"2024-04-25T10:03:15.663186+0000","last_fullsized":"2024-04-25T10:03:15.663186+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:59.362567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032734+0000","last_change":"2024-04-25T10:03:14.673302+0000","last_active":"2024-04-25T10:03:16.032734+0000","last_peered":"2024-04-25T10:03:16.032734+0000","last_clean":"2024-04-25T10:03:16.032734+0000","last_became_active":"2024-04-25T10:03:14.673003+0000","last_became_peered":"2024-04-25T10:03:14.673003+0000","last_unstale":"2024-04-25T10:03:16.032734+0000","last_undegraded":"2024-04-25T10:03:16.032734+0000","last_fullsized":"2024-04-25T10:03:16.032734+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:05.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695652+0000","last_change":"2024-04-25T10:03:14.693330+0000","last_active":"2024-04-25T10:03:15.695652+0000","last_peered":"2024-04-25T10:03:15.695652+0000","last_clean":"2024-04-25T10:03:15.695652+0000","last_became_active":"2024-04-25T10:03:14.692664+0000","last_became_peered":"2024-04-25T10:03:14.692664+0000","last_unstale":"2024-04-25T10:03:15.695652+0000","last_undegraded":"2024-04-25T10:03:15.695652+0000","last_fullsized":"2024-04-25T10:03:15.695652+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:33.297902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662549+0000","last_change":"2024-04-25T10:03:14.680270+0000","last_active":"2024-04-25T10:03:15.662549+0000","last_peered":"2024-04-25T10:03:15.662549+0000","last_clean":"2024-04-25T10:03:15.662549+0000","last_became_active":"2024-04-25T10:03:14.680005+0000","last_became_peered":"2024-04-25T10:03:14.680005+0000","last_unstale":"2024-04-25T10:03:15.662549+0000","last_undegraded":"2024-04-25T10:03:15.662549+0000","last_fullsized":"2024-04-25T10:03:15.662549+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:36.801468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032504+0000","last_change":"2024-04-25T10:03:14.666917+0000","last_active":"2024-04-25T10:03:16.032504+0000","last_peered":"2024-04-25T10:03:16.032504+0000","last_clean":"2024-04-25T10:03:16.032504+0000","last_became_active":"2024-04-25T10:03:14.666580+0000","last_became_peered":"2024-04-25T10:03:14.666580+0000","last_unstale":"2024-04-25T10:03:16.032504+0000","last_undegraded":"2024-04-25T10:03:16.032504+0000","last_fullsized":"2024-04-25T10:03:16.032504+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:43.733533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662990+0000","last_change":"2024-04-25T10:03:14.675411+0000","last_active":"2024-04-25T10:03:15.662990+0000","last_peered":"2024-04-25T10:03:15.662990+0000","last_clean":"2024-04-25T10:03:15.662990+0000","last_became_active":"2024-04-25T10:03:14.675071+0000","last_became_peered":"2024-04-25T10:03:14.675071+0000","last_unstale":"2024-04-25T10:03:15.662990+0000","last_undegraded":"2024-04-25T10:03:15.662990+0000","last_fullsized":"2024-04-25T10:03:15.662990+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:34.175141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032478+0000","last_change":"2024-04-25T10:03:14.680054+0000","last_active":"2024-04-25T10:03:16.032478+0000","last_peered":"2024-04-25T10:03:16.032478+0000","last_clean":"2024-04-25T10:03:16.032478+0000","last_became_active":"2024-04-25T10:03:14.679780+0000","last_became_peered":"2024-04-25T10:03:14.679780+0000","last_unstale":"2024-04-25T10:03:16.032478+0000","last_undegraded":"2024-04-25T10:03:16.032478+0000","last_fullsized":"2024-04-25T10:03:16.032478+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:55.709812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032215+0000","last_change":"2024-04-25T10:03:14.678473+0000","last_active":"2024-04-25T10:03:16.032215+0000","last_peered":"2024-04-25T10:03:16.032215+0000","last_clean":"2024-04-25T10:03:16.032215+0000","last_became_active":"2024-04-25T10:03:14.677596+0000","last_became_peered":"2024-04-25T10:03:14.677596+0000","last_unstale":"2024-04-25T10:03:16.032215+0000","last_undegraded":"2024-04-25T10:03:16.032215+0000","last_fullsized":"2024-04-25T10:03:16.032215+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:04.438091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032098+0000","last_change":"2024-04-25T10:03:14.672679+0000","last_active":"2024-04-25T10:03:16.032098+0000","last_peered":"2024-04-25T10:03:16.032098+0000","last_clean":"2024-04-25T10:03:16.032098+0000","last_became_active":"2024-04-25T10:03:14.672415+0000","last_became_peered":"2024-04-25T10:03:14.672415+0000","last_unstale":"2024-04-25T10:03:16.032098+0000","last_undegraded":"2024-04-25T10:03:16.032098+0000","last_fullsized":"2024-04-25T10:03:16.032098+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:59.362567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662222+0000","last_change":"2024-04-25T10:03:14.668281+0000","last_active":"2024-04-25T10:03:15.662222+0000","last_peered":"2024-04-25T10:03:15.662222+0000","last_clean":"2024-04-25T10:03:15.662222+0000","last_became_active":"2024-04-25T10:03:14.668002+0000","last_became_peered":"2024-04-25T10:03:14.668002+0000","last_unstale":"2024-04-25T10:03:15.662222+0000","last_undegraded":"2024-04-25T10:03:15.662222+0000","last_fullsized":"2024-04-25T10:03:15.662222+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:44.343693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"153'4","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663624+0000","last_change":"2024-04-25T10:03:13.674884+0000","last_active":"2024-04-25T10:03:15.663624+0000","last_peered":"2024-04-25T10:03:15.663624+0000","last_clean":"2024-04-25T10:03:15.663624+0000","last_became_active":"2024-04-25T10:03:13.674524+0000","last_became_peered":"2024-04-25T10:03:13.674524+0000","last_unstale":"2024-04-25T10:03:15.663624+0000","last_undegraded":"2024-04-25T10:03:15.663624+0000","last_fullsized":"2024-04-25T10:03:15.663624+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:30.635663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032047+0000","last_change":"2024-04-25T10:03:14.676233+0000","last_active":"2024-04-25T10:03:16.032047+0000","last_peered":"2024-04-25T10:03:16.032047+0000","last_clean":"2024-04-25T10:03:16.032047+0000","last_became_active":"2024-04-25T10:03:14.675959+0000","last_became_peered":"2024-04-25T10:03:14.675959+0000","last_unstale":"2024-04-25T10:03:16.032047+0000","last_undegraded":"2024-04-25T10:03:16.032047+0000","last_fullsized":"2024-04-25T10:03:16.032047+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:10.838625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661524+0000","last_change":"2024-04-25T10:03:13.679019+0000","last_active":"2024-04-25T10:03:15.661524+0000","last_peered":"2024-04-25T10:03:15.661524+0000","last_clean":"2024-04-25T10:03:15.661524+0000","last_became_active":"2024-04-25T10:03:13.678461+0000","last_became_peered":"2024-04-25T10:03:13.678461+0000","last_unstale":"2024-04-25T10:03:15.661524+0000","last_undegraded":"2024-04-25T10:03:15.661524+0000","last_fullsized":"2024-04-25T10:03:15.661524+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:08.056138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661367+0000","last_change":"2024-04-25T10:03:14.694677+0000","last_active":"2024-04-25T10:03:15.661367+0000","last_peered":"2024-04-25T10:03:15.661367+0000","last_clean":"2024-04-25T10:03:15.661367+0000","last_became_active":"2024-04-25T10:03:14.694420+0000","last_became_peered":"2024-04-25T10:03:14.694420+0000","last_unstale":"2024-04-25T10:03:15.661367+0000","last_undegraded":"2024-04-25T10:03:15.661367+0000","last_fullsized":"2024-04-25T10:03:15.661367+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:49.159049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032736+0000","last_change":"2024-04-25T10:03:13.674905+0000","last_active":"2024-04-25T10:03:16.032736+0000","last_peered":"2024-04-25T10:03:16.032736+0000","last_clean":"2024-04-25T10:03:16.032736+0000","last_became_active":"2024-04-25T10:03:13.674590+0000","last_became_peered":"2024-04-25T10:03:13.674590+0000","last_unstale":"2024-04-25T10:03:16.032736+0000","last_undegraded":"2024-04-25T10:03:16.032736+0000","last_fullsized":"2024-04-25T10:03:16.032736+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:10.022627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663029+0000","last_change":"2024-04-25T10:03:14.667446+0000","last_active":"2024-04-25T10:03:15.663029+0000","last_peered":"2024-04-25T10:03:15.663029+0000","last_clean":"2024-04-25T10:03:15.663029+0000","last_became_active":"2024-04-25T10:03:14.667170+0000","last_became_peered":"2024-04-25T10:03:14.667170+0000","last_unstale":"2024-04-25T10:03:15.663029+0000","last_undegraded":"2024-04-25T10:03:15.663029+0000","last_fullsized":"2024-04-25T10:03:15.663029+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:58.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695558+0000","last_change":"2024-04-25T10:03:13.673989+0000","last_active":"2024-04-25T10:03:15.695558+0000","last_peered":"2024-04-25T10:03:15.695558+0000","last_clean":"2024-04-25T10:03:15.695558+0000","last_became_active":"2024-04-25T10:03:13.673670+0000","last_became_peered":"2024-04-25T10:03:13.673670+0000","last_unstale":"2024-04-25T10:03:15.695558+0000","last_undegraded":"2024-04-25T10:03:15.695558+0000","last_fullsized":"2024-04-25T10:03:15.695558+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:40.623717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.033073+0000","last_change":"2024-04-25T10:03:14.684249+0000","last_active":"2024-04-25T10:03:16.033073+0000","last_peered":"2024-04-25T10:03:16.033073+0000","last_clean":"2024-04-25T10:03:16.033073+0000","last_became_active":"2024-04-25T10:03:14.683952+0000","last_became_peered":"2024-04-25T10:03:14.683952+0000","last_unstale":"2024-04-25T10:03:16.033073+0000","last_undegraded":"2024-04-25T10:03:16.033073+0000","last_fullsized":"2024-04-25T10:03:16.033073+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:12.246737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032435+0000","last_change":"2024-04-25T10:03:13.677056+0000","last_active":"2024-04-25T10:03:16.032435+0000","last_peered":"2024-04-25T10:03:16.032435+0000","last_clean":"2024-04-25T10:03:16.032435+0000","last_became_active":"2024-04-25T10:03:13.676751+0000","last_became_peered":"2024-04-25T10:03:13.676751+0000","last_unstale":"2024-04-25T10:03:16.032435+0000","last_undegraded":"2024-04-25T10:03:16.032435+0000","last_fullsized":"2024-04-25T10:03:16.032435+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:30.425664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695669+0000","last_change":"2024-04-25T10:03:14.692243+0000","last_active":"2024-04-25T10:03:15.695669+0000","last_peered":"2024-04-25T10:03:15.695669+0000","last_clean":"2024-04-25T10:03:15.695669+0000","last_became_active":"2024-04-25T10:03:14.691969+0000","last_became_peered":"2024-04-25T10:03:14.691969+0000","last_unstale":"2024-04-25T10:03:15.695669+0000","last_undegraded":"2024-04-25T10:03:15.695669+0000","last_fullsized":"2024-04-25T10:03:15.695669+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:40.965938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661463+0000","last_change":"2024-04-25T10:03:13.674339+0000","last_active":"2024-04-25T10:03:15.661463+0000","last_peered":"2024-04-25T10:03:15.661463+0000","last_clean":"2024-04-25T10:03:15.661463+0000","last_became_active":"2024-04-25T10:03:13.674074+0000","last_became_peered":"2024-04-25T10:03:13.674074+0000","last_unstale":"2024-04-25T10:03:15.661463+0000","last_undegraded":"2024-04-25T10:03:15.661463+0000","last_fullsized":"2024-04-25T10:03:15.661463+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:42.731289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695673+0000","last_change":"2024-04-25T10:03:14.693007+0000","last_active":"2024-04-25T10:03:15.695673+0000","last_peered":"2024-04-25T10:03:15.695673+0000","last_clean":"2024-04-25T10:03:15.695673+0000","last_became_active":"2024-04-25T10:03:14.692321+0000","last_became_peered":"2024-04-25T10:03:14.692321+0000","last_unstale":"2024-04-25T10:03:15.695673+0000","last_undegraded":"2024-04-25T10:03:15.695673+0000","last_fullsized":"2024-04-25T10:03:15.695673+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:29.049939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695399+0000","last_change":"2024-04-25T10:03:13.681030+0000","last_active":"2024-04-25T10:03:15.695399+0000","last_peered":"2024-04-25T10:03:15.695399+0000","last_clean":"2024-04-25T10:03:15.695399+0000","last_became_active":"2024-04-25T10:03:13.680669+0000","last_became_peered":"2024-04-25T10:03:13.680669+0000","last_unstale":"2024-04-25T10:03:15.695399+0000","last_undegraded":"2024-04-25T10:03:15.695399+0000","last_fullsized":"2024-04-25T10:03:15.695399+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:11.777899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663547+0000","last_change":"2024-04-25T10:03:14.671062+0000","last_active":"2024-04-25T10:03:15.663547+0000","last_peered":"2024-04-25T10:03:15.663547+0000","last_clean":"2024-04-25T10:03:15.663547+0000","last_became_active":"2024-04-25T10:03:14.670731+0000","last_became_peered":"2024-04-25T10:03:14.670731+0000","last_unstale":"2024-04-25T10:03:15.663547+0000","last_undegraded":"2024-04-25T10:03:15.663547+0000","last_fullsized":"2024-04-25T10:03:15.663547+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:17.438859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032058+0000","last_change":"2024-04-25T10:03:13.670929+0000","last_active":"2024-04-25T10:03:16.032058+0000","last_peered":"2024-04-25T10:03:16.032058+0000","last_clean":"2024-04-25T10:03:16.032058+0000","last_became_active":"2024-04-25T10:03:13.670647+0000","last_became_peered":"2024-04-25T10:03:13.670647+0000","last_unstale":"2024-04-25T10:03:16.032058+0000","last_undegraded":"2024-04-25T10:03:16.032058+0000","last_fullsized":"2024-04-25T10:03:16.032058+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:54.474670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662254+0000","last_change":"2024-04-25T10:03:14.667661+0000","last_active":"2024-04-25T10:03:15.662254+0000","last_peered":"2024-04-25T10:03:15.662254+0000","last_clean":"2024-04-25T10:03:15.662254+0000","last_became_active":"2024-04-25T10:03:14.667384+0000","last_became_peered":"2024-04-25T10:03:14.667384+0000","last_unstale":"2024-04-25T10:03:15.662254+0000","last_undegraded":"2024-04-25T10:03:15.662254+0000","last_fullsized":"2024-04-25T10:03:15.662254+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:23.927100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032503+0000","last_change":"2024-04-25T10:03:13.680361+0000","last_active":"2024-04-25T10:03:16.032503+0000","last_peered":"2024-04-25T10:03:16.032503+0000","last_clean":"2024-04-25T10:03:16.032503+0000","last_became_active":"2024-04-25T10:03:13.680075+0000","last_became_peered":"2024-04-25T10:03:13.680075+0000","last_unstale":"2024-04-25T10:03:16.032503+0000","last_undegraded":"2024-04-25T10:03:16.032503+0000","last_fullsized":"2024-04-25T10:03:16.032503+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:41.835542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032121+0000","last_change":"2024-04-25T10:03:14.679029+0000","last_active":"2024-04-25T10:03:16.032121+0000","last_peered":"2024-04-25T10:03:16.032121+0000","last_clean":"2024-04-25T10:03:16.032121+0000","last_became_active":"2024-04-25T10:03:14.678623+0000","last_became_peered":"2024-04-25T10:03:14.678623+0000","last_unstale":"2024-04-25T10:03:16.032121+0000","last_undegraded":"2024-04-25T10:03:16.032121+0000","last_fullsized":"2024-04-25T10:03:16.032121+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:09.058381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"153'3","reported_seq":17,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663312+0000","last_change":"2024-04-25T10:03:13.669655+0000","last_active":"2024-04-25T10:03:15.663312+0000","last_peered":"2024-04-25T10:03:15.663312+0000","last_clean":"2024-04-25T10:03:15.663312+0000","last_became_active":"2024-04-25T10:03:13.669366+0000","last_became_peered":"2024-04-25T10:03:13.669366+0000","last_unstale":"2024-04-25T10:03:15.663312+0000","last_undegraded":"2024-04-25T10:03:15.663312+0000","last_fullsized":"2024-04-25T10:03:15.663312+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:18.239201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695492+0000","last_change":"2024-04-25T10:03:14.688513+0000","last_active":"2024-04-25T10:03:15.695492+0000","last_peered":"2024-04-25T10:03:15.695492+0000","last_clean":"2024-04-25T10:03:15.695492+0000","last_became_active":"2024-04-25T10:03:14.687663+0000","last_became_peered":"2024-04-25T10:03:14.687663+0000","last_unstale":"2024-04-25T10:03:15.695492+0000","last_undegraded":"2024-04-25T10:03:15.695492+0000","last_fullsized":"2024-04-25T10:03:15.695492+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.827804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"153'6","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032704+0000","last_change":"2024-04-25T10:03:13.681202+0000","last_active":"2024-04-25T10:03:16.032704+0000","last_peered":"2024-04-25T10:03:16.032704+0000","last_clean":"2024-04-25T10:03:16.032704+0000","last_became_active":"2024-04-25T10:03:13.680736+0000","last_became_peered":"2024-04-25T10:03:13.680736+0000","last_unstale":"2024-04-25T10:03:16.032704+0000","last_undegraded":"2024-04-25T10:03:16.032704+0000","last_fullsized":"2024-04-25T10:03:16.032704+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:22.924857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032567+0000","last_change":"2024-04-25T10:03:14.672705+0000","last_active":"2024-04-25T10:03:16.032567+0000","last_peered":"2024-04-25T10:03:16.032567+0000","last_clean":"2024-04-25T10:03:16.032567+0000","last_became_active":"2024-04-25T10:03:14.672098+0000","last_became_peered":"2024-04-25T10:03:14.672098+0000","last_unstale":"2024-04-25T10:03:16.032567+0000","last_undegraded":"2024-04-25T10:03:16.032567+0000","last_fullsized":"2024-04-25T10:03:16.032567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:06.108933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661873+0000","last_change":"2024-04-25T10:03:13.671104+0000","last_active":"2024-04-25T10:03:15.661873+0000","last_peered":"2024-04-25T10:03:15.661873+0000","last_clean":"2024-04-25T10:03:15.661873+0000","last_became_active":"2024-04-25T10:03:13.670810+0000","last_became_peered":"2024-04-25T10:03:13.670810+0000","last_unstale":"2024-04-25T10:03:15.661873+0000","last_undegraded":"2024-04-25T10:03:15.661873+0000","last_fullsized":"2024-04-25T10:03:15.661873+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:26.230423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032411+0000","last_change":"2024-04-25T10:03:14.683180+0000","last_active":"2024-04-25T10:03:16.032411+0000","last_peered":"2024-04-25T10:03:16.032411+0000","last_clean":"2024-04-25T10:03:16.032411+0000","last_became_active":"2024-04-25T10:03:14.681022+0000","last_became_peered":"2024-04-25T10:03:14.681022+0000","last_unstale":"2024-04-25T10:03:16.032411+0000","last_undegraded":"2024-04-25T10:03:16.032411+0000","last_fullsized":"2024-04-25T10:03:16.032411+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:31.946288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032321+0000","last_change":"2024-04-25T10:03:13.672839+0000","last_active":"2024-04-25T10:03:16.032321+0000","last_peered":"2024-04-25T10:03:16.032321+0000","last_clean":"2024-04-25T10:03:16.032321+0000","last_became_active":"2024-04-25T10:03:13.671105+0000","last_became_peered":"2024-04-25T10:03:13.671105+0000","last_unstale":"2024-04-25T10:03:16.032321+0000","last_undegraded":"2024-04-25T10:03:16.032321+0000","last_fullsized":"2024-04-25T10:03:16.032321+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:30.944045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032890+0000","last_change":"2024-04-25T10:03:14.678441+0000","last_active":"2024-04-25T10:03:16.032890+0000","last_peered":"2024-04-25T10:03:16.032890+0000","last_clean":"2024-04-25T10:03:16.032890+0000","last_became_active":"2024-04-25T10:03:14.678057+0000","last_became_peered":"2024-04-25T10:03:14.678057+0000","last_unstale":"2024-04-25T10:03:16.032890+0000","last_undegraded":"2024-04-25T10:03:16.032890+0000","last_fullsized":"2024-04-25T10:03:16.032890+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:42.837785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032322+0000","last_change":"2024-04-25T10:03:13.680518+0000","last_active":"2024-04-25T10:03:16.032322+0000","last_peered":"2024-04-25T10:03:16.032322+0000","last_clean":"2024-04-25T10:03:16.032322+0000","last_became_active":"2024-04-25T10:03:13.680260+0000","last_became_peered":"2024-04-25T10:03:13.680260+0000","last_unstale":"2024-04-25T10:03:16.032322+0000","last_undegraded":"2024-04-25T10:03:16.032322+0000","last_fullsized":"2024-04-25T10:03:16.032322+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:04.430596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695620+0000","last_change":"2024-04-25T10:03:14.690620+0000","last_active":"2024-04-25T10:03:15.695620+0000","last_peered":"2024-04-25T10:03:15.695620+0000","last_clean":"2024-04-25T10:03:15.695620+0000","last_became_active":"2024-04-25T10:03:14.690062+0000","last_became_peered":"2024-04-25T10:03:14.690062+0000","last_unstale":"2024-04-25T10:03:15.695620+0000","last_undegraded":"2024-04-25T10:03:15.695620+0000","last_fullsized":"2024-04-25T10:03:15.695620+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:32.342537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"153'8","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032174+0000","last_change":"2024-04-25T10:03:13.675759+0000","last_active":"2024-04-25T10:03:16.032174+0000","last_peered":"2024-04-25T10:03:16.032174+0000","last_clean":"2024-04-25T10:03:16.032174+0000","last_became_active":"2024-04-25T10:03:13.675488+0000","last_became_peered":"2024-04-25T10:03:13.675488+0000","last_unstale":"2024-04-25T10:03:16.032174+0000","last_undegraded":"2024-04-25T10:03:16.032174+0000","last_fullsized":"2024-04-25T10:03:16.032174+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:46.101956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.031966+0000","last_change":"2024-04-25T10:03:14.693995+0000","last_active":"2024-04-25T10:03:16.031966+0000","last_peered":"2024-04-25T10:03:16.031966+0000","last_clean":"2024-04-25T10:03:16.031966+0000","last_became_active":"2024-04-25T10:03:14.693721+0000","last_became_peered":"2024-04-25T10:03:14.693721+0000","last_unstale":"2024-04-25T10:03:16.031966+0000","last_undegraded":"2024-04-25T10:03:16.031966+0000","last_fullsized":"2024-04-25T10:03:16.031966+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:46.234899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032693+0000","last_change":"2024-04-25T10:03:13.673662+0000","last_active":"2024-04-25T10:03:16.032693+0000","last_peered":"2024-04-25T10:03:16.032693+0000","last_clean":"2024-04-25T10:03:16.032693+0000","last_became_active":"2024-04-25T10:03:13.673355+0000","last_became_peered":"2024-04-25T10:03:13.673355+0000","last_unstale":"2024-04-25T10:03:16.032693+0000","last_undegraded":"2024-04-25T10:03:16.032693+0000","last_fullsized":"2024-04-25T10:03:16.032693+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:27.463128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032537+0000","last_change":"2024-04-25T10:03:14.677433+0000","last_active":"2024-04-25T10:03:16.032537+0000","last_peered":"2024-04-25T10:03:16.032537+0000","last_clean":"2024-04-25T10:03:16.032537+0000","last_became_active":"2024-04-25T10:03:14.676902+0000","last_became_peered":"2024-04-25T10:03:14.676902+0000","last_unstale":"2024-04-25T10:03:16.032537+0000","last_undegraded":"2024-04-25T10:03:16.032537+0000","last_fullsized":"2024-04-25T10:03:16.032537+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:17.438859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.033020+0000","last_change":"2024-04-25T10:03:13.678163+0000","last_active":"2024-04-25T10:03:16.033020+0000","last_peered":"2024-04-25T10:03:16.033020+0000","last_clean":"2024-04-25T10:03:16.033020+0000","last_became_active":"2024-04-25T10:03:13.677884+0000","last_became_peered":"2024-04-25T10:03:13.677884+0000","last_unstale":"2024-04-25T10:03:16.033020+0000","last_undegraded":"2024-04-25T10:03:16.033020+0000","last_fullsized":"2024-04-25T10:03:16.033020+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:01.697483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032567+0000","last_change":"2024-04-25T10:03:14.673455+0000","last_active":"2024-04-25T10:03:16.032567+0000","last_peered":"2024-04-25T10:03:16.032567+0000","last_clean":"2024-04-25T10:03:16.032567+0000","last_became_active":"2024-04-25T10:03:14.673174+0000","last_became_peered":"2024-04-25T10:03:14.673174+0000","last_unstale":"2024-04-25T10:03:16.032567+0000","last_undegraded":"2024-04-25T10:03:16.032567+0000","last_fullsized":"2024-04-25T10:03:16.032567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:32.794361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661808+0000","last_change":"2024-04-25T10:03:13.667893+0000","last_active":"2024-04-25T10:03:15.661808+0000","last_peered":"2024-04-25T10:03:15.661808+0000","last_clean":"2024-04-25T10:03:15.661808+0000","last_became_active":"2024-04-25T10:03:13.667483+0000","last_became_peered":"2024-04-25T10:03:13.667483+0000","last_unstale":"2024-04-25T10:03:15.661808+0000","last_undegraded":"2024-04-25T10:03:15.661808+0000","last_fullsized":"2024-04-25T10:03:15.661808+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:43.289383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032270+0000","last_change":"2024-04-25T10:03:14.680270+0000","last_active":"2024-04-25T10:03:16.032270+0000","last_peered":"2024-04-25T10:03:16.032270+0000","last_clean":"2024-04-25T10:03:16.032270+0000","last_became_active":"2024-04-25T10:03:14.679992+0000","last_became_peered":"2024-04-25T10:03:14.679992+0000","last_unstale":"2024-04-25T10:03:16.032270+0000","last_undegraded":"2024-04-25T10:03:16.032270+0000","last_fullsized":"2024-04-25T10:03:16.032270+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:32.008962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.033043+0000","last_change":"2024-04-25T10:03:13.678499+0000","last_active":"2024-04-25T10:03:16.033043+0000","last_peered":"2024-04-25T10:03:16.033043+0000","last_clean":"2024-04-25T10:03:16.033043+0000","last_became_active":"2024-04-25T10:03:13.678219+0000","last_became_peered":"2024-04-25T10:03:13.678219+0000","last_unstale":"2024-04-25T10:03:16.033043+0000","last_undegraded":"2024-04-25T10:03:16.033043+0000","last_fullsized":"2024-04-25T10:03:16.033043+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:45.649559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032593+0000","last_change":"2024-04-25T10:03:14.690253+0000","last_active":"2024-04-25T10:03:16.032593+0000","last_peered":"2024-04-25T10:03:16.032593+0000","last_clean":"2024-04-25T10:03:16.032593+0000","last_became_active":"2024-04-25T10:03:14.689918+0000","last_became_peered":"2024-04-25T10:03:14.689918+0000","last_unstale":"2024-04-25T10:03:16.032593+0000","last_undegraded":"2024-04-25T10:03:16.032593+0000","last_fullsized":"2024-04-25T10:03:16.032593+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:58.106950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661895+0000","last_change":"2024-04-25T10:03:13.670893+0000","last_active":"2024-04-25T10:03:15.661895+0000","last_peered":"2024-04-25T10:03:15.661895+0000","last_clean":"2024-04-25T10:03:15.661895+0000","last_became_active":"2024-04-25T10:03:13.670599+0000","last_became_peered":"2024-04-25T10:03:13.670599+0000","last_unstale":"2024-04-25T10:03:15.661895+0000","last_undegraded":"2024-04-25T10:03:15.661895+0000","last_fullsized":"2024-04-25T10:03:15.661895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:24.954579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661809+0000","last_change":"2024-04-25T10:03:14.687032+0000","last_active":"2024-04-25T10:03:15.661809+0000","last_peered":"2024-04-25T10:03:15.661809+0000","last_clean":"2024-04-25T10:03:15.661809+0000","last_became_active":"2024-04-25T10:03:14.686714+0000","last_became_peered":"2024-04-25T10:03:14.686714+0000","last_unstale":"2024-04-25T10:03:15.661809+0000","last_undegraded":"2024-04-25T10:03:15.661809+0000","last_fullsized":"2024-04-25T10:03:15.661809+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:03.973242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032641+0000","last_change":"2024-04-25T10:03:13.676633+0000","last_active":"2024-04-25T10:03:16.032641+0000","last_peered":"2024-04-25T10:03:16.032641+0000","last_clean":"2024-04-25T10:03:16.032641+0000","last_became_active":"2024-04-25T10:03:13.676353+0000","last_became_peered":"2024-04-25T10:03:13.676353+0000","last_unstale":"2024-04-25T10:03:16.032641+0000","last_undegraded":"2024-04-25T10:03:16.032641+0000","last_fullsized":"2024-04-25T10:03:16.032641+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:25.428507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661506+0000","last_change":"2024-04-25T10:03:14.675835+0000","last_active":"2024-04-25T10:03:15.661506+0000","last_peered":"2024-04-25T10:03:15.661506+0000","last_clean":"2024-04-25T10:03:15.661506+0000","last_became_active":"2024-04-25T10:03:14.675561+0000","last_became_peered":"2024-04-25T10:03:14.675561+0000","last_unstale":"2024-04-25T10:03:15.661506+0000","last_undegraded":"2024-04-25T10:03:15.661506+0000","last_fullsized":"2024-04-25T10:03:15.661506+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:31.454154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695547+0000","last_change":"2024-04-25T10:03:13.673943+0000","last_active":"2024-04-25T10:03:15.695547+0000","last_peered":"2024-04-25T10:03:15.695547+0000","last_clean":"2024-04-25T10:03:15.695547+0000","last_became_active":"2024-04-25T10:03:13.673646+0000","last_became_peered":"2024-04-25T10:03:13.673646+0000","last_unstale":"2024-04-25T10:03:15.695547+0000","last_undegraded":"2024-04-25T10:03:15.695547+0000","last_fullsized":"2024-04-25T10:03:15.695547+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:02.140986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663038+0000","last_change":"2024-04-25T10:03:14.676320+0000","last_active":"2024-04-25T10:03:15.663038+0000","last_peered":"2024-04-25T10:03:15.663038+0000","last_clean":"2024-04-25T10:03:15.663038+0000","last_became_active":"2024-04-25T10:03:14.675981+0000","last_became_peered":"2024-04-25T10:03:14.675981+0000","last_unstale":"2024-04-25T10:03:15.663038+0000","last_undegraded":"2024-04-25T10:03:15.663038+0000","last_fullsized":"2024-04-25T10:03:15.663038+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:27.232666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662332+0000","last_change":"2024-04-25T10:03:13.671428+0000","last_active":"2024-04-25T10:03:15.662332+0000","last_peered":"2024-04-25T10:03:15.662332+0000","last_clean":"2024-04-25T10:03:15.662332+0000","last_became_active":"2024-04-25T10:03:13.671080+0000","last_became_peered":"2024-04-25T10:03:13.671080+0000","last_unstale":"2024-04-25T10:03:15.662332+0000","last_undegraded":"2024-04-25T10:03:15.662332+0000","last_fullsized":"2024-04-25T10:03:15.662332+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:53.770956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032066+0000","last_change":"2024-04-25T10:03:14.674783+0000","last_active":"2024-04-25T10:03:16.032066+0000","last_peered":"2024-04-25T10:03:16.032066+0000","last_clean":"2024-04-25T10:03:16.032066+0000","last_became_active":"2024-04-25T10:03:14.674513+0000","last_became_peered":"2024-04-25T10:03:14.674513+0000","last_unstale":"2024-04-25T10:03:16.032066+0000","last_undegraded":"2024-04-25T10:03:16.032066+0000","last_fullsized":"2024-04-25T10:03:16.032066+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:52.242049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661552+0000","last_change":"2024-04-25T10:03:13.669724+0000","last_active":"2024-04-25T10:03:15.661552+0000","last_peered":"2024-04-25T10:03:15.661552+0000","last_clean":"2024-04-25T10:03:15.661552+0000","last_became_active":"2024-04-25T10:03:13.669446+0000","last_became_peered":"2024-04-25T10:03:13.669446+0000","last_unstale":"2024-04-25T10:03:15.661552+0000","last_undegraded":"2024-04-25T10:03:15.661552+0000","last_fullsized":"2024-04-25T10:03:15.661552+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:09.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032122+0000","last_change":"2024-04-25T10:03:14.696579+0000","last_active":"2024-04-25T10:03:16.032122+0000","last_peered":"2024-04-25T10:03:16.032122+0000","last_clean":"2024-04-25T10:03:16.032122+0000","last_became_active":"2024-04-25T10:03:14.696291+0000","last_became_peered":"2024-04-25T10:03:14.696291+0000","last_unstale":"2024-04-25T10:03:16.032122+0000","last_undegraded":"2024-04-25T10:03:16.032122+0000","last_fullsized":"2024-04-25T10:03:16.032122+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:53.141987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"154'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032973+0000","last_change":"2024-04-25T10:03:13.681265+0000","last_active":"2024-04-25T10:03:16.032973+0000","last_peered":"2024-04-25T10:03:16.032973+0000","last_clean":"2024-04-25T10:03:16.032973+0000","last_became_active":"2024-04-25T10:03:13.680824+0000","last_became_peered":"2024-04-25T10:03:13.680824+0000","last_unstale":"2024-04-25T10:03:16.032973+0000","last_undegraded":"2024-04-25T10:03:16.032973+0000","last_fullsized":"2024-04-25T10:03:16.032973+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:40.866203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695539+0000","last_change":"2024-04-25T10:03:14.692915+0000","last_active":"2024-04-25T10:03:15.695539+0000","last_peered":"2024-04-25T10:03:15.695539+0000","last_clean":"2024-04-25T10:03:15.695539+0000","last_became_active":"2024-04-25T10:03:14.692006+0000","last_became_peered":"2024-04-25T10:03:14.692006+0000","last_unstale":"2024-04-25T10:03:15.695539+0000","last_undegraded":"2024-04-25T10:03:15.695539+0000","last_fullsized":"2024-04-25T10:03:15.695539+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:42.744369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032353+0000","last_change":"2024-04-25T10:03:13.670694+0000","last_active":"2024-04-25T10:03:16.032353+0000","last_peered":"2024-04-25T10:03:16.032353+0000","last_clean":"2024-04-25T10:03:16.032353+0000","last_became_active":"2024-04-25T10:03:13.670408+0000","last_became_peered":"2024-04-25T10:03:13.670408+0000","last_unstale":"2024-04-25T10:03:16.032353+0000","last_undegraded":"2024-04-25T10:03:16.032353+0000","last_fullsized":"2024-04-25T10:03:16.032353+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:52.139744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032337+0000","last_change":"2024-04-25T10:03:14.693604+0000","last_active":"2024-04-25T10:03:16.032337+0000","last_peered":"2024-04-25T10:03:16.032337+0000","last_clean":"2024-04-25T10:03:16.032337+0000","last_became_active":"2024-04-25T10:03:14.693315+0000","last_became_peered":"2024-04-25T10:03:14.693315+0000","last_unstale":"2024-04-25T10:03:16.032337+0000","last_undegraded":"2024-04-25T10:03:16.032337+0000","last_fullsized":"2024-04-25T10:03:16.032337+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:41.868446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695686+0000","last_change":"2024-04-25T10:03:13.679719+0000","last_active":"2024-04-25T10:03:15.695686+0000","last_peered":"2024-04-25T10:03:15.695686+0000","last_clean":"2024-04-25T10:03:15.695686+0000","last_became_active":"2024-04-25T10:03:13.679453+0000","last_became_peered":"2024-04-25T10:03:13.679453+0000","last_unstale":"2024-04-25T10:03:15.695686+0000","last_undegraded":"2024-04-25T10:03:15.695686+0000","last_fullsized":"2024-04-25T10:03:15.695686+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:07.634628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663006+0000","last_change":"2024-04-25T10:03:14.676318+0000","last_active":"2024-04-25T10:03:15.663006+0000","last_peered":"2024-04-25T10:03:15.663006+0000","last_clean":"2024-04-25T10:03:15.663006+0000","last_became_active":"2024-04-25T10:03:14.676037+0000","last_became_peered":"2024-04-25T10:03:14.676037+0000","last_unstale":"2024-04-25T10:03:15.663006+0000","last_undegraded":"2024-04-25T10:03:15.663006+0000","last_fullsized":"2024-04-25T10:03:15.663006+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:27.054273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.663488+0000","last_change":"2024-04-25T10:03:13.670937+0000","last_active":"2024-04-25T10:03:15.663488+0000","last_peered":"2024-04-25T10:03:15.663488+0000","last_clean":"2024-04-25T10:03:15.663488+0000","last_became_active":"2024-04-25T10:03:13.670561+0000","last_became_peered":"2024-04-25T10:03:13.670561+0000","last_unstale":"2024-04-25T10:03:15.663488+0000","last_undegraded":"2024-04-25T10:03:15.663488+0000","last_fullsized":"2024-04-25T10:03:15.663488+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:04.722407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662724+0000","last_change":"2024-04-25T10:03:14.668203+0000","last_active":"2024-04-25T10:03:15.662724+0000","last_peered":"2024-04-25T10:03:15.662724+0000","last_clean":"2024-04-25T10:03:15.662724+0000","last_became_active":"2024-04-25T10:03:14.667905+0000","last_became_peered":"2024-04-25T10:03:14.667905+0000","last_unstale":"2024-04-25T10:03:15.662724+0000","last_undegraded":"2024-04-25T10:03:15.662724+0000","last_fullsized":"2024-04-25T10:03:15.662724+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:14.453093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032426+0000","last_change":"2024-04-25T10:03:13.671727+0000","last_active":"2024-04-25T10:03:16.032426+0000","last_peered":"2024-04-25T10:03:16.032426+0000","last_clean":"2024-04-25T10:03:16.032426+0000","last_became_active":"2024-04-25T10:03:13.671439+0000","last_became_peered":"2024-04-25T10:03:13.671439+0000","last_unstale":"2024-04-25T10:03:16.032426+0000","last_undegraded":"2024-04-25T10:03:16.032426+0000","last_fullsized":"2024-04-25T10:03:16.032426+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:43.341450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032829+0000","last_change":"2024-04-25T10:03:14.689063+0000","last_active":"2024-04-25T10:03:16.032829+0000","last_peered":"2024-04-25T10:03:16.032829+0000","last_clean":"2024-04-25T10:03:16.032829+0000","last_became_active":"2024-04-25T10:03:14.688747+0000","last_became_peered":"2024-04-25T10:03:14.688747+0000","last_unstale":"2024-04-25T10:03:16.032829+0000","last_undegraded":"2024-04-25T10:03:16.032829+0000","last_fullsized":"2024-04-25T10:03:16.032829+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:39.228952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032435+0000","last_change":"2024-04-25T10:03:13.674966+0000","last_active":"2024-04-25T10:03:16.032435+0000","last_peered":"2024-04-25T10:03:16.032435+0000","last_clean":"2024-04-25T10:03:16.032435+0000","last_became_active":"2024-04-25T10:03:13.674633+0000","last_became_peered":"2024-04-25T10:03:13.674633+0000","last_unstale":"2024-04-25T10:03:16.032435+0000","last_undegraded":"2024-04-25T10:03:16.032435+0000","last_fullsized":"2024-04-25T10:03:16.032435+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:58.961175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.695657+0000","last_change":"2024-04-25T10:03:14.690356+0000","last_active":"2024-04-25T10:03:15.695657+0000","last_peered":"2024-04-25T10:03:15.695657+0000","last_clean":"2024-04-25T10:03:15.695657+0000","last_became_active":"2024-04-25T10:03:14.689751+0000","last_became_peered":"2024-04-25T10:03:14.689751+0000","last_unstale":"2024-04-25T10:03:15.695657+0000","last_undegraded":"2024-04-25T10:03:15.695657+0000","last_fullsized":"2024-04-25T10:03:15.695657+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:29.903213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.661164+0000","last_change":"2024-04-25T10:03:13.670435+0000","last_active":"2024-04-25T10:03:15.661164+0000","last_peered":"2024-04-25T10:03:15.661164+0000","last_clean":"2024-04-25T10:03:15.661164+0000","last_became_active":"2024-04-25T10:03:13.670146+0000","last_became_peered":"2024-04-25T10:03:13.670146+0000","last_unstale":"2024-04-25T10:03:15.661164+0000","last_undegraded":"2024-04-25T10:03:15.661164+0000","last_fullsized":"2024-04-25T10:03:15.661164+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:57.556755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662567+0000","last_change":"2024-04-25T10:03:14.676525+0000","last_active":"2024-04-25T10:03:15.662567+0000","last_peered":"2024-04-25T10:03:15.662567+0000","last_clean":"2024-04-25T10:03:15.662567+0000","last_became_active":"2024-04-25T10:03:14.675930+0000","last_became_peered":"2024-04-25T10:03:14.675930+0000","last_unstale":"2024-04-25T10:03:15.662567+0000","last_undegraded":"2024-04-25T10:03:15.662567+0000","last_fullsized":"2024-04-25T10:03:15.662567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:13.651178+0000","last_clean_scrub_stamp":"2024-04-25T10:03:13.651178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:12.246737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:16.032895+0000","last_change":"2024-04-25T10:03:13.681369+0000","last_active":"2024-04-25T10:03:16.032895+0000","last_peered":"2024-04-25T10:03:16.032895+0000","last_clean":"2024-04-25T10:03:16.032895+0000","last_became_active":"2024-04-25T10:03:13.681075+0000","last_became_peered":"2024-04-25T10:03:13.681075+0000","last_unstale":"2024-04-25T10:03:16.032895+0000","last_undegraded":"2024-04-25T10:03:16.032895+0000","last_fullsized":"2024-04-25T10:03:16.032895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:12.648934+0000","last_clean_scrub_stamp":"2024-04-25T10:03:12.648934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:45.232656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":443,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:03:15.662442+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:03:15.662442+0000","last_peered":"2024-04-25T10:03:15.662442+0000","last_clean":"2024-04-25T10:03:15.662442+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:03:15.662442+0000","last_undegraded":"2024-04-25T10:03:15.662442+0000","last_fullsized":"2024-04-25T10:03:15.662442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51704,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320136,"statfs":{"total":96636764160,"available":96583819264,"internally_reserved":0,"allocated":2531328,"data_stored":1498834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51120,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320720,"statfs":{"total":96636764160,"available":96584417280,"internally_reserved":0,"allocated":1937408,"data_stored":908466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51128,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320712,"statfs":{"total":96636764160,"available":96584409088,"internally_reserved":0,"allocated":1945600,"data_stored":908590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59456,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312384,"statfs":{"total":96636764160,"available":96575881216,"internally_reserved":0,"allocated":1953792,"data_stored":909230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59464,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312376,"statfs":{"total":96636764160,"available":96575873024,"internally_reserved":0,"allocated":1957888,"data_stored":910682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574977,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55284,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316556,"statfs":{"total":96636764160,"available":96580153344,"internally_reserved":0,"allocated":1937408,"data_stored":908466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55296,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316544,"statfs":{"total":96636764160,"available":96580141056,"internally_reserved":0,"allocated":1949696,"data_stored":910078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55872,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315968,"statfs":{"total":96636764160,"available":96579551232,"internally_reserved":0,"allocated":2539520,"data_stored":1498938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:23.440 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:03:23.440 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:03:23.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.580+0000 7fdd3b190640 1 Processor -- start 2024-04-25T10:03:23.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.580+0000 7fdd3b190640 1 -- start start 2024-04-25T10:03:23.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd3b190640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd3415b7d0 0x7fdd340cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd3b190640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd3406cf50 con 0x7fdd340b5620 2024-04-25T10:03:23.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd3b190640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd340ce530 con 0x7fdd3406c9e0 2024-04-25T10:03:23.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd3b190640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd340ce810 con 0x7fdd3415b7d0 2024-04-25T10:03:23.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd33fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdd340b5620 0x7fdd3406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46164/0 (socket says 172.21.15.19:46164) 2024-04-25T10:03:23.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd33fff640 1 -- 172.21.15.19:0/3865629584 learned_addr learned my addr 172.21.15.19:0/3865629584 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:23.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd38f2c640 1 --2- 172.21.15.19:0/3865629584 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd3415b7d0 0x7fdd340cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd337fe640 1 -- 172.21.15.19:0/3865629584 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265938520 0 0) 0x7fdd3406cf50 con 0x7fdd340b5620 2024-04-25T10:03:23.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd337fe640 1 -- 172.21.15.19:0/3865629584 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd24003520 con 0x7fdd340b5620 2024-04-25T10:03:23.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd337fe640 1 -- 172.21.15.19:0/3865629584 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774118317 0 0) 0x7fdd340ce530 con 0x7fdd3406c9e0 2024-04-25T10:03:23.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd38f2c640 1 -- 172.21.15.19:0/3865629584 >> v1:172.21.15.112:6789/0 conn(0x7fdd3406c9e0 legacy=0x7fdd340d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd38f2c640 1 -- 172.21.15.19:0/3865629584 >> v1:172.21.15.19:6789/0 conn(0x7fdd340b5620 legacy=0x7fdd3406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.581+0000 7fdd38f2c640 1 -- 172.21.15.19:0/3865629584 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd340ceaf0 con 0x7fdd3415b7d0 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.582+0000 7fdd38f2c640 1 --2- 172.21.15.19:0/3865629584 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd3415b7d0 0x7fdd340cf010 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd280097d0 tx=0x7fdd2802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34ab4421dd5fd127 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.582+0000 7fdd337fe640 1 -- 172.21.15.19:0/3865629584 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28037070 con 0x7fdd3415b7d0 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.582+0000 7fdd337fe640 1 -- 172.21.15.19:0/3865629584 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2802ac10 con 0x7fdd3415b7d0 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.582+0000 7fdd337fe640 1 -- 172.21.15.19:0/3865629584 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20001670 con 0x7fdd3415b7d0 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.583+0000 7fdd3b190640 1 -- 172.21.15.19:0/3865629584 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd3415b7d0 msgr2=0x7fdd340cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.583+0000 7fdd3b190640 1 --2- 172.21.15.19:0/3865629584 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd3415b7d0 0x7fdd340cf010 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd280097d0 tx=0x7fdd2802a180 comp rx=0 tx=0).stop 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.583+0000 7fdd3b190640 1 -- 172.21.15.19:0/3865629584 shutdown_connections 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.583+0000 7fdd3b190640 1 --2- 172.21.15.19:0/3865629584 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd3415b7d0 0x7fdd340cf010 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.583+0000 7fdd3b190640 1 -- 172.21.15.19:0/3865629584 >> 172.21.15.19:0/3865629584 conn(0x7fdd340d0800 msgr2=0x7fdd340d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.583+0000 7fdd3b190640 1 -- 172.21.15.19:0/3865629584 shutdown_connections 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.583+0000 7fdd3b190640 1 -- 172.21.15.19:0/3865629584 wait complete. 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.584+0000 7fdd3b190640 1 Processor -- start 2024-04-25T10:03:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.584+0000 7fdd3b190640 1 -- start start 2024-04-25T10:03:23.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd3b190640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd3406c9e0 0x7fdd341f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd3b190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd340b5620 0x7fdd340cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd3b190640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd3415b7d0 0x7fdd341f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd3b190640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd340cf6d0 con 0x7fdd340b5620 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd3b190640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd340cf550 con 0x7fdd3415b7d0 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd3b190640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3415bbb0 con 0x7fdd3406c9e0 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd33fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd340b5620 0x7fdd340cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd33fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd340b5620 0x7fdd340cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49696/0 (socket says 172.21.15.19:49696) 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd33fff640 1 -- 172.21.15.19:0/4280608613 learned_addr learned my addr 172.21.15.19:0/4280608613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd38f2c640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd3406c9e0 0x7fdd341f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd3972d640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd3415b7d0 0x7fdd341f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd33fff640 1 -- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd3406c9e0 msgr2=0x7fdd341f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd33fff640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd3406c9e0 0x7fdd341f3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd33fff640 1 -- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd3415b7d0 msgr2=0x7fdd341f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd33fff640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd3415b7d0 0x7fdd341f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.585+0000 7fdd33fff640 1 -- 172.21.15.19:0/4280608613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd340cd240 con 0x7fdd340b5620 2024-04-25T10:03:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.586+0000 7fdd33fff640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd340b5620 0x7fdd340cc7d0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c790 tx=0x7fdd2000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.586+0000 7fdd317fa640 1 -- 172.21.15.19:0/4280608613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20018070 con 0x7fdd340b5620 2024-04-25T10:03:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.586+0000 7fdd317fa640 1 -- 172.21.15.19:0/4280608613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd200074e0 con 0x7fdd340b5620 2024-04-25T10:03:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.586+0000 7fdd3b190640 1 -- 172.21.15.19:0/4280608613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd340cd520 con 0x7fdd340b5620 2024-04-25T10:03:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.586+0000 7fdd317fa640 1 -- 172.21.15.19:0/4280608613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20004a00 con 0x7fdd340b5620 2024-04-25T10:03:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.586+0000 7fdd3b190640 1 -- 172.21.15.19:0/4280608613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd340cdae0 con 0x7fdd340b5620 2024-04-25T10:03:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.587+0000 7fdd3b190640 1 -- 172.21.15.19:0/4280608613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd34073a00 con 0x7fdd340b5620 2024-04-25T10:03:23.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.587+0000 7fdd317fa640 1 -- 172.21.15.19:0/4280608613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd2001b070 con 0x7fdd340b5620 2024-04-25T10:03:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.588+0000 7fdd317fa640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd1806b4d0 0x7fdd1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.588+0000 7fdd317fa640 1 -- 172.21.15.19:0/4280608613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd200898c0 con 0x7fdd340b5620 2024-04-25T10:03:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.588+0000 7fdd38f2c640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd1806b4d0 0x7fdd1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.588+0000 7fdd38f2c640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd1806b4d0 0x7fdd1806d990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fdd28034000 tx=0x7fdd2802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.592+0000 7fdd317fa640 1 -- 172.21.15.19:0/4280608613 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd20058640 con 0x7fdd340b5620 2024-04-25T10:03:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.839+0000 7fdd3b190640 1 -- 172.21.15.19:0/4280608613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdd3406b210 con 0x7fdd340b5620 2024-04-25T10:03:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.843+0000 7fdd317fa640 1 -- 172.21.15.19:0/4280608613 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7fdd20057f40 con 0x7fdd340b5620 2024-04-25T10:03:23.848 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T10:03:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T10:03:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T10:03:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T10:03:23.848 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:03:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T10:03:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 -- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd1806b4d0 msgr2=0x7fdd1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd1806b4d0 0x7fdd1806d990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fdd28034000 tx=0x7fdd2802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 -- 172.21.15.19:0/4280608613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd340b5620 msgr2=0x7fdd340cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd340b5620 0x7fdd340cc7d0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c790 tx=0x7fdd2000cc50 comp rx=0 tx=0).stop 2024-04-25T10:03:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 -- 172.21.15.19:0/4280608613 shutdown_connections 2024-04-25T10:03:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdd1806b4d0 0x7fdd1806d990 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd3415b7d0 0x7fdd341f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd340b5620 0x7fdd340cc7d0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 --2- 172.21.15.19:0/4280608613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd3406c9e0 0x7fdd341f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 -- 172.21.15.19:0/4280608613 >> 172.21.15.19:0/4280608613 conn(0x7fdd340d0800 msgr2=0x7fdd34160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 -- 172.21.15.19:0/4280608613 shutdown_connections 2024-04-25T10:03:23.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.845+0000 7fdd3b190640 1 -- 172.21.15.19:0/4280608613 wait complete. 2024-04-25T10:03:23.861 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.957+0000 7f758ba9a640 1 Processor -- start 2024-04-25T10:03:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.957+0000 7f758ba9a640 1 -- start start 2024-04-25T10:03:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.958+0000 7f758ba9a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f758415db70 0x7f7584162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.958+0000 7f758ba9a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 0x7f75840b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.958+0000 7f758ba9a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7584162690 con 0x7f75840b5a60 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.958+0000 7f758ba9a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75840d3800 con 0x7f758415db70 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.958+0000 7f758ba9a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75840ceee0 con 0x7f75840b6cb0 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.958+0000 7f7589035640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 0x7f75840b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.958+0000 7f7589836640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f758415db70 0x7f7584162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.958+0000 7f7589836640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f758415db70 0x7f7584162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:58566/0 (socket says 172.21.15.19:58566) 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.958+0000 7f7589836640 1 -- 172.21.15.19:0/3602166912 learned_addr learned my addr 172.21.15.19:0/3602166912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.958+0000 7f7589035640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 0x7f75840b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:55432/0 (socket says 172.21.15.19:55432) 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.959+0000 7f7589035640 1 -- 172.21.15.19:0/3602166912 >> v2:172.21.15.112:3300/0 conn(0x7f758415db70 msgr2=0x7f7584162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.959+0000 7f7589035640 1 --2- 172.21.15.19:0/3602166912 >> v2:172.21.15.112:3300/0 conn(0x7f758415db70 0x7f7584162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.959+0000 7f7589035640 1 -- 172.21.15.19:0/3602166912 >> v1:172.21.15.112:6789/0 conn(0x7f75840b5a60 legacy=0x7f758406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.959+0000 7f7588834640 1 -- 172.21.15.19:0/3602166912 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451072517 0 0) 0x7f7584162690 con 0x7f75840b5a60 2024-04-25T10:03:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.959+0000 7f7589035640 1 -- 172.21.15.19:0/3602166912 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75840d1d10 con 0x7f75840b6cb0 2024-04-25T10:03:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.959+0000 7f7589836640 1 --2- 172.21.15.19:0/3602166912 >> v2:172.21.15.112:3300/0 conn(0x7f758415db70 0x7f7584162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:23.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.959+0000 7f7589035640 1 --2- 172.21.15.19:0/3602166912 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 0x7f75840b5520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f75740097a0 tx=0x7f757402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fe63e4f2a712444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.960+0000 7f7588834640 1 -- 172.21.15.19:0/3602166912 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574037070 con 0x7f75840b6cb0 2024-04-25T10:03:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.960+0000 7f7588834640 1 -- 172.21.15.19:0/3602166912 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7574032440 con 0x7f75840b6cb0 2024-04-25T10:03:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.960+0000 7f7588834640 1 -- 172.21.15.19:0/3602166912 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574032810 con 0x7f75840b6cb0 2024-04-25T10:03:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.960+0000 7f758ba9a640 1 -- 172.21.15.19:0/3602166912 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 msgr2=0x7f75840b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.960+0000 7f758ba9a640 1 --2- 172.21.15.19:0/3602166912 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 0x7f75840b5520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f75740097a0 tx=0x7f757402a180 comp rx=0 tx=0).stop 2024-04-25T10:03:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.960+0000 7f758ba9a640 1 -- 172.21.15.19:0/3602166912 shutdown_connections 2024-04-25T10:03:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.960+0000 7f758ba9a640 1 --2- 172.21.15.19:0/3602166912 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 0x7f75840b5520 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.960+0000 7f758ba9a640 1 --2- 172.21.15.19:0/3602166912 >> v2:172.21.15.112:3300/0 conn(0x7f758415db70 0x7f7584162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.960+0000 7f758ba9a640 1 -- 172.21.15.19:0/3602166912 >> 172.21.15.19:0/3602166912 conn(0x7f75840d0520 msgr2=0x7f75840d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.960+0000 7f758ba9a640 1 -- 172.21.15.19:0/3602166912 shutdown_connections 2024-04-25T10:03:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.960+0000 7f758ba9a640 1 -- 172.21.15.19:0/3602166912 wait complete. 2024-04-25T10:03:23.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.961+0000 7f758ba9a640 1 Processor -- start 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.961+0000 7f758ba9a640 1 -- start start 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.962+0000 7f758ba9a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75840b5a60 0x7f75840cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.962+0000 7f758ba9a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 0x7f75840c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.962+0000 7f758ba9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f758415db70 0x7f75840aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.962+0000 7f758ba9a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758415df50 con 0x7f75840b5a60 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.962+0000 7f758ba9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75840d22d0 con 0x7f758415db70 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.962+0000 7f758ba9a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75840ce530 con 0x7f75840b6cb0 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.962+0000 7f7589836640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75840b5a60 0x7f75840cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.962+0000 7f7589836640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75840b5a60 0x7f75840cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49698/0 (socket says 172.21.15.19:49698) 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.962+0000 7f7589836640 1 -- 172.21.15.19:0/2356230505 learned_addr learned my addr 172.21.15.19:0/2356230505 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.962+0000 7f758a037640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f758415db70 0x7f75840aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.962+0000 7f7589035640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 0x7f75840c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.963+0000 7f7589836640 1 -- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 msgr2=0x7f75840c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.963+0000 7f7589836640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 0x7f75840c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.963+0000 7f7589836640 1 -- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f758415db70 msgr2=0x7f75840aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.963+0000 7f7589836640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f758415db70 0x7f75840aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.963+0000 7f7589836640 1 -- 172.21.15.19:0/2356230505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584146dc0 con 0x7f75840b5a60 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.963+0000 7f7589836640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75840b5a60 0x7f75840cba50 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f758000e9d0 tx=0x7f758000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.963+0000 7f757a7fc640 1 -- 172.21.15.19:0/2356230505 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580017070 con 0x7f75840b5a60 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.963+0000 7f757a7fc640 1 -- 172.21.15.19:0/2356230505 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758000ca60 con 0x7f75840b5a60 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.963+0000 7f758ba9a640 1 -- 172.21.15.19:0/2356230505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75841470a0 con 0x7f75840b5a60 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.963+0000 7f757a7fc640 1 -- 172.21.15.19:0/2356230505 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758000ce10 con 0x7f75840b5a60 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.963+0000 7f758ba9a640 1 -- 172.21.15.19:0/2356230505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7584147660 con 0x7f75840b5a60 2024-04-25T10:03:23.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.964+0000 7f758ba9a640 1 -- 172.21.15.19:0/2356230505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7584073990 con 0x7f75840b5a60 2024-04-25T10:03:23.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.964+0000 7f757a7fc640 1 -- 172.21.15.19:0/2356230505 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f758001a070 con 0x7f75840b5a60 2024-04-25T10:03:23.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.965+0000 7f757a7fc640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756806b4d0 0x7f756806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.965+0000 7f757a7fc640 1 -- 172.21.15.19:0/2356230505 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f758008a570 con 0x7f75840b5a60 2024-04-25T10:03:23.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.965+0000 7f7589035640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756806b4d0 0x7f756806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.966+0000 7f7589035640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756806b4d0 0x7f756806d990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7574034000 tx=0x7f757402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:23.969+0000 7f757a7fc640 1 -- 172.21.15.19:0/2356230505 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7580058ca0 con 0x7f75840b5a60 2024-04-25T10:03:24.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.188+0000 7f758ba9a640 1 -- 172.21.15.19:0/2356230505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75840d1820 con 0x7f75840b5a60 2024-04-25T10:03:24.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.189+0000 7f757a7fc640 1 -- 172.21.15.19:0/2356230505 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f758005cd40 con 0x7f75840b5a60 2024-04-25T10:03:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:24.194 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":102,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3725393673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3725393673},{"type":"v1","addr":"172.21.15.112:6837","nonce":3725393673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8866,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2539453196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2539453196},{"type":"v1","addr":"172.21.15.112:6836","nonce":2539453196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10221,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3896097924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3896097924},{"type":"v1","addr":"172.21.15.19:6834","nonce":3896097924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:14.030067+0000","modified":"2024-04-25T10:03:17.045113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10224":{"gid":10224,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/717842662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":717842662},{"type":"v1","addr":"172.21.15.19:6835","nonce":717842662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:03:24.194 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 102 2024-04-25T10:03:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 -- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756806b4d0 msgr2=0x7f756806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756806b4d0 0x7f756806d990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7574034000 tx=0x7f757402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 -- 172.21.15.19:0/2356230505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75840b5a60 msgr2=0x7f75840cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75840b5a60 0x7f75840cba50 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f758000e9d0 tx=0x7f758000ee90 comp rx=0 tx=0).stop 2024-04-25T10:03:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 -- 172.21.15.19:0/2356230505 shutdown_connections 2024-04-25T10:03:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f756806b4d0 0x7f756806d990 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f758415db70 0x7f75840aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75840b6cb0 0x7f75840c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 --2- 172.21.15.19:0/2356230505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75840b5a60 0x7f75840cba50 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 -- 172.21.15.19:0/2356230505 >> 172.21.15.19:0/2356230505 conn(0x7f75840d0520 msgr2=0x7f7584162150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 -- 172.21.15.19:0/2356230505 shutdown_connections 2024-04-25T10:03:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.191+0000 7f758ba9a640 1 -- 172.21.15.19:0/2356230505 wait complete. 2024-04-25T10:03:24.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:03:14.030067+0000', 'modified': '2024-04-25T10:03:17.045113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10224}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10224': {'gid': 10224, 'name': 'a', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/717842662', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 717842662}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 717842662}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:24.610 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:03:24.610 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:03:24.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.712+0000 7f8344007640 1 Processor -- start 2024-04-25T10:03:24.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.712+0000 7f8344007640 1 -- start start 2024-04-25T10:03:24.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8344007640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f833c161710 0x7f833c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8344007640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c15e000 con 0x7f833c15dc20 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8344007640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c0d1de0 con 0x7f833c0ced80 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8344007640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833c161c60 con 0x7f833c161710 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8341da3640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f833c15dc20 0x7f833c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46178/0 (socket says 172.21.15.19:46178) 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8341da3640 1 -- 172.21.15.19:0/106020198 learned_addr learned my addr 172.21.15.19:0/106020198 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f83425a4640 1 --2- 172.21.15.19:0/106020198 >> v2:172.21.15.112:3300/0 conn(0x7f833c161710 0x7f833c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183172509 0 0) 0x7f833c15e000 con 0x7f833c15dc20 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8328003520 con 0x7f833c15dc20 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 391447073 0 0) 0x7f8328003520 con 0x7f833c15dc20 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 >> v2:172.21.15.112:3300/0 conn(0x7f833c161710 msgr2=0x7f833c0d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8340da1640 1 --2- 172.21.15.19:0/106020198 >> v2:172.21.15.112:3300/0 conn(0x7f833c161710 0x7f833c0d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 >> v1:172.21.15.112:6789/0 conn(0x7f833c0ced80 legacy=0x7f833c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c161f40 con 0x7f833c15dc20 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.713+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8338001670 con 0x7f833c15dc20 2024-04-25T10:03:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 >> v1:172.21.15.19:6789/0 conn(0x7f833c15dc20 legacy=0x7f833c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8340da1640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83280039b0 0x7f8328003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8340da1640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8328004800 0x7f8328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8340da1640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280051c0 0x7f832800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c161c60 con 0x7f83280039b0 2024-04-25T10:03:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832800c060 con 0x7f83280051c0 2024-04-25T10:03:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832800c260 con 0x7f8328004800 2024-04-25T10:03:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8341da3640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83280039b0 0x7f8328003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f83415a2640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8328004800 0x7f8328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f83425a4640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280051c0 0x7f832800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8341da3640 1 -- 172.21.15.19:0/106020198 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8328004800 msgr2=0x7f8328004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8341da3640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8328004800 0x7f8328004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8341da3640 1 -- 172.21.15.19:0/106020198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280051c0 msgr2=0x7f832800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8341da3640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280051c0 0x7f832800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f83425a4640 1 -- 172.21.15.19:0/106020198 reap_dead start 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f8341da3640 1 -- 172.21.15.19:0/106020198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8328003520 con 0x7f83280039b0 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.714+0000 7f83415a2640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8328004800 0x7f8328004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.715+0000 7f8341da3640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83280039b0 0x7f8328003dd0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f833800c240 tx=0x7f8328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ebb8d181ba5cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.715+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338007d10 con 0x7f83280039b0 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.715+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83380058d0 con 0x7f83280039b0 2024-04-25T10:03:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.715+0000 7f8340da1640 1 -- 172.21.15.19:0/106020198 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338005d40 con 0x7f83280039b0 2024-04-25T10:03:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.715+0000 7f8344007640 1 -- 172.21.15.19:0/106020198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83280039b0 msgr2=0x7f8328003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.715+0000 7f8344007640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83280039b0 0x7f8328003dd0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f833800c240 tx=0x7f8328005830 comp rx=0 tx=0).stop 2024-04-25T10:03:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.715+0000 7f8344007640 1 -- 172.21.15.19:0/106020198 shutdown_connections 2024-04-25T10:03:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.715+0000 7f8344007640 1 --2- 172.21.15.19:0/106020198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83280039b0 0x7f8328003dd0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.715+0000 7f8344007640 1 -- 172.21.15.19:0/106020198 >> 172.21.15.19:0/106020198 conn(0x7f833c0d0420 msgr2=0x7f833c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.715+0000 7f8344007640 1 -- 172.21.15.19:0/106020198 shutdown_connections 2024-04-25T10:03:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.715+0000 7f8344007640 1 -- 172.21.15.19:0/106020198 wait complete. 2024-04-25T10:03:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.717+0000 7f8344007640 1 Processor -- start 2024-04-25T10:03:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.717+0000 7f8344007640 1 -- start start 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.717+0000 7f8344007640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280039b0 0x7f833c0acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.717+0000 7f8344007640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f833c1f3c10 0x7f833c152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.717+0000 7f8344007640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f833c152d40 0x7f833c0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.717+0000 7f8344007640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0b5e50 con 0x7f833c1f3c10 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.717+0000 7f8344007640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c162500 con 0x7f83280039b0 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.717+0000 7f8344007640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0b5fd0 con 0x7f833c152d40 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.717+0000 7f83415a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f833c1f3c10 0x7f833c152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.717+0000 7f83415a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f833c1f3c10 0x7f833c152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49714/0 (socket says 172.21.15.19:49714) 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.717+0000 7f83415a2640 1 -- 172.21.15.19:0/2867619282 learned_addr learned my addr 172.21.15.19:0/2867619282 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f83425a4640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f833c152d40 0x7f833c0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f8341da3640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280039b0 0x7f833c0acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f83415a2640 1 -- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f833c152d40 msgr2=0x7f833c0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f83415a2640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f833c152d40 0x7f833c0cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f83415a2640 1 -- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280039b0 msgr2=0x7f833c0acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f83415a2640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280039b0 0x7f833c0acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f83415a2640 1 -- 172.21.15.19:0/2867619282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c0aa680 con 0x7f833c1f3c10 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f83425a4640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f833c152d40 0x7f833c0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f8341da3640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280039b0 0x7f833c0acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f83415a2640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f833c1f3c10 0x7f833c152800 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f832c00e480 tx=0x7f832c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f83327fc640 1 -- 172.21.15.19:0/2867619282 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c0052e0 con 0x7f833c1f3c10 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f83327fc640 1 -- 172.21.15.19:0/2867619282 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c007020 con 0x7f833c1f3c10 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f8344007640 1 -- 172.21.15.19:0/2867619282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833c0aa960 con 0x7f833c1f3c10 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f83327fc640 1 -- 172.21.15.19:0/2867619282 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c007210 con 0x7f833c1f3c10 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.718+0000 7f8344007640 1 -- 172.21.15.19:0/2867619282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c0aaf20 con 0x7f833c1f3c10 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.720+0000 7f83327fc640 1 -- 172.21.15.19:0/2867619282 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f832c01a020 con 0x7f833c1f3c10 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.720+0000 7f83327fc640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f832006b500 0x7f832006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.720+0000 7f83327fc640 1 -- 172.21.15.19:0/2867619282 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f832c0897f0 con 0x7f833c1f3c10 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.720+0000 7f8341da3640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f832006b500 0x7f832006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.721+0000 7f8341da3640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f832006b500 0x7f832006d9c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8338033000 tx=0x7f833802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.721+0000 7f8346de8740 1 Processor -- start 2024-04-25T10:03:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.721+0000 7f8346de8740 1 -- start start 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f8346de8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe25b230 0x55befe25b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f8346de8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe279020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f8346de8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe230f10 0x55befe277190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f8346de8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befe193a10 con 0x55befe230f10 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f8346de8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befe13cb30 con 0x55befe25b230 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f8346de8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55befe18fc50 con 0x55befe278ad0 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f83425a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe230f10 0x55befe277190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f8341da3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe25b230 0x55befe25b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f83425a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe230f10 0x55befe277190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49716/0 (socket says 172.21.15.19:49716) 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f83415a2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe279020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f83425a4640 1 -- 172.21.15.19:0/1251063832 learned_addr learned my addr 172.21.15.19:0/1251063832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f83425a4640 1 -- 172.21.15.19:0/1251063832 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 msgr2=0x55befe279020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f83425a4640 1 --2- 172.21.15.19:0/1251063832 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe279020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f83425a4640 1 -- 172.21.15.19:0/1251063832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe25b230 msgr2=0x55befe25b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f83425a4640 1 --2- 172.21.15.19:0/1251063832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe25b230 0x55befe25b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.722+0000 7f83425a4640 1 -- 172.21.15.19:0/1251063832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55befe151e10 con 0x55befe230f10 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f83425a4640 1 --2- 172.21.15.19:0/1251063832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe230f10 0x55befe277190 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f833c0b90f0 tx=0x7f8334009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6019d82c8118ed2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f831f7fe640 1 -- 172.21.15.19:0/1251063832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833401a020 con 0x55befe230f10 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f831f7fe640 1 -- 172.21.15.19:0/1251063832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334016460 con 0x55befe230f10 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f831f7fe640 1 -- 172.21.15.19:0/1251063832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334016810 con 0x55befe230f10 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f8346de8740 1 -- 172.21.15.19:0/1251063832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe230f10 msgr2=0x55befe277190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f8346de8740 1 --2- 172.21.15.19:0/1251063832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe230f10 0x55befe277190 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f833c0b90f0 tx=0x7f8334009750 comp rx=0 tx=0).stop 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f8346de8740 1 -- 172.21.15.19:0/1251063832 shutdown_connections 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f8346de8740 1 --2- 172.21.15.19:0/1251063832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe230f10 0x55befe277190 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f8346de8740 1 --2- 172.21.15.19:0/1251063832 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe279020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f8346de8740 1 --2- 172.21.15.19:0/1251063832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe25b230 0x55befe25b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f8346de8740 1 -- 172.21.15.19:0/1251063832 >> 172.21.15.19:0/1251063832 conn(0x55befe22d840 msgr2=0x55befe276040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f8346de8740 1 -- 172.21.15.19:0/1251063832 shutdown_connections 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.723+0000 7f8346de8740 1 -- 172.21.15.19:0/1251063832 wait complete. 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.724+0000 7f8346de8740 1 -- start start 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.724+0000 7f8346de8740 1 Processor -- start 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8346de8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe25ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8346de8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe22a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8346de8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe22a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8346de8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befe231cb0 con 0x55befe25b230 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8346de8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befe1500c0 con 0x55befe257e40 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8346de8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55befe22b770 con 0x55befe278ad0 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8346de8740 10 client.0 _ll_register_callbacks cb 0x55befe143ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8341da3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe25ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8341da3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe25ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49724/0 (socket says 172.21.15.19:49724) 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8341da3640 1 -- 172.21.15.19:0/3335588387 learned_addr learned my addr 172.21.15.19:0/3335588387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f83425a4640 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe22a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f83415a2640 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe22a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8341da3640 1 -- 172.21.15.19:0/3335588387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 msgr2=0x55befe22a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8341da3640 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe22a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8341da3640 1 -- 172.21.15.19:0/3335588387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 msgr2=0x55befe22a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8341da3640 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe22a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.725+0000 7f8341da3640 1 -- 172.21.15.19:0/3335588387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55befdea82d0 con 0x55befe25b230 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f8341da3640 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe25ef10 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f833800c240 tx=0x7f833802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea97e09e07db8a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f831cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f831cff9640 1 -- 172.21.15.19:0/3335588387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833804f020 con 0x55befe25b230 2024-04-25T10:03:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f831cff9640 1 -- 172.21.15.19:0/3335588387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f833804b880 con 0x55befe25b230 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f8346de8740 1 -- 172.21.15.19:0/3335588387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8334016460 con 0x55befe25b230 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f831cff9640 1 -- 172.21.15.19:0/3335588387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8338059050 con 0x55befe25b230 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f8346de8740 10 client.10392 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f8346de8740 1 -- 172.21.15.19:0/3335588387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55befdf726b0 con 0x55befe25b230 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f831cff9640 1 -- 172.21.15.19:0/3335588387 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334016460 con 0x55befe25b230 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f831cff9640 1 -- 172.21.15.19:0/3335588387 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f833805c020 con 0x55befe25b230 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f8346de8740 10 client.10392 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f8346de8740 10 client.10392 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f8346de8740 20 client.10392 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f8346de8740 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55befe26dc10 0x55befe294fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f8346de8740 4 client.10392 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.726+0000 7f8346de8740 1 -- 172.21.15.19:0/3335588387 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55befe1f2920 con 0x55befe26dc10 2024-04-25T10:03:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.727+0000 7f83415a2640 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55befe26dc10 0x55befe294fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.727+0000 7f831cff9640 10 client.10392 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.727+0000 7f83415a2640 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55befe26dc10 0x55befe294fa0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bfc6348bdc36c76 server_cookie=644d5b97143c6dc7 in_seq=0 out_seq=0 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.727+0000 7f831cff9640 1 -- 172.21.15.19:0/3335588387 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55befe1f2920 con 0x55befe26dc10 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.727+0000 7f831cff9640 10 client.10392 handle_command_reply: tid=0 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.727+0000 7f8346de8740 1 client.10392 shutdown 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f8346de8740 1 -- 172.21.15.19:0/3335588387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 msgr2=0x55befe25ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f8346de8740 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe25ef10 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f833800c240 tx=0x7f833802d000 comp rx=0 tx=0).stop 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f8346de8740 1 -- 172.21.15.19:0/3335588387 shutdown_connections 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f8346de8740 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55befe26dc10 0x55befe294fa0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f8346de8740 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe22a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f831cff9640 0 client.10392 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f8346de8740 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe22a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f8346de8740 1 --2- 172.21.15.19:0/3335588387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe25ef10 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f8346de8740 1 -- 172.21.15.19:0/3335588387 >> 172.21.15.19:0/3335588387 conn(0x55befe230f10 msgr2=0x55befe2795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f8346de8740 1 -- 172.21.15.19:0/3335588387 shutdown_connections 2024-04-25T10:03:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f8346de8740 1 -- 172.21.15.19:0/3335588387 wait complete. 2024-04-25T10:03:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.728+0000 7f8346de8740 20 client.10392 trim_cache size 0 max 16384 2024-04-25T10:03:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.745+0000 7f8346de8740 1 Processor -- start 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.745+0000 7f8346de8740 1 -- start start 2024-04-25T10:03:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f8346de8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe25a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f8346de8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe25b230 0x55befe25df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f8346de8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe278ad0 0x55befe25e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f8346de8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c200b90 con 0x55befe278ad0 2024-04-25T10:03:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f8346de8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c1ffd30 con 0x55befe257e40 2024-04-25T10:03:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f8346de8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833c1ff130 con 0x55befe25b230 2024-04-25T10:03:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f83425a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe278ad0 0x55befe25e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f83425a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe278ad0 0x55befe25e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49728/0 (socket says 172.21.15.19:49728) 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f83425a4640 1 -- 172.21.15.19:0/624212663 learned_addr learned my addr 172.21.15.19:0/624212663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f83415a2640 1 --2- 172.21.15.19:0/624212663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe25b230 0x55befe25df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f8341da3640 1 --2- 172.21.15.19:0/624212663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe25a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f83425a4640 1 -- 172.21.15.19:0/624212663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe25b230 msgr2=0x55befe25df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f83425a4640 1 --2- 172.21.15.19:0/624212663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe25b230 0x55befe25df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f83425a4640 1 -- 172.21.15.19:0/624212663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 msgr2=0x55befe25a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f83425a4640 1 --2- 172.21.15.19:0/624212663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe25a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f83425a4640 1 -- 172.21.15.19:0/624212663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55befe286ab0 con 0x55befe278ad0 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f83415a2640 1 --2- 172.21.15.19:0/624212663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe25b230 0x55befe25df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f8341da3640 1 --2- 172.21.15.19:0/624212663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe25a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f83425a4640 1 --2- 172.21.15.19:0/624212663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe278ad0 0x55befe25e440 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f833c0b6880 tx=0x7f8334016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5592711eac73e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f831effd640 1 -- 172.21.15.19:0/624212663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833401a050 con 0x55befe278ad0 2024-04-25T10:03:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f831effd640 1 -- 172.21.15.19:0/624212663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334004800 con 0x55befe278ad0 2024-04-25T10:03:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.746+0000 7f831effd640 1 -- 172.21.15.19:0/624212663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334009750 con 0x55befe278ad0 2024-04-25T10:03:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.747+0000 7f8346de8740 1 -- 172.21.15.19:0/624212663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe278ad0 msgr2=0x55befe25e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.747+0000 7f8346de8740 1 --2- 172.21.15.19:0/624212663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe278ad0 0x55befe25e440 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f833c0b6880 tx=0x7f8334016000 comp rx=0 tx=0).stop 2024-04-25T10:03:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.747+0000 7f8346de8740 1 -- 172.21.15.19:0/624212663 shutdown_connections 2024-04-25T10:03:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.747+0000 7f8346de8740 1 --2- 172.21.15.19:0/624212663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe278ad0 0x55befe25e440 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.747+0000 7f8346de8740 1 --2- 172.21.15.19:0/624212663 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe25b230 0x55befe25df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.747+0000 7f8346de8740 1 --2- 172.21.15.19:0/624212663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe25a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.747+0000 7f8346de8740 1 -- 172.21.15.19:0/624212663 >> 172.21.15.19:0/624212663 conn(0x55befe230f10 msgr2=0x55befe25ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.747+0000 7f8346de8740 1 -- 172.21.15.19:0/624212663 shutdown_connections 2024-04-25T10:03:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.747+0000 7f8346de8740 1 -- 172.21.15.19:0/624212663 wait complete. 2024-04-25T10:03:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.748+0000 7f8346de8740 1 -- start start 2024-04-25T10:03:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.748+0000 7f8346de8740 1 Processor -- start 2024-04-25T10:03:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f8346de8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe255980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f8346de8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe278f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f8346de8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe25df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f8346de8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befe287630 con 0x55befe25b230 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f8346de8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c1ff730 con 0x55befe257e40 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f8346de8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55befe287070 con 0x55befe278ad0 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f8346de8740 10 client.0 _ll_register_callbacks cb 0x55befe22bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f83415a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe278f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f83415a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe278f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49736/0 (socket says 172.21.15.19:49736) 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f83415a2640 1 -- 172.21.15.19:0/166470209 learned_addr learned my addr 172.21.15.19:0/166470209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f8341da3640 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe255980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f83425a4640 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe25df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f83415a2640 1 -- 172.21.15.19:0/166470209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 msgr2=0x55befe25df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f83415a2640 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe25df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f83415a2640 1 -- 172.21.15.19:0/166470209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 msgr2=0x55befe255980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f83415a2640 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe255980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f83415a2640 1 -- 172.21.15.19:0/166470209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55befe2e0e60 con 0x55befe25b230 2024-04-25T10:03:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f83415a2640 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe278f80 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f832c00ee40 tx=0x7f832c087af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a73ed9f582766d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f831cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.749+0000 7f831cff9640 1 -- 172.21.15.19:0/166470209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c075060 con 0x55befe25b230 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f831cff9640 1 -- 172.21.15.19:0/166470209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f832c0544e0 con 0x55befe25b230 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f8346de8740 1 -- 172.21.15.19:0/166470209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8334004800 con 0x55befe25b230 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f831cff9640 1 -- 172.21.15.19:0/166470209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f832c07b050 con 0x55befe25b230 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f8346de8740 10 client.10398 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f8346de8740 1 -- 172.21.15.19:0/166470209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55befe2e0e60 con 0x55befe25b230 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f831cff9640 1 -- 172.21.15.19:0/166470209 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334004800 con 0x55befe25b230 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f831cff9640 1 -- 172.21.15.19:0/166470209 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f832c07e020 con 0x55befe25b230 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f8346de8740 10 client.10398 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f8346de8740 10 client.10398 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f8346de8740 20 client.10398 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f8346de8740 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55befe25e440 0x55befe2e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f8346de8740 4 client.10398 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f8346de8740 1 -- 172.21.15.19:0/166470209 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55befe1a0f50 con 0x55befe25e440 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f8341da3640 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55befe25e440 0x55befe2e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.750+0000 7f831cff9640 10 client.10398 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8341da3640 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55befe25e440 0x55befe2e4590 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52583f28f600299 server_cookie=d5a239d5448063d7 in_seq=0 out_seq=0 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f831cff9640 1 -- 172.21.15.19:0/166470209 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55befe1a0f50 con 0x55befe25e440 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f831cff9640 10 client.10398 handle_command_reply: tid=0 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 1 client.10398 shutdown 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 1 -- 172.21.15.19:0/166470209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 msgr2=0x55befe278f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe278f80 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f832c00ee40 tx=0x7f832c087af0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 1 -- 172.21.15.19:0/166470209 shutdown_connections 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55befe25e440 0x55befe2e4590 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55befe278ad0 0x55befe25df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55befe25b230 0x55befe278f80 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f831cff9640 0 client.10398 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 1 --2- 172.21.15.19:0/166470209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55befe257e40 0x55befe255980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 1 -- 172.21.15.19:0/166470209 >> 172.21.15.19:0/166470209 conn(0x55befe230f10 msgr2=0x55befe2312f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 1 -- 172.21.15.19:0/166470209 shutdown_connections 2024-04-25T10:03:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 1 -- 172.21.15.19:0/166470209 wait complete. 2024-04-25T10:03:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.751+0000 7f8346de8740 20 client.10398 trim_cache size 0 max 16384 2024-04-25T10:03:24.762 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:03:24.762 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T10:03:24.762 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:03:24.762 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 10224, 2024-04-25T10:03:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:03:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:03:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:03:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 10.093021176000001, 2024-04-25T10:03:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 102, 2024-04-25T10:03:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 154, 2024-04-25T10:03:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 154, 2024-04-25T10:03:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 15.450877909000001 2024-04-25T10:03:24.763 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:03:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 -- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f832006b500 msgr2=0x7f832006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f832006b500 0x7f832006d9c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8338033000 tx=0x7f833802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 -- 172.21.15.19:0/2867619282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f833c1f3c10 msgr2=0x7f833c152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f833c1f3c10 0x7f833c152800 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f832c00e480 tx=0x7f832c00e940 comp rx=0 tx=0).stop 2024-04-25T10:03:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 -- 172.21.15.19:0/2867619282 shutdown_connections 2024-04-25T10:03:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f832006b500 0x7f832006d9c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f833c152d40 0x7f833c0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f833c1f3c10 0x7f833c152800 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 --2- 172.21.15.19:0/2867619282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83280039b0 0x7f833c0acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 -- 172.21.15.19:0/2867619282 >> 172.21.15.19:0/2867619282 conn(0x7f833c0d0420 msgr2=0x7f833c0b8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 -- 172.21.15.19:0/2867619282 shutdown_connections 2024-04-25T10:03:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.752+0000 7f8344007640 1 -- 172.21.15.19:0/2867619282 wait complete. 2024-04-25T10:03:24.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:03:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.863+0000 7eff84ff4640 1 Processor -- start 2024-04-25T10:03:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.863+0000 7eff84ff4640 1 -- start start 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff84ff4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff8015dc90 0x7eff801625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff84ff4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7eff800cc8e0 0x7eff8015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff84ff4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8015f390 con 0x7eff800d1810 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff84ff4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff800cce50 con 0x7eff800cc8e0 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff84ff4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff80161af0 con 0x7eff8015dc90 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff77fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7eff800cc8e0 0x7eff8015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff77fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7eff800cc8e0 0x7eff8015ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49744/0 (socket says 172.21.15.19:49744) 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff77fff640 1 -- 172.21.15.19:0/2370059231 learned_addr learned my addr 172.21.15.19:0/2370059231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff7e59c640 1 --2- 172.21.15.19:0/2370059231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff8015dc90 0x7eff801625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff77fff640 1 -- 172.21.15.19:0/2370059231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff8015dc90 msgr2=0x7eff801625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff77fff640 1 --2- 172.21.15.19:0/2370059231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff8015dc90 0x7eff801625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff77fff640 1 -- 172.21.15.19:0/2370059231 >> v1:172.21.15.112:6789/0 conn(0x7eff800d1810 legacy=0x7eff800cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.864+0000 7eff77fff640 1 -- 172.21.15.19:0/2370059231 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80161dd0 con 0x7eff800cc8e0 2024-04-25T10:03:24.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.865+0000 7eff77fff640 1 --2- 172.21.15.19:0/2370059231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff800cc8e0 0x7eff8015ee50 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7eff68009870 tx=0x7eff6802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f81165349e6ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.865+0000 7eff7dd9b640 1 -- 172.21.15.19:0/2370059231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68037070 con 0x7eff800cc8e0 2024-04-25T10:03:24.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.865+0000 7eff7dd9b640 1 -- 172.21.15.19:0/2370059231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff68032440 con 0x7eff800cc8e0 2024-04-25T10:03:24.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.865+0000 7eff7dd9b640 1 -- 172.21.15.19:0/2370059231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68032810 con 0x7eff800cc8e0 2024-04-25T10:03:24.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.865+0000 7eff84ff4640 1 -- 172.21.15.19:0/2370059231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff800cc8e0 msgr2=0x7eff8015ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.865+0000 7eff84ff4640 1 --2- 172.21.15.19:0/2370059231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff800cc8e0 0x7eff8015ee50 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7eff68009870 tx=0x7eff6802a180 comp rx=0 tx=0).stop 2024-04-25T10:03:24.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.866+0000 7eff84ff4640 1 -- 172.21.15.19:0/2370059231 shutdown_connections 2024-04-25T10:03:24.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.866+0000 7eff84ff4640 1 --2- 172.21.15.19:0/2370059231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff800cc8e0 0x7eff8015ee50 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.866+0000 7eff84ff4640 1 --2- 172.21.15.19:0/2370059231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff8015dc90 0x7eff801625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.866+0000 7eff84ff4640 1 -- 172.21.15.19:0/2370059231 >> 172.21.15.19:0/2370059231 conn(0x7eff8006f590 msgr2=0x7eff8006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.866+0000 7eff84ff4640 1 -- 172.21.15.19:0/2370059231 shutdown_connections 2024-04-25T10:03:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.866+0000 7eff84ff4640 1 -- 172.21.15.19:0/2370059231 wait complete. 2024-04-25T10:03:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.867+0000 7eff84ff4640 1 Processor -- start 2024-04-25T10:03:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.867+0000 7eff84ff4640 1 -- start start 2024-04-25T10:03:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff84ff4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff800cc8e0 0x7eff80153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff84ff4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff800d1810 0x7eff800b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff84ff4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff8015dc90 0x7eff800ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff84ff4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8015e070 con 0x7eff800cc8e0 2024-04-25T10:03:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff84ff4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff800cd130 con 0x7eff800d1810 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff84ff4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8015e1f0 con 0x7eff8015dc90 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff7e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff800cc8e0 0x7eff80153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff7e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff800cc8e0 0x7eff80153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49754/0 (socket says 172.21.15.19:49754) 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff7e59c640 1 -- 172.21.15.19:0/3375480140 learned_addr learned my addr 172.21.15.19:0/3375480140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff77fff640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff800d1810 0x7eff800b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff7ed9d640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff8015dc90 0x7eff800ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff7e59c640 1 -- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff8015dc90 msgr2=0x7eff800ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff7e59c640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff8015dc90 0x7eff800ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff7e59c640 1 -- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff800d1810 msgr2=0x7eff800b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff7e59c640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff800d1810 0x7eff800b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff7e59c640 1 -- 172.21.15.19:0/3375480140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff801f7630 con 0x7eff800cc8e0 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff7e59c640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff800cc8e0 0x7eff80153ed0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7eff7000d800 tx=0x7eff7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff76ffd640 1 -- 172.21.15.19:0/3375480140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7000b660 con 0x7eff800cc8e0 2024-04-25T10:03:24.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff76ffd640 1 -- 172.21.15.19:0/3375480140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff70019070 con 0x7eff800cc8e0 2024-04-25T10:03:24.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff84ff4640 1 -- 172.21.15.19:0/3375480140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff801f78d0 con 0x7eff800cc8e0 2024-04-25T10:03:24.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff76ffd640 1 -- 172.21.15.19:0/3375480140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7000ba20 con 0x7eff800cc8e0 2024-04-25T10:03:24.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.868+0000 7eff84ff4640 1 -- 172.21.15.19:0/3375480140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff801f7e90 con 0x7eff800cc8e0 2024-04-25T10:03:24.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.869+0000 7eff84ff4640 1 -- 172.21.15.19:0/3375480140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff80073990 con 0x7eff800cc8e0 2024-04-25T10:03:24.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.870+0000 7eff76ffd640 1 -- 172.21.15.19:0/3375480140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff7000bbc0 con 0x7eff800cc8e0 2024-04-25T10:03:24.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.870+0000 7eff76ffd640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff6006b4d0 0x7eff6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.870+0000 7eff76ffd640 1 -- 172.21.15.19:0/3375480140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7eff7008a790 con 0x7eff800cc8e0 2024-04-25T10:03:24.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.870+0000 7eff77fff640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff6006b4d0 0x7eff6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.871+0000 7eff77fff640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff6006b4d0 0x7eff6006d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7eff68034000 tx=0x7eff6802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:24.875+0000 7eff76ffd640 1 -- 172.21.15.19:0/3375480140 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff7001c360 con 0x7eff800cc8e0 2024-04-25T10:03:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.091+0000 7eff84ff4640 1 -- 172.21.15.19:0/3375480140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7eff800cf6b0 con 0x7eff800cc8e0 2024-04-25T10:03:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.091+0000 7eff76ffd640 1 -- 172.21.15.19:0/3375480140 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) v1 ==== 53+0+73 (secure 0 0 0) 0x7eff70059510 con 0x7eff800cc8e0 2024-04-25T10:03:25.096 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:03:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 -- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff6006b4d0 msgr2=0x7eff6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff6006b4d0 0x7eff6006d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7eff68034000 tx=0x7eff6802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 -- 172.21.15.19:0/3375480140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff800cc8e0 msgr2=0x7eff80153ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff800cc8e0 0x7eff80153ed0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7eff7000d800 tx=0x7eff7000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 -- 172.21.15.19:0/3375480140 shutdown_connections 2024-04-25T10:03:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7eff6006b4d0 0x7eff6006d990 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff8015dc90 0x7eff800ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff800d1810 0x7eff800b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 --2- 172.21.15.19:0/3375480140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff800cc8e0 0x7eff80153ed0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 -- 172.21.15.19:0/3375480140 >> 172.21.15.19:0/3375480140 conn(0x7eff8006f590 msgr2=0x7eff8015cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 -- 172.21.15.19:0/3375480140 shutdown_connections 2024-04-25T10:03:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.093+0000 7eff84ff4640 1 -- 172.21.15.19:0/3375480140 wait complete. 2024-04-25T10:03:25.109 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:03:25.109 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:03:25.109 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T10:03:25.109 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:03:25.109 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:25.109 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:03:25.110 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:03:25.110 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:03:25.110 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:25.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.110 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:25.126 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42141sec preferred_lft 42141sec 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:03:25.127 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:25.128 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T10:03:25.128 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:25.128 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:25.128 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:25.128 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T10:03:25.128 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:25.128 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:25.128 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:25.128 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T10:03:25.128 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:25.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.129 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:03:25.189 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T10:03:25.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.190 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T10:03:25.246 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T10:03:25.246 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:03:25.246 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:03:25.247 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:03:25.247 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:03:25.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.247 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T10:03:25.247 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:03:25.247 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:03:25.247 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:03:25.247 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:03:25.247 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:03:25.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.333 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T10:03:25.347 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:03:25.348 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:03:25.348 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:25.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T10:03:25.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:25.412 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:25.467 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:25.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:25.468 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:25.522 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:03:25.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:25.605 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:25.605 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:03:25.612 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:25.612 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:03:25.612 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T10:03:25.612 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T10:03:25.612 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T10:03:25.613 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:03:25.613 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:03:25.613 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:03:25.613 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:03:25.613 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:03:25.614 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:03:25.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.614 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.680 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:03:25.741 INFO:teuthology.orchestra.run.smithi019.stdout:client7838 2024-04-25T10:03:25.742 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:03:25.742 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:25.797 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:03:25.798 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:03:25.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.935+0000 7f4d1d789640 1 Processor -- start 2024-04-25T10:03:25.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.936+0000 7f4d1d789640 1 -- start start 2024-04-25T10:03:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.936+0000 7f4d1d789640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d1806c330 0x7f4d180b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.936+0000 7f4d1d789640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4d1806c7e0 0x7f4d180b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.936+0000 7f4d1d789640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d180d1d50 con 0x7f4d18158230 2024-04-25T10:03:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.936+0000 7f4d1d789640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d18158610 con 0x7f4d1806c7e0 2024-04-25T10:03:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.936+0000 7f4d1d789640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1806bea0 con 0x7f4d1806c330 2024-04-25T10:03:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d16ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4d18158230 0x7f4d180d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46194/0 (socket says 172.21.15.19:46194) 2024-04-25T10:03:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d16ffd640 1 -- 172.21.15.19:0/3631273222 learned_addr learned my addr 172.21.15.19:0/3631273222 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3631273222 >> v2:172.21.15.112:3300/0 conn(0x7f4d1806c7e0 0x7f4d180b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d167fc640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d1806c330 0x7f4d180b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214239330 0 0) 0x7f4d180d1d50 con 0x7f4d18158230 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d04003520 con 0x7f4d18158230 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 719970953 0 0) 0x7f4d04003520 con 0x7f4d18158230 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d1806c330 msgr2=0x7f4d180b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d1806c330 0x7f4d180b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 >> v2:172.21.15.112:3300/0 conn(0x7f4d1806c7e0 msgr2=0x7f4d180b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 --2- 172.21.15.19:0/3631273222 >> v2:172.21.15.112:3300/0 conn(0x7f4d1806c7e0 0x7f4d180b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180d2020 con 0x7f4d18158230 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f4d10001670 con 0x7f4d18158230 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 >> v1:172.21.15.19:6789/0 conn(0x7f4d18158230 legacy=0x7f4d180d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d1806c330 0x7f4d040039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d04003f30 0x7f4d04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d04004d40 0x7f4d040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d18158610 con 0x7f4d04004d40 2024-04-25T10:03:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1806bea0 con 0x7f4d04003f30 2024-04-25T10:03:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0400b9c0 con 0x7f4d1806c330 2024-04-25T10:03:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d04004d40 0x7f4d040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.937+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d1806c330 0x7f4d040039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.938+0000 7f4d167fc640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d04003f30 0x7f4d04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.938+0000 7f4d177fe640 1 -- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d1806c330 msgr2=0x7f4d040039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.938+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d1806c330 0x7f4d040039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.938+0000 7f4d177fe640 1 -- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d04003f30 msgr2=0x7f4d04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.938+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d04003f30 0x7f4d04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.938+0000 7f4d177fe640 1 -- 172.21.15.19:0/3631273222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d04003520 con 0x7f4d04004d40 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.938+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d1806c330 0x7f4d040039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.938+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d04004d40 0x7f4d040051c0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009fc0 tx=0x7f4d04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8621f51829e322 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.939+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08009c10 con 0x7f4d04004d40 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.939+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d08036070 con 0x7f4d04004d40 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.939+0000 7f4d15ffb640 1 -- 172.21.15.19:0/3631273222 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0802d070 con 0x7f4d04004d40 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.939+0000 7f4d1d789640 1 -- 172.21.15.19:0/3631273222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d04004d40 msgr2=0x7f4d040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.939+0000 7f4d1d789640 1 --2- 172.21.15.19:0/3631273222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d04004d40 0x7f4d040051c0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009fc0 tx=0x7f4d04005830 comp rx=0 tx=0).stop 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.939+0000 7f4d177fe640 1 -- 172.21.15.19:0/3631273222 reap_dead start 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.939+0000 7f4d1d789640 1 -- 172.21.15.19:0/3631273222 shutdown_connections 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.939+0000 7f4d1d789640 1 -- 172.21.15.19:0/3631273222 >> 172.21.15.19:0/3631273222 conn(0x7f4d18061600 msgr2=0x7f4d180619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.939+0000 7f4d1d789640 1 -- 172.21.15.19:0/3631273222 shutdown_connections 2024-04-25T10:03:25.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.939+0000 7f4d1d789640 1 -- 172.21.15.19:0/3631273222 wait complete. 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.940+0000 7f4d1d789640 1 Processor -- start 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.940+0000 7f4d1d789640 1 -- start start 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.941+0000 7f4d1d789640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d04004d40 0x7f4d181461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.941+0000 7f4d1d789640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18158790 0x7f4d18154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.941+0000 7f4d1d789640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d18158c10 0x7f4d181547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.941+0000 7f4d1d789640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180d25e0 con 0x7f4d18158790 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.941+0000 7f4d1d789640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1806bb30 con 0x7f4d18158c10 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.941+0000 7f4d1d789640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1815e320 con 0x7f4d04004d40 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.941+0000 7f4d167fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18158790 0x7f4d18154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.941+0000 7f4d167fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18158790 0x7f4d18154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49762/0 (socket says 172.21.15.19:49762) 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.941+0000 7f4d167fc640 1 -- 172.21.15.19:0/3957090481 learned_addr learned my addr 172.21.15.19:0/3957090481 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.941+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d18158c10 0x7f4d181547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.941+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d04004d40 0x7f4d181461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.942+0000 7f4d167fc640 1 -- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d04004d40 msgr2=0x7f4d181461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.942+0000 7f4d167fc640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d04004d40 0x7f4d181461b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.942+0000 7f4d167fc640 1 -- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d18158c10 msgr2=0x7f4d181547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.942+0000 7f4d167fc640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d18158c10 0x7f4d181547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.942+0000 7f4d167fc640 1 -- 172.21.15.19:0/3957090481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180cada0 con 0x7f4d18158790 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.942+0000 7f4d167fc640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18158790 0x7f4d18154280 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f4d00011910 tx=0x7f4d00011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.942+0000 7f4d0f7fe640 1 -- 172.21.15.19:0/3957090481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00005ba0 con 0x7f4d18158790 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.942+0000 7f4d0f7fe640 1 -- 172.21.15.19:0/3957090481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d00005d40 con 0x7f4d18158790 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.942+0000 7f4d1d789640 1 -- 172.21.15.19:0/3957090481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1815b3b0 con 0x7f4d18158790 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.942+0000 7f4d0f7fe640 1 -- 172.21.15.19:0/3957090481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000fa40 con 0x7f4d18158790 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.942+0000 7f4d1d789640 1 -- 172.21.15.19:0/3957090481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d180cb320 con 0x7f4d18158790 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.943+0000 7f4d0f7fe640 1 -- 172.21.15.19:0/3957090481 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d00012040 con 0x7f4d18158790 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.944+0000 7f4d0f7fe640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cf806b4d0 0x7f4cf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.944+0000 7f4d0f7fe640 1 -- 172.21.15.19:0/3957090481 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d0008a8c0 con 0x7f4d18158790 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.944+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cf806b4d0 0x7f4cf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.945+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cf806b4d0 0x7f4cf806d990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000fb0 tx=0x7f4d10007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.945+0000 7f4d2056f740 1 Processor -- start 2024-04-25T10:03:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.945+0000 7f4d2056f740 1 -- start start 2024-04-25T10:03:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1aa0 0x55e2191d42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191a4700 0x55e2191a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d2056f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e21910de00 con 0x55e2191a4700 2024-04-25T10:03:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d2056f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2190b6d20 con 0x55e2191d1e80 2024-04-25T10:03:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d2056f740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e219109ea0 con 0x55e2191d1aa0 2024-04-25T10:03:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191a4700 0x55e2191a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d16ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1aa0 0x55e2191d42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191a4700 0x55e2191a7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49768/0 (socket says 172.21.15.19:49768) 2024-04-25T10:03:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d177fe640 1 -- 172.21.15.19:0/3977496655 learned_addr learned my addr 172.21.15.19:0/3977496655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d167fc640 1 --2- 172.21.15.19:0/3977496655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d177fe640 1 -- 172.21.15.19:0/3977496655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1aa0 msgr2=0x55e2191d42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3977496655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1aa0 0x55e2191d42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d177fe640 1 -- 172.21.15.19:0/3977496655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 msgr2=0x55e2191d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3977496655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d177fe640 1 -- 172.21.15.19:0/3977496655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2190d8df0 con 0x55e2191a4700 2024-04-25T10:03:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3977496655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191a4700 0x55e2191a7f10 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f4d08007090 tx=0x7f4d0802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4329656418dac665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.946+0000 7f4d0cff9640 1 -- 172.21.15.19:0/3977496655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08003cd0 con 0x55e2191a4700 2024-04-25T10:03:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.947+0000 7f4d0cff9640 1 -- 172.21.15.19:0/3977496655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d08036040 con 0x55e2191a4700 2024-04-25T10:03:25.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.947+0000 7f4d0cff9640 1 -- 172.21.15.19:0/3977496655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08004280 con 0x55e2191a4700 2024-04-25T10:03:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.947+0000 7f4d2056f740 1 -- 172.21.15.19:0/3977496655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191a4700 msgr2=0x55e2191a7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.947+0000 7f4d2056f740 1 --2- 172.21.15.19:0/3977496655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191a4700 0x55e2191a7f10 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f4d08007090 tx=0x7f4d0802d000 comp rx=0 tx=0).stop 2024-04-25T10:03:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.947+0000 7f4d2056f740 1 -- 172.21.15.19:0/3977496655 shutdown_connections 2024-04-25T10:03:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.947+0000 7f4d2056f740 1 --2- 172.21.15.19:0/3977496655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191a4700 0x55e2191a7f10 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.947+0000 7f4d2056f740 1 --2- 172.21.15.19:0/3977496655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.947+0000 7f4d2056f740 1 --2- 172.21.15.19:0/3977496655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1aa0 0x55e2191d42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.947+0000 7f4d2056f740 1 -- 172.21.15.19:0/3977496655 >> 172.21.15.19:0/3977496655 conn(0x55e2191a6970 msgr2=0x55e2191ef2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.947+0000 7f4d2056f740 1 -- 172.21.15.19:0/3977496655 shutdown_connections 2024-04-25T10:03:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.947+0000 7f4d2056f740 1 -- 172.21.15.19:0/3977496655 wait complete. 2024-04-25T10:03:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.948+0000 7f4d2056f740 1 -- start start 2024-04-25T10:03:25.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.948+0000 7f4d2056f740 1 Processor -- start 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e2191cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191ec2e0 0x55e2191cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d2056f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2191a9b20 con 0x55e2191d1aa0 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d2056f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2190ac590 con 0x55e2191d1e80 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d2056f740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2191a9e00 con 0x55e2191ec2e0 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d2056f740 10 client.0 _ll_register_callbacks cb 0x55e2190b8e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e2191cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e2191cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49778/0 (socket says 172.21.15.19:49778) 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d16ffd640 1 -- 172.21.15.19:0/567206176 learned_addr learned my addr 172.21.15.19:0/567206176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d167fc640 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d177fe640 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191ec2e0 0x55e2191cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d16ffd640 1 -- 172.21.15.19:0/567206176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191ec2e0 msgr2=0x55e2191cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191ec2e0 0x55e2191cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d16ffd640 1 -- 172.21.15.19:0/567206176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 msgr2=0x55e2191ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d16ffd640 1 -- 172.21.15.19:0/567206176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e218e223f0 con 0x55e2191d1aa0 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e2191cfb40 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f4d10015a20 tx=0x7f4d10000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba2043644fe2dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/567206176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1001b030 con 0x55e2191d1aa0 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/567206176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d10014b60 con 0x55e2191d1aa0 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4d2056f740 1 -- 172.21.15.19:0/567206176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d08036040 con 0x55e2191d1aa0 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.949+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/567206176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d10030050 con 0x55e2191d1aa0 2024-04-25T10:03:25.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4d2056f740 10 client.10416 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:25.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4d2056f740 1 -- 172.21.15.19:0/567206176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55e218eec7d0 con 0x55e2191d1aa0 2024-04-25T10:03:25.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/567206176 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d08036040 con 0x55e2191d1aa0 2024-04-25T10:03:25.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/567206176 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d10033020 con 0x55e2191d1aa0 2024-04-25T10:03:25.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4d2056f740 10 client.10416 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:25.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4d2056f740 10 client.10416 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:03:25.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4d2056f740 20 client.10416 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:25.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4d2056f740 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e2191eaf30 0x55e21920cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4d2056f740 4 client.10416 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:25.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4d2056f740 1 -- 172.21.15.19:0/567206176 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e21911b340 con 0x55e2191eaf30 2024-04-25T10:03:25.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4d167fc640 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e2191eaf30 0x55e21920cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4cf5ffb640 10 client.10416 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:25.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.950+0000 7f4d167fc640 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e2191eaf30 0x55e21920cb60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c504ae9cd8b7e951 server_cookie=67d4808ac135ba48 in_seq=0 out_seq=0 2024-04-25T10:03:25.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/567206176 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e21911b340 con 0x55e2191eaf30 2024-04-25T10:03:25.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4cf5ffb640 10 client.10416 handle_command_reply: tid=0 2024-04-25T10:03:25.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 1 client.10416 shutdown 2024-04-25T10:03:25.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 1 -- 172.21.15.19:0/567206176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 msgr2=0x55e2191cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e2191cfb40 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f4d10015a20 tx=0x7f4d10000fe0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 1 -- 172.21.15.19:0/567206176 shutdown_connections 2024-04-25T10:03:25.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e2191eaf30 0x55e21920cb60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191ec2e0 0x55e2191cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4cf5ffb640 0 client.10416 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:25.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 1 --2- 172.21.15.19:0/567206176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e2191cfb40 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 1 -- 172.21.15.19:0/567206176 >> 172.21.15.19:0/567206176 conn(0x55e2191a4700 msgr2=0x55e2191cc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 1 -- 172.21.15.19:0/567206176 shutdown_connections 2024-04-25T10:03:25.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 1 -- 172.21.15.19:0/567206176 wait complete. 2024-04-25T10:03:25.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.951+0000 7f4d2056f740 20 client.10416 trim_cache size 0 max 16384 2024-04-25T10:03:25.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.963+0000 7f4d2056f740 1 Processor -- start 2024-04-25T10:03:25.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.963+0000 7f4d2056f740 1 -- start start 2024-04-25T10:03:25.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1aa0 0x55e2191cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191ec2e0 0x55e2191ef310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d2056f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d18200b60 con 0x55e2191ec2e0 2024-04-25T10:03:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d2056f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d181ffd90 con 0x55e2191d1e80 2024-04-25T10:03:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d2056f740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d181ff190 con 0x55e2191d1aa0 2024-04-25T10:03:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191ec2e0 0x55e2191ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191ec2e0 0x55e2191ef310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49786/0 (socket says 172.21.15.19:49786) 2024-04-25T10:03:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d177fe640 1 -- 172.21.15.19:0/3317053784 learned_addr learned my addr 172.21.15.19:0/3317053784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d167fc640 1 --2- 172.21.15.19:0/3317053784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/3317053784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1aa0 0x55e2191cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d177fe640 1 -- 172.21.15.19:0/3317053784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1aa0 msgr2=0x55e2191cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3317053784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1aa0 0x55e2191cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d177fe640 1 -- 172.21.15.19:0/3317053784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 msgr2=0x55e2191d3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3317053784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191d3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d177fe640 1 -- 172.21.15.19:0/3317053784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2191d85e0 con 0x55e2191ec2e0 2024-04-25T10:03:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.964+0000 7f4d177fe640 1 --2- 172.21.15.19:0/3317053784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191ec2e0 0x55e2191ef310 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800e200 tx=0x7f4d0800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc8ba97c7831060 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4cf7fff640 1 -- 172.21.15.19:0/3317053784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08031670 con 0x55e2191ec2e0 2024-04-25T10:03:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4cf7fff640 1 -- 172.21.15.19:0/3317053784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d08036070 con 0x55e2191ec2e0 2024-04-25T10:03:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4cf7fff640 1 -- 172.21.15.19:0/3317053784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08031a30 con 0x55e2191ec2e0 2024-04-25T10:03:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4d2056f740 1 -- 172.21.15.19:0/3317053784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191ec2e0 msgr2=0x55e2191ef310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4d2056f740 1 --2- 172.21.15.19:0/3317053784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191ec2e0 0x55e2191ef310 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800e200 tx=0x7f4d0800b040 comp rx=0 tx=0).stop 2024-04-25T10:03:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4d2056f740 1 -- 172.21.15.19:0/3317053784 shutdown_connections 2024-04-25T10:03:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4d2056f740 1 --2- 172.21.15.19:0/3317053784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191ec2e0 0x55e2191ef310 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4d2056f740 1 --2- 172.21.15.19:0/3317053784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191d1e80 0x55e2191d3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4d2056f740 1 --2- 172.21.15.19:0/3317053784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1aa0 0x55e2191cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4d2056f740 1 -- 172.21.15.19:0/3317053784 >> 172.21.15.19:0/3317053784 conn(0x55e2191a4700 msgr2=0x55e2191eb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4d2056f740 1 -- 172.21.15.19:0/3317053784 shutdown_connections 2024-04-25T10:03:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.965+0000 7f4d2056f740 1 -- 172.21.15.19:0/3317053784 wait complete. 2024-04-25T10:03:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.966+0000 7f4d2056f740 1 -- start start 2024-04-25T10:03:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.966+0000 7f4d2056f740 1 Processor -- start 2024-04-25T10:03:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.966+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e21925da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.966+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1e80 0x55e2191a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.966+0000 7f4d2056f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191ec2e0 0x55e2191d7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.966+0000 7f4d2056f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e21920f340 con 0x55e2191d1aa0 2024-04-25T10:03:25.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.966+0000 7f4d2056f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d181ff790 con 0x55e2191ec2e0 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.966+0000 7f4d2056f740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2191ec720 con 0x55e2191d1e80 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.966+0000 7f4d2056f740 10 client.0 _ll_register_callbacks cb 0x7f4d00006f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e21925da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d16ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e21925da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49802/0 (socket says 172.21.15.19:49802) 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d16ffd640 1 -- 172.21.15.19:0/1998787998 learned_addr learned my addr 172.21.15.19:0/1998787998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d177fe640 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191ec2e0 0x55e2191d7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d167fc640 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1e80 0x55e2191a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d16ffd640 1 -- 172.21.15.19:0/1998787998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1e80 msgr2=0x55e2191a7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1e80 0x55e2191a7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d16ffd640 1 -- 172.21.15.19:0/1998787998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191ec2e0 msgr2=0x55e2191d7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191ec2e0 0x55e2191d7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d16ffd640 1 -- 172.21.15.19:0/1998787998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2191cade0 con 0x55e2191d1aa0 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d16ffd640 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e21925da40 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f4d10015a20 tx=0x7f4d10015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8520e03e4e6f3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/1998787998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1001b040 con 0x55e2191d1aa0 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/1998787998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d10023650 con 0x55e2191d1aa0 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d2056f740 1 -- 172.21.15.19:0/1998787998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d08036070 con 0x55e2191d1aa0 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/1998787998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d10043050 con 0x55e2191d1aa0 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d2056f740 10 client.10422 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.967+0000 7f4d2056f740 1 -- 172.21.15.19:0/1998787998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55e2191a82c0 con 0x55e2191d1aa0 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/1998787998 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d08036070 con 0x55e2191d1aa0 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/1998787998 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d10046020 con 0x55e2191d1aa0 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 10 client.10422 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 10 client.10422 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 20 client.10422 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e2191ca360 0x55e219201c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 4 client.10422 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 1 -- 172.21.15.19:0/1998787998 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e219115690 con 0x55e2191ca360 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d167fc640 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e2191ca360 0x55e219201c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4cf5ffb640 10 client.10422 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d167fc640 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e2191ca360 0x55e219201c20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e464324fcab8f5e3 server_cookie=281c4b18929beb06 in_seq=0 out_seq=0 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4cf5ffb640 1 -- 172.21.15.19:0/1998787998 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e219115690 con 0x55e2191ca360 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4cf5ffb640 10 client.10422 handle_command_reply: tid=0 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 1 client.10422 shutdown 2024-04-25T10:03:25.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 1 -- 172.21.15.19:0/1998787998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 msgr2=0x55e21925da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e21925da40 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f4d10015a20 tx=0x7f4d10015000 comp rx=0 tx=0).stop 2024-04-25T10:03:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 1 -- 172.21.15.19:0/1998787998 shutdown_connections 2024-04-25T10:03:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e2191ca360 0x55e219201c20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e2191ec2e0 0x55e2191d7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e2191d1e80 0x55e2191a7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 1 --2- 172.21.15.19:0/1998787998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2191d1aa0 0x55e21925da40 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4cf5ffb640 0 client.10422 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.968+0000 7f4d2056f740 1 -- 172.21.15.19:0/1998787998 >> 172.21.15.19:0/1998787998 conn(0x55e2191a4700 msgr2=0x55e2191ef310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d2056f740 1 -- 172.21.15.19:0/1998787998 shutdown_connections 2024-04-25T10:03:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d2056f740 1 -- 172.21.15.19:0/1998787998 wait complete. 2024-04-25T10:03:25.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d2056f740 20 client.10422 trim_cache size 0 max 16384 2024-04-25T10:03:25.983 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:03:25.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 -- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cf806b4d0 msgr2=0x7f4cf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cf806b4d0 0x7f4cf806d990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000fb0 tx=0x7f4d10007040 comp rx=0 tx=0).stop 2024-04-25T10:03:25.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 -- 172.21.15.19:0/3957090481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18158790 msgr2=0x7f4d18154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18158790 0x7f4d18154280 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f4d00011910 tx=0x7f4d00011d10 comp rx=0 tx=0).stop 2024-04-25T10:03:25.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 -- 172.21.15.19:0/3957090481 shutdown_connections 2024-04-25T10:03:25.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4cf806b4d0 0x7f4cf806d990 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d18158c10 0x7f4d181547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d18158790 0x7f4d18154280 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 --2- 172.21.15.19:0/3957090481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d04004d40 0x7f4d181461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 -- 172.21.15.19:0/3957090481 >> 172.21.15.19:0/3957090481 conn(0x7f4d18061600 msgr2=0x7f4d1815e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 -- 172.21.15.19:0/3957090481 shutdown_connections 2024-04-25T10:03:25.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:25.969+0000 7f4d1d789640 1 -- 172.21.15.19:0/3957090481 wait complete. 2024-04-25T10:03:25.986 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:03:26.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.081+0000 7f517f4ee640 1 Processor -- start 2024-04-25T10:03:26.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.081+0000 7f517f4ee640 1 -- start start 2024-04-25T10:03:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.082+0000 7f517f4ee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5178154ed0 0x7f51780aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.082+0000 7f517f4ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f517806c1d0 0x7f517806afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.082+0000 7f517f4ee640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517815ec60 con 0x7f5178071ac0 2024-04-25T10:03:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.082+0000 7f517f4ee640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517815ef40 con 0x7f5178154ed0 2024-04-25T10:03:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.082+0000 7f517f4ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517806db90 con 0x7f517806c1d0 2024-04-25T10:03:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.082+0000 7f517ca89640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5178071ac0 0x7f51780ad410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46206/0 (socket says 172.21.15.19:46206) 2024-04-25T10:03:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.082+0000 7f517ca89640 1 -- 172.21.15.19:0/3087341264 learned_addr learned my addr 172.21.15.19:0/3087341264 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.082+0000 7f517d28a640 1 --2- 172.21.15.19:0/3087341264 >> v2:172.21.15.112:3300/0 conn(0x7f5178154ed0 0x7f51780aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.082+0000 7f517da8b640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f517806c1d0 0x7f517806afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.082+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836772580 0 0) 0x7f517815ec60 con 0x7f5178071ac0 2024-04-25T10:03:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.082+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5160003520 con 0x7f5178071ac0 2024-04-25T10:03:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3041046926 0 0) 0x7f5160003520 con 0x7f5178071ac0 2024-04-25T10:03:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f517806c1d0 msgr2=0x7f517806afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f517806c1d0 0x7f517806afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 >> v2:172.21.15.112:3300/0 conn(0x7f5178154ed0 msgr2=0x7f51780aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 --2- 172.21.15.19:0/3087341264 >> v2:172.21.15.112:3300/0 conn(0x7f5178154ed0 0x7f51780aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517806de70 con 0x7f5178071ac0 2024-04-25T10:03:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f516c001670 con 0x7f5178071ac0 2024-04-25T10:03:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 >> v1:172.21.15.19:6789/0 conn(0x7f5178071ac0 legacy=0x7f51780ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f517806c1d0 0x7f51600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5160003f30 0x7f5160004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5160004d40 0x7f51600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517815ef40 con 0x7f5160004d40 2024-04-25T10:03:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517806db90 con 0x7f5160003f30 2024-04-25T10:03:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516000b9c0 con 0x7f517806c1d0 2024-04-25T10:03:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f517da8b640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5160004d40 0x7f51600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f517d28a640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f517806c1d0 0x7f51600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f517ca89640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5160003f30 0x7f5160004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f517da8b640 1 -- 172.21.15.19:0/3087341264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f517806c1d0 msgr2=0x7f51600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f517da8b640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f517806c1d0 0x7f51600039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f517da8b640 1 -- 172.21.15.19:0/3087341264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5160003f30 msgr2=0x7f5160004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f517da8b640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5160003f30 0x7f5160004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.083+0000 7f517da8b640 1 -- 172.21.15.19:0/3087341264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160003520 con 0x7f5160004d40 2024-04-25T10:03:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.084+0000 7f517da8b640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5160004d40 0x7f51600051c0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f5170009ac0 tx=0x7f5160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb76684ba7b1933 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.084+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170036070 con 0x7f5160004d40 2024-04-25T10:03:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.084+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5170031400 con 0x7f5160004d40 2024-04-25T10:03:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.084+0000 7f5177fff640 1 -- 172.21.15.19:0/3087341264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170009690 con 0x7f5160004d40 2024-04-25T10:03:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.084+0000 7f517f4ee640 1 -- 172.21.15.19:0/3087341264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5160004d40 msgr2=0x7f51600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.085+0000 7f517f4ee640 1 --2- 172.21.15.19:0/3087341264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5160004d40 0x7f51600051c0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f5170009ac0 tx=0x7f5160005830 comp rx=0 tx=0).stop 2024-04-25T10:03:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.085+0000 7f517da8b640 1 -- 172.21.15.19:0/3087341264 reap_dead start 2024-04-25T10:03:26.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.085+0000 7f517f4ee640 1 -- 172.21.15.19:0/3087341264 shutdown_connections 2024-04-25T10:03:26.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.085+0000 7f517f4ee640 1 -- 172.21.15.19:0/3087341264 >> 172.21.15.19:0/3087341264 conn(0x7f51780af670 msgr2=0x7f51780b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.085+0000 7f517f4ee640 1 -- 172.21.15.19:0/3087341264 shutdown_connections 2024-04-25T10:03:26.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.085+0000 7f517f4ee640 1 -- 172.21.15.19:0/3087341264 wait complete. 2024-04-25T10:03:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.086+0000 7f517f4ee640 1 Processor -- start 2024-04-25T10:03:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.086+0000 7f517f4ee640 1 -- start start 2024-04-25T10:03:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517f4ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51781560f0 0x7f51780cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517f4ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51781564d0 0x7f51781552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517f4ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f517814e520 0x7f517814e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517f4ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517806b120 con 0x7f517814e520 2024-04-25T10:03:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517f4ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517806afa0 con 0x7f51781560f0 2024-04-25T10:03:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517f4ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517806b2a0 con 0x7f51781564d0 2024-04-25T10:03:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517da8b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f517814e520 0x7f517814e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517da8b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f517814e520 0x7f517814e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49820/0 (socket says 172.21.15.19:49820) 2024-04-25T10:03:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517da8b640 1 -- 172.21.15.19:0/523825323 learned_addr learned my addr 172.21.15.19:0/523825323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517d28a640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51781560f0 0x7f51780cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517ca89640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51781564d0 0x7f51781552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517da8b640 1 -- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51781564d0 msgr2=0x7f51781552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517da8b640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51781564d0 0x7f51781552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517da8b640 1 -- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51781560f0 msgr2=0x7f51780cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517da8b640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51781560f0 0x7f51780cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517da8b640 1 -- 172.21.15.19:0/523825323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517814fcf0 con 0x7f517814e520 2024-04-25T10:03:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517da8b640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f517814e520 0x7f517814e9d0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f5170034040 tx=0x7f5170004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f5175ffb640 1 -- 172.21.15.19:0/523825323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170036040 con 0x7f517814e520 2024-04-25T10:03:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f5175ffb640 1 -- 172.21.15.19:0/523825323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5170041070 con 0x7f517814e520 2024-04-25T10:03:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f517f4ee640 1 -- 172.21.15.19:0/523825323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51780ce090 con 0x7f517814e520 2024-04-25T10:03:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.087+0000 7f5175ffb640 1 -- 172.21.15.19:0/523825323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170003c80 con 0x7f517814e520 2024-04-25T10:03:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.088+0000 7f517f4ee640 1 -- 172.21.15.19:0/523825323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5178150240 con 0x7f517814e520 2024-04-25T10:03:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.088+0000 7f517f4ee640 1 -- 172.21.15.19:0/523825323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51780b3c90 con 0x7f517814e520 2024-04-25T10:03:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.089+0000 7f5175ffb640 1 -- 172.21.15.19:0/523825323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5170039070 con 0x7f517814e520 2024-04-25T10:03:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.089+0000 7f5175ffb640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f516406b4d0 0x7f516406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.089+0000 7f5175ffb640 1 -- 172.21.15.19:0/523825323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f51700a9a80 con 0x7f517814e520 2024-04-25T10:03:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.090+0000 7f517d28a640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f516406b4d0 0x7f516406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.090+0000 7f517d28a640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f516406b4d0 0x7f516406d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5168007520 tx=0x7f516800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.093+0000 7f5175ffb640 1 -- 172.21.15.19:0/523825323 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f517004a020 con 0x7f517814e520 2024-04-25T10:03:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.311+0000 7f517f4ee640 1 -- 172.21.15.19:0/523825323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f51780cde00 con 0x7f517814e520 2024-04-25T10:03:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.312+0000 7f5175ffb640 1 -- 172.21.15.19:0/523825323 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5170003e20 con 0x7f517814e520 2024-04-25T10:03:26.317 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:26.317 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:03:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.313+0000 7f517f4ee640 1 -- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f516406b4d0 msgr2=0x7f516406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.313+0000 7f517f4ee640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f516406b4d0 0x7f516406d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5168007520 tx=0x7f516800b040 comp rx=0 tx=0).stop 2024-04-25T10:03:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.313+0000 7f517f4ee640 1 -- 172.21.15.19:0/523825323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f517814e520 msgr2=0x7f517814e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.313+0000 7f517f4ee640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f517814e520 0x7f517814e9d0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f5170034040 tx=0x7f5170004730 comp rx=0 tx=0).stop 2024-04-25T10:03:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.313+0000 7f517f4ee640 1 -- 172.21.15.19:0/523825323 shutdown_connections 2024-04-25T10:03:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.314+0000 7f517f4ee640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f516406b4d0 0x7f516406d990 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.314+0000 7f517f4ee640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f517814e520 0x7f517814e9d0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.314+0000 7f517f4ee640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51781564d0 0x7f51781552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.314+0000 7f517f4ee640 1 --2- 172.21.15.19:0/523825323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51781560f0 0x7f51780cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.314+0000 7f517f4ee640 1 -- 172.21.15.19:0/523825323 >> 172.21.15.19:0/523825323 conn(0x7f5160004d40 msgr2=0x7f51780aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.314+0000 7f517f4ee640 1 -- 172.21.15.19:0/523825323 shutdown_connections 2024-04-25T10:03:26.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.314+0000 7f517f4ee640 1 -- 172.21.15.19:0/523825323 wait complete. 2024-04-25T10:03:26.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:26.330 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub start '~mdsdir' recursive 2024-04-25T10:03:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.430+0000 7f4afab29640 1 Processor -- start 2024-04-25T10:03:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.430+0000 7f4afab29640 1 -- start start 2024-04-25T10:03:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.430+0000 7f4afab29640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4af40cc4f0 0x7f4af40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.430+0000 7f4afab29640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 0x7f4af40cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.430+0000 7f4afab29640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af4072280 con 0x7f4af415b270 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.430+0000 7f4afab29640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af40ce4c0 con 0x7f4af40cc4f0 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.430+0000 7f4afab29640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af4071270 con 0x7f4af40cdb20 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.431+0000 7f4af88c5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4af40cc4f0 0x7f4af40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.431+0000 7f4af90c6640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4af415b270 0x7f4af415b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46222/0 (socket says 172.21.15.19:46222) 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.431+0000 7f4af88c5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4af40cc4f0 0x7f4af40cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49830/0 (socket says 172.21.15.19:49830) 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.431+0000 7f4af90c6640 1 -- 172.21.15.19:0/692348009 learned_addr learned my addr 172.21.15.19:0/692348009 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.431+0000 7f4aebfff640 1 --2- 172.21.15.19:0/692348009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 0x7f4af40cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.431+0000 7f4af88c5640 1 -- 172.21.15.19:0/692348009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 msgr2=0x7f4af40cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.431+0000 7f4af88c5640 1 --2- 172.21.15.19:0/692348009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 0x7f4af40cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.431+0000 7f4af88c5640 1 -- 172.21.15.19:0/692348009 >> v1:172.21.15.19:6789/0 conn(0x7f4af415b270 legacy=0x7f4af415b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.431+0000 7f4af88c5640 1 -- 172.21.15.19:0/692348009 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af40714c0 con 0x7f4af40cc4f0 2024-04-25T10:03:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.431+0000 7f4aeb7fe640 1 -- 172.21.15.19:0/692348009 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366132852 0 0) 0x7f4af4072280 con 0x7f4af415b270 2024-04-25T10:03:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.431+0000 7f4aebfff640 1 --2- 172.21.15.19:0/692348009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 0x7f4af40cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4af88c5640 1 --2- 172.21.15.19:0/692348009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af40cc4f0 0x7f4af40cc8d0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f4af00097a0 tx=0x7f4af002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279e1bda5bf546f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4aeb7fe640 1 -- 172.21.15.19:0/692348009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0037070 con 0x7f4af40cc4f0 2024-04-25T10:03:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4aeb7fe640 1 -- 172.21.15.19:0/692348009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af0032440 con 0x7f4af40cc4f0 2024-04-25T10:03:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4aeb7fe640 1 -- 172.21.15.19:0/692348009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0032810 con 0x7f4af40cc4f0 2024-04-25T10:03:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4afab29640 1 -- 172.21.15.19:0/692348009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af40cc4f0 msgr2=0x7f4af40cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4afab29640 1 --2- 172.21.15.19:0/692348009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af40cc4f0 0x7f4af40cc8d0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f4af00097a0 tx=0x7f4af002a180 comp rx=0 tx=0).stop 2024-04-25T10:03:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4afab29640 1 -- 172.21.15.19:0/692348009 shutdown_connections 2024-04-25T10:03:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4afab29640 1 --2- 172.21.15.19:0/692348009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 0x7f4af40cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4afab29640 1 --2- 172.21.15.19:0/692348009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af40cc4f0 0x7f4af40cc8d0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4afab29640 1 -- 172.21.15.19:0/692348009 >> 172.21.15.19:0/692348009 conn(0x7f4af406abe0 msgr2=0x7f4af406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4afab29640 1 -- 172.21.15.19:0/692348009 shutdown_connections 2024-04-25T10:03:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.432+0000 7f4afab29640 1 -- 172.21.15.19:0/692348009 wait complete. 2024-04-25T10:03:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.434+0000 7f4afab29640 1 Processor -- start 2024-04-25T10:03:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.434+0000 7f4afab29640 1 -- start start 2024-04-25T10:03:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.434+0000 7f4afab29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af40cc4f0 0x7f4af4149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.434+0000 7f4afab29640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 0x7f4af4155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.434+0000 7f4afab29640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af415b270 0x7f4af4157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.434+0000 7f4afab29640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af4071a80 con 0x7f4af415b270 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.434+0000 7f4afab29640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af415bd90 con 0x7f4af40cc4f0 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.434+0000 7f4afab29640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af4071d60 con 0x7f4af40cdb20 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4af90c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af415b270 0x7f4af4157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4af90c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af415b270 0x7f4af4157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49844/0 (socket says 172.21.15.19:49844) 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4af90c6640 1 -- 172.21.15.19:0/4080914983 learned_addr learned my addr 172.21.15.19:0/4080914983 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4af88c5640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af40cc4f0 0x7f4af4149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4aebfff640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 0x7f4af4155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4af90c6640 1 -- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 msgr2=0x7f4af4155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4af90c6640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 0x7f4af4155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4af90c6640 1 -- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af40cc4f0 msgr2=0x7f4af4149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4af90c6640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af40cc4f0 0x7f4af4149b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4af90c6640 1 -- 172.21.15.19:0/4080914983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af4155c90 con 0x7f4af415b270 2024-04-25T10:03:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4aebfff640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 0x7f4af4155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4af90c6640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af415b270 0x7f4af4157090 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00c7a0 tx=0x7f4aec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4ae97fa640 1 -- 172.21.15.19:0/4080914983 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec018070 con 0x7f4af415b270 2024-04-25T10:03:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4ae97fa640 1 -- 172.21.15.19:0/4080914983 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aec00a640 con 0x7f4af415b270 2024-04-25T10:03:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4afab29640 1 -- 172.21.15.19:0/4080914983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af4155f70 con 0x7f4af415b270 2024-04-25T10:03:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4ae97fa640 1 -- 172.21.15.19:0/4080914983 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec00aa70 con 0x7f4af415b270 2024-04-25T10:03:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.435+0000 7f4afab29640 1 -- 172.21.15.19:0/4080914983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af4156530 con 0x7f4af415b270 2024-04-25T10:03:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.437+0000 7f4ae97fa640 1 -- 172.21.15.19:0/4080914983 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4aec01b070 con 0x7f4af415b270 2024-04-25T10:03:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.437+0000 7f4ae97fa640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4ae006b4d0 0x7f4ae006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.437+0000 7f4ae97fa640 1 -- 172.21.15.19:0/4080914983 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4aec089930 con 0x7f4af415b270 2024-04-25T10:03:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.438+0000 7f4af88c5640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4ae006b4d0 0x7f4ae006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.438+0000 7f4af88c5640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4ae006b4d0 0x7f4ae006d990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4af0034000 tx=0x7f4af002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.438+0000 7f4afd82b740 1 Processor -- start 2024-04-25T10:03:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.438+0000 7f4afd82b740 1 -- start start 2024-04-25T10:03:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1088080 0x55e5e1088460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088aa0 0x55e5e1058200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1058740 0x55e5e1080500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4afd82b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e0fc10a0 con 0x55e5e1088aa0 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4afd82b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e0f69d40 con 0x55e5e1088080 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4afd82b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e0fbd140 con 0x55e5e1058740 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088aa0 0x55e5e1058200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088aa0 0x55e5e1058200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49858/0 (socket says 172.21.15.19:49858) 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4aebfff640 1 -- 172.21.15.19:0/1721878541 learned_addr learned my addr 172.21.15.19:0/1721878541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4af88c5640 1 --2- 172.21.15.19:0/1721878541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1088080 0x55e5e1088460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4af90c6640 1 --2- 172.21.15.19:0/1721878541 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1058740 0x55e5e1080500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4aebfff640 1 -- 172.21.15.19:0/1721878541 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1058740 msgr2=0x55e5e1080500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4aebfff640 1 --2- 172.21.15.19:0/1721878541 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1058740 0x55e5e1080500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4aebfff640 1 -- 172.21.15.19:0/1721878541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1088080 msgr2=0x55e5e1088460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4aebfff640 1 --2- 172.21.15.19:0/1721878541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1088080 0x55e5e1088460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.439+0000 7f4aebfff640 1 -- 172.21.15.19:0/1721878541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e0da1a20 con 0x55e5e1088aa0 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4aebfff640 1 --2- 172.21.15.19:0/1721878541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088aa0 0x55e5e1058200 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400a8b0 tx=0x7f4ae400ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300400bf6d8aea68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4ada7fc640 1 -- 172.21.15.19:0/1721878541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4018020 con 0x55e5e1088aa0 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4ada7fc640 1 -- 172.21.15.19:0/1721878541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae400f9c0 con 0x55e5e1088aa0 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4ada7fc640 1 -- 172.21.15.19:0/1721878541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400fd90 con 0x55e5e1088aa0 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4afd82b740 1 -- 172.21.15.19:0/1721878541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088aa0 msgr2=0x55e5e1058200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1721878541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088aa0 0x55e5e1058200 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400a8b0 tx=0x7f4ae400ad70 comp rx=0 tx=0).stop 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4afd82b740 1 -- 172.21.15.19:0/1721878541 shutdown_connections 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1721878541 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1058740 0x55e5e1080500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1721878541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088aa0 0x55e5e1058200 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1721878541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1088080 0x55e5e1088460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4afd82b740 1 -- 172.21.15.19:0/1721878541 >> 172.21.15.19:0/1721878541 conn(0x55e5e105a470 msgr2=0x55e5e10a2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4afd82b740 1 -- 172.21.15.19:0/1721878541 shutdown_connections 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.440+0000 7f4afd82b740 1 -- 172.21.15.19:0/1721878541 wait complete. 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.441+0000 7f4afd82b740 1 -- start start 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.441+0000 7f4afd82b740 1 Processor -- start 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e1086c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1088aa0 0x55e5e10838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 0x55e5e1081380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4afd82b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e10847d0 con 0x55e5e1088080 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4afd82b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e0f5f3d0 con 0x55e5e1083e10 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4afd82b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e1084a70 con 0x55e5e1088aa0 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4afd82b740 10 client.0 _ll_register_callbacks cb 0x55e5e0cd4e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4af88c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e1086c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4af88c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e1086c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49872/0 (socket says 172.21.15.19:49872) 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4af88c5640 1 -- 172.21.15.19:0/947813790 learned_addr learned my addr 172.21.15.19:0/947813790 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4aebfff640 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1088aa0 0x55e5e10838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4af90c6640 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 0x55e5e1081380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4af88c5640 1 -- 172.21.15.19:0/947813790 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1088aa0 msgr2=0x55e5e10838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4af88c5640 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1088aa0 0x55e5e10838d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4af88c5640 1 -- 172.21.15.19:0/947813790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 msgr2=0x55e5e1081380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4af88c5640 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 0x55e5e1081380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4af88c5640 1 -- 172.21.15.19:0/947813790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5e0c95310 con 0x55e5e1088080 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4af90c6640 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 0x55e5e1081380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.442+0000 7f4aebfff640 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1088aa0 0x55e5e10838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4af88c5640 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e1086c30 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f4af00097a0 tx=0x7f4af0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9310d9f022a0e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4ad37fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4ad37fe640 1 -- 172.21.15.19:0/947813790 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af004f020 con 0x55e5e1088080 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4ad37fe640 1 -- 172.21.15.19:0/947813790 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4af003c9d0 con 0x55e5e1088080 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4afd82b740 1 -- 172.21.15.19:0/947813790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ae400f9c0 con 0x55e5e1088080 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4ad37fe640 1 -- 172.21.15.19:0/947813790 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af0059050 con 0x55e5e1088080 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4afd82b740 10 client.10443 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4afd82b740 1 -- 172.21.15.19:0/947813790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55e5e0e021b0 con 0x55e5e1088080 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4ad37fe640 1 -- 172.21.15.19:0/947813790 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae400f9c0 con 0x55e5e1088080 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4ad37fe640 1 -- 172.21.15.19:0/947813790 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4af005c020 con 0x55e5e1088080 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4afd82b740 10 client.10443 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4afd82b740 10 client.10443 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4afd82b740 20 client.10443 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4afd82b740 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e5e10aa590 0x55e5e10aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4afd82b740 4 client.10443 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4afd82b740 1 -- 172.21.15.19:0/947813790 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5e0fce5e0 con 0x55e5e10aa590 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.443+0000 7f4aebfff640 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e5e10aa590 0x55e5e10aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4aebfff640 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e5e10aa590 0x55e5e10aa9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c9604c907d3f19 server_cookie=5ffa118b3544dddf in_seq=0 out_seq=0 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4ad37fe640 10 client.10443 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4ad37fe640 1 -- 172.21.15.19:0/947813790 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5e0fce5e0 con 0x55e5e10aa590 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4ad37fe640 10 client.10443 handle_command_reply: tid=0 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4afd82b740 1 client.10443 shutdown 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4afd82b740 1 -- 172.21.15.19:0/947813790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 msgr2=0x55e5e1086c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4afd82b740 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e1086c30 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f4af00097a0 tx=0x7f4af0004730 comp rx=0 tx=0).stop 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4afd82b740 1 -- 172.21.15.19:0/947813790 shutdown_connections 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4afd82b740 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e5e10aa590 0x55e5e10aa9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4afd82b740 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 0x55e5e1081380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4afd82b740 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1088aa0 0x55e5e10838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4ad37fe640 0 client.10443 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4afd82b740 1 --2- 172.21.15.19:0/947813790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e1086c30 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.444+0000 7f4afd82b740 1 -- 172.21.15.19:0/947813790 >> 172.21.15.19:0/947813790 conn(0x55e5e1058740 msgr2=0x55e5e10578d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.445+0000 7f4afd82b740 1 -- 172.21.15.19:0/947813790 shutdown_connections 2024-04-25T10:03:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.445+0000 7f4afd82b740 1 -- 172.21.15.19:0/947813790 wait complete. 2024-04-25T10:03:26.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.445+0000 7f4afd82b740 20 client.10443 trim_cache size 0 max 16384 2024-04-25T10:03:26.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.460+0000 7f4afd82b740 1 Processor -- start 2024-04-25T10:03:26.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.460+0000 7f4afd82b740 1 -- start start 2024-04-25T10:03:26.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1083e10 0x55e5e105e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10aab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1088aa0 0x55e5e108b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4afd82b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af4200280 con 0x55e5e1088080 2024-04-25T10:03:26.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4afd82b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af41ff400 con 0x55e5e1088aa0 2024-04-25T10:03:26.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4afd82b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af41542b0 con 0x55e5e1083e10 2024-04-25T10:03:26.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10aab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49876/0 (socket says 172.21.15.19:49876) 2024-04-25T10:03:26.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4aebfff640 1 -- 172.21.15.19:0/1899681876 learned_addr learned my addr 172.21.15.19:0/1899681876 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4aebfff640 1 -- 172.21.15.19:0/1899681876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1083e10 msgr2=0x55e5e105e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4aebfff640 1 --2- 172.21.15.19:0/1899681876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1083e10 0x55e5e105e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4aebfff640 1 -- 172.21.15.19:0/1899681876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1088aa0 msgr2=0x55e5e108b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4aebfff640 1 --2- 172.21.15.19:0/1899681876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1088aa0 0x55e5e108b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4aebfff640 1 -- 172.21.15.19:0/1899681876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e0c6e490 con 0x55e5e1088080 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4aebfff640 1 --2- 172.21.15.19:0/1899681876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10aab60 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400adb0 tx=0x7f4ae4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e47c07ef8410f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4ad9ffb640 1 -- 172.21.15.19:0/1899681876 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae402c070 con 0x55e5e1088080 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4ad9ffb640 1 -- 172.21.15.19:0/1899681876 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae400f8e0 con 0x55e5e1088080 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.461+0000 7f4ad9ffb640 1 -- 172.21.15.19:0/1899681876 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4018040 con 0x55e5e1088080 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.462+0000 7f4afd82b740 1 -- 172.21.15.19:0/1899681876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 msgr2=0x55e5e10aab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.462+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1899681876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10aab60 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400adb0 tx=0x7f4ae4014000 comp rx=0 tx=0).stop 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.462+0000 7f4afd82b740 1 -- 172.21.15.19:0/1899681876 shutdown_connections 2024-04-25T10:03:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.462+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1899681876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1088aa0 0x55e5e108b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.462+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1899681876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10aab60 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.462+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1899681876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1083e10 0x55e5e105e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.462+0000 7f4afd82b740 1 -- 172.21.15.19:0/1899681876 >> 172.21.15.19:0/1899681876 conn(0x55e5e1058740 msgr2=0x55e5e10bdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.462+0000 7f4afd82b740 1 -- 172.21.15.19:0/1899681876 shutdown_connections 2024-04-25T10:03:26.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.462+0000 7f4afd82b740 1 -- 172.21.15.19:0/1899681876 wait complete. 2024-04-25T10:03:26.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.462+0000 7f4afd82b740 1 -- start start 2024-04-25T10:03:26.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.463+0000 7f4afd82b740 1 Processor -- start 2024-04-25T10:03:26.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.463+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 0x55e5e1111840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.463+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10a9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.463+0000 7f4afd82b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1088aa0 0x55e5e10a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.463+0000 7f4afd82b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af41feec0 con 0x55e5e1088080 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.463+0000 7f4afd82b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e107e670 con 0x55e5e1083e10 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.463+0000 7f4afd82b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e108bbd0 con 0x55e5e1088aa0 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.463+0000 7f4afd82b740 10 client.0 _ll_register_callbacks cb 0x7f4ae400a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.463+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.463+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10a9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49886/0 (socket says 172.21.15.19:49886) 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.463+0000 7f4aebfff640 1 -- 172.21.15.19:0/1538090479 learned_addr learned my addr 172.21.15.19:0/1538090479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4af88c5640 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 0x55e5e1111840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4af90c6640 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1088aa0 0x55e5e10a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4aebfff640 1 -- 172.21.15.19:0/1538090479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1088aa0 msgr2=0x55e5e10a9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4aebfff640 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1088aa0 0x55e5e10a9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4aebfff640 1 -- 172.21.15.19:0/1538090479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 msgr2=0x55e5e1111840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4aebfff640 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 0x55e5e1111840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4aebfff640 1 -- 172.21.15.19:0/1538090479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5e10c6d10 con 0x55e5e1088080 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4af88c5640 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 0x55e5e1111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4aebfff640 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10a9810 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400a8b0 tx=0x7f4ae4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db4b2f62dd04f4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4ad37fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4ad37fe640 1 -- 172.21.15.19:0/1538090479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae402c030 con 0x55e5e1088080 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4ad37fe640 1 -- 172.21.15.19:0/1538090479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ae4026040 con 0x55e5e1088080 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4afd82b740 1 -- 172.21.15.19:0/1538090479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ae400f8e0 con 0x55e5e1088080 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4ad37fe640 1 -- 172.21.15.19:0/1538090479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ae4018040 con 0x55e5e1088080 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4afd82b740 10 client.10449 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:26.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4afd82b740 1 -- 172.21.15.19:0/1538090479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55e5e1070770 con 0x55e5e1088080 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4ad37fe640 1 -- 172.21.15.19:0/1538090479 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae400f8e0 con 0x55e5e1088080 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4ad37fe640 1 -- 172.21.15.19:0/1538090479 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4ae4005990 con 0x55e5e1088080 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4afd82b740 10 client.10449 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4afd82b740 10 client.10449 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.464+0000 7f4afd82b740 20 client.10449 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.465+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e5e105e3d0 0x55e5e108aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.465+0000 7f4afd82b740 4 client.10449 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.465+0000 7f4afd82b740 1 -- 172.21.15.19:0/1538090479 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x55e5e0fc8930 con 0x55e5e105e3d0 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.465+0000 7f4af88c5640 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e5e105e3d0 0x55e5e108aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.465+0000 7f4ad37fe640 10 client.10449 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.465+0000 7f4af88c5640 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e5e105e3d0 0x55e5e108aac0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c089ffadc5bd06a3 server_cookie=436fb939aa1542e4 in_seq=0 out_seq=0 2024-04-25T10:03:26.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.465+0000 7f4ad37fe640 1 -- 172.21.15.19:0/1538090479 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e5e0fc8930 con 0x55e5e105e3d0 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.465+0000 7f4ad37fe640 10 client.10449 handle_command_reply: tid=0 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.465+0000 7f4afd82b740 1 client.10449 shutdown 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afd82b740 1 -- 172.21.15.19:0/1538090479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 msgr2=0x55e5e10a9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10a9810 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400a8b0 tx=0x7f4ae4004730 comp rx=0 tx=0).stop 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afd82b740 1 -- 172.21.15.19:0/1538090479 shutdown_connections 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x55e5e105e3d0 0x55e5e108aac0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55e5e1088aa0 0x55e5e10a9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5e1088080 0x55e5e10a9810 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4ad37fe640 0 client.10449 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afd82b740 1 --2- 172.21.15.19:0/1538090479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e1083e10 0x55e5e1111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afd82b740 1 -- 172.21.15.19:0/1538090479 >> 172.21.15.19:0/1538090479 conn(0x55e5e1058740 msgr2=0x55e5e10aab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afd82b740 1 -- 172.21.15.19:0/1538090479 shutdown_connections 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afd82b740 1 -- 172.21.15.19:0/1538090479 wait complete. 2024-04-25T10:03:26.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afd82b740 20 client.10449 trim_cache size 0 max 16384 2024-04-25T10:03:26.478 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:03:26.478 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:03:26.478 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "2d73e2e0-105e-4dab-91d2-4910256a8bda", 2024-04-25T10:03:26.478 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:03:26.478 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:03:26.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afab29640 1 -- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4ae006b4d0 msgr2=0x7f4ae006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afab29640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4ae006b4d0 0x7f4ae006d990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4af0034000 tx=0x7f4af002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:26.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afab29640 1 -- 172.21.15.19:0/4080914983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af415b270 msgr2=0x7f4af4157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afab29640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af415b270 0x7f4af4157090 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00c7a0 tx=0x7f4aec00cc60 comp rx=0 tx=0).stop 2024-04-25T10:03:26.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afab29640 1 -- 172.21.15.19:0/4080914983 shutdown_connections 2024-04-25T10:03:26.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afab29640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4ae006b4d0 0x7f4ae006d990 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afab29640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af415b270 0x7f4af4157090 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.466+0000 7f4afab29640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4af40cdb20 0x7f4af4155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.467+0000 7f4afab29640 1 --2- 172.21.15.19:0/4080914983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af40cc4f0 0x7f4af4149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.467+0000 7f4afab29640 1 -- 172.21.15.19:0/4080914983 >> 172.21.15.19:0/4080914983 conn(0x7f4af406abe0 msgr2=0x7f4af406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.467+0000 7f4afab29640 1 -- 172.21.15.19:0/4080914983 shutdown_connections 2024-04-25T10:03:26.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.467+0000 7f4afab29640 1 -- 172.21.15.19:0/4080914983 wait complete. 2024-04-25T10:03:26.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:26.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:03:26.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.580+0000 7f79527ce640 1 Processor -- start 2024-04-25T10:03:26.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.580+0000 7f79527ce640 1 -- start start 2024-04-25T10:03:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.581+0000 7f79527ce640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f794c15dc20 0x7f794c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.581+0000 7f79527ce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f794c161710 0x7f794c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.581+0000 7f79527ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f794c0b9330 0x7f794c06bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.581+0000 7f79527ce640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794c15e000 con 0x7f794c15dc20 2024-04-25T10:03:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.581+0000 7f79527ce640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794c0d1de0 con 0x7f794c161710 2024-04-25T10:03:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.581+0000 7f79527ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794c161c60 con 0x7f794c0b9330 2024-04-25T10:03:26.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.581+0000 7f794bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f794c15dc20 0x7f794c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.581+0000 7f794bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f794c15dc20 0x7f794c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49902/0 (socket says 172.21.15.19:49902) 2024-04-25T10:03:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.581+0000 7f794bfff640 1 -- 172.21.15.19:0/805095141 learned_addr learned my addr 172.21.15.19:0/805095141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.581+0000 7f7950d6b640 1 --2- 172.21.15.19:0/805095141 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f794c0b9330 0x7f794c06bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.581+0000 7f794b7fe640 1 --2- 172.21.15.19:0/805095141 >> v2:172.21.15.112:3300/0 conn(0x7f794c161710 0x7f794c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.582+0000 7f794bfff640 1 -- 172.21.15.19:0/805095141 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f794c0b9330 msgr2=0x7f794c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.582+0000 7f794bfff640 1 --2- 172.21.15.19:0/805095141 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f794c0b9330 0x7f794c06bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.582+0000 7f794bfff640 1 -- 172.21.15.19:0/805095141 >> v2:172.21.15.112:3300/0 conn(0x7f794c161710 msgr2=0x7f794c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.582+0000 7f794bfff640 1 --2- 172.21.15.19:0/805095141 >> v2:172.21.15.112:3300/0 conn(0x7f794c161710 0x7f794c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.582+0000 7f794bfff640 1 -- 172.21.15.19:0/805095141 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c161f40 con 0x7f794c15dc20 2024-04-25T10:03:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.582+0000 7f794bfff640 1 --2- 172.21.15.19:0/805095141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c15dc20 0x7f794c0ce670 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f794c06da90 tx=0x7f794002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f5272533e746af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.582+0000 7f794affd640 1 -- 172.21.15.19:0/805095141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940037070 con 0x7f794c15dc20 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.582+0000 7f794affd640 1 -- 172.21.15.19:0/805095141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7940032440 con 0x7f794c15dc20 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.583+0000 7f794affd640 1 -- 172.21.15.19:0/805095141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940032810 con 0x7f794c15dc20 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.583+0000 7f79527ce640 1 -- 172.21.15.19:0/805095141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c15dc20 msgr2=0x7f794c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.583+0000 7f79527ce640 1 --2- 172.21.15.19:0/805095141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c15dc20 0x7f794c0ce670 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f794c06da90 tx=0x7f794002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.583+0000 7f79527ce640 1 -- 172.21.15.19:0/805095141 shutdown_connections 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.583+0000 7f79527ce640 1 --2- 172.21.15.19:0/805095141 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f794c0b9330 0x7f794c06bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.583+0000 7f79527ce640 1 --2- 172.21.15.19:0/805095141 >> v2:172.21.15.112:3300/0 conn(0x7f794c161710 0x7f794c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.583+0000 7f79527ce640 1 --2- 172.21.15.19:0/805095141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c15dc20 0x7f794c0ce670 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.583+0000 7f79527ce640 1 -- 172.21.15.19:0/805095141 >> 172.21.15.19:0/805095141 conn(0x7f794c0d0420 msgr2=0x7f794c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.583+0000 7f79527ce640 1 -- 172.21.15.19:0/805095141 shutdown_connections 2024-04-25T10:03:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.583+0000 7f79527ce640 1 -- 172.21.15.19:0/805095141 wait complete. 2024-04-25T10:03:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.584+0000 7f79527ce640 1 Processor -- start 2024-04-25T10:03:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.584+0000 7f79527ce640 1 -- start start 2024-04-25T10:03:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.585+0000 7f79527ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f794c0b9330 0x7f794c0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.585+0000 7f79527ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f794c15dc20 0x7f794c1f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.585+0000 7f79527ce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c161710 0x7f794c1f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.585+0000 7f79527ce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c0b5e50 con 0x7f794c161710 2024-04-25T10:03:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.585+0000 7f79527ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c162500 con 0x7f794c0b9330 2024-04-25T10:03:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.585+0000 7f79527ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794c0b5fd0 con 0x7f794c15dc20 2024-04-25T10:03:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.585+0000 7f7950d6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c161710 0x7f794c1f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.585+0000 7f7950d6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c161710 0x7f794c1f3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49906/0 (socket says 172.21.15.19:49906) 2024-04-25T10:03:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.585+0000 7f7950d6b640 1 -- 172.21.15.19:0/1936493511 learned_addr learned my addr 172.21.15.19:0/1936493511 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.585+0000 7f794bfff640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f794c0b9330 0x7f794c0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.585+0000 7f794b7fe640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f794c15dc20 0x7f794c1f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.586+0000 7f7950d6b640 1 -- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f794c15dc20 msgr2=0x7f794c1f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.586+0000 7f7950d6b640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f794c15dc20 0x7f794c1f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.586+0000 7f7950d6b640 1 -- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f794c0b9330 msgr2=0x7f794c0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.586+0000 7f7950d6b640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f794c0b9330 0x7f794c0cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.586+0000 7f7950d6b640 1 -- 172.21.15.19:0/1936493511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c160b90 con 0x7f794c161710 2024-04-25T10:03:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.586+0000 7f7950d6b640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c161710 0x7f794c1f3dd0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f7944007d10 tx=0x7f794400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.586+0000 7f7948ff9640 1 -- 172.21.15.19:0/1936493511 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944014420 con 0x7f794c161710 2024-04-25T10:03:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.586+0000 7f7948ff9640 1 -- 172.21.15.19:0/1936493511 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7944019070 con 0x7f794c161710 2024-04-25T10:03:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.586+0000 7f79527ce640 1 -- 172.21.15.19:0/1936493511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794c0cd980 con 0x7f794c161710 2024-04-25T10:03:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.586+0000 7f7948ff9640 1 -- 172.21.15.19:0/1936493511 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944014800 con 0x7f794c161710 2024-04-25T10:03:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.586+0000 7f79527ce640 1 -- 172.21.15.19:0/1936493511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c1602f0 con 0x7f794c161710 2024-04-25T10:03:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.588+0000 7f7948ff9640 1 -- 172.21.15.19:0/1936493511 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79440149a0 con 0x7f794c161710 2024-04-25T10:03:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.588+0000 7f7948ff9640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f793806b4d0 0x7f793806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.588+0000 7f7948ff9640 1 -- 172.21.15.19:0/1936493511 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f794408aa10 con 0x7f794c161710 2024-04-25T10:03:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.588+0000 7f794bfff640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f793806b4d0 0x7f793806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.589+0000 7f794bfff640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f793806b4d0 0x7f793806d990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7940034000 tx=0x7f794002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.589+0000 7f79555ac740 1 Processor -- start 2024-04-25T10:03:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.589+0000 7f79555ac740 1 -- start start 2024-04-25T10:03:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.589+0000 7f79555ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22b10 0x5577b7f25320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.589+0000 7f79555ac740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f22ef0 0x5577b7f25860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.589+0000 7f79555ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7ef5770 0x5577b7ef8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.589+0000 7f79555ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7e5eda0 con 0x5577b7ef5770 2024-04-25T10:03:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.589+0000 7f79555ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7e07b80 con 0x5577b7f22b10 2024-04-25T10:03:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f79555ac740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577b7e5ae40 con 0x5577b7f22ef0 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7950d6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7ef5770 0x5577b7ef8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7950d6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7ef5770 0x5577b7ef8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49920/0 (socket says 172.21.15.19:49920) 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7950d6b640 1 -- 172.21.15.19:0/1517011191 learned_addr learned my addr 172.21.15.19:0/1517011191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f794bfff640 1 --2- 172.21.15.19:0/1517011191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22b10 0x5577b7f25320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f794b7fe640 1 --2- 172.21.15.19:0/1517011191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f22ef0 0x5577b7f25860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7950d6b640 1 -- 172.21.15.19:0/1517011191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f22ef0 msgr2=0x5577b7f25860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7950d6b640 1 --2- 172.21.15.19:0/1517011191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f22ef0 0x5577b7f25860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7950d6b640 1 -- 172.21.15.19:0/1517011191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22b10 msgr2=0x5577b7f25320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7950d6b640 1 --2- 172.21.15.19:0/1517011191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22b10 0x5577b7f25320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7950d6b640 1 -- 172.21.15.19:0/1517011191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577b7e11f10 con 0x5577b7ef5770 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7950d6b640 1 --2- 172.21.15.19:0/1517011191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7ef5770 0x5577b7ef8f80 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f7944061270 tx=0x7f7944058af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800de515a040879c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7931ffb640 1 -- 172.21.15.19:0/1517011191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79440617a0 con 0x5577b7ef5770 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7931ffb640 1 -- 172.21.15.19:0/1517011191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7944057130 con 0x5577b7ef5770 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.590+0000 7f7931ffb640 1 -- 172.21.15.19:0/1517011191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944012dc0 con 0x5577b7ef5770 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.591+0000 7f79555ac740 1 -- 172.21.15.19:0/1517011191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7ef5770 msgr2=0x5577b7ef8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.591+0000 7f79555ac740 1 --2- 172.21.15.19:0/1517011191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7ef5770 0x5577b7ef8f80 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f7944061270 tx=0x7f7944058af0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.591+0000 7f79555ac740 1 -- 172.21.15.19:0/1517011191 shutdown_connections 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.591+0000 7f79555ac740 1 --2- 172.21.15.19:0/1517011191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7ef5770 0x5577b7ef8f80 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.591+0000 7f79555ac740 1 --2- 172.21.15.19:0/1517011191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f22ef0 0x5577b7f25860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.591+0000 7f79555ac740 1 --2- 172.21.15.19:0/1517011191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22b10 0x5577b7f25320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.591+0000 7f79555ac740 1 -- 172.21.15.19:0/1517011191 >> 172.21.15.19:0/1517011191 conn(0x5577b7ef79e0 msgr2=0x5577b7f40320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.591+0000 7f79555ac740 1 -- 172.21.15.19:0/1517011191 shutdown_connections 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.591+0000 7f79555ac740 1 -- 172.21.15.19:0/1517011191 wait complete. 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.591+0000 7f79555ac740 1 -- start start 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.592+0000 7f79555ac740 1 Processor -- start 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.592+0000 7f79555ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f1e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.592+0000 7f79555ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7f431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.592+0000 7f79555ac740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f1b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.592+0000 7f79555ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7efab90 con 0x5577b7f22b10 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.592+0000 7f79555ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7dfd4b0 con 0x5577b7f22ef0 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.592+0000 7f79555ac740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577b7efae70 con 0x5577b7f1bc90 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.592+0000 7f79555ac740 10 client.0 _ll_register_callbacks cb 0x5577b7e3d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.592+0000 7f794bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f1e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f794bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f1e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49928/0 (socket says 172.21.15.19:49928) 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f794bfff640 1 -- 172.21.15.19:0/414406088 learned_addr learned my addr 172.21.15.19:0/414406088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f794b7fe640 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7f431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f7950d6b640 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f1b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f794bfff640 1 -- 172.21.15.19:0/414406088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 msgr2=0x5577b7f1b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f794bfff640 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f1b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f794bfff640 1 -- 172.21.15.19:0/414406088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 msgr2=0x5577b7f431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f794bfff640 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7f431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f794bfff640 1 -- 172.21.15.19:0/414406088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577b7c3d7d0 con 0x5577b7f22b10 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f794b7fe640 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7f431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f794bfff640 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f1e380 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f79400090b0 tx=0x7f794002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2a0230d9472676 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f792affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f792affd640 1 -- 172.21.15.19:0/414406088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794004f060 con 0x5577b7f22b10 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f79555ac740 1 -- 172.21.15.19:0/414406088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7944057130 con 0x5577b7f22b10 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f792affd640 1 -- 172.21.15.19:0/414406088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7940032810 con 0x5577b7f22b10 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f792affd640 1 -- 172.21.15.19:0/414406088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7940055050 con 0x5577b7f22b10 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f79555ac740 10 client.10461 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.593+0000 7f79555ac740 1 -- 172.21.15.19:0/414406088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x5577b7b733f0 con 0x5577b7f22b10 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.594+0000 7f792affd640 1 -- 172.21.15.19:0/414406088 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7944057130 con 0x5577b7f22b10 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.594+0000 7f792affd640 1 -- 172.21.15.19:0/414406088 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7940058020 con 0x5577b7f22b10 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.594+0000 7f79555ac740 10 client.10461 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.594+0000 7f79555ac740 10 client.10461 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.594+0000 7f79555ac740 20 client.10461 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.594+0000 7f79555ac740 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x5577b7f248f0 0x5577b7f24cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.594+0000 7f79555ac740 4 client.10461 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.594+0000 7f79555ac740 1 -- 172.21.15.19:0/414406088 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577b7e6c2e0 con 0x5577b7f248f0 2024-04-25T10:03:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.594+0000 7f794b7fe640 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x5577b7f248f0 0x5577b7f24cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.594+0000 7f792affd640 10 client.10461 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.594+0000 7f794b7fe640 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x5577b7f248f0 0x5577b7f24cd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=851ddb38c1d19beb server_cookie=a6a50d22a1712b99 in_seq=0 out_seq=0 2024-04-25T10:03:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f792affd640 1 -- 172.21.15.19:0/414406088 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5577b7e6c2e0 con 0x5577b7f248f0 2024-04-25T10:03:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f792affd640 10 client.10461 handle_command_reply: tid=0 2024-04-25T10:03:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 1 client.10461 shutdown 2024-04-25T10:03:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 1 -- 172.21.15.19:0/414406088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 msgr2=0x5577b7f1e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f1e380 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f79400090b0 tx=0x7f794002c070 comp rx=0 tx=0).stop 2024-04-25T10:03:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 1 -- 172.21.15.19:0/414406088 shutdown_connections 2024-04-25T10:03:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x5577b7f248f0 0x5577b7f24cd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f1b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7f431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f792affd640 0 client.10461 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 1 --2- 172.21.15.19:0/414406088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f1e380 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 1 -- 172.21.15.19:0/414406088 >> 172.21.15.19:0/414406088 conn(0x5577b7ef5770 msgr2=0x5577b7ef6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 1 -- 172.21.15.19:0/414406088 shutdown_connections 2024-04-25T10:03:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 1 -- 172.21.15.19:0/414406088 wait complete. 2024-04-25T10:03:26.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.595+0000 7f79555ac740 20 client.10461 trim_cache size 0 max 16384 2024-04-25T10:03:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.610+0000 7f79555ac740 1 Processor -- start 2024-04-25T10:03:26.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.610+0000 7f79555ac740 1 -- start start 2024-04-25T10:03:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f79555ac740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f52190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f79555ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f79555ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7ef8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f79555ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c200510 con 0x5577b7f22b10 2024-04-25T10:03:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f79555ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c1ff6f0 con 0x5577b7f22ef0 2024-04-25T10:03:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f79555ac740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794c1feaf0 con 0x5577b7f1bc90 2024-04-25T10:03:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f794b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f794b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49942/0 (socket says 172.21.15.19:49942) 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f794b7fe640 1 -- 172.21.15.19:0/1808987390 learned_addr learned my addr 172.21.15.19:0/1808987390 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f7950d6b640 1 --2- 172.21.15.19:0/1808987390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7ef8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f794bfff640 1 --2- 172.21.15.19:0/1808987390 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f52190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f794b7fe640 1 -- 172.21.15.19:0/1808987390 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 msgr2=0x5577b7f52190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f794b7fe640 1 --2- 172.21.15.19:0/1808987390 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f52190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f794b7fe640 1 -- 172.21.15.19:0/1808987390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 msgr2=0x5577b7ef8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f794b7fe640 1 --2- 172.21.15.19:0/1808987390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7ef8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f794b7fe640 1 -- 172.21.15.19:0/1808987390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577b7f25040 con 0x5577b7f22b10 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f794bfff640 1 --2- 172.21.15.19:0/1808987390 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f52190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.611+0000 7f794b7fe640 1 --2- 172.21.15.19:0/1808987390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x5577b7ef8580 tx=0x7f793c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9207bcbc2ec240 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79317fa640 1 -- 172.21.15.19:0/1808987390 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c01e4b0 con 0x5577b7f22b10 2024-04-25T10:03:26.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79317fa640 1 -- 172.21.15.19:0/1808987390 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c01e650 con 0x5577b7f22b10 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79317fa640 1 -- 172.21.15.19:0/1808987390 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c003610 con 0x5577b7f22b10 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79555ac740 1 -- 172.21.15.19:0/1808987390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 msgr2=0x5577b7f526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79555ac740 1 --2- 172.21.15.19:0/1808987390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x5577b7ef8580 tx=0x7f793c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79555ac740 1 -- 172.21.15.19:0/1808987390 shutdown_connections 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79555ac740 1 --2- 172.21.15.19:0/1808987390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7ef8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79555ac740 1 --2- 172.21.15.19:0/1808987390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79555ac740 1 --2- 172.21.15.19:0/1808987390 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f52190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79555ac740 1 -- 172.21.15.19:0/1808987390 >> 172.21.15.19:0/1808987390 conn(0x5577b7ef5770 msgr2=0x5577b7f40c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79555ac740 1 -- 172.21.15.19:0/1808987390 shutdown_connections 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.612+0000 7f79555ac740 1 -- 172.21.15.19:0/1808987390 wait complete. 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.613+0000 7f79555ac740 1 -- start start 2024-04-25T10:03:26.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.613+0000 7f79555ac740 1 Processor -- start 2024-04-25T10:03:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f79555ac740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f52190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f79555ac740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f79555ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7f592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f79555ac740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c1ff0f0 con 0x5577b7f22b10 2024-04-25T10:03:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f79555ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577b7ef1410 con 0x5577b7f22ef0 2024-04-25T10:03:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f79555ac740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577b7f4f680 con 0x5577b7f1bc90 2024-04-25T10:03:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f79555ac740 10 client.0 _ll_register_callbacks cb 0x7f7944009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f794b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f794b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49950/0 (socket says 172.21.15.19:49950) 2024-04-25T10:03:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f794b7fe640 1 -- 172.21.15.19:0/4009898151 learned_addr learned my addr 172.21.15.19:0/4009898151 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f794bfff640 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f52190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f7950d6b640 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7f592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f794b7fe640 1 -- 172.21.15.19:0/4009898151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 msgr2=0x5577b7f52190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f794b7fe640 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f52190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f794b7fe640 1 -- 172.21.15.19:0/4009898151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 msgr2=0x5577b7f592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f794b7fe640 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7f592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f794b7fe640 1 -- 172.21.15.19:0/4009898151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577b7b330f0 con 0x5577b7f22b10 2024-04-25T10:03:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f794b7fe640 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f793c020000 tx=0x7f793c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d203acb4979e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.614+0000 7f792affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f792affd640 1 -- 172.21.15.19:0/4009898151 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c02b070 con 0x5577b7f22b10 2024-04-25T10:03:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f792affd640 1 -- 172.21.15.19:0/4009898151 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f793c003a20 con 0x5577b7f22b10 2024-04-25T10:03:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f79555ac740 1 -- 172.21.15.19:0/4009898151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f793c01e650 con 0x5577b7f22b10 2024-04-25T10:03:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f792affd640 1 -- 172.21.15.19:0/4009898151 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f793c038050 con 0x5577b7f22b10 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f79555ac740 10 client.10467 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f79555ac740 1 -- 172.21.15.19:0/4009898151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x5577b7f20ca0 con 0x5577b7f22b10 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f792affd640 1 -- 172.21.15.19:0/4009898151 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c01e650 con 0x5577b7f22b10 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f792affd640 1 -- 172.21.15.19:0/4009898151 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f793c03b020 con 0x5577b7f22b10 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f79555ac740 10 client.10467 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f79555ac740 10 client.10467 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f79555ac740 20 client.10467 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f79555ac740 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x5577b7faa320 0x5577b7f59820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f79555ac740 4 client.10467 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f79555ac740 1 -- 172.21.15.19:0/4009898151 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577b7e66630 con 0x5577b7faa320 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f794bfff640 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x5577b7faa320 0x5577b7f59820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f792affd640 10 client.10467 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.615+0000 7f794bfff640 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x5577b7faa320 0x5577b7f59820 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f4a33d14400ed0 server_cookie=3b2e6e7fddf83101 in_seq=0 out_seq=0 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f792affd640 1 -- 172.21.15.19:0/4009898151 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+257 (crc 0 0 0) 0x5577b7e66630 con 0x5577b7faa320 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f792affd640 10 client.10467 handle_command_reply: tid=0 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 1 client.10467 shutdown 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 1 -- 172.21.15.19:0/4009898151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 msgr2=0x5577b7f526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f793c020000 tx=0x7f793c004730 comp rx=0 tx=0).stop 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 1 -- 172.21.15.19:0/4009898151 shutdown_connections 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x5577b7faa320 0x5577b7f59820 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577b7f22ef0 0x5577b7f592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b7f22b10 0x5577b7f526d0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f792affd640 0 client.10467 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 1 --2- 172.21.15.19:0/4009898151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5577b7f1bc90 0x5577b7f52190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 1 -- 172.21.15.19:0/4009898151 >> 172.21.15.19:0/4009898151 conn(0x5577b7ef5770 msgr2=0x5577b7f191d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 1 -- 172.21.15.19:0/4009898151 shutdown_connections 2024-04-25T10:03:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 1 -- 172.21.15.19:0/4009898151 wait complete. 2024-04-25T10:03:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79555ac740 20 client.10467 trim_cache size 0 max 16384 2024-04-25T10:03:26.629 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:03:26.629 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:03:26.629 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:03:26.629 INFO:teuthology.orchestra.run.smithi019.stdout: "2d73e2e0-105e-4dab-91d2-4910256a8bda": { 2024-04-25T10:03:26.630 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "~mds0", 2024-04-25T10:03:26.630 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "2d73e2e0-105e-4dab-91d2-4910256a8bda", 2024-04-25T10:03:26.630 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive" 2024-04-25T10:03:26.630 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:26.630 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:03:26.630 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:03:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79527ce640 1 -- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f793806b4d0 msgr2=0x7f793806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79527ce640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f793806b4d0 0x7f793806d990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7940034000 tx=0x7f794002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79527ce640 1 -- 172.21.15.19:0/1936493511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c161710 msgr2=0x7f794c1f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79527ce640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c161710 0x7f794c1f3dd0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f7944007d10 tx=0x7f794400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79527ce640 1 -- 172.21.15.19:0/1936493511 shutdown_connections 2024-04-25T10:03:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79527ce640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f793806b4d0 0x7f793806d990 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79527ce640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c161710 0x7f794c1f3dd0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79527ce640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f794c15dc20 0x7f794c1f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79527ce640 1 --2- 172.21.15.19:0/1936493511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f794c0b9330 0x7f794c0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.616+0000 7f79527ce640 1 -- 172.21.15.19:0/1936493511 >> 172.21.15.19:0/1936493511 conn(0x7f794c0d0420 msgr2=0x7f794c0c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.617+0000 7f79527ce640 1 -- 172.21.15.19:0/1936493511 shutdown_connections 2024-04-25T10:03:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:26.617+0000 7f79527ce640 1 -- 172.21.15.19:0/1936493511 wait complete. 2024-04-25T10:03:26.633 INFO:tasks.cephfs.filesystem:scrub status for tag:2d73e2e0-105e-4dab-91d2-4910256a8bda - {'path': '~mds0', 'tag': '2d73e2e0-105e-4dab-91d2-4910256a8bda', 'options': 'recursive'} 2024-04-25T10:03:38.926 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:38.929 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:56.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:56.635 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:03:56.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.747+0000 7f9b49038640 1 Processor -- start 2024-04-25T10:03:56.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.747+0000 7f9b49038640 1 -- start start 2024-04-25T10:03:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.748+0000 7f9b49038640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b44161240 0x7f9b440b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.748+0000 7f9b49038640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b440cf1d0 con 0x7f9b44161890 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.748+0000 7f9b49038640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b44161620 con 0x7f9b440b54a0 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.748+0000 7f9b49038640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b440b7120 con 0x7f9b44161240 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.748+0000 7f9b42d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b44161240 0x7f9b440b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.748+0000 7f9b4259c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9b44161890 0x7f9b440ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38202/0 (socket says 172.21.15.19:38202) 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.748+0000 7f9b4259c640 1 -- 172.21.15.19:0/2558787025 learned_addr learned my addr 172.21.15.19:0/2558787025 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.749+0000 7f9b42d9d640 1 -- 172.21.15.19:0/2558787025 >> v1:172.21.15.112:6789/0 conn(0x7f9b440b54a0 legacy=0x7f9b440d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.749+0000 7f9b42d9d640 1 -- 172.21.15.19:0/2558787025 >> v1:172.21.15.19:6789/0 conn(0x7f9b44161890 legacy=0x7f9b440ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.749+0000 7f9b42d9d640 1 -- 172.21.15.19:0/2558787025 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b44161e00 con 0x7f9b44161240 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.749+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/2558787025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b44161240 0x7f9b440b6be0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009d90 tx=0x7f9b3402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec56a87aa952700 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.750+0000 7f9b41d9b640 1 -- 172.21.15.19:0/2558787025 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34007740 con 0x7f9b44161240 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.750+0000 7f9b41d9b640 1 -- 172.21.15.19:0/2558787025 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b340078e0 con 0x7f9b44161240 2024-04-25T10:03:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.750+0000 7f9b41d9b640 1 -- 172.21.15.19:0/2558787025 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34007cb0 con 0x7f9b44161240 2024-04-25T10:03:56.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.750+0000 7f9b49038640 1 -- 172.21.15.19:0/2558787025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b44161240 msgr2=0x7f9b440b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.750+0000 7f9b49038640 1 --2- 172.21.15.19:0/2558787025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b44161240 0x7f9b440b6be0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009d90 tx=0x7f9b3402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.750+0000 7f9b49038640 1 -- 172.21.15.19:0/2558787025 shutdown_connections 2024-04-25T10:03:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.750+0000 7f9b49038640 1 --2- 172.21.15.19:0/2558787025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b44161240 0x7f9b440b6be0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.750+0000 7f9b49038640 1 -- 172.21.15.19:0/2558787025 >> 172.21.15.19:0/2558787025 conn(0x7f9b440d04a0 msgr2=0x7f9b440d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.750+0000 7f9b49038640 1 -- 172.21.15.19:0/2558787025 shutdown_connections 2024-04-25T10:03:56.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.750+0000 7f9b49038640 1 -- 172.21.15.19:0/2558787025 wait complete. 2024-04-25T10:03:56.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.752+0000 7f9b49038640 1 Processor -- start 2024-04-25T10:03:56.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.752+0000 7f9b49038640 1 -- start start 2024-04-25T10:03:56.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b49038640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440b54a0 0x7f9b44154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b49038640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b44161240 0x7f9b440cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b49038640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b44161890 0x7f9b440c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b49038640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441626a0 con 0x7f9b44161890 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b49038640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441623c0 con 0x7f9b44161240 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b49038640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440d15f0 con 0x7f9b440b54a0 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b4359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b44161890 0x7f9b440c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b4359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b44161890 0x7f9b440c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57756/0 (socket says 172.21.15.19:57756) 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b4359e640 1 -- 172.21.15.19:0/304807386 learned_addr learned my addr 172.21.15.19:0/304807386 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b4259c640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b44161240 0x7f9b440cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440b54a0 0x7f9b44154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b4359e640 1 -- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440b54a0 msgr2=0x7f9b44154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b4359e640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440b54a0 0x7f9b44154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b4359e640 1 -- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b44161240 msgr2=0x7f9b440cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b4359e640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b44161240 0x7f9b440cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b4359e640 1 -- 172.21.15.19:0/304807386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b441e7b70 con 0x7f9b44161890 2024-04-25T10:03:56.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440b54a0 0x7f9b44154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.753+0000 7f9b4359e640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b44161890 0x7f9b440c7a10 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c001eb0 tx=0x7f9b3c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.754+0000 7f9b2b7fe640 1 -- 172.21.15.19:0/304807386 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c017070 con 0x7f9b44161890 2024-04-25T10:03:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.754+0000 7f9b2b7fe640 1 -- 172.21.15.19:0/304807386 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c0052d0 con 0x7f9b44161890 2024-04-25T10:03:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.754+0000 7f9b49038640 1 -- 172.21.15.19:0/304807386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b441e7e50 con 0x7f9b44161890 2024-04-25T10:03:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.754+0000 7f9b2b7fe640 1 -- 172.21.15.19:0/304807386 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c003810 con 0x7f9b44161890 2024-04-25T10:03:56.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.754+0000 7f9b49038640 1 -- 172.21.15.19:0/304807386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b441e86f0 con 0x7f9b44161890 2024-04-25T10:03:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.755+0000 7f9b2b7fe640 1 -- 172.21.15.19:0/304807386 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b3c01a070 con 0x7f9b44161890 2024-04-25T10:03:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.756+0000 7f9b2b7fe640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9b3006b500 0x7f9b3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.756+0000 7f9b2b7fe640 1 -- 172.21.15.19:0/304807386 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b3c089730 con 0x7f9b44161890 2024-04-25T10:03:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.756+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9b3006b500 0x7f9b3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.756+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9b3006b500 0x7f9b3006d9c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9b34034000 tx=0x7f9b3402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.757+0000 7f9b4be1c740 1 Processor -- start 2024-04-25T10:03:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.757+0000 7f9b4be1c740 1 -- start start 2024-04-25T10:03:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b48e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b45e770 0x56526b461f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526b3c7da0 con 0x56526b48bef0 2024-04-25T10:03:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526b370b80 con 0x56526b45e770 2024-04-25T10:03:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56526b3c3e40 con 0x56526b48bb10 2024-04-25T10:03:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57770/0 (socket says 172.21.15.19:57770) 2024-04-25T10:03:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4259c640 1 -- 172.21.15.19:0/1199152683 learned_addr learned my addr 172.21.15.19:0/1199152683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/1199152683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b48e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4359e640 1 --2- 172.21.15.19:0/1199152683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b45e770 0x56526b461f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4259c640 1 -- 172.21.15.19:0/1199152683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 msgr2=0x56526b48e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4259c640 1 --2- 172.21.15.19:0/1199152683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b48e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4259c640 1 -- 172.21.15.19:0/1199152683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b45e770 msgr2=0x56526b461f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4259c640 1 --2- 172.21.15.19:0/1199152683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b45e770 0x56526b461f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4259c640 1 -- 172.21.15.19:0/1199152683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56526b37af10 con 0x56526b48bef0 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b4259c640 1 --2- 172.21.15.19:0/1199152683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48e860 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f9b38000f60 tx=0x7f9b38001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2f8f68bc22f063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b28ff9640 1 -- 172.21.15.19:0/1199152683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38019070 con 0x56526b48bef0 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b28ff9640 1 -- 172.21.15.19:0/1199152683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b38005ac0 con 0x56526b48bef0 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.758+0000 7f9b28ff9640 1 -- 172.21.15.19:0/1199152683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b380070d0 con 0x56526b48bef0 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.759+0000 7f9b4be1c740 1 -- 172.21.15.19:0/1199152683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 msgr2=0x56526b48e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.759+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/1199152683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48e860 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f9b38000f60 tx=0x7f9b38001970 comp rx=0 tx=0).stop 2024-04-25T10:03:56.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.759+0000 7f9b4be1c740 1 -- 172.21.15.19:0/1199152683 shutdown_connections 2024-04-25T10:03:56.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.759+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/1199152683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b45e770 0x56526b461f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.759+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/1199152683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48e860 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.759+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/1199152683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b48e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.759+0000 7f9b4be1c740 1 -- 172.21.15.19:0/1199152683 >> 172.21.15.19:0/1199152683 conn(0x56526b4609e0 msgr2=0x56526b4a9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.759+0000 7f9b4be1c740 1 -- 172.21.15.19:0/1199152683 shutdown_connections 2024-04-25T10:03:56.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.759+0000 7f9b4be1c740 1 -- 172.21.15.19:0/1199152683 wait complete. 2024-04-25T10:03:56.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.760+0000 7f9b4be1c740 1 -- start start 2024-04-25T10:03:56.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.760+0000 7f9b4be1c740 1 Processor -- start 2024-04-25T10:03:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b489af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48fe80 0x56526b4abd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526b463b90 con 0x56526b48bef0 2024-04-25T10:03:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526b3664b0 con 0x56526b48fe80 2024-04-25T10:03:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56526b463e70 con 0x56526b48bb10 2024-04-25T10:03:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4be1c740 10 client.0 _ll_register_callbacks cb 0x56526b3a6180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57776/0 (socket says 172.21.15.19:57776) 2024-04-25T10:03:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4259c640 1 -- 172.21.15.19:0/710612643 learned_addr learned my addr 172.21.15.19:0/710612643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:56.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4359e640 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48fe80 0x56526b4abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b489af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4259c640 1 -- 172.21.15.19:0/710612643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 msgr2=0x56526b489af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4259c640 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b489af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4259c640 1 -- 172.21.15.19:0/710612643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48fe80 msgr2=0x56526b4abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4259c640 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48fe80 0x56526b4abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4259c640 1 -- 172.21.15.19:0/710612643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56526b1a67d0 con 0x56526b48bef0 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b4259c640 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48a030 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009010 tx=0x7f9b38000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b52127005552e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.761+0000 7f9b21ffb640 1 -- 172.21.15.19:0/710612643 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38022020 con 0x56526b48bef0 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b21ffb640 1 -- 172.21.15.19:0/710612643 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b380074e0 con 0x56526b48bef0 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b4be1c740 1 -- 172.21.15.19:0/710612643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b38005ac0 con 0x56526b48bef0 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b21ffb640 1 -- 172.21.15.19:0/710612643 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b3802c050 con 0x56526b48bef0 2024-04-25T10:03:56.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b4be1c740 10 client.10482 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b4be1c740 1 -- 172.21.15.19:0/710612643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x56526b0dc3f0 con 0x56526b48bef0 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b21ffb640 1 -- 172.21.15.19:0/710612643 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b38005ac0 con 0x56526b48bef0 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b21ffb640 1 -- 172.21.15.19:0/710612643 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9b3802f020 con 0x56526b48bef0 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b4be1c740 10 client.10482 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b4be1c740 10 client.10482 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b4be1c740 20 client.10482 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x56526b4b0a00 0x56526b4ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b4be1c740 4 client.10482 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b4be1c740 1 -- 172.21.15.19:0/710612643 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56526b3d52e0 con 0x56526b4b0a00 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x56526b4b0a00 0x56526b4ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b21ffb640 10 client.10482 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.762+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x56526b4b0a00 0x56526b4ac760 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b55140de7db162fe server_cookie=48cafbd2e98b1542 in_seq=0 out_seq=0 2024-04-25T10:03:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b21ffb640 1 -- 172.21.15.19:0/710612643 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56526b3d52e0 con 0x56526b4b0a00 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b21ffb640 10 client.10482 handle_command_reply: tid=0 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 1 client.10482 shutdown 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 1 -- 172.21.15.19:0/710612643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 msgr2=0x56526b48a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48a030 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009010 tx=0x7f9b38000ed0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 1 -- 172.21.15.19:0/710612643 shutdown_connections 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x56526b4b0a00 0x56526b4ac760 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48fe80 0x56526b4abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48bef0 0x56526b48a030 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b21ffb640 0 client.10482 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/710612643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b489af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 1 -- 172.21.15.19:0/710612643 >> 172.21.15.19:0/710612643 conn(0x56526b45e770 msgr2=0x56526b45f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 1 -- 172.21.15.19:0/710612643 shutdown_connections 2024-04-25T10:03:56.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 1 -- 172.21.15.19:0/710612643 wait complete. 2024-04-25T10:03:56.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.763+0000 7f9b4be1c740 20 client.10482 trim_cache size 0 max 16384 2024-04-25T10:03:56.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.780+0000 7f9b4be1c740 1 Processor -- start 2024-04-25T10:03:56.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.780+0000 7f9b4be1c740 1 -- start start 2024-04-25T10:03:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b463dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bef0 0x56526b4acec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b4ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44200760 con 0x56526b48fe80 2024-04-25T10:03:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441ff8c0 con 0x56526b48bef0 2024-04-25T10:03:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440cb450 con 0x56526b48bb10 2024-04-25T10:03:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b4ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b4ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57790/0 (socket says 172.21.15.19:57790) 2024-04-25T10:03:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4359e640 1 -- 172.21.15.19:0/3299065721 learned_addr learned my addr 172.21.15.19:0/3299065721 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4259c640 1 --2- 172.21.15.19:0/3299065721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bef0 0x56526b4acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/3299065721 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b463dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4359e640 1 -- 172.21.15.19:0/3299065721 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 msgr2=0x56526b463dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4359e640 1 --2- 172.21.15.19:0/3299065721 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b463dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4359e640 1 -- 172.21.15.19:0/3299065721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bef0 msgr2=0x56526b4acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.781+0000 7f9b4359e640 1 --2- 172.21.15.19:0/3299065721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bef0 0x56526b4acec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b4359e640 1 -- 172.21.15.19:0/3299065721 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56526b09c0f0 con 0x56526b48fe80 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/3299065721 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b463dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b4259c640 1 --2- 172.21.15.19:0/3299065721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bef0 0x56526b4acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b4359e640 1 --2- 172.21.15.19:0/3299065721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b4ad400 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c057c20 tx=0x7f9b3c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e56823369a35457 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b23fff640 1 -- 172.21.15.19:0/3299065721 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c055d30 con 0x56526b48fe80 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b23fff640 1 -- 172.21.15.19:0/3299065721 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c055f90 con 0x56526b48fe80 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b23fff640 1 -- 172.21.15.19:0/3299065721 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c0570e0 con 0x56526b48fe80 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3299065721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 msgr2=0x56526b4ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/3299065721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b4ad400 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c057c20 tx=0x7f9b3c01e040 comp rx=0 tx=0).stop 2024-04-25T10:03:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3299065721 shutdown_connections 2024-04-25T10:03:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/3299065721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b4ad400 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/3299065721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bef0 0x56526b4acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.782+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/3299065721 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bb10 0x56526b463dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.783+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3299065721 >> 172.21.15.19:0/3299065721 conn(0x56526b45e770 msgr2=0x56526b464810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.783+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3299065721 shutdown_connections 2024-04-25T10:03:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.783+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3299065721 wait complete. 2024-04-25T10:03:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.783+0000 7f9b4be1c740 1 -- start start 2024-04-25T10:03:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.783+0000 7f9b4be1c740 1 Processor -- start 2024-04-25T10:03:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bb10 0x56526b4640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bef0 0x56526b50dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4be1c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b49f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526b4a6bb0 con 0x56526b48fe80 2024-04-25T10:03:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441ff390 con 0x56526b48bb10 2024-04-25T10:03:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4be1c740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56526b4a6e00 con 0x56526b48bef0 2024-04-25T10:03:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4be1c740 10 client.0 _ll_register_callbacks cb 0x7f9b38007f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b49f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b49f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57798/0 (socket says 172.21.15.19:57798) 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4359e640 1 -- 172.21.15.19:0/3370858082 learned_addr learned my addr 172.21.15.19:0/3370858082 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bb10 0x56526b4640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4259c640 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bef0 0x56526b50dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4359e640 1 -- 172.21.15.19:0/3370858082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bef0 msgr2=0x56526b50dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4359e640 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bef0 0x56526b50dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4359e640 1 -- 172.21.15.19:0/3370858082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bb10 msgr2=0x56526b4640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4359e640 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bb10 0x56526b4640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b4359e640 1 -- 172.21.15.19:0/3370858082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56526b49ed50 con 0x56526b48fe80 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.784+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bb10 0x56526b4640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b4359e640 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b49f6e0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c000fb0 tx=0x7f9b3c089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6df6734ee676137 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b21ffb640 1 -- 172.21.15.19:0/3370858082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c0570e0 con 0x56526b48fe80 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b21ffb640 1 -- 172.21.15.19:0/3370858082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b3c057280 con 0x56526b48fe80 2024-04-25T10:03:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3370858082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b3c055f90 con 0x56526b48fe80 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b21ffb640 1 -- 172.21.15.19:0/3370858082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b3c07e050 con 0x56526b48fe80 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b4be1c740 10 client.10488 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3370858082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x56526b49fc20 con 0x56526b48fe80 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b21ffb640 1 -- 172.21.15.19:0/3370858082 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c055f90 con 0x56526b48fe80 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b21ffb640 1 -- 172.21.15.19:0/3370858082 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9b3c081020 con 0x56526b48fe80 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b4be1c740 10 client.10488 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b4be1c740 10 client.10488 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b4be1c740 20 client.10488 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x56526b4b0e80 0x56526b4b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b4be1c740 4 client.10488 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3370858082 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56526b3cf630 con 0x56526b4b0e80 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.785+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x56526b4b0e80 0x56526b4b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b42d9d640 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x56526b4b0e80 0x56526b4b8470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6462cda993cedeb6 server_cookie=b18644e806156659 in_seq=0 out_seq=0 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b21ffb640 10 client.10488 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b21ffb640 1 -- 172.21.15.19:0/3370858082 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56526b3cf630 con 0x56526b4b0e80 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b21ffb640 10 client.10488 handle_command_reply: tid=0 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 1 client.10488 shutdown 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3370858082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 msgr2=0x56526b49f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b49f6e0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c000fb0 tx=0x7f9b3c089040 comp rx=0 tx=0).stop 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3370858082 shutdown_connections 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x56526b4b0e80 0x56526b4b8470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56526b48fe80 0x56526b49f6e0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56526b48bef0 0x56526b50dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b21ffb640 0 client.10488 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 1 --2- 172.21.15.19:0/3370858082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56526b48bb10 0x56526b4640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3370858082 >> 172.21.15.19:0/3370858082 conn(0x56526b45e770 msgr2=0x56526b4ace20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3370858082 shutdown_connections 2024-04-25T10:03:56.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 1 -- 172.21.15.19:0/3370858082 wait complete. 2024-04-25T10:03:56.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.786+0000 7f9b4be1c740 20 client.10488 trim_cache size 0 max 16384 2024-04-25T10:03:56.798 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:03:56.798 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:03:56.798 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:03:56.798 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:03:56.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 -- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9b3006b500 msgr2=0x7f9b3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9b3006b500 0x7f9b3006d9c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9b34034000 tx=0x7f9b3402f000 comp rx=0 tx=0).stop 2024-04-25T10:03:56.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 -- 172.21.15.19:0/304807386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b44161890 msgr2=0x7f9b440c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b44161890 0x7f9b440c7a10 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c001eb0 tx=0x7f9b3c00c890 comp rx=0 tx=0).stop 2024-04-25T10:03:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 -- 172.21.15.19:0/304807386 shutdown_connections 2024-04-25T10:03:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9b3006b500 0x7f9b3006d9c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b44161890 0x7f9b440c7a10 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b44161240 0x7f9b440cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 --2- 172.21.15.19:0/304807386 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440b54a0 0x7f9b44154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 -- 172.21.15.19:0/304807386 >> 172.21.15.19:0/304807386 conn(0x7f9b440d04a0 msgr2=0x7f9b440d19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 -- 172.21.15.19:0/304807386 shutdown_connections 2024-04-25T10:03:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.787+0000 7f9b49038640 1 -- 172.21.15.19:0/304807386 wait complete. 2024-04-25T10:03:56.803 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:03:56.803 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:03:56.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:03:56.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.901+0000 7f79df438640 1 Processor -- start 2024-04-25T10:03:56.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.901+0000 7f79df438640 1 -- start start 2024-04-25T10:03:56.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.902+0000 7f79df438640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79d815b7d0 0x7f79d80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.902+0000 7f79df438640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d806cf50 con 0x7f79d806c9e0 2024-04-25T10:03:56.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.902+0000 7f79df438640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80ce530 con 0x7f79d80b5620 2024-04-25T10:03:56.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.902+0000 7f79df438640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d80ce810 con 0x7f79d815b7d0 2024-04-25T10:03:56.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.902+0000 7f79dd1d4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79d815b7d0 0x7f79d80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.902+0000 7f79dd1d4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79d815b7d0 0x7f79d80cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57804/0 (socket says 172.21.15.19:57804) 2024-04-25T10:03:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.902+0000 7f79dd1d4640 1 -- 172.21.15.19:0/2496459835 learned_addr learned my addr 172.21.15.19:0/2496459835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.902+0000 7f79dd9d5640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f79d806c9e0 0x7f79d80d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38212/0 (socket says 172.21.15.19:38212) 2024-04-25T10:03:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.902+0000 7f79dd1d4640 1 -- 172.21.15.19:0/2496459835 >> v1:172.21.15.112:6789/0 conn(0x7f79d80b5620 legacy=0x7f79d806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.902+0000 7f79dd1d4640 1 -- 172.21.15.19:0/2496459835 >> v1:172.21.15.19:6789/0 conn(0x7f79d806c9e0 legacy=0x7f79d80d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.902+0000 7f79dd1d4640 1 -- 172.21.15.19:0/2496459835 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80ceaf0 con 0x7f79d815b7d0 2024-04-25T10:03:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.903+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/2496459835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d815b7d0 0x7f79d80cf010 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009870 tx=0x7f79d402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1489c0506d3eb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.903+0000 7f79cffff640 1 -- 172.21.15.19:0/2496459835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4037070 con 0x7f79d815b7d0 2024-04-25T10:03:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.903+0000 7f79cffff640 1 -- 172.21.15.19:0/2496459835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d4032440 con 0x7f79d815b7d0 2024-04-25T10:03:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.903+0000 7f79cffff640 1 -- 172.21.15.19:0/2496459835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4032810 con 0x7f79d815b7d0 2024-04-25T10:03:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.904+0000 7f79df438640 1 -- 172.21.15.19:0/2496459835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d815b7d0 msgr2=0x7f79d80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.904+0000 7f79df438640 1 --2- 172.21.15.19:0/2496459835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d815b7d0 0x7f79d80cf010 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009870 tx=0x7f79d402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.904+0000 7f79df438640 1 -- 172.21.15.19:0/2496459835 shutdown_connections 2024-04-25T10:03:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.904+0000 7f79df438640 1 --2- 172.21.15.19:0/2496459835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d815b7d0 0x7f79d80cf010 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.904+0000 7f79df438640 1 -- 172.21.15.19:0/2496459835 >> 172.21.15.19:0/2496459835 conn(0x7f79d80d0800 msgr2=0x7f79d80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.904+0000 7f79df438640 1 -- 172.21.15.19:0/2496459835 shutdown_connections 2024-04-25T10:03:56.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.904+0000 7f79df438640 1 -- 172.21.15.19:0/2496459835 wait complete. 2024-04-25T10:03:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.905+0000 7f79df438640 1 Processor -- start 2024-04-25T10:03:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.905+0000 7f79df438640 1 -- start start 2024-04-25T10:03:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79df438640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f79d806c9e0 0x7f79d80b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79df438640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d80b5620 0x7f79d8159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79df438640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79d815b7d0 0x7f79d8159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79df438640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80cf6d0 con 0x7f79d80b5620 2024-04-25T10:03:56.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79df438640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80cf550 con 0x7f79d815b7d0 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79df438640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d815bbb0 con 0x7f79d806c9e0 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79dc9d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d80b5620 0x7f79d8159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79dc9d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d80b5620 0x7f79d8159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57816/0 (socket says 172.21.15.19:57816) 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79dc9d3640 1 -- 172.21.15.19:0/1895045518 learned_addr learned my addr 172.21.15.19:0/1895045518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f79d806c9e0 0x7f79d80b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79dd9d5640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79d815b7d0 0x7f79d8159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79dc9d3640 1 -- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f79d806c9e0 msgr2=0x7f79d80b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79dc9d3640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f79d806c9e0 0x7f79d80b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79dc9d3640 1 -- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79d815b7d0 msgr2=0x7f79d8159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79dc9d3640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79d815b7d0 0x7f79d8159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.906+0000 7f79dc9d3640 1 -- 172.21.15.19:0/1895045518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80b3080 con 0x7f79d80b5620 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.907+0000 7f79dc9d3640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d80b5620 0x7f79d8159400 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f79c800c430 tx=0x7f79c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.907+0000 7f79cdffb640 1 -- 172.21.15.19:0/1895045518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8011790 con 0x7f79d80b5620 2024-04-25T10:03:56.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.907+0000 7f79cdffb640 1 -- 172.21.15.19:0/1895045518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c8017070 con 0x7f79d80b5620 2024-04-25T10:03:56.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.907+0000 7f79df438640 1 -- 172.21.15.19:0/1895045518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d80cb490 con 0x7f79d80b5620 2024-04-25T10:03:56.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.907+0000 7f79cdffb640 1 -- 172.21.15.19:0/1895045518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8011b70 con 0x7f79d80b5620 2024-04-25T10:03:56.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.907+0000 7f79df438640 1 -- 172.21.15.19:0/1895045518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d80cba50 con 0x7f79d80b5620 2024-04-25T10:03:56.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.908+0000 7f79cdffb640 1 -- 172.21.15.19:0/1895045518 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79c8011d10 con 0x7f79d80b5620 2024-04-25T10:03:56.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.909+0000 7f79cdffb640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f79bc06b4d0 0x7f79bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.909+0000 7f79cdffb640 1 -- 172.21.15.19:0/1895045518 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79c808a280 con 0x7f79d80b5620 2024-04-25T10:03:56.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.909+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f79bc06b4d0 0x7f79bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.910+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f79bc06b4d0 0x7f79bc06d990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f79d4034000 tx=0x7f79d402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.910+0000 7f79e2211740 1 Processor -- start 2024-04-25T10:03:56.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.910+0000 7f79e2211740 1 -- start start 2024-04-25T10:03:56.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79e2211740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09aa0 0x560f3dc0c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79e2211740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09e80 0x560f3dc0c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79e2211740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dbdc700 0x560f3dbdff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79e2211740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3db45e00 con 0x560f3dbdc700 2024-04-25T10:03:56.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79e2211740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3daeed20 con 0x560f3dc09aa0 2024-04-25T10:03:56.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79e2211740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f3db41ea0 con 0x560f3dc09e80 2024-04-25T10:03:56.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79dd9d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dbdc700 0x560f3dbdff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79dd9d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dbdc700 0x560f3dbdff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57830/0 (socket says 172.21.15.19:57830) 2024-04-25T10:03:56.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79dd9d5640 1 -- 172.21.15.19:0/1435606901 learned_addr learned my addr 172.21.15.19:0/1435606901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:56.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/1435606901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09aa0 0x560f3dc0c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79dc9d3640 1 --2- 172.21.15.19:0/1435606901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09e80 0x560f3dc0c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79dd9d5640 1 -- 172.21.15.19:0/1435606901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09e80 msgr2=0x560f3dc0c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79dd9d5640 1 --2- 172.21.15.19:0/1435606901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09e80 0x560f3dc0c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79dd9d5640 1 -- 172.21.15.19:0/1435606901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09aa0 msgr2=0x560f3dc0c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79dd9d5640 1 --2- 172.21.15.19:0/1435606901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09aa0 0x560f3dc0c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79dd9d5640 1 -- 172.21.15.19:0/1435606901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f3db10df0 con 0x560f3dbdc700 2024-04-25T10:03:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.911+0000 7f79dd9d5640 1 --2- 172.21.15.19:0/1435606901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dbdc700 0x560f3dbdff10 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f79d0007400 tx=0x7f79d00078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc02bfb1e0e6e135 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79c2ffd640 1 -- 172.21.15.19:0/1435606901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0019070 con 0x560f3dbdc700 2024-04-25T10:03:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79c2ffd640 1 -- 172.21.15.19:0/1435606901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0001ca0 con 0x560f3dbdc700 2024-04-25T10:03:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79c2ffd640 1 -- 172.21.15.19:0/1435606901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0002070 con 0x560f3dbdc700 2024-04-25T10:03:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79e2211740 1 -- 172.21.15.19:0/1435606901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dbdc700 msgr2=0x560f3dbdff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79e2211740 1 --2- 172.21.15.19:0/1435606901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dbdc700 0x560f3dbdff10 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f79d0007400 tx=0x7f79d00078c0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79e2211740 1 -- 172.21.15.19:0/1435606901 shutdown_connections 2024-04-25T10:03:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79e2211740 1 --2- 172.21.15.19:0/1435606901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dbdc700 0x560f3dbdff10 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79e2211740 1 --2- 172.21.15.19:0/1435606901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09e80 0x560f3dc0c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79e2211740 1 --2- 172.21.15.19:0/1435606901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09aa0 0x560f3dc0c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79e2211740 1 -- 172.21.15.19:0/1435606901 >> 172.21.15.19:0/1435606901 conn(0x560f3dbde970 msgr2=0x560f3dc272b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79e2211740 1 -- 172.21.15.19:0/1435606901 shutdown_connections 2024-04-25T10:03:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.912+0000 7f79e2211740 1 -- 172.21.15.19:0/1435606901 wait complete. 2024-04-25T10:03:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.913+0000 7f79e2211740 1 -- start start 2024-04-25T10:03:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.913+0000 7f79e2211740 1 Processor -- start 2024-04-25T10:03:56.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79e2211740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09aa0 0x560f3dc04fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79e2211740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09e80 0x560f3dc070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79e2211740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc29aa0 0x560f3dc00790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79e2211740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3dbe1b20 con 0x560f3dc09e80 2024-04-25T10:03:56.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79e2211740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3dae4590 con 0x560f3dc09aa0 2024-04-25T10:03:56.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79e2211740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f3dbe1e00 con 0x560f3dc29aa0 2024-04-25T10:03:56.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79e2211740 10 client.0 _ll_register_callbacks cb 0x560f3daf0e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79dc9d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09e80 0x560f3dc070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79dc9d3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09e80 0x560f3dc070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57834/0 (socket says 172.21.15.19:57834) 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79dc9d3640 1 -- 172.21.15.19:0/2571593544 learned_addr learned my addr 172.21.15.19:0/2571593544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09aa0 0x560f3dc04fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79dd9d5640 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc29aa0 0x560f3dc00790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79dc9d3640 1 -- 172.21.15.19:0/2571593544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc29aa0 msgr2=0x560f3dc00790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79dc9d3640 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc29aa0 0x560f3dc00790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79dc9d3640 1 -- 172.21.15.19:0/2571593544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09aa0 msgr2=0x560f3dc04fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79dc9d3640 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09aa0 0x560f3dc04fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79dc9d3640 1 -- 172.21.15.19:0/2571593544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f3d85a3f0 con 0x560f3dc09e80 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79dc9d3640 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09e80 0x560f3dc070e0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f79c800cdf0 tx=0x7f79c8058890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc85ceb19782039 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.914+0000 7f79b7fff640 1 -- 172.21.15.19:0/2571593544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8086060 con 0x560f3dc09e80 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79b7fff640 1 -- 172.21.15.19:0/2571593544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79c8054e40 con 0x560f3dc09e80 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79e2211740 1 -- 172.21.15.19:0/2571593544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79d0001ca0 con 0x560f3dc09e80 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79b7fff640 1 -- 172.21.15.19:0/2571593544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79c8075050 con 0x560f3dc09e80 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79e2211740 10 client.10503 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79e2211740 1 -- 172.21.15.19:0/2571593544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x560f3d9247d0 con 0x560f3dc09e80 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79b7fff640 1 -- 172.21.15.19:0/2571593544 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0001ca0 con 0x560f3dc09e80 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79b7fff640 1 -- 172.21.15.19:0/2571593544 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f79c8060a50 con 0x560f3dc09e80 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79e2211740 10 client.10503 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79e2211740 10 client.10503 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79e2211740 20 client.10503 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79e2211740 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x560f3dc02670 0x560f3dc05dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79e2211740 4 client.10503 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79e2211740 1 -- 172.21.15.19:0/2571593544 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f3db53340 con 0x560f3dc02670 2024-04-25T10:03:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x560f3dc02670 0x560f3dc05dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79b7fff640 10 client.10503 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.915+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x560f3dc02670 0x560f3dc05dc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1636ed000727da70 server_cookie=e133fc57b32b70ee in_seq=0 out_seq=0 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79b7fff640 1 -- 172.21.15.19:0/2571593544 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f3db53340 con 0x560f3dc02670 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79b7fff640 10 client.10503 handle_command_reply: tid=0 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 1 client.10503 shutdown 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 1 -- 172.21.15.19:0/2571593544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09e80 msgr2=0x560f3dc070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09e80 0x560f3dc070e0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f79c800cdf0 tx=0x7f79c8058890 comp rx=0 tx=0).stop 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 1 -- 172.21.15.19:0/2571593544 shutdown_connections 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x560f3dc02670 0x560f3dc05dc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc29aa0 0x560f3dc00790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09e80 0x560f3dc070e0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79b7fff640 0 client.10503 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 1 --2- 172.21.15.19:0/2571593544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09aa0 0x560f3dc04fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 1 -- 172.21.15.19:0/2571593544 >> 172.21.15.19:0/2571593544 conn(0x560f3dbdc700 msgr2=0x560f3dc08cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 1 -- 172.21.15.19:0/2571593544 shutdown_connections 2024-04-25T10:03:56.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 1 -- 172.21.15.19:0/2571593544 wait complete. 2024-04-25T10:03:56.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.916+0000 7f79e2211740 20 client.10503 trim_cache size 0 max 16384 2024-04-25T10:03:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.928+0000 7f79e2211740 1 Processor -- start 2024-04-25T10:03:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.928+0000 7f79e2211740 1 -- start start 2024-04-25T10:03:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79e2211740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09aa0 0x560f3dc2a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79e2211740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09e80 0x560f3dc2acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79e2211740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc29aa0 0x560f3dbdd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79e2211740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f3dc26ec0 con 0x560f3dc09aa0 2024-04-25T10:03:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79e2211740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8200440 con 0x560f3dc29aa0 2024-04-25T10:03:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79e2211740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d81ff640 con 0x560f3dc09e80 2024-04-25T10:03:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79dd1d4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09aa0 0x560f3dc2a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79dd1d4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09aa0 0x560f3dc2a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57848/0 (socket says 172.21.15.19:57848) 2024-04-25T10:03:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79dd1d4640 1 -- 172.21.15.19:0/1793482615 learned_addr learned my addr 172.21.15.19:0/1793482615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79dd9d5640 1 --2- 172.21.15.19:0/1793482615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc29aa0 0x560f3dbdd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79dc9d3640 1 --2- 172.21.15.19:0/1793482615 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09e80 0x560f3dc2acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79dd1d4640 1 -- 172.21.15.19:0/1793482615 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09e80 msgr2=0x560f3dc2acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/1793482615 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09e80 0x560f3dc2acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79dd1d4640 1 -- 172.21.15.19:0/1793482615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc29aa0 msgr2=0x560f3dbdd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/1793482615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc29aa0 0x560f3dbdd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79dd1d4640 1 -- 172.21.15.19:0/1793482615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f3dbdbf40 con 0x560f3dc09aa0 2024-04-25T10:03:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.929+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/1793482615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09aa0 0x560f3dc2a790 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009630 tx=0x7f79d402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419bc9c52f496bd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79c27fc640 1 -- 172.21.15.19:0/1793482615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4053070 con 0x560f3dc09aa0 2024-04-25T10:03:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79c27fc640 1 -- 172.21.15.19:0/1793482615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d403ecb0 con 0x560f3dc09aa0 2024-04-25T10:03:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79c27fc640 1 -- 172.21.15.19:0/1793482615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4037040 con 0x560f3dc09aa0 2024-04-25T10:03:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79e2211740 1 -- 172.21.15.19:0/1793482615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09aa0 msgr2=0x560f3dc2a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79e2211740 1 --2- 172.21.15.19:0/1793482615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09aa0 0x560f3dc2a790 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009630 tx=0x7f79d402c000 comp rx=0 tx=0).stop 2024-04-25T10:03:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79e2211740 1 -- 172.21.15.19:0/1793482615 shutdown_connections 2024-04-25T10:03:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79e2211740 1 --2- 172.21.15.19:0/1793482615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc29aa0 0x560f3dbdd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79e2211740 1 --2- 172.21.15.19:0/1793482615 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09e80 0x560f3dc2acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79e2211740 1 --2- 172.21.15.19:0/1793482615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc09aa0 0x560f3dc2a790 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79e2211740 1 -- 172.21.15.19:0/1793482615 >> 172.21.15.19:0/1793482615 conn(0x560f3dbdc700 msgr2=0x560f3dc00560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79e2211740 1 -- 172.21.15.19:0/1793482615 shutdown_connections 2024-04-25T10:03:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.930+0000 7f79e2211740 1 -- 172.21.15.19:0/1793482615 wait complete. 2024-04-25T10:03:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.931+0000 7f79e2211740 1 -- start start 2024-04-25T10:03:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.931+0000 7f79e2211740 1 Processor -- start 2024-04-25T10:03:56.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79e2211740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09aa0 0x560f3dc06ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79e2211740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09e80 0x560f3dc1d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79e2211740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc29aa0 0x560f3dbdd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79e2211740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80cd440 con 0x560f3dc29aa0 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79e2211740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80cce40 con 0x560f3dc09e80 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79e2211740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f3dc91b00 con 0x560f3dc09aa0 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79e2211740 10 client.0 _ll_register_callbacks cb 0x7f79d8062560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79dd9d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc29aa0 0x560f3dbdd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79dd9d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc29aa0 0x560f3dbdd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57858/0 (socket says 172.21.15.19:57858) 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79dd9d5640 1 -- 172.21.15.19:0/1389540234 learned_addr learned my addr 172.21.15.19:0/1389540234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79dc9d3640 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09e80 0x560f3dc1d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09aa0 0x560f3dc06ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79dd9d5640 1 -- 172.21.15.19:0/1389540234 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09aa0 msgr2=0x560f3dc06ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79dd9d5640 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09aa0 0x560f3dc06ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79dd9d5640 1 -- 172.21.15.19:0/1389540234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09e80 msgr2=0x560f3dc1d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79dd9d5640 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09e80 0x560f3dc1d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79dd9d5640 1 -- 172.21.15.19:0/1389540234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f3dc39fa0 con 0x560f3dc29aa0 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79dd9d5640 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc29aa0 0x560f3dbdd250 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f79d0013000 tx=0x7f79d0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fd5465116b750c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79b7fff640 1 -- 172.21.15.19:0/1389540234 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0019040 con 0x560f3dc29aa0 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79b7fff640 1 -- 172.21.15.19:0/1389540234 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79d000f070 con 0x560f3dc29aa0 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.932+0000 7f79e2211740 1 -- 172.21.15.19:0/1389540234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79d403ecb0 con 0x560f3dc29aa0 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79b7fff640 1 -- 172.21.15.19:0/1389540234 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79d002c050 con 0x560f3dc29aa0 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79e2211740 10 client.10509 fetch_fsmap learned FSMap version 102 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79e2211740 1 -- 172.21.15.19:0/1389540234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x560f3dc070e0 con 0x560f3dc29aa0 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79b7fff640 1 -- 172.21.15.19:0/1389540234 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d403ecb0 con 0x560f3dc29aa0 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79b7fff640 1 -- 172.21.15.19:0/1389540234 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f79d002f020 con 0x560f3dc29aa0 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79e2211740 10 client.10509 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79e2211740 10 client.10509 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:03:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79e2211740 20 client.10509 populate_metadata read hostname 'smithi019' 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79e2211740 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x560f3dc916f0 0x560f3dbdd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79e2211740 4 client.10509 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79e2211740 1 -- 172.21.15.19:0/1389540234 --> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560f3db4d690 con 0x560f3dc916f0 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x560f3dc916f0 0x560f3dbdd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79dd1d4640 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x560f3dc916f0 0x560f3dbdd790 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983743d547b243e5 server_cookie=c3448c8405bb82f1 in_seq=0 out_seq=0 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.933+0000 7f79b7fff640 10 client.10509 ms_handle_connect on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79b7fff640 1 -- 172.21.15.19:0/1389540234 <== mds.0 v2:172.21.15.19:6833/717842662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560f3db4d690 con 0x560f3dc916f0 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79b7fff640 10 client.10509 handle_command_reply: tid=0 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 1 client.10509 shutdown 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 1 -- 172.21.15.19:0/1389540234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc29aa0 msgr2=0x560f3dbdd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc29aa0 0x560f3dbdd250 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f79d0013000 tx=0x7f79d0011000 comp rx=0 tx=0).stop 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 1 -- 172.21.15.19:0/1389540234 shutdown_connections 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.19:6833/717842662,v1:172.21.15.19:6835/717842662] conn(0x560f3dc916f0 0x560f3dbdd790 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f3dc29aa0 0x560f3dbdd250 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f3dc09e80 0x560f3dc1d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 1 --2- 172.21.15.19:0/1389540234 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x560f3dc09aa0 0x560f3dc06ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79b7fff640 0 client.10509 ms_handle_reset on v2:172.21.15.19:6833/717842662 2024-04-25T10:03:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 1 -- 172.21.15.19:0/1389540234 >> 172.21.15.19:0/1389540234 conn(0x560f3dbdc700 msgr2=0x560f3dc49b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 1 -- 172.21.15.19:0/1389540234 shutdown_connections 2024-04-25T10:03:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 1 -- 172.21.15.19:0/1389540234 wait complete. 2024-04-25T10:03:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79e2211740 20 client.10509 trim_cache size 0 max 16384 2024-04-25T10:03:56.946 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:03:56.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79df438640 1 -- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f79bc06b4d0 msgr2=0x7f79bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79df438640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f79bc06b4d0 0x7f79bc06d990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f79d4034000 tx=0x7f79d402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79df438640 1 -- 172.21.15.19:0/1895045518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d80b5620 msgr2=0x7f79d8159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.934+0000 7f79df438640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d80b5620 0x7f79d8159400 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f79c800c430 tx=0x7f79c800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.935+0000 7f79df438640 1 -- 172.21.15.19:0/1895045518 shutdown_connections 2024-04-25T10:03:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.935+0000 7f79df438640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f79bc06b4d0 0x7f79bc06d990 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.935+0000 7f79df438640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79d815b7d0 0x7f79d8159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.935+0000 7f79df438640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d80b5620 0x7f79d8159400 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.935+0000 7f79df438640 1 --2- 172.21.15.19:0/1895045518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f79d806c9e0 0x7f79d80b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.935+0000 7f79df438640 1 -- 172.21.15.19:0/1895045518 >> 172.21.15.19:0/1895045518 conn(0x7f79d80d0800 msgr2=0x7f79d8158cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.935+0000 7f79df438640 1 -- 172.21.15.19:0/1895045518 shutdown_connections 2024-04-25T10:03:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:56.935+0000 7f79df438640 1 -- 172.21.15.19:0/1895045518 wait complete. 2024-04-25T10:03:56.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:03:57.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.045+0000 7f0c20728640 1 Processor -- start 2024-04-25T10:03:57.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.045+0000 7f0c20728640 1 -- start start 2024-04-25T10:03:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c20728640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c1815db70 0x7f0c18162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c20728640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0c180b6cb0 0x7f0c180b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c20728640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c18162690 con 0x7f0c180b5a60 2024-04-25T10:03:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c20728640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c180d3800 con 0x7f0c180b6cb0 2024-04-25T10:03:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c20728640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180ceee0 con 0x7f0c1815db70 2024-04-25T10:03:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c1ecc5640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0c180b5a60 0x7f0c1806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38224/0 (socket says 172.21.15.19:38224) 2024-04-25T10:03:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c1ecc5640 1 -- 172.21.15.19:0/1744069356 learned_addr learned my addr 172.21.15.19:0/1744069356 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c1e4c4640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c1815db70 0x7f0c18162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464767058 0 0) 0x7f0c18162690 con 0x7f0c180b5a60 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c1dcc3640 1 --2- 172.21.15.19:0/1744069356 >> v2:172.21.15.112:3300/0 conn(0x7f0c180b6cb0 0x7f0c180b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c04003520 con 0x7f0c180b5a60 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3672446102 0 0) 0x7f0c04003520 con 0x7f0c180b5a60 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c1815db70 msgr2=0x7f0c18162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c1d4c2640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c1815db70 0x7f0c18162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 >> v2:172.21.15.112:3300/0 conn(0x7f0c180b6cb0 msgr2=0x7f0c180b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.046+0000 7f0c1d4c2640 1 --2- 172.21.15.19:0/1744069356 >> v2:172.21.15.112:3300/0 conn(0x7f0c180b6cb0 0x7f0c180b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c180d1d10 con 0x7f0c180b5a60 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0c100019f0 con 0x7f0c180b5a60 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 >> v1:172.21.15.19:6789/0 conn(0x7f0c180b5a60 legacy=0x7f0c1806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1d4c2640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c040039b0 0x7f0c04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1d4c2640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c04004800 0x7f0c04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1d4c2640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c1815db70 0x7f0c040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180d3800 con 0x7f0c04004800 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180ceee0 con 0x7f0c040039b0 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0400b9c0 con 0x7f0c1815db70 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1dcc3640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c04004800 0x7f0c04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1e4c4640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c040039b0 0x7f0c04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1ecc5640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c1815db70 0x7f0c040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1dcc3640 1 -- 172.21.15.19:0/1744069356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c1815db70 msgr2=0x7f0c040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1dcc3640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c1815db70 0x7f0c040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1dcc3640 1 -- 172.21.15.19:0/1744069356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c040039b0 msgr2=0x7f0c04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1dcc3640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c040039b0 0x7f0c04003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.047+0000 7f0c1dcc3640 1 -- 172.21.15.19:0/1744069356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c04003520 con 0x7f0c04004800 2024-04-25T10:03:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.048+0000 7f0c1dcc3640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c04004800 0x7f0c04004c80 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009a90 tx=0x7f0c04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb573e49cc37656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:57.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.048+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08036070 con 0x7f0c04004800 2024-04-25T10:03:57.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.048+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c08031400 con 0x7f0c04004800 2024-04-25T10:03:57.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.048+0000 7f0c1d4c2640 1 -- 172.21.15.19:0/1744069356 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08009690 con 0x7f0c04004800 2024-04-25T10:03:57.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.048+0000 7f0c20728640 1 -- 172.21.15.19:0/1744069356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c04004800 msgr2=0x7f0c04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.048+0000 7f0c20728640 1 --2- 172.21.15.19:0/1744069356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c04004800 0x7f0c04004c80 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009a90 tx=0x7f0c04005830 comp rx=0 tx=0).stop 2024-04-25T10:03:57.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.048+0000 7f0c1ecc5640 1 -- 172.21.15.19:0/1744069356 reap_dead start 2024-04-25T10:03:57.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.049+0000 7f0c20728640 1 -- 172.21.15.19:0/1744069356 shutdown_connections 2024-04-25T10:03:57.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.049+0000 7f0c20728640 1 -- 172.21.15.19:0/1744069356 >> 172.21.15.19:0/1744069356 conn(0x7f0c180d0520 msgr2=0x7f0c180d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:57.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.049+0000 7f0c20728640 1 -- 172.21.15.19:0/1744069356 shutdown_connections 2024-04-25T10:03:57.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.049+0000 7f0c20728640 1 -- 172.21.15.19:0/1744069356 wait complete. 2024-04-25T10:03:57.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.050+0000 7f0c20728640 1 Processor -- start 2024-04-25T10:03:57.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.050+0000 7f0c20728640 1 -- start start 2024-04-25T10:03:57.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c20728640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c04004800 0x7f0c180cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c20728640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c18155c00 0x7f0c18156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c20728640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c181442d0 0x7f0c18144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c20728640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1815df50 con 0x7f0c181442d0 2024-04-25T10:03:57.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c20728640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180d22d0 con 0x7f0c04004800 2024-04-25T10:03:57.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c20728640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180ce530 con 0x7f0c18155c00 2024-04-25T10:03:57.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c1ecc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c181442d0 0x7f0c18144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c1ecc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c181442d0 0x7f0c18144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57888/0 (socket says 172.21.15.19:57888) 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c1ecc5640 1 -- 172.21.15.19:0/723238726 learned_addr learned my addr 172.21.15.19:0/723238726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c1e4c4640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c04004800 0x7f0c180cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c1dcc3640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c18155c00 0x7f0c18156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c1ecc5640 1 -- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c18155c00 msgr2=0x7f0c18156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c1ecc5640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c18155c00 0x7f0c18156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c1ecc5640 1 -- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c04004800 msgr2=0x7f0c180cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c1ecc5640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c04004800 0x7f0c180cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c1ecc5640 1 -- 172.21.15.19:0/723238726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18154b60 con 0x7f0c181442d0 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c1ecc5640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c181442d0 0x7f0c18144750 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f0c180b6570 tx=0x7f0c1000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c0effd640 1 -- 172.21.15.19:0/723238726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000ebb0 con 0x7f0c181442d0 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c0effd640 1 -- 172.21.15.19:0/723238726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1000ed70 con 0x7f0c181442d0 2024-04-25T10:03:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c0effd640 1 -- 172.21.15.19:0/723238726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10014670 con 0x7f0c181442d0 2024-04-25T10:03:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.051+0000 7f0c20728640 1 -- 172.21.15.19:0/723238726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c18154e40 con 0x7f0c181442d0 2024-04-25T10:03:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.052+0000 7f0c20728640 1 -- 172.21.15.19:0/723238726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c18155400 con 0x7f0c181442d0 2024-04-25T10:03:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.052+0000 7f0c20728640 1 -- 172.21.15.19:0/723238726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c18073990 con 0x7f0c181442d0 2024-04-25T10:03:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.053+0000 7f0c0effd640 1 -- 172.21.15.19:0/723238726 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c10014810 con 0x7f0c181442d0 2024-04-25T10:03:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.053+0000 7f0c0effd640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bfc06b4d0 0x7f0bfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.053+0000 7f0c0effd640 1 -- 172.21.15.19:0/723238726 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c1008a150 con 0x7f0c181442d0 2024-04-25T10:03:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.054+0000 7f0c1e4c4640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bfc06b4d0 0x7f0bfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.054+0000 7f0c1e4c4640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bfc06b4d0 0x7f0bfc06d990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009ae0 tx=0x7f0c14009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.058+0000 7f0c0effd640 1 -- 172.21.15.19:0/723238726 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c10058750 con 0x7f0c181442d0 2024-04-25T10:03:57.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.277+0000 7f0c20728640 1 -- 172.21.15.19:0/723238726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0c18161490 con 0x7f0c181442d0 2024-04-25T10:03:57.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.277+0000 7f0c0effd640 1 -- 172.21.15.19:0/723238726 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0c1005c7f0 con 0x7f0c181442d0 2024-04-25T10:03:57.282 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:57.282 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:03:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 -- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bfc06b4d0 msgr2=0x7f0bfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bfc06b4d0 0x7f0bfc06d990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009ae0 tx=0x7f0c14009190 comp rx=0 tx=0).stop 2024-04-25T10:03:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 -- 172.21.15.19:0/723238726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c181442d0 msgr2=0x7f0c18144750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c181442d0 0x7f0c18144750 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f0c180b6570 tx=0x7f0c1000e530 comp rx=0 tx=0).stop 2024-04-25T10:03:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 -- 172.21.15.19:0/723238726 shutdown_connections 2024-04-25T10:03:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bfc06b4d0 0x7f0bfc06d990 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c181442d0 0x7f0c18144750 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c18155c00 0x7f0c18156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 --2- 172.21.15.19:0/723238726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c04004800 0x7f0c180cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 -- 172.21.15.19:0/723238726 >> 172.21.15.19:0/723238726 conn(0x7f0c180d0520 msgr2=0x7f0c180d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 -- 172.21.15.19:0/723238726 shutdown_connections 2024-04-25T10:03:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.279+0000 7f0c20728640 1 -- 172.21.15.19:0/723238726 wait complete. 2024-04-25T10:03:57.295 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:03:57.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:57.331 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:57.389 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:03:57.389 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:03:57.389 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:57.405 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:03:57.405 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:57.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:57.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.596+0000 7efe37658640 1 Processor -- start 2024-04-25T10:03:57.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.596+0000 7efe37658640 1 -- start start 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.597+0000 7efe37658640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dc90 0x7efe301625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.597+0000 7efe37658640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe300cc8e0 0x7efe3015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.597+0000 7efe37658640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3015f390 con 0x7efe300d1810 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.597+0000 7efe37658640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe300cce50 con 0x7efe300cc8e0 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.597+0000 7efe37658640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe30161af0 con 0x7efe3015dc90 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.597+0000 7efe34bf3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe300cc8e0 0x7efe3015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.597+0000 7efe34bf3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe300cc8e0 0x7efe3015ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57902/0 (socket says 172.21.15.19:57902) 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.597+0000 7efe34bf3640 1 -- 172.21.15.19:0/2938522635 learned_addr learned my addr 172.21.15.19:0/2938522635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.598+0000 7efe353f4640 1 --2- 172.21.15.19:0/2938522635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dc90 0x7efe301625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.598+0000 7efe34bf3640 1 -- 172.21.15.19:0/2938522635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dc90 msgr2=0x7efe301625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.598+0000 7efe34bf3640 1 --2- 172.21.15.19:0/2938522635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dc90 0x7efe301625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.598+0000 7efe34bf3640 1 -- 172.21.15.19:0/2938522635 >> v1:172.21.15.112:6789/0 conn(0x7efe300d1810 legacy=0x7efe300cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.598+0000 7efe34bf3640 1 -- 172.21.15.19:0/2938522635 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe30161dd0 con 0x7efe300cc8e0 2024-04-25T10:03:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.598+0000 7efe353f4640 1 --2- 172.21.15.19:0/2938522635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dc90 0x7efe301625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.598+0000 7efe34bf3640 1 --2- 172.21.15.19:0/2938522635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300cc8e0 0x7efe3015ee50 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7efe20009870 tx=0x7efe2002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8516a319120aec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.599+0000 7efe27fff640 1 -- 172.21.15.19:0/2938522635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20037070 con 0x7efe300cc8e0 2024-04-25T10:03:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.599+0000 7efe27fff640 1 -- 172.21.15.19:0/2938522635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe20032440 con 0x7efe300cc8e0 2024-04-25T10:03:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.599+0000 7efe27fff640 1 -- 172.21.15.19:0/2938522635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20032810 con 0x7efe300cc8e0 2024-04-25T10:03:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.599+0000 7efe37658640 1 -- 172.21.15.19:0/2938522635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300cc8e0 msgr2=0x7efe3015ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.599+0000 7efe37658640 1 --2- 172.21.15.19:0/2938522635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300cc8e0 0x7efe3015ee50 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7efe20009870 tx=0x7efe2002a180 comp rx=0 tx=0).stop 2024-04-25T10:03:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.599+0000 7efe37658640 1 -- 172.21.15.19:0/2938522635 shutdown_connections 2024-04-25T10:03:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.599+0000 7efe37658640 1 --2- 172.21.15.19:0/2938522635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300cc8e0 0x7efe3015ee50 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.599+0000 7efe37658640 1 --2- 172.21.15.19:0/2938522635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dc90 0x7efe301625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.599+0000 7efe37658640 1 -- 172.21.15.19:0/2938522635 >> 172.21.15.19:0/2938522635 conn(0x7efe3006f590 msgr2=0x7efe3006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.599+0000 7efe37658640 1 -- 172.21.15.19:0/2938522635 shutdown_connections 2024-04-25T10:03:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.599+0000 7efe37658640 1 -- 172.21.15.19:0/2938522635 wait complete. 2024-04-25T10:03:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.600+0000 7efe37658640 1 Processor -- start 2024-04-25T10:03:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.600+0000 7efe37658640 1 -- start start 2024-04-25T10:03:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.601+0000 7efe37658640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe300cc8e0 0x7efe300ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.601+0000 7efe37658640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe300d1810 0x7efe3015b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.601+0000 7efe37658640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3015dc90 0x7efe300cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.601+0000 7efe37658640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3015e070 con 0x7efe3015dc90 2024-04-25T10:03:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.601+0000 7efe37658640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe300cd130 con 0x7efe300d1810 2024-04-25T10:03:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.601+0000 7efe37658640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3015e1f0 con 0x7efe300cc8e0 2024-04-25T10:03:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.601+0000 7efe35bf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3015dc90 0x7efe300cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.601+0000 7efe35bf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3015dc90 0x7efe300cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57904/0 (socket says 172.21.15.19:57904) 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.601+0000 7efe35bf5640 1 -- 172.21.15.19:0/4001485226 learned_addr learned my addr 172.21.15.19:0/4001485226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.601+0000 7efe34bf3640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe300d1810 0x7efe3015b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.601+0000 7efe353f4640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe300cc8e0 0x7efe300ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.602+0000 7efe35bf5640 1 -- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe300cc8e0 msgr2=0x7efe300ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.602+0000 7efe35bf5640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe300cc8e0 0x7efe300ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.602+0000 7efe35bf5640 1 -- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe300d1810 msgr2=0x7efe3015b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.602+0000 7efe35bf5640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe300d1810 0x7efe3015b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.602+0000 7efe35bf5640 1 -- 172.21.15.19:0/4001485226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300afde0 con 0x7efe3015dc90 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.602+0000 7efe35bf5640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3015dc90 0x7efe300cb6c0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7efe2800c7b0 tx=0x7efe2800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.602+0000 7efe25ffb640 1 -- 172.21.15.19:0/4001485226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800ce60 con 0x7efe3015dc90 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.602+0000 7efe25ffb640 1 -- 172.21.15.19:0/4001485226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe28004b10 con 0x7efe3015dc90 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.602+0000 7efe37658640 1 -- 172.21.15.19:0/4001485226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe300b0080 con 0x7efe3015dc90 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.602+0000 7efe25ffb640 1 -- 172.21.15.19:0/4001485226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800a850 con 0x7efe3015dc90 2024-04-25T10:03:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.602+0000 7efe37658640 1 -- 172.21.15.19:0/4001485226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe300cbc00 con 0x7efe3015dc90 2024-04-25T10:03:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.603+0000 7efe37658640 1 -- 172.21.15.19:0/4001485226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe30067ef0 con 0x7efe3015dc90 2024-04-25T10:03:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.603+0000 7efe25ffb640 1 -- 172.21.15.19:0/4001485226 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe2800a9f0 con 0x7efe3015dc90 2024-04-25T10:03:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.604+0000 7efe25ffb640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1406b4d0 0x7efe1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.604+0000 7efe25ffb640 1 -- 172.21.15.19:0/4001485226 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe2808a380 con 0x7efe3015dc90 2024-04-25T10:03:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.604+0000 7efe353f4640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1406b4d0 0x7efe1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.605+0000 7efe353f4640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1406b4d0 0x7efe1406d990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00ab60 tx=0x7efe2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.608+0000 7efe25ffb640 1 -- 172.21.15.19:0/4001485226 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe30067ef0 con 0x7efe3015dc90 2024-04-25T10:03:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.791+0000 7efe37658640 1 -- 172.21.15.19:0/4001485226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe300cf6b0 con 0x7efe3015dc90 2024-04-25T10:03:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.792+0000 7efe25ffb640 1 -- 172.21.15.19:0/4001485226 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+16872 (secure 0 0 0) 0x7efe28020070 con 0x7efe3015dc90 2024-04-25T10:03:57.797 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:57.797 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":154,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:03:15.658204+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:12.307052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:13.005066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 -- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1406b4d0 msgr2=0x7efe1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1406b4d0 0x7efe1406d990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00ab60 tx=0x7efe2c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 -- 172.21.15.19:0/4001485226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3015dc90 msgr2=0x7efe300cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3015dc90 0x7efe300cb6c0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7efe2800c7b0 tx=0x7efe2800cc70 comp rx=0 tx=0).stop 2024-04-25T10:03:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 -- 172.21.15.19:0/4001485226 shutdown_connections 2024-04-25T10:03:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1406b4d0 0x7efe1406d990 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3015dc90 0x7efe300cb6c0 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe300d1810 0x7efe3015b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 --2- 172.21.15.19:0/4001485226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe300cc8e0 0x7efe300ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 -- 172.21.15.19:0/4001485226 >> 172.21.15.19:0/4001485226 conn(0x7efe3006f590 msgr2=0x7efe3015cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:57.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 -- 172.21.15.19:0/4001485226 shutdown_connections 2024-04-25T10:03:57.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.794+0000 7efe37658640 1 -- 172.21.15.19:0/4001485226 wait complete. 2024-04-25T10:03:57.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:12.307052+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:13.005066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:57.810 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:57.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.909+0000 7f7566190640 1 Processor -- start 2024-04-25T10:03:57.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.909+0000 7f7566190640 1 -- start start 2024-04-25T10:03:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.910+0000 7f7566190640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75601610f0 0x7f756006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.910+0000 7f7566190640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f75601616f0 0x7f7560161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.910+0000 7f7566190640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75600c6960 con 0x7f75600ce0a0 2024-04-25T10:03:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.910+0000 7f7566190640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75601614d0 con 0x7f75601610f0 2024-04-25T10:03:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.910+0000 7f7566190640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7560162260 con 0x7f75601616f0 2024-04-25T10:03:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.910+0000 7f755f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75601610f0 0x7f756006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.910+0000 7f755f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75601610f0 0x7f756006c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49102/0 (socket says 172.21.15.19:49102) 2024-04-25T10:03:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.910+0000 7f755f7fe640 1 -- 172.21.15.19:0/3229513144 learned_addr learned my addr 172.21.15.19:0/3229513144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.911+0000 7f755effd640 1 --2- 172.21.15.19:0/3229513144 >> v2:172.21.15.112:3300/0 conn(0x7f75601616f0 0x7f7560161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.911+0000 7f755f7fe640 1 -- 172.21.15.19:0/3229513144 >> v2:172.21.15.112:3300/0 conn(0x7f75601616f0 msgr2=0x7f7560161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.911+0000 7f755f7fe640 1 --2- 172.21.15.19:0/3229513144 >> v2:172.21.15.112:3300/0 conn(0x7f75601616f0 0x7f7560161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.911+0000 7f755f7fe640 1 -- 172.21.15.19:0/3229513144 >> v1:172.21.15.112:6789/0 conn(0x7f75600ce0a0 legacy=0x7f75600d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.911+0000 7f755f7fe640 1 -- 172.21.15.19:0/3229513144 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7560162540 con 0x7f75601610f0 2024-04-25T10:03:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.911+0000 7f755effd640 1 --2- 172.21.15.19:0/3229513144 >> v2:172.21.15.112:3300/0 conn(0x7f75601616f0 0x7f7560161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.911+0000 7f755f7fe640 1 --2- 172.21.15.19:0/3229513144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601610f0 0x7f756006c1c0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f756006f220 tx=0x7f755002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252d89400c6615ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.912+0000 7f755e7fc640 1 -- 172.21.15.19:0/3229513144 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550037070 con 0x7f75601610f0 2024-04-25T10:03:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.912+0000 7f755e7fc640 1 -- 172.21.15.19:0/3229513144 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7550032440 con 0x7f75601610f0 2024-04-25T10:03:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.912+0000 7f755e7fc640 1 -- 172.21.15.19:0/3229513144 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550032810 con 0x7f75601610f0 2024-04-25T10:03:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.912+0000 7f7566190640 1 -- 172.21.15.19:0/3229513144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601610f0 msgr2=0x7f756006c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.912+0000 7f7566190640 1 --2- 172.21.15.19:0/3229513144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601610f0 0x7f756006c1c0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f756006f220 tx=0x7f755002a230 comp rx=0 tx=0).stop 2024-04-25T10:03:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.912+0000 7f7566190640 1 -- 172.21.15.19:0/3229513144 shutdown_connections 2024-04-25T10:03:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.912+0000 7f7566190640 1 --2- 172.21.15.19:0/3229513144 >> v2:172.21.15.112:3300/0 conn(0x7f75601616f0 0x7f7560161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.912+0000 7f7566190640 1 --2- 172.21.15.19:0/3229513144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601610f0 0x7f756006c1c0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.912+0000 7f7566190640 1 -- 172.21.15.19:0/3229513144 >> 172.21.15.19:0/3229513144 conn(0x7f756006f500 msgr2=0x7f756006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.912+0000 7f7566190640 1 -- 172.21.15.19:0/3229513144 shutdown_connections 2024-04-25T10:03:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.912+0000 7f7566190640 1 -- 172.21.15.19:0/3229513144 wait complete. 2024-04-25T10:03:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.913+0000 7f7566190640 1 Processor -- start 2024-04-25T10:03:57.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.914+0000 7f7566190640 1 -- start start 2024-04-25T10:03:57.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.914+0000 7f7566190640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75600ce0a0 0x7f75600af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.914+0000 7f7566190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601610f0 0x7f7560152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.914+0000 7f7566190640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75601616f0 0x7f75600b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.914+0000 7f7566190640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75600cf690 con 0x7f75601610f0 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.914+0000 7f7566190640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75600cfce0 con 0x7f75600ce0a0 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.914+0000 7f7566190640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756006c700 con 0x7f75601616f0 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.914+0000 7f755effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601610f0 0x7f7560152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.914+0000 7f755effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601610f0 0x7f7560152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49108/0 (socket says 172.21.15.19:49108) 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.914+0000 7f755effd640 1 -- 172.21.15.19:0/845155502 learned_addr learned my addr 172.21.15.19:0/845155502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f755f7fe640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75600ce0a0 0x7f75600af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f755ffff640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75601616f0 0x7f75600b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f755effd640 1 -- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75601616f0 msgr2=0x7f75600b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f755effd640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75601616f0 0x7f75600b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f755effd640 1 -- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75600ce0a0 msgr2=0x7f75600af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f755effd640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75600ce0a0 0x7f75600af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f755effd640 1 -- 172.21.15.19:0/845155502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75600b01f0 con 0x7f75601610f0 2024-04-25T10:03:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f755effd640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601610f0 0x7f7560152bc0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f755400c7f0 tx=0x7f755400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f7547fff640 1 -- 172.21.15.19:0/845155502 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554015070 con 0x7f75601610f0 2024-04-25T10:03:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f7547fff640 1 -- 172.21.15.19:0/845155502 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7554007820 con 0x7f75601610f0 2024-04-25T10:03:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f7547fff640 1 -- 172.21.15.19:0/845155502 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554007bd0 con 0x7f75601610f0 2024-04-25T10:03:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f7566190640 1 -- 172.21.15.19:0/845155502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75600b04d0 con 0x7f75601610f0 2024-04-25T10:03:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.915+0000 7f7566190640 1 -- 172.21.15.19:0/845155502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75600b0a90 con 0x7f75601610f0 2024-04-25T10:03:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.916+0000 7f7566190640 1 -- 172.21.15.19:0/845155502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7560073a00 con 0x7f75601610f0 2024-04-25T10:03:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.917+0000 7f7547fff640 1 -- 172.21.15.19:0/845155502 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7554018070 con 0x7f75601610f0 2024-04-25T10:03:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.917+0000 7f7547fff640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f754c06b4d0 0x7f754c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.917+0000 7f7547fff640 1 -- 172.21.15.19:0/845155502 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f755408a4a0 con 0x7f75601610f0 2024-04-25T10:03:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.917+0000 7f755f7fe640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f754c06b4d0 0x7f754c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.918+0000 7f755f7fe640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f754c06b4d0 0x7f754c06d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7550034000 tx=0x7f755002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:57.922+0000 7f7547fff640 1 -- 172.21.15.19:0/845155502 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7554058ac0 con 0x7f75601610f0 2024-04-25T10:03:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.105+0000 7f7566190640 1 -- 172.21.15.19:0/845155502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75601608a0 con 0x7f75601610f0 2024-04-25T10:03:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.105+0000 7f7547fff640 1 -- 172.21.15.19:0/845155502 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v154) v1 ==== 93+0+10 (secure 0 0 0) 0x7f755405cb60 con 0x7f75601610f0 2024-04-25T10:03:58.110 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:03:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 -- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f754c06b4d0 msgr2=0x7f754c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f754c06b4d0 0x7f754c06d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7550034000 tx=0x7f755002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 -- 172.21.15.19:0/845155502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601610f0 msgr2=0x7f7560152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601610f0 0x7f7560152bc0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f755400c7f0 tx=0x7f755400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 -- 172.21.15.19:0/845155502 shutdown_connections 2024-04-25T10:03:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f754c06b4d0 0x7f754c06d990 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75601616f0 0x7f75600b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75601610f0 0x7f7560152bc0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 --2- 172.21.15.19:0/845155502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75600ce0a0 0x7f75600af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 -- 172.21.15.19:0/845155502 >> 172.21.15.19:0/845155502 conn(0x7f756006f500 msgr2=0x7f75600b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 -- 172.21.15.19:0/845155502 shutdown_connections 2024-04-25T10:03:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.107+0000 7f7566190640 1 -- 172.21.15.19:0/845155502 wait complete. 2024-04-25T10:03:58.123 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:58.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.222+0000 7f087b2fc640 1 Processor -- start 2024-04-25T10:03:58.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.222+0000 7f087b2fc640 1 -- start start 2024-04-25T10:03:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f087b2fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f087415da90 0x7f0874161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f087b2fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08740b9750 0x7f08740ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f087b2fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0874161ba0 0x7f08740cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f087b2fc640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0874162600 con 0x7f0874161ba0 2024-04-25T10:03:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f087b2fc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0874162040 con 0x7f08740b9750 2024-04-25T10:03:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f087b2fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0874162320 con 0x7f087415da90 2024-04-25T10:03:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f0879899640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0874161ba0 0x7f08740cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f0879899640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0874161ba0 0x7f08740cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49114/0 (socket says 172.21.15.19:49114) 2024-04-25T10:03:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f0879899640 1 -- 172.21.15.19:0/767887257 learned_addr learned my addr 172.21.15.19:0/767887257 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f0879098640 1 --2- 172.21.15.19:0/767887257 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f087415da90 0x7f0874161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f0879899640 1 -- 172.21.15.19:0/767887257 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f087415da90 msgr2=0x7f0874161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f0878897640 1 --2- 172.21.15.19:0/767887257 >> v2:172.21.15.112:3300/0 conn(0x7f08740b9750 0x7f08740ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f0879899640 1 --2- 172.21.15.19:0/767887257 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f087415da90 0x7f0874161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f0879899640 1 -- 172.21.15.19:0/767887257 >> v2:172.21.15.112:3300/0 conn(0x7f08740b9750 msgr2=0x7f08740ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f0879899640 1 --2- 172.21.15.19:0/767887257 >> v2:172.21.15.112:3300/0 conn(0x7f08740b9750 0x7f08740ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.223+0000 7f0879899640 1 -- 172.21.15.19:0/767887257 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740d1b10 con 0x7f0874161ba0 2024-04-25T10:03:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.224+0000 7f0879899640 1 --2- 172.21.15.19:0/767887257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0874161ba0 0x7f08740cebb0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f087406d130 tx=0x7f086c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f5fda265e737aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.224+0000 7f086bfff640 1 -- 172.21.15.19:0/767887257 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c036070 con 0x7f0874161ba0 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.224+0000 7f086bfff640 1 -- 172.21.15.19:0/767887257 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c031600 con 0x7f0874161ba0 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.224+0000 7f086bfff640 1 -- 172.21.15.19:0/767887257 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c0319d0 con 0x7f0874161ba0 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.224+0000 7f087b2fc640 1 -- 172.21.15.19:0/767887257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0874161ba0 msgr2=0x7f08740cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.224+0000 7f087b2fc640 1 --2- 172.21.15.19:0/767887257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0874161ba0 0x7f08740cebb0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f087406d130 tx=0x7f086c02a610 comp rx=0 tx=0).stop 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.225+0000 7f087b2fc640 1 -- 172.21.15.19:0/767887257 shutdown_connections 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.225+0000 7f087b2fc640 1 --2- 172.21.15.19:0/767887257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0874161ba0 0x7f08740cebb0 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.225+0000 7f087b2fc640 1 --2- 172.21.15.19:0/767887257 >> v2:172.21.15.112:3300/0 conn(0x7f08740b9750 0x7f08740ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.225+0000 7f087b2fc640 1 --2- 172.21.15.19:0/767887257 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f087415da90 0x7f0874161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.225+0000 7f087b2fc640 1 -- 172.21.15.19:0/767887257 >> 172.21.15.19:0/767887257 conn(0x7f0874061a20 msgr2=0x7f0874061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.225+0000 7f087b2fc640 1 -- 172.21.15.19:0/767887257 shutdown_connections 2024-04-25T10:03:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.225+0000 7f087b2fc640 1 -- 172.21.15.19:0/767887257 wait complete. 2024-04-25T10:03:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.226+0000 7f087b2fc640 1 Processor -- start 2024-04-25T10:03:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.226+0000 7f087b2fc640 1 -- start start 2024-04-25T10:03:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.226+0000 7f087b2fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f087415da90 0x7f08740c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f087b2fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0874161ba0 0x7f087415eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f087b2fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08740d3ef0 0x7f08740b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f087b2fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087406c120 con 0x7f0874161ba0 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f087b2fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087415de70 con 0x7f087415da90 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f087b2fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087406c2a0 con 0x7f08740d3ef0 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0878897640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0874161ba0 0x7f087415eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0878897640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0874161ba0 0x7f087415eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49126/0 (socket says 172.21.15.19:49126) 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0878897640 1 -- 172.21.15.19:0/3557243904 learned_addr learned my addr 172.21.15.19:0/3557243904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0879899640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08740d3ef0 0x7f08740b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0879098640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f087415da90 0x7f08740c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0878897640 1 -- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08740d3ef0 msgr2=0x7f08740b7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0878897640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08740d3ef0 0x7f08740b7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0878897640 1 -- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f087415da90 msgr2=0x7f08740c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0878897640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f087415da90 0x7f08740c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0878897640 1 -- 172.21.15.19:0/3557243904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08741f3910 con 0x7f0874161ba0 2024-04-25T10:03:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0878897640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0874161ba0 0x7f087415eff0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f086400c7f0 tx=0x7f086400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0869ffb640 1 -- 172.21.15.19:0/3557243904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864015070 con 0x7f0874161ba0 2024-04-25T10:03:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0869ffb640 1 -- 172.21.15.19:0/3557243904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0864004240 con 0x7f0874161ba0 2024-04-25T10:03:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f087b2fc640 1 -- 172.21.15.19:0/3557243904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08741f3bf0 con 0x7f0874161ba0 2024-04-25T10:03:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.227+0000 7f0869ffb640 1 -- 172.21.15.19:0/3557243904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864007620 con 0x7f0874161ba0 2024-04-25T10:03:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.228+0000 7f087b2fc640 1 -- 172.21.15.19:0/3557243904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08741f41b0 con 0x7f0874161ba0 2024-04-25T10:03:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.229+0000 7f087b2fc640 1 -- 172.21.15.19:0/3557243904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0874073a00 con 0x7f0874161ba0 2024-04-25T10:03:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.229+0000 7f0869ffb640 1 -- 172.21.15.19:0/3557243904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08640079d0 con 0x7f0874161ba0 2024-04-25T10:03:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.230+0000 7f0869ffb640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f085806b4d0 0x7f085806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.230+0000 7f0869ffb640 1 -- 172.21.15.19:0/3557243904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f086408a780 con 0x7f0874161ba0 2024-04-25T10:03:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.230+0000 7f0879098640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f085806b4d0 0x7f085806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.230+0000 7f0879098640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f085806b4d0 0x7f085806d990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0870006fd0 tx=0x7f0870008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.234+0000 7f0869ffb640 1 -- 172.21.15.19:0/3557243904 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0864059500 con 0x7f0874161ba0 2024-04-25T10:03:58.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.414+0000 7f087b2fc640 1 -- 172.21.15.19:0/3557243904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08740c7720 con 0x7f0874161ba0 2024-04-25T10:03:58.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.414+0000 7f0869ffb640 1 -- 172.21.15.19:0/3557243904 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v154) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0864058e00 con 0x7f0874161ba0 2024-04-25T10:03:58.419 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:03:58.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 -- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f085806b4d0 msgr2=0x7f085806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f085806b4d0 0x7f085806d990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0870006fd0 tx=0x7f0870008040 comp rx=0 tx=0).stop 2024-04-25T10:03:58.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 -- 172.21.15.19:0/3557243904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0874161ba0 msgr2=0x7f087415eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0874161ba0 0x7f087415eff0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f086400c7f0 tx=0x7f086400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 -- 172.21.15.19:0/3557243904 shutdown_connections 2024-04-25T10:03:58.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f085806b4d0 0x7f085806d990 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08740d3ef0 0x7f08740b7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0874161ba0 0x7f087415eff0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 --2- 172.21.15.19:0/3557243904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f087415da90 0x7f08740c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 -- 172.21.15.19:0/3557243904 >> 172.21.15.19:0/3557243904 conn(0x7f0874061a20 msgr2=0x7f0874061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 -- 172.21.15.19:0/3557243904 shutdown_connections 2024-04-25T10:03:58.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.416+0000 7f087b2fc640 1 -- 172.21.15.19:0/3557243904 wait complete. 2024-04-25T10:03:58.432 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.531+0000 7f389d06b640 1 Processor -- start 2024-04-25T10:03:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.531+0000 7f389d06b640 1 -- start start 2024-04-25T10:03:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389d06b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f389815dc20 0x7f38980ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389d06b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38980d32f0 0x7f38980cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389d06b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389815e000 con 0x7f3898161710 2024-04-25T10:03:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389d06b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38980d1de0 con 0x7f38980d32f0 2024-04-25T10:03:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389d06b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3898161c60 con 0x7f389815dc20 2024-04-25T10:03:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389759e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38980d32f0 0x7f38980cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389759e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38980d32f0 0x7f38980cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49142/0 (socket says 172.21.15.19:49142) 2024-04-25T10:03:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389759e640 1 -- 172.21.15.19:0/910844922 learned_addr learned my addr 172.21.15.19:0/910844922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f3896d9d640 1 --2- 172.21.15.19:0/910844922 >> v2:172.21.15.112:3300/0 conn(0x7f389815dc20 0x7f38980ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389759e640 1 -- 172.21.15.19:0/910844922 >> v2:172.21.15.112:3300/0 conn(0x7f389815dc20 msgr2=0x7f38980ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389759e640 1 --2- 172.21.15.19:0/910844922 >> v2:172.21.15.112:3300/0 conn(0x7f389815dc20 0x7f38980ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389759e640 1 -- 172.21.15.19:0/910844922 >> v1:172.21.15.112:6789/0 conn(0x7f3898161710 legacy=0x7f389806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f389759e640 1 -- 172.21.15.19:0/910844922 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3898161f40 con 0x7f38980d32f0 2024-04-25T10:03:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.532+0000 7f3896d9d640 1 --2- 172.21.15.19:0/910844922 >> v2:172.21.15.112:3300/0 conn(0x7f389815dc20 0x7f38980ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:58.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.533+0000 7f389759e640 1 --2- 172.21.15.19:0/910844922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38980d32f0 0x7f38980cebb0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f38980d1520 tx=0x7f389002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85aa71d05b84aac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.533+0000 7f389659c640 1 -- 172.21.15.19:0/910844922 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890036070 con 0x7f38980d32f0 2024-04-25T10:03:58.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.533+0000 7f389659c640 1 -- 172.21.15.19:0/910844922 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3890031600 con 0x7f38980d32f0 2024-04-25T10:03:58.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.533+0000 7f389659c640 1 -- 172.21.15.19:0/910844922 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38900319d0 con 0x7f38980d32f0 2024-04-25T10:03:58.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.534+0000 7f389d06b640 1 -- 172.21.15.19:0/910844922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38980d32f0 msgr2=0x7f38980cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.534+0000 7f389d06b640 1 --2- 172.21.15.19:0/910844922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38980d32f0 0x7f38980cebb0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f38980d1520 tx=0x7f389002a730 comp rx=0 tx=0).stop 2024-04-25T10:03:58.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.534+0000 7f389d06b640 1 -- 172.21.15.19:0/910844922 shutdown_connections 2024-04-25T10:03:58.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.534+0000 7f389d06b640 1 --2- 172.21.15.19:0/910844922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38980d32f0 0x7f38980cebb0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.534+0000 7f389d06b640 1 --2- 172.21.15.19:0/910844922 >> v2:172.21.15.112:3300/0 conn(0x7f389815dc20 0x7f38980ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.534+0000 7f389d06b640 1 -- 172.21.15.19:0/910844922 >> 172.21.15.19:0/910844922 conn(0x7f38980d0420 msgr2=0x7f38980d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.534+0000 7f389d06b640 1 -- 172.21.15.19:0/910844922 shutdown_connections 2024-04-25T10:03:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.534+0000 7f389d06b640 1 -- 172.21.15.19:0/910844922 wait complete. 2024-04-25T10:03:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.535+0000 7f389d06b640 1 Processor -- start 2024-04-25T10:03:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.535+0000 7f389d06b640 1 -- start start 2024-04-25T10:03:58.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389d06b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38980d32f0 0x7f38980cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389d06b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f389815dc20 0x7f38981f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389d06b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3898161710 0x7f38981f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389d06b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38980b5e50 con 0x7f3898161710 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389d06b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3898162500 con 0x7f38980d32f0 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389d06b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38980b5fd0 con 0x7f389815dc20 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3898161710 0x7f38981f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3898161710 0x7f38981f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49148/0 (socket says 172.21.15.19:49148) 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389759e640 1 -- 172.21.15.19:0/3735136097 learned_addr learned my addr 172.21.15.19:0/3735136097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f388ffff640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f389815dc20 0x7f38981f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f3896d9d640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38980d32f0 0x7f38980cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389759e640 1 -- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f389815dc20 msgr2=0x7f38981f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389759e640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f389815dc20 0x7f38981f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389759e640 1 -- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38980d32f0 msgr2=0x7f38980cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389759e640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38980d32f0 0x7f38980cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.536+0000 7f389759e640 1 -- 172.21.15.19:0/3735136097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389815f000 con 0x7f3898161710 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.537+0000 7f389759e640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3898161710 0x7f38981f6a80 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f3890009f40 tx=0x7f3890004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.537+0000 7f388f7fe640 1 -- 172.21.15.19:0/3735136097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890036050 con 0x7f3898161710 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.537+0000 7f388f7fe640 1 -- 172.21.15.19:0/3735136097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3890042070 con 0x7f3898161710 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.537+0000 7f389d06b640 1 -- 172.21.15.19:0/3735136097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389815f2e0 con 0x7f3898161710 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.537+0000 7f388f7fe640 1 -- 172.21.15.19:0/3735136097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890009510 con 0x7f3898161710 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.537+0000 7f389d06b640 1 -- 172.21.15.19:0/3735136097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389815f8a0 con 0x7f3898161710 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.538+0000 7f389d06b640 1 -- 172.21.15.19:0/3735136097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3898073a00 con 0x7f3898161710 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.538+0000 7f388f7fe640 1 -- 172.21.15.19:0/3735136097 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3890007bf0 con 0x7f3898161710 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.539+0000 7f388f7fe640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387806b4d0 0x7f387806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.539+0000 7f388f7fe640 1 -- 172.21.15.19:0/3735136097 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f38900ab090 con 0x7f3898161710 2024-04-25T10:03:58.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.539+0000 7f3896d9d640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387806b4d0 0x7f387806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.539+0000 7f3896d9d640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387806b4d0 0x7f387806d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f38980d1610 tx=0x7f3888008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.543+0000 7f388f7fe640 1 -- 172.21.15.19:0/3735136097 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3890045360 con 0x7f3898161710 2024-04-25T10:03:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.723+0000 7f389d06b640 1 -- 172.21.15.19:0/3735136097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38980d16b0 con 0x7f3898161710 2024-04-25T10:03:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.723+0000 7f388f7fe640 1 -- 172.21.15.19:0/3735136097 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v154) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3890079e10 con 0x7f3898161710 2024-04-25T10:03:58.728 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:03:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 -- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387806b4d0 msgr2=0x7f387806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387806b4d0 0x7f387806d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f38980d1610 tx=0x7f3888008040 comp rx=0 tx=0).stop 2024-04-25T10:03:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 -- 172.21.15.19:0/3735136097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3898161710 msgr2=0x7f38981f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3898161710 0x7f38981f6a80 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f3890009f40 tx=0x7f3890004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 -- 172.21.15.19:0/3735136097 shutdown_connections 2024-04-25T10:03:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f387806b4d0 0x7f387806d990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3898161710 0x7f38981f6a80 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f389815dc20 0x7f38981f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 --2- 172.21.15.19:0/3735136097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38980d32f0 0x7f38980cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 -- 172.21.15.19:0/3735136097 >> 172.21.15.19:0/3735136097 conn(0x7f38980d0420 msgr2=0x7f38980d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 -- 172.21.15.19:0/3735136097 shutdown_connections 2024-04-25T10:03:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.725+0000 7f389d06b640 1 -- 172.21.15.19:0/3735136097 wait complete. 2024-04-25T10:03:58.741 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:58.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.838+0000 7f14465c8640 1 Processor -- start 2024-04-25T10:03:58.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.838+0000 7f14465c8640 1 -- start start 2024-04-25T10:03:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f14465c8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14400b9750 0x7f14400ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f14465c8640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1440161d70 con 0x7f14400cebb0 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f14465c8640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14400cf020 con 0x7f144015da90 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f14465c8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1440162600 con 0x7f14400b9750 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f1444b65640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f14400cebb0 0x7f14400d2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56948/0 (socket says 172.21.15.19:56948) 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f1444b65640 1 -- 172.21.15.19:0/4286678960 learned_addr learned my addr 172.21.15.19:0/4286678960 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143f7fe640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14400b9750 0x7f14400ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3054393053 0 0) 0x7f1440161d70 con 0x7f14400cebb0 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1428003520 con 0x7f14400cebb0 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4289482905 0 0) 0x7f1428003520 con 0x7f14400cebb0 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14400b9750 msgr2=0x7f14400ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14400b9750 0x7f14400ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 >> v1:172.21.15.112:6789/0 conn(0x7f144015da90 legacy=0x7f1440161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440162040 con 0x7f14400cebb0 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f14380019f0 con 0x7f14400cebb0 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 >> v1:172.21.15.19:6789/0 conn(0x7f14400cebb0 legacy=0x7f14400d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14280039e0 0x7f1428003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1428004340 0x7f1428004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1428005020 0x7f142800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1440162600 con 0x7f1428005020 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142800bf30 con 0x7f1428004340 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.839+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142800c130 con 0x7f14280039e0 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.840+0000 7f1444b65640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1428005020 0x7f142800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.840+0000 7f143ffff640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14280039e0 0x7f1428003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.840+0000 7f143f7fe640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1428004340 0x7f1428004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.840+0000 7f1444b65640 1 -- 172.21.15.19:0/4286678960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14280039e0 msgr2=0x7f1428003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.840+0000 7f1444b65640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14280039e0 0x7f1428003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.840+0000 7f1444b65640 1 -- 172.21.15.19:0/4286678960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1428004340 msgr2=0x7f1428004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.840+0000 7f1444b65640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1428004340 0x7f1428004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.840+0000 7f1444b65640 1 -- 172.21.15.19:0/4286678960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1428003520 con 0x7f1428005020 2024-04-25T10:03:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.841+0000 7f1444b65640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1428005020 0x7f142800b9c0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f143800a8d0 tx=0x7f1428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15255e203bf82e3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.841+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800a4a0 con 0x7f1428005020 2024-04-25T10:03:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.841+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143802f070 con 0x7f1428005020 2024-04-25T10:03:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.841+0000 7f143effd640 1 -- 172.21.15.19:0/4286678960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143802bcb0 con 0x7f1428005020 2024-04-25T10:03:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.841+0000 7f14465c8640 1 -- 172.21.15.19:0/4286678960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1428005020 msgr2=0x7f142800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.841+0000 7f14465c8640 1 --2- 172.21.15.19:0/4286678960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1428005020 0x7f142800b9c0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f143800a8d0 tx=0x7f1428005830 comp rx=0 tx=0).stop 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.841+0000 7f1444b65640 1 -- 172.21.15.19:0/4286678960 reap_dead start 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.841+0000 7f14465c8640 1 -- 172.21.15.19:0/4286678960 shutdown_connections 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.841+0000 7f14465c8640 1 -- 172.21.15.19:0/4286678960 >> 172.21.15.19:0/4286678960 conn(0x7f1440061a20 msgr2=0x7f1440061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.841+0000 7f14465c8640 1 -- 172.21.15.19:0/4286678960 shutdown_connections 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.841+0000 7f14465c8640 1 -- 172.21.15.19:0/4286678960 wait complete. 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.842+0000 7f14465c8640 1 Processor -- start 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.843+0000 7f14465c8640 1 -- start start 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.843+0000 7f14465c8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1428005020 0x7f14400c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.843+0000 7f14465c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14401f6960 0x7f14401f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.843+0000 7f14465c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14401f7320 0x7f14401f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.843+0000 7f14465c8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14400d1df0 con 0x7f14401f7320 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.843+0000 7f14465c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14400d1b10 con 0x7f14401f6960 2024-04-25T10:03:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.843+0000 7f14465c8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144015de70 con 0x7f1428005020 2024-04-25T10:03:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.843+0000 7f1444b65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14401f7320 0x7f14401f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.843+0000 7f1444b65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14401f7320 0x7f14401f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49166/0 (socket says 172.21.15.19:49166) 2024-04-25T10:03:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.843+0000 7f1444b65640 1 -- 172.21.15.19:0/3640089292 learned_addr learned my addr 172.21.15.19:0/3640089292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f143ffff640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1428005020 0x7f14400c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f143f7fe640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14401f6960 0x7f14401f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f1444b65640 1 -- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1428005020 msgr2=0x7f14400c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f1444b65640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1428005020 0x7f14400c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f1444b65640 1 -- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14401f6960 msgr2=0x7f14401f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f1444b65640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14401f6960 0x7f14401f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f1444b65640 1 -- 172.21.15.19:0/3640089292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14401f4640 con 0x7f14401f7320 2024-04-25T10:03:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f143f7fe640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14401f6960 0x7f14401f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f1444b65640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14401f7320 0x7f14401f4b20 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f1438001cf0 tx=0x7f1438001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f143cff9640 1 -- 172.21.15.19:0/3640089292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800a4a0 con 0x7f14401f7320 2024-04-25T10:03:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f143cff9640 1 -- 172.21.15.19:0/3640089292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1438001a20 con 0x7f14401f7320 2024-04-25T10:03:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f143cff9640 1 -- 172.21.15.19:0/3640089292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438002000 con 0x7f14401f7320 2024-04-25T10:03:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f14465c8640 1 -- 172.21.15.19:0/3640089292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14400c8660 con 0x7f14401f7320 2024-04-25T10:03:58.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.844+0000 7f14465c8640 1 -- 172.21.15.19:0/3640089292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14400c8c20 con 0x7f14401f7320 2024-04-25T10:03:58.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.845+0000 7f14465c8640 1 -- 172.21.15.19:0/3640089292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14401f4810 con 0x7f14401f7320 2024-04-25T10:03:58.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.846+0000 7f143cff9640 1 -- 172.21.15.19:0/3640089292 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1438046060 con 0x7f14401f7320 2024-04-25T10:03:58.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.846+0000 7f143cff9640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f142c06b500 0x7f142c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.846+0000 7f143cff9640 1 -- 172.21.15.19:0/3640089292 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14380aa1d0 con 0x7f14401f7320 2024-04-25T10:03:58.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.846+0000 7f143ffff640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f142c06b500 0x7f142c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.847+0000 7f143ffff640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f142c06b500 0x7f142c06d9c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f14400b7fe0 tx=0x7f1430009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:58.850+0000 7f143cff9640 1 -- 172.21.15.19:0/3640089292 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1438078f50 con 0x7f14401f7320 2024-04-25T10:03:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.042+0000 7f14465c8640 1 -- 172.21.15.19:0/3640089292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1440073a00 con 0x7f14401f7320 2024-04-25T10:03:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.042+0000 7f143cff9640 1 -- 172.21.15.19:0/3640089292 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v154) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1438078850 con 0x7f14401f7320 2024-04-25T10:03:59.047 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.044+0000 7f14465c8640 1 -- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f142c06b500 msgr2=0x7f142c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.044+0000 7f14465c8640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f142c06b500 0x7f142c06d9c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f14400b7fe0 tx=0x7f1430009040 comp rx=0 tx=0).stop 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.044+0000 7f14465c8640 1 -- 172.21.15.19:0/3640089292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14401f7320 msgr2=0x7f14401f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.044+0000 7f14465c8640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14401f7320 0x7f14401f4b20 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f1438001cf0 tx=0x7f1438001360 comp rx=0 tx=0).stop 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.044+0000 7f14465c8640 1 -- 172.21.15.19:0/3640089292 shutdown_connections 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.044+0000 7f14465c8640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f142c06b500 0x7f142c06d9c0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.044+0000 7f14465c8640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14401f7320 0x7f14401f4b20 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.044+0000 7f14465c8640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14401f6960 0x7f14401f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.044+0000 7f14465c8640 1 --2- 172.21.15.19:0/3640089292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1428005020 0x7f14400c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.044+0000 7f14465c8640 1 -- 172.21.15.19:0/3640089292 >> 172.21.15.19:0/3640089292 conn(0x7f1440061a20 msgr2=0x7f14400d3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.045+0000 7f14465c8640 1 -- 172.21.15.19:0/3640089292 shutdown_connections 2024-04-25T10:03:59.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.045+0000 7f14465c8640 1 -- 172.21.15.19:0/3640089292 wait complete. 2024-04-25T10:03:59.060 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:03:59.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:59.061 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:59.088 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:03:59.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:59.089 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:03:59.144 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:03:59.144 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:59.163 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:59.163 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:03:59.163 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:59.200 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:03:59.200 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:03:59.200 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:59.220 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:59.221 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.335+0000 7fc0735f9640 1 Processor -- start 2024-04-25T10:03:59.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.335+0000 7fc0735f9640 1 -- start start 2024-04-25T10:03:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc0735f9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc06c15dd30 0x7fc06c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc0735f9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc06c0b9770 0x7fc06c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc0735f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc06c0d2e90 0x7fc06c0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc0735f9640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06c0d1df0 con 0x7fc06c0b9770 2024-04-25T10:03:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc0735f9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06c0d20d0 con 0x7fc06c15dd30 2024-04-25T10:03:59.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc0735f9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0d23b0 con 0x7fc06c0d2e90 2024-04-25T10:03:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc070b94640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc06c0b9770 0x7fc06c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc070b94640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc06c0b9770 0x7fc06c0d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49178/0 (socket says 172.21.15.19:49178) 2024-04-25T10:03:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc070b94640 1 -- 172.21.15.19:0/1162701944 learned_addr learned my addr 172.21.15.19:0/1162701944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc070b94640 1 -- 172.21.15.19:0/1162701944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc06c0d2e90 msgr2=0x7fc06c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc070b94640 1 --2- 172.21.15.19:0/1162701944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc06c0d2e90 0x7fc06c0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc070b94640 1 -- 172.21.15.19:0/1162701944 >> v2:172.21.15.112:3300/0 conn(0x7fc06c15dd30 msgr2=0x7fc06c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc070b94640 1 --2- 172.21.15.19:0/1162701944 >> v2:172.21.15.112:3300/0 conn(0x7fc06c15dd30 0x7fc06c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.336+0000 7fc070b94640 1 -- 172.21.15.19:0/1162701944 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c0d2690 con 0x7fc06c0b9770 2024-04-25T10:03:59.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.337+0000 7fc070b94640 1 --2- 172.21.15.19:0/1162701944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc06c0b9770 0x7fc06c0d03c0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fc06c06d000 tx=0x7fc05c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de3ded0fee524ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.337+0000 7fc063fff640 1 -- 172.21.15.19:0/1162701944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c004030 con 0x7fc06c0b9770 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.337+0000 7fc063fff640 1 -- 172.21.15.19:0/1162701944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c0041d0 con 0x7fc06c0b9770 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.337+0000 7fc063fff640 1 -- 172.21.15.19:0/1162701944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c003610 con 0x7fc06c0b9770 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.337+0000 7fc0735f9640 1 -- 172.21.15.19:0/1162701944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc06c0b9770 msgr2=0x7fc06c0d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.337+0000 7fc0735f9640 1 --2- 172.21.15.19:0/1162701944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc06c0b9770 0x7fc06c0d03c0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fc06c06d000 tx=0x7fc05c02a130 comp rx=0 tx=0).stop 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.338+0000 7fc0735f9640 1 -- 172.21.15.19:0/1162701944 shutdown_connections 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.338+0000 7fc0735f9640 1 --2- 172.21.15.19:0/1162701944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc06c0d2e90 0x7fc06c0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.338+0000 7fc0735f9640 1 --2- 172.21.15.19:0/1162701944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc06c0b9770 0x7fc06c0d03c0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.338+0000 7fc0735f9640 1 --2- 172.21.15.19:0/1162701944 >> v2:172.21.15.112:3300/0 conn(0x7fc06c15dd30 0x7fc06c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.338+0000 7fc0735f9640 1 -- 172.21.15.19:0/1162701944 >> 172.21.15.19:0/1162701944 conn(0x7fc06c061c30 msgr2=0x7fc06c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.338+0000 7fc0735f9640 1 -- 172.21.15.19:0/1162701944 shutdown_connections 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.338+0000 7fc0735f9640 1 -- 172.21.15.19:0/1162701944 wait complete. 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.339+0000 7fc0735f9640 1 Processor -- start 2024-04-25T10:03:59.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.339+0000 7fc0735f9640 1 -- start start 2024-04-25T10:03:59.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc0735f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc06c0b9770 0x7fc06c1f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc0735f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc06c0d2e90 0x7fc06c1f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc0735f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc06c15dd30 0x7fc06c0c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc0735f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0d1b90 con 0x7fc06c15dd30 2024-04-25T10:03:59.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc0735f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0d1a10 con 0x7fc06c0d2e90 2024-04-25T10:03:59.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc0735f9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0b5430 con 0x7fc06c0b9770 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc071b96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc06c15dd30 0x7fc06c0c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc071b96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc06c15dd30 0x7fc06c0c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49188/0 (socket says 172.21.15.19:49188) 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc071b96640 1 -- 172.21.15.19:0/3176728640 learned_addr learned my addr 172.21.15.19:0/3176728640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc070b94640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc06c0d2e90 0x7fc06c1f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc071395640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc06c0b9770 0x7fc06c1f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc071b96640 1 -- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc06c0b9770 msgr2=0x7fc06c1f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc071b96640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc06c0b9770 0x7fc06c1f33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc071b96640 1 -- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc06c0d2e90 msgr2=0x7fc06c1f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc071b96640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc06c0d2e90 0x7fc06c1f7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc071b96640 1 -- 172.21.15.19:0/3176728640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c0c8760 con 0x7fc06c15dd30 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc071b96640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc06c15dd30 0x7fc06c0c9800 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fc06400b970 tx=0x7fc06400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc061ffb640 1 -- 172.21.15.19:0/3176728640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064019070 con 0x7fc06c15dd30 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc061ffb640 1 -- 172.21.15.19:0/3176728640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06400cb60 con 0x7fc06c15dd30 2024-04-25T10:03:59.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc0735f9640 1 -- 172.21.15.19:0/3176728640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06c0c8a40 con 0x7fc06c15dd30 2024-04-25T10:03:59.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.340+0000 7fc061ffb640 1 -- 172.21.15.19:0/3176728640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064014440 con 0x7fc06c15dd30 2024-04-25T10:03:59.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.341+0000 7fc0735f9640 1 -- 172.21.15.19:0/3176728640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06c0c9000 con 0x7fc06c15dd30 2024-04-25T10:03:59.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.342+0000 7fc0735f9640 1 -- 172.21.15.19:0/3176728640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06c073990 con 0x7fc06c15dd30 2024-04-25T10:03:59.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.342+0000 7fc061ffb640 1 -- 172.21.15.19:0/3176728640 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc06401c070 con 0x7fc06c15dd30 2024-04-25T10:03:59.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.342+0000 7fc061ffb640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc05006b4d0 0x7fc05006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.343+0000 7fc061ffb640 1 -- 172.21.15.19:0/3176728640 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc06408a0d0 con 0x7fc06c15dd30 2024-04-25T10:03:59.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.343+0000 7fc071395640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc05006b4d0 0x7fc05006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.343+0000 7fc071395640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc05006b4d0 0x7fc05006d990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc06800a770 tx=0x7fc068008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.347+0000 7fc061ffb640 1 -- 172.21.15.19:0/3176728640 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc064058dc0 con 0x7fc06c15dd30 2024-04-25T10:03:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.563+0000 7fc0735f9640 1 -- 172.21.15.19:0/3176728640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc06c0b5fb0 con 0x7fc06c15dd30 2024-04-25T10:03:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.564+0000 7fc061ffb640 1 -- 172.21.15.19:0/3176728640 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4951 (secure 0 0 0) 0x7fc0640586c0 con 0x7fc06c15dd30 2024-04-25T10:03:59.569 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:59.569 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":102,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3725393673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3725393673},{"type":"v1","addr":"172.21.15.112:6837","nonce":3725393673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8866,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2539453196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2539453196},{"type":"v1","addr":"172.21.15.112:6836","nonce":2539453196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10221,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3896097924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3896097924},{"type":"v1","addr":"172.21.15.19:6834","nonce":3896097924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:14.030067+0000","modified":"2024-04-25T10:03:17.045113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10224":{"gid":10224,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/717842662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":717842662},{"type":"v1","addr":"172.21.15.19:6835","nonce":717842662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:03:59.569 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 102 2024-04-25T10:03:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 -- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc05006b4d0 msgr2=0x7fc05006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc05006b4d0 0x7fc05006d990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc06800a770 tx=0x7fc068008040 comp rx=0 tx=0).stop 2024-04-25T10:03:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 -- 172.21.15.19:0/3176728640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc06c15dd30 msgr2=0x7fc06c0c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc06c15dd30 0x7fc06c0c9800 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fc06400b970 tx=0x7fc06400be30 comp rx=0 tx=0).stop 2024-04-25T10:03:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 -- 172.21.15.19:0/3176728640 shutdown_connections 2024-04-25T10:03:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc05006b4d0 0x7fc05006d990 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc06c15dd30 0x7fc06c0c9800 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc06c0d2e90 0x7fc06c1f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 --2- 172.21.15.19:0/3176728640 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc06c0b9770 0x7fc06c1f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 -- 172.21.15.19:0/3176728640 >> 172.21.15.19:0/3176728640 conn(0x7fc06c061c30 msgr2=0x7fc06c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 -- 172.21.15.19:0/3176728640 shutdown_connections 2024-04-25T10:03:59.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.566+0000 7fc0735f9640 1 -- 172.21.15.19:0/3176728640 wait complete. 2024-04-25T10:03:59.582 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:59.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.678+0000 7fd87b66f640 1 Processor -- start 2024-04-25T10:03:59.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.678+0000 7fd87b66f640 1 -- start start 2024-04-25T10:03:59.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.679+0000 7fd87b66f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd8740b96d0 0x7fd8740d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.679+0000 7fd87b66f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd8740d0fe0 0x7fd8740d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.679+0000 7fd87b66f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87415e810 con 0x7fd87415dd20 2024-04-25T10:03:59.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.679+0000 7fd87b66f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8740d1c00 con 0x7fd8740b96d0 2024-04-25T10:03:59.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.679+0000 7fd87b66f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8740d1420 con 0x7fd8740d0fe0 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.679+0000 7fd878c0a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd8740b96d0 0x7fd8740d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.679+0000 7fd878c0a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd8740b96d0 0x7fd8740d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49204/0 (socket says 172.21.15.19:49204) 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.679+0000 7fd87940b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd87415dd20 0x7fd87415e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56954/0 (socket says 172.21.15.19:56954) 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.679+0000 7fd878c0a640 1 -- 172.21.15.19:0/1650984244 learned_addr learned my addr 172.21.15.19:0/1650984244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.679+0000 7fd879c0c640 1 --2- 172.21.15.19:0/1650984244 >> v2:172.21.15.112:3300/0 conn(0x7fd8740d0fe0 0x7fd8740d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.680+0000 7fd878c0a640 1 -- 172.21.15.19:0/1650984244 >> v2:172.21.15.112:3300/0 conn(0x7fd8740d0fe0 msgr2=0x7fd8740d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.680+0000 7fd878c0a640 1 --2- 172.21.15.19:0/1650984244 >> v2:172.21.15.112:3300/0 conn(0x7fd8740d0fe0 0x7fd8740d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.680+0000 7fd878c0a640 1 -- 172.21.15.19:0/1650984244 >> v1:172.21.15.19:6789/0 conn(0x7fd87415dd20 legacy=0x7fd87415e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.680+0000 7fd873fff640 1 -- 172.21.15.19:0/1650984244 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644618807 0 0) 0x7fd87415e810 con 0x7fd87415dd20 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.680+0000 7fd878c0a640 1 -- 172.21.15.19:0/1650984244 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87415eac0 con 0x7fd8740b96d0 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.680+0000 7fd879c0c640 1 --2- 172.21.15.19:0/1650984244 >> v2:172.21.15.112:3300/0 conn(0x7fd8740d0fe0 0x7fd8740d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.680+0000 7fd878c0a640 1 --2- 172.21.15.19:0/1650984244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8740b96d0 0x7fd8740d0aa0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fd8740d1990 tx=0x7fd86802a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38d8e7956d72795 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.680+0000 7fd873fff640 1 -- 172.21.15.19:0/1650984244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868037070 con 0x7fd8740b96d0 2024-04-25T10:03:59.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.680+0000 7fd873fff640 1 -- 172.21.15.19:0/1650984244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd868032440 con 0x7fd8740b96d0 2024-04-25T10:03:59.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.680+0000 7fd873fff640 1 -- 172.21.15.19:0/1650984244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868032810 con 0x7fd8740b96d0 2024-04-25T10:03:59.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.681+0000 7fd87b66f640 1 -- 172.21.15.19:0/1650984244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8740b96d0 msgr2=0x7fd8740d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.681+0000 7fd87b66f640 1 --2- 172.21.15.19:0/1650984244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8740b96d0 0x7fd8740d0aa0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fd8740d1990 tx=0x7fd86802a1f0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.681+0000 7fd87b66f640 1 -- 172.21.15.19:0/1650984244 shutdown_connections 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.681+0000 7fd87b66f640 1 --2- 172.21.15.19:0/1650984244 >> v2:172.21.15.112:3300/0 conn(0x7fd8740d0fe0 0x7fd8740d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.681+0000 7fd87b66f640 1 --2- 172.21.15.19:0/1650984244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8740b96d0 0x7fd8740d0aa0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.681+0000 7fd87b66f640 1 -- 172.21.15.19:0/1650984244 >> 172.21.15.19:0/1650984244 conn(0x7fd874061a20 msgr2=0x7fd874061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.681+0000 7fd87b66f640 1 -- 172.21.15.19:0/1650984244 shutdown_connections 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.681+0000 7fd87b66f640 1 -- 172.21.15.19:0/1650984244 wait complete. 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.682+0000 7fd87b66f640 1 Processor -- start 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.682+0000 7fd87b66f640 1 -- start start 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd87b66f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8740b96d0 0x7fd8740cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd87b66f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8740d0fe0 0x7fd8740c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd87b66f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd87415dd20 0x7fd874151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd87b66f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8740d1e50 con 0x7fd87415dd20 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd87b66f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87415f080 con 0x7fd8740b96d0 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd87b66f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8740d28b0 con 0x7fd8740d0fe0 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd879c0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd87415dd20 0x7fd874151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd879c0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd87415dd20 0x7fd874151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49210/0 (socket says 172.21.15.19:49210) 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd879c0c640 1 -- 172.21.15.19:0/4282103166 learned_addr learned my addr 172.21.15.19:0/4282103166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd879c0c640 1 -- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8740d0fe0 msgr2=0x7fd8740c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd879c0c640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8740d0fe0 0x7fd8740c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd87940b640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8740b96d0 0x7fd8740cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.683+0000 7fd879c0c640 1 -- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8740b96d0 msgr2=0x7fd8740cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.684+0000 7fd879c0c640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8740b96d0 0x7fd8740cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.684+0000 7fd879c0c640 1 -- 172.21.15.19:0/4282103166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8740c8e10 con 0x7fd87415dd20 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.684+0000 7fd879c0c640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd87415dd20 0x7fd874151fd0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fd86400cef0 tx=0x7fd86400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.684+0000 7fd871ffb640 1 -- 172.21.15.19:0/4282103166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86401a070 con 0x7fd87415dd20 2024-04-25T10:03:59.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.684+0000 7fd871ffb640 1 -- 172.21.15.19:0/4282103166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86400d860 con 0x7fd87415dd20 2024-04-25T10:03:59.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.684+0000 7fd871ffb640 1 -- 172.21.15.19:0/4282103166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86400dc50 con 0x7fd87415dd20 2024-04-25T10:03:59.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.684+0000 7fd87b66f640 1 -- 172.21.15.19:0/4282103166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8740c7d70 con 0x7fd87415dd20 2024-04-25T10:03:59.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.684+0000 7fd87b66f640 1 -- 172.21.15.19:0/4282103166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8740c8330 con 0x7fd87415dd20 2024-04-25T10:03:59.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.685+0000 7fd87b66f640 1 -- 172.21.15.19:0/4282103166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd874073990 con 0x7fd87415dd20 2024-04-25T10:03:59.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.685+0000 7fd871ffb640 1 -- 172.21.15.19:0/4282103166 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd86401d070 con 0x7fd87415dd20 2024-04-25T10:03:59.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.686+0000 7fd871ffb640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd86006b500 0x7fd86006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.686+0000 7fd871ffb640 1 -- 172.21.15.19:0/4282103166 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd86408ab20 con 0x7fd87415dd20 2024-04-25T10:03:59.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.686+0000 7fd87940b640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd86006b500 0x7fd86006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.687+0000 7fd87940b640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd86006b500 0x7fd86006d9c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd86c000f60 tx=0x7fd86c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.690+0000 7fd871ffb640 1 -- 172.21.15.19:0/4282103166 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8640598a0 con 0x7fd87415dd20 2024-04-25T10:03:59.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.870+0000 7fd87b66f640 1 -- 172.21.15.19:0/4282103166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd87415f770 con 0x7fd87415dd20 2024-04-25T10:03:59.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.871+0000 7fd871ffb640 1 -- 172.21.15.19:0/4282103166 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+16872 (secure 0 0 0) 0x7fd8640591a0 con 0x7fd87415dd20 2024-04-25T10:03:59.876 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:03:59.876 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":154,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:03:15.658204+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:12.307052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:13.005066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:59.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 -- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd86006b500 msgr2=0x7fd86006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd86006b500 0x7fd86006d9c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd86c000f60 tx=0x7fd86c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:59.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 -- 172.21.15.19:0/4282103166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd87415dd20 msgr2=0x7fd874151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd87415dd20 0x7fd874151fd0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fd86400cef0 tx=0x7fd86400a710 comp rx=0 tx=0).stop 2024-04-25T10:03:59.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 -- 172.21.15.19:0/4282103166 shutdown_connections 2024-04-25T10:03:59.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd86006b500 0x7fd86006d9c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd87415dd20 0x7fd874151fd0 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8740d0fe0 0x7fd8740c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 --2- 172.21.15.19:0/4282103166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8740b96d0 0x7fd8740cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 -- 172.21.15.19:0/4282103166 >> 172.21.15.19:0/4282103166 conn(0x7fd874061a20 msgr2=0x7fd8740ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 -- 172.21.15.19:0/4282103166 shutdown_connections 2024-04-25T10:03:59.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.873+0000 7fd87b66f640 1 -- 172.21.15.19:0/4282103166 wait complete. 2024-04-25T10:03:59.889 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:12.307052+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:13.005066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:59.890 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:59.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.988+0000 7f8ef895c640 1 Processor -- start 2024-04-25T10:03:59.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.989+0000 7f8ef895c640 1 -- start start 2024-04-25T10:03:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.989+0000 7f8ef895c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef415b980 0x7f8ef40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.989+0000 7f8ef895c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef4072280 con 0x7f8ef40cc4f0 2024-04-25T10:03:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.989+0000 7f8ef895c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef40714c0 con 0x7f8ef40ce230 2024-04-25T10:03:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.989+0000 7f8ef895c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef415bdc0 con 0x7f8ef415b980 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.989+0000 7f8ef259c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8ef40cc4f0 0x7f8ef40cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56966/0 (socket says 172.21.15.19:56966) 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.989+0000 7f8ef259c640 1 -- 172.21.15.19:0/358141707 learned_addr learned my addr 172.21.15.19:0/358141707 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef2d9d640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef415b980 0x7f8ef40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125431599 0 0) 0x7f8ef4072280 con 0x7f8ef40cc4f0 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ee0003520 con 0x7f8ef40cc4f0 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509309218 0 0) 0x7f8ef40714c0 con 0x7f8ef40ce230 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ef4072280 con 0x7f8ef40ce230 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 261268365 0 0) 0x7f8ee0003520 con 0x7f8ef40cc4f0 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef415b980 msgr2=0x7f8ef40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef415b980 0x7f8ef40cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 >> v1:172.21.15.112:6789/0 conn(0x7f8ef40ce230 legacy=0x7f8ef415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40717a0 con 0x7f8ef40cc4f0 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8ee4001670 con 0x7f8ef40cc4f0 2024-04-25T10:03:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 >> v1:172.21.15.19:6789/0 conn(0x7f8ef40cc4f0 legacy=0x7f8ef40cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 0x7f8ee0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee0004800 0x7f8ee0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef415b980 0x7f8ee00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef415bdc0 con 0x7f8ee0003b70 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee000dc80 con 0x7f8ee0004800 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee000de80 con 0x7f8ef415b980 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef259c640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 0x7f8ee0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8eebfff640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee0004800 0x7f8ee0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef2d9d640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef415b980 0x7f8ee00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.990+0000 7f8ef259c640 1 -- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef415b980 msgr2=0x7f8ee00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.991+0000 7f8ef259c640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef415b980 0x7f8ee00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.991+0000 7f8ef259c640 1 -- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee0004800 msgr2=0x7f8ee0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.991+0000 7f8ef259c640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee0004800 0x7f8ee0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.991+0000 7f8ef259c640 1 -- 172.21.15.19:0/358141707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee0003520 con 0x7f8ee0003b70 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.991+0000 7f8ef2d9d640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef415b980 0x7f8ee00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.991+0000 7f8eebfff640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee0004800 0x7f8ee0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.991+0000 7f8ef259c640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 0x7f8ee0003f70 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c0e0 tx=0x7f8ee0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756091eed8993cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.991+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4007c00 con 0x7f8ee0003b70 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.991+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee4005ad0 con 0x7f8ee0003b70 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.991+0000 7f8ef1d9b640 1 -- 172.21.15.19:0/358141707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4007070 con 0x7f8ee0003b70 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.992+0000 7f8ef895c640 1 -- 172.21.15.19:0/358141707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 msgr2=0x7f8ee0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.992+0000 7f8ef895c640 1 --2- 172.21.15.19:0/358141707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 0x7f8ee0003f70 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c0e0 tx=0x7f8ee0005830 comp rx=0 tx=0).stop 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.992+0000 7f8ef2d9d640 1 -- 172.21.15.19:0/358141707 reap_dead start 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.992+0000 7f8ef895c640 1 -- 172.21.15.19:0/358141707 shutdown_connections 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.992+0000 7f8ef895c640 1 -- 172.21.15.19:0/358141707 >> 172.21.15.19:0/358141707 conn(0x7f8ef406abe0 msgr2=0x7f8ef406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.992+0000 7f8ef895c640 1 -- 172.21.15.19:0/358141707 shutdown_connections 2024-04-25T10:04:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.992+0000 7f8ef895c640 1 -- 172.21.15.19:0/358141707 wait complete. 2024-04-25T10:04:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.993+0000 7f8ef895c640 1 Processor -- start 2024-04-25T10:04:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.993+0000 7f8ef895c640 1 -- start start 2024-04-25T10:04:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef895c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 0x7f8ef4157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef895c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef41580c0 0x7f8ef41e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef895c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ef41e46f0 0x7f8ef41e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef895c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4071d60 con 0x7f8ee0003b70 2024-04-25T10:04:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef895c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4071270 con 0x7f8ef41e46f0 2024-04-25T10:04:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef895c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef406e8a0 con 0x7f8ef41580c0 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 0x7f8ef4157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 0x7f8ef4157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49238/0 (socket says 172.21.15.19:49238) 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef259c640 1 -- 172.21.15.19:0/4252079775 learned_addr learned my addr 172.21.15.19:0/4252079775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef259c640 1 -- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef41580c0 msgr2=0x7f8ef41e41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef259c640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef41580c0 0x7f8ef41e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef259c640 1 -- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ef41e46f0 msgr2=0x7f8ef41e4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.994+0000 7f8ef2d9d640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ef41e46f0 0x7f8ef41e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.995+0000 7f8ef259c640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ef41e46f0 0x7f8ef41e4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.995+0000 7f8ef259c640 1 -- 172.21.15.19:0/4252079775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef414eac0 con 0x7f8ee0003b70 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.995+0000 7f8ef259c640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 0x7f8ef4157b80 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4000f60 tx=0x7f8ef40cf190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.995+0000 7f8eeaffd640 1 -- 172.21.15.19:0/4252079775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4041020 con 0x7f8ee0003b70 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.995+0000 7f8eeaffd640 1 -- 172.21.15.19:0/4252079775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee4001470 con 0x7f8ee0003b70 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.995+0000 7f8ef895c640 1 -- 172.21.15.19:0/4252079775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef41556c0 con 0x7f8ee0003b70 2024-04-25T10:04:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.995+0000 7f8eeaffd640 1 -- 172.21.15.19:0/4252079775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4007070 con 0x7f8ee0003b70 2024-04-25T10:04:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.995+0000 7f8ef895c640 1 -- 172.21.15.19:0/4252079775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef4155c80 con 0x7f8ee0003b70 2024-04-25T10:04:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.996+0000 7f8ef895c640 1 -- 172.21.15.19:0/4252079775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef40ce610 con 0x7f8ee0003b70 2024-04-25T10:04:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.996+0000 7f8eeaffd640 1 -- 172.21.15.19:0/4252079775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ee4044020 con 0x7f8ee0003b70 2024-04-25T10:04:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.997+0000 7f8eeaffd640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ed406b4d0 0x7f8ed406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.997+0000 7f8eeaffd640 1 -- 172.21.15.19:0/4252079775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ee40aa720 con 0x7f8ee0003b70 2024-04-25T10:04:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.997+0000 7f8eebfff640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ed406b4d0 0x7f8ed406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:03:59.998+0000 7f8eebfff640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ed406b4d0 0x7f8ed406d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40b1b70 tx=0x7f8edc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.001+0000 7f8eeaffd640 1 -- 172.21.15.19:0/4252079775 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ef40ce610 con 0x7f8ee0003b70 2024-04-25T10:04:00.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.179+0000 7f8ef895c640 1 -- 172.21.15.19:0/4252079775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ef4073a00 con 0x7f8ee0003b70 2024-04-25T10:04:00.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.179+0000 7f8eeaffd640 1 -- 172.21.15.19:0/4252079775 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v154) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ee4005aa0 con 0x7f8ee0003b70 2024-04-25T10:04:00.185 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:04:00.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 -- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ed406b4d0 msgr2=0x7f8ed406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ed406b4d0 0x7f8ed406d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40b1b70 tx=0x7f8edc009040 comp rx=0 tx=0).stop 2024-04-25T10:04:00.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 -- 172.21.15.19:0/4252079775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 msgr2=0x7f8ef4157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 0x7f8ef4157b80 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4000f60 tx=0x7f8ef40cf190 comp rx=0 tx=0).stop 2024-04-25T10:04:00.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 -- 172.21.15.19:0/4252079775 shutdown_connections 2024-04-25T10:04:00.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8ed406b4d0 0x7f8ed406d990 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ef41e46f0 0x7f8ef41e4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ef41580c0 0x7f8ef41e41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 --2- 172.21.15.19:0/4252079775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ee0003b70 0x7f8ef4157b80 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 -- 172.21.15.19:0/4252079775 >> 172.21.15.19:0/4252079775 conn(0x7f8ef406abe0 msgr2=0x7f8ef406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:00.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 -- 172.21.15.19:0/4252079775 shutdown_connections 2024-04-25T10:04:00.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.181+0000 7f8ef895c640 1 -- 172.21.15.19:0/4252079775 wait complete. 2024-04-25T10:04:00.197 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:04:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.296+0000 7fc617acb640 1 Processor -- start 2024-04-25T10:04:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.296+0000 7fc617acb640 1 -- start start 2024-04-25T10:04:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.297+0000 7fc617acb640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6100ced80 0x7fc6100d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.297+0000 7fc617acb640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61015e000 con 0x7fc61015dc20 2024-04-25T10:04:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.297+0000 7fc617acb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6100d1de0 con 0x7fc610161710 2024-04-25T10:04:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.297+0000 7fc617acb640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc610161c60 con 0x7fc6100ced80 2024-04-25T10:04:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.297+0000 7fc615066640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6100ced80 0x7fc6100d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.297+0000 7fc615066640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6100ced80 0x7fc6100d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49252/0 (socket says 172.21.15.19:49252) 2024-04-25T10:04:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.297+0000 7fc615867640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc61015dc20 0x7fc6100ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56972/0 (socket says 172.21.15.19:56972) 2024-04-25T10:04:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.297+0000 7fc615066640 1 -- 172.21.15.19:0/3169966014 learned_addr learned my addr 172.21.15.19:0/3169966014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.297+0000 7fc615066640 1 -- 172.21.15.19:0/3169966014 >> v1:172.21.15.112:6789/0 conn(0x7fc610161710 legacy=0x7fc6100d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.297+0000 7fc615066640 1 -- 172.21.15.19:0/3169966014 >> v1:172.21.15.19:6789/0 conn(0x7fc61015dc20 legacy=0x7fc6100ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.297+0000 7fc615066640 1 -- 172.21.15.19:0/3169966014 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610161f40 con 0x7fc6100ced80 2024-04-25T10:04:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.298+0000 7fc615066640 1 --2- 172.21.15.19:0/3169966014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6100ced80 0x7fc6100d22e0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fc600009870 tx=0x7fc60002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b1e6c6bc8591d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.298+0000 7fc614865640 1 -- 172.21.15.19:0/3169966014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600037070 con 0x7fc6100ced80 2024-04-25T10:04:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.298+0000 7fc614865640 1 -- 172.21.15.19:0/3169966014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600032440 con 0x7fc6100ced80 2024-04-25T10:04:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.298+0000 7fc614865640 1 -- 172.21.15.19:0/3169966014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600032810 con 0x7fc6100ced80 2024-04-25T10:04:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.299+0000 7fc617acb640 1 -- 172.21.15.19:0/3169966014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6100ced80 msgr2=0x7fc6100d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.299+0000 7fc617acb640 1 --2- 172.21.15.19:0/3169966014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6100ced80 0x7fc6100d22e0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fc600009870 tx=0x7fc60002a2a0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.299+0000 7fc617acb640 1 -- 172.21.15.19:0/3169966014 shutdown_connections 2024-04-25T10:04:00.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.299+0000 7fc617acb640 1 --2- 172.21.15.19:0/3169966014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6100ced80 0x7fc6100d22e0 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.299+0000 7fc617acb640 1 -- 172.21.15.19:0/3169966014 >> 172.21.15.19:0/3169966014 conn(0x7fc6100d0420 msgr2=0x7fc6100d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:00.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.299+0000 7fc617acb640 1 -- 172.21.15.19:0/3169966014 shutdown_connections 2024-04-25T10:04:00.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.299+0000 7fc617acb640 1 -- 172.21.15.19:0/3169966014 wait complete. 2024-04-25T10:04:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.300+0000 7fc617acb640 1 Processor -- start 2024-04-25T10:04:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.300+0000 7fc617acb640 1 -- start start 2024-04-25T10:04:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc617acb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100ced80 0x7fc6100c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc617acb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc61015dc20 0x7fc6101585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc617acb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610161710 0x7fc6101f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc617acb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6100b5e50 con 0x7fc610161710 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc617acb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc610162500 con 0x7fc61015dc20 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc617acb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6100b5fd0 con 0x7fc6100ced80 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc616068640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610161710 0x7fc6101f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc616068640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610161710 0x7fc6101f72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49264/0 (socket says 172.21.15.19:49264) 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc616068640 1 -- 172.21.15.19:0/2290372831 learned_addr learned my addr 172.21.15.19:0/2290372831 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc615867640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100ced80 0x7fc6100c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc615066640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc61015dc20 0x7fc6101585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc616068640 1 -- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100ced80 msgr2=0x7fc6100c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc616068640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100ced80 0x7fc6100c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc616068640 1 -- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc61015dc20 msgr2=0x7fc6101585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc616068640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc61015dc20 0x7fc6101585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc616068640 1 -- 172.21.15.19:0/2290372831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6100c9920 con 0x7fc610161710 2024-04-25T10:04:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc616068640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610161710 0x7fc6101f72a0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fc60800c780 tx=0x7fc60800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.301+0000 7fc6067fc640 1 -- 172.21.15.19:0/2290372831 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608004b10 con 0x7fc610161710 2024-04-25T10:04:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.302+0000 7fc6067fc640 1 -- 172.21.15.19:0/2290372831 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc608004cb0 con 0x7fc610161710 2024-04-25T10:04:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.302+0000 7fc617acb640 1 -- 172.21.15.19:0/2290372831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6100c9c00 con 0x7fc610161710 2024-04-25T10:04:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.302+0000 7fc6067fc640 1 -- 172.21.15.19:0/2290372831 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60800aa90 con 0x7fc610161710 2024-04-25T10:04:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.302+0000 7fc617acb640 1 -- 172.21.15.19:0/2290372831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6100ca1c0 con 0x7fc610161710 2024-04-25T10:04:00.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.302+0000 7fc617acb640 1 -- 172.21.15.19:0/2290372831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc610073a00 con 0x7fc610161710 2024-04-25T10:04:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.303+0000 7fc6067fc640 1 -- 172.21.15.19:0/2290372831 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc60800ac30 con 0x7fc610161710 2024-04-25T10:04:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.303+0000 7fc6067fc640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5f406b4d0 0x7fc5f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.303+0000 7fc6067fc640 1 -- 172.21.15.19:0/2290372831 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc60808a880 con 0x7fc610161710 2024-04-25T10:04:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.304+0000 7fc615867640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5f406b4d0 0x7fc5f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.304+0000 7fc615867640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5f406b4d0 0x7fc5f406d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc60c000f60 tx=0x7fc60c007cc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.308+0000 7fc6067fc640 1 -- 172.21.15.19:0/2290372831 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc608059570 con 0x7fc610161710 2024-04-25T10:04:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.487+0000 7fc617acb640 1 -- 172.21.15.19:0/2290372831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6100d16b0 con 0x7fc610161710 2024-04-25T10:04:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.487+0000 7fc6067fc640 1 -- 172.21.15.19:0/2290372831 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v154) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc608058e70 con 0x7fc610161710 2024-04-25T10:04:00.492 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:04:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.489+0000 7fc617acb640 1 -- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5f406b4d0 msgr2=0x7fc5f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.489+0000 7fc617acb640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5f406b4d0 0x7fc5f406d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc60c000f60 tx=0x7fc60c007cc0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.489+0000 7fc617acb640 1 -- 172.21.15.19:0/2290372831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610161710 msgr2=0x7fc6101f72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.489+0000 7fc617acb640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610161710 0x7fc6101f72a0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fc60800c780 tx=0x7fc60800cc40 comp rx=0 tx=0).stop 2024-04-25T10:04:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.489+0000 7fc617acb640 1 -- 172.21.15.19:0/2290372831 shutdown_connections 2024-04-25T10:04:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.489+0000 7fc617acb640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5f406b4d0 0x7fc5f406d990 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.489+0000 7fc617acb640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc610161710 0x7fc6101f72a0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.489+0000 7fc617acb640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc61015dc20 0x7fc6101585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.489+0000 7fc617acb640 1 --2- 172.21.15.19:0/2290372831 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100ced80 0x7fc6100c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.490+0000 7fc617acb640 1 -- 172.21.15.19:0/2290372831 >> 172.21.15.19:0/2290372831 conn(0x7fc6100d0420 msgr2=0x7fc6100d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.490+0000 7fc617acb640 1 -- 172.21.15.19:0/2290372831 shutdown_connections 2024-04-25T10:04:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.490+0000 7fc617acb640 1 -- 172.21.15.19:0/2290372831 wait complete. 2024-04-25T10:04:00.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:04:00.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.604+0000 7f1ff668f640 1 Processor -- start 2024-04-25T10:04:00.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.604+0000 7f1ff668f640 1 -- start start 2024-04-25T10:04:00.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.605+0000 7f1ff668f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ff00ce7b0 0x7f1ff0161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.605+0000 7f1ff668f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff006c8d0 con 0x7f1ff01616f0 2024-04-25T10:04:00.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.605+0000 7f1ff668f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff00c6960 con 0x7f1ff01610f0 2024-04-25T10:04:00.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.605+0000 7f1ff668f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff01614d0 con 0x7f1ff00ce7b0 2024-04-25T10:04:00.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.605+0000 7f1fef7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1ff01616f0 0x7f1ff00ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56978/0 (socket says 172.21.15.19:56978) 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.605+0000 7f1fef7fe640 1 -- 172.21.15.19:0/729834237 learned_addr learned my addr 172.21.15.19:0/729834237 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1ff4c2c640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ff00ce7b0 0x7f1ff0161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836163089 0 0) 0x7f1ff006c8d0 con 0x7f1ff01616f0 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fd8003520 con 0x7f1ff01616f0 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805093995 0 0) 0x7f1ff00c6960 con 0x7f1ff01610f0 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ff006c8d0 con 0x7f1ff01610f0 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3802144179 0 0) 0x7f1fd8003520 con 0x7f1ff01616f0 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ff00ce7b0 msgr2=0x7f1ff0161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ff00ce7b0 0x7f1ff0161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 >> v1:172.21.15.112:6789/0 conn(0x7f1ff01610f0 legacy=0x7f1ff006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff0162260 con 0x7f1ff01616f0 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f1fe40015f0 con 0x7f1ff01616f0 2024-04-25T10:04:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 >> v1:172.21.15.19:6789/0 conn(0x7f1ff01616f0 legacy=0x7f1ff00ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ff00ce7b0 0x7f1fd8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd8004120 0x7f1fd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd8004d40 0x7f1fd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff01614d0 con 0x7f1fd8004120 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd800bb00 con 0x7f1fd8004d40 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd800bd00 con 0x7f1ff00ce7b0 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1fef7fe640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd8004120 0x7f1fd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1ff4c2c640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd8004d40 0x7f1fd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1feffff640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ff00ce7b0 0x7f1fd8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1fef7fe640 1 -- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ff00ce7b0 msgr2=0x7f1fd8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1fef7fe640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ff00ce7b0 0x7f1fd8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1fef7fe640 1 -- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd8004d40 msgr2=0x7f1fd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1fef7fe640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd8004d40 0x7f1fd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.606+0000 7f1fef7fe640 1 -- 172.21.15.19:0/729834237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8003520 con 0x7f1fd8004120 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.607+0000 7f1ff4c2c640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fd8004d40 0x7f1fd80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.607+0000 7f1feffff640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ff00ce7b0 0x7f1fd8003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.607+0000 7f1fef7fe640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd8004120 0x7f1fd8004800 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4007f60 tx=0x7f1fd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59e5789e6936151 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.607+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4007b80 con 0x7f1fd8004120 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.607+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe4005bb0 con 0x7f1fd8004120 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.607+0000 7f1feeffd640 1 -- 172.21.15.19:0/729834237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4007070 con 0x7f1fd8004120 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.608+0000 7f1ff668f640 1 -- 172.21.15.19:0/729834237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd8004120 msgr2=0x7f1fd8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.608+0000 7f1ff668f640 1 --2- 172.21.15.19:0/729834237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd8004120 0x7f1fd8004800 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4007f60 tx=0x7f1fd8005830 comp rx=0 tx=0).stop 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.608+0000 7f1ff4c2c640 1 -- 172.21.15.19:0/729834237 reap_dead start 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.608+0000 7f1ff668f640 1 -- 172.21.15.19:0/729834237 shutdown_connections 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.608+0000 7f1ff668f640 1 -- 172.21.15.19:0/729834237 >> 172.21.15.19:0/729834237 conn(0x7f1ff006f500 msgr2=0x7f1ff006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.608+0000 7f1ff668f640 1 -- 172.21.15.19:0/729834237 shutdown_connections 2024-04-25T10:04:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.608+0000 7f1ff668f640 1 -- 172.21.15.19:0/729834237 wait complete. 2024-04-25T10:04:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.609+0000 7f1ff668f640 1 Processor -- start 2024-04-25T10:04:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.609+0000 7f1ff668f640 1 -- start start 2024-04-25T10:04:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.610+0000 7f1ff668f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd8004120 0x7f1ff0145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.610+0000 7f1ff668f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ff015df20 0x7f1ff015e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.610+0000 7f1ff668f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015fa00 0x7f1ff015fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.610+0000 7f1ff668f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff00cfce0 con 0x7f1ff015fa00 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.610+0000 7f1ff668f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff00cfa00 con 0x7f1ff015df20 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.610+0000 7f1ff668f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff00cf690 con 0x7f1fd8004120 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.610+0000 7f1ff4c2c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015fa00 0x7f1ff015fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.610+0000 7f1ff4c2c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015fa00 0x7f1ff015fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49280/0 (socket says 172.21.15.19:49280) 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.610+0000 7f1ff4c2c640 1 -- 172.21.15.19:0/4288317284 learned_addr learned my addr 172.21.15.19:0/4288317284 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1fef7fe640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ff015df20 0x7f1ff015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1ff4c2c640 1 -- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd8004120 msgr2=0x7f1ff0145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1feffff640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd8004120 0x7f1ff0145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1ff4c2c640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd8004120 0x7f1ff0145610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1ff4c2c640 1 -- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ff015df20 msgr2=0x7f1ff015e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1ff4c2c640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ff015df20 0x7f1ff015e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1ff4c2c640 1 -- 172.21.15.19:0/4288317284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff015e8c0 con 0x7f1ff015fa00 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1fef7fe640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ff015df20 0x7f1ff015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1ff4c2c640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015fa00 0x7f1ff015fe80 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00cf120 tx=0x7f1fe8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1fecff9640 1 -- 172.21.15.19:0/4288317284 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8010d20 con 0x7f1ff015fa00 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1fecff9640 1 -- 172.21.15.19:0/4288317284 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe8003990 con 0x7f1ff015fa00 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1ff668f640 1 -- 172.21.15.19:0/4288317284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff01603c0 con 0x7f1ff015fa00 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1fecff9640 1 -- 172.21.15.19:0/4288317284 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8003dc0 con 0x7f1ff015fa00 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.611+0000 7f1ff668f640 1 -- 172.21.15.19:0/4288317284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff015ee10 con 0x7f1ff015fa00 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.612+0000 7f1ff668f640 1 -- 172.21.15.19:0/4288317284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff00adad0 con 0x7f1ff015fa00 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.612+0000 7f1fecff9640 1 -- 172.21.15.19:0/4288317284 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1fe8005d10 con 0x7f1ff015fa00 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.613+0000 7f1fecff9640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.613+0000 7f1fecff9640 1 -- 172.21.15.19:0/4288317284 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1fe808a8d0 con 0x7f1ff015fa00 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.613+0000 7f1feffff640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.613+0000 7f1feffff640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0007fd0 tx=0x7f1fe0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.617+0000 7f1fecff9640 1 -- 172.21.15.19:0/4288317284 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ff00adad0 con 0x7f1ff015fa00 2024-04-25T10:04:00.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.801+0000 7f1ff668f640 1 -- 172.21.15.19:0/4288317284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ff0073a00 con 0x7f1ff015fa00 2024-04-25T10:04:00.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.801+0000 7f1fecff9640 1 -- 172.21.15.19:0/4288317284 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v154) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1fe8025020 con 0x7f1ff015fa00 2024-04-25T10:04:00.806 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:04:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.803+0000 7f1ff668f640 1 -- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fdc06b4d0 msgr2=0x7f1fdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.803+0000 7f1ff668f640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0007fd0 tx=0x7f1fe0009040 comp rx=0 tx=0).stop 2024-04-25T10:04:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.803+0000 7f1ff668f640 1 -- 172.21.15.19:0/4288317284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015fa00 msgr2=0x7f1ff015fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.803+0000 7f1ff668f640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015fa00 0x7f1ff015fe80 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00cf120 tx=0x7f1fe8010590 comp rx=0 tx=0).stop 2024-04-25T10:04:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.803+0000 7f1ff668f640 1 -- 172.21.15.19:0/4288317284 shutdown_connections 2024-04-25T10:04:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.803+0000 7f1ff668f640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.803+0000 7f1ff668f640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015fa00 0x7f1ff015fe80 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.803+0000 7f1ff668f640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ff015df20 0x7f1ff015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.803+0000 7f1ff668f640 1 --2- 172.21.15.19:0/4288317284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1fd8004120 0x7f1ff0145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.803+0000 7f1ff668f640 1 -- 172.21.15.19:0/4288317284 >> 172.21.15.19:0/4288317284 conn(0x7f1ff006f500 msgr2=0x7f1ff00d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.804+0000 7f1ff668f640 1 -- 172.21.15.19:0/4288317284 shutdown_connections 2024-04-25T10:04:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.804+0000 7f1ff668f640 1 -- 172.21.15.19:0/4288317284 wait complete. 2024-04-25T10:04:00.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.915+0000 7f954661f640 1 Processor -- start 2024-04-25T10:04:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.915+0000 7f954661f640 1 -- start start 2024-04-25T10:04:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.916+0000 7f954661f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f954015dc00 0x7f95400c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.916+0000 7f954661f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95400d1dd0 0x7f95400ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.916+0000 7f954661f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954015dfe0 con 0x7f95400ceb20 2024-04-25T10:04:00.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.916+0000 7f954661f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95400d00c0 con 0x7f95400d1dd0 2024-04-25T10:04:00.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.916+0000 7f954661f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95400d03a0 con 0x7f954015dc00 2024-04-25T10:04:00.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.917+0000 7f953ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f954015dc00 0x7f95400c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.917+0000 7f9544bbc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95400d1dd0 0x7f95400ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.917+0000 7f953ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f954015dc00 0x7f95400c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:48542/0 (socket says 172.21.15.19:48542) 2024-04-25T10:04:00.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.917+0000 7f953ffff640 1 -- 172.21.15.19:0/197513588 learned_addr learned my addr 172.21.15.19:0/197513588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.917+0000 7f9544bbc640 1 -- 172.21.15.19:0/197513588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f954015dc00 msgr2=0x7f95400c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.917+0000 7f9544bbc640 1 --2- 172.21.15.19:0/197513588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f954015dc00 0x7f95400c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.917+0000 7f9544bbc640 1 -- 172.21.15.19:0/197513588 >> v1:172.21.15.112:6789/0 conn(0x7f95400ceb20 legacy=0x7f95400d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.917+0000 7f9544bbc640 1 -- 172.21.15.19:0/197513588 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95400cc440 con 0x7f95400d1dd0 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.917+0000 7f953ffff640 1 --2- 172.21.15.19:0/197513588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f954015dc00 0x7f95400c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f9544bbc640 1 --2- 172.21.15.19:0/197513588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95400d1dd0 0x7f95400ccc40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9538009d90 tx=0x7f953802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b0bfcda33dcad1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f953effd640 1 -- 172.21.15.19:0/197513588 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538007740 con 0x7f95400d1dd0 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f953effd640 1 -- 172.21.15.19:0/197513588 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95380078e0 con 0x7f95400d1dd0 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f953effd640 1 -- 172.21.15.19:0/197513588 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538007cb0 con 0x7f95400d1dd0 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f954661f640 1 -- 172.21.15.19:0/197513588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95400d1dd0 msgr2=0x7f95400ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f954661f640 1 --2- 172.21.15.19:0/197513588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95400d1dd0 0x7f95400ccc40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9538009d90 tx=0x7f953802a660 comp rx=0 tx=0).stop 2024-04-25T10:04:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f954661f640 1 -- 172.21.15.19:0/197513588 shutdown_connections 2024-04-25T10:04:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f954661f640 1 --2- 172.21.15.19:0/197513588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95400d1dd0 0x7f95400ccc40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f954661f640 1 --2- 172.21.15.19:0/197513588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f954015dc00 0x7f95400c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f954661f640 1 -- 172.21.15.19:0/197513588 >> 172.21.15.19:0/197513588 conn(0x7f954006f5d0 msgr2=0x7f954006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f954661f640 1 -- 172.21.15.19:0/197513588 shutdown_connections 2024-04-25T10:04:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.918+0000 7f954661f640 1 -- 172.21.15.19:0/197513588 wait complete. 2024-04-25T10:04:00.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.920+0000 7f954661f640 1 Processor -- start 2024-04-25T10:04:00.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.920+0000 7f954661f640 1 -- start start 2024-04-25T10:04:00.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.920+0000 7f954661f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95400ceb20 0x7f954006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.920+0000 7f954661f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95400d1dd0 0x7f95400ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.920+0000 7f954661f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954015dc00 0x7f95400b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.920+0000 7f954661f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95400cfd90 con 0x7f954015dc00 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.920+0000 7f954661f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95400cc130 con 0x7f95400d1dd0 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.920+0000 7f954661f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95400cdd70 con 0x7f95400ceb20 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f9544bbc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954015dc00 0x7f95400b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f9544bbc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954015dc00 0x7f95400b6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49282/0 (socket says 172.21.15.19:49282) 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f9544bbc640 1 -- 172.21.15.19:0/3742170274 learned_addr learned my addr 172.21.15.19:0/3742170274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f953f7fe640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95400d1dd0 0x7f95400ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f953ffff640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95400ceb20 0x7f954006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f9544bbc640 1 -- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95400ceb20 msgr2=0x7f954006d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f9544bbc640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95400ceb20 0x7f954006d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f9544bbc640 1 -- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95400d1dd0 msgr2=0x7f95400ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f9544bbc640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95400d1dd0 0x7f95400ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f9544bbc640 1 -- 172.21.15.19:0/3742170274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95400cb450 con 0x7f954015dc00 2024-04-25T10:04:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f9544bbc640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954015dc00 0x7f95400b6f80 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f9538009eb0 tx=0x7f9538004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f953cff9640 1 -- 172.21.15.19:0/3742170274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538040070 con 0x7f954015dc00 2024-04-25T10:04:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f953cff9640 1 -- 172.21.15.19:0/3742170274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9538031bc0 con 0x7f954015dc00 2024-04-25T10:04:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f953cff9640 1 -- 172.21.15.19:0/3742170274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538009450 con 0x7f954015dc00 2024-04-25T10:04:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f954661f640 1 -- 172.21.15.19:0/3742170274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95400cae90 con 0x7f954015dc00 2024-04-25T10:04:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.921+0000 7f954661f640 1 -- 172.21.15.19:0/3742170274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954006bb30 con 0x7f954015dc00 2024-04-25T10:04:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.922+0000 7f954661f640 1 -- 172.21.15.19:0/3742170274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9540073990 con 0x7f954015dc00 2024-04-25T10:04:00.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.923+0000 7f953cff9640 1 -- 172.21.15.19:0/3742170274 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9538039040 con 0x7f954015dc00 2024-04-25T10:04:00.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.923+0000 7f953cff9640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952006b4d0 0x7f952006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:00.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.923+0000 7f953cff9640 1 -- 172.21.15.19:0/3742170274 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f95380abc80 con 0x7f954015dc00 2024-04-25T10:04:00.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.924+0000 7f953ffff640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952006b4d0 0x7f952006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:00.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.924+0000 7f953ffff640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952006b4d0 0x7f952006d990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f95400b6b10 tx=0x7f9528008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:00.928+0000 7f953cff9640 1 -- 172.21.15.19:0/3742170274 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f953807a970 con 0x7f954015dc00 2024-04-25T10:04:01.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.149+0000 7f954661f640 1 -- 172.21.15.19:0/3742170274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95400c66c0 con 0x7f954015dc00 2024-04-25T10:04:01.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.150+0000 7f953cff9640 1 -- 172.21.15.19:0/3742170274 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f953807a270 con 0x7f954015dc00 2024-04-25T10:04:01.156 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:01.156 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":102,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3725393673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3725393673},{"type":"v1","addr":"172.21.15.112:6837","nonce":3725393673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8866,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2539453196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2539453196},{"type":"v1","addr":"172.21.15.112:6836","nonce":2539453196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10221,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3896097924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3896097924},{"type":"v1","addr":"172.21.15.19:6834","nonce":3896097924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:14.030067+0000","modified":"2024-04-25T10:03:17.045113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10224":{"gid":10224,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/717842662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":717842662},{"type":"v1","addr":"172.21.15.19:6835","nonce":717842662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:04:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 102 2024-04-25T10:04:01.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 -- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952006b4d0 msgr2=0x7f952006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952006b4d0 0x7f952006d990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f95400b6b10 tx=0x7f9528008040 comp rx=0 tx=0).stop 2024-04-25T10:04:01.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 -- 172.21.15.19:0/3742170274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954015dc00 msgr2=0x7f95400b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954015dc00 0x7f95400b6f80 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f9538009eb0 tx=0x7f9538004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:01.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 -- 172.21.15.19:0/3742170274 shutdown_connections 2024-04-25T10:04:01.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f952006b4d0 0x7f952006d990 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f954015dc00 0x7f95400b6f80 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95400d1dd0 0x7f95400ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 --2- 172.21.15.19:0/3742170274 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95400ceb20 0x7f954006d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 -- 172.21.15.19:0/3742170274 >> 172.21.15.19:0/3742170274 conn(0x7f954006f5d0 msgr2=0x7f95400b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 -- 172.21.15.19:0/3742170274 shutdown_connections 2024-04-25T10:04:01.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.152+0000 7f954661f640 1 -- 172.21.15.19:0/3742170274 wait complete. 2024-04-25T10:04:01.168 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:01.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.265+0000 7f15c095c640 1 Processor -- start 2024-04-25T10:04:01.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.265+0000 7f15c095c640 1 -- start start 2024-04-25T10:04:01.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.265+0000 7f15c095c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15bc15dc20 0x7f15bc1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.265+0000 7f15c095c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc15e000 con 0x7f15bc0cedf0 2024-04-25T10:04:01.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.266+0000 7f15c095c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc0b6bb0 con 0x7f15bc0b93a0 2024-04-25T10:04:01.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.266+0000 7f15c095c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15bc0b6030 con 0x7f15bc15dc20 2024-04-25T10:04:01.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.266+0000 7f15ba59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15bc15dc20 0x7f15bc1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.266+0000 7f15bad9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f15bc0cedf0 0x7f15bc0d2510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56988/0 (socket says 172.21.15.19:56988) 2024-04-25T10:04:01.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.266+0000 7f15bad9d640 1 -- 172.21.15.19:0/2340016715 learned_addr learned my addr 172.21.15.19:0/2340016715 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:01.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.266+0000 7f15ba59c640 1 -- 172.21.15.19:0/2340016715 >> v1:172.21.15.112:6789/0 conn(0x7f15bc0b93a0 legacy=0x7f15bc0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.266+0000 7f15ba59c640 1 -- 172.21.15.19:0/2340016715 >> v1:172.21.15.19:6789/0 conn(0x7f15bc0cedf0 legacy=0x7f15bc0d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.266+0000 7f15ba59c640 1 -- 172.21.15.19:0/2340016715 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc161de0 con 0x7f15bc15dc20 2024-04-25T10:04:01.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.266+0000 7f15b9d9b640 1 -- 172.21.15.19:0/2340016715 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969004968 0 0) 0x7f15b4001490 con 0x7f15bc0cedf0 2024-04-25T10:04:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.267+0000 7f15ba59c640 1 --2- 172.21.15.19:0/2340016715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc15dc20 0x7f15bc1616b0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0097a0 tx=0x7f15ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80fdfddb427c81b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.267+0000 7f15b9d9b640 1 -- 172.21.15.19:0/2340016715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac037070 con 0x7f15bc15dc20 2024-04-25T10:04:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.267+0000 7f15b9d9b640 1 -- 172.21.15.19:0/2340016715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac032440 con 0x7f15bc15dc20 2024-04-25T10:04:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.267+0000 7f15b9d9b640 1 -- 172.21.15.19:0/2340016715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac032810 con 0x7f15bc15dc20 2024-04-25T10:04:01.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.267+0000 7f15c095c640 1 -- 172.21.15.19:0/2340016715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc15dc20 msgr2=0x7f15bc1616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.267+0000 7f15c095c640 1 --2- 172.21.15.19:0/2340016715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc15dc20 0x7f15bc1616b0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0097a0 tx=0x7f15ac02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.267+0000 7f15c095c640 1 -- 172.21.15.19:0/2340016715 shutdown_connections 2024-04-25T10:04:01.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.267+0000 7f15c095c640 1 --2- 172.21.15.19:0/2340016715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc15dc20 0x7f15bc1616b0 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.267+0000 7f15c095c640 1 -- 172.21.15.19:0/2340016715 >> 172.21.15.19:0/2340016715 conn(0x7f15bc0d0490 msgr2=0x7f15bc0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.268+0000 7f15c095c640 1 -- 172.21.15.19:0/2340016715 shutdown_connections 2024-04-25T10:04:01.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.268+0000 7f15c095c640 1 -- 172.21.15.19:0/2340016715 wait complete. 2024-04-25T10:04:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.269+0000 7f15c095c640 1 Processor -- start 2024-04-25T10:04:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.269+0000 7f15c095c640 1 -- start start 2024-04-25T10:04:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15c095c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0b93a0 0x7f15bc1e7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15c095c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15bc0cedf0 0x7f15bc158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15c095c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc15dc20 0x7f15bc1e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15c095c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0d2c20 con 0x7f15bc0b93a0 2024-04-25T10:04:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15c095c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc1623a0 con 0x7f15bc15dc20 2024-04-25T10:04:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15c095c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0d2da0 con 0x7f15bc0cedf0 2024-04-25T10:04:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15ba59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0b93a0 0x7f15bc1e7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15ba59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0b93a0 0x7f15bc1e7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49290/0 (socket says 172.21.15.19:49290) 2024-04-25T10:04:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15ba59c640 1 -- 172.21.15.19:0/1743916443 learned_addr learned my addr 172.21.15.19:0/1743916443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15bad9d640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc15dc20 0x7f15bc1e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15ba59c640 1 -- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15bc0cedf0 msgr2=0x7f15bc158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15b3fff640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15bc0cedf0 0x7f15bc158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15ba59c640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15bc0cedf0 0x7f15bc158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15ba59c640 1 -- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc15dc20 msgr2=0x7f15bc1e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15ba59c640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc15dc20 0x7f15bc1e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15ba59c640 1 -- 172.21.15.19:0/1743916443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc15e280 con 0x7f15bc0b93a0 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15bad9d640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc15dc20 0x7f15bc1e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15ba59c640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0b93a0 0x7f15bc1e7390 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0098c0 tx=0x7f15ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15b2ffd640 1 -- 172.21.15.19:0/1743916443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac041070 con 0x7f15bc0b93a0 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15b2ffd640 1 -- 172.21.15.19:0/1743916443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac03c870 con 0x7f15bc0b93a0 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.270+0000 7f15c095c640 1 -- 172.21.15.19:0/1743916443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15bc15e4f0 con 0x7f15bc0b93a0 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.271+0000 7f15b2ffd640 1 -- 172.21.15.19:0/1743916443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac037040 con 0x7f15bc0b93a0 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.271+0000 7f15c095c640 1 -- 172.21.15.19:0/1743916443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc1e78d0 con 0x7f15bc0b93a0 2024-04-25T10:04:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.271+0000 7f15c095c640 1 -- 172.21.15.19:0/1743916443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc15e8e0 con 0x7f15bc0b93a0 2024-04-25T10:04:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.272+0000 7f15b2ffd640 1 -- 172.21.15.19:0/1743916443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15ac044070 con 0x7f15bc0b93a0 2024-04-25T10:04:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.272+0000 7f15b2ffd640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f159c06b4d0 0x7f159c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.272+0000 7f15b2ffd640 1 -- 172.21.15.19:0/1743916443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f15ac0ab0b0 con 0x7f15bc0b93a0 2024-04-25T10:04:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.273+0000 7f15b3fff640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f159c06b4d0 0x7f159c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.273+0000 7f15b3fff640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f159c06b4d0 0x7f159c06d990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f15bc1f7100 tx=0x7f15a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.277+0000 7f15b2ffd640 1 -- 172.21.15.19:0/1743916443 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15bc0b6bb0 con 0x7f15bc0b93a0 2024-04-25T10:04:01.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.458+0000 7f15c095c640 1 -- 172.21.15.19:0/1743916443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15bc15e6c0 con 0x7f15bc0b93a0 2024-04-25T10:04:01.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.459+0000 7f15b2ffd640 1 -- 172.21.15.19:0/1743916443 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f15ac079f50 con 0x7f15bc0b93a0 2024-04-25T10:04:01.464 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:01.464 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":154,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:03:15.658204+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:12.307052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:13.005066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:01.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 -- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f159c06b4d0 msgr2=0x7f159c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f159c06b4d0 0x7f159c06d990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f15bc1f7100 tx=0x7f15a4009040 comp rx=0 tx=0).stop 2024-04-25T10:04:01.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 -- 172.21.15.19:0/1743916443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0b93a0 msgr2=0x7f15bc1e7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0b93a0 0x7f15bc1e7390 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0098c0 tx=0x7f15ac0046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 -- 172.21.15.19:0/1743916443 shutdown_connections 2024-04-25T10:04:01.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f159c06b4d0 0x7f159c06d990 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc15dc20 0x7f15bc1e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15bc0cedf0 0x7f15bc158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 --2- 172.21.15.19:0/1743916443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0b93a0 0x7f15bc1e7390 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 -- 172.21.15.19:0/1743916443 >> 172.21.15.19:0/1743916443 conn(0x7f15bc0d0490 msgr2=0x7f15bc0b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 -- 172.21.15.19:0/1743916443 shutdown_connections 2024-04-25T10:04:01.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.461+0000 7f15c095c640 1 -- 172.21.15.19:0/1743916443 wait complete. 2024-04-25T10:04:01.477 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:04:01.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:01.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.573+0000 7f55fa68f640 1 Processor -- start 2024-04-25T10:04:01.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.573+0000 7f55fa68f640 1 -- start start 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55fa68f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f415dd30 0x7f55f40b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55fa68f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f55f40b9770 0x7f55f40d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55fa68f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f40d1df0 con 0x7f55f40d2e90 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55fa68f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f40b7a50 con 0x7f55f40b9770 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55fa68f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40d20d0 con 0x7f55f415dd30 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55f37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f55f40b9770 0x7f55f40d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55f3fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f415dd30 0x7f55f40b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55f8c2c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f55f40d2e90 0x7f55f40b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:59606/0 (socket says 172.21.15.19:59606) 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55f8c2c640 1 -- 172.21.15.19:0/1327662395 learned_addr learned my addr 172.21.15.19:0/1327662395 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55f37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f55f40b9770 0x7f55f40d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:40000/0 (socket says 172.21.15.19:40000) 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55f37fe640 1 -- 172.21.15.19:0/1327662395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f415dd30 msgr2=0x7f55f40b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55f37fe640 1 --2- 172.21.15.19:0/1327662395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f415dd30 0x7f55f40b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55f37fe640 1 -- 172.21.15.19:0/1327662395 >> v1:172.21.15.112:6789/0 conn(0x7f55f40d2e90 legacy=0x7f55f40b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.574+0000 7f55f37fe640 1 -- 172.21.15.19:0/1327662395 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40d23b0 con 0x7f55f40b9770 2024-04-25T10:04:01.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.575+0000 7f55f37fe640 1 --2- 172.21.15.19:0/1327662395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f40b9770 0x7f55f40d03c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009790 tx=0x7f55e402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=723c874dfebdc3f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.575+0000 7f55f2ffd640 1 -- 172.21.15.19:0/1327662395 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4037070 con 0x7f55f40b9770 2024-04-25T10:04:01.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.575+0000 7f55f2ffd640 1 -- 172.21.15.19:0/1327662395 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e402ae50 con 0x7f55f40b9770 2024-04-25T10:04:01.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.575+0000 7f55f2ffd640 1 -- 172.21.15.19:0/1327662395 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4032660 con 0x7f55f40b9770 2024-04-25T10:04:01.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.576+0000 7f55fa68f640 1 -- 172.21.15.19:0/1327662395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f40b9770 msgr2=0x7f55f40d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.576+0000 7f55fa68f640 1 --2- 172.21.15.19:0/1327662395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f40b9770 0x7f55f40d03c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009790 tx=0x7f55e402a100 comp rx=0 tx=0).stop 2024-04-25T10:04:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.576+0000 7f55fa68f640 1 -- 172.21.15.19:0/1327662395 shutdown_connections 2024-04-25T10:04:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.576+0000 7f55fa68f640 1 --2- 172.21.15.19:0/1327662395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f40b9770 0x7f55f40d03c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.576+0000 7f55fa68f640 1 --2- 172.21.15.19:0/1327662395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f415dd30 0x7f55f40b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.576+0000 7f55fa68f640 1 -- 172.21.15.19:0/1327662395 >> 172.21.15.19:0/1327662395 conn(0x7f55f4061c30 msgr2=0x7f55f406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.576+0000 7f55fa68f640 1 -- 172.21.15.19:0/1327662395 shutdown_connections 2024-04-25T10:04:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.576+0000 7f55fa68f640 1 -- 172.21.15.19:0/1327662395 wait complete. 2024-04-25T10:04:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.577+0000 7f55fa68f640 1 Processor -- start 2024-04-25T10:04:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.577+0000 7f55fa68f640 1 -- start start 2024-04-25T10:04:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.578+0000 7f55fa68f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f40b9770 0x7f55f40b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.578+0000 7f55fa68f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f40d2e90 0x7f55f40cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.578+0000 7f55fa68f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f415dd30 0x7f55f41606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.578+0000 7f55fa68f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40d1a10 con 0x7f55f40d2e90 2024-04-25T10:04:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.578+0000 7f55fa68f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40d2970 con 0x7f55f415dd30 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.578+0000 7f55fa68f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40d1b90 con 0x7f55f40b9770 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.578+0000 7f55f37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f40d2e90 0x7f55f40cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.578+0000 7f55f37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f40d2e90 0x7f55f40cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49306/0 (socket says 172.21.15.19:49306) 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.578+0000 7f55f37fe640 1 -- 172.21.15.19:0/3019629671 learned_addr learned my addr 172.21.15.19:0/3019629671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.578+0000 7f55f3fff640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f40b9770 0x7f55f40b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.578+0000 7f55f8c2c640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f415dd30 0x7f55f41606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.579+0000 7f55f37fe640 1 -- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f40b9770 msgr2=0x7f55f40b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.579+0000 7f55f37fe640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f40b9770 0x7f55f40b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.579+0000 7f55f37fe640 1 -- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f415dd30 msgr2=0x7f55f41606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.579+0000 7f55f37fe640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f415dd30 0x7f55f41606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.579+0000 7f55f37fe640 1 -- 172.21.15.19:0/3019629671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40cc810 con 0x7f55f40d2e90 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.579+0000 7f55f37fe640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f40d2e90 0x7f55f40cbd70 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f55e40098b0 tx=0x7f55e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.579+0000 7f55f0ff9640 1 -- 172.21.15.19:0/3019629671 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4037040 con 0x7f55f40d2e90 2024-04-25T10:04:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.579+0000 7f55f0ff9640 1 -- 172.21.15.19:0/3019629671 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e4042070 con 0x7f55f40d2e90 2024-04-25T10:04:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.579+0000 7f55fa68f640 1 -- 172.21.15.19:0/3019629671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f40ccaf0 con 0x7f55f40d2e90 2024-04-25T10:04:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.579+0000 7f55f0ff9640 1 -- 172.21.15.19:0/3019629671 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4005de0 con 0x7f55f40d2e90 2024-04-25T10:04:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.579+0000 7f55fa68f640 1 -- 172.21.15.19:0/3019629671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f40cd0b0 con 0x7f55f40d2e90 2024-04-25T10:04:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.580+0000 7f55fa68f640 1 -- 172.21.15.19:0/3019629671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f4073990 con 0x7f55f40d2e90 2024-04-25T10:04:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.581+0000 7f55f0ff9640 1 -- 172.21.15.19:0/3019629671 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55e403a070 con 0x7f55f40d2e90 2024-04-25T10:04:01.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.581+0000 7f55f0ff9640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f55d406b4d0 0x7f55d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.581+0000 7f55f0ff9640 1 -- 172.21.15.19:0/3019629671 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55e40aa400 con 0x7f55f40d2e90 2024-04-25T10:04:01.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.581+0000 7f55f3fff640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f55d406b4d0 0x7f55d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.582+0000 7f55f3fff640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f55d406b4d0 0x7f55d406d990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009a30 tx=0x7f55dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.585+0000 7f55f0ff9640 1 -- 172.21.15.19:0/3019629671 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55e4079180 con 0x7f55f40d2e90 2024-04-25T10:04:01.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.807+0000 7f55fa68f640 1 -- 172.21.15.19:0/3019629671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55f4161360 con 0x7f55f40d2e90 2024-04-25T10:04:01.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.808+0000 7f55f0ff9640 1 -- 172.21.15.19:0/3019629671 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f55e4078a80 con 0x7f55f40d2e90 2024-04-25T10:04:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:01.813 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":102,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3725393673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3725393673},{"type":"v1","addr":"172.21.15.112:6837","nonce":3725393673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8866,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2539453196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2539453196},{"type":"v1","addr":"172.21.15.112:6836","nonce":2539453196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10221,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3896097924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3896097924},{"type":"v1","addr":"172.21.15.19:6834","nonce":3896097924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:14.030067+0000","modified":"2024-04-25T10:03:17.045113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10224":{"gid":10224,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/717842662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":717842662},{"type":"v1","addr":"172.21.15.19:6835","nonce":717842662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:04:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 102 2024-04-25T10:04:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 -- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f55d406b4d0 msgr2=0x7f55d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f55d406b4d0 0x7f55d406d990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009a30 tx=0x7f55dc008040 comp rx=0 tx=0).stop 2024-04-25T10:04:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 -- 172.21.15.19:0/3019629671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f40d2e90 msgr2=0x7f55f40cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f40d2e90 0x7f55f40cbd70 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f55e40098b0 tx=0x7f55e40046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 -- 172.21.15.19:0/3019629671 shutdown_connections 2024-04-25T10:04:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f55d406b4d0 0x7f55d406d990 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f415dd30 0x7f55f41606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55f40d2e90 0x7f55f40cbd70 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 --2- 172.21.15.19:0/3019629671 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f40b9770 0x7f55f40b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 -- 172.21.15.19:0/3019629671 >> 172.21.15.19:0/3019629671 conn(0x7f55f4061c30 msgr2=0x7f55f40d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 -- 172.21.15.19:0/3019629671 shutdown_connections 2024-04-25T10:04:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.810+0000 7f55fa68f640 1 -- 172.21.15.19:0/3019629671 wait complete. 2024-04-25T10:04:01.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.922+0000 7f9b12af5640 1 Processor -- start 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.922+0000 7f9b12af5640 1 -- start start 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.922+0000 7f9b12af5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b0c15dbe0 0x7f9b0c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.922+0000 7f9b12af5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b0c0b92a0 0x7f9b0c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.922+0000 7f9b12af5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0c0b97a0 con 0x7f9b0c0b2cb0 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.922+0000 7f9b12af5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0c15dfc0 con 0x7f9b0c15dbe0 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.922+0000 7f9b12af5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0d1a00 con 0x7f9b0c0b92a0 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.923+0000 7f9b10891640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b0c15dbe0 0x7f9b0c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.923+0000 7f9b10891640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b0c15dbe0 0x7f9b0c0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49318/0 (socket says 172.21.15.19:49318) 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.923+0000 7f9b10891640 1 -- 172.21.15.19:0/3942283784 learned_addr learned my addr 172.21.15.19:0/3942283784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.923+0000 7f9b11092640 1 --2- 172.21.15.19:0/3942283784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b0c0b92a0 0x7f9b0c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.923+0000 7f9b10891640 1 -- 172.21.15.19:0/3942283784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b0c0b92a0 msgr2=0x7f9b0c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.923+0000 7f9b10891640 1 --2- 172.21.15.19:0/3942283784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b0c0b92a0 0x7f9b0c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.923+0000 7f9b10891640 1 -- 172.21.15.19:0/3942283784 >> v1:172.21.15.112:6789/0 conn(0x7f9b0c0b2cb0 legacy=0x7f9b0c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.923+0000 7f9b10891640 1 -- 172.21.15.19:0/3942283784 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0d1c50 con 0x7f9b0c15dbe0 2024-04-25T10:04:01.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.924+0000 7f9b10891640 1 --2- 172.21.15.19:0/3942283784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b0c15dbe0 0x7f9b0c0d0910 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009e50 tx=0x7f9b0402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229434aed5cd2cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.924+0000 7f9b0b7fe640 1 -- 172.21.15.19:0/3942283784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04036070 con 0x7f9b0c15dbe0 2024-04-25T10:04:01.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.924+0000 7f9b0b7fe640 1 -- 172.21.15.19:0/3942283784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b04031600 con 0x7f9b0c15dbe0 2024-04-25T10:04:01.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.924+0000 7f9b0b7fe640 1 -- 172.21.15.19:0/3942283784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b040319d0 con 0x7f9b0c15dbe0 2024-04-25T10:04:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.924+0000 7f9b12af5640 1 -- 172.21.15.19:0/3942283784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b0c15dbe0 msgr2=0x7f9b0c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.924+0000 7f9b12af5640 1 --2- 172.21.15.19:0/3942283784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b0c15dbe0 0x7f9b0c0d0910 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009e50 tx=0x7f9b0402a710 comp rx=0 tx=0).stop 2024-04-25T10:04:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.924+0000 7f9b12af5640 1 -- 172.21.15.19:0/3942283784 shutdown_connections 2024-04-25T10:04:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.924+0000 7f9b12af5640 1 --2- 172.21.15.19:0/3942283784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b0c0b92a0 0x7f9b0c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.924+0000 7f9b12af5640 1 --2- 172.21.15.19:0/3942283784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b0c15dbe0 0x7f9b0c0d0910 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.924+0000 7f9b12af5640 1 -- 172.21.15.19:0/3942283784 >> 172.21.15.19:0/3942283784 conn(0x7f9b0c061a20 msgr2=0x7f9b0c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.925+0000 7f9b12af5640 1 -- 172.21.15.19:0/3942283784 shutdown_connections 2024-04-25T10:04:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.925+0000 7f9b12af5640 1 -- 172.21.15.19:0/3942283784 wait complete. 2024-04-25T10:04:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.926+0000 7f9b12af5640 1 Processor -- start 2024-04-25T10:04:01.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.926+0000 7f9b12af5640 1 -- start start 2024-04-25T10:04:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b12af5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c0b2cb0 0x7f9b0c0c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b12af5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b0c0b92a0 0x7f9b0c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b12af5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b0c15dbe0 0x7f9b0c0b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b12af5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0b6ab0 con 0x7f9b0c0b92a0 2024-04-25T10:04:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b12af5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0d2210 con 0x7f9b0c0b2cb0 2024-04-25T10:04:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b12af5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0b6c30 con 0x7f9b0c15dbe0 2024-04-25T10:04:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b0c0b92a0 0x7f9b0c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b0c0b92a0 0x7f9b0c0cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49324/0 (socket says 172.21.15.19:49324) 2024-04-25T10:04:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b0bfff640 1 -- 172.21.15.19:0/461924318 learned_addr learned my addr 172.21.15.19:0/461924318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b0bfff640 1 -- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b0c15dbe0 msgr2=0x7f9b0c0b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b11092640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b0c15dbe0 0x7f9b0c0b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b10891640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c0b2cb0 0x7f9b0c0c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b0bfff640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b0c15dbe0 0x7f9b0c0b84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b0bfff640 1 -- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c0b2cb0 msgr2=0x7f9b0c0c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b0bfff640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c0b2cb0 0x7f9b0c0c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b0bfff640 1 -- 172.21.15.19:0/461924318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0cab80 con 0x7f9b0c0b92a0 2024-04-25T10:04:01.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b0bfff640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b0c0b92a0 0x7f9b0c0cc6e0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00c470 tx=0x7f9afc00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b097fa640 1 -- 172.21.15.19:0/461924318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0052d0 con 0x7f9b0c0b92a0 2024-04-25T10:04:01.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b097fa640 1 -- 172.21.15.19:0/461924318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc005490 con 0x7f9b0c0b92a0 2024-04-25T10:04:01.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b097fa640 1 -- 172.21.15.19:0/461924318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc007260 con 0x7f9b0c0b92a0 2024-04-25T10:04:01.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.927+0000 7f9b12af5640 1 -- 172.21.15.19:0/461924318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0c0cae60 con 0x7f9b0c0b92a0 2024-04-25T10:04:01.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.928+0000 7f9b12af5640 1 -- 172.21.15.19:0/461924318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0c0cb420 con 0x7f9b0c0b92a0 2024-04-25T10:04:01.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.928+0000 7f9b12af5640 1 -- 172.21.15.19:0/461924318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0c073990 con 0x7f9b0c0b92a0 2024-04-25T10:04:01.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.929+0000 7f9b097fa640 1 -- 172.21.15.19:0/461924318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9afc007400 con 0x7f9b0c0b92a0 2024-04-25T10:04:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.930+0000 7f9b097fa640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9aec06b4d0 0x7f9aec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.930+0000 7f9b097fa640 1 -- 172.21.15.19:0/461924318 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9afc089780 con 0x7f9b0c0b92a0 2024-04-25T10:04:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.930+0000 7f9b10891640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9aec06b4d0 0x7f9aec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.930+0000 7f9b10891640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9aec06b4d0 0x7f9aec06d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9b04033000 tx=0x7f9b04009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:01.934+0000 7f9b097fa640 1 -- 172.21.15.19:0/461924318 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9afc058500 con 0x7f9b0c0b92a0 2024-04-25T10:04:02.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.151+0000 7f9b12af5640 1 -- 172.21.15.19:0/461924318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b0c15d1b0 con 0x7f9b0c0b92a0 2024-04-25T10:04:02.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.151+0000 7f9b097fa640 1 -- 172.21.15.19:0/461924318 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f9afc057e00 con 0x7f9b0c0b92a0 2024-04-25T10:04:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:02.157 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":102,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3725393673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3725393673},{"type":"v1","addr":"172.21.15.112:6837","nonce":3725393673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8866,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2539453196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2539453196},{"type":"v1","addr":"172.21.15.112:6836","nonce":2539453196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10221,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3896097924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3896097924},{"type":"v1","addr":"172.21.15.19:6834","nonce":3896097924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:14.030067+0000","modified":"2024-04-25T10:03:17.045113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10224":{"gid":10224,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/717842662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":717842662},{"type":"v1","addr":"172.21.15.19:6835","nonce":717842662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:04:02.157 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 102 2024-04-25T10:04:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.153+0000 7f9b12af5640 1 -- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9aec06b4d0 msgr2=0x7f9aec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.153+0000 7f9b12af5640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9aec06b4d0 0x7f9aec06d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9b04033000 tx=0x7f9b04009000 comp rx=0 tx=0).stop 2024-04-25T10:04:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.153+0000 7f9b12af5640 1 -- 172.21.15.19:0/461924318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b0c0b92a0 msgr2=0x7f9b0c0cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.153+0000 7f9b12af5640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b0c0b92a0 0x7f9b0c0cc6e0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00c470 tx=0x7f9afc00c930 comp rx=0 tx=0).stop 2024-04-25T10:04:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.153+0000 7f9b12af5640 1 -- 172.21.15.19:0/461924318 shutdown_connections 2024-04-25T10:04:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.153+0000 7f9b12af5640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9aec06b4d0 0x7f9aec06d990 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.153+0000 7f9b12af5640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b0c15dbe0 0x7f9b0c0b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.153+0000 7f9b12af5640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b0c0b92a0 0x7f9b0c0cc6e0 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.153+0000 7f9b12af5640 1 --2- 172.21.15.19:0/461924318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b0c0b2cb0 0x7f9b0c0c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.154+0000 7f9b12af5640 1 -- 172.21.15.19:0/461924318 >> 172.21.15.19:0/461924318 conn(0x7f9b0c061a20 msgr2=0x7f9b0c15b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.154+0000 7f9b12af5640 1 -- 172.21.15.19:0/461924318 shutdown_connections 2024-04-25T10:04:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.154+0000 7f9b12af5640 1 -- 172.21.15.19:0/461924318 wait complete. 2024-04-25T10:04:02.169 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.265+0000 7f35c7893640 1 Processor -- start 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.265+0000 7f35c7893640 1 -- start start 2024-04-25T10:04:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.266+0000 7f35c7893640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f35c006abe0 0x7f35c006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.266+0000 7f35c7893640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c01626e0 con 0x7f35c00ce330 2024-04-25T10:04:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.266+0000 7f35c7893640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c006b500 con 0x7f35c015d370 2024-04-25T10:04:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.266+0000 7f35c7893640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c0070270 con 0x7f35c006abe0 2024-04-25T10:04:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.266+0000 7f35c4e2e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f35c00ce330 0x7f35c006fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56994/0 (socket says 172.21.15.19:56994) 2024-04-25T10:04:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.266+0000 7f35c4e2e640 1 -- 172.21.15.19:0/1179060548 learned_addr learned my addr 172.21.15.19:0/1179060548 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.266+0000 7f35c562f640 1 --2- 172.21.15.19:0/1179060548 >> v2:172.21.15.112:3300/0 conn(0x7f35c006abe0 0x7f35c006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079731588 0 0) 0x7f35c01626e0 con 0x7f35c00ce330 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35ac003520 con 0x7f35c00ce330 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382830174 0 0) 0x7f35c006b500 con 0x7f35c015d370 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c01626e0 con 0x7f35c015d370 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 965458705 0 0) 0x7f35ac003520 con 0x7f35c00ce330 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 >> v2:172.21.15.112:3300/0 conn(0x7f35c006abe0 msgr2=0x7f35c006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 --2- 172.21.15.19:0/1179060548 >> v2:172.21.15.112:3300/0 conn(0x7f35c006abe0 0x7f35c006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 >> v1:172.21.15.112:6789/0 conn(0x7f35c015d370 legacy=0x7f35c015d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c00cd270 con 0x7f35c00ce330 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f35b0001670 con 0x7f35c00ce330 2024-04-25T10:04:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 >> v1:172.21.15.19:6789/0 conn(0x7f35c00ce330 legacy=0x7f35c006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35ac003b40 0x7f35ac003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ac004800 0x7f35ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ac0051c0 0x7f35ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c0070270 con 0x7f35ac003b40 2024-04-25T10:04:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac00c0f0 con 0x7f35ac004800 2024-04-25T10:04:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac00c2f0 con 0x7f35ac0051c0 2024-04-25T10:04:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35c562f640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35ac003b40 0x7f35ac003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35c5e30640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ac0051c0 0x7f35ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35c4e2e640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ac004800 0x7f35ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35c562f640 1 -- 172.21.15.19:0/1179060548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ac0051c0 msgr2=0x7f35ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35c562f640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35ac0051c0 0x7f35ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35c562f640 1 -- 172.21.15.19:0/1179060548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ac004800 msgr2=0x7f35ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35c562f640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ac004800 0x7f35ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35c562f640 1 -- 172.21.15.19:0/1179060548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac003520 con 0x7f35ac003b40 2024-04-25T10:04:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35c5e30640 1 -- 172.21.15.19:0/1179060548 reap_dead start 2024-04-25T10:04:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.267+0000 7f35c4e2e640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ac004800 0x7f35ac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.268+0000 7f35c562f640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35ac003b40 0x7f35ac003f40 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009ac0 tx=0x7f35ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7122fa099b6da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.268+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc036070 con 0x7f35ac003b40 2024-04-25T10:04:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.268+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc031400 con 0x7f35ac003b40 2024-04-25T10:04:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.268+0000 7f35b7fff640 1 -- 172.21.15.19:0/1179060548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc009690 con 0x7f35ac003b40 2024-04-25T10:04:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.269+0000 7f35c7893640 1 -- 172.21.15.19:0/1179060548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35ac003b40 msgr2=0x7f35ac003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.269+0000 7f35c7893640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35ac003b40 0x7f35ac003f40 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009ac0 tx=0x7f35ac005830 comp rx=0 tx=0).stop 2024-04-25T10:04:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.269+0000 7f35c7893640 1 -- 172.21.15.19:0/1179060548 shutdown_connections 2024-04-25T10:04:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.269+0000 7f35c7893640 1 --2- 172.21.15.19:0/1179060548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35ac003b40 0x7f35ac003f40 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.269+0000 7f35c7893640 1 -- 172.21.15.19:0/1179060548 >> 172.21.15.19:0/1179060548 conn(0x7f35c00b2220 msgr2=0x7f35c00b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.269+0000 7f35c7893640 1 -- 172.21.15.19:0/1179060548 shutdown_connections 2024-04-25T10:04:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.269+0000 7f35c7893640 1 -- 172.21.15.19:0/1179060548 wait complete. 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.270+0000 7f35c7893640 1 Processor -- start 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.270+0000 7f35c7893640 1 -- start start 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c7893640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ac003b40 0x7f35c00ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c7893640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35c00ca8b0 0x7f35c00ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c7893640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35c00b5a20 0x7f35c00adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c7893640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c00cdb10 con 0x7f35c00ca8b0 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c7893640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c00cd830 con 0x7f35ac003b40 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c7893640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c0161780 con 0x7f35c00b5a20 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c4e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35c00ca8b0 0x7f35c00ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c4e2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35c00ca8b0 0x7f35c00ada40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49340/0 (socket says 172.21.15.19:49340) 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c4e2e640 1 -- 172.21.15.19:0/3375771222 learned_addr learned my addr 172.21.15.19:0/3375771222 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c562f640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ac003b40 0x7f35c00ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c5e30640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35c00b5a20 0x7f35c00adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c4e2e640 1 -- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35c00b5a20 msgr2=0x7f35c00adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c4e2e640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35c00b5a20 0x7f35c00adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c4e2e640 1 -- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ac003b40 msgr2=0x7f35c00ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c4e2e640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ac003b40 0x7f35c00ad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c4e2e640 1 -- 172.21.15.19:0/3375771222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c00b5f20 con 0x7f35c00ca8b0 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.271+0000 7f35c4e2e640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35c00ca8b0 0x7f35c00ada40 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f35b000e780 tx=0x7f35b000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.272+0000 7f35b5ffb640 1 -- 172.21.15.19:0/3375771222 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b000c660 con 0x7f35c00ca8b0 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.272+0000 7f35b5ffb640 1 -- 172.21.15.19:0/3375771222 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b0018070 con 0x7f35c00ca8b0 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.272+0000 7f35c7893640 1 -- 172.21.15.19:0/3375771222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c00b6200 con 0x7f35c00ca8b0 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.272+0000 7f35b5ffb640 1 -- 172.21.15.19:0/3375771222 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b000ca20 con 0x7f35c00ca8b0 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.272+0000 7f35c7893640 1 -- 172.21.15.19:0/3375771222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c00ca0b0 con 0x7f35c00ca8b0 2024-04-25T10:04:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.273+0000 7f35c7893640 1 -- 172.21.15.19:0/3375771222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c0073990 con 0x7f35c00ca8b0 2024-04-25T10:04:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.273+0000 7f35b5ffb640 1 -- 172.21.15.19:0/3375771222 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35b000cbc0 con 0x7f35c00ca8b0 2024-04-25T10:04:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.273+0000 7f35b5ffb640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35a406b4d0 0x7f35a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.274+0000 7f35b5ffb640 1 -- 172.21.15.19:0/3375771222 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f35b0089f50 con 0x7f35c00ca8b0 2024-04-25T10:04:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.274+0000 7f35c562f640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35a406b4d0 0x7f35a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.274+0000 7f35c562f640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35a406b4d0 0x7f35a406d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f35bc033000 tx=0x7f35bc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.278+0000 7f35b5ffb640 1 -- 172.21.15.19:0/3375771222 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f35b0058cd0 con 0x7f35c00ca8b0 2024-04-25T10:04:02.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.460+0000 7f35c7893640 1 -- 172.21.15.19:0/3375771222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35c00b2ff0 con 0x7f35c00ca8b0 2024-04-25T10:04:02.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.461+0000 7f35b5ffb640 1 -- 172.21.15.19:0/3375771222 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f35b00585d0 con 0x7f35c00ca8b0 2024-04-25T10:04:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:02.467 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":154,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:03:15.658204+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:12.307052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:13.005066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 -- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35a406b4d0 msgr2=0x7f35a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35a406b4d0 0x7f35a406d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f35bc033000 tx=0x7f35bc008030 comp rx=0 tx=0).stop 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 -- 172.21.15.19:0/3375771222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35c00ca8b0 msgr2=0x7f35c00ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35c00ca8b0 0x7f35c00ada40 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f35b000e780 tx=0x7f35b000eb80 comp rx=0 tx=0).stop 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 -- 172.21.15.19:0/3375771222 shutdown_connections 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f35a406b4d0 0x7f35a406d990 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35c00b5a20 0x7f35c00adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35c00ca8b0 0x7f35c00ada40 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 --2- 172.21.15.19:0/3375771222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35ac003b40 0x7f35c00ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 -- 172.21.15.19:0/3375771222 >> 172.21.15.19:0/3375771222 conn(0x7f35c00b2220 msgr2=0x7f35c0158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 -- 172.21.15.19:0/3375771222 shutdown_connections 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.463+0000 7f35c7893640 1 -- 172.21.15.19:0/3375771222 wait complete. 2024-04-25T10:04:02.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.576+0000 7fc3aee8f640 1 Processor -- start 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.576+0000 7fc3aee8f640 1 -- start start 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.577+0000 7fc3aee8f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3a815de80 0x7fc3a80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.577+0000 7fc3aee8f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3a80ccfe0 0x7fc3a80cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.577+0000 7fc3aee8f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a80afc80 con 0x7fc3a80cefb0 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.577+0000 7fc3aee8f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a80ce4d0 con 0x7fc3a815de80 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.577+0000 7fc3aee8f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80ce7b0 con 0x7fc3a80ccfe0 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.577+0000 7fc3acc2b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3a815de80 0x7fc3a80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.577+0000 7fc3acc2b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3a815de80 0x7fc3a80cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49348/0 (socket says 172.21.15.19:49348) 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.577+0000 7fc3acc2b640 1 -- 172.21.15.19:0/577649402 learned_addr learned my addr 172.21.15.19:0/577649402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.578+0000 7fc3ad42c640 1 --2- 172.21.15.19:0/577649402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3a80ccfe0 0x7fc3a80cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.578+0000 7fc3acc2b640 1 -- 172.21.15.19:0/577649402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3a80ccfe0 msgr2=0x7fc3a80cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.578+0000 7fc3acc2b640 1 --2- 172.21.15.19:0/577649402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3a80ccfe0 0x7fc3a80cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.578+0000 7fc3acc2b640 1 -- 172.21.15.19:0/577649402 >> v1:172.21.15.112:6789/0 conn(0x7fc3a80cefb0 legacy=0x7fc3a80cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.578+0000 7fc3acc2b640 1 -- 172.21.15.19:0/577649402 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a80cea90 con 0x7fc3a815de80 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.578+0000 7fc3ad42c640 1 --2- 172.21.15.19:0/577649402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3a80ccfe0 0x7fc3a80cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.578+0000 7fc3acc2b640 1 --2- 172.21.15.19:0/577649402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3a815de80 0x7fc3a80cadb0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009e50 tx=0x7fc3a002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb2dc27865fa0e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.579+0000 7fc3a77fe640 1 -- 172.21.15.19:0/577649402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0036070 con 0x7fc3a815de80 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.579+0000 7fc3a77fe640 1 -- 172.21.15.19:0/577649402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a0031600 con 0x7fc3a815de80 2024-04-25T10:04:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.579+0000 7fc3a77fe640 1 -- 172.21.15.19:0/577649402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00319d0 con 0x7fc3a815de80 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.579+0000 7fc3aee8f640 1 -- 172.21.15.19:0/577649402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3a815de80 msgr2=0x7fc3a80cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.579+0000 7fc3aee8f640 1 --2- 172.21.15.19:0/577649402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3a815de80 0x7fc3a80cadb0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009e50 tx=0x7fc3a002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.579+0000 7fc3aee8f640 1 -- 172.21.15.19:0/577649402 shutdown_connections 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.579+0000 7fc3aee8f640 1 --2- 172.21.15.19:0/577649402 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3a80ccfe0 0x7fc3a80cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.579+0000 7fc3aee8f640 1 --2- 172.21.15.19:0/577649402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3a815de80 0x7fc3a80cadb0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.579+0000 7fc3aee8f640 1 -- 172.21.15.19:0/577649402 >> 172.21.15.19:0/577649402 conn(0x7fc3a80b2390 msgr2=0x7fc3a80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.579+0000 7fc3aee8f640 1 -- 172.21.15.19:0/577649402 shutdown_connections 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.579+0000 7fc3aee8f640 1 -- 172.21.15.19:0/577649402 wait complete. 2024-04-25T10:04:02.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.580+0000 7fc3aee8f640 1 Processor -- start 2024-04-25T10:04:02.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.581+0000 7fc3aee8f640 1 -- start start 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.581+0000 7fc3aee8f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3a80ccfe0 0x7fc3a815b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.581+0000 7fc3aee8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3a80cefb0 0x7fc3a815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.581+0000 7fc3aee8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3a815de80 0x7fc3a80b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.581+0000 7fc3aee8f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80cfb50 con 0x7fc3a80cefb0 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.581+0000 7fc3aee8f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80b2eb0 con 0x7fc3a815de80 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.581+0000 7fc3aee8f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80c8400 con 0x7fc3a80ccfe0 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.581+0000 7fc3a7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3a80cefb0 0x7fc3a815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3a7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3a80cefb0 0x7fc3a815bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49362/0 (socket says 172.21.15.19:49362) 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3a7fff640 1 -- 172.21.15.19:0/4194403327 learned_addr learned my addr 172.21.15.19:0/4194403327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3acc2b640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3a80ccfe0 0x7fc3a815b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3ad42c640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3a815de80 0x7fc3a80b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3a7fff640 1 -- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3a80ccfe0 msgr2=0x7fc3a815b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3a7fff640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3a80ccfe0 0x7fc3a815b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3a7fff640 1 -- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3a815de80 msgr2=0x7fc3a80b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3a7fff640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3a815de80 0x7fc3a80b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3a7fff640 1 -- 172.21.15.19:0/4194403327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a815c7b0 con 0x7fc3a80cefb0 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3ad42c640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3a815de80 0x7fc3a80b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3a7fff640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3a80cefb0 0x7fc3a815bf30 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c430 tx=0x7fc39c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3a57fa640 1 -- 172.21.15.19:0/4194403327 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c011790 con 0x7fc3a80cefb0 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3a57fa640 1 -- 172.21.15.19:0/4194403327 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c017070 con 0x7fc3a80cefb0 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3aee8f640 1 -- 172.21.15.19:0/4194403327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a80c7ad0 con 0x7fc3a80cefb0 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3a57fa640 1 -- 172.21.15.19:0/4194403327 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c011bb0 con 0x7fc3a80cefb0 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.582+0000 7fc3aee8f640 1 -- 172.21.15.19:0/4194403327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a80c8090 con 0x7fc3a80cefb0 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.583+0000 7fc3aee8f640 1 -- 172.21.15.19:0/4194403327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a8071b70 con 0x7fc3a80cefb0 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.584+0000 7fc3a57fa640 1 -- 172.21.15.19:0/4194403327 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc39c01c460 con 0x7fc3a80cefb0 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.584+0000 7fc3a57fa640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc39406b500 0x7fc39406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.584+0000 7fc3a57fa640 1 -- 172.21.15.19:0/4194403327 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc39c08a140 con 0x7fc3a80cefb0 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.584+0000 7fc3acc2b640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc39406b500 0x7fc39406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.585+0000 7fc3acc2b640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc39406b500 0x7fc39406d9c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0033000 tx=0x7fc3a0009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.588+0000 7fc3a57fa640 1 -- 172.21.15.19:0/4194403327 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3a8071b70 con 0x7fc3a80cefb0 2024-04-25T10:04:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.810+0000 7fc3aee8f640 1 -- 172.21.15.19:0/4194403327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc3a8067be0 con 0x7fc3a80cefb0 2024-04-25T10:04:02.815 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T10:04:02.811+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:04:02.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.812+0000 7fc3a57fa640 1 -- 172.21.15.19:0/4194403327 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v103) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc3a8067be0 con 0x7fc3a80cefb0 2024-04-25T10:04:02.817 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:04:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 -- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc39406b500 msgr2=0x7fc39406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc39406b500 0x7fc39406d9c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0033000 tx=0x7fc3a0009000 comp rx=0 tx=0).stop 2024-04-25T10:04:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 -- 172.21.15.19:0/4194403327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3a80cefb0 msgr2=0x7fc3a815bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3a80cefb0 0x7fc3a815bf30 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c430 tx=0x7fc39c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 -- 172.21.15.19:0/4194403327 shutdown_connections 2024-04-25T10:04:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc39406b500 0x7fc39406d9c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3a815de80 0x7fc3a80b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3a80cefb0 0x7fc3a815bf30 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 --2- 172.21.15.19:0/4194403327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3a80ccfe0 0x7fc3a815b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 -- 172.21.15.19:0/4194403327 >> 172.21.15.19:0/4194403327 conn(0x7fc3a80b2390 msgr2=0x7fc3a806e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 -- 172.21.15.19:0/4194403327 shutdown_connections 2024-04-25T10:04:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.815+0000 7fc3aee8f640 1 -- 172.21.15.19:0/4194403327 wait complete. 2024-04-25T10:04:02.822 INFO:tasks.ceph.mds.a.smithi019.stderr: -3408> 2024-04-25T10:03:20.312+0000 7f79d1af1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:04:02.831 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:04:02.851 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T10:04:02.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.934+0000 7f711a476640 1 Processor -- start 2024-04-25T10:04:02.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.934+0000 7f711a476640 1 -- start start 2024-04-25T10:04:02.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.934+0000 7f711a476640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71141616f0 0x7f7114161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.934+0000 7f711a476640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711406c8d0 con 0x7f71141610f0 2024-04-25T10:04:02.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.934+0000 7f711a476640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71140c6960 con 0x7f71140ce0a0 2024-04-25T10:04:02.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.934+0000 7f711a476640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71141614d0 con 0x7f71141616f0 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7113fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f71141610f0 0x7f711406c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57004/0 (socket says 172.21.15.19:57004) 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7113fff640 1 -- 172.21.15.19:0/2959472175 learned_addr learned my addr 172.21.15.19:0/2959472175 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f71137fe640 1 --2- 172.21.15.19:0/2959472175 >> v2:172.21.15.112:3300/0 conn(0x7f71141616f0 0x7f7114161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232697354 0 0) 0x7f711406c8d0 con 0x7f71141610f0 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70fc003520 con 0x7f71141610f0 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577269238 0 0) 0x7f71140c6960 con 0x7f71140ce0a0 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f711406c8d0 con 0x7f71140ce0a0 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 750712819 0 0) 0x7f70fc003520 con 0x7f71141610f0 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 >> v2:172.21.15.112:3300/0 conn(0x7f71141616f0 msgr2=0x7f7114161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 --2- 172.21.15.19:0/2959472175 >> v2:172.21.15.112:3300/0 conn(0x7f71141616f0 0x7f7114161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 >> v1:172.21.15.112:6789/0 conn(0x7f71140ce0a0 legacy=0x7f71140d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7114162260 con 0x7f71141610f0 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f7104001670 con 0x7f71141610f0 2024-04-25T10:04:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f71137fe640 1 --2- 172.21.15.19:0/2959472175 >> v2:172.21.15.112:3300/0 conn(0x7f71141616f0 0x7f7114161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 >> v1:172.21.15.19:6789/0 conn(0x7f71141610f0 legacy=0x7f711406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 --2- 172.21.15.19:0/2959472175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70fc003b40 0x7f70fc003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 --2- 172.21.15.19:0/2959472175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc004800 0x7f70fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 --2- 172.21.15.19:0/2959472175 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc0051c0 0x7f70fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71141614d0 con 0x7f70fc004800 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc00c040 con 0x7f70fc003b40 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc00c240 con 0x7f70fc0051c0 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f71137fe640 1 --2- 172.21.15.19:0/2959472175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc004800 0x7f70fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7113fff640 1 --2- 172.21.15.19:0/2959472175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70fc003b40 0x7f70fc003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.935+0000 7f7118a13640 1 --2- 172.21.15.19:0/2959472175 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc0051c0 0x7f70fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.936+0000 7f71137fe640 1 -- 172.21.15.19:0/2959472175 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc0051c0 msgr2=0x7f70fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:02.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.936+0000 7f71137fe640 1 --2- 172.21.15.19:0/2959472175 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc0051c0 0x7f70fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.936+0000 7f71137fe640 1 -- 172.21.15.19:0/2959472175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70fc003b40 msgr2=0x7f70fc003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.936+0000 7f71137fe640 1 --2- 172.21.15.19:0/2959472175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70fc003b40 0x7f70fc003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.936+0000 7f71137fe640 1 -- 172.21.15.19:0/2959472175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc003520 con 0x7f70fc004800 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.936+0000 7f7118a13640 1 -- 172.21.15.19:0/2959472175 reap_dead start 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.936+0000 7f71137fe640 1 --2- 172.21.15.19:0/2959472175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc004800 0x7f70fc004c80 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f7108009c90 tx=0x7f70fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436355bdcf980cff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.936+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108036070 con 0x7f70fc004800 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.937+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108031400 con 0x7f70fc004800 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.937+0000 7f7112ffd640 1 -- 172.21.15.19:0/2959472175 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108009860 con 0x7f70fc004800 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.937+0000 7f711a476640 1 -- 172.21.15.19:0/2959472175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc004800 msgr2=0x7f70fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.937+0000 7f711a476640 1 --2- 172.21.15.19:0/2959472175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc004800 0x7f70fc004c80 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f7108009c90 tx=0x7f70fc005830 comp rx=0 tx=0).stop 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.937+0000 7f711a476640 1 -- 172.21.15.19:0/2959472175 shutdown_connections 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.937+0000 7f711a476640 1 --2- 172.21.15.19:0/2959472175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70fc004800 0x7f70fc004c80 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.937+0000 7f711a476640 1 -- 172.21.15.19:0/2959472175 >> 172.21.15.19:0/2959472175 conn(0x7f711406f500 msgr2=0x7f711406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.937+0000 7f711a476640 1 -- 172.21.15.19:0/2959472175 shutdown_connections 2024-04-25T10:04:02.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.937+0000 7f711a476640 1 -- 172.21.15.19:0/2959472175 wait complete. 2024-04-25T10:04:02.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.938+0000 7f711a476640 1 Processor -- start 2024-04-25T10:04:02.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.939+0000 7f711a476640 1 -- start start 2024-04-25T10:04:02.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.939+0000 7f711a476640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc004800 0x7f71140b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.939+0000 7f711a476640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71140b66f0 0x7f71140b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.939+0000 7f711a476640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114149170 0x7f71141495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.939+0000 7f711a476640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71140cfce0 con 0x7f7114149170 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.939+0000 7f711a476640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71140cfa00 con 0x7f71140b66f0 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.939+0000 7f711a476640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71140cf690 con 0x7f70fc004800 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.939+0000 7f7118a13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114149170 0x7f71141495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.939+0000 7f7118a13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114149170 0x7f71141495f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49406/0 (socket says 172.21.15.19:49406) 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.939+0000 7f7118a13640 1 -- 172.21.15.19:0/1452408464 learned_addr learned my addr 172.21.15.19:0/1452408464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f71137fe640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71140b66f0 0x7f71140b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f7113fff640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc004800 0x7f71140b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f7118a13640 1 -- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc004800 msgr2=0x7f71140b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f7118a13640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc004800 0x7f71140b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f7118a13640 1 -- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71140b66f0 msgr2=0x7f71140b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f7118a13640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71140b66f0 0x7f71140b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f7118a13640 1 -- 172.21.15.19:0/1452408464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711414a0d0 con 0x7f7114149170 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f71137fe640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71140b66f0 0x7f71140b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f7118a13640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114149170 0x7f71141495f0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f710c00fae0 tx=0x7f710c0020b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f7110ff9640 1 -- 172.21.15.19:0/1452408464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c019070 con 0x7f7114149170 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f7110ff9640 1 -- 172.21.15.19:0/1452408464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c006e20 con 0x7f7114149170 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f711a476640 1 -- 172.21.15.19:0/1452408464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7114149b30 con 0x7f7114149170 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f7110ff9640 1 -- 172.21.15.19:0/1452408464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c003b90 con 0x7f7114149170 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.940+0000 7f711a476640 1 -- 172.21.15.19:0/1452408464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71141471f0 con 0x7f7114149170 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.941+0000 7f711a476640 1 -- 172.21.15.19:0/1452408464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7114073a00 con 0x7f7114149170 2024-04-25T10:04:02.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.941+0000 7f7110ff9640 1 -- 172.21.15.19:0/1452408464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f710c01c070 con 0x7f7114149170 2024-04-25T10:04:02.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.942+0000 7f7110ff9640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f710006b4d0 0x7f710006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.942+0000 7f7110ff9640 1 -- 172.21.15.19:0/1452408464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f710c089630 con 0x7f7114149170 2024-04-25T10:04:02.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.942+0000 7f7113fff640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f710006b4d0 0x7f710006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.943+0000 7f7113fff640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f710006b4d0 0x7f710006d990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7104007c00 tx=0x7f7104009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:02.946+0000 7f7110ff9640 1 -- 172.21.15.19:0/1452408464 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f710c058240 con 0x7f7114149170 2024-04-25T10:04:03.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.162+0000 7f711a476640 1 -- 172.21.15.19:0/1452408464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f71140ce9b0 con 0x7f7114149170 2024-04-25T10:04:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.821+0000 7f7110ff9640 1 -- 172.21.15.19:0/1452408464 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v104) v1 ==== 104+0+0 (secure 0 0 0) 0x7f710c057b40 con 0x7f7114149170 2024-04-25T10:04:03.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 -- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f710006b4d0 msgr2=0x7f710006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f710006b4d0 0x7f710006d990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7104007c00 tx=0x7f7104009040 comp rx=0 tx=0).stop 2024-04-25T10:04:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 -- 172.21.15.19:0/1452408464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114149170 msgr2=0x7f71141495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114149170 0x7f71141495f0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f710c00fae0 tx=0x7f710c0020b0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 -- 172.21.15.19:0/1452408464 shutdown_connections 2024-04-25T10:04:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f710006b4d0 0x7f710006d990 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7114149170 0x7f71141495f0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71140b66f0 0x7f71140b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 --2- 172.21.15.19:0/1452408464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70fc004800 0x7f71140b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 -- 172.21.15.19:0/1452408464 >> 172.21.15.19:0/1452408464 conn(0x7f711406f500 msgr2=0x7f71140b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 -- 172.21.15.19:0/1452408464 shutdown_connections 2024-04-25T10:04:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.823+0000 7f711a476640 1 -- 172.21.15.19:0/1452408464 wait complete. 2024-04-25T10:04:03.842 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:04:03.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.990+0000 7f711f3f9640 1 Processor -- start 2024-04-25T10:04:03.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.990+0000 7f711f3f9640 1 -- start start 2024-04-25T10:04:03.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.991+0000 7f711f3f9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7118161890 0x7f71180b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.991+0000 7f711f3f9640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71180cf1d0 con 0x7f7118161240 2024-04-25T10:04:03.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.991+0000 7f711f3f9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7118161620 con 0x7f71180b54a0 2024-04-25T10:04:03.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.991+0000 7f711f3f9640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71180b7120 con 0x7f7118161890 2024-04-25T10:04:03.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.991+0000 7f711d195640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7118161240 0x7f71180ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57020/0 (socket says 172.21.15.19:57020) 2024-04-25T10:04:03.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.991+0000 7f711d195640 1 -- 172.21.15.19:0/304863365 learned_addr learned my addr 172.21.15.19:0/304863365 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:03.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.991+0000 7f711c994640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7118161890 0x7f71180b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.991+0000 7f711c994640 1 -- 172.21.15.19:0/304863365 >> v1:172.21.15.112:6789/0 conn(0x7f71180b54a0 legacy=0x7f71180d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:03.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.991+0000 7f711c994640 1 -- 172.21.15.19:0/304863365 >> v1:172.21.15.19:6789/0 conn(0x7f7118161240 legacy=0x7f71180ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.991+0000 7f711c994640 1 -- 172.21.15.19:0/304863365 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7118161e00 con 0x7f7118161890 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.992+0000 7f711c994640 1 --2- 172.21.15.19:0/304863365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7118161890 0x7f71180b6be0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f71080097a0 tx=0x7f710802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418c3aafb4920f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.992+0000 7f710ffff640 1 -- 172.21.15.19:0/304863365 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108037070 con 0x7f7118161890 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.992+0000 7f710ffff640 1 -- 172.21.15.19:0/304863365 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108032440 con 0x7f7118161890 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.992+0000 7f710ffff640 1 -- 172.21.15.19:0/304863365 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108032810 con 0x7f7118161890 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.993+0000 7f711f3f9640 1 -- 172.21.15.19:0/304863365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7118161890 msgr2=0x7f71180b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.993+0000 7f711f3f9640 1 --2- 172.21.15.19:0/304863365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7118161890 0x7f71180b6be0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f71080097a0 tx=0x7f710802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.993+0000 7f711f3f9640 1 -- 172.21.15.19:0/304863365 shutdown_connections 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.993+0000 7f711f3f9640 1 --2- 172.21.15.19:0/304863365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7118161890 0x7f71180b6be0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.993+0000 7f711f3f9640 1 -- 172.21.15.19:0/304863365 >> 172.21.15.19:0/304863365 conn(0x7f71180d04a0 msgr2=0x7f71180d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.993+0000 7f711f3f9640 1 -- 172.21.15.19:0/304863365 shutdown_connections 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.993+0000 7f711f3f9640 1 -- 172.21.15.19:0/304863365 wait complete. 2024-04-25T10:04:03.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.994+0000 7f711f3f9640 1 Processor -- start 2024-04-25T10:04:03.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.994+0000 7f711f3f9640 1 -- start start 2024-04-25T10:04:04.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711f3f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71180b54a0 0x7f711814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711f3f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7118161240 0x7f711814e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711f3f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7118161890 0x7f71180c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711f3f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71181626a0 con 0x7f7118161890 2024-04-25T10:04:04.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711f3f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71181623c0 con 0x7f71180b54a0 2024-04-25T10:04:04.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711f3f9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71180d15f0 con 0x7f7118161240 2024-04-25T10:04:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711d996640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7118161890 0x7f71180c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711d996640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7118161890 0x7f71180c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49424/0 (socket says 172.21.15.19:49424) 2024-04-25T10:04:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711d996640 1 -- 172.21.15.19:0/3345376882 learned_addr learned my addr 172.21.15.19:0/3345376882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711d195640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71180b54a0 0x7f711814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711c994640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7118161240 0x7f711814e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711d996640 1 -- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7118161240 msgr2=0x7f711814e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711d996640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7118161240 0x7f711814e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711d996640 1 -- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71180b54a0 msgr2=0x7f711814def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711d996640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71180b54a0 0x7f711814def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711d996640 1 -- 172.21.15.19:0/3345376882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180c8a00 con 0x7f7118161890 2024-04-25T10:04:04.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.995+0000 7f711d996640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7118161890 0x7f71180c7f60 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f711000c7b0 tx=0x7f711000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.996+0000 7f710dffb640 1 -- 172.21.15.19:0/3345376882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7110004b60 con 0x7f7118161890 2024-04-25T10:04:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.996+0000 7f710dffb640 1 -- 172.21.15.19:0/3345376882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7110004d00 con 0x7f7118161890 2024-04-25T10:04:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.996+0000 7f711f3f9640 1 -- 172.21.15.19:0/3345376882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71180cbc50 con 0x7f7118161890 2024-04-25T10:04:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.996+0000 7f710dffb640 1 -- 172.21.15.19:0/3345376882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711000aa90 con 0x7f7118161890 2024-04-25T10:04:04.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.996+0000 7f711f3f9640 1 -- 172.21.15.19:0/3345376882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71180cc210 con 0x7f7118161890 2024-04-25T10:04:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.997+0000 7f711f3f9640 1 -- 172.21.15.19:0/3345376882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7118067be0 con 0x7f7118161890 2024-04-25T10:04:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.997+0000 7f710dffb640 1 -- 172.21.15.19:0/3345376882 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f711000ac30 con 0x7f7118161890 2024-04-25T10:04:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.998+0000 7f710dffb640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.998+0000 7f710dffb640 1 -- 172.21.15.19:0/3345376882 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f711008a520 con 0x7f7118161890 2024-04-25T10:04:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.998+0000 7f711d195640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:03.998+0000 7f711d195640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70fc06b4d0 0x7f70fc06d990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7114007fd0 tx=0x7f7114007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.002+0000 7f710dffb640 1 -- 172.21.15.19:0/3345376882 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7118067be0 con 0x7f7118161890 2024-04-25T10:04:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.184+0000 7f711f3f9640 1 -- 172.21.15.19:0/3345376882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f71180b3fc0 con 0x7f7118161890 2024-04-25T10:04:04.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.824+0000 7f710dffb640 1 -- 172.21.15.19:0/3345376882 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v156) v1 ==== 181+0+0 (secure 0 0 0) 0x7f71180b3fc0 con 0x7f7118161890 2024-04-25T10:04:04.829 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:04:04.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 -- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70fc06b4d0 msgr2=0x7f70fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70fc06b4d0 0x7f70fc06d990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7114007fd0 tx=0x7f7114007e80 comp rx=0 tx=0).stop 2024-04-25T10:04:04.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 -- 172.21.15.19:0/3345376882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7118161890 msgr2=0x7f71180c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7118161890 0x7f71180c7f60 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f711000c7b0 tx=0x7f711000cc70 comp rx=0 tx=0).stop 2024-04-25T10:04:04.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 -- 172.21.15.19:0/3345376882 shutdown_connections 2024-04-25T10:04:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7118161890 0x7f71180c7f60 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7118161240 0x7f711814e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 --2- 172.21.15.19:0/3345376882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71180b54a0 0x7f711814def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 -- 172.21.15.19:0/3345376882 >> 172.21.15.19:0/3345376882 conn(0x7f71180d04a0 msgr2=0x7f711815de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 -- 172.21.15.19:0/3345376882 shutdown_connections 2024-04-25T10:04:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.828+0000 7f711f3f9640 1 -- 172.21.15.19:0/3345376882 wait complete. 2024-04-25T10:04:04.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:04:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.959+0000 7fd39927c640 1 Processor -- start 2024-04-25T10:04:04.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.959+0000 7fd39927c640 1 -- start start 2024-04-25T10:04:04.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd39927c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd39415dc20 0x7fd3940ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd39927c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd3940d32f0 0x7fd3940cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd39927c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39415e000 con 0x7fd394161710 2024-04-25T10:04:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd39927c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3940d1de0 con 0x7fd39415dc20 2024-04-25T10:04:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd39927c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd394161c60 con 0x7fd3940d32f0 2024-04-25T10:04:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd392ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd39415dc20 0x7fd3940ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd3927fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd394161710 0x7fd39406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57036/0 (socket says 172.21.15.19:57036) 2024-04-25T10:04:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd3927fc640 1 -- 172.21.15.19:0/2367645268 learned_addr learned my addr 172.21.15.19:0/2367645268 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd3937fe640 1 --2- 172.21.15.19:0/2367645268 >> v2:172.21.15.112:3300/0 conn(0x7fd3940d32f0 0x7fd3940cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd392ffd640 1 -- 172.21.15.19:0/2367645268 >> v2:172.21.15.112:3300/0 conn(0x7fd3940d32f0 msgr2=0x7fd3940cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd392ffd640 1 --2- 172.21.15.19:0/2367645268 >> v2:172.21.15.112:3300/0 conn(0x7fd3940d32f0 0x7fd3940cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd392ffd640 1 -- 172.21.15.19:0/2367645268 >> v1:172.21.15.19:6789/0 conn(0x7fd394161710 legacy=0x7fd39406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd392ffd640 1 -- 172.21.15.19:0/2367645268 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd394161f40 con 0x7fd39415dc20 2024-04-25T10:04:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.960+0000 7fd391ffb640 1 -- 172.21.15.19:0/2367645268 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045727233 0 0) 0x7fd39415e000 con 0x7fd394161710 2024-04-25T10:04:04.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.961+0000 7fd3937fe640 1 --2- 172.21.15.19:0/2367645268 >> v2:172.21.15.112:3300/0 conn(0x7fd3940d32f0 0x7fd3940cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.961+0000 7fd392ffd640 1 --2- 172.21.15.19:0/2367645268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39415dc20 0x7fd3940ce670 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fd3940b6910 tx=0x7fd38c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684a101d0ba9329a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.961+0000 7fd391ffb640 1 -- 172.21.15.19:0/2367645268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c037070 con 0x7fd39415dc20 2024-04-25T10:04:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.961+0000 7fd391ffb640 1 -- 172.21.15.19:0/2367645268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38c032440 con 0x7fd39415dc20 2024-04-25T10:04:04.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.961+0000 7fd391ffb640 1 -- 172.21.15.19:0/2367645268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c032810 con 0x7fd39415dc20 2024-04-25T10:04:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.962+0000 7fd39927c640 1 -- 172.21.15.19:0/2367645268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39415dc20 msgr2=0x7fd3940ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.962+0000 7fd39927c640 1 --2- 172.21.15.19:0/2367645268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39415dc20 0x7fd3940ce670 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fd3940b6910 tx=0x7fd38c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.962+0000 7fd39927c640 1 -- 172.21.15.19:0/2367645268 shutdown_connections 2024-04-25T10:04:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.962+0000 7fd39927c640 1 --2- 172.21.15.19:0/2367645268 >> v2:172.21.15.112:3300/0 conn(0x7fd3940d32f0 0x7fd3940cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.962+0000 7fd39927c640 1 --2- 172.21.15.19:0/2367645268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39415dc20 0x7fd3940ce670 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.962+0000 7fd39927c640 1 -- 172.21.15.19:0/2367645268 >> 172.21.15.19:0/2367645268 conn(0x7fd3940d0420 msgr2=0x7fd3940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.962+0000 7fd39927c640 1 -- 172.21.15.19:0/2367645268 shutdown_connections 2024-04-25T10:04:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.962+0000 7fd39927c640 1 -- 172.21.15.19:0/2367645268 wait complete. 2024-04-25T10:04:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.963+0000 7fd39927c640 1 Processor -- start 2024-04-25T10:04:04.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.963+0000 7fd39927c640 1 -- start start 2024-04-25T10:04:04.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd39927c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3940d32f0 0x7fd3940c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd39927c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39415dc20 0x7fd394149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd39927c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd394161710 0x7fd39414b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd39927c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940b5e50 con 0x7fd39415dc20 2024-04-25T10:04:04.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd39927c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd394162500 con 0x7fd394161710 2024-04-25T10:04:04.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd39927c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940b5fd0 con 0x7fd3940d32f0 2024-04-25T10:04:04.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd3927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39415dc20 0x7fd394149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd3927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39415dc20 0x7fd394149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49446/0 (socket says 172.21.15.19:49446) 2024-04-25T10:04:04.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd3927fc640 1 -- 172.21.15.19:0/1157108324 learned_addr learned my addr 172.21.15.19:0/1157108324 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd392ffd640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3940d32f0 0x7fd3940c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd3937fe640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd394161710 0x7fd39414b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd3927fc640 1 -- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3940d32f0 msgr2=0x7fd3940c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd3927fc640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3940d32f0 0x7fd3940c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.964+0000 7fd3927fc640 1 -- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd394161710 msgr2=0x7fd39414b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.965+0000 7fd3927fc640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd394161710 0x7fd39414b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.965+0000 7fd3927fc640 1 -- 172.21.15.19:0/1157108324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3941f3260 con 0x7fd39415dc20 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.965+0000 7fd392ffd640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3940d32f0 0x7fd3940c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.965+0000 7fd3937fe640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd394161710 0x7fd39414b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.965+0000 7fd3927fc640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39415dc20 0x7fd394149b90 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c430 tx=0x7fd38800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.965+0000 7fd37b7fe640 1 -- 172.21.15.19:0/1157108324 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3880074e0 con 0x7fd39415dc20 2024-04-25T10:04:04.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.965+0000 7fd37b7fe640 1 -- 172.21.15.19:0/1157108324 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3880047e0 con 0x7fd39415dc20 2024-04-25T10:04:04.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.965+0000 7fd39927c640 1 -- 172.21.15.19:0/1157108324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39414a1e0 con 0x7fd39415dc20 2024-04-25T10:04:04.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.965+0000 7fd37b7fe640 1 -- 172.21.15.19:0/1157108324 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388004bd0 con 0x7fd39415dc20 2024-04-25T10:04:04.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.965+0000 7fd39927c640 1 -- 172.21.15.19:0/1157108324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39414a7a0 con 0x7fd39415dc20 2024-04-25T10:04:04.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.966+0000 7fd39927c640 1 -- 172.21.15.19:0/1157108324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd394073a00 con 0x7fd39415dc20 2024-04-25T10:04:04.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.966+0000 7fd37b7fe640 1 -- 172.21.15.19:0/1157108324 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd388011070 con 0x7fd39415dc20 2024-04-25T10:04:04.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.967+0000 7fd37b7fe640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd38006b4d0 0x7fd38006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.967+0000 7fd37b7fe640 1 -- 172.21.15.19:0/1157108324 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fd388089e50 con 0x7fd39415dc20 2024-04-25T10:04:04.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.967+0000 7fd392ffd640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd38006b4d0 0x7fd38006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.968+0000 7fd392ffd640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd38006b4d0 0x7fd38006d990 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd38c034000 tx=0x7fd38c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:04.971+0000 7fd37b7fe640 1 -- 172.21.15.19:0/1157108324 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd388058ca0 con 0x7fd39415dc20 2024-04-25T10:04:05.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.154+0000 7fd39927c640 1 -- 172.21.15.19:0/1157108324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd39406b010 con 0x7fd39415dc20 2024-04-25T10:04:05.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.830+0000 7fd37b7fe640 1 -- 172.21.15.19:0/1157108324 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v157) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd3880585a0 con 0x7fd39415dc20 2024-04-25T10:04:05.835 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T10:04:05.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.833+0000 7fd39927c640 1 -- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd38006b4d0 msgr2=0x7fd38006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.833+0000 7fd39927c640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd38006b4d0 0x7fd38006d990 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd38c034000 tx=0x7fd38c02e000 comp rx=0 tx=0).stop 2024-04-25T10:04:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.833+0000 7fd39927c640 1 -- 172.21.15.19:0/1157108324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39415dc20 msgr2=0x7fd394149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.833+0000 7fd39927c640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39415dc20 0x7fd394149b90 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c430 tx=0x7fd38800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.833+0000 7fd39927c640 1 -- 172.21.15.19:0/1157108324 shutdown_connections 2024-04-25T10:04:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.833+0000 7fd39927c640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd38006b4d0 0x7fd38006d990 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.833+0000 7fd39927c640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd394161710 0x7fd39414b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.833+0000 7fd39927c640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39415dc20 0x7fd394149b90 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.833+0000 7fd39927c640 1 --2- 172.21.15.19:0/1157108324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd3940d32f0 0x7fd3940c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.833+0000 7fd39927c640 1 -- 172.21.15.19:0/1157108324 >> 172.21.15.19:0/1157108324 conn(0x7fd3940d0420 msgr2=0x7fd3940d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:05.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.834+0000 7fd39927c640 1 -- 172.21.15.19:0/1157108324 shutdown_connections 2024-04-25T10:04:05.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.834+0000 7fd39927c640 1 -- 172.21.15.19:0/1157108324 wait complete. 2024-04-25T10:04:05.854 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:04:05.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.958+0000 7fee41f6a640 1 Processor -- start 2024-04-25T10:04:05.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.958+0000 7fee41f6a640 1 -- start start 2024-04-25T10:04:05.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.958+0000 7fee41f6a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee3c15dc20 0x7fee3c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.958+0000 7fee41f6a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fee3c0d32f0 0x7fee3c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.958+0000 7fee41f6a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3c15e000 con 0x7fee3c161710 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.958+0000 7fee41f6a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3c0d1de0 con 0x7fee3c0d32f0 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.958+0000 7fee41f6a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c161c60 con 0x7fee3c15dc20 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.959+0000 7fee3bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fee3c0d32f0 0x7fee3c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.959+0000 7fee3bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fee3c0d32f0 0x7fee3c0cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49458/0 (socket says 172.21.15.19:49458) 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.959+0000 7fee3bfff640 1 -- 172.21.15.19:0/1746835796 learned_addr learned my addr 172.21.15.19:0/1746835796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.959+0000 7fee3b7fe640 1 --2- 172.21.15.19:0/1746835796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee3c15dc20 0x7fee3c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.959+0000 7fee3bfff640 1 -- 172.21.15.19:0/1746835796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee3c15dc20 msgr2=0x7fee3c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.959+0000 7fee3bfff640 1 --2- 172.21.15.19:0/1746835796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee3c15dc20 0x7fee3c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.959+0000 7fee3bfff640 1 -- 172.21.15.19:0/1746835796 >> v1:172.21.15.112:6789/0 conn(0x7fee3c161710 legacy=0x7fee3c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.959+0000 7fee3bfff640 1 -- 172.21.15.19:0/1746835796 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c161f40 con 0x7fee3c0d32f0 2024-04-25T10:04:05.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.959+0000 7fee3b7fe640 1 --2- 172.21.15.19:0/1746835796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee3c15dc20 0x7fee3c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:05.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee3bfff640 1 --2- 172.21.15.19:0/1746835796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee3c0d32f0 0x7fee3c0cebb0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fee34009e50 tx=0x7fee3402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e843b17bc9b8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:05.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee3a7fc640 1 -- 172.21.15.19:0/1746835796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34036070 con 0x7fee3c0d32f0 2024-04-25T10:04:05.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee3a7fc640 1 -- 172.21.15.19:0/1746835796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee34031600 con 0x7fee3c0d32f0 2024-04-25T10:04:05.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee3a7fc640 1 -- 172.21.15.19:0/1746835796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340319d0 con 0x7fee3c0d32f0 2024-04-25T10:04:05.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee41f6a640 1 -- 172.21.15.19:0/1746835796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee3c0d32f0 msgr2=0x7fee3c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee41f6a640 1 --2- 172.21.15.19:0/1746835796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee3c0d32f0 0x7fee3c0cebb0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fee34009e50 tx=0x7fee3402a710 comp rx=0 tx=0).stop 2024-04-25T10:04:05.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee41f6a640 1 -- 172.21.15.19:0/1746835796 shutdown_connections 2024-04-25T10:04:05.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee41f6a640 1 --2- 172.21.15.19:0/1746835796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee3c0d32f0 0x7fee3c0cebb0 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee41f6a640 1 --2- 172.21.15.19:0/1746835796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee3c15dc20 0x7fee3c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee41f6a640 1 -- 172.21.15.19:0/1746835796 >> 172.21.15.19:0/1746835796 conn(0x7fee3c0d0420 msgr2=0x7fee3c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:05.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee41f6a640 1 -- 172.21.15.19:0/1746835796 shutdown_connections 2024-04-25T10:04:05.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.960+0000 7fee41f6a640 1 -- 172.21.15.19:0/1746835796 wait complete. 2024-04-25T10:04:05.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.962+0000 7fee41f6a640 1 Processor -- start 2024-04-25T10:04:05.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.962+0000 7fee41f6a640 1 -- start start 2024-04-25T10:04:05.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee41f6a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee3c0d32f0 0x7fee3c14c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee41f6a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee3c15dc20 0x7fee3c0acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee41f6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee3c161710 0x7fee3c0ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee41f6a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0b5e50 con 0x7fee3c161710 2024-04-25T10:04:05.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee41f6a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c162500 con 0x7fee3c15dc20 2024-04-25T10:04:05.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee41f6a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0b5fd0 con 0x7fee3c0d32f0 2024-04-25T10:04:05.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee3bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee3c161710 0x7fee3c0ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee3bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee3c161710 0x7fee3c0ad460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49470/0 (socket says 172.21.15.19:49470) 2024-04-25T10:04:05.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee3bfff640 1 -- 172.21.15.19:0/4181551579 learned_addr learned my addr 172.21.15.19:0/4181551579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:05.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee3b7fe640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee3c0d32f0 0x7fee3c14c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee3affd640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee3c15dc20 0x7fee3c0acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee3bfff640 1 -- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee3c0d32f0 msgr2=0x7fee3c14c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee3bfff640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee3c0d32f0 0x7fee3c14c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee3bfff640 1 -- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee3c15dc20 msgr2=0x7fee3c0acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee3bfff640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee3c15dc20 0x7fee3c0acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee3bfff640 1 -- 172.21.15.19:0/4181551579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c0cd350 con 0x7fee3c161710 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee3bfff640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee3c161710 0x7fee3c0ad460 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fee34009e50 tx=0x7fee34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee23fff640 1 -- 172.21.15.19:0/4181551579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34041020 con 0x7fee3c161710 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee23fff640 1 -- 172.21.15.19:0/4181551579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee340077a0 con 0x7fee3c161710 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee41f6a640 1 -- 172.21.15.19:0/4181551579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c14cc30 con 0x7fee3c161710 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.963+0000 7fee23fff640 1 -- 172.21.15.19:0/4181551579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34036040 con 0x7fee3c161710 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.964+0000 7fee41f6a640 1 -- 172.21.15.19:0/4181551579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c0cc5d0 con 0x7fee3c161710 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.965+0000 7fee41f6a640 1 -- 172.21.15.19:0/4181551579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3c14ca50 con 0x7fee3c161710 2024-04-25T10:04:05.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.965+0000 7fee23fff640 1 -- 172.21.15.19:0/4181551579 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee34044020 con 0x7fee3c161710 2024-04-25T10:04:05.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.965+0000 7fee23fff640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee2806b500 0x7fee2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.966+0000 7fee23fff640 1 -- 172.21.15.19:0/4181551579 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fee340aaba0 con 0x7fee3c161710 2024-04-25T10:04:05.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.966+0000 7fee3b7fe640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee2806b500 0x7fee2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.966+0000 7fee3b7fe640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee2806b500 0x7fee2806d9c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0aa590 tx=0x7fee2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:05.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:05.970+0000 7fee23fff640 1 -- 172.21.15.19:0/4181551579 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee3c14ca50 con 0x7fee3c161710 2024-04-25T10:04:06.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.149+0000 7fee41f6a640 1 -- 172.21.15.19:0/4181551579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fee3c073a00 con 0x7fee3c161710 2024-04-25T10:04:06.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.152+0000 7fee23fff640 1 -- 172.21.15.19:0/4181551579 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fee3404b030 con 0x7fee3c161710 2024-04-25T10:04:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 -- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee2806b500 msgr2=0x7fee2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee2806b500 0x7fee2806d9c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0aa590 tx=0x7fee2c008040 comp rx=0 tx=0).stop 2024-04-25T10:04:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 -- 172.21.15.19:0/4181551579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee3c161710 msgr2=0x7fee3c0ad460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee3c161710 0x7fee3c0ad460 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fee34009e50 tx=0x7fee34004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 -- 172.21.15.19:0/4181551579 shutdown_connections 2024-04-25T10:04:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee2806b500 0x7fee2806d9c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee3c161710 0x7fee3c0ad460 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee3c15dc20 0x7fee3c0acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 --2- 172.21.15.19:0/4181551579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee3c0d32f0 0x7fee3c14c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 -- 172.21.15.19:0/4181551579 >> 172.21.15.19:0/4181551579 conn(0x7fee3c0d0420 msgr2=0x7fee3c0b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 -- 172.21.15.19:0/4181551579 shutdown_connections 2024-04-25T10:04:06.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.155+0000 7fee41f6a640 1 -- 172.21.15.19:0/4181551579 wait complete. 2024-04-25T10:04:06.171 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:04:06.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.266+0000 7f1a53948640 1 Processor -- start 2024-04-25T10:04:06.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.267+0000 7f1a53948640 1 -- start start 2024-04-25T10:04:06.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.267+0000 7f1a53948640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c15db70 0x7f1a4c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.267+0000 7f1a53948640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4c162690 con 0x7f1a4c0b6cb0 2024-04-25T10:04:06.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.267+0000 7f1a53948640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4c0ceee0 con 0x7f1a4c0b5c30 2024-04-25T10:04:06.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.267+0000 7f1a53948640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0d1d10 con 0x7f1a4c15db70 2024-04-25T10:04:06.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a50ee3640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1a4c0b6cb0 0x7f1a4c0b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57048/0 (socket says 172.21.15.19:57048) 2024-04-25T10:04:06.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a50ee3640 1 -- 172.21.15.19:0/1318190258 learned_addr learned my addr 172.21.15.19:0/1318190258 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111802822 0 0) 0x7f1a4c162690 con 0x7f1a4c0b6cb0 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a34003520 con 0x7f1a4c0b6cb0 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a516e4640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c15db70 0x7f1a4c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3256622819 0 0) 0x7f1a34003520 con 0x7f1a4c0b6cb0 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c15db70 msgr2=0x7f1a4c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c15db70 0x7f1a4c162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 >> v1:172.21.15.112:6789/0 conn(0x7f1a4c0b5c30 legacy=0x7f1a4c0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c0d1ff0 con 0x7f1a4c0b6cb0 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f1a40001670 con 0x7f1a4c0b6cb0 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 >> v1:172.21.15.19:6789/0 conn(0x7f1a4c0b6cb0 legacy=0x7f1a4c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a340039e0 0x7f1a34003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34004800 0x7f1a34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c15db70 0x7f1a34005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0d1d10 con 0x7f1a34004800 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3400baf0 con 0x7f1a340039e0 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3400bcf0 con 0x7f1a4c15db70 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a50ee3640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34004800 0x7f1a34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.268+0000 7f1a516e4640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a340039e0 0x7f1a34003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.269+0000 7f1a51ee5640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c15db70 0x7f1a34005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.269+0000 7f1a50ee3640 1 -- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c15db70 msgr2=0x7f1a34005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.269+0000 7f1a50ee3640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c15db70 0x7f1a34005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.269+0000 7f1a50ee3640 1 -- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a340039e0 msgr2=0x7f1a34003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.269+0000 7f1a50ee3640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a340039e0 0x7f1a34003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.269+0000 7f1a50ee3640 1 -- 172.21.15.19:0/1318190258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34003520 con 0x7f1a34004800 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.269+0000 7f1a51ee5640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c15db70 0x7f1a34005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.269+0000 7f1a50ee3640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34004800 0x7f1a34004c80 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000c210 tx=0x7f1a34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c3c191f6a4022e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.270+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40007cf0 con 0x7f1a34004800 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.270+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a40005af0 con 0x7f1a34004800 2024-04-25T10:04:06.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.270+0000 7f1a4bfff640 1 -- 172.21.15.19:0/1318190258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40007070 con 0x7f1a34004800 2024-04-25T10:04:06.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.270+0000 7f1a53948640 1 -- 172.21.15.19:0/1318190258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34004800 msgr2=0x7f1a34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.270+0000 7f1a53948640 1 --2- 172.21.15.19:0/1318190258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a34004800 0x7f1a34004c80 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000c210 tx=0x7f1a34005830 comp rx=0 tx=0).stop 2024-04-25T10:04:06.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.270+0000 7f1a51ee5640 1 -- 172.21.15.19:0/1318190258 reap_dead start 2024-04-25T10:04:06.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.270+0000 7f1a53948640 1 -- 172.21.15.19:0/1318190258 shutdown_connections 2024-04-25T10:04:06.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.270+0000 7f1a53948640 1 -- 172.21.15.19:0/1318190258 >> 172.21.15.19:0/1318190258 conn(0x7f1a4c0d0520 msgr2=0x7f1a4c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.270+0000 7f1a53948640 1 -- 172.21.15.19:0/1318190258 shutdown_connections 2024-04-25T10:04:06.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.270+0000 7f1a53948640 1 -- 172.21.15.19:0/1318190258 wait complete. 2024-04-25T10:04:06.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.271+0000 7f1a53948640 1 Processor -- start 2024-04-25T10:04:06.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.271+0000 7f1a53948640 1 -- start start 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a53948640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a34004800 0x7f1a4c0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a53948640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c0cab70 0x7f1a4c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a53948640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0ccd80 0x7f1a4c0cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a53948640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0ce530 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a53948640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c15df50 con 0x7f1a34004800 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a53948640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0cf1c0 con 0x7f1a4c0cab70 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a51ee5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0ccd80 0x7f1a4c0cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a51ee5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0ccd80 0x7f1a4c0cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49488/0 (socket says 172.21.15.19:49488) 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a51ee5640 1 -- 172.21.15.19:0/68653918 learned_addr learned my addr 172.21.15.19:0/68653918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a50ee3640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c0cab70 0x7f1a4c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a516e4640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a34004800 0x7f1a4c0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a51ee5640 1 -- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c0cab70 msgr2=0x7f1a4c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a51ee5640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c0cab70 0x7f1a4c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a51ee5640 1 -- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a34004800 msgr2=0x7f1a4c0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a51ee5640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a34004800 0x7f1a4c0c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.272+0000 7f1a51ee5640 1 -- 172.21.15.19:0/68653918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c0cdaf0 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.273+0000 7f1a51ee5640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0ccd80 0x7f1a4c0cd1e0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c000f80 tx=0x7f1a3c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.273+0000 7f1a49ffb640 1 -- 172.21.15.19:0/68653918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c006b50 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.273+0000 7f1a49ffb640 1 -- 172.21.15.19:0/68653918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c006d10 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.273+0000 7f1a53948640 1 -- 172.21.15.19:0/68653918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c0cddd0 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.273+0000 7f1a49ffb640 1 -- 172.21.15.19:0/68653918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c003970 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.273+0000 7f1a53948640 1 -- 172.21.15.19:0/68653918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c154ad0 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.274+0000 7f1a53948640 1 -- 172.21.15.19:0/68653918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4c073990 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.275+0000 7f1a49ffb640 1 -- 172.21.15.19:0/68653918 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a3c01a020 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.275+0000 7f1a49ffb640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a3806b4d0 0x7f1a3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.275+0000 7f1a49ffb640 1 -- 172.21.15.19:0/68653918 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1a3c089770 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.275+0000 7f1a516e4640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a3806b4d0 0x7f1a3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.276+0000 7f1a516e4640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a3806b4d0 0x7f1a3806d990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c156820 tx=0x7f1a44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.279+0000 7f1a49ffb640 1 -- 172.21.15.19:0/68653918 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a3c058750 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.527+0000 7f1a53948640 1 -- 172.21.15.19:0/68653918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7f1a4c0d1820 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.831+0000 7f1a49ffb640 1 -- 172.21.15.19:0/68653918 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v542) v1 ==== 140+0+0 (secure 0 0 0) 0x7f1a3c058050 con 0x7f1a4c0ccd80 2024-04-25T10:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.833+0000 7f1a53948640 1 -- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a3806b4d0 msgr2=0x7f1a3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.833+0000 7f1a53948640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a3806b4d0 0x7f1a3806d990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c156820 tx=0x7f1a44008040 comp rx=0 tx=0).stop 2024-04-25T10:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.833+0000 7f1a53948640 1 -- 172.21.15.19:0/68653918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0ccd80 msgr2=0x7f1a4c0cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.833+0000 7f1a53948640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0ccd80 0x7f1a4c0cd1e0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c000f80 tx=0x7f1a3c00eb90 comp rx=0 tx=0).stop 2024-04-25T10:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.833+0000 7f1a53948640 1 -- 172.21.15.19:0/68653918 shutdown_connections 2024-04-25T10:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.833+0000 7f1a53948640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a3806b4d0 0x7f1a3806d990 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.833+0000 7f1a53948640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0ccd80 0x7f1a4c0cd1e0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.833+0000 7f1a53948640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a4c0cab70 0x7f1a4c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.833+0000 7f1a53948640 1 --2- 172.21.15.19:0/68653918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a34004800 0x7f1a4c0c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.833+0000 7f1a53948640 1 -- 172.21.15.19:0/68653918 >> 172.21.15.19:0/68653918 conn(0x7f1a4c0d0520 msgr2=0x7f1a4c0d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.834+0000 7f1a53948640 1 -- 172.21.15.19:0/68653918 shutdown_connections 2024-04-25T10:04:06.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.834+0000 7f1a53948640 1 -- 172.21.15.19:0/68653918 wait complete. 2024-04-25T10:04:06.854 INFO:tasks.cephfs_test_runner:test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:04:06.854 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:04:06.855 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:06.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.951+0000 7f6037f09640 1 Processor -- start 2024-04-25T10:04:06.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.952+0000 7f6037f09640 1 -- start start 2024-04-25T10:04:06.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.952+0000 7f6037f09640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f60300d0b10 0x7f60300ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.952+0000 7f6037f09640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60300cec00 con 0x7f603015db60 2024-04-25T10:04:06.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.952+0000 7f6037f09640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60300ceee0 con 0x7f60300b5f00 2024-04-25T10:04:06.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.952+0000 7f6037f09640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60300ce930 con 0x7f60300d0b10 2024-04-25T10:04:06.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.952+0000 7f6035ca5640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f603015db60 0x7f60300d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57056/0 (socket says 172.21.15.19:57056) 2024-04-25T10:04:06.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.952+0000 7f6035ca5640 1 -- 172.21.15.19:0/2084840328 learned_addr learned my addr 172.21.15.19:0/2084840328 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:06.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f60354a4640 1 --2- 172.21.15.19:0/2084840328 >> v2:172.21.15.112:3300/0 conn(0x7f60300d0b10 0x7f60300ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443311850 0 0) 0x7f60300cec00 con 0x7f603015db60 2024-04-25T10:04:06.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6018003520 con 0x7f603015db60 2024-04-25T10:04:06.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3281000848 0 0) 0x7f6018003520 con 0x7f603015db60 2024-04-25T10:04:06.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 >> v2:172.21.15.112:3300/0 conn(0x7f60300d0b10 msgr2=0x7f60300ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 --2- 172.21.15.19:0/2084840328 >> v2:172.21.15.112:3300/0 conn(0x7f60300d0b10 0x7f60300ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 >> v1:172.21.15.112:6789/0 conn(0x7f60300b5f00 legacy=0x7f60300d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300cf1c0 con 0x7f603015db60 2024-04-25T10:04:06.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f6028001670 con 0x7f603015db60 2024-04-25T10:04:06.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 >> v1:172.21.15.19:6789/0 conn(0x7f603015db60 legacy=0x7f60300d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60180039e0 0x7f6018003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6018004800 0x7f6018004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6018005210 0x7f601800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300ce930 con 0x7f60180039e0 2024-04-25T10:04:06.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601800c140 con 0x7f6018005210 2024-04-25T10:04:06.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601800c340 con 0x7f6018004800 2024-04-25T10:04:06.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6035ca5640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60180039e0 0x7f6018003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f60364a6640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6018005210 0x7f601800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f60354a4640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6018004800 0x7f6018004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6035ca5640 1 -- 172.21.15.19:0/2084840328 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6018004800 msgr2=0x7f6018004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6035ca5640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6018004800 0x7f6018004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6035ca5640 1 -- 172.21.15.19:0/2084840328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6018005210 msgr2=0x7f601800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.953+0000 7f6035ca5640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6018005210 0x7f601800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.954+0000 7f60364a6640 1 -- 172.21.15.19:0/2084840328 reap_dead start 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.954+0000 7f6035ca5640 1 -- 172.21.15.19:0/2084840328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6018003520 con 0x7f60180039e0 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.954+0000 7f60354a4640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6018004800 0x7f6018004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.954+0000 7f6035ca5640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60180039e0 0x7f6018003e00 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f602800c240 tx=0x7f6018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4811a483dc87f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.954+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028007cf0 con 0x7f60180039e0 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.954+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60280058d0 con 0x7f60180039e0 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.954+0000 7f6034ca3640 1 -- 172.21.15.19:0/2084840328 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028005ca0 con 0x7f60180039e0 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.955+0000 7f6037f09640 1 -- 172.21.15.19:0/2084840328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60180039e0 msgr2=0x7f6018003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.955+0000 7f6037f09640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60180039e0 0x7f6018003e00 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f602800c240 tx=0x7f6018005830 comp rx=0 tx=0).stop 2024-04-25T10:04:06.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.955+0000 7f6037f09640 1 -- 172.21.15.19:0/2084840328 shutdown_connections 2024-04-25T10:04:06.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.955+0000 7f6037f09640 1 --2- 172.21.15.19:0/2084840328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60180039e0 0x7f6018003e00 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.955+0000 7f6037f09640 1 -- 172.21.15.19:0/2084840328 >> 172.21.15.19:0/2084840328 conn(0x7f6030061600 msgr2=0x7f6030061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.955+0000 7f6037f09640 1 -- 172.21.15.19:0/2084840328 shutdown_connections 2024-04-25T10:04:06.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.955+0000 7f6037f09640 1 -- 172.21.15.19:0/2084840328 wait complete. 2024-04-25T10:04:06.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.956+0000 7f6037f09640 1 Processor -- start 2024-04-25T10:04:06.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.956+0000 7f6037f09640 1 -- start start 2024-04-25T10:04:06.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f6037f09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60180039e0 0x7f60301607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f6037f09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60301f3a10 0x7f603015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f6037f09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60301f3e90 0x7f603015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f6037f09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603015df40 con 0x7f60301f3a10 2024-04-25T10:04:06.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f6037f09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300b6450 con 0x7f60180039e0 2024-04-25T10:04:06.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f6037f09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603015e0c0 con 0x7f60301f3e90 2024-04-25T10:04:06.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f60354a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60301f3a10 0x7f603015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f60354a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60301f3a10 0x7f603015ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49502/0 (socket says 172.21.15.19:49502) 2024-04-25T10:04:06.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f60354a4640 1 -- 172.21.15.19:0/3214402459 learned_addr learned my addr 172.21.15.19:0/3214402459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:06.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f6035ca5640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60180039e0 0x7f60301607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f60364a6640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60301f3e90 0x7f603015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f60354a4640 1 -- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60301f3e90 msgr2=0x7f603015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f60354a4640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60301f3e90 0x7f603015f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f60354a4640 1 -- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60180039e0 msgr2=0x7f60301607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f60354a4640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60180039e0 0x7f60301607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.957+0000 7f60354a4640 1 -- 172.21.15.19:0/3214402459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6030159940 con 0x7f60301f3a10 2024-04-25T10:04:06.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.958+0000 7f60354a4640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60301f3a10 0x7f603015ee60 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f60240114c0 tx=0x7f6024011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.958+0000 7f602e7fc640 1 -- 172.21.15.19:0/3214402459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024015930 con 0x7f60301f3a10 2024-04-25T10:04:06.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.958+0000 7f602e7fc640 1 -- 172.21.15.19:0/3214402459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6024015ad0 con 0x7f60301f3a10 2024-04-25T10:04:06.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.958+0000 7f602e7fc640 1 -- 172.21.15.19:0/3214402459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602401d400 con 0x7f60301f3a10 2024-04-25T10:04:06.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.958+0000 7f6037f09640 1 -- 172.21.15.19:0/3214402459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6030159c20 con 0x7f60301f3a10 2024-04-25T10:04:06.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.958+0000 7f6037f09640 1 -- 172.21.15.19:0/3214402459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603015a1e0 con 0x7f60301f3a10 2024-04-25T10:04:06.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.959+0000 7f6037f09640 1 -- 172.21.15.19:0/3214402459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6030073270 con 0x7f60301f3a10 2024-04-25T10:04:06.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.959+0000 7f602e7fc640 1 -- 172.21.15.19:0/3214402459 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f602401d5a0 con 0x7f60301f3a10 2024-04-25T10:04:06.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.960+0000 7f602e7fc640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601c06b4d0 0x7f601c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.960+0000 7f602e7fc640 1 -- 172.21.15.19:0/3214402459 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f602408a810 con 0x7f60301f3a10 2024-04-25T10:04:06.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.960+0000 7f6035ca5640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601c06b4d0 0x7f601c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.960+0000 7f6035ca5640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601c06b4d0 0x7f601c06d990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6028033000 tx=0x7f6028009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:06.964+0000 7f602e7fc640 1 -- 172.21.15.19:0/3214402459 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6024019030 con 0x7f60301f3a10 2024-04-25T10:04:07.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.146+0000 7f6037f09640 1 -- 172.21.15.19:0/3214402459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6030073990 con 0x7f60301f3a10 2024-04-25T10:04:07.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.147+0000 7f602e7fc640 1 -- 172.21.15.19:0/3214402459 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+13289 (secure 0 0 0) 0x7f60240597f0 con 0x7f60301f3a10 2024-04-25T10:04:07.152 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:07.152 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":157,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:04:05.825485+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/717842662":"2024-04-26T10:04:02.811602+0000","172.21.15.19:6833/717842662":"2024-04-26T10:04:02.811602+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.148+0000 7f6037f09640 1 -- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601c06b4d0 msgr2=0x7f601c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.148+0000 7f6037f09640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601c06b4d0 0x7f601c06d990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6028033000 tx=0x7f6028009030 comp rx=0 tx=0).stop 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.148+0000 7f6037f09640 1 -- 172.21.15.19:0/3214402459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60301f3a10 msgr2=0x7f603015ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.148+0000 7f6037f09640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60301f3a10 0x7f603015ee60 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f60240114c0 tx=0x7f6024011980 comp rx=0 tx=0).stop 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.149+0000 7f6037f09640 1 -- 172.21.15.19:0/3214402459 shutdown_connections 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.149+0000 7f6037f09640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f601c06b4d0 0x7f601c06d990 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.149+0000 7f6037f09640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f60301f3e90 0x7f603015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.149+0000 7f6037f09640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60301f3a10 0x7f603015ee60 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.149+0000 7f6037f09640 1 --2- 172.21.15.19:0/3214402459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60180039e0 0x7f60301607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.149+0000 7f6037f09640 1 -- 172.21.15.19:0/3214402459 >> 172.21.15.19:0/3214402459 conn(0x7f6030061600 msgr2=0x7f60300b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:07.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.149+0000 7f6037f09640 1 -- 172.21.15.19:0/3214402459 shutdown_connections 2024-04-25T10:04:07.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.149+0000 7f6037f09640 1 -- 172.21.15.19:0/3214402459 wait complete. 2024-04-25T10:04:07.165 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:07.165 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:07.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.264+0000 7f93e03d5640 1 Processor -- start 2024-04-25T10:04:07.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.264+0000 7f93e03d5640 1 -- start start 2024-04-25T10:04:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93e03d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93d80cd4c0 0x7f93d80cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93e03d5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f93d815b1c0 0x7f93d815dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93e03d5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f93d815e1a0 0x7f93d8150b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93e03d5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d815b620 con 0x7f93d815b1c0 2024-04-25T10:04:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93e03d5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d815e6e0 con 0x7f93d815e1a0 2024-04-25T10:04:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93e03d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d815e9c0 con 0x7f93d80cd4c0 2024-04-25T10:04:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93dd970640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f93d815b1c0 0x7f93d815dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93dd970640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f93d815b1c0 0x7f93d815dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49504/0 (socket says 172.21.15.19:49504) 2024-04-25T10:04:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93dd970640 1 -- 172.21.15.19:0/1264700059 learned_addr learned my addr 172.21.15.19:0/1264700059 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93dd970640 1 -- 172.21.15.19:0/1264700059 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93d80cd4c0 msgr2=0x7f93d80cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93dd970640 1 --2- 172.21.15.19:0/1264700059 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93d80cd4c0 0x7f93d80cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93de972640 1 --2- 172.21.15.19:0/1264700059 >> v2:172.21.15.112:3300/0 conn(0x7f93d815e1a0 0x7f93d8150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93dd970640 1 -- 172.21.15.19:0/1264700059 >> v2:172.21.15.112:3300/0 conn(0x7f93d815e1a0 msgr2=0x7f93d8150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93dd970640 1 --2- 172.21.15.19:0/1264700059 >> v2:172.21.15.112:3300/0 conn(0x7f93d815e1a0 0x7f93d8150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93dd970640 1 -- 172.21.15.19:0/1264700059 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d815a9c0 con 0x7f93d815b1c0 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.265+0000 7f93de972640 1 --2- 172.21.15.19:0/1264700059 >> v2:172.21.15.112:3300/0 conn(0x7f93d815e1a0 0x7f93d8150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.266+0000 7f93dd970640 1 --2- 172.21.15.19:0/1264700059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93d815b1c0 0x7f93d815dc60 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f93d80af3a0 tx=0x7f93c802b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e589d1e9e044e794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.266+0000 7f93dd16f640 1 -- 172.21.15.19:0/1264700059 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c800b040 con 0x7f93d815b1c0 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.266+0000 7f93dd16f640 1 -- 172.21.15.19:0/1264700059 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c8030710 con 0x7f93d815b1c0 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.266+0000 7f93dd16f640 1 -- 172.21.15.19:0/1264700059 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8030ae0 con 0x7f93d815b1c0 2024-04-25T10:04:07.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.266+0000 7f93e03d5640 1 -- 172.21.15.19:0/1264700059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93d815b1c0 msgr2=0x7f93d815dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.266+0000 7f93e03d5640 1 --2- 172.21.15.19:0/1264700059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93d815b1c0 0x7f93d815dc60 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f93d80af3a0 tx=0x7f93c802b5a0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.267+0000 7f93e03d5640 1 -- 172.21.15.19:0/1264700059 shutdown_connections 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.267+0000 7f93e03d5640 1 --2- 172.21.15.19:0/1264700059 >> v2:172.21.15.112:3300/0 conn(0x7f93d815e1a0 0x7f93d8150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.267+0000 7f93e03d5640 1 --2- 172.21.15.19:0/1264700059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93d815b1c0 0x7f93d815dc60 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.267+0000 7f93e03d5640 1 --2- 172.21.15.19:0/1264700059 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93d80cd4c0 0x7f93d80cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.267+0000 7f93e03d5640 1 -- 172.21.15.19:0/1264700059 >> 172.21.15.19:0/1264700059 conn(0x7f93d806abe0 msgr2=0x7f93d806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.267+0000 7f93e03d5640 1 -- 172.21.15.19:0/1264700059 shutdown_connections 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.267+0000 7f93e03d5640 1 -- 172.21.15.19:0/1264700059 wait complete. 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.268+0000 7f93e03d5640 1 Processor -- start 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.268+0000 7f93e03d5640 1 -- start start 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93e03d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93d80cd4c0 0x7f93d813b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93e03d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93d815b1c0 0x7f93d80b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93e03d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93d815e1a0 0x7f93d8153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93e03d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d806eca0 con 0x7f93d815e1a0 2024-04-25T10:04:07.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93e03d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d815a670 con 0x7f93d815b1c0 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93e03d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d806ee20 con 0x7f93d80cd4c0 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93de972640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93d815e1a0 0x7f93d8153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93de972640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93d815e1a0 0x7f93d8153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49518/0 (socket says 172.21.15.19:49518) 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93de972640 1 -- 172.21.15.19:0/1584635374 learned_addr learned my addr 172.21.15.19:0/1584635374 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93dd970640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93d815b1c0 0x7f93d80b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93de171640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93d80cd4c0 0x7f93d813b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93de972640 1 -- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93d80cd4c0 msgr2=0x7f93d813b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93de972640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93d80cd4c0 0x7f93d813b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93de972640 1 -- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93d815b1c0 msgr2=0x7f93d80b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93de972640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93d815b1c0 0x7f93d80b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93de972640 1 -- 172.21.15.19:0/1584635374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d80b0bb0 con 0x7f93d815e1a0 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93de972640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93d815e1a0 0x7f93d8153f70 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f93d000cd80 tx=0x7f93d000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.269+0000 7f93ceffd640 1 -- 172.21.15.19:0/1584635374 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d000f680 con 0x7f93d815e1a0 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.270+0000 7f93ceffd640 1 -- 172.21.15.19:0/1584635374 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d000f840 con 0x7f93d815e1a0 2024-04-25T10:04:07.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.270+0000 7f93ceffd640 1 -- 172.21.15.19:0/1584635374 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d000fc30 con 0x7f93d815e1a0 2024-04-25T10:04:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.270+0000 7f93e03d5640 1 -- 172.21.15.19:0/1584635374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d80b0e90 con 0x7f93d815e1a0 2024-04-25T10:04:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.270+0000 7f93e03d5640 1 -- 172.21.15.19:0/1584635374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d80b1450 con 0x7f93d815e1a0 2024-04-25T10:04:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.270+0000 7f93e03d5640 1 -- 172.21.15.19:0/1584635374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d8073a00 con 0x7f93d815e1a0 2024-04-25T10:04:07.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.271+0000 7f93ceffd640 1 -- 172.21.15.19:0/1584635374 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93d00223f0 con 0x7f93d815e1a0 2024-04-25T10:04:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.271+0000 7f93ceffd640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f93bc06b4d0 0x7f93bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.271+0000 7f93ceffd640 1 -- 172.21.15.19:0/1584635374 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f93d008aeb0 con 0x7f93d815e1a0 2024-04-25T10:04:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.272+0000 7f93de171640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f93bc06b4d0 0x7f93bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.272+0000 7f93de171640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f93bc06b4d0 0x7f93bc06d990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009890 tx=0x7f93d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.276+0000 7f93ceffd640 1 -- 172.21.15.19:0/1584635374 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93d0014030 con 0x7f93d815e1a0 2024-04-25T10:04:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.456+0000 7f93e03d5640 1 -- 172.21.15.19:0/1584635374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93d80ccb10 con 0x7f93d815e1a0 2024-04-25T10:04:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.456+0000 7f93ceffd640 1 -- 172.21.15.19:0/1584635374 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93d0059e00 con 0x7f93d815e1a0 2024-04-25T10:04:07.462 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:04:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 -- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f93bc06b4d0 msgr2=0x7f93bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f93bc06b4d0 0x7f93bc06d990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009890 tx=0x7f93d4008040 comp rx=0 tx=0).stop 2024-04-25T10:04:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 -- 172.21.15.19:0/1584635374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93d815e1a0 msgr2=0x7f93d8153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93d815e1a0 0x7f93d8153f70 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f93d000cd80 tx=0x7f93d000a650 comp rx=0 tx=0).stop 2024-04-25T10:04:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 -- 172.21.15.19:0/1584635374 shutdown_connections 2024-04-25T10:04:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f93bc06b4d0 0x7f93bc06d990 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93d815e1a0 0x7f93d8153f70 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93d815b1c0 0x7f93d80b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 --2- 172.21.15.19:0/1584635374 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93d80cd4c0 0x7f93d813b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 -- 172.21.15.19:0/1584635374 >> 172.21.15.19:0/1584635374 conn(0x7f93d806abe0 msgr2=0x7f93d814fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 -- 172.21.15.19:0/1584635374 shutdown_connections 2024-04-25T10:04:07.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.458+0000 7f93e03d5640 1 -- 172.21.15.19:0/1584635374 wait complete. 2024-04-25T10:04:07.474 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:04:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.570+0000 7f353a0f5640 1 Processor -- start 2024-04-25T10:04:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.570+0000 7f353a0f5640 1 -- start start 2024-04-25T10:04:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.571+0000 7f353a0f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f353406abe0 0x7f353406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.571+0000 7f353a0f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35340ce330 0x7f353406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.571+0000 7f353a0f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35341626e0 con 0x7f35340700a0 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.571+0000 7f353a0f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f353406b500 con 0x7f35340ce330 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.571+0000 7f353a0f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35340cd270 con 0x7f353406abe0 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.571+0000 7f3532ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35340ce330 0x7f353406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.571+0000 7f3532ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35340ce330 0x7f353406fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49526/0 (socket says 172.21.15.19:49526) 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.571+0000 7f3532ffd640 1 -- 172.21.15.19:0/503991541 learned_addr learned my addr 172.21.15.19:0/503991541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.572+0000 7f35337fe640 1 --2- 172.21.15.19:0/503991541 >> v2:172.21.15.112:3300/0 conn(0x7f353406abe0 0x7f353406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.572+0000 7f3532ffd640 1 -- 172.21.15.19:0/503991541 >> v2:172.21.15.112:3300/0 conn(0x7f353406abe0 msgr2=0x7f353406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.572+0000 7f3532ffd640 1 --2- 172.21.15.19:0/503991541 >> v2:172.21.15.112:3300/0 conn(0x7f353406abe0 0x7f353406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.572+0000 7f3532ffd640 1 -- 172.21.15.19:0/503991541 >> v1:172.21.15.112:6789/0 conn(0x7f35340700a0 legacy=0x7f353406e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.572+0000 7f3532ffd640 1 -- 172.21.15.19:0/503991541 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35340cd550 con 0x7f35340ce330 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.572+0000 7f3532ffd640 1 --2- 172.21.15.19:0/503991541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340ce330 0x7f353406fb60 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f353406e200 tx=0x7f352802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f986ed2e2c7cd1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.572+0000 7f35327fc640 1 -- 172.21.15.19:0/503991541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528037070 con 0x7f35340ce330 2024-04-25T10:04:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.573+0000 7f35327fc640 1 -- 172.21.15.19:0/503991541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3528032440 con 0x7f35340ce330 2024-04-25T10:04:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.573+0000 7f35327fc640 1 -- 172.21.15.19:0/503991541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528032810 con 0x7f35340ce330 2024-04-25T10:04:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.573+0000 7f353a0f5640 1 -- 172.21.15.19:0/503991541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340ce330 msgr2=0x7f353406fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.573+0000 7f353a0f5640 1 --2- 172.21.15.19:0/503991541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340ce330 0x7f353406fb60 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f353406e200 tx=0x7f352802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.573+0000 7f353a0f5640 1 -- 172.21.15.19:0/503991541 shutdown_connections 2024-04-25T10:04:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.573+0000 7f353a0f5640 1 --2- 172.21.15.19:0/503991541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340ce330 0x7f353406fb60 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.573+0000 7f353a0f5640 1 --2- 172.21.15.19:0/503991541 >> v2:172.21.15.112:3300/0 conn(0x7f353406abe0 0x7f353406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.573+0000 7f353a0f5640 1 -- 172.21.15.19:0/503991541 >> 172.21.15.19:0/503991541 conn(0x7f35340b2220 msgr2=0x7f35340b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.573+0000 7f353a0f5640 1 -- 172.21.15.19:0/503991541 shutdown_connections 2024-04-25T10:04:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.573+0000 7f353a0f5640 1 -- 172.21.15.19:0/503991541 wait complete. 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.574+0000 7f353a0f5640 1 Processor -- start 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.574+0000 7f353a0f5640 1 -- start start 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.575+0000 7f353a0f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353406abe0 0x7f3534159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.575+0000 7f353a0f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340700a0 0x7f35341e7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.575+0000 7f353a0f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35340ce330 0x7f35341e8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.575+0000 7f353a0f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3534161780 con 0x7f35340700a0 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.575+0000 7f353a0f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35340cdb10 con 0x7f353406abe0 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.575+0000 7f353a0f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353406eb00 con 0x7f35340ce330 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.575+0000 7f3532ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340700a0 0x7f35341e7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.575+0000 7f3532ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340700a0 0x7f35341e7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49528/0 (socket says 172.21.15.19:49528) 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.575+0000 7f3532ffd640 1 -- 172.21.15.19:0/353954868 learned_addr learned my addr 172.21.15.19:0/353954868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f35337fe640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353406abe0 0x7f3534159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f3533fff640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35340ce330 0x7f35341e8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f3532ffd640 1 -- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35340ce330 msgr2=0x7f35341e8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f3532ffd640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35340ce330 0x7f35341e8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f3532ffd640 1 -- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353406abe0 msgr2=0x7f3534159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f3532ffd640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353406abe0 0x7f3534159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f3532ffd640 1 -- 172.21.15.19:0/353954868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353415c9e0 con 0x7f35340700a0 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f35337fe640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353406abe0 0x7f3534159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f3533fff640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35340ce330 0x7f35341e8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f3532ffd640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340700a0 0x7f35341e7ed0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f3528000c00 tx=0x7f35280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f351bfff640 1 -- 172.21.15.19:0/353954868 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528041070 con 0x7f35340700a0 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f351bfff640 1 -- 172.21.15.19:0/353954868 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352803c870 con 0x7f35340700a0 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f353a0f5640 1 -- 172.21.15.19:0/353954868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353415ccc0 con 0x7f35340700a0 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f351bfff640 1 -- 172.21.15.19:0/353954868 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528037040 con 0x7f35340700a0 2024-04-25T10:04:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.576+0000 7f353a0f5640 1 -- 172.21.15.19:0/353954868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353415d280 con 0x7f35340700a0 2024-04-25T10:04:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.577+0000 7f353a0f5640 1 -- 172.21.15.19:0/353954868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3534073990 con 0x7f35340700a0 2024-04-25T10:04:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.577+0000 7f351bfff640 1 -- 172.21.15.19:0/353954868 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3528044070 con 0x7f35340700a0 2024-04-25T10:04:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.578+0000 7f351bfff640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 0x7f352006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.578+0000 7f351bfff640 1 -- 172.21.15.19:0/353954868 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f35280aacd0 con 0x7f35340700a0 2024-04-25T10:04:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.578+0000 7f35337fe640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 0x7f352006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.578+0000 7f35337fe640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 0x7f352006d990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3524006fd0 tx=0x7f3524008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.582+0000 7f351bfff640 1 -- 172.21.15.19:0/353954868 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3528079e70 con 0x7f35340700a0 2024-04-25T10:04:07.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:07.833+0000 7f353a0f5640 1 -- 172.21.15.19:0/353954868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7f35340afe50 con 0x7f35340700a0 2024-04-25T10:04:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.837+0000 7f351bfff640 1 -- 172.21.15.19:0/353954868 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v544) v1 ==== 136+0+0 (secure 0 0 0) 0x7f3528079770 con 0x7f35340700a0 2024-04-25T10:04:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 -- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 msgr2=0x7f352006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 0x7f352006d990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3524006fd0 tx=0x7f3524008040 comp rx=0 tx=0).stop 2024-04-25T10:04:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 -- 172.21.15.19:0/353954868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340700a0 msgr2=0x7f35341e7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340700a0 0x7f35341e7ed0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f3528000c00 tx=0x7f35280046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 -- 172.21.15.19:0/353954868 shutdown_connections 2024-04-25T10:04:08.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f352006b4d0 0x7f352006d990 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35340ce330 0x7f35341e8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35340700a0 0x7f35341e7ed0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 --2- 172.21.15.19:0/353954868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f353406abe0 0x7f3534159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 -- 172.21.15.19:0/353954868 >> 172.21.15.19:0/353954868 conn(0x7f35340b2220 msgr2=0x7f35340b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:08.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 -- 172.21.15.19:0/353954868 shutdown_connections 2024-04-25T10:04:08.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.841+0000 7f353a0f5640 1 -- 172.21.15.19:0/353954868 wait complete. 2024-04-25T10:04:08.860 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:04:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.964+0000 7f00830b3640 1 Processor -- start 2024-04-25T10:04:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.964+0000 7f00830b3640 1 -- start start 2024-04-25T10:04:08.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f00830b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f007c144d20 0x7f007c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f00830b3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f007c0714d0 0x7f007c0b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f00830b3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007c0ceba0 con 0x7f007c1454b0 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f00830b3640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007c0b4830 con 0x7f007c0714d0 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f00830b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c0cee80 con 0x7f007c144d20 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f0081650640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f007c0714d0 0x7f007c0b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f0081650640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f007c0714d0 0x7f007c0b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37842/0 (socket says 172.21.15.19:37842) 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f0081650640 1 -- 172.21.15.19:0/97766403 learned_addr learned my addr 172.21.15.19:0/97766403 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f0080e4f640 1 --2- 172.21.15.19:0/97766403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f007c144d20 0x7f007c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f0081650640 1 -- 172.21.15.19:0/97766403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f007c144d20 msgr2=0x7f007c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f0081650640 1 --2- 172.21.15.19:0/97766403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f007c144d20 0x7f007c0ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f0081650640 1 -- 172.21.15.19:0/97766403 >> v1:172.21.15.112:6789/0 conn(0x7f007c1454b0 legacy=0x7f007c070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.965+0000 7f0081650640 1 -- 172.21.15.19:0/97766403 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c0cf160 con 0x7f007c0714d0 2024-04-25T10:04:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.966+0000 7f0081650640 1 --2- 172.21.15.19:0/97766403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c0714d0 0x7f007c0b4ca0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f0074009e50 tx=0x7f007402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745eb1cf185e1272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.966+0000 7f007b7fe640 1 -- 172.21.15.19:0/97766403 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074036070 con 0x7f007c0714d0 2024-04-25T10:04:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.966+0000 7f007b7fe640 1 -- 172.21.15.19:0/97766403 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0074031600 con 0x7f007c0714d0 2024-04-25T10:04:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.966+0000 7f007b7fe640 1 -- 172.21.15.19:0/97766403 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00740319d0 con 0x7f007c0714d0 2024-04-25T10:04:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.967+0000 7f00830b3640 1 -- 172.21.15.19:0/97766403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c0714d0 msgr2=0x7f007c0b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.967+0000 7f00830b3640 1 --2- 172.21.15.19:0/97766403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c0714d0 0x7f007c0b4ca0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f0074009e50 tx=0x7f007402a710 comp rx=0 tx=0).stop 2024-04-25T10:04:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.967+0000 7f00830b3640 1 -- 172.21.15.19:0/97766403 shutdown_connections 2024-04-25T10:04:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.967+0000 7f00830b3640 1 --2- 172.21.15.19:0/97766403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c0714d0 0x7f007c0b4ca0 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.967+0000 7f00830b3640 1 --2- 172.21.15.19:0/97766403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f007c144d20 0x7f007c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.967+0000 7f00830b3640 1 -- 172.21.15.19:0/97766403 >> 172.21.15.19:0/97766403 conn(0x7f007c0b3530 msgr2=0x7f007c0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.967+0000 7f00830b3640 1 -- 172.21.15.19:0/97766403 shutdown_connections 2024-04-25T10:04:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.967+0000 7f00830b3640 1 -- 172.21.15.19:0/97766403 wait complete. 2024-04-25T10:04:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.968+0000 7f00830b3640 1 Processor -- start 2024-04-25T10:04:08.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.968+0000 7f00830b3640 1 -- start start 2024-04-25T10:04:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f00830b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f007c0714d0 0x7f007c13f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f00830b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f007c144d20 0x7f007c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f00830b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c1454b0 0x7f007c0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f00830b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c145100 con 0x7f007c1454b0 2024-04-25T10:04:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f00830b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c0ce890 con 0x7f007c0714d0 2024-04-25T10:04:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f00830b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c145280 con 0x7f007c144d20 2024-04-25T10:04:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f0081650640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c1454b0 0x7f007c0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f0081650640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c1454b0 0x7f007c0cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37858/0 (socket says 172.21.15.19:37858) 2024-04-25T10:04:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f0081650640 1 -- 172.21.15.19:0/1370559643 learned_addr learned my addr 172.21.15.19:0/1370559643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f0080e4f640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f007c0714d0 0x7f007c13f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f007bfff640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f007c144d20 0x7f007c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f0081650640 1 -- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f007c144d20 msgr2=0x7f007c0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f0081650640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f007c144d20 0x7f007c0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f0081650640 1 -- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f007c0714d0 msgr2=0x7f007c13f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.969+0000 7f0081650640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f007c0714d0 0x7f007c13f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.970+0000 7f0081650640 1 -- 172.21.15.19:0/1370559643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c13fa80 con 0x7f007c1454b0 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.970+0000 7f0081650640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c1454b0 0x7f007c0cdac0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f0074034040 tx=0x7f0074004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.970+0000 7f00797fa640 1 -- 172.21.15.19:0/1370559643 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074036050 con 0x7f007c1454b0 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.970+0000 7f00797fa640 1 -- 172.21.15.19:0/1370559643 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0074042070 con 0x7f007c1454b0 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.970+0000 7f00797fa640 1 -- 172.21.15.19:0/1370559643 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074009510 con 0x7f007c1454b0 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.970+0000 7f00830b3640 1 -- 172.21.15.19:0/1370559643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007c0cbb00 con 0x7f007c1454b0 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.970+0000 7f00830b3640 1 -- 172.21.15.19:0/1370559643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c0cc0c0 con 0x7f007c1454b0 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.971+0000 7f00830b3640 1 -- 172.21.15.19:0/1370559643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007c073a00 con 0x7f007c1454b0 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.971+0000 7f00797fa640 1 -- 172.21.15.19:0/1370559643 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0074007bd0 con 0x7f007c1454b0 2024-04-25T10:04:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.972+0000 7f00797fa640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006006b500 0x7f006006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.972+0000 7f00797fa640 1 -- 172.21.15.19:0/1370559643 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f00740aadd0 con 0x7f007c1454b0 2024-04-25T10:04:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.972+0000 7f0080e4f640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006006b500 0x7f006006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.972+0000 7f0080e4f640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006006b500 0x7f006006d9c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0070006fd0 tx=0x7f0070008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:08.976+0000 7f00797fa640 1 -- 172.21.15.19:0/1370559643 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f007c13fa80 con 0x7f007c1454b0 2024-04-25T10:04:08.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:08.992 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:09.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.157+0000 7f00830b3640 1 -- 172.21.15.19:0/1370559643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f007c0b0f50 con 0x7f007c1454b0 2024-04-25T10:04:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.160+0000 7f00797fa640 1 -- 172.21.15.19:0/1370559643 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f00740ac0c0 con 0x7f007c1454b0 2024-04-25T10:04:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.162+0000 7f00830b3640 1 -- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006006b500 msgr2=0x7f006006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.162+0000 7f00830b3640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006006b500 0x7f006006d9c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0070006fd0 tx=0x7f0070008040 comp rx=0 tx=0).stop 2024-04-25T10:04:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.162+0000 7f00830b3640 1 -- 172.21.15.19:0/1370559643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c1454b0 msgr2=0x7f007c0cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.162+0000 7f00830b3640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c1454b0 0x7f007c0cdac0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f0074034040 tx=0x7f0074004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.162+0000 7f00830b3640 1 -- 172.21.15.19:0/1370559643 shutdown_connections 2024-04-25T10:04:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.162+0000 7f00830b3640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f006006b500 0x7f006006d9c0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.162+0000 7f00830b3640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c1454b0 0x7f007c0cdac0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.162+0000 7f00830b3640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f007c144d20 0x7f007c0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.163+0000 7f00830b3640 1 --2- 172.21.15.19:0/1370559643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f007c0714d0 0x7f007c13f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.163+0000 7f00830b3640 1 -- 172.21.15.19:0/1370559643 >> 172.21.15.19:0/1370559643 conn(0x7f007c0b3530 msgr2=0x7f007c0b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.163+0000 7f00830b3640 1 -- 172.21.15.19:0/1370559643 shutdown_connections 2024-04-25T10:04:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.163+0000 7f00830b3640 1 -- 172.21.15.19:0/1370559643 wait complete. 2024-04-25T10:04:09.178 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:04:09.179 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:09.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:09.259 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:04:09.259 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:04:09.259 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:09.276 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:09.276 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:04:09.276 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:09.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:09.317 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:04:09.317 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:09.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:09.333 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:04:09.333 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:09.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:09.374 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:04:09.374 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:09.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.470+0000 7fcd52570640 1 Processor -- start 2024-04-25T10:04:09.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.470+0000 7fcd52570640 1 -- start start 2024-04-25T10:04:09.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.471+0000 7fcd52570640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd4c0b96d0 0x7fcd4c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.471+0000 7fcd52570640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcd4c0d0fe0 0x7fcd4c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.471+0000 7fcd52570640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4c15e810 con 0x7fcd4c15dd20 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.471+0000 7fcd52570640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4c0d1c00 con 0x7fcd4c0b96d0 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.471+0000 7fcd52570640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4c0d1420 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.471+0000 7fcd4b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd4c0b96d0 0x7fcd4c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.471+0000 7fcd4b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd4c0b96d0 0x7fcd4c0d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37866/0 (socket says 172.21.15.19:37866) 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.471+0000 7fcd4b7fe640 1 -- 172.21.15.19:0/2300307832 learned_addr learned my addr 172.21.15.19:0/2300307832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.471+0000 7fcd50b0d640 1 --2- 172.21.15.19:0/2300307832 >> v2:172.21.15.112:3300/0 conn(0x7fcd4c0d0fe0 0x7fcd4c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.472+0000 7fcd4b7fe640 1 -- 172.21.15.19:0/2300307832 >> v2:172.21.15.112:3300/0 conn(0x7fcd4c0d0fe0 msgr2=0x7fcd4c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.472+0000 7fcd4b7fe640 1 --2- 172.21.15.19:0/2300307832 >> v2:172.21.15.112:3300/0 conn(0x7fcd4c0d0fe0 0x7fcd4c0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.472+0000 7fcd4b7fe640 1 -- 172.21.15.19:0/2300307832 >> v1:172.21.15.112:6789/0 conn(0x7fcd4c15dd20 legacy=0x7fcd4c15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.472+0000 7fcd4b7fe640 1 -- 172.21.15.19:0/2300307832 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c15eac0 con 0x7fcd4c0b96d0 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.472+0000 7fcd4b7fe640 1 --2- 172.21.15.19:0/2300307832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0b96d0 0x7fcd4c0d0aa0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0d1990 tx=0x7fcd4002a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295a5cebb1ac0c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.473+0000 7fcd4affd640 1 -- 172.21.15.19:0/2300307832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40037070 con 0x7fcd4c0b96d0 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.473+0000 7fcd4affd640 1 -- 172.21.15.19:0/2300307832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd40032440 con 0x7fcd4c0b96d0 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.473+0000 7fcd4affd640 1 -- 172.21.15.19:0/2300307832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40032810 con 0x7fcd4c0b96d0 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.473+0000 7fcd52570640 1 -- 172.21.15.19:0/2300307832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0b96d0 msgr2=0x7fcd4c0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.473+0000 7fcd52570640 1 --2- 172.21.15.19:0/2300307832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0b96d0 0x7fcd4c0d0aa0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0d1990 tx=0x7fcd4002a1f0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.473+0000 7fcd52570640 1 -- 172.21.15.19:0/2300307832 shutdown_connections 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.473+0000 7fcd52570640 1 --2- 172.21.15.19:0/2300307832 >> v2:172.21.15.112:3300/0 conn(0x7fcd4c0d0fe0 0x7fcd4c0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.473+0000 7fcd52570640 1 --2- 172.21.15.19:0/2300307832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0b96d0 0x7fcd4c0d0aa0 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.473+0000 7fcd52570640 1 -- 172.21.15.19:0/2300307832 >> 172.21.15.19:0/2300307832 conn(0x7fcd4c061a20 msgr2=0x7fcd4c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.473+0000 7fcd52570640 1 -- 172.21.15.19:0/2300307832 shutdown_connections 2024-04-25T10:04:09.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.473+0000 7fcd52570640 1 -- 172.21.15.19:0/2300307832 wait complete. 2024-04-25T10:04:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.474+0000 7fcd52570640 1 Processor -- start 2024-04-25T10:04:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.475+0000 7fcd52570640 1 -- start start 2024-04-25T10:04:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.475+0000 7fcd52570640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c0b96d0 0x7fcd4c14ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.475+0000 7fcd52570640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0d0fe0 0x7fcd4c0c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.475+0000 7fcd52570640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd4c15dd20 0x7fcd4c0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.475+0000 7fcd52570640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0d1e50 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.475+0000 7fcd52570640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c15f080 con 0x7fcd4c0b96d0 2024-04-25T10:04:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.475+0000 7fcd52570640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0d28b0 con 0x7fcd4c15dd20 2024-04-25T10:04:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.475+0000 7fcd4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0d0fe0 0x7fcd4c0c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.475+0000 7fcd4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0d0fe0 0x7fcd4c0c99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37876/0 (socket says 172.21.15.19:37876) 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.475+0000 7fcd4b7fe640 1 -- 172.21.15.19:0/4225483009 learned_addr learned my addr 172.21.15.19:0/4225483009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd50b0d640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd4c15dd20 0x7fcd4c0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd4bfff640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c0b96d0 0x7fcd4c14ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd4b7fe640 1 -- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd4c15dd20 msgr2=0x7fcd4c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd4b7fe640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd4c15dd20 0x7fcd4c0cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd4b7fe640 1 -- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c0b96d0 msgr2=0x7fcd4c14ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd4b7fe640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c0b96d0 0x7fcd4c14ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd4b7fe640 1 -- 172.21.15.19:0/4225483009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c0c8330 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd4b7fe640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0d0fe0 0x7fcd4c0c99c0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fcd4002a230 tx=0x7fcd400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd48ff9640 1 -- 172.21.15.19:0/4225483009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40037040 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd48ff9640 1 -- 172.21.15.19:0/4225483009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd40042070 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd52570640 1 -- 172.21.15.19:0/4225483009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4c0c8610 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd48ff9640 1 -- 172.21.15.19:0/4225483009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40030880 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.476+0000 7fcd52570640 1 -- 172.21.15.19:0/4225483009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4c0c8eb0 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.477+0000 7fcd52570640 1 -- 172.21.15.19:0/4225483009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4c0c87e0 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.478+0000 7fcd48ff9640 1 -- 172.21.15.19:0/4225483009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd40030a20 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.478+0000 7fcd48ff9640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd3806b4d0 0x7fcd3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.478+0000 7fcd48ff9640 1 -- 172.21.15.19:0/4225483009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcd400aa950 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.478+0000 7fcd4bfff640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd3806b4d0 0x7fcd3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.479+0000 7fcd4bfff640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd3806b4d0 0x7fcd3806d990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007fd0 tx=0x7fcd3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.482+0000 7fcd48ff9640 1 -- 172.21.15.19:0/4225483009 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd40079940 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.699+0000 7fcd52570640 1 -- 172.21.15.19:0/4225483009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd4c073990 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.699+0000 7fcd48ff9640 1 -- 172.21.15.19:0/4225483009 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+3706 (secure 0 0 0) 0x7fcd40079240 con 0x7fcd4c0d0fe0 2024-04-25T10:04:09.704 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:09.704 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":104,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3725393673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3725393673},{"type":"v1","addr":"172.21.15.112:6837","nonce":3725393673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8866,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2539453196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2539453196},{"type":"v1","addr":"172.21.15.112:6836","nonce":2539453196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10221,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3896097924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3896097924},{"type":"v1","addr":"172.21.15.19:6834","nonce":3896097924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10620,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/451312990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":451312990},{"type":"v1","addr":"172.21.15.19:6835","nonce":451312990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":104}],"filesystems":[]} 2024-04-25T10:04:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 104 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 -- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd3806b4d0 msgr2=0x7fcd3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd3806b4d0 0x7fcd3806d990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007fd0 tx=0x7fcd3c009040 comp rx=0 tx=0).stop 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 -- 172.21.15.19:0/4225483009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0d0fe0 msgr2=0x7fcd4c0c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0d0fe0 0x7fcd4c0c99c0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fcd4002a230 tx=0x7fcd400046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 -- 172.21.15.19:0/4225483009 shutdown_connections 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd3806b4d0 0x7fcd3806d990 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd4c15dd20 0x7fcd4c0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0d0fe0 0x7fcd4c0c99c0 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 --2- 172.21.15.19:0/4225483009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c0b96d0 0x7fcd4c14ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 -- 172.21.15.19:0/4225483009 >> 172.21.15.19:0/4225483009 conn(0x7fcd4c061a20 msgr2=0x7fcd4c0b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 -- 172.21.15.19:0/4225483009 shutdown_connections 2024-04-25T10:04:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.701+0000 7fcd52570640 1 -- 172.21.15.19:0/4225483009 wait complete. 2024-04-25T10:04:09.717 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:04:09.718 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:04:09.718 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:04:09.718 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:09.718 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:04:09.718 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:04:09.719 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:04:09.719 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:09.719 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:04:09.719 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:04:09.719 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:04:09.719 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:09.719 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:04:09.720 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:04:09.720 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:04:09.720 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:09.730 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:04:09.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:04:09.731 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:04:09.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:04:09.733 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:04:09.733 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:04:09.734 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:04:09.734 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:04:09.773 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:04:09.774 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:09.775 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:04:09.775 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:09.776 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:04:09.776 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:09.777 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:04:09.777 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:09.778 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:04:09.853 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T10:04:09.853 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:04:09.858 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T10:04:09.858 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T10:04:09.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.880+0000 7fd0871a5640 1 Processor -- start 2024-04-25T10:04:09.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.880+0000 7fd0871a5640 1 -- start start 2024-04-25T10:04:09.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.881+0000 7fd0871a5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd080161710 0x7fd0800d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.881+0000 7fd0871a5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08015e000 con 0x7fd08015dc20 2024-04-25T10:04:09.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.881+0000 7fd0871a5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0800d1de0 con 0x7fd0800ced80 2024-04-25T10:04:09.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.881+0000 7fd0871a5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd080161c60 con 0x7fd080161710 2024-04-25T10:04:09.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.881+0000 7fd084f41640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd08015dc20 0x7fd0800ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41346/0 (socket says 172.21.15.19:41346) 2024-04-25T10:04:09.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.881+0000 7fd084f41640 1 -- 172.21.15.19:0/3544617843 learned_addr learned my addr 172.21.15.19:0/3544617843 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:09.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.881+0000 7fd085742640 1 --2- 172.21.15.19:0/3544617843 >> v2:172.21.15.19:3300/0 conn(0x7fd080161710 0x7fd0800d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.882+0000 7fd07f7fe640 1 -- 172.21.15.19:0/3544617843 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151393264 0 0) 0x7fd08015e000 con 0x7fd08015dc20 2024-04-25T10:04:09.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.882+0000 7fd07f7fe640 1 -- 172.21.15.19:0/3544617843 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd068003520 con 0x7fd08015dc20 2024-04-25T10:04:09.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.882+0000 7fd07f7fe640 1 -- 172.21.15.19:0/3544617843 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586334454 0 0) 0x7fd0800d1de0 con 0x7fd0800ced80 2024-04-25T10:04:09.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.882+0000 7fd07f7fe640 1 -- 172.21.15.19:0/3544617843 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd08015e000 con 0x7fd0800ced80 2024-04-25T10:04:09.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.882+0000 7fd085742640 1 -- 172.21.15.19:0/3544617843 >> v1:172.21.15.112:6789/0 conn(0x7fd0800ced80 legacy=0x7fd0800d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.882+0000 7fd085742640 1 -- 172.21.15.19:0/3544617843 >> v1:172.21.15.19:6789/0 conn(0x7fd08015dc20 legacy=0x7fd0800ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.882+0000 7fd07f7fe640 1 -- 172.21.15.19:0/3544617843 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3307612582 0 0) 0x7fd068003520 con 0x7fd08015dc20 2024-04-25T10:04:09.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.882+0000 7fd085742640 1 -- 172.21.15.19:0/3544617843 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd080161f40 con 0x7fd080161710 2024-04-25T10:04:09.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.883+0000 7fd085742640 1 --2- 172.21.15.19:0/3544617843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080161710 0x7fd0800d29f0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fd0700097a0 tx=0x7fd07002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d305e2a303f580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.883+0000 7fd07f7fe640 1 -- 172.21.15.19:0/3544617843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070037070 con 0x7fd080161710 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.883+0000 7fd07f7fe640 1 -- 172.21.15.19:0/3544617843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd070032440 con 0x7fd080161710 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.883+0000 7fd07f7fe640 1 -- 172.21.15.19:0/3544617843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070032810 con 0x7fd080161710 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.883+0000 7fd0871a5640 1 -- 172.21.15.19:0/3544617843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080161710 msgr2=0x7fd0800d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.883+0000 7fd0871a5640 1 --2- 172.21.15.19:0/3544617843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080161710 0x7fd0800d29f0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fd0700097a0 tx=0x7fd07002a230 comp rx=0 tx=0).stop 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.883+0000 7fd0871a5640 1 -- 172.21.15.19:0/3544617843 shutdown_connections 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.883+0000 7fd0871a5640 1 --2- 172.21.15.19:0/3544617843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080161710 0x7fd0800d29f0 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.883+0000 7fd0871a5640 1 -- 172.21.15.19:0/3544617843 >> 172.21.15.19:0/3544617843 conn(0x7fd0800d0420 msgr2=0x7fd0800d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.883+0000 7fd0871a5640 1 -- 172.21.15.19:0/3544617843 shutdown_connections 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.883+0000 7fd0871a5640 1 -- 172.21.15.19:0/3544617843 wait complete. 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.885+0000 7fd0871a5640 1 Processor -- start 2024-04-25T10:04:09.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.885+0000 7fd0871a5640 1 -- start start 2024-04-25T10:04:09.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd0871a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0800ced80 0x7fd080155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd084f41640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0800ced80 0x7fd080155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd084f41640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0800ced80 0x7fd080155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37898/0 (socket says 172.21.15.19:37898) 2024-04-25T10:04:09.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd0871a5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd08015dc20 0x7fd0800c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd084f41640 1 -- 172.21.15.19:0/2686533919 learned_addr learned my addr 172.21.15.19:0/2686533919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:09.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd0871a5640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd080161710 0x7fd0800c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0800b5e50 con 0x7fd0800ced80 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080162500 con 0x7fd080161710 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0800b5fd0 con 0x7fd08015dc20 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd07ffff640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd08015dc20 0x7fd0800c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd085742640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd080161710 0x7fd0800c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd084f41640 1 -- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd08015dc20 msgr2=0x7fd0800c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd084f41640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd08015dc20 0x7fd0800c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd084f41640 1 -- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd080161710 msgr2=0x7fd0800c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd084f41640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd080161710 0x7fd0800c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd084f41640 1 -- 172.21.15.19:0/2686533919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08015f410 con 0x7fd0800ced80 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd085742640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd080161710 0x7fd0800c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd07ffff640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd08015dc20 0x7fd0800c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:09.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd084f41640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0800ced80 0x7fd080155aa0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fd07800cb60 tx=0x7fd07800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.886+0000 7fd07d7fa640 1 -- 172.21.15.19:0/2686533919 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078017070 con 0x7fd0800ced80 2024-04-25T10:04:09.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.887+0000 7fd07d7fa640 1 -- 172.21.15.19:0/2686533919 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07800ad40 con 0x7fd0800ced80 2024-04-25T10:04:09.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.887+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08015f6f0 con 0x7fd0800ced80 2024-04-25T10:04:09.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.887+0000 7fd07d7fa640 1 -- 172.21.15.19:0/2686533919 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078012600 con 0x7fd0800ced80 2024-04-25T10:04:09.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.887+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd080155fe0 con 0x7fd0800ced80 2024-04-25T10:04:09.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.888+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0800cd030 con 0x7fd0800ced80 2024-04-25T10:04:09.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.888+0000 7fd07d7fa640 1 -- 172.21.15.19:0/2686533919 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd07801a070 con 0x7fd0800ced80 2024-04-25T10:04:09.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.888+0000 7fd07d7fa640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd06c06b500 0x7fd06c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.888+0000 7fd07d7fa640 1 -- 172.21.15.19:0/2686533919 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd078089ed0 con 0x7fd0800ced80 2024-04-25T10:04:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.889+0000 7fd07ffff640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd06c06b500 0x7fd06c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.889+0000 7fd07ffff640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd06c06b500 0x7fd06c06d9c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd0801565c0 tx=0x7fd074009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:09.893+0000 7fd07d7fa640 1 -- 172.21.15.19:0/2686533919 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd078058750 con 0x7fd0800ced80 2024-04-25T10:04:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.084+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd0800d16b0 con 0x7fd0800ced80 2024-04-25T10:04:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.860+0000 7fd07d7fa640 1 -- 172.21.15.19:0/2686533919 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v159) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd07805c7f0 con 0x7fd0800ced80 2024-04-25T10:04:10.865 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T10:04:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd06c06b500 msgr2=0x7fd06c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd06c06b500 0x7fd06c06d9c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd0801565c0 tx=0x7fd074009040 comp rx=0 tx=0).stop 2024-04-25T10:04:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0800ced80 msgr2=0x7fd080155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0800ced80 0x7fd080155aa0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fd07800cb60 tx=0x7fd07800a430 comp rx=0 tx=0).stop 2024-04-25T10:04:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 shutdown_connections 2024-04-25T10:04:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd06c06b500 0x7fd06c06d9c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd080161710 0x7fd0800c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd08015dc20 0x7fd0800c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 --2- 172.21.15.19:0/2686533919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0800ced80 0x7fd080155aa0 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 >> 172.21.15.19:0/2686533919 conn(0x7fd0800d0420 msgr2=0x7fd0800d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 shutdown_connections 2024-04-25T10:04:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.864+0000 7fd0871a5640 1 -- 172.21.15.19:0/2686533919 wait complete. 2024-04-25T10:04:10.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:04:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.986+0000 7f5d41334640 1 Processor -- start 2024-04-25T10:04:10.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.986+0000 7f5d41334640 1 -- start start 2024-04-25T10:04:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d41334640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d3c15db70 0x7f5d3c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d41334640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d3c0b5c30 0x7f5d3c0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d41334640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c162690 con 0x7f5d3c0b6cb0 2024-04-25T10:04:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d41334640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3c0ceee0 con 0x7f5d3c15db70 2024-04-25T10:04:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d41334640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0d1d10 con 0x7f5d3c0b5c30 2024-04-25T10:04:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d3affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d3c15db70 0x7f5d3c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d3affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d3c15db70 0x7f5d3c162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37912/0 (socket says 172.21.15.19:37912) 2024-04-25T10:04:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d3affd640 1 -- 172.21.15.19:0/3219481395 learned_addr learned my addr 172.21.15.19:0/3219481395 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d3b7fe640 1 --2- 172.21.15.19:0/3219481395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d3c0b5c30 0x7f5d3c0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d3affd640 1 -- 172.21.15.19:0/3219481395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d3c0b5c30 msgr2=0x7f5d3c0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d3affd640 1 --2- 172.21.15.19:0/3219481395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d3c0b5c30 0x7f5d3c0d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d3affd640 1 -- 172.21.15.19:0/3219481395 >> v1:172.21.15.112:6789/0 conn(0x7f5d3c0b6cb0 legacy=0x7f5d3c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d3affd640 1 -- 172.21.15.19:0/3219481395 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0d1ff0 con 0x7f5d3c15db70 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.987+0000 7f5d3b7fe640 1 --2- 172.21.15.19:0/3219481395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d3c0b5c30 0x7f5d3c0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.988+0000 7f5d3affd640 1 --2- 172.21.15.19:0/3219481395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d3c15db70 0x7f5d3c162150 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009870 tx=0x7f5d2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4646719adc8034d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.988+0000 7f5d3a7fc640 1 -- 172.21.15.19:0/3219481395 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c037070 con 0x7f5d3c15db70 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.988+0000 7f5d3a7fc640 1 -- 172.21.15.19:0/3219481395 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c032440 con 0x7f5d3c15db70 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.988+0000 7f5d3a7fc640 1 -- 172.21.15.19:0/3219481395 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c032810 con 0x7f5d3c15db70 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.989+0000 7f5d41334640 1 -- 172.21.15.19:0/3219481395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d3c15db70 msgr2=0x7f5d3c162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.989+0000 7f5d41334640 1 --2- 172.21.15.19:0/3219481395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d3c15db70 0x7f5d3c162150 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009870 tx=0x7f5d2c02a270 comp rx=0 tx=0).stop 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.989+0000 7f5d41334640 1 -- 172.21.15.19:0/3219481395 shutdown_connections 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.989+0000 7f5d41334640 1 --2- 172.21.15.19:0/3219481395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d3c0b5c30 0x7f5d3c0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.989+0000 7f5d41334640 1 --2- 172.21.15.19:0/3219481395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d3c15db70 0x7f5d3c162150 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.989+0000 7f5d41334640 1 -- 172.21.15.19:0/3219481395 >> 172.21.15.19:0/3219481395 conn(0x7f5d3c0d0520 msgr2=0x7f5d3c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:10.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.989+0000 7f5d41334640 1 -- 172.21.15.19:0/3219481395 shutdown_connections 2024-04-25T10:04:10.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.989+0000 7f5d41334640 1 -- 172.21.15.19:0/3219481395 wait complete. 2024-04-25T10:04:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.990+0000 7f5d41334640 1 Processor -- start 2024-04-25T10:04:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.990+0000 7f5d41334640 1 -- start start 2024-04-25T10:04:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d41334640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d3c0b5c30 0x7f5d3c0cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d41334640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c0b6cb0 0x7f5d3c0d2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d41334640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d3c15db70 0x7f5d3c14bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d41334640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0ce530 con 0x7f5d3c15db70 2024-04-25T10:04:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d41334640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c15df50 con 0x7f5d3c0b6cb0 2024-04-25T10:04:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d41334640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0cf1c0 con 0x7f5d3c0b5c30 2024-04-25T10:04:10.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d3b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d3c15db70 0x7f5d3c14bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d3b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d3c15db70 0x7f5d3c14bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37918/0 (socket says 172.21.15.19:37918) 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d3b7fe640 1 -- 172.21.15.19:0/1833064436 learned_addr learned my addr 172.21.15.19:0/1833064436 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d33fff640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c0b6cb0 0x7f5d3c0d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d3affd640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d3c0b5c30 0x7f5d3c0cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d3b7fe640 1 -- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d3c0b5c30 msgr2=0x7f5d3c0cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d3b7fe640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d3c0b5c30 0x7f5d3c0cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d3b7fe640 1 -- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c0b6cb0 msgr2=0x7f5d3c0d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d3b7fe640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c0b6cb0 0x7f5d3c0d2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.991+0000 7f5d3b7fe640 1 -- 172.21.15.19:0/1833064436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0cac10 con 0x7f5d3c15db70 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.992+0000 7f5d3b7fe640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d3c15db70 0x7f5d3c14bb20 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400cd10 tx=0x7f5d3400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.992+0000 7f5d337fe640 1 -- 172.21.15.19:0/1833064436 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34003b90 con 0x7f5d3c15db70 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.992+0000 7f5d337fe640 1 -- 172.21.15.19:0/1833064436 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d34003d50 con 0x7f5d3c15db70 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.992+0000 7f5d41334640 1 -- 172.21.15.19:0/1833064436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c0caef0 con 0x7f5d3c15db70 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.992+0000 7f5d337fe640 1 -- 172.21.15.19:0/1833064436 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34004260 con 0x7f5d3c15db70 2024-04-25T10:04:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.992+0000 7f5d41334640 1 -- 172.21.15.19:0/1833064436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c1e9600 con 0x7f5d3c15db70 2024-04-25T10:04:11.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.993+0000 7f5d41334640 1 -- 172.21.15.19:0/1833064436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c073990 con 0x7f5d3c15db70 2024-04-25T10:04:11.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.993+0000 7f5d337fe640 1 -- 172.21.15.19:0/1833064436 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d34004400 con 0x7f5d3c15db70 2024-04-25T10:04:11.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.994+0000 7f5d337fe640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d1c06b4d0 0x7f5d1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.994+0000 7f5d337fe640 1 -- 172.21.15.19:0/1833064436 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5d3408a960 con 0x7f5d3c15db70 2024-04-25T10:04:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.994+0000 7f5d3affd640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d1c06b4d0 0x7f5d1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.994+0000 7f5d3affd640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d1c06b4d0 0x7f5d1c06d990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c034000 tx=0x7f5d2c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:10.998+0000 7f5d337fe640 1 -- 172.21.15.19:0/1833064436 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d34014030 con 0x7f5d3c15db70 2024-04-25T10:04:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.243+0000 7f5d41334640 1 -- 172.21.15.19:0/1833064436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5d3c0d1820 con 0x7f5d3c15db70 2024-04-25T10:04:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.244+0000 7f5d337fe640 1 -- 172.21.15.19:0/1833064436 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5d340593e0 con 0x7f5d3c15db70 2024-04-25T10:04:11.249 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:11.249 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:04:11.249 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T10:04:11.249 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.249 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T10:04:11.249 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T10:04:11.249 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:04:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.251 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:11.252 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.255 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:04:11.256 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T10:04:11.257 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:04:11.258 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.259 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.260 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.261 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T10:04:11.262 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:04:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 -- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d1c06b4d0 msgr2=0x7f5d1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d1c06b4d0 0x7f5d1c06d990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c034000 tx=0x7f5d2c02e000 comp rx=0 tx=0).stop 2024-04-25T10:04:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 -- 172.21.15.19:0/1833064436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d3c15db70 msgr2=0x7f5d3c14bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d3c15db70 0x7f5d3c14bb20 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400cd10 tx=0x7f5d3400a650 comp rx=0 tx=0).stop 2024-04-25T10:04:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 -- 172.21.15.19:0/1833064436 shutdown_connections 2024-04-25T10:04:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5d1c06b4d0 0x7f5d1c06d990 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d3c15db70 0x7f5d3c14bb20 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d3c0b6cb0 0x7f5d3c0d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 --2- 172.21.15.19:0/1833064436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d3c0b5c30 0x7f5d3c0cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 -- 172.21.15.19:0/1833064436 >> 172.21.15.19:0/1833064436 conn(0x7f5d3c0d0520 msgr2=0x7f5d3c159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 -- 172.21.15.19:0/1833064436 shutdown_connections 2024-04-25T10:04:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.246+0000 7f5d41334640 1 -- 172.21.15.19:0/1833064436 wait complete. 2024-04-25T10:04:11.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.361+0000 7ff62ab34640 1 Processor -- start 2024-04-25T10:04:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.361+0000 7ff62ab34640 1 -- start start 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff62ab34640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff62415dbe0 0x7ff6240d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff62ab34640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6240b2cb0 0x7ff6240d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff62ab34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6240b92a0 0x7ff6240b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff62ab34640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62415dfc0 con 0x7ff6240b2cb0 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff62ab34640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6240d1a00 con 0x7ff62415dbe0 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff62ab34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6240b97a0 con 0x7ff6240b92a0 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff623fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6240b2cb0 0x7ff6240d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff623fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6240b2cb0 0x7ff6240d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37922/0 (socket says 172.21.15.19:37922) 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff623fff640 1 -- 172.21.15.19:0/1622874839 learned_addr learned my addr 172.21.15.19:0/1622874839 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff623fff640 1 -- 172.21.15.19:0/1622874839 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6240b92a0 msgr2=0x7ff6240b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff623fff640 1 --2- 172.21.15.19:0/1622874839 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6240b92a0 0x7ff6240b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff623fff640 1 -- 172.21.15.19:0/1622874839 >> v2:172.21.15.112:3300/0 conn(0x7ff62415dbe0 msgr2=0x7ff6240d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff6288d0640 1 --2- 172.21.15.19:0/1622874839 >> v2:172.21.15.112:3300/0 conn(0x7ff62415dbe0 0x7ff6240d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff623fff640 1 --2- 172.21.15.19:0/1622874839 >> v2:172.21.15.112:3300/0 conn(0x7ff62415dbe0 0x7ff6240d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.362+0000 7ff623fff640 1 -- 172.21.15.19:0/1622874839 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6240d1c50 con 0x7ff6240b2cb0 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.363+0000 7ff623fff640 1 --2- 172.21.15.19:0/1622874839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6240b2cb0 0x7ff6240d0e50 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7ff62406c140 tx=0x7ff61002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5996f49624cd1980 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.363+0000 7ff6237fe640 1 -- 172.21.15.19:0/1622874839 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610037070 con 0x7ff6240b2cb0 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.363+0000 7ff6237fe640 1 -- 172.21.15.19:0/1622874839 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff610032440 con 0x7ff6240b2cb0 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.363+0000 7ff6237fe640 1 -- 172.21.15.19:0/1622874839 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610032810 con 0x7ff6240b2cb0 2024-04-25T10:04:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.363+0000 7ff62ab34640 1 -- 172.21.15.19:0/1622874839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6240b2cb0 msgr2=0x7ff6240d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.363+0000 7ff62ab34640 1 --2- 172.21.15.19:0/1622874839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6240b2cb0 0x7ff6240d0e50 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7ff62406c140 tx=0x7ff61002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.364+0000 7ff62ab34640 1 -- 172.21.15.19:0/1622874839 shutdown_connections 2024-04-25T10:04:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.364+0000 7ff62ab34640 1 --2- 172.21.15.19:0/1622874839 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6240b92a0 0x7ff6240b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.364+0000 7ff62ab34640 1 --2- 172.21.15.19:0/1622874839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6240b2cb0 0x7ff6240d0e50 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.364+0000 7ff62ab34640 1 --2- 172.21.15.19:0/1622874839 >> v2:172.21.15.112:3300/0 conn(0x7ff62415dbe0 0x7ff6240d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.364+0000 7ff62ab34640 1 -- 172.21.15.19:0/1622874839 >> 172.21.15.19:0/1622874839 conn(0x7ff624061a20 msgr2=0x7ff624061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.364+0000 7ff62ab34640 1 -- 172.21.15.19:0/1622874839 shutdown_connections 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.364+0000 7ff62ab34640 1 -- 172.21.15.19:0/1622874839 wait complete. 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.365+0000 7ff62ab34640 1 Processor -- start 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.365+0000 7ff62ab34640 1 -- start start 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff62ab34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6240b2cb0 0x7ff6241f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff62ab34640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6240b92a0 0x7ff6241531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff62ab34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff62415dbe0 0x7ff624153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff62ab34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6240b6ab0 con 0x7ff62415dbe0 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff62ab34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6240d2210 con 0x7ff6240b2cb0 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff62ab34640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6240b6c30 con 0x7ff6240b92a0 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6290d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff62415dbe0 0x7ff624153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6290d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff62415dbe0 0x7ff624153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37936/0 (socket says 172.21.15.19:37936) 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6290d1640 1 -- 172.21.15.19:0/2348481746 learned_addr learned my addr 172.21.15.19:0/2348481746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6288d0640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6240b2cb0 0x7ff6241f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff623fff640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6240b92a0 0x7ff6241531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6290d1640 1 -- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6240b92a0 msgr2=0x7ff6241531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6290d1640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6240b92a0 0x7ff6241531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6290d1640 1 -- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6240b2cb0 msgr2=0x7ff6241f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6290d1640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6240b2cb0 0x7ff6241f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6290d1640 1 -- 172.21.15.19:0/2348481746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff624160190 con 0x7ff62415dbe0 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6290d1640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff62415dbe0 0x7ff624153700 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00b9e0 tx=0x7ff61c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6217fa640 1 -- 172.21.15.19:0/2348481746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c019070 con 0x7ff62415dbe0 2024-04-25T10:04:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6217fa640 1 -- 172.21.15.19:0/2348481746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c00cb40 con 0x7ff62415dbe0 2024-04-25T10:04:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff62ab34640 1 -- 172.21.15.19:0/2348481746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff624160470 con 0x7ff62415dbe0 2024-04-25T10:04:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.366+0000 7ff6217fa640 1 -- 172.21.15.19:0/2348481746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c014440 con 0x7ff62415dbe0 2024-04-25T10:04:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.367+0000 7ff62ab34640 1 -- 172.21.15.19:0/2348481746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff624156840 con 0x7ff62415dbe0 2024-04-25T10:04:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.367+0000 7ff62ab34640 1 -- 172.21.15.19:0/2348481746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff624156370 con 0x7ff62415dbe0 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.368+0000 7ff6217fa640 1 -- 172.21.15.19:0/2348481746 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff61c01c070 con 0x7ff62415dbe0 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.368+0000 7ff6217fa640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff60806b4d0 0x7ff60806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.368+0000 7ff6217fa640 1 -- 172.21.15.19:0/2348481746 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff61c089ce0 con 0x7ff62415dbe0 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.369+0000 7ff6288d0640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff60806b4d0 0x7ff60806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.369+0000 7ff6288d0640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff60806b4d0 0x7ff60806d990 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff618006fd0 tx=0x7ff618008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.372+0000 7ff6217fa640 1 -- 172.21.15.19:0/2348481746 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff61c058db0 con 0x7ff62415dbe0 2024-04-25T10:04:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.556+0000 7ff62ab34640 1 -- 172.21.15.19:0/2348481746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff624073990 con 0x7ff62415dbe0 2024-04-25T10:04:11.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.557+0000 7ff6217fa640 1 -- 172.21.15.19:0/2348481746 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+13162 (secure 0 0 0) 0x7ff61c0586b0 con 0x7ff62415dbe0 2024-04-25T10:04:11.562 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:11.562 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":161,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:04:10.872953+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 -- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff60806b4d0 msgr2=0x7ff60806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff60806b4d0 0x7ff60806d990 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff618006fd0 tx=0x7ff618008040 comp rx=0 tx=0).stop 2024-04-25T10:04:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 -- 172.21.15.19:0/2348481746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff62415dbe0 msgr2=0x7ff624153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff62415dbe0 0x7ff624153700 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00b9e0 tx=0x7ff61c00bea0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 -- 172.21.15.19:0/2348481746 shutdown_connections 2024-04-25T10:04:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff60806b4d0 0x7ff60806d990 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff62415dbe0 0x7ff624153700 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6240b92a0 0x7ff6241531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 --2- 172.21.15.19:0/2348481746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6240b2cb0 0x7ff6241f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 -- 172.21.15.19:0/2348481746 >> 172.21.15.19:0/2348481746 conn(0x7ff624061a20 msgr2=0x7ff6240b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 -- 172.21.15.19:0/2348481746 shutdown_connections 2024-04-25T10:04:11.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.559+0000 7ff62ab34640 1 -- 172.21.15.19:0/2348481746 wait complete. 2024-04-25T10:04:11.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:11.575 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.673+0000 7f1c032e1640 1 Processor -- start 2024-04-25T10:04:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.674+0000 7f1c032e1640 1 -- start start 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.674+0000 7f1c032e1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1bfc1616f0 0x7f1bfc161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.674+0000 7f1c032e1640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bfc06c8d0 con 0x7f1bfc1610f0 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.674+0000 7f1c032e1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bfc0c6960 con 0x7f1bfc0ce0a0 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.674+0000 7f1c032e1640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bfc1614d0 con 0x7f1bfc1616f0 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.674+0000 7f1c0087c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1bfc1616f0 0x7f1bfc161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.674+0000 7f1c0107d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1bfc1610f0 0x7f1bfc06c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41358/0 (socket says 172.21.15.19:41358) 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.675+0000 7f1c0107d640 1 -- 172.21.15.19:0/4139569020 learned_addr learned my addr 172.21.15.19:0/4139569020 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.675+0000 7f1bf3fff640 1 -- 172.21.15.19:0/4139569020 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992254424 0 0) 0x7f1bfc06c8d0 con 0x7f1bfc1610f0 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.675+0000 7f1bf3fff640 1 -- 172.21.15.19:0/4139569020 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1be8003520 con 0x7f1bfc1610f0 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.675+0000 7f1c0087c640 1 -- 172.21.15.19:0/4139569020 >> v1:172.21.15.112:6789/0 conn(0x7f1bfc0ce0a0 legacy=0x7f1bfc0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.675+0000 7f1c0087c640 1 -- 172.21.15.19:0/4139569020 >> v1:172.21.15.19:6789/0 conn(0x7f1bfc1610f0 legacy=0x7f1bfc06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.675+0000 7f1c0087c640 1 -- 172.21.15.19:0/4139569020 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc162260 con 0x7f1bfc1616f0 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.676+0000 7f1c0087c640 1 --2- 172.21.15.19:0/4139569020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bfc1616f0 0x7f1bfc161b50 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0097a0 tx=0x7f1bec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c093cf0d4ef0e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.676+0000 7f1bf3fff640 1 -- 172.21.15.19:0/4139569020 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec037070 con 0x7f1bfc1616f0 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.676+0000 7f1bf3fff640 1 -- 172.21.15.19:0/4139569020 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec032440 con 0x7f1bfc1616f0 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.676+0000 7f1bf3fff640 1 -- 172.21.15.19:0/4139569020 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec032810 con 0x7f1bfc1616f0 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.676+0000 7f1c032e1640 1 -- 172.21.15.19:0/4139569020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bfc1616f0 msgr2=0x7f1bfc161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.676+0000 7f1c032e1640 1 --2- 172.21.15.19:0/4139569020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bfc1616f0 0x7f1bfc161b50 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0097a0 tx=0x7f1bec02a230 comp rx=0 tx=0).stop 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.676+0000 7f1c032e1640 1 -- 172.21.15.19:0/4139569020 shutdown_connections 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.676+0000 7f1c032e1640 1 --2- 172.21.15.19:0/4139569020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bfc1616f0 0x7f1bfc161b50 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.676+0000 7f1c032e1640 1 -- 172.21.15.19:0/4139569020 >> 172.21.15.19:0/4139569020 conn(0x7f1bfc06f500 msgr2=0x7f1bfc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.676+0000 7f1c032e1640 1 -- 172.21.15.19:0/4139569020 shutdown_connections 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.676+0000 7f1c032e1640 1 -- 172.21.15.19:0/4139569020 wait complete. 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.677+0000 7f1c032e1640 1 Processor -- start 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.678+0000 7f1c032e1640 1 -- start start 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.678+0000 7f1c032e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc0ce0a0 0x7f1bfc0b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.678+0000 7f1c032e1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bfc1610f0 0x7f1bfc14b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.678+0000 7f1c032e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bfc1616f0 0x7f1bfc15cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.678+0000 7f1c032e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc0cfce0 con 0x7f1bfc1616f0 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.678+0000 7f1c032e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc0cfa00 con 0x7f1bfc0ce0a0 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.678+0000 7f1c032e1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc0cf690 con 0x7f1bfc1610f0 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.678+0000 7f1c0187e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bfc1616f0 0x7f1bfc15cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.678+0000 7f1c0187e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bfc1616f0 0x7f1bfc15cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37952/0 (socket says 172.21.15.19:37952) 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.678+0000 7f1c0187e640 1 -- 172.21.15.19:0/1801442384 learned_addr learned my addr 172.21.15.19:0/1801442384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1c0107d640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc0ce0a0 0x7f1bfc0b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1c0087c640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bfc1610f0 0x7f1bfc14b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1c0187e640 1 -- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bfc1610f0 msgr2=0x7f1bfc14b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1c0187e640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bfc1610f0 0x7f1bfc14b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1c0187e640 1 -- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc0ce0a0 msgr2=0x7f1bfc0b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1c0187e640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc0ce0a0 0x7f1bfc0b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1c0187e640 1 -- 172.21.15.19:0/1801442384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc0b4a30 con 0x7f1bfc1616f0 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1c0107d640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc0ce0a0 0x7f1bfc0b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1c0187e640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bfc1616f0 0x7f1bfc15cb30 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400c7a0 tx=0x7f1bf400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1bf1ffb640 1 -- 172.21.15.19:0/1801442384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4003b70 con 0x7f1bfc1616f0 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1bf1ffb640 1 -- 172.21.15.19:0/1801442384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf4003d30 con 0x7f1bfc1616f0 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1c032e1640 1 -- 172.21.15.19:0/1801442384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bfc15db20 con 0x7f1bfc1616f0 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1bf1ffb640 1 -- 172.21.15.19:0/1801442384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf400aa90 con 0x7f1bfc1616f0 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.679+0000 7f1c032e1640 1 -- 172.21.15.19:0/1801442384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc15e0e0 con 0x7f1bfc1616f0 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.680+0000 7f1c032e1640 1 -- 172.21.15.19:0/1801442384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bfc0b4c00 con 0x7f1bfc1616f0 2024-04-25T10:04:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.681+0000 7f1bf1ffb640 1 -- 172.21.15.19:0/1801442384 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bf400ac30 con 0x7f1bfc1616f0 2024-04-25T10:04:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.681+0000 7f1bf1ffb640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be006b4d0 0x7f1be006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.681+0000 7f1bf1ffb640 1 -- 172.21.15.19:0/1801442384 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1bf4089f90 con 0x7f1bfc1616f0 2024-04-25T10:04:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.681+0000 7f1c0107d640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be006b4d0 0x7f1be006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.682+0000 7f1c0107d640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be006b4d0 0x7f1be006d990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80011e0 tx=0x7f1bf8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.685+0000 7f1bf1ffb640 1 -- 172.21.15.19:0/1801442384 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1bf4058fd0 con 0x7f1bfc1616f0 2024-04-25T10:04:11.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.868+0000 7f1c032e1640 1 -- 172.21.15.19:0/1801442384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1bfc15ae40 con 0x7f1bfc1616f0 2024-04-25T10:04:11.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.868+0000 7f1bf1ffb640 1 -- 172.21.15.19:0/1801442384 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v161) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1bf40588d0 con 0x7f1bfc1616f0 2024-04-25T10:04:11.874 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:04:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 -- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be006b4d0 msgr2=0x7f1be006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be006b4d0 0x7f1be006d990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80011e0 tx=0x7f1bf8009040 comp rx=0 tx=0).stop 2024-04-25T10:04:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 -- 172.21.15.19:0/1801442384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bfc1616f0 msgr2=0x7f1bfc15cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bfc1616f0 0x7f1bfc15cb30 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400c7a0 tx=0x7f1bf400cc60 comp rx=0 tx=0).stop 2024-04-25T10:04:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 -- 172.21.15.19:0/1801442384 shutdown_connections 2024-04-25T10:04:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1be006b4d0 0x7f1be006d990 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bfc1616f0 0x7f1bfc15cb30 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bfc1610f0 0x7f1bfc14b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 --2- 172.21.15.19:0/1801442384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bfc0ce0a0 0x7f1bfc0b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 -- 172.21.15.19:0/1801442384 >> 172.21.15.19:0/1801442384 conn(0x7f1bfc06f500 msgr2=0x7f1bfc0d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 -- 172.21.15.19:0/1801442384 shutdown_connections 2024-04-25T10:04:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.871+0000 7f1c032e1640 1 -- 172.21.15.19:0/1801442384 wait complete. 2024-04-25T10:04:11.887 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:04:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.982+0000 7fb81443f640 1 Processor -- start 2024-04-25T10:04:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.983+0000 7fb81443f640 1 -- start start 2024-04-25T10:04:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.983+0000 7fb81443f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb80c154ee0 0x7fb80c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.983+0000 7fb81443f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb80c0ce590 0x7fb80c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb81443f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80c071480 con 0x7fb80c0cfcf0 2024-04-25T10:04:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb81443f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80c072090 con 0x7fb80c154ee0 2024-04-25T10:04:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb81443f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c071230 con 0x7fb80c0ce590 2024-04-25T10:04:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb8121db640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb80c154ee0 0x7fb80c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb8129dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb80c0ce590 0x7fb80c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb8121db640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb80c154ee0 0x7fb80c0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:36970/0 (socket says 172.21.15.19:36970) 2024-04-25T10:04:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb8121db640 1 -- 172.21.15.19:0/3381692162 learned_addr learned my addr 172.21.15.19:0/3381692162 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb8119da640 1 --1- 172.21.15.19:0/3381692162 >> v1:172.21.15.112:6789/0 conn(0x7fb80c0cfcf0 0x7fb80c0cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:47082/0 (socket says 172.21.15.19:47082) 2024-04-25T10:04:11.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb8129dc640 1 -- 172.21.15.19:0/3381692162 >> v2:172.21.15.112:3300/0 conn(0x7fb80c154ee0 msgr2=0x7fb80c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:11.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb8129dc640 1 --2- 172.21.15.19:0/3381692162 >> v2:172.21.15.112:3300/0 conn(0x7fb80c154ee0 0x7fb80c0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb8129dc640 1 -- 172.21.15.19:0/3381692162 >> v1:172.21.15.112:6789/0 conn(0x7fb80c0cfcf0 legacy=0x7fb80c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb8129dc640 1 -- 172.21.15.19:0/3381692162 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c071760 con 0x7fb80c0ce590 2024-04-25T10:04:11.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.984+0000 7fb8121db640 1 --2- 172.21.15.19:0/3381692162 >> v2:172.21.15.112:3300/0 conn(0x7fb80c154ee0 0x7fb80c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:11.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.985+0000 7fb8129dc640 1 --2- 172.21.15.19:0/3381692162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb80c0ce590 0x7fb80c06d6f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb804009d90 tx=0x7fb80402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f38fd7f103194fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.985+0000 7fb8111d9640 1 -- 172.21.15.19:0/3381692162 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804007740 con 0x7fb80c0ce590 2024-04-25T10:04:11.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.985+0000 7fb8111d9640 1 -- 172.21.15.19:0/3381692162 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8040078e0 con 0x7fb80c0ce590 2024-04-25T10:04:11.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.985+0000 7fb8111d9640 1 -- 172.21.15.19:0/3381692162 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804007cb0 con 0x7fb80c0ce590 2024-04-25T10:04:11.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.986+0000 7fb81443f640 1 -- 172.21.15.19:0/3381692162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb80c0ce590 msgr2=0x7fb80c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.986+0000 7fb81443f640 1 --2- 172.21.15.19:0/3381692162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb80c0ce590 0x7fb80c06d6f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb804009d90 tx=0x7fb80402a660 comp rx=0 tx=0).stop 2024-04-25T10:04:11.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.986+0000 7fb81443f640 1 -- 172.21.15.19:0/3381692162 shutdown_connections 2024-04-25T10:04:11.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.986+0000 7fb81443f640 1 --2- 172.21.15.19:0/3381692162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb80c0ce590 0x7fb80c06d6f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.986+0000 7fb81443f640 1 --2- 172.21.15.19:0/3381692162 >> v2:172.21.15.112:3300/0 conn(0x7fb80c154ee0 0x7fb80c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.986+0000 7fb81443f640 1 -- 172.21.15.19:0/3381692162 >> 172.21.15.19:0/3381692162 conn(0x7fb80c06abe0 msgr2=0x7fb80c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.986+0000 7fb81443f640 1 -- 172.21.15.19:0/3381692162 shutdown_connections 2024-04-25T10:04:11.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.986+0000 7fb81443f640 1 -- 172.21.15.19:0/3381692162 wait complete. 2024-04-25T10:04:11.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.987+0000 7fb81443f640 1 Processor -- start 2024-04-25T10:04:11.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.987+0000 7fb81443f640 1 -- start start 2024-04-25T10:04:11.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb81443f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb80c0ce590 0x7fb80c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb81443f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb80c0cfcf0 0x7fb80c1f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb81443f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb80c154ee0 0x7fb80c0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb81443f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c1552c0 con 0x7fb80c0cfcf0 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb81443f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c071d20 con 0x7fb80c0ce590 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb81443f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c155440 con 0x7fb80c154ee0 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb8119da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb80c0cfcf0 0x7fb80c1f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb8119da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb80c0cfcf0 0x7fb80c1f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37956/0 (socket says 172.21.15.19:37956) 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb8119da640 1 -- 172.21.15.19:0/2788182545 learned_addr learned my addr 172.21.15.19:0/2788182545 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb8129dc640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb80c154ee0 0x7fb80c0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb8121db640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb80c0ce590 0x7fb80c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb8119da640 1 -- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb80c154ee0 msgr2=0x7fb80c0cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb8119da640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb80c154ee0 0x7fb80c0cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb8119da640 1 -- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb80c0ce590 msgr2=0x7fb80c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb8119da640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb80c0ce590 0x7fb80c0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.988+0000 7fb8119da640 1 -- 172.21.15.19:0/2788182545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c1f4870 con 0x7fb80c0cfcf0 2024-04-25T10:04:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.989+0000 7fb8119da640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb80c0cfcf0 0x7fb80c1f4330 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00c430 tx=0x7fb7fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.989+0000 7fb802ffd640 1 -- 172.21.15.19:0/2788182545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0074e0 con 0x7fb80c0cfcf0 2024-04-25T10:04:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.989+0000 7fb802ffd640 1 -- 172.21.15.19:0/2788182545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc007680 con 0x7fb80c0cfcf0 2024-04-25T10:04:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.989+0000 7fb802ffd640 1 -- 172.21.15.19:0/2788182545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc004a00 con 0x7fb80c0cfcf0 2024-04-25T10:04:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.989+0000 7fb81443f640 1 -- 172.21.15.19:0/2788182545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c0ace90 con 0x7fb80c0cfcf0 2024-04-25T10:04:11.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.989+0000 7fb81443f640 1 -- 172.21.15.19:0/2788182545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c0ad450 con 0x7fb80c0cfcf0 2024-04-25T10:04:11.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.990+0000 7fb81443f640 1 -- 172.21.15.19:0/2788182545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80c073990 con 0x7fb80c0cfcf0 2024-04-25T10:04:11.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.990+0000 7fb802ffd640 1 -- 172.21.15.19:0/2788182545 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7fc01a020 con 0x7fb80c0cfcf0 2024-04-25T10:04:11.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.991+0000 7fb802ffd640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7f006b500 0x7fb7f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.991+0000 7fb802ffd640 1 -- 172.21.15.19:0/2788182545 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb7fc0896f0 con 0x7fb80c0cfcf0 2024-04-25T10:04:11.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.991+0000 7fb8121db640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7f006b500 0x7fb7f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.992+0000 7fb8121db640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7f006b500 0x7fb7f006d9c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb808006ca0 tx=0x7fb808009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:11.995+0000 7fb802ffd640 1 -- 172.21.15.19:0/2788182545 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb80c1f4870 con 0x7fb80c0cfcf0 2024-04-25T10:04:12.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.175+0000 7fb81443f640 1 -- 172.21.15.19:0/2788182545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb80c0cf6c0 con 0x7fb80c0cfcf0 2024-04-25T10:04:12.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.175+0000 7fb802ffd640 1 -- 172.21.15.19:0/2788182545 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v161) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb7fc0580e0 con 0x7fb80c0cfcf0 2024-04-25T10:04:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:12.180 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T10:04:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T10:04:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T10:04:12.180 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:12.181 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T10:04:12.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 -- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7f006b500 msgr2=0x7fb7f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7f006b500 0x7fb7f006d9c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb808006ca0 tx=0x7fb808009240 comp rx=0 tx=0).stop 2024-04-25T10:04:12.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 -- 172.21.15.19:0/2788182545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb80c0cfcf0 msgr2=0x7fb80c1f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb80c0cfcf0 0x7fb80c1f4330 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00c430 tx=0x7fb7fc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 -- 172.21.15.19:0/2788182545 shutdown_connections 2024-04-25T10:04:12.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb7f006b500 0x7fb7f006d9c0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb80c154ee0 0x7fb80c0cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb80c0cfcf0 0x7fb80c1f4330 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 --2- 172.21.15.19:0/2788182545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb80c0ce590 0x7fb80c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 -- 172.21.15.19:0/2788182545 >> 172.21.15.19:0/2788182545 conn(0x7fb80c06abe0 msgr2=0x7fb80c15c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:12.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 -- 172.21.15.19:0/2788182545 shutdown_connections 2024-04-25T10:04:12.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.177+0000 7fb81443f640 1 -- 172.21.15.19:0/2788182545 wait complete. 2024-04-25T10:04:12.193 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:04:12.193 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:04:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.292+0000 7f83baab6640 1 Processor -- start 2024-04-25T10:04:12.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.292+0000 7f83baab6640 1 -- start start 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.293+0000 7f83baab6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f83b4161710 0x7f83b40d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.293+0000 7f83baab6640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b415e000 con 0x7f83b415dc20 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.293+0000 7f83baab6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b40d1de0 con 0x7f83b40ced80 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.293+0000 7f83baab6640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b4161c60 con 0x7f83b4161710 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.293+0000 7f83b9053640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f83b4161710 0x7f83b40d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.293+0000 7f83b8852640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f83b415dc20 0x7f83b40ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41360/0 (socket says 172.21.15.19:41360) 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.293+0000 7f83b8852640 1 -- 172.21.15.19:0/1952107979 learned_addr learned my addr 172.21.15.19:0/1952107979 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.294+0000 7f83b9053640 1 -- 172.21.15.19:0/1952107979 >> v1:172.21.15.112:6789/0 conn(0x7f83b40ced80 legacy=0x7f83b40d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.294+0000 7f83b9053640 1 -- 172.21.15.19:0/1952107979 >> v1:172.21.15.19:6789/0 conn(0x7f83b415dc20 legacy=0x7f83b40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.294+0000 7f83b9053640 1 -- 172.21.15.19:0/1952107979 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b4161f40 con 0x7f83b4161710 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.294+0000 7f83b37fe640 1 -- 172.21.15.19:0/1952107979 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994971915 0 0) 0x7f83b415e000 con 0x7f83b415dc20 2024-04-25T10:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.294+0000 7f83b9053640 1 --2- 172.21.15.19:0/1952107979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83b4161710 0x7f83b40d29f0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009d90 tx=0x7f83ac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75590fcdfed6bc29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.295+0000 7f83b37fe640 1 -- 172.21.15.19:0/1952107979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac007740 con 0x7f83b4161710 2024-04-25T10:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.295+0000 7f83b37fe640 1 -- 172.21.15.19:0/1952107979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ac0078e0 con 0x7f83b4161710 2024-04-25T10:04:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.295+0000 7f83b37fe640 1 -- 172.21.15.19:0/1952107979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac007cb0 con 0x7f83b4161710 2024-04-25T10:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.295+0000 7f83baab6640 1 -- 172.21.15.19:0/1952107979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83b4161710 msgr2=0x7f83b40d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.295+0000 7f83baab6640 1 --2- 172.21.15.19:0/1952107979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83b4161710 0x7f83b40d29f0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009d90 tx=0x7f83ac02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.295+0000 7f83baab6640 1 -- 172.21.15.19:0/1952107979 shutdown_connections 2024-04-25T10:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.295+0000 7f83baab6640 1 --2- 172.21.15.19:0/1952107979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83b4161710 0x7f83b40d29f0 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.295+0000 7f83baab6640 1 -- 172.21.15.19:0/1952107979 >> 172.21.15.19:0/1952107979 conn(0x7f83b40d0420 msgr2=0x7f83b40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.295+0000 7f83baab6640 1 -- 172.21.15.19:0/1952107979 shutdown_connections 2024-04-25T10:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.295+0000 7f83baab6640 1 -- 172.21.15.19:0/1952107979 wait complete. 2024-04-25T10:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.296+0000 7f83baab6640 1 Processor -- start 2024-04-25T10:04:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.296+0000 7f83baab6640 1 -- start start 2024-04-25T10:04:12.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.297+0000 7f83baab6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83b40ced80 0x7f83b415f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.297+0000 7f83baab6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83b415dc20 0x7f83b40d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.297+0000 7f83baab6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83b4161710 0x7f83b4154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.297+0000 7f83baab6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40b5e50 con 0x7f83b40ced80 2024-04-25T10:04:12.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.297+0000 7f83baab6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b4162500 con 0x7f83b415dc20 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.297+0000 7f83baab6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40b5fd0 con 0x7f83b4161710 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.297+0000 7f83b8852640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83b40ced80 0x7f83b415f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.297+0000 7f83b8852640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83b40ced80 0x7f83b415f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37980/0 (socket says 172.21.15.19:37980) 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.297+0000 7f83b8852640 1 -- 172.21.15.19:0/257980676 learned_addr learned my addr 172.21.15.19:0/257980676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83b3fff640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83b415dc20 0x7f83b40d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83b9053640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83b4161710 0x7f83b4154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83b8852640 1 -- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83b4161710 msgr2=0x7f83b4154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83b8852640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83b4161710 0x7f83b4154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83b8852640 1 -- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83b415dc20 msgr2=0x7f83b40d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83b8852640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83b415dc20 0x7f83b40d2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83b8852640 1 -- 172.21.15.19:0/257980676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b41f3560 con 0x7f83b40ced80 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83b8852640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83b40ced80 0x7f83b415f930 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f83a400c4a0 tx=0x7f83a400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83b17fa640 1 -- 172.21.15.19:0/257980676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4018070 con 0x7f83b40ced80 2024-04-25T10:04:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83b17fa640 1 -- 172.21.15.19:0/257980676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a40074e0 con 0x7f83b40ced80 2024-04-25T10:04:12.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83b17fa640 1 -- 172.21.15.19:0/257980676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4004a20 con 0x7f83b40ced80 2024-04-25T10:04:12.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83baab6640 1 -- 172.21.15.19:0/257980676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b41f3820 con 0x7f83b40ced80 2024-04-25T10:04:12.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.298+0000 7f83baab6640 1 -- 172.21.15.19:0/257980676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b41f3de0 con 0x7f83b40ced80 2024-04-25T10:04:12.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.299+0000 7f83baab6640 1 -- 172.21.15.19:0/257980676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b4073a00 con 0x7f83b40ced80 2024-04-25T10:04:12.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.300+0000 7f83b17fa640 1 -- 172.21.15.19:0/257980676 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83a4004bc0 con 0x7f83b40ced80 2024-04-25T10:04:12.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.300+0000 7f83b17fa640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83a006b500 0x7f83a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.300+0000 7f83b17fa640 1 -- 172.21.15.19:0/257980676 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f83a40892f0 con 0x7f83b40ced80 2024-04-25T10:04:12.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.300+0000 7f83b3fff640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83a006b500 0x7f83a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.301+0000 7f83b3fff640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83a006b500 0x7f83a006d9c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f83b40c6e90 tx=0x7f83a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.304+0000 7f83b17fa640 1 -- 172.21.15.19:0/257980676 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83a4058330 con 0x7f83b40ced80 2024-04-25T10:04:12.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.486+0000 7f83baab6640 1 -- 172.21.15.19:0/257980676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f83b40d16b0 con 0x7f83b40ced80 2024-04-25T10:04:12.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.877+0000 7f83b17fa640 1 -- 172.21.15.19:0/257980676 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v162) v1 ==== 138+0+0 (secure 0 0 0) 0x7f83a4057c30 con 0x7f83b40ced80 2024-04-25T10:04:12.883 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T10:04:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 -- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83a006b500 msgr2=0x7f83a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83a006b500 0x7f83a006d9c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f83b40c6e90 tx=0x7f83a8009040 comp rx=0 tx=0).stop 2024-04-25T10:04:12.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 -- 172.21.15.19:0/257980676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83b40ced80 msgr2=0x7f83b415f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83b40ced80 0x7f83b415f930 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f83a400c4a0 tx=0x7f83a400c960 comp rx=0 tx=0).stop 2024-04-25T10:04:12.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 -- 172.21.15.19:0/257980676 shutdown_connections 2024-04-25T10:04:12.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f83a006b500 0x7f83a006d9c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83b4161710 0x7f83b4154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83b415dc20 0x7f83b40d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 --2- 172.21.15.19:0/257980676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83b40ced80 0x7f83b415f930 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 -- 172.21.15.19:0/257980676 >> 172.21.15.19:0/257980676 conn(0x7f83b40d0420 msgr2=0x7f83b40d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:12.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 -- 172.21.15.19:0/257980676 shutdown_connections 2024-04-25T10:04:12.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:12.880+0000 7f83baab6640 1 -- 172.21.15.19:0/257980676 wait complete. 2024-04-25T10:04:12.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:04:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.005+0000 7f2f13ff9640 1 Processor -- start 2024-04-25T10:04:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.005+0000 7f2f13ff9640 1 -- start start 2024-04-25T10:04:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.006+0000 7f2f13ff9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f0c0ced80 0x7f2f0c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.006+0000 7f2f13ff9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f0c161710 0x7f2f0c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.006+0000 7f2f13ff9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f0c15e000 con 0x7f2f0c15dc20 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.006+0000 7f2f13ff9640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0c0d1de0 con 0x7f2f0c161710 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.006+0000 7f2f13ff9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c161c60 con 0x7f2f0c0ced80 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.006+0000 7f2f12596640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f0c161710 0x7f2f0c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.006+0000 7f2f12596640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f0c161710 0x7f2f0c0d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37986/0 (socket says 172.21.15.19:37986) 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.006+0000 7f2f12596640 1 -- 172.21.15.19:0/4027110221 learned_addr learned my addr 172.21.15.19:0/4027110221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.006+0000 7f2f11594640 1 --2- 172.21.15.19:0/4027110221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f0c0ced80 0x7f2f0c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.007+0000 7f2f12596640 1 -- 172.21.15.19:0/4027110221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f0c0ced80 msgr2=0x7f2f0c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.007+0000 7f2f12596640 1 --2- 172.21.15.19:0/4027110221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f0c0ced80 0x7f2f0c0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.007+0000 7f2f12596640 1 -- 172.21.15.19:0/4027110221 >> v1:172.21.15.112:6789/0 conn(0x7f2f0c15dc20 legacy=0x7f2f0c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.007+0000 7f2f12596640 1 -- 172.21.15.19:0/4027110221 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c161f40 con 0x7f2f0c161710 2024-04-25T10:04:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.007+0000 7f2f12596640 1 --2- 172.21.15.19:0/4027110221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0c161710 0x7f2f0c0d2820 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009870 tx=0x7f2f0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436b1dcecd3894f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.007+0000 7f2f10d93640 1 -- 172.21.15.19:0/4027110221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04037070 con 0x7f2f0c161710 2024-04-25T10:04:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.007+0000 7f2f10d93640 1 -- 172.21.15.19:0/4027110221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f04032440 con 0x7f2f0c161710 2024-04-25T10:04:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.007+0000 7f2f10d93640 1 -- 172.21.15.19:0/4027110221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04032810 con 0x7f2f0c161710 2024-04-25T10:04:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.008+0000 7f2f13ff9640 1 -- 172.21.15.19:0/4027110221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0c161710 msgr2=0x7f2f0c0d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.008+0000 7f2f13ff9640 1 --2- 172.21.15.19:0/4027110221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0c161710 0x7f2f0c0d2820 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009870 tx=0x7f2f0402a270 comp rx=0 tx=0).stop 2024-04-25T10:04:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.008+0000 7f2f13ff9640 1 -- 172.21.15.19:0/4027110221 shutdown_connections 2024-04-25T10:04:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.008+0000 7f2f13ff9640 1 --2- 172.21.15.19:0/4027110221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0c161710 0x7f2f0c0d2820 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.008+0000 7f2f13ff9640 1 --2- 172.21.15.19:0/4027110221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f0c0ced80 0x7f2f0c0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.008+0000 7f2f13ff9640 1 -- 172.21.15.19:0/4027110221 >> 172.21.15.19:0/4027110221 conn(0x7f2f0c0d0420 msgr2=0x7f2f0c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.008+0000 7f2f13ff9640 1 -- 172.21.15.19:0/4027110221 shutdown_connections 2024-04-25T10:04:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.008+0000 7f2f13ff9640 1 -- 172.21.15.19:0/4027110221 wait complete. 2024-04-25T10:04:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.009+0000 7f2f13ff9640 1 Processor -- start 2024-04-25T10:04:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.009+0000 7f2f13ff9640 1 -- start start 2024-04-25T10:04:13.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.010+0000 7f2f13ff9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c0ced80 0x7f2f0c15ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.010+0000 7f2f13ff9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0c15dc20 0x7f2f0c15c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.010+0000 7f2f13ff9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f0c161710 0x7f2f0c0d3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.010+0000 7f2f13ff9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c0b5e50 con 0x7f2f0c15dc20 2024-04-25T10:04:13.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.010+0000 7f2f13ff9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c162500 con 0x7f2f0c0ced80 2024-04-25T10:04:13.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.010+0000 7f2f13ff9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c0b5fd0 con 0x7f2f0c161710 2024-04-25T10:04:13.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.010+0000 7f2f11594640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0c15dc20 0x7f2f0c15c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.010+0000 7f2f11594640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0c15dc20 0x7f2f0c15c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37992/0 (socket says 172.21.15.19:37992) 2024-04-25T10:04:13.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.010+0000 7f2f11594640 1 -- 172.21.15.19:0/1689641214 learned_addr learned my addr 172.21.15.19:0/1689641214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.010+0000 7f2f11d95640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c0ced80 0x7f2f0c15ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.010+0000 7f2f12596640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f0c161710 0x7f2f0c0d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.011+0000 7f2f11594640 1 -- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f0c161710 msgr2=0x7f2f0c0d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.011+0000 7f2f11594640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f0c161710 0x7f2f0c0d3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.011+0000 7f2f11594640 1 -- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c0ced80 msgr2=0x7f2f0c15ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.011+0000 7f2f11594640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c0ced80 0x7f2f0c15ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.011+0000 7f2f11594640 1 -- 172.21.15.19:0/1689641214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c1e9270 con 0x7f2f0c15dc20 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.011+0000 7f2f11594640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0c15dc20 0x7f2f0c15c410 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000d620 tx=0x7f2f0000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.011+0000 7f2f0a7fc640 1 -- 172.21.15.19:0/1689641214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000b620 con 0x7f2f0c15dc20 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.011+0000 7f2f0a7fc640 1 -- 172.21.15.19:0/1689641214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f00019070 con 0x7f2f0c15dc20 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.011+0000 7f2f13ff9640 1 -- 172.21.15.19:0/1689641214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0c15e420 con 0x7f2f0c15dc20 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.011+0000 7f2f0a7fc640 1 -- 172.21.15.19:0/1689641214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000ba00 con 0x7f2f0c15dc20 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.011+0000 7f2f13ff9640 1 -- 172.21.15.19:0/1689641214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0c15e9e0 con 0x7f2f0c15dc20 2024-04-25T10:04:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.012+0000 7f2f13ff9640 1 -- 172.21.15.19:0/1689641214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0c15e230 con 0x7f2f0c15dc20 2024-04-25T10:04:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.013+0000 7f2f0a7fc640 1 -- 172.21.15.19:0/1689641214 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f0000bba0 con 0x7f2f0c15dc20 2024-04-25T10:04:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.013+0000 7f2f0a7fc640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef806b4d0 0x7f2ef806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.013+0000 7f2f0a7fc640 1 -- 172.21.15.19:0/1689641214 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f2f0008a3e0 con 0x7f2f0c15dc20 2024-04-25T10:04:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.013+0000 7f2f11d95640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef806b4d0 0x7f2ef806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.014+0000 7f2f11d95640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef806b4d0 0x7f2ef806d990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2efc001e70 tx=0x7f2efc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.018+0000 7f2f0a7fc640 1 -- 172.21.15.19:0/1689641214 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f00058cf0 con 0x7f2f0c15dc20 2024-04-25T10:04:13.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.200+0000 7f2f13ff9640 1 -- 172.21.15.19:0/1689641214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2f0c073a00 con 0x7f2f0c15dc20 2024-04-25T10:04:13.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.883+0000 7f2f0a7fc640 1 -- 172.21.15.19:0/1689641214 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v163) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2f0005cd90 con 0x7f2f0c15dc20 2024-04-25T10:04:13.889 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T10:04:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.885+0000 7f2f13ff9640 1 -- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef806b4d0 msgr2=0x7f2ef806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.885+0000 7f2f13ff9640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef806b4d0 0x7f2ef806d990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2efc001e70 tx=0x7f2efc009040 comp rx=0 tx=0).stop 2024-04-25T10:04:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.885+0000 7f2f13ff9640 1 -- 172.21.15.19:0/1689641214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0c15dc20 msgr2=0x7f2f0c15c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.885+0000 7f2f13ff9640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0c15dc20 0x7f2f0c15c410 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000d620 tx=0x7f2f0000dae0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.886+0000 7f2f13ff9640 1 -- 172.21.15.19:0/1689641214 shutdown_connections 2024-04-25T10:04:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.886+0000 7f2f13ff9640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ef806b4d0 0x7f2ef806d990 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.886+0000 7f2f13ff9640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f0c161710 0x7f2f0c0d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.886+0000 7f2f13ff9640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f0c15dc20 0x7f2f0c15c410 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.886+0000 7f2f13ff9640 1 --2- 172.21.15.19:0/1689641214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f0c0ced80 0x7f2f0c15ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.886+0000 7f2f13ff9640 1 -- 172.21.15.19:0/1689641214 >> 172.21.15.19:0/1689641214 conn(0x7f2f0c0d0420 msgr2=0x7f2f0c0cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.886+0000 7f2f13ff9640 1 -- 172.21.15.19:0/1689641214 shutdown_connections 2024-04-25T10:04:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:13.886+0000 7f2f13ff9640 1 -- 172.21.15.19:0/1689641214 wait complete. 2024-04-25T10:04:13.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:04:14.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.019+0000 7f8cc7053640 1 Processor -- start 2024-04-25T10:04:14.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.019+0000 7f8cc7053640 1 -- start start 2024-04-25T10:04:14.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.020+0000 7f8cc7053640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8cc015dc20 0x7f8cc00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.020+0000 7f8cc7053640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cc00d32f0 0x7f8cc00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.020+0000 7f8cc7053640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc015e000 con 0x7f8cc0161710 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.020+0000 7f8cc7053640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc00d1de0 con 0x7f8cc00d32f0 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.020+0000 7f8cc7053640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc0161c60 con 0x7f8cc015dc20 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.020+0000 7f8cc55f0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cc00d32f0 0x7f8cc00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.020+0000 7f8cc55f0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cc00d32f0 0x7f8cc00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38000/0 (socket says 172.21.15.19:38000) 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.020+0000 7f8cc55f0640 1 -- 172.21.15.19:0/3272310964 learned_addr learned my addr 172.21.15.19:0/3272310964 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.020+0000 7f8cc4def640 1 --2- 172.21.15.19:0/3272310964 >> v2:172.21.15.112:3300/0 conn(0x7f8cc015dc20 0x7f8cc00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.021+0000 7f8cc55f0640 1 -- 172.21.15.19:0/3272310964 >> v2:172.21.15.112:3300/0 conn(0x7f8cc015dc20 msgr2=0x7f8cc00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.021+0000 7f8cc55f0640 1 --2- 172.21.15.19:0/3272310964 >> v2:172.21.15.112:3300/0 conn(0x7f8cc015dc20 0x7f8cc00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.021+0000 7f8cc55f0640 1 -- 172.21.15.19:0/3272310964 >> v1:172.21.15.112:6789/0 conn(0x7f8cc0161710 legacy=0x7f8cc006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.021+0000 7f8cc55f0640 1 -- 172.21.15.19:0/3272310964 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0161f40 con 0x7f8cc00d32f0 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.021+0000 7f8cc4def640 1 --2- 172.21.15.19:0/3272310964 >> v2:172.21.15.112:3300/0 conn(0x7f8cc015dc20 0x7f8cc00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.021+0000 7f8cc55f0640 1 --2- 172.21.15.19:0/3272310964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc00d32f0 0x7f8cc00cebb0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00d1520 tx=0x7f8cb802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253c419918fceab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.022+0000 7f8cbf7fe640 1 -- 172.21.15.19:0/3272310964 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8036070 con 0x7f8cc00d32f0 2024-04-25T10:04:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.022+0000 7f8cbf7fe640 1 -- 172.21.15.19:0/3272310964 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb8031600 con 0x7f8cc00d32f0 2024-04-25T10:04:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.022+0000 7f8cbf7fe640 1 -- 172.21.15.19:0/3272310964 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb80319d0 con 0x7f8cc00d32f0 2024-04-25T10:04:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.022+0000 7f8cc7053640 1 -- 172.21.15.19:0/3272310964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc00d32f0 msgr2=0x7f8cc00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.022+0000 7f8cc7053640 1 --2- 172.21.15.19:0/3272310964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc00d32f0 0x7f8cc00cebb0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00d1520 tx=0x7f8cb802a730 comp rx=0 tx=0).stop 2024-04-25T10:04:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.022+0000 7f8cc7053640 1 -- 172.21.15.19:0/3272310964 shutdown_connections 2024-04-25T10:04:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.022+0000 7f8cc7053640 1 --2- 172.21.15.19:0/3272310964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc00d32f0 0x7f8cc00cebb0 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.022+0000 7f8cc7053640 1 --2- 172.21.15.19:0/3272310964 >> v2:172.21.15.112:3300/0 conn(0x7f8cc015dc20 0x7f8cc00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.022+0000 7f8cc7053640 1 -- 172.21.15.19:0/3272310964 >> 172.21.15.19:0/3272310964 conn(0x7f8cc00d0420 msgr2=0x7f8cc00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.022+0000 7f8cc7053640 1 -- 172.21.15.19:0/3272310964 shutdown_connections 2024-04-25T10:04:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.022+0000 7f8cc7053640 1 -- 172.21.15.19:0/3272310964 wait complete. 2024-04-25T10:04:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.023+0000 7f8cc7053640 1 Processor -- start 2024-04-25T10:04:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.023+0000 7f8cc7053640 1 -- start start 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.024+0000 7f8cc7053640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cc00d32f0 0x7f8cc00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.024+0000 7f8cc7053640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc015dc20 0x7f8cc01f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.024+0000 7f8cc7053640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc0161710 0x7f8cc01f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.024+0000 7f8cc7053640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00b5e50 con 0x7f8cc015dc20 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.024+0000 7f8cc7053640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0162500 con 0x7f8cc0161710 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.024+0000 7f8cc7053640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00b5fd0 con 0x7f8cc00d32f0 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.024+0000 7f8cbffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc015dc20 0x7f8cc01f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.024+0000 7f8cbffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc015dc20 0x7f8cc01f6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38006/0 (socket says 172.21.15.19:38006) 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.024+0000 7f8cbffff640 1 -- 172.21.15.19:0/4217445791 learned_addr learned my addr 172.21.15.19:0/4217445791 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cc4def640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cc00d32f0 0x7f8cc00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cc55f0640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc0161710 0x7f8cc01f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cbffff640 1 -- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cc00d32f0 msgr2=0x7f8cc00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cbffff640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cc00d32f0 0x7f8cc00cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cbffff640 1 -- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc0161710 msgr2=0x7f8cc01f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cbffff640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc0161710 0x7f8cc01f6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cbffff640 1 -- 172.21.15.19:0/4217445791 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc015f000 con 0x7f8cc015dc20 2024-04-25T10:04:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cc55f0640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc0161710 0x7f8cc01f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cc4def640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cc00d32f0 0x7f8cc00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cbffff640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc015dc20 0x7f8cc01f6540 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400c3c0 tx=0x7f8cb400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cbd7fa640 1 -- 172.21.15.19:0/4217445791 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb40052d0 con 0x7f8cc015dc20 2024-04-25T10:04:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cbd7fa640 1 -- 172.21.15.19:0/4217445791 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb4005490 con 0x7f8cc015dc20 2024-04-25T10:04:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cc7053640 1 -- 172.21.15.19:0/4217445791 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc015f2e0 con 0x7f8cc015dc20 2024-04-25T10:04:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cbd7fa640 1 -- 172.21.15.19:0/4217445791 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4007260 con 0x7f8cc015dc20 2024-04-25T10:04:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.025+0000 7f8cc7053640 1 -- 172.21.15.19:0/4217445791 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc015f8a0 con 0x7f8cc015dc20 2024-04-25T10:04:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.026+0000 7f8cc7053640 1 -- 172.21.15.19:0/4217445791 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc0073a00 con 0x7f8cc015dc20 2024-04-25T10:04:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.027+0000 7f8cbd7fa640 1 -- 172.21.15.19:0/4217445791 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cb4007400 con 0x7f8cc015dc20 2024-04-25T10:04:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.027+0000 7f8cbd7fa640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cac06b4d0 0x7f8cac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.027+0000 7f8cbd7fa640 1 -- 172.21.15.19:0/4217445791 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f8cb4089820 con 0x7f8cc015dc20 2024-04-25T10:04:14.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.028+0000 7f8cc4def640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cac06b4d0 0x7f8cac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.028+0000 7f8cc4def640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cac06b4d0 0x7f8cac06d990 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00d1610 tx=0x7f8cb0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.032+0000 7f8cbd7fa640 1 -- 172.21.15.19:0/4217445791 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8cb4058560 con 0x7f8cc015dc20 2024-04-25T10:04:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.249+0000 7f8cc7053640 1 -- 172.21.15.19:0/4217445791 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8cc00d16b0 con 0x7f8cc015dc20 2024-04-25T10:04:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.885+0000 7f8cbd7fa640 1 -- 172.21.15.19:0/4217445791 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v107) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8cb4057e60 con 0x7f8cc015dc20 2024-04-25T10:04:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 25 and data pool 26 2024-04-25T10:04:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.888+0000 7f8cc7053640 1 -- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cac06b4d0 msgr2=0x7f8cac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.888+0000 7f8cc7053640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cac06b4d0 0x7f8cac06d990 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00d1610 tx=0x7f8cb0008040 comp rx=0 tx=0).stop 2024-04-25T10:04:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.888+0000 7f8cc7053640 1 -- 172.21.15.19:0/4217445791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc015dc20 msgr2=0x7f8cc01f6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.888+0000 7f8cc7053640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc015dc20 0x7f8cc01f6540 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400c3c0 tx=0x7f8cb400c880 comp rx=0 tx=0).stop 2024-04-25T10:04:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.888+0000 7f8cc7053640 1 -- 172.21.15.19:0/4217445791 shutdown_connections 2024-04-25T10:04:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.888+0000 7f8cc7053640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8cac06b4d0 0x7f8cac06d990 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.888+0000 7f8cc7053640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc0161710 0x7f8cc01f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.888+0000 7f8cc7053640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc015dc20 0x7f8cc01f6540 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.888+0000 7f8cc7053640 1 --2- 172.21.15.19:0/4217445791 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cc00d32f0 0x7f8cc00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.888+0000 7f8cc7053640 1 -- 172.21.15.19:0/4217445791 >> 172.21.15.19:0/4217445791 conn(0x7f8cc00d0420 msgr2=0x7f8cc00d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.888+0000 7f8cc7053640 1 -- 172.21.15.19:0/4217445791 shutdown_connections 2024-04-25T10:04:14.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:14.889+0000 7f8cc7053640 1 -- 172.21.15.19:0/4217445791 wait complete. 2024-04-25T10:04:14.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.025+0000 7f1689734640 1 Processor -- start 2024-04-25T10:04:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.025+0000 7f1689734640 1 -- start start 2024-04-25T10:04:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f1689734640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f16840d1020 0x7f16840d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f1689734640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16840b97a0 con 0x7f168415dbe0 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f1689734640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16840d1c50 con 0x7f16840b2cb0 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f1689734640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168415dfc0 con 0x7f16840d1020 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f1682ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f168415dbe0 0x7f16840d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41374/0 (socket says 172.21.15.19:41374) 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f1682ffd640 1 -- 172.21.15.19:0/4102142067 learned_addr learned my addr 172.21.15.19:0/4102142067 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f16837fe640 1 --2- 172.21.15.19:0/4102142067 >> v2:172.21.15.112:3300/0 conn(0x7f16840d1020 0x7f16840d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f1681ffb640 1 -- 172.21.15.19:0/4102142067 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190790394 0 0) 0x7f16840b97a0 con 0x7f168415dbe0 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f1681ffb640 1 -- 172.21.15.19:0/4102142067 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f166c003520 con 0x7f168415dbe0 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f16837fe640 1 -- 172.21.15.19:0/4102142067 >> v1:172.21.15.112:6789/0 conn(0x7f16840b2cb0 legacy=0x7f16840d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f1681ffb640 1 -- 172.21.15.19:0/4102142067 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199975442 0 0) 0x7f16840d1c50 con 0x7f16840b2cb0 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f16837fe640 1 -- 172.21.15.19:0/4102142067 >> v1:172.21.15.19:6789/0 conn(0x7f168415dbe0 legacy=0x7f16840d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.026+0000 7f16837fe640 1 -- 172.21.15.19:0/4102142067 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16840d1f30 con 0x7f16840d1020 2024-04-25T10:04:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.027+0000 7f16837fe640 1 --2- 172.21.15.19:0/4102142067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16840d1020 0x7f16840d2e40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f167c009d90 tx=0x7f167c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40071f4c27415a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.027+0000 7f1681ffb640 1 -- 172.21.15.19:0/4102142067 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c007740 con 0x7f16840d1020 2024-04-25T10:04:15.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.027+0000 7f1681ffb640 1 -- 172.21.15.19:0/4102142067 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167c0078e0 con 0x7f16840d1020 2024-04-25T10:04:15.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.027+0000 7f1681ffb640 1 -- 172.21.15.19:0/4102142067 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c007cb0 con 0x7f16840d1020 2024-04-25T10:04:15.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.028+0000 7f1689734640 1 -- 172.21.15.19:0/4102142067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16840d1020 msgr2=0x7f16840d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.028+0000 7f1689734640 1 --2- 172.21.15.19:0/4102142067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16840d1020 0x7f16840d2e40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f167c009d90 tx=0x7f167c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.028+0000 7f1689734640 1 -- 172.21.15.19:0/4102142067 shutdown_connections 2024-04-25T10:04:15.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.028+0000 7f1689734640 1 --2- 172.21.15.19:0/4102142067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16840d1020 0x7f16840d2e40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.028+0000 7f1689734640 1 -- 172.21.15.19:0/4102142067 >> 172.21.15.19:0/4102142067 conn(0x7f1684061a20 msgr2=0x7f1684061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.028+0000 7f1689734640 1 -- 172.21.15.19:0/4102142067 shutdown_connections 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.028+0000 7f1689734640 1 -- 172.21.15.19:0/4102142067 wait complete. 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.029+0000 7f1689734640 1 Processor -- start 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.029+0000 7f1689734640 1 -- start start 2024-04-25T10:04:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1689734640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16840b2cb0 0x7f16840cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1689734640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16840d1020 0x7f16841f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1689734640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168415dbe0 0x7f16841539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1689734640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16840b92a0 con 0x7f16840b2cb0 2024-04-25T10:04:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1689734640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16840d1a00 con 0x7f168415dbe0 2024-04-25T10:04:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1689734640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16840b9420 con 0x7f16840d1020 2024-04-25T10:04:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1682ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16840b2cb0 0x7f16840cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1682ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16840b2cb0 0x7f16840cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38012/0 (socket says 172.21.15.19:38012) 2024-04-25T10:04:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1682ffd640 1 -- 172.21.15.19:0/2486937922 learned_addr learned my addr 172.21.15.19:0/2486937922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f16837fe640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168415dbe0 0x7f16841539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f16827fc640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16840d1020 0x7f16841f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1682ffd640 1 -- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16840d1020 msgr2=0x7f16841f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1682ffd640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16840d1020 0x7f16841f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1682ffd640 1 -- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168415dbe0 msgr2=0x7f16841539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1682ffd640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168415dbe0 0x7f16841539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1682ffd640 1 -- 172.21.15.19:0/2486937922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16841f79c0 con 0x7f16840b2cb0 2024-04-25T10:04:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f16837fe640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168415dbe0 0x7f16841539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.030+0000 7f1682ffd640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16840b2cb0 0x7f16840cbb80 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f167400c390 tx=0x7f167400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.031+0000 7f166b7fe640 1 -- 172.21.15.19:0/2486937922 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674017070 con 0x7f16840b2cb0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.031+0000 7f166b7fe640 1 -- 172.21.15.19:0/2486937922 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1674005ce0 con 0x7f16840b2cb0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.031+0000 7f1689734640 1 -- 172.21.15.19:0/2486937922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16841f4410 con 0x7f16840b2cb0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.031+0000 7f166b7fe640 1 -- 172.21.15.19:0/2486937922 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16740072f0 con 0x7f16840b2cb0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.031+0000 7f1689734640 1 -- 172.21.15.19:0/2486937922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16841f3b70 con 0x7f16840b2cb0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.032+0000 7f1689734640 1 -- 172.21.15.19:0/2486937922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1684073990 con 0x7f16840b2cb0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.032+0000 7f166b7fe640 1 -- 172.21.15.19:0/2486937922 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f167401a070 con 0x7f16840b2cb0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.032+0000 7f166b7fe640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b4d0 0x7f167006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.033+0000 7f166b7fe640 1 -- 172.21.15.19:0/2486937922 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1674089820 con 0x7f16840b2cb0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.033+0000 7f16827fc640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b4d0 0x7f167006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.033+0000 7f16827fc640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b4d0 0x7f167006d990 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f16840b24a0 tx=0x7f1678009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.037+0000 7f166b7fe640 1 -- 172.21.15.19:0/2486937922 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16841f79c0 con 0x7f16840b2cb0 2024-04-25T10:04:15.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.217+0000 7f1689734640 1 -- 172.21.15.19:0/2486937922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f168415cc90 con 0x7f16840b2cb0 2024-04-25T10:04:15.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.219+0000 7f166b7fe640 1 -- 172.21.15.19:0/2486937922 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f1674058480 con 0x7f16840b2cb0 2024-04-25T10:04:15.225 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:15.225 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":164,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:04:14.252144+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:12.518429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:13.232769+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 -- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b4d0 msgr2=0x7f167006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b4d0 0x7f167006d990 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f16840b24a0 tx=0x7f1678009040 comp rx=0 tx=0).stop 2024-04-25T10:04:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 -- 172.21.15.19:0/2486937922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16840b2cb0 msgr2=0x7f16840cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16840b2cb0 0x7f16840cbb80 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f167400c390 tx=0x7f167400c850 comp rx=0 tx=0).stop 2024-04-25T10:04:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 -- 172.21.15.19:0/2486937922 shutdown_connections 2024-04-25T10:04:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f167006b4d0 0x7f167006d990 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f168415dbe0 0x7f16841539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16840d1020 0x7f16841f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 --2- 172.21.15.19:0/2486937922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16840b2cb0 0x7f16840cbb80 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 -- 172.21.15.19:0/2486937922 >> 172.21.15.19:0/2486937922 conn(0x7f1684061a20 msgr2=0x7f1684158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 -- 172.21.15.19:0/2486937922 shutdown_connections 2024-04-25T10:04:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.221+0000 7f1689734640 1 -- 172.21.15.19:0/2486937922 wait complete. 2024-04-25T10:04:15.238 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.333+0000 7f2f5439b640 1 Processor -- start 2024-04-25T10:04:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.333+0000 7f2f5439b640 1 -- start start 2024-04-25T10:04:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f5439b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f4c15dbe0 0x7f2f4c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f5439b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c0b6220 con 0x7f2f4c0b2cb0 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f5439b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c0b97a0 con 0x7f2f4c0b92a0 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f5439b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4c15dfc0 con 0x7f2f4c15dbe0 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f51936640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2f4c0b2cb0 0x7f2f4c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41378/0 (socket says 172.21.15.19:41378) 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f51936640 1 -- 172.21.15.19:0/997416457 learned_addr learned my addr 172.21.15.19:0/997416457 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227923178 0 0) 0x7f2f4c0b6220 con 0x7f2f4c0b2cb0 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f38003520 con 0x7f2f4c0b2cb0 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f52137640 1 --2- 172.21.15.19:0/997416457 >> v2:172.21.15.112:3300/0 conn(0x7f2f4c15dbe0 0x7f2f4c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3440758897 0 0) 0x7f2f38003520 con 0x7f2f4c0b2cb0 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 >> v2:172.21.15.112:3300/0 conn(0x7f2f4c15dbe0 msgr2=0x7f2f4c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f51135640 1 --2- 172.21.15.19:0/997416457 >> v2:172.21.15.112:3300/0 conn(0x7f2f4c15dbe0 0x7f2f4c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 >> v1:172.21.15.112:6789/0 conn(0x7f2f4c0b92a0 legacy=0x7f2f4c0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c0d1c50 con 0x7f2f4c0b2cb0 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f2f3c001670 con 0x7f2f4c0b2cb0 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.334+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 >> v1:172.21.15.19:6789/0 conn(0x7f2f4c0b2cb0 legacy=0x7f2f4c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f51135640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f380039b0 0x7f2f38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f51135640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f38004800 0x7f2f38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f51135640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 0x7f2f3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3800c130 con 0x7f2f380051c0 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3800c2e0 con 0x7f2f380039b0 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3800c4e0 con 0x7f2f38004800 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f52938640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 0x7f2f3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f52137640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f380039b0 0x7f2f38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f51936640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f38004800 0x7f2f38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f52938640 1 -- 172.21.15.19:0/997416457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f38004800 msgr2=0x7f2f38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f52938640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f38004800 0x7f2f38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f52938640 1 -- 172.21.15.19:0/997416457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f380039b0 msgr2=0x7f2f38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f52938640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f380039b0 0x7f2f38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f52938640 1 -- 172.21.15.19:0/997416457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38003520 con 0x7f2f380051c0 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.335+0000 7f2f52137640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f380039b0 0x7f2f38003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.336+0000 7f2f52938640 1 -- 172.21.15.19:0/997416457 reap_dead start 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.336+0000 7f2f52938640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 0x7f2f3800b9c0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f2f44001e20 tx=0x7f2f38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c068a8eb2cd01cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.336+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48009690 con 0x7f2f380051c0 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.336+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4402ae40 con 0x7f2f380051c0 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.336+0000 7f2f51135640 1 -- 172.21.15.19:0/997416457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44001a70 con 0x7f2f380051c0 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.336+0000 7f2f5439b640 1 -- 172.21.15.19:0/997416457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 msgr2=0x7f2f3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.336+0000 7f2f5439b640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 0x7f2f3800b9c0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f2f44001e20 tx=0x7f2f38005830 comp rx=0 tx=0).stop 2024-04-25T10:04:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.336+0000 7f2f5439b640 1 -- 172.21.15.19:0/997416457 shutdown_connections 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.336+0000 7f2f5439b640 1 --2- 172.21.15.19:0/997416457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 0x7f2f3800b9c0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.336+0000 7f2f5439b640 1 -- 172.21.15.19:0/997416457 >> 172.21.15.19:0/997416457 conn(0x7f2f4c061a20 msgr2=0x7f2f4c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.337+0000 7f2f5439b640 1 -- 172.21.15.19:0/997416457 shutdown_connections 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.337+0000 7f2f5439b640 1 -- 172.21.15.19:0/997416457 wait complete. 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.338+0000 7f2f5439b640 1 Processor -- start 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.338+0000 7f2f5439b640 1 -- start start 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.338+0000 7f2f5439b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 0x7f2f4c159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.338+0000 7f2f5439b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f4c0ab060 0x7f2f4c15f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.338+0000 7f2f5439b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c15fa50 0x7f2f4c0aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.338+0000 7f2f5439b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0d2210 con 0x7f2f380051c0 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.338+0000 7f2f5439b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0d1a00 con 0x7f2f4c15fa50 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.338+0000 7f2f5439b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0b6ab0 con 0x7f2f4c0ab060 2024-04-25T10:04:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.338+0000 7f2f52137640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 0x7f2f4c159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f52137640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 0x7f2f4c159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38030/0 (socket says 172.21.15.19:38030) 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f52137640 1 -- 172.21.15.19:0/3336471495 learned_addr learned my addr 172.21.15.19:0/3336471495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f52938640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c15fa50 0x7f2f4c0aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f51936640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f4c0ab060 0x7f2f4c15f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f52137640 1 -- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f4c0ab060 msgr2=0x7f2f4c15f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f52137640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f4c0ab060 0x7f2f4c15f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f52137640 1 -- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c15fa50 msgr2=0x7f2f4c0aa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f52137640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c15fa50 0x7f2f4c0aa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f52137640 1 -- 172.21.15.19:0/3336471495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c15ff70 con 0x7f2f380051c0 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f52938640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c15fa50 0x7f2f4c0aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f52137640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 0x7f2f4c159000 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f2f48011930 tx=0x7f2f48011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f42ffd640 1 -- 172.21.15.19:0/3336471495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4800f9d0 con 0x7f2f380051c0 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f42ffd640 1 -- 172.21.15.19:0/3336471495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4800fb70 con 0x7f2f380051c0 2024-04-25T10:04:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f5439b640 1 -- 172.21.15.19:0/3336471495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4c160250 con 0x7f2f380051c0 2024-04-25T10:04:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f42ffd640 1 -- 172.21.15.19:0/3336471495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48014440 con 0x7f2f380051c0 2024-04-25T10:04:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.339+0000 7f2f5439b640 1 -- 172.21.15.19:0/3336471495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c0cbeb0 con 0x7f2f380051c0 2024-04-25T10:04:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.340+0000 7f2f5439b640 1 -- 172.21.15.19:0/3336471495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4c073990 con 0x7f2f380051c0 2024-04-25T10:04:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.341+0000 7f2f42ffd640 1 -- 172.21.15.19:0/3336471495 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f48012040 con 0x7f2f380051c0 2024-04-25T10:04:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.341+0000 7f2f42ffd640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f3006b500 0x7f2f3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.341+0000 7f2f42ffd640 1 -- 172.21.15.19:0/3336471495 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f4808a940 con 0x7f2f380051c0 2024-04-25T10:04:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.341+0000 7f2f51936640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f3006b500 0x7f2f3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.342+0000 7f2f51936640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f3006b500 0x7f2f3006d9c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c001050 tx=0x7f2f3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.345+0000 7f2f42ffd640 1 -- 172.21.15.19:0/3336471495 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f48019030 con 0x7f2f380051c0 2024-04-25T10:04:15.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.528+0000 7f2f5439b640 1 -- 172.21.15.19:0/3336471495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f4c159ea0 con 0x7f2f380051c0 2024-04-25T10:04:15.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.529+0000 7f2f42ffd640 1 -- 172.21.15.19:0/3336471495 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f2f48059630 con 0x7f2f380051c0 2024-04-25T10:04:15.535 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:15.536 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":164,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:04:14.252144+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:12.518429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:13.232769+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 -- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f3006b500 msgr2=0x7f2f3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f3006b500 0x7f2f3006d9c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c001050 tx=0x7f2f3c009040 comp rx=0 tx=0).stop 2024-04-25T10:04:15.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 -- 172.21.15.19:0/3336471495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 msgr2=0x7f2f4c159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 0x7f2f4c159000 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f2f48011930 tx=0x7f2f48011df0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 -- 172.21.15.19:0/3336471495 shutdown_connections 2024-04-25T10:04:15.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f3006b500 0x7f2f3006d9c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c15fa50 0x7f2f4c0aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f4c0ab060 0x7f2f4c15f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 --2- 172.21.15.19:0/3336471495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f380051c0 0x7f2f4c159000 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 -- 172.21.15.19:0/3336471495 >> 172.21.15.19:0/3336471495 conn(0x7f2f4c061a20 msgr2=0x7f2f4c0d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 -- 172.21.15.19:0/3336471495 shutdown_connections 2024-04-25T10:04:15.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.531+0000 7f2f5439b640 1 -- 172.21.15.19:0/3336471495 wait complete. 2024-04-25T10:04:15.549 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:04:15.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.647+0000 7f16f044a640 1 Processor -- start 2024-04-25T10:04:15.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.647+0000 7f16f044a640 1 -- start start 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16f044a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16e815dac0 0x7f16e80d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16f044a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f16e80d1f30 0x7f16e80d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16f044a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e80d2420 con 0x7f16e80d34c0 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16f044a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e80d3030 con 0x7f16e80d1f30 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16f044a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80d2700 con 0x7f16e815dac0 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16ee9e7640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f16e80d34c0 0x7f16e80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41384/0 (socket says 172.21.15.19:41384) 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16ee9e7640 1 -- 172.21.15.19:0/4166514113 learned_addr learned my addr 172.21.15.19:0/4166514113 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16ed9e5640 1 --2- 172.21.15.19:0/4166514113 >> v2:172.21.15.112:3300/0 conn(0x7f16e80d1f30 0x7f16e80d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16ee1e6640 1 --2- 172.21.15.19:0/4166514113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16e815dac0 0x7f16e80d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16ed1e4640 1 -- 172.21.15.19:0/4166514113 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808881638 0 0) 0x7f16e80d2420 con 0x7f16e80d34c0 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16ed1e4640 1 -- 172.21.15.19:0/4166514113 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16d4003520 con 0x7f16e80d34c0 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16ed9e5640 1 -- 172.21.15.19:0/4166514113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16e815dac0 msgr2=0x7f16e80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16ed9e5640 1 --2- 172.21.15.19:0/4166514113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16e815dac0 0x7f16e80d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16ed9e5640 1 -- 172.21.15.19:0/4166514113 >> v1:172.21.15.19:6789/0 conn(0x7f16e80d34c0 legacy=0x7f16e80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.648+0000 7f16ed9e5640 1 -- 172.21.15.19:0/4166514113 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e80d29e0 con 0x7f16e80d1f30 2024-04-25T10:04:15.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.649+0000 7f16ed9e5640 1 --2- 172.21.15.19:0/4166514113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e80d1f30 0x7f16e80d03a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009790 tx=0x7f16d802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3792cd1c2f9e486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.649+0000 7f16ed1e4640 1 -- 172.21.15.19:0/4166514113 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8037070 con 0x7f16e80d1f30 2024-04-25T10:04:15.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.649+0000 7f16ed1e4640 1 -- 172.21.15.19:0/4166514113 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8032440 con 0x7f16e80d1f30 2024-04-25T10:04:15.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.649+0000 7f16ed1e4640 1 -- 172.21.15.19:0/4166514113 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8032810 con 0x7f16e80d1f30 2024-04-25T10:04:15.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.650+0000 7f16f044a640 1 -- 172.21.15.19:0/4166514113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e80d1f30 msgr2=0x7f16e80d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.650+0000 7f16f044a640 1 --2- 172.21.15.19:0/4166514113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e80d1f30 0x7f16e80d03a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009790 tx=0x7f16d802a210 comp rx=0 tx=0).stop 2024-04-25T10:04:15.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.650+0000 7f16f044a640 1 -- 172.21.15.19:0/4166514113 shutdown_connections 2024-04-25T10:04:15.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.650+0000 7f16f044a640 1 --2- 172.21.15.19:0/4166514113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e80d1f30 0x7f16e80d03a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.650+0000 7f16f044a640 1 --2- 172.21.15.19:0/4166514113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16e815dac0 0x7f16e80d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.650+0000 7f16f044a640 1 -- 172.21.15.19:0/4166514113 >> 172.21.15.19:0/4166514113 conn(0x7f16e8061c30 msgr2=0x7f16e806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.650+0000 7f16f044a640 1 -- 172.21.15.19:0/4166514113 shutdown_connections 2024-04-25T10:04:15.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.650+0000 7f16f044a640 1 -- 172.21.15.19:0/4166514113 wait complete. 2024-04-25T10:04:15.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.651+0000 7f16f044a640 1 Processor -- start 2024-04-25T10:04:15.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.651+0000 7f16f044a640 1 -- start start 2024-04-25T10:04:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.652+0000 7f16f044a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16e80d1f30 0x7f16e814bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.652+0000 7f16f044a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16e80d34c0 0x7f16e815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.652+0000 7f16f044a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e815dac0 0x7f16e815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.652+0000 7f16f044a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80b97d0 con 0x7f16e80d1f30 2024-04-25T10:04:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.652+0000 7f16f044a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e815dea0 con 0x7f16e815dac0 2024-04-25T10:04:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.652+0000 7f16f044a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80b9950 con 0x7f16e80d34c0 2024-04-25T10:04:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.652+0000 7f16ee1e6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16e80d1f30 0x7f16e814bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.652+0000 7f16ee1e6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16e80d1f30 0x7f16e814bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38044/0 (socket says 172.21.15.19:38044) 2024-04-25T10:04:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.652+0000 7f16ee1e6640 1 -- 172.21.15.19:0/965161300 learned_addr learned my addr 172.21.15.19:0/965161300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.652+0000 7f16ee9e7640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e815dac0 0x7f16e815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.652+0000 7f16ed9e5640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16e80d34c0 0x7f16e815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16ee1e6640 1 -- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16e80d34c0 msgr2=0x7f16e815e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16ee1e6640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16e80d34c0 0x7f16e815e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16ee1e6640 1 -- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e815dac0 msgr2=0x7f16e815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16ee1e6640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e815dac0 0x7f16e815fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16ee1e6640 1 -- 172.21.15.19:0/965161300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e80b33c0 con 0x7f16e80d1f30 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16ed9e5640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16e80d34c0 0x7f16e815e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16ee1e6640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16e80d1f30 0x7f16e814bef0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f16e400e870 tx=0x7f16e400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16deffd640 1 -- 172.21.15.19:0/965161300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4017070 con 0x7f16e80d1f30 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16deffd640 1 -- 172.21.15.19:0/965161300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e400c940 con 0x7f16e80d1f30 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16f044a640 1 -- 172.21.15.19:0/965161300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e815eb00 con 0x7f16e80d1f30 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16deffd640 1 -- 172.21.15.19:0/965161300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e400cd30 con 0x7f16e80d1f30 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.653+0000 7f16f044a640 1 -- 172.21.15.19:0/965161300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e815f0c0 con 0x7f16e80d1f30 2024-04-25T10:04:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.654+0000 7f16f044a640 1 -- 172.21.15.19:0/965161300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e8073a00 con 0x7f16e80d1f30 2024-04-25T10:04:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.655+0000 7f16deffd640 1 -- 172.21.15.19:0/965161300 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16e401a070 con 0x7f16e80d1f30 2024-04-25T10:04:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.655+0000 7f16deffd640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16cc06b4d0 0x7f16cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.655+0000 7f16deffd640 1 -- 172.21.15.19:0/965161300 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16e408a1c0 con 0x7f16e80d1f30 2024-04-25T10:04:15.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.655+0000 7f16ed9e5640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16cc06b4d0 0x7f16cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.656+0000 7f16ed9e5640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16cc06b4d0 0x7f16cc06d990 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f16d8034000 tx=0x7f16d802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.659+0000 7f16deffd640 1 -- 172.21.15.19:0/965161300 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16e4058f40 con 0x7f16e80d1f30 2024-04-25T10:04:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.885+0000 7f16f044a640 1 -- 172.21.15.19:0/965161300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f16e806c470 con 0x7f16e80d1f30 2024-04-25T10:04:15.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.904+0000 7f16deffd640 1 -- 172.21.15.19:0/965161300 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v108) v1 ==== 118+0+0 (secure 0 0 0) 0x7f16e4058840 con 0x7f16e80d1f30 2024-04-25T10:04:15.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 -- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16cc06b4d0 msgr2=0x7f16cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16cc06b4d0 0x7f16cc06d990 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f16d8034000 tx=0x7f16d802e000 comp rx=0 tx=0).stop 2024-04-25T10:04:15.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 -- 172.21.15.19:0/965161300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16e80d1f30 msgr2=0x7f16e814bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16e80d1f30 0x7f16e814bef0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f16e400e870 tx=0x7f16e400ed30 comp rx=0 tx=0).stop 2024-04-25T10:04:15.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 -- 172.21.15.19:0/965161300 shutdown_connections 2024-04-25T10:04:15.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f16cc06b4d0 0x7f16cc06d990 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e815dac0 0x7f16e815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16e80d34c0 0x7f16e815e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 --2- 172.21.15.19:0/965161300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16e80d1f30 0x7f16e814bef0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 -- 172.21.15.19:0/965161300 >> 172.21.15.19:0/965161300 conn(0x7f16e8061c30 msgr2=0x7f16e80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 -- 172.21.15.19:0/965161300 shutdown_connections 2024-04-25T10:04:15.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:15.907+0000 7f16f044a640 1 -- 172.21.15.19:0/965161300 wait complete. 2024-04-25T10:04:15.924 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:04:15.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:04:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.022+0000 7fa5b54dd640 1 Processor -- start 2024-04-25T10:04:16.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.022+0000 7fa5b54dd640 1 -- start start 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.022+0000 7fa5b54dd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa5b015dac0 0x7fa5b00d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.022+0000 7fa5b54dd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b00d2420 con 0x7fa5b00d0ab0 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.022+0000 7fa5b54dd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b00d2700 con 0x7fa5b00d1f30 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.022+0000 7fa5b54dd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b00d3030 con 0x7fa5b015dac0 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5af7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa5b00d0ab0 0x7fa5b00b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41400/0 (socket says 172.21.15.19:41400) 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5af7fe640 1 -- 172.21.15.19:0/1577230250 learned_addr learned my addr 172.21.15.19:0/1577230250 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661194901 0 0) 0x7fa5b00d2420 con 0x7fa5b00d0ab0 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa59c003520 con 0x7fa5b00d0ab0 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5aeffd640 1 --2- 172.21.15.19:0/1577230250 >> v2:172.21.15.112:3300/0 conn(0x7fa5b015dac0 0x7fa5b00d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 83251898 0 0) 0x7fa59c003520 con 0x7fa5b00d0ab0 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 >> v2:172.21.15.112:3300/0 conn(0x7fa5b015dac0 msgr2=0x7fa5b00d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 --2- 172.21.15.19:0/1577230250 >> v2:172.21.15.112:3300/0 conn(0x7fa5b015dac0 0x7fa5b00d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 >> v1:172.21.15.112:6789/0 conn(0x7fa5b00d1f30 legacy=0x7fa5b00d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b00d29e0 con 0x7fa5b00d0ab0 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fa5a00019f0 con 0x7fa5b00d0ab0 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 >> v1:172.21.15.19:6789/0 conn(0x7fa5b00d0ab0 legacy=0x7fa5b00b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 --2- 172.21.15.19:0/1577230250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa59c0039b0 0x7fa59c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 --2- 172.21.15.19:0/1577230250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa59c004800 0x7fa59c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 --2- 172.21.15.19:0/1577230250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa59c0051c0 0x7fa59c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00d3030 con 0x7fa59c004800 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c00c070 con 0x7fa59c0039b0 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c00c270 con 0x7fa59c0051c0 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5ae7fc640 1 --2- 172.21.15.19:0/1577230250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa59c004800 0x7fa59c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5aeffd640 1 --2- 172.21.15.19:0/1577230250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa59c0039b0 0x7fa59c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.023+0000 7fa5af7fe640 1 --2- 172.21.15.19:0/1577230250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa59c0051c0 0x7fa59c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.024+0000 7fa5ae7fc640 1 -- 172.21.15.19:0/1577230250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa59c0051c0 msgr2=0x7fa59c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.024+0000 7fa5ae7fc640 1 --2- 172.21.15.19:0/1577230250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa59c0051c0 0x7fa59c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.024+0000 7fa5ae7fc640 1 -- 172.21.15.19:0/1577230250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa59c0039b0 msgr2=0x7fa59c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.024+0000 7fa5ae7fc640 1 --2- 172.21.15.19:0/1577230250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa59c0039b0 0x7fa59c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.024+0000 7fa5ae7fc640 1 -- 172.21.15.19:0/1577230250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c003520 con 0x7fa59c004800 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.024+0000 7fa5af7fe640 1 -- 172.21.15.19:0/1577230250 reap_dead start 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.024+0000 7fa5ae7fc640 1 --2- 172.21.15.19:0/1577230250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa59c004800 0x7fa59c004c80 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fa598001770 tx=0x7fa59c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2e02621c867b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.025+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5980013c0 con 0x7fa59c004800 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.025+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5980052d0 con 0x7fa59c004800 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.025+0000 7fa5adffb640 1 -- 172.21.15.19:0/1577230250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59802a8a0 con 0x7fa59c004800 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.025+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1577230250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa59c004800 msgr2=0x7fa59c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.025+0000 7fa5b54dd640 1 --2- 172.21.15.19:0/1577230250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa59c004800 0x7fa59c004c80 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fa598001770 tx=0x7fa59c005830 comp rx=0 tx=0).stop 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.025+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1577230250 shutdown_connections 2024-04-25T10:04:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.025+0000 7fa5b54dd640 1 --2- 172.21.15.19:0/1577230250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa59c004800 0x7fa59c004c80 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.025+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1577230250 >> 172.21.15.19:0/1577230250 conn(0x7fa5b0061c30 msgr2=0x7fa5b006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.025+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1577230250 shutdown_connections 2024-04-25T10:04:16.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.025+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1577230250 wait complete. 2024-04-25T10:04:16.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.026+0000 7fa5b54dd640 1 Processor -- start 2024-04-25T10:04:16.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5b54dd640 1 -- start start 2024-04-25T10:04:16.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5b54dd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa59c004800 0x7fa5b0153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5b54dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b00caaa0 0x7fa5b00cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5b54dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b00cce80 0x7fa5b00cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5b54dd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00b97d0 con 0x7fa5b00caaa0 2024-04-25T10:04:16.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5b54dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b015dea0 con 0x7fa5b00cce80 2024-04-25T10:04:16.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5b54dd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00b9950 con 0x7fa59c004800 2024-04-25T10:04:16.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5ae7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b00caaa0 0x7fa5b00cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5ae7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b00caaa0 0x7fa5b00cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38072/0 (socket says 172.21.15.19:38072) 2024-04-25T10:04:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5ae7fc640 1 -- 172.21.15.19:0/1917537485 learned_addr learned my addr 172.21.15.19:0/1917537485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5aeffd640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa59c004800 0x7fa5b0153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.027+0000 7fa5af7fe640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b00cce80 0x7fa5b00cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.028+0000 7fa5ae7fc640 1 -- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa59c004800 msgr2=0x7fa5b0153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.028+0000 7fa5ae7fc640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa59c004800 0x7fa5b0153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.028+0000 7fa5ae7fc640 1 -- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b00cce80 msgr2=0x7fa5b00cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.028+0000 7fa5ae7fc640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b00cce80 0x7fa5b00cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.028+0000 7fa5ae7fc640 1 -- 172.21.15.19:0/1917537485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0153a80 con 0x7fa5b00caaa0 2024-04-25T10:04:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.028+0000 7fa5ae7fc640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b00caaa0 0x7fa5b00cb410 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fa598000c00 tx=0x7fa598001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.028+0000 7fa5a77fe640 1 -- 172.21.15.19:0/1917537485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598040020 con 0x7fa5b00caaa0 2024-04-25T10:04:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.028+0000 7fa5a77fe640 1 -- 172.21.15.19:0/1917537485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5980050d0 con 0x7fa5b00caaa0 2024-04-25T10:04:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.028+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1917537485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b00c7c80 con 0x7fa5b00caaa0 2024-04-25T10:04:16.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.028+0000 7fa5a77fe640 1 -- 172.21.15.19:0/1917537485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598006a60 con 0x7fa5b00caaa0 2024-04-25T10:04:16.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.028+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1917537485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b00c8240 con 0x7fa5b00caaa0 2024-04-25T10:04:16.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.029+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1917537485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b0073a00 con 0x7fa5b00caaa0 2024-04-25T10:04:16.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.030+0000 7fa5a77fe640 1 -- 172.21.15.19:0/1917537485 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa598043020 con 0x7fa5b00caaa0 2024-04-25T10:04:16.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.030+0000 7fa5a77fe640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa59006b4d0 0x7fa59006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.030+0000 7fa5a77fe640 1 -- 172.21.15.19:0/1917537485 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5980aa0f0 con 0x7fa5b00caaa0 2024-04-25T10:04:16.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.030+0000 7fa5aeffd640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa59006b4d0 0x7fa59006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.031+0000 7fa5aeffd640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa59006b4d0 0x7fa59006d990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0068c30 tx=0x7fa5a8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.034+0000 7fa5a77fe640 1 -- 172.21.15.19:0/1917537485 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa59804a030 con 0x7fa5b00caaa0 2024-04-25T10:04:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.257+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1917537485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa5b00b2ac0 con 0x7fa5b00caaa0 2024-04-25T10:04:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.907+0000 7fa5a77fe640 1 -- 172.21.15.19:0/1917537485 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v109) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa5b0153a80 con 0x7fa5b00caaa0 2024-04-25T10:04:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa59006b4d0 msgr2=0x7fa59006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa59006b4d0 0x7fa59006d990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0068c30 tx=0x7fa5a8009190 comp rx=0 tx=0).stop 2024-04-25T10:04:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1917537485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b00caaa0 msgr2=0x7fa5b00cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b00caaa0 0x7fa5b00cb410 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fa598000c00 tx=0x7fa598001170 comp rx=0 tx=0).stop 2024-04-25T10:04:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1917537485 shutdown_connections 2024-04-25T10:04:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa59006b4d0 0x7fa59006d990 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b00cce80 0x7fa5b00cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5b00caaa0 0x7fa5b00cb410 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 --2- 172.21.15.19:0/1917537485 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa59c004800 0x7fa5b0153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1917537485 >> 172.21.15.19:0/1917537485 conn(0x7fa5b0061c30 msgr2=0x7fa5b00b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1917537485 shutdown_connections 2024-04-25T10:04:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:16.910+0000 7fa5b54dd640 1 -- 172.21.15.19:0/1917537485 wait complete. 2024-04-25T10:04:16.929 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:04:16.929 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.030+0000 7f2f3abb7640 1 Processor -- start 2024-04-25T10:04:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.030+0000 7f2f3abb7640 1 -- start start 2024-04-25T10:04:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.031+0000 7f2f3abb7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f3415dc00 0x7f2f340c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.031+0000 7f2f3abb7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f340ceb20 0x7f2f340d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.031+0000 7f2f3abb7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3415dfe0 con 0x7f2f340d1c00 2024-04-25T10:04:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.031+0000 7f2f3abb7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f340d2060 con 0x7f2f3415dc00 2024-04-25T10:04:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.031+0000 7f2f3abb7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340d00c0 con 0x7f2f340ceb20 2024-04-25T10:04:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.032+0000 7f2f33fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f340ceb20 0x7f2f340d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.032+0000 7f2f38953640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f3415dc00 0x7f2f340c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.032+0000 7f2f38953640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f3415dc00 0x7f2f340c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:37112/0 (socket says 172.21.15.19:37112) 2024-04-25T10:04:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.032+0000 7f2f38953640 1 -- 172.21.15.19:0/1143304902 learned_addr learned my addr 172.21.15.19:0/1143304902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.032+0000 7f2f39154640 1 --1- 172.21.15.19:0/1143304902 >> v1:172.21.15.112:6789/0 conn(0x7f2f340d1c00 0x7f2f340ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:47144/0 (socket says 172.21.15.19:47144) 2024-04-25T10:04:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.032+0000 7f2f38953640 1 -- 172.21.15.19:0/1143304902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f340ceb20 msgr2=0x7f2f340d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.032+0000 7f2f38953640 1 --2- 172.21.15.19:0/1143304902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f340ceb20 0x7f2f340d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.032+0000 7f2f38953640 1 -- 172.21.15.19:0/1143304902 >> v1:172.21.15.112:6789/0 conn(0x7f2f340d1c00 legacy=0x7f2f340ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.032+0000 7f2f38953640 1 -- 172.21.15.19:0/1143304902 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f340d03a0 con 0x7f2f3415dc00 2024-04-25T10:04:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.032+0000 7f2f33fff640 1 --2- 172.21.15.19:0/1143304902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f340ceb20 0x7f2f340d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.032+0000 7f2f337fe640 1 -- 172.21.15.19:0/1143304902 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298396758 0 0) 0x7f2f1c001110 con 0x7f2f340d1c00 2024-04-25T10:04:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.033+0000 7f2f38953640 1 --2- 172.21.15.19:0/1143304902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3415dc00 0x7f2f340c6df0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009870 tx=0x7f2f2c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1fd3796a6a76c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.033+0000 7f2f337fe640 1 -- 172.21.15.19:0/1143304902 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c037070 con 0x7f2f3415dc00 2024-04-25T10:04:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.033+0000 7f2f337fe640 1 -- 172.21.15.19:0/1143304902 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c032440 con 0x7f2f3415dc00 2024-04-25T10:04:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.033+0000 7f2f337fe640 1 -- 172.21.15.19:0/1143304902 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c032810 con 0x7f2f3415dc00 2024-04-25T10:04:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.034+0000 7f2f3abb7640 1 -- 172.21.15.19:0/1143304902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3415dc00 msgr2=0x7f2f340c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.034+0000 7f2f3abb7640 1 --2- 172.21.15.19:0/1143304902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3415dc00 0x7f2f340c6df0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009870 tx=0x7f2f2c02a270 comp rx=0 tx=0).stop 2024-04-25T10:04:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.034+0000 7f2f3abb7640 1 -- 172.21.15.19:0/1143304902 shutdown_connections 2024-04-25T10:04:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.034+0000 7f2f3abb7640 1 --2- 172.21.15.19:0/1143304902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f340ceb20 0x7f2f340d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.034+0000 7f2f3abb7640 1 --2- 172.21.15.19:0/1143304902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f3415dc00 0x7f2f340c6df0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.034+0000 7f2f3abb7640 1 -- 172.21.15.19:0/1143304902 >> 172.21.15.19:0/1143304902 conn(0x7f2f3406f5d0 msgr2=0x7f2f3406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.034+0000 7f2f3abb7640 1 -- 172.21.15.19:0/1143304902 shutdown_connections 2024-04-25T10:04:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.034+0000 7f2f3abb7640 1 -- 172.21.15.19:0/1143304902 wait complete. 2024-04-25T10:04:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.035+0000 7f2f3abb7640 1 Processor -- start 2024-04-25T10:04:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.035+0000 7f2f3abb7640 1 -- start start 2024-04-25T10:04:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f3abb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f340ceb20 0x7f2f3414f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f3abb7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f340d1c00 0x7f2f3415f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f3abb7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3415dc00 0x7f2f3415c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f3abb7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340cc130 con 0x7f2f3415dc00 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f3abb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340cc720 con 0x7f2f340ceb20 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f3abb7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340cfd90 con 0x7f2f340d1c00 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f39154640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3415dc00 0x7f2f3415c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f39154640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3415dc00 0x7f2f3415c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38080/0 (socket says 172.21.15.19:38080) 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f39154640 1 -- 172.21.15.19:0/2334885035 learned_addr learned my addr 172.21.15.19:0/2334885035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f38953640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f340ceb20 0x7f2f3414f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f33fff640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f340d1c00 0x7f2f3415f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f39154640 1 -- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f340d1c00 msgr2=0x7f2f3415f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f39154640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f340d1c00 0x7f2f3415f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f39154640 1 -- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f340ceb20 msgr2=0x7f2f3414f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f39154640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f340ceb20 0x7f2f3414f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f39154640 1 -- 172.21.15.19:0/2334885035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f340b4a80 con 0x7f2f3415dc00 2024-04-25T10:04:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.036+0000 7f2f39154640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3415dc00 0x7f2f3415c720 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00c780 tx=0x7f2f1c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.037+0000 7f2f317fa640 1 -- 172.21.15.19:0/2334885035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c018070 con 0x7f2f3415dc00 2024-04-25T10:04:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.037+0000 7f2f317fa640 1 -- 172.21.15.19:0/2334885035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c0070d0 con 0x7f2f3415dc00 2024-04-25T10:04:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.037+0000 7f2f317fa640 1 -- 172.21.15.19:0/2334885035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0074c0 con 0x7f2f3415dc00 2024-04-25T10:04:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.037+0000 7f2f3abb7640 1 -- 172.21.15.19:0/2334885035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f340b4d60 con 0x7f2f3415dc00 2024-04-25T10:04:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.037+0000 7f2f3abb7640 1 -- 172.21.15.19:0/2334885035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f340b5320 con 0x7f2f3415dc00 2024-04-25T10:04:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.038+0000 7f2f3abb7640 1 -- 172.21.15.19:0/2334885035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f34073990 con 0x7f2f3415dc00 2024-04-25T10:04:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.038+0000 7f2f317fa640 1 -- 172.21.15.19:0/2334885035 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f1c01b070 con 0x7f2f3415dc00 2024-04-25T10:04:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.038+0000 7f2f317fa640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f1406b4d0 0x7f2f1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.039+0000 7f2f317fa640 1 -- 172.21.15.19:0/2334885035 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f1c0894e0 con 0x7f2f3415dc00 2024-04-25T10:04:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.039+0000 7f2f38953640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f1406b4d0 0x7f2f1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.039+0000 7f2f38953640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f1406b4d0 0x7f2f1406d990 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c034000 tx=0x7f2f2c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.043+0000 7f2f317fa640 1 -- 172.21.15.19:0/2334885035 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f3415dfe0 con 0x7f2f3415dc00 2024-04-25T10:04:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.262+0000 7f2f3abb7640 1 -- 172.21.15.19:0/2334885035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f3406e570 con 0x7f2f3415dc00 2024-04-25T10:04:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.262+0000 7f2f317fa640 1 -- 172.21.15.19:0/2334885035 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4954 (secure 0 0 0) 0x7f2f1c057c10 con 0x7f2f3415dc00 2024-04-25T10:04:17.269 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:17.269 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1300145056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1300145056},{"type":"v1","addr":"172.21.15.19:6835","nonce":1300145056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":8027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2001687418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2001687418},{"type":"v1","addr":"172.21.15.112:6836","nonce":2001687418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721457153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721457153},{"type":"v1","addr":"172.21.15.112:6837","nonce":721457153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:14.252458+0000","modified":"2024-04-25T10:04:16.259643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10698":{"gid":10698,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3942829201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3942829201},{"type":"v1","addr":"172.21.15.19:6834","nonce":3942829201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:04:17.269 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 109 2024-04-25T10:04:17.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 -- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f1406b4d0 msgr2=0x7f2f1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f1406b4d0 0x7f2f1406d990 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c034000 tx=0x7f2f2c02e000 comp rx=0 tx=0).stop 2024-04-25T10:04:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 -- 172.21.15.19:0/2334885035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3415dc00 msgr2=0x7f2f3415c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3415dc00 0x7f2f3415c720 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00c780 tx=0x7f2f1c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:04:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 -- 172.21.15.19:0/2334885035 shutdown_connections 2024-04-25T10:04:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2f1406b4d0 0x7f2f1406d990 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f3415dc00 0x7f2f3415c720 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f340d1c00 0x7f2f3415f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 --2- 172.21.15.19:0/2334885035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f340ceb20 0x7f2f3414f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 -- 172.21.15.19:0/2334885035 >> 172.21.15.19:0/2334885035 conn(0x7f2f3406f5d0 msgr2=0x7f2f3406c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 -- 172.21.15.19:0/2334885035 shutdown_connections 2024-04-25T10:04:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.264+0000 7f2f3abb7640 1 -- 172.21.15.19:0/2334885035 wait complete. 2024-04-25T10:04:17.281 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:17.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.375+0000 7f53f2f2b640 1 Processor -- start 2024-04-25T10:04:17.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.375+0000 7f53f2f2b640 1 -- start start 2024-04-25T10:04:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.376+0000 7f53f2f2b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec1594f0 0x7f53ec0b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.376+0000 7f53f2f2b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f53ec15c3d0 0x7f53ec0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.376+0000 7f53f2f2b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ec0b5630 con 0x7f53ec15bf50 2024-04-25T10:04:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.376+0000 7f53f2f2b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec1598d0 con 0x7f53ec15c3d0 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.376+0000 7f53f2f2b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0b4c60 con 0x7f53ec1594f0 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.376+0000 7f53ebfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f53ec15bf50 0x7f53ec0d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41410/0 (socket says 172.21.15.19:41410) 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.376+0000 7f53ebfff640 1 -- 172.21.15.19:0/1673779879 learned_addr learned my addr 172.21.15.19:0/1673779879 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53f14c8640 1 --2- 172.21.15.19:0/1673779879 >> v2:172.21.15.112:3300/0 conn(0x7f53ec15c3d0 0x7f53ec0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53f0cc7640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec1594f0 0x7f53ec0b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246933255 0 0) 0x7f53ec0b5630 con 0x7f53ec15bf50 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d4003520 con 0x7f53ec15bf50 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2216129841 0 0) 0x7f53d4003520 con 0x7f53ec15bf50 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec1594f0 msgr2=0x7f53ec0b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec1594f0 0x7f53ec0b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 >> v2:172.21.15.112:3300/0 conn(0x7f53ec15c3d0 msgr2=0x7f53ec0b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 --2- 172.21.15.19:0/1673779879 >> v2:172.21.15.112:3300/0 conn(0x7f53ec15c3d0 0x7f53ec0b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec15c930 con 0x7f53ec15bf50 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f53e0001670 con 0x7f53ec15bf50 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 >> v1:172.21.15.19:6789/0 conn(0x7f53ec15bf50 legacy=0x7f53ec0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53d40039b0 0x7f53d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec1594f0 0x7f53d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4004d40 0x7f53d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1598d0 con 0x7f53d4004d40 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0b4c60 con 0x7f53d40039b0 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d400b9c0 con 0x7f53ec1594f0 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53f14c8640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4004d40 0x7f53d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53f0cc7640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53d40039b0 0x7f53d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53ebfff640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec1594f0 0x7f53d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53f14c8640 1 -- 172.21.15.19:0/1673779879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec1594f0 msgr2=0x7f53d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53f14c8640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec1594f0 0x7f53d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53f14c8640 1 -- 172.21.15.19:0/1673779879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53d40039b0 msgr2=0x7f53d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53f14c8640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53d40039b0 0x7f53d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.377+0000 7f53f14c8640 1 -- 172.21.15.19:0/1673779879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4003520 con 0x7f53d4004d40 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.378+0000 7f53f14c8640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4004d40 0x7f53d40051c0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009a90 tx=0x7f53d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4063716d7fcb1acc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.378+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc036070 con 0x7f53d4004d40 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.378+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc031400 con 0x7f53d4004d40 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.378+0000 7f53eb7fe640 1 -- 172.21.15.19:0/1673779879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc009690 con 0x7f53d4004d40 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.379+0000 7f53f2f2b640 1 -- 172.21.15.19:0/1673779879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4004d40 msgr2=0x7f53d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.379+0000 7f53f2f2b640 1 --2- 172.21.15.19:0/1673779879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4004d40 0x7f53d40051c0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009a90 tx=0x7f53d4005830 comp rx=0 tx=0).stop 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.379+0000 7f53f14c8640 1 -- 172.21.15.19:0/1673779879 reap_dead start 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.379+0000 7f53f2f2b640 1 -- 172.21.15.19:0/1673779879 shutdown_connections 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.379+0000 7f53f2f2b640 1 -- 172.21.15.19:0/1673779879 >> 172.21.15.19:0/1673779879 conn(0x7f53ec06f520 msgr2=0x7f53ec06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.379+0000 7f53f2f2b640 1 -- 172.21.15.19:0/1673779879 shutdown_connections 2024-04-25T10:04:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.379+0000 7f53f2f2b640 1 -- 172.21.15.19:0/1673779879 wait complete. 2024-04-25T10:04:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.380+0000 7f53f2f2b640 1 Processor -- start 2024-04-25T10:04:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.380+0000 7f53f2f2b640 1 -- start start 2024-04-25T10:04:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f2f2b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ec0aa400 0x7f53ec0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f2f2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53ec0aa7e0 0x7f53ec153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f2f2b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec0b20a0 0x7f53ec153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f2f2b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec070e10 con 0x7f53ec0aa400 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f2f2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0b48f0 con 0x7f53ec0aa7e0 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f2f2b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec070f90 con 0x7f53ec0b20a0 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f0cc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ec0aa400 0x7f53ec0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f0cc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ec0aa400 0x7f53ec0b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38098/0 (socket says 172.21.15.19:38098) 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f0cc7640 1 -- 172.21.15.19:0/904689555 learned_addr learned my addr 172.21.15.19:0/904689555 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53ebfff640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53ec0aa7e0 0x7f53ec153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f14c8640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec0b20a0 0x7f53ec153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f0cc7640 1 -- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec0b20a0 msgr2=0x7f53ec153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f0cc7640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec0b20a0 0x7f53ec153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f0cc7640 1 -- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53ec0aa7e0 msgr2=0x7f53ec153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f0cc7640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53ec0aa7e0 0x7f53ec153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f0cc7640 1 -- 172.21.15.19:0/904689555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec15a0f0 con 0x7f53ec0aa400 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f14c8640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec0b20a0 0x7f53ec153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.381+0000 7f53f0cc7640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ec0aa400 0x7f53ec0b1b60 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009f90 tx=0x7f53e400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.382+0000 7f53e97fa640 1 -- 172.21.15.19:0/904689555 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e400fea0 con 0x7f53ec0aa400 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.382+0000 7f53e97fa640 1 -- 172.21.15.19:0/904689555 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e4015630 con 0x7f53ec0aa400 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.382+0000 7f53f2f2b640 1 -- 172.21.15.19:0/904689555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec15a3d0 con 0x7f53ec0aa400 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.382+0000 7f53e97fa640 1 -- 172.21.15.19:0/904689555 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4015a20 con 0x7f53ec0aa400 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.382+0000 7f53f2f2b640 1 -- 172.21.15.19:0/904689555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec15a990 con 0x7f53ec0aa400 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.383+0000 7f53f2f2b640 1 -- 172.21.15.19:0/904689555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec073990 con 0x7f53ec0aa400 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.383+0000 7f53e97fa640 1 -- 172.21.15.19:0/904689555 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53e4015bc0 con 0x7f53ec0aa400 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.384+0000 7f53e97fa640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53d806b500 0x7f53d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.384+0000 7f53e97fa640 1 -- 172.21.15.19:0/904689555 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53e408b520 con 0x7f53ec0aa400 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.384+0000 7f53ebfff640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53d806b500 0x7f53d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.384+0000 7f53ebfff640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53d806b500 0x7f53d806d9c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0b4680 tx=0x7f53e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.388+0000 7f53e97fa640 1 -- 172.21.15.19:0/904689555 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53e4019030 con 0x7f53ec0aa400 2024-04-25T10:04:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.570+0000 7f53f2f2b640 1 -- 172.21.15.19:0/904689555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53ec0700e0 con 0x7f53ec0aa400 2024-04-25T10:04:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.571+0000 7f53e97fa640 1 -- 172.21.15.19:0/904689555 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f53e4010040 con 0x7f53ec0aa400 2024-04-25T10:04:17.577 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:17.577 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":165,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:04:15.886607+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:12.518429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:13.232769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:17.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 -- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53d806b500 msgr2=0x7f53d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53d806b500 0x7f53d806d9c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0b4680 tx=0x7f53e0009040 comp rx=0 tx=0).stop 2024-04-25T10:04:17.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 -- 172.21.15.19:0/904689555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ec0aa400 msgr2=0x7f53ec0b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ec0aa400 0x7f53ec0b1b60 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009f90 tx=0x7f53e400f580 comp rx=0 tx=0).stop 2024-04-25T10:04:17.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 -- 172.21.15.19:0/904689555 shutdown_connections 2024-04-25T10:04:17.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53d806b500 0x7f53d806d9c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53ec0b20a0 0x7f53ec153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53ec0aa7e0 0x7f53ec153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 --2- 172.21.15.19:0/904689555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53ec0aa400 0x7f53ec0b1b60 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 -- 172.21.15.19:0/904689555 >> 172.21.15.19:0/904689555 conn(0x7f53ec06f520 msgr2=0x7f53ec15e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 -- 172.21.15.19:0/904689555 shutdown_connections 2024-04-25T10:04:17.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.573+0000 7f53f2f2b640 1 -- 172.21.15.19:0/904689555 wait complete. 2024-04-25T10:04:17.590 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:04:17.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.685+0000 7f55276b3640 1 Processor -- start 2024-04-25T10:04:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.685+0000 7f55276b3640 1 -- start start 2024-04-25T10:04:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.686+0000 7f55276b3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f552015dbe0 0x7f55200d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.686+0000 7f55276b3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55200b6220 con 0x7f55200b2cb0 2024-04-25T10:04:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.686+0000 7f55276b3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55200b97a0 con 0x7f55200b92a0 2024-04-25T10:04:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.686+0000 7f55276b3640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552015dfc0 con 0x7f552015dbe0 2024-04-25T10:04:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.686+0000 7f552544f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f552015dbe0 0x7f55200d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.686+0000 7f5524c4e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f55200b2cb0 0x7f55200b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57548/0 (socket says 172.21.15.19:57548) 2024-04-25T10:04:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.686+0000 7f5524c4e640 1 -- 172.21.15.19:0/2771710397 learned_addr learned my addr 172.21.15.19:0/2771710397 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.686+0000 7f552544f640 1 -- 172.21.15.19:0/2771710397 >> v1:172.21.15.112:6789/0 conn(0x7f55200b92a0 legacy=0x7f55200b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.686+0000 7f552544f640 1 -- 172.21.15.19:0/2771710397 >> v1:172.21.15.19:6789/0 conn(0x7f55200b2cb0 legacy=0x7f55200b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.686+0000 7f5517fff640 1 -- 172.21.15.19:0/2771710397 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399433606 0 0) 0x7f55200b6220 con 0x7f55200b2cb0 2024-04-25T10:04:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.686+0000 7f552544f640 1 -- 172.21.15.19:0/2771710397 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55200d1c50 con 0x7f552015dbe0 2024-04-25T10:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.687+0000 7f552544f640 1 --2- 172.21.15.19:0/2771710397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f552015dbe0 0x7f55200d0910 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f551c0097a0 tx=0x7f551c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774c7be09f817302 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.687+0000 7f5517fff640 1 -- 172.21.15.19:0/2771710397 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c037070 con 0x7f552015dbe0 2024-04-25T10:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.687+0000 7f5517fff640 1 -- 172.21.15.19:0/2771710397 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c032440 con 0x7f552015dbe0 2024-04-25T10:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.687+0000 7f5517fff640 1 -- 172.21.15.19:0/2771710397 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c032810 con 0x7f552015dbe0 2024-04-25T10:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.687+0000 7f55276b3640 1 -- 172.21.15.19:0/2771710397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f552015dbe0 msgr2=0x7f55200d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.687+0000 7f55276b3640 1 --2- 172.21.15.19:0/2771710397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f552015dbe0 0x7f55200d0910 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f551c0097a0 tx=0x7f551c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.687+0000 7f55276b3640 1 -- 172.21.15.19:0/2771710397 shutdown_connections 2024-04-25T10:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.687+0000 7f55276b3640 1 --2- 172.21.15.19:0/2771710397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f552015dbe0 0x7f55200d0910 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.688+0000 7f55276b3640 1 -- 172.21.15.19:0/2771710397 >> 172.21.15.19:0/2771710397 conn(0x7f5520061a20 msgr2=0x7f5520061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.688+0000 7f55276b3640 1 -- 172.21.15.19:0/2771710397 shutdown_connections 2024-04-25T10:04:17.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.688+0000 7f55276b3640 1 -- 172.21.15.19:0/2771710397 wait complete. 2024-04-25T10:04:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.689+0000 7f55276b3640 1 Processor -- start 2024-04-25T10:04:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.689+0000 7f55276b3640 1 -- start start 2024-04-25T10:04:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.689+0000 7f55276b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55200b2cb0 0x7f55200caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.689+0000 7f55276b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55200b92a0 0x7f5520151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f55276b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f552015dbe0 0x7f552015e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f55276b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55200d2210 con 0x7f552015dbe0 2024-04-25T10:04:17.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f55276b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55200d1a00 con 0x7f55200b2cb0 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f55276b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55200b6ab0 con 0x7f55200b92a0 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5525c50640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f552015dbe0 0x7f552015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5525c50640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f552015dbe0 0x7f552015e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56924/0 (socket says 172.21.15.19:56924) 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5525c50640 1 -- 172.21.15.19:0/3840998766 learned_addr learned my addr 172.21.15.19:0/3840998766 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f552544f640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55200b2cb0 0x7f55200caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5524c4e640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55200b92a0 0x7f5520151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5525c50640 1 -- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55200b92a0 msgr2=0x7f5520151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5525c50640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55200b92a0 0x7f5520151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5525c50640 1 -- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55200b2cb0 msgr2=0x7f55200caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5525c50640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55200b2cb0 0x7f55200caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5525c50640 1 -- 172.21.15.19:0/3840998766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552015f0f0 con 0x7f552015dbe0 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5524c4e640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55200b92a0 0x7f5520151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:17.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5525c50640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f552015dbe0 0x7f552015e150 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f551800c730 tx=0x7f551800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5515ffb640 1 -- 172.21.15.19:0/3840998766 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518003b70 con 0x7f552015dbe0 2024-04-25T10:04:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5515ffb640 1 -- 172.21.15.19:0/3840998766 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5518003d10 con 0x7f552015dbe0 2024-04-25T10:04:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f55276b3640 1 -- 172.21.15.19:0/3840998766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552015eb30 con 0x7f552015dbe0 2024-04-25T10:04:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.690+0000 7f5515ffb640 1 -- 172.21.15.19:0/3840998766 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800aa90 con 0x7f552015dbe0 2024-04-25T10:04:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.691+0000 7f55276b3640 1 -- 172.21.15.19:0/3840998766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552015fad0 con 0x7f552015dbe0 2024-04-25T10:04:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.692+0000 7f55276b3640 1 -- 172.21.15.19:0/3840998766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5520073990 con 0x7f552015dbe0 2024-04-25T10:04:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.692+0000 7f5515ffb640 1 -- 172.21.15.19:0/3840998766 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f551800ac30 con 0x7f552015dbe0 2024-04-25T10:04:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.692+0000 7f5515ffb640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550406b4d0 0x7f550406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.692+0000 7f5515ffb640 1 -- 172.21.15.19:0/3840998766 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f551808a220 con 0x7f552015dbe0 2024-04-25T10:04:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.692+0000 7f552544f640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550406b4d0 0x7f550406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.693+0000 7f552544f640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550406b4d0 0x7f550406d990 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f551c034000 tx=0x7f551c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.697+0000 7f5515ffb640 1 -- 172.21.15.19:0/3840998766 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5518058fa0 con 0x7f552015dbe0 2024-04-25T10:04:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.873+0000 7f55276b3640 1 -- 172.21.15.19:0/3840998766 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5520061e00 con 0x7f550406b4d0 2024-04-25T10:04:17.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.880+0000 7f5515ffb640 1 -- 172.21.15.19:0/3840998766 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7f5520061e00 con 0x7f550406b4d0 2024-04-25T10:04:17.889 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:17.890 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:04:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.882+0000 7f55276b3640 1 -- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550406b4d0 msgr2=0x7f550406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.882+0000 7f55276b3640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550406b4d0 0x7f550406d990 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f551c034000 tx=0x7f551c02e000 comp rx=0 tx=0).stop 2024-04-25T10:04:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.882+0000 7f55276b3640 1 -- 172.21.15.19:0/3840998766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f552015dbe0 msgr2=0x7f552015e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.882+0000 7f55276b3640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f552015dbe0 0x7f552015e150 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f551800c730 tx=0x7f551800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.882+0000 7f55276b3640 1 -- 172.21.15.19:0/3840998766 shutdown_connections 2024-04-25T10:04:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.882+0000 7f55276b3640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f550406b4d0 0x7f550406d990 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.882+0000 7f55276b3640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f552015dbe0 0x7f552015e150 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.882+0000 7f55276b3640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55200b92a0 0x7f5520151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.882+0000 7f55276b3640 1 --2- 172.21.15.19:0/3840998766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55200b2cb0 0x7f55200caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.882+0000 7f55276b3640 1 -- 172.21.15.19:0/3840998766 >> 172.21.15.19:0/3840998766 conn(0x7f5520061a20 msgr2=0x7f552015b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.883+0000 7f55276b3640 1 -- 172.21.15.19:0/3840998766 shutdown_connections 2024-04-25T10:04:17.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:17.883+0000 7f55276b3640 1 -- 172.21.15.19:0/3840998766 wait complete. 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-04-25T10:04:17.068592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440128,"kb_used_data":17164,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754534592,"statfs":{"total":773094113280,"available":772643422208,"internally_reserved":0,"allocated":17575936,"data_stored":8844353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.184409"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.906917+0000","last_change":"2024-04-25T10:04:13.906917+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.906917+0000","last_undegraded":"2024-04-25T10:04:13.906917+0000","last_fullsized":"2024-04-25T10:04:13.906917+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.906400+0000","last_change":"2024-04-25T10:04:13.906400+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.906400+0000","last_undegraded":"2024-04-25T10:04:13.906400+0000","last_fullsized":"2024-04-25T10:04:13.906400+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894277+0000","last_change":"2024-04-25T10:04:13.894277+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894277+0000","last_undegraded":"2024-04-25T10:04:13.894277+0000","last_fullsized":"2024-04-25T10:04:13.894277+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.899956+0000","last_change":"2024-04-25T10:04:13.899956+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.899956+0000","last_undegraded":"2024-04-25T10:04:13.899956+0000","last_fullsized":"2024-04-25T10:04:13.899956+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.907015+0000","last_change":"2024-04-25T10:04:13.907015+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.907015+0000","last_undegraded":"2024-04-25T10:04:13.907015+0000","last_fullsized":"2024-04-25T10:04:13.907015+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.905958+0000","last_change":"2024-04-25T10:04:13.905958+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.905958+0000","last_undegraded":"2024-04-25T10:04:13.905958+0000","last_fullsized":"2024-04-25T10:04:13.905958+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898174+0000","last_change":"2024-04-25T10:04:13.898174+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898174+0000","last_undegraded":"2024-04-25T10:04:13.898174+0000","last_fullsized":"2024-04-25T10:04:13.898174+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888128+0000","last_change":"2024-04-25T10:04:13.888128+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888128+0000","last_undegraded":"2024-04-25T10:04:13.888128+0000","last_fullsized":"2024-04-25T10:04:13.888128+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.906338+0000","last_change":"2024-04-25T10:04:13.906338+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.906338+0000","last_undegraded":"2024-04-25T10:04:13.906338+0000","last_fullsized":"2024-04-25T10:04:13.906338+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904643+0000","last_change":"2024-04-25T10:04:13.904643+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904643+0000","last_undegraded":"2024-04-25T10:04:13.904643+0000","last_fullsized":"2024-04-25T10:04:13.904643+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894877+0000","last_change":"2024-04-25T10:04:13.894877+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894877+0000","last_undegraded":"2024-04-25T10:04:13.894877+0000","last_fullsized":"2024-04-25T10:04:13.894877+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897174+0000","last_change":"2024-04-25T10:04:13.897174+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897174+0000","last_undegraded":"2024-04-25T10:04:13.897174+0000","last_fullsized":"2024-04-25T10:04:13.897174+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.891816+0000","last_change":"2024-04-25T10:04:13.891816+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.891816+0000","last_undegraded":"2024-04-25T10:04:13.891816+0000","last_fullsized":"2024-04-25T10:04:13.891816+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897714+0000","last_change":"2024-04-25T10:04:13.897714+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897714+0000","last_undegraded":"2024-04-25T10:04:13.897714+0000","last_fullsized":"2024-04-25T10:04:13.897714+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904789+0000","last_change":"2024-04-25T10:04:13.904789+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904789+0000","last_undegraded":"2024-04-25T10:04:13.904789+0000","last_fullsized":"2024-04-25T10:04:13.904789+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903622+0000","last_change":"2024-04-25T10:04:13.903622+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903622+0000","last_undegraded":"2024-04-25T10:04:13.903622+0000","last_fullsized":"2024-04-25T10:04:13.903622+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898964+0000","last_change":"2024-04-25T10:04:13.898964+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898964+0000","last_undegraded":"2024-04-25T10:04:13.898964+0000","last_fullsized":"2024-04-25T10:04:13.898964+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.891283+0000","last_change":"2024-04-25T10:04:13.891283+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.891283+0000","last_undegraded":"2024-04-25T10:04:13.891283+0000","last_fullsized":"2024-04-25T10:04:13.891283+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.889985+0000","last_change":"2024-04-25T10:04:13.889985+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.889985+0000","last_undegraded":"2024-04-25T10:04:13.889985+0000","last_fullsized":"2024-04-25T10:04:13.889985+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903899+0000","last_change":"2024-04-25T10:04:13.903899+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903899+0000","last_undegraded":"2024-04-25T10:04:13.903899+0000","last_fullsized":"2024-04-25T10:04:13.903899+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904325+0000","last_change":"2024-04-25T10:04:13.904325+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904325+0000","last_undegraded":"2024-04-25T10:04:13.904325+0000","last_fullsized":"2024-04-25T10:04:13.904325+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904219+0000","last_change":"2024-04-25T10:04:13.904219+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904219+0000","last_undegraded":"2024-04-25T10:04:13.904219+0000","last_fullsized":"2024-04-25T10:04:13.904219+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888564+0000","last_change":"2024-04-25T10:04:13.888564+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888564+0000","last_undegraded":"2024-04-25T10:04:13.888564+0000","last_fullsized":"2024-04-25T10:04:13.888564+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903639+0000","last_change":"2024-04-25T10:04:13.903639+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903639+0000","last_undegraded":"2024-04-25T10:04:13.903639+0000","last_fullsized":"2024-04-25T10:04:13.903639+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898089+0000","last_change":"2024-04-25T10:04:13.898089+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898089+0000","last_undegraded":"2024-04-25T10:04:13.898089+0000","last_fullsized":"2024-04-25T10:04:13.898089+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896303+0000","last_change":"2024-04-25T10:04:13.896303+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896303+0000","last_undegraded":"2024-04-25T10:04:13.896303+0000","last_fullsized":"2024-04-25T10:04:13.896303+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888597+0000","last_change":"2024-04-25T10:04:13.888597+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888597+0000","last_undegraded":"2024-04-25T10:04:13.888597+0000","last_fullsized":"2024-04-25T10:04:13.888597+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.899739+0000","last_change":"2024-04-25T10:04:13.899739+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.899739+0000","last_undegraded":"2024-04-25T10:04:13.899739+0000","last_fullsized":"2024-04-25T10:04:13.899739+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894259+0000","last_change":"2024-04-25T10:04:13.894259+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894259+0000","last_undegraded":"2024-04-25T10:04:13.894259+0000","last_fullsized":"2024-04-25T10:04:13.894259+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.893981+0000","last_change":"2024-04-25T10:04:13.893981+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.893981+0000","last_undegraded":"2024-04-25T10:04:13.893981+0000","last_fullsized":"2024-04-25T10:04:13.893981+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897098+0000","last_change":"2024-04-25T10:04:13.897098+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897098+0000","last_undegraded":"2024-04-25T10:04:13.897098+0000","last_fullsized":"2024-04-25T10:04:13.897098+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895659+0000","last_change":"2024-04-25T10:04:13.895659+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895659+0000","last_undegraded":"2024-04-25T10:04:13.895659+0000","last_fullsized":"2024-04-25T10:04:13.895659+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888515+0000","last_change":"2024-04-25T10:04:13.888515+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888515+0000","last_undegraded":"2024-04-25T10:04:13.888515+0000","last_fullsized":"2024-04-25T10:04:13.888515+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.909538+0000","last_change":"2024-04-25T10:04:13.905310+0000","last_active":"2024-04-25T10:04:13.909538+0000","last_peered":"2024-04-25T10:04:13.909538+0000","last_clean":"2024-04-25T10:04:13.909538+0000","last_became_active":"2024-04-25T10:04:13.905060+0000","last_became_peered":"2024-04-25T10:04:13.905060+0000","last_unstale":"2024-04-25T10:04:13.909538+0000","last_undegraded":"2024-04-25T10:04:13.909538+0000","last_fullsized":"2024-04-25T10:04:13.909538+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:03.466270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897689+0000","last_change":"2024-04-25T10:04:13.897689+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897689+0000","last_undegraded":"2024-04-25T10:04:13.897689+0000","last_fullsized":"2024-04-25T10:04:13.897689+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913761+0000","last_change":"2024-04-25T10:04:13.911506+0000","last_active":"2024-04-25T10:04:13.913761+0000","last_peered":"2024-04-25T10:04:13.913761+0000","last_clean":"2024-04-25T10:04:13.913761+0000","last_became_active":"2024-04-25T10:04:13.909233+0000","last_became_peered":"2024-04-25T10:04:13.909233+0000","last_unstale":"2024-04-25T10:04:13.913761+0000","last_undegraded":"2024-04-25T10:04:13.913761+0000","last_fullsized":"2024-04-25T10:04:13.913761+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:44.444031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895862+0000","last_change":"2024-04-25T10:04:13.895862+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895862+0000","last_undegraded":"2024-04-25T10:04:13.895862+0000","last_fullsized":"2024-04-25T10:04:13.895862+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900428+0000","last_change":"2024-04-25T10:04:13.899686+0000","last_active":"2024-04-25T10:04:13.900428+0000","last_peered":"2024-04-25T10:04:13.900428+0000","last_clean":"2024-04-25T10:04:13.900428+0000","last_became_active":"2024-04-25T10:04:13.899387+0000","last_became_peered":"2024-04-25T10:04:13.899387+0000","last_unstale":"2024-04-25T10:04:13.900428+0000","last_undegraded":"2024-04-25T10:04:13.900428+0000","last_fullsized":"2024-04-25T10:04:13.900428+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.333329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.893994+0000","last_change":"2024-04-25T10:04:13.893994+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.893994+0000","last_undegraded":"2024-04-25T10:04:13.893994+0000","last_fullsized":"2024-04-25T10:04:13.893994+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900727+0000","last_change":"2024-04-25T10:04:13.899482+0000","last_active":"2024-04-25T10:04:13.900727+0000","last_peered":"2024-04-25T10:04:13.900727+0000","last_clean":"2024-04-25T10:04:13.900727+0000","last_became_active":"2024-04-25T10:04:13.899195+0000","last_became_peered":"2024-04-25T10:04:13.899195+0000","last_unstale":"2024-04-25T10:04:13.900727+0000","last_undegraded":"2024-04-25T10:04:13.900727+0000","last_fullsized":"2024-04-25T10:04:13.900727+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:42.064164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.889782+0000","last_change":"2024-04-25T10:04:13.889782+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.889782+0000","last_undegraded":"2024-04-25T10:04:13.889782+0000","last_fullsized":"2024-04-25T10:04:13.889782+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913248+0000","last_change":"2024-04-25T10:04:13.911021+0000","last_active":"2024-04-25T10:04:13.913248+0000","last_peered":"2024-04-25T10:04:13.913248+0000","last_clean":"2024-04-25T10:04:13.913248+0000","last_became_active":"2024-04-25T10:04:13.909391+0000","last_became_peered":"2024-04-25T10:04:13.909391+0000","last_unstale":"2024-04-25T10:04:13.913248+0000","last_undegraded":"2024-04-25T10:04:13.913248+0000","last_fullsized":"2024-04-25T10:04:13.913248+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:11.455636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.902584+0000","last_change":"2024-04-25T10:04:13.902584+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.902584+0000","last_undegraded":"2024-04-25T10:04:13.902584+0000","last_fullsized":"2024-04-25T10:04:13.902584+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.911557+0000","last_change":"2024-04-25T10:04:13.907057+0000","last_active":"2024-04-25T10:04:13.911557+0000","last_peered":"2024-04-25T10:04:13.911557+0000","last_clean":"2024-04-25T10:04:13.911557+0000","last_became_active":"2024-04-25T10:04:13.906778+0000","last_became_peered":"2024-04-25T10:04:13.906778+0000","last_unstale":"2024-04-25T10:04:13.911557+0000","last_undegraded":"2024-04-25T10:04:13.911557+0000","last_fullsized":"2024-04-25T10:04:13.911557+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:59.124388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903095+0000","last_change":"2024-04-25T10:04:13.903095+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903095+0000","last_undegraded":"2024-04-25T10:04:13.903095+0000","last_fullsized":"2024-04-25T10:04:13.903095+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.895543+0000","last_change":"2024-04-25T10:04:13.894824+0000","last_active":"2024-04-25T10:04:13.895543+0000","last_peered":"2024-04-25T10:04:13.895543+0000","last_clean":"2024-04-25T10:04:13.895543+0000","last_became_active":"2024-04-25T10:04:13.894557+0000","last_became_peered":"2024-04-25T10:04:13.894557+0000","last_unstale":"2024-04-25T10:04:13.895543+0000","last_undegraded":"2024-04-25T10:04:13.895543+0000","last_fullsized":"2024-04-25T10:04:13.895543+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:38.732229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.887704+0000","last_change":"2024-04-25T10:04:13.887704+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.887704+0000","last_undegraded":"2024-04-25T10:04:13.887704+0000","last_fullsized":"2024-04-25T10:04:13.887704+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913799+0000","last_change":"2024-04-25T10:04:13.912389+0000","last_active":"2024-04-25T10:04:13.913799+0000","last_peered":"2024-04-25T10:04:13.913799+0000","last_clean":"2024-04-25T10:04:13.913799+0000","last_became_active":"2024-04-25T10:04:13.912117+0000","last_became_peered":"2024-04-25T10:04:13.912117+0000","last_unstale":"2024-04-25T10:04:13.913799+0000","last_undegraded":"2024-04-25T10:04:13.913799+0000","last_fullsized":"2024-04-25T10:04:13.913799+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:31.979599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897391+0000","last_change":"2024-04-25T10:04:13.897391+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897391+0000","last_undegraded":"2024-04-25T10:04:13.897391+0000","last_fullsized":"2024-04-25T10:04:13.897391+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.909291+0000","last_change":"2024-04-25T10:04:13.908763+0000","last_active":"2024-04-25T10:04:13.909291+0000","last_peered":"2024-04-25T10:04:13.909291+0000","last_clean":"2024-04-25T10:04:13.909291+0000","last_became_active":"2024-04-25T10:04:13.908481+0000","last_became_peered":"2024-04-25T10:04:13.908481+0000","last_unstale":"2024-04-25T10:04:13.909291+0000","last_undegraded":"2024-04-25T10:04:13.909291+0000","last_fullsized":"2024-04-25T10:04:13.909291+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:48.385428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.890204+0000","last_change":"2024-04-25T10:04:13.890204+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.890204+0000","last_undegraded":"2024-04-25T10:04:13.890204+0000","last_fullsized":"2024-04-25T10:04:13.890204+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.912017+0000","last_change":"2024-04-25T10:04:13.909365+0000","last_active":"2024-04-25T10:04:13.912017+0000","last_peered":"2024-04-25T10:04:13.912017+0000","last_clean":"2024-04-25T10:04:13.912017+0000","last_became_active":"2024-04-25T10:04:13.907222+0000","last_became_peered":"2024-04-25T10:04:13.907222+0000","last_unstale":"2024-04-25T10:04:13.912017+0000","last_undegraded":"2024-04-25T10:04:13.912017+0000","last_fullsized":"2024-04-25T10:04:13.912017+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:14.655743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.902109+0000","last_change":"2024-04-25T10:04:13.902109+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.902109+0000","last_undegraded":"2024-04-25T10:04:13.902109+0000","last_fullsized":"2024-04-25T10:04:13.902109+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900704+0000","last_change":"2024-04-25T10:04:13.898686+0000","last_active":"2024-04-25T10:04:13.900704+0000","last_peered":"2024-04-25T10:04:13.900704+0000","last_clean":"2024-04-25T10:04:13.900704+0000","last_became_active":"2024-04-25T10:04:13.898413+0000","last_became_peered":"2024-04-25T10:04:13.898413+0000","last_unstale":"2024-04-25T10:04:13.900704+0000","last_undegraded":"2024-04-25T10:04:13.900704+0000","last_fullsized":"2024-04-25T10:04:13.900704+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:30.648919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896588+0000","last_change":"2024-04-25T10:04:13.896588+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896588+0000","last_undegraded":"2024-04-25T10:04:13.896588+0000","last_fullsized":"2024-04-25T10:04:13.896588+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899891+0000","last_change":"2024-04-25T10:04:13.897853+0000","last_active":"2024-04-25T10:04:13.899891+0000","last_peered":"2024-04-25T10:04:13.899891+0000","last_clean":"2024-04-25T10:04:13.899891+0000","last_became_active":"2024-04-25T10:04:13.897547+0000","last_became_peered":"2024-04-25T10:04:13.897547+0000","last_unstale":"2024-04-25T10:04:13.899891+0000","last_undegraded":"2024-04-25T10:04:13.899891+0000","last_fullsized":"2024-04-25T10:04:13.899891+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:32.020740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894510+0000","last_change":"2024-04-25T10:04:13.894510+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894510+0000","last_undegraded":"2024-04-25T10:04:13.894510+0000","last_fullsized":"2024-04-25T10:04:13.894510+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905135+0000","last_change":"2024-04-25T10:04:13.901634+0000","last_active":"2024-04-25T10:04:13.905135+0000","last_peered":"2024-04-25T10:04:13.905135+0000","last_clean":"2024-04-25T10:04:13.905135+0000","last_became_active":"2024-04-25T10:04:13.900799+0000","last_became_peered":"2024-04-25T10:04:13.900799+0000","last_unstale":"2024-04-25T10:04:13.905135+0000","last_undegraded":"2024-04-25T10:04:13.905135+0000","last_fullsized":"2024-04-25T10:04:13.905135+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:08.935454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.890080+0000","last_change":"2024-04-25T10:04:13.890080+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.890080+0000","last_undegraded":"2024-04-25T10:04:13.890080+0000","last_fullsized":"2024-04-25T10:04:13.890080+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900787+0000","last_change":"2024-04-25T10:04:13.898306+0000","last_active":"2024-04-25T10:04:13.900787+0000","last_peered":"2024-04-25T10:04:13.900787+0000","last_clean":"2024-04-25T10:04:13.900787+0000","last_became_active":"2024-04-25T10:04:13.898057+0000","last_became_peered":"2024-04-25T10:04:13.898057+0000","last_unstale":"2024-04-25T10:04:13.900787+0000","last_undegraded":"2024-04-25T10:04:13.900787+0000","last_fullsized":"2024-04-25T10:04:13.900787+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:53.024047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.890799+0000","last_change":"2024-04-25T10:04:13.890799+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.890799+0000","last_undegraded":"2024-04-25T10:04:13.890799+0000","last_fullsized":"2024-04-25T10:04:13.890799+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.902746+0000","last_change":"2024-04-25T10:04:13.902317+0000","last_active":"2024-04-25T10:04:13.902746+0000","last_peered":"2024-04-25T10:04:13.902746+0000","last_clean":"2024-04-25T10:04:13.902746+0000","last_became_active":"2024-04-25T10:04:13.902008+0000","last_became_peered":"2024-04-25T10:04:13.902008+0000","last_unstale":"2024-04-25T10:04:13.902746+0000","last_undegraded":"2024-04-25T10:04:13.902746+0000","last_fullsized":"2024-04-25T10:04:13.902746+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:15.960516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896084+0000","last_change":"2024-04-25T10:04:13.896084+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896084+0000","last_undegraded":"2024-04-25T10:04:13.896084+0000","last_fullsized":"2024-04-25T10:04:13.896084+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.902930+0000","last_change":"2024-04-25T10:04:13.902390+0000","last_active":"2024-04-25T10:04:13.902930+0000","last_peered":"2024-04-25T10:04:13.902930+0000","last_clean":"2024-04-25T10:04:13.902930+0000","last_became_active":"2024-04-25T10:04:13.902012+0000","last_became_peered":"2024-04-25T10:04:13.902012+0000","last_unstale":"2024-04-25T10:04:13.902930+0000","last_undegraded":"2024-04-25T10:04:13.902930+0000","last_fullsized":"2024-04-25T10:04:13.902930+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:04.306029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896982+0000","last_change":"2024-04-25T10:04:13.896982+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896982+0000","last_undegraded":"2024-04-25T10:04:13.896982+0000","last_fullsized":"2024-04-25T10:04:13.896982+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899990+0000","last_change":"2024-04-25T10:04:13.896683+0000","last_active":"2024-04-25T10:04:13.899990+0000","last_peered":"2024-04-25T10:04:13.899990+0000","last_clean":"2024-04-25T10:04:13.899990+0000","last_became_active":"2024-04-25T10:04:13.896386+0000","last_became_peered":"2024-04-25T10:04:13.896386+0000","last_unstale":"2024-04-25T10:04:13.899990+0000","last_undegraded":"2024-04-25T10:04:13.899990+0000","last_fullsized":"2024-04-25T10:04:13.899990+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:14.367579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895399+0000","last_change":"2024-04-25T10:04:13.895399+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895399+0000","last_undegraded":"2024-04-25T10:04:13.895399+0000","last_fullsized":"2024-04-25T10:04:13.895399+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.906438+0000","last_change":"2024-04-25T10:04:13.905850+0000","last_active":"2024-04-25T10:04:13.906438+0000","last_peered":"2024-04-25T10:04:13.906438+0000","last_clean":"2024-04-25T10:04:13.906438+0000","last_became_active":"2024-04-25T10:04:13.905565+0000","last_became_peered":"2024-04-25T10:04:13.905565+0000","last_unstale":"2024-04-25T10:04:13.906438+0000","last_undegraded":"2024-04-25T10:04:13.906438+0000","last_fullsized":"2024-04-25T10:04:13.906438+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:42.064164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.901364+0000","last_change":"2024-04-25T10:04:13.901364+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.901364+0000","last_undegraded":"2024-04-25T10:04:13.901364+0000","last_fullsized":"2024-04-25T10:04:13.901364+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900445+0000","last_change":"2024-04-25T10:04:13.899466+0000","last_active":"2024-04-25T10:04:13.900445+0000","last_peered":"2024-04-25T10:04:13.900445+0000","last_clean":"2024-04-25T10:04:13.900445+0000","last_became_active":"2024-04-25T10:04:13.899146+0000","last_became_peered":"2024-04-25T10:04:13.899146+0000","last_unstale":"2024-04-25T10:04:13.900445+0000","last_undegraded":"2024-04-25T10:04:13.900445+0000","last_fullsized":"2024-04-25T10:04:13.900445+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:43.518006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.900952+0000","last_change":"2024-04-25T10:04:13.900952+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.900952+0000","last_undegraded":"2024-04-25T10:04:13.900952+0000","last_fullsized":"2024-04-25T10:04:13.900952+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.908254+0000","last_change":"2024-04-25T10:04:13.907912+0000","last_active":"2024-04-25T10:04:13.908254+0000","last_peered":"2024-04-25T10:04:13.908254+0000","last_clean":"2024-04-25T10:04:13.908254+0000","last_became_active":"2024-04-25T10:04:13.907651+0000","last_became_peered":"2024-04-25T10:04:13.907651+0000","last_unstale":"2024-04-25T10:04:13.908254+0000","last_undegraded":"2024-04-25T10:04:13.908254+0000","last_fullsized":"2024-04-25T10:04:13.908254+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:27.691750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896135+0000","last_change":"2024-04-25T10:04:13.896135+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896135+0000","last_undegraded":"2024-04-25T10:04:13.896135+0000","last_fullsized":"2024-04-25T10:04:13.896135+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905973+0000","last_change":"2024-04-25T10:04:13.904679+0000","last_active":"2024-04-25T10:04:13.905973+0000","last_peered":"2024-04-25T10:04:13.905973+0000","last_clean":"2024-04-25T10:04:13.905973+0000","last_became_active":"2024-04-25T10:04:13.904426+0000","last_became_peered":"2024-04-25T10:04:13.904426+0000","last_unstale":"2024-04-25T10:04:13.905973+0000","last_undegraded":"2024-04-25T10:04:13.905973+0000","last_fullsized":"2024-04-25T10:04:13.905973+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:44.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896661+0000","last_change":"2024-04-25T10:04:13.896661+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896661+0000","last_undegraded":"2024-04-25T10:04:13.896661+0000","last_fullsized":"2024-04-25T10:04:13.896661+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.904794+0000","last_change":"2024-04-25T10:04:13.898826+0000","last_active":"2024-04-25T10:04:13.904794+0000","last_peered":"2024-04-25T10:04:13.904794+0000","last_clean":"2024-04-25T10:04:13.904794+0000","last_became_active":"2024-04-25T10:04:13.898462+0000","last_became_peered":"2024-04-25T10:04:13.898462+0000","last_unstale":"2024-04-25T10:04:13.904794+0000","last_undegraded":"2024-04-25T10:04:13.904794+0000","last_fullsized":"2024-04-25T10:04:13.904794+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:30.864286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898514+0000","last_change":"2024-04-25T10:04:13.898514+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898514+0000","last_undegraded":"2024-04-25T10:04:13.898514+0000","last_fullsized":"2024-04-25T10:04:13.898514+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899831+0000","last_change":"2024-04-25T10:04:13.899083+0000","last_active":"2024-04-25T10:04:13.899831+0000","last_peered":"2024-04-25T10:04:13.899831+0000","last_clean":"2024-04-25T10:04:13.899831+0000","last_became_active":"2024-04-25T10:04:13.898810+0000","last_became_peered":"2024-04-25T10:04:13.898810+0000","last_unstale":"2024-04-25T10:04:13.899831+0000","last_undegraded":"2024-04-25T10:04:13.899831+0000","last_fullsized":"2024-04-25T10:04:13.899831+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:30.648919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894391+0000","last_change":"2024-04-25T10:04:13.894391+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894391+0000","last_undegraded":"2024-04-25T10:04:13.894391+0000","last_fullsized":"2024-04-25T10:04:13.894391+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900693+0000","last_change":"2024-04-25T10:04:13.899508+0000","last_active":"2024-04-25T10:04:13.900693+0000","last_peered":"2024-04-25T10:04:13.900693+0000","last_clean":"2024-04-25T10:04:13.900693+0000","last_became_active":"2024-04-25T10:04:13.898618+0000","last_became_peered":"2024-04-25T10:04:13.898618+0000","last_unstale":"2024-04-25T10:04:13.900693+0000","last_undegraded":"2024-04-25T10:04:13.900693+0000","last_fullsized":"2024-04-25T10:04:13.900693+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:37.888496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.900372+0000","last_change":"2024-04-25T10:04:13.900372+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.900372+0000","last_undegraded":"2024-04-25T10:04:13.900372+0000","last_fullsized":"2024-04-25T10:04:13.900372+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899333+0000","last_change":"2024-04-25T10:04:13.895684+0000","last_active":"2024-04-25T10:04:13.899333+0000","last_peered":"2024-04-25T10:04:13.899333+0000","last_clean":"2024-04-25T10:04:13.899333+0000","last_became_active":"2024-04-25T10:04:13.895390+0000","last_became_peered":"2024-04-25T10:04:13.895390+0000","last_unstale":"2024-04-25T10:04:13.899333+0000","last_undegraded":"2024-04-25T10:04:13.899333+0000","last_fullsized":"2024-04-25T10:04:13.899333+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:18.467823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888644+0000","last_change":"2024-04-25T10:04:13.888644+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888644+0000","last_undegraded":"2024-04-25T10:04:13.888644+0000","last_fullsized":"2024-04-25T10:04:13.888644+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913155+0000","last_change":"2024-04-25T10:04:13.911061+0000","last_active":"2024-04-25T10:04:13.913155+0000","last_peered":"2024-04-25T10:04:13.913155+0000","last_clean":"2024-04-25T10:04:13.913155+0000","last_became_active":"2024-04-25T10:04:13.910591+0000","last_became_peered":"2024-04-25T10:04:13.910591+0000","last_unstale":"2024-04-25T10:04:13.913155+0000","last_undegraded":"2024-04-25T10:04:13.913155+0000","last_fullsized":"2024-04-25T10:04:13.913155+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:40.288371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895546+0000","last_change":"2024-04-25T10:04:13.895546+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895546+0000","last_undegraded":"2024-04-25T10:04:13.895546+0000","last_fullsized":"2024-04-25T10:04:13.895546+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913837+0000","last_change":"2024-04-25T10:04:13.913137+0000","last_active":"2024-04-25T10:04:13.913837+0000","last_peered":"2024-04-25T10:04:13.913837+0000","last_clean":"2024-04-25T10:04:13.913837+0000","last_became_active":"2024-04-25T10:04:13.912835+0000","last_became_peered":"2024-04-25T10:04:13.912835+0000","last_unstale":"2024-04-25T10:04:13.913837+0000","last_undegraded":"2024-04-25T10:04:13.913837+0000","last_fullsized":"2024-04-25T10:04:13.913837+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:25.196171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895070+0000","last_change":"2024-04-25T10:04:13.895070+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895070+0000","last_undegraded":"2024-04-25T10:04:13.895070+0000","last_fullsized":"2024-04-25T10:04:13.895070+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.897923+0000","last_change":"2024-04-25T10:04:13.896282+0000","last_active":"2024-04-25T10:04:13.897923+0000","last_peered":"2024-04-25T10:04:13.897923+0000","last_clean":"2024-04-25T10:04:13.897923+0000","last_became_active":"2024-04-25T10:04:13.894939+0000","last_became_peered":"2024-04-25T10:04:13.894939+0000","last_unstale":"2024-04-25T10:04:13.897923+0000","last_undegraded":"2024-04-25T10:04:13.897923+0000","last_fullsized":"2024-04-25T10:04:13.897923+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:17.612850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.889558+0000","last_change":"2024-04-25T10:04:13.889558+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.889558+0000","last_undegraded":"2024-04-25T10:04:13.889558+0000","last_fullsized":"2024-04-25T10:04:13.889558+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.910509+0000","last_change":"2024-04-25T10:04:13.910016+0000","last_active":"2024-04-25T10:04:13.910509+0000","last_peered":"2024-04-25T10:04:13.910509+0000","last_clean":"2024-04-25T10:04:13.910509+0000","last_became_active":"2024-04-25T10:04:13.909591+0000","last_became_peered":"2024-04-25T10:04:13.909591+0000","last_unstale":"2024-04-25T10:04:13.910509+0000","last_undegraded":"2024-04-25T10:04:13.910509+0000","last_fullsized":"2024-04-25T10:04:13.910509+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:31.416889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.901336+0000","last_change":"2024-04-25T10:04:13.901336+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.901336+0000","last_undegraded":"2024-04-25T10:04:13.901336+0000","last_fullsized":"2024-04-25T10:04:13.901336+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905824+0000","last_change":"2024-04-25T10:04:13.899980+0000","last_active":"2024-04-25T10:04:13.905824+0000","last_peered":"2024-04-25T10:04:13.905824+0000","last_clean":"2024-04-25T10:04:13.905824+0000","last_became_active":"2024-04-25T10:04:13.899671+0000","last_became_peered":"2024-04-25T10:04:13.899671+0000","last_unstale":"2024-04-25T10:04:13.905824+0000","last_undegraded":"2024-04-25T10:04:13.905824+0000","last_fullsized":"2024-04-25T10:04:13.905824+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:22.542468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888539+0000","last_change":"2024-04-25T10:04:13.888539+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888539+0000","last_undegraded":"2024-04-25T10:04:13.888539+0000","last_fullsized":"2024-04-25T10:04:13.888539+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.910748+0000","last_change":"2024-04-25T10:04:13.910423+0000","last_active":"2024-04-25T10:04:13.910748+0000","last_peered":"2024-04-25T10:04:13.910748+0000","last_clean":"2024-04-25T10:04:13.910748+0000","last_became_active":"2024-04-25T10:04:13.910113+0000","last_became_peered":"2024-04-25T10:04:13.910113+0000","last_unstale":"2024-04-25T10:04:13.910748+0000","last_undegraded":"2024-04-25T10:04:13.910748+0000","last_fullsized":"2024-04-25T10:04:13.910748+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:27.691750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.893539+0000","last_change":"2024-04-25T10:04:13.893539+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.893539+0000","last_undegraded":"2024-04-25T10:04:13.893539+0000","last_fullsized":"2024-04-25T10:04:13.893539+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905911+0000","last_change":"2024-04-25T10:04:13.903829+0000","last_active":"2024-04-25T10:04:13.905911+0000","last_peered":"2024-04-25T10:04:13.905911+0000","last_clean":"2024-04-25T10:04:13.905911+0000","last_became_active":"2024-04-25T10:04:13.901705+0000","last_became_peered":"2024-04-25T10:04:13.901705+0000","last_unstale":"2024-04-25T10:04:13.905911+0000","last_undegraded":"2024-04-25T10:04:13.905911+0000","last_fullsized":"2024-04-25T10:04:13.905911+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:17.450538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":461,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.888932+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:04:13.888932+0000","last_peered":"2024-04-25T10:04:13.888932+0000","last_clean":"2024-04-25T10:04:13.888932+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:04:13.888932+0000","last_undegraded":"2024-04-25T10:04:13.888932+0000","last_fullsized":"2024-04-25T10:04:13.888932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51820,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320020,"statfs":{"total":96636764160,"available":96583700480,"internally_reserved":0,"allocated":2650112,"data_stored":1552487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574988,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51224,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320616,"statfs":{"total":96636764160,"available":96584310784,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51236,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320604,"statfs":{"total":96636764160,"available":96584298496,"internally_reserved":0,"allocated":2056192,"data_stored":962119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59560,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312280,"statfs":{"total":96636764160,"available":96575774720,"internally_reserved":0,"allocated":2056192,"data_stored":962119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59544,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312296,"statfs":{"total":96636764160,"available":96575791104,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55392,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316448,"statfs":{"total":96636764160,"available":96580042752,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55388,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316452,"statfs":{"total":96636764160,"available":96580046848,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55964,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315876,"statfs":{"total":96636764160,"available":96579457024,"internally_reserved":0,"allocated":2637824,"data_stored":1545820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:17.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.034+0000 7fe45531d640 1 Processor -- start 2024-04-25T10:04:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.034+0000 7fe45531d640 1 -- start start 2024-04-25T10:04:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.035+0000 7fe45531d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4500cfec0 0x7fe4500cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.035+0000 7fe45531d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 0x7fe45006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.035+0000 7fe45531d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe450071480 con 0x7fe450154ee0 2024-04-25T10:04:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.035+0000 7fe45531d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4500ce830 con 0x7fe4500cfec0 2024-04-25T10:04:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.035+0000 7fe45531d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe450072090 con 0x7fe4500ce3c0 2024-04-25T10:04:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.036+0000 7fe44f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 0x7fe45006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.036+0000 7fe44e7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4500cfec0 0x7fe4500cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.036+0000 7fe44f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 0x7fe45006d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:35846/0 (socket says 172.21.15.19:35846) 2024-04-25T10:04:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.036+0000 7fe44f7fe640 1 -- 172.21.15.19:0/1271326971 learned_addr learned my addr 172.21.15.19:0/1271326971 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.036+0000 7fe44f7fe640 1 -- 172.21.15.19:0/1271326971 >> v2:172.21.15.112:3300/0 conn(0x7fe4500cfec0 msgr2=0x7fe4500cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.036+0000 7fe44f7fe640 1 --2- 172.21.15.19:0/1271326971 >> v2:172.21.15.112:3300/0 conn(0x7fe4500cfec0 0x7fe4500cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.036+0000 7fe44f7fe640 1 -- 172.21.15.19:0/1271326971 >> v1:172.21.15.112:6789/0 conn(0x7fe450154ee0 legacy=0x7fe4500cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.036+0000 7fe44f7fe640 1 -- 172.21.15.19:0/1271326971 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe450071760 con 0x7fe4500ce3c0 2024-04-25T10:04:18.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.037+0000 7fe44f7fe640 1 --2- 172.21.15.19:0/1271326971 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 0x7fe45006d6f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe44000d150 tx=0x7fe44002c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde77cb159bbafcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.037+0000 7fe44dffb640 1 -- 172.21.15.19:0/1271326971 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440009600 con 0x7fe4500ce3c0 2024-04-25T10:04:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.037+0000 7fe44dffb640 1 -- 172.21.15.19:0/1271326971 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440037070 con 0x7fe4500ce3c0 2024-04-25T10:04:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.037+0000 7fe44dffb640 1 -- 172.21.15.19:0/1271326971 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4400099c0 con 0x7fe4500ce3c0 2024-04-25T10:04:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.037+0000 7fe45531d640 1 -- 172.21.15.19:0/1271326971 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 msgr2=0x7fe45006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.037+0000 7fe45531d640 1 --2- 172.21.15.19:0/1271326971 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 0x7fe45006d6f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe44000d150 tx=0x7fe44002c930 comp rx=0 tx=0).stop 2024-04-25T10:04:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.038+0000 7fe45531d640 1 -- 172.21.15.19:0/1271326971 shutdown_connections 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.038+0000 7fe45531d640 1 --2- 172.21.15.19:0/1271326971 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 0x7fe45006d6f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.038+0000 7fe45531d640 1 --2- 172.21.15.19:0/1271326971 >> v2:172.21.15.112:3300/0 conn(0x7fe4500cfec0 0x7fe4500cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.038+0000 7fe45531d640 1 -- 172.21.15.19:0/1271326971 >> 172.21.15.19:0/1271326971 conn(0x7fe45006abe0 msgr2=0x7fe45006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.038+0000 7fe45531d640 1 -- 172.21.15.19:0/1271326971 shutdown_connections 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.038+0000 7fe45531d640 1 -- 172.21.15.19:0/1271326971 wait complete. 2024-04-25T10:04:18.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.039+0000 7fe45531d640 1 Processor -- start 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.039+0000 7fe45531d640 1 -- start start 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe45531d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 0x7fe450151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe45531d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4500cfec0 0x7fe450158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe45531d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe450154ee0 0x7fe450158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe45531d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe450071d20 con 0x7fe450154ee0 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe45531d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe450071230 con 0x7fe4500cfec0 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe45531d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4501552c0 con 0x7fe4500ce3c0 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe450154ee0 0x7fe450158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe450154ee0 0x7fe450158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56936/0 (socket says 172.21.15.19:56936) 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44f7fe640 1 -- 172.21.15.19:0/56865414 learned_addr learned my addr 172.21.15.19:0/56865414 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44e7fc640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4500cfec0 0x7fe450158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44effd640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 0x7fe450151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44f7fe640 1 -- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 msgr2=0x7fe450151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44f7fe640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 0x7fe450151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44f7fe640 1 -- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4500cfec0 msgr2=0x7fe450158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44f7fe640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4500cfec0 0x7fe450158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44f7fe640 1 -- 172.21.15.19:0/56865414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4501573a0 con 0x7fe450154ee0 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44e7fc640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4500cfec0 0x7fe450158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.040+0000 7fe44f7fe640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe450154ee0 0x7fe450158750 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fe440034040 tx=0x7fe440004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.041+0000 7fe4377fe640 1 -- 172.21.15.19:0/56865414 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440009e40 con 0x7fe450154ee0 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.041+0000 7fe4377fe640 1 -- 172.21.15.19:0/56865414 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440047480 con 0x7fe450154ee0 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.041+0000 7fe45531d640 1 -- 172.21.15.19:0/56865414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4501e4d80 con 0x7fe450154ee0 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.041+0000 7fe4377fe640 1 -- 172.21.15.19:0/56865414 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440047850 con 0x7fe450154ee0 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.041+0000 7fe45531d640 1 -- 172.21.15.19:0/56865414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe450152650 con 0x7fe450154ee0 2024-04-25T10:04:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.042+0000 7fe45531d640 1 -- 172.21.15.19:0/56865414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe450073990 con 0x7fe450154ee0 2024-04-25T10:04:18.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.042+0000 7fe4377fe640 1 -- 172.21.15.19:0/56865414 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe440032420 con 0x7fe450154ee0 2024-04-25T10:04:18.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.043+0000 7fe4377fe640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe43c06b4d0 0x7fe43c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.043+0000 7fe4377fe640 1 -- 172.21.15.19:0/56865414 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe4400ad580 con 0x7fe450154ee0 2024-04-25T10:04:18.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.043+0000 7fe44effd640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe43c06b4d0 0x7fe43c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.043+0000 7fe44effd640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe43c06b4d0 0x7fe43c06d990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe448007fd0 tx=0x7fe448009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.047+0000 7fe4377fe640 1 -- 172.21.15.19:0/56865414 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe44007c280 con 0x7fe450154ee0 2024-04-25T10:04:18.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.224+0000 7fe45531d640 1 -- 172.21.15.19:0/56865414 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe450151d60 con 0x7fe43c06b4d0 2024-04-25T10:04:18.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.230+0000 7fe4377fe640 1 -- 172.21.15.19:0/56865414 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7fe450151d60 con 0x7fe43c06b4d0 2024-04-25T10:04:18.239 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 -- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe43c06b4d0 msgr2=0x7fe43c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe43c06b4d0 0x7fe43c06d990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe448007fd0 tx=0x7fe448009040 comp rx=0 tx=0).stop 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 -- 172.21.15.19:0/56865414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe450154ee0 msgr2=0x7fe450158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe450154ee0 0x7fe450158750 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fe440034040 tx=0x7fe440004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 -- 172.21.15.19:0/56865414 shutdown_connections 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe43c06b4d0 0x7fe43c06d990 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe450154ee0 0x7fe450158750 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4500cfec0 0x7fe450158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 --2- 172.21.15.19:0/56865414 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4500ce3c0 0x7fe450151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 -- 172.21.15.19:0/56865414 >> 172.21.15.19:0/56865414 conn(0x7fe45006abe0 msgr2=0x7fe45006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 -- 172.21.15.19:0/56865414 shutdown_connections 2024-04-25T10:04:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.233+0000 7fe45531d640 1 -- 172.21.15.19:0/56865414 wait complete. 2024-04-25T10:04:18.250 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-04-25T10:04:17.068592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440128,"kb_used_data":17164,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754534592,"statfs":{"total":773094113280,"available":772643422208,"internally_reserved":0,"allocated":17575936,"data_stored":8844353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.184409"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.906917+0000","last_change":"2024-04-25T10:04:13.906917+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.906917+0000","last_undegraded":"2024-04-25T10:04:13.906917+0000","last_fullsized":"2024-04-25T10:04:13.906917+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.906400+0000","last_change":"2024-04-25T10:04:13.906400+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.906400+0000","last_undegraded":"2024-04-25T10:04:13.906400+0000","last_fullsized":"2024-04-25T10:04:13.906400+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894277+0000","last_change":"2024-04-25T10:04:13.894277+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894277+0000","last_undegraded":"2024-04-25T10:04:13.894277+0000","last_fullsized":"2024-04-25T10:04:13.894277+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.899956+0000","last_change":"2024-04-25T10:04:13.899956+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.899956+0000","last_undegraded":"2024-04-25T10:04:13.899956+0000","last_fullsized":"2024-04-25T10:04:13.899956+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.907015+0000","last_change":"2024-04-25T10:04:13.907015+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.907015+0000","last_undegraded":"2024-04-25T10:04:13.907015+0000","last_fullsized":"2024-04-25T10:04:13.907015+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.905958+0000","last_change":"2024-04-25T10:04:13.905958+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.905958+0000","last_undegraded":"2024-04-25T10:04:13.905958+0000","last_fullsized":"2024-04-25T10:04:13.905958+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898174+0000","last_change":"2024-04-25T10:04:13.898174+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898174+0000","last_undegraded":"2024-04-25T10:04:13.898174+0000","last_fullsized":"2024-04-25T10:04:13.898174+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888128+0000","last_change":"2024-04-25T10:04:13.888128+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888128+0000","last_undegraded":"2024-04-25T10:04:13.888128+0000","last_fullsized":"2024-04-25T10:04:13.888128+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.906338+0000","last_change":"2024-04-25T10:04:13.906338+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.906338+0000","last_undegraded":"2024-04-25T10:04:13.906338+0000","last_fullsized":"2024-04-25T10:04:13.906338+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904643+0000","last_change":"2024-04-25T10:04:13.904643+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904643+0000","last_undegraded":"2024-04-25T10:04:13.904643+0000","last_fullsized":"2024-04-25T10:04:13.904643+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894877+0000","last_change":"2024-04-25T10:04:13.894877+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894877+0000","last_undegraded":"2024-04-25T10:04:13.894877+0000","last_fullsized":"2024-04-25T10:04:13.894877+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897174+0000","last_change":"2024-04-25T10:04:13.897174+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897174+0000","last_undegraded":"2024-04-25T10:04:13.897174+0000","last_fullsized":"2024-04-25T10:04:13.897174+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.891816+0000","last_change":"2024-04-25T10:04:13.891816+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.891816+0000","last_undegraded":"2024-04-25T10:04:13.891816+0000","last_fullsized":"2024-04-25T10:04:13.891816+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897714+0000","last_change":"2024-04-25T10:04:13.897714+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897714+0000","last_undegraded":"2024-04-25T10:04:13.897714+0000","last_fullsized":"2024-04-25T10:04:13.897714+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904789+0000","last_change":"2024-04-25T10:04:13.904789+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904789+0000","last_undegraded":"2024-04-25T10:04:13.904789+0000","last_fullsized":"2024-04-25T10:04:13.904789+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903622+0000","last_change":"2024-04-25T10:04:13.903622+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903622+0000","last_undegraded":"2024-04-25T10:04:13.903622+0000","last_fullsized":"2024-04-25T10:04:13.903622+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898964+0000","last_change":"2024-04-25T10:04:13.898964+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898964+0000","last_undegraded":"2024-04-25T10:04:13.898964+0000","last_fullsized":"2024-04-25T10:04:13.898964+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.891283+0000","last_change":"2024-04-25T10:04:13.891283+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.891283+0000","last_undegraded":"2024-04-25T10:04:13.891283+0000","last_fullsized":"2024-04-25T10:04:13.891283+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.889985+0000","last_change":"2024-04-25T10:04:13.889985+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.889985+0000","last_undegraded":"2024-04-25T10:04:13.889985+0000","last_fullsized":"2024-04-25T10:04:13.889985+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903899+0000","last_change":"2024-04-25T10:04:13.903899+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903899+0000","last_undegraded":"2024-04-25T10:04:13.903899+0000","last_fullsized":"2024-04-25T10:04:13.903899+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904325+0000","last_change":"2024-04-25T10:04:13.904325+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904325+0000","last_undegraded":"2024-04-25T10:04:13.904325+0000","last_fullsized":"2024-04-25T10:04:13.904325+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904219+0000","last_change":"2024-04-25T10:04:13.904219+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904219+0000","last_undegraded":"2024-04-25T10:04:13.904219+0000","last_fullsized":"2024-04-25T10:04:13.904219+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888564+0000","last_change":"2024-04-25T10:04:13.888564+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888564+0000","last_undegraded":"2024-04-25T10:04:13.888564+0000","last_fullsized":"2024-04-25T10:04:13.888564+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903639+0000","last_change":"2024-04-25T10:04:13.903639+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903639+0000","last_undegraded":"2024-04-25T10:04:13.903639+0000","last_fullsized":"2024-04-25T10:04:13.903639+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898089+0000","last_change":"2024-04-25T10:04:13.898089+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898089+0000","last_undegraded":"2024-04-25T10:04:13.898089+0000","last_fullsized":"2024-04-25T10:04:13.898089+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896303+0000","last_change":"2024-04-25T10:04:13.896303+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896303+0000","last_undegraded":"2024-04-25T10:04:13.896303+0000","last_fullsized":"2024-04-25T10:04:13.896303+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888597+0000","last_change":"2024-04-25T10:04:13.888597+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888597+0000","last_undegraded":"2024-04-25T10:04:13.888597+0000","last_fullsized":"2024-04-25T10:04:13.888597+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.899739+0000","last_change":"2024-04-25T10:04:13.899739+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.899739+0000","last_undegraded":"2024-04-25T10:04:13.899739+0000","last_fullsized":"2024-04-25T10:04:13.899739+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894259+0000","last_change":"2024-04-25T10:04:13.894259+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894259+0000","last_undegraded":"2024-04-25T10:04:13.894259+0000","last_fullsized":"2024-04-25T10:04:13.894259+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.893981+0000","last_change":"2024-04-25T10:04:13.893981+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.893981+0000","last_undegraded":"2024-04-25T10:04:13.893981+0000","last_fullsized":"2024-04-25T10:04:13.893981+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897098+0000","last_change":"2024-04-25T10:04:13.897098+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897098+0000","last_undegraded":"2024-04-25T10:04:13.897098+0000","last_fullsized":"2024-04-25T10:04:13.897098+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895659+0000","last_change":"2024-04-25T10:04:13.895659+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895659+0000","last_undegraded":"2024-04-25T10:04:13.895659+0000","last_fullsized":"2024-04-25T10:04:13.895659+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888515+0000","last_change":"2024-04-25T10:04:13.888515+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888515+0000","last_undegraded":"2024-04-25T10:04:13.888515+0000","last_fullsized":"2024-04-25T10:04:13.888515+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.909538+0000","last_change":"2024-04-25T10:04:13.905310+0000","last_active":"2024-04-25T10:04:13.909538+0000","last_peered":"2024-04-25T10:04:13.909538+0000","last_clean":"2024-04-25T10:04:13.909538+0000","last_became_active":"2024-04-25T10:04:13.905060+0000","last_became_peered":"2024-04-25T10:04:13.905060+0000","last_unstale":"2024-04-25T10:04:13.909538+0000","last_undegraded":"2024-04-25T10:04:13.909538+0000","last_fullsized":"2024-04-25T10:04:13.909538+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:03.466270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897689+0000","last_change":"2024-04-25T10:04:13.897689+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897689+0000","last_undegraded":"2024-04-25T10:04:13.897689+0000","last_fullsized":"2024-04-25T10:04:13.897689+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913761+0000","last_change":"2024-04-25T10:04:13.911506+0000","last_active":"2024-04-25T10:04:13.913761+0000","last_peered":"2024-04-25T10:04:13.913761+0000","last_clean":"2024-04-25T10:04:13.913761+0000","last_became_active":"2024-04-25T10:04:13.909233+0000","last_became_peered":"2024-04-25T10:04:13.909233+0000","last_unstale":"2024-04-25T10:04:13.913761+0000","last_undegraded":"2024-04-25T10:04:13.913761+0000","last_fullsized":"2024-04-25T10:04:13.913761+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:44.444031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895862+0000","last_change":"2024-04-25T10:04:13.895862+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895862+0000","last_undegraded":"2024-04-25T10:04:13.895862+0000","last_fullsized":"2024-04-25T10:04:13.895862+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900428+0000","last_change":"2024-04-25T10:04:13.899686+0000","last_active":"2024-04-25T10:04:13.900428+0000","last_peered":"2024-04-25T10:04:13.900428+0000","last_clean":"2024-04-25T10:04:13.900428+0000","last_became_active":"2024-04-25T10:04:13.899387+0000","last_became_peered":"2024-04-25T10:04:13.899387+0000","last_unstale":"2024-04-25T10:04:13.900428+0000","last_undegraded":"2024-04-25T10:04:13.900428+0000","last_fullsized":"2024-04-25T10:04:13.900428+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.333329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.893994+0000","last_change":"2024-04-25T10:04:13.893994+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.893994+0000","last_undegraded":"2024-04-25T10:04:13.893994+0000","last_fullsized":"2024-04-25T10:04:13.893994+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900727+0000","last_change":"2024-04-25T10:04:13.899482+0000","last_active":"2024-04-25T10:04:13.900727+0000","last_peered":"2024-04-25T10:04:13.900727+0000","last_clean":"2024-04-25T10:04:13.900727+0000","last_became_active":"2024-04-25T10:04:13.899195+0000","last_became_peered":"2024-04-25T10:04:13.899195+0000","last_unstale":"2024-04-25T10:04:13.900727+0000","last_undegraded":"2024-04-25T10:04:13.900727+0000","last_fullsized":"2024-04-25T10:04:13.900727+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:42.064164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.889782+0000","last_change":"2024-04-25T10:04:13.889782+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.889782+0000","last_undegraded":"2024-04-25T10:04:13.889782+0000","last_fullsized":"2024-04-25T10:04:13.889782+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913248+0000","last_change":"2024-04-25T10:04:13.911021+0000","last_active":"2024-04-25T10:04:13.913248+0000","last_peered":"2024-04-25T10:04:13.913248+0000","last_clean":"2024-04-25T10:04:13.913248+0000","last_became_active":"2024-04-25T10:04:13.909391+0000","last_became_peered":"2024-04-25T10:04:13.909391+0000","last_unstale":"2024-04-25T10:04:13.913248+0000","last_undegraded":"2024-04-25T10:04:13.913248+0000","last_fullsized":"2024-04-25T10:04:13.913248+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:11.455636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.902584+0000","last_change":"2024-04-25T10:04:13.902584+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.902584+0000","last_undegraded":"2024-04-25T10:04:13.902584+0000","last_fullsized":"2024-04-25T10:04:13.902584+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.911557+0000","last_change":"2024-04-25T10:04:13.907057+0000","last_active":"2024-04-25T10:04:13.911557+0000","last_peered":"2024-04-25T10:04:13.911557+0000","last_clean":"2024-04-25T10:04:13.911557+0000","last_became_active":"2024-04-25T10:04:13.906778+0000","last_became_peered":"2024-04-25T10:04:13.906778+0000","last_unstale":"2024-04-25T10:04:13.911557+0000","last_undegraded":"2024-04-25T10:04:13.911557+0000","last_fullsized":"2024-04-25T10:04:13.911557+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:59.124388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903095+0000","last_change":"2024-04-25T10:04:13.903095+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903095+0000","last_undegraded":"2024-04-25T10:04:13.903095+0000","last_fullsized":"2024-04-25T10:04:13.903095+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.895543+0000","last_change":"2024-04-25T10:04:13.894824+0000","last_active":"2024-04-25T10:04:13.895543+0000","last_peered":"2024-04-25T10:04:13.895543+0000","last_clean":"2024-04-25T10:04:13.895543+0000","last_became_active":"2024-04-25T10:04:13.894557+0000","last_became_peered":"2024-04-25T10:04:13.894557+0000","last_unstale":"2024-04-25T10:04:13.895543+0000","last_undegraded":"2024-04-25T10:04:13.895543+0000","last_fullsized":"2024-04-25T10:04:13.895543+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:38.732229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.887704+0000","last_change":"2024-04-25T10:04:13.887704+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.887704+0000","last_undegraded":"2024-04-25T10:04:13.887704+0000","last_fullsized":"2024-04-25T10:04:13.887704+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913799+0000","last_change":"2024-04-25T10:04:13.912389+0000","last_active":"2024-04-25T10:04:13.913799+0000","last_peered":"2024-04-25T10:04:13.913799+0000","last_clean":"2024-04-25T10:04:13.913799+0000","last_became_active":"2024-04-25T10:04:13.912117+0000","last_became_peered":"2024-04-25T10:04:13.912117+0000","last_unstale":"2024-04-25T10:04:13.913799+0000","last_undegraded":"2024-04-25T10:04:13.913799+0000","last_fullsized":"2024-04-25T10:04:13.913799+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:31.979599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897391+0000","last_change":"2024-04-25T10:04:13.897391+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897391+0000","last_undegraded":"2024-04-25T10:04:13.897391+0000","last_fullsized":"2024-04-25T10:04:13.897391+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.909291+0000","last_change":"2024-04-25T10:04:13.908763+0000","last_active":"2024-04-25T10:04:13.909291+0000","last_peered":"2024-04-25T10:04:13.909291+0000","last_clean":"2024-04-25T10:04:13.909291+0000","last_became_active":"2024-04-25T10:04:13.908481+0000","last_became_peered":"2024-04-25T10:04:13.908481+0000","last_unstale":"2024-04-25T10:04:13.909291+0000","last_undegraded":"2024-04-25T10:04:13.909291+0000","last_fullsized":"2024-04-25T10:04:13.909291+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:48.385428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.890204+0000","last_change":"2024-04-25T10:04:13.890204+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.890204+0000","last_undegraded":"2024-04-25T10:04:13.890204+0000","last_fullsized":"2024-04-25T10:04:13.890204+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.912017+0000","last_change":"2024-04-25T10:04:13.909365+0000","last_active":"2024-04-25T10:04:13.912017+0000","last_peered":"2024-04-25T10:04:13.912017+0000","last_clean":"2024-04-25T10:04:13.912017+0000","last_became_active":"2024-04-25T10:04:13.907222+0000","last_became_peered":"2024-04-25T10:04:13.907222+0000","last_unstale":"2024-04-25T10:04:13.912017+0000","last_undegraded":"2024-04-25T10:04:13.912017+0000","last_fullsized":"2024-04-25T10:04:13.912017+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:14.655743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.902109+0000","last_change":"2024-04-25T10:04:13.902109+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.902109+0000","last_undegraded":"2024-04-25T10:04:13.902109+0000","last_fullsized":"2024-04-25T10:04:13.902109+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900704+0000","last_change":"2024-04-25T10:04:13.898686+0000","last_active":"2024-04-25T10:04:13.900704+0000","last_peered":"2024-04-25T10:04:13.900704+0000","last_clean":"2024-04-25T10:04:13.900704+0000","last_became_active":"2024-04-25T10:04:13.898413+0000","last_became_peered":"2024-04-25T10:04:13.898413+0000","last_unstale":"2024-04-25T10:04:13.900704+0000","last_undegraded":"2024-04-25T10:04:13.900704+0000","last_fullsized":"2024-04-25T10:04:13.900704+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:30.648919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896588+0000","last_change":"2024-04-25T10:04:13.896588+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896588+0000","last_undegraded":"2024-04-25T10:04:13.896588+0000","last_fullsized":"2024-04-25T10:04:13.896588+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899891+0000","last_change":"2024-04-25T10:04:13.897853+0000","last_active":"2024-04-25T10:04:13.899891+0000","last_peered":"2024-04-25T10:04:13.899891+0000","last_clean":"2024-04-25T10:04:13.899891+0000","last_became_active":"2024-04-25T10:04:13.897547+0000","last_became_peered":"2024-04-25T10:04:13.897547+0000","last_unstale":"2024-04-25T10:04:13.899891+0000","last_undegraded":"2024-04-25T10:04:13.899891+0000","last_fullsized":"2024-04-25T10:04:13.899891+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:32.020740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894510+0000","last_change":"2024-04-25T10:04:13.894510+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894510+0000","last_undegraded":"2024-04-25T10:04:13.894510+0000","last_fullsized":"2024-04-25T10:04:13.894510+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905135+0000","last_change":"2024-04-25T10:04:13.901634+0000","last_active":"2024-04-25T10:04:13.905135+0000","last_peered":"2024-04-25T10:04:13.905135+0000","last_clean":"2024-04-25T10:04:13.905135+0000","last_became_active":"2024-04-25T10:04:13.900799+0000","last_became_peered":"2024-04-25T10:04:13.900799+0000","last_unstale":"2024-04-25T10:04:13.905135+0000","last_undegraded":"2024-04-25T10:04:13.905135+0000","last_fullsized":"2024-04-25T10:04:13.905135+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:08.935454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.890080+0000","last_change":"2024-04-25T10:04:13.890080+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.890080+0000","last_undegraded":"2024-04-25T10:04:13.890080+0000","last_fullsized":"2024-04-25T10:04:13.890080+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900787+0000","last_change":"2024-04-25T10:04:13.898306+0000","last_active":"2024-04-25T10:04:13.900787+0000","last_peered":"2024-04-25T10:04:13.900787+0000","last_clean":"2024-04-25T10:04:13.900787+0000","last_became_active":"2024-04-25T10:04:13.898057+0000","last_became_peered":"2024-04-25T10:04:13.898057+0000","last_unstale":"2024-04-25T10:04:13.900787+0000","last_undegraded":"2024-04-25T10:04:13.900787+0000","last_fullsized":"2024-04-25T10:04:13.900787+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:53.024047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.890799+0000","last_change":"2024-04-25T10:04:13.890799+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.890799+0000","last_undegraded":"2024-04-25T10:04:13.890799+0000","last_fullsized":"2024-04-25T10:04:13.890799+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.902746+0000","last_change":"2024-04-25T10:04:13.902317+0000","last_active":"2024-04-25T10:04:13.902746+0000","last_peered":"2024-04-25T10:04:13.902746+0000","last_clean":"2024-04-25T10:04:13.902746+0000","last_became_active":"2024-04-25T10:04:13.902008+0000","last_became_peered":"2024-04-25T10:04:13.902008+0000","last_unstale":"2024-04-25T10:04:13.902746+0000","last_undegraded":"2024-04-25T10:04:13.902746+0000","last_fullsized":"2024-04-25T10:04:13.902746+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:15.960516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896084+0000","last_change":"2024-04-25T10:04:13.896084+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896084+0000","last_undegraded":"2024-04-25T10:04:13.896084+0000","last_fullsized":"2024-04-25T10:04:13.896084+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.902930+0000","last_change":"2024-04-25T10:04:13.902390+0000","last_active":"2024-04-25T10:04:13.902930+0000","last_peered":"2024-04-25T10:04:13.902930+0000","last_clean":"2024-04-25T10:04:13.902930+0000","last_became_active":"2024-04-25T10:04:13.902012+0000","last_became_peered":"2024-04-25T10:04:13.902012+0000","last_unstale":"2024-04-25T10:04:13.902930+0000","last_undegraded":"2024-04-25T10:04:13.902930+0000","last_fullsized":"2024-04-25T10:04:13.902930+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:04.306029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896982+0000","last_change":"2024-04-25T10:04:13.896982+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896982+0000","last_undegraded":"2024-04-25T10:04:13.896982+0000","last_fullsized":"2024-04-25T10:04:13.896982+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899990+0000","last_change":"2024-04-25T10:04:13.896683+0000","last_active":"2024-04-25T10:04:13.899990+0000","last_peered":"2024-04-25T10:04:13.899990+0000","last_clean":"2024-04-25T10:04:13.899990+0000","last_became_active":"2024-04-25T10:04:13.896386+0000","last_became_peered":"2024-04-25T10:04:13.896386+0000","last_unstale":"2024-04-25T10:04:13.899990+0000","last_undegraded":"2024-04-25T10:04:13.899990+0000","last_fullsized":"2024-04-25T10:04:13.899990+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:14.367579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895399+0000","last_change":"2024-04-25T10:04:13.895399+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895399+0000","last_undegraded":"2024-04-25T10:04:13.895399+0000","last_fullsized":"2024-04-25T10:04:13.895399+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.906438+0000","last_change":"2024-04-25T10:04:13.905850+0000","last_active":"2024-04-25T10:04:13.906438+0000","last_peered":"2024-04-25T10:04:13.906438+0000","last_clean":"2024-04-25T10:04:13.906438+0000","last_became_active":"2024-04-25T10:04:13.905565+0000","last_became_peered":"2024-04-25T10:04:13.905565+0000","last_unstale":"2024-04-25T10:04:13.906438+0000","last_undegraded":"2024-04-25T10:04:13.906438+0000","last_fullsized":"2024-04-25T10:04:13.906438+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:42.064164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.901364+0000","last_change":"2024-04-25T10:04:13.901364+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.901364+0000","last_undegraded":"2024-04-25T10:04:13.901364+0000","last_fullsized":"2024-04-25T10:04:13.901364+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900445+0000","last_change":"2024-04-25T10:04:13.899466+0000","last_active":"2024-04-25T10:04:13.900445+0000","last_peered":"2024-04-25T10:04:13.900445+0000","last_clean":"2024-04-25T10:04:13.900445+0000","last_became_active":"2024-04-25T10:04:13.899146+0000","last_became_peered":"2024-04-25T10:04:13.899146+0000","last_unstale":"2024-04-25T10:04:13.900445+0000","last_undegraded":"2024-04-25T10:04:13.900445+0000","last_fullsized":"2024-04-25T10:04:13.900445+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:43.518006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.900952+0000","last_change":"2024-04-25T10:04:13.900952+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.900952+0000","last_undegraded":"2024-04-25T10:04:13.900952+0000","last_fullsized":"2024-04-25T10:04:13.900952+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.908254+0000","last_change":"2024-04-25T10:04:13.907912+0000","last_active":"2024-04-25T10:04:13.908254+0000","last_peered":"2024-04-25T10:04:13.908254+0000","last_clean":"2024-04-25T10:04:13.908254+0000","last_became_active":"2024-04-25T10:04:13.907651+0000","last_became_peered":"2024-04-25T10:04:13.907651+0000","last_unstale":"2024-04-25T10:04:13.908254+0000","last_undegraded":"2024-04-25T10:04:13.908254+0000","last_fullsized":"2024-04-25T10:04:13.908254+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:27.691750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896135+0000","last_change":"2024-04-25T10:04:13.896135+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896135+0000","last_undegraded":"2024-04-25T10:04:13.896135+0000","last_fullsized":"2024-04-25T10:04:13.896135+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905973+0000","last_change":"2024-04-25T10:04:13.904679+0000","last_active":"2024-04-25T10:04:13.905973+0000","last_peered":"2024-04-25T10:04:13.905973+0000","last_clean":"2024-04-25T10:04:13.905973+0000","last_became_active":"2024-04-25T10:04:13.904426+0000","last_became_peered":"2024-04-25T10:04:13.904426+0000","last_unstale":"2024-04-25T10:04:13.905973+0000","last_undegraded":"2024-04-25T10:04:13.905973+0000","last_fullsized":"2024-04-25T10:04:13.905973+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:44.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896661+0000","last_change":"2024-04-25T10:04:13.896661+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896661+0000","last_undegraded":"2024-04-25T10:04:13.896661+0000","last_fullsized":"2024-04-25T10:04:13.896661+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.904794+0000","last_change":"2024-04-25T10:04:13.898826+0000","last_active":"2024-04-25T10:04:13.904794+0000","last_peered":"2024-04-25T10:04:13.904794+0000","last_clean":"2024-04-25T10:04:13.904794+0000","last_became_active":"2024-04-25T10:04:13.898462+0000","last_became_peered":"2024-04-25T10:04:13.898462+0000","last_unstale":"2024-04-25T10:04:13.904794+0000","last_undegraded":"2024-04-25T10:04:13.904794+0000","last_fullsized":"2024-04-25T10:04:13.904794+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:30.864286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898514+0000","last_change":"2024-04-25T10:04:13.898514+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898514+0000","last_undegraded":"2024-04-25T10:04:13.898514+0000","last_fullsized":"2024-04-25T10:04:13.898514+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899831+0000","last_change":"2024-04-25T10:04:13.899083+0000","last_active":"2024-04-25T10:04:13.899831+0000","last_peered":"2024-04-25T10:04:13.899831+0000","last_clean":"2024-04-25T10:04:13.899831+0000","last_became_active":"2024-04-25T10:04:13.898810+0000","last_became_peered":"2024-04-25T10:04:13.898810+0000","last_unstale":"2024-04-25T10:04:13.899831+0000","last_undegraded":"2024-04-25T10:04:13.899831+0000","last_fullsized":"2024-04-25T10:04:13.899831+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:30.648919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894391+0000","last_change":"2024-04-25T10:04:13.894391+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894391+0000","last_undegraded":"2024-04-25T10:04:13.894391+0000","last_fullsized":"2024-04-25T10:04:13.894391+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900693+0000","last_change":"2024-04-25T10:04:13.899508+0000","last_active":"2024-04-25T10:04:13.900693+0000","last_peered":"2024-04-25T10:04:13.900693+0000","last_clean":"2024-04-25T10:04:13.900693+0000","last_became_active":"2024-04-25T10:04:13.898618+0000","last_became_peered":"2024-04-25T10:04:13.898618+0000","last_unstale":"2024-04-25T10:04:13.900693+0000","last_undegraded":"2024-04-25T10:04:13.900693+0000","last_fullsized":"2024-04-25T10:04:13.900693+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:37.888496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.900372+0000","last_change":"2024-04-25T10:04:13.900372+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.900372+0000","last_undegraded":"2024-04-25T10:04:13.900372+0000","last_fullsized":"2024-04-25T10:04:13.900372+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899333+0000","last_change":"2024-04-25T10:04:13.895684+0000","last_active":"2024-04-25T10:04:13.899333+0000","last_peered":"2024-04-25T10:04:13.899333+0000","last_clean":"2024-04-25T10:04:13.899333+0000","last_became_active":"2024-04-25T10:04:13.895390+0000","last_became_peered":"2024-04-25T10:04:13.895390+0000","last_unstale":"2024-04-25T10:04:13.899333+0000","last_undegraded":"2024-04-25T10:04:13.899333+0000","last_fullsized":"2024-04-25T10:04:13.899333+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:18.467823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888644+0000","last_change":"2024-04-25T10:04:13.888644+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888644+0000","last_undegraded":"2024-04-25T10:04:13.888644+0000","last_fullsized":"2024-04-25T10:04:13.888644+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913155+0000","last_change":"2024-04-25T10:04:13.911061+0000","last_active":"2024-04-25T10:04:13.913155+0000","last_peered":"2024-04-25T10:04:13.913155+0000","last_clean":"2024-04-25T10:04:13.913155+0000","last_became_active":"2024-04-25T10:04:13.910591+0000","last_became_peered":"2024-04-25T10:04:13.910591+0000","last_unstale":"2024-04-25T10:04:13.913155+0000","last_undegraded":"2024-04-25T10:04:13.913155+0000","last_fullsized":"2024-04-25T10:04:13.913155+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:40.288371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895546+0000","last_change":"2024-04-25T10:04:13.895546+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895546+0000","last_undegraded":"2024-04-25T10:04:13.895546+0000","last_fullsized":"2024-04-25T10:04:13.895546+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913837+0000","last_change":"2024-04-25T10:04:13.913137+0000","last_active":"2024-04-25T10:04:13.913837+0000","last_peered":"2024-04-25T10:04:13.913837+0000","last_clean":"2024-04-25T10:04:13.913837+0000","last_became_active":"2024-04-25T10:04:13.912835+0000","last_became_peered":"2024-04-25T10:04:13.912835+0000","last_unstale":"2024-04-25T10:04:13.913837+0000","last_undegraded":"2024-04-25T10:04:13.913837+0000","last_fullsized":"2024-04-25T10:04:13.913837+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:25.196171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895070+0000","last_change":"2024-04-25T10:04:13.895070+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895070+0000","last_undegraded":"2024-04-25T10:04:13.895070+0000","last_fullsized":"2024-04-25T10:04:13.895070+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.897923+0000","last_change":"2024-04-25T10:04:13.896282+0000","last_active":"2024-04-25T10:04:13.897923+0000","last_peered":"2024-04-25T10:04:13.897923+0000","last_clean":"2024-04-25T10:04:13.897923+0000","last_became_active":"2024-04-25T10:04:13.894939+0000","last_became_peered":"2024-04-25T10:04:13.894939+0000","last_unstale":"2024-04-25T10:04:13.897923+0000","last_undegraded":"2024-04-25T10:04:13.897923+0000","last_fullsized":"2024-04-25T10:04:13.897923+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:17.612850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.889558+0000","last_change":"2024-04-25T10:04:13.889558+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.889558+0000","last_undegraded":"2024-04-25T10:04:13.889558+0000","last_fullsized":"2024-04-25T10:04:13.889558+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.910509+0000","last_change":"2024-04-25T10:04:13.910016+0000","last_active":"2024-04-25T10:04:13.910509+0000","last_peered":"2024-04-25T10:04:13.910509+0000","last_clean":"2024-04-25T10:04:13.910509+0000","last_became_active":"2024-04-25T10:04:13.909591+0000","last_became_peered":"2024-04-25T10:04:13.909591+0000","last_unstale":"2024-04-25T10:04:13.910509+0000","last_undegraded":"2024-04-25T10:04:13.910509+0000","last_fullsized":"2024-04-25T10:04:13.910509+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:31.416889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.901336+0000","last_change":"2024-04-25T10:04:13.901336+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.901336+0000","last_undegraded":"2024-04-25T10:04:13.901336+0000","last_fullsized":"2024-04-25T10:04:13.901336+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905824+0000","last_change":"2024-04-25T10:04:13.899980+0000","last_active":"2024-04-25T10:04:13.905824+0000","last_peered":"2024-04-25T10:04:13.905824+0000","last_clean":"2024-04-25T10:04:13.905824+0000","last_became_active":"2024-04-25T10:04:13.899671+0000","last_became_peered":"2024-04-25T10:04:13.899671+0000","last_unstale":"2024-04-25T10:04:13.905824+0000","last_undegraded":"2024-04-25T10:04:13.905824+0000","last_fullsized":"2024-04-25T10:04:13.905824+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:22.542468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888539+0000","last_change":"2024-04-25T10:04:13.888539+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888539+0000","last_undegraded":"2024-04-25T10:04:13.888539+0000","last_fullsized":"2024-04-25T10:04:13.888539+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.910748+0000","last_change":"2024-04-25T10:04:13.910423+0000","last_active":"2024-04-25T10:04:13.910748+0000","last_peered":"2024-04-25T10:04:13.910748+0000","last_clean":"2024-04-25T10:04:13.910748+0000","last_became_active":"2024-04-25T10:04:13.910113+0000","last_became_peered":"2024-04-25T10:04:13.910113+0000","last_unstale":"2024-04-25T10:04:13.910748+0000","last_undegraded":"2024-04-25T10:04:13.910748+0000","last_fullsized":"2024-04-25T10:04:13.910748+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:27.691750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.893539+0000","last_change":"2024-04-25T10:04:13.893539+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.893539+0000","last_undegraded":"2024-04-25T10:04:13.893539+0000","last_fullsized":"2024-04-25T10:04:13.893539+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905911+0000","last_change":"2024-04-25T10:04:13.903829+0000","last_active":"2024-04-25T10:04:13.905911+0000","last_peered":"2024-04-25T10:04:13.905911+0000","last_clean":"2024-04-25T10:04:13.905911+0000","last_became_active":"2024-04-25T10:04:13.901705+0000","last_became_peered":"2024-04-25T10:04:13.901705+0000","last_unstale":"2024-04-25T10:04:13.905911+0000","last_undegraded":"2024-04-25T10:04:13.905911+0000","last_fullsized":"2024-04-25T10:04:13.905911+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:17.450538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":461,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.888932+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:04:13.888932+0000","last_peered":"2024-04-25T10:04:13.888932+0000","last_clean":"2024-04-25T10:04:13.888932+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:04:13.888932+0000","last_undegraded":"2024-04-25T10:04:13.888932+0000","last_fullsized":"2024-04-25T10:04:13.888932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51820,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320020,"statfs":{"total":96636764160,"available":96583700480,"internally_reserved":0,"allocated":2650112,"data_stored":1552487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574988,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51224,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320616,"statfs":{"total":96636764160,"available":96584310784,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51236,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320604,"statfs":{"total":96636764160,"available":96584298496,"internally_reserved":0,"allocated":2056192,"data_stored":962119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59560,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312280,"statfs":{"total":96636764160,"available":96575774720,"internally_reserved":0,"allocated":2056192,"data_stored":962119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59544,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312296,"statfs":{"total":96636764160,"available":96575791104,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55392,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316448,"statfs":{"total":96636764160,"available":96580042752,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55388,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316452,"statfs":{"total":96636764160,"available":96580046848,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55964,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315876,"statfs":{"total":96636764160,"available":96579457024,"internally_reserved":0,"allocated":2637824,"data_stored":1545820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:18.254 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:18.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.385+0000 7fce67a5a640 1 Processor -- start 2024-04-25T10:04:18.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.386+0000 7fce67a5a640 1 -- start start 2024-04-25T10:04:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.386+0000 7fce67a5a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce600d0b10 0x7fce600ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.386+0000 7fce67a5a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fce600b5f00 0x7fce600d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce67a5a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce600cec00 con 0x7fce6015db60 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce67a5a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce600ce930 con 0x7fce600b5f00 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce67a5a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce600b6450 con 0x7fce600d0b10 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce657f6640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fce6015db60 0x7fce600d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57562/0 (socket says 172.21.15.19:57562) 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce657f6640 1 -- 172.21.15.19:0/410328347 learned_addr learned my addr 172.21.15.19:0/410328347 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce65ff7640 1 --2- 172.21.15.19:0/410328347 >> v2:172.21.15.112:3300/0 conn(0x7fce600b5f00 0x7fce600d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce64ff5640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce600d0b10 0x7fce600ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010871362 0 0) 0x7fce600cec00 con 0x7fce6015db60 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce4c003520 con 0x7fce6015db60 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2065897005 0 0) 0x7fce4c003520 con 0x7fce6015db60 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce600d0b10 msgr2=0x7fce600ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce600d0b10 0x7fce600ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 >> v2:172.21.15.112:3300/0 conn(0x7fce600b5f00 msgr2=0x7fce600d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 --2- 172.21.15.19:0/410328347 >> v2:172.21.15.112:3300/0 conn(0x7fce600b5f00 0x7fce600d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce600ceee0 con 0x7fce6015db60 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fce5c001670 con 0x7fce6015db60 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce65ff7640 1 --2- 172.21.15.19:0/410328347 >> v2:172.21.15.112:3300/0 conn(0x7fce600b5f00 0x7fce600d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 >> v1:172.21.15.19:6789/0 conn(0x7fce6015db60 legacy=0x7fce600d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce4c0039e0 0x7fce4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce600d0b10 0x7fce4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce4c004d40 0x7fce4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce600ce930 con 0x7fce4c004d40 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce600b6450 con 0x7fce4c0039e0 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c00b9f0 con 0x7fce600d0b10 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.387+0000 7fce65ff7640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce4c004d40 0x7fce4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.388+0000 7fce657f6640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce4c0039e0 0x7fce4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.388+0000 7fce64ff5640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce600d0b10 0x7fce4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.388+0000 7fce65ff7640 1 -- 172.21.15.19:0/410328347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce600d0b10 msgr2=0x7fce4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.388+0000 7fce65ff7640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce600d0b10 0x7fce4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.388+0000 7fce65ff7640 1 -- 172.21.15.19:0/410328347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce4c0039e0 msgr2=0x7fce4c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.388+0000 7fce65ff7640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce4c0039e0 0x7fce4c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.388+0000 7fce65ff7640 1 -- 172.21.15.19:0/410328347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c003520 con 0x7fce4c004d40 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.388+0000 7fce65ff7640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce4c004d40 0x7fce4c0051c0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fce5800e350 tx=0x7fce4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bf9d9b2d0c95fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.389+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58009de0 con 0x7fce4c004d40 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.389+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5802e660 con 0x7fce4c004d40 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.389+0000 7fce57fff640 1 -- 172.21.15.19:0/410328347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5802ea10 con 0x7fce4c004d40 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.389+0000 7fce67a5a640 1 -- 172.21.15.19:0/410328347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce4c004d40 msgr2=0x7fce4c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.389+0000 7fce67a5a640 1 --2- 172.21.15.19:0/410328347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce4c004d40 0x7fce4c0051c0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fce5800e350 tx=0x7fce4c005830 comp rx=0 tx=0).stop 2024-04-25T10:04:18.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.389+0000 7fce65ff7640 1 -- 172.21.15.19:0/410328347 reap_dead start 2024-04-25T10:04:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.389+0000 7fce67a5a640 1 -- 172.21.15.19:0/410328347 shutdown_connections 2024-04-25T10:04:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.389+0000 7fce67a5a640 1 -- 172.21.15.19:0/410328347 >> 172.21.15.19:0/410328347 conn(0x7fce60061600 msgr2=0x7fce60061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.389+0000 7fce67a5a640 1 -- 172.21.15.19:0/410328347 shutdown_connections 2024-04-25T10:04:18.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.389+0000 7fce67a5a640 1 -- 172.21.15.19:0/410328347 wait complete. 2024-04-25T10:04:18.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.390+0000 7fce67a5a640 1 Processor -- start 2024-04-25T10:04:18.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.391+0000 7fce67a5a640 1 -- start start 2024-04-25T10:04:18.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.391+0000 7fce67a5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce4c004d40 0x7fce601624a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.391+0000 7fce67a5a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce6014d1c0 0x7fce600c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.391+0000 7fce67a5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce600ccd10 0x7fce600cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.391+0000 7fce67a5a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6015df40 con 0x7fce600ccd10 2024-04-25T10:04:18.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.391+0000 7fce67a5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce600d1d50 con 0x7fce4c004d40 2024-04-25T10:04:18.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.391+0000 7fce67a5a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6015e0c0 con 0x7fce6014d1c0 2024-04-25T10:04:18.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.391+0000 7fce65ff7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce600ccd10 0x7fce600cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.391+0000 7fce65ff7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce600ccd10 0x7fce600cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56964/0 (socket says 172.21.15.19:56964) 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.391+0000 7fce65ff7640 1 -- 172.21.15.19:0/563104340 learned_addr learned my addr 172.21.15.19:0/563104340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce657f6640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce4c004d40 0x7fce601624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce64ff5640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce6014d1c0 0x7fce600c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce65ff7640 1 -- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce6014d1c0 msgr2=0x7fce600c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce65ff7640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce6014d1c0 0x7fce600c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce65ff7640 1 -- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce4c004d40 msgr2=0x7fce601624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce65ff7640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce4c004d40 0x7fce601624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce65ff7640 1 -- 172.21.15.19:0/563104340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce600c94f0 con 0x7fce600ccd10 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce657f6640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce4c004d40 0x7fce601624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce65ff7640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce600ccd10 0x7fce600cd190 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fce5802b0e0 tx=0x7fce58009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce55ffb640 1 -- 172.21.15.19:0/563104340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58009de0 con 0x7fce600ccd10 2024-04-25T10:04:18.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce55ffb640 1 -- 172.21.15.19:0/563104340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5802e460 con 0x7fce600ccd10 2024-04-25T10:04:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce67a5a640 1 -- 172.21.15.19:0/563104340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce600c97d0 con 0x7fce600ccd10 2024-04-25T10:04:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce55ffb640 1 -- 172.21.15.19:0/563104340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58003bd0 con 0x7fce600ccd10 2024-04-25T10:04:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.392+0000 7fce67a5a640 1 -- 172.21.15.19:0/563104340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce600c9d90 con 0x7fce600ccd10 2024-04-25T10:04:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.393+0000 7fce67a5a640 1 -- 172.21.15.19:0/563104340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce600ca0c0 con 0x7fce600ccd10 2024-04-25T10:04:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.394+0000 7fce55ffb640 1 -- 172.21.15.19:0/563104340 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce58048070 con 0x7fce600ccd10 2024-04-25T10:04:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.394+0000 7fce55ffb640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce4406b4d0 0x7fce4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.394+0000 7fce55ffb640 1 -- 172.21.15.19:0/563104340 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce580ab340 con 0x7fce600ccd10 2024-04-25T10:04:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.394+0000 7fce657f6640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce4406b4d0 0x7fce4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.395+0000 7fce657f6640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce4406b4d0 0x7fce4406d990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fce5c007c70 tx=0x7fce5c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.398+0000 7fce55ffb640 1 -- 172.21.15.19:0/563104340 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce58032030 con 0x7fce600ccd10 2024-04-25T10:04:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.574+0000 7fce67a5a640 1 -- 172.21.15.19:0/563104340 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce60073270 con 0x7fce4406b4d0 2024-04-25T10:04:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.581+0000 7fce55ffb640 1 -- 172.21.15.19:0/563104340 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7fce60073270 con 0x7fce4406b4d0 2024-04-25T10:04:18.589 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:04:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 -- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce4406b4d0 msgr2=0x7fce4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce4406b4d0 0x7fce4406d990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fce5c007c70 tx=0x7fce5c009040 comp rx=0 tx=0).stop 2024-04-25T10:04:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 -- 172.21.15.19:0/563104340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce600ccd10 msgr2=0x7fce600cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce600ccd10 0x7fce600cd190 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fce5802b0e0 tx=0x7fce58009750 comp rx=0 tx=0).stop 2024-04-25T10:04:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 -- 172.21.15.19:0/563104340 shutdown_connections 2024-04-25T10:04:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fce4406b4d0 0x7fce4406d990 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce600ccd10 0x7fce600cd190 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce6014d1c0 0x7fce600c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 --2- 172.21.15.19:0/563104340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce4c004d40 0x7fce601624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 -- 172.21.15.19:0/563104340 >> 172.21.15.19:0/563104340 conn(0x7fce60061600 msgr2=0x7fce600b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 -- 172.21.15.19:0/563104340 shutdown_connections 2024-04-25T10:04:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.584+0000 7fce67a5a640 1 -- 172.21.15.19:0/563104340 wait complete. 2024-04-25T10:04:18.601 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-04-25T10:04:17.068592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440128,"kb_used_data":17164,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754534592,"statfs":{"total":773094113280,"available":772643422208,"internally_reserved":0,"allocated":17575936,"data_stored":8844353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.184409"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.906917+0000","last_change":"2024-04-25T10:04:13.906917+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.906917+0000","last_undegraded":"2024-04-25T10:04:13.906917+0000","last_fullsized":"2024-04-25T10:04:13.906917+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.906400+0000","last_change":"2024-04-25T10:04:13.906400+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.906400+0000","last_undegraded":"2024-04-25T10:04:13.906400+0000","last_fullsized":"2024-04-25T10:04:13.906400+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894277+0000","last_change":"2024-04-25T10:04:13.894277+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894277+0000","last_undegraded":"2024-04-25T10:04:13.894277+0000","last_fullsized":"2024-04-25T10:04:13.894277+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.899956+0000","last_change":"2024-04-25T10:04:13.899956+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.899956+0000","last_undegraded":"2024-04-25T10:04:13.899956+0000","last_fullsized":"2024-04-25T10:04:13.899956+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.907015+0000","last_change":"2024-04-25T10:04:13.907015+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.907015+0000","last_undegraded":"2024-04-25T10:04:13.907015+0000","last_fullsized":"2024-04-25T10:04:13.907015+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.905958+0000","last_change":"2024-04-25T10:04:13.905958+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.905958+0000","last_undegraded":"2024-04-25T10:04:13.905958+0000","last_fullsized":"2024-04-25T10:04:13.905958+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898174+0000","last_change":"2024-04-25T10:04:13.898174+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898174+0000","last_undegraded":"2024-04-25T10:04:13.898174+0000","last_fullsized":"2024-04-25T10:04:13.898174+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888128+0000","last_change":"2024-04-25T10:04:13.888128+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888128+0000","last_undegraded":"2024-04-25T10:04:13.888128+0000","last_fullsized":"2024-04-25T10:04:13.888128+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.906338+0000","last_change":"2024-04-25T10:04:13.906338+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.906338+0000","last_undegraded":"2024-04-25T10:04:13.906338+0000","last_fullsized":"2024-04-25T10:04:13.906338+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904643+0000","last_change":"2024-04-25T10:04:13.904643+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904643+0000","last_undegraded":"2024-04-25T10:04:13.904643+0000","last_fullsized":"2024-04-25T10:04:13.904643+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894877+0000","last_change":"2024-04-25T10:04:13.894877+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894877+0000","last_undegraded":"2024-04-25T10:04:13.894877+0000","last_fullsized":"2024-04-25T10:04:13.894877+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897174+0000","last_change":"2024-04-25T10:04:13.897174+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897174+0000","last_undegraded":"2024-04-25T10:04:13.897174+0000","last_fullsized":"2024-04-25T10:04:13.897174+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.891816+0000","last_change":"2024-04-25T10:04:13.891816+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.891816+0000","last_undegraded":"2024-04-25T10:04:13.891816+0000","last_fullsized":"2024-04-25T10:04:13.891816+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897714+0000","last_change":"2024-04-25T10:04:13.897714+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897714+0000","last_undegraded":"2024-04-25T10:04:13.897714+0000","last_fullsized":"2024-04-25T10:04:13.897714+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904789+0000","last_change":"2024-04-25T10:04:13.904789+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904789+0000","last_undegraded":"2024-04-25T10:04:13.904789+0000","last_fullsized":"2024-04-25T10:04:13.904789+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903622+0000","last_change":"2024-04-25T10:04:13.903622+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903622+0000","last_undegraded":"2024-04-25T10:04:13.903622+0000","last_fullsized":"2024-04-25T10:04:13.903622+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898964+0000","last_change":"2024-04-25T10:04:13.898964+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898964+0000","last_undegraded":"2024-04-25T10:04:13.898964+0000","last_fullsized":"2024-04-25T10:04:13.898964+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.891283+0000","last_change":"2024-04-25T10:04:13.891283+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.891283+0000","last_undegraded":"2024-04-25T10:04:13.891283+0000","last_fullsized":"2024-04-25T10:04:13.891283+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.889985+0000","last_change":"2024-04-25T10:04:13.889985+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.889985+0000","last_undegraded":"2024-04-25T10:04:13.889985+0000","last_fullsized":"2024-04-25T10:04:13.889985+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903899+0000","last_change":"2024-04-25T10:04:13.903899+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903899+0000","last_undegraded":"2024-04-25T10:04:13.903899+0000","last_fullsized":"2024-04-25T10:04:13.903899+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904325+0000","last_change":"2024-04-25T10:04:13.904325+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904325+0000","last_undegraded":"2024-04-25T10:04:13.904325+0000","last_fullsized":"2024-04-25T10:04:13.904325+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.904219+0000","last_change":"2024-04-25T10:04:13.904219+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.904219+0000","last_undegraded":"2024-04-25T10:04:13.904219+0000","last_fullsized":"2024-04-25T10:04:13.904219+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888564+0000","last_change":"2024-04-25T10:04:13.888564+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888564+0000","last_undegraded":"2024-04-25T10:04:13.888564+0000","last_fullsized":"2024-04-25T10:04:13.888564+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903639+0000","last_change":"2024-04-25T10:04:13.903639+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903639+0000","last_undegraded":"2024-04-25T10:04:13.903639+0000","last_fullsized":"2024-04-25T10:04:13.903639+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898089+0000","last_change":"2024-04-25T10:04:13.898089+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898089+0000","last_undegraded":"2024-04-25T10:04:13.898089+0000","last_fullsized":"2024-04-25T10:04:13.898089+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896303+0000","last_change":"2024-04-25T10:04:13.896303+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896303+0000","last_undegraded":"2024-04-25T10:04:13.896303+0000","last_fullsized":"2024-04-25T10:04:13.896303+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888597+0000","last_change":"2024-04-25T10:04:13.888597+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888597+0000","last_undegraded":"2024-04-25T10:04:13.888597+0000","last_fullsized":"2024-04-25T10:04:13.888597+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.899739+0000","last_change":"2024-04-25T10:04:13.899739+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.899739+0000","last_undegraded":"2024-04-25T10:04:13.899739+0000","last_fullsized":"2024-04-25T10:04:13.899739+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894259+0000","last_change":"2024-04-25T10:04:13.894259+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894259+0000","last_undegraded":"2024-04-25T10:04:13.894259+0000","last_fullsized":"2024-04-25T10:04:13.894259+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.893981+0000","last_change":"2024-04-25T10:04:13.893981+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.893981+0000","last_undegraded":"2024-04-25T10:04:13.893981+0000","last_fullsized":"2024-04-25T10:04:13.893981+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897098+0000","last_change":"2024-04-25T10:04:13.897098+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897098+0000","last_undegraded":"2024-04-25T10:04:13.897098+0000","last_fullsized":"2024-04-25T10:04:13.897098+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895659+0000","last_change":"2024-04-25T10:04:13.895659+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895659+0000","last_undegraded":"2024-04-25T10:04:13.895659+0000","last_fullsized":"2024-04-25T10:04:13.895659+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888515+0000","last_change":"2024-04-25T10:04:13.888515+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888515+0000","last_undegraded":"2024-04-25T10:04:13.888515+0000","last_fullsized":"2024-04-25T10:04:13.888515+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.909538+0000","last_change":"2024-04-25T10:04:13.905310+0000","last_active":"2024-04-25T10:04:13.909538+0000","last_peered":"2024-04-25T10:04:13.909538+0000","last_clean":"2024-04-25T10:04:13.909538+0000","last_became_active":"2024-04-25T10:04:13.905060+0000","last_became_peered":"2024-04-25T10:04:13.905060+0000","last_unstale":"2024-04-25T10:04:13.909538+0000","last_undegraded":"2024-04-25T10:04:13.909538+0000","last_fullsized":"2024-04-25T10:04:13.909538+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:03.466270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897689+0000","last_change":"2024-04-25T10:04:13.897689+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897689+0000","last_undegraded":"2024-04-25T10:04:13.897689+0000","last_fullsized":"2024-04-25T10:04:13.897689+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913761+0000","last_change":"2024-04-25T10:04:13.911506+0000","last_active":"2024-04-25T10:04:13.913761+0000","last_peered":"2024-04-25T10:04:13.913761+0000","last_clean":"2024-04-25T10:04:13.913761+0000","last_became_active":"2024-04-25T10:04:13.909233+0000","last_became_peered":"2024-04-25T10:04:13.909233+0000","last_unstale":"2024-04-25T10:04:13.913761+0000","last_undegraded":"2024-04-25T10:04:13.913761+0000","last_fullsized":"2024-04-25T10:04:13.913761+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:44.444031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895862+0000","last_change":"2024-04-25T10:04:13.895862+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895862+0000","last_undegraded":"2024-04-25T10:04:13.895862+0000","last_fullsized":"2024-04-25T10:04:13.895862+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900428+0000","last_change":"2024-04-25T10:04:13.899686+0000","last_active":"2024-04-25T10:04:13.900428+0000","last_peered":"2024-04-25T10:04:13.900428+0000","last_clean":"2024-04-25T10:04:13.900428+0000","last_became_active":"2024-04-25T10:04:13.899387+0000","last_became_peered":"2024-04-25T10:04:13.899387+0000","last_unstale":"2024-04-25T10:04:13.900428+0000","last_undegraded":"2024-04-25T10:04:13.900428+0000","last_fullsized":"2024-04-25T10:04:13.900428+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.333329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.893994+0000","last_change":"2024-04-25T10:04:13.893994+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.893994+0000","last_undegraded":"2024-04-25T10:04:13.893994+0000","last_fullsized":"2024-04-25T10:04:13.893994+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900727+0000","last_change":"2024-04-25T10:04:13.899482+0000","last_active":"2024-04-25T10:04:13.900727+0000","last_peered":"2024-04-25T10:04:13.900727+0000","last_clean":"2024-04-25T10:04:13.900727+0000","last_became_active":"2024-04-25T10:04:13.899195+0000","last_became_peered":"2024-04-25T10:04:13.899195+0000","last_unstale":"2024-04-25T10:04:13.900727+0000","last_undegraded":"2024-04-25T10:04:13.900727+0000","last_fullsized":"2024-04-25T10:04:13.900727+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:42.064164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.889782+0000","last_change":"2024-04-25T10:04:13.889782+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.889782+0000","last_undegraded":"2024-04-25T10:04:13.889782+0000","last_fullsized":"2024-04-25T10:04:13.889782+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913248+0000","last_change":"2024-04-25T10:04:13.911021+0000","last_active":"2024-04-25T10:04:13.913248+0000","last_peered":"2024-04-25T10:04:13.913248+0000","last_clean":"2024-04-25T10:04:13.913248+0000","last_became_active":"2024-04-25T10:04:13.909391+0000","last_became_peered":"2024-04-25T10:04:13.909391+0000","last_unstale":"2024-04-25T10:04:13.913248+0000","last_undegraded":"2024-04-25T10:04:13.913248+0000","last_fullsized":"2024-04-25T10:04:13.913248+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:11.455636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.902584+0000","last_change":"2024-04-25T10:04:13.902584+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.902584+0000","last_undegraded":"2024-04-25T10:04:13.902584+0000","last_fullsized":"2024-04-25T10:04:13.902584+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.911557+0000","last_change":"2024-04-25T10:04:13.907057+0000","last_active":"2024-04-25T10:04:13.911557+0000","last_peered":"2024-04-25T10:04:13.911557+0000","last_clean":"2024-04-25T10:04:13.911557+0000","last_became_active":"2024-04-25T10:04:13.906778+0000","last_became_peered":"2024-04-25T10:04:13.906778+0000","last_unstale":"2024-04-25T10:04:13.911557+0000","last_undegraded":"2024-04-25T10:04:13.911557+0000","last_fullsized":"2024-04-25T10:04:13.911557+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:59.124388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.903095+0000","last_change":"2024-04-25T10:04:13.903095+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.903095+0000","last_undegraded":"2024-04-25T10:04:13.903095+0000","last_fullsized":"2024-04-25T10:04:13.903095+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.895543+0000","last_change":"2024-04-25T10:04:13.894824+0000","last_active":"2024-04-25T10:04:13.895543+0000","last_peered":"2024-04-25T10:04:13.895543+0000","last_clean":"2024-04-25T10:04:13.895543+0000","last_became_active":"2024-04-25T10:04:13.894557+0000","last_became_peered":"2024-04-25T10:04:13.894557+0000","last_unstale":"2024-04-25T10:04:13.895543+0000","last_undegraded":"2024-04-25T10:04:13.895543+0000","last_fullsized":"2024-04-25T10:04:13.895543+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:38.732229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.887704+0000","last_change":"2024-04-25T10:04:13.887704+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.887704+0000","last_undegraded":"2024-04-25T10:04:13.887704+0000","last_fullsized":"2024-04-25T10:04:13.887704+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913799+0000","last_change":"2024-04-25T10:04:13.912389+0000","last_active":"2024-04-25T10:04:13.913799+0000","last_peered":"2024-04-25T10:04:13.913799+0000","last_clean":"2024-04-25T10:04:13.913799+0000","last_became_active":"2024-04-25T10:04:13.912117+0000","last_became_peered":"2024-04-25T10:04:13.912117+0000","last_unstale":"2024-04-25T10:04:13.913799+0000","last_undegraded":"2024-04-25T10:04:13.913799+0000","last_fullsized":"2024-04-25T10:04:13.913799+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:31.979599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.897391+0000","last_change":"2024-04-25T10:04:13.897391+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.897391+0000","last_undegraded":"2024-04-25T10:04:13.897391+0000","last_fullsized":"2024-04-25T10:04:13.897391+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.909291+0000","last_change":"2024-04-25T10:04:13.908763+0000","last_active":"2024-04-25T10:04:13.909291+0000","last_peered":"2024-04-25T10:04:13.909291+0000","last_clean":"2024-04-25T10:04:13.909291+0000","last_became_active":"2024-04-25T10:04:13.908481+0000","last_became_peered":"2024-04-25T10:04:13.908481+0000","last_unstale":"2024-04-25T10:04:13.909291+0000","last_undegraded":"2024-04-25T10:04:13.909291+0000","last_fullsized":"2024-04-25T10:04:13.909291+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:48.385428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.890204+0000","last_change":"2024-04-25T10:04:13.890204+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.890204+0000","last_undegraded":"2024-04-25T10:04:13.890204+0000","last_fullsized":"2024-04-25T10:04:13.890204+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.912017+0000","last_change":"2024-04-25T10:04:13.909365+0000","last_active":"2024-04-25T10:04:13.912017+0000","last_peered":"2024-04-25T10:04:13.912017+0000","last_clean":"2024-04-25T10:04:13.912017+0000","last_became_active":"2024-04-25T10:04:13.907222+0000","last_became_peered":"2024-04-25T10:04:13.907222+0000","last_unstale":"2024-04-25T10:04:13.912017+0000","last_undegraded":"2024-04-25T10:04:13.912017+0000","last_fullsized":"2024-04-25T10:04:13.912017+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:14.655743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.902109+0000","last_change":"2024-04-25T10:04:13.902109+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.902109+0000","last_undegraded":"2024-04-25T10:04:13.902109+0000","last_fullsized":"2024-04-25T10:04:13.902109+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900704+0000","last_change":"2024-04-25T10:04:13.898686+0000","last_active":"2024-04-25T10:04:13.900704+0000","last_peered":"2024-04-25T10:04:13.900704+0000","last_clean":"2024-04-25T10:04:13.900704+0000","last_became_active":"2024-04-25T10:04:13.898413+0000","last_became_peered":"2024-04-25T10:04:13.898413+0000","last_unstale":"2024-04-25T10:04:13.900704+0000","last_undegraded":"2024-04-25T10:04:13.900704+0000","last_fullsized":"2024-04-25T10:04:13.900704+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:30.648919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896588+0000","last_change":"2024-04-25T10:04:13.896588+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896588+0000","last_undegraded":"2024-04-25T10:04:13.896588+0000","last_fullsized":"2024-04-25T10:04:13.896588+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899891+0000","last_change":"2024-04-25T10:04:13.897853+0000","last_active":"2024-04-25T10:04:13.899891+0000","last_peered":"2024-04-25T10:04:13.899891+0000","last_clean":"2024-04-25T10:04:13.899891+0000","last_became_active":"2024-04-25T10:04:13.897547+0000","last_became_peered":"2024-04-25T10:04:13.897547+0000","last_unstale":"2024-04-25T10:04:13.899891+0000","last_undegraded":"2024-04-25T10:04:13.899891+0000","last_fullsized":"2024-04-25T10:04:13.899891+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:32.020740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894510+0000","last_change":"2024-04-25T10:04:13.894510+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894510+0000","last_undegraded":"2024-04-25T10:04:13.894510+0000","last_fullsized":"2024-04-25T10:04:13.894510+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905135+0000","last_change":"2024-04-25T10:04:13.901634+0000","last_active":"2024-04-25T10:04:13.905135+0000","last_peered":"2024-04-25T10:04:13.905135+0000","last_clean":"2024-04-25T10:04:13.905135+0000","last_became_active":"2024-04-25T10:04:13.900799+0000","last_became_peered":"2024-04-25T10:04:13.900799+0000","last_unstale":"2024-04-25T10:04:13.905135+0000","last_undegraded":"2024-04-25T10:04:13.905135+0000","last_fullsized":"2024-04-25T10:04:13.905135+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:08.935454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.890080+0000","last_change":"2024-04-25T10:04:13.890080+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.890080+0000","last_undegraded":"2024-04-25T10:04:13.890080+0000","last_fullsized":"2024-04-25T10:04:13.890080+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900787+0000","last_change":"2024-04-25T10:04:13.898306+0000","last_active":"2024-04-25T10:04:13.900787+0000","last_peered":"2024-04-25T10:04:13.900787+0000","last_clean":"2024-04-25T10:04:13.900787+0000","last_became_active":"2024-04-25T10:04:13.898057+0000","last_became_peered":"2024-04-25T10:04:13.898057+0000","last_unstale":"2024-04-25T10:04:13.900787+0000","last_undegraded":"2024-04-25T10:04:13.900787+0000","last_fullsized":"2024-04-25T10:04:13.900787+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:53.024047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.890799+0000","last_change":"2024-04-25T10:04:13.890799+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.890799+0000","last_undegraded":"2024-04-25T10:04:13.890799+0000","last_fullsized":"2024-04-25T10:04:13.890799+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.902746+0000","last_change":"2024-04-25T10:04:13.902317+0000","last_active":"2024-04-25T10:04:13.902746+0000","last_peered":"2024-04-25T10:04:13.902746+0000","last_clean":"2024-04-25T10:04:13.902746+0000","last_became_active":"2024-04-25T10:04:13.902008+0000","last_became_peered":"2024-04-25T10:04:13.902008+0000","last_unstale":"2024-04-25T10:04:13.902746+0000","last_undegraded":"2024-04-25T10:04:13.902746+0000","last_fullsized":"2024-04-25T10:04:13.902746+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:15.960516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896084+0000","last_change":"2024-04-25T10:04:13.896084+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896084+0000","last_undegraded":"2024-04-25T10:04:13.896084+0000","last_fullsized":"2024-04-25T10:04:13.896084+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.902930+0000","last_change":"2024-04-25T10:04:13.902390+0000","last_active":"2024-04-25T10:04:13.902930+0000","last_peered":"2024-04-25T10:04:13.902930+0000","last_clean":"2024-04-25T10:04:13.902930+0000","last_became_active":"2024-04-25T10:04:13.902012+0000","last_became_peered":"2024-04-25T10:04:13.902012+0000","last_unstale":"2024-04-25T10:04:13.902930+0000","last_undegraded":"2024-04-25T10:04:13.902930+0000","last_fullsized":"2024-04-25T10:04:13.902930+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:04.306029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896982+0000","last_change":"2024-04-25T10:04:13.896982+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896982+0000","last_undegraded":"2024-04-25T10:04:13.896982+0000","last_fullsized":"2024-04-25T10:04:13.896982+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899990+0000","last_change":"2024-04-25T10:04:13.896683+0000","last_active":"2024-04-25T10:04:13.899990+0000","last_peered":"2024-04-25T10:04:13.899990+0000","last_clean":"2024-04-25T10:04:13.899990+0000","last_became_active":"2024-04-25T10:04:13.896386+0000","last_became_peered":"2024-04-25T10:04:13.896386+0000","last_unstale":"2024-04-25T10:04:13.899990+0000","last_undegraded":"2024-04-25T10:04:13.899990+0000","last_fullsized":"2024-04-25T10:04:13.899990+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:14.367579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895399+0000","last_change":"2024-04-25T10:04:13.895399+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895399+0000","last_undegraded":"2024-04-25T10:04:13.895399+0000","last_fullsized":"2024-04-25T10:04:13.895399+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.906438+0000","last_change":"2024-04-25T10:04:13.905850+0000","last_active":"2024-04-25T10:04:13.906438+0000","last_peered":"2024-04-25T10:04:13.906438+0000","last_clean":"2024-04-25T10:04:13.906438+0000","last_became_active":"2024-04-25T10:04:13.905565+0000","last_became_peered":"2024-04-25T10:04:13.905565+0000","last_unstale":"2024-04-25T10:04:13.906438+0000","last_undegraded":"2024-04-25T10:04:13.906438+0000","last_fullsized":"2024-04-25T10:04:13.906438+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:42.064164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.901364+0000","last_change":"2024-04-25T10:04:13.901364+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.901364+0000","last_undegraded":"2024-04-25T10:04:13.901364+0000","last_fullsized":"2024-04-25T10:04:13.901364+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900445+0000","last_change":"2024-04-25T10:04:13.899466+0000","last_active":"2024-04-25T10:04:13.900445+0000","last_peered":"2024-04-25T10:04:13.900445+0000","last_clean":"2024-04-25T10:04:13.900445+0000","last_became_active":"2024-04-25T10:04:13.899146+0000","last_became_peered":"2024-04-25T10:04:13.899146+0000","last_unstale":"2024-04-25T10:04:13.900445+0000","last_undegraded":"2024-04-25T10:04:13.900445+0000","last_fullsized":"2024-04-25T10:04:13.900445+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:43.518006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.900952+0000","last_change":"2024-04-25T10:04:13.900952+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.900952+0000","last_undegraded":"2024-04-25T10:04:13.900952+0000","last_fullsized":"2024-04-25T10:04:13.900952+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.908254+0000","last_change":"2024-04-25T10:04:13.907912+0000","last_active":"2024-04-25T10:04:13.908254+0000","last_peered":"2024-04-25T10:04:13.908254+0000","last_clean":"2024-04-25T10:04:13.908254+0000","last_became_active":"2024-04-25T10:04:13.907651+0000","last_became_peered":"2024-04-25T10:04:13.907651+0000","last_unstale":"2024-04-25T10:04:13.908254+0000","last_undegraded":"2024-04-25T10:04:13.908254+0000","last_fullsized":"2024-04-25T10:04:13.908254+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:27.691750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896135+0000","last_change":"2024-04-25T10:04:13.896135+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896135+0000","last_undegraded":"2024-04-25T10:04:13.896135+0000","last_fullsized":"2024-04-25T10:04:13.896135+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905973+0000","last_change":"2024-04-25T10:04:13.904679+0000","last_active":"2024-04-25T10:04:13.905973+0000","last_peered":"2024-04-25T10:04:13.905973+0000","last_clean":"2024-04-25T10:04:13.905973+0000","last_became_active":"2024-04-25T10:04:13.904426+0000","last_became_peered":"2024-04-25T10:04:13.904426+0000","last_unstale":"2024-04-25T10:04:13.905973+0000","last_undegraded":"2024-04-25T10:04:13.905973+0000","last_fullsized":"2024-04-25T10:04:13.905973+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:44.452539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.896661+0000","last_change":"2024-04-25T10:04:13.896661+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.896661+0000","last_undegraded":"2024-04-25T10:04:13.896661+0000","last_fullsized":"2024-04-25T10:04:13.896661+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.904794+0000","last_change":"2024-04-25T10:04:13.898826+0000","last_active":"2024-04-25T10:04:13.904794+0000","last_peered":"2024-04-25T10:04:13.904794+0000","last_clean":"2024-04-25T10:04:13.904794+0000","last_became_active":"2024-04-25T10:04:13.898462+0000","last_became_peered":"2024-04-25T10:04:13.898462+0000","last_unstale":"2024-04-25T10:04:13.904794+0000","last_undegraded":"2024-04-25T10:04:13.904794+0000","last_fullsized":"2024-04-25T10:04:13.904794+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:30.864286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.898514+0000","last_change":"2024-04-25T10:04:13.898514+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.898514+0000","last_undegraded":"2024-04-25T10:04:13.898514+0000","last_fullsized":"2024-04-25T10:04:13.898514+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899831+0000","last_change":"2024-04-25T10:04:13.899083+0000","last_active":"2024-04-25T10:04:13.899831+0000","last_peered":"2024-04-25T10:04:13.899831+0000","last_clean":"2024-04-25T10:04:13.899831+0000","last_became_active":"2024-04-25T10:04:13.898810+0000","last_became_peered":"2024-04-25T10:04:13.898810+0000","last_unstale":"2024-04-25T10:04:13.899831+0000","last_undegraded":"2024-04-25T10:04:13.899831+0000","last_fullsized":"2024-04-25T10:04:13.899831+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:30.648919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.894391+0000","last_change":"2024-04-25T10:04:13.894391+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.894391+0000","last_undegraded":"2024-04-25T10:04:13.894391+0000","last_fullsized":"2024-04-25T10:04:13.894391+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.900693+0000","last_change":"2024-04-25T10:04:13.899508+0000","last_active":"2024-04-25T10:04:13.900693+0000","last_peered":"2024-04-25T10:04:13.900693+0000","last_clean":"2024-04-25T10:04:13.900693+0000","last_became_active":"2024-04-25T10:04:13.898618+0000","last_became_peered":"2024-04-25T10:04:13.898618+0000","last_unstale":"2024-04-25T10:04:13.900693+0000","last_undegraded":"2024-04-25T10:04:13.900693+0000","last_fullsized":"2024-04-25T10:04:13.900693+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:37.888496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.900372+0000","last_change":"2024-04-25T10:04:13.900372+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.900372+0000","last_undegraded":"2024-04-25T10:04:13.900372+0000","last_fullsized":"2024-04-25T10:04:13.900372+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.899333+0000","last_change":"2024-04-25T10:04:13.895684+0000","last_active":"2024-04-25T10:04:13.899333+0000","last_peered":"2024-04-25T10:04:13.899333+0000","last_clean":"2024-04-25T10:04:13.899333+0000","last_became_active":"2024-04-25T10:04:13.895390+0000","last_became_peered":"2024-04-25T10:04:13.895390+0000","last_unstale":"2024-04-25T10:04:13.899333+0000","last_undegraded":"2024-04-25T10:04:13.899333+0000","last_fullsized":"2024-04-25T10:04:13.899333+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:18.467823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888644+0000","last_change":"2024-04-25T10:04:13.888644+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888644+0000","last_undegraded":"2024-04-25T10:04:13.888644+0000","last_fullsized":"2024-04-25T10:04:13.888644+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913155+0000","last_change":"2024-04-25T10:04:13.911061+0000","last_active":"2024-04-25T10:04:13.913155+0000","last_peered":"2024-04-25T10:04:13.913155+0000","last_clean":"2024-04-25T10:04:13.913155+0000","last_became_active":"2024-04-25T10:04:13.910591+0000","last_became_peered":"2024-04-25T10:04:13.910591+0000","last_unstale":"2024-04-25T10:04:13.913155+0000","last_undegraded":"2024-04-25T10:04:13.913155+0000","last_fullsized":"2024-04-25T10:04:13.913155+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:40.288371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895546+0000","last_change":"2024-04-25T10:04:13.895546+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895546+0000","last_undegraded":"2024-04-25T10:04:13.895546+0000","last_fullsized":"2024-04-25T10:04:13.895546+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.913837+0000","last_change":"2024-04-25T10:04:13.913137+0000","last_active":"2024-04-25T10:04:13.913837+0000","last_peered":"2024-04-25T10:04:13.913837+0000","last_clean":"2024-04-25T10:04:13.913837+0000","last_became_active":"2024-04-25T10:04:13.912835+0000","last_became_peered":"2024-04-25T10:04:13.912835+0000","last_unstale":"2024-04-25T10:04:13.913837+0000","last_undegraded":"2024-04-25T10:04:13.913837+0000","last_fullsized":"2024-04-25T10:04:13.913837+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:25.196171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.895070+0000","last_change":"2024-04-25T10:04:13.895070+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.895070+0000","last_undegraded":"2024-04-25T10:04:13.895070+0000","last_fullsized":"2024-04-25T10:04:13.895070+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.897923+0000","last_change":"2024-04-25T10:04:13.896282+0000","last_active":"2024-04-25T10:04:13.897923+0000","last_peered":"2024-04-25T10:04:13.897923+0000","last_clean":"2024-04-25T10:04:13.897923+0000","last_became_active":"2024-04-25T10:04:13.894939+0000","last_became_peered":"2024-04-25T10:04:13.894939+0000","last_unstale":"2024-04-25T10:04:13.897923+0000","last_undegraded":"2024-04-25T10:04:13.897923+0000","last_fullsized":"2024-04-25T10:04:13.897923+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:17.612850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.889558+0000","last_change":"2024-04-25T10:04:13.889558+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.889558+0000","last_undegraded":"2024-04-25T10:04:13.889558+0000","last_fullsized":"2024-04-25T10:04:13.889558+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.910509+0000","last_change":"2024-04-25T10:04:13.910016+0000","last_active":"2024-04-25T10:04:13.910509+0000","last_peered":"2024-04-25T10:04:13.910509+0000","last_clean":"2024-04-25T10:04:13.910509+0000","last_became_active":"2024-04-25T10:04:13.909591+0000","last_became_peered":"2024-04-25T10:04:13.909591+0000","last_unstale":"2024-04-25T10:04:13.910509+0000","last_undegraded":"2024-04-25T10:04:13.910509+0000","last_fullsized":"2024-04-25T10:04:13.910509+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:31.416889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.901336+0000","last_change":"2024-04-25T10:04:13.901336+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.901336+0000","last_undegraded":"2024-04-25T10:04:13.901336+0000","last_fullsized":"2024-04-25T10:04:13.901336+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905824+0000","last_change":"2024-04-25T10:04:13.899980+0000","last_active":"2024-04-25T10:04:13.905824+0000","last_peered":"2024-04-25T10:04:13.905824+0000","last_clean":"2024-04-25T10:04:13.905824+0000","last_became_active":"2024-04-25T10:04:13.899671+0000","last_became_peered":"2024-04-25T10:04:13.899671+0000","last_unstale":"2024-04-25T10:04:13.905824+0000","last_undegraded":"2024-04-25T10:04:13.905824+0000","last_fullsized":"2024-04-25T10:04:13.905824+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:22.542468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.888539+0000","last_change":"2024-04-25T10:04:13.888539+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.888539+0000","last_undegraded":"2024-04-25T10:04:13.888539+0000","last_fullsized":"2024-04-25T10:04:13.888539+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.910748+0000","last_change":"2024-04-25T10:04:13.910423+0000","last_active":"2024-04-25T10:04:13.910748+0000","last_peered":"2024-04-25T10:04:13.910748+0000","last_clean":"2024-04-25T10:04:13.910748+0000","last_became_active":"2024-04-25T10:04:13.910113+0000","last_became_peered":"2024-04-25T10:04:13.910113+0000","last_unstale":"2024-04-25T10:04:13.910748+0000","last_undegraded":"2024-04-25T10:04:13.910748+0000","last_fullsized":"2024-04-25T10:04:13.910748+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:27.691750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":1,"reported_epoch":163,"state":"creating+peering","last_fresh":"2024-04-25T10:04:13.893539+0000","last_change":"2024-04-25T10:04:13.893539+0000","last_active":"2024-04-25T10:04:13.879667+0000","last_peered":"2024-04-25T10:04:13.879667+0000","last_clean":"2024-04-25T10:04:13.879667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:13.893539+0000","last_undegraded":"2024-04-25T10:04:13.893539+0000","last_fullsized":"2024-04-25T10:04:13.893539+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.905911+0000","last_change":"2024-04-25T10:04:13.903829+0000","last_active":"2024-04-25T10:04:13.905911+0000","last_peered":"2024-04-25T10:04:13.905911+0000","last_clean":"2024-04-25T10:04:13.905911+0000","last_became_active":"2024-04-25T10:04:13.901705+0000","last_became_peered":"2024-04-25T10:04:13.901705+0000","last_unstale":"2024-04-25T10:04:13.905911+0000","last_undegraded":"2024-04-25T10:04:13.905911+0000","last_fullsized":"2024-04-25T10:04:13.905911+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:17.450538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":461,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.888932+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:04:13.888932+0000","last_peered":"2024-04-25T10:04:13.888932+0000","last_clean":"2024-04-25T10:04:13.888932+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:04:13.888932+0000","last_undegraded":"2024-04-25T10:04:13.888932+0000","last_fullsized":"2024-04-25T10:04:13.888932+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51820,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320020,"statfs":{"total":96636764160,"available":96583700480,"internally_reserved":0,"allocated":2650112,"data_stored":1552487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574988,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51224,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320616,"statfs":{"total":96636764160,"available":96584310784,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51236,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320604,"statfs":{"total":96636764160,"available":96584298496,"internally_reserved":0,"allocated":2056192,"data_stored":962119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59560,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312280,"statfs":{"total":96636764160,"available":96575774720,"internally_reserved":0,"allocated":2056192,"data_stored":962119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59544,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312296,"statfs":{"total":96636764160,"available":96575791104,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55392,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316448,"statfs":{"total":96636764160,"available":96580042752,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55388,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316452,"statfs":{"total":96636764160,"available":96580046848,"internally_reserved":0,"allocated":2043904,"data_stored":955452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55964,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315876,"statfs":{"total":96636764160,"available":96579457024,"internally_reserved":0,"allocated":2637824,"data_stored":1545820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3f is not active+clean 2024-04-25T10:04:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.906917+0000', 'last_change': '2024-04-25T10:04:13.906917+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.906917+0000', 'last_undegraded': '2024-04-25T10:04:13.906917+0000', 'last_fullsized': '2024-04-25T10:04:13.906917+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3e is not active+clean 2024-04-25T10:04:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.906400+0000', 'last_change': '2024-04-25T10:04:13.906400+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.906400+0000', 'last_undegraded': '2024-04-25T10:04:13.906400+0000', 'last_fullsized': '2024-04-25T10:04:13.906400+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3d is not active+clean 2024-04-25T10:04:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.894277+0000', 'last_change': '2024-04-25T10:04:13.894277+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.894277+0000', 'last_undegraded': '2024-04-25T10:04:13.894277+0000', 'last_fullsized': '2024-04-25T10:04:13.894277+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:04:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3c is not active+clean 2024-04-25T10:04:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.899956+0000', 'last_change': '2024-04-25T10:04:13.899956+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.899956+0000', 'last_undegraded': '2024-04-25T10:04:13.899956+0000', 'last_fullsized': '2024-04-25T10:04:13.899956+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:04:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3b is not active+clean 2024-04-25T10:04:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.907015+0000', 'last_change': '2024-04-25T10:04:13.907015+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.907015+0000', 'last_undegraded': '2024-04-25T10:04:13.907015+0000', 'last_fullsized': '2024-04-25T10:04:13.907015+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3a is not active+clean 2024-04-25T10:04:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.905958+0000', 'last_change': '2024-04-25T10:04:13.905958+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.905958+0000', 'last_undegraded': '2024-04-25T10:04:13.905958+0000', 'last_fullsized': '2024-04-25T10:04:13.905958+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.39 is not active+clean 2024-04-25T10:04:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.898174+0000', 'last_change': '2024-04-25T10:04:13.898174+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.898174+0000', 'last_undegraded': '2024-04-25T10:04:13.898174+0000', 'last_fullsized': '2024-04-25T10:04:13.898174+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.38 is not active+clean 2024-04-25T10:04:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.888128+0000', 'last_change': '2024-04-25T10:04:13.888128+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.888128+0000', 'last_undegraded': '2024-04-25T10:04:13.888128+0000', 'last_fullsized': '2024-04-25T10:04:13.888128+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:04:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.37 is not active+clean 2024-04-25T10:04:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.906338+0000', 'last_change': '2024-04-25T10:04:13.906338+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.906338+0000', 'last_undegraded': '2024-04-25T10:04:13.906338+0000', 'last_fullsized': '2024-04-25T10:04:13.906338+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.36 is not active+clean 2024-04-25T10:04:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.904643+0000', 'last_change': '2024-04-25T10:04:13.904643+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.904643+0000', 'last_undegraded': '2024-04-25T10:04:13.904643+0000', 'last_fullsized': '2024-04-25T10:04:13.904643+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.35 is not active+clean 2024-04-25T10:04:18.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.894877+0000', 'last_change': '2024-04-25T10:04:13.894877+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.894877+0000', 'last_undegraded': '2024-04-25T10:04:13.894877+0000', 'last_fullsized': '2024-04-25T10:04:13.894877+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:04:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.34 is not active+clean 2024-04-25T10:04:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.897174+0000', 'last_change': '2024-04-25T10:04:13.897174+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.897174+0000', 'last_undegraded': '2024-04-25T10:04:13.897174+0000', 'last_fullsized': '2024-04-25T10:04:13.897174+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:04:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.33 is not active+clean 2024-04-25T10:04:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.891816+0000', 'last_change': '2024-04-25T10:04:13.891816+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.891816+0000', 'last_undegraded': '2024-04-25T10:04:13.891816+0000', 'last_fullsized': '2024-04-25T10:04:13.891816+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:04:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.32 is not active+clean 2024-04-25T10:04:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.897714+0000', 'last_change': '2024-04-25T10:04:13.897714+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.897714+0000', 'last_undegraded': '2024-04-25T10:04:13.897714+0000', 'last_fullsized': '2024-04-25T10:04:13.897714+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.31 is not active+clean 2024-04-25T10:04:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.904789+0000', 'last_change': '2024-04-25T10:04:13.904789+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.904789+0000', 'last_undegraded': '2024-04-25T10:04:13.904789+0000', 'last_fullsized': '2024-04-25T10:04:13.904789+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.30 is not active+clean 2024-04-25T10:04:18.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.903622+0000', 'last_change': '2024-04-25T10:04:13.903622+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.903622+0000', 'last_undegraded': '2024-04-25T10:04:13.903622+0000', 'last_fullsized': '2024-04-25T10:04:13.903622+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2f is not active+clean 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.898964+0000', 'last_change': '2024-04-25T10:04:13.898964+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.898964+0000', 'last_undegraded': '2024-04-25T10:04:13.898964+0000', 'last_fullsized': '2024-04-25T10:04:13.898964+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2e is not active+clean 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.891283+0000', 'last_change': '2024-04-25T10:04:13.891283+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.891283+0000', 'last_undegraded': '2024-04-25T10:04:13.891283+0000', 'last_fullsized': '2024-04-25T10:04:13.891283+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2d is not active+clean 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.889985+0000', 'last_change': '2024-04-25T10:04:13.889985+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.889985+0000', 'last_undegraded': '2024-04-25T10:04:13.889985+0000', 'last_fullsized': '2024-04-25T10:04:13.889985+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2c is not active+clean 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.903899+0000', 'last_change': '2024-04-25T10:04:13.903899+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.903899+0000', 'last_undegraded': '2024-04-25T10:04:13.903899+0000', 'last_fullsized': '2024-04-25T10:04:13.903899+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2b is not active+clean 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.904325+0000', 'last_change': '2024-04-25T10:04:13.904325+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.904325+0000', 'last_undegraded': '2024-04-25T10:04:13.904325+0000', 'last_fullsized': '2024-04-25T10:04:13.904325+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2a is not active+clean 2024-04-25T10:04:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.904219+0000', 'last_change': '2024-04-25T10:04:13.904219+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.904219+0000', 'last_undegraded': '2024-04-25T10:04:13.904219+0000', 'last_fullsized': '2024-04-25T10:04:13.904219+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.29 is not active+clean 2024-04-25T10:04:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.888564+0000', 'last_change': '2024-04-25T10:04:13.888564+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.888564+0000', 'last_undegraded': '2024-04-25T10:04:13.888564+0000', 'last_fullsized': '2024-04-25T10:04:13.888564+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:04:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.28 is not active+clean 2024-04-25T10:04:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.903639+0000', 'last_change': '2024-04-25T10:04:13.903639+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.903639+0000', 'last_undegraded': '2024-04-25T10:04:13.903639+0000', 'last_fullsized': '2024-04-25T10:04:13.903639+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.27 is not active+clean 2024-04-25T10:04:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.898089+0000', 'last_change': '2024-04-25T10:04:13.898089+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.898089+0000', 'last_undegraded': '2024-04-25T10:04:13.898089+0000', 'last_fullsized': '2024-04-25T10:04:13.898089+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.26 is not active+clean 2024-04-25T10:04:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.896303+0000', 'last_change': '2024-04-25T10:04:13.896303+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.896303+0000', 'last_undegraded': '2024-04-25T10:04:13.896303+0000', 'last_fullsized': '2024-04-25T10:04:13.896303+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:04:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.25 is not active+clean 2024-04-25T10:04:18.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.888597+0000', 'last_change': '2024-04-25T10:04:13.888597+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.888597+0000', 'last_undegraded': '2024-04-25T10:04:13.888597+0000', 'last_fullsized': '2024-04-25T10:04:13.888597+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.24 is not active+clean 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.899739+0000', 'last_change': '2024-04-25T10:04:13.899739+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.899739+0000', 'last_undegraded': '2024-04-25T10:04:13.899739+0000', 'last_fullsized': '2024-04-25T10:04:13.899739+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.23 is not active+clean 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.894259+0000', 'last_change': '2024-04-25T10:04:13.894259+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.894259+0000', 'last_undegraded': '2024-04-25T10:04:13.894259+0000', 'last_fullsized': '2024-04-25T10:04:13.894259+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.22 is not active+clean 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.893981+0000', 'last_change': '2024-04-25T10:04:13.893981+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.893981+0000', 'last_undegraded': '2024-04-25T10:04:13.893981+0000', 'last_fullsized': '2024-04-25T10:04:13.893981+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.21 is not active+clean 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.897098+0000', 'last_change': '2024-04-25T10:04:13.897098+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.897098+0000', 'last_undegraded': '2024-04-25T10:04:13.897098+0000', 'last_fullsized': '2024-04-25T10:04:13.897098+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.20 is not active+clean 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.895659+0000', 'last_change': '2024-04-25T10:04:13.895659+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.895659+0000', 'last_undegraded': '2024-04-25T10:04:13.895659+0000', 'last_fullsized': '2024-04-25T10:04:13.895659+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1c is not active+clean 2024-04-25T10:04:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.888515+0000', 'last_change': '2024-04-25T10:04:13.888515+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.888515+0000', 'last_undegraded': '2024-04-25T10:04:13.888515+0000', 'last_fullsized': '2024-04-25T10:04:13.888515+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:04:18.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1d is not active+clean 2024-04-25T10:04:18.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.897689+0000', 'last_change': '2024-04-25T10:04:13.897689+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.897689+0000', 'last_undegraded': '2024-04-25T10:04:13.897689+0000', 'last_fullsized': '2024-04-25T10:04:13.897689+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:18.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1e is not active+clean 2024-04-25T10:04:18.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.895862+0000', 'last_change': '2024-04-25T10:04:13.895862+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.895862+0000', 'last_undegraded': '2024-04-25T10:04:13.895862+0000', 'last_fullsized': '2024-04-25T10:04:13.895862+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:04:18.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1f is not active+clean 2024-04-25T10:04:18.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.893994+0000', 'last_change': '2024-04-25T10:04:13.893994+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.893994+0000', 'last_undegraded': '2024-04-25T10:04:13.893994+0000', 'last_fullsized': '2024-04-25T10:04:13.893994+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:04:18.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.18 is not active+clean 2024-04-25T10:04:18.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.889782+0000', 'last_change': '2024-04-25T10:04:13.889782+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.889782+0000', 'last_undegraded': '2024-04-25T10:04:13.889782+0000', 'last_fullsized': '2024-04-25T10:04:13.889782+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:04:18.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.19 is not active+clean 2024-04-25T10:04:18.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.902584+0000', 'last_change': '2024-04-25T10:04:13.902584+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.902584+0000', 'last_undegraded': '2024-04-25T10:04:13.902584+0000', 'last_fullsized': '2024-04-25T10:04:13.902584+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1a is not active+clean 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.903095+0000', 'last_change': '2024-04-25T10:04:13.903095+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.903095+0000', 'last_undegraded': '2024-04-25T10:04:13.903095+0000', 'last_fullsized': '2024-04-25T10:04:13.903095+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.14 is not active+clean 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.887704+0000', 'last_change': '2024-04-25T10:04:13.887704+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.887704+0000', 'last_undegraded': '2024-04-25T10:04:13.887704+0000', 'last_fullsized': '2024-04-25T10:04:13.887704+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.15 is not active+clean 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.897391+0000', 'last_change': '2024-04-25T10:04:13.897391+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.897391+0000', 'last_undegraded': '2024-04-25T10:04:13.897391+0000', 'last_fullsized': '2024-04-25T10:04:13.897391+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.16 is not active+clean 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.890204+0000', 'last_change': '2024-04-25T10:04:13.890204+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.890204+0000', 'last_undegraded': '2024-04-25T10:04:13.890204+0000', 'last_fullsized': '2024-04-25T10:04:13.890204+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.17 is not active+clean 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.902109+0000', 'last_change': '2024-04-25T10:04:13.902109+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.902109+0000', 'last_undegraded': '2024-04-25T10:04:13.902109+0000', 'last_fullsized': '2024-04-25T10:04:13.902109+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.10 is not active+clean 2024-04-25T10:04:18.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.896588+0000', 'last_change': '2024-04-25T10:04:13.896588+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.896588+0000', 'last_undegraded': '2024-04-25T10:04:13.896588+0000', 'last_fullsized': '2024-04-25T10:04:13.896588+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.11 is not active+clean 2024-04-25T10:04:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.894510+0000', 'last_change': '2024-04-25T10:04:13.894510+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.894510+0000', 'last_undegraded': '2024-04-25T10:04:13.894510+0000', 'last_fullsized': '2024-04-25T10:04:13.894510+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.12 is not active+clean 2024-04-25T10:04:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.890080+0000', 'last_change': '2024-04-25T10:04:13.890080+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.890080+0000', 'last_undegraded': '2024-04-25T10:04:13.890080+0000', 'last_fullsized': '2024-04-25T10:04:13.890080+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:04:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.13 is not active+clean 2024-04-25T10:04:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.890799+0000', 'last_change': '2024-04-25T10:04:13.890799+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.890799+0000', 'last_undegraded': '2024-04-25T10:04:13.890799+0000', 'last_fullsized': '2024-04-25T10:04:13.890799+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:04:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.c is not active+clean 2024-04-25T10:04:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.896084+0000', 'last_change': '2024-04-25T10:04:13.896084+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.896084+0000', 'last_undegraded': '2024-04-25T10:04:13.896084+0000', 'last_fullsized': '2024-04-25T10:04:13.896084+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.d is not active+clean 2024-04-25T10:04:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.896982+0000', 'last_change': '2024-04-25T10:04:13.896982+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.896982+0000', 'last_undegraded': '2024-04-25T10:04:13.896982+0000', 'last_fullsized': '2024-04-25T10:04:13.896982+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.e is not active+clean 2024-04-25T10:04:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.895399+0000', 'last_change': '2024-04-25T10:04:13.895399+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.895399+0000', 'last_undegraded': '2024-04-25T10:04:13.895399+0000', 'last_fullsized': '2024-04-25T10:04:13.895399+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:04:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.f is not active+clean 2024-04-25T10:04:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.901364+0000', 'last_change': '2024-04-25T10:04:13.901364+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.901364+0000', 'last_undegraded': '2024-04-25T10:04:13.901364+0000', 'last_fullsized': '2024-04-25T10:04:13.901364+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.8 is not active+clean 2024-04-25T10:04:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.900952+0000', 'last_change': '2024-04-25T10:04:13.900952+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.900952+0000', 'last_undegraded': '2024-04-25T10:04:13.900952+0000', 'last_fullsized': '2024-04-25T10:04:13.900952+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.9 is not active+clean 2024-04-25T10:04:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.896135+0000', 'last_change': '2024-04-25T10:04:13.896135+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.896135+0000', 'last_undegraded': '2024-04-25T10:04:13.896135+0000', 'last_fullsized': '2024-04-25T10:04:13.896135+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:04:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.a is not active+clean 2024-04-25T10:04:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.896661+0000', 'last_change': '2024-04-25T10:04:13.896661+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.896661+0000', 'last_undegraded': '2024-04-25T10:04:13.896661+0000', 'last_fullsized': '2024-04-25T10:04:13.896661+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:04:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.b is not active+clean 2024-04-25T10:04:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.898514+0000', 'last_change': '2024-04-25T10:04:13.898514+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.898514+0000', 'last_undegraded': '2024-04-25T10:04:13.898514+0000', 'last_fullsized': '2024-04-25T10:04:13.898514+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:18.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.4 is not active+clean 2024-04-25T10:04:18.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.894391+0000', 'last_change': '2024-04-25T10:04:13.894391+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.894391+0000', 'last_undegraded': '2024-04-25T10:04:13.894391+0000', 'last_fullsized': '2024-04-25T10:04:13.894391+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:18.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.5 is not active+clean 2024-04-25T10:04:18.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.900372+0000', 'last_change': '2024-04-25T10:04:13.900372+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.900372+0000', 'last_undegraded': '2024-04-25T10:04:13.900372+0000', 'last_fullsized': '2024-04-25T10:04:13.900372+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.6 is not active+clean 2024-04-25T10:04:18.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.888644+0000', 'last_change': '2024-04-25T10:04:13.888644+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.888644+0000', 'last_undegraded': '2024-04-25T10:04:13.888644+0000', 'last_fullsized': '2024-04-25T10:04:13.888644+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:04:18.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.7 is not active+clean 2024-04-25T10:04:18.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.895546+0000', 'last_change': '2024-04-25T10:04:13.895546+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.895546+0000', 'last_undegraded': '2024-04-25T10:04:13.895546+0000', 'last_fullsized': '2024-04-25T10:04:13.895546+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:18.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.0 is not active+clean 2024-04-25T10:04:18.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.895070+0000', 'last_change': '2024-04-25T10:04:13.895070+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.895070+0000', 'last_undegraded': '2024-04-25T10:04:13.895070+0000', 'last_fullsized': '2024-04-25T10:04:13.895070+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:04:18.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1 is not active+clean 2024-04-25T10:04:18.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.889558+0000', 'last_change': '2024-04-25T10:04:13.889558+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.889558+0000', 'last_undegraded': '2024-04-25T10:04:13.889558+0000', 'last_fullsized': '2024-04-25T10:04:13.889558+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:04:18.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2 is not active+clean 2024-04-25T10:04:18.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.901336+0000', 'last_change': '2024-04-25T10:04:13.901336+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.901336+0000', 'last_undegraded': '2024-04-25T10:04:13.901336+0000', 'last_fullsized': '2024-04-25T10:04:13.901336+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:18.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3 is not active+clean 2024-04-25T10:04:18.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.888539+0000', 'last_change': '2024-04-25T10:04:13.888539+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.888539+0000', 'last_undegraded': '2024-04-25T10:04:13.888539+0000', 'last_fullsized': '2024-04-25T10:04:13.888539+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:04:18.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1b is not active+clean 2024-04-25T10:04:18.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 163, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:13.893539+0000', 'last_change': '2024-04-25T10:04:13.893539+0000', 'last_active': '2024-04-25T10:04:13.879667+0000', 'last_peered': '2024-04-25T10:04:13.879667+0000', 'last_clean': '2024-04-25T10:04:13.879667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:13.893539+0000', 'last_undegraded': '2024-04-25T10:04:13.893539+0000', 'last_fullsized': '2024-04-25T10:04:13.893539+0000', 'mapping_epoch': 163, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 163, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:13.879667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:04:18.617 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:04:18.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.734+0000 7fcf4d9d5640 1 Processor -- start 2024-04-25T10:04:18.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.734+0000 7fcf4d9d5640 1 -- start start 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf4d9d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf480b9730 0x7fcf480ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf4d9d5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf48161d50 con 0x7fcf480ceb90 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf4d9d5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf480cf000 con 0x7fcf4815da70 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf4d9d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf481625e0 con 0x7fcf480b9730 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf477fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcf480ceb90 0x7fcf480d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57576/0 (socket says 172.21.15.19:57576) 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf477fe640 1 -- 172.21.15.19:0/2389338158 learned_addr learned my addr 172.21.15.19:0/2389338158 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf467fc640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf480b9730 0x7fcf480ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690158676 0 0) 0x7fcf48161d50 con 0x7fcf480ceb90 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf30003520 con 0x7fcf480ceb90 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110898023 0 0) 0x7fcf480cf000 con 0x7fcf4815da70 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf48161d50 con 0x7fcf4815da70 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926192662 0 0) 0x7fcf30003520 con 0x7fcf480ceb90 2024-04-25T10:04:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.735+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf480b9730 msgr2=0x7fcf480ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf45ffb640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf480b9730 0x7fcf480ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 >> v1:172.21.15.112:6789/0 conn(0x7fcf4815da70 legacy=0x7fcf48161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf48162020 con 0x7fcf480ceb90 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fcf400019f0 con 0x7fcf480ceb90 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 >> v1:172.21.15.19:6789/0 conn(0x7fcf480ceb90 legacy=0x7fcf480d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf45ffb640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf30003b70 0x7fcf30003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf45ffb640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf30004800 0x7fcf30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf45ffb640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf300051c0 0x7fcf3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf481625e0 con 0x7fcf30004800 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3000bf30 con 0x7fcf300051c0 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3000c130 con 0x7fcf30003b70 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf467fc640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf30004800 0x7fcf30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf46ffd640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf30003b70 0x7fcf30003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf477fe640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf300051c0 0x7fcf3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf467fc640 1 -- 172.21.15.19:0/2389338158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf30003b70 msgr2=0x7fcf30003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf467fc640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf30003b70 0x7fcf30003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf467fc640 1 -- 172.21.15.19:0/2389338158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf300051c0 msgr2=0x7fcf3000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf467fc640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf300051c0 0x7fcf3000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.736+0000 7fcf467fc640 1 -- 172.21.15.19:0/2389338158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf30003520 con 0x7fcf30004800 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.737+0000 7fcf467fc640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf30004800 0x7fcf30004c80 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009a90 tx=0x7fcf30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9730c4142871920e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.737+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c036070 con 0x7fcf30004800 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.737+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c031400 con 0x7fcf30004800 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.737+0000 7fcf45ffb640 1 -- 172.21.15.19:0/2389338158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c009690 con 0x7fcf30004800 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.737+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/2389338158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf30004800 msgr2=0x7fcf30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.737+0000 7fcf4d9d5640 1 --2- 172.21.15.19:0/2389338158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf30004800 0x7fcf30004c80 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009a90 tx=0x7fcf30005830 comp rx=0 tx=0).stop 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.737+0000 7fcf477fe640 1 -- 172.21.15.19:0/2389338158 reap_dead start 2024-04-25T10:04:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.737+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/2389338158 shutdown_connections 2024-04-25T10:04:18.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.737+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/2389338158 >> 172.21.15.19:0/2389338158 conn(0x7fcf48061a20 msgr2=0x7fcf48061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.738+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/2389338158 shutdown_connections 2024-04-25T10:04:18.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.738+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/2389338158 wait complete. 2024-04-25T10:04:18.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.739+0000 7fcf4d9d5640 1 Processor -- start 2024-04-25T10:04:18.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.739+0000 7fcf4d9d5640 1 -- start start 2024-04-25T10:04:18.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.739+0000 7fcf4d9d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf30004800 0x7fcf48157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf4d9d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4815f420 0x7fcf4815f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf4d9d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf480b4980 0x7fcf480b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf4d9d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480d1dd0 con 0x7fcf480b4980 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf4d9d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480d1af0 con 0x7fcf4815f420 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf4d9d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4815de50 con 0x7fcf30004800 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf480b4980 0x7fcf480b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf477fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf480b4980 0x7fcf480b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56978/0 (socket says 172.21.15.19:56978) 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf477fe640 1 -- 172.21.15.19:0/327328018 learned_addr learned my addr 172.21.15.19:0/327328018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf46ffd640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf30004800 0x7fcf48157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf467fc640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4815f420 0x7fcf4815f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf477fe640 1 -- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf30004800 msgr2=0x7fcf48157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf477fe640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf30004800 0x7fcf48157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf477fe640 1 -- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4815f420 msgr2=0x7fcf4815f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf477fe640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4815f420 0x7fcf4815f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf477fe640 1 -- 172.21.15.19:0/327328018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4815fe70 con 0x7fcf480b4980 2024-04-25T10:04:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf467fc640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4815f420 0x7fcf4815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf46ffd640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf30004800 0x7fcf48157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf477fe640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf480b4980 0x7fcf480b4de0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fcf40000f80 tx=0x7fcf4000e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf2ffff640 1 -- 172.21.15.19:0/327328018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4000eb70 con 0x7fcf480b4980 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf2ffff640 1 -- 172.21.15.19:0/327328018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf40018070 con 0x7fcf480b4980 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf2ffff640 1 -- 172.21.15.19:0/327328018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40013420 con 0x7fcf480b4980 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/327328018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf48160150 con 0x7fcf480b4980 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.740+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/327328018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf48160710 con 0x7fcf480b4980 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.741+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/327328018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf48073990 con 0x7fcf480b4980 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.742+0000 7fcf2ffff640 1 -- 172.21.15.19:0/327328018 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf400135c0 con 0x7fcf480b4980 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.742+0000 7fcf2ffff640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf3406b4d0 0x7fcf3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.742+0000 7fcf2ffff640 1 -- 172.21.15.19:0/327328018 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcf4008a6c0 con 0x7fcf480b4980 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.742+0000 7fcf46ffd640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf3406b4d0 0x7fcf3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.743+0000 7fcf46ffd640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf3406b4d0 0x7fcf3406d990 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcf380035f0 tx=0x7fcf38009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.746+0000 7fcf2ffff640 1 -- 172.21.15.19:0/327328018 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf40059420 con 0x7fcf480b4980 2024-04-25T10:04:19.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.998+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/327328018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcf480d1570 con 0x7fcf480b4980 2024-04-25T10:04:19.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:18.999+0000 7fcf2ffff640 1 -- 172.21.15.19:0/327328018 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1330 (secure 0 0 0) 0x7fcf40058d20 con 0x7fcf480b4980 2024-04-25T10:04:19.005 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":696,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":165,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":450691072,"bytes_avail":772643422208,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":109,"id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"c","status":"up:active","gid":10698}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-04-25T10:04:11.067289+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:04:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf3406b4d0 msgr2=0x7fcf3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf3406b4d0 0x7fcf3406d990 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcf380035f0 tx=0x7fcf38009040 comp rx=0 tx=0).stop 2024-04-25T10:04:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/327328018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf480b4980 msgr2=0x7fcf480b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf480b4980 0x7fcf480b4de0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fcf40000f80 tx=0x7fcf4000e160 comp rx=0 tx=0).stop 2024-04-25T10:04:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/327328018 shutdown_connections 2024-04-25T10:04:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcf3406b4d0 0x7fcf3406d990 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf480b4980 0x7fcf480b4de0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4815f420 0x7fcf4815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 --2- 172.21.15.19:0/327328018 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf30004800 0x7fcf48157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/327328018 >> 172.21.15.19:0/327328018 conn(0x7fcf48061a20 msgr2=0x7fcf480d3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/327328018 shutdown_connections 2024-04-25T10:04:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.001+0000 7fcf4d9d5640 1 -- 172.21.15.19:0/327328018 wait complete. 2024-04-25T10:04:19.018 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:04:19.018 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.113+0000 7f0c9865a640 1 Processor -- start 2024-04-25T10:04:19.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.113+0000 7f0c9865a640 1 -- start start 2024-04-25T10:04:19.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.114+0000 7f0c9865a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c900b3410 0x7f0c900720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.114+0000 7f0c9865a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c900ace70 con 0x7f0c900b37f0 2024-04-25T10:04:19.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.114+0000 7f0c9865a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c900b43b0 con 0x7f0c9006e3c0 2024-04-25T10:04:19.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.114+0000 7f0c9865a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9006df30 con 0x7f0c900b3410 2024-04-25T10:04:19.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.114+0000 7f0c963f6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c900b3410 0x7f0c900720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.114+0000 7f0c95bf5640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0c900b37f0 0x7f0c900b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57582/0 (socket says 172.21.15.19:57582) 2024-04-25T10:04:19.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.114+0000 7f0c95bf5640 1 -- 172.21.15.19:0/788247522 learned_addr learned my addr 172.21.15.19:0/788247522 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:19.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.114+0000 7f0c963f6640 1 -- 172.21.15.19:0/788247522 >> v1:172.21.15.112:6789/0 conn(0x7f0c9006e3c0 legacy=0x7f0c9006e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:19.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.114+0000 7f0c963f6640 1 -- 172.21.15.19:0/788247522 >> v1:172.21.15.19:6789/0 conn(0x7f0c900b37f0 legacy=0x7f0c900b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.114+0000 7f0c963f6640 1 -- 172.21.15.19:0/788247522 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c900726d0 con 0x7f0c900b3410 2024-04-25T10:04:19.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.114+0000 7f0c953f4640 1 -- 172.21.15.19:0/788247522 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933271655 0 0) 0x7f0c80001110 con 0x7f0c900b37f0 2024-04-25T10:04:19.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.115+0000 7f0c963f6640 1 --2- 172.21.15.19:0/788247522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c900b3410 0x7f0c900720f0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0097a0 tx=0x7f0c8c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d80fdb0e9351e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.115+0000 7f0c953f4640 1 -- 172.21.15.19:0/788247522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c037070 con 0x7f0c900b3410 2024-04-25T10:04:19.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.115+0000 7f0c953f4640 1 -- 172.21.15.19:0/788247522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8c032440 con 0x7f0c900b3410 2024-04-25T10:04:19.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.115+0000 7f0c953f4640 1 -- 172.21.15.19:0/788247522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c032810 con 0x7f0c900b3410 2024-04-25T10:04:19.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.116+0000 7f0c9865a640 1 -- 172.21.15.19:0/788247522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c900b3410 msgr2=0x7f0c900720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.116+0000 7f0c9865a640 1 --2- 172.21.15.19:0/788247522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c900b3410 0x7f0c900720f0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0097a0 tx=0x7f0c8c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.116+0000 7f0c9865a640 1 -- 172.21.15.19:0/788247522 shutdown_connections 2024-04-25T10:04:19.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.116+0000 7f0c9865a640 1 --2- 172.21.15.19:0/788247522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c900b3410 0x7f0c900720f0 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.116+0000 7f0c9865a640 1 -- 172.21.15.19:0/788247522 >> 172.21.15.19:0/788247522 conn(0x7f0c900cfae0 msgr2=0x7f0c900cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.116+0000 7f0c9865a640 1 -- 172.21.15.19:0/788247522 shutdown_connections 2024-04-25T10:04:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.116+0000 7f0c9865a640 1 -- 172.21.15.19:0/788247522 wait complete. 2024-04-25T10:04:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.117+0000 7f0c9865a640 1 Processor -- start 2024-04-25T10:04:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.117+0000 7f0c9865a640 1 -- start start 2024-04-25T10:04:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c9865a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c9006e3c0 0x7f0c900c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c9865a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c900b3410 0x7f0c90144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c9865a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c900b37f0 0x7f0c90145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c9865a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c90147890 con 0x7f0c900b37f0 2024-04-25T10:04:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c9865a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9006dbc0 con 0x7f0c900b3410 2024-04-25T10:04:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c9865a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c90147230 con 0x7f0c9006e3c0 2024-04-25T10:04:19.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c96bf7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c900b37f0 0x7f0c90145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c96bf7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c900b37f0 0x7f0c90145220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57000/0 (socket says 172.21.15.19:57000) 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c96bf7640 1 -- 172.21.15.19:0/3261142895 learned_addr learned my addr 172.21.15.19:0/3261142895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c95bf5640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c900b3410 0x7f0c90144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c96bf7640 1 -- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c9006e3c0 msgr2=0x7f0c900c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c96bf7640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c9006e3c0 0x7f0c900c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c96bf7640 1 -- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c900b3410 msgr2=0x7f0c90144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c96bf7640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c900b3410 0x7f0c90144ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c96bf7640 1 -- 172.21.15.19:0/3261142895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9012f4e0 con 0x7f0c900b37f0 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c96bf7640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c900b37f0 0x7f0c90145220 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800c780 tx=0x7f0c8800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c86ffd640 1 -- 172.21.15.19:0/3261142895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88017070 con 0x7f0c900b37f0 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c86ffd640 1 -- 172.21.15.19:0/3261142895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c88003da0 con 0x7f0c900b37f0 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c9865a640 1 -- 172.21.15.19:0/3261142895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c900c55e0 con 0x7f0c900b37f0 2024-04-25T10:04:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c86ffd640 1 -- 172.21.15.19:0/3261142895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c880052b0 con 0x7f0c900b37f0 2024-04-25T10:04:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.118+0000 7f0c9865a640 1 -- 172.21.15.19:0/3261142895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9012fa60 con 0x7f0c900b37f0 2024-04-25T10:04:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.119+0000 7f0c9865a640 1 -- 172.21.15.19:0/3261142895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c90073990 con 0x7f0c900b37f0 2024-04-25T10:04:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.120+0000 7f0c86ffd640 1 -- 172.21.15.19:0/3261142895 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c8801a070 con 0x7f0c900b37f0 2024-04-25T10:04:19.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.120+0000 7f0c86ffd640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c7406b4d0 0x7f0c7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.120+0000 7f0c86ffd640 1 -- 172.21.15.19:0/3261142895 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c88089980 con 0x7f0c900b37f0 2024-04-25T10:04:19.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.120+0000 7f0c963f6640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c7406b4d0 0x7f0c7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.121+0000 7f0c963f6640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c7406b4d0 0x7f0c7406d990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c034000 tx=0x7f0c8c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.124+0000 7f0c86ffd640 1 -- 172.21.15.19:0/3261142895 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c88058700 con 0x7f0c900b37f0 2024-04-25T10:04:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.301+0000 7f0c9865a640 1 -- 172.21.15.19:0/3261142895 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c900ae410 con 0x7f0c7406b4d0 2024-04-25T10:04:19.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.307+0000 7f0c86ffd640 1 -- 172.21.15.19:0/3261142895 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252472 (secure 0 0 0) 0x7f0c900ae410 con 0x7f0c7406b4d0 2024-04-25T10:04:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:19.317 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:04:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 -- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c7406b4d0 msgr2=0x7f0c7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c7406b4d0 0x7f0c7406d990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c034000 tx=0x7f0c8c02c000 comp rx=0 tx=0).stop 2024-04-25T10:04:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 -- 172.21.15.19:0/3261142895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c900b37f0 msgr2=0x7f0c90145220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c900b37f0 0x7f0c90145220 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800c780 tx=0x7f0c8800cc40 comp rx=0 tx=0).stop 2024-04-25T10:04:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 -- 172.21.15.19:0/3261142895 shutdown_connections 2024-04-25T10:04:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0c7406b4d0 0x7f0c7406d990 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c900b37f0 0x7f0c90145220 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c900b3410 0x7f0c90144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 --2- 172.21.15.19:0/3261142895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c9006e3c0 0x7f0c900c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 -- 172.21.15.19:0/3261142895 >> 172.21.15.19:0/3261142895 conn(0x7f0c900cfae0 msgr2=0x7f0c90146530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 -- 172.21.15.19:0/3261142895 shutdown_connections 2024-04-25T10:04:19.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:19.310+0000 7f0c9865a640 1 -- 172.21.15.19:0/3261142895 wait complete. 2024-04-25T10:04:19.327 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-04-25T10:04:19.068971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440452,"kb_used_data":17480,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754534268,"statfs":{"total":773094113280,"available":772643090432,"internally_reserved":0,"allocated":17899520,"data_stored":8993380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001231"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238494+0000","last_change":"2024-04-25T10:04:14.914827+0000","last_active":"2024-04-25T10:04:16.238494+0000","last_peered":"2024-04-25T10:04:16.238494+0000","last_clean":"2024-04-25T10:04:16.238494+0000","last_became_active":"2024-04-25T10:04:14.914441+0000","last_became_peered":"2024-04-25T10:04:14.914441+0000","last_unstale":"2024-04-25T10:04:16.238494+0000","last_undegraded":"2024-04-25T10:04:16.238494+0000","last_fullsized":"2024-04-25T10:04:16.238494+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:20.833068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237589+0000","last_change":"2024-04-25T10:04:14.923366+0000","last_active":"2024-04-25T10:04:16.237589+0000","last_peered":"2024-04-25T10:04:16.237589+0000","last_clean":"2024-04-25T10:04:16.237589+0000","last_became_active":"2024-04-25T10:04:14.923076+0000","last_became_peered":"2024-04-25T10:04:14.923076+0000","last_unstale":"2024-04-25T10:04:16.237589+0000","last_undegraded":"2024-04-25T10:04:16.237589+0000","last_fullsized":"2024-04-25T10:04:16.237589+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:38.890605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237663+0000","last_change":"2024-04-25T10:04:14.902149+0000","last_active":"2024-04-25T10:04:16.237663+0000","last_peered":"2024-04-25T10:04:16.237663+0000","last_clean":"2024-04-25T10:04:16.237663+0000","last_became_active":"2024-04-25T10:04:14.901865+0000","last_became_peered":"2024-04-25T10:04:14.901865+0000","last_unstale":"2024-04-25T10:04:16.237663+0000","last_undegraded":"2024-04-25T10:04:16.237663+0000","last_fullsized":"2024-04-25T10:04:16.237663+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:34.403630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238455+0000","last_change":"2024-04-25T10:04:14.895964+0000","last_active":"2024-04-25T10:04:16.238455+0000","last_peered":"2024-04-25T10:04:16.238455+0000","last_clean":"2024-04-25T10:04:16.238455+0000","last_became_active":"2024-04-25T10:04:14.895669+0000","last_became_peered":"2024-04-25T10:04:14.895669+0000","last_unstale":"2024-04-25T10:04:16.238455+0000","last_undegraded":"2024-04-25T10:04:16.238455+0000","last_fullsized":"2024-04-25T10:04:16.238455+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.428364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889944+0000","last_change":"2024-04-25T10:04:14.916200+0000","last_active":"2024-04-25T10:04:15.889944+0000","last_peered":"2024-04-25T10:04:15.889944+0000","last_clean":"2024-04-25T10:04:15.889944+0000","last_became_active":"2024-04-25T10:04:14.915920+0000","last_became_peered":"2024-04-25T10:04:14.915920+0000","last_unstale":"2024-04-25T10:04:15.889944+0000","last_undegraded":"2024-04-25T10:04:15.889944+0000","last_fullsized":"2024-04-25T10:04:15.889944+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:05.202335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889851+0000","last_change":"2024-04-25T10:04:14.922953+0000","last_active":"2024-04-25T10:04:15.889851+0000","last_peered":"2024-04-25T10:04:15.889851+0000","last_clean":"2024-04-25T10:04:15.889851+0000","last_became_active":"2024-04-25T10:04:14.922631+0000","last_became_peered":"2024-04-25T10:04:14.922631+0000","last_unstale":"2024-04-25T10:04:15.889851+0000","last_undegraded":"2024-04-25T10:04:15.889851+0000","last_fullsized":"2024-04-25T10:04:15.889851+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:08.347746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237488+0000","last_change":"2024-04-25T10:04:14.910528+0000","last_active":"2024-04-25T10:04:16.237488+0000","last_peered":"2024-04-25T10:04:16.237488+0000","last_clean":"2024-04-25T10:04:16.237488+0000","last_became_active":"2024-04-25T10:04:14.910239+0000","last_became_peered":"2024-04-25T10:04:14.910239+0000","last_unstale":"2024-04-25T10:04:16.237488+0000","last_undegraded":"2024-04-25T10:04:16.237488+0000","last_fullsized":"2024-04-25T10:04:16.237488+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:26.659240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892111+0000","last_change":"2024-04-25T10:04:14.908178+0000","last_active":"2024-04-25T10:04:15.892111+0000","last_peered":"2024-04-25T10:04:15.892111+0000","last_clean":"2024-04-25T10:04:15.892111+0000","last_became_active":"2024-04-25T10:04:14.907839+0000","last_became_peered":"2024-04-25T10:04:14.907839+0000","last_unstale":"2024-04-25T10:04:15.892111+0000","last_undegraded":"2024-04-25T10:04:15.892111+0000","last_fullsized":"2024-04-25T10:04:15.892111+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:59.027629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238568+0000","last_change":"2024-04-25T10:04:14.913322+0000","last_active":"2024-04-25T10:04:16.238568+0000","last_peered":"2024-04-25T10:04:16.238568+0000","last_clean":"2024-04-25T10:04:16.238568+0000","last_became_active":"2024-04-25T10:04:14.908095+0000","last_became_peered":"2024-04-25T10:04:14.908095+0000","last_unstale":"2024-04-25T10:04:16.238568+0000","last_undegraded":"2024-04-25T10:04:16.238568+0000","last_fullsized":"2024-04-25T10:04:16.238568+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.715750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890301+0000","last_change":"2024-04-25T10:04:14.921222+0000","last_active":"2024-04-25T10:04:15.890301+0000","last_peered":"2024-04-25T10:04:15.890301+0000","last_clean":"2024-04-25T10:04:15.890301+0000","last_became_active":"2024-04-25T10:04:14.920566+0000","last_became_peered":"2024-04-25T10:04:14.920566+0000","last_unstale":"2024-04-25T10:04:15.890301+0000","last_undegraded":"2024-04-25T10:04:15.890301+0000","last_fullsized":"2024-04-25T10:04:15.890301+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.428364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237859+0000","last_change":"2024-04-25T10:04:14.907617+0000","last_active":"2024-04-25T10:04:16.237859+0000","last_peered":"2024-04-25T10:04:16.237859+0000","last_clean":"2024-04-25T10:04:16.237859+0000","last_became_active":"2024-04-25T10:04:14.907049+0000","last_became_peered":"2024-04-25T10:04:14.907049+0000","last_unstale":"2024-04-25T10:04:16.237859+0000","last_undegraded":"2024-04-25T10:04:16.237859+0000","last_fullsized":"2024-04-25T10:04:16.237859+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:49.307475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238601+0000","last_change":"2024-04-25T10:04:14.906433+0000","last_active":"2024-04-25T10:04:16.238601+0000","last_peered":"2024-04-25T10:04:16.238601+0000","last_clean":"2024-04-25T10:04:16.238601+0000","last_became_active":"2024-04-25T10:04:14.906171+0000","last_became_peered":"2024-04-25T10:04:14.906171+0000","last_unstale":"2024-04-25T10:04:16.238601+0000","last_undegraded":"2024-04-25T10:04:16.238601+0000","last_fullsized":"2024-04-25T10:04:16.238601+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:19.360586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891131+0000","last_change":"2024-04-25T10:04:14.898838+0000","last_active":"2024-04-25T10:04:15.891131+0000","last_peered":"2024-04-25T10:04:15.891131+0000","last_clean":"2024-04-25T10:04:15.891131+0000","last_became_active":"2024-04-25T10:04:14.898538+0000","last_became_peered":"2024-04-25T10:04:14.898538+0000","last_unstale":"2024-04-25T10:04:15.891131+0000","last_undegraded":"2024-04-25T10:04:15.891131+0000","last_fullsized":"2024-04-25T10:04:15.891131+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:44.520115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237248+0000","last_change":"2024-04-25T10:04:14.910913+0000","last_active":"2024-04-25T10:04:16.237248+0000","last_peered":"2024-04-25T10:04:16.237248+0000","last_clean":"2024-04-25T10:04:16.237248+0000","last_became_active":"2024-04-25T10:04:14.910077+0000","last_became_peered":"2024-04-25T10:04:14.910077+0000","last_unstale":"2024-04-25T10:04:16.237248+0000","last_undegraded":"2024-04-25T10:04:16.237248+0000","last_fullsized":"2024-04-25T10:04:16.237248+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:52.598811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238115+0000","last_change":"2024-04-25T10:04:14.917611+0000","last_active":"2024-04-25T10:04:16.238115+0000","last_peered":"2024-04-25T10:04:16.238115+0000","last_clean":"2024-04-25T10:04:16.238115+0000","last_became_active":"2024-04-25T10:04:14.913917+0000","last_became_peered":"2024-04-25T10:04:14.913917+0000","last_unstale":"2024-04-25T10:04:16.238115+0000","last_undegraded":"2024-04-25T10:04:16.238115+0000","last_fullsized":"2024-04-25T10:04:16.238115+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:18.452648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889810+0000","last_change":"2024-04-25T10:04:14.911686+0000","last_active":"2024-04-25T10:04:15.889810+0000","last_peered":"2024-04-25T10:04:15.889810+0000","last_clean":"2024-04-25T10:04:15.889810+0000","last_became_active":"2024-04-25T10:04:14.909395+0000","last_became_peered":"2024-04-25T10:04:14.909395+0000","last_unstale":"2024-04-25T10:04:15.889810+0000","last_undegraded":"2024-04-25T10:04:15.889810+0000","last_fullsized":"2024-04-25T10:04:15.889810+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:36.960234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890412+0000","last_change":"2024-04-25T10:04:14.922314+0000","last_active":"2024-04-25T10:04:15.890412+0000","last_peered":"2024-04-25T10:04:15.890412+0000","last_clean":"2024-04-25T10:04:15.890412+0000","last_became_active":"2024-04-25T10:04:14.922033+0000","last_became_peered":"2024-04-25T10:04:14.922033+0000","last_unstale":"2024-04-25T10:04:15.890412+0000","last_undegraded":"2024-04-25T10:04:15.890412+0000","last_fullsized":"2024-04-25T10:04:15.890412+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:19.360586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891442+0000","last_change":"2024-04-25T10:04:14.893601+0000","last_active":"2024-04-25T10:04:15.891442+0000","last_peered":"2024-04-25T10:04:15.891442+0000","last_clean":"2024-04-25T10:04:15.891442+0000","last_became_active":"2024-04-25T10:04:14.893293+0000","last_became_peered":"2024-04-25T10:04:14.893293+0000","last_unstale":"2024-04-25T10:04:15.891442+0000","last_undegraded":"2024-04-25T10:04:15.891442+0000","last_fullsized":"2024-04-25T10:04:15.891442+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:58.335439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891513+0000","last_change":"2024-04-25T10:04:14.902677+0000","last_active":"2024-04-25T10:04:15.891513+0000","last_peered":"2024-04-25T10:04:15.891513+0000","last_clean":"2024-04-25T10:04:15.891513+0000","last_became_active":"2024-04-25T10:04:14.902386+0000","last_became_peered":"2024-04-25T10:04:14.902386+0000","last_unstale":"2024-04-25T10:04:15.891513+0000","last_undegraded":"2024-04-25T10:04:15.891513+0000","last_fullsized":"2024-04-25T10:04:15.891513+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:05.308139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889925+0000","last_change":"2024-04-25T10:04:14.923102+0000","last_active":"2024-04-25T10:04:15.889925+0000","last_peered":"2024-04-25T10:04:15.889925+0000","last_clean":"2024-04-25T10:04:15.889925+0000","last_became_active":"2024-04-25T10:04:14.922796+0000","last_became_peered":"2024-04-25T10:04:14.922796+0000","last_unstale":"2024-04-25T10:04:15.889925+0000","last_undegraded":"2024-04-25T10:04:15.889925+0000","last_fullsized":"2024-04-25T10:04:15.889925+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:34.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238136+0000","last_change":"2024-04-25T10:04:14.908964+0000","last_active":"2024-04-25T10:04:16.238136+0000","last_peered":"2024-04-25T10:04:16.238136+0000","last_clean":"2024-04-25T10:04:16.238136+0000","last_became_active":"2024-04-25T10:04:14.908648+0000","last_became_peered":"2024-04-25T10:04:14.908648+0000","last_unstale":"2024-04-25T10:04:16.238136+0000","last_undegraded":"2024-04-25T10:04:16.238136+0000","last_fullsized":"2024-04-25T10:04:16.238136+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:45.454649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238390+0000","last_change":"2024-04-25T10:04:14.923545+0000","last_active":"2024-04-25T10:04:16.238390+0000","last_peered":"2024-04-25T10:04:16.238390+0000","last_clean":"2024-04-25T10:04:16.238390+0000","last_became_active":"2024-04-25T10:04:14.923209+0000","last_became_peered":"2024-04-25T10:04:14.923209+0000","last_unstale":"2024-04-25T10:04:16.238390+0000","last_undegraded":"2024-04-25T10:04:16.238390+0000","last_fullsized":"2024-04-25T10:04:16.238390+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:00.126498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892096+0000","last_change":"2024-04-25T10:04:14.905495+0000","last_active":"2024-04-25T10:04:15.892096+0000","last_peered":"2024-04-25T10:04:15.892096+0000","last_clean":"2024-04-25T10:04:15.892096+0000","last_became_active":"2024-04-25T10:04:14.905164+0000","last_became_peered":"2024-04-25T10:04:14.905164+0000","last_unstale":"2024-04-25T10:04:15.892096+0000","last_undegraded":"2024-04-25T10:04:15.892096+0000","last_fullsized":"2024-04-25T10:04:15.892096+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:45.287354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237196+0000","last_change":"2024-04-25T10:04:14.914850+0000","last_active":"2024-04-25T10:04:16.237196+0000","last_peered":"2024-04-25T10:04:16.237196+0000","last_clean":"2024-04-25T10:04:16.237196+0000","last_became_active":"2024-04-25T10:04:14.914592+0000","last_became_peered":"2024-04-25T10:04:14.914592+0000","last_unstale":"2024-04-25T10:04:16.237196+0000","last_undegraded":"2024-04-25T10:04:16.237196+0000","last_fullsized":"2024-04-25T10:04:16.237196+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:54.434833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237253+0000","last_change":"2024-04-25T10:04:14.917923+0000","last_active":"2024-04-25T10:04:16.237253+0000","last_peered":"2024-04-25T10:04:16.237253+0000","last_clean":"2024-04-25T10:04:16.237253+0000","last_became_active":"2024-04-25T10:04:14.917630+0000","last_became_peered":"2024-04-25T10:04:14.917630+0000","last_unstale":"2024-04-25T10:04:16.237253+0000","last_undegraded":"2024-04-25T10:04:16.237253+0000","last_fullsized":"2024-04-25T10:04:16.237253+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.253359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237884+0000","last_change":"2024-04-25T10:04:14.917336+0000","last_active":"2024-04-25T10:04:16.237884+0000","last_peered":"2024-04-25T10:04:16.237884+0000","last_clean":"2024-04-25T10:04:16.237884+0000","last_became_active":"2024-04-25T10:04:14.916975+0000","last_became_peered":"2024-04-25T10:04:14.916975+0000","last_unstale":"2024-04-25T10:04:16.237884+0000","last_undegraded":"2024-04-25T10:04:16.237884+0000","last_fullsized":"2024-04-25T10:04:16.237884+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:42.694901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892333+0000","last_change":"2024-04-25T10:04:14.909230+0000","last_active":"2024-04-25T10:04:15.892333+0000","last_peered":"2024-04-25T10:04:15.892333+0000","last_clean":"2024-04-25T10:04:15.892333+0000","last_became_active":"2024-04-25T10:04:14.908930+0000","last_became_peered":"2024-04-25T10:04:14.908930+0000","last_unstale":"2024-04-25T10:04:15.892333+0000","last_undegraded":"2024-04-25T10:04:15.892333+0000","last_fullsized":"2024-04-25T10:04:15.892333+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:54.434833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889972+0000","last_change":"2024-04-25T10:04:14.915440+0000","last_active":"2024-04-25T10:04:15.889972+0000","last_peered":"2024-04-25T10:04:15.889972+0000","last_clean":"2024-04-25T10:04:15.889972+0000","last_became_active":"2024-04-25T10:04:14.915147+0000","last_became_peered":"2024-04-25T10:04:14.915147+0000","last_unstale":"2024-04-25T10:04:15.889972+0000","last_undegraded":"2024-04-25T10:04:15.889972+0000","last_fullsized":"2024-04-25T10:04:15.889972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:17.123316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237693+0000","last_change":"2024-04-25T10:04:14.920213+0000","last_active":"2024-04-25T10:04:16.237693+0000","last_peered":"2024-04-25T10:04:16.237693+0000","last_clean":"2024-04-25T10:04:16.237693+0000","last_became_active":"2024-04-25T10:04:14.919900+0000","last_became_peered":"2024-04-25T10:04:14.919900+0000","last_unstale":"2024-04-25T10:04:16.237693+0000","last_undegraded":"2024-04-25T10:04:16.237693+0000","last_fullsized":"2024-04-25T10:04:16.237693+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:22.561454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237383+0000","last_change":"2024-04-25T10:04:14.919732+0000","last_active":"2024-04-25T10:04:16.237383+0000","last_peered":"2024-04-25T10:04:16.237383+0000","last_clean":"2024-04-25T10:04:16.237383+0000","last_became_active":"2024-04-25T10:04:14.919449+0000","last_became_peered":"2024-04-25T10:04:14.919449+0000","last_unstale":"2024-04-25T10:04:16.237383+0000","last_undegraded":"2024-04-25T10:04:16.237383+0000","last_fullsized":"2024-04-25T10:04:16.237383+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:32.418999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237629+0000","last_change":"2024-04-25T10:04:14.905796+0000","last_active":"2024-04-25T10:04:16.237629+0000","last_peered":"2024-04-25T10:04:16.237629+0000","last_clean":"2024-04-25T10:04:16.237629+0000","last_became_active":"2024-04-25T10:04:14.905497+0000","last_became_peered":"2024-04-25T10:04:14.905497+0000","last_unstale":"2024-04-25T10:04:16.237629+0000","last_undegraded":"2024-04-25T10:04:16.237629+0000","last_fullsized":"2024-04-25T10:04:16.237629+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:05.661329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889889+0000","last_change":"2024-04-25T10:04:14.920427+0000","last_active":"2024-04-25T10:04:15.889889+0000","last_peered":"2024-04-25T10:04:15.889889+0000","last_clean":"2024-04-25T10:04:15.889889+0000","last_became_active":"2024-04-25T10:04:14.920014+0000","last_became_peered":"2024-04-25T10:04:14.920014+0000","last_unstale":"2024-04-25T10:04:15.889889+0000","last_undegraded":"2024-04-25T10:04:15.889889+0000","last_fullsized":"2024-04-25T10:04:15.889889+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:41.695527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892417+0000","last_change":"2024-04-25T10:04:14.896609+0000","last_active":"2024-04-25T10:04:15.892417+0000","last_peered":"2024-04-25T10:04:15.892417+0000","last_clean":"2024-04-25T10:04:15.892417+0000","last_became_active":"2024-04-25T10:04:14.896337+0000","last_became_peered":"2024-04-25T10:04:14.896337+0000","last_unstale":"2024-04-25T10:04:15.892417+0000","last_undegraded":"2024-04-25T10:04:15.892417+0000","last_fullsized":"2024-04-25T10:04:15.892417+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:15.369689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"164'4","reported_seq":18,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237842+0000","last_change":"2024-04-25T10:04:13.905310+0000","last_active":"2024-04-25T10:04:16.237842+0000","last_peered":"2024-04-25T10:04:16.237842+0000","last_clean":"2024-04-25T10:04:16.237842+0000","last_became_active":"2024-04-25T10:04:13.905060+0000","last_became_peered":"2024-04-25T10:04:13.905060+0000","last_unstale":"2024-04-25T10:04:16.237842+0000","last_undegraded":"2024-04-25T10:04:16.237842+0000","last_fullsized":"2024-04-25T10:04:16.237842+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:35.958124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238175+0000","last_change":"2024-04-25T10:04:14.911957+0000","last_active":"2024-04-25T10:04:16.238175+0000","last_peered":"2024-04-25T10:04:16.238175+0000","last_clean":"2024-04-25T10:04:16.238175+0000","last_became_active":"2024-04-25T10:04:14.911671+0000","last_became_peered":"2024-04-25T10:04:14.911671+0000","last_unstale":"2024-04-25T10:04:16.238175+0000","last_undegraded":"2024-04-25T10:04:16.238175+0000","last_fullsized":"2024-04-25T10:04:16.238175+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:31.656397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890389+0000","last_change":"2024-04-25T10:04:13.911506+0000","last_active":"2024-04-25T10:04:15.890389+0000","last_peered":"2024-04-25T10:04:15.890389+0000","last_clean":"2024-04-25T10:04:15.890389+0000","last_became_active":"2024-04-25T10:04:13.909233+0000","last_became_peered":"2024-04-25T10:04:13.909233+0000","last_unstale":"2024-04-25T10:04:15.890389+0000","last_undegraded":"2024-04-25T10:04:15.890389+0000","last_fullsized":"2024-04-25T10:04:15.890389+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:44.285244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238015+0000","last_change":"2024-04-25T10:04:14.908949+0000","last_active":"2024-04-25T10:04:16.238015+0000","last_peered":"2024-04-25T10:04:16.238015+0000","last_clean":"2024-04-25T10:04:16.238015+0000","last_became_active":"2024-04-25T10:04:14.908607+0000","last_became_peered":"2024-04-25T10:04:14.908607+0000","last_unstale":"2024-04-25T10:04:16.238015+0000","last_undegraded":"2024-04-25T10:04:16.238015+0000","last_fullsized":"2024-04-25T10:04:16.238015+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:44.644245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237922+0000","last_change":"2024-04-25T10:04:13.899686+0000","last_active":"2024-04-25T10:04:16.237922+0000","last_peered":"2024-04-25T10:04:16.237922+0000","last_clean":"2024-04-25T10:04:16.237922+0000","last_became_active":"2024-04-25T10:04:13.899387+0000","last_became_peered":"2024-04-25T10:04:13.899387+0000","last_unstale":"2024-04-25T10:04:16.237922+0000","last_undegraded":"2024-04-25T10:04:16.237922+0000","last_fullsized":"2024-04-25T10:04:16.237922+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:43.080781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237638+0000","last_change":"2024-04-25T10:04:14.911170+0000","last_active":"2024-04-25T10:04:16.237638+0000","last_peered":"2024-04-25T10:04:16.237638+0000","last_clean":"2024-04-25T10:04:16.237638+0000","last_became_active":"2024-04-25T10:04:14.910835+0000","last_became_peered":"2024-04-25T10:04:14.910835+0000","last_unstale":"2024-04-25T10:04:16.237638+0000","last_undegraded":"2024-04-25T10:04:16.237638+0000","last_fullsized":"2024-04-25T10:04:16.237638+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:14.681582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891074+0000","last_change":"2024-04-25T10:04:13.899482+0000","last_active":"2024-04-25T10:04:15.891074+0000","last_peered":"2024-04-25T10:04:15.891074+0000","last_clean":"2024-04-25T10:04:15.891074+0000","last_became_active":"2024-04-25T10:04:13.899195+0000","last_became_peered":"2024-04-25T10:04:13.899195+0000","last_unstale":"2024-04-25T10:04:15.891074+0000","last_undegraded":"2024-04-25T10:04:15.891074+0000","last_fullsized":"2024-04-25T10:04:15.891074+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:41.094825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890757+0000","last_change":"2024-04-25T10:04:14.918739+0000","last_active":"2024-04-25T10:04:15.890757+0000","last_peered":"2024-04-25T10:04:15.890757+0000","last_clean":"2024-04-25T10:04:15.890757+0000","last_became_active":"2024-04-25T10:04:14.918447+0000","last_became_peered":"2024-04-25T10:04:14.918447+0000","last_unstale":"2024-04-25T10:04:15.890757+0000","last_undegraded":"2024-04-25T10:04:15.890757+0000","last_fullsized":"2024-04-25T10:04:15.890757+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:11.067114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890102+0000","last_change":"2024-04-25T10:04:13.911021+0000","last_active":"2024-04-25T10:04:15.890102+0000","last_peered":"2024-04-25T10:04:15.890102+0000","last_clean":"2024-04-25T10:04:15.890102+0000","last_became_active":"2024-04-25T10:04:13.909391+0000","last_became_peered":"2024-04-25T10:04:13.909391+0000","last_unstale":"2024-04-25T10:04:15.890102+0000","last_undegraded":"2024-04-25T10:04:15.890102+0000","last_fullsized":"2024-04-25T10:04:15.890102+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:53.432723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238267+0000","last_change":"2024-04-25T10:04:14.907532+0000","last_active":"2024-04-25T10:04:16.238267+0000","last_peered":"2024-04-25T10:04:16.238267+0000","last_clean":"2024-04-25T10:04:16.238267+0000","last_became_active":"2024-04-25T10:04:14.906617+0000","last_became_peered":"2024-04-25T10:04:14.906617+0000","last_unstale":"2024-04-25T10:04:16.238267+0000","last_undegraded":"2024-04-25T10:04:16.238267+0000","last_fullsized":"2024-04-25T10:04:16.238267+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:04.468380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238041+0000","last_change":"2024-04-25T10:04:13.907057+0000","last_active":"2024-04-25T10:04:16.238041+0000","last_peered":"2024-04-25T10:04:16.238041+0000","last_clean":"2024-04-25T10:04:16.238041+0000","last_became_active":"2024-04-25T10:04:13.906778+0000","last_became_peered":"2024-04-25T10:04:13.906778+0000","last_unstale":"2024-04-25T10:04:16.238041+0000","last_undegraded":"2024-04-25T10:04:16.238041+0000","last_fullsized":"2024-04-25T10:04:16.238041+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:55.338829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238412+0000","last_change":"2024-04-25T10:04:14.913844+0000","last_active":"2024-04-25T10:04:16.238412+0000","last_peered":"2024-04-25T10:04:16.238412+0000","last_clean":"2024-04-25T10:04:16.238412+0000","last_became_active":"2024-04-25T10:04:14.913525+0000","last_became_peered":"2024-04-25T10:04:14.913525+0000","last_unstale":"2024-04-25T10:04:16.238412+0000","last_undegraded":"2024-04-25T10:04:16.238412+0000","last_fullsized":"2024-04-25T10:04:16.238412+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:15.657853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890982+0000","last_change":"2024-04-25T10:04:13.894824+0000","last_active":"2024-04-25T10:04:15.890982+0000","last_peered":"2024-04-25T10:04:15.890982+0000","last_clean":"2024-04-25T10:04:15.890982+0000","last_became_active":"2024-04-25T10:04:13.894557+0000","last_became_peered":"2024-04-25T10:04:13.894557+0000","last_unstale":"2024-04-25T10:04:15.890982+0000","last_undegraded":"2024-04-25T10:04:15.890982+0000","last_fullsized":"2024-04-25T10:04:15.890982+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:45.461278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892468+0000","last_change":"2024-04-25T10:04:14.893650+0000","last_active":"2024-04-25T10:04:15.892468+0000","last_peered":"2024-04-25T10:04:15.892468+0000","last_clean":"2024-04-25T10:04:15.892468+0000","last_became_active":"2024-04-25T10:04:14.893293+0000","last_became_peered":"2024-04-25T10:04:14.893293+0000","last_unstale":"2024-04-25T10:04:15.892468+0000","last_undegraded":"2024-04-25T10:04:15.892468+0000","last_fullsized":"2024-04-25T10:04:15.892468+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.026157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890380+0000","last_change":"2024-04-25T10:04:13.912389+0000","last_active":"2024-04-25T10:04:15.890380+0000","last_peered":"2024-04-25T10:04:15.890380+0000","last_clean":"2024-04-25T10:04:15.890380+0000","last_became_active":"2024-04-25T10:04:13.912117+0000","last_became_peered":"2024-04-25T10:04:13.912117+0000","last_unstale":"2024-04-25T10:04:15.890380+0000","last_undegraded":"2024-04-25T10:04:15.890380+0000","last_fullsized":"2024-04-25T10:04:15.890380+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:58.025520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890246+0000","last_change":"2024-04-25T10:04:14.920422+0000","last_active":"2024-04-25T10:04:15.890246+0000","last_peered":"2024-04-25T10:04:15.890246+0000","last_clean":"2024-04-25T10:04:15.890246+0000","last_became_active":"2024-04-25T10:04:14.919739+0000","last_became_peered":"2024-04-25T10:04:14.919739+0000","last_unstale":"2024-04-25T10:04:15.890246+0000","last_undegraded":"2024-04-25T10:04:15.890246+0000","last_fullsized":"2024-04-25T10:04:15.890246+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:15.701395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237736+0000","last_change":"2024-04-25T10:04:13.908763+0000","last_active":"2024-04-25T10:04:16.237736+0000","last_peered":"2024-04-25T10:04:16.237736+0000","last_clean":"2024-04-25T10:04:16.237736+0000","last_became_active":"2024-04-25T10:04:13.908481+0000","last_became_peered":"2024-04-25T10:04:13.908481+0000","last_unstale":"2024-04-25T10:04:16.237736+0000","last_undegraded":"2024-04-25T10:04:16.237736+0000","last_fullsized":"2024-04-25T10:04:16.237736+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:38.122061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891488+0000","last_change":"2024-04-25T10:04:14.910090+0000","last_active":"2024-04-25T10:04:15.891488+0000","last_peered":"2024-04-25T10:04:15.891488+0000","last_clean":"2024-04-25T10:04:15.891488+0000","last_became_active":"2024-04-25T10:04:14.909490+0000","last_became_peered":"2024-04-25T10:04:14.909490+0000","last_unstale":"2024-04-25T10:04:15.891488+0000","last_undegraded":"2024-04-25T10:04:15.891488+0000","last_fullsized":"2024-04-25T10:04:15.891488+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:16.962626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"164'3","reported_seq":17,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238146+0000","last_change":"2024-04-25T10:04:13.909365+0000","last_active":"2024-04-25T10:04:16.238146+0000","last_peered":"2024-04-25T10:04:16.238146+0000","last_clean":"2024-04-25T10:04:16.238146+0000","last_became_active":"2024-04-25T10:04:13.907222+0000","last_became_peered":"2024-04-25T10:04:13.907222+0000","last_unstale":"2024-04-25T10:04:16.238146+0000","last_undegraded":"2024-04-25T10:04:16.238146+0000","last_fullsized":"2024-04-25T10:04:16.238146+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:07.345636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238653+0000","last_change":"2024-04-25T10:04:14.919133+0000","last_active":"2024-04-25T10:04:16.238653+0000","last_peered":"2024-04-25T10:04:16.238653+0000","last_clean":"2024-04-25T10:04:16.238653+0000","last_became_active":"2024-04-25T10:04:14.918862+0000","last_became_peered":"2024-04-25T10:04:14.918862+0000","last_unstale":"2024-04-25T10:04:16.238653+0000","last_undegraded":"2024-04-25T10:04:16.238653+0000","last_fullsized":"2024-04-25T10:04:16.238653+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:56.340939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"164'6","reported_seq":20,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891075+0000","last_change":"2024-04-25T10:04:13.898686+0000","last_active":"2024-04-25T10:04:15.891075+0000","last_peered":"2024-04-25T10:04:15.891075+0000","last_clean":"2024-04-25T10:04:15.891075+0000","last_became_active":"2024-04-25T10:04:13.898413+0000","last_became_peered":"2024-04-25T10:04:13.898413+0000","last_unstale":"2024-04-25T10:04:15.891075+0000","last_undegraded":"2024-04-25T10:04:15.891075+0000","last_fullsized":"2024-04-25T10:04:15.891075+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:52.368366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237057+0000","last_change":"2024-04-25T10:04:14.910641+0000","last_active":"2024-04-25T10:04:16.237057+0000","last_peered":"2024-04-25T10:04:16.237057+0000","last_clean":"2024-04-25T10:04:16.237057+0000","last_became_active":"2024-04-25T10:04:14.910312+0000","last_became_peered":"2024-04-25T10:04:14.910312+0000","last_unstale":"2024-04-25T10:04:16.237057+0000","last_undegraded":"2024-04-25T10:04:16.237057+0000","last_fullsized":"2024-04-25T10:04:16.237057+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:02.199324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891306+0000","last_change":"2024-04-25T10:04:13.897853+0000","last_active":"2024-04-25T10:04:15.891306+0000","last_peered":"2024-04-25T10:04:15.891306+0000","last_clean":"2024-04-25T10:04:15.891306+0000","last_became_active":"2024-04-25T10:04:13.897547+0000","last_became_peered":"2024-04-25T10:04:13.897547+0000","last_unstale":"2024-04-25T10:04:15.891306+0000","last_undegraded":"2024-04-25T10:04:15.891306+0000","last_fullsized":"2024-04-25T10:04:15.891306+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:31.172667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889824+0000","last_change":"2024-04-25T10:04:14.914301+0000","last_active":"2024-04-25T10:04:15.889824+0000","last_peered":"2024-04-25T10:04:15.889824+0000","last_clean":"2024-04-25T10:04:15.889824+0000","last_became_active":"2024-04-25T10:04:14.909471+0000","last_became_peered":"2024-04-25T10:04:14.909471+0000","last_unstale":"2024-04-25T10:04:15.889824+0000","last_undegraded":"2024-04-25T10:04:15.889824+0000","last_fullsized":"2024-04-25T10:04:15.889824+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:46.662983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238511+0000","last_change":"2024-04-25T10:04:13.901634+0000","last_active":"2024-04-25T10:04:16.238511+0000","last_peered":"2024-04-25T10:04:16.238511+0000","last_clean":"2024-04-25T10:04:16.238511+0000","last_became_active":"2024-04-25T10:04:13.900799+0000","last_became_peered":"2024-04-25T10:04:13.900799+0000","last_unstale":"2024-04-25T10:04:16.238511+0000","last_undegraded":"2024-04-25T10:04:16.238511+0000","last_fullsized":"2024-04-25T10:04:16.238511+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:03.199621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891670+0000","last_change":"2024-04-25T10:04:14.901604+0000","last_active":"2024-04-25T10:04:15.891670+0000","last_peered":"2024-04-25T10:04:15.891670+0000","last_clean":"2024-04-25T10:04:15.891670+0000","last_became_active":"2024-04-25T10:04:14.901362+0000","last_became_peered":"2024-04-25T10:04:14.901362+0000","last_unstale":"2024-04-25T10:04:15.891670+0000","last_undegraded":"2024-04-25T10:04:15.891670+0000","last_fullsized":"2024-04-25T10:04:15.891670+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:39.457441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237632+0000","last_change":"2024-04-25T10:04:13.898306+0000","last_active":"2024-04-25T10:04:16.237632+0000","last_peered":"2024-04-25T10:04:16.237632+0000","last_clean":"2024-04-25T10:04:16.237632+0000","last_became_active":"2024-04-25T10:04:13.898057+0000","last_became_peered":"2024-04-25T10:04:13.898057+0000","last_unstale":"2024-04-25T10:04:16.237632+0000","last_undegraded":"2024-04-25T10:04:16.237632+0000","last_fullsized":"2024-04-25T10:04:16.237632+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:10.713640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891204+0000","last_change":"2024-04-25T10:04:14.905089+0000","last_active":"2024-04-25T10:04:15.891204+0000","last_peered":"2024-04-25T10:04:15.891204+0000","last_clean":"2024-04-25T10:04:15.891204+0000","last_became_active":"2024-04-25T10:04:14.904817+0000","last_became_peered":"2024-04-25T10:04:14.904817+0000","last_unstale":"2024-04-25T10:04:15.891204+0000","last_undegraded":"2024-04-25T10:04:15.891204+0000","last_fullsized":"2024-04-25T10:04:15.891204+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:43.962022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"164'8","reported_seq":22,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237225+0000","last_change":"2024-04-25T10:04:13.902317+0000","last_active":"2024-04-25T10:04:16.237225+0000","last_peered":"2024-04-25T10:04:16.237225+0000","last_clean":"2024-04-25T10:04:16.237225+0000","last_became_active":"2024-04-25T10:04:13.902008+0000","last_became_peered":"2024-04-25T10:04:13.902008+0000","last_unstale":"2024-04-25T10:04:16.237225+0000","last_undegraded":"2024-04-25T10:04:16.237225+0000","last_fullsized":"2024-04-25T10:04:16.237225+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.551407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237639+0000","last_change":"2024-04-25T10:04:14.906405+0000","last_active":"2024-04-25T10:04:16.237639+0000","last_peered":"2024-04-25T10:04:16.237639+0000","last_clean":"2024-04-25T10:04:16.237639+0000","last_became_active":"2024-04-25T10:04:14.906126+0000","last_became_peered":"2024-04-25T10:04:14.906126+0000","last_unstale":"2024-04-25T10:04:16.237639+0000","last_undegraded":"2024-04-25T10:04:16.237639+0000","last_fullsized":"2024-04-25T10:04:16.237639+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:26.159293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238023+0000","last_change":"2024-04-25T10:04:13.902390+0000","last_active":"2024-04-25T10:04:16.238023+0000","last_peered":"2024-04-25T10:04:16.238023+0000","last_clean":"2024-04-25T10:04:16.238023+0000","last_became_active":"2024-04-25T10:04:13.902012+0000","last_became_peered":"2024-04-25T10:04:13.902012+0000","last_unstale":"2024-04-25T10:04:16.238023+0000","last_undegraded":"2024-04-25T10:04:16.238023+0000","last_fullsized":"2024-04-25T10:04:16.238023+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:29.242265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890166+0000","last_change":"2024-04-25T10:04:14.921007+0000","last_active":"2024-04-25T10:04:15.890166+0000","last_peered":"2024-04-25T10:04:15.890166+0000","last_clean":"2024-04-25T10:04:15.890166+0000","last_became_active":"2024-04-25T10:04:14.920141+0000","last_became_peered":"2024-04-25T10:04:14.920141+0000","last_unstale":"2024-04-25T10:04:15.890166+0000","last_undegraded":"2024-04-25T10:04:15.890166+0000","last_fullsized":"2024-04-25T10:04:15.890166+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:47.794132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237595+0000","last_change":"2024-04-25T10:04:13.896683+0000","last_active":"2024-04-25T10:04:16.237595+0000","last_peered":"2024-04-25T10:04:16.237595+0000","last_clean":"2024-04-25T10:04:16.237595+0000","last_became_active":"2024-04-25T10:04:13.896386+0000","last_became_peered":"2024-04-25T10:04:13.896386+0000","last_unstale":"2024-04-25T10:04:16.237595+0000","last_undegraded":"2024-04-25T10:04:16.237595+0000","last_fullsized":"2024-04-25T10:04:16.237595+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:59.155982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238115+0000","last_change":"2024-04-25T10:04:14.917146+0000","last_active":"2024-04-25T10:04:16.238115+0000","last_peered":"2024-04-25T10:04:16.238115+0000","last_clean":"2024-04-25T10:04:16.238115+0000","last_became_active":"2024-04-25T10:04:14.916788+0000","last_became_peered":"2024-04-25T10:04:14.916788+0000","last_unstale":"2024-04-25T10:04:16.238115+0000","last_undegraded":"2024-04-25T10:04:16.238115+0000","last_fullsized":"2024-04-25T10:04:16.238115+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:12.457746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237767+0000","last_change":"2024-04-25T10:04:13.905850+0000","last_active":"2024-04-25T10:04:16.237767+0000","last_peered":"2024-04-25T10:04:16.237767+0000","last_clean":"2024-04-25T10:04:16.237767+0000","last_became_active":"2024-04-25T10:04:13.905565+0000","last_became_peered":"2024-04-25T10:04:13.905565+0000","last_unstale":"2024-04-25T10:04:16.237767+0000","last_undegraded":"2024-04-25T10:04:16.237767+0000","last_fullsized":"2024-04-25T10:04:16.237767+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:41.094825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238677+0000","last_change":"2024-04-25T10:04:14.922523+0000","last_active":"2024-04-25T10:04:16.238677+0000","last_peered":"2024-04-25T10:04:16.238677+0000","last_clean":"2024-04-25T10:04:16.238677+0000","last_became_active":"2024-04-25T10:04:14.922112+0000","last_became_peered":"2024-04-25T10:04:14.922112+0000","last_unstale":"2024-04-25T10:04:16.238677+0000","last_undegraded":"2024-04-25T10:04:16.238677+0000","last_fullsized":"2024-04-25T10:04:16.238677+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:17.961068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237753+0000","last_change":"2024-04-25T10:04:13.899466+0000","last_active":"2024-04-25T10:04:16.237753+0000","last_peered":"2024-04-25T10:04:16.237753+0000","last_clean":"2024-04-25T10:04:16.237753+0000","last_became_active":"2024-04-25T10:04:13.899146+0000","last_became_peered":"2024-04-25T10:04:13.899146+0000","last_unstale":"2024-04-25T10:04:16.237753+0000","last_undegraded":"2024-04-25T10:04:16.237753+0000","last_fullsized":"2024-04-25T10:04:16.237753+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:12.909171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237348+0000","last_change":"2024-04-25T10:04:14.921433+0000","last_active":"2024-04-25T10:04:16.237348+0000","last_peered":"2024-04-25T10:04:16.237348+0000","last_clean":"2024-04-25T10:04:16.237348+0000","last_became_active":"2024-04-25T10:04:14.921129+0000","last_became_peered":"2024-04-25T10:04:14.921129+0000","last_unstale":"2024-04-25T10:04:16.237348+0000","last_undegraded":"2024-04-25T10:04:16.237348+0000","last_fullsized":"2024-04-25T10:04:16.237348+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:15.369689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891325+0000","last_change":"2024-04-25T10:04:13.907912+0000","last_active":"2024-04-25T10:04:15.891325+0000","last_peered":"2024-04-25T10:04:15.891325+0000","last_clean":"2024-04-25T10:04:15.891325+0000","last_became_active":"2024-04-25T10:04:13.907651+0000","last_became_peered":"2024-04-25T10:04:13.907651+0000","last_unstale":"2024-04-25T10:04:15.891325+0000","last_undegraded":"2024-04-25T10:04:15.891325+0000","last_fullsized":"2024-04-25T10:04:15.891325+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:43.570072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237535+0000","last_change":"2024-04-25T10:04:14.896251+0000","last_active":"2024-04-25T10:04:16.237535+0000","last_peered":"2024-04-25T10:04:16.237535+0000","last_clean":"2024-04-25T10:04:16.237535+0000","last_became_active":"2024-04-25T10:04:14.895304+0000","last_became_peered":"2024-04-25T10:04:14.895304+0000","last_unstale":"2024-04-25T10:04:16.237535+0000","last_undegraded":"2024-04-25T10:04:16.237535+0000","last_fullsized":"2024-04-25T10:04:16.237535+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:47.794132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238147+0000","last_change":"2024-04-25T10:04:13.904679+0000","last_active":"2024-04-25T10:04:16.238147+0000","last_peered":"2024-04-25T10:04:16.238147+0000","last_clean":"2024-04-25T10:04:16.238147+0000","last_became_active":"2024-04-25T10:04:13.904426+0000","last_became_peered":"2024-04-25T10:04:13.904426+0000","last_unstale":"2024-04-25T10:04:16.238147+0000","last_undegraded":"2024-04-25T10:04:16.238147+0000","last_fullsized":"2024-04-25T10:04:16.238147+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:16.958959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237950+0000","last_change":"2024-04-25T10:04:14.908665+0000","last_active":"2024-04-25T10:04:16.237950+0000","last_peered":"2024-04-25T10:04:16.237950+0000","last_clean":"2024-04-25T10:04:16.237950+0000","last_became_active":"2024-04-25T10:04:14.908392+0000","last_became_peered":"2024-04-25T10:04:14.908392+0000","last_unstale":"2024-04-25T10:04:16.237950+0000","last_undegraded":"2024-04-25T10:04:16.237950+0000","last_fullsized":"2024-04-25T10:04:16.237950+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:15.701395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237792+0000","last_change":"2024-04-25T10:04:13.898826+0000","last_active":"2024-04-25T10:04:16.237792+0000","last_peered":"2024-04-25T10:04:16.237792+0000","last_clean":"2024-04-25T10:04:16.237792+0000","last_became_active":"2024-04-25T10:04:13.898462+0000","last_became_peered":"2024-04-25T10:04:13.898462+0000","last_unstale":"2024-04-25T10:04:16.237792+0000","last_undegraded":"2024-04-25T10:04:16.237792+0000","last_fullsized":"2024-04-25T10:04:16.237792+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:41.233807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890041+0000","last_change":"2024-04-25T10:04:14.917009+0000","last_active":"2024-04-25T10:04:15.890041+0000","last_peered":"2024-04-25T10:04:15.890041+0000","last_clean":"2024-04-25T10:04:15.890041+0000","last_became_active":"2024-04-25T10:04:14.916661+0000","last_became_peered":"2024-04-25T10:04:14.916661+0000","last_unstale":"2024-04-25T10:04:15.890041+0000","last_undegraded":"2024-04-25T10:04:15.890041+0000","last_fullsized":"2024-04-25T10:04:15.890041+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:17.029553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237272+0000","last_change":"2024-04-25T10:04:13.899083+0000","last_active":"2024-04-25T10:04:16.237272+0000","last_peered":"2024-04-25T10:04:16.237272+0000","last_clean":"2024-04-25T10:04:16.237272+0000","last_became_active":"2024-04-25T10:04:13.898810+0000","last_became_peered":"2024-04-25T10:04:13.898810+0000","last_unstale":"2024-04-25T10:04:16.237272+0000","last_undegraded":"2024-04-25T10:04:16.237272+0000","last_fullsized":"2024-04-25T10:04:16.237272+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:45.461278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237783+0000","last_change":"2024-04-25T10:04:14.908754+0000","last_active":"2024-04-25T10:04:16.237783+0000","last_peered":"2024-04-25T10:04:16.237783+0000","last_clean":"2024-04-25T10:04:16.237783+0000","last_became_active":"2024-04-25T10:04:14.908503+0000","last_became_peered":"2024-04-25T10:04:14.908503+0000","last_unstale":"2024-04-25T10:04:16.237783+0000","last_undegraded":"2024-04-25T10:04:16.237783+0000","last_fullsized":"2024-04-25T10:04:16.237783+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:55.705402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237952+0000","last_change":"2024-04-25T10:04:13.899508+0000","last_active":"2024-04-25T10:04:16.237952+0000","last_peered":"2024-04-25T10:04:16.237952+0000","last_clean":"2024-04-25T10:04:16.237952+0000","last_became_active":"2024-04-25T10:04:13.898618+0000","last_became_peered":"2024-04-25T10:04:13.898618+0000","last_unstale":"2024-04-25T10:04:16.237952+0000","last_undegraded":"2024-04-25T10:04:16.237952+0000","last_fullsized":"2024-04-25T10:04:16.237952+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:45.660873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237429+0000","last_change":"2024-04-25T10:04:14.913939+0000","last_active":"2024-04-25T10:04:16.237429+0000","last_peered":"2024-04-25T10:04:16.237429+0000","last_clean":"2024-04-25T10:04:16.237429+0000","last_became_active":"2024-04-25T10:04:14.913637+0000","last_became_peered":"2024-04-25T10:04:14.913637+0000","last_unstale":"2024-04-25T10:04:16.237429+0000","last_undegraded":"2024-04-25T10:04:16.237429+0000","last_fullsized":"2024-04-25T10:04:16.237429+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.026157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"165'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237342+0000","last_change":"2024-04-25T10:04:13.895684+0000","last_active":"2024-04-25T10:04:16.237342+0000","last_peered":"2024-04-25T10:04:16.237342+0000","last_clean":"2024-04-25T10:04:16.237342+0000","last_became_active":"2024-04-25T10:04:13.895390+0000","last_became_peered":"2024-04-25T10:04:13.895390+0000","last_unstale":"2024-04-25T10:04:16.237342+0000","last_undegraded":"2024-04-25T10:04:16.237342+0000","last_fullsized":"2024-04-25T10:04:16.237342+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:59.957477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892462+0000","last_change":"2024-04-25T10:04:14.897994+0000","last_active":"2024-04-25T10:04:15.892462+0000","last_peered":"2024-04-25T10:04:15.892462+0000","last_clean":"2024-04-25T10:04:15.892462+0000","last_became_active":"2024-04-25T10:04:14.897723+0000","last_became_peered":"2024-04-25T10:04:14.897723+0000","last_unstale":"2024-04-25T10:04:15.892462+0000","last_undegraded":"2024-04-25T10:04:15.892462+0000","last_fullsized":"2024-04-25T10:04:15.892462+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:38.890605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890263+0000","last_change":"2024-04-25T10:04:13.911061+0000","last_active":"2024-04-25T10:04:15.890263+0000","last_peered":"2024-04-25T10:04:15.890263+0000","last_clean":"2024-04-25T10:04:15.890263+0000","last_became_active":"2024-04-25T10:04:13.910591+0000","last_became_peered":"2024-04-25T10:04:13.910591+0000","last_unstale":"2024-04-25T10:04:15.890263+0000","last_undegraded":"2024-04-25T10:04:15.890263+0000","last_fullsized":"2024-04-25T10:04:15.890263+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:14.367579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237404+0000","last_change":"2024-04-25T10:04:14.908689+0000","last_active":"2024-04-25T10:04:16.237404+0000","last_peered":"2024-04-25T10:04:16.237404+0000","last_clean":"2024-04-25T10:04:16.237404+0000","last_became_active":"2024-04-25T10:04:14.908375+0000","last_became_peered":"2024-04-25T10:04:14.908375+0000","last_unstale":"2024-04-25T10:04:16.237404+0000","last_undegraded":"2024-04-25T10:04:16.237404+0000","last_fullsized":"2024-04-25T10:04:16.237404+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:00.191907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890212+0000","last_change":"2024-04-25T10:04:13.913137+0000","last_active":"2024-04-25T10:04:15.890212+0000","last_peered":"2024-04-25T10:04:15.890212+0000","last_clean":"2024-04-25T10:04:15.890212+0000","last_became_active":"2024-04-25T10:04:13.912835+0000","last_became_peered":"2024-04-25T10:04:13.912835+0000","last_unstale":"2024-04-25T10:04:15.890212+0000","last_undegraded":"2024-04-25T10:04:15.890212+0000","last_fullsized":"2024-04-25T10:04:15.890212+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:22.383618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237200+0000","last_change":"2024-04-25T10:04:14.904973+0000","last_active":"2024-04-25T10:04:16.237200+0000","last_peered":"2024-04-25T10:04:16.237200+0000","last_clean":"2024-04-25T10:04:16.237200+0000","last_became_active":"2024-04-25T10:04:14.904700+0000","last_became_peered":"2024-04-25T10:04:14.904700+0000","last_unstale":"2024-04-25T10:04:16.237200+0000","last_undegraded":"2024-04-25T10:04:16.237200+0000","last_fullsized":"2024-04-25T10:04:16.237200+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:37.029957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892236+0000","last_change":"2024-04-25T10:04:13.896282+0000","last_active":"2024-04-25T10:04:15.892236+0000","last_peered":"2024-04-25T10:04:15.892236+0000","last_clean":"2024-04-25T10:04:15.892236+0000","last_became_active":"2024-04-25T10:04:13.894939+0000","last_became_peered":"2024-04-25T10:04:13.894939+0000","last_unstale":"2024-04-25T10:04:15.892236+0000","last_undegraded":"2024-04-25T10:04:15.892236+0000","last_fullsized":"2024-04-25T10:04:15.892236+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:48.305366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890983+0000","last_change":"2024-04-25T10:04:14.901523+0000","last_active":"2024-04-25T10:04:15.890983+0000","last_peered":"2024-04-25T10:04:15.890983+0000","last_clean":"2024-04-25T10:04:15.890983+0000","last_became_active":"2024-04-25T10:04:14.901256+0000","last_became_peered":"2024-04-25T10:04:14.901256+0000","last_unstale":"2024-04-25T10:04:15.890983+0000","last_undegraded":"2024-04-25T10:04:15.890983+0000","last_fullsized":"2024-04-25T10:04:15.890983+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:09.286870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892036+0000","last_change":"2024-04-25T10:04:13.910016+0000","last_active":"2024-04-25T10:04:15.892036+0000","last_peered":"2024-04-25T10:04:15.892036+0000","last_clean":"2024-04-25T10:04:15.892036+0000","last_became_active":"2024-04-25T10:04:13.909591+0000","last_became_peered":"2024-04-25T10:04:13.909591+0000","last_unstale":"2024-04-25T10:04:15.892036+0000","last_undegraded":"2024-04-25T10:04:15.892036+0000","last_fullsized":"2024-04-25T10:04:15.892036+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:21.559345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238551+0000","last_change":"2024-04-25T10:04:14.922323+0000","last_active":"2024-04-25T10:04:16.238551+0000","last_peered":"2024-04-25T10:04:16.238551+0000","last_clean":"2024-04-25T10:04:16.238551+0000","last_became_active":"2024-04-25T10:04:14.922051+0000","last_became_peered":"2024-04-25T10:04:14.922051+0000","last_unstale":"2024-04-25T10:04:16.238551+0000","last_undegraded":"2024-04-25T10:04:16.238551+0000","last_fullsized":"2024-04-25T10:04:16.238551+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:00.158092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237892+0000","last_change":"2024-04-25T10:04:13.899980+0000","last_active":"2024-04-25T10:04:16.237892+0000","last_peered":"2024-04-25T10:04:16.237892+0000","last_clean":"2024-04-25T10:04:16.237892+0000","last_became_active":"2024-04-25T10:04:13.899671+0000","last_became_peered":"2024-04-25T10:04:13.899671+0000","last_unstale":"2024-04-25T10:04:16.237892+0000","last_undegraded":"2024-04-25T10:04:16.237892+0000","last_fullsized":"2024-04-25T10:04:16.237892+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:12.291155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892155+0000","last_change":"2024-04-25T10:04:14.906536+0000","last_active":"2024-04-25T10:04:15.892155+0000","last_peered":"2024-04-25T10:04:15.892155+0000","last_clean":"2024-04-25T10:04:15.892155+0000","last_became_active":"2024-04-25T10:04:14.906253+0000","last_became_peered":"2024-04-25T10:04:14.906253+0000","last_unstale":"2024-04-25T10:04:15.892155+0000","last_undegraded":"2024-04-25T10:04:15.892155+0000","last_fullsized":"2024-04-25T10:04:15.892155+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:23.385728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237327+0000","last_change":"2024-04-25T10:04:13.910423+0000","last_active":"2024-04-25T10:04:16.237327+0000","last_peered":"2024-04-25T10:04:16.237327+0000","last_clean":"2024-04-25T10:04:16.237327+0000","last_became_active":"2024-04-25T10:04:13.910113+0000","last_became_peered":"2024-04-25T10:04:13.910113+0000","last_unstale":"2024-04-25T10:04:16.237327+0000","last_undegraded":"2024-04-25T10:04:16.237327+0000","last_fullsized":"2024-04-25T10:04:16.237327+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:52.368366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237587+0000","last_change":"2024-04-25T10:04:14.917448+0000","last_active":"2024-04-25T10:04:16.237587+0000","last_peered":"2024-04-25T10:04:16.237587+0000","last_clean":"2024-04-25T10:04:16.237587+0000","last_became_active":"2024-04-25T10:04:14.917173+0000","last_became_peered":"2024-04-25T10:04:14.917173+0000","last_unstale":"2024-04-25T10:04:16.237587+0000","last_undegraded":"2024-04-25T10:04:16.237587+0000","last_fullsized":"2024-04-25T10:04:16.237587+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:12.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237474+0000","last_change":"2024-04-25T10:04:13.903829+0000","last_active":"2024-04-25T10:04:16.237474+0000","last_peered":"2024-04-25T10:04:16.237474+0000","last_clean":"2024-04-25T10:04:16.237474+0000","last_became_active":"2024-04-25T10:04:13.901705+0000","last_became_peered":"2024-04-25T10:04:13.901705+0000","last_unstale":"2024-04-25T10:04:16.237474+0000","last_undegraded":"2024-04-25T10:04:16.237474+0000","last_fullsized":"2024-04-25T10:04:16.237474+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:19.830958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":465,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892090+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:04:15.892090+0000","last_peered":"2024-04-25T10:04:15.892090+0000","last_clean":"2024-04-25T10:04:15.892090+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:04:15.892090+0000","last_undegraded":"2024-04-25T10:04:15.892090+0000","last_fullsized":"2024-04-25T10:04:15.892090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51844,"kb_used_data":2612,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319996,"statfs":{"total":96636764160,"available":96583675904,"internally_reserved":0,"allocated":2674688,"data_stored":1566346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51264,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320576,"statfs":{"total":96636764160,"available":96584269824,"internally_reserved":0,"allocated":2084864,"data_stored":976048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51276,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320564,"statfs":{"total":96636764160,"available":96584257536,"internally_reserved":0,"allocated":2097152,"data_stored":977612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59596,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312244,"statfs":{"total":96636764160,"available":96575737856,"internally_reserved":0,"allocated":2097152,"data_stored":977624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59596,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312244,"statfs":{"total":96636764160,"available":96575737856,"internally_reserved":0,"allocated":2088960,"data_stored":976138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55432,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316408,"statfs":{"total":96636764160,"available":96580001792,"internally_reserved":0,"allocated":2084864,"data_stored":976560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55424,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316416,"statfs":{"total":96636764160,"available":96580009984,"internally_reserved":0,"allocated":2080768,"data_stored":975978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56020,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315820,"statfs":{"total":96636764160,"available":96579399680,"internally_reserved":0,"allocated":2691072,"data_stored":1567074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:20.863 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T10:04:20.857+0000 7ffb5370b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:04:22.332 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.440+0000 7f1ad6922640 1 Processor -- start 2024-04-25T10:04:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.440+0000 7f1ad6922640 1 -- start start 2024-04-25T10:04:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1ad6922640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1ad015da70 0x7f1ad0161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1ad6922640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ad00b9730 0x7f1ad00ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1ad6922640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad0162020 con 0x7f1ad0161b80 2024-04-25T10:04:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1ad6922640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad01625e0 con 0x7f1ad00b9730 2024-04-25T10:04:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1ad6922640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad0162300 con 0x7f1ad015da70 2024-04-25T10:04:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1acf7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ad00b9730 0x7f1ad00ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1acf7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ad00b9730 0x7f1ad00ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57002/0 (socket says 172.21.15.19:57002) 2024-04-25T10:04:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1acf7fe640 1 -- 172.21.15.19:0/2531744973 learned_addr learned my addr 172.21.15.19:0/2531744973 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1acf7fe640 1 -- 172.21.15.19:0/2531744973 >> v2:172.21.15.112:3300/0 conn(0x7f1ad015da70 msgr2=0x7f1ad0161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1acffff640 1 --2- 172.21.15.19:0/2531744973 >> v2:172.21.15.112:3300/0 conn(0x7f1ad015da70 0x7f1ad0161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1acf7fe640 1 --2- 172.21.15.19:0/2531744973 >> v2:172.21.15.112:3300/0 conn(0x7f1ad015da70 0x7f1ad0161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1acf7fe640 1 -- 172.21.15.19:0/2531744973 >> v1:172.21.15.112:6789/0 conn(0x7f1ad0161b80 legacy=0x7f1ad006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1acf7fe640 1 -- 172.21.15.19:0/2531744973 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00d1af0 con 0x7f1ad00b9730 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.441+0000 7f1acffff640 1 --2- 172.21.15.19:0/2531744973 >> v2:172.21.15.112:3300/0 conn(0x7f1ad015da70 0x7f1ad0161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.442+0000 7f1acf7fe640 1 --2- 172.21.15.19:0/2531744973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad00b9730 0x7f1ad00ce650 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00b61a0 tx=0x7f1ab802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f2b5a62b1da19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.442+0000 7f1aceffd640 1 -- 172.21.15.19:0/2531744973 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8037070 con 0x7f1ad00b9730 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.442+0000 7f1aceffd640 1 -- 172.21.15.19:0/2531744973 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab8032440 con 0x7f1ad00b9730 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.442+0000 7f1aceffd640 1 -- 172.21.15.19:0/2531744973 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8032810 con 0x7f1ad00b9730 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.443+0000 7f1ad6922640 1 -- 172.21.15.19:0/2531744973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad00b9730 msgr2=0x7f1ad00ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.443+0000 7f1ad6922640 1 --2- 172.21.15.19:0/2531744973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad00b9730 0x7f1ad00ce650 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00b61a0 tx=0x7f1ab802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.443+0000 7f1ad6922640 1 -- 172.21.15.19:0/2531744973 shutdown_connections 2024-04-25T10:04:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.443+0000 7f1ad6922640 1 --2- 172.21.15.19:0/2531744973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad00b9730 0x7f1ad00ce650 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.443+0000 7f1ad6922640 1 --2- 172.21.15.19:0/2531744973 >> v2:172.21.15.112:3300/0 conn(0x7f1ad015da70 0x7f1ad0161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.443+0000 7f1ad6922640 1 -- 172.21.15.19:0/2531744973 >> 172.21.15.19:0/2531744973 conn(0x7f1ad0061a20 msgr2=0x7f1ad0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.443+0000 7f1ad6922640 1 -- 172.21.15.19:0/2531744973 shutdown_connections 2024-04-25T10:04:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.443+0000 7f1ad6922640 1 -- 172.21.15.19:0/2531744973 wait complete. 2024-04-25T10:04:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.444+0000 7f1ad6922640 1 Processor -- start 2024-04-25T10:04:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.444+0000 7f1ad6922640 1 -- start start 2024-04-25T10:04:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.445+0000 7f1ad6922640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad015da70 0x7f1ad01523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.445+0000 7f1ad6922640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad0161b80 0x7f1ad00c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.445+0000 7f1ad6922640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ad00c83a0 0x7f1ad0154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.445+0000 7f1ad6922640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00d3300 con 0x7f1ad015da70 2024-04-25T10:04:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.445+0000 7f1ad6922640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad015de50 con 0x7f1ad0161b80 2024-04-25T10:04:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.445+0000 7f1ad6922640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00d3480 con 0x7f1ad00c83a0 2024-04-25T10:04:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.445+0000 7f1acffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad015da70 0x7f1ad01523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.445+0000 7f1acffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad015da70 0x7f1ad01523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57010/0 (socket says 172.21.15.19:57010) 2024-04-25T10:04:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.445+0000 7f1acffff640 1 -- 172.21.15.19:0/3710126228 learned_addr learned my addr 172.21.15.19:0/3710126228 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.445+0000 7f1acf7fe640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad0161b80 0x7f1ad00c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.445+0000 7f1ad4ebf640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ad00c83a0 0x7f1ad0154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1acffff640 1 -- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ad00c83a0 msgr2=0x7f1ad0154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1acffff640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ad00c83a0 0x7f1ad0154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1acffff640 1 -- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad0161b80 msgr2=0x7f1ad00c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1acffff640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad0161b80 0x7f1ad00c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1acffff640 1 -- 172.21.15.19:0/3710126228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00cade0 con 0x7f1ad015da70 2024-04-25T10:04:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1ad4ebf640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ad00c83a0 0x7f1ad0154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1acf7fe640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad0161b80 0x7f1ad00c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1acffff640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad015da70 0x7f1ad01523e0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c7f0 tx=0x7f1ac000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1accff9640 1 -- 172.21.15.19:0/3710126228 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0015070 con 0x7f1ad015da70 2024-04-25T10:04:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1accff9640 1 -- 172.21.15.19:0/3710126228 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac0007a30 con 0x7f1ad015da70 2024-04-25T10:04:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1ad6922640 1 -- 172.21.15.19:0/3710126228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad00c8820 con 0x7f1ad015da70 2024-04-25T10:04:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1accff9640 1 -- 172.21.15.19:0/3710126228 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0007e00 con 0x7f1ad015da70 2024-04-25T10:04:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.446+0000 7f1ad6922640 1 -- 172.21.15.19:0/3710126228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad00c8de0 con 0x7f1ad015da70 2024-04-25T10:04:22.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.447+0000 7f1ad6922640 1 -- 172.21.15.19:0/3710126228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad0073990 con 0x7f1ad015da70 2024-04-25T10:04:22.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.448+0000 7f1accff9640 1 -- 172.21.15.19:0/3710126228 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ac0018070 con 0x7f1ad015da70 2024-04-25T10:04:22.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.448+0000 7f1accff9640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ab006b4d0 0x7f1ab006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.448+0000 7f1accff9640 1 -- 172.21.15.19:0/3710126228 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ac008a250 con 0x7f1ad015da70 2024-04-25T10:04:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.448+0000 7f1acf7fe640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ab006b4d0 0x7f1ab006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.449+0000 7f1acf7fe640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ab006b4d0 0x7f1ab006d990 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8034000 tx=0x7f1ab802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.452+0000 7f1accff9640 1 -- 172.21.15.19:0/3710126228 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ac0058fd0 con 0x7f1ad015da70 2024-04-25T10:04:22.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.630+0000 7f1ad6922640 1 -- 172.21.15.19:0/3710126228 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ad00d1570 con 0x7f1ab006b4d0 2024-04-25T10:04:22.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.638+0000 7f1accff9640 1 -- 172.21.15.19:0/3710126228 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252472 (secure 0 0 0) 0x7f1ab807fa50 con 0x7f1ab006b4d0 2024-04-25T10:04:22.649 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:22.650 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:04:22.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.640+0000 7f1ad6922640 1 -- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ab006b4d0 msgr2=0x7f1ab006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.640+0000 7f1ad6922640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ab006b4d0 0x7f1ab006d990 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8034000 tx=0x7f1ab802e000 comp rx=0 tx=0).stop 2024-04-25T10:04:22.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.641+0000 7f1ad6922640 1 -- 172.21.15.19:0/3710126228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad015da70 msgr2=0x7f1ad01523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.641+0000 7f1ad6922640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad015da70 0x7f1ad01523e0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c7f0 tx=0x7f1ac000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.641+0000 7f1ad6922640 1 -- 172.21.15.19:0/3710126228 shutdown_connections 2024-04-25T10:04:22.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.641+0000 7f1ad6922640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1ab006b4d0 0x7f1ab006d990 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.641+0000 7f1ad6922640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ad00c83a0 0x7f1ad0154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.641+0000 7f1ad6922640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ad0161b80 0x7f1ad00c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.641+0000 7f1ad6922640 1 --2- 172.21.15.19:0/3710126228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad015da70 0x7f1ad01523e0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.641+0000 7f1ad6922640 1 -- 172.21.15.19:0/3710126228 >> 172.21.15.19:0/3710126228 conn(0x7f1ad0061a20 msgr2=0x7f1ad00ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.641+0000 7f1ad6922640 1 -- 172.21.15.19:0/3710126228 shutdown_connections 2024-04-25T10:04:22.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.641+0000 7f1ad6922640 1 -- 172.21.15.19:0/3710126228 wait complete. 2024-04-25T10:04:22.657 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-04-25T10:04:21.069666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440452,"kb_used_data":17480,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754534268,"statfs":{"total":773094113280,"available":772643090432,"internally_reserved":0,"allocated":17899520,"data_stored":8993380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.183815"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238494+0000","last_change":"2024-04-25T10:04:14.914827+0000","last_active":"2024-04-25T10:04:16.238494+0000","last_peered":"2024-04-25T10:04:16.238494+0000","last_clean":"2024-04-25T10:04:16.238494+0000","last_became_active":"2024-04-25T10:04:14.914441+0000","last_became_peered":"2024-04-25T10:04:14.914441+0000","last_unstale":"2024-04-25T10:04:16.238494+0000","last_undegraded":"2024-04-25T10:04:16.238494+0000","last_fullsized":"2024-04-25T10:04:16.238494+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:20.833068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237589+0000","last_change":"2024-04-25T10:04:14.923366+0000","last_active":"2024-04-25T10:04:16.237589+0000","last_peered":"2024-04-25T10:04:16.237589+0000","last_clean":"2024-04-25T10:04:16.237589+0000","last_became_active":"2024-04-25T10:04:14.923076+0000","last_became_peered":"2024-04-25T10:04:14.923076+0000","last_unstale":"2024-04-25T10:04:16.237589+0000","last_undegraded":"2024-04-25T10:04:16.237589+0000","last_fullsized":"2024-04-25T10:04:16.237589+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:38.890605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237663+0000","last_change":"2024-04-25T10:04:14.902149+0000","last_active":"2024-04-25T10:04:16.237663+0000","last_peered":"2024-04-25T10:04:16.237663+0000","last_clean":"2024-04-25T10:04:16.237663+0000","last_became_active":"2024-04-25T10:04:14.901865+0000","last_became_peered":"2024-04-25T10:04:14.901865+0000","last_unstale":"2024-04-25T10:04:16.237663+0000","last_undegraded":"2024-04-25T10:04:16.237663+0000","last_fullsized":"2024-04-25T10:04:16.237663+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:34.403630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238455+0000","last_change":"2024-04-25T10:04:14.895964+0000","last_active":"2024-04-25T10:04:16.238455+0000","last_peered":"2024-04-25T10:04:16.238455+0000","last_clean":"2024-04-25T10:04:16.238455+0000","last_became_active":"2024-04-25T10:04:14.895669+0000","last_became_peered":"2024-04-25T10:04:14.895669+0000","last_unstale":"2024-04-25T10:04:16.238455+0000","last_undegraded":"2024-04-25T10:04:16.238455+0000","last_fullsized":"2024-04-25T10:04:16.238455+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.428364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889944+0000","last_change":"2024-04-25T10:04:14.916200+0000","last_active":"2024-04-25T10:04:15.889944+0000","last_peered":"2024-04-25T10:04:15.889944+0000","last_clean":"2024-04-25T10:04:15.889944+0000","last_became_active":"2024-04-25T10:04:14.915920+0000","last_became_peered":"2024-04-25T10:04:14.915920+0000","last_unstale":"2024-04-25T10:04:15.889944+0000","last_undegraded":"2024-04-25T10:04:15.889944+0000","last_fullsized":"2024-04-25T10:04:15.889944+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:05.202335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889851+0000","last_change":"2024-04-25T10:04:14.922953+0000","last_active":"2024-04-25T10:04:15.889851+0000","last_peered":"2024-04-25T10:04:15.889851+0000","last_clean":"2024-04-25T10:04:15.889851+0000","last_became_active":"2024-04-25T10:04:14.922631+0000","last_became_peered":"2024-04-25T10:04:14.922631+0000","last_unstale":"2024-04-25T10:04:15.889851+0000","last_undegraded":"2024-04-25T10:04:15.889851+0000","last_fullsized":"2024-04-25T10:04:15.889851+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:08.347746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237488+0000","last_change":"2024-04-25T10:04:14.910528+0000","last_active":"2024-04-25T10:04:16.237488+0000","last_peered":"2024-04-25T10:04:16.237488+0000","last_clean":"2024-04-25T10:04:16.237488+0000","last_became_active":"2024-04-25T10:04:14.910239+0000","last_became_peered":"2024-04-25T10:04:14.910239+0000","last_unstale":"2024-04-25T10:04:16.237488+0000","last_undegraded":"2024-04-25T10:04:16.237488+0000","last_fullsized":"2024-04-25T10:04:16.237488+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:26.659240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892111+0000","last_change":"2024-04-25T10:04:14.908178+0000","last_active":"2024-04-25T10:04:15.892111+0000","last_peered":"2024-04-25T10:04:15.892111+0000","last_clean":"2024-04-25T10:04:15.892111+0000","last_became_active":"2024-04-25T10:04:14.907839+0000","last_became_peered":"2024-04-25T10:04:14.907839+0000","last_unstale":"2024-04-25T10:04:15.892111+0000","last_undegraded":"2024-04-25T10:04:15.892111+0000","last_fullsized":"2024-04-25T10:04:15.892111+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:59.027629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238568+0000","last_change":"2024-04-25T10:04:14.913322+0000","last_active":"2024-04-25T10:04:16.238568+0000","last_peered":"2024-04-25T10:04:16.238568+0000","last_clean":"2024-04-25T10:04:16.238568+0000","last_became_active":"2024-04-25T10:04:14.908095+0000","last_became_peered":"2024-04-25T10:04:14.908095+0000","last_unstale":"2024-04-25T10:04:16.238568+0000","last_undegraded":"2024-04-25T10:04:16.238568+0000","last_fullsized":"2024-04-25T10:04:16.238568+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.715750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890301+0000","last_change":"2024-04-25T10:04:14.921222+0000","last_active":"2024-04-25T10:04:15.890301+0000","last_peered":"2024-04-25T10:04:15.890301+0000","last_clean":"2024-04-25T10:04:15.890301+0000","last_became_active":"2024-04-25T10:04:14.920566+0000","last_became_peered":"2024-04-25T10:04:14.920566+0000","last_unstale":"2024-04-25T10:04:15.890301+0000","last_undegraded":"2024-04-25T10:04:15.890301+0000","last_fullsized":"2024-04-25T10:04:15.890301+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.428364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237859+0000","last_change":"2024-04-25T10:04:14.907617+0000","last_active":"2024-04-25T10:04:16.237859+0000","last_peered":"2024-04-25T10:04:16.237859+0000","last_clean":"2024-04-25T10:04:16.237859+0000","last_became_active":"2024-04-25T10:04:14.907049+0000","last_became_peered":"2024-04-25T10:04:14.907049+0000","last_unstale":"2024-04-25T10:04:16.237859+0000","last_undegraded":"2024-04-25T10:04:16.237859+0000","last_fullsized":"2024-04-25T10:04:16.237859+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:49.307475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238601+0000","last_change":"2024-04-25T10:04:14.906433+0000","last_active":"2024-04-25T10:04:16.238601+0000","last_peered":"2024-04-25T10:04:16.238601+0000","last_clean":"2024-04-25T10:04:16.238601+0000","last_became_active":"2024-04-25T10:04:14.906171+0000","last_became_peered":"2024-04-25T10:04:14.906171+0000","last_unstale":"2024-04-25T10:04:16.238601+0000","last_undegraded":"2024-04-25T10:04:16.238601+0000","last_fullsized":"2024-04-25T10:04:16.238601+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:19.360586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891131+0000","last_change":"2024-04-25T10:04:14.898838+0000","last_active":"2024-04-25T10:04:15.891131+0000","last_peered":"2024-04-25T10:04:15.891131+0000","last_clean":"2024-04-25T10:04:15.891131+0000","last_became_active":"2024-04-25T10:04:14.898538+0000","last_became_peered":"2024-04-25T10:04:14.898538+0000","last_unstale":"2024-04-25T10:04:15.891131+0000","last_undegraded":"2024-04-25T10:04:15.891131+0000","last_fullsized":"2024-04-25T10:04:15.891131+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:44.520115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237248+0000","last_change":"2024-04-25T10:04:14.910913+0000","last_active":"2024-04-25T10:04:16.237248+0000","last_peered":"2024-04-25T10:04:16.237248+0000","last_clean":"2024-04-25T10:04:16.237248+0000","last_became_active":"2024-04-25T10:04:14.910077+0000","last_became_peered":"2024-04-25T10:04:14.910077+0000","last_unstale":"2024-04-25T10:04:16.237248+0000","last_undegraded":"2024-04-25T10:04:16.237248+0000","last_fullsized":"2024-04-25T10:04:16.237248+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:52.598811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238115+0000","last_change":"2024-04-25T10:04:14.917611+0000","last_active":"2024-04-25T10:04:16.238115+0000","last_peered":"2024-04-25T10:04:16.238115+0000","last_clean":"2024-04-25T10:04:16.238115+0000","last_became_active":"2024-04-25T10:04:14.913917+0000","last_became_peered":"2024-04-25T10:04:14.913917+0000","last_unstale":"2024-04-25T10:04:16.238115+0000","last_undegraded":"2024-04-25T10:04:16.238115+0000","last_fullsized":"2024-04-25T10:04:16.238115+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:18.452648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889810+0000","last_change":"2024-04-25T10:04:14.911686+0000","last_active":"2024-04-25T10:04:15.889810+0000","last_peered":"2024-04-25T10:04:15.889810+0000","last_clean":"2024-04-25T10:04:15.889810+0000","last_became_active":"2024-04-25T10:04:14.909395+0000","last_became_peered":"2024-04-25T10:04:14.909395+0000","last_unstale":"2024-04-25T10:04:15.889810+0000","last_undegraded":"2024-04-25T10:04:15.889810+0000","last_fullsized":"2024-04-25T10:04:15.889810+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:36.960234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890412+0000","last_change":"2024-04-25T10:04:14.922314+0000","last_active":"2024-04-25T10:04:15.890412+0000","last_peered":"2024-04-25T10:04:15.890412+0000","last_clean":"2024-04-25T10:04:15.890412+0000","last_became_active":"2024-04-25T10:04:14.922033+0000","last_became_peered":"2024-04-25T10:04:14.922033+0000","last_unstale":"2024-04-25T10:04:15.890412+0000","last_undegraded":"2024-04-25T10:04:15.890412+0000","last_fullsized":"2024-04-25T10:04:15.890412+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:19.360586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891442+0000","last_change":"2024-04-25T10:04:14.893601+0000","last_active":"2024-04-25T10:04:15.891442+0000","last_peered":"2024-04-25T10:04:15.891442+0000","last_clean":"2024-04-25T10:04:15.891442+0000","last_became_active":"2024-04-25T10:04:14.893293+0000","last_became_peered":"2024-04-25T10:04:14.893293+0000","last_unstale":"2024-04-25T10:04:15.891442+0000","last_undegraded":"2024-04-25T10:04:15.891442+0000","last_fullsized":"2024-04-25T10:04:15.891442+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:58.335439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891513+0000","last_change":"2024-04-25T10:04:14.902677+0000","last_active":"2024-04-25T10:04:15.891513+0000","last_peered":"2024-04-25T10:04:15.891513+0000","last_clean":"2024-04-25T10:04:15.891513+0000","last_became_active":"2024-04-25T10:04:14.902386+0000","last_became_peered":"2024-04-25T10:04:14.902386+0000","last_unstale":"2024-04-25T10:04:15.891513+0000","last_undegraded":"2024-04-25T10:04:15.891513+0000","last_fullsized":"2024-04-25T10:04:15.891513+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:05.308139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889925+0000","last_change":"2024-04-25T10:04:14.923102+0000","last_active":"2024-04-25T10:04:15.889925+0000","last_peered":"2024-04-25T10:04:15.889925+0000","last_clean":"2024-04-25T10:04:15.889925+0000","last_became_active":"2024-04-25T10:04:14.922796+0000","last_became_peered":"2024-04-25T10:04:14.922796+0000","last_unstale":"2024-04-25T10:04:15.889925+0000","last_undegraded":"2024-04-25T10:04:15.889925+0000","last_fullsized":"2024-04-25T10:04:15.889925+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:34.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238136+0000","last_change":"2024-04-25T10:04:14.908964+0000","last_active":"2024-04-25T10:04:16.238136+0000","last_peered":"2024-04-25T10:04:16.238136+0000","last_clean":"2024-04-25T10:04:16.238136+0000","last_became_active":"2024-04-25T10:04:14.908648+0000","last_became_peered":"2024-04-25T10:04:14.908648+0000","last_unstale":"2024-04-25T10:04:16.238136+0000","last_undegraded":"2024-04-25T10:04:16.238136+0000","last_fullsized":"2024-04-25T10:04:16.238136+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:45.454649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238390+0000","last_change":"2024-04-25T10:04:14.923545+0000","last_active":"2024-04-25T10:04:16.238390+0000","last_peered":"2024-04-25T10:04:16.238390+0000","last_clean":"2024-04-25T10:04:16.238390+0000","last_became_active":"2024-04-25T10:04:14.923209+0000","last_became_peered":"2024-04-25T10:04:14.923209+0000","last_unstale":"2024-04-25T10:04:16.238390+0000","last_undegraded":"2024-04-25T10:04:16.238390+0000","last_fullsized":"2024-04-25T10:04:16.238390+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:00.126498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892096+0000","last_change":"2024-04-25T10:04:14.905495+0000","last_active":"2024-04-25T10:04:15.892096+0000","last_peered":"2024-04-25T10:04:15.892096+0000","last_clean":"2024-04-25T10:04:15.892096+0000","last_became_active":"2024-04-25T10:04:14.905164+0000","last_became_peered":"2024-04-25T10:04:14.905164+0000","last_unstale":"2024-04-25T10:04:15.892096+0000","last_undegraded":"2024-04-25T10:04:15.892096+0000","last_fullsized":"2024-04-25T10:04:15.892096+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:45.287354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237196+0000","last_change":"2024-04-25T10:04:14.914850+0000","last_active":"2024-04-25T10:04:16.237196+0000","last_peered":"2024-04-25T10:04:16.237196+0000","last_clean":"2024-04-25T10:04:16.237196+0000","last_became_active":"2024-04-25T10:04:14.914592+0000","last_became_peered":"2024-04-25T10:04:14.914592+0000","last_unstale":"2024-04-25T10:04:16.237196+0000","last_undegraded":"2024-04-25T10:04:16.237196+0000","last_fullsized":"2024-04-25T10:04:16.237196+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:54.434833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237253+0000","last_change":"2024-04-25T10:04:14.917923+0000","last_active":"2024-04-25T10:04:16.237253+0000","last_peered":"2024-04-25T10:04:16.237253+0000","last_clean":"2024-04-25T10:04:16.237253+0000","last_became_active":"2024-04-25T10:04:14.917630+0000","last_became_peered":"2024-04-25T10:04:14.917630+0000","last_unstale":"2024-04-25T10:04:16.237253+0000","last_undegraded":"2024-04-25T10:04:16.237253+0000","last_fullsized":"2024-04-25T10:04:16.237253+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.253359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237884+0000","last_change":"2024-04-25T10:04:14.917336+0000","last_active":"2024-04-25T10:04:16.237884+0000","last_peered":"2024-04-25T10:04:16.237884+0000","last_clean":"2024-04-25T10:04:16.237884+0000","last_became_active":"2024-04-25T10:04:14.916975+0000","last_became_peered":"2024-04-25T10:04:14.916975+0000","last_unstale":"2024-04-25T10:04:16.237884+0000","last_undegraded":"2024-04-25T10:04:16.237884+0000","last_fullsized":"2024-04-25T10:04:16.237884+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:42.694901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892333+0000","last_change":"2024-04-25T10:04:14.909230+0000","last_active":"2024-04-25T10:04:15.892333+0000","last_peered":"2024-04-25T10:04:15.892333+0000","last_clean":"2024-04-25T10:04:15.892333+0000","last_became_active":"2024-04-25T10:04:14.908930+0000","last_became_peered":"2024-04-25T10:04:14.908930+0000","last_unstale":"2024-04-25T10:04:15.892333+0000","last_undegraded":"2024-04-25T10:04:15.892333+0000","last_fullsized":"2024-04-25T10:04:15.892333+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:54.434833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889972+0000","last_change":"2024-04-25T10:04:14.915440+0000","last_active":"2024-04-25T10:04:15.889972+0000","last_peered":"2024-04-25T10:04:15.889972+0000","last_clean":"2024-04-25T10:04:15.889972+0000","last_became_active":"2024-04-25T10:04:14.915147+0000","last_became_peered":"2024-04-25T10:04:14.915147+0000","last_unstale":"2024-04-25T10:04:15.889972+0000","last_undegraded":"2024-04-25T10:04:15.889972+0000","last_fullsized":"2024-04-25T10:04:15.889972+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:17.123316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237693+0000","last_change":"2024-04-25T10:04:14.920213+0000","last_active":"2024-04-25T10:04:16.237693+0000","last_peered":"2024-04-25T10:04:16.237693+0000","last_clean":"2024-04-25T10:04:16.237693+0000","last_became_active":"2024-04-25T10:04:14.919900+0000","last_became_peered":"2024-04-25T10:04:14.919900+0000","last_unstale":"2024-04-25T10:04:16.237693+0000","last_undegraded":"2024-04-25T10:04:16.237693+0000","last_fullsized":"2024-04-25T10:04:16.237693+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:22.561454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237383+0000","last_change":"2024-04-25T10:04:14.919732+0000","last_active":"2024-04-25T10:04:16.237383+0000","last_peered":"2024-04-25T10:04:16.237383+0000","last_clean":"2024-04-25T10:04:16.237383+0000","last_became_active":"2024-04-25T10:04:14.919449+0000","last_became_peered":"2024-04-25T10:04:14.919449+0000","last_unstale":"2024-04-25T10:04:16.237383+0000","last_undegraded":"2024-04-25T10:04:16.237383+0000","last_fullsized":"2024-04-25T10:04:16.237383+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:32.418999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237629+0000","last_change":"2024-04-25T10:04:14.905796+0000","last_active":"2024-04-25T10:04:16.237629+0000","last_peered":"2024-04-25T10:04:16.237629+0000","last_clean":"2024-04-25T10:04:16.237629+0000","last_became_active":"2024-04-25T10:04:14.905497+0000","last_became_peered":"2024-04-25T10:04:14.905497+0000","last_unstale":"2024-04-25T10:04:16.237629+0000","last_undegraded":"2024-04-25T10:04:16.237629+0000","last_fullsized":"2024-04-25T10:04:16.237629+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:05.661329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889889+0000","last_change":"2024-04-25T10:04:14.920427+0000","last_active":"2024-04-25T10:04:15.889889+0000","last_peered":"2024-04-25T10:04:15.889889+0000","last_clean":"2024-04-25T10:04:15.889889+0000","last_became_active":"2024-04-25T10:04:14.920014+0000","last_became_peered":"2024-04-25T10:04:14.920014+0000","last_unstale":"2024-04-25T10:04:15.889889+0000","last_undegraded":"2024-04-25T10:04:15.889889+0000","last_fullsized":"2024-04-25T10:04:15.889889+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:41.695527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892417+0000","last_change":"2024-04-25T10:04:14.896609+0000","last_active":"2024-04-25T10:04:15.892417+0000","last_peered":"2024-04-25T10:04:15.892417+0000","last_clean":"2024-04-25T10:04:15.892417+0000","last_became_active":"2024-04-25T10:04:14.896337+0000","last_became_peered":"2024-04-25T10:04:14.896337+0000","last_unstale":"2024-04-25T10:04:15.892417+0000","last_undegraded":"2024-04-25T10:04:15.892417+0000","last_fullsized":"2024-04-25T10:04:15.892417+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:15.369689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"164'4","reported_seq":18,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237842+0000","last_change":"2024-04-25T10:04:13.905310+0000","last_active":"2024-04-25T10:04:16.237842+0000","last_peered":"2024-04-25T10:04:16.237842+0000","last_clean":"2024-04-25T10:04:16.237842+0000","last_became_active":"2024-04-25T10:04:13.905060+0000","last_became_peered":"2024-04-25T10:04:13.905060+0000","last_unstale":"2024-04-25T10:04:16.237842+0000","last_undegraded":"2024-04-25T10:04:16.237842+0000","last_fullsized":"2024-04-25T10:04:16.237842+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:35.958124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238175+0000","last_change":"2024-04-25T10:04:14.911957+0000","last_active":"2024-04-25T10:04:16.238175+0000","last_peered":"2024-04-25T10:04:16.238175+0000","last_clean":"2024-04-25T10:04:16.238175+0000","last_became_active":"2024-04-25T10:04:14.911671+0000","last_became_peered":"2024-04-25T10:04:14.911671+0000","last_unstale":"2024-04-25T10:04:16.238175+0000","last_undegraded":"2024-04-25T10:04:16.238175+0000","last_fullsized":"2024-04-25T10:04:16.238175+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:31.656397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890389+0000","last_change":"2024-04-25T10:04:13.911506+0000","last_active":"2024-04-25T10:04:15.890389+0000","last_peered":"2024-04-25T10:04:15.890389+0000","last_clean":"2024-04-25T10:04:15.890389+0000","last_became_active":"2024-04-25T10:04:13.909233+0000","last_became_peered":"2024-04-25T10:04:13.909233+0000","last_unstale":"2024-04-25T10:04:15.890389+0000","last_undegraded":"2024-04-25T10:04:15.890389+0000","last_fullsized":"2024-04-25T10:04:15.890389+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:44.285244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238015+0000","last_change":"2024-04-25T10:04:14.908949+0000","last_active":"2024-04-25T10:04:16.238015+0000","last_peered":"2024-04-25T10:04:16.238015+0000","last_clean":"2024-04-25T10:04:16.238015+0000","last_became_active":"2024-04-25T10:04:14.908607+0000","last_became_peered":"2024-04-25T10:04:14.908607+0000","last_unstale":"2024-04-25T10:04:16.238015+0000","last_undegraded":"2024-04-25T10:04:16.238015+0000","last_fullsized":"2024-04-25T10:04:16.238015+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:44.644245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237922+0000","last_change":"2024-04-25T10:04:13.899686+0000","last_active":"2024-04-25T10:04:16.237922+0000","last_peered":"2024-04-25T10:04:16.237922+0000","last_clean":"2024-04-25T10:04:16.237922+0000","last_became_active":"2024-04-25T10:04:13.899387+0000","last_became_peered":"2024-04-25T10:04:13.899387+0000","last_unstale":"2024-04-25T10:04:16.237922+0000","last_undegraded":"2024-04-25T10:04:16.237922+0000","last_fullsized":"2024-04-25T10:04:16.237922+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:43.080781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237638+0000","last_change":"2024-04-25T10:04:14.911170+0000","last_active":"2024-04-25T10:04:16.237638+0000","last_peered":"2024-04-25T10:04:16.237638+0000","last_clean":"2024-04-25T10:04:16.237638+0000","last_became_active":"2024-04-25T10:04:14.910835+0000","last_became_peered":"2024-04-25T10:04:14.910835+0000","last_unstale":"2024-04-25T10:04:16.237638+0000","last_undegraded":"2024-04-25T10:04:16.237638+0000","last_fullsized":"2024-04-25T10:04:16.237638+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:14.681582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891074+0000","last_change":"2024-04-25T10:04:13.899482+0000","last_active":"2024-04-25T10:04:15.891074+0000","last_peered":"2024-04-25T10:04:15.891074+0000","last_clean":"2024-04-25T10:04:15.891074+0000","last_became_active":"2024-04-25T10:04:13.899195+0000","last_became_peered":"2024-04-25T10:04:13.899195+0000","last_unstale":"2024-04-25T10:04:15.891074+0000","last_undegraded":"2024-04-25T10:04:15.891074+0000","last_fullsized":"2024-04-25T10:04:15.891074+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:41.094825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890757+0000","last_change":"2024-04-25T10:04:14.918739+0000","last_active":"2024-04-25T10:04:15.890757+0000","last_peered":"2024-04-25T10:04:15.890757+0000","last_clean":"2024-04-25T10:04:15.890757+0000","last_became_active":"2024-04-25T10:04:14.918447+0000","last_became_peered":"2024-04-25T10:04:14.918447+0000","last_unstale":"2024-04-25T10:04:15.890757+0000","last_undegraded":"2024-04-25T10:04:15.890757+0000","last_fullsized":"2024-04-25T10:04:15.890757+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:11.067114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890102+0000","last_change":"2024-04-25T10:04:13.911021+0000","last_active":"2024-04-25T10:04:15.890102+0000","last_peered":"2024-04-25T10:04:15.890102+0000","last_clean":"2024-04-25T10:04:15.890102+0000","last_became_active":"2024-04-25T10:04:13.909391+0000","last_became_peered":"2024-04-25T10:04:13.909391+0000","last_unstale":"2024-04-25T10:04:15.890102+0000","last_undegraded":"2024-04-25T10:04:15.890102+0000","last_fullsized":"2024-04-25T10:04:15.890102+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:53.432723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238267+0000","last_change":"2024-04-25T10:04:14.907532+0000","last_active":"2024-04-25T10:04:16.238267+0000","last_peered":"2024-04-25T10:04:16.238267+0000","last_clean":"2024-04-25T10:04:16.238267+0000","last_became_active":"2024-04-25T10:04:14.906617+0000","last_became_peered":"2024-04-25T10:04:14.906617+0000","last_unstale":"2024-04-25T10:04:16.238267+0000","last_undegraded":"2024-04-25T10:04:16.238267+0000","last_fullsized":"2024-04-25T10:04:16.238267+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:04.468380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238041+0000","last_change":"2024-04-25T10:04:13.907057+0000","last_active":"2024-04-25T10:04:16.238041+0000","last_peered":"2024-04-25T10:04:16.238041+0000","last_clean":"2024-04-25T10:04:16.238041+0000","last_became_active":"2024-04-25T10:04:13.906778+0000","last_became_peered":"2024-04-25T10:04:13.906778+0000","last_unstale":"2024-04-25T10:04:16.238041+0000","last_undegraded":"2024-04-25T10:04:16.238041+0000","last_fullsized":"2024-04-25T10:04:16.238041+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:55.338829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238412+0000","last_change":"2024-04-25T10:04:14.913844+0000","last_active":"2024-04-25T10:04:16.238412+0000","last_peered":"2024-04-25T10:04:16.238412+0000","last_clean":"2024-04-25T10:04:16.238412+0000","last_became_active":"2024-04-25T10:04:14.913525+0000","last_became_peered":"2024-04-25T10:04:14.913525+0000","last_unstale":"2024-04-25T10:04:16.238412+0000","last_undegraded":"2024-04-25T10:04:16.238412+0000","last_fullsized":"2024-04-25T10:04:16.238412+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:15.657853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890982+0000","last_change":"2024-04-25T10:04:13.894824+0000","last_active":"2024-04-25T10:04:15.890982+0000","last_peered":"2024-04-25T10:04:15.890982+0000","last_clean":"2024-04-25T10:04:15.890982+0000","last_became_active":"2024-04-25T10:04:13.894557+0000","last_became_peered":"2024-04-25T10:04:13.894557+0000","last_unstale":"2024-04-25T10:04:15.890982+0000","last_undegraded":"2024-04-25T10:04:15.890982+0000","last_fullsized":"2024-04-25T10:04:15.890982+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:45.461278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892468+0000","last_change":"2024-04-25T10:04:14.893650+0000","last_active":"2024-04-25T10:04:15.892468+0000","last_peered":"2024-04-25T10:04:15.892468+0000","last_clean":"2024-04-25T10:04:15.892468+0000","last_became_active":"2024-04-25T10:04:14.893293+0000","last_became_peered":"2024-04-25T10:04:14.893293+0000","last_unstale":"2024-04-25T10:04:15.892468+0000","last_undegraded":"2024-04-25T10:04:15.892468+0000","last_fullsized":"2024-04-25T10:04:15.892468+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.026157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890380+0000","last_change":"2024-04-25T10:04:13.912389+0000","last_active":"2024-04-25T10:04:15.890380+0000","last_peered":"2024-04-25T10:04:15.890380+0000","last_clean":"2024-04-25T10:04:15.890380+0000","last_became_active":"2024-04-25T10:04:13.912117+0000","last_became_peered":"2024-04-25T10:04:13.912117+0000","last_unstale":"2024-04-25T10:04:15.890380+0000","last_undegraded":"2024-04-25T10:04:15.890380+0000","last_fullsized":"2024-04-25T10:04:15.890380+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:58.025520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890246+0000","last_change":"2024-04-25T10:04:14.920422+0000","last_active":"2024-04-25T10:04:15.890246+0000","last_peered":"2024-04-25T10:04:15.890246+0000","last_clean":"2024-04-25T10:04:15.890246+0000","last_became_active":"2024-04-25T10:04:14.919739+0000","last_became_peered":"2024-04-25T10:04:14.919739+0000","last_unstale":"2024-04-25T10:04:15.890246+0000","last_undegraded":"2024-04-25T10:04:15.890246+0000","last_fullsized":"2024-04-25T10:04:15.890246+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:15.701395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237736+0000","last_change":"2024-04-25T10:04:13.908763+0000","last_active":"2024-04-25T10:04:16.237736+0000","last_peered":"2024-04-25T10:04:16.237736+0000","last_clean":"2024-04-25T10:04:16.237736+0000","last_became_active":"2024-04-25T10:04:13.908481+0000","last_became_peered":"2024-04-25T10:04:13.908481+0000","last_unstale":"2024-04-25T10:04:16.237736+0000","last_undegraded":"2024-04-25T10:04:16.237736+0000","last_fullsized":"2024-04-25T10:04:16.237736+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:38.122061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891488+0000","last_change":"2024-04-25T10:04:14.910090+0000","last_active":"2024-04-25T10:04:15.891488+0000","last_peered":"2024-04-25T10:04:15.891488+0000","last_clean":"2024-04-25T10:04:15.891488+0000","last_became_active":"2024-04-25T10:04:14.909490+0000","last_became_peered":"2024-04-25T10:04:14.909490+0000","last_unstale":"2024-04-25T10:04:15.891488+0000","last_undegraded":"2024-04-25T10:04:15.891488+0000","last_fullsized":"2024-04-25T10:04:15.891488+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:16.962626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"164'3","reported_seq":17,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238146+0000","last_change":"2024-04-25T10:04:13.909365+0000","last_active":"2024-04-25T10:04:16.238146+0000","last_peered":"2024-04-25T10:04:16.238146+0000","last_clean":"2024-04-25T10:04:16.238146+0000","last_became_active":"2024-04-25T10:04:13.907222+0000","last_became_peered":"2024-04-25T10:04:13.907222+0000","last_unstale":"2024-04-25T10:04:16.238146+0000","last_undegraded":"2024-04-25T10:04:16.238146+0000","last_fullsized":"2024-04-25T10:04:16.238146+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:07.345636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238653+0000","last_change":"2024-04-25T10:04:14.919133+0000","last_active":"2024-04-25T10:04:16.238653+0000","last_peered":"2024-04-25T10:04:16.238653+0000","last_clean":"2024-04-25T10:04:16.238653+0000","last_became_active":"2024-04-25T10:04:14.918862+0000","last_became_peered":"2024-04-25T10:04:14.918862+0000","last_unstale":"2024-04-25T10:04:16.238653+0000","last_undegraded":"2024-04-25T10:04:16.238653+0000","last_fullsized":"2024-04-25T10:04:16.238653+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:56.340939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"164'6","reported_seq":20,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891075+0000","last_change":"2024-04-25T10:04:13.898686+0000","last_active":"2024-04-25T10:04:15.891075+0000","last_peered":"2024-04-25T10:04:15.891075+0000","last_clean":"2024-04-25T10:04:15.891075+0000","last_became_active":"2024-04-25T10:04:13.898413+0000","last_became_peered":"2024-04-25T10:04:13.898413+0000","last_unstale":"2024-04-25T10:04:15.891075+0000","last_undegraded":"2024-04-25T10:04:15.891075+0000","last_fullsized":"2024-04-25T10:04:15.891075+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:52.368366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237057+0000","last_change":"2024-04-25T10:04:14.910641+0000","last_active":"2024-04-25T10:04:16.237057+0000","last_peered":"2024-04-25T10:04:16.237057+0000","last_clean":"2024-04-25T10:04:16.237057+0000","last_became_active":"2024-04-25T10:04:14.910312+0000","last_became_peered":"2024-04-25T10:04:14.910312+0000","last_unstale":"2024-04-25T10:04:16.237057+0000","last_undegraded":"2024-04-25T10:04:16.237057+0000","last_fullsized":"2024-04-25T10:04:16.237057+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:02.199324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891306+0000","last_change":"2024-04-25T10:04:13.897853+0000","last_active":"2024-04-25T10:04:15.891306+0000","last_peered":"2024-04-25T10:04:15.891306+0000","last_clean":"2024-04-25T10:04:15.891306+0000","last_became_active":"2024-04-25T10:04:13.897547+0000","last_became_peered":"2024-04-25T10:04:13.897547+0000","last_unstale":"2024-04-25T10:04:15.891306+0000","last_undegraded":"2024-04-25T10:04:15.891306+0000","last_fullsized":"2024-04-25T10:04:15.891306+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:31.172667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.889824+0000","last_change":"2024-04-25T10:04:14.914301+0000","last_active":"2024-04-25T10:04:15.889824+0000","last_peered":"2024-04-25T10:04:15.889824+0000","last_clean":"2024-04-25T10:04:15.889824+0000","last_became_active":"2024-04-25T10:04:14.909471+0000","last_became_peered":"2024-04-25T10:04:14.909471+0000","last_unstale":"2024-04-25T10:04:15.889824+0000","last_undegraded":"2024-04-25T10:04:15.889824+0000","last_fullsized":"2024-04-25T10:04:15.889824+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:46.662983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238511+0000","last_change":"2024-04-25T10:04:13.901634+0000","last_active":"2024-04-25T10:04:16.238511+0000","last_peered":"2024-04-25T10:04:16.238511+0000","last_clean":"2024-04-25T10:04:16.238511+0000","last_became_active":"2024-04-25T10:04:13.900799+0000","last_became_peered":"2024-04-25T10:04:13.900799+0000","last_unstale":"2024-04-25T10:04:16.238511+0000","last_undegraded":"2024-04-25T10:04:16.238511+0000","last_fullsized":"2024-04-25T10:04:16.238511+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:03.199621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891670+0000","last_change":"2024-04-25T10:04:14.901604+0000","last_active":"2024-04-25T10:04:15.891670+0000","last_peered":"2024-04-25T10:04:15.891670+0000","last_clean":"2024-04-25T10:04:15.891670+0000","last_became_active":"2024-04-25T10:04:14.901362+0000","last_became_peered":"2024-04-25T10:04:14.901362+0000","last_unstale":"2024-04-25T10:04:15.891670+0000","last_undegraded":"2024-04-25T10:04:15.891670+0000","last_fullsized":"2024-04-25T10:04:15.891670+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:39.457441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237632+0000","last_change":"2024-04-25T10:04:13.898306+0000","last_active":"2024-04-25T10:04:16.237632+0000","last_peered":"2024-04-25T10:04:16.237632+0000","last_clean":"2024-04-25T10:04:16.237632+0000","last_became_active":"2024-04-25T10:04:13.898057+0000","last_became_peered":"2024-04-25T10:04:13.898057+0000","last_unstale":"2024-04-25T10:04:16.237632+0000","last_undegraded":"2024-04-25T10:04:16.237632+0000","last_fullsized":"2024-04-25T10:04:16.237632+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:10.713640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891204+0000","last_change":"2024-04-25T10:04:14.905089+0000","last_active":"2024-04-25T10:04:15.891204+0000","last_peered":"2024-04-25T10:04:15.891204+0000","last_clean":"2024-04-25T10:04:15.891204+0000","last_became_active":"2024-04-25T10:04:14.904817+0000","last_became_peered":"2024-04-25T10:04:14.904817+0000","last_unstale":"2024-04-25T10:04:15.891204+0000","last_undegraded":"2024-04-25T10:04:15.891204+0000","last_fullsized":"2024-04-25T10:04:15.891204+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:43.962022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"164'8","reported_seq":22,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237225+0000","last_change":"2024-04-25T10:04:13.902317+0000","last_active":"2024-04-25T10:04:16.237225+0000","last_peered":"2024-04-25T10:04:16.237225+0000","last_clean":"2024-04-25T10:04:16.237225+0000","last_became_active":"2024-04-25T10:04:13.902008+0000","last_became_peered":"2024-04-25T10:04:13.902008+0000","last_unstale":"2024-04-25T10:04:16.237225+0000","last_undegraded":"2024-04-25T10:04:16.237225+0000","last_fullsized":"2024-04-25T10:04:16.237225+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.551407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237639+0000","last_change":"2024-04-25T10:04:14.906405+0000","last_active":"2024-04-25T10:04:16.237639+0000","last_peered":"2024-04-25T10:04:16.237639+0000","last_clean":"2024-04-25T10:04:16.237639+0000","last_became_active":"2024-04-25T10:04:14.906126+0000","last_became_peered":"2024-04-25T10:04:14.906126+0000","last_unstale":"2024-04-25T10:04:16.237639+0000","last_undegraded":"2024-04-25T10:04:16.237639+0000","last_fullsized":"2024-04-25T10:04:16.237639+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:26.159293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238023+0000","last_change":"2024-04-25T10:04:13.902390+0000","last_active":"2024-04-25T10:04:16.238023+0000","last_peered":"2024-04-25T10:04:16.238023+0000","last_clean":"2024-04-25T10:04:16.238023+0000","last_became_active":"2024-04-25T10:04:13.902012+0000","last_became_peered":"2024-04-25T10:04:13.902012+0000","last_unstale":"2024-04-25T10:04:16.238023+0000","last_undegraded":"2024-04-25T10:04:16.238023+0000","last_fullsized":"2024-04-25T10:04:16.238023+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:29.242265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890166+0000","last_change":"2024-04-25T10:04:14.921007+0000","last_active":"2024-04-25T10:04:15.890166+0000","last_peered":"2024-04-25T10:04:15.890166+0000","last_clean":"2024-04-25T10:04:15.890166+0000","last_became_active":"2024-04-25T10:04:14.920141+0000","last_became_peered":"2024-04-25T10:04:14.920141+0000","last_unstale":"2024-04-25T10:04:15.890166+0000","last_undegraded":"2024-04-25T10:04:15.890166+0000","last_fullsized":"2024-04-25T10:04:15.890166+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:47.794132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237595+0000","last_change":"2024-04-25T10:04:13.896683+0000","last_active":"2024-04-25T10:04:16.237595+0000","last_peered":"2024-04-25T10:04:16.237595+0000","last_clean":"2024-04-25T10:04:16.237595+0000","last_became_active":"2024-04-25T10:04:13.896386+0000","last_became_peered":"2024-04-25T10:04:13.896386+0000","last_unstale":"2024-04-25T10:04:16.237595+0000","last_undegraded":"2024-04-25T10:04:16.237595+0000","last_fullsized":"2024-04-25T10:04:16.237595+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:59.155982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238115+0000","last_change":"2024-04-25T10:04:14.917146+0000","last_active":"2024-04-25T10:04:16.238115+0000","last_peered":"2024-04-25T10:04:16.238115+0000","last_clean":"2024-04-25T10:04:16.238115+0000","last_became_active":"2024-04-25T10:04:14.916788+0000","last_became_peered":"2024-04-25T10:04:14.916788+0000","last_unstale":"2024-04-25T10:04:16.238115+0000","last_undegraded":"2024-04-25T10:04:16.238115+0000","last_fullsized":"2024-04-25T10:04:16.238115+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:12.457746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237767+0000","last_change":"2024-04-25T10:04:13.905850+0000","last_active":"2024-04-25T10:04:16.237767+0000","last_peered":"2024-04-25T10:04:16.237767+0000","last_clean":"2024-04-25T10:04:16.237767+0000","last_became_active":"2024-04-25T10:04:13.905565+0000","last_became_peered":"2024-04-25T10:04:13.905565+0000","last_unstale":"2024-04-25T10:04:16.237767+0000","last_undegraded":"2024-04-25T10:04:16.237767+0000","last_fullsized":"2024-04-25T10:04:16.237767+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:41.094825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238677+0000","last_change":"2024-04-25T10:04:14.922523+0000","last_active":"2024-04-25T10:04:16.238677+0000","last_peered":"2024-04-25T10:04:16.238677+0000","last_clean":"2024-04-25T10:04:16.238677+0000","last_became_active":"2024-04-25T10:04:14.922112+0000","last_became_peered":"2024-04-25T10:04:14.922112+0000","last_unstale":"2024-04-25T10:04:16.238677+0000","last_undegraded":"2024-04-25T10:04:16.238677+0000","last_fullsized":"2024-04-25T10:04:16.238677+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:17.961068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237753+0000","last_change":"2024-04-25T10:04:13.899466+0000","last_active":"2024-04-25T10:04:16.237753+0000","last_peered":"2024-04-25T10:04:16.237753+0000","last_clean":"2024-04-25T10:04:16.237753+0000","last_became_active":"2024-04-25T10:04:13.899146+0000","last_became_peered":"2024-04-25T10:04:13.899146+0000","last_unstale":"2024-04-25T10:04:16.237753+0000","last_undegraded":"2024-04-25T10:04:16.237753+0000","last_fullsized":"2024-04-25T10:04:16.237753+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:12.909171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237348+0000","last_change":"2024-04-25T10:04:14.921433+0000","last_active":"2024-04-25T10:04:16.237348+0000","last_peered":"2024-04-25T10:04:16.237348+0000","last_clean":"2024-04-25T10:04:16.237348+0000","last_became_active":"2024-04-25T10:04:14.921129+0000","last_became_peered":"2024-04-25T10:04:14.921129+0000","last_unstale":"2024-04-25T10:04:16.237348+0000","last_undegraded":"2024-04-25T10:04:16.237348+0000","last_fullsized":"2024-04-25T10:04:16.237348+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:15.369689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.891325+0000","last_change":"2024-04-25T10:04:13.907912+0000","last_active":"2024-04-25T10:04:15.891325+0000","last_peered":"2024-04-25T10:04:15.891325+0000","last_clean":"2024-04-25T10:04:15.891325+0000","last_became_active":"2024-04-25T10:04:13.907651+0000","last_became_peered":"2024-04-25T10:04:13.907651+0000","last_unstale":"2024-04-25T10:04:15.891325+0000","last_undegraded":"2024-04-25T10:04:15.891325+0000","last_fullsized":"2024-04-25T10:04:15.891325+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:43.570072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237535+0000","last_change":"2024-04-25T10:04:14.896251+0000","last_active":"2024-04-25T10:04:16.237535+0000","last_peered":"2024-04-25T10:04:16.237535+0000","last_clean":"2024-04-25T10:04:16.237535+0000","last_became_active":"2024-04-25T10:04:14.895304+0000","last_became_peered":"2024-04-25T10:04:14.895304+0000","last_unstale":"2024-04-25T10:04:16.237535+0000","last_undegraded":"2024-04-25T10:04:16.237535+0000","last_fullsized":"2024-04-25T10:04:16.237535+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:47.794132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238147+0000","last_change":"2024-04-25T10:04:13.904679+0000","last_active":"2024-04-25T10:04:16.238147+0000","last_peered":"2024-04-25T10:04:16.238147+0000","last_clean":"2024-04-25T10:04:16.238147+0000","last_became_active":"2024-04-25T10:04:13.904426+0000","last_became_peered":"2024-04-25T10:04:13.904426+0000","last_unstale":"2024-04-25T10:04:16.238147+0000","last_undegraded":"2024-04-25T10:04:16.238147+0000","last_fullsized":"2024-04-25T10:04:16.238147+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:16.958959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237950+0000","last_change":"2024-04-25T10:04:14.908665+0000","last_active":"2024-04-25T10:04:16.237950+0000","last_peered":"2024-04-25T10:04:16.237950+0000","last_clean":"2024-04-25T10:04:16.237950+0000","last_became_active":"2024-04-25T10:04:14.908392+0000","last_became_peered":"2024-04-25T10:04:14.908392+0000","last_unstale":"2024-04-25T10:04:16.237950+0000","last_undegraded":"2024-04-25T10:04:16.237950+0000","last_fullsized":"2024-04-25T10:04:16.237950+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:15.701395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237792+0000","last_change":"2024-04-25T10:04:13.898826+0000","last_active":"2024-04-25T10:04:16.237792+0000","last_peered":"2024-04-25T10:04:16.237792+0000","last_clean":"2024-04-25T10:04:16.237792+0000","last_became_active":"2024-04-25T10:04:13.898462+0000","last_became_peered":"2024-04-25T10:04:13.898462+0000","last_unstale":"2024-04-25T10:04:16.237792+0000","last_undegraded":"2024-04-25T10:04:16.237792+0000","last_fullsized":"2024-04-25T10:04:16.237792+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:41.233807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890041+0000","last_change":"2024-04-25T10:04:14.917009+0000","last_active":"2024-04-25T10:04:15.890041+0000","last_peered":"2024-04-25T10:04:15.890041+0000","last_clean":"2024-04-25T10:04:15.890041+0000","last_became_active":"2024-04-25T10:04:14.916661+0000","last_became_peered":"2024-04-25T10:04:14.916661+0000","last_unstale":"2024-04-25T10:04:15.890041+0000","last_undegraded":"2024-04-25T10:04:15.890041+0000","last_fullsized":"2024-04-25T10:04:15.890041+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:17.029553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237272+0000","last_change":"2024-04-25T10:04:13.899083+0000","last_active":"2024-04-25T10:04:16.237272+0000","last_peered":"2024-04-25T10:04:16.237272+0000","last_clean":"2024-04-25T10:04:16.237272+0000","last_became_active":"2024-04-25T10:04:13.898810+0000","last_became_peered":"2024-04-25T10:04:13.898810+0000","last_unstale":"2024-04-25T10:04:16.237272+0000","last_undegraded":"2024-04-25T10:04:16.237272+0000","last_fullsized":"2024-04-25T10:04:16.237272+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:45.461278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237783+0000","last_change":"2024-04-25T10:04:14.908754+0000","last_active":"2024-04-25T10:04:16.237783+0000","last_peered":"2024-04-25T10:04:16.237783+0000","last_clean":"2024-04-25T10:04:16.237783+0000","last_became_active":"2024-04-25T10:04:14.908503+0000","last_became_peered":"2024-04-25T10:04:14.908503+0000","last_unstale":"2024-04-25T10:04:16.237783+0000","last_undegraded":"2024-04-25T10:04:16.237783+0000","last_fullsized":"2024-04-25T10:04:16.237783+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:55.705402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237952+0000","last_change":"2024-04-25T10:04:13.899508+0000","last_active":"2024-04-25T10:04:16.237952+0000","last_peered":"2024-04-25T10:04:16.237952+0000","last_clean":"2024-04-25T10:04:16.237952+0000","last_became_active":"2024-04-25T10:04:13.898618+0000","last_became_peered":"2024-04-25T10:04:13.898618+0000","last_unstale":"2024-04-25T10:04:16.237952+0000","last_undegraded":"2024-04-25T10:04:16.237952+0000","last_fullsized":"2024-04-25T10:04:16.237952+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:45.660873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237429+0000","last_change":"2024-04-25T10:04:14.913939+0000","last_active":"2024-04-25T10:04:16.237429+0000","last_peered":"2024-04-25T10:04:16.237429+0000","last_clean":"2024-04-25T10:04:16.237429+0000","last_became_active":"2024-04-25T10:04:14.913637+0000","last_became_peered":"2024-04-25T10:04:14.913637+0000","last_unstale":"2024-04-25T10:04:16.237429+0000","last_undegraded":"2024-04-25T10:04:16.237429+0000","last_fullsized":"2024-04-25T10:04:16.237429+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.026157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"165'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237342+0000","last_change":"2024-04-25T10:04:13.895684+0000","last_active":"2024-04-25T10:04:16.237342+0000","last_peered":"2024-04-25T10:04:16.237342+0000","last_clean":"2024-04-25T10:04:16.237342+0000","last_became_active":"2024-04-25T10:04:13.895390+0000","last_became_peered":"2024-04-25T10:04:13.895390+0000","last_unstale":"2024-04-25T10:04:16.237342+0000","last_undegraded":"2024-04-25T10:04:16.237342+0000","last_fullsized":"2024-04-25T10:04:16.237342+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:59.957477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892462+0000","last_change":"2024-04-25T10:04:14.897994+0000","last_active":"2024-04-25T10:04:15.892462+0000","last_peered":"2024-04-25T10:04:15.892462+0000","last_clean":"2024-04-25T10:04:15.892462+0000","last_became_active":"2024-04-25T10:04:14.897723+0000","last_became_peered":"2024-04-25T10:04:14.897723+0000","last_unstale":"2024-04-25T10:04:15.892462+0000","last_undegraded":"2024-04-25T10:04:15.892462+0000","last_fullsized":"2024-04-25T10:04:15.892462+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:38.890605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890263+0000","last_change":"2024-04-25T10:04:13.911061+0000","last_active":"2024-04-25T10:04:15.890263+0000","last_peered":"2024-04-25T10:04:15.890263+0000","last_clean":"2024-04-25T10:04:15.890263+0000","last_became_active":"2024-04-25T10:04:13.910591+0000","last_became_peered":"2024-04-25T10:04:13.910591+0000","last_unstale":"2024-04-25T10:04:15.890263+0000","last_undegraded":"2024-04-25T10:04:15.890263+0000","last_fullsized":"2024-04-25T10:04:15.890263+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:14.367579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237404+0000","last_change":"2024-04-25T10:04:14.908689+0000","last_active":"2024-04-25T10:04:16.237404+0000","last_peered":"2024-04-25T10:04:16.237404+0000","last_clean":"2024-04-25T10:04:16.237404+0000","last_became_active":"2024-04-25T10:04:14.908375+0000","last_became_peered":"2024-04-25T10:04:14.908375+0000","last_unstale":"2024-04-25T10:04:16.237404+0000","last_undegraded":"2024-04-25T10:04:16.237404+0000","last_fullsized":"2024-04-25T10:04:16.237404+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:00.191907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890212+0000","last_change":"2024-04-25T10:04:13.913137+0000","last_active":"2024-04-25T10:04:15.890212+0000","last_peered":"2024-04-25T10:04:15.890212+0000","last_clean":"2024-04-25T10:04:15.890212+0000","last_became_active":"2024-04-25T10:04:13.912835+0000","last_became_peered":"2024-04-25T10:04:13.912835+0000","last_unstale":"2024-04-25T10:04:15.890212+0000","last_undegraded":"2024-04-25T10:04:15.890212+0000","last_fullsized":"2024-04-25T10:04:15.890212+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:22.383618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237200+0000","last_change":"2024-04-25T10:04:14.904973+0000","last_active":"2024-04-25T10:04:16.237200+0000","last_peered":"2024-04-25T10:04:16.237200+0000","last_clean":"2024-04-25T10:04:16.237200+0000","last_became_active":"2024-04-25T10:04:14.904700+0000","last_became_peered":"2024-04-25T10:04:14.904700+0000","last_unstale":"2024-04-25T10:04:16.237200+0000","last_undegraded":"2024-04-25T10:04:16.237200+0000","last_fullsized":"2024-04-25T10:04:16.237200+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:37.029957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892236+0000","last_change":"2024-04-25T10:04:13.896282+0000","last_active":"2024-04-25T10:04:15.892236+0000","last_peered":"2024-04-25T10:04:15.892236+0000","last_clean":"2024-04-25T10:04:15.892236+0000","last_became_active":"2024-04-25T10:04:13.894939+0000","last_became_peered":"2024-04-25T10:04:13.894939+0000","last_unstale":"2024-04-25T10:04:15.892236+0000","last_undegraded":"2024-04-25T10:04:15.892236+0000","last_fullsized":"2024-04-25T10:04:15.892236+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:48.305366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.890983+0000","last_change":"2024-04-25T10:04:14.901523+0000","last_active":"2024-04-25T10:04:15.890983+0000","last_peered":"2024-04-25T10:04:15.890983+0000","last_clean":"2024-04-25T10:04:15.890983+0000","last_became_active":"2024-04-25T10:04:14.901256+0000","last_became_peered":"2024-04-25T10:04:14.901256+0000","last_unstale":"2024-04-25T10:04:15.890983+0000","last_undegraded":"2024-04-25T10:04:15.890983+0000","last_fullsized":"2024-04-25T10:04:15.890983+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:09.286870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892036+0000","last_change":"2024-04-25T10:04:13.910016+0000","last_active":"2024-04-25T10:04:15.892036+0000","last_peered":"2024-04-25T10:04:15.892036+0000","last_clean":"2024-04-25T10:04:15.892036+0000","last_became_active":"2024-04-25T10:04:13.909591+0000","last_became_peered":"2024-04-25T10:04:13.909591+0000","last_unstale":"2024-04-25T10:04:15.892036+0000","last_undegraded":"2024-04-25T10:04:15.892036+0000","last_fullsized":"2024-04-25T10:04:15.892036+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:21.559345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.238551+0000","last_change":"2024-04-25T10:04:14.922323+0000","last_active":"2024-04-25T10:04:16.238551+0000","last_peered":"2024-04-25T10:04:16.238551+0000","last_clean":"2024-04-25T10:04:16.238551+0000","last_became_active":"2024-04-25T10:04:14.922051+0000","last_became_peered":"2024-04-25T10:04:14.922051+0000","last_unstale":"2024-04-25T10:04:16.238551+0000","last_undegraded":"2024-04-25T10:04:16.238551+0000","last_fullsized":"2024-04-25T10:04:16.238551+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:00.158092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237892+0000","last_change":"2024-04-25T10:04:13.899980+0000","last_active":"2024-04-25T10:04:16.237892+0000","last_peered":"2024-04-25T10:04:16.237892+0000","last_clean":"2024-04-25T10:04:16.237892+0000","last_became_active":"2024-04-25T10:04:13.899671+0000","last_became_peered":"2024-04-25T10:04:13.899671+0000","last_unstale":"2024-04-25T10:04:16.237892+0000","last_undegraded":"2024-04-25T10:04:16.237892+0000","last_fullsized":"2024-04-25T10:04:16.237892+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:12.291155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892155+0000","last_change":"2024-04-25T10:04:14.906536+0000","last_active":"2024-04-25T10:04:15.892155+0000","last_peered":"2024-04-25T10:04:15.892155+0000","last_clean":"2024-04-25T10:04:15.892155+0000","last_became_active":"2024-04-25T10:04:14.906253+0000","last_became_peered":"2024-04-25T10:04:14.906253+0000","last_unstale":"2024-04-25T10:04:15.892155+0000","last_undegraded":"2024-04-25T10:04:15.892155+0000","last_fullsized":"2024-04-25T10:04:15.892155+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:23.385728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237327+0000","last_change":"2024-04-25T10:04:13.910423+0000","last_active":"2024-04-25T10:04:16.237327+0000","last_peered":"2024-04-25T10:04:16.237327+0000","last_clean":"2024-04-25T10:04:16.237327+0000","last_became_active":"2024-04-25T10:04:13.910113+0000","last_became_peered":"2024-04-25T10:04:13.910113+0000","last_unstale":"2024-04-25T10:04:16.237327+0000","last_undegraded":"2024-04-25T10:04:16.237327+0000","last_fullsized":"2024-04-25T10:04:16.237327+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:52.368366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237587+0000","last_change":"2024-04-25T10:04:14.917448+0000","last_active":"2024-04-25T10:04:16.237587+0000","last_peered":"2024-04-25T10:04:16.237587+0000","last_clean":"2024-04-25T10:04:16.237587+0000","last_became_active":"2024-04-25T10:04:14.917173+0000","last_became_peered":"2024-04-25T10:04:14.917173+0000","last_unstale":"2024-04-25T10:04:16.237587+0000","last_undegraded":"2024-04-25T10:04:16.237587+0000","last_fullsized":"2024-04-25T10:04:16.237587+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:13.879667+0000","last_clean_scrub_stamp":"2024-04-25T10:04:13.879667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:12.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:16.237474+0000","last_change":"2024-04-25T10:04:13.903829+0000","last_active":"2024-04-25T10:04:16.237474+0000","last_peered":"2024-04-25T10:04:16.237474+0000","last_clean":"2024-04-25T10:04:16.237474+0000","last_became_active":"2024-04-25T10:04:13.901705+0000","last_became_peered":"2024-04-25T10:04:13.901705+0000","last_unstale":"2024-04-25T10:04:16.237474+0000","last_undegraded":"2024-04-25T10:04:16.237474+0000","last_fullsized":"2024-04-25T10:04:16.237474+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:12.877557+0000","last_clean_scrub_stamp":"2024-04-25T10:04:12.877557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:19.830958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":465,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-04-25T10:04:15.892090+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:04:15.892090+0000","last_peered":"2024-04-25T10:04:15.892090+0000","last_clean":"2024-04-25T10:04:15.892090+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:04:15.892090+0000","last_undegraded":"2024-04-25T10:04:15.892090+0000","last_fullsized":"2024-04-25T10:04:15.892090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51844,"kb_used_data":2612,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319996,"statfs":{"total":96636764160,"available":96583675904,"internally_reserved":0,"allocated":2674688,"data_stored":1566346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51264,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320576,"statfs":{"total":96636764160,"available":96584269824,"internally_reserved":0,"allocated":2084864,"data_stored":976048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51276,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320564,"statfs":{"total":96636764160,"available":96584257536,"internally_reserved":0,"allocated":2097152,"data_stored":977612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59596,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312244,"statfs":{"total":96636764160,"available":96575737856,"internally_reserved":0,"allocated":2097152,"data_stored":977624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59596,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312244,"statfs":{"total":96636764160,"available":96575737856,"internally_reserved":0,"allocated":2088960,"data_stored":976138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55432,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316408,"statfs":{"total":96636764160,"available":96580001792,"internally_reserved":0,"allocated":2084864,"data_stored":976560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55424,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316416,"statfs":{"total":96636764160,"available":96580009984,"internally_reserved":0,"allocated":2080768,"data_stored":975978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56020,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315820,"statfs":{"total":96636764160,"available":96579399680,"internally_reserved":0,"allocated":2691072,"data_stored":1567074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:22.661 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:04:22.662 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:04:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.800+0000 7eff15a5d640 1 Processor -- start 2024-04-25T10:04:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.800+0000 7eff15a5d640 1 -- start start 2024-04-25T10:04:22.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.801+0000 7eff15a5d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff1015dc20 0x7eff100ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.801+0000 7eff15a5d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7eff100d32f0 0x7eff100cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.801+0000 7eff15a5d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1015e000 con 0x7eff10161710 2024-04-25T10:04:22.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.801+0000 7eff15a5d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff100d1de0 con 0x7eff100d32f0 2024-04-25T10:04:22.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.801+0000 7eff15a5d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff10161c60 con 0x7eff1015dc20 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.801+0000 7eff0e7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7eff10161710 0x7eff1006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57588/0 (socket says 172.21.15.19:57588) 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.801+0000 7eff0e7fc640 1 -- 172.21.15.19:0/1076896124 learned_addr learned my addr 172.21.15.19:0/1076896124 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.801+0000 7eff0f7fe640 1 --2- 172.21.15.19:0/1076896124 >> v2:172.21.15.112:3300/0 conn(0x7eff100d32f0 0x7eff100cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.801+0000 7eff0effd640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff1015dc20 0x7eff100ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.801+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253296076 0 0) 0x7eff1015e000 con 0x7eff10161710 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efef8003520 con 0x7eff10161710 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 666873721 0 0) 0x7efef8003520 con 0x7eff10161710 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff1015dc20 msgr2=0x7eff100ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff1015dc20 0x7eff100ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 >> v2:172.21.15.112:3300/0 conn(0x7eff100d32f0 msgr2=0x7eff100cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 --2- 172.21.15.19:0/1076896124 >> v2:172.21.15.112:3300/0 conn(0x7eff100d32f0 0x7eff100cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff10161f40 con 0x7eff10161710 2024-04-25T10:04:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7eff04001670 con 0x7eff10161710 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 >> v1:172.21.15.19:6789/0 conn(0x7eff10161710 legacy=0x7eff1006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff1015dc20 0x7efef80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef8003f30 0x7efef8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef8004d40 0x7efef80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff100d1de0 con 0x7efef8004d40 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff10161c60 con 0x7efef8003f30 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef800b9c0 con 0x7eff1015dc20 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0f7fe640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef8004d40 0x7efef80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0effd640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff1015dc20 0x7efef80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0e7fc640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef8003f30 0x7efef8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0f7fe640 1 -- 172.21.15.19:0/1076896124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff1015dc20 msgr2=0x7efef80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0f7fe640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff1015dc20 0x7efef80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0f7fe640 1 -- 172.21.15.19:0/1076896124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef8003f30 msgr2=0x7efef8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0f7fe640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef8003f30 0x7efef8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.802+0000 7eff0f7fe640 1 -- 172.21.15.19:0/1076896124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef8003520 con 0x7efef8004d40 2024-04-25T10:04:22.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.803+0000 7eff0f7fe640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef8004d40 0x7efef80051c0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7eff08009f90 tx=0x7efef8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235d00a45036c2fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.803+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08009c10 con 0x7efef8004d40 2024-04-25T10:04:22.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.803+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff08036070 con 0x7efef8004d40 2024-04-25T10:04:22.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.803+0000 7eff0dffb640 1 -- 172.21.15.19:0/1076896124 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08031630 con 0x7efef8004d40 2024-04-25T10:04:22.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.804+0000 7eff15a5d640 1 -- 172.21.15.19:0/1076896124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef8004d40 msgr2=0x7efef80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.804+0000 7eff15a5d640 1 --2- 172.21.15.19:0/1076896124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef8004d40 0x7efef80051c0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7eff08009f90 tx=0x7efef8005830 comp rx=0 tx=0).stop 2024-04-25T10:04:22.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.804+0000 7eff0f7fe640 1 -- 172.21.15.19:0/1076896124 reap_dead start 2024-04-25T10:04:22.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.804+0000 7eff15a5d640 1 -- 172.21.15.19:0/1076896124 shutdown_connections 2024-04-25T10:04:22.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.804+0000 7eff15a5d640 1 -- 172.21.15.19:0/1076896124 >> 172.21.15.19:0/1076896124 conn(0x7eff100d0420 msgr2=0x7eff100d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:22.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.804+0000 7eff15a5d640 1 -- 172.21.15.19:0/1076896124 shutdown_connections 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.804+0000 7eff15a5d640 1 -- 172.21.15.19:0/1076896124 wait complete. 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.805+0000 7eff15a5d640 1 Processor -- start 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.805+0000 7eff15a5d640 1 -- start start 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff15a5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff100cdb40 0x7eff10151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff15a5d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff100cdf20 0x7eff10151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff15a5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1015f500 0x7eff1015f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff15a5d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff100b5e50 con 0x7eff100cdb40 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff15a5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff10162500 con 0x7eff1015f500 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff15a5d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff100b5fd0 con 0x7eff100cdf20 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff100cdb40 0x7eff10151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff0effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff100cdb40 0x7eff10151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57036/0 (socket says 172.21.15.19:57036) 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff0effd640 1 -- 172.21.15.19:0/3079371110 learned_addr learned my addr 172.21.15.19:0/3079371110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff0e7fc640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff100cdf20 0x7eff10151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff0f7fe640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1015f500 0x7eff1015f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff0effd640 1 -- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff100cdf20 msgr2=0x7eff10151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff0effd640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff100cdf20 0x7eff10151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff0effd640 1 -- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1015f500 msgr2=0x7eff1015f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff0effd640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1015f500 0x7eff1015f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff0effd640 1 -- 172.21.15.19:0/3079371110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff10160230 con 0x7eff100cdb40 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7eff0effd640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff100cdb40 0x7eff10151980 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7eff000119a0 tx=0x7eff00011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.806+0000 7efef7fff640 1 -- 172.21.15.19:0/3079371110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0000faf0 con 0x7eff100cdb40 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.807+0000 7efef7fff640 1 -- 172.21.15.19:0/3079371110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0000fc90 con 0x7eff100cdb40 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.807+0000 7eff15a5d640 1 -- 172.21.15.19:0/3079371110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff100cb250 con 0x7eff100cdb40 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.807+0000 7efef7fff640 1 -- 172.21.15.19:0/3079371110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00014440 con 0x7eff100cdb40 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.807+0000 7eff15a5d640 1 -- 172.21.15.19:0/3079371110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff100cb810 con 0x7eff100cdb40 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.808+0000 7eff15a5d640 1 -- 172.21.15.19:0/3079371110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff10073a00 con 0x7eff100cdb40 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.808+0000 7efef7fff640 1 -- 172.21.15.19:0/3079371110 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff00012040 con 0x7eff100cdb40 2024-04-25T10:04:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.808+0000 7efef7fff640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efefc06b4d0 0x7efefc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.808+0000 7efef7fff640 1 -- 172.21.15.19:0/3079371110 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7eff0008aa10 con 0x7eff100cdb40 2024-04-25T10:04:22.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.809+0000 7eff0e7fc640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efefc06b4d0 0x7efefc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.809+0000 7eff0e7fc640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efefc06b4d0 0x7efefc06d990 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7eff04004840 tx=0x7eff04009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:22.813+0000 7efef7fff640 1 -- 172.21.15.19:0/3079371110 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff00019030 con 0x7eff100cdb40 2024-04-25T10:04:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.065+0000 7eff15a5d640 1 -- 172.21.15.19:0/3079371110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7eff100d16b0 con 0x7eff100cdb40 2024-04-25T10:04:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.068+0000 7efef7fff640 1 -- 172.21.15.19:0/3079371110 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7eff00059790 con 0x7eff100cdb40 2024-04-25T10:04:23.075 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T10:04:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T10:04:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T10:04:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T10:04:23.075 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:04:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T10:04:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 -- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efefc06b4d0 msgr2=0x7efefc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efefc06b4d0 0x7efefc06d990 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7eff04004840 tx=0x7eff04009040 comp rx=0 tx=0).stop 2024-04-25T10:04:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 -- 172.21.15.19:0/3079371110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff100cdb40 msgr2=0x7eff10151980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff100cdb40 0x7eff10151980 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7eff000119a0 tx=0x7eff00011e60 comp rx=0 tx=0).stop 2024-04-25T10:04:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 -- 172.21.15.19:0/3079371110 shutdown_connections 2024-04-25T10:04:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efefc06b4d0 0x7efefc06d990 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1015f500 0x7eff1015f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7eff100cdf20 0x7eff10151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 --2- 172.21.15.19:0/3079371110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff100cdb40 0x7eff10151980 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 -- 172.21.15.19:0/3079371110 >> 172.21.15.19:0/3079371110 conn(0x7eff100d0420 msgr2=0x7eff100d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 -- 172.21.15.19:0/3079371110 shutdown_connections 2024-04-25T10:04:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.070+0000 7eff15a5d640 1 -- 172.21.15.19:0/3079371110 wait complete. 2024-04-25T10:04:23.087 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:23.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.182+0000 7f857c9f9640 1 Processor -- start 2024-04-25T10:04:23.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.182+0000 7f857c9f9640 1 -- start start 2024-04-25T10:04:23.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.183+0000 7f857c9f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85780d2e90 0x7f85780d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.183+0000 7f857c9f9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f85780d0900 0x7f85780b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.183+0000 7f857c9f9640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85780b9940 con 0x7f857815dd30 2024-04-25T10:04:23.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.183+0000 7f857c9f9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85780d0d70 con 0x7f85780d0900 2024-04-25T10:04:23.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.183+0000 7f857c9f9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85780d1df0 con 0x7f85780d2e90 2024-04-25T10:04:23.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.183+0000 7f857659c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f857815dd30 0x7f85780b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57590/0 (socket says 172.21.15.19:57590) 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.183+0000 7f857659c640 1 -- 172.21.15.19:0/3463857048 learned_addr learned my addr 172.21.15.19:0/3463857048 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273638951 0 0) 0x7f85780b9940 con 0x7f857815dd30 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8564003520 con 0x7f857815dd30 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f8576d9d640 1 --2- 172.21.15.19:0/3463857048 >> v2:172.21.15.112:3300/0 conn(0x7f85780d0900 0x7f85780b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f8575d9b640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85780d2e90 0x7f85780d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3243907328 0 0) 0x7f8564003520 con 0x7f857815dd30 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85780d2e90 msgr2=0x7f85780d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85780d2e90 0x7f85780d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 >> v2:172.21.15.112:3300/0 conn(0x7f85780d0900 msgr2=0x7f85780b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 --2- 172.21.15.19:0/3463857048 >> v2:172.21.15.112:3300/0 conn(0x7f85780d0900 0x7f85780b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85780d20d0 con 0x7f857815dd30 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8570001670 con 0x7f857815dd30 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 >> v1:172.21.15.19:6789/0 conn(0x7f857815dd30 legacy=0x7f85780b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85780d2e90 0x7f85640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8564003f30 0x7f8564004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564004d40 0x7f85640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85780d0d70 con 0x7f8564003f30 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85780d1df0 con 0x7f8564004d40 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856400ba10 con 0x7f85780d2e90 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f8575d9b640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8564003f30 0x7f8564004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f857659c640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85780d2e90 0x7f85640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.184+0000 7f8576d9d640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564004d40 0x7f85640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.185+0000 7f8575d9b640 1 -- 172.21.15.19:0/3463857048 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85780d2e90 msgr2=0x7f85640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.185+0000 7f8575d9b640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85780d2e90 0x7f85640039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.185+0000 7f8575d9b640 1 -- 172.21.15.19:0/3463857048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564004d40 msgr2=0x7f85640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.185+0000 7f8575d9b640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564004d40 0x7f85640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.185+0000 7f8575d9b640 1 -- 172.21.15.19:0/3463857048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8564003520 con 0x7f8564003f30 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.185+0000 7f8575d9b640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8564003f30 0x7f8564004800 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f8560009ae0 tx=0x7f8564005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916f538881ef5fae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.185+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560036070 con 0x7f8564003f30 2024-04-25T10:04:23.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.185+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8560005990 con 0x7f8564003f30 2024-04-25T10:04:23.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.185+0000 7f857559a640 1 -- 172.21.15.19:0/3463857048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560009680 con 0x7f8564003f30 2024-04-25T10:04:23.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.186+0000 7f857c9f9640 1 -- 172.21.15.19:0/3463857048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8564003f30 msgr2=0x7f8564004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.186+0000 7f857c9f9640 1 --2- 172.21.15.19:0/3463857048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8564003f30 0x7f8564004800 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f8560009ae0 tx=0x7f8564005830 comp rx=0 tx=0).stop 2024-04-25T10:04:23.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.186+0000 7f8576d9d640 1 -- 172.21.15.19:0/3463857048 reap_dead start 2024-04-25T10:04:23.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.186+0000 7f857c9f9640 1 -- 172.21.15.19:0/3463857048 shutdown_connections 2024-04-25T10:04:23.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.186+0000 7f857c9f9640 1 -- 172.21.15.19:0/3463857048 >> 172.21.15.19:0/3463857048 conn(0x7f8578061c30 msgr2=0x7f857806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.186+0000 7f857c9f9640 1 -- 172.21.15.19:0/3463857048 shutdown_connections 2024-04-25T10:04:23.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.186+0000 7f857c9f9640 1 -- 172.21.15.19:0/3463857048 wait complete. 2024-04-25T10:04:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.187+0000 7f857c9f9640 1 Processor -- start 2024-04-25T10:04:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.187+0000 7f857c9f9640 1 -- start start 2024-04-25T10:04:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.188+0000 7f857c9f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564003f30 0x7f85780b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.188+0000 7f857c9f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85781f4cd0 0x7f8578158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.188+0000 7f857c9f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f857814d950 0x7f857814de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.188+0000 7f857c9f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85780d2970 con 0x7f85781f4cd0 2024-04-25T10:04:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.188+0000 7f857c9f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85780d2690 con 0x7f8564003f30 2024-04-25T10:04:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.188+0000 7f857c9f9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85780d1a10 con 0x7f857814d950 2024-04-25T10:04:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.188+0000 7f8575d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85781f4cd0 0x7f8578158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.188+0000 7f8575d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85781f4cd0 0x7f8578158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57048/0 (socket says 172.21.15.19:57048) 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.188+0000 7f8575d9b640 1 -- 172.21.15.19:0/3799232272 learned_addr learned my addr 172.21.15.19:0/3799232272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.188+0000 7f857659c640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564003f30 0x7f85780b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.188+0000 7f8576d9d640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f857814d950 0x7f857814de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.189+0000 7f8575d9b640 1 -- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f857814d950 msgr2=0x7f857814de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.189+0000 7f8575d9b640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f857814d950 0x7f857814de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.189+0000 7f8575d9b640 1 -- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564003f30 msgr2=0x7f85780b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.189+0000 7f8575d9b640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564003f30 0x7f85780b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.189+0000 7f8575d9b640 1 -- 172.21.15.19:0/3799232272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8578156e80 con 0x7f85781f4cd0 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.189+0000 7f8575d9b640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85781f4cd0 0x7f8578158c10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f8560034040 tx=0x7f8560004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.189+0000 7f856effd640 1 -- 172.21.15.19:0/3799232272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560040020 con 0x7f85781f4cd0 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.189+0000 7f856effd640 1 -- 172.21.15.19:0/3799232272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8560037040 con 0x7f85781f4cd0 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.189+0000 7f856effd640 1 -- 172.21.15.19:0/3799232272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560036030 con 0x7f85781f4cd0 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.189+0000 7f857c9f9640 1 -- 172.21.15.19:0/3799232272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8578157160 con 0x7f85781f4cd0 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.189+0000 7f857c9f9640 1 -- 172.21.15.19:0/3799232272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8578157720 con 0x7f85781f4cd0 2024-04-25T10:04:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.190+0000 7f857c9f9640 1 -- 172.21.15.19:0/3799232272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8578073990 con 0x7f85781f4cd0 2024-04-25T10:04:23.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.190+0000 7f856effd640 1 -- 172.21.15.19:0/3799232272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8560043020 con 0x7f85781f4cd0 2024-04-25T10:04:23.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.191+0000 7f856effd640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f855806b4d0 0x7f855806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.191+0000 7f856effd640 1 -- 172.21.15.19:0/3799232272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85600aa3f0 con 0x7f85781f4cd0 2024-04-25T10:04:23.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.191+0000 7f857659c640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f855806b4d0 0x7f855806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.192+0000 7f857659c640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f855806b4d0 0x7f855806d990 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8570001050 tx=0x7f8570007970 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.195+0000 7f856effd640 1 -- 172.21.15.19:0/3799232272 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f856004a030 con 0x7f85781f4cd0 2024-04-25T10:04:23.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.412+0000 7f857c9f9640 1 -- 172.21.15.19:0/3799232272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f857815c370 con 0x7f85781f4cd0 2024-04-25T10:04:23.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.413+0000 7f856effd640 1 -- 172.21.15.19:0/3799232272 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4954 (secure 0 0 0) 0x7f8560078a90 con 0x7f85781f4cd0 2024-04-25T10:04:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:23.419 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1300145056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1300145056},{"type":"v1","addr":"172.21.15.19:6835","nonce":1300145056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":8027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2001687418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2001687418},{"type":"v1","addr":"172.21.15.112:6836","nonce":2001687418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721457153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721457153},{"type":"v1","addr":"172.21.15.112:6837","nonce":721457153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:14.252458+0000","modified":"2024-04-25T10:04:16.259643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10698":{"gid":10698,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3942829201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3942829201},{"type":"v1","addr":"172.21.15.19:6834","nonce":3942829201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:04:23.419 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 109 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 -- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f855806b4d0 msgr2=0x7f855806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f855806b4d0 0x7f855806d990 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8570001050 tx=0x7f8570007970 comp rx=0 tx=0).stop 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 -- 172.21.15.19:0/3799232272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85781f4cd0 msgr2=0x7f8578158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85781f4cd0 0x7f8578158c10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f8560034040 tx=0x7f8560004730 comp rx=0 tx=0).stop 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 -- 172.21.15.19:0/3799232272 shutdown_connections 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f855806b4d0 0x7f855806d990 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f857814d950 0x7f857814de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85781f4cd0 0x7f8578158c10 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 --2- 172.21.15.19:0/3799232272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564003f30 0x7f85780b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 -- 172.21.15.19:0/3799232272 >> 172.21.15.19:0/3799232272 conn(0x7f8578061c30 msgr2=0x7f857815e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 -- 172.21.15.19:0/3799232272 shutdown_connections 2024-04-25T10:04:23.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.415+0000 7f857c9f9640 1 -- 172.21.15.19:0/3799232272 wait complete. 2024-04-25T10:04:23.836 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:04:14.252458+0000', 'modified': '2024-04-25T10:04:16.259643+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10698}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10698': {'gid': 10698, 'name': 'c', 'rank': 0, 'incarnation': 107, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6834/3942829201', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 3942829201}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 3942829201}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:04:23.836 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:04:23.836 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:04:23.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.935+0000 7fcc6da37640 1 Processor -- start 2024-04-25T10:04:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.935+0000 7fcc6da37640 1 -- start start 2024-04-25T10:04:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.936+0000 7fcc6da37640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcc6815bb50 0x7fcc680c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.936+0000 7fcc6da37640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcc680c70d0 0x7fcc680cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.936+0000 7fcc6da37640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680c75e0 con 0x7fcc680cfb40 2024-04-25T10:04:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.936+0000 7fcc6da37640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc680c8680 con 0x7fcc680c70d0 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.936+0000 7fcc6da37640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc680c78c0 con 0x7fcc6815bb50 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.936+0000 7fcc667fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcc680c70d0 0x7fcc680cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.936+0000 7fcc677fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcc680cfb40 0x7fcc68070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57596/0 (socket says 172.21.15.19:57596) 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.936+0000 7fcc677fe640 1 -- 172.21.15.19:0/2021607369 learned_addr learned my addr 172.21.15.19:0/2021607369 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.936+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/2021607369 >> v2:172.21.15.112:3300/0 conn(0x7fcc6815bb50 0x7fcc680c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.937+0000 7fcc667fc640 1 -- 172.21.15.19:0/2021607369 >> v2:172.21.15.112:3300/0 conn(0x7fcc6815bb50 msgr2=0x7fcc680c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.937+0000 7fcc667fc640 1 --2- 172.21.15.19:0/2021607369 >> v2:172.21.15.112:3300/0 conn(0x7fcc6815bb50 0x7fcc680c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.937+0000 7fcc667fc640 1 -- 172.21.15.19:0/2021607369 >> v1:172.21.15.19:6789/0 conn(0x7fcc680cfb40 legacy=0x7fcc68070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.937+0000 7fcc667fc640 1 -- 172.21.15.19:0/2021607369 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680c7ba0 con 0x7fcc680c70d0 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.937+0000 7fcc65ffb640 1 -- 172.21.15.19:0/2021607369 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983821080 0 0) 0x7fcc680c75e0 con 0x7fcc680cfb40 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.937+0000 7fcc667fc640 1 --2- 172.21.15.19:0/2021607369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc680c70d0 0x7fcc680cf600 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fcc68070ab0 tx=0x7fcc5002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4acbb2efdb9deb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.937+0000 7fcc65ffb640 1 -- 172.21.15.19:0/2021607369 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50037070 con 0x7fcc680c70d0 2024-04-25T10:04:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.938+0000 7fcc65ffb640 1 -- 172.21.15.19:0/2021607369 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc50032440 con 0x7fcc680c70d0 2024-04-25T10:04:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.938+0000 7fcc65ffb640 1 -- 172.21.15.19:0/2021607369 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50032810 con 0x7fcc680c70d0 2024-04-25T10:04:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.938+0000 7fcc6da37640 1 -- 172.21.15.19:0/2021607369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc680c70d0 msgr2=0x7fcc680cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.938+0000 7fcc6da37640 1 --2- 172.21.15.19:0/2021607369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc680c70d0 0x7fcc680cf600 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fcc68070ab0 tx=0x7fcc5002a250 comp rx=0 tx=0).stop 2024-04-25T10:04:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.938+0000 7fcc6da37640 1 -- 172.21.15.19:0/2021607369 shutdown_connections 2024-04-25T10:04:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.938+0000 7fcc6da37640 1 --2- 172.21.15.19:0/2021607369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc680c70d0 0x7fcc680cf600 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.938+0000 7fcc6da37640 1 --2- 172.21.15.19:0/2021607369 >> v2:172.21.15.112:3300/0 conn(0x7fcc6815bb50 0x7fcc680c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.938+0000 7fcc6da37640 1 -- 172.21.15.19:0/2021607369 >> 172.21.15.19:0/2021607369 conn(0x7fcc6806f500 msgr2=0x7fcc6806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.938+0000 7fcc6da37640 1 -- 172.21.15.19:0/2021607369 shutdown_connections 2024-04-25T10:04:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.938+0000 7fcc6da37640 1 -- 172.21.15.19:0/2021607369 wait complete. 2024-04-25T10:04:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.939+0000 7fcc6da37640 1 Processor -- start 2024-04-25T10:04:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.939+0000 7fcc6da37640 1 -- start start 2024-04-25T10:04:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc6da37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc680c70d0 0x7fcc68154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc6da37640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc680cfb40 0x7fcc680c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc6da37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6815bb50 0x7fcc680c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc6da37640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6815f6d0 con 0x7fcc680c70d0 2024-04-25T10:04:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc6da37640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680c8160 con 0x7fcc6815bb50 2024-04-25T10:04:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc6da37640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6815f850 con 0x7fcc680cfb40 2024-04-25T10:04:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc680c70d0 0x7fcc68154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc680c70d0 0x7fcc68154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57064/0 (socket says 172.21.15.19:57064) 2024-04-25T10:04:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc66ffd640 1 -- 172.21.15.19:0/1823310701 learned_addr learned my addr 172.21.15.19:0/1823310701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc677fe640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6815bb50 0x7fcc680c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc667fc640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc680cfb40 0x7fcc680c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc66ffd640 1 -- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc680cfb40 msgr2=0x7fcc680c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc680cfb40 0x7fcc680c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc66ffd640 1 -- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6815bb50 msgr2=0x7fcc680c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6815bb50 0x7fcc680c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc66ffd640 1 -- 172.21.15.19:0/1823310701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680ca3e0 con 0x7fcc680c70d0 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.940+0000 7fcc667fc640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc680cfb40 0x7fcc680c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.941+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc680c70d0 0x7fcc68154d00 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000e870 tx=0x7fcc6000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.941+0000 7fcc5ffff640 1 -- 172.21.15.19:0/1823310701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60017070 con 0x7fcc680c70d0 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.941+0000 7fcc5ffff640 1 -- 172.21.15.19:0/1823310701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6000c940 con 0x7fcc680c70d0 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.941+0000 7fcc6da37640 1 -- 172.21.15.19:0/1823310701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc680ca6c0 con 0x7fcc680c70d0 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.941+0000 7fcc5ffff640 1 -- 172.21.15.19:0/1823310701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000cd30 con 0x7fcc680c70d0 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.941+0000 7fcc6da37640 1 -- 172.21.15.19:0/1823310701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc680cac80 con 0x7fcc680c70d0 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.942+0000 7fcc5ffff640 1 -- 172.21.15.19:0/1823310701 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc6001a070 con 0x7fcc680c70d0 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.943+0000 7fcc5ffff640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcc4806b4d0 0x7fcc4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.943+0000 7fcc5ffff640 1 -- 172.21.15.19:0/1823310701 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc6008a620 con 0x7fcc680c70d0 2024-04-25T10:04:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.943+0000 7fcc667fc640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcc4806b4d0 0x7fcc4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.944+0000 7fcc667fc640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcc4806b4d0 0x7fcc4806d990 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcc50034000 tx=0x7fcc5002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.944+0000 7fcc7080b740 1 Processor -- start 2024-04-25T10:04:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.944+0000 7fcc7080b740 1 -- start start 2024-04-25T10:04:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c04b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c068ad0 0x56261c069020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c020f10 0x56261c067190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc7080b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261bf83a10 con 0x56261c04b230 2024-04-25T10:04:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc7080b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261bf2cb30 con 0x56261c068ad0 2024-04-25T10:04:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc7080b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56261bf7fc50 con 0x56261c020f10 2024-04-25T10:04:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c04b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c04b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57068/0 (socket says 172.21.15.19:57068) 2024-04-25T10:04:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc66ffd640 1 -- 172.21.15.19:0/1983367232 learned_addr learned my addr 172.21.15.19:0/1983367232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc677fe640 1 --2- 172.21.15.19:0/1983367232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c020f10 0x56261c067190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc667fc640 1 --2- 172.21.15.19:0/1983367232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c068ad0 0x56261c069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc66ffd640 1 -- 172.21.15.19:0/1983367232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c020f10 msgr2=0x56261c067190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/1983367232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c020f10 0x56261c067190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc66ffd640 1 -- 172.21.15.19:0/1983367232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c068ad0 msgr2=0x56261c069020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/1983367232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c068ad0 0x56261c069020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc66ffd640 1 -- 172.21.15.19:0/1983367232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56261bf41e10 con 0x56261c04b230 2024-04-25T10:04:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.945+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/1983367232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c04b610 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcc60058900 tx=0x7fcc60058800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c1f086f425f889 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc5d7fa640 1 -- 172.21.15.19:0/1983367232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60021070 con 0x56261c04b230 2024-04-25T10:04:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc5d7fa640 1 -- 172.21.15.19:0/1983367232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6005cd50 con 0x56261c04b230 2024-04-25T10:04:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc5d7fa640 1 -- 172.21.15.19:0/1983367232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60061310 con 0x56261c04b230 2024-04-25T10:04:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc7080b740 1 -- 172.21.15.19:0/1983367232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 msgr2=0x56261c04b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc7080b740 1 --2- 172.21.15.19:0/1983367232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c04b610 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcc60058900 tx=0x7fcc60058800 comp rx=0 tx=0).stop 2024-04-25T10:04:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc7080b740 1 -- 172.21.15.19:0/1983367232 shutdown_connections 2024-04-25T10:04:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc7080b740 1 --2- 172.21.15.19:0/1983367232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c020f10 0x56261c067190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc7080b740 1 --2- 172.21.15.19:0/1983367232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c068ad0 0x56261c069020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc7080b740 1 --2- 172.21.15.19:0/1983367232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c04b610 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc7080b740 1 -- 172.21.15.19:0/1983367232 >> 172.21.15.19:0/1983367232 conn(0x56261c01d840 msgr2=0x56261c066040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc7080b740 1 -- 172.21.15.19:0/1983367232 shutdown_connections 2024-04-25T10:04:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.946+0000 7fcc7080b740 1 -- 172.21.15.19:0/1983367232 wait complete. 2024-04-25T10:04:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.947+0000 7fcc7080b740 1 -- start start 2024-04-25T10:04:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.947+0000 7fcc7080b740 1 Processor -- start 2024-04-25T10:04:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c04b230 0x56261c038af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c068ad0 0x56261c0794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04e190 0x56261c0356b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc7080b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261c021cb0 con 0x56261c04e190 2024-04-25T10:04:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc7080b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261bf400c0 con 0x56261c04b230 2024-04-25T10:04:23.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc7080b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56261c01b770 con 0x56261c068ad0 2024-04-25T10:04:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc7080b740 10 client.0 _ll_register_callbacks cb 0x56261bf33ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04e190 0x56261c0356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04e190 0x56261c0356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57074/0 (socket says 172.21.15.19:57074) 2024-04-25T10:04:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc677fe640 1 -- 172.21.15.19:0/790615871 learned_addr learned my addr 172.21.15.19:0/790615871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c04b230 0x56261c038af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc667fc640 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c068ad0 0x56261c0794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc677fe640 1 -- 172.21.15.19:0/790615871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c068ad0 msgr2=0x56261c0794e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc677fe640 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c068ad0 0x56261c0794e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc677fe640 1 -- 172.21.15.19:0/790615871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c04b230 msgr2=0x56261c038af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc677fe640 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c04b230 0x56261c038af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc677fe640 1 -- 172.21.15.19:0/790615871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56261bc982d0 con 0x56261c04e190 2024-04-25T10:04:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc677fe640 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04e190 0x56261c0356b0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcc58007b30 tx=0x7fcc58001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12198bc2cde78396 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.948+0000 7fcc467fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:04:23.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc467fc640 1 -- 172.21.15.19:0/790615871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5801a060 con 0x56261c04e190 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc467fc640 1 -- 172.21.15.19:0/790615871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc58004d60 con 0x56261c04e190 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc7080b740 1 -- 172.21.15.19:0/790615871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc6005cd50 con 0x56261c04e190 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc467fc640 1 -- 172.21.15.19:0/790615871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc5800d070 con 0x56261c04e190 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc7080b740 10 client.10860 fetch_fsmap learned FSMap version 109 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc7080b740 1 -- 172.21.15.19:0/790615871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x56261bd626b0 con 0x56261c04e190 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc467fc640 1 -- 172.21.15.19:0/790615871 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6005cd50 con 0x56261c04e190 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc467fc640 1 -- 172.21.15.19:0/790615871 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 109) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcc58013030 con 0x56261c04e190 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc7080b740 10 client.10860 fetch_fsmap finished waiting for FSMap version 109 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc7080b740 10 client.10860 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc7080b740 20 client.10860 populate_metadata read hostname 'smithi019' 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc7080b740 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x56261c04d4b0 0x56261c075f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc7080b740 4 client.10860 mds_command: new command op to 10698 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc7080b740 1 -- 172.21.15.19:0/790615871 --> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56261bfe2920 con 0x56261c04d4b0 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.949+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x56261c04d4b0 0x56261c075f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x56261c04d4b0 0x56261c075f10 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54aa605da8c7b02 server_cookie=2a823f60f015a3f1 in_seq=0 out_seq=0 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc467fc640 10 client.10860 ms_handle_connect on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc467fc640 1 -- 172.21.15.19:0/790615871 <== mds.0 v2:172.21.15.19:6832/3942829201 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56261bfe2920 con 0x56261c04d4b0 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc467fc640 10 client.10860 handle_command_reply: tid=0 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc7080b740 1 client.10860 shutdown 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc7080b740 1 -- 172.21.15.19:0/790615871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04e190 msgr2=0x56261c0356b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc7080b740 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04e190 0x56261c0356b0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcc58007b30 tx=0x7fcc58001ac0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc7080b740 1 -- 172.21.15.19:0/790615871 shutdown_connections 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc7080b740 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x56261c04d4b0 0x56261c075f10 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc7080b740 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04e190 0x56261c0356b0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc7080b740 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c068ad0 0x56261c0794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc7080b740 1 --2- 172.21.15.19:0/790615871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c04b230 0x56261c038af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc467fc640 0 client.10860 ms_handle_reset on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:23.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc7080b740 1 -- 172.21.15.19:0/790615871 >> 172.21.15.19:0/790615871 conn(0x56261c020f10 msgr2=0x56261c069e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc7080b740 1 -- 172.21.15.19:0/790615871 shutdown_connections 2024-04-25T10:04:23.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.950+0000 7fcc7080b740 1 -- 172.21.15.19:0/790615871 wait complete. 2024-04-25T10:04:23.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.951+0000 7fcc7080b740 20 client.10860 trim_cache size 0 max 16384 2024-04-25T10:04:23.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.966+0000 7fcc7080b740 1 Processor -- start 2024-04-25T10:04:23.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.966+0000 7fcc7080b740 1 -- start start 2024-04-25T10:04:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c04b230 0x56261c0355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c04e190 0x56261c0214e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c068ad0 0x56261c01ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc7080b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc681ff5b0 con 0x56261c068ad0 2024-04-25T10:04:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc7080b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6815dae0 con 0x56261c04b230 2024-04-25T10:04:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc7080b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc681ff080 con 0x56261c04e190 2024-04-25T10:04:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c068ad0 0x56261c01ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc677fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c068ad0 0x56261c01ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57076/0 (socket says 172.21.15.19:57076) 2024-04-25T10:04:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc677fe640 1 -- 172.21.15.19:0/2922698007 learned_addr learned my addr 172.21.15.19:0/2922698007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc677fe640 1 -- 172.21.15.19:0/2922698007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c04e190 msgr2=0x56261c0214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc677fe640 1 --2- 172.21.15.19:0/2922698007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c04e190 0x56261c0214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc677fe640 1 -- 172.21.15.19:0/2922698007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c04b230 msgr2=0x56261c0355c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc677fe640 1 --2- 172.21.15.19:0/2922698007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c04b230 0x56261c0355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.967+0000 7fcc677fe640 1 -- 172.21.15.19:0/2922698007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56261c04dfc0 con 0x56261c068ad0 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc677fe640 1 --2- 172.21.15.19:0/2922698007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c068ad0 0x56261c01ede0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fcc58007470 tx=0x7fcc58012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1853a6a3d6ac9b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc5cff9640 1 -- 172.21.15.19:0/2922698007 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58010040 con 0x56261c068ad0 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc5cff9640 1 -- 172.21.15.19:0/2922698007 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc58004b60 con 0x56261c068ad0 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc5cff9640 1 -- 172.21.15.19:0/2922698007 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc580051f0 con 0x56261c068ad0 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc7080b740 1 -- 172.21.15.19:0/2922698007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c068ad0 msgr2=0x56261c01ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc7080b740 1 --2- 172.21.15.19:0/2922698007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c068ad0 0x56261c01ede0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fcc58007470 tx=0x7fcc58012000 comp rx=0 tx=0).stop 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc7080b740 1 -- 172.21.15.19:0/2922698007 shutdown_connections 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc7080b740 1 --2- 172.21.15.19:0/2922698007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c068ad0 0x56261c01ede0 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc7080b740 1 --2- 172.21.15.19:0/2922698007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c04e190 0x56261c0214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc7080b740 1 --2- 172.21.15.19:0/2922698007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c04b230 0x56261c0355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc7080b740 1 -- 172.21.15.19:0/2922698007 >> 172.21.15.19:0/2922698007 conn(0x56261c020f10 msgr2=0x56261c022250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc7080b740 1 -- 172.21.15.19:0/2922698007 shutdown_connections 2024-04-25T10:04:23.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.968+0000 7fcc7080b740 1 -- 172.21.15.19:0/2922698007 wait complete. 2024-04-25T10:04:23.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.969+0000 7fcc7080b740 1 -- start start 2024-04-25T10:04:23.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.969+0000 7fcc7080b740 1 Processor -- start 2024-04-25T10:04:23.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c078780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c04e190 0x56261c04bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc7080b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c068ad0 0x56261c04c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc7080b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261c07a2f0 con 0x56261c04b230 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc7080b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261c05d0f0 con 0x56261c068ad0 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc7080b740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56261c05dd80 con 0x56261c04e190 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc7080b740 10 client.0 _ll_register_callbacks cb 0x56261c01bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c078780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc66ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c078780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57090/0 (socket says 172.21.15.19:57090) 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc66ffd640 1 -- 172.21.15.19:0/3751236978 learned_addr learned my addr 172.21.15.19:0/3751236978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc677fe640 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c068ad0 0x56261c04c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc667fc640 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c04e190 0x56261c04bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc66ffd640 1 -- 172.21.15.19:0/3751236978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c04e190 msgr2=0x56261c04bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c04e190 0x56261c04bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc66ffd640 1 -- 172.21.15.19:0/3751236978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c068ad0 msgr2=0x56261c04c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c068ad0 0x56261c04c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc66ffd640 1 -- 172.21.15.19:0/3751236978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56261c05c950 con 0x56261c04b230 2024-04-25T10:04:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc677fe640 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c068ad0 0x56261c04c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc66ffd640 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c078780 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fcc60058900 tx=0x7fcc60000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aacd6f767f8901f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc467fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.970+0000 7fcc467fc640 1 -- 172.21.15.19:0/3751236978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60021040 con 0x56261c04b230 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc467fc640 1 -- 172.21.15.19:0/3751236978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc60055270 con 0x56261c04b230 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc7080b740 1 -- 172.21.15.19:0/3751236978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc58004b60 con 0x56261c04b230 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc467fc640 1 -- 172.21.15.19:0/3751236978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc6005cd50 con 0x56261c04b230 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc7080b740 10 client.10866 fetch_fsmap learned FSMap version 109 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc7080b740 1 -- 172.21.15.19:0/3751236978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x56261c05c640 con 0x56261c04b230 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc467fc640 1 -- 172.21.15.19:0/3751236978 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc58004b60 con 0x56261c04b230 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc467fc640 1 -- 172.21.15.19:0/3751236978 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 109) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcc6006cbc0 con 0x56261c04b230 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc7080b740 10 client.10866 fetch_fsmap finished waiting for FSMap version 109 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc7080b740 10 client.10866 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:04:23.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc7080b740 20 client.10866 populate_metadata read hostname 'smithi019' 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc7080b740 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x56261c0463a0 0x56261c046780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc7080b740 4 client.10866 mds_command: new command op to 10698 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc7080b740 1 -- 172.21.15.19:0/3751236978 --> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56261bf90f50 con 0x56261c0463a0 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc667fc640 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x56261c0463a0 0x56261c046780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc467fc640 10 client.10866 ms_handle_connect on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.971+0000 7fcc667fc640 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x56261c0463a0 0x56261c046780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599c4858e033d73e server_cookie=fa2a4131ef974132 in_seq=0 out_seq=0 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc467fc640 1 -- 172.21.15.19:0/3751236978 <== mds.0 v2:172.21.15.19:6832/3942829201 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x56261bf90f50 con 0x56261c0463a0 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc467fc640 10 client.10866 handle_command_reply: tid=0 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 1 client.10866 shutdown 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 1 -- 172.21.15.19:0/3751236978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 msgr2=0x56261c078780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c078780 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fcc60058900 tx=0x7fcc60000bc0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 1 -- 172.21.15.19:0/3751236978 shutdown_connections 2024-04-25T10:04:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x56261c0463a0 0x56261c046780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56261c068ad0 0x56261c04c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56261c04e190 0x56261c04bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 1 --2- 172.21.15.19:0/3751236978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56261c04b230 0x56261c078780 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc467fc640 0 client.10866 ms_handle_reset on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 1 -- 172.21.15.19:0/3751236978 >> 172.21.15.19:0/3751236978 conn(0x56261c020f10 msgr2=0x56261c022230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 1 -- 172.21.15.19:0/3751236978 shutdown_connections 2024-04-25T10:04:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 1 -- 172.21.15.19:0/3751236978 wait complete. 2024-04-25T10:04:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc7080b740 20 client.10866 trim_cache size 0 max 16384 2024-04-25T10:04:23.984 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:04:23.984 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T10:04:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:04:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 10698, 2024-04-25T10:04:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:04:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:04:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:04:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 9.0864855789999996, 2024-04-25T10:04:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 109, 2024-04-25T10:04:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 165, 2024-04-25T10:04:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 165, 2024-04-25T10:04:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 14.122819056999999 2024-04-25T10:04:23.985 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:04:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc6da37640 1 -- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcc4806b4d0 msgr2=0x7fcc4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc6da37640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcc4806b4d0 0x7fcc4806d990 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcc50034000 tx=0x7fcc5002e000 comp rx=0 tx=0).stop 2024-04-25T10:04:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc6da37640 1 -- 172.21.15.19:0/1823310701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc680c70d0 msgr2=0x7fcc68154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.972+0000 7fcc6da37640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc680c70d0 0x7fcc68154d00 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000e870 tx=0x7fcc6000ed30 comp rx=0 tx=0).stop 2024-04-25T10:04:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.973+0000 7fcc6da37640 1 -- 172.21.15.19:0/1823310701 shutdown_connections 2024-04-25T10:04:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.973+0000 7fcc6da37640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcc4806b4d0 0x7fcc4806d990 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.973+0000 7fcc6da37640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc6815bb50 0x7fcc680c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.973+0000 7fcc6da37640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc680cfb40 0x7fcc680c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.973+0000 7fcc6da37640 1 --2- 172.21.15.19:0/1823310701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc680c70d0 0x7fcc68154d00 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.973+0000 7fcc6da37640 1 -- 172.21.15.19:0/1823310701 >> 172.21.15.19:0/1823310701 conn(0x7fcc6806f500 msgr2=0x7fcc6806f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.973+0000 7fcc6da37640 1 -- 172.21.15.19:0/1823310701 shutdown_connections 2024-04-25T10:04:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:23.973+0000 7fcc6da37640 1 -- 172.21.15.19:0/1823310701 wait complete. 2024-04-25T10:04:23.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:04:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.084+0000 7f135b934640 1 Processor -- start 2024-04-25T10:04:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.085+0000 7f135b934640 1 -- start start 2024-04-25T10:04:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.085+0000 7f135b934640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13540b2cb0 0x7f13540d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.085+0000 7f135b934640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13540b97a0 con 0x7f13540d1020 2024-04-25T10:04:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.085+0000 7f135b934640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13540d1c50 con 0x7f135415dbe0 2024-04-25T10:04:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.085+0000 7f135b934640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135415dfc0 con 0x7f13540b2cb0 2024-04-25T10:04:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f1359ed1640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f13540d1020 0x7f13540d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57606/0 (socket says 172.21.15.19:57606) 2024-04-25T10:04:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f1359ed1640 1 -- 172.21.15.19:0/65777808 learned_addr learned my addr 172.21.15.19:0/65777808 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f1358ecf640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13540b2cb0 0x7f13540d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060130272 0 0) 0x7f13540b97a0 con 0x7f13540d1020 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1340003520 con 0x7f13540d1020 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756483295 0 0) 0x7f13540d1c50 con 0x7f135415dbe0 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13540b97a0 con 0x7f135415dbe0 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 383000677 0 0) 0x7f1340003520 con 0x7f13540d1020 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13540b2cb0 msgr2=0x7f13540d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13540b2cb0 0x7f13540d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 >> v1:172.21.15.112:6789/0 conn(0x7f135415dbe0 legacy=0x7f13540d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13540d1f30 con 0x7f13540d1020 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f13500019f0 con 0x7f13540d1020 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 >> v1:172.21.15.19:6789/0 conn(0x7f13540d1020 legacy=0x7f13540d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1340003b70 0x7f1340003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1340004800 0x7f1340004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 0x7f134000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134000bfb0 con 0x7f13400051c0 2024-04-25T10:04:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134000c160 con 0x7f1340003b70 2024-04-25T10:04:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134000c360 con 0x7f1340004800 2024-04-25T10:04:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f1358ecf640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13540b2cb0 0x7f13540d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f1359ed1640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 0x7f134000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f13596d0640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1340003b70 0x7f1340003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.086+0000 7f1358ecf640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1340004800 0x7f1340004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.087+0000 7f1359ed1640 1 -- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1340004800 msgr2=0x7f1340004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.087+0000 7f1359ed1640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1340004800 0x7f1340004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.087+0000 7f1359ed1640 1 -- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1340003b70 msgr2=0x7f1340003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.087+0000 7f1359ed1640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1340003b70 0x7f1340003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.087+0000 7f1359ed1640 1 -- 172.21.15.19:0/65777808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1340003520 con 0x7f13400051c0 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.087+0000 7f1358ecf640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1340004800 0x7f1340004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.087+0000 7f1359ed1640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 0x7f134000b9c0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f135000a700 tx=0x7f1340005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d991acb26f9aa67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.087+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135000a350 con 0x7f13400051c0 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.087+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135002f070 con 0x7f13400051c0 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.087+0000 7f134bfff640 1 -- 172.21.15.19:0/65777808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135002bb50 con 0x7f13400051c0 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.088+0000 7f135b934640 1 -- 172.21.15.19:0/65777808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 msgr2=0x7f134000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.088+0000 7f135b934640 1 --2- 172.21.15.19:0/65777808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 0x7f134000b9c0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f135000a700 tx=0x7f1340005830 comp rx=0 tx=0).stop 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.088+0000 7f1359ed1640 1 -- 172.21.15.19:0/65777808 reap_dead start 2024-04-25T10:04:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.088+0000 7f135b934640 1 -- 172.21.15.19:0/65777808 shutdown_connections 2024-04-25T10:04:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.088+0000 7f135b934640 1 -- 172.21.15.19:0/65777808 >> 172.21.15.19:0/65777808 conn(0x7f1354061a20 msgr2=0x7f1354061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.088+0000 7f135b934640 1 -- 172.21.15.19:0/65777808 shutdown_connections 2024-04-25T10:04:24.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.088+0000 7f135b934640 1 -- 172.21.15.19:0/65777808 wait complete. 2024-04-25T10:04:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.089+0000 7f135b934640 1 Processor -- start 2024-04-25T10:04:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.089+0000 7f135b934640 1 -- start start 2024-04-25T10:04:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f135b934640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 0x7f135415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f135b934640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13540cdb10 0x7f13540b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f135b934640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13540c9880 0x7f13540c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f135b934640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13540b92a0 con 0x7f13400051c0 2024-04-25T10:04:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f135b934640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13540d1a00 con 0x7f13540c9880 2024-04-25T10:04:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f135b934640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13540b9420 con 0x7f13540cdb10 2024-04-25T10:04:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f13596d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 0x7f135415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f13596d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 0x7f135415dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57112/0 (socket says 172.21.15.19:57112) 2024-04-25T10:04:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f13596d0640 1 -- 172.21.15.19:0/539168322 learned_addr learned my addr 172.21.15.19:0/539168322 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f1359ed1640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13540c9880 0x7f13540c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f1358ecf640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13540cdb10 0x7f13540b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f13596d0640 1 -- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13540cdb10 msgr2=0x7f13540b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f13596d0640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13540cdb10 0x7f13540b7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f13596d0640 1 -- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13540c9880 msgr2=0x7f13540c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f13596d0640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13540c9880 0x7f13540c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f13596d0640 1 -- 172.21.15.19:0/539168322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135415e8f0 con 0x7f13400051c0 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f1358ecf640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13540cdb10 0x7f13540b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.090+0000 7f1359ed1640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13540c9880 0x7f13540c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.091+0000 7f13596d0640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 0x7f135415dfc0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f134c00f820 tx=0x7f134c00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.091+0000 7f1349ffb640 1 -- 172.21.15.19:0/539168322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c0052e0 con 0x7f13400051c0 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.091+0000 7f1349ffb640 1 -- 172.21.15.19:0/539168322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c018070 con 0x7f13400051c0 2024-04-25T10:04:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.091+0000 7f135b934640 1 -- 172.21.15.19:0/539168322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135415ebd0 con 0x7f13400051c0 2024-04-25T10:04:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.091+0000 7f1349ffb640 1 -- 172.21.15.19:0/539168322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c00d970 con 0x7f13400051c0 2024-04-25T10:04:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.091+0000 7f135b934640 1 -- 172.21.15.19:0/539168322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135415f8f0 con 0x7f13400051c0 2024-04-25T10:04:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.092+0000 7f135b934640 1 -- 172.21.15.19:0/539168322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135415faf0 con 0x7f13400051c0 2024-04-25T10:04:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.092+0000 7f1349ffb640 1 -- 172.21.15.19:0/539168322 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f134c00db10 con 0x7f13400051c0 2024-04-25T10:04:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.092+0000 7f1349ffb640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f133806b4d0 0x7f133806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.093+0000 7f1349ffb640 1 -- 172.21.15.19:0/539168322 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f134c08a3a0 con 0x7f13400051c0 2024-04-25T10:04:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.093+0000 7f1358ecf640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f133806b4d0 0x7f133806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.093+0000 7f1358ecf640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f133806b4d0 0x7f133806d990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f13541f7b10 tx=0x7f1344008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.097+0000 7f1349ffb640 1 -- 172.21.15.19:0/539168322 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f135415faf0 con 0x7f13400051c0 2024-04-25T10:04:24.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.313+0000 7f135b934640 1 -- 172.21.15.19:0/539168322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1354073990 con 0x7f13400051c0 2024-04-25T10:04:24.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.313+0000 7f1349ffb640 1 -- 172.21.15.19:0/539168322 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v109) v1 ==== 53+0+73 (secure 0 0 0) 0x7f134c01a310 con 0x7f13400051c0 2024-04-25T10:04:24.320 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 -- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f133806b4d0 msgr2=0x7f133806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f133806b4d0 0x7f133806d990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f13541f7b10 tx=0x7f1344008040 comp rx=0 tx=0).stop 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 -- 172.21.15.19:0/539168322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 msgr2=0x7f135415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 0x7f135415dfc0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f134c00f820 tx=0x7f134c00fce0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 -- 172.21.15.19:0/539168322 shutdown_connections 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f133806b4d0 0x7f133806d990 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13540c9880 0x7f13540c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13540cdb10 0x7f13540b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 --2- 172.21.15.19:0/539168322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13400051c0 0x7f135415dfc0 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 -- 172.21.15.19:0/539168322 >> 172.21.15.19:0/539168322 conn(0x7f1354061a20 msgr2=0x7f135415b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 -- 172.21.15.19:0/539168322 shutdown_connections 2024-04-25T10:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:24.315+0000 7f135b934640 1 -- 172.21.15.19:0/539168322 wait complete. 2024-04-25T10:04:24.332 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:04:24.332 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:04:24.332 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T10:04:24.332 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:04:24.332 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:24.332 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:04:24.333 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:04:24.333 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:04:24.333 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:24.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:24.333 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T10:04:24.348 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T10:04:24.349 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42081sec preferred_lft 42081sec 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:04:24.350 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T10:04:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:24.351 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:04:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:04:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T10:04:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:24.351 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:04:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:04:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T10:04:24.351 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:24.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:24.352 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:04:24.411 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T10:04:24.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:24.413 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T10:04:24.469 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T10:04:24.469 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:04:24.469 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:04:24.469 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:04:24.469 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:04:24.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:24.469 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T10:04:24.469 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:04:24.469 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:04:24.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:04:24.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:04:24.470 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:04:24.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:24.555 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T10:04:24.569 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:04:24.570 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:04:24.570 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:24.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:24.570 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T10:04:24.634 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:04:24.634 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:24.689 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:24.689 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:04:24.689 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:24.744 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:04:24.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:24.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:04:24.826 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:04:24.826 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:04:24.833 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:04:24.833 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:04:24.833 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T10:04:24.833 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T10:04:24.833 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T10:04:24.834 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:04:24.834 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:04:24.834 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:04:24.834 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:04:24.834 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:04:24.836 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:04:24.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:24.836 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:24.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:24.901 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:04:24.962 INFO:teuthology.orchestra.run.smithi019.stdout:client9373 2024-04-25T10:04:24.962 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:04:24.962 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:25.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:25.018 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-04-25T10:04:25.077 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:04:25.077 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:25.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:25.134 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:04:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records in 2024-04-25T10:04:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records out 2024-04-25T10:04:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0362059 s, 174 MB/s 2024-04-25T10:04:25.228 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:04:25.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:04:25.270 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-04-25T10:04:25.270 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-04-25T10:04:25.271 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:04:25.272 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-04-25T10:04:25.273 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on skipped message = False 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-04-25T10:04:25.274 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 20 2024-04-25T10:04:25.276 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 20 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 10 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T10:04:25.277 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-04-25T10:04:25.278 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: mon op complaint time = 120 2024-04-25T10:04:25.279 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout: client mount timeout = 600 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = "" 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-04-25T10:04:25.280 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout:[mds] 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds = 20 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds balancer = 20 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: mds cache max size = 1000 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug frag = True 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug scatterstat = True 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: mds log max segments = 1 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: mds op complaint time = 180 2024-04-25T10:04:25.281 INFO:teuthology.orchestra.run.smithi019.stdout: mds verify scatter = False 2024-04-25T10:04:25.282 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T10:04:25.282 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T10:04:25.282 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T10:04:25.282 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-04-25T10:04:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-04-25T10:04:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:04:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-04-25T10:04:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:04:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:04:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.19,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T10:04:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-04-25T10:04:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-04-25T10:04:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-04-25T10:04:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:04:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: mon op complaint time = 120 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-04-25T10:04:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: client mount timeout = 600 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-04-25T10:04:25.292 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds = 20 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds balancer = 20 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: mds cache max size = 1000 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug frag = True 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug scatterstat = True 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: mds log max segments = 1 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: mds op complaint time = 180 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: mds verify scatter = False 2024-04-25T10:04:25.294 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T10:04:25.294 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T10:04:25.294 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T10:04:25.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:04:25.337 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-04-25T10:04:25.377 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-04-25T10:04:25.378 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on skipped message = False 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:04:25.380 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-04-25T10:04:25.381 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 20 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 20 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-04-25T10:04:25.382 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 10 2024-04-25T10:04:25.383 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:04:25.383 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-04-25T10:04:25.383 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:04:25.383 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:04:25.383 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-04-25T10:04:25.383 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T10:04:25.383 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.383 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-04-25T10:04:25.383 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:04:25.383 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-04-25T10:04:25.383 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:25.384 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:04:25.385 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout: mon op complaint time = 120 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout: client mount timeout = 600 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-04-25T10:04:25.386 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:04:25.387 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = "" 2024-04-25T10:04:25.387 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T10:04:25.387 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T10:04:25.387 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-04-25T10:04:25.387 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-04-25T10:04:25.387 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-04-25T10:04:25.387 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-04-25T10:04:25.387 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-04-25T10:04:25.387 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-04-25T10:04:25.387 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout:[mds] 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds = 20 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds balancer = 20 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: mds cache max size = 1000 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug frag = True 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug scatterstat = False 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: mds log max segments = 1 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: mds op complaint time = 180 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: mds verify scatter = False 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T10:04:25.388 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T10:04:25.389 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-04-25T10:04:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-04-25T10:04:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:04:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-04-25T10:04:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-04-25T10:04:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:04:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:04:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.19,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2024-04-25T10:04:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-04-25T10:04:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-04-25T10:04:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-04-25T10:04:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-04-25T10:04:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-04-25T10:04:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:04:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:04:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-04-25T10:04:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-04-25T10:04:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:04:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:04:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: mon op complaint time = 120 2024-04-25T10:04:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:04:25.399 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-04-25T10:04:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-04-25T10:04:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-04-25T10:04:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-04-25T10:04:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:04:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:04:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: client mount timeout = 600 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2024-04-25T10:04:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds = 20 2024-04-25T10:04:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds balancer = 20 2024-04-25T10:04:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:04:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: mds cache max size = 1000 2024-04-25T10:04:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug frag = True 2024-04-25T10:04:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug scatterstat = False 2024-04-25T10:04:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: mds log max segments = 1 2024-04-25T10:04:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: mds op complaint time = 180 2024-04-25T10:04:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: mds verify scatter = False 2024-04-25T10:04:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T10:04:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T10:04:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T10:04:25.403 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T10:04:25.403 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T10:04:25.403 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T10:04:25.403 DEBUG:teuthology.orchestra.run.smithi019:> import json 2024-04-25T10:04:25.403 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-04-25T10:04:25.403 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T10:04:25.404 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T10:04:25.404 DEBUG:teuthology.orchestra.run.smithi019:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:04:25.404 DEBUG:teuthology.orchestra.run.smithi019:> except OSError as e: 2024-04-25T10:04:25.404 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(e.errno) 2024-04-25T10:04:25.404 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T10:04:25.404 DEBUG:teuthology.orchestra.run.smithi019:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:04:25.404 DEBUG:teuthology.orchestra.run.smithi019:> print(json.dumps( 2024-04-25T10:04:25.404 DEBUG:teuthology.orchestra.run.smithi019:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:04:25.404 DEBUG:teuthology.orchestra.run.smithi019:> indent=2)) 2024-04-25T10:04:25.405 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T10:04:25.511 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:04:25.511 INFO:teuthology.orchestra.run.smithi019.stdout: "st_mode": 33188, 2024-04-25T10:04:25.511 INFO:teuthology.orchestra.run.smithi019.stdout: "st_ino": 1099511627777, 2024-04-25T10:04:25.511 INFO:teuthology.orchestra.run.smithi019.stdout: "st_dev": 43, 2024-04-25T10:04:25.511 INFO:teuthology.orchestra.run.smithi019.stdout: "st_nlink": 1, 2024-04-25T10:04:25.511 INFO:teuthology.orchestra.run.smithi019.stdout: "st_uid": 1000, 2024-04-25T10:04:25.511 INFO:teuthology.orchestra.run.smithi019.stdout: "st_gid": 1316, 2024-04-25T10:04:25.511 INFO:teuthology.orchestra.run.smithi019.stdout: "st_size": 6291456, 2024-04-25T10:04:25.511 INFO:teuthology.orchestra.run.smithi019.stdout: "st_atime": 1714039465.1838725, 2024-04-25T10:04:25.511 INFO:teuthology.orchestra.run.smithi019.stdout: "st_mtime": 1714039465.2018723, 2024-04-25T10:04:25.511 INFO:teuthology.orchestra.run.smithi019.stdout: "st_ctime": 1714039465.2018723 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:04:25.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:04:25.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:25.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.656+0000 7f74cd26b640 1 Processor -- start 2024-04-25T10:04:25.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.656+0000 7f74cd26b640 1 -- start start 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.656+0000 7f74cd26b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f74c80b96d0 0x7f74c80d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.656+0000 7f74cd26b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f74c80d0fe0 0x7f74c80d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.656+0000 7f74cd26b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c815e810 con 0x7f74c815dd20 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.656+0000 7f74cd26b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c80d1c00 con 0x7f74c80d0fe0 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.656+0000 7f74cd26b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c80d1420 con 0x7f74c80b96d0 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.657+0000 7f74c77fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f74c80d0fe0 0x7f74c80d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.657+0000 7f74c6ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f74c815dd20 0x7f74c815e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57614/0 (socket says 172.21.15.19:57614) 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.657+0000 7f74c6ffd640 1 -- 172.21.15.19:0/763078117 learned_addr learned my addr 172.21.15.19:0/763078117 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.657+0000 7f74c67fc640 1 --2- 172.21.15.19:0/763078117 >> v2:172.21.15.112:3300/0 conn(0x7f74c80b96d0 0x7f74c80d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.657+0000 7f74c77fe640 1 -- 172.21.15.19:0/763078117 >> v2:172.21.15.112:3300/0 conn(0x7f74c80b96d0 msgr2=0x7f74c80d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.657+0000 7f74c77fe640 1 --2- 172.21.15.19:0/763078117 >> v2:172.21.15.112:3300/0 conn(0x7f74c80b96d0 0x7f74c80d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.657+0000 7f74c77fe640 1 -- 172.21.15.19:0/763078117 >> v1:172.21.15.19:6789/0 conn(0x7f74c815dd20 legacy=0x7f74c815e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.657+0000 7f74c77fe640 1 -- 172.21.15.19:0/763078117 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c815eac0 con 0x7f74c80d0fe0 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.657+0000 7f74c5ffb640 1 -- 172.21.15.19:0/763078117 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302452828 0 0) 0x7f74c815e810 con 0x7f74c815dd20 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74c77fe640 1 --2- 172.21.15.19:0/763078117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c80d0fe0 0x7f74c80d2370 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f74c80b5d20 tx=0x7f74c002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3468228c6ff0de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74c5ffb640 1 -- 172.21.15.19:0/763078117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0007740 con 0x7f74c80d0fe0 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74c5ffb640 1 -- 172.21.15.19:0/763078117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c00078e0 con 0x7f74c80d0fe0 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74c5ffb640 1 -- 172.21.15.19:0/763078117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0007cb0 con 0x7f74c80d0fe0 2024-04-25T10:04:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74cd26b640 1 -- 172.21.15.19:0/763078117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c80d0fe0 msgr2=0x7f74c80d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74cd26b640 1 --2- 172.21.15.19:0/763078117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c80d0fe0 0x7f74c80d2370 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f74c80b5d20 tx=0x7f74c002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74cd26b640 1 -- 172.21.15.19:0/763078117 shutdown_connections 2024-04-25T10:04:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74cd26b640 1 --2- 172.21.15.19:0/763078117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c80d0fe0 0x7f74c80d2370 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74cd26b640 1 --2- 172.21.15.19:0/763078117 >> v2:172.21.15.112:3300/0 conn(0x7f74c80b96d0 0x7f74c80d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74cd26b640 1 -- 172.21.15.19:0/763078117 >> 172.21.15.19:0/763078117 conn(0x7f74c8061a20 msgr2=0x7f74c8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74cd26b640 1 -- 172.21.15.19:0/763078117 shutdown_connections 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.658+0000 7f74cd26b640 1 -- 172.21.15.19:0/763078117 wait complete. 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.660+0000 7f74cd26b640 1 Processor -- start 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.660+0000 7f74cd26b640 1 -- start start 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.660+0000 7f74cd26b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74c80b96d0 0x7f74c80cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.660+0000 7f74cd26b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74c80d0fe0 0x7f74c80b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.660+0000 7f74cd26b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c815dd20 0x7f74c81f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.660+0000 7f74cd26b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80d1e50 con 0x7f74c815dd20 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.660+0000 7f74cd26b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c815f080 con 0x7f74c80b96d0 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.660+0000 7f74cd26b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80d28b0 con 0x7f74c80d0fe0 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74c77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c815dd20 0x7f74c81f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74c77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c815dd20 0x7f74c81f3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57128/0 (socket says 172.21.15.19:57128) 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74c77fe640 1 -- 172.21.15.19:0/1980264099 learned_addr learned my addr 172.21.15.19:0/1980264099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74c67fc640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74c80d0fe0 0x7f74c80b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74c6ffd640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74c80b96d0 0x7f74c80cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74c77fe640 1 -- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74c80d0fe0 msgr2=0x7f74c80b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74c77fe640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74c80d0fe0 0x7f74c80b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74c77fe640 1 -- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74c80b96d0 msgr2=0x7f74c80cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74c77fe640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74c80b96d0 0x7f74c80cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74c77fe640 1 -- 172.21.15.19:0/1980264099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80caef0 con 0x7f74c815dd20 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74c77fe640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c815dd20 0x7f74c81f3a20 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f74c002abe0 tx=0x7f74c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74af7fe640 1 -- 172.21.15.19:0/1980264099 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0040070 con 0x7f74c815dd20 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74af7fe640 1 -- 172.21.15.19:0/1980264099 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c0031bc0 con 0x7f74c815dd20 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74cd26b640 1 -- 172.21.15.19:0/1980264099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c80cb4b0 con 0x7f74c815dd20 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74af7fe640 1 -- 172.21.15.19:0/1980264099 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0045460 con 0x7f74c815dd20 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.661+0000 7f74cd26b640 1 -- 172.21.15.19:0/1980264099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c80cba70 con 0x7f74c815dd20 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.662+0000 7f74cd26b640 1 -- 172.21.15.19:0/1980264099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c80cbc70 con 0x7f74c815dd20 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.663+0000 7f74af7fe640 1 -- 172.21.15.19:0/1980264099 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f74c0039040 con 0x7f74c815dd20 2024-04-25T10:04:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.663+0000 7f74af7fe640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f74b406b4d0 0x7f74b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.664+0000 7f74af7fe640 1 -- 172.21.15.19:0/1980264099 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f74c00abb40 con 0x7f74c815dd20 2024-04-25T10:04:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.664+0000 7f74c6ffd640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f74b406b4d0 0x7f74b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.664+0000 7f74c6ffd640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f74b406b4d0 0x7f74b406d990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f74b8007fd0 tx=0x7f74b8007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.668+0000 7f74af7fe640 1 -- 172.21.15.19:0/1980264099 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74c80cbc70 con 0x7f74c815dd20 2024-04-25T10:04:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.888+0000 7f74cd26b640 1 -- 172.21.15.19:0/1980264099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74c8073990 con 0x7f74c815dd20 2024-04-25T10:04:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.888+0000 7f74af7fe640 1 -- 172.21.15.19:0/1980264099 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4954 (secure 0 0 0) 0x7f74c007a1e0 con 0x7f74c815dd20 2024-04-25T10:04:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:25.895 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1300145056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1300145056},{"type":"v1","addr":"172.21.15.19:6835","nonce":1300145056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":8027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2001687418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2001687418},{"type":"v1","addr":"172.21.15.112:6836","nonce":2001687418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721457153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721457153},{"type":"v1","addr":"172.21.15.112:6837","nonce":721457153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:14.252458+0000","modified":"2024-04-25T10:04:16.259643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10698":{"gid":10698,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3942829201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3942829201},{"type":"v1","addr":"172.21.15.19:6834","nonce":3942829201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:04:25.895 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 109 2024-04-25T10:04:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 -- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f74b406b4d0 msgr2=0x7f74b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f74b406b4d0 0x7f74b406d990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f74b8007fd0 tx=0x7f74b8007de0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 -- 172.21.15.19:0/1980264099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c815dd20 msgr2=0x7f74c81f3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c815dd20 0x7f74c81f3a20 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f74c002abe0 tx=0x7f74c0004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 -- 172.21.15.19:0/1980264099 shutdown_connections 2024-04-25T10:04:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f74b406b4d0 0x7f74b406d990 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74c815dd20 0x7f74c81f3a20 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74c80d0fe0 0x7f74c80b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 --2- 172.21.15.19:0/1980264099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74c80b96d0 0x7f74c80cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 -- 172.21.15.19:0/1980264099 >> 172.21.15.19:0/1980264099 conn(0x7f74c8061a20 msgr2=0x7f74c80d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 -- 172.21.15.19:0/1980264099 shutdown_connections 2024-04-25T10:04:25.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:25.891+0000 7f74cd26b640 1 -- 172.21.15.19:0/1980264099 wait complete. 2024-04-25T10:04:25.908 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:04:26.031 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-04-25T10:04:26.032 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:04:26.032 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent '' 2024-04-25T10:04:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.070+0000 7f4b4b47af40 1 Processor -- start 2024-04-25T10:04:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.070+0000 7f4b4b47af40 1 -- start start 2024-04-25T10:04:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.070+0000 7f4b4b47af40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55b0394abfb0 0x55b0395829c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.070+0000 7f4b4b47af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0395850a0 0x55b039579050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.070+0000 7f4b4b47af40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0394d8bb0 0x55b0394d9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.070+0000 7f4b4b47af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0394ae9a0 con 0x55b0395850a0 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.070+0000 7f4b4b47af40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b039454df0 con 0x55b0394d8bb0 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.070+0000 7f4b4b47af40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0394f71a0 con 0x55b0394abfb0 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.070+0000 7f4b4877a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0395850a0 0x55b039579050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.070+0000 7f4b4877a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0395850a0 0x55b039579050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57150/0 (socket says 172.21.15.19:57150) 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.070+0000 7f4b4877a640 1 -- 172.21.15.19:0/3627256451 learned_addr learned my addr 172.21.15.19:0/3627256451 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4977c640 1 --2- 172.21.15.19:0/3627256451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0394d8bb0 0x55b0394d9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b48f7b640 1 --2- 172.21.15.19:0/3627256451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55b0394abfb0 0x55b0395829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4877a640 1 -- 172.21.15.19:0/3627256451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55b0394abfb0 msgr2=0x55b0395829c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4877a640 1 --2- 172.21.15.19:0/3627256451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55b0394abfb0 0x55b0395829c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4877a640 1 -- 172.21.15.19:0/3627256451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0394d8bb0 msgr2=0x55b0394d9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4877a640 1 --2- 172.21.15.19:0/3627256451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0394d8bb0 0x55b0394d9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4877a640 1 -- 172.21.15.19:0/3627256451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0393dac50 con 0x55b0395850a0 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4877a640 1 --2- 172.21.15.19:0/3627256451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0395850a0 0x55b039579050 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f4b30002e10 tx=0x7f4b3000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfdf522abda33e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b47778640 1 -- 172.21.15.19:0/3627256451 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30013020 con 0x55b0395850a0 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b47778640 1 -- 172.21.15.19:0/3627256451 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3000f400 con 0x55b0395850a0 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b47778640 1 -- 172.21.15.19:0/3627256451 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000f7f0 con 0x55b0395850a0 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4b47af40 1 -- 172.21.15.19:0/3627256451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0395850a0 msgr2=0x55b039579050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4b47af40 1 --2- 172.21.15.19:0/3627256451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0395850a0 0x55b039579050 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f4b30002e10 tx=0x7f4b3000b5b0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4b47af40 1 -- 172.21.15.19:0/3627256451 shutdown_connections 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4b47af40 1 --2- 172.21.15.19:0/3627256451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0394d8bb0 0x55b0394d9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4b47af40 1 --2- 172.21.15.19:0/3627256451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0395850a0 0x55b039579050 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4b47af40 1 --2- 172.21.15.19:0/3627256451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55b0394abfb0 0x55b0395829c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.071+0000 7f4b4b47af40 1 -- 172.21.15.19:0/3627256451 >> 172.21.15.19:0/3627256451 conn(0x55b0394ac9b0 msgr2=0x55b0394f6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.072+0000 7f4b4b47af40 1 -- 172.21.15.19:0/3627256451 shutdown_connections 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.072+0000 7f4b4b47af40 1 -- 172.21.15.19:0/3627256451 wait complete. 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.072+0000 7f4b4b47af40 1 Processor -- start 2024-04-25T10:04:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.072+0000 7f4b4b47af40 1 -- start start 2024-04-25T10:04:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4b47af40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0394abfb0 0x55b0395704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4b47af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0394d8bb0 0x55b0394db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4b47af40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55b0395850a0 0x55b0394db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4b47af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b039582f00 con 0x55b0394d8bb0 2024-04-25T10:04:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4b47af40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0395806b0 con 0x55b0394abfb0 2024-04-25T10:04:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4b47af40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0394af310 con 0x55b0395850a0 2024-04-25T10:04:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4877a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0394d8bb0 0x55b0394db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4877a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0394d8bb0 0x55b0394db0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57160/0 (socket says 172.21.15.19:57160) 2024-04-25T10:04:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4877a640 1 -- 172.21.15.19:0/2785340366 learned_addr learned my addr 172.21.15.19:0/2785340366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b48f7b640 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0394abfb0 0x55b0395704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4977c640 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55b0395850a0 0x55b0394db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4877a640 1 -- 172.21.15.19:0/2785340366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55b0395850a0 msgr2=0x55b0394db5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4877a640 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55b0395850a0 0x55b0394db5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4877a640 1 -- 172.21.15.19:0/2785340366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0394abfb0 msgr2=0x55b0395704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4877a640 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0394abfb0 0x55b0395704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.073+0000 7f4b4877a640 1 -- 172.21.15.19:0/2785340366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b039574f90 con 0x55b0394d8bb0 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.074+0000 7f4b4877a640 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0394d8bb0 0x55b0394db0b0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f4b30000c00 tx=0x7f4b30001fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.074+0000 7f4b46776640 1 -- 172.21.15.19:0/2785340366 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3001a850 con 0x55b0394d8bb0 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.074+0000 7f4b46776640 1 -- 172.21.15.19:0/2785340366 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3001a9f0 con 0x55b0394d8bb0 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.074+0000 7f4b4b47af40 1 -- 172.21.15.19:0/2785340366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b0395856d0 con 0x55b0394d8bb0 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.074+0000 7f4b46776640 1 -- 172.21.15.19:0/2785340366 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3001ae20 con 0x55b0394d8bb0 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.074+0000 7f4b4b47af40 1 -- 172.21.15.19:0/2785340366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b039585c90 con 0x55b0394d8bb0 2024-04-25T10:04:26.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.075+0000 7f4b46776640 1 -- 172.21.15.19:0/2785340366 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b30016020 con 0x55b0394d8bb0 2024-04-25T10:04:26.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.076+0000 7f4b46776640 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b28069eb0 0x7f4b2806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.076+0000 7f4b46776640 1 -- 172.21.15.19:0/2785340366 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b3008a7f0 con 0x55b0394d8bb0 2024-04-25T10:04:26.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.076+0000 7f4b48f7b640 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b28069eb0 0x7f4b2806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.076+0000 7f4b4b47af40 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x55b0395709e0 0x55b039586c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.076+0000 7f4b4b47af40 1 -- 172.21.15.19:0/2785340366 --> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] -- osd_op(unknown.0.0:1 26.7 26:e2e07930:::10000000001.00000000:head [setxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165) v9 -- 0x55b03957ab60 con 0x55b0395709e0 2024-04-25T10:04:26.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.076+0000 7f4b4977c640 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x55b0395709e0 0x55b039586c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.076+0000 7f4b48f7b640 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b28069eb0 0x7f4b2806c370 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4b40007600 tx=0x7f4b400059c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.076+0000 7f4b4977c640 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x55b0395709e0 0x55b039586c80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.077+0000 7f4b4977c640 1 -- 172.21.15.19:0/2785340366 <== osd.2 v2:172.21.15.19:6816/3205329770 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr] v165'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b38056030 con 0x55b0395709e0 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.078+0000 7f4b4b47af40 1 -- 172.21.15.19:0/2785340366 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x55b0395709e0 msgr2=0x55b039586c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.078+0000 7f4b4b47af40 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x55b0395709e0 0x55b039586c80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.078+0000 7f4b4b47af40 1 -- 172.21.15.19:0/2785340366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b28069eb0 msgr2=0x7f4b2806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.078+0000 7f4b4b47af40 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b28069eb0 0x7f4b2806c370 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4b40007600 tx=0x7f4b400059c0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.078+0000 7f4b4b47af40 1 -- 172.21.15.19:0/2785340366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0394d8bb0 msgr2=0x55b0394db0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.078+0000 7f4b4b47af40 1 --2- 172.21.15.19:0/2785340366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0394d8bb0 0x55b0394db0b0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f4b30000c00 tx=0x7f4b30001fe0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.078+0000 7f4b4977c640 1 -- 172.21.15.19:0/2785340366 reap_dead start 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.078+0000 7f4b4b47af40 1 -- 172.21.15.19:0/2785340366 shutdown_connections 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.078+0000 7f4b4b47af40 1 -- 172.21.15.19:0/2785340366 >> 172.21.15.19:0/2785340366 conn(0x55b0394ac9b0 msgr2=0x55b039579050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.078+0000 7f4b4b47af40 1 -- 172.21.15.19:0/2785340366 shutdown_connections 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.078+0000 7f4b4b47af40 1 -- 172.21.15.19:0/2785340366 wait complete. 2024-04-25T10:04:26.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:26.089 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub start / recursive,repair 2024-04-25T10:04:26.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.188+0000 7f72a8d5c640 1 Processor -- start 2024-04-25T10:04:26.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.188+0000 7f72a8d5c640 1 -- start start 2024-04-25T10:04:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f72a8d5c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f72a40cc4f0 0x7f72a40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f72a8d5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f72a415b980 0x7f72a40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f72a8d5c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a4072280 con 0x7f72a40cdb20 2024-04-25T10:04:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f72a8d5c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a4071270 con 0x7f72a40cc4f0 2024-04-25T10:04:26.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f72a8d5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a415bd80 con 0x7f72a415b980 2024-04-25T10:04:26.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f72a259c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f72a40cc4f0 0x7f72a40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f729bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f72a40cdb20 0x7f72a415b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57616/0 (socket says 172.21.15.19:57616) 2024-04-25T10:04:26.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f729bfff640 1 -- 172.21.15.19:0/688394454 learned_addr learned my addr 172.21.15.19:0/688394454 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:26.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f72a2d9d640 1 --2- 172.21.15.19:0/688394454 >> v2:172.21.15.112:3300/0 conn(0x7f72a415b980 0x7f72a40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f72a259c640 1 -- 172.21.15.19:0/688394454 >> v2:172.21.15.112:3300/0 conn(0x7f72a415b980 msgr2=0x7f72a40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f72a259c640 1 --2- 172.21.15.19:0/688394454 >> v2:172.21.15.112:3300/0 conn(0x7f72a415b980 0x7f72a40cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f72a259c640 1 -- 172.21.15.19:0/688394454 >> v1:172.21.15.19:6789/0 conn(0x7f72a40cdb20 legacy=0x7f72a415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.189+0000 7f72a259c640 1 -- 172.21.15.19:0/688394454 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a40714c0 con 0x7f72a40cc4f0 2024-04-25T10:04:26.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.190+0000 7f72a259c640 1 --2- 172.21.15.19:0/688394454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a40cc4f0 0x7f72a40cc8d0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f72a40b4c00 tx=0x7f729402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2031b9dac25555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.190+0000 7f72a1d9b640 1 -- 172.21.15.19:0/688394454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294037070 con 0x7f72a40cc4f0 2024-04-25T10:04:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.190+0000 7f72a1d9b640 1 -- 172.21.15.19:0/688394454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7294032440 con 0x7f72a40cc4f0 2024-04-25T10:04:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.190+0000 7f72a1d9b640 1 -- 172.21.15.19:0/688394454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294032810 con 0x7f72a40cc4f0 2024-04-25T10:04:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.191+0000 7f72a8d5c640 1 -- 172.21.15.19:0/688394454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a40cc4f0 msgr2=0x7f72a40cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.191+0000 7f72a8d5c640 1 --2- 172.21.15.19:0/688394454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a40cc4f0 0x7f72a40cc8d0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f72a40b4c00 tx=0x7f729402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.191+0000 7f72a8d5c640 1 -- 172.21.15.19:0/688394454 shutdown_connections 2024-04-25T10:04:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.191+0000 7f72a8d5c640 1 --2- 172.21.15.19:0/688394454 >> v2:172.21.15.112:3300/0 conn(0x7f72a415b980 0x7f72a40cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.191+0000 7f72a8d5c640 1 --2- 172.21.15.19:0/688394454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a40cc4f0 0x7f72a40cc8d0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.191+0000 7f72a8d5c640 1 -- 172.21.15.19:0/688394454 >> 172.21.15.19:0/688394454 conn(0x7f72a406abe0 msgr2=0x7f72a406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.191+0000 7f72a8d5c640 1 -- 172.21.15.19:0/688394454 shutdown_connections 2024-04-25T10:04:26.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.191+0000 7f72a8d5c640 1 -- 172.21.15.19:0/688394454 wait complete. 2024-04-25T10:04:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.192+0000 7f72a8d5c640 1 Processor -- start 2024-04-25T10:04:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.192+0000 7f72a8d5c640 1 -- start start 2024-04-25T10:04:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f72a8d5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72a40cc4f0 0x7f72a41f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f72a8d5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a40cdb20 0x7f72a40ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f72a8d5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72a415b980 0x7f72a414c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f72a8d5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a4071d60 con 0x7f72a40cdb20 2024-04-25T10:04:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f72a8d5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a4071a80 con 0x7f72a415b980 2024-04-25T10:04:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f72a8d5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40ae500 con 0x7f72a40cc4f0 2024-04-25T10:04:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f729bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a40cdb20 0x7f72a40ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f729bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a40cdb20 0x7f72a40ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57186/0 (socket says 172.21.15.19:57186) 2024-04-25T10:04:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f729bfff640 1 -- 172.21.15.19:0/3127463466 learned_addr learned my addr 172.21.15.19:0/3127463466 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f72a259c640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72a40cc4f0 0x7f72a41f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f72a2d9d640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72a415b980 0x7f72a414c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f729bfff640 1 -- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72a40cc4f0 msgr2=0x7f72a41f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f729bfff640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72a40cc4f0 0x7f72a41f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f729bfff640 1 -- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72a415b980 msgr2=0x7f72a414c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f729bfff640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72a415b980 0x7f72a414c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.193+0000 7f729bfff640 1 -- 172.21.15.19:0/3127463466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a41f7e90 con 0x7f72a40cdb20 2024-04-25T10:04:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.194+0000 7f729bfff640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a40cdb20 0x7f72a40ade70 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c430 tx=0x7f728c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.194+0000 7f729affd640 1 -- 172.21.15.19:0/3127463466 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c017070 con 0x7f72a40cdb20 2024-04-25T10:04:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.194+0000 7f729affd640 1 -- 172.21.15.19:0/3127463466 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c005ce0 con 0x7f72a40cdb20 2024-04-25T10:04:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.194+0000 7f729affd640 1 -- 172.21.15.19:0/3127463466 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c0072f0 con 0x7f72a40cdb20 2024-04-25T10:04:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.194+0000 7f72a8d5c640 1 -- 172.21.15.19:0/3127463466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a41f8170 con 0x7f72a40cdb20 2024-04-25T10:04:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.194+0000 7f72a8d5c640 1 -- 172.21.15.19:0/3127463466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a414ca40 con 0x7f72a40cdb20 2024-04-25T10:04:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.195+0000 7f729affd640 1 -- 172.21.15.19:0/3127463466 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f728c01a070 con 0x7f72a40cdb20 2024-04-25T10:04:26.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.196+0000 7f729affd640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728406b4d0 0x7f728406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.196+0000 7f729affd640 1 -- 172.21.15.19:0/3127463466 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f728c089710 con 0x7f72a40cdb20 2024-04-25T10:04:26.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.196+0000 7f72a259c640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728406b4d0 0x7f728406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.197+0000 7f72a259c640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728406b4d0 0x7f728406d990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7294034000 tx=0x7f729402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.197+0000 7f72aba64740 1 Processor -- start 2024-04-25T10:04:26.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.197+0000 7f72aba64740 1 -- start start 2024-04-25T10:04:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72aba64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e122bb0 0x564b1e122f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72aba64740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e11d830 0x564b1e1217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72aba64740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e122130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72aba64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1e086ac0 con 0x564b1e122bb0 2024-04-25T10:04:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72aba64740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1e02fd80 con 0x564b1e11d830 2024-04-25T10:04:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72aba64740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1e082b60 con 0x564b1e121cf0 2024-04-25T10:04:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e122bb0 0x564b1e122f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e122bb0 0x564b1e122f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57190/0 (socket says 172.21.15.19:57190) 2024-04-25T10:04:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72a259c640 1 -- 172.21.15.19:0/25874231 learned_addr learned my addr 172.21.15.19:0/25874231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72a2d9d640 1 --2- 172.21.15.19:0/25874231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e122130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f729bfff640 1 --2- 172.21.15.19:0/25874231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e11d830 0x564b1e1217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72a259c640 1 -- 172.21.15.19:0/25874231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 msgr2=0x564b1e122130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72a259c640 1 --2- 172.21.15.19:0/25874231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e122130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72a259c640 1 -- 172.21.15.19:0/25874231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e11d830 msgr2=0x564b1e1217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72a259c640 1 --2- 172.21.15.19:0/25874231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e11d830 0x564b1e1217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72a259c640 1 -- 172.21.15.19:0/25874231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b1de637e0 con 0x564b1e122bb0 2024-04-25T10:04:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f729bfff640 1 --2- 172.21.15.19:0/25874231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e11d830 0x564b1e1217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f72a259c640 1 --2- 172.21.15.19:0/25874231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e122bb0 0x564b1e122f90 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f729402a790 tx=0x7f729402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cd942d824375f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f7283fff640 1 -- 172.21.15.19:0/25874231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729404d070 con 0x564b1e122bb0 2024-04-25T10:04:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f7283fff640 1 -- 172.21.15.19:0/25874231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7294032c20 con 0x564b1e122bb0 2024-04-25T10:04:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.198+0000 7f7283fff640 1 -- 172.21.15.19:0/25874231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294037040 con 0x564b1e122bb0 2024-04-25T10:04:26.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.199+0000 7f72aba64740 1 -- 172.21.15.19:0/25874231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e122bb0 msgr2=0x564b1e122f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.199+0000 7f72aba64740 1 --2- 172.21.15.19:0/25874231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e122bb0 0x564b1e122f90 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f729402a790 tx=0x7f729402c000 comp rx=0 tx=0).stop 2024-04-25T10:04:26.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.199+0000 7f72aba64740 1 -- 172.21.15.19:0/25874231 shutdown_connections 2024-04-25T10:04:26.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.199+0000 7f72aba64740 1 --2- 172.21.15.19:0/25874231 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e122130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.199+0000 7f72aba64740 1 --2- 172.21.15.19:0/25874231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e11d830 0x564b1e1217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.199+0000 7f72aba64740 1 --2- 172.21.15.19:0/25874231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e122bb0 0x564b1e122f90 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.199+0000 7f72aba64740 1 -- 172.21.15.19:0/25874231 >> 172.21.15.19:0/25874231 conn(0x564b1e11fcb0 msgr2=0x564b1e168770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.199+0000 7f72aba64740 1 -- 172.21.15.19:0/25874231 shutdown_connections 2024-04-25T10:04:26.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.199+0000 7f72aba64740 1 -- 172.21.15.19:0/25874231 wait complete. 2024-04-25T10:04:26.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.200+0000 7f72aba64740 1 -- start start 2024-04-25T10:04:26.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.200+0000 7f72aba64740 1 Processor -- start 2024-04-25T10:04:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72aba64740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e13dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72aba64740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 0x564b1e13e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72aba64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e13ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72aba64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1e14e740 con 0x564b1e13e5e0 2024-04-25T10:04:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72aba64740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1e025530 con 0x564b1e122bb0 2024-04-25T10:04:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72aba64740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1e14dbc0 con 0x564b1e121cf0 2024-04-25T10:04:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72aba64740 10 client.0 _ll_register_callbacks cb 0x564b1e07d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e13ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e13ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57200/0 (socket says 172.21.15.19:57200) 2024-04-25T10:04:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72a2d9d640 1 -- 172.21.15.19:0/3974682615 learned_addr learned my addr 172.21.15.19:0/3974682615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f729bfff640 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 0x564b1e13e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72a259c640 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e13dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72a2d9d640 1 -- 172.21.15.19:0/3974682615 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 msgr2=0x564b1e13dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72a2d9d640 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e13dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72a2d9d640 1 -- 172.21.15.19:0/3974682615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 msgr2=0x564b1e13e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72a2d9d640 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 0x564b1e13e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72a2d9d640 1 -- 172.21.15.19:0/3974682615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b1dec3f70 con 0x564b1e13e5e0 2024-04-25T10:04:26.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72a259c640 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e13dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:26.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72a2d9d640 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e13ec90 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f729c0042a0 tx=0x7f729c004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d6d5e5eb8f3ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72817fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:04:26.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72817fa640 1 -- 172.21.15.19:0/3974682615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c01a020 con 0x564b1e13e5e0 2024-04-25T10:04:26.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72817fa640 1 -- 172.21.15.19:0/3974682615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f729c007830 con 0x564b1e13e5e0 2024-04-25T10:04:26.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.201+0000 7f72aba64740 1 -- 172.21.15.19:0/3974682615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7294032c20 con 0x564b1e13e5e0 2024-04-25T10:04:26.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72817fa640 1 -- 172.21.15.19:0/3974682615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f729c024050 con 0x564b1e13e5e0 2024-04-25T10:04:26.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72aba64740 10 client.10905 fetch_fsmap learned FSMap version 109 2024-04-25T10:04:26.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72aba64740 1 -- 172.21.15.19:0/3974682615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x564b1dd5b170 con 0x564b1e13e5e0 2024-04-25T10:04:26.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72817fa640 1 -- 172.21.15.19:0/3974682615 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7294032c20 con 0x564b1e13e5e0 2024-04-25T10:04:26.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72817fa640 1 -- 172.21.15.19:0/3974682615 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 109) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f729c027020 con 0x564b1e13e5e0 2024-04-25T10:04:26.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72aba64740 10 client.10905 fetch_fsmap finished waiting for FSMap version 109 2024-04-25T10:04:26.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72aba64740 10 client.10905 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:04:26.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72aba64740 20 client.10905 populate_metadata read hostname 'smithi019' 2024-04-25T10:04:26.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72aba64740 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x564b1e13f6a0 0x564b1e17bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72aba64740 4 client.10905 mds_command: new command op to 10698 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72aba64740 1 -- 172.21.15.19:0/3974682615 --> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b1e094000 con 0x564b1e13f6a0 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72a259c640 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x564b1e13f6a0 0x564b1e17bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72817fa640 10 client.10905 ms_handle_connect on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.202+0000 7f72a259c640 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x564b1e13f6a0 0x564b1e17bdb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b31fc4e888f4a9a2 server_cookie=d0542e38cbffeda9 in_seq=0 out_seq=0 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72817fa640 1 -- 172.21.15.19:0/3974682615 <== mds.0 v2:172.21.15.19:6832/3942829201 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b1e094000 con 0x564b1e13f6a0 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72817fa640 10 client.10905 handle_command_reply: tid=0 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 1 client.10905 shutdown 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 1 -- 172.21.15.19:0/3974682615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 msgr2=0x564b1e13ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e13ec90 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f729c0042a0 tx=0x7f729c004760 comp rx=0 tx=0).stop 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 1 -- 172.21.15.19:0/3974682615 shutdown_connections 2024-04-25T10:04:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x564b1e13f6a0 0x564b1e17bdb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e13ec90 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 0x564b1e13e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 1 --2- 172.21.15.19:0/3974682615 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e13dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72817fa640 0 client.10905 ms_handle_reset on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 1 -- 172.21.15.19:0/3974682615 >> 172.21.15.19:0/3974682615 conn(0x564b1e11d830 msgr2=0x564b1e11cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 1 -- 172.21.15.19:0/3974682615 shutdown_connections 2024-04-25T10:04:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 1 -- 172.21.15.19:0/3974682615 wait complete. 2024-04-25T10:04:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.203+0000 7f72aba64740 20 client.10905 trim_cache size 0 max 16384 2024-04-25T10:04:26.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.219+0000 7f72aba64740 1 Processor -- start 2024-04-25T10:04:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.219+0000 7f72aba64740 1 -- start start 2024-04-25T10:04:26.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.219+0000 7f72aba64740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e177c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.219+0000 7f72aba64740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 0x564b1e145090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.219+0000 7f72aba64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e1220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.219+0000 7f72aba64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a4200590 con 0x564b1e13e5e0 2024-04-25T10:04:26.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.219+0000 7f72aba64740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40d0390 con 0x564b1e122bb0 2024-04-25T10:04:26.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.219+0000 7f72aba64740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40cf790 con 0x564b1e121cf0 2024-04-25T10:04:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.219+0000 7f72a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e1220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.219+0000 7f72a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e1220d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57214/0 (socket says 172.21.15.19:57214) 2024-04-25T10:04:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.219+0000 7f72a2d9d640 1 -- 172.21.15.19:0/1279883884 learned_addr learned my addr 172.21.15.19:0/1279883884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.220+0000 7f72a2d9d640 1 -- 172.21.15.19:0/1279883884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 msgr2=0x564b1e177c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.220+0000 7f72a2d9d640 1 --2- 172.21.15.19:0/1279883884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e177c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.220+0000 7f72a2d9d640 1 -- 172.21.15.19:0/1279883884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 msgr2=0x564b1e145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.220+0000 7f72a2d9d640 1 --2- 172.21.15.19:0/1279883884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 0x564b1e145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.220+0000 7f72a2d9d640 1 -- 172.21.15.19:0/1279883884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b1e119750 con 0x564b1e13e5e0 2024-04-25T10:04:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.220+0000 7f72a2d9d640 1 --2- 172.21.15.19:0/1279883884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e1220d0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f729c003d60 tx=0x7f729c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c661a5754feb22a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.220+0000 7f72837fe640 1 -- 172.21.15.19:0/1279883884 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c01a070 con 0x564b1e13e5e0 2024-04-25T10:04:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.220+0000 7f72837fe640 1 -- 172.21.15.19:0/1279883884 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c0077c0 con 0x564b1e13e5e0 2024-04-25T10:04:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.220+0000 7f72837fe640 1 -- 172.21.15.19:0/1279883884 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c0090c0 con 0x564b1e13e5e0 2024-04-25T10:04:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.220+0000 7f72aba64740 1 -- 172.21.15.19:0/1279883884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 msgr2=0x564b1e1220d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.220+0000 7f72aba64740 1 --2- 172.21.15.19:0/1279883884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e1220d0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f729c003d60 tx=0x7f729c010000 comp rx=0 tx=0).stop 2024-04-25T10:04:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.221+0000 7f72aba64740 1 -- 172.21.15.19:0/1279883884 shutdown_connections 2024-04-25T10:04:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.221+0000 7f72aba64740 1 --2- 172.21.15.19:0/1279883884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e13e5e0 0x564b1e1220d0 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.221+0000 7f72aba64740 1 --2- 172.21.15.19:0/1279883884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 0x564b1e145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.221+0000 7f72aba64740 1 --2- 172.21.15.19:0/1279883884 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e121cf0 0x564b1e177c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.221+0000 7f72aba64740 1 -- 172.21.15.19:0/1279883884 >> 172.21.15.19:0/1279883884 conn(0x564b1e11d830 msgr2=0x564b1e146140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.221+0000 7f72aba64740 1 -- 172.21.15.19:0/1279883884 shutdown_connections 2024-04-25T10:04:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.221+0000 7f72aba64740 1 -- 172.21.15.19:0/1279883884 wait complete. 2024-04-25T10:04:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.221+0000 7f72aba64740 1 -- start start 2024-04-25T10:04:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.221+0000 7f72aba64740 1 Processor -- start 2024-04-25T10:04:26.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72aba64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e121cf0 0x564b1e1bc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72aba64740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 0x564b1e16c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72aba64740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e13e5e0 0x564b1e13d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72aba64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1e13ec60 con 0x564b1e121cf0 2024-04-25T10:04:26.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72aba64740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40cfd90 con 0x564b1e122bb0 2024-04-25T10:04:26.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72aba64740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1e1455d0 con 0x564b1e13e5e0 2024-04-25T10:04:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72aba64740 10 client.0 _ll_register_callbacks cb 0x7f7294009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e121cf0 0x564b1e1bc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72a259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e121cf0 0x564b1e1bc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57218/0 (socket says 172.21.15.19:57218) 2024-04-25T10:04:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72a259c640 1 -- 172.21.15.19:0/1912192087 learned_addr learned my addr 172.21.15.19:0/1912192087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f729bfff640 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 0x564b1e16c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72a2d9d640 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e13e5e0 0x564b1e13d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72a259c640 1 -- 172.21.15.19:0/1912192087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e13e5e0 msgr2=0x564b1e13d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72a259c640 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e13e5e0 0x564b1e13d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72a259c640 1 -- 172.21.15.19:0/1912192087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 msgr2=0x564b1e16c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72a259c640 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 0x564b1e16c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.222+0000 7f72a259c640 1 -- 172.21.15.19:0/1912192087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b1e13c2e0 con 0x564b1e121cf0 2024-04-25T10:04:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72a259c640 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e121cf0 0x564b1e1bc4c0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f7294032fd0 tx=0x7f72940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6146c87c570f13de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72817fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:04:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72817fa640 1 -- 172.21.15.19:0/1912192087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729404d040 con 0x564b1e121cf0 2024-04-25T10:04:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72817fa640 1 -- 172.21.15.19:0/1912192087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72940040b0 con 0x564b1e121cf0 2024-04-25T10:04:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72aba64740 1 -- 172.21.15.19:0/1912192087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f729c0077c0 con 0x564b1e121cf0 2024-04-25T10:04:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72817fa640 1 -- 172.21.15.19:0/1912192087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7294061050 con 0x564b1e121cf0 2024-04-25T10:04:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72aba64740 10 client.10911 fetch_fsmap learned FSMap version 109 2024-04-25T10:04:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72aba64740 1 -- 172.21.15.19:0/1912192087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x564b1e123450 con 0x564b1e121cf0 2024-04-25T10:04:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72817fa640 1 -- 172.21.15.19:0/1912192087 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c0077c0 con 0x564b1e121cf0 2024-04-25T10:04:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72817fa640 1 -- 172.21.15.19:0/1912192087 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 109) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7294064020 con 0x564b1e121cf0 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72aba64740 10 client.10911 fetch_fsmap finished waiting for FSMap version 109 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72aba64740 10 client.10911 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72aba64740 20 client.10911 populate_metadata read hostname 'smithi019' 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72aba64740 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x564b1e13dab0 0x564b1e14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72aba64740 4 client.10911 mds_command: new command op to 10698 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f72aba64740 1 -- 172.21.15.19:0/1912192087 --> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x564b1e14df30 con 0x564b1e13dab0 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.223+0000 7f729bfff640 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x564b1e13dab0 0x564b1e14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.224+0000 7f72817fa640 10 client.10911 ms_handle_connect on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.224+0000 7f729bfff640 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x564b1e13dab0 0x564b1e14c9d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac5bfd62e580adf server_cookie=5228eee7b216e04a in_seq=0 out_seq=0 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.224+0000 7f72817fa640 1 -- 172.21.15.19:0/1912192087 <== mds.0 v2:172.21.15.19:6832/3942829201 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564b1e14df30 con 0x564b1e13dab0 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.224+0000 7f72817fa640 10 client.10911 handle_command_reply: tid=0 2024-04-25T10:04:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.224+0000 7f72aba64740 1 client.10911 shutdown 2024-04-25T10:04:26.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72aba64740 1 -- 172.21.15.19:0/1912192087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e121cf0 msgr2=0x564b1e1bc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72aba64740 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e121cf0 0x564b1e1bc4c0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f7294032fd0 tx=0x7f72940046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72aba64740 1 -- 172.21.15.19:0/1912192087 shutdown_connections 2024-04-25T10:04:26.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72aba64740 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x564b1e13dab0 0x564b1e14c9d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.238 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T10:04:26.225+0000 7ffb4f703640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:04:25.199872+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={9373=0-12582912@1} caps={9373=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=9373 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x559473591600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(26)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:04:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72aba64740 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564b1e13e5e0 0x564b1e13d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72aba64740 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b1e122bb0 0x564b1e16c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72aba64740 1 --2- 172.21.15.19:0/1912192087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564b1e121cf0 0x564b1e1bc4c0 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72817fa640 0 client.10911 ms_handle_reset on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72aba64740 1 -- 172.21.15.19:0/1912192087 >> 172.21.15.19:0/1912192087 conn(0x564b1e11d830 msgr2=0x564b1e169310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72aba64740 1 -- 172.21.15.19:0/1912192087 shutdown_connections 2024-04-25T10:04:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72aba64740 1 -- 172.21.15.19:0/1912192087 wait complete. 2024-04-25T10:04:26.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72aba64740 20 client.10911 trim_cache size 0 max 16384 2024-04-25T10:04:26.240 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:04:26.240 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:04:26.240 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "75988a3d-9d2a-4a59-90d2-86be625cf529", 2024-04-25T10:04:26.240 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:04:26.240 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:04:26.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72a8d5c640 1 -- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728406b4d0 msgr2=0x7f728406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72a8d5c640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728406b4d0 0x7f728406d990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7294034000 tx=0x7f729402e000 comp rx=0 tx=0).stop 2024-04-25T10:04:26.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72a8d5c640 1 -- 172.21.15.19:0/3127463466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a40cdb20 msgr2=0x7f72a40ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72a8d5c640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a40cdb20 0x7f72a40ade70 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c430 tx=0x7f728c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72a8d5c640 1 -- 172.21.15.19:0/3127463466 shutdown_connections 2024-04-25T10:04:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72a8d5c640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f728406b4d0 0x7f728406d990 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72a8d5c640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72a415b980 0x7f72a414c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72a8d5c640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72a40cdb20 0x7f72a40ade70 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72a8d5c640 1 --2- 172.21.15.19:0/3127463466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72a40cc4f0 0x7f72a41f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.225+0000 7f72a8d5c640 1 -- 172.21.15.19:0/3127463466 >> 172.21.15.19:0/3127463466 conn(0x7f72a406abe0 msgr2=0x7f72a4159590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.226+0000 7f72a8d5c640 1 -- 172.21.15.19:0/3127463466 shutdown_connections 2024-04-25T10:04:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.226+0000 7f72a8d5c640 1 -- 172.21.15.19:0/3127463466 wait complete. 2024-04-25T10:04:26.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:26.243 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:04:26.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.340+0000 7f932f7e7640 1 Processor -- start 2024-04-25T10:04:26.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.340+0000 7f932f7e7640 1 -- start start 2024-04-25T10:04:26.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.341+0000 7f932f7e7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93280c6010 0x7f93280ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.341+0000 7f932f7e7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280c6940 con 0x7f93280cd520 2024-04-25T10:04:26.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.341+0000 7f932f7e7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932815add0 con 0x7f932815a9f0 2024-04-25T10:04:26.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.341+0000 7f932f7e7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93280b0020 con 0x7f93280c6010 2024-04-25T10:04:26.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.341+0000 7f932dd84640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f93280cd520 0x7f93280cba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57618/0 (socket says 172.21.15.19:57618) 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.341+0000 7f932dd84640 1 -- 172.21.15.19:0/2891468987 learned_addr learned my addr 172.21.15.19:0/2891468987 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f932cd82640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93280c6010 0x7f93280ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387184201 0 0) 0x7f9324001490 con 0x7f93280cd520 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9314003520 con 0x7f93280cd520 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1125460848 0 0) 0x7f9314003520 con 0x7f93280cd520 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93280c6010 msgr2=0x7f93280ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93280c6010 0x7f93280ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 >> v1:172.21.15.112:6789/0 conn(0x7f932815a9f0 legacy=0x7f93280cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280c6660 con 0x7f93280cd520 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f9324001bc0 con 0x7f93280cd520 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 >> v1:172.21.15.19:6789/0 conn(0x7f93280cd520 legacy=0x7f93280cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93280c6010 0x7f93140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9314003f20 0x7f9314004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9314004d40 0x7f93140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931400b9f0 con 0x7f9314003f20 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931400bba0 con 0x7f9314004d40 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931400bda0 con 0x7f93280c6010 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f932cd82640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9314003f20 0x7f9314004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f932d583640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93280c6010 0x7f93140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.342+0000 7f932dd84640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9314004d40 0x7f93140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.343+0000 7f932cd82640 1 -- 172.21.15.19:0/2891468987 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93280c6010 msgr2=0x7f93140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.343+0000 7f932cd82640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93280c6010 0x7f93140039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.343+0000 7f932cd82640 1 -- 172.21.15.19:0/2891468987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9314004d40 msgr2=0x7f93140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.343+0000 7f932cd82640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9314004d40 0x7f93140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.343+0000 7f932cd82640 1 -- 172.21.15.19:0/2891468987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314003520 con 0x7f9314003f20 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.343+0000 7f932cd82640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9314003f20 0x7f9314004800 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f9318009aa0 tx=0x7f9314005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb929185d8d9040a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.343+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318036070 con 0x7f9314003f20 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.343+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9318031400 con 0x7f9314003f20 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.343+0000 7f931ffff640 1 -- 172.21.15.19:0/2891468987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318009670 con 0x7f9314003f20 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.344+0000 7f932f7e7640 1 -- 172.21.15.19:0/2891468987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9314003f20 msgr2=0x7f9314004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.344+0000 7f932f7e7640 1 --2- 172.21.15.19:0/2891468987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9314003f20 0x7f9314004800 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f9318009aa0 tx=0x7f9314005830 comp rx=0 tx=0).stop 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.344+0000 7f932dd84640 1 -- 172.21.15.19:0/2891468987 reap_dead start 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.344+0000 7f932f7e7640 1 -- 172.21.15.19:0/2891468987 shutdown_connections 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.344+0000 7f932f7e7640 1 -- 172.21.15.19:0/2891468987 >> 172.21.15.19:0/2891468987 conn(0x7f93280b2390 msgr2=0x7f93280b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.344+0000 7f932f7e7640 1 -- 172.21.15.19:0/2891468987 shutdown_connections 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.344+0000 7f932f7e7640 1 -- 172.21.15.19:0/2891468987 wait complete. 2024-04-25T10:04:26.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.345+0000 7f932f7e7640 1 Processor -- start 2024-04-25T10:04:26.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.345+0000 7f932f7e7640 1 -- start start 2024-04-25T10:04:26.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932f7e7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9314003f20 0x7f932815c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932f7e7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93280b4400 0x7f9328155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932f7e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93280b3510 0x7f93280b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932f7e7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280cb230 con 0x7f93280b4400 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932f7e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280cb5a0 con 0x7f93280b3510 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932f7e7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93280ccfe0 con 0x7f9314003f20 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932cd82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93280b4400 0x7f9328155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932cd82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93280b4400 0x7f9328155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57246/0 (socket says 172.21.15.19:57246) 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932cd82640 1 -- 172.21.15.19:0/536536096 learned_addr learned my addr 172.21.15.19:0/536536096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932d583640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9314003f20 0x7f932815c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932dd84640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93280b3510 0x7f93280b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932cd82640 1 -- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9314003f20 msgr2=0x7f932815c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932cd82640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9314003f20 0x7f932815c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932cd82640 1 -- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93280b3510 msgr2=0x7f93280b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932cd82640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93280b3510 0x7f93280b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.346+0000 7f932cd82640 1 -- 172.21.15.19:0/536536096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280b4910 con 0x7f93280b4400 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.347+0000 7f932cd82640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93280b4400 0x7f9328155430 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f9318009b00 tx=0x7f9318004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.347+0000 7f931dffb640 1 -- 172.21.15.19:0/536536096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318041070 con 0x7f93280b4400 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.347+0000 7f931dffb640 1 -- 172.21.15.19:0/536536096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931803d680 con 0x7f93280b4400 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.347+0000 7f932f7e7640 1 -- 172.21.15.19:0/536536096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93280ad7a0 con 0x7f93280b4400 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.347+0000 7f931dffb640 1 -- 172.21.15.19:0/536536096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318036040 con 0x7f93280b4400 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.347+0000 7f932f7e7640 1 -- 172.21.15.19:0/536536096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9328155970 con 0x7f93280b4400 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.348+0000 7f931dffb640 1 -- 172.21.15.19:0/536536096 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9318044070 con 0x7f93280b4400 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.349+0000 7f931dffb640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f930c06b4d0 0x7f930c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.349+0000 7f931dffb640 1 -- 172.21.15.19:0/536536096 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f93180aa7f0 con 0x7f93280b4400 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.349+0000 7f932d583640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f930c06b4d0 0x7f930c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.350+0000 7f932d583640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f930c06b4d0 0x7f930c06d990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f93280b2360 tx=0x7f9320009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.350+0000 7f93325c8740 1 Processor -- start 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.350+0000 7f93325c8740 1 -- start start 2024-04-25T10:04:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f93325c8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbba9a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f93325c8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbba6a770 0x556fbba707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f93325c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbbab6470 0x556fbbab68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f93325c8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbb9d3da0 con 0x556fbba9a510 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f93325c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbb97cb80 con 0x556fbbab6470 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f93325c8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x556fbb9cfe40 con 0x556fbba6a770 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f932d583640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbba9a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f932d583640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbba9a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57256/0 (socket says 172.21.15.19:57256) 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f932d583640 1 -- 172.21.15.19:0/2109922209 learned_addr learned my addr 172.21.15.19:0/2109922209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f932cd82640 1 --2- 172.21.15.19:0/2109922209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbba6a770 0x556fbba707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f932dd84640 1 --2- 172.21.15.19:0/2109922209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbbab6470 0x556fbbab68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f932d583640 1 -- 172.21.15.19:0/2109922209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbba6a770 msgr2=0x556fbba707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f932d583640 1 --2- 172.21.15.19:0/2109922209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbba6a770 0x556fbba707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f932d583640 1 -- 172.21.15.19:0/2109922209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbbab6470 msgr2=0x556fbbab68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f932d583640 1 --2- 172.21.15.19:0/2109922209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbbab6470 0x556fbbab68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f932d583640 1 -- 172.21.15.19:0/2109922209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fbb986f10 con 0x556fbba9a510 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.351+0000 7f932d583640 1 --2- 172.21.15.19:0/2109922209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbba9a8f0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f9320012d00 tx=0x7f9320015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fff75f03933f48f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f9312ffd640 1 -- 172.21.15.19:0/2109922209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320019070 con 0x556fbba9a510 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f9312ffd640 1 -- 172.21.15.19:0/2109922209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9320026070 con 0x556fbba9a510 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f9312ffd640 1 -- 172.21.15.19:0/2109922209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320015dd0 con 0x556fbba9a510 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f93325c8740 1 -- 172.21.15.19:0/2109922209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 msgr2=0x556fbba9a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f93325c8740 1 --2- 172.21.15.19:0/2109922209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbba9a8f0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f9320012d00 tx=0x7f9320015000 comp rx=0 tx=0).stop 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f93325c8740 1 -- 172.21.15.19:0/2109922209 shutdown_connections 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f93325c8740 1 --2- 172.21.15.19:0/2109922209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbbab6470 0x556fbbab68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f93325c8740 1 --2- 172.21.15.19:0/2109922209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbba6a770 0x556fbba707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f93325c8740 1 --2- 172.21.15.19:0/2109922209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbba9a8f0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f93325c8740 1 -- 172.21.15.19:0/2109922209 >> 172.21.15.19:0/2109922209 conn(0x556fbba6c9e0 msgr2=0x556fbbab5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f93325c8740 1 -- 172.21.15.19:0/2109922209 shutdown_connections 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.352+0000 7f93325c8740 1 -- 172.21.15.19:0/2109922209 wait complete. 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.353+0000 7f93325c8740 1 -- start start 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.353+0000 7f93325c8740 1 Processor -- start 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f93325c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbba9a510 0x556fbba938c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f93325c8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab6470 0x556fbbab99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f93325c8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab68f0 0x556fbbab6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f93325c8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbba70ce0 con 0x556fbbab68f0 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f93325c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbb9724b0 con 0x556fbba9a510 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f93325c8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x556fbba70310 con 0x556fbbab6470 2024-04-25T10:04:26.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f93325c8740 10 client.0 _ll_register_callbacks cb 0x556fbb9b2180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f932dd84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab68f0 0x556fbbab6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f932dd84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab68f0 0x556fbbab6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57268/0 (socket says 172.21.15.19:57268) 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f932dd84640 1 -- 172.21.15.19:0/534001841 learned_addr learned my addr 172.21.15.19:0/534001841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f932d583640 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbba9a510 0x556fbba938c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f932cd82640 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab6470 0x556fbbab99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f932dd84640 1 -- 172.21.15.19:0/534001841 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab6470 msgr2=0x556fbbab99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f932dd84640 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab6470 0x556fbbab99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f932dd84640 1 -- 172.21.15.19:0/534001841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbba9a510 msgr2=0x556fbba938c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f932dd84640 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbba9a510 0x556fbba938c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f932dd84640 1 -- 172.21.15.19:0/534001841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fbb7b27d0 con 0x556fbbab68f0 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f932dd84640 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab68f0 0x556fbbab6d70 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f9328062550 tx=0x7f932400a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329878a325ceff50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f9307fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f9307fff640 1 -- 172.21.15.19:0/534001841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324007340 con 0x556fbbab68f0 2024-04-25T10:04:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f9307fff640 1 -- 172.21.15.19:0/534001841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f93240074e0 con 0x556fbbab68f0 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.354+0000 7f93325c8740 1 -- 172.21.15.19:0/534001841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9320026070 con 0x556fbbab68f0 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f9307fff640 1 -- 172.21.15.19:0/534001841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9324024050 con 0x556fbbab68f0 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f93325c8740 10 client.10923 fetch_fsmap learned FSMap version 109 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f93325c8740 1 -- 172.21.15.19:0/534001841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x556fbb6e83f0 con 0x556fbbab68f0 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f9307fff640 1 -- 172.21.15.19:0/534001841 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9320026070 con 0x556fbbab68f0 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f9307fff640 1 -- 172.21.15.19:0/534001841 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 109) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9324027020 con 0x556fbbab68f0 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f93325c8740 10 client.10923 fetch_fsmap finished waiting for FSMap version 109 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f93325c8740 10 client.10923 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f93325c8740 20 client.10923 populate_metadata read hostname 'smithi019' 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f93325c8740 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x556fbba9cab0 0x556fbba9cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f93325c8740 4 client.10923 mds_command: new command op to 10698 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f93325c8740 1 -- 172.21.15.19:0/534001841 --> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fbb9e12e0 con 0x556fbba9cab0 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f932d583640 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x556fbba9cab0 0x556fbba9cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f932d583640 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x556fbba9cab0 0x556fbba9cee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79fdc95821664276 server_cookie=a809656f61a21a49 in_seq=0 out_seq=0 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.355+0000 7f9307fff640 10 client.10923 ms_handle_connect on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f9307fff640 1 -- 172.21.15.19:0/534001841 <== mds.0 v2:172.21.15.19:6832/3942829201 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556fbb9e12e0 con 0x556fbba9cab0 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f9307fff640 10 client.10923 handle_command_reply: tid=0 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 1 client.10923 shutdown 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 1 -- 172.21.15.19:0/534001841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab68f0 msgr2=0x556fbbab6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab68f0 0x556fbbab6d70 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f9328062550 tx=0x7f932400a850 comp rx=0 tx=0).stop 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 1 -- 172.21.15.19:0/534001841 shutdown_connections 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x556fbba9cab0 0x556fbba9cee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab68f0 0x556fbbab6d70 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab6470 0x556fbbab99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f9307fff640 0 client.10923 ms_handle_reset on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 1 --2- 172.21.15.19:0/534001841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbba9a510 0x556fbba938c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 1 -- 172.21.15.19:0/534001841 >> 172.21.15.19:0/534001841 conn(0x556fbba6a770 msgr2=0x556fbbab8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 1 -- 172.21.15.19:0/534001841 shutdown_connections 2024-04-25T10:04:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 1 -- 172.21.15.19:0/534001841 wait complete. 2024-04-25T10:04:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.356+0000 7f93325c8740 20 client.10923 trim_cache size 0 max 16384 2024-04-25T10:04:26.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.371+0000 7f93325c8740 1 Processor -- start 2024-04-25T10:04:26.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.371+0000 7f93325c8740 1 -- start start 2024-04-25T10:04:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.372+0000 7f93325c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbba9a510 0x556fbba71330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.372+0000 7f93325c8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab6470 0x556fbbab8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.372+0000 7f93325c8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab68f0 0x556fbbab8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.372+0000 7f93325c8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281fc770 con 0x556fbbab6470 2024-04-25T10:04:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.372+0000 7f93325c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281fbad0 con 0x556fbba9a510 2024-04-25T10:04:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.372+0000 7f93325c8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932806b870 con 0x556fbbab68f0 2024-04-25T10:04:26.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.372+0000 7f932cd82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab6470 0x556fbbab8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.372+0000 7f932cd82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab6470 0x556fbbab8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57274/0 (socket says 172.21.15.19:57274) 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.372+0000 7f932cd82640 1 -- 172.21.15.19:0/3361869949 learned_addr learned my addr 172.21.15.19:0/3361869949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.372+0000 7f932dd84640 1 --2- 172.21.15.19:0/3361869949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab68f0 0x556fbbab8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.372+0000 7f932d583640 1 --2- 172.21.15.19:0/3361869949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbba9a510 0x556fbba71330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f932cd82640 1 -- 172.21.15.19:0/3361869949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab68f0 msgr2=0x556fbbab8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f932cd82640 1 --2- 172.21.15.19:0/3361869949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab68f0 0x556fbbab8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f932cd82640 1 -- 172.21.15.19:0/3361869949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbba9a510 msgr2=0x556fbba71330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f932cd82640 1 --2- 172.21.15.19:0/3361869949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbba9a510 0x556fbba71330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f932cd82640 1 -- 172.21.15.19:0/3361869949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fbb6a80f0 con 0x556fbbab6470 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f932cd82640 1 --2- 172.21.15.19:0/3361869949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab6470 0x556fbbab8830 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f9318075380 tx=0x7f93180aa040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22baf977b7b8c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93127fc640 1 -- 172.21.15.19:0/3361869949 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318009670 con 0x556fbbab6470 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93127fc640 1 -- 172.21.15.19:0/3361869949 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9318075590 con 0x556fbbab6470 2024-04-25T10:04:26.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93127fc640 1 -- 172.21.15.19:0/3361869949 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93180810f0 con 0x556fbbab6470 2024-04-25T10:04:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93325c8740 1 -- 172.21.15.19:0/3361869949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab6470 msgr2=0x556fbbab8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93325c8740 1 --2- 172.21.15.19:0/3361869949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab6470 0x556fbbab8830 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f9318075380 tx=0x7f93180aa040 comp rx=0 tx=0).stop 2024-04-25T10:04:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93325c8740 1 -- 172.21.15.19:0/3361869949 shutdown_connections 2024-04-25T10:04:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93325c8740 1 --2- 172.21.15.19:0/3361869949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab68f0 0x556fbbab8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93325c8740 1 --2- 172.21.15.19:0/3361869949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbbab6470 0x556fbbab8830 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93325c8740 1 --2- 172.21.15.19:0/3361869949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbba9a510 0x556fbba71330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93325c8740 1 -- 172.21.15.19:0/3361869949 >> 172.21.15.19:0/3361869949 conn(0x556fbba6a770 msgr2=0x556fbbab9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93325c8740 1 -- 172.21.15.19:0/3361869949 shutdown_connections 2024-04-25T10:04:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.373+0000 7f93325c8740 1 -- 172.21.15.19:0/3361869949 wait complete. 2024-04-25T10:04:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.374+0000 7f93325c8740 1 -- start start 2024-04-25T10:04:26.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.374+0000 7f93325c8740 1 Processor -- start 2024-04-25T10:04:26.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f93325c8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbbb23170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f93325c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbbab6470 0x556fbba6f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f93325c8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab68f0 0x556fbbab4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f93325c8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbbacd4c0 con 0x556fbba9a510 2024-04-25T10:04:26.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f93325c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281f4020 con 0x556fbbab6470 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f93325c8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x556fbba69820 con 0x556fbbab68f0 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f93325c8740 10 client.0 _ll_register_callbacks cb 0x7f9320001720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932d583640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbbb23170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932d583640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbbb23170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57278/0 (socket says 172.21.15.19:57278) 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932d583640 1 -- 172.21.15.19:0/887920956 learned_addr learned my addr 172.21.15.19:0/887920956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932cd82640 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbbab6470 0x556fbba6f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932dd84640 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab68f0 0x556fbbab4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932d583640 1 -- 172.21.15.19:0/887920956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab68f0 msgr2=0x556fbbab4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932d583640 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab68f0 0x556fbbab4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932d583640 1 -- 172.21.15.19:0/887920956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbbab6470 msgr2=0x556fbba6f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932d583640 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbbab6470 0x556fbba6f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932d583640 1 -- 172.21.15.19:0/887920956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fbba94b10 con 0x556fbba9a510 2024-04-25T10:04:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932cd82640 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbbab6470 0x556fbba6f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:26.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.375+0000 7f932dd84640 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab68f0 0x556fbbab4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:26.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f932d583640 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbbb23170 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f9320012690 tx=0x7f9320001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7294d988ca59ba61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f9307fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:04:26.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f9307fff640 1 -- 172.21.15.19:0/887920956 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320019040 con 0x556fbba9a510 2024-04-25T10:04:26.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f9307fff640 1 -- 172.21.15.19:0/887920956 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9320036740 con 0x556fbba9a510 2024-04-25T10:04:26.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f93325c8740 1 -- 172.21.15.19:0/887920956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9318075590 con 0x556fbba9a510 2024-04-25T10:04:26.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f9307fff640 1 -- 172.21.15.19:0/887920956 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9320047050 con 0x556fbba9a510 2024-04-25T10:04:26.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f93325c8740 10 client.10929 fetch_fsmap learned FSMap version 109 2024-04-25T10:04:26.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f93325c8740 1 -- 172.21.15.19:0/887920956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x556fbba6edd0 con 0x556fbba9a510 2024-04-25T10:04:26.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f9307fff640 1 -- 172.21.15.19:0/887920956 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9318075590 con 0x556fbba9a510 2024-04-25T10:04:26.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f9307fff640 1 -- 172.21.15.19:0/887920956 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 109) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9324001a40 con 0x556fbba9a510 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f93325c8740 10 client.10929 fetch_fsmap finished waiting for FSMap version 109 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f93325c8740 10 client.10929 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f93325c8740 20 client.10929 populate_metadata read hostname 'smithi019' 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f93325c8740 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x556fbba6b540 0x556fbba712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f93325c8740 4 client.10929 mds_command: new command op to 10698 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f93325c8740 1 -- 172.21.15.19:0/887920956 --> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556fbba9c150 con 0x556fbba6b540 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.376+0000 7f932cd82640 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x556fbba6b540 0x556fbba712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f932cd82640 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x556fbba6b540 0x556fbba712a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e549c97709927d5 server_cookie=d4a945511769e70a in_seq=0 out_seq=0 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f9307fff640 10 client.10929 ms_handle_connect on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f9307fff640 1 -- 172.21.15.19:0/887920956 <== mds.0 v2:172.21.15.19:6832/3942829201 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x556fbba9c150 con 0x556fbba6b540 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f9307fff640 10 client.10929 handle_command_reply: tid=0 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 1 client.10929 shutdown 2024-04-25T10:04:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 1 -- 172.21.15.19:0/887920956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 msgr2=0x556fbbb23170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbbb23170 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f9320012690 tx=0x7f9320001230 comp rx=0 tx=0).stop 2024-04-25T10:04:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 1 -- 172.21.15.19:0/887920956 shutdown_connections 2024-04-25T10:04:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x556fbba6b540 0x556fbba712a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556fbbab68f0 0x556fbbab4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fbbab6470 0x556fbba6f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f9307fff640 0 client.10929 ms_handle_reset on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 1 --2- 172.21.15.19:0/887920956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fbba9a510 0x556fbbb23170 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 1 -- 172.21.15.19:0/887920956 >> 172.21.15.19:0/887920956 conn(0x556fbba6a770 msgr2=0x556fbbab8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 1 -- 172.21.15.19:0/887920956 shutdown_connections 2024-04-25T10:04:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 1 -- 172.21.15.19:0/887920956 wait complete. 2024-04-25T10:04:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.377+0000 7f93325c8740 20 client.10929 trim_cache size 0 max 16384 2024-04-25T10:04:26.391 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:04:26.391 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:04:26.391 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:04:26.391 INFO:teuthology.orchestra.run.smithi019.stdout: "75988a3d-9d2a-4a59-90d2-86be625cf529": { 2024-04-25T10:04:26.391 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:04:26.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "75988a3d-9d2a-4a59-90d2-86be625cf529", 2024-04-25T10:04:26.391 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,repair" 2024-04-25T10:04:26.391 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:26.391 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:26.391 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:04:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 -- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f930c06b4d0 msgr2=0x7f930c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f930c06b4d0 0x7f930c06d990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f93280b2360 tx=0x7f9320009040 comp rx=0 tx=0).stop 2024-04-25T10:04:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 -- 172.21.15.19:0/536536096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93280b4400 msgr2=0x7f9328155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93280b4400 0x7f9328155430 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f9318009b00 tx=0x7f9318004730 comp rx=0 tx=0).stop 2024-04-25T10:04:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 -- 172.21.15.19:0/536536096 shutdown_connections 2024-04-25T10:04:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f930c06b4d0 0x7f930c06d990 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93280b3510 0x7f93280b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93280b4400 0x7f9328155430 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 --2- 172.21.15.19:0/536536096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9314003f20 0x7f932815c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 -- 172.21.15.19:0/536536096 >> 172.21.15.19:0/536536096 conn(0x7f93280b2390 msgr2=0x7f9328157bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 -- 172.21.15.19:0/536536096 shutdown_connections 2024-04-25T10:04:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:26.378+0000 7f932f7e7640 1 -- 172.21.15.19:0/536536096 wait complete. 2024-04-25T10:04:26.395 INFO:tasks.cephfs.filesystem:scrub status for tag:75988a3d-9d2a-4a59-90d2-86be625cf529 - {'path': '/', 'tag': '75988a3d-9d2a-4a59-90d2-86be625cf529', 'options': 'recursive,repair'} 2024-04-25T10:04:39.121 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:39.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:56.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:56.396 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:04:56.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.508+0000 7f7553930640 1 Processor -- start 2024-04-25T10:04:56.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.508+0000 7f7553930640 1 -- start start 2024-04-25T10:04:56.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.509+0000 7f7553930640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f754c0ced80 0x7f754c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.509+0000 7f7553930640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c15e000 con 0x7f754c161710 2024-04-25T10:04:56.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.509+0000 7f7553930640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c0d1de0 con 0x7f754c15dc20 2024-04-25T10:04:56.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.509+0000 7f7553930640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754c161c60 con 0x7f754c0ced80 2024-04-25T10:04:56.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.509+0000 7f7551ecd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f754c161710 0x7f754c0d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40738/0 (socket says 172.21.15.19:40738) 2024-04-25T10:04:56.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.509+0000 7f7550ecb640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f754c0ced80 0x7f754c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.509+0000 7f7551ecd640 1 -- 172.21.15.19:0/1521321923 learned_addr learned my addr 172.21.15.19:0/1521321923 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:56.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.509+0000 7f7550ecb640 1 -- 172.21.15.19:0/1521321923 >> v1:172.21.15.112:6789/0 conn(0x7f754c15dc20 legacy=0x7f754c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.509+0000 7f7550ecb640 1 -- 172.21.15.19:0/1521321923 >> v1:172.21.15.19:6789/0 conn(0x7f754c161710 legacy=0x7f754c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.509+0000 7f7543fff640 1 -- 172.21.15.19:0/1521321923 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102510371 0 0) 0x7f754c15e000 con 0x7f754c161710 2024-04-25T10:04:56.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.509+0000 7f7550ecb640 1 -- 172.21.15.19:0/1521321923 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c161f40 con 0x7f754c0ced80 2024-04-25T10:04:56.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.510+0000 7f7550ecb640 1 --2- 172.21.15.19:0/1521321923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0ced80 0x7f754c0d22e0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f753c0097a0 tx=0x7f753c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3fcf107b679b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.510+0000 7f7543fff640 1 -- 172.21.15.19:0/1521321923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c037070 con 0x7f754c0ced80 2024-04-25T10:04:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.510+0000 7f7543fff640 1 -- 172.21.15.19:0/1521321923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c032440 con 0x7f754c0ced80 2024-04-25T10:04:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.510+0000 7f7543fff640 1 -- 172.21.15.19:0/1521321923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c032810 con 0x7f754c0ced80 2024-04-25T10:04:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.511+0000 7f7553930640 1 -- 172.21.15.19:0/1521321923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0ced80 msgr2=0x7f754c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.511+0000 7f7553930640 1 --2- 172.21.15.19:0/1521321923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0ced80 0x7f754c0d22e0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f753c0097a0 tx=0x7f753c02a230 comp rx=0 tx=0).stop 2024-04-25T10:04:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.511+0000 7f7553930640 1 -- 172.21.15.19:0/1521321923 shutdown_connections 2024-04-25T10:04:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.511+0000 7f7553930640 1 --2- 172.21.15.19:0/1521321923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0ced80 0x7f754c0d22e0 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.511+0000 7f7553930640 1 -- 172.21.15.19:0/1521321923 >> 172.21.15.19:0/1521321923 conn(0x7f754c0d0420 msgr2=0x7f754c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.511+0000 7f7553930640 1 -- 172.21.15.19:0/1521321923 shutdown_connections 2024-04-25T10:04:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.511+0000 7f7553930640 1 -- 172.21.15.19:0/1521321923 wait complete. 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.512+0000 7f7553930640 1 Processor -- start 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.512+0000 7f7553930640 1 -- start start 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f7553930640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0ced80 0x7f754c154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f7553930640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f754c15dc20 0x7f754c0c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f7553930640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f754c161710 0x7f754c15f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f7553930640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0b5e50 con 0x7f754c0ced80 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f7553930640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c162500 con 0x7f754c15dc20 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f7553930640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0b5fd0 con 0x7f754c161710 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f75516cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0ced80 0x7f754c154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f75516cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0ced80 0x7f754c154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55304/0 (socket says 172.21.15.19:55304) 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f75516cc640 1 -- 172.21.15.19:0/2288668564 learned_addr learned my addr 172.21.15.19:0/2288668564 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f75516cc640 1 -- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f754c161710 msgr2=0x7f754c15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f75516cc640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f754c161710 0x7f754c15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f75516cc640 1 -- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f754c15dc20 msgr2=0x7f754c0c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f75516cc640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f754c15dc20 0x7f754c0c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.513+0000 7f75516cc640 1 -- 172.21.15.19:0/2288668564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0c7c00 con 0x7f754c0ced80 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.514+0000 7f75516cc640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0ced80 0x7f754c154400 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f754800c450 tx=0x7f754800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.514+0000 7f7541ffb640 1 -- 172.21.15.19:0/2288668564 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75480052d0 con 0x7f754c0ced80 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.514+0000 7f7541ffb640 1 -- 172.21.15.19:0/2288668564 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548005470 con 0x7f754c0ced80 2024-04-25T10:04:56.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.514+0000 7f7553930640 1 -- 172.21.15.19:0/2288668564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c0c7ee0 con 0x7f754c0ced80 2024-04-25T10:04:56.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.514+0000 7f7541ffb640 1 -- 172.21.15.19:0/2288668564 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548012870 con 0x7f754c0ced80 2024-04-25T10:04:56.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.514+0000 7f7553930640 1 -- 172.21.15.19:0/2288668564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c0c8710 con 0x7f754c0ced80 2024-04-25T10:04:56.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.515+0000 7f7541ffb640 1 -- 172.21.15.19:0/2288668564 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7548012a10 con 0x7f754c0ced80 2024-04-25T10:04:56.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.516+0000 7f7541ffb640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f753006b500 0x7f753006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.516+0000 7f7541ffb640 1 -- 172.21.15.19:0/2288668564 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f7548089f70 con 0x7f754c0ced80 2024-04-25T10:04:56.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.516+0000 7f7550ecb640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f753006b500 0x7f753006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.517+0000 7f7550ecb640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f753006b500 0x7f753006d9c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f753c034000 tx=0x7f753c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.517+0000 7f7556632740 1 Processor -- start 2024-04-25T10:04:56.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.517+0000 7f7556632740 1 -- start start 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7556632740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e2db10 0x555624e30320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7556632740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2def0 0x555624e30860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7556632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e00770 0x555624e03f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7556632740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x555624d69da0 con 0x555624e2def0 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7556632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x555624d12b80 con 0x555624e00770 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7556632740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x555624d65e40 con 0x555624e2db10 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7550ecb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2def0 0x555624e30860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7550ecb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2def0 0x555624e30860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55314/0 (socket says 172.21.15.19:55314) 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7550ecb640 1 -- 172.21.15.19:0/3476022824 learned_addr learned my addr 172.21.15.19:0/3476022824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f75516cc640 1 --2- 172.21.15.19:0/3476022824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e2db10 0x555624e30320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7551ecd640 1 --2- 172.21.15.19:0/3476022824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e00770 0x555624e03f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7550ecb640 1 -- 172.21.15.19:0/3476022824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e2db10 msgr2=0x555624e30320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7550ecb640 1 --2- 172.21.15.19:0/3476022824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e2db10 0x555624e30320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7550ecb640 1 -- 172.21.15.19:0/3476022824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e00770 msgr2=0x555624e03f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7550ecb640 1 --2- 172.21.15.19:0/3476022824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e00770 0x555624e03f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.518+0000 7f7550ecb640 1 -- 172.21.15.19:0/3476022824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555624d1cf10 con 0x555624e2def0 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.519+0000 7f7550ecb640 1 --2- 172.21.15.19:0/3476022824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2def0 0x555624e30860 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f753c009560 tx=0x7f753c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297aea356361c1d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.519+0000 7f7536ffd640 1 -- 172.21.15.19:0/3476022824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c037040 con 0x555624e2def0 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.519+0000 7f7536ffd640 1 -- 172.21.15.19:0/3476022824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c04c070 con 0x555624e2def0 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.519+0000 7f7536ffd640 1 -- 172.21.15.19:0/3476022824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c032d60 con 0x555624e2def0 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.519+0000 7f7556632740 1 -- 172.21.15.19:0/3476022824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2def0 msgr2=0x555624e30860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.519+0000 7f7556632740 1 --2- 172.21.15.19:0/3476022824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2def0 0x555624e30860 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f753c009560 tx=0x7f753c008030 comp rx=0 tx=0).stop 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.519+0000 7f7556632740 1 -- 172.21.15.19:0/3476022824 shutdown_connections 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.520+0000 7f7556632740 1 --2- 172.21.15.19:0/3476022824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e00770 0x555624e03f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.520+0000 7f7556632740 1 --2- 172.21.15.19:0/3476022824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2def0 0x555624e30860 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.520+0000 7f7556632740 1 --2- 172.21.15.19:0/3476022824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e2db10 0x555624e30320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.520+0000 7f7556632740 1 -- 172.21.15.19:0/3476022824 >> 172.21.15.19:0/3476022824 conn(0x555624e029e0 msgr2=0x555624e4b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.520+0000 7f7556632740 1 -- 172.21.15.19:0/3476022824 shutdown_connections 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.520+0000 7f7556632740 1 -- 172.21.15.19:0/3476022824 wait complete. 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.520+0000 7f7556632740 1 -- start start 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.520+0000 7f7556632740 1 Processor -- start 2024-04-25T10:04:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f7556632740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e24da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f7556632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e25ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f7556632740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 0x555624e2c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f7556632740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x555624e05b90 con 0x555624e2db10 2024-04-25T10:04:56.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f7556632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x555624d084b0 con 0x555624e2def0 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f7556632740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x555624e05e70 con 0x555624e4c450 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f7556632740 10 client.0 _ll_register_callbacks cb 0x555624d48180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f75516cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e24da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f75516cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e24da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55330/0 (socket says 172.21.15.19:55330) 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f75516cc640 1 -- 172.21.15.19:0/1329278017 learned_addr learned my addr 172.21.15.19:0/1329278017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f7551ecd640 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 0x555624e2c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.521+0000 7f7550ecb640 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e25ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f75516cc640 1 -- 172.21.15.19:0/1329278017 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 msgr2=0x555624e2c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f75516cc640 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 0x555624e2c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f75516cc640 1 -- 172.21.15.19:0/1329278017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 msgr2=0x555624e25ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f75516cc640 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e25ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f75516cc640 1 -- 172.21.15.19:0/1329278017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555624b487d0 con 0x555624e2db10 2024-04-25T10:04:56.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f7550ecb640 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e25ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f75516cc640 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e24da0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f75480549e0 tx=0x7f7548060c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714f6b2764cbec64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f752bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f752bfff640 1 -- 172.21.15.19:0/1329278017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548060800 con 0x555624e2db10 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f752bfff640 1 -- 172.21.15.19:0/1329278017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f75480609a0 con 0x555624e2db10 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f7556632740 1 -- 172.21.15.19:0/1329278017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f753c04c070 con 0x555624e2db10 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f752bfff640 1 -- 172.21.15.19:0/1329278017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75480579d0 con 0x555624e2db10 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f7556632740 10 client.10944 fetch_fsmap learned FSMap version 109 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f7556632740 1 -- 172.21.15.19:0/1329278017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x555624a7e3f0 con 0x555624e2db10 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f752bfff640 1 -- 172.21.15.19:0/1329278017 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c04c070 con 0x555624e2db10 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f752bfff640 1 -- 172.21.15.19:0/1329278017 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 109) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7548057b70 con 0x555624e2db10 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f7556632740 10 client.10944 fetch_fsmap finished waiting for FSMap version 109 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f7556632740 10 client.10944 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f7556632740 20 client.10944 populate_metadata read hostname 'smithi019' 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f7556632740 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x555624e27fa0 0x555624e31ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f7556632740 4 client.10944 mds_command: new command op to 10698 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.522+0000 7f7556632740 1 -- 172.21.15.19:0/1329278017 --> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555624d772e0 con 0x555624e27fa0 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.523+0000 7f7550ecb640 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x555624e27fa0 0x555624e31ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.523+0000 7f7550ecb640 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x555624e27fa0 0x555624e31ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bada5b943ffe4348 server_cookie=c11fb1f18b3b5ae5 in_seq=0 out_seq=0 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.523+0000 7f752bfff640 10 client.10944 ms_handle_connect on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.523+0000 7f752bfff640 1 -- 172.21.15.19:0/1329278017 <== mds.0 v2:172.21.15.19:6832/3942829201 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555624d772e0 con 0x555624e27fa0 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.523+0000 7f752bfff640 10 client.10944 handle_command_reply: tid=0 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.523+0000 7f7556632740 1 client.10944 shutdown 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f7556632740 1 -- 172.21.15.19:0/1329278017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 msgr2=0x555624e24da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f7556632740 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e24da0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f75480549e0 tx=0x7f7548060c90 comp rx=0 tx=0).stop 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f7556632740 1 -- 172.21.15.19:0/1329278017 shutdown_connections 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f7556632740 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x555624e27fa0 0x555624e31ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f7556632740 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 0x555624e2c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f752bfff640 0 client.10944 ms_handle_reset on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f7556632740 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e25ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f7556632740 1 --2- 172.21.15.19:0/1329278017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e24da0 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f7556632740 1 -- 172.21.15.19:0/1329278017 >> 172.21.15.19:0/1329278017 conn(0x555624e00770 msgr2=0x555624e01400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f7556632740 1 -- 172.21.15.19:0/1329278017 shutdown_connections 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f7556632740 1 -- 172.21.15.19:0/1329278017 wait complete. 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.524+0000 7f7556632740 20 client.10944 trim_cache size 0 max 16384 2024-04-25T10:04:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.539+0000 7f7556632740 1 Processor -- start 2024-04-25T10:04:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.539+0000 7f7556632740 1 -- start start 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f7556632740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e4e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f7556632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e4e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f7556632740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 0x555624dff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f7556632740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c200570 con 0x555624e2db10 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f7556632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c1ff690 con 0x555624e2def0 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f7556632740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0c9f30 con 0x555624e4c450 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f75516cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e4e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f75516cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e4e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55344/0 (socket says 172.21.15.19:55344) 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f75516cc640 1 -- 172.21.15.19:0/1168302503 learned_addr learned my addr 172.21.15.19:0/1168302503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f75516cc640 1 -- 172.21.15.19:0/1168302503 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 msgr2=0x555624dff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f75516cc640 1 --2- 172.21.15.19:0/1168302503 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 0x555624dff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f75516cc640 1 -- 172.21.15.19:0/1168302503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 msgr2=0x555624e4e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f75516cc640 1 --2- 172.21.15.19:0/1168302503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e4e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f75516cc640 1 -- 172.21.15.19:0/1168302503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555624a3e0f0 con 0x555624e2db10 2024-04-25T10:04:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.540+0000 7f75516cc640 1 --2- 172.21.15.19:0/1168302503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e4e140 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f7548058280 tx=0x7f754806d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0eea99485d1ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f75367fc640 1 -- 172.21.15.19:0/1168302503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548057ee0 con 0x555624e2db10 2024-04-25T10:04:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f75367fc640 1 -- 172.21.15.19:0/1168302503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548057b70 con 0x555624e2db10 2024-04-25T10:04:56.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f75367fc640 1 -- 172.21.15.19:0/1168302503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548060800 con 0x555624e2db10 2024-04-25T10:04:56.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f7556632740 1 -- 172.21.15.19:0/1168302503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 msgr2=0x555624e4e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f7556632740 1 --2- 172.21.15.19:0/1168302503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e4e140 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f7548058280 tx=0x7f754806d000 comp rx=0 tx=0).stop 2024-04-25T10:04:56.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f7556632740 1 -- 172.21.15.19:0/1168302503 shutdown_connections 2024-04-25T10:04:56.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f7556632740 1 --2- 172.21.15.19:0/1168302503 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 0x555624dff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f7556632740 1 --2- 172.21.15.19:0/1168302503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e4e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f7556632740 1 --2- 172.21.15.19:0/1168302503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624e4e140 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f7556632740 1 -- 172.21.15.19:0/1168302503 >> 172.21.15.19:0/1168302503 conn(0x555624e00770 msgr2=0x555624e241d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:56.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f7556632740 1 -- 172.21.15.19:0/1168302503 shutdown_connections 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.541+0000 7f7556632740 1 -- 172.21.15.19:0/1168302503 wait complete. 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.542+0000 7f7556632740 1 -- start start 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.542+0000 7f7556632740 1 Processor -- start 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f7556632740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624eb00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f7556632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e40cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f7556632740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 0x555624e411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f7556632740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x555624e4aa10 con 0x555624e2db10 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f7556632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c1ff070 con 0x555624e2def0 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f7556632740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x555624e2be40 con 0x555624e4c450 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f7556632740 10 client.0 _ll_register_callbacks cb 0x7f753c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f75516cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624eb00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f75516cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624eb00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55352/0 (socket says 172.21.15.19:55352) 2024-04-25T10:04:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f75516cc640 1 -- 172.21.15.19:0/4104574665 learned_addr learned my addr 172.21.15.19:0/4104574665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f7550ecb640 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e40cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f7551ecd640 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 0x555624e411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f75516cc640 1 -- 172.21.15.19:0/4104574665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 msgr2=0x555624e411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f75516cc640 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 0x555624e411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f75516cc640 1 -- 172.21.15.19:0/4104574665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 msgr2=0x555624e40cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f75516cc640 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e40cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f75516cc640 1 -- 172.21.15.19:0/4104574665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555624e33d50 con 0x555624e2db10 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f7550ecb640 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e40cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f75516cc640 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624eb00c0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f754800ce10 tx=0x7f7548088200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3faded366a95e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.543+0000 7f752bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f752bfff640 1 -- 172.21.15.19:0/4104574665 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548057ee0 con 0x555624e2db10 2024-04-25T10:04:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f752bfff640 1 -- 172.21.15.19:0/4104574665 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f75480544d0 con 0x555624e2db10 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f7556632740 1 -- 172.21.15.19:0/4104574665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7548057b70 con 0x555624e2db10 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f752bfff640 1 -- 172.21.15.19:0/4104574665 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7548057440 con 0x555624e2db10 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f7556632740 10 client.10950 fetch_fsmap learned FSMap version 109 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f7556632740 1 -- 172.21.15.19:0/4104574665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x555624e302e0 con 0x555624e2db10 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f752bfff640 1 -- 172.21.15.19:0/4104574665 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548057b70 con 0x555624e2db10 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f752bfff640 1 -- 172.21.15.19:0/4104574665 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 109) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f75480575e0 con 0x555624e2db10 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f7556632740 10 client.10950 fetch_fsmap finished waiting for FSMap version 109 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f7556632740 10 client.10950 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f7556632740 20 client.10950 populate_metadata read hostname 'smithi019' 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f7556632740 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x555624e24240 0x555624eb64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f7556632740 4 client.10950 mds_command: new command op to 10698 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f7556632740 1 -- 172.21.15.19:0/4104574665 --> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555624e00fc0 con 0x555624e24240 2024-04-25T10:04:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.544+0000 7f7550ecb640 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x555624e24240 0x555624eb64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f752bfff640 10 client.10950 ms_handle_connect on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7550ecb640 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x555624e24240 0x555624eb64c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd57f7653f1d43e server_cookie=3891dd857e86166a in_seq=0 out_seq=0 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f752bfff640 1 -- 172.21.15.19:0/4104574665 <== mds.0 v2:172.21.15.19:6832/3942829201 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555624e00fc0 con 0x555624e24240 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f752bfff640 10 client.10950 handle_command_reply: tid=0 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 1 client.10950 shutdown 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 1 -- 172.21.15.19:0/4104574665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 msgr2=0x555624eb00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624eb00c0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f754800ce10 tx=0x7f7548088200 comp rx=0 tx=0).stop 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 1 -- 172.21.15.19:0/4104574665 shutdown_connections 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.19:6832/3942829201,v1:172.21.15.19:6834/3942829201] conn(0x555624e24240 0x555624eb64c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555624e4c450 0x555624e411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555624e2def0 0x555624e40cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 1 --2- 172.21.15.19:0/4104574665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555624e2db10 0x555624eb00c0 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 1 -- 172.21.15.19:0/4104574665 >> 172.21.15.19:0/4104574665 conn(0x555624e00770 msgr2=0x555624e4e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f752bfff640 0 client.10950 ms_handle_reset on v2:172.21.15.19:6832/3942829201 2024-04-25T10:04:56.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 1 -- 172.21.15.19:0/4104574665 shutdown_connections 2024-04-25T10:04:56.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 1 -- 172.21.15.19:0/4104574665 wait complete. 2024-04-25T10:04:56.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.545+0000 7f7556632740 20 client.10950 trim_cache size 0 max 16384 2024-04-25T10:04:56.557 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:04:56.557 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:04:56.557 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:04:56.557 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:04:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 -- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f753006b500 msgr2=0x7f753006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f753006b500 0x7f753006d9c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f753c034000 tx=0x7f753c02c000 comp rx=0 tx=0).stop 2024-04-25T10:04:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 -- 172.21.15.19:0/2288668564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0ced80 msgr2=0x7f754c154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0ced80 0x7f754c154400 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f754800c450 tx=0x7f754800c910 comp rx=0 tx=0).stop 2024-04-25T10:04:56.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 -- 172.21.15.19:0/2288668564 shutdown_connections 2024-04-25T10:04:56.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f753006b500 0x7f753006d9c0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f754c161710 0x7f754c15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f754c15dc20 0x7f754c0c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 --2- 172.21.15.19:0/2288668564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0ced80 0x7f754c154400 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 -- 172.21.15.19:0/2288668564 >> 172.21.15.19:0/2288668564 conn(0x7f754c0d0420 msgr2=0x7f754c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:56.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 -- 172.21.15.19:0/2288668564 shutdown_connections 2024-04-25T10:04:56.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.546+0000 7f7553930640 1 -- 172.21.15.19:0/2288668564 wait complete. 2024-04-25T10:04:56.562 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:04:56.562 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T10:04:56.562 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T10:04:56.562 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> import json 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> except OSError as e: 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(e.errno) 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> print(json.dumps( 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> indent=2)) 2024-04-25T10:04:56.563 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T10:04:56.632 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:04:56.632 INFO:teuthology.orchestra.run.smithi019.stdout: "st_mode": 33188, 2024-04-25T10:04:56.632 INFO:teuthology.orchestra.run.smithi019.stdout: "st_ino": 1099511627777, 2024-04-25T10:04:56.632 INFO:teuthology.orchestra.run.smithi019.stdout: "st_dev": 43, 2024-04-25T10:04:56.632 INFO:teuthology.orchestra.run.smithi019.stdout: "st_nlink": 1, 2024-04-25T10:04:56.633 INFO:teuthology.orchestra.run.smithi019.stdout: "st_uid": 1000, 2024-04-25T10:04:56.633 INFO:teuthology.orchestra.run.smithi019.stdout: "st_gid": 1316, 2024-04-25T10:04:56.633 INFO:teuthology.orchestra.run.smithi019.stdout: "st_size": 6291456, 2024-04-25T10:04:56.633 INFO:teuthology.orchestra.run.smithi019.stdout: "st_atime": 1714039465.1838725, 2024-04-25T10:04:56.633 INFO:teuthology.orchestra.run.smithi019.stdout: "st_mtime": 1714039465.2018723, 2024-04-25T10:04:56.633 INFO:teuthology.orchestra.run.smithi019.stdout: "st_ctime": 1714039465.2018723 2024-04-25T10:04:56.633 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:04:56.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:04:56.642 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:56.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.778+0000 7fc62d4fc640 1 Processor -- start 2024-04-25T10:04:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.778+0000 7fc62d4fc640 1 -- start start 2024-04-25T10:04:56.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.779+0000 7fc62d4fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc6280b6cb0 0x7fc6280b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.779+0000 7fc62d4fc640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6280ceee0 con 0x7fc62815db70 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.779+0000 7fc62d4fc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6280d1d10 con 0x7fc6280b5a60 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.779+0000 7fc62d4fc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6280d1ff0 con 0x7fc6280b6cb0 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.779+0000 7fc626ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc62815db70 0x7fc628162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40748/0 (socket says 172.21.15.19:40748) 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.779+0000 7fc626ffd640 1 -- 172.21.15.19:0/9496439 learned_addr learned my addr 172.21.15.19:0/9496439 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.779+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857380444 0 0) 0x7fc6280ceee0 con 0x7fc62815db70 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.779+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc610003520 con 0x7fc62815db70 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc6267fc640 1 --2- 172.21.15.19:0/9496439 >> v2:172.21.15.112:3300/0 conn(0x7fc6280b6cb0 0x7fc6280b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3816306017 0 0) 0x7fc610003520 con 0x7fc62815db70 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 >> v2:172.21.15.112:3300/0 conn(0x7fc6280b6cb0 msgr2=0x7fc6280b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 --2- 172.21.15.19:0/9496439 >> v2:172.21.15.112:3300/0 conn(0x7fc6280b6cb0 0x7fc6280b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 >> v1:172.21.15.112:6789/0 conn(0x7fc6280b5a60 legacy=0x7fc6280d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6280d22d0 con 0x7fc62815db70 2024-04-25T10:04:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fc620001670 con 0x7fc62815db70 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 >> v1:172.21.15.19:6789/0 conn(0x7fc62815db70 legacy=0x7fc628162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 --2- 172.21.15.19:0/9496439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6100039b0 0x7fc610003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 --2- 172.21.15.19:0/9496439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610004800 0x7fc610004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 --2- 172.21.15.19:0/9496439 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100052e0 0x7fc61000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6280d1ff0 con 0x7fc6100039b0 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61000c1b0 con 0x7fc610004800 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61000c3b0 con 0x7fc6100052e0 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc626ffd640 1 --2- 172.21.15.19:0/9496439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6100039b0 0x7fc610003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc6267fc640 1 --2- 172.21.15.19:0/9496439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610004800 0x7fc610004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc6277fe640 1 --2- 172.21.15.19:0/9496439 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100052e0 0x7fc61000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc626ffd640 1 -- 172.21.15.19:0/9496439 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100052e0 msgr2=0x7fc61000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc626ffd640 1 --2- 172.21.15.19:0/9496439 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100052e0 0x7fc61000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc626ffd640 1 -- 172.21.15.19:0/9496439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610004800 msgr2=0x7fc610004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc626ffd640 1 --2- 172.21.15.19:0/9496439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610004800 0x7fc610004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc626ffd640 1 -- 172.21.15.19:0/9496439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610003520 con 0x7fc6100039b0 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.780+0000 7fc6277fe640 1 -- 172.21.15.19:0/9496439 reap_dead start 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.781+0000 7fc626ffd640 1 --2- 172.21.15.19:0/9496439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6100039b0 0x7fc610003dd0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c210 tx=0x7fc610005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fd049fd281a1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.781+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620007cf0 con 0x7fc6100039b0 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.781+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc620005af0 con 0x7fc6100039b0 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.781+0000 7fc625ffb640 1 -- 172.21.15.19:0/9496439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620007070 con 0x7fc6100039b0 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.782+0000 7fc62d4fc640 1 -- 172.21.15.19:0/9496439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6100039b0 msgr2=0x7fc610003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.782+0000 7fc62d4fc640 1 --2- 172.21.15.19:0/9496439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6100039b0 0x7fc610003dd0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c210 tx=0x7fc610005830 comp rx=0 tx=0).stop 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.782+0000 7fc62d4fc640 1 -- 172.21.15.19:0/9496439 shutdown_connections 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.782+0000 7fc62d4fc640 1 --2- 172.21.15.19:0/9496439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6100039b0 0x7fc610003dd0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.782+0000 7fc62d4fc640 1 -- 172.21.15.19:0/9496439 >> 172.21.15.19:0/9496439 conn(0x7fc6280d0520 msgr2=0x7fc6280d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.782+0000 7fc62d4fc640 1 -- 172.21.15.19:0/9496439 shutdown_connections 2024-04-25T10:04:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.782+0000 7fc62d4fc640 1 -- 172.21.15.19:0/9496439 wait complete. 2024-04-25T10:04:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.783+0000 7fc62d4fc640 1 Processor -- start 2024-04-25T10:04:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.783+0000 7fc62d4fc640 1 -- start start 2024-04-25T10:04:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc62d4fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100039b0 0x7fc6281f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc62d4fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62815f210 0x7fc628153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc62d4fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628153cb0 0x7fc628155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc62d4fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6280cf1c0 con 0x7fc62815f210 2024-04-25T10:04:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc62d4fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6280ce530 con 0x7fc628153cb0 2024-04-25T10:04:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc62d4fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6280cf340 con 0x7fc6100039b0 2024-04-25T10:04:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc6267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62815f210 0x7fc628153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc6267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62815f210 0x7fc628153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55370/0 (socket says 172.21.15.19:55370) 2024-04-25T10:04:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc6267fc640 1 -- 172.21.15.19:0/2206391909 learned_addr learned my addr 172.21.15.19:0/2206391909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc626ffd640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100039b0 0x7fc6281f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc6277fe640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628153cb0 0x7fc628155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc6267fc640 1 -- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100039b0 msgr2=0x7fc6281f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc6267fc640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100039b0 0x7fc6281f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc6267fc640 1 -- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628153cb0 msgr2=0x7fc628155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc6267fc640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628153cb0 0x7fc628155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc6267fc640 1 -- 172.21.15.19:0/2206391909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6281f3d70 con 0x7fc62815f210 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc626ffd640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100039b0 0x7fc6281f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.784+0000 7fc6267fc640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62815f210 0x7fc628153770 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00b720 tx=0x7fc61c00bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.785+0000 7fc60f7fe640 1 -- 172.21.15.19:0/2206391909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c018070 con 0x7fc62815f210 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.785+0000 7fc60f7fe640 1 -- 172.21.15.19:0/2206391909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c010750 con 0x7fc62815f210 2024-04-25T10:04:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.785+0000 7fc62d4fc640 1 -- 172.21.15.19:0/2206391909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6281f4050 con 0x7fc62815f210 2024-04-25T10:04:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.785+0000 7fc60f7fe640 1 -- 172.21.15.19:0/2206391909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c010b60 con 0x7fc62815f210 2024-04-25T10:04:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.785+0000 7fc62d4fc640 1 -- 172.21.15.19:0/2206391909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc628155990 con 0x7fc62815f210 2024-04-25T10:04:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.786+0000 7fc62d4fc640 1 -- 172.21.15.19:0/2206391909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc628073990 con 0x7fc62815f210 2024-04-25T10:04:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.786+0000 7fc60f7fe640 1 -- 172.21.15.19:0/2206391909 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc61c01b070 con 0x7fc62815f210 2024-04-25T10:04:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.787+0000 7fc60f7fe640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 0x7fc61406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.787+0000 7fc60f7fe640 1 -- 172.21.15.19:0/2206391909 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc61c08a5a0 con 0x7fc62815f210 2024-04-25T10:04:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.787+0000 7fc626ffd640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 0x7fc61406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.787+0000 7fc626ffd640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 0x7fc61406d990 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc620032000 tx=0x7fc620009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:56.791+0000 7fc60f7fe640 1 -- 172.21.15.19:0/2206391909 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc61c058bd0 con 0x7fc62815f210 2024-04-25T10:04:57.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.009+0000 7fc62d4fc640 1 -- 172.21.15.19:0/2206391909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6280d1820 con 0x7fc62815f210 2024-04-25T10:04:57.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.009+0000 7fc60f7fe640 1 -- 172.21.15.19:0/2206391909 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4954 (secure 0 0 0) 0x7fc61c05cc70 con 0x7fc62815f210 2024-04-25T10:04:57.015 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:57.015 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1300145056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1300145056},{"type":"v1","addr":"172.21.15.19:6835","nonce":1300145056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":8027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2001687418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2001687418},{"type":"v1","addr":"172.21.15.112:6836","nonce":2001687418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721457153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721457153},{"type":"v1","addr":"172.21.15.112:6837","nonce":721457153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:14.252458+0000","modified":"2024-04-25T10:04:16.259643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10698":{"gid":10698,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3942829201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3942829201},{"type":"v1","addr":"172.21.15.19:6834","nonce":3942829201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:04:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 109 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 -- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 msgr2=0x7fc61406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 0x7fc61406d990 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc620032000 tx=0x7fc620009030 comp rx=0 tx=0).stop 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 -- 172.21.15.19:0/2206391909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62815f210 msgr2=0x7fc628153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62815f210 0x7fc628153770 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00b720 tx=0x7fc61c00bbe0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 -- 172.21.15.19:0/2206391909 shutdown_connections 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc61406b4d0 0x7fc61406d990 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628153cb0 0x7fc628155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc62815f210 0x7fc628153770 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 --2- 172.21.15.19:0/2206391909 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6100039b0 0x7fc6281f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 -- 172.21.15.19:0/2206391909 >> 172.21.15.19:0/2206391909 conn(0x7fc6280d0520 msgr2=0x7fc6280d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 -- 172.21.15.19:0/2206391909 shutdown_connections 2024-04-25T10:04:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.011+0000 7fc62d4fc640 1 -- 172.21.15.19:0/2206391909 wait complete. 2024-04-25T10:04:57.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:04:57.149 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-04-25T10:04:57.149 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:04:57.149 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:04:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.186+0000 7fc2659dbf40 1 Processor -- start 2024-04-25T10:04:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.186+0000 7fc2659dbf40 1 -- start start 2024-04-25T10:04:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.187+0000 7fc2659dbf40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fd8f88d610 0x55fd8f92a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.187+0000 7fc2659dbf40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd8f88d9f0 0x55fd8f88e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.187+0000 7fc2659dbf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 0x55fd8f8acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.187+0000 7fc2659dbf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8f93d0f0 con 0x55fd8f939ee0 2024-04-25T10:04:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.187+0000 7fc2659dbf40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8f80adf0 con 0x55fd8f88d9f0 2024-04-25T10:04:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.187+0000 7fc2659dbf40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd8f93d390 con 0x55fd8f88d610 2024-04-25T10:04:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.187+0000 7fc263cdf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 0x55fd8f8acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.187+0000 7fc263cdf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 0x55fd8f8acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55396/0 (socket says 172.21.15.19:55396) 2024-04-25T10:04:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.187+0000 7fc263cdf640 1 -- 172.21.15.19:0/3171218123 learned_addr learned my addr 172.21.15.19:0/3171218123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc263cdf640 1 -- 172.21.15.19:0/3171218123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fd8f88d610 msgr2=0x55fd8f92a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc2634de640 1 --2- 172.21.15.19:0/3171218123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fd8f88d610 0x55fd8f92a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc263cdf640 1 --2- 172.21.15.19:0/3171218123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fd8f88d610 0x55fd8f92a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc263cdf640 1 -- 172.21.15.19:0/3171218123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd8f88d9f0 msgr2=0x55fd8f88e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc262cdd640 1 --2- 172.21.15.19:0/3171218123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd8f88d9f0 0x55fd8f88e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc263cdf640 1 --2- 172.21.15.19:0/3171218123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd8f88d9f0 0x55fd8f88e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc263cdf640 1 -- 172.21.15.19:0/3171218123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd8f790c50 con 0x55fd8f939ee0 2024-04-25T10:04:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc263cdf640 1 --2- 172.21.15.19:0/3171218123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 0x55fd8f8acac0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fc25400a4e0 tx=0x7fc25400a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36d8fc0ce3dc3f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc261cdb640 1 -- 172.21.15.19:0/3171218123 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25400f7b0 con 0x55fd8f939ee0 2024-04-25T10:04:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc261cdb640 1 -- 172.21.15.19:0/3171218123 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254015070 con 0x55fd8f939ee0 2024-04-25T10:04:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc261cdb640 1 -- 172.21.15.19:0/3171218123 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25400fb70 con 0x55fd8f939ee0 2024-04-25T10:04:57.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc2659dbf40 1 -- 172.21.15.19:0/3171218123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 msgr2=0x55fd8f8acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc2659dbf40 1 --2- 172.21.15.19:0/3171218123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 0x55fd8f8acac0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fc25400a4e0 tx=0x7fc25400a9a0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc2659dbf40 1 -- 172.21.15.19:0/3171218123 shutdown_connections 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc2659dbf40 1 --2- 172.21.15.19:0/3171218123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 0x55fd8f8acac0 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc2659dbf40 1 --2- 172.21.15.19:0/3171218123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd8f88d9f0 0x55fd8f88e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc2659dbf40 1 --2- 172.21.15.19:0/3171218123 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fd8f88d610 0x55fd8f92a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.188+0000 7fc2659dbf40 1 -- 172.21.15.19:0/3171218123 >> 172.21.15.19:0/3171218123 conn(0x55fd8f8629b0 msgr2=0x55fd8f8910d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.189+0000 7fc2659dbf40 1 -- 172.21.15.19:0/3171218123 shutdown_connections 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.189+0000 7fc2659dbf40 1 -- 172.21.15.19:0/3171218123 wait complete. 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.189+0000 7fc2659dbf40 1 Processor -- start 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.189+0000 7fc2659dbf40 1 -- start start 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc2659dbf40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fd8f88d610 0x55fd8f926b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc2659dbf40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd8f88d9f0 0x55fd8f930ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc2659dbf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 0x55fd8f931020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc2659dbf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8f92db00 con 0x55fd8f939ee0 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc2659dbf40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8f938300 con 0x55fd8f88d9f0 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc2659dbf40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd8f92dde0 con 0x55fd8f88d610 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc263cdf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 0x55fd8f931020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc263cdf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 0x55fd8f931020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55402/0 (socket says 172.21.15.19:55402) 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc263cdf640 1 -- 172.21.15.19:0/1917153125 learned_addr learned my addr 172.21.15.19:0/1917153125 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc2634de640 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fd8f88d610 0x55fd8f926b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc262cdd640 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd8f88d9f0 0x55fd8f930ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc263cdf640 1 -- 172.21.15.19:0/1917153125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fd8f88d610 msgr2=0x55fd8f926b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc263cdf640 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55fd8f88d610 0x55fd8f926b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc263cdf640 1 -- 172.21.15.19:0/1917153125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd8f88d9f0 msgr2=0x55fd8f930ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc263cdf640 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd8f88d9f0 0x55fd8f930ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc263cdf640 1 -- 172.21.15.19:0/1917153125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd8f93be80 con 0x55fd8f939ee0 2024-04-25T10:04:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.190+0000 7fc263cdf640 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 0x55fd8f931020 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fc25400b010 tx=0x7fc254003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.191+0000 7fc260cd9640 1 -- 172.21.15.19:0/1917153125 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254010950 con 0x55fd8f939ee0 2024-04-25T10:04:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.191+0000 7fc260cd9640 1 -- 172.21.15.19:0/1917153125 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254020070 con 0x55fd8f939ee0 2024-04-25T10:04:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.191+0000 7fc2659dbf40 1 -- 172.21.15.19:0/1917153125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fd8f862f30 con 0x55fd8f939ee0 2024-04-25T10:04:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.191+0000 7fc260cd9640 1 -- 172.21.15.19:0/1917153125 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254010d90 con 0x55fd8f939ee0 2024-04-25T10:04:57.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.191+0000 7fc2659dbf40 1 -- 172.21.15.19:0/1917153125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd8f9270d0 con 0x55fd8f939ee0 2024-04-25T10:04:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.192+0000 7fc260cd9640 1 -- 172.21.15.19:0/1917153125 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc254025470 con 0x55fd8f939ee0 2024-04-25T10:04:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.192+0000 7fc260cd9640 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc244069eb0 0x7fc24406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.193+0000 7fc260cd9640 1 -- 172.21.15.19:0/1917153125 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc25408b440 con 0x55fd8f939ee0 2024-04-25T10:04:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.193+0000 7fc2659dbf40 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x55fd8f927450 0x55fd8f93a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.193+0000 7fc2659dbf40 1 -- 172.21.15.19:0/1917153125 --> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] -- osd_op(unknown.0.0:1 26.7 26:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167) v9 -- 0x55fd8f93ad60 con 0x55fd8f927450 2024-04-25T10:04:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.193+0000 7fc2634de640 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc244069eb0 0x7fc24406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.193+0000 7fc262cdd640 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x55fd8f927450 0x55fd8f93a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.193+0000 7fc262cdd640 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x55fd8f927450 0x55fd8f93a820 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.193+0000 7fc2634de640 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc244069eb0 0x7fc24406c370 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc25c057000 tx=0x7fc25c052000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc262cdd640 1 -- 172.21.15.19:0/1917153125 <== osd.2 v2:172.21.15.19:6816/3205329770 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fc24c00b150 con 0x55fd8f927450 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc2659dbf40 1 -- 172.21.15.19:0/1917153125 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x55fd8f927450 msgr2=0x55fd8f93a820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc2659dbf40 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x55fd8f927450 0x55fd8f93a820 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc2659dbf40 1 -- 172.21.15.19:0/1917153125 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc244069eb0 msgr2=0x7fc24406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc2659dbf40 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc244069eb0 0x7fc24406c370 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc25c057000 tx=0x7fc25c052000 comp rx=0 tx=0).stop 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc2659dbf40 1 -- 172.21.15.19:0/1917153125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 msgr2=0x55fd8f931020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc2659dbf40 1 --2- 172.21.15.19:0/1917153125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd8f939ee0 0x55fd8f931020 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fc25400b010 tx=0x7fc254003330 comp rx=0 tx=0).stop 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc263cdf640 1 -- 172.21.15.19:0/1917153125 reap_dead start 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc2659dbf40 1 -- 172.21.15.19:0/1917153125 shutdown_connections 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc2659dbf40 1 -- 172.21.15.19:0/1917153125 >> 172.21.15.19:0/1917153125 conn(0x55fd8f8629b0 msgr2=0x55fd8f8910b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc2659dbf40 1 -- 172.21.15.19:0/1917153125 shutdown_connections 2024-04-25T10:04:57.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.194+0000 7fc2659dbf40 1 -- 172.21.15.19:0/1917153125 wait complete. 2024-04-25T10:04:57.205 INFO:teuthology.orchestra.run.smithi019.stdout:Ysixmegssubdir  2024-04-25T10:04:57.205 DEBUG:teuthology.orchestra.run.smithi019:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:04:57.317 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:04:57.317 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627777, 2024-04-25T10:04:57.317 INFO:teuthology.orchestra.run.smithi019.stdout: "ancestors": [ 2024-04-25T10:04:57.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:57.317 INFO:teuthology.orchestra.run.smithi019.stdout: "dirino": 1099511627776, 2024-04-25T10:04:57.317 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "sixmegs", 2024-04-25T10:04:57.317 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 8 2024-04-25T10:04:57.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:04:57.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:04:57.317 INFO:teuthology.orchestra.run.smithi019.stdout: "dirino": 1, 2024-04-25T10:04:57.318 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "subdir", 2024-04-25T10:04:57.318 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 11 2024-04-25T10:04:57.318 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:04:57.318 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T10:04:57.318 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 26, 2024-04-25T10:04:57.318 INFO:teuthology.orchestra.run.smithi019.stdout: "old_pools": [] 2024-04-25T10:04:57.318 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:04:57.318 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:57.332 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:04:57.332 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:57.420 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:57.466 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:04:57.466 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:04:57.466 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:57.482 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:04:57.482 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:57.539 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:57.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.674+0000 7f96d906b640 1 Processor -- start 2024-04-25T10:04:57.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.674+0000 7f96d906b640 1 -- start start 2024-04-25T10:04:57.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d906b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f96d415dbe0 0x7f96d40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d906b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d40b6220 con 0x7f96d40b2cb0 2024-04-25T10:04:57.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d906b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d40b97a0 con 0x7f96d40b92a0 2024-04-25T10:04:57.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d906b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d415dfc0 con 0x7f96d415dbe0 2024-04-25T10:04:57.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d259c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f96d40b2cb0 0x7f96d40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55388/0 (socket says 172.21.15.19:55388) 2024-04-25T10:04:57.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d259c640 1 -- 172.21.15.19:0/202206983 learned_addr learned my addr 172.21.15.19:0/202206983 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:57.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d2d9d640 1 --2- 172.21.15.19:0/202206983 >> v2:172.21.15.112:3300/0 conn(0x7f96d415dbe0 0x7f96d40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242766087 0 0) 0x7f96d40b6220 con 0x7f96d40b2cb0 2024-04-25T10:04:57.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96bc003520 con 0x7f96d40b2cb0 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3762456336 0 0) 0x7f96bc003520 con 0x7f96d40b2cb0 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 >> v2:172.21.15.112:3300/0 conn(0x7f96d415dbe0 msgr2=0x7f96d40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d1d9b640 1 --2- 172.21.15.19:0/202206983 >> v2:172.21.15.112:3300/0 conn(0x7f96d415dbe0 0x7f96d40d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 >> v1:172.21.15.112:6789/0 conn(0x7f96d40b92a0 legacy=0x7f96d40b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d40d1c50 con 0x7f96d40b2cb0 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.675+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f96c8001670 con 0x7f96d40b2cb0 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 >> v1:172.21.15.19:6789/0 conn(0x7f96d40b2cb0 legacy=0x7f96d40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d1d9b640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96bc0039b0 0x7f96bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d1d9b640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96bc004800 0x7f96bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d1d9b640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96bc0051c0 0x7f96bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc00c130 con 0x7f96bc0039b0 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc00c2e0 con 0x7f96bc004800 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc00c4e0 con 0x7f96bc0051c0 2024-04-25T10:04:57.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d2d9d640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96bc0039b0 0x7f96bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d259c640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96bc004800 0x7f96bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d359e640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96bc0051c0 0x7f96bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d2d9d640 1 -- 172.21.15.19:0/202206983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96bc0051c0 msgr2=0x7f96bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d2d9d640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96bc0051c0 0x7f96bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d2d9d640 1 -- 172.21.15.19:0/202206983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96bc004800 msgr2=0x7f96bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d2d9d640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96bc004800 0x7f96bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d2d9d640 1 -- 172.21.15.19:0/202206983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc003520 con 0x7f96bc0039b0 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d359e640 1 -- 172.21.15.19:0/202206983 reap_dead start 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.676+0000 7f96d259c640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96bc004800 0x7f96bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.677+0000 7f96d2d9d640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96bc0039b0 0x7f96bc003dd0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009ac0 tx=0x7f96bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b84230d325ae36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.677+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4036070 con 0x7f96bc0039b0 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.677+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c4031400 con 0x7f96bc0039b0 2024-04-25T10:04:57.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.677+0000 7f96d1d9b640 1 -- 172.21.15.19:0/202206983 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4009690 con 0x7f96bc0039b0 2024-04-25T10:04:57.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.677+0000 7f96d906b640 1 -- 172.21.15.19:0/202206983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96bc0039b0 msgr2=0x7f96bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.677+0000 7f96d906b640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96bc0039b0 0x7f96bc003dd0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009ac0 tx=0x7f96bc005830 comp rx=0 tx=0).stop 2024-04-25T10:04:57.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.677+0000 7f96d906b640 1 -- 172.21.15.19:0/202206983 shutdown_connections 2024-04-25T10:04:57.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.677+0000 7f96d906b640 1 --2- 172.21.15.19:0/202206983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96bc0039b0 0x7f96bc003dd0 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.677+0000 7f96d906b640 1 -- 172.21.15.19:0/202206983 >> 172.21.15.19:0/202206983 conn(0x7f96d4061a20 msgr2=0x7f96d4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.677+0000 7f96d906b640 1 -- 172.21.15.19:0/202206983 shutdown_connections 2024-04-25T10:04:57.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.677+0000 7f96d906b640 1 -- 172.21.15.19:0/202206983 wait complete. 2024-04-25T10:04:57.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.679+0000 7f96d906b640 1 Processor -- start 2024-04-25T10:04:57.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.679+0000 7f96d906b640 1 -- start start 2024-04-25T10:04:57.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d906b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96bc0039b0 0x7f96d415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d906b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96d4154130 0x7f96d40b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d906b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96d40c8ff0 0x7f96d40c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d906b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40d2210 con 0x7f96d4154130 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d906b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40d1a00 con 0x7f96d40c8ff0 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d906b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40b6ab0 con 0x7f96bc0039b0 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96d4154130 0x7f96d40b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96d4154130 0x7f96d40b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53160/0 (socket says 172.21.15.19:53160) 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d259c640 1 -- 172.21.15.19:0/61396482 learned_addr learned my addr 172.21.15.19:0/61396482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d2d9d640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96bc0039b0 0x7f96d415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d259c640 1 -- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96bc0039b0 msgr2=0x7f96d415dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d259c640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96bc0039b0 0x7f96d415dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d259c640 1 -- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96d40c8ff0 msgr2=0x7f96d40c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d359e640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96d40c8ff0 0x7f96d40c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d259c640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96d40c8ff0 0x7f96d40c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d259c640 1 -- 172.21.15.19:0/61396482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d415e500 con 0x7f96d4154130 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d259c640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96d4154130 0x7f96d40b49e0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f96c80017e0 tx=0x7f96c800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96bb7fe640 1 -- 172.21.15.19:0/61396482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c800ee60 con 0x7f96d4154130 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96bb7fe640 1 -- 172.21.15.19:0/61396482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c8018070 con 0x7f96d4154130 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96d906b640 1 -- 172.21.15.19:0/61396482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d40c99c0 con 0x7f96d4154130 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.680+0000 7f96bb7fe640 1 -- 172.21.15.19:0/61396482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c800ca90 con 0x7f96d4154130 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.681+0000 7f96d906b640 1 -- 172.21.15.19:0/61396482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d41f7170 con 0x7f96d4154130 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.681+0000 7f96d906b640 1 -- 172.21.15.19:0/61396482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d4073990 con 0x7f96d4154130 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.682+0000 7f96bb7fe640 1 -- 172.21.15.19:0/61396482 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96c800cc30 con 0x7f96d4154130 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.682+0000 7f96bb7fe640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96c006b4d0 0x7f96c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.682+0000 7f96bb7fe640 1 -- 172.21.15.19:0/61396482 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f96c808a160 con 0x7f96d4154130 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.683+0000 7f96d2d9d640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96c006b4d0 0x7f96c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.683+0000 7f96d2d9d640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96c006b4d0 0x7f96c006d990 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f96c4033000 tx=0x7f96c4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.687+0000 7f96bb7fe640 1 -- 172.21.15.19:0/61396482 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96c8058e70 con 0x7f96d4154130 2024-04-25T10:04:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.868+0000 7f96d906b640 1 -- 172.21.15.19:0/61396482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96d40d4440 con 0x7f96d4154130 2024-04-25T10:04:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.869+0000 7f96bb7fe640 1 -- 172.21.15.19:0/61396482 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+17066 (secure 0 0 0) 0x7f96c8058770 con 0x7f96d4154130 2024-04-25T10:04:57.874 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:57.874 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":167,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:04:49.642130+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:12.518429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:13.232769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2c","mappings":[{"from":3,"to":0}]},{"pgid":"26.31","mappings":[{"from":1,"to":0}]},{"pgid":"26.37","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 -- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96c006b4d0 msgr2=0x7f96c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96c006b4d0 0x7f96c006d990 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f96c4033000 tx=0x7f96c4008030 comp rx=0 tx=0).stop 2024-04-25T10:04:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 -- 172.21.15.19:0/61396482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96d4154130 msgr2=0x7f96d40b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96d4154130 0x7f96d40b49e0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f96c80017e0 tx=0x7f96c800ec10 comp rx=0 tx=0).stop 2024-04-25T10:04:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 -- 172.21.15.19:0/61396482 shutdown_connections 2024-04-25T10:04:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f96c006b4d0 0x7f96c006d990 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96d40c8ff0 0x7f96d40c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96d4154130 0x7f96d40b49e0 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 --2- 172.21.15.19:0/61396482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96bc0039b0 0x7f96d415dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 -- 172.21.15.19:0/61396482 >> 172.21.15.19:0/61396482 conn(0x7f96d4061a20 msgr2=0x7f96d4061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 -- 172.21.15.19:0/61396482 shutdown_connections 2024-04-25T10:04:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.871+0000 7f96d906b640 1 -- 172.21.15.19:0/61396482 wait complete. 2024-04-25T10:04:57.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:04:12.518429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:04:13.232769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '165', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:57.888 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.986+0000 7fbb1ca3a640 1 Processor -- start 2024-04-25T10:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.986+0000 7fbb1ca3a640 1 -- start start 2024-04-25T10:04:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.987+0000 7fbb1ca3a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbb1815dc20 0x7fbb180d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.987+0000 7fbb1ca3a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb180d1ce0 0x7fbb180b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.987+0000 7fbb1ca3a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbb180ce670 0x7fbb180ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.987+0000 7fbb1ca3a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb180d2120 con 0x7fbb180ce670 2024-04-25T10:04:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.987+0000 7fbb1ca3a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb180d23a0 con 0x7fbb1815dc20 2024-04-25T10:04:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.987+0000 7fbb1ca3a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180d2680 con 0x7fbb180d1ce0 2024-04-25T10:04:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.987+0000 7fbb16d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbb180ce670 0x7fbb180ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.987+0000 7fbb16d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbb180ce670 0x7fbb180ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53170/0 (socket says 172.21.15.19:53170) 2024-04-25T10:04:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.987+0000 7fbb16d9d640 1 -- 172.21.15.19:0/2870862910 learned_addr learned my addr 172.21.15.19:0/2870862910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.987+0000 7fbb1659c640 1 --2- 172.21.15.19:0/2870862910 >> v2:172.21.15.112:3300/0 conn(0x7fbb1815dc20 0x7fbb180d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.988+0000 7fbb15d9b640 1 --2- 172.21.15.19:0/2870862910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb180d1ce0 0x7fbb180b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.988+0000 7fbb16d9d640 1 -- 172.21.15.19:0/2870862910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb180d1ce0 msgr2=0x7fbb180b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.988+0000 7fbb16d9d640 1 --2- 172.21.15.19:0/2870862910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb180d1ce0 0x7fbb180b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.988+0000 7fbb16d9d640 1 -- 172.21.15.19:0/2870862910 >> v2:172.21.15.112:3300/0 conn(0x7fbb1815dc20 msgr2=0x7fbb180d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.988+0000 7fbb16d9d640 1 --2- 172.21.15.19:0/2870862910 >> v2:172.21.15.112:3300/0 conn(0x7fbb1815dc20 0x7fbb180d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.988+0000 7fbb16d9d640 1 -- 172.21.15.19:0/2870862910 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180d2960 con 0x7fbb180ce670 2024-04-25T10:04:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.988+0000 7fbb16d9d640 1 --2- 172.21.15.19:0/2870862910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb180ce670 0x7fbb180ceb00 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fbb180d1500 tx=0x7fbb0802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4756a427f542924 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1559a640 1 -- 172.21.15.19:0/2870862910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08036070 con 0x7fbb180ce670 2024-04-25T10:04:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1559a640 1 -- 172.21.15.19:0/2870862910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb08031600 con 0x7fbb180ce670 2024-04-25T10:04:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1559a640 1 -- 172.21.15.19:0/2870862910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb080319d0 con 0x7fbb180ce670 2024-04-25T10:04:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/2870862910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb180ce670 msgr2=0x7fbb180ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1ca3a640 1 --2- 172.21.15.19:0/2870862910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb180ce670 0x7fbb180ceb00 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fbb180d1500 tx=0x7fbb0802a610 comp rx=0 tx=0).stop 2024-04-25T10:04:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/2870862910 shutdown_connections 2024-04-25T10:04:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1ca3a640 1 --2- 172.21.15.19:0/2870862910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb180d1ce0 0x7fbb180b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1ca3a640 1 --2- 172.21.15.19:0/2870862910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb180ce670 0x7fbb180ceb00 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1ca3a640 1 --2- 172.21.15.19:0/2870862910 >> v2:172.21.15.112:3300/0 conn(0x7fbb1815dc20 0x7fbb180d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/2870862910 >> 172.21.15.19:0/2870862910 conn(0x7fbb18061a20 msgr2=0x7fbb18061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/2870862910 shutdown_connections 2024-04-25T10:04:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.989+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/2870862910 wait complete. 2024-04-25T10:04:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.991+0000 7fbb1ca3a640 1 Processor -- start 2024-04-25T10:04:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.991+0000 7fbb1ca3a640 1 -- start start 2024-04-25T10:04:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.991+0000 7fbb1ca3a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb180ce670 0x7fbb180b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.991+0000 7fbb1ca3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb180d1ce0 0x7fbb180c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.991+0000 7fbb1ca3a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1815dc20 0x7fbb180c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.991+0000 7fbb1ca3a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1806c120 con 0x7fbb1815dc20 2024-04-25T10:04:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.991+0000 7fbb1ca3a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180d2f20 con 0x7fbb180d1ce0 2024-04-25T10:04:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.991+0000 7fbb1ca3a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1806c2a0 con 0x7fbb180ce670 2024-04-25T10:04:58.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb16d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1815dc20 0x7fbb180c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb16d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1815dc20 0x7fbb180c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53178/0 (socket says 172.21.15.19:53178) 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb16d9d640 1 -- 172.21.15.19:0/3767203539 learned_addr learned my addr 172.21.15.19:0/3767203539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb15d9b640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb180d1ce0 0x7fbb180c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb1659c640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb180ce670 0x7fbb180b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb16d9d640 1 -- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb180ce670 msgr2=0x7fbb180b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb16d9d640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb180ce670 0x7fbb180b7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb16d9d640 1 -- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb180d1ce0 msgr2=0x7fbb180c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb16d9d640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb180d1ce0 0x7fbb180c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb16d9d640 1 -- 172.21.15.19:0/3767203539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1815e3e0 con 0x7fbb1815dc20 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb15d9b640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb180d1ce0 0x7fbb180c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb1659c640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb180ce670 0x7fbb180b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb16d9d640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1815dc20 0x7fbb180c84e0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009f40 tx=0x7fbb08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb0effd640 1 -- 172.21.15.19:0/3767203539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08036050 con 0x7fbb1815dc20 2024-04-25T10:04:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb0effd640 1 -- 172.21.15.19:0/3767203539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb08042070 con 0x7fbb1815dc20 2024-04-25T10:04:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/3767203539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1815e6c0 con 0x7fbb1815dc20 2024-04-25T10:04:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.992+0000 7fbb0effd640 1 -- 172.21.15.19:0/3767203539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08009510 con 0x7fbb1815dc20 2024-04-25T10:04:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.993+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/3767203539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1815ef60 con 0x7fbb1815dc20 2024-04-25T10:04:58.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.994+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/3767203539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1815e890 con 0x7fbb1815dc20 2024-04-25T10:04:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.994+0000 7fbb0effd640 1 -- 172.21.15.19:0/3767203539 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb08007c70 con 0x7fbb1815dc20 2024-04-25T10:04:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.994+0000 7fbb0effd640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbaf806b4d0 0x7fbaf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.995+0000 7fbb0effd640 1 -- 172.21.15.19:0/3767203539 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fbb080ab200 con 0x7fbb1815dc20 2024-04-25T10:04:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.995+0000 7fbb1659c640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbaf806b4d0 0x7fbaf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.995+0000 7fbb1659c640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbaf806b4d0 0x7fbaf806d990 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009a70 tx=0x7fbb10008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:57.999+0000 7fbb0effd640 1 -- 172.21.15.19:0/3767203539 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb08079e90 con 0x7fbb1815dc20 2024-04-25T10:04:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.181+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/3767203539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb18073a00 con 0x7fbb1815dc20 2024-04-25T10:04:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.182+0000 7fbb0effd640 1 -- 172.21.15.19:0/3767203539 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb08079790 con 0x7fbb1815dc20 2024-04-25T10:04:58.187 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:04:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbaf806b4d0 msgr2=0x7fbaf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbaf806b4d0 0x7fbaf806d990 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009a70 tx=0x7fbb10008040 comp rx=0 tx=0).stop 2024-04-25T10:04:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/3767203539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1815dc20 msgr2=0x7fbb180c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1815dc20 0x7fbb180c84e0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009f40 tx=0x7fbb08004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/3767203539 shutdown_connections 2024-04-25T10:04:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbaf806b4d0 0x7fbaf806d990 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1815dc20 0x7fbb180c84e0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb180d1ce0 0x7fbb180c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 --2- 172.21.15.19:0/3767203539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb180ce670 0x7fbb180b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/3767203539 >> 172.21.15.19:0/3767203539 conn(0x7fbb18061a20 msgr2=0x7fbb180b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/3767203539 shutdown_connections 2024-04-25T10:04:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.184+0000 7fbb1ca3a640 1 -- 172.21.15.19:0/3767203539 wait complete. 2024-04-25T10:04:58.200 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:04:58.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.298+0000 7fcccac07640 1 Processor -- start 2024-04-25T10:04:58.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.299+0000 7fcccac07640 1 -- start start 2024-04-25T10:04:58.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.299+0000 7fcccac07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc4161240 0x7fccc40b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.299+0000 7fcccac07640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fccc40b54a0 0x7fccc40b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.299+0000 7fcccac07640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc40cf1d0 con 0x7fccc4161890 2024-04-25T10:04:58.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.299+0000 7fcccac07640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc40b7120 con 0x7fccc40b54a0 2024-04-25T10:04:58.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.299+0000 7fcccac07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4161620 con 0x7fccc4161240 2024-04-25T10:04:58.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.300+0000 7fccc91a4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fccc40b54a0 0x7fccc40b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.300+0000 7fccc91a4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fccc40b54a0 0x7fccc40b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53180/0 (socket says 172.21.15.19:53180) 2024-04-25T10:04:58.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.300+0000 7fccc91a4640 1 -- 172.21.15.19:0/3821073720 learned_addr learned my addr 172.21.15.19:0/3821073720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:58.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.300+0000 7fccc89a3640 1 --2- 172.21.15.19:0/3821073720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc4161240 0x7fccc40b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.300+0000 7fccc91a4640 1 -- 172.21.15.19:0/3821073720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc4161240 msgr2=0x7fccc40b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.300+0000 7fccc91a4640 1 --2- 172.21.15.19:0/3821073720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc4161240 0x7fccc40b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.300+0000 7fccc91a4640 1 -- 172.21.15.19:0/3821073720 >> v1:172.21.15.112:6789/0 conn(0x7fccc4161890 legacy=0x7fccc40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.300+0000 7fccc91a4640 1 -- 172.21.15.19:0/3821073720 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4161e00 con 0x7fccc40b54a0 2024-04-25T10:04:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.300+0000 7fccc89a3640 1 --2- 172.21.15.19:0/3821073720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc4161240 0x7fccc40b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fccc91a4640 1 --2- 172.21.15.19:0/3821073720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40b54a0 0x7fccc40b5930 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009e50 tx=0x7fccbc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ddc2bce4c4cbdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fccc37fe640 1 -- 172.21.15.19:0/3821073720 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc036070 con 0x7fccc40b54a0 2024-04-25T10:04:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fccc37fe640 1 -- 172.21.15.19:0/3821073720 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc031600 con 0x7fccc40b54a0 2024-04-25T10:04:58.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fccc37fe640 1 -- 172.21.15.19:0/3821073720 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc0319d0 con 0x7fccc40b54a0 2024-04-25T10:04:58.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fcccac07640 1 -- 172.21.15.19:0/3821073720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40b54a0 msgr2=0x7fccc40b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fcccac07640 1 --2- 172.21.15.19:0/3821073720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40b54a0 0x7fccc40b5930 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009e50 tx=0x7fccbc02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fcccac07640 1 -- 172.21.15.19:0/3821073720 shutdown_connections 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fcccac07640 1 --2- 172.21.15.19:0/3821073720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40b54a0 0x7fccc40b5930 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fcccac07640 1 --2- 172.21.15.19:0/3821073720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc4161240 0x7fccc40b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fcccac07640 1 -- 172.21.15.19:0/3821073720 >> 172.21.15.19:0/3821073720 conn(0x7fccc40d04a0 msgr2=0x7fccc40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fcccac07640 1 -- 172.21.15.19:0/3821073720 shutdown_connections 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.301+0000 7fcccac07640 1 -- 172.21.15.19:0/3821073720 wait complete. 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.302+0000 7fcccac07640 1 Processor -- start 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.303+0000 7fcccac07640 1 -- start start 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.303+0000 7fcccac07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40b54a0 0x7fccc40cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.303+0000 7fcccac07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc4161240 0x7fccc41f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.303+0000 7fcccac07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc4161890 0x7fccc414ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.303+0000 7fcccac07640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc41626a0 con 0x7fccc40b54a0 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.303+0000 7fcccac07640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc41623c0 con 0x7fccc4161890 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.303+0000 7fcccac07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40d15f0 con 0x7fccc4161240 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.303+0000 7fccc89a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40b54a0 0x7fccc40cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.303+0000 7fccc89a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40b54a0 0x7fccc40cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53184/0 (socket says 172.21.15.19:53184) 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.303+0000 7fccc89a3640 1 -- 172.21.15.19:0/567602087 learned_addr learned my addr 172.21.15.19:0/567602087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fccc91a4640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc4161890 0x7fccc414ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fccc3fff640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc4161240 0x7fccc41f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fccc89a3640 1 -- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc4161240 msgr2=0x7fccc41f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fccc89a3640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc4161240 0x7fccc41f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fccc89a3640 1 -- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc4161890 msgr2=0x7fccc414ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fccc89a3640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc4161890 0x7fccc414ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fccc89a3640 1 -- 172.21.15.19:0/567602087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc41f7670 con 0x7fccc40b54a0 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fccc89a3640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40b54a0 0x7fccc40cb420 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fccb800c7f0 tx=0x7fccb800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fccc17fa640 1 -- 172.21.15.19:0/567602087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8015070 con 0x7fccc40b54a0 2024-04-25T10:04:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fccc17fa640 1 -- 172.21.15.19:0/567602087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb8007a30 con 0x7fccc40b54a0 2024-04-25T10:04:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fccc17fa640 1 -- 172.21.15.19:0/567602087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8007e00 con 0x7fccc40b54a0 2024-04-25T10:04:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fcccac07640 1 -- 172.21.15.19:0/567602087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc41f7950 con 0x7fccc40b54a0 2024-04-25T10:04:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.304+0000 7fcccac07640 1 -- 172.21.15.19:0/567602087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc40caf00 con 0x7fccc40b54a0 2024-04-25T10:04:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.305+0000 7fcccac07640 1 -- 172.21.15.19:0/567602087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc40cb100 con 0x7fccc40b54a0 2024-04-25T10:04:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.306+0000 7fccc17fa640 1 -- 172.21.15.19:0/567602087 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccb8018070 con 0x7fccc40b54a0 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.306+0000 7fccc17fa640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 0x7fcca806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.306+0000 7fccc17fa640 1 -- 172.21.15.19:0/567602087 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fccb808a270 con 0x7fccc40b54a0 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.306+0000 7fccc3fff640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 0x7fcca806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.307+0000 7fccc3fff640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 0x7fcca806d990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fccb0006fd0 tx=0x7fccb0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.310+0000 7fccc17fa640 1 -- 172.21.15.19:0/567602087 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fccc40cb100 con 0x7fccc40b54a0 2024-04-25T10:04:58.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.487+0000 7fcccac07640 1 -- 172.21.15.19:0/567602087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fccc4073a00 con 0x7fccc40b54a0 2024-04-25T10:04:58.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.488+0000 7fccc17fa640 1 -- 172.21.15.19:0/567602087 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v167) v1 ==== 104+0+11 (secure 0 0 0) 0x7fccb8058930 con 0x7fccc40b54a0 2024-04-25T10:04:58.493 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:04:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 -- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 msgr2=0x7fcca806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 0x7fcca806d990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fccb0006fd0 tx=0x7fccb0008040 comp rx=0 tx=0).stop 2024-04-25T10:04:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 -- 172.21.15.19:0/567602087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40b54a0 msgr2=0x7fccc40cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40b54a0 0x7fccc40cb420 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fccb800c7f0 tx=0x7fccb800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 -- 172.21.15.19:0/567602087 shutdown_connections 2024-04-25T10:04:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca806b4d0 0x7fcca806d990 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc4161890 0x7fccc414ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc4161240 0x7fccc41f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 --2- 172.21.15.19:0/567602087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40b54a0 0x7fccc40cb420 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 -- 172.21.15.19:0/567602087 >> 172.21.15.19:0/567602087 conn(0x7fccc40d04a0 msgr2=0x7fccc40d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 -- 172.21.15.19:0/567602087 shutdown_connections 2024-04-25T10:04:58.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.490+0000 7fcccac07640 1 -- 172.21.15.19:0/567602087 wait complete. 2024-04-25T10:04:58.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:04:58.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.604+0000 7efc5c46f640 1 Processor -- start 2024-04-25T10:04:58.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.604+0000 7efc5c46f640 1 -- start start 2024-04-25T10:04:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.605+0000 7efc5c46f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc540cadb0 0x7efc540cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.605+0000 7efc5c46f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc540ccfe0 0x7efc540cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.605+0000 7efc5c46f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc540afc80 con 0x7efc5415de80 2024-04-25T10:04:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.605+0000 7efc5c46f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc540cb210 con 0x7efc540ccfe0 2024-04-25T10:04:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.605+0000 7efc5c46f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc540ce4d0 con 0x7efc540cadb0 2024-04-25T10:04:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.605+0000 7efc5a20b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efc5415de80 0x7efc540cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55400/0 (socket says 172.21.15.19:55400) 2024-04-25T10:04:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc5a20b640 1 -- 172.21.15.19:0/4098807403 learned_addr learned my addr 172.21.15.19:0/4098807403 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221989265 0 0) 0x7efc540afc80 con 0x7efc5415de80 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc3c003520 con 0x7efc5415de80 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59a0a640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc540cadb0 0x7efc540cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc5aa0c640 1 --2- 172.21.15.19:0/4098807403 >> v2:172.21.15.112:3300/0 conn(0x7efc540ccfe0 0x7efc540cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3447982291 0 0) 0x7efc3c003520 con 0x7efc5415de80 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc540cadb0 msgr2=0x7efc540cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc540cadb0 0x7efc540cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 >> v2:172.21.15.112:3300/0 conn(0x7efc540ccfe0 msgr2=0x7efc540cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 --2- 172.21.15.19:0/4098807403 >> v2:172.21.15.112:3300/0 conn(0x7efc540ccfe0 0x7efc540cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc540ce7b0 con 0x7efc5415de80 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7efc4c001670 con 0x7efc5415de80 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 >> v1:172.21.15.19:6789/0 conn(0x7efc5415de80 legacy=0x7efc540cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c0039b0 0x7efc3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc3c004800 0x7efc3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc540cadb0 0x7efc3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc540cb210 con 0x7efc3c0039b0 2024-04-25T10:04:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc540ce4d0 con 0x7efc3c004800 2024-04-25T10:04:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c00b9c0 con 0x7efc540cadb0 2024-04-25T10:04:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc5a20b640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c0039b0 0x7efc3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc5aa0c640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc540cadb0 0x7efc3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.606+0000 7efc59a0a640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc3c004800 0x7efc3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.607+0000 7efc5a20b640 1 -- 172.21.15.19:0/4098807403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc540cadb0 msgr2=0x7efc3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.607+0000 7efc5a20b640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc540cadb0 0x7efc3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.607+0000 7efc5a20b640 1 -- 172.21.15.19:0/4098807403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc3c004800 msgr2=0x7efc3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.607+0000 7efc5a20b640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc3c004800 0x7efc3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.607+0000 7efc5a20b640 1 -- 172.21.15.19:0/4098807403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c003520 con 0x7efc3c0039b0 2024-04-25T10:04:58.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.607+0000 7efc5a20b640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c0039b0 0x7efc3c003dd0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7efc4c001050 tx=0x7efc3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c930c6541d99af24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.607+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c02e070 con 0x7efc3c0039b0 2024-04-25T10:04:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.607+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c005af0 con 0x7efc3c0039b0 2024-04-25T10:04:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.607+0000 7efc59209640 1 -- 172.21.15.19:0/4098807403 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c007d70 con 0x7efc3c0039b0 2024-04-25T10:04:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.608+0000 7efc5c46f640 1 -- 172.21.15.19:0/4098807403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c0039b0 msgr2=0x7efc3c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.608+0000 7efc5c46f640 1 --2- 172.21.15.19:0/4098807403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc3c0039b0 0x7efc3c003dd0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7efc4c001050 tx=0x7efc3c005830 comp rx=0 tx=0).stop 2024-04-25T10:04:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.608+0000 7efc5aa0c640 1 -- 172.21.15.19:0/4098807403 reap_dead start 2024-04-25T10:04:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.608+0000 7efc5c46f640 1 -- 172.21.15.19:0/4098807403 shutdown_connections 2024-04-25T10:04:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.608+0000 7efc5c46f640 1 -- 172.21.15.19:0/4098807403 >> 172.21.15.19:0/4098807403 conn(0x7efc540b2390 msgr2=0x7efc540b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.608+0000 7efc5c46f640 1 -- 172.21.15.19:0/4098807403 shutdown_connections 2024-04-25T10:04:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.608+0000 7efc5c46f640 1 -- 172.21.15.19:0/4098807403 wait complete. 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.609+0000 7efc5c46f640 1 Processor -- start 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.609+0000 7efc5c46f640 1 -- start start 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.610+0000 7efc5c46f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc3c0039b0 0x7efc5415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.610+0000 7efc5c46f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc540c7c70 0x7efc54153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.610+0000 7efc5c46f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc540cf730 0x7efc540cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.610+0000 7efc5c46f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc540b2eb0 con 0x7efc540cf730 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.610+0000 7efc5c46f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5415e260 con 0x7efc540c7c70 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.610+0000 7efc5c46f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc540b3030 con 0x7efc3c0039b0 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.610+0000 7efc5aa0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc540cf730 0x7efc540cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.610+0000 7efc5aa0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc540cf730 0x7efc540cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53194/0 (socket says 172.21.15.19:53194) 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.610+0000 7efc5aa0c640 1 -- 172.21.15.19:0/723231934 learned_addr learned my addr 172.21.15.19:0/723231934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.610+0000 7efc5a20b640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc3c0039b0 0x7efc5415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.610+0000 7efc59a0a640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc540c7c70 0x7efc54153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc5aa0c640 1 -- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc3c0039b0 msgr2=0x7efc5415ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc5aa0c640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc3c0039b0 0x7efc5415ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc5aa0c640 1 -- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc540c7c70 msgr2=0x7efc54153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc5aa0c640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc540c7c70 0x7efc54153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc5aa0c640 1 -- 172.21.15.19:0/723231934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc540b45f0 con 0x7efc540cf730 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc59a0a640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc540c7c70 0x7efc54153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc5a20b640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc3c0039b0 0x7efc5415ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc5aa0c640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc540cf730 0x7efc540cfbb0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7efc44011bf0 tx=0x7efc4400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc52ffd640 1 -- 172.21.15.19:0/723231934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4400fe10 con 0x7efc540cf730 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc52ffd640 1 -- 172.21.15.19:0/723231934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44015430 con 0x7efc540cf730 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc5c46f640 1 -- 172.21.15.19:0/723231934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc54153bb0 con 0x7efc540cf730 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc52ffd640 1 -- 172.21.15.19:0/723231934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44015820 con 0x7efc540cf730 2024-04-25T10:04:58.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.611+0000 7efc5c46f640 1 -- 172.21.15.19:0/723231934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc540c86d0 con 0x7efc540cf730 2024-04-25T10:04:58.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.612+0000 7efc5c46f640 1 -- 172.21.15.19:0/723231934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc54073a00 con 0x7efc540cf730 2024-04-25T10:04:58.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.612+0000 7efc52ffd640 1 -- 172.21.15.19:0/723231934 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc440159c0 con 0x7efc540cf730 2024-04-25T10:04:58.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.613+0000 7efc52ffd640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc4006b500 0x7efc4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.613+0000 7efc52ffd640 1 -- 172.21.15.19:0/723231934 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7efc4408b0f0 con 0x7efc540cf730 2024-04-25T10:04:58.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.613+0000 7efc5a20b640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc4006b500 0x7efc4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.614+0000 7efc5a20b640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc4006b500 0x7efc4006d9c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7efc4c033000 tx=0x7efc4c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.617+0000 7efc52ffd640 1 -- 172.21.15.19:0/723231934 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc54153bb0 con 0x7efc540cf730 2024-04-25T10:04:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.798+0000 7efc5c46f640 1 -- 172.21.15.19:0/723231934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efc540cef30 con 0x7efc540cf730 2024-04-25T10:04:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.798+0000 7efc52ffd640 1 -- 172.21.15.19:0/723231934 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v167) v1 ==== 100+0+11 (secure 0 0 0) 0x7efc540b45f0 con 0x7efc540cf730 2024-04-25T10:04:58.803 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:04:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 -- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc4006b500 msgr2=0x7efc4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc4006b500 0x7efc4006d9c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7efc4c033000 tx=0x7efc4c009030 comp rx=0 tx=0).stop 2024-04-25T10:04:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 -- 172.21.15.19:0/723231934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc540cf730 msgr2=0x7efc540cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc540cf730 0x7efc540cfbb0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7efc44011bf0 tx=0x7efc4400f580 comp rx=0 tx=0).stop 2024-04-25T10:04:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 -- 172.21.15.19:0/723231934 shutdown_connections 2024-04-25T10:04:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efc4006b500 0x7efc4006d9c0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc540cf730 0x7efc540cfbb0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc540c7c70 0x7efc54153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 --2- 172.21.15.19:0/723231934 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc3c0039b0 0x7efc5415ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 -- 172.21.15.19:0/723231934 >> 172.21.15.19:0/723231934 conn(0x7efc540b2390 msgr2=0x7efc5406e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 -- 172.21.15.19:0/723231934 shutdown_connections 2024-04-25T10:04:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.800+0000 7efc5c46f640 1 -- 172.21.15.19:0/723231934 wait complete. 2024-04-25T10:04:58.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:04:58.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.912+0000 7f0389c7d640 1 Processor -- start 2024-04-25T10:04:58.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.912+0000 7f0389c7d640 1 -- start start 2024-04-25T10:04:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.913+0000 7f0389c7d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0384161710 0x7f03840d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.913+0000 7f0389c7d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038415e000 con 0x7f03840ced80 2024-04-25T10:04:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.913+0000 7f0389c7d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03840d1de0 con 0x7f038415dc20 2024-04-25T10:04:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.913+0000 7f0389c7d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0384161c60 con 0x7f0384161710 2024-04-25T10:04:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.913+0000 7f0382ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f03840ced80 0x7f03840d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55406/0 (socket says 172.21.15.19:55406) 2024-04-25T10:04:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.913+0000 7f0382ffd640 1 -- 172.21.15.19:0/2509890258 learned_addr learned my addr 172.21.15.19:0/2509890258 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.913+0000 7f0383fff640 1 --2- 172.21.15.19:0/2509890258 >> v2:172.21.15.112:3300/0 conn(0x7f0384161710 0x7f03840d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.914+0000 7f03827fc640 1 -- 172.21.15.19:0/2509890258 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47307269 0 0) 0x7f038415e000 con 0x7f03840ced80 2024-04-25T10:04:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.914+0000 7f03827fc640 1 -- 172.21.15.19:0/2509890258 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f036c003520 con 0x7f03840ced80 2024-04-25T10:04:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.914+0000 7f03827fc640 1 -- 172.21.15.19:0/2509890258 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548920091 0 0) 0x7f03840d1de0 con 0x7f038415dc20 2024-04-25T10:04:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.914+0000 7f03827fc640 1 -- 172.21.15.19:0/2509890258 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f038415e000 con 0x7f038415dc20 2024-04-25T10:04:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.914+0000 7f0383fff640 1 -- 172.21.15.19:0/2509890258 >> v1:172.21.15.112:6789/0 conn(0x7f038415dc20 legacy=0x7f03840ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.914+0000 7f0383fff640 1 -- 172.21.15.19:0/2509890258 >> v1:172.21.15.19:6789/0 conn(0x7f03840ced80 legacy=0x7f03840d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.914+0000 7f0383fff640 1 -- 172.21.15.19:0/2509890258 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0384161f40 con 0x7f0384161710 2024-04-25T10:04:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.915+0000 7f0383fff640 1 --2- 172.21.15.19:0/2509890258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0384161710 0x7f03840d29f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f037c009d90 tx=0x7f037c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95dc1412efe6b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.915+0000 7f03827fc640 1 -- 172.21.15.19:0/2509890258 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c007740 con 0x7f0384161710 2024-04-25T10:04:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.915+0000 7f03827fc640 1 -- 172.21.15.19:0/2509890258 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c0078e0 con 0x7f0384161710 2024-04-25T10:04:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.915+0000 7f03827fc640 1 -- 172.21.15.19:0/2509890258 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c007cb0 con 0x7f0384161710 2024-04-25T10:04:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.915+0000 7f0389c7d640 1 -- 172.21.15.19:0/2509890258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0384161710 msgr2=0x7f03840d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.915+0000 7f0389c7d640 1 --2- 172.21.15.19:0/2509890258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0384161710 0x7f03840d29f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f037c009d90 tx=0x7f037c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.915+0000 7f0389c7d640 1 -- 172.21.15.19:0/2509890258 shutdown_connections 2024-04-25T10:04:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.915+0000 7f0389c7d640 1 --2- 172.21.15.19:0/2509890258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0384161710 0x7f03840d29f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.915+0000 7f0389c7d640 1 -- 172.21.15.19:0/2509890258 >> 172.21.15.19:0/2509890258 conn(0x7f03840d0420 msgr2=0x7f03840d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.915+0000 7f0389c7d640 1 -- 172.21.15.19:0/2509890258 shutdown_connections 2024-04-25T10:04:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.916+0000 7f0389c7d640 1 -- 172.21.15.19:0/2509890258 wait complete. 2024-04-25T10:04:58.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.917+0000 7f0389c7d640 1 Processor -- start 2024-04-25T10:04:58.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.917+0000 7f0389c7d640 1 -- start start 2024-04-25T10:04:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.917+0000 7f0389c7d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03840ced80 0x7f0384160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.917+0000 7f0389c7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f038415dc20 0x7f03841e9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0389c7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0384161710 0x7f038414b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0389c7d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03840b5e50 con 0x7f0384161710 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0389c7d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0384162500 con 0x7f038415dc20 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0389c7d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03840b5fd0 con 0x7f03840ced80 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0383fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0384161710 0x7f038414b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0383fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0384161710 0x7f038414b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53210/0 (socket says 172.21.15.19:53210) 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0383fff640 1 -- 172.21.15.19:0/1152410894 learned_addr learned my addr 172.21.15.19:0/1152410894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0382ffd640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f038415dc20 0x7f03841e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f03837fe640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03840ced80 0x7f0384160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0383fff640 1 -- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03840ced80 msgr2=0x7f0384160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0383fff640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03840ced80 0x7f0384160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0383fff640 1 -- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f038415dc20 msgr2=0x7f03841e9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0383fff640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f038415dc20 0x7f03841e9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0383fff640 1 -- 172.21.15.19:0/1152410894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03840c7320 con 0x7f0384161710 2024-04-25T10:04:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0383fff640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0384161710 0x7f038414b920 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f037c009eb0 tx=0x7f037c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f036bfff640 1 -- 172.21.15.19:0/1152410894 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c040070 con 0x7f0384161710 2024-04-25T10:04:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f036bfff640 1 -- 172.21.15.19:0/1152410894 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c031bc0 con 0x7f0384161710 2024-04-25T10:04:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0389c7d640 1 -- 172.21.15.19:0/1152410894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038415f7b0 con 0x7f0384161710 2024-04-25T10:04:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f036bfff640 1 -- 172.21.15.19:0/1152410894 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c009450 con 0x7f0384161710 2024-04-25T10:04:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.918+0000 7f0389c7d640 1 -- 172.21.15.19:0/1152410894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038415fd70 con 0x7f0384161710 2024-04-25T10:04:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.919+0000 7f0389c7d640 1 -- 172.21.15.19:0/1152410894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03840c74f0 con 0x7f0384161710 2024-04-25T10:04:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.920+0000 7f036bfff640 1 -- 172.21.15.19:0/1152410894 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f037c039040 con 0x7f0384161710 2024-04-25T10:04:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.920+0000 7f036bfff640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f037006b4d0 0x7f037006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.920+0000 7f036bfff640 1 -- 172.21.15.19:0/1152410894 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f037c0ab970 con 0x7f0384161710 2024-04-25T10:04:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.920+0000 7f03837fe640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f037006b4d0 0x7f037006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.921+0000 7f03837fe640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f037006b4d0 0x7f037006d990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0374007fd0 tx=0x7f0374009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:58.925+0000 7f036bfff640 1 -- 172.21.15.19:0/1152410894 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f037c07a0c0 con 0x7f0384161710 2024-04-25T10:04:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.119+0000 7f0389c7d640 1 -- 172.21.15.19:0/1152410894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0384073a00 con 0x7f0384161710 2024-04-25T10:04:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.120+0000 7f036bfff640 1 -- 172.21.15.19:0/1152410894 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v167) v1 ==== 80+0+0 (secure 0 0 0) 0x7f037c07e160 con 0x7f0384161710 2024-04-25T10:04:59.125 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T10:04:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 -- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f037006b4d0 msgr2=0x7f037006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f037006b4d0 0x7f037006d990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0374007fd0 tx=0x7f0374009040 comp rx=0 tx=0).stop 2024-04-25T10:04:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 -- 172.21.15.19:0/1152410894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0384161710 msgr2=0x7f038414b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0384161710 0x7f038414b920 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f037c009eb0 tx=0x7f037c004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 -- 172.21.15.19:0/1152410894 shutdown_connections 2024-04-25T10:04:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f037006b4d0 0x7f037006d990 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0384161710 0x7f038414b920 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f038415dc20 0x7f03841e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 --2- 172.21.15.19:0/1152410894 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03840ced80 0x7f0384160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 -- 172.21.15.19:0/1152410894 >> 172.21.15.19:0/1152410894 conn(0x7f03840d0420 msgr2=0x7f03840b79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:59.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 -- 172.21.15.19:0/1152410894 shutdown_connections 2024-04-25T10:04:59.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.122+0000 7f0389c7d640 1 -- 172.21.15.19:0/1152410894 wait complete. 2024-04-25T10:04:59.138 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:04:59.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:59.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:59.176 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:04:59.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:59.176 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:04:59.231 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:04:59.232 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:59.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:59.249 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:04:59.249 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:59.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:59.287 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:04:59.287 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:59.305 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:59.305 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:59.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.422+0000 7fc5c13b6640 1 Processor -- start 2024-04-25T10:04:59.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.422+0000 7fc5c13b6640 1 -- start start 2024-04-25T10:04:59.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.422+0000 7fc5c13b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc15dbe0 0x7fc5bc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.422+0000 7fc5c13b6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5bc0b2cb0 0x7fc5bc0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.422+0000 7fc5c13b6640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5bc0b97a0 con 0x7fc5bc0b92a0 2024-04-25T10:04:59.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.422+0000 7fc5c13b6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc15dfc0 con 0x7fc5bc0b2cb0 2024-04-25T10:04:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.422+0000 7fc5c13b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0d1a00 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5bb7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc5bc0b92a0 0x7fc5bc0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55412/0 (socket says 172.21.15.19:55412) 2024-04-25T10:04:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5bb7fe640 1 -- 172.21.15.19:0/2638884183 learned_addr learned my addr 172.21.15.19:0/2638884183 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/2638884183 >> v2:172.21.15.112:3300/0 conn(0x7fc5bc0b2cb0 0x7fc5bc0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5baffd640 1 --2- 172.21.15.19:0/2638884183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc15dbe0 0x7fc5bc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5b9ffb640 1 -- 172.21.15.19:0/2638884183 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941688308 0 0) 0x7fc5bc0b97a0 con 0x7fc5bc0b92a0 2024-04-25T10:04:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5b9ffb640 1 -- 172.21.15.19:0/2638884183 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a4003520 con 0x7fc5bc0b92a0 2024-04-25T10:04:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5b9ffb640 1 -- 172.21.15.19:0/2638884183 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1275020605 0 0) 0x7fc5a4003520 con 0x7fc5bc0b92a0 2024-04-25T10:04:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5baffd640 1 -- 172.21.15.19:0/2638884183 >> v2:172.21.15.112:3300/0 conn(0x7fc5bc0b2cb0 msgr2=0x7fc5bc0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5baffd640 1 --2- 172.21.15.19:0/2638884183 >> v2:172.21.15.112:3300/0 conn(0x7fc5bc0b2cb0 0x7fc5bc0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5baffd640 1 -- 172.21.15.19:0/2638884183 >> v1:172.21.15.19:6789/0 conn(0x7fc5bc0b92a0 legacy=0x7fc5bc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5baffd640 1 -- 172.21.15.19:0/2638884183 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc0d1c50 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.423+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/2638884183 >> v2:172.21.15.112:3300/0 conn(0x7fc5bc0b2cb0 0x7fc5bc0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:59.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.424+0000 7fc5baffd640 1 --2- 172.21.15.19:0/2638884183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc15dbe0 0x7fc5bc0d0910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009d90 tx=0x7fc5ac02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60ace8a5ed099bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:59.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.424+0000 7fc5b9ffb640 1 -- 172.21.15.19:0/2638884183 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4001670 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.424+0000 7fc5b9ffb640 1 -- 172.21.15.19:0/2638884183 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac007540 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.424+0000 7fc5b9ffb640 1 -- 172.21.15.19:0/2638884183 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0078f0 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.425+0000 7fc5c13b6640 1 -- 172.21.15.19:0/2638884183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc15dbe0 msgr2=0x7fc5bc0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.425+0000 7fc5c13b6640 1 --2- 172.21.15.19:0/2638884183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc15dbe0 0x7fc5bc0d0910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009d90 tx=0x7fc5ac02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.425+0000 7fc5c13b6640 1 -- 172.21.15.19:0/2638884183 shutdown_connections 2024-04-25T10:04:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.425+0000 7fc5c13b6640 1 --2- 172.21.15.19:0/2638884183 >> v2:172.21.15.112:3300/0 conn(0x7fc5bc0b2cb0 0x7fc5bc0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.425+0000 7fc5c13b6640 1 --2- 172.21.15.19:0/2638884183 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc15dbe0 0x7fc5bc0d0910 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.425+0000 7fc5c13b6640 1 -- 172.21.15.19:0/2638884183 >> 172.21.15.19:0/2638884183 conn(0x7fc5bc061a20 msgr2=0x7fc5bc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.425+0000 7fc5c13b6640 1 -- 172.21.15.19:0/2638884183 shutdown_connections 2024-04-25T10:04:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.425+0000 7fc5c13b6640 1 -- 172.21.15.19:0/2638884183 wait complete. 2024-04-25T10:04:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.426+0000 7fc5c13b6640 1 Processor -- start 2024-04-25T10:04:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.426+0000 7fc5c13b6640 1 -- start start 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5c13b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5bc0b2cb0 0x7fc5bc0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5c13b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc0b92a0 0x7fc5bc0b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5c13b6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc15dbe0 0x7fc5bc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5c13b6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b6ab0 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5c13b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0d2210 con 0x7fc5bc0b2cb0 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5c13b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b6c30 con 0x7fc5bc0b92a0 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc15dbe0 0x7fc5bc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc15dbe0 0x7fc5bc0b8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53218/0 (socket says 172.21.15.19:53218) 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5bb7fe640 1 -- 172.21.15.19:0/3860016166 learned_addr learned my addr 172.21.15.19:0/3860016166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5baffd640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5bc0b2cb0 0x7fc5bc0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc0b92a0 0x7fc5bc0b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5bb7fe640 1 -- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc0b92a0 msgr2=0x7fc5bc0b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5bb7fe640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc0b92a0 0x7fc5bc0b85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5bb7fe640 1 -- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5bc0b2cb0 msgr2=0x7fc5bc0c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5bb7fe640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5bc0b2cb0 0x7fc5bc0c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5bb7fe640 1 -- 172.21.15.19:0/3860016166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc0cabf0 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5ba7fc640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc0b92a0 0x7fc5bc0b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.427+0000 7fc5bb7fe640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc15dbe0 0x7fc5bc0b8af0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c580 tx=0x7fc5b400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.428+0000 7fc5a37fe640 1 -- 172.21.15.19:0/3860016166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4017070 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.428+0000 7fc5a37fe640 1 -- 172.21.15.19:0/3860016166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b4005ce0 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.428+0000 7fc5c13b6640 1 -- 172.21.15.19:0/3860016166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc0cae90 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.428+0000 7fc5a37fe640 1 -- 172.21.15.19:0/3860016166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4007330 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.428+0000 7fc5c13b6640 1 -- 172.21.15.19:0/3860016166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc0cb450 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.429+0000 7fc5c13b6640 1 -- 172.21.15.19:0/3860016166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc073990 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.429+0000 7fc5a37fe640 1 -- 172.21.15.19:0/3860016166 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5b401a070 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.430+0000 7fc5a37fe640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 0x7fc5a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.430+0000 7fc5a37fe640 1 -- 172.21.15.19:0/3860016166 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc5b4089900 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.430+0000 7fc5baffd640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 0x7fc5a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.430+0000 7fc5baffd640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 0x7fc5a806d990 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac034000 tx=0x7fc5ac02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.434+0000 7fc5a37fe640 1 -- 172.21.15.19:0/3860016166 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5b4057f40 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.649+0000 7fc5c13b6640 1 -- 172.21.15.19:0/3860016166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5bc06c920 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.649+0000 7fc5a37fe640 1 -- 172.21.15.19:0/3860016166 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4954 (secure 0 0 0) 0x7fc5b405bfe0 con 0x7fc5bc15dbe0 2024-04-25T10:04:59.655 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:59.655 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1300145056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1300145056},{"type":"v1","addr":"172.21.15.19:6835","nonce":1300145056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":8027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2001687418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2001687418},{"type":"v1","addr":"172.21.15.112:6836","nonce":2001687418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721457153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721457153},{"type":"v1","addr":"172.21.15.112:6837","nonce":721457153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:14.252458+0000","modified":"2024-04-25T10:04:16.259643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10698":{"gid":10698,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3942829201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3942829201},{"type":"v1","addr":"172.21.15.19:6834","nonce":3942829201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:04:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 109 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 -- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 msgr2=0x7fc5a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 0x7fc5a806d990 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac034000 tx=0x7fc5ac02f000 comp rx=0 tx=0).stop 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 -- 172.21.15.19:0/3860016166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc15dbe0 msgr2=0x7fc5bc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc15dbe0 0x7fc5bc0b8af0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c580 tx=0x7fc5b400ca40 comp rx=0 tx=0).stop 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 -- 172.21.15.19:0/3860016166 shutdown_connections 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc5a806b4d0 0x7fc5a806d990 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5bc15dbe0 0x7fc5bc0b8af0 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5bc0b92a0 0x7fc5bc0b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 --2- 172.21.15.19:0/3860016166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5bc0b2cb0 0x7fc5bc0c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 -- 172.21.15.19:0/3860016166 >> 172.21.15.19:0/3860016166 conn(0x7fc5bc061a20 msgr2=0x7fc5bc15c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 -- 172.21.15.19:0/3860016166 shutdown_connections 2024-04-25T10:04:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.651+0000 7fc5c13b6640 1 -- 172.21.15.19:0/3860016166 wait complete. 2024-04-25T10:04:59.667 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.764+0000 7f42dfe07640 1 Processor -- start 2024-04-25T10:04:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.764+0000 7f42dfe07640 1 -- start start 2024-04-25T10:04:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.764+0000 7f42dfe07640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42d815dd30 0x7f42d80b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.764+0000 7f42dfe07640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f42d80d0ad0 0x7f42d80b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.764+0000 7f42dfe07640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d80d1df0 con 0x7f42d80b9770 2024-04-25T10:04:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42dfe07640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d80d20d0 con 0x7f42d815dd30 2024-04-25T10:04:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42dfe07640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d80d23b0 con 0x7f42d80d0ad0 2024-04-25T10:04:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42ddba3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42d815dd30 0x7f42d80b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42dd3a2640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f42d80b9770 0x7f42d80d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55418/0 (socket says 172.21.15.19:55418) 2024-04-25T10:04:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42dd3a2640 1 -- 172.21.15.19:0/979119518 learned_addr learned my addr 172.21.15.19:0/979119518 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:04:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42de3a4640 1 --2- 172.21.15.19:0/979119518 >> v2:172.21.15.112:3300/0 conn(0x7f42d80d0ad0 0x7f42d80b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42dcba1640 1 -- 172.21.15.19:0/979119518 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837109964 0 0) 0x7f42d80d1df0 con 0x7f42d80b9770 2024-04-25T10:04:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42dcba1640 1 -- 172.21.15.19:0/979119518 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42c4003520 con 0x7f42d80b9770 2024-04-25T10:04:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42ddba3640 1 -- 172.21.15.19:0/979119518 >> v2:172.21.15.112:3300/0 conn(0x7f42d80d0ad0 msgr2=0x7f42d80b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42ddba3640 1 --2- 172.21.15.19:0/979119518 >> v2:172.21.15.112:3300/0 conn(0x7f42d80d0ad0 0x7f42d80b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42ddba3640 1 -- 172.21.15.19:0/979119518 >> v1:172.21.15.19:6789/0 conn(0x7f42d80b9770 legacy=0x7f42d80d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.765+0000 7f42ddba3640 1 -- 172.21.15.19:0/979119518 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d80d2690 con 0x7f42d815dd30 2024-04-25T10:04:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.766+0000 7f42ddba3640 1 --2- 172.21.15.19:0/979119518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d815dd30 0x7f42d80b9230 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f42d80ce200 tx=0x7f42d402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365864fff13ced0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.766+0000 7f42dcba1640 1 -- 172.21.15.19:0/979119518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4037070 con 0x7f42d815dd30 2024-04-25T10:04:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.766+0000 7f42dcba1640 1 -- 172.21.15.19:0/979119518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d402add0 con 0x7f42d815dd30 2024-04-25T10:04:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.766+0000 7f42dcba1640 1 -- 172.21.15.19:0/979119518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4032640 con 0x7f42d815dd30 2024-04-25T10:04:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.766+0000 7f42dfe07640 1 -- 172.21.15.19:0/979119518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d815dd30 msgr2=0x7f42d80b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.766+0000 7f42dfe07640 1 --2- 172.21.15.19:0/979119518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d815dd30 0x7f42d80b9230 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f42d80ce200 tx=0x7f42d402a080 comp rx=0 tx=0).stop 2024-04-25T10:04:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.766+0000 7f42dfe07640 1 -- 172.21.15.19:0/979119518 shutdown_connections 2024-04-25T10:04:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.766+0000 7f42dfe07640 1 --2- 172.21.15.19:0/979119518 >> v2:172.21.15.112:3300/0 conn(0x7f42d80d0ad0 0x7f42d80b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.766+0000 7f42dfe07640 1 --2- 172.21.15.19:0/979119518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d815dd30 0x7f42d80b9230 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.766+0000 7f42dfe07640 1 -- 172.21.15.19:0/979119518 >> 172.21.15.19:0/979119518 conn(0x7f42d8061c30 msgr2=0x7f42d806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.767+0000 7f42dfe07640 1 -- 172.21.15.19:0/979119518 shutdown_connections 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.767+0000 7f42dfe07640 1 -- 172.21.15.19:0/979119518 wait complete. 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.768+0000 7f42dfe07640 1 Processor -- start 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.768+0000 7f42dfe07640 1 -- start start 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.768+0000 7f42dfe07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d80b9770 0x7f42d81543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.768+0000 7f42dfe07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42d80d0ad0 0x7f42d8152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.768+0000 7f42dfe07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42d815dd30 0x7f42d81f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.768+0000 7f42dfe07640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80d1b90 con 0x7f42d80b9770 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.768+0000 7f42dfe07640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80d1a10 con 0x7f42d815dd30 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.768+0000 7f42dfe07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80d2e90 con 0x7f42d80d0ad0 2024-04-25T10:04:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ddba3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d80b9770 0x7f42d81543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ddba3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d80b9770 0x7f42d81543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53238/0 (socket says 172.21.15.19:53238) 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ddba3640 1 -- 172.21.15.19:0/1190086255 learned_addr learned my addr 172.21.15.19:0/1190086255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ddba3640 1 -- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42d80d0ad0 msgr2=0x7f42d8152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ddba3640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42d80d0ad0 0x7f42d8152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ddba3640 1 -- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42d815dd30 msgr2=0x7f42d81f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ddba3640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42d815dd30 0x7f42d81f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ddba3640 1 -- 172.21.15.19:0/1190086255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d81f4020 con 0x7f42d80b9770 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ddba3640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d80b9770 0x7f42d81543c0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f42d4000c00 tx=0x7f42d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ce7fc640 1 -- 172.21.15.19:0/1190086255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4041030 con 0x7f42d80b9770 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ce7fc640 1 -- 172.21.15.19:0/1190086255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d402abd0 con 0x7f42d80b9770 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42dfe07640 1 -- 172.21.15.19:0/1190086255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d81f4300 con 0x7f42d80b9770 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42ce7fc640 1 -- 172.21.15.19:0/1190086255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4037040 con 0x7f42d80b9770 2024-04-25T10:04:59.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.769+0000 7f42dfe07640 1 -- 172.21.15.19:0/1190086255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d8153ea0 con 0x7f42d80b9770 2024-04-25T10:04:59.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.770+0000 7f42dfe07640 1 -- 172.21.15.19:0/1190086255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d8073990 con 0x7f42d80b9770 2024-04-25T10:04:59.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.771+0000 7f42ce7fc640 1 -- 172.21.15.19:0/1190086255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42d4044070 con 0x7f42d80b9770 2024-04-25T10:04:59.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.771+0000 7f42ce7fc640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.771+0000 7f42ce7fc640 1 -- 172.21.15.19:0/1190086255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f42d40aa700 con 0x7f42d80b9770 2024-04-25T10:04:59.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.771+0000 7f42dd3a2640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.772+0000 7f42dd3a2640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f42bc06b4d0 0x7f42bc06d990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f42c8007fd0 tx=0x7f42c80077c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:59.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.775+0000 7f42ce7fc640 1 -- 172.21.15.19:0/1190086255 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42d4078dc0 con 0x7f42d80b9770 2024-04-25T10:04:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.958+0000 7f42dfe07640 1 -- 172.21.15.19:0/1190086255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42d815d180 con 0x7f42d80b9770 2024-04-25T10:04:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.959+0000 7f42ce7fc640 1 -- 172.21.15.19:0/1190086255 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+17066 (secure 0 0 0) 0x7f42d407ce60 con 0x7f42d80b9770 2024-04-25T10:04:59.964 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:04:59.964 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":167,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:04:49.642130+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:12.518429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:13.232769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2c","mappings":[{"from":3,"to":0}]},{"pgid":"26.31","mappings":[{"from":1,"to":0}]},{"pgid":"26.37","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 -- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f42bc06b4d0 msgr2=0x7f42bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f42bc06b4d0 0x7f42bc06d990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f42c8007fd0 tx=0x7f42c80077c0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 -- 172.21.15.19:0/1190086255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d80b9770 msgr2=0x7f42d81543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d80b9770 0x7f42d81543c0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f42d4000c00 tx=0x7f42d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 -- 172.21.15.19:0/1190086255 shutdown_connections 2024-04-25T10:04:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42d815dd30 0x7f42d81f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42d80d0ad0 0x7f42d8152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 --2- 172.21.15.19:0/1190086255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d80b9770 0x7f42d81543c0 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 -- 172.21.15.19:0/1190086255 >> 172.21.15.19:0/1190086255 conn(0x7f42d8061c30 msgr2=0x7f42d815fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 -- 172.21.15.19:0/1190086255 shutdown_connections 2024-04-25T10:04:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:04:59.961+0000 7f42dfe07640 1 -- 172.21.15.19:0/1190086255 wait complete. 2024-04-25T10:04:59.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:04:12.518429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:04:13.232769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '165', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:59.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.076+0000 7fbb96522640 1 Processor -- start 2024-04-25T10:05:00.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.076+0000 7fbb96522640 1 -- start start 2024-04-25T10:05:00.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.077+0000 7fbb96522640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbb9015dc20 0x7fbb900ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.077+0000 7fbb96522640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9006cda0 con 0x7fbb90161710 2024-04-25T10:05:00.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.077+0000 7fbb96522640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9015e000 con 0x7fbb900d32f0 2024-04-25T10:05:00.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.077+0000 7fbb96522640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb900d1de0 con 0x7fbb9015dc20 2024-04-25T10:05:00.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.077+0000 7fbb8ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbb9015dc20 0x7fbb900ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.077+0000 7fbb8f7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbb90161710 0x7fbb9006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55420/0 (socket says 172.21.15.19:55420) 2024-04-25T10:05:00.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.077+0000 7fbb8f7fe640 1 -- 172.21.15.19:0/2130144833 learned_addr learned my addr 172.21.15.19:0/2130144833 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:00.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.077+0000 7fbb8ffff640 1 -- 172.21.15.19:0/2130144833 >> v1:172.21.15.112:6789/0 conn(0x7fbb900d32f0 legacy=0x7fbb9006c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.077+0000 7fbb8ffff640 1 -- 172.21.15.19:0/2130144833 >> v1:172.21.15.19:6789/0 conn(0x7fbb90161710 legacy=0x7fbb9006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.077+0000 7fbb8ffff640 1 -- 172.21.15.19:0/2130144833 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb90161c60 con 0x7fbb9015dc20 2024-04-25T10:05:00.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.077+0000 7fbb8effd640 1 -- 172.21.15.19:0/2130144833 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947708201 0 0) 0x7fbb9006cda0 con 0x7fbb90161710 2024-04-25T10:05:00.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.078+0000 7fbb8ffff640 1 --2- 172.21.15.19:0/2130144833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9015dc20 0x7fbb900ce670 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fbb780097a0 tx=0x7fbb7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618d1390e3cc4593 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.078+0000 7fbb8effd640 1 -- 172.21.15.19:0/2130144833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78037070 con 0x7fbb9015dc20 2024-04-25T10:05:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.078+0000 7fbb8effd640 1 -- 172.21.15.19:0/2130144833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb78032440 con 0x7fbb9015dc20 2024-04-25T10:05:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.078+0000 7fbb8effd640 1 -- 172.21.15.19:0/2130144833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78032810 con 0x7fbb9015dc20 2024-04-25T10:05:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.079+0000 7fbb96522640 1 -- 172.21.15.19:0/2130144833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9015dc20 msgr2=0x7fbb900ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.079+0000 7fbb96522640 1 --2- 172.21.15.19:0/2130144833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9015dc20 0x7fbb900ce670 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fbb780097a0 tx=0x7fbb7802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.079+0000 7fbb96522640 1 -- 172.21.15.19:0/2130144833 shutdown_connections 2024-04-25T10:05:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.079+0000 7fbb96522640 1 --2- 172.21.15.19:0/2130144833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9015dc20 0x7fbb900ce670 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.079+0000 7fbb96522640 1 -- 172.21.15.19:0/2130144833 >> 172.21.15.19:0/2130144833 conn(0x7fbb900d0420 msgr2=0x7fbb900d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.079+0000 7fbb96522640 1 -- 172.21.15.19:0/2130144833 shutdown_connections 2024-04-25T10:05:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.079+0000 7fbb96522640 1 -- 172.21.15.19:0/2130144833 wait complete. 2024-04-25T10:05:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.080+0000 7fbb96522640 1 Processor -- start 2024-04-25T10:05:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.080+0000 7fbb96522640 1 -- start start 2024-04-25T10:05:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb96522640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb900d32f0 0x7fbb900c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb96522640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9015dc20 0x7fbb9014b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb96522640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb90161710 0x7fbb9014bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb96522640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90162500 con 0x7fbb9015dc20 2024-04-25T10:05:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb96522640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90162220 con 0x7fbb900d32f0 2024-04-25T10:05:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb96522640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb900b5e50 con 0x7fbb90161710 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9015dc20 0x7fbb9014b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb8f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9015dc20 0x7fbb9014b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53246/0 (socket says 172.21.15.19:53246) 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb8f7fe640 1 -- 172.21.15.19:0/787943638 learned_addr learned my addr 172.21.15.19:0/787943638 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb8ffff640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb900d32f0 0x7fbb900c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb8f7fe640 1 -- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb90161710 msgr2=0x7fbb9014bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.081+0000 7fbb94abf640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb90161710 0x7fbb9014bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb8f7fe640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb90161710 0x7fbb9014bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb8f7fe640 1 -- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb900d32f0 msgr2=0x7fbb900c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb8f7fe640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb900d32f0 0x7fbb900c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb8f7fe640 1 -- 172.21.15.19:0/787943638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb901e9ed0 con 0x7fbb9015dc20 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb8ffff640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb900d32f0 0x7fbb900c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb94abf640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb90161710 0x7fbb9014bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb8f7fe640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9015dc20 0x7fbb9014b870 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c500 tx=0x7fbb8000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb8cff9640 1 -- 172.21.15.19:0/787943638 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80018070 con 0x7fbb9015dc20 2024-04-25T10:05:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb8cff9640 1 -- 172.21.15.19:0/787943638 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb80007580 con 0x7fbb9015dc20 2024-04-25T10:05:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb96522640 1 -- 172.21.15.19:0/787943638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb901e9910 con 0x7fbb9015dc20 2024-04-25T10:05:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb8cff9640 1 -- 172.21.15.19:0/787943638 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80004a20 con 0x7fbb9015dc20 2024-04-25T10:05:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.082+0000 7fbb96522640 1 -- 172.21.15.19:0/787943638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb901498f0 con 0x7fbb9015dc20 2024-04-25T10:05:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.083+0000 7fbb96522640 1 -- 172.21.15.19:0/787943638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb90073a00 con 0x7fbb9015dc20 2024-04-25T10:05:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.083+0000 7fbb8cff9640 1 -- 172.21.15.19:0/787943638 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb8001b070 con 0x7fbb9015dc20 2024-04-25T10:05:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.084+0000 7fbb8cff9640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb7006b4d0 0x7fbb7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.084+0000 7fbb8cff9640 1 -- 172.21.15.19:0/787943638 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fbb800896c0 con 0x7fbb9015dc20 2024-04-25T10:05:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.084+0000 7fbb8ffff640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb7006b4d0 0x7fbb7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.084+0000 7fbb8ffff640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb7006b4d0 0x7fbb7006d990 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbb78034000 tx=0x7fbb7802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.088+0000 7fbb8cff9640 1 -- 172.21.15.19:0/787943638 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb800583d0 con 0x7fbb9015dc20 2024-04-25T10:05:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.267+0000 7fbb96522640 1 -- 172.21.15.19:0/787943638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb9006b010 con 0x7fbb9015dc20 2024-04-25T10:05:00.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.267+0000 7fbb8cff9640 1 -- 172.21.15.19:0/787943638 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb80057cd0 con 0x7fbb9015dc20 2024-04-25T10:05:00.272 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:05:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 -- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb7006b4d0 msgr2=0x7fbb7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb7006b4d0 0x7fbb7006d990 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbb78034000 tx=0x7fbb7802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 -- 172.21.15.19:0/787943638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9015dc20 msgr2=0x7fbb9014b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9015dc20 0x7fbb9014b870 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c500 tx=0x7fbb8000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 -- 172.21.15.19:0/787943638 shutdown_connections 2024-04-25T10:05:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbb7006b4d0 0x7fbb7006d990 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb90161710 0x7fbb9014bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9015dc20 0x7fbb9014b870 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 --2- 172.21.15.19:0/787943638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb900d32f0 0x7fbb900c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 -- 172.21.15.19:0/787943638 >> 172.21.15.19:0/787943638 conn(0x7fbb900d0420 msgr2=0x7fbb900c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 -- 172.21.15.19:0/787943638 shutdown_connections 2024-04-25T10:05:00.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.269+0000 7fbb96522640 1 -- 172.21.15.19:0/787943638 wait complete. 2024-04-25T10:05:00.285 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:00.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.383+0000 7f953adb7640 1 Processor -- start 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.384+0000 7f953adb7640 1 -- start start 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.384+0000 7f953adb7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f953415dc20 0x7f95340ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.384+0000 7f953adb7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953406cda0 con 0x7f95340d32f0 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.384+0000 7f953adb7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953415e000 con 0x7f9534161710 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.384+0000 7f953adb7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95340d1de0 con 0x7f953415dc20 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.384+0000 7f9539354640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f95340d32f0 0x7f953406c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55426/0 (socket says 172.21.15.19:55426) 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.384+0000 7f9539354640 1 -- 172.21.15.19:0/2534500514 learned_addr learned my addr 172.21.15.19:0/2534500514 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f9538b53640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f953415dc20 0x7f95340ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925937569 0 0) 0x7f953406cda0 con 0x7f95340d32f0 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9520003520 con 0x7f95340d32f0 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 52073773 0 0) 0x7f9520003520 con 0x7f95340d32f0 2024-04-25T10:05:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f953415dc20 msgr2=0x7f95340ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f953415dc20 0x7f95340ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 >> v1:172.21.15.112:6789/0 conn(0x7f9534161710 legacy=0x7f953406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9534161c60 con 0x7f95340d32f0 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f952c0019f0 con 0x7f95340d32f0 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 >> v1:172.21.15.19:6789/0 conn(0x7f95340d32f0 legacy=0x7f953406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95200039e0 0x7f9520003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520004800 0x7f9520004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f953415dc20 0x7f95200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95340d1de0 con 0x7f9520004800 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952000ba70 con 0x7f95200039e0 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952000bc70 con 0x7f953415dc20 2024-04-25T10:05:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f9533fff640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520004800 0x7f9520004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f9538b53640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95200039e0 0x7f9520003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f9539354640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f953415dc20 0x7f95200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f9533fff640 1 -- 172.21.15.19:0/2534500514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f953415dc20 msgr2=0x7f95200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.385+0000 7f9533fff640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f953415dc20 0x7f95200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.386+0000 7f9533fff640 1 -- 172.21.15.19:0/2534500514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95200039e0 msgr2=0x7f9520003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.386+0000 7f9533fff640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95200039e0 0x7f9520003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.386+0000 7f9533fff640 1 -- 172.21.15.19:0/2534500514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9520003520 con 0x7f9520004800 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.386+0000 7f9533fff640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520004800 0x7f9520004c80 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f951c001720 tx=0x7f9520005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83828055cb9e0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.386+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c001370 con 0x7f9520004800 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.386+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c0052d0 con 0x7f9520004800 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.386+0000 7f95337fe640 1 -- 172.21.15.19:0/2534500514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c02ab90 con 0x7f9520004800 2024-04-25T10:05:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.387+0000 7f953adb7640 1 -- 172.21.15.19:0/2534500514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520004800 msgr2=0x7f9520004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.387+0000 7f953adb7640 1 --2- 172.21.15.19:0/2534500514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9520004800 0x7f9520004c80 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f951c001720 tx=0x7f9520005830 comp rx=0 tx=0).stop 2024-04-25T10:05:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.387+0000 7f9539354640 1 -- 172.21.15.19:0/2534500514 reap_dead start 2024-04-25T10:05:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.387+0000 7f953adb7640 1 -- 172.21.15.19:0/2534500514 shutdown_connections 2024-04-25T10:05:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.387+0000 7f953adb7640 1 -- 172.21.15.19:0/2534500514 >> 172.21.15.19:0/2534500514 conn(0x7f95340d0420 msgr2=0x7f95340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.387+0000 7f953adb7640 1 -- 172.21.15.19:0/2534500514 shutdown_connections 2024-04-25T10:05:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.387+0000 7f953adb7640 1 -- 172.21.15.19:0/2534500514 wait complete. 2024-04-25T10:05:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.388+0000 7f953adb7640 1 Processor -- start 2024-04-25T10:05:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.388+0000 7f953adb7640 1 -- start start 2024-04-25T10:05:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f953adb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520004800 0x7f953414c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f953adb7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95340ac950 0x7f95340acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f953adb7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95340cc700 0x7f95340ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f953adb7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9534162500 con 0x7f95340ac950 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f953adb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9534162220 con 0x7f9520004800 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f953adb7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95340b5e50 con 0x7f95340cc700 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f9533fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95340ac950 0x7f95340acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f9533fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95340ac950 0x7f95340acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53274/0 (socket says 172.21.15.19:53274) 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f9533fff640 1 -- 172.21.15.19:0/2090009854 learned_addr learned my addr 172.21.15.19:0/2090009854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f9538b53640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520004800 0x7f953414c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f9533fff640 1 -- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95340cc700 msgr2=0x7f95340ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f9533fff640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95340cc700 0x7f95340ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f9533fff640 1 -- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520004800 msgr2=0x7f953414c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f9533fff640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520004800 0x7f953414c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f9533fff640 1 -- 172.21.15.19:0/2090009854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95340ad310 con 0x7f95340ac950 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.389+0000 7f9538b53640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520004800 0x7f953414c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.390+0000 7f9533fff640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95340ac950 0x7f95340acdd0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f951c005040 tx=0x7f951c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.390+0000 7f95317fa640 1 -- 172.21.15.19:0/2090009854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c040020 con 0x7f95340ac950 2024-04-25T10:05:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.390+0000 7f95317fa640 1 -- 172.21.15.19:0/2090009854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c005480 con 0x7f95340ac950 2024-04-25T10:05:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.390+0000 7f953adb7640 1 -- 172.21.15.19:0/2090009854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95340cd430 con 0x7f95340ac950 2024-04-25T10:05:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.390+0000 7f95317fa640 1 -- 172.21.15.19:0/2090009854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c007a30 con 0x7f95340ac950 2024-04-25T10:05:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.390+0000 7f953adb7640 1 -- 172.21.15.19:0/2090009854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953414d370 con 0x7f95340ac950 2024-04-25T10:05:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.391+0000 7f953adb7640 1 -- 172.21.15.19:0/2090009854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953414ce80 con 0x7f95340ac950 2024-04-25T10:05:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.391+0000 7f95317fa640 1 -- 172.21.15.19:0/2090009854 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f951c043020 con 0x7f95340ac950 2024-04-25T10:05:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.392+0000 7f95317fa640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f951406b4d0 0x7f951406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.392+0000 7f95317fa640 1 -- 172.21.15.19:0/2090009854 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f951c0050d0 con 0x7f95340ac950 2024-04-25T10:05:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.392+0000 7f9538b53640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f951406b4d0 0x7f951406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.392+0000 7f9538b53640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f951406b4d0 0x7f951406d990 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9524009a40 tx=0x7f9524008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.396+0000 7f95317fa640 1 -- 172.21.15.19:0/2090009854 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f951c04a030 con 0x7f95340ac950 2024-04-25T10:05:00.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.573+0000 7f953adb7640 1 -- 172.21.15.19:0/2090009854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9534073a00 con 0x7f95340ac950 2024-04-25T10:05:00.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.574+0000 7f95317fa640 1 -- 172.21.15.19:0/2090009854 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v167) v1 ==== 104+0+11 (secure 0 0 0) 0x7f951c0791e0 con 0x7f95340ac950 2024-04-25T10:05:00.579 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:05:00.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.575+0000 7f953adb7640 1 -- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f951406b4d0 msgr2=0x7f951406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.575+0000 7f953adb7640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f951406b4d0 0x7f951406d990 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9524009a40 tx=0x7f9524008040 comp rx=0 tx=0).stop 2024-04-25T10:05:00.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.575+0000 7f953adb7640 1 -- 172.21.15.19:0/2090009854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95340ac950 msgr2=0x7f95340acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.575+0000 7f953adb7640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95340ac950 0x7f95340acdd0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f951c005040 tx=0x7f951c001120 comp rx=0 tx=0).stop 2024-04-25T10:05:00.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.575+0000 7f953adb7640 1 -- 172.21.15.19:0/2090009854 shutdown_connections 2024-04-25T10:05:00.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.575+0000 7f953adb7640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f951406b4d0 0x7f951406d990 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.575+0000 7f953adb7640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95340cc700 0x7f95340ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.575+0000 7f953adb7640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95340ac950 0x7f95340acdd0 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.575+0000 7f953adb7640 1 --2- 172.21.15.19:0/2090009854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520004800 0x7f953414c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.575+0000 7f953adb7640 1 -- 172.21.15.19:0/2090009854 >> 172.21.15.19:0/2090009854 conn(0x7f95340d0420 msgr2=0x7f95340b8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.576+0000 7f953adb7640 1 -- 172.21.15.19:0/2090009854 shutdown_connections 2024-04-25T10:05:00.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.576+0000 7f953adb7640 1 -- 172.21.15.19:0/2090009854 wait complete. 2024-04-25T10:05:00.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:00.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.690+0000 7f893670a640 1 Processor -- start 2024-04-25T10:05:00.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.690+0000 7f893670a640 1 -- start start 2024-04-25T10:05:00.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.691+0000 7f893670a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8930166a10 0x7f8930168e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.691+0000 7f893670a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89300afc80 con 0x7f893015de80 2024-04-25T10:05:00.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.691+0000 7f893670a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89300ce4d0 con 0x7f89300cadb0 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.691+0000 7f893670a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89300cb240 con 0x7f8930166a10 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.691+0000 7f892ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f893015de80 0x7f89300cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55430/0 (socket says 172.21.15.19:55430) 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.691+0000 7f892ffff640 1 -- 172.21.15.19:0/1466542552 learned_addr learned my addr 172.21.15.19:0/1466542552 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f8934ca7640 1 --2- 172.21.15.19:0/1466542552 >> v2:172.21.15.112:3300/0 conn(0x7f8930166a10 0x7f8930168e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259756573 0 0) 0x7f89300afc80 con 0x7f893015de80 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8918003520 con 0x7f893015de80 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1430218128 0 0) 0x7f8918003520 con 0x7f893015de80 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 >> v2:172.21.15.112:3300/0 conn(0x7f8930166a10 msgr2=0x7f8930168e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 --2- 172.21.15.19:0/1466542552 >> v2:172.21.15.112:3300/0 conn(0x7f8930166a10 0x7f8930168e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 >> v1:172.21.15.112:6789/0 conn(0x7f89300cadb0 legacy=0x7f89300cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89300ce7b0 con 0x7f893015de80 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8920001670 con 0x7f893015de80 2024-04-25T10:05:00.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 >> v1:172.21.15.19:6789/0 conn(0x7f893015de80 legacy=0x7f89300cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 --2- 172.21.15.19:0/1466542552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89180039b0 0x7f8918003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 --2- 172.21.15.19:0/1466542552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 0x7f8918004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 --2- 172.21.15.19:0/1466542552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89180051c0 0x7f891800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891800c130 con 0x7f8918004800 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891800c2e0 con 0x7f89180051c0 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891800c4e0 con 0x7f89180039b0 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892f7fe640 1 --2- 172.21.15.19:0/1466542552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 0x7f8918004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f8934ca7640 1 --2- 172.21.15.19:0/1466542552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89180051c0 0x7f891800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892ffff640 1 --2- 172.21.15.19:0/1466542552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89180039b0 0x7f8918003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892f7fe640 1 -- 172.21.15.19:0/1466542552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89180039b0 msgr2=0x7f8918003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892f7fe640 1 --2- 172.21.15.19:0/1466542552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89180039b0 0x7f8918003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892f7fe640 1 -- 172.21.15.19:0/1466542552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89180051c0 msgr2=0x7f891800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.692+0000 7f892f7fe640 1 --2- 172.21.15.19:0/1466542552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89180051c0 0x7f891800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.693+0000 7f892f7fe640 1 -- 172.21.15.19:0/1466542552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918003520 con 0x7f8918004800 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.693+0000 7f8934ca7640 1 -- 172.21.15.19:0/1466542552 reap_dead start 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.693+0000 7f892f7fe640 1 --2- 172.21.15.19:0/1466542552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 0x7f8918004c80 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f8924001750 tx=0x7f8918005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d244b9c672f1af0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.693+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924001370 con 0x7f8918004800 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.693+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89240052d0 con 0x7f8918004800 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.693+0000 7f892effd640 1 -- 172.21.15.19:0/1466542552 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892402abc0 con 0x7f8918004800 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.694+0000 7f893670a640 1 -- 172.21.15.19:0/1466542552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 msgr2=0x7f8918004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.694+0000 7f893670a640 1 --2- 172.21.15.19:0/1466542552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 0x7f8918004c80 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f8924001750 tx=0x7f8918005830 comp rx=0 tx=0).stop 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.694+0000 7f893670a640 1 -- 172.21.15.19:0/1466542552 shutdown_connections 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.694+0000 7f893670a640 1 --2- 172.21.15.19:0/1466542552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 0x7f8918004c80 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.694+0000 7f893670a640 1 -- 172.21.15.19:0/1466542552 >> 172.21.15.19:0/1466542552 conn(0x7f89300b2390 msgr2=0x7f89300b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.694+0000 7f893670a640 1 -- 172.21.15.19:0/1466542552 shutdown_connections 2024-04-25T10:05:00.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.694+0000 7f893670a640 1 -- 172.21.15.19:0/1466542552 wait complete. 2024-04-25T10:05:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.695+0000 7f893670a640 1 Processor -- start 2024-04-25T10:05:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.695+0000 7f893670a640 1 -- start start 2024-04-25T10:05:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f893670a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 0x7f8930153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f893670a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89301548a0 0x7f89300c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f893670a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8930154330 0x7f89300ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f893670a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89300b2eb0 con 0x7f8918004800 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f893670a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893015e260 con 0x7f8930154330 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f893670a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89300b3030 con 0x7f89301548a0 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f892ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 0x7f8930153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f892ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 0x7f8930153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53290/0 (socket says 172.21.15.19:53290) 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f892ffff640 1 -- 172.21.15.19:0/2115108111 learned_addr learned my addr 172.21.15.19:0/2115108111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f8934ca7640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8930154330 0x7f89300ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f892f7fe640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89301548a0 0x7f89300c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.696+0000 7f892ffff640 1 -- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89301548a0 msgr2=0x7f89300c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.697+0000 7f892ffff640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89301548a0 0x7f89300c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.697+0000 7f892ffff640 1 -- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8930154330 msgr2=0x7f89300ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.697+0000 7f892ffff640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8930154330 0x7f89300ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.697+0000 7f892ffff640 1 -- 172.21.15.19:0/2115108111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893015b920 con 0x7f8918004800 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.697+0000 7f892ffff640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 0x7f8930153540 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f8920007340 tx=0x7f8920007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.697+0000 7f892cff9640 1 -- 172.21.15.19:0/2115108111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892000f8c0 con 0x7f8918004800 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.697+0000 7f892cff9640 1 -- 172.21.15.19:0/2115108111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8920018070 con 0x7f8918004800 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.697+0000 7f893670a640 1 -- 172.21.15.19:0/2115108111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893015b640 con 0x7f8918004800 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.697+0000 7f892cff9640 1 -- 172.21.15.19:0/2115108111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892000fca0 con 0x7f8918004800 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.697+0000 7f893670a640 1 -- 172.21.15.19:0/2115108111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893015aa70 con 0x7f8918004800 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.698+0000 7f893670a640 1 -- 172.21.15.19:0/2115108111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8930073a00 con 0x7f8918004800 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.698+0000 7f892cff9640 1 -- 172.21.15.19:0/2115108111 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f892001d460 con 0x7f8918004800 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.699+0000 7f892cff9640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f891c06b4d0 0x7f891c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.699+0000 7f892cff9640 1 -- 172.21.15.19:0/2115108111 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f892008a450 con 0x7f8918004800 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.699+0000 7f892f7fe640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f891c06b4d0 0x7f891c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.699+0000 7f892f7fe640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f891c06b4d0 0x7f891c06d990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8924032000 tx=0x7f892402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.703+0000 7f892cff9640 1 -- 172.21.15.19:0/2115108111 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89200590d0 con 0x7f8918004800 2024-04-25T10:05:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.885+0000 7f893670a640 1 -- 172.21.15.19:0/2115108111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89300cf880 con 0x7f8918004800 2024-04-25T10:05:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.886+0000 7f892cff9640 1 -- 172.21.15.19:0/2115108111 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v167) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89200589d0 con 0x7f8918004800 2024-04-25T10:05:00.891 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:05:00.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 -- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f891c06b4d0 msgr2=0x7f891c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f891c06b4d0 0x7f891c06d990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8924032000 tx=0x7f892402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:00.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 -- 172.21.15.19:0/2115108111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 msgr2=0x7f8930153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 0x7f8930153540 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f8920007340 tx=0x7f8920007800 comp rx=0 tx=0).stop 2024-04-25T10:05:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 -- 172.21.15.19:0/2115108111 shutdown_connections 2024-04-25T10:05:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f891c06b4d0 0x7f891c06d990 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8930154330 0x7f89300ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89301548a0 0x7f89300c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 --2- 172.21.15.19:0/2115108111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8918004800 0x7f8930153540 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 -- 172.21.15.19:0/2115108111 >> 172.21.15.19:0/2115108111 conn(0x7f89300b2390 msgr2=0x7f893015bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 -- 172.21.15.19:0/2115108111 shutdown_connections 2024-04-25T10:05:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.888+0000 7f893670a640 1 -- 172.21.15.19:0/2115108111 wait complete. 2024-04-25T10:05:00.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:00.999+0000 7f3313407640 1 Processor -- start 2024-04-25T10:05:01.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.000+0000 7f3313407640 1 -- start start 2024-04-25T10:05:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.000+0000 7f3313407640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f330c158220 0x7f330c15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.000+0000 7f3313407640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f330c0af720 0x7f330c0afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.000+0000 7f3313407640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330c15f180 con 0x7f330c1617b0 2024-04-25T10:05:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.000+0000 7f3313407640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330c0b3670 con 0x7f330c158220 2024-04-25T10:05:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.000+0000 7f3313407640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c070a10 con 0x7f330c0af720 2024-04-25T10:05:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.001+0000 7f33111a3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f330c158220 0x7f330c15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.001+0000 7f33109a2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f330c0af720 0x7f330c0afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.001+0000 7f33119a4640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f330c1617b0 0x7f330c161c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:41056/0 (socket says 172.21.15.19:41056) 2024-04-25T10:05:01.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.001+0000 7f33119a4640 1 -- 172.21.15.19:0/3643434298 learned_addr learned my addr 172.21.15.19:0/3643434298 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:01.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.001+0000 7f33111a3640 1 -- 172.21.15.19:0/3643434298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f330c0af720 msgr2=0x7f330c0afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:01.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.001+0000 7f33111a3640 1 --2- 172.21.15.19:0/3643434298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f330c0af720 0x7f330c0afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.001+0000 7f33111a3640 1 -- 172.21.15.19:0/3643434298 >> v1:172.21.15.112:6789/0 conn(0x7f330c1617b0 legacy=0x7f330c161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.001+0000 7f33111a3640 1 -- 172.21.15.19:0/3643434298 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c06fe00 con 0x7f330c158220 2024-04-25T10:05:01.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.001+0000 7f33109a2640 1 --2- 172.21.15.19:0/3643434298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f330c0af720 0x7f330c0afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.002+0000 7f33111a3640 1 --2- 172.21.15.19:0/3643434298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f330c158220 0x7f330c15ec40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3304009d90 tx=0x7f330402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a68b951391ba965 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.002+0000 7f330bfff640 1 -- 172.21.15.19:0/3643434298 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304007740 con 0x7f330c158220 2024-04-25T10:05:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.002+0000 7f330bfff640 1 -- 172.21.15.19:0/3643434298 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33040078e0 con 0x7f330c158220 2024-04-25T10:05:01.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.002+0000 7f330bfff640 1 -- 172.21.15.19:0/3643434298 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304007cb0 con 0x7f330c158220 2024-04-25T10:05:01.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.002+0000 7f3313407640 1 -- 172.21.15.19:0/3643434298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f330c158220 msgr2=0x7f330c15ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.002+0000 7f3313407640 1 --2- 172.21.15.19:0/3643434298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f330c158220 0x7f330c15ec40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3304009d90 tx=0x7f330402a660 comp rx=0 tx=0).stop 2024-04-25T10:05:01.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.003+0000 7f3313407640 1 -- 172.21.15.19:0/3643434298 shutdown_connections 2024-04-25T10:05:01.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.003+0000 7f3313407640 1 --2- 172.21.15.19:0/3643434298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f330c0af720 0x7f330c0afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.003+0000 7f3313407640 1 --2- 172.21.15.19:0/3643434298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f330c158220 0x7f330c15ec40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.003+0000 7f3313407640 1 -- 172.21.15.19:0/3643434298 >> 172.21.15.19:0/3643434298 conn(0x7f330c0b2370 msgr2=0x7f330c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:01.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.003+0000 7f3313407640 1 -- 172.21.15.19:0/3643434298 shutdown_connections 2024-04-25T10:05:01.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.003+0000 7f3313407640 1 -- 172.21.15.19:0/3643434298 wait complete. 2024-04-25T10:05:01.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.004+0000 7f3313407640 1 Processor -- start 2024-04-25T10:05:01.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.004+0000 7f3313407640 1 -- start start 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.004+0000 7f3313407640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f330c0af720 0x7f330c152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.004+0000 7f3313407640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f330c158220 0x7f330c1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f3313407640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f330c1617b0 0x7f330c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f3313407640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0706a0 con 0x7f330c158220 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f3313407640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0703c0 con 0x7f330c0af720 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f3313407640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c158600 con 0x7f330c1617b0 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33109a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f330c158220 0x7f330c1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33109a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f330c158220 0x7f330c1549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53298/0 (socket says 172.21.15.19:53298) 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33109a2640 1 -- 172.21.15.19:0/2417023051 learned_addr learned my addr 172.21.15.19:0/2417023051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33111a3640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f330c0af720 0x7f330c152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33119a4640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f330c1617b0 0x7f330c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33109a2640 1 -- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f330c1617b0 msgr2=0x7f330c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33109a2640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f330c1617b0 0x7f330c0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33109a2640 1 -- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f330c0af720 msgr2=0x7f330c152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33109a2640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f330c0af720 0x7f330c152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33109a2640 1 -- 172.21.15.19:0/2417023051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c0cb290 con 0x7f330c158220 2024-04-25T10:05:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33111a3640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f330c0af720 0x7f330c152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f33109a2640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f330c158220 0x7f330c1549d0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f330000ec10 tx=0x7f330000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f3309ffb640 1 -- 172.21.15.19:0/2417023051 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300017070 con 0x7f330c158220 2024-04-25T10:05:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f3309ffb640 1 -- 172.21.15.19:0/2417023051 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330000cc30 con 0x7f330c158220 2024-04-25T10:05:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f3313407640 1 -- 172.21.15.19:0/2417023051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c0cb570 con 0x7f330c158220 2024-04-25T10:05:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.005+0000 7f3309ffb640 1 -- 172.21.15.19:0/2417023051 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300013470 con 0x7f330c158220 2024-04-25T10:05:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.006+0000 7f3313407640 1 -- 172.21.15.19:0/2417023051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c0cbb30 con 0x7f330c158220 2024-04-25T10:05:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.007+0000 7f3313407640 1 -- 172.21.15.19:0/2417023051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c0b5730 con 0x7f330c158220 2024-04-25T10:05:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.007+0000 7f3309ffb640 1 -- 172.21.15.19:0/2417023051 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f330001a070 con 0x7f330c158220 2024-04-25T10:05:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.007+0000 7f3309ffb640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32f806b500 0x7f32f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.008+0000 7f3309ffb640 1 -- 172.21.15.19:0/2417023051 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f330008a5f0 con 0x7f330c158220 2024-04-25T10:05:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.008+0000 7f33111a3640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32f806b500 0x7f32f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.008+0000 7f33111a3640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32f806b500 0x7f32f806d9c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3304034000 tx=0x7f330402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.012+0000 7f3309ffb640 1 -- 172.21.15.19:0/2417023051 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f330c0b5730 con 0x7f330c158220 2024-04-25T10:05:01.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.230+0000 7f3313407640 1 -- 172.21.15.19:0/2417023051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f330c156be0 con 0x7f330c158220 2024-04-25T10:05:01.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.231+0000 7f3309ffb640 1 -- 172.21.15.19:0/2417023051 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4954 (secure 0 0 0) 0x7f330c156be0 con 0x7f330c158220 2024-04-25T10:05:01.236 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:01.236 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1300145056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1300145056},{"type":"v1","addr":"172.21.15.19:6835","nonce":1300145056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":8027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2001687418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2001687418},{"type":"v1","addr":"172.21.15.112:6836","nonce":2001687418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721457153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721457153},{"type":"v1","addr":"172.21.15.112:6837","nonce":721457153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:14.252458+0000","modified":"2024-04-25T10:04:16.259643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10698":{"gid":10698,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3942829201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3942829201},{"type":"v1","addr":"172.21.15.19:6834","nonce":3942829201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:05:01.236 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 109 2024-04-25T10:05:01.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.232+0000 7f3313407640 1 -- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32f806b500 msgr2=0x7f32f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.232+0000 7f3313407640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32f806b500 0x7f32f806d9c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3304034000 tx=0x7f330402f000 comp rx=0 tx=0).stop 2024-04-25T10:05:01.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.233+0000 7f3313407640 1 -- 172.21.15.19:0/2417023051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f330c158220 msgr2=0x7f330c1549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.233+0000 7f3313407640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f330c158220 0x7f330c1549d0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f330000ec10 tx=0x7f330000c520 comp rx=0 tx=0).stop 2024-04-25T10:05:01.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.233+0000 7f3313407640 1 -- 172.21.15.19:0/2417023051 shutdown_connections 2024-04-25T10:05:01.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.233+0000 7f3313407640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f32f806b500 0x7f32f806d9c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.233+0000 7f3313407640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f330c1617b0 0x7f330c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.233+0000 7f3313407640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f330c158220 0x7f330c1549d0 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.233+0000 7f3313407640 1 --2- 172.21.15.19:0/2417023051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f330c0af720 0x7f330c152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.233+0000 7f3313407640 1 -- 172.21.15.19:0/2417023051 >> 172.21.15.19:0/2417023051 conn(0x7f330c0b2370 msgr2=0x7f330c160a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.233+0000 7f3313407640 1 -- 172.21.15.19:0/2417023051 shutdown_connections 2024-04-25T10:05:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.233+0000 7f3313407640 1 -- 172.21.15.19:0/2417023051 wait complete. 2024-04-25T10:05:01.249 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.344+0000 7fee1a5c8640 1 Processor -- start 2024-04-25T10:05:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.344+0000 7fee1a5c8640 1 -- start start 2024-04-25T10:05:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee1a5c8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fee1415dd30 0x7fee140b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee1a5c8640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fee140d0ad0 0x7fee140b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee1a5c8640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee140d1df0 con 0x7fee140b9770 2024-04-25T10:05:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee1a5c8640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee140d20d0 con 0x7fee140d0ad0 2024-04-25T10:05:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee1a5c8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee140d23b0 con 0x7fee1415dd30 2024-04-25T10:05:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee137fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fee140b9770 0x7fee140d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55434/0 (socket says 172.21.15.19:55434) 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee137fe640 1 -- 172.21.15.19:0/2914940297 learned_addr learned my addr 172.21.15.19:0/2914940297 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee18b65640 1 --2- 172.21.15.19:0/2914940297 >> v2:172.21.15.19:3300/0 conn(0x7fee140d0ad0 0x7fee140b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee13fff640 1 --2- 172.21.15.19:0/2914940297 >> v2:172.21.15.112:3300/0 conn(0x7fee1415dd30 0x7fee140b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee12ffd640 1 -- 172.21.15.19:0/2914940297 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117703162 0 0) 0x7fee140d1df0 con 0x7fee140b9770 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee12ffd640 1 -- 172.21.15.19:0/2914940297 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedfc003520 con 0x7fee140b9770 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee18b65640 1 -- 172.21.15.19:0/2914940297 >> v2:172.21.15.112:3300/0 conn(0x7fee1415dd30 msgr2=0x7fee140b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee18b65640 1 --2- 172.21.15.19:0/2914940297 >> v2:172.21.15.112:3300/0 conn(0x7fee1415dd30 0x7fee140b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee18b65640 1 -- 172.21.15.19:0/2914940297 >> v1:172.21.15.19:6789/0 conn(0x7fee140b9770 legacy=0x7fee140d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.345+0000 7fee18b65640 1 -- 172.21.15.19:0/2914940297 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee140d2690 con 0x7fee140d0ad0 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.346+0000 7fee18b65640 1 --2- 172.21.15.19:0/2914940297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140d0ad0 0x7fee140b6a40 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fee1406d000 tx=0x7fee0c02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f71f9558989872e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.346+0000 7fee12ffd640 1 -- 172.21.15.19:0/2914940297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c007740 con 0x7fee140d0ad0 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.346+0000 7fee12ffd640 1 -- 172.21.15.19:0/2914940297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c0078e0 con 0x7fee140d0ad0 2024-04-25T10:05:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.346+0000 7fee12ffd640 1 -- 172.21.15.19:0/2914940297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c007c90 con 0x7fee140d0ad0 2024-04-25T10:05:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.347+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2914940297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140d0ad0 msgr2=0x7fee140b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.347+0000 7fee1a5c8640 1 --2- 172.21.15.19:0/2914940297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140d0ad0 0x7fee140b6a40 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fee1406d000 tx=0x7fee0c02a560 comp rx=0 tx=0).stop 2024-04-25T10:05:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.347+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2914940297 shutdown_connections 2024-04-25T10:05:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.347+0000 7fee1a5c8640 1 --2- 172.21.15.19:0/2914940297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140d0ad0 0x7fee140b6a40 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.347+0000 7fee1a5c8640 1 --2- 172.21.15.19:0/2914940297 >> v2:172.21.15.112:3300/0 conn(0x7fee1415dd30 0x7fee140b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.347+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2914940297 >> 172.21.15.19:0/2914940297 conn(0x7fee14061c30 msgr2=0x7fee1406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.347+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2914940297 shutdown_connections 2024-04-25T10:05:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.347+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2914940297 wait complete. 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.348+0000 7fee1a5c8640 1 Processor -- start 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.348+0000 7fee1a5c8640 1 -- start start 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee1a5c8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee140b9770 0x7fee14154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee1a5c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140d0ad0 0x7fee141594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee1a5c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1415dd30 0x7fee140b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee1a5c8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee140d1b90 con 0x7fee140d0ad0 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee1a5c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee140d1a10 con 0x7fee1415dd30 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee1a5c8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee140d2e90 con 0x7fee140b9770 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140d0ad0 0x7fee141594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140d0ad0 0x7fee141594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53310/0 (socket says 172.21.15.19:53310) 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee137fe640 1 -- 172.21.15.19:0/2157648969 learned_addr learned my addr 172.21.15.19:0/2157648969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee18b65640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1415dd30 0x7fee140b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee13fff640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee140b9770 0x7fee14154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee137fe640 1 -- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee140b9770 msgr2=0x7fee14154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee137fe640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee140b9770 0x7fee14154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee137fe640 1 -- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1415dd30 msgr2=0x7fee140b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee137fe640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1415dd30 0x7fee140b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee137fe640 1 -- 172.21.15.19:0/2157648969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee140cabb0 con 0x7fee140d0ad0 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee18b65640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1415dd30 0x7fee140b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.349+0000 7fee137fe640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140d0ad0 0x7fee141594e0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fee0800c700 tx=0x7fee0800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.350+0000 7fee10ff9640 1 -- 172.21.15.19:0/2157648969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08017070 con 0x7fee140d0ad0 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.350+0000 7fee10ff9640 1 -- 172.21.15.19:0/2157648969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee08007290 con 0x7fee140d0ad0 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.350+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2157648969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee140cae90 con 0x7fee140d0ad0 2024-04-25T10:05:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.350+0000 7fee10ff9640 1 -- 172.21.15.19:0/2157648969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08001d40 con 0x7fee140d0ad0 2024-04-25T10:05:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.350+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2157648969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee14159a20 con 0x7fee140d0ad0 2024-04-25T10:05:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.351+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2157648969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee14073990 con 0x7fee140d0ad0 2024-04-25T10:05:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.351+0000 7fee10ff9640 1 -- 172.21.15.19:0/2157648969 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee0801a070 con 0x7fee140d0ad0 2024-04-25T10:05:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.352+0000 7fee10ff9640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee0006b500 0x7fee0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.352+0000 7fee10ff9640 1 -- 172.21.15.19:0/2157648969 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fee08089740 con 0x7fee140d0ad0 2024-04-25T10:05:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.352+0000 7fee13fff640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee0006b500 0x7fee0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.352+0000 7fee13fff640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee0006b500 0x7fee0006d9c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fee04006fd0 tx=0x7fee04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.356+0000 7fee10ff9640 1 -- 172.21.15.19:0/2157648969 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee080583b0 con 0x7fee140d0ad0 2024-04-25T10:05:01.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.537+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2157648969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee140b83b0 con 0x7fee140d0ad0 2024-04-25T10:05:01.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.539+0000 7fee10ff9640 1 -- 172.21.15.19:0/2157648969 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+17066 (secure 0 0 0) 0x7fee08057cb0 con 0x7fee140d0ad0 2024-04-25T10:05:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:01.544 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":167,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:04:49.642130+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:12.518429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:13.232769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2c","mappings":[{"from":3,"to":0}]},{"pgid":"26.31","mappings":[{"from":1,"to":0}]},{"pgid":"26.37","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:01.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.540+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee0006b500 msgr2=0x7fee0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.540+0000 7fee1a5c8640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee0006b500 0x7fee0006d9c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fee04006fd0 tx=0x7fee04008040 comp rx=0 tx=0).stop 2024-04-25T10:05:01.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.540+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2157648969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140d0ad0 msgr2=0x7fee141594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.540+0000 7fee1a5c8640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140d0ad0 0x7fee141594e0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fee0800c700 tx=0x7fee0800cbc0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.541+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2157648969 shutdown_connections 2024-04-25T10:05:01.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.541+0000 7fee1a5c8640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fee0006b500 0x7fee0006d9c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.541+0000 7fee1a5c8640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee1415dd30 0x7fee140b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.541+0000 7fee1a5c8640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140d0ad0 0x7fee141594e0 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.541+0000 7fee1a5c8640 1 --2- 172.21.15.19:0/2157648969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee140b9770 0x7fee14154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.541+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2157648969 >> 172.21.15.19:0/2157648969 conn(0x7fee14061c30 msgr2=0x7fee1415f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.541+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2157648969 shutdown_connections 2024-04-25T10:05:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.541+0000 7fee1a5c8640 1 -- 172.21.15.19:0/2157648969 wait complete. 2024-04-25T10:05:01.557 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:05:01.557 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:01.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.653+0000 7ff2acfd5640 1 Processor -- start 2024-04-25T10:05:01.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.653+0000 7ff2acfd5640 1 -- start start 2024-04-25T10:05:01.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.654+0000 7ff2acfd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80b6cb0 0x7ff2a80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.654+0000 7ff2acfd5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a80ceee0 con 0x7ff2a815db70 2024-04-25T10:05:01.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.654+0000 7ff2acfd5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a80d1d10 con 0x7ff2a80b5a60 2024-04-25T10:05:01.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.654+0000 7ff2acfd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80d1ff0 con 0x7ff2a80b6cb0 2024-04-25T10:05:01.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.654+0000 7ff2a659c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff2a815db70 0x7ff2a8162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55440/0 (socket says 172.21.15.19:55440) 2024-04-25T10:05:01.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.654+0000 7ff2a659c640 1 -- 172.21.15.19:0/3447813520 learned_addr learned my addr 172.21.15.19:0/3447813520 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.654+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184868773 0 0) 0x7ff2a80ceee0 con 0x7ff2a815db70 2024-04-25T10:05:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.654+0000 7ff2a5d9b640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80b6cb0 0x7ff2a80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.654+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff290003520 con 0x7ff2a815db70 2024-04-25T10:05:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1152149652 0 0) 0x7ff290003520 con 0x7ff2a815db70 2024-04-25T10:05:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80b6cb0 msgr2=0x7ff2a80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80b6cb0 0x7ff2a80b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 >> v1:172.21.15.112:6789/0 conn(0x7ff2a80b5a60 legacy=0x7ff2a80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80d22d0 con 0x7ff2a815db70 2024-04-25T10:05:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7ff298001670 con 0x7ff2a815db70 2024-04-25T10:05:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 >> v1:172.21.15.19:6789/0 conn(0x7ff2a815db70 legacy=0x7ff2a8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80b6cb0 0x7ff2900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff290003f60 0x7ff290004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290004d40 0x7ff2900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80d1ff0 con 0x7ff290004d40 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29000baa0 con 0x7ff290003f60 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29000bca0 con 0x7ff2a80b6cb0 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a5d9b640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff290003f60 0x7ff290004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a6d9d640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290004d40 0x7ff2900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a659c640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80b6cb0 0x7ff2900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a5d9b640 1 -- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80b6cb0 msgr2=0x7ff2900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a5d9b640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80b6cb0 0x7ff2900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a5d9b640 1 -- 172.21.15.19:0/3447813520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290004d40 msgr2=0x7ff2900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a5d9b640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290004d40 0x7ff2900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a5d9b640 1 -- 172.21.15.19:0/3447813520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff290003520 con 0x7ff290003f60 2024-04-25T10:05:01.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.655+0000 7ff2a6d9d640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290004d40 0x7ff2900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:01.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.656+0000 7ff2a5d9b640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff290003f60 0x7ff290004800 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009a90 tx=0x7ff290005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b37c3294717e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.656+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c036070 con 0x7ff290003f60 2024-04-25T10:05:01.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.656+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c031400 con 0x7ff290003f60 2024-04-25T10:05:01.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.656+0000 7ff2a559a640 1 -- 172.21.15.19:0/3447813520 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c009690 con 0x7ff290003f60 2024-04-25T10:05:01.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.657+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff290003f60 msgr2=0x7ff290004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.657+0000 7ff2acfd5640 1 --2- 172.21.15.19:0/3447813520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff290003f60 0x7ff290004800 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009a90 tx=0x7ff290005830 comp rx=0 tx=0).stop 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.657+0000 7ff2a6d9d640 1 -- 172.21.15.19:0/3447813520 reap_dead start 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.657+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3447813520 shutdown_connections 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.657+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3447813520 >> 172.21.15.19:0/3447813520 conn(0x7ff2a80d0520 msgr2=0x7ff2a80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.657+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3447813520 shutdown_connections 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.657+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3447813520 wait complete. 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.658+0000 7ff2acfd5640 1 Processor -- start 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.658+0000 7ff2acfd5640 1 -- start start 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2acfd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290003f60 0x7ff2a8149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2acfd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a80c8ef0 0x7ff2a80c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2acfd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80cab70 0x7ff2a80caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2acfd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80cf1c0 con 0x7ff290003f60 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2acfd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80ce530 con 0x7ff2a80c8ef0 2024-04-25T10:05:01.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2acfd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80cf340 con 0x7ff2a80cab70 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290003f60 0x7ff2a8149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290003f60 0x7ff2a8149140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53334/0 (socket says 172.21.15.19:53334) 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a659c640 1 -- 172.21.15.19:0/3442736400 learned_addr learned my addr 172.21.15.19:0/3442736400 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a6d9d640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80cab70 0x7ff2a80caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a5d9b640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a80c8ef0 0x7ff2a80c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a659c640 1 -- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80cab70 msgr2=0x7ff2a80caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a659c640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80cab70 0x7ff2a80caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a659c640 1 -- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a80c8ef0 msgr2=0x7ff2a80c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a659c640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a80c8ef0 0x7ff2a80c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a659c640 1 -- 172.21.15.19:0/3442736400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80cc7f0 con 0x7ff290003f60 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a5d9b640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a80c8ef0 0x7ff2a80c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a6d9d640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80cab70 0x7ff2a80caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.659+0000 7ff2a659c640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290003f60 0x7ff2a8149140 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7ff298007340 tx=0x7ff298007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.660+0000 7ff296ffd640 1 -- 172.21.15.19:0/3442736400 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29801a070 con 0x7ff290003f60 2024-04-25T10:05:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.660+0000 7ff296ffd640 1 -- 172.21.15.19:0/3442736400 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29800f8e0 con 0x7ff290003f60 2024-04-25T10:05:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.660+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3442736400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a81533b0 con 0x7ff290003f60 2024-04-25T10:05:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.660+0000 7ff296ffd640 1 -- 172.21.15.19:0/3442736400 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29800fcb0 con 0x7ff290003f60 2024-04-25T10:05:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.660+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3442736400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a80cba30 con 0x7ff290003f60 2024-04-25T10:05:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.661+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3442736400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a80b62b0 con 0x7ff290003f60 2024-04-25T10:05:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.661+0000 7ff296ffd640 1 -- 172.21.15.19:0/3442736400 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff29801d070 con 0x7ff290003f60 2024-04-25T10:05:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.662+0000 7ff296ffd640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28806b500 0x7ff28806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.662+0000 7ff296ffd640 1 -- 172.21.15.19:0/3442736400 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff298089c70 con 0x7ff290003f60 2024-04-25T10:05:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.662+0000 7ff2a5d9b640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28806b500 0x7ff28806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.662+0000 7ff2a5d9b640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28806b500 0x7ff28806d9c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff29c033000 tx=0x7ff29c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.666+0000 7ff296ffd640 1 -- 172.21.15.19:0/3442736400 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2a80b62b0 con 0x7ff290003f60 2024-04-25T10:05:01.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.884+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3442736400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2a8073990 con 0x7ff290003f60 2024-04-25T10:05:01.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.884+0000 7ff296ffd640 1 -- 172.21.15.19:0/3442736400 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4954 (secure 0 0 0) 0x7ff2980588f0 con 0x7ff290003f60 2024-04-25T10:05:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:01.890 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1300145056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1300145056},{"type":"v1","addr":"172.21.15.19:6835","nonce":1300145056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":8027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2001687418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2001687418},{"type":"v1","addr":"172.21.15.112:6836","nonce":2001687418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721457153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721457153},{"type":"v1","addr":"172.21.15.112:6837","nonce":721457153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:14.252458+0000","modified":"2024-04-25T10:04:16.259643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10698":{"gid":10698,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3942829201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3942829201},{"type":"v1","addr":"172.21.15.19:6834","nonce":3942829201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:05:01.890 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 109 2024-04-25T10:05:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28806b500 msgr2=0x7ff28806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28806b500 0x7ff28806d9c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff29c033000 tx=0x7ff29c008030 comp rx=0 tx=0).stop 2024-04-25T10:05:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3442736400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290003f60 msgr2=0x7ff2a8149140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290003f60 0x7ff2a8149140 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7ff298007340 tx=0x7ff298007800 comp rx=0 tx=0).stop 2024-04-25T10:05:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3442736400 shutdown_connections 2024-04-25T10:05:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff28806b500 0x7ff28806d9c0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2a80cab70 0x7ff2a80caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a80c8ef0 0x7ff2a80c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 --2- 172.21.15.19:0/3442736400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff290003f60 0x7ff2a8149140 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3442736400 >> 172.21.15.19:0/3442736400 conn(0x7ff2a80d0520 msgr2=0x7ff2a815c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:01.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3442736400 shutdown_connections 2024-04-25T10:05:01.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.887+0000 7ff2acfd5640 1 -- 172.21.15.19:0/3442736400 wait complete. 2024-04-25T10:05:01.903 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.998+0000 7fd00a934640 1 Processor -- start 2024-04-25T10:05:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.998+0000 7fd00a934640 1 -- start start 2024-04-25T10:05:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd00a934640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd00415da70 0x7fd004161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd00a934640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd004161b80 0x7fd0040d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd00a934640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd004162020 con 0x7fd0040b9730 2024-04-25T10:05:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd00a934640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0041625e0 con 0x7fd004161b80 2024-04-25T10:05:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd00a934640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd004162300 con 0x7fd00415da70 2024-04-25T10:05:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd008ed1640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd004161b80 0x7fd0040d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd0037fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd0040b9730 0x7fd0040ce650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55442/0 (socket says 172.21.15.19:55442) 2024-04-25T10:05:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd0037fe640 1 -- 172.21.15.19:0/332113145 learned_addr learned my addr 172.21.15.19:0/332113145 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd003fff640 1 --2- 172.21.15.19:0/332113145 >> v2:172.21.15.112:3300/0 conn(0x7fd00415da70 0x7fd004161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd008ed1640 1 -- 172.21.15.19:0/332113145 >> v2:172.21.15.112:3300/0 conn(0x7fd00415da70 msgr2=0x7fd004161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd008ed1640 1 --2- 172.21.15.19:0/332113145 >> v2:172.21.15.112:3300/0 conn(0x7fd00415da70 0x7fd004161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd008ed1640 1 -- 172.21.15.19:0/332113145 >> v1:172.21.15.19:6789/0 conn(0x7fd0040b9730 legacy=0x7fd0040ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd008ed1640 1 -- 172.21.15.19:0/332113145 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0040d1af0 con 0x7fd004161b80 2024-04-25T10:05:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd002ffd640 1 -- 172.21.15.19:0/332113145 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102149251 0 0) 0x7fd004162020 con 0x7fd0040b9730 2024-04-25T10:05:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:01.999+0000 7fd003fff640 1 --2- 172.21.15.19:0/332113145 >> v2:172.21.15.112:3300/0 conn(0x7fd00415da70 0x7fd004161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.000+0000 7fd008ed1640 1 --2- 172.21.15.19:0/332113145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd004161b80 0x7fd0040d22f0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fd0040b61a0 tx=0x7fcffc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d993b3317967907a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.000+0000 7fd002ffd640 1 -- 172.21.15.19:0/332113145 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc007740 con 0x7fd004161b80 2024-04-25T10:05:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.000+0000 7fd002ffd640 1 -- 172.21.15.19:0/332113145 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc0078e0 con 0x7fd004161b80 2024-04-25T10:05:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.000+0000 7fd002ffd640 1 -- 172.21.15.19:0/332113145 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc007cb0 con 0x7fd004161b80 2024-04-25T10:05:02.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.000+0000 7fd00a934640 1 -- 172.21.15.19:0/332113145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd004161b80 msgr2=0x7fd0040d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.000+0000 7fd00a934640 1 --2- 172.21.15.19:0/332113145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd004161b80 0x7fd0040d22f0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fd0040b61a0 tx=0x7fcffc02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.001+0000 7fd00a934640 1 -- 172.21.15.19:0/332113145 shutdown_connections 2024-04-25T10:05:02.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.001+0000 7fd00a934640 1 --2- 172.21.15.19:0/332113145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd004161b80 0x7fd0040d22f0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.001+0000 7fd00a934640 1 --2- 172.21.15.19:0/332113145 >> v2:172.21.15.112:3300/0 conn(0x7fd00415da70 0x7fd004161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.001+0000 7fd00a934640 1 -- 172.21.15.19:0/332113145 >> 172.21.15.19:0/332113145 conn(0x7fd004061a20 msgr2=0x7fd004061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.001+0000 7fd00a934640 1 -- 172.21.15.19:0/332113145 shutdown_connections 2024-04-25T10:05:02.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.001+0000 7fd00a934640 1 -- 172.21.15.19:0/332113145 wait complete. 2024-04-25T10:05:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.002+0000 7fd00a934640 1 Processor -- start 2024-04-25T10:05:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.002+0000 7fd00a934640 1 -- start start 2024-04-25T10:05:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd00a934640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00415da70 0x7fd00414adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd00a934640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd004161b80 0x7fd0040c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd00a934640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0040d3ed0 0x7fd004147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd00a934640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040ced60 con 0x7fd00415da70 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd00a934640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00415de50 con 0x7fd004161b80 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd00a934640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040ceee0 con 0x7fd0040d3ed0 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd003fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00415da70 0x7fd00414adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd003fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00415da70 0x7fd00414adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53340/0 (socket says 172.21.15.19:53340) 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd003fff640 1 -- 172.21.15.19:0/1382321342 learned_addr learned my addr 172.21.15.19:0/1382321342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd0037fe640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd004161b80 0x7fd0040c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd008ed1640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0040d3ed0 0x7fd004147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd003fff640 1 -- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0040d3ed0 msgr2=0x7fd004147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd003fff640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0040d3ed0 0x7fd004147470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd003fff640 1 -- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd004161b80 msgr2=0x7fd0040c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd003fff640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd004161b80 0x7fd0040c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd003fff640 1 -- 172.21.15.19:0/1382321342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0040c9310 con 0x7fd00415da70 2024-04-25T10:05:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd008ed1640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0040d3ed0 0x7fd004147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.003+0000 7fd003fff640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00415da70 0x7fd00414adc0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fcff400c820 tx=0x7fcff400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.004+0000 7fd000ff9640 1 -- 172.21.15.19:0/1382321342 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4015070 con 0x7fd00415da70 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.004+0000 7fd000ff9640 1 -- 172.21.15.19:0/1382321342 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff4007a30 con 0x7fd00415da70 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.004+0000 7fd00a934640 1 -- 172.21.15.19:0/1382321342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0040c95f0 con 0x7fd00415da70 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.004+0000 7fd000ff9640 1 -- 172.21.15.19:0/1382321342 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4007e00 con 0x7fd00415da70 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.004+0000 7fd00a934640 1 -- 172.21.15.19:0/1382321342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0040c9bb0 con 0x7fd00415da70 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.005+0000 7fd00a934640 1 -- 172.21.15.19:0/1382321342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd004073990 con 0x7fd00415da70 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.005+0000 7fd000ff9640 1 -- 172.21.15.19:0/1382321342 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcff4018070 con 0x7fd00415da70 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.006+0000 7fd000ff9640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 0x7fcff006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.006+0000 7fd000ff9640 1 -- 172.21.15.19:0/1382321342 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fcff408a320 con 0x7fd00415da70 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.006+0000 7fd0037fe640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 0x7fcff006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.006+0000 7fd0037fe640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 0x7fcff006d990 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd00406c030 tx=0x7fcff8007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.010+0000 7fd000ff9640 1 -- 172.21.15.19:0/1382321342 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcff4059030 con 0x7fd00415da70 2024-04-25T10:05:02.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.228+0000 7fd00a934640 1 -- 172.21.15.19:0/1382321342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0040d1570 con 0x7fd00415da70 2024-04-25T10:05:02.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.228+0000 7fd000ff9640 1 -- 172.21.15.19:0/1382321342 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4954 (secure 0 0 0) 0x7fcff4058930 con 0x7fd00415da70 2024-04-25T10:05:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:02.234 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1300145056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1300145056},{"type":"v1","addr":"172.21.15.19:6835","nonce":1300145056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":8027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2001687418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2001687418},{"type":"v1","addr":"172.21.15.112:6836","nonce":2001687418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721457153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721457153},{"type":"v1","addr":"172.21.15.112:6837","nonce":721457153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:14.252458+0000","modified":"2024-04-25T10:04:16.259643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10698":{"gid":10698,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/3942829201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3942829201},{"type":"v1","addr":"172.21.15.19:6834","nonce":3942829201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:05:02.234 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 109 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 -- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 msgr2=0x7fcff006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 0x7fcff006d990 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd00406c030 tx=0x7fcff8007b50 comp rx=0 tx=0).stop 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 -- 172.21.15.19:0/1382321342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00415da70 msgr2=0x7fd00414adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00415da70 0x7fd00414adc0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fcff400c820 tx=0x7fcff400cc20 comp rx=0 tx=0).stop 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 -- 172.21.15.19:0/1382321342 shutdown_connections 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcff006b4d0 0x7fcff006d990 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0040d3ed0 0x7fd004147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd004161b80 0x7fd0040c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 --2- 172.21.15.19:0/1382321342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00415da70 0x7fd00414adc0 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 -- 172.21.15.19:0/1382321342 >> 172.21.15.19:0/1382321342 conn(0x7fd004061a20 msgr2=0x7fd004061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 -- 172.21.15.19:0/1382321342 shutdown_connections 2024-04-25T10:05:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.230+0000 7fd00a934640 1 -- 172.21.15.19:0/1382321342 wait complete. 2024-04-25T10:05:02.247 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:02.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.343+0000 7f454fc09640 1 Processor -- start 2024-04-25T10:05:02.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.343+0000 7f454fc09640 1 -- start start 2024-04-25T10:05:02.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454fc09640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f454815dd20 0x7f45480b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454fc09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45480d11b0 0x7f45480d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454fc09640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454815eac0 con 0x7f454815e100 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454fc09640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45480d1c00 con 0x7f454815dd20 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454fc09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454815eda0 con 0x7f45480d11b0 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454d9a5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f454815dd20 0x7f45480b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454d1a4640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f454815e100 0x7f45480d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55448/0 (socket says 172.21.15.19:55448) 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454d1a4640 1 -- 172.21.15.19:0/2553417874 learned_addr learned my addr 172.21.15.19:0/2553417874 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454e1a6640 1 --2- 172.21.15.19:0/2553417874 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45480d11b0 0x7f45480d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454d9a5640 1 -- 172.21.15.19:0/2553417874 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45480d11b0 msgr2=0x7f45480d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454d9a5640 1 --2- 172.21.15.19:0/2553417874 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45480d11b0 0x7f45480d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454d9a5640 1 -- 172.21.15.19:0/2553417874 >> v1:172.21.15.19:6789/0 conn(0x7f454815e100 legacy=0x7f45480d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.344+0000 7f454d9a5640 1 -- 172.21.15.19:0/2553417874 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454815f080 con 0x7f454815dd20 2024-04-25T10:05:02.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.345+0000 7f454d9a5640 1 --2- 172.21.15.19:0/2553417874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454815dd20 0x7f45480b96d0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f45440097a0 tx=0x7f454402a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee53691b12e8f967 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.345+0000 7f454c9a3640 1 -- 172.21.15.19:0/2553417874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544037070 con 0x7f454815dd20 2024-04-25T10:05:02.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.345+0000 7f454c9a3640 1 -- 172.21.15.19:0/2553417874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454402ac90 con 0x7f454815dd20 2024-04-25T10:05:02.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.345+0000 7f454c9a3640 1 -- 172.21.15.19:0/2553417874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544032660 con 0x7f454815dd20 2024-04-25T10:05:02.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.346+0000 7f454fc09640 1 -- 172.21.15.19:0/2553417874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454815dd20 msgr2=0x7f45480b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.346+0000 7f454fc09640 1 --2- 172.21.15.19:0/2553417874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454815dd20 0x7f45480b96d0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f45440097a0 tx=0x7f454402a000 comp rx=0 tx=0).stop 2024-04-25T10:05:02.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.346+0000 7f454fc09640 1 -- 172.21.15.19:0/2553417874 shutdown_connections 2024-04-25T10:05:02.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.346+0000 7f454fc09640 1 --2- 172.21.15.19:0/2553417874 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45480d11b0 0x7f45480d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.346+0000 7f454fc09640 1 --2- 172.21.15.19:0/2553417874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454815dd20 0x7f45480b96d0 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.346+0000 7f454fc09640 1 -- 172.21.15.19:0/2553417874 >> 172.21.15.19:0/2553417874 conn(0x7f4548061a20 msgr2=0x7f4548061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.346+0000 7f454fc09640 1 -- 172.21.15.19:0/2553417874 shutdown_connections 2024-04-25T10:05:02.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.346+0000 7f454fc09640 1 -- 172.21.15.19:0/2553417874 wait complete. 2024-04-25T10:05:02.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.347+0000 7f454fc09640 1 Processor -- start 2024-04-25T10:05:02.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.347+0000 7f454fc09640 1 -- start start 2024-04-25T10:05:02.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454fc09640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45480d11b0 0x7f454814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454fc09640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454815dd20 0x7f45480c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454fc09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f454815e100 0x7f45480c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454fc09640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454815e710 con 0x7f454815dd20 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454fc09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454815e590 con 0x7f454815e100 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454fc09640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4548158bf0 con 0x7f45480d11b0 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454d1a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454815dd20 0x7f45480c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454d1a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454815dd20 0x7f45480c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53358/0 (socket says 172.21.15.19:53358) 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454d1a4640 1 -- 172.21.15.19:0/2920374760 learned_addr learned my addr 172.21.15.19:0/2920374760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454e1a6640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f454815e100 0x7f45480c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454d9a5640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45480d11b0 0x7f454814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454d1a4640 1 -- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45480d11b0 msgr2=0x7f454814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454d1a4640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45480d11b0 0x7f454814f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454d1a4640 1 -- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f454815e100 msgr2=0x7f45480c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454d1a4640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f454815e100 0x7f45480c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454d1a4640 1 -- 172.21.15.19:0/2920374760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454814f610 con 0x7f454815dd20 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454d1a4640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454815dd20 0x7f45480c82b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f453800c570 tx=0x7f453800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f453e7fc640 1 -- 172.21.15.19:0/2920374760 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538018070 con 0x7f454815dd20 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f453e7fc640 1 -- 172.21.15.19:0/2920374760 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45380074e0 con 0x7f454815dd20 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f453e7fc640 1 -- 172.21.15.19:0/2920374760 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538004a20 con 0x7f454815dd20 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454fc09640 1 -- 172.21.15.19:0/2920374760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45480c8f40 con 0x7f454815dd20 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.348+0000 7f454fc09640 1 -- 172.21.15.19:0/2920374760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45480cabe0 con 0x7f454815dd20 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.349+0000 7f454fc09640 1 -- 172.21.15.19:0/2920374760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4548073990 con 0x7f454815dd20 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.350+0000 7f453e7fc640 1 -- 172.21.15.19:0/2920374760 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f453801b070 con 0x7f454815dd20 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.350+0000 7f453e7fc640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f452c06b4d0 0x7f452c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.350+0000 7f453e7fc640 1 -- 172.21.15.19:0/2920374760 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f45380896d0 con 0x7f454815dd20 2024-04-25T10:05:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.351+0000 7f454d9a5640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f452c06b4d0 0x7f452c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.351+0000 7f454d9a5640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f452c06b4d0 0x7f452c06d990 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4544034000 tx=0x7f454402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.355+0000 7f453e7fc640 1 -- 172.21.15.19:0/2920374760 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45380583e0 con 0x7f454815dd20 2024-04-25T10:05:02.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.534+0000 7f454fc09640 1 -- 172.21.15.19:0/2920374760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45480cf080 con 0x7f454815dd20 2024-04-25T10:05:02.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.535+0000 7f453e7fc640 1 -- 172.21.15.19:0/2920374760 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+17066 (secure 0 0 0) 0x7f4538087e20 con 0x7f454815dd20 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":167,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:04:49.642130+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:12.518429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:13.232769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2c","mappings":[{"from":3,"to":0}]},{"pgid":"26.31","mappings":[{"from":1,"to":0}]},{"pgid":"26.37","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 -- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f452c06b4d0 msgr2=0x7f452c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f452c06b4d0 0x7f452c06d990 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4544034000 tx=0x7f454402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 -- 172.21.15.19:0/2920374760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454815dd20 msgr2=0x7f45480c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454815dd20 0x7f45480c82b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f453800c570 tx=0x7f453800ca30 comp rx=0 tx=0).stop 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 -- 172.21.15.19:0/2920374760 shutdown_connections 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f452c06b4d0 0x7f452c06d990 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f454815e100 0x7f45480c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f454815dd20 0x7f45480c82b0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 --2- 172.21.15.19:0/2920374760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45480d11b0 0x7f454814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 -- 172.21.15.19:0/2920374760 >> 172.21.15.19:0/2920374760 conn(0x7f4548061a20 msgr2=0x7f45480b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 -- 172.21.15.19:0/2920374760 shutdown_connections 2024-04-25T10:05:02.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.537+0000 7f454fc09640 1 -- 172.21.15.19:0/2920374760 wait complete. 2024-04-25T10:05:02.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:05:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.650+0000 7f61b08fc640 1 Processor -- start 2024-04-25T10:05:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.650+0000 7f61b08fc640 1 -- start start 2024-04-25T10:05:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61b08fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61ac06abe0 0x7f61ac06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61b08fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f61ac0ad570 0x7f61ac06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61b08fc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac06b500 con 0x7f61ac15dd80 2024-04-25T10:05:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61b08fc640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ac06e410 con 0x7f61ac06abe0 2024-04-25T10:05:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61b08fc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ac0b3930 con 0x7f61ac0ad570 2024-04-25T10:05:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61aa59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61ac06abe0 0x7f61ac06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61aa59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61ac06abe0 0x7f61ac06afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53366/0 (socket says 172.21.15.19:53366) 2024-04-25T10:05:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61aa59c640 1 -- 172.21.15.19:0/1274454923 learned_addr learned my addr 172.21.15.19:0/1274454923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61aa59c640 1 -- 172.21.15.19:0/1274454923 >> v2:172.21.15.112:3300/0 conn(0x7f61ac0ad570 msgr2=0x7f61ac06ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61aa59c640 1 --2- 172.21.15.19:0/1274454923 >> v2:172.21.15.112:3300/0 conn(0x7f61ac0ad570 0x7f61ac06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61aa59c640 1 -- 172.21.15.19:0/1274454923 >> v1:172.21.15.112:6789/0 conn(0x7f61ac15dd80 legacy=0x7f61ac0b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.651+0000 7f61aa59c640 1 -- 172.21.15.19:0/1274454923 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac0b3c10 con 0x7f61ac06abe0 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.652+0000 7f61aa59c640 1 --2- 172.21.15.19:0/1274454923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac06abe0 0x7f61ac06afc0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f61ac06e590 tx=0x7f619c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4839ba206b1a92e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.652+0000 7f61a959a640 1 -- 172.21.15.19:0/1274454923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c037070 con 0x7f61ac06abe0 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.652+0000 7f61a959a640 1 -- 172.21.15.19:0/1274454923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c032440 con 0x7f61ac06abe0 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.652+0000 7f61a959a640 1 -- 172.21.15.19:0/1274454923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c032810 con 0x7f61ac06abe0 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.653+0000 7f61b08fc640 1 -- 172.21.15.19:0/1274454923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac06abe0 msgr2=0x7f61ac06afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.653+0000 7f61b08fc640 1 --2- 172.21.15.19:0/1274454923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac06abe0 0x7f61ac06afc0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f61ac06e590 tx=0x7f619c02a210 comp rx=0 tx=0).stop 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.653+0000 7f61b08fc640 1 -- 172.21.15.19:0/1274454923 shutdown_connections 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.653+0000 7f61b08fc640 1 --2- 172.21.15.19:0/1274454923 >> v2:172.21.15.112:3300/0 conn(0x7f61ac0ad570 0x7f61ac06ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.653+0000 7f61b08fc640 1 --2- 172.21.15.19:0/1274454923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac06abe0 0x7f61ac06afc0 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.653+0000 7f61b08fc640 1 -- 172.21.15.19:0/1274454923 >> 172.21.15.19:0/1274454923 conn(0x7f61ac0b5af0 msgr2=0x7f61ac0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.653+0000 7f61b08fc640 1 -- 172.21.15.19:0/1274454923 shutdown_connections 2024-04-25T10:05:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.653+0000 7f61b08fc640 1 -- 172.21.15.19:0/1274454923 wait complete. 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.654+0000 7f61b08fc640 1 Processor -- start 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.654+0000 7f61b08fc640 1 -- start start 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61b08fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac06abe0 0x7f61ac152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61b08fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61ac0ad570 0x7f61ac159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61b08fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac15dd80 0x7f61ac15c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61b08fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac070280 con 0x7f61ac06abe0 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61b08fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac06e940 con 0x7f61ac15dd80 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61b08fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac070400 con 0x7f61ac0ad570 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac06abe0 0x7f61ac152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac06abe0 0x7f61ac152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53372/0 (socket says 172.21.15.19:53372) 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61aa59c640 1 -- 172.21.15.19:0/869307227 learned_addr learned my addr 172.21.15.19:0/869307227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61aad9d640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac15dd80 0x7f61ac15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61a9d9b640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61ac0ad570 0x7f61ac159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61aa59c640 1 -- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61ac0ad570 msgr2=0x7f61ac159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61aa59c640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61ac0ad570 0x7f61ac159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61aa59c640 1 -- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac15dd80 msgr2=0x7f61ac15c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61aa59c640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac15dd80 0x7f61ac15c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.655+0000 7f61aa59c640 1 -- 172.21.15.19:0/869307227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac15a6f0 con 0x7f61ac06abe0 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.656+0000 7f61aad9d640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac15dd80 0x7f61ac15c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.656+0000 7f61aa59c640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac06abe0 0x7f61ac152d60 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f619c000c00 tx=0x7f619c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.656+0000 7f619affd640 1 -- 172.21.15.19:0/869307227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c041070 con 0x7f61ac06abe0 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.656+0000 7f619affd640 1 -- 172.21.15.19:0/869307227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c03c870 con 0x7f61ac06abe0 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.656+0000 7f61b08fc640 1 -- 172.21.15.19:0/869307227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac15a9d0 con 0x7f61ac06abe0 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.656+0000 7f619affd640 1 -- 172.21.15.19:0/869307227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c037040 con 0x7f61ac06abe0 2024-04-25T10:05:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.656+0000 7f61b08fc640 1 -- 172.21.15.19:0/869307227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac15af90 con 0x7f61ac06abe0 2024-04-25T10:05:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.657+0000 7f61b08fc640 1 -- 172.21.15.19:0/869307227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac15b770 con 0x7f61ac06abe0 2024-04-25T10:05:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.657+0000 7f619affd640 1 -- 172.21.15.19:0/869307227 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f619c044070 con 0x7f61ac06abe0 2024-04-25T10:05:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.658+0000 7f619affd640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f618c06b4d0 0x7f618c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.658+0000 7f619affd640 1 -- 172.21.15.19:0/869307227 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f619c0aaf10 con 0x7f61ac06abe0 2024-04-25T10:05:02.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.658+0000 7f61a9d9b640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f618c06b4d0 0x7f618c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.659+0000 7f61a9d9b640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f618c06b4d0 0x7f618c06d990 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f61a0006fd0 tx=0x7f61a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.662+0000 7f619affd640 1 -- 172.21.15.19:0/869307227 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61ac15b770 con 0x7f61ac06abe0 2024-04-25T10:05:02.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.879+0000 7f61b08fc640 1 -- 172.21.15.19:0/869307227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f61ac0b6320 con 0x7f61ac06abe0 2024-04-25T10:05:02.885 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T10:05:02.880+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:05:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.881+0000 7f619affd640 1 -- 172.21.15.19:0/869307227 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v110) v1 ==== 161+0+0 (secure 0 0 0) 0x7f619c079d40 con 0x7f61ac06abe0 2024-04-25T10:05:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:05:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 -- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f618c06b4d0 msgr2=0x7f618c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f618c06b4d0 0x7f618c06d990 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f61a0006fd0 tx=0x7f61a0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 -- 172.21.15.19:0/869307227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac06abe0 msgr2=0x7f61ac152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac06abe0 0x7f61ac152d60 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f619c000c00 tx=0x7f619c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 -- 172.21.15.19:0/869307227 shutdown_connections 2024-04-25T10:05:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f618c06b4d0 0x7f618c06d990 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac15dd80 0x7f61ac15c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61ac0ad570 0x7f61ac159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 --2- 172.21.15.19:0/869307227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61ac06abe0 0x7f61ac152d60 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 -- 172.21.15.19:0/869307227 >> 172.21.15.19:0/869307227 conn(0x7f61ac0b5af0 msgr2=0x7f61ac0b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 -- 172.21.15.19:0/869307227 shutdown_connections 2024-04-25T10:05:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:02.884+0000 7f61b08fc640 1 -- 172.21.15.19:0/869307227 wait complete. 2024-04-25T10:05:02.892 INFO:tasks.ceph.mds.c.smithi019.stderr: -6228> 2024-04-25T10:04:20.857+0000 7ffb5370b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:02.895 INFO:tasks.ceph.mds.c.smithi019.stderr: -3283> 2024-04-25T10:04:26.225+0000 7ffb4f703640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:04:25.199872+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={9373=0-12582912@1} caps={9373=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=9373 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x559473591600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(26)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:05:02.900 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:05:02.923 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:05:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.001+0000 7f04d60a3640 1 Processor -- start 2024-04-25T10:05:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.001+0000 7f04d60a3640 1 -- start start 2024-04-25T10:05:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.002+0000 7f04d60a3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04d006c9e0 0x7f04d00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.002+0000 7f04d60a3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d006cf50 con 0x7f04d00b5620 2024-04-25T10:05:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.002+0000 7f04d60a3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d00ce530 con 0x7f04d015b7d0 2024-04-25T10:05:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.002+0000 7f04d60a3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d00ce810 con 0x7f04d006c9e0 2024-04-25T10:05:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.002+0000 7f04ceffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f04d00b5620 0x7f04d006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55452/0 (socket says 172.21.15.19:55452) 2024-04-25T10:05:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.002+0000 7f04ceffd640 1 -- 172.21.15.19:0/2163197589 learned_addr learned my addr 172.21.15.19:0/2163197589 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.002+0000 7f04cffff640 1 --2- 172.21.15.19:0/2163197589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04d006c9e0 0x7f04d00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.002+0000 7f04ce7fc640 1 -- 172.21.15.19:0/2163197589 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379903654 0 0) 0x7f04d006cf50 con 0x7f04d00b5620 2024-04-25T10:05:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.002+0000 7f04ce7fc640 1 -- 172.21.15.19:0/2163197589 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04b8003520 con 0x7f04d00b5620 2024-04-25T10:05:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.002+0000 7f04ce7fc640 1 -- 172.21.15.19:0/2163197589 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792127014 0 0) 0x7f04d00ce530 con 0x7f04d015b7d0 2024-04-25T10:05:03.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.002+0000 7f04ce7fc640 1 -- 172.21.15.19:0/2163197589 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04d006cf50 con 0x7f04d015b7d0 2024-04-25T10:05:03.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.003+0000 7f04cffff640 1 -- 172.21.15.19:0/2163197589 >> v1:172.21.15.112:6789/0 conn(0x7f04d015b7d0 legacy=0x7f04d00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.003+0000 7f04cffff640 1 -- 172.21.15.19:0/2163197589 >> v1:172.21.15.19:6789/0 conn(0x7f04d00b5620 legacy=0x7f04d006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.003+0000 7f04cffff640 1 -- 172.21.15.19:0/2163197589 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d00ceaf0 con 0x7f04d006c9e0 2024-04-25T10:05:03.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.003+0000 7f04cffff640 1 --2- 172.21.15.19:0/2163197589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04d006c9e0 0x7f04d00b5ab0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009dc0 tx=0x7f04c802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7332f3905f88a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.004+0000 7f04ce7fc640 1 -- 172.21.15.19:0/2163197589 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8007740 con 0x7f04d006c9e0 2024-04-25T10:05:03.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.004+0000 7f04ce7fc640 1 -- 172.21.15.19:0/2163197589 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c80078e0 con 0x7f04d006c9e0 2024-04-25T10:05:03.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.004+0000 7f04ce7fc640 1 -- 172.21.15.19:0/2163197589 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8007c90 con 0x7f04d006c9e0 2024-04-25T10:05:03.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.004+0000 7f04d60a3640 1 -- 172.21.15.19:0/2163197589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04d006c9e0 msgr2=0x7f04d00b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.004+0000 7f04d60a3640 1 --2- 172.21.15.19:0/2163197589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04d006c9e0 0x7f04d00b5ab0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009dc0 tx=0x7f04c802a6d0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.004+0000 7f04d60a3640 1 -- 172.21.15.19:0/2163197589 shutdown_connections 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.004+0000 7f04d60a3640 1 --2- 172.21.15.19:0/2163197589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04d006c9e0 0x7f04d00b5ab0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.004+0000 7f04d60a3640 1 -- 172.21.15.19:0/2163197589 >> 172.21.15.19:0/2163197589 conn(0x7f04d00d0800 msgr2=0x7f04d00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.004+0000 7f04d60a3640 1 -- 172.21.15.19:0/2163197589 shutdown_connections 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.004+0000 7f04d60a3640 1 -- 172.21.15.19:0/2163197589 wait complete. 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.005+0000 7f04d60a3640 1 Processor -- start 2024-04-25T10:05:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.006+0000 7f04d60a3640 1 -- start start 2024-04-25T10:05:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.006+0000 7f04d60a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d006c9e0 0x7f04d0159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.006+0000 7f04d60a3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04d00b5620 0x7f04d00c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.006+0000 7f04d60a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d015b7d0 0x7f04d00b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.006+0000 7f04d60a3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d015bd30 con 0x7f04d015b7d0 2024-04-25T10:05:03.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.006+0000 7f04d60a3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d015bbb0 con 0x7f04d006c9e0 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.006+0000 7f04d60a3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d00d1f40 con 0x7f04d00b5620 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.006+0000 7f04cffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d015b7d0 0x7f04d00b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.006+0000 7f04cffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d015b7d0 0x7f04d00b85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53392/0 (socket says 172.21.15.19:53392) 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.006+0000 7f04cffff640 1 -- 172.21.15.19:0/3595386135 learned_addr learned my addr 172.21.15.19:0/3595386135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04cf7fe640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d006c9e0 0x7f04d0159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04ceffd640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04d00b5620 0x7f04d00c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04cffff640 1 -- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04d00b5620 msgr2=0x7f04d00c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04cffff640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04d00b5620 0x7f04d00c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04cffff640 1 -- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d006c9e0 msgr2=0x7f04d0159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04cffff640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d006c9e0 0x7f04d0159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04cffff640 1 -- 172.21.15.19:0/3595386135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d00c89d0 con 0x7f04d015b7d0 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04cffff640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d015b7d0 0x7f04d00b85f0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f04c802acc0 tx=0x7f04c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04b7fff640 1 -- 172.21.15.19:0/3595386135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8040070 con 0x7f04d015b7d0 2024-04-25T10:05:03.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04b7fff640 1 -- 172.21.15.19:0/3595386135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c8031c40 con 0x7f04d015b7d0 2024-04-25T10:05:03.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04d60a3640 1 -- 172.21.15.19:0/3595386135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d00c8cb0 con 0x7f04d015b7d0 2024-04-25T10:05:03.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04b7fff640 1 -- 172.21.15.19:0/3595386135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8009450 con 0x7f04d015b7d0 2024-04-25T10:05:03.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.007+0000 7f04d60a3640 1 -- 172.21.15.19:0/3595386135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d00c9270 con 0x7f04d015b7d0 2024-04-25T10:05:03.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.008+0000 7f04d60a3640 1 -- 172.21.15.19:0/3595386135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d01f6ff0 con 0x7f04d015b7d0 2024-04-25T10:05:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.009+0000 7f04b7fff640 1 -- 172.21.15.19:0/3595386135 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04c8039040 con 0x7f04d015b7d0 2024-04-25T10:05:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.009+0000 7f04b7fff640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f04bc06b500 0x7f04bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.009+0000 7f04b7fff640 1 -- 172.21.15.19:0/3595386135 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f04c80ab810 con 0x7f04d015b7d0 2024-04-25T10:05:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.009+0000 7f04cf7fe640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f04bc06b500 0x7f04bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.010+0000 7f04cf7fe640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f04bc06b500 0x7f04bc06d9c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f04c00015b0 tx=0x7f04c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.013+0000 7f04b7fff640 1 -- 172.21.15.19:0/3595386135 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04c804a060 con 0x7f04d015b7d0 2024-04-25T10:05:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.236+0000 7f04d60a3640 1 -- 172.21.15.19:0/3595386135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f04d0161400 con 0x7f04d015b7d0 2024-04-25T10:05:03.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.886+0000 7f04b7fff640 1 -- 172.21.15.19:0/3595386135 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v111) v1 ==== 104+0+0 (secure 0 0 0) 0x7f04c8038350 con 0x7f04d015b7d0 2024-04-25T10:05:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 -- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f04bc06b500 msgr2=0x7f04bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f04bc06b500 0x7f04bc06d9c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f04c00015b0 tx=0x7f04c0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 -- 172.21.15.19:0/3595386135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d015b7d0 msgr2=0x7f04d00b85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d015b7d0 0x7f04d00b85f0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f04c802acc0 tx=0x7f04c8004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 -- 172.21.15.19:0/3595386135 shutdown_connections 2024-04-25T10:05:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f04bc06b500 0x7f04bc06d9c0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04d015b7d0 0x7f04d00b85f0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04d00b5620 0x7f04d00c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 --2- 172.21.15.19:0/3595386135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d006c9e0 0x7f04d0159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 -- 172.21.15.19:0/3595386135 >> 172.21.15.19:0/3595386135 conn(0x7f04d00d0800 msgr2=0x7f04d015ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 -- 172.21.15.19:0/3595386135 shutdown_connections 2024-04-25T10:05:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:03.889+0000 7f04d60a3640 1 -- 172.21.15.19:0/3595386135 wait complete. 2024-04-25T10:05:03.908 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:05:04.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.053+0000 7f7a567a9640 1 Processor -- start 2024-04-25T10:05:04.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.054+0000 7f7a567a9640 1 -- start start 2024-04-25T10:05:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.054+0000 7f7a567a9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a5015bb50 0x7f7a500c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.054+0000 7f7a567a9640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a500c75e0 con 0x7f7a500cfd10 2024-04-25T10:05:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.054+0000 7f7a567a9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a500c78c0 con 0x7f7a500c70d0 2024-04-25T10:05:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.054+0000 7f7a567a9640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a500c8680 con 0x7f7a5015bb50 2024-04-25T10:05:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.055+0000 7f7a4ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a5015bb50 0x7f7a500c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.055+0000 7f7a54d46640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7a500cfd10 0x7f7a500cbf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55464/0 (socket says 172.21.15.19:55464) 2024-04-25T10:05:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.055+0000 7f7a54d46640 1 -- 172.21.15.19:0/4232163337 learned_addr learned my addr 172.21.15.19:0/4232163337 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.055+0000 7f7a4ffff640 1 -- 172.21.15.19:0/4232163337 >> v1:172.21.15.112:6789/0 conn(0x7f7a500c70d0 legacy=0x7f7a500cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.055+0000 7f7a4ffff640 1 -- 172.21.15.19:0/4232163337 >> v1:172.21.15.19:6789/0 conn(0x7f7a500cfd10 legacy=0x7f7a500cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.055+0000 7f7a4ffff640 1 -- 172.21.15.19:0/4232163337 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a500c7ba0 con 0x7f7a5015bb50 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.056+0000 7f7a4ffff640 1 --2- 172.21.15.19:0/4232163337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a5015bb50 0x7f7a500c6b90 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f7a400097a0 tx=0x7f7a4002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9457c60bcbba08e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.056+0000 7f7a4effd640 1 -- 172.21.15.19:0/4232163337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40037070 con 0x7f7a5015bb50 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.056+0000 7f7a4effd640 1 -- 172.21.15.19:0/4232163337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a40032440 con 0x7f7a5015bb50 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.056+0000 7f7a4effd640 1 -- 172.21.15.19:0/4232163337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40032810 con 0x7f7a5015bb50 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.056+0000 7f7a567a9640 1 -- 172.21.15.19:0/4232163337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a5015bb50 msgr2=0x7f7a500c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.056+0000 7f7a567a9640 1 --2- 172.21.15.19:0/4232163337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a5015bb50 0x7f7a500c6b90 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f7a400097a0 tx=0x7f7a4002a270 comp rx=0 tx=0).stop 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.056+0000 7f7a567a9640 1 -- 172.21.15.19:0/4232163337 shutdown_connections 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.056+0000 7f7a567a9640 1 --2- 172.21.15.19:0/4232163337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a5015bb50 0x7f7a500c6b90 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.056+0000 7f7a567a9640 1 -- 172.21.15.19:0/4232163337 >> 172.21.15.19:0/4232163337 conn(0x7f7a5006f500 msgr2=0x7f7a5006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.056+0000 7f7a567a9640 1 -- 172.21.15.19:0/4232163337 shutdown_connections 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.056+0000 7f7a567a9640 1 -- 172.21.15.19:0/4232163337 wait complete. 2024-04-25T10:05:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.058+0000 7f7a567a9640 1 Processor -- start 2024-04-25T10:05:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.058+0000 7f7a567a9640 1 -- start start 2024-04-25T10:05:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.058+0000 7f7a567a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a500c70d0 0x7f7a5015d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a567a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500cfd10 0x7f7a50150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a567a9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a5015bb50 0x7f7a5013ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a567a9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5015f6d0 con 0x7f7a500c70d0 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a567a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500c8160 con 0x7f7a500cfd10 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a567a9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5015f850 con 0x7f7a5015bb50 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a500c70d0 0x7f7a5015d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a500c70d0 0x7f7a5015d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53420/0 (socket says 172.21.15.19:53420) 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4ffff640 1 -- 172.21.15.19:0/2575881698 learned_addr learned my addr 172.21.15.19:0/2575881698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4ffff640 1 -- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a5015bb50 msgr2=0x7f7a5013ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4ffff640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a5015bb50 0x7f7a5013ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4ffff640 1 -- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500cfd10 msgr2=0x7f7a50150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4ffff640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500cfd10 0x7f7a50150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4ffff640 1 -- 172.21.15.19:0/2575881698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5015d9f0 con 0x7f7a500c70d0 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4ffff640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a500c70d0 0x7f7a5015d010 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f7a400098c0 tx=0x7f7a400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4cff9640 1 -- 172.21.15.19:0/2575881698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40037040 con 0x7f7a500c70d0 2024-04-25T10:05:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4cff9640 1 -- 172.21.15.19:0/2575881698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a40042070 con 0x7f7a500c70d0 2024-04-25T10:05:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a4cff9640 1 -- 172.21.15.19:0/2575881698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40030a70 con 0x7f7a500c70d0 2024-04-25T10:05:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a567a9640 1 -- 172.21.15.19:0/2575881698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5015dcd0 con 0x7f7a500c70d0 2024-04-25T10:05:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.059+0000 7f7a567a9640 1 -- 172.21.15.19:0/2575881698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5015e290 con 0x7f7a500c70d0 2024-04-25T10:05:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.060+0000 7f7a567a9640 1 -- 172.21.15.19:0/2575881698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a50073a00 con 0x7f7a500c70d0 2024-04-25T10:05:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.061+0000 7f7a4cff9640 1 -- 172.21.15.19:0/2575881698 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a40030c10 con 0x7f7a500c70d0 2024-04-25T10:05:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.061+0000 7f7a4cff9640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a3c06b4d0 0x7f7a3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.061+0000 7f7a4cff9640 1 -- 172.21.15.19:0/2575881698 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f7a400aafc0 con 0x7f7a500c70d0 2024-04-25T10:05:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.061+0000 7f7a4f7fe640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a3c06b4d0 0x7f7a3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.062+0000 7f7a4f7fe640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a3c06b4d0 0x7f7a3c06d990 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7a5015ec00 tx=0x7f7a440079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.065+0000 7f7a4cff9640 1 -- 172.21.15.19:0/2575881698 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a40045360 con 0x7f7a500c70d0 2024-04-25T10:05:04.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.247+0000 7f7a567a9640 1 -- 172.21.15.19:0/2575881698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7a500b6fa0 con 0x7f7a500c70d0 2024-04-25T10:05:04.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.888+0000 7f7a4cff9640 1 -- 172.21.15.19:0/2575881698 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v169) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7a40079b60 con 0x7f7a500c70d0 2024-04-25T10:05:04.893 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:05:04.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.890+0000 7f7a567a9640 1 -- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a3c06b4d0 msgr2=0x7f7a3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.890+0000 7f7a567a9640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a3c06b4d0 0x7f7a3c06d990 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7a5015ec00 tx=0x7f7a440079d0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.890+0000 7f7a567a9640 1 -- 172.21.15.19:0/2575881698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a500c70d0 msgr2=0x7f7a5015d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.890+0000 7f7a567a9640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a500c70d0 0x7f7a5015d010 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f7a400098c0 tx=0x7f7a400046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.891+0000 7f7a567a9640 1 -- 172.21.15.19:0/2575881698 shutdown_connections 2024-04-25T10:05:04.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.891+0000 7f7a567a9640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f7a3c06b4d0 0x7f7a3c06d990 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.891+0000 7f7a567a9640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a5015bb50 0x7f7a5013ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.891+0000 7f7a567a9640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a500cfd10 0x7f7a50150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.891+0000 7f7a567a9640 1 --2- 172.21.15.19:0/2575881698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a500c70d0 0x7f7a5015d010 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.891+0000 7f7a567a9640 1 -- 172.21.15.19:0/2575881698 >> 172.21.15.19:0/2575881698 conn(0x7f7a5006f500 msgr2=0x7f7a500b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.891+0000 7f7a567a9640 1 -- 172.21.15.19:0/2575881698 shutdown_connections 2024-04-25T10:05:04.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:04.891+0000 7f7a567a9640 1 -- 172.21.15.19:0/2575881698 wait complete. 2024-04-25T10:05:04.912 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:05:05.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.019+0000 7f2ef5476640 1 Processor -- start 2024-04-25T10:05:05.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.019+0000 7f2ef5476640 1 -- start start 2024-04-25T10:05:05.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.019+0000 7f2ef5476640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ef015de80 0x7f2ef00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.020+0000 7f2ef5476640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef00afc80 con 0x7f2ef00cefb0 2024-04-25T10:05:05.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.020+0000 7f2ef5476640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef00ce4d0 con 0x7f2ef00ccfe0 2024-04-25T10:05:05.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.020+0000 7f2ef5476640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef00ce7b0 con 0x7f2ef015de80 2024-04-25T10:05:05.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.020+0000 7f2eeeffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ef015de80 0x7f2ef00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.020+0000 7f2ee7fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2ef00cefb0 0x7f2ef00cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55480/0 (socket says 172.21.15.19:55480) 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.020+0000 7f2ee7fff640 1 -- 172.21.15.19:0/3462551376 learned_addr learned my addr 172.21.15.19:0/3462551376 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.020+0000 7f2eeeffd640 1 -- 172.21.15.19:0/3462551376 >> v1:172.21.15.112:6789/0 conn(0x7f2ef00ccfe0 legacy=0x7f2ef00cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.020+0000 7f2eeeffd640 1 -- 172.21.15.19:0/3462551376 >> v1:172.21.15.19:6789/0 conn(0x7f2ef00cefb0 legacy=0x7f2ef00cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.020+0000 7f2eee7fc640 1 -- 172.21.15.19:0/3462551376 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255076182 0 0) 0x7f2ef00afc80 con 0x7f2ef00cefb0 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.020+0000 7f2eeeffd640 1 -- 172.21.15.19:0/3462551376 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00cea90 con 0x7f2ef015de80 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.021+0000 7f2eeeffd640 1 --2- 172.21.15.19:0/3462551376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ef015de80 0x7f2ef00cadb0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00097a0 tx=0x7f2ee002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22ae126719e8c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.021+0000 7f2eee7fc640 1 -- 172.21.15.19:0/3462551376 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0037070 con 0x7f2ef015de80 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.021+0000 7f2eee7fc640 1 -- 172.21.15.19:0/3462551376 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee0032440 con 0x7f2ef015de80 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.021+0000 7f2eee7fc640 1 -- 172.21.15.19:0/3462551376 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0032810 con 0x7f2ef015de80 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.021+0000 7f2ef5476640 1 -- 172.21.15.19:0/3462551376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ef015de80 msgr2=0x7f2ef00cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.021+0000 7f2ef5476640 1 --2- 172.21.15.19:0/3462551376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ef015de80 0x7f2ef00cadb0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00097a0 tx=0x7f2ee002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.022+0000 7f2ef5476640 1 -- 172.21.15.19:0/3462551376 shutdown_connections 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.022+0000 7f2ef5476640 1 --2- 172.21.15.19:0/3462551376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ef015de80 0x7f2ef00cadb0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.022+0000 7f2ef5476640 1 -- 172.21.15.19:0/3462551376 >> 172.21.15.19:0/3462551376 conn(0x7f2ef00b2390 msgr2=0x7f2ef00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.022+0000 7f2ef5476640 1 -- 172.21.15.19:0/3462551376 shutdown_connections 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.022+0000 7f2ef5476640 1 -- 172.21.15.19:0/3462551376 wait complete. 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.023+0000 7f2ef5476640 1 Processor -- start 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.023+0000 7f2ef5476640 1 -- start start 2024-04-25T10:05:05.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.023+0000 7f2ef5476640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ef00ccfe0 0x7f2ef0152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.023+0000 7f2ef5476640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ef00cefb0 0x7f2ef00ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.023+0000 7f2ef5476640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef015de80 0x7f2ef01e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.023+0000 7f2ef5476640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00cfb50 con 0x7f2ef00cefb0 2024-04-25T10:05:05.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.023+0000 7f2ef5476640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00b2eb0 con 0x7f2ef015de80 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.023+0000 7f2ef5476640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef0153ae0 con 0x7f2ef00ccfe0 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ef00cefb0 0x7f2ef00ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ef00cefb0 0x7f2ef00ae980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53434/0 (socket says 172.21.15.19:53434) 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee7fff640 1 -- 172.21.15.19:0/1600424676 learned_addr learned my addr 172.21.15.19:0/1600424676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2eeeffd640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ef00ccfe0 0x7f2ef0152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2eef7fe640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef015de80 0x7f2ef01e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee7fff640 1 -- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ef00ccfe0 msgr2=0x7f2ef0152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee7fff640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ef00ccfe0 0x7f2ef0152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee7fff640 1 -- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef015de80 msgr2=0x7f2ef01e4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee7fff640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef015de80 0x7f2ef01e4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee7fff640 1 -- 172.21.15.19:0/1600424676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00ad600 con 0x7f2ef00cefb0 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2eeeffd640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ef00ccfe0 0x7f2ef0152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2eef7fe640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef015de80 0x7f2ef01e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:05.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee7fff640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ef00cefb0 0x7f2ef00ae980 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c4d0 tx=0x7f2ed800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee77fe640 1 -- 172.21.15.19:0/1600424676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8018070 con 0x7f2ef00cefb0 2024-04-25T10:05:05.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee77fe640 1 -- 172.21.15.19:0/1600424676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed8007500 con 0x7f2ef00cefb0 2024-04-25T10:05:05.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ef5476640 1 -- 172.21.15.19:0/1600424676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef00ad8e0 con 0x7f2ef00cefb0 2024-04-25T10:05:05.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ee77fe640 1 -- 172.21.15.19:0/1600424676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8004a20 con 0x7f2ef00cefb0 2024-04-25T10:05:05.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.024+0000 7f2ef5476640 1 -- 172.21.15.19:0/1600424676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef00adea0 con 0x7f2ef00cefb0 2024-04-25T10:05:05.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.025+0000 7f2ef5476640 1 -- 172.21.15.19:0/1600424676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef0073a00 con 0x7f2ef00cefb0 2024-04-25T10:05:05.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.026+0000 7f2ee77fe640 1 -- 172.21.15.19:0/1600424676 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ed801b070 con 0x7f2ef00cefb0 2024-04-25T10:05:05.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.026+0000 7f2ee77fe640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ed006b4d0 0x7f2ed006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.026+0000 7f2ee77fe640 1 -- 172.21.15.19:0/1600424676 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5865+0+0 (secure 0 0 0) 0x7f2ed80895a0 con 0x7f2ef00cefb0 2024-04-25T10:05:05.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.027+0000 7f2eeeffd640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ed006b4d0 0x7f2ed006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.027+0000 7f2eeeffd640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ed006b4d0 0x7f2ed006d990 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0034000 tx=0x7f2ee002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.030+0000 7f2ee77fe640 1 -- 172.21.15.19:0/1600424676 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ed8058380 con 0x7f2ef00cefb0 2024-04-25T10:05:05.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.211+0000 7f2ef5476640 1 -- 172.21.15.19:0/1600424676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2ef0153530 con 0x7f2ef00cefb0 2024-04-25T10:05:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.895+0000 7f2ee77fe640 1 -- 172.21.15.19:0/1600424676 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v170) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2ed8057c80 con 0x7f2ef00cefb0 2024-04-25T10:05:05.900 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T10:05:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 -- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ed006b4d0 msgr2=0x7f2ed006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ed006b4d0 0x7f2ed006d990 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0034000 tx=0x7f2ee002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 -- 172.21.15.19:0/1600424676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ef00cefb0 msgr2=0x7f2ef00ae980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ef00cefb0 0x7f2ef00ae980 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c4d0 tx=0x7f2ed800c990 comp rx=0 tx=0).stop 2024-04-25T10:05:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 -- 172.21.15.19:0/1600424676 shutdown_connections 2024-04-25T10:05:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2ed006b4d0 0x7f2ed006d990 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ef015de80 0x7f2ef01e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ef00cefb0 0x7f2ef00ae980 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 --2- 172.21.15.19:0/1600424676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ef00ccfe0 0x7f2ef0152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 -- 172.21.15.19:0/1600424676 >> 172.21.15.19:0/1600424676 conn(0x7f2ef00b2390 msgr2=0x7f2ef0153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 -- 172.21.15.19:0/1600424676 shutdown_connections 2024-04-25T10:05:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:05.897+0000 7f2ef5476640 1 -- 172.21.15.19:0/1600424676 wait complete. 2024-04-25T10:05:05.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:05:06.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.022+0000 7f17f8615640 1 Processor -- start 2024-04-25T10:05:06.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.022+0000 7f17f8615640 1 -- start start 2024-04-25T10:05:06.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.023+0000 7f17f8615640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f17f00cfa40 0x7f17f00cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.023+0000 7f17f8615640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f00c75e0 con 0x7f17f00cf600 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.023+0000 7f17f8615640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f00c78c0 con 0x7f17f015bb50 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.023+0000 7f17f8615640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f00c8680 con 0x7f17f00cfa40 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.023+0000 7f17f5bb0640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f17f00cf600 0x7f17f00cbf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55488/0 (socket says 172.21.15.19:55488) 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f5bb0640 1 -- 172.21.15.19:0/4253464273 learned_addr learned my addr 172.21.15.19:0/4253464273 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f6bb2640 1 --2- 172.21.15.19:0/4253464273 >> v2:172.21.15.112:3300/0 conn(0x7f17f00cfa40 0x7f17f00cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764919521 0 0) 0x7f17f00c75e0 con 0x7f17f00cf600 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17dc003520 con 0x7f17f00cf600 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 110040480 0 0) 0x7f17dc003520 con 0x7f17f00cf600 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 >> v2:172.21.15.112:3300/0 conn(0x7f17f00cfa40 msgr2=0x7f17f00cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 --2- 172.21.15.19:0/4253464273 >> v2:172.21.15.112:3300/0 conn(0x7f17f00cfa40 0x7f17f00cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 >> v1:172.21.15.112:6789/0 conn(0x7f17f015bb50 legacy=0x7f17f00c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f00c7ba0 con 0x7f17f00cf600 2024-04-25T10:05:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f17e0001670 con 0x7f17f00cf600 2024-04-25T10:05:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 >> v1:172.21.15.19:6789/0 conn(0x7f17f00cf600 legacy=0x7f17f00cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17dc0039b0 0x7f17dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc0043b0 0x7f17dc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17dc005080 0x7f17dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f00c8680 con 0x7f17dc0043b0 2024-04-25T10:05:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc00bf30 con 0x7f17dc005080 2024-04-25T10:05:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc00c130 con 0x7f17dc0039b0 2024-04-25T10:05:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f5bb0640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc0043b0 0x7f17dc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f63b1640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17dc0039b0 0x7f17dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.024+0000 7f17f6bb2640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17dc005080 0x7f17dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.025+0000 7f17f5bb0640 1 -- 172.21.15.19:0/4253464273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17dc0039b0 msgr2=0x7f17dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.025+0000 7f17f5bb0640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17dc0039b0 0x7f17dc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.025+0000 7f17f5bb0640 1 -- 172.21.15.19:0/4253464273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17dc005080 msgr2=0x7f17dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.025+0000 7f17f5bb0640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17dc005080 0x7f17dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.025+0000 7f17f5bb0640 1 -- 172.21.15.19:0/4253464273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc003520 con 0x7f17dc0043b0 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.025+0000 7f17f6bb2640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17dc005080 0x7f17dc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.025+0000 7f17f6bb2640 1 -- 172.21.15.19:0/4253464273 reap_dead start 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.025+0000 7f17f5bb0640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc0043b0 0x7f17dc004880 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f17e000c290 tx=0x7f17dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9365751786750f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.025+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0007d50 con 0x7f17dc0043b0 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.026+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e00058d0 con 0x7f17dc0043b0 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.026+0000 7f17f53af640 1 -- 172.21.15.19:0/4253464273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0005d20 con 0x7f17dc0043b0 2024-04-25T10:05:06.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.026+0000 7f17f8615640 1 -- 172.21.15.19:0/4253464273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc0043b0 msgr2=0x7f17dc004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.026+0000 7f17f8615640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc0043b0 0x7f17dc004880 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f17e000c290 tx=0x7f17dc005830 comp rx=0 tx=0).stop 2024-04-25T10:05:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.026+0000 7f17f8615640 1 -- 172.21.15.19:0/4253464273 shutdown_connections 2024-04-25T10:05:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.026+0000 7f17f8615640 1 --2- 172.21.15.19:0/4253464273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17dc0043b0 0x7f17dc004880 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.026+0000 7f17f8615640 1 -- 172.21.15.19:0/4253464273 >> 172.21.15.19:0/4253464273 conn(0x7f17f006f500 msgr2=0x7f17f006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.026+0000 7f17f8615640 1 -- 172.21.15.19:0/4253464273 shutdown_connections 2024-04-25T10:05:06.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.026+0000 7f17f8615640 1 -- 172.21.15.19:0/4253464273 wait complete. 2024-04-25T10:05:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.027+0000 7f17f8615640 1 Processor -- start 2024-04-25T10:05:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.027+0000 7f17f8615640 1 -- start start 2024-04-25T10:05:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f8615640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17dc0043b0 0x7f17f00cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f8615640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17f015d5c0 0x7f17f00afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f8615640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f00c9260 0x7f17f0150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f8615640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f00cfe90 con 0x7f17f00c9260 2024-04-25T10:05:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f8615640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f00c8160 con 0x7f17f015d5c0 2024-04-25T10:05:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f8615640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f015f6d0 con 0x7f17dc0043b0 2024-04-25T10:05:06.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f6bb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f00c9260 0x7f17f0150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f6bb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f00c9260 0x7f17f0150e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53448/0 (socket says 172.21.15.19:53448) 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f6bb2640 1 -- 172.21.15.19:0/811671315 learned_addr learned my addr 172.21.15.19:0/811671315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f5bb0640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17f015d5c0 0x7f17f00afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f63b1640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17dc0043b0 0x7f17f00cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f6bb2640 1 -- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17dc0043b0 msgr2=0x7f17f00cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f6bb2640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17dc0043b0 0x7f17f00cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f6bb2640 1 -- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17f015d5c0 msgr2=0x7f17f00afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f6bb2640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17f015d5c0 0x7f17f00afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.028+0000 7f17f6bb2640 1 -- 172.21.15.19:0/811671315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f015da40 con 0x7f17f00c9260 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.029+0000 7f17f5bb0640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17f015d5c0 0x7f17f00afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.029+0000 7f17f63b1640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17dc0043b0 0x7f17f00cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.029+0000 7f17f6bb2640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f00c9260 0x7f17f0150e10 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f17f015f450 tx=0x7f17e8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.029+0000 7f17e6ffd640 1 -- 172.21.15.19:0/811671315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8015440 con 0x7f17f00c9260 2024-04-25T10:05:06.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.029+0000 7f17e6ffd640 1 -- 172.21.15.19:0/811671315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e801c070 con 0x7f17f00c9260 2024-04-25T10:05:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.029+0000 7f17f8615640 1 -- 172.21.15.19:0/811671315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f015dcd0 con 0x7f17f00c9260 2024-04-25T10:05:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.029+0000 7f17e6ffd640 1 -- 172.21.15.19:0/811671315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8015820 con 0x7f17f00c9260 2024-04-25T10:05:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.029+0000 7f17f8615640 1 -- 172.21.15.19:0/811671315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f015e290 con 0x7f17f00c9260 2024-04-25T10:05:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.030+0000 7f17f8615640 1 -- 172.21.15.19:0/811671315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f0073a00 con 0x7f17f00c9260 2024-04-25T10:05:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.030+0000 7f17e6ffd640 1 -- 172.21.15.19:0/811671315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17e8005d10 con 0x7f17f00c9260 2024-04-25T10:05:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.031+0000 7f17e6ffd640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17d406b4d0 0x7f17d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.031+0000 7f17e6ffd640 1 -- 172.21.15.19:0/811671315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f17e808aac0 con 0x7f17f00c9260 2024-04-25T10:05:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.031+0000 7f17f63b1640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17d406b4d0 0x7f17d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.031+0000 7f17f63b1640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17d406b4d0 0x7f17d406d990 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f17f00b30e0 tx=0x7f17ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.035+0000 7f17e6ffd640 1 -- 172.21.15.19:0/811671315 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17e8059a10 con 0x7f17f00c9260 2024-04-25T10:05:06.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.214+0000 7f17f8615640 1 -- 172.21.15.19:0/811671315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f17f015b520 con 0x7f17f00c9260 2024-04-25T10:05:06.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.219+0000 7f17e6ffd640 1 -- 172.21.15.19:0/811671315 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f17e808ae20 con 0x7f17f00c9260 2024-04-25T10:05:06.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 -- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17d406b4d0 msgr2=0x7f17d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17d406b4d0 0x7f17d406d990 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f17f00b30e0 tx=0x7f17ec009040 comp rx=0 tx=0).stop 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 -- 172.21.15.19:0/811671315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f00c9260 msgr2=0x7f17f0150e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f00c9260 0x7f17f0150e10 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f17f015f450 tx=0x7f17e8010590 comp rx=0 tx=0).stop 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 -- 172.21.15.19:0/811671315 shutdown_connections 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17d406b4d0 0x7f17d406d990 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17f00c9260 0x7f17f0150e10 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17f015d5c0 0x7f17f00afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 --2- 172.21.15.19:0/811671315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17dc0043b0 0x7f17f00cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 -- 172.21.15.19:0/811671315 >> 172.21.15.19:0/811671315 conn(0x7f17f006f500 msgr2=0x7f17f00d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 -- 172.21.15.19:0/811671315 shutdown_connections 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.221+0000 7f17f8615640 1 -- 172.21.15.19:0/811671315 wait complete. 2024-04-25T10:05:06.238 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:05:06.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.333+0000 7f508b0d8640 1 Processor -- start 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.333+0000 7f508b0d8640 1 -- start start 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f508b0d8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508415dd30 0x7f50840b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f508b0d8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f50840b9770 0x7f50840d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f508b0d8640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50840d2e90 0x7f50840d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f508b0d8640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50840d1df0 con 0x7f50840d2e90 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f508b0d8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50840d20d0 con 0x7f50840b9770 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f508b0d8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50840d23b0 con 0x7f508415dd30 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f5089675640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50840d2e90 0x7f50840d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f5089675640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50840d2e90 0x7f50840d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53456/0 (socket says 172.21.15.19:53456) 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f5089675640 1 -- 172.21.15.19:0/3445249499 learned_addr learned my addr 172.21.15.19:0/3445249499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f5088e74640 1 --2- 172.21.15.19:0/3445249499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508415dd30 0x7f50840b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f5083fff640 1 --2- 172.21.15.19:0/3445249499 >> v2:172.21.15.112:3300/0 conn(0x7f50840b9770 0x7f50840d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f5089675640 1 -- 172.21.15.19:0/3445249499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508415dd30 msgr2=0x7f50840b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:06.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f5089675640 1 --2- 172.21.15.19:0/3445249499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508415dd30 0x7f50840b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f5089675640 1 -- 172.21.15.19:0/3445249499 >> v2:172.21.15.112:3300/0 conn(0x7f50840b9770 msgr2=0x7f50840d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:06.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f5089675640 1 --2- 172.21.15.19:0/3445249499 >> v2:172.21.15.112:3300/0 conn(0x7f50840b9770 0x7f50840d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.334+0000 7f5089675640 1 -- 172.21.15.19:0/3445249499 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50840d2690 con 0x7f50840d2e90 2024-04-25T10:05:06.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.335+0000 7f5083fff640 1 --2- 172.21.15.19:0/3445249499 >> v2:172.21.15.112:3300/0 conn(0x7f50840b9770 0x7f50840d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:06.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.335+0000 7f5089675640 1 --2- 172.21.15.19:0/3445249499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50840d2e90 0x7f50840d0900 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f50840d49c0 tx=0x7f507402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f813e3c0e34f6964 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.335+0000 7f50837fe640 1 -- 172.21.15.19:0/3445249499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074037070 con 0x7f50840d2e90 2024-04-25T10:05:06.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.335+0000 7f50837fe640 1 -- 172.21.15.19:0/3445249499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5074032440 con 0x7f50840d2e90 2024-04-25T10:05:06.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.335+0000 7f50837fe640 1 -- 172.21.15.19:0/3445249499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074032810 con 0x7f50840d2e90 2024-04-25T10:05:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.336+0000 7f508b0d8640 1 -- 172.21.15.19:0/3445249499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50840d2e90 msgr2=0x7f50840d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.336+0000 7f508b0d8640 1 --2- 172.21.15.19:0/3445249499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50840d2e90 0x7f50840d0900 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f50840d49c0 tx=0x7f507402a140 comp rx=0 tx=0).stop 2024-04-25T10:05:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.336+0000 7f508b0d8640 1 -- 172.21.15.19:0/3445249499 shutdown_connections 2024-04-25T10:05:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.336+0000 7f508b0d8640 1 --2- 172.21.15.19:0/3445249499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50840d2e90 0x7f50840d0900 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.336+0000 7f508b0d8640 1 --2- 172.21.15.19:0/3445249499 >> v2:172.21.15.112:3300/0 conn(0x7f50840b9770 0x7f50840d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.336+0000 7f508b0d8640 1 --2- 172.21.15.19:0/3445249499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508415dd30 0x7f50840b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.336+0000 7f508b0d8640 1 -- 172.21.15.19:0/3445249499 >> 172.21.15.19:0/3445249499 conn(0x7f5084061c30 msgr2=0x7f508406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.336+0000 7f508b0d8640 1 -- 172.21.15.19:0/3445249499 shutdown_connections 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.336+0000 7f508b0d8640 1 -- 172.21.15.19:0/3445249499 wait complete. 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.337+0000 7f508b0d8640 1 Processor -- start 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.337+0000 7f508b0d8640 1 -- start start 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.338+0000 7f508b0d8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50840b9770 0x7f50841f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.338+0000 7f508b0d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50840d2e90 0x7f50840b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.338+0000 7f508b0d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f508415dd30 0x7f508414d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.338+0000 7f508b0d8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50840d1b90 con 0x7f508415dd30 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.338+0000 7f508b0d8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50840d1a10 con 0x7f50840d2e90 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.338+0000 7f508b0d8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50840b5430 con 0x7f50840b9770 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.338+0000 7f5089675640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f508415dd30 0x7f508414d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.338+0000 7f5089675640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f508415dd30 0x7f508414d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53472/0 (socket says 172.21.15.19:53472) 2024-04-25T10:05:06.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.338+0000 7f5089675640 1 -- 172.21.15.19:0/1384024554 learned_addr learned my addr 172.21.15.19:0/1384024554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.338+0000 7f5083fff640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50840d2e90 0x7f50840b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.338+0000 7f5088e74640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50840b9770 0x7f50841f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f5089675640 1 -- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50840b9770 msgr2=0x7f50841f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f5089675640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50840b9770 0x7f50841f39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f5089675640 1 -- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50840d2e90 msgr2=0x7f50840b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f5089675640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50840d2e90 0x7f50840b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f5089675640 1 -- 172.21.15.19:0/1384024554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5084157c80 con 0x7f508415dd30 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f5083fff640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50840d2e90 0x7f50840b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f5089675640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f508415dd30 0x7f508414d850 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f50740090b0 tx=0x7f50740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f50817fa640 1 -- 172.21.15.19:0/1384024554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074037040 con 0x7f508415dd30 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f50817fa640 1 -- 172.21.15.19:0/1384024554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5074042070 con 0x7f508415dd30 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f508b0d8640 1 -- 172.21.15.19:0/1384024554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508415f6a0 con 0x7f508415dd30 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f50817fa640 1 -- 172.21.15.19:0/1384024554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074030920 con 0x7f508415dd30 2024-04-25T10:05:06.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.339+0000 7f508b0d8640 1 -- 172.21.15.19:0/1384024554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508415fc60 con 0x7f508415dd30 2024-04-25T10:05:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.340+0000 7f508b0d8640 1 -- 172.21.15.19:0/1384024554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5084160150 con 0x7f508415dd30 2024-04-25T10:05:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.340+0000 7f50817fa640 1 -- 172.21.15.19:0/1384024554 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5074030ac0 con 0x7f508415dd30 2024-04-25T10:05:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.341+0000 7f50817fa640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f506406b4d0 0x7f506406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.341+0000 7f50817fa640 1 -- 172.21.15.19:0/1384024554 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f50740aa9f0 con 0x7f508415dd30 2024-04-25T10:05:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.341+0000 7f5088e74640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f506406b4d0 0x7f506406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.342+0000 7f5088e74640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f506406b4d0 0x7f506406d990 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f507c009940 tx=0x7f507c003cd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.345+0000 7f50817fa640 1 -- 172.21.15.19:0/1384024554 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5074079300 con 0x7f508415dd30 2024-04-25T10:05:06.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.608+0000 7f508b0d8640 1 -- 172.21.15.19:0/1384024554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7f50840ce940 con 0x7f508415dd30 2024-04-25T10:05:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.896+0000 7f50817fa640 1 -- 172.21.15.19:0/1384024554 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v587) v1 ==== 133+0+0 (secure 0 0 0) 0x7f50840ce940 con 0x7f508415dd30 2024-04-25T10:05:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.898+0000 7f508b0d8640 1 -- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f506406b4d0 msgr2=0x7f506406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.898+0000 7f508b0d8640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f506406b4d0 0x7f506406d990 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f507c009940 tx=0x7f507c003cd0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.898+0000 7f508b0d8640 1 -- 172.21.15.19:0/1384024554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f508415dd30 msgr2=0x7f508414d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.898+0000 7f508b0d8640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f508415dd30 0x7f508414d850 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f50740090b0 tx=0x7f50740046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.898+0000 7f508b0d8640 1 -- 172.21.15.19:0/1384024554 shutdown_connections 2024-04-25T10:05:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.898+0000 7f508b0d8640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f506406b4d0 0x7f506406d990 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.898+0000 7f508b0d8640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f508415dd30 0x7f508414d850 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.898+0000 7f508b0d8640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50840d2e90 0x7f50840b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.898+0000 7f508b0d8640 1 --2- 172.21.15.19:0/1384024554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50840b9770 0x7f50841f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.898+0000 7f508b0d8640 1 -- 172.21.15.19:0/1384024554 >> 172.21.15.19:0/1384024554 conn(0x7f5084061c30 msgr2=0x7f50840ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:06.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.899+0000 7f508b0d8640 1 -- 172.21.15.19:0/1384024554 shutdown_connections 2024-04-25T10:05:06.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:06.899+0000 7f508b0d8640 1 -- 172.21.15.19:0/1384024554 wait complete. 2024-04-25T10:05:06.916 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:05:06.916 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:05:06.916 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:07.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.013+0000 7f7550b63640 1 Processor -- start 2024-04-25T10:05:07.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.013+0000 7f7550b63640 1 -- start start 2024-04-25T10:05:07.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.013+0000 7f7550b63640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f754c0aecf0 0x7f754c0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.013+0000 7f7550b63640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f754c0b2650 0x7f754c0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.013+0000 7f7550b63640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c0cf330 con 0x7f754c165680 2024-04-25T10:05:07.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.013+0000 7f7550b63640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754c072340 con 0x7f754c0aecf0 2024-04-25T10:05:07.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.013+0000 7f7550b63640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754c071670 con 0x7f754c0b2650 2024-04-25T10:05:07.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.014+0000 7f7541d9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f754c0aecf0 0x7f754c0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.014+0000 7f754a59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f754c165680 0x7f754c0cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55498/0 (socket says 172.21.15.19:55498) 2024-04-25T10:05:07.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.014+0000 7f754a59c640 1 -- 172.21.15.19:0/2508280960 learned_addr learned my addr 172.21.15.19:0/2508280960 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:07.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.014+0000 7f754ad9d640 1 --2- 172.21.15.19:0/2508280960 >> v2:172.21.15.112:3300/0 conn(0x7f754c0b2650 0x7f754c0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.014+0000 7f7541d9b640 1 -- 172.21.15.19:0/2508280960 >> v2:172.21.15.112:3300/0 conn(0x7f754c0b2650 msgr2=0x7f754c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.014+0000 7f7541d9b640 1 --2- 172.21.15.19:0/2508280960 >> v2:172.21.15.112:3300/0 conn(0x7f754c0b2650 0x7f754c0cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.014+0000 7f7541d9b640 1 -- 172.21.15.19:0/2508280960 >> v1:172.21.15.19:6789/0 conn(0x7f754c165680 legacy=0x7f754c0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.014+0000 7f7549d9b640 1 -- 172.21.15.19:0/2508280960 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013526069 0 0) 0x7f754c0cf330 con 0x7f754c165680 2024-04-25T10:05:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.014+0000 7f7541d9b640 1 -- 172.21.15.19:0/2508280960 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c071d80 con 0x7f754c0aecf0 2024-04-25T10:05:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.014+0000 7f754ad9d640 1 --2- 172.21.15.19:0/2508280960 >> v2:172.21.15.112:3300/0 conn(0x7f754c0b2650 0x7f754c0cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7541d9b640 1 --2- 172.21.15.19:0/2508280960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0aecf0 0x7f754c0cec70 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f754c0724c0 tx=0x7f753402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c6b8f53a1a3441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7549d9b640 1 -- 172.21.15.19:0/2508280960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7534037070 con 0x7f754c0aecf0 2024-04-25T10:05:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7549d9b640 1 -- 172.21.15.19:0/2508280960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7534032440 con 0x7f754c0aecf0 2024-04-25T10:05:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7549d9b640 1 -- 172.21.15.19:0/2508280960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753402ae60 con 0x7f754c0aecf0 2024-04-25T10:05:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7550b63640 1 -- 172.21.15.19:0/2508280960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0aecf0 msgr2=0x7f754c0cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7550b63640 1 --2- 172.21.15.19:0/2508280960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0aecf0 0x7f754c0cec70 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f754c0724c0 tx=0x7f753402a110 comp rx=0 tx=0).stop 2024-04-25T10:05:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7550b63640 1 -- 172.21.15.19:0/2508280960 shutdown_connections 2024-04-25T10:05:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7550b63640 1 --2- 172.21.15.19:0/2508280960 >> v2:172.21.15.112:3300/0 conn(0x7f754c0b2650 0x7f754c0cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7550b63640 1 --2- 172.21.15.19:0/2508280960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0aecf0 0x7f754c0cec70 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7550b63640 1 -- 172.21.15.19:0/2508280960 >> 172.21.15.19:0/2508280960 conn(0x7f754c06abe0 msgr2=0x7f754c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7550b63640 1 -- 172.21.15.19:0/2508280960 shutdown_connections 2024-04-25T10:05:07.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.015+0000 7f7550b63640 1 -- 172.21.15.19:0/2508280960 wait complete. 2024-04-25T10:05:07.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.017+0000 7f7550b63640 1 Processor -- start 2024-04-25T10:05:07.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.017+0000 7f7550b63640 1 -- start start 2024-04-25T10:05:07.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7550b63640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f754c0aecf0 0x7f754c155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7550b63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0b2650 0x7f754c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7550b63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f754c165680 0x7f754c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7550b63640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c165a60 con 0x7f754c0b2650 2024-04-25T10:05:07.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7550b63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c06ea30 con 0x7f754c165680 2024-04-25T10:05:07.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7550b63640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0b2a60 con 0x7f754c0aecf0 2024-04-25T10:05:07.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7541d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0b2650 0x7f754c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7541d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0b2650 0x7f754c15aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53498/0 (socket says 172.21.15.19:53498) 2024-04-25T10:05:07.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7541d9b640 1 -- 172.21.15.19:0/3537399483 learned_addr learned my addr 172.21.15.19:0/3537399483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f754a59c640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f754c0aecf0 0x7f754c155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f754ad9d640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f754c165680 0x7f754c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7541d9b640 1 -- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f754c0aecf0 msgr2=0x7f754c155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7541d9b640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f754c0aecf0 0x7f754c155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7541d9b640 1 -- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f754c165680 msgr2=0x7f754c1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7541d9b640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f754c165680 0x7f754c1529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7541d9b640 1 -- 172.21.15.19:0/3537399483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c154ac0 con 0x7f754c0b2650 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.018+0000 7f7541d9b640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0b2650 0x7f754c15aad0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f7534000c00 tx=0x7f75340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.019+0000 7f75437fe640 1 -- 172.21.15.19:0/3537399483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7534037040 con 0x7f754c0b2650 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.019+0000 7f75437fe640 1 -- 172.21.15.19:0/3537399483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7534042070 con 0x7f754c0b2650 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.019+0000 7f7550b63640 1 -- 172.21.15.19:0/3537399483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c154da0 con 0x7f754c0b2650 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.019+0000 7f75437fe640 1 -- 172.21.15.19:0/3537399483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7534030a50 con 0x7f754c0b2650 2024-04-25T10:05:07.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.019+0000 7f7550b63640 1 -- 172.21.15.19:0/3537399483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c155360 con 0x7f754c0b2650 2024-04-25T10:05:07.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.020+0000 7f7550b63640 1 -- 172.21.15.19:0/3537399483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c073990 con 0x7f754c0b2650 2024-04-25T10:05:07.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.020+0000 7f75437fe640 1 -- 172.21.15.19:0/3537399483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7534030bf0 con 0x7f754c0b2650 2024-04-25T10:05:07.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.021+0000 7f75437fe640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f752c06b4d0 0x7f752c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.021+0000 7f75437fe640 1 -- 172.21.15.19:0/3537399483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f75340aa990 con 0x7f754c0b2650 2024-04-25T10:05:07.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.021+0000 7f754a59c640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f752c06b4d0 0x7f752c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.021+0000 7f754a59c640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f752c06b4d0 0x7f752c06d990 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f753c007fd0 tx=0x7f753c007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.025+0000 7f75437fe640 1 -- 172.21.15.19:0/3537399483 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7534079450 con 0x7f754c0b2650 2024-04-25T10:05:07.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.206+0000 7f7550b63640 1 -- 172.21.15.19:0/3537399483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f754c0cc990 con 0x7f754c0b2650 2024-04-25T10:05:07.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.207+0000 7f75437fe640 1 -- 172.21.15.19:0/3537399483 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+13291 (secure 0 0 0) 0x7f753407d4f0 con 0x7f754c0b2650 2024-04-25T10:05:07.213 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:07.213 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":170,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:05.889131+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/3942829201":"2024-04-26T10:05:02.880632+0000","172.21.15.19:6832/3942829201":"2024-04-26T10:05:02.880632+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 -- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f752c06b4d0 msgr2=0x7f752c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f752c06b4d0 0x7f752c06d990 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f753c007fd0 tx=0x7f753c007de0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 -- 172.21.15.19:0/3537399483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0b2650 msgr2=0x7f754c15aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0b2650 0x7f754c15aad0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f7534000c00 tx=0x7f75340046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 -- 172.21.15.19:0/3537399483 shutdown_connections 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f752c06b4d0 0x7f752c06d990 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f754c165680 0x7f754c1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0b2650 0x7f754c15aad0 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 --2- 172.21.15.19:0/3537399483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f754c0aecf0 0x7f754c155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 -- 172.21.15.19:0/3537399483 >> 172.21.15.19:0/3537399483 conn(0x7f754c06abe0 msgr2=0x7f754c06d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 -- 172.21.15.19:0/3537399483 shutdown_connections 2024-04-25T10:05:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.209+0000 7f7550b63640 1 -- 172.21.15.19:0/3537399483 wait complete. 2024-04-25T10:05:07.226 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:07.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:07.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.325+0000 7fc0e1fdb640 1 Processor -- start 2024-04-25T10:05:07.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.325+0000 7fc0e1fdb640 1 -- start start 2024-04-25T10:05:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.326+0000 7fc0e1fdb640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0dc0b5620 0x7fc0dc0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.326+0000 7fc0e1fdb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0dc06c2d0 0x7fc0dc06c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.326+0000 7fc0e1fdb640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0dc06cc80 con 0x7fc0dc15b7d0 2024-04-25T10:05:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.326+0000 7fc0e1fdb640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0dc06cf50 con 0x7fc0dc0b5620 2024-04-25T10:05:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.326+0000 7fc0e1fdb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc0ce530 con 0x7fc0dc06c2d0 2024-04-25T10:05:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.326+0000 7fc0daffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0dc0b5620 0x7fc0dc0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.326+0000 7fc0db7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc0dc15b7d0 0x7fc0dc0cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55512/0 (socket says 172.21.15.19:55512) 2024-04-25T10:05:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.326+0000 7fc0db7fe640 1 -- 172.21.15.19:0/908114511 learned_addr learned my addr 172.21.15.19:0/908114511 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.326+0000 7fc0dbfff640 1 --2- 172.21.15.19:0/908114511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0dc06c2d0 0x7fc0dc06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.327+0000 7fc0daffd640 1 -- 172.21.15.19:0/908114511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0dc06c2d0 msgr2=0x7fc0dc06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.327+0000 7fc0daffd640 1 --2- 172.21.15.19:0/908114511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0dc06c2d0 0x7fc0dc06c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.327+0000 7fc0daffd640 1 -- 172.21.15.19:0/908114511 >> v1:172.21.15.19:6789/0 conn(0x7fc0dc15b7d0 legacy=0x7fc0dc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.327+0000 7fc0daffd640 1 -- 172.21.15.19:0/908114511 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc0ce810 con 0x7fc0dc0b5620 2024-04-25T10:05:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.327+0000 7fc0daffd640 1 --2- 172.21.15.19:0/908114511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0dc0b5620 0x7fc0dc0b5ab0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00097a0 tx=0x7fc0d002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d646b178211f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.327+0000 7fc0da7fc640 1 -- 172.21.15.19:0/908114511 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0037070 con 0x7fc0dc0b5620 2024-04-25T10:05:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.327+0000 7fc0da7fc640 1 -- 172.21.15.19:0/908114511 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d0032440 con 0x7fc0dc0b5620 2024-04-25T10:05:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.327+0000 7fc0da7fc640 1 -- 172.21.15.19:0/908114511 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0032810 con 0x7fc0dc0b5620 2024-04-25T10:05:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.328+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/908114511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0dc0b5620 msgr2=0x7fc0dc0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.328+0000 7fc0e1fdb640 1 --2- 172.21.15.19:0/908114511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0dc0b5620 0x7fc0dc0b5ab0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00097a0 tx=0x7fc0d002a180 comp rx=0 tx=0).stop 2024-04-25T10:05:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.328+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/908114511 shutdown_connections 2024-04-25T10:05:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.328+0000 7fc0e1fdb640 1 --2- 172.21.15.19:0/908114511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0dc06c2d0 0x7fc0dc06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.328+0000 7fc0e1fdb640 1 --2- 172.21.15.19:0/908114511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0dc0b5620 0x7fc0dc0b5ab0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.328+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/908114511 >> 172.21.15.19:0/908114511 conn(0x7fc0dc0d0800 msgr2=0x7fc0dc0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.328+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/908114511 shutdown_connections 2024-04-25T10:05:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.328+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/908114511 wait complete. 2024-04-25T10:05:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.329+0000 7fc0e1fdb640 1 Processor -- start 2024-04-25T10:05:07.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.329+0000 7fc0e1fdb640 1 -- start start 2024-04-25T10:05:07.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0e1fdb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0dc06c2d0 0x7fc0dc14cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0e1fdb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0dc0b5620 0x7fc0dc0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0e1fdb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0dc15b7d0 0x7fc0dc0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0e1fdb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc15bbb0 con 0x7fc0dc0b5620 2024-04-25T10:05:07.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0e1fdb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc06d230 con 0x7fc0dc15b7d0 2024-04-25T10:05:07.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0e1fdb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc15bd30 con 0x7fc0dc06c2d0 2024-04-25T10:05:07.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0daffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0dc0b5620 0x7fc0dc0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0daffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0dc0b5620 0x7fc0dc0cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53518/0 (socket says 172.21.15.19:53518) 2024-04-25T10:05:07.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0daffd640 1 -- 172.21.15.19:0/2497371883 learned_addr learned my addr 172.21.15.19:0/2497371883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:07.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0dbfff640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0dc15b7d0 0x7fc0dc0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0db7fe640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0dc06c2d0 0x7fc0dc14cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0daffd640 1 -- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0dc06c2d0 msgr2=0x7fc0dc14cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0daffd640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0dc06c2d0 0x7fc0dc14cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0daffd640 1 -- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0dc15b7d0 msgr2=0x7fc0dc0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0daffd640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0dc15b7d0 0x7fc0dc0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.330+0000 7fc0daffd640 1 -- 172.21.15.19:0/2497371883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc14d6d0 con 0x7fc0dc0b5620 2024-04-25T10:05:07.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.331+0000 7fc0db7fe640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0dc06c2d0 0x7fc0dc14cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:07.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.331+0000 7fc0daffd640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0dc0b5620 0x7fc0dc0cac70 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0035040 tx=0x7fc0d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.331+0000 7fc0c3fff640 1 -- 172.21.15.19:0/2497371883 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0037040 con 0x7fc0dc0b5620 2024-04-25T10:05:07.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.331+0000 7fc0c3fff640 1 -- 172.21.15.19:0/2497371883 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d0030770 con 0x7fc0dc0b5620 2024-04-25T10:05:07.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.331+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/2497371883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0dc14d9b0 con 0x7fc0dc0b5620 2024-04-25T10:05:07.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.331+0000 7fc0c3fff640 1 -- 172.21.15.19:0/2497371883 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0030b80 con 0x7fc0dc0b5620 2024-04-25T10:05:07.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.331+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/2497371883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc14df70 con 0x7fc0dc0b5620 2024-04-25T10:05:07.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.332+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/2497371883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0dc14c850 con 0x7fc0dc0b5620 2024-04-25T10:05:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.332+0000 7fc0c3fff640 1 -- 172.21.15.19:0/2497371883 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0d0044020 con 0x7fc0dc0b5620 2024-04-25T10:05:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.333+0000 7fc0c3fff640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc0c806b4d0 0x7fc0c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.333+0000 7fc0c3fff640 1 -- 172.21.15.19:0/2497371883 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc0d00aa440 con 0x7fc0dc0b5620 2024-04-25T10:05:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.333+0000 7fc0db7fe640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc0c806b4d0 0x7fc0c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.334+0000 7fc0db7fe640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc0c806b4d0 0x7fc0c806d990 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc000f60 tx=0x7fc0cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.337+0000 7fc0c3fff640 1 -- 172.21.15.19:0/2497371883 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0dc14c850 con 0x7fc0dc0b5620 2024-04-25T10:05:07.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.516+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/2497371883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0dc06b210 con 0x7fc0dc0b5620 2024-04-25T10:05:07.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.516+0000 7fc0c3fff640 1 -- 172.21.15.19:0/2497371883 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0d0032e20 con 0x7fc0dc0b5620 2024-04-25T10:05:07.522 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:05:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc0c806b4d0 msgr2=0x7fc0c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc0c806b4d0 0x7fc0c806d990 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc000f60 tx=0x7fc0cc009040 comp rx=0 tx=0).stop 2024-04-25T10:05:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/2497371883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0dc0b5620 msgr2=0x7fc0dc0cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0dc0b5620 0x7fc0dc0cac70 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0035040 tx=0x7fc0d00046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/2497371883 shutdown_connections 2024-04-25T10:05:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc0c806b4d0 0x7fc0c806d990 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0dc15b7d0 0x7fc0dc0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0dc0b5620 0x7fc0dc0cac70 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 --2- 172.21.15.19:0/2497371883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc0dc06c2d0 0x7fc0dc14cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/2497371883 >> 172.21.15.19:0/2497371883 conn(0x7fc0dc0d0800 msgr2=0x7fc0dc15bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/2497371883 shutdown_connections 2024-04-25T10:05:07.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.518+0000 7fc0e1fdb640 1 -- 172.21.15.19:0/2497371883 wait complete. 2024-04-25T10:05:07.534 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:05:07.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.629+0000 7f012d8bd640 1 Processor -- start 2024-04-25T10:05:07.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.629+0000 7f012d8bd640 1 -- start start 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.630+0000 7f012d8bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01281602e0 0x7f01280ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.630+0000 7f012d8bd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01280cd710 con 0x7f012815ba00 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.630+0000 7f012d8bd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0128099b70 con 0x7f012815f5f0 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.630+0000 7f012d8bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0128099e50 con 0x7f01281602e0 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.630+0000 7f0126ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f012815ba00 0x7f012809a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55528/0 (socket says 172.21.15.19:55528) 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.630+0000 7f0126ffd640 1 -- 172.21.15.19:0/2257704649 learned_addr learned my addr 172.21.15.19:0/2257704649 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.630+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583178110 0 0) 0x7f0120001110 con 0x7f012815ba00 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0114003520 con 0x7f012815ba00 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f01267fc640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01281602e0 0x7f01280ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3395591643 0 0) 0x7f0114003520 con 0x7f012815ba00 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01281602e0 msgr2=0x7f01280ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01281602e0 0x7f01280ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 >> v1:172.21.15.112:6789/0 conn(0x7f012815f5f0 legacy=0x7f01280c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012809a130 con 0x7f012815ba00 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0120001840 con 0x7f012815ba00 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 >> v1:172.21.15.19:6789/0 conn(0x7f012815ba00 legacy=0x7f012809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01140039e0 0x7f0114003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 0x7f0114004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01281602e0 0x7f0114005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0128099e50 con 0x7f0114004800 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011400bb80 con 0x7f01140039e0 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011400bd80 con 0x7f01281602e0 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f01267fc640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 0x7f0114004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0126ffd640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01140039e0 0x7f0114003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f01277fe640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01281602e0 0x7f0114005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f01267fc640 1 -- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01281602e0 msgr2=0x7f0114005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f01267fc640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01281602e0 0x7f0114005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f01267fc640 1 -- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01140039e0 msgr2=0x7f0114003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f01267fc640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01140039e0 0x7f0114003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f01267fc640 1 -- 172.21.15.19:0/2257704649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0114003520 con 0x7f0114004800 2024-04-25T10:05:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f0126ffd640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01140039e0 0x7f0114003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.631+0000 7f01277fe640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01281602e0 0x7f0114005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.632+0000 7f01267fc640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 0x7f0114004c80 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f0110009ac0 tx=0x7f0114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35386f53c348e42c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.632+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110036070 con 0x7f0114004800 2024-04-25T10:05:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.632+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0110031400 con 0x7f0114004800 2024-04-25T10:05:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.632+0000 7f0125ffb640 1 -- 172.21.15.19:0/2257704649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110009690 con 0x7f0114004800 2024-04-25T10:05:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.633+0000 7f012d8bd640 1 -- 172.21.15.19:0/2257704649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 msgr2=0x7f0114004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.633+0000 7f012d8bd640 1 --2- 172.21.15.19:0/2257704649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 0x7f0114004c80 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f0110009ac0 tx=0x7f0114005830 comp rx=0 tx=0).stop 2024-04-25T10:05:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.633+0000 7f01277fe640 1 -- 172.21.15.19:0/2257704649 reap_dead start 2024-04-25T10:05:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.633+0000 7f012d8bd640 1 -- 172.21.15.19:0/2257704649 shutdown_connections 2024-04-25T10:05:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.633+0000 7f012d8bd640 1 -- 172.21.15.19:0/2257704649 >> 172.21.15.19:0/2257704649 conn(0x7f01280b3270 msgr2=0x7f01280b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.633+0000 7f012d8bd640 1 -- 172.21.15.19:0/2257704649 shutdown_connections 2024-04-25T10:05:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.633+0000 7f012d8bd640 1 -- 172.21.15.19:0/2257704649 wait complete. 2024-04-25T10:05:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.634+0000 7f012d8bd640 1 Processor -- start 2024-04-25T10:05:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.634+0000 7f012d8bd640 1 -- start start 2024-04-25T10:05:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f012d8bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 0x7f01281f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f012d8bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01281f4770 0x7f0128070750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f012d8bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0128070c90 0x7f01280c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f012d8bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01280c5b60 con 0x7f0114004800 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f012d8bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01280c59e0 con 0x7f01281f4770 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f012d8bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01280c5ce0 con 0x7f0128070c90 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f0126ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 0x7f01281f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f0126ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 0x7f01281f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53536/0 (socket says 172.21.15.19:53536) 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f0126ffd640 1 -- 172.21.15.19:0/895953496 learned_addr learned my addr 172.21.15.19:0/895953496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f01267fc640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01281f4770 0x7f0128070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f01277fe640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0128070c90 0x7f01280c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f0126ffd640 1 -- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0128070c90 msgr2=0x7f01280c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f0126ffd640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0128070c90 0x7f01280c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f0126ffd640 1 -- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01281f4770 msgr2=0x7f0128070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f0126ffd640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01281f4770 0x7f0128070750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f0126ffd640 1 -- 172.21.15.19:0/895953496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01280c9430 con 0x7f0114004800 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f0126ffd640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 0x7f01281f1960 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f0120007330 tx=0x7f01200077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.635+0000 7f011ffff640 1 -- 172.21.15.19:0/895953496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012000e8f0 con 0x7f0114004800 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.636+0000 7f011ffff640 1 -- 172.21.15.19:0/895953496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0120018070 con 0x7f0114004800 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.636+0000 7f012d8bd640 1 -- 172.21.15.19:0/895953496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01280c9710 con 0x7f0114004800 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.636+0000 7f011ffff640 1 -- 172.21.15.19:0/895953496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012000ec90 con 0x7f0114004800 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.636+0000 7f012d8bd640 1 -- 172.21.15.19:0/895953496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01280c9cd0 con 0x7f0114004800 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.637+0000 7f012d8bd640 1 -- 172.21.15.19:0/895953496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0128073990 con 0x7f0114004800 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.637+0000 7f011ffff640 1 -- 172.21.15.19:0/895953496 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f012001d460 con 0x7f0114004800 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.637+0000 7f011ffff640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f010806b4d0 0x7f010806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.637+0000 7f011ffff640 1 -- 172.21.15.19:0/895953496 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f012008a040 con 0x7f0114004800 2024-04-25T10:05:07.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.637+0000 7f01267fc640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f010806b4d0 0x7f010806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.638+0000 7f01267fc640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f010806b4d0 0x7f010806d990 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0110033000 tx=0x7f0110008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.642+0000 7f011ffff640 1 -- 172.21.15.19:0/895953496 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01280cd710 con 0x7f0114004800 2024-04-25T10:05:07.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.890+0000 7f012d8bd640 1 -- 172.21.15.19:0/895953496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f0128071070 con 0x7f0114004800 2024-04-25T10:05:07.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.901+0000 7f011ffff640 1 -- 172.21.15.19:0/895953496 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v588) v1 ==== 150+0+0 (secure 0 0 0) 0x7f0120059030 con 0x7f0114004800 2024-04-25T10:05:07.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 -- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f010806b4d0 msgr2=0x7f010806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f010806b4d0 0x7f010806d990 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0110033000 tx=0x7f0110008030 comp rx=0 tx=0).stop 2024-04-25T10:05:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 -- 172.21.15.19:0/895953496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 msgr2=0x7f01281f1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 0x7f01281f1960 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f0120007330 tx=0x7f01200077f0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 -- 172.21.15.19:0/895953496 shutdown_connections 2024-04-25T10:05:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f010806b4d0 0x7f010806d990 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0128070c90 0x7f01280c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01281f4770 0x7f0128070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 --2- 172.21.15.19:0/895953496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0114004800 0x7f01281f1960 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 -- 172.21.15.19:0/895953496 >> 172.21.15.19:0/895953496 conn(0x7f01280b3270 msgr2=0x7f01280cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 -- 172.21.15.19:0/895953496 shutdown_connections 2024-04-25T10:05:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:07.903+0000 7f012d8bd640 1 -- 172.21.15.19:0/895953496 wait complete. 2024-04-25T10:05:07.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:05:08.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.019+0000 7fa821d5c640 1 Processor -- start 2024-04-25T10:05:08.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.019+0000 7fa821d5c640 1 -- start start 2024-04-25T10:05:08.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.019+0000 7fa821d5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa81c15bb20 0x7fa81c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.019+0000 7fa821d5c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa81c0ccdd0 0x7fa81c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.019+0000 7fa821d5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa81c0cdcd0 0x7fa81c0cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.019+0000 7fa821d5c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c0d0450 con 0x7fa81c0ccdd0 2024-04-25T10:05:08.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.019+0000 7fa821d5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c0cb190 con 0x7fa81c15bb20 2024-04-25T10:05:08.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.019+0000 7fa821d5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0cb470 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.020+0000 7fa81affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa81c0ccdd0 0x7fa81c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.020+0000 7fa81affd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa81c0ccdd0 0x7fa81c0cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37916/0 (socket says 172.21.15.19:37916) 2024-04-25T10:05:08.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.020+0000 7fa81affd640 1 -- 172.21.15.19:0/1383164084 learned_addr learned my addr 172.21.15.19:0/1383164084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.020+0000 7fa81b7fe640 1 --2- 172.21.15.19:0/1383164084 >> v2:172.21.15.112:3300/0 conn(0x7fa81c15bb20 0x7fa81c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.020+0000 7fa81bfff640 1 --2- 172.21.15.19:0/1383164084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa81c0cdcd0 0x7fa81c0cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.020+0000 7fa81affd640 1 -- 172.21.15.19:0/1383164084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa81c0cdcd0 msgr2=0x7fa81c0cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.020+0000 7fa81affd640 1 --2- 172.21.15.19:0/1383164084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa81c0cdcd0 0x7fa81c0cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.020+0000 7fa81affd640 1 -- 172.21.15.19:0/1383164084 >> v2:172.21.15.112:3300/0 conn(0x7fa81c15bb20 msgr2=0x7fa81c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.020+0000 7fa81affd640 1 --2- 172.21.15.19:0/1383164084 >> v2:172.21.15.112:3300/0 conn(0x7fa81c15bb20 0x7fa81c0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.020+0000 7fa81affd640 1 -- 172.21.15.19:0/1383164084 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0cb750 con 0x7fa81c0ccdd0 2024-04-25T10:05:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.021+0000 7fa81affd640 1 --2- 172.21.15.19:0/1383164084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c0ccdd0 0x7fa81c0cf9d0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0d1800 tx=0x7fa81002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985f127731f0e4e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.021+0000 7fa81a7fc640 1 -- 172.21.15.19:0/1383164084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810037070 con 0x7fa81c0ccdd0 2024-04-25T10:05:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.021+0000 7fa81a7fc640 1 -- 172.21.15.19:0/1383164084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa810032440 con 0x7fa81c0ccdd0 2024-04-25T10:05:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.021+0000 7fa81a7fc640 1 -- 172.21.15.19:0/1383164084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810032810 con 0x7fa81c0ccdd0 2024-04-25T10:05:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.021+0000 7fa821d5c640 1 -- 172.21.15.19:0/1383164084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c0ccdd0 msgr2=0x7fa81c0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.021+0000 7fa821d5c640 1 --2- 172.21.15.19:0/1383164084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c0ccdd0 0x7fa81c0cf9d0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0d1800 tx=0x7fa81002a140 comp rx=0 tx=0).stop 2024-04-25T10:05:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.021+0000 7fa821d5c640 1 -- 172.21.15.19:0/1383164084 shutdown_connections 2024-04-25T10:05:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.021+0000 7fa821d5c640 1 --2- 172.21.15.19:0/1383164084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa81c0cdcd0 0x7fa81c0cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.021+0000 7fa821d5c640 1 --2- 172.21.15.19:0/1383164084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c0ccdd0 0x7fa81c0cf9d0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.021+0000 7fa821d5c640 1 --2- 172.21.15.19:0/1383164084 >> v2:172.21.15.112:3300/0 conn(0x7fa81c15bb20 0x7fa81c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.021+0000 7fa821d5c640 1 -- 172.21.15.19:0/1383164084 >> 172.21.15.19:0/1383164084 conn(0x7fa81c0b2390 msgr2=0x7fa81c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.022+0000 7fa821d5c640 1 -- 172.21.15.19:0/1383164084 shutdown_connections 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.022+0000 7fa821d5c640 1 -- 172.21.15.19:0/1383164084 wait complete. 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.023+0000 7fa821d5c640 1 Processor -- start 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.023+0000 7fa821d5c640 1 -- start start 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa821d5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa81c0ccdd0 0x7fa81c146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa821d5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c0cdcd0 0x7fa81c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa821d5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa81c15bb20 0x7fa81c154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa821d5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0cc770 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa821d5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0cbd10 con 0x7fa81c0ccdd0 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa821d5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c15bf00 con 0x7fa81c15bb20 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa81affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c0cdcd0 0x7fa81c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa81affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c0cdcd0 0x7fa81c154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37924/0 (socket says 172.21.15.19:37924) 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa81affd640 1 -- 172.21.15.19:0/1778561765 learned_addr learned my addr 172.21.15.19:0/1778561765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa81affd640 1 -- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa81c15bb20 msgr2=0x7fa81c154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa81affd640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa81c15bb20 0x7fa81c154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa81affd640 1 -- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa81c0ccdd0 msgr2=0x7fa81c146260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa81affd640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa81c0ccdd0 0x7fa81c146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa81affd640 1 -- 172.21.15.19:0/1778561765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0b4b30 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa81affd640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c0cdcd0 0x7fa81c154960 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7fa81002a180 tx=0x7fa8100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa803fff640 1 -- 172.21.15.19:0/1778561765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810037040 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa803fff640 1 -- 172.21.15.19:0/1778561765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa810042070 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa821d5c640 1 -- 172.21.15.19:0/1778561765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81c0b4e10 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa803fff640 1 -- 172.21.15.19:0/1778561765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81003d880 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.024+0000 7fa821d5c640 1 -- 172.21.15.19:0/1778561765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81c0b53d0 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.026+0000 7fa821d5c640 1 -- 172.21.15.19:0/1778561765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81c073a00 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.026+0000 7fa803fff640 1 -- 172.21.15.19:0/1778561765 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa81003da20 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.026+0000 7fa803fff640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa80806b4d0 0x7fa80806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.026+0000 7fa803fff640 1 -- 172.21.15.19:0/1778561765 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa8100aa8d0 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.027+0000 7fa81b7fe640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa80806b4d0 0x7fa80806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.027+0000 7fa81b7fe640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa80806b4d0 0x7fa80806d990 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa80c006a90 tx=0x7fa80c006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.031+0000 7fa803fff640 1 -- 172.21.15.19:0/1778561765 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8100798c0 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.212+0000 7fa821d5c640 1 -- 172.21.15.19:0/1778561765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa81c15a370 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.215+0000 7fa803fff640 1 -- 172.21.15.19:0/1778561765 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa8100791c0 con 0x7fa81c0cdcd0 2024-04-25T10:05:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.217+0000 7fa821d5c640 1 -- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa80806b4d0 msgr2=0x7fa80806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.217+0000 7fa821d5c640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa80806b4d0 0x7fa80806d990 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa80c006a90 tx=0x7fa80c006a20 comp rx=0 tx=0).stop 2024-04-25T10:05:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.217+0000 7fa821d5c640 1 -- 172.21.15.19:0/1778561765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c0cdcd0 msgr2=0x7fa81c154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.217+0000 7fa821d5c640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c0cdcd0 0x7fa81c154960 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7fa81002a180 tx=0x7fa8100046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.217+0000 7fa821d5c640 1 -- 172.21.15.19:0/1778561765 shutdown_connections 2024-04-25T10:05:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.217+0000 7fa821d5c640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa80806b4d0 0x7fa80806d990 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.217+0000 7fa821d5c640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa81c15bb20 0x7fa81c154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.217+0000 7fa821d5c640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c0cdcd0 0x7fa81c154960 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.217+0000 7fa821d5c640 1 --2- 172.21.15.19:0/1778561765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa81c0ccdd0 0x7fa81c146260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.217+0000 7fa821d5c640 1 -- 172.21.15.19:0/1778561765 >> 172.21.15.19:0/1778561765 conn(0x7fa81c0b2390 msgr2=0x7fa81c0aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.217+0000 7fa821d5c640 1 -- 172.21.15.19:0/1778561765 shutdown_connections 2024-04-25T10:05:08.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.218+0000 7fa821d5c640 1 -- 172.21.15.19:0/1778561765 wait complete. 2024-04-25T10:05:08.234 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:08.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:08.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:08.330 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:08.330 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:08.330 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:08.346 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:08.346 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:05:08.346 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:08.388 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:08.388 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:08.388 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:08.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:08.404 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:05:08.404 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:08.445 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:08.445 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:05:08.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:08.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.540+0000 7f17e839e640 1 Processor -- start 2024-04-25T10:05:08.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.540+0000 7f17e839e640 1 -- start start 2024-04-25T10:05:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e839e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17e015dc00 0x7f17e00c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e839e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f17e00ceb20 0x7f17e00d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e839e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e015dfe0 con 0x7f17e00d1c00 2024-04-25T10:05:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e839e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e00d2090 con 0x7f17e015dc00 2024-04-25T10:05:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e839e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e00d00c0 con 0x7f17e00ceb20 2024-04-25T10:05:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e613a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17e015dc00 0x7f17e00c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e613a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17e015dc00 0x7f17e00c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37930/0 (socket says 172.21.15.19:37930) 2024-04-25T10:05:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e613a640 1 -- 172.21.15.19:0/1103104825 learned_addr learned my addr 172.21.15.19:0/1103104825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e5939640 1 --2- 172.21.15.19:0/1103104825 >> v2:172.21.15.112:3300/0 conn(0x7f17e00ceb20 0x7f17e00d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e613a640 1 -- 172.21.15.19:0/1103104825 >> v2:172.21.15.112:3300/0 conn(0x7f17e00ceb20 msgr2=0x7f17e00d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e613a640 1 --2- 172.21.15.19:0/1103104825 >> v2:172.21.15.112:3300/0 conn(0x7f17e00ceb20 0x7f17e00d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e613a640 1 -- 172.21.15.19:0/1103104825 >> v1:172.21.15.112:6789/0 conn(0x7f17e00d1c00 legacy=0x7f17e00ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e613a640 1 -- 172.21.15.19:0/1103104825 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e00d03a0 con 0x7f17e015dc00 2024-04-25T10:05:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.541+0000 7f17e5939640 1 --2- 172.21.15.19:0/1103104825 >> v2:172.21.15.112:3300/0 conn(0x7f17e00ceb20 0x7f17e00d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.542+0000 7f17e613a640 1 --2- 172.21.15.19:0/1103104825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17e015dc00 0x7f17e00c6df0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f17e00c8820 tx=0x7f17dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4482e87acb0237e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.542+0000 7f17e5138640 1 -- 172.21.15.19:0/1103104825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc037070 con 0x7f17e015dc00 2024-04-25T10:05:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.542+0000 7f17e5138640 1 -- 172.21.15.19:0/1103104825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc032440 con 0x7f17e015dc00 2024-04-25T10:05:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.542+0000 7f17e5138640 1 -- 172.21.15.19:0/1103104825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc032810 con 0x7f17e015dc00 2024-04-25T10:05:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.543+0000 7f17e839e640 1 -- 172.21.15.19:0/1103104825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17e015dc00 msgr2=0x7f17e00c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.543+0000 7f17e839e640 1 --2- 172.21.15.19:0/1103104825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17e015dc00 0x7f17e00c6df0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f17e00c8820 tx=0x7f17dc02a230 comp rx=0 tx=0).stop 2024-04-25T10:05:08.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.543+0000 7f17e839e640 1 -- 172.21.15.19:0/1103104825 shutdown_connections 2024-04-25T10:05:08.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.543+0000 7f17e839e640 1 --2- 172.21.15.19:0/1103104825 >> v2:172.21.15.112:3300/0 conn(0x7f17e00ceb20 0x7f17e00d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.543+0000 7f17e839e640 1 --2- 172.21.15.19:0/1103104825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17e015dc00 0x7f17e00c6df0 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.543+0000 7f17e839e640 1 -- 172.21.15.19:0/1103104825 >> 172.21.15.19:0/1103104825 conn(0x7f17e006f5d0 msgr2=0x7f17e006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:08.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.543+0000 7f17e839e640 1 -- 172.21.15.19:0/1103104825 shutdown_connections 2024-04-25T10:05:08.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.543+0000 7f17e839e640 1 -- 172.21.15.19:0/1103104825 wait complete. 2024-04-25T10:05:08.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.544+0000 7f17e839e640 1 Processor -- start 2024-04-25T10:05:08.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.544+0000 7f17e839e640 1 -- start start 2024-04-25T10:05:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e839e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e00ceb20 0x7f17e0158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e839e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17e00d1c00 0x7f17e01582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e839e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17e015dc00 0x7f17e015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e839e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e00cc130 con 0x7f17e015dc00 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e839e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e00cc720 con 0x7f17e00ceb20 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e839e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e00cfd90 con 0x7f17e00d1c00 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e693b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17e015dc00 0x7f17e015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e693b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17e015dc00 0x7f17e015e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37940/0 (socket says 172.21.15.19:37940) 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e693b640 1 -- 172.21.15.19:0/4194927678 learned_addr learned my addr 172.21.15.19:0/4194927678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e613a640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e00ceb20 0x7f17e0158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e5939640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17e00d1c00 0x7f17e01582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e693b640 1 -- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17e00d1c00 msgr2=0x7f17e01582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e693b640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17e00d1c00 0x7f17e01582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e693b640 1 -- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e00ceb20 msgr2=0x7f17e0158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e693b640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e00ceb20 0x7f17e0158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e693b640 1 -- 172.21.15.19:0/4194927678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e01587f0 con 0x7f17e015dc00 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e5939640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17e00d1c00 0x7f17e01582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e693b640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17e015dc00 0x7f17e015e460 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f17d800c7b0 tx=0x7f17d800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17d6ffd640 1 -- 172.21.15.19:0/4194927678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8004b60 con 0x7f17e015dc00 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17d6ffd640 1 -- 172.21.15.19:0/4194927678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d8004d00 con 0x7f17e015dc00 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17e839e640 1 -- 172.21.15.19:0/4194927678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e00ca540 con 0x7f17e015dc00 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.545+0000 7f17d6ffd640 1 -- 172.21.15.19:0/4194927678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d800aa90 con 0x7f17e015dc00 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.546+0000 7f17e839e640 1 -- 172.21.15.19:0/4194927678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e00cab00 con 0x7f17e015dc00 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.547+0000 7f17e839e640 1 -- 172.21.15.19:0/4194927678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e0073990 con 0x7f17e015dc00 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.547+0000 7f17d6ffd640 1 -- 172.21.15.19:0/4194927678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17d800ac30 con 0x7f17e015dc00 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.547+0000 7f17d6ffd640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17c406b4d0 0x7f17c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.548+0000 7f17d6ffd640 1 -- 172.21.15.19:0/4194927678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f17d808a250 con 0x7f17e015dc00 2024-04-25T10:05:08.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.548+0000 7f17e613a640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17c406b4d0 0x7f17c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.548+0000 7f17e613a640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17c406b4d0 0x7f17c406d990 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f17dc034000 tx=0x7f17dc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.552+0000 7f17d6ffd640 1 -- 172.21.15.19:0/4194927678 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17d8020070 con 0x7f17e015dc00 2024-04-25T10:05:08.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.770+0000 7f17e839e640 1 -- 172.21.15.19:0/4194927678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17e00cf470 con 0x7f17e015dc00 2024-04-25T10:05:08.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.771+0000 7f17d6ffd640 1 -- 172.21.15.19:0/4194927678 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+3708 (secure 0 0 0) 0x7f17d8059240 con 0x7f17e015dc00 2024-04-25T10:05:08.776 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:08.776 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":111,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1300145056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1300145056},{"type":"v1","addr":"172.21.15.19:6835","nonce":1300145056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":8027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2001687418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2001687418},{"type":"v1","addr":"172.21.15.112:6836","nonce":2001687418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721457153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721457153},{"type":"v1","addr":"172.21.15.112:6837","nonce":721457153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":11079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1724832908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1724832908},{"type":"v1","addr":"172.21.15.19:6834","nonce":1724832908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":111}],"filesystems":[]} 2024-04-25T10:05:08.776 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 111 2024-04-25T10:05:08.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 -- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17c406b4d0 msgr2=0x7f17c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17c406b4d0 0x7f17c406d990 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f17dc034000 tx=0x7f17dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:08.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 -- 172.21.15.19:0/4194927678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17e015dc00 msgr2=0x7f17e015e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17e015dc00 0x7f17e015e460 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f17d800c7b0 tx=0x7f17d800cc70 comp rx=0 tx=0).stop 2024-04-25T10:05:08.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 -- 172.21.15.19:0/4194927678 shutdown_connections 2024-04-25T10:05:08.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17c406b4d0 0x7f17c406d990 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17e015dc00 0x7f17e015e460 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17e00d1c00 0x7f17e01582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 --2- 172.21.15.19:0/4194927678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e00ceb20 0x7f17e0158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 -- 172.21.15.19:0/4194927678 >> 172.21.15.19:0/4194927678 conn(0x7f17e006f5d0 msgr2=0x7f17e015bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:08.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 -- 172.21.15.19:0/4194927678 shutdown_connections 2024-04-25T10:05:08.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.773+0000 7f17e839e640 1 -- 172.21.15.19:0/4194927678 wait complete. 2024-04-25T10:05:08.789 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:05:08.790 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:05:08.790 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:05:08.790 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:08.790 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:05:08.790 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:05:08.790 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:05:08.790 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:08.791 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:05:08.791 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:05:08.791 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:05:08.791 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:08.791 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:05:08.791 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:05:08.791 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:05:08.792 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:08.799 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:05:08.799 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:05:08.800 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:05:08.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:05:08.802 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:05:08.802 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:05:08.803 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:05:08.803 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:05:08.842 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:05:08.843 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:08.844 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:05:08.844 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:08.845 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:05:08.845 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:08.845 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:05:08.846 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:08.846 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:05:08.918 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:05:08.919 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T10:05:08.923 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T10:05:08.923 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T10:05:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.948+0000 7fccc8d7e640 1 Processor -- start 2024-04-25T10:05:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.948+0000 7fccc8d7e640 1 -- start start 2024-04-25T10:05:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.949+0000 7fccc8d7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40ced80 0x7fccc40d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.949+0000 7fccc8d7e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc415e000 con 0x7fccc4161710 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.949+0000 7fccc8d7e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc40d1de0 con 0x7fccc415dc20 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.949+0000 7fccc8d7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4161c60 con 0x7fccc40ced80 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.949+0000 7fccc2d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fccc4161710 0x7fccc40d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37374/0 (socket says 172.21.15.19:37374) 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.949+0000 7fccc2d9d640 1 -- 172.21.15.19:0/2087207999 learned_addr learned my addr 172.21.15.19:0/2087207999 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672190871 0 0) 0x7fccc415e000 con 0x7fccc4161710 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccac003520 con 0x7fccc4161710 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc1d9b640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40ced80 0x7fccc40d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3332074704 0 0) 0x7fccac003520 con 0x7fccc4161710 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40ced80 msgr2=0x7fccc40d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40ced80 0x7fccc40d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 >> v1:172.21.15.112:6789/0 conn(0x7fccc415dc20 legacy=0x7fccc40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4161f40 con 0x7fccc4161710 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fccbc0019f0 con 0x7fccc4161710 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 >> v1:172.21.15.19:6789/0 conn(0x7fccc4161710 legacy=0x7fccc40d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccac0039e0 0x7fccac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40ced80 0x7fccac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccac004d40 0x7fccac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4161c60 con 0x7fccac0039e0 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac00b9c0 con 0x7fccac004d40 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac00bbc0 con 0x7fccc40ced80 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc1d9b640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40ced80 0x7fccac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc2d9d640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccac004d40 0x7fccac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.950+0000 7fccc259c640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccac0039e0 0x7fccac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.951+0000 7fccc1d9b640 1 -- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccac004d40 msgr2=0x7fccac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.951+0000 7fccc1d9b640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccac004d40 0x7fccac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.951+0000 7fccc1d9b640 1 -- 172.21.15.19:0/2087207999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccac0039e0 msgr2=0x7fccac003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.951+0000 7fccc1d9b640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccac0039e0 0x7fccac003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.951+0000 7fccc1d9b640 1 -- 172.21.15.19:0/2087207999 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac003520 con 0x7fccc40ced80 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.951+0000 7fccc2d9d640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccac004d40 0x7fccac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.951+0000 7fccc259c640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccac0039e0 0x7fccac003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.951+0000 7fccc1d9b640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40ced80 0x7fccac004800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009ac0 tx=0x7fccac005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55302f63482efb3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.952+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8036070 con 0x7fccc40ced80 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.952+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb8031400 con 0x7fccc40ced80 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.952+0000 7fccc159a640 1 -- 172.21.15.19:0/2087207999 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8009690 con 0x7fccc40ced80 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.952+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40ced80 msgr2=0x7fccac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.952+0000 7fccc8d7e640 1 --2- 172.21.15.19:0/2087207999 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40ced80 0x7fccac004800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009ac0 tx=0x7fccac005830 comp rx=0 tx=0).stop 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.952+0000 7fccc2d9d640 1 -- 172.21.15.19:0/2087207999 reap_dead start 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.952+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2087207999 shutdown_connections 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.952+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2087207999 >> 172.21.15.19:0/2087207999 conn(0x7fccc40d0420 msgr2=0x7fccc40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:08.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.952+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2087207999 shutdown_connections 2024-04-25T10:05:08.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.952+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2087207999 wait complete. 2024-04-25T10:05:08.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.954+0000 7fccc8d7e640 1 Processor -- start 2024-04-25T10:05:08.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.954+0000 7fccc8d7e640 1 -- start start 2024-04-25T10:05:08.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.954+0000 7fccc8d7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40ced80 0x7fccc415f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.954+0000 7fccc8d7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc414c540 0x7fccc41f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.954+0000 7fccc8d7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40b1ed0 0x7fccc40b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc8d7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40b5e50 con 0x7fccc40ced80 2024-04-25T10:05:08.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc8d7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4162500 con 0x7fccc414c540 2024-04-25T10:05:08.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc8d7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40b5fd0 con 0x7fccc40b1ed0 2024-04-25T10:05:08.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40ced80 0x7fccc415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40ced80 0x7fccc415f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38002/0 (socket says 172.21.15.19:38002) 2024-04-25T10:05:08.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc259c640 1 -- 172.21.15.19:0/2278984702 learned_addr learned my addr 172.21.15.19:0/2278984702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc1d9b640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc414c540 0x7fccc41f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc2d9d640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40b1ed0 0x7fccc40b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc259c640 1 -- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40b1ed0 msgr2=0x7fccc40b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc259c640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40b1ed0 0x7fccc40b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc259c640 1 -- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc414c540 msgr2=0x7fccc41f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc259c640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc414c540 0x7fccc41f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc259c640 1 -- 172.21.15.19:0/2278984702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc415f8a0 con 0x7fccc40ced80 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc2d9d640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40b1ed0 0x7fccc40b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc259c640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40ced80 0x7fccc415f200 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7fccb400f850 tx=0x7fccb400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccb2ffd640 1 -- 172.21.15.19:0/2278984702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb40052e0 con 0x7fccc40ced80 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccb2ffd640 1 -- 172.21.15.19:0/2278984702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb4018070 con 0x7fccc40ced80 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2278984702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc415fb80 con 0x7fccc40ced80 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.955+0000 7fccb2ffd640 1 -- 172.21.15.19:0/2278984702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb400d950 con 0x7fccc40ced80 2024-04-25T10:05:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.956+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2278984702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc4160140 con 0x7fccc40ced80 2024-04-25T10:05:08.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.956+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2278984702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc4073a00 con 0x7fccc40ced80 2024-04-25T10:05:08.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.957+0000 7fccb2ffd640 1 -- 172.21.15.19:0/2278984702 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccb400daf0 con 0x7fccc40ced80 2024-04-25T10:05:08.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.957+0000 7fccb2ffd640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca406b500 0x7fcca406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.957+0000 7fccb2ffd640 1 -- 172.21.15.19:0/2278984702 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fccb408a160 con 0x7fccc40ced80 2024-04-25T10:05:08.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.958+0000 7fccc1d9b640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca406b500 0x7fcca406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.958+0000 7fccc1d9b640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca406b500 0x7fcca406d9c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fccb8033000 tx=0x7fccb8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:08.962+0000 7fccb2ffd640 1 -- 172.21.15.19:0/2278984702 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fccb4059030 con 0x7fccc40ced80 2024-04-25T10:05:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.157+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2278984702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fccc40d16b0 con 0x7fccc40ced80 2024-04-25T10:05:09.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:09.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:09.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.914+0000 7fccb2ffd640 1 -- 172.21.15.19:0/2278984702 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v172) v1 ==== 97+0+0 (secure 0 0 0) 0x7fccb4058930 con 0x7fccc40ced80 2024-04-25T10:05:09.919 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T10:05:09.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.917+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca406b500 msgr2=0x7fcca406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.917+0000 7fccc8d7e640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca406b500 0x7fcca406d9c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fccb8033000 tx=0x7fccb8008030 comp rx=0 tx=0).stop 2024-04-25T10:05:09.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.917+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2278984702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40ced80 msgr2=0x7fccc415f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.917+0000 7fccc8d7e640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40ced80 0x7fccc415f200 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7fccb400f850 tx=0x7fccb400fd10 comp rx=0 tx=0).stop 2024-04-25T10:05:09.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.918+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2278984702 shutdown_connections 2024-04-25T10:05:09.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.918+0000 7fccc8d7e640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcca406b500 0x7fcca406d9c0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.918+0000 7fccc8d7e640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccc40b1ed0 0x7fccc40b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.918+0000 7fccc8d7e640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccc414c540 0x7fccc41f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.918+0000 7fccc8d7e640 1 --2- 172.21.15.19:0/2278984702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc40ced80 0x7fccc415f200 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.918+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2278984702 >> 172.21.15.19:0/2278984702 conn(0x7fccc40d0420 msgr2=0x7fccc40b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:09.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.918+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2278984702 shutdown_connections 2024-04-25T10:05:09.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:09.918+0000 7fccc8d7e640 1 -- 172.21.15.19:0/2278984702 wait complete. 2024-04-25T10:05:09.937 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:05:10.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.039+0000 7f1c836fc640 1 Processor -- start 2024-04-25T10:05:10.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.040+0000 7f1c836fc640 1 -- start start 2024-04-25T10:05:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.040+0000 7f1c836fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 0x7f1c7c0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.040+0000 7f1c836fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c7c09a450 0x7f1c7c0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.040+0000 7f1c836fc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7c06c220 con 0x7f1c7c06c700 2024-04-25T10:05:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.040+0000 7f1c836fc640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c09a8b0 con 0x7f1c7c09a450 2024-04-25T10:05:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.040+0000 7f1c836fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0d0c90 con 0x7f1c7c15dc70 2024-04-25T10:05:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.041+0000 7f1c80c97640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c7c09a450 0x7f1c7c0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.041+0000 7f1c80c97640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c7c09a450 0x7f1c7c0cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38016/0 (socket says 172.21.15.19:38016) 2024-04-25T10:05:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.041+0000 7f1c80c97640 1 -- 172.21.15.19:0/1719273943 learned_addr learned my addr 172.21.15.19:0/1719273943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.041+0000 7f1c81498640 1 --2- 172.21.15.19:0/1719273943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 0x7f1c7c0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.041+0000 7f1c80c97640 1 -- 172.21.15.19:0/1719273943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 msgr2=0x7f1c7c0d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:10.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.041+0000 7f1c80c97640 1 --2- 172.21.15.19:0/1719273943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 0x7f1c7c0d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.041+0000 7f1c80c97640 1 -- 172.21.15.19:0/1719273943 >> v1:172.21.15.112:6789/0 conn(0x7f1c7c06c700 legacy=0x7f1c7c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.041+0000 7f1c80c97640 1 -- 172.21.15.19:0/1719273943 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0d0080 con 0x7f1c7c09a450 2024-04-25T10:05:10.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.041+0000 7f1c81498640 1 --2- 172.21.15.19:0/1719273943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 0x7f1c7c0d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:10.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c80c97640 1 --2- 172.21.15.19:0/1719273943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c09a450 0x7f1c7c0cf9c0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009870 tx=0x7f1c7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a89eb80ff62dc3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c7bfff640 1 -- 172.21.15.19:0/1719273943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70037070 con 0x7f1c7c09a450 2024-04-25T10:05:10.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c7bfff640 1 -- 172.21.15.19:0/1719273943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c70032440 con 0x7f1c7c09a450 2024-04-25T10:05:10.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c7bfff640 1 -- 172.21.15.19:0/1719273943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70032810 con 0x7f1c7c09a450 2024-04-25T10:05:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c836fc640 1 -- 172.21.15.19:0/1719273943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c09a450 msgr2=0x7f1c7c0cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c836fc640 1 --2- 172.21.15.19:0/1719273943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c09a450 0x7f1c7c0cf9c0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009870 tx=0x7f1c7002a270 comp rx=0 tx=0).stop 2024-04-25T10:05:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c836fc640 1 -- 172.21.15.19:0/1719273943 shutdown_connections 2024-04-25T10:05:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c836fc640 1 --2- 172.21.15.19:0/1719273943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c09a450 0x7f1c7c0cf9c0 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c836fc640 1 --2- 172.21.15.19:0/1719273943 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 0x7f1c7c0d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c836fc640 1 -- 172.21.15.19:0/1719273943 >> 172.21.15.19:0/1719273943 conn(0x7f1c7c0b3270 msgr2=0x7f1c7c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c836fc640 1 -- 172.21.15.19:0/1719273943 shutdown_connections 2024-04-25T10:05:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.042+0000 7f1c836fc640 1 -- 172.21.15.19:0/1719273943 wait complete. 2024-04-25T10:05:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.044+0000 7f1c836fc640 1 Processor -- start 2024-04-25T10:05:10.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.044+0000 7f1c836fc640 1 -- start start 2024-04-25T10:05:10.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.044+0000 7f1c836fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c7c06c700 0x7f1c7c159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.044+0000 7f1c836fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c09a450 0x7f1c7c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.044+0000 7f1c836fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 0x7f1c7c0ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.044+0000 7f1c836fc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0d0920 con 0x7f1c7c09a450 2024-04-25T10:05:10.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.044+0000 7f1c836fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0d0640 con 0x7f1c7c06c700 2024-04-25T10:05:10.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.044+0000 7f1c836fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c15e050 con 0x7f1c7c15dc70 2024-04-25T10:05:10.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.044+0000 7f1c80c97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c09a450 0x7f1c7c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c80c97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c09a450 0x7f1c7c0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38028/0 (socket says 172.21.15.19:38028) 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c80c97640 1 -- 172.21.15.19:0/2856972834 learned_addr learned my addr 172.21.15.19:0/2856972834 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c81498640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c7c06c700 0x7f1c7c159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c81c99640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 0x7f1c7c0ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c80c97640 1 -- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 msgr2=0x7f1c7c0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c80c97640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 0x7f1c7c0ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c80c97640 1 -- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c7c06c700 msgr2=0x7f1c7c159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c80c97640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c7c06c700 0x7f1c7c159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c80c97640 1 -- 172.21.15.19:0/2856972834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0b2aa0 con 0x7f1c7c09a450 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c81c99640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 0x7f1c7c0ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c81498640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c7c06c700 0x7f1c7c159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c80c97640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c09a450 0x7f1c7c0cdf80 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000c00 tx=0x7f1c700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c79ffb640 1 -- 172.21.15.19:0/2856972834 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70041020 con 0x7f1c7c09a450 2024-04-25T10:05:10.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c79ffb640 1 -- 172.21.15.19:0/2856972834 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c700306c0 con 0x7f1c7c09a450 2024-04-25T10:05:10.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c836fc640 1 -- 172.21.15.19:0/2856972834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c1f7930 con 0x7f1c7c09a450 2024-04-25T10:05:10.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c79ffb640 1 -- 172.21.15.19:0/2856972834 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70037040 con 0x7f1c7c09a450 2024-04-25T10:05:10.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.045+0000 7f1c836fc640 1 -- 172.21.15.19:0/2856972834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c0c5310 con 0x7f1c7c09a450 2024-04-25T10:05:10.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.047+0000 7f1c836fc640 1 -- 172.21.15.19:0/2856972834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c073990 con 0x7f1c7c09a450 2024-04-25T10:05:10.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.047+0000 7f1c79ffb640 1 -- 172.21.15.19:0/2856972834 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c70044020 con 0x7f1c7c09a450 2024-04-25T10:05:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.047+0000 7f1c79ffb640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c6806b4d0 0x7f1c6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.048+0000 7f1c79ffb640 1 -- 172.21.15.19:0/2856972834 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1c700aa7d0 con 0x7f1c7c09a450 2024-04-25T10:05:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.048+0000 7f1c81498640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c6806b4d0 0x7f1c6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.048+0000 7f1c81498640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c6806b4d0 0x7f1c6806d990 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1c74006fd0 tx=0x7f1c74008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.052+0000 7f1c79ffb640 1 -- 172.21.15.19:0/2856972834 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c7004b030 con 0x7f1c7c09a450 2024-04-25T10:05:10.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.298+0000 7f1c836fc640 1 -- 172.21.15.19:0/2856972834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1c7c09a210 con 0x7f1c7c09a450 2024-04-25T10:05:10.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.298+0000 7f1c79ffb640 1 -- 172.21.15.19:0/2856972834 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1c7c0b2aa0 con 0x7f1c7c09a450 2024-04-25T10:05:10.303 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.304 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.305 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:05:10.306 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T10:05:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:10.308 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T10:05:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T10:05:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T10:05:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T10:05:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T10:05:10.312 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T10:05:10.312 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.312 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:10.313 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:10.313 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.313 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.314 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T10:05:10.314 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T10:05:10.314 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.314 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:10.314 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:10.314 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.315 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.315 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T10:05:10.315 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T10:05:10.316 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.316 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:10.316 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T10:05:10.316 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:10.316 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.316 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T10:05:10.317 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T10:05:10.317 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.317 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:10.317 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:05:10.317 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:05:10.317 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:05:10.317 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.318 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T10:05:10.318 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T10:05:10.318 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.318 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:10.319 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:05:10.319 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:05:10.319 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:05:10.319 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.319 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T10:05:10.320 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T10:05:10.320 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.320 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:10.320 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:05:10.320 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:05:10.320 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:05:10.320 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.321 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:10.323 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T10:05:10.323 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:05:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 -- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c6806b4d0 msgr2=0x7f1c6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c6806b4d0 0x7f1c6806d990 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1c74006fd0 tx=0x7f1c74008040 comp rx=0 tx=0).stop 2024-04-25T10:05:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 -- 172.21.15.19:0/2856972834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c09a450 msgr2=0x7f1c7c0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c09a450 0x7f1c7c0cdf80 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000c00 tx=0x7f1c700046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 -- 172.21.15.19:0/2856972834 shutdown_connections 2024-04-25T10:05:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1c6806b4d0 0x7f1c6806d990 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c7c15dc70 0x7f1c7c0ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c09a450 0x7f1c7c0cdf80 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 --2- 172.21.15.19:0/2856972834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c7c06c700 0x7f1c7c159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 -- 172.21.15.19:0/2856972834 >> 172.21.15.19:0/2856972834 conn(0x7f1c7c0b3270 msgr2=0x7f1c7c070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 -- 172.21.15.19:0/2856972834 shutdown_connections 2024-04-25T10:05:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.300+0000 7f1c836fc640 1 -- 172.21.15.19:0/2856972834 wait complete. 2024-04-25T10:05:10.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:10.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.421+0000 7fba1cf6a640 1 Processor -- start 2024-04-25T10:05:10.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.421+0000 7fba1cf6a640 1 -- start start 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.422+0000 7fba1cf6a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba1815b9c0 0x7fba1815bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.422+0000 7fba1cf6a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fba18072d30 0x7fba1806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.422+0000 7fba1cf6a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba180cfa60 con 0x7fba1806abe0 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.422+0000 7fba1cf6a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba180731a0 con 0x7fba18072d30 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.422+0000 7fba1cf6a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1806b580 con 0x7fba1815b9c0 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.422+0000 7fba16d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fba18072d30 0x7fba1806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.422+0000 7fba16d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fba18072d30 0x7fba1806b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38036/0 (socket says 172.21.15.19:38036) 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.422+0000 7fba16d9d640 1 -- 172.21.15.19:0/3490211389 learned_addr learned my addr 172.21.15.19:0/3490211389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.423+0000 7fba1659c640 1 --2- 172.21.15.19:0/3490211389 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba1815b9c0 0x7fba1815bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.423+0000 7fba16d9d640 1 -- 172.21.15.19:0/3490211389 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba1815b9c0 msgr2=0x7fba1815bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.423+0000 7fba16d9d640 1 --2- 172.21.15.19:0/3490211389 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba1815b9c0 0x7fba1815bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.423+0000 7fba16d9d640 1 -- 172.21.15.19:0/3490211389 >> v1:172.21.15.112:6789/0 conn(0x7fba1806abe0 legacy=0x7fba18072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.423+0000 7fba16d9d640 1 -- 172.21.15.19:0/3490211389 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba180cfd40 con 0x7fba18072d30 2024-04-25T10:05:10.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.423+0000 7fba16d9d640 1 --2- 172.21.15.19:0/3490211389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba18072d30 0x7fba1806b040 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fba10009870 tx=0x7fba1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db56132672c56d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.424+0000 7fba1559a640 1 -- 172.21.15.19:0/3490211389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10037070 con 0x7fba18072d30 2024-04-25T10:05:10.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.424+0000 7fba1559a640 1 -- 172.21.15.19:0/3490211389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba10032440 con 0x7fba18072d30 2024-04-25T10:05:10.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.424+0000 7fba1559a640 1 -- 172.21.15.19:0/3490211389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10032810 con 0x7fba18072d30 2024-04-25T10:05:10.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.424+0000 7fba1cf6a640 1 -- 172.21.15.19:0/3490211389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba18072d30 msgr2=0x7fba1806b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.424+0000 7fba1cf6a640 1 --2- 172.21.15.19:0/3490211389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba18072d30 0x7fba1806b040 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fba10009870 tx=0x7fba1002a270 comp rx=0 tx=0).stop 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.424+0000 7fba1cf6a640 1 -- 172.21.15.19:0/3490211389 shutdown_connections 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.424+0000 7fba1cf6a640 1 --2- 172.21.15.19:0/3490211389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba18072d30 0x7fba1806b040 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.424+0000 7fba1cf6a640 1 --2- 172.21.15.19:0/3490211389 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba1815b9c0 0x7fba1815bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.424+0000 7fba1cf6a640 1 -- 172.21.15.19:0/3490211389 >> 172.21.15.19:0/3490211389 conn(0x7fba180b2370 msgr2=0x7fba180b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.424+0000 7fba1cf6a640 1 -- 172.21.15.19:0/3490211389 shutdown_connections 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.424+0000 7fba1cf6a640 1 -- 172.21.15.19:0/3490211389 wait complete. 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.426+0000 7fba1cf6a640 1 Processor -- start 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.426+0000 7fba1cf6a640 1 -- start start 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.426+0000 7fba1cf6a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1806abe0 0x7fba181562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.426+0000 7fba1cf6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba18072d30 0x7fba18154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.426+0000 7fba1cf6a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba1815b9c0 0x7fba181f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.426+0000 7fba1cf6a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba180cf710 con 0x7fba18072d30 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.426+0000 7fba1cf6a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba180d0300 con 0x7fba1806abe0 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.426+0000 7fba1cf6a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1815c2e0 con 0x7fba1815b9c0 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.426+0000 7fba15d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba18072d30 0x7fba18154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba15d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba18072d30 0x7fba18154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38042/0 (socket says 172.21.15.19:38042) 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba15d9b640 1 -- 172.21.15.19:0/1246587452 learned_addr learned my addr 172.21.15.19:0/1246587452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba1659c640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1806abe0 0x7fba181562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba16d9d640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba1815b9c0 0x7fba181f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba15d9b640 1 -- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba1815b9c0 msgr2=0x7fba181f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba15d9b640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba1815b9c0 0x7fba181f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba15d9b640 1 -- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1806abe0 msgr2=0x7fba181562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba15d9b640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1806abe0 0x7fba181562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba15d9b640 1 -- 172.21.15.19:0/1246587452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba181f1d20 con 0x7fba18072d30 2024-04-25T10:05:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba1659c640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1806abe0 0x7fba181562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:10.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba15d9b640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba18072d30 0x7fba18154490 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fba0000c3f0 tx=0x7fba0000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba0effd640 1 -- 172.21.15.19:0/1246587452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba000052d0 con 0x7fba18072d30 2024-04-25T10:05:10.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba0effd640 1 -- 172.21.15.19:0/1246587452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba00005490 con 0x7fba18072d30 2024-04-25T10:05:10.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba0effd640 1 -- 172.21.15.19:0/1246587452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00007260 con 0x7fba18072d30 2024-04-25T10:05:10.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba1cf6a640 1 -- 172.21.15.19:0/1246587452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba181f2000 con 0x7fba18072d30 2024-04-25T10:05:10.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.427+0000 7fba1cf6a640 1 -- 172.21.15.19:0/1246587452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba181f25c0 con 0x7fba18072d30 2024-04-25T10:05:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.428+0000 7fba1cf6a640 1 -- 172.21.15.19:0/1246587452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba18073990 con 0x7fba18072d30 2024-04-25T10:05:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.429+0000 7fba0effd640 1 -- 172.21.15.19:0/1246587452 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba00007400 con 0x7fba18072d30 2024-04-25T10:05:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.429+0000 7fba0effd640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb9f806b4d0 0x7fb9f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.429+0000 7fba0effd640 1 -- 172.21.15.19:0/1246587452 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fba000895a0 con 0x7fba18072d30 2024-04-25T10:05:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.429+0000 7fba1659c640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb9f806b4d0 0x7fb9f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.430+0000 7fba1659c640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb9f806b4d0 0x7fb9f806d990 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fba1814d4a0 tx=0x7fba0800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.433+0000 7fba0effd640 1 -- 172.21.15.19:0/1246587452 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba00058560 con 0x7fba18072d30 2024-04-25T10:05:10.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.617+0000 7fba1cf6a640 1 -- 172.21.15.19:0/1246587452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba180b0530 con 0x7fba18072d30 2024-04-25T10:05:10.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.618+0000 7fba0effd640 1 -- 172.21.15.19:0/1246587452 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+13162 (secure 0 0 0) 0x7fba00057e60 con 0x7fba18072d30 2024-04-25T10:05:10.623 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:10.624 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":174,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:09.924423+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:10.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 -- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb9f806b4d0 msgr2=0x7fb9f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb9f806b4d0 0x7fb9f806d990 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fba1814d4a0 tx=0x7fba0800b040 comp rx=0 tx=0).stop 2024-04-25T10:05:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 -- 172.21.15.19:0/1246587452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba18072d30 msgr2=0x7fba18154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba18072d30 0x7fba18154490 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fba0000c3f0 tx=0x7fba0000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 -- 172.21.15.19:0/1246587452 shutdown_connections 2024-04-25T10:05:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb9f806b4d0 0x7fb9f806d990 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba1815b9c0 0x7fba181f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba18072d30 0x7fba18154490 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 --2- 172.21.15.19:0/1246587452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1806abe0 0x7fba181562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 -- 172.21.15.19:0/1246587452 >> 172.21.15.19:0/1246587452 conn(0x7fba180b2370 msgr2=0x7fba180cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 -- 172.21.15.19:0/1246587452 shutdown_connections 2024-04-25T10:05:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.620+0000 7fba1cf6a640 1 -- 172.21.15.19:0/1246587452 wait complete. 2024-04-25T10:05:10.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:10.637 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.735+0000 7f94481f4640 1 Processor -- start 2024-04-25T10:05:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.735+0000 7f94481f4640 1 -- start start 2024-04-25T10:05:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.736+0000 7f94481f4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9440166a10 0x7f9440168e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.736+0000 7f94481f4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94400afc80 con 0x7f94400cadb0 2024-04-25T10:05:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.736+0000 7f94481f4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94400ce4d0 con 0x7f944015de80 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.736+0000 7f94481f4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94400cb240 con 0x7f9440166a10 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.736+0000 7f944578f640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f94400cadb0 0x7f94400cf6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37386/0 (socket says 172.21.15.19:37386) 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.736+0000 7f944578f640 1 -- 172.21.15.19:0/3538711820 learned_addr learned my addr 172.21.15.19:0/3538711820 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784469826 0 0) 0x7f94400afc80 con 0x7f94400cadb0 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9428003520 con 0x7f94400cadb0 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9446791640 1 --2- 172.21.15.19:0/3538711820 >> v2:172.21.15.112:3300/0 conn(0x7f9440166a10 0x7f9440168e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2950317078 0 0) 0x7f9428003520 con 0x7f94400cadb0 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 >> v2:172.21.15.112:3300/0 conn(0x7f9440166a10 msgr2=0x7f9440168e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 --2- 172.21.15.19:0/3538711820 >> v2:172.21.15.112:3300/0 conn(0x7f9440166a10 0x7f9440168e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 >> v1:172.21.15.112:6789/0 conn(0x7f944015de80 legacy=0x7f94400cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94400ce7b0 con 0x7f94400cadb0 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f9434001670 con 0x7f94400cadb0 2024-04-25T10:05:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 >> v1:172.21.15.19:6789/0 conn(0x7f94400cadb0 legacy=0x7f94400cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 --2- 172.21.15.19:0/3538711820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94280039b0 0x7f9428003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 --2- 172.21.15.19:0/3538711820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 0x7f9428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 --2- 172.21.15.19:0/3538711820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94280051c0 0x7f942800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942800c130 con 0x7f9428004800 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942800c2e0 con 0x7f94280039b0 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942800c4e0 con 0x7f94280051c0 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f944578f640 1 --2- 172.21.15.19:0/3538711820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 0x7f9428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9445f90640 1 --2- 172.21.15.19:0/3538711820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94280039b0 0x7f9428003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9446791640 1 --2- 172.21.15.19:0/3538711820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94280051c0 0x7f942800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f944578f640 1 -- 172.21.15.19:0/3538711820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94280051c0 msgr2=0x7f942800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f944578f640 1 --2- 172.21.15.19:0/3538711820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94280051c0 0x7f942800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f944578f640 1 -- 172.21.15.19:0/3538711820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94280039b0 msgr2=0x7f9428003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f944578f640 1 --2- 172.21.15.19:0/3538711820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94280039b0 0x7f9428003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f944578f640 1 -- 172.21.15.19:0/3538711820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9428003520 con 0x7f9428004800 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.737+0000 7f9446791640 1 -- 172.21.15.19:0/3538711820 reap_dead start 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.738+0000 7f944578f640 1 --2- 172.21.15.19:0/3538711820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 0x7f9428004c80 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f943400c240 tx=0x7f9428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bdd380a01b974d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.738+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434007cf0 con 0x7f9428004800 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.738+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9434005af0 con 0x7f9428004800 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.738+0000 7f9444f8e640 1 -- 172.21.15.19:0/3538711820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434007070 con 0x7f9428004800 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.739+0000 7f94481f4640 1 -- 172.21.15.19:0/3538711820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 msgr2=0x7f9428004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.739+0000 7f94481f4640 1 --2- 172.21.15.19:0/3538711820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 0x7f9428004c80 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f943400c240 tx=0x7f9428005830 comp rx=0 tx=0).stop 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.739+0000 7f94481f4640 1 -- 172.21.15.19:0/3538711820 shutdown_connections 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.739+0000 7f94481f4640 1 --2- 172.21.15.19:0/3538711820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 0x7f9428004c80 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.739+0000 7f94481f4640 1 -- 172.21.15.19:0/3538711820 >> 172.21.15.19:0/3538711820 conn(0x7f94400b2390 msgr2=0x7f94400b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.739+0000 7f94481f4640 1 -- 172.21.15.19:0/3538711820 shutdown_connections 2024-04-25T10:05:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.739+0000 7f94481f4640 1 -- 172.21.15.19:0/3538711820 wait complete. 2024-04-25T10:05:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.740+0000 7f94481f4640 1 Processor -- start 2024-04-25T10:05:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.740+0000 7f94481f4640 1 -- start start 2024-04-25T10:05:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f94481f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 0x7f944014f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f94481f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f944015cb20 0x7f944014e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f94481f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94400cc700 0x7f94400ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f94481f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400b2eb0 con 0x7f9428004800 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f94481f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944015e260 con 0x7f94400cc700 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f94481f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94400b3030 con 0x7f944015cb20 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f9445f90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 0x7f944014f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f9445f90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 0x7f944014f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38056/0 (socket says 172.21.15.19:38056) 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f9445f90640 1 -- 172.21.15.19:0/3997819664 learned_addr learned my addr 172.21.15.19:0/3997819664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f944578f640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f944015cb20 0x7f944014e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f9446791640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94400cc700 0x7f94400ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f9445f90640 1 -- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f944015cb20 msgr2=0x7f944014e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f9445f90640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f944015cb20 0x7f944014e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f9445f90640 1 -- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94400cc700 msgr2=0x7f94400ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f9445f90640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94400cc700 0x7f94400ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.741+0000 7f9445f90640 1 -- 172.21.15.19:0/3997819664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944014f070 con 0x7f9428004800 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.742+0000 7f9445f90640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 0x7f944014f310 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f943000f990 tx=0x7f943000fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.742+0000 7f943e7fc640 1 -- 172.21.15.19:0/3997819664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430003b80 con 0x7f9428004800 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.742+0000 7f943e7fc640 1 -- 172.21.15.19:0/3997819664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9430018070 con 0x7f9428004800 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.742+0000 7f943e7fc640 1 -- 172.21.15.19:0/3997819664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94300050d0 con 0x7f9428004800 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.742+0000 7f94481f4640 1 -- 172.21.15.19:0/3997819664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94400cd4b0 con 0x7f9428004800 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.742+0000 7f94481f4640 1 -- 172.21.15.19:0/3997819664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94400ca5f0 con 0x7f9428004800 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.743+0000 7f94481f4640 1 -- 172.21.15.19:0/3997819664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9440073a00 con 0x7f9428004800 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.743+0000 7f943e7fc640 1 -- 172.21.15.19:0/3997819664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9430005270 con 0x7f9428004800 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.744+0000 7f943e7fc640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f942c06b4d0 0x7f942c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.744+0000 7f943e7fc640 1 -- 172.21.15.19:0/3997819664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9430089fd0 con 0x7f9428004800 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.744+0000 7f944578f640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f942c06b4d0 0x7f942c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.745+0000 7f944578f640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f942c06b4d0 0x7f942c06d990 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9434032000 tx=0x7f9434009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.748+0000 7f943e7fc640 1 -- 172.21.15.19:0/3997819664 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9430059000 con 0x7f9428004800 2024-04-25T10:05:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.929+0000 7f94481f4640 1 -- 172.21.15.19:0/3997819664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94400b2910 con 0x7f9428004800 2024-04-25T10:05:10.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.929+0000 7f943e7fc640 1 -- 172.21.15.19:0/3997819664 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v174) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9430058900 con 0x7f9428004800 2024-04-25T10:05:10.934 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:05:10.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 -- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f942c06b4d0 msgr2=0x7f942c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f942c06b4d0 0x7f942c06d990 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9434032000 tx=0x7f9434009030 comp rx=0 tx=0).stop 2024-04-25T10:05:10.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 -- 172.21.15.19:0/3997819664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 msgr2=0x7f944014f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 0x7f944014f310 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f943000f990 tx=0x7f943000fd90 comp rx=0 tx=0).stop 2024-04-25T10:05:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 -- 172.21.15.19:0/3997819664 shutdown_connections 2024-04-25T10:05:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f942c06b4d0 0x7f942c06d990 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94400cc700 0x7f94400ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f944015cb20 0x7f944014e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 --2- 172.21.15.19:0/3997819664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9428004800 0x7f944014f310 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 -- 172.21.15.19:0/3997819664 >> 172.21.15.19:0/3997819664 conn(0x7f94400b2390 msgr2=0x7f94400cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 -- 172.21.15.19:0/3997819664 shutdown_connections 2024-04-25T10:05:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:10.931+0000 7f94481f4640 1 -- 172.21.15.19:0/3997819664 wait complete. 2024-04-25T10:05:10.947 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:05:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.043+0000 7efe36522640 1 Processor -- start 2024-04-25T10:05:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.044+0000 7efe36522640 1 -- start start 2024-04-25T10:05:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.044+0000 7efe36522640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dbe0 0x7efe300d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.044+0000 7efe36522640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efe300b2cb0 0x7efe300d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.044+0000 7efe36522640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe300b92a0 0x7efe300b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.044+0000 7efe36522640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3015dfc0 con 0x7efe300b92a0 2024-04-25T10:05:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.044+0000 7efe36522640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe300d1a00 con 0x7efe300b2cb0 2024-04-25T10:05:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.044+0000 7efe36522640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe300b97a0 con 0x7efe3015dbe0 2024-04-25T10:05:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.044+0000 7efe34abf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe300b92a0 0x7efe300b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.045+0000 7efe34abf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe300b92a0 0x7efe300b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38062/0 (socket says 172.21.15.19:38062) 2024-04-25T10:05:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.045+0000 7efe34abf640 1 -- 172.21.15.19:0/1428718296 learned_addr learned my addr 172.21.15.19:0/1428718296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:11.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.045+0000 7efe2f7fe640 1 --2- 172.21.15.19:0/1428718296 >> v2:172.21.15.112:3300/0 conn(0x7efe300b2cb0 0x7efe300d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.045+0000 7efe2ffff640 1 --2- 172.21.15.19:0/1428718296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dbe0 0x7efe300d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.045+0000 7efe34abf640 1 -- 172.21.15.19:0/1428718296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dbe0 msgr2=0x7efe300d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.045+0000 7efe34abf640 1 --2- 172.21.15.19:0/1428718296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dbe0 0x7efe300d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.045+0000 7efe34abf640 1 -- 172.21.15.19:0/1428718296 >> v2:172.21.15.112:3300/0 conn(0x7efe300b2cb0 msgr2=0x7efe300d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.045+0000 7efe34abf640 1 --2- 172.21.15.19:0/1428718296 >> v2:172.21.15.112:3300/0 conn(0x7efe300b2cb0 0x7efe300d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.045+0000 7efe34abf640 1 -- 172.21.15.19:0/1428718296 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300d1c50 con 0x7efe300b92a0 2024-04-25T10:05:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe34abf640 1 --2- 172.21.15.19:0/1428718296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300b92a0 0x7efe300b5400 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7efe300b64d0 tx=0x7efe2802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b427169193dbc08a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe2effd640 1 -- 172.21.15.19:0/1428718296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28036070 con 0x7efe300b92a0 2024-04-25T10:05:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe2effd640 1 -- 172.21.15.19:0/1428718296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe28031600 con 0x7efe300b92a0 2024-04-25T10:05:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe2effd640 1 -- 172.21.15.19:0/1428718296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe280319d0 con 0x7efe300b92a0 2024-04-25T10:05:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe36522640 1 -- 172.21.15.19:0/1428718296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300b92a0 msgr2=0x7efe300b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe36522640 1 --2- 172.21.15.19:0/1428718296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300b92a0 0x7efe300b5400 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7efe300b64d0 tx=0x7efe2802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe36522640 1 -- 172.21.15.19:0/1428718296 shutdown_connections 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe36522640 1 --2- 172.21.15.19:0/1428718296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300b92a0 0x7efe300b5400 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe36522640 1 --2- 172.21.15.19:0/1428718296 >> v2:172.21.15.112:3300/0 conn(0x7efe300b2cb0 0x7efe300d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe36522640 1 --2- 172.21.15.19:0/1428718296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dbe0 0x7efe300d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe36522640 1 -- 172.21.15.19:0/1428718296 >> 172.21.15.19:0/1428718296 conn(0x7efe30061a20 msgr2=0x7efe30061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe36522640 1 -- 172.21.15.19:0/1428718296 shutdown_connections 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.046+0000 7efe36522640 1 -- 172.21.15.19:0/1428718296 wait complete. 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.047+0000 7efe36522640 1 Processor -- start 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.048+0000 7efe36522640 1 -- start start 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.048+0000 7efe36522640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300b2cb0 0x7efe300b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.048+0000 7efe36522640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe300b92a0 0x7efe3015fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.048+0000 7efe36522640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dbe0 0x7efe3015c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.048+0000 7efe36522640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe300b6ab0 con 0x7efe300b2cb0 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.048+0000 7efe36522640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe300d2210 con 0x7efe300b92a0 2024-04-25T10:05:11.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.048+0000 7efe36522640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe300b6c30 con 0x7efe3015dbe0 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.048+0000 7efe2ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300b2cb0 0x7efe300b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.048+0000 7efe2ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300b2cb0 0x7efe300b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38076/0 (socket says 172.21.15.19:38076) 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.048+0000 7efe2ffff640 1 -- 172.21.15.19:0/1710565958 learned_addr learned my addr 172.21.15.19:0/1710565958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe2f7fe640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe300b92a0 0x7efe3015fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe34abf640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dbe0 0x7efe3015c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe2ffff640 1 -- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dbe0 msgr2=0x7efe3015c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe2ffff640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dbe0 0x7efe3015c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe2ffff640 1 -- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe300b92a0 msgr2=0x7efe3015fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe2ffff640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe300b92a0 0x7efe3015fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe2ffff640 1 -- 172.21.15.19:0/1710565958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe301600c0 con 0x7efe300b2cb0 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe2f7fe640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe300b92a0 0x7efe3015fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe34abf640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dbe0 0x7efe3015c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe2ffff640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300b2cb0 0x7efe300b5d60 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7efe2000c820 tx=0x7efe2000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe2cff9640 1 -- 172.21.15.19:0/1710565958 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20015070 con 0x7efe300b2cb0 2024-04-25T10:05:11.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe2cff9640 1 -- 172.21.15.19:0/1710565958 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe20004020 con 0x7efe300b2cb0 2024-04-25T10:05:11.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe2cff9640 1 -- 172.21.15.19:0/1710565958 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe200043f0 con 0x7efe300b2cb0 2024-04-25T10:05:11.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe36522640 1 -- 172.21.15.19:0/1710565958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe300cc990 con 0x7efe300b2cb0 2024-04-25T10:05:11.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.049+0000 7efe36522640 1 -- 172.21.15.19:0/1710565958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe300ccf50 con 0x7efe300b2cb0 2024-04-25T10:05:11.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.050+0000 7efe36522640 1 -- 172.21.15.19:0/1710565958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe30073990 con 0x7efe300b2cb0 2024-04-25T10:05:11.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.051+0000 7efe2cff9640 1 -- 172.21.15.19:0/1710565958 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe20007830 con 0x7efe300b2cb0 2024-04-25T10:05:11.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.051+0000 7efe2cff9640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1c06b4d0 0x7efe1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.051+0000 7efe2cff9640 1 -- 172.21.15.19:0/1710565958 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efe20089de0 con 0x7efe300b2cb0 2024-04-25T10:05:11.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.051+0000 7efe2f7fe640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1c06b4d0 0x7efe1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.052+0000 7efe2f7fe640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1c06b4d0 0x7efe1c06d990 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7efe30157900 tx=0x7efe24008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.055+0000 7efe2cff9640 1 -- 172.21.15.19:0/1710565958 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe20058e20 con 0x7efe300b2cb0 2024-04-25T10:05:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.238+0000 7efe36522640 1 -- 172.21.15.19:0/1710565958 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efe30159ca0 con 0x7efe300b2cb0 2024-04-25T10:05:11.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.238+0000 7efe2cff9640 1 -- 172.21.15.19:0/1710565958 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v174) v1 ==== 84+0+65 (secure 0 0 0) 0x7efe20058720 con 0x7efe300b2cb0 2024-04-25T10:05:11.244 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:11.244 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T10:05:11.244 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:11.244 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T10:05:11.244 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T10:05:11.244 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:11.244 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T10:05:11.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 -- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1c06b4d0 msgr2=0x7efe1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1c06b4d0 0x7efe1c06d990 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7efe30157900 tx=0x7efe24008040 comp rx=0 tx=0).stop 2024-04-25T10:05:11.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 -- 172.21.15.19:0/1710565958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300b2cb0 msgr2=0x7efe300b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300b2cb0 0x7efe300b5d60 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7efe2000c820 tx=0x7efe2000cce0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 -- 172.21.15.19:0/1710565958 shutdown_connections 2024-04-25T10:05:11.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efe1c06b4d0 0x7efe1c06d990 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efe3015dbe0 0x7efe3015c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe300b92a0 0x7efe3015fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 --2- 172.21.15.19:0/1710565958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe300b2cb0 0x7efe300b5d60 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 -- 172.21.15.19:0/1710565958 >> 172.21.15.19:0/1710565958 conn(0x7efe30061a20 msgr2=0x7efe300b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 -- 172.21.15.19:0/1710565958 shutdown_connections 2024-04-25T10:05:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.240+0000 7efe36522640 1 -- 172.21.15.19:0/1710565958 wait complete. 2024-04-25T10:05:11.257 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:05:11.257 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:05:11.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.356+0000 7f2a4228f640 1 Processor -- start 2024-04-25T10:05:11.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.356+0000 7f2a4228f640 1 -- start start 2024-04-25T10:05:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a4228f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2a3c15dc20 0x7f2a3c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a4228f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2a3c0d32f0 0x7f2a3c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a4228f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3c15e000 con 0x7f2a3c161710 2024-04-25T10:05:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a4228f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c0d1de0 con 0x7f2a3c0d32f0 2024-04-25T10:05:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a4228f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c161c60 con 0x7f2a3c15dc20 2024-04-25T10:05:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a4082c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2a3c0d32f0 0x7f2a3c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a3b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2a3c161710 0x7f2a3c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37390/0 (socket says 172.21.15.19:37390) 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a3b7fe640 1 -- 172.21.15.19:0/2851506152 learned_addr learned my addr 172.21.15.19:0/2851506152 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a3bfff640 1 --2- 172.21.15.19:0/2851506152 >> v2:172.21.15.112:3300/0 conn(0x7f2a3c15dc20 0x7f2a3c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a4082c640 1 -- 172.21.15.19:0/2851506152 >> v2:172.21.15.112:3300/0 conn(0x7f2a3c15dc20 msgr2=0x7f2a3c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a4082c640 1 --2- 172.21.15.19:0/2851506152 >> v2:172.21.15.112:3300/0 conn(0x7f2a3c15dc20 0x7f2a3c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a4082c640 1 -- 172.21.15.19:0/2851506152 >> v1:172.21.15.19:6789/0 conn(0x7f2a3c161710 legacy=0x7f2a3c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a4082c640 1 -- 172.21.15.19:0/2851506152 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c161f40 con 0x7f2a3c0d32f0 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.357+0000 7f2a3affd640 1 -- 172.21.15.19:0/2851506152 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843157537 0 0) 0x7f2a3c15e000 con 0x7f2a3c161710 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.358+0000 7f2a4082c640 1 --2- 172.21.15.19:0/2851506152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3c0d32f0 0x7f2a3c0cebb0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0d1520 tx=0x7f2a3402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96abf0b089e04a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.358+0000 7f2a3affd640 1 -- 172.21.15.19:0/2851506152 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34007740 con 0x7f2a3c0d32f0 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.358+0000 7f2a3affd640 1 -- 172.21.15.19:0/2851506152 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a340078e0 con 0x7f2a3c0d32f0 2024-04-25T10:05:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.358+0000 7f2a3affd640 1 -- 172.21.15.19:0/2851506152 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34007cb0 con 0x7f2a3c0d32f0 2024-04-25T10:05:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.359+0000 7f2a4228f640 1 -- 172.21.15.19:0/2851506152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3c0d32f0 msgr2=0x7f2a3c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.359+0000 7f2a4228f640 1 --2- 172.21.15.19:0/2851506152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3c0d32f0 0x7f2a3c0cebb0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0d1520 tx=0x7f2a3402a680 comp rx=0 tx=0).stop 2024-04-25T10:05:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.359+0000 7f2a4228f640 1 -- 172.21.15.19:0/2851506152 shutdown_connections 2024-04-25T10:05:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.359+0000 7f2a4228f640 1 --2- 172.21.15.19:0/2851506152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3c0d32f0 0x7f2a3c0cebb0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.359+0000 7f2a4228f640 1 --2- 172.21.15.19:0/2851506152 >> v2:172.21.15.112:3300/0 conn(0x7f2a3c15dc20 0x7f2a3c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.359+0000 7f2a4228f640 1 -- 172.21.15.19:0/2851506152 >> 172.21.15.19:0/2851506152 conn(0x7f2a3c0d0420 msgr2=0x7f2a3c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.359+0000 7f2a4228f640 1 -- 172.21.15.19:0/2851506152 shutdown_connections 2024-04-25T10:05:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.359+0000 7f2a4228f640 1 -- 172.21.15.19:0/2851506152 wait complete. 2024-04-25T10:05:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.360+0000 7f2a4228f640 1 Processor -- start 2024-04-25T10:05:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.360+0000 7f2a4228f640 1 -- start start 2024-04-25T10:05:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4228f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a3c0d32f0 0x7f2a3c152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4228f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a3c15dc20 0x7f2a3c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4228f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3c161710 0x7f2a3c157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4228f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b5e50 con 0x7f2a3c161710 2024-04-25T10:05:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4228f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c162500 con 0x7f2a3c15dc20 2024-04-25T10:05:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4228f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b5fd0 con 0x7f2a3c0d32f0 2024-04-25T10:05:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4082c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3c161710 0x7f2a3c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4082c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3c161710 0x7f2a3c157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38102/0 (socket says 172.21.15.19:38102) 2024-04-25T10:05:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4082c640 1 -- 172.21.15.19:0/931668901 learned_addr learned my addr 172.21.15.19:0/931668901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a3b7fe640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a3c15dc20 0x7f2a3c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a3bfff640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a3c0d32f0 0x7f2a3c152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4082c640 1 -- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a3c0d32f0 msgr2=0x7f2a3c152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4082c640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a3c0d32f0 0x7f2a3c152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4082c640 1 -- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a3c15dc20 msgr2=0x7f2a3c156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4082c640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a3c15dc20 0x7f2a3c156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.361+0000 7f2a4082c640 1 -- 172.21.15.19:0/931668901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c157380 con 0x7f2a3c161710 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.362+0000 7f2a4082c640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3c161710 0x7f2a3c157750 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009e50 tx=0x7f2a34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.362+0000 7f2a38ff9640 1 -- 172.21.15.19:0/931668901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3403f070 con 0x7f2a3c161710 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.362+0000 7f2a38ff9640 1 -- 172.21.15.19:0/931668901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a34031bf0 con 0x7f2a3c161710 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.362+0000 7f2a38ff9640 1 -- 172.21.15.19:0/931668901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34009450 con 0x7f2a3c161710 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.362+0000 7f2a4228f640 1 -- 172.21.15.19:0/931668901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c15fec0 con 0x7f2a3c161710 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.362+0000 7f2a4228f640 1 -- 172.21.15.19:0/931668901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c160480 con 0x7f2a3c161710 2024-04-25T10:05:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.363+0000 7f2a4228f640 1 -- 172.21.15.19:0/931668901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c0cf940 con 0x7f2a3c161710 2024-04-25T10:05:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.363+0000 7f2a38ff9640 1 -- 172.21.15.19:0/931668901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a3403b410 con 0x7f2a3c161710 2024-04-25T10:05:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.364+0000 7f2a38ff9640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a1c06b4d0 0x7f2a1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.364+0000 7f2a38ff9640 1 -- 172.21.15.19:0/931668901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2a340ab590 con 0x7f2a3c161710 2024-04-25T10:05:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.364+0000 7f2a3bfff640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a1c06b4d0 0x7f2a1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.364+0000 7f2a3bfff640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a1c06b4d0 0x7f2a1c06d990 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009aa0 tx=0x7f2a24009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.368+0000 7f2a38ff9640 1 -- 172.21.15.19:0/931668901 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a3c0cf940 con 0x7f2a3c161710 2024-04-25T10:05:11.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.547+0000 7f2a4228f640 1 -- 172.21.15.19:0/931668901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2a3c0cf380 con 0x7f2a3c161710 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.922+0000 7f2a38ff9640 1 -- 172.21.15.19:0/931668901 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v175) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2a3c0cf380 con 0x7f2a3c161710 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T10:05:11.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 -- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a1c06b4d0 msgr2=0x7f2a1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a1c06b4d0 0x7f2a1c06d990 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009aa0 tx=0x7f2a24009190 comp rx=0 tx=0).stop 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 -- 172.21.15.19:0/931668901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3c161710 msgr2=0x7f2a3c157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3c161710 0x7f2a3c157750 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009e50 tx=0x7f2a34004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 -- 172.21.15.19:0/931668901 shutdown_connections 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2a1c06b4d0 0x7f2a1c06d990 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3c161710 0x7f2a3c157750 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a3c15dc20 0x7f2a3c156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 --2- 172.21.15.19:0/931668901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2a3c0d32f0 0x7f2a3c152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 -- 172.21.15.19:0/931668901 >> 172.21.15.19:0/931668901 conn(0x7f2a3c0d0420 msgr2=0x7f2a3c0d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 -- 172.21.15.19:0/931668901 shutdown_connections 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:11.924+0000 7f2a4228f640 1 -- 172.21.15.19:0/931668901 wait complete. 2024-04-25T10:05:11.945 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:05:12.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.050+0000 7fd942934640 1 Processor -- start 2024-04-25T10:05:12.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.050+0000 7fd942934640 1 -- start start 2024-04-25T10:05:12.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.050+0000 7fd942934640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd93c0cfa40 0x7fd93c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.050+0000 7fd942934640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93c0c75e0 con 0x7fd93c15bb50 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.050+0000 7fd942934640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93c0c78c0 con 0x7fd93c0cf600 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.050+0000 7fd942934640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd93c0c8680 con 0x7fd93c0cfa40 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd93c15bb50 0x7fd93c0c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37404/0 (socket says 172.21.15.19:37404) 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93bfff640 1 -- 172.21.15.19:0/3435918878 learned_addr learned my addr 172.21.15.19:0/3435918878 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd940ed1640 1 --2- 172.21.15.19:0/3435918878 >> v2:172.21.15.112:3300/0 conn(0x7fd93c0cfa40 0x7fd93c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89514208 0 0) 0x7fd93c0c75e0 con 0x7fd93c15bb50 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd924003520 con 0x7fd93c15bb50 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 765658977 0 0) 0x7fd924003520 con 0x7fd93c15bb50 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 >> v2:172.21.15.112:3300/0 conn(0x7fd93c0cfa40 msgr2=0x7fd93c0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 --2- 172.21.15.19:0/3435918878 >> v2:172.21.15.112:3300/0 conn(0x7fd93c0cfa40 0x7fd93c0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 >> v1:172.21.15.112:6789/0 conn(0x7fd93c0cf600 legacy=0x7fd93c0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c0c7ba0 con 0x7fd93c15bb50 2024-04-25T10:05:12.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fd92c001670 con 0x7fd93c15bb50 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 >> v1:172.21.15.19:6789/0 conn(0x7fd93c15bb50 legacy=0x7fd93c0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 --2- 172.21.15.19:0/3435918878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9240039b0 0x7fd924003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 --2- 172.21.15.19:0/3435918878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd924004800 0x7fd924004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 --2- 172.21.15.19:0/3435918878 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9240051c0 0x7fd92400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c0c8680 con 0x7fd9240039b0 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92400c070 con 0x7fd924004800 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92400c270 con 0x7fd9240051c0 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93bfff640 1 --2- 172.21.15.19:0/3435918878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9240039b0 0x7fd924003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.051+0000 7fd93b7fe640 1 --2- 172.21.15.19:0/3435918878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd924004800 0x7fd924004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.052+0000 7fd940ed1640 1 --2- 172.21.15.19:0/3435918878 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9240051c0 0x7fd92400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.052+0000 7fd93bfff640 1 -- 172.21.15.19:0/3435918878 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9240051c0 msgr2=0x7fd92400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.052+0000 7fd93bfff640 1 --2- 172.21.15.19:0/3435918878 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9240051c0 0x7fd92400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.052+0000 7fd93bfff640 1 -- 172.21.15.19:0/3435918878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd924004800 msgr2=0x7fd924004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.052+0000 7fd93bfff640 1 --2- 172.21.15.19:0/3435918878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd924004800 0x7fd924004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.052+0000 7fd93bfff640 1 -- 172.21.15.19:0/3435918878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd924003520 con 0x7fd9240039b0 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.052+0000 7fd940ed1640 1 -- 172.21.15.19:0/3435918878 reap_dead start 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.052+0000 7fd93bfff640 1 --2- 172.21.15.19:0/3435918878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9240039b0 0x7fd924003dd0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00c210 tx=0x7fd924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e135a3458d9cea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.052+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c007d30 con 0x7fd9240039b0 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.053+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c005ad0 con 0x7fd9240039b0 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.053+0000 7fd93affd640 1 -- 172.21.15.19:0/3435918878 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c007070 con 0x7fd9240039b0 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.053+0000 7fd942934640 1 -- 172.21.15.19:0/3435918878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9240039b0 msgr2=0x7fd924003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.053+0000 7fd942934640 1 --2- 172.21.15.19:0/3435918878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9240039b0 0x7fd924003dd0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00c210 tx=0x7fd924005830 comp rx=0 tx=0).stop 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.053+0000 7fd942934640 1 -- 172.21.15.19:0/3435918878 shutdown_connections 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.053+0000 7fd942934640 1 --2- 172.21.15.19:0/3435918878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9240039b0 0x7fd924003dd0 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.053+0000 7fd942934640 1 -- 172.21.15.19:0/3435918878 >> 172.21.15.19:0/3435918878 conn(0x7fd93c06f500 msgr2=0x7fd93c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.053+0000 7fd942934640 1 -- 172.21.15.19:0/3435918878 shutdown_connections 2024-04-25T10:05:12.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.053+0000 7fd942934640 1 -- 172.21.15.19:0/3435918878 wait complete. 2024-04-25T10:05:12.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.054+0000 7fd942934640 1 Processor -- start 2024-04-25T10:05:12.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.055+0000 7fd942934640 1 -- start start 2024-04-25T10:05:12.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.055+0000 7fd942934640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9240039b0 0x7fd93c13f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.055+0000 7fd942934640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd93c142e80 0x7fd93c15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.055+0000 7fd942934640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd93c1e5290 0x7fd93c1e5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.055+0000 7fd942934640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c0cfe90 con 0x7fd93c142e80 2024-04-25T10:05:12.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.055+0000 7fd942934640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c0c8160 con 0x7fd9240039b0 2024-04-25T10:05:12.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.055+0000 7fd942934640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c15f6d0 con 0x7fd93c1e5290 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.055+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd93c142e80 0x7fd93c15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.055+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd93c142e80 0x7fd93c15cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38128/0 (socket says 172.21.15.19:38128) 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.055+0000 7fd93b7fe640 1 -- 172.21.15.19:0/1353570681 learned_addr learned my addr 172.21.15.19:0/1353570681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd93bfff640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9240039b0 0x7fd93c13f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd940ed1640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd93c1e5290 0x7fd93c1e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd93b7fe640 1 -- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd93c1e5290 msgr2=0x7fd93c1e5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd93b7fe640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd93c1e5290 0x7fd93c1e5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd93b7fe640 1 -- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9240039b0 msgr2=0x7fd93c13f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd93b7fe640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9240039b0 0x7fd93c13f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd93b7fe640 1 -- 172.21.15.19:0/1353570681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c15d8f0 con 0x7fd93c142e80 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd93b7fe640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd93c142e80 0x7fd93c15cf80 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7fd93000f5f0 tx=0x7fd93000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd938ff9640 1 -- 172.21.15.19:0/1353570681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9300052e0 con 0x7fd93c142e80 2024-04-25T10:05:12.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd938ff9640 1 -- 172.21.15.19:0/1353570681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd930018070 con 0x7fd93c142e80 2024-04-25T10:05:12.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd938ff9640 1 -- 172.21.15.19:0/1353570681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930003810 con 0x7fd93c142e80 2024-04-25T10:05:12.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd942934640 1 -- 172.21.15.19:0/1353570681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93c15dbd0 con 0x7fd93c142e80 2024-04-25T10:05:12.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.056+0000 7fd942934640 1 -- 172.21.15.19:0/1353570681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93c15e190 con 0x7fd93c142e80 2024-04-25T10:05:12.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.057+0000 7fd942934640 1 -- 172.21.15.19:0/1353570681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93c073a00 con 0x7fd93c142e80 2024-04-25T10:05:12.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.058+0000 7fd938ff9640 1 -- 172.21.15.19:0/1353570681 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9300039b0 con 0x7fd93c142e80 2024-04-25T10:05:12.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.058+0000 7fd938ff9640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd92806b500 0x7fd92806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.058+0000 7fd938ff9640 1 -- 172.21.15.19:0/1353570681 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fd930089e90 con 0x7fd93c142e80 2024-04-25T10:05:12.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.058+0000 7fd93bfff640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd92806b500 0x7fd92806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.059+0000 7fd93bfff640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd92806b500 0x7fd92806d9c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd92c032000 tx=0x7fd92c02d000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.063+0000 7fd938ff9640 1 -- 172.21.15.19:0/1353570681 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd930058670 con 0x7fd93c142e80 2024-04-25T10:05:12.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.242+0000 7fd942934640 1 -- 172.21.15.19:0/1353570681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd93c15feb0 con 0x7fd93c142e80 2024-04-25T10:05:12.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.932+0000 7fd938ff9640 1 -- 172.21.15.19:0/1353570681 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v176) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd93005c710 con 0x7fd93c142e80 2024-04-25T10:05:12.937 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T10:05:12.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 -- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd92806b500 msgr2=0x7fd92806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd92806b500 0x7fd92806d9c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd92c032000 tx=0x7fd92c02d000 comp rx=0 tx=0).stop 2024-04-25T10:05:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 -- 172.21.15.19:0/1353570681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd93c142e80 msgr2=0x7fd93c15cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd93c142e80 0x7fd93c15cf80 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7fd93000f5f0 tx=0x7fd93000fab0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 -- 172.21.15.19:0/1353570681 shutdown_connections 2024-04-25T10:05:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd92806b500 0x7fd92806d9c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd93c1e5290 0x7fd93c1e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd93c142e80 0x7fd93c15cf80 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 --2- 172.21.15.19:0/1353570681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9240039b0 0x7fd93c13f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 -- 172.21.15.19:0/1353570681 >> 172.21.15.19:0/1353570681 conn(0x7fd93c06f500 msgr2=0x7fd93c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 -- 172.21.15.19:0/1353570681 shutdown_connections 2024-04-25T10:05:12.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:12.935+0000 7fd942934640 1 -- 172.21.15.19:0/1353570681 wait complete. 2024-04-25T10:05:12.966 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:05:13.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.069+0000 7f4eb44cb640 1 Processor -- start 2024-04-25T10:05:13.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.069+0000 7f4eb44cb640 1 -- start start 2024-04-25T10:05:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.070+0000 7f4eb44cb640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4eac1610f0 0x7f4eac06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.070+0000 7f4eb44cb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4eac1616f0 0x7f4eac161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.070+0000 7f4eb44cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4eac0ce0a0 0x7f4eac0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.070+0000 7f4eb44cb640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eac1614d0 con 0x7f4eac1610f0 2024-04-25T10:05:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.070+0000 7f4eb44cb640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eac0c6960 con 0x7f4eac1616f0 2024-04-25T10:05:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.070+0000 7f4eb44cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac162260 con 0x7f4eac0ce0a0 2024-04-25T10:05:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.070+0000 7f4eb2267640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4eac1610f0 0x7f4eac06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.070+0000 7f4eb2267640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4eac1610f0 0x7f4eac06c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38136/0 (socket says 172.21.15.19:38136) 2024-04-25T10:05:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.070+0000 7f4eb2267640 1 -- 172.21.15.19:0/2005762431 learned_addr learned my addr 172.21.15.19:0/2005762431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.071+0000 7f4eb1a66640 1 --2- 172.21.15.19:0/2005762431 >> v2:172.21.15.112:3300/0 conn(0x7f4eac1616f0 0x7f4eac161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.071+0000 7f4eb2a68640 1 --2- 172.21.15.19:0/2005762431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4eac0ce0a0 0x7f4eac0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.071+0000 7f4eb2267640 1 -- 172.21.15.19:0/2005762431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4eac0ce0a0 msgr2=0x7f4eac0ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.071+0000 7f4eb2267640 1 --2- 172.21.15.19:0/2005762431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4eac0ce0a0 0x7f4eac0ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.071+0000 7f4eb2267640 1 -- 172.21.15.19:0/2005762431 >> v2:172.21.15.112:3300/0 conn(0x7f4eac1616f0 msgr2=0x7f4eac161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.071+0000 7f4eb2267640 1 --2- 172.21.15.19:0/2005762431 >> v2:172.21.15.112:3300/0 conn(0x7f4eac1616f0 0x7f4eac161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.071+0000 7f4eb2267640 1 -- 172.21.15.19:0/2005762431 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac162540 con 0x7f4eac1610f0 2024-04-25T10:05:13.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.071+0000 7f4eb2267640 1 --2- 172.21.15.19:0/2005762431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1610f0 0x7f4eac06c1c0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f4eac06f220 tx=0x7f4ea402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843eeace6c599cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb1265640 1 -- 172.21.15.19:0/2005762431 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4037070 con 0x7f4eac1610f0 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb1265640 1 -- 172.21.15.19:0/2005762431 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea4032440 con 0x7f4eac1610f0 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb1265640 1 -- 172.21.15.19:0/2005762431 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4032810 con 0x7f4eac1610f0 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb44cb640 1 -- 172.21.15.19:0/2005762431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1610f0 msgr2=0x7f4eac06c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb44cb640 1 --2- 172.21.15.19:0/2005762431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1610f0 0x7f4eac06c1c0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f4eac06f220 tx=0x7f4ea402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb44cb640 1 -- 172.21.15.19:0/2005762431 shutdown_connections 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb44cb640 1 --2- 172.21.15.19:0/2005762431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4eac0ce0a0 0x7f4eac0ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb44cb640 1 --2- 172.21.15.19:0/2005762431 >> v2:172.21.15.112:3300/0 conn(0x7f4eac1616f0 0x7f4eac161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb44cb640 1 --2- 172.21.15.19:0/2005762431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1610f0 0x7f4eac06c1c0 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb44cb640 1 -- 172.21.15.19:0/2005762431 >> 172.21.15.19:0/2005762431 conn(0x7f4eac06f500 msgr2=0x7f4eac06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb44cb640 1 -- 172.21.15.19:0/2005762431 shutdown_connections 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.072+0000 7f4eb44cb640 1 -- 172.21.15.19:0/2005762431 wait complete. 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.073+0000 7f4eb44cb640 1 Processor -- start 2024-04-25T10:05:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.074+0000 7f4eb44cb640 1 -- start start 2024-04-25T10:05:13.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.074+0000 7f4eb44cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac0ce0a0 0x7f4eac1f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.074+0000 7f4eb44cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1610f0 0x7f4eac155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.074+0000 7f4eb44cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4eac1616f0 0x7f4eac0ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.074+0000 7f4eb44cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac0cf690 con 0x7f4eac1610f0 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.074+0000 7f4eb44cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac0cfce0 con 0x7f4eac0ce0a0 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.074+0000 7f4eb44cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac06c700 con 0x7f4eac1616f0 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.074+0000 7f4eb1a66640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1610f0 0x7f4eac155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.074+0000 7f4eb1a66640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1610f0 0x7f4eac155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38140/0 (socket says 172.21.15.19:38140) 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.074+0000 7f4eb1a66640 1 -- 172.21.15.19:0/3024831817 learned_addr learned my addr 172.21.15.19:0/3024831817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.074+0000 7f4eb2267640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac0ce0a0 0x7f4eac1f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eb2a68640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4eac1616f0 0x7f4eac0ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eb1a66640 1 -- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4eac1616f0 msgr2=0x7f4eac0ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eb1a66640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4eac1616f0 0x7f4eac0ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eb1a66640 1 -- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac0ce0a0 msgr2=0x7f4eac1f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eb1a66640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac0ce0a0 0x7f4eac1f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eb1a66640 1 -- 172.21.15.19:0/3024831817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac160030 con 0x7f4eac1610f0 2024-04-25T10:05:13.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eb1a66640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1610f0 0x7f4eac155fb0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000d5f0 tx=0x7f4ea000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eaaffd640 1 -- 172.21.15.19:0/3024831817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00125f0 con 0x7f4eac1610f0 2024-04-25T10:05:13.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eaaffd640 1 -- 172.21.15.19:0/3024831817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea0019070 con 0x7f4eac1610f0 2024-04-25T10:05:13.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eb44cb640 1 -- 172.21.15.19:0/3024831817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eac160310 con 0x7f4eac1610f0 2024-04-25T10:05:13.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eaaffd640 1 -- 172.21.15.19:0/3024831817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00129d0 con 0x7f4eac1610f0 2024-04-25T10:05:13.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.075+0000 7f4eb44cb640 1 -- 172.21.15.19:0/3024831817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac1608d0 con 0x7f4eac1610f0 2024-04-25T10:05:13.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.076+0000 7f4eb44cb640 1 -- 172.21.15.19:0/3024831817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eac073a00 con 0x7f4eac1610f0 2024-04-25T10:05:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.077+0000 7f4eaaffd640 1 -- 172.21.15.19:0/3024831817 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ea0012b70 con 0x7f4eac1610f0 2024-04-25T10:05:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.077+0000 7f4eaaffd640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4e9806b4d0 0x7f4e9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.078+0000 7f4eaaffd640 1 -- 172.21.15.19:0/3024831817 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f4ea008a5f0 con 0x7f4eac1610f0 2024-04-25T10:05:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.078+0000 7f4eb2267640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4e9806b4d0 0x7f4e9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.078+0000 7f4eb2267640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4e9806b4d0 0x7f4e9806d990 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4034000 tx=0x7f4ea402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.082+0000 7f4eaaffd640 1 -- 172.21.15.19:0/3024831817 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ea0058cf0 con 0x7f4eac1610f0 2024-04-25T10:05:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.298+0000 7f4eb44cb640 1 -- 172.21.15.19:0/3024831817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4eac0cdc70 con 0x7f4eac1610f0 2024-04-25T10:05:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.933+0000 7f4eaaffd640 1 -- 172.21.15.19:0/3024831817 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v114) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4ea005cd90 con 0x7f4eac1610f0 2024-04-25T10:05:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 27 and data pool 28 2024-04-25T10:05:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.938+0000 7f4eb44cb640 1 -- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4e9806b4d0 msgr2=0x7f4e9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.938+0000 7f4eb44cb640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4e9806b4d0 0x7f4e9806d990 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4034000 tx=0x7f4ea402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.938+0000 7f4eb44cb640 1 -- 172.21.15.19:0/3024831817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1610f0 msgr2=0x7f4eac155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.938+0000 7f4eb44cb640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1610f0 0x7f4eac155fb0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000d5f0 tx=0x7f4ea000dab0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.938+0000 7f4eb44cb640 1 -- 172.21.15.19:0/3024831817 shutdown_connections 2024-04-25T10:05:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.938+0000 7f4eb44cb640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4e9806b4d0 0x7f4e9806d990 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.938+0000 7f4eb44cb640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4eac1616f0 0x7f4eac0ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.938+0000 7f4eb44cb640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eac1610f0 0x7f4eac155fb0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.938+0000 7f4eb44cb640 1 --2- 172.21.15.19:0/3024831817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4eac0ce0a0 0x7f4eac1f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.938+0000 7f4eb44cb640 1 -- 172.21.15.19:0/3024831817 >> 172.21.15.19:0/3024831817 conn(0x7f4eac06f500 msgr2=0x7f4eac0d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.939+0000 7f4eb44cb640 1 -- 172.21.15.19:0/3024831817 shutdown_connections 2024-04-25T10:05:13.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:13.939+0000 7f4eb44cb640 1 -- 172.21.15.19:0/3024831817 wait complete. 2024-04-25T10:05:13.975 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:14.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.075+0000 7fd3a2858640 1 Processor -- start 2024-04-25T10:05:14.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.075+0000 7fd3a2858640 1 -- start start 2024-04-25T10:05:14.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.076+0000 7fd3a2858640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd39c06abe0 0x7fd39c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.076+0000 7fd3a2858640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd39c15d370 0x7fd39c15d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.076+0000 7fd3a2858640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39c1626e0 con 0x7fd39c0ce330 2024-04-25T10:05:14.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.076+0000 7fd3a2858640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39c06b500 con 0x7fd39c06abe0 2024-04-25T10:05:14.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.076+0000 7fd3a2858640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39c070270 con 0x7fd39c15d370 2024-04-25T10:05:14.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.076+0000 7fd39bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd39c06abe0 0x7fd39c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.076+0000 7fd39b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd39c0ce330 0x7fd39c06fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37410/0 (socket says 172.21.15.19:37410) 2024-04-25T10:05:14.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.076+0000 7fd39b7fe640 1 -- 172.21.15.19:0/652933237 learned_addr learned my addr 172.21.15.19:0/652933237 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.077+0000 7fd3a0df5640 1 --2- 172.21.15.19:0/652933237 >> v2:172.21.15.112:3300/0 conn(0x7fd39c15d370 0x7fd39c15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.077+0000 7fd39bfff640 1 -- 172.21.15.19:0/652933237 >> v2:172.21.15.112:3300/0 conn(0x7fd39c15d370 msgr2=0x7fd39c15d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.077+0000 7fd39bfff640 1 --2- 172.21.15.19:0/652933237 >> v2:172.21.15.112:3300/0 conn(0x7fd39c15d370 0x7fd39c15d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.077+0000 7fd39bfff640 1 -- 172.21.15.19:0/652933237 >> v1:172.21.15.19:6789/0 conn(0x7fd39c0ce330 legacy=0x7fd39c06fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.077+0000 7fd39affd640 1 -- 172.21.15.19:0/652933237 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006318544 0 0) 0x7fd39c1626e0 con 0x7fd39c0ce330 2024-04-25T10:05:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.077+0000 7fd39bfff640 1 -- 172.21.15.19:0/652933237 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c0cd270 con 0x7fd39c06abe0 2024-04-25T10:05:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.077+0000 7fd3a0df5640 1 --2- 172.21.15.19:0/652933237 >> v2:172.21.15.112:3300/0 conn(0x7fd39c15d370 0x7fd39c15d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.077+0000 7fd39bfff640 1 --2- 172.21.15.19:0/652933237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39c06abe0 0x7fd39c06afc0 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0b32a0 tx=0x7fd38402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a196c118dab2bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.078+0000 7fd39affd640 1 -- 172.21.15.19:0/652933237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384037070 con 0x7fd39c06abe0 2024-04-25T10:05:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.078+0000 7fd39affd640 1 -- 172.21.15.19:0/652933237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd384032440 con 0x7fd39c06abe0 2024-04-25T10:05:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.078+0000 7fd39affd640 1 -- 172.21.15.19:0/652933237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38402ae60 con 0x7fd39c06abe0 2024-04-25T10:05:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.078+0000 7fd3a2858640 1 -- 172.21.15.19:0/652933237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39c06abe0 msgr2=0x7fd39c06afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.078+0000 7fd3a2858640 1 --2- 172.21.15.19:0/652933237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39c06abe0 0x7fd39c06afc0 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0b32a0 tx=0x7fd38402a110 comp rx=0 tx=0).stop 2024-04-25T10:05:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.078+0000 7fd3a2858640 1 -- 172.21.15.19:0/652933237 shutdown_connections 2024-04-25T10:05:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.078+0000 7fd3a2858640 1 --2- 172.21.15.19:0/652933237 >> v2:172.21.15.112:3300/0 conn(0x7fd39c15d370 0x7fd39c15d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.078+0000 7fd3a2858640 1 --2- 172.21.15.19:0/652933237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39c06abe0 0x7fd39c06afc0 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.078+0000 7fd3a2858640 1 -- 172.21.15.19:0/652933237 >> 172.21.15.19:0/652933237 conn(0x7fd39c0b2220 msgr2=0x7fd39c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.078+0000 7fd3a2858640 1 -- 172.21.15.19:0/652933237 shutdown_connections 2024-04-25T10:05:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.078+0000 7fd3a2858640 1 -- 172.21.15.19:0/652933237 wait complete. 2024-04-25T10:05:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.079+0000 7fd3a2858640 1 Processor -- start 2024-04-25T10:05:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.080+0000 7fd3a2858640 1 -- start start 2024-04-25T10:05:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.080+0000 7fd3a2858640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd39c06abe0 0x7fd39c157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.080+0000 7fd3a2858640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39c0ce330 0x7fd39c157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.080+0000 7fd3a2858640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd39c15d370 0x7fd39c1469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.080+0000 7fd3a2858640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c0cdb10 con 0x7fd39c0ce330 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.080+0000 7fd3a2858640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c0cd830 con 0x7fd39c15d370 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.080+0000 7fd3a2858640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c161780 con 0x7fd39c06abe0 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.080+0000 7fd39b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39c0ce330 0x7fd39c157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.080+0000 7fd39b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39c0ce330 0x7fd39c157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38156/0 (socket says 172.21.15.19:38156) 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.080+0000 7fd39b7fe640 1 -- 172.21.15.19:0/342793525 learned_addr learned my addr 172.21.15.19:0/342793525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd3a0df5640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd39c15d370 0x7fd39c1469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd39bfff640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd39c06abe0 0x7fd39c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd39b7fe640 1 -- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd39c06abe0 msgr2=0x7fd39c157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd39b7fe640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd39c06abe0 0x7fd39c157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd39b7fe640 1 -- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd39c15d370 msgr2=0x7fd39c1469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd39b7fe640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd39c15d370 0x7fd39c1469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd39b7fe640 1 -- 172.21.15.19:0/342793525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c1584b0 con 0x7fd39c0ce330 2024-04-25T10:05:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd39b7fe640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39c0ce330 0x7fd39c157f70 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00c4e0 tx=0x7fd38c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd398ff9640 1 -- 172.21.15.19:0/342793525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c018070 con 0x7fd39c0ce330 2024-04-25T10:05:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd398ff9640 1 -- 172.21.15.19:0/342793525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38c007500 con 0x7fd39c0ce330 2024-04-25T10:05:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd3a2858640 1 -- 172.21.15.19:0/342793525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39c0b4200 con 0x7fd39c0ce330 2024-04-25T10:05:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd398ff9640 1 -- 172.21.15.19:0/342793525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c004a20 con 0x7fd39c0ce330 2024-04-25T10:05:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.081+0000 7fd3a2858640 1 -- 172.21.15.19:0/342793525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39c0b4a60 con 0x7fd39c0ce330 2024-04-25T10:05:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.082+0000 7fd3a2858640 1 -- 172.21.15.19:0/342793525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39c073990 con 0x7fd39c0ce330 2024-04-25T10:05:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.083+0000 7fd398ff9640 1 -- 172.21.15.19:0/342793525 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd38c01b070 con 0x7fd39c0ce330 2024-04-25T10:05:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.083+0000 7fd398ff9640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd37c06b500 0x7fd37c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.083+0000 7fd398ff9640 1 -- 172.21.15.19:0/342793525 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd38c089590 con 0x7fd39c0ce330 2024-04-25T10:05:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.083+0000 7fd39bfff640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd37c06b500 0x7fd37c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.084+0000 7fd39bfff640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd37c06b500 0x7fd37c06d9c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd384034000 tx=0x7fd38402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.088+0000 7fd398ff9640 1 -- 172.21.15.19:0/342793525 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd39c1584b0 con 0x7fd39c0ce330 2024-04-25T10:05:14.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.270+0000 7fd3a2858640 1 -- 172.21.15.19:0/342793525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd39c0b2ff0 con 0x7fd39c0ce330 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.271+0000 7fd398ff9640 1 -- 172.21.15.19:0/342793525 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+16953 (secure 0 0 0) 0x7fd38c057cf0 con 0x7fd39c0ce330 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":177,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:13.300075+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:11.578636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:12.274609+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 -- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd37c06b500 msgr2=0x7fd37c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd37c06b500 0x7fd37c06d9c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd384034000 tx=0x7fd38402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 -- 172.21.15.19:0/342793525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39c0ce330 msgr2=0x7fd39c157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39c0ce330 0x7fd39c157f70 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00c4e0 tx=0x7fd38c00c9a0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 -- 172.21.15.19:0/342793525 shutdown_connections 2024-04-25T10:05:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd37c06b500 0x7fd37c06d9c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd39c15d370 0x7fd39c1469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd39c0ce330 0x7fd39c157f70 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 --2- 172.21.15.19:0/342793525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd39c06abe0 0x7fd39c157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 -- 172.21.15.19:0/342793525 >> 172.21.15.19:0/342793525 conn(0x7fd39c0b2220 msgr2=0x7fd39c0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 -- 172.21.15.19:0/342793525 shutdown_connections 2024-04-25T10:05:14.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.273+0000 7fd3a2858640 1 -- 172.21.15.19:0/342793525 wait complete. 2024-04-25T10:05:14.289 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.385+0000 7f75e5389640 1 Processor -- start 2024-04-25T10:05:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.385+0000 7f75e5389640 1 -- start start 2024-04-25T10:05:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.385+0000 7f75e5389640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f75e015dd20 0x7f75e00b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.385+0000 7f75e5389640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75e015e100 0x7f75e00d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75e5389640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e015eac0 con 0x7f75e015e590 2024-04-25T10:05:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75e5389640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e00d1c00 con 0x7f75e015e100 2024-04-25T10:05:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75e5389640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e015eda0 con 0x7f75e015dd20 2024-04-25T10:05:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75d7fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75e015e100 0x7f75e00d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75df7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f75e015e590 0x7f75e00d3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37424/0 (socket says 172.21.15.19:37424) 2024-04-25T10:05:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75df7fe640 1 -- 172.21.15.19:0/2450556002 learned_addr learned my addr 172.21.15.19:0/2450556002 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75deffd640 1 --2- 172.21.15.19:0/2450556002 >> v2:172.21.15.112:3300/0 conn(0x7f75e015dd20 0x7f75e00b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75de7fc640 1 -- 172.21.15.19:0/2450556002 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410291154 0 0) 0x7f75e015eac0 con 0x7f75e015e590 2024-04-25T10:05:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75de7fc640 1 -- 172.21.15.19:0/2450556002 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75cc003520 con 0x7f75e015e590 2024-04-25T10:05:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75d7fff640 1 -- 172.21.15.19:0/2450556002 >> v2:172.21.15.112:3300/0 conn(0x7f75e015dd20 msgr2=0x7f75e00b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75d7fff640 1 --2- 172.21.15.19:0/2450556002 >> v2:172.21.15.112:3300/0 conn(0x7f75e015dd20 0x7f75e00b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75d7fff640 1 -- 172.21.15.19:0/2450556002 >> v1:172.21.15.19:6789/0 conn(0x7f75e015e590 legacy=0x7f75e00d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.386+0000 7f75d7fff640 1 -- 172.21.15.19:0/2450556002 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e015f080 con 0x7f75e015e100 2024-04-25T10:05:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.387+0000 7f75d7fff640 1 --2- 172.21.15.19:0/2450556002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75e015e100 0x7f75e00d0aa0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f75e00b6520 tx=0x7f75c802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cf90a5b107879e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.387+0000 7f75de7fc640 1 -- 172.21.15.19:0/2450556002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8037070 con 0x7f75e015e100 2024-04-25T10:05:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.387+0000 7f75de7fc640 1 -- 172.21.15.19:0/2450556002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c8032440 con 0x7f75e015e100 2024-04-25T10:05:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.387+0000 7f75de7fc640 1 -- 172.21.15.19:0/2450556002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c802ae60 con 0x7f75e015e100 2024-04-25T10:05:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.387+0000 7f75e5389640 1 -- 172.21.15.19:0/2450556002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75e015e100 msgr2=0x7f75e00d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.387+0000 7f75e5389640 1 --2- 172.21.15.19:0/2450556002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75e015e100 0x7f75e00d0aa0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f75e00b6520 tx=0x7f75c802a110 comp rx=0 tx=0).stop 2024-04-25T10:05:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.387+0000 7f75e5389640 1 -- 172.21.15.19:0/2450556002 shutdown_connections 2024-04-25T10:05:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.387+0000 7f75e5389640 1 --2- 172.21.15.19:0/2450556002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75e015e100 0x7f75e00d0aa0 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.387+0000 7f75e5389640 1 --2- 172.21.15.19:0/2450556002 >> v2:172.21.15.112:3300/0 conn(0x7f75e015dd20 0x7f75e00b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.388+0000 7f75e5389640 1 -- 172.21.15.19:0/2450556002 >> 172.21.15.19:0/2450556002 conn(0x7f75e0061a20 msgr2=0x7f75e0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.388+0000 7f75e5389640 1 -- 172.21.15.19:0/2450556002 shutdown_connections 2024-04-25T10:05:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.388+0000 7f75e5389640 1 -- 172.21.15.19:0/2450556002 wait complete. 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.389+0000 7f75e5389640 1 Processor -- start 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.389+0000 7f75e5389640 1 -- start start 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.389+0000 7f75e5389640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75e015dd20 0x7f75e015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.389+0000 7f75e5389640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e015e100 0x7f75e015f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.389+0000 7f75e5389640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75e015e590 0x7f75e015f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75e5389640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00d3500 con 0x7f75e015dd20 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75e5389640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00d3380 con 0x7f75e015e100 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75e5389640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00d3680 con 0x7f75e015e590 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75e015dd20 0x7f75e015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75e015dd20 0x7f75e015d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38178/0 (socket says 172.21.15.19:38178) 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75deffd640 1 -- 172.21.15.19:0/2392517243 learned_addr learned my addr 172.21.15.19:0/2392517243 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75d7fff640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e015e100 0x7f75e015f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75df7fe640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75e015e590 0x7f75e015f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75deffd640 1 -- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75e015e590 msgr2=0x7f75e015f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75deffd640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75e015e590 0x7f75e015f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75deffd640 1 -- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e015e100 msgr2=0x7f75e015f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75deffd640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e015e100 0x7f75e015f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75deffd640 1 -- 172.21.15.19:0/2392517243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e01f4a20 con 0x7f75e015dd20 2024-04-25T10:05:14.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75d7fff640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e015e100 0x7f75e015f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:14.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75df7fe640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75e015e590 0x7f75e015f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:14.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75deffd640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75e015dd20 0x7f75e015d140 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d830 tx=0x7f75d000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75d77fe640 1 -- 172.21.15.19:0/2392517243 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000b6c0 con 0x7f75e015dd20 2024-04-25T10:05:14.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75d77fe640 1 -- 172.21.15.19:0/2392517243 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0019070 con 0x7f75e015dd20 2024-04-25T10:05:14.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75d77fe640 1 -- 172.21.15.19:0/2392517243 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000ba80 con 0x7f75e015dd20 2024-04-25T10:05:14.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75e5389640 1 -- 172.21.15.19:0/2392517243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e01f4d00 con 0x7f75e015dd20 2024-04-25T10:05:14.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.390+0000 7f75e5389640 1 -- 172.21.15.19:0/2392517243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e01f52c0 con 0x7f75e015dd20 2024-04-25T10:05:14.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.391+0000 7f75e5389640 1 -- 172.21.15.19:0/2392517243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e0160e00 con 0x7f75e015dd20 2024-04-25T10:05:14.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.392+0000 7f75d77fe640 1 -- 172.21.15.19:0/2392517243 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75d001e460 con 0x7f75e015dd20 2024-04-25T10:05:14.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.392+0000 7f75d77fe640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75c006b4d0 0x7f75c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.392+0000 7f75d77fe640 1 -- 172.21.15.19:0/2392517243 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f75d008a9b0 con 0x7f75e015dd20 2024-04-25T10:05:14.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.392+0000 7f75d7fff640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75c006b4d0 0x7f75c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.393+0000 7f75d7fff640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75c006b4d0 0x7f75c006d990 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f75c8034000 tx=0x7f75c802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.396+0000 7f75d77fe640 1 -- 172.21.15.19:0/2392517243 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75e0160e00 con 0x7f75e015dd20 2024-04-25T10:05:14.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.575+0000 7f75e5389640 1 -- 172.21.15.19:0/2392517243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75e0073990 con 0x7f75e015dd20 2024-04-25T10:05:14.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.576+0000 7f75d77fe640 1 -- 172.21.15.19:0/2392517243 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+16953 (secure 0 0 0) 0x7f75d0059730 con 0x7f75e015dd20 2024-04-25T10:05:14.582 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:14.582 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":177,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:13.300075+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:11.578636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:12.274609+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 -- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75c006b4d0 msgr2=0x7f75c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75c006b4d0 0x7f75c006d990 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f75c8034000 tx=0x7f75c802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 -- 172.21.15.19:0/2392517243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75e015dd20 msgr2=0x7f75e015d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75e015dd20 0x7f75e015d140 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d830 tx=0x7f75d000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 -- 172.21.15.19:0/2392517243 shutdown_connections 2024-04-25T10:05:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f75c006b4d0 0x7f75c006d990 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75e015e590 0x7f75e015f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e015e100 0x7f75e015f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 --2- 172.21.15.19:0/2392517243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75e015dd20 0x7f75e015d140 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 -- 172.21.15.19:0/2392517243 >> 172.21.15.19:0/2392517243 conn(0x7f75e0061a20 msgr2=0x7f75e0160420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 -- 172.21.15.19:0/2392517243 shutdown_connections 2024-04-25T10:05:14.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.578+0000 7f75e5389640 1 -- 172.21.15.19:0/2392517243 wait complete. 2024-04-25T10:05:14.595 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.693+0000 7fab33c6b640 1 Processor -- start 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.693+0000 7fab33c6b640 1 -- start start 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.694+0000 7fab33c6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c15dac0 0x7fab2c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.694+0000 7fab33c6b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fab2c0d0ab0 0x7fab2c0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.694+0000 7fab33c6b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c0d2420 con 0x7fab2c0d1f30 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.694+0000 7fab33c6b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c0d3030 con 0x7fab2c0d0ab0 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.694+0000 7fab33c6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c0d2700 con 0x7fab2c15dac0 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.694+0000 7fab32208640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fab2c0d0ab0 0x7fab2c0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.694+0000 7fab31a07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c15dac0 0x7fab2c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.694+0000 7fab32208640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fab2c0d0ab0 0x7fab2c0b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:57390/0 (socket says 172.21.15.19:57390) 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.694+0000 7fab32208640 1 -- 172.21.15.19:0/4142783138 learned_addr learned my addr 172.21.15.19:0/4142783138 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.695+0000 7fab32208640 1 -- 172.21.15.19:0/4142783138 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c15dac0 msgr2=0x7fab2c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.695+0000 7fab32208640 1 --2- 172.21.15.19:0/4142783138 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c15dac0 0x7fab2c0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.695+0000 7fab32208640 1 -- 172.21.15.19:0/4142783138 >> v1:172.21.15.112:6789/0 conn(0x7fab2c0d1f30 legacy=0x7fab2c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.695+0000 7fab32208640 1 -- 172.21.15.19:0/4142783138 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c0d29e0 con 0x7fab2c0d0ab0 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.695+0000 7fab32208640 1 --2- 172.21.15.19:0/4142783138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c0d0ab0 0x7fab2c0b6990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fab24009e50 tx=0x7fab2402a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1325da0de7868757 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.696+0000 7fab30a05640 1 -- 172.21.15.19:0/4142783138 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24036070 con 0x7fab2c0d0ab0 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.696+0000 7fab30a05640 1 -- 172.21.15.19:0/4142783138 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab24031600 con 0x7fab2c0d0ab0 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.696+0000 7fab30a05640 1 -- 172.21.15.19:0/4142783138 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab240319d0 con 0x7fab2c0d0ab0 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.696+0000 7fab33c6b640 1 -- 172.21.15.19:0/4142783138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c0d0ab0 msgr2=0x7fab2c0b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.696+0000 7fab33c6b640 1 --2- 172.21.15.19:0/4142783138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c0d0ab0 0x7fab2c0b6990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fab24009e50 tx=0x7fab2402a7a0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.696+0000 7fab33c6b640 1 -- 172.21.15.19:0/4142783138 shutdown_connections 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.696+0000 7fab33c6b640 1 --2- 172.21.15.19:0/4142783138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c0d0ab0 0x7fab2c0b6990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.696+0000 7fab33c6b640 1 --2- 172.21.15.19:0/4142783138 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c15dac0 0x7fab2c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.696+0000 7fab33c6b640 1 -- 172.21.15.19:0/4142783138 >> 172.21.15.19:0/4142783138 conn(0x7fab2c061c30 msgr2=0x7fab2c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.696+0000 7fab33c6b640 1 -- 172.21.15.19:0/4142783138 shutdown_connections 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.696+0000 7fab33c6b640 1 -- 172.21.15.19:0/4142783138 wait complete. 2024-04-25T10:05:14.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.697+0000 7fab33c6b640 1 Processor -- start 2024-04-25T10:05:14.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.697+0000 7fab33c6b640 1 -- start start 2024-04-25T10:05:14.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.698+0000 7fab33c6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c0d0ab0 0x7fab2c1f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.698+0000 7fab33c6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab2c0d1f30 0x7fab2c1494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.698+0000 7fab33c6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c15dac0 0x7fab2c149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.698+0000 7fab33c6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c0b97d0 con 0x7fab2c0d1f30 2024-04-25T10:05:14.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.698+0000 7fab33c6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c15dea0 con 0x7fab2c0d0ab0 2024-04-25T10:05:14.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.698+0000 7fab33c6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c0b9950 con 0x7fab2c15dac0 2024-04-25T10:05:14.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.698+0000 7fab31206640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab2c0d1f30 0x7fab2c1494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.698+0000 7fab31206640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab2c0d1f30 0x7fab2c1494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38192/0 (socket says 172.21.15.19:38192) 2024-04-25T10:05:14.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.698+0000 7fab31206640 1 -- 172.21.15.19:0/1562934786 learned_addr learned my addr 172.21.15.19:0/1562934786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.698+0000 7fab31a07640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c0d0ab0 0x7fab2c1f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.698+0000 7fab32208640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c15dac0 0x7fab2c149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab31206640 1 -- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c15dac0 msgr2=0x7fab2c149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab31206640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c15dac0 0x7fab2c149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab31206640 1 -- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c0d0ab0 msgr2=0x7fab2c1f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab31206640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c0d0ab0 0x7fab2c1f2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab31206640 1 -- 172.21.15.19:0/1562934786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c0cdfc0 con 0x7fab2c0d1f30 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab32208640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c15dac0 0x7fab2c149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab31206640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab2c0d1f30 0x7fab2c1494e0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c3c0 tx=0x7fab1c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab227fc640 1 -- 172.21.15.19:0/1562934786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0052d0 con 0x7fab2c0d1f30 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab227fc640 1 -- 172.21.15.19:0/1562934786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c005490 con 0x7fab2c0d1f30 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab227fc640 1 -- 172.21.15.19:0/1562934786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c006a40 con 0x7fab2c0d1f30 2024-04-25T10:05:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab33c6b640 1 -- 172.21.15.19:0/1562934786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c0cdce0 con 0x7fab2c0d1f30 2024-04-25T10:05:14.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.699+0000 7fab33c6b640 1 -- 172.21.15.19:0/1562934786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c14a850 con 0x7fab2c0d1f30 2024-04-25T10:05:14.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.700+0000 7fab33c6b640 1 -- 172.21.15.19:0/1562934786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c073a00 con 0x7fab2c0d1f30 2024-04-25T10:05:14.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.701+0000 7fab227fc640 1 -- 172.21.15.19:0/1562934786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab1c003640 con 0x7fab2c0d1f30 2024-04-25T10:05:14.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.701+0000 7fab227fc640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.701+0000 7fab227fc640 1 -- 172.21.15.19:0/1562934786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab1c0897b0 con 0x7fab2c0d1f30 2024-04-25T10:05:14.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.701+0000 7fab31a07640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.702+0000 7fab31a07640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab1006b4d0 0x7fab1006d990 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fab28009aa0 tx=0x7fab28009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.705+0000 7fab227fc640 1 -- 172.21.15.19:0/1562934786 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab1c057e50 con 0x7fab2c0d1f30 2024-04-25T10:05:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.922+0000 7fab33c6b640 1 -- 172.21.15.19:0/1562934786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fab2c0b8b70 con 0x7fab2c0d1f30 2024-04-25T10:05:14.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.945+0000 7fab227fc640 1 -- 172.21.15.19:0/1562934786 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v115) v1 ==== 118+0+0 (secure 0 0 0) 0x7fab1c05bef0 con 0x7fab2c0d1f30 2024-04-25T10:05:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 -- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab1006b4d0 msgr2=0x7fab1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab1006b4d0 0x7fab1006d990 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fab28009aa0 tx=0x7fab28009190 comp rx=0 tx=0).stop 2024-04-25T10:05:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 -- 172.21.15.19:0/1562934786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab2c0d1f30 msgr2=0x7fab2c1494e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab2c0d1f30 0x7fab2c1494e0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c3c0 tx=0x7fab1c00c880 comp rx=0 tx=0).stop 2024-04-25T10:05:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 -- 172.21.15.19:0/1562934786 shutdown_connections 2024-04-25T10:05:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c15dac0 0x7fab2c149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab2c0d1f30 0x7fab2c1494e0 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 --2- 172.21.15.19:0/1562934786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c0d0ab0 0x7fab2c1f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 -- 172.21.15.19:0/1562934786 >> 172.21.15.19:0/1562934786 conn(0x7fab2c061c30 msgr2=0x7fab2c0b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 -- 172.21.15.19:0/1562934786 shutdown_connections 2024-04-25T10:05:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:14.948+0000 7fab33c6b640 1 -- 172.21.15.19:0/1562934786 wait complete. 2024-04-25T10:05:14.965 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:05:14.965 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:05:15.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.064+0000 7f85e3c6b640 1 Processor -- start 2024-04-25T10:05:15.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.064+0000 7f85e3c6b640 1 -- start start 2024-04-25T10:05:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.065+0000 7f85e3c6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85dc0ced80 0x7f85dc0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.065+0000 7f85e3c6b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f85dc161710 0x7f85dc0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.065+0000 7f85e3c6b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85dc15e000 con 0x7f85dc15dc20 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.065+0000 7f85e3c6b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85dc0d1de0 con 0x7f85dc161710 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.065+0000 7f85e3c6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc161c60 con 0x7f85dc0ced80 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.065+0000 7f85e2208640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f85dc161710 0x7f85dc0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.065+0000 7f85e1a07640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f85dc15dc20 0x7f85dc0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37426/0 (socket says 172.21.15.19:37426) 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.065+0000 7f85e1a07640 1 -- 172.21.15.19:0/441245508 learned_addr learned my addr 172.21.15.19:0/441245508 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.065+0000 7f85e1206640 1 --2- 172.21.15.19:0/441245508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85dc0ced80 0x7f85dc0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.066+0000 7f85e2208640 1 -- 172.21.15.19:0/441245508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85dc0ced80 msgr2=0x7f85dc0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.066+0000 7f85e2208640 1 --2- 172.21.15.19:0/441245508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85dc0ced80 0x7f85dc0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.066+0000 7f85e2208640 1 -- 172.21.15.19:0/441245508 >> v1:172.21.15.19:6789/0 conn(0x7f85dc15dc20 legacy=0x7f85dc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.066+0000 7f85e2208640 1 -- 172.21.15.19:0/441245508 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc161f40 con 0x7f85dc161710 2024-04-25T10:05:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.066+0000 7f85e1206640 1 --2- 172.21.15.19:0/441245508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85dc0ced80 0x7f85dc0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.066+0000 7f85e0a05640 1 -- 172.21.15.19:0/441245508 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1088023049 0 0) 0x7f85dc15e000 con 0x7f85dc15dc20 2024-04-25T10:05:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.066+0000 7f85e2208640 1 --2- 172.21.15.19:0/441245508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc161710 0x7f85dc0d2820 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009d90 tx=0x7f85d802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5822397b7d503133 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.067+0000 7f85e0a05640 1 -- 172.21.15.19:0/441245508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8007740 con 0x7f85dc161710 2024-04-25T10:05:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.067+0000 7f85e0a05640 1 -- 172.21.15.19:0/441245508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d80078e0 con 0x7f85dc161710 2024-04-25T10:05:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.067+0000 7f85e0a05640 1 -- 172.21.15.19:0/441245508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8007cb0 con 0x7f85dc161710 2024-04-25T10:05:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.067+0000 7f85e3c6b640 1 -- 172.21.15.19:0/441245508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc161710 msgr2=0x7f85dc0d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.067+0000 7f85e3c6b640 1 --2- 172.21.15.19:0/441245508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc161710 0x7f85dc0d2820 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009d90 tx=0x7f85d802a660 comp rx=0 tx=0).stop 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.067+0000 7f85e3c6b640 1 -- 172.21.15.19:0/441245508 shutdown_connections 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.067+0000 7f85e3c6b640 1 --2- 172.21.15.19:0/441245508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc161710 0x7f85dc0d2820 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.067+0000 7f85e3c6b640 1 --2- 172.21.15.19:0/441245508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85dc0ced80 0x7f85dc0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.067+0000 7f85e3c6b640 1 -- 172.21.15.19:0/441245508 >> 172.21.15.19:0/441245508 conn(0x7f85dc0d0420 msgr2=0x7f85dc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.067+0000 7f85e3c6b640 1 -- 172.21.15.19:0/441245508 shutdown_connections 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.067+0000 7f85e3c6b640 1 -- 172.21.15.19:0/441245508 wait complete. 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.068+0000 7f85e3c6b640 1 Processor -- start 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.068+0000 7f85e3c6b640 1 -- start start 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e3c6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85dc0ced80 0x7f85dc15fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e3c6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85dc15dc20 0x7f85dc1f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e3c6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc161710 0x7f85dc1f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e3c6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0b5e50 con 0x7f85dc161710 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e3c6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc162500 con 0x7f85dc0ced80 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e3c6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0b5fd0 con 0x7f85dc15dc20 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e2208640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc161710 0x7f85dc1f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e2208640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc161710 0x7f85dc1f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38204/0 (socket says 172.21.15.19:38204) 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e2208640 1 -- 172.21.15.19:0/3042086517 learned_addr learned my addr 172.21.15.19:0/3042086517 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e1206640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85dc15dc20 0x7f85dc1f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e1a07640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85dc0ced80 0x7f85dc15fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e2208640 1 -- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85dc15dc20 msgr2=0x7f85dc1f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.069+0000 7f85e2208640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85dc15dc20 0x7f85dc1f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.070+0000 7f85e2208640 1 -- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85dc0ced80 msgr2=0x7f85dc15fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.070+0000 7f85e2208640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85dc0ced80 0x7f85dc15fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.070+0000 7f85e2208640 1 -- 172.21.15.19:0/3042086517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc160540 con 0x7f85dc161710 2024-04-25T10:05:15.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.070+0000 7f85e1a07640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85dc0ced80 0x7f85dc15fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:15.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.070+0000 7f85e2208640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc161710 0x7f85dc1f3950 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009d90 tx=0x7f85d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.070+0000 7f85d27fc640 1 -- 172.21.15.19:0/3042086517 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8040070 con 0x7f85dc161710 2024-04-25T10:05:15.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.070+0000 7f85d27fc640 1 -- 172.21.15.19:0/3042086517 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d8031c40 con 0x7f85dc161710 2024-04-25T10:05:15.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.070+0000 7f85e3c6b640 1 -- 172.21.15.19:0/3042086517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc160820 con 0x7f85dc161710 2024-04-25T10:05:15.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.070+0000 7f85d27fc640 1 -- 172.21.15.19:0/3042086517 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8009450 con 0x7f85dc161710 2024-04-25T10:05:15.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.070+0000 7f85e3c6b640 1 -- 172.21.15.19:0/3042086517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc0cae30 con 0x7f85dc161710 2024-04-25T10:05:15.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.071+0000 7f85e3c6b640 1 -- 172.21.15.19:0/3042086517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85dc0cb030 con 0x7f85dc161710 2024-04-25T10:05:15.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.071+0000 7f85d27fc640 1 -- 172.21.15.19:0/3042086517 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85d8039040 con 0x7f85dc161710 2024-04-25T10:05:15.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.072+0000 7f85d27fc640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c006b4d0 0x7f85c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.072+0000 7f85d27fc640 1 -- 172.21.15.19:0/3042086517 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85d80ab100 con 0x7f85dc161710 2024-04-25T10:05:15.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.072+0000 7f85e1a07640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c006b4d0 0x7f85c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.073+0000 7f85e1a07640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c006b4d0 0x7f85c006d990 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f85d4000f60 tx=0x7f85d4007cc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.076+0000 7f85d27fc640 1 -- 172.21.15.19:0/3042086517 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85d8079e80 con 0x7f85dc161710 2024-04-25T10:05:15.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.293+0000 7f85e3c6b640 1 -- 172.21.15.19:0/3042086517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f85dc0cac20 con 0x7f85dc161710 2024-04-25T10:05:15.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.956+0000 7f85d27fc640 1 -- 172.21.15.19:0/3042086517 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v116) v1 ==== 122+0+0 (secure 0 0 0) 0x7f85d8079780 con 0x7f85dc161710 2024-04-25T10:05:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.958+0000 7f85e3c6b640 1 -- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c006b4d0 msgr2=0x7f85c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.958+0000 7f85e3c6b640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c006b4d0 0x7f85c006d990 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f85d4000f60 tx=0x7f85d4007cc0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.958+0000 7f85e3c6b640 1 -- 172.21.15.19:0/3042086517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc161710 msgr2=0x7f85dc1f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.958+0000 7f85e3c6b640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc161710 0x7f85dc1f3950 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009d90 tx=0x7f85d8004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.958+0000 7f85e3c6b640 1 -- 172.21.15.19:0/3042086517 shutdown_connections 2024-04-25T10:05:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.958+0000 7f85e3c6b640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f85c006b4d0 0x7f85c006d990 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.958+0000 7f85e3c6b640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc161710 0x7f85dc1f3950 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.958+0000 7f85e3c6b640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85dc15dc20 0x7f85dc1f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.958+0000 7f85e3c6b640 1 --2- 172.21.15.19:0/3042086517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85dc0ced80 0x7f85dc15fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.958+0000 7f85e3c6b640 1 -- 172.21.15.19:0/3042086517 >> 172.21.15.19:0/3042086517 conn(0x7f85dc0d0420 msgr2=0x7f85dc0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.959+0000 7f85e3c6b640 1 -- 172.21.15.19:0/3042086517 shutdown_connections 2024-04-25T10:05:15.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:15.959+0000 7f85e3c6b640 1 -- 172.21.15.19:0/3042086517 wait complete. 2024-04-25T10:05:15.978 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:05:15.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.080+0000 7f5b48370640 1 Processor -- start 2024-04-25T10:05:16.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.080+0000 7f5b48370640 1 -- start start 2024-04-25T10:05:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.081+0000 7f5b48370640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b400b2cb0 0x7f5b400d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.081+0000 7f5b48370640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b400d1020 0x7f5b400d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.081+0000 7f5b48370640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b400b97a0 con 0x7f5b4015dbe0 2024-04-25T10:05:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.081+0000 7f5b48370640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4015dfc0 con 0x7f5b400b2cb0 2024-04-25T10:05:16.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.081+0000 7f5b48370640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b400d1a00 con 0x7f5b400d1020 2024-04-25T10:05:16.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.082+0000 7f5b4590b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b400b2cb0 0x7f5b400d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.082+0000 7f5b4610c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5b4015dbe0 0x7f5b400d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37440/0 (socket says 172.21.15.19:37440) 2024-04-25T10:05:16.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.082+0000 7f5b4610c640 1 -- 172.21.15.19:0/3266427869 learned_addr learned my addr 172.21.15.19:0/3266427869 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:16.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.082+0000 7f5b4690d640 1 --2- 172.21.15.19:0/3266427869 >> v2:172.21.15.112:3300/0 conn(0x7f5b400d1020 0x7f5b400d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.082+0000 7f5b4590b640 1 -- 172.21.15.19:0/3266427869 >> v2:172.21.15.112:3300/0 conn(0x7f5b400d1020 msgr2=0x7f5b400d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:16.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.082+0000 7f5b4590b640 1 --2- 172.21.15.19:0/3266427869 >> v2:172.21.15.112:3300/0 conn(0x7f5b400d1020 0x7f5b400d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.082+0000 7f5b4590b640 1 -- 172.21.15.19:0/3266427869 >> v1:172.21.15.19:6789/0 conn(0x7f5b4015dbe0 legacy=0x7f5b400d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.082+0000 7f5b4510a640 1 -- 172.21.15.19:0/3266427869 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968142926 0 0) 0x7f5b400b97a0 con 0x7f5b4015dbe0 2024-04-25T10:05:16.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.082+0000 7f5b4590b640 1 -- 172.21.15.19:0/3266427869 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b400d1c50 con 0x7f5b400b2cb0 2024-04-25T10:05:16.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.082+0000 7f5b4690d640 1 --2- 172.21.15.19:0/3266427869 >> v2:172.21.15.112:3300/0 conn(0x7f5b400d1020 0x7f5b400d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b4590b640 1 --2- 172.21.15.19:0/3266427869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b400b2cb0 0x7f5b400d2730 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f5b400cf570 tx=0x7f5b3002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47300177a4a1e2a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b4510a640 1 -- 172.21.15.19:0/3266427869 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30037070 con 0x7f5b400b2cb0 2024-04-25T10:05:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b4510a640 1 -- 172.21.15.19:0/3266427869 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b30032440 con 0x7f5b400b2cb0 2024-04-25T10:05:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b4510a640 1 -- 172.21.15.19:0/3266427869 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30032810 con 0x7f5b400b2cb0 2024-04-25T10:05:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b48370640 1 -- 172.21.15.19:0/3266427869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b400b2cb0 msgr2=0x7f5b400d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b48370640 1 --2- 172.21.15.19:0/3266427869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b400b2cb0 0x7f5b400d2730 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f5b400cf570 tx=0x7f5b3002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b48370640 1 -- 172.21.15.19:0/3266427869 shutdown_connections 2024-04-25T10:05:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b48370640 1 --2- 172.21.15.19:0/3266427869 >> v2:172.21.15.112:3300/0 conn(0x7f5b400d1020 0x7f5b400d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b48370640 1 --2- 172.21.15.19:0/3266427869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b400b2cb0 0x7f5b400d2730 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b48370640 1 -- 172.21.15.19:0/3266427869 >> 172.21.15.19:0/3266427869 conn(0x7f5b40061a20 msgr2=0x7f5b40061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b48370640 1 -- 172.21.15.19:0/3266427869 shutdown_connections 2024-04-25T10:05:16.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.083+0000 7f5b48370640 1 -- 172.21.15.19:0/3266427869 wait complete. 2024-04-25T10:05:16.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.085+0000 7f5b48370640 1 Processor -- start 2024-04-25T10:05:16.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.085+0000 7f5b48370640 1 -- start start 2024-04-25T10:05:16.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.085+0000 7f5b48370640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b400b2cb0 0x7f5b40148540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.085+0000 7f5b48370640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b400d1020 0x7f5b401f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.085+0000 7f5b48370640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b4015dbe0 0x7f5b400caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.085+0000 7f5b48370640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b400b92a0 con 0x7f5b4015dbe0 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.085+0000 7f5b48370640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b400d2210 con 0x7f5b400b2cb0 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.085+0000 7f5b48370640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b400b9420 con 0x7f5b400d1020 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.085+0000 7f5b4690d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b4015dbe0 0x7f5b400caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.085+0000 7f5b4690d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b4015dbe0 0x7f5b400caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38216/0 (socket says 172.21.15.19:38216) 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.085+0000 7f5b4690d640 1 -- 172.21.15.19:0/4018736793 learned_addr learned my addr 172.21.15.19:0/4018736793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b4590b640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b400d1020 0x7f5b401f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b4610c640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b400b2cb0 0x7f5b40148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b4690d640 1 -- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b400d1020 msgr2=0x7f5b401f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b4690d640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b400d1020 0x7f5b401f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b4690d640 1 -- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b400b2cb0 msgr2=0x7f5b40148540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b4690d640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b400b2cb0 0x7f5b40148540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b4690d640 1 -- 172.21.15.19:0/4018736793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b401f6ab0 con 0x7f5b4015dbe0 2024-04-25T10:05:16.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b4610c640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b400b2cb0 0x7f5b40148540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:16.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b4690d640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b4015dbe0 0x7f5b400caef0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800c7f0 tx=0x7f5b3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b36ffd640 1 -- 172.21.15.19:0/4018736793 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38015070 con 0x7f5b4015dbe0 2024-04-25T10:05:16.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b36ffd640 1 -- 172.21.15.19:0/4018736793 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b38007820 con 0x7f5b4015dbe0 2024-04-25T10:05:16.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b48370640 1 -- 172.21.15.19:0/4018736793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b401f6d80 con 0x7f5b4015dbe0 2024-04-25T10:05:16.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b36ffd640 1 -- 172.21.15.19:0/4018736793 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38007c50 con 0x7f5b4015dbe0 2024-04-25T10:05:16.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.086+0000 7f5b48370640 1 -- 172.21.15.19:0/4018736793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b400ccba0 con 0x7f5b4015dbe0 2024-04-25T10:05:16.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.087+0000 7f5b48370640 1 -- 172.21.15.19:0/4018736793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b40073990 con 0x7f5b4015dbe0 2024-04-25T10:05:16.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.088+0000 7f5b36ffd640 1 -- 172.21.15.19:0/4018736793 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b38018070 con 0x7f5b4015dbe0 2024-04-25T10:05:16.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.088+0000 7f5b36ffd640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b2406b4d0 0x7f5b2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.088+0000 7f5b36ffd640 1 -- 172.21.15.19:0/4018736793 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b3808a430 con 0x7f5b4015dbe0 2024-04-25T10:05:16.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.088+0000 7f5b4610c640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b2406b4d0 0x7f5b2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.089+0000 7f5b4610c640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b2406b4d0 0x7f5b2406d990 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c007fd0 tx=0x7f5b3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.092+0000 7f5b36ffd640 1 -- 172.21.15.19:0/4018736793 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b380591b0 con 0x7f5b4015dbe0 2024-04-25T10:05:16.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.309+0000 7f5b48370640 1 -- 172.21.15.19:0/4018736793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b400ccf80 con 0x7f5b4015dbe0 2024-04-25T10:05:16.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.309+0000 7f5b36ffd640 1 -- 172.21.15.19:0/4018736793 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4948 (secure 0 0 0) 0x7f5b38058ab0 con 0x7f5b4015dbe0 2024-04-25T10:05:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:16.315 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":116,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/662861975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":662861975},{"type":"v1","addr":"172.21.15.19:6835","nonce":662861975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3439611393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3439611393},{"type":"v1","addr":"172.21.15.112:6836","nonce":3439611393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721666508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721666508},{"type":"v1","addr":"172.21.15.112:6837","nonce":721666508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:13.300379+0000","modified":"2024-04-25T10:05:15.294971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11166":{"gid":11166,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6833/829980210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":829980210},{"type":"v1","addr":"172.21.15.19:6833","nonce":829980210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:05:16.315 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 116 2024-04-25T10:05:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.311+0000 7f5b48370640 1 -- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b2406b4d0 msgr2=0x7f5b2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.311+0000 7f5b48370640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b2406b4d0 0x7f5b2406d990 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c007fd0 tx=0x7f5b3c009040 comp rx=0 tx=0).stop 2024-04-25T10:05:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.311+0000 7f5b48370640 1 -- 172.21.15.19:0/4018736793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b4015dbe0 msgr2=0x7f5b400caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.311+0000 7f5b48370640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b4015dbe0 0x7f5b400caef0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800c7f0 tx=0x7f5b3800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.311+0000 7f5b48370640 1 -- 172.21.15.19:0/4018736793 shutdown_connections 2024-04-25T10:05:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.311+0000 7f5b48370640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5b2406b4d0 0x7f5b2406d990 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.311+0000 7f5b48370640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b4015dbe0 0x7f5b400caef0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.311+0000 7f5b48370640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b400d1020 0x7f5b401f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.311+0000 7f5b48370640 1 --2- 172.21.15.19:0/4018736793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b400b2cb0 0x7f5b40148540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.311+0000 7f5b48370640 1 -- 172.21.15.19:0/4018736793 >> 172.21.15.19:0/4018736793 conn(0x7f5b40061a20 msgr2=0x7f5b400b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.311+0000 7f5b48370640 1 -- 172.21.15.19:0/4018736793 shutdown_connections 2024-04-25T10:05:16.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.312+0000 7f5b48370640 1 -- 172.21.15.19:0/4018736793 wait complete. 2024-04-25T10:05:16.328 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:16.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.423+0000 7f473f493640 1 Processor -- start 2024-04-25T10:05:16.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.423+0000 7f473f493640 1 -- start start 2024-04-25T10:05:16.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.424+0000 7f473f493640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f473815dbe0 0x7f47380d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.424+0000 7f473f493640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47380b2cb0 0x7f47380d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.424+0000 7f473f493640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47380b97a0 con 0x7f47380b92a0 2024-04-25T10:05:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.424+0000 7f473f493640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473815dfc0 con 0x7f473815dbe0 2024-04-25T10:05:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.424+0000 7f473f493640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47380d1a00 con 0x7f47380b2cb0 2024-04-25T10:05:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.424+0000 7f473d22f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f473815dbe0 0x7f47380d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.424+0000 7f473da30640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f47380b92a0 0x7f47380b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37448/0 (socket says 172.21.15.19:37448) 2024-04-25T10:05:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.424+0000 7f473da30640 1 -- 172.21.15.19:0/1659732104 learned_addr learned my addr 172.21.15.19:0/1659732104 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.425+0000 7f473ca2e640 1 --2- 172.21.15.19:0/1659732104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47380b2cb0 0x7f47380d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.425+0000 7f473d22f640 1 -- 172.21.15.19:0/1659732104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47380b2cb0 msgr2=0x7f47380d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:16.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.425+0000 7f473d22f640 1 --2- 172.21.15.19:0/1659732104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47380b2cb0 0x7f47380d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.425+0000 7f473d22f640 1 -- 172.21.15.19:0/1659732104 >> v1:172.21.15.19:6789/0 conn(0x7f47380b92a0 legacy=0x7f47380b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.425+0000 7f473d22f640 1 -- 172.21.15.19:0/1659732104 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47380d1c50 con 0x7f473815dbe0 2024-04-25T10:05:16.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.425+0000 7f4737fff640 1 -- 172.21.15.19:0/1659732104 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601808560 0 0) 0x7f47380b97a0 con 0x7f47380b92a0 2024-04-25T10:05:16.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.425+0000 7f473ca2e640 1 --2- 172.21.15.19:0/1659732104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47380b2cb0 0x7f47380d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:16.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.425+0000 7f473d22f640 1 --2- 172.21.15.19:0/1659732104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f473815dbe0 0x7f47380d0910 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f4730009d90 tx=0x7f473002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873d905d18ada826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.426+0000 7f4737fff640 1 -- 172.21.15.19:0/1659732104 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730007740 con 0x7f473815dbe0 2024-04-25T10:05:16.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.426+0000 7f4737fff640 1 -- 172.21.15.19:0/1659732104 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47300078e0 con 0x7f473815dbe0 2024-04-25T10:05:16.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.426+0000 7f4737fff640 1 -- 172.21.15.19:0/1659732104 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730007cb0 con 0x7f473815dbe0 2024-04-25T10:05:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.426+0000 7f473f493640 1 -- 172.21.15.19:0/1659732104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f473815dbe0 msgr2=0x7f47380d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.426+0000 7f473f493640 1 --2- 172.21.15.19:0/1659732104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f473815dbe0 0x7f47380d0910 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f4730009d90 tx=0x7f473002a660 comp rx=0 tx=0).stop 2024-04-25T10:05:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.426+0000 7f473f493640 1 -- 172.21.15.19:0/1659732104 shutdown_connections 2024-04-25T10:05:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.426+0000 7f473f493640 1 --2- 172.21.15.19:0/1659732104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47380b2cb0 0x7f47380d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.426+0000 7f473f493640 1 --2- 172.21.15.19:0/1659732104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f473815dbe0 0x7f47380d0910 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.426+0000 7f473f493640 1 -- 172.21.15.19:0/1659732104 >> 172.21.15.19:0/1659732104 conn(0x7f4738061a20 msgr2=0x7f4738061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.426+0000 7f473f493640 1 -- 172.21.15.19:0/1659732104 shutdown_connections 2024-04-25T10:05:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.426+0000 7f473f493640 1 -- 172.21.15.19:0/1659732104 wait complete. 2024-04-25T10:05:16.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.427+0000 7f473f493640 1 Processor -- start 2024-04-25T10:05:16.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.427+0000 7f473f493640 1 -- start start 2024-04-25T10:05:16.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.428+0000 7f473f493640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47380b2cb0 0x7f473815f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.428+0000 7f473f493640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47380b92a0 0x7f47380cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.428+0000 7f473f493640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f473815dbe0 0x7f4738151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.428+0000 7f473f493640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47380b6ab0 con 0x7f47380b92a0 2024-04-25T10:05:16.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.428+0000 7f473f493640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47380d2210 con 0x7f47380b2cb0 2024-04-25T10:05:16.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.428+0000 7f473f493640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47380b6c30 con 0x7f473815dbe0 2024-04-25T10:05:16.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.428+0000 7f473ca2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47380b92a0 0x7f47380cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.428+0000 7f473ca2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47380b92a0 0x7f47380cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38236/0 (socket says 172.21.15.19:38236) 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.428+0000 7f473ca2e640 1 -- 172.21.15.19:0/3360156415 learned_addr learned my addr 172.21.15.19:0/3360156415 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.428+0000 7f473da30640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f473815dbe0 0x7f4738151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.428+0000 7f473d22f640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47380b2cb0 0x7f473815f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.429+0000 7f473ca2e640 1 -- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f473815dbe0 msgr2=0x7f4738151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.429+0000 7f473ca2e640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f473815dbe0 0x7f4738151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.429+0000 7f473ca2e640 1 -- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47380b2cb0 msgr2=0x7f473815f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.429+0000 7f473ca2e640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47380b2cb0 0x7f473815f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.429+0000 7f473ca2e640 1 -- 172.21.15.19:0/3360156415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4738152420 con 0x7f47380b92a0 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.429+0000 7f473ca2e640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47380b92a0 0x7f47380cac10 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f472c00c7f0 tx=0x7f472c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.429+0000 7f4735ffb640 1 -- 172.21.15.19:0/3360156415 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c015070 con 0x7f47380b92a0 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.429+0000 7f4735ffb640 1 -- 172.21.15.19:0/3360156415 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c007a30 con 0x7f47380b92a0 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.429+0000 7f473f493640 1 -- 172.21.15.19:0/3360156415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473815f830 con 0x7f47380b92a0 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.429+0000 7f4735ffb640 1 -- 172.21.15.19:0/3360156415 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c007e00 con 0x7f47380b92a0 2024-04-25T10:05:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.429+0000 7f473f493640 1 -- 172.21.15.19:0/3360156415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47380b4a50 con 0x7f47380b92a0 2024-04-25T10:05:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.430+0000 7f473f493640 1 -- 172.21.15.19:0/3360156415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4738073990 con 0x7f47380b92a0 2024-04-25T10:05:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.430+0000 7f4735ffb640 1 -- 172.21.15.19:0/3360156415 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f472c018070 con 0x7f47380b92a0 2024-04-25T10:05:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.431+0000 7f4735ffb640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f472406b4d0 0x7f472406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.431+0000 7f4735ffb640 1 -- 172.21.15.19:0/3360156415 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f472c08a250 con 0x7f47380b92a0 2024-04-25T10:05:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.431+0000 7f473d22f640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f472406b4d0 0x7f472406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.432+0000 7f473d22f640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f472406b4d0 0x7f472406d990 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4730034000 tx=0x7f4730033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.435+0000 7f4735ffb640 1 -- 172.21.15.19:0/3360156415 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f472c058fd0 con 0x7f47380b92a0 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.614+0000 7f473f493640 1 -- 172.21.15.19:0/3360156415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4738160f60 con 0x7f47380b92a0 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.616+0000 7f4735ffb640 1 -- 172.21.15.19:0/3360156415 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f472c0588d0 con 0x7f47380b92a0 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":178,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:14.934502+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:11.578636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:12.274609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 -- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f472406b4d0 msgr2=0x7f472406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f472406b4d0 0x7f472406d990 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4730034000 tx=0x7f4730033040 comp rx=0 tx=0).stop 2024-04-25T10:05:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 -- 172.21.15.19:0/3360156415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47380b92a0 msgr2=0x7f47380cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47380b92a0 0x7f47380cac10 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f472c00c7f0 tx=0x7f472c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 -- 172.21.15.19:0/3360156415 shutdown_connections 2024-04-25T10:05:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f472406b4d0 0x7f472406d990 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f473815dbe0 0x7f4738151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47380b92a0 0x7f47380cac10 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 --2- 172.21.15.19:0/3360156415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47380b2cb0 0x7f473815f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 -- 172.21.15.19:0/3360156415 >> 172.21.15.19:0/3360156415 conn(0x7f4738061a20 msgr2=0x7f473815b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 -- 172.21.15.19:0/3360156415 shutdown_connections 2024-04-25T10:05:16.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.618+0000 7f473f493640 1 -- 172.21.15.19:0/3360156415 wait complete. 2024-04-25T10:05:16.634 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:05:16.635 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:16.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.730+0000 7fe406e9e640 1 Processor -- start 2024-04-25T10:05:16.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.730+0000 7fe406e9e640 1 -- start start 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.731+0000 7fe406e9e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe40015dbe0 0x7fe4000d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.731+0000 7fe406e9e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4000b2cb0 0x7fe4000d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.731+0000 7fe406e9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4000b97a0 con 0x7fe4000b92a0 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.731+0000 7fe406e9e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40015dfc0 con 0x7fe4000b2cb0 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.731+0000 7fe406e9e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4000d1a00 con 0x7fe40015dbe0 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.731+0000 7fe3fffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4000b2cb0 0x7fe4000d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.731+0000 7fe3fffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4000b2cb0 0x7fe4000d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38244/0 (socket says 172.21.15.19:38244) 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.731+0000 7fe3fffff640 1 -- 172.21.15.19:0/1054024303 learned_addr learned my addr 172.21.15.19:0/1054024303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.732+0000 7fe404c3a640 1 --2- 172.21.15.19:0/1054024303 >> v2:172.21.15.112:3300/0 conn(0x7fe40015dbe0 0x7fe4000d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.732+0000 7fe3fffff640 1 -- 172.21.15.19:0/1054024303 >> v2:172.21.15.112:3300/0 conn(0x7fe40015dbe0 msgr2=0x7fe4000d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.732+0000 7fe3fffff640 1 --2- 172.21.15.19:0/1054024303 >> v2:172.21.15.112:3300/0 conn(0x7fe40015dbe0 0x7fe4000d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.732+0000 7fe3fffff640 1 -- 172.21.15.19:0/1054024303 >> v1:172.21.15.112:6789/0 conn(0x7fe4000b92a0 legacy=0x7fe4000b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:16.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.732+0000 7fe3fffff640 1 -- 172.21.15.19:0/1054024303 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4000d1c50 con 0x7fe4000b2cb0 2024-04-25T10:05:16.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.732+0000 7fe3fffff640 1 --2- 172.21.15.19:0/1054024303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4000b2cb0 0x7fe4000d0e50 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fe40006c140 tx=0x7fe3f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f00bbd3967d0fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.732+0000 7fe3ff7fe640 1 -- 172.21.15.19:0/1054024303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4037070 con 0x7fe4000b2cb0 2024-04-25T10:05:16.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.733+0000 7fe3ff7fe640 1 -- 172.21.15.19:0/1054024303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f4032440 con 0x7fe4000b2cb0 2024-04-25T10:05:16.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.733+0000 7fe3ff7fe640 1 -- 172.21.15.19:0/1054024303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4032810 con 0x7fe4000b2cb0 2024-04-25T10:05:16.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.733+0000 7fe406e9e640 1 -- 172.21.15.19:0/1054024303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4000b2cb0 msgr2=0x7fe4000d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.733+0000 7fe406e9e640 1 --2- 172.21.15.19:0/1054024303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4000b2cb0 0x7fe4000d0e50 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fe40006c140 tx=0x7fe3f402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.733+0000 7fe406e9e640 1 -- 172.21.15.19:0/1054024303 shutdown_connections 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.733+0000 7fe406e9e640 1 --2- 172.21.15.19:0/1054024303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4000b2cb0 0x7fe4000d0e50 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.733+0000 7fe406e9e640 1 --2- 172.21.15.19:0/1054024303 >> v2:172.21.15.112:3300/0 conn(0x7fe40015dbe0 0x7fe4000d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.733+0000 7fe406e9e640 1 -- 172.21.15.19:0/1054024303 >> 172.21.15.19:0/1054024303 conn(0x7fe400061a20 msgr2=0x7fe400061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.733+0000 7fe406e9e640 1 -- 172.21.15.19:0/1054024303 shutdown_connections 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.733+0000 7fe406e9e640 1 -- 172.21.15.19:0/1054024303 wait complete. 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.734+0000 7fe406e9e640 1 Processor -- start 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.734+0000 7fe406e9e640 1 -- start start 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.735+0000 7fe406e9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4000b2cb0 0x7fe4000c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.735+0000 7fe406e9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4000b92a0 0x7fe4000b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.735+0000 7fe406e9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe40015dbe0 0x7fe4000b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.735+0000 7fe406e9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b6ab0 con 0x7fe40015dbe0 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.735+0000 7fe406e9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000d2210 con 0x7fe4000b2cb0 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.735+0000 7fe406e9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b6c30 con 0x7fe4000b92a0 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.735+0000 7fe40543b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe40015dbe0 0x7fe4000b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.735+0000 7fe40543b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe40015dbe0 0x7fe4000b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38258/0 (socket says 172.21.15.19:38258) 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.735+0000 7fe40543b640 1 -- 172.21.15.19:0/2773869665 learned_addr learned my addr 172.21.15.19:0/2773869665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe3fffff640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4000b92a0 0x7fe4000b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe404c3a640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4000b2cb0 0x7fe4000c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe40543b640 1 -- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4000b92a0 msgr2=0x7fe4000b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe40543b640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4000b92a0 0x7fe4000b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe40543b640 1 -- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4000b2cb0 msgr2=0x7fe4000c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe40543b640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4000b2cb0 0x7fe4000c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe40543b640 1 -- 172.21.15.19:0/2773869665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4000b45f0 con 0x7fe40015dbe0 2024-04-25T10:05:16.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe40543b640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe40015dbe0 0x7fe4000b51e0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c730 tx=0x7fe3f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe3fd7fa640 1 -- 172.21.15.19:0/2773869665 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8016020 con 0x7fe40015dbe0 2024-04-25T10:05:16.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe3fd7fa640 1 -- 172.21.15.19:0/2773869665 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f800aa50 con 0x7fe40015dbe0 2024-04-25T10:05:16.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe406e9e640 1 -- 172.21.15.19:0/2773869665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4000b48d0 con 0x7fe40015dbe0 2024-04-25T10:05:16.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe3fd7fa640 1 -- 172.21.15.19:0/2773869665 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800ce40 con 0x7fe40015dbe0 2024-04-25T10:05:16.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.736+0000 7fe406e9e640 1 -- 172.21.15.19:0/2773869665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40015e680 con 0x7fe40015dbe0 2024-04-25T10:05:16.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.737+0000 7fe406e9e640 1 -- 172.21.15.19:0/2773869665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe400073990 con 0x7fe40015dbe0 2024-04-25T10:05:16.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.738+0000 7fe3fd7fa640 1 -- 172.21.15.19:0/2773869665 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3f8019020 con 0x7fe40015dbe0 2024-04-25T10:05:16.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.738+0000 7fe3fd7fa640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe3ec06b500 0x7fe3ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.738+0000 7fe3fd7fa640 1 -- 172.21.15.19:0/2773869665 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe3f808a2f0 con 0x7fe40015dbe0 2024-04-25T10:05:16.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.738+0000 7fe404c3a640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe3ec06b500 0x7fe3ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.739+0000 7fe404c3a640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe3ec06b500 0x7fe3ec06d9c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0006fd0 tx=0x7fe3f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.742+0000 7fe3fd7fa640 1 -- 172.21.15.19:0/2773869665 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3f8020030 con 0x7fe40015dbe0 2024-04-25T10:05:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.920+0000 7fe406e9e640 1 -- 172.21.15.19:0/2773869665 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe40015ba10 con 0x7fe3ec06b500 2024-04-25T10:05:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.927+0000 7fe3fd7fa640 1 -- 172.21.15.19:0/2773869665 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252421 (secure 0 0 0) 0x7fe40015ba10 con 0x7fe3ec06b500 2024-04-25T10:05:16.935 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:16.936 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:05:16.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 -- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe3ec06b500 msgr2=0x7fe3ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe3ec06b500 0x7fe3ec06d9c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0006fd0 tx=0x7fe3f0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:16.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 -- 172.21.15.19:0/2773869665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe40015dbe0 msgr2=0x7fe4000b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe40015dbe0 0x7fe4000b51e0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c730 tx=0x7fe3f800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 -- 172.21.15.19:0/2773869665 shutdown_connections 2024-04-25T10:05:16.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe3ec06b500 0x7fe3ec06d9c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe40015dbe0 0x7fe4000b51e0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4000b92a0 0x7fe4000b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 --2- 172.21.15.19:0/2773869665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4000b2cb0 0x7fe4000c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 -- 172.21.15.19:0/2773869665 >> 172.21.15.19:0/2773869665 conn(0x7fe400061a20 msgr2=0x7fe40015ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 -- 172.21.15.19:0/2773869665 shutdown_connections 2024-04-25T10:05:16.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:16.930+0000 7fe406e9e640 1 -- 172.21.15.19:0/2773869665 wait complete. 2024-04-25T10:05:16.946 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":542,"stamp":"2024-04-25T10:05:15.082587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":441504,"kb_used_data":18536,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754533216,"statfs":{"total":773094113280,"available":772642013184,"internally_reserved":0,"allocated":18980864,"data_stored":9523912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000705"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.961986+0000","last_change":"2024-04-25T10:05:13.960275+0000","last_active":"2024-04-25T10:05:13.961986+0000","last_peered":"2024-04-25T10:05:13.961986+0000","last_clean":"2024-04-25T10:05:13.961986+0000","last_became_active":"2024-04-25T10:05:13.959940+0000","last_became_peered":"2024-04-25T10:05:13.959940+0000","last_unstale":"2024-04-25T10:05:13.961986+0000","last_undegraded":"2024-04-25T10:05:13.961986+0000","last_fullsized":"2024-04-25T10:05:13.961986+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:18.404329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960031+0000","last_change":"2024-04-25T10:05:13.950499+0000","last_active":"2024-04-25T10:05:13.960031+0000","last_peered":"2024-04-25T10:05:13.960031+0000","last_clean":"2024-04-25T10:05:13.960031+0000","last_became_active":"2024-04-25T10:05:13.950095+0000","last_became_peered":"2024-04-25T10:05:13.950095+0000","last_unstale":"2024-04-25T10:05:13.960031+0000","last_undegraded":"2024-04-25T10:05:13.960031+0000","last_fullsized":"2024-04-25T10:05:13.960031+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:20.914289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.952965+0000","last_change":"2024-04-25T10:05:13.951403+0000","last_active":"2024-04-25T10:05:13.952965+0000","last_peered":"2024-04-25T10:05:13.952965+0000","last_clean":"2024-04-25T10:05:13.952965+0000","last_became_active":"2024-04-25T10:05:13.951112+0000","last_became_peered":"2024-04-25T10:05:13.951112+0000","last_unstale":"2024-04-25T10:05:13.952965+0000","last_undegraded":"2024-04-25T10:05:13.952965+0000","last_fullsized":"2024-04-25T10:05:13.952965+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:06.660471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.975909+0000","last_change":"2024-04-25T10:05:13.968980+0000","last_active":"2024-04-25T10:05:13.975909+0000","last_peered":"2024-04-25T10:05:13.975909+0000","last_clean":"2024-04-25T10:05:13.975909+0000","last_became_active":"2024-04-25T10:05:13.968704+0000","last_became_peered":"2024-04-25T10:05:13.968704+0000","last_unstale":"2024-04-25T10:05:13.975909+0000","last_undegraded":"2024-04-25T10:05:13.975909+0000","last_fullsized":"2024-04-25T10:05:13.975909+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.121744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.954162+0000","last_change":"2024-04-25T10:05:13.946171+0000","last_active":"2024-04-25T10:05:13.954162+0000","last_peered":"2024-04-25T10:05:13.954162+0000","last_clean":"2024-04-25T10:05:13.954162+0000","last_became_active":"2024-04-25T10:05:13.945873+0000","last_became_peered":"2024-04-25T10:05:13.945873+0000","last_unstale":"2024-04-25T10:05:13.954162+0000","last_undegraded":"2024-04-25T10:05:13.954162+0000","last_fullsized":"2024-04-25T10:05:13.954162+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:43.005765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.953553+0000","last_change":"2024-04-25T10:05:13.950200+0000","last_active":"2024-04-25T10:05:13.953553+0000","last_peered":"2024-04-25T10:05:13.953553+0000","last_clean":"2024-04-25T10:05:13.953553+0000","last_became_active":"2024-04-25T10:05:13.949883+0000","last_became_peered":"2024-04-25T10:05:13.949883+0000","last_unstale":"2024-04-25T10:05:13.953553+0000","last_undegraded":"2024-04-25T10:05:13.953553+0000","last_fullsized":"2024-04-25T10:05:13.953553+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.564661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.945443+0000","last_change":"2024-04-25T10:05:13.941215+0000","last_active":"2024-04-25T10:05:13.945443+0000","last_peered":"2024-04-25T10:05:13.945443+0000","last_clean":"2024-04-25T10:05:13.945443+0000","last_became_active":"2024-04-25T10:05:13.940832+0000","last_became_peered":"2024-04-25T10:05:13.940832+0000","last_unstale":"2024-04-25T10:05:13.945443+0000","last_undegraded":"2024-04-25T10:05:13.945443+0000","last_fullsized":"2024-04-25T10:05:13.945443+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:10.110857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962486+0000","last_change":"2024-04-25T10:05:13.955816+0000","last_active":"2024-04-25T10:05:13.962486+0000","last_peered":"2024-04-25T10:05:13.962486+0000","last_clean":"2024-04-25T10:05:13.962486+0000","last_became_active":"2024-04-25T10:05:13.955379+0000","last_became_peered":"2024-04-25T10:05:13.955379+0000","last_unstale":"2024-04-25T10:05:13.962486+0000","last_undegraded":"2024-04-25T10:05:13.962486+0000","last_fullsized":"2024-04-25T10:05:13.962486+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:31.462742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.975249+0000","last_change":"2024-04-25T10:05:13.974885+0000","last_active":"2024-04-25T10:05:13.975249+0000","last_peered":"2024-04-25T10:05:13.975249+0000","last_clean":"2024-04-25T10:05:13.975249+0000","last_became_active":"2024-04-25T10:05:13.974626+0000","last_became_peered":"2024-04-25T10:05:13.974626+0000","last_unstale":"2024-04-25T10:05:13.975249+0000","last_undegraded":"2024-04-25T10:05:13.975249+0000","last_fullsized":"2024-04-25T10:05:13.975249+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:19.706548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.961701+0000","last_change":"2024-04-25T10:05:13.956287+0000","last_active":"2024-04-25T10:05:13.961701+0000","last_peered":"2024-04-25T10:05:13.961701+0000","last_clean":"2024-04-25T10:05:13.961701+0000","last_became_active":"2024-04-25T10:05:13.956006+0000","last_became_peered":"2024-04-25T10:05:13.956006+0000","last_unstale":"2024-04-25T10:05:13.961701+0000","last_undegraded":"2024-04-25T10:05:13.961701+0000","last_fullsized":"2024-04-25T10:05:13.961701+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:21.605197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966654+0000","last_change":"2024-04-25T10:05:13.965752+0000","last_active":"2024-04-25T10:05:13.966654+0000","last_peered":"2024-04-25T10:05:13.966654+0000","last_clean":"2024-04-25T10:05:13.966654+0000","last_became_active":"2024-04-25T10:05:13.965460+0000","last_became_peered":"2024-04-25T10:05:13.965460+0000","last_unstale":"2024-04-25T10:05:13.966654+0000","last_undegraded":"2024-04-25T10:05:13.966654+0000","last_fullsized":"2024-04-25T10:05:13.966654+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:56.713012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.968729+0000","last_change":"2024-04-25T10:05:13.967277+0000","last_active":"2024-04-25T10:05:13.968729+0000","last_peered":"2024-04-25T10:05:13.968729+0000","last_clean":"2024-04-25T10:05:13.968729+0000","last_became_active":"2024-04-25T10:05:13.967001+0000","last_became_peered":"2024-04-25T10:05:13.967001+0000","last_unstale":"2024-04-25T10:05:13.968729+0000","last_undegraded":"2024-04-25T10:05:13.968729+0000","last_fullsized":"2024-04-25T10:05:13.968729+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:04.351882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.949431+0000","last_change":"2024-04-25T10:05:13.948369+0000","last_active":"2024-04-25T10:05:13.949431+0000","last_peered":"2024-04-25T10:05:13.949431+0000","last_clean":"2024-04-25T10:05:13.949431+0000","last_became_active":"2024-04-25T10:05:13.948066+0000","last_became_peered":"2024-04-25T10:05:13.948066+0000","last_unstale":"2024-04-25T10:05:13.949431+0000","last_undegraded":"2024-04-25T10:05:13.949431+0000","last_fullsized":"2024-04-25T10:05:13.949431+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:11.317829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.963946+0000","last_change":"2024-04-25T10:05:13.961114+0000","last_active":"2024-04-25T10:05:13.963946+0000","last_peered":"2024-04-25T10:05:13.963946+0000","last_clean":"2024-04-25T10:05:13.963946+0000","last_became_active":"2024-04-25T10:05:13.959327+0000","last_became_peered":"2024-04-25T10:05:13.959327+0000","last_unstale":"2024-04-25T10:05:13.963946+0000","last_undegraded":"2024-04-25T10:05:13.963946+0000","last_fullsized":"2024-04-25T10:05:13.963946+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:40.334224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.957085+0000","last_change":"2024-04-25T10:05:13.955462+0000","last_active":"2024-04-25T10:05:13.957085+0000","last_peered":"2024-04-25T10:05:13.957085+0000","last_clean":"2024-04-25T10:05:13.957085+0000","last_became_active":"2024-04-25T10:05:13.955177+0000","last_became_peered":"2024-04-25T10:05:13.955177+0000","last_unstale":"2024-04-25T10:05:13.957085+0000","last_undegraded":"2024-04-25T10:05:13.957085+0000","last_fullsized":"2024-04-25T10:05:13.957085+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:14.413431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.952238+0000","last_change":"2024-04-25T10:05:13.950588+0000","last_active":"2024-04-25T10:05:13.952238+0000","last_peered":"2024-04-25T10:05:13.952238+0000","last_clean":"2024-04-25T10:05:13.952238+0000","last_became_active":"2024-04-25T10:05:13.950293+0000","last_became_peered":"2024-04-25T10:05:13.950293+0000","last_unstale":"2024-04-25T10:05:13.952238+0000","last_undegraded":"2024-04-25T10:05:13.952238+0000","last_fullsized":"2024-04-25T10:05:13.952238+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:35.796402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.967850+0000","last_change":"2024-04-25T10:05:13.966966+0000","last_active":"2024-04-25T10:05:13.967850+0000","last_peered":"2024-04-25T10:05:13.967850+0000","last_clean":"2024-04-25T10:05:13.967850+0000","last_became_active":"2024-04-25T10:05:13.966346+0000","last_became_peered":"2024-04-25T10:05:13.966346+0000","last_unstale":"2024-04-25T10:05:13.967850+0000","last_undegraded":"2024-04-25T10:05:13.967850+0000","last_fullsized":"2024-04-25T10:05:13.967850+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:35.065296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.970988+0000","last_change":"2024-04-25T10:05:13.970408+0000","last_active":"2024-04-25T10:05:13.970988+0000","last_peered":"2024-04-25T10:05:13.970988+0000","last_clean":"2024-04-25T10:05:13.970988+0000","last_became_active":"2024-04-25T10:05:13.970020+0000","last_became_peered":"2024-04-25T10:05:13.970020+0000","last_unstale":"2024-04-25T10:05:13.970988+0000","last_undegraded":"2024-04-25T10:05:13.970988+0000","last_fullsized":"2024-04-25T10:05:13.970988+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:29.729358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966373+0000","last_change":"2024-04-25T10:05:13.965944+0000","last_active":"2024-04-25T10:05:13.966373+0000","last_peered":"2024-04-25T10:05:13.966373+0000","last_clean":"2024-04-25T10:05:13.966373+0000","last_became_active":"2024-04-25T10:05:13.965616+0000","last_became_peered":"2024-04-25T10:05:13.965616+0000","last_unstale":"2024-04-25T10:05:13.966373+0000","last_undegraded":"2024-04-25T10:05:13.966373+0000","last_fullsized":"2024-04-25T10:05:13.966373+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:44.331097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964143+0000","last_change":"2024-04-25T10:05:13.959469+0000","last_active":"2024-04-25T10:05:13.964143+0000","last_peered":"2024-04-25T10:05:13.964143+0000","last_clean":"2024-04-25T10:05:13.964143+0000","last_became_active":"2024-04-25T10:05:13.959147+0000","last_became_peered":"2024-04-25T10:05:13.959147+0000","last_unstale":"2024-04-25T10:05:13.964143+0000","last_undegraded":"2024-04-25T10:05:13.964143+0000","last_fullsized":"2024-04-25T10:05:13.964143+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:11.501488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964876+0000","last_change":"2024-04-25T10:05:13.963537+0000","last_active":"2024-04-25T10:05:13.964876+0000","last_peered":"2024-04-25T10:05:13.964876+0000","last_clean":"2024-04-25T10:05:13.964876+0000","last_became_active":"2024-04-25T10:05:13.963270+0000","last_became_peered":"2024-04-25T10:05:13.963270+0000","last_unstale":"2024-04-25T10:05:13.964876+0000","last_undegraded":"2024-04-25T10:05:13.964876+0000","last_fullsized":"2024-04-25T10:05:13.964876+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:25.242024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964660+0000","last_change":"2024-04-25T10:05:13.959371+0000","last_active":"2024-04-25T10:05:13.964660+0000","last_peered":"2024-04-25T10:05:13.964660+0000","last_clean":"2024-04-25T10:05:13.964660+0000","last_became_active":"2024-04-25T10:05:13.959091+0000","last_became_peered":"2024-04-25T10:05:13.959091+0000","last_unstale":"2024-04-25T10:05:13.964660+0000","last_undegraded":"2024-04-25T10:05:13.964660+0000","last_fullsized":"2024-04-25T10:05:13.964660+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:43.687988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960875+0000","last_change":"2024-04-25T10:05:13.958796+0000","last_active":"2024-04-25T10:05:13.960875+0000","last_peered":"2024-04-25T10:05:13.960875+0000","last_clean":"2024-04-25T10:05:13.960875+0000","last_became_active":"2024-04-25T10:05:13.958357+0000","last_became_peered":"2024-04-25T10:05:13.958357+0000","last_unstale":"2024-04-25T10:05:13.960875+0000","last_undegraded":"2024-04-25T10:05:13.960875+0000","last_fullsized":"2024-04-25T10:05:13.960875+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:24.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.956384+0000","last_change":"2024-04-25T10:05:13.950809+0000","last_active":"2024-04-25T10:05:13.956384+0000","last_peered":"2024-04-25T10:05:13.956384+0000","last_clean":"2024-04-25T10:05:13.956384+0000","last_became_active":"2024-04-25T10:05:13.950541+0000","last_became_peered":"2024-04-25T10:05:13.950541+0000","last_unstale":"2024-04-25T10:05:13.956384+0000","last_undegraded":"2024-04-25T10:05:13.956384+0000","last_fullsized":"2024-04-25T10:05:13.956384+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:43.563858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.970513+0000","last_change":"2024-04-25T10:05:13.969018+0000","last_active":"2024-04-25T10:05:13.970513+0000","last_peered":"2024-04-25T10:05:13.970513+0000","last_clean":"2024-04-25T10:05:13.970513+0000","last_became_active":"2024-04-25T10:05:13.968630+0000","last_became_peered":"2024-04-25T10:05:13.968630+0000","last_unstale":"2024-04-25T10:05:13.970513+0000","last_undegraded":"2024-04-25T10:05:13.970513+0000","last_fullsized":"2024-04-25T10:05:13.970513+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:40.769585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.965600+0000","last_change":"2024-04-25T10:05:13.964653+0000","last_active":"2024-04-25T10:05:13.965600+0000","last_peered":"2024-04-25T10:05:13.965600+0000","last_clean":"2024-04-25T10:05:13.965600+0000","last_became_active":"2024-04-25T10:05:13.964225+0000","last_became_peered":"2024-04-25T10:05:13.964225+0000","last_unstale":"2024-04-25T10:05:13.965600+0000","last_undegraded":"2024-04-25T10:05:13.965600+0000","last_fullsized":"2024-04-25T10:05:13.965600+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:05.254847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.956362+0000","last_change":"2024-04-25T10:05:13.955939+0000","last_active":"2024-04-25T10:05:13.956362+0000","last_peered":"2024-04-25T10:05:13.956362+0000","last_clean":"2024-04-25T10:05:13.956362+0000","last_became_active":"2024-04-25T10:05:13.955628+0000","last_became_peered":"2024-04-25T10:05:13.955628+0000","last_unstale":"2024-04-25T10:05:13.956362+0000","last_undegraded":"2024-04-25T10:05:13.956362+0000","last_fullsized":"2024-04-25T10:05:13.956362+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:43.843086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.953619+0000","last_change":"2024-04-25T10:05:13.943450+0000","last_active":"2024-04-25T10:05:13.953619+0000","last_peered":"2024-04-25T10:05:13.953619+0000","last_clean":"2024-04-25T10:05:13.953619+0000","last_became_active":"2024-04-25T10:05:13.943181+0000","last_became_peered":"2024-04-25T10:05:13.943181+0000","last_unstale":"2024-04-25T10:05:13.953619+0000","last_undegraded":"2024-04-25T10:05:13.953619+0000","last_fullsized":"2024-04-25T10:05:13.953619+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:08.330613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.967004+0000","last_change":"2024-04-25T10:05:13.965852+0000","last_active":"2024-04-25T10:05:13.967004+0000","last_peered":"2024-04-25T10:05:13.967004+0000","last_clean":"2024-04-25T10:05:13.967004+0000","last_became_active":"2024-04-25T10:05:13.965595+0000","last_became_peered":"2024-04-25T10:05:13.965595+0000","last_unstale":"2024-04-25T10:05:13.967004+0000","last_undegraded":"2024-04-25T10:05:13.967004+0000","last_fullsized":"2024-04-25T10:05:13.967004+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:19.674779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.951424+0000","last_change":"2024-04-25T10:05:13.948527+0000","last_active":"2024-04-25T10:05:13.951424+0000","last_peered":"2024-04-25T10:05:13.951424+0000","last_clean":"2024-04-25T10:05:13.951424+0000","last_became_active":"2024-04-25T10:05:13.947332+0000","last_became_peered":"2024-04-25T10:05:13.947332+0000","last_unstale":"2024-04-25T10:05:13.951424+0000","last_undegraded":"2024-04-25T10:05:13.951424+0000","last_fullsized":"2024-04-25T10:05:13.951424+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:44.331097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.953582+0000","last_change":"2024-04-25T10:05:13.951937+0000","last_active":"2024-04-25T10:05:13.953582+0000","last_peered":"2024-04-25T10:05:13.953582+0000","last_clean":"2024-04-25T10:05:13.953582+0000","last_became_active":"2024-04-25T10:05:13.951122+0000","last_became_peered":"2024-04-25T10:05:13.951122+0000","last_unstale":"2024-04-25T10:05:13.953582+0000","last_undegraded":"2024-04-25T10:05:13.953582+0000","last_fullsized":"2024-04-25T10:05:13.953582+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:35.796402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.946711+0000","last_change":"2024-04-25T10:05:13.945709+0000","last_active":"2024-04-25T10:05:13.946711+0000","last_peered":"2024-04-25T10:05:13.946711+0000","last_clean":"2024-04-25T10:05:13.946711+0000","last_became_active":"2024-04-25T10:05:13.945323+0000","last_became_peered":"2024-04-25T10:05:13.945323+0000","last_unstale":"2024-04-25T10:05:13.946711+0000","last_undegraded":"2024-04-25T10:05:13.946711+0000","last_fullsized":"2024-04-25T10:05:13.946711+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:09.118377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960921+0000","last_change":"2024-04-25T10:05:13.958986+0000","last_active":"2024-04-25T10:05:13.960921+0000","last_peered":"2024-04-25T10:05:13.960921+0000","last_clean":"2024-04-25T10:05:13.960921+0000","last_became_active":"2024-04-25T10:05:13.958261+0000","last_became_peered":"2024-04-25T10:05:13.958261+0000","last_unstale":"2024-04-25T10:05:13.960921+0000","last_undegraded":"2024-04-25T10:05:13.960921+0000","last_fullsized":"2024-04-25T10:05:13.960921+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:50.517981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"177'4","reported_seq":16,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.965096+0000","last_change":"2024-04-25T10:05:12.955599+0000","last_active":"2024-04-25T10:05:13.965096+0000","last_peered":"2024-04-25T10:05:13.965096+0000","last_clean":"2024-04-25T10:05:13.965096+0000","last_became_active":"2024-04-25T10:05:12.954861+0000","last_became_peered":"2024-04-25T10:05:12.954861+0000","last_unstale":"2024-04-25T10:05:13.965096+0000","last_undegraded":"2024-04-25T10:05:13.965096+0000","last_fullsized":"2024-04-25T10:05:13.965096+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:31.018350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.950678+0000","last_change":"2024-04-25T10:05:13.950029+0000","last_active":"2024-04-25T10:05:13.950678+0000","last_peered":"2024-04-25T10:05:13.950678+0000","last_clean":"2024-04-25T10:05:13.950678+0000","last_became_active":"2024-04-25T10:05:13.949756+0000","last_became_peered":"2024-04-25T10:05:13.949756+0000","last_unstale":"2024-04-25T10:05:13.950678+0000","last_undegraded":"2024-04-25T10:05:13.950678+0000","last_fullsized":"2024-04-25T10:05:13.950678+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:14.745138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"177'1","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.963071+0000","last_change":"2024-04-25T10:05:12.951502+0000","last_active":"2024-04-25T10:05:13.963071+0000","last_peered":"2024-04-25T10:05:13.963071+0000","last_clean":"2024-04-25T10:05:13.963071+0000","last_became_active":"2024-04-25T10:05:12.950400+0000","last_became_peered":"2024-04-25T10:05:12.950400+0000","last_unstale":"2024-04-25T10:05:13.963071+0000","last_undegraded":"2024-04-25T10:05:13.963071+0000","last_fullsized":"2024-04-25T10:05:13.963071+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:45.830773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964633+0000","last_change":"2024-04-25T10:05:13.961715+0000","last_active":"2024-04-25T10:05:13.964633+0000","last_peered":"2024-04-25T10:05:13.964633+0000","last_clean":"2024-04-25T10:05:13.964633+0000","last_became_active":"2024-04-25T10:05:13.961434+0000","last_became_peered":"2024-04-25T10:05:13.961434+0000","last_unstale":"2024-04-25T10:05:13.964633+0000","last_undegraded":"2024-04-25T10:05:13.964633+0000","last_fullsized":"2024-04-25T10:05:13.964633+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:32.025451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960484+0000","last_change":"2024-04-25T10:05:12.950304+0000","last_active":"2024-04-25T10:05:13.960484+0000","last_peered":"2024-04-25T10:05:13.960484+0000","last_clean":"2024-04-25T10:05:13.960484+0000","last_became_active":"2024-04-25T10:05:12.950022+0000","last_became_peered":"2024-04-25T10:05:12.950022+0000","last_unstale":"2024-04-25T10:05:13.960484+0000","last_undegraded":"2024-04-25T10:05:13.960484+0000","last_fullsized":"2024-04-25T10:05:13.960484+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:18.449777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962676+0000","last_change":"2024-04-25T10:05:13.953992+0000","last_active":"2024-04-25T10:05:13.962676+0000","last_peered":"2024-04-25T10:05:13.962676+0000","last_clean":"2024-04-25T10:05:13.962676+0000","last_became_active":"2024-04-25T10:05:13.953684+0000","last_became_peered":"2024-04-25T10:05:13.953684+0000","last_unstale":"2024-04-25T10:05:13.962676+0000","last_undegraded":"2024-04-25T10:05:13.962676+0000","last_fullsized":"2024-04-25T10:05:13.962676+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:57.379182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.976265+0000","last_change":"2024-04-25T10:05:12.956302+0000","last_active":"2024-04-25T10:05:13.976265+0000","last_peered":"2024-04-25T10:05:13.976265+0000","last_clean":"2024-04-25T10:05:13.976265+0000","last_became_active":"2024-04-25T10:05:12.955985+0000","last_became_peered":"2024-04-25T10:05:12.955985+0000","last_unstale":"2024-04-25T10:05:13.976265+0000","last_undegraded":"2024-04-25T10:05:13.976265+0000","last_fullsized":"2024-04-25T10:05:13.976265+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:19.774500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.955156+0000","last_change":"2024-04-25T10:05:13.952110+0000","last_active":"2024-04-25T10:05:13.955156+0000","last_peered":"2024-04-25T10:05:13.955156+0000","last_clean":"2024-04-25T10:05:13.955156+0000","last_became_active":"2024-04-25T10:05:13.951771+0000","last_became_peered":"2024-04-25T10:05:13.951771+0000","last_unstale":"2024-04-25T10:05:13.955156+0000","last_undegraded":"2024-04-25T10:05:13.955156+0000","last_fullsized":"2024-04-25T10:05:13.955156+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:53.478576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"177'1","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962018+0000","last_change":"2024-04-25T10:05:12.953881+0000","last_active":"2024-04-25T10:05:13.962018+0000","last_peered":"2024-04-25T10:05:13.962018+0000","last_clean":"2024-04-25T10:05:13.962018+0000","last_became_active":"2024-04-25T10:05:12.953463+0000","last_became_peered":"2024-04-25T10:05:12.953463+0000","last_unstale":"2024-04-25T10:05:13.962018+0000","last_undegraded":"2024-04-25T10:05:13.962018+0000","last_fullsized":"2024-04-25T10:05:13.962018+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:05.737767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.951308+0000","last_change":"2024-04-25T10:05:13.949897+0000","last_active":"2024-04-25T10:05:13.951308+0000","last_peered":"2024-04-25T10:05:13.951308+0000","last_clean":"2024-04-25T10:05:13.951308+0000","last_became_active":"2024-04-25T10:05:13.949605+0000","last_became_peered":"2024-04-25T10:05:13.949605+0000","last_unstale":"2024-04-25T10:05:13.951308+0000","last_undegraded":"2024-04-25T10:05:13.951308+0000","last_fullsized":"2024-04-25T10:05:13.951308+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:22.429471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.939404+0000","last_change":"2024-04-25T10:05:12.951447+0000","last_active":"2024-04-25T10:05:13.939404+0000","last_peered":"2024-04-25T10:05:13.939404+0000","last_clean":"2024-04-25T10:05:13.939404+0000","last_became_active":"2024-04-25T10:05:12.951174+0000","last_became_peered":"2024-04-25T10:05:12.951174+0000","last_unstale":"2024-04-25T10:05:13.939404+0000","last_undegraded":"2024-04-25T10:05:13.939404+0000","last_fullsized":"2024-04-25T10:05:13.939404+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:21.289661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.967112+0000","last_change":"2024-04-25T10:05:13.965921+0000","last_active":"2024-04-25T10:05:13.967112+0000","last_peered":"2024-04-25T10:05:13.967112+0000","last_clean":"2024-04-25T10:05:13.967112+0000","last_became_active":"2024-04-25T10:05:13.965580+0000","last_became_peered":"2024-04-25T10:05:13.965580+0000","last_unstale":"2024-04-25T10:05:13.967112+0000","last_undegraded":"2024-04-25T10:05:13.967112+0000","last_fullsized":"2024-04-25T10:05:13.967112+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:42.945698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.941893+0000","last_change":"2024-04-25T10:05:12.943891+0000","last_active":"2024-04-25T10:05:13.941893+0000","last_peered":"2024-04-25T10:05:13.941893+0000","last_clean":"2024-04-25T10:05:13.941893+0000","last_became_active":"2024-04-25T10:05:12.943601+0000","last_became_peered":"2024-04-25T10:05:12.943601+0000","last_unstale":"2024-04-25T10:05:13.941893+0000","last_undegraded":"2024-04-25T10:05:13.941893+0000","last_fullsized":"2024-04-25T10:05:13.941893+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:58.699989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.975912+0000","last_change":"2024-04-25T10:05:13.968875+0000","last_active":"2024-04-25T10:05:13.975912+0000","last_peered":"2024-04-25T10:05:13.975912+0000","last_clean":"2024-04-25T10:05:13.975912+0000","last_became_active":"2024-04-25T10:05:13.968542+0000","last_became_peered":"2024-04-25T10:05:13.968542+0000","last_unstale":"2024-04-25T10:05:13.975912+0000","last_undegraded":"2024-04-25T10:05:13.975912+0000","last_fullsized":"2024-04-25T10:05:13.975912+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:53.069900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.938493+0000","last_change":"2024-04-25T10:05:12.948032+0000","last_active":"2024-04-25T10:05:13.938493+0000","last_peered":"2024-04-25T10:05:13.938493+0000","last_clean":"2024-04-25T10:05:13.938493+0000","last_became_active":"2024-04-25T10:05:12.947736+0000","last_became_peered":"2024-04-25T10:05:12.947736+0000","last_unstale":"2024-04-25T10:05:13.938493+0000","last_undegraded":"2024-04-25T10:05:13.938493+0000","last_fullsized":"2024-04-25T10:05:13.938493+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:25.497796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966338+0000","last_change":"2024-04-25T10:05:13.959472+0000","last_active":"2024-04-25T10:05:13.966338+0000","last_peered":"2024-04-25T10:05:13.966338+0000","last_clean":"2024-04-25T10:05:13.966338+0000","last_became_active":"2024-04-25T10:05:13.958930+0000","last_became_peered":"2024-04-25T10:05:13.958930+0000","last_unstale":"2024-04-25T10:05:13.966338+0000","last_undegraded":"2024-04-25T10:05:13.966338+0000","last_fullsized":"2024-04-25T10:05:13.966338+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:47.924488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.939366+0000","last_change":"2024-04-25T10:05:12.947552+0000","last_active":"2024-04-25T10:05:13.939366+0000","last_peered":"2024-04-25T10:05:13.939366+0000","last_clean":"2024-04-25T10:05:13.939366+0000","last_became_active":"2024-04-25T10:05:12.947269+0000","last_became_peered":"2024-04-25T10:05:12.947269+0000","last_unstale":"2024-04-25T10:05:13.939366+0000","last_undegraded":"2024-04-25T10:05:13.939366+0000","last_fullsized":"2024-04-25T10:05:13.939366+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:43.100356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966541+0000","last_change":"2024-04-25T10:05:13.964972+0000","last_active":"2024-04-25T10:05:13.966541+0000","last_peered":"2024-04-25T10:05:13.966541+0000","last_clean":"2024-04-25T10:05:13.966541+0000","last_became_active":"2024-04-25T10:05:13.964686+0000","last_became_peered":"2024-04-25T10:05:13.964686+0000","last_unstale":"2024-04-25T10:05:13.966541+0000","last_undegraded":"2024-04-25T10:05:13.966541+0000","last_fullsized":"2024-04-25T10:05:13.966541+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:36.670877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"177'3","reported_seq":15,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964127+0000","last_change":"2024-04-25T10:05:12.952574+0000","last_active":"2024-04-25T10:05:13.964127+0000","last_peered":"2024-04-25T10:05:13.964127+0000","last_clean":"2024-04-25T10:05:13.964127+0000","last_became_active":"2024-04-25T10:05:12.951545+0000","last_became_peered":"2024-04-25T10:05:12.951545+0000","last_unstale":"2024-04-25T10:05:13.964127+0000","last_undegraded":"2024-04-25T10:05:13.964127+0000","last_fullsized":"2024-04-25T10:05:13.964127+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:19.907187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962736+0000","last_change":"2024-04-25T10:05:13.960221+0000","last_active":"2024-04-25T10:05:13.962736+0000","last_peered":"2024-04-25T10:05:13.962736+0000","last_clean":"2024-04-25T10:05:13.962736+0000","last_became_active":"2024-04-25T10:05:13.959943+0000","last_became_peered":"2024-04-25T10:05:13.959943+0000","last_unstale":"2024-04-25T10:05:13.962736+0000","last_undegraded":"2024-04-25T10:05:13.962736+0000","last_fullsized":"2024-04-25T10:05:13.962736+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:16.006369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"177'6","reported_seq":18,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.976274+0000","last_change":"2024-04-25T10:05:12.945845+0000","last_active":"2024-04-25T10:05:13.976274+0000","last_peered":"2024-04-25T10:05:13.976274+0000","last_clean":"2024-04-25T10:05:13.976274+0000","last_became_active":"2024-04-25T10:05:12.945557+0000","last_became_peered":"2024-04-25T10:05:12.945557+0000","last_unstale":"2024-04-25T10:05:13.976274+0000","last_undegraded":"2024-04-25T10:05:13.976274+0000","last_fullsized":"2024-04-25T10:05:13.976274+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:01.436152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.967117+0000","last_change":"2024-04-25T10:05:13.965741+0000","last_active":"2024-04-25T10:05:13.967117+0000","last_peered":"2024-04-25T10:05:13.967117+0000","last_clean":"2024-04-25T10:05:13.967117+0000","last_became_active":"2024-04-25T10:05:13.965105+0000","last_became_peered":"2024-04-25T10:05:13.965105+0000","last_unstale":"2024-04-25T10:05:13.967117+0000","last_undegraded":"2024-04-25T10:05:13.967117+0000","last_fullsized":"2024-04-25T10:05:13.967117+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:15.566682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960390+0000","last_change":"2024-04-25T10:05:12.950836+0000","last_active":"2024-04-25T10:05:13.960390+0000","last_peered":"2024-04-25T10:05:13.960390+0000","last_clean":"2024-04-25T10:05:13.960390+0000","last_became_active":"2024-04-25T10:05:12.950534+0000","last_became_peered":"2024-04-25T10:05:12.950534+0000","last_unstale":"2024-04-25T10:05:13.960390+0000","last_undegraded":"2024-04-25T10:05:13.960390+0000","last_fullsized":"2024-04-25T10:05:13.960390+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:32.358510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.968690+0000","last_change":"2024-04-25T10:05:13.968109+0000","last_active":"2024-04-25T10:05:13.968690+0000","last_peered":"2024-04-25T10:05:13.968690+0000","last_clean":"2024-04-25T10:05:13.968690+0000","last_became_active":"2024-04-25T10:05:13.967853+0000","last_became_peered":"2024-04-25T10:05:13.967853+0000","last_unstale":"2024-04-25T10:05:13.968690+0000","last_undegraded":"2024-04-25T10:05:13.968690+0000","last_fullsized":"2024-04-25T10:05:13.968690+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:21.905060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.943954+0000","last_change":"2024-04-25T10:05:12.945794+0000","last_active":"2024-04-25T10:05:13.943954+0000","last_peered":"2024-04-25T10:05:13.943954+0000","last_clean":"2024-04-25T10:05:13.943954+0000","last_became_active":"2024-04-25T10:05:12.945501+0000","last_became_peered":"2024-04-25T10:05:12.945501+0000","last_unstale":"2024-04-25T10:05:13.943954+0000","last_undegraded":"2024-04-25T10:05:13.943954+0000","last_fullsized":"2024-04-25T10:05:13.943954+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:56.557559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962696+0000","last_change":"2024-04-25T10:05:13.961715+0000","last_active":"2024-04-25T10:05:13.962696+0000","last_peered":"2024-04-25T10:05:13.962696+0000","last_clean":"2024-04-25T10:05:13.962696+0000","last_became_active":"2024-04-25T10:05:13.961442+0000","last_became_peered":"2024-04-25T10:05:13.961442+0000","last_unstale":"2024-04-25T10:05:13.962696+0000","last_undegraded":"2024-04-25T10:05:13.962696+0000","last_fullsized":"2024-04-25T10:05:13.962696+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:42.849418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.954806+0000","last_change":"2024-04-25T10:05:12.953254+0000","last_active":"2024-04-25T10:05:13.954806+0000","last_peered":"2024-04-25T10:05:13.954806+0000","last_clean":"2024-04-25T10:05:13.954806+0000","last_became_active":"2024-04-25T10:05:12.952946+0000","last_became_peered":"2024-04-25T10:05:12.952946+0000","last_unstale":"2024-04-25T10:05:13.954806+0000","last_undegraded":"2024-04-25T10:05:13.954806+0000","last_fullsized":"2024-04-25T10:05:13.954806+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:28.587109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.953619+0000","last_change":"2024-04-25T10:05:13.952744+0000","last_active":"2024-04-25T10:05:13.953619+0000","last_peered":"2024-04-25T10:05:13.953619+0000","last_clean":"2024-04-25T10:05:13.953619+0000","last_became_active":"2024-04-25T10:05:13.951443+0000","last_became_peered":"2024-04-25T10:05:13.951443+0000","last_unstale":"2024-04-25T10:05:13.953619+0000","last_undegraded":"2024-04-25T10:05:13.953619+0000","last_fullsized":"2024-04-25T10:05:13.953619+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:06.660471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"177'8","reported_seq":20,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.970113+0000","last_change":"2024-04-25T10:05:12.948493+0000","last_active":"2024-04-25T10:05:13.970113+0000","last_peered":"2024-04-25T10:05:13.970113+0000","last_clean":"2024-04-25T10:05:13.970113+0000","last_became_active":"2024-04-25T10:05:12.947793+0000","last_became_peered":"2024-04-25T10:05:12.947793+0000","last_unstale":"2024-04-25T10:05:13.970113+0000","last_undegraded":"2024-04-25T10:05:13.970113+0000","last_fullsized":"2024-04-25T10:05:13.970113+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:12.718223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962650+0000","last_change":"2024-04-25T10:05:13.956792+0000","last_active":"2024-04-25T10:05:13.962650+0000","last_peered":"2024-04-25T10:05:13.962650+0000","last_clean":"2024-04-25T10:05:13.962650+0000","last_became_active":"2024-04-25T10:05:13.956520+0000","last_became_peered":"2024-04-25T10:05:13.956520+0000","last_unstale":"2024-04-25T10:05:13.962650+0000","last_undegraded":"2024-04-25T10:05:13.962650+0000","last_fullsized":"2024-04-25T10:05:13.962650+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:48.351218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"177'1","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.969221+0000","last_change":"2024-04-25T10:05:12.956205+0000","last_active":"2024-04-25T10:05:13.969221+0000","last_peered":"2024-04-25T10:05:13.969221+0000","last_clean":"2024-04-25T10:05:13.969221+0000","last_became_active":"2024-04-25T10:05:12.955887+0000","last_became_peered":"2024-04-25T10:05:12.955887+0000","last_unstale":"2024-04-25T10:05:13.969221+0000","last_undegraded":"2024-04-25T10:05:13.969221+0000","last_fullsized":"2024-04-25T10:05:13.969221+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:15.066195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962252+0000","last_change":"2024-04-25T10:05:13.960447+0000","last_active":"2024-04-25T10:05:13.962252+0000","last_peered":"2024-04-25T10:05:13.962252+0000","last_clean":"2024-04-25T10:05:13.962252+0000","last_became_active":"2024-04-25T10:05:13.960133+0000","last_became_peered":"2024-04-25T10:05:13.960133+0000","last_unstale":"2024-04-25T10:05:13.962252+0000","last_undegraded":"2024-04-25T10:05:13.962252+0000","last_fullsized":"2024-04-25T10:05:13.962252+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:33.847180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.958024+0000","last_change":"2024-04-25T10:05:12.949358+0000","last_active":"2024-04-25T10:05:13.958024+0000","last_peered":"2024-04-25T10:05:13.958024+0000","last_clean":"2024-04-25T10:05:13.958024+0000","last_became_active":"2024-04-25T10:05:12.949055+0000","last_became_peered":"2024-04-25T10:05:12.949055+0000","last_unstale":"2024-04-25T10:05:13.958024+0000","last_undegraded":"2024-04-25T10:05:13.958024+0000","last_fullsized":"2024-04-25T10:05:13.958024+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:32.054191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.963653+0000","last_change":"2024-04-25T10:05:13.962978+0000","last_active":"2024-04-25T10:05:13.963653+0000","last_peered":"2024-04-25T10:05:13.963653+0000","last_clean":"2024-04-25T10:05:13.963653+0000","last_became_active":"2024-04-25T10:05:13.962710+0000","last_became_peered":"2024-04-25T10:05:13.962710+0000","last_unstale":"2024-04-25T10:05:13.963653+0000","last_undegraded":"2024-04-25T10:05:13.963653+0000","last_fullsized":"2024-04-25T10:05:13.963653+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:03.552292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"177'1","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960591+0000","last_change":"2024-04-25T10:05:12.954249+0000","last_active":"2024-04-25T10:05:13.960591+0000","last_peered":"2024-04-25T10:05:13.960591+0000","last_clean":"2024-04-25T10:05:13.960591+0000","last_became_active":"2024-04-25T10:05:12.953965+0000","last_became_peered":"2024-04-25T10:05:12.953965+0000","last_unstale":"2024-04-25T10:05:13.960591+0000","last_undegraded":"2024-04-25T10:05:13.960591+0000","last_fullsized":"2024-04-25T10:05:13.960591+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:28.587109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.947369+0000","last_change":"2024-04-25T10:05:13.945229+0000","last_active":"2024-04-25T10:05:13.947369+0000","last_peered":"2024-04-25T10:05:13.947369+0000","last_clean":"2024-04-25T10:05:13.947369+0000","last_became_active":"2024-04-25T10:05:13.944936+0000","last_became_peered":"2024-04-25T10:05:13.944936+0000","last_unstale":"2024-04-25T10:05:13.947369+0000","last_undegraded":"2024-04-25T10:05:13.947369+0000","last_fullsized":"2024-04-25T10:05:13.947369+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:03.245474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.938781+0000","last_change":"2024-04-25T10:05:12.947755+0000","last_active":"2024-04-25T10:05:13.938781+0000","last_peered":"2024-04-25T10:05:13.938781+0000","last_clean":"2024-04-25T10:05:13.938781+0000","last_became_active":"2024-04-25T10:05:12.947420+0000","last_became_peered":"2024-04-25T10:05:12.947420+0000","last_unstale":"2024-04-25T10:05:13.938781+0000","last_undegraded":"2024-04-25T10:05:13.938781+0000","last_fullsized":"2024-04-25T10:05:13.938781+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:24.221953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.965126+0000","last_change":"2024-04-25T10:05:13.959514+0000","last_active":"2024-04-25T10:05:13.965126+0000","last_peered":"2024-04-25T10:05:13.965126+0000","last_clean":"2024-04-25T10:05:13.965126+0000","last_became_active":"2024-04-25T10:05:13.959247+0000","last_became_peered":"2024-04-25T10:05:13.959247+0000","last_unstale":"2024-04-25T10:05:13.965126+0000","last_undegraded":"2024-04-25T10:05:13.965126+0000","last_fullsized":"2024-04-25T10:05:13.965126+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:20.781602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.954324+0000","last_change":"2024-04-25T10:05:12.944909+0000","last_active":"2024-04-25T10:05:13.954324+0000","last_peered":"2024-04-25T10:05:13.954324+0000","last_clean":"2024-04-25T10:05:13.954324+0000","last_became_active":"2024-04-25T10:05:12.943909+0000","last_became_peered":"2024-04-25T10:05:12.943909+0000","last_unstale":"2024-04-25T10:05:13.954324+0000","last_undegraded":"2024-04-25T10:05:13.954324+0000","last_fullsized":"2024-04-25T10:05:13.954324+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:56.824128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962848+0000","last_change":"2024-04-25T10:05:13.961018+0000","last_active":"2024-04-25T10:05:13.962848+0000","last_peered":"2024-04-25T10:05:13.962848+0000","last_clean":"2024-04-25T10:05:13.962848+0000","last_became_active":"2024-04-25T10:05:13.960611+0000","last_became_peered":"2024-04-25T10:05:13.960611+0000","last_unstale":"2024-04-25T10:05:13.962848+0000","last_undegraded":"2024-04-25T10:05:13.962848+0000","last_fullsized":"2024-04-25T10:05:13.962848+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:33.632465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.946423+0000","last_change":"2024-04-25T10:05:12.945654+0000","last_active":"2024-04-25T10:05:13.946423+0000","last_peered":"2024-04-25T10:05:13.946423+0000","last_clean":"2024-04-25T10:05:13.946423+0000","last_became_active":"2024-04-25T10:05:12.945359+0000","last_became_peered":"2024-04-25T10:05:12.945359+0000","last_unstale":"2024-04-25T10:05:13.946423+0000","last_undegraded":"2024-04-25T10:05:13.946423+0000","last_fullsized":"2024-04-25T10:05:13.946423+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:26.765381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966505+0000","last_change":"2024-04-25T10:05:13.964967+0000","last_active":"2024-04-25T10:05:13.966505+0000","last_peered":"2024-04-25T10:05:13.966505+0000","last_clean":"2024-04-25T10:05:13.966505+0000","last_became_active":"2024-04-25T10:05:13.964640+0000","last_became_peered":"2024-04-25T10:05:13.964640+0000","last_unstale":"2024-04-25T10:05:13.966505+0000","last_undegraded":"2024-04-25T10:05:13.966505+0000","last_fullsized":"2024-04-25T10:05:13.966505+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:46.750710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.938086+0000","last_change":"2024-04-25T10:05:12.956001+0000","last_active":"2024-04-25T10:05:13.938086+0000","last_peered":"2024-04-25T10:05:13.938086+0000","last_clean":"2024-04-25T10:05:13.938086+0000","last_became_active":"2024-04-25T10:05:12.955706+0000","last_became_peered":"2024-04-25T10:05:12.955706+0000","last_unstale":"2024-04-25T10:05:13.938086+0000","last_undegraded":"2024-04-25T10:05:13.938086+0000","last_fullsized":"2024-04-25T10:05:13.938086+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:57.064270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964829+0000","last_change":"2024-04-25T10:05:13.961221+0000","last_active":"2024-04-25T10:05:13.964829+0000","last_peered":"2024-04-25T10:05:13.964829+0000","last_clean":"2024-04-25T10:05:13.964829+0000","last_became_active":"2024-04-25T10:05:13.960779+0000","last_became_peered":"2024-04-25T10:05:13.960779+0000","last_unstale":"2024-04-25T10:05:13.964829+0000","last_undegraded":"2024-04-25T10:05:13.964829+0000","last_fullsized":"2024-04-25T10:05:13.964829+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:44.489884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.942752+0000","last_change":"2024-04-25T10:05:12.951562+0000","last_active":"2024-04-25T10:05:13.942752+0000","last_peered":"2024-04-25T10:05:13.942752+0000","last_clean":"2024-04-25T10:05:13.942752+0000","last_became_active":"2024-04-25T10:05:12.951296+0000","last_became_peered":"2024-04-25T10:05:12.951296+0000","last_unstale":"2024-04-25T10:05:13.942752+0000","last_undegraded":"2024-04-25T10:05:13.942752+0000","last_fullsized":"2024-04-25T10:05:13.942752+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:03.238976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.965985+0000","last_change":"2024-04-25T10:05:13.965040+0000","last_active":"2024-04-25T10:05:13.965985+0000","last_peered":"2024-04-25T10:05:13.965985+0000","last_clean":"2024-04-25T10:05:13.965985+0000","last_became_active":"2024-04-25T10:05:13.964591+0000","last_became_peered":"2024-04-25T10:05:13.964591+0000","last_unstale":"2024-04-25T10:05:13.965985+0000","last_undegraded":"2024-04-25T10:05:13.965985+0000","last_fullsized":"2024-04-25T10:05:13.965985+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:58.071372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966798+0000","last_change":"2024-04-25T10:05:12.955761+0000","last_active":"2024-04-25T10:05:13.966798+0000","last_peered":"2024-04-25T10:05:13.966798+0000","last_clean":"2024-04-25T10:05:13.966798+0000","last_became_active":"2024-04-25T10:05:12.955436+0000","last_became_peered":"2024-04-25T10:05:12.955436+0000","last_unstale":"2024-04-25T10:05:13.966798+0000","last_undegraded":"2024-04-25T10:05:13.966798+0000","last_fullsized":"2024-04-25T10:05:13.966798+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:14.559580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966299+0000","last_change":"2024-04-25T10:05:13.959206+0000","last_active":"2024-04-25T10:05:13.966299+0000","last_peered":"2024-04-25T10:05:13.966299+0000","last_clean":"2024-04-25T10:05:13.966299+0000","last_became_active":"2024-04-25T10:05:13.958559+0000","last_became_peered":"2024-04-25T10:05:13.958559+0000","last_unstale":"2024-04-25T10:05:13.966299+0000","last_undegraded":"2024-04-25T10:05:13.966299+0000","last_fullsized":"2024-04-25T10:05:13.966299+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.121744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.947211+0000","last_change":"2024-04-25T10:05:12.948445+0000","last_active":"2024-04-25T10:05:13.947211+0000","last_peered":"2024-04-25T10:05:13.947211+0000","last_clean":"2024-04-25T10:05:13.947211+0000","last_became_active":"2024-04-25T10:05:12.948181+0000","last_became_peered":"2024-04-25T10:05:12.948181+0000","last_unstale":"2024-04-25T10:05:13.947211+0000","last_undegraded":"2024-04-25T10:05:13.947211+0000","last_fullsized":"2024-04-25T10:05:13.947211+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:16.651601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964129+0000","last_change":"2024-04-25T10:05:13.962531+0000","last_active":"2024-04-25T10:05:13.964129+0000","last_peered":"2024-04-25T10:05:13.964129+0000","last_clean":"2024-04-25T10:05:13.964129+0000","last_became_active":"2024-04-25T10:05:13.962152+0000","last_became_peered":"2024-04-25T10:05:13.962152+0000","last_unstale":"2024-04-25T10:05:13.964129+0000","last_undegraded":"2024-04-25T10:05:13.964129+0000","last_fullsized":"2024-04-25T10:05:13.964129+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:30.154935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.937732+0000","last_change":"2024-04-25T10:05:12.953798+0000","last_active":"2024-04-25T10:05:13.937732+0000","last_peered":"2024-04-25T10:05:13.937732+0000","last_clean":"2024-04-25T10:05:13.937732+0000","last_became_active":"2024-04-25T10:05:12.953524+0000","last_became_peered":"2024-04-25T10:05:12.953524+0000","last_unstale":"2024-04-25T10:05:13.937732+0000","last_undegraded":"2024-04-25T10:05:13.937732+0000","last_fullsized":"2024-04-25T10:05:13.937732+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:24.234922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.956903+0000","last_change":"2024-04-25T10:05:13.955422+0000","last_active":"2024-04-25T10:05:13.956903+0000","last_peered":"2024-04-25T10:05:13.956903+0000","last_clean":"2024-04-25T10:05:13.956903+0000","last_became_active":"2024-04-25T10:05:13.955149+0000","last_became_peered":"2024-04-25T10:05:13.955149+0000","last_unstale":"2024-04-25T10:05:13.956903+0000","last_undegraded":"2024-04-25T10:05:13.956903+0000","last_fullsized":"2024-04-25T10:05:13.956903+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:00.472107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.942134+0000","last_change":"2024-04-25T10:05:12.954577+0000","last_active":"2024-04-25T10:05:13.942134+0000","last_peered":"2024-04-25T10:05:13.942134+0000","last_clean":"2024-04-25T10:05:13.942134+0000","last_became_active":"2024-04-25T10:05:12.954112+0000","last_became_peered":"2024-04-25T10:05:12.954112+0000","last_unstale":"2024-04-25T10:05:13.942134+0000","last_undegraded":"2024-04-25T10:05:13.942134+0000","last_fullsized":"2024-04-25T10:05:13.942134+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:39.732169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962634+0000","last_change":"2024-04-25T10:05:13.958351+0000","last_active":"2024-04-25T10:05:13.962634+0000","last_peered":"2024-04-25T10:05:13.962634+0000","last_clean":"2024-04-25T10:05:13.962634+0000","last_became_active":"2024-04-25T10:05:13.957327+0000","last_became_peered":"2024-04-25T10:05:13.957327+0000","last_unstale":"2024-04-25T10:05:13.962634+0000","last_undegraded":"2024-04-25T10:05:13.962634+0000","last_fullsized":"2024-04-25T10:05:13.962634+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:41.738644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.947018+0000","last_change":"2024-04-25T10:05:12.951998+0000","last_active":"2024-04-25T10:05:13.947018+0000","last_peered":"2024-04-25T10:05:13.947018+0000","last_clean":"2024-04-25T10:05:13.947018+0000","last_became_active":"2024-04-25T10:05:12.951734+0000","last_became_peered":"2024-04-25T10:05:12.951734+0000","last_unstale":"2024-04-25T10:05:13.947018+0000","last_undegraded":"2024-04-25T10:05:13.947018+0000","last_fullsized":"2024-04-25T10:05:13.947018+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:02.303005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.953740+0000","last_change":"2024-04-25T10:05:13.949241+0000","last_active":"2024-04-25T10:05:13.953740+0000","last_peered":"2024-04-25T10:05:13.953740+0000","last_clean":"2024-04-25T10:05:13.953740+0000","last_became_active":"2024-04-25T10:05:13.948915+0000","last_became_peered":"2024-04-25T10:05:13.948915+0000","last_unstale":"2024-04-25T10:05:13.953740+0000","last_undegraded":"2024-04-25T10:05:13.953740+0000","last_fullsized":"2024-04-25T10:05:13.953740+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:34.018215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.956251+0000","last_change":"2024-04-25T10:05:12.950171+0000","last_active":"2024-04-25T10:05:13.956251+0000","last_peered":"2024-04-25T10:05:13.956251+0000","last_clean":"2024-04-25T10:05:13.956251+0000","last_became_active":"2024-04-25T10:05:12.949738+0000","last_became_peered":"2024-04-25T10:05:12.949738+0000","last_unstale":"2024-04-25T10:05:13.956251+0000","last_undegraded":"2024-04-25T10:05:13.956251+0000","last_fullsized":"2024-04-25T10:05:13.956251+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:49.510879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.957737+0000","last_change":"2024-04-25T10:05:13.956416+0000","last_active":"2024-04-25T10:05:13.957737+0000","last_peered":"2024-04-25T10:05:13.957737+0000","last_clean":"2024-04-25T10:05:13.957737+0000","last_became_active":"2024-04-25T10:05:13.955891+0000","last_became_peered":"2024-04-25T10:05:13.955891+0000","last_unstale":"2024-04-25T10:05:13.957737+0000","last_undegraded":"2024-04-25T10:05:13.957737+0000","last_fullsized":"2024-04-25T10:05:13.957737+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:43.843086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.943844+0000","last_change":"2024-04-25T10:05:12.954777+0000","last_active":"2024-04-25T10:05:13.943844+0000","last_peered":"2024-04-25T10:05:13.943844+0000","last_clean":"2024-04-25T10:05:13.943844+0000","last_became_active":"2024-04-25T10:05:12.953061+0000","last_became_peered":"2024-04-25T10:05:12.953061+0000","last_unstale":"2024-04-25T10:05:13.943844+0000","last_undegraded":"2024-04-25T10:05:13.943844+0000","last_fullsized":"2024-04-25T10:05:13.943844+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:15.159957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966169+0000","last_change":"2024-04-25T10:05:13.958218+0000","last_active":"2024-04-25T10:05:13.966169+0000","last_peered":"2024-04-25T10:05:13.966169+0000","last_clean":"2024-04-25T10:05:13.966169+0000","last_became_active":"2024-04-25T10:05:13.957905+0000","last_became_peered":"2024-04-25T10:05:13.957905+0000","last_unstale":"2024-04-25T10:05:13.966169+0000","last_undegraded":"2024-04-25T10:05:13.966169+0000","last_fullsized":"2024-04-25T10:05:13.966169+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:33.632465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.937689+0000","last_change":"2024-04-25T10:05:12.948244+0000","last_active":"2024-04-25T10:05:13.937689+0000","last_peered":"2024-04-25T10:05:13.937689+0000","last_clean":"2024-04-25T10:05:13.937689+0000","last_became_active":"2024-04-25T10:05:12.947552+0000","last_became_peered":"2024-04-25T10:05:12.947552+0000","last_unstale":"2024-04-25T10:05:13.937689+0000","last_undegraded":"2024-04-25T10:05:13.937689+0000","last_fullsized":"2024-04-25T10:05:13.937689+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:32.440271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.961994+0000","last_change":"2024-04-25T10:05:13.960690+0000","last_active":"2024-04-25T10:05:13.961994+0000","last_peered":"2024-04-25T10:05:13.961994+0000","last_clean":"2024-04-25T10:05:13.961994+0000","last_became_active":"2024-04-25T10:05:13.960406+0000","last_became_peered":"2024-04-25T10:05:13.960406+0000","last_unstale":"2024-04-25T10:05:13.961994+0000","last_undegraded":"2024-04-25T10:05:13.961994+0000","last_fullsized":"2024-04-25T10:05:13.961994+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:12.790722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"177'1","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962369+0000","last_change":"2024-04-25T10:05:12.952811+0000","last_active":"2024-04-25T10:05:13.962369+0000","last_peered":"2024-04-25T10:05:13.962369+0000","last_clean":"2024-04-25T10:05:13.962369+0000","last_became_active":"2024-04-25T10:05:12.952488+0000","last_became_peered":"2024-04-25T10:05:12.952488+0000","last_unstale":"2024-04-25T10:05:13.962369+0000","last_undegraded":"2024-04-25T10:05:13.962369+0000","last_fullsized":"2024-04-25T10:05:13.962369+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:23.930697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":490,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.938660+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:05:13.938660+0000","last_peered":"2024-04-25T10:05:13.938660+0000","last_clean":"2024-04-25T10:05:13.938660+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:05:13.938660+0000","last_undegraded":"2024-04-25T10:05:13.938660+0000","last_fullsized":"2024-04-25T10:05:13.938660+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51984,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319856,"statfs":{"total":96636764160,"available":96583532544,"internally_reserved":0,"allocated":2818048,"data_stored":1633265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51400,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320440,"statfs":{"total":96636764160,"available":96584130560,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51412,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320428,"statfs":{"total":96636764160,"available":96584118272,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59720,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312120,"statfs":{"total":96636764160,"available":96575610880,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59724,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312116,"statfs":{"total":96636764160,"available":96575606784,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55564,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316276,"statfs":{"total":96636764160,"available":96579866624,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55560,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316280,"statfs":{"total":96636764160,"available":96579870720,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56140,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315700,"statfs":{"total":96636764160,"available":96579276800,"internally_reserved":0,"allocated":2818048,"data_stored":1633265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:16.951 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:17.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.081+0000 7fb1cbff4640 1 Processor -- start 2024-04-25T10:05:17.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.081+0000 7fb1cbff4640 1 -- start start 2024-04-25T10:05:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.082+0000 7fb1cbff4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1c40d1020 0x7fb1c40d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.082+0000 7fb1cbff4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40b97a0 con 0x7fb1c415dbe0 2024-04-25T10:05:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.082+0000 7fb1cbff4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40d1c50 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.082+0000 7fb1cbff4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c415dfc0 con 0x7fb1c40d1020 2024-04-25T10:05:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.082+0000 7fb1c9d90640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb1c415dbe0 0x7fb1c40d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37452/0 (socket says 172.21.15.19:37452) 2024-04-25T10:05:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.082+0000 7fb1c9d90640 1 -- 172.21.15.19:0/4085877543 learned_addr learned my addr 172.21.15.19:0/4085877543 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.082+0000 7fb1ca591640 1 --2- 172.21.15.19:0/4085877543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1c40d1020 0x7fb1c40d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.083+0000 7fb1c8d8e640 1 -- 172.21.15.19:0/4085877543 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406841019 0 0) 0x7fb1c40b97a0 con 0x7fb1c415dbe0 2024-04-25T10:05:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.083+0000 7fb1c8d8e640 1 -- 172.21.15.19:0/4085877543 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1ac003520 con 0x7fb1c415dbe0 2024-04-25T10:05:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.083+0000 7fb1c8d8e640 1 -- 172.21.15.19:0/4085877543 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516499399 0 0) 0x7fb1c40d1c50 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.083+0000 7fb1c8d8e640 1 -- 172.21.15.19:0/4085877543 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c40b97a0 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.083+0000 7fb1ca591640 1 -- 172.21.15.19:0/4085877543 >> v1:172.21.15.112:6789/0 conn(0x7fb1c40b2cb0 legacy=0x7fb1c40d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.083+0000 7fb1ca591640 1 -- 172.21.15.19:0/4085877543 >> v1:172.21.15.19:6789/0 conn(0x7fb1c415dbe0 legacy=0x7fb1c40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.083+0000 7fb1ca591640 1 -- 172.21.15.19:0/4085877543 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c40d1f30 con 0x7fb1c40d1020 2024-04-25T10:05:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.084+0000 7fb1ca591640 1 --2- 172.21.15.19:0/4085877543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1c40d1020 0x7fb1c40d2e40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0097d0 tx=0x7fb1bc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ccdc2555decaee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.084+0000 7fb1c8d8e640 1 -- 172.21.15.19:0/4085877543 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc037070 con 0x7fb1c40d1020 2024-04-25T10:05:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.084+0000 7fb1c8d8e640 1 -- 172.21.15.19:0/4085877543 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc032440 con 0x7fb1c40d1020 2024-04-25T10:05:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.084+0000 7fb1c8d8e640 1 -- 172.21.15.19:0/4085877543 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0327f0 con 0x7fb1c40d1020 2024-04-25T10:05:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.084+0000 7fb1cbff4640 1 -- 172.21.15.19:0/4085877543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1c40d1020 msgr2=0x7fb1c40d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.084+0000 7fb1cbff4640 1 --2- 172.21.15.19:0/4085877543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1c40d1020 0x7fb1c40d2e40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0097d0 tx=0x7fb1bc02a210 comp rx=0 tx=0).stop 2024-04-25T10:05:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.084+0000 7fb1cbff4640 1 -- 172.21.15.19:0/4085877543 shutdown_connections 2024-04-25T10:05:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.084+0000 7fb1cbff4640 1 --2- 172.21.15.19:0/4085877543 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1c40d1020 0x7fb1c40d2e40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.084+0000 7fb1cbff4640 1 -- 172.21.15.19:0/4085877543 >> 172.21.15.19:0/4085877543 conn(0x7fb1c4061a20 msgr2=0x7fb1c4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.084+0000 7fb1cbff4640 1 -- 172.21.15.19:0/4085877543 shutdown_connections 2024-04-25T10:05:17.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.085+0000 7fb1cbff4640 1 -- 172.21.15.19:0/4085877543 wait complete. 2024-04-25T10:05:17.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.086+0000 7fb1cbff4640 1 Processor -- start 2024-04-25T10:05:17.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.086+0000 7fb1cbff4640 1 -- start start 2024-04-25T10:05:17.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.086+0000 7fb1cbff4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1c40b2cb0 0x7fb1c415fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.086+0000 7fb1cbff4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1c40d1020 0x7fb1c4160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.086+0000 7fb1cbff4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c415dbe0 0x7fb1c41f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.086+0000 7fb1cbff4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40b92a0 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.086+0000 7fb1cbff4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40d1a00 con 0x7fb1c415dbe0 2024-04-25T10:05:17.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.086+0000 7fb1cbff4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40b9420 con 0x7fb1c40d1020 2024-04-25T10:05:17.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.086+0000 7fb1c9d90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1c40b2cb0 0x7fb1c415fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.086+0000 7fb1c9d90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1c40b2cb0 0x7fb1c415fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38268/0 (socket says 172.21.15.19:38268) 2024-04-25T10:05:17.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1c9d90640 1 -- 172.21.15.19:0/2230652417 learned_addr learned my addr 172.21.15.19:0/2230652417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1c958f640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1c40d1020 0x7fb1c4160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1c9d90640 1 -- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1c40d1020 msgr2=0x7fb1c4160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1ca591640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c415dbe0 0x7fb1c41f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1c9d90640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1c40d1020 0x7fb1c4160ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1c9d90640 1 -- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c415dbe0 msgr2=0x7fb1c41f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1c9d90640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c415dbe0 0x7fb1c41f6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1c9d90640 1 -- 172.21.15.19:0/2230652417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c4160500 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1c9d90640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1c40b2cb0 0x7fb1c415fe40 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400cb70 tx=0x7fb1b400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1c27fc640 1 -- 172.21.15.19:0/2230652417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4017070 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1c27fc640 1 -- 172.21.15.19:0/2230652417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b400ad40 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1cbff4640 1 -- 172.21.15.19:0/2230652417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c415eb60 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1c27fc640 1 -- 172.21.15.19:0/2230652417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4012600 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.087+0000 7fb1cbff4640 1 -- 172.21.15.19:0/2230652417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c415f120 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.088+0000 7fb1cbff4640 1 -- 172.21.15.19:0/2230652417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c4073990 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.089+0000 7fb1c27fc640 1 -- 172.21.15.19:0/2230652417 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1b401a070 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.089+0000 7fb1c27fc640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1b006b4d0 0x7fb1b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.089+0000 7fb1c27fc640 1 -- 172.21.15.19:0/2230652417 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb1b408a030 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.089+0000 7fb1c958f640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1b006b4d0 0x7fb1b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.090+0000 7fb1c958f640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1b006b4d0 0x7fb1b006d990 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80010e0 tx=0x7fb1b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.093+0000 7fb1c27fc640 1 -- 172.21.15.19:0/2230652417 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1b4058760 con 0x7fb1c40b2cb0 2024-04-25T10:05:17.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.268+0000 7fb1cbff4640 1 -- 172.21.15.19:0/2230652417 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1c40ca2e0 con 0x7fb1b006b4d0 2024-04-25T10:05:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.275+0000 7fb1c27fc640 1 -- 172.21.15.19:0/2230652417 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252421 (secure 0 0 0) 0x7fb1c40ca2e0 con 0x7fb1b006b4d0 2024-04-25T10:05:17.283 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:05:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 -- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1b006b4d0 msgr2=0x7fb1b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1b006b4d0 0x7fb1b006d990 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80010e0 tx=0x7fb1b8009040 comp rx=0 tx=0).stop 2024-04-25T10:05:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 -- 172.21.15.19:0/2230652417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1c40b2cb0 msgr2=0x7fb1c415fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1c40b2cb0 0x7fb1c415fe40 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400cb70 tx=0x7fb1b400a430 comp rx=0 tx=0).stop 2024-04-25T10:05:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 -- 172.21.15.19:0/2230652417 shutdown_connections 2024-04-25T10:05:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb1b006b4d0 0x7fb1b006d990 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c415dbe0 0x7fb1c41f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1c40d1020 0x7fb1c4160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 --2- 172.21.15.19:0/2230652417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1c40b2cb0 0x7fb1c415fe40 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 -- 172.21.15.19:0/2230652417 >> 172.21.15.19:0/2230652417 conn(0x7fb1c4061a20 msgr2=0x7fb1c41593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 -- 172.21.15.19:0/2230652417 shutdown_connections 2024-04-25T10:05:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.278+0000 7fb1cbff4640 1 -- 172.21.15.19:0/2230652417 wait complete. 2024-04-25T10:05:17.294 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":543,"stamp":"2024-04-25T10:05:17.082924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":441504,"kb_used_data":18536,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754533216,"statfs":{"total":773094113280,"available":772642013184,"internally_reserved":0,"allocated":18980864,"data_stored":9523912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":432983602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.157775"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.961986+0000","last_change":"2024-04-25T10:05:13.960275+0000","last_active":"2024-04-25T10:05:13.961986+0000","last_peered":"2024-04-25T10:05:13.961986+0000","last_clean":"2024-04-25T10:05:13.961986+0000","last_became_active":"2024-04-25T10:05:13.959940+0000","last_became_peered":"2024-04-25T10:05:13.959940+0000","last_unstale":"2024-04-25T10:05:13.961986+0000","last_undegraded":"2024-04-25T10:05:13.961986+0000","last_fullsized":"2024-04-25T10:05:13.961986+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:18.404329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960031+0000","last_change":"2024-04-25T10:05:13.950499+0000","last_active":"2024-04-25T10:05:13.960031+0000","last_peered":"2024-04-25T10:05:13.960031+0000","last_clean":"2024-04-25T10:05:13.960031+0000","last_became_active":"2024-04-25T10:05:13.950095+0000","last_became_peered":"2024-04-25T10:05:13.950095+0000","last_unstale":"2024-04-25T10:05:13.960031+0000","last_undegraded":"2024-04-25T10:05:13.960031+0000","last_fullsized":"2024-04-25T10:05:13.960031+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:20.914289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.952965+0000","last_change":"2024-04-25T10:05:13.951403+0000","last_active":"2024-04-25T10:05:13.952965+0000","last_peered":"2024-04-25T10:05:13.952965+0000","last_clean":"2024-04-25T10:05:13.952965+0000","last_became_active":"2024-04-25T10:05:13.951112+0000","last_became_peered":"2024-04-25T10:05:13.951112+0000","last_unstale":"2024-04-25T10:05:13.952965+0000","last_undegraded":"2024-04-25T10:05:13.952965+0000","last_fullsized":"2024-04-25T10:05:13.952965+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:06.660471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.975909+0000","last_change":"2024-04-25T10:05:13.968980+0000","last_active":"2024-04-25T10:05:13.975909+0000","last_peered":"2024-04-25T10:05:13.975909+0000","last_clean":"2024-04-25T10:05:13.975909+0000","last_became_active":"2024-04-25T10:05:13.968704+0000","last_became_peered":"2024-04-25T10:05:13.968704+0000","last_unstale":"2024-04-25T10:05:13.975909+0000","last_undegraded":"2024-04-25T10:05:13.975909+0000","last_fullsized":"2024-04-25T10:05:13.975909+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.121744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.954162+0000","last_change":"2024-04-25T10:05:13.946171+0000","last_active":"2024-04-25T10:05:13.954162+0000","last_peered":"2024-04-25T10:05:13.954162+0000","last_clean":"2024-04-25T10:05:13.954162+0000","last_became_active":"2024-04-25T10:05:13.945873+0000","last_became_peered":"2024-04-25T10:05:13.945873+0000","last_unstale":"2024-04-25T10:05:13.954162+0000","last_undegraded":"2024-04-25T10:05:13.954162+0000","last_fullsized":"2024-04-25T10:05:13.954162+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:43.005765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.953553+0000","last_change":"2024-04-25T10:05:13.950200+0000","last_active":"2024-04-25T10:05:13.953553+0000","last_peered":"2024-04-25T10:05:13.953553+0000","last_clean":"2024-04-25T10:05:13.953553+0000","last_became_active":"2024-04-25T10:05:13.949883+0000","last_became_peered":"2024-04-25T10:05:13.949883+0000","last_unstale":"2024-04-25T10:05:13.953553+0000","last_undegraded":"2024-04-25T10:05:13.953553+0000","last_fullsized":"2024-04-25T10:05:13.953553+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:57.564661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.945443+0000","last_change":"2024-04-25T10:05:13.941215+0000","last_active":"2024-04-25T10:05:13.945443+0000","last_peered":"2024-04-25T10:05:13.945443+0000","last_clean":"2024-04-25T10:05:13.945443+0000","last_became_active":"2024-04-25T10:05:13.940832+0000","last_became_peered":"2024-04-25T10:05:13.940832+0000","last_unstale":"2024-04-25T10:05:13.945443+0000","last_undegraded":"2024-04-25T10:05:13.945443+0000","last_fullsized":"2024-04-25T10:05:13.945443+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:10.110857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962486+0000","last_change":"2024-04-25T10:05:13.955816+0000","last_active":"2024-04-25T10:05:13.962486+0000","last_peered":"2024-04-25T10:05:13.962486+0000","last_clean":"2024-04-25T10:05:13.962486+0000","last_became_active":"2024-04-25T10:05:13.955379+0000","last_became_peered":"2024-04-25T10:05:13.955379+0000","last_unstale":"2024-04-25T10:05:13.962486+0000","last_undegraded":"2024-04-25T10:05:13.962486+0000","last_fullsized":"2024-04-25T10:05:13.962486+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:31.462742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.975249+0000","last_change":"2024-04-25T10:05:13.974885+0000","last_active":"2024-04-25T10:05:13.975249+0000","last_peered":"2024-04-25T10:05:13.975249+0000","last_clean":"2024-04-25T10:05:13.975249+0000","last_became_active":"2024-04-25T10:05:13.974626+0000","last_became_peered":"2024-04-25T10:05:13.974626+0000","last_unstale":"2024-04-25T10:05:13.975249+0000","last_undegraded":"2024-04-25T10:05:13.975249+0000","last_fullsized":"2024-04-25T10:05:13.975249+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:19.706548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.961701+0000","last_change":"2024-04-25T10:05:13.956287+0000","last_active":"2024-04-25T10:05:13.961701+0000","last_peered":"2024-04-25T10:05:13.961701+0000","last_clean":"2024-04-25T10:05:13.961701+0000","last_became_active":"2024-04-25T10:05:13.956006+0000","last_became_peered":"2024-04-25T10:05:13.956006+0000","last_unstale":"2024-04-25T10:05:13.961701+0000","last_undegraded":"2024-04-25T10:05:13.961701+0000","last_fullsized":"2024-04-25T10:05:13.961701+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:21.605197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966654+0000","last_change":"2024-04-25T10:05:13.965752+0000","last_active":"2024-04-25T10:05:13.966654+0000","last_peered":"2024-04-25T10:05:13.966654+0000","last_clean":"2024-04-25T10:05:13.966654+0000","last_became_active":"2024-04-25T10:05:13.965460+0000","last_became_peered":"2024-04-25T10:05:13.965460+0000","last_unstale":"2024-04-25T10:05:13.966654+0000","last_undegraded":"2024-04-25T10:05:13.966654+0000","last_fullsized":"2024-04-25T10:05:13.966654+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:56.713012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.968729+0000","last_change":"2024-04-25T10:05:13.967277+0000","last_active":"2024-04-25T10:05:13.968729+0000","last_peered":"2024-04-25T10:05:13.968729+0000","last_clean":"2024-04-25T10:05:13.968729+0000","last_became_active":"2024-04-25T10:05:13.967001+0000","last_became_peered":"2024-04-25T10:05:13.967001+0000","last_unstale":"2024-04-25T10:05:13.968729+0000","last_undegraded":"2024-04-25T10:05:13.968729+0000","last_fullsized":"2024-04-25T10:05:13.968729+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:04.351882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.949431+0000","last_change":"2024-04-25T10:05:13.948369+0000","last_active":"2024-04-25T10:05:13.949431+0000","last_peered":"2024-04-25T10:05:13.949431+0000","last_clean":"2024-04-25T10:05:13.949431+0000","last_became_active":"2024-04-25T10:05:13.948066+0000","last_became_peered":"2024-04-25T10:05:13.948066+0000","last_unstale":"2024-04-25T10:05:13.949431+0000","last_undegraded":"2024-04-25T10:05:13.949431+0000","last_fullsized":"2024-04-25T10:05:13.949431+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:11.317829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.963946+0000","last_change":"2024-04-25T10:05:13.961114+0000","last_active":"2024-04-25T10:05:13.963946+0000","last_peered":"2024-04-25T10:05:13.963946+0000","last_clean":"2024-04-25T10:05:13.963946+0000","last_became_active":"2024-04-25T10:05:13.959327+0000","last_became_peered":"2024-04-25T10:05:13.959327+0000","last_unstale":"2024-04-25T10:05:13.963946+0000","last_undegraded":"2024-04-25T10:05:13.963946+0000","last_fullsized":"2024-04-25T10:05:13.963946+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:40.334224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.957085+0000","last_change":"2024-04-25T10:05:13.955462+0000","last_active":"2024-04-25T10:05:13.957085+0000","last_peered":"2024-04-25T10:05:13.957085+0000","last_clean":"2024-04-25T10:05:13.957085+0000","last_became_active":"2024-04-25T10:05:13.955177+0000","last_became_peered":"2024-04-25T10:05:13.955177+0000","last_unstale":"2024-04-25T10:05:13.957085+0000","last_undegraded":"2024-04-25T10:05:13.957085+0000","last_fullsized":"2024-04-25T10:05:13.957085+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:14.413431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.952238+0000","last_change":"2024-04-25T10:05:13.950588+0000","last_active":"2024-04-25T10:05:13.952238+0000","last_peered":"2024-04-25T10:05:13.952238+0000","last_clean":"2024-04-25T10:05:13.952238+0000","last_became_active":"2024-04-25T10:05:13.950293+0000","last_became_peered":"2024-04-25T10:05:13.950293+0000","last_unstale":"2024-04-25T10:05:13.952238+0000","last_undegraded":"2024-04-25T10:05:13.952238+0000","last_fullsized":"2024-04-25T10:05:13.952238+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:35.796402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.967850+0000","last_change":"2024-04-25T10:05:13.966966+0000","last_active":"2024-04-25T10:05:13.967850+0000","last_peered":"2024-04-25T10:05:13.967850+0000","last_clean":"2024-04-25T10:05:13.967850+0000","last_became_active":"2024-04-25T10:05:13.966346+0000","last_became_peered":"2024-04-25T10:05:13.966346+0000","last_unstale":"2024-04-25T10:05:13.967850+0000","last_undegraded":"2024-04-25T10:05:13.967850+0000","last_fullsized":"2024-04-25T10:05:13.967850+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:35.065296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.970988+0000","last_change":"2024-04-25T10:05:13.970408+0000","last_active":"2024-04-25T10:05:13.970988+0000","last_peered":"2024-04-25T10:05:13.970988+0000","last_clean":"2024-04-25T10:05:13.970988+0000","last_became_active":"2024-04-25T10:05:13.970020+0000","last_became_peered":"2024-04-25T10:05:13.970020+0000","last_unstale":"2024-04-25T10:05:13.970988+0000","last_undegraded":"2024-04-25T10:05:13.970988+0000","last_fullsized":"2024-04-25T10:05:13.970988+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:29.729358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966373+0000","last_change":"2024-04-25T10:05:13.965944+0000","last_active":"2024-04-25T10:05:13.966373+0000","last_peered":"2024-04-25T10:05:13.966373+0000","last_clean":"2024-04-25T10:05:13.966373+0000","last_became_active":"2024-04-25T10:05:13.965616+0000","last_became_peered":"2024-04-25T10:05:13.965616+0000","last_unstale":"2024-04-25T10:05:13.966373+0000","last_undegraded":"2024-04-25T10:05:13.966373+0000","last_fullsized":"2024-04-25T10:05:13.966373+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:44.331097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964143+0000","last_change":"2024-04-25T10:05:13.959469+0000","last_active":"2024-04-25T10:05:13.964143+0000","last_peered":"2024-04-25T10:05:13.964143+0000","last_clean":"2024-04-25T10:05:13.964143+0000","last_became_active":"2024-04-25T10:05:13.959147+0000","last_became_peered":"2024-04-25T10:05:13.959147+0000","last_unstale":"2024-04-25T10:05:13.964143+0000","last_undegraded":"2024-04-25T10:05:13.964143+0000","last_fullsized":"2024-04-25T10:05:13.964143+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:11.501488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964876+0000","last_change":"2024-04-25T10:05:13.963537+0000","last_active":"2024-04-25T10:05:13.964876+0000","last_peered":"2024-04-25T10:05:13.964876+0000","last_clean":"2024-04-25T10:05:13.964876+0000","last_became_active":"2024-04-25T10:05:13.963270+0000","last_became_peered":"2024-04-25T10:05:13.963270+0000","last_unstale":"2024-04-25T10:05:13.964876+0000","last_undegraded":"2024-04-25T10:05:13.964876+0000","last_fullsized":"2024-04-25T10:05:13.964876+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:25.242024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964660+0000","last_change":"2024-04-25T10:05:13.959371+0000","last_active":"2024-04-25T10:05:13.964660+0000","last_peered":"2024-04-25T10:05:13.964660+0000","last_clean":"2024-04-25T10:05:13.964660+0000","last_became_active":"2024-04-25T10:05:13.959091+0000","last_became_peered":"2024-04-25T10:05:13.959091+0000","last_unstale":"2024-04-25T10:05:13.964660+0000","last_undegraded":"2024-04-25T10:05:13.964660+0000","last_fullsized":"2024-04-25T10:05:13.964660+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:43.687988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960875+0000","last_change":"2024-04-25T10:05:13.958796+0000","last_active":"2024-04-25T10:05:13.960875+0000","last_peered":"2024-04-25T10:05:13.960875+0000","last_clean":"2024-04-25T10:05:13.960875+0000","last_became_active":"2024-04-25T10:05:13.958357+0000","last_became_peered":"2024-04-25T10:05:13.958357+0000","last_unstale":"2024-04-25T10:05:13.960875+0000","last_undegraded":"2024-04-25T10:05:13.960875+0000","last_fullsized":"2024-04-25T10:05:13.960875+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:24.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.956384+0000","last_change":"2024-04-25T10:05:13.950809+0000","last_active":"2024-04-25T10:05:13.956384+0000","last_peered":"2024-04-25T10:05:13.956384+0000","last_clean":"2024-04-25T10:05:13.956384+0000","last_became_active":"2024-04-25T10:05:13.950541+0000","last_became_peered":"2024-04-25T10:05:13.950541+0000","last_unstale":"2024-04-25T10:05:13.956384+0000","last_undegraded":"2024-04-25T10:05:13.956384+0000","last_fullsized":"2024-04-25T10:05:13.956384+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:43.563858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.970513+0000","last_change":"2024-04-25T10:05:13.969018+0000","last_active":"2024-04-25T10:05:13.970513+0000","last_peered":"2024-04-25T10:05:13.970513+0000","last_clean":"2024-04-25T10:05:13.970513+0000","last_became_active":"2024-04-25T10:05:13.968630+0000","last_became_peered":"2024-04-25T10:05:13.968630+0000","last_unstale":"2024-04-25T10:05:13.970513+0000","last_undegraded":"2024-04-25T10:05:13.970513+0000","last_fullsized":"2024-04-25T10:05:13.970513+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:40.769585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.965600+0000","last_change":"2024-04-25T10:05:13.964653+0000","last_active":"2024-04-25T10:05:13.965600+0000","last_peered":"2024-04-25T10:05:13.965600+0000","last_clean":"2024-04-25T10:05:13.965600+0000","last_became_active":"2024-04-25T10:05:13.964225+0000","last_became_peered":"2024-04-25T10:05:13.964225+0000","last_unstale":"2024-04-25T10:05:13.965600+0000","last_undegraded":"2024-04-25T10:05:13.965600+0000","last_fullsized":"2024-04-25T10:05:13.965600+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:05.254847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.956362+0000","last_change":"2024-04-25T10:05:13.955939+0000","last_active":"2024-04-25T10:05:13.956362+0000","last_peered":"2024-04-25T10:05:13.956362+0000","last_clean":"2024-04-25T10:05:13.956362+0000","last_became_active":"2024-04-25T10:05:13.955628+0000","last_became_peered":"2024-04-25T10:05:13.955628+0000","last_unstale":"2024-04-25T10:05:13.956362+0000","last_undegraded":"2024-04-25T10:05:13.956362+0000","last_fullsized":"2024-04-25T10:05:13.956362+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:43.843086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.953619+0000","last_change":"2024-04-25T10:05:13.943450+0000","last_active":"2024-04-25T10:05:13.953619+0000","last_peered":"2024-04-25T10:05:13.953619+0000","last_clean":"2024-04-25T10:05:13.953619+0000","last_became_active":"2024-04-25T10:05:13.943181+0000","last_became_peered":"2024-04-25T10:05:13.943181+0000","last_unstale":"2024-04-25T10:05:13.953619+0000","last_undegraded":"2024-04-25T10:05:13.953619+0000","last_fullsized":"2024-04-25T10:05:13.953619+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:08.330613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.967004+0000","last_change":"2024-04-25T10:05:13.965852+0000","last_active":"2024-04-25T10:05:13.967004+0000","last_peered":"2024-04-25T10:05:13.967004+0000","last_clean":"2024-04-25T10:05:13.967004+0000","last_became_active":"2024-04-25T10:05:13.965595+0000","last_became_peered":"2024-04-25T10:05:13.965595+0000","last_unstale":"2024-04-25T10:05:13.967004+0000","last_undegraded":"2024-04-25T10:05:13.967004+0000","last_fullsized":"2024-04-25T10:05:13.967004+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:19.674779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.951424+0000","last_change":"2024-04-25T10:05:13.948527+0000","last_active":"2024-04-25T10:05:13.951424+0000","last_peered":"2024-04-25T10:05:13.951424+0000","last_clean":"2024-04-25T10:05:13.951424+0000","last_became_active":"2024-04-25T10:05:13.947332+0000","last_became_peered":"2024-04-25T10:05:13.947332+0000","last_unstale":"2024-04-25T10:05:13.951424+0000","last_undegraded":"2024-04-25T10:05:13.951424+0000","last_fullsized":"2024-04-25T10:05:13.951424+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:44.331097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.953582+0000","last_change":"2024-04-25T10:05:13.951937+0000","last_active":"2024-04-25T10:05:13.953582+0000","last_peered":"2024-04-25T10:05:13.953582+0000","last_clean":"2024-04-25T10:05:13.953582+0000","last_became_active":"2024-04-25T10:05:13.951122+0000","last_became_peered":"2024-04-25T10:05:13.951122+0000","last_unstale":"2024-04-25T10:05:13.953582+0000","last_undegraded":"2024-04-25T10:05:13.953582+0000","last_fullsized":"2024-04-25T10:05:13.953582+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:35.796402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.946711+0000","last_change":"2024-04-25T10:05:13.945709+0000","last_active":"2024-04-25T10:05:13.946711+0000","last_peered":"2024-04-25T10:05:13.946711+0000","last_clean":"2024-04-25T10:05:13.946711+0000","last_became_active":"2024-04-25T10:05:13.945323+0000","last_became_peered":"2024-04-25T10:05:13.945323+0000","last_unstale":"2024-04-25T10:05:13.946711+0000","last_undegraded":"2024-04-25T10:05:13.946711+0000","last_fullsized":"2024-04-25T10:05:13.946711+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:09.118377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960921+0000","last_change":"2024-04-25T10:05:13.958986+0000","last_active":"2024-04-25T10:05:13.960921+0000","last_peered":"2024-04-25T10:05:13.960921+0000","last_clean":"2024-04-25T10:05:13.960921+0000","last_became_active":"2024-04-25T10:05:13.958261+0000","last_became_peered":"2024-04-25T10:05:13.958261+0000","last_unstale":"2024-04-25T10:05:13.960921+0000","last_undegraded":"2024-04-25T10:05:13.960921+0000","last_fullsized":"2024-04-25T10:05:13.960921+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:50.517981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"177'4","reported_seq":16,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.965096+0000","last_change":"2024-04-25T10:05:12.955599+0000","last_active":"2024-04-25T10:05:13.965096+0000","last_peered":"2024-04-25T10:05:13.965096+0000","last_clean":"2024-04-25T10:05:13.965096+0000","last_became_active":"2024-04-25T10:05:12.954861+0000","last_became_peered":"2024-04-25T10:05:12.954861+0000","last_unstale":"2024-04-25T10:05:13.965096+0000","last_undegraded":"2024-04-25T10:05:13.965096+0000","last_fullsized":"2024-04-25T10:05:13.965096+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:31.018350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.950678+0000","last_change":"2024-04-25T10:05:13.950029+0000","last_active":"2024-04-25T10:05:13.950678+0000","last_peered":"2024-04-25T10:05:13.950678+0000","last_clean":"2024-04-25T10:05:13.950678+0000","last_became_active":"2024-04-25T10:05:13.949756+0000","last_became_peered":"2024-04-25T10:05:13.949756+0000","last_unstale":"2024-04-25T10:05:13.950678+0000","last_undegraded":"2024-04-25T10:05:13.950678+0000","last_fullsized":"2024-04-25T10:05:13.950678+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:14.745138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"177'1","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.963071+0000","last_change":"2024-04-25T10:05:12.951502+0000","last_active":"2024-04-25T10:05:13.963071+0000","last_peered":"2024-04-25T10:05:13.963071+0000","last_clean":"2024-04-25T10:05:13.963071+0000","last_became_active":"2024-04-25T10:05:12.950400+0000","last_became_peered":"2024-04-25T10:05:12.950400+0000","last_unstale":"2024-04-25T10:05:13.963071+0000","last_undegraded":"2024-04-25T10:05:13.963071+0000","last_fullsized":"2024-04-25T10:05:13.963071+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:45.830773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964633+0000","last_change":"2024-04-25T10:05:13.961715+0000","last_active":"2024-04-25T10:05:13.964633+0000","last_peered":"2024-04-25T10:05:13.964633+0000","last_clean":"2024-04-25T10:05:13.964633+0000","last_became_active":"2024-04-25T10:05:13.961434+0000","last_became_peered":"2024-04-25T10:05:13.961434+0000","last_unstale":"2024-04-25T10:05:13.964633+0000","last_undegraded":"2024-04-25T10:05:13.964633+0000","last_fullsized":"2024-04-25T10:05:13.964633+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:32.025451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960484+0000","last_change":"2024-04-25T10:05:12.950304+0000","last_active":"2024-04-25T10:05:13.960484+0000","last_peered":"2024-04-25T10:05:13.960484+0000","last_clean":"2024-04-25T10:05:13.960484+0000","last_became_active":"2024-04-25T10:05:12.950022+0000","last_became_peered":"2024-04-25T10:05:12.950022+0000","last_unstale":"2024-04-25T10:05:13.960484+0000","last_undegraded":"2024-04-25T10:05:13.960484+0000","last_fullsized":"2024-04-25T10:05:13.960484+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:18.449777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962676+0000","last_change":"2024-04-25T10:05:13.953992+0000","last_active":"2024-04-25T10:05:13.962676+0000","last_peered":"2024-04-25T10:05:13.962676+0000","last_clean":"2024-04-25T10:05:13.962676+0000","last_became_active":"2024-04-25T10:05:13.953684+0000","last_became_peered":"2024-04-25T10:05:13.953684+0000","last_unstale":"2024-04-25T10:05:13.962676+0000","last_undegraded":"2024-04-25T10:05:13.962676+0000","last_fullsized":"2024-04-25T10:05:13.962676+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:57.379182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.976265+0000","last_change":"2024-04-25T10:05:12.956302+0000","last_active":"2024-04-25T10:05:13.976265+0000","last_peered":"2024-04-25T10:05:13.976265+0000","last_clean":"2024-04-25T10:05:13.976265+0000","last_became_active":"2024-04-25T10:05:12.955985+0000","last_became_peered":"2024-04-25T10:05:12.955985+0000","last_unstale":"2024-04-25T10:05:13.976265+0000","last_undegraded":"2024-04-25T10:05:13.976265+0000","last_fullsized":"2024-04-25T10:05:13.976265+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:19.774500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.955156+0000","last_change":"2024-04-25T10:05:13.952110+0000","last_active":"2024-04-25T10:05:13.955156+0000","last_peered":"2024-04-25T10:05:13.955156+0000","last_clean":"2024-04-25T10:05:13.955156+0000","last_became_active":"2024-04-25T10:05:13.951771+0000","last_became_peered":"2024-04-25T10:05:13.951771+0000","last_unstale":"2024-04-25T10:05:13.955156+0000","last_undegraded":"2024-04-25T10:05:13.955156+0000","last_fullsized":"2024-04-25T10:05:13.955156+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:53.478576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"177'1","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962018+0000","last_change":"2024-04-25T10:05:12.953881+0000","last_active":"2024-04-25T10:05:13.962018+0000","last_peered":"2024-04-25T10:05:13.962018+0000","last_clean":"2024-04-25T10:05:13.962018+0000","last_became_active":"2024-04-25T10:05:12.953463+0000","last_became_peered":"2024-04-25T10:05:12.953463+0000","last_unstale":"2024-04-25T10:05:13.962018+0000","last_undegraded":"2024-04-25T10:05:13.962018+0000","last_fullsized":"2024-04-25T10:05:13.962018+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:05.737767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.951308+0000","last_change":"2024-04-25T10:05:13.949897+0000","last_active":"2024-04-25T10:05:13.951308+0000","last_peered":"2024-04-25T10:05:13.951308+0000","last_clean":"2024-04-25T10:05:13.951308+0000","last_became_active":"2024-04-25T10:05:13.949605+0000","last_became_peered":"2024-04-25T10:05:13.949605+0000","last_unstale":"2024-04-25T10:05:13.951308+0000","last_undegraded":"2024-04-25T10:05:13.951308+0000","last_fullsized":"2024-04-25T10:05:13.951308+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:22.429471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.939404+0000","last_change":"2024-04-25T10:05:12.951447+0000","last_active":"2024-04-25T10:05:13.939404+0000","last_peered":"2024-04-25T10:05:13.939404+0000","last_clean":"2024-04-25T10:05:13.939404+0000","last_became_active":"2024-04-25T10:05:12.951174+0000","last_became_peered":"2024-04-25T10:05:12.951174+0000","last_unstale":"2024-04-25T10:05:13.939404+0000","last_undegraded":"2024-04-25T10:05:13.939404+0000","last_fullsized":"2024-04-25T10:05:13.939404+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:21.289661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.967112+0000","last_change":"2024-04-25T10:05:13.965921+0000","last_active":"2024-04-25T10:05:13.967112+0000","last_peered":"2024-04-25T10:05:13.967112+0000","last_clean":"2024-04-25T10:05:13.967112+0000","last_became_active":"2024-04-25T10:05:13.965580+0000","last_became_peered":"2024-04-25T10:05:13.965580+0000","last_unstale":"2024-04-25T10:05:13.967112+0000","last_undegraded":"2024-04-25T10:05:13.967112+0000","last_fullsized":"2024-04-25T10:05:13.967112+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:42.945698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.941893+0000","last_change":"2024-04-25T10:05:12.943891+0000","last_active":"2024-04-25T10:05:13.941893+0000","last_peered":"2024-04-25T10:05:13.941893+0000","last_clean":"2024-04-25T10:05:13.941893+0000","last_became_active":"2024-04-25T10:05:12.943601+0000","last_became_peered":"2024-04-25T10:05:12.943601+0000","last_unstale":"2024-04-25T10:05:13.941893+0000","last_undegraded":"2024-04-25T10:05:13.941893+0000","last_fullsized":"2024-04-25T10:05:13.941893+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:58.699989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.975912+0000","last_change":"2024-04-25T10:05:13.968875+0000","last_active":"2024-04-25T10:05:13.975912+0000","last_peered":"2024-04-25T10:05:13.975912+0000","last_clean":"2024-04-25T10:05:13.975912+0000","last_became_active":"2024-04-25T10:05:13.968542+0000","last_became_peered":"2024-04-25T10:05:13.968542+0000","last_unstale":"2024-04-25T10:05:13.975912+0000","last_undegraded":"2024-04-25T10:05:13.975912+0000","last_fullsized":"2024-04-25T10:05:13.975912+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:53.069900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.938493+0000","last_change":"2024-04-25T10:05:12.948032+0000","last_active":"2024-04-25T10:05:13.938493+0000","last_peered":"2024-04-25T10:05:13.938493+0000","last_clean":"2024-04-25T10:05:13.938493+0000","last_became_active":"2024-04-25T10:05:12.947736+0000","last_became_peered":"2024-04-25T10:05:12.947736+0000","last_unstale":"2024-04-25T10:05:13.938493+0000","last_undegraded":"2024-04-25T10:05:13.938493+0000","last_fullsized":"2024-04-25T10:05:13.938493+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:25.497796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966338+0000","last_change":"2024-04-25T10:05:13.959472+0000","last_active":"2024-04-25T10:05:13.966338+0000","last_peered":"2024-04-25T10:05:13.966338+0000","last_clean":"2024-04-25T10:05:13.966338+0000","last_became_active":"2024-04-25T10:05:13.958930+0000","last_became_peered":"2024-04-25T10:05:13.958930+0000","last_unstale":"2024-04-25T10:05:13.966338+0000","last_undegraded":"2024-04-25T10:05:13.966338+0000","last_fullsized":"2024-04-25T10:05:13.966338+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:47.924488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.939366+0000","last_change":"2024-04-25T10:05:12.947552+0000","last_active":"2024-04-25T10:05:13.939366+0000","last_peered":"2024-04-25T10:05:13.939366+0000","last_clean":"2024-04-25T10:05:13.939366+0000","last_became_active":"2024-04-25T10:05:12.947269+0000","last_became_peered":"2024-04-25T10:05:12.947269+0000","last_unstale":"2024-04-25T10:05:13.939366+0000","last_undegraded":"2024-04-25T10:05:13.939366+0000","last_fullsized":"2024-04-25T10:05:13.939366+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:43.100356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966541+0000","last_change":"2024-04-25T10:05:13.964972+0000","last_active":"2024-04-25T10:05:13.966541+0000","last_peered":"2024-04-25T10:05:13.966541+0000","last_clean":"2024-04-25T10:05:13.966541+0000","last_became_active":"2024-04-25T10:05:13.964686+0000","last_became_peered":"2024-04-25T10:05:13.964686+0000","last_unstale":"2024-04-25T10:05:13.966541+0000","last_undegraded":"2024-04-25T10:05:13.966541+0000","last_fullsized":"2024-04-25T10:05:13.966541+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:36.670877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"177'3","reported_seq":15,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964127+0000","last_change":"2024-04-25T10:05:12.952574+0000","last_active":"2024-04-25T10:05:13.964127+0000","last_peered":"2024-04-25T10:05:13.964127+0000","last_clean":"2024-04-25T10:05:13.964127+0000","last_became_active":"2024-04-25T10:05:12.951545+0000","last_became_peered":"2024-04-25T10:05:12.951545+0000","last_unstale":"2024-04-25T10:05:13.964127+0000","last_undegraded":"2024-04-25T10:05:13.964127+0000","last_fullsized":"2024-04-25T10:05:13.964127+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:19.907187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962736+0000","last_change":"2024-04-25T10:05:13.960221+0000","last_active":"2024-04-25T10:05:13.962736+0000","last_peered":"2024-04-25T10:05:13.962736+0000","last_clean":"2024-04-25T10:05:13.962736+0000","last_became_active":"2024-04-25T10:05:13.959943+0000","last_became_peered":"2024-04-25T10:05:13.959943+0000","last_unstale":"2024-04-25T10:05:13.962736+0000","last_undegraded":"2024-04-25T10:05:13.962736+0000","last_fullsized":"2024-04-25T10:05:13.962736+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:16.006369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"177'6","reported_seq":18,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.976274+0000","last_change":"2024-04-25T10:05:12.945845+0000","last_active":"2024-04-25T10:05:13.976274+0000","last_peered":"2024-04-25T10:05:13.976274+0000","last_clean":"2024-04-25T10:05:13.976274+0000","last_became_active":"2024-04-25T10:05:12.945557+0000","last_became_peered":"2024-04-25T10:05:12.945557+0000","last_unstale":"2024-04-25T10:05:13.976274+0000","last_undegraded":"2024-04-25T10:05:13.976274+0000","last_fullsized":"2024-04-25T10:05:13.976274+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:01.436152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.967117+0000","last_change":"2024-04-25T10:05:13.965741+0000","last_active":"2024-04-25T10:05:13.967117+0000","last_peered":"2024-04-25T10:05:13.967117+0000","last_clean":"2024-04-25T10:05:13.967117+0000","last_became_active":"2024-04-25T10:05:13.965105+0000","last_became_peered":"2024-04-25T10:05:13.965105+0000","last_unstale":"2024-04-25T10:05:13.967117+0000","last_undegraded":"2024-04-25T10:05:13.967117+0000","last_fullsized":"2024-04-25T10:05:13.967117+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:15.566682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960390+0000","last_change":"2024-04-25T10:05:12.950836+0000","last_active":"2024-04-25T10:05:13.960390+0000","last_peered":"2024-04-25T10:05:13.960390+0000","last_clean":"2024-04-25T10:05:13.960390+0000","last_became_active":"2024-04-25T10:05:12.950534+0000","last_became_peered":"2024-04-25T10:05:12.950534+0000","last_unstale":"2024-04-25T10:05:13.960390+0000","last_undegraded":"2024-04-25T10:05:13.960390+0000","last_fullsized":"2024-04-25T10:05:13.960390+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:32.358510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.968690+0000","last_change":"2024-04-25T10:05:13.968109+0000","last_active":"2024-04-25T10:05:13.968690+0000","last_peered":"2024-04-25T10:05:13.968690+0000","last_clean":"2024-04-25T10:05:13.968690+0000","last_became_active":"2024-04-25T10:05:13.967853+0000","last_became_peered":"2024-04-25T10:05:13.967853+0000","last_unstale":"2024-04-25T10:05:13.968690+0000","last_undegraded":"2024-04-25T10:05:13.968690+0000","last_fullsized":"2024-04-25T10:05:13.968690+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:21.905060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.943954+0000","last_change":"2024-04-25T10:05:12.945794+0000","last_active":"2024-04-25T10:05:13.943954+0000","last_peered":"2024-04-25T10:05:13.943954+0000","last_clean":"2024-04-25T10:05:13.943954+0000","last_became_active":"2024-04-25T10:05:12.945501+0000","last_became_peered":"2024-04-25T10:05:12.945501+0000","last_unstale":"2024-04-25T10:05:13.943954+0000","last_undegraded":"2024-04-25T10:05:13.943954+0000","last_fullsized":"2024-04-25T10:05:13.943954+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:56.557559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962696+0000","last_change":"2024-04-25T10:05:13.961715+0000","last_active":"2024-04-25T10:05:13.962696+0000","last_peered":"2024-04-25T10:05:13.962696+0000","last_clean":"2024-04-25T10:05:13.962696+0000","last_became_active":"2024-04-25T10:05:13.961442+0000","last_became_peered":"2024-04-25T10:05:13.961442+0000","last_unstale":"2024-04-25T10:05:13.962696+0000","last_undegraded":"2024-04-25T10:05:13.962696+0000","last_fullsized":"2024-04-25T10:05:13.962696+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:42.849418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.954806+0000","last_change":"2024-04-25T10:05:12.953254+0000","last_active":"2024-04-25T10:05:13.954806+0000","last_peered":"2024-04-25T10:05:13.954806+0000","last_clean":"2024-04-25T10:05:13.954806+0000","last_became_active":"2024-04-25T10:05:12.952946+0000","last_became_peered":"2024-04-25T10:05:12.952946+0000","last_unstale":"2024-04-25T10:05:13.954806+0000","last_undegraded":"2024-04-25T10:05:13.954806+0000","last_fullsized":"2024-04-25T10:05:13.954806+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:28.587109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.953619+0000","last_change":"2024-04-25T10:05:13.952744+0000","last_active":"2024-04-25T10:05:13.953619+0000","last_peered":"2024-04-25T10:05:13.953619+0000","last_clean":"2024-04-25T10:05:13.953619+0000","last_became_active":"2024-04-25T10:05:13.951443+0000","last_became_peered":"2024-04-25T10:05:13.951443+0000","last_unstale":"2024-04-25T10:05:13.953619+0000","last_undegraded":"2024-04-25T10:05:13.953619+0000","last_fullsized":"2024-04-25T10:05:13.953619+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:06.660471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"177'8","reported_seq":20,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.970113+0000","last_change":"2024-04-25T10:05:12.948493+0000","last_active":"2024-04-25T10:05:13.970113+0000","last_peered":"2024-04-25T10:05:13.970113+0000","last_clean":"2024-04-25T10:05:13.970113+0000","last_became_active":"2024-04-25T10:05:12.947793+0000","last_became_peered":"2024-04-25T10:05:12.947793+0000","last_unstale":"2024-04-25T10:05:13.970113+0000","last_undegraded":"2024-04-25T10:05:13.970113+0000","last_fullsized":"2024-04-25T10:05:13.970113+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:12.718223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962650+0000","last_change":"2024-04-25T10:05:13.956792+0000","last_active":"2024-04-25T10:05:13.962650+0000","last_peered":"2024-04-25T10:05:13.962650+0000","last_clean":"2024-04-25T10:05:13.962650+0000","last_became_active":"2024-04-25T10:05:13.956520+0000","last_became_peered":"2024-04-25T10:05:13.956520+0000","last_unstale":"2024-04-25T10:05:13.962650+0000","last_undegraded":"2024-04-25T10:05:13.962650+0000","last_fullsized":"2024-04-25T10:05:13.962650+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:48.351218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"177'1","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.969221+0000","last_change":"2024-04-25T10:05:12.956205+0000","last_active":"2024-04-25T10:05:13.969221+0000","last_peered":"2024-04-25T10:05:13.969221+0000","last_clean":"2024-04-25T10:05:13.969221+0000","last_became_active":"2024-04-25T10:05:12.955887+0000","last_became_peered":"2024-04-25T10:05:12.955887+0000","last_unstale":"2024-04-25T10:05:13.969221+0000","last_undegraded":"2024-04-25T10:05:13.969221+0000","last_fullsized":"2024-04-25T10:05:13.969221+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:15.066195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962252+0000","last_change":"2024-04-25T10:05:13.960447+0000","last_active":"2024-04-25T10:05:13.962252+0000","last_peered":"2024-04-25T10:05:13.962252+0000","last_clean":"2024-04-25T10:05:13.962252+0000","last_became_active":"2024-04-25T10:05:13.960133+0000","last_became_peered":"2024-04-25T10:05:13.960133+0000","last_unstale":"2024-04-25T10:05:13.962252+0000","last_undegraded":"2024-04-25T10:05:13.962252+0000","last_fullsized":"2024-04-25T10:05:13.962252+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:33.847180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.958024+0000","last_change":"2024-04-25T10:05:12.949358+0000","last_active":"2024-04-25T10:05:13.958024+0000","last_peered":"2024-04-25T10:05:13.958024+0000","last_clean":"2024-04-25T10:05:13.958024+0000","last_became_active":"2024-04-25T10:05:12.949055+0000","last_became_peered":"2024-04-25T10:05:12.949055+0000","last_unstale":"2024-04-25T10:05:13.958024+0000","last_undegraded":"2024-04-25T10:05:13.958024+0000","last_fullsized":"2024-04-25T10:05:13.958024+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:32.054191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.963653+0000","last_change":"2024-04-25T10:05:13.962978+0000","last_active":"2024-04-25T10:05:13.963653+0000","last_peered":"2024-04-25T10:05:13.963653+0000","last_clean":"2024-04-25T10:05:13.963653+0000","last_became_active":"2024-04-25T10:05:13.962710+0000","last_became_peered":"2024-04-25T10:05:13.962710+0000","last_unstale":"2024-04-25T10:05:13.963653+0000","last_undegraded":"2024-04-25T10:05:13.963653+0000","last_fullsized":"2024-04-25T10:05:13.963653+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:03.552292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"177'1","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.960591+0000","last_change":"2024-04-25T10:05:12.954249+0000","last_active":"2024-04-25T10:05:13.960591+0000","last_peered":"2024-04-25T10:05:13.960591+0000","last_clean":"2024-04-25T10:05:13.960591+0000","last_became_active":"2024-04-25T10:05:12.953965+0000","last_became_peered":"2024-04-25T10:05:12.953965+0000","last_unstale":"2024-04-25T10:05:13.960591+0000","last_undegraded":"2024-04-25T10:05:13.960591+0000","last_fullsized":"2024-04-25T10:05:13.960591+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:28.587109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.947369+0000","last_change":"2024-04-25T10:05:13.945229+0000","last_active":"2024-04-25T10:05:13.947369+0000","last_peered":"2024-04-25T10:05:13.947369+0000","last_clean":"2024-04-25T10:05:13.947369+0000","last_became_active":"2024-04-25T10:05:13.944936+0000","last_became_peered":"2024-04-25T10:05:13.944936+0000","last_unstale":"2024-04-25T10:05:13.947369+0000","last_undegraded":"2024-04-25T10:05:13.947369+0000","last_fullsized":"2024-04-25T10:05:13.947369+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:03.245474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.938781+0000","last_change":"2024-04-25T10:05:12.947755+0000","last_active":"2024-04-25T10:05:13.938781+0000","last_peered":"2024-04-25T10:05:13.938781+0000","last_clean":"2024-04-25T10:05:13.938781+0000","last_became_active":"2024-04-25T10:05:12.947420+0000","last_became_peered":"2024-04-25T10:05:12.947420+0000","last_unstale":"2024-04-25T10:05:13.938781+0000","last_undegraded":"2024-04-25T10:05:13.938781+0000","last_fullsized":"2024-04-25T10:05:13.938781+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:24.221953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.965126+0000","last_change":"2024-04-25T10:05:13.959514+0000","last_active":"2024-04-25T10:05:13.965126+0000","last_peered":"2024-04-25T10:05:13.965126+0000","last_clean":"2024-04-25T10:05:13.965126+0000","last_became_active":"2024-04-25T10:05:13.959247+0000","last_became_peered":"2024-04-25T10:05:13.959247+0000","last_unstale":"2024-04-25T10:05:13.965126+0000","last_undegraded":"2024-04-25T10:05:13.965126+0000","last_fullsized":"2024-04-25T10:05:13.965126+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:20.781602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.954324+0000","last_change":"2024-04-25T10:05:12.944909+0000","last_active":"2024-04-25T10:05:13.954324+0000","last_peered":"2024-04-25T10:05:13.954324+0000","last_clean":"2024-04-25T10:05:13.954324+0000","last_became_active":"2024-04-25T10:05:12.943909+0000","last_became_peered":"2024-04-25T10:05:12.943909+0000","last_unstale":"2024-04-25T10:05:13.954324+0000","last_undegraded":"2024-04-25T10:05:13.954324+0000","last_fullsized":"2024-04-25T10:05:13.954324+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:56.824128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962848+0000","last_change":"2024-04-25T10:05:13.961018+0000","last_active":"2024-04-25T10:05:13.962848+0000","last_peered":"2024-04-25T10:05:13.962848+0000","last_clean":"2024-04-25T10:05:13.962848+0000","last_became_active":"2024-04-25T10:05:13.960611+0000","last_became_peered":"2024-04-25T10:05:13.960611+0000","last_unstale":"2024-04-25T10:05:13.962848+0000","last_undegraded":"2024-04-25T10:05:13.962848+0000","last_fullsized":"2024-04-25T10:05:13.962848+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:33.632465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.946423+0000","last_change":"2024-04-25T10:05:12.945654+0000","last_active":"2024-04-25T10:05:13.946423+0000","last_peered":"2024-04-25T10:05:13.946423+0000","last_clean":"2024-04-25T10:05:13.946423+0000","last_became_active":"2024-04-25T10:05:12.945359+0000","last_became_peered":"2024-04-25T10:05:12.945359+0000","last_unstale":"2024-04-25T10:05:13.946423+0000","last_undegraded":"2024-04-25T10:05:13.946423+0000","last_fullsized":"2024-04-25T10:05:13.946423+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:26.765381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966505+0000","last_change":"2024-04-25T10:05:13.964967+0000","last_active":"2024-04-25T10:05:13.966505+0000","last_peered":"2024-04-25T10:05:13.966505+0000","last_clean":"2024-04-25T10:05:13.966505+0000","last_became_active":"2024-04-25T10:05:13.964640+0000","last_became_peered":"2024-04-25T10:05:13.964640+0000","last_unstale":"2024-04-25T10:05:13.966505+0000","last_undegraded":"2024-04-25T10:05:13.966505+0000","last_fullsized":"2024-04-25T10:05:13.966505+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:46.750710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.938086+0000","last_change":"2024-04-25T10:05:12.956001+0000","last_active":"2024-04-25T10:05:13.938086+0000","last_peered":"2024-04-25T10:05:13.938086+0000","last_clean":"2024-04-25T10:05:13.938086+0000","last_became_active":"2024-04-25T10:05:12.955706+0000","last_became_peered":"2024-04-25T10:05:12.955706+0000","last_unstale":"2024-04-25T10:05:13.938086+0000","last_undegraded":"2024-04-25T10:05:13.938086+0000","last_fullsized":"2024-04-25T10:05:13.938086+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:57.064270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964829+0000","last_change":"2024-04-25T10:05:13.961221+0000","last_active":"2024-04-25T10:05:13.964829+0000","last_peered":"2024-04-25T10:05:13.964829+0000","last_clean":"2024-04-25T10:05:13.964829+0000","last_became_active":"2024-04-25T10:05:13.960779+0000","last_became_peered":"2024-04-25T10:05:13.960779+0000","last_unstale":"2024-04-25T10:05:13.964829+0000","last_undegraded":"2024-04-25T10:05:13.964829+0000","last_fullsized":"2024-04-25T10:05:13.964829+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:44.489884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.942752+0000","last_change":"2024-04-25T10:05:12.951562+0000","last_active":"2024-04-25T10:05:13.942752+0000","last_peered":"2024-04-25T10:05:13.942752+0000","last_clean":"2024-04-25T10:05:13.942752+0000","last_became_active":"2024-04-25T10:05:12.951296+0000","last_became_peered":"2024-04-25T10:05:12.951296+0000","last_unstale":"2024-04-25T10:05:13.942752+0000","last_undegraded":"2024-04-25T10:05:13.942752+0000","last_fullsized":"2024-04-25T10:05:13.942752+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:03.238976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.965985+0000","last_change":"2024-04-25T10:05:13.965040+0000","last_active":"2024-04-25T10:05:13.965985+0000","last_peered":"2024-04-25T10:05:13.965985+0000","last_clean":"2024-04-25T10:05:13.965985+0000","last_became_active":"2024-04-25T10:05:13.964591+0000","last_became_peered":"2024-04-25T10:05:13.964591+0000","last_unstale":"2024-04-25T10:05:13.965985+0000","last_undegraded":"2024-04-25T10:05:13.965985+0000","last_fullsized":"2024-04-25T10:05:13.965985+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:58.071372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"177'2","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966798+0000","last_change":"2024-04-25T10:05:12.955761+0000","last_active":"2024-04-25T10:05:13.966798+0000","last_peered":"2024-04-25T10:05:13.966798+0000","last_clean":"2024-04-25T10:05:13.966798+0000","last_became_active":"2024-04-25T10:05:12.955436+0000","last_became_peered":"2024-04-25T10:05:12.955436+0000","last_unstale":"2024-04-25T10:05:13.966798+0000","last_undegraded":"2024-04-25T10:05:13.966798+0000","last_fullsized":"2024-04-25T10:05:13.966798+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:14.559580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966299+0000","last_change":"2024-04-25T10:05:13.959206+0000","last_active":"2024-04-25T10:05:13.966299+0000","last_peered":"2024-04-25T10:05:13.966299+0000","last_clean":"2024-04-25T10:05:13.966299+0000","last_became_active":"2024-04-25T10:05:13.958559+0000","last_became_peered":"2024-04-25T10:05:13.958559+0000","last_unstale":"2024-04-25T10:05:13.966299+0000","last_undegraded":"2024-04-25T10:05:13.966299+0000","last_fullsized":"2024-04-25T10:05:13.966299+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:49.121744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.947211+0000","last_change":"2024-04-25T10:05:12.948445+0000","last_active":"2024-04-25T10:05:13.947211+0000","last_peered":"2024-04-25T10:05:13.947211+0000","last_clean":"2024-04-25T10:05:13.947211+0000","last_became_active":"2024-04-25T10:05:12.948181+0000","last_became_peered":"2024-04-25T10:05:12.948181+0000","last_unstale":"2024-04-25T10:05:13.947211+0000","last_undegraded":"2024-04-25T10:05:13.947211+0000","last_fullsized":"2024-04-25T10:05:13.947211+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:16.651601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.964129+0000","last_change":"2024-04-25T10:05:13.962531+0000","last_active":"2024-04-25T10:05:13.964129+0000","last_peered":"2024-04-25T10:05:13.964129+0000","last_clean":"2024-04-25T10:05:13.964129+0000","last_became_active":"2024-04-25T10:05:13.962152+0000","last_became_peered":"2024-04-25T10:05:13.962152+0000","last_unstale":"2024-04-25T10:05:13.964129+0000","last_undegraded":"2024-04-25T10:05:13.964129+0000","last_fullsized":"2024-04-25T10:05:13.964129+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:30.154935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.937732+0000","last_change":"2024-04-25T10:05:12.953798+0000","last_active":"2024-04-25T10:05:13.937732+0000","last_peered":"2024-04-25T10:05:13.937732+0000","last_clean":"2024-04-25T10:05:13.937732+0000","last_became_active":"2024-04-25T10:05:12.953524+0000","last_became_peered":"2024-04-25T10:05:12.953524+0000","last_unstale":"2024-04-25T10:05:13.937732+0000","last_undegraded":"2024-04-25T10:05:13.937732+0000","last_fullsized":"2024-04-25T10:05:13.937732+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:24.234922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.956903+0000","last_change":"2024-04-25T10:05:13.955422+0000","last_active":"2024-04-25T10:05:13.956903+0000","last_peered":"2024-04-25T10:05:13.956903+0000","last_clean":"2024-04-25T10:05:13.956903+0000","last_became_active":"2024-04-25T10:05:13.955149+0000","last_became_peered":"2024-04-25T10:05:13.955149+0000","last_unstale":"2024-04-25T10:05:13.956903+0000","last_undegraded":"2024-04-25T10:05:13.956903+0000","last_fullsized":"2024-04-25T10:05:13.956903+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:00.472107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.942134+0000","last_change":"2024-04-25T10:05:12.954577+0000","last_active":"2024-04-25T10:05:13.942134+0000","last_peered":"2024-04-25T10:05:13.942134+0000","last_clean":"2024-04-25T10:05:13.942134+0000","last_became_active":"2024-04-25T10:05:12.954112+0000","last_became_peered":"2024-04-25T10:05:12.954112+0000","last_unstale":"2024-04-25T10:05:13.942134+0000","last_undegraded":"2024-04-25T10:05:13.942134+0000","last_fullsized":"2024-04-25T10:05:13.942134+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:39.732169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962634+0000","last_change":"2024-04-25T10:05:13.958351+0000","last_active":"2024-04-25T10:05:13.962634+0000","last_peered":"2024-04-25T10:05:13.962634+0000","last_clean":"2024-04-25T10:05:13.962634+0000","last_became_active":"2024-04-25T10:05:13.957327+0000","last_became_peered":"2024-04-25T10:05:13.957327+0000","last_unstale":"2024-04-25T10:05:13.962634+0000","last_undegraded":"2024-04-25T10:05:13.962634+0000","last_fullsized":"2024-04-25T10:05:13.962634+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:41.738644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.947018+0000","last_change":"2024-04-25T10:05:12.951998+0000","last_active":"2024-04-25T10:05:13.947018+0000","last_peered":"2024-04-25T10:05:13.947018+0000","last_clean":"2024-04-25T10:05:13.947018+0000","last_became_active":"2024-04-25T10:05:12.951734+0000","last_became_peered":"2024-04-25T10:05:12.951734+0000","last_unstale":"2024-04-25T10:05:13.947018+0000","last_undegraded":"2024-04-25T10:05:13.947018+0000","last_fullsized":"2024-04-25T10:05:13.947018+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:02.303005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.953740+0000","last_change":"2024-04-25T10:05:13.949241+0000","last_active":"2024-04-25T10:05:13.953740+0000","last_peered":"2024-04-25T10:05:13.953740+0000","last_clean":"2024-04-25T10:05:13.953740+0000","last_became_active":"2024-04-25T10:05:13.948915+0000","last_became_peered":"2024-04-25T10:05:13.948915+0000","last_unstale":"2024-04-25T10:05:13.953740+0000","last_undegraded":"2024-04-25T10:05:13.953740+0000","last_fullsized":"2024-04-25T10:05:13.953740+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:34.018215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.956251+0000","last_change":"2024-04-25T10:05:12.950171+0000","last_active":"2024-04-25T10:05:13.956251+0000","last_peered":"2024-04-25T10:05:13.956251+0000","last_clean":"2024-04-25T10:05:13.956251+0000","last_became_active":"2024-04-25T10:05:12.949738+0000","last_became_peered":"2024-04-25T10:05:12.949738+0000","last_unstale":"2024-04-25T10:05:13.956251+0000","last_undegraded":"2024-04-25T10:05:13.956251+0000","last_fullsized":"2024-04-25T10:05:13.956251+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:49.510879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.957737+0000","last_change":"2024-04-25T10:05:13.956416+0000","last_active":"2024-04-25T10:05:13.957737+0000","last_peered":"2024-04-25T10:05:13.957737+0000","last_clean":"2024-04-25T10:05:13.957737+0000","last_became_active":"2024-04-25T10:05:13.955891+0000","last_became_peered":"2024-04-25T10:05:13.955891+0000","last_unstale":"2024-04-25T10:05:13.957737+0000","last_undegraded":"2024-04-25T10:05:13.957737+0000","last_fullsized":"2024-04-25T10:05:13.957737+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:43.843086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.943844+0000","last_change":"2024-04-25T10:05:12.954777+0000","last_active":"2024-04-25T10:05:13.943844+0000","last_peered":"2024-04-25T10:05:13.943844+0000","last_clean":"2024-04-25T10:05:13.943844+0000","last_became_active":"2024-04-25T10:05:12.953061+0000","last_became_peered":"2024-04-25T10:05:12.953061+0000","last_unstale":"2024-04-25T10:05:13.943844+0000","last_undegraded":"2024-04-25T10:05:13.943844+0000","last_fullsized":"2024-04-25T10:05:13.943844+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:15.159957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.966169+0000","last_change":"2024-04-25T10:05:13.958218+0000","last_active":"2024-04-25T10:05:13.966169+0000","last_peered":"2024-04-25T10:05:13.966169+0000","last_clean":"2024-04-25T10:05:13.966169+0000","last_became_active":"2024-04-25T10:05:13.957905+0000","last_became_peered":"2024-04-25T10:05:13.957905+0000","last_unstale":"2024-04-25T10:05:13.966169+0000","last_undegraded":"2024-04-25T10:05:13.966169+0000","last_fullsized":"2024-04-25T10:05:13.966169+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:33.632465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.937689+0000","last_change":"2024-04-25T10:05:12.948244+0000","last_active":"2024-04-25T10:05:13.937689+0000","last_peered":"2024-04-25T10:05:13.937689+0000","last_clean":"2024-04-25T10:05:13.937689+0000","last_became_active":"2024-04-25T10:05:12.947552+0000","last_became_peered":"2024-04-25T10:05:12.947552+0000","last_unstale":"2024-04-25T10:05:13.937689+0000","last_undegraded":"2024-04-25T10:05:13.937689+0000","last_fullsized":"2024-04-25T10:05:13.937689+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:32.440271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.961994+0000","last_change":"2024-04-25T10:05:13.960690+0000","last_active":"2024-04-25T10:05:13.961994+0000","last_peered":"2024-04-25T10:05:13.961994+0000","last_clean":"2024-04-25T10:05:13.961994+0000","last_became_active":"2024-04-25T10:05:13.960406+0000","last_became_peered":"2024-04-25T10:05:13.960406+0000","last_unstale":"2024-04-25T10:05:13.961994+0000","last_undegraded":"2024-04-25T10:05:13.961994+0000","last_fullsized":"2024-04-25T10:05:13.961994+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:12.923410+0000","last_clean_scrub_stamp":"2024-04-25T10:05:12.923410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:12.790722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"177'1","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.962369+0000","last_change":"2024-04-25T10:05:12.952811+0000","last_active":"2024-04-25T10:05:13.962369+0000","last_peered":"2024-04-25T10:05:13.962369+0000","last_clean":"2024-04-25T10:05:13.962369+0000","last_became_active":"2024-04-25T10:05:12.952488+0000","last_became_peered":"2024-04-25T10:05:12.952488+0000","last_unstale":"2024-04-25T10:05:13.962369+0000","last_undegraded":"2024-04-25T10:05:13.962369+0000","last_fullsized":"2024-04-25T10:05:13.962369+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:11.916308+0000","last_clean_scrub_stamp":"2024-04-25T10:05:11.916308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:23.930697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":490,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-04-25T10:05:13.938660+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:05:13.938660+0000","last_peered":"2024-04-25T10:05:13.938660+0000","last_clean":"2024-04-25T10:05:13.938660+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:05:13.938660+0000","last_undegraded":"2024-04-25T10:05:13.938660+0000","last_fullsized":"2024-04-25T10:05:13.938660+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51984,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319856,"statfs":{"total":96636764160,"available":96583532544,"internally_reserved":0,"allocated":2818048,"data_stored":1633265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51400,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320440,"statfs":{"total":96636764160,"available":96584130560,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51412,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320428,"statfs":{"total":96636764160,"available":96584118272,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59720,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312120,"statfs":{"total":96636764160,"available":96575610880,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59724,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312116,"statfs":{"total":96636764160,"available":96575606784,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55564,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316276,"statfs":{"total":96636764160,"available":96579866624,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55560,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316280,"statfs":{"total":96636764160,"available":96579870720,"internally_reserved":0,"allocated":2224128,"data_stored":1042897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56140,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315700,"statfs":{"total":96636764160,"available":96579276800,"internally_reserved":0,"allocated":2818048,"data_stored":1633265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:17.299 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:05:17.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:05:17.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.437+0000 7fa076cbd640 1 Processor -- start 2024-04-25T10:05:17.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.437+0000 7fa076cbd640 1 -- start start 2024-04-25T10:05:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa076cbd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa07015dcd0 0x7fa07006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa076cbd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa07006e2f0 0x7fa07006e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa076cbd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0700b3450 con 0x7fa07006b8c0 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa076cbd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0700b36f0 con 0x7fa07006e2f0 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa076cbd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700b39d0 con 0x7fa07015dcd0 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa067fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa07006b8c0 0x7fa0700b44b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37456/0 (socket says 172.21.15.19:37456) 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa067fff640 1 -- 172.21.15.19:0/2612731820 learned_addr learned my addr 172.21.15.19:0/2612731820 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa074a59640 1 --2- 172.21.15.19:0/2612731820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa07015dcd0 0x7fa07006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa07525a640 1 --2- 172.21.15.19:0/2612731820 >> v2:172.21.15.112:3300/0 conn(0x7fa07006e2f0 0x7fa07006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa0677fe640 1 -- 172.21.15.19:0/2612731820 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615899883 0 0) 0x7fa0700b3450 con 0x7fa07006b8c0 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa0677fe640 1 -- 172.21.15.19:0/2612731820 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa058003520 con 0x7fa07006b8c0 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa0677fe640 1 -- 172.21.15.19:0/2612731820 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3441691678 0 0) 0x7fa058003520 con 0x7fa07006b8c0 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa074a59640 1 -- 172.21.15.19:0/2612731820 >> v2:172.21.15.112:3300/0 conn(0x7fa07006e2f0 msgr2=0x7fa07006e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.438+0000 7fa074a59640 1 --2- 172.21.15.19:0/2612731820 >> v2:172.21.15.112:3300/0 conn(0x7fa07006e2f0 0x7fa07006e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.439+0000 7fa074a59640 1 -- 172.21.15.19:0/2612731820 >> v1:172.21.15.19:6789/0 conn(0x7fa07006b8c0 legacy=0x7fa0700b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.439+0000 7fa074a59640 1 -- 172.21.15.19:0/2612731820 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0700b3cb0 con 0x7fa07015dcd0 2024-04-25T10:05:17.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.439+0000 7fa074a59640 1 --2- 172.21.15.19:0/2612731820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa07015dcd0 0x7fa07006d6e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0097a0 tx=0x7fa06c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3779f4717192b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.439+0000 7fa0677fe640 1 -- 172.21.15.19:0/2612731820 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c037070 con 0x7fa07015dcd0 2024-04-25T10:05:17.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.440+0000 7fa0677fe640 1 -- 172.21.15.19:0/2612731820 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c02aba0 con 0x7fa07015dcd0 2024-04-25T10:05:17.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.440+0000 7fa0677fe640 1 -- 172.21.15.19:0/2612731820 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060001670 con 0x7fa07015dcd0 2024-04-25T10:05:17.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.440+0000 7fa076cbd640 1 -- 172.21.15.19:0/2612731820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa07015dcd0 msgr2=0x7fa07006d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.440+0000 7fa076cbd640 1 --2- 172.21.15.19:0/2612731820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa07015dcd0 0x7fa07006d6e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0097a0 tx=0x7fa06c02a110 comp rx=0 tx=0).stop 2024-04-25T10:05:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.440+0000 7fa076cbd640 1 -- 172.21.15.19:0/2612731820 shutdown_connections 2024-04-25T10:05:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.440+0000 7fa076cbd640 1 --2- 172.21.15.19:0/2612731820 >> v2:172.21.15.112:3300/0 conn(0x7fa07006e2f0 0x7fa07006e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.440+0000 7fa076cbd640 1 --2- 172.21.15.19:0/2612731820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa07015dcd0 0x7fa07006d6e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.440+0000 7fa076cbd640 1 -- 172.21.15.19:0/2612731820 >> 172.21.15.19:0/2612731820 conn(0x7fa07006abe0 msgr2=0x7fa07006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.440+0000 7fa076cbd640 1 -- 172.21.15.19:0/2612731820 shutdown_connections 2024-04-25T10:05:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.440+0000 7fa076cbd640 1 -- 172.21.15.19:0/2612731820 wait complete. 2024-04-25T10:05:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.441+0000 7fa076cbd640 1 Processor -- start 2024-04-25T10:05:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.441+0000 7fa076cbd640 1 -- start start 2024-04-25T10:05:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.442+0000 7fa076cbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07006b8c0 0x7fa0700b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.442+0000 7fa076cbd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa07006e2f0 0x7fa0700b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.442+0000 7fa076cbd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07015dcd0 0x7fa07015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.442+0000 7fa076cbd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07006dc20 con 0x7fa07015dcd0 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.442+0000 7fa076cbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700b4bc0 con 0x7fa07006b8c0 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.442+0000 7fa076cbd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07006dda0 con 0x7fa07006e2f0 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.442+0000 7fa07525a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07015dcd0 0x7fa07015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.442+0000 7fa07525a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07015dcd0 0x7fa07015fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38272/0 (socket says 172.21.15.19:38272) 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.442+0000 7fa07525a640 1 -- 172.21.15.19:0/2992517589 learned_addr learned my addr 172.21.15.19:0/2992517589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.442+0000 7fa067fff640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa07006e2f0 0x7fa0700b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.442+0000 7fa074a59640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07006b8c0 0x7fa0700b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa07525a640 1 -- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa07006e2f0 msgr2=0x7fa0700b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa07525a640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa07006e2f0 0x7fa0700b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa07525a640 1 -- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07006b8c0 msgr2=0x7fa0700b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa07525a640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07006b8c0 0x7fa0700b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa07525a640 1 -- 172.21.15.19:0/2992517589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0701609a0 con 0x7fa07015dcd0 2024-04-25T10:05:17.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa074a59640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07006b8c0 0x7fa0700b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:17.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa07525a640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07015dcd0 0x7fa07015fcb0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fa06800efc0 tx=0x7fa06800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa0657fa640 1 -- 172.21.15.19:0/2992517589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068018020 con 0x7fa07015dcd0 2024-04-25T10:05:17.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa0657fa640 1 -- 172.21.15.19:0/2992517589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa068014480 con 0x7fa07015dcd0 2024-04-25T10:05:17.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa076cbd640 1 -- 172.21.15.19:0/2992517589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa070160c80 con 0x7fa07015dcd0 2024-04-25T10:05:17.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa0657fa640 1 -- 172.21.15.19:0/2992517589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068014870 con 0x7fa07015dcd0 2024-04-25T10:05:17.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.443+0000 7fa076cbd640 1 -- 172.21.15.19:0/2992517589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0701498d0 con 0x7fa07015dcd0 2024-04-25T10:05:17.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.444+0000 7fa076cbd640 1 -- 172.21.15.19:0/2992517589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa070149b90 con 0x7fa07015dcd0 2024-04-25T10:05:17.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.445+0000 7fa0657fa640 1 -- 172.21.15.19:0/2992517589 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa068005d10 con 0x7fa07015dcd0 2024-04-25T10:05:17.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.445+0000 7fa0657fa640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05c06b4d0 0x7fa05c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.445+0000 7fa0657fa640 1 -- 172.21.15.19:0/2992517589 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa06808b1c0 con 0x7fa07015dcd0 2024-04-25T10:05:17.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.445+0000 7fa074a59640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05c06b4d0 0x7fa05c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.446+0000 7fa074a59640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05c06b4d0 0x7fa05c06d990 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa06c034000 tx=0x7fa06c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.449+0000 7fa0657fa640 1 -- 172.21.15.19:0/2992517589 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa068022030 con 0x7fa07015dcd0 2024-04-25T10:05:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.694+0000 7fa076cbd640 1 -- 172.21.15.19:0/2992517589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa070073a00 con 0x7fa07015dcd0 2024-04-25T10:05:17.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.696+0000 7fa0657fa640 1 -- 172.21.15.19:0/2992517589 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa068010040 con 0x7fa07015dcd0 2024-04-25T10:05:17.702 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T10:05:17.702 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T10:05:17.702 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T10:05:17.702 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T10:05:17.702 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:17.702 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T10:05:17.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.698+0000 7fa076cbd640 1 -- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05c06b4d0 msgr2=0x7fa05c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.699+0000 7fa076cbd640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05c06b4d0 0x7fa05c06d990 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa06c034000 tx=0x7fa06c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:17.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.699+0000 7fa076cbd640 1 -- 172.21.15.19:0/2992517589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07015dcd0 msgr2=0x7fa07015fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.699+0000 7fa076cbd640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07015dcd0 0x7fa07015fcb0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fa06800efc0 tx=0x7fa06800c490 comp rx=0 tx=0).stop 2024-04-25T10:05:17.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.699+0000 7fa076cbd640 1 -- 172.21.15.19:0/2992517589 shutdown_connections 2024-04-25T10:05:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.699+0000 7fa076cbd640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05c06b4d0 0x7fa05c06d990 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.699+0000 7fa076cbd640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07015dcd0 0x7fa07015fcb0 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.699+0000 7fa076cbd640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa07006e2f0 0x7fa0700b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.699+0000 7fa076cbd640 1 --2- 172.21.15.19:0/2992517589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa07006b8c0 0x7fa0700b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.699+0000 7fa076cbd640 1 -- 172.21.15.19:0/2992517589 >> 172.21.15.19:0/2992517589 conn(0x7fa07006abe0 msgr2=0x7fa0700b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.699+0000 7fa076cbd640 1 -- 172.21.15.19:0/2992517589 shutdown_connections 2024-04-25T10:05:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.699+0000 7fa076cbd640 1 -- 172.21.15.19:0/2992517589 wait complete. 2024-04-25T10:05:17.715 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.810+0000 7f0be98f5640 1 Processor -- start 2024-04-25T10:05:17.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.810+0000 7f0be98f5640 1 -- start start 2024-04-25T10:05:17.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.811+0000 7f0be98f5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0be406abe0 0x7f0be406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.811+0000 7f0be98f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0be415ec20 0x7f0be40b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.811+0000 7f0be98f5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be40b3e10 con 0x7f0be406e430 2024-04-25T10:05:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.811+0000 7f0be98f5640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be406b500 con 0x7f0be406abe0 2024-04-25T10:05:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.811+0000 7f0be98f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be40b3590 con 0x7f0be415ec20 2024-04-25T10:05:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.811+0000 7f0be2ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0be406abe0 0x7f0be406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.811+0000 7f0be37fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0be406e430 0x7f0be406e840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47000/0 (socket says 172.21.15.19:47000) 2024-04-25T10:05:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.811+0000 7f0be37fe640 1 -- 172.21.15.19:0/1815918562 learned_addr learned my addr 172.21.15.19:0/1815918562 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.811+0000 7f0be2ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0be406abe0 0x7f0be406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45874/0 (socket says 172.21.15.19:45874) 2024-04-25T10:05:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.811+0000 7f0be27fc640 1 --2- 172.21.15.19:0/1815918562 >> v2:172.21.15.112:3300/0 conn(0x7f0be415ec20 0x7f0be40b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.812+0000 7f0be2ffd640 1 -- 172.21.15.19:0/1815918562 >> v2:172.21.15.112:3300/0 conn(0x7f0be415ec20 msgr2=0x7f0be40b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.812+0000 7f0be2ffd640 1 --2- 172.21.15.19:0/1815918562 >> v2:172.21.15.112:3300/0 conn(0x7f0be415ec20 0x7f0be40b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.812+0000 7f0be2ffd640 1 -- 172.21.15.19:0/1815918562 >> v1:172.21.15.19:6789/0 conn(0x7f0be406e430 legacy=0x7f0be406e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.812+0000 7f0be2ffd640 1 -- 172.21.15.19:0/1815918562 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be40b3850 con 0x7f0be406abe0 2024-04-25T10:05:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.812+0000 7f0be2ffd640 1 --2- 172.21.15.19:0/1815918562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be406abe0 0x7f0be406afc0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f0be40713c0 tx=0x7f0bdc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29c7a0093ac39a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.812+0000 7f0be1ffb640 1 -- 172.21.15.19:0/1815918562 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc037070 con 0x7f0be406abe0 2024-04-25T10:05:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.812+0000 7f0be1ffb640 1 -- 172.21.15.19:0/1815918562 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc032440 con 0x7f0be406abe0 2024-04-25T10:05:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.812+0000 7f0be1ffb640 1 -- 172.21.15.19:0/1815918562 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc032810 con 0x7f0be406abe0 2024-04-25T10:05:17.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.813+0000 7f0be98f5640 1 -- 172.21.15.19:0/1815918562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be406abe0 msgr2=0x7f0be406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.813+0000 7f0be98f5640 1 --2- 172.21.15.19:0/1815918562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be406abe0 0x7f0be406afc0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f0be40713c0 tx=0x7f0bdc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.813+0000 7f0be98f5640 1 -- 172.21.15.19:0/1815918562 shutdown_connections 2024-04-25T10:05:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.813+0000 7f0be98f5640 1 --2- 172.21.15.19:0/1815918562 >> v2:172.21.15.112:3300/0 conn(0x7f0be415ec20 0x7f0be40b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.813+0000 7f0be98f5640 1 --2- 172.21.15.19:0/1815918562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be406abe0 0x7f0be406afc0 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.813+0000 7f0be98f5640 1 -- 172.21.15.19:0/1815918562 >> 172.21.15.19:0/1815918562 conn(0x7f0be40b5af0 msgr2=0x7f0be40b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.813+0000 7f0be98f5640 1 -- 172.21.15.19:0/1815918562 shutdown_connections 2024-04-25T10:05:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.813+0000 7f0be98f5640 1 -- 172.21.15.19:0/1815918562 wait complete. 2024-04-25T10:05:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.814+0000 7f0be98f5640 1 Processor -- start 2024-04-25T10:05:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.814+0000 7f0be98f5640 1 -- start start 2024-04-25T10:05:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.815+0000 7f0be98f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0be406abe0 0x7f0be40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.815+0000 7f0be98f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be406e430 0x7f0be4148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.815+0000 7f0be98f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be415ec20 0x7f0be41563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.815+0000 7f0be98f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be415e700 con 0x7f0be415ec20 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.815+0000 7f0be98f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be415e420 con 0x7f0be406e430 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.815+0000 7f0be98f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be406ef50 con 0x7f0be406abe0 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.815+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be415ec20 0x7f0be41563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.815+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be415ec20 0x7f0be41563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45880/0 (socket says 172.21.15.19:45880) 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.815+0000 7f0be37fe640 1 -- 172.21.15.19:0/3358445821 learned_addr learned my addr 172.21.15.19:0/3358445821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0be27fc640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be406e430 0x7f0be4148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0be2ffd640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0be406abe0 0x7f0be40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0be37fe640 1 -- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0be406abe0 msgr2=0x7f0be40b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0be37fe640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0be406abe0 0x7f0be40b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0be37fe640 1 -- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be406e430 msgr2=0x7f0be4148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0be37fe640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be406e430 0x7f0be4148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0be37fe640 1 -- 172.21.15.19:0/3358445821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be40b09f0 con 0x7f0be415ec20 2024-04-25T10:05:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0be27fc640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be406e430 0x7f0be4148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0be37fe640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be415ec20 0x7f0be41563b0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400ca80 tx=0x7f0bd400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0bdbfff640 1 -- 172.21.15.19:0/3358445821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4017070 con 0x7f0be415ec20 2024-04-25T10:05:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0bdbfff640 1 -- 172.21.15.19:0/3358445821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd400ac30 con 0x7f0be415ec20 2024-04-25T10:05:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0be98f5640 1 -- 172.21.15.19:0/3358445821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be40af9f0 con 0x7f0be415ec20 2024-04-25T10:05:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0bdbfff640 1 -- 172.21.15.19:0/3358445821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4012400 con 0x7f0be415ec20 2024-04-25T10:05:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.816+0000 7f0be98f5640 1 -- 172.21.15.19:0/3358445821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be4155af0 con 0x7f0be415ec20 2024-04-25T10:05:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.817+0000 7f0be98f5640 1 -- 172.21.15.19:0/3358445821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be406cc80 con 0x7f0be415ec20 2024-04-25T10:05:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.817+0000 7f0bdbfff640 1 -- 172.21.15.19:0/3358445821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bd401a070 con 0x7f0be415ec20 2024-04-25T10:05:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.818+0000 7f0bdbfff640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bc406b4d0 0x7f0bc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.818+0000 7f0bdbfff640 1 -- 172.21.15.19:0/3358445821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bd4089e70 con 0x7f0be415ec20 2024-04-25T10:05:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.818+0000 7f0be2ffd640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bc406b4d0 0x7f0bc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.818+0000 7f0be2ffd640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bc406b4d0 0x7f0bc406d990 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc034000 tx=0x7f0bdc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:17.822+0000 7f0bdbfff640 1 -- 172.21.15.19:0/3358445821 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0be40b3e10 con 0x7f0be415ec20 2024-04-25T10:05:18.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.042+0000 7f0be98f5640 1 -- 172.21.15.19:0/3358445821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0be415b720 con 0x7f0be415ec20 2024-04-25T10:05:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.043+0000 7f0bdbfff640 1 -- 172.21.15.19:0/3358445821 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4948 (secure 0 0 0) 0x7f0bd40585a0 con 0x7f0be415ec20 2024-04-25T10:05:18.048 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:18.048 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":116,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/662861975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":662861975},{"type":"v1","addr":"172.21.15.19:6835","nonce":662861975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3439611393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3439611393},{"type":"v1","addr":"172.21.15.112:6836","nonce":3439611393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721666508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721666508},{"type":"v1","addr":"172.21.15.112:6837","nonce":721666508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:13.300379+0000","modified":"2024-04-25T10:05:15.294971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11166":{"gid":11166,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6833/829980210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":829980210},{"type":"v1","addr":"172.21.15.19:6833","nonce":829980210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:05:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 116 2024-04-25T10:05:18.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 -- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bc406b4d0 msgr2=0x7f0bc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bc406b4d0 0x7f0bc406d990 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc034000 tx=0x7f0bdc02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 -- 172.21.15.19:0/3358445821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be415ec20 msgr2=0x7f0be41563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be415ec20 0x7f0be41563b0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400ca80 tx=0x7f0bd400cf40 comp rx=0 tx=0).stop 2024-04-25T10:05:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 -- 172.21.15.19:0/3358445821 shutdown_connections 2024-04-25T10:05:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0bc406b4d0 0x7f0bc406d990 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be415ec20 0x7f0be41563b0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be406e430 0x7f0be4148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 --2- 172.21.15.19:0/3358445821 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0be406abe0 0x7f0be40b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 -- 172.21.15.19:0/3358445821 >> 172.21.15.19:0/3358445821 conn(0x7f0be40b5af0 msgr2=0x7f0be406e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 -- 172.21.15.19:0/3358445821 shutdown_connections 2024-04-25T10:05:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.045+0000 7f0be98f5640 1 -- 172.21.15.19:0/3358445821 wait complete. 2024-04-25T10:05:18.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 116, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:13.300379+0000', 'modified': '2024-04-25T10:05:15.294971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11166}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11166': {'gid': 11166, 'name': 'c', 'rank': 0, 'incarnation': 114, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6833/829980210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 829980210}, {'type': 'v1', 'addr': '172.21.15.19:6833', 'nonce': 829980210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:05:18.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:05:18.464 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:05:18.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.566+0000 7f0e530bd640 1 Processor -- start 2024-04-25T10:05:18.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.566+0000 7f0e530bd640 1 -- start start 2024-04-25T10:05:18.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e530bd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15dac0 0x7f0e4c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e530bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0d1f30 0x7f0e4c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e530bd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4c0d2420 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e530bd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4c0d3030 con 0x7f0e4c15dac0 2024-04-25T10:05:18.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e530bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0d2700 con 0x7f0e4c0d1f30 2024-04-25T10:05:18.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e5165a640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0e4c0d34c0 0x7f0e4c0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47002/0 (socket says 172.21.15.19:47002) 2024-04-25T10:05:18.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e5165a640 1 -- 172.21.15.19:0/1932761689 learned_addr learned my addr 172.21.15.19:0/1932761689 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:18.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e43fff640 1 --2- 172.21.15.19:0/1932761689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0d1f30 0x7f0e4c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e50e59640 1 --2- 172.21.15.19:0/1932761689 >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15dac0 0x7f0e4c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e437fe640 1 -- 172.21.15.19:0/1932761689 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331830683 0 0) 0x7f0e4c0d2420 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e437fe640 1 -- 172.21.15.19:0/1932761689 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e34003520 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e43fff640 1 -- 172.21.15.19:0/1932761689 >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15dac0 msgr2=0x7f0e4c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e43fff640 1 --2- 172.21.15.19:0/1932761689 >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15dac0 0x7f0e4c0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e43fff640 1 -- 172.21.15.19:0/1932761689 >> v1:172.21.15.19:6789/0 conn(0x7f0e4c0d34c0 legacy=0x7f0e4c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e43fff640 1 -- 172.21.15.19:0/1932761689 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c0d29e0 con 0x7f0e4c0d1f30 2024-04-25T10:05:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.567+0000 7f0e50e59640 1 --2- 172.21.15.19:0/1932761689 >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15dac0 0x7f0e4c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:18.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.568+0000 7f0e43fff640 1 --2- 172.21.15.19:0/1932761689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0d1f30 0x7f0e4c0d03a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009790 tx=0x7f0e3c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0b4f52f2dc9b5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.568+0000 7f0e437fe640 1 -- 172.21.15.19:0/1932761689 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c037070 con 0x7f0e4c0d1f30 2024-04-25T10:05:18.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.569+0000 7f0e437fe640 1 -- 172.21.15.19:0/1932761689 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c032440 con 0x7f0e4c0d1f30 2024-04-25T10:05:18.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.569+0000 7f0e437fe640 1 -- 172.21.15.19:0/1932761689 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0327f0 con 0x7f0e4c0d1f30 2024-04-25T10:05:18.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.569+0000 7f0e530bd640 1 -- 172.21.15.19:0/1932761689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0d1f30 msgr2=0x7f0e4c0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.569+0000 7f0e530bd640 1 --2- 172.21.15.19:0/1932761689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0d1f30 0x7f0e4c0d03a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009790 tx=0x7f0e3c02a210 comp rx=0 tx=0).stop 2024-04-25T10:05:18.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.569+0000 7f0e530bd640 1 -- 172.21.15.19:0/1932761689 shutdown_connections 2024-04-25T10:05:18.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.569+0000 7f0e530bd640 1 --2- 172.21.15.19:0/1932761689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0d1f30 0x7f0e4c0d03a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.569+0000 7f0e530bd640 1 --2- 172.21.15.19:0/1932761689 >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15dac0 0x7f0e4c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.569+0000 7f0e530bd640 1 -- 172.21.15.19:0/1932761689 >> 172.21.15.19:0/1932761689 conn(0x7f0e4c061c30 msgr2=0x7f0e4c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.569+0000 7f0e530bd640 1 -- 172.21.15.19:0/1932761689 shutdown_connections 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.569+0000 7f0e530bd640 1 -- 172.21.15.19:0/1932761689 wait complete. 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.570+0000 7f0e530bd640 1 Processor -- start 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.570+0000 7f0e530bd640 1 -- start start 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e530bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0d1f30 0x7f0e4c14ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e530bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c0d34c0 0x7f0e4c0cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e530bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e4c15dac0 0x7f0e4c0c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e530bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0b97d0 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e530bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c15dea0 con 0x7f0e4c15dac0 2024-04-25T10:05:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e530bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0b9950 con 0x7f0e4c0d1f30 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c0d34c0 0x7f0e4c0cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c0d34c0 0x7f0e4c0cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45886/0 (socket says 172.21.15.19:45886) 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e43fff640 1 -- 172.21.15.19:0/2549081100 learned_addr learned my addr 172.21.15.19:0/2549081100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e5165a640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e4c15dac0 0x7f0e4c0c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e43fff640 1 -- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0d1f30 msgr2=0x7f0e4c14ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.571+0000 7f0e50e59640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0d1f30 0x7f0e4c14ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.572+0000 7f0e43fff640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0d1f30 0x7f0e4c14ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.572+0000 7f0e43fff640 1 -- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e4c15dac0 msgr2=0x7f0e4c0c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.572+0000 7f0e43fff640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e4c15dac0 0x7f0e4c0c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.572+0000 7f0e43fff640 1 -- 172.21.15.19:0/2549081100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c0cbd70 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.572+0000 7f0e5165a640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e4c15dac0 0x7f0e4c0c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.572+0000 7f0e43fff640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c0d34c0 0x7f0e4c0cce10 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c035040 tx=0x7f0e3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.572+0000 7f0e417fa640 1 -- 172.21.15.19:0/2549081100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c037040 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.572+0000 7f0e417fa640 1 -- 172.21.15.19:0/2549081100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c042070 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.572+0000 7f0e417fa640 1 -- 172.21.15.19:0/2549081100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0044e0 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.572+0000 7f0e530bd640 1 -- 172.21.15.19:0/2549081100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4c0cc050 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.572+0000 7f0e530bd640 1 -- 172.21.15.19:0/2549081100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c0cc610 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.573+0000 7f0e417fa640 1 -- 172.21.15.19:0/2549081100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e3c032810 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.574+0000 7f0e417fa640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0e3806b500 0x7f0e3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.574+0000 7f0e417fa640 1 -- 172.21.15.19:0/2549081100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e3c0aabf0 con 0x7f0e4c0d34c0 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.574+0000 7f0e50e59640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0e3806b500 0x7f0e3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.575+0000 7f0e50e59640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0e3806b500 0x7f0e3806d9c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009950 tx=0x7f0e48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.575+0000 7f0e55dc4740 1 Processor -- start 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.575+0000 7f0e55dc4740 1 -- start start 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539d06610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539d24020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539cdbf10 0x55f539d22190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f539c3ea10 con 0x55f539cdbf10 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f539be7b30 con 0x55f539d06230 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f539c3ac50 con 0x55f539d23ad0 2024-04-25T10:05:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e5165a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539cdbf10 0x55f539d22190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e5165a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539cdbf10 0x55f539d22190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45902/0 (socket says 172.21.15.19:45902) 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e5165a640 1 -- 172.21.15.19:0/3246937617 learned_addr learned my addr 172.21.15.19:0/3246937617 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e50e59640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539d06610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e43fff640 1 --2- 172.21.15.19:0/3246937617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539d24020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e5165a640 1 -- 172.21.15.19:0/3246937617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 msgr2=0x55f539d24020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e5165a640 1 --2- 172.21.15.19:0/3246937617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539d24020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e5165a640 1 -- 172.21.15.19:0/3246937617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 msgr2=0x55f539d06610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e5165a640 1 --2- 172.21.15.19:0/3246937617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539d06610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e5165a640 1 -- 172.21.15.19:0/3246937617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f539bfce10 con 0x55f539cdbf10 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e5165a640 1 --2- 172.21.15.19:0/3246937617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539cdbf10 0x55f539d22190 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f0e440021e0 tx=0x7f0e4400d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bacc5fec944d1e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.576+0000 7f0e327fc640 1 -- 172.21.15.19:0/3246937617 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44018070 con 0x55f539cdbf10 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.577+0000 7f0e327fc640 1 -- 172.21.15.19:0/3246937617 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e440074e0 con 0x55f539cdbf10 2024-04-25T10:05:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.577+0000 7f0e327fc640 1 -- 172.21.15.19:0/3246937617 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44007890 con 0x55f539cdbf10 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.577+0000 7f0e55dc4740 1 -- 172.21.15.19:0/3246937617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539cdbf10 msgr2=0x55f539d22190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.577+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/3246937617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539cdbf10 0x55f539d22190 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f0e440021e0 tx=0x7f0e4400d5e0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.577+0000 7f0e55dc4740 1 -- 172.21.15.19:0/3246937617 shutdown_connections 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.577+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/3246937617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539cdbf10 0x55f539d22190 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.577+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/3246937617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539d24020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.577+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/3246937617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539d06610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.577+0000 7f0e55dc4740 1 -- 172.21.15.19:0/3246937617 >> 172.21.15.19:0/3246937617 conn(0x55f539cd8840 msgr2=0x55f539d21040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.577+0000 7f0e55dc4740 1 -- 172.21.15.19:0/3246937617 shutdown_connections 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.577+0000 7f0e55dc4740 1 -- 172.21.15.19:0/3246937617 wait complete. 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.578+0000 7f0e55dc4740 1 -- start start 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.578+0000 7f0e55dc4740 1 Processor -- start 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539d27170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539cd58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539cf0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f539cdccb0 con 0x55f539d09ef0 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f539bfb0c0 con 0x55f539d06230 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f539cd6770 con 0x55f539d23ad0 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e55dc4740 10 client.0 _ll_register_callbacks cb 0x55f539beeec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e5165a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539cf0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e5165a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539cf0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45904/0 (socket says 172.21.15.19:45904) 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e5165a640 1 -- 172.21.15.19:0/1858722538 learned_addr learned my addr 172.21.15.19:0/1858722538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e43fff640 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539cd58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e50e59640 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539d27170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e5165a640 1 -- 172.21.15.19:0/1858722538 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 msgr2=0x55f539cd58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e5165a640 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539cd58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e5165a640 1 -- 172.21.15.19:0/1858722538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 msgr2=0x55f539d27170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e5165a640 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539d27170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e5165a640 1 -- 172.21.15.19:0/1858722538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5399532d0 con 0x55f539d09ef0 2024-04-25T10:05:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e5165a640 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539cf0120 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f0e44016040 tx=0x7f0e44001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa39abaed78a0064 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.579+0000 7f0e2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/1858722538 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44023020 con 0x55f539d09ef0 2024-04-25T10:05:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/1858722538 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e4401f480 con 0x55f539d09ef0 2024-04-25T10:05:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1858722538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e440074e0 con 0x55f539d09ef0 2024-04-25T10:05:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/1858722538 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e4402d050 con 0x55f539d09ef0 2024-04-25T10:05:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e55dc4740 10 client.11313 fetch_fsmap learned FSMap version 116 2024-04-25T10:05:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1858722538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) v3 -- 0x55f539a1d6b0 con 0x55f539d09ef0 2024-04-25T10:05:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/1858722538 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e440074e0 con 0x55f539d09ef0 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/1858722538 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 116) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0e44030020 con 0x55f539d09ef0 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e55dc4740 10 client.11313 fetch_fsmap finished waiting for FSMap version 116 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e55dc4740 10 client.11313 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e55dc4740 20 client.11313 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55f539d34c40 0x55f539d35020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e55dc4740 4 client.11313 mds_command: new command op to 11166 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1858722538 --> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f539c9d920 con 0x55f539d34c40 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.580+0000 7f0e50e59640 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55f539d34c40 0x55f539d35020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e2b7fe640 10 client.11313 ms_handle_connect on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e50e59640 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55f539d34c40 0x55f539d35020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b11f71b1ca47b654 server_cookie=6f0dbf65a2ebfdff in_seq=0 out_seq=0 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/1858722538 <== mds.0 v2:172.21.15.19:6832/829980210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f539c9d920 con 0x55f539d34c40 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e2b7fe640 10 client.11313 handle_command_reply: tid=0 2024-04-25T10:05:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e55dc4740 1 client.11313 shutdown 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1858722538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 msgr2=0x55f539cf0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539cf0120 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f0e44016040 tx=0x7f0e44001250 comp rx=0 tx=0).stop 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1858722538 shutdown_connections 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55f539d34c40 0x55f539d35020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539cf0120 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539cd58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e2b7fe640 0 client.11313 ms_handle_reset on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/1858722538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539d27170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.581+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1858722538 >> 172.21.15.19:0/1858722538 conn(0x55f539cdbf10 msgr2=0x55f539d24e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.582+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1858722538 shutdown_connections 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.582+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1858722538 wait complete. 2024-04-25T10:05:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.582+0000 7f0e55dc4740 20 client.11313 trim_cache size 0 max 16384 2024-04-25T10:05:18.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.597+0000 7f0e55dc4740 1 Processor -- start 2024-04-25T10:05:18.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.597+0000 7f0e55dc4740 1 -- start start 2024-04-25T10:05:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d06230 0x55f539d3b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d23ad0 0x55f539d02450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c1ff520 con 0x55f539d09ef0 2024-04-25T10:05:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0cb390 con 0x55f539d23ad0 2024-04-25T10:05:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c1ff110 con 0x55f539d06230 2024-04-25T10:05:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45918/0 (socket says 172.21.15.19:45918) 2024-04-25T10:05:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e43fff640 1 -- 172.21.15.19:0/1948675281 learned_addr learned my addr 172.21.15.19:0/1948675281 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e50e59640 1 --2- 172.21.15.19:0/1948675281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d06230 0x55f539d3b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e43fff640 1 -- 172.21.15.19:0/1948675281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d06230 msgr2=0x55f539d3b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e5165a640 1 --2- 172.21.15.19:0/1948675281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d23ad0 0x55f539d02450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e43fff640 1 --2- 172.21.15.19:0/1948675281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d06230 0x55f539d3b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e43fff640 1 -- 172.21.15.19:0/1948675281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d23ad0 msgr2=0x55f539d02450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e43fff640 1 --2- 172.21.15.19:0/1948675281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d23ad0 0x55f539d02450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e43fff640 1 -- 172.21.15.19:0/1948675281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f539d22a80 con 0x55f539d09ef0 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e50e59640 1 --2- 172.21.15.19:0/1948675281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d06230 0x55f539d3b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.598+0000 7f0e5165a640 1 --2- 172.21.15.19:0/1948675281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d23ad0 0x55f539d02450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e43fff640 1 --2- 172.21.15.19:0/1948675281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c000c00 tx=0x7f0e3c075460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de88535a1628a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e31ffb640 1 -- 172.21.15.19:0/1948675281 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0a6020 con 0x55f539d09ef0 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e31ffb640 1 -- 172.21.15.19:0/1948675281 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c08e040 con 0x55f539d09ef0 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e31ffb640 1 -- 172.21.15.19:0/1948675281 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0791c0 con 0x55f539d09ef0 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1948675281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 msgr2=0x55f539d05460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/1948675281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c000c00 tx=0x7f0e3c075460 comp rx=0 tx=0).stop 2024-04-25T10:05:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1948675281 shutdown_connections 2024-04-25T10:05:18.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/1948675281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d23ad0 0x55f539d02450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/1948675281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/1948675281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d06230 0x55f539d3b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1948675281 >> 172.21.15.19:0/1948675281 conn(0x55f539cdbf10 msgr2=0x55f539d30990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1948675281 shutdown_connections 2024-04-25T10:05:18.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.599+0000 7f0e55dc4740 1 -- 172.21.15.19:0/1948675281 wait complete. 2024-04-25T10:05:18.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.600+0000 7f0e55dc4740 1 -- start start 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.600+0000 7f0e55dc4740 1 Processor -- start 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539cdd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e55dc4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539d059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f539cef920 con 0x55f539d09ef0 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f539cef0c0 con 0x55f539d06230 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e55dc4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f539cf1580 con 0x55f539d23ad0 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e55dc4740 10 client.0 _ll_register_callbacks cb 0x55f539cd6c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45926/0 (socket says 172.21.15.19:45926) 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e43fff640 1 -- 172.21.15.19:0/4289488407 learned_addr learned my addr 172.21.15.19:0/4289488407 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e5165a640 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539d059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e50e59640 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539cdd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e43fff640 1 -- 172.21.15.19:0/4289488407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 msgr2=0x55f539d059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e43fff640 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539d059a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e43fff640 1 -- 172.21.15.19:0/4289488407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 msgr2=0x55f539cdd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e43fff640 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539cdd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.601+0000 7f0e43fff640 1 -- 172.21.15.19:0/4289488407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f539d8d0d0 con 0x55f539d09ef0 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e43fff640 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c077470 tx=0x7f0e3c009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943b68603f97f0fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/4289488407 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0a6040 con 0x55f539d09ef0 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/4289488407 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e3c079420 con 0x55f539d09ef0 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e55dc4740 1 -- 172.21.15.19:0/4289488407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e3c08e040 con 0x55f539d09ef0 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/4289488407 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e3c03a550 con 0x55f539d09ef0 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e55dc4740 10 client.11319 fetch_fsmap learned FSMap version 116 2024-04-25T10:05:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e55dc4740 1 -- 172.21.15.19:0/4289488407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) v3 -- 0x55f539d02f30 con 0x55f539d09ef0 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/4289488407 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c08e040 con 0x55f539d09ef0 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/4289488407 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 116) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0e3c03a030 con 0x55f539d09ef0 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e55dc4740 10 client.11319 fetch_fsmap finished waiting for FSMap version 116 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e55dc4740 10 client.11319 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e55dc4740 20 client.11319 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55f539d8c450 0x55f539d8c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e55dc4740 4 client.11319 mds_command: new command op to 11166 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e55dc4740 1 -- 172.21.15.19:0/4289488407 --> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f539c4bf50 con 0x55f539d8c450 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.602+0000 7f0e50e59640 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55f539d8c450 0x55f539d8c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e2b7fe640 10 client.11319 ms_handle_connect on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e50e59640 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55f539d8c450 0x55f539d8c830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0722f40cf336281 server_cookie=c606a72aa57a6d66 in_seq=0 out_seq=0 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e2b7fe640 1 -- 172.21.15.19:0/4289488407 <== mds.0 v2:172.21.15.19:6832/829980210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55f539c4bf50 con 0x55f539d8c450 2024-04-25T10:05:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e2b7fe640 10 client.11319 handle_command_reply: tid=0 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 1 client.11319 shutdown 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 1 -- 172.21.15.19:0/4289488407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 msgr2=0x55f539d05460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c077470 tx=0x7f0e3c009110 comp rx=0 tx=0).stop 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 1 -- 172.21.15.19:0/4289488407 shutdown_connections 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55f539d8c450 0x55f539d8c830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f539d23ad0 0x55f539d059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f539d09ef0 0x55f539d05460 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e2b7fe640 0 client.11319 ms_handle_reset on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 1 --2- 172.21.15.19:0/4289488407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f539d06230 0x55f539cdd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 1 -- 172.21.15.19:0/4289488407 >> 172.21.15.19:0/4289488407 conn(0x55f539cdbf10 msgr2=0x55f539d3b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 1 -- 172.21.15.19:0/4289488407 shutdown_connections 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 1 -- 172.21.15.19:0/4289488407 wait complete. 2024-04-25T10:05:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.603+0000 7f0e55dc4740 20 client.11319 trim_cache size 0 max 16384 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 11166, 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 4.6688450619999999, 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 116, 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 178, 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 178, 2024-04-25T10:05:18.616 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 9.6875988339999992 2024-04-25T10:05:18.616 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:05:18.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 -- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0e3806b500 msgr2=0x7f0e3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0e3806b500 0x7f0e3806d9c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009950 tx=0x7f0e48008040 comp rx=0 tx=0).stop 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 -- 172.21.15.19:0/2549081100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c0d34c0 msgr2=0x7f0e4c0cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c0d34c0 0x7f0e4c0cce10 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c035040 tx=0x7f0e3c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 -- 172.21.15.19:0/2549081100 shutdown_connections 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0e3806b500 0x7f0e3806d9c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e4c15dac0 0x7f0e4c0c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e4c0d34c0 0x7f0e4c0cce10 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 --2- 172.21.15.19:0/2549081100 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0d1f30 0x7f0e4c14ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 -- 172.21.15.19:0/2549081100 >> 172.21.15.19:0/2549081100 conn(0x7f0e4c061c30 msgr2=0x7f0e4c0c7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 -- 172.21.15.19:0/2549081100 shutdown_connections 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.604+0000 7f0e530bd640 1 -- 172.21.15.19:0/2549081100 wait complete. 2024-04-25T10:05:18.620 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.714+0000 7f835c8ab640 1 Processor -- start 2024-04-25T10:05:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.715+0000 7f835c8ab640 1 -- start start 2024-04-25T10:05:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.715+0000 7f835c8ab640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f835815db60 0x7f83580ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.715+0000 7f835c8ab640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f83580b5f00 0x7f83580d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.715+0000 7f835c8ab640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83580cec00 con 0x7f835815df40 2024-04-25T10:05:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.715+0000 7f835c8ab640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83580ce930 con 0x7f83580b5f00 2024-04-25T10:05:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.715+0000 7f835c8ab640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83580b6450 con 0x7f835815db60 2024-04-25T10:05:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.715+0000 7f834dd9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f83580b5f00 0x7f83580d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.715+0000 7f8356d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f835815df40 0x7f8358161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47012/0 (socket says 172.21.15.19:47012) 2024-04-25T10:05:18.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.715+0000 7f8356d9d640 1 -- 172.21.15.19:0/3241913513 learned_addr learned my addr 172.21.15.19:0/3241913513 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:18.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.716+0000 7f835659c640 1 --2- 172.21.15.19:0/3241913513 >> v2:172.21.15.112:3300/0 conn(0x7f835815db60 0x7f83580ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.716+0000 7f834dd9b640 1 -- 172.21.15.19:0/3241913513 >> v2:172.21.15.112:3300/0 conn(0x7f835815db60 msgr2=0x7f83580ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.716+0000 7f834dd9b640 1 --2- 172.21.15.19:0/3241913513 >> v2:172.21.15.112:3300/0 conn(0x7f835815db60 0x7f83580ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.716+0000 7f8355d9b640 1 -- 172.21.15.19:0/3241913513 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382068926 0 0) 0x7f83580cec00 con 0x7f835815df40 2024-04-25T10:05:18.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.716+0000 7f834dd9b640 1 -- 172.21.15.19:0/3241913513 >> v1:172.21.15.19:6789/0 conn(0x7f835815df40 legacy=0x7f8358161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.716+0000 7f834dd9b640 1 -- 172.21.15.19:0/3241913513 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83580ceee0 con 0x7f83580b5f00 2024-04-25T10:05:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.716+0000 7f834dd9b640 1 --2- 172.21.15.19:0/3241913513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83580b5f00 0x7f83580d0400 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f8358062440 tx=0x7f834002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b60730704bf9bd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.717+0000 7f8355d9b640 1 -- 172.21.15.19:0/3241913513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340037070 con 0x7f83580b5f00 2024-04-25T10:05:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.717+0000 7f8355d9b640 1 -- 172.21.15.19:0/3241913513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834002ae50 con 0x7f83580b5f00 2024-04-25T10:05:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.717+0000 7f8355d9b640 1 -- 172.21.15.19:0/3241913513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340032660 con 0x7f83580b5f00 2024-04-25T10:05:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.717+0000 7f835c8ab640 1 -- 172.21.15.19:0/3241913513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83580b5f00 msgr2=0x7f83580d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.717+0000 7f835c8ab640 1 --2- 172.21.15.19:0/3241913513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83580b5f00 0x7f83580d0400 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f8358062440 tx=0x7f834002a140 comp rx=0 tx=0).stop 2024-04-25T10:05:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.717+0000 7f835c8ab640 1 -- 172.21.15.19:0/3241913513 shutdown_connections 2024-04-25T10:05:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.717+0000 7f835c8ab640 1 --2- 172.21.15.19:0/3241913513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83580b5f00 0x7f83580d0400 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.717+0000 7f835c8ab640 1 --2- 172.21.15.19:0/3241913513 >> v2:172.21.15.112:3300/0 conn(0x7f835815db60 0x7f83580ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.717+0000 7f835c8ab640 1 -- 172.21.15.19:0/3241913513 >> 172.21.15.19:0/3241913513 conn(0x7f8358061600 msgr2=0x7f8358061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.717+0000 7f835c8ab640 1 -- 172.21.15.19:0/3241913513 shutdown_connections 2024-04-25T10:05:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.717+0000 7f835c8ab640 1 -- 172.21.15.19:0/3241913513 wait complete. 2024-04-25T10:05:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.719+0000 7f835c8ab640 1 Processor -- start 2024-04-25T10:05:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.719+0000 7f835c8ab640 1 -- start start 2024-04-25T10:05:18.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.719+0000 7f835c8ab640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83580b5f00 0x7f83581f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.719+0000 7f835c8ab640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835815db60 0x7f835815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.719+0000 7f835c8ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835815df40 0x7f83580b8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.719+0000 7f835c8ab640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83580d3280 con 0x7f835815db60 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.719+0000 7f835c8ab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83580d1d50 con 0x7f835815df40 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.719+0000 7f835c8ab640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83580d3400 con 0x7f83580b5f00 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835815db60 0x7f835815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835815db60 0x7f835815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45952/0 (socket says 172.21.15.19:45952) 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834dd9b640 1 -- 172.21.15.19:0/674250250 learned_addr learned my addr 172.21.15.19:0/674250250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f835659c640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83580b5f00 0x7f83581f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f8356d9d640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835815df40 0x7f83580b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834dd9b640 1 -- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83580b5f00 msgr2=0x7f83581f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834dd9b640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83580b5f00 0x7f83581f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834dd9b640 1 -- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835815df40 msgr2=0x7f83580b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834dd9b640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835815df40 0x7f83580b8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834dd9b640 1 -- 172.21.15.19:0/674250250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83581f35f0 con 0x7f835815db60 2024-04-25T10:05:18.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834dd9b640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835815db60 0x7f835815e3b0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f8340035040 tx=0x7f83400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834f7fe640 1 -- 172.21.15.19:0/674250250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340041070 con 0x7f835815db60 2024-04-25T10:05:18.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834f7fe640 1 -- 172.21.15.19:0/674250250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834003c460 con 0x7f835815db60 2024-04-25T10:05:18.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f835c8ab640 1 -- 172.21.15.19:0/674250250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83580c8b40 con 0x7f835815db60 2024-04-25T10:05:18.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f834f7fe640 1 -- 172.21.15.19:0/674250250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340037040 con 0x7f835815db60 2024-04-25T10:05:18.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.720+0000 7f835c8ab640 1 -- 172.21.15.19:0/674250250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83580c9100 con 0x7f835815db60 2024-04-25T10:05:18.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.721+0000 7f835c8ab640 1 -- 172.21.15.19:0/674250250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8358073270 con 0x7f835815db60 2024-04-25T10:05:18.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.722+0000 7f834f7fe640 1 -- 172.21.15.19:0/674250250 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f834003a070 con 0x7f835815db60 2024-04-25T10:05:18.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.722+0000 7f834f7fe640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f833806b4d0 0x7f833806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.722+0000 7f834f7fe640 1 -- 172.21.15.19:0/674250250 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83400aa630 con 0x7f835815db60 2024-04-25T10:05:18.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.722+0000 7f835659c640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f833806b4d0 0x7f833806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.723+0000 7f835659c640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f833806b4d0 0x7f833806d990 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8348009b40 tx=0x7f8348009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.726+0000 7f834f7fe640 1 -- 172.21.15.19:0/674250250 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8340078c40 con 0x7f835815db60 2024-04-25T10:05:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.942+0000 7f835c8ab640 1 -- 172.21.15.19:0/674250250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8358073990 con 0x7f835815db60 2024-04-25T10:05:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.943+0000 7f834f7fe640 1 -- 172.21.15.19:0/674250250 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v116) v1 ==== 53+0+73 (secure 0 0 0) 0x7f834007cce0 con 0x7f835815db60 2024-04-25T10:05:18.948 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.944+0000 7f835c8ab640 1 -- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f833806b4d0 msgr2=0x7f833806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.944+0000 7f835c8ab640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f833806b4d0 0x7f833806d990 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8348009b40 tx=0x7f8348009190 comp rx=0 tx=0).stop 2024-04-25T10:05:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.944+0000 7f835c8ab640 1 -- 172.21.15.19:0/674250250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835815db60 msgr2=0x7f835815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.944+0000 7f835c8ab640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835815db60 0x7f835815e3b0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f8340035040 tx=0x7f83400046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.944+0000 7f835c8ab640 1 -- 172.21.15.19:0/674250250 shutdown_connections 2024-04-25T10:05:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.944+0000 7f835c8ab640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f833806b4d0 0x7f833806d990 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.944+0000 7f835c8ab640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835815df40 0x7f83580b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.944+0000 7f835c8ab640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835815db60 0x7f835815e3b0 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.944+0000 7f835c8ab640 1 --2- 172.21.15.19:0/674250250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83580b5f00 0x7f83581f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.944+0000 7f835c8ab640 1 -- 172.21.15.19:0/674250250 >> 172.21.15.19:0/674250250 conn(0x7f8358061600 msgr2=0x7f83580b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.944+0000 7f835c8ab640 1 -- 172.21.15.19:0/674250250 shutdown_connections 2024-04-25T10:05:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:18.945+0000 7f835c8ab640 1 -- 172.21.15.19:0/674250250 wait complete. 2024-04-25T10:05:18.960 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:05:18.960 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:18.961 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T10:05:18.961 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:05:18.961 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:18.961 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:05:18.961 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:05:18.961 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:18.961 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:18.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:18.961 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T10:05:18.977 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:18.977 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:18.977 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:18.977 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:18.977 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:18.977 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:18.977 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:18.977 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:18.977 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T10:05:18.977 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:18.978 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:18.978 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T10:05:18.978 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:18.978 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:18.978 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T10:05:18.978 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:05:18.978 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42027sec preferred_lft 42027sec 2024-04-25T10:05:18.978 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T10:05:18.978 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:18.978 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:18.978 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:18.979 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T10:05:18.979 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:18.979 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:18.979 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:05:18.979 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:18.979 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T10:05:18.979 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:18.979 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:18.980 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:18.980 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T10:05:18.980 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:18.980 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:18.980 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:18.980 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T10:05:18.980 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:18.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:18.980 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:19.040 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T10:05:19.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:19.041 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T10:05:19.098 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T10:05:19.098 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:19.098 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:05:19.098 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:05:19.098 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:19.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:19.098 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T10:05:19.099 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:05:19.099 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:05:19.099 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:05:19.099 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:05:19.099 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:05:19.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:19.183 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:05:19.199 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:19.200 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T10:05:19.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:19.263 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:19.319 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:19.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:19.319 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:19.375 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:05:19.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:19.375 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:05:19.457 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:05:19.457 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:05:19.468 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:05:19.468 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:05:19.468 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T10:05:19.468 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T10:05:19.468 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T10:05:19.468 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:05:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:05:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:05:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:05:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:05:19.471 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:05:19.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:19.471 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:19.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:19.536 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:19.596 INFO:teuthology.orchestra.run.smithi019.stdout:client11331 2024-04-25T10:05:19.597 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:19.597 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:19.652 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:19.653 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:19.708 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> import uuid 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> n = 1000 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> create_hard_links = False 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> if 0 > 0: 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> create_hard_links = True 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> path = "/home/ubuntu/cephtest/mnt.0/scrub-new-files/file" 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> dpath = os.path.dirname(path) 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> fnameprefix = os.path.basename(path) 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T10:05:19.709 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> for i in range(n): 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> f.write(f"{i}") 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> f.flush() 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> os.fsync(f.fileno()) 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> os.unlink(fpath) 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> os.fsync(dirfd) 2024-04-25T10:05:19.710 DEBUG:teuthology.orchestra.run.smithi019:> if create_hard_links: 2024-04-25T10:05:19.711 DEBUG:teuthology.orchestra.run.smithi019:> for j in range(0): 2024-04-25T10:05:19.711 DEBUG:teuthology.orchestra.run.smithi019:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:05:19.711 DEBUG:teuthology.orchestra.run.smithi019:> if False: 2024-04-25T10:05:19.711 DEBUG:teuthology.orchestra.run.smithi019:> os.fsync(dirfd) 2024-04-25T10:05:19.711 DEBUG:teuthology.orchestra.run.smithi019:> finally: 2024-04-25T10:05:19.711 DEBUG:teuthology.orchestra.run.smithi019:> os.close(dirfd) 2024-04-25T10:05:19.711 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T10:05:19.923 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T10:05:19.918+0000 7fe4cd160640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:05:20.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:20.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-25T10:05:20.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.173+0000 7f58fa815640 1 Processor -- start 2024-04-25T10:05:20.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.173+0000 7f58fa815640 1 -- start start 2024-04-25T10:05:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.174+0000 7f58fa815640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58f40ced80 0x7f58f40d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.174+0000 7f58fa815640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f58f4161710 0x7f58f40d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.174+0000 7f58fa815640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f415e000 con 0x7f58f415dc20 2024-04-25T10:05:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.174+0000 7f58fa815640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f40d1de0 con 0x7f58f40ced80 2024-04-25T10:05:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.174+0000 7f58fa815640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f4161c60 con 0x7f58f4161710 2024-04-25T10:05:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.174+0000 7f58f37fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58f40ced80 0x7f58f40d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.174+0000 7f58f37fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58f40ced80 0x7f58f40d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45956/0 (socket says 172.21.15.19:45956) 2024-04-25T10:05:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.174+0000 7f58f37fe640 1 -- 172.21.15.19:0/1864589425 learned_addr learned my addr 172.21.15.19:0/1864589425 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.175+0000 7f58f8db2640 1 --2- 172.21.15.19:0/1864589425 >> v2:172.21.15.112:3300/0 conn(0x7f58f4161710 0x7f58f40d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.175+0000 7f58f37fe640 1 -- 172.21.15.19:0/1864589425 >> v2:172.21.15.112:3300/0 conn(0x7f58f4161710 msgr2=0x7f58f40d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.175+0000 7f58f37fe640 1 --2- 172.21.15.19:0/1864589425 >> v2:172.21.15.112:3300/0 conn(0x7f58f4161710 0x7f58f40d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.175+0000 7f58f37fe640 1 -- 172.21.15.19:0/1864589425 >> v1:172.21.15.112:6789/0 conn(0x7f58f415dc20 legacy=0x7f58f40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.175+0000 7f58f37fe640 1 -- 172.21.15.19:0/1864589425 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4161f40 con 0x7f58f40ced80 2024-04-25T10:05:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.175+0000 7f58f37fe640 1 --2- 172.21.15.19:0/1864589425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f40ced80 0x7f58f40d22e0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f58f40b6570 tx=0x7f58e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f616600095b367e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.176+0000 7f58f2ffd640 1 -- 172.21.15.19:0/1864589425 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8037070 con 0x7f58f40ced80 2024-04-25T10:05:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.176+0000 7f58f2ffd640 1 -- 172.21.15.19:0/1864589425 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e8032440 con 0x7f58f40ced80 2024-04-25T10:05:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.176+0000 7f58f2ffd640 1 -- 172.21.15.19:0/1864589425 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8032810 con 0x7f58f40ced80 2024-04-25T10:05:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.176+0000 7f58fa815640 1 -- 172.21.15.19:0/1864589425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f40ced80 msgr2=0x7f58f40d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.176+0000 7f58fa815640 1 --2- 172.21.15.19:0/1864589425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f40ced80 0x7f58f40d22e0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f58f40b6570 tx=0x7f58e802a230 comp rx=0 tx=0).stop 2024-04-25T10:05:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.176+0000 7f58fa815640 1 -- 172.21.15.19:0/1864589425 shutdown_connections 2024-04-25T10:05:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.176+0000 7f58fa815640 1 --2- 172.21.15.19:0/1864589425 >> v2:172.21.15.112:3300/0 conn(0x7f58f4161710 0x7f58f40d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.176+0000 7f58fa815640 1 --2- 172.21.15.19:0/1864589425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f40ced80 0x7f58f40d22e0 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.176+0000 7f58fa815640 1 -- 172.21.15.19:0/1864589425 >> 172.21.15.19:0/1864589425 conn(0x7f58f40d0420 msgr2=0x7f58f40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.176+0000 7f58fa815640 1 -- 172.21.15.19:0/1864589425 shutdown_connections 2024-04-25T10:05:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.176+0000 7f58fa815640 1 -- 172.21.15.19:0/1864589425 wait complete. 2024-04-25T10:05:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.177+0000 7f58fa815640 1 Processor -- start 2024-04-25T10:05:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.178+0000 7f58fa815640 1 -- start start 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.178+0000 7f58fa815640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58f40ced80 0x7f58f41534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.178+0000 7f58fa815640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f415dc20 0x7f58f4156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.178+0000 7f58fa815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f4161710 0x7f58f40c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.178+0000 7f58fa815640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40b5e50 con 0x7f58f415dc20 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.178+0000 7f58fa815640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f4162500 con 0x7f58f4161710 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.178+0000 7f58fa815640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40b5fd0 con 0x7f58f40ced80 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.178+0000 7f58f37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f415dc20 0x7f58f4156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.178+0000 7f58f37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f415dc20 0x7f58f4156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45964/0 (socket says 172.21.15.19:45964) 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.178+0000 7f58f37fe640 1 -- 172.21.15.19:0/3613191755 learned_addr learned my addr 172.21.15.19:0/3613191755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58f3fff640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58f40ced80 0x7f58f41534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58f8db2640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f4161710 0x7f58f40c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58f37fe640 1 -- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58f40ced80 msgr2=0x7f58f41534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58f37fe640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58f40ced80 0x7f58f41534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58f37fe640 1 -- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f4161710 msgr2=0x7f58f40c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58f37fe640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f4161710 0x7f58f40c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58f37fe640 1 -- 172.21.15.19:0/3613191755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4156650 con 0x7f58f415dc20 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58f37fe640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f415dc20 0x7f58f4156110 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f58e8000c00 tx=0x7f58e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58f0ff9640 1 -- 172.21.15.19:0/3613191755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8041020 con 0x7f58f415dc20 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58f0ff9640 1 -- 172.21.15.19:0/3613191755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e80306c0 con 0x7f58f415dc20 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58fa815640 1 -- 172.21.15.19:0/3613191755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f40c9cf0 con 0x7f58f415dc20 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58f0ff9640 1 -- 172.21.15.19:0/3613191755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8037040 con 0x7f58f415dc20 2024-04-25T10:05:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.179+0000 7f58fa815640 1 -- 172.21.15.19:0/3613191755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f415e9f0 con 0x7f58f415dc20 2024-04-25T10:05:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.181+0000 7f58f0ff9640 1 -- 172.21.15.19:0/3613191755 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58e8044020 con 0x7f58f415dc20 2024-04-25T10:05:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.181+0000 7f58f0ff9640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58e006b4d0 0x7f58e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.181+0000 7f58f0ff9640 1 -- 172.21.15.19:0/3613191755 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58e80aac10 con 0x7f58f415dc20 2024-04-25T10:05:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.181+0000 7f58f3fff640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58e006b4d0 0x7f58e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.182+0000 7f58fd5f8740 1 Processor -- start 2024-04-25T10:05:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.182+0000 7f58f3fff640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58e006b4d0 0x7f58e006d990 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f58e400a750 tx=0x7f58e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.182+0000 7f58fd5f8740 1 -- start start 2024-04-25T10:05:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556974537b10 0x55697453a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x55697453a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55697450a770 0x55697450df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x556974473da0 con 0x556974537ef0 2024-04-25T10:05:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697441cb80 con 0x556974537b10 2024-04-25T10:05:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55697446fe40 con 0x55697450a770 2024-04-25T10:05:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58f37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x55697453a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58f3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556974537b10 0x55697453a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58f37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x55697453a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45966/0 (socket says 172.21.15.19:45966) 2024-04-25T10:05:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58f37fe640 1 -- 172.21.15.19:0/86057363 learned_addr learned my addr 172.21.15.19:0/86057363 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58f8db2640 1 --2- 172.21.15.19:0/86057363 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55697450a770 0x55697450df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58f37fe640 1 -- 172.21.15.19:0/86057363 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55697450a770 msgr2=0x55697450df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58f37fe640 1 --2- 172.21.15.19:0/86057363 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55697450a770 0x55697450df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58f37fe640 1 -- 172.21.15.19:0/86057363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556974537b10 msgr2=0x55697453a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58f37fe640 1 --2- 172.21.15.19:0/86057363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556974537b10 0x55697453a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58f37fe640 1 -- 172.21.15.19:0/86057363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556974426f10 con 0x556974537ef0 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58f37fe640 1 --2- 172.21.15.19:0/86057363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x55697453a860 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f58e8079190 tx=0x7f58e8079900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f40b4b5e52f2962 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58d9ffb640 1 -- 172.21.15.19:0/86057363 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8079200 con 0x556974537ef0 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58d9ffb640 1 -- 172.21.15.19:0/86057363 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e807d350 con 0x556974537ef0 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.183+0000 7f58d9ffb640 1 -- 172.21.15.19:0/86057363 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8081910 con 0x556974537ef0 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.184+0000 7f58fd5f8740 1 -- 172.21.15.19:0/86057363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 msgr2=0x55697453a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.184+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/86057363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x55697453a860 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f58e8079190 tx=0x7f58e8079900 comp rx=0 tx=0).stop 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.184+0000 7f58fd5f8740 1 -- 172.21.15.19:0/86057363 shutdown_connections 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.184+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/86057363 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55697450a770 0x55697450df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.184+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/86057363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x55697453a860 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.184+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/86057363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556974537b10 0x55697453a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.184+0000 7f58fd5f8740 1 -- 172.21.15.19:0/86057363 >> 172.21.15.19:0/86057363 conn(0x55697450c9e0 msgr2=0x556974555320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.184+0000 7f58fd5f8740 1 -- 172.21.15.19:0/86057363 shutdown_connections 2024-04-25T10:05:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.184+0000 7f58fd5f8740 1 -- 172.21.15.19:0/86057363 wait complete. 2024-04-25T10:05:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.185+0000 7f58fd5f8740 1 -- start start 2024-04-25T10:05:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.185+0000 7f58fd5f8740 1 Processor -- start 2024-04-25T10:05:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537b10 0x556974529ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x556974558cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55697453b2e0 0x55697450ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x556974473da0 con 0x556974537ef0 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569744124b0 con 0x55697453b2e0 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55697450fb90 con 0x556974537b10 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58fd5f8740 10 client.0 _ll_register_callbacks cb 0x556974452180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x556974558cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x556974558cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45970/0 (socket says 172.21.15.19:45970) 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f37fe640 1 -- 172.21.15.19:0/3594889226 learned_addr learned my addr 172.21.15.19:0/3594889226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f3fff640 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537b10 0x556974529ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f8db2640 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55697453b2e0 0x55697450ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f37fe640 1 -- 172.21.15.19:0/3594889226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537b10 msgr2=0x556974529ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f37fe640 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537b10 0x556974529ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f37fe640 1 -- 172.21.15.19:0/3594889226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55697453b2e0 msgr2=0x55697450ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f37fe640 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55697453b2e0 0x55697450ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f37fe640 1 -- 172.21.15.19:0/3594889226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569742527d0 con 0x556974537ef0 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f3fff640 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537b10 0x556974529ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58f37fe640 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x556974558cb0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f58e8032040 tx=0x7f58e8008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fb78a98b71ef65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3594889226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8030c90 con 0x556974537ef0 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3594889226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58e8081480 con 0x556974537ef0 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3594889226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58e807d350 con 0x556974537ef0 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.186+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3594889226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58e809e050 con 0x556974537ef0 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58fd5f8740 10 client.11343 fetch_fsmap learned FSMap version 116 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3594889226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) v3 -- 0x5569741883f0 con 0x556974537ef0 2024-04-25T10:05:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3594889226 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e807d350 con 0x556974537ef0 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3594889226 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 116) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f58e80a1020 con 0x556974537ef0 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58fd5f8740 10 client.11343 fetch_fsmap finished waiting for FSMap version 116 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58fd5f8740 10 client.11343 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58fd5f8740 20 client.11343 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556974550b90 0x556974550fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58fd5f8740 4 client.11343 mds_command: new command op to 11166 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3594889226 --> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569744812e0 con 0x556974550b90 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58f3fff640 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556974550b90 0x556974550fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58d2ffd640 10 client.11343 ms_handle_connect on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.187+0000 7f58f3fff640 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556974550b90 0x556974550fa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a87d7fbc6ca71c02 server_cookie=d16513f84073569c in_seq=0 out_seq=0 2024-04-25T10:05:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3594889226 <== mds.0 v2:172.21.15.19:6832/829980210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5569744812e0 con 0x556974550b90 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58d2ffd640 10 client.11343 handle_command_reply: tid=0 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 1 client.11343 shutdown 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3594889226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 msgr2=0x556974558cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x556974558cb0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f58e8032040 tx=0x7f58e8008070 comp rx=0 tx=0).stop 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3594889226 shutdown_connections 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556974550b90 0x556974550fa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55697453b2e0 0x55697450ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x556974558cb0 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3594889226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537b10 0x556974529ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58d2ffd640 0 client.11343 ms_handle_reset on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3594889226 >> 172.21.15.19:0/3594889226 conn(0x55697450a770 msgr2=0x556974536f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3594889226 shutdown_connections 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3594889226 wait complete. 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.188+0000 7f58fd5f8740 20 client.11343 trim_cache size 0 max 16384 2024-04-25T10:05:20.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.203+0000 7f58fd5f8740 1 Processor -- start 2024-04-25T10:05:20.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.203+0000 7f58fd5f8740 1 -- start start 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537b10 0x556974555b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537ef0 0x55697450ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55697453b2e0 0x556974509600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f42006d0 con 0x556974537b10 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41ff870 con 0x55697453b2e0 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40b4160 con 0x556974537ef0 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537b10 0x556974555b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537b10 0x556974555b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45974/0 (socket says 172.21.15.19:45974) 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f3fff640 1 -- 172.21.15.19:0/2863780452 learned_addr learned my addr 172.21.15.19:0/2863780452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f8db2640 1 --2- 172.21.15.19:0/2863780452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55697453b2e0 0x556974509600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f37fe640 1 --2- 172.21.15.19:0/2863780452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537ef0 0x55697450ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f3fff640 1 -- 172.21.15.19:0/2863780452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537ef0 msgr2=0x55697450ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f3fff640 1 --2- 172.21.15.19:0/2863780452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537ef0 0x55697450ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f3fff640 1 -- 172.21.15.19:0/2863780452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55697453b2e0 msgr2=0x556974509600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f3fff640 1 --2- 172.21.15.19:0/2863780452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55697453b2e0 0x556974509600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f3fff640 1 -- 172.21.15.19:0/2863780452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569741480f0 con 0x556974537b10 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f8db2640 1 --2- 172.21.15.19:0/2863780452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55697453b2e0 0x556974509600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.204+0000 7f58f37fe640 1 --2- 172.21.15.19:0/2863780452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537ef0 0x55697450ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58f3fff640 1 --2- 172.21.15.19:0/2863780452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537b10 0x556974555b00 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x556974188540 tx=0x7f58e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a96d4ff1e9a1659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58d97fa640 1 -- 172.21.15.19:0/2863780452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4019040 con 0x556974537b10 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58d97fa640 1 -- 172.21.15.19:0/2863780452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e402b070 con 0x556974537b10 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58d97fa640 1 -- 172.21.15.19:0/2863780452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4021d30 con 0x556974537b10 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58fd5f8740 1 -- 172.21.15.19:0/2863780452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537b10 msgr2=0x556974555b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/2863780452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537b10 0x556974555b00 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x556974188540 tx=0x7f58e4013000 comp rx=0 tx=0).stop 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58fd5f8740 1 -- 172.21.15.19:0/2863780452 shutdown_connections 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/2863780452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55697453b2e0 0x556974509600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/2863780452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556974537ef0 0x55697450ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/2863780452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537b10 0x556974555b00 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58fd5f8740 1 -- 172.21.15.19:0/2863780452 >> 172.21.15.19:0/2863780452 conn(0x55697450a770 msgr2=0x55697452de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58fd5f8740 1 -- 172.21.15.19:0/2863780452 shutdown_connections 2024-04-25T10:05:20.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.205+0000 7f58fd5f8740 1 -- 172.21.15.19:0/2863780452 wait complete. 2024-04-25T10:05:20.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.206+0000 7f58fd5f8740 1 -- start start 2024-04-25T10:05:20.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.206+0000 7f58fd5f8740 1 Processor -- start 2024-04-25T10:05:20.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556974537b10 0x5569745779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x5569745be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58fd5f8740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55697453b2e0 0x556974558230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697452e5f0 con 0x556974537ef0 2024-04-25T10:05:20.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41ff270 con 0x556974537b10 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58fd5f8740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55697453ae30 con 0x55697453b2e0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58fd5f8740 10 client.0 _ll_register_callbacks cb 0x7f58e8009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58f37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x5569745be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58f37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x5569745be900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45978/0 (socket says 172.21.15.19:45978) 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58f37fe640 1 -- 172.21.15.19:0/3330367852 learned_addr learned my addr 172.21.15.19:0/3330367852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58f3fff640 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556974537b10 0x5569745779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58f8db2640 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55697453b2e0 0x556974558230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58f37fe640 1 -- 172.21.15.19:0/3330367852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55697453b2e0 msgr2=0x556974558230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58f37fe640 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55697453b2e0 0x556974558230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58f37fe640 1 -- 172.21.15.19:0/3330367852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556974537b10 msgr2=0x5569745779f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58f37fe640 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556974537b10 0x5569745779f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58f37fe640 1 -- 172.21.15.19:0/3330367852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55697454e8f0 con 0x556974537ef0 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58f37fe640 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x5569745be900 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f58e80813e0 tx=0x7f58e8075520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f59f34097a0665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.207+0000 7f58d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3330367852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8086060 con 0x556974537ef0 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3330367852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58e8081b60 con 0x556974537ef0 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3330367852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58e402b070 con 0x556974537ef0 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3330367852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58e808c050 con 0x556974537ef0 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58fd5f8740 10 client.11349 fetch_fsmap learned FSMap version 116 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3330367852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) v3 -- 0x55697454e8f0 con 0x556974537ef0 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3330367852 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e402b070 con 0x556974537ef0 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3330367852 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 116) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f58e80a1040 con 0x556974537ef0 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58fd5f8740 10 client.11349 fetch_fsmap finished waiting for FSMap version 116 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58fd5f8740 10 client.11349 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58fd5f8740 20 client.11349 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55697454b510 0x5569745641b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58fd5f8740 4 client.11349 mds_command: new command op to 11166 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3330367852 --> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55697447b630 con 0x55697454b510 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.208+0000 7f58f3fff640 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55697454b510 0x5569745641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58d2ffd640 10 client.11349 ms_handle_connect on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58f3fff640 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55697454b510 0x5569745641b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6206efa062a1d10b server_cookie=e08b0c1f0f69166c in_seq=0 out_seq=0 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58d2ffd640 1 -- 172.21.15.19:0/3330367852 <== mds.0 v2:172.21.15.19:6832/829980210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55697447b630 con 0x55697454b510 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58d2ffd640 10 client.11349 handle_command_reply: tid=0 2024-04-25T10:05:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 1 client.11349 shutdown 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3330367852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 msgr2=0x5569745be900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x5569745be900 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f58e80813e0 tx=0x7f58e8075520 comp rx=0 tx=0).stop 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3330367852 shutdown_connections 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x55697454b510 0x5569745641b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55697453b2e0 0x556974558230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556974537ef0 0x5569745be900 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 1 --2- 172.21.15.19:0/3330367852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556974537b10 0x5569745779f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58d2ffd640 0 client.11349 ms_handle_reset on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3330367852 >> 172.21.15.19:0/3330367852 conn(0x55697450a770 msgr2=0x55697450fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3330367852 shutdown_connections 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 1 -- 172.21.15.19:0/3330367852 wait complete. 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.209+0000 7f58fd5f8740 20 client.11349 trim_cache size 0 max 16384 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 -- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58e006b4d0 msgr2=0x7f58e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58e006b4d0 0x7f58e006d990 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f58e400a750 tx=0x7f58e4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 -- 172.21.15.19:0/3613191755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f415dc20 msgr2=0x7f58f4156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f415dc20 0x7f58f4156110 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f58e8000c00 tx=0x7f58e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 -- 172.21.15.19:0/3613191755 shutdown_connections 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f58e006b4d0 0x7f58e006d990 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f4161710 0x7f58f40c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58f415dc20 0x7f58f4156110 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 --2- 172.21.15.19:0/3613191755 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58f40ced80 0x7f58f41534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 -- 172.21.15.19:0/3613191755 >> 172.21.15.19:0/3613191755 conn(0x7f58f40d0420 msgr2=0x7f58f40b76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 -- 172.21.15.19:0/3613191755 shutdown_connections 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.210+0000 7f58fa815640 1 -- 172.21.15.19:0/3613191755 wait complete. 2024-04-25T10:05:20.226 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:05:20.226 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:05:20.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 damage ls 2024-04-25T10:05:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.325+0000 7ffba5fce640 1 Processor -- start 2024-04-25T10:05:20.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.325+0000 7ffba5fce640 1 -- start start 2024-04-25T10:05:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.325+0000 7ffba5fce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffba0161710 0x7ffba00d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.325+0000 7ffba5fce640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba015e000 con 0x7ffba015dc20 2024-04-25T10:05:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.325+0000 7ffba5fce640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba00d1de0 con 0x7ffba00ced80 2024-04-25T10:05:20.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.325+0000 7ffba5fce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba0161c60 con 0x7ffba0161710 2024-04-25T10:05:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.326+0000 7ffb9f7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ffba015dc20 0x7ffba00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47022/0 (socket says 172.21.15.19:47022) 2024-04-25T10:05:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.326+0000 7ffb9f7fe640 1 -- 172.21.15.19:0/317402939 learned_addr learned my addr 172.21.15.19:0/317402939 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.326+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/317402939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffba0161710 0x7ffba00d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.326+0000 7ffb9e7fc640 1 -- 172.21.15.19:0/317402939 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244836156 0 0) 0x7ffba015e000 con 0x7ffba015dc20 2024-04-25T10:05:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.326+0000 7ffb9e7fc640 1 -- 172.21.15.19:0/317402939 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb88003520 con 0x7ffba015dc20 2024-04-25T10:05:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.326+0000 7ffb9e7fc640 1 -- 172.21.15.19:0/317402939 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160448596 0 0) 0x7ffba00d1de0 con 0x7ffba00ced80 2024-04-25T10:05:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.326+0000 7ffb9ffff640 1 -- 172.21.15.19:0/317402939 >> v1:172.21.15.112:6789/0 conn(0x7ffba00ced80 legacy=0x7ffba00d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.326+0000 7ffb9ffff640 1 -- 172.21.15.19:0/317402939 >> v1:172.21.15.19:6789/0 conn(0x7ffba015dc20 legacy=0x7ffba00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.326+0000 7ffb9ffff640 1 -- 172.21.15.19:0/317402939 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba0161f40 con 0x7ffba0161710 2024-04-25T10:05:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.327+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/317402939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffba0161710 0x7ffba00d29f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009dc0 tx=0x7ffb9802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f11893f944ee507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.327+0000 7ffb9e7fc640 1 -- 172.21.15.19:0/317402939 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98007740 con 0x7ffba0161710 2024-04-25T10:05:20.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.327+0000 7ffb9e7fc640 1 -- 172.21.15.19:0/317402939 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb980078e0 con 0x7ffba0161710 2024-04-25T10:05:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.327+0000 7ffb9e7fc640 1 -- 172.21.15.19:0/317402939 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98007c90 con 0x7ffba0161710 2024-04-25T10:05:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.327+0000 7ffba5fce640 1 -- 172.21.15.19:0/317402939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffba0161710 msgr2=0x7ffba00d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.328+0000 7ffba5fce640 1 --2- 172.21.15.19:0/317402939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffba0161710 0x7ffba00d29f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009dc0 tx=0x7ffb9802a6d0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.328+0000 7ffba5fce640 1 -- 172.21.15.19:0/317402939 shutdown_connections 2024-04-25T10:05:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.328+0000 7ffba5fce640 1 --2- 172.21.15.19:0/317402939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffba0161710 0x7ffba00d29f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.328+0000 7ffba5fce640 1 -- 172.21.15.19:0/317402939 >> 172.21.15.19:0/317402939 conn(0x7ffba00d0420 msgr2=0x7ffba00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.328+0000 7ffba5fce640 1 -- 172.21.15.19:0/317402939 shutdown_connections 2024-04-25T10:05:20.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.328+0000 7ffba5fce640 1 -- 172.21.15.19:0/317402939 wait complete. 2024-04-25T10:05:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.329+0000 7ffba5fce640 1 Processor -- start 2024-04-25T10:05:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.329+0000 7ffba5fce640 1 -- start start 2024-04-25T10:05:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffba5fce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffba00ced80 0x7ffba015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffba5fce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffba015dc20 0x7ffba0160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffba5fce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba0161710 0x7ffba0160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffba5fce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba00b5e50 con 0x7ffba0161710 2024-04-25T10:05:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffba5fce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba0162500 con 0x7ffba00ced80 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffba5fce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba00b5fd0 con 0x7ffba015dc20 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba0161710 0x7ffba0160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba0161710 0x7ffba0160760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45994/0 (socket says 172.21.15.19:45994) 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb9ffff640 1 -- 172.21.15.19:0/2599998982 learned_addr learned my addr 172.21.15.19:0/2599998982 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffba00ced80 0x7ffba015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb9effd640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffba015dc20 0x7ffba0160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb9ffff640 1 -- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffba015dc20 msgr2=0x7ffba0160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffba015dc20 0x7ffba0160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb9ffff640 1 -- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffba00ced80 msgr2=0x7ffba015c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffba00ced80 0x7ffba015c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb9ffff640 1 -- 172.21.15.19:0/2599998982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba00cab10 con 0x7ffba0161710 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba0161710 0x7ffba0160760 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7ffb9802acc0 tx=0x7ffb98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.330+0000 7ffb87fff640 1 -- 172.21.15.19:0/2599998982 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98040070 con 0x7ffba0161710 2024-04-25T10:05:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.331+0000 7ffb87fff640 1 -- 172.21.15.19:0/2599998982 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb98031c40 con 0x7ffba0161710 2024-04-25T10:05:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.331+0000 7ffb87fff640 1 -- 172.21.15.19:0/2599998982 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98009450 con 0x7ffba0161710 2024-04-25T10:05:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.331+0000 7ffba5fce640 1 -- 172.21.15.19:0/2599998982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba01f39a0 con 0x7ffba0161710 2024-04-25T10:05:20.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.331+0000 7ffba5fce640 1 -- 172.21.15.19:0/2599998982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba00c9f60 con 0x7ffba0161710 2024-04-25T10:05:20.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.332+0000 7ffb87fff640 1 -- 172.21.15.19:0/2599998982 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb98039040 con 0x7ffba0161710 2024-04-25T10:05:20.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.333+0000 7ffb87fff640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8c06b4d0 0x7ffb8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.333+0000 7ffb87fff640 1 -- 172.21.15.19:0/2599998982 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb980abb30 con 0x7ffba0161710 2024-04-25T10:05:20.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.333+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8c06b4d0 0x7ffb8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.334+0000 7ffba8dad740 1 Processor -- start 2024-04-25T10:05:20.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.334+0000 7ffba8dad740 1 -- start start 2024-04-25T10:05:20.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.334+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8c06b4d0 0x7ffb8c06d990 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ffb90005880 tx=0x7ffb90009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.334+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15dc12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.334+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15dbee80 0x556b15dc17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15d91700 0x556b15d94f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffba8dad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b15cfae00 con 0x556b15dbee80 2024-04-25T10:05:20.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffba8dad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b15ca3d20 con 0x556b15dbeaa0 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffba8dad740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b15cf6ea0 con 0x556b15d91700 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb9effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15dbee80 0x556b15dc17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb9effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15dbee80 0x556b15dc17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45998/0 (socket says 172.21.15.19:45998) 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb9effd640 1 -- 172.21.15.19:0/2781949495 learned_addr learned my addr 172.21.15.19:0/2781949495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/2781949495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15d91700 0x556b15d94f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/2781949495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15dc12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb9effd640 1 -- 172.21.15.19:0/2781949495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15d91700 msgr2=0x556b15d94f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb9effd640 1 --2- 172.21.15.19:0/2781949495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15d91700 0x556b15d94f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb9effd640 1 -- 172.21.15.19:0/2781949495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 msgr2=0x556b15dc12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb9effd640 1 --2- 172.21.15.19:0/2781949495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15dc12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb9effd640 1 -- 172.21.15.19:0/2781949495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b15cc5df0 con 0x556b15dbee80 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb9effd640 1 --2- 172.21.15.19:0/2781949495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15dbee80 0x556b15dc17f0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7ffb94000f60 tx=0x7ffb940019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35134bb037847e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb857fa640 1 -- 172.21.15.19:0/2781949495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94019070 con 0x556b15dbee80 2024-04-25T10:05:20.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb857fa640 1 -- 172.21.15.19:0/2781949495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb940052e0 con 0x556b15dbee80 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.335+0000 7ffb857fa640 1 -- 172.21.15.19:0/2781949495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94005490 con 0x556b15dbee80 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.336+0000 7ffba8dad740 1 -- 172.21.15.19:0/2781949495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15dbee80 msgr2=0x556b15dc17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.336+0000 7ffba8dad740 1 --2- 172.21.15.19:0/2781949495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15dbee80 0x556b15dc17f0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7ffb94000f60 tx=0x7ffb940019b0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.336+0000 7ffba8dad740 1 -- 172.21.15.19:0/2781949495 shutdown_connections 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.336+0000 7ffba8dad740 1 --2- 172.21.15.19:0/2781949495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15d91700 0x556b15d94f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.336+0000 7ffba8dad740 1 --2- 172.21.15.19:0/2781949495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15dbee80 0x556b15dc17f0 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.336+0000 7ffba8dad740 1 --2- 172.21.15.19:0/2781949495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15dc12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.336+0000 7ffba8dad740 1 -- 172.21.15.19:0/2781949495 >> 172.21.15.19:0/2781949495 conn(0x556b15d93970 msgr2=0x556b15ddc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.336+0000 7ffba8dad740 1 -- 172.21.15.19:0/2781949495 shutdown_connections 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.336+0000 7ffba8dad740 1 -- 172.21.15.19:0/2781949495 wait complete. 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.337+0000 7ffba8dad740 1 -- start start 2024-04-25T10:05:20.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.337+0000 7ffba8dad740 1 Processor -- start 2024-04-25T10:05:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15df0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15ddb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15ddb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffba8dad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b15d96b20 con 0x556b15db4a80 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffba8dad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b15c99590 con 0x556b15dbeaa0 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffba8dad740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b15d96e00 con 0x556b15dbee80 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffba8dad740 10 client.0 _ll_register_callbacks cb 0x556b15ca5e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15ddb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15ddb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46008/0 (socket says 172.21.15.19:46008) 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9ffff640 1 -- 172.21.15.19:0/1408607796 learned_addr learned my addr 172.21.15.19:0/1408607796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15df0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9effd640 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15ddb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9ffff640 1 -- 172.21.15.19:0/1408607796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 msgr2=0x556b15ddb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15ddb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9ffff640 1 -- 172.21.15.19:0/1408607796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 msgr2=0x556b15df0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15df0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9ffff640 1 -- 172.21.15.19:0/1408607796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b15a0f3f0 con 0x556b15db4a80 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15df0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15ddb990 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7ffb9807a280 tx=0x7ffb9807a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04f93cdafa6aa89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/1408607796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb980a8060 con 0x556b15db4a80 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/1408607796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb9807e0b0 con 0x556b15db4a80 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffba8dad740 1 -- 172.21.15.19:0/1408607796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb940052e0 con 0x556b15db4a80 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/1408607796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb98087050 con 0x556b15db4a80 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffba8dad740 10 client.11361 fetch_fsmap learned FSMap version 116 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.338+0000 7ffba8dad740 1 -- 172.21.15.19:0/1408607796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) v3 -- 0x556b15ad97d0 con 0x556b15db4a80 2024-04-25T10:05:20.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.339+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/1408607796 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb940052e0 con 0x556b15db4a80 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.339+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/1408607796 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 116) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffb9808a020 con 0x556b15db4a80 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.339+0000 7ffba8dad740 10 client.11361 fetch_fsmap finished waiting for FSMap version 116 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.339+0000 7ffba8dad740 10 client.11361 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.339+0000 7ffba8dad740 20 client.11361 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.339+0000 7ffba8dad740 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556b15dd3510 0x556b15dd3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.339+0000 7ffba8dad740 4 client.11361 mds_command: new command op to 11166 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.339+0000 7ffba8dad740 1 -- 172.21.15.19:0/1408607796 --> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b15d08340 con 0x556b15dd3510 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.339+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556b15dd3510 0x556b15dd3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.339+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556b15dd3510 0x556b15dd3920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=579e5489d9086e5d server_cookie=c9c7d009933a5eb6 in_seq=0 out_seq=0 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.339+0000 7ffb7e7fc640 10 client.11361 ms_handle_connect on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/1408607796 <== mds.0 v2:172.21.15.19:6832/829980210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b15d08340 con 0x556b15dd3510 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffb7e7fc640 10 client.11361 handle_command_reply: tid=0 2024-04-25T10:05:20.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 1 client.11361 shutdown 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 1 -- 172.21.15.19:0/1408607796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 msgr2=0x556b15ddb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15ddb990 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7ffb9807a280 tx=0x7ffb9807a430 comp rx=0 tx=0).stop 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 1 -- 172.21.15.19:0/1408607796 shutdown_connections 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556b15dd3510 0x556b15dd3920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15ddb990 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15ddb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffb7e7fc640 0 client.11361 ms_handle_reset on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 1 --2- 172.21.15.19:0/1408607796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15df0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 1 -- 172.21.15.19:0/1408607796 >> 172.21.15.19:0/1408607796 conn(0x556b15d91700 msgr2=0x556b15d923b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 1 -- 172.21.15.19:0/1408607796 shutdown_connections 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 1 -- 172.21.15.19:0/1408607796 wait complete. 2024-04-25T10:05:20.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.340+0000 7ffba8dad740 20 client.11361 trim_cache size 0 max 16384 2024-04-25T10:05:20.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.352+0000 7ffba8dad740 1 Processor -- start 2024-04-25T10:05:20.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.352+0000 7ffba8dad740 1 -- start start 2024-04-25T10:05:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15dd7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15defae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15df0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffba8dad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba0200380 con 0x556b15db4a80 2024-04-25T10:05:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffba8dad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba01ff500 con 0x556b15dbeaa0 2024-04-25T10:05:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffba8dad740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba00cc250 con 0x556b15dbee80 2024-04-25T10:05:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15dd7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15dd7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46012/0 (socket says 172.21.15.19:46012) 2024-04-25T10:05:20.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffb9f7fe640 1 -- 172.21.15.19:0/2602159246 learned_addr learned my addr 172.21.15.19:0/2602159246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/2602159246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15df0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffb9f7fe640 1 -- 172.21.15.19:0/2602159246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 msgr2=0x556b15df0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/2602159246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15df0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffb9effd640 1 --2- 172.21.15.19:0/2602159246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15defae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffb9f7fe640 1 -- 172.21.15.19:0/2602159246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 msgr2=0x556b15defae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/2602159246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15defae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.353+0000 7ffb9f7fe640 1 -- 172.21.15.19:0/2602159246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b15d98bb0 con 0x556b15db4a80 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/2602159246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15dd7d40 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x556b15db5110 tx=0x7ffb90012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23871683e5927440 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffb84ff9640 1 -- 172.21.15.19:0/2602159246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90007560 con 0x556b15db4a80 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffb84ff9640 1 -- 172.21.15.19:0/2602159246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9002b070 con 0x556b15db4a80 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffb84ff9640 1 -- 172.21.15.19:0/2602159246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90007700 con 0x556b15db4a80 2024-04-25T10:05:20.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffba8dad740 1 -- 172.21.15.19:0/2602159246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 msgr2=0x556b15dd7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffba8dad740 1 --2- 172.21.15.19:0/2602159246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15dd7d40 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x556b15db5110 tx=0x7ffb90012000 comp rx=0 tx=0).stop 2024-04-25T10:05:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffba8dad740 1 -- 172.21.15.19:0/2602159246 shutdown_connections 2024-04-25T10:05:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffba8dad740 1 --2- 172.21.15.19:0/2602159246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15df0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffba8dad740 1 --2- 172.21.15.19:0/2602159246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15defae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffba8dad740 1 --2- 172.21.15.19:0/2602159246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15dd7d40 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffba8dad740 1 -- 172.21.15.19:0/2602159246 >> 172.21.15.19:0/2602159246 conn(0x556b15d91700 msgr2=0x556b15dd3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffba8dad740 1 -- 172.21.15.19:0/2602159246 shutdown_connections 2024-04-25T10:05:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.354+0000 7ffba8dad740 1 -- 172.21.15.19:0/2602159246 wait complete. 2024-04-25T10:05:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.355+0000 7ffba8dad740 1 -- start start 2024-04-25T10:05:20.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.355+0000 7ffba8dad740 1 Processor -- start 2024-04-25T10:05:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15d96450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15dd8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffba8dad740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15dd8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffba8dad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba01fef30 con 0x556b15db4a80 2024-04-25T10:05:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffba8dad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b15db95d0 con 0x556b15dbeaa0 2024-04-25T10:05:20.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffba8dad740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b15deeca0 con 0x556b15dbee80 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffba8dad740 10 client.0 _ll_register_callbacks cb 0x7ffb94007df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15d96450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15d96450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46028/0 (socket says 172.21.15.19:46028) 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9f7fe640 1 -- 172.21.15.19:0/3879310433 learned_addr learned my addr 172.21.15.19:0/3879310433 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15dd8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9effd640 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15dd8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9f7fe640 1 -- 172.21.15.19:0/3879310433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 msgr2=0x556b15dd8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15dd8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9f7fe640 1 -- 172.21.15.19:0/3879310433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 msgr2=0x556b15dd8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15dd8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9f7fe640 1 -- 172.21.15.19:0/3879310433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b15defb80 con 0x556b15db4a80 2024-04-25T10:05:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9ffff640 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15dd8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb9f7fe640 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15d96450 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7ffb90000f60 tx=0x7ffb900010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d798e0790d123b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/3879310433 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90026250 con 0x556b15db4a80 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/3879310433 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb900263f0 con 0x556b15db4a80 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.356+0000 7ffba8dad740 1 -- 172.21.15.19:0/3879310433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb9002b070 con 0x556b15db4a80 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/3879310433 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb9003d050 con 0x556b15db4a80 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffba8dad740 10 client.11367 fetch_fsmap learned FSMap version 116 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffba8dad740 1 -- 172.21.15.19:0/3879310433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) v3 -- 0x556b15deea40 con 0x556b15db4a80 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/3879310433 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9002b070 con 0x556b15db4a80 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/3879310433 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 116) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffb90040020 con 0x556b15db4a80 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffba8dad740 10 client.11367 fetch_fsmap finished waiting for FSMap version 116 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffba8dad740 10 client.11367 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffba8dad740 20 client.11367 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:20.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffba8dad740 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556b15d92710 0x556b15df6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffba8dad740 4 client.11367 mds_command: new command op to 11166 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffba8dad740 1 -- 172.21.15.19:0/3879310433 --> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556b15dd5ba0 con 0x556b15d92710 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffb9effd640 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556b15d92710 0x556b15df6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffb7e7fc640 10 client.11367 ms_handle_connect on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffb9effd640 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556b15d92710 0x556b15df6e00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85cfa362eea3386b server_cookie=ccfe02763cb27ade in_seq=0 out_seq=0 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffb7e7fc640 1 -- 172.21.15.19:0/3879310433 <== mds.0 v2:172.21.15.19:6832/829980210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556b15dd5ba0 con 0x556b15d92710 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffb7e7fc640 10 client.11367 handle_command_reply: tid=0 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.357+0000 7ffba8dad740 1 client.11367 shutdown 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba8dad740 1 -- 172.21.15.19:0/3879310433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 msgr2=0x556b15d96450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba8dad740 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15d96450 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7ffb90000f60 tx=0x7ffb900010a0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba8dad740 1 -- 172.21.15.19:0/3879310433 shutdown_connections 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba8dad740 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.19:6832/829980210,v1:172.21.15.19:6833/829980210] conn(0x556b15d92710 0x556b15df6e00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba8dad740 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x556b15dbee80 0x556b15dd8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba8dad740 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b15dbeaa0 0x556b15dd8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffb7e7fc640 0 client.11367 ms_handle_reset on v2:172.21.15.19:6832/829980210 2024-04-25T10:05:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba8dad740 1 --2- 172.21.15.19:0/3879310433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b15db4a80 0x556b15d96450 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba8dad740 1 -- 172.21.15.19:0/3879310433 >> 172.21.15.19:0/3879310433 conn(0x556b15d91700 msgr2=0x556b15dd7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba8dad740 1 -- 172.21.15.19:0/3879310433 shutdown_connections 2024-04-25T10:05:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba8dad740 1 -- 172.21.15.19:0/3879310433 wait complete. 2024-04-25T10:05:20.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba8dad740 20 client.11367 trim_cache size 0 max 16384 2024-04-25T10:05:20.370 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-04-25T10:05:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 -- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8c06b4d0 msgr2=0x7ffb8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8c06b4d0 0x7ffb8c06d990 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ffb90005880 tx=0x7ffb90009040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 -- 172.21.15.19:0/2599998982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba0161710 msgr2=0x7ffba0160760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba0161710 0x7ffba0160760 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7ffb9802acc0 tx=0x7ffb98004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 -- 172.21.15.19:0/2599998982 shutdown_connections 2024-04-25T10:05:20.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb8c06b4d0 0x7ffb8c06d990 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba0161710 0x7ffba0160760 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffba015dc20 0x7ffba0160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 --2- 172.21.15.19:0/2599998982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffba00ced80 0x7ffba015c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 -- 172.21.15.19:0/2599998982 >> 172.21.15.19:0/2599998982 conn(0x7ffba00d0420 msgr2=0x7ffba015cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 -- 172.21.15.19:0/2599998982 shutdown_connections 2024-04-25T10:05:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.358+0000 7ffba5fce640 1 -- 172.21.15.19:0/2599998982 wait complete. 2024-04-25T10:05:20.374 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:20.374 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:20.412 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:20.451 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:20.451 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:20.451 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:20.468 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:20.468 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:20.525 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.660+0000 7f161a15c640 1 Processor -- start 2024-04-25T10:05:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.660+0000 7f161a15c640 1 -- start start 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.661+0000 7f161a15c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f161415dbe0 0x7f16140d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.661+0000 7f161a15c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16140b92a0 0x7f16140d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.661+0000 7f161a15c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16140b97a0 con 0x7f16140b2cb0 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.661+0000 7f161a15c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161415dfc0 con 0x7f161415dbe0 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.661+0000 7f161a15c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16140d1a00 con 0x7f16140b92a0 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.661+0000 7f16137fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f161415dbe0 0x7f16140d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.661+0000 7f16137fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f161415dbe0 0x7f16140d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46044/0 (socket says 172.21.15.19:46044) 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.661+0000 7f16137fe640 1 -- 172.21.15.19:0/1281489314 learned_addr learned my addr 172.21.15.19:0/1281489314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.662+0000 7f1613fff640 1 --2- 172.21.15.19:0/1281489314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16140b92a0 0x7f16140d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.662+0000 7f16137fe640 1 -- 172.21.15.19:0/1281489314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16140b92a0 msgr2=0x7f16140d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.662+0000 7f16137fe640 1 --2- 172.21.15.19:0/1281489314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16140b92a0 0x7f16140d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.662+0000 7f16137fe640 1 -- 172.21.15.19:0/1281489314 >> v1:172.21.15.112:6789/0 conn(0x7f16140b2cb0 legacy=0x7f16140b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.662+0000 7f16137fe640 1 -- 172.21.15.19:0/1281489314 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140d1c50 con 0x7f161415dbe0 2024-04-25T10:05:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.662+0000 7f16137fe640 1 --2- 172.21.15.19:0/1281489314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f161415dbe0 0x7f16140d0910 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f1604009870 tx=0x7f160402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e332dc2ab3e365 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.662+0000 7f16127fc640 1 -- 172.21.15.19:0/1281489314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604037070 con 0x7f161415dbe0 2024-04-25T10:05:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.663+0000 7f16127fc640 1 -- 172.21.15.19:0/1281489314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1604032440 con 0x7f161415dbe0 2024-04-25T10:05:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.663+0000 7f16127fc640 1 -- 172.21.15.19:0/1281489314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604032810 con 0x7f161415dbe0 2024-04-25T10:05:20.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.663+0000 7f161a15c640 1 -- 172.21.15.19:0/1281489314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f161415dbe0 msgr2=0x7f16140d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.663+0000 7f161a15c640 1 --2- 172.21.15.19:0/1281489314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f161415dbe0 0x7f16140d0910 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f1604009870 tx=0x7f160402a270 comp rx=0 tx=0).stop 2024-04-25T10:05:20.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.663+0000 7f161a15c640 1 -- 172.21.15.19:0/1281489314 shutdown_connections 2024-04-25T10:05:20.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.663+0000 7f161a15c640 1 --2- 172.21.15.19:0/1281489314 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16140b92a0 0x7f16140d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.663+0000 7f161a15c640 1 --2- 172.21.15.19:0/1281489314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f161415dbe0 0x7f16140d0910 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.663+0000 7f161a15c640 1 -- 172.21.15.19:0/1281489314 >> 172.21.15.19:0/1281489314 conn(0x7f1614061a20 msgr2=0x7f1614061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.663+0000 7f161a15c640 1 -- 172.21.15.19:0/1281489314 shutdown_connections 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.663+0000 7f161a15c640 1 -- 172.21.15.19:0/1281489314 wait complete. 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.664+0000 7f161a15c640 1 Processor -- start 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.664+0000 7f161a15c640 1 -- start start 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.665+0000 7f161a15c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16140b2cb0 0x7f16140c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.665+0000 7f161a15c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16140b92a0 0x7f161415fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.665+0000 7f161a15c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f161415dbe0 0x7f1614160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.665+0000 7f161a15c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16140b6ab0 con 0x7f161415dbe0 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.665+0000 7f161a15c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16140d2210 con 0x7f16140b92a0 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.665+0000 7f161a15c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16140b6c30 con 0x7f16140b2cb0 2024-04-25T10:05:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.665+0000 7f1613fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f161415dbe0 0x7f1614160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.665+0000 7f1613fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f161415dbe0 0x7f1614160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46056/0 (socket says 172.21.15.19:46056) 2024-04-25T10:05:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.665+0000 7f1613fff640 1 -- 172.21.15.19:0/3137108907 learned_addr learned my addr 172.21.15.19:0/3137108907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.665+0000 7f16137fe640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16140b2cb0 0x7f16140c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.665+0000 7f1612ffd640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16140b92a0 0x7f161415fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f1613fff640 1 -- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16140b2cb0 msgr2=0x7f16140c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f1613fff640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16140b2cb0 0x7f16140c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f1613fff640 1 -- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16140b92a0 msgr2=0x7f161415fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f1613fff640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16140b92a0 0x7f161415fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f1613fff640 1 -- 172.21.15.19:0/3137108907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140cb940 con 0x7f161415dbe0 2024-04-25T10:05:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f1612ffd640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16140b92a0 0x7f161415fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f1613fff640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f161415dbe0 0x7f1614160230 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f160c007e30 tx=0x7f160c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f15fbfff640 1 -- 172.21.15.19:0/3137108907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c014620 con 0x7f161415dbe0 2024-04-25T10:05:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f15fbfff640 1 -- 172.21.15.19:0/3137108907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c019070 con 0x7f161415dbe0 2024-04-25T10:05:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f161a15c640 1 -- 172.21.15.19:0/3137108907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16140cbc20 con 0x7f161415dbe0 2024-04-25T10:05:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f15fbfff640 1 -- 172.21.15.19:0/3137108907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c014a00 con 0x7f161415dbe0 2024-04-25T10:05:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.666+0000 7f161a15c640 1 -- 172.21.15.19:0/3137108907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16140cc1e0 con 0x7f161415dbe0 2024-04-25T10:05:20.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.667+0000 7f161a15c640 1 -- 172.21.15.19:0/3137108907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1614073990 con 0x7f161415dbe0 2024-04-25T10:05:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.668+0000 7f15fbfff640 1 -- 172.21.15.19:0/3137108907 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f160c014ba0 con 0x7f161415dbe0 2024-04-25T10:05:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.668+0000 7f15fbfff640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f160006b4d0 0x7f160006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.668+0000 7f15fbfff640 1 -- 172.21.15.19:0/3137108907 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f160c08abe0 con 0x7f161415dbe0 2024-04-25T10:05:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.668+0000 7f16137fe640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f160006b4d0 0x7f160006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.669+0000 7f16137fe640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f160006b4d0 0x7f160006d990 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1604034000 tx=0x7f160402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.672+0000 7f15fbfff640 1 -- 172.21.15.19:0/3137108907 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f160c0598c0 con 0x7f161415dbe0 2024-04-25T10:05:20.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.853+0000 7f161a15c640 1 -- 172.21.15.19:0/3137108907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f161406b080 con 0x7f161415dbe0 2024-04-25T10:05:20.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.854+0000 7f15fbfff640 1 -- 172.21.15.19:0/3137108907 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f160c0591c0 con 0x7f161415dbe0 2024-04-25T10:05:20.859 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:20.859 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":178,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:14.934502+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:11.578636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:12.274609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:20.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 -- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f160006b4d0 msgr2=0x7f160006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f160006b4d0 0x7f160006d990 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1604034000 tx=0x7f160402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 -- 172.21.15.19:0/3137108907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f161415dbe0 msgr2=0x7f1614160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f161415dbe0 0x7f1614160230 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f160c007e30 tx=0x7f160c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 -- 172.21.15.19:0/3137108907 shutdown_connections 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f160006b4d0 0x7f160006d990 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f161415dbe0 0x7f1614160230 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16140b92a0 0x7f161415fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 --2- 172.21.15.19:0/3137108907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16140b2cb0 0x7f16140c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 -- 172.21.15.19:0/3137108907 >> 172.21.15.19:0/3137108907 conn(0x7f1614061a20 msgr2=0x7f16140b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 -- 172.21.15.19:0/3137108907 shutdown_connections 2024-04-25T10:05:20.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.856+0000 7f161a15c640 1 -- 172.21.15.19:0/3137108907 wait complete. 2024-04-25T10:05:20.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:11.578636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '177', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:12.274609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '178', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:20.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.976+0000 7fcfee5d5640 1 Processor -- start 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.976+0000 7fcfee5d5640 1 -- start start 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfee5d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe815dc20 0x7fcfe80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfee5d5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcfe80d32f0 0x7fcfe80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfee5d5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe815e000 con 0x7fcfe8161710 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfee5d5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe80d1de0 con 0x7fcfe80d32f0 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfee5d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8161c60 con 0x7fcfe815dc20 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfe7fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe815dc20 0x7fcfe80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfe7fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe815dc20 0x7fcfe80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:45448/0 (socket says 172.21.15.19:45448) 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfe7fff640 1 -- 172.21.15.19:0/3304853886 learned_addr learned my addr 172.21.15.19:0/3304853886 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfecb72640 1 --2- 172.21.15.19:0/3304853886 >> v2:172.21.15.112:3300/0 conn(0x7fcfe80d32f0 0x7fcfe80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfe7fff640 1 -- 172.21.15.19:0/3304853886 >> v2:172.21.15.112:3300/0 conn(0x7fcfe80d32f0 msgr2=0x7fcfe80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfe7fff640 1 --2- 172.21.15.19:0/3304853886 >> v2:172.21.15.112:3300/0 conn(0x7fcfe80d32f0 0x7fcfe80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfe7fff640 1 -- 172.21.15.19:0/3304853886 >> v1:172.21.15.112:6789/0 conn(0x7fcfe8161710 legacy=0x7fcfe806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.977+0000 7fcfe7fff640 1 -- 172.21.15.19:0/3304853886 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8161f40 con 0x7fcfe815dc20 2024-04-25T10:05:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.978+0000 7fcfe7fff640 1 --2- 172.21.15.19:0/3304853886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe815dc20 0x7fcfe80ce670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009870 tx=0x7fcfdc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d7bb8c2957315a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.978+0000 7fcfe6ffd640 1 -- 172.21.15.19:0/3304853886 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc037070 con 0x7fcfe815dc20 2024-04-25T10:05:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.978+0000 7fcfe6ffd640 1 -- 172.21.15.19:0/3304853886 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfdc032440 con 0x7fcfe815dc20 2024-04-25T10:05:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.978+0000 7fcfe6ffd640 1 -- 172.21.15.19:0/3304853886 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc032810 con 0x7fcfe815dc20 2024-04-25T10:05:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.979+0000 7fcfee5d5640 1 -- 172.21.15.19:0/3304853886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe815dc20 msgr2=0x7fcfe80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.979+0000 7fcfee5d5640 1 --2- 172.21.15.19:0/3304853886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe815dc20 0x7fcfe80ce670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009870 tx=0x7fcfdc02a270 comp rx=0 tx=0).stop 2024-04-25T10:05:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.979+0000 7fcfee5d5640 1 -- 172.21.15.19:0/3304853886 shutdown_connections 2024-04-25T10:05:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.979+0000 7fcfee5d5640 1 --2- 172.21.15.19:0/3304853886 >> v2:172.21.15.112:3300/0 conn(0x7fcfe80d32f0 0x7fcfe80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.979+0000 7fcfee5d5640 1 --2- 172.21.15.19:0/3304853886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe815dc20 0x7fcfe80ce670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.979+0000 7fcfee5d5640 1 -- 172.21.15.19:0/3304853886 >> 172.21.15.19:0/3304853886 conn(0x7fcfe80d0420 msgr2=0x7fcfe80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.979+0000 7fcfee5d5640 1 -- 172.21.15.19:0/3304853886 shutdown_connections 2024-04-25T10:05:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.979+0000 7fcfee5d5640 1 -- 172.21.15.19:0/3304853886 wait complete. 2024-04-25T10:05:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.980+0000 7fcfee5d5640 1 Processor -- start 2024-04-25T10:05:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.980+0000 7fcfee5d5640 1 -- start start 2024-04-25T10:05:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfee5d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe80d32f0 0x7fcfe8160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfee5d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe815dc20 0x7fcfe80b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfee5d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfe8161710 0x7fcfe80ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfee5d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80b5e50 con 0x7fcfe8161710 2024-04-25T10:05:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfee5d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8162500 con 0x7fcfe815dc20 2024-04-25T10:05:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfee5d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80b5fd0 con 0x7fcfe80d32f0 2024-04-25T10:05:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfecb72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfe8161710 0x7fcfe80ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfecb72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfe8161710 0x7fcfe80ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46068/0 (socket says 172.21.15.19:46068) 2024-04-25T10:05:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfecb72640 1 -- 172.21.15.19:0/630705295 learned_addr learned my addr 172.21.15.19:0/630705295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfe7fff640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe80d32f0 0x7fcfe8160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfe77fe640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe815dc20 0x7fcfe80b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfecb72640 1 -- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe80d32f0 msgr2=0x7fcfe8160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfecb72640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe80d32f0 0x7fcfe8160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.981+0000 7fcfecb72640 1 -- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe815dc20 msgr2=0x7fcfe80b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.982+0000 7fcfecb72640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe815dc20 0x7fcfe80b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.982+0000 7fcfecb72640 1 -- 172.21.15.19:0/630705295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe815ecf0 con 0x7fcfe8161710 2024-04-25T10:05:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.982+0000 7fcfe77fe640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe815dc20 0x7fcfe80b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.982+0000 7fcfecb72640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfe8161710 0x7fcfe80ac190 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000ec10 tx=0x7fcfe000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.982+0000 7fcfe4ff9640 1 -- 172.21.15.19:0/630705295 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0018020 con 0x7fcfe8161710 2024-04-25T10:05:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.982+0000 7fcfe4ff9640 1 -- 172.21.15.19:0/630705295 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe000ccc0 con 0x7fcfe8161710 2024-04-25T10:05:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.982+0000 7fcfee5d5640 1 -- 172.21.15.19:0/630705295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe815efd0 con 0x7fcfe8161710 2024-04-25T10:05:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.982+0000 7fcfe4ff9640 1 -- 172.21.15.19:0/630705295 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00146c0 con 0x7fcfe8161710 2024-04-25T10:05:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.982+0000 7fcfee5d5640 1 -- 172.21.15.19:0/630705295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe815f590 con 0x7fcfe8161710 2024-04-25T10:05:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.983+0000 7fcfee5d5640 1 -- 172.21.15.19:0/630705295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe8073a00 con 0x7fcfe8161710 2024-04-25T10:05:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.983+0000 7fcfe4ff9640 1 -- 172.21.15.19:0/630705295 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfe001b020 con 0x7fcfe8161710 2024-04-25T10:05:20.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.984+0000 7fcfe4ff9640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfd406b4d0 0x7fcfd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.984+0000 7fcfe4ff9640 1 -- 172.21.15.19:0/630705295 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcfe008b330 con 0x7fcfe8161710 2024-04-25T10:05:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.984+0000 7fcfe7fff640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfd406b4d0 0x7fcfd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.985+0000 7fcfe7fff640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfd406b4d0 0x7fcfd406d990 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc034000 tx=0x7fcfdc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:20.988+0000 7fcfe4ff9640 1 -- 172.21.15.19:0/630705295 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcfe0022020 con 0x7fcfe8161710 2024-04-25T10:05:21.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.169+0000 7fcfee5d5640 1 -- 172.21.15.19:0/630705295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcfe80d16b0 con 0x7fcfe8161710 2024-04-25T10:05:21.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.170+0000 7fcfe4ff9640 1 -- 172.21.15.19:0/630705295 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v178) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcfe0010040 con 0x7fcfe8161710 2024-04-25T10:05:21.175 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:05:21.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 -- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfd406b4d0 msgr2=0x7fcfd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfd406b4d0 0x7fcfd406d990 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc034000 tx=0x7fcfdc02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:21.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 -- 172.21.15.19:0/630705295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfe8161710 msgr2=0x7fcfe80ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfe8161710 0x7fcfe80ac190 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000ec10 tx=0x7fcfe000c490 comp rx=0 tx=0).stop 2024-04-25T10:05:21.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 -- 172.21.15.19:0/630705295 shutdown_connections 2024-04-25T10:05:21.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcfd406b4d0 0x7fcfd406d990 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfe8161710 0x7fcfe80ac190 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfe815dc20 0x7fcfe80b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 --2- 172.21.15.19:0/630705295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcfe80d32f0 0x7fcfe8160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 -- 172.21.15.19:0/630705295 >> 172.21.15.19:0/630705295 conn(0x7fcfe80d0420 msgr2=0x7fcfe80cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 -- 172.21.15.19:0/630705295 shutdown_connections 2024-04-25T10:05:21.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.172+0000 7fcfee5d5640 1 -- 172.21.15.19:0/630705295 wait complete. 2024-04-25T10:05:21.188 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.287+0000 7f0d827ce640 1 Processor -- start 2024-04-25T10:05:21.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.287+0000 7f0d827ce640 1 -- start start 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d827ce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0d7c0ced80 0x7f0d7c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d827ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c161710 0x7f0d7c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d827ce640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7c15e000 con 0x7f0d7c15dc20 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d827ce640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c0d1de0 con 0x7f0d7c0ced80 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d827ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c161c60 con 0x7f0d7c161710 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d80d6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c161710 0x7f0d7c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d7b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0d7c0ced80 0x7f0d7c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d7bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0d7c15dc20 0x7f0d7c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:56002/0 (socket says 172.21.15.19:56002) 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d80d6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c161710 0x7f0d7c0d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:45466/0 (socket says 172.21.15.19:45466) 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d7bfff640 1 -- 172.21.15.19:0/4183427846 learned_addr learned my addr 172.21.15.19:0/4183427846 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d80d6b640 1 -- 172.21.15.19:0/4183427846 >> v2:172.21.15.112:3300/0 conn(0x7f0d7c0ced80 msgr2=0x7f0d7c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.288+0000 7f0d80d6b640 1 --2- 172.21.15.19:0/4183427846 >> v2:172.21.15.112:3300/0 conn(0x7f0d7c0ced80 0x7f0d7c0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.289+0000 7f0d80d6b640 1 -- 172.21.15.19:0/4183427846 >> v1:172.21.15.112:6789/0 conn(0x7f0d7c15dc20 legacy=0x7f0d7c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.289+0000 7f0d80d6b640 1 -- 172.21.15.19:0/4183427846 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c161f40 con 0x7f0d7c161710 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.289+0000 7f0d7b7fe640 1 --2- 172.21.15.19:0/4183427846 >> v2:172.21.15.112:3300/0 conn(0x7f0d7c0ced80 0x7f0d7c0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.289+0000 7f0d7affd640 1 -- 172.21.15.19:0/4183427846 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524364114 0 0) 0x7f0d7c15e000 con 0x7f0d7c15dc20 2024-04-25T10:05:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.289+0000 7f0d80d6b640 1 --2- 172.21.15.19:0/4183427846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c161710 0x7f0d7c0d2820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009d90 tx=0x7f0d7402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f6f5d11b2d18f83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.289+0000 7f0d7affd640 1 -- 172.21.15.19:0/4183427846 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74007740 con 0x7f0d7c161710 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.290+0000 7f0d7affd640 1 -- 172.21.15.19:0/4183427846 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d740078e0 con 0x7f0d7c161710 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.290+0000 7f0d7affd640 1 -- 172.21.15.19:0/4183427846 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74007cb0 con 0x7f0d7c161710 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.290+0000 7f0d827ce640 1 -- 172.21.15.19:0/4183427846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c161710 msgr2=0x7f0d7c0d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.290+0000 7f0d827ce640 1 --2- 172.21.15.19:0/4183427846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c161710 0x7f0d7c0d2820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009d90 tx=0x7f0d7402a660 comp rx=0 tx=0).stop 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.290+0000 7f0d827ce640 1 -- 172.21.15.19:0/4183427846 shutdown_connections 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.290+0000 7f0d827ce640 1 --2- 172.21.15.19:0/4183427846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c161710 0x7f0d7c0d2820 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.290+0000 7f0d827ce640 1 --2- 172.21.15.19:0/4183427846 >> v2:172.21.15.112:3300/0 conn(0x7f0d7c0ced80 0x7f0d7c0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.290+0000 7f0d827ce640 1 -- 172.21.15.19:0/4183427846 >> 172.21.15.19:0/4183427846 conn(0x7f0d7c0d0420 msgr2=0x7f0d7c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.290+0000 7f0d827ce640 1 -- 172.21.15.19:0/4183427846 shutdown_connections 2024-04-25T10:05:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.290+0000 7f0d827ce640 1 -- 172.21.15.19:0/4183427846 wait complete. 2024-04-25T10:05:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.291+0000 7f0d827ce640 1 Processor -- start 2024-04-25T10:05:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.291+0000 7f0d827ce640 1 -- start start 2024-04-25T10:05:21.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.292+0000 7f0d827ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c0ced80 0x7f0d7c0c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.292+0000 7f0d827ce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c15dc20 0x7f0d7c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.292+0000 7f0d827ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d7c161710 0x7f0d7c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.292+0000 7f0d827ce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0b5e50 con 0x7f0d7c15dc20 2024-04-25T10:05:21.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.292+0000 7f0d827ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c162500 con 0x7f0d7c161710 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.292+0000 7f0d827ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0b5fd0 con 0x7f0d7c0ced80 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.292+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c15dc20 0x7f0d7c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.292+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c15dc20 0x7f0d7c0c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46084/0 (socket says 172.21.15.19:46084) 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.292+0000 7f0d7b7fe640 1 -- 172.21.15.19:0/2782049495 learned_addr learned my addr 172.21.15.19:0/2782049495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.292+0000 7f0d7bfff640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c0ced80 0x7f0d7c0c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.292+0000 7f0d80d6b640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d7c161710 0x7f0d7c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.293+0000 7f0d7b7fe640 1 -- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c0ced80 msgr2=0x7f0d7c0c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.293+0000 7f0d7b7fe640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c0ced80 0x7f0d7c0c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.293+0000 7f0d7b7fe640 1 -- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d7c161710 msgr2=0x7f0d7c0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.293+0000 7f0d7b7fe640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d7c161710 0x7f0d7c0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.293+0000 7f0d7b7fe640 1 -- 172.21.15.19:0/2782049495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0c7be0 con 0x7f0d7c15dc20 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.293+0000 7f0d7b7fe640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c15dc20 0x7f0d7c0c7320 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000e990 tx=0x7f0d7000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.293+0000 7f0d78ff9640 1 -- 172.21.15.19:0/2782049495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70017070 con 0x7f0d7c15dc20 2024-04-25T10:05:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.293+0000 7f0d78ff9640 1 -- 172.21.15.19:0/2782049495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7000c9e0 con 0x7f0d7c15dc20 2024-04-25T10:05:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.293+0000 7f0d827ce640 1 -- 172.21.15.19:0/2782049495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c0c7ec0 con 0x7f0d7c15dc20 2024-04-25T10:05:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.293+0000 7f0d78ff9640 1 -- 172.21.15.19:0/2782049495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7000cdb0 con 0x7f0d7c15dc20 2024-04-25T10:05:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.293+0000 7f0d827ce640 1 -- 172.21.15.19:0/2782049495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c0c8480 con 0x7f0d7c15dc20 2024-04-25T10:05:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.294+0000 7f0d827ce640 1 -- 172.21.15.19:0/2782049495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c073a00 con 0x7f0d7c15dc20 2024-04-25T10:05:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.294+0000 7f0d78ff9640 1 -- 172.21.15.19:0/2782049495 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d7001a070 con 0x7f0d7c15dc20 2024-04-25T10:05:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.295+0000 7f0d78ff9640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d6806b4d0 0x7f0d6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.295+0000 7f0d78ff9640 1 -- 172.21.15.19:0/2782049495 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0d7008a540 con 0x7f0d7c15dc20 2024-04-25T10:05:21.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.295+0000 7f0d7bfff640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d6806b4d0 0x7f0d6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.296+0000 7f0d7bfff640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d6806b4d0 0x7f0d6806d990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c007fd0 tx=0x7f0d6c007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.299+0000 7f0d78ff9640 1 -- 172.21.15.19:0/2782049495 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d70058be0 con 0x7f0d7c15dc20 2024-04-25T10:05:21.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.479+0000 7f0d827ce640 1 -- 172.21.15.19:0/2782049495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d7c0c8880 con 0x7f0d7c15dc20 2024-04-25T10:05:21.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.480+0000 7f0d78ff9640 1 -- 172.21.15.19:0/2782049495 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v178) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d7005cc80 con 0x7f0d7c15dc20 2024-04-25T10:05:21.485 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:05:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 -- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d6806b4d0 msgr2=0x7f0d6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d6806b4d0 0x7f0d6806d990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c007fd0 tx=0x7f0d6c007de0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 -- 172.21.15.19:0/2782049495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c15dc20 msgr2=0x7f0d7c0c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c15dc20 0x7f0d7c0c7320 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000e990 tx=0x7f0d7000ee50 comp rx=0 tx=0).stop 2024-04-25T10:05:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 -- 172.21.15.19:0/2782049495 shutdown_connections 2024-04-25T10:05:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0d6806b4d0 0x7f0d6806d990 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d7c161710 0x7f0d7c0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c15dc20 0x7f0d7c0c7320 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 --2- 172.21.15.19:0/2782049495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d7c0ced80 0x7f0d7c0c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 -- 172.21.15.19:0/2782049495 >> 172.21.15.19:0/2782049495 conn(0x7f0d7c0d0420 msgr2=0x7f0d7c0b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 -- 172.21.15.19:0/2782049495 shutdown_connections 2024-04-25T10:05:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.482+0000 7f0d827ce640 1 -- 172.21.15.19:0/2782049495 wait complete. 2024-04-25T10:05:21.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:21.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.596+0000 7fcd86458640 1 Processor -- start 2024-04-25T10:05:21.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.596+0000 7fcd86458640 1 -- start start 2024-04-25T10:05:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd86458640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd8015b5a0 0x7fcd800b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd86458640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcd8015b980 0x7fcd800b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd86458640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd80162450 0x7fcd800ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd86458640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd800cf320 con 0x7fcd8015b5a0 2024-04-25T10:05:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd86458640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd800cf040 con 0x7fcd8015b980 2024-04-25T10:05:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd86458640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80161970 con 0x7fcd80162450 2024-04-25T10:05:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd7ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd8015b5a0 0x7fcd800b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd7ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd8015b5a0 0x7fcd800b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46096/0 (socket says 172.21.15.19:46096) 2024-04-25T10:05:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd7ffff640 1 -- 172.21.15.19:0/2526990351 learned_addr learned my addr 172.21.15.19:0/2526990351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd7f7fe640 1 --2- 172.21.15.19:0/2526990351 >> v2:172.21.15.112:3300/0 conn(0x7fcd8015b980 0x7fcd800b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd849f5640 1 --2- 172.21.15.19:0/2526990351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd80162450 0x7fcd800ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd7ffff640 1 -- 172.21.15.19:0/2526990351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd80162450 msgr2=0x7fcd800ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd7ffff640 1 --2- 172.21.15.19:0/2526990351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd80162450 0x7fcd800ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd7ffff640 1 -- 172.21.15.19:0/2526990351 >> v2:172.21.15.112:3300/0 conn(0x7fcd8015b980 msgr2=0x7fcd800b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd7ffff640 1 --2- 172.21.15.19:0/2526990351 >> v2:172.21.15.112:3300/0 conn(0x7fcd8015b980 0x7fcd800b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd7ffff640 1 -- 172.21.15.19:0/2526990351 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd80161c50 con 0x7fcd8015b5a0 2024-04-25T10:05:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.597+0000 7fcd849f5640 1 --2- 172.21.15.19:0/2526990351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd80162450 0x7fcd800ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.598+0000 7fcd7ffff640 1 --2- 172.21.15.19:0/2526990351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd8015b5a0 0x7fcd800b5530 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fcd800b9bf0 tx=0x7fcd6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd82e441c2a79741 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.598+0000 7fcd7effd640 1 -- 172.21.15.19:0/2526990351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68037070 con 0x7fcd8015b5a0 2024-04-25T10:05:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.598+0000 7fcd7effd640 1 -- 172.21.15.19:0/2526990351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd68032440 con 0x7fcd8015b5a0 2024-04-25T10:05:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.598+0000 7fcd7effd640 1 -- 172.21.15.19:0/2526990351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68032810 con 0x7fcd8015b5a0 2024-04-25T10:05:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.599+0000 7fcd86458640 1 -- 172.21.15.19:0/2526990351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd8015b5a0 msgr2=0x7fcd800b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.599+0000 7fcd86458640 1 --2- 172.21.15.19:0/2526990351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd8015b5a0 0x7fcd800b5530 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fcd800b9bf0 tx=0x7fcd6802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.599+0000 7fcd86458640 1 -- 172.21.15.19:0/2526990351 shutdown_connections 2024-04-25T10:05:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.599+0000 7fcd86458640 1 --2- 172.21.15.19:0/2526990351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd80162450 0x7fcd800ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.599+0000 7fcd86458640 1 --2- 172.21.15.19:0/2526990351 >> v2:172.21.15.112:3300/0 conn(0x7fcd8015b980 0x7fcd800b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.599+0000 7fcd86458640 1 --2- 172.21.15.19:0/2526990351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd8015b5a0 0x7fcd800b5530 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.599+0000 7fcd86458640 1 -- 172.21.15.19:0/2526990351 >> 172.21.15.19:0/2526990351 conn(0x7fcd800d0530 msgr2=0x7fcd800d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.599+0000 7fcd86458640 1 -- 172.21.15.19:0/2526990351 shutdown_connections 2024-04-25T10:05:21.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.599+0000 7fcd86458640 1 -- 172.21.15.19:0/2526990351 wait complete. 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.600+0000 7fcd86458640 1 Processor -- start 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.600+0000 7fcd86458640 1 -- start start 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd86458640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd8015b5a0 0x7fcd801f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd86458640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd8015b980 0x7fcd801f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd86458640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80162450 0x7fcd800cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd86458640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800ce9d0 con 0x7fcd8015b5a0 2024-04-25T10:05:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd86458640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800ce850 con 0x7fcd80162450 2024-04-25T10:05:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd86458640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800ceb50 con 0x7fcd8015b980 2024-04-25T10:05:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd8015b5a0 0x7fcd801f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd8015b5a0 0x7fcd801f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46100/0 (socket says 172.21.15.19:46100) 2024-04-25T10:05:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd7ffff640 1 -- 172.21.15.19:0/3806673743 learned_addr learned my addr 172.21.15.19:0/3806673743 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd7f7fe640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd8015b980 0x7fcd801f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd849f5640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80162450 0x7fcd800cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd7ffff640 1 -- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd8015b980 msgr2=0x7fcd801f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd7ffff640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd8015b980 0x7fcd801f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd7ffff640 1 -- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80162450 msgr2=0x7fcd800cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd7ffff640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80162450 0x7fcd800cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.601+0000 7fcd7ffff640 1 -- 172.21.15.19:0/3806673743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd801f48b0 con 0x7fcd8015b5a0 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.602+0000 7fcd7ffff640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd8015b5a0 0x7fcd801f0f70 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fcd6802a220 tx=0x7fcd680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.602+0000 7fcd7cff9640 1 -- 172.21.15.19:0/3806673743 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68037040 con 0x7fcd8015b5a0 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.602+0000 7fcd7cff9640 1 -- 172.21.15.19:0/3806673743 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd68042070 con 0x7fcd8015b5a0 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.602+0000 7fcd86458640 1 -- 172.21.15.19:0/3806673743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd800cc240 con 0x7fcd8015b5a0 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.602+0000 7fcd7cff9640 1 -- 172.21.15.19:0/3806673743 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68030880 con 0x7fcd8015b5a0 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.602+0000 7fcd86458640 1 -- 172.21.15.19:0/3806673743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8015f850 con 0x7fcd8015b5a0 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.603+0000 7fcd86458640 1 -- 172.21.15.19:0/3806673743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8015c7a0 con 0x7fcd8015b5a0 2024-04-25T10:05:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.603+0000 7fcd7cff9640 1 -- 172.21.15.19:0/3806673743 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd68030a20 con 0x7fcd8015b5a0 2024-04-25T10:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.604+0000 7fcd7cff9640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6006b4d0 0x7fcd6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.604+0000 7fcd7cff9640 1 -- 172.21.15.19:0/3806673743 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd680aaae0 con 0x7fcd8015b5a0 2024-04-25T10:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.604+0000 7fcd7f7fe640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6006b4d0 0x7fcd6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.604+0000 7fcd7f7fe640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6006b4d0 0x7fcd6006d990 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fcd800b6960 tx=0x7fcd70009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.608+0000 7fcd7cff9640 1 -- 172.21.15.19:0/3806673743 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd8015c7a0 con 0x7fcd8015b5a0 2024-04-25T10:05:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.788+0000 7fcd86458640 1 -- 172.21.15.19:0/3806673743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd800d17c0 con 0x7fcd8015b5a0 2024-04-25T10:05:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.788+0000 7fcd7cff9640 1 -- 172.21.15.19:0/3806673743 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v178) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd801f48b0 con 0x7fcd8015b5a0 2024-04-25T10:05:21.794 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:05:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.790+0000 7fcd86458640 1 -- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6006b4d0 msgr2=0x7fcd6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.790+0000 7fcd86458640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6006b4d0 0x7fcd6006d990 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fcd800b6960 tx=0x7fcd70009110 comp rx=0 tx=0).stop 2024-04-25T10:05:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.790+0000 7fcd86458640 1 -- 172.21.15.19:0/3806673743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd8015b5a0 msgr2=0x7fcd801f0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.790+0000 7fcd86458640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd8015b5a0 0x7fcd801f0f70 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fcd6802a220 tx=0x7fcd680046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.791+0000 7fcd86458640 1 -- 172.21.15.19:0/3806673743 shutdown_connections 2024-04-25T10:05:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.791+0000 7fcd86458640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fcd6006b4d0 0x7fcd6006d990 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.791+0000 7fcd86458640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80162450 0x7fcd800cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.791+0000 7fcd86458640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd8015b980 0x7fcd801f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.791+0000 7fcd86458640 1 --2- 172.21.15.19:0/3806673743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd8015b5a0 0x7fcd801f0f70 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.791+0000 7fcd86458640 1 -- 172.21.15.19:0/3806673743 >> 172.21.15.19:0/3806673743 conn(0x7fcd800d0530 msgr2=0x7fcd800b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.791+0000 7fcd86458640 1 -- 172.21.15.19:0/3806673743 shutdown_connections 2024-04-25T10:05:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.791+0000 7fcd86458640 1 -- 172.21.15.19:0/3806673743 wait complete. 2024-04-25T10:05:21.807 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.903+0000 7f22783be640 1 Processor -- start 2024-04-25T10:05:21.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.903+0000 7f22783be640 1 -- start start 2024-04-25T10:05:21.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.904+0000 7f22783be640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22701446d0 0x7f22700b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.904+0000 7f22783be640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2270144d50 0x7f22700b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.904+0000 7f22783be640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22700cccc0 con 0x7f22701452a0 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.904+0000 7f22783be640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22700b2b20 con 0x7f2270144d50 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.904+0000 7f22783be640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227006d400 con 0x7f22701446d0 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.905+0000 7f2275959640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2270144d50 0x7f22700b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.905+0000 7f227695b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f22701452a0 0x7f22700cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47030/0 (socket says 172.21.15.19:47030) 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.905+0000 7f227695b640 1 -- 172.21.15.19:0/2635207159 learned_addr learned my addr 172.21.15.19:0/2635207159 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.905+0000 7f227615a640 1 --2- 172.21.15.19:0/2635207159 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22701446d0 0x7f22700b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.905+0000 7f2275959640 1 -- 172.21.15.19:0/2635207159 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22701446d0 msgr2=0x7f22700b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.905+0000 7f2275959640 1 --2- 172.21.15.19:0/2635207159 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22701446d0 0x7f22700b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.905+0000 7f2275959640 1 -- 172.21.15.19:0/2635207159 >> v1:172.21.15.19:6789/0 conn(0x7f22701452a0 legacy=0x7f22700cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.905+0000 7f2275959640 1 -- 172.21.15.19:0/2635207159 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22700cc140 con 0x7f2270144d50 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f2275959640 1 --2- 172.21.15.19:0/2635207159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270144d50 0x7f22700b4460 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f22600097a0 tx=0x7f226002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343638cb56bd462e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f2275158640 1 -- 172.21.15.19:0/2635207159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260037070 con 0x7f2270144d50 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f2275158640 1 -- 172.21.15.19:0/2635207159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2260032440 con 0x7f2270144d50 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f2275158640 1 -- 172.21.15.19:0/2635207159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260032810 con 0x7f2270144d50 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f22783be640 1 -- 172.21.15.19:0/2635207159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270144d50 msgr2=0x7f22700b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f22783be640 1 --2- 172.21.15.19:0/2635207159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270144d50 0x7f22700b4460 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f22600097a0 tx=0x7f226002a180 comp rx=0 tx=0).stop 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f22783be640 1 -- 172.21.15.19:0/2635207159 shutdown_connections 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f22783be640 1 --2- 172.21.15.19:0/2635207159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2270144d50 0x7f22700b4460 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f22783be640 1 --2- 172.21.15.19:0/2635207159 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22701446d0 0x7f22700b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f22783be640 1 -- 172.21.15.19:0/2635207159 >> 172.21.15.19:0/2635207159 conn(0x7f227006abe0 msgr2=0x7f227006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f22783be640 1 -- 172.21.15.19:0/2635207159 shutdown_connections 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.906+0000 7f22783be640 1 -- 172.21.15.19:0/2635207159 wait complete. 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.908+0000 7f22783be640 1 Processor -- start 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.908+0000 7f22783be640 1 -- start start 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.908+0000 7f22783be640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22701446d0 0x7f22701f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.908+0000 7f22783be640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2270144d50 0x7f22700cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.908+0000 7f22783be640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22701452a0 0x7f227015b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.908+0000 7f22783be640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700cc700 con 0x7f22701446d0 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.908+0000 7f22783be640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700b49a0 con 0x7f2270144d50 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.908+0000 7f22783be640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22700cc9e0 con 0x7f22701452a0 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.908+0000 7f227615a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22701446d0 0x7f22701f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.908+0000 7f227615a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22701446d0 0x7f22701f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46120/0 (socket says 172.21.15.19:46120) 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.908+0000 7f227615a640 1 -- 172.21.15.19:0/2394862195 learned_addr learned my addr 172.21.15.19:0/2394862195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f2275959640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2270144d50 0x7f22700cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f227695b640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22701452a0 0x7f227015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f227615a640 1 -- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22701452a0 msgr2=0x7f227015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f227615a640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22701452a0 0x7f227015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f227615a640 1 -- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2270144d50 msgr2=0x7f22700cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f227615a640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2270144d50 0x7f22700cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f227615a640 1 -- 172.21.15.19:0/2394862195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22701f39b0 con 0x7f22701446d0 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f227695b640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22701452a0 0x7f227015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f227615a640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22701446d0 0x7f22701f4470 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f226c00d830 tx=0x7f226c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f2266ffd640 1 -- 172.21.15.19:0/2394862195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00b6c0 con 0x7f22701446d0 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f2266ffd640 1 -- 172.21.15.19:0/2394862195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c019070 con 0x7f22701446d0 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f22783be640 1 -- 172.21.15.19:0/2394862195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227013a6b0 con 0x7f22701446d0 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f2266ffd640 1 -- 172.21.15.19:0/2394862195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00ba80 con 0x7f22701446d0 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.909+0000 7f22783be640 1 -- 172.21.15.19:0/2394862195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227013ac70 con 0x7f22701446d0 2024-04-25T10:05:21.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.910+0000 7f22783be640 1 -- 172.21.15.19:0/2394862195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2270073a00 con 0x7f22701446d0 2024-04-25T10:05:21.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.911+0000 7f2266ffd640 1 -- 172.21.15.19:0/2394862195 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f226c01e460 con 0x7f22701446d0 2024-04-25T10:05:21.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.911+0000 7f2266ffd640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f225406b4d0 0x7f225406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.911+0000 7f2266ffd640 1 -- 172.21.15.19:0/2394862195 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f226c08a9c0 con 0x7f22701446d0 2024-04-25T10:05:21.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.911+0000 7f2275959640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f225406b4d0 0x7f225406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.912+0000 7f2275959640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f225406b4d0 0x7f225406d990 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2260034000 tx=0x7f226002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:21.915+0000 7f2266ffd640 1 -- 172.21.15.19:0/2394862195 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f226c08ce20 con 0x7f22701446d0 2024-04-25T10:05:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.113+0000 7f22783be640 1 -- 172.21.15.19:0/2394862195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f22700ce630 con 0x7f22701446d0 2024-04-25T10:05:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.114+0000 7f2266ffd640 1 -- 172.21.15.19:0/2394862195 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v178) v1 ==== 80+0+0 (secure 0 0 0) 0x7f226c059740 con 0x7f22701446d0 2024-04-25T10:05:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T10:05:22.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 -- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f225406b4d0 msgr2=0x7f225406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f225406b4d0 0x7f225406d990 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2260034000 tx=0x7f226002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:22.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 -- 172.21.15.19:0/2394862195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22701446d0 msgr2=0x7f22701f4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22701446d0 0x7f22701f4470 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f226c00d830 tx=0x7f226c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 -- 172.21.15.19:0/2394862195 shutdown_connections 2024-04-25T10:05:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f225406b4d0 0x7f225406d990 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22701452a0 0x7f227015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2270144d50 0x7f22700cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 --2- 172.21.15.19:0/2394862195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22701446d0 0x7f22701f4470 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 -- 172.21.15.19:0/2394862195 >> 172.21.15.19:0/2394862195 conn(0x7f227006abe0 msgr2=0x7f22700cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 -- 172.21.15.19:0/2394862195 shutdown_connections 2024-04-25T10:05:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:22.116+0000 7f22783be640 1 -- 172.21.15.19:0/2394862195 wait complete. 2024-04-25T10:05:22.131 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:05:22.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:22.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:22.954 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:05:22.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:22.955 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:05:22.970 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:05:22.970 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:22.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:22.986 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:22.986 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:23.026 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:23.026 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:05:23.026 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:23.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:23.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:23.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.162+0000 7f17ce16a640 1 Processor -- start 2024-04-25T10:05:23.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.162+0000 7f17ce16a640 1 -- start start 2024-04-25T10:05:23.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17ce16a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c815dbe0 0x7f17c80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17ce16a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17c80b2cb0 0x7f17c80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17ce16a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c80b97a0 con 0x7f17c80b92a0 2024-04-25T10:05:23.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17ce16a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c815dfc0 con 0x7f17c80b2cb0 2024-04-25T10:05:23.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17ce16a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80d1a00 con 0x7f17c815dbe0 2024-04-25T10:05:23.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17c6ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17c80b2cb0 0x7f17c80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17c6ffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17c80b2cb0 0x7f17c80d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46136/0 (socket says 172.21.15.19:46136) 2024-04-25T10:05:23.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17c6ffd640 1 -- 172.21.15.19:0/638648813 learned_addr learned my addr 172.21.15.19:0/638648813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:23.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17c77fe640 1 --2- 172.21.15.19:0/638648813 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c815dbe0 0x7f17c80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17c6ffd640 1 -- 172.21.15.19:0/638648813 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c815dbe0 msgr2=0x7f17c80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17c6ffd640 1 --2- 172.21.15.19:0/638648813 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c815dbe0 0x7f17c80d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17c6ffd640 1 -- 172.21.15.19:0/638648813 >> v1:172.21.15.112:6789/0 conn(0x7f17c80b92a0 legacy=0x7f17c80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.163+0000 7f17c6ffd640 1 -- 172.21.15.19:0/638648813 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c80d1c50 con 0x7f17c80b2cb0 2024-04-25T10:05:23.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.164+0000 7f17c6ffd640 1 --2- 172.21.15.19:0/638648813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c80b2cb0 0x7f17c80d0e50 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009870 tx=0x7f17bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3598f11ac1c14b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.164+0000 7f17c67fc640 1 -- 172.21.15.19:0/638648813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc037070 con 0x7f17c80b2cb0 2024-04-25T10:05:23.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.164+0000 7f17c67fc640 1 -- 172.21.15.19:0/638648813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc032440 con 0x7f17c80b2cb0 2024-04-25T10:05:23.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.164+0000 7f17c67fc640 1 -- 172.21.15.19:0/638648813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc032810 con 0x7f17c80b2cb0 2024-04-25T10:05:23.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.165+0000 7f17ce16a640 1 -- 172.21.15.19:0/638648813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c80b2cb0 msgr2=0x7f17c80d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.165+0000 7f17ce16a640 1 --2- 172.21.15.19:0/638648813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c80b2cb0 0x7f17c80d0e50 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009870 tx=0x7f17bc02a270 comp rx=0 tx=0).stop 2024-04-25T10:05:23.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.165+0000 7f17ce16a640 1 -- 172.21.15.19:0/638648813 shutdown_connections 2024-04-25T10:05:23.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.165+0000 7f17ce16a640 1 --2- 172.21.15.19:0/638648813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c80b2cb0 0x7f17c80d0e50 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.165+0000 7f17ce16a640 1 --2- 172.21.15.19:0/638648813 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c815dbe0 0x7f17c80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.165+0000 7f17ce16a640 1 -- 172.21.15.19:0/638648813 >> 172.21.15.19:0/638648813 conn(0x7f17c8061a20 msgr2=0x7f17c8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.165+0000 7f17ce16a640 1 -- 172.21.15.19:0/638648813 shutdown_connections 2024-04-25T10:05:23.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.165+0000 7f17ce16a640 1 -- 172.21.15.19:0/638648813 wait complete. 2024-04-25T10:05:23.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.166+0000 7f17ce16a640 1 Processor -- start 2024-04-25T10:05:23.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.166+0000 7f17ce16a640 1 -- start start 2024-04-25T10:05:23.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17ce16a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c80b2cb0 0x7f17c815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17ce16a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c80b92a0 0x7f17c80c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17ce16a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c815dbe0 0x7f17c80b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17ce16a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80b6ab0 con 0x7f17c80b92a0 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17ce16a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80d2210 con 0x7f17c80b2cb0 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17ce16a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80b6c30 con 0x7f17c815dbe0 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c80b92a0 0x7f17c80c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17c6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c80b92a0 0x7f17c80c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46150/0 (socket says 172.21.15.19:46150) 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17c6ffd640 1 -- 172.21.15.19:0/1203207827 learned_addr learned my addr 172.21.15.19:0/1203207827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17c77fe640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c80b2cb0 0x7f17c815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17c7fff640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c815dbe0 0x7f17c80b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17c6ffd640 1 -- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c815dbe0 msgr2=0x7f17c80b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17c6ffd640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c815dbe0 0x7f17c80b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17c6ffd640 1 -- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c80b2cb0 msgr2=0x7f17c815f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17c6ffd640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c80b2cb0 0x7f17c815f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17c6ffd640 1 -- 172.21.15.19:0/1203207827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c815e710 con 0x7f17c80b92a0 2024-04-25T10:05:23.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.167+0000 7f17c6ffd640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c80b92a0 0x7f17c80c9450 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000c00 tx=0x7f17bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.168+0000 7f17affff640 1 -- 172.21.15.19:0/1203207827 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc037040 con 0x7f17c80b92a0 2024-04-25T10:05:23.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.168+0000 7f17affff640 1 -- 172.21.15.19:0/1203207827 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc042070 con 0x7f17c80b92a0 2024-04-25T10:05:23.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.168+0000 7f17ce16a640 1 -- 172.21.15.19:0/1203207827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c815e9f0 con 0x7f17c80b92a0 2024-04-25T10:05:23.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.168+0000 7f17affff640 1 -- 172.21.15.19:0/1203207827 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc030880 con 0x7f17c80b92a0 2024-04-25T10:05:23.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.168+0000 7f17ce16a640 1 -- 172.21.15.19:0/1203207827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c815efb0 con 0x7f17c80b92a0 2024-04-25T10:05:23.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.169+0000 7f17ce16a640 1 -- 172.21.15.19:0/1203207827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c8073990 con 0x7f17c80b92a0 2024-04-25T10:05:23.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.169+0000 7f17affff640 1 -- 172.21.15.19:0/1203207827 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17bc030a20 con 0x7f17c80b92a0 2024-04-25T10:05:23.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.170+0000 7f17affff640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17b406b4d0 0x7f17b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.170+0000 7f17affff640 1 -- 172.21.15.19:0/1203207827 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f17bc0aabf0 con 0x7f17c80b92a0 2024-04-25T10:05:23.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.170+0000 7f17c77fe640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17b406b4d0 0x7f17b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.170+0000 7f17c77fe640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17b406b4d0 0x7f17b406d990 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009a30 tx=0x7f17b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.174+0000 7f17affff640 1 -- 172.21.15.19:0/1203207827 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17bc0798e0 con 0x7f17c80b92a0 2024-04-25T10:05:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.392+0000 7f17ce16a640 1 -- 172.21.15.19:0/1203207827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17c80b48b0 con 0x7f17c80b92a0 2024-04-25T10:05:23.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.392+0000 7f17affff640 1 -- 172.21.15.19:0/1203207827 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4948 (secure 0 0 0) 0x7f17bc0791e0 con 0x7f17c80b92a0 2024-04-25T10:05:23.398 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:23.398 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":116,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/662861975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":662861975},{"type":"v1","addr":"172.21.15.19:6835","nonce":662861975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3439611393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3439611393},{"type":"v1","addr":"172.21.15.112:6836","nonce":3439611393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721666508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721666508},{"type":"v1","addr":"172.21.15.112:6837","nonce":721666508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:13.300379+0000","modified":"2024-04-25T10:05:15.294971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11166":{"gid":11166,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6833/829980210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":829980210},{"type":"v1","addr":"172.21.15.19:6833","nonce":829980210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:05:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 116 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 -- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17b406b4d0 msgr2=0x7f17b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17b406b4d0 0x7f17b406d990 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009a30 tx=0x7f17b8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 -- 172.21.15.19:0/1203207827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c80b92a0 msgr2=0x7f17c80c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c80b92a0 0x7f17c80c9450 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000c00 tx=0x7f17bc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 -- 172.21.15.19:0/1203207827 shutdown_connections 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f17b406b4d0 0x7f17b406d990 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c815dbe0 0x7f17c80b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c80b92a0 0x7f17c80c9450 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 --2- 172.21.15.19:0/1203207827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c80b2cb0 0x7f17c815f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 -- 172.21.15.19:0/1203207827 >> 172.21.15.19:0/1203207827 conn(0x7f17c8061a20 msgr2=0x7f17c80b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 -- 172.21.15.19:0/1203207827 shutdown_connections 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.394+0000 7f17ce16a640 1 -- 172.21.15.19:0/1203207827 wait complete. 2024-04-25T10:05:23.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:23.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.506+0000 7f156d076640 1 Processor -- start 2024-04-25T10:05:23.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.506+0000 7f156d076640 1 -- start start 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f156d076640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f156815dd30 0x7f15680b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f156d076640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15680b9770 0x7f15680d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f156d076640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15680d1df0 con 0x7f15680d2e90 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f156d076640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15680b7a50 con 0x7f15680b9770 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f156d076640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15680d20d0 con 0x7f156815dd30 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f155ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15680b9770 0x7f15680d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f155ffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15680b9770 0x7f15680d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46166/0 (socket says 172.21.15.19:46166) 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f155ffff640 1 -- 172.21.15.19:0/3553752266 learned_addr learned my addr 172.21.15.19:0/3553752266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f1566d9d640 1 --2- 172.21.15.19:0/3553752266 >> v2:172.21.15.112:3300/0 conn(0x7f156815dd30 0x7f15680b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f155ffff640 1 -- 172.21.15.19:0/3553752266 >> v2:172.21.15.112:3300/0 conn(0x7f156815dd30 msgr2=0x7f15680b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f155ffff640 1 --2- 172.21.15.19:0/3553752266 >> v2:172.21.15.112:3300/0 conn(0x7f156815dd30 0x7f15680b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f155ffff640 1 -- 172.21.15.19:0/3553752266 >> v1:172.21.15.112:6789/0 conn(0x7f15680d2e90 legacy=0x7f15680b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f155ffff640 1 -- 172.21.15.19:0/3553752266 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15680d23b0 con 0x7f15680b9770 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.507+0000 7f1566d9d640 1 --2- 172.21.15.19:0/3553752266 >> v2:172.21.15.112:3300/0 conn(0x7f156815dd30 0x7f15680b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.508+0000 7f155ffff640 1 --2- 172.21.15.19:0/3553752266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15680b9770 0x7f15680d03c0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f156806d000 tx=0x7f155002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db629c1f6068b121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.508+0000 7f156659c640 1 -- 172.21.15.19:0/3553752266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550004030 con 0x7f15680b9770 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.508+0000 7f156659c640 1 -- 172.21.15.19:0/3553752266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15500041d0 con 0x7f15680b9770 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.508+0000 7f156659c640 1 -- 172.21.15.19:0/3553752266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550003610 con 0x7f15680b9770 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.509+0000 7f156d076640 1 -- 172.21.15.19:0/3553752266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15680b9770 msgr2=0x7f15680d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.509+0000 7f156d076640 1 --2- 172.21.15.19:0/3553752266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15680b9770 0x7f15680d03c0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f156806d000 tx=0x7f155002a130 comp rx=0 tx=0).stop 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.509+0000 7f156d076640 1 -- 172.21.15.19:0/3553752266 shutdown_connections 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.509+0000 7f156d076640 1 --2- 172.21.15.19:0/3553752266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15680b9770 0x7f15680d03c0 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.509+0000 7f156d076640 1 --2- 172.21.15.19:0/3553752266 >> v2:172.21.15.112:3300/0 conn(0x7f156815dd30 0x7f15680b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.509+0000 7f156d076640 1 -- 172.21.15.19:0/3553752266 >> 172.21.15.19:0/3553752266 conn(0x7f1568061c30 msgr2=0x7f156806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.509+0000 7f156d076640 1 -- 172.21.15.19:0/3553752266 shutdown_connections 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.509+0000 7f156d076640 1 -- 172.21.15.19:0/3553752266 wait complete. 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.510+0000 7f156d076640 1 Processor -- start 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.510+0000 7f156d076640 1 -- start start 2024-04-25T10:05:23.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f156d076640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15680b9770 0x7f1568160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f156d076640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15680d2e90 0x7f15681545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f156d076640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f156815dd30 0x7f156815a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f156d076640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15680d1a10 con 0x7f15680d2e90 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f156d076640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15680d2970 con 0x7f156815dd30 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f156d076640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15680d1b90 con 0x7f15680b9770 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15680d2e90 0x7f15681545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15680d2e90 0x7f15681545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46182/0 (socket says 172.21.15.19:46182) 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155ffff640 1 -- 172.21.15.19:0/1327530571 learned_addr learned my addr 172.21.15.19:0/1327530571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f156759e640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f156815dd30 0x7f156815a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f1566d9d640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15680b9770 0x7f1568160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155ffff640 1 -- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15680b9770 msgr2=0x7f1568160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155ffff640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15680b9770 0x7f1568160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155ffff640 1 -- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f156815dd30 msgr2=0x7f156815a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155ffff640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f156815dd30 0x7f156815a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155ffff640 1 -- 172.21.15.19:0/1327530571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1568156550 con 0x7f15680d2e90 2024-04-25T10:05:23.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155ffff640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15680d2e90 0x7f15681545e0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f155002a170 tx=0x7f15500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155f7fe640 1 -- 172.21.15.19:0/1327530571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550041020 con 0x7f15680d2e90 2024-04-25T10:05:23.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155f7fe640 1 -- 172.21.15.19:0/1327530571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1550003c50 con 0x7f15680d2e90 2024-04-25T10:05:23.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f156d076640 1 -- 172.21.15.19:0/1327530571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1568154e30 con 0x7f15680d2e90 2024-04-25T10:05:23.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.511+0000 7f155f7fe640 1 -- 172.21.15.19:0/1327530571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550004030 con 0x7f15680d2e90 2024-04-25T10:05:23.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.512+0000 7f156d076640 1 -- 172.21.15.19:0/1327530571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1568156ab0 con 0x7f15680d2e90 2024-04-25T10:05:23.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.513+0000 7f156d076640 1 -- 172.21.15.19:0/1327530571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1568073990 con 0x7f15680d2e90 2024-04-25T10:05:23.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.513+0000 7f155f7fe640 1 -- 172.21.15.19:0/1327530571 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1550044020 con 0x7f15680d2e90 2024-04-25T10:05:23.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.513+0000 7f155f7fe640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f154806b4d0 0x7f154806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.513+0000 7f155f7fe640 1 -- 172.21.15.19:0/1327530571 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f15500aafd0 con 0x7f15680d2e90 2024-04-25T10:05:23.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.513+0000 7f1566d9d640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f154806b4d0 0x7f154806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.514+0000 7f1566d9d640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f154806b4d0 0x7f154806d990 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1558006fd0 tx=0x7f1558008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.517+0000 7f155f7fe640 1 -- 172.21.15.19:0/1327530571 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f155004b030 con 0x7f15680d2e90 2024-04-25T10:05:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.698+0000 7f156d076640 1 -- 172.21.15.19:0/1327530571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1568156f10 con 0x7f15680d2e90 2024-04-25T10:05:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.700+0000 7f155f7fe640 1 -- 172.21.15.19:0/1327530571 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f1568156550 con 0x7f15680d2e90 2024-04-25T10:05:23.705 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:23.705 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":178,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:14.934502+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:11.578636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:12.274609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:23.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.701+0000 7f156d076640 1 -- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f154806b4d0 msgr2=0x7f154806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.701+0000 7f156d076640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f154806b4d0 0x7f154806d990 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1558006fd0 tx=0x7f1558008040 comp rx=0 tx=0).stop 2024-04-25T10:05:23.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.701+0000 7f156d076640 1 -- 172.21.15.19:0/1327530571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15680d2e90 msgr2=0x7f15681545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.701+0000 7f156d076640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15680d2e90 0x7f15681545e0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f155002a170 tx=0x7f15500046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.702+0000 7f156d076640 1 -- 172.21.15.19:0/1327530571 shutdown_connections 2024-04-25T10:05:23.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.702+0000 7f156d076640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f154806b4d0 0x7f154806d990 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.702+0000 7f156d076640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f156815dd30 0x7f156815a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.702+0000 7f156d076640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15680d2e90 0x7f15681545e0 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.702+0000 7f156d076640 1 --2- 172.21.15.19:0/1327530571 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15680b9770 0x7f1568160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.702+0000 7f156d076640 1 -- 172.21.15.19:0/1327530571 >> 172.21.15.19:0/1327530571 conn(0x7f1568061c30 msgr2=0x7f15680d3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.702+0000 7f156d076640 1 -- 172.21.15.19:0/1327530571 shutdown_connections 2024-04-25T10:05:23.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.702+0000 7f156d076640 1 -- 172.21.15.19:0/1327530571 wait complete. 2024-04-25T10:05:23.718 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:11.578636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '177', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:12.274609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '178', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:23.719 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.817+0000 7fca87128640 1 Processor -- start 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.817+0000 7fca87128640 1 -- start start 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca87128640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fca8015b7d0 0x7fca800cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca87128640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fca8006c9e0 0x7fca800b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca87128640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8006cf50 con 0x7fca800b5620 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca87128640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca800ce530 con 0x7fca8006c9e0 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca87128640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca800ce810 con 0x7fca8015b7d0 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca856c5640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fca8006c9e0 0x7fca800b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca7ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fca800b5620 0x7fca8006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47036/0 (socket says 172.21.15.19:47036) 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca7ffff640 1 -- 172.21.15.19:0/1566700921 learned_addr learned my addr 172.21.15.19:0/1566700921 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca84ec4640 1 --2- 172.21.15.19:0/1566700921 >> v2:172.21.15.112:3300/0 conn(0x7fca8015b7d0 0x7fca800cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca856c5640 1 -- 172.21.15.19:0/1566700921 >> v2:172.21.15.112:3300/0 conn(0x7fca8015b7d0 msgr2=0x7fca800cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca856c5640 1 --2- 172.21.15.19:0/1566700921 >> v2:172.21.15.112:3300/0 conn(0x7fca8015b7d0 0x7fca800cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca856c5640 1 -- 172.21.15.19:0/1566700921 >> v1:172.21.15.19:6789/0 conn(0x7fca800b5620 legacy=0x7fca8006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.818+0000 7fca856c5640 1 -- 172.21.15.19:0/1566700921 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800ceaf0 con 0x7fca8006c9e0 2024-04-25T10:05:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.819+0000 7fca856c5640 1 --2- 172.21.15.19:0/1566700921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8006c9e0 0x7fca800b5ab0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7fca8008c0d0 tx=0x7fca6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380482f580523839 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.819+0000 7fca7f7fe640 1 -- 172.21.15.19:0/1566700921 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68037070 con 0x7fca8006c9e0 2024-04-25T10:05:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.819+0000 7fca7f7fe640 1 -- 172.21.15.19:0/1566700921 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68032440 con 0x7fca8006c9e0 2024-04-25T10:05:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.819+0000 7fca7f7fe640 1 -- 172.21.15.19:0/1566700921 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68032810 con 0x7fca8006c9e0 2024-04-25T10:05:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.820+0000 7fca87128640 1 -- 172.21.15.19:0/1566700921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8006c9e0 msgr2=0x7fca800b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.820+0000 7fca87128640 1 --2- 172.21.15.19:0/1566700921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8006c9e0 0x7fca800b5ab0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7fca8008c0d0 tx=0x7fca6802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.820+0000 7fca87128640 1 -- 172.21.15.19:0/1566700921 shutdown_connections 2024-04-25T10:05:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.820+0000 7fca87128640 1 --2- 172.21.15.19:0/1566700921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8006c9e0 0x7fca800b5ab0 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.820+0000 7fca87128640 1 --2- 172.21.15.19:0/1566700921 >> v2:172.21.15.112:3300/0 conn(0x7fca8015b7d0 0x7fca800cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.820+0000 7fca87128640 1 -- 172.21.15.19:0/1566700921 >> 172.21.15.19:0/1566700921 conn(0x7fca800d0800 msgr2=0x7fca800d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.820+0000 7fca87128640 1 -- 172.21.15.19:0/1566700921 shutdown_connections 2024-04-25T10:05:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.820+0000 7fca87128640 1 -- 172.21.15.19:0/1566700921 wait complete. 2024-04-25T10:05:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.821+0000 7fca87128640 1 Processor -- start 2024-04-25T10:05:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.821+0000 7fca87128640 1 -- start start 2024-04-25T10:05:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca87128640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca8006c9e0 0x7fca800c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca87128640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca800b5620 0x7fca8015ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca87128640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8015b7d0 0x7fca8015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca87128640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800cf6d0 con 0x7fca8015b7d0 2024-04-25T10:05:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca87128640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800cf550 con 0x7fca8006c9e0 2024-04-25T10:05:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca87128640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8015bbb0 con 0x7fca800b5620 2024-04-25T10:05:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca856c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8015b7d0 0x7fca8015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca856c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8015b7d0 0x7fca8015d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46210/0 (socket says 172.21.15.19:46210) 2024-04-25T10:05:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca856c5640 1 -- 172.21.15.19:0/4141732180 learned_addr learned my addr 172.21.15.19:0/4141732180 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca7ffff640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca800b5620 0x7fca8015ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca84ec4640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca8006c9e0 0x7fca800c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca856c5640 1 -- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca800b5620 msgr2=0x7fca8015ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca856c5640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca800b5620 0x7fca8015ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca856c5640 1 -- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca8006c9e0 msgr2=0x7fca800c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca856c5640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca8006c9e0 0x7fca800c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca856c5640 1 -- 172.21.15.19:0/4141732180 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800cb550 con 0x7fca8015b7d0 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.822+0000 7fca856c5640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8015b7d0 0x7fca8015d1f0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7fca68035040 tx=0x7fca680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.823+0000 7fca7d7fa640 1 -- 172.21.15.19:0/4141732180 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68041020 con 0x7fca8015b7d0 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.823+0000 7fca7d7fa640 1 -- 172.21.15.19:0/4141732180 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6803d640 con 0x7fca8015b7d0 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.823+0000 7fca87128640 1 -- 172.21.15.19:0/4141732180 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca800ca640 con 0x7fca8015b7d0 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.823+0000 7fca7d7fa640 1 -- 172.21.15.19:0/4141732180 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68037040 con 0x7fca8015b7d0 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.823+0000 7fca87128640 1 -- 172.21.15.19:0/4141732180 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca801f3d50 con 0x7fca8015b7d0 2024-04-25T10:05:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.824+0000 7fca87128640 1 -- 172.21.15.19:0/4141732180 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca80073a00 con 0x7fca8015b7d0 2024-04-25T10:05:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.824+0000 7fca7d7fa640 1 -- 172.21.15.19:0/4141732180 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca68044020 con 0x7fca8015b7d0 2024-04-25T10:05:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.825+0000 7fca7d7fa640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca6006b4d0 0x7fca6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.825+0000 7fca7d7fa640 1 -- 172.21.15.19:0/4141732180 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fca680aab10 con 0x7fca8015b7d0 2024-04-25T10:05:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.825+0000 7fca84ec4640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca6006b4d0 0x7fca6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.825+0000 7fca84ec4640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca6006b4d0 0x7fca6006d990 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fca800c7520 tx=0x7fca78008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:23.829+0000 7fca7d7fa640 1 -- 172.21.15.19:0/4141732180 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca6804b030 con 0x7fca8015b7d0 2024-04-25T10:05:24.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.015+0000 7fca87128640 1 -- 172.21.15.19:0/4141732180 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca800cded0 con 0x7fca8015b7d0 2024-04-25T10:05:24.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.015+0000 7fca7d7fa640 1 -- 172.21.15.19:0/4141732180 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v178) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca68079810 con 0x7fca8015b7d0 2024-04-25T10:05:24.020 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:05:24.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 -- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca6006b4d0 msgr2=0x7fca6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca6006b4d0 0x7fca6006d990 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fca800c7520 tx=0x7fca78008040 comp rx=0 tx=0).stop 2024-04-25T10:05:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 -- 172.21.15.19:0/4141732180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8015b7d0 msgr2=0x7fca8015d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8015b7d0 0x7fca8015d1f0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7fca68035040 tx=0x7fca680046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 -- 172.21.15.19:0/4141732180 shutdown_connections 2024-04-25T10:05:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fca6006b4d0 0x7fca6006d990 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8015b7d0 0x7fca8015d1f0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca800b5620 0x7fca8015ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 --2- 172.21.15.19:0/4141732180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca8006c9e0 0x7fca800c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 -- 172.21.15.19:0/4141732180 >> 172.21.15.19:0/4141732180 conn(0x7fca800d0800 msgr2=0x7fca80160320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 -- 172.21.15.19:0/4141732180 shutdown_connections 2024-04-25T10:05:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.017+0000 7fca87128640 1 -- 172.21.15.19:0/4141732180 wait complete. 2024-04-25T10:05:24.034 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.132+0000 7fc13905d640 1 Processor -- start 2024-04-25T10:05:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.132+0000 7fc13905d640 1 -- start start 2024-04-25T10:05:24.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.132+0000 7fc13905d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc13406c9e0 0x7fc1340b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.132+0000 7fc13905d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc13406cf50 con 0x7fc13415b7d0 2024-04-25T10:05:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13905d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1340ce530 con 0x7fc1340b5620 2024-04-25T10:05:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13905d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1340ce810 con 0x7fc13406c9e0 2024-04-25T10:05:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc132d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc13415b7d0 0x7fc1340cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47052/0 (socket says 172.21.15.19:47052) 2024-04-25T10:05:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc132d9d640 1 -- 172.21.15.19:0/3421527088 learned_addr learned my addr 172.21.15.19:0/3421527088 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13359e640 1 --2- 172.21.15.19:0/3421527088 >> v2:172.21.15.112:3300/0 conn(0x7fc13406c9e0 0x7fc1340b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052723502 0 0) 0x7fc13406cf50 con 0x7fc13415b7d0 2024-04-25T10:05:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc120003520 con 0x7fc13415b7d0 2024-04-25T10:05:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2257264526 0 0) 0x7fc120003520 con 0x7fc13415b7d0 2024-04-25T10:05:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 >> v2:172.21.15.112:3300/0 conn(0x7fc13406c9e0 msgr2=0x7fc1340b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 --2- 172.21.15.19:0/3421527088 >> v2:172.21.15.112:3300/0 conn(0x7fc13406c9e0 0x7fc1340b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 >> v1:172.21.15.112:6789/0 conn(0x7fc1340b5620 legacy=0x7fc13406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340ceaf0 con 0x7fc13415b7d0 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fc124001670 con 0x7fc13415b7d0 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 >> v1:172.21.15.19:6789/0 conn(0x7fc13415b7d0 legacy=0x7fc1340cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 --2- 172.21.15.19:0/3421527088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1200039b0 0x7fc120003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 --2- 172.21.15.19:0/3421527088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc120004800 0x7fc120004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 --2- 172.21.15.19:0/3421527088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1200051c0 0x7fc12000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340ce810 con 0x7fc1200051c0 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12000c060 con 0x7fc1200039b0 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12000c260 con 0x7fc120004800 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc13359e640 1 --2- 172.21.15.19:0/3421527088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1200051c0 0x7fc12000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.133+0000 7fc132d9d640 1 --2- 172.21.15.19:0/3421527088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1200039b0 0x7fc120003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.134+0000 7fc12a59c640 1 --2- 172.21.15.19:0/3421527088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc120004800 0x7fc120004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.134+0000 7fc13359e640 1 -- 172.21.15.19:0/3421527088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc120004800 msgr2=0x7fc120004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.134+0000 7fc13359e640 1 --2- 172.21.15.19:0/3421527088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc120004800 0x7fc120004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.134+0000 7fc13359e640 1 -- 172.21.15.19:0/3421527088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1200039b0 msgr2=0x7fc120003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.134+0000 7fc13359e640 1 --2- 172.21.15.19:0/3421527088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1200039b0 0x7fc120003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.134+0000 7fc13359e640 1 -- 172.21.15.19:0/3421527088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc120003520 con 0x7fc1200051c0 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.134+0000 7fc13359e640 1 -- 172.21.15.19:0/3421527088 reap_dead start 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.134+0000 7fc13359e640 1 --2- 172.21.15.19:0/3421527088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1200051c0 0x7fc12000b9c0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009fc0 tx=0x7fc120005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125e33590381b738 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.135+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c035020 con 0x7fc1200051c0 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.135+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12c003bd0 con 0x7fc1200051c0 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.135+0000 7fc13259c640 1 -- 172.21.15.19:0/3421527088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c009c10 con 0x7fc1200051c0 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.135+0000 7fc13905d640 1 -- 172.21.15.19:0/3421527088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1200051c0 msgr2=0x7fc12000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.135+0000 7fc13905d640 1 --2- 172.21.15.19:0/3421527088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1200051c0 0x7fc12000b9c0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009fc0 tx=0x7fc120005830 comp rx=0 tx=0).stop 2024-04-25T10:05:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.135+0000 7fc13905d640 1 -- 172.21.15.19:0/3421527088 shutdown_connections 2024-04-25T10:05:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.135+0000 7fc13905d640 1 --2- 172.21.15.19:0/3421527088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1200051c0 0x7fc12000b9c0 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.135+0000 7fc13905d640 1 -- 172.21.15.19:0/3421527088 >> 172.21.15.19:0/3421527088 conn(0x7fc1340d0800 msgr2=0x7fc1340d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.135+0000 7fc13905d640 1 -- 172.21.15.19:0/3421527088 shutdown_connections 2024-04-25T10:05:24.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.135+0000 7fc13905d640 1 -- 172.21.15.19:0/3421527088 wait complete. 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.136+0000 7fc13905d640 1 Processor -- start 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.137+0000 7fc13905d640 1 -- start start 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.137+0000 7fc13905d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1200051c0 0x7fc1340cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.137+0000 7fc13905d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1340cd1d0 0x7fc1340b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.137+0000 7fc13905d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340c8a40 0x7fc1340c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.137+0000 7fc13905d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13415bd30 con 0x7fc1340c8a40 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.137+0000 7fc13905d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13415bbb0 con 0x7fc1200051c0 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.137+0000 7fc13905d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340b5ab0 con 0x7fc1340cd1d0 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.137+0000 7fc13359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340c8a40 0x7fc1340c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.137+0000 7fc13359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340c8a40 0x7fc1340c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46226/0 (socket says 172.21.15.19:46226) 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.137+0000 7fc13359e640 1 -- 172.21.15.19:0/463992107 learned_addr learned my addr 172.21.15.19:0/463992107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:24.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc132d9d640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1200051c0 0x7fc1340cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc12a59c640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1340cd1d0 0x7fc1340b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc13359e640 1 -- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1340cd1d0 msgr2=0x7fc1340b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc13359e640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1340cd1d0 0x7fc1340b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc13359e640 1 -- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1200051c0 msgr2=0x7fc1340cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc13359e640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1200051c0 0x7fc1340cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc13359e640 1 -- 172.21.15.19:0/463992107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340cda00 con 0x7fc1340c8a40 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc132d9d640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1200051c0 0x7fc1340cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc13359e640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340c8a40 0x7fc1340c8ec0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00e230 tx=0x7fc12c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc12bfff640 1 -- 172.21.15.19:0/463992107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c040070 con 0x7fc1340c8a40 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc12bfff640 1 -- 172.21.15.19:0/463992107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12c03c680 con 0x7fc1340c8a40 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc13905d640 1 -- 172.21.15.19:0/463992107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13414a8f0 con 0x7fc1340c8a40 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc12bfff640 1 -- 172.21.15.19:0/463992107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c035030 con 0x7fc1340c8a40 2024-04-25T10:05:24.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.138+0000 7fc13905d640 1 -- 172.21.15.19:0/463992107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13414aeb0 con 0x7fc1340c8a40 2024-04-25T10:05:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.139+0000 7fc13905d640 1 -- 172.21.15.19:0/463992107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc134073a00 con 0x7fc1340c8a40 2024-04-25T10:05:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.140+0000 7fc12bfff640 1 -- 172.21.15.19:0/463992107 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc12c043070 con 0x7fc1340c8a40 2024-04-25T10:05:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.140+0000 7fc12bfff640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc11406b4d0 0x7fc11406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.140+0000 7fc12bfff640 1 -- 172.21.15.19:0/463992107 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc12c0ab520 con 0x7fc1340c8a40 2024-04-25T10:05:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.140+0000 7fc132d9d640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc11406b4d0 0x7fc11406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.141+0000 7fc132d9d640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc11406b4d0 0x7fc11406d990 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc124001470 tx=0x7fc124009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.144+0000 7fc12bfff640 1 -- 172.21.15.19:0/463992107 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc12c079b40 con 0x7fc1340c8a40 2024-04-25T10:05:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.322+0000 7fc13905d640 1 -- 172.21.15.19:0/463992107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc13414b080 con 0x7fc1340c8a40 2024-04-25T10:05:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.322+0000 7fc12bfff640 1 -- 172.21.15.19:0/463992107 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v178) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc12c07dbe0 con 0x7fc1340c8a40 2024-04-25T10:05:24.328 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 -- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc11406b4d0 msgr2=0x7fc11406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc11406b4d0 0x7fc11406d990 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc124001470 tx=0x7fc124009040 comp rx=0 tx=0).stop 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 -- 172.21.15.19:0/463992107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340c8a40 msgr2=0x7fc1340c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340c8a40 0x7fc1340c8ec0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00e230 tx=0x7fc12c009750 comp rx=0 tx=0).stop 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 -- 172.21.15.19:0/463992107 shutdown_connections 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc11406b4d0 0x7fc11406d990 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340c8a40 0x7fc1340c8ec0 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1340cd1d0 0x7fc1340b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 --2- 172.21.15.19:0/463992107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1200051c0 0x7fc1340cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 -- 172.21.15.19:0/463992107 >> 172.21.15.19:0/463992107 conn(0x7fc1340d0800 msgr2=0x7fc13415b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 -- 172.21.15.19:0/463992107 shutdown_connections 2024-04-25T10:05:24.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.324+0000 7fc13905d640 1 -- 172.21.15.19:0/463992107 wait complete. 2024-04-25T10:05:24.340 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:24.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.438+0000 7f74afab3640 1 Processor -- start 2024-04-25T10:05:24.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.438+0000 7f74afab3640 1 -- start start 2024-04-25T10:05:24.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.439+0000 7f74afab3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f74a815de80 0x7f74a80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.439+0000 7f74afab3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74a80ccfe0 0x7f74a80cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.439+0000 7f74afab3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a80afc80 con 0x7f74a80cefb0 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.439+0000 7f74afab3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a80ce4d0 con 0x7f74a815de80 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.439+0000 7f74afab3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80ce7b0 con 0x7f74a80ccfe0 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.439+0000 7f74ad04e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f74a80cefb0 0x7f74a80cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47062/0 (socket says 172.21.15.19:47062) 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.439+0000 7f74ad04e640 1 -- 172.21.15.19:0/1564387194 learned_addr learned my addr 172.21.15.19:0/1564387194 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.440+0000 7f74ad84f640 1 --2- 172.21.15.19:0/1564387194 >> v2:172.21.15.112:3300/0 conn(0x7f74a815de80 0x7f74a80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.440+0000 7f74ae050640 1 --2- 172.21.15.19:0/1564387194 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74a80ccfe0 0x7f74a80cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.440+0000 7f74ac84d640 1 -- 172.21.15.19:0/1564387194 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396850681 0 0) 0x7f74a80afc80 con 0x7f74a80cefb0 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.440+0000 7f74ac84d640 1 -- 172.21.15.19:0/1564387194 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7490003520 con 0x7f74a80cefb0 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.440+0000 7f74ac84d640 1 -- 172.21.15.19:0/1564387194 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 763633976 0 0) 0x7f7490003520 con 0x7f74a80cefb0 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.440+0000 7f74ad84f640 1 -- 172.21.15.19:0/1564387194 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74a80ccfe0 msgr2=0x7f74a80cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.440+0000 7f74ad84f640 1 --2- 172.21.15.19:0/1564387194 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74a80ccfe0 0x7f74a80cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.440+0000 7f74ad84f640 1 -- 172.21.15.19:0/1564387194 >> v1:172.21.15.19:6789/0 conn(0x7f74a80cefb0 legacy=0x7f74a80cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.440+0000 7f74ad84f640 1 -- 172.21.15.19:0/1564387194 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a80cea90 con 0x7f74a815de80 2024-04-25T10:05:24.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.440+0000 7f74ae050640 1 --2- 172.21.15.19:0/1564387194 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74a80ccfe0 0x7f74a80cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.441+0000 7f74ad84f640 1 --2- 172.21.15.19:0/1564387194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a815de80 0x7f74a80cadb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f74a00097a0 tx=0x7f74a002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=774e6cdb1e614aed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.441+0000 7f74ac84d640 1 -- 172.21.15.19:0/1564387194 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0037070 con 0x7f74a815de80 2024-04-25T10:05:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.441+0000 7f74ac84d640 1 -- 172.21.15.19:0/1564387194 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a002aba0 con 0x7f74a815de80 2024-04-25T10:05:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.441+0000 7f74ac84d640 1 -- 172.21.15.19:0/1564387194 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c001670 con 0x7f74a815de80 2024-04-25T10:05:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.441+0000 7f74afab3640 1 -- 172.21.15.19:0/1564387194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a815de80 msgr2=0x7f74a80cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.441+0000 7f74afab3640 1 --2- 172.21.15.19:0/1564387194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a815de80 0x7f74a80cadb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f74a00097a0 tx=0x7f74a002a110 comp rx=0 tx=0).stop 2024-04-25T10:05:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.441+0000 7f74afab3640 1 -- 172.21.15.19:0/1564387194 shutdown_connections 2024-04-25T10:05:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.441+0000 7f74afab3640 1 --2- 172.21.15.19:0/1564387194 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74a80ccfe0 0x7f74a80cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.441+0000 7f74afab3640 1 --2- 172.21.15.19:0/1564387194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a815de80 0x7f74a80cadb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.441+0000 7f74afab3640 1 -- 172.21.15.19:0/1564387194 >> 172.21.15.19:0/1564387194 conn(0x7f74a80b2390 msgr2=0x7f74a80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.442+0000 7f74afab3640 1 -- 172.21.15.19:0/1564387194 shutdown_connections 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.442+0000 7f74afab3640 1 -- 172.21.15.19:0/1564387194 wait complete. 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.443+0000 7f74afab3640 1 Processor -- start 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.443+0000 7f74afab3640 1 -- start start 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.443+0000 7f74afab3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a80ccfe0 0x7f74a814a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.443+0000 7f74afab3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74a80cefb0 0x7f74a8147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.443+0000 7f74afab3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74a815de80 0x7f74a8147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.443+0000 7f74afab3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80cfb50 con 0x7f74a80cefb0 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.443+0000 7f74afab3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80b2eb0 con 0x7f74a80ccfe0 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.443+0000 7f74afab3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80b5550 con 0x7f74a815de80 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74ad04e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74a80cefb0 0x7f74a8147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74ad04e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74a80cefb0 0x7f74a8147380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46236/0 (socket says 172.21.15.19:46236) 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74ad04e640 1 -- 172.21.15.19:0/390153641 learned_addr learned my addr 172.21.15.19:0/390153641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74ad84f640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a80ccfe0 0x7f74a814a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74ae050640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74a815de80 0x7f74a8147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74ad04e640 1 -- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74a815de80 msgr2=0x7f74a8147bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74ad04e640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74a815de80 0x7f74a8147bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74ad04e640 1 -- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a80ccfe0 msgr2=0x7f74a814a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74ad04e640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a80ccfe0 0x7f74a814a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74ad04e640 1 -- 172.21.15.19:0/390153641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a814dea0 con 0x7f74a80cefb0 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74ad04e640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74a80cefb0 0x7f74a8147380 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c780 tx=0x7f749c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74a67fc640 1 -- 172.21.15.19:0/390153641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c017070 con 0x7f74a80cefb0 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74a67fc640 1 -- 172.21.15.19:0/390153641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c005ad0 con 0x7f74a80cefb0 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74afab3640 1 -- 172.21.15.19:0/390153641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a814ab90 con 0x7f74a80cefb0 2024-04-25T10:05:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74a67fc640 1 -- 172.21.15.19:0/390153641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c007070 con 0x7f74a80cefb0 2024-04-25T10:05:24.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.444+0000 7f74afab3640 1 -- 172.21.15.19:0/390153641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a80b4c60 con 0x7f74a80cefb0 2024-04-25T10:05:24.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.445+0000 7f74afab3640 1 -- 172.21.15.19:0/390153641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a8073a00 con 0x7f74a80cefb0 2024-04-25T10:05:24.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.445+0000 7f74a67fc640 1 -- 172.21.15.19:0/390153641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f749c01a070 con 0x7f74a80cefb0 2024-04-25T10:05:24.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.446+0000 7f74a67fc640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f749406b4d0 0x7f749406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.446+0000 7f74a67fc640 1 -- 172.21.15.19:0/390153641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f749c089980 con 0x7f74a80cefb0 2024-04-25T10:05:24.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.446+0000 7f74ad84f640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f749406b4d0 0x7f749406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.447+0000 7f74ad84f640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f749406b4d0 0x7f749406d990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f74a0034000 tx=0x7f74a002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.450+0000 7f74a67fc640 1 -- 172.21.15.19:0/390153641 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74a814ab90 con 0x7f74a80cefb0 2024-04-25T10:05:24.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.629+0000 7f74afab3640 1 -- 172.21.15.19:0/390153641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74a815a3b0 con 0x7f74a80cefb0 2024-04-25T10:05:24.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.629+0000 7f74a67fc640 1 -- 172.21.15.19:0/390153641 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v178) v1 ==== 100+0+11 (secure 0 0 0) 0x7f74a814dea0 con 0x7f74a80cefb0 2024-04-25T10:05:24.635 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:05:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.631+0000 7f74afab3640 1 -- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f749406b4d0 msgr2=0x7f749406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.631+0000 7f74afab3640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f749406b4d0 0x7f749406d990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f74a0034000 tx=0x7f74a002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.631+0000 7f74afab3640 1 -- 172.21.15.19:0/390153641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74a80cefb0 msgr2=0x7f74a8147380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.631+0000 7f74afab3640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74a80cefb0 0x7f74a8147380 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c780 tx=0x7f749c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:05:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.631+0000 7f74afab3640 1 -- 172.21.15.19:0/390153641 shutdown_connections 2024-04-25T10:05:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.631+0000 7f74afab3640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f749406b4d0 0x7f749406d990 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.631+0000 7f74afab3640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74a815de80 0x7f74a8147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.631+0000 7f74afab3640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74a80cefb0 0x7f74a8147380 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.631+0000 7f74afab3640 1 --2- 172.21.15.19:0/390153641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a80ccfe0 0x7f74a814a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.631+0000 7f74afab3640 1 -- 172.21.15.19:0/390153641 >> 172.21.15.19:0/390153641 conn(0x7f74a80b2390 msgr2=0x7f74a80c6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.632+0000 7f74afab3640 1 -- 172.21.15.19:0/390153641 shutdown_connections 2024-04-25T10:05:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.632+0000 7f74afab3640 1 -- 172.21.15.19:0/390153641 wait complete. 2024-04-25T10:05:24.647 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.742+0000 7fbdf20ac640 1 Processor -- start 2024-04-25T10:05:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.743+0000 7fbdf20ac640 1 -- start start 2024-04-25T10:05:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.743+0000 7fbdf20ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdec15ba00 0x7fbdec09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.743+0000 7fbdf20ac640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbdec15f5f0 0x7fbdec15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.743+0000 7fbdf20ac640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdec0cd710 con 0x7fbdec1602e0 2024-04-25T10:05:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.743+0000 7fbdf20ac640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdec0cd4b0 con 0x7fbdec15f5f0 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.743+0000 7fbdf20ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec099b70 con 0x7fbdec15ba00 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.743+0000 7fbdebfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbdec15f5f0 0x7fbdec15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.743+0000 7fbdeaffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbdec1602e0 0x7fbdec0ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47068/0 (socket says 172.21.15.19:47068) 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.743+0000 7fbdeaffd640 1 -- 172.21.15.19:0/636113439 learned_addr learned my addr 172.21.15.19:0/636113439 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.744+0000 7fbdeb7fe640 1 --2- 172.21.15.19:0/636113439 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdec15ba00 0x7fbdec09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.744+0000 7fbdebfff640 1 -- 172.21.15.19:0/636113439 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdec15ba00 msgr2=0x7fbdec09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.744+0000 7fbdebfff640 1 --2- 172.21.15.19:0/636113439 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdec15ba00 0x7fbdec09a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.744+0000 7fbdebfff640 1 -- 172.21.15.19:0/636113439 >> v1:172.21.15.19:6789/0 conn(0x7fbdec1602e0 legacy=0x7fbdec0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.744+0000 7fbdebfff640 1 -- 172.21.15.19:0/636113439 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec099e50 con 0x7fbdec15f5f0 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.744+0000 7fbdea7fc640 1 -- 172.21.15.19:0/636113439 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2116237258 0 0) 0x7fbde0001110 con 0x7fbdec1602e0 2024-04-25T10:05:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdebfff640 1 --2- 172.21.15.19:0/636113439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdec15f5f0 0x7fbdec15faa0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009d90 tx=0x7fbde402a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f37a4d65f7bc84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdea7fc640 1 -- 172.21.15.19:0/636113439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4007740 con 0x7fbdec15f5f0 2024-04-25T10:05:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdea7fc640 1 -- 172.21.15.19:0/636113439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde40078e0 con 0x7fbdec15f5f0 2024-04-25T10:05:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdea7fc640 1 -- 172.21.15.19:0/636113439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4007cb0 con 0x7fbdec15f5f0 2024-04-25T10:05:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdf20ac640 1 -- 172.21.15.19:0/636113439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdec15f5f0 msgr2=0x7fbdec15faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdf20ac640 1 --2- 172.21.15.19:0/636113439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdec15f5f0 0x7fbdec15faa0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009d90 tx=0x7fbde402a530 comp rx=0 tx=0).stop 2024-04-25T10:05:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdf20ac640 1 -- 172.21.15.19:0/636113439 shutdown_connections 2024-04-25T10:05:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdf20ac640 1 --2- 172.21.15.19:0/636113439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdec15f5f0 0x7fbdec15faa0 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdf20ac640 1 --2- 172.21.15.19:0/636113439 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdec15ba00 0x7fbdec09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdf20ac640 1 -- 172.21.15.19:0/636113439 >> 172.21.15.19:0/636113439 conn(0x7fbdec0b3270 msgr2=0x7fbdec0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdf20ac640 1 -- 172.21.15.19:0/636113439 shutdown_connections 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.745+0000 7fbdf20ac640 1 -- 172.21.15.19:0/636113439 wait complete. 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.746+0000 7fbdf20ac640 1 Processor -- start 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.747+0000 7fbdf20ac640 1 -- start start 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.747+0000 7fbdf20ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdec15ba00 0x7fbdec158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.747+0000 7fbdf20ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdec15f5f0 0x7fbdec0b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.747+0000 7fbdf20ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdec1602e0 0x7fbdec0b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.747+0000 7fbdf20ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec15bde0 con 0x7fbdec1602e0 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.747+0000 7fbdf20ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec09ae70 con 0x7fbdec15f5f0 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.747+0000 7fbdf20ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec1591b0 con 0x7fbdec15ba00 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.747+0000 7fbdebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdec1602e0 0x7fbdec0b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.747+0000 7fbdebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdec1602e0 0x7fbdec0b60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46246/0 (socket says 172.21.15.19:46246) 2024-04-25T10:05:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.747+0000 7fbdebfff640 1 -- 172.21.15.19:0/965747248 learned_addr learned my addr 172.21.15.19:0/965747248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdeb7fe640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdec15ba00 0x7fbdec158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdeaffd640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdec15f5f0 0x7fbdec0b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdebfff640 1 -- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdec15ba00 msgr2=0x7fbdec158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdebfff640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdec15ba00 0x7fbdec158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdebfff640 1 -- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdec15f5f0 msgr2=0x7fbdec0b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdebfff640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdec15f5f0 0x7fbdec0b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdebfff640 1 -- 172.21.15.19:0/965747248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec0b0b80 con 0x7fbdec1602e0 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdebfff640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdec1602e0 0x7fbdec0b60f0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fbde402ab20 tx=0x7fbde4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdd3fff640 1 -- 172.21.15.19:0/965747248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4040070 con 0x7fbdec1602e0 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdd3fff640 1 -- 172.21.15.19:0/965747248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde4031bc0 con 0x7fbdec1602e0 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdf20ac640 1 -- 172.21.15.19:0/965747248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdec0b0e00 con 0x7fbdec1602e0 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdd3fff640 1 -- 172.21.15.19:0/965747248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4045460 con 0x7fbdec1602e0 2024-04-25T10:05:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.748+0000 7fbdf20ac640 1 -- 172.21.15.19:0/965747248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdec0b13c0 con 0x7fbdec1602e0 2024-04-25T10:05:24.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.749+0000 7fbdf20ac640 1 -- 172.21.15.19:0/965747248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdec073990 con 0x7fbdec1602e0 2024-04-25T10:05:24.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.750+0000 7fbdd3fff640 1 -- 172.21.15.19:0/965747248 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbde4039040 con 0x7fbdec1602e0 2024-04-25T10:05:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.750+0000 7fbdd3fff640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbdd806b4d0 0x7fbdd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.750+0000 7fbdd3fff640 1 -- 172.21.15.19:0/965747248 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbde40abcb0 con 0x7fbdec1602e0 2024-04-25T10:05:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.750+0000 7fbdeb7fe640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbdd806b4d0 0x7fbdd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.751+0000 7fbdeb7fe640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbdd806b4d0 0x7fbdd806d990 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fbddc006fd0 tx=0x7fbddc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.754+0000 7fbdd3fff640 1 -- 172.21.15.19:0/965747248 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbdec0b0b80 con 0x7fbdec1602e0 2024-04-25T10:05:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.979+0000 7fbdf20ac640 1 -- 172.21.15.19:0/965747248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdec0703e0 con 0x7fbdec1602e0 2024-04-25T10:05:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.979+0000 7fbdd3fff640 1 -- 172.21.15.19:0/965747248 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4948 (secure 0 0 0) 0x7fbde407a920 con 0x7fbdec1602e0 2024-04-25T10:05:24.985 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:24.985 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":116,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/662861975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":662861975},{"type":"v1","addr":"172.21.15.19:6835","nonce":662861975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3439611393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3439611393},{"type":"v1","addr":"172.21.15.112:6836","nonce":3439611393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721666508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721666508},{"type":"v1","addr":"172.21.15.112:6837","nonce":721666508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:13.300379+0000","modified":"2024-04-25T10:05:15.294971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11166":{"gid":11166,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6833/829980210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":829980210},{"type":"v1","addr":"172.21.15.19:6833","nonce":829980210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:05:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 116 2024-04-25T10:05:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.981+0000 7fbdf20ac640 1 -- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbdd806b4d0 msgr2=0x7fbdd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.981+0000 7fbdf20ac640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbdd806b4d0 0x7fbdd806d990 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fbddc006fd0 tx=0x7fbddc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.981+0000 7fbdf20ac640 1 -- 172.21.15.19:0/965747248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdec1602e0 msgr2=0x7fbdec0b60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.981+0000 7fbdf20ac640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdec1602e0 0x7fbdec0b60f0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fbde402ab20 tx=0x7fbde4004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.981+0000 7fbdf20ac640 1 -- 172.21.15.19:0/965747248 shutdown_connections 2024-04-25T10:05:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.981+0000 7fbdf20ac640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fbdd806b4d0 0x7fbdd806d990 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.981+0000 7fbdf20ac640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdec1602e0 0x7fbdec0b60f0 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.981+0000 7fbdf20ac640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdec15f5f0 0x7fbdec0b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.981+0000 7fbdf20ac640 1 --2- 172.21.15.19:0/965747248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdec15ba00 0x7fbdec158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.981+0000 7fbdf20ac640 1 -- 172.21.15.19:0/965747248 >> 172.21.15.19:0/965747248 conn(0x7fbdec0b3270 msgr2=0x7fbdec15e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.982+0000 7fbdf20ac640 1 -- 172.21.15.19:0/965747248 shutdown_connections 2024-04-25T10:05:24.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:24.982+0000 7fbdf20ac640 1 -- 172.21.15.19:0/965747248 wait complete. 2024-04-25T10:05:24.998 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.094+0000 7f424faac640 1 Processor -- start 2024-04-25T10:05:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.094+0000 7f424faac640 1 -- start start 2024-04-25T10:05:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424faac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42480b2cb0 0x7f42480d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424faac640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42480d1020 0x7f42480d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424faac640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42480b97a0 con 0x7f424815dbe0 2024-04-25T10:05:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424faac640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424815dfc0 con 0x7f42480d1020 2024-04-25T10:05:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424faac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42480d1a00 con 0x7f42480b2cb0 2024-04-25T10:05:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424e049640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42480d1020 0x7f42480d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424e049640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42480d1020 0x7f42480d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46252/0 (socket says 172.21.15.19:46252) 2024-04-25T10:05:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424e049640 1 -- 172.21.15.19:0/1440817809 learned_addr learned my addr 172.21.15.19:0/1440817809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424d047640 1 --2- 172.21.15.19:0/1440817809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42480b2cb0 0x7f42480d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424e049640 1 -- 172.21.15.19:0/1440817809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42480b2cb0 msgr2=0x7f42480d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424e049640 1 --2- 172.21.15.19:0/1440817809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42480b2cb0 0x7f42480d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424e049640 1 -- 172.21.15.19:0/1440817809 >> v1:172.21.15.112:6789/0 conn(0x7f424815dbe0 legacy=0x7f42480d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424e049640 1 -- 172.21.15.19:0/1440817809 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480d1c50 con 0x7f42480d1020 2024-04-25T10:05:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.095+0000 7f424d047640 1 --2- 172.21.15.19:0/1440817809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42480b2cb0 0x7f42480d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.096+0000 7f424e049640 1 --2- 172.21.15.19:0/1440817809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42480d1020 0x7f42480d2c70 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f4244009e50 tx=0x7f424402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a41a8b05062ca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.096+0000 7f424c846640 1 -- 172.21.15.19:0/1440817809 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244036070 con 0x7f42480d1020 2024-04-25T10:05:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.096+0000 7f424c846640 1 -- 172.21.15.19:0/1440817809 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4244031600 con 0x7f42480d1020 2024-04-25T10:05:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.096+0000 7f424c846640 1 -- 172.21.15.19:0/1440817809 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42440319d0 con 0x7f42480d1020 2024-04-25T10:05:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.096+0000 7f424faac640 1 -- 172.21.15.19:0/1440817809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42480d1020 msgr2=0x7f42480d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.096+0000 7f424faac640 1 --2- 172.21.15.19:0/1440817809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42480d1020 0x7f42480d2c70 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f4244009e50 tx=0x7f424402a6c0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.096+0000 7f424faac640 1 -- 172.21.15.19:0/1440817809 shutdown_connections 2024-04-25T10:05:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.096+0000 7f424faac640 1 --2- 172.21.15.19:0/1440817809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42480d1020 0x7f42480d2c70 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.096+0000 7f424faac640 1 --2- 172.21.15.19:0/1440817809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42480b2cb0 0x7f42480d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.096+0000 7f424faac640 1 -- 172.21.15.19:0/1440817809 >> 172.21.15.19:0/1440817809 conn(0x7f4248061a20 msgr2=0x7f4248061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.096+0000 7f424faac640 1 -- 172.21.15.19:0/1440817809 shutdown_connections 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.097+0000 7f424faac640 1 -- 172.21.15.19:0/1440817809 wait complete. 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.098+0000 7f424faac640 1 Processor -- start 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.098+0000 7f424faac640 1 -- start start 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.098+0000 7f424faac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42480b2cb0 0x7f42480b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.098+0000 7f424faac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42480d1020 0x7f42481541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.098+0000 7f424faac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f424815dbe0 0x7f4248155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.098+0000 7f424faac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480b92a0 con 0x7f42480b2cb0 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.098+0000 7f424faac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480d2210 con 0x7f424815dbe0 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.098+0000 7f424faac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42480b9420 con 0x7f42480d1020 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.098+0000 7f424d848640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42480b2cb0 0x7f42480b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424d848640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42480b2cb0 0x7f42480b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46258/0 (socket says 172.21.15.19:46258) 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424d848640 1 -- 172.21.15.19:0/558678422 learned_addr learned my addr 172.21.15.19:0/558678422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424e049640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f424815dbe0 0x7f4248155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424d047640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42480d1020 0x7f42481541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424d848640 1 -- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42480d1020 msgr2=0x7f42481541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424d848640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42480d1020 0x7f42481541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424d848640 1 -- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f424815dbe0 msgr2=0x7f4248155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424d848640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f424815dbe0 0x7f4248155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424d848640 1 -- 172.21.15.19:0/558678422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480b4620 con 0x7f42480b2cb0 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424e049640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f424815dbe0 0x7f4248155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424d848640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42480b2cb0 0x7f42480b40e0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f424000b3f0 tx=0x7f424000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f423e7fc640 1 -- 172.21.15.19:0/558678422 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240015020 con 0x7f42480b2cb0 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f423e7fc640 1 -- 172.21.15.19:0/558678422 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240011830 con 0x7f42480b2cb0 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424faac640 1 -- 172.21.15.19:0/558678422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42481546f0 con 0x7f42480b2cb0 2024-04-25T10:05:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f423e7fc640 1 -- 172.21.15.19:0/558678422 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240011c20 con 0x7f42480b2cb0 2024-04-25T10:05:25.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.099+0000 7f424faac640 1 -- 172.21.15.19:0/558678422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42480b4b80 con 0x7f42480b2cb0 2024-04-25T10:05:25.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.100+0000 7f424faac640 1 -- 172.21.15.19:0/558678422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4248073990 con 0x7f42480b2cb0 2024-04-25T10:05:25.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.101+0000 7f423e7fc640 1 -- 172.21.15.19:0/558678422 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4240018020 con 0x7f42480b2cb0 2024-04-25T10:05:25.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.101+0000 7f423e7fc640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f422c06b4d0 0x7f422c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.101+0000 7f423e7fc640 1 -- 172.21.15.19:0/558678422 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4240089f80 con 0x7f42480b2cb0 2024-04-25T10:05:25.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.101+0000 7f424d047640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f422c06b4d0 0x7f422c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.102+0000 7f424d047640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f422c06b4d0 0x7f422c06d990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4248160df0 tx=0x7f4238008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.105+0000 7f423e7fc640 1 -- 172.21.15.19:0/558678422 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f424001f030 con 0x7f42480b2cb0 2024-04-25T10:05:25.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.287+0000 7f424faac640 1 -- 172.21.15.19:0/558678422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42481590e0 con 0x7f42480b2cb0 2024-04-25T10:05:25.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.288+0000 7f423e7fc640 1 -- 172.21.15.19:0/558678422 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f42480b4620 con 0x7f42480b2cb0 2024-04-25T10:05:25.294 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:25.294 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":178,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:14.934502+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:11.578636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:12.274609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 -- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f422c06b4d0 msgr2=0x7f422c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f422c06b4d0 0x7f422c06d990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4248160df0 tx=0x7f4238008040 comp rx=0 tx=0).stop 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 -- 172.21.15.19:0/558678422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42480b2cb0 msgr2=0x7f42480b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42480b2cb0 0x7f42480b40e0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f424000b3f0 tx=0x7f424000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 -- 172.21.15.19:0/558678422 shutdown_connections 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f422c06b4d0 0x7f422c06d990 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f424815dbe0 0x7f4248155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42480d1020 0x7f42481541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 --2- 172.21.15.19:0/558678422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42480b2cb0 0x7f42480b40e0 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 -- 172.21.15.19:0/558678422 >> 172.21.15.19:0/558678422 conn(0x7f4248061a20 msgr2=0x7f424815b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 -- 172.21.15.19:0/558678422 shutdown_connections 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.290+0000 7f424faac640 1 -- 172.21.15.19:0/558678422 wait complete. 2024-04-25T10:05:25.307 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:05:25.307 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:25.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.402+0000 7f8031b7e640 1 Processor -- start 2024-04-25T10:05:25.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.402+0000 7f8031b7e640 1 -- start start 2024-04-25T10:05:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f8031b7e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f802c0b6cb0 0x7f802c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f8031b7e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f802c0b5a60 0x7f802c0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f8031b7e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c0ceee0 con 0x7f802c15db70 2024-04-25T10:05:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f8031b7e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c0d1d10 con 0x7f802c0b5a60 2024-04-25T10:05:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f8031b7e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c0d1ff0 con 0x7f802c0b6cb0 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f802bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f802c0b5a60 0x7f802c0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f802bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f802c0b5a60 0x7f802c0d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46266/0 (socket says 172.21.15.19:46266) 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f802bfff640 1 -- 172.21.15.19:0/3913031460 learned_addr learned my addr 172.21.15.19:0/3913031460 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f802affd640 1 --2- 172.21.15.19:0/3913031460 >> v2:172.21.15.112:3300/0 conn(0x7f802c0b6cb0 0x7f802c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f802bfff640 1 -- 172.21.15.19:0/3913031460 >> v2:172.21.15.112:3300/0 conn(0x7f802c0b6cb0 msgr2=0x7f802c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f802bfff640 1 --2- 172.21.15.19:0/3913031460 >> v2:172.21.15.112:3300/0 conn(0x7f802c0b6cb0 0x7f802c0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f802bfff640 1 -- 172.21.15.19:0/3913031460 >> v1:172.21.15.112:6789/0 conn(0x7f802c15db70 legacy=0x7f802c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.403+0000 7f802bfff640 1 -- 172.21.15.19:0/3913031460 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0d22d0 con 0x7f802c0b5a60 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.404+0000 7f802bfff640 1 --2- 172.21.15.19:0/3913031460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0b5a60 0x7f802c0d27f0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f802c0b6aa0 tx=0x7f802402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dec551c56014f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.404+0000 7f802a7fc640 1 -- 172.21.15.19:0/3913031460 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024036070 con 0x7f802c0b5a60 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.404+0000 7f802a7fc640 1 -- 172.21.15.19:0/3913031460 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8024031600 con 0x7f802c0b5a60 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.404+0000 7f802a7fc640 1 -- 172.21.15.19:0/3913031460 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80240319d0 con 0x7f802c0b5a60 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.405+0000 7f8031b7e640 1 -- 172.21.15.19:0/3913031460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0b5a60 msgr2=0x7f802c0d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.405+0000 7f8031b7e640 1 --2- 172.21.15.19:0/3913031460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0b5a60 0x7f802c0d27f0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f802c0b6aa0 tx=0x7f802402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.405+0000 7f8031b7e640 1 -- 172.21.15.19:0/3913031460 shutdown_connections 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.405+0000 7f8031b7e640 1 --2- 172.21.15.19:0/3913031460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0b5a60 0x7f802c0d27f0 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.405+0000 7f8031b7e640 1 --2- 172.21.15.19:0/3913031460 >> v2:172.21.15.112:3300/0 conn(0x7f802c0b6cb0 0x7f802c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.405+0000 7f8031b7e640 1 -- 172.21.15.19:0/3913031460 >> 172.21.15.19:0/3913031460 conn(0x7f802c0d0520 msgr2=0x7f802c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.405+0000 7f8031b7e640 1 -- 172.21.15.19:0/3913031460 shutdown_connections 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.405+0000 7f8031b7e640 1 -- 172.21.15.19:0/3913031460 wait complete. 2024-04-25T10:05:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.406+0000 7f8031b7e640 1 Processor -- start 2024-04-25T10:05:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.406+0000 7f8031b7e640 1 -- start start 2024-04-25T10:05:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f8031b7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0b5a60 0x7f802c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f8031b7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f802c0b6cb0 0x7f802c0aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f8031b7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15db70 0x7f802c0aa890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f8031b7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0cf1c0 con 0x7f802c0b5a60 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f8031b7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0ce530 con 0x7f802c15db70 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f8031b7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0cf340 con 0x7f802c0b6cb0 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f802b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0b5a60 0x7f802c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f802b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0b5a60 0x7f802c157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46276/0 (socket says 172.21.15.19:46276) 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f802b7fe640 1 -- 172.21.15.19:0/2408910773 learned_addr learned my addr 172.21.15.19:0/2408910773 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f802affd640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f802c0b6cb0 0x7f802c0aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f802bfff640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15db70 0x7f802c0aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f802b7fe640 1 -- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f802c0b6cb0 msgr2=0x7f802c0aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f802b7fe640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f802c0b6cb0 0x7f802c0aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f802b7fe640 1 -- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15db70 msgr2=0x7f802c0aa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.407+0000 7f802b7fe640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15db70 0x7f802c0aa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.408+0000 7f802b7fe640 1 -- 172.21.15.19:0/2408910773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0cc8d0 con 0x7f802c0b5a60 2024-04-25T10:05:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.408+0000 7f802b7fe640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0b5a60 0x7f802c157050 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f801c00c3f0 tx=0x7f801c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.408+0000 7f8013fff640 1 -- 172.21.15.19:0/2408910773 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c0052d0 con 0x7f802c0b5a60 2024-04-25T10:05:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.408+0000 7f8013fff640 1 -- 172.21.15.19:0/2408910773 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c005490 con 0x7f802c0b5a60 2024-04-25T10:05:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.408+0000 7f8031b7e640 1 -- 172.21.15.19:0/2408910773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c0ccbb0 con 0x7f802c0b5a60 2024-04-25T10:05:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.408+0000 7f8013fff640 1 -- 172.21.15.19:0/2408910773 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c007260 con 0x7f802c0b5a60 2024-04-25T10:05:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.408+0000 7f8031b7e640 1 -- 172.21.15.19:0/2408910773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c0cd170 con 0x7f802c0b5a60 2024-04-25T10:05:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.409+0000 7f8031b7e640 1 -- 172.21.15.19:0/2408910773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c073990 con 0x7f802c0b5a60 2024-04-25T10:05:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.409+0000 7f8013fff640 1 -- 172.21.15.19:0/2408910773 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f801c007400 con 0x7f802c0b5a60 2024-04-25T10:05:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.410+0000 7f8013fff640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f801806b4d0 0x7f801806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.410+0000 7f8013fff640 1 -- 172.21.15.19:0/2408910773 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f801c0897c0 con 0x7f802c0b5a60 2024-04-25T10:05:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.410+0000 7f802affd640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f801806b4d0 0x7f801806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.410+0000 7f802affd640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f801806b4d0 0x7f801806d990 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f802c15b680 tx=0x7f8020008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.414+0000 7f8013fff640 1 -- 172.21.15.19:0/2408910773 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f801c057e60 con 0x7f802c0b5a60 2024-04-25T10:05:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.636+0000 7f8031b7e640 1 -- 172.21.15.19:0/2408910773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f802c0b83d0 con 0x7f802c0b5a60 2024-04-25T10:05:25.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.637+0000 7f8013fff640 1 -- 172.21.15.19:0/2408910773 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4948 (secure 0 0 0) 0x7f801c05ce20 con 0x7f802c0b5a60 2024-04-25T10:05:25.642 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:25.642 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":116,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/662861975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":662861975},{"type":"v1","addr":"172.21.15.19:6835","nonce":662861975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3439611393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3439611393},{"type":"v1","addr":"172.21.15.112:6836","nonce":3439611393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721666508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721666508},{"type":"v1","addr":"172.21.15.112:6837","nonce":721666508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:13.300379+0000","modified":"2024-04-25T10:05:15.294971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11166":{"gid":11166,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6833/829980210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":829980210},{"type":"v1","addr":"172.21.15.19:6833","nonce":829980210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:05:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 116 2024-04-25T10:05:25.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.638+0000 7f8031b7e640 1 -- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f801806b4d0 msgr2=0x7f801806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.638+0000 7f8031b7e640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f801806b4d0 0x7f801806d990 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f802c15b680 tx=0x7f8020008040 comp rx=0 tx=0).stop 2024-04-25T10:05:25.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.638+0000 7f8031b7e640 1 -- 172.21.15.19:0/2408910773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0b5a60 msgr2=0x7f802c157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.638+0000 7f8031b7e640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0b5a60 0x7f802c157050 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f801c00c3f0 tx=0x7f801c00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.638+0000 7f8031b7e640 1 -- 172.21.15.19:0/2408910773 shutdown_connections 2024-04-25T10:05:25.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.639+0000 7f8031b7e640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f801806b4d0 0x7f801806d990 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.639+0000 7f8031b7e640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c15db70 0x7f802c0aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.639+0000 7f8031b7e640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f802c0b6cb0 0x7f802c0aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.639+0000 7f8031b7e640 1 --2- 172.21.15.19:0/2408910773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0b5a60 0x7f802c157050 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.639+0000 7f8031b7e640 1 -- 172.21.15.19:0/2408910773 >> 172.21.15.19:0/2408910773 conn(0x7f802c0d0520 msgr2=0x7f802c15c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.639+0000 7f8031b7e640 1 -- 172.21.15.19:0/2408910773 shutdown_connections 2024-04-25T10:05:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.639+0000 7f8031b7e640 1 -- 172.21.15.19:0/2408910773 wait complete. 2024-04-25T10:05:25.655 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.750+0000 7f92c7316640 1 Processor -- start 2024-04-25T10:05:25.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.750+0000 7f92c7316640 1 -- start start 2024-04-25T10:05:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c7316640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92c015da70 0x7f92c0161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c7316640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f92c00b9730 0x7f92c00ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c7316640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c0161b80 0x7f92c00ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c7316640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c01625e0 con 0x7f92c015da70 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c7316640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c0162020 con 0x7f92c00b9730 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c7316640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c0162300 con 0x7f92c0161b80 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c50b2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92c015da70 0x7f92c0161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c50b2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92c015da70 0x7f92c0161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46292/0 (socket says 172.21.15.19:46292) 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c50b2640 1 -- 172.21.15.19:0/3351131772 learned_addr learned my addr 172.21.15.19:0/3351131772 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c58b3640 1 --2- 172.21.15.19:0/3351131772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c0161b80 0x7f92c00ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c48b1640 1 --2- 172.21.15.19:0/3351131772 >> v2:172.21.15.112:3300/0 conn(0x7f92c00b9730 0x7f92c00ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c50b2640 1 -- 172.21.15.19:0/3351131772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c0161b80 msgr2=0x7f92c00ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c50b2640 1 --2- 172.21.15.19:0/3351131772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c0161b80 0x7f92c00ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c50b2640 1 -- 172.21.15.19:0/3351131772 >> v2:172.21.15.112:3300/0 conn(0x7f92c00b9730 msgr2=0x7f92c00ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.751+0000 7f92c50b2640 1 --2- 172.21.15.19:0/3351131772 >> v2:172.21.15.112:3300/0 conn(0x7f92c00b9730 0x7f92c00ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.752+0000 7f92c50b2640 1 -- 172.21.15.19:0/3351131772 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c00d1af0 con 0x7f92c015da70 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.752+0000 7f92c48b1640 1 --2- 172.21.15.19:0/3351131772 >> v2:172.21.15.112:3300/0 conn(0x7f92c00b9730 0x7f92c00ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.752+0000 7f92c50b2640 1 --2- 172.21.15.19:0/3351131772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c015da70 0x7f92c0161640 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f92c00c72a0 tx=0x7f92bc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939b2b3911fb0c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.752+0000 7f92b7fff640 1 -- 172.21.15.19:0/3351131772 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc037070 con 0x7f92c015da70 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.752+0000 7f92b7fff640 1 -- 172.21.15.19:0/3351131772 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc032440 con 0x7f92c015da70 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.752+0000 7f92b7fff640 1 -- 172.21.15.19:0/3351131772 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc032810 con 0x7f92c015da70 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.753+0000 7f92c7316640 1 -- 172.21.15.19:0/3351131772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c015da70 msgr2=0x7f92c0161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.753+0000 7f92c7316640 1 --2- 172.21.15.19:0/3351131772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c015da70 0x7f92c0161640 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f92c00c72a0 tx=0x7f92bc02a140 comp rx=0 tx=0).stop 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.753+0000 7f92c7316640 1 -- 172.21.15.19:0/3351131772 shutdown_connections 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.753+0000 7f92c7316640 1 --2- 172.21.15.19:0/3351131772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c0161b80 0x7f92c00ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.753+0000 7f92c7316640 1 --2- 172.21.15.19:0/3351131772 >> v2:172.21.15.112:3300/0 conn(0x7f92c00b9730 0x7f92c00ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.753+0000 7f92c7316640 1 --2- 172.21.15.19:0/3351131772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c015da70 0x7f92c0161640 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.753+0000 7f92c7316640 1 -- 172.21.15.19:0/3351131772 >> 172.21.15.19:0/3351131772 conn(0x7f92c0061a20 msgr2=0x7f92c0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.753+0000 7f92c7316640 1 -- 172.21.15.19:0/3351131772 shutdown_connections 2024-04-25T10:05:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.753+0000 7f92c7316640 1 -- 172.21.15.19:0/3351131772 wait complete. 2024-04-25T10:05:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.754+0000 7f92c7316640 1 Processor -- start 2024-04-25T10:05:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.754+0000 7f92c7316640 1 -- start start 2024-04-25T10:05:25.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c7316640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c015da70 0x7f92c014ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c7316640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c0161b80 0x7f92c015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c7316640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c015ea50 0x7f92c015eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c7316640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c006c120 con 0x7f92c0161b80 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c7316640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c015de50 con 0x7f92c015da70 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c7316640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c006c2a0 con 0x7f92c015ea50 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c48b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c0161b80 0x7f92c015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c48b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c0161b80 0x7f92c015aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46306/0 (socket says 172.21.15.19:46306) 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c48b1640 1 -- 172.21.15.19:0/2218901787 learned_addr learned my addr 172.21.15.19:0/2218901787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c50b2640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c015da70 0x7f92c014ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c58b3640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c015ea50 0x7f92c015eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c48b1640 1 -- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c015ea50 msgr2=0x7f92c015eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c48b1640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c015ea50 0x7f92c015eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c48b1640 1 -- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c015da70 msgr2=0x7f92c014ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c48b1640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c015da70 0x7f92c014ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c48b1640 1 -- 172.21.15.19:0/2218901787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c014b3d0 con 0x7f92c0161b80 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c58b3640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c015ea50 0x7f92c015eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.755+0000 7f92c50b2640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c015da70 0x7f92c014ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.756+0000 7f92c48b1640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c0161b80 0x7f92c015aa50 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c7f0 tx=0x7f92b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.756+0000 7f92b5ffb640 1 -- 172.21.15.19:0/2218901787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0015070 con 0x7f92c0161b80 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.756+0000 7f92b5ffb640 1 -- 172.21.15.19:0/2218901787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b0007a30 con 0x7f92c0161b80 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.756+0000 7f92b5ffb640 1 -- 172.21.15.19:0/2218901787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0007e00 con 0x7f92c0161b80 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.756+0000 7f92c7316640 1 -- 172.21.15.19:0/2218901787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c015af90 con 0x7f92c0161b80 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.756+0000 7f92c7316640 1 -- 172.21.15.19:0/2218901787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c0159c90 con 0x7f92c0161b80 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.757+0000 7f92c7316640 1 -- 172.21.15.19:0/2218901787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c0073990 con 0x7f92c0161b80 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.757+0000 7f92b5ffb640 1 -- 172.21.15.19:0/2218901787 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92b0018070 con 0x7f92c0161b80 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.758+0000 7f92b5ffb640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92a406b4d0 0x7f92a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.758+0000 7f92b5ffb640 1 -- 172.21.15.19:0/2218901787 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f92b008a210 con 0x7f92c0161b80 2024-04-25T10:05:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.758+0000 7f92c50b2640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92a406b4d0 0x7f92a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.758+0000 7f92c50b2640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92a406b4d0 0x7f92a406d990 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f92bc034000 tx=0x7f92bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.762+0000 7f92b5ffb640 1 -- 172.21.15.19:0/2218901787 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92b0058f90 con 0x7f92c0161b80 2024-04-25T10:05:25.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.977+0000 7f92c7316640 1 -- 172.21.15.19:0/2218901787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92c00d1570 con 0x7f92c0161b80 2024-04-25T10:05:25.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.978+0000 7f92b5ffb640 1 -- 172.21.15.19:0/2218901787 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4948 (secure 0 0 0) 0x7f92b0058890 con 0x7f92c0161b80 2024-04-25T10:05:25.983 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:25.983 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":116,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/662861975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":662861975},{"type":"v1","addr":"172.21.15.19:6835","nonce":662861975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3439611393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3439611393},{"type":"v1","addr":"172.21.15.112:6836","nonce":3439611393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721666508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721666508},{"type":"v1","addr":"172.21.15.112:6837","nonce":721666508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:13.300379+0000","modified":"2024-04-25T10:05:15.294971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11166},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11166":{"gid":11166,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6833/829980210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":829980210},{"type":"v1","addr":"172.21.15.19:6833","nonce":829980210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:05:25.984 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 116 2024-04-25T10:05:25.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 -- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92a406b4d0 msgr2=0x7f92a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92a406b4d0 0x7f92a406d990 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f92bc034000 tx=0x7f92bc02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:25.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 -- 172.21.15.19:0/2218901787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c0161b80 msgr2=0x7f92c015aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c0161b80 0x7f92c015aa50 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c7f0 tx=0x7f92b000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 -- 172.21.15.19:0/2218901787 shutdown_connections 2024-04-25T10:05:25.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f92a406b4d0 0x7f92a406d990 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f92c015ea50 0x7f92c015eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92c0161b80 0x7f92c015aa50 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 --2- 172.21.15.19:0/2218901787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c015da70 0x7f92c014ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 -- 172.21.15.19:0/2218901787 >> 172.21.15.19:0/2218901787 conn(0x7f92c0061a20 msgr2=0x7f92c006c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 -- 172.21.15.19:0/2218901787 shutdown_connections 2024-04-25T10:05:25.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:25.980+0000 7f92c7316640 1 -- 172.21.15.19:0/2218901787 wait complete. 2024-04-25T10:05:25.996 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.092+0000 7f9bcb09a640 1 Processor -- start 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.092+0000 7f9bcb09a640 1 -- start start 2024-04-25T10:05:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.093+0000 7f9bcb09a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bc4144750 0x7f9bc406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.093+0000 7f9bcb09a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc4144b30 con 0x7f9bc40cf030 2024-04-25T10:05:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.093+0000 7f9bcb09a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc40712d0 con 0x7f9bc406eb10 2024-04-25T10:05:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.093+0000 7f9bcb09a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40715b0 con 0x7f9bc4144750 2024-04-25T10:05:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.093+0000 7f9bc9637640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9bc40cf030 0x7f9bc406d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47080/0 (socket says 172.21.15.19:47080) 2024-04-25T10:05:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.093+0000 7f9bc9637640 1 -- 172.21.15.19:0/1689887109 learned_addr learned my addr 172.21.15.19:0/1689887109 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.093+0000 7f9bc8e36640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bc4144750 0x7f9bc406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.093+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779368453 0 0) 0x7f9bc4144b30 con 0x7f9bc40cf030 2024-04-25T10:05:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.093+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb4003520 con 0x7f9bc40cf030 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3135838146 0 0) 0x7f9bb4003520 con 0x7f9bc40cf030 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bc4144750 msgr2=0x7f9bc406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bc4144750 0x7f9bc406e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 >> v1:172.21.15.112:6789/0 conn(0x7f9bc406eb10 legacy=0x7f9bc40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc4071890 con 0x7f9bc40cf030 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f9bbc0019f0 con 0x7f9bc40cf030 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 >> v1:172.21.15.19:6789/0 conn(0x7f9bc40cf030 legacy=0x7f9bc406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bc4144750 0x7f9bb40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bb4003f20 0x7f9bb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bb4004d40 0x7f9bb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40715b0 con 0x7f9bb4004d40 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb400bad0 con 0x7f9bb4003f20 2024-04-25T10:05:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb400bcd0 con 0x7f9bc4144750 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc9637640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bb4004d40 0x7f9bb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc3fff640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bb4003f20 0x7f9bb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc8e36640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bc4144750 0x7f9bb40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc9637640 1 -- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bc4144750 msgr2=0x7f9bb40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc9637640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bc4144750 0x7f9bb40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc9637640 1 -- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bb4003f20 msgr2=0x7f9bb4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc9637640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bb4003f20 0x7f9bb4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc9637640 1 -- 172.21.15.19:0/1689887109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb4003520 con 0x7f9bb4004d40 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc8e36640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bc4144750 0x7f9bb40039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.094+0000 7f9bc3fff640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bb4003f20 0x7f9bb4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.095+0000 7f9bc9637640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bb4004d40 0x7f9bb40051c0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00a880 tx=0x7f9bb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e52e2aa8e27e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.095+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bb4004d40 2024-04-25T10:05:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.095+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc02f070 con 0x7f9bb4004d40 2024-04-25T10:05:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.095+0000 7f9bc37fe640 1 -- 172.21.15.19:0/1689887109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc02bd20 con 0x7f9bb4004d40 2024-04-25T10:05:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.095+0000 7f9bcb09a640 1 -- 172.21.15.19:0/1689887109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bb4004d40 msgr2=0x7f9bb40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.095+0000 7f9bcb09a640 1 --2- 172.21.15.19:0/1689887109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bb4004d40 0x7f9bb40051c0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00a880 tx=0x7f9bb4005830 comp rx=0 tx=0).stop 2024-04-25T10:05:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.095+0000 7f9bc9637640 1 -- 172.21.15.19:0/1689887109 reap_dead start 2024-04-25T10:05:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.095+0000 7f9bcb09a640 1 -- 172.21.15.19:0/1689887109 shutdown_connections 2024-04-25T10:05:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.096+0000 7f9bcb09a640 1 -- 172.21.15.19:0/1689887109 >> 172.21.15.19:0/1689887109 conn(0x7f9bc406abe0 msgr2=0x7f9bc406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.096+0000 7f9bcb09a640 1 -- 172.21.15.19:0/1689887109 shutdown_connections 2024-04-25T10:05:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.096+0000 7f9bcb09a640 1 -- 172.21.15.19:0/1689887109 wait complete. 2024-04-25T10:05:26.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.097+0000 7f9bcb09a640 1 Processor -- start 2024-04-25T10:05:26.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.097+0000 7f9bcb09a640 1 -- start start 2024-04-25T10:05:26.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bcb09a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bb4004d40 0x7f9bc4141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bcb09a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc4141bd0 0x7f9bc40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bcb09a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc415e400 0x7f9bc40acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bcb09a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc41455d0 con 0x7f9bc415e400 2024-04-25T10:05:26.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bcb09a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc4145450 con 0x7f9bc4141bd0 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bcb09a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc415c000 con 0x7f9bb4004d40 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc9637640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc415e400 0x7f9bc40acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc9637640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc415e400 0x7f9bc40acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46322/0 (socket says 172.21.15.19:46322) 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc9637640 1 -- 172.21.15.19:0/4060255344 learned_addr learned my addr 172.21.15.19:0/4060255344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc8e36640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bb4004d40 0x7f9bc4141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc9637640 1 -- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bb4004d40 msgr2=0x7f9bc4141690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc3fff640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc4141bd0 0x7f9bc40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc9637640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bb4004d40 0x7f9bc4141690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc9637640 1 -- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc4141bd0 msgr2=0x7f9bc40ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc9637640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc4141bd0 0x7f9bc40ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc9637640 1 -- 172.21.15.19:0/4060255344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc4142090 con 0x7f9bc415e400 2024-04-25T10:05:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc8e36640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bb4004d40 0x7f9bc4141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc3fff640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc4141bd0 0x7f9bc40ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc9637640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc415e400 0x7f9bc40acdf0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc02b530 tx=0x7f9bc4071b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc17fa640 1 -- 172.21.15.19:0/4060255344 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bc415e400 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc17fa640 1 -- 172.21.15.19:0/4060255344 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc02bc90 con 0x7f9bc415e400 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bc17fa640 1 -- 172.21.15.19:0/4060255344 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc0346b0 con 0x7f9bc415e400 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.098+0000 7f9bcb09a640 1 -- 172.21.15.19:0/4060255344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc4142370 con 0x7f9bc415e400 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.099+0000 7f9bcb09a640 1 -- 172.21.15.19:0/4060255344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc4142930 con 0x7f9bc415e400 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.099+0000 7f9bcb09a640 1 -- 172.21.15.19:0/4060255344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc4073990 con 0x7f9bc415e400 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.100+0000 7f9bc17fa640 1 -- 172.21.15.19:0/4060255344 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bbc034850 con 0x7f9bc415e400 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.100+0000 7f9bc17fa640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b500 0x7f9ba806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.100+0000 7f9bc17fa640 1 -- 172.21.15.19:0/4060255344 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9bbc0aa4c0 con 0x7f9bc415e400 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.101+0000 7f9bc8e36640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b500 0x7f9ba806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.101+0000 7f9bc8e36640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b500 0x7f9ba806d9c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009a80 tx=0x7f9bb8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.105+0000 7f9bc17fa640 1 -- 172.21.15.19:0/4060255344 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9bbc0791b0 con 0x7f9bc415e400 2024-04-25T10:05:26.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.284+0000 7f9bcb09a640 1 -- 172.21.15.19:0/4060255344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bc406b090 con 0x7f9bc415e400 2024-04-25T10:05:26.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.285+0000 7f9bc17fa640 1 -- 172.21.15.19:0/4060255344 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f9bbc078ab0 con 0x7f9bc415e400 2024-04-25T10:05:26.290 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:26.291 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":178,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:14.934502+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:11.578636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:12.274609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 -- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b500 msgr2=0x7f9ba806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b500 0x7f9ba806d9c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009a80 tx=0x7f9bb8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 -- 172.21.15.19:0/4060255344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc415e400 msgr2=0x7f9bc40acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc415e400 0x7f9bc40acdf0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc02b530 tx=0x7f9bc4071b70 comp rx=0 tx=0).stop 2024-04-25T10:05:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 -- 172.21.15.19:0/4060255344 shutdown_connections 2024-04-25T10:05:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9ba806b500 0x7f9ba806d9c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc415e400 0x7f9bc40acdf0 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc4141bd0 0x7f9bc40ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 --2- 172.21.15.19:0/4060255344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9bb4004d40 0x7f9bc4141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 -- 172.21.15.19:0/4060255344 >> 172.21.15.19:0/4060255344 conn(0x7f9bc406abe0 msgr2=0x7f9bc40b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 -- 172.21.15.19:0/4060255344 shutdown_connections 2024-04-25T10:05:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.287+0000 7f9bcb09a640 1 -- 172.21.15.19:0/4060255344 wait complete. 2024-04-25T10:05:26.303 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:05:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.400+0000 7f2185c07640 1 Processor -- start 2024-04-25T10:05:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.400+0000 7f2185c07640 1 -- start start 2024-04-25T10:05:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.400+0000 7f2185c07640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21800ced80 0x7f21800d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.400+0000 7f2185c07640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218015e000 con 0x7f2180161710 2024-04-25T10:05:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.400+0000 7f2185c07640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800d1de0 con 0x7f218015dc20 2024-04-25T10:05:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.400+0000 7f2185c07640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2180161c60 con 0x7f21800ced80 2024-04-25T10:05:26.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.401+0000 7f217effd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21800ced80 0x7f21800d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.401+0000 7f217ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2180161710 0x7f21800d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47096/0 (socket says 172.21.15.19:47096) 2024-04-25T10:05:26.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.401+0000 7f217ffff640 1 -- 172.21.15.19:0/262982798 learned_addr learned my addr 172.21.15.19:0/262982798 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:26.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.401+0000 7f217effd640 1 -- 172.21.15.19:0/262982798 >> v1:172.21.15.112:6789/0 conn(0x7f218015dc20 legacy=0x7f21800ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.401+0000 7f217effd640 1 -- 172.21.15.19:0/262982798 >> v1:172.21.15.19:6789/0 conn(0x7f2180161710 legacy=0x7f21800d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.401+0000 7f217effd640 1 -- 172.21.15.19:0/262982798 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2180161f40 con 0x7f21800ced80 2024-04-25T10:05:26.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.401+0000 7f217e7fc640 1 -- 172.21.15.19:0/262982798 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593226502 0 0) 0x7f218015e000 con 0x7f2180161710 2024-04-25T10:05:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.402+0000 7f217effd640 1 --2- 172.21.15.19:0/262982798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21800ced80 0x7f21800d22e0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f21740097a0 tx=0x7f217402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c43185100d6d90e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.402+0000 7f217e7fc640 1 -- 172.21.15.19:0/262982798 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174037070 con 0x7f21800ced80 2024-04-25T10:05:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.402+0000 7f217e7fc640 1 -- 172.21.15.19:0/262982798 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2174032440 con 0x7f21800ced80 2024-04-25T10:05:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.402+0000 7f217e7fc640 1 -- 172.21.15.19:0/262982798 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174032810 con 0x7f21800ced80 2024-04-25T10:05:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.402+0000 7f2185c07640 1 -- 172.21.15.19:0/262982798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21800ced80 msgr2=0x7f21800d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.402+0000 7f2185c07640 1 --2- 172.21.15.19:0/262982798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21800ced80 0x7f21800d22e0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f21740097a0 tx=0x7f217402a230 comp rx=0 tx=0).stop 2024-04-25T10:05:26.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.402+0000 7f2185c07640 1 -- 172.21.15.19:0/262982798 shutdown_connections 2024-04-25T10:05:26.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.402+0000 7f2185c07640 1 --2- 172.21.15.19:0/262982798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21800ced80 0x7f21800d22e0 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.402+0000 7f2185c07640 1 -- 172.21.15.19:0/262982798 >> 172.21.15.19:0/262982798 conn(0x7f21800d0420 msgr2=0x7f21800d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.402+0000 7f2185c07640 1 -- 172.21.15.19:0/262982798 shutdown_connections 2024-04-25T10:05:26.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.402+0000 7f2185c07640 1 -- 172.21.15.19:0/262982798 wait complete. 2024-04-25T10:05:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.403+0000 7f2185c07640 1 Processor -- start 2024-04-25T10:05:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.404+0000 7f2185c07640 1 -- start start 2024-04-25T10:05:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.404+0000 7f2185c07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21800ced80 0x7f2180154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.404+0000 7f2185c07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f218015dc20 0x7f21800c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.404+0000 7f2185c07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2180161710 0x7f218015f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.404+0000 7f2185c07640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800b5e50 con 0x7f2180161710 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.404+0000 7f2185c07640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180162500 con 0x7f21800ced80 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.404+0000 7f2185c07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21800b5fd0 con 0x7f218015dc20 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.404+0000 7f217ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2180161710 0x7f218015f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f217ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2180161710 0x7f218015f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46346/0 (socket says 172.21.15.19:46346) 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f217ffff640 1 -- 172.21.15.19:0/1528682090 learned_addr learned my addr 172.21.15.19:0/1528682090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f217f7fe640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21800ced80 0x7f2180154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f217effd640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f218015dc20 0x7f21800c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f217ffff640 1 -- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f218015dc20 msgr2=0x7f21800c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f217ffff640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f218015dc20 0x7f21800c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f217ffff640 1 -- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21800ced80 msgr2=0x7f2180154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f217ffff640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21800ced80 0x7f2180154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f217ffff640 1 -- 172.21.15.19:0/1528682090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800c7c00 con 0x7f2180161710 2024-04-25T10:05:26.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f217ffff640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2180161710 0x7f218015f870 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f217800c800 tx=0x7f217800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f2167fff640 1 -- 172.21.15.19:0/1528682090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178018070 con 0x7f2180161710 2024-04-25T10:05:26.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f2167fff640 1 -- 172.21.15.19:0/1528682090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2178004d80 con 0x7f2180161710 2024-04-25T10:05:26.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f2167fff640 1 -- 172.21.15.19:0/1528682090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217800a880 con 0x7f2180161710 2024-04-25T10:05:26.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f2185c07640 1 -- 172.21.15.19:0/1528682090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21800c7ee0 con 0x7f2180161710 2024-04-25T10:05:26.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.405+0000 7f2185c07640 1 -- 172.21.15.19:0/1528682090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21800c8710 con 0x7f2180161710 2024-04-25T10:05:26.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.406+0000 7f2185c07640 1 -- 172.21.15.19:0/1528682090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2180073a00 con 0x7f2180161710 2024-04-25T10:05:26.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.407+0000 7f2167fff640 1 -- 172.21.15.19:0/1528682090 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f217800aa20 con 0x7f2180161710 2024-04-25T10:05:26.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.407+0000 7f2167fff640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f216c06b500 0x7f216c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.407+0000 7f2167fff640 1 -- 172.21.15.19:0/1528682090 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2178089830 con 0x7f2180161710 2024-04-25T10:05:26.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.407+0000 7f217f7fe640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f216c06b500 0x7f216c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.408+0000 7f217f7fe640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f216c06b500 0x7f216c06d9c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2170000f60 tx=0x7f2170009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.411+0000 7f2167fff640 1 -- 172.21.15.19:0/1528682090 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21780585f0 con 0x7f2180161710 2024-04-25T10:05:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.630+0000 7f2185c07640 1 -- 172.21.15.19:0/1528682090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f218015b590 con 0x7f2180161710 2024-04-25T10:05:26.635 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T10:05:26.631+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.632+0000 7f2167fff640 1 -- 172.21.15.19:0/1528682090 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v117) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2178057ef0 con 0x7f2180161710 2024-04-25T10:05:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:05:26.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 -- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f216c06b500 msgr2=0x7f216c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f216c06b500 0x7f216c06d9c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2170000f60 tx=0x7f2170009040 comp rx=0 tx=0).stop 2024-04-25T10:05:26.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 -- 172.21.15.19:0/1528682090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2180161710 msgr2=0x7f218015f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2180161710 0x7f218015f870 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f217800c800 tx=0x7f217800ccc0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 -- 172.21.15.19:0/1528682090 shutdown_connections 2024-04-25T10:05:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f216c06b500 0x7f216c06d9c0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2180161710 0x7f218015f870 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f218015dc20 0x7f21800c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 --2- 172.21.15.19:0/1528682090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21800ced80 0x7f2180154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 -- 172.21.15.19:0/1528682090 >> 172.21.15.19:0/1528682090 conn(0x7f21800d0420 msgr2=0x7f21800d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 -- 172.21.15.19:0/1528682090 shutdown_connections 2024-04-25T10:05:26.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.635+0000 7f2185c07640 1 -- 172.21.15.19:0/1528682090 wait complete. 2024-04-25T10:05:26.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:05:26.679 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:05:26.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.753+0000 7ff15a19e640 1 Processor -- start 2024-04-25T10:05:26.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.753+0000 7ff15a19e640 1 -- start start 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15a19e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff15415dd30 0x7ff1540cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15a19e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff1540b5b40 0x7ff1540d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15a19e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15415e110 con 0x7ff15406bff0 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15a19e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15406d000 con 0x7ff15415dd30 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15a19e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1540b2fd0 con 0x7ff1540b5b40 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15919c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff15415dd30 0x7ff1540cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15919c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff15415dd30 0x7ff1540cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46366/0 (socket says 172.21.15.19:46366) 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15919c640 1 -- 172.21.15.19:0/1568351833 learned_addr learned my addr 172.21.15.19:0/1568351833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15999d640 1 --2- 172.21.15.19:0/1568351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff1540b5b40 0x7ff1540d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15919c640 1 -- 172.21.15.19:0/1568351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff1540b5b40 msgr2=0x7ff1540d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15919c640 1 --2- 172.21.15.19:0/1568351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff1540b5b40 0x7ff1540d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15919c640 1 -- 172.21.15.19:0/1568351833 >> v1:172.21.15.112:6789/0 conn(0x7ff15406bff0 legacy=0x7ff1540b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15919c640 1 -- 172.21.15.19:0/1568351833 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15406c480 con 0x7ff15415dd30 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.754+0000 7ff15999d640 1 --2- 172.21.15.19:0/1568351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff1540b5b40 0x7ff1540d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.755+0000 7ff15919c640 1 --2- 172.21.15.19:0/1568351833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff15415dd30 0x7ff1540cebd0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009870 tx=0x7ff14c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa653636ac03a3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.755+0000 7ff153fff640 1 -- 172.21.15.19:0/1568351833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c037070 con 0x7ff15415dd30 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.755+0000 7ff153fff640 1 -- 172.21.15.19:0/1568351833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c032440 con 0x7ff15415dd30 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.755+0000 7ff153fff640 1 -- 172.21.15.19:0/1568351833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c032810 con 0x7ff15415dd30 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.755+0000 7ff15a19e640 1 -- 172.21.15.19:0/1568351833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff15415dd30 msgr2=0x7ff1540cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.755+0000 7ff15a19e640 1 --2- 172.21.15.19:0/1568351833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff15415dd30 0x7ff1540cebd0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009870 tx=0x7ff14c02a200 comp rx=0 tx=0).stop 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.755+0000 7ff15a19e640 1 -- 172.21.15.19:0/1568351833 shutdown_connections 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.755+0000 7ff15a19e640 1 --2- 172.21.15.19:0/1568351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff1540b5b40 0x7ff1540d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.755+0000 7ff15a19e640 1 --2- 172.21.15.19:0/1568351833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff15415dd30 0x7ff1540cebd0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.755+0000 7ff15a19e640 1 -- 172.21.15.19:0/1568351833 >> 172.21.15.19:0/1568351833 conn(0x7ff1540d0610 msgr2=0x7ff1540d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.756+0000 7ff15a19e640 1 -- 172.21.15.19:0/1568351833 shutdown_connections 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.756+0000 7ff15a19e640 1 -- 172.21.15.19:0/1568351833 wait complete. 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.757+0000 7ff15a19e640 1 Processor -- start 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.757+0000 7ff15a19e640 1 -- start start 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15a19e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff1540b5b40 0x7ff154160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15a19e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff15415dd30 0x7ff1541e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15a19e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15415b360 0x7ff1540cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15a19e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15406cd20 con 0x7ff15415dd30 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15a19e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15406ca40 con 0x7ff15415b360 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15a19e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1540cf110 con 0x7ff1540b5b40 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15899b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff15415dd30 0x7ff1541e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15899b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff15415dd30 0x7ff1541e6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46370/0 (socket says 172.21.15.19:46370) 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15899b640 1 -- 172.21.15.19:0/3292270092 learned_addr learned my addr 172.21.15.19:0/3292270092 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15999d640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15415b360 0x7ff1540cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15919c640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff1540b5b40 0x7ff154160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15899b640 1 -- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff1540b5b40 msgr2=0x7ff154160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15899b640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff1540b5b40 0x7ff154160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15899b640 1 -- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15415b360 msgr2=0x7ff1540cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15899b640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15415b360 0x7ff1540cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15899b640 1 -- 172.21.15.19:0/3292270092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff154160270 con 0x7ff15415dd30 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15999d640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15415b360 0x7ff1540cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15899b640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff15415dd30 0x7ff1541e6f80 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7ff14800c430 tx=0x7ff14800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff151ffb640 1 -- 172.21.15.19:0/3292270092 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148011790 con 0x7ff15415dd30 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff151ffb640 1 -- 172.21.15.19:0/3292270092 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff148017070 con 0x7ff15415dd30 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff15a19e640 1 -- 172.21.15.19:0/3292270092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15415f4b0 con 0x7ff15415dd30 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.758+0000 7ff151ffb640 1 -- 172.21.15.19:0/3292270092 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148011bb0 con 0x7ff15415dd30 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.759+0000 7ff15a19e640 1 -- 172.21.15.19:0/3292270092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15415fa70 con 0x7ff15415dd30 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.760+0000 7ff15a19e640 1 -- 172.21.15.19:0/3292270092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff154073990 con 0x7ff15415dd30 2024-04-25T10:05:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.760+0000 7ff151ffb640 1 -- 172.21.15.19:0/3292270092 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff14801c460 con 0x7ff15415dd30 2024-04-25T10:05:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.760+0000 7ff151ffb640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff14006b4d0 0x7ff14006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.760+0000 7ff151ffb640 1 -- 172.21.15.19:0/3292270092 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff14808a290 con 0x7ff15415dd30 2024-04-25T10:05:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.761+0000 7ff15919c640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff14006b4d0 0x7ff14006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.761+0000 7ff15919c640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff14006b4d0 0x7ff14006d990 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff14c034000 tx=0x7ff14c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.764+0000 7ff151ffb640 1 -- 172.21.15.19:0/3292270092 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff148058f40 con 0x7ff15415dd30 2024-04-25T10:05:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:26.982+0000 7ff15a19e640 1 -- 172.21.15.19:0/3292270092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff1540d16c0 con 0x7ff15415dd30 2024-04-25T10:05:27.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.634+0000 7ff151ffb640 1 -- 172.21.15.19:0/3292270092 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v118) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff148058840 con 0x7ff15415dd30 2024-04-25T10:05:27.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 -- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff14006b4d0 msgr2=0x7ff14006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff14006b4d0 0x7ff14006d990 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff14c034000 tx=0x7ff14c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:27.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 -- 172.21.15.19:0/3292270092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff15415dd30 msgr2=0x7ff1541e6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff15415dd30 0x7ff1541e6f80 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7ff14800c430 tx=0x7ff14800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 -- 172.21.15.19:0/3292270092 shutdown_connections 2024-04-25T10:05:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff14006b4d0 0x7ff14006d990 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15415b360 0x7ff1540cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff15415dd30 0x7ff1541e6f80 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 --2- 172.21.15.19:0/3292270092 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff1540b5b40 0x7ff154160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 -- 172.21.15.19:0/3292270092 >> 172.21.15.19:0/3292270092 conn(0x7ff1540d0610 msgr2=0x7ff1540d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 -- 172.21.15.19:0/3292270092 shutdown_connections 2024-04-25T10:05:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.638+0000 7ff15a19e640 1 -- 172.21.15.19:0/3292270092 wait complete. 2024-04-25T10:05:27.657 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:05:27.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.807+0000 7f8eb5031640 1 Processor -- start 2024-04-25T10:05:27.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.807+0000 7f8eb5031640 1 -- start start 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eb5031640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8eb015dc20 0x7f8eb00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eb5031640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8eb0161710 0x7f8eb00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eb5031640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8eb00b9330 0x7f8eb006bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eb5031640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb015e000 con 0x7f8eb00b9330 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eb5031640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb00d1de0 con 0x7f8eb0161710 2024-04-25T10:05:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eb5031640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0161c60 con 0x7f8eb015dc20 2024-04-25T10:05:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eaf59e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8eb00b9330 0x7f8eb006bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eaf59e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8eb00b9330 0x7f8eb006bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44798/0 (socket says 172.21.15.19:44798) 2024-04-25T10:05:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eaf59e640 1 -- 172.21.15.19:0/1969378293 learned_addr learned my addr 172.21.15.19:0/1969378293 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eae59c640 1 --2- 172.21.15.19:0/1969378293 >> v2:172.21.15.112:3300/0 conn(0x7f8eb0161710 0x7f8eb00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eaf59e640 1 -- 172.21.15.19:0/1969378293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8eb015dc20 msgr2=0x7f8eb00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eaf59e640 1 --2- 172.21.15.19:0/1969378293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8eb015dc20 0x7f8eb00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eaf59e640 1 -- 172.21.15.19:0/1969378293 >> v2:172.21.15.112:3300/0 conn(0x7f8eb0161710 msgr2=0x7f8eb00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eaf59e640 1 --2- 172.21.15.19:0/1969378293 >> v2:172.21.15.112:3300/0 conn(0x7f8eb0161710 0x7f8eb00cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.808+0000 7f8eaf59e640 1 -- 172.21.15.19:0/1969378293 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb0161f40 con 0x7f8eb00b9330 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.809+0000 7f8eaf59e640 1 --2- 172.21.15.19:0/1969378293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb00b9330 0x7f8eb006bf80 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f8eb006d9a0 tx=0x7f8ea802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d679aa22415514b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.809+0000 7f8eadd9b640 1 -- 172.21.15.19:0/1969378293 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8036070 con 0x7f8eb00b9330 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.809+0000 7f8eadd9b640 1 -- 172.21.15.19:0/1969378293 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea8031600 con 0x7f8eb00b9330 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.809+0000 7f8eadd9b640 1 -- 172.21.15.19:0/1969378293 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80319d0 con 0x7f8eb00b9330 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.810+0000 7f8eb5031640 1 -- 172.21.15.19:0/1969378293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb00b9330 msgr2=0x7f8eb006bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.810+0000 7f8eb5031640 1 --2- 172.21.15.19:0/1969378293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb00b9330 0x7f8eb006bf80 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f8eb006d9a0 tx=0x7f8ea802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.810+0000 7f8eb5031640 1 -- 172.21.15.19:0/1969378293 shutdown_connections 2024-04-25T10:05:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.810+0000 7f8eb5031640 1 --2- 172.21.15.19:0/1969378293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb00b9330 0x7f8eb006bf80 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.810+0000 7f8eb5031640 1 --2- 172.21.15.19:0/1969378293 >> v2:172.21.15.112:3300/0 conn(0x7f8eb0161710 0x7f8eb00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.810+0000 7f8eb5031640 1 --2- 172.21.15.19:0/1969378293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8eb015dc20 0x7f8eb00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.810+0000 7f8eb5031640 1 -- 172.21.15.19:0/1969378293 >> 172.21.15.19:0/1969378293 conn(0x7f8eb00d0420 msgr2=0x7f8eb00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.810+0000 7f8eb5031640 1 -- 172.21.15.19:0/1969378293 shutdown_connections 2024-04-25T10:05:27.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.810+0000 7f8eb5031640 1 -- 172.21.15.19:0/1969378293 wait complete. 2024-04-25T10:05:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.811+0000 7f8eb5031640 1 Processor -- start 2024-04-25T10:05:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.811+0000 7f8eb5031640 1 -- start start 2024-04-25T10:05:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eb5031640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8eb00b9330 0x7f8eb015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eb5031640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb015dc20 0x7f8eb00cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eb5031640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0161710 0x7f8eb01f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eb5031640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00b5e50 con 0x7f8eb0161710 2024-04-25T10:05:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eb5031640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0162500 con 0x7f8eb015dc20 2024-04-25T10:05:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eb5031640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00b5fd0 con 0x7f8eb00b9330 2024-04-25T10:05:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eaf59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0161710 0x7f8eb01f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eaf59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0161710 0x7f8eb01f3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44802/0 (socket says 172.21.15.19:44802) 2024-04-25T10:05:27.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eaf59e640 1 -- 172.21.15.19:0/3413629266 learned_addr learned my addr 172.21.15.19:0/3413629266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:27.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eae59c640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb015dc20 0x7f8eb00cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eaed9d640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8eb00b9330 0x7f8eb015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eaf59e640 1 -- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8eb00b9330 msgr2=0x7f8eb015bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eaf59e640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8eb00b9330 0x7f8eb015bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eaf59e640 1 -- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb015dc20 msgr2=0x7f8eb00cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eaf59e640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb015dc20 0x7f8eb00cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.812+0000 7f8eaf59e640 1 -- 172.21.15.19:0/3413629266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb00aa740 con 0x7f8eb0161710 2024-04-25T10:05:27.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.813+0000 7f8eae59c640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb015dc20 0x7f8eb00cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:27.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.813+0000 7f8eaf59e640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0161710 0x7f8eb01f3740 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009f40 tx=0x7f8ea8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.813+0000 7f8e977fe640 1 -- 172.21.15.19:0/3413629266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8041020 con 0x7f8eb0161710 2024-04-25T10:05:27.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.813+0000 7f8eb5031640 1 -- 172.21.15.19:0/3413629266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb00aaa20 con 0x7f8eb0161710 2024-04-25T10:05:27.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.813+0000 7f8e977fe640 1 -- 172.21.15.19:0/3413629266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea80077a0 con 0x7f8eb0161710 2024-04-25T10:05:27.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.813+0000 7f8e977fe640 1 -- 172.21.15.19:0/3413629266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8036040 con 0x7f8eb0161710 2024-04-25T10:05:27.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.813+0000 7f8eb5031640 1 -- 172.21.15.19:0/3413629266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb00ab2c0 con 0x7f8eb0161710 2024-04-25T10:05:27.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.814+0000 7f8eb5031640 1 -- 172.21.15.19:0/3413629266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb00aabf0 con 0x7f8eb0161710 2024-04-25T10:05:27.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.814+0000 7f8e977fe640 1 -- 172.21.15.19:0/3413629266 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ea8044020 con 0x7f8eb0161710 2024-04-25T10:05:27.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.815+0000 7f8e977fe640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.815+0000 7f8e977fe640 1 -- 172.21.15.19:0/3413629266 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8ea80aaff0 con 0x7f8eb0161710 2024-04-25T10:05:27.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.815+0000 7f8eaed9d640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.816+0000 7f8eaed9d640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000b810 tx=0x7f8ea0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.819+0000 7f8e977fe640 1 -- 172.21.15.19:0/3413629266 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ea804b030 con 0x7f8eb0161710 2024-04-25T10:05:28.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:27.998+0000 7f8eb5031640 1 -- 172.21.15.19:0/3413629266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8eb0073a00 con 0x7f8eb0161710 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.003+0000 7f8e977fe640 1 -- 172.21.15.19:0/3413629266 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v180) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8ea8079d60 con 0x7f8eb0161710 2024-04-25T10:05:29.009 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.007+0000 7f8eb5031640 1 -- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8e9c06b4d0 msgr2=0x7f8e9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.007+0000 7f8eb5031640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000b810 tx=0x7f8ea0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.007+0000 7f8eb5031640 1 -- 172.21.15.19:0/3413629266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0161710 msgr2=0x7f8eb01f3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.007+0000 7f8eb5031640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0161710 0x7f8eb01f3740 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009f40 tx=0x7f8ea8004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.007+0000 7f8eb5031640 1 -- 172.21.15.19:0/3413629266 shutdown_connections 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.007+0000 7f8eb5031640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.007+0000 7f8eb5031640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0161710 0x7f8eb01f3740 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.007+0000 7f8eb5031640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb015dc20 0x7f8eb00cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.007+0000 7f8eb5031640 1 --2- 172.21.15.19:0/3413629266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8eb00b9330 0x7f8eb015bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.007+0000 7f8eb5031640 1 -- 172.21.15.19:0/3413629266 >> 172.21.15.19:0/3413629266 conn(0x7f8eb00d0420 msgr2=0x7f8eb015cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.008+0000 7f8eb5031640 1 -- 172.21.15.19:0/3413629266 shutdown_connections 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.008+0000 7f8eb5031640 1 -- 172.21.15.19:0/3413629266 wait complete. 2024-04-25T10:05:29.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:05:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.137+0000 7f4389a6b640 1 Processor -- start 2024-04-25T10:05:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.137+0000 7f4389a6b640 1 -- start start 2024-04-25T10:05:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.138+0000 7f4389a6b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f438415dc20 0x7f43840ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.138+0000 7f4389a6b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438406cda0 con 0x7f43840d32f0 2024-04-25T10:05:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.138+0000 7f4389a6b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438415e000 con 0x7f4384161710 2024-04-25T10:05:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.138+0000 7f4389a6b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43840d1de0 con 0x7f438415dc20 2024-04-25T10:05:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.138+0000 7f43837fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f438415dc20 0x7f43840ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.138+0000 7f4383fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f43840d32f0 0x7f438406c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46618/0 (socket says 172.21.15.19:46618) 2024-04-25T10:05:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.138+0000 7f4383fff640 1 -- 172.21.15.19:0/3437090337 learned_addr learned my addr 172.21.15.19:0/3437090337 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.138+0000 7f43837fe640 1 -- 172.21.15.19:0/3437090337 >> v1:172.21.15.112:6789/0 conn(0x7f4384161710 legacy=0x7f438406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.138+0000 7f43837fe640 1 -- 172.21.15.19:0/3437090337 >> v1:172.21.15.19:6789/0 conn(0x7f43840d32f0 legacy=0x7f438406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.138+0000 7f43837fe640 1 -- 172.21.15.19:0/3437090337 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4384161c60 con 0x7f438415dc20 2024-04-25T10:05:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.139+0000 7f43837fe640 1 --2- 172.21.15.19:0/3437090337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f438415dc20 0x7f43840ce670 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f43780097a0 tx=0x7f437802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d196c644bf8d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.139+0000 7f43827fc640 1 -- 172.21.15.19:0/3437090337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378037070 con 0x7f438415dc20 2024-04-25T10:05:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.139+0000 7f43827fc640 1 -- 172.21.15.19:0/3437090337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4378032440 con 0x7f438415dc20 2024-04-25T10:05:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.139+0000 7f43827fc640 1 -- 172.21.15.19:0/3437090337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378032810 con 0x7f438415dc20 2024-04-25T10:05:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.140+0000 7f4389a6b640 1 -- 172.21.15.19:0/3437090337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f438415dc20 msgr2=0x7f43840ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.140+0000 7f4389a6b640 1 --2- 172.21.15.19:0/3437090337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f438415dc20 0x7f43840ce670 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f43780097a0 tx=0x7f437802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.140+0000 7f4389a6b640 1 -- 172.21.15.19:0/3437090337 shutdown_connections 2024-04-25T10:05:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.140+0000 7f4389a6b640 1 --2- 172.21.15.19:0/3437090337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f438415dc20 0x7f43840ce670 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.140+0000 7f4389a6b640 1 -- 172.21.15.19:0/3437090337 >> 172.21.15.19:0/3437090337 conn(0x7f43840d0420 msgr2=0x7f43840d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.140+0000 7f4389a6b640 1 -- 172.21.15.19:0/3437090337 shutdown_connections 2024-04-25T10:05:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.140+0000 7f4389a6b640 1 -- 172.21.15.19:0/3437090337 wait complete. 2024-04-25T10:05:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.141+0000 7f4389a6b640 1 Processor -- start 2024-04-25T10:05:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.141+0000 7f4389a6b640 1 -- start start 2024-04-25T10:05:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4389a6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43840d32f0 0x7f438415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4389a6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f438415dc20 0x7f43841f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4389a6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4384161710 0x7f4384158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4389a6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4384162500 con 0x7f4384161710 2024-04-25T10:05:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4389a6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4384162220 con 0x7f438415dc20 2024-04-25T10:05:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4389a6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43840b5e50 con 0x7f43840d32f0 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4383fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4384161710 0x7f4384158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4383fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4384161710 0x7f4384158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44818/0 (socket says 172.21.15.19:44818) 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4383fff640 1 -- 172.21.15.19:0/3033610950 learned_addr learned my addr 172.21.15.19:0/3033610950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f43837fe640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43840d32f0 0x7f438415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4382ffd640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f438415dc20 0x7f43841f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4383fff640 1 -- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43840d32f0 msgr2=0x7f438415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4383fff640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43840d32f0 0x7f438415e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4383fff640 1 -- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f438415dc20 msgr2=0x7f43841f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4383fff640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f438415dc20 0x7f43841f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4383fff640 1 -- 172.21.15.19:0/3033610950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4384158d50 con 0x7f4384161710 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4383fff640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4384161710 0x7f4384158810 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f437c00cad0 tx=0x7f437c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f436bfff640 1 -- 172.21.15.19:0/3033610950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c017070 con 0x7f4384161710 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f436bfff640 1 -- 172.21.15.19:0/3033610950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437c00ab70 con 0x7f4384161710 2024-04-25T10:05:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f436bfff640 1 -- 172.21.15.19:0/3033610950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c012400 con 0x7f4384161710 2024-04-25T10:05:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4389a6b640 1 -- 172.21.15.19:0/3033610950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43841f76b0 con 0x7f4384161710 2024-04-25T10:05:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.142+0000 7f4389a6b640 1 -- 172.21.15.19:0/3033610950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43841f7c70 con 0x7f4384161710 2024-04-25T10:05:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.143+0000 7f4389a6b640 1 -- 172.21.15.19:0/3033610950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4384073a00 con 0x7f4384161710 2024-04-25T10:05:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.144+0000 7f436bfff640 1 -- 172.21.15.19:0/3033610950 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f437c01a070 con 0x7f4384161710 2024-04-25T10:05:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.144+0000 7f436bfff640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f437006b4d0 0x7f437006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.145+0000 7f436bfff640 1 -- 172.21.15.19:0/3033610950 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f437c089c10 con 0x7f4384161710 2024-04-25T10:05:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.145+0000 7f43837fe640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f437006b4d0 0x7f437006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.145+0000 7f43837fe640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f437006b4d0 0x7f437006d990 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4378034000 tx=0x7f437802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.149+0000 7f436bfff640 1 -- 172.21.15.19:0/3033610950 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f437c0584a0 con 0x7f4384161710 2024-04-25T10:05:29.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:29.329+0000 7f4389a6b640 1 -- 172.21.15.19:0/3033610950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f43840d16b0 con 0x7f4384161710 2024-04-25T10:05:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.009+0000 7f436bfff640 1 -- 172.21.15.19:0/3033610950 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v181) v1 ==== 169+0+0 (secure 0 0 0) 0x7f437c05c540 con 0x7f4384161710 2024-04-25T10:05:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T10:05:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.012+0000 7f4389a6b640 1 -- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f437006b4d0 msgr2=0x7f437006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.012+0000 7f4389a6b640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f437006b4d0 0x7f437006d990 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4378034000 tx=0x7f437802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.012+0000 7f4389a6b640 1 -- 172.21.15.19:0/3033610950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4384161710 msgr2=0x7f4384158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.012+0000 7f4389a6b640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4384161710 0x7f4384158810 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f437c00cad0 tx=0x7f437c00cf90 comp rx=0 tx=0).stop 2024-04-25T10:05:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.013+0000 7f4389a6b640 1 -- 172.21.15.19:0/3033610950 shutdown_connections 2024-04-25T10:05:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.013+0000 7f4389a6b640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f437006b4d0 0x7f437006d990 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.013+0000 7f4389a6b640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4384161710 0x7f4384158810 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.013+0000 7f4389a6b640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f438415dc20 0x7f43841f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.013+0000 7f4389a6b640 1 --2- 172.21.15.19:0/3033610950 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43840d32f0 0x7f438415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.013+0000 7f4389a6b640 1 -- 172.21.15.19:0/3033610950 >> 172.21.15.19:0/3033610950 conn(0x7f43840d0420 msgr2=0x7f438415bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.014+0000 7f4389a6b640 1 -- 172.21.15.19:0/3033610950 shutdown_connections 2024-04-25T10:05:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.014+0000 7f4389a6b640 1 -- 172.21.15.19:0/3033610950 wait complete. 2024-04-25T10:05:30.036 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:05:30.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.140+0000 7f046743f640 1 Processor -- start 2024-04-25T10:05:30.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.140+0000 7f046743f640 1 -- start start 2024-04-25T10:05:30.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f046743f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f046015b7d0 0x7f04600cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f046743f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04600b5620 0x7f04600b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f046743f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f046006c2d0 0x7f046006c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f046743f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04600b5fc0 con 0x7f04600b5620 2024-04-25T10:05:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f046743f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046006cca0 con 0x7f046006c2d0 2024-04-25T10:05:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f046743f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046006cf50 con 0x7f046015b7d0 2024-04-25T10:05:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f04649da640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04600b5620 0x7f04600b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f04649da640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04600b5620 0x7f04600b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44834/0 (socket says 172.21.15.19:44834) 2024-04-25T10:05:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f04649da640 1 -- 172.21.15.19:0/3420391785 learned_addr learned my addr 172.21.15.19:0/3420391785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f04651db640 1 --2- 172.21.15.19:0/3420391785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f046015b7d0 0x7f04600cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f04659dc640 1 --2- 172.21.15.19:0/3420391785 >> v2:172.21.15.112:3300/0 conn(0x7f046006c2d0 0x7f046006c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f04649da640 1 -- 172.21.15.19:0/3420391785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f046015b7d0 msgr2=0x7f04600cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f04649da640 1 --2- 172.21.15.19:0/3420391785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f046015b7d0 0x7f04600cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f04649da640 1 -- 172.21.15.19:0/3420391785 >> v2:172.21.15.112:3300/0 conn(0x7f046006c2d0 msgr2=0x7f046006c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f04649da640 1 --2- 172.21.15.19:0/3420391785 >> v2:172.21.15.112:3300/0 conn(0x7f046006c2d0 0x7f046006c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.141+0000 7f04649da640 1 -- 172.21.15.19:0/3420391785 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04600ce530 con 0x7f04600b5620 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.142+0000 7f04649da640 1 --2- 172.21.15.19:0/3420391785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04600b5620 0x7f04600b5a80 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f046015e340 tx=0x7f045002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70f325dd12dc433 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.142+0000 7f0457fff640 1 -- 172.21.15.19:0/3420391785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450037070 con 0x7f04600b5620 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.142+0000 7f0457fff640 1 -- 172.21.15.19:0/3420391785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0450032440 con 0x7f04600b5620 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.142+0000 7f0457fff640 1 -- 172.21.15.19:0/3420391785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450032810 con 0x7f04600b5620 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.143+0000 7f046743f640 1 -- 172.21.15.19:0/3420391785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04600b5620 msgr2=0x7f04600b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.143+0000 7f046743f640 1 --2- 172.21.15.19:0/3420391785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04600b5620 0x7f04600b5a80 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f046015e340 tx=0x7f045002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.143+0000 7f046743f640 1 -- 172.21.15.19:0/3420391785 shutdown_connections 2024-04-25T10:05:30.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.143+0000 7f046743f640 1 --2- 172.21.15.19:0/3420391785 >> v2:172.21.15.112:3300/0 conn(0x7f046006c2d0 0x7f046006c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.143+0000 7f046743f640 1 --2- 172.21.15.19:0/3420391785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04600b5620 0x7f04600b5a80 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.143+0000 7f046743f640 1 --2- 172.21.15.19:0/3420391785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f046015b7d0 0x7f04600cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.143+0000 7f046743f640 1 -- 172.21.15.19:0/3420391785 >> 172.21.15.19:0/3420391785 conn(0x7f04600d0800 msgr2=0x7f04600d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.143+0000 7f046743f640 1 -- 172.21.15.19:0/3420391785 shutdown_connections 2024-04-25T10:05:30.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.143+0000 7f046743f640 1 -- 172.21.15.19:0/3420391785 wait complete. 2024-04-25T10:05:30.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.144+0000 7f046743f640 1 Processor -- start 2024-04-25T10:05:30.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.144+0000 7f046743f640 1 -- start start 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f046743f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f046006c2d0 0x7f0460157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f046743f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04600b5620 0x7f04600cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f046743f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046015b7d0 0x7f04601603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f046743f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046006d230 con 0x7f04600b5620 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f046743f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04600ceaf0 con 0x7f046015b7d0 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f046743f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04600cf550 con 0x7f046006c2d0 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f04649da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04600b5620 0x7f04600cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f04649da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04600b5620 0x7f04600cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44844/0 (socket says 172.21.15.19:44844) 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f04649da640 1 -- 172.21.15.19:0/1537058267 learned_addr learned my addr 172.21.15.19:0/1537058267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f04651db640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f046006c2d0 0x7f0460157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f04659dc640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046015b7d0 0x7f04601603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f04649da640 1 -- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f046006c2d0 msgr2=0x7f0460157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f04649da640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f046006c2d0 0x7f0460157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f04649da640 1 -- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046015b7d0 msgr2=0x7f04601603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f04649da640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046015b7d0 0x7f04601603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.145+0000 7f04649da640 1 -- 172.21.15.19:0/1537058267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04601586e0 con 0x7f04600b5620 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.146+0000 7f04649da640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04600b5620 0x7f04600cb580 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f045002a220 tx=0x7f04500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.146+0000 7f0455ffb640 1 -- 172.21.15.19:0/1537058267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450041020 con 0x7f04600b5620 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.146+0000 7f0455ffb640 1 -- 172.21.15.19:0/1537058267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04500306c0 con 0x7f04600b5620 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.146+0000 7f046743f640 1 -- 172.21.15.19:0/1537058267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04600cd580 con 0x7f04600b5620 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.146+0000 7f0455ffb640 1 -- 172.21.15.19:0/1537058267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450037040 con 0x7f04600b5620 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.146+0000 7f046743f640 1 -- 172.21.15.19:0/1537058267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04600cdb40 con 0x7f04600b5620 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.147+0000 7f046743f640 1 -- 172.21.15.19:0/1537058267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0460073a00 con 0x7f04600b5620 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.148+0000 7f0455ffb640 1 -- 172.21.15.19:0/1537058267 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0450044020 con 0x7f04600b5620 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.148+0000 7f0455ffb640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f044406b500 0x7f044406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.148+0000 7f0455ffb640 1 -- 172.21.15.19:0/1537058267 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f04500aa5e0 con 0x7f04600b5620 2024-04-25T10:05:30.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.148+0000 7f04651db640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f044406b500 0x7f044406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.149+0000 7f04651db640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f044406b500 0x7f044406d9c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f04600b2790 tx=0x7f045c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.152+0000 7f0455ffb640 1 -- 172.21.15.19:0/1537058267 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f045004b030 con 0x7f04600b5620 2024-04-25T10:05:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.334+0000 7f046743f640 1 -- 172.21.15.19:0/1537058267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f046015dbc0 con 0x7f04600b5620 2024-04-25T10:05:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.337+0000 7f0455ffb640 1 -- 172.21.15.19:0/1537058267 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0450079010 con 0x7f04600b5620 2024-04-25T10:05:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 -- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f044406b500 msgr2=0x7f044406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f044406b500 0x7f044406d9c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f04600b2790 tx=0x7f045c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 -- 172.21.15.19:0/1537058267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04600b5620 msgr2=0x7f04600cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04600b5620 0x7f04600cb580 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f045002a220 tx=0x7f04500046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 -- 172.21.15.19:0/1537058267 shutdown_connections 2024-04-25T10:05:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f044406b500 0x7f044406d9c0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f046015b7d0 0x7f04601603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04600b5620 0x7f04600cb580 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 --2- 172.21.15.19:0/1537058267 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f046006c2d0 0x7f0460157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 -- 172.21.15.19:0/1537058267 >> 172.21.15.19:0/1537058267 conn(0x7f04600d0800 msgr2=0x7f04600b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 -- 172.21.15.19:0/1537058267 shutdown_connections 2024-04-25T10:05:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.339+0000 7f046743f640 1 -- 172.21.15.19:0/1537058267 wait complete. 2024-04-25T10:05:30.356 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:05:30.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.451+0000 7f8b2ad89640 1 Processor -- start 2024-04-25T10:05:30.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.451+0000 7f8b2ad89640 1 -- start start 2024-04-25T10:05:30.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.452+0000 7f8b2ad89640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8b24154ed0 0x7f8b240aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.452+0000 7f8b2ad89640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2415ec60 con 0x7f8b24071ac0 2024-04-25T10:05:30.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.452+0000 7f8b2ad89640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2415ef40 con 0x7f8b2406c1d0 2024-04-25T10:05:30.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.452+0000 7f8b2ad89640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2406db90 con 0x7f8b24154ed0 2024-04-25T10:05:30.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.452+0000 7f8b1bfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8b24071ac0 0x7f8b240ad410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46622/0 (socket says 172.21.15.19:46622) 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.452+0000 7f8b1bfff640 1 -- 172.21.15.19:0/3897384117 learned_addr learned my addr 172.21.15.19:0/3897384117 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.452+0000 7f8b28b25640 1 --2- 172.21.15.19:0/3897384117 >> v2:172.21.15.112:3300/0 conn(0x7f8b24154ed0 0x7f8b240aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.452+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367894413 0 0) 0x7f8b2415ec60 con 0x7f8b24071ac0 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.452+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b0c003520 con 0x7f8b24071ac0 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2845007377 0 0) 0x7f8b0c003520 con 0x7f8b24071ac0 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 >> v2:172.21.15.112:3300/0 conn(0x7f8b24154ed0 msgr2=0x7f8b240aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 --2- 172.21.15.19:0/3897384117 >> v2:172.21.15.112:3300/0 conn(0x7f8b24154ed0 0x7f8b240aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 >> v1:172.21.15.112:6789/0 conn(0x7f8b2406c1d0 legacy=0x7f8b2406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2406de70 con 0x7f8b24071ac0 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f8b14001670 con 0x7f8b24071ac0 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 >> v1:172.21.15.19:6789/0 conn(0x7f8b24071ac0 legacy=0x7f8b240ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b0c0039b0 0x7f8b0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b0c004800 0x7f8b0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b0c0051c0 0x7f8b0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2406db90 con 0x7f8b0c004800 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c00c160 con 0x7f8b0c0039b0 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c00c360 con 0x7f8b0c0051c0 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b28b25640 1 --2- 172.21.15.19:0/3897384117 >> v2:172.21.15.112:3300/0 conn(0x7f8b24154ed0 0x7f8b240aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1bfff640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b0c004800 0x7f8b0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b29326640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b0c0051c0 0x7f8b0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b28b25640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b0c0039b0 0x7f8b0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1bfff640 1 -- 172.21.15.19:0/3897384117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b0c0051c0 msgr2=0x7f8b0c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1bfff640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b0c0051c0 0x7f8b0c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1bfff640 1 -- 172.21.15.19:0/3897384117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b0c0039b0 msgr2=0x7f8b0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1bfff640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b0c0039b0 0x7f8b0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b1bfff640 1 -- 172.21.15.19:0/3897384117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c003520 con 0x7f8b0c004800 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b29326640 1 -- 172.21.15.19:0/3897384117 reap_dead start 2024-04-25T10:05:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.453+0000 7f8b28b25640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b0c0039b0 0x7f8b0c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.454+0000 7f8b1bfff640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b0c004800 0x7f8b0c004c80 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c210 tx=0x7f8b0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15d6c7e1389a023 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.454+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14007d10 con 0x7f8b0c004800 2024-04-25T10:05:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.454+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b140058d0 con 0x7f8b0c004800 2024-04-25T10:05:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.454+0000 7f8b1b7fe640 1 -- 172.21.15.19:0/3897384117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14005d20 con 0x7f8b0c004800 2024-04-25T10:05:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.454+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3897384117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b0c004800 msgr2=0x7f8b0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.454+0000 7f8b2ad89640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b0c004800 0x7f8b0c004c80 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c210 tx=0x7f8b0c005830 comp rx=0 tx=0).stop 2024-04-25T10:05:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.454+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3897384117 shutdown_connections 2024-04-25T10:05:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.454+0000 7f8b2ad89640 1 --2- 172.21.15.19:0/3897384117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b0c004800 0x7f8b0c004c80 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.454+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3897384117 >> 172.21.15.19:0/3897384117 conn(0x7f8b240af670 msgr2=0x7f8b240b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.454+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3897384117 shutdown_connections 2024-04-25T10:05:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.455+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3897384117 wait complete. 2024-04-25T10:05:30.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.456+0000 7f8b2ad89640 1 Processor -- start 2024-04-25T10:05:30.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.456+0000 7f8b2ad89640 1 -- start start 2024-04-25T10:05:30.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b2ad89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b24145440 0x7f8b24147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b2ad89640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b241f7ef0 0x7f8b241575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b2ad89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b24157b30 0x7f8b24146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b2ad89640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b240ad260 con 0x7f8b241f7ef0 2024-04-25T10:05:30.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b2ad89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b240ad0e0 con 0x7f8b24157b30 2024-04-25T10:05:30.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b2ad89640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b240ad3e0 con 0x7f8b24145440 2024-04-25T10:05:30.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b1bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b241f7ef0 0x7f8b241575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b1bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b241f7ef0 0x7f8b241575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44852/0 (socket says 172.21.15.19:44852) 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b1bfff640 1 -- 172.21.15.19:0/3051936744 learned_addr learned my addr 172.21.15.19:0/3051936744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b28b25640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b24145440 0x7f8b24147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b29326640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b24157b30 0x7f8b24146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b1bfff640 1 -- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b24145440 msgr2=0x7f8b24147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b1bfff640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b24145440 0x7f8b24147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b1bfff640 1 -- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b24157b30 msgr2=0x7f8b24146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b1bfff640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b24157b30 0x7f8b24146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b1bfff640 1 -- 172.21.15.19:0/3051936744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b24145820 con 0x7f8b241f7ef0 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b28b25640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b24145440 0x7f8b24147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b29326640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b24157b30 0x7f8b24146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b1bfff640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b241f7ef0 0x7f8b241575f0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c330 tx=0x7f8b24158830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b197fa640 1 -- 172.21.15.19:0/3051936744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14041020 con 0x7f8b241f7ef0 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b197fa640 1 -- 172.21.15.19:0/3051936744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14000ed0 con 0x7f8b241f7ef0 2024-04-25T10:05:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3051936744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b241481b0 con 0x7f8b241f7ef0 2024-04-25T10:05:30.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b197fa640 1 -- 172.21.15.19:0/3051936744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14038040 con 0x7f8b241f7ef0 2024-04-25T10:05:30.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.457+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3051936744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b24148770 con 0x7f8b241f7ef0 2024-04-25T10:05:30.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.459+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3051936744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b240b2b10 con 0x7f8b241f7ef0 2024-04-25T10:05:30.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.459+0000 7f8b197fa640 1 -- 172.21.15.19:0/3051936744 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b14044020 con 0x7f8b241f7ef0 2024-04-25T10:05:30.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.459+0000 7f8b197fa640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b1006b4d0 0x7f8b1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.460+0000 7f8b197fa640 1 -- 172.21.15.19:0/3051936744 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8b140a9d90 con 0x7f8b241f7ef0 2024-04-25T10:05:30.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.460+0000 7f8b28b25640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b1006b4d0 0x7f8b1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.460+0000 7f8b28b25640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b1006b4d0 0x7f8b1006d990 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8b240692f0 tx=0x7f8b20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.464+0000 7f8b197fa640 1 -- 172.21.15.19:0/3051936744 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b14078c50 con 0x7f8b241f7ef0 2024-04-25T10:05:30.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:30.712+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3051936744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f8b24070020 con 0x7f8b241f7ef0 2024-04-25T10:05:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.010+0000 7f8b197fa640 1 -- 172.21.15.19:0/3051936744 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v611) v1 ==== 147+0+0 (secure 0 0 0) 0x7f8b14078550 con 0x7f8b241f7ef0 2024-04-25T10:05:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b1006b4d0 msgr2=0x7f8b1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b1006b4d0 0x7f8b1006d990 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8b240692f0 tx=0x7f8b20008040 comp rx=0 tx=0).stop 2024-04-25T10:05:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3051936744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b241f7ef0 msgr2=0x7f8b241575f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b241f7ef0 0x7f8b241575f0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c330 tx=0x7f8b24158830 comp rx=0 tx=0).stop 2024-04-25T10:05:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3051936744 shutdown_connections 2024-04-25T10:05:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8b1006b4d0 0x7f8b1006d990 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b24157b30 0x7f8b24146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b241f7ef0 0x7f8b241575f0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 --2- 172.21.15.19:0/3051936744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b24145440 0x7f8b24147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3051936744 >> 172.21.15.19:0/3051936744 conn(0x7f8b0c004800 msgr2=0x7f8b2406ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3051936744 shutdown_connections 2024-04-25T10:05:31.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.012+0000 7f8b2ad89640 1 -- 172.21.15.19:0/3051936744 wait complete. 2024-04-25T10:05:31.029 INFO:tasks.cephfs_test_runner:test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:05:31.029 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:05:31.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:31.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.134+0000 7f58a644d640 1 Processor -- start 2024-04-25T10:05:31.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.134+0000 7f58a644d640 1 -- start start 2024-04-25T10:05:31.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f58a644d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f58a015dbe0 0x7f58a00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f58a644d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58a00b2cb0 0x7f58a00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f58a644d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a00b97a0 con 0x7f58a00b92a0 2024-04-25T10:05:31.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f58a644d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a015dfc0 con 0x7f58a00b2cb0 2024-04-25T10:05:31.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f58a644d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a00d1a00 con 0x7f58a015dbe0 2024-04-25T10:05:31.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f589f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58a00b2cb0 0x7f58a00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f589f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58a00b2cb0 0x7f58a00d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44858/0 (socket says 172.21.15.19:44858) 2024-04-25T10:05:31.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f589f7fe640 1 -- 172.21.15.19:0/2225528462 learned_addr learned my addr 172.21.15.19:0/2225528462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:31.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f589ffff640 1 --2- 172.21.15.19:0/2225528462 >> v2:172.21.15.112:3300/0 conn(0x7f58a015dbe0 0x7f58a00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f589f7fe640 1 -- 172.21.15.19:0/2225528462 >> v2:172.21.15.112:3300/0 conn(0x7f58a015dbe0 msgr2=0x7f58a00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f589f7fe640 1 --2- 172.21.15.19:0/2225528462 >> v2:172.21.15.112:3300/0 conn(0x7f58a015dbe0 0x7f58a00d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f589f7fe640 1 -- 172.21.15.19:0/2225528462 >> v1:172.21.15.112:6789/0 conn(0x7f58a00b92a0 legacy=0x7f58a00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.135+0000 7f589f7fe640 1 -- 172.21.15.19:0/2225528462 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00d1c50 con 0x7f58a00b2cb0 2024-04-25T10:05:31.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.136+0000 7f589f7fe640 1 --2- 172.21.15.19:0/2225528462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a00b2cb0 0x7f58a00d0e50 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f58a006c140 tx=0x7f589002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ea3590b71fccdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.136+0000 7f589effd640 1 -- 172.21.15.19:0/2225528462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890037070 con 0x7f58a00b2cb0 2024-04-25T10:05:31.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.136+0000 7f589effd640 1 -- 172.21.15.19:0/2225528462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5890032440 con 0x7f58a00b2cb0 2024-04-25T10:05:31.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.136+0000 7f589effd640 1 -- 172.21.15.19:0/2225528462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890032810 con 0x7f58a00b2cb0 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.137+0000 7f58a644d640 1 -- 172.21.15.19:0/2225528462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a00b2cb0 msgr2=0x7f58a00d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.137+0000 7f58a644d640 1 --2- 172.21.15.19:0/2225528462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a00b2cb0 0x7f58a00d0e50 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f58a006c140 tx=0x7f589002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.137+0000 7f58a644d640 1 -- 172.21.15.19:0/2225528462 shutdown_connections 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.137+0000 7f58a644d640 1 --2- 172.21.15.19:0/2225528462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a00b2cb0 0x7f58a00d0e50 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.137+0000 7f58a644d640 1 --2- 172.21.15.19:0/2225528462 >> v2:172.21.15.112:3300/0 conn(0x7f58a015dbe0 0x7f58a00d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.137+0000 7f58a644d640 1 -- 172.21.15.19:0/2225528462 >> 172.21.15.19:0/2225528462 conn(0x7f58a0061a20 msgr2=0x7f58a0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.137+0000 7f58a644d640 1 -- 172.21.15.19:0/2225528462 shutdown_connections 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.137+0000 7f58a644d640 1 -- 172.21.15.19:0/2225528462 wait complete. 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.138+0000 7f58a644d640 1 Processor -- start 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.138+0000 7f58a644d640 1 -- start start 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a644d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a00b2cb0 0x7f58a00b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a644d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58a00b92a0 0x7f58a0148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a644d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a015dbe0 0x7f58a0160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a644d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00b6ab0 con 0x7f58a015dbe0 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a644d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00d2210 con 0x7f58a00b2cb0 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a644d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00b6c30 con 0x7f58a00b92a0 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a49ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a015dbe0 0x7f58a0160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a49ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a015dbe0 0x7f58a0160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44860/0 (socket says 172.21.15.19:44860) 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a49ea640 1 -- 172.21.15.19:0/2828044400 learned_addr learned my addr 172.21.15.19:0/2828044400 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f589ffff640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a00b2cb0 0x7f58a00b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f589f7fe640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58a00b92a0 0x7f58a0148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a49ea640 1 -- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58a00b92a0 msgr2=0x7f58a0148c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a49ea640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58a00b92a0 0x7f58a0148c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a49ea640 1 -- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a00b2cb0 msgr2=0x7f58a00b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a49ea640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a00b2cb0 0x7f58a00b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a49ea640 1 -- 172.21.15.19:0/2828044400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a0160d40 con 0x7f58a015dbe0 2024-04-25T10:05:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f589ffff640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a00b2cb0 0x7f58a00b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.139+0000 7f58a49ea640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a015dbe0 0x7f58a0160210 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f589800c730 tx=0x7f589800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.140+0000 7f589cff9640 1 -- 172.21.15.19:0/2828044400 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898003b70 con 0x7f58a015dbe0 2024-04-25T10:05:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.140+0000 7f589cff9640 1 -- 172.21.15.19:0/2828044400 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5898003d10 con 0x7f58a015dbe0 2024-04-25T10:05:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.140+0000 7f58a644d640 1 -- 172.21.15.19:0/2828044400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a0149180 con 0x7f58a015dbe0 2024-04-25T10:05:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.140+0000 7f589cff9640 1 -- 172.21.15.19:0/2828044400 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589800aa90 con 0x7f58a015dbe0 2024-04-25T10:05:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.140+0000 7f58a644d640 1 -- 172.21.15.19:0/2828044400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a015e1f0 con 0x7f58a015dbe0 2024-04-25T10:05:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.141+0000 7f58a644d640 1 -- 172.21.15.19:0/2828044400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a0073990 con 0x7f58a015dbe0 2024-04-25T10:05:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.141+0000 7f589cff9640 1 -- 172.21.15.19:0/2828044400 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f589800ac30 con 0x7f58a015dbe0 2024-04-25T10:05:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.142+0000 7f589cff9640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588c06b500 0x7f588c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.142+0000 7f589cff9640 1 -- 172.21.15.19:0/2828044400 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5898089f20 con 0x7f58a015dbe0 2024-04-25T10:05:31.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.142+0000 7f589ffff640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588c06b500 0x7f588c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.142+0000 7f589ffff640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588c06b500 0x7f588c06d9c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5894009af0 tx=0x7f5894008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.146+0000 7f589cff9640 1 -- 172.21.15.19:0/2828044400 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5898058f10 con 0x7f58a015dbe0 2024-04-25T10:05:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.330+0000 7f58a644d640 1 -- 172.21.15.19:0/2828044400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58a0160750 con 0x7f58a015dbe0 2024-04-25T10:05:31.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.331+0000 7f589cff9640 1 -- 172.21.15.19:0/2828044400 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+13289 (secure 0 0 0) 0x7f5898058810 con 0x7f58a015dbe0 2024-04-25T10:05:31.337 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:31.337 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":181,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:30.004499+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6833/829980210":"2024-04-26T10:05:26.631670+0000","172.21.15.19:6832/829980210":"2024-04-26T10:05:26.631670+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.333+0000 7f58a644d640 1 -- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588c06b500 msgr2=0x7f588c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.333+0000 7f58a644d640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588c06b500 0x7f588c06d9c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5894009af0 tx=0x7f5894008040 comp rx=0 tx=0).stop 2024-04-25T10:05:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.333+0000 7f58a644d640 1 -- 172.21.15.19:0/2828044400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a015dbe0 msgr2=0x7f58a0160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.333+0000 7f58a644d640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a015dbe0 0x7f58a0160210 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f589800c730 tx=0x7f589800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.334+0000 7f58a644d640 1 -- 172.21.15.19:0/2828044400 shutdown_connections 2024-04-25T10:05:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.334+0000 7f58a644d640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f588c06b500 0x7f588c06d9c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.334+0000 7f58a644d640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a015dbe0 0x7f58a0160210 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.334+0000 7f58a644d640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58a00b92a0 0x7f58a0148c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.334+0000 7f58a644d640 1 --2- 172.21.15.19:0/2828044400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58a00b2cb0 0x7f58a00b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.334+0000 7f58a644d640 1 -- 172.21.15.19:0/2828044400 >> 172.21.15.19:0/2828044400 conn(0x7f58a0061a20 msgr2=0x7f58a00b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.334+0000 7f58a644d640 1 -- 172.21.15.19:0/2828044400 shutdown_connections 2024-04-25T10:05:31.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.334+0000 7f58a644d640 1 -- 172.21.15.19:0/2828044400 wait complete. 2024-04-25T10:05:31.350 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:31.350 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.448+0000 7f48e796a640 1 Processor -- start 2024-04-25T10:05:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.448+0000 7f48e796a640 1 -- start start 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e796a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f48e00ce230 0x7f48e00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e796a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f48e015b270 0x7f48e015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e796a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e0072280 con 0x7f48e00cc4f0 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e796a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e0071270 con 0x7f48e00ce230 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e796a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e00714c0 con 0x7f48e015b270 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e4f05640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f48e00ce230 0x7f48e00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e5706640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f48e00cc4f0 0x7f48e00cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46634/0 (socket says 172.21.15.19:46634) 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e5706640 1 -- 172.21.15.19:0/3817210751 learned_addr learned my addr 172.21.15.19:0/3817210751 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e5f07640 1 --2- 172.21.15.19:0/3817210751 >> v2:172.21.15.112:3300/0 conn(0x7f48e015b270 0x7f48e015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e4f05640 1 -- 172.21.15.19:0/3817210751 >> v2:172.21.15.112:3300/0 conn(0x7f48e015b270 msgr2=0x7f48e015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e4f05640 1 --2- 172.21.15.19:0/3817210751 >> v2:172.21.15.112:3300/0 conn(0x7f48e015b270 0x7f48e015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e4f05640 1 -- 172.21.15.19:0/3817210751 >> v1:172.21.15.19:6789/0 conn(0x7f48e00cc4f0 legacy=0x7f48e00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48e4f05640 1 -- 172.21.15.19:0/3817210751 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e00717a0 con 0x7f48e00ce230 2024-04-25T10:05:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.449+0000 7f48dffff640 1 -- 172.21.15.19:0/3817210751 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132777717 0 0) 0x7f48e0072280 con 0x7f48e00cc4f0 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.450+0000 7f48e4f05640 1 --2- 172.21.15.19:0/3817210751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48e00ce230 0x7f48e00cc8d0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f48e015af60 tx=0x7f48d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223efce5219eef9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.450+0000 7f48dffff640 1 -- 172.21.15.19:0/3817210751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4037070 con 0x7f48e00ce230 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.450+0000 7f48dffff640 1 -- 172.21.15.19:0/3817210751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d4032440 con 0x7f48e00ce230 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.450+0000 7f48dffff640 1 -- 172.21.15.19:0/3817210751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4032810 con 0x7f48e00ce230 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.451+0000 7f48e796a640 1 -- 172.21.15.19:0/3817210751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48e00ce230 msgr2=0x7f48e00cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.451+0000 7f48e796a640 1 --2- 172.21.15.19:0/3817210751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48e00ce230 0x7f48e00cc8d0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f48e015af60 tx=0x7f48d402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.451+0000 7f48e796a640 1 -- 172.21.15.19:0/3817210751 shutdown_connections 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.451+0000 7f48e796a640 1 --2- 172.21.15.19:0/3817210751 >> v2:172.21.15.112:3300/0 conn(0x7f48e015b270 0x7f48e015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.451+0000 7f48e796a640 1 --2- 172.21.15.19:0/3817210751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48e00ce230 0x7f48e00cc8d0 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.451+0000 7f48e796a640 1 -- 172.21.15.19:0/3817210751 >> 172.21.15.19:0/3817210751 conn(0x7f48e006abe0 msgr2=0x7f48e006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.451+0000 7f48e796a640 1 -- 172.21.15.19:0/3817210751 shutdown_connections 2024-04-25T10:05:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.451+0000 7f48e796a640 1 -- 172.21.15.19:0/3817210751 wait complete. 2024-04-25T10:05:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.452+0000 7f48e796a640 1 Processor -- start 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.452+0000 7f48e796a640 1 -- start start 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e796a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e00cc4f0 0x7f48e01562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e796a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e00ce230 0x7f48e0158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e796a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48e015b270 0x7f48e0157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e796a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e015bc20 con 0x7f48e015b270 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e796a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e0071d60 con 0x7f48e00ce230 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e796a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e015bda0 con 0x7f48e00cc4f0 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e5f07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48e015b270 0x7f48e0157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e5f07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48e015b270 0x7f48e0157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44876/0 (socket says 172.21.15.19:44876) 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e5f07640 1 -- 172.21.15.19:0/4046909560 learned_addr learned my addr 172.21.15.19:0/4046909560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e5706640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e00cc4f0 0x7f48e01562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e4f05640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e00ce230 0x7f48e0158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e5f07640 1 -- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e00cc4f0 msgr2=0x7f48e01562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e5f07640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e00cc4f0 0x7f48e01562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e5f07640 1 -- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e00ce230 msgr2=0x7f48e0158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e5f07640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e00ce230 0x7f48e0158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.453+0000 7f48e5f07640 1 -- 172.21.15.19:0/4046909560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e0158f90 con 0x7f48e015b270 2024-04-25T10:05:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.454+0000 7f48e5f07640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48e015b270 0x7f48e0157680 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f48d0007e40 tx=0x7f48d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.454+0000 7f48ddffb640 1 -- 172.21.15.19:0/4046909560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0014620 con 0x7f48e015b270 2024-04-25T10:05:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.454+0000 7f48ddffb640 1 -- 172.21.15.19:0/4046909560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d0019070 con 0x7f48e015b270 2024-04-25T10:05:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.454+0000 7f48e796a640 1 -- 172.21.15.19:0/4046909560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e0159260 con 0x7f48e015b270 2024-04-25T10:05:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.454+0000 7f48ddffb640 1 -- 172.21.15.19:0/4046909560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0014a20 con 0x7f48e015b270 2024-04-25T10:05:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.454+0000 7f48e796a640 1 -- 172.21.15.19:0/4046909560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e0156e80 con 0x7f48e015b270 2024-04-25T10:05:31.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.455+0000 7f48e796a640 1 -- 172.21.15.19:0/4046909560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e0073a00 con 0x7f48e015b270 2024-04-25T10:05:31.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.455+0000 7f48ddffb640 1 -- 172.21.15.19:0/4046909560 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48d0014bc0 con 0x7f48e015b270 2024-04-25T10:05:31.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.456+0000 7f48ddffb640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48cc06b4d0 0x7f48cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.456+0000 7f48ddffb640 1 -- 172.21.15.19:0/4046909560 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f48d008a7a0 con 0x7f48e015b270 2024-04-25T10:05:31.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.456+0000 7f48e5706640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48cc06b4d0 0x7f48cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.457+0000 7f48e5706640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48cc06b4d0 0x7f48cc06d990 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f48d8007fd0 tx=0x7f48d8007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.460+0000 7f48ddffb640 1 -- 172.21.15.19:0/4046909560 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48d00591d0 con 0x7f48e015b270 2024-04-25T10:05:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.644+0000 7f48e796a640 1 -- 172.21.15.19:0/4046909560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48e006d3a0 con 0x7f48e015b270 2024-04-25T10:05:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.645+0000 7f48ddffb640 1 -- 172.21.15.19:0/4046909560 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v181) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48d005d270 con 0x7f48e015b270 2024-04-25T10:05:31.650 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:05:31.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 -- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48cc06b4d0 msgr2=0x7f48cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48cc06b4d0 0x7f48cc06d990 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f48d8007fd0 tx=0x7f48d8007de0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 -- 172.21.15.19:0/4046909560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48e015b270 msgr2=0x7f48e0157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48e015b270 0x7f48e0157680 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f48d0007e40 tx=0x7f48d000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 -- 172.21.15.19:0/4046909560 shutdown_connections 2024-04-25T10:05:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f48cc06b4d0 0x7f48cc06d990 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48e015b270 0x7f48e0157680 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e00ce230 0x7f48e0158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 --2- 172.21.15.19:0/4046909560 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e00cc4f0 0x7f48e01562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 -- 172.21.15.19:0/4046909560 >> 172.21.15.19:0/4046909560 conn(0x7f48e006abe0 msgr2=0x7f48e00b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 -- 172.21.15.19:0/4046909560 shutdown_connections 2024-04-25T10:05:31.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.647+0000 7f48e796a640 1 -- 172.21.15.19:0/4046909560 wait complete. 2024-04-25T10:05:31.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.759+0000 7f10a59b7640 1 Processor -- start 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.759+0000 7f10a59b7640 1 -- start start 2024-04-25T10:05:31.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.760+0000 7f10a59b7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f10a00ce590 0x7f10a0165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.760+0000 7f10a59b7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a0071480 con 0x7f10a00cfec0 2024-04-25T10:05:31.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.760+0000 7f10a59b7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a0071760 con 0x7f10a0154ee0 2024-04-25T10:05:31.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.760+0000 7f10a59b7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a0072090 con 0x7f10a00ce590 2024-04-25T10:05:31.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.760+0000 7f1097fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f10a00cfec0 0x7f10a00cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46642/0 (socket says 172.21.15.19:46642) 2024-04-25T10:05:31.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.760+0000 7f109f7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f10a00ce590 0x7f10a0165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.760+0000 7f1097fff640 1 -- 172.21.15.19:0/3348526771 learned_addr learned my addr 172.21.15.19:0/3348526771 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:31.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.760+0000 7f109f7fe640 1 -- 172.21.15.19:0/3348526771 >> v1:172.21.15.112:6789/0 conn(0x7f10a0154ee0 legacy=0x7f10a00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.760+0000 7f109f7fe640 1 -- 172.21.15.19:0/3348526771 >> v1:172.21.15.19:6789/0 conn(0x7f10a00cfec0 legacy=0x7f10a00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.760+0000 7f109e7fc640 1 -- 172.21.15.19:0/3348526771 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799774143 0 0) 0x7f10a0071480 con 0x7f10a00cfec0 2024-04-25T10:05:31.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.760+0000 7f109f7fe640 1 -- 172.21.15.19:0/3348526771 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a0071a40 con 0x7f10a00ce590 2024-04-25T10:05:31.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.761+0000 7f109f7fe640 1 --2- 172.21.15.19:0/3348526771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a00ce590 0x7f10a0165900 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f1098009d90 tx=0x7f109802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26864421d7e18fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.761+0000 7f109e7fc640 1 -- 172.21.15.19:0/3348526771 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098007740 con 0x7f10a00ce590 2024-04-25T10:05:31.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.761+0000 7f109e7fc640 1 -- 172.21.15.19:0/3348526771 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10980078e0 con 0x7f10a00ce590 2024-04-25T10:05:31.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.761+0000 7f109e7fc640 1 -- 172.21.15.19:0/3348526771 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098007cb0 con 0x7f10a00ce590 2024-04-25T10:05:31.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.761+0000 7f10a59b7640 1 -- 172.21.15.19:0/3348526771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a00ce590 msgr2=0x7f10a0165900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.761+0000 7f10a59b7640 1 --2- 172.21.15.19:0/3348526771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a00ce590 0x7f10a0165900 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f1098009d90 tx=0x7f109802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.761+0000 7f10a59b7640 1 -- 172.21.15.19:0/3348526771 shutdown_connections 2024-04-25T10:05:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.761+0000 7f10a59b7640 1 --2- 172.21.15.19:0/3348526771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a00ce590 0x7f10a0165900 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.762+0000 7f10a59b7640 1 -- 172.21.15.19:0/3348526771 >> 172.21.15.19:0/3348526771 conn(0x7f10a006abe0 msgr2=0x7f10a006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.762+0000 7f10a59b7640 1 -- 172.21.15.19:0/3348526771 shutdown_connections 2024-04-25T10:05:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.762+0000 7f10a59b7640 1 -- 172.21.15.19:0/3348526771 wait complete. 2024-04-25T10:05:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.763+0000 7f10a59b7640 1 Processor -- start 2024-04-25T10:05:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.763+0000 7f10a59b7640 1 -- start start 2024-04-25T10:05:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.763+0000 7f10a59b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10a00ce590 0x7f10a00a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.763+0000 7f10a59b7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10a00cfec0 0x7f10a00abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.763+0000 7f10a59b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0154ee0 0x7f10a01542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.763+0000 7f10a59b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01552c0 con 0x7f10a0154ee0 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.763+0000 7f10a59b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a0071230 con 0x7f10a00ce590 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.763+0000 7f10a59b7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a0155440 con 0x7f10a00cfec0 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0154ee0 0x7f10a01542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0154ee0 0x7f10a01542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44900/0 (socket says 172.21.15.19:44900) 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109f7fe640 1 -- 172.21.15.19:0/1316161332 learned_addr learned my addr 172.21.15.19:0/1316161332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f1097fff640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10a00cfec0 0x7f10a00abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109effd640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10a00ce590 0x7f10a00a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109f7fe640 1 -- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10a00cfec0 msgr2=0x7f10a00abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109f7fe640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10a00cfec0 0x7f10a00abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109f7fe640 1 -- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10a00ce590 msgr2=0x7f10a00a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109f7fe640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10a00ce590 0x7f10a00a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109f7fe640 1 -- 172.21.15.19:0/1316161332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a014f640 con 0x7f10a0154ee0 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109effd640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10a00ce590 0x7f10a00a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109f7fe640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0154ee0 0x7f10a01542f0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f1098009eb0 tx=0x7f1098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109cff9640 1 -- 172.21.15.19:0/1316161332 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098040070 con 0x7f10a0154ee0 2024-04-25T10:05:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109cff9640 1 -- 172.21.15.19:0/1316161332 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098031c90 con 0x7f10a0154ee0 2024-04-25T10:05:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f109cff9640 1 -- 172.21.15.19:0/1316161332 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109803c640 con 0x7f10a0154ee0 2024-04-25T10:05:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f10a59b7640 1 -- 172.21.15.19:0/1316161332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a00ac540 con 0x7f10a0154ee0 2024-04-25T10:05:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.764+0000 7f10a59b7640 1 -- 172.21.15.19:0/1316161332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a00acb00 con 0x7f10a0154ee0 2024-04-25T10:05:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.765+0000 7f10a59b7640 1 -- 172.21.15.19:0/1316161332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a0073990 con 0x7f10a0154ee0 2024-04-25T10:05:31.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.766+0000 7f109cff9640 1 -- 172.21.15.19:0/1316161332 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1098039040 con 0x7f10a0154ee0 2024-04-25T10:05:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.766+0000 7f109cff9640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f108006b500 0x7f108006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.766+0000 7f109cff9640 1 -- 172.21.15.19:0/1316161332 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f10980ab590 con 0x7f10a0154ee0 2024-04-25T10:05:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.766+0000 7f109effd640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f108006b500 0x7f108006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.767+0000 7f109effd640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f108006b500 0x7f108006d9c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f10a01527b0 tx=0x7f1090009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:31.771+0000 7f109cff9640 1 -- 172.21.15.19:0/1316161332 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f109804b340 con 0x7f10a0154ee0 2024-04-25T10:05:32.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:32.023+0000 7f10a59b7640 1 -- 172.21.15.19:0/1316161332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7f10a00b2130 con 0x7f10a0154ee0 2024-04-25T10:05:33.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.019+0000 7f109cff9640 1 -- 172.21.15.19:0/1316161332 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v613) v1 ==== 136+0+0 (secure 0 0 0) 0x7f109807a6d0 con 0x7f10a0154ee0 2024-04-25T10:05:33.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.021+0000 7f10a59b7640 1 -- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f108006b500 msgr2=0x7f108006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.021+0000 7f10a59b7640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f108006b500 0x7f108006d9c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f10a01527b0 tx=0x7f1090009040 comp rx=0 tx=0).stop 2024-04-25T10:05:33.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.021+0000 7f10a59b7640 1 -- 172.21.15.19:0/1316161332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0154ee0 msgr2=0x7f10a01542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.021+0000 7f10a59b7640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0154ee0 0x7f10a01542f0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f1098009eb0 tx=0x7f1098004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.022+0000 7f10a59b7640 1 -- 172.21.15.19:0/1316161332 shutdown_connections 2024-04-25T10:05:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.022+0000 7f10a59b7640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f108006b500 0x7f108006d9c0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.022+0000 7f10a59b7640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a0154ee0 0x7f10a01542f0 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.022+0000 7f10a59b7640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10a00cfec0 0x7f10a00abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.022+0000 7f10a59b7640 1 --2- 172.21.15.19:0/1316161332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10a00ce590 0x7f10a00a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.022+0000 7f10a59b7640 1 -- 172.21.15.19:0/1316161332 >> 172.21.15.19:0/1316161332 conn(0x7f10a006abe0 msgr2=0x7f10a015c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.022+0000 7f10a59b7640 1 -- 172.21.15.19:0/1316161332 shutdown_connections 2024-04-25T10:05:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.022+0000 7f10a59b7640 1 -- 172.21.15.19:0/1316161332 wait complete. 2024-04-25T10:05:33.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:05:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.146+0000 7f81f8b70640 1 Processor -- start 2024-04-25T10:05:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.146+0000 7f81f8b70640 1 -- start start 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.147+0000 7f81f8b70640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f81f415dac0 0x7f81f40d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.147+0000 7f81f8b70640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d0ab0 0x7f81f40b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.147+0000 7f81f8b70640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f40d2420 con 0x7f81f40d1f30 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.147+0000 7f81f8b70640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f40d3030 con 0x7f81f415dac0 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.147+0000 7f81f8b70640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f40d2700 con 0x7f81f40d0ab0 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.147+0000 7f81f2d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d0ab0 0x7f81f40b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.147+0000 7f81f2d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d0ab0 0x7f81f40b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:40550/0 (socket says 172.21.15.19:40550) 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.147+0000 7f81f2d9d640 1 -- 172.21.15.19:0/2621873708 learned_addr learned my addr 172.21.15.19:0/2621873708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.147+0000 7f81f259c640 1 --2- 172.21.15.19:0/2621873708 >> v2:172.21.15.112:3300/0 conn(0x7f81f415dac0 0x7f81f40d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.148+0000 7f81f2d9d640 1 -- 172.21.15.19:0/2621873708 >> v2:172.21.15.112:3300/0 conn(0x7f81f415dac0 msgr2=0x7f81f40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.148+0000 7f81f2d9d640 1 --2- 172.21.15.19:0/2621873708 >> v2:172.21.15.112:3300/0 conn(0x7f81f415dac0 0x7f81f40d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.148+0000 7f81f2d9d640 1 -- 172.21.15.19:0/2621873708 >> v1:172.21.15.112:6789/0 conn(0x7f81f40d1f30 legacy=0x7f81f40d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.148+0000 7f81f2d9d640 1 -- 172.21.15.19:0/2621873708 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f40d29e0 con 0x7f81f40d0ab0 2024-04-25T10:05:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.148+0000 7f81f2d9d640 1 --2- 172.21.15.19:0/2621873708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d0ab0 0x7f81f40b6990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009e50 tx=0x7f81ec02a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6f1081178d0b686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.149+0000 7f81f1d9b640 1 -- 172.21.15.19:0/2621873708 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec036070 con 0x7f81f40d0ab0 2024-04-25T10:05:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.149+0000 7f81f1d9b640 1 -- 172.21.15.19:0/2621873708 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec031600 con 0x7f81f40d0ab0 2024-04-25T10:05:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.149+0000 7f81f1d9b640 1 -- 172.21.15.19:0/2621873708 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0319d0 con 0x7f81f40d0ab0 2024-04-25T10:05:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.149+0000 7f81f8b70640 1 -- 172.21.15.19:0/2621873708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d0ab0 msgr2=0x7f81f40b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.149+0000 7f81f8b70640 1 --2- 172.21.15.19:0/2621873708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d0ab0 0x7f81f40b6990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009e50 tx=0x7f81ec02a7a0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.149+0000 7f81f8b70640 1 -- 172.21.15.19:0/2621873708 shutdown_connections 2024-04-25T10:05:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.149+0000 7f81f8b70640 1 --2- 172.21.15.19:0/2621873708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d0ab0 0x7f81f40b6990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.149+0000 7f81f8b70640 1 --2- 172.21.15.19:0/2621873708 >> v2:172.21.15.112:3300/0 conn(0x7f81f415dac0 0x7f81f40d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.149+0000 7f81f8b70640 1 -- 172.21.15.19:0/2621873708 >> 172.21.15.19:0/2621873708 conn(0x7f81f4061c30 msgr2=0x7f81f406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.149+0000 7f81f8b70640 1 -- 172.21.15.19:0/2621873708 shutdown_connections 2024-04-25T10:05:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.149+0000 7f81f8b70640 1 -- 172.21.15.19:0/2621873708 wait complete. 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.150+0000 7f81f8b70640 1 Processor -- start 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.151+0000 7f81f8b70640 1 -- start start 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.151+0000 7f81f8b70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f40d0ab0 0x7f81f4153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.151+0000 7f81f8b70640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d1f30 0x7f81f40b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.151+0000 7f81f8b70640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81f415dac0 0x7f81f41f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.151+0000 7f81f8b70640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f40b97d0 con 0x7f81f415dac0 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.151+0000 7f81f8b70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f415dea0 con 0x7f81f40d0ab0 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.151+0000 7f81f8b70640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f40b9950 con 0x7f81f40d1f30 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.151+0000 7f81f2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81f415dac0 0x7f81f41f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.151+0000 7f81f2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81f415dac0 0x7f81f41f7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44916/0 (socket says 172.21.15.19:44916) 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.151+0000 7f81f2d9d640 1 -- 172.21.15.19:0/3275749608 learned_addr learned my addr 172.21.15.19:0/3275749608 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81ebfff640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d1f30 0x7f81f40b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81f259c640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f40d0ab0 0x7f81f4153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81f2d9d640 1 -- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d1f30 msgr2=0x7f81f40b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81f2d9d640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d1f30 0x7f81f40b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81f2d9d640 1 -- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f40d0ab0 msgr2=0x7f81f4153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81f2d9d640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f40d0ab0 0x7f81f4153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81f2d9d640 1 -- 172.21.15.19:0/3275749608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f40b4600 con 0x7f81f415dac0 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81f259c640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f40d0ab0 0x7f81f4153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81f2d9d640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81f415dac0 0x7f81f41f7530 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f81ec034040 tx=0x7f81ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81eaffd640 1 -- 172.21.15.19:0/3275749608 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec036050 con 0x7f81f415dac0 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81eaffd640 1 -- 172.21.15.19:0/3275749608 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec042070 con 0x7f81f415dac0 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81f8b70640 1 -- 172.21.15.19:0/3275749608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f40b4320 con 0x7f81f415dac0 2024-04-25T10:05:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81eaffd640 1 -- 172.21.15.19:0/3275749608 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec009510 con 0x7f81f415dac0 2024-04-25T10:05:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.152+0000 7f81f8b70640 1 -- 172.21.15.19:0/3275749608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f40cbbd0 con 0x7f81f415dac0 2024-04-25T10:05:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.153+0000 7f81f8b70640 1 -- 172.21.15.19:0/3275749608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f4073a00 con 0x7f81f415dac0 2024-04-25T10:05:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.154+0000 7f81eaffd640 1 -- 172.21.15.19:0/3275749608 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81ec007bf0 con 0x7f81f415dac0 2024-04-25T10:05:33.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.154+0000 7f81eaffd640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81d406b4d0 0x7f81d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.154+0000 7f81eaffd640 1 -- 172.21.15.19:0/3275749608 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f81ec0aae50 con 0x7f81f415dac0 2024-04-25T10:05:33.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.154+0000 7f81f259c640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81d406b4d0 0x7f81d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.155+0000 7f81f259c640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81d406b4d0 0x7f81d406d990 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009a20 tx=0x7f81e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.158+0000 7f81eaffd640 1 -- 172.21.15.19:0/3275749608 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81ec079e40 con 0x7f81f415dac0 2024-04-25T10:05:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.336+0000 7f81f8b70640 1 -- 172.21.15.19:0/3275749608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f81f40b8a80 con 0x7f81f415dac0 2024-04-25T10:05:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.339+0000 7f81eaffd640 1 -- 172.21.15.19:0/3275749608 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f81ec079740 con 0x7f81f415dac0 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 -- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81d406b4d0 msgr2=0x7f81d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81d406b4d0 0x7f81d406d990 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009a20 tx=0x7f81e4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 -- 172.21.15.19:0/3275749608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81f415dac0 msgr2=0x7f81f41f7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81f415dac0 0x7f81f41f7530 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f81ec034040 tx=0x7f81ec004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 -- 172.21.15.19:0/3275749608 shutdown_connections 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f81d406b4d0 0x7f81d406d990 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81f415dac0 0x7f81f41f7530 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f81f40d1f30 0x7f81f40b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 --2- 172.21.15.19:0/3275749608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81f40d0ab0 0x7f81f4153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 -- 172.21.15.19:0/3275749608 >> 172.21.15.19:0/3275749608 conn(0x7f81f4061c30 msgr2=0x7f81f40c75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 -- 172.21.15.19:0/3275749608 shutdown_connections 2024-04-25T10:05:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.341+0000 7f81f8b70640 1 -- 172.21.15.19:0/3275749608 wait complete. 2024-04-25T10:05:33.357 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:33.358 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:33.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:33.444 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:33.445 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:33.445 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:33.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:33.461 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:05:33.461 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:33.502 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:33.502 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:33.502 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:33.522 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:33.522 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:05:33.522 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:33.559 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:33.560 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:05:33.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.657+0000 7fc45f276640 1 Processor -- start 2024-04-25T10:05:33.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.657+0000 7fc45f276640 1 -- start start 2024-04-25T10:05:33.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45f276640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4580c6df0 0x7fc4580d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45f276640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4580d1c00 0x7fc4580ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45f276640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45815dfe0 con 0x7fc45815dc00 2024-04-25T10:05:33.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45f276640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4580d2060 con 0x7fc4580d1c00 2024-04-25T10:05:33.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45f276640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580d00c0 con 0x7fc4580c6df0 2024-04-25T10:05:33.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45d012640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc45815dc00 0x7fc4580ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46646/0 (socket says 172.21.15.19:46646) 2024-04-25T10:05:33.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45d012640 1 -- 172.21.15.19:0/1796272432 learned_addr learned my addr 172.21.15.19:0/1796272432 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:33.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45d813640 1 --2- 172.21.15.19:0/1796272432 >> v2:172.21.15.112:3300/0 conn(0x7fc4580d1c00 0x7fc4580ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45c811640 1 --2- 172.21.15.19:0/1796272432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4580c6df0 0x7fc4580d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc44ffff640 1 -- 172.21.15.19:0/1796272432 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449056033 0 0) 0x7fc454001110 con 0x7fc45815dc00 2024-04-25T10:05:33.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc44ffff640 1 -- 172.21.15.19:0/1796272432 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc444003520 con 0x7fc45815dc00 2024-04-25T10:05:33.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45d813640 1 -- 172.21.15.19:0/1796272432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4580c6df0 msgr2=0x7fc4580d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45d813640 1 --2- 172.21.15.19:0/1796272432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4580c6df0 0x7fc4580d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45d813640 1 -- 172.21.15.19:0/1796272432 >> v1:172.21.15.19:6789/0 conn(0x7fc45815dc00 legacy=0x7fc4580ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc44ffff640 1 -- 172.21.15.19:0/1796272432 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3312115656 0 0) 0x7fc444003520 con 0x7fc45815dc00 2024-04-25T10:05:33.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.658+0000 7fc45d813640 1 -- 172.21.15.19:0/1796272432 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4580d03a0 con 0x7fc4580d1c00 2024-04-25T10:05:33.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.659+0000 7fc45d813640 1 --2- 172.21.15.19:0/1796272432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4580d1c00 0x7fc4580ccc40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc450009d90 tx=0x7fc45002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52d82d9991d41672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.659+0000 7fc44ffff640 1 -- 172.21.15.19:0/1796272432 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454001840 con 0x7fc4580d1c00 2024-04-25T10:05:33.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.659+0000 7fc44ffff640 1 -- 172.21.15.19:0/1796272432 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc450007540 con 0x7fc4580d1c00 2024-04-25T10:05:33.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.659+0000 7fc44ffff640 1 -- 172.21.15.19:0/1796272432 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4500078f0 con 0x7fc4580d1c00 2024-04-25T10:05:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.660+0000 7fc45f276640 1 -- 172.21.15.19:0/1796272432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4580d1c00 msgr2=0x7fc4580ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.660+0000 7fc45f276640 1 --2- 172.21.15.19:0/1796272432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4580d1c00 0x7fc4580ccc40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc450009d90 tx=0x7fc45002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.660+0000 7fc45f276640 1 -- 172.21.15.19:0/1796272432 shutdown_connections 2024-04-25T10:05:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.660+0000 7fc45f276640 1 --2- 172.21.15.19:0/1796272432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4580d1c00 0x7fc4580ccc40 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.660+0000 7fc45f276640 1 --2- 172.21.15.19:0/1796272432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4580c6df0 0x7fc4580d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.660+0000 7fc45f276640 1 -- 172.21.15.19:0/1796272432 >> 172.21.15.19:0/1796272432 conn(0x7fc45806f5d0 msgr2=0x7fc45806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.660+0000 7fc45f276640 1 -- 172.21.15.19:0/1796272432 shutdown_connections 2024-04-25T10:05:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.660+0000 7fc45f276640 1 -- 172.21.15.19:0/1796272432 wait complete. 2024-04-25T10:05:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.661+0000 7fc45f276640 1 Processor -- start 2024-04-25T10:05:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.661+0000 7fc45f276640 1 -- start start 2024-04-25T10:05:33.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45f276640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4580c6df0 0x7fc45814d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45f276640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4580d1c00 0x7fc4580b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45f276640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45815dc00 0x7fc4580b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45f276640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580cc130 con 0x7fc45815dc00 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45f276640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580cc720 con 0x7fc4580d1c00 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45f276640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580cfd90 con 0x7fc4580c6df0 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45d813640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45815dc00 0x7fc4580b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45d813640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45815dc00 0x7fc4580b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44924/0 (socket says 172.21.15.19:44924) 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45d813640 1 -- 172.21.15.19:0/2673123293 learned_addr learned my addr 172.21.15.19:0/2673123293 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45d012640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4580c6df0 0x7fc45814d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45c811640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4580d1c00 0x7fc4580b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45d813640 1 -- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4580c6df0 msgr2=0x7fc45814d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45d813640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4580c6df0 0x7fc45814d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45d813640 1 -- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4580d1c00 msgr2=0x7fc4580b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45d813640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4580d1c00 0x7fc4580b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45d813640 1 -- 172.21.15.19:0/2673123293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4580b5130 con 0x7fc45815dc00 2024-04-25T10:05:33.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.662+0000 7fc45d813640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45815dc00 0x7fc4580b1b00 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fc450009eb0 tx=0x7fc450004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.663+0000 7fc44dffb640 1 -- 172.21.15.19:0/2673123293 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450040070 con 0x7fc45815dc00 2024-04-25T10:05:33.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.663+0000 7fc44dffb640 1 -- 172.21.15.19:0/2673123293 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc450031c40 con 0x7fc45815dc00 2024-04-25T10:05:33.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.663+0000 7fc45f276640 1 -- 172.21.15.19:0/2673123293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4580b5410 con 0x7fc45815dc00 2024-04-25T10:05:33.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.663+0000 7fc44dffb640 1 -- 172.21.15.19:0/2673123293 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450009450 con 0x7fc45815dc00 2024-04-25T10:05:33.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.663+0000 7fc45f276640 1 -- 172.21.15.19:0/2673123293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4580b59d0 con 0x7fc45815dc00 2024-04-25T10:05:33.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.664+0000 7fc45f276640 1 -- 172.21.15.19:0/2673123293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4580cdd70 con 0x7fc45815dc00 2024-04-25T10:05:33.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.665+0000 7fc44dffb640 1 -- 172.21.15.19:0/2673123293 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc450039040 con 0x7fc45815dc00 2024-04-25T10:05:33.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.665+0000 7fc44dffb640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc43c06b4d0 0x7fc43c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.665+0000 7fc44dffb640 1 -- 172.21.15.19:0/2673123293 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc4500ab2c0 con 0x7fc45815dc00 2024-04-25T10:05:33.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.665+0000 7fc45d012640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc43c06b4d0 0x7fc43c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.666+0000 7fc45d012640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc43c06b4d0 0x7fc43c06d990 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc454005880 tx=0x7fc454001230 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.669+0000 7fc44dffb640 1 -- 172.21.15.19:0/2673123293 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4580cdd70 con 0x7fc45815dc00 2024-04-25T10:05:33.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.885+0000 7fc45f276640 1 -- 172.21.15.19:0/2673123293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc458073990 con 0x7fc45815dc00 2024-04-25T10:05:33.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.886+0000 7fc44dffb640 1 -- 172.21.15.19:0/2673123293 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+3705 (secure 0 0 0) 0x7fc450079e10 con 0x7fc45815dc00 2024-04-25T10:05:33.891 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:33.891 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":118,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8258,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/662861975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":662861975},{"type":"v1","addr":"172.21.15.19:6835","nonce":662861975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3439611393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3439611393},{"type":"v1","addr":"172.21.15.112:6836","nonce":3439611393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/721666508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":721666508},{"type":"v1","addr":"172.21.15.112:6837","nonce":721666508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":11478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1624678115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1624678115},{"type":"v1","addr":"172.21.15.19:6833","nonce":1624678115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[]} 2024-04-25T10:05:33.892 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 118 2024-04-25T10:05:33.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 -- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc43c06b4d0 msgr2=0x7fc43c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc43c06b4d0 0x7fc43c06d990 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc454005880 tx=0x7fc454001230 comp rx=0 tx=0).stop 2024-04-25T10:05:33.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 -- 172.21.15.19:0/2673123293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45815dc00 msgr2=0x7fc4580b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45815dc00 0x7fc4580b1b00 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fc450009eb0 tx=0x7fc450004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:33.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 -- 172.21.15.19:0/2673123293 shutdown_connections 2024-04-25T10:05:33.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc43c06b4d0 0x7fc43c06d990 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45815dc00 0x7fc4580b1b00 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4580d1c00 0x7fc4580b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 --2- 172.21.15.19:0/2673123293 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4580c6df0 0x7fc45814d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 -- 172.21.15.19:0/2673123293 >> 172.21.15.19:0/2673123293 conn(0x7fc45806f5d0 msgr2=0x7fc45815b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 -- 172.21.15.19:0/2673123293 shutdown_connections 2024-04-25T10:05:33.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:33.888+0000 7fc45f276640 1 -- 172.21.15.19:0/2673123293 wait complete. 2024-04-25T10:05:33.904 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:05:33.904 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:05:33.904 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:05:33.904 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:33.905 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:05:33.905 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:05:33.905 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:05:33.905 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:33.905 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:05:33.905 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:05:33.905 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:05:33.905 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:33.906 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:05:33.906 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:05:33.906 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:05:33.906 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:33.914 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:05:33.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:05:33.915 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:05:33.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:05:33.917 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:05:33.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:05:33.918 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:05:33.918 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:05:33.957 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:05:33.958 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:33.959 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:05:33.959 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:33.960 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:05:33.960 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:33.961 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:05:33.961 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:33.961 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:05:34.035 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T10:05:34.035 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:05:34.041 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-04-25T10:05:34.043 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-04-25T10:05:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.063+0000 7f9a40909640 1 Processor -- start 2024-04-25T10:05:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.063+0000 7f9a40909640 1 -- start start 2024-04-25T10:05:34.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a40909640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a3c1616f0 0x7f9a3c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a40909640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3c06c8d0 con 0x7f9a3c1610f0 2024-04-25T10:05:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a40909640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3c0c6960 con 0x7f9a3c0ce0a0 2024-04-25T10:05:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a40909640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c1614d0 con 0x7f9a3c1616f0 2024-04-25T10:05:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a3a59c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9a3c1610f0 0x7f9a3c06c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46670/0 (socket says 172.21.15.19:46670) 2024-04-25T10:05:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a3a59c640 1 -- 172.21.15.19:0/3984196066 learned_addr learned my addr 172.21.15.19:0/3984196066 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a39d9b640 1 --2- 172.21.15.19:0/3984196066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a3c1616f0 0x7f9a3c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a3959a640 1 -- 172.21.15.19:0/3984196066 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305377572 0 0) 0x7f9a3c06c8d0 con 0x7f9a3c1610f0 2024-04-25T10:05:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a3959a640 1 -- 172.21.15.19:0/3984196066 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a24003520 con 0x7f9a3c1610f0 2024-04-25T10:05:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a3959a640 1 -- 172.21.15.19:0/3984196066 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596010747 0 0) 0x7f9a3c0c6960 con 0x7f9a3c0ce0a0 2024-04-25T10:05:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a3959a640 1 -- 172.21.15.19:0/3984196066 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a3c06c8d0 con 0x7f9a3c0ce0a0 2024-04-25T10:05:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a39d9b640 1 -- 172.21.15.19:0/3984196066 >> v1:172.21.15.112:6789/0 conn(0x7f9a3c0ce0a0 legacy=0x7f9a3c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a39d9b640 1 -- 172.21.15.19:0/3984196066 >> v1:172.21.15.19:6789/0 conn(0x7f9a3c1610f0 legacy=0x7f9a3c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.064+0000 7f9a39d9b640 1 -- 172.21.15.19:0/3984196066 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c162260 con 0x7f9a3c1616f0 2024-04-25T10:05:34.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.065+0000 7f9a39d9b640 1 --2- 172.21.15.19:0/3984196066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a3c1616f0 0x7f9a3c161b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a300097d0 tx=0x7f9a3002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35a8fbab7ba23c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.065+0000 7f9a3959a640 1 -- 172.21.15.19:0/3984196066 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30037070 con 0x7f9a3c1616f0 2024-04-25T10:05:34.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.065+0000 7f9a3959a640 1 -- 172.21.15.19:0/3984196066 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a30032440 con 0x7f9a3c1616f0 2024-04-25T10:05:34.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.065+0000 7f9a3959a640 1 -- 172.21.15.19:0/3984196066 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30032810 con 0x7f9a3c1616f0 2024-04-25T10:05:34.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.066+0000 7f9a40909640 1 -- 172.21.15.19:0/3984196066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a3c1616f0 msgr2=0x7f9a3c161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.066+0000 7f9a40909640 1 --2- 172.21.15.19:0/3984196066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a3c1616f0 0x7f9a3c161b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a300097d0 tx=0x7f9a3002a210 comp rx=0 tx=0).stop 2024-04-25T10:05:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.066+0000 7f9a40909640 1 -- 172.21.15.19:0/3984196066 shutdown_connections 2024-04-25T10:05:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.066+0000 7f9a40909640 1 --2- 172.21.15.19:0/3984196066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a3c1616f0 0x7f9a3c161b50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.066+0000 7f9a40909640 1 -- 172.21.15.19:0/3984196066 >> 172.21.15.19:0/3984196066 conn(0x7f9a3c06f500 msgr2=0x7f9a3c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.066+0000 7f9a40909640 1 -- 172.21.15.19:0/3984196066 shutdown_connections 2024-04-25T10:05:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.066+0000 7f9a40909640 1 -- 172.21.15.19:0/3984196066 wait complete. 2024-04-25T10:05:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.067+0000 7f9a40909640 1 Processor -- start 2024-04-25T10:05:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.067+0000 7f9a40909640 1 -- start start 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a40909640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a3c0ce0a0 0x7f9a3c15c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a40909640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a3c1610f0 0x7f9a3c0b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a40909640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c1616f0 0x7f9a3c15b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a40909640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c0cfce0 con 0x7f9a3c1610f0 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a40909640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c0cfa00 con 0x7f9a3c1616f0 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a40909640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c0cf690 con 0x7f9a3c0ce0a0 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a39d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a3c1610f0 0x7f9a3c0b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a39d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a3c1610f0 0x7f9a3c0b7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44960/0 (socket says 172.21.15.19:44960) 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a39d9b640 1 -- 172.21.15.19:0/2395931580 learned_addr learned my addr 172.21.15.19:0/2395931580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a3a59c640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a3c0ce0a0 0x7f9a3c15c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a3ad9d640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c1616f0 0x7f9a3c15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a39d9b640 1 -- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a3c0ce0a0 msgr2=0x7f9a3c15c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a39d9b640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a3c0ce0a0 0x7f9a3c15c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a39d9b640 1 -- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c1616f0 msgr2=0x7f9a3c15b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a39d9b640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c1616f0 0x7f9a3c15b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a39d9b640 1 -- 172.21.15.19:0/2395931580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c15b860 con 0x7f9a3c1610f0 2024-04-25T10:05:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a39d9b640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a3c1610f0 0x7f9a3c0b7340 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f9a300098f0 tx=0x7f9a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.068+0000 7f9a2affd640 1 -- 172.21.15.19:0/2395931580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30041020 con 0x7f9a3c1610f0 2024-04-25T10:05:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.069+0000 7f9a2affd640 1 -- 172.21.15.19:0/2395931580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3003d6e0 con 0x7f9a3c1610f0 2024-04-25T10:05:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.069+0000 7f9a40909640 1 -- 172.21.15.19:0/2395931580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3c15bb40 con 0x7f9a3c1610f0 2024-04-25T10:05:34.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.069+0000 7f9a2affd640 1 -- 172.21.15.19:0/2395931580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30037040 con 0x7f9a3c1610f0 2024-04-25T10:05:34.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.069+0000 7f9a40909640 1 -- 172.21.15.19:0/2395931580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c15c100 con 0x7f9a3c1610f0 2024-04-25T10:05:34.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.069+0000 7f9a40909640 1 -- 172.21.15.19:0/2395931580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3c06afe0 con 0x7f9a3c1610f0 2024-04-25T10:05:34.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.070+0000 7f9a2affd640 1 -- 172.21.15.19:0/2395931580 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a30044020 con 0x7f9a3c1610f0 2024-04-25T10:05:34.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.070+0000 7f9a2affd640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a1c06b4d0 0x7f9a1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.070+0000 7f9a2affd640 1 -- 172.21.15.19:0/2395931580 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9a300aa5b0 con 0x7f9a3c1610f0 2024-04-25T10:05:34.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.071+0000 7f9a3a59c640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a1c06b4d0 0x7f9a1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.071+0000 7f9a3a59c640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a1c06b4d0 0x7f9a1c06d990 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c000f60 tx=0x7f9a2c007d70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.075+0000 7f9a2affd640 1 -- 172.21.15.19:0/2395931580 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a3c06afe0 con 0x7f9a3c1610f0 2024-04-25T10:05:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:34.266+0000 7f9a40909640 1 -- 172.21.15.19:0/2395931580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9a3c073a00 con 0x7f9a3c1610f0 2024-04-25T10:05:35.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.035+0000 7f9a2affd640 1 -- 172.21.15.19:0/2395931580 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v183) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9a3004b030 con 0x7f9a3c1610f0 2024-04-25T10:05:35.041 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T10:05:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 -- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a1c06b4d0 msgr2=0x7f9a1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a1c06b4d0 0x7f9a1c06d990 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c000f60 tx=0x7f9a2c007d70 comp rx=0 tx=0).stop 2024-04-25T10:05:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 -- 172.21.15.19:0/2395931580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a3c1610f0 msgr2=0x7f9a3c0b7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a3c1610f0 0x7f9a3c0b7340 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f9a300098f0 tx=0x7f9a300046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 -- 172.21.15.19:0/2395931580 shutdown_connections 2024-04-25T10:05:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9a1c06b4d0 0x7f9a1c06d990 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a3c1616f0 0x7f9a3c15b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a3c1610f0 0x7f9a3c0b7340 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 --2- 172.21.15.19:0/2395931580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9a3c0ce0a0 0x7f9a3c15c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 -- 172.21.15.19:0/2395931580 >> 172.21.15.19:0/2395931580 conn(0x7f9a3c06f500 msgr2=0x7f9a3c070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 -- 172.21.15.19:0/2395931580 shutdown_connections 2024-04-25T10:05:35.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.039+0000 7f9a40909640 1 -- 172.21.15.19:0/2395931580 wait complete. 2024-04-25T10:05:35.061 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:05:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.163+0000 7fb32bd5c640 1 Processor -- start 2024-04-25T10:05:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.163+0000 7fb32bd5c640 1 -- start start 2024-04-25T10:05:35.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb32bd5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb32413d190 0x7fb3240cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb32bd5c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb32415b320 0x7fb3240cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb32bd5c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb324071ed0 con 0x7fb324071230 2024-04-25T10:05:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb32bd5c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32413d570 con 0x7fb32415b320 2024-04-25T10:05:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb32bd5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32415a840 con 0x7fb32413d190 2024-04-25T10:05:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb3292f7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb32415b320 0x7fb3240cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb3292f7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb32415b320 0x7fb3240cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44972/0 (socket says 172.21.15.19:44972) 2024-04-25T10:05:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb3292f7640 1 -- 172.21.15.19:0/4253765344 learned_addr learned my addr 172.21.15.19:0/4253765344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb329af8640 1 --2- 172.21.15.19:0/4253765344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb32413d190 0x7fb3240cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb3292f7640 1 -- 172.21.15.19:0/4253765344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb32413d190 msgr2=0x7fb3240cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb3292f7640 1 --2- 172.21.15.19:0/4253765344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb32413d190 0x7fb3240cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb3292f7640 1 -- 172.21.15.19:0/4253765344 >> v1:172.21.15.112:6789/0 conn(0x7fb324071230 legacy=0x7fb3240716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.164+0000 7fb3292f7640 1 -- 172.21.15.19:0/4253765344 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32415ab20 con 0x7fb32415b320 2024-04-25T10:05:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.165+0000 7fb3292f7640 1 --2- 172.21.15.19:0/4253765344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb32415b320 0x7fb3240cc4f0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fb31400bb10 tx=0x7fb31402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69e4f71cc6e1c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.165+0000 7fb328af6640 1 -- 172.21.15.19:0/4253765344 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314032790 con 0x7fb32415b320 2024-04-25T10:05:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.165+0000 7fb328af6640 1 -- 172.21.15.19:0/4253765344 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb314038070 con 0x7fb32415b320 2024-04-25T10:05:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.165+0000 7fb328af6640 1 -- 172.21.15.19:0/4253765344 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314032b50 con 0x7fb32415b320 2024-04-25T10:05:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.166+0000 7fb32bd5c640 1 -- 172.21.15.19:0/4253765344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb32415b320 msgr2=0x7fb3240cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.166+0000 7fb32bd5c640 1 --2- 172.21.15.19:0/4253765344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb32415b320 0x7fb3240cc4f0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fb31400bb10 tx=0x7fb31402c890 comp rx=0 tx=0).stop 2024-04-25T10:05:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.166+0000 7fb32bd5c640 1 -- 172.21.15.19:0/4253765344 shutdown_connections 2024-04-25T10:05:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.166+0000 7fb32bd5c640 1 --2- 172.21.15.19:0/4253765344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb32415b320 0x7fb3240cc4f0 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.166+0000 7fb32bd5c640 1 --2- 172.21.15.19:0/4253765344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb32413d190 0x7fb3240cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.166+0000 7fb32bd5c640 1 -- 172.21.15.19:0/4253765344 >> 172.21.15.19:0/4253765344 conn(0x7fb32406abe0 msgr2=0x7fb32406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.166+0000 7fb32bd5c640 1 -- 172.21.15.19:0/4253765344 shutdown_connections 2024-04-25T10:05:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.166+0000 7fb32bd5c640 1 -- 172.21.15.19:0/4253765344 wait complete. 2024-04-25T10:05:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.167+0000 7fb32bd5c640 1 Processor -- start 2024-04-25T10:05:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.167+0000 7fb32bd5c640 1 -- start start 2024-04-25T10:05:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32bd5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb324071230 0x7fb324139fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32bd5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb32413d190 0x7fb32415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32bd5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb32415b320 0x7fb3240b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32bd5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240cf7a0 con 0x7fb32415b320 2024-04-25T10:05:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32bd5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240cf620 con 0x7fb324071230 2024-04-25T10:05:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32bd5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240cf920 con 0x7fb32413d190 2024-04-25T10:05:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32a2f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb32415b320 0x7fb3240b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32a2f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb32415b320 0x7fb3240b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44974/0 (socket says 172.21.15.19:44974) 2024-04-25T10:05:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32a2f9640 1 -- 172.21.15.19:0/489459809 learned_addr learned my addr 172.21.15.19:0/489459809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb3292f7640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb32413d190 0x7fb32415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb329af8640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb324071230 0x7fb324139fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32a2f9640 1 -- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb32413d190 msgr2=0x7fb32415b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32a2f9640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb32413d190 0x7fb32415b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32a2f9640 1 -- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb324071230 msgr2=0x7fb324139fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32a2f9640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb324071230 0x7fb324139fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32a2f9640 1 -- 172.21.15.19:0/489459809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32415c4a0 con 0x7fb32415b320 2024-04-25T10:05:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb3292f7640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb32413d190 0x7fb32415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.168+0000 7fb32a2f9640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb32415b320 0x7fb3240b3900 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00c7d0 tx=0x7fb31c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.169+0000 7fb31a7fc640 1 -- 172.21.15.19:0/489459809 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c003b70 con 0x7fb32415b320 2024-04-25T10:05:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.169+0000 7fb31a7fc640 1 -- 172.21.15.19:0/489459809 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c003d30 con 0x7fb32415b320 2024-04-25T10:05:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.169+0000 7fb32bd5c640 1 -- 172.21.15.19:0/489459809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32415c780 con 0x7fb32415b320 2024-04-25T10:05:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.169+0000 7fb31a7fc640 1 -- 172.21.15.19:0/489459809 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c00aa90 con 0x7fb32415b320 2024-04-25T10:05:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.169+0000 7fb32bd5c640 1 -- 172.21.15.19:0/489459809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32415cd40 con 0x7fb32415b320 2024-04-25T10:05:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.170+0000 7fb32bd5c640 1 -- 172.21.15.19:0/489459809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb324073990 con 0x7fb32415b320 2024-04-25T10:05:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.170+0000 7fb31a7fc640 1 -- 172.21.15.19:0/489459809 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb31c00ac30 con 0x7fb32415b320 2024-04-25T10:05:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.171+0000 7fb31a7fc640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb30806b4d0 0x7fb30806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.171+0000 7fb31a7fc640 1 -- 172.21.15.19:0/489459809 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb31c089df0 con 0x7fb32415b320 2024-04-25T10:05:35.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.171+0000 7fb329af8640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb30806b4d0 0x7fb30806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.171+0000 7fb329af8640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb30806b4d0 0x7fb30806d990 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb320009b40 tx=0x7fb320009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.175+0000 7fb31a7fc640 1 -- 172.21.15.19:0/489459809 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb31c0587e0 con 0x7fb32415b320 2024-04-25T10:05:35.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.428+0000 7fb32bd5c640 1 -- 172.21.15.19:0/489459809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb3240b4580 con 0x7fb32415b320 2024-04-25T10:05:35.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.428+0000 7fb31a7fc640 1 -- 172.21.15.19:0/489459809 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb31c05c880 con 0x7fb32415b320 2024-04-25T10:05:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:35.434 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:05:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T10:05:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T10:05:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm75A+DBAAFoAwb0Ek4VUgQwPT8Uws5g==", 2024-04-25T10:05:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:35.434 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmCkjoFxAAsNrZLz4ZRIap93lD5mAfWg==", 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmrHAxExAArWQCjBtrkjqTxHYf23aAgQ==", 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T10:05:35.436 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm7VbUHhAAll0pEybfAxZGkdHJnyorxw==", 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNJypmyzQHBRAA55lFR+1t21TVoexf3TD+wQ==", 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T10:05:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQJypm8A21AxAAtA6BrfFCVcZegpt3fex0xQ==", 2024-04-25T10:05:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTJypmRH4QAhAAGHfuiPFSfBdUhfELxVQOcw==", 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDWJypmFfOzABAANdlmvhw4UoBt8TdOaZvZlA==", 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:35.439 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDcJypmok4pBxAATO5O8oCcTVC7nH3gcMfr1Q==", 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfJypmGTsYBRAAdDQ94XJIjiLrr9+BhG5X6Q==", 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiJypmuIC+AhAAMTqj8lDNixOcxs4JaeGXPA==", 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.441 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlJypmfnugABAAjAZNiYzwXw/ujYylG07o1w==", 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA==", 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T10:05:35.443 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm+J5SKxAAFkGsf269GJE2ulsqa+PqCg==", 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmt59IKBAA0H2cYqgqJaMBLd9vsbKnhA==", 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:35.444 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm3UXBMBAA/y5jQmDhYuwMCpFxhNCvPw==", 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDIJypmW1z/KhAAdHXkCthhUxJ/fYwVdpUuzw==", 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.446 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypm9MQdCRAAnvnn1qfHH6Ya9EqYrgquig==", 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDJJypmcDEsBBAAuVjS/mGd3enrXlyONy02Rg==", 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T10:05:35.447 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T10:05:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:35.448 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T10:05:35.448 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:05:35.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 -- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb30806b4d0 msgr2=0x7fb30806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb30806b4d0 0x7fb30806d990 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb320009b40 tx=0x7fb320009190 comp rx=0 tx=0).stop 2024-04-25T10:05:35.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 -- 172.21.15.19:0/489459809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb32415b320 msgr2=0x7fb3240b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb32415b320 0x7fb3240b3900 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00c7d0 tx=0x7fb31c00cc90 comp rx=0 tx=0).stop 2024-04-25T10:05:35.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 -- 172.21.15.19:0/489459809 shutdown_connections 2024-04-25T10:05:35.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb30806b4d0 0x7fb30806d990 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb32415b320 0x7fb3240b3900 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb32413d190 0x7fb32415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 --2- 172.21.15.19:0/489459809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb324071230 0x7fb324139fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 -- 172.21.15.19:0/489459809 >> 172.21.15.19:0/489459809 conn(0x7fb32406abe0 msgr2=0x7fb3240716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 -- 172.21.15.19:0/489459809 shutdown_connections 2024-04-25T10:05:35.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.430+0000 7fb32bd5c640 1 -- 172.21.15.19:0/489459809 wait complete. 2024-04-25T10:05:35.451 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.546+0000 7ff77b9f4640 1 Processor -- start 2024-04-25T10:05:35.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.546+0000 7ff77b9f4640 1 -- start start 2024-04-25T10:05:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.547+0000 7ff77b9f4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff77415db70 0x7ff774162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.547+0000 7ff77b9f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff7740b5c30 0x7ff7740d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.547+0000 7ff77b9f4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff774162690 con 0x7ff7740b6cb0 2024-04-25T10:05:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.547+0000 7ff77b9f4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7740ceee0 con 0x7ff77415db70 2024-04-25T10:05:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.547+0000 7ff77b9f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740d1d10 con 0x7ff7740b5c30 2024-04-25T10:05:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.547+0000 7ff778f8f640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff7740b6cb0 0x7ff7740b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46682/0 (socket says 172.21.15.19:46682) 2024-04-25T10:05:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.547+0000 7ff778f8f640 1 -- 172.21.15.19:0/3703846189 learned_addr learned my addr 172.21.15.19:0/3703846189 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:35.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.547+0000 7ff779f91640 1 --2- 172.21.15.19:0/3703846189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff7740b5c30 0x7ff7740d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.547+0000 7ff779790640 1 --2- 172.21.15.19:0/3703846189 >> v2:172.21.15.112:3300/0 conn(0x7ff77415db70 0x7ff774162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.548+0000 7ff773fff640 1 -- 172.21.15.19:0/3703846189 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190881731 0 0) 0x7ff774162690 con 0x7ff7740b6cb0 2024-04-25T10:05:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.548+0000 7ff773fff640 1 -- 172.21.15.19:0/3703846189 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff75c003520 con 0x7ff7740b6cb0 2024-04-25T10:05:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.548+0000 7ff779f91640 1 -- 172.21.15.19:0/3703846189 >> v2:172.21.15.112:3300/0 conn(0x7ff77415db70 msgr2=0x7ff774162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.548+0000 7ff779f91640 1 --2- 172.21.15.19:0/3703846189 >> v2:172.21.15.112:3300/0 conn(0x7ff77415db70 0x7ff774162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.548+0000 7ff779f91640 1 -- 172.21.15.19:0/3703846189 >> v1:172.21.15.19:6789/0 conn(0x7ff7740b6cb0 legacy=0x7ff7740b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.548+0000 7ff779f91640 1 -- 172.21.15.19:0/3703846189 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740d1ff0 con 0x7ff7740b5c30 2024-04-25T10:05:35.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.548+0000 7ff779f91640 1 --2- 172.21.15.19:0/3703846189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff7740b5c30 0x7ff7740d27f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff764009d90 tx=0x7ff76402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b652fef2ac76d374 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.549+0000 7ff773fff640 1 -- 172.21.15.19:0/3703846189 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764007740 con 0x7ff7740b5c30 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.549+0000 7ff773fff640 1 -- 172.21.15.19:0/3703846189 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7640078e0 con 0x7ff7740b5c30 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.549+0000 7ff773fff640 1 -- 172.21.15.19:0/3703846189 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764007c90 con 0x7ff7740b5c30 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.549+0000 7ff77b9f4640 1 -- 172.21.15.19:0/3703846189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff7740b5c30 msgr2=0x7ff7740d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.549+0000 7ff77b9f4640 1 --2- 172.21.15.19:0/3703846189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff7740b5c30 0x7ff7740d27f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff764009d90 tx=0x7ff76402a660 comp rx=0 tx=0).stop 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.549+0000 7ff77b9f4640 1 -- 172.21.15.19:0/3703846189 shutdown_connections 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.549+0000 7ff77b9f4640 1 --2- 172.21.15.19:0/3703846189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff7740b5c30 0x7ff7740d27f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.549+0000 7ff77b9f4640 1 --2- 172.21.15.19:0/3703846189 >> v2:172.21.15.112:3300/0 conn(0x7ff77415db70 0x7ff774162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.549+0000 7ff77b9f4640 1 -- 172.21.15.19:0/3703846189 >> 172.21.15.19:0/3703846189 conn(0x7ff7740d0520 msgr2=0x7ff7740d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.549+0000 7ff77b9f4640 1 -- 172.21.15.19:0/3703846189 shutdown_connections 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.549+0000 7ff77b9f4640 1 -- 172.21.15.19:0/3703846189 wait complete. 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.550+0000 7ff77b9f4640 1 Processor -- start 2024-04-25T10:05:35.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.550+0000 7ff77b9f4640 1 -- start start 2024-04-25T10:05:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.551+0000 7ff77b9f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7740b5c30 0x7ff7740cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.551+0000 7ff77b9f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7740b6cb0 0x7ff7741f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.551+0000 7ff77b9f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff77415db70 0x7ff7741f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.551+0000 7ff77b9f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740ce530 con 0x7ff7740b5c30 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.551+0000 7ff77b9f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77415df50 con 0x7ff7740b6cb0 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.551+0000 7ff77b9f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740cf1c0 con 0x7ff77415db70 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.551+0000 7ff779790640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7740b5c30 0x7ff7740cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.551+0000 7ff779790640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7740b5c30 0x7ff7740cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44990/0 (socket says 172.21.15.19:44990) 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.551+0000 7ff779790640 1 -- 172.21.15.19:0/4199078634 learned_addr learned my addr 172.21.15.19:0/4199078634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.552+0000 7ff779790640 1 -- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff77415db70 msgr2=0x7ff7741f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.552+0000 7ff779790640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff77415db70 0x7ff7741f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.552+0000 7ff779790640 1 -- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7740b6cb0 msgr2=0x7ff7741f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.552+0000 7ff779790640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7740b6cb0 0x7ff7741f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.552+0000 7ff779790640 1 -- 172.21.15.19:0/4199078634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7741f7730 con 0x7ff7740b5c30 2024-04-25T10:05:35.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.552+0000 7ff779790640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7740b5c30 0x7ff7740cda50 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00e870 tx=0x7ff76c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.552+0000 7ff771ffb640 1 -- 172.21.15.19:0/4199078634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c017070 con 0x7ff7740b5c30 2024-04-25T10:05:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.552+0000 7ff771ffb640 1 -- 172.21.15.19:0/4199078634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c00c940 con 0x7ff7740b5c30 2024-04-25T10:05:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.552+0000 7ff77b9f4640 1 -- 172.21.15.19:0/4199078634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7741f7a10 con 0x7ff7740b5c30 2024-04-25T10:05:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.552+0000 7ff771ffb640 1 -- 172.21.15.19:0/4199078634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c00cd30 con 0x7ff7740b5c30 2024-04-25T10:05:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.552+0000 7ff77b9f4640 1 -- 172.21.15.19:0/4199078634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7741f3c90 con 0x7ff7740b5c30 2024-04-25T10:05:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.553+0000 7ff77b9f4640 1 -- 172.21.15.19:0/4199078634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff774073990 con 0x7ff7740b5c30 2024-04-25T10:05:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.553+0000 7ff771ffb640 1 -- 172.21.15.19:0/4199078634 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff76c01a070 con 0x7ff7740b5c30 2024-04-25T10:05:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.554+0000 7ff771ffb640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff76006b4d0 0x7ff76006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.554+0000 7ff771ffb640 1 -- 172.21.15.19:0/4199078634 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff76c08a230 con 0x7ff7740b5c30 2024-04-25T10:05:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.554+0000 7ff778f8f640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff76006b4d0 0x7ff76006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.555+0000 7ff778f8f640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff76006b4d0 0x7ff76006d990 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff768000f60 tx=0x7ff7680078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.558+0000 7ff771ffb640 1 -- 172.21.15.19:0/4199078634 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff76c059300 con 0x7ff7740b5c30 2024-04-25T10:05:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.738+0000 7ff77b9f4640 1 -- 172.21.15.19:0/4199078634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7740d1820 con 0x7ff7740b5c30 2024-04-25T10:05:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.739+0000 7ff771ffb640 1 -- 172.21.15.19:0/4199078634 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+13162 (secure 0 0 0) 0x7ff76c058c00 con 0x7ff7740b5c30 2024-04-25T10:05:35.745 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:35.745 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":185,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:35.047913+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 -- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff76006b4d0 msgr2=0x7ff76006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff76006b4d0 0x7ff76006d990 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff768000f60 tx=0x7ff7680078f0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 -- 172.21.15.19:0/4199078634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7740b5c30 msgr2=0x7ff7740cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7740b5c30 0x7ff7740cda50 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00e870 tx=0x7ff76c00ed30 comp rx=0 tx=0).stop 2024-04-25T10:05:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 -- 172.21.15.19:0/4199078634 shutdown_connections 2024-04-25T10:05:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff76006b4d0 0x7ff76006d990 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff77415db70 0x7ff7741f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7740b6cb0 0x7ff7741f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 --2- 172.21.15.19:0/4199078634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7740b5c30 0x7ff7740cda50 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 -- 172.21.15.19:0/4199078634 >> 172.21.15.19:0/4199078634 conn(0x7ff7740d0520 msgr2=0x7ff7740d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 -- 172.21.15.19:0/4199078634 shutdown_connections 2024-04-25T10:05:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.741+0000 7ff77b9f4640 1 -- 172.21.15.19:0/4199078634 wait complete. 2024-04-25T10:05:35.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:35.758 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:35.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.856+0000 7fdca002a640 1 Processor -- start 2024-04-25T10:05:35.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.857+0000 7fdca002a640 1 -- start start 2024-04-25T10:05:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.857+0000 7fdca002a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc98161240 0x7fdc980b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.857+0000 7fdca002a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdc980b54a0 0x7fdc980b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.857+0000 7fdca002a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc980cf1d0 con 0x7fdc98161890 2024-04-25T10:05:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.857+0000 7fdca002a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc980b7120 con 0x7fdc980b54a0 2024-04-25T10:05:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.857+0000 7fdca002a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc98161620 con 0x7fdc98161240 2024-04-25T10:05:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.858+0000 7fdc9e5c7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdc980b54a0 0x7fdc980b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.858+0000 7fdc9d5c5640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fdc98161890 0x7fdc980ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:47540/0 (socket says 172.21.15.19:47540) 2024-04-25T10:05:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.858+0000 7fdc9d5c5640 1 -- 172.21.15.19:0/1251262187 learned_addr learned my addr 172.21.15.19:0/1251262187 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.858+0000 7fdc9ddc6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc98161240 0x7fdc980b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.858+0000 7fdc9e5c7640 1 -- 172.21.15.19:0/1251262187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc98161240 msgr2=0x7fdc980b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.858+0000 7fdc9e5c7640 1 --2- 172.21.15.19:0/1251262187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc98161240 0x7fdc980b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.858+0000 7fdc9e5c7640 1 -- 172.21.15.19:0/1251262187 >> v1:172.21.15.112:6789/0 conn(0x7fdc98161890 legacy=0x7fdc980ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.858+0000 7fdc9e5c7640 1 -- 172.21.15.19:0/1251262187 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98161e00 con 0x7fdc980b54a0 2024-04-25T10:05:35.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdc9e5c7640 1 --2- 172.21.15.19:0/1251262187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc980b54a0 0x7fdc980b5930 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009d90 tx=0x7fdc9002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c2259f335f80696 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdc9cdc4640 1 -- 172.21.15.19:0/1251262187 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90007740 con 0x7fdc980b54a0 2024-04-25T10:05:35.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdc9cdc4640 1 -- 172.21.15.19:0/1251262187 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc900078e0 con 0x7fdc980b54a0 2024-04-25T10:05:35.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdc9cdc4640 1 -- 172.21.15.19:0/1251262187 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90007cb0 con 0x7fdc980b54a0 2024-04-25T10:05:35.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdca002a640 1 -- 172.21.15.19:0/1251262187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc980b54a0 msgr2=0x7fdc980b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdca002a640 1 --2- 172.21.15.19:0/1251262187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc980b54a0 0x7fdc980b5930 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009d90 tx=0x7fdc9002a660 comp rx=0 tx=0).stop 2024-04-25T10:05:35.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdca002a640 1 -- 172.21.15.19:0/1251262187 shutdown_connections 2024-04-25T10:05:35.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdca002a640 1 --2- 172.21.15.19:0/1251262187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc980b54a0 0x7fdc980b5930 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdca002a640 1 --2- 172.21.15.19:0/1251262187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc98161240 0x7fdc980b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdca002a640 1 -- 172.21.15.19:0/1251262187 >> 172.21.15.19:0/1251262187 conn(0x7fdc980d04a0 msgr2=0x7fdc980d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdca002a640 1 -- 172.21.15.19:0/1251262187 shutdown_connections 2024-04-25T10:05:35.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.859+0000 7fdca002a640 1 -- 172.21.15.19:0/1251262187 wait complete. 2024-04-25T10:05:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.861+0000 7fdca002a640 1 Processor -- start 2024-04-25T10:05:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.861+0000 7fdca002a640 1 -- start start 2024-04-25T10:05:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.861+0000 7fdca002a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc980b54a0 0x7fdc9815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.861+0000 7fdca002a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc98161240 0x7fdc980b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.861+0000 7fdca002a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc98161890 0x7fdc980b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.861+0000 7fdca002a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981626a0 con 0x7fdc980b54a0 2024-04-25T10:05:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.861+0000 7fdca002a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981623c0 con 0x7fdc98161240 2024-04-25T10:05:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.861+0000 7fdca002a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980d15f0 con 0x7fdc98161890 2024-04-25T10:05:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.861+0000 7fdc9ddc6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc980b54a0 0x7fdc9815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.861+0000 7fdc9ddc6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc980b54a0 0x7fdc9815ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44998/0 (socket says 172.21.15.19:44998) 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.861+0000 7fdc9ddc6640 1 -- 172.21.15.19:0/864456010 learned_addr learned my addr 172.21.15.19:0/864456010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdc9d5c5640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc98161240 0x7fdc980b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdc9e5c7640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc98161890 0x7fdc980b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdc9ddc6640 1 -- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc98161890 msgr2=0x7fdc980b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdc9ddc6640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc98161890 0x7fdc980b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdc9ddc6640 1 -- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc98161240 msgr2=0x7fdc980b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdc9ddc6640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc98161240 0x7fdc980b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdc9ddc6640 1 -- 172.21.15.19:0/864456010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98160320 con 0x7fdc980b54a0 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdc9ddc6640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc980b54a0 0x7fdc9815ad00 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00e9b0 tx=0x7fdc8c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdc8a7fc640 1 -- 172.21.15.19:0/864456010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c017070 con 0x7fdc980b54a0 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdc8a7fc640 1 -- 172.21.15.19:0/864456010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c00c9a0 con 0x7fdc980b54a0 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdca002a640 1 -- 172.21.15.19:0/864456010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc980c7730 con 0x7fdc980b54a0 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdc8a7fc640 1 -- 172.21.15.19:0/864456010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00cd90 con 0x7fdc980b54a0 2024-04-25T10:05:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.862+0000 7fdca002a640 1 -- 172.21.15.19:0/864456010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9815b240 con 0x7fdc980b54a0 2024-04-25T10:05:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.863+0000 7fdca002a640 1 -- 172.21.15.19:0/864456010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc980c7550 con 0x7fdc980b54a0 2024-04-25T10:05:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.864+0000 7fdc8a7fc640 1 -- 172.21.15.19:0/864456010 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc8c01a070 con 0x7fdc980b54a0 2024-04-25T10:05:35.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.864+0000 7fdc8a7fc640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.864+0000 7fdc8a7fc640 1 -- 172.21.15.19:0/864456010 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fdc8c08a390 con 0x7fdc980b54a0 2024-04-25T10:05:35.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.864+0000 7fdc9d5c5640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.865+0000 7fdc9d5c5640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fdc980b1bd0 tx=0x7fdc94009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:35.868+0000 7fdc8a7fc640 1 -- 172.21.15.19:0/864456010 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc980c7550 con 0x7fdc980b54a0 2024-04-25T10:05:36.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.049+0000 7fdca002a640 1 -- 172.21.15.19:0/864456010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc98073a00 con 0x7fdc980b54a0 2024-04-25T10:05:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.049+0000 7fdc8a7fc640 1 -- 172.21.15.19:0/864456010 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v185) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc8c059460 con 0x7fdc980b54a0 2024-04-25T10:05:36.055 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:05:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 -- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdc7c06b4d0 msgr2=0x7fdc7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fdc980b1bd0 tx=0x7fdc94009040 comp rx=0 tx=0).stop 2024-04-25T10:05:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 -- 172.21.15.19:0/864456010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc980b54a0 msgr2=0x7fdc9815ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc980b54a0 0x7fdc9815ad00 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00e9b0 tx=0x7fdc8c00ee70 comp rx=0 tx=0).stop 2024-04-25T10:05:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 -- 172.21.15.19:0/864456010 shutdown_connections 2024-04-25T10:05:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc98161890 0x7fdc980b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc98161240 0x7fdc980b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 --2- 172.21.15.19:0/864456010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc980b54a0 0x7fdc9815ad00 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 -- 172.21.15.19:0/864456010 >> 172.21.15.19:0/864456010 conn(0x7fdc980d04a0 msgr2=0x7fdc980d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 -- 172.21.15.19:0/864456010 shutdown_connections 2024-04-25T10:05:36.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.051+0000 7fdca002a640 1 -- 172.21.15.19:0/864456010 wait complete. 2024-04-25T10:05:36.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.164+0000 7f5dfb170640 1 Processor -- start 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.164+0000 7f5dfb170640 1 -- start start 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5dfb170640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5df415da70 0x7f5df40d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5dfb170640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5df40c7730 0x7f5df40ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5dfb170640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df40c8030 con 0x7f5df40ce700 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5dfb170640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df40ceb30 con 0x7f5df40c7730 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5dfb170640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df40c8310 con 0x7f5df415da70 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5df3fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5df40c7730 0x7f5df40ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5df970d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5df40ce700 0x7f5df4070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46690/0 (socket says 172.21.15.19:46690) 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5df970d640 1 -- 172.21.15.19:0/1211068778 learned_addr learned my addr 172.21.15.19:0/1211068778 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5df8f0c640 1 --2- 172.21.15.19:0/1211068778 >> v2:172.21.15.112:3300/0 conn(0x7f5df415da70 0x7f5df40d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5df37fe640 1 -- 172.21.15.19:0/1211068778 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296022697 0 0) 0x7f5df40c8030 con 0x7f5df40ce700 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5df37fe640 1 -- 172.21.15.19:0/1211068778 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de0003520 con 0x7f5df40ce700 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5df3fff640 1 -- 172.21.15.19:0/1211068778 >> v2:172.21.15.112:3300/0 conn(0x7f5df415da70 msgr2=0x7f5df40d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5df3fff640 1 --2- 172.21.15.19:0/1211068778 >> v2:172.21.15.112:3300/0 conn(0x7f5df415da70 0x7f5df40d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5df3fff640 1 -- 172.21.15.19:0/1211068778 >> v1:172.21.15.19:6789/0 conn(0x7f5df40ce700 legacy=0x7f5df4070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5df3fff640 1 -- 172.21.15.19:0/1211068778 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df40c85f0 con 0x7f5df40c7730 2024-04-25T10:05:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.165+0000 7f5df8f0c640 1 --2- 172.21.15.19:0/1211068778 >> v2:172.21.15.112:3300/0 conn(0x7f5df415da70 0x7f5df40d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.166+0000 7f5df3fff640 1 --2- 172.21.15.19:0/1211068778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df40c7730 0x7f5df40ce1c0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f5df406b900 tx=0x7f5ddc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89482be39d7bf08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.166+0000 7f5df37fe640 1 -- 172.21.15.19:0/1211068778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc037070 con 0x7f5df40c7730 2024-04-25T10:05:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.166+0000 7f5df37fe640 1 -- 172.21.15.19:0/1211068778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ddc032440 con 0x7f5df40c7730 2024-04-25T10:05:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.166+0000 7f5df37fe640 1 -- 172.21.15.19:0/1211068778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc032810 con 0x7f5df40c7730 2024-04-25T10:05:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.167+0000 7f5dfb170640 1 -- 172.21.15.19:0/1211068778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df40c7730 msgr2=0x7f5df40ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.167+0000 7f5dfb170640 1 --2- 172.21.15.19:0/1211068778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df40c7730 0x7f5df40ce1c0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f5df406b900 tx=0x7f5ddc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.167+0000 7f5dfb170640 1 -- 172.21.15.19:0/1211068778 shutdown_connections 2024-04-25T10:05:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.167+0000 7f5dfb170640 1 --2- 172.21.15.19:0/1211068778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df40c7730 0x7f5df40ce1c0 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.167+0000 7f5dfb170640 1 --2- 172.21.15.19:0/1211068778 >> v2:172.21.15.112:3300/0 conn(0x7f5df415da70 0x7f5df40d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.167+0000 7f5dfb170640 1 -- 172.21.15.19:0/1211068778 >> 172.21.15.19:0/1211068778 conn(0x7f5df40b33e0 msgr2=0x7f5df40b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.167+0000 7f5dfb170640 1 -- 172.21.15.19:0/1211068778 shutdown_connections 2024-04-25T10:05:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.167+0000 7f5dfb170640 1 -- 172.21.15.19:0/1211068778 wait complete. 2024-04-25T10:05:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.168+0000 7f5dfb170640 1 Processor -- start 2024-04-25T10:05:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.168+0000 7f5dfb170640 1 -- start start 2024-04-25T10:05:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5dfb170640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df40c7730 0x7f5df4156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5dfb170640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df40ce700 0x7f5df40c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5dfb170640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df415da70 0x7f5df4160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5dfb170640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df415de50 con 0x7f5df415da70 2024-04-25T10:05:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5dfb170640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df406be10 con 0x7f5df40c7730 2024-04-25T10:05:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5dfb170640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df40c7bc0 con 0x7f5df40ce700 2024-04-25T10:05:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5df970d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df415da70 0x7f5df4160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5df970d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df415da70 0x7f5df4160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45002/0 (socket says 172.21.15.19:45002) 2024-04-25T10:05:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5df970d640 1 -- 172.21.15.19:0/3484422482 learned_addr learned my addr 172.21.15.19:0/3484422482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5df8f0c640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df40c7730 0x7f5df4156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5df3fff640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df40ce700 0x7f5df40c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5df970d640 1 -- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df40ce700 msgr2=0x7f5df40c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5df970d640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df40ce700 0x7f5df40c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5df970d640 1 -- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df40c7730 msgr2=0x7f5df4156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.169+0000 7f5df970d640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df40c7730 0x7f5df4156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.170+0000 7f5df970d640 1 -- 172.21.15.19:0/3484422482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df40c9540 con 0x7f5df415da70 2024-04-25T10:05:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.170+0000 7f5df970d640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df415da70 0x7f5df4160600 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f5de400c430 tx=0x7f5de400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.170+0000 7f5df17fa640 1 -- 172.21.15.19:0/3484422482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4017070 con 0x7f5df415da70 2024-04-25T10:05:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.170+0000 7f5df17fa640 1 -- 172.21.15.19:0/3484422482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de4005ce0 con 0x7f5df415da70 2024-04-25T10:05:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.170+0000 7f5dfb170640 1 -- 172.21.15.19:0/3484422482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df40c9820 con 0x7f5df415da70 2024-04-25T10:05:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.170+0000 7f5df17fa640 1 -- 172.21.15.19:0/3484422482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4007310 con 0x7f5df415da70 2024-04-25T10:05:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.170+0000 7f5dfb170640 1 -- 172.21.15.19:0/3484422482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df40c9de0 con 0x7f5df415da70 2024-04-25T10:05:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.171+0000 7f5dfb170640 1 -- 172.21.15.19:0/3484422482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df4073990 con 0x7f5df415da70 2024-04-25T10:05:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.172+0000 7f5df17fa640 1 -- 172.21.15.19:0/3484422482 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5de401a070 con 0x7f5df415da70 2024-04-25T10:05:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.172+0000 7f5df17fa640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b500 0x7f5dd406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.172+0000 7f5df17fa640 1 -- 172.21.15.19:0/3484422482 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5de4089420 con 0x7f5df415da70 2024-04-25T10:05:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.172+0000 7f5df8f0c640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b500 0x7f5dd406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.173+0000 7f5df8f0c640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b500 0x7f5dd406d9c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5df40cbc40 tx=0x7f5dec009870 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.176+0000 7f5df17fa640 1 -- 172.21.15.19:0/3484422482 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5de4057e10 con 0x7f5df415da70 2024-04-25T10:05:36.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.358+0000 7f5dfb170640 1 -- 172.21.15.19:0/3484422482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5df406d860 con 0x7f5df415da70 2024-04-25T10:05:36.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.358+0000 7f5df17fa640 1 -- 172.21.15.19:0/3484422482 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v185) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5de405beb0 con 0x7f5df415da70 2024-04-25T10:05:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:36.364 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T10:05:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T10:05:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T10:05:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T10:05:36.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:36.364 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T10:05:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 -- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b500 msgr2=0x7f5dd406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b500 0x7f5dd406d9c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5df40cbc40 tx=0x7f5dec009870 comp rx=0 tx=0).stop 2024-04-25T10:05:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 -- 172.21.15.19:0/3484422482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df415da70 msgr2=0x7f5df4160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df415da70 0x7f5df4160600 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f5de400c430 tx=0x7f5de400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 -- 172.21.15.19:0/3484422482 shutdown_connections 2024-04-25T10:05:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5dd406b500 0x7f5dd406d9c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df415da70 0x7f5df4160600 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5df40ce700 0x7f5df40c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 --2- 172.21.15.19:0/3484422482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5df40c7730 0x7f5df4156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 -- 172.21.15.19:0/3484422482 >> 172.21.15.19:0/3484422482 conn(0x7f5df40b33e0 msgr2=0x7f5df406ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 -- 172.21.15.19:0/3484422482 shutdown_connections 2024-04-25T10:05:36.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.360+0000 7f5dfb170640 1 -- 172.21.15.19:0/3484422482 wait complete. 2024-04-25T10:05:36.377 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:05:36.377 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:05:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.477+0000 7f73a005d640 1 Processor -- start 2024-04-25T10:05:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.477+0000 7f73a005d640 1 -- start start 2024-04-25T10:05:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f73a005d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f739815b7d0 0x7f73980cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f73a005d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f739806c9e0 0x7f73980b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f73a005d640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739806cf50 con 0x7f73980b5620 2024-04-25T10:05:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f73a005d640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73980ce530 con 0x7f739815b7d0 2024-04-25T10:05:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f73a005d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73980ce810 con 0x7f739806c9e0 2024-04-25T10:05:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739ddf9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f739815b7d0 0x7f73980cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739d5f8640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f73980b5620 0x7f739806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46698/0 (socket says 172.21.15.19:46698) 2024-04-25T10:05:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739ddf9640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f739815b7d0 0x7f73980cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45010/0 (socket says 172.21.15.19:45010) 2024-04-25T10:05:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739d5f8640 1 -- 172.21.15.19:0/2233180412 learned_addr learned my addr 172.21.15.19:0/2233180412 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739e5fa640 1 --2- 172.21.15.19:0/2233180412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f739806c9e0 0x7f73980b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739cdf7640 1 -- 172.21.15.19:0/2233180412 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553638376 0 0) 0x7f739806cf50 con 0x7f73980b5620 2024-04-25T10:05:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739cdf7640 1 -- 172.21.15.19:0/2233180412 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7384003520 con 0x7f73980b5620 2024-04-25T10:05:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739ddf9640 1 -- 172.21.15.19:0/2233180412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f739806c9e0 msgr2=0x7f73980b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739ddf9640 1 --2- 172.21.15.19:0/2233180412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f739806c9e0 0x7f73980b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739ddf9640 1 -- 172.21.15.19:0/2233180412 >> v1:172.21.15.19:6789/0 conn(0x7f73980b5620 legacy=0x7f739806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739ddf9640 1 -- 172.21.15.19:0/2233180412 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73980ceaf0 con 0x7f739815b7d0 2024-04-25T10:05:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.478+0000 7f739e5fa640 1 --2- 172.21.15.19:0/2233180412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f739806c9e0 0x7f73980b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.479+0000 7f739ddf9640 1 --2- 172.21.15.19:0/2233180412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f739815b7d0 0x7f73980cf010 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f73940097a0 tx=0x7f739402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbcce6a4c374b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.479+0000 7f739cdf7640 1 -- 172.21.15.19:0/2233180412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394037070 con 0x7f739815b7d0 2024-04-25T10:05:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.479+0000 7f739cdf7640 1 -- 172.21.15.19:0/2233180412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394032440 con 0x7f739815b7d0 2024-04-25T10:05:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.479+0000 7f739cdf7640 1 -- 172.21.15.19:0/2233180412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394032810 con 0x7f739815b7d0 2024-04-25T10:05:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.480+0000 7f73a005d640 1 -- 172.21.15.19:0/2233180412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f739815b7d0 msgr2=0x7f73980cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.480+0000 7f73a005d640 1 --2- 172.21.15.19:0/2233180412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f739815b7d0 0x7f73980cf010 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f73940097a0 tx=0x7f739402a180 comp rx=0 tx=0).stop 2024-04-25T10:05:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.480+0000 7f73a005d640 1 -- 172.21.15.19:0/2233180412 shutdown_connections 2024-04-25T10:05:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.480+0000 7f73a005d640 1 --2- 172.21.15.19:0/2233180412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f739806c9e0 0x7f73980b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.480+0000 7f73a005d640 1 --2- 172.21.15.19:0/2233180412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f739815b7d0 0x7f73980cf010 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.480+0000 7f73a005d640 1 -- 172.21.15.19:0/2233180412 >> 172.21.15.19:0/2233180412 conn(0x7f73980d0800 msgr2=0x7f73980d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.480+0000 7f73a005d640 1 -- 172.21.15.19:0/2233180412 shutdown_connections 2024-04-25T10:05:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.480+0000 7f73a005d640 1 -- 172.21.15.19:0/2233180412 wait complete. 2024-04-25T10:05:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.481+0000 7f73a005d640 1 Processor -- start 2024-04-25T10:05:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.481+0000 7f73a005d640 1 -- start start 2024-04-25T10:05:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f73a005d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f739806c9e0 0x7f73980c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f73a005d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73980b5620 0x7f73980cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f73a005d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f739815b7d0 0x7f73980b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f73a005d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73980cf6d0 con 0x7f739815b7d0 2024-04-25T10:05:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f73a005d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73980cf550 con 0x7f73980b5620 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f73a005d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739815bbb0 con 0x7f739806c9e0 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f739e5fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f739815b7d0 0x7f73980b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f739e5fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f739815b7d0 0x7f73980b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45016/0 (socket says 172.21.15.19:45016) 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f739e5fa640 1 -- 172.21.15.19:0/3043496511 learned_addr learned my addr 172.21.15.19:0/3043496511 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f739d5f8640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73980b5620 0x7f73980cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f739ddf9640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f739806c9e0 0x7f73980c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f739e5fa640 1 -- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f739806c9e0 msgr2=0x7f73980c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f739e5fa640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f739806c9e0 0x7f73980c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f739e5fa640 1 -- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73980b5620 msgr2=0x7f73980cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f739e5fa640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73980b5620 0x7f73980cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f739e5fa640 1 -- 172.21.15.19:0/3043496511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73980cbed0 con 0x7f739815b7d0 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.482+0000 7f739d5f8640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73980b5620 0x7f73980cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.483+0000 7f739e5fa640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f739815b7d0 0x7f73980b47c0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f7390007de0 tx=0x7f739000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.483+0000 7f738e7fc640 1 -- 172.21.15.19:0/3043496511 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739001a070 con 0x7f739815b7d0 2024-04-25T10:05:36.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.483+0000 7f738e7fc640 1 -- 172.21.15.19:0/3043496511 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7390003990 con 0x7f739815b7d0 2024-04-25T10:05:36.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.483+0000 7f73a005d640 1 -- 172.21.15.19:0/3043496511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73980cc1b0 con 0x7f739815b7d0 2024-04-25T10:05:36.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.483+0000 7f738e7fc640 1 -- 172.21.15.19:0/3043496511 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7390003d80 con 0x7f739815b7d0 2024-04-25T10:05:36.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.483+0000 7f73a005d640 1 -- 172.21.15.19:0/3043496511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73980cc770 con 0x7f739815b7d0 2024-04-25T10:05:36.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.484+0000 7f73a005d640 1 -- 172.21.15.19:0/3043496511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73980cc940 con 0x7f739815b7d0 2024-04-25T10:05:36.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.484+0000 7f738e7fc640 1 -- 172.21.15.19:0/3043496511 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f739001d070 con 0x7f739815b7d0 2024-04-25T10:05:36.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.485+0000 7f738e7fc640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737c06b4d0 0x7f737c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.485+0000 7f738e7fc640 1 -- 172.21.15.19:0/3043496511 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f739008a3e0 con 0x7f739815b7d0 2024-04-25T10:05:36.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.485+0000 7f739ddf9640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737c06b4d0 0x7f737c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.486+0000 7f739ddf9640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737c06b4d0 0x7f737c06d990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7394034000 tx=0x7f739402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.489+0000 7f738e7fc640 1 -- 172.21.15.19:0/3043496511 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7390010070 con 0x7f739815b7d0 2024-04-25T10:05:36.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:36.668+0000 7f73a005d640 1 -- 172.21.15.19:0/3043496511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f73980cbbf0 con 0x7f739815b7d0 2024-04-25T10:05:37.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.051+0000 7f738e7fc640 1 -- 172.21.15.19:0/3043496511 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v186) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7390059410 con 0x7f739815b7d0 2024-04-25T10:05:37.057 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T10:05:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.053+0000 7f73a005d640 1 -- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737c06b4d0 msgr2=0x7f737c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.053+0000 7f73a005d640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737c06b4d0 0x7f737c06d990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7394034000 tx=0x7f739402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.053+0000 7f73a005d640 1 -- 172.21.15.19:0/3043496511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f739815b7d0 msgr2=0x7f73980b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.053+0000 7f73a005d640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f739815b7d0 0x7f73980b47c0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f7390007de0 tx=0x7f739000d4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.053+0000 7f73a005d640 1 -- 172.21.15.19:0/3043496511 shutdown_connections 2024-04-25T10:05:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.053+0000 7f73a005d640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f737c06b4d0 0x7f737c06d990 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.053+0000 7f73a005d640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f739815b7d0 0x7f73980b47c0 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.053+0000 7f73a005d640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73980b5620 0x7f73980cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.053+0000 7f73a005d640 1 --2- 172.21.15.19:0/3043496511 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f739806c9e0 0x7f73980c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.054+0000 7f73a005d640 1 -- 172.21.15.19:0/3043496511 >> 172.21.15.19:0/3043496511 conn(0x7f73980d0800 msgr2=0x7f739815ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.054+0000 7f73a005d640 1 -- 172.21.15.19:0/3043496511 shutdown_connections 2024-04-25T10:05:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.054+0000 7f73a005d640 1 -- 172.21.15.19:0/3043496511 wait complete. 2024-04-25T10:05:37.073 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:05:37.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.177+0000 7f540765a640 1 Processor -- start 2024-04-25T10:05:37.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.177+0000 7f540765a640 1 -- start start 2024-04-25T10:05:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f540765a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f540015dc20 0x7f54000ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f540765a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f54000d32f0 0x7f54000cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f540765a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540015e000 con 0x7f5400161710 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f540765a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54000d1de0 con 0x7f540015dc20 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f540765a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5400161c60 con 0x7f54000d32f0 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f54053f6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f540015dc20 0x7f54000ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f54053f6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f540015dc20 0x7f54000ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45024/0 (socket says 172.21.15.19:45024) 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f54053f6640 1 -- 172.21.15.19:0/3041019247 learned_addr learned my addr 172.21.15.19:0/3041019247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f5404bf5640 1 --1- 172.21.15.19:0/3041019247 >> v1:172.21.15.19:6789/0 conn(0x7f5400161710 0x7f540006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46708/0 (socket says 172.21.15.19:46708) 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f5405bf7640 1 --2- 172.21.15.19:0/3041019247 >> v2:172.21.15.112:3300/0 conn(0x7f54000d32f0 0x7f54000cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f54053f6640 1 -- 172.21.15.19:0/3041019247 >> v2:172.21.15.112:3300/0 conn(0x7f54000d32f0 msgr2=0x7f54000cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f54053f6640 1 --2- 172.21.15.19:0/3041019247 >> v2:172.21.15.112:3300/0 conn(0x7f54000d32f0 0x7f54000cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f54053f6640 1 -- 172.21.15.19:0/3041019247 >> v1:172.21.15.19:6789/0 conn(0x7f5400161710 legacy=0x7f540006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f54053f6640 1 -- 172.21.15.19:0/3041019247 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5400161f40 con 0x7f540015dc20 2024-04-25T10:05:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.178+0000 7f53f7fff640 1 -- 172.21.15.19:0/3041019247 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740559586 0 0) 0x7f540015e000 con 0x7f5400161710 2024-04-25T10:05:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.179+0000 7f5405bf7640 1 --2- 172.21.15.19:0/3041019247 >> v2:172.21.15.112:3300/0 conn(0x7f54000d32f0 0x7f54000cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.179+0000 7f54053f6640 1 --2- 172.21.15.19:0/3041019247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f540015dc20 0x7f54000ce670 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f54000b6910 tx=0x7f53fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e169f1545f25d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.179+0000 7f53f7fff640 1 -- 172.21.15.19:0/3041019247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc037070 con 0x7f540015dc20 2024-04-25T10:05:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.179+0000 7f53f7fff640 1 -- 172.21.15.19:0/3041019247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc032440 con 0x7f540015dc20 2024-04-25T10:05:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.179+0000 7f53f7fff640 1 -- 172.21.15.19:0/3041019247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc032810 con 0x7f540015dc20 2024-04-25T10:05:37.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.180+0000 7f540765a640 1 -- 172.21.15.19:0/3041019247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f540015dc20 msgr2=0x7f54000ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.180+0000 7f540765a640 1 --2- 172.21.15.19:0/3041019247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f540015dc20 0x7f54000ce670 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f54000b6910 tx=0x7f53fc02a270 comp rx=0 tx=0).stop 2024-04-25T10:05:37.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.180+0000 7f540765a640 1 -- 172.21.15.19:0/3041019247 shutdown_connections 2024-04-25T10:05:37.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.180+0000 7f540765a640 1 --2- 172.21.15.19:0/3041019247 >> v2:172.21.15.112:3300/0 conn(0x7f54000d32f0 0x7f54000cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.180+0000 7f540765a640 1 --2- 172.21.15.19:0/3041019247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f540015dc20 0x7f54000ce670 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.180+0000 7f540765a640 1 -- 172.21.15.19:0/3041019247 >> 172.21.15.19:0/3041019247 conn(0x7f54000d0420 msgr2=0x7f54000d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.180+0000 7f540765a640 1 -- 172.21.15.19:0/3041019247 shutdown_connections 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.180+0000 7f540765a640 1 -- 172.21.15.19:0/3041019247 wait complete. 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.181+0000 7f540765a640 1 Processor -- start 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.181+0000 7f540765a640 1 -- start start 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f540765a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000d32f0 0x7f54000aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f540765a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f540015dc20 0x7f5400159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f540765a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5400161710 0x7f54000b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f540765a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54000b5e50 con 0x7f5400161710 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f540765a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5400162500 con 0x7f54000d32f0 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f540765a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54000b5fd0 con 0x7f540015dc20 2024-04-25T10:05:37.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f5405bf7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5400161710 0x7f54000b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f5405bf7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5400161710 0x7f54000b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45026/0 (socket says 172.21.15.19:45026) 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f5405bf7640 1 -- 172.21.15.19:0/1377600191 learned_addr learned my addr 172.21.15.19:0/1377600191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f5404bf5640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f540015dc20 0x7f5400159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f54053f6640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000d32f0 0x7f54000aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f5405bf7640 1 -- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f540015dc20 msgr2=0x7f5400159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f5405bf7640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f540015dc20 0x7f5400159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f5405bf7640 1 -- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000d32f0 msgr2=0x7f54000aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f5405bf7640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000d32f0 0x7f54000aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f5405bf7640 1 -- 172.21.15.19:0/1377600191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54000cbb20 con 0x7f5400161710 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.182+0000 7f5404bf5640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f540015dc20 0x7f5400159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.183+0000 7f5405bf7640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5400161710 0x7f54000b21b0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f53f800ccc0 tx=0x7f53f800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.183+0000 7f53f5ffb640 1 -- 172.21.15.19:0/1377600191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8003b90 con 0x7f5400161710 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.183+0000 7f53f5ffb640 1 -- 172.21.15.19:0/1377600191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f8003d50 con 0x7f5400161710 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.183+0000 7f540765a640 1 -- 172.21.15.19:0/1377600191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54000cbe00 con 0x7f5400161710 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.183+0000 7f53f5ffb640 1 -- 172.21.15.19:0/1377600191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8004260 con 0x7f5400161710 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.183+0000 7f540765a640 1 -- 172.21.15.19:0/1377600191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54000cc3c0 con 0x7f5400161710 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.184+0000 7f540765a640 1 -- 172.21.15.19:0/1377600191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54000cc5c0 con 0x7f5400161710 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.184+0000 7f53f5ffb640 1 -- 172.21.15.19:0/1377600191 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53f8004400 con 0x7f5400161710 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.185+0000 7f53f5ffb640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53e406b4d0 0x7f53e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.185+0000 7f53f5ffb640 1 -- 172.21.15.19:0/1377600191 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f53f808aa90 con 0x7f5400161710 2024-04-25T10:05:37.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.185+0000 7f54053f6640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53e406b4d0 0x7f53e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.186+0000 7f54053f6640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53e406b4d0 0x7f53e406d990 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f53fc034000 tx=0x7f53fc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.189+0000 7f53f5ffb640 1 -- 172.21.15.19:0/1377600191 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54000cc5c0 con 0x7f5400161710 2024-04-25T10:05:37.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:37.370+0000 7f540765a640 1 -- 172.21.15.19:0/1377600191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5400073a00 con 0x7f5400161710 2024-04-25T10:05:38.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.061+0000 7f53f5ffb640 1 -- 172.21.15.19:0/1377600191 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v187) v1 ==== 170+0+0 (secure 0 0 0) 0x7f53f8014030 con 0x7f5400161710 2024-04-25T10:05:38.067 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T10:05:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 -- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53e406b4d0 msgr2=0x7f53e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53e406b4d0 0x7f53e406d990 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f53fc034000 tx=0x7f53fc02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 -- 172.21.15.19:0/1377600191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5400161710 msgr2=0x7f54000b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5400161710 0x7f54000b21b0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f53f800ccc0 tx=0x7f53f800a650 comp rx=0 tx=0).stop 2024-04-25T10:05:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 -- 172.21.15.19:0/1377600191 shutdown_connections 2024-04-25T10:05:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f53e406b4d0 0x7f53e406d990 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5400161710 0x7f54000b21b0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f540015dc20 0x7f5400159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 --2- 172.21.15.19:0/1377600191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000d32f0 0x7f54000aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 -- 172.21.15.19:0/1377600191 >> 172.21.15.19:0/1377600191 conn(0x7f54000d0420 msgr2=0x7f54000d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 -- 172.21.15.19:0/1377600191 shutdown_connections 2024-04-25T10:05:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.064+0000 7f540765a640 1 -- 172.21.15.19:0/1377600191 wait complete. 2024-04-25T10:05:38.092 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:05:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.193+0000 7f38ab658640 1 Processor -- start 2024-04-25T10:05:38.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.193+0000 7f38ab658640 1 -- start start 2024-04-25T10:05:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.194+0000 7f38ab658640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38a4161240 0x7f38a40b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.194+0000 7f38ab658640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38a4161890 0x7f38a40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.194+0000 7f38ab658640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a4161620 con 0x7f38a40b54a0 2024-04-25T10:05:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.194+0000 7f38ab658640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a40b7120 con 0x7f38a4161240 2024-04-25T10:05:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.194+0000 7f38ab658640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4161e00 con 0x7f38a4161890 2024-04-25T10:05:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.194+0000 7f38a93f4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38a4161240 0x7f38a40b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.194+0000 7f38a93f4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38a4161240 0x7f38a40b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35692/0 (socket says 172.21.15.19:35692) 2024-04-25T10:05:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.194+0000 7f38a93f4640 1 -- 172.21.15.19:0/197312035 learned_addr learned my addr 172.21.15.19:0/197312035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.195+0000 7f38a8bf3640 1 --2- 172.21.15.19:0/197312035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38a4161890 0x7f38a40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.195+0000 7f38a93f4640 1 -- 172.21.15.19:0/197312035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38a4161890 msgr2=0x7f38a40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.195+0000 7f38a93f4640 1 --2- 172.21.15.19:0/197312035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38a4161890 0x7f38a40ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.195+0000 7f38a93f4640 1 -- 172.21.15.19:0/197312035 >> v1:172.21.15.112:6789/0 conn(0x7f38a40b54a0 legacy=0x7f38a40d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.195+0000 7f38a93f4640 1 -- 172.21.15.19:0/197312035 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a41620e0 con 0x7f38a4161240 2024-04-25T10:05:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.195+0000 7f38a93f4640 1 --2- 172.21.15.19:0/197312035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a4161240 0x7f38a40b6be0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009870 tx=0x7f38a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c525218f7ceb7cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.196+0000 7f389bfff640 1 -- 172.21.15.19:0/197312035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0037070 con 0x7f38a4161240 2024-04-25T10:05:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.196+0000 7f389bfff640 1 -- 172.21.15.19:0/197312035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a0032440 con 0x7f38a4161240 2024-04-25T10:05:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.196+0000 7f389bfff640 1 -- 172.21.15.19:0/197312035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0032810 con 0x7f38a4161240 2024-04-25T10:05:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.196+0000 7f38ab658640 1 -- 172.21.15.19:0/197312035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a4161240 msgr2=0x7f38a40b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.196+0000 7f38ab658640 1 --2- 172.21.15.19:0/197312035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a4161240 0x7f38a40b6be0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009870 tx=0x7f38a002a270 comp rx=0 tx=0).stop 2024-04-25T10:05:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.196+0000 7f38ab658640 1 -- 172.21.15.19:0/197312035 shutdown_connections 2024-04-25T10:05:38.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.196+0000 7f38ab658640 1 --2- 172.21.15.19:0/197312035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38a4161890 0x7f38a40ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.196+0000 7f38ab658640 1 --2- 172.21.15.19:0/197312035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a4161240 0x7f38a40b6be0 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.196+0000 7f38ab658640 1 -- 172.21.15.19:0/197312035 >> 172.21.15.19:0/197312035 conn(0x7f38a40d04a0 msgr2=0x7f38a40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.196+0000 7f38ab658640 1 -- 172.21.15.19:0/197312035 shutdown_connections 2024-04-25T10:05:38.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.196+0000 7f38ab658640 1 -- 172.21.15.19:0/197312035 wait complete. 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.197+0000 7f38ab658640 1 Processor -- start 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.198+0000 7f38ab658640 1 -- start start 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.198+0000 7f38ab658640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38a40b54a0 0x7f38a40cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.198+0000 7f38ab658640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38a4161240 0x7f38a41e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.198+0000 7f38ab658640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a4161890 0x7f38a4151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.198+0000 7f38ab658640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40d15f0 con 0x7f38a4161890 2024-04-25T10:05:38.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.198+0000 7f38ab658640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41626a0 con 0x7f38a40b54a0 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.198+0000 7f38ab658640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40cf000 con 0x7f38a4161240 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.198+0000 7f38a9bf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a4161890 0x7f38a4151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.198+0000 7f38a9bf5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a4161890 0x7f38a4151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35706/0 (socket says 172.21.15.19:35706) 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.198+0000 7f38a9bf5640 1 -- 172.21.15.19:0/2626543135 learned_addr learned my addr 172.21.15.19:0/2626543135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f38a8bf3640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38a4161240 0x7f38a41e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f38a93f4640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38a40b54a0 0x7f38a40cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f38a9bf5640 1 -- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38a4161240 msgr2=0x7f38a41e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f38a9bf5640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38a4161240 0x7f38a41e80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f38a9bf5640 1 -- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38a40b54a0 msgr2=0x7f38a40cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f38a9bf5640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38a40b54a0 0x7f38a40cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f38a9bf5640 1 -- 172.21.15.19:0/2626543135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a41e87b0 con 0x7f38a4161890 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f38a93f4640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38a40b54a0 0x7f38a40cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f38a9bf5640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a4161890 0x7f38a4151f30 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c740 tx=0x7f389c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f3899ffb640 1 -- 172.21.15.19:0/2626543135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c016020 con 0x7f38a4161890 2024-04-25T10:05:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f3899ffb640 1 -- 172.21.15.19:0/2626543135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c00ce50 con 0x7f38a4161890 2024-04-25T10:05:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f38ab658640 1 -- 172.21.15.19:0/2626543135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a40cc690 con 0x7f38a4161890 2024-04-25T10:05:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f3899ffb640 1 -- 172.21.15.19:0/2626543135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c003970 con 0x7f38a4161890 2024-04-25T10:05:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.199+0000 7f38ab658640 1 -- 172.21.15.19:0/2626543135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a40ccc50 con 0x7f38a4161890 2024-04-25T10:05:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.200+0000 7f38ab658640 1 -- 172.21.15.19:0/2626543135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a4073a00 con 0x7f38a4161890 2024-04-25T10:05:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.201+0000 7f3899ffb640 1 -- 172.21.15.19:0/2626543135 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f389c003b10 con 0x7f38a4161890 2024-04-25T10:05:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.201+0000 7f3899ffb640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f388806b500 0x7f388806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.201+0000 7f3899ffb640 1 -- 172.21.15.19:0/2626543135 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f389c089ef0 con 0x7f38a4161890 2024-04-25T10:05:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.201+0000 7f38a93f4640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f388806b500 0x7f388806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.202+0000 7f38a93f4640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f388806b500 0x7f388806d9c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f38a0034000 tx=0x7f38a002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.206+0000 7f3899ffb640 1 -- 172.21.15.19:0/2626543135 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f389c020020 con 0x7f38a4161890 2024-04-25T10:05:38.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:38.422+0000 7f38ab658640 1 -- 172.21.15.19:0/2626543135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f38a40cecb0 con 0x7f38a4161890 2024-04-25T10:05:39.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.062+0000 7f3899ffb640 1 -- 172.21.15.19:0/2626543135 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 29 and data pool 30 v121) v1 ==== 174+0+0 (secure 0 0 0) 0x7f389c058c30 con 0x7f38a4161890 2024-04-25T10:05:39.068 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 29 and data pool 30 2024-04-25T10:05:39.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.070+0000 7f38ab658640 1 -- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f388806b500 msgr2=0x7f388806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.070+0000 7f38ab658640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f388806b500 0x7f388806d9c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f38a0034000 tx=0x7f38a002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:39.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.070+0000 7f38ab658640 1 -- 172.21.15.19:0/2626543135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a4161890 msgr2=0x7f38a4151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.070+0000 7f38ab658640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a4161890 0x7f38a4151f30 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c740 tx=0x7f389c00cc00 comp rx=0 tx=0).stop 2024-04-25T10:05:39.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.071+0000 7f38ab658640 1 -- 172.21.15.19:0/2626543135 shutdown_connections 2024-04-25T10:05:39.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.071+0000 7f38ab658640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f388806b500 0x7f388806d9c0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.071+0000 7f38ab658640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a4161890 0x7f38a4151f30 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.071+0000 7f38ab658640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38a4161240 0x7f38a41e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.071+0000 7f38ab658640 1 --2- 172.21.15.19:0/2626543135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38a40b54a0 0x7f38a40cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.071+0000 7f38ab658640 1 -- 172.21.15.19:0/2626543135 >> 172.21.15.19:0/2626543135 conn(0x7f38a40d04a0 msgr2=0x7f38a40b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.076+0000 7f38ab658640 1 -- 172.21.15.19:0/2626543135 shutdown_connections 2024-04-25T10:05:39.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.081+0000 7f38ab658640 1 -- 172.21.15.19:0/2626543135 wait complete. 2024-04-25T10:05:39.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:39.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.206+0000 7f1aa8dbe640 1 Processor -- start 2024-04-25T10:05:39.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.206+0000 7f1aa8dbe640 1 -- start start 2024-04-25T10:05:39.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1aa8dbe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa415ef00 0x7f1aa40d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1aa8dbe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1aa40d1ca0 0x7f1aa40c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1aa8dbe640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa40d20e0 con 0x7f1aa40c7140 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1aa8dbe640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa415f2e0 con 0x7f1aa40d1ca0 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1aa8dbe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40c7970 con 0x7f1aa415ef00 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1aa2d9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1aa40c7140 0x7f1aa415f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34860/0 (socket says 172.21.15.19:34860) 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1aa2d9d640 1 -- 172.21.15.19:0/2301260438 learned_addr learned my addr 172.21.15.19:0/2301260438 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1aa259c640 1 --2- 172.21.15.19:0/2301260438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa415ef00 0x7f1aa40d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1a99d9b640 1 --2- 172.21.15.19:0/2301260438 >> v2:172.21.15.112:3300/0 conn(0x7f1aa40d1ca0 0x7f1aa40c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1aa1d9b640 1 -- 172.21.15.19:0/2301260438 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190362981 0 0) 0x7f1aa40d20e0 con 0x7f1aa40c7140 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1aa1d9b640 1 -- 172.21.15.19:0/2301260438 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a90003520 con 0x7f1aa40c7140 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1a99d9b640 1 -- 172.21.15.19:0/2301260438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa415ef00 msgr2=0x7f1aa40d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1a99d9b640 1 --2- 172.21.15.19:0/2301260438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa415ef00 0x7f1aa40d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1a99d9b640 1 -- 172.21.15.19:0/2301260438 >> v1:172.21.15.19:6789/0 conn(0x7f1aa40c7140 legacy=0x7f1aa415f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1a99d9b640 1 -- 172.21.15.19:0/2301260438 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40c7c50 con 0x7f1aa40d1ca0 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.207+0000 7f1aa259c640 1 --2- 172.21.15.19:0/2301260438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa415ef00 0x7f1aa40d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.208+0000 7f1a99d9b640 1 --2- 172.21.15.19:0/2301260438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa40d1ca0 0x7f1aa40c6c00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0097a0 tx=0x7f1a8c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f8b0119dfc1e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.208+0000 7f1aa1d9b640 1 -- 172.21.15.19:0/2301260438 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c037070 con 0x7f1aa40d1ca0 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.208+0000 7f1aa1d9b640 1 -- 172.21.15.19:0/2301260438 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c032440 con 0x7f1aa40d1ca0 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.208+0000 7f1aa1d9b640 1 -- 172.21.15.19:0/2301260438 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c032810 con 0x7f1aa40d1ca0 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.209+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2301260438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa40d1ca0 msgr2=0x7f1aa40c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.209+0000 7f1aa8dbe640 1 --2- 172.21.15.19:0/2301260438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa40d1ca0 0x7f1aa40c6c00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0097a0 tx=0x7f1a8c02a210 comp rx=0 tx=0).stop 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.209+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2301260438 shutdown_connections 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.209+0000 7f1aa8dbe640 1 --2- 172.21.15.19:0/2301260438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa40d1ca0 0x7f1aa40c6c00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.209+0000 7f1aa8dbe640 1 --2- 172.21.15.19:0/2301260438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa415ef00 0x7f1aa40d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.209+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2301260438 >> 172.21.15.19:0/2301260438 conn(0x7f1aa406f5d0 msgr2=0x7f1aa406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.209+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2301260438 shutdown_connections 2024-04-25T10:05:39.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.209+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2301260438 wait complete. 2024-04-25T10:05:39.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.210+0000 7f1aa8dbe640 1 Processor -- start 2024-04-25T10:05:39.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.210+0000 7f1aa8dbe640 1 -- start start 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.211+0000 7f1aa8dbe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa40c7140 0x7f1aa40b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.211+0000 7f1aa8dbe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa40d1ca0 0x7f1aa40b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.211+0000 7f1aa8dbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa415ef00 0x7f1aa41f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.211+0000 7f1aa8dbe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40c84f0 con 0x7f1aa40c7140 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.211+0000 7f1aa8dbe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40c8210 con 0x7f1aa415ef00 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.211+0000 7f1aa8dbe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40c7560 con 0x7f1aa40d1ca0 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.211+0000 7f1aa259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa40c7140 0x7f1aa40b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.211+0000 7f1aa259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa40c7140 0x7f1aa40b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35712/0 (socket says 172.21.15.19:35712) 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.211+0000 7f1aa259c640 1 -- 172.21.15.19:0/2525529587 learned_addr learned my addr 172.21.15.19:0/2525529587 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1a99d9b640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa40d1ca0 0x7f1aa40b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1aa2d9d640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa415ef00 0x7f1aa41f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1aa259c640 1 -- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa40d1ca0 msgr2=0x7f1aa40b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1aa259c640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa40d1ca0 0x7f1aa40b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1aa259c640 1 -- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa415ef00 msgr2=0x7f1aa41f1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1aa259c640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa415ef00 0x7f1aa41f1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1aa259c640 1 -- 172.21.15.19:0/2525529587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40b40f0 con 0x7f1aa40c7140 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1aa2d9d640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa415ef00 0x7f1aa41f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1a99d9b640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa40d1ca0 0x7f1aa40b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1aa259c640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa40c7140 0x7f1aa40b1280 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400e960 tx=0x7f1a9400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1a9b7fe640 1 -- 172.21.15.19:0/2525529587 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94017070 con 0x7f1aa40c7140 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1a9b7fe640 1 -- 172.21.15.19:0/2525529587 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9400ca00 con 0x7f1aa40c7140 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2525529587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa4156c50 con 0x7f1aa40c7140 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1a9b7fe640 1 -- 172.21.15.19:0/2525529587 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400cdd0 con 0x7f1aa40c7140 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.212+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2525529587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa41f2a80 con 0x7f1aa40c7140 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.213+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2525529587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa40c68d0 con 0x7f1aa40c7140 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.214+0000 7f1a9b7fe640 1 -- 172.21.15.19:0/2525529587 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a9401a070 con 0x7f1aa40c7140 2024-04-25T10:05:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.214+0000 7f1a9b7fe640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8406b4d0 0x7f1a8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.214+0000 7f1a9b7fe640 1 -- 172.21.15.19:0/2525529587 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a9408a600 con 0x7f1aa40c7140 2024-04-25T10:05:39.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.214+0000 7f1a99d9b640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8406b4d0 0x7f1a8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.215+0000 7f1a99d9b640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8406b4d0 0x7f1a8406d990 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c034000 tx=0x7f1a8c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.218+0000 7f1a9b7fe640 1 -- 172.21.15.19:0/2525529587 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1aa40c68d0 con 0x7f1aa40c7140 2024-04-25T10:05:39.266 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:39.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:39.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.405+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2525529587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1aa4073990 con 0x7f1aa40c7140 2024-04-25T10:05:39.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.406+0000 7f1a9b7fe640 1 -- 172.21.15.19:0/2525529587 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f1a94059300 con 0x7f1aa40c7140 2024-04-25T10:05:39.411 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:39.412 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":188,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:38.423852+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:36.700326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:37.402558+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:39.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8406b4d0 msgr2=0x7f1a8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8406b4d0 0x7f1a8406d990 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c034000 tx=0x7f1a8c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:39.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2525529587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa40c7140 msgr2=0x7f1aa40b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa40c7140 0x7f1aa40b1280 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400e960 tx=0x7f1a9400ee20 comp rx=0 tx=0).stop 2024-04-25T10:05:39.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2525529587 shutdown_connections 2024-04-25T10:05:39.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1a8406b4d0 0x7f1a8406d990 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1aa415ef00 0x7f1aa41f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1aa40d1ca0 0x7f1aa40b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 --2- 172.21.15.19:0/2525529587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1aa40c7140 0x7f1aa40b1280 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2525529587 >> 172.21.15.19:0/2525529587 conn(0x7f1aa406f5d0 msgr2=0x7f1aa40ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2525529587 shutdown_connections 2024-04-25T10:05:39.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.408+0000 7f1aa8dbe640 1 -- 172.21.15.19:0/2525529587 wait complete. 2024-04-25T10:05:39.425 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.521+0000 7f5398322640 1 Processor -- start 2024-04-25T10:05:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.521+0000 7f5398322640 1 -- start start 2024-04-25T10:05:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f5398322640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53900b6cb0 0x7f53900b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f5398322640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f53900b5a60 0x7f53900d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f5398322640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53900ceee0 con 0x7f539015db70 2024-04-25T10:05:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f5398322640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53900d1d10 con 0x7f53900b5a60 2024-04-25T10:05:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f5398322640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53900d1ff0 con 0x7f53900b6cb0 2024-04-25T10:05:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f53968bf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f53900b5a60 0x7f53900d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f53968bf640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f53900b5a60 0x7f53900d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35714/0 (socket says 172.21.15.19:35714) 2024-04-25T10:05:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f53968bf640 1 -- 172.21.15.19:0/3015062720 learned_addr learned my addr 172.21.15.19:0/3015062720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f53958bd640 1 --2- 172.21.15.19:0/3015062720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53900b6cb0 0x7f53900b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f53968bf640 1 -- 172.21.15.19:0/3015062720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53900b6cb0 msgr2=0x7f53900b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f53968bf640 1 --2- 172.21.15.19:0/3015062720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53900b6cb0 0x7f53900b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f53968bf640 1 -- 172.21.15.19:0/3015062720 >> v1:172.21.15.112:6789/0 conn(0x7f539015db70 legacy=0x7f5390162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.522+0000 7f53968bf640 1 -- 172.21.15.19:0/3015062720 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53900d22d0 con 0x7f53900b5a60 2024-04-25T10:05:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.523+0000 7f53968bf640 1 --2- 172.21.15.19:0/3015062720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900b5a60 0x7f53900d27f0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f5388009e50 tx=0x7f538802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c18edfca0fe0314 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.523+0000 7f53950bc640 1 -- 172.21.15.19:0/3015062720 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388036070 con 0x7f53900b5a60 2024-04-25T10:05:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.523+0000 7f53950bc640 1 -- 172.21.15.19:0/3015062720 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5388031600 con 0x7f53900b5a60 2024-04-25T10:05:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.523+0000 7f53950bc640 1 -- 172.21.15.19:0/3015062720 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53880319d0 con 0x7f53900b5a60 2024-04-25T10:05:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.523+0000 7f5398322640 1 -- 172.21.15.19:0/3015062720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900b5a60 msgr2=0x7f53900d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.523+0000 7f5398322640 1 --2- 172.21.15.19:0/3015062720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900b5a60 0x7f53900d27f0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f5388009e50 tx=0x7f538802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.524+0000 7f5398322640 1 -- 172.21.15.19:0/3015062720 shutdown_connections 2024-04-25T10:05:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.524+0000 7f5398322640 1 --2- 172.21.15.19:0/3015062720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900b5a60 0x7f53900d27f0 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.524+0000 7f5398322640 1 --2- 172.21.15.19:0/3015062720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53900b6cb0 0x7f53900b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.524+0000 7f5398322640 1 -- 172.21.15.19:0/3015062720 >> 172.21.15.19:0/3015062720 conn(0x7f53900d0520 msgr2=0x7f53900d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.524+0000 7f5398322640 1 -- 172.21.15.19:0/3015062720 shutdown_connections 2024-04-25T10:05:39.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.524+0000 7f5398322640 1 -- 172.21.15.19:0/3015062720 wait complete. 2024-04-25T10:05:39.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.525+0000 7f5398322640 1 Processor -- start 2024-04-25T10:05:39.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.525+0000 7f5398322640 1 -- start start 2024-04-25T10:05:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f5398322640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53900b5a60 0x7f53900c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f5398322640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900b6cb0 0x7f53900cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f5398322640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539015db70 0x7f53901489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f5398322640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900cf1c0 con 0x7f53900b6cb0 2024-04-25T10:05:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f5398322640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900ce530 con 0x7f539015db70 2024-04-25T10:05:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f5398322640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53900cf340 con 0x7f53900b5a60 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f53958bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900b6cb0 0x7f53900cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f53958bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900b6cb0 0x7f53900cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35728/0 (socket says 172.21.15.19:35728) 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f53958bd640 1 -- 172.21.15.19:0/3542818501 learned_addr learned my addr 172.21.15.19:0/3542818501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f53968bf640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539015db70 0x7f53901489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f53960be640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53900b5a60 0x7f53900c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f53958bd640 1 -- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53900b5a60 msgr2=0x7f53900c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f53958bd640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53900b5a60 0x7f53900c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f53958bd640 1 -- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539015db70 msgr2=0x7f53901489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f53958bd640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539015db70 0x7f53901489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f53958bd640 1 -- 172.21.15.19:0/3542818501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53900cd360 con 0x7f53900b6cb0 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f53958bd640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900b6cb0 0x7f53900cc8c0 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f538000d7d0 tx=0x7f538000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f5386ffd640 1 -- 172.21.15.19:0/3542818501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538000b6c0 con 0x7f53900b6cb0 2024-04-25T10:05:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f5386ffd640 1 -- 172.21.15.19:0/3542818501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5380019070 con 0x7f53900b6cb0 2024-04-25T10:05:39.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f5386ffd640 1 -- 172.21.15.19:0/3542818501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538000bac0 con 0x7f53900b6cb0 2024-04-25T10:05:39.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.526+0000 7f5398322640 1 -- 172.21.15.19:0/3542818501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53900cd640 con 0x7f53900b6cb0 2024-04-25T10:05:39.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.527+0000 7f5398322640 1 -- 172.21.15.19:0/3542818501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53900cdc00 con 0x7f53900b6cb0 2024-04-25T10:05:39.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.527+0000 7f5398322640 1 -- 172.21.15.19:0/3542818501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5390073990 con 0x7f53900b6cb0 2024-04-25T10:05:39.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.528+0000 7f5386ffd640 1 -- 172.21.15.19:0/3542818501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f538000bc60 con 0x7f53900b6cb0 2024-04-25T10:05:39.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.528+0000 7f5386ffd640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f537406b4d0 0x7f537406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.528+0000 7f5386ffd640 1 -- 172.21.15.19:0/3542818501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f538008a6e0 con 0x7f53900b6cb0 2024-04-25T10:05:39.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.529+0000 7f53960be640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f537406b4d0 0x7f537406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.529+0000 7f53960be640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f537406b4d0 0x7f537406d990 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f538c004f60 tx=0x7f538c007e30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.533+0000 7f5386ffd640 1 -- 172.21.15.19:0/3542818501 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5380058d70 con 0x7f53900b6cb0 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.715+0000 7f5398322640 1 -- 172.21.15.19:0/3542818501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f539015c410 con 0x7f53900b6cb0 2024-04-25T10:05:39.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.716+0000 7f5386ffd640 1 -- 172.21.15.19:0/3542818501 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f538005f0c0 con 0x7f53900b6cb0 2024-04-25T10:05:39.722 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:39.722 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":188,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:38.423852+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:36.700326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:37.402558+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:39.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.718+0000 7f5398322640 1 -- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f537406b4d0 msgr2=0x7f537406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.718+0000 7f5398322640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f537406b4d0 0x7f537406d990 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f538c004f60 tx=0x7f538c007e30 comp rx=0 tx=0).stop 2024-04-25T10:05:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.718+0000 7f5398322640 1 -- 172.21.15.19:0/3542818501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900b6cb0 msgr2=0x7f53900cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.718+0000 7f5398322640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900b6cb0 0x7f53900cc8c0 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f538000d7d0 tx=0x7f538000dc90 comp rx=0 tx=0).stop 2024-04-25T10:05:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.719+0000 7f5398322640 1 -- 172.21.15.19:0/3542818501 shutdown_connections 2024-04-25T10:05:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.719+0000 7f5398322640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f537406b4d0 0x7f537406d990 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.719+0000 7f5398322640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539015db70 0x7f53901489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.719+0000 7f5398322640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900b6cb0 0x7f53900cc8c0 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.719+0000 7f5398322640 1 --2- 172.21.15.19:0/3542818501 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53900b5a60 0x7f53900c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.719+0000 7f5398322640 1 -- 172.21.15.19:0/3542818501 >> 172.21.15.19:0/3542818501 conn(0x7f53900d0520 msgr2=0x7f5390156fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.719+0000 7f5398322640 1 -- 172.21.15.19:0/3542818501 shutdown_connections 2024-04-25T10:05:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.719+0000 7f5398322640 1 -- 172.21.15.19:0/3542818501 wait complete. 2024-04-25T10:05:39.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:05:39.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.834+0000 7f0f1a329640 1 Processor -- start 2024-04-25T10:05:39.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.834+0000 7f0f1a329640 1 -- start start 2024-04-25T10:05:39.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f1a329640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0f1415dc20 0x7f0f140ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f1a329640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0f14161710 0x7f0f140cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f1a329640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f140b9330 0x7f0f1406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f1a329640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1415e000 con 0x7f0f14161710 2024-04-25T10:05:39.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f1a329640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f140d1de0 con 0x7f0f1415dc20 2024-04-25T10:05:39.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f1a329640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f14161c60 con 0x7f0f140b9330 2024-04-25T10:05:39.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f137fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0f14161710 0x7f0f140cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f137fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0f14161710 0x7f0f140cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35730/0 (socket says 172.21.15.19:35730) 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f137fe640 1 -- 172.21.15.19:0/1266910675 learned_addr learned my addr 172.21.15.19:0/1266910675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f188c6640 1 --2- 172.21.15.19:0/1266910675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f140b9330 0x7f0f1406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f13fff640 1 --2- 172.21.15.19:0/1266910675 >> v2:172.21.15.112:3300/0 conn(0x7f0f1415dc20 0x7f0f140ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f137fe640 1 -- 172.21.15.19:0/1266910675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f140b9330 msgr2=0x7f0f1406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f137fe640 1 --2- 172.21.15.19:0/1266910675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f140b9330 0x7f0f1406bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f137fe640 1 -- 172.21.15.19:0/1266910675 >> v2:172.21.15.112:3300/0 conn(0x7f0f1415dc20 msgr2=0x7f0f140ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f137fe640 1 --2- 172.21.15.19:0/1266910675 >> v2:172.21.15.112:3300/0 conn(0x7f0f1415dc20 0x7f0f140ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f137fe640 1 -- 172.21.15.19:0/1266910675 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f14161f40 con 0x7f0f14161710 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f188c6640 1 --2- 172.21.15.19:0/1266910675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f140b9330 0x7f0f1406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.835+0000 7f0f13fff640 1 --2- 172.21.15.19:0/1266910675 >> v2:172.21.15.112:3300/0 conn(0x7f0f1415dc20 0x7f0f140ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.836+0000 7f0f137fe640 1 --2- 172.21.15.19:0/1266910675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f14161710 0x7f0f140cebb0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f0f140d1520 tx=0x7f0f0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6d53da27858c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.836+0000 7f0f12ffd640 1 -- 172.21.15.19:0/1266910675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04037070 con 0x7f0f14161710 2024-04-25T10:05:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.836+0000 7f0f12ffd640 1 -- 172.21.15.19:0/1266910675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f04032440 con 0x7f0f14161710 2024-04-25T10:05:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.836+0000 7f0f12ffd640 1 -- 172.21.15.19:0/1266910675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04032810 con 0x7f0f14161710 2024-04-25T10:05:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.836+0000 7f0f1a329640 1 -- 172.21.15.19:0/1266910675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f14161710 msgr2=0x7f0f140cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.836+0000 7f0f1a329640 1 --2- 172.21.15.19:0/1266910675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f14161710 0x7f0f140cebb0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f0f140d1520 tx=0x7f0f0402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.837+0000 7f0f1a329640 1 -- 172.21.15.19:0/1266910675 shutdown_connections 2024-04-25T10:05:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.837+0000 7f0f1a329640 1 --2- 172.21.15.19:0/1266910675 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f140b9330 0x7f0f1406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.837+0000 7f0f1a329640 1 --2- 172.21.15.19:0/1266910675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f14161710 0x7f0f140cebb0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.837+0000 7f0f1a329640 1 --2- 172.21.15.19:0/1266910675 >> v2:172.21.15.112:3300/0 conn(0x7f0f1415dc20 0x7f0f140ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.837+0000 7f0f1a329640 1 -- 172.21.15.19:0/1266910675 >> 172.21.15.19:0/1266910675 conn(0x7f0f140d0420 msgr2=0x7f0f140d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.837+0000 7f0f1a329640 1 -- 172.21.15.19:0/1266910675 shutdown_connections 2024-04-25T10:05:39.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.837+0000 7f0f1a329640 1 -- 172.21.15.19:0/1266910675 wait complete. 2024-04-25T10:05:39.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.838+0000 7f0f1a329640 1 Processor -- start 2024-04-25T10:05:39.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.838+0000 7f0f1a329640 1 -- start start 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f1a329640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f140b9330 0x7f0f140d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f1a329640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f1415dc20 0x7f0f14155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f1a329640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f14161710 0x7f0f14160720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f1a329640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f140b5e50 con 0x7f0f140b9330 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f1a329640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f14162500 con 0x7f0f1415dc20 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f1a329640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f140b5fd0 con 0x7f0f14161710 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f140b9330 0x7f0f140d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f140b9330 0x7f0f140d3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35734/0 (socket says 172.21.15.19:35734) 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f13fff640 1 -- 172.21.15.19:0/2571026528 learned_addr learned my addr 172.21.15.19:0/2571026528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f188c6640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f14161710 0x7f0f14160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f137fe640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f1415dc20 0x7f0f14155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f13fff640 1 -- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f14161710 msgr2=0x7f0f14160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f13fff640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f14161710 0x7f0f14160720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f13fff640 1 -- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f1415dc20 msgr2=0x7f0f14155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f13fff640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f1415dc20 0x7f0f14155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f13fff640 1 -- 172.21.15.19:0/2571026528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f140cac80 con 0x7f0f140b9330 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f188c6640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f14161710 0x7f0f14160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f13fff640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f140b9330 0x7f0f140d3bf0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800c820 tx=0x7f0f0800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f10ff9640 1 -- 172.21.15.19:0/2571026528 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08015070 con 0x7f0f140b9330 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f10ff9640 1 -- 172.21.15.19:0/2571026528 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f080079d0 con 0x7f0f140b9330 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f1a329640 1 -- 172.21.15.19:0/2571026528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f140caf60 con 0x7f0f140b9330 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f10ff9640 1 -- 172.21.15.19:0/2571026528 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08007d80 con 0x7f0f140b9330 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.839+0000 7f0f1a329640 1 -- 172.21.15.19:0/2571026528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f140cb520 con 0x7f0f140b9330 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.840+0000 7f0f1a329640 1 -- 172.21.15.19:0/2571026528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f14073a00 con 0x7f0f140b9330 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.841+0000 7f0f10ff9640 1 -- 172.21.15.19:0/2571026528 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f08018070 con 0x7f0f140b9330 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.841+0000 7f0f10ff9640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f0006b4d0 0x7f0f0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.841+0000 7f0f10ff9640 1 -- 172.21.15.19:0/2571026528 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f0808a1c0 con 0x7f0f140b9330 2024-04-25T10:05:39.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.842+0000 7f0f137fe640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f0006b4d0 0x7f0f0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.842+0000 7f0f137fe640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f0006b4d0 0x7f0f0006d990 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f0f04034000 tx=0x7f0f0402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:39.846+0000 7f0f10ff9640 1 -- 172.21.15.19:0/2571026528 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f08058860 con 0x7f0f140b9330 2024-04-25T10:05:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.063+0000 7f0f1a329640 1 -- 172.21.15.19:0/2571026528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0f140b4df0 con 0x7f0f140b9330 2024-04-25T10:05:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.078+0000 7f0f10ff9640 1 -- 172.21.15.19:0/2571026528 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v122) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0f0805c900 con 0x7f0f140b9330 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.080+0000 7f0f1a329640 1 -- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f0006b4d0 msgr2=0x7f0f0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.080+0000 7f0f1a329640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f0006b4d0 0x7f0f0006d990 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f0f04034000 tx=0x7f0f0402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.080+0000 7f0f1a329640 1 -- 172.21.15.19:0/2571026528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f140b9330 msgr2=0x7f0f140d3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.080+0000 7f0f1a329640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f140b9330 0x7f0f140d3bf0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800c820 tx=0x7f0f0800cce0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.080+0000 7f0f1a329640 1 -- 172.21.15.19:0/2571026528 shutdown_connections 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.080+0000 7f0f1a329640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0f0006b4d0 0x7f0f0006d990 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.080+0000 7f0f1a329640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f14161710 0x7f0f14160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.080+0000 7f0f1a329640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f1415dc20 0x7f0f14155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.080+0000 7f0f1a329640 1 --2- 172.21.15.19:0/2571026528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f140b9330 0x7f0f140d3bf0 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.080+0000 7f0f1a329640 1 -- 172.21.15.19:0/2571026528 >> 172.21.15.19:0/2571026528 conn(0x7f0f140d0420 msgr2=0x7f0f140b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.081+0000 7f0f1a329640 1 -- 172.21.15.19:0/2571026528 shutdown_connections 2024-04-25T10:05:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.081+0000 7f0f1a329640 1 -- 172.21.15.19:0/2571026528 wait complete. 2024-04-25T10:05:40.097 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:05:40.098 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:05:40.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.196+0000 7f305f86f640 1 Processor -- start 2024-04-25T10:05:40.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.196+0000 7f305f86f640 1 -- start start 2024-04-25T10:05:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305f86f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30580b5620 0x7f30580b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305f86f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f305806c2d0 0x7f305806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305f86f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305806cc80 con 0x7f305815b7d0 2024-04-25T10:05:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305f86f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305806cf50 con 0x7f30580b5620 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305f86f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30580ce530 con 0x7f305806c2d0 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305ce0a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30580b5620 0x7f30580b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305d60b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f305815b7d0 0x7f30580cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34874/0 (socket says 172.21.15.19:34874) 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305ce0a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30580b5620 0x7f30580b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35736/0 (socket says 172.21.15.19:35736) 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305d60b640 1 -- 172.21.15.19:0/928790783 learned_addr learned my addr 172.21.15.19:0/928790783 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305ce0a640 1 -- 172.21.15.19:0/928790783 >> v2:172.21.15.112:3300/0 conn(0x7f305806c2d0 msgr2=0x7f305806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305ce0a640 1 --2- 172.21.15.19:0/928790783 >> v2:172.21.15.112:3300/0 conn(0x7f305806c2d0 0x7f305806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305ce0a640 1 -- 172.21.15.19:0/928790783 >> v1:172.21.15.19:6789/0 conn(0x7f305815b7d0 legacy=0x7f30580cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f305ce0a640 1 -- 172.21.15.19:0/928790783 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30580ce810 con 0x7f30580b5620 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.197+0000 7f304ffff640 1 -- 172.21.15.19:0/928790783 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346011452 0 0) 0x7f305806cc80 con 0x7f305815b7d0 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.198+0000 7f305ce0a640 1 --2- 172.21.15.19:0/928790783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30580b5620 0x7f30580b5ab0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f3058161890 tx=0x7f304802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fc61d059024bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.198+0000 7f304ffff640 1 -- 172.21.15.19:0/928790783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048037070 con 0x7f30580b5620 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.198+0000 7f304ffff640 1 -- 172.21.15.19:0/928790783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3048032440 con 0x7f30580b5620 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.198+0000 7f304ffff640 1 -- 172.21.15.19:0/928790783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048032810 con 0x7f30580b5620 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.199+0000 7f305f86f640 1 -- 172.21.15.19:0/928790783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30580b5620 msgr2=0x7f30580b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.199+0000 7f305f86f640 1 --2- 172.21.15.19:0/928790783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30580b5620 0x7f30580b5ab0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f3058161890 tx=0x7f304802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.199+0000 7f305f86f640 1 -- 172.21.15.19:0/928790783 shutdown_connections 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.199+0000 7f305f86f640 1 --2- 172.21.15.19:0/928790783 >> v2:172.21.15.112:3300/0 conn(0x7f305806c2d0 0x7f305806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.199+0000 7f305f86f640 1 --2- 172.21.15.19:0/928790783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30580b5620 0x7f30580b5ab0 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.199+0000 7f305f86f640 1 -- 172.21.15.19:0/928790783 >> 172.21.15.19:0/928790783 conn(0x7f30580d0800 msgr2=0x7f30580d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.199+0000 7f305f86f640 1 -- 172.21.15.19:0/928790783 shutdown_connections 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.199+0000 7f305f86f640 1 -- 172.21.15.19:0/928790783 wait complete. 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.200+0000 7f305f86f640 1 Processor -- start 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.200+0000 7f305f86f640 1 -- start start 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305f86f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305806c2d0 0x7f30580b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305f86f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30580b5620 0x7f30580c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305f86f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f305815b7d0 0x7f305814ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305f86f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305815bbb0 con 0x7f305815b7d0 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305f86f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305806d230 con 0x7f305806c2d0 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305f86f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305815bd30 con 0x7f30580b5620 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305de0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f305815b7d0 0x7f305814ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305de0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f305815b7d0 0x7f305814ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35740/0 (socket says 172.21.15.19:35740) 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305de0c640 1 -- 172.21.15.19:0/182293171 learned_addr learned my addr 172.21.15.19:0/182293171 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305d60b640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305806c2d0 0x7f30580b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305ce0a640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30580b5620 0x7f30580c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305de0c640 1 -- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30580b5620 msgr2=0x7f30580c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305de0c640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30580b5620 0x7f30580c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305de0c640 1 -- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305806c2d0 msgr2=0x7f30580b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305de0c640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305806c2d0 0x7f30580b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305de0c640 1 -- 172.21.15.19:0/182293171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30580c8aa0 con 0x7f305815b7d0 2024-04-25T10:05:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305ce0a640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30580b5620 0x7f30580c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305de0c640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f305815b7d0 0x7f305814ccd0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f305000b9e0 tx=0x7f305000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f304dffb640 1 -- 172.21.15.19:0/182293171 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050019070 con 0x7f305815b7d0 2024-04-25T10:05:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f304dffb640 1 -- 172.21.15.19:0/182293171 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305000cac0 con 0x7f305815b7d0 2024-04-25T10:05:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f305f86f640 1 -- 172.21.15.19:0/182293171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30580cc180 con 0x7f305815b7d0 2024-04-25T10:05:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.201+0000 7f304dffb640 1 -- 172.21.15.19:0/182293171 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050014440 con 0x7f305815b7d0 2024-04-25T10:05:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.202+0000 7f305f86f640 1 -- 172.21.15.19:0/182293171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30580cc740 con 0x7f305815b7d0 2024-04-25T10:05:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.203+0000 7f305f86f640 1 -- 172.21.15.19:0/182293171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30580cbf70 con 0x7f305815b7d0 2024-04-25T10:05:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.203+0000 7f304dffb640 1 -- 172.21.15.19:0/182293171 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f305001c070 con 0x7f305815b7d0 2024-04-25T10:05:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.203+0000 7f304dffb640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f303c06b4d0 0x7f303c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.204+0000 7f304dffb640 1 -- 172.21.15.19:0/182293171 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f305008a060 con 0x7f305815b7d0 2024-04-25T10:05:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.204+0000 7f305d60b640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f303c06b4d0 0x7f303c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.204+0000 7f305d60b640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f303c06b4d0 0x7f303c06d990 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3054000f60 tx=0x7f3054007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.208+0000 7f304dffb640 1 -- 172.21.15.19:0/182293171 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3050058d50 con 0x7f305815b7d0 2024-04-25T10:05:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:40.425+0000 7f305f86f640 1 -- 172.21.15.19:0/182293171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3058073a00 con 0x7f305815b7d0 2024-04-25T10:05:41.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.080+0000 7f304dffb640 1 -- 172.21.15.19:0/182293171 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v123) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3050058650 con 0x7f305815b7d0 2024-04-25T10:05:41.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.082+0000 7f305f86f640 1 -- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f303c06b4d0 msgr2=0x7f303c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.082+0000 7f305f86f640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f303c06b4d0 0x7f303c06d990 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3054000f60 tx=0x7f3054007c00 comp rx=0 tx=0).stop 2024-04-25T10:05:41.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.082+0000 7f305f86f640 1 -- 172.21.15.19:0/182293171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f305815b7d0 msgr2=0x7f305814ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.082+0000 7f305f86f640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f305815b7d0 0x7f305814ccd0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f305000b9e0 tx=0x7f305000bea0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.083+0000 7f305f86f640 1 -- 172.21.15.19:0/182293171 shutdown_connections 2024-04-25T10:05:41.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.083+0000 7f305f86f640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f303c06b4d0 0x7f303c06d990 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.083+0000 7f305f86f640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f305815b7d0 0x7f305814ccd0 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.083+0000 7f305f86f640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30580b5620 0x7f30580c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.083+0000 7f305f86f640 1 --2- 172.21.15.19:0/182293171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305806c2d0 0x7f30580b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.083+0000 7f305f86f640 1 -- 172.21.15.19:0/182293171 >> 172.21.15.19:0/182293171 conn(0x7f30580d0800 msgr2=0x7f30580d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.083+0000 7f305f86f640 1 -- 172.21.15.19:0/182293171 shutdown_connections 2024-04-25T10:05:41.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.083+0000 7f305f86f640 1 -- 172.21.15.19:0/182293171 wait complete. 2024-04-25T10:05:41.101 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:05:41.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:41.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.203+0000 7f2171502640 1 Processor -- start 2024-04-25T10:05:41.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.203+0000 7f2171502640 1 -- start start 2024-04-25T10:05:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f2171502640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f216c15dbe0 0x7f216c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f2171502640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216c0b2cb0 0x7f216c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f2171502640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f216c0b92a0 0x7f216c0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f2171502640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c15dfc0 con 0x7f216c0b92a0 2024-04-25T10:05:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f2171502640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c0d1a00 con 0x7f216c15dbe0 2024-04-25T10:05:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f2171502640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0b97a0 con 0x7f216c0b2cb0 2024-04-25T10:05:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f216b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f216c0b92a0 0x7f216c0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f216b7fe640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f216c0b92a0 0x7f216c0b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35744/0 (socket says 172.21.15.19:35744) 2024-04-25T10:05:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f216b7fe640 1 -- 172.21.15.19:0/3082577083 learned_addr learned my addr 172.21.15.19:0/3082577083 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f2163fff640 1 --2- 172.21.15.19:0/3082577083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216c0b2cb0 0x7f216c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f216affd640 1 --2- 172.21.15.19:0/3082577083 >> v2:172.21.15.112:3300/0 conn(0x7f216c15dbe0 0x7f216c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f216b7fe640 1 -- 172.21.15.19:0/3082577083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216c0b2cb0 msgr2=0x7f216c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f216b7fe640 1 --2- 172.21.15.19:0/3082577083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216c0b2cb0 0x7f216c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f216b7fe640 1 -- 172.21.15.19:0/3082577083 >> v2:172.21.15.112:3300/0 conn(0x7f216c15dbe0 msgr2=0x7f216c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f216b7fe640 1 --2- 172.21.15.19:0/3082577083 >> v2:172.21.15.112:3300/0 conn(0x7f216c15dbe0 0x7f216c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.204+0000 7f216b7fe640 1 -- 172.21.15.19:0/3082577083 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0d1c50 con 0x7f216c0b92a0 2024-04-25T10:05:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.205+0000 7f216b7fe640 1 --2- 172.21.15.19:0/3082577083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216c0b92a0 0x7f216c0b5400 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f216c06c140 tx=0x7f216402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0083bfbd916b0b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.205+0000 7f216a7fc640 1 -- 172.21.15.19:0/3082577083 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164036070 con 0x7f216c0b92a0 2024-04-25T10:05:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.205+0000 7f216a7fc640 1 -- 172.21.15.19:0/3082577083 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2164031600 con 0x7f216c0b92a0 2024-04-25T10:05:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.205+0000 7f216a7fc640 1 -- 172.21.15.19:0/3082577083 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21640319d0 con 0x7f216c0b92a0 2024-04-25T10:05:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.206+0000 7f2171502640 1 -- 172.21.15.19:0/3082577083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216c0b92a0 msgr2=0x7f216c0b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.206+0000 7f2171502640 1 --2- 172.21.15.19:0/3082577083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216c0b92a0 0x7f216c0b5400 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f216c06c140 tx=0x7f216402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.206+0000 7f2171502640 1 -- 172.21.15.19:0/3082577083 shutdown_connections 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.206+0000 7f2171502640 1 --2- 172.21.15.19:0/3082577083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216c0b2cb0 0x7f216c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.206+0000 7f2171502640 1 --2- 172.21.15.19:0/3082577083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216c0b92a0 0x7f216c0b5400 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.206+0000 7f2171502640 1 --2- 172.21.15.19:0/3082577083 >> v2:172.21.15.112:3300/0 conn(0x7f216c15dbe0 0x7f216c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.206+0000 7f2171502640 1 -- 172.21.15.19:0/3082577083 >> 172.21.15.19:0/3082577083 conn(0x7f216c061a20 msgr2=0x7f216c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.206+0000 7f2171502640 1 -- 172.21.15.19:0/3082577083 shutdown_connections 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.206+0000 7f2171502640 1 -- 172.21.15.19:0/3082577083 wait complete. 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.207+0000 7f2171502640 1 Processor -- start 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.207+0000 7f2171502640 1 -- start start 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f2171502640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216c0b2cb0 0x7f216c15e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f2171502640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c0b92a0 0x7f216c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f2171502640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216c15dbe0 0x7f216c0b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f2171502640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0b6ab0 con 0x7f216c0b2cb0 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f2171502640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0d2210 con 0x7f216c0b92a0 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f2171502640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0b6c30 con 0x7f216c15dbe0 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f216affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216c0b2cb0 0x7f216c15e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f216affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216c0b2cb0 0x7f216c15e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35746/0 (socket says 172.21.15.19:35746) 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f216affd640 1 -- 172.21.15.19:0/4228007467 learned_addr learned my addr 172.21.15.19:0/4228007467 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f2163fff640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c0b92a0 0x7f216c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f216b7fe640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216c15dbe0 0x7f216c0b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f216affd640 1 -- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216c15dbe0 msgr2=0x7f216c0b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f216affd640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216c15dbe0 0x7f216c0b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f216affd640 1 -- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c0b92a0 msgr2=0x7f216c0b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f216affd640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c0b92a0 0x7f216c0b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f216affd640 1 -- 172.21.15.19:0/4228007467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c15eb30 con 0x7f216c0b2cb0 2024-04-25T10:05:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f2163fff640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c0b92a0 0x7f216c0b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:41.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f216affd640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216c0b2cb0 0x7f216c15e150 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f215c00e8a0 tx=0x7f215c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.208+0000 7f21637fe640 1 -- 172.21.15.19:0/4228007467 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c017070 con 0x7f216c0b2cb0 2024-04-25T10:05:41.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.209+0000 7f21637fe640 1 -- 172.21.15.19:0/4228007467 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c00c940 con 0x7f216c0b2cb0 2024-04-25T10:05:41.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.209+0000 7f2171502640 1 -- 172.21.15.19:0/4228007467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c15ee10 con 0x7f216c0b2cb0 2024-04-25T10:05:41.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.209+0000 7f21637fe640 1 -- 172.21.15.19:0/4228007467 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c00cd30 con 0x7f216c0b2cb0 2024-04-25T10:05:41.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.209+0000 7f2171502640 1 -- 172.21.15.19:0/4228007467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c15f3d0 con 0x7f216c0b2cb0 2024-04-25T10:05:41.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.209+0000 7f2171502640 1 -- 172.21.15.19:0/4228007467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c073990 con 0x7f216c0b2cb0 2024-04-25T10:05:41.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.210+0000 7f21637fe640 1 -- 172.21.15.19:0/4228007467 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f215c01a070 con 0x7f216c0b2cb0 2024-04-25T10:05:41.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.210+0000 7f21637fe640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f214c06b4d0 0x7f214c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.210+0000 7f21637fe640 1 -- 172.21.15.19:0/4228007467 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f215c08a510 con 0x7f216c0b2cb0 2024-04-25T10:05:41.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.211+0000 7f2163fff640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f214c06b4d0 0x7f214c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.211+0000 7f2163fff640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f214c06b4d0 0x7f214c06d990 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f216c154860 tx=0x7f2154008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.215+0000 7f21637fe640 1 -- 172.21.15.19:0/4228007467 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f215c058bb0 con 0x7f216c0b2cb0 2024-04-25T10:05:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.431+0000 7f2171502640 1 -- 172.21.15.19:0/4228007467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f216c0b29a0 con 0x7f216c0b2cb0 2024-04-25T10:05:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.432+0000 7f21637fe640 1 -- 172.21.15.19:0/4228007467 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f215c05cc50 con 0x7f216c0b2cb0 2024-04-25T10:05:41.438 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:41.438 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":123,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:38.424174+0000","modified":"2024-04-25T10:05:40.426910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11559":{"gid":11559,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/392508344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":392508344},{"type":"v1","addr":"172.21.15.19:6835","nonce":392508344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:05:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 123 2024-04-25T10:05:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 -- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f214c06b4d0 msgr2=0x7f214c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f214c06b4d0 0x7f214c06d990 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f216c154860 tx=0x7f2154008040 comp rx=0 tx=0).stop 2024-04-25T10:05:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 -- 172.21.15.19:0/4228007467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216c0b2cb0 msgr2=0x7f216c15e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216c0b2cb0 0x7f216c15e150 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f215c00e8a0 tx=0x7f215c00ed60 comp rx=0 tx=0).stop 2024-04-25T10:05:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 -- 172.21.15.19:0/4228007467 shutdown_connections 2024-04-25T10:05:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f214c06b4d0 0x7f214c06d990 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216c15dbe0 0x7f216c0b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c0b92a0 0x7f216c0b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 --2- 172.21.15.19:0/4228007467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216c0b2cb0 0x7f216c15e150 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 -- 172.21.15.19:0/4228007467 >> 172.21.15.19:0/4228007467 conn(0x7f216c061a20 msgr2=0x7f216c0b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 -- 172.21.15.19:0/4228007467 shutdown_connections 2024-04-25T10:05:41.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.434+0000 7f2171502640 1 -- 172.21.15.19:0/4228007467 wait complete. 2024-04-25T10:05:41.451 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.546+0000 7f202e6ac640 1 Processor -- start 2024-04-25T10:05:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.546+0000 7f202e6ac640 1 -- start start 2024-04-25T10:05:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f202e6ac640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f202815dd30 0x7f20280b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f202e6ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20280d0ad0 0x7f20280b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f202e6ac640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20280d1df0 con 0x7f20280b9770 2024-04-25T10:05:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f202e6ac640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20280d20d0 con 0x7f202815dd30 2024-04-25T10:05:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f202e6ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20280d23b0 con 0x7f20280d0ad0 2024-04-25T10:05:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f2027fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f202815dd30 0x7f20280b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f20277fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f20280b9770 0x7f20280d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34880/0 (socket says 172.21.15.19:34880) 2024-04-25T10:05:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f2027fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f202815dd30 0x7f20280b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35756/0 (socket says 172.21.15.19:35756) 2024-04-25T10:05:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f20277fe640 1 -- 172.21.15.19:0/1571080172 learned_addr learned my addr 172.21.15.19:0/1571080172 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f2027fff640 1 -- 172.21.15.19:0/1571080172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20280d0ad0 msgr2=0x7f20280b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f202cc49640 1 --2- 172.21.15.19:0/1571080172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20280d0ad0 0x7f20280b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f2026ffd640 1 -- 172.21.15.19:0/1571080172 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843628058 0 0) 0x7f20280d1df0 con 0x7f20280b9770 2024-04-25T10:05:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f2027fff640 1 --2- 172.21.15.19:0/1571080172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20280d0ad0 0x7f20280b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f2027fff640 1 -- 172.21.15.19:0/1571080172 >> v1:172.21.15.19:6789/0 conn(0x7f20280b9770 legacy=0x7f20280d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f2027fff640 1 -- 172.21.15.19:0/1571080172 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280d2690 con 0x7f202815dd30 2024-04-25T10:05:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.547+0000 7f202cc49640 1 --2- 172.21.15.19:0/1571080172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20280d0ad0 0x7f20280b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.548+0000 7f2027fff640 1 --2- 172.21.15.19:0/1571080172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f202815dd30 0x7f20280b9230 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f20180097a0 tx=0x7f201802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732b8da0941460d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.548+0000 7f2026ffd640 1 -- 172.21.15.19:0/1571080172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018037070 con 0x7f202815dd30 2024-04-25T10:05:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.548+0000 7f2026ffd640 1 -- 172.21.15.19:0/1571080172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201802ae50 con 0x7f202815dd30 2024-04-25T10:05:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.548+0000 7f2026ffd640 1 -- 172.21.15.19:0/1571080172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018032660 con 0x7f202815dd30 2024-04-25T10:05:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.549+0000 7f202e6ac640 1 -- 172.21.15.19:0/1571080172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f202815dd30 msgr2=0x7f20280b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.549+0000 7f202e6ac640 1 --2- 172.21.15.19:0/1571080172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f202815dd30 0x7f20280b9230 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f20180097a0 tx=0x7f201802a100 comp rx=0 tx=0).stop 2024-04-25T10:05:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.549+0000 7f202e6ac640 1 -- 172.21.15.19:0/1571080172 shutdown_connections 2024-04-25T10:05:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.549+0000 7f202e6ac640 1 --2- 172.21.15.19:0/1571080172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20280d0ad0 0x7f20280b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.549+0000 7f202e6ac640 1 --2- 172.21.15.19:0/1571080172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f202815dd30 0x7f20280b9230 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.549+0000 7f202e6ac640 1 -- 172.21.15.19:0/1571080172 >> 172.21.15.19:0/1571080172 conn(0x7f2028061c30 msgr2=0x7f202806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.549+0000 7f202e6ac640 1 -- 172.21.15.19:0/1571080172 shutdown_connections 2024-04-25T10:05:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.549+0000 7f202e6ac640 1 -- 172.21.15.19:0/1571080172 wait complete. 2024-04-25T10:05:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.550+0000 7f202e6ac640 1 Processor -- start 2024-04-25T10:05:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.550+0000 7f202e6ac640 1 -- start start 2024-04-25T10:05:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f202e6ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20280b9770 0x7f20280b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f202e6ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20280d0ad0 0x7f20280cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f202e6ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f202815dd30 0x7f2028152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f202e6ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280d1b90 con 0x7f20280d0ad0 2024-04-25T10:05:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f202e6ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280d1a10 con 0x7f202815dd30 2024-04-25T10:05:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f202e6ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20280d2e90 con 0x7f20280b9770 2024-04-25T10:05:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f20277fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20280d0ad0 0x7f20280cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f20277fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20280d0ad0 0x7f20280cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35758/0 (socket says 172.21.15.19:35758) 2024-04-25T10:05:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f20277fe640 1 -- 172.21.15.19:0/2248581985 learned_addr learned my addr 172.21.15.19:0/2248581985 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f2027fff640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20280b9770 0x7f20280b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f202cc49640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f202815dd30 0x7f2028152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f20277fe640 1 -- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20280b9770 msgr2=0x7f20280b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f20277fe640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20280b9770 0x7f20280b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f20277fe640 1 -- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f202815dd30 msgr2=0x7f2028152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f20277fe640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f202815dd30 0x7f2028152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f20277fe640 1 -- 172.21.15.19:0/2248581985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280cac00 con 0x7f20280d0ad0 2024-04-25T10:05:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.551+0000 7f20277fe640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20280d0ad0 0x7f20280cd150 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f201000c4d0 tx=0x7f201000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.552+0000 7f2024ff9640 1 -- 172.21.15.19:0/2248581985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010007480 con 0x7f20280d0ad0 2024-04-25T10:05:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.552+0000 7f2024ff9640 1 -- 172.21.15.19:0/2248581985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20100057a0 con 0x7f20280d0ad0 2024-04-25T10:05:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.552+0000 7f202e6ac640 1 -- 172.21.15.19:0/2248581985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20280caee0 con 0x7f20280d0ad0 2024-04-25T10:05:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.552+0000 7f2024ff9640 1 -- 172.21.15.19:0/2248581985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010007620 con 0x7f20280d0ad0 2024-04-25T10:05:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.552+0000 7f202e6ac640 1 -- 172.21.15.19:0/2248581985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20280cb4a0 con 0x7f20280d0ad0 2024-04-25T10:05:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.553+0000 7f202e6ac640 1 -- 172.21.15.19:0/2248581985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2028073990 con 0x7f20280d0ad0 2024-04-25T10:05:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.553+0000 7f2024ff9640 1 -- 172.21.15.19:0/2248581985 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2010005b50 con 0x7f20280d0ad0 2024-04-25T10:05:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.553+0000 7f2024ff9640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200806b4d0 0x7f200806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.554+0000 7f2024ff9640 1 -- 172.21.15.19:0/2248581985 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f201008a020 con 0x7f20280d0ad0 2024-04-25T10:05:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.554+0000 7f2027fff640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200806b4d0 0x7f200806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.554+0000 7f2027fff640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200806b4d0 0x7f200806d990 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2018034000 tx=0x7f201802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.558+0000 7f2024ff9640 1 -- 172.21.15.19:0/2248581985 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20100182b0 con 0x7f20280d0ad0 2024-04-25T10:05:41.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.737+0000 7f202e6ac640 1 -- 172.21.15.19:0/2248581985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20280cd690 con 0x7f20280d0ad0 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.739+0000 7f2024ff9640 1 -- 172.21.15.19:0/2248581985 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f2010058d10 con 0x7f20280d0ad0 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":189,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:40.072490+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:36.700326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:37.402558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:41.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.740+0000 7f202e6ac640 1 -- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200806b4d0 msgr2=0x7f200806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.740+0000 7f202e6ac640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200806b4d0 0x7f200806d990 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2018034000 tx=0x7f201802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:41.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.740+0000 7f202e6ac640 1 -- 172.21.15.19:0/2248581985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20280d0ad0 msgr2=0x7f20280cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.740+0000 7f202e6ac640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20280d0ad0 0x7f20280cd150 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f201000c4d0 tx=0x7f201000c990 comp rx=0 tx=0).stop 2024-04-25T10:05:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.741+0000 7f202e6ac640 1 -- 172.21.15.19:0/2248581985 shutdown_connections 2024-04-25T10:05:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.741+0000 7f202e6ac640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200806b4d0 0x7f200806d990 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.741+0000 7f202e6ac640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f202815dd30 0x7f2028152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.741+0000 7f202e6ac640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20280d0ad0 0x7f20280cd150 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.741+0000 7f202e6ac640 1 --2- 172.21.15.19:0/2248581985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20280b9770 0x7f20280b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.741+0000 7f202e6ac640 1 -- 172.21.15.19:0/2248581985 >> 172.21.15.19:0/2248581985 conn(0x7f2028061c30 msgr2=0x7f20280c8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.741+0000 7f202e6ac640 1 -- 172.21.15.19:0/2248581985 shutdown_connections 2024-04-25T10:05:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.741+0000 7f202e6ac640 1 -- 172.21.15.19:0/2248581985 wait complete. 2024-04-25T10:05:41.757 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:05:41.758 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.852+0000 7f8aefd63640 1 Processor -- start 2024-04-25T10:05:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.852+0000 7f8aefd63640 1 -- start start 2024-04-25T10:05:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.853+0000 7f8aefd63640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ae815db70 0x7f8ae8162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.853+0000 7f8aefd63640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8ae80b5c30 0x7f8ae80d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.853+0000 7f8aefd63640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae8162690 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.853+0000 7f8aefd63640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae80ceee0 con 0x7f8ae815db70 2024-04-25T10:05:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.853+0000 7f8aefd63640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae80d1d10 con 0x7f8ae80b5c30 2024-04-25T10:05:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.853+0000 7f8aedaff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ae815db70 0x7f8ae8162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.853+0000 7f8aed2fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8ae80b6cb0 0x7f8ae80b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34884/0 (socket says 172.21.15.19:34884) 2024-04-25T10:05:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.853+0000 7f8aed2fe640 1 -- 172.21.15.19:0/836490443 learned_addr learned my addr 172.21.15.19:0/836490443 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.854+0000 7f8aecafd640 1 -- 172.21.15.19:0/836490443 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11823873 0 0) 0x7f8ae8162690 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.854+0000 7f8aee300640 1 --2- 172.21.15.19:0/836490443 >> v2:172.21.15.112:3300/0 conn(0x7f8ae80b5c30 0x7f8ae80d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.854+0000 7f8aecafd640 1 -- 172.21.15.19:0/836490443 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad4003520 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.854+0000 7f8aedaff640 1 -- 172.21.15.19:0/836490443 >> v2:172.21.15.112:3300/0 conn(0x7f8ae80b5c30 msgr2=0x7f8ae80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.854+0000 7f8aedaff640 1 --2- 172.21.15.19:0/836490443 >> v2:172.21.15.112:3300/0 conn(0x7f8ae80b5c30 0x7f8ae80d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.854+0000 7f8aedaff640 1 -- 172.21.15.19:0/836490443 >> v1:172.21.15.19:6789/0 conn(0x7f8ae80b6cb0 legacy=0x7f8ae80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.854+0000 7f8aedaff640 1 -- 172.21.15.19:0/836490443 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80d1ff0 con 0x7f8ae815db70 2024-04-25T10:05:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.854+0000 7f8aee300640 1 --2- 172.21.15.19:0/836490443 >> v2:172.21.15.112:3300/0 conn(0x7f8ae80b5c30 0x7f8ae80d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.854+0000 7f8aedaff640 1 --2- 172.21.15.19:0/836490443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae815db70 0x7f8ae8162150 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f8ae808ccc0 tx=0x7f8ae402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1677b7cac1a85a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.855+0000 7f8aecafd640 1 -- 172.21.15.19:0/836490443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4037070 con 0x7f8ae815db70 2024-04-25T10:05:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.855+0000 7f8aecafd640 1 -- 172.21.15.19:0/836490443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae4032440 con 0x7f8ae815db70 2024-04-25T10:05:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.855+0000 7f8aecafd640 1 -- 172.21.15.19:0/836490443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4032810 con 0x7f8ae815db70 2024-04-25T10:05:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.855+0000 7f8aefd63640 1 -- 172.21.15.19:0/836490443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae815db70 msgr2=0x7f8ae8162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.855+0000 7f8aefd63640 1 --2- 172.21.15.19:0/836490443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae815db70 0x7f8ae8162150 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f8ae808ccc0 tx=0x7f8ae402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.855+0000 7f8aefd63640 1 -- 172.21.15.19:0/836490443 shutdown_connections 2024-04-25T10:05:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.855+0000 7f8aefd63640 1 --2- 172.21.15.19:0/836490443 >> v2:172.21.15.112:3300/0 conn(0x7f8ae80b5c30 0x7f8ae80d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.855+0000 7f8aefd63640 1 --2- 172.21.15.19:0/836490443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae815db70 0x7f8ae8162150 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.855+0000 7f8aefd63640 1 -- 172.21.15.19:0/836490443 >> 172.21.15.19:0/836490443 conn(0x7f8ae80d0520 msgr2=0x7f8ae80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.855+0000 7f8aefd63640 1 -- 172.21.15.19:0/836490443 shutdown_connections 2024-04-25T10:05:41.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.855+0000 7f8aefd63640 1 -- 172.21.15.19:0/836490443 wait complete. 2024-04-25T10:05:41.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.857+0000 7f8aefd63640 1 Processor -- start 2024-04-25T10:05:41.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.857+0000 7f8aefd63640 1 -- start start 2024-04-25T10:05:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.857+0000 7f8aefd63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ae80b5c30 0x7f8ae80c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.857+0000 7f8aefd63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae80b6cb0 0x7f8ae815c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.857+0000 7f8aefd63640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ae815db70 0x7f8ae815c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.857+0000 7f8aefd63640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80ce530 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.857+0000 7f8aefd63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae815df50 con 0x7f8ae80b5c30 2024-04-25T10:05:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.857+0000 7f8aefd63640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80cf1c0 con 0x7f8ae815db70 2024-04-25T10:05:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.857+0000 7f8aed2fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae80b6cb0 0x7f8ae815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.857+0000 7f8aed2fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae80b6cb0 0x7f8ae815c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35776/0 (socket says 172.21.15.19:35776) 2024-04-25T10:05:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.857+0000 7f8aed2fe640 1 -- 172.21.15.19:0/3021174106 learned_addr learned my addr 172.21.15.19:0/3021174106 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8aee300640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ae815db70 0x7f8ae815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8aedaff640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ae80b5c30 0x7f8ae80c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8aed2fe640 1 -- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ae815db70 msgr2=0x7f8ae815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8aed2fe640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ae815db70 0x7f8ae815c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8aed2fe640 1 -- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ae80b5c30 msgr2=0x7f8ae80c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8aed2fe640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ae80b5c30 0x7f8ae80c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8aed2fe640 1 -- 172.21.15.19:0/3021174106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae81550f0 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8aed2fe640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae80b6cb0 0x7f8ae815c240 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800c400 tx=0x7f8ad800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8ade7fc640 1 -- 172.21.15.19:0/3021174106 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8017070 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8ade7fc640 1 -- 172.21.15.19:0/3021174106 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8005d00 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8aefd63640 1 -- 172.21.15.19:0/3021174106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae81553d0 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8ade7fc640 1 -- 172.21.15.19:0/3021174106 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8007310 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.858+0000 7f8aefd63640 1 -- 172.21.15.19:0/3021174106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae815e0d0 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.859+0000 7f8aefd63640 1 -- 172.21.15.19:0/3021174106 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae8073990 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.860+0000 7f8ade7fc640 1 -- 172.21.15.19:0/3021174106 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ad801a070 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.860+0000 7f8ade7fc640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8acc06b4d0 0x7f8acc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.860+0000 7f8ade7fc640 1 -- 172.21.15.19:0/3021174106 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ad80896e0 con 0x7f8ae80b6cb0 2024-04-25T10:05:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.860+0000 7f8aedaff640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8acc06b4d0 0x7f8acc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.861+0000 7f8aedaff640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8acc06b4d0 0x7f8acc06d990 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4034000 tx=0x7f8ae402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:41.865+0000 7f8ade7fc640 1 -- 172.21.15.19:0/3021174106 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ad8057e10 con 0x7f8ae80b6cb0 2024-04-25T10:05:42.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.042+0000 7f8aefd63640 1 -- 172.21.15.19:0/3021174106 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ae80ce310 con 0x7f8acc06b4d0 2024-04-25T10:05:42.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.049+0000 7f8ade7fc640 1 -- 172.21.15.19:0/3021174106 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247082 (secure 0 0 0) 0x7f8ae80ce310 con 0x7f8acc06b4d0 2024-04-25T10:05:42.057 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:42.059 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:05:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 -- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8acc06b4d0 msgr2=0x7f8acc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8acc06b4d0 0x7f8acc06d990 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4034000 tx=0x7f8ae402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 -- 172.21.15.19:0/3021174106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae80b6cb0 msgr2=0x7f8ae815c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae80b6cb0 0x7f8ae815c240 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800c400 tx=0x7f8ad800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 -- 172.21.15.19:0/3021174106 shutdown_connections 2024-04-25T10:05:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8acc06b4d0 0x7f8acc06d990 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8ae815db70 0x7f8ae815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae80b6cb0 0x7f8ae815c240 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 --2- 172.21.15.19:0/3021174106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ae80b5c30 0x7f8ae80c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 -- 172.21.15.19:0/3021174106 >> 172.21.15.19:0/3021174106 conn(0x7f8ae80d0520 msgr2=0x7f8ae80d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 -- 172.21.15.19:0/3021174106 shutdown_connections 2024-04-25T10:05:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.052+0000 7f8aefd63640 1 -- 172.21.15.19:0/3021174106 wait complete. 2024-04-25T10:05:42.069 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":566,"stamp":"2024-04-25T10:05:41.088763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":446772,"kb_used_data":19644,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754527948,"statfs":{"total":773094113280,"available":772636618752,"internally_reserved":0,"allocated":20115456,"data_stored":10057341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":437243442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035420"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066637+0000","last_change":"2024-04-25T10:05:38.066637+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066637+0000","last_undegraded":"2024-04-25T10:05:38.066637+0000","last_fullsized":"2024-04-25T10:05:38.066637+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074792+0000","last_change":"2024-04-25T10:05:38.074792+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074792+0000","last_undegraded":"2024-04-25T10:05:38.074792+0000","last_fullsized":"2024-04-25T10:05:38.074792+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077295+0000","last_change":"2024-04-25T10:05:38.077295+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077295+0000","last_undegraded":"2024-04-25T10:05:38.077295+0000","last_fullsized":"2024-04-25T10:05:38.077295+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067681+0000","last_change":"2024-04-25T10:05:38.067681+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067681+0000","last_undegraded":"2024-04-25T10:05:38.067681+0000","last_fullsized":"2024-04-25T10:05:38.067681+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066530+0000","last_change":"2024-04-25T10:05:38.066530+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066530+0000","last_undegraded":"2024-04-25T10:05:38.066530+0000","last_fullsized":"2024-04-25T10:05:38.066530+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071870+0000","last_change":"2024-04-25T10:05:38.071870+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071870+0000","last_undegraded":"2024-04-25T10:05:38.071870+0000","last_fullsized":"2024-04-25T10:05:38.071870+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077706+0000","last_change":"2024-04-25T10:05:38.077706+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077706+0000","last_undegraded":"2024-04-25T10:05:38.077706+0000","last_fullsized":"2024-04-25T10:05:38.077706+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071345+0000","last_change":"2024-04-25T10:05:38.071345+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071345+0000","last_undegraded":"2024-04-25T10:05:38.071345+0000","last_fullsized":"2024-04-25T10:05:38.071345+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076215+0000","last_change":"2024-04-25T10:05:38.076215+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076215+0000","last_undegraded":"2024-04-25T10:05:38.076215+0000","last_fullsized":"2024-04-25T10:05:38.076215+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.081044+0000","last_change":"2024-04-25T10:05:38.081044+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.081044+0000","last_undegraded":"2024-04-25T10:05:38.081044+0000","last_fullsized":"2024-04-25T10:05:38.081044+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.068089+0000","last_change":"2024-04-25T10:05:38.068089+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.068089+0000","last_undegraded":"2024-04-25T10:05:38.068089+0000","last_fullsized":"2024-04-25T10:05:38.068089+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076169+0000","last_change":"2024-04-25T10:05:38.076169+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076169+0000","last_undegraded":"2024-04-25T10:05:38.076169+0000","last_fullsized":"2024-04-25T10:05:38.076169+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070409+0000","last_change":"2024-04-25T10:05:38.070409+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070409+0000","last_undegraded":"2024-04-25T10:05:38.070409+0000","last_fullsized":"2024-04-25T10:05:38.070409+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076413+0000","last_change":"2024-04-25T10:05:38.076413+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076413+0000","last_undegraded":"2024-04-25T10:05:38.076413+0000","last_fullsized":"2024-04-25T10:05:38.076413+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076156+0000","last_change":"2024-04-25T10:05:38.076156+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076156+0000","last_undegraded":"2024-04-25T10:05:38.076156+0000","last_fullsized":"2024-04-25T10:05:38.076156+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070878+0000","last_change":"2024-04-25T10:05:38.070878+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070878+0000","last_undegraded":"2024-04-25T10:05:38.070878+0000","last_fullsized":"2024-04-25T10:05:38.070878+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078479+0000","last_change":"2024-04-25T10:05:38.078479+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078479+0000","last_undegraded":"2024-04-25T10:05:38.078479+0000","last_fullsized":"2024-04-25T10:05:38.078479+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.080617+0000","last_change":"2024-04-25T10:05:38.080617+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.080617+0000","last_undegraded":"2024-04-25T10:05:38.080617+0000","last_fullsized":"2024-04-25T10:05:38.080617+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065954+0000","last_change":"2024-04-25T10:05:38.065954+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065954+0000","last_undegraded":"2024-04-25T10:05:38.065954+0000","last_fullsized":"2024-04-25T10:05:38.065954+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070674+0000","last_change":"2024-04-25T10:05:38.070674+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070674+0000","last_undegraded":"2024-04-25T10:05:38.070674+0000","last_fullsized":"2024-04-25T10:05:38.070674+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077115+0000","last_change":"2024-04-25T10:05:38.077115+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077115+0000","last_undegraded":"2024-04-25T10:05:38.077115+0000","last_fullsized":"2024-04-25T10:05:38.077115+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076601+0000","last_change":"2024-04-25T10:05:38.076601+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076601+0000","last_undegraded":"2024-04-25T10:05:38.076601+0000","last_fullsized":"2024-04-25T10:05:38.076601+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075635+0000","last_change":"2024-04-25T10:05:38.075635+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075635+0000","last_undegraded":"2024-04-25T10:05:38.075635+0000","last_fullsized":"2024-04-25T10:05:38.075635+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065899+0000","last_change":"2024-04-25T10:05:38.065899+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065899+0000","last_undegraded":"2024-04-25T10:05:38.065899+0000","last_fullsized":"2024-04-25T10:05:38.065899+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065933+0000","last_change":"2024-04-25T10:05:38.065933+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065933+0000","last_undegraded":"2024-04-25T10:05:38.065933+0000","last_fullsized":"2024-04-25T10:05:38.065933+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.080059+0000","last_change":"2024-04-25T10:05:38.080059+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.080059+0000","last_undegraded":"2024-04-25T10:05:38.080059+0000","last_fullsized":"2024-04-25T10:05:38.080059+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067528+0000","last_change":"2024-04-25T10:05:38.067528+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067528+0000","last_undegraded":"2024-04-25T10:05:38.067528+0000","last_fullsized":"2024-04-25T10:05:38.067528+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074828+0000","last_change":"2024-04-25T10:05:38.074828+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074828+0000","last_undegraded":"2024-04-25T10:05:38.074828+0000","last_fullsized":"2024-04-25T10:05:38.074828+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074206+0000","last_change":"2024-04-25T10:05:38.074206+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074206+0000","last_undegraded":"2024-04-25T10:05:38.074206+0000","last_fullsized":"2024-04-25T10:05:38.074206+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.072514+0000","last_change":"2024-04-25T10:05:38.072514+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.072514+0000","last_undegraded":"2024-04-25T10:05:38.072514+0000","last_fullsized":"2024-04-25T10:05:38.072514+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074387+0000","last_change":"2024-04-25T10:05:38.074387+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074387+0000","last_undegraded":"2024-04-25T10:05:38.074387+0000","last_fullsized":"2024-04-25T10:05:38.074387+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070297+0000","last_change":"2024-04-25T10:05:38.070297+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070297+0000","last_undegraded":"2024-04-25T10:05:38.070297+0000","last_fullsized":"2024-04-25T10:05:38.070297+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077732+0000","last_change":"2024-04-25T10:05:38.077732+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077732+0000","last_undegraded":"2024-04-25T10:05:38.077732+0000","last_fullsized":"2024-04-25T10:05:38.077732+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.086219+0000","last_change":"2024-04-25T10:05:38.085541+0000","last_active":"2024-04-25T10:05:38.086219+0000","last_peered":"2024-04-25T10:05:38.086219+0000","last_clean":"2024-04-25T10:05:38.086219+0000","last_became_active":"2024-04-25T10:05:38.085242+0000","last_became_peered":"2024-04-25T10:05:38.085242+0000","last_unstale":"2024-04-25T10:05:38.086219+0000","last_undegraded":"2024-04-25T10:05:38.086219+0000","last_fullsized":"2024-04-25T10:05:38.086219+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:07.815434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076803+0000","last_change":"2024-04-25T10:05:38.076803+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076803+0000","last_undegraded":"2024-04-25T10:05:38.076803+0000","last_fullsized":"2024-04-25T10:05:38.076803+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084928+0000","last_change":"2024-04-25T10:05:38.081472+0000","last_active":"2024-04-25T10:05:38.084928+0000","last_peered":"2024-04-25T10:05:38.084928+0000","last_clean":"2024-04-25T10:05:38.084928+0000","last_became_active":"2024-04-25T10:05:38.081187+0000","last_became_peered":"2024-04-25T10:05:38.081187+0000","last_unstale":"2024-04-25T10:05:38.084928+0000","last_undegraded":"2024-04-25T10:05:38.084928+0000","last_fullsized":"2024-04-25T10:05:38.084928+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:38.872584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074333+0000","last_change":"2024-04-25T10:05:38.074333+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074333+0000","last_undegraded":"2024-04-25T10:05:38.074333+0000","last_fullsized":"2024-04-25T10:05:38.074333+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.080465+0000","last_change":"2024-04-25T10:05:38.079620+0000","last_active":"2024-04-25T10:05:38.080465+0000","last_peered":"2024-04-25T10:05:38.080465+0000","last_clean":"2024-04-25T10:05:38.080465+0000","last_became_active":"2024-04-25T10:05:38.079361+0000","last_became_peered":"2024-04-25T10:05:38.079361+0000","last_unstale":"2024-04-25T10:05:38.080465+0000","last_undegraded":"2024-04-25T10:05:38.080465+0000","last_fullsized":"2024-04-25T10:05:38.080465+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:46.032506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079284+0000","last_change":"2024-04-25T10:05:38.079284+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079284+0000","last_undegraded":"2024-04-25T10:05:38.079284+0000","last_fullsized":"2024-04-25T10:05:38.079284+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084474+0000","last_change":"2024-04-25T10:05:38.083940+0000","last_active":"2024-04-25T10:05:38.084474+0000","last_peered":"2024-04-25T10:05:38.084474+0000","last_clean":"2024-04-25T10:05:38.084474+0000","last_became_active":"2024-04-25T10:05:38.083619+0000","last_became_peered":"2024-04-25T10:05:38.083619+0000","last_unstale":"2024-04-25T10:05:38.084474+0000","last_undegraded":"2024-04-25T10:05:38.084474+0000","last_fullsized":"2024-04-25T10:05:38.084474+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.073144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073657+0000","last_change":"2024-04-25T10:05:38.073657+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073657+0000","last_undegraded":"2024-04-25T10:05:38.073657+0000","last_fullsized":"2024-04-25T10:05:38.073657+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.083185+0000","last_change":"2024-04-25T10:05:38.079975+0000","last_active":"2024-04-25T10:05:38.083185+0000","last_peered":"2024-04-25T10:05:38.083185+0000","last_clean":"2024-04-25T10:05:38.083185+0000","last_became_active":"2024-04-25T10:05:38.079671+0000","last_became_peered":"2024-04-25T10:05:38.079671+0000","last_unstale":"2024-04-25T10:05:38.083185+0000","last_undegraded":"2024-04-25T10:05:38.083185+0000","last_fullsized":"2024-04-25T10:05:38.083185+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:43.802225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074714+0000","last_change":"2024-04-25T10:05:38.074714+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074714+0000","last_undegraded":"2024-04-25T10:05:38.074714+0000","last_fullsized":"2024-04-25T10:05:38.074714+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084915+0000","last_change":"2024-04-25T10:05:38.082341+0000","last_active":"2024-04-25T10:05:38.084915+0000","last_peered":"2024-04-25T10:05:38.084915+0000","last_clean":"2024-04-25T10:05:38.084915+0000","last_became_active":"2024-04-25T10:05:38.082061+0000","last_became_peered":"2024-04-25T10:05:38.082061+0000","last_unstale":"2024-04-25T10:05:38.084915+0000","last_undegraded":"2024-04-25T10:05:38.084915+0000","last_fullsized":"2024-04-25T10:05:38.084915+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:28.373524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079217+0000","last_change":"2024-04-25T10:05:38.079217+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079217+0000","last_undegraded":"2024-04-25T10:05:38.079217+0000","last_fullsized":"2024-04-25T10:05:38.079217+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081588+0000","last_change":"2024-04-25T10:05:38.080620+0000","last_active":"2024-04-25T10:05:38.081588+0000","last_peered":"2024-04-25T10:05:38.081588+0000","last_clean":"2024-04-25T10:05:38.081588+0000","last_became_active":"2024-04-25T10:05:38.075233+0000","last_became_peered":"2024-04-25T10:05:38.075233+0000","last_unstale":"2024-04-25T10:05:38.081588+0000","last_undegraded":"2024-04-25T10:05:38.081588+0000","last_fullsized":"2024-04-25T10:05:38.081588+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:49.754709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073656+0000","last_change":"2024-04-25T10:05:38.073656+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073656+0000","last_undegraded":"2024-04-25T10:05:38.073656+0000","last_fullsized":"2024-04-25T10:05:38.073656+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.075711+0000","last_change":"2024-04-25T10:05:38.074918+0000","last_active":"2024-04-25T10:05:38.075711+0000","last_peered":"2024-04-25T10:05:38.075711+0000","last_clean":"2024-04-25T10:05:38.075711+0000","last_became_active":"2024-04-25T10:05:38.074613+0000","last_became_peered":"2024-04-25T10:05:38.074613+0000","last_unstale":"2024-04-25T10:05:38.075711+0000","last_undegraded":"2024-04-25T10:05:38.075711+0000","last_fullsized":"2024-04-25T10:05:38.075711+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:54.282381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074188+0000","last_change":"2024-04-25T10:05:38.074188+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074188+0000","last_undegraded":"2024-04-25T10:05:38.074188+0000","last_fullsized":"2024-04-25T10:05:38.074188+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084181+0000","last_change":"2024-04-25T10:05:38.082985+0000","last_active":"2024-04-25T10:05:38.084181+0000","last_peered":"2024-04-25T10:05:38.084181+0000","last_clean":"2024-04-25T10:05:38.084181+0000","last_became_active":"2024-04-25T10:05:38.082719+0000","last_became_peered":"2024-04-25T10:05:38.082719+0000","last_unstale":"2024-04-25T10:05:38.084181+0000","last_undegraded":"2024-04-25T10:05:38.084181+0000","last_fullsized":"2024-04-25T10:05:38.084181+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:40.200742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079561+0000","last_change":"2024-04-25T10:05:38.079561+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079561+0000","last_undegraded":"2024-04-25T10:05:38.079561+0000","last_fullsized":"2024-04-25T10:05:38.079561+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076035+0000","last_change":"2024-04-25T10:05:38.075322+0000","last_active":"2024-04-25T10:05:38.076035+0000","last_peered":"2024-04-25T10:05:38.076035+0000","last_clean":"2024-04-25T10:05:38.076035+0000","last_became_active":"2024-04-25T10:05:38.074688+0000","last_became_peered":"2024-04-25T10:05:38.074688+0000","last_unstale":"2024-04-25T10:05:38.076035+0000","last_undegraded":"2024-04-25T10:05:38.076035+0000","last_fullsized":"2024-04-25T10:05:38.076035+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:10.878156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.068047+0000","last_change":"2024-04-25T10:05:38.068047+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.068047+0000","last_undegraded":"2024-04-25T10:05:38.068047+0000","last_fullsized":"2024-04-25T10:05:38.068047+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081990+0000","last_change":"2024-04-25T10:05:38.080464+0000","last_active":"2024-04-25T10:05:38.081990+0000","last_peered":"2024-04-25T10:05:38.081990+0000","last_clean":"2024-04-25T10:05:38.081990+0000","last_became_active":"2024-04-25T10:05:38.080184+0000","last_became_peered":"2024-04-25T10:05:38.080184+0000","last_unstale":"2024-04-25T10:05:38.081990+0000","last_undegraded":"2024-04-25T10:05:38.081990+0000","last_fullsized":"2024-04-25T10:05:38.081990+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:06.519993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074213+0000","last_change":"2024-04-25T10:05:38.074213+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074213+0000","last_undegraded":"2024-04-25T10:05:38.074213+0000","last_fullsized":"2024-04-25T10:05:38.074213+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.087722+0000","last_change":"2024-04-25T10:05:38.087415+0000","last_active":"2024-04-25T10:05:38.087722+0000","last_peered":"2024-04-25T10:05:38.087722+0000","last_clean":"2024-04-25T10:05:38.087722+0000","last_became_active":"2024-04-25T10:05:38.087167+0000","last_became_peered":"2024-04-25T10:05:38.087167+0000","last_unstale":"2024-04-25T10:05:38.087722+0000","last_undegraded":"2024-04-25T10:05:38.087722+0000","last_fullsized":"2024-04-25T10:05:38.087722+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:30.872314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073003+0000","last_change":"2024-04-25T10:05:38.073003+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073003+0000","last_undegraded":"2024-04-25T10:05:38.073003+0000","last_fullsized":"2024-04-25T10:05:38.073003+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084546+0000","last_change":"2024-04-25T10:05:38.081783+0000","last_active":"2024-04-25T10:05:38.084546+0000","last_peered":"2024-04-25T10:05:38.084546+0000","last_clean":"2024-04-25T10:05:38.084546+0000","last_became_active":"2024-04-25T10:05:38.081463+0000","last_became_peered":"2024-04-25T10:05:38.081463+0000","last_unstale":"2024-04-25T10:05:38.084546+0000","last_undegraded":"2024-04-25T10:05:38.084546+0000","last_fullsized":"2024-04-25T10:05:38.084546+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:10.390750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077577+0000","last_change":"2024-04-25T10:05:38.077577+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077577+0000","last_undegraded":"2024-04-25T10:05:38.077577+0000","last_fullsized":"2024-04-25T10:05:38.077577+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.077536+0000","last_change":"2024-04-25T10:05:38.076940+0000","last_active":"2024-04-25T10:05:38.077536+0000","last_peered":"2024-04-25T10:05:38.077536+0000","last_clean":"2024-04-25T10:05:38.077536+0000","last_became_active":"2024-04-25T10:05:38.076639+0000","last_became_peered":"2024-04-25T10:05:38.076639+0000","last_unstale":"2024-04-25T10:05:38.077536+0000","last_undegraded":"2024-04-25T10:05:38.077536+0000","last_fullsized":"2024-04-25T10:05:38.077536+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:20.840458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073109+0000","last_change":"2024-04-25T10:05:38.073109+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073109+0000","last_undegraded":"2024-04-25T10:05:38.073109+0000","last_fullsized":"2024-04-25T10:05:38.073109+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085058+0000","last_change":"2024-04-25T10:05:38.083896+0000","last_active":"2024-04-25T10:05:38.085058+0000","last_peered":"2024-04-25T10:05:38.085058+0000","last_clean":"2024-04-25T10:05:38.085058+0000","last_became_active":"2024-04-25T10:05:38.083309+0000","last_became_peered":"2024-04-25T10:05:38.083309+0000","last_unstale":"2024-04-25T10:05:38.085058+0000","last_undegraded":"2024-04-25T10:05:38.085058+0000","last_fullsized":"2024-04-25T10:05:38.085058+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:27.679738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074786+0000","last_change":"2024-04-25T10:05:38.074786+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074786+0000","last_undegraded":"2024-04-25T10:05:38.074786+0000","last_fullsized":"2024-04-25T10:05:38.074786+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084860+0000","last_change":"2024-04-25T10:05:38.083645+0000","last_active":"2024-04-25T10:05:38.084860+0000","last_peered":"2024-04-25T10:05:38.084860+0000","last_clean":"2024-04-25T10:05:38.084860+0000","last_became_active":"2024-04-25T10:05:38.083357+0000","last_became_peered":"2024-04-25T10:05:38.083357+0000","last_unstale":"2024-04-25T10:05:38.084860+0000","last_undegraded":"2024-04-25T10:05:38.084860+0000","last_fullsized":"2024-04-25T10:05:38.084860+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:32.268641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067142+0000","last_change":"2024-04-25T10:05:38.067142+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067142+0000","last_undegraded":"2024-04-25T10:05:38.067142+0000","last_fullsized":"2024-04-25T10:05:38.067142+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085150+0000","last_change":"2024-04-25T10:05:38.083600+0000","last_active":"2024-04-25T10:05:38.085150+0000","last_peered":"2024-04-25T10:05:38.085150+0000","last_clean":"2024-04-25T10:05:38.085150+0000","last_became_active":"2024-04-25T10:05:38.083303+0000","last_became_peered":"2024-04-25T10:05:38.083303+0000","last_unstale":"2024-04-25T10:05:38.085150+0000","last_undegraded":"2024-04-25T10:05:38.085150+0000","last_fullsized":"2024-04-25T10:05:38.085150+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:05.866090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078449+0000","last_change":"2024-04-25T10:05:38.078449+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078449+0000","last_undegraded":"2024-04-25T10:05:38.078449+0000","last_fullsized":"2024-04-25T10:05:38.078449+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084965+0000","last_change":"2024-04-25T10:05:38.081964+0000","last_active":"2024-04-25T10:05:38.084965+0000","last_peered":"2024-04-25T10:05:38.084965+0000","last_clean":"2024-04-25T10:05:38.084965+0000","last_became_active":"2024-04-25T10:05:38.081673+0000","last_became_peered":"2024-04-25T10:05:38.081673+0000","last_unstale":"2024-04-25T10:05:38.084965+0000","last_undegraded":"2024-04-25T10:05:38.084965+0000","last_fullsized":"2024-04-25T10:05:38.084965+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.599553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078013+0000","last_change":"2024-04-25T10:05:38.078013+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078013+0000","last_undegraded":"2024-04-25T10:05:38.078013+0000","last_fullsized":"2024-04-25T10:05:38.078013+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085904+0000","last_change":"2024-04-25T10:05:38.085345+0000","last_active":"2024-04-25T10:05:38.085904+0000","last_peered":"2024-04-25T10:05:38.085904+0000","last_clean":"2024-04-25T10:05:38.085904+0000","last_became_active":"2024-04-25T10:05:38.082298+0000","last_became_peered":"2024-04-25T10:05:38.082298+0000","last_unstale":"2024-04-25T10:05:38.085904+0000","last_undegraded":"2024-04-25T10:05:38.085904+0000","last_fullsized":"2024-04-25T10:05:38.085904+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:46.424209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074518+0000","last_change":"2024-04-25T10:05:38.074518+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074518+0000","last_undegraded":"2024-04-25T10:05:38.074518+0000","last_fullsized":"2024-04-25T10:05:38.074518+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084654+0000","last_change":"2024-04-25T10:05:38.083542+0000","last_active":"2024-04-25T10:05:38.084654+0000","last_peered":"2024-04-25T10:05:38.084654+0000","last_clean":"2024-04-25T10:05:38.084654+0000","last_became_active":"2024-04-25T10:05:38.082652+0000","last_became_peered":"2024-04-25T10:05:38.082652+0000","last_unstale":"2024-04-25T10:05:38.084654+0000","last_undegraded":"2024-04-25T10:05:38.084654+0000","last_fullsized":"2024-04-25T10:05:38.084654+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:36.918168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065904+0000","last_change":"2024-04-25T10:05:38.065904+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065904+0000","last_undegraded":"2024-04-25T10:05:38.065904+0000","last_fullsized":"2024-04-25T10:05:38.065904+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.080166+0000","last_change":"2024-04-25T10:05:38.077397+0000","last_active":"2024-04-25T10:05:38.080166+0000","last_peered":"2024-04-25T10:05:38.080166+0000","last_clean":"2024-04-25T10:05:38.080166+0000","last_became_active":"2024-04-25T10:05:38.077107+0000","last_became_peered":"2024-04-25T10:05:38.077107+0000","last_unstale":"2024-04-25T10:05:38.080166+0000","last_undegraded":"2024-04-25T10:05:38.080166+0000","last_fullsized":"2024-04-25T10:05:38.080166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:43.802225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073892+0000","last_change":"2024-04-25T10:05:38.073892+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073892+0000","last_undegraded":"2024-04-25T10:05:38.073892+0000","last_fullsized":"2024-04-25T10:05:38.073892+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.079630+0000","last_change":"2024-04-25T10:05:38.075167+0000","last_active":"2024-04-25T10:05:38.079630+0000","last_peered":"2024-04-25T10:05:38.079630+0000","last_clean":"2024-04-25T10:05:38.079630+0000","last_became_active":"2024-04-25T10:05:38.074890+0000","last_became_peered":"2024-04-25T10:05:38.074890+0000","last_unstale":"2024-04-25T10:05:38.079630+0000","last_undegraded":"2024-04-25T10:05:38.079630+0000","last_fullsized":"2024-04-25T10:05:38.079630+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:00.798323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076641+0000","last_change":"2024-04-25T10:05:38.076641+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076641+0000","last_undegraded":"2024-04-25T10:05:38.076641+0000","last_fullsized":"2024-04-25T10:05:38.076641+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084195+0000","last_change":"2024-04-25T10:05:38.082353+0000","last_active":"2024-04-25T10:05:38.084195+0000","last_peered":"2024-04-25T10:05:38.084195+0000","last_clean":"2024-04-25T10:05:38.084195+0000","last_became_active":"2024-04-25T10:05:38.082047+0000","last_became_peered":"2024-04-25T10:05:38.082047+0000","last_unstale":"2024-04-25T10:05:38.084195+0000","last_undegraded":"2024-04-25T10:05:38.084195+0000","last_fullsized":"2024-04-25T10:05:38.084195+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:12.478664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074243+0000","last_change":"2024-04-25T10:05:38.074243+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074243+0000","last_undegraded":"2024-04-25T10:05:38.074243+0000","last_fullsized":"2024-04-25T10:05:38.074243+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085023+0000","last_change":"2024-04-25T10:05:38.083029+0000","last_active":"2024-04-25T10:05:38.085023+0000","last_peered":"2024-04-25T10:05:38.085023+0000","last_clean":"2024-04-25T10:05:38.085023+0000","last_became_active":"2024-04-25T10:05:38.082535+0000","last_became_peered":"2024-04-25T10:05:38.082535+0000","last_unstale":"2024-04-25T10:05:38.085023+0000","last_undegraded":"2024-04-25T10:05:38.085023+0000","last_fullsized":"2024-04-25T10:05:38.085023+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:42.531775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066667+0000","last_change":"2024-04-25T10:05:38.066667+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066667+0000","last_undegraded":"2024-04-25T10:05:38.066667+0000","last_fullsized":"2024-04-25T10:05:38.066667+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076016+0000","last_change":"2024-04-25T10:05:38.073711+0000","last_active":"2024-04-25T10:05:38.076016+0000","last_peered":"2024-04-25T10:05:38.076016+0000","last_clean":"2024-04-25T10:05:38.076016+0000","last_became_active":"2024-04-25T10:05:38.073414+0000","last_became_peered":"2024-04-25T10:05:38.073414+0000","last_unstale":"2024-04-25T10:05:38.076016+0000","last_undegraded":"2024-04-25T10:05:38.076016+0000","last_fullsized":"2024-04-25T10:05:38.076016+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:10.878156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071187+0000","last_change":"2024-04-25T10:05:38.071187+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071187+0000","last_undegraded":"2024-04-25T10:05:38.071187+0000","last_fullsized":"2024-04-25T10:05:38.071187+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.073860+0000","last_change":"2024-04-25T10:05:38.071762+0000","last_active":"2024-04-25T10:05:38.073860+0000","last_peered":"2024-04-25T10:05:38.073860+0000","last_clean":"2024-04-25T10:05:38.073860+0000","last_became_active":"2024-04-25T10:05:38.071429+0000","last_became_peered":"2024-04-25T10:05:38.071429+0000","last_unstale":"2024-04-25T10:05:38.073860+0000","last_undegraded":"2024-04-25T10:05:38.073860+0000","last_fullsized":"2024-04-25T10:05:38.073860+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:54.282381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077423+0000","last_change":"2024-04-25T10:05:38.077423+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077423+0000","last_undegraded":"2024-04-25T10:05:38.077423+0000","last_fullsized":"2024-04-25T10:05:38.077423+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085218+0000","last_change":"2024-04-25T10:05:38.084273+0000","last_active":"2024-04-25T10:05:38.085218+0000","last_peered":"2024-04-25T10:05:38.085218+0000","last_clean":"2024-04-25T10:05:38.085218+0000","last_became_active":"2024-04-25T10:05:38.082528+0000","last_became_peered":"2024-04-25T10:05:38.082528+0000","last_unstale":"2024-04-25T10:05:38.085218+0000","last_undegraded":"2024-04-25T10:05:38.085218+0000","last_fullsized":"2024-04-25T10:05:38.085218+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:29.382293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.072844+0000","last_change":"2024-04-25T10:05:38.072844+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.072844+0000","last_undegraded":"2024-04-25T10:05:38.072844+0000","last_fullsized":"2024-04-25T10:05:38.072844+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.087513+0000","last_change":"2024-04-25T10:05:38.086909+0000","last_active":"2024-04-25T10:05:38.087513+0000","last_peered":"2024-04-25T10:05:38.087513+0000","last_clean":"2024-04-25T10:05:38.087513+0000","last_became_active":"2024-04-25T10:05:38.086646+0000","last_became_peered":"2024-04-25T10:05:38.086646+0000","last_unstale":"2024-04-25T10:05:38.087513+0000","last_undegraded":"2024-04-25T10:05:38.087513+0000","last_fullsized":"2024-04-25T10:05:38.087513+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:08.234904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075592+0000","last_change":"2024-04-25T10:05:38.075592+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075592+0000","last_undegraded":"2024-04-25T10:05:38.075592+0000","last_fullsized":"2024-04-25T10:05:38.075592+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081925+0000","last_change":"2024-04-25T10:05:38.081354+0000","last_active":"2024-04-25T10:05:38.081925+0000","last_peered":"2024-04-25T10:05:38.081925+0000","last_clean":"2024-04-25T10:05:38.081925+0000","last_became_active":"2024-04-25T10:05:38.081054+0000","last_became_peered":"2024-04-25T10:05:38.081054+0000","last_unstale":"2024-04-25T10:05:38.081925+0000","last_undegraded":"2024-04-25T10:05:38.081925+0000","last_fullsized":"2024-04-25T10:05:38.081925+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:04.866716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075995+0000","last_change":"2024-04-25T10:05:38.075995+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075995+0000","last_undegraded":"2024-04-25T10:05:38.075995+0000","last_fullsized":"2024-04-25T10:05:38.075995+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076017+0000","last_change":"2024-04-25T10:05:38.075546+0000","last_active":"2024-04-25T10:05:38.076017+0000","last_peered":"2024-04-25T10:05:38.076017+0000","last_clean":"2024-04-25T10:05:38.076017+0000","last_became_active":"2024-04-25T10:05:38.074936+0000","last_became_peered":"2024-04-25T10:05:38.074936+0000","last_unstale":"2024-04-25T10:05:38.076017+0000","last_undegraded":"2024-04-25T10:05:38.076017+0000","last_fullsized":"2024-04-25T10:05:38.076017+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:00.798323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077978+0000","last_change":"2024-04-25T10:05:38.077978+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077978+0000","last_undegraded":"2024-04-25T10:05:38.077978+0000","last_fullsized":"2024-04-25T10:05:38.077978+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084725+0000","last_change":"2024-04-25T10:05:38.083077+0000","last_active":"2024-04-25T10:05:38.084725+0000","last_peered":"2024-04-25T10:05:38.084725+0000","last_clean":"2024-04-25T10:05:38.084725+0000","last_became_active":"2024-04-25T10:05:38.082606+0000","last_became_peered":"2024-04-25T10:05:38.082606+0000","last_unstale":"2024-04-25T10:05:38.084725+0000","last_undegraded":"2024-04-25T10:05:38.084725+0000","last_fullsized":"2024-04-25T10:05:38.084725+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:57.974626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077178+0000","last_change":"2024-04-25T10:05:38.077178+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077178+0000","last_undegraded":"2024-04-25T10:05:38.077178+0000","last_fullsized":"2024-04-25T10:05:38.077178+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.075802+0000","last_change":"2024-04-25T10:05:38.072474+0000","last_active":"2024-04-25T10:05:38.075802+0000","last_peered":"2024-04-25T10:05:38.075802+0000","last_clean":"2024-04-25T10:05:38.075802+0000","last_became_active":"2024-04-25T10:05:38.072160+0000","last_became_peered":"2024-04-25T10:05:38.072160+0000","last_unstale":"2024-04-25T10:05:38.075802+0000","last_undegraded":"2024-04-25T10:05:38.075802+0000","last_fullsized":"2024-04-25T10:05:38.075802+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:06.976864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":510,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.069800+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:05:38.069800+0000","last_peered":"2024-04-25T10:05:38.069800+0000","last_clean":"2024-04-25T10:05:38.069800+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:05:38.069800+0000","last_undegraded":"2024-04-25T10:05:38.069800+0000","last_fullsized":"2024-04-25T10:05:38.069800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56284,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315556,"statfs":{"total":96636764160,"available":96579129344,"internally_reserved":0,"allocated":2961408,"data_stored":1700777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51544,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320296,"statfs":{"total":96636764160,"available":96583983104,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51540,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320300,"statfs":{"total":96636764160,"available":96583987200,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2355200,"data_stored":1103742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59868,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311972,"statfs":{"total":96636764160,"available":96575459328,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55696,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55708,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316132,"statfs":{"total":96636764160,"available":96579719168,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56284,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315556,"statfs":{"total":96636764160,"available":96579129344,"internally_reserved":0,"allocated":2961408,"data_stored":1700777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:42.074 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.205+0000 7fd4ced7e640 1 Processor -- start 2024-04-25T10:05:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.205+0000 7fd4ced7e640 1 -- start start 2024-04-25T10:05:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4ced7e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd4c815dd30 0x7fd4c80b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4ced7e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd4c80d0ad0 0x7fd4c80b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4ced7e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c80d1df0 con 0x7fd4c80b9770 2024-04-25T10:05:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4ced7e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c80d20d0 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4ced7e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c80d23b0 con 0x7fd4c815dd30 2024-04-25T10:05:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4cd31b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd4c80d0ad0 0x7fd4c80b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4cd31b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd4c80d0ad0 0x7fd4c80b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35782/0 (socket says 172.21.15.19:35782) 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4cd31b640 1 -- 172.21.15.19:0/380254602 learned_addr learned my addr 172.21.15.19:0/380254602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4ccb1a640 1 --2- 172.21.15.19:0/380254602 >> v2:172.21.15.112:3300/0 conn(0x7fd4c815dd30 0x7fd4c80b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4cd31b640 1 -- 172.21.15.19:0/380254602 >> v2:172.21.15.112:3300/0 conn(0x7fd4c815dd30 msgr2=0x7fd4c80b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4cd31b640 1 --2- 172.21.15.19:0/380254602 >> v2:172.21.15.112:3300/0 conn(0x7fd4c815dd30 0x7fd4c80b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4cd31b640 1 -- 172.21.15.19:0/380254602 >> v1:172.21.15.112:6789/0 conn(0x7fd4c80b9770 legacy=0x7fd4c80d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4cd31b640 1 -- 172.21.15.19:0/380254602 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c80d2690 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.206+0000 7fd4ccb1a640 1 --2- 172.21.15.19:0/380254602 >> v2:172.21.15.112:3300/0 conn(0x7fd4c815dd30 0x7fd4c80b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.207+0000 7fd4cd31b640 1 --2- 172.21.15.19:0/380254602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c80d0ad0 0x7fd4c80b6a40 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7fd4c806d000 tx=0x7fd4c002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12bd33babc29f9ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.207+0000 7fd4c77fe640 1 -- 172.21.15.19:0/380254602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0036070 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.207+0000 7fd4c77fe640 1 -- 172.21.15.19:0/380254602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c0031600 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.207+0000 7fd4c77fe640 1 -- 172.21.15.19:0/380254602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c00319d0 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.207+0000 7fd4ced7e640 1 -- 172.21.15.19:0/380254602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c80d0ad0 msgr2=0x7fd4c80b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.207+0000 7fd4ced7e640 1 --2- 172.21.15.19:0/380254602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c80d0ad0 0x7fd4c80b6a40 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7fd4c806d000 tx=0x7fd4c002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.208+0000 7fd4ced7e640 1 -- 172.21.15.19:0/380254602 shutdown_connections 2024-04-25T10:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.208+0000 7fd4ced7e640 1 --2- 172.21.15.19:0/380254602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c80d0ad0 0x7fd4c80b6a40 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.208+0000 7fd4ced7e640 1 --2- 172.21.15.19:0/380254602 >> v2:172.21.15.112:3300/0 conn(0x7fd4c815dd30 0x7fd4c80b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.208+0000 7fd4ced7e640 1 -- 172.21.15.19:0/380254602 >> 172.21.15.19:0/380254602 conn(0x7fd4c8061c30 msgr2=0x7fd4c806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.208+0000 7fd4ced7e640 1 -- 172.21.15.19:0/380254602 shutdown_connections 2024-04-25T10:05:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.208+0000 7fd4ced7e640 1 -- 172.21.15.19:0/380254602 wait complete. 2024-04-25T10:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.209+0000 7fd4ced7e640 1 Processor -- start 2024-04-25T10:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.209+0000 7fd4ced7e640 1 -- start start 2024-04-25T10:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4ced7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4c80b9770 0x7fd4c814b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4ced7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c80d0ad0 0x7fd4c80cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4ced7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4c815dd30 0x7fd4c80cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4ced7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c80d1b90 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4ced7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c80d1a10 con 0x7fd4c80b9770 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4ced7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c80d2e90 con 0x7fd4c815dd30 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c80d0ad0 0x7fd4c80cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c80d0ad0 0x7fd4c80cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35798/0 (socket says 172.21.15.19:35798) 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4c7fff640 1 -- 172.21.15.19:0/606293732 learned_addr learned my addr 172.21.15.19:0/606293732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4ccb1a640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4c80b9770 0x7fd4c814b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4cd31b640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4c815dd30 0x7fd4c80cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4c7fff640 1 -- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4c815dd30 msgr2=0x7fd4c80cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4c7fff640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4c815dd30 0x7fd4c80cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4c7fff640 1 -- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4c80b9770 msgr2=0x7fd4c814b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4c7fff640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4c80b9770 0x7fd4c814b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4c7fff640 1 -- 172.21.15.19:0/606293732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c814a5d0 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.210+0000 7fd4c7fff640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c80d0ad0 0x7fd4c80cb090 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4004640 tx=0x7fd4b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.211+0000 7fd4c57fa640 1 -- 172.21.15.19:0/606293732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4011790 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.211+0000 7fd4c57fa640 1 -- 172.21.15.19:0/606293732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b4017070 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.211+0000 7fd4ced7e640 1 -- 172.21.15.19:0/606293732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c814a8b0 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.211+0000 7fd4c57fa640 1 -- 172.21.15.19:0/606293732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4011bb0 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.211+0000 7fd4ced7e640 1 -- 172.21.15.19:0/606293732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c814ae70 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.212+0000 7fd4ced7e640 1 -- 172.21.15.19:0/606293732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c8073990 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.212+0000 7fd4c57fa640 1 -- 172.21.15.19:0/606293732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4b4011d50 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.213+0000 7fd4c57fa640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4ac06b4d0 0x7fd4ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.213+0000 7fd4c57fa640 1 -- 172.21.15.19:0/606293732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4b4089e00 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.213+0000 7fd4ccb1a640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4ac06b4d0 0x7fd4ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.213+0000 7fd4ccb1a640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4ac06b4d0 0x7fd4ac06d990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00aac0 tx=0x7fd4bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.217+0000 7fd4c57fa640 1 -- 172.21.15.19:0/606293732 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4b40584a0 con 0x7fd4c80d0ad0 2024-04-25T10:05:42.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.395+0000 7fd4ced7e640 1 -- 172.21.15.19:0/606293732 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4c80008d0 con 0x7fd4ac06b4d0 2024-04-25T10:05:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.401+0000 7fd4c57fa640 1 -- 172.21.15.19:0/606293732 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247082 (secure 0 0 0) 0x7fd4c80008d0 con 0x7fd4ac06b4d0 2024-04-25T10:05:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:42.411 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:05:42.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 -- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4ac06b4d0 msgr2=0x7fd4ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4ac06b4d0 0x7fd4ac06d990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00aac0 tx=0x7fd4bc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:42.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 -- 172.21.15.19:0/606293732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c80d0ad0 msgr2=0x7fd4c80cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c80d0ad0 0x7fd4c80cb090 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4004640 tx=0x7fd4b400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 -- 172.21.15.19:0/606293732 shutdown_connections 2024-04-25T10:05:42.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4ac06b4d0 0x7fd4ac06d990 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4c815dd30 0x7fd4c80cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c80d0ad0 0x7fd4c80cb090 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 --2- 172.21.15.19:0/606293732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4c80b9770 0x7fd4c814b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 -- 172.21.15.19:0/606293732 >> 172.21.15.19:0/606293732 conn(0x7fd4c8061c30 msgr2=0x7fd4c80ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 -- 172.21.15.19:0/606293732 shutdown_connections 2024-04-25T10:05:42.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.405+0000 7fd4ced7e640 1 -- 172.21.15.19:0/606293732 wait complete. 2024-04-25T10:05:42.421 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":566,"stamp":"2024-04-25T10:05:41.088763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":446772,"kb_used_data":19644,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754527948,"statfs":{"total":773094113280,"available":772636618752,"internally_reserved":0,"allocated":20115456,"data_stored":10057341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":437243442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035420"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066637+0000","last_change":"2024-04-25T10:05:38.066637+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066637+0000","last_undegraded":"2024-04-25T10:05:38.066637+0000","last_fullsized":"2024-04-25T10:05:38.066637+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074792+0000","last_change":"2024-04-25T10:05:38.074792+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074792+0000","last_undegraded":"2024-04-25T10:05:38.074792+0000","last_fullsized":"2024-04-25T10:05:38.074792+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077295+0000","last_change":"2024-04-25T10:05:38.077295+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077295+0000","last_undegraded":"2024-04-25T10:05:38.077295+0000","last_fullsized":"2024-04-25T10:05:38.077295+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067681+0000","last_change":"2024-04-25T10:05:38.067681+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067681+0000","last_undegraded":"2024-04-25T10:05:38.067681+0000","last_fullsized":"2024-04-25T10:05:38.067681+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066530+0000","last_change":"2024-04-25T10:05:38.066530+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066530+0000","last_undegraded":"2024-04-25T10:05:38.066530+0000","last_fullsized":"2024-04-25T10:05:38.066530+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071870+0000","last_change":"2024-04-25T10:05:38.071870+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071870+0000","last_undegraded":"2024-04-25T10:05:38.071870+0000","last_fullsized":"2024-04-25T10:05:38.071870+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077706+0000","last_change":"2024-04-25T10:05:38.077706+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077706+0000","last_undegraded":"2024-04-25T10:05:38.077706+0000","last_fullsized":"2024-04-25T10:05:38.077706+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071345+0000","last_change":"2024-04-25T10:05:38.071345+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071345+0000","last_undegraded":"2024-04-25T10:05:38.071345+0000","last_fullsized":"2024-04-25T10:05:38.071345+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076215+0000","last_change":"2024-04-25T10:05:38.076215+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076215+0000","last_undegraded":"2024-04-25T10:05:38.076215+0000","last_fullsized":"2024-04-25T10:05:38.076215+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.081044+0000","last_change":"2024-04-25T10:05:38.081044+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.081044+0000","last_undegraded":"2024-04-25T10:05:38.081044+0000","last_fullsized":"2024-04-25T10:05:38.081044+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.068089+0000","last_change":"2024-04-25T10:05:38.068089+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.068089+0000","last_undegraded":"2024-04-25T10:05:38.068089+0000","last_fullsized":"2024-04-25T10:05:38.068089+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076169+0000","last_change":"2024-04-25T10:05:38.076169+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076169+0000","last_undegraded":"2024-04-25T10:05:38.076169+0000","last_fullsized":"2024-04-25T10:05:38.076169+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070409+0000","last_change":"2024-04-25T10:05:38.070409+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070409+0000","last_undegraded":"2024-04-25T10:05:38.070409+0000","last_fullsized":"2024-04-25T10:05:38.070409+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076413+0000","last_change":"2024-04-25T10:05:38.076413+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076413+0000","last_undegraded":"2024-04-25T10:05:38.076413+0000","last_fullsized":"2024-04-25T10:05:38.076413+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076156+0000","last_change":"2024-04-25T10:05:38.076156+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076156+0000","last_undegraded":"2024-04-25T10:05:38.076156+0000","last_fullsized":"2024-04-25T10:05:38.076156+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070878+0000","last_change":"2024-04-25T10:05:38.070878+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070878+0000","last_undegraded":"2024-04-25T10:05:38.070878+0000","last_fullsized":"2024-04-25T10:05:38.070878+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078479+0000","last_change":"2024-04-25T10:05:38.078479+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078479+0000","last_undegraded":"2024-04-25T10:05:38.078479+0000","last_fullsized":"2024-04-25T10:05:38.078479+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.080617+0000","last_change":"2024-04-25T10:05:38.080617+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.080617+0000","last_undegraded":"2024-04-25T10:05:38.080617+0000","last_fullsized":"2024-04-25T10:05:38.080617+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065954+0000","last_change":"2024-04-25T10:05:38.065954+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065954+0000","last_undegraded":"2024-04-25T10:05:38.065954+0000","last_fullsized":"2024-04-25T10:05:38.065954+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070674+0000","last_change":"2024-04-25T10:05:38.070674+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070674+0000","last_undegraded":"2024-04-25T10:05:38.070674+0000","last_fullsized":"2024-04-25T10:05:38.070674+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077115+0000","last_change":"2024-04-25T10:05:38.077115+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077115+0000","last_undegraded":"2024-04-25T10:05:38.077115+0000","last_fullsized":"2024-04-25T10:05:38.077115+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076601+0000","last_change":"2024-04-25T10:05:38.076601+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076601+0000","last_undegraded":"2024-04-25T10:05:38.076601+0000","last_fullsized":"2024-04-25T10:05:38.076601+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075635+0000","last_change":"2024-04-25T10:05:38.075635+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075635+0000","last_undegraded":"2024-04-25T10:05:38.075635+0000","last_fullsized":"2024-04-25T10:05:38.075635+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065899+0000","last_change":"2024-04-25T10:05:38.065899+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065899+0000","last_undegraded":"2024-04-25T10:05:38.065899+0000","last_fullsized":"2024-04-25T10:05:38.065899+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065933+0000","last_change":"2024-04-25T10:05:38.065933+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065933+0000","last_undegraded":"2024-04-25T10:05:38.065933+0000","last_fullsized":"2024-04-25T10:05:38.065933+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.080059+0000","last_change":"2024-04-25T10:05:38.080059+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.080059+0000","last_undegraded":"2024-04-25T10:05:38.080059+0000","last_fullsized":"2024-04-25T10:05:38.080059+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067528+0000","last_change":"2024-04-25T10:05:38.067528+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067528+0000","last_undegraded":"2024-04-25T10:05:38.067528+0000","last_fullsized":"2024-04-25T10:05:38.067528+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074828+0000","last_change":"2024-04-25T10:05:38.074828+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074828+0000","last_undegraded":"2024-04-25T10:05:38.074828+0000","last_fullsized":"2024-04-25T10:05:38.074828+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074206+0000","last_change":"2024-04-25T10:05:38.074206+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074206+0000","last_undegraded":"2024-04-25T10:05:38.074206+0000","last_fullsized":"2024-04-25T10:05:38.074206+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.072514+0000","last_change":"2024-04-25T10:05:38.072514+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.072514+0000","last_undegraded":"2024-04-25T10:05:38.072514+0000","last_fullsized":"2024-04-25T10:05:38.072514+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074387+0000","last_change":"2024-04-25T10:05:38.074387+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074387+0000","last_undegraded":"2024-04-25T10:05:38.074387+0000","last_fullsized":"2024-04-25T10:05:38.074387+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070297+0000","last_change":"2024-04-25T10:05:38.070297+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070297+0000","last_undegraded":"2024-04-25T10:05:38.070297+0000","last_fullsized":"2024-04-25T10:05:38.070297+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077732+0000","last_change":"2024-04-25T10:05:38.077732+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077732+0000","last_undegraded":"2024-04-25T10:05:38.077732+0000","last_fullsized":"2024-04-25T10:05:38.077732+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.086219+0000","last_change":"2024-04-25T10:05:38.085541+0000","last_active":"2024-04-25T10:05:38.086219+0000","last_peered":"2024-04-25T10:05:38.086219+0000","last_clean":"2024-04-25T10:05:38.086219+0000","last_became_active":"2024-04-25T10:05:38.085242+0000","last_became_peered":"2024-04-25T10:05:38.085242+0000","last_unstale":"2024-04-25T10:05:38.086219+0000","last_undegraded":"2024-04-25T10:05:38.086219+0000","last_fullsized":"2024-04-25T10:05:38.086219+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:07.815434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076803+0000","last_change":"2024-04-25T10:05:38.076803+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076803+0000","last_undegraded":"2024-04-25T10:05:38.076803+0000","last_fullsized":"2024-04-25T10:05:38.076803+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084928+0000","last_change":"2024-04-25T10:05:38.081472+0000","last_active":"2024-04-25T10:05:38.084928+0000","last_peered":"2024-04-25T10:05:38.084928+0000","last_clean":"2024-04-25T10:05:38.084928+0000","last_became_active":"2024-04-25T10:05:38.081187+0000","last_became_peered":"2024-04-25T10:05:38.081187+0000","last_unstale":"2024-04-25T10:05:38.084928+0000","last_undegraded":"2024-04-25T10:05:38.084928+0000","last_fullsized":"2024-04-25T10:05:38.084928+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:38.872584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074333+0000","last_change":"2024-04-25T10:05:38.074333+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074333+0000","last_undegraded":"2024-04-25T10:05:38.074333+0000","last_fullsized":"2024-04-25T10:05:38.074333+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.080465+0000","last_change":"2024-04-25T10:05:38.079620+0000","last_active":"2024-04-25T10:05:38.080465+0000","last_peered":"2024-04-25T10:05:38.080465+0000","last_clean":"2024-04-25T10:05:38.080465+0000","last_became_active":"2024-04-25T10:05:38.079361+0000","last_became_peered":"2024-04-25T10:05:38.079361+0000","last_unstale":"2024-04-25T10:05:38.080465+0000","last_undegraded":"2024-04-25T10:05:38.080465+0000","last_fullsized":"2024-04-25T10:05:38.080465+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:46.032506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079284+0000","last_change":"2024-04-25T10:05:38.079284+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079284+0000","last_undegraded":"2024-04-25T10:05:38.079284+0000","last_fullsized":"2024-04-25T10:05:38.079284+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084474+0000","last_change":"2024-04-25T10:05:38.083940+0000","last_active":"2024-04-25T10:05:38.084474+0000","last_peered":"2024-04-25T10:05:38.084474+0000","last_clean":"2024-04-25T10:05:38.084474+0000","last_became_active":"2024-04-25T10:05:38.083619+0000","last_became_peered":"2024-04-25T10:05:38.083619+0000","last_unstale":"2024-04-25T10:05:38.084474+0000","last_undegraded":"2024-04-25T10:05:38.084474+0000","last_fullsized":"2024-04-25T10:05:38.084474+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.073144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073657+0000","last_change":"2024-04-25T10:05:38.073657+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073657+0000","last_undegraded":"2024-04-25T10:05:38.073657+0000","last_fullsized":"2024-04-25T10:05:38.073657+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.083185+0000","last_change":"2024-04-25T10:05:38.079975+0000","last_active":"2024-04-25T10:05:38.083185+0000","last_peered":"2024-04-25T10:05:38.083185+0000","last_clean":"2024-04-25T10:05:38.083185+0000","last_became_active":"2024-04-25T10:05:38.079671+0000","last_became_peered":"2024-04-25T10:05:38.079671+0000","last_unstale":"2024-04-25T10:05:38.083185+0000","last_undegraded":"2024-04-25T10:05:38.083185+0000","last_fullsized":"2024-04-25T10:05:38.083185+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:43.802225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074714+0000","last_change":"2024-04-25T10:05:38.074714+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074714+0000","last_undegraded":"2024-04-25T10:05:38.074714+0000","last_fullsized":"2024-04-25T10:05:38.074714+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084915+0000","last_change":"2024-04-25T10:05:38.082341+0000","last_active":"2024-04-25T10:05:38.084915+0000","last_peered":"2024-04-25T10:05:38.084915+0000","last_clean":"2024-04-25T10:05:38.084915+0000","last_became_active":"2024-04-25T10:05:38.082061+0000","last_became_peered":"2024-04-25T10:05:38.082061+0000","last_unstale":"2024-04-25T10:05:38.084915+0000","last_undegraded":"2024-04-25T10:05:38.084915+0000","last_fullsized":"2024-04-25T10:05:38.084915+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:28.373524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079217+0000","last_change":"2024-04-25T10:05:38.079217+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079217+0000","last_undegraded":"2024-04-25T10:05:38.079217+0000","last_fullsized":"2024-04-25T10:05:38.079217+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081588+0000","last_change":"2024-04-25T10:05:38.080620+0000","last_active":"2024-04-25T10:05:38.081588+0000","last_peered":"2024-04-25T10:05:38.081588+0000","last_clean":"2024-04-25T10:05:38.081588+0000","last_became_active":"2024-04-25T10:05:38.075233+0000","last_became_peered":"2024-04-25T10:05:38.075233+0000","last_unstale":"2024-04-25T10:05:38.081588+0000","last_undegraded":"2024-04-25T10:05:38.081588+0000","last_fullsized":"2024-04-25T10:05:38.081588+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:49.754709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073656+0000","last_change":"2024-04-25T10:05:38.073656+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073656+0000","last_undegraded":"2024-04-25T10:05:38.073656+0000","last_fullsized":"2024-04-25T10:05:38.073656+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.075711+0000","last_change":"2024-04-25T10:05:38.074918+0000","last_active":"2024-04-25T10:05:38.075711+0000","last_peered":"2024-04-25T10:05:38.075711+0000","last_clean":"2024-04-25T10:05:38.075711+0000","last_became_active":"2024-04-25T10:05:38.074613+0000","last_became_peered":"2024-04-25T10:05:38.074613+0000","last_unstale":"2024-04-25T10:05:38.075711+0000","last_undegraded":"2024-04-25T10:05:38.075711+0000","last_fullsized":"2024-04-25T10:05:38.075711+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:54.282381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074188+0000","last_change":"2024-04-25T10:05:38.074188+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074188+0000","last_undegraded":"2024-04-25T10:05:38.074188+0000","last_fullsized":"2024-04-25T10:05:38.074188+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084181+0000","last_change":"2024-04-25T10:05:38.082985+0000","last_active":"2024-04-25T10:05:38.084181+0000","last_peered":"2024-04-25T10:05:38.084181+0000","last_clean":"2024-04-25T10:05:38.084181+0000","last_became_active":"2024-04-25T10:05:38.082719+0000","last_became_peered":"2024-04-25T10:05:38.082719+0000","last_unstale":"2024-04-25T10:05:38.084181+0000","last_undegraded":"2024-04-25T10:05:38.084181+0000","last_fullsized":"2024-04-25T10:05:38.084181+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:40.200742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079561+0000","last_change":"2024-04-25T10:05:38.079561+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079561+0000","last_undegraded":"2024-04-25T10:05:38.079561+0000","last_fullsized":"2024-04-25T10:05:38.079561+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076035+0000","last_change":"2024-04-25T10:05:38.075322+0000","last_active":"2024-04-25T10:05:38.076035+0000","last_peered":"2024-04-25T10:05:38.076035+0000","last_clean":"2024-04-25T10:05:38.076035+0000","last_became_active":"2024-04-25T10:05:38.074688+0000","last_became_peered":"2024-04-25T10:05:38.074688+0000","last_unstale":"2024-04-25T10:05:38.076035+0000","last_undegraded":"2024-04-25T10:05:38.076035+0000","last_fullsized":"2024-04-25T10:05:38.076035+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:10.878156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.068047+0000","last_change":"2024-04-25T10:05:38.068047+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.068047+0000","last_undegraded":"2024-04-25T10:05:38.068047+0000","last_fullsized":"2024-04-25T10:05:38.068047+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081990+0000","last_change":"2024-04-25T10:05:38.080464+0000","last_active":"2024-04-25T10:05:38.081990+0000","last_peered":"2024-04-25T10:05:38.081990+0000","last_clean":"2024-04-25T10:05:38.081990+0000","last_became_active":"2024-04-25T10:05:38.080184+0000","last_became_peered":"2024-04-25T10:05:38.080184+0000","last_unstale":"2024-04-25T10:05:38.081990+0000","last_undegraded":"2024-04-25T10:05:38.081990+0000","last_fullsized":"2024-04-25T10:05:38.081990+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:06.519993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074213+0000","last_change":"2024-04-25T10:05:38.074213+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074213+0000","last_undegraded":"2024-04-25T10:05:38.074213+0000","last_fullsized":"2024-04-25T10:05:38.074213+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.087722+0000","last_change":"2024-04-25T10:05:38.087415+0000","last_active":"2024-04-25T10:05:38.087722+0000","last_peered":"2024-04-25T10:05:38.087722+0000","last_clean":"2024-04-25T10:05:38.087722+0000","last_became_active":"2024-04-25T10:05:38.087167+0000","last_became_peered":"2024-04-25T10:05:38.087167+0000","last_unstale":"2024-04-25T10:05:38.087722+0000","last_undegraded":"2024-04-25T10:05:38.087722+0000","last_fullsized":"2024-04-25T10:05:38.087722+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:30.872314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073003+0000","last_change":"2024-04-25T10:05:38.073003+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073003+0000","last_undegraded":"2024-04-25T10:05:38.073003+0000","last_fullsized":"2024-04-25T10:05:38.073003+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084546+0000","last_change":"2024-04-25T10:05:38.081783+0000","last_active":"2024-04-25T10:05:38.084546+0000","last_peered":"2024-04-25T10:05:38.084546+0000","last_clean":"2024-04-25T10:05:38.084546+0000","last_became_active":"2024-04-25T10:05:38.081463+0000","last_became_peered":"2024-04-25T10:05:38.081463+0000","last_unstale":"2024-04-25T10:05:38.084546+0000","last_undegraded":"2024-04-25T10:05:38.084546+0000","last_fullsized":"2024-04-25T10:05:38.084546+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:10.390750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077577+0000","last_change":"2024-04-25T10:05:38.077577+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077577+0000","last_undegraded":"2024-04-25T10:05:38.077577+0000","last_fullsized":"2024-04-25T10:05:38.077577+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.077536+0000","last_change":"2024-04-25T10:05:38.076940+0000","last_active":"2024-04-25T10:05:38.077536+0000","last_peered":"2024-04-25T10:05:38.077536+0000","last_clean":"2024-04-25T10:05:38.077536+0000","last_became_active":"2024-04-25T10:05:38.076639+0000","last_became_peered":"2024-04-25T10:05:38.076639+0000","last_unstale":"2024-04-25T10:05:38.077536+0000","last_undegraded":"2024-04-25T10:05:38.077536+0000","last_fullsized":"2024-04-25T10:05:38.077536+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:20.840458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073109+0000","last_change":"2024-04-25T10:05:38.073109+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073109+0000","last_undegraded":"2024-04-25T10:05:38.073109+0000","last_fullsized":"2024-04-25T10:05:38.073109+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085058+0000","last_change":"2024-04-25T10:05:38.083896+0000","last_active":"2024-04-25T10:05:38.085058+0000","last_peered":"2024-04-25T10:05:38.085058+0000","last_clean":"2024-04-25T10:05:38.085058+0000","last_became_active":"2024-04-25T10:05:38.083309+0000","last_became_peered":"2024-04-25T10:05:38.083309+0000","last_unstale":"2024-04-25T10:05:38.085058+0000","last_undegraded":"2024-04-25T10:05:38.085058+0000","last_fullsized":"2024-04-25T10:05:38.085058+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:27.679738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074786+0000","last_change":"2024-04-25T10:05:38.074786+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074786+0000","last_undegraded":"2024-04-25T10:05:38.074786+0000","last_fullsized":"2024-04-25T10:05:38.074786+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084860+0000","last_change":"2024-04-25T10:05:38.083645+0000","last_active":"2024-04-25T10:05:38.084860+0000","last_peered":"2024-04-25T10:05:38.084860+0000","last_clean":"2024-04-25T10:05:38.084860+0000","last_became_active":"2024-04-25T10:05:38.083357+0000","last_became_peered":"2024-04-25T10:05:38.083357+0000","last_unstale":"2024-04-25T10:05:38.084860+0000","last_undegraded":"2024-04-25T10:05:38.084860+0000","last_fullsized":"2024-04-25T10:05:38.084860+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:32.268641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067142+0000","last_change":"2024-04-25T10:05:38.067142+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067142+0000","last_undegraded":"2024-04-25T10:05:38.067142+0000","last_fullsized":"2024-04-25T10:05:38.067142+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085150+0000","last_change":"2024-04-25T10:05:38.083600+0000","last_active":"2024-04-25T10:05:38.085150+0000","last_peered":"2024-04-25T10:05:38.085150+0000","last_clean":"2024-04-25T10:05:38.085150+0000","last_became_active":"2024-04-25T10:05:38.083303+0000","last_became_peered":"2024-04-25T10:05:38.083303+0000","last_unstale":"2024-04-25T10:05:38.085150+0000","last_undegraded":"2024-04-25T10:05:38.085150+0000","last_fullsized":"2024-04-25T10:05:38.085150+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:05.866090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078449+0000","last_change":"2024-04-25T10:05:38.078449+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078449+0000","last_undegraded":"2024-04-25T10:05:38.078449+0000","last_fullsized":"2024-04-25T10:05:38.078449+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084965+0000","last_change":"2024-04-25T10:05:38.081964+0000","last_active":"2024-04-25T10:05:38.084965+0000","last_peered":"2024-04-25T10:05:38.084965+0000","last_clean":"2024-04-25T10:05:38.084965+0000","last_became_active":"2024-04-25T10:05:38.081673+0000","last_became_peered":"2024-04-25T10:05:38.081673+0000","last_unstale":"2024-04-25T10:05:38.084965+0000","last_undegraded":"2024-04-25T10:05:38.084965+0000","last_fullsized":"2024-04-25T10:05:38.084965+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.599553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078013+0000","last_change":"2024-04-25T10:05:38.078013+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078013+0000","last_undegraded":"2024-04-25T10:05:38.078013+0000","last_fullsized":"2024-04-25T10:05:38.078013+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085904+0000","last_change":"2024-04-25T10:05:38.085345+0000","last_active":"2024-04-25T10:05:38.085904+0000","last_peered":"2024-04-25T10:05:38.085904+0000","last_clean":"2024-04-25T10:05:38.085904+0000","last_became_active":"2024-04-25T10:05:38.082298+0000","last_became_peered":"2024-04-25T10:05:38.082298+0000","last_unstale":"2024-04-25T10:05:38.085904+0000","last_undegraded":"2024-04-25T10:05:38.085904+0000","last_fullsized":"2024-04-25T10:05:38.085904+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:46.424209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074518+0000","last_change":"2024-04-25T10:05:38.074518+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074518+0000","last_undegraded":"2024-04-25T10:05:38.074518+0000","last_fullsized":"2024-04-25T10:05:38.074518+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084654+0000","last_change":"2024-04-25T10:05:38.083542+0000","last_active":"2024-04-25T10:05:38.084654+0000","last_peered":"2024-04-25T10:05:38.084654+0000","last_clean":"2024-04-25T10:05:38.084654+0000","last_became_active":"2024-04-25T10:05:38.082652+0000","last_became_peered":"2024-04-25T10:05:38.082652+0000","last_unstale":"2024-04-25T10:05:38.084654+0000","last_undegraded":"2024-04-25T10:05:38.084654+0000","last_fullsized":"2024-04-25T10:05:38.084654+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:36.918168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065904+0000","last_change":"2024-04-25T10:05:38.065904+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065904+0000","last_undegraded":"2024-04-25T10:05:38.065904+0000","last_fullsized":"2024-04-25T10:05:38.065904+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.080166+0000","last_change":"2024-04-25T10:05:38.077397+0000","last_active":"2024-04-25T10:05:38.080166+0000","last_peered":"2024-04-25T10:05:38.080166+0000","last_clean":"2024-04-25T10:05:38.080166+0000","last_became_active":"2024-04-25T10:05:38.077107+0000","last_became_peered":"2024-04-25T10:05:38.077107+0000","last_unstale":"2024-04-25T10:05:38.080166+0000","last_undegraded":"2024-04-25T10:05:38.080166+0000","last_fullsized":"2024-04-25T10:05:38.080166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:43.802225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073892+0000","last_change":"2024-04-25T10:05:38.073892+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073892+0000","last_undegraded":"2024-04-25T10:05:38.073892+0000","last_fullsized":"2024-04-25T10:05:38.073892+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.079630+0000","last_change":"2024-04-25T10:05:38.075167+0000","last_active":"2024-04-25T10:05:38.079630+0000","last_peered":"2024-04-25T10:05:38.079630+0000","last_clean":"2024-04-25T10:05:38.079630+0000","last_became_active":"2024-04-25T10:05:38.074890+0000","last_became_peered":"2024-04-25T10:05:38.074890+0000","last_unstale":"2024-04-25T10:05:38.079630+0000","last_undegraded":"2024-04-25T10:05:38.079630+0000","last_fullsized":"2024-04-25T10:05:38.079630+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:00.798323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076641+0000","last_change":"2024-04-25T10:05:38.076641+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076641+0000","last_undegraded":"2024-04-25T10:05:38.076641+0000","last_fullsized":"2024-04-25T10:05:38.076641+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084195+0000","last_change":"2024-04-25T10:05:38.082353+0000","last_active":"2024-04-25T10:05:38.084195+0000","last_peered":"2024-04-25T10:05:38.084195+0000","last_clean":"2024-04-25T10:05:38.084195+0000","last_became_active":"2024-04-25T10:05:38.082047+0000","last_became_peered":"2024-04-25T10:05:38.082047+0000","last_unstale":"2024-04-25T10:05:38.084195+0000","last_undegraded":"2024-04-25T10:05:38.084195+0000","last_fullsized":"2024-04-25T10:05:38.084195+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:12.478664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074243+0000","last_change":"2024-04-25T10:05:38.074243+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074243+0000","last_undegraded":"2024-04-25T10:05:38.074243+0000","last_fullsized":"2024-04-25T10:05:38.074243+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085023+0000","last_change":"2024-04-25T10:05:38.083029+0000","last_active":"2024-04-25T10:05:38.085023+0000","last_peered":"2024-04-25T10:05:38.085023+0000","last_clean":"2024-04-25T10:05:38.085023+0000","last_became_active":"2024-04-25T10:05:38.082535+0000","last_became_peered":"2024-04-25T10:05:38.082535+0000","last_unstale":"2024-04-25T10:05:38.085023+0000","last_undegraded":"2024-04-25T10:05:38.085023+0000","last_fullsized":"2024-04-25T10:05:38.085023+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:42.531775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066667+0000","last_change":"2024-04-25T10:05:38.066667+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066667+0000","last_undegraded":"2024-04-25T10:05:38.066667+0000","last_fullsized":"2024-04-25T10:05:38.066667+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076016+0000","last_change":"2024-04-25T10:05:38.073711+0000","last_active":"2024-04-25T10:05:38.076016+0000","last_peered":"2024-04-25T10:05:38.076016+0000","last_clean":"2024-04-25T10:05:38.076016+0000","last_became_active":"2024-04-25T10:05:38.073414+0000","last_became_peered":"2024-04-25T10:05:38.073414+0000","last_unstale":"2024-04-25T10:05:38.076016+0000","last_undegraded":"2024-04-25T10:05:38.076016+0000","last_fullsized":"2024-04-25T10:05:38.076016+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:10.878156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071187+0000","last_change":"2024-04-25T10:05:38.071187+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071187+0000","last_undegraded":"2024-04-25T10:05:38.071187+0000","last_fullsized":"2024-04-25T10:05:38.071187+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.073860+0000","last_change":"2024-04-25T10:05:38.071762+0000","last_active":"2024-04-25T10:05:38.073860+0000","last_peered":"2024-04-25T10:05:38.073860+0000","last_clean":"2024-04-25T10:05:38.073860+0000","last_became_active":"2024-04-25T10:05:38.071429+0000","last_became_peered":"2024-04-25T10:05:38.071429+0000","last_unstale":"2024-04-25T10:05:38.073860+0000","last_undegraded":"2024-04-25T10:05:38.073860+0000","last_fullsized":"2024-04-25T10:05:38.073860+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:54.282381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077423+0000","last_change":"2024-04-25T10:05:38.077423+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077423+0000","last_undegraded":"2024-04-25T10:05:38.077423+0000","last_fullsized":"2024-04-25T10:05:38.077423+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085218+0000","last_change":"2024-04-25T10:05:38.084273+0000","last_active":"2024-04-25T10:05:38.085218+0000","last_peered":"2024-04-25T10:05:38.085218+0000","last_clean":"2024-04-25T10:05:38.085218+0000","last_became_active":"2024-04-25T10:05:38.082528+0000","last_became_peered":"2024-04-25T10:05:38.082528+0000","last_unstale":"2024-04-25T10:05:38.085218+0000","last_undegraded":"2024-04-25T10:05:38.085218+0000","last_fullsized":"2024-04-25T10:05:38.085218+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:29.382293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.072844+0000","last_change":"2024-04-25T10:05:38.072844+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.072844+0000","last_undegraded":"2024-04-25T10:05:38.072844+0000","last_fullsized":"2024-04-25T10:05:38.072844+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.087513+0000","last_change":"2024-04-25T10:05:38.086909+0000","last_active":"2024-04-25T10:05:38.087513+0000","last_peered":"2024-04-25T10:05:38.087513+0000","last_clean":"2024-04-25T10:05:38.087513+0000","last_became_active":"2024-04-25T10:05:38.086646+0000","last_became_peered":"2024-04-25T10:05:38.086646+0000","last_unstale":"2024-04-25T10:05:38.087513+0000","last_undegraded":"2024-04-25T10:05:38.087513+0000","last_fullsized":"2024-04-25T10:05:38.087513+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:08.234904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075592+0000","last_change":"2024-04-25T10:05:38.075592+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075592+0000","last_undegraded":"2024-04-25T10:05:38.075592+0000","last_fullsized":"2024-04-25T10:05:38.075592+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081925+0000","last_change":"2024-04-25T10:05:38.081354+0000","last_active":"2024-04-25T10:05:38.081925+0000","last_peered":"2024-04-25T10:05:38.081925+0000","last_clean":"2024-04-25T10:05:38.081925+0000","last_became_active":"2024-04-25T10:05:38.081054+0000","last_became_peered":"2024-04-25T10:05:38.081054+0000","last_unstale":"2024-04-25T10:05:38.081925+0000","last_undegraded":"2024-04-25T10:05:38.081925+0000","last_fullsized":"2024-04-25T10:05:38.081925+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:04.866716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075995+0000","last_change":"2024-04-25T10:05:38.075995+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075995+0000","last_undegraded":"2024-04-25T10:05:38.075995+0000","last_fullsized":"2024-04-25T10:05:38.075995+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076017+0000","last_change":"2024-04-25T10:05:38.075546+0000","last_active":"2024-04-25T10:05:38.076017+0000","last_peered":"2024-04-25T10:05:38.076017+0000","last_clean":"2024-04-25T10:05:38.076017+0000","last_became_active":"2024-04-25T10:05:38.074936+0000","last_became_peered":"2024-04-25T10:05:38.074936+0000","last_unstale":"2024-04-25T10:05:38.076017+0000","last_undegraded":"2024-04-25T10:05:38.076017+0000","last_fullsized":"2024-04-25T10:05:38.076017+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:00.798323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077978+0000","last_change":"2024-04-25T10:05:38.077978+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077978+0000","last_undegraded":"2024-04-25T10:05:38.077978+0000","last_fullsized":"2024-04-25T10:05:38.077978+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084725+0000","last_change":"2024-04-25T10:05:38.083077+0000","last_active":"2024-04-25T10:05:38.084725+0000","last_peered":"2024-04-25T10:05:38.084725+0000","last_clean":"2024-04-25T10:05:38.084725+0000","last_became_active":"2024-04-25T10:05:38.082606+0000","last_became_peered":"2024-04-25T10:05:38.082606+0000","last_unstale":"2024-04-25T10:05:38.084725+0000","last_undegraded":"2024-04-25T10:05:38.084725+0000","last_fullsized":"2024-04-25T10:05:38.084725+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:57.974626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077178+0000","last_change":"2024-04-25T10:05:38.077178+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077178+0000","last_undegraded":"2024-04-25T10:05:38.077178+0000","last_fullsized":"2024-04-25T10:05:38.077178+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.075802+0000","last_change":"2024-04-25T10:05:38.072474+0000","last_active":"2024-04-25T10:05:38.075802+0000","last_peered":"2024-04-25T10:05:38.075802+0000","last_clean":"2024-04-25T10:05:38.075802+0000","last_became_active":"2024-04-25T10:05:38.072160+0000","last_became_peered":"2024-04-25T10:05:38.072160+0000","last_unstale":"2024-04-25T10:05:38.075802+0000","last_undegraded":"2024-04-25T10:05:38.075802+0000","last_fullsized":"2024-04-25T10:05:38.075802+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:06.976864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":510,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.069800+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:05:38.069800+0000","last_peered":"2024-04-25T10:05:38.069800+0000","last_clean":"2024-04-25T10:05:38.069800+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:05:38.069800+0000","last_undegraded":"2024-04-25T10:05:38.069800+0000","last_fullsized":"2024-04-25T10:05:38.069800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56284,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315556,"statfs":{"total":96636764160,"available":96579129344,"internally_reserved":0,"allocated":2961408,"data_stored":1700777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51544,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320296,"statfs":{"total":96636764160,"available":96583983104,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51540,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320300,"statfs":{"total":96636764160,"available":96583987200,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2355200,"data_stored":1103742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59868,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311972,"statfs":{"total":96636764160,"available":96575459328,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55696,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55708,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316132,"statfs":{"total":96636764160,"available":96579719168,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56284,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315556,"statfs":{"total":96636764160,"available":96579129344,"internally_reserved":0,"allocated":2961408,"data_stored":1700777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:42.425 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.556+0000 7fd7a3d5c640 1 Processor -- start 2024-04-25T10:05:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.557+0000 7fd7a3d5c640 1 -- start start 2024-04-25T10:05:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.557+0000 7fd7a3d5c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd79c1602e0 0x7fd79c0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.557+0000 7fd7a3d5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd79c15f5f0 0x7fd79c15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.557+0000 7fd7a3d5c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79c0cd710 con 0x7fd79c15ba00 2024-04-25T10:05:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.557+0000 7fd7a3d5c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79c15ff80 con 0x7fd79c1602e0 2024-04-25T10:05:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.557+0000 7fd7a3d5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79c099b70 con 0x7fd79c15f5f0 2024-04-25T10:05:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.558+0000 7fd7a12f7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd79c1602e0 0x7fd79c0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.558+0000 7fd7a12f7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd79c1602e0 0x7fd79c0ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35800/0 (socket says 172.21.15.19:35800) 2024-04-25T10:05:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.558+0000 7fd7a12f7640 1 -- 172.21.15.19:0/1349406481 learned_addr learned my addr 172.21.15.19:0/1349406481 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.558+0000 7fd7a22f9640 1 --2- 172.21.15.19:0/1349406481 >> v2:172.21.15.112:3300/0 conn(0x7fd79c15f5f0 0x7fd79c15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.558+0000 7fd7a12f7640 1 -- 172.21.15.19:0/1349406481 >> v2:172.21.15.112:3300/0 conn(0x7fd79c15f5f0 msgr2=0x7fd79c15fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.558+0000 7fd7a12f7640 1 --2- 172.21.15.19:0/1349406481 >> v2:172.21.15.112:3300/0 conn(0x7fd79c15f5f0 0x7fd79c15fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.558+0000 7fd7a12f7640 1 -- 172.21.15.19:0/1349406481 >> v1:172.21.15.112:6789/0 conn(0x7fd79c15ba00 legacy=0x7fd79c09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.558+0000 7fd7a12f7640 1 -- 172.21.15.19:0/1349406481 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c099e50 con 0x7fd79c1602e0 2024-04-25T10:05:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.558+0000 7fd7a22f9640 1 --2- 172.21.15.19:0/1349406481 >> v2:172.21.15.112:3300/0 conn(0x7fd79c15f5f0 0x7fd79c15fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a12f7640 1 --2- 172.21.15.19:0/1349406481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79c1602e0 0x7fd79c0ccda0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7fd79c072b10 tx=0x7fd78c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb3e28d0eee01d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a0af6640 1 -- 172.21.15.19:0/1349406481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c037070 con 0x7fd79c1602e0 2024-04-25T10:05:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a0af6640 1 -- 172.21.15.19:0/1349406481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c02ae20 con 0x7fd79c1602e0 2024-04-25T10:05:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a0af6640 1 -- 172.21.15.19:0/1349406481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c032660 con 0x7fd79c1602e0 2024-04-25T10:05:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/1349406481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79c1602e0 msgr2=0x7fd79c0ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a3d5c640 1 --2- 172.21.15.19:0/1349406481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79c1602e0 0x7fd79c0ccda0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7fd79c072b10 tx=0x7fd78c02a110 comp rx=0 tx=0).stop 2024-04-25T10:05:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/1349406481 shutdown_connections 2024-04-25T10:05:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a3d5c640 1 --2- 172.21.15.19:0/1349406481 >> v2:172.21.15.112:3300/0 conn(0x7fd79c15f5f0 0x7fd79c15fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a3d5c640 1 --2- 172.21.15.19:0/1349406481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79c1602e0 0x7fd79c0ccda0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/1349406481 >> 172.21.15.19:0/1349406481 conn(0x7fd79c0b3270 msgr2=0x7fd79c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/1349406481 shutdown_connections 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.559+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/1349406481 wait complete. 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.561+0000 7fd7a3d5c640 1 Processor -- start 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.561+0000 7fd7a3d5c640 1 -- start start 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.561+0000 7fd7a3d5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd79c15ba00 0x7fd79c152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.561+0000 7fd7a3d5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd79c15f5f0 0x7fd79c15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.561+0000 7fd7a3d5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79c1602e0 0x7fd79c0c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.561+0000 7fd7a3d5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c15bde0 con 0x7fd79c1602e0 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.561+0000 7fd7a3d5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c0cd2e0 con 0x7fd79c15ba00 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.561+0000 7fd7a3d5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c0cb180 con 0x7fd79c15f5f0 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.561+0000 7fd7a22f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79c1602e0 0x7fd79c0c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.561+0000 7fd7a22f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79c1602e0 0x7fd79c0c7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35804/0 (socket says 172.21.15.19:35804) 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.561+0000 7fd7a22f9640 1 -- 172.21.15.19:0/2822736366 learned_addr learned my addr 172.21.15.19:0/2822736366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7a12f7640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd79c15f5f0 0x7fd79c15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7a1af8640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd79c15ba00 0x7fd79c152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7a22f9640 1 -- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd79c15f5f0 msgr2=0x7fd79c15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7a22f9640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd79c15f5f0 0x7fd79c15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7a22f9640 1 -- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd79c15ba00 msgr2=0x7fd79c152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7a22f9640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd79c15ba00 0x7fd79c152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7a22f9640 1 -- 172.21.15.19:0/2822736366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c0ca0e0 con 0x7fd79c1602e0 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7a22f9640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79c1602e0 0x7fd79c0c7010 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7fd79400cf20 tx=0x7fd79400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7927fc640 1 -- 172.21.15.19:0/2822736366 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79401a070 con 0x7fd79c1602e0 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7927fc640 1 -- 172.21.15.19:0/2822736366 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79400d840 con 0x7fd79c1602e0 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7927fc640 1 -- 172.21.15.19:0/2822736366 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79400dc30 con 0x7fd79c1602e0 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/2822736366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c0ca3c0 con 0x7fd79c1602e0 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.562+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/2822736366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79c0ca980 con 0x7fd79c1602e0 2024-04-25T10:05:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.563+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/2822736366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79c073990 con 0x7fd79c1602e0 2024-04-25T10:05:42.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.563+0000 7fd7927fc640 1 -- 172.21.15.19:0/2822736366 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd79401d070 con 0x7fd79c1602e0 2024-04-25T10:05:42.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.564+0000 7fd7927fc640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd78006b4d0 0x7fd78006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.564+0000 7fd7927fc640 1 -- 172.21.15.19:0/2822736366 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd79408ac50 con 0x7fd79c1602e0 2024-04-25T10:05:42.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.564+0000 7fd7a1af8640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd78006b4d0 0x7fd78006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.565+0000 7fd7a1af8640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd78006b4d0 0x7fd78006d990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0cc580 tx=0x7fd798009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.568+0000 7fd7927fc640 1 -- 172.21.15.19:0/2822736366 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd794090020 con 0x7fd79c1602e0 2024-04-25T10:05:42.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.744+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/2822736366 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd79c0b40a0 con 0x7fd78006b4d0 2024-04-25T10:05:42.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.750+0000 7fd7927fc640 1 -- 172.21.15.19:0/2822736366 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247082 (secure 0 0 0) 0x7fd79c0b40a0 con 0x7fd78006b4d0 2024-04-25T10:05:42.759 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:42.760 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:05:42.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd78006b4d0 msgr2=0x7fd78006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd78006b4d0 0x7fd78006d990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0cc580 tx=0x7fd798009040 comp rx=0 tx=0).stop 2024-04-25T10:05:42.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/2822736366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79c1602e0 msgr2=0x7fd79c0c7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79c1602e0 0x7fd79c0c7010 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7fd79400cf20 tx=0x7fd79400a7d0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/2822736366 shutdown_connections 2024-04-25T10:05:42.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd78006b4d0 0x7fd78006d990 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79c1602e0 0x7fd79c0c7010 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd79c15f5f0 0x7fd79c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 --2- 172.21.15.19:0/2822736366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd79c15ba00 0x7fd79c152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/2822736366 >> 172.21.15.19:0/2822736366 conn(0x7fd79c0b3270 msgr2=0x7fd79c0b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/2822736366 shutdown_connections 2024-04-25T10:05:42.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.753+0000 7fd7a3d5c640 1 -- 172.21.15.19:0/2822736366 wait complete. 2024-04-25T10:05:42.769 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":566,"stamp":"2024-04-25T10:05:41.088763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":446772,"kb_used_data":19644,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754527948,"statfs":{"total":773094113280,"available":772636618752,"internally_reserved":0,"allocated":20115456,"data_stored":10057341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":437243442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035420"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066637+0000","last_change":"2024-04-25T10:05:38.066637+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066637+0000","last_undegraded":"2024-04-25T10:05:38.066637+0000","last_fullsized":"2024-04-25T10:05:38.066637+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074792+0000","last_change":"2024-04-25T10:05:38.074792+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074792+0000","last_undegraded":"2024-04-25T10:05:38.074792+0000","last_fullsized":"2024-04-25T10:05:38.074792+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077295+0000","last_change":"2024-04-25T10:05:38.077295+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077295+0000","last_undegraded":"2024-04-25T10:05:38.077295+0000","last_fullsized":"2024-04-25T10:05:38.077295+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067681+0000","last_change":"2024-04-25T10:05:38.067681+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067681+0000","last_undegraded":"2024-04-25T10:05:38.067681+0000","last_fullsized":"2024-04-25T10:05:38.067681+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066530+0000","last_change":"2024-04-25T10:05:38.066530+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066530+0000","last_undegraded":"2024-04-25T10:05:38.066530+0000","last_fullsized":"2024-04-25T10:05:38.066530+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071870+0000","last_change":"2024-04-25T10:05:38.071870+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071870+0000","last_undegraded":"2024-04-25T10:05:38.071870+0000","last_fullsized":"2024-04-25T10:05:38.071870+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077706+0000","last_change":"2024-04-25T10:05:38.077706+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077706+0000","last_undegraded":"2024-04-25T10:05:38.077706+0000","last_fullsized":"2024-04-25T10:05:38.077706+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071345+0000","last_change":"2024-04-25T10:05:38.071345+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071345+0000","last_undegraded":"2024-04-25T10:05:38.071345+0000","last_fullsized":"2024-04-25T10:05:38.071345+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076215+0000","last_change":"2024-04-25T10:05:38.076215+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076215+0000","last_undegraded":"2024-04-25T10:05:38.076215+0000","last_fullsized":"2024-04-25T10:05:38.076215+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.081044+0000","last_change":"2024-04-25T10:05:38.081044+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.081044+0000","last_undegraded":"2024-04-25T10:05:38.081044+0000","last_fullsized":"2024-04-25T10:05:38.081044+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.068089+0000","last_change":"2024-04-25T10:05:38.068089+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.068089+0000","last_undegraded":"2024-04-25T10:05:38.068089+0000","last_fullsized":"2024-04-25T10:05:38.068089+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076169+0000","last_change":"2024-04-25T10:05:38.076169+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076169+0000","last_undegraded":"2024-04-25T10:05:38.076169+0000","last_fullsized":"2024-04-25T10:05:38.076169+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070409+0000","last_change":"2024-04-25T10:05:38.070409+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070409+0000","last_undegraded":"2024-04-25T10:05:38.070409+0000","last_fullsized":"2024-04-25T10:05:38.070409+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076413+0000","last_change":"2024-04-25T10:05:38.076413+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076413+0000","last_undegraded":"2024-04-25T10:05:38.076413+0000","last_fullsized":"2024-04-25T10:05:38.076413+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076156+0000","last_change":"2024-04-25T10:05:38.076156+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076156+0000","last_undegraded":"2024-04-25T10:05:38.076156+0000","last_fullsized":"2024-04-25T10:05:38.076156+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070878+0000","last_change":"2024-04-25T10:05:38.070878+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070878+0000","last_undegraded":"2024-04-25T10:05:38.070878+0000","last_fullsized":"2024-04-25T10:05:38.070878+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078479+0000","last_change":"2024-04-25T10:05:38.078479+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078479+0000","last_undegraded":"2024-04-25T10:05:38.078479+0000","last_fullsized":"2024-04-25T10:05:38.078479+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.080617+0000","last_change":"2024-04-25T10:05:38.080617+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.080617+0000","last_undegraded":"2024-04-25T10:05:38.080617+0000","last_fullsized":"2024-04-25T10:05:38.080617+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065954+0000","last_change":"2024-04-25T10:05:38.065954+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065954+0000","last_undegraded":"2024-04-25T10:05:38.065954+0000","last_fullsized":"2024-04-25T10:05:38.065954+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070674+0000","last_change":"2024-04-25T10:05:38.070674+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070674+0000","last_undegraded":"2024-04-25T10:05:38.070674+0000","last_fullsized":"2024-04-25T10:05:38.070674+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077115+0000","last_change":"2024-04-25T10:05:38.077115+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077115+0000","last_undegraded":"2024-04-25T10:05:38.077115+0000","last_fullsized":"2024-04-25T10:05:38.077115+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076601+0000","last_change":"2024-04-25T10:05:38.076601+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076601+0000","last_undegraded":"2024-04-25T10:05:38.076601+0000","last_fullsized":"2024-04-25T10:05:38.076601+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075635+0000","last_change":"2024-04-25T10:05:38.075635+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075635+0000","last_undegraded":"2024-04-25T10:05:38.075635+0000","last_fullsized":"2024-04-25T10:05:38.075635+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065899+0000","last_change":"2024-04-25T10:05:38.065899+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065899+0000","last_undegraded":"2024-04-25T10:05:38.065899+0000","last_fullsized":"2024-04-25T10:05:38.065899+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065933+0000","last_change":"2024-04-25T10:05:38.065933+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065933+0000","last_undegraded":"2024-04-25T10:05:38.065933+0000","last_fullsized":"2024-04-25T10:05:38.065933+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.080059+0000","last_change":"2024-04-25T10:05:38.080059+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.080059+0000","last_undegraded":"2024-04-25T10:05:38.080059+0000","last_fullsized":"2024-04-25T10:05:38.080059+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067528+0000","last_change":"2024-04-25T10:05:38.067528+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067528+0000","last_undegraded":"2024-04-25T10:05:38.067528+0000","last_fullsized":"2024-04-25T10:05:38.067528+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074828+0000","last_change":"2024-04-25T10:05:38.074828+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074828+0000","last_undegraded":"2024-04-25T10:05:38.074828+0000","last_fullsized":"2024-04-25T10:05:38.074828+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074206+0000","last_change":"2024-04-25T10:05:38.074206+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074206+0000","last_undegraded":"2024-04-25T10:05:38.074206+0000","last_fullsized":"2024-04-25T10:05:38.074206+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.072514+0000","last_change":"2024-04-25T10:05:38.072514+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.072514+0000","last_undegraded":"2024-04-25T10:05:38.072514+0000","last_fullsized":"2024-04-25T10:05:38.072514+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074387+0000","last_change":"2024-04-25T10:05:38.074387+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074387+0000","last_undegraded":"2024-04-25T10:05:38.074387+0000","last_fullsized":"2024-04-25T10:05:38.074387+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070297+0000","last_change":"2024-04-25T10:05:38.070297+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070297+0000","last_undegraded":"2024-04-25T10:05:38.070297+0000","last_fullsized":"2024-04-25T10:05:38.070297+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077732+0000","last_change":"2024-04-25T10:05:38.077732+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077732+0000","last_undegraded":"2024-04-25T10:05:38.077732+0000","last_fullsized":"2024-04-25T10:05:38.077732+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.086219+0000","last_change":"2024-04-25T10:05:38.085541+0000","last_active":"2024-04-25T10:05:38.086219+0000","last_peered":"2024-04-25T10:05:38.086219+0000","last_clean":"2024-04-25T10:05:38.086219+0000","last_became_active":"2024-04-25T10:05:38.085242+0000","last_became_peered":"2024-04-25T10:05:38.085242+0000","last_unstale":"2024-04-25T10:05:38.086219+0000","last_undegraded":"2024-04-25T10:05:38.086219+0000","last_fullsized":"2024-04-25T10:05:38.086219+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:07.815434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076803+0000","last_change":"2024-04-25T10:05:38.076803+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076803+0000","last_undegraded":"2024-04-25T10:05:38.076803+0000","last_fullsized":"2024-04-25T10:05:38.076803+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084928+0000","last_change":"2024-04-25T10:05:38.081472+0000","last_active":"2024-04-25T10:05:38.084928+0000","last_peered":"2024-04-25T10:05:38.084928+0000","last_clean":"2024-04-25T10:05:38.084928+0000","last_became_active":"2024-04-25T10:05:38.081187+0000","last_became_peered":"2024-04-25T10:05:38.081187+0000","last_unstale":"2024-04-25T10:05:38.084928+0000","last_undegraded":"2024-04-25T10:05:38.084928+0000","last_fullsized":"2024-04-25T10:05:38.084928+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:38.872584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074333+0000","last_change":"2024-04-25T10:05:38.074333+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074333+0000","last_undegraded":"2024-04-25T10:05:38.074333+0000","last_fullsized":"2024-04-25T10:05:38.074333+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.080465+0000","last_change":"2024-04-25T10:05:38.079620+0000","last_active":"2024-04-25T10:05:38.080465+0000","last_peered":"2024-04-25T10:05:38.080465+0000","last_clean":"2024-04-25T10:05:38.080465+0000","last_became_active":"2024-04-25T10:05:38.079361+0000","last_became_peered":"2024-04-25T10:05:38.079361+0000","last_unstale":"2024-04-25T10:05:38.080465+0000","last_undegraded":"2024-04-25T10:05:38.080465+0000","last_fullsized":"2024-04-25T10:05:38.080465+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:46.032506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079284+0000","last_change":"2024-04-25T10:05:38.079284+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079284+0000","last_undegraded":"2024-04-25T10:05:38.079284+0000","last_fullsized":"2024-04-25T10:05:38.079284+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084474+0000","last_change":"2024-04-25T10:05:38.083940+0000","last_active":"2024-04-25T10:05:38.084474+0000","last_peered":"2024-04-25T10:05:38.084474+0000","last_clean":"2024-04-25T10:05:38.084474+0000","last_became_active":"2024-04-25T10:05:38.083619+0000","last_became_peered":"2024-04-25T10:05:38.083619+0000","last_unstale":"2024-04-25T10:05:38.084474+0000","last_undegraded":"2024-04-25T10:05:38.084474+0000","last_fullsized":"2024-04-25T10:05:38.084474+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.073144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073657+0000","last_change":"2024-04-25T10:05:38.073657+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073657+0000","last_undegraded":"2024-04-25T10:05:38.073657+0000","last_fullsized":"2024-04-25T10:05:38.073657+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.083185+0000","last_change":"2024-04-25T10:05:38.079975+0000","last_active":"2024-04-25T10:05:38.083185+0000","last_peered":"2024-04-25T10:05:38.083185+0000","last_clean":"2024-04-25T10:05:38.083185+0000","last_became_active":"2024-04-25T10:05:38.079671+0000","last_became_peered":"2024-04-25T10:05:38.079671+0000","last_unstale":"2024-04-25T10:05:38.083185+0000","last_undegraded":"2024-04-25T10:05:38.083185+0000","last_fullsized":"2024-04-25T10:05:38.083185+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:43.802225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074714+0000","last_change":"2024-04-25T10:05:38.074714+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074714+0000","last_undegraded":"2024-04-25T10:05:38.074714+0000","last_fullsized":"2024-04-25T10:05:38.074714+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084915+0000","last_change":"2024-04-25T10:05:38.082341+0000","last_active":"2024-04-25T10:05:38.084915+0000","last_peered":"2024-04-25T10:05:38.084915+0000","last_clean":"2024-04-25T10:05:38.084915+0000","last_became_active":"2024-04-25T10:05:38.082061+0000","last_became_peered":"2024-04-25T10:05:38.082061+0000","last_unstale":"2024-04-25T10:05:38.084915+0000","last_undegraded":"2024-04-25T10:05:38.084915+0000","last_fullsized":"2024-04-25T10:05:38.084915+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:28.373524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079217+0000","last_change":"2024-04-25T10:05:38.079217+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079217+0000","last_undegraded":"2024-04-25T10:05:38.079217+0000","last_fullsized":"2024-04-25T10:05:38.079217+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081588+0000","last_change":"2024-04-25T10:05:38.080620+0000","last_active":"2024-04-25T10:05:38.081588+0000","last_peered":"2024-04-25T10:05:38.081588+0000","last_clean":"2024-04-25T10:05:38.081588+0000","last_became_active":"2024-04-25T10:05:38.075233+0000","last_became_peered":"2024-04-25T10:05:38.075233+0000","last_unstale":"2024-04-25T10:05:38.081588+0000","last_undegraded":"2024-04-25T10:05:38.081588+0000","last_fullsized":"2024-04-25T10:05:38.081588+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:49.754709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073656+0000","last_change":"2024-04-25T10:05:38.073656+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073656+0000","last_undegraded":"2024-04-25T10:05:38.073656+0000","last_fullsized":"2024-04-25T10:05:38.073656+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.075711+0000","last_change":"2024-04-25T10:05:38.074918+0000","last_active":"2024-04-25T10:05:38.075711+0000","last_peered":"2024-04-25T10:05:38.075711+0000","last_clean":"2024-04-25T10:05:38.075711+0000","last_became_active":"2024-04-25T10:05:38.074613+0000","last_became_peered":"2024-04-25T10:05:38.074613+0000","last_unstale":"2024-04-25T10:05:38.075711+0000","last_undegraded":"2024-04-25T10:05:38.075711+0000","last_fullsized":"2024-04-25T10:05:38.075711+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:54.282381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074188+0000","last_change":"2024-04-25T10:05:38.074188+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074188+0000","last_undegraded":"2024-04-25T10:05:38.074188+0000","last_fullsized":"2024-04-25T10:05:38.074188+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084181+0000","last_change":"2024-04-25T10:05:38.082985+0000","last_active":"2024-04-25T10:05:38.084181+0000","last_peered":"2024-04-25T10:05:38.084181+0000","last_clean":"2024-04-25T10:05:38.084181+0000","last_became_active":"2024-04-25T10:05:38.082719+0000","last_became_peered":"2024-04-25T10:05:38.082719+0000","last_unstale":"2024-04-25T10:05:38.084181+0000","last_undegraded":"2024-04-25T10:05:38.084181+0000","last_fullsized":"2024-04-25T10:05:38.084181+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:40.200742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079561+0000","last_change":"2024-04-25T10:05:38.079561+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079561+0000","last_undegraded":"2024-04-25T10:05:38.079561+0000","last_fullsized":"2024-04-25T10:05:38.079561+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076035+0000","last_change":"2024-04-25T10:05:38.075322+0000","last_active":"2024-04-25T10:05:38.076035+0000","last_peered":"2024-04-25T10:05:38.076035+0000","last_clean":"2024-04-25T10:05:38.076035+0000","last_became_active":"2024-04-25T10:05:38.074688+0000","last_became_peered":"2024-04-25T10:05:38.074688+0000","last_unstale":"2024-04-25T10:05:38.076035+0000","last_undegraded":"2024-04-25T10:05:38.076035+0000","last_fullsized":"2024-04-25T10:05:38.076035+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:10.878156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.068047+0000","last_change":"2024-04-25T10:05:38.068047+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.068047+0000","last_undegraded":"2024-04-25T10:05:38.068047+0000","last_fullsized":"2024-04-25T10:05:38.068047+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081990+0000","last_change":"2024-04-25T10:05:38.080464+0000","last_active":"2024-04-25T10:05:38.081990+0000","last_peered":"2024-04-25T10:05:38.081990+0000","last_clean":"2024-04-25T10:05:38.081990+0000","last_became_active":"2024-04-25T10:05:38.080184+0000","last_became_peered":"2024-04-25T10:05:38.080184+0000","last_unstale":"2024-04-25T10:05:38.081990+0000","last_undegraded":"2024-04-25T10:05:38.081990+0000","last_fullsized":"2024-04-25T10:05:38.081990+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:06.519993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074213+0000","last_change":"2024-04-25T10:05:38.074213+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074213+0000","last_undegraded":"2024-04-25T10:05:38.074213+0000","last_fullsized":"2024-04-25T10:05:38.074213+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.087722+0000","last_change":"2024-04-25T10:05:38.087415+0000","last_active":"2024-04-25T10:05:38.087722+0000","last_peered":"2024-04-25T10:05:38.087722+0000","last_clean":"2024-04-25T10:05:38.087722+0000","last_became_active":"2024-04-25T10:05:38.087167+0000","last_became_peered":"2024-04-25T10:05:38.087167+0000","last_unstale":"2024-04-25T10:05:38.087722+0000","last_undegraded":"2024-04-25T10:05:38.087722+0000","last_fullsized":"2024-04-25T10:05:38.087722+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:30.872314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073003+0000","last_change":"2024-04-25T10:05:38.073003+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073003+0000","last_undegraded":"2024-04-25T10:05:38.073003+0000","last_fullsized":"2024-04-25T10:05:38.073003+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084546+0000","last_change":"2024-04-25T10:05:38.081783+0000","last_active":"2024-04-25T10:05:38.084546+0000","last_peered":"2024-04-25T10:05:38.084546+0000","last_clean":"2024-04-25T10:05:38.084546+0000","last_became_active":"2024-04-25T10:05:38.081463+0000","last_became_peered":"2024-04-25T10:05:38.081463+0000","last_unstale":"2024-04-25T10:05:38.084546+0000","last_undegraded":"2024-04-25T10:05:38.084546+0000","last_fullsized":"2024-04-25T10:05:38.084546+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:10.390750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077577+0000","last_change":"2024-04-25T10:05:38.077577+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077577+0000","last_undegraded":"2024-04-25T10:05:38.077577+0000","last_fullsized":"2024-04-25T10:05:38.077577+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.077536+0000","last_change":"2024-04-25T10:05:38.076940+0000","last_active":"2024-04-25T10:05:38.077536+0000","last_peered":"2024-04-25T10:05:38.077536+0000","last_clean":"2024-04-25T10:05:38.077536+0000","last_became_active":"2024-04-25T10:05:38.076639+0000","last_became_peered":"2024-04-25T10:05:38.076639+0000","last_unstale":"2024-04-25T10:05:38.077536+0000","last_undegraded":"2024-04-25T10:05:38.077536+0000","last_fullsized":"2024-04-25T10:05:38.077536+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:20.840458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073109+0000","last_change":"2024-04-25T10:05:38.073109+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073109+0000","last_undegraded":"2024-04-25T10:05:38.073109+0000","last_fullsized":"2024-04-25T10:05:38.073109+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085058+0000","last_change":"2024-04-25T10:05:38.083896+0000","last_active":"2024-04-25T10:05:38.085058+0000","last_peered":"2024-04-25T10:05:38.085058+0000","last_clean":"2024-04-25T10:05:38.085058+0000","last_became_active":"2024-04-25T10:05:38.083309+0000","last_became_peered":"2024-04-25T10:05:38.083309+0000","last_unstale":"2024-04-25T10:05:38.085058+0000","last_undegraded":"2024-04-25T10:05:38.085058+0000","last_fullsized":"2024-04-25T10:05:38.085058+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:27.679738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074786+0000","last_change":"2024-04-25T10:05:38.074786+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074786+0000","last_undegraded":"2024-04-25T10:05:38.074786+0000","last_fullsized":"2024-04-25T10:05:38.074786+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084860+0000","last_change":"2024-04-25T10:05:38.083645+0000","last_active":"2024-04-25T10:05:38.084860+0000","last_peered":"2024-04-25T10:05:38.084860+0000","last_clean":"2024-04-25T10:05:38.084860+0000","last_became_active":"2024-04-25T10:05:38.083357+0000","last_became_peered":"2024-04-25T10:05:38.083357+0000","last_unstale":"2024-04-25T10:05:38.084860+0000","last_undegraded":"2024-04-25T10:05:38.084860+0000","last_fullsized":"2024-04-25T10:05:38.084860+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:32.268641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067142+0000","last_change":"2024-04-25T10:05:38.067142+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067142+0000","last_undegraded":"2024-04-25T10:05:38.067142+0000","last_fullsized":"2024-04-25T10:05:38.067142+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085150+0000","last_change":"2024-04-25T10:05:38.083600+0000","last_active":"2024-04-25T10:05:38.085150+0000","last_peered":"2024-04-25T10:05:38.085150+0000","last_clean":"2024-04-25T10:05:38.085150+0000","last_became_active":"2024-04-25T10:05:38.083303+0000","last_became_peered":"2024-04-25T10:05:38.083303+0000","last_unstale":"2024-04-25T10:05:38.085150+0000","last_undegraded":"2024-04-25T10:05:38.085150+0000","last_fullsized":"2024-04-25T10:05:38.085150+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:05.866090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078449+0000","last_change":"2024-04-25T10:05:38.078449+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078449+0000","last_undegraded":"2024-04-25T10:05:38.078449+0000","last_fullsized":"2024-04-25T10:05:38.078449+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084965+0000","last_change":"2024-04-25T10:05:38.081964+0000","last_active":"2024-04-25T10:05:38.084965+0000","last_peered":"2024-04-25T10:05:38.084965+0000","last_clean":"2024-04-25T10:05:38.084965+0000","last_became_active":"2024-04-25T10:05:38.081673+0000","last_became_peered":"2024-04-25T10:05:38.081673+0000","last_unstale":"2024-04-25T10:05:38.084965+0000","last_undegraded":"2024-04-25T10:05:38.084965+0000","last_fullsized":"2024-04-25T10:05:38.084965+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.599553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078013+0000","last_change":"2024-04-25T10:05:38.078013+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078013+0000","last_undegraded":"2024-04-25T10:05:38.078013+0000","last_fullsized":"2024-04-25T10:05:38.078013+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085904+0000","last_change":"2024-04-25T10:05:38.085345+0000","last_active":"2024-04-25T10:05:38.085904+0000","last_peered":"2024-04-25T10:05:38.085904+0000","last_clean":"2024-04-25T10:05:38.085904+0000","last_became_active":"2024-04-25T10:05:38.082298+0000","last_became_peered":"2024-04-25T10:05:38.082298+0000","last_unstale":"2024-04-25T10:05:38.085904+0000","last_undegraded":"2024-04-25T10:05:38.085904+0000","last_fullsized":"2024-04-25T10:05:38.085904+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:46.424209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074518+0000","last_change":"2024-04-25T10:05:38.074518+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074518+0000","last_undegraded":"2024-04-25T10:05:38.074518+0000","last_fullsized":"2024-04-25T10:05:38.074518+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084654+0000","last_change":"2024-04-25T10:05:38.083542+0000","last_active":"2024-04-25T10:05:38.084654+0000","last_peered":"2024-04-25T10:05:38.084654+0000","last_clean":"2024-04-25T10:05:38.084654+0000","last_became_active":"2024-04-25T10:05:38.082652+0000","last_became_peered":"2024-04-25T10:05:38.082652+0000","last_unstale":"2024-04-25T10:05:38.084654+0000","last_undegraded":"2024-04-25T10:05:38.084654+0000","last_fullsized":"2024-04-25T10:05:38.084654+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:36.918168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065904+0000","last_change":"2024-04-25T10:05:38.065904+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065904+0000","last_undegraded":"2024-04-25T10:05:38.065904+0000","last_fullsized":"2024-04-25T10:05:38.065904+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.080166+0000","last_change":"2024-04-25T10:05:38.077397+0000","last_active":"2024-04-25T10:05:38.080166+0000","last_peered":"2024-04-25T10:05:38.080166+0000","last_clean":"2024-04-25T10:05:38.080166+0000","last_became_active":"2024-04-25T10:05:38.077107+0000","last_became_peered":"2024-04-25T10:05:38.077107+0000","last_unstale":"2024-04-25T10:05:38.080166+0000","last_undegraded":"2024-04-25T10:05:38.080166+0000","last_fullsized":"2024-04-25T10:05:38.080166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:43.802225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073892+0000","last_change":"2024-04-25T10:05:38.073892+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073892+0000","last_undegraded":"2024-04-25T10:05:38.073892+0000","last_fullsized":"2024-04-25T10:05:38.073892+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.079630+0000","last_change":"2024-04-25T10:05:38.075167+0000","last_active":"2024-04-25T10:05:38.079630+0000","last_peered":"2024-04-25T10:05:38.079630+0000","last_clean":"2024-04-25T10:05:38.079630+0000","last_became_active":"2024-04-25T10:05:38.074890+0000","last_became_peered":"2024-04-25T10:05:38.074890+0000","last_unstale":"2024-04-25T10:05:38.079630+0000","last_undegraded":"2024-04-25T10:05:38.079630+0000","last_fullsized":"2024-04-25T10:05:38.079630+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:00.798323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076641+0000","last_change":"2024-04-25T10:05:38.076641+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076641+0000","last_undegraded":"2024-04-25T10:05:38.076641+0000","last_fullsized":"2024-04-25T10:05:38.076641+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084195+0000","last_change":"2024-04-25T10:05:38.082353+0000","last_active":"2024-04-25T10:05:38.084195+0000","last_peered":"2024-04-25T10:05:38.084195+0000","last_clean":"2024-04-25T10:05:38.084195+0000","last_became_active":"2024-04-25T10:05:38.082047+0000","last_became_peered":"2024-04-25T10:05:38.082047+0000","last_unstale":"2024-04-25T10:05:38.084195+0000","last_undegraded":"2024-04-25T10:05:38.084195+0000","last_fullsized":"2024-04-25T10:05:38.084195+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:12.478664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074243+0000","last_change":"2024-04-25T10:05:38.074243+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074243+0000","last_undegraded":"2024-04-25T10:05:38.074243+0000","last_fullsized":"2024-04-25T10:05:38.074243+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085023+0000","last_change":"2024-04-25T10:05:38.083029+0000","last_active":"2024-04-25T10:05:38.085023+0000","last_peered":"2024-04-25T10:05:38.085023+0000","last_clean":"2024-04-25T10:05:38.085023+0000","last_became_active":"2024-04-25T10:05:38.082535+0000","last_became_peered":"2024-04-25T10:05:38.082535+0000","last_unstale":"2024-04-25T10:05:38.085023+0000","last_undegraded":"2024-04-25T10:05:38.085023+0000","last_fullsized":"2024-04-25T10:05:38.085023+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:42.531775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066667+0000","last_change":"2024-04-25T10:05:38.066667+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066667+0000","last_undegraded":"2024-04-25T10:05:38.066667+0000","last_fullsized":"2024-04-25T10:05:38.066667+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076016+0000","last_change":"2024-04-25T10:05:38.073711+0000","last_active":"2024-04-25T10:05:38.076016+0000","last_peered":"2024-04-25T10:05:38.076016+0000","last_clean":"2024-04-25T10:05:38.076016+0000","last_became_active":"2024-04-25T10:05:38.073414+0000","last_became_peered":"2024-04-25T10:05:38.073414+0000","last_unstale":"2024-04-25T10:05:38.076016+0000","last_undegraded":"2024-04-25T10:05:38.076016+0000","last_fullsized":"2024-04-25T10:05:38.076016+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:10.878156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071187+0000","last_change":"2024-04-25T10:05:38.071187+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071187+0000","last_undegraded":"2024-04-25T10:05:38.071187+0000","last_fullsized":"2024-04-25T10:05:38.071187+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.073860+0000","last_change":"2024-04-25T10:05:38.071762+0000","last_active":"2024-04-25T10:05:38.073860+0000","last_peered":"2024-04-25T10:05:38.073860+0000","last_clean":"2024-04-25T10:05:38.073860+0000","last_became_active":"2024-04-25T10:05:38.071429+0000","last_became_peered":"2024-04-25T10:05:38.071429+0000","last_unstale":"2024-04-25T10:05:38.073860+0000","last_undegraded":"2024-04-25T10:05:38.073860+0000","last_fullsized":"2024-04-25T10:05:38.073860+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:54.282381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077423+0000","last_change":"2024-04-25T10:05:38.077423+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077423+0000","last_undegraded":"2024-04-25T10:05:38.077423+0000","last_fullsized":"2024-04-25T10:05:38.077423+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085218+0000","last_change":"2024-04-25T10:05:38.084273+0000","last_active":"2024-04-25T10:05:38.085218+0000","last_peered":"2024-04-25T10:05:38.085218+0000","last_clean":"2024-04-25T10:05:38.085218+0000","last_became_active":"2024-04-25T10:05:38.082528+0000","last_became_peered":"2024-04-25T10:05:38.082528+0000","last_unstale":"2024-04-25T10:05:38.085218+0000","last_undegraded":"2024-04-25T10:05:38.085218+0000","last_fullsized":"2024-04-25T10:05:38.085218+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:29.382293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.072844+0000","last_change":"2024-04-25T10:05:38.072844+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.072844+0000","last_undegraded":"2024-04-25T10:05:38.072844+0000","last_fullsized":"2024-04-25T10:05:38.072844+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.087513+0000","last_change":"2024-04-25T10:05:38.086909+0000","last_active":"2024-04-25T10:05:38.087513+0000","last_peered":"2024-04-25T10:05:38.087513+0000","last_clean":"2024-04-25T10:05:38.087513+0000","last_became_active":"2024-04-25T10:05:38.086646+0000","last_became_peered":"2024-04-25T10:05:38.086646+0000","last_unstale":"2024-04-25T10:05:38.087513+0000","last_undegraded":"2024-04-25T10:05:38.087513+0000","last_fullsized":"2024-04-25T10:05:38.087513+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:08.234904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075592+0000","last_change":"2024-04-25T10:05:38.075592+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075592+0000","last_undegraded":"2024-04-25T10:05:38.075592+0000","last_fullsized":"2024-04-25T10:05:38.075592+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081925+0000","last_change":"2024-04-25T10:05:38.081354+0000","last_active":"2024-04-25T10:05:38.081925+0000","last_peered":"2024-04-25T10:05:38.081925+0000","last_clean":"2024-04-25T10:05:38.081925+0000","last_became_active":"2024-04-25T10:05:38.081054+0000","last_became_peered":"2024-04-25T10:05:38.081054+0000","last_unstale":"2024-04-25T10:05:38.081925+0000","last_undegraded":"2024-04-25T10:05:38.081925+0000","last_fullsized":"2024-04-25T10:05:38.081925+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:04.866716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075995+0000","last_change":"2024-04-25T10:05:38.075995+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075995+0000","last_undegraded":"2024-04-25T10:05:38.075995+0000","last_fullsized":"2024-04-25T10:05:38.075995+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076017+0000","last_change":"2024-04-25T10:05:38.075546+0000","last_active":"2024-04-25T10:05:38.076017+0000","last_peered":"2024-04-25T10:05:38.076017+0000","last_clean":"2024-04-25T10:05:38.076017+0000","last_became_active":"2024-04-25T10:05:38.074936+0000","last_became_peered":"2024-04-25T10:05:38.074936+0000","last_unstale":"2024-04-25T10:05:38.076017+0000","last_undegraded":"2024-04-25T10:05:38.076017+0000","last_fullsized":"2024-04-25T10:05:38.076017+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:00.798323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077978+0000","last_change":"2024-04-25T10:05:38.077978+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077978+0000","last_undegraded":"2024-04-25T10:05:38.077978+0000","last_fullsized":"2024-04-25T10:05:38.077978+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084725+0000","last_change":"2024-04-25T10:05:38.083077+0000","last_active":"2024-04-25T10:05:38.084725+0000","last_peered":"2024-04-25T10:05:38.084725+0000","last_clean":"2024-04-25T10:05:38.084725+0000","last_became_active":"2024-04-25T10:05:38.082606+0000","last_became_peered":"2024-04-25T10:05:38.082606+0000","last_unstale":"2024-04-25T10:05:38.084725+0000","last_undegraded":"2024-04-25T10:05:38.084725+0000","last_fullsized":"2024-04-25T10:05:38.084725+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:57.974626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077178+0000","last_change":"2024-04-25T10:05:38.077178+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077178+0000","last_undegraded":"2024-04-25T10:05:38.077178+0000","last_fullsized":"2024-04-25T10:05:38.077178+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.075802+0000","last_change":"2024-04-25T10:05:38.072474+0000","last_active":"2024-04-25T10:05:38.075802+0000","last_peered":"2024-04-25T10:05:38.075802+0000","last_clean":"2024-04-25T10:05:38.075802+0000","last_became_active":"2024-04-25T10:05:38.072160+0000","last_became_peered":"2024-04-25T10:05:38.072160+0000","last_unstale":"2024-04-25T10:05:38.075802+0000","last_undegraded":"2024-04-25T10:05:38.075802+0000","last_fullsized":"2024-04-25T10:05:38.075802+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:06.976864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":510,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.069800+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:05:38.069800+0000","last_peered":"2024-04-25T10:05:38.069800+0000","last_clean":"2024-04-25T10:05:38.069800+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:05:38.069800+0000","last_undegraded":"2024-04-25T10:05:38.069800+0000","last_fullsized":"2024-04-25T10:05:38.069800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56284,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315556,"statfs":{"total":96636764160,"available":96579129344,"internally_reserved":0,"allocated":2961408,"data_stored":1700777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51544,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320296,"statfs":{"total":96636764160,"available":96583983104,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51540,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320300,"statfs":{"total":96636764160,"available":96583987200,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2355200,"data_stored":1103742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59868,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311972,"statfs":{"total":96636764160,"available":96575459328,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55696,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55708,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316132,"statfs":{"total":96636764160,"available":96579719168,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56284,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315556,"statfs":{"total":96636764160,"available":96579129344,"internally_reserved":0,"allocated":2961408,"data_stored":1700777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:42.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3f is not active+clean 2024-04-25T10:05:42.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.066637+0000', 'last_change': '2024-04-25T10:05:38.066637+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.066637+0000', 'last_undegraded': '2024-04-25T10:05:38.066637+0000', 'last_fullsized': '2024-04-25T10:05:38.066637+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:05:42.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3e is not active+clean 2024-04-25T10:05:42.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.074792+0000', 'last_change': '2024-04-25T10:05:38.074792+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.074792+0000', 'last_undegraded': '2024-04-25T10:05:38.074792+0000', 'last_fullsized': '2024-04-25T10:05:38.074792+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:05:42.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3d is not active+clean 2024-04-25T10:05:42.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.077295+0000', 'last_change': '2024-04-25T10:05:38.077295+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.077295+0000', 'last_undegraded': '2024-04-25T10:05:38.077295+0000', 'last_fullsized': '2024-04-25T10:05:38.077295+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3c is not active+clean 2024-04-25T10:05:42.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.067681+0000', 'last_change': '2024-04-25T10:05:38.067681+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.067681+0000', 'last_undegraded': '2024-04-25T10:05:38.067681+0000', 'last_fullsized': '2024-04-25T10:05:38.067681+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:05:42.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3b is not active+clean 2024-04-25T10:05:42.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.066530+0000', 'last_change': '2024-04-25T10:05:38.066530+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.066530+0000', 'last_undegraded': '2024-04-25T10:05:38.066530+0000', 'last_fullsized': '2024-04-25T10:05:38.066530+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:05:42.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3a is not active+clean 2024-04-25T10:05:42.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.071870+0000', 'last_change': '2024-04-25T10:05:38.071870+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.071870+0000', 'last_undegraded': '2024-04-25T10:05:38.071870+0000', 'last_fullsized': '2024-04-25T10:05:38.071870+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:42.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.39 is not active+clean 2024-04-25T10:05:42.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.077706+0000', 'last_change': '2024-04-25T10:05:38.077706+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.077706+0000', 'last_undegraded': '2024-04-25T10:05:38.077706+0000', 'last_fullsized': '2024-04-25T10:05:38.077706+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.775 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.38 is not active+clean 2024-04-25T10:05:42.775 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.071345+0000', 'last_change': '2024-04-25T10:05:38.071345+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.071345+0000', 'last_undegraded': '2024-04-25T10:05:38.071345+0000', 'last_fullsized': '2024-04-25T10:05:38.071345+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:42.775 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.37 is not active+clean 2024-04-25T10:05:42.775 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.076215+0000', 'last_change': '2024-04-25T10:05:38.076215+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.076215+0000', 'last_undegraded': '2024-04-25T10:05:38.076215+0000', 'last_fullsized': '2024-04-25T10:05:38.076215+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:05:42.775 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.36 is not active+clean 2024-04-25T10:05:42.775 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.081044+0000', 'last_change': '2024-04-25T10:05:38.081044+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.081044+0000', 'last_undegraded': '2024-04-25T10:05:38.081044+0000', 'last_fullsized': '2024-04-25T10:05:38.081044+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.775 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.35 is not active+clean 2024-04-25T10:05:42.775 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.068089+0000', 'last_change': '2024-04-25T10:05:38.068089+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.068089+0000', 'last_undegraded': '2024-04-25T10:05:38.068089+0000', 'last_fullsized': '2024-04-25T10:05:38.068089+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:05:42.775 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.34 is not active+clean 2024-04-25T10:05:42.775 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.076169+0000', 'last_change': '2024-04-25T10:05:38.076169+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.076169+0000', 'last_undegraded': '2024-04-25T10:05:38.076169+0000', 'last_fullsized': '2024-04-25T10:05:38.076169+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:05:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.33 is not active+clean 2024-04-25T10:05:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.070409+0000', 'last_change': '2024-04-25T10:05:38.070409+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.070409+0000', 'last_undegraded': '2024-04-25T10:05:38.070409+0000', 'last_fullsized': '2024-04-25T10:05:38.070409+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.32 is not active+clean 2024-04-25T10:05:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.076413+0000', 'last_change': '2024-04-25T10:05:38.076413+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.076413+0000', 'last_undegraded': '2024-04-25T10:05:38.076413+0000', 'last_fullsized': '2024-04-25T10:05:38.076413+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:05:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.31 is not active+clean 2024-04-25T10:05:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.076156+0000', 'last_change': '2024-04-25T10:05:38.076156+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.076156+0000', 'last_undegraded': '2024-04-25T10:05:38.076156+0000', 'last_fullsized': '2024-04-25T10:05:38.076156+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:05:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.30 is not active+clean 2024-04-25T10:05:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.070878+0000', 'last_change': '2024-04-25T10:05:38.070878+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.070878+0000', 'last_undegraded': '2024-04-25T10:05:38.070878+0000', 'last_fullsized': '2024-04-25T10:05:38.070878+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2f is not active+clean 2024-04-25T10:05:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.078479+0000', 'last_change': '2024-04-25T10:05:38.078479+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.078479+0000', 'last_undegraded': '2024-04-25T10:05:38.078479+0000', 'last_fullsized': '2024-04-25T10:05:38.078479+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:05:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2e is not active+clean 2024-04-25T10:05:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.080617+0000', 'last_change': '2024-04-25T10:05:38.080617+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.080617+0000', 'last_undegraded': '2024-04-25T10:05:38.080617+0000', 'last_fullsized': '2024-04-25T10:05:38.080617+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2d is not active+clean 2024-04-25T10:05:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.065954+0000', 'last_change': '2024-04-25T10:05:38.065954+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.065954+0000', 'last_undegraded': '2024-04-25T10:05:38.065954+0000', 'last_fullsized': '2024-04-25T10:05:38.065954+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:05:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2c is not active+clean 2024-04-25T10:05:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.070674+0000', 'last_change': '2024-04-25T10:05:38.070674+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.070674+0000', 'last_undegraded': '2024-04-25T10:05:38.070674+0000', 'last_fullsized': '2024-04-25T10:05:38.070674+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2b is not active+clean 2024-04-25T10:05:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.077115+0000', 'last_change': '2024-04-25T10:05:38.077115+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.077115+0000', 'last_undegraded': '2024-04-25T10:05:38.077115+0000', 'last_fullsized': '2024-04-25T10:05:38.077115+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2a is not active+clean 2024-04-25T10:05:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.076601+0000', 'last_change': '2024-04-25T10:05:38.076601+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.076601+0000', 'last_undegraded': '2024-04-25T10:05:38.076601+0000', 'last_fullsized': '2024-04-25T10:05:38.076601+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:05:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.29 is not active+clean 2024-04-25T10:05:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.075635+0000', 'last_change': '2024-04-25T10:05:38.075635+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.075635+0000', 'last_undegraded': '2024-04-25T10:05:38.075635+0000', 'last_fullsized': '2024-04-25T10:05:38.075635+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:05:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.28 is not active+clean 2024-04-25T10:05:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.065899+0000', 'last_change': '2024-04-25T10:05:38.065899+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.065899+0000', 'last_undegraded': '2024-04-25T10:05:38.065899+0000', 'last_fullsized': '2024-04-25T10:05:38.065899+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:05:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.27 is not active+clean 2024-04-25T10:05:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.065933+0000', 'last_change': '2024-04-25T10:05:38.065933+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.065933+0000', 'last_undegraded': '2024-04-25T10:05:38.065933+0000', 'last_fullsized': '2024-04-25T10:05:38.065933+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.26 is not active+clean 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.080059+0000', 'last_change': '2024-04-25T10:05:38.080059+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.080059+0000', 'last_undegraded': '2024-04-25T10:05:38.080059+0000', 'last_fullsized': '2024-04-25T10:05:38.080059+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.25 is not active+clean 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.067528+0000', 'last_change': '2024-04-25T10:05:38.067528+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.067528+0000', 'last_undegraded': '2024-04-25T10:05:38.067528+0000', 'last_fullsized': '2024-04-25T10:05:38.067528+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.24 is not active+clean 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.074828+0000', 'last_change': '2024-04-25T10:05:38.074828+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.074828+0000', 'last_undegraded': '2024-04-25T10:05:38.074828+0000', 'last_fullsized': '2024-04-25T10:05:38.074828+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.23 is not active+clean 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.074206+0000', 'last_change': '2024-04-25T10:05:38.074206+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.074206+0000', 'last_undegraded': '2024-04-25T10:05:38.074206+0000', 'last_fullsized': '2024-04-25T10:05:38.074206+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.22 is not active+clean 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.072514+0000', 'last_change': '2024-04-25T10:05:38.072514+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.072514+0000', 'last_undegraded': '2024-04-25T10:05:38.072514+0000', 'last_fullsized': '2024-04-25T10:05:38.072514+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.21 is not active+clean 2024-04-25T10:05:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.074387+0000', 'last_change': '2024-04-25T10:05:38.074387+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.074387+0000', 'last_undegraded': '2024-04-25T10:05:38.074387+0000', 'last_fullsized': '2024-04-25T10:05:38.074387+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:05:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.20 is not active+clean 2024-04-25T10:05:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.070297+0000', 'last_change': '2024-04-25T10:05:38.070297+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.070297+0000', 'last_undegraded': '2024-04-25T10:05:38.070297+0000', 'last_fullsized': '2024-04-25T10:05:38.070297+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1c is not active+clean 2024-04-25T10:05:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.077732+0000', 'last_change': '2024-04-25T10:05:38.077732+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.077732+0000', 'last_undegraded': '2024-04-25T10:05:38.077732+0000', 'last_fullsized': '2024-04-25T10:05:38.077732+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1d is not active+clean 2024-04-25T10:05:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.076803+0000', 'last_change': '2024-04-25T10:05:38.076803+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.076803+0000', 'last_undegraded': '2024-04-25T10:05:38.076803+0000', 'last_fullsized': '2024-04-25T10:05:38.076803+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1e is not active+clean 2024-04-25T10:05:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.074333+0000', 'last_change': '2024-04-25T10:05:38.074333+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.074333+0000', 'last_undegraded': '2024-04-25T10:05:38.074333+0000', 'last_fullsized': '2024-04-25T10:05:38.074333+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:05:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.18 is not active+clean 2024-04-25T10:05:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.079284+0000', 'last_change': '2024-04-25T10:05:38.079284+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.079284+0000', 'last_undegraded': '2024-04-25T10:05:38.079284+0000', 'last_fullsized': '2024-04-25T10:05:38.079284+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.19 is not active+clean 2024-04-25T10:05:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.073657+0000', 'last_change': '2024-04-25T10:05:38.073657+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.073657+0000', 'last_undegraded': '2024-04-25T10:05:38.073657+0000', 'last_fullsized': '2024-04-25T10:05:38.073657+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:05:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1a is not active+clean 2024-04-25T10:05:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.074714+0000', 'last_change': '2024-04-25T10:05:38.074714+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.074714+0000', 'last_undegraded': '2024-04-25T10:05:38.074714+0000', 'last_fullsized': '2024-04-25T10:05:38.074714+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:05:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1b is not active+clean 2024-04-25T10:05:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.079217+0000', 'last_change': '2024-04-25T10:05:38.079217+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.079217+0000', 'last_undegraded': '2024-04-25T10:05:38.079217+0000', 'last_fullsized': '2024-04-25T10:05:38.079217+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:05:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.14 is not active+clean 2024-04-25T10:05:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.073656+0000', 'last_change': '2024-04-25T10:05:38.073656+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.073656+0000', 'last_undegraded': '2024-04-25T10:05:38.073656+0000', 'last_fullsized': '2024-04-25T10:05:38.073656+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:05:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.15 is not active+clean 2024-04-25T10:05:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.074188+0000', 'last_change': '2024-04-25T10:05:38.074188+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.074188+0000', 'last_undegraded': '2024-04-25T10:05:38.074188+0000', 'last_fullsized': '2024-04-25T10:05:38.074188+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:05:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.16 is not active+clean 2024-04-25T10:05:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.079561+0000', 'last_change': '2024-04-25T10:05:38.079561+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.079561+0000', 'last_undegraded': '2024-04-25T10:05:38.079561+0000', 'last_fullsized': '2024-04-25T10:05:38.079561+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.17 is not active+clean 2024-04-25T10:05:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.068047+0000', 'last_change': '2024-04-25T10:05:38.068047+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.068047+0000', 'last_undegraded': '2024-04-25T10:05:38.068047+0000', 'last_fullsized': '2024-04-25T10:05:38.068047+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:05:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.10 is not active+clean 2024-04-25T10:05:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.074213+0000', 'last_change': '2024-04-25T10:05:38.074213+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.074213+0000', 'last_undegraded': '2024-04-25T10:05:38.074213+0000', 'last_fullsized': '2024-04-25T10:05:38.074213+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:05:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.11 is not active+clean 2024-04-25T10:05:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.073003+0000', 'last_change': '2024-04-25T10:05:38.073003+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.073003+0000', 'last_undegraded': '2024-04-25T10:05:38.073003+0000', 'last_fullsized': '2024-04-25T10:05:38.073003+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:05:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.12 is not active+clean 2024-04-25T10:05:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.077577+0000', 'last_change': '2024-04-25T10:05:38.077577+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.077577+0000', 'last_undegraded': '2024-04-25T10:05:38.077577+0000', 'last_fullsized': '2024-04-25T10:05:38.077577+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.13 is not active+clean 2024-04-25T10:05:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.073109+0000', 'last_change': '2024-04-25T10:05:38.073109+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.073109+0000', 'last_undegraded': '2024-04-25T10:05:38.073109+0000', 'last_fullsized': '2024-04-25T10:05:38.073109+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:05:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.c is not active+clean 2024-04-25T10:05:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.074786+0000', 'last_change': '2024-04-25T10:05:38.074786+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.074786+0000', 'last_undegraded': '2024-04-25T10:05:38.074786+0000', 'last_fullsized': '2024-04-25T10:05:38.074786+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:05:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.d is not active+clean 2024-04-25T10:05:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.067142+0000', 'last_change': '2024-04-25T10:05:38.067142+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.067142+0000', 'last_undegraded': '2024-04-25T10:05:38.067142+0000', 'last_fullsized': '2024-04-25T10:05:38.067142+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:05:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.e is not active+clean 2024-04-25T10:05:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.078449+0000', 'last_change': '2024-04-25T10:05:38.078449+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.078449+0000', 'last_undegraded': '2024-04-25T10:05:38.078449+0000', 'last_fullsized': '2024-04-25T10:05:38.078449+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.f is not active+clean 2024-04-25T10:05:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.078013+0000', 'last_change': '2024-04-25T10:05:38.078013+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.078013+0000', 'last_undegraded': '2024-04-25T10:05:38.078013+0000', 'last_fullsized': '2024-04-25T10:05:38.078013+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:05:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.8 is not active+clean 2024-04-25T10:05:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.074518+0000', 'last_change': '2024-04-25T10:05:38.074518+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.074518+0000', 'last_undegraded': '2024-04-25T10:05:38.074518+0000', 'last_fullsized': '2024-04-25T10:05:38.074518+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:05:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.9 is not active+clean 2024-04-25T10:05:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.065904+0000', 'last_change': '2024-04-25T10:05:38.065904+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.065904+0000', 'last_undegraded': '2024-04-25T10:05:38.065904+0000', 'last_fullsized': '2024-04-25T10:05:38.065904+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:05:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.a is not active+clean 2024-04-25T10:05:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.073892+0000', 'last_change': '2024-04-25T10:05:38.073892+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.073892+0000', 'last_undegraded': '2024-04-25T10:05:38.073892+0000', 'last_fullsized': '2024-04-25T10:05:38.073892+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:05:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.b is not active+clean 2024-04-25T10:05:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.076641+0000', 'last_change': '2024-04-25T10:05:38.076641+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.076641+0000', 'last_undegraded': '2024-04-25T10:05:38.076641+0000', 'last_fullsized': '2024-04-25T10:05:38.076641+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.4 is not active+clean 2024-04-25T10:05:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.074243+0000', 'last_change': '2024-04-25T10:05:38.074243+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.074243+0000', 'last_undegraded': '2024-04-25T10:05:38.074243+0000', 'last_fullsized': '2024-04-25T10:05:38.074243+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:05:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.5 is not active+clean 2024-04-25T10:05:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.066667+0000', 'last_change': '2024-04-25T10:05:38.066667+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.066667+0000', 'last_undegraded': '2024-04-25T10:05:38.066667+0000', 'last_fullsized': '2024-04-25T10:05:38.066667+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:05:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.6 is not active+clean 2024-04-25T10:05:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.071187+0000', 'last_change': '2024-04-25T10:05:38.071187+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.071187+0000', 'last_undegraded': '2024-04-25T10:05:38.071187+0000', 'last_fullsized': '2024-04-25T10:05:38.071187+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:05:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.7 is not active+clean 2024-04-25T10:05:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.077423+0000', 'last_change': '2024-04-25T10:05:38.077423+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.077423+0000', 'last_undegraded': '2024-04-25T10:05:38.077423+0000', 'last_fullsized': '2024-04-25T10:05:38.077423+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:05:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.0 is not active+clean 2024-04-25T10:05:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.072844+0000', 'last_change': '2024-04-25T10:05:38.072844+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.072844+0000', 'last_undegraded': '2024-04-25T10:05:38.072844+0000', 'last_fullsized': '2024-04-25T10:05:38.072844+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1 is not active+clean 2024-04-25T10:05:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.075592+0000', 'last_change': '2024-04-25T10:05:38.075592+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.075592+0000', 'last_undegraded': '2024-04-25T10:05:38.075592+0000', 'last_fullsized': '2024-04-25T10:05:38.075592+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2 is not active+clean 2024-04-25T10:05:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.075995+0000', 'last_change': '2024-04-25T10:05:38.075995+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.075995+0000', 'last_undegraded': '2024-04-25T10:05:38.075995+0000', 'last_fullsized': '2024-04-25T10:05:38.075995+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:05:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3 is not active+clean 2024-04-25T10:05:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.077978+0000', 'last_change': '2024-04-25T10:05:38.077978+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.077978+0000', 'last_undegraded': '2024-04-25T10:05:38.077978+0000', 'last_fullsized': '2024-04-25T10:05:38.077978+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:05:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1f is not active+clean 2024-04-25T10:05:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 187, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:05:38.077178+0000', 'last_change': '2024-04-25T10:05:38.077178+0000', 'last_active': '2024-04-25T10:05:38.052841+0000', 'last_peered': '2024-04-25T10:05:38.052841+0000', 'last_clean': '2024-04-25T10:05:38.052841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:38.077178+0000', 'last_undegraded': '2024-04-25T10:05:38.077178+0000', 'last_fullsized': '2024-04-25T10:05:38.077178+0000', 'mapping_epoch': 187, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 187, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:38.052841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:05:42.786 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:05:42.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.904+0000 7fdf10e6b640 1 Processor -- start 2024-04-25T10:05:42.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.904+0000 7fdf10e6b640 1 -- start start 2024-04-25T10:05:42.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf10e6b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdf0c15dbe0 0x7fdf0c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf10e6b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf0c0b2cb0 0x7fdf0c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf10e6b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0b97a0 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf10e6b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c15dfc0 con 0x7fdf0c0b2cb0 2024-04-25T10:05:42.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf10e6b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c0d1a00 con 0x7fdf0c15dbe0 2024-04-25T10:05:42.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf03fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf0c0b2cb0 0x7fdf0c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf0ad9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdf0c0b92a0 0x7fdf0c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34892/0 (socket says 172.21.15.19:34892) 2024-04-25T10:05:42.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf03fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf0c0b2cb0 0x7fdf0c0d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35814/0 (socket says 172.21.15.19:35814) 2024-04-25T10:05:42.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf0ad9d640 1 -- 172.21.15.19:0/458386136 learned_addr learned my addr 172.21.15.19:0/458386136 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:42.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf03fff640 1 -- 172.21.15.19:0/458386136 >> v2:172.21.15.112:3300/0 conn(0x7fdf0c15dbe0 msgr2=0x7fdf0c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:42.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf03fff640 1 --2- 172.21.15.19:0/458386136 >> v2:172.21.15.112:3300/0 conn(0x7fdf0c15dbe0 0x7fdf0c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf03fff640 1 -- 172.21.15.19:0/458386136 >> v1:172.21.15.19:6789/0 conn(0x7fdf0c0b92a0 legacy=0x7fdf0c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf03fff640 1 -- 172.21.15.19:0/458386136 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0d1c50 con 0x7fdf0c0b2cb0 2024-04-25T10:05:42.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.905+0000 7fdf09d9b640 1 -- 172.21.15.19:0/458386136 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868287532 0 0) 0x7fdf0c0b97a0 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.906+0000 7fdf03fff640 1 --2- 172.21.15.19:0/458386136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c0b2cb0 0x7fdf0c0d0e50 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c06c140 tx=0x7fdef402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1e6acc7b72eaba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.906+0000 7fdf09d9b640 1 -- 172.21.15.19:0/458386136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4037070 con 0x7fdf0c0b2cb0 2024-04-25T10:05:42.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.906+0000 7fdf09d9b640 1 -- 172.21.15.19:0/458386136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef4032440 con 0x7fdf0c0b2cb0 2024-04-25T10:05:42.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.906+0000 7fdf09d9b640 1 -- 172.21.15.19:0/458386136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef402ae60 con 0x7fdf0c0b2cb0 2024-04-25T10:05:42.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.906+0000 7fdf10e6b640 1 -- 172.21.15.19:0/458386136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c0b2cb0 msgr2=0x7fdf0c0d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.906+0000 7fdf10e6b640 1 --2- 172.21.15.19:0/458386136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c0b2cb0 0x7fdf0c0d0e50 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c06c140 tx=0x7fdef402a110 comp rx=0 tx=0).stop 2024-04-25T10:05:42.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.907+0000 7fdf10e6b640 1 -- 172.21.15.19:0/458386136 shutdown_connections 2024-04-25T10:05:42.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.907+0000 7fdf10e6b640 1 --2- 172.21.15.19:0/458386136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c0b2cb0 0x7fdf0c0d0e50 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.907+0000 7fdf10e6b640 1 --2- 172.21.15.19:0/458386136 >> v2:172.21.15.112:3300/0 conn(0x7fdf0c15dbe0 0x7fdf0c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.907+0000 7fdf10e6b640 1 -- 172.21.15.19:0/458386136 >> 172.21.15.19:0/458386136 conn(0x7fdf0c061a20 msgr2=0x7fdf0c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.907+0000 7fdf10e6b640 1 -- 172.21.15.19:0/458386136 shutdown_connections 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.907+0000 7fdf10e6b640 1 -- 172.21.15.19:0/458386136 wait complete. 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.908+0000 7fdf10e6b640 1 Processor -- start 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.908+0000 7fdf10e6b640 1 -- start start 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf10e6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf0c0b2cb0 0x7fdf0c0b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf10e6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c0b92a0 0x7fdf0c151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf10e6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf0c15dbe0 0x7fdf0c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf10e6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0b6ab0 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf10e6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0d2210 con 0x7fdf0c15dbe0 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf10e6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0b6c30 con 0x7fdf0c0b2cb0 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf03fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c0b92a0 0x7fdf0c151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf03fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c0b92a0 0x7fdf0c151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35816/0 (socket says 172.21.15.19:35816) 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf03fff640 1 -- 172.21.15.19:0/599629863 learned_addr learned my addr 172.21.15.19:0/599629863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf0ad9d640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf0c15dbe0 0x7fdf0c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf0a59c640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf0c0b2cb0 0x7fdf0c0b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf03fff640 1 -- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf0c0b2cb0 msgr2=0x7fdf0c0b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf03fff640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf0c0b2cb0 0x7fdf0c0b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf03fff640 1 -- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf0c15dbe0 msgr2=0x7fdf0c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf03fff640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf0c15dbe0 0x7fdf0c0b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf03fff640 1 -- 172.21.15.19:0/599629863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c1520d0 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf03fff640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c0b92a0 0x7fdf0c151b90 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fdef4000c00 tx=0x7fdef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf02ffd640 1 -- 172.21.15.19:0/599629863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4041070 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf02ffd640 1 -- 172.21.15.19:0/599629863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef403c950 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf10e6b640 1 -- 172.21.15.19:0/599629863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c0b4b90 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf02ffd640 1 -- 172.21.15.19:0/599629863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4037040 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.909+0000 7fdf10e6b640 1 -- 172.21.15.19:0/599629863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c0b5150 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.910+0000 7fdf10e6b640 1 -- 172.21.15.19:0/599629863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c073990 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.911+0000 7fdf02ffd640 1 -- 172.21.15.19:0/599629863 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdef4044070 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.911+0000 7fdf02ffd640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdeec06b4d0 0x7fdeec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.911+0000 7fdf02ffd640 1 -- 172.21.15.19:0/599629863 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdef40ab2e0 con 0x7fdf0c0b92a0 2024-04-25T10:05:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.911+0000 7fdf0a59c640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdeec06b4d0 0x7fdeec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.912+0000 7fdf0a59c640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdeec06b4d0 0x7fdeec06d990 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c1608f0 tx=0x7fdefc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:42.916+0000 7fdf02ffd640 1 -- 172.21.15.19:0/599629863 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdef407a0e0 con 0x7fdf0c0b92a0 2024-04-25T10:05:43.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.164+0000 7fdf10e6b640 1 -- 172.21.15.19:0/599629863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fdf0c0b6280 con 0x7fdf0c0b92a0 2024-04-25T10:05:43.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.164+0000 7fdf02ffd640 1 -- 172.21.15.19:0/599629863 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1459 (secure 0 0 0) 0x7fdef40799e0 con 0x7fdf0c0b92a0 2024-04-25T10:05:43.170 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:43.170 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":780,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":189,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038773,"num_in_osds":8,"osd_in_since":1714038764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":457494528,"bytes_avail":772636618752,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":123,"id":14,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":14,"rank":0,"name":"c","status":"up:active","gid":11559}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":35,"modified":"2024-04-25T10:05:27.085700+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:05:43.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 -- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdeec06b4d0 msgr2=0x7fdeec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdeec06b4d0 0x7fdeec06d990 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c1608f0 tx=0x7fdefc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:43.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 -- 172.21.15.19:0/599629863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c0b92a0 msgr2=0x7fdf0c151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c0b92a0 0x7fdf0c151b90 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fdef4000c00 tx=0x7fdef40046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 -- 172.21.15.19:0/599629863 shutdown_connections 2024-04-25T10:05:43.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdeec06b4d0 0x7fdeec06d990 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf0c15dbe0 0x7fdf0c0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf0c0b92a0 0x7fdf0c151b90 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 --2- 172.21.15.19:0/599629863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf0c0b2cb0 0x7fdf0c0b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 -- 172.21.15.19:0/599629863 >> 172.21.15.19:0/599629863 conn(0x7fdf0c061a20 msgr2=0x7fdf0c15c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 -- 172.21.15.19:0/599629863 shutdown_connections 2024-04-25T10:05:43.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.166+0000 7fdf10e6b640 1 -- 172.21.15.19:0/599629863 wait complete. 2024-04-25T10:05:43.183 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:05:43.183 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.278+0000 7ff5cf09a640 1 Processor -- start 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.279+0000 7ff5cf09a640 1 -- start start 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.279+0000 7ff5cf09a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c806ce10 0x7ff5c80d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.279+0000 7ff5cf09a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c806c220 con 0x7ff5c809a450 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.279+0000 7ff5cf09a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c80d0080 con 0x7ff5c815dc70 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.279+0000 7ff5cf09a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80d0c90 con 0x7ff5c806ce10 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.279+0000 7ff5cd637640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff5c809a450 0x7ff5c80cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34902/0 (socket says 172.21.15.19:34902) 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.279+0000 7ff5cd637640 1 -- 172.21.15.19:0/3468686771 learned_addr learned my addr 172.21.15.19:0/3468686771 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c7fff640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c806ce10 0x7ff5c80d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436629838 0 0) 0x7ff5c806c220 con 0x7ff5c809a450 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5b8003520 con 0x7ff5c809a450 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881689264 0 0) 0x7ff5c80d0080 con 0x7ff5c815dc70 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5c806c220 con 0x7ff5c815dc70 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2912003410 0 0) 0x7ff5b8003520 con 0x7ff5c809a450 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c806ce10 msgr2=0x7ff5c80d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c806ce10 0x7ff5c80d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 >> v1:172.21.15.112:6789/0 conn(0x7ff5c815dc70 legacy=0x7ff5c806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80d0360 con 0x7ff5c809a450 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7ff5c00019f0 con 0x7ff5c809a450 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c7fff640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c806ce10 0x7ff5c80d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 >> v1:172.21.15.19:6789/0 conn(0x7ff5c809a450 legacy=0x7ff5c80cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 0x7ff5b8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5b8004800 0x7ff5b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c806ce10 0x7ff5b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80d0c90 con 0x7ff5b8003b70 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b800ba70 con 0x7ff5b8004800 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b800bc40 con 0x7ff5c806ce10 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5cce36640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 0x7ff5b8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5c7fff640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5b8004800 0x7ff5b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.280+0000 7ff5cd637640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c806ce10 0x7ff5b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.281+0000 7ff5cce36640 1 -- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c806ce10 msgr2=0x7ff5b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.281+0000 7ff5cce36640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c806ce10 0x7ff5b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.281+0000 7ff5cce36640 1 -- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5b8004800 msgr2=0x7ff5b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.281+0000 7ff5cce36640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5b8004800 0x7ff5b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.281+0000 7ff5cce36640 1 -- 172.21.15.19:0/3468686771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b8003520 con 0x7ff5b8003b70 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.281+0000 7ff5cce36640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 0x7ff5b8003f70 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0018a0 tx=0x7ff5b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fc5b78a9c3e024 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.281+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc0014f0 con 0x7ff5b8003b70 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.281+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc0052d0 con 0x7ff5b8003b70 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.281+0000 7ff5c77fe640 1 -- 172.21.15.19:0/3468686771 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc02ac40 con 0x7ff5b8003b70 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.282+0000 7ff5cf09a640 1 -- 172.21.15.19:0/3468686771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 msgr2=0x7ff5b8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.282+0000 7ff5cf09a640 1 --2- 172.21.15.19:0/3468686771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 0x7ff5b8003f70 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0018a0 tx=0x7ff5b8005830 comp rx=0 tx=0).stop 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.282+0000 7ff5cd637640 1 -- 172.21.15.19:0/3468686771 reap_dead start 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.282+0000 7ff5cf09a640 1 -- 172.21.15.19:0/3468686771 shutdown_connections 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.282+0000 7ff5cf09a640 1 -- 172.21.15.19:0/3468686771 >> 172.21.15.19:0/3468686771 conn(0x7ff5c80b3270 msgr2=0x7ff5c80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.282+0000 7ff5cf09a640 1 -- 172.21.15.19:0/3468686771 shutdown_connections 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.282+0000 7ff5cf09a640 1 -- 172.21.15.19:0/3468686771 wait complete. 2024-04-25T10:05:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.283+0000 7ff5cf09a640 1 Processor -- start 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.283+0000 7ff5cf09a640 1 -- start start 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5cf09a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 0x7ff5c8157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5cf09a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c80cacf0 0x7ff5c815b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5cf09a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c815b760 0x7ff5c81587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5cf09a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c815e050 con 0x7ff5b8003b70 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5cf09a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80d0920 con 0x7ff5c815b760 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5cf09a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80d1120 con 0x7ff5c80cacf0 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5cce36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 0x7ff5c8157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5cce36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 0x7ff5c8157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35832/0 (socket says 172.21.15.19:35832) 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5cce36640 1 -- 172.21.15.19:0/2422019189 learned_addr learned my addr 172.21.15.19:0/2422019189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5c7fff640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c80cacf0 0x7ff5c815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5cce36640 1 -- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c80cacf0 msgr2=0x7ff5c815b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.284+0000 7ff5cd637640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c815b760 0x7ff5c81587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.285+0000 7ff5cce36640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c80cacf0 0x7ff5c815b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.285+0000 7ff5cce36640 1 -- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c815b760 msgr2=0x7ff5c81587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.285+0000 7ff5cce36640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c815b760 0x7ff5c81587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.285+0000 7ff5cce36640 1 -- 172.21.15.19:0/2422019189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80c9c50 con 0x7ff5b8003b70 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.285+0000 7ff5cce36640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 0x7ff5c8157ea0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc001990 tx=0x7ff5bc0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.285+0000 7ff5c57fa640 1 -- 172.21.15.19:0/2422019189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc03f020 con 0x7ff5b8003b70 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.285+0000 7ff5c57fa640 1 -- 172.21.15.19:0/2422019189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc005480 con 0x7ff5b8003b70 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.285+0000 7ff5c57fa640 1 -- 172.21.15.19:0/2422019189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc037040 con 0x7ff5b8003b70 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.285+0000 7ff5cf09a640 1 -- 172.21.15.19:0/2422019189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c80c9f30 con 0x7ff5b8003b70 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.285+0000 7ff5cf09a640 1 -- 172.21.15.19:0/2422019189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c80ca4f0 con 0x7ff5b8003b70 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.286+0000 7ff5cf09a640 1 -- 172.21.15.19:0/2422019189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c8073990 con 0x7ff5b8003b70 2024-04-25T10:05:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.286+0000 7ff5c57fa640 1 -- 172.21.15.19:0/2422019189 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5bc042020 con 0x7ff5b8003b70 2024-04-25T10:05:43.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.287+0000 7ff5c57fa640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5ac06b4d0 0x7ff5ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.287+0000 7ff5c57fa640 1 -- 172.21.15.19:0/2422019189 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5bc0aa1a0 con 0x7ff5b8003b70 2024-04-25T10:05:43.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.287+0000 7ff5c7fff640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5ac06b4d0 0x7ff5ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.287+0000 7ff5c7fff640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5ac06b4d0 0x7ff5ac06d990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80c8550 tx=0x7ff5b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.291+0000 7ff5c57fa640 1 -- 172.21.15.19:0/2422019189 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5bc049030 con 0x7ff5b8003b70 2024-04-25T10:05:43.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.467+0000 7ff5cf09a640 1 -- 172.21.15.19:0/2422019189 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5c80b3720 con 0x7ff5ac06b4d0 2024-04-25T10:05:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.473+0000 7ff5c57fa640 1 -- 172.21.15.19:0/2422019189 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247082 (secure 0 0 0) 0x7ff5c80b3720 con 0x7ff5ac06b4d0 2024-04-25T10:05:43.481 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:05:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 -- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5ac06b4d0 msgr2=0x7ff5ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5ac06b4d0 0x7ff5ac06d990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80c8550 tx=0x7ff5b4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 -- 172.21.15.19:0/2422019189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 msgr2=0x7ff5c8157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 0x7ff5c8157ea0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc001990 tx=0x7ff5bc0012d0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 -- 172.21.15.19:0/2422019189 shutdown_connections 2024-04-25T10:05:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff5ac06b4d0 0x7ff5ac06d990 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5c815b760 0x7ff5c81587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5c80cacf0 0x7ff5c815b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 --2- 172.21.15.19:0/2422019189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5b8003b70 0x7ff5c8157ea0 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 -- 172.21.15.19:0/2422019189 >> 172.21.15.19:0/2422019189 conn(0x7ff5c80b3270 msgr2=0x7ff5c8070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 -- 172.21.15.19:0/2422019189 shutdown_connections 2024-04-25T10:05:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:43.477+0000 7ff5cf09a640 1 -- 172.21.15.19:0/2422019189 wait complete. 2024-04-25T10:05:43.493 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":567,"stamp":"2024-04-25T10:05:43.089160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":446772,"kb_used_data":19644,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754527948,"statfs":{"total":773094113280,"available":772636618752,"internally_reserved":0,"allocated":20115456,"data_stored":10057341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":437243442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001342"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066637+0000","last_change":"2024-04-25T10:05:38.066637+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066637+0000","last_undegraded":"2024-04-25T10:05:38.066637+0000","last_fullsized":"2024-04-25T10:05:38.066637+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074792+0000","last_change":"2024-04-25T10:05:38.074792+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074792+0000","last_undegraded":"2024-04-25T10:05:38.074792+0000","last_fullsized":"2024-04-25T10:05:38.074792+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077295+0000","last_change":"2024-04-25T10:05:38.077295+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077295+0000","last_undegraded":"2024-04-25T10:05:38.077295+0000","last_fullsized":"2024-04-25T10:05:38.077295+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067681+0000","last_change":"2024-04-25T10:05:38.067681+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067681+0000","last_undegraded":"2024-04-25T10:05:38.067681+0000","last_fullsized":"2024-04-25T10:05:38.067681+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066530+0000","last_change":"2024-04-25T10:05:38.066530+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066530+0000","last_undegraded":"2024-04-25T10:05:38.066530+0000","last_fullsized":"2024-04-25T10:05:38.066530+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071870+0000","last_change":"2024-04-25T10:05:38.071870+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071870+0000","last_undegraded":"2024-04-25T10:05:38.071870+0000","last_fullsized":"2024-04-25T10:05:38.071870+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077706+0000","last_change":"2024-04-25T10:05:38.077706+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077706+0000","last_undegraded":"2024-04-25T10:05:38.077706+0000","last_fullsized":"2024-04-25T10:05:38.077706+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071345+0000","last_change":"2024-04-25T10:05:38.071345+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071345+0000","last_undegraded":"2024-04-25T10:05:38.071345+0000","last_fullsized":"2024-04-25T10:05:38.071345+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076215+0000","last_change":"2024-04-25T10:05:38.076215+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076215+0000","last_undegraded":"2024-04-25T10:05:38.076215+0000","last_fullsized":"2024-04-25T10:05:38.076215+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.081044+0000","last_change":"2024-04-25T10:05:38.081044+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.081044+0000","last_undegraded":"2024-04-25T10:05:38.081044+0000","last_fullsized":"2024-04-25T10:05:38.081044+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.068089+0000","last_change":"2024-04-25T10:05:38.068089+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.068089+0000","last_undegraded":"2024-04-25T10:05:38.068089+0000","last_fullsized":"2024-04-25T10:05:38.068089+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076169+0000","last_change":"2024-04-25T10:05:38.076169+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076169+0000","last_undegraded":"2024-04-25T10:05:38.076169+0000","last_fullsized":"2024-04-25T10:05:38.076169+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070409+0000","last_change":"2024-04-25T10:05:38.070409+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070409+0000","last_undegraded":"2024-04-25T10:05:38.070409+0000","last_fullsized":"2024-04-25T10:05:38.070409+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076413+0000","last_change":"2024-04-25T10:05:38.076413+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076413+0000","last_undegraded":"2024-04-25T10:05:38.076413+0000","last_fullsized":"2024-04-25T10:05:38.076413+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076156+0000","last_change":"2024-04-25T10:05:38.076156+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076156+0000","last_undegraded":"2024-04-25T10:05:38.076156+0000","last_fullsized":"2024-04-25T10:05:38.076156+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070878+0000","last_change":"2024-04-25T10:05:38.070878+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070878+0000","last_undegraded":"2024-04-25T10:05:38.070878+0000","last_fullsized":"2024-04-25T10:05:38.070878+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078479+0000","last_change":"2024-04-25T10:05:38.078479+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078479+0000","last_undegraded":"2024-04-25T10:05:38.078479+0000","last_fullsized":"2024-04-25T10:05:38.078479+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.080617+0000","last_change":"2024-04-25T10:05:38.080617+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.080617+0000","last_undegraded":"2024-04-25T10:05:38.080617+0000","last_fullsized":"2024-04-25T10:05:38.080617+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065954+0000","last_change":"2024-04-25T10:05:38.065954+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065954+0000","last_undegraded":"2024-04-25T10:05:38.065954+0000","last_fullsized":"2024-04-25T10:05:38.065954+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070674+0000","last_change":"2024-04-25T10:05:38.070674+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070674+0000","last_undegraded":"2024-04-25T10:05:38.070674+0000","last_fullsized":"2024-04-25T10:05:38.070674+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077115+0000","last_change":"2024-04-25T10:05:38.077115+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077115+0000","last_undegraded":"2024-04-25T10:05:38.077115+0000","last_fullsized":"2024-04-25T10:05:38.077115+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076601+0000","last_change":"2024-04-25T10:05:38.076601+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076601+0000","last_undegraded":"2024-04-25T10:05:38.076601+0000","last_fullsized":"2024-04-25T10:05:38.076601+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075635+0000","last_change":"2024-04-25T10:05:38.075635+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075635+0000","last_undegraded":"2024-04-25T10:05:38.075635+0000","last_fullsized":"2024-04-25T10:05:38.075635+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065899+0000","last_change":"2024-04-25T10:05:38.065899+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065899+0000","last_undegraded":"2024-04-25T10:05:38.065899+0000","last_fullsized":"2024-04-25T10:05:38.065899+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065933+0000","last_change":"2024-04-25T10:05:38.065933+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065933+0000","last_undegraded":"2024-04-25T10:05:38.065933+0000","last_fullsized":"2024-04-25T10:05:38.065933+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.080059+0000","last_change":"2024-04-25T10:05:38.080059+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.080059+0000","last_undegraded":"2024-04-25T10:05:38.080059+0000","last_fullsized":"2024-04-25T10:05:38.080059+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067528+0000","last_change":"2024-04-25T10:05:38.067528+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067528+0000","last_undegraded":"2024-04-25T10:05:38.067528+0000","last_fullsized":"2024-04-25T10:05:38.067528+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074828+0000","last_change":"2024-04-25T10:05:38.074828+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074828+0000","last_undegraded":"2024-04-25T10:05:38.074828+0000","last_fullsized":"2024-04-25T10:05:38.074828+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074206+0000","last_change":"2024-04-25T10:05:38.074206+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074206+0000","last_undegraded":"2024-04-25T10:05:38.074206+0000","last_fullsized":"2024-04-25T10:05:38.074206+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.072514+0000","last_change":"2024-04-25T10:05:38.072514+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.072514+0000","last_undegraded":"2024-04-25T10:05:38.072514+0000","last_fullsized":"2024-04-25T10:05:38.072514+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074387+0000","last_change":"2024-04-25T10:05:38.074387+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074387+0000","last_undegraded":"2024-04-25T10:05:38.074387+0000","last_fullsized":"2024-04-25T10:05:38.074387+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.070297+0000","last_change":"2024-04-25T10:05:38.070297+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.070297+0000","last_undegraded":"2024-04-25T10:05:38.070297+0000","last_fullsized":"2024-04-25T10:05:38.070297+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077732+0000","last_change":"2024-04-25T10:05:38.077732+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077732+0000","last_undegraded":"2024-04-25T10:05:38.077732+0000","last_fullsized":"2024-04-25T10:05:38.077732+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.086219+0000","last_change":"2024-04-25T10:05:38.085541+0000","last_active":"2024-04-25T10:05:38.086219+0000","last_peered":"2024-04-25T10:05:38.086219+0000","last_clean":"2024-04-25T10:05:38.086219+0000","last_became_active":"2024-04-25T10:05:38.085242+0000","last_became_peered":"2024-04-25T10:05:38.085242+0000","last_unstale":"2024-04-25T10:05:38.086219+0000","last_undegraded":"2024-04-25T10:05:38.086219+0000","last_fullsized":"2024-04-25T10:05:38.086219+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:07.815434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076803+0000","last_change":"2024-04-25T10:05:38.076803+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076803+0000","last_undegraded":"2024-04-25T10:05:38.076803+0000","last_fullsized":"2024-04-25T10:05:38.076803+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084928+0000","last_change":"2024-04-25T10:05:38.081472+0000","last_active":"2024-04-25T10:05:38.084928+0000","last_peered":"2024-04-25T10:05:38.084928+0000","last_clean":"2024-04-25T10:05:38.084928+0000","last_became_active":"2024-04-25T10:05:38.081187+0000","last_became_peered":"2024-04-25T10:05:38.081187+0000","last_unstale":"2024-04-25T10:05:38.084928+0000","last_undegraded":"2024-04-25T10:05:38.084928+0000","last_fullsized":"2024-04-25T10:05:38.084928+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:38.872584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074333+0000","last_change":"2024-04-25T10:05:38.074333+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074333+0000","last_undegraded":"2024-04-25T10:05:38.074333+0000","last_fullsized":"2024-04-25T10:05:38.074333+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.080465+0000","last_change":"2024-04-25T10:05:38.079620+0000","last_active":"2024-04-25T10:05:38.080465+0000","last_peered":"2024-04-25T10:05:38.080465+0000","last_clean":"2024-04-25T10:05:38.080465+0000","last_became_active":"2024-04-25T10:05:38.079361+0000","last_became_peered":"2024-04-25T10:05:38.079361+0000","last_unstale":"2024-04-25T10:05:38.080465+0000","last_undegraded":"2024-04-25T10:05:38.080465+0000","last_fullsized":"2024-04-25T10:05:38.080465+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:46.032506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079284+0000","last_change":"2024-04-25T10:05:38.079284+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079284+0000","last_undegraded":"2024-04-25T10:05:38.079284+0000","last_fullsized":"2024-04-25T10:05:38.079284+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084474+0000","last_change":"2024-04-25T10:05:38.083940+0000","last_active":"2024-04-25T10:05:38.084474+0000","last_peered":"2024-04-25T10:05:38.084474+0000","last_clean":"2024-04-25T10:05:38.084474+0000","last_became_active":"2024-04-25T10:05:38.083619+0000","last_became_peered":"2024-04-25T10:05:38.083619+0000","last_unstale":"2024-04-25T10:05:38.084474+0000","last_undegraded":"2024-04-25T10:05:38.084474+0000","last_fullsized":"2024-04-25T10:05:38.084474+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.073144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073657+0000","last_change":"2024-04-25T10:05:38.073657+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073657+0000","last_undegraded":"2024-04-25T10:05:38.073657+0000","last_fullsized":"2024-04-25T10:05:38.073657+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.083185+0000","last_change":"2024-04-25T10:05:38.079975+0000","last_active":"2024-04-25T10:05:38.083185+0000","last_peered":"2024-04-25T10:05:38.083185+0000","last_clean":"2024-04-25T10:05:38.083185+0000","last_became_active":"2024-04-25T10:05:38.079671+0000","last_became_peered":"2024-04-25T10:05:38.079671+0000","last_unstale":"2024-04-25T10:05:38.083185+0000","last_undegraded":"2024-04-25T10:05:38.083185+0000","last_fullsized":"2024-04-25T10:05:38.083185+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:43.802225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074714+0000","last_change":"2024-04-25T10:05:38.074714+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074714+0000","last_undegraded":"2024-04-25T10:05:38.074714+0000","last_fullsized":"2024-04-25T10:05:38.074714+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084915+0000","last_change":"2024-04-25T10:05:38.082341+0000","last_active":"2024-04-25T10:05:38.084915+0000","last_peered":"2024-04-25T10:05:38.084915+0000","last_clean":"2024-04-25T10:05:38.084915+0000","last_became_active":"2024-04-25T10:05:38.082061+0000","last_became_peered":"2024-04-25T10:05:38.082061+0000","last_unstale":"2024-04-25T10:05:38.084915+0000","last_undegraded":"2024-04-25T10:05:38.084915+0000","last_fullsized":"2024-04-25T10:05:38.084915+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:28.373524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079217+0000","last_change":"2024-04-25T10:05:38.079217+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079217+0000","last_undegraded":"2024-04-25T10:05:38.079217+0000","last_fullsized":"2024-04-25T10:05:38.079217+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081588+0000","last_change":"2024-04-25T10:05:38.080620+0000","last_active":"2024-04-25T10:05:38.081588+0000","last_peered":"2024-04-25T10:05:38.081588+0000","last_clean":"2024-04-25T10:05:38.081588+0000","last_became_active":"2024-04-25T10:05:38.075233+0000","last_became_peered":"2024-04-25T10:05:38.075233+0000","last_unstale":"2024-04-25T10:05:38.081588+0000","last_undegraded":"2024-04-25T10:05:38.081588+0000","last_fullsized":"2024-04-25T10:05:38.081588+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:49.754709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073656+0000","last_change":"2024-04-25T10:05:38.073656+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073656+0000","last_undegraded":"2024-04-25T10:05:38.073656+0000","last_fullsized":"2024-04-25T10:05:38.073656+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.075711+0000","last_change":"2024-04-25T10:05:38.074918+0000","last_active":"2024-04-25T10:05:38.075711+0000","last_peered":"2024-04-25T10:05:38.075711+0000","last_clean":"2024-04-25T10:05:38.075711+0000","last_became_active":"2024-04-25T10:05:38.074613+0000","last_became_peered":"2024-04-25T10:05:38.074613+0000","last_unstale":"2024-04-25T10:05:38.075711+0000","last_undegraded":"2024-04-25T10:05:38.075711+0000","last_fullsized":"2024-04-25T10:05:38.075711+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:54.282381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074188+0000","last_change":"2024-04-25T10:05:38.074188+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074188+0000","last_undegraded":"2024-04-25T10:05:38.074188+0000","last_fullsized":"2024-04-25T10:05:38.074188+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084181+0000","last_change":"2024-04-25T10:05:38.082985+0000","last_active":"2024-04-25T10:05:38.084181+0000","last_peered":"2024-04-25T10:05:38.084181+0000","last_clean":"2024-04-25T10:05:38.084181+0000","last_became_active":"2024-04-25T10:05:38.082719+0000","last_became_peered":"2024-04-25T10:05:38.082719+0000","last_unstale":"2024-04-25T10:05:38.084181+0000","last_undegraded":"2024-04-25T10:05:38.084181+0000","last_fullsized":"2024-04-25T10:05:38.084181+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:40.200742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.079561+0000","last_change":"2024-04-25T10:05:38.079561+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.079561+0000","last_undegraded":"2024-04-25T10:05:38.079561+0000","last_fullsized":"2024-04-25T10:05:38.079561+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076035+0000","last_change":"2024-04-25T10:05:38.075322+0000","last_active":"2024-04-25T10:05:38.076035+0000","last_peered":"2024-04-25T10:05:38.076035+0000","last_clean":"2024-04-25T10:05:38.076035+0000","last_became_active":"2024-04-25T10:05:38.074688+0000","last_became_peered":"2024-04-25T10:05:38.074688+0000","last_unstale":"2024-04-25T10:05:38.076035+0000","last_undegraded":"2024-04-25T10:05:38.076035+0000","last_fullsized":"2024-04-25T10:05:38.076035+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:10.878156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.068047+0000","last_change":"2024-04-25T10:05:38.068047+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.068047+0000","last_undegraded":"2024-04-25T10:05:38.068047+0000","last_fullsized":"2024-04-25T10:05:38.068047+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081990+0000","last_change":"2024-04-25T10:05:38.080464+0000","last_active":"2024-04-25T10:05:38.081990+0000","last_peered":"2024-04-25T10:05:38.081990+0000","last_clean":"2024-04-25T10:05:38.081990+0000","last_became_active":"2024-04-25T10:05:38.080184+0000","last_became_peered":"2024-04-25T10:05:38.080184+0000","last_unstale":"2024-04-25T10:05:38.081990+0000","last_undegraded":"2024-04-25T10:05:38.081990+0000","last_fullsized":"2024-04-25T10:05:38.081990+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:06.519993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074213+0000","last_change":"2024-04-25T10:05:38.074213+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074213+0000","last_undegraded":"2024-04-25T10:05:38.074213+0000","last_fullsized":"2024-04-25T10:05:38.074213+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.087722+0000","last_change":"2024-04-25T10:05:38.087415+0000","last_active":"2024-04-25T10:05:38.087722+0000","last_peered":"2024-04-25T10:05:38.087722+0000","last_clean":"2024-04-25T10:05:38.087722+0000","last_became_active":"2024-04-25T10:05:38.087167+0000","last_became_peered":"2024-04-25T10:05:38.087167+0000","last_unstale":"2024-04-25T10:05:38.087722+0000","last_undegraded":"2024-04-25T10:05:38.087722+0000","last_fullsized":"2024-04-25T10:05:38.087722+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:30.872314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073003+0000","last_change":"2024-04-25T10:05:38.073003+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073003+0000","last_undegraded":"2024-04-25T10:05:38.073003+0000","last_fullsized":"2024-04-25T10:05:38.073003+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084546+0000","last_change":"2024-04-25T10:05:38.081783+0000","last_active":"2024-04-25T10:05:38.084546+0000","last_peered":"2024-04-25T10:05:38.084546+0000","last_clean":"2024-04-25T10:05:38.084546+0000","last_became_active":"2024-04-25T10:05:38.081463+0000","last_became_peered":"2024-04-25T10:05:38.081463+0000","last_unstale":"2024-04-25T10:05:38.084546+0000","last_undegraded":"2024-04-25T10:05:38.084546+0000","last_fullsized":"2024-04-25T10:05:38.084546+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:10.390750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077577+0000","last_change":"2024-04-25T10:05:38.077577+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077577+0000","last_undegraded":"2024-04-25T10:05:38.077577+0000","last_fullsized":"2024-04-25T10:05:38.077577+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.077536+0000","last_change":"2024-04-25T10:05:38.076940+0000","last_active":"2024-04-25T10:05:38.077536+0000","last_peered":"2024-04-25T10:05:38.077536+0000","last_clean":"2024-04-25T10:05:38.077536+0000","last_became_active":"2024-04-25T10:05:38.076639+0000","last_became_peered":"2024-04-25T10:05:38.076639+0000","last_unstale":"2024-04-25T10:05:38.077536+0000","last_undegraded":"2024-04-25T10:05:38.077536+0000","last_fullsized":"2024-04-25T10:05:38.077536+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:20.840458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073109+0000","last_change":"2024-04-25T10:05:38.073109+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073109+0000","last_undegraded":"2024-04-25T10:05:38.073109+0000","last_fullsized":"2024-04-25T10:05:38.073109+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085058+0000","last_change":"2024-04-25T10:05:38.083896+0000","last_active":"2024-04-25T10:05:38.085058+0000","last_peered":"2024-04-25T10:05:38.085058+0000","last_clean":"2024-04-25T10:05:38.085058+0000","last_became_active":"2024-04-25T10:05:38.083309+0000","last_became_peered":"2024-04-25T10:05:38.083309+0000","last_unstale":"2024-04-25T10:05:38.085058+0000","last_undegraded":"2024-04-25T10:05:38.085058+0000","last_fullsized":"2024-04-25T10:05:38.085058+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:27.679738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074786+0000","last_change":"2024-04-25T10:05:38.074786+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074786+0000","last_undegraded":"2024-04-25T10:05:38.074786+0000","last_fullsized":"2024-04-25T10:05:38.074786+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084860+0000","last_change":"2024-04-25T10:05:38.083645+0000","last_active":"2024-04-25T10:05:38.084860+0000","last_peered":"2024-04-25T10:05:38.084860+0000","last_clean":"2024-04-25T10:05:38.084860+0000","last_became_active":"2024-04-25T10:05:38.083357+0000","last_became_peered":"2024-04-25T10:05:38.083357+0000","last_unstale":"2024-04-25T10:05:38.084860+0000","last_undegraded":"2024-04-25T10:05:38.084860+0000","last_fullsized":"2024-04-25T10:05:38.084860+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:32.268641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.067142+0000","last_change":"2024-04-25T10:05:38.067142+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.067142+0000","last_undegraded":"2024-04-25T10:05:38.067142+0000","last_fullsized":"2024-04-25T10:05:38.067142+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085150+0000","last_change":"2024-04-25T10:05:38.083600+0000","last_active":"2024-04-25T10:05:38.085150+0000","last_peered":"2024-04-25T10:05:38.085150+0000","last_clean":"2024-04-25T10:05:38.085150+0000","last_became_active":"2024-04-25T10:05:38.083303+0000","last_became_peered":"2024-04-25T10:05:38.083303+0000","last_unstale":"2024-04-25T10:05:38.085150+0000","last_undegraded":"2024-04-25T10:05:38.085150+0000","last_fullsized":"2024-04-25T10:05:38.085150+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:05.866090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078449+0000","last_change":"2024-04-25T10:05:38.078449+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078449+0000","last_undegraded":"2024-04-25T10:05:38.078449+0000","last_fullsized":"2024-04-25T10:05:38.078449+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084965+0000","last_change":"2024-04-25T10:05:38.081964+0000","last_active":"2024-04-25T10:05:38.084965+0000","last_peered":"2024-04-25T10:05:38.084965+0000","last_clean":"2024-04-25T10:05:38.084965+0000","last_became_active":"2024-04-25T10:05:38.081673+0000","last_became_peered":"2024-04-25T10:05:38.081673+0000","last_unstale":"2024-04-25T10:05:38.084965+0000","last_undegraded":"2024-04-25T10:05:38.084965+0000","last_fullsized":"2024-04-25T10:05:38.084965+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.599553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.078013+0000","last_change":"2024-04-25T10:05:38.078013+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.078013+0000","last_undegraded":"2024-04-25T10:05:38.078013+0000","last_fullsized":"2024-04-25T10:05:38.078013+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085904+0000","last_change":"2024-04-25T10:05:38.085345+0000","last_active":"2024-04-25T10:05:38.085904+0000","last_peered":"2024-04-25T10:05:38.085904+0000","last_clean":"2024-04-25T10:05:38.085904+0000","last_became_active":"2024-04-25T10:05:38.082298+0000","last_became_peered":"2024-04-25T10:05:38.082298+0000","last_unstale":"2024-04-25T10:05:38.085904+0000","last_undegraded":"2024-04-25T10:05:38.085904+0000","last_fullsized":"2024-04-25T10:05:38.085904+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:46.424209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074518+0000","last_change":"2024-04-25T10:05:38.074518+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074518+0000","last_undegraded":"2024-04-25T10:05:38.074518+0000","last_fullsized":"2024-04-25T10:05:38.074518+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084654+0000","last_change":"2024-04-25T10:05:38.083542+0000","last_active":"2024-04-25T10:05:38.084654+0000","last_peered":"2024-04-25T10:05:38.084654+0000","last_clean":"2024-04-25T10:05:38.084654+0000","last_became_active":"2024-04-25T10:05:38.082652+0000","last_became_peered":"2024-04-25T10:05:38.082652+0000","last_unstale":"2024-04-25T10:05:38.084654+0000","last_undegraded":"2024-04-25T10:05:38.084654+0000","last_fullsized":"2024-04-25T10:05:38.084654+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:36.918168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.065904+0000","last_change":"2024-04-25T10:05:38.065904+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.065904+0000","last_undegraded":"2024-04-25T10:05:38.065904+0000","last_fullsized":"2024-04-25T10:05:38.065904+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.080166+0000","last_change":"2024-04-25T10:05:38.077397+0000","last_active":"2024-04-25T10:05:38.080166+0000","last_peered":"2024-04-25T10:05:38.080166+0000","last_clean":"2024-04-25T10:05:38.080166+0000","last_became_active":"2024-04-25T10:05:38.077107+0000","last_became_peered":"2024-04-25T10:05:38.077107+0000","last_unstale":"2024-04-25T10:05:38.080166+0000","last_undegraded":"2024-04-25T10:05:38.080166+0000","last_fullsized":"2024-04-25T10:05:38.080166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:43.802225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.073892+0000","last_change":"2024-04-25T10:05:38.073892+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.073892+0000","last_undegraded":"2024-04-25T10:05:38.073892+0000","last_fullsized":"2024-04-25T10:05:38.073892+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.079630+0000","last_change":"2024-04-25T10:05:38.075167+0000","last_active":"2024-04-25T10:05:38.079630+0000","last_peered":"2024-04-25T10:05:38.079630+0000","last_clean":"2024-04-25T10:05:38.079630+0000","last_became_active":"2024-04-25T10:05:38.074890+0000","last_became_peered":"2024-04-25T10:05:38.074890+0000","last_unstale":"2024-04-25T10:05:38.079630+0000","last_undegraded":"2024-04-25T10:05:38.079630+0000","last_fullsized":"2024-04-25T10:05:38.079630+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:00.798323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.076641+0000","last_change":"2024-04-25T10:05:38.076641+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.076641+0000","last_undegraded":"2024-04-25T10:05:38.076641+0000","last_fullsized":"2024-04-25T10:05:38.076641+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084195+0000","last_change":"2024-04-25T10:05:38.082353+0000","last_active":"2024-04-25T10:05:38.084195+0000","last_peered":"2024-04-25T10:05:38.084195+0000","last_clean":"2024-04-25T10:05:38.084195+0000","last_became_active":"2024-04-25T10:05:38.082047+0000","last_became_peered":"2024-04-25T10:05:38.082047+0000","last_unstale":"2024-04-25T10:05:38.084195+0000","last_undegraded":"2024-04-25T10:05:38.084195+0000","last_fullsized":"2024-04-25T10:05:38.084195+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:12.478664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.074243+0000","last_change":"2024-04-25T10:05:38.074243+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.074243+0000","last_undegraded":"2024-04-25T10:05:38.074243+0000","last_fullsized":"2024-04-25T10:05:38.074243+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085023+0000","last_change":"2024-04-25T10:05:38.083029+0000","last_active":"2024-04-25T10:05:38.085023+0000","last_peered":"2024-04-25T10:05:38.085023+0000","last_clean":"2024-04-25T10:05:38.085023+0000","last_became_active":"2024-04-25T10:05:38.082535+0000","last_became_peered":"2024-04-25T10:05:38.082535+0000","last_unstale":"2024-04-25T10:05:38.085023+0000","last_undegraded":"2024-04-25T10:05:38.085023+0000","last_fullsized":"2024-04-25T10:05:38.085023+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:42.531775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.066667+0000","last_change":"2024-04-25T10:05:38.066667+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.066667+0000","last_undegraded":"2024-04-25T10:05:38.066667+0000","last_fullsized":"2024-04-25T10:05:38.066667+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076016+0000","last_change":"2024-04-25T10:05:38.073711+0000","last_active":"2024-04-25T10:05:38.076016+0000","last_peered":"2024-04-25T10:05:38.076016+0000","last_clean":"2024-04-25T10:05:38.076016+0000","last_became_active":"2024-04-25T10:05:38.073414+0000","last_became_peered":"2024-04-25T10:05:38.073414+0000","last_unstale":"2024-04-25T10:05:38.076016+0000","last_undegraded":"2024-04-25T10:05:38.076016+0000","last_fullsized":"2024-04-25T10:05:38.076016+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:10.878156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.071187+0000","last_change":"2024-04-25T10:05:38.071187+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.071187+0000","last_undegraded":"2024-04-25T10:05:38.071187+0000","last_fullsized":"2024-04-25T10:05:38.071187+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.073860+0000","last_change":"2024-04-25T10:05:38.071762+0000","last_active":"2024-04-25T10:05:38.073860+0000","last_peered":"2024-04-25T10:05:38.073860+0000","last_clean":"2024-04-25T10:05:38.073860+0000","last_became_active":"2024-04-25T10:05:38.071429+0000","last_became_peered":"2024-04-25T10:05:38.071429+0000","last_unstale":"2024-04-25T10:05:38.073860+0000","last_undegraded":"2024-04-25T10:05:38.073860+0000","last_fullsized":"2024-04-25T10:05:38.073860+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:54.282381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077423+0000","last_change":"2024-04-25T10:05:38.077423+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077423+0000","last_undegraded":"2024-04-25T10:05:38.077423+0000","last_fullsized":"2024-04-25T10:05:38.077423+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.085218+0000","last_change":"2024-04-25T10:05:38.084273+0000","last_active":"2024-04-25T10:05:38.085218+0000","last_peered":"2024-04-25T10:05:38.085218+0000","last_clean":"2024-04-25T10:05:38.085218+0000","last_became_active":"2024-04-25T10:05:38.082528+0000","last_became_peered":"2024-04-25T10:05:38.082528+0000","last_unstale":"2024-04-25T10:05:38.085218+0000","last_undegraded":"2024-04-25T10:05:38.085218+0000","last_fullsized":"2024-04-25T10:05:38.085218+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:29.382293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.072844+0000","last_change":"2024-04-25T10:05:38.072844+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.072844+0000","last_undegraded":"2024-04-25T10:05:38.072844+0000","last_fullsized":"2024-04-25T10:05:38.072844+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.087513+0000","last_change":"2024-04-25T10:05:38.086909+0000","last_active":"2024-04-25T10:05:38.087513+0000","last_peered":"2024-04-25T10:05:38.087513+0000","last_clean":"2024-04-25T10:05:38.087513+0000","last_became_active":"2024-04-25T10:05:38.086646+0000","last_became_peered":"2024-04-25T10:05:38.086646+0000","last_unstale":"2024-04-25T10:05:38.087513+0000","last_undegraded":"2024-04-25T10:05:38.087513+0000","last_fullsized":"2024-04-25T10:05:38.087513+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:08.234904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075592+0000","last_change":"2024-04-25T10:05:38.075592+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075592+0000","last_undegraded":"2024-04-25T10:05:38.075592+0000","last_fullsized":"2024-04-25T10:05:38.075592+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.081925+0000","last_change":"2024-04-25T10:05:38.081354+0000","last_active":"2024-04-25T10:05:38.081925+0000","last_peered":"2024-04-25T10:05:38.081925+0000","last_clean":"2024-04-25T10:05:38.081925+0000","last_became_active":"2024-04-25T10:05:38.081054+0000","last_became_peered":"2024-04-25T10:05:38.081054+0000","last_unstale":"2024-04-25T10:05:38.081925+0000","last_undegraded":"2024-04-25T10:05:38.081925+0000","last_fullsized":"2024-04-25T10:05:38.081925+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:04.866716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.075995+0000","last_change":"2024-04-25T10:05:38.075995+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.075995+0000","last_undegraded":"2024-04-25T10:05:38.075995+0000","last_fullsized":"2024-04-25T10:05:38.075995+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.076017+0000","last_change":"2024-04-25T10:05:38.075546+0000","last_active":"2024-04-25T10:05:38.076017+0000","last_peered":"2024-04-25T10:05:38.076017+0000","last_clean":"2024-04-25T10:05:38.076017+0000","last_became_active":"2024-04-25T10:05:38.074936+0000","last_became_peered":"2024-04-25T10:05:38.074936+0000","last_unstale":"2024-04-25T10:05:38.076017+0000","last_undegraded":"2024-04-25T10:05:38.076017+0000","last_fullsized":"2024-04-25T10:05:38.076017+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:00.798323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077978+0000","last_change":"2024-04-25T10:05:38.077978+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077978+0000","last_undegraded":"2024-04-25T10:05:38.077978+0000","last_fullsized":"2024-04-25T10:05:38.077978+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.084725+0000","last_change":"2024-04-25T10:05:38.083077+0000","last_active":"2024-04-25T10:05:38.084725+0000","last_peered":"2024-04-25T10:05:38.084725+0000","last_clean":"2024-04-25T10:05:38.084725+0000","last_became_active":"2024-04-25T10:05:38.082606+0000","last_became_peered":"2024-04-25T10:05:38.082606+0000","last_unstale":"2024-04-25T10:05:38.084725+0000","last_undegraded":"2024-04-25T10:05:38.084725+0000","last_fullsized":"2024-04-25T10:05:38.084725+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:57.974626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":1,"reported_epoch":187,"state":"creating+peering","last_fresh":"2024-04-25T10:05:38.077178+0000","last_change":"2024-04-25T10:05:38.077178+0000","last_active":"2024-04-25T10:05:38.052841+0000","last_peered":"2024-04-25T10:05:38.052841+0000","last_clean":"2024-04-25T10:05:38.052841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:38.077178+0000","last_undegraded":"2024-04-25T10:05:38.077178+0000","last_fullsized":"2024-04-25T10:05:38.077178+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.075802+0000","last_change":"2024-04-25T10:05:38.072474+0000","last_active":"2024-04-25T10:05:38.075802+0000","last_peered":"2024-04-25T10:05:38.075802+0000","last_clean":"2024-04-25T10:05:38.075802+0000","last_became_active":"2024-04-25T10:05:38.072160+0000","last_became_peered":"2024-04-25T10:05:38.072160+0000","last_unstale":"2024-04-25T10:05:38.075802+0000","last_undegraded":"2024-04-25T10:05:38.075802+0000","last_fullsized":"2024-04-25T10:05:38.075802+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:06.976864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":510,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.069800+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:05:38.069800+0000","last_peered":"2024-04-25T10:05:38.069800+0000","last_clean":"2024-04-25T10:05:38.069800+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:05:38.069800+0000","last_undegraded":"2024-04-25T10:05:38.069800+0000","last_fullsized":"2024-04-25T10:05:38.069800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56284,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315556,"statfs":{"total":96636764160,"available":96579129344,"internally_reserved":0,"allocated":2961408,"data_stored":1700777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51544,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320296,"statfs":{"total":96636764160,"available":96583983104,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51540,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320300,"statfs":{"total":96636764160,"available":96583987200,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2355200,"data_stored":1103742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59868,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311972,"statfs":{"total":96636764160,"available":96575459328,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55696,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55708,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316132,"statfs":{"total":96636764160,"available":96579719168,"internally_reserved":0,"allocated":2367488,"data_stored":1110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56284,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315556,"statfs":{"total":96636764160,"available":96579129344,"internally_reserved":0,"allocated":2961408,"data_stored":1700777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:45.044 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T10:05:45.039+0000 7fcd7c22b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:46.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.607+0000 7fa1d3838640 1 Processor -- start 2024-04-25T10:05:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.607+0000 7fa1d3838640 1 -- start start 2024-04-25T10:05:46.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d3838640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa1cc15dd30 0x7fa1cc0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d3838640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa1cc0d0ad0 0x7fa1cc0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d3838640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1cc0d1df0 con 0x7fa1cc0b9770 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d3838640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1cc0d20d0 con 0x7fa1cc15dd30 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d3838640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1cc0d23b0 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d15d4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa1cc15dd30 0x7fa1cc0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d0dd3640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa1cc0b9770 0x7fa1cc0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34912/0 (socket says 172.21.15.19:34912) 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d0dd3640 1 -- 172.21.15.19:0/2172574668 learned_addr learned my addr 172.21.15.19:0/2172574668 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d1dd5640 1 --2- 172.21.15.19:0/2172574668 >> v2:172.21.15.112:3300/0 conn(0x7fa1cc0d0ad0 0x7fa1cc0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d15d4640 1 -- 172.21.15.19:0/2172574668 >> v2:172.21.15.112:3300/0 conn(0x7fa1cc0d0ad0 msgr2=0x7fa1cc0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d15d4640 1 --2- 172.21.15.19:0/2172574668 >> v2:172.21.15.112:3300/0 conn(0x7fa1cc0d0ad0 0x7fa1cc0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d15d4640 1 -- 172.21.15.19:0/2172574668 >> v1:172.21.15.19:6789/0 conn(0x7fa1cc0b9770 legacy=0x7fa1cc0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1cbfff640 1 -- 172.21.15.19:0/2172574668 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698772843 0 0) 0x7fa1cc0d1df0 con 0x7fa1cc0b9770 2024-04-25T10:05:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.608+0000 7fa1d15d4640 1 -- 172.21.15.19:0/2172574668 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc0d2690 con 0x7fa1cc15dd30 2024-04-25T10:05:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.609+0000 7fa1d15d4640 1 --2- 172.21.15.19:0/2172574668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1cc15dd30 0x7fa1cc0b9230 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0ce200 tx=0x7fa1c402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec67c70170d90f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.609+0000 7fa1cbfff640 1 -- 172.21.15.19:0/2172574668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4037070 con 0x7fa1cc15dd30 2024-04-25T10:05:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.609+0000 7fa1cbfff640 1 -- 172.21.15.19:0/2172574668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c402add0 con 0x7fa1cc15dd30 2024-04-25T10:05:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.609+0000 7fa1cbfff640 1 -- 172.21.15.19:0/2172574668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4032660 con 0x7fa1cc15dd30 2024-04-25T10:05:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.610+0000 7fa1d3838640 1 -- 172.21.15.19:0/2172574668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1cc15dd30 msgr2=0x7fa1cc0b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.610+0000 7fa1d3838640 1 --2- 172.21.15.19:0/2172574668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1cc15dd30 0x7fa1cc0b9230 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0ce200 tx=0x7fa1c402a080 comp rx=0 tx=0).stop 2024-04-25T10:05:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.610+0000 7fa1d3838640 1 -- 172.21.15.19:0/2172574668 shutdown_connections 2024-04-25T10:05:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.610+0000 7fa1d3838640 1 --2- 172.21.15.19:0/2172574668 >> v2:172.21.15.112:3300/0 conn(0x7fa1cc0d0ad0 0x7fa1cc0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.610+0000 7fa1d3838640 1 --2- 172.21.15.19:0/2172574668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1cc15dd30 0x7fa1cc0b9230 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.610+0000 7fa1d3838640 1 -- 172.21.15.19:0/2172574668 >> 172.21.15.19:0/2172574668 conn(0x7fa1cc061c30 msgr2=0x7fa1cc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.610+0000 7fa1d3838640 1 -- 172.21.15.19:0/2172574668 shutdown_connections 2024-04-25T10:05:46.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.610+0000 7fa1d3838640 1 -- 172.21.15.19:0/2172574668 wait complete. 2024-04-25T10:05:46.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.611+0000 7fa1d3838640 1 Processor -- start 2024-04-25T10:05:46.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.611+0000 7fa1d3838640 1 -- start start 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d3838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1cc0b9770 0x7fa1cc0b8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d3838640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1cc0d0ad0 0x7fa1cc1f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d3838640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1cc15dd30 0x7fa1cc156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d3838640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0d1b90 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d3838640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0d1a10 con 0x7fa1cc0b9770 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d3838640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0d2e90 con 0x7fa1cc15dd30 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d0dd3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1cc0d0ad0 0x7fa1cc1f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d0dd3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1cc0d0ad0 0x7fa1cc1f7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35848/0 (socket says 172.21.15.19:35848) 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d0dd3640 1 -- 172.21.15.19:0/2919043131 learned_addr learned my addr 172.21.15.19:0/2919043131 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d15d4640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1cc0b9770 0x7fa1cc0b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d1dd5640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1cc15dd30 0x7fa1cc156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d0dd3640 1 -- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1cc15dd30 msgr2=0x7fa1cc156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d0dd3640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1cc15dd30 0x7fa1cc156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d0dd3640 1 -- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1cc0b9770 msgr2=0x7fa1cc0b8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d0dd3640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1cc0b9770 0x7fa1cc0b8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d0dd3640 1 -- 172.21.15.19:0/2919043131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc158440 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d15d4640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1cc0b9770 0x7fa1cc0b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d1dd5640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1cc15dd30 0x7fa1cc156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d0dd3640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1cc0d0ad0 0x7fa1cc1f7670 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000c4d0 tx=0x7fa1c000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1c9ffb640 1 -- 172.21.15.19:0/2919043131 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0007480 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1c9ffb640 1 -- 172.21.15.19:0/2919043131 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c00057a0 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1d3838640 1 -- 172.21.15.19:0/2919043131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1cc158720 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.612+0000 7fa1c9ffb640 1 -- 172.21.15.19:0/2919043131 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0007620 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.613+0000 7fa1d3838640 1 -- 172.21.15.19:0/2919043131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1cc158110 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.614+0000 7fa1d3838640 1 -- 172.21.15.19:0/2919043131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1cc073990 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.614+0000 7fa1c9ffb640 1 -- 172.21.15.19:0/2919043131 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1c0005b50 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.615+0000 7fa1c9ffb640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1b806b4d0 0x7fa1b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.615+0000 7fa1c9ffb640 1 -- 172.21.15.19:0/2919043131 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1c0089fe0 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.615+0000 7fa1d15d4640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1b806b4d0 0x7fa1b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.615+0000 7fa1d15d4640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1b806b4d0 0x7fa1b806d990 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4034000 tx=0x7fa1c402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.619+0000 7fa1c9ffb640 1 -- 172.21.15.19:0/2919043131 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1c00182b0 con 0x7fa1cc0d0ad0 2024-04-25T10:05:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.796+0000 7fa1d3838640 1 -- 172.21.15.19:0/2919043131 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1cc161310 con 0x7fa1b806b4d0 2024-04-25T10:05:46.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.802+0000 7fa1c9ffb640 1 -- 172.21.15.19:0/2919043131 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252489 (secure 0 0 0) 0x7fa1cc161310 con 0x7fa1b806b4d0 2024-04-25T10:05:46.810 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:46.811 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:05:46.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 -- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1b806b4d0 msgr2=0x7fa1b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1b806b4d0 0x7fa1b806d990 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4034000 tx=0x7fa1c402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:46.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 -- 172.21.15.19:0/2919043131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1cc0d0ad0 msgr2=0x7fa1cc1f7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1cc0d0ad0 0x7fa1cc1f7670 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000c4d0 tx=0x7fa1c000c990 comp rx=0 tx=0).stop 2024-04-25T10:05:46.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 -- 172.21.15.19:0/2919043131 shutdown_connections 2024-04-25T10:05:46.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa1b806b4d0 0x7fa1b806d990 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1cc15dd30 0x7fa1cc156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1cc0d0ad0 0x7fa1cc1f7670 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 --2- 172.21.15.19:0/2919043131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1cc0b9770 0x7fa1cc0b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 -- 172.21.15.19:0/2919043131 >> 172.21.15.19:0/2919043131 conn(0x7fa1cc061c30 msgr2=0x7fa1cc0d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 -- 172.21.15.19:0/2919043131 shutdown_connections 2024-04-25T10:05:46.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.805+0000 7fa1d3838640 1 -- 172.21.15.19:0/2919043131 wait complete. 2024-04-25T10:05:46.821 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":568,"stamp":"2024-04-25T10:05:45.089543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":447032,"kb_used_data":19912,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754527688,"statfs":{"total":773094113280,"available":772636352512,"internally_reserved":0,"allocated":20389888,"data_stored":10179700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":437243442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.025809"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.087211+0000","last_change":"2024-04-25T10:05:39.083990+0000","last_active":"2024-04-25T10:05:40.087211+0000","last_peered":"2024-04-25T10:05:40.087211+0000","last_clean":"2024-04-25T10:05:40.087211+0000","last_became_active":"2024-04-25T10:05:39.083717+0000","last_became_peered":"2024-04-25T10:05:39.083717+0000","last_unstale":"2024-04-25T10:05:40.087211+0000","last_undegraded":"2024-04-25T10:05:40.087211+0000","last_fullsized":"2024-04-25T10:05:40.087211+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:59.725163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488727+0000","last_change":"2024-04-25T10:05:39.084533+0000","last_active":"2024-04-25T10:05:40.488727+0000","last_peered":"2024-04-25T10:05:40.488727+0000","last_clean":"2024-04-25T10:05:40.488727+0000","last_became_active":"2024-04-25T10:05:39.084178+0000","last_became_peered":"2024-04-25T10:05:39.084178+0000","last_unstale":"2024-04-25T10:05:40.488727+0000","last_undegraded":"2024-04-25T10:05:40.488727+0000","last_fullsized":"2024-04-25T10:05:40.488727+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:34.691545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488625+0000","last_change":"2024-04-25T10:05:39.077629+0000","last_active":"2024-04-25T10:05:40.488625+0000","last_peered":"2024-04-25T10:05:40.488625+0000","last_clean":"2024-04-25T10:05:40.488625+0000","last_became_active":"2024-04-25T10:05:39.077336+0000","last_became_peered":"2024-04-25T10:05:39.077336+0000","last_unstale":"2024-04-25T10:05:40.488625+0000","last_undegraded":"2024-04-25T10:05:40.488625+0000","last_fullsized":"2024-04-25T10:05:40.488625+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:54.291770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.086545+0000","last_change":"2024-04-25T10:05:39.083971+0000","last_active":"2024-04-25T10:05:40.086545+0000","last_peered":"2024-04-25T10:05:40.086545+0000","last_clean":"2024-04-25T10:05:40.086545+0000","last_became_active":"2024-04-25T10:05:39.083691+0000","last_became_peered":"2024-04-25T10:05:39.083691+0000","last_unstale":"2024-04-25T10:05:40.086545+0000","last_undegraded":"2024-04-25T10:05:40.086545+0000","last_fullsized":"2024-04-25T10:05:40.086545+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:28.681723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.086896+0000","last_change":"2024-04-25T10:05:39.084639+0000","last_active":"2024-04-25T10:05:40.086896+0000","last_peered":"2024-04-25T10:05:40.086896+0000","last_clean":"2024-04-25T10:05:40.086896+0000","last_became_active":"2024-04-25T10:05:39.084377+0000","last_became_peered":"2024-04-25T10:05:39.084377+0000","last_unstale":"2024-04-25T10:05:40.086896+0000","last_undegraded":"2024-04-25T10:05:40.086896+0000","last_fullsized":"2024-04-25T10:05:40.086896+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:52.844876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488889+0000","last_change":"2024-04-25T10:05:39.081507+0000","last_active":"2024-04-25T10:05:40.488889+0000","last_peered":"2024-04-25T10:05:40.488889+0000","last_clean":"2024-04-25T10:05:40.488889+0000","last_became_active":"2024-04-25T10:05:39.081187+0000","last_became_peered":"2024-04-25T10:05:39.081187+0000","last_unstale":"2024-04-25T10:05:40.488889+0000","last_undegraded":"2024-04-25T10:05:40.488889+0000","last_fullsized":"2024-04-25T10:05:40.488889+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:05.899016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488313+0000","last_change":"2024-04-25T10:05:39.075980+0000","last_active":"2024-04-25T10:05:40.488313+0000","last_peered":"2024-04-25T10:05:40.488313+0000","last_clean":"2024-04-25T10:05:40.488313+0000","last_became_active":"2024-04-25T10:05:39.075711+0000","last_became_peered":"2024-04-25T10:05:39.075711+0000","last_unstale":"2024-04-25T10:05:40.488313+0000","last_undegraded":"2024-04-25T10:05:40.488313+0000","last_fullsized":"2024-04-25T10:05:40.488313+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:52.919626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.487938+0000","last_change":"2024-04-25T10:05:39.081410+0000","last_active":"2024-04-25T10:05:40.487938+0000","last_peered":"2024-04-25T10:05:40.487938+0000","last_clean":"2024-04-25T10:05:40.487938+0000","last_became_active":"2024-04-25T10:05:39.080912+0000","last_became_peered":"2024-04-25T10:05:39.080912+0000","last_unstale":"2024-04-25T10:05:40.487938+0000","last_undegraded":"2024-04-25T10:05:40.487938+0000","last_fullsized":"2024-04-25T10:05:40.487938+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:07.978849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.487846+0000","last_change":"2024-04-25T10:05:39.083998+0000","last_active":"2024-04-25T10:05:40.487846+0000","last_peered":"2024-04-25T10:05:40.487846+0000","last_clean":"2024-04-25T10:05:40.487846+0000","last_became_active":"2024-04-25T10:05:39.083693+0000","last_became_peered":"2024-04-25T10:05:39.083693+0000","last_unstale":"2024-04-25T10:05:40.487846+0000","last_undegraded":"2024-04-25T10:05:40.487846+0000","last_fullsized":"2024-04-25T10:05:40.487846+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:03.063780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488774+0000","last_change":"2024-04-25T10:05:39.080314+0000","last_active":"2024-04-25T10:05:40.488774+0000","last_peered":"2024-04-25T10:05:40.488774+0000","last_clean":"2024-04-25T10:05:40.488774+0000","last_became_active":"2024-04-25T10:05:39.080034+0000","last_became_peered":"2024-04-25T10:05:39.080034+0000","last_unstale":"2024-04-25T10:05:40.488774+0000","last_undegraded":"2024-04-25T10:05:40.488774+0000","last_fullsized":"2024-04-25T10:05:40.488774+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:06.995643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.087178+0000","last_change":"2024-04-25T10:05:39.082423+0000","last_active":"2024-04-25T10:05:40.087178+0000","last_peered":"2024-04-25T10:05:40.087178+0000","last_clean":"2024-04-25T10:05:40.087178+0000","last_became_active":"2024-04-25T10:05:39.082152+0000","last_became_peered":"2024-04-25T10:05:39.082152+0000","last_unstale":"2024-04-25T10:05:40.087178+0000","last_undegraded":"2024-04-25T10:05:40.087178+0000","last_fullsized":"2024-04-25T10:05:40.087178+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:47.426194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.489199+0000","last_change":"2024-04-25T10:05:39.099770+0000","last_active":"2024-04-25T10:05:40.489199+0000","last_peered":"2024-04-25T10:05:40.489199+0000","last_clean":"2024-04-25T10:05:40.489199+0000","last_became_active":"2024-04-25T10:05:39.099004+0000","last_became_peered":"2024-04-25T10:05:39.099004+0000","last_unstale":"2024-04-25T10:05:40.489199+0000","last_undegraded":"2024-04-25T10:05:40.489199+0000","last_fullsized":"2024-04-25T10:05:40.489199+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:11.880142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488471+0000","last_change":"2024-04-25T10:05:39.076944+0000","last_active":"2024-04-25T10:05:40.488471+0000","last_peered":"2024-04-25T10:05:40.488471+0000","last_clean":"2024-04-25T10:05:40.488471+0000","last_became_active":"2024-04-25T10:05:39.076674+0000","last_became_peered":"2024-04-25T10:05:39.076674+0000","last_unstale":"2024-04-25T10:05:40.488471+0000","last_undegraded":"2024-04-25T10:05:40.488471+0000","last_fullsized":"2024-04-25T10:05:40.488471+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:44.804210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488512+0000","last_change":"2024-04-25T10:05:39.093974+0000","last_active":"2024-04-25T10:05:40.488512+0000","last_peered":"2024-04-25T10:05:40.488512+0000","last_clean":"2024-04-25T10:05:40.488512+0000","last_became_active":"2024-04-25T10:05:39.093656+0000","last_became_peered":"2024-04-25T10:05:39.093656+0000","last_unstale":"2024-04-25T10:05:40.488512+0000","last_undegraded":"2024-04-25T10:05:40.488512+0000","last_fullsized":"2024-04-25T10:05:40.488512+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:50.067230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.083472+0000","last_change":"2024-04-25T10:05:39.079467+0000","last_active":"2024-04-25T10:05:40.083472+0000","last_peered":"2024-04-25T10:05:40.083472+0000","last_clean":"2024-04-25T10:05:40.083472+0000","last_became_active":"2024-04-25T10:05:39.079176+0000","last_became_peered":"2024-04-25T10:05:39.079176+0000","last_unstale":"2024-04-25T10:05:40.083472+0000","last_undegraded":"2024-04-25T10:05:40.083472+0000","last_fullsized":"2024-04-25T10:05:40.083472+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:44.876070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488044+0000","last_change":"2024-04-25T10:05:39.085160+0000","last_active":"2024-04-25T10:05:40.488044+0000","last_peered":"2024-04-25T10:05:40.488044+0000","last_clean":"2024-04-25T10:05:40.488044+0000","last_became_active":"2024-04-25T10:05:39.084870+0000","last_became_peered":"2024-04-25T10:05:39.084870+0000","last_unstale":"2024-04-25T10:05:40.488044+0000","last_undegraded":"2024-04-25T10:05:40.488044+0000","last_fullsized":"2024-04-25T10:05:40.488044+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:55.284366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488904+0000","last_change":"2024-04-25T10:05:39.091810+0000","last_active":"2024-04-25T10:05:40.488904+0000","last_peered":"2024-04-25T10:05:40.488904+0000","last_clean":"2024-04-25T10:05:40.488904+0000","last_became_active":"2024-04-25T10:05:39.091492+0000","last_became_peered":"2024-04-25T10:05:39.091492+0000","last_unstale":"2024-04-25T10:05:40.488904+0000","last_undegraded":"2024-04-25T10:05:40.488904+0000","last_fullsized":"2024-04-25T10:05:40.488904+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:07.978849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488861+0000","last_change":"2024-04-25T10:05:39.090584+0000","last_active":"2024-04-25T10:05:40.488861+0000","last_peered":"2024-04-25T10:05:40.488861+0000","last_clean":"2024-04-25T10:05:40.488861+0000","last_became_active":"2024-04-25T10:05:39.090114+0000","last_became_peered":"2024-04-25T10:05:39.090114+0000","last_unstale":"2024-04-25T10:05:40.488861+0000","last_undegraded":"2024-04-25T10:05:40.488861+0000","last_fullsized":"2024-04-25T10:05:40.488861+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:59.565614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.087579+0000","last_change":"2024-04-25T10:05:39.077698+0000","last_active":"2024-04-25T10:05:40.087579+0000","last_peered":"2024-04-25T10:05:40.087579+0000","last_clean":"2024-04-25T10:05:40.087579+0000","last_became_active":"2024-04-25T10:05:39.077408+0000","last_became_peered":"2024-04-25T10:05:39.077408+0000","last_unstale":"2024-04-25T10:05:40.087579+0000","last_undegraded":"2024-04-25T10:05:40.087579+0000","last_fullsized":"2024-04-25T10:05:40.087579+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:02.513237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488690+0000","last_change":"2024-04-25T10:05:39.084106+0000","last_active":"2024-04-25T10:05:40.488690+0000","last_peered":"2024-04-25T10:05:40.488690+0000","last_clean":"2024-04-25T10:05:40.488690+0000","last_became_active":"2024-04-25T10:05:39.083820+0000","last_became_peered":"2024-04-25T10:05:39.083820+0000","last_unstale":"2024-04-25T10:05:40.488690+0000","last_undegraded":"2024-04-25T10:05:40.488690+0000","last_fullsized":"2024-04-25T10:05:40.488690+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:08.075129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.487977+0000","last_change":"2024-04-25T10:05:39.075141+0000","last_active":"2024-04-25T10:05:40.487977+0000","last_peered":"2024-04-25T10:05:40.487977+0000","last_clean":"2024-04-25T10:05:40.487977+0000","last_became_active":"2024-04-25T10:05:39.074846+0000","last_became_peered":"2024-04-25T10:05:39.074846+0000","last_unstale":"2024-04-25T10:05:40.487977+0000","last_undegraded":"2024-04-25T10:05:40.487977+0000","last_fullsized":"2024-04-25T10:05:40.487977+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:02.200478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.083516+0000","last_change":"2024-04-25T10:05:39.079445+0000","last_active":"2024-04-25T10:05:40.083516+0000","last_peered":"2024-04-25T10:05:40.083516+0000","last_clean":"2024-04-25T10:05:40.083516+0000","last_became_active":"2024-04-25T10:05:39.079145+0000","last_became_peered":"2024-04-25T10:05:39.079145+0000","last_unstale":"2024-04-25T10:05:40.083516+0000","last_undegraded":"2024-04-25T10:05:40.083516+0000","last_fullsized":"2024-04-25T10:05:40.083516+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:34.080155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.487889+0000","last_change":"2024-04-25T10:05:39.086847+0000","last_active":"2024-04-25T10:05:40.487889+0000","last_peered":"2024-04-25T10:05:40.487889+0000","last_clean":"2024-04-25T10:05:40.487889+0000","last_became_active":"2024-04-25T10:05:39.083331+0000","last_became_peered":"2024-04-25T10:05:39.083331+0000","last_unstale":"2024-04-25T10:05:40.487889+0000","last_undegraded":"2024-04-25T10:05:40.487889+0000","last_fullsized":"2024-04-25T10:05:40.487889+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:42.625822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.086275+0000","last_change":"2024-04-25T10:05:39.080581+0000","last_active":"2024-04-25T10:05:40.086275+0000","last_peered":"2024-04-25T10:05:40.086275+0000","last_clean":"2024-04-25T10:05:40.086275+0000","last_became_active":"2024-04-25T10:05:39.080287+0000","last_became_peered":"2024-04-25T10:05:39.080287+0000","last_unstale":"2024-04-25T10:05:40.086275+0000","last_undegraded":"2024-04-25T10:05:40.086275+0000","last_fullsized":"2024-04-25T10:05:40.086275+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:44.876070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.087252+0000","last_change":"2024-04-25T10:05:39.081871+0000","last_active":"2024-04-25T10:05:40.087252+0000","last_peered":"2024-04-25T10:05:40.087252+0000","last_clean":"2024-04-25T10:05:40.087252+0000","last_became_active":"2024-04-25T10:05:39.081264+0000","last_became_peered":"2024-04-25T10:05:39.081264+0000","last_unstale":"2024-04-25T10:05:40.087252+0000","last_undegraded":"2024-04-25T10:05:40.087252+0000","last_fullsized":"2024-04-25T10:05:40.087252+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:54.522337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488962+0000","last_change":"2024-04-25T10:05:39.091308+0000","last_active":"2024-04-25T10:05:40.488962+0000","last_peered":"2024-04-25T10:05:40.488962+0000","last_clean":"2024-04-25T10:05:40.488962+0000","last_became_active":"2024-04-25T10:05:39.090466+0000","last_became_peered":"2024-04-25T10:05:39.090466+0000","last_unstale":"2024-04-25T10:05:40.488962+0000","last_undegraded":"2024-04-25T10:05:40.488962+0000","last_fullsized":"2024-04-25T10:05:40.488962+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:49.168634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.087426+0000","last_change":"2024-04-25T10:05:39.084627+0000","last_active":"2024-04-25T10:05:40.087426+0000","last_peered":"2024-04-25T10:05:40.087426+0000","last_clean":"2024-04-25T10:05:40.087426+0000","last_became_active":"2024-04-25T10:05:39.084361+0000","last_became_peered":"2024-04-25T10:05:39.084361+0000","last_unstale":"2024-04-25T10:05:40.087426+0000","last_undegraded":"2024-04-25T10:05:40.087426+0000","last_fullsized":"2024-04-25T10:05:40.087426+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:09.236890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.489186+0000","last_change":"2024-04-25T10:05:39.099535+0000","last_active":"2024-04-25T10:05:40.489186+0000","last_peered":"2024-04-25T10:05:40.489186+0000","last_clean":"2024-04-25T10:05:40.489186+0000","last_became_active":"2024-04-25T10:05:39.098493+0000","last_became_peered":"2024-04-25T10:05:39.098493+0000","last_unstale":"2024-04-25T10:05:40.489186+0000","last_undegraded":"2024-04-25T10:05:40.489186+0000","last_fullsized":"2024-04-25T10:05:40.489186+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:55.284366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488068+0000","last_change":"2024-04-25T10:05:39.083297+0000","last_active":"2024-04-25T10:05:40.488068+0000","last_peered":"2024-04-25T10:05:40.488068+0000","last_clean":"2024-04-25T10:05:40.488068+0000","last_became_active":"2024-04-25T10:05:39.082950+0000","last_became_peered":"2024-04-25T10:05:39.082950+0000","last_unstale":"2024-04-25T10:05:40.488068+0000","last_undegraded":"2024-04-25T10:05:40.488068+0000","last_fullsized":"2024-04-25T10:05:40.488068+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:17.885596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488116+0000","last_change":"2024-04-25T10:05:39.095864+0000","last_active":"2024-04-25T10:05:40.488116+0000","last_peered":"2024-04-25T10:05:40.488116+0000","last_clean":"2024-04-25T10:05:40.488116+0000","last_became_active":"2024-04-25T10:05:39.095587+0000","last_became_peered":"2024-04-25T10:05:39.095587+0000","last_unstale":"2024-04-25T10:05:40.488116+0000","last_undegraded":"2024-04-25T10:05:40.488116+0000","last_fullsized":"2024-04-25T10:05:40.488116+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:00.194728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488038+0000","last_change":"2024-04-25T10:05:39.086772+0000","last_active":"2024-04-25T10:05:40.488038+0000","last_peered":"2024-04-25T10:05:40.488038+0000","last_clean":"2024-04-25T10:05:40.488038+0000","last_became_active":"2024-04-25T10:05:39.082964+0000","last_became_peered":"2024-04-25T10:05:39.082964+0000","last_unstale":"2024-04-25T10:05:40.488038+0000","last_undegraded":"2024-04-25T10:05:40.488038+0000","last_fullsized":"2024-04-25T10:05:40.488038+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:09.627823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488166+0000","last_change":"2024-04-25T10:05:39.092580+0000","last_active":"2024-04-25T10:05:40.488166+0000","last_peered":"2024-04-25T10:05:40.488166+0000","last_clean":"2024-04-25T10:05:40.488166+0000","last_became_active":"2024-04-25T10:05:39.092285+0000","last_became_peered":"2024-04-25T10:05:39.092285+0000","last_unstale":"2024-04-25T10:05:40.488166+0000","last_undegraded":"2024-04-25T10:05:40.488166+0000","last_fullsized":"2024-04-25T10:05:40.488166+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:11.880142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488673+0000","last_change":"2024-04-25T10:05:39.084842+0000","last_active":"2024-04-25T10:05:40.488673+0000","last_peered":"2024-04-25T10:05:40.488673+0000","last_clean":"2024-04-25T10:05:40.488673+0000","last_became_active":"2024-04-25T10:05:39.084558+0000","last_became_peered":"2024-04-25T10:05:39.084558+0000","last_unstale":"2024-04-25T10:05:40.488673+0000","last_undegraded":"2024-04-25T10:05:40.488673+0000","last_fullsized":"2024-04-25T10:05:40.488673+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:04.347829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"188'4","reported_seq":18,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.487938+0000","last_change":"2024-04-25T10:05:38.085541+0000","last_active":"2024-04-25T10:05:40.487938+0000","last_peered":"2024-04-25T10:05:40.487938+0000","last_clean":"2024-04-25T10:05:40.487938+0000","last_became_active":"2024-04-25T10:05:38.085242+0000","last_became_peered":"2024-04-25T10:05:38.085242+0000","last_unstale":"2024-04-25T10:05:40.487938+0000","last_undegraded":"2024-04-25T10:05:40.487938+0000","last_fullsized":"2024-04-25T10:05:40.487938+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:08.617330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488773+0000","last_change":"2024-04-25T10:05:39.085670+0000","last_active":"2024-04-25T10:05:40.488773+0000","last_peered":"2024-04-25T10:05:40.488773+0000","last_clean":"2024-04-25T10:05:40.488773+0000","last_became_active":"2024-04-25T10:05:39.085398+0000","last_became_peered":"2024-04-25T10:05:39.085398+0000","last_unstale":"2024-04-25T10:05:40.488773+0000","last_undegraded":"2024-04-25T10:05:40.488773+0000","last_fullsized":"2024-04-25T10:05:40.488773+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:31.975901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"188'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.489196+0000","last_change":"2024-04-25T10:05:38.081472+0000","last_active":"2024-04-25T10:05:40.489196+0000","last_peered":"2024-04-25T10:05:40.489196+0000","last_clean":"2024-04-25T10:05:40.489196+0000","last_became_active":"2024-04-25T10:05:38.081187+0000","last_became_peered":"2024-04-25T10:05:38.081187+0000","last_unstale":"2024-04-25T10:05:40.489196+0000","last_undegraded":"2024-04-25T10:05:40.489196+0000","last_fullsized":"2024-04-25T10:05:40.489196+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:23.834536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.489022+0000","last_change":"2024-04-25T10:05:39.089569+0000","last_active":"2024-04-25T10:05:40.489022+0000","last_peered":"2024-04-25T10:05:40.489022+0000","last_clean":"2024-04-25T10:05:40.489022+0000","last_became_active":"2024-04-25T10:05:39.089275+0000","last_became_peered":"2024-04-25T10:05:39.089275+0000","last_unstale":"2024-04-25T10:05:40.489022+0000","last_undegraded":"2024-04-25T10:05:40.489022+0000","last_fullsized":"2024-04-25T10:05:40.489022+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:40.696113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"188'2","reported_seq":16,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488161+0000","last_change":"2024-04-25T10:05:38.079620+0000","last_active":"2024-04-25T10:05:40.488161+0000","last_peered":"2024-04-25T10:05:40.488161+0000","last_clean":"2024-04-25T10:05:40.488161+0000","last_became_active":"2024-04-25T10:05:38.079361+0000","last_became_peered":"2024-04-25T10:05:38.079361+0000","last_unstale":"2024-04-25T10:05:40.488161+0000","last_undegraded":"2024-04-25T10:05:40.488161+0000","last_fullsized":"2024-04-25T10:05:40.488161+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:09.669880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.487821+0000","last_change":"2024-04-25T10:05:39.079693+0000","last_active":"2024-04-25T10:05:40.487821+0000","last_peered":"2024-04-25T10:05:40.487821+0000","last_clean":"2024-04-25T10:05:40.487821+0000","last_became_active":"2024-04-25T10:05:39.079406+0000","last_became_peered":"2024-04-25T10:05:39.079406+0000","last_unstale":"2024-04-25T10:05:40.487821+0000","last_undegraded":"2024-04-25T10:05:40.487821+0000","last_fullsized":"2024-04-25T10:05:40.487821+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:17.885596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"188'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.083751+0000","last_change":"2024-04-25T10:05:38.083940+0000","last_active":"2024-04-25T10:05:40.083751+0000","last_peered":"2024-04-25T10:05:40.083751+0000","last_clean":"2024-04-25T10:05:40.083751+0000","last_became_active":"2024-04-25T10:05:38.083619+0000","last_became_peered":"2024-04-25T10:05:38.083619+0000","last_unstale":"2024-04-25T10:05:40.083751+0000","last_undegraded":"2024-04-25T10:05:40.083751+0000","last_fullsized":"2024-04-25T10:05:40.083751+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:14.753650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488248+0000","last_change":"2024-04-25T10:05:39.098221+0000","last_active":"2024-04-25T10:05:40.488248+0000","last_peered":"2024-04-25T10:05:40.488248+0000","last_clean":"2024-04-25T10:05:40.488248+0000","last_became_active":"2024-04-25T10:05:39.097918+0000","last_became_peered":"2024-04-25T10:05:39.097918+0000","last_unstale":"2024-04-25T10:05:40.488248+0000","last_undegraded":"2024-04-25T10:05:40.488248+0000","last_fullsized":"2024-04-25T10:05:40.488248+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:44.586311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"188'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.086789+0000","last_change":"2024-04-25T10:05:38.079975+0000","last_active":"2024-04-25T10:05:40.086789+0000","last_peered":"2024-04-25T10:05:40.086789+0000","last_clean":"2024-04-25T10:05:40.086789+0000","last_became_active":"2024-04-25T10:05:38.079671+0000","last_became_peered":"2024-04-25T10:05:38.079671+0000","last_unstale":"2024-04-25T10:05:40.086789+0000","last_undegraded":"2024-04-25T10:05:40.086789+0000","last_fullsized":"2024-04-25T10:05:40.086789+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:43.833994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.083518+0000","last_change":"2024-04-25T10:05:39.080081+0000","last_active":"2024-04-25T10:05:40.083518+0000","last_peered":"2024-04-25T10:05:40.083518+0000","last_clean":"2024-04-25T10:05:40.083518+0000","last_became_active":"2024-04-25T10:05:39.079796+0000","last_became_peered":"2024-04-25T10:05:39.079796+0000","last_unstale":"2024-04-25T10:05:40.083518+0000","last_undegraded":"2024-04-25T10:05:40.083518+0000","last_fullsized":"2024-04-25T10:05:40.083518+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:52.844876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.086545+0000","last_change":"2024-04-25T10:05:38.082341+0000","last_active":"2024-04-25T10:05:40.086545+0000","last_peered":"2024-04-25T10:05:40.086545+0000","last_clean":"2024-04-25T10:05:40.086545+0000","last_became_active":"2024-04-25T10:05:38.082061+0000","last_became_peered":"2024-04-25T10:05:38.082061+0000","last_unstale":"2024-04-25T10:05:40.086545+0000","last_undegraded":"2024-04-25T10:05:40.086545+0000","last_fullsized":"2024-04-25T10:05:40.086545+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:10.965321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488483+0000","last_change":"2024-04-25T10:05:39.085333+0000","last_active":"2024-04-25T10:05:40.488483+0000","last_peered":"2024-04-25T10:05:40.488483+0000","last_clean":"2024-04-25T10:05:40.488483+0000","last_became_active":"2024-04-25T10:05:39.085005+0000","last_became_peered":"2024-04-25T10:05:39.085005+0000","last_unstale":"2024-04-25T10:05:40.488483+0000","last_undegraded":"2024-04-25T10:05:40.488483+0000","last_fullsized":"2024-04-25T10:05:40.488483+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:24.299672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488070+0000","last_change":"2024-04-25T10:05:38.080620+0000","last_active":"2024-04-25T10:05:40.488070+0000","last_peered":"2024-04-25T10:05:40.488070+0000","last_clean":"2024-04-25T10:05:40.488070+0000","last_became_active":"2024-04-25T10:05:38.075233+0000","last_became_peered":"2024-04-25T10:05:38.075233+0000","last_unstale":"2024-04-25T10:05:40.488070+0000","last_undegraded":"2024-04-25T10:05:40.488070+0000","last_fullsized":"2024-04-25T10:05:40.488070+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:19.223866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.083696+0000","last_change":"2024-04-25T10:05:39.086048+0000","last_active":"2024-04-25T10:05:40.083696+0000","last_peered":"2024-04-25T10:05:40.083696+0000","last_clean":"2024-04-25T10:05:40.083696+0000","last_became_active":"2024-04-25T10:05:39.085528+0000","last_became_peered":"2024-04-25T10:05:39.085528+0000","last_unstale":"2024-04-25T10:05:40.083696+0000","last_undegraded":"2024-04-25T10:05:40.083696+0000","last_fullsized":"2024-04-25T10:05:40.083696+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:59.725163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.087332+0000","last_change":"2024-04-25T10:05:38.074918+0000","last_active":"2024-04-25T10:05:40.087332+0000","last_peered":"2024-04-25T10:05:40.087332+0000","last_clean":"2024-04-25T10:05:40.087332+0000","last_became_active":"2024-04-25T10:05:38.074613+0000","last_became_peered":"2024-04-25T10:05:38.074613+0000","last_unstale":"2024-04-25T10:05:40.087332+0000","last_undegraded":"2024-04-25T10:05:40.087332+0000","last_fullsized":"2024-04-25T10:05:40.087332+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:04.897031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.083855+0000","last_change":"2024-04-25T10:05:39.084834+0000","last_active":"2024-04-25T10:05:40.083855+0000","last_peered":"2024-04-25T10:05:40.083855+0000","last_clean":"2024-04-25T10:05:40.083855+0000","last_became_active":"2024-04-25T10:05:39.084493+0000","last_became_peered":"2024-04-25T10:05:39.084493+0000","last_unstale":"2024-04-25T10:05:40.083855+0000","last_undegraded":"2024-04-25T10:05:40.083855+0000","last_fullsized":"2024-04-25T10:05:40.083855+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:59.378346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.086733+0000","last_change":"2024-04-25T10:05:38.082985+0000","last_active":"2024-04-25T10:05:40.086733+0000","last_peered":"2024-04-25T10:05:40.086733+0000","last_clean":"2024-04-25T10:05:40.086733+0000","last_became_active":"2024-04-25T10:05:38.082719+0000","last_became_peered":"2024-04-25T10:05:38.082719+0000","last_unstale":"2024-04-25T10:05:40.086733+0000","last_undegraded":"2024-04-25T10:05:40.086733+0000","last_fullsized":"2024-04-25T10:05:40.086733+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:57.188739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488960+0000","last_change":"2024-04-25T10:05:39.086470+0000","last_active":"2024-04-25T10:05:40.488960+0000","last_peered":"2024-04-25T10:05:40.488960+0000","last_clean":"2024-04-25T10:05:40.488960+0000","last_became_active":"2024-04-25T10:05:39.086063+0000","last_became_peered":"2024-04-25T10:05:39.086063+0000","last_unstale":"2024-04-25T10:05:40.488960+0000","last_undegraded":"2024-04-25T10:05:40.488960+0000","last_fullsized":"2024-04-25T10:05:40.488960+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:27.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"188'3","reported_seq":17,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.087633+0000","last_change":"2024-04-25T10:05:38.075322+0000","last_active":"2024-04-25T10:05:40.087633+0000","last_peered":"2024-04-25T10:05:40.087633+0000","last_clean":"2024-04-25T10:05:40.087633+0000","last_became_active":"2024-04-25T10:05:38.074688+0000","last_became_peered":"2024-04-25T10:05:38.074688+0000","last_unstale":"2024-04-25T10:05:40.087633+0000","last_undegraded":"2024-04-25T10:05:40.087633+0000","last_fullsized":"2024-04-25T10:05:40.087633+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:27.770003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.086941+0000","last_change":"2024-04-25T10:05:39.083213+0000","last_active":"2024-04-25T10:05:40.086941+0000","last_peered":"2024-04-25T10:05:40.086941+0000","last_clean":"2024-04-25T10:05:40.086941+0000","last_became_active":"2024-04-25T10:05:39.082872+0000","last_became_peered":"2024-04-25T10:05:39.082872+0000","last_unstale":"2024-04-25T10:05:40.086941+0000","last_undegraded":"2024-04-25T10:05:40.086941+0000","last_fullsized":"2024-04-25T10:05:40.086941+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:30.384278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"188'6","reported_seq":20,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488675+0000","last_change":"2024-04-25T10:05:38.080464+0000","last_active":"2024-04-25T10:05:40.488675+0000","last_peered":"2024-04-25T10:05:40.488675+0000","last_clean":"2024-04-25T10:05:40.488675+0000","last_became_active":"2024-04-25T10:05:38.080184+0000","last_became_peered":"2024-04-25T10:05:38.080184+0000","last_unstale":"2024-04-25T10:05:40.488675+0000","last_undegraded":"2024-04-25T10:05:40.488675+0000","last_fullsized":"2024-04-25T10:05:40.488675+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:51.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.487583+0000","last_change":"2024-04-25T10:05:39.083955+0000","last_active":"2024-04-25T10:05:40.487583+0000","last_peered":"2024-04-25T10:05:40.487583+0000","last_clean":"2024-04-25T10:05:40.487583+0000","last_became_active":"2024-04-25T10:05:39.083678+0000","last_became_peered":"2024-04-25T10:05:39.083678+0000","last_unstale":"2024-04-25T10:05:40.487583+0000","last_undegraded":"2024-04-25T10:05:40.487583+0000","last_fullsized":"2024-04-25T10:05:40.487583+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:34.354323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"188'2","reported_seq":16,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488899+0000","last_change":"2024-04-25T10:05:38.087415+0000","last_active":"2024-04-25T10:05:40.488899+0000","last_peered":"2024-04-25T10:05:40.488899+0000","last_clean":"2024-04-25T10:05:40.488899+0000","last_became_active":"2024-04-25T10:05:38.087167+0000","last_became_peered":"2024-04-25T10:05:38.087167+0000","last_unstale":"2024-04-25T10:05:40.488899+0000","last_undegraded":"2024-04-25T10:05:40.488899+0000","last_fullsized":"2024-04-25T10:05:40.488899+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:58.145661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488325+0000","last_change":"2024-04-25T10:05:39.098979+0000","last_active":"2024-04-25T10:05:40.488325+0000","last_peered":"2024-04-25T10:05:40.488325+0000","last_clean":"2024-04-25T10:05:40.488325+0000","last_became_active":"2024-04-25T10:05:39.098546+0000","last_became_peered":"2024-04-25T10:05:39.098546+0000","last_unstale":"2024-04-25T10:05:40.488325+0000","last_undegraded":"2024-04-25T10:05:40.488325+0000","last_fullsized":"2024-04-25T10:05:40.488325+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:54.723642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488212+0000","last_change":"2024-04-25T10:05:38.081783+0000","last_active":"2024-04-25T10:05:40.488212+0000","last_peered":"2024-04-25T10:05:40.488212+0000","last_clean":"2024-04-25T10:05:40.488212+0000","last_became_active":"2024-04-25T10:05:38.081463+0000","last_became_peered":"2024-04-25T10:05:38.081463+0000","last_unstale":"2024-04-25T10:05:40.488212+0000","last_undegraded":"2024-04-25T10:05:40.488212+0000","last_fullsized":"2024-04-25T10:05:40.488212+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:52.988550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.487812+0000","last_change":"2024-04-25T10:05:39.073464+0000","last_active":"2024-04-25T10:05:40.487812+0000","last_peered":"2024-04-25T10:05:40.487812+0000","last_clean":"2024-04-25T10:05:40.487812+0000","last_became_active":"2024-04-25T10:05:39.073201+0000","last_became_peered":"2024-04-25T10:05:39.073201+0000","last_unstale":"2024-04-25T10:05:40.487812+0000","last_undegraded":"2024-04-25T10:05:40.487812+0000","last_fullsized":"2024-04-25T10:05:40.487812+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:34.354323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488193+0000","last_change":"2024-04-25T10:05:38.076940+0000","last_active":"2024-04-25T10:05:40.488193+0000","last_peered":"2024-04-25T10:05:40.488193+0000","last_clean":"2024-04-25T10:05:40.488193+0000","last_became_active":"2024-04-25T10:05:38.076639+0000","last_became_peered":"2024-04-25T10:05:38.076639+0000","last_unstale":"2024-04-25T10:05:40.488193+0000","last_undegraded":"2024-04-25T10:05:40.488193+0000","last_fullsized":"2024-04-25T10:05:40.488193+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:37.355154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.083730+0000","last_change":"2024-04-25T10:05:39.083157+0000","last_active":"2024-04-25T10:05:40.083730+0000","last_peered":"2024-04-25T10:05:40.083730+0000","last_clean":"2024-04-25T10:05:40.083730+0000","last_became_active":"2024-04-25T10:05:39.082886+0000","last_became_peered":"2024-04-25T10:05:39.082886+0000","last_unstale":"2024-04-25T10:05:40.083730+0000","last_undegraded":"2024-04-25T10:05:40.083730+0000","last_fullsized":"2024-04-25T10:05:40.083730+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:54.522337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"188'8","reported_seq":22,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488583+0000","last_change":"2024-04-25T10:05:38.083896+0000","last_active":"2024-04-25T10:05:40.488583+0000","last_peered":"2024-04-25T10:05:40.488583+0000","last_clean":"2024-04-25T10:05:40.488583+0000","last_became_active":"2024-04-25T10:05:38.083309+0000","last_became_peered":"2024-04-25T10:05:38.083309+0000","last_unstale":"2024-04-25T10:05:40.488583+0000","last_undegraded":"2024-04-25T10:05:40.488583+0000","last_fullsized":"2024-04-25T10:05:40.488583+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:26.570699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488615+0000","last_change":"2024-04-25T10:05:39.094125+0000","last_active":"2024-04-25T10:05:40.488615+0000","last_peered":"2024-04-25T10:05:40.488615+0000","last_clean":"2024-04-25T10:05:40.488615+0000","last_became_active":"2024-04-25T10:05:39.093716+0000","last_became_peered":"2024-04-25T10:05:39.093716+0000","last_unstale":"2024-04-25T10:05:40.488615+0000","last_undegraded":"2024-04-25T10:05:40.488615+0000","last_fullsized":"2024-04-25T10:05:40.488615+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:02.200478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"188'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488445+0000","last_change":"2024-04-25T10:05:38.083645+0000","last_active":"2024-04-25T10:05:40.488445+0000","last_peered":"2024-04-25T10:05:40.488445+0000","last_clean":"2024-04-25T10:05:40.488445+0000","last_became_active":"2024-04-25T10:05:38.083357+0000","last_became_peered":"2024-04-25T10:05:38.083357+0000","last_unstale":"2024-04-25T10:05:40.488445+0000","last_undegraded":"2024-04-25T10:05:40.488445+0000","last_fullsized":"2024-04-25T10:05:40.488445+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:32.773871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.087445+0000","last_change":"2024-04-25T10:05:39.078169+0000","last_active":"2024-04-25T10:05:40.087445+0000","last_peered":"2024-04-25T10:05:40.087445+0000","last_clean":"2024-04-25T10:05:40.087445+0000","last_became_active":"2024-04-25T10:05:39.077273+0000","last_became_peered":"2024-04-25T10:05:39.077273+0000","last_unstale":"2024-04-25T10:05:40.087445+0000","last_undegraded":"2024-04-25T10:05:40.087445+0000","last_fullsized":"2024-04-25T10:05:40.087445+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:59.378346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"188'2","reported_seq":16,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488133+0000","last_change":"2024-04-25T10:05:38.083600+0000","last_active":"2024-04-25T10:05:40.488133+0000","last_peered":"2024-04-25T10:05:40.488133+0000","last_clean":"2024-04-25T10:05:40.488133+0000","last_became_active":"2024-04-25T10:05:38.083303+0000","last_became_peered":"2024-04-25T10:05:38.083303+0000","last_unstale":"2024-04-25T10:05:40.488133+0000","last_undegraded":"2024-04-25T10:05:40.488133+0000","last_fullsized":"2024-04-25T10:05:40.488133+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:04.461670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.489036+0000","last_change":"2024-04-25T10:05:39.087038+0000","last_active":"2024-04-25T10:05:40.489036+0000","last_peered":"2024-04-25T10:05:40.489036+0000","last_clean":"2024-04-25T10:05:40.489036+0000","last_became_active":"2024-04-25T10:05:39.086442+0000","last_became_peered":"2024-04-25T10:05:39.086442+0000","last_unstale":"2024-04-25T10:05:40.489036+0000","last_undegraded":"2024-04-25T10:05:40.489036+0000","last_fullsized":"2024-04-25T10:05:40.489036+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:06.388844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"188'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.489166+0000","last_change":"2024-04-25T10:05:38.081964+0000","last_active":"2024-04-25T10:05:40.489166+0000","last_peered":"2024-04-25T10:05:40.489166+0000","last_clean":"2024-04-25T10:05:40.489166+0000","last_became_active":"2024-04-25T10:05:38.081673+0000","last_became_peered":"2024-04-25T10:05:38.081673+0000","last_unstale":"2024-04-25T10:05:40.489166+0000","last_undegraded":"2024-04-25T10:05:40.489166+0000","last_fullsized":"2024-04-25T10:05:40.489166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:59.923848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488989+0000","last_change":"2024-04-25T10:05:39.090956+0000","last_active":"2024-04-25T10:05:40.488989+0000","last_peered":"2024-04-25T10:05:40.488989+0000","last_clean":"2024-04-25T10:05:40.488989+0000","last_became_active":"2024-04-25T10:05:39.089066+0000","last_became_peered":"2024-04-25T10:05:39.089066+0000","last_unstale":"2024-04-25T10:05:40.488989+0000","last_undegraded":"2024-04-25T10:05:40.488989+0000","last_fullsized":"2024-04-25T10:05:40.488989+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:45.911033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.489286+0000","last_change":"2024-04-25T10:05:38.085345+0000","last_active":"2024-04-25T10:05:40.489286+0000","last_peered":"2024-04-25T10:05:40.489286+0000","last_clean":"2024-04-25T10:05:40.489286+0000","last_became_active":"2024-04-25T10:05:38.082298+0000","last_became_peered":"2024-04-25T10:05:38.082298+0000","last_unstale":"2024-04-25T10:05:40.489286+0000","last_undegraded":"2024-04-25T10:05:40.489286+0000","last_fullsized":"2024-04-25T10:05:40.489286+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:46.032506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488439+0000","last_change":"2024-04-25T10:05:39.089692+0000","last_active":"2024-04-25T10:05:40.488439+0000","last_peered":"2024-04-25T10:05:40.488439+0000","last_clean":"2024-04-25T10:05:40.488439+0000","last_became_active":"2024-04-25T10:05:39.089403+0000","last_became_peered":"2024-04-25T10:05:39.089403+0000","last_unstale":"2024-04-25T10:05:40.488439+0000","last_undegraded":"2024-04-25T10:05:40.488439+0000","last_fullsized":"2024-04-25T10:05:40.488439+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:46.043721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"188'2","reported_seq":16,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.489036+0000","last_change":"2024-04-25T10:05:38.083542+0000","last_active":"2024-04-25T10:05:40.489036+0000","last_peered":"2024-04-25T10:05:40.489036+0000","last_clean":"2024-04-25T10:05:40.489036+0000","last_became_active":"2024-04-25T10:05:38.082652+0000","last_became_peered":"2024-04-25T10:05:38.082652+0000","last_unstale":"2024-04-25T10:05:40.489036+0000","last_undegraded":"2024-04-25T10:05:40.489036+0000","last_fullsized":"2024-04-25T10:05:40.489036+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:33.245823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.086844+0000","last_change":"2024-04-25T10:05:39.076245+0000","last_active":"2024-04-25T10:05:40.086844+0000","last_peered":"2024-04-25T10:05:40.086844+0000","last_clean":"2024-04-25T10:05:40.086844+0000","last_became_active":"2024-04-25T10:05:39.075946+0000","last_became_peered":"2024-04-25T10:05:39.075946+0000","last_unstale":"2024-04-25T10:05:40.086844+0000","last_undegraded":"2024-04-25T10:05:40.086844+0000","last_fullsized":"2024-04-25T10:05:40.086844+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:34.080155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.083617+0000","last_change":"2024-04-25T10:05:38.077397+0000","last_active":"2024-04-25T10:05:40.083617+0000","last_peered":"2024-04-25T10:05:40.083617+0000","last_clean":"2024-04-25T10:05:40.083617+0000","last_became_active":"2024-04-25T10:05:38.077107+0000","last_became_peered":"2024-04-25T10:05:38.077107+0000","last_unstale":"2024-04-25T10:05:40.083617+0000","last_undegraded":"2024-04-25T10:05:40.083617+0000","last_fullsized":"2024-04-25T10:05:40.083617+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:47.593866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488503+0000","last_change":"2024-04-25T10:05:39.090169+0000","last_active":"2024-04-25T10:05:40.488503+0000","last_peered":"2024-04-25T10:05:40.488503+0000","last_clean":"2024-04-25T10:05:40.488503+0000","last_became_active":"2024-04-25T10:05:39.089802+0000","last_became_peered":"2024-04-25T10:05:39.089802+0000","last_unstale":"2024-04-25T10:05:40.488503+0000","last_undegraded":"2024-04-25T10:05:40.488503+0000","last_fullsized":"2024-04-25T10:05:40.488503+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:09.805857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.083473+0000","last_change":"2024-04-25T10:05:38.075167+0000","last_active":"2024-04-25T10:05:40.083473+0000","last_peered":"2024-04-25T10:05:40.083473+0000","last_clean":"2024-04-25T10:05:40.083473+0000","last_became_active":"2024-04-25T10:05:38.074890+0000","last_became_peered":"2024-04-25T10:05:38.074890+0000","last_unstale":"2024-04-25T10:05:40.083473+0000","last_undegraded":"2024-04-25T10:05:40.083473+0000","last_fullsized":"2024-04-25T10:05:40.083473+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:43.833994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488138+0000","last_change":"2024-04-25T10:05:39.089724+0000","last_active":"2024-04-25T10:05:40.488138+0000","last_peered":"2024-04-25T10:05:40.488138+0000","last_clean":"2024-04-25T10:05:40.488138+0000","last_became_active":"2024-04-25T10:05:39.089450+0000","last_became_peered":"2024-04-25T10:05:39.089450+0000","last_unstale":"2024-04-25T10:05:40.488138+0000","last_undegraded":"2024-04-25T10:05:40.488138+0000","last_fullsized":"2024-04-25T10:05:40.488138+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:34.691545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.487583+0000","last_change":"2024-04-25T10:05:38.082353+0000","last_active":"2024-04-25T10:05:40.487583+0000","last_peered":"2024-04-25T10:05:40.487583+0000","last_clean":"2024-04-25T10:05:40.487583+0000","last_became_active":"2024-04-25T10:05:38.082047+0000","last_became_peered":"2024-04-25T10:05:38.082047+0000","last_unstale":"2024-04-25T10:05:40.487583+0000","last_undegraded":"2024-04-25T10:05:40.487583+0000","last_fullsized":"2024-04-25T10:05:40.487583+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:35.628934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488771+0000","last_change":"2024-04-25T10:05:39.081317+0000","last_active":"2024-04-25T10:05:40.488771+0000","last_peered":"2024-04-25T10:05:40.488771+0000","last_clean":"2024-04-25T10:05:40.488771+0000","last_became_active":"2024-04-25T10:05:39.081021+0000","last_became_peered":"2024-04-25T10:05:39.081021+0000","last_unstale":"2024-04-25T10:05:40.488771+0000","last_undegraded":"2024-04-25T10:05:40.488771+0000","last_fullsized":"2024-04-25T10:05:40.488771+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:16.588995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"188'2","reported_seq":16,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.489279+0000","last_change":"2024-04-25T10:05:38.083029+0000","last_active":"2024-04-25T10:05:40.489279+0000","last_peered":"2024-04-25T10:05:40.489279+0000","last_clean":"2024-04-25T10:05:40.489279+0000","last_became_active":"2024-04-25T10:05:38.082535+0000","last_became_peered":"2024-04-25T10:05:38.082535+0000","last_unstale":"2024-04-25T10:05:40.489279+0000","last_undegraded":"2024-04-25T10:05:40.489279+0000","last_fullsized":"2024-04-25T10:05:40.489279+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:13.249190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.087553+0000","last_change":"2024-04-25T10:05:39.078923+0000","last_active":"2024-04-25T10:05:40.087553+0000","last_peered":"2024-04-25T10:05:40.087553+0000","last_clean":"2024-04-25T10:05:40.087553+0000","last_became_active":"2024-04-25T10:05:39.078654+0000","last_became_peered":"2024-04-25T10:05:39.078654+0000","last_unstale":"2024-04-25T10:05:40.087553+0000","last_undegraded":"2024-04-25T10:05:40.087553+0000","last_fullsized":"2024-04-25T10:05:40.087553+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:16.395964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"189'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.083846+0000","last_change":"2024-04-25T10:05:38.073711+0000","last_active":"2024-04-25T10:05:40.083846+0000","last_peered":"2024-04-25T10:05:40.083846+0000","last_clean":"2024-04-25T10:05:40.083846+0000","last_became_active":"2024-04-25T10:05:38.073414+0000","last_became_peered":"2024-04-25T10:05:38.073414+0000","last_unstale":"2024-04-25T10:05:40.083846+0000","last_undegraded":"2024-04-25T10:05:40.083846+0000","last_fullsized":"2024-04-25T10:05:40.083846+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:53.856804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488348+0000","last_change":"2024-04-25T10:05:39.091979+0000","last_active":"2024-04-25T10:05:40.488348+0000","last_peered":"2024-04-25T10:05:40.488348+0000","last_clean":"2024-04-25T10:05:40.488348+0000","last_became_active":"2024-04-25T10:05:39.091705+0000","last_became_peered":"2024-04-25T10:05:39.091705+0000","last_unstale":"2024-04-25T10:05:40.488348+0000","last_undegraded":"2024-04-25T10:05:40.488348+0000","last_fullsized":"2024-04-25T10:05:40.488348+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:01.800308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.083855+0000","last_change":"2024-04-25T10:05:38.071762+0000","last_active":"2024-04-25T10:05:40.083855+0000","last_peered":"2024-04-25T10:05:40.083855+0000","last_clean":"2024-04-25T10:05:40.083855+0000","last_became_active":"2024-04-25T10:05:38.071429+0000","last_became_peered":"2024-04-25T10:05:38.071429+0000","last_unstale":"2024-04-25T10:05:40.083855+0000","last_undegraded":"2024-04-25T10:05:40.083855+0000","last_fullsized":"2024-04-25T10:05:40.083855+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:27.770003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.487687+0000","last_change":"2024-04-25T10:05:39.084607+0000","last_active":"2024-04-25T10:05:40.487687+0000","last_peered":"2024-04-25T10:05:40.487687+0000","last_clean":"2024-04-25T10:05:40.487687+0000","last_became_active":"2024-04-25T10:05:39.084322+0000","last_became_peered":"2024-04-25T10:05:39.084322+0000","last_unstale":"2024-04-25T10:05:40.487687+0000","last_undegraded":"2024-04-25T10:05:40.487687+0000","last_fullsized":"2024-04-25T10:05:40.487687+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:18.199331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.489305+0000","last_change":"2024-04-25T10:05:38.084273+0000","last_active":"2024-04-25T10:05:40.489305+0000","last_peered":"2024-04-25T10:05:40.489305+0000","last_clean":"2024-04-25T10:05:40.489305+0000","last_became_active":"2024-04-25T10:05:38.082528+0000","last_became_peered":"2024-04-25T10:05:38.082528+0000","last_unstale":"2024-04-25T10:05:40.489305+0000","last_undegraded":"2024-04-25T10:05:40.489305+0000","last_fullsized":"2024-04-25T10:05:40.489305+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:20.840458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.487977+0000","last_change":"2024-04-25T10:05:39.090857+0000","last_active":"2024-04-25T10:05:40.487977+0000","last_peered":"2024-04-25T10:05:40.487977+0000","last_clean":"2024-04-25T10:05:40.487977+0000","last_became_active":"2024-04-25T10:05:39.090218+0000","last_became_peered":"2024-04-25T10:05:39.090218+0000","last_unstale":"2024-04-25T10:05:40.487977+0000","last_undegraded":"2024-04-25T10:05:40.487977+0000","last_fullsized":"2024-04-25T10:05:40.487977+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:09.805857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.489091+0000","last_change":"2024-04-25T10:05:38.086909+0000","last_active":"2024-04-25T10:05:40.489091+0000","last_peered":"2024-04-25T10:05:40.489091+0000","last_clean":"2024-04-25T10:05:40.489091+0000","last_became_active":"2024-04-25T10:05:38.086646+0000","last_became_peered":"2024-04-25T10:05:38.086646+0000","last_unstale":"2024-04-25T10:05:40.489091+0000","last_undegraded":"2024-04-25T10:05:40.489091+0000","last_fullsized":"2024-04-25T10:05:40.489091+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:35.445275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488448+0000","last_change":"2024-04-25T10:05:39.080681+0000","last_active":"2024-04-25T10:05:40.488448+0000","last_peered":"2024-04-25T10:05:40.488448+0000","last_clean":"2024-04-25T10:05:40.488448+0000","last_became_active":"2024-04-25T10:05:39.080352+0000","last_became_peered":"2024-04-25T10:05:39.080352+0000","last_unstale":"2024-04-25T10:05:40.488448+0000","last_undegraded":"2024-04-25T10:05:40.488448+0000","last_fullsized":"2024-04-25T10:05:40.488448+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:45.234556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.086357+0000","last_change":"2024-04-25T10:05:38.081354+0000","last_active":"2024-04-25T10:05:40.086357+0000","last_peered":"2024-04-25T10:05:40.086357+0000","last_clean":"2024-04-25T10:05:40.086357+0000","last_became_active":"2024-04-25T10:05:38.081054+0000","last_became_peered":"2024-04-25T10:05:38.081054+0000","last_unstale":"2024-04-25T10:05:40.086357+0000","last_undegraded":"2024-04-25T10:05:40.086357+0000","last_fullsized":"2024-04-25T10:05:40.086357+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:40.294505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488317+0000","last_change":"2024-04-25T10:05:39.088105+0000","last_active":"2024-04-25T10:05:40.488317+0000","last_peered":"2024-04-25T10:05:40.488317+0000","last_clean":"2024-04-25T10:05:40.488317+0000","last_became_active":"2024-04-25T10:05:39.087840+0000","last_became_peered":"2024-04-25T10:05:39.087840+0000","last_unstale":"2024-04-25T10:05:40.488317+0000","last_undegraded":"2024-04-25T10:05:40.488317+0000","last_fullsized":"2024-04-25T10:05:40.488317+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:28.641554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.087163+0000","last_change":"2024-04-25T10:05:38.075546+0000","last_active":"2024-04-25T10:05:40.087163+0000","last_peered":"2024-04-25T10:05:40.087163+0000","last_clean":"2024-04-25T10:05:40.087163+0000","last_became_active":"2024-04-25T10:05:38.074936+0000","last_became_peered":"2024-04-25T10:05:38.074936+0000","last_unstale":"2024-04-25T10:05:40.087163+0000","last_undegraded":"2024-04-25T10:05:40.087163+0000","last_fullsized":"2024-04-25T10:05:40.087163+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:53.856804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488728+0000","last_change":"2024-04-25T10:05:39.089658+0000","last_active":"2024-04-25T10:05:40.488728+0000","last_peered":"2024-04-25T10:05:40.488728+0000","last_clean":"2024-04-25T10:05:40.488728+0000","last_became_active":"2024-04-25T10:05:39.089377+0000","last_became_peered":"2024-04-25T10:05:39.089377+0000","last_unstale":"2024-04-25T10:05:40.488728+0000","last_undegraded":"2024-04-25T10:05:40.488728+0000","last_fullsized":"2024-04-25T10:05:40.488728+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:15.647412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488693+0000","last_change":"2024-04-25T10:05:38.083077+0000","last_active":"2024-04-25T10:05:40.488693+0000","last_peered":"2024-04-25T10:05:40.488693+0000","last_clean":"2024-04-25T10:05:40.488693+0000","last_became_active":"2024-04-25T10:05:38.082606+0000","last_became_peered":"2024-04-25T10:05:38.082606+0000","last_unstale":"2024-04-25T10:05:40.488693+0000","last_undegraded":"2024-04-25T10:05:40.488693+0000","last_fullsized":"2024-04-25T10:05:40.488693+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:21.692107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.488285+0000","last_change":"2024-04-25T10:05:39.094049+0000","last_active":"2024-04-25T10:05:40.488285+0000","last_peered":"2024-04-25T10:05:40.488285+0000","last_clean":"2024-04-25T10:05:40.488285+0000","last_became_active":"2024-04-25T10:05:39.093713+0000","last_became_peered":"2024-04-25T10:05:39.093713+0000","last_unstale":"2024-04-25T10:05:40.488285+0000","last_undegraded":"2024-04-25T10:05:40.488285+0000","last_fullsized":"2024-04-25T10:05:40.488285+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:38.052841+0000","last_clean_scrub_stamp":"2024-04-25T10:05:38.052841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:16.588995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"188'2","reported_seq":16,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.087095+0000","last_change":"2024-04-25T10:05:38.072474+0000","last_active":"2024-04-25T10:05:40.087095+0000","last_peered":"2024-04-25T10:05:40.087095+0000","last_clean":"2024-04-25T10:05:40.087095+0000","last_became_active":"2024-04-25T10:05:38.072160+0000","last_became_peered":"2024-04-25T10:05:38.072160+0000","last_unstale":"2024-04-25T10:05:40.087095+0000","last_undegraded":"2024-04-25T10:05:40.087095+0000","last_fullsized":"2024-04-25T10:05:40.087095+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:37.050856+0000","last_clean_scrub_stamp":"2024-04-25T10:05:37.050856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:59.192742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":514,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-04-25T10:05:40.086241+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:05:40.086241+0000","last_peered":"2024-04-25T10:05:40.086241+0000","last_clean":"2024-04-25T10:05:40.086241+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:05:40.086241+0000","last_undegraded":"2024-04-25T10:05:40.086241+0000","last_fullsized":"2024-04-25T10:05:40.086241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56316,"kb_used_data":2924,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315524,"statfs":{"total":96636764160,"available":96579096576,"internally_reserved":0,"allocated":2994176,"data_stored":1715308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575006,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51588,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320252,"statfs":{"total":96636764160,"available":96583938048,"internally_reserved":0,"allocated":2416640,"data_stored":1126006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575005,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51580,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320260,"statfs":{"total":96636764160,"available":96583946240,"internally_reserved":0,"allocated":2408448,"data_stored":1125950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575005,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59888,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311952,"statfs":{"total":96636764160,"available":96575438848,"internally_reserved":0,"allocated":2396160,"data_stored":1124358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575006,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59892,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311948,"statfs":{"total":96636764160,"available":96575434752,"internally_reserved":0,"allocated":2396160,"data_stored":1124290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55736,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316104,"statfs":{"total":96636764160,"available":96579690496,"internally_reserved":0,"allocated":2400256,"data_stored":1124884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55728,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316112,"statfs":{"total":96636764160,"available":96579698688,"internally_reserved":0,"allocated":2392064,"data_stored":1124268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56304,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315536,"statfs":{"total":96636764160,"available":96579108864,"internally_reserved":0,"allocated":2985984,"data_stored":1714636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:46.825 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:05:46.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:05:46.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.965+0000 7f73ed916640 1 Processor -- start 2024-04-25T10:05:46.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.965+0000 7f73ed916640 1 -- start start 2024-04-25T10:05:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73ed916640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f73e80b5620 0x7f73e80b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73ed916640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e806cf50 con 0x7f73e806c2d0 2024-04-25T10:05:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73ed916640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e80ce530 con 0x7f73e815b7d0 2024-04-25T10:05:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73ed916640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e80ce810 con 0x7f73e80b5620 2024-04-25T10:05:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e77fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f73e806c2d0 0x7f73e806c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34918/0 (socket says 172.21.15.19:34918) 2024-04-25T10:05:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e77fe640 1 -- 172.21.15.19:0/971340590 learned_addr learned my addr 172.21.15.19:0/971340590 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e67fc640 1 --2- 172.21.15.19:0/971340590 >> v2:172.21.15.112:3300/0 conn(0x7f73e80b5620 0x7f73e80b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581204442 0 0) 0x7f73e806cf50 con 0x7f73e806c2d0 2024-04-25T10:05:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73d0003520 con 0x7f73e806c2d0 2024-04-25T10:05:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 664435890 0 0) 0x7f73d0003520 con 0x7f73e806c2d0 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 >> v2:172.21.15.112:3300/0 conn(0x7f73e80b5620 msgr2=0x7f73e80b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e5ffb640 1 --2- 172.21.15.19:0/971340590 >> v2:172.21.15.112:3300/0 conn(0x7f73e80b5620 0x7f73e80b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 >> v1:172.21.15.112:6789/0 conn(0x7f73e815b7d0 legacy=0x7f73e80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e80ceaf0 con 0x7f73e806c2d0 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f73e00019f0 con 0x7f73e806c2d0 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 >> v1:172.21.15.19:6789/0 conn(0x7f73e806c2d0 legacy=0x7f73e806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.966+0000 7f73e5ffb640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d00039b0 0x7f73d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e5ffb640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d0004800 0x7f73d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e5ffb640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73d00051c0 0x7f73d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80ce810 con 0x7f73d0004800 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d000c060 con 0x7f73d00039b0 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d000c260 con 0x7f73d00051c0 2024-04-25T10:05:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e67fc640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d0004800 0x7f73d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e6ffd640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d00039b0 0x7f73d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e77fe640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73d00051c0 0x7f73d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e67fc640 1 -- 172.21.15.19:0/971340590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73d00051c0 msgr2=0x7f73d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e67fc640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73d00051c0 0x7f73d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e67fc640 1 -- 172.21.15.19:0/971340590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d00039b0 msgr2=0x7f73d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e67fc640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d00039b0 0x7f73d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e67fc640 1 -- 172.21.15.19:0/971340590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d0003520 con 0x7f73d0004800 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e77fe640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73d00051c0 0x7f73d000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.967+0000 7f73e77fe640 1 -- 172.21.15.19:0/971340590 reap_dead start 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.968+0000 7f73e67fc640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d0004800 0x7f73d0004c80 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009ac0 tx=0x7f73d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1f89e744b0b883 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.968+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc036070 con 0x7f73d0004800 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.968+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc031400 con 0x7f73d0004800 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.968+0000 7f73e5ffb640 1 -- 172.21.15.19:0/971340590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc009690 con 0x7f73d0004800 2024-04-25T10:05:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.968+0000 7f73ed916640 1 -- 172.21.15.19:0/971340590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d0004800 msgr2=0x7f73d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.968+0000 7f73ed916640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d0004800 0x7f73d0004c80 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009ac0 tx=0x7f73d0005830 comp rx=0 tx=0).stop 2024-04-25T10:05:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.968+0000 7f73ed916640 1 -- 172.21.15.19:0/971340590 shutdown_connections 2024-04-25T10:05:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.968+0000 7f73ed916640 1 --2- 172.21.15.19:0/971340590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d0004800 0x7f73d0004c80 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.968+0000 7f73ed916640 1 -- 172.21.15.19:0/971340590 >> 172.21.15.19:0/971340590 conn(0x7f73e80d0800 msgr2=0x7f73e80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.968+0000 7f73ed916640 1 -- 172.21.15.19:0/971340590 shutdown_connections 2024-04-25T10:05:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.968+0000 7f73ed916640 1 -- 172.21.15.19:0/971340590 wait complete. 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.969+0000 7f73ed916640 1 Processor -- start 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.970+0000 7f73ed916640 1 -- start start 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.970+0000 7f73ed916640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73d0004800 0x7f73e8148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.970+0000 7f73ed916640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e814bcb0 0x7f73e80cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.970+0000 7f73ed916640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73e80b48b0 0x7f73e80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.970+0000 7f73ed916640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e815bd30 con 0x7f73e814bcb0 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.970+0000 7f73ed916640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e815bbb0 con 0x7f73e80b48b0 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.970+0000 7f73ed916640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80b5ab0 con 0x7f73d0004800 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.970+0000 7f73e67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e814bcb0 0x7f73e80cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73e67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e814bcb0 0x7f73e80cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35862/0 (socket says 172.21.15.19:35862) 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73e67fc640 1 -- 172.21.15.19:0/2054826239 learned_addr learned my addr 172.21.15.19:0/2054826239 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73e6ffd640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73d0004800 0x7f73e8148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73e77fe640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73e80b48b0 0x7f73e80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73e67fc640 1 -- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73d0004800 msgr2=0x7f73e8148bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73e67fc640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73d0004800 0x7f73e8148bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73e67fc640 1 -- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73e80b48b0 msgr2=0x7f73e80b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73e67fc640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73e80b48b0 0x7f73e80b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73e67fc640 1 -- 172.21.15.19:0/2054826239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e8158650 con 0x7f73e814bcb0 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73e6ffd640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73d0004800 0x7f73e8148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73e67fc640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e814bcb0 0x7f73e80cac70 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009be0 tx=0x7f73dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73cffff640 1 -- 172.21.15.19:0/2054826239 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc03f020 con 0x7f73e814bcb0 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73cffff640 1 -- 172.21.15.19:0/2054826239 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc037040 con 0x7f73e814bcb0 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73cffff640 1 -- 172.21.15.19:0/2054826239 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc036070 con 0x7f73e814bcb0 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73ed916640 1 -- 172.21.15.19:0/2054826239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e80cdb60 con 0x7f73e814bcb0 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.971+0000 7f73ed916640 1 -- 172.21.15.19:0/2054826239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e80cd5a0 con 0x7f73e814bcb0 2024-04-25T10:05:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.972+0000 7f73ed916640 1 -- 172.21.15.19:0/2054826239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e8073a00 con 0x7f73e814bcb0 2024-04-25T10:05:46.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.973+0000 7f73cffff640 1 -- 172.21.15.19:0/2054826239 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73dc042020 con 0x7f73e814bcb0 2024-04-25T10:05:46.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.973+0000 7f73cffff640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f73d406b500 0x7f73d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.973+0000 7f73cffff640 1 -- 172.21.15.19:0/2054826239 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f73dc0aa450 con 0x7f73e814bcb0 2024-04-25T10:05:46.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.974+0000 7f73e6ffd640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f73d406b500 0x7f73d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.974+0000 7f73e6ffd640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f73d406b500 0x7f73d406d9c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f73d8005600 tx=0x7f73d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:46.978+0000 7f73cffff640 1 -- 172.21.15.19:0/2054826239 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73dc049030 con 0x7f73e814bcb0 2024-04-25T10:05:47.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.230+0000 7f73ed916640 1 -- 172.21.15.19:0/2054826239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f73e80b6660 con 0x7f73e814bcb0 2024-04-25T10:05:47.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.232+0000 7f73cffff640 1 -- 172.21.15.19:0/2054826239 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+147 (secure 0 0 0) 0x7f73e8158650 con 0x7f73e814bcb0 2024-04-25T10:05:47.238 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T10:05:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDJJypmAGLDIhAAh7PMNi+2Wmm/xN967sxbdA== 2024-04-25T10:05:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T10:05:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T10:05:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T10:05:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 -- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f73d406b500 msgr2=0x7f73d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f73d406b500 0x7f73d406d9c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f73d8005600 tx=0x7f73d8009040 comp rx=0 tx=0).stop 2024-04-25T10:05:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 -- 172.21.15.19:0/2054826239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e814bcb0 msgr2=0x7f73e80cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e814bcb0 0x7f73e80cac70 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009be0 tx=0x7f73dc004730 comp rx=0 tx=0).stop 2024-04-25T10:05:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 -- 172.21.15.19:0/2054826239 shutdown_connections 2024-04-25T10:05:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f73d406b500 0x7f73d406d9c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73e80b48b0 0x7f73e80b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73e814bcb0 0x7f73e80cac70 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 --2- 172.21.15.19:0/2054826239 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f73d0004800 0x7f73e8148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 -- 172.21.15.19:0/2054826239 >> 172.21.15.19:0/2054826239 conn(0x7f73e80d0800 msgr2=0x7f73e815fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 -- 172.21.15.19:0/2054826239 shutdown_connections 2024-04-25T10:05:47.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.234+0000 7f73ed916640 1 -- 172.21.15.19:0/2054826239 wait complete. 2024-04-25T10:05:47.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.346+0000 7f0c017bd640 1 Processor -- start 2024-04-25T10:05:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.346+0000 7f0c017bd640 1 -- start start 2024-04-25T10:05:47.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0c017bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc15dc20 0x7f0bfc1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0c017bd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bfc0b93a0 0x7f0bfc0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0c017bd640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bfc15e000 con 0x7f0bfc0cec20 2024-04-25T10:05:47.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0c017bd640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bfc0b6bb0 con 0x7f0bfc0b93a0 2024-04-25T10:05:47.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0c017bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0b6030 con 0x7f0bfc15dc20 2024-04-25T10:05:47.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0bfa7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bfc0b93a0 0x7f0bfc0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0bfa7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bfc0b93a0 0x7f0bfc0ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35878/0 (socket says 172.21.15.19:35878) 2024-04-25T10:05:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0bfa7fc640 1 -- 172.21.15.19:0/1034465697 learned_addr learned my addr 172.21.15.19:0/1034465697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0bfaffd640 1 --2- 172.21.15.19:0/1034465697 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc15dc20 0x7f0bfc1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0bfa7fc640 1 -- 172.21.15.19:0/1034465697 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc15dc20 msgr2=0x7f0bfc1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0bfa7fc640 1 --2- 172.21.15.19:0/1034465697 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc15dc20 0x7f0bfc1616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0bfa7fc640 1 -- 172.21.15.19:0/1034465697 >> v1:172.21.15.19:6789/0 conn(0x7f0bfc0cec20 legacy=0x7f0bfc06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.347+0000 7f0bfa7fc640 1 -- 172.21.15.19:0/1034465697 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc161de0 con 0x7f0bfc0b93a0 2024-04-25T10:05:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.348+0000 7f0bfa7fc640 1 --2- 172.21.15.19:0/1034465697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bfc0b93a0 0x7f0bfc0ce6e0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009870 tx=0x7f0bf002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e846f8f376546f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.348+0000 7f0bf9ffb640 1 -- 172.21.15.19:0/1034465697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0037070 con 0x7f0bfc0b93a0 2024-04-25T10:05:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.348+0000 7f0bf9ffb640 1 -- 172.21.15.19:0/1034465697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf0032440 con 0x7f0bfc0b93a0 2024-04-25T10:05:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.348+0000 7f0bf9ffb640 1 -- 172.21.15.19:0/1034465697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0032810 con 0x7f0bfc0b93a0 2024-04-25T10:05:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.348+0000 7f0c017bd640 1 -- 172.21.15.19:0/1034465697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bfc0b93a0 msgr2=0x7f0bfc0ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.348+0000 7f0c017bd640 1 --2- 172.21.15.19:0/1034465697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bfc0b93a0 0x7f0bfc0ce6e0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009870 tx=0x7f0bf002a270 comp rx=0 tx=0).stop 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.349+0000 7f0c017bd640 1 -- 172.21.15.19:0/1034465697 shutdown_connections 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.349+0000 7f0c017bd640 1 --2- 172.21.15.19:0/1034465697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bfc0b93a0 0x7f0bfc0ce6e0 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.349+0000 7f0c017bd640 1 --2- 172.21.15.19:0/1034465697 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc15dc20 0x7f0bfc1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.349+0000 7f0c017bd640 1 -- 172.21.15.19:0/1034465697 >> 172.21.15.19:0/1034465697 conn(0x7f0bfc0d0490 msgr2=0x7f0bfc0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.349+0000 7f0c017bd640 1 -- 172.21.15.19:0/1034465697 shutdown_connections 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.349+0000 7f0c017bd640 1 -- 172.21.15.19:0/1034465697 wait complete. 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.350+0000 7f0c017bd640 1 Processor -- start 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.350+0000 7f0c017bd640 1 -- start start 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0c017bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc0b93a0 0x7f0bfc14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0c017bd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bfc0cec20 0x7f0bfc1e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0c017bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc15dc20 0x7f0bfc15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0c017bd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0d3520 con 0x7f0bfc0cec20 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0c017bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc1623a0 con 0x7f0bfc15dc20 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0c017bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0d36a0 con 0x7f0bfc0b93a0 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfa7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bfc0cec20 0x7f0bfc1e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfa7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bfc0cec20 0x7f0bfc1e8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35882/0 (socket says 172.21.15.19:35882) 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfa7fc640 1 -- 172.21.15.19:0/4229976017 learned_addr learned my addr 172.21.15.19:0/4229976017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfaffd640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc0b93a0 0x7f0bfc14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfb7fe640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc15dc20 0x7f0bfc15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfa7fc640 1 -- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc0b93a0 msgr2=0x7f0bfc14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfa7fc640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc0b93a0 0x7f0bfc14cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfa7fc640 1 -- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc15dc20 msgr2=0x7f0bfc15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfa7fc640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc15dc20 0x7f0bfc15f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfa7fc640 1 -- 172.21.15.19:0/4229976017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc1e7770 con 0x7f0bfc0cec20 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfb7fe640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc15dc20 0x7f0bfc15f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0bfa7fc640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bfc0cec20 0x7f0bfc1e8810 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0000c00 tx=0x7f0bf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0be37fe640 1 -- 172.21.15.19:0/4229976017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0041020 con 0x7f0bfc0cec20 2024-04-25T10:05:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0be37fe640 1 -- 172.21.15.19:0/4229976017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf00306c0 con 0x7f0bfc0cec20 2024-04-25T10:05:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0c017bd640 1 -- 172.21.15.19:0/4229976017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bfc1e7a50 con 0x7f0bfc0cec20 2024-04-25T10:05:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.351+0000 7f0be37fe640 1 -- 172.21.15.19:0/4229976017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0037040 con 0x7f0bfc0cec20 2024-04-25T10:05:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.352+0000 7f0c017bd640 1 -- 172.21.15.19:0/4229976017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bfc1e8010 con 0x7f0bfc0cec20 2024-04-25T10:05:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.353+0000 7f0c017bd640 1 -- 172.21.15.19:0/4229976017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bfc073990 con 0x7f0bfc0cec20 2024-04-25T10:05:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.353+0000 7f0be37fe640 1 -- 172.21.15.19:0/4229976017 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bf0044020 con 0x7f0bfc0cec20 2024-04-25T10:05:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.353+0000 7f0be37fe640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0be806b4d0 0x7f0be806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.353+0000 7f0be37fe640 1 -- 172.21.15.19:0/4229976017 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bf00aab00 con 0x7f0bfc0cec20 2024-04-25T10:05:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.354+0000 7f0bfaffd640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0be806b4d0 0x7f0be806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.354+0000 7f0bfaffd640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0be806b4d0 0x7f0be806d990 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0bec006fd0 tx=0x7f0bec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.358+0000 7f0be37fe640 1 -- 172.21.15.19:0/4229976017 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0bf004b030 con 0x7f0bfc0cec20 2024-04-25T10:05:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.575+0000 7f0c017bd640 1 -- 172.21.15.19:0/4229976017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bfc0b42a0 con 0x7f0bfc0cec20 2024-04-25T10:05:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.576+0000 7f0be37fe640 1 -- 172.21.15.19:0/4229976017 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f0bf00797f0 con 0x7f0bfc0cec20 2024-04-25T10:05:47.581 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:47.582 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":123,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:38.424174+0000","modified":"2024-04-25T10:05:40.426910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11559":{"gid":11559,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/392508344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":392508344},{"type":"v1","addr":"172.21.15.19:6835","nonce":392508344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:05:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 123 2024-04-25T10:05:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 -- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0be806b4d0 msgr2=0x7f0be806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0be806b4d0 0x7f0be806d990 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0bec006fd0 tx=0x7f0bec008040 comp rx=0 tx=0).stop 2024-04-25T10:05:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 -- 172.21.15.19:0/4229976017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bfc0cec20 msgr2=0x7f0bfc1e8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bfc0cec20 0x7f0bfc1e8810 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0000c00 tx=0x7f0bf00046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 -- 172.21.15.19:0/4229976017 shutdown_connections 2024-04-25T10:05:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0be806b4d0 0x7f0be806d990 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc15dc20 0x7f0bfc15f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bfc0cec20 0x7f0bfc1e8810 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 --2- 172.21.15.19:0/4229976017 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc0b93a0 0x7f0bfc14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 -- 172.21.15.19:0/4229976017 >> 172.21.15.19:0/4229976017 conn(0x7f0bfc0d0490 msgr2=0x7f0bfc06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 -- 172.21.15.19:0/4229976017 shutdown_connections 2024-04-25T10:05:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:47.578+0000 7f0c017bd640 1 -- 172.21.15.19:0/4229976017 wait complete. 2024-04-25T10:05:47.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:38.424174+0000', 'modified': '2024-04-25T10:05:40.426910+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11559}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11559': {'gid': 11559, 'name': 'c', 'rank': 0, 'incarnation': 121, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/392508344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 392508344}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 392508344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:05:47.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:05:47.998 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:05:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.099+0000 7fb4532b3640 1 Processor -- start 2024-04-25T10:05:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.100+0000 7fb4532b3640 1 -- start start 2024-04-25T10:05:48.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.100+0000 7fb4532b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cc4f0 0x7fb44c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.100+0000 7fb4532b3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb44c15b980 0x7fb44c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.100+0000 7fb4532b3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c072280 con 0x7fb44c0cdb20 2024-04-25T10:05:48.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.100+0000 7fb4532b3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44c0ce4c0 con 0x7fb44c15b980 2024-04-25T10:05:48.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.100+0000 7fb4532b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c071270 con 0x7fb44c0cc4f0 2024-04-25T10:05:48.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.101+0000 7fb45104f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cc4f0 0x7fb44c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.101+0000 7fb45104f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cc4f0 0x7fb44c0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:59444/0 (socket says 172.21.15.19:59444) 2024-04-25T10:05:48.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.101+0000 7fb451850640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb44c15b980 0x7fb44c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.101+0000 7fb45104f640 1 -- 172.21.15.19:0/983184863 learned_addr learned my addr 172.21.15.19:0/983184863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:48.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.101+0000 7fb45104f640 1 -- 172.21.15.19:0/983184863 >> v2:172.21.15.112:3300/0 conn(0x7fb44c15b980 msgr2=0x7fb44c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.101+0000 7fb45104f640 1 --2- 172.21.15.19:0/983184863 >> v2:172.21.15.112:3300/0 conn(0x7fb44c15b980 0x7fb44c0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.101+0000 7fb45104f640 1 -- 172.21.15.19:0/983184863 >> v1:172.21.15.112:6789/0 conn(0x7fb44c0cdb20 legacy=0x7fb44c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.101+0000 7fb45104f640 1 -- 172.21.15.19:0/983184863 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c0714c0 con 0x7fb44c0cc4f0 2024-04-25T10:05:48.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.102+0000 7fb45104f640 1 --2- 172.21.15.19:0/983184863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cc4f0 0x7fb44c0cc8d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb44800bb10 tx=0x7fb44802c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a438f6bd9ebd00eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.102+0000 7fb443fff640 1 -- 172.21.15.19:0/983184863 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448036020 con 0x7fb44c0cc4f0 2024-04-25T10:05:48.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.102+0000 7fb443fff640 1 -- 172.21.15.19:0/983184863 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb448032820 con 0x7fb44c0cc4f0 2024-04-25T10:05:48.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.103+0000 7fb443fff640 1 -- 172.21.15.19:0/983184863 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448032bf0 con 0x7fb44c0cc4f0 2024-04-25T10:05:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.103+0000 7fb4532b3640 1 -- 172.21.15.19:0/983184863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cc4f0 msgr2=0x7fb44c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.103+0000 7fb4532b3640 1 --2- 172.21.15.19:0/983184863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cc4f0 0x7fb44c0cc8d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb44800bb10 tx=0x7fb44802c920 comp rx=0 tx=0).stop 2024-04-25T10:05:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.103+0000 7fb4532b3640 1 -- 172.21.15.19:0/983184863 shutdown_connections 2024-04-25T10:05:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.103+0000 7fb4532b3640 1 --2- 172.21.15.19:0/983184863 >> v2:172.21.15.112:3300/0 conn(0x7fb44c15b980 0x7fb44c0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.103+0000 7fb4532b3640 1 --2- 172.21.15.19:0/983184863 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cc4f0 0x7fb44c0cc8d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.103+0000 7fb4532b3640 1 -- 172.21.15.19:0/983184863 >> 172.21.15.19:0/983184863 conn(0x7fb44c06abe0 msgr2=0x7fb44c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.103+0000 7fb4532b3640 1 -- 172.21.15.19:0/983184863 shutdown_connections 2024-04-25T10:05:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.103+0000 7fb4532b3640 1 -- 172.21.15.19:0/983184863 wait complete. 2024-04-25T10:05:48.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.104+0000 7fb4532b3640 1 Processor -- start 2024-04-25T10:05:48.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.104+0000 7fb4532b3640 1 -- start start 2024-04-25T10:05:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.105+0000 7fb4532b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb44c0cc4f0 0x7fb44c1e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.105+0000 7fb4532b3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cdb20 0x7fb44c1e3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.105+0000 7fb4532b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c15b980 0x7fb44c06de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.105+0000 7fb4532b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c071a80 con 0x7fb44c15b980 2024-04-25T10:05:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.105+0000 7fb4532b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c15bd90 con 0x7fb44c0cc4f0 2024-04-25T10:05:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.105+0000 7fb4532b3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c071d60 con 0x7fb44c0cdb20 2024-04-25T10:05:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.105+0000 7fb451850640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c15b980 0x7fb44c06de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.105+0000 7fb451850640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c15b980 0x7fb44c06de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46746/0 (socket says 172.21.15.19:46746) 2024-04-25T10:05:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.105+0000 7fb451850640 1 -- 172.21.15.19:0/1301203499 learned_addr learned my addr 172.21.15.19:0/1301203499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.105+0000 7fb45104f640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb44c0cc4f0 0x7fb44c1e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.105+0000 7fb45084e640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cdb20 0x7fb44c1e3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.106+0000 7fb451850640 1 -- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cdb20 msgr2=0x7fb44c1e3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.106+0000 7fb451850640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cdb20 0x7fb44c1e3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.106+0000 7fb451850640 1 -- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb44c0cc4f0 msgr2=0x7fb44c1e52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.106+0000 7fb451850640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb44c0cc4f0 0x7fb44c1e52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.106+0000 7fb451850640 1 -- 172.21.15.19:0/1301203499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c1e3c20 con 0x7fb44c15b980 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.106+0000 7fb451850640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c15b980 0x7fb44c06de30 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fb44400efc0 tx=0x7fb44400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.106+0000 7fb441ffb640 1 -- 172.21.15.19:0/1301203499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444018020 con 0x7fb44c15b980 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.106+0000 7fb441ffb640 1 -- 172.21.15.19:0/1301203499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb444014480 con 0x7fb44c15b980 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.106+0000 7fb4532b3640 1 -- 172.21.15.19:0/1301203499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c1e3f00 con 0x7fb44c15b980 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.106+0000 7fb441ffb640 1 -- 172.21.15.19:0/1301203499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444014870 con 0x7fb44c15b980 2024-04-25T10:05:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.106+0000 7fb4532b3640 1 -- 172.21.15.19:0/1301203499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c1e44c0 con 0x7fb44c15b980 2024-04-25T10:05:48.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.108+0000 7fb441ffb640 1 -- 172.21.15.19:0/1301203499 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb444005d10 con 0x7fb44c15b980 2024-04-25T10:05:48.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.108+0000 7fb441ffb640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb43006b4d0 0x7fb43006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.109+0000 7fb441ffb640 1 -- 172.21.15.19:0/1301203499 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb44408ae00 con 0x7fb44c15b980 2024-04-25T10:05:48.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.109+0000 7fb45104f640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb43006b4d0 0x7fb43006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.109+0000 7fb45104f640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb43006b4d0 0x7fb43006d990 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb448037000 tx=0x7fb44803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.110+0000 7fb455fba740 1 Processor -- start 2024-04-25T10:05:48.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.110+0000 7fb455fba740 1 -- start start 2024-04-25T10:05:48.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.110+0000 7fb455fba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33af87e0 0x559e33ad0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.110+0000 7fb455fba740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33afb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.110+0000 7fb455fba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33afb580 0x559e33acede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.110+0000 7fb455fba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e33a33a10 con 0x559e33af87e0 2024-04-25T10:05:48.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.110+0000 7fb455fba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e339dcb30 con 0x559e33afb580 2024-04-25T10:05:48.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.110+0000 7fb455fba740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e33a2fc50 con 0x559e33af8bc0 2024-04-25T10:05:48.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.110+0000 7fb45104f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33af87e0 0x559e33ad0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.110+0000 7fb45104f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33af87e0 0x559e33ad0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46758/0 (socket says 172.21.15.19:46758) 2024-04-25T10:05:48.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.110+0000 7fb45104f640 1 -- 172.21.15.19:0/3850840742 learned_addr learned my addr 172.21.15.19:0/3850840742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:48.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.111+0000 7fb45084e640 1 --2- 172.21.15.19:0/3850840742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33afb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.111+0000 7fb451850640 1 --2- 172.21.15.19:0/3850840742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33afb580 0x559e33acede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.111+0000 7fb45104f640 1 -- 172.21.15.19:0/3850840742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 msgr2=0x559e33afb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.111+0000 7fb45104f640 1 --2- 172.21.15.19:0/3850840742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33afb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.111+0000 7fb45104f640 1 -- 172.21.15.19:0/3850840742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33afb580 msgr2=0x559e33acede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.111+0000 7fb45104f640 1 --2- 172.21.15.19:0/3850840742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33afb580 0x559e33acede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.111+0000 7fb45104f640 1 -- 172.21.15.19:0/3850840742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e339f1e10 con 0x559e33af87e0 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.111+0000 7fb45104f640 1 --2- 172.21.15.19:0/3850840742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33af87e0 0x559e33ad0e40 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fb44802cf10 tx=0x7fb448031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c42a2554457190b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.111+0000 7fb436ffd640 1 -- 172.21.15.19:0/3850840742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448036040 con 0x559e33af87e0 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.111+0000 7fb436ffd640 1 -- 172.21.15.19:0/3850840742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44803baf0 con 0x559e33af87e0 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.111+0000 7fb436ffd640 1 -- 172.21.15.19:0/3850840742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448032e40 con 0x559e33af87e0 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.112+0000 7fb455fba740 1 -- 172.21.15.19:0/3850840742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33af87e0 msgr2=0x559e33ad0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.112+0000 7fb455fba740 1 --2- 172.21.15.19:0/3850840742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33af87e0 0x559e33ad0e40 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fb44802cf10 tx=0x7fb448031000 comp rx=0 tx=0).stop 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.112+0000 7fb455fba740 1 -- 172.21.15.19:0/3850840742 shutdown_connections 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.112+0000 7fb455fba740 1 --2- 172.21.15.19:0/3850840742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33afb580 0x559e33acede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.112+0000 7fb455fba740 1 --2- 172.21.15.19:0/3850840742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33afb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.112+0000 7fb455fba740 1 --2- 172.21.15.19:0/3850840742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33af87e0 0x559e33ad0e40 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.112+0000 7fb455fba740 1 -- 172.21.15.19:0/3850840742 >> 172.21.15.19:0/3850840742 conn(0x559e33acd840 msgr2=0x559e33b16040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.112+0000 7fb455fba740 1 -- 172.21.15.19:0/3850840742 shutdown_connections 2024-04-25T10:05:48.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.112+0000 7fb455fba740 1 -- 172.21.15.19:0/3850840742 wait complete. 2024-04-25T10:05:48.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.112+0000 7fb455fba740 1 -- start start 2024-04-25T10:05:48.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.112+0000 7fb455fba740 1 Processor -- start 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.113+0000 7fb455fba740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33af9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.113+0000 7fb455fba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.113+0000 7fb455fba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33b12ca0 0x559e33b0d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.113+0000 7fb455fba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e339f00c0 con 0x559e33afb580 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.113+0000 7fb455fba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e33b13d40 con 0x559e33b12ca0 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.113+0000 7fb455fba740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e33b2db40 con 0x559e33af8bc0 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.113+0000 7fb455fba740 10 client.0 _ll_register_callbacks cb 0x559e339e3ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.113+0000 7fb45084e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.113+0000 7fb45084e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46760/0 (socket says 172.21.15.19:46760) 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.113+0000 7fb45084e640 1 -- 172.21.15.19:0/577699480 learned_addr learned my addr 172.21.15.19:0/577699480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb45104f640 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33af9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb451850640 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33b12ca0 0x559e33b0d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb45084e640 1 -- 172.21.15.19:0/577699480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 msgr2=0x559e33af9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb45084e640 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33af9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb45084e640 1 -- 172.21.15.19:0/577699480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33b12ca0 msgr2=0x559e33b0d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb45084e640 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33b12ca0 0x559e33b0d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb45084e640 1 -- 172.21.15.19:0/577699480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e337482d0 con 0x559e33afb580 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb45084e640 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d0b0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fb43c001eb0 tx=0x7fb43c012750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9ab5488d7d86a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb42bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb42bfff640 1 -- 172.21.15.19:0/577699480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c005320 con 0x559e33afb580 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb42bfff640 1 -- 172.21.15.19:0/577699480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb43c018910 con 0x559e33afb580 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb455fba740 1 -- 172.21.15.19:0/577699480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb44803baf0 con 0x559e33afb580 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb42bfff640 1 -- 172.21.15.19:0/577699480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb43c018b10 con 0x559e33afb580 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb455fba740 10 client.11733 fetch_fsmap learned FSMap version 123 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb455fba740 1 -- 172.21.15.19:0/577699480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=2+}) v3 -- 0x559e338126b0 con 0x559e33afb580 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.114+0000 7fb42bfff640 1 -- 172.21.15.19:0/577699480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44803baf0 con 0x559e33afb580 2024-04-25T10:05:48.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.115+0000 7fb42bfff640 1 -- 172.21.15.19:0/577699480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 123) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb43c018cb0 con 0x559e33afb580 2024-04-25T10:05:48.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.115+0000 7fb455fba740 10 client.11733 fetch_fsmap finished waiting for FSMap version 123 2024-04-25T10:05:48.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.115+0000 7fb455fba740 10 client.11733 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:48.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.115+0000 7fb455fba740 20 client.11733 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:48.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.115+0000 7fb455fba740 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.19:6833/392508344,v1:172.21.15.19:6835/392508344] conn(0x559e33b2b600 0x559e33b39650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.115+0000 7fb455fba740 4 client.11733 mds_command: new command op to 11559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:48.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.115+0000 7fb455fba740 1 -- 172.21.15.19:0/577699480 --> [v2:172.21.15.19:6833/392508344,v1:172.21.15.19:6835/392508344] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e33a92920 con 0x559e33b2b600 2024-04-25T10:05:48.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.115+0000 7fb45104f640 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.19:6833/392508344,v1:172.21.15.19:6835/392508344] conn(0x559e33b2b600 0x559e33b39650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.115+0000 7fb45104f640 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.19:6833/392508344,v1:172.21.15.19:6835/392508344] conn(0x559e33b2b600 0x559e33b39650 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e9b913f344c3e60 server_cookie=4eb04662f2fa58ee in_seq=0 out_seq=0 2024-04-25T10:05:48.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.115+0000 7fb42bfff640 10 client.11733 ms_handle_connect on v2:172.21.15.19:6833/392508344 2024-04-25T10:05:48.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb42bfff640 1 -- 172.21.15.19:0/577699480 <== mds.0 v2:172.21.15.19:6833/392508344 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559e33a92920 con 0x559e33b2b600 2024-04-25T10:05:48.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb42bfff640 10 client.11733 handle_command_reply: tid=0 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 1 client.11733 shutdown 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 1 -- 172.21.15.19:0/577699480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 msgr2=0x559e33b0d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d0b0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fb43c001eb0 tx=0x7fb43c012750 comp rx=0 tx=0).stop 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 1 -- 172.21.15.19:0/577699480 shutdown_connections 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.19:6833/392508344,v1:172.21.15.19:6835/392508344] conn(0x559e33b2b600 0x559e33b39650 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33b12ca0 0x559e33b0d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d0b0 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 1 --2- 172.21.15.19:0/577699480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33af9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 1 -- 172.21.15.19:0/577699480 >> 172.21.15.19:0/577699480 conn(0x559e33af87e0 msgr2=0x559e33acede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb42bfff640 0 client.11733 ms_handle_reset on v2:172.21.15.19:6833/392508344 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 1 -- 172.21.15.19:0/577699480 shutdown_connections 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 1 -- 172.21.15.19:0/577699480 wait complete. 2024-04-25T10:05:48.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.116+0000 7fb455fba740 20 client.11733 trim_cache size 0 max 16384 2024-04-25T10:05:48.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.132+0000 7fb455fba740 1 Processor -- start 2024-04-25T10:05:48.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.132+0000 7fb455fba740 1 -- start start 2024-04-25T10:05:48.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb455fba740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33ace610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb455fba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb455fba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33b12ca0 0x559e33b33d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb455fba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c1fbf50 con 0x559e33afb580 2024-04-25T10:05:48.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb455fba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c1fb0b0 con 0x559e33b12ca0 2024-04-25T10:05:48.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb455fba740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c157cf0 con 0x559e33af8bc0 2024-04-25T10:05:48.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb45084e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb45084e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46774/0 (socket says 172.21.15.19:46774) 2024-04-25T10:05:48.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb45084e640 1 -- 172.21.15.19:0/2445615890 learned_addr learned my addr 172.21.15.19:0/2445615890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:48.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb45104f640 1 --2- 172.21.15.19:0/2445615890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33ace610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb451850640 1 --2- 172.21.15.19:0/2445615890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33b12ca0 0x559e33b33d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb45084e640 1 -- 172.21.15.19:0/2445615890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 msgr2=0x559e33ace610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb45084e640 1 --2- 172.21.15.19:0/2445615890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33ace610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb45084e640 1 -- 172.21.15.19:0/2445615890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33b12ca0 msgr2=0x559e33b33d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb45084e640 1 --2- 172.21.15.19:0/2445615890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33b12ca0 0x559e33b33d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb45084e640 1 -- 172.21.15.19:0/2445615890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e33af70d0 con 0x559e33afb580 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb45084e640 1 --2- 172.21.15.19:0/2445615890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d190 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fb43c000fb0 tx=0x7fb43c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e7e56ab5707637 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb4367fc640 1 -- 172.21.15.19:0/2445615890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c02f070 con 0x559e33afb580 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb4367fc640 1 -- 172.21.15.19:0/2445615890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c012c50 con 0x559e33afb580 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.133+0000 7fb4367fc640 1 -- 172.21.15.19:0/2445615890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c005320 con 0x559e33afb580 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.134+0000 7fb455fba740 1 -- 172.21.15.19:0/2445615890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 msgr2=0x559e33b0d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.134+0000 7fb455fba740 1 --2- 172.21.15.19:0/2445615890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d190 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fb43c000fb0 tx=0x7fb43c00f000 comp rx=0 tx=0).stop 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.134+0000 7fb455fba740 1 -- 172.21.15.19:0/2445615890 shutdown_connections 2024-04-25T10:05:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.134+0000 7fb455fba740 1 --2- 172.21.15.19:0/2445615890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33b12ca0 0x559e33b33d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.134+0000 7fb455fba740 1 --2- 172.21.15.19:0/2445615890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b0d190 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.134+0000 7fb455fba740 1 --2- 172.21.15.19:0/2445615890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33af8bc0 0x559e33ace610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.134+0000 7fb455fba740 1 -- 172.21.15.19:0/2445615890 >> 172.21.15.19:0/2445615890 conn(0x559e33af87e0 msgr2=0x559e33afea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.134+0000 7fb455fba740 1 -- 172.21.15.19:0/2445615890 shutdown_connections 2024-04-25T10:05:48.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.134+0000 7fb455fba740 1 -- 172.21.15.19:0/2445615890 wait complete. 2024-04-25T10:05:48.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.134+0000 7fb455fba740 1 -- start start 2024-04-25T10:05:48.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.134+0000 7fb455fba740 1 Processor -- start 2024-04-25T10:05:48.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.135+0000 7fb455fba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33af8bc0 0x559e33b3a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.135+0000 7fb455fba740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b3aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.135+0000 7fb455fba740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33b12ca0 0x559e33b84900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.135+0000 7fb455fba740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e33afa500 con 0x559e33afb580 2024-04-25T10:05:48.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.135+0000 7fb455fba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c1582f0 con 0x559e33af8bc0 2024-04-25T10:05:48.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.135+0000 7fb455fba740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e33b2c050 con 0x559e33b12ca0 2024-04-25T10:05:48.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.135+0000 7fb455fba740 10 client.0 _ll_register_callbacks cb 0x7fb44800b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:48.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.135+0000 7fb45084e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b3aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.135+0000 7fb45084e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b3aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46778/0 (socket says 172.21.15.19:46778) 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.135+0000 7fb45084e640 1 -- 172.21.15.19:0/904238913 learned_addr learned my addr 172.21.15.19:0/904238913 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb451850640 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33b12ca0 0x559e33b84900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb45104f640 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33af8bc0 0x559e33b3a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb45084e640 1 -- 172.21.15.19:0/904238913 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33b12ca0 msgr2=0x559e33b84900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb45084e640 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33b12ca0 0x559e33b84900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb45084e640 1 -- 172.21.15.19:0/904238913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33af8bc0 msgr2=0x559e33b3a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb45084e640 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33af8bc0 0x559e33b3a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb45084e640 1 -- 172.21.15.19:0/904238913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e33b0c690 con 0x559e33afb580 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb45084e640 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b3aae0 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7fb43c005620 tx=0x7fb43c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51546c51b5ffa78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb42bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb42bfff640 1 -- 172.21.15.19:0/904238913 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c02f040 con 0x559e33afb580 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb42bfff640 1 -- 172.21.15.19:0/904238913 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb43c0073f0 con 0x559e33afb580 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb455fba740 1 -- 172.21.15.19:0/904238913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb43c012c50 con 0x559e33afb580 2024-04-25T10:05:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb42bfff640 1 -- 172.21.15.19:0/904238913 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb43c042050 con 0x559e33afb580 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb455fba740 10 client.11739 fetch_fsmap learned FSMap version 123 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb455fba740 1 -- 172.21.15.19:0/904238913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=2+}) v3 -- 0x559e33aff110 con 0x559e33afb580 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb42bfff640 1 -- 172.21.15.19:0/904238913 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c012c50 con 0x559e33afb580 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb42bfff640 1 -- 172.21.15.19:0/904238913 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 123) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb43c045020 con 0x559e33afb580 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb455fba740 10 client.11739 fetch_fsmap finished waiting for FSMap version 123 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb455fba740 10 client.11739 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb455fba740 20 client.11739 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb455fba740 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.19:6833/392508344,v1:172.21.15.19:6835/392508344] conn(0x559e33ad0a70 0x559e33ad0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb455fba740 4 client.11739 mds_command: new command op to 11559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.136+0000 7fb455fba740 1 -- 172.21.15.19:0/904238913 --> [v2:172.21.15.19:6833/392508344,v1:172.21.15.19:6835/392508344] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559e33a40f50 con 0x559e33ad0a70 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb45104f640 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.19:6833/392508344,v1:172.21.15.19:6835/392508344] conn(0x559e33ad0a70 0x559e33ad0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb42bfff640 10 client.11739 ms_handle_connect on v2:172.21.15.19:6833/392508344 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb45104f640 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.19:6833/392508344,v1:172.21.15.19:6835/392508344] conn(0x559e33ad0a70 0x559e33ad0e50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbdf798bc9cee66f server_cookie=6d7419348c76df31 in_seq=0 out_seq=0 2024-04-25T10:05:48.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb42bfff640 1 -- 172.21.15.19:0/904238913 <== mds.0 v2:172.21.15.19:6833/392508344 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x559e33a40f50 con 0x559e33ad0a70 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb42bfff640 10 client.11739 handle_command_reply: tid=0 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb455fba740 1 client.11739 shutdown 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb455fba740 1 -- 172.21.15.19:0/904238913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 msgr2=0x559e33b3aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb455fba740 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b3aae0 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7fb43c005620 tx=0x7fb43c000ed0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb455fba740 1 -- 172.21.15.19:0/904238913 shutdown_connections 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb455fba740 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.19:6833/392508344,v1:172.21.15.19:6835/392508344] conn(0x559e33ad0a70 0x559e33ad0e50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb455fba740 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559e33b12ca0 0x559e33b84900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb455fba740 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559e33afb580 0x559e33b3aae0 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb42bfff640 0 client.11739 ms_handle_reset on v2:172.21.15.19:6833/392508344 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb455fba740 1 --2- 172.21.15.19:0/904238913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e33af8bc0 0x559e33b3a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb455fba740 1 -- 172.21.15.19:0/904238913 >> 172.21.15.19:0/904238913 conn(0x559e33af87e0 msgr2=0x559e33afea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb455fba740 1 -- 172.21.15.19:0/904238913 shutdown_connections 2024-04-25T10:05:48.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.137+0000 7fb455fba740 1 -- 172.21.15.19:0/904238913 wait complete. 2024-04-25T10:05:48.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb455fba740 20 client.11739 trim_cache size 0 max 16384 2024-04-25T10:05:48.150 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:05:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T10:05:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:05:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 11559, 2024-04-25T10:05:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:05:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:05:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:05:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 9.0740438260000005, 2024-04-25T10:05:48.150 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 123, 2024-04-25T10:05:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 189, 2024-04-25T10:05:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 189, 2024-04-25T10:05:48.151 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 14.105051453 2024-04-25T10:05:48.151 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 -- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb43006b4d0 msgr2=0x7fb43006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb43006b4d0 0x7fb43006d990 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb448037000 tx=0x7fb44803d040 comp rx=0 tx=0).stop 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 -- 172.21.15.19:0/1301203499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c15b980 msgr2=0x7fb44c06de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c15b980 0x7fb44c06de30 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fb44400efc0 tx=0x7fb44400c490 comp rx=0 tx=0).stop 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 -- 172.21.15.19:0/1301203499 shutdown_connections 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb43006b4d0 0x7fb43006d990 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c15b980 0x7fb44c06de30 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb44c0cdb20 0x7fb44c1e3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 --2- 172.21.15.19:0/1301203499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb44c0cc4f0 0x7fb44c1e52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 -- 172.21.15.19:0/1301203499 >> 172.21.15.19:0/1301203499 conn(0x7fb44c06abe0 msgr2=0x7fb44c15b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 -- 172.21.15.19:0/1301203499 shutdown_connections 2024-04-25T10:05:48.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.138+0000 7fb4532b3640 1 -- 172.21.15.19:0/1301203499 wait complete. 2024-04-25T10:05:48.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:48.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.249+0000 7f9c43770640 1 Processor -- start 2024-04-25T10:05:48.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.249+0000 7f9c43770640 1 -- start start 2024-04-25T10:05:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.249+0000 7f9c43770640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c3c15db70 0x7f9c3c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c43770640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3c0b6cb0 0x7f9c3c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c43770640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9c3c0b5a60 0x7f9c3c06c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c43770640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c162690 con 0x7f9c3c15db70 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c43770640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c0ceee0 con 0x7f9c3c0b5a60 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c43770640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0d1d10 con 0x7f9c3c0b6cb0 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c4150c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c3c15db70 0x7f9c3c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c4150c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c3c15db70 0x7f9c3c162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46792/0 (socket says 172.21.15.19:46792) 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c4150c640 1 -- 172.21.15.19:0/1915500182 learned_addr learned my addr 172.21.15.19:0/1915500182 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c40d0b640 1 --2- 172.21.15.19:0/1915500182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3c0b6cb0 0x7f9c3c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c41d0d640 1 --2- 172.21.15.19:0/1915500182 >> v2:172.21.15.112:3300/0 conn(0x7f9c3c0b5a60 0x7f9c3c06c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c4150c640 1 -- 172.21.15.19:0/1915500182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3c0b6cb0 msgr2=0x7f9c3c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c4150c640 1 --2- 172.21.15.19:0/1915500182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3c0b6cb0 0x7f9c3c0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c4150c640 1 -- 172.21.15.19:0/1915500182 >> v2:172.21.15.112:3300/0 conn(0x7f9c3c0b5a60 msgr2=0x7f9c3c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c4150c640 1 --2- 172.21.15.19:0/1915500182 >> v2:172.21.15.112:3300/0 conn(0x7f9c3c0b5a60 0x7f9c3c06c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.250+0000 7f9c4150c640 1 -- 172.21.15.19:0/1915500182 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0d1ff0 con 0x7f9c3c15db70 2024-04-25T10:05:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.251+0000 7f9c4150c640 1 --2- 172.21.15.19:0/1915500182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c15db70 0x7f9c3c162150 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c08bfb0 tx=0x7f9c3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1448de6b26e16263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.251+0000 7f9c33fff640 1 -- 172.21.15.19:0/1915500182 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38037070 con 0x7f9c3c15db70 2024-04-25T10:05:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.251+0000 7f9c33fff640 1 -- 172.21.15.19:0/1915500182 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c38032440 con 0x7f9c3c15db70 2024-04-25T10:05:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.251+0000 7f9c33fff640 1 -- 172.21.15.19:0/1915500182 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38032810 con 0x7f9c3c15db70 2024-04-25T10:05:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.251+0000 7f9c43770640 1 -- 172.21.15.19:0/1915500182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c15db70 msgr2=0x7f9c3c162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.251+0000 7f9c43770640 1 --2- 172.21.15.19:0/1915500182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c15db70 0x7f9c3c162150 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c08bfb0 tx=0x7f9c3802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.251+0000 7f9c43770640 1 -- 172.21.15.19:0/1915500182 shutdown_connections 2024-04-25T10:05:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.251+0000 7f9c43770640 1 --2- 172.21.15.19:0/1915500182 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3c0b6cb0 0x7f9c3c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.252+0000 7f9c43770640 1 --2- 172.21.15.19:0/1915500182 >> v2:172.21.15.112:3300/0 conn(0x7f9c3c0b5a60 0x7f9c3c06c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.252+0000 7f9c43770640 1 --2- 172.21.15.19:0/1915500182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c15db70 0x7f9c3c162150 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.252+0000 7f9c43770640 1 -- 172.21.15.19:0/1915500182 >> 172.21.15.19:0/1915500182 conn(0x7f9c3c0d0520 msgr2=0x7f9c3c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.252+0000 7f9c43770640 1 -- 172.21.15.19:0/1915500182 shutdown_connections 2024-04-25T10:05:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.252+0000 7f9c43770640 1 -- 172.21.15.19:0/1915500182 wait complete. 2024-04-25T10:05:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.253+0000 7f9c43770640 1 Processor -- start 2024-04-25T10:05:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.253+0000 7f9c43770640 1 -- start start 2024-04-25T10:05:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.253+0000 7f9c43770640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3c0b5a60 0x7f9c3c159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.253+0000 7f9c43770640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3c0b6cb0 0x7f9c3c15b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.253+0000 7f9c43770640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c15db70 0x7f9c3c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.253+0000 7f9c43770640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0ce530 con 0x7f9c3c15db70 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.253+0000 7f9c43770640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c15df50 con 0x7f9c3c0b5a60 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.253+0000 7f9c43770640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0cf1c0 con 0x7f9c3c0b6cb0 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c41d0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c15db70 0x7f9c3c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c41d0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c15db70 0x7f9c3c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46800/0 (socket says 172.21.15.19:46800) 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c41d0d640 1 -- 172.21.15.19:0/4070592916 learned_addr learned my addr 172.21.15.19:0/4070592916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c40d0b640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3c0b6cb0 0x7f9c3c15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c4150c640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3c0b5a60 0x7f9c3c159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c41d0d640 1 -- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3c0b6cb0 msgr2=0x7f9c3c15b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c41d0d640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3c0b6cb0 0x7f9c3c15b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c41d0d640 1 -- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3c0b5a60 msgr2=0x7f9c3c159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c41d0d640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3c0b5a60 0x7f9c3c159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c41d0d640 1 -- 172.21.15.19:0/4070592916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0aa5d0 con 0x7f9c3c15db70 2024-04-25T10:05:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c41d0d640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c15db70 0x7f9c3c15b6a0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f9c34007e60 tx=0x7f9c3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c31ffb640 1 -- 172.21.15.19:0/4070592916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34014620 con 0x7f9c3c15db70 2024-04-25T10:05:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c31ffb640 1 -- 172.21.15.19:0/4070592916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c34019070 con 0x7f9c3c15db70 2024-04-25T10:05:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c43770640 1 -- 172.21.15.19:0/4070592916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c0cbaa0 con 0x7f9c3c15db70 2024-04-25T10:05:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c31ffb640 1 -- 172.21.15.19:0/4070592916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34014a20 con 0x7f9c3c15db70 2024-04-25T10:05:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.254+0000 7f9c43770640 1 -- 172.21.15.19:0/4070592916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c0cc060 con 0x7f9c3c15db70 2024-04-25T10:05:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.255+0000 7f9c43770640 1 -- 172.21.15.19:0/4070592916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c0d3760 con 0x7f9c3c15db70 2024-04-25T10:05:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.256+0000 7f9c31ffb640 1 -- 172.21.15.19:0/4070592916 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c34014bc0 con 0x7f9c3c15db70 2024-04-25T10:05:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.256+0000 7f9c31ffb640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c2006b4d0 0x7f9c2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.256+0000 7f9c31ffb640 1 -- 172.21.15.19:0/4070592916 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c3408a9d0 con 0x7f9c3c15db70 2024-04-25T10:05:48.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.257+0000 7f9c4150c640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c2006b4d0 0x7f9c2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.257+0000 7f9c4150c640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c2006b4d0 0x7f9c2006d990 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9c38034000 tx=0x7f9c3802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.261+0000 7f9c31ffb640 1 -- 172.21.15.19:0/4070592916 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c3c0d3760 con 0x7f9c3c15db70 2024-04-25T10:05:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.476+0000 7f9c43770640 1 -- 172.21.15.19:0/4070592916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9c3c073990 con 0x7f9c3c15db70 2024-04-25T10:05:48.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.476+0000 7f9c31ffb640 1 -- 172.21.15.19:0/4070592916 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v123) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9c34059100 con 0x7f9c3c15db70 2024-04-25T10:05:48.482 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:48.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 -- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c2006b4d0 msgr2=0x7f9c2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c2006b4d0 0x7f9c2006d990 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9c38034000 tx=0x7f9c3802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:48.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 -- 172.21.15.19:0/4070592916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c15db70 msgr2=0x7f9c3c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c15db70 0x7f9c3c15b6a0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f9c34007e60 tx=0x7f9c3400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 -- 172.21.15.19:0/4070592916 shutdown_connections 2024-04-25T10:05:48.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f9c2006b4d0 0x7f9c2006d990 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c15db70 0x7f9c3c15b6a0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c3c0b6cb0 0x7f9c3c15b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 --2- 172.21.15.19:0/4070592916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c3c0b5a60 0x7f9c3c159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 -- 172.21.15.19:0/4070592916 >> 172.21.15.19:0/4070592916 conn(0x7f9c3c0d0520 msgr2=0x7f9c3c0c74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 -- 172.21.15.19:0/4070592916 shutdown_connections 2024-04-25T10:05:48.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:48.478+0000 7f9c43770640 1 -- 172.21.15.19:0/4070592916 wait complete. 2024-04-25T10:05:48.495 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:05:48.495 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:48.495 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T10:05:48.495 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:05:48.495 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:48.495 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:05:48.495 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:05:48.495 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:48.495 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:48.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.496 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:48.512 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41997sec preferred_lft 41997sec 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:05:48.513 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::34e0:a4ff:fe39:5ba1/64 scope link 2024-04-25T10:05:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:48.514 INFO:teuthology.orchestra.run.smithi019.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 5e:bc:f8:f3:7b:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5cbc:f8ff:fef3:7b8f/64 scope link 2024-04-25T10:05:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:48.514 INFO:teuthology.orchestra.run.smithi019.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:25:6c:bd:81:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c425:6cff:febd:81f2/64 scope link 2024-04-25T10:05:48.514 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:48.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.515 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:48.575 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T10:05:48.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.577 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T10:05:48.634 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T10:05:48.634 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:48.635 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:05:48.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:05:48.635 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:48.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.636 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-04-25T10:05:48.636 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T10:05:48.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:05:48.637 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:05:48.637 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:05:48.637 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T10:05:48.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.720 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T10:05:48.736 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:05:48.736 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:05:48.736 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:48.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.736 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T10:05:48.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:48.803 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:48.859 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:48.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:48.859 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:48.914 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:05:48.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:05:48.996 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:05:48.997 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:05:49.004 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:05:49.004 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:05:49.004 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-04-25T10:05:49.004 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@342f98bd-d5a6-4934-bb27-7f8bea67a3be.cephfs=/ 2024-04-25T10:05:49.004 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-04-25T10:05:49.005 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:05:49.005 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:05:49.005 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:05:49.005 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:05:49.005 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:05:49.006 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:05:49.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:49.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:49.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:49.072 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:49.134 INFO:teuthology.orchestra.run.smithi019.stdout:client11748 2024-04-25T10:05:49.135 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:49.135 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:49.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:49.190 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent) 2024-04-25T10:05:49.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:49.251 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/child) 2024-04-25T10:05:49.309 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:49.309 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:49.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:49.365 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/parentfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:05:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records in 2024-04-25T10:05:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records out 2024-04-25T10:05:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0332423 s, 189 MB/s 2024-04-25T10:05:49.454 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:49.454 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:49.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:49.470 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/child/childfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:05:49.559 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records in 2024-04-25T10:05:49.559 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records out 2024-04-25T10:05:49.559 INFO:teuthology.orchestra.run.smithi019.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0328534 s, 192 MB/s 2024-04-25T10:05:49.561 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:05:49.563 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:05:49.603 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-04-25T10:05:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-04-25T10:05:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:05:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-04-25T10:05:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on skipped message = False 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:05:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 20 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 20 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:05:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 10 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T10:05:49.610 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-04-25T10:05:49.611 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:05:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: mon op complaint time = 120 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: client mount timeout = 600 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = "" 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T10:05:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout:[mds] 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds = 20 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds balancer = 20 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: mds cache max size = 1000 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug frag = True 2024-04-25T10:05:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug scatterstat = False 2024-04-25T10:05:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: mds log max segments = 1 2024-04-25T10:05:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: mds op complaint time = 180 2024-04-25T10:05:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: mds verify scatter = False 2024-04-25T10:05:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T10:05:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T10:05:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-04-25T10:05:49.616 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-04-25T10:05:49.617 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.19,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.619 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-04-25T10:05:49.620 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-04-25T10:05:49.621 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-04-25T10:05:49.622 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:05:49.623 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-04-25T10:05:49.623 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T10:05:49.623 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T10:05:49.623 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:49.623 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:49.623 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-04-25T10:05:49.623 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.623 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-04-25T10:05:49.623 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:05:49.624 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:05:49.625 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.625 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:05:49.625 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:05:49.625 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:05:49.625 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.625 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-04-25T10:05:49.625 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:05:49.625 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.625 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:05:49.625 INFO:teuthology.orchestra.run.smithi112.stdout: mon op complaint time = 120 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: client mount timeout = 600 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T10:05:49.626 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds = 20 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds balancer = 20 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout: mds cache max size = 1000 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug frag = True 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug scatterstat = False 2024-04-25T10:05:49.627 INFO:teuthology.orchestra.run.smithi112.stdout: mds log max segments = 1 2024-04-25T10:05:49.628 INFO:teuthology.orchestra.run.smithi112.stdout: mds op complaint time = 180 2024-04-25T10:05:49.628 INFO:teuthology.orchestra.run.smithi112.stdout: mds verify scatter = False 2024-04-25T10:05:49.628 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T10:05:49.628 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T10:05:49.628 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T10:05:49.630 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:05:49.671 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:05:49.711 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-04-25T10:05:49.711 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-04-25T10:05:49.711 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:05:49.711 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-04-25T10:05:49.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:05:49.713 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on skipped message = False 2024-04-25T10:05:49.714 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-04-25T10:05:49.715 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.716 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 20 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 20 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 10 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T10:05:49.717 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T10:05:49.718 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:05:49.719 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: mon op complaint time = 120 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:05:49.720 INFO:teuthology.orchestra.run.smithi019.stdout: client mount timeout = 600 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = "" 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout:[mds] 2024-04-25T10:05:49.721 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds = 20 2024-04-25T10:05:49.722 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds balancer = 20 2024-04-25T10:05:49.722 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T10:05:49.722 INFO:teuthology.orchestra.run.smithi019.stdout: mds cache max size = 1000 2024-04-25T10:05:49.722 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug frag = True 2024-04-25T10:05:49.722 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug scatterstat = False 2024-04-25T10:05:49.722 INFO:teuthology.orchestra.run.smithi019.stdout: mds log max segments = 1 2024-04-25T10:05:49.722 INFO:teuthology.orchestra.run.smithi019.stdout: mds op complaint time = 180 2024-04-25T10:05:49.722 INFO:teuthology.orchestra.run.smithi019.stdout: mds verify scatter = False 2024-04-25T10:05:49.722 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T10:05:49.722 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T10:05:49.722 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T10:05:49.723 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-04-25T10:05:49.723 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-04-25T10:05:49.723 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:05:49.723 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-04-25T10:05:49.723 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.723 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-04-25T10:05:49.723 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.723 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-04-25T10:05:49.724 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.725 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.19,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 342f98bd-d5a6-4934-bb27-7f8bea67a3be 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.726 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.727 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2024-04-25T10:05:49.728 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T10:05:49.729 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-04-25T10:05:49.730 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:05:49.731 INFO:teuthology.orchestra.run.smithi112.stdout: mon op complaint time = 120 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: client mount timeout = 600 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T10:05:49.732 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds = 20 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds balancer = 20 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout: mds cache max size = 1000 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug frag = True 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug scatterstat = False 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout: mds log max segments = 1 2024-04-25T10:05:49.733 INFO:teuthology.orchestra.run.smithi112.stdout: mds op complaint time = 180 2024-04-25T10:05:49.734 INFO:teuthology.orchestra.run.smithi112.stdout: mds verify scatter = False 2024-04-25T10:05:49.734 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T10:05:49.734 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T10:05:49.734 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T10:05:49.735 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:49.735 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:49.791 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:05:49.791 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:49.847 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.982+0000 7fc7159d5640 1 Processor -- start 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.982+0000 7fc7159d5640 1 -- start start 2024-04-25T10:05:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.983+0000 7fc7159d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7100af720 0x7fc7100afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.983+0000 7fc7159d5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71006fe00 con 0x7fc710158220 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.983+0000 7fc7159d5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7100700e0 con 0x7fc7100d02f0 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.983+0000 7fc7159d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7100b3670 con 0x7fc7100af720 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.983+0000 7fc70effd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc710158220 0x7fc71015ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42446/0 (socket says 172.21.15.19:42446) 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.983+0000 7fc70effd640 1 -- 172.21.15.19:0/1852753132 learned_addr learned my addr 172.21.15.19:0/1852753132 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70e7fc640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7100af720 0x7fc7100afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181248346 0 0) 0x7fc71006fe00 con 0x7fc710158220 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f8003520 con 0x7fc710158220 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 86165859 0 0) 0x7fc6f8003520 con 0x7fc710158220 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7100af720 msgr2=0x7fc7100afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7100af720 0x7fc7100afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 >> v1:172.21.15.112:6789/0 conn(0x7fc7100d02f0 legacy=0x7fc7100d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7100703c0 con 0x7fc710158220 2024-04-25T10:05:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fc708001670 con 0x7fc710158220 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 >> v1:172.21.15.19:6789/0 conn(0x7fc710158220 legacy=0x7fc71015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7100af720 0x7fc6f80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f8003fb0 0x7fc6f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f8004d40 0x7fc6f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f800bad0 con 0x7fc6f8004d40 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f800bc80 con 0x7fc6f8003fb0 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f800be80 con 0x7fc7100af720 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70effd640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7100af720 0x7fc6f80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70f7fe640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f8004d40 0x7fc6f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.984+0000 7fc70e7fc640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f8003fb0 0x7fc6f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.985+0000 7fc70f7fe640 1 -- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7100af720 msgr2=0x7fc6f80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.985+0000 7fc70f7fe640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7100af720 0x7fc6f80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.985+0000 7fc70f7fe640 1 -- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f8003fb0 msgr2=0x7fc6f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.985+0000 7fc70f7fe640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f8003fb0 0x7fc6f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.985+0000 7fc70f7fe640 1 -- 172.21.15.19:0/1852753132 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f8003520 con 0x7fc6f8004d40 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.985+0000 7fc70e7fc640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f8003fb0 0x7fc6f8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.985+0000 7fc70f7fe640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f8004d40 0x7fc6f80051c0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fc700001e50 tx=0x7fc6f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39da3121e0375878 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.985+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700001a70 con 0x7fc6f8004d40 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.985+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc700003970 con 0x7fc6f8004d40 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.985+0000 7fc70dffb640 1 -- 172.21.15.19:0/1852753132 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700003d60 con 0x7fc6f8004d40 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.986+0000 7fc7159d5640 1 -- 172.21.15.19:0/1852753132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f8004d40 msgr2=0x7fc6f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.986+0000 7fc7159d5640 1 --2- 172.21.15.19:0/1852753132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6f8004d40 0x7fc6f80051c0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fc700001e50 tx=0x7fc6f8005830 comp rx=0 tx=0).stop 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.986+0000 7fc70f7fe640 1 -- 172.21.15.19:0/1852753132 reap_dead start 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.986+0000 7fc7159d5640 1 -- 172.21.15.19:0/1852753132 shutdown_connections 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.986+0000 7fc7159d5640 1 -- 172.21.15.19:0/1852753132 >> 172.21.15.19:0/1852753132 conn(0x7fc7100b2370 msgr2=0x7fc7100b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.986+0000 7fc7159d5640 1 -- 172.21.15.19:0/1852753132 shutdown_connections 2024-04-25T10:05:49.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.986+0000 7fc7159d5640 1 -- 172.21.15.19:0/1852753132 wait complete. 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.987+0000 7fc7159d5640 1 Processor -- start 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.987+0000 7fc7159d5640 1 -- start start 2024-04-25T10:05:49.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.988+0000 7fc7159d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6f8004d40 0x7fc71014c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.988+0000 7fc7159d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7100b0680 0x7fc7100cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.988+0000 7fc7159d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7100cb8c0 0x7fc7100ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.988+0000 7fc7159d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc710158600 con 0x7fc7100b0680 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.988+0000 7fc7159d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc710070a10 con 0x7fc7100cb8c0 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.988+0000 7fc7159d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc710072960 con 0x7fc6f8004d40 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.988+0000 7fc70e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7100b0680 0x7fc7100cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.988+0000 7fc70e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7100b0680 0x7fc7100cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46820/0 (socket says 172.21.15.19:46820) 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.988+0000 7fc70e7fc640 1 -- 172.21.15.19:0/3691706922 learned_addr learned my addr 172.21.15.19:0/3691706922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.988+0000 7fc70effd640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6f8004d40 0x7fc71014c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.988+0000 7fc70f7fe640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7100cb8c0 0x7fc7100ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.989+0000 7fc70e7fc640 1 -- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6f8004d40 msgr2=0x7fc71014c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.989+0000 7fc70e7fc640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6f8004d40 0x7fc71014c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.989+0000 7fc70e7fc640 1 -- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7100cb8c0 msgr2=0x7fc7100ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.989+0000 7fc70e7fc640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7100cb8c0 0x7fc7100ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.989+0000 7fc70e7fc640 1 -- 172.21.15.19:0/3691706922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7100cbdd0 con 0x7fc7100b0680 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.989+0000 7fc70e7fc640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7100b0680 0x7fc7100cb380 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fc704011cd0 tx=0x7fc70400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.989+0000 7fc6f7fff640 1 -- 172.21.15.19:0/3691706922 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70400fd20 con 0x7fc7100b0680 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.989+0000 7fc6f7fff640 1 -- 172.21.15.19:0/3691706922 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc704014440 con 0x7fc7100b0680 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.989+0000 7fc7159d5640 1 -- 172.21.15.19:0/3691706922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7100afba0 con 0x7fc7100b0680 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.989+0000 7fc6f7fff640 1 -- 172.21.15.19:0/3691706922 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704014830 con 0x7fc7100b0680 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.989+0000 7fc7159d5640 1 -- 172.21.15.19:0/3691706922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7100b0160 con 0x7fc7100b0680 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.990+0000 7fc7159d5640 1 -- 172.21.15.19:0/3691706922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc710073990 con 0x7fc7100b0680 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.990+0000 7fc6f7fff640 1 -- 172.21.15.19:0/3691706922 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc704012040 con 0x7fc7100b0680 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.991+0000 7fc6f7fff640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc6fc06b500 0x7fc6fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.991+0000 7fc6f7fff640 1 -- 172.21.15.19:0/3691706922 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc70408ab90 con 0x7fc7100b0680 2024-04-25T10:05:49.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.991+0000 7fc70effd640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc6fc06b500 0x7fc6fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.992+0000 7fc70effd640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc6fc06b500 0x7fc6fc06d9c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc710156e70 tx=0x7fc708009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:49.995+0000 7fc6f7fff640 1 -- 172.21.15.19:0/3691706922 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc704019030 con 0x7fc7100b0680 2024-04-25T10:05:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.187+0000 7fc7159d5640 1 -- 172.21.15.19:0/3691706922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7101563a0 con 0x7fc7100b0680 2024-04-25T10:05:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.188+0000 7fc6f7fff640 1 -- 172.21.15.19:0/3691706922 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+17068 (secure 0 0 0) 0x7fc704059250 con 0x7fc7100b0680 2024-04-25T10:05:50.193 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:50.194 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":191,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:50.105747+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:36.700326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:37.402558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":6,"to":0}]},{"pgid":"29.b","mappings":[{"from":6,"to":0}]},{"pgid":"29.13","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:50.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 -- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc6fc06b500 msgr2=0x7fc6fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc6fc06b500 0x7fc6fc06d9c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc710156e70 tx=0x7fc708009040 comp rx=0 tx=0).stop 2024-04-25T10:05:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 -- 172.21.15.19:0/3691706922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7100b0680 msgr2=0x7fc7100cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7100b0680 0x7fc7100cb380 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fc704011cd0 tx=0x7fc70400f610 comp rx=0 tx=0).stop 2024-04-25T10:05:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 -- 172.21.15.19:0/3691706922 shutdown_connections 2024-04-25T10:05:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fc6fc06b500 0x7fc6fc06d9c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7100cb8c0 0x7fc7100ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7100b0680 0x7fc7100cb380 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 --2- 172.21.15.19:0/3691706922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc6f8004d40 0x7fc71014c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 -- 172.21.15.19:0/3691706922 >> 172.21.15.19:0/3691706922 conn(0x7fc7100b2370 msgr2=0x7fc7101570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 -- 172.21.15.19:0/3691706922 shutdown_connections 2024-04-25T10:05:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.190+0000 7fc7159d5640 1 -- 172.21.15.19:0/3691706922 wait complete. 2024-04-25T10:05:50.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:36.700326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:37.402558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:50.207 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.305+0000 7f5e6334f640 1 Processor -- start 2024-04-25T10:05:50.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.306+0000 7f5e6334f640 1 -- start start 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.306+0000 7f5e6334f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c070dc0 0x7f5e5c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.306+0000 7f5e6334f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5e5c0711a0 0x7f5e5c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.306+0000 7f5e6334f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5c0d1770 con 0x7f5e5c0b4830 2024-04-25T10:05:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.306+0000 7f5e6334f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c1622b0 con 0x7f5e5c0711a0 2024-04-25T10:05:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.306+0000 7f5e6334f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0b10f0 con 0x7f5e5c070dc0 2024-04-25T10:05:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.307+0000 7f5e610eb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c070dc0 0x7f5e5c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.307+0000 7f5e608ea640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5e5c0711a0 0x7f5e5c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.307+0000 7f5e610eb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c070dc0 0x7f5e5c06ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:59512/0 (socket says 172.21.15.19:59512) 2024-04-25T10:05:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.307+0000 7f5e610eb640 1 -- 172.21.15.19:0/3315208002 learned_addr learned my addr 172.21.15.19:0/3315208002 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.307+0000 7f5e610eb640 1 -- 172.21.15.19:0/3315208002 >> v2:172.21.15.112:3300/0 conn(0x7f5e5c0711a0 msgr2=0x7f5e5c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.307+0000 7f5e610eb640 1 --2- 172.21.15.19:0/3315208002 >> v2:172.21.15.112:3300/0 conn(0x7f5e5c0711a0 0x7f5e5c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.307+0000 7f5e610eb640 1 -- 172.21.15.19:0/3315208002 >> v1:172.21.15.112:6789/0 conn(0x7f5e5c0b4830 legacy=0x7f5e5c0b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.307+0000 7f5e610eb640 1 -- 172.21.15.19:0/3315208002 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c162740 con 0x7f5e5c070dc0 2024-04-25T10:05:50.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.308+0000 7f5e610eb640 1 --2- 172.21.15.19:0/3315208002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c070dc0 0x7f5e5c06ccd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009870 tx=0x7f5e5802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56a8229f96530017 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.308+0000 7f5e53fff640 1 -- 172.21.15.19:0/3315208002 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58037070 con 0x7f5e5c070dc0 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.308+0000 7f5e53fff640 1 -- 172.21.15.19:0/3315208002 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e58032440 con 0x7f5e5c070dc0 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.308+0000 7f5e53fff640 1 -- 172.21.15.19:0/3315208002 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58032810 con 0x7f5e5c070dc0 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.309+0000 7f5e6334f640 1 -- 172.21.15.19:0/3315208002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c070dc0 msgr2=0x7f5e5c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.309+0000 7f5e6334f640 1 --2- 172.21.15.19:0/3315208002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c070dc0 0x7f5e5c06ccd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009870 tx=0x7f5e5802a130 comp rx=0 tx=0).stop 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.309+0000 7f5e6334f640 1 -- 172.21.15.19:0/3315208002 shutdown_connections 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.309+0000 7f5e6334f640 1 --2- 172.21.15.19:0/3315208002 >> v2:172.21.15.112:3300/0 conn(0x7f5e5c0711a0 0x7f5e5c0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.309+0000 7f5e6334f640 1 --2- 172.21.15.19:0/3315208002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c070dc0 0x7f5e5c06ccd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.309+0000 7f5e6334f640 1 -- 172.21.15.19:0/3315208002 >> 172.21.15.19:0/3315208002 conn(0x7f5e5c0b3530 msgr2=0x7f5e5c0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.309+0000 7f5e6334f640 1 -- 172.21.15.19:0/3315208002 shutdown_connections 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.309+0000 7f5e6334f640 1 -- 172.21.15.19:0/3315208002 wait complete. 2024-04-25T10:05:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.310+0000 7f5e6334f640 1 Processor -- start 2024-04-25T10:05:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.310+0000 7f5e6334f640 1 -- start start 2024-04-25T10:05:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e6334f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e5c070dc0 0x7f5e5c152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e6334f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c0711a0 0x7f5e5c160030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e6334f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e5c0b4830 0x7f5e5c1f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e6334f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c161f40 con 0x7f5e5c0b4830 2024-04-25T10:05:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e6334f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c161c60 con 0x7f5e5c070dc0 2024-04-25T10:05:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e6334f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c071600 con 0x7f5e5c0711a0 2024-04-25T10:05:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e618ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e5c0b4830 0x7f5e5c1f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e618ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e5c0b4830 0x7f5e5c1f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46832/0 (socket says 172.21.15.19:46832) 2024-04-25T10:05:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e618ec640 1 -- 172.21.15.19:0/345063103 learned_addr learned my addr 172.21.15.19:0/345063103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e608ea640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c0711a0 0x7f5e5c160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e610eb640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e5c070dc0 0x7f5e5c152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e618ec640 1 -- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c0711a0 msgr2=0x7f5e5c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e618ec640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c0711a0 0x7f5e5c160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e618ec640 1 -- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e5c070dc0 msgr2=0x7f5e5c152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e618ec640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e5c070dc0 0x7f5e5c152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e618ec640 1 -- 172.21.15.19:0/345063103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c0cabb0 con 0x7f5e5c0b4830 2024-04-25T10:05:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e610eb640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e5c070dc0 0x7f5e5c152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e618ec640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e5c0b4830 0x7f5e5c1f4a10 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400c730 tx=0x7f5e5400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e51ffb640 1 -- 172.21.15.19:0/345063103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54016020 con 0x7f5e5c0b4830 2024-04-25T10:05:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e6334f640 1 -- 172.21.15.19:0/345063103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5c0cae70 con 0x7f5e5c0b4830 2024-04-25T10:05:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e51ffb640 1 -- 172.21.15.19:0/345063103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54006d50 con 0x7f5e5c0b4830 2024-04-25T10:05:50.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.311+0000 7f5e51ffb640 1 -- 172.21.15.19:0/345063103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54003bb0 con 0x7f5e5c0b4830 2024-04-25T10:05:50.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.312+0000 7f5e6334f640 1 -- 172.21.15.19:0/345063103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5c0cb430 con 0x7f5e5c0b4830 2024-04-25T10:05:50.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.313+0000 7f5e6334f640 1 -- 172.21.15.19:0/345063103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5c06da80 con 0x7f5e5c0b4830 2024-04-25T10:05:50.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.313+0000 7f5e51ffb640 1 -- 172.21.15.19:0/345063103 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e540050d0 con 0x7f5e5c0b4830 2024-04-25T10:05:50.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.314+0000 7f5e51ffb640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e4006b4d0 0x7f5e4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.314+0000 7f5e51ffb640 1 -- 172.21.15.19:0/345063103 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f5e54089fe0 con 0x7f5e5c0b4830 2024-04-25T10:05:50.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.314+0000 7f5e610eb640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e4006b4d0 0x7f5e4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.314+0000 7f5e610eb640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e4006b4d0 0x7f5e4006d990 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5e58034000 tx=0x7f5e5802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.318+0000 7f5e51ffb640 1 -- 172.21.15.19:0/345063103 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e5c06da80 con 0x7f5e5c0b4830 2024-04-25T10:05:50.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.499+0000 7f5e6334f640 1 -- 172.21.15.19:0/345063103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e5c073a00 con 0x7f5e5c0b4830 2024-04-25T10:05:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.499+0000 7f5e51ffb640 1 -- 172.21.15.19:0/345063103 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v191) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e54020020 con 0x7f5e5c0b4830 2024-04-25T10:05:50.505 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:05:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 -- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e4006b4d0 msgr2=0x7f5e4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e4006b4d0 0x7f5e4006d990 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5e58034000 tx=0x7f5e5802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 -- 172.21.15.19:0/345063103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e5c0b4830 msgr2=0x7f5e5c1f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e5c0b4830 0x7f5e5c1f4a10 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400c730 tx=0x7f5e5400cb30 comp rx=0 tx=0).stop 2024-04-25T10:05:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 -- 172.21.15.19:0/345063103 shutdown_connections 2024-04-25T10:05:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5e4006b4d0 0x7f5e4006d990 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e5c0b4830 0x7f5e5c1f4a10 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e5c0711a0 0x7f5e5c160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 --2- 172.21.15.19:0/345063103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e5c070dc0 0x7f5e5c152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 -- 172.21.15.19:0/345063103 >> 172.21.15.19:0/345063103 conn(0x7f5e5c0b3530 msgr2=0x7f5e5c0ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 -- 172.21.15.19:0/345063103 shutdown_connections 2024-04-25T10:05:50.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.501+0000 7f5e6334f640 1 -- 172.21.15.19:0/345063103 wait complete. 2024-04-25T10:05:50.517 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:50.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.616+0000 7f04aaf4c640 1 Processor -- start 2024-04-25T10:05:50.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.616+0000 7f04aaf4c640 1 -- start start 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.616+0000 7f04aaf4c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04a40c6010 0x7f04a40ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.616+0000 7f04aaf4c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f04a40cd520 0x7f04a40cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.616+0000 7f04aaf4c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a40c6940 con 0x7f04a415a9f0 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.616+0000 7f04aaf4c640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a40b0020 con 0x7f04a40c6010 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.616+0000 7f04aaf4c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a40cb5a0 con 0x7f04a40cd520 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.617+0000 7f04a3fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04a40c6010 0x7f04a40ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.617+0000 7f04a8ce8640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f04a415a9f0 0x7f04a40cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42454/0 (socket says 172.21.15.19:42454) 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.617+0000 7f04a8ce8640 1 -- 172.21.15.19:0/1828870847 learned_addr learned my addr 172.21.15.19:0/1828870847 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.617+0000 7f04a94e9640 1 --2- 172.21.15.19:0/1828870847 >> v2:172.21.15.112:3300/0 conn(0x7f04a40cd520 0x7f04a40cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.617+0000 7f04a3fff640 1 -- 172.21.15.19:0/1828870847 >> v2:172.21.15.112:3300/0 conn(0x7f04a40cd520 msgr2=0x7f04a40cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.617+0000 7f04a3fff640 1 --2- 172.21.15.19:0/1828870847 >> v2:172.21.15.112:3300/0 conn(0x7f04a40cd520 0x7f04a40cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.617+0000 7f04a3fff640 1 -- 172.21.15.19:0/1828870847 >> v1:172.21.15.19:6789/0 conn(0x7f04a415a9f0 legacy=0x7f04a40cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.617+0000 7f04a3fff640 1 -- 172.21.15.19:0/1828870847 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a415add0 con 0x7f04a40c6010 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.617+0000 7f04a37fe640 1 -- 172.21.15.19:0/1828870847 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098595358 0 0) 0x7f049c001490 con 0x7f04a415a9f0 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.617+0000 7f04a94e9640 1 --2- 172.21.15.19:0/1828870847 >> v2:172.21.15.112:3300/0 conn(0x7f04a40cd520 0x7f04a40cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04a3fff640 1 --2- 172.21.15.19:0/1828870847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a40c6010 0x7f04a40ccfe0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f04a40afd90 tx=0x7f048c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb944697b93257b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04a37fe640 1 -- 172.21.15.19:0/1828870847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c037070 con 0x7f04a40c6010 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04a37fe640 1 -- 172.21.15.19:0/1828870847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c032440 con 0x7f04a40c6010 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04a37fe640 1 -- 172.21.15.19:0/1828870847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c032810 con 0x7f04a40c6010 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04aaf4c640 1 -- 172.21.15.19:0/1828870847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a40c6010 msgr2=0x7f04a40ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04aaf4c640 1 --2- 172.21.15.19:0/1828870847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a40c6010 0x7f04a40ccfe0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f04a40afd90 tx=0x7f048c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04aaf4c640 1 -- 172.21.15.19:0/1828870847 shutdown_connections 2024-04-25T10:05:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04aaf4c640 1 --2- 172.21.15.19:0/1828870847 >> v2:172.21.15.112:3300/0 conn(0x7f04a40cd520 0x7f04a40cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04aaf4c640 1 --2- 172.21.15.19:0/1828870847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a40c6010 0x7f04a40ccfe0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04aaf4c640 1 -- 172.21.15.19:0/1828870847 >> 172.21.15.19:0/1828870847 conn(0x7f04a40b2390 msgr2=0x7f04a40b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04aaf4c640 1 -- 172.21.15.19:0/1828870847 shutdown_connections 2024-04-25T10:05:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.618+0000 7f04aaf4c640 1 -- 172.21.15.19:0/1828870847 wait complete. 2024-04-25T10:05:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.620+0000 7f04aaf4c640 1 Processor -- start 2024-04-25T10:05:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.620+0000 7f04aaf4c640 1 -- start start 2024-04-25T10:05:50.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.620+0000 7f04aaf4c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a40c6010 0x7f04a40c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.620+0000 7f04aaf4c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04a40cd520 0x7f04a41f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04aaf4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04a415a9f0 0x7f04a40c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04aaf4c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40cb230 con 0x7f04a40c6010 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04aaf4c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40caf50 con 0x7f04a415a9f0 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04aaf4c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40cbf70 con 0x7f04a40cd520 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a8ce8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a40c6010 0x7f04a40c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a8ce8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a40c6010 0x7f04a40c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46856/0 (socket says 172.21.15.19:46856) 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a8ce8640 1 -- 172.21.15.19:0/303959777 learned_addr learned my addr 172.21.15.19:0/303959777 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a8ce8640 1 -- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04a40cd520 msgr2=0x7f04a41f12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a8ce8640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04a40cd520 0x7f04a41f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a8ce8640 1 -- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04a415a9f0 msgr2=0x7f04a40c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a8ce8640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04a415a9f0 0x7f04a40c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a8ce8640 1 -- 172.21.15.19:0/303959777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a40c8600 con 0x7f04a40c6010 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a8ce8640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a40c6010 0x7f04a40c96a0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f049c00c7c0 tx=0x7f049c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a17fa640 1 -- 172.21.15.19:0/303959777 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c018070 con 0x7f04a40c6010 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a17fa640 1 -- 172.21.15.19:0/303959777 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c00a640 con 0x7f04a40c6010 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04aaf4c640 1 -- 172.21.15.19:0/303959777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a40c88e0 con 0x7f04a40c6010 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04a17fa640 1 -- 172.21.15.19:0/303959777 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c00aa30 con 0x7f04a40c6010 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.621+0000 7f04aaf4c640 1 -- 172.21.15.19:0/303959777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a40c8ea0 con 0x7f04a40c6010 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.622+0000 7f04aaf4c640 1 -- 172.21.15.19:0/303959777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a4073a00 con 0x7f04a40c6010 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.623+0000 7f04a17fa640 1 -- 172.21.15.19:0/303959777 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f049c01b070 con 0x7f04a40c6010 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.623+0000 7f04a17fa640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f048406b4d0 0x7f048406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.623+0000 7f04a17fa640 1 -- 172.21.15.19:0/303959777 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f049c089a30 con 0x7f04a40c6010 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.624+0000 7f04a3fff640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f048406b4d0 0x7f048406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.624+0000 7f04a3fff640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f048406b4d0 0x7f048406d990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f048c034000 tx=0x7f048c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.627+0000 7f04a17fa640 1 -- 172.21.15.19:0/303959777 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04a40c6940 con 0x7f04a40c6010 2024-04-25T10:05:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.806+0000 7f04aaf4c640 1 -- 172.21.15.19:0/303959777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04a41f17e0 con 0x7f04a40c6010 2024-04-25T10:05:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.806+0000 7f04a17fa640 1 -- 172.21.15.19:0/303959777 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v191) v1 ==== 104+0+11 (secure 0 0 0) 0x7f049c0586b0 con 0x7f04a40c6010 2024-04-25T10:05:50.812 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:05:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 -- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f048406b4d0 msgr2=0x7f048406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f048406b4d0 0x7f048406d990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f048c034000 tx=0x7f048c02c000 comp rx=0 tx=0).stop 2024-04-25T10:05:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 -- 172.21.15.19:0/303959777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a40c6010 msgr2=0x7f04a40c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a40c6010 0x7f04a40c96a0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f049c00c7c0 tx=0x7f049c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:05:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 -- 172.21.15.19:0/303959777 shutdown_connections 2024-04-25T10:05:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f048406b4d0 0x7f048406d990 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04a415a9f0 0x7f04a40c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04a40cd520 0x7f04a41f12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 --2- 172.21.15.19:0/303959777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a40c6010 0x7f04a40c96a0 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 -- 172.21.15.19:0/303959777 >> 172.21.15.19:0/303959777 conn(0x7f04a40b2390 msgr2=0x7f04a40af610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 -- 172.21.15.19:0/303959777 shutdown_connections 2024-04-25T10:05:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.808+0000 7f04aaf4c640 1 -- 172.21.15.19:0/303959777 wait complete. 2024-04-25T10:05:50.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:50.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.923+0000 7f187ca1f640 1 Processor -- start 2024-04-25T10:05:50.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.923+0000 7f187ca1f640 1 -- start start 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.924+0000 7f187ca1f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f187815b7d0 0x7f18780cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.924+0000 7f187ca1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f187806c9e0 0x7f18780b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.924+0000 7f187ca1f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187806cf50 con 0x7f18780b5620 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.924+0000 7f187ca1f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18780ce530 con 0x7f187815b7d0 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.924+0000 7f187ca1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18780ce810 con 0x7f187806c9e0 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.924+0000 7f187659c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f187815b7d0 0x7f18780cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.924+0000 7f1875d9b640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f18780b5620 0x7f187806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42460/0 (socket says 172.21.15.19:42460) 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.924+0000 7f1875d9b640 1 -- 172.21.15.19:0/2592339292 learned_addr learned my addr 172.21.15.19:0/2592339292 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.924+0000 7f1876d9d640 1 --2- 172.21.15.19:0/2592339292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f187806c9e0 0x7f18780b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.925+0000 7f187659c640 1 -- 172.21.15.19:0/2592339292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f187806c9e0 msgr2=0x7f18780b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.925+0000 7f187659c640 1 --2- 172.21.15.19:0/2592339292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f187806c9e0 0x7f18780b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.925+0000 7f187659c640 1 -- 172.21.15.19:0/2592339292 >> v1:172.21.15.19:6789/0 conn(0x7f18780b5620 legacy=0x7f187806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.925+0000 7f187559a640 1 -- 172.21.15.19:0/2592339292 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607159925 0 0) 0x7f187806cf50 con 0x7f18780b5620 2024-04-25T10:05:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.925+0000 7f187659c640 1 -- 172.21.15.19:0/2592339292 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18780ceaf0 con 0x7f187815b7d0 2024-04-25T10:05:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.925+0000 7f1876d9d640 1 --2- 172.21.15.19:0/2592339292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f187806c9e0 0x7f18780b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.925+0000 7f187659c640 1 --2- 172.21.15.19:0/2592339292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f187815b7d0 0x7f18780cf010 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f18680097a0 tx=0x7f186802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ee9b469e6fa18d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.925+0000 7f187559a640 1 -- 172.21.15.19:0/2592339292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868037070 con 0x7f187815b7d0 2024-04-25T10:05:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.926+0000 7f187559a640 1 -- 172.21.15.19:0/2592339292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1868032440 con 0x7f187815b7d0 2024-04-25T10:05:50.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.926+0000 7f187559a640 1 -- 172.21.15.19:0/2592339292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868032810 con 0x7f187815b7d0 2024-04-25T10:05:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.926+0000 7f187ca1f640 1 -- 172.21.15.19:0/2592339292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f187815b7d0 msgr2=0x7f18780cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.926+0000 7f187ca1f640 1 --2- 172.21.15.19:0/2592339292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f187815b7d0 0x7f18780cf010 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f18680097a0 tx=0x7f186802a180 comp rx=0 tx=0).stop 2024-04-25T10:05:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.926+0000 7f187ca1f640 1 -- 172.21.15.19:0/2592339292 shutdown_connections 2024-04-25T10:05:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.926+0000 7f187ca1f640 1 --2- 172.21.15.19:0/2592339292 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f187806c9e0 0x7f18780b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.926+0000 7f187ca1f640 1 --2- 172.21.15.19:0/2592339292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f187815b7d0 0x7f18780cf010 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.926+0000 7f187ca1f640 1 -- 172.21.15.19:0/2592339292 >> 172.21.15.19:0/2592339292 conn(0x7f18780d0800 msgr2=0x7f18780d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.926+0000 7f187ca1f640 1 -- 172.21.15.19:0/2592339292 shutdown_connections 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.926+0000 7f187ca1f640 1 -- 172.21.15.19:0/2592339292 wait complete. 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.927+0000 7f187ca1f640 1 Processor -- start 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.928+0000 7f187ca1f640 1 -- start start 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.928+0000 7f187ca1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f187806c9e0 0x7f18780c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.928+0000 7f187ca1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18780b5620 0x7f18780c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.928+0000 7f187ca1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f187815b7d0 0x7f18781486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.928+0000 7f187ca1f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18780cf6d0 con 0x7f18780b5620 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.928+0000 7f187ca1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18780cf550 con 0x7f187815b7d0 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.928+0000 7f187ca1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187815bbb0 con 0x7f187806c9e0 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.928+0000 7f1875d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18780b5620 0x7f18780c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.928+0000 7f1875d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18780b5620 0x7f18780c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46880/0 (socket says 172.21.15.19:46880) 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.928+0000 7f1875d9b640 1 -- 172.21.15.19:0/3610493539 learned_addr learned my addr 172.21.15.19:0/3610493539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f187659c640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f187806c9e0 0x7f18780c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f1876d9d640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f187815b7d0 0x7f18781486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f1875d9b640 1 -- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f187806c9e0 msgr2=0x7f18780c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f1875d9b640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f187806c9e0 0x7f18780c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f1875d9b640 1 -- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f187815b7d0 msgr2=0x7f18781486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f1875d9b640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f187815b7d0 0x7f18781486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f1875d9b640 1 -- 172.21.15.19:0/3610493539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18780c9cb0 con 0x7f18780b5620 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f1875d9b640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18780b5620 0x7f18780c9460 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f186c00c460 tx=0x7f186c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f1866ffd640 1 -- 172.21.15.19:0/3610493539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c005ce0 con 0x7f18780b5620 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f1866ffd640 1 -- 172.21.15.19:0/3610493539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186c007070 con 0x7f18780b5620 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f1866ffd640 1 -- 172.21.15.19:0/3610493539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c007460 con 0x7f18780b5620 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f187ca1f640 1 -- 172.21.15.19:0/3610493539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18780c9f90 con 0x7f18780b5620 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.929+0000 7f187ca1f640 1 -- 172.21.15.19:0/3610493539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18780ca550 con 0x7f18780b5620 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.930+0000 7f187ca1f640 1 -- 172.21.15.19:0/3610493539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1878073a00 con 0x7f18780b5620 2024-04-25T10:05:50.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.931+0000 7f1866ffd640 1 -- 172.21.15.19:0/3610493539 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f186c007600 con 0x7f18780b5620 2024-04-25T10:05:50.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.931+0000 7f1866ffd640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f185806b4d0 0x7f185806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.931+0000 7f1866ffd640 1 -- 172.21.15.19:0/3610493539 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f186c08a0e0 con 0x7f18780b5620 2024-04-25T10:05:50.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.931+0000 7f187659c640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f185806b4d0 0x7f185806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.932+0000 7f187659c640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f185806b4d0 0x7f185806d990 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1868034000 tx=0x7f186802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:50.935+0000 7f1866ffd640 1 -- 172.21.15.19:0/3610493539 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f186c058d60 con 0x7f18780b5620 2024-04-25T10:05:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.115+0000 7f187ca1f640 1 -- 172.21.15.19:0/3610493539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f187815f840 con 0x7f18780b5620 2024-04-25T10:05:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.116+0000 7f1866ffd640 1 -- 172.21.15.19:0/3610493539 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v191) v1 ==== 100+0+11 (secure 0 0 0) 0x7f186c058660 con 0x7f18780b5620 2024-04-25T10:05:51.122 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:05:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 -- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f185806b4d0 msgr2=0x7f185806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f185806b4d0 0x7f185806d990 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1868034000 tx=0x7f186802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 -- 172.21.15.19:0/3610493539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18780b5620 msgr2=0x7f18780c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18780b5620 0x7f18780c9460 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f186c00c460 tx=0x7f186c00c920 comp rx=0 tx=0).stop 2024-04-25T10:05:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 -- 172.21.15.19:0/3610493539 shutdown_connections 2024-04-25T10:05:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f185806b4d0 0x7f185806d990 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f187815b7d0 0x7f18781486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18780b5620 0x7f18780c9460 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 --2- 172.21.15.19:0/3610493539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f187806c9e0 0x7f18780c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 -- 172.21.15.19:0/3610493539 >> 172.21.15.19:0/3610493539 conn(0x7f18780d0800 msgr2=0x7f1878160e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 -- 172.21.15.19:0/3610493539 shutdown_connections 2024-04-25T10:05:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.119+0000 7f187ca1f640 1 -- 172.21.15.19:0/3610493539 wait complete. 2024-04-25T10:05:51.136 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:51.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.231+0000 7f71c3b0a640 1 Processor -- start 2024-04-25T10:05:51.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.231+0000 7f71c3b0a640 1 -- start start 2024-04-25T10:05:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c3b0a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71bc0ced80 0x7f71bc0d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c3b0a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f71bc0d1ce0 0x7f71bc0b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c3b0a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71bc0d2120 con 0x7f71bc15dc20 2024-04-25T10:05:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c3b0a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71bc0d23a0 con 0x7f71bc0d1ce0 2024-04-25T10:05:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c3b0a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71bc0d2680 con 0x7f71bc0ced80 2024-04-25T10:05:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c20a7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f71bc0d1ce0 0x7f71bc0b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c18a6640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f71bc15dc20 0x7f71bc0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42468/0 (socket says 172.21.15.19:42468) 2024-04-25T10:05:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c18a6640 1 -- 172.21.15.19:0/4116443183 learned_addr learned my addr 172.21.15.19:0/4116443183 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c20a7640 1 -- 172.21.15.19:0/4116443183 >> v2:172.21.15.112:3300/0 conn(0x7f71bc0ced80 msgr2=0x7f71bc0d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c20a7640 1 --2- 172.21.15.19:0/4116443183 >> v2:172.21.15.112:3300/0 conn(0x7f71bc0ced80 0x7f71bc0d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c20a7640 1 -- 172.21.15.19:0/4116443183 >> v1:172.21.15.19:6789/0 conn(0x7f71bc15dc20 legacy=0x7f71bc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c20a7640 1 -- 172.21.15.19:0/4116443183 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc0d2960 con 0x7f71bc0d1ce0 2024-04-25T10:05:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.232+0000 7f71c08a4640 1 -- 172.21.15.19:0/4116443183 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888099003 0 0) 0x7f71b8001110 con 0x7f71bc15dc20 2024-04-25T10:05:51.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.233+0000 7f71c20a7640 1 --2- 172.21.15.19:0/4116443183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc0d1ce0 0x7f71bc0b9490 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0b61c0 tx=0x7f71b402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fea5c8ea748ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.233+0000 7f71c08a4640 1 -- 172.21.15.19:0/4116443183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4007740 con 0x7f71bc0d1ce0 2024-04-25T10:05:51.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.233+0000 7f71c08a4640 1 -- 172.21.15.19:0/4116443183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b40078e0 con 0x7f71bc0d1ce0 2024-04-25T10:05:51.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.233+0000 7f71c08a4640 1 -- 172.21.15.19:0/4116443183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4007cb0 con 0x7f71bc0d1ce0 2024-04-25T10:05:51.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.234+0000 7f71c3b0a640 1 -- 172.21.15.19:0/4116443183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc0d1ce0 msgr2=0x7f71bc0b9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.234+0000 7f71c3b0a640 1 --2- 172.21.15.19:0/4116443183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc0d1ce0 0x7f71bc0b9490 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0b61c0 tx=0x7f71b402a5f0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.234+0000 7f71c3b0a640 1 -- 172.21.15.19:0/4116443183 shutdown_connections 2024-04-25T10:05:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.234+0000 7f71c3b0a640 1 --2- 172.21.15.19:0/4116443183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc0d1ce0 0x7f71bc0b9490 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.234+0000 7f71c3b0a640 1 --2- 172.21.15.19:0/4116443183 >> v2:172.21.15.112:3300/0 conn(0x7f71bc0ced80 0x7f71bc0d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.234+0000 7f71c3b0a640 1 -- 172.21.15.19:0/4116443183 >> 172.21.15.19:0/4116443183 conn(0x7f71bc061a20 msgr2=0x7f71bc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.234+0000 7f71c3b0a640 1 -- 172.21.15.19:0/4116443183 shutdown_connections 2024-04-25T10:05:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.234+0000 7f71c3b0a640 1 -- 172.21.15.19:0/4116443183 wait complete. 2024-04-25T10:05:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.235+0000 7f71c3b0a640 1 Processor -- start 2024-04-25T10:05:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.235+0000 7f71c3b0a640 1 -- start start 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c3b0a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71bc0ced80 0x7f71bc1f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c3b0a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71bc0d1ce0 0x7f71bc149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c3b0a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc15dc20 0x7f71bc0c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c3b0a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0b6a30 con 0x7f71bc15dc20 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c3b0a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0d2f20 con 0x7f71bc0d1ce0 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c3b0a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0b6bb0 con 0x7f71bc0ced80 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c20a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc15dc20 0x7f71bc0c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c20a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc15dc20 0x7f71bc0c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46900/0 (socket says 172.21.15.19:46900) 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c20a7640 1 -- 172.21.15.19:0/944887664 learned_addr learned my addr 172.21.15.19:0/944887664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c18a6640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71bc0ced80 0x7f71bc1f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c10a5640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71bc0d1ce0 0x7f71bc149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c20a7640 1 -- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71bc0ced80 msgr2=0x7f71bc1f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c20a7640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71bc0ced80 0x7f71bc1f6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c20a7640 1 -- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71bc0d1ce0 msgr2=0x7f71bc149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c20a7640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71bc0d1ce0 0x7f71bc149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c20a7640 1 -- 172.21.15.19:0/944887664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc0c8730 con 0x7f71bc15dc20 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c20a7640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc15dc20 0x7f71bc0c94f0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f71b402abe0 tx=0x7f71b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71b27fc640 1 -- 172.21.15.19:0/944887664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4040070 con 0x7f71bc15dc20 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71b27fc640 1 -- 172.21.15.19:0/944887664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4031c90 con 0x7f71bc15dc20 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71c3b0a640 1 -- 172.21.15.19:0/944887664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc0c8a10 con 0x7f71bc15dc20 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.236+0000 7f71b27fc640 1 -- 172.21.15.19:0/944887664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4045680 con 0x7f71bc15dc20 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.237+0000 7f71c3b0a640 1 -- 172.21.15.19:0/944887664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc0c8fd0 con 0x7f71bc15dc20 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.238+0000 7f71c3b0a640 1 -- 172.21.15.19:0/944887664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71bc073a00 con 0x7f71bc15dc20 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.238+0000 7f71b27fc640 1 -- 172.21.15.19:0/944887664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71b4039040 con 0x7f71bc15dc20 2024-04-25T10:05:51.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.238+0000 7f71b27fc640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a006b4d0 0x7f71a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.239+0000 7f71b27fc640 1 -- 172.21.15.19:0/944887664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f71b40abec0 con 0x7f71bc15dc20 2024-04-25T10:05:51.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.239+0000 7f71c18a6640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a006b4d0 0x7f71a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.239+0000 7f71c18a6640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a006b4d0 0x7f71a006d990 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f71b8000f60 tx=0x7f71b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.243+0000 7f71b27fc640 1 -- 172.21.15.19:0/944887664 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71b407ab60 con 0x7f71bc15dc20 2024-04-25T10:05:51.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.436+0000 7f71c3b0a640 1 -- 172.21.15.19:0/944887664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f71bc0c91a0 con 0x7f71bc15dc20 2024-04-25T10:05:51.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.437+0000 7f71b27fc640 1 -- 172.21.15.19:0/944887664 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v191) v1 ==== 80+0+0 (secure 0 0 0) 0x7f71b407a460 con 0x7f71bc15dc20 2024-04-25T10:05:51.443 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T10:05:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 -- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a006b4d0 msgr2=0x7f71a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a006b4d0 0x7f71a006d990 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f71b8000f60 tx=0x7f71b8009040 comp rx=0 tx=0).stop 2024-04-25T10:05:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 -- 172.21.15.19:0/944887664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc15dc20 msgr2=0x7f71bc0c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc15dc20 0x7f71bc0c94f0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f71b402abe0 tx=0x7f71b4004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 -- 172.21.15.19:0/944887664 shutdown_connections 2024-04-25T10:05:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f71a006b4d0 0x7f71a006d990 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc15dc20 0x7f71bc0c94f0 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71bc0d1ce0 0x7f71bc149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 --2- 172.21.15.19:0/944887664 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71bc0ced80 0x7f71bc1f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 -- 172.21.15.19:0/944887664 >> 172.21.15.19:0/944887664 conn(0x7f71bc061a20 msgr2=0x7f71bc0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 -- 172.21.15.19:0/944887664 shutdown_connections 2024-04-25T10:05:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.439+0000 7f71c3b0a640 1 -- 172.21.15.19:0/944887664 wait complete. 2024-04-25T10:05:51.456 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:05:51.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:51.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:51.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:05:51.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:51.507 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:05:51.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.670+0000 7f9799893640 1 Processor -- start 2024-04-25T10:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.671+0000 7f9799893640 1 -- start start 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.671+0000 7f9799893640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f979406e430 0x7f979406e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.671+0000 7f9799893640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97940b3e10 con 0x7f979415ec20 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.671+0000 7f9799893640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97940b3590 con 0x7f979406abe0 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.671+0000 7f9799893640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97940b3850 con 0x7f979406e430 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f97927fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f979415ec20 0x7f97940b5270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42474/0 (socket says 172.21.15.19:42474) 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f97927fc640 1 -- 172.21.15.19:0/3041415644 learned_addr learned my addr 172.21.15.19:0/3041415644 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f97937fe640 1 --2- 172.21.15.19:0/3041415644 >> v2:172.21.15.112:3300/0 conn(0x7f979406e430 0x7f979406e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493547666 0 0) 0x7f977c001110 con 0x7f979415ec20 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9780003520 con 0x7f979415ec20 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4243586712 0 0) 0x7f9780003520 con 0x7f979415ec20 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 >> v2:172.21.15.112:3300/0 conn(0x7f979406e430 msgr2=0x7f979406e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 --2- 172.21.15.19:0/3041415644 >> v2:172.21.15.112:3300/0 conn(0x7f979406e430 0x7f979406e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 >> v1:172.21.15.112:6789/0 conn(0x7f979406abe0 legacy=0x7f979406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940b3b30 con 0x7f979415ec20 2024-04-25T10:05:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f977c001840 con 0x7f979415ec20 2024-04-25T10:05:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 >> v1:172.21.15.19:6789/0 conn(0x7f979415ec20 legacy=0x7f97940b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97800039b0 0x7f9780003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97800043b0 0x7f9780004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9780005080 0x7f978000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97940b3850 con 0x7f97800039b0 2024-04-25T10:05:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978000bfe0 con 0x7f97800043b0 2024-04-25T10:05:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978000c1e0 con 0x7f9780005080 2024-04-25T10:05:51.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f9792ffd640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97800039b0 0x7f9780003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f97927fc640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97800043b0 0x7f9780004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.672+0000 7f97937fe640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9780005080 0x7f978000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.673+0000 7f9792ffd640 1 -- 172.21.15.19:0/3041415644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9780005080 msgr2=0x7f978000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.673+0000 7f9792ffd640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9780005080 0x7f978000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.673+0000 7f9792ffd640 1 -- 172.21.15.19:0/3041415644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97800043b0 msgr2=0x7f9780004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.673+0000 7f9792ffd640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97800043b0 0x7f9780004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.673+0000 7f9792ffd640 1 -- 172.21.15.19:0/3041415644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9780003520 con 0x7f97800039b0 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.673+0000 7f97937fe640 1 -- 172.21.15.19:0/3041415644 reap_dead start 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.673+0000 7f97927fc640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97800043b0 0x7f9780004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.673+0000 7f9792ffd640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97800039b0 0x7f9780003d90 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f978c001860 tx=0x7f9780005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d665d21693f3aa69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.673+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c037070 con 0x7f97800039b0 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.673+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c02abf0 con 0x7f97800039b0 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.673+0000 7f9791ffb640 1 -- 172.21.15.19:0/3041415644 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c001430 con 0x7f97800039b0 2024-04-25T10:05:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.674+0000 7f9799893640 1 -- 172.21.15.19:0/3041415644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97800039b0 msgr2=0x7f9780003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.674+0000 7f9799893640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97800039b0 0x7f9780003d90 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f978c001860 tx=0x7f9780005830 comp rx=0 tx=0).stop 2024-04-25T10:05:51.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.674+0000 7f9799893640 1 -- 172.21.15.19:0/3041415644 shutdown_connections 2024-04-25T10:05:51.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.674+0000 7f9799893640 1 --2- 172.21.15.19:0/3041415644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97800039b0 0x7f9780003d90 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.674+0000 7f9799893640 1 -- 172.21.15.19:0/3041415644 >> 172.21.15.19:0/3041415644 conn(0x7f97940b5af0 msgr2=0x7f97940b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.674+0000 7f9799893640 1 -- 172.21.15.19:0/3041415644 shutdown_connections 2024-04-25T10:05:51.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.674+0000 7f9799893640 1 -- 172.21.15.19:0/3041415644 wait complete. 2024-04-25T10:05:51.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.675+0000 7f9799893640 1 Processor -- start 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.675+0000 7f9799893640 1 -- start start 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.676+0000 7f9799893640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97800039b0 0x7f9794155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.676+0000 7f9799893640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97941f3550 0x7f9794145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.676+0000 7f9799893640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9794160400 0x7f97941608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.676+0000 7f9799893640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979415f0b0 con 0x7f97941f3550 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.676+0000 7f9799893640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979415e700 con 0x7f97800039b0 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.676+0000 7f9799893640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979415f230 con 0x7f9794160400 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.676+0000 7f97927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97941f3550 0x7f9794145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.676+0000 7f97927fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97941f3550 0x7f9794145080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46918/0 (socket says 172.21.15.19:46918) 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.676+0000 7f97927fc640 1 -- 172.21.15.19:0/696637728 learned_addr learned my addr 172.21.15.19:0/696637728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.676+0000 7f97937fe640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9794160400 0x7f97941608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f9792ffd640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97800039b0 0x7f9794155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f97927fc640 1 -- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9794160400 msgr2=0x7f97941608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f97927fc640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9794160400 0x7f97941608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f97927fc640 1 -- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97800039b0 msgr2=0x7f9794155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f97927fc640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97800039b0 0x7f9794155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f97927fc640 1 -- 172.21.15.19:0/696637728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9794155ad0 con 0x7f97941f3550 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f97937fe640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9794160400 0x7f97941608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f97927fc640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97941f3550 0x7f9794145080 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f977c005e10 tx=0x7f977c007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f978bfff640 1 -- 172.21.15.19:0/696637728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c00e740 con 0x7f97941f3550 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f978bfff640 1 -- 172.21.15.19:0/696637728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c018070 con 0x7f97941f3550 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f978bfff640 1 -- 172.21.15.19:0/696637728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c00eae0 con 0x7f97941f3550 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f9799893640 1 -- 172.21.15.19:0/696637728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97941f39e0 con 0x7f97941f3550 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.677+0000 7f9799893640 1 -- 172.21.15.19:0/696637728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97941f3fa0 con 0x7f97941f3550 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.678+0000 7f9799893640 1 -- 172.21.15.19:0/696637728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979406cc80 con 0x7f97941f3550 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.678+0000 7f978bfff640 1 -- 172.21.15.19:0/696637728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f977c00ec80 con 0x7f97941f3550 2024-04-25T10:05:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.679+0000 7f978bfff640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977406b4d0 0x7f977406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.679+0000 7f978bfff640 1 -- 172.21.15.19:0/696637728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f977c08a380 con 0x7f97941f3550 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.679+0000 7f9792ffd640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977406b4d0 0x7f977406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.680+0000 7f9792ffd640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977406b4d0 0x7f977406d990 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f978c034000 tx=0x7f978c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.683+0000 7f978bfff640 1 -- 172.21.15.19:0/696637728 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97940b3e10 con 0x7f97941f3550 2024-04-25T10:05:51.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.906+0000 7f9799893640 1 -- 172.21.15.19:0/696637728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9794073990 con 0x7f97941f3550 2024-04-25T10:05:51.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.907+0000 7f978bfff640 1 -- 172.21.15.19:0/696637728 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f977c0590c0 con 0x7f97941f3550 2024-04-25T10:05:51.914 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:51.914 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":123,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:38.424174+0000","modified":"2024-04-25T10:05:40.426910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11559":{"gid":11559,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/392508344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":392508344},{"type":"v1","addr":"172.21.15.19:6835","nonce":392508344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:05:51.914 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 123 2024-04-25T10:05:51.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 -- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977406b4d0 msgr2=0x7f977406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977406b4d0 0x7f977406d990 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f978c034000 tx=0x7f978c009030 comp rx=0 tx=0).stop 2024-04-25T10:05:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 -- 172.21.15.19:0/696637728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97941f3550 msgr2=0x7f9794145080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97941f3550 0x7f9794145080 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f977c005e10 tx=0x7f977c007420 comp rx=0 tx=0).stop 2024-04-25T10:05:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 -- 172.21.15.19:0/696637728 shutdown_connections 2024-04-25T10:05:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f977406b4d0 0x7f977406d990 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9794160400 0x7f97941608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97941f3550 0x7f9794145080 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 --2- 172.21.15.19:0/696637728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97800039b0 0x7f9794155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 -- 172.21.15.19:0/696637728 >> 172.21.15.19:0/696637728 conn(0x7f97940b5af0 msgr2=0x7f97940b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 -- 172.21.15.19:0/696637728 shutdown_connections 2024-04-25T10:05:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:51.909+0000 7f9799893640 1 -- 172.21.15.19:0/696637728 wait complete. 2024-04-25T10:05:51.926 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:05:52.051 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-04-25T10:05:52.051 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:05:52.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:05:52.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.147+0000 7f0dceb5c640 1 Processor -- start 2024-04-25T10:05:52.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.147+0000 7f0dceb5c640 1 -- start start 2024-04-25T10:05:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.148+0000 7f0dceb5c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0dc815dc20 0x7f0dc80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.148+0000 7f0dceb5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc8161710 0x7f0dc80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.148+0000 7f0dceb5c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc815e000 con 0x7f0dc80b9330 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.148+0000 7f0dceb5c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc80d1de0 con 0x7f0dc815dc20 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.148+0000 7f0dceb5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc8161c60 con 0x7f0dc8161710 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.148+0000 7f0dcd0f9640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0dc80b9330 0x7f0dc806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42478/0 (socket says 172.21.15.19:42478) 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.148+0000 7f0dcd0f9640 1 -- 172.21.15.19:0/3523854707 learned_addr learned my addr 172.21.15.19:0/3523854707 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.148+0000 7f0dc7fff640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc8161710 0x7f0dc80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.148+0000 7f0dcc8f8640 1 --2- 172.21.15.19:0/3523854707 >> v2:172.21.15.112:3300/0 conn(0x7f0dc815dc20 0x7f0dc80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823746049 0 0) 0x7f0dc815e000 con 0x7f0dc80b9330 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0db8003520 con 0x7f0dc80b9330 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3767774895 0 0) 0x7f0db8003520 con 0x7f0dc80b9330 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc8161710 msgr2=0x7f0dc80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc8161710 0x7f0dc80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 >> v2:172.21.15.112:3300/0 conn(0x7f0dc815dc20 msgr2=0x7f0dc80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 --2- 172.21.15.19:0/3523854707 >> v2:172.21.15.112:3300/0 conn(0x7f0dc815dc20 0x7f0dc80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc8161f40 con 0x7f0dc80b9330 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7f0dc00019f0 con 0x7f0dc80b9330 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 >> v1:172.21.15.19:6789/0 conn(0x7f0dc80b9330 legacy=0x7f0dc806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db80039b0 0x7f0db8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db8004800 0x7f0db8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc8161710 0x7f0db80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc80d1de0 con 0x7f0db80039b0 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc8161c60 con 0x7f0db8004800 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db800b9c0 con 0x7f0dc8161710 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dcc8f8640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db80039b0 0x7f0db8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dcd0f9640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc8161710 0x7f0db80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dc7fff640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db8004800 0x7f0db8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dcc8f8640 1 -- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc8161710 msgr2=0x7f0db80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dcc8f8640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc8161710 0x7f0db80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dcc8f8640 1 -- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db8004800 msgr2=0x7f0db8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dcc8f8640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db8004800 0x7f0db8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dcc8f8640 1 -- 172.21.15.19:0/3523854707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db8003520 con 0x7f0db80039b0 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.149+0000 7f0dcd0f9640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc8161710 0x7f0db80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.150+0000 7f0dcc8f8640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db80039b0 0x7f0db8003dd0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009ac0 tx=0x7f0db8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dcebb3207b6efd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.150+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc036070 con 0x7f0db80039b0 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.150+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc031400 con 0x7f0db80039b0 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.150+0000 7f0dc77fe640 1 -- 172.21.15.19:0/3523854707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc009690 con 0x7f0db80039b0 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.151+0000 7f0dceb5c640 1 -- 172.21.15.19:0/3523854707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db80039b0 msgr2=0x7f0db8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.151+0000 7f0dceb5c640 1 --2- 172.21.15.19:0/3523854707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0db80039b0 0x7f0db8003dd0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009ac0 tx=0x7f0db8005830 comp rx=0 tx=0).stop 2024-04-25T10:05:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.151+0000 7f0dcd0f9640 1 -- 172.21.15.19:0/3523854707 reap_dead start 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.151+0000 7f0dceb5c640 1 -- 172.21.15.19:0/3523854707 shutdown_connections 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.151+0000 7f0dceb5c640 1 -- 172.21.15.19:0/3523854707 >> 172.21.15.19:0/3523854707 conn(0x7f0dc80d0420 msgr2=0x7f0dc80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.151+0000 7f0dceb5c640 1 -- 172.21.15.19:0/3523854707 shutdown_connections 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.151+0000 7f0dceb5c640 1 -- 172.21.15.19:0/3523854707 wait complete. 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.152+0000 7f0dceb5c640 1 Processor -- start 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.152+0000 7f0dceb5c640 1 -- start start 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dceb5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80039b0 0x7f0dc815cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dceb5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc80d2130 0x7f0dc80d2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dceb5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80ac820 0x7f0dc80acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dceb5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc80b5e50 con 0x7f0dc80ac820 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dceb5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc8162500 con 0x7f0db80039b0 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dceb5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc80b5fd0 con 0x7f0dc80d2130 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dcd0f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80ac820 0x7f0dc80acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dcd0f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80ac820 0x7f0dc80acca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46938/0 (socket says 172.21.15.19:46938) 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dcd0f9640 1 -- 172.21.15.19:0/1492858030 learned_addr learned my addr 172.21.15.19:0/1492858030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dc7fff640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc80d2130 0x7f0dc80d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dcc8f8640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80039b0 0x7f0dc815cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dcd0f9640 1 -- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc80d2130 msgr2=0x7f0dc80d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dcd0f9640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc80d2130 0x7f0dc80d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dcd0f9640 1 -- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80039b0 msgr2=0x7f0dc815cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dcd0f9640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80039b0 0x7f0dc815cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dcd0f9640 1 -- 172.21.15.19:0/1492858030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc80ad590 con 0x7f0dc80ac820 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.153+0000 7f0dcd0f9640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80ac820 0x7f0dc80acca0 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00013d0 tx=0x7f0dc000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.154+0000 7f0dc57fa640 1 -- 172.21.15.19:0/1492858030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc000eb60 con 0x7f0dc80ac820 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.154+0000 7f0dc57fa640 1 -- 172.21.15.19:0/1492858030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc000ed00 con 0x7f0dc80ac820 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.154+0000 7f0dc57fa640 1 -- 172.21.15.19:0/1492858030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0014670 con 0x7f0dc80ac820 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.154+0000 7f0dceb5c640 1 -- 172.21.15.19:0/1492858030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc815d440 con 0x7f0dc80ac820 2024-04-25T10:05:52.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.154+0000 7f0dceb5c640 1 -- 172.21.15.19:0/1492858030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc81f71c0 con 0x7f0dc80ac820 2024-04-25T10:05:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.155+0000 7f0dceb5c640 1 -- 172.21.15.19:0/1492858030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc80d2ad0 con 0x7f0dc80ac820 2024-04-25T10:05:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.155+0000 7f0dc57fa640 1 -- 172.21.15.19:0/1492858030 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0dc0014810 con 0x7f0dc80ac820 2024-04-25T10:05:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.156+0000 7f0dc57fa640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0dac06b4d0 0x7f0dac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.156+0000 7f0dc57fa640 1 -- 172.21.15.19:0/1492858030 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f0dc008a100 con 0x7f0dc80ac820 2024-04-25T10:05:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.156+0000 7f0dcc8f8640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0dac06b4d0 0x7f0dac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.156+0000 7f0dcc8f8640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0dac06b4d0 0x7f0dac06d990 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc033000 tx=0x7f0dbc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.160+0000 7f0dc57fa640 1 -- 172.21.15.19:0/1492858030 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0dc0058730 con 0x7f0dc80ac820 2024-04-25T10:05:52.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.376+0000 7f0dceb5c640 1 -- 172.21.15.19:0/1492858030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0dc80d16b0 con 0x7f0dc80ac820 2024-04-25T10:05:52.382 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T10:05:52.376+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:05:52.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.378+0000 7f0dc57fa640 1 -- 172.21.15.19:0/1492858030 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v124) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0dc005c7d0 con 0x7f0dc80ac820 2024-04-25T10:05:52.384 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:05:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.380+0000 7f0dceb5c640 1 -- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0dac06b4d0 msgr2=0x7f0dac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.380+0000 7f0dceb5c640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0dac06b4d0 0x7f0dac06d990 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc033000 tx=0x7f0dbc008030 comp rx=0 tx=0).stop 2024-04-25T10:05:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.380+0000 7f0dceb5c640 1 -- 172.21.15.19:0/1492858030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80ac820 msgr2=0x7f0dc80acca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.380+0000 7f0dceb5c640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80ac820 0x7f0dc80acca0 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00013d0 tx=0x7f0dc000e530 comp rx=0 tx=0).stop 2024-04-25T10:05:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.380+0000 7f0dceb5c640 1 -- 172.21.15.19:0/1492858030 shutdown_connections 2024-04-25T10:05:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.380+0000 7f0dceb5c640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0dac06b4d0 0x7f0dac06d990 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.380+0000 7f0dceb5c640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80ac820 0x7f0dc80acca0 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.380+0000 7f0dceb5c640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0dc80d2130 0x7f0dc80d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.380+0000 7f0dceb5c640 1 --2- 172.21.15.19:0/1492858030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80039b0 0x7f0dc815cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.380+0000 7f0dceb5c640 1 -- 172.21.15.19:0/1492858030 >> 172.21.15.19:0/1492858030 conn(0x7f0dc80d0420 msgr2=0x7f0dc80b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.381+0000 7f0dceb5c640 1 -- 172.21.15.19:0/1492858030 shutdown_connections 2024-04-25T10:05:52.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.381+0000 7f0dceb5c640 1 -- 172.21.15.19:0/1492858030 wait complete. 2024-04-25T10:05:52.389 INFO:tasks.ceph.mds.c.smithi019.stderr: -5415> 2024-04-25T10:05:45.039+0000 7fcd7c22b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:52.399 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapval 10000000000.00000000 parentfile_head - 2024-04-25T10:05:52.419 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:05:52.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.438+0000 7f78d1fd6f40 1 Processor -- start 2024-04-25T10:05:52.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.438+0000 7f78d1fd6f40 1 -- start start 2024-04-25T10:05:52.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78d1fd6f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b8003ab0 0x56141c2449c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78d1fd6f40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56141c315a40 0x56141c243d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78d1fd6f40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56141c2442d0 0x56141c26ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78d1fd6f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141c1ebdf0 con 0x7f78b8003ab0 2024-04-25T10:05:52.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78d1fd6f40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141c1efd50 con 0x56141c2442d0 2024-04-25T10:05:52.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78d1fd6f40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56141c28df30 con 0x56141c315a40 2024-04-25T10:05:52.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78cfad7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b8003ab0 0x56141c2449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78cfad7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b8003ab0 0x56141c2449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46970/0 (socket says 172.21.15.19:46970) 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78cfad7640 1 -- 172.21.15.19:0/1219034910 learned_addr learned my addr 172.21.15.19:0/1219034910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78d02d8640 1 --2- 172.21.15.19:0/1219034910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56141c2442d0 0x56141c26ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78cf2d6640 1 --2- 172.21.15.19:0/1219034910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56141c315a40 0x56141c243d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78cfad7640 1 -- 172.21.15.19:0/1219034910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56141c315a40 msgr2=0x56141c243d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78cfad7640 1 --2- 172.21.15.19:0/1219034910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56141c315a40 0x56141c243d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78cfad7640 1 -- 172.21.15.19:0/1219034910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56141c2442d0 msgr2=0x56141c26ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78cfad7640 1 --2- 172.21.15.19:0/1219034910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56141c2442d0 0x56141c26ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78cfad7640 1 -- 172.21.15.19:0/1219034910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141c274410 con 0x7f78b8003ab0 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78cf2d6640 1 --2- 172.21.15.19:0/1219034910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56141c315a40 0x56141c243d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78cfad7640 1 --2- 172.21.15.19:0/1219034910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b8003ab0 0x56141c2449c0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f78c0001750 tx=0x7f78c0009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a66d42ea07988d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78ce2d4640 1 -- 172.21.15.19:0/1219034910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c000e860 con 0x7f78b8003ab0 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78ce2d4640 1 -- 172.21.15.19:0/1219034910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c0013030 con 0x7f78b8003ab0 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.439+0000 7f78ce2d4640 1 -- 172.21.15.19:0/1219034910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c000ec40 con 0x7f78b8003ab0 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.440+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/1219034910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b8003ab0 msgr2=0x56141c2449c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.440+0000 7f78d1fd6f40 1 --2- 172.21.15.19:0/1219034910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b8003ab0 0x56141c2449c0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f78c0001750 tx=0x7f78c0009a90 comp rx=0 tx=0).stop 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.440+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/1219034910 shutdown_connections 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.440+0000 7f78d1fd6f40 1 --2- 172.21.15.19:0/1219034910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56141c2442d0 0x56141c26ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.440+0000 7f78d1fd6f40 1 --2- 172.21.15.19:0/1219034910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56141c315a40 0x56141c243d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.440+0000 7f78d1fd6f40 1 --2- 172.21.15.19:0/1219034910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78b8003ab0 0x56141c2449c0 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.440+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/1219034910 >> 172.21.15.19:0/1219034910 conn(0x56141c2439b0 msgr2=0x56141c273b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.440+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/1219034910 shutdown_connections 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.440+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/1219034910 wait complete. 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.441+0000 7f78d1fd6f40 1 Processor -- start 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.441+0000 7f78d1fd6f40 1 -- start start 2024-04-25T10:05:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.441+0000 7f78d1fd6f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56141c2442d0 0x56141c3128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.441+0000 7f78d1fd6f40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56141c315a40 0x56141c310ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.441+0000 7f78d1fd6f40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b8003ab0 0x56141c315080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.441+0000 7f78d1fd6f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141c3165a0 con 0x56141c2442d0 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.441+0000 7f78d1fd6f40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141c285390 con 0x7f78b8003ab0 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.441+0000 7f78d1fd6f40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56141c3160f0 con 0x56141c315a40 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.441+0000 7f78cfad7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56141c2442d0 0x56141c3128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.441+0000 7f78cfad7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56141c2442d0 0x56141c3128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46982/0 (socket says 172.21.15.19:46982) 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.441+0000 7f78cfad7640 1 -- 172.21.15.19:0/867393136 learned_addr learned my addr 172.21.15.19:0/867393136 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78cf2d6640 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56141c315a40 0x56141c310ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78d02d8640 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b8003ab0 0x56141c315080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78cfad7640 1 -- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56141c315a40 msgr2=0x56141c310ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78cfad7640 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56141c315a40 0x56141c310ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78cfad7640 1 -- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b8003ab0 msgr2=0x56141c315080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78cfad7640 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78b8003ab0 0x56141c315080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78cfad7640 1 -- 172.21.15.19:0/867393136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141c3108a0 con 0x56141c2442d0 2024-04-25T10:05:52.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78cfad7640 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56141c2442d0 0x56141c3128b0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f78c0001a10 tx=0x7f78c0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78cd2d2640 1 -- 172.21.15.19:0/867393136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c000f650 con 0x56141c2442d0 2024-04-25T10:05:52.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78cd2d2640 1 -- 172.21.15.19:0/867393136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c000f810 con 0x56141c2442d0 2024-04-25T10:05:52.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/867393136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56141c3155c0 con 0x56141c2442d0 2024-04-25T10:05:52.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78cd2d2640 1 -- 172.21.15.19:0/867393136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c000fc00 con 0x56141c2442d0 2024-04-25T10:05:52.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.442+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/867393136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56141c3102e0 con 0x56141c2442d0 2024-04-25T10:05:52.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.443+0000 7f78cd2d2640 1 -- 172.21.15.19:0/867393136 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78c0023070 con 0x56141c2442d0 2024-04-25T10:05:52.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.444+0000 7f78cd2d2640 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78bc069eb0 0x7f78bc06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.444+0000 7f78cd2d2640 1 -- 172.21.15.19:0/867393136 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f78c008b630 con 0x56141c2442d0 2024-04-25T10:05:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.444+0000 7f78cf2d6640 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78bc069eb0 0x7f78bc06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.444+0000 7f78d1fd6f40 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x56141c30c5b0 0x56141c30c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.444+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/867393136 --> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] -- osd_op(unknown.0.0:1 29.10 29:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e192) v9 -- 0x56141c270ae0 con 0x56141c30c5b0 2024-04-25T10:05:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.444+0000 7f78d02d8640 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x56141c30c5b0 0x56141c30c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.444+0000 7f78cf2d6640 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78bc069eb0 0x7f78bc06c370 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f78c4005400 tx=0x7f78c400c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.445+0000 7f78d02d8640 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x56141c30c5b0 0x56141c30c9e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d02d8640 1 -- 172.21.15.19:0/867393136 <== osd.6 v2:172.21.15.112:6816/3693987094 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-vals-by-keys out=588b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+588 (crc 0 0 0) 0x7f78c805b040 con 0x56141c30c5b0 2024-04-25T10:05:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:Writing to - 2024-04-25T10:05:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x56141c30c5b0 msgr2=0x56141c30c9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d1fd6f40 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x56141c30c5b0 0x56141c30c9e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78bc069eb0 msgr2=0x7f78bc06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d1fd6f40 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f78bc069eb0 0x7f78bc06c370 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f78c4005400 tx=0x7f78c400c040 comp rx=0 tx=0).stop 2024-04-25T10:05:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/867393136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56141c2442d0 msgr2=0x56141c3128b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d1fd6f40 1 --2- 172.21.15.19:0/867393136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56141c2442d0 0x56141c3128b0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f78c0001a10 tx=0x7f78c0001a40 comp rx=0 tx=0).stop 2024-04-25T10:05:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d02d8640 1 -- 172.21.15.19:0/867393136 reap_dead start 2024-04-25T10:05:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/867393136 shutdown_connections 2024-04-25T10:05:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/867393136 >> 172.21.15.19:0/867393136 conn(0x56141c2439b0 msgr2=0x56141c2430a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/867393136 shutdown_connections 2024-04-25T10:05:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.447+0000 7f78d1fd6f40 1 -- 172.21.15.19:0/867393136 wait complete. 2024-04-25T10:05:52.458 INFO:teuthology.orchestra.run.smithi019.stdout:i"��**fIU����$@@`���������**fIU��**fJk�(8`�**fIU�8`�**fIU����������**fJk�����security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� 2024-04-25T10:05:52.458 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 shadow_head 2024-04-25T10:05:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.534+0000 7fe717d69f40 1 Processor -- start 2024-04-25T10:05:52.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.534+0000 7fe717d69f40 1 -- start start 2024-04-25T10:05:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe717d69f40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db070dd40 0x558db06ee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe717d69f40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558db0797c30 0x558db0795210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe717d69f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db0798010 0x558db06ef400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe717d69f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db06f21e0 con 0x558db0798010 2024-04-25T10:05:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe717d69f40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db066adf0 con 0x558db070dd40 2024-04-25T10:05:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe717d69f40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x558db06f2480 con 0x558db0797c30 2024-04-25T10:05:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe71606b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db0798010 0x558db06ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe71606b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db0798010 0x558db06ef400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46988/0 (socket says 172.21.15.19:46988) 2024-04-25T10:05:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe71606b640 1 -- 172.21.15.19:0/3424866810 learned_addr learned my addr 172.21.15.19:0/3424866810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe715069640 1 --2- 172.21.15.19:0/3424866810 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558db0797c30 0x558db0795210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe71586a640 1 --2- 172.21.15.19:0/3424866810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db070dd40 0x558db06ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe71606b640 1 -- 172.21.15.19:0/3424866810 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558db0797c30 msgr2=0x558db0795210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe71606b640 1 --2- 172.21.15.19:0/3424866810 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558db0797c30 0x558db0795210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe71606b640 1 -- 172.21.15.19:0/3424866810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db070dd40 msgr2=0x558db06ee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe71606b640 1 --2- 172.21.15.19:0/3424866810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db070dd40 0x558db06ee540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe71606b640 1 -- 172.21.15.19:0/3424866810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558db05f0c50 con 0x558db0798010 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe71586a640 1 --2- 172.21.15.19:0/3424866810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db070dd40 0x558db06ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe71606b640 1 --2- 172.21.15.19:0/3424866810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db0798010 0x558db06ef400 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7fe7080065e0 tx=0x7fe708006aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8711baef1dffc17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe707fff640 1 -- 172.21.15.19:0/3424866810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708016070 con 0x558db0798010 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe707fff640 1 -- 172.21.15.19:0/3424866810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708004540 con 0x558db0798010 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.535+0000 7fe707fff640 1 -- 172.21.15.19:0/3424866810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708004970 con 0x558db0798010 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.536+0000 7fe717d69f40 1 -- 172.21.15.19:0/3424866810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db0798010 msgr2=0x558db06ef400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.536+0000 7fe717d69f40 1 --2- 172.21.15.19:0/3424866810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db0798010 0x558db06ef400 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7fe7080065e0 tx=0x7fe708006aa0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.536+0000 7fe717d69f40 1 -- 172.21.15.19:0/3424866810 shutdown_connections 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.536+0000 7fe717d69f40 1 --2- 172.21.15.19:0/3424866810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db0798010 0x558db06ef400 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.536+0000 7fe717d69f40 1 --2- 172.21.15.19:0/3424866810 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558db0797c30 0x558db0795210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.536+0000 7fe717d69f40 1 --2- 172.21.15.19:0/3424866810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db070dd40 0x558db06ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.536+0000 7fe717d69f40 1 -- 172.21.15.19:0/3424866810 >> 172.21.15.19:0/3424866810 conn(0x558db06c29b0 msgr2=0x558db06c5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.536+0000 7fe717d69f40 1 -- 172.21.15.19:0/3424866810 shutdown_connections 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.536+0000 7fe717d69f40 1 -- 172.21.15.19:0/3424866810 wait complete. 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe717d69f40 1 Processor -- start 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe717d69f40 1 -- start start 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe717d69f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db070dd40 0x558db0794260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe717d69f40 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558db0797c30 0x558db07a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe717d69f40 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db0798010 0x558db07a3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe717d69f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db06c3480 con 0x558db070dd40 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe717d69f40 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db0793d80 con 0x558db0798010 2024-04-25T10:05:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe717d69f40 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x558db079b860 con 0x558db0797c30 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe71586a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db070dd40 0x558db0794260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe71586a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db070dd40 0x558db0794260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46992/0 (socket says 172.21.15.19:46992) 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe71586a640 1 -- 172.21.15.19:0/907750698 learned_addr learned my addr 172.21.15.19:0/907750698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.537+0000 7fe715069640 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558db0797c30 0x558db07a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe71606b640 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db0798010 0x558db07a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe71586a640 1 -- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558db0797c30 msgr2=0x558db07a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe71586a640 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558db0797c30 0x558db07a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe71586a640 1 -- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db0798010 msgr2=0x558db07a3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe71586a640 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db0798010 0x558db07a3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe71586a640 1 -- 172.21.15.19:0/907750698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558db078e7b0 con 0x558db070dd40 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe71586a640 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db070dd40 0x558db0794260 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7fe710005fd0 tx=0x7fe710008400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe706ffd640 1 -- 172.21.15.19:0/907750698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710008e40 con 0x558db070dd40 2024-04-25T10:05:52.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe706ffd640 1 -- 172.21.15.19:0/907750698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe710015460 con 0x558db070dd40 2024-04-25T10:05:52.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe717d69f40 1 -- 172.21.15.19:0/907750698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558db078ea90 con 0x558db070dd40 2024-04-25T10:05:52.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe706ffd640 1 -- 172.21.15.19:0/907750698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710015850 con 0x558db070dd40 2024-04-25T10:05:52.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.538+0000 7fe717d69f40 1 -- 172.21.15.19:0/907750698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558db0791050 con 0x558db070dd40 2024-04-25T10:05:52.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.539+0000 7fe706ffd640 1 -- 172.21.15.19:0/907750698 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe71001c020 con 0x558db070dd40 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.540+0000 7fe706ffd640 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6f8069eb0 0x7fe6f806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.540+0000 7fe706ffd640 1 -- 172.21.15.19:0/907750698 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe710088fe0 con 0x558db070dd40 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.540+0000 7fe715069640 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6f8069eb0 0x7fe6f806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.540+0000 7fe717d69f40 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x558db078a260 0x558db0799e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.540+0000 7fe717d69f40 1 -- 172.21.15.19:0/907750698 --> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] -- osd_op(unknown.0.0:1 29.10 29:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e192) v9 -- 0x558db06efb30 con 0x558db078a260 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.540+0000 7fe71606b640 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x558db078a260 0x558db0799e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.540+0000 7fe715069640 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6f8069eb0 0x7fe6f806c370 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe70c056000 tx=0x7fe70c050000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.541+0000 7fe71606b640 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x558db078a260 0x558db0799e00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe71606b640 1 -- 172.21.15.19:0/907750698 <== osd.6 v2:172.21.15.112:6816/3693987094 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v192'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe708019040 con 0x558db078a260 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe717d69f40 1 -- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x558db078a260 msgr2=0x558db0799e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe717d69f40 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x558db078a260 0x558db0799e00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe717d69f40 1 -- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6f8069eb0 msgr2=0x7fe6f806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe717d69f40 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6f8069eb0 0x7fe6f806c370 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe70c056000 tx=0x7fe70c050000 comp rx=0 tx=0).stop 2024-04-25T10:05:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe717d69f40 1 -- 172.21.15.19:0/907750698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db070dd40 msgr2=0x558db0794260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe717d69f40 1 --2- 172.21.15.19:0/907750698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558db070dd40 0x558db0794260 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7fe710005fd0 tx=0x7fe710008400 comp rx=0 tx=0).stop 2024-04-25T10:05:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe71606b640 1 -- 172.21.15.19:0/907750698 reap_dead start 2024-04-25T10:05:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe717d69f40 1 -- 172.21.15.19:0/907750698 shutdown_connections 2024-04-25T10:05:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe717d69f40 1 -- 172.21.15.19:0/907750698 >> 172.21.15.19:0/907750698 conn(0x558db06c29b0 msgr2=0x558db06c58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe717d69f40 1 -- 172.21.15.19:0/907750698 shutdown_connections 2024-04-25T10:05:52.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.542+0000 7fe717d69f40 1 -- 172.21.15.19:0/907750698 wait complete. 2024-04-25T10:05:52.553 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_hack_allow_loading_invalid_metadata True 2024-04-25T10:05:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.691+0000 7fa0a64f5640 1 Processor -- start 2024-04-25T10:05:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.691+0000 7fa0a64f5640 1 -- start start 2024-04-25T10:05:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa0a64f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0144d50 0x7fa0a006d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa0a64f5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0a01452a0 0x7fa0a006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa0a64f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a00cccc0 con 0x7fa0a01446d0 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa0a64f5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a00b2b20 con 0x7fa0a01452a0 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa0a64f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a006d400 con 0x7fa0a0144d50 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa09f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0144d50 0x7fa0a006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa0a4a92640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0a01452a0 0x7fa0a006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa09f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0144d50 0x7fa0a006d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:59684/0 (socket says 172.21.15.19:59684) 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa09f7fe640 1 -- 172.21.15.19:0/2873045110 learned_addr learned my addr 172.21.15.19:0/2873045110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa0a4a92640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0a01452a0 0x7fa0a006ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:36362/0 (socket says 172.21.15.19:36362) 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa09ffff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa0a01446d0 0x7fa0a00b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:52286/0 (socket says 172.21.15.19:52286) 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa09f7fe640 1 -- 172.21.15.19:0/2873045110 >> v2:172.21.15.112:3300/0 conn(0x7fa0a01452a0 msgr2=0x7fa0a006ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa09f7fe640 1 --2- 172.21.15.19:0/2873045110 >> v2:172.21.15.112:3300/0 conn(0x7fa0a01452a0 0x7fa0a006ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa09f7fe640 1 -- 172.21.15.19:0/2873045110 >> v1:172.21.15.112:6789/0 conn(0x7fa0a01446d0 legacy=0x7fa0a00b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.692+0000 7fa09f7fe640 1 -- 172.21.15.19:0/2873045110 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a00cc140 con 0x7fa0a0144d50 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.693+0000 7fa09f7fe640 1 --2- 172.21.15.19:0/2873045110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0144d50 0x7fa0a006d890 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa090009870 tx=0x7fa09002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1cb15b930d5ac38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.693+0000 7fa09effd640 1 -- 172.21.15.19:0/2873045110 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090037070 con 0x7fa0a0144d50 2024-04-25T10:05:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.693+0000 7fa09effd640 1 -- 172.21.15.19:0/2873045110 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa090032440 con 0x7fa0a0144d50 2024-04-25T10:05:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.693+0000 7fa09effd640 1 -- 172.21.15.19:0/2873045110 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090032810 con 0x7fa0a0144d50 2024-04-25T10:05:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.694+0000 7fa0a64f5640 1 -- 172.21.15.19:0/2873045110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0144d50 msgr2=0x7fa0a006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.694+0000 7fa0a64f5640 1 --2- 172.21.15.19:0/2873045110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0144d50 0x7fa0a006d890 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa090009870 tx=0x7fa09002a270 comp rx=0 tx=0).stop 2024-04-25T10:05:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.694+0000 7fa0a64f5640 1 -- 172.21.15.19:0/2873045110 shutdown_connections 2024-04-25T10:05:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.694+0000 7fa0a64f5640 1 --2- 172.21.15.19:0/2873045110 >> v2:172.21.15.112:3300/0 conn(0x7fa0a01452a0 0x7fa0a006ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.694+0000 7fa0a64f5640 1 --2- 172.21.15.19:0/2873045110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0144d50 0x7fa0a006d890 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.694+0000 7fa0a64f5640 1 -- 172.21.15.19:0/2873045110 >> 172.21.15.19:0/2873045110 conn(0x7fa0a006abe0 msgr2=0x7fa0a006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.694+0000 7fa0a64f5640 1 -- 172.21.15.19:0/2873045110 shutdown_connections 2024-04-25T10:05:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.694+0000 7fa0a64f5640 1 -- 172.21.15.19:0/2873045110 wait complete. 2024-04-25T10:05:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.695+0000 7fa0a64f5640 1 Processor -- start 2024-04-25T10:05:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.695+0000 7fa0a64f5640 1 -- start start 2024-04-25T10:05:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa0a64f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a01446d0 0x7fa0a00a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa0a64f5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0144d50 0x7fa0a013cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa0a64f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0a01452a0 0x7fa0a01e42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa0a64f5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00cc9e0 con 0x7fa0a0144d50 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa0a64f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00cc700 con 0x7fa0a01452a0 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa0a64f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a0144ab0 con 0x7fa0a01446d0 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa09f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0144d50 0x7fa0a013cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa09f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0144d50 0x7fa0a013cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47000/0 (socket says 172.21.15.19:47000) 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa09f7fe640 1 -- 172.21.15.19:0/3197136610 learned_addr learned my addr 172.21.15.19:0/3197136610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa09ffff640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a01446d0 0x7fa0a00a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa0a4a92640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0a01452a0 0x7fa0a01e42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa09f7fe640 1 -- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a01446d0 msgr2=0x7fa0a00a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa09f7fe640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a01446d0 0x7fa0a00a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa09f7fe640 1 -- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0a01452a0 msgr2=0x7fa0a01e42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa09f7fe640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0a01452a0 0x7fa0a01e42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.696+0000 7fa09f7fe640 1 -- 172.21.15.19:0/3197136610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a00a8aa0 con 0x7fa0a0144d50 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.697+0000 7fa09f7fe640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0144d50 0x7fa0a013cff0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7fa090000c00 tx=0x7fa0900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.697+0000 7fa09cff9640 1 -- 172.21.15.19:0/3197136610 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090037040 con 0x7fa0a0144d50 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.697+0000 7fa09cff9640 1 -- 172.21.15.19:0/3197136610 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa090042070 con 0x7fa0a0144d50 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.697+0000 7fa0a64f5640 1 -- 172.21.15.19:0/3197136610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a01e37c0 con 0x7fa0a0144d50 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.697+0000 7fa09cff9640 1 -- 172.21.15.19:0/3197136610 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090030880 con 0x7fa0a0144d50 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.697+0000 7fa0a64f5640 1 -- 172.21.15.19:0/3197136610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a01e3d80 con 0x7fa0a0144d50 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.698+0000 7fa0a64f5640 1 -- 172.21.15.19:0/3197136610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a0073a00 con 0x7fa0a0144d50 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.698+0000 7fa09cff9640 1 -- 172.21.15.19:0/3197136610 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa090030a20 con 0x7fa0a0144d50 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.699+0000 7fa09cff9640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa08c06b4d0 0x7fa08c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.699+0000 7fa09cff9640 1 -- 172.21.15.19:0/3197136610 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa0900aab80 con 0x7fa0a0144d50 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.699+0000 7fa09ffff640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa08c06b4d0 0x7fa08c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.699+0000 7fa09ffff640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa08c06b4d0 0x7fa08c06d990 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa094007fd0 tx=0x7fa094007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.703+0000 7fa09cff9640 1 -- 172.21.15.19:0/3197136610 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0900790e0 con 0x7fa0a0144d50 2024-04-25T10:05:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.888+0000 7fa0a64f5640 1 -- 172.21.15.19:0/3197136610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}] v 0) v1 -- 0x7fa0a0142160 con 0x7fa0a0144d50 2024-04-25T10:05:52.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.891+0000 7fa09cff9640 1 -- 172.21.15.19:0/3197136610 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}]=0 v35)=0 v35) v1 ==== 140+0+0 (secure 0 0 0) 0x7fa09007d180 con 0x7fa0a0144d50 2024-04-25T10:05:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.893+0000 7fa0a64f5640 1 -- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa08c06b4d0 msgr2=0x7fa08c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.893+0000 7fa0a64f5640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa08c06b4d0 0x7fa08c06d990 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa094007fd0 tx=0x7fa094007e80 comp rx=0 tx=0).stop 2024-04-25T10:05:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.893+0000 7fa0a64f5640 1 -- 172.21.15.19:0/3197136610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0144d50 msgr2=0x7fa0a013cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.893+0000 7fa0a64f5640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0144d50 0x7fa0a013cff0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7fa090000c00 tx=0x7fa0900046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.893+0000 7fa0a64f5640 1 -- 172.21.15.19:0/3197136610 shutdown_connections 2024-04-25T10:05:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.893+0000 7fa0a64f5640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa08c06b4d0 0x7fa08c06d990 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.893+0000 7fa0a64f5640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0a01452a0 0x7fa0a01e42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.893+0000 7fa0a64f5640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0144d50 0x7fa0a013cff0 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.893+0000 7fa0a64f5640 1 --2- 172.21.15.19:0/3197136610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a01446d0 0x7fa0a00a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.893+0000 7fa0a64f5640 1 -- 172.21.15.19:0/3197136610 >> 172.21.15.19:0/3197136610 conn(0x7fa0a006abe0 msgr2=0x7fa0a00af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.894+0000 7fa0a64f5640 1 -- 172.21.15.19:0/3197136610 shutdown_connections 2024-04-25T10:05:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:52.894+0000 7fa0a64f5640 1 -- 172.21.15.19:0/3197136610 wait complete. 2024-04-25T10:05:52.911 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:05:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.007+0000 7f2dcaa12640 1 Processor -- start 2024-04-25T10:05:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.007+0000 7f2dcaa12640 1 -- start start 2024-04-25T10:05:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.008+0000 7f2dcaa12640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 0x7f2dc4161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.008+0000 7f2dcaa12640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2dc40b9750 0x7f2dc40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.008+0000 7f2dcaa12640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc4162040 con 0x7f2dc4161ba0 2024-04-25T10:05:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.008+0000 7f2dcaa12640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc4162600 con 0x7f2dc40b9750 2024-04-25T10:05:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.008+0000 7f2dcaa12640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc4162320 con 0x7f2dc415da90 2024-04-25T10:05:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.009+0000 7f2dc3fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 0x7f2dc4161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.009+0000 7f2dc37fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2dc40b9750 0x7f2dc40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.009+0000 7f2dc3fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 0x7f2dc4161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:59702/0 (socket says 172.21.15.19:59702) 2024-04-25T10:05:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.009+0000 7f2dc3fff640 1 -- 172.21.15.19:0/1808757895 learned_addr learned my addr 172.21.15.19:0/1808757895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.009+0000 7f2dc3fff640 1 -- 172.21.15.19:0/1808757895 >> v2:172.21.15.112:3300/0 conn(0x7f2dc40b9750 msgr2=0x7f2dc40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.009+0000 7f2dc3fff640 1 --2- 172.21.15.19:0/1808757895 >> v2:172.21.15.112:3300/0 conn(0x7f2dc40b9750 0x7f2dc40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.009+0000 7f2dc3fff640 1 -- 172.21.15.19:0/1808757895 >> v1:172.21.15.112:6789/0 conn(0x7f2dc4161ba0 legacy=0x7f2dc406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.009+0000 7f2dc3fff640 1 -- 172.21.15.19:0/1808757895 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc40d1b10 con 0x7f2dc415da90 2024-04-25T10:05:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.009+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/1808757895 >> v2:172.21.15.112:3300/0 conn(0x7f2dc40b9750 0x7f2dc40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.010+0000 7f2dc3fff640 1 --2- 172.21.15.19:0/1808757895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 0x7f2dc4161660 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cac69fbc4145ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.010+0000 7f2dc2ffd640 1 -- 172.21.15.19:0/1808757895 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac037070 con 0x7f2dc415da90 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.010+0000 7f2dc2ffd640 1 -- 172.21.15.19:0/1808757895 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac032440 con 0x7f2dc415da90 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.010+0000 7f2dc2ffd640 1 -- 172.21.15.19:0/1808757895 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac032810 con 0x7f2dc415da90 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.010+0000 7f2dcaa12640 1 -- 172.21.15.19:0/1808757895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 msgr2=0x7f2dc4161660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.010+0000 7f2dcaa12640 1 --2- 172.21.15.19:0/1808757895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 0x7f2dc4161660 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.011+0000 7f2dcaa12640 1 -- 172.21.15.19:0/1808757895 shutdown_connections 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.011+0000 7f2dcaa12640 1 --2- 172.21.15.19:0/1808757895 >> v2:172.21.15.112:3300/0 conn(0x7f2dc40b9750 0x7f2dc40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.011+0000 7f2dcaa12640 1 --2- 172.21.15.19:0/1808757895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 0x7f2dc4161660 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.011+0000 7f2dcaa12640 1 -- 172.21.15.19:0/1808757895 >> 172.21.15.19:0/1808757895 conn(0x7f2dc4061a20 msgr2=0x7f2dc4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.011+0000 7f2dcaa12640 1 -- 172.21.15.19:0/1808757895 shutdown_connections 2024-04-25T10:05:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.011+0000 7f2dcaa12640 1 -- 172.21.15.19:0/1808757895 wait complete. 2024-04-25T10:05:53.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.012+0000 7f2dcaa12640 1 Processor -- start 2024-04-25T10:05:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.012+0000 7f2dcaa12640 1 -- start start 2024-04-25T10:05:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dcaa12640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 0x7f2dc40c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dcaa12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc4161ba0 0x7f2dc40ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dcaa12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc414ee60 0x7f2dc40b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dcaa12640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40d3320 con 0x7f2dc4161ba0 2024-04-25T10:05:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dcaa12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc415de70 con 0x7f2dc414ee60 2024-04-25T10:05:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dcaa12640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40d34a0 con 0x7f2dc415da90 2024-04-25T10:05:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc4161ba0 0x7f2dc40ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc4161ba0 0x7f2dc40ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47008/0 (socket says 172.21.15.19:47008) 2024-04-25T10:05:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc37fe640 1 -- 172.21.15.19:0/3774136191 learned_addr learned my addr 172.21.15.19:0/3774136191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc3fff640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 0x7f2dc40c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc8faf640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc414ee60 0x7f2dc40b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc37fe640 1 -- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 msgr2=0x7f2dc40c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 0x7f2dc40c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc37fe640 1 -- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc414ee60 msgr2=0x7f2dc40b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc414ee60 0x7f2dc40b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc37fe640 1 -- 172.21.15.19:0/3774136191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc414f2f0 con 0x7f2dc4161ba0 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc3fff640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 0x7f2dc40c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc37fe640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc4161ba0 0x7f2dc40ccd40 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f2db400e960 tx=0x7f2db400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc0ff9640 1 -- 172.21.15.19:0/3774136191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4017070 con 0x7f2dc4161ba0 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc0ff9640 1 -- 172.21.15.19:0/3774136191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db400ca00 con 0x7f2dc4161ba0 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.013+0000 7f2dc0ff9640 1 -- 172.21.15.19:0/3774136191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db400cdd0 con 0x7f2dc4161ba0 2024-04-25T10:05:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.014+0000 7f2dcaa12640 1 -- 172.21.15.19:0/3774136191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc40c7b70 con 0x7f2dc4161ba0 2024-04-25T10:05:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.014+0000 7f2dcaa12640 1 -- 172.21.15.19:0/3774136191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc40c8130 con 0x7f2dc4161ba0 2024-04-25T10:05:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.015+0000 7f2dcaa12640 1 -- 172.21.15.19:0/3774136191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc4073a00 con 0x7f2dc4161ba0 2024-04-25T10:05:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.015+0000 7f2dc0ff9640 1 -- 172.21.15.19:0/3774136191 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2db401a070 con 0x7f2dc4161ba0 2024-04-25T10:05:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.015+0000 7f2dc0ff9640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2da406b4d0 0x7f2da406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.016+0000 7f2dc0ff9640 1 -- 172.21.15.19:0/3774136191 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f2db408a6b0 con 0x7f2dc4161ba0 2024-04-25T10:05:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.016+0000 7f2dc3fff640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2da406b4d0 0x7f2da406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.016+0000 7f2dc3fff640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2da406b4d0 0x7f2da406d990 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2dac034000 tx=0x7f2dac02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.020+0000 7f2dc0ff9640 1 -- 172.21.15.19:0/3774136191 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2db4058bf0 con 0x7f2dc4161ba0 2024-04-25T10:05:53.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.243+0000 7f2dcaa12640 1 -- 172.21.15.19:0/3774136191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f2dc40d1590 con 0x7f2dc4161ba0 2024-04-25T10:05:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.379+0000 7f2dc0ff9640 1 -- 172.21.15.19:0/3774136191 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v125) v1 ==== 162+0+0 (secure 0 0 0) 0x7f2db405cc90 con 0x7f2dc4161ba0 2024-04-25T10:05:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:05:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 -- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2da406b4d0 msgr2=0x7f2da406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2da406b4d0 0x7f2da406d990 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2dac034000 tx=0x7f2dac02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 -- 172.21.15.19:0/3774136191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc4161ba0 msgr2=0x7f2dc40ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc4161ba0 0x7f2dc40ccd40 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f2db400e960 tx=0x7f2db400ee20 comp rx=0 tx=0).stop 2024-04-25T10:05:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 -- 172.21.15.19:0/3774136191 shutdown_connections 2024-04-25T10:05:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f2da406b4d0 0x7f2da406d990 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc414ee60 0x7f2dc40b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc4161ba0 0x7f2dc40ccd40 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 --2- 172.21.15.19:0/3774136191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dc415da90 0x7f2dc40c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 -- 172.21.15.19:0/3774136191 >> 172.21.15.19:0/3774136191 conn(0x7f2dc4061a20 msgr2=0x7f2dc40b7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 -- 172.21.15.19:0/3774136191 shutdown_connections 2024-04-25T10:05:53.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.382+0000 7f2dcaa12640 1 -- 172.21.15.19:0/3774136191 wait complete. 2024-04-25T10:05:53.401 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:53.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.495+0000 7f5561153640 1 Processor -- start 2024-04-25T10:05:53.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.495+0000 7f5561153640 1 -- start start 2024-04-25T10:05:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.496+0000 7f5561153640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f555c0cfec0 0x7f555c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.496+0000 7f5561153640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c071480 con 0x7f555c154ee0 2024-04-25T10:05:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.496+0000 7f5561153640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c071760 con 0x7f555c0ce3c0 2024-04-25T10:05:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.496+0000 7f5561153640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c0ce830 con 0x7f555c0cfec0 2024-04-25T10:05:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.496+0000 7f555ad9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f555c154ee0 0x7f555c0cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42502/0 (socket says 172.21.15.19:42502) 2024-04-25T10:05:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.496+0000 7f555ad9d640 1 -- 172.21.15.19:0/3686895607 learned_addr learned my addr 172.21.15.19:0/3686895607 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.496+0000 7f555a59c640 1 --2- 172.21.15.19:0/3686895607 >> v2:172.21.15.19:3300/0 conn(0x7f555c0cfec0 0x7f555c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.496+0000 7f5559d9b640 1 -- 172.21.15.19:0/3686895607 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650384198 0 0) 0x7f555c071480 con 0x7f555c154ee0 2024-04-25T10:05:53.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.496+0000 7f5559d9b640 1 -- 172.21.15.19:0/3686895607 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5548005730 con 0x7f555c154ee0 2024-04-25T10:05:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.497+0000 7f555a59c640 1 -- 172.21.15.19:0/3686895607 >> v1:172.21.15.112:6789/0 conn(0x7f555c0ce3c0 legacy=0x7f555c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.497+0000 7f555a59c640 1 -- 172.21.15.19:0/3686895607 >> v1:172.21.15.19:6789/0 conn(0x7f555c154ee0 legacy=0x7f555c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.497+0000 7f555a59c640 1 -- 172.21.15.19:0/3686895607 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c071a40 con 0x7f555c0cfec0 2024-04-25T10:05:53.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.497+0000 7f555a59c640 1 --2- 172.21.15.19:0/3686895607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c0cfec0 0x7f555c0cde80 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f55440097a0 tx=0x7f554402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118926f64095ba65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.497+0000 7f5559d9b640 1 -- 172.21.15.19:0/3686895607 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544037070 con 0x7f555c0cfec0 2024-04-25T10:05:53.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.497+0000 7f5559d9b640 1 -- 172.21.15.19:0/3686895607 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5544032440 con 0x7f555c0cfec0 2024-04-25T10:05:53.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.497+0000 7f5559d9b640 1 -- 172.21.15.19:0/3686895607 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544032810 con 0x7f555c0cfec0 2024-04-25T10:05:53.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.498+0000 7f5561153640 1 -- 172.21.15.19:0/3686895607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c0cfec0 msgr2=0x7f555c0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.498+0000 7f5561153640 1 --2- 172.21.15.19:0/3686895607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c0cfec0 0x7f555c0cde80 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f55440097a0 tx=0x7f554402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.498+0000 7f5561153640 1 -- 172.21.15.19:0/3686895607 shutdown_connections 2024-04-25T10:05:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.498+0000 7f5561153640 1 --2- 172.21.15.19:0/3686895607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c0cfec0 0x7f555c0cde80 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.498+0000 7f5561153640 1 -- 172.21.15.19:0/3686895607 >> 172.21.15.19:0/3686895607 conn(0x7f555c06abe0 msgr2=0x7f555c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.498+0000 7f5561153640 1 -- 172.21.15.19:0/3686895607 shutdown_connections 2024-04-25T10:05:53.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.498+0000 7f5561153640 1 -- 172.21.15.19:0/3686895607 wait complete. 2024-04-25T10:05:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.500+0000 7f5561153640 1 Processor -- start 2024-04-25T10:05:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.500+0000 7f5561153640 1 -- start start 2024-04-25T10:05:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.500+0000 7f5561153640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f555c0ce3c0 0x7f555c14dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.500+0000 7f5561153640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c0cfec0 0x7f555c150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.500+0000 7f5561153640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f555c154ee0 0x7f555c150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.500+0000 7f5561153640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c071230 con 0x7f555c0cfec0 2024-04-25T10:05:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.500+0000 7f5561153640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c1552c0 con 0x7f555c154ee0 2024-04-25T10:05:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.500+0000 7f5561153640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c155440 con 0x7f555c0ce3c0 2024-04-25T10:05:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.500+0000 7f555a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c0cfec0 0x7f555c150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.500+0000 7f555a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c0cfec0 0x7f555c150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47018/0 (socket says 172.21.15.19:47018) 2024-04-25T10:05:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.500+0000 7f555a59c640 1 -- 172.21.15.19:0/3779295566 learned_addr learned my addr 172.21.15.19:0/3779295566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f555ad9d640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f555c0ce3c0 0x7f555c14dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f555b59e640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f555c154ee0 0x7f555c150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f555a59c640 1 -- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f555c0ce3c0 msgr2=0x7f555c14dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f555a59c640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f555c0ce3c0 0x7f555c14dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f555a59c640 1 -- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f555c154ee0 msgr2=0x7f555c150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f555a59c640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f555c154ee0 0x7f555c150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f555a59c640 1 -- 172.21.15.19:0/3779295566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c14fa40 con 0x7f555c0cfec0 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f555a59c640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c0cfec0 0x7f555c150540 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f5544000c00 tx=0x7f55440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f55537fe640 1 -- 172.21.15.19:0/3779295566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544037040 con 0x7f555c0cfec0 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f55537fe640 1 -- 172.21.15.19:0/3779295566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5544042070 con 0x7f555c0cfec0 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f5561153640 1 -- 172.21.15.19:0/3779295566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c14fd20 con 0x7f555c0cfec0 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f55537fe640 1 -- 172.21.15.19:0/3779295566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544030940 con 0x7f555c0cfec0 2024-04-25T10:05:53.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.501+0000 7f5561153640 1 -- 172.21.15.19:0/3779295566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c1e3180 con 0x7f555c0cfec0 2024-04-25T10:05:53.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.502+0000 7f5561153640 1 -- 172.21.15.19:0/3779295566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555c073990 con 0x7f555c0cfec0 2024-04-25T10:05:53.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.503+0000 7f55537fe640 1 -- 172.21.15.19:0/3779295566 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5544030ae0 con 0x7f555c0cfec0 2024-04-25T10:05:53.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.503+0000 7f55537fe640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f553c06b4d0 0x7f553c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.503+0000 7f55537fe640 1 -- 172.21.15.19:0/3779295566 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f55440aad40 con 0x7f555c0cfec0 2024-04-25T10:05:53.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.503+0000 7f555ad9d640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f553c06b4d0 0x7f553c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.504+0000 7f555ad9d640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f553c06b4d0 0x7f553c06d990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5554000f60 tx=0x7f5554009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.508+0000 7f55537fe640 1 -- 172.21.15.19:0/3779295566 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5544045360 con 0x7f555c0cfec0 2024-04-25T10:05:53.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.722+0000 7f5561153640 1 -- 172.21.15.19:0/3779295566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f555c06d360 con 0x7f555c0cfec0 2024-04-25T10:05:53.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.723+0000 7f55537fe640 1 -- 172.21.15.19:0/3779295566 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4956 (secure 0 0 0) 0x7f5544079970 con 0x7f555c0cfec0 2024-04-25T10:05:53.729 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:53.730 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":126,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:38.424174+0000","modified":"2024-04-25T10:05:53.384217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":192,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11799":{"gid":11799,"name":"c","rank":0,"incarnation":126,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/1758455116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1758455116},{"type":"v1","addr":"172.21.15.19:6835","nonce":1758455116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:05:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 126 2024-04-25T10:05:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 -- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f553c06b4d0 msgr2=0x7f553c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f553c06b4d0 0x7f553c06d990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5554000f60 tx=0x7f5554009040 comp rx=0 tx=0).stop 2024-04-25T10:05:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 -- 172.21.15.19:0/3779295566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c0cfec0 msgr2=0x7f555c150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c0cfec0 0x7f555c150540 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f5544000c00 tx=0x7f55440046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 -- 172.21.15.19:0/3779295566 shutdown_connections 2024-04-25T10:05:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f553c06b4d0 0x7f553c06d990 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f555c154ee0 0x7f555c150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f555c0cfec0 0x7f555c150540 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 --2- 172.21.15.19:0/3779295566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f555c0ce3c0 0x7f555c14dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 -- 172.21.15.19:0/3779295566 >> 172.21.15.19:0/3779295566 conn(0x7f555c06abe0 msgr2=0x7f555c15d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 -- 172.21.15.19:0/3779295566 shutdown_connections 2024-04-25T10:05:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:53.725+0000 7f5561153640 1 -- 172.21.15.19:0/3779295566 wait complete. 2024-04-25T10:05:54.145 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 126, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:38.424174+0000', 'modified': '2024-04-25T10:05:53.384217+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 192, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11799': {'gid': 11799, 'name': 'c', 'rank': 0, 'incarnation': 126, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/1758455116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 1758455116}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 1758455116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:05:54.145 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11799:up:replay 2024-04-25T10:05:55.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:55.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.252+0000 7ffb51c88640 1 Processor -- start 2024-04-25T10:05:55.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.253+0000 7ffb51c88640 1 -- start start 2024-04-25T10:05:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.253+0000 7ffb51c88640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb4c0b2cb0 0x7ffb4c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.253+0000 7ffb51c88640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb4c0d1020 0x7ffb4c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.253+0000 7ffb51c88640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4c0b97a0 con 0x7ffb4c15dbe0 2024-04-25T10:05:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.253+0000 7ffb51c88640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c15dfc0 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.253+0000 7ffb51c88640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0d1a00 con 0x7ffb4c0d1020 2024-04-25T10:05:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.254+0000 7ffb4affd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb4c0b2cb0 0x7ffb4c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.254+0000 7ffb4bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb4c0d1020 0x7ffb4c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.254+0000 7ffb4affd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffb4c0b2cb0 0x7ffb4c0d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:36410/0 (socket says 172.21.15.19:36410) 2024-04-25T10:05:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.254+0000 7ffb4affd640 1 -- 172.21.15.19:0/2083655661 learned_addr learned my addr 172.21.15.19:0/2083655661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.254+0000 7ffb4b7fe640 1 --1- 172.21.15.19:0/2083655661 >> v1:172.21.15.112:6789/0 conn(0x7ffb4c15dbe0 0x7ffb4c0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:52306/0 (socket says 172.21.15.19:52306) 2024-04-25T10:05:55.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.254+0000 7ffb4affd640 1 -- 172.21.15.19:0/2083655661 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb4c0d1020 msgr2=0x7ffb4c0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.254+0000 7ffb4affd640 1 --2- 172.21.15.19:0/2083655661 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb4c0d1020 0x7ffb4c0d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.254+0000 7ffb4affd640 1 -- 172.21.15.19:0/2083655661 >> v1:172.21.15.112:6789/0 conn(0x7ffb4c15dbe0 legacy=0x7ffb4c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.254+0000 7ffb4affd640 1 -- 172.21.15.19:0/2083655661 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c0d1c50 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.255+0000 7ffb4affd640 1 --2- 172.21.15.19:0/2083655661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb4c0b2cb0 0x7ffb4c0d2730 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009870 tx=0x7ffb4002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abdbb00518962deb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.255+0000 7ffb4a7fc640 1 -- 172.21.15.19:0/2083655661 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40037070 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.255+0000 7ffb4a7fc640 1 -- 172.21.15.19:0/2083655661 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb40032440 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.255+0000 7ffb4a7fc640 1 -- 172.21.15.19:0/2083655661 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40032810 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.256+0000 7ffb51c88640 1 -- 172.21.15.19:0/2083655661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb4c0b2cb0 msgr2=0x7ffb4c0d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.256+0000 7ffb51c88640 1 --2- 172.21.15.19:0/2083655661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb4c0b2cb0 0x7ffb4c0d2730 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009870 tx=0x7ffb4002a270 comp rx=0 tx=0).stop 2024-04-25T10:05:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.256+0000 7ffb51c88640 1 -- 172.21.15.19:0/2083655661 shutdown_connections 2024-04-25T10:05:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.256+0000 7ffb51c88640 1 --2- 172.21.15.19:0/2083655661 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb4c0d1020 0x7ffb4c0d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.256+0000 7ffb51c88640 1 --2- 172.21.15.19:0/2083655661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb4c0b2cb0 0x7ffb4c0d2730 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.256+0000 7ffb51c88640 1 -- 172.21.15.19:0/2083655661 >> 172.21.15.19:0/2083655661 conn(0x7ffb4c061a20 msgr2=0x7ffb4c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.256+0000 7ffb51c88640 1 -- 172.21.15.19:0/2083655661 shutdown_connections 2024-04-25T10:05:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.256+0000 7ffb51c88640 1 -- 172.21.15.19:0/2083655661 wait complete. 2024-04-25T10:05:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.257+0000 7ffb51c88640 1 Processor -- start 2024-04-25T10:05:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.257+0000 7ffb51c88640 1 -- start start 2024-04-25T10:05:55.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.258+0000 7ffb51c88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb4c0b2cb0 0x7ffb4c1f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.258+0000 7ffb51c88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb4c0d1020 0x7ffb4c15e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.258+0000 7ffb51c88640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb4c15dbe0 0x7ffb4c0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.258+0000 7ffb51c88640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0b92a0 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.258+0000 7ffb51c88640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0d2210 con 0x7ffb4c0d1020 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.258+0000 7ffb51c88640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0b9420 con 0x7ffb4c15dbe0 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.258+0000 7ffb4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb4c0b2cb0 0x7ffb4c1f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.258+0000 7ffb4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb4c0b2cb0 0x7ffb4c1f6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47026/0 (socket says 172.21.15.19:47026) 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.258+0000 7ffb4b7fe640 1 -- 172.21.15.19:0/1936010229 learned_addr learned my addr 172.21.15.19:0/1936010229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.258+0000 7ffb4affd640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb4c0d1020 0x7ffb4c15e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.258+0000 7ffb4bfff640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb4c15dbe0 0x7ffb4c0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.259+0000 7ffb4b7fe640 1 -- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb4c15dbe0 msgr2=0x7ffb4c0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.259+0000 7ffb4b7fe640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb4c15dbe0 0x7ffb4c0b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.259+0000 7ffb4b7fe640 1 -- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb4c0d1020 msgr2=0x7ffb4c15e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.259+0000 7ffb4b7fe640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb4c0d1020 0x7ffb4c15e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.259+0000 7ffb4b7fe640 1 -- 172.21.15.19:0/1936010229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c15ec10 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.259+0000 7ffb4b7fe640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb4c0b2cb0 0x7ffb4c1f6620 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00c3c0 tx=0x7ffb3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.259+0000 7ffb33fff640 1 -- 172.21.15.19:0/1936010229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c0074e0 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.259+0000 7ffb33fff640 1 -- 172.21.15.19:0/1936010229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c007680 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.259+0000 7ffb51c88640 1 -- 172.21.15.19:0/1936010229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c15eef0 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.259+0000 7ffb33fff640 1 -- 172.21.15.19:0/1936010229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c004a00 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.259+0000 7ffb51c88640 1 -- 172.21.15.19:0/1936010229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c15f4b0 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.260+0000 7ffb51c88640 1 -- 172.21.15.19:0/1936010229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c073990 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.261+0000 7ffb33fff640 1 -- 172.21.15.19:0/1936010229 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb3c01a020 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.261+0000 7ffb33fff640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb3806b4d0 0x7ffb3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.261+0000 7ffb33fff640 1 -- 172.21.15.19:0/1936010229 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ffb3c0899e0 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.262+0000 7ffb4affd640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb3806b4d0 0x7ffb3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.262+0000 7ffb4affd640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb3806b4d0 0x7ffb3806d990 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ffb40034000 tx=0x7ffb4002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.266+0000 7ffb33fff640 1 -- 172.21.15.19:0/1936010229 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb3c058050 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.485+0000 7ffb51c88640 1 -- 172.21.15.19:0/1936010229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb4c0d4790 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.485+0000 7ffb33fff640 1 -- 172.21.15.19:0/1936010229 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4956 (secure 0 0 0) 0x7ffb3c05c0f0 con 0x7ffb4c0b2cb0 2024-04-25T10:05:55.492 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:55.492 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":128,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:38.424174+0000","modified":"2024-04-25T10:05:54.392922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":192,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11799":{"gid":11799,"name":"c","rank":0,"incarnation":126,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.19:6835/1758455116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1758455116},{"type":"v1","addr":"172.21.15.19:6835","nonce":1758455116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:05:55.492 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 128 2024-04-25T10:05:55.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 -- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb3806b4d0 msgr2=0x7ffb3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb3806b4d0 0x7ffb3806d990 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ffb40034000 tx=0x7ffb4002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:55.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 -- 172.21.15.19:0/1936010229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb4c0b2cb0 msgr2=0x7ffb4c1f6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb4c0b2cb0 0x7ffb4c1f6620 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00c3c0 tx=0x7ffb3c00c880 comp rx=0 tx=0).stop 2024-04-25T10:05:55.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 -- 172.21.15.19:0/1936010229 shutdown_connections 2024-04-25T10:05:55.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffb3806b4d0 0x7ffb3806d990 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb4c15dbe0 0x7ffb4c0b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb4c0d1020 0x7ffb4c15e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 --2- 172.21.15.19:0/1936010229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb4c0b2cb0 0x7ffb4c1f6620 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 -- 172.21.15.19:0/1936010229 >> 172.21.15.19:0/1936010229 conn(0x7ffb4c061a20 msgr2=0x7ffb4c0d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:55.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 -- 172.21.15.19:0/1936010229 shutdown_connections 2024-04-25T10:05:55.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:55.487+0000 7ffb51c88640 1 -- 172.21.15.19:0/1936010229 wait complete. 2024-04-25T10:05:55.907 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 128, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:38.424174+0000', 'modified': '2024-04-25T10:05:54.392922+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 192, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11799': {'gid': 11799, 'name': 'c', 'rank': 0, 'incarnation': 126, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.19:6835/1758455116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 1758455116}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 1758455116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:05:55.907 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11799:up:rejoin 2024-04-25T10:05:56.908 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.015+0000 7f9142977640 1 Processor -- start 2024-04-25T10:05:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.015+0000 7f9142977640 1 -- start start 2024-04-25T10:05:57.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.016+0000 7f9142977640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f913c0cfec0 0x7f913c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.016+0000 7f9142977640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f913c0ce3c0 0x7f913c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.016+0000 7f9142977640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c071480 con 0x7f913c154ee0 2024-04-25T10:05:57.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.016+0000 7f9142977640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c0ce830 con 0x7f913c0cfec0 2024-04-25T10:05:57.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.016+0000 7f9142977640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c072090 con 0x7f913c0ce3c0 2024-04-25T10:05:57.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.016+0000 7f913b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f913c0cfec0 0x7f913c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.016+0000 7f9140f14640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f913c0ce3c0 0x7f913c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.016+0000 7f913bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f913c154ee0 0x7f913c0cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:52316/0 (socket says 172.21.15.19:52316) 2024-04-25T10:05:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.017+0000 7f913bfff640 1 -- 172.21.15.19:0/1517024016 learned_addr learned my addr 172.21.15.19:0/1517024016 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.017+0000 7f913b7fe640 1 -- 172.21.15.19:0/1517024016 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f913c0ce3c0 msgr2=0x7f913c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.017+0000 7f913b7fe640 1 --2- 172.21.15.19:0/1517024016 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f913c0ce3c0 0x7f913c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.017+0000 7f913b7fe640 1 -- 172.21.15.19:0/1517024016 >> v1:172.21.15.112:6789/0 conn(0x7f913c154ee0 legacy=0x7f913c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.017+0000 7f913b7fe640 1 -- 172.21.15.19:0/1517024016 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c071760 con 0x7f913c0cfec0 2024-04-25T10:05:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.017+0000 7f9140f14640 1 --2- 172.21.15.19:0/1517024016 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f913c0ce3c0 0x7f913c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:57.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.018+0000 7f913b7fe640 1 --2- 172.21.15.19:0/1517024016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f913c0cfec0 0x7f913c0cde80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f912c00ba40 tx=0x7f912c02c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d42dc422409bc39b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.018+0000 7f913affd640 1 -- 172.21.15.19:0/1517024016 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c032750 con 0x7f913c0cfec0 2024-04-25T10:05:57.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.018+0000 7f913affd640 1 -- 172.21.15.19:0/1517024016 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c030050 con 0x7f913c0cfec0 2024-04-25T10:05:57.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.018+0000 7f913affd640 1 -- 172.21.15.19:0/1517024016 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c032b10 con 0x7f913c0cfec0 2024-04-25T10:05:57.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.018+0000 7f9142977640 1 -- 172.21.15.19:0/1517024016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f913c0cfec0 msgr2=0x7f913c0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.018+0000 7f9142977640 1 --2- 172.21.15.19:0/1517024016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f913c0cfec0 0x7f913c0cde80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f912c00ba40 tx=0x7f912c02c850 comp rx=0 tx=0).stop 2024-04-25T10:05:57.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.018+0000 7f9142977640 1 -- 172.21.15.19:0/1517024016 shutdown_connections 2024-04-25T10:05:57.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.018+0000 7f9142977640 1 --2- 172.21.15.19:0/1517024016 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f913c0ce3c0 0x7f913c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.018+0000 7f9142977640 1 --2- 172.21.15.19:0/1517024016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f913c0cfec0 0x7f913c0cde80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.018+0000 7f9142977640 1 -- 172.21.15.19:0/1517024016 >> 172.21.15.19:0/1517024016 conn(0x7f913c06abe0 msgr2=0x7f913c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.019+0000 7f9142977640 1 -- 172.21.15.19:0/1517024016 shutdown_connections 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.019+0000 7f9142977640 1 -- 172.21.15.19:0/1517024016 wait complete. 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.020+0000 7f9142977640 1 Processor -- start 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.020+0000 7f9142977640 1 -- start start 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f9142977640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f913c0ce3c0 0x7f913c144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f9142977640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c0cfec0 0x7f913c155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f9142977640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f913c154ee0 0x7f913c150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f9142977640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c071d20 con 0x7f913c0cfec0 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f9142977640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c071230 con 0x7f913c0ce3c0 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f9142977640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c1552c0 con 0x7f913c154ee0 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f913b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c0cfec0 0x7f913c155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f913b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c0cfec0 0x7f913c155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47038/0 (socket says 172.21.15.19:47038) 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f913b7fe640 1 -- 172.21.15.19:0/2261737111 learned_addr learned my addr 172.21.15.19:0/2261737111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f9140f14640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f913c154ee0 0x7f913c150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f913bfff640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f913c0ce3c0 0x7f913c144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f913b7fe640 1 -- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f913c154ee0 msgr2=0x7f913c150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f913b7fe640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f913c154ee0 0x7f913c150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f913b7fe640 1 -- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f913c0ce3c0 msgr2=0x7f913c144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f913b7fe640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f913c0ce3c0 0x7f913c144100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f913b7fe640 1 -- 172.21.15.19:0/2261737111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c155b50 con 0x7f913c0cfec0 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.021+0000 7f913b7fe640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c0cfec0 0x7f913c155610 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f912c00bb30 tx=0x7f912c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.022+0000 7f9138ff9640 1 -- 172.21.15.19:0/2261737111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c034a10 con 0x7f913c0cfec0 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.022+0000 7f9138ff9640 1 -- 172.21.15.19:0/2261737111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c034bb0 con 0x7f913c0cfec0 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.022+0000 7f9142977640 1 -- 172.21.15.19:0/2261737111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c14eaf0 con 0x7f913c0cfec0 2024-04-25T10:05:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.022+0000 7f9138ff9640 1 -- 172.21.15.19:0/2261737111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c03f420 con 0x7f913c0cfec0 2024-04-25T10:05:57.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.022+0000 7f9142977640 1 -- 172.21.15.19:0/2261737111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c14f0b0 con 0x7f913c0cfec0 2024-04-25T10:05:57.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.023+0000 7f9142977640 1 -- 172.21.15.19:0/2261737111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c073990 con 0x7f913c0cfec0 2024-04-25T10:05:57.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.023+0000 7f9138ff9640 1 -- 172.21.15.19:0/2261737111 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f912c03c040 con 0x7f913c0cfec0 2024-04-25T10:05:57.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.024+0000 7f9138ff9640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911c06b4d0 0x7f911c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.024+0000 7f9138ff9640 1 -- 172.21.15.19:0/2261737111 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f912c0adb90 con 0x7f913c0cfec0 2024-04-25T10:05:57.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.024+0000 7f913bfff640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911c06b4d0 0x7f911c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.024+0000 7f913bfff640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911c06b4d0 0x7f911c06d990 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9124000f60 tx=0x7f9124009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.028+0000 7f9138ff9640 1 -- 172.21.15.19:0/2261737111 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f912c043030 con 0x7f913c0cfec0 2024-04-25T10:05:57.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.245+0000 7f9142977640 1 -- 172.21.15.19:0/2261737111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f913c06d360 con 0x7f913c0cfec0 2024-04-25T10:05:57.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.246+0000 7f9138ff9640 1 -- 172.21.15.19:0/2261737111 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+4956 (secure 0 0 0) 0x7f912c07c730 con 0x7f913c0cfec0 2024-04-25T10:05:57.253 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:05:57.253 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":129,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:38.424174+0000","modified":"2024-04-25T10:05:55.398362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":192,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11799":{"gid":11799,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/1758455116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1758455116},{"type":"v1","addr":"172.21.15.19:6835","nonce":1758455116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:05:57.253 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 129 2024-04-25T10:05:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 -- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911c06b4d0 msgr2=0x7f911c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911c06b4d0 0x7f911c06d990 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9124000f60 tx=0x7f9124009040 comp rx=0 tx=0).stop 2024-04-25T10:05:57.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 -- 172.21.15.19:0/2261737111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c0cfec0 msgr2=0x7f913c155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c0cfec0 0x7f913c155610 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f912c00bb30 tx=0x7f912c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 -- 172.21.15.19:0/2261737111 shutdown_connections 2024-04-25T10:05:57.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f911c06b4d0 0x7f911c06d990 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f913c154ee0 0x7f913c150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f913c0cfec0 0x7f913c155610 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 --2- 172.21.15.19:0/2261737111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f913c0ce3c0 0x7f913c144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 -- 172.21.15.19:0/2261737111 >> 172.21.15.19:0/2261737111 conn(0x7f913c06abe0 msgr2=0x7f913c0b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 -- 172.21.15.19:0/2261737111 shutdown_connections 2024-04-25T10:05:57.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.248+0000 7f9142977640 1 -- 172.21.15.19:0/2261737111 wait complete. 2024-04-25T10:05:57.669 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 129, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:38.424174+0000', 'modified': '2024-04-25T10:05:55.398362+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 192, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11799': {'gid': 11799, 'name': 'c', 'rank': 0, 'incarnation': 126, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6835/1758455116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 1758455116}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 1758455116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:05:57.669 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:05:57.669 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:05:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.769+0000 7f91ef49e640 1 Processor -- start 2024-04-25T10:05:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.769+0000 7f91ef49e640 1 -- start start 2024-04-25T10:05:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.770+0000 7f91ef49e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91e80d0ab0 0x7f91e8168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.770+0000 7f91ef49e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e80d2420 con 0x7f91e80d34c0 2024-04-25T10:05:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.770+0000 7f91ef49e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e80d2700 con 0x7f91e815dac0 2024-04-25T10:05:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.770+0000 7f91ef49e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e80d3030 con 0x7f91e80d0ab0 2024-04-25T10:05:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.770+0000 7f91eca39640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f91e80d34c0 0x7f91e80d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38034/0 (socket says 172.21.15.19:38034) 2024-04-25T10:05:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.770+0000 7f91eca39640 1 -- 172.21.15.19:0/2904857568 learned_addr learned my addr 172.21.15.19:0/2904857568 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.770+0000 7f91eda3b640 1 --2- 172.21.15.19:0/2904857568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91e80d0ab0 0x7f91e8168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.770+0000 7f91dffff640 1 -- 172.21.15.19:0/2904857568 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500391984 0 0) 0x7f91e80d2420 con 0x7f91e80d34c0 2024-04-25T10:05:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.770+0000 7f91dffff640 1 -- 172.21.15.19:0/2904857568 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91d4003520 con 0x7f91e80d34c0 2024-04-25T10:05:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.770+0000 7f91dffff640 1 -- 172.21.15.19:0/2904857568 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712394348 0 0) 0x7f91e80d2700 con 0x7f91e815dac0 2024-04-25T10:05:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.770+0000 7f91dffff640 1 -- 172.21.15.19:0/2904857568 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e80d2420 con 0x7f91e815dac0 2024-04-25T10:05:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.771+0000 7f91dffff640 1 -- 172.21.15.19:0/2904857568 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2378282470 0 0) 0x7f91d4003520 con 0x7f91e80d34c0 2024-04-25T10:05:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.771+0000 7f91eda3b640 1 -- 172.21.15.19:0/2904857568 >> v1:172.21.15.112:6789/0 conn(0x7f91e815dac0 legacy=0x7f91e80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.771+0000 7f91eda3b640 1 -- 172.21.15.19:0/2904857568 >> v1:172.21.15.19:6789/0 conn(0x7f91e80d34c0 legacy=0x7f91e80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.771+0000 7f91eda3b640 1 -- 172.21.15.19:0/2904857568 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e80d29e0 con 0x7f91e80d0ab0 2024-04-25T10:05:57.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.771+0000 7f91eda3b640 1 --2- 172.21.15.19:0/2904857568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91e80d0ab0 0x7f91e8168670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f91e80d31b0 tx=0x7f91e002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e25614817980219 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.771+0000 7f91dffff640 1 -- 172.21.15.19:0/2904857568 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d80016c0 con 0x7f91e80d0ab0 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.771+0000 7f91dffff640 1 -- 172.21.15.19:0/2904857568 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e002ae50 con 0x7f91e80d0ab0 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.772+0000 7f91dffff640 1 -- 172.21.15.19:0/2904857568 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0007760 con 0x7f91e80d0ab0 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.772+0000 7f91ef49e640 1 -- 172.21.15.19:0/2904857568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91e80d0ab0 msgr2=0x7f91e8168670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.772+0000 7f91ef49e640 1 --2- 172.21.15.19:0/2904857568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91e80d0ab0 0x7f91e8168670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f91e80d31b0 tx=0x7f91e002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.772+0000 7f91ef49e640 1 -- 172.21.15.19:0/2904857568 shutdown_connections 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.772+0000 7f91ef49e640 1 --2- 172.21.15.19:0/2904857568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91e80d0ab0 0x7f91e8168670 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.772+0000 7f91ef49e640 1 -- 172.21.15.19:0/2904857568 >> 172.21.15.19:0/2904857568 conn(0x7f91e8061c30 msgr2=0x7f91e806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.772+0000 7f91ef49e640 1 -- 172.21.15.19:0/2904857568 shutdown_connections 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.772+0000 7f91ef49e640 1 -- 172.21.15.19:0/2904857568 wait complete. 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.773+0000 7f91ef49e640 1 Processor -- start 2024-04-25T10:05:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.773+0000 7f91ef49e640 1 -- start start 2024-04-25T10:05:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91ef49e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91e80d0ab0 0x7f91e8152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91ef49e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e80d34c0 0x7f91e80b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91ef49e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91e815dac0 0x7f91e80cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91ef49e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e815dea0 con 0x7f91e815dac0 2024-04-25T10:05:57.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91ef49e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e80d2100 con 0x7f91e80d34c0 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91ef49e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e80b97d0 con 0x7f91e80d0ab0 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91eda3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91e815dac0 0x7f91e80cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91eda3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91e815dac0 0x7f91e80cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32886/0 (socket says 172.21.15.19:32886) 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91eda3b640 1 -- 172.21.15.19:0/2861532015 learned_addr learned my addr 172.21.15.19:0/2861532015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91eca39640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e80d34c0 0x7f91e80b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91ed23a640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91e80d0ab0 0x7f91e8152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91eda3b640 1 -- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91e80d0ab0 msgr2=0x7f91e8152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91eda3b640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91e80d0ab0 0x7f91e8152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91eda3b640 1 -- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e80d34c0 msgr2=0x7f91e80b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91eda3b640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e80d34c0 0x7f91e80b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.774+0000 7f91eda3b640 1 -- 172.21.15.19:0/2861532015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e80b5db0 con 0x7f91e815dac0 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.775+0000 7f91eda3b640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91e815dac0 0x7f91e80cd0b0 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009d90 tx=0x7f91e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.775+0000 7f91ddffb640 1 -- 172.21.15.19:0/2861532015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0040070 con 0x7f91e815dac0 2024-04-25T10:05:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.775+0000 7f91ddffb640 1 -- 172.21.15.19:0/2861532015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e003c830 con 0x7f91e815dac0 2024-04-25T10:05:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.775+0000 7f91ddffb640 1 -- 172.21.15.19:0/2861532015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e003cbe0 con 0x7f91e815dac0 2024-04-25T10:05:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.775+0000 7f91ef49e640 1 -- 172.21.15.19:0/2861532015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e80b4d10 con 0x7f91e815dac0 2024-04-25T10:05:57.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.775+0000 7f91ef49e640 1 -- 172.21.15.19:0/2861532015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e80b55b0 con 0x7f91e815dac0 2024-04-25T10:05:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.777+0000 7f91ddffb640 1 -- 172.21.15.19:0/2861532015 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91e0039040 con 0x7f91e815dac0 2024-04-25T10:05:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.777+0000 7f91ddffb640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f91cc06b4d0 0x7f91cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.777+0000 7f91ddffb640 1 -- 172.21.15.19:0/2861532015 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f91e00ab520 con 0x7f91e815dac0 2024-04-25T10:05:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.778+0000 7f91ed23a640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f91cc06b4d0 0x7f91cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.778+0000 7f91ed23a640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f91cc06b4d0 0x7f91cc06d990 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f91e40011e0 tx=0x7f91e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.778+0000 7f91f21a7740 1 Processor -- start 2024-04-25T10:05:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.778+0000 7f91f21a7740 1 -- start start 2024-04-25T10:05:57.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f5ee230 0x557f5f5ee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f60bad0 0x557f5f60c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5c3f10 0x557f5f60a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91f21a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5f526a10 con 0x557f5f5c3f10 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91f21a7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5f4cfb30 con 0x557f5f5ee230 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91f21a7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f5f522c50 con 0x557f5f60bad0 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91eda3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5c3f10 0x557f5f60a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91eda3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5c3f10 0x557f5f60a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32888/0 (socket says 172.21.15.19:32888) 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91eda3b640 1 -- 172.21.15.19:0/1464203906 learned_addr learned my addr 172.21.15.19:0/1464203906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91ed23a640 1 --2- 172.21.15.19:0/1464203906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f5ee230 0x557f5f5ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91eca39640 1 --2- 172.21.15.19:0/1464203906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f60bad0 0x557f5f60c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91eda3b640 1 -- 172.21.15.19:0/1464203906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f60bad0 msgr2=0x557f5f60c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91eda3b640 1 --2- 172.21.15.19:0/1464203906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f60bad0 0x557f5f60c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91eda3b640 1 -- 172.21.15.19:0/1464203906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f5ee230 msgr2=0x557f5f5ee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91eda3b640 1 --2- 172.21.15.19:0/1464203906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f5ee230 0x557f5f5ee610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.779+0000 7f91eda3b640 1 -- 172.21.15.19:0/1464203906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f5f4e4e10 con 0x557f5f5c3f10 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91eda3b640 1 --2- 172.21.15.19:0/1464203906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5c3f10 0x557f5f60a190 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7f91e00a97a0 tx=0x7f91e0079640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287028a51539e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91caffd640 1 -- 172.21.15.19:0/1464203906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0077f20 con 0x557f5f5c3f10 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91caffd640 1 -- 172.21.15.19:0/1464203906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e0076290 con 0x557f5f5c3f10 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91caffd640 1 -- 172.21.15.19:0/1464203906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e007ede0 con 0x557f5f5c3f10 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91f21a7740 1 -- 172.21.15.19:0/1464203906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5c3f10 msgr2=0x557f5f60a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91f21a7740 1 --2- 172.21.15.19:0/1464203906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5c3f10 0x557f5f60a190 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7f91e00a97a0 tx=0x7f91e0079640 comp rx=0 tx=0).stop 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91f21a7740 1 -- 172.21.15.19:0/1464203906 shutdown_connections 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91f21a7740 1 --2- 172.21.15.19:0/1464203906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5c3f10 0x557f5f60a190 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91f21a7740 1 --2- 172.21.15.19:0/1464203906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f60bad0 0x557f5f60c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91f21a7740 1 --2- 172.21.15.19:0/1464203906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f5ee230 0x557f5f5ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91f21a7740 1 -- 172.21.15.19:0/1464203906 >> 172.21.15.19:0/1464203906 conn(0x557f5f5c0840 msgr2=0x557f5f609040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91f21a7740 1 -- 172.21.15.19:0/1464203906 shutdown_connections 2024-04-25T10:05:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.780+0000 7f91f21a7740 1 -- 172.21.15.19:0/1464203906 wait complete. 2024-04-25T10:05:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.781+0000 7f91f21a7740 1 -- start start 2024-04-25T10:05:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.781+0000 7f91f21a7740 1 Processor -- start 2024-04-25T10:05:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f5ee230 0x557f5f5d99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f60bad0 0x557f5f60e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60ef20 0x557f5f5bd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91f21a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5f5c4cb0 con 0x557f5f60ef20 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91f21a7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5f4e30c0 con 0x557f5f5ee230 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91f21a7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f5f5be770 con 0x557f5f60bad0 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91f21a7740 10 client.0 _ll_register_callbacks cb 0x557f5f4d6ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91eda3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60ef20 0x557f5f5bd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91eda3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60ef20 0x557f5f5bd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32894/0 (socket says 172.21.15.19:32894) 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91eda3b640 1 -- 172.21.15.19:0/34829926 learned_addr learned my addr 172.21.15.19:0/34829926 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91ed23a640 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f5ee230 0x557f5f5d99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91eca39640 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f60bad0 0x557f5f60e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91eda3b640 1 -- 172.21.15.19:0/34829926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f60bad0 msgr2=0x557f5f60e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91eda3b640 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f60bad0 0x557f5f60e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91eda3b640 1 -- 172.21.15.19:0/34829926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f5ee230 msgr2=0x557f5f5d99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91eda3b640 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f5ee230 0x557f5f5d99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91eda3b640 1 -- 172.21.15.19:0/34829926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f5f23b2d0 con 0x557f5f60ef20 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91eca39640 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f60bad0 0x557f5f60e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91eda3b640 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60ef20 0x557f5f5bd4a0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f91e002a730 tx=0x7f91e0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa71f5a8b5bb7f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.782+0000 7f91c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91c3fff640 1 -- 172.21.15.19:0/34829926 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0079e80 con 0x557f5f60ef20 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91c3fff640 1 -- 172.21.15.19:0/34829926 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f91e007a020 con 0x557f5f60ef20 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91f21a7740 1 -- 172.21.15.19:0/34829926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91e0076290 con 0x557f5f60ef20 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91c3fff640 1 -- 172.21.15.19:0/34829926 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91e0081df0 con 0x557f5f60ef20 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91f21a7740 10 client.11850 fetch_fsmap learned FSMap version 129 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91f21a7740 1 -- 172.21.15.19:0/34829926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x557f5f3056b0 con 0x557f5f60ef20 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91c3fff640 1 -- 172.21.15.19:0/34829926 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e0076290 con 0x557f5f60ef20 2024-04-25T10:05:57.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91c3fff640 1 -- 172.21.15.19:0/34829926 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91e00a7040 con 0x557f5f60ef20 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91f21a7740 10 client.11850 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91f21a7740 10 client.11850 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91f21a7740 20 client.11850 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91f21a7740 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x557f5f624e50 0x557f5f627240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91f21a7740 4 client.11850 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91f21a7740 1 -- 172.21.15.19:0/34829926 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f5f585920 con 0x557f5f624e50 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.783+0000 7f91ed23a640 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x557f5f624e50 0x557f5f627240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91ed23a640 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x557f5f624e50 0x557f5f627240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=211b52e30d50c3e0 server_cookie=3ac10af9446b9830 in_seq=0 out_seq=0 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91c3fff640 10 client.11850 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91c3fff640 1 -- 172.21.15.19:0/34829926 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f5f585920 con 0x557f5f624e50 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91c3fff640 10 client.11850 handle_command_reply: tid=0 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91f21a7740 1 client.11850 shutdown 2024-04-25T10:05:57.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91f21a7740 1 -- 172.21.15.19:0/34829926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60ef20 msgr2=0x557f5f5bd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91f21a7740 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60ef20 0x557f5f5bd4a0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f91e002a730 tx=0x7f91e0009750 comp rx=0 tx=0).stop 2024-04-25T10:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91f21a7740 1 -- 172.21.15.19:0/34829926 shutdown_connections 2024-04-25T10:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91f21a7740 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x557f5f624e50 0x557f5f627240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91f21a7740 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60ef20 0x557f5f5bd4a0 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91f21a7740 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f60bad0 0x557f5f60e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91c3fff640 0 client.11850 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91f21a7740 1 --2- 172.21.15.19:0/34829926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f5ee230 0x557f5f5d99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.784+0000 7f91f21a7740 1 -- 172.21.15.19:0/34829926 >> 172.21.15.19:0/34829926 conn(0x557f5f5c3f10 msgr2=0x557f5f60c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.785+0000 7f91f21a7740 1 -- 172.21.15.19:0/34829926 shutdown_connections 2024-04-25T10:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.785+0000 7f91f21a7740 1 -- 172.21.15.19:0/34829926 wait complete. 2024-04-25T10:05:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.785+0000 7f91f21a7740 20 client.11850 trim_cache size 0 max 16384 2024-04-25T10:05:57.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.800+0000 7f91f21a7740 1 Processor -- start 2024-04-25T10:05:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.800+0000 7f91f21a7740 1 -- start start 2024-04-25T10:05:57.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f5ee230 0x557f5f60fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60bad0 0x557f5f6101d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f60ef20 0x557f5f66aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91f21a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e81ff890 con 0x557f5f60bad0 2024-04-25T10:05:57.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91f21a7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e815f660 con 0x557f5f60ef20 2024-04-25T10:05:57.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91f21a7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e815fc60 con 0x557f5f5ee230 2024-04-25T10:05:57.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91eca39640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60bad0 0x557f5f6101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91eca39640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60bad0 0x557f5f6101d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32898/0 (socket says 172.21.15.19:32898) 2024-04-25T10:05:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91eca39640 1 -- 172.21.15.19:0/1935325034 learned_addr learned my addr 172.21.15.19:0/1935325034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91eca39640 1 -- 172.21.15.19:0/1935325034 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f5ee230 msgr2=0x557f5f60fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91eca39640 1 --2- 172.21.15.19:0/1935325034 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f5ee230 0x557f5f60fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91eca39640 1 -- 172.21.15.19:0/1935325034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f60ef20 msgr2=0x557f5f66aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91eca39640 1 --2- 172.21.15.19:0/1935325034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f60ef20 0x557f5f66aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.801+0000 7f91eca39640 1 -- 172.21.15.19:0/1935325034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f5f55b460 con 0x557f5f60bad0 2024-04-25T10:05:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91eca39640 1 --2- 172.21.15.19:0/1935325034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60bad0 0x557f5f6101d0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f91d8006c00 tx=0x7f91d8001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a94ee3c85344f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91ca7fc640 1 -- 172.21.15.19:0/1935325034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8006cf0 con 0x557f5f60bad0 2024-04-25T10:05:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91ca7fc640 1 -- 172.21.15.19:0/1935325034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8016420 con 0x557f5f60bad0 2024-04-25T10:05:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91ca7fc640 1 -- 172.21.15.19:0/1935325034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d80167d0 con 0x557f5f60bad0 2024-04-25T10:05:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91f21a7740 1 -- 172.21.15.19:0/1935325034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60bad0 msgr2=0x557f5f6101d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91f21a7740 1 --2- 172.21.15.19:0/1935325034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60bad0 0x557f5f6101d0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f91d8006c00 tx=0x7f91d8001630 comp rx=0 tx=0).stop 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91f21a7740 1 -- 172.21.15.19:0/1935325034 shutdown_connections 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91f21a7740 1 --2- 172.21.15.19:0/1935325034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f60ef20 0x557f5f66aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91f21a7740 1 --2- 172.21.15.19:0/1935325034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f60bad0 0x557f5f6101d0 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91f21a7740 1 --2- 172.21.15.19:0/1935325034 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f5ee230 0x557f5f60fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91f21a7740 1 -- 172.21.15.19:0/1935325034 >> 172.21.15.19:0/1935325034 conn(0x557f5f5c3f10 msgr2=0x557f5f608f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91f21a7740 1 -- 172.21.15.19:0/1935325034 shutdown_connections 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.802+0000 7f91f21a7740 1 -- 172.21.15.19:0/1935325034 wait complete. 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.803+0000 7f91f21a7740 1 -- start start 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.803+0000 7f91f21a7740 1 Processor -- start 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5ee230 0x557f5f666e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f60ef20 0x557f5f5f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91f21a7740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f5ba080 0x557f5f60d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91f21a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5f5c4cb0 con 0x557f5f5ee230 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91f21a7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5f4e30c0 con 0x557f5f60ef20 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91f21a7740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f5f61d040 con 0x557f5f5ba080 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91f21a7740 10 client.0 _ll_register_callbacks cb 0x557f5f5bec50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91ed23a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5ee230 0x557f5f666e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91ed23a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5ee230 0x557f5f666e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32914/0 (socket says 172.21.15.19:32914) 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91ed23a640 1 -- 172.21.15.19:0/144387651 learned_addr learned my addr 172.21.15.19:0/144387651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91eda3b640 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f5ba080 0x557f5f60d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91eca39640 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f60ef20 0x557f5f5f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91ed23a640 1 -- 172.21.15.19:0/144387651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f5ba080 msgr2=0x557f5f60d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91ed23a640 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f5ba080 0x557f5f60d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91ed23a640 1 -- 172.21.15.19:0/144387651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f60ef20 msgr2=0x557f5f5f1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91ed23a640 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f60ef20 0x557f5f5f1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91ed23a640 1 -- 172.21.15.19:0/144387651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f5f60b620 con 0x557f5f5ee230 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91eda3b640 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f5ba080 0x557f5f60d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.804+0000 7f91ed23a640 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5ee230 0x557f5f666e70 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f91e400f5a0 tx=0x7f91e4007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a86aa1d73fb9812 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91c3fff640 1 -- 172.21.15.19:0/144387651 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e402f060 con 0x557f5f5ee230 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91c3fff640 1 -- 172.21.15.19:0/144387651 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f91e4014450 con 0x557f5f5ee230 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91f21a7740 1 -- 172.21.15.19:0/144387651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91d8016420 con 0x557f5f5ee230 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91c3fff640 1 -- 172.21.15.19:0/144387651 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91e4035050 con 0x557f5f5ee230 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91f21a7740 10 client.11856 fetch_fsmap learned FSMap version 129 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91f21a7740 1 -- 172.21.15.19:0/144387651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x557f5f60e170 con 0x557f5f5ee230 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91c3fff640 1 -- 172.21.15.19:0/144387651 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8016420 con 0x557f5f5ee230 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91c3fff640 1 -- 172.21.15.19:0/144387651 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91e4038020 con 0x557f5f5ee230 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91f21a7740 10 client.11856 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:05:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91f21a7740 10 client.11856 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91f21a7740 20 client.11856 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91f21a7740 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x557f5f60bf00 0x557f5f60c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91f21a7740 4 client.11856 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91f21a7740 1 -- 172.21.15.19:0/144387651 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557f5f533f50 con 0x557f5f60bf00 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.805+0000 7f91eca39640 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x557f5f60bf00 0x557f5f60c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91c3fff640 10 client.11856 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91eca39640 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x557f5f60bf00 0x557f5f60c2e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a35cafafa6b44bfd server_cookie=f67933dfc4986db5 in_seq=0 out_seq=0 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91c3fff640 1 -- 172.21.15.19:0/144387651 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x557f5f533f50 con 0x557f5f60bf00 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91c3fff640 10 client.11856 handle_command_reply: tid=0 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 1 client.11856 shutdown 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 1 -- 172.21.15.19:0/144387651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5ee230 msgr2=0x557f5f666e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5ee230 0x557f5f666e70 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f91e400f5a0 tx=0x7f91e4007d20 comp rx=0 tx=0).stop 2024-04-25T10:05:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 1 -- 172.21.15.19:0/144387651 shutdown_connections 2024-04-25T10:05:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x557f5f60bf00 0x557f5f60c2e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557f5f5ba080 0x557f5f60d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f5f60ef20 0x557f5f5f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91c3fff640 0 client.11856 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 1 --2- 172.21.15.19:0/144387651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557f5f5ee230 0x557f5f666e70 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 1 -- 172.21.15.19:0/144387651 >> 172.21.15.19:0/144387651 conn(0x557f5f5c3f10 msgr2=0x557f5f60fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 1 -- 172.21.15.19:0/144387651 shutdown_connections 2024-04-25T10:05:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 1 -- 172.21.15.19:0/144387651 wait complete. 2024-04-25T10:05:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.806+0000 7f91f21a7740 20 client.11856 trim_cache size 0 max 16384 2024-04-25T10:05:57.818 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:05:57.818 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T10:05:57.819 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:05:57.819 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 11799, 2024-04-25T10:05:57.819 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:05:57.819 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:05:57.819 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:05:57.819 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 4.4181574619999999, 2024-04-25T10:05:57.819 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 129, 2024-04-25T10:05:57.819 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 192, 2024-04-25T10:05:57.819 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 192, 2024-04-25T10:05:57.819 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 5.3906310700000004 2024-04-25T10:05:57.819 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:05:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 -- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f91cc06b4d0 msgr2=0x7f91cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f91cc06b4d0 0x7f91cc06d990 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f91e40011e0 tx=0x7f91e4009040 comp rx=0 tx=0).stop 2024-04-25T10:05:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 -- 172.21.15.19:0/2861532015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91e815dac0 msgr2=0x7f91e80cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91e815dac0 0x7f91e80cd0b0 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009d90 tx=0x7f91e0004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 -- 172.21.15.19:0/2861532015 shutdown_connections 2024-04-25T10:05:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f91cc06b4d0 0x7f91cc06d990 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91e815dac0 0x7f91e80cd0b0 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e80d34c0 0x7f91e80b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 --2- 172.21.15.19:0/2861532015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f91e80d0ab0 0x7f91e8152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 -- 172.21.15.19:0/2861532015 >> 172.21.15.19:0/2861532015 conn(0x7f91e8061c30 msgr2=0x7f91e80c79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 -- 172.21.15.19:0/2861532015 shutdown_connections 2024-04-25T10:05:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.807+0000 7f91ef49e640 1 -- 172.21.15.19:0/2861532015 wait complete. 2024-04-25T10:05:57.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:57.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:05:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.922+0000 7f77f361b640 1 Processor -- start 2024-04-25T10:05:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.922+0000 7f77f361b640 1 -- start start 2024-04-25T10:05:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.923+0000 7f77f361b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f77ec0d34c0 0x7f77ec0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.923+0000 7f77f361b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f77ec0d08e0 0x7f77ec0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.923+0000 7f77f361b640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77ec0d2420 con 0x7f77ec15dac0 2024-04-25T10:05:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.923+0000 7f77f361b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77ec0d0d50 con 0x7f77ec0d34c0 2024-04-25T10:05:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.923+0000 7f77f361b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77ec0d3030 con 0x7f77ec0d08e0 2024-04-25T10:05:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.923+0000 7f77f0bb6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f77ec0d34c0 0x7f77ec0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.923+0000 7f77f13b7640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f77ec15dac0 0x7f77ec0d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38040/0 (socket says 172.21.15.19:38040) 2024-04-25T10:05:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.923+0000 7f77f13b7640 1 -- 172.21.15.19:0/1933434063 learned_addr learned my addr 172.21.15.19:0/1933434063 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.923+0000 7f77f1bb8640 1 --2- 172.21.15.19:0/1933434063 >> v2:172.21.15.112:3300/0 conn(0x7f77ec0d08e0 0x7f77ec0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.924+0000 7f77f0bb6640 1 -- 172.21.15.19:0/1933434063 >> v2:172.21.15.112:3300/0 conn(0x7f77ec0d08e0 msgr2=0x7f77ec0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.924+0000 7f77f0bb6640 1 --2- 172.21.15.19:0/1933434063 >> v2:172.21.15.112:3300/0 conn(0x7f77ec0d08e0 0x7f77ec0b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.924+0000 7f77f0bb6640 1 -- 172.21.15.19:0/1933434063 >> v1:172.21.15.19:6789/0 conn(0x7f77ec15dac0 legacy=0x7f77ec0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.924+0000 7f77f0bb6640 1 -- 172.21.15.19:0/1933434063 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec0d2700 con 0x7f77ec0d34c0 2024-04-25T10:05:57.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.924+0000 7f77f0bb6640 1 --2- 172.21.15.19:0/1933434063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ec0d34c0 0x7f77ec0d03a0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0d2fa0 tx=0x7f77e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253b45431793f7db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.924+0000 7f77ebfff640 1 -- 172.21.15.19:0/1933434063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0037070 con 0x7f77ec0d34c0 2024-04-25T10:05:57.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.924+0000 7f77ebfff640 1 -- 172.21.15.19:0/1933434063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e0032440 con 0x7f77ec0d34c0 2024-04-25T10:05:57.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.924+0000 7f77ebfff640 1 -- 172.21.15.19:0/1933434063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0032810 con 0x7f77ec0d34c0 2024-04-25T10:05:57.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.925+0000 7f77f361b640 1 -- 172.21.15.19:0/1933434063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ec0d34c0 msgr2=0x7f77ec0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.925+0000 7f77f361b640 1 --2- 172.21.15.19:0/1933434063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ec0d34c0 0x7f77ec0d03a0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0d2fa0 tx=0x7f77e002a230 comp rx=0 tx=0).stop 2024-04-25T10:05:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.925+0000 7f77f361b640 1 -- 172.21.15.19:0/1933434063 shutdown_connections 2024-04-25T10:05:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.925+0000 7f77f361b640 1 --2- 172.21.15.19:0/1933434063 >> v2:172.21.15.112:3300/0 conn(0x7f77ec0d08e0 0x7f77ec0b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.925+0000 7f77f361b640 1 --2- 172.21.15.19:0/1933434063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ec0d34c0 0x7f77ec0d03a0 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.925+0000 7f77f361b640 1 -- 172.21.15.19:0/1933434063 >> 172.21.15.19:0/1933434063 conn(0x7f77ec061c30 msgr2=0x7f77ec06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.925+0000 7f77f361b640 1 -- 172.21.15.19:0/1933434063 shutdown_connections 2024-04-25T10:05:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.925+0000 7f77f361b640 1 -- 172.21.15.19:0/1933434063 wait complete. 2024-04-25T10:05:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.926+0000 7f77f361b640 1 Processor -- start 2024-04-25T10:05:57.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.926+0000 7f77f361b640 1 -- start start 2024-04-25T10:05:57.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f361b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ec0d08e0 0x7f77ec0b7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f361b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77ec0d34c0 0x7f77ec0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f361b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77ec15dac0 0x7f77ec0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f361b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec0d2100 con 0x7f77ec0d08e0 2024-04-25T10:05:57.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f361b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec0d2cc0 con 0x7f77ec0d34c0 2024-04-25T10:05:57.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f361b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec15dea0 con 0x7f77ec15dac0 2024-04-25T10:05:57.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f13b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ec0d08e0 0x7f77ec0b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f13b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ec0d08e0 0x7f77ec0b7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32932/0 (socket says 172.21.15.19:32932) 2024-04-25T10:05:57.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f13b7640 1 -- 172.21.15.19:0/1298704765 learned_addr learned my addr 172.21.15.19:0/1298704765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:57.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f1bb8640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77ec15dac0 0x7f77ec0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f0bb6640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77ec0d34c0 0x7f77ec0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f13b7640 1 -- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77ec15dac0 msgr2=0x7f77ec0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77ec15dac0 0x7f77ec0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f13b7640 1 -- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77ec0d34c0 msgr2=0x7f77ec0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77ec0d34c0 0x7f77ec0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.927+0000 7f77f13b7640 1 -- 172.21.15.19:0/1298704765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec1f7710 con 0x7f77ec0d08e0 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.928+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ec0d08e0 0x7f77ec0b7890 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00ca70 tx=0x7f77dc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.928+0000 7f77e9ffb640 1 -- 172.21.15.19:0/1298704765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc017070 con 0x7f77ec0d08e0 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.928+0000 7f77e9ffb640 1 -- 172.21.15.19:0/1298704765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77dc00abd0 con 0x7f77ec0d08e0 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.928+0000 7f77e9ffb640 1 -- 172.21.15.19:0/1298704765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc012400 con 0x7f77ec0d08e0 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.928+0000 7f77f361b640 1 -- 172.21.15.19:0/1298704765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ec1f7980 con 0x7f77ec0d08e0 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.928+0000 7f77f361b640 1 -- 172.21.15.19:0/1298704765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ec1f7f40 con 0x7f77ec0d08e0 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.929+0000 7f77e9ffb640 1 -- 172.21.15.19:0/1298704765 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f77dc01a070 con 0x7f77ec0d08e0 2024-04-25T10:05:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.930+0000 7f77e9ffb640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f77d806b4d0 0x7f77d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.930+0000 7f77e9ffb640 1 -- 172.21.15.19:0/1298704765 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f77dc08a110 con 0x7f77ec0d08e0 2024-04-25T10:05:57.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.930+0000 7f77f0bb6640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f77d806b4d0 0x7f77d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.930+0000 7f77f0bb6640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f77d806b4d0 0x7f77d806d990 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f77e0034000 tx=0x7f77e002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.931+0000 7f77f63f0740 1 Processor -- start 2024-04-25T10:05:57.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.931+0000 7f77f63f0740 1 -- start start 2024-04-25T10:05:57.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.931+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c34bb0 0x555da2c34f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.931+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c2f830 0x555da2c337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.931+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c33cf0 0x555da2c34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.931+0000 7f77f63f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x555da2b98ac0 con 0x555da2c2f830 2024-04-25T10:05:57.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.931+0000 7f77f63f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x555da2b41d80 con 0x555da2c34bb0 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.931+0000 7f77f63f0740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x555da2b94b60 con 0x555da2c33cf0 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77f0bb6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c2f830 0x555da2c337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77f0bb6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c2f830 0x555da2c337b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32948/0 (socket says 172.21.15.19:32948) 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77f0bb6640 1 -- 172.21.15.19:0/3253214568 learned_addr learned my addr 172.21.15.19:0/3253214568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77f13b7640 1 --2- 172.21.15.19:0/3253214568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c34bb0 0x555da2c34f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77f1bb8640 1 --2- 172.21.15.19:0/3253214568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c33cf0 0x555da2c34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77f0bb6640 1 -- 172.21.15.19:0/3253214568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c33cf0 msgr2=0x555da2c34130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77f0bb6640 1 --2- 172.21.15.19:0/3253214568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c33cf0 0x555da2c34130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77f0bb6640 1 -- 172.21.15.19:0/3253214568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c34bb0 msgr2=0x555da2c34f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77f0bb6640 1 --2- 172.21.15.19:0/3253214568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c34bb0 0x555da2c34f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77f0bb6640 1 -- 172.21.15.19:0/3253214568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555da29757e0 con 0x555da2c2f830 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77f0bb6640 1 --2- 172.21.15.19:0/3253214568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c2f830 0x555da2c337b0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f77e0035040 tx=0x7f77e002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e708faaa60e13242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77d2ffd640 1 -- 172.21.15.19:0/3253214568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e004e070 con 0x555da2c2f830 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77d2ffd640 1 -- 172.21.15.19:0/3253214568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e0032d90 con 0x555da2c2f830 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.932+0000 7f77d2ffd640 1 -- 172.21.15.19:0/3253214568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0037040 con 0x555da2c2f830 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.933+0000 7f77f63f0740 1 -- 172.21.15.19:0/3253214568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c2f830 msgr2=0x555da2c337b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.933+0000 7f77f63f0740 1 --2- 172.21.15.19:0/3253214568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c2f830 0x555da2c337b0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f77e0035040 tx=0x7f77e002c000 comp rx=0 tx=0).stop 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.933+0000 7f77f63f0740 1 -- 172.21.15.19:0/3253214568 shutdown_connections 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.933+0000 7f77f63f0740 1 --2- 172.21.15.19:0/3253214568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c33cf0 0x555da2c34130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.933+0000 7f77f63f0740 1 --2- 172.21.15.19:0/3253214568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c2f830 0x555da2c337b0 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.933+0000 7f77f63f0740 1 --2- 172.21.15.19:0/3253214568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c34bb0 0x555da2c34f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.933+0000 7f77f63f0740 1 -- 172.21.15.19:0/3253214568 >> 172.21.15.19:0/3253214568 conn(0x555da2c31cb0 msgr2=0x555da2c7a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.933+0000 7f77f63f0740 1 -- 172.21.15.19:0/3253214568 shutdown_connections 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.933+0000 7f77f63f0740 1 -- 172.21.15.19:0/3253214568 wait complete. 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.933+0000 7f77f63f0740 1 -- start start 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.934+0000 7f77f63f0740 1 Processor -- start 2024-04-25T10:05:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.934+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c33cf0 0x555da2c5f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.934+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c34bb0 0x555da2c51d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.934+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c52280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.934+0000 7f77f63f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x555da2c60740 con 0x555da2c82c30 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.934+0000 7f77f63f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x555da2b37530 con 0x555da2c33cf0 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.934+0000 7f77f63f0740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x555da2c5fbc0 con 0x555da2c34bb0 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f63f0740 10 client.0 _ll_register_callbacks cb 0x555da2b8f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f1bb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c52280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f1bb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c52280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32958/0 (socket says 172.21.15.19:32958) 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f1bb8640 1 -- 172.21.15.19:0/1719693188 learned_addr learned my addr 172.21.15.19:0/1719693188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f0bb6640 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c34bb0 0x555da2c51d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c33cf0 0x555da2c5f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f1bb8640 1 -- 172.21.15.19:0/1719693188 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c34bb0 msgr2=0x555da2c51d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f1bb8640 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c34bb0 0x555da2c51d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f1bb8640 1 -- 172.21.15.19:0/1719693188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c33cf0 msgr2=0x555da2c5f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f1bb8640 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c33cf0 0x555da2c5f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f1bb8640 1 -- 172.21.15.19:0/1719693188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555da29d5f70 con 0x555da2c82c30 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c33cf0 0x555da2c5f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f1bb8640 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c52280 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f77e400bfc0 tx=0x7f77e4012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21136cab8039d14d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77c7fff640 1 -- 172.21.15.19:0/1719693188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4019030 con 0x555da2c82c30 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77c7fff640 1 -- 172.21.15.19:0/1719693188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f77e400f040 con 0x555da2c82c30 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f63f0740 1 -- 172.21.15.19:0/1719693188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77e0032d90 con 0x555da2c82c30 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77c7fff640 1 -- 172.21.15.19:0/1719693188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77e4004020 con 0x555da2c82c30 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f63f0740 10 client.11871 fetch_fsmap learned FSMap version 129 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.935+0000 7f77f63f0740 1 -- 172.21.15.19:0/1719693188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x555da286d170 con 0x555da2c82c30 2024-04-25T10:05:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.936+0000 7f77c7fff640 1 -- 172.21.15.19:0/1719693188 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e0032d90 con 0x555da2c82c30 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.936+0000 7f77c7fff640 1 -- 172.21.15.19:0/1719693188 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f77e4027080 con 0x555da2c82c30 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.936+0000 7f77f63f0740 10 client.11871 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.936+0000 7f77f63f0740 10 client.11871 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.936+0000 7f77f63f0740 20 client.11871 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.936+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x555da2c5dc00 0x555da2c5e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.936+0000 7f77f63f0740 4 client.11871 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.936+0000 7f77f63f0740 1 -- 172.21.15.19:0/1719693188 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555da2ba6000 con 0x555da2c5dc00 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.936+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x555da2c5dc00 0x555da2c5e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.936+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x555da2c5dc00 0x555da2c5e010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed9e63b8bd0c6d6 server_cookie=7ee6449083a78dae in_seq=0 out_seq=0 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.936+0000 7f77c7fff640 10 client.11871 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77c7fff640 1 -- 172.21.15.19:0/1719693188 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555da2ba6000 con 0x555da2c5dc00 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77c7fff640 10 client.11871 handle_command_reply: tid=0 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 1 client.11871 shutdown 2024-04-25T10:05:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 1 -- 172.21.15.19:0/1719693188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 msgr2=0x555da2c52280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c52280 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f77e400bfc0 tx=0x7f77e4012760 comp rx=0 tx=0).stop 2024-04-25T10:05:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 1 -- 172.21.15.19:0/1719693188 shutdown_connections 2024-04-25T10:05:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x555da2c5dc00 0x555da2c5e010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c52280 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c34bb0 0x555da2c51d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77c7fff640 0 client.11871 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1719693188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c33cf0 0x555da2c5f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 1 -- 172.21.15.19:0/1719693188 >> 172.21.15.19:0/1719693188 conn(0x555da2c2f830 msgr2=0x555da2c356a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 1 -- 172.21.15.19:0/1719693188 shutdown_connections 2024-04-25T10:05:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 1 -- 172.21.15.19:0/1719693188 wait complete. 2024-04-25T10:05:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.937+0000 7f77f63f0740 20 client.11871 trim_cache size 0 max 16384 2024-04-25T10:05:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.952+0000 7f77f63f0740 1 Processor -- start 2024-04-25T10:05:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.952+0000 7f77f63f0740 1 -- start start 2024-04-25T10:05:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c33cf0 0x555da2c2e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c34bb0 0x555da2c34410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c82c30 0x555da2c7ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f63f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec200750 con 0x555da2c33cf0 2024-04-25T10:05:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f63f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec1ffa00 con 0x555da2c82c30 2024-04-25T10:05:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f63f0740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec1f4440 con 0x555da2c34bb0 2024-04-25T10:05:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f13b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c33cf0 0x555da2c2e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f13b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c33cf0 0x555da2c2e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32972/0 (socket says 172.21.15.19:32972) 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f13b7640 1 -- 172.21.15.19:0/1936497002 learned_addr learned my addr 172.21.15.19:0/1936497002 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f13b7640 1 -- 172.21.15.19:0/1936497002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c34bb0 msgr2=0x555da2c34410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1936497002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c34bb0 0x555da2c34410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f13b7640 1 -- 172.21.15.19:0/1936497002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c82c30 msgr2=0x555da2c7ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1936497002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c82c30 0x555da2c7ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.953+0000 7f77f13b7640 1 -- 172.21.15.19:0/1936497002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555da2c2f450 con 0x555da2c33cf0 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1936497002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c33cf0 0x555da2c2e7c0 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f77dc088480 tx=0x7f77dc088350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e21b198f3a10ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77d27fc640 1 -- 172.21.15.19:0/1936497002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc070210 con 0x555da2c33cf0 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77d27fc640 1 -- 172.21.15.19:0/1936497002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77dc0880b0 con 0x555da2c33cf0 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77d27fc640 1 -- 172.21.15.19:0/1936497002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc060950 con 0x555da2c33cf0 2024-04-25T10:05:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77f63f0740 1 -- 172.21.15.19:0/1936497002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c33cf0 msgr2=0x555da2c2e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1936497002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c33cf0 0x555da2c2e7c0 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f77dc088480 tx=0x7f77dc088350 comp rx=0 tx=0).stop 2024-04-25T10:05:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77f63f0740 1 -- 172.21.15.19:0/1936497002 shutdown_connections 2024-04-25T10:05:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1936497002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c82c30 0x555da2c7ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1936497002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c34bb0 0x555da2c34410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1936497002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c33cf0 0x555da2c2e7c0 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77f63f0740 1 -- 172.21.15.19:0/1936497002 >> 172.21.15.19:0/1936497002 conn(0x555da2c2f830 msgr2=0x555da2c5dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77f63f0740 1 -- 172.21.15.19:0/1936497002 shutdown_connections 2024-04-25T10:05:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.954+0000 7f77f63f0740 1 -- 172.21.15.19:0/1936497002 wait complete. 2024-04-25T10:05:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.955+0000 7f77f63f0740 1 -- start start 2024-04-25T10:05:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.955+0000 7f77f63f0740 1 Processor -- start 2024-04-25T10:05:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c33cf0 0x555da2c9d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c34bb0 0x555da2ce5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f63f0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c4ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f63f0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x555da2c31aa0 con 0x555da2c82c30 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f63f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec1f4a40 con 0x555da2c34bb0 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f63f0740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x555da2c7aaa0 con 0x555da2c33cf0 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f63f0740 10 client.0 _ll_register_callbacks cb 0x7f77e0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f1bb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c4ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f1bb8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c4ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32986/0 (socket says 172.21.15.19:32986) 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f1bb8640 1 -- 172.21.15.19:0/1748625848 learned_addr learned my addr 172.21.15.19:0/1748625848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c33cf0 0x555da2c9d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f0bb6640 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c34bb0 0x555da2ce5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f1bb8640 1 -- 172.21.15.19:0/1748625848 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c33cf0 msgr2=0x555da2c9d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f1bb8640 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c33cf0 0x555da2c9d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f1bb8640 1 -- 172.21.15.19:0/1748625848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c34bb0 msgr2=0x555da2ce5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f1bb8640 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c34bb0 0x555da2ce5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f1bb8640 1 -- 172.21.15.19:0/1748625848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555da2c55720 con 0x555da2c82c30 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.956+0000 7f77f0bb6640 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c34bb0 0x555da2ce5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77f1bb8640 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c4ed00 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f77e4000c00 tx=0x7f77e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9300d0f6e92d2ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77c7fff640 1 -- 172.21.15.19:0/1748625848 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4019040 con 0x555da2c82c30 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77c7fff640 1 -- 172.21.15.19:0/1748625848 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f77e4012c60 con 0x555da2c82c30 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77f63f0740 1 -- 172.21.15.19:0/1748625848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77dc0880b0 con 0x555da2c82c30 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77c7fff640 1 -- 172.21.15.19:0/1748625848 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77e4034050 con 0x555da2c82c30 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77f63f0740 10 client.11877 fetch_fsmap learned FSMap version 129 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77f63f0740 1 -- 172.21.15.19:0/1748625848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x555da2c79990 con 0x555da2c82c30 2024-04-25T10:05:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77c7fff640 1 -- 172.21.15.19:0/1748625848 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77dc0880b0 con 0x555da2c82c30 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77c7fff640 1 -- 172.21.15.19:0/1748625848 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f77e4037020 con 0x555da2c82c30 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77f63f0740 10 client.11877 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77f63f0740 10 client.11877 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77f63f0740 20 client.11877 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x555da2c79eb0 0x555da2c7a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77f63f0740 4 client.11877 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77f63f0740 1 -- 172.21.15.19:0/1748625848 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x555da2ba0350 con 0x555da2c79eb0 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.957+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x555da2c79eb0 0x555da2c7a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.958+0000 7f77c7fff640 10 client.11877 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.958+0000 7f77f13b7640 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x555da2c79eb0 0x555da2c7a2c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c4062917d0e475f server_cookie=bbb286231c41be7c in_seq=0 out_seq=0 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.958+0000 7f77c7fff640 1 -- 172.21.15.19:0/1748625848 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555da2ba0350 con 0x555da2c79eb0 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.958+0000 7f77c7fff640 10 client.11877 handle_command_reply: tid=0 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.958+0000 7f77f63f0740 1 client.11877 shutdown 2024-04-25T10:05:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f63f0740 1 -- 172.21.15.19:0/1748625848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 msgr2=0x555da2c4ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c4ed00 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f77e4000c00 tx=0x7f77e4004730 comp rx=0 tx=0).stop 2024-04-25T10:05:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f63f0740 1 -- 172.21.15.19:0/1748625848 shutdown_connections 2024-04-25T10:05:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x555da2c79eb0 0x555da2c7a2c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555da2c82c30 0x555da2c4ed00 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555da2c34bb0 0x555da2ce5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f63f0740 1 --2- 172.21.15.19:0/1748625848 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x555da2c33cf0 0x555da2c9d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f63f0740 1 -- 172.21.15.19:0/1748625848 >> 172.21.15.19:0/1748625848 conn(0x555da2c2f830 msgr2=0x555da2c58020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77c7fff640 0 client.11877 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f63f0740 1 -- 172.21.15.19:0/1748625848 shutdown_connections 2024-04-25T10:05:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f63f0740 1 -- 172.21.15.19:0/1748625848 wait complete. 2024-04-25T10:05:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f63f0740 20 client.11877 trim_cache size 0 max 16384 2024-04-25T10:05:57.972 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:05:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:05:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "5d98a00f-ca00-4f1a-9f6e-f93ac22d95a9", 2024-04-25T10:05:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:05:57.972 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:05:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.959+0000 7f77f361b640 1 -- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f77d806b4d0 msgr2=0x7f77d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.960+0000 7f77f361b640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f77d806b4d0 0x7f77d806d990 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f77e0034000 tx=0x7f77e002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.960+0000 7f77f361b640 1 -- 172.21.15.19:0/1298704765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ec0d08e0 msgr2=0x7f77ec0b7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.960+0000 7f77f361b640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ec0d08e0 0x7f77ec0b7890 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00ca70 tx=0x7f77dc00cf30 comp rx=0 tx=0).stop 2024-04-25T10:05:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.960+0000 7f77f361b640 1 -- 172.21.15.19:0/1298704765 shutdown_connections 2024-04-25T10:05:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.960+0000 7f77f361b640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f77d806b4d0 0x7f77d806d990 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.960+0000 7f77f361b640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77ec15dac0 0x7f77ec0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.960+0000 7f77f361b640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77ec0d34c0 0x7f77ec0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.960+0000 7f77f361b640 1 --2- 172.21.15.19:0/1298704765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77ec0d08e0 0x7f77ec0b7890 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.960+0000 7f77f361b640 1 -- 172.21.15.19:0/1298704765 >> 172.21.15.19:0/1298704765 conn(0x7f77ec061c30 msgr2=0x7f77ec0b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.960+0000 7f77f361b640 1 -- 172.21.15.19:0/1298704765 shutdown_connections 2024-04-25T10:05:57.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:57.960+0000 7f77f361b640 1 -- 172.21.15.19:0/1298704765 wait complete. 2024-04-25T10:05:57.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:57.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.073+0000 7f4bbf50a640 1 Processor -- start 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.073+0000 7f4bbf50a640 1 -- start start 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbf50a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4bb815b7d0 0x7f4bb80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbf50a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4bb806c9e0 0x7f4bb80b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbf50a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb806cf50 con 0x7f4bb80b5620 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbf50a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb80ce530 con 0x7f4bb815b7d0 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbf50a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80ce810 con 0x7f4bb806c9e0 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbd2a6640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4bb815b7d0 0x7f4bb80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbcaa5640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4bb80b5620 0x7f4bb806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38046/0 (socket says 172.21.15.19:38046) 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbcaa5640 1 -- 172.21.15.19:0/2713650885 learned_addr learned my addr 172.21.15.19:0/2713650885 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/2713650885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4bb806c9e0 0x7f4bb80b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4baffff640 1 -- 172.21.15.19:0/2713650885 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851181409 0 0) 0x7f4bb806cf50 con 0x7f4bb80b5620 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4baffff640 1 -- 172.21.15.19:0/2713650885 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ba4003520 con 0x7f4bb80b5620 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbd2a6640 1 -- 172.21.15.19:0/2713650885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4bb806c9e0 msgr2=0x7f4bb80b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/2713650885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4bb806c9e0 0x7f4bb80b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbd2a6640 1 -- 172.21.15.19:0/2713650885 >> v1:172.21.15.19:6789/0 conn(0x7f4bb80b5620 legacy=0x7f4bb806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.074+0000 7f4bbd2a6640 1 -- 172.21.15.19:0/2713650885 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb80ceaf0 con 0x7f4bb815b7d0 2024-04-25T10:05:58.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.075+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/2713650885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb815b7d0 0x7f4bb80cf010 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40097a0 tx=0x7f4bb402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52c9bb092940fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.075+0000 7f4baffff640 1 -- 172.21.15.19:0/2713650885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4037070 con 0x7f4bb815b7d0 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.075+0000 7f4baffff640 1 -- 172.21.15.19:0/2713650885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb4032440 con 0x7f4bb815b7d0 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.075+0000 7f4baffff640 1 -- 172.21.15.19:0/2713650885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4032810 con 0x7f4bb815b7d0 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.076+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2713650885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb815b7d0 msgr2=0x7f4bb80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.076+0000 7f4bbf50a640 1 --2- 172.21.15.19:0/2713650885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb815b7d0 0x7f4bb80cf010 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40097a0 tx=0x7f4bb402a180 comp rx=0 tx=0).stop 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.076+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2713650885 shutdown_connections 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.076+0000 7f4bbf50a640 1 --2- 172.21.15.19:0/2713650885 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4bb806c9e0 0x7f4bb80b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.076+0000 7f4bbf50a640 1 --2- 172.21.15.19:0/2713650885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb815b7d0 0x7f4bb80cf010 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.076+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2713650885 >> 172.21.15.19:0/2713650885 conn(0x7f4bb80d0800 msgr2=0x7f4bb80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.076+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2713650885 shutdown_connections 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.076+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2713650885 wait complete. 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.077+0000 7f4bbf50a640 1 Processor -- start 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.077+0000 7f4bbf50a640 1 -- start start 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbf50a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb806c9e0 0x7f4bb81571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbf50a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80b5620 0x7f4bb80b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbf50a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4bb815b7d0 0x7f4bb80b8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbf50a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80cf6d0 con 0x7f4bb80b5620 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbf50a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80cf550 con 0x7f4bb806c9e0 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbf50a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb815bbb0 con 0x7f4bb815b7d0 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbcaa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80b5620 0x7f4bb80b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbcaa5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80b5620 0x7f4bb80b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33006/0 (socket says 172.21.15.19:33006) 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbcaa5640 1 -- 172.21.15.19:0/2090797630 learned_addr learned my addr 172.21.15.19:0/2090797630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4bb815b7d0 0x7f4bb80b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb806c9e0 0x7f4bb81571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbcaa5640 1 -- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4bb815b7d0 msgr2=0x7f4bb80b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4bb815b7d0 0x7f4bb80b8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbcaa5640 1 -- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb806c9e0 msgr2=0x7f4bb81571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb806c9e0 0x7f4bb81571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbcaa5640 1 -- 172.21.15.19:0/2090797630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb80c76a0 con 0x7f4bb80b5620 2024-04-25T10:05:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4bb815b7d0 0x7f4bb80b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.078+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb806c9e0 0x7f4bb81571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.079+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80b5620 0x7f4bb80b8050 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800c4d0 tx=0x7f4ba800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.079+0000 7f4badffb640 1 -- 172.21.15.19:0/2090797630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8017070 con 0x7f4bb80b5620 2024-04-25T10:05:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.079+0000 7f4badffb640 1 -- 172.21.15.19:0/2090797630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba8005d00 con 0x7f4bb80b5620 2024-04-25T10:05:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.079+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2090797630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb80c7980 con 0x7f4bb80b5620 2024-04-25T10:05:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.079+0000 7f4badffb640 1 -- 172.21.15.19:0/2090797630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80072b0 con 0x7f4bb80b5620 2024-04-25T10:05:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.079+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2090797630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb80c7f40 con 0x7f4bb80b5620 2024-04-25T10:05:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.080+0000 7f4badffb640 1 -- 172.21.15.19:0/2090797630 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ba801a070 con 0x7f4bb80b5620 2024-04-25T10:05:58.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.081+0000 7f4badffb640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b9c06b500 0x7f4b9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.081+0000 7f4badffb640 1 -- 172.21.15.19:0/2090797630 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f4ba8089800 con 0x7f4bb80b5620 2024-04-25T10:05:58.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.081+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b9c06b500 0x7f4b9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.081+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b9c06b500 0x7f4b9c06d9c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4034000 tx=0x7f4bb402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.082+0000 7f4bc2207740 1 Processor -- start 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.082+0000 7f4bc2207740 1 -- start start 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f87320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283f87860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f57770 0x559283f5af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bc2207740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559283ec0da0 con 0x559283f57770 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bc2207740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559283e69b80 con 0x559283f84ef0 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bc2207740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559283ebce40 con 0x559283f84b10 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbdaa7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f57770 0x559283f5af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbdaa7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f57770 0x559283f5af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33016/0 (socket says 172.21.15.19:33016) 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbdaa7640 1 -- 172.21.15.19:0/3539172784 learned_addr learned my addr 172.21.15.19:0/3539172784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/3539172784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283f87860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/3539172784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f87320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbdaa7640 1 -- 172.21.15.19:0/3539172784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 msgr2=0x559283f87320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/3539172784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f87320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbdaa7640 1 -- 172.21.15.19:0/3539172784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 msgr2=0x559283f87860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/3539172784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283f87860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbdaa7640 1 -- 172.21.15.19:0/3539172784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559283e73f10 con 0x559283f57770 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/3539172784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f87320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/3539172784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f57770 0x559283f5af80 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00074e0 tx=0x7f4bb00079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b583ebcc79ce95fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4ba2ffd640 1 -- 172.21.15.19:0/3539172784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00147e0 con 0x559283f57770 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4ba2ffd640 1 -- 172.21.15.19:0/3539172784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb001a070 con 0x559283f57770 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.083+0000 7f4ba2ffd640 1 -- 172.21.15.19:0/3539172784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0014ba0 con 0x559283f57770 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.084+0000 7f4bc2207740 1 -- 172.21.15.19:0/3539172784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f57770 msgr2=0x559283f5af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.084+0000 7f4bc2207740 1 --2- 172.21.15.19:0/3539172784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f57770 0x559283f5af80 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00074e0 tx=0x7f4bb00079a0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.084+0000 7f4bc2207740 1 -- 172.21.15.19:0/3539172784 shutdown_connections 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.084+0000 7f4bc2207740 1 --2- 172.21.15.19:0/3539172784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f57770 0x559283f5af80 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.084+0000 7f4bc2207740 1 --2- 172.21.15.19:0/3539172784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283f87860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.084+0000 7f4bc2207740 1 --2- 172.21.15.19:0/3539172784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f87320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.084+0000 7f4bc2207740 1 -- 172.21.15.19:0/3539172784 >> 172.21.15.19:0/3539172784 conn(0x559283f599e0 msgr2=0x559283fa2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.084+0000 7f4bc2207740 1 -- 172.21.15.19:0/3539172784 shutdown_connections 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.084+0000 7f4bc2207740 1 -- 172.21.15.19:0/3539172784 wait complete. 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.085+0000 7f4bc2207740 1 -- start start 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.085+0000 7f4bc2207740 1 Processor -- start 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84b10 0x559283f58820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283fa4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f7b670 0x559283f7baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bc2207740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559283f5cb90 con 0x559283f84b10 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bc2207740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x559283e5f4b0 con 0x559283f84ef0 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bc2207740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559283f5ce70 con 0x559283f7b670 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bc2207740 10 client.0 _ll_register_callbacks cb 0x559283e9f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bbd2a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84b10 0x559283f58820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bbd2a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84b10 0x559283f58820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33020/0 (socket says 172.21.15.19:33020) 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bbd2a6640 1 -- 172.21.15.19:0/2629767259 learned_addr learned my addr 172.21.15.19:0/2629767259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f7b670 0x559283f7baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283fa4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bbd2a6640 1 -- 172.21.15.19:0/2629767259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f7b670 msgr2=0x559283f7baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f7b670 0x559283f7baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bbd2a6640 1 -- 172.21.15.19:0/2629767259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 msgr2=0x559283fa4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283fa4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bbd2a6640 1 -- 172.21.15.19:0/2629767259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559283c9f7d0 con 0x559283f84b10 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84b10 0x559283f58820 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4000c00 tx=0x7f4bb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e4aa094b1e1889 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4b97fff640 1 -- 172.21.15.19:0/2629767259 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4037040 con 0x559283f84b10 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4b97fff640 1 -- 172.21.15.19:0/2629767259 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f4bb403eaf0 con 0x559283f84b10 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bc2207740 1 -- 172.21.15.19:0/2629767259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bb001a070 con 0x559283f84b10 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4b97fff640 1 -- 172.21.15.19:0/2629767259 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bb4056050 con 0x559283f84b10 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bc2207740 10 client.11892 fetch_fsmap learned FSMap version 129 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.086+0000 7f4bc2207740 1 -- 172.21.15.19:0/2629767259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x559283bd53f0 con 0x559283f84b10 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.087+0000 7f4b97fff640 1 -- 172.21.15.19:0/2629767259 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb001a070 con 0x559283f84b10 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.087+0000 7f4b97fff640 1 -- 172.21.15.19:0/2629767259 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4bb4059020 con 0x559283f84b10 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.087+0000 7f4bc2207740 10 client.11892 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.087+0000 7f4bc2207740 10 client.11892 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.087+0000 7f4bc2207740 20 client.11892 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.087+0000 7f4bc2207740 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x559283fb57f0 0x559283fb5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.087+0000 7f4bc2207740 4 client.11892 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.087+0000 7f4bc2207740 1 -- 172.21.15.19:0/2629767259 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559283ece2e0 con 0x559283fb57f0 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.087+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x559283fb57f0 0x559283fb5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.087+0000 7f4b97fff640 10 client.11892 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.087+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x559283fb57f0 0x559283fb5c50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2146a918c6f51003 server_cookie=9085fcc7571787f5 in_seq=0 out_seq=0 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4b97fff640 1 -- 172.21.15.19:0/2629767259 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559283ece2e0 con 0x559283fb57f0 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4b97fff640 10 client.11892 handle_command_reply: tid=0 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 1 client.11892 shutdown 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 1 -- 172.21.15.19:0/2629767259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84b10 msgr2=0x559283f58820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84b10 0x559283f58820 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4000c00 tx=0x7f4bb4004730 comp rx=0 tx=0).stop 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 1 -- 172.21.15.19:0/2629767259 shutdown_connections 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x559283fb57f0 0x559283fb5c50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f7b670 0x559283f7baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283fa4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4b97fff640 0 client.11892 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 1 --2- 172.21.15.19:0/2629767259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84b10 0x559283f58820 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 1 -- 172.21.15.19:0/2629767259 >> 172.21.15.19:0/2629767259 conn(0x559283f57770 msgr2=0x559283f83dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 1 -- 172.21.15.19:0/2629767259 shutdown_connections 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 1 -- 172.21.15.19:0/2629767259 wait complete. 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.088+0000 7f4bc2207740 20 client.11892 trim_cache size 0 max 16384 2024-04-25T10:05:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.103+0000 7f4bc2207740 1 Processor -- start 2024-04-25T10:05:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.103+0000 7f4bc2207740 1 -- start start 2024-04-25T10:05:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f7b670 0x559283f83790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f7baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84ef0 0x559283f56600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bc2207740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb8200420 con 0x559283f84ef0 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bc2207740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb81ff580 con 0x559283f7b670 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bc2207740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb81fe980 con 0x559283f84b10 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbdaa7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84ef0 0x559283f56600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbdaa7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84ef0 0x559283f56600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33028/0 (socket says 172.21.15.19:33028) 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbdaa7640 1 -- 172.21.15.19:0/1600749649 learned_addr learned my addr 172.21.15.19:0/1600749649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/1600749649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f7baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/1600749649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f7b670 0x559283f83790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbdaa7640 1 -- 172.21.15.19:0/1600749649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 msgr2=0x559283f7baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/1600749649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f7baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbdaa7640 1 -- 172.21.15.19:0/1600749649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f7b670 msgr2=0x559283f83790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/1600749649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f7b670 0x559283f83790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbdaa7640 1 -- 172.21.15.19:0/1600749649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559283f5c920 con 0x559283f84ef0 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/1600749649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f7baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.104+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/1600749649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f7b670 0x559283f83790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/1600749649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84ef0 0x559283f56600 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x559283f5f070 tx=0x7f4bb0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2815be42b6cefc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4ba27fc640 1 -- 172.21.15.19:0/1600749649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb002d070 con 0x559283f84ef0 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4ba27fc640 1 -- 172.21.15.19:0/1600749649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb001a040 con 0x559283f84ef0 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4ba27fc640 1 -- 172.21.15.19:0/1600749649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0014800 con 0x559283f84ef0 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4bc2207740 1 -- 172.21.15.19:0/1600749649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84ef0 msgr2=0x559283f56600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4bc2207740 1 --2- 172.21.15.19:0/1600749649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84ef0 0x559283f56600 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x559283f5f070 tx=0x7f4bb0014000 comp rx=0 tx=0).stop 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4bc2207740 1 -- 172.21.15.19:0/1600749649 shutdown_connections 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4bc2207740 1 --2- 172.21.15.19:0/1600749649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f84ef0 0x559283f56600 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4bc2207740 1 --2- 172.21.15.19:0/1600749649 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f7baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4bc2207740 1 --2- 172.21.15.19:0/1600749649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f7b670 0x559283f83790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4bc2207740 1 -- 172.21.15.19:0/1600749649 >> 172.21.15.19:0/1600749649 conn(0x559283f57770 msgr2=0x559283f7aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4bc2207740 1 -- 172.21.15.19:0/1600749649 shutdown_connections 2024-04-25T10:05:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.105+0000 7f4bc2207740 1 -- 172.21.15.19:0/1600749649 wait complete. 2024-04-25T10:05:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.106+0000 7f4bc2207740 1 -- start start 2024-04-25T10:05:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.106+0000 7f4bc2207740 1 Processor -- start 2024-04-25T10:05:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f7b670 0x559283ff6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f5ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bc2207740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283f56600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bc2207740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x559283f841f0 con 0x559283f7b670 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bc2207740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb81fef80 con 0x559283f84ef0 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bc2207740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x559283f83550 con 0x559283f84b10 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bc2207740 10 client.0 _ll_register_callbacks cb 0x7f4bb8062560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbd2a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f7b670 0x559283ff6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbd2a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f7b670 0x559283ff6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33040/0 (socket says 172.21.15.19:33040) 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbd2a6640 1 -- 172.21.15.19:0/4205969506 learned_addr learned my addr 172.21.15.19:0/4205969506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f5ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283f56600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbd2a6640 1 -- 172.21.15.19:0/4205969506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 msgr2=0x559283f5ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f5ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbd2a6640 1 -- 172.21.15.19:0/4205969506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 msgr2=0x559283f56600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283f56600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbd2a6640 1 -- 172.21.15.19:0/4205969506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559283b950f0 con 0x559283f7b670 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f5ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbdaa7640 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283f56600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4bbd2a6640 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f7b670 0x559283ff6350 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4051600 tx=0x7f4bb4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4a74d75d526737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.107+0000 7f4b97fff640 1 -- 172.21.15.19:0/4205969506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb405f020 con 0x559283f7b670 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4b97fff640 1 -- 172.21.15.19:0/4205969506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f4bb403c430 con 0x559283f7b670 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4bc2207740 1 -- 172.21.15.19:0/4205969506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bb001a040 con 0x559283f7b670 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4b97fff640 1 -- 172.21.15.19:0/4205969506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bb4069050 con 0x559283f7b670 2024-04-25T10:05:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4bc2207740 10 client.11898 fetch_fsmap learned FSMap version 129 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4bc2207740 1 -- 172.21.15.19:0/4205969506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x559283f8ae40 con 0x559283f7b670 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4b97fff640 1 -- 172.21.15.19:0/4205969506 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb001a040 con 0x559283f7b670 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4b97fff640 1 -- 172.21.15.19:0/4205969506 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4bb406c020 con 0x559283f7b670 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4bc2207740 10 client.11898 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4bc2207740 10 client.11898 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4bc2207740 20 client.11898 populate_metadata read hostname 'smithi019' 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4bc2207740 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x559283fa2770 0x559283f56b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4bc2207740 4 client.11898 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4bc2207740 1 -- 172.21.15.19:0/4205969506 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559283ec8630 con 0x559283fa2770 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x559283fa2770 0x559283f56b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4b97fff640 10 client.11898 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.108+0000 7f4bbcaa5640 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x559283fa2770 0x559283f56b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e1b76272182e36 server_cookie=3c0973a430e95bef in_seq=0 out_seq=0 2024-04-25T10:05:58.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4b97fff640 1 -- 172.21.15.19:0/4205969506 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x559283ec8630 con 0x559283fa2770 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4b97fff640 10 client.11898 handle_command_reply: tid=0 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 1 client.11898 shutdown 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 1 -- 172.21.15.19:0/4205969506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f7b670 msgr2=0x559283ff6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f7b670 0x559283ff6350 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4051600 tx=0x7f4bb4009190 comp rx=0 tx=0).stop 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 1 -- 172.21.15.19:0/4205969506 shutdown_connections 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x559283fa2770 0x559283f56b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559283f84ef0 0x559283f56600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x559283f84b10 0x559283f5ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4b97fff640 0 client.11898 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 1 --2- 172.21.15.19:0/4205969506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559283f7b670 0x559283ff6350 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 1 -- 172.21.15.19:0/4205969506 >> 172.21.15.19:0/4205969506 conn(0x559283f57770 msgr2=0x559283fbbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 1 -- 172.21.15.19:0/4205969506 shutdown_connections 2024-04-25T10:05:58.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 1 -- 172.21.15.19:0/4205969506 wait complete. 2024-04-25T10:05:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bc2207740 20 client.11898 trim_cache size 0 max 16384 2024-04-25T10:05:58.124 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:05:58.124 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:05:58.124 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:05:58.124 INFO:teuthology.orchestra.run.smithi019.stdout: "5d98a00f-ca00-4f1a-9f6e-f93ac22d95a9": { 2024-04-25T10:05:58.124 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:05:58.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "5d98a00f-ca00-4f1a-9f6e-f93ac22d95a9", 2024-04-25T10:05:58.124 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,repair" 2024-04-25T10:05:58.125 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:58.125 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:05:58.125 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:05:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b9c06b500 msgr2=0x7f4b9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.109+0000 7f4bbf50a640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b9c06b500 0x7f4b9c06d9c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4034000 tx=0x7f4bb402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.110+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2090797630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80b5620 msgr2=0x7f4bb80b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.110+0000 7f4bbf50a640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80b5620 0x7f4bb80b8050 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800c4d0 tx=0x7f4ba800c990 comp rx=0 tx=0).stop 2024-04-25T10:05:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.110+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2090797630 shutdown_connections 2024-04-25T10:05:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.110+0000 7f4bbf50a640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f4b9c06b500 0x7f4b9c06d9c0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.110+0000 7f4bbf50a640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4bb815b7d0 0x7f4bb80b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.110+0000 7f4bbf50a640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80b5620 0x7f4bb80b8050 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.110+0000 7f4bbf50a640 1 --2- 172.21.15.19:0/2090797630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb806c9e0 0x7f4bb81571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.110+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2090797630 >> 172.21.15.19:0/2090797630 conn(0x7f4bb80d0800 msgr2=0x7f4bb815afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.110+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2090797630 shutdown_connections 2024-04-25T10:05:58.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:05:58.110+0000 7f4bbf50a640 1 -- 172.21.15.19:0/2090797630 wait complete. 2024-04-25T10:05:58.127 INFO:tasks.cephfs.filesystem:scrub status for tag:5d98a00f-ca00-4f1a-9f6e-f93ac22d95a9 - {'path': '/', 'tag': '5d98a00f-ca00-4f1a-9f6e-f93ac22d95a9', 'options': 'recursive,repair'} 2024-04-25T10:05:59.393 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T10:05:59.387+0000 7f99eaed3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:09.325 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:09.328 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:28.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:28.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:06:28.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.241+0000 7fa0b8e1f640 1 Processor -- start 2024-04-25T10:06:28.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.241+0000 7fa0b8e1f640 1 -- start start 2024-04-25T10:06:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.241+0000 7fa0b8e1f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0b415dc20 0x7fa0b40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b8e1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0b40d32f0 0x7fa0b40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b8e1f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b415e000 con 0x7fa0b4161710 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b8e1f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b40d1de0 con 0x7fa0b415dc20 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b8e1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4161c60 con 0x7fa0b40d32f0 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0abfff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa0b4161710 0x7fa0b406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47146/0 (socket says 172.21.15.19:47146) 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0abfff640 1 -- 172.21.15.19:0/3721471516 learned_addr learned my addr 172.21.15.19:0/3721471516 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b2d9d640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0b40d32f0 0x7fa0b40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b259c640 1 --2- 172.21.15.19:0/3721471516 >> v2:172.21.15.112:3300/0 conn(0x7fa0b415dc20 0x7fa0b40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148024096 0 0) 0x7fa0b415e000 con 0x7fa0b4161710 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a0003520 con 0x7fa0b4161710 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3430964767 0 0) 0x7fa0a0003520 con 0x7fa0b4161710 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0b40d32f0 msgr2=0x7fa0b40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0b40d32f0 0x7fa0b40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 >> v2:172.21.15.112:3300/0 conn(0x7fa0b415dc20 msgr2=0x7fa0b40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 --2- 172.21.15.19:0/3721471516 >> v2:172.21.15.112:3300/0 conn(0x7fa0b415dc20 0x7fa0b40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b4161f40 con 0x7fa0b4161710 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fa09c001670 con 0x7fa0b4161710 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 >> v1:172.21.15.19:6789/0 conn(0x7fa0b4161710 legacy=0x7fa0b406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0a00039b0 0x7fa0a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0004800 0x7fa0a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0b40d32f0 0x7fa0a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40d1de0 con 0x7fa0a0004800 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4161c60 con 0x7fa0a00039b0 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.242+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a000b9c0 con 0x7fa0b40d32f0 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.243+0000 7fa0abfff640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0004800 0x7fa0a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.243+0000 7fa0b2d9d640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0b40d32f0 0x7fa0a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.243+0000 7fa0b259c640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0a00039b0 0x7fa0a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.243+0000 7fa0abfff640 1 -- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0b40d32f0 msgr2=0x7fa0a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.243+0000 7fa0abfff640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0b40d32f0 0x7fa0a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.243+0000 7fa0abfff640 1 -- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0a00039b0 msgr2=0x7fa0a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.243+0000 7fa0abfff640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0a00039b0 0x7fa0a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.243+0000 7fa0abfff640 1 -- 172.21.15.19:0/3721471516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a0003520 con 0x7fa0a0004800 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.243+0000 7fa0b259c640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0a00039b0 0x7fa0a0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.244+0000 7fa0abfff640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0004800 0x7fa0a0004c80 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fa09c007fd0 tx=0x7fa0a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8531cc48fbf0bc45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.244+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c007d50 con 0x7fa0a0004800 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.244+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c005ad0 con 0x7fa0a0004800 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.244+0000 7fa0b1d9b640 1 -- 172.21.15.19:0/3721471516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c007070 con 0x7fa0a0004800 2024-04-25T10:06:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.244+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/3721471516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0004800 msgr2=0x7fa0a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.244+0000 7fa0b8e1f640 1 --2- 172.21.15.19:0/3721471516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0a0004800 0x7fa0a0004c80 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fa09c007fd0 tx=0x7fa0a0005830 comp rx=0 tx=0).stop 2024-04-25T10:06:28.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.244+0000 7fa0b2d9d640 1 -- 172.21.15.19:0/3721471516 reap_dead start 2024-04-25T10:06:28.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.244+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/3721471516 shutdown_connections 2024-04-25T10:06:28.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.244+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/3721471516 >> 172.21.15.19:0/3721471516 conn(0x7fa0b40d0420 msgr2=0x7fa0b40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.244+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/3721471516 shutdown_connections 2024-04-25T10:06:28.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.244+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/3721471516 wait complete. 2024-04-25T10:06:28.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.246+0000 7fa0b8e1f640 1 Processor -- start 2024-04-25T10:06:28.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.246+0000 7fa0b8e1f640 1 -- start start 2024-04-25T10:06:28.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b8e1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0004800 0x7fa0b40cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b8e1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b40cc0f0 0x7fa0b41f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b8e1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b41f3d00 0x7fa0b40aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b8e1f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40b5e50 con 0x7fa0b41f3d00 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b8e1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4162500 con 0x7fa0b40cc0f0 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b8e1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40b5fd0 con 0x7fa0a0004800 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b41f3d00 0x7fa0b40aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b41f3d00 0x7fa0b40aad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40732/0 (socket says 172.21.15.19:40732) 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b2d9d640 1 -- 172.21.15.19:0/1331445857 learned_addr learned my addr 172.21.15.19:0/1331445857 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b259c640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0004800 0x7fa0b40cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0abfff640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b40cc0f0 0x7fa0b41f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b2d9d640 1 -- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0004800 msgr2=0x7fa0b40cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b2d9d640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0004800 0x7fa0b40cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b2d9d640 1 -- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b40cc0f0 msgr2=0x7fa0b41f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b2d9d640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b40cc0f0 0x7fa0b41f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b2d9d640 1 -- 172.21.15.19:0/1331445857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b40cab10 con 0x7fa0b41f3d00 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0b2d9d640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b41f3d00 0x7fa0b40aad90 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40b90f0 tx=0x7fa0ac00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.247+0000 7fa0aaffd640 1 -- 172.21.15.19:0/1331445857 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00fce0 con 0x7fa0b41f3d00 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.248+0000 7fa0aaffd640 1 -- 172.21.15.19:0/1331445857 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac00fe80 con 0x7fa0b41f3d00 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.248+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/1331445857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b40cadf0 con 0x7fa0b41f3d00 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.248+0000 7fa0aaffd640 1 -- 172.21.15.19:0/1331445857 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac015670 con 0x7fa0b41f3d00 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.248+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/1331445857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b40cb3b0 con 0x7fa0b41f3d00 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.249+0000 7fa0aaffd640 1 -- 172.21.15.19:0/1331445857 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0ac015810 con 0x7fa0b41f3d00 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.249+0000 7fa0aaffd640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa09406b500 0x7fa09406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.250+0000 7fa0aaffd640 1 -- 172.21.15.19:0/1331445857 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa0ac08aff0 con 0x7fa0b41f3d00 2024-04-25T10:06:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.250+0000 7fa0b259c640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa09406b500 0x7fa09406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.250+0000 7fa0b259c640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa09406b500 0x7fa09406d9c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4006f80 tx=0x7fa0a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.250+0000 7fa0bbc00740 1 Processor -- start 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.251+0000 7fa0bbc00740 1 -- start start 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.251+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351acdb10 0x55f351ad0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.251+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 0x55f351ad0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.251+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351aa0770 0x55f351aa3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.251+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f351a09da0 con 0x55f351aa0770 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.251+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3519b2b80 con 0x55f351acdef0 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.251+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f351a05e40 con 0x55f351acdb10 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0b2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351aa0770 0x55f351aa3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0b2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351aa0770 0x55f351aa3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40742/0 (socket says 172.21.15.19:40742) 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0b2d9d640 1 -- 172.21.15.19:0/266671588 learned_addr learned my addr 172.21.15.19:0/266671588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0b259c640 1 --2- 172.21.15.19:0/266671588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351acdb10 0x55f351ad0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0abfff640 1 --2- 172.21.15.19:0/266671588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 0x55f351ad0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0b2d9d640 1 -- 172.21.15.19:0/266671588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351acdb10 msgr2=0x55f351ad0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0b2d9d640 1 --2- 172.21.15.19:0/266671588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351acdb10 0x55f351ad0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0b2d9d640 1 -- 172.21.15.19:0/266671588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 msgr2=0x55f351ad0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0b2d9d640 1 --2- 172.21.15.19:0/266671588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 0x55f351ad0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0b2d9d640 1 -- 172.21.15.19:0/266671588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3519bcf10 con 0x55f351aa0770 2024-04-25T10:06:28.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0abfff640 1 --2- 172.21.15.19:0/266671588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 0x55f351ad0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:28.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa0b2d9d640 1 --2- 172.21.15.19:0/266671588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351aa0770 0x55f351aa3f80 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac089270 tx=0x7fa0ac059270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa99be857cf21f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa093fff640 1 -- 172.21.15.19:0/266671588 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac055c70 con 0x55f351aa0770 2024-04-25T10:06:28.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa093fff640 1 -- 172.21.15.19:0/266671588 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac0617d0 con 0x55f351aa0770 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.252+0000 7fa093fff640 1 -- 172.21.15.19:0/266671588 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac055e10 con 0x55f351aa0770 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.253+0000 7fa0bbc00740 1 -- 172.21.15.19:0/266671588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351aa0770 msgr2=0x55f351aa3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.253+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/266671588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351aa0770 0x55f351aa3f80 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac089270 tx=0x7fa0ac059270 comp rx=0 tx=0).stop 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.253+0000 7fa0bbc00740 1 -- 172.21.15.19:0/266671588 shutdown_connections 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.253+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/266671588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351aa0770 0x55f351aa3f80 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.253+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/266671588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 0x55f351ad0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.253+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/266671588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351acdb10 0x55f351ad0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.253+0000 7fa0bbc00740 1 -- 172.21.15.19:0/266671588 >> 172.21.15.19:0/266671588 conn(0x55f351aa29e0 msgr2=0x55f351aeb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.253+0000 7fa0bbc00740 1 -- 172.21.15.19:0/266671588 shutdown_connections 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.253+0000 7fa0bbc00740 1 -- 172.21.15.19:0/266671588 wait complete. 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.254+0000 7fa0bbc00740 1 -- start start 2024-04-25T10:06:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.254+0000 7fa0bbc00740 1 Processor -- start 2024-04-25T10:06:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351acdb10 0x55f351ac7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351aca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351aeda30 0x55f351acc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f351aa5b90 con 0x55f351acdef0 2024-04-25T10:06:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3519a84b0 con 0x55f351aeda30 2024-04-25T10:06:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f351aa5e70 con 0x55f351acdb10 2024-04-25T10:06:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0bbc00740 10 client.0 _ll_register_callbacks cb 0x55f3519e8180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351aca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0b259c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351acdb10 0x55f351ac7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351aca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40748/0 (socket says 172.21.15.19:40748) 2024-04-25T10:06:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0abfff640 1 -- 172.21.15.19:0/795757074 learned_addr learned my addr 172.21.15.19:0/795757074 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0b2d9d640 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351aeda30 0x55f351acc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0abfff640 1 -- 172.21.15.19:0/795757074 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351acdb10 msgr2=0x55f351ac7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0abfff640 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351acdb10 0x55f351ac7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0abfff640 1 -- 172.21.15.19:0/795757074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351aeda30 msgr2=0x55f351acc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0abfff640 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351aeda30 0x55f351acc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0abfff640 1 -- 172.21.15.19:0/795757074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3517e87d0 con 0x55f351acdef0 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0abfff640 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351aca1a0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7fa09c000f60 tx=0x7fa09c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d064cf97d363ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0917fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.255+0000 7fa0917fa640 1 -- 172.21.15.19:0/795757074 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c004cb0 con 0x55f351acdef0 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0917fa640 1 -- 172.21.15.19:0/795757074 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa09c007070 con 0x55f351acdef0 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0bbc00740 1 -- 172.21.15.19:0/795757074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0ac0617d0 con 0x55f351acdef0 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0917fa640 1 -- 172.21.15.19:0/795757074 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa09c004e50 con 0x55f351acdef0 2024-04-25T10:06:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0bbc00740 10 client.11910 fetch_fsmap learned FSMap version 129 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0bbc00740 1 -- 172.21.15.19:0/795757074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x55f35171e3f0 con 0x55f351acdef0 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0917fa640 1 -- 172.21.15.19:0/795757074 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac0617d0 con 0x55f351acdef0 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0917fa640 1 -- 172.21.15.19:0/795757074 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa09c048060 con 0x55f351acdef0 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0bbc00740 10 client.11910 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0bbc00740 10 client.11910 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0bbc00740 20 client.11910 populate_metadata read hostname 'smithi019' 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55f351afb390 0x55f351aec450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0bbc00740 4 client.11910 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0bbc00740 1 -- 172.21.15.19:0/795757074 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f351a172e0 con 0x55f351afb390 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0b259c640 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55f351afb390 0x55f351aec450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0917fa640 10 client.11910 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.256+0000 7fa0b259c640 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55f351afb390 0x55f351aec450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c67a40ef6e94afcf server_cookie=f721dfa4453fa2af in_seq=0 out_seq=0 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0917fa640 1 -- 172.21.15.19:0/795757074 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f351a172e0 con 0x55f351afb390 2024-04-25T10:06:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0917fa640 10 client.11910 handle_command_reply: tid=0 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0bbc00740 1 client.11910 shutdown 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0bbc00740 1 -- 172.21.15.19:0/795757074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 msgr2=0x55f351aca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351aca1a0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7fa09c000f60 tx=0x7fa09c000fe0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0bbc00740 1 -- 172.21.15.19:0/795757074 shutdown_connections 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55f351afb390 0x55f351aec450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351aeda30 0x55f351acc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351aca1a0 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0917fa640 0 client.11910 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/795757074 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351acdb10 0x55f351ac7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0bbc00740 1 -- 172.21.15.19:0/795757074 >> 172.21.15.19:0/795757074 conn(0x55f351aa0770 msgr2=0x55f351accf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0bbc00740 1 -- 172.21.15.19:0/795757074 shutdown_connections 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.257+0000 7fa0bbc00740 1 -- 172.21.15.19:0/795757074 wait complete. 2024-04-25T10:06:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.258+0000 7fa0bbc00740 20 client.11910 trim_cache size 0 max 16384 2024-04-25T10:06:28.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.273+0000 7fa0bbc00740 1 Processor -- start 2024-04-25T10:06:28.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.273+0000 7fa0bbc00740 1 -- start start 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdb10 0x55f351ad4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351a9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351aeda30 0x55f351a9fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f351aef510 con 0x55f351acdef0 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f351aeaf30 con 0x55f351acdb10 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4200ed0 con 0x55f351aeda30 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351a9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0abfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351a9f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40762/0 (socket says 172.21.15.19:40762) 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0abfff640 1 -- 172.21.15.19:0/4197615566 learned_addr learned my addr 172.21.15.19:0/4197615566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0abfff640 1 -- 172.21.15.19:0/4197615566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351aeda30 msgr2=0x55f351a9fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0abfff640 1 --2- 172.21.15.19:0/4197615566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351aeda30 0x55f351a9fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0abfff640 1 -- 172.21.15.19:0/4197615566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdb10 msgr2=0x55f351ad4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0abfff640 1 --2- 172.21.15.19:0/4197615566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdb10 0x55f351ad4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0abfff640 1 -- 172.21.15.19:0/4197615566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3516de0f0 con 0x55f351acdef0 2024-04-25T10:06:28.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0abfff640 1 --2- 172.21.15.19:0/4197615566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351a9f600 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fa09c001d30 tx=0x7fa09c042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332f1ab3e6488bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0937fe640 1 -- 172.21.15.19:0/4197615566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c056070 con 0x55f351acdef0 2024-04-25T10:06:28.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0937fe640 1 -- 172.21.15.19:0/4197615566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c004e50 con 0x55f351acdef0 2024-04-25T10:06:28.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.274+0000 7fa0937fe640 1 -- 172.21.15.19:0/4197615566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c007070 con 0x55f351acdef0 2024-04-25T10:06:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.275+0000 7fa0bbc00740 1 -- 172.21.15.19:0/4197615566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 msgr2=0x55f351a9f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.275+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/4197615566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351a9f600 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fa09c001d30 tx=0x7fa09c042040 comp rx=0 tx=0).stop 2024-04-25T10:06:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.275+0000 7fa0bbc00740 1 -- 172.21.15.19:0/4197615566 shutdown_connections 2024-04-25T10:06:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.275+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/4197615566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351aeda30 0x55f351a9fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.275+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/4197615566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdef0 0x55f351a9f600 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.275+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/4197615566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdb10 0x55f351ad4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.275+0000 7fa0bbc00740 1 -- 172.21.15.19:0/4197615566 >> 172.21.15.19:0/4197615566 conn(0x55f351aa0770 msgr2=0x55f351ac8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.275+0000 7fa0bbc00740 1 -- 172.21.15.19:0/4197615566 shutdown_connections 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.275+0000 7fa0bbc00740 1 -- 172.21.15.19:0/4197615566 wait complete. 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.275+0000 7fa0bbc00740 1 -- start start 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.276+0000 7fa0bbc00740 1 Processor -- start 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.276+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdb10 0x55f351b58bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.276+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 0x55f351af9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.276+0000 7fa0bbc00740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351aeda30 0x55f351afa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.276+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b41ff4b0 con 0x55f351acdb10 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.276+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b42000b0 con 0x55f351acdef0 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.276+0000 7fa0bbc00740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b41ffab0 con 0x55f351aeda30 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0bbc00740 10 client.0 _ll_register_callbacks cb 0x7fa0ac055920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0b259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdb10 0x55f351b58bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0b259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdb10 0x55f351b58bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40768/0 (socket says 172.21.15.19:40768) 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0b259c640 1 -- 172.21.15.19:0/329583311 learned_addr learned my addr 172.21.15.19:0/329583311 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0abfff640 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 0x55f351af9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0b2d9d640 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351aeda30 0x55f351afa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0b259c640 1 -- 172.21.15.19:0/329583311 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351aeda30 msgr2=0x55f351afa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0b259c640 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351aeda30 0x55f351afa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0b259c640 1 -- 172.21.15.19:0/329583311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 msgr2=0x55f351af9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0b259c640 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 0x55f351af9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0b259c640 1 -- 172.21.15.19:0/329583311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f351aecf00 con 0x55f351acdb10 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0abfff640 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 0x55f351af9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0b2d9d640 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351aeda30 0x55f351afa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0b259c640 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdb10 0x55f351b58bb0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x55f351aef140 tx=0x7fa0a4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4568ca3eaff5994 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0917fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:06:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0917fa640 1 -- 172.21.15.19:0/329583311 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a402f630 con 0x55f351acdb10 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0917fa640 1 -- 172.21.15.19:0/329583311 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa0a402f7d0 con 0x55f351acdb10 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0bbc00740 1 -- 172.21.15.19:0/329583311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa09c004e50 con 0x55f351acdb10 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0917fa640 1 -- 172.21.15.19:0/329583311 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0a4040050 con 0x55f351acdb10 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0bbc00740 10 client.11916 fetch_fsmap learned FSMap version 129 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.277+0000 7fa0bbc00740 1 -- 172.21.15.19:0/329583311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x55f351ac63f0 con 0x55f351acdb10 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0917fa640 1 -- 172.21.15.19:0/329583311 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c004e50 con 0x55f351acdb10 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0917fa640 1 -- 172.21.15.19:0/329583311 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa0a4043020 con 0x55f351acdb10 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0bbc00740 10 client.11916 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0bbc00740 10 client.11916 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0bbc00740 20 client.11916 populate_metadata read hostname 'smithi019' 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55f351aca660 0x55f351a9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0bbc00740 4 client.11916 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0bbc00740 1 -- 172.21.15.19:0/329583311 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f351a11630 con 0x55f351aca660 2024-04-25T10:06:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0abfff640 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55f351aca660 0x55f351a9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0917fa640 10 client.11916 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0abfff640 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55f351aca660 0x55f351a9f600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26ef05ea111f1ee0 server_cookie=1b3ed700882fc53e in_seq=0 out_seq=0 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0917fa640 1 -- 172.21.15.19:0/329583311 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f351a11630 con 0x55f351aca660 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0917fa640 10 client.11916 handle_command_reply: tid=0 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.278+0000 7fa0bbc00740 1 client.11916 shutdown 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0bbc00740 1 -- 172.21.15.19:0/329583311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdb10 msgr2=0x55f351b58bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdb10 0x55f351b58bb0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x55f351aef140 tx=0x7fa0a4009190 comp rx=0 tx=0).stop 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0bbc00740 1 -- 172.21.15.19:0/329583311 shutdown_connections 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55f351aca660 0x55f351a9f600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55f351aeda30 0x55f351afa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f351acdef0 0x55f351af9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0bbc00740 1 --2- 172.21.15.19:0/329583311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f351acdb10 0x55f351b58bb0 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0917fa640 0 client.11916 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0bbc00740 1 -- 172.21.15.19:0/329583311 >> 172.21.15.19:0/329583311 conn(0x55f351aa0770 msgr2=0x55f351b0d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0bbc00740 1 -- 172.21.15.19:0/329583311 shutdown_connections 2024-04-25T10:06:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0bbc00740 1 -- 172.21.15.19:0/329583311 wait complete. 2024-04-25T10:06:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0bbc00740 20 client.11916 trim_cache size 0 max 16384 2024-04-25T10:06:28.291 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:06:28.291 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:06:28.291 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:06:28.291 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:06:28.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa09406b500 msgr2=0x7fa09406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0b8e1f640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa09406b500 0x7fa09406d9c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4006f80 tx=0x7fa0a4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:28.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/1331445857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b41f3d00 msgr2=0x7fa0b40aad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0b8e1f640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b41f3d00 0x7fa0b40aad90 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40b90f0 tx=0x7fa0ac00f1b0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/1331445857 shutdown_connections 2024-04-25T10:06:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0b8e1f640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa09406b500 0x7fa09406d9c0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0b8e1f640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b41f3d00 0x7fa0b40aad90 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0b8e1f640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b40cc0f0 0x7fa0b41f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0b8e1f640 1 --2- 172.21.15.19:0/1331445857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0a0004800 0x7fa0b40cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.279+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/1331445857 >> 172.21.15.19:0/1331445857 conn(0x7fa0b40d0420 msgr2=0x7fa0b40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.280+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/1331445857 shutdown_connections 2024-04-25T10:06:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.280+0000 7fa0b8e1f640 1 -- 172.21.15.19:0/1331445857 wait complete. 2024-04-25T10:06:28.296 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:06:28.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:28.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:06:28.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.395+0000 7fe5008cb640 1 Processor -- start 2024-04-25T10:06:28.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.395+0000 7fe5008cb640 1 -- start start 2024-04-25T10:06:28.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.396+0000 7fe5008cb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4fc1610f0 0x7fe4fc06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.396+0000 7fe5008cb640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc0c6960 con 0x7fe4fc0ce7b0 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.396+0000 7fe5008cb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc162260 con 0x7fe4fc1616f0 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.396+0000 7fe5008cb640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4fc1614d0 con 0x7fe4fc1610f0 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.396+0000 7fe4fad9d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe4fc0ce7b0 0x7fe4fc0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47148/0 (socket says 172.21.15.19:47148) 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.396+0000 7fe4fad9d640 1 -- 172.21.15.19:0/85589735 learned_addr learned my addr 172.21.15.19:0/85589735 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689639578 0 0) 0x7fe4fc0c6960 con 0x7fe4fc0ce7b0 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/85589735 >> v2:172.21.15.112:3300/0 conn(0x7fe4fc1610f0 0x7fe4fc06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4e8003520 con 0x7fe4fc0ce7b0 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1285259161 0 0) 0x7fe4e8003520 con 0x7fe4fc0ce7b0 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 >> v2:172.21.15.112:3300/0 conn(0x7fe4fc1610f0 msgr2=0x7fe4fc06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 --2- 172.21.15.19:0/85589735 >> v2:172.21.15.112:3300/0 conn(0x7fe4fc1610f0 0x7fe4fc06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 >> v1:172.21.15.112:6789/0 conn(0x7fe4fc1616f0 legacy=0x7fe4fc0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc162540 con 0x7fe4fc0ce7b0 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fe4ec0019f0 con 0x7fe4fc0ce7b0 2024-04-25T10:06:28.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 >> v1:172.21.15.19:6789/0 conn(0x7fe4fc0ce7b0 legacy=0x7fe4fc0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4e80039b0 0x7fe4e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4e8004800 0x7fe4e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e80051c0 0x7fe4e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1614d0 con 0x7fe4e8004800 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e800c060 con 0x7fe4e80051c0 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e800c260 con 0x7fe4e80039b0 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f9d9b640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4e8004800 0x7fe4e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4e80039b0 0x7fe4e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4fad9d640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e80051c0 0x7fe4e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f9d9b640 1 -- 172.21.15.19:0/85589735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4e80039b0 msgr2=0x7fe4e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f9d9b640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4e80039b0 0x7fe4e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f9d9b640 1 -- 172.21.15.19:0/85589735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e80051c0 msgr2=0x7fe4e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f9d9b640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e80051c0 0x7fe4e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4f9d9b640 1 -- 172.21.15.19:0/85589735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e8003520 con 0x7fe4e8004800 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4fad9d640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e80051c0 0x7fe4e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.397+0000 7fe4fad9d640 1 -- 172.21.15.19:0/85589735 reap_dead start 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.398+0000 7fe4f9d9b640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4e8004800 0x7fe4e8004c80 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4001750 tx=0x7fe4e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a87add4d92cfde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.398+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4001370 con 0x7fe4e8004800 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.398+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e40052d0 con 0x7fe4e8004800 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.398+0000 7fe4f959a640 1 -- 172.21.15.19:0/85589735 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e402abc0 con 0x7fe4e8004800 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.399+0000 7fe5008cb640 1 -- 172.21.15.19:0/85589735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4e8004800 msgr2=0x7fe4e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.399+0000 7fe5008cb640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4e8004800 0x7fe4e8004c80 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4001750 tx=0x7fe4e8005830 comp rx=0 tx=0).stop 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.399+0000 7fe5008cb640 1 -- 172.21.15.19:0/85589735 shutdown_connections 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.399+0000 7fe5008cb640 1 --2- 172.21.15.19:0/85589735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4e8004800 0x7fe4e8004c80 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.399+0000 7fe5008cb640 1 -- 172.21.15.19:0/85589735 >> 172.21.15.19:0/85589735 conn(0x7fe4fc06f500 msgr2=0x7fe4fc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.399+0000 7fe5008cb640 1 -- 172.21.15.19:0/85589735 shutdown_connections 2024-04-25T10:06:28.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.399+0000 7fe5008cb640 1 -- 172.21.15.19:0/85589735 wait complete. 2024-04-25T10:06:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.400+0000 7fe5008cb640 1 Processor -- start 2024-04-25T10:06:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.400+0000 7fe5008cb640 1 -- start start 2024-04-25T10:06:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe5008cb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4e8004800 0x7fe4fc15f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe5008cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4fc0b27e0 0x7fe4fc0b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe5008cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc15cfd0 0x7fe4fc15d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe5008cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0cf690 con 0x7fe4fc15cfd0 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe5008cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0cfce0 con 0x7fe4fc0b27e0 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe5008cb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc161b50 con 0x7fe4e8004800 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4fad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc15cfd0 0x7fe4fc15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4fad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc15cfd0 0x7fe4fc15d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40794/0 (socket says 172.21.15.19:40794) 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4fad9d640 1 -- 172.21.15.19:0/3822632287 learned_addr learned my addr 172.21.15.19:0/3822632287 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4f9d9b640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4fc0b27e0 0x7fe4fc0b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4e8004800 0x7fe4fc15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4fad9d640 1 -- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4e8004800 msgr2=0x7fe4fc15f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4fad9d640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4e8004800 0x7fe4fc15f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4fad9d640 1 -- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4fc0b27e0 msgr2=0x7fe4fc0b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4fad9d640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4fc0b27e0 0x7fe4fc0b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4fad9d640 1 -- 172.21.15.19:0/3822632287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc148a00 con 0x7fe4fc15cfd0 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4f9d9b640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4fc0b27e0 0x7fe4fc0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.401+0000 7fe4fad9d640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc15cfd0 0x7fe4fc15d450 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec002230 tx=0x7fe4ec007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.402+0000 7fe4f2ffd640 1 -- 172.21.15.19:0/3822632287 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00ea20 con 0x7fe4fc15cfd0 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.402+0000 7fe4f2ffd640 1 -- 172.21.15.19:0/3822632287 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec018070 con 0x7fe4fc15cfd0 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.402+0000 7fe5008cb640 1 -- 172.21.15.19:0/3822632287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc147c40 con 0x7fe4fc15cfd0 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.402+0000 7fe4f2ffd640 1 -- 172.21.15.19:0/3822632287 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00ee40 con 0x7fe4fc15cfd0 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.402+0000 7fe5008cb640 1 -- 172.21.15.19:0/3822632287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc148200 con 0x7fe4fc15cfd0 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.403+0000 7fe4f2ffd640 1 -- 172.21.15.19:0/3822632287 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4ec01d460 con 0x7fe4fc15cfd0 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.403+0000 7fe4f2ffd640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4dc06b4d0 0x7fe4dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.404+0000 7fe4f2ffd640 1 -- 172.21.15.19:0/3822632287 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe4ec08a620 con 0x7fe4fc15cfd0 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.404+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4dc06b4d0 0x7fe4dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.404+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4dc06b4d0 0x7fe4dc06d990 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009b80 tx=0x7fe4f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.405+0000 7fe5035ce740 1 Processor -- start 2024-04-25T10:06:28.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.405+0000 7fe5035ce740 1 -- start start 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.405+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.405+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa819f830 0x55caa81a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.405+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa81a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.405+0000 7fe5035ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa8108ac0 con 0x55caa81a4bb0 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.405+0000 7fe5035ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa80b1d80 con 0x55caa819f830 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.405+0000 7fe5035ce740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55caa8104b60 con 0x55caa81a3cf0 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40804/0 (socket says 172.21.15.19:40804) 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4fa59c640 1 -- 172.21.15.19:0/1727203607 learned_addr learned my addr 172.21.15.19:0/1727203607 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4fad9d640 1 --2- 172.21.15.19:0/1727203607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa81a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4f9d9b640 1 --2- 172.21.15.19:0/1727203607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa819f830 0x55caa81a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4fa59c640 1 -- 172.21.15.19:0/1727203607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 msgr2=0x55caa81a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/1727203607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa81a4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4fa59c640 1 -- 172.21.15.19:0/1727203607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa819f830 msgr2=0x55caa81a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/1727203607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa819f830 0x55caa81a37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4fa59c640 1 -- 172.21.15.19:0/1727203607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caa7ee57e0 con 0x55caa81a4bb0 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4fad9d640 1 --2- 172.21.15.19:0/1727203607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa81a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/1727203607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81a4f90 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4014e80 tx=0x7fe4f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad34577959e7347a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4dbfff640 1 -- 172.21.15.19:0/1727203607 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4016e40 con 0x55caa81a4bb0 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4dbfff640 1 -- 172.21.15.19:0/1727203607 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f40097d0 con 0x55caa81a4bb0 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.406+0000 7fe4dbfff640 1 -- 172.21.15.19:0/1727203607 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4004230 con 0x55caa81a4bb0 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.407+0000 7fe5035ce740 1 -- 172.21.15.19:0/1727203607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 msgr2=0x55caa81a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.407+0000 7fe5035ce740 1 --2- 172.21.15.19:0/1727203607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81a4f90 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4014e80 tx=0x7fe4f4017000 comp rx=0 tx=0).stop 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.407+0000 7fe5035ce740 1 -- 172.21.15.19:0/1727203607 shutdown_connections 2024-04-25T10:06:28.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.407+0000 7fe5035ce740 1 --2- 172.21.15.19:0/1727203607 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa81a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.407+0000 7fe5035ce740 1 --2- 172.21.15.19:0/1727203607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa819f830 0x55caa81a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.407+0000 7fe5035ce740 1 --2- 172.21.15.19:0/1727203607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81a4f90 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.407+0000 7fe5035ce740 1 -- 172.21.15.19:0/1727203607 >> 172.21.15.19:0/1727203607 conn(0x55caa81a1cb0 msgr2=0x55caa81ea770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.407+0000 7fe5035ce740 1 -- 172.21.15.19:0/1727203607 shutdown_connections 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.407+0000 7fe5035ce740 1 -- 172.21.15.19:0/1727203607 wait complete. 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.408+0000 7fe5035ce740 1 -- start start 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.408+0000 7fe5035ce740 1 Processor -- start 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.408+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa81c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81f0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81f07d0 0x55caa81c4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe5035ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa81d0740 con 0x55caa81a4bb0 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe5035ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa80a7530 con 0x55caa81f07d0 2024-04-25T10:06:28.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe5035ce740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55caa81cfbc0 con 0x55caa81a3cf0 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe5035ce740 10 client.0 _ll_register_callbacks cb 0x55caa80ff130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81f0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4f9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81f0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40818/0 (socket says 172.21.15.19:40818) 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4f9d9b640 1 -- 172.21.15.19:0/875325781 learned_addr learned my addr 172.21.15.19:0/875325781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa81c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4fad9d640 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81f07d0 0x55caa81c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4f9d9b640 1 -- 172.21.15.19:0/875325781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 msgr2=0x55caa81c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4f9d9b640 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa81c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4f9d9b640 1 -- 172.21.15.19:0/875325781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81f07d0 msgr2=0x55caa81c4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4f9d9b640 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81f07d0 0x55caa81c4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4f9d9b640 1 -- 172.21.15.19:0/875325781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caa7f45f70 con 0x55caa81a4bb0 2024-04-25T10:06:28.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4f9d9b640 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81f0290 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4001750 tx=0x7fe4e4001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e4e46b810eb849 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4d97fa640 1 -- 172.21.15.19:0/875325781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4042060 con 0x55caa81a4bb0 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4d97fa640 1 -- 172.21.15.19:0/875325781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe4e402ad90 con 0x55caa81a4bb0 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe5035ce740 1 -- 172.21.15.19:0/875325781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55caa8116000 con 0x55caa81a4bb0 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe4d97fa640 1 -- 172.21.15.19:0/875325781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e4007040 con 0x55caa81a4bb0 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe5035ce740 10 client.11928 fetch_fsmap learned FSMap version 129 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.409+0000 7fe5035ce740 1 -- 172.21.15.19:0/875325781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x55caa7ddd170 con 0x55caa81a4bb0 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.410+0000 7fe4d97fa640 1 -- 172.21.15.19:0/875325781 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55caa8116000 con 0x55caa81a4bb0 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.410+0000 7fe4d97fa640 1 -- 172.21.15.19:0/875325781 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe4e40071e0 con 0x55caa81a4bb0 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.410+0000 7fe5035ce740 10 client.11928 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.410+0000 7fe5035ce740 10 client.11928 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.410+0000 7fe5035ce740 20 client.11928 populate_metadata read hostname 'smithi019' 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.410+0000 7fe5035ce740 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55caa81ef310 0x55caa81ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.410+0000 7fe5035ce740 4 client.11928 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.410+0000 7fe5035ce740 1 -- 172.21.15.19:0/875325781 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55caa8110350 con 0x55caa81ef310 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.410+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55caa81ef310 0x55caa81ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.410+0000 7fe4d97fa640 10 client.11928 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.410+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55caa81ef310 0x55caa81ef6f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee6e0f433dca320 server_cookie=f70625b48d161332 in_seq=0 out_seq=0 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe4d97fa640 1 -- 172.21.15.19:0/875325781 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55caa8110350 con 0x55caa81ef310 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe4d97fa640 10 client.11928 handle_command_reply: tid=0 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 1 client.11928 shutdown 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 1 -- 172.21.15.19:0/875325781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 msgr2=0x55caa81f0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81f0290 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4001750 tx=0x7fe4e4001190 comp rx=0 tx=0).stop 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 1 -- 172.21.15.19:0/875325781 shutdown_connections 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55caa81ef310 0x55caa81ef6f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81f07d0 0x55caa81c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a4bb0 0x55caa81f0290 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe4d97fa640 0 client.11928 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 1 --2- 172.21.15.19:0/875325781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa81c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 1 -- 172.21.15.19:0/875325781 >> 172.21.15.19:0/875325781 conn(0x55caa819f830 msgr2=0x55caa81a56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 1 -- 172.21.15.19:0/875325781 shutdown_connections 2024-04-25T10:06:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 1 -- 172.21.15.19:0/875325781 wait complete. 2024-04-25T10:06:28.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.411+0000 7fe5035ce740 20 client.11928 trim_cache size 0 max 16384 2024-04-25T10:06:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.426+0000 7fe5035ce740 1 Processor -- start 2024-04-25T10:06:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.426+0000 7fe5035ce740 1 -- start start 2024-04-25T10:06:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a3cf0 0x55caa81f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81a4bb0 0x55caa82043f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81f07d0 0x55caa8204930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe5035ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc200e80 con 0x55caa81a3cf0 2024-04-25T10:06:28.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe5035ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1f7f00 con 0x55caa81a4bb0 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe5035ce740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0b5fb0 con 0x55caa81f07d0 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a3cf0 0x55caa81f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a3cf0 0x55caa81f00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40820/0 (socket says 172.21.15.19:40820) 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe4fa59c640 1 -- 172.21.15.19:0/2624377224 learned_addr learned my addr 172.21.15.19:0/2624377224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe4fa59c640 1 -- 172.21.15.19:0/2624377224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81f07d0 msgr2=0x55caa8204930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/2624377224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81f07d0 0x55caa8204930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe4fa59c640 1 -- 172.21.15.19:0/2624377224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81a4bb0 msgr2=0x55caa82043f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/2624377224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81a4bb0 0x55caa82043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.427+0000 7fe4fa59c640 1 -- 172.21.15.19:0/2624377224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caa81c2ac0 con 0x55caa81a3cf0 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/2624377224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a3cf0 0x55caa81f00a0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4004000 tx=0x7fe4f401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0ed0b51118551f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe4db7fe640 1 -- 172.21.15.19:0/2624377224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f403a070 con 0x55caa81a3cf0 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe4db7fe640 1 -- 172.21.15.19:0/2624377224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f40355b0 con 0x55caa81a3cf0 2024-04-25T10:06:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe4db7fe640 1 -- 172.21.15.19:0/2624377224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4023660 con 0x55caa81a3cf0 2024-04-25T10:06:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe5035ce740 1 -- 172.21.15.19:0/2624377224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a3cf0 msgr2=0x55caa81f00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe5035ce740 1 --2- 172.21.15.19:0/2624377224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a3cf0 0x55caa81f00a0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4004000 tx=0x7fe4f401b000 comp rx=0 tx=0).stop 2024-04-25T10:06:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe5035ce740 1 -- 172.21.15.19:0/2624377224 shutdown_connections 2024-04-25T10:06:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe5035ce740 1 --2- 172.21.15.19:0/2624377224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81f07d0 0x55caa8204930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe5035ce740 1 --2- 172.21.15.19:0/2624377224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81a4bb0 0x55caa82043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe5035ce740 1 --2- 172.21.15.19:0/2624377224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81a3cf0 0x55caa81f00a0 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe5035ce740 1 -- 172.21.15.19:0/2624377224 >> 172.21.15.19:0/2624377224 conn(0x55caa819f830 msgr2=0x55caa81d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.428+0000 7fe5035ce740 1 -- 172.21.15.19:0/2624377224 shutdown_connections 2024-04-25T10:06:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.429+0000 7fe5035ce740 1 -- 172.21.15.19:0/2624377224 wait complete. 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.429+0000 7fe5035ce740 1 -- start start 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.429+0000 7fe5035ce740 1 Processor -- start 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa824f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81a4bb0 0x55caa81a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe5035ce740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81f07d0 0x55caa81a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe5035ce740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1f7900 con 0x55caa81f07d0 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe5035ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa81ee040 con 0x55caa81a4bb0 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe5035ce740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55caa81c6b10 con 0x55caa81a3cf0 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe5035ce740 10 client.0 _ll_register_callbacks cb 0x7fe4f401a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe4fad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81f07d0 0x55caa81a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe4fad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81f07d0 0x55caa81a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40832/0 (socket says 172.21.15.19:40832) 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe4fad9d640 1 -- 172.21.15.19:0/1497779632 learned_addr learned my addr 172.21.15.19:0/1497779632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa824f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe4fad9d640 1 -- 172.21.15.19:0/1497779632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 msgr2=0x55caa824f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe4f9d9b640 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81a4bb0 0x55caa81a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe4fad9d640 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa824f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe4fad9d640 1 -- 172.21.15.19:0/1497779632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81a4bb0 msgr2=0x55caa81a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe4fad9d640 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81a4bb0 0x55caa81a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe4fad9d640 1 -- 172.21.15.19:0/1497779632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caa82555d0 con 0x55caa81f07d0 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.430+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa824f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe4fad9d640 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81f07d0 0x55caa81a54d0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000f80 tx=0x7fe4ec058c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e249bba7269bad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe4d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe4d97fa640 1 -- 172.21.15.19:0/1497779632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec060dd0 con 0x55caa81f07d0 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe4d97fa640 1 -- 172.21.15.19:0/1497779632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe4ec060f70 con 0x55caa81f07d0 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe5035ce740 1 -- 172.21.15.19:0/1497779632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4f40355b0 con 0x55caa81f07d0 2024-04-25T10:06:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe4d97fa640 1 -- 172.21.15.19:0/1497779632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4ec05dbc0 con 0x55caa81f07d0 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe5035ce740 10 client.11934 fetch_fsmap learned FSMap version 129 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe5035ce740 1 -- 172.21.15.19:0/1497779632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x55caa81ed940 con 0x55caa81f07d0 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe4d97fa640 1 -- 172.21.15.19:0/1497779632 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f40355b0 con 0x55caa81f07d0 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe4d97fa640 1 -- 172.21.15.19:0/1497779632 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe4ec057fa0 con 0x55caa81f07d0 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe5035ce740 10 client.11934 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe5035ce740 10 client.11934 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe5035ce740 20 client.11934 populate_metadata read hostname 'smithi019' 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe5035ce740 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55caa81c51e0 0x55caa8255ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe5035ce740 4 client.11934 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe5035ce740 1 -- 172.21.15.19:0/1497779632 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55caa81c9f70 con 0x55caa81c51e0 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.431+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55caa81c51e0 0x55caa8255ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe4d97fa640 10 client.11934 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe4fa59c640 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55caa81c51e0 0x55caa8255ad0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf1d2eb12d94f25 server_cookie=7a4c54fc698f8c2e in_seq=0 out_seq=0 2024-04-25T10:06:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe4d97fa640 1 -- 172.21.15.19:0/1497779632 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55caa81c9f70 con 0x55caa81c51e0 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe4d97fa640 10 client.11934 handle_command_reply: tid=0 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe5035ce740 1 client.11934 shutdown 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe5035ce740 1 -- 172.21.15.19:0/1497779632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81f07d0 msgr2=0x55caa81a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe5035ce740 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81f07d0 0x55caa81a54d0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000f80 tx=0x7fe4ec058c10 comp rx=0 tx=0).stop 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe5035ce740 1 -- 172.21.15.19:0/1497779632 shutdown_connections 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe5035ce740 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55caa81c51e0 0x55caa8255ad0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe5035ce740 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caa81f07d0 0x55caa81a54d0 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe5035ce740 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55caa81a4bb0 0x55caa81a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe4d97fa640 0 client.11934 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe5035ce740 1 --2- 172.21.15.19:0/1497779632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55caa81a3cf0 0x55caa824f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.432+0000 7fe5035ce740 1 -- 172.21.15.19:0/1497779632 >> 172.21.15.19:0/1497779632 conn(0x55caa819f830 msgr2=0x55caa82043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5035ce740 1 -- 172.21.15.19:0/1497779632 shutdown_connections 2024-04-25T10:06:28.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5035ce740 1 -- 172.21.15.19:0/1497779632 wait complete. 2024-04-25T10:06:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5035ce740 20 client.11934 trim_cache size 0 max 16384 2024-04-25T10:06:28.444 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:06:28.444 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T10:06:28.444 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "1b9ee9ba-28f4-4a3e-9d7f-958a1037d6e2", 2024-04-25T10:06:28.444 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T10:06:28.444 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:06:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 -- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4dc06b4d0 msgr2=0x7fe4dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4dc06b4d0 0x7fe4dc06d990 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009b80 tx=0x7fe4f4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 -- 172.21.15.19:0/3822632287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc15cfd0 msgr2=0x7fe4fc15d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc15cfd0 0x7fe4fc15d450 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec002230 tx=0x7fe4ec007bc0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 -- 172.21.15.19:0/3822632287 shutdown_connections 2024-04-25T10:06:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe4dc06b4d0 0x7fe4dc06d990 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4fc15cfd0 0x7fe4fc15d450 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4fc0b27e0 0x7fe4fc0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 --2- 172.21.15.19:0/3822632287 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4e8004800 0x7fe4fc15f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 -- 172.21.15.19:0/3822632287 >> 172.21.15.19:0/3822632287 conn(0x7fe4fc06f500 msgr2=0x7fe4fc0ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 -- 172.21.15.19:0/3822632287 shutdown_connections 2024-04-25T10:06:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.433+0000 7fe5008cb640 1 -- 172.21.15.19:0/3822632287 wait complete. 2024-04-25T10:06:28.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:28.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:06:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.547+0000 7f6e1257e640 1 Processor -- start 2024-04-25T10:06:28.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.547+0000 7f6e1257e640 1 -- start start 2024-04-25T10:06:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.547+0000 7f6e1257e640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6e0c15b7d0 0x7f6e0c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.547+0000 7f6e1257e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0c0b5620 0x7f6e0c0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.547+0000 7f6e1257e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6e0c06c2d0 0x7f6e0c06c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.547+0000 7f6e1257e640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0c0b5fc0 con 0x7f6e0c15b7d0 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.547+0000 7f6e1257e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0c06cca0 con 0x7f6e0c06c2d0 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.547+0000 7f6e1257e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c06cf50 con 0x7f6e0c0b5620 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.548+0000 7f6e0bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6e0c15b7d0 0x7f6e0c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.548+0000 7f6e0bfff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6e0c15b7d0 0x7f6e0c0cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40836/0 (socket says 172.21.15.19:40836) 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.548+0000 7f6e0bfff640 1 -- 172.21.15.19:0/1213925890 learned_addr learned my addr 172.21.15.19:0/1213925890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.548+0000 7f6e0b7fe640 1 --2- 172.21.15.19:0/1213925890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0c0b5620 0x7f6e0c0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.548+0000 7f6e0bfff640 1 -- 172.21.15.19:0/1213925890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0c0b5620 msgr2=0x7f6e0c0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.548+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/1213925890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0c0b5620 0x7f6e0c0b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.548+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/1213925890 >> v2:172.21.15.112:3300/0 conn(0x7f6e0c06c2d0 0x7f6e0c06c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.548+0000 7f6e0bfff640 1 -- 172.21.15.19:0/1213925890 >> v2:172.21.15.112:3300/0 conn(0x7f6e0c06c2d0 msgr2=0x7f6e0c06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.548+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/1213925890 >> v2:172.21.15.112:3300/0 conn(0x7f6e0c06c2d0 0x7f6e0c06c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.548+0000 7f6e0bfff640 1 -- 172.21.15.19:0/1213925890 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c0ce530 con 0x7f6e0c15b7d0 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/1213925890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e0c15b7d0 0x7f6e0c0cf010 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c15e340 tx=0x7f6dfc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96d6be517db312e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e0affd640 1 -- 172.21.15.19:0/1213925890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc037070 con 0x7f6e0c15b7d0 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e0affd640 1 -- 172.21.15.19:0/1213925890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc032440 con 0x7f6e0c15b7d0 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e0affd640 1 -- 172.21.15.19:0/1213925890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc032810 con 0x7f6e0c15b7d0 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e1257e640 1 -- 172.21.15.19:0/1213925890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e0c15b7d0 msgr2=0x7f6e0c0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e1257e640 1 --2- 172.21.15.19:0/1213925890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e0c15b7d0 0x7f6e0c0cf010 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c15e340 tx=0x7f6dfc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e1257e640 1 -- 172.21.15.19:0/1213925890 shutdown_connections 2024-04-25T10:06:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e1257e640 1 --2- 172.21.15.19:0/1213925890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0c0b5620 0x7f6e0c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e1257e640 1 --2- 172.21.15.19:0/1213925890 >> v2:172.21.15.112:3300/0 conn(0x7f6e0c06c2d0 0x7f6e0c06c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e1257e640 1 --2- 172.21.15.19:0/1213925890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e0c15b7d0 0x7f6e0c0cf010 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e1257e640 1 -- 172.21.15.19:0/1213925890 >> 172.21.15.19:0/1213925890 conn(0x7f6e0c0d0800 msgr2=0x7f6e0c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e1257e640 1 -- 172.21.15.19:0/1213925890 shutdown_connections 2024-04-25T10:06:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.549+0000 7f6e1257e640 1 -- 172.21.15.19:0/1213925890 wait complete. 2024-04-25T10:06:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.551+0000 7f6e1257e640 1 Processor -- start 2024-04-25T10:06:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.551+0000 7f6e1257e640 1 -- start start 2024-04-25T10:06:28.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.551+0000 7f6e1257e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0c06c2d0 0x7f6e0c0aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e1257e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e0c0b5620 0x7f6e0c0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e1257e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e0c15b7d0 0x7f6e0c0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e1257e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c06d230 con 0x7f6e0c0b5620 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e1257e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c0ceaf0 con 0x7f6e0c15b7d0 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e1257e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c0cf550 con 0x7f6e0c06c2d0 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e0b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e0c0b5620 0x7f6e0c0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e0b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e0c0b5620 0x7f6e0c0ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40852/0 (socket says 172.21.15.19:40852) 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e0b7fe640 1 -- 172.21.15.19:0/3700397412 learned_addr learned my addr 172.21.15.19:0/3700397412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0c06c2d0 0x7f6e0c0aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e0c15b7d0 0x7f6e0c0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e0b7fe640 1 -- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0c06c2d0 msgr2=0x7f6e0c0aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e0b7fe640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0c06c2d0 0x7f6e0c0aaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e0b7fe640 1 -- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e0c15b7d0 msgr2=0x7f6e0c0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e0b7fe640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e0c15b7d0 0x7f6e0c0ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e0b7fe640 1 -- 172.21.15.19:0/3700397412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c0ab3e0 con 0x7f6e0c0b5620 2024-04-25T10:06:28.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0c06c2d0 0x7f6e0c0aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e0c15b7d0 0x7f6e0c0ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e0b7fe640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e0c0b5620 0x7f6e0c0ca070 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c7f0 tx=0x7f6e0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e08ff9640 1 -- 172.21.15.19:0/3700397412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00015070 con 0x7f6e0c0b5620 2024-04-25T10:06:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e08ff9640 1 -- 172.21.15.19:0/3700397412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e00004240 con 0x7f6e0c0b5620 2024-04-25T10:06:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e1257e640 1 -- 172.21.15.19:0/3700397412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c0cbe60 con 0x7f6e0c0b5620 2024-04-25T10:06:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e08ff9640 1 -- 172.21.15.19:0/3700397412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00007620 con 0x7f6e0c0b5620 2024-04-25T10:06:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.552+0000 7f6e1257e640 1 -- 172.21.15.19:0/3700397412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c0cc420 con 0x7f6e0c0b5620 2024-04-25T10:06:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.554+0000 7f6e08ff9640 1 -- 172.21.15.19:0/3700397412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e000079d0 con 0x7f6e0c0b5620 2024-04-25T10:06:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.555+0000 7f6e08ff9640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6df806b4d0 0x7f6df806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.555+0000 7f6e08ff9640 1 -- 172.21.15.19:0/3700397412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6e0008a800 con 0x7f6e0c0b5620 2024-04-25T10:06:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.555+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6df806b4d0 0x7f6df806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.555+0000 7f6e15280740 1 Processor -- start 2024-04-25T10:06:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.555+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6df806b4d0 0x7f6df806d990 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc034000 tx=0x7f6dfc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.555+0000 7f6e15280740 1 -- start start 2024-04-25T10:06:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e15280740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9b10 0x55d485ddc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e15280740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9ef0 0x55d485ddc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e15280740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dac770 0x55d485daff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e15280740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d485d15da0 con 0x55d485dd9b10 2024-04-25T10:06:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e15280740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d485cbeb80 con 0x55d485dd9ef0 2024-04-25T10:06:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e15280740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d485d11e40 con 0x55d485dac770 2024-04-25T10:06:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9b10 0x55d485ddc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9b10 0x55d485ddc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40868/0 (socket says 172.21.15.19:40868) 2024-04-25T10:06:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e0bfff640 1 -- 172.21.15.19:0/901229039 learned_addr learned my addr 172.21.15.19:0/901229039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/901229039 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dac770 0x55d485daff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e0b7fe640 1 --2- 172.21.15.19:0/901229039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9ef0 0x55d485ddc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e0bfff640 1 -- 172.21.15.19:0/901229039 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dac770 msgr2=0x55d485daff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/901229039 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dac770 0x55d485daff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e0bfff640 1 -- 172.21.15.19:0/901229039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9ef0 msgr2=0x55d485ddc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/901229039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9ef0 0x55d485ddc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e0bfff640 1 -- 172.21.15.19:0/901229039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d485cc8f10 con 0x55d485dd9b10 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e0b7fe640 1 --2- 172.21.15.19:0/901229039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9ef0 0x55d485ddc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.556+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/901229039 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dac770 0x55d485daff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/901229039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9b10 0x55d485ddc320 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc035040 tx=0x7f6dfc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7e4b415b06000b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6df1ffb640 1 -- 172.21.15.19:0/901229039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc04d070 con 0x55d485dd9b10 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6df1ffb640 1 -- 172.21.15.19:0/901229039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc03c460 con 0x55d485dd9b10 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6df1ffb640 1 -- 172.21.15.19:0/901229039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc037040 con 0x55d485dd9b10 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6e15280740 1 -- 172.21.15.19:0/901229039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9b10 msgr2=0x55d485ddc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6e15280740 1 --2- 172.21.15.19:0/901229039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9b10 0x55d485ddc320 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc035040 tx=0x7f6dfc02c000 comp rx=0 tx=0).stop 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6e15280740 1 -- 172.21.15.19:0/901229039 shutdown_connections 2024-04-25T10:06:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6e15280740 1 --2- 172.21.15.19:0/901229039 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dac770 0x55d485daff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6e15280740 1 --2- 172.21.15.19:0/901229039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9ef0 0x55d485ddc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6e15280740 1 --2- 172.21.15.19:0/901229039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9b10 0x55d485ddc320 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6e15280740 1 -- 172.21.15.19:0/901229039 >> 172.21.15.19:0/901229039 conn(0x55d485dae9e0 msgr2=0x55d485df7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6e15280740 1 -- 172.21.15.19:0/901229039 shutdown_connections 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.557+0000 7f6e15280740 1 -- 172.21.15.19:0/901229039 wait complete. 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.558+0000 7f6e15280740 1 -- start start 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.558+0000 7f6e15280740 1 Processor -- start 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e15280740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485df1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e15280740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd9ef0 0x55d485e06e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e15280740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e15280740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d485db1b90 con 0x55d485dd8540 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e15280740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d485cb44b0 con 0x55d485dd9b10 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e15280740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d485db1e70 con 0x55d485dd9ef0 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e15280740 10 client.0 _ll_register_callbacks cb 0x55d485cf4180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e10b1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e10b1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dee8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40882/0 (socket says 172.21.15.19:40882) 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e10b1b640 1 -- 172.21.15.19:0/254045609 learned_addr learned my addr 172.21.15.19:0/254045609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485df1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e0b7fe640 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd9ef0 0x55d485e06e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e10b1b640 1 -- 172.21.15.19:0/254045609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd9ef0 msgr2=0x55d485e06e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd9ef0 0x55d485e06e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e10b1b640 1 -- 172.21.15.19:0/254045609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 msgr2=0x55d485df1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485df1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.559+0000 7f6e10b1b640 1 -- 172.21.15.19:0/254045609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d485af47d0 con 0x55d485dd8540 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dee8c0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400f840 tx=0x7f6e0400fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa1ec7c20d28400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6deaffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6deaffd640 1 -- 172.21.15.19:0/254045609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0400dd70 con 0x55d485dd8540 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6deaffd640 1 -- 172.21.15.19:0/254045609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6e04019d40 con 0x55d485dd8540 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6e15280740 1 -- 172.21.15.19:0/254045609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dfc03c460 con 0x55d485dd8540 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6deaffd640 1 -- 172.21.15.19:0/254045609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e04003990 con 0x55d485dd8540 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6e15280740 10 client.11946 fetch_fsmap learned FSMap version 129 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6e15280740 1 -- 172.21.15.19:0/254045609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x55d485a2a3f0 con 0x55d485dd8540 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6deaffd640 1 -- 172.21.15.19:0/254045609 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc03c460 con 0x55d485dd8540 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6deaffd640 1 -- 172.21.15.19:0/254045609 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6e04020050 con 0x55d485dd8540 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6e15280740 10 client.11946 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6e15280740 10 client.11946 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6e15280740 20 client.11946 populate_metadata read hostname 'smithi019' 2024-04-25T10:06:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6e15280740 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55d485df3bd0 0x55d485df3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6e15280740 4 client.11946 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6e15280740 1 -- 172.21.15.19:0/254045609 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d485d232e0 con 0x55d485df3bd0 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.560+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55d485df3bd0 0x55d485df3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55d485df3bd0 0x55d485df3ff0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72c44e10a0b2c03c server_cookie=30490c98d690a40e in_seq=0 out_seq=0 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6deaffd640 10 client.11946 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6deaffd640 1 -- 172.21.15.19:0/254045609 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d485d232e0 con 0x55d485df3bd0 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6deaffd640 10 client.11946 handle_command_reply: tid=0 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e15280740 1 client.11946 shutdown 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e15280740 1 -- 172.21.15.19:0/254045609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 msgr2=0x55d485dee8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e15280740 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dee8c0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400f840 tx=0x7f6e0400fd00 comp rx=0 tx=0).stop 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e15280740 1 -- 172.21.15.19:0/254045609 shutdown_connections 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e15280740 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55d485df3bd0 0x55d485df3ff0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e15280740 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dee8c0 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e15280740 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd9ef0 0x55d485e06e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6deaffd640 0 client.11946 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e15280740 1 --2- 172.21.15.19:0/254045609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485df1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e15280740 1 -- 172.21.15.19:0/254045609 >> 172.21.15.19:0/254045609 conn(0x55d485dac770 msgr2=0x55d485dab600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e15280740 1 -- 172.21.15.19:0/254045609 shutdown_connections 2024-04-25T10:06:28.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.561+0000 7f6e15280740 1 -- 172.21.15.19:0/254045609 wait complete. 2024-04-25T10:06:28.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.562+0000 7f6e15280740 20 client.11946 trim_cache size 0 max 16384 2024-04-25T10:06:28.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.577+0000 7f6e15280740 1 Processor -- start 2024-04-25T10:06:28.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.577+0000 7f6e15280740 1 -- start start 2024-04-25T10:06:28.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.577+0000 7f6e15280740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.577+0000 7f6e15280740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485df06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.577+0000 7f6e15280740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd9ef0 0x55d485df0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.577+0000 7f6e15280740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c200520 con 0x55d485dd8540 2024-04-25T10:06:28.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.577+0000 7f6e15280740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c1ff6b0 con 0x55d485dd9b10 2024-04-25T10:06:28.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.577+0000 7f6e15280740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c1f7700 con 0x55d485dd9ef0 2024-04-25T10:06:28.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.577+0000 7f6e0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.577+0000 7f6e0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dab7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40888/0 (socket says 172.21.15.19:40888) 2024-04-25T10:06:28.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.577+0000 7f6e0bfff640 1 -- 172.21.15.19:0/3044722169 learned_addr learned my addr 172.21.15.19:0/3044722169 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/3044722169 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd9ef0 0x55d485df0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6e0b7fe640 1 --2- 172.21.15.19:0/3044722169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485df06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6e0bfff640 1 -- 172.21.15.19:0/3044722169 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd9ef0 msgr2=0x55d485df0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/3044722169 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd9ef0 0x55d485df0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6e0bfff640 1 -- 172.21.15.19:0/3044722169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 msgr2=0x55d485df06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/3044722169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485df06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6e0bfff640 1 -- 172.21.15.19:0/3044722169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4859ea0f0 con 0x55d485dd8540 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6e0b7fe640 1 --2- 172.21.15.19:0/3044722169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485df06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/3044722169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dab7f0 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009960 tx=0x7f6dfc009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556c434717ba25be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6df17fa640 1 -- 172.21.15.19:0/3044722169 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc04d040 con 0x55d485dd8540 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6df17fa640 1 -- 172.21.15.19:0/3044722169 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc057070 con 0x55d485dd8540 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6df17fa640 1 -- 172.21.15.19:0/3044722169 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc037040 con 0x55d485dd8540 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6e15280740 1 -- 172.21.15.19:0/3044722169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 msgr2=0x55d485dab7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.578+0000 7f6e15280740 1 --2- 172.21.15.19:0/3044722169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dab7f0 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009960 tx=0x7f6dfc009240 comp rx=0 tx=0).stop 2024-04-25T10:06:28.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.579+0000 7f6e15280740 1 -- 172.21.15.19:0/3044722169 shutdown_connections 2024-04-25T10:06:28.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.579+0000 7f6e15280740 1 --2- 172.21.15.19:0/3044722169 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd9ef0 0x55d485df0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.579+0000 7f6e15280740 1 --2- 172.21.15.19:0/3044722169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485df06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.579+0000 7f6e15280740 1 --2- 172.21.15.19:0/3044722169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd8540 0x55d485dab7f0 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.579+0000 7f6e15280740 1 -- 172.21.15.19:0/3044722169 >> 172.21.15.19:0/3044722169 conn(0x55d485dac770 msgr2=0x55d485df8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.579+0000 7f6e15280740 1 -- 172.21.15.19:0/3044722169 shutdown_connections 2024-04-25T10:06:28.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.579+0000 7f6e15280740 1 -- 172.21.15.19:0/3044722169 wait complete. 2024-04-25T10:06:28.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.579+0000 7f6e15280740 1 -- start start 2024-04-25T10:06:28.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.579+0000 7f6e15280740 1 Processor -- start 2024-04-25T10:06:28.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e15280740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd8540 0x55d485dd4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e15280740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485dee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e15280740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9ef0 0x55d485db3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e15280740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c1f7d00 con 0x55d485dd9ef0 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e15280740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d485dee490 con 0x55d485dd9b10 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e15280740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d485db0000 con 0x55d485dd8540 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e15280740 10 client.0 _ll_register_callbacks cb 0x7f6dfc035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e10b1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9ef0 0x55d485db3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e10b1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9ef0 0x55d485db3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40904/0 (socket says 172.21.15.19:40904) 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e10b1b640 1 -- 172.21.15.19:0/4274340215 learned_addr learned my addr 172.21.15.19:0/4274340215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e10b1b640 1 -- 172.21.15.19:0/4274340215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd8540 msgr2=0x55d485dd4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e0b7fe640 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485dee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd8540 0x55d485dd4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd8540 0x55d485dd4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e10b1b640 1 -- 172.21.15.19:0/4274340215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 msgr2=0x55d485dee670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485dee670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.580+0000 7f6e10b1b640 1 -- 172.21.15.19:0/4274340215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d485dafdf0 con 0x55d485dd9ef0 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6e10b1b640 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9ef0 0x55d485db3810 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f6e040073c0 tx=0x7f6e04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3f2c63b3fa5e82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6deaffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6deaffd640 1 -- 172.21.15.19:0/4274340215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04022470 con 0x55d485dd9ef0 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6deaffd640 1 -- 172.21.15.19:0/4274340215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6e04022610 con 0x55d485dd9ef0 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6e15280740 1 -- 172.21.15.19:0/4274340215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dfc057070 con 0x55d485dd9ef0 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6deaffd640 1 -- 172.21.15.19:0/4274340215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e04035050 con 0x55d485dd9ef0 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6e15280740 10 client.11952 fetch_fsmap learned FSMap version 129 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6e15280740 1 -- 172.21.15.19:0/4274340215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x55d485ddf700 con 0x55d485dd9ef0 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6deaffd640 1 -- 172.21.15.19:0/4274340215 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc057070 con 0x55d485dd9ef0 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6deaffd640 1 -- 172.21.15.19:0/4274340215 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6e04038020 con 0x55d485dd9ef0 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6e15280740 10 client.11952 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6e15280740 10 client.11952 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6e15280740 20 client.11952 populate_metadata read hostname 'smithi019' 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6e15280740 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55d485ddfa00 0x55d485e647a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6e15280740 4 client.11952 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6e15280740 1 -- 172.21.15.19:0/4274340215 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d485d1d630 con 0x55d485ddfa00 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.581+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55d485ddfa00 0x55d485e647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6deaffd640 10 client.11952 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e0bfff640 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55d485ddfa00 0x55d485e647a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb4db79b6974171 server_cookie=e8f5399ad3607740 in_seq=0 out_seq=0 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6deaffd640 1 -- 172.21.15.19:0/4274340215 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55d485d1d630 con 0x55d485ddfa00 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6deaffd640 10 client.11952 handle_command_reply: tid=0 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 1 client.11952 shutdown 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 1 -- 172.21.15.19:0/4274340215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9ef0 msgr2=0x55d485db3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9ef0 0x55d485db3810 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f6e040073c0 tx=0x7f6e04004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 1 -- 172.21.15.19:0/4274340215 shutdown_connections 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55d485ddfa00 0x55d485e647a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d485dd9ef0 0x55d485db3810 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d485dd9b10 0x55d485dee670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6deaffd640 0 client.11952 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 1 --2- 172.21.15.19:0/4274340215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55d485dd8540 0x55d485dd4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 1 -- 172.21.15.19:0/4274340215 >> 172.21.15.19:0/4274340215 conn(0x55d485dac770 msgr2=0x55d485e06d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 1 -- 172.21.15.19:0/4274340215 shutdown_connections 2024-04-25T10:06:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 1 -- 172.21.15.19:0/4274340215 wait complete. 2024-04-25T10:06:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.582+0000 7f6e15280740 20 client.11952 trim_cache size 0 max 16384 2024-04-25T10:06:28.595 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:06:28.595 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:06:28.595 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T10:06:28.595 INFO:teuthology.orchestra.run.smithi019.stdout: "1b9ee9ba-28f4-4a3e-9d7f-958a1037d6e2": { 2024-04-25T10:06:28.596 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-04-25T10:06:28.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "1b9ee9ba-28f4-4a3e-9d7f-958a1037d6e2", 2024-04-25T10:06:28.596 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,repair" 2024-04-25T10:06:28.596 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:06:28.596 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T10:06:28.596 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:06:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 -- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6df806b4d0 msgr2=0x7f6df806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6df806b4d0 0x7f6df806d990 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc034000 tx=0x7f6dfc02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 -- 172.21.15.19:0/3700397412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e0c0b5620 msgr2=0x7f6e0c0ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e0c0b5620 0x7f6e0c0ca070 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c7f0 tx=0x7f6e0000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 -- 172.21.15.19:0/3700397412 shutdown_connections 2024-04-25T10:06:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6df806b4d0 0x7f6df806d990 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e0c15b7d0 0x7f6e0c0ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e0c0b5620 0x7f6e0c0ca070 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 --2- 172.21.15.19:0/3700397412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0c06c2d0 0x7f6e0c0aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 -- 172.21.15.19:0/3700397412 >> 172.21.15.19:0/3700397412 conn(0x7f6e0c0d0800 msgr2=0x7f6e0c15fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 -- 172.21.15.19:0/3700397412 shutdown_connections 2024-04-25T10:06:28.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:28.583+0000 7f6e1257e640 1 -- 172.21.15.19:0/3700397412 wait complete. 2024-04-25T10:06:28.599 INFO:tasks.cephfs.filesystem:scrub status for tag:1b9ee9ba-28f4-4a3e-9d7f-958a1037d6e2 - {'path': '/', 'tag': '1b9ee9ba-28f4-4a3e-9d7f-958a1037d6e2', 'options': 'recursive,repair'} 2024-04-25T10:06:39.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:39.460 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:58.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:58.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:06:58.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.711+0000 7fec265c8640 1 Processor -- start 2024-04-25T10:06:58.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.711+0000 7fec265c8640 1 -- start start 2024-04-25T10:06:58.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.712+0000 7fec265c8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec2015dac0 0x7fec200d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.712+0000 7fec265c8640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec200d2420 con 0x7fec200d0ab0 2024-04-25T10:06:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.712+0000 7fec265c8640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec200d2700 con 0x7fec200d1f30 2024-04-25T10:06:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.712+0000 7fec265c8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec200d3030 con 0x7fec2015dac0 2024-04-25T10:06:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.712+0000 7fec24b65640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fec200d0ab0 0x7fec200b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54338/0 (socket says 172.21.15.19:54338) 2024-04-25T10:06:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.712+0000 7fec24b65640 1 -- 172.21.15.19:0/1478285126 learned_addr learned my addr 172.21.15.19:0/1478285126 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:06:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.712+0000 7fec1ffff640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec2015dac0 0x7fec200d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244725923 0 0) 0x7fec200d2420 con 0x7fec200d0ab0 2024-04-25T10:06:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec0c003520 con 0x7fec200d0ab0 2024-04-25T10:06:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150239879 0 0) 0x7fec200d2700 con 0x7fec200d1f30 2024-04-25T10:06:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec200d2420 con 0x7fec200d1f30 2024-04-25T10:06:58.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365908774 0 0) 0x7fec0c003520 con 0x7fec200d0ab0 2024-04-25T10:06:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec2015dac0 msgr2=0x7fec200d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec2015dac0 0x7fec200d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 >> v1:172.21.15.112:6789/0 conn(0x7fec200d1f30 legacy=0x7fec200d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200d29e0 con 0x7fec200d0ab0 2024-04-25T10:06:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fec180019f0 con 0x7fec200d0ab0 2024-04-25T10:06:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 >> v1:172.21.15.19:6789/0 conn(0x7fec200d0ab0 legacy=0x7fec200b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec0c003b70 0x7fec0c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec0c004800 0x7fec0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec2015dac0 0x7fec0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec200d3030 con 0x7fec0c004800 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c00baa0 con 0x7fec0c003b70 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c00bca0 con 0x7fec2015dac0 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1f7fe640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec0c004800 0x7fec0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1ffff640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec0c003b70 0x7fec0c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec24b65640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec2015dac0 0x7fec0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1f7fe640 1 -- 172.21.15.19:0/1478285126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec2015dac0 msgr2=0x7fec0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1f7fe640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec2015dac0 0x7fec0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1f7fe640 1 -- 172.21.15.19:0/1478285126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec0c003b70 msgr2=0x7fec0c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1f7fe640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec0c003b70 0x7fec0c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.713+0000 7fec1f7fe640 1 -- 172.21.15.19:0/1478285126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c003520 con 0x7fec0c004800 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.714+0000 7fec1f7fe640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec0c004800 0x7fec0c004c80 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7fec10007fd0 tx=0x7fec0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995b5a8e13c13fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.714+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10001540 con 0x7fec0c004800 2024-04-25T10:06:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.714+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec100052d0 con 0x7fec0c004800 2024-04-25T10:06:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.714+0000 7fec1effd640 1 -- 172.21.15.19:0/1478285126 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1002a9d0 con 0x7fec0c004800 2024-04-25T10:06:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.715+0000 7fec265c8640 1 -- 172.21.15.19:0/1478285126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec0c004800 msgr2=0x7fec0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.715+0000 7fec265c8640 1 --2- 172.21.15.19:0/1478285126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec0c004800 0x7fec0c004c80 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7fec10007fd0 tx=0x7fec0c005830 comp rx=0 tx=0).stop 2024-04-25T10:06:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.715+0000 7fec24b65640 1 -- 172.21.15.19:0/1478285126 reap_dead start 2024-04-25T10:06:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.715+0000 7fec265c8640 1 -- 172.21.15.19:0/1478285126 shutdown_connections 2024-04-25T10:06:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.715+0000 7fec265c8640 1 -- 172.21.15.19:0/1478285126 >> 172.21.15.19:0/1478285126 conn(0x7fec20061c30 msgr2=0x7fec2006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.715+0000 7fec265c8640 1 -- 172.21.15.19:0/1478285126 shutdown_connections 2024-04-25T10:06:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.715+0000 7fec265c8640 1 -- 172.21.15.19:0/1478285126 wait complete. 2024-04-25T10:06:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.716+0000 7fec265c8640 1 Processor -- start 2024-04-25T10:06:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.716+0000 7fec265c8640 1 -- start start 2024-04-25T10:06:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.717+0000 7fec265c8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec0c004800 0x7fec201f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.717+0000 7fec265c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec20152b30 0x7fec20159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.717+0000 7fec265c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec200cd0b0 0x7fec200cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.717+0000 7fec265c8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec200b97d0 con 0x7fec200cd0b0 2024-04-25T10:06:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.717+0000 7fec265c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2015dea0 con 0x7fec20152b30 2024-04-25T10:06:58.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.717+0000 7fec265c8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec200b9950 con 0x7fec0c004800 2024-04-25T10:06:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.717+0000 7fec24b65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec200cd0b0 0x7fec200cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.717+0000 7fec24b65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec200cd0b0 0x7fec200cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45718/0 (socket says 172.21.15.19:45718) 2024-04-25T10:06:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.717+0000 7fec24b65640 1 -- 172.21.15.19:0/368830254 learned_addr learned my addr 172.21.15.19:0/368830254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.717+0000 7fec1ffff640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec0c004800 0x7fec201f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec1f7fe640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec20152b30 0x7fec20159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec24b65640 1 -- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec0c004800 msgr2=0x7fec201f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec24b65640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec0c004800 0x7fec201f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec24b65640 1 -- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec20152b30 msgr2=0x7fec20159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec24b65640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec20152b30 0x7fec20159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec24b65640 1 -- 172.21.15.19:0/368830254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200cb550 con 0x7fec200cd0b0 2024-04-25T10:06:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec24b65640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec200cd0b0 0x7fec200cd530 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fec200b6520 tx=0x7fec18007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec1cff9640 1 -- 172.21.15.19:0/368830254 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18017020 con 0x7fec200cd0b0 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec1cff9640 1 -- 172.21.15.19:0/368830254 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1800aab0 con 0x7fec200cd0b0 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec265c8640 1 -- 172.21.15.19:0/368830254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec200cb830 con 0x7fec200cd0b0 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec1cff9640 1 -- 172.21.15.19:0/368830254 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec180133f0 con 0x7fec200cd0b0 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.718+0000 7fec265c8640 1 -- 172.21.15.19:0/368830254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec200cda70 con 0x7fec200cd0b0 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.720+0000 7fec1cff9640 1 -- 172.21.15.19:0/368830254 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec1801a020 con 0x7fec200cd0b0 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.720+0000 7fec1cff9640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec0006b4d0 0x7fec0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.720+0000 7fec1cff9640 1 -- 172.21.15.19:0/368830254 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fec1808aaf0 con 0x7fec200cd0b0 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.720+0000 7fec1ffff640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec0006b4d0 0x7fec0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.721+0000 7fec1ffff640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec0006b4d0 0x7fec0006d990 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fec08009a20 tx=0x7fec08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.721+0000 7fec293a6740 1 Processor -- start 2024-04-25T10:06:58.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.721+0000 7fec293a6740 1 -- start start 2024-04-25T10:06:58.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec293a6740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088db10 0x5627f0890320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec293a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088def0 0x5627f0890860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec293a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f0860770 0x5627f0863f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec293a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f07c9da0 con 0x5627f0860770 2024-04-25T10:06:58.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec293a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f0772b80 con 0x5627f088def0 2024-04-25T10:06:58.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec293a6740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627f07c5e40 con 0x5627f088db10 2024-04-25T10:06:58.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec24b65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f0860770 0x5627f0863f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088db10 0x5627f0890320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec24b65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f0860770 0x5627f0863f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45722/0 (socket says 172.21.15.19:45722) 2024-04-25T10:06:58.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec24b65640 1 -- 172.21.15.19:0/2625948233 learned_addr learned my addr 172.21.15.19:0/2625948233 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:58.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec1f7fe640 1 --2- 172.21.15.19:0/2625948233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088def0 0x5627f0890860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec24b65640 1 -- 172.21.15.19:0/2625948233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088db10 msgr2=0x5627f0890320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec24b65640 1 --2- 172.21.15.19:0/2625948233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088db10 0x5627f0890320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec24b65640 1 -- 172.21.15.19:0/2625948233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088def0 msgr2=0x5627f0890860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec24b65640 1 --2- 172.21.15.19:0/2625948233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088def0 0x5627f0890860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec24b65640 1 -- 172.21.15.19:0/2625948233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627f077cf10 con 0x5627f0860770 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.722+0000 7fec1ffff640 1 --2- 172.21.15.19:0/2625948233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088db10 0x5627f0890320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.723+0000 7fec24b65640 1 --2- 172.21.15.19:0/2625948233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f0860770 0x5627f0863f80 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fec18061290 tx=0x7fec1805d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf96399ae0ea5d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.723+0000 7fec15ffb640 1 -- 172.21.15.19:0/2625948233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec180557d0 con 0x5627f0860770 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.723+0000 7fec15ffb640 1 -- 172.21.15.19:0/2625948233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec180585b0 con 0x5627f0860770 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.723+0000 7fec15ffb640 1 -- 172.21.15.19:0/2625948233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18055970 con 0x5627f0860770 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.723+0000 7fec293a6740 1 -- 172.21.15.19:0/2625948233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f0860770 msgr2=0x5627f0863f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.723+0000 7fec293a6740 1 --2- 172.21.15.19:0/2625948233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f0860770 0x5627f0863f80 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fec18061290 tx=0x7fec1805d340 comp rx=0 tx=0).stop 2024-04-25T10:06:58.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.723+0000 7fec293a6740 1 -- 172.21.15.19:0/2625948233 shutdown_connections 2024-04-25T10:06:58.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.723+0000 7fec293a6740 1 --2- 172.21.15.19:0/2625948233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f0860770 0x5627f0863f80 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.723+0000 7fec293a6740 1 --2- 172.21.15.19:0/2625948233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088def0 0x5627f0890860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.723+0000 7fec293a6740 1 --2- 172.21.15.19:0/2625948233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088db10 0x5627f0890320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.723+0000 7fec293a6740 1 -- 172.21.15.19:0/2625948233 >> 172.21.15.19:0/2625948233 conn(0x5627f08629e0 msgr2=0x5627f08ab320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.724+0000 7fec293a6740 1 -- 172.21.15.19:0/2625948233 shutdown_connections 2024-04-25T10:06:58.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.724+0000 7fec293a6740 1 -- 172.21.15.19:0/2625948233 wait complete. 2024-04-25T10:06:58.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.724+0000 7fec293a6740 1 -- start start 2024-04-25T10:06:58.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.724+0000 7fec293a6740 1 Processor -- start 2024-04-25T10:06:58.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec293a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088db10 0x5627f0883550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec293a6740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088def0 0x5627f08878c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec293a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f0864f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec293a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f0865b90 con 0x5627f08aef90 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec293a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f07684b0 con 0x5627f088db10 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec293a6740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627f0865e70 con 0x5627f088def0 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec293a6740 10 client.0 _ll_register_callbacks cb 0x5627f07a8180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec24b65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f0864f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec24b65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f0864f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45738/0 (socket says 172.21.15.19:45738) 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec24b65640 1 -- 172.21.15.19:0/2474400548 learned_addr learned my addr 172.21.15.19:0/2474400548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec1f7fe640 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088def0 0x5627f08878c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.725+0000 7fec1ffff640 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088db10 0x5627f0883550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec24b65640 1 -- 172.21.15.19:0/2474400548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088def0 msgr2=0x5627f08878c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec24b65640 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088def0 0x5627f08878c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec24b65640 1 -- 172.21.15.19:0/2474400548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088db10 msgr2=0x5627f0883550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec24b65640 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088db10 0x5627f0883550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec24b65640 1 -- 172.21.15.19:0/2474400548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627f05a87d0 con 0x5627f08aef90 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec24b65640 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f0864f90 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7fec18061980 tx=0x7fec1805d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59d7e20d714c425 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7febfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7febfeffd640 1 -- 172.21.15.19:0/2474400548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1805f3a0 con 0x5627f08aef90 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7febfeffd640 1 -- 172.21.15.19:0/2474400548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fec18054eb0 con 0x5627f08aef90 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec293a6740 1 -- 172.21.15.19:0/2474400548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec180585b0 con 0x5627f08aef90 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7febfeffd640 1 -- 172.21.15.19:0/2474400548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec18076050 con 0x5627f08aef90 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec293a6740 10 client.11964 fetch_fsmap learned FSMap version 129 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec293a6740 1 -- 172.21.15.19:0/2474400548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x5627f04de3f0 con 0x5627f08aef90 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7febfeffd640 1 -- 172.21.15.19:0/2474400548 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec180585b0 con 0x5627f08aef90 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7febfeffd640 1 -- 172.21.15.19:0/2474400548 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fec180550b0 con 0x5627f08aef90 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec293a6740 10 client.11964 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec293a6740 10 client.11964 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec293a6740 20 client.11964 populate_metadata read hostname 'smithi019' 2024-04-25T10:06:58.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec293a6740 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x5627f08ac920 0x5627f08acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec293a6740 4 client.11964 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:58.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.726+0000 7fec293a6740 1 -- 172.21.15.19:0/2474400548 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627f07d72e0 con 0x5627f08ac920 2024-04-25T10:06:58.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.727+0000 7fec1ffff640 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x5627f08ac920 0x5627f08acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.727+0000 7fec1ffff640 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x5627f08ac920 0x5627f08acd30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73742b7fd1d8d1ac server_cookie=dbfb8cff4fe0f27e in_seq=0 out_seq=0 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.727+0000 7febfeffd640 10 client.11964 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.727+0000 7febfeffd640 1 -- 172.21.15.19:0/2474400548 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5627f07d72e0 con 0x5627f08ac920 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.727+0000 7febfeffd640 10 client.11964 handle_command_reply: tid=0 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.727+0000 7fec293a6740 1 client.11964 shutdown 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7fec293a6740 1 -- 172.21.15.19:0/2474400548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 msgr2=0x5627f0864f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7fec293a6740 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f0864f90 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7fec18061980 tx=0x7fec1805d070 comp rx=0 tx=0).stop 2024-04-25T10:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7fec293a6740 1 -- 172.21.15.19:0/2474400548 shutdown_connections 2024-04-25T10:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7fec293a6740 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x5627f08ac920 0x5627f08acd30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7fec293a6740 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f0864f90 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7fec293a6740 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088def0 0x5627f08878c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7febfeffd640 0 client.11964 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7fec293a6740 1 --2- 172.21.15.19:0/2474400548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088db10 0x5627f0883550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7fec293a6740 1 -- 172.21.15.19:0/2474400548 >> 172.21.15.19:0/2474400548 conn(0x5627f0860770 msgr2=0x5627f0861420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7fec293a6740 1 -- 172.21.15.19:0/2474400548 shutdown_connections 2024-04-25T10:06:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7fec293a6740 1 -- 172.21.15.19:0/2474400548 wait complete. 2024-04-25T10:06:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.728+0000 7fec293a6740 20 client.11964 trim_cache size 0 max 16384 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.743+0000 7fec293a6740 1 Processor -- start 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.743+0000 7fec293a6740 1 -- start start 2024-04-25T10:06:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec293a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088db10 0x5627f08ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec293a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f088def0 0x5627f08ab510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec293a6740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f08aef90 0x5627f08bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec293a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec20200eb0 con 0x5627f088def0 2024-04-25T10:06:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec293a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec20200050 con 0x5627f088db10 2024-04-25T10:06:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec293a6740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec201ff570 con 0x5627f08aef90 2024-04-25T10:06:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f088def0 0x5627f08ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec1f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f088def0 0x5627f08ab510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45750/0 (socket says 172.21.15.19:45750) 2024-04-25T10:06:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec1f7fe640 1 -- 172.21.15.19:0/2146298754 learned_addr learned my addr 172.21.15.19:0/2146298754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec1ffff640 1 --2- 172.21.15.19:0/2146298754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088db10 0x5627f08ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec24b65640 1 --2- 172.21.15.19:0/2146298754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f08aef90 0x5627f08bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec1f7fe640 1 -- 172.21.15.19:0/2146298754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f08aef90 msgr2=0x5627f08bcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec1f7fe640 1 --2- 172.21.15.19:0/2146298754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f08aef90 0x5627f08bcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec1f7fe640 1 -- 172.21.15.19:0/2146298754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088db10 msgr2=0x5627f08ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec1f7fe640 1 --2- 172.21.15.19:0/2146298754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088db10 0x5627f08ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.744+0000 7fec1f7fe640 1 -- 172.21.15.19:0/2146298754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627f049e0f0 con 0x5627f088def0 2024-04-25T10:06:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec1f7fe640 1 --2- 172.21.15.19:0/2146298754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f088def0 0x5627f08ab510 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fec200d2350 tx=0x7fec1002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950a90c7fd26963c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec157fa640 1 -- 172.21.15.19:0/2146298754 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec100050d0 con 0x5627f088def0 2024-04-25T10:06:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec157fa640 1 -- 172.21.15.19:0/2146298754 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec10042070 con 0x5627f088def0 2024-04-25T10:06:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec157fa640 1 -- 172.21.15.19:0/2146298754 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec100054b0 con 0x5627f088def0 2024-04-25T10:06:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec293a6740 1 -- 172.21.15.19:0/2146298754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f088def0 msgr2=0x5627f08ab510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec293a6740 1 --2- 172.21.15.19:0/2146298754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f088def0 0x5627f08ab510 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fec200d2350 tx=0x7fec1002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec293a6740 1 -- 172.21.15.19:0/2146298754 shutdown_connections 2024-04-25T10:06:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec293a6740 1 --2- 172.21.15.19:0/2146298754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f08aef90 0x5627f08bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec293a6740 1 --2- 172.21.15.19:0/2146298754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f088def0 0x5627f08ab510 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec293a6740 1 --2- 172.21.15.19:0/2146298754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088db10 0x5627f08ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec293a6740 1 -- 172.21.15.19:0/2146298754 >> 172.21.15.19:0/2146298754 conn(0x5627f0860770 msgr2=0x5627f08876d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec293a6740 1 -- 172.21.15.19:0/2146298754 shutdown_connections 2024-04-25T10:06:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.745+0000 7fec293a6740 1 -- 172.21.15.19:0/2146298754 wait complete. 2024-04-25T10:06:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.746+0000 7fec293a6740 1 -- start start 2024-04-25T10:06:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.746+0000 7fec293a6740 1 Processor -- start 2024-04-25T10:06:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec293a6740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088db10 0x5627f08aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec293a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088def0 0x5627f09148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec293a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f08ba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec293a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec201ffa50 con 0x5627f08aef90 2024-04-25T10:06:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec293a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f0890380 con 0x5627f088def0 2024-04-25T10:06:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec293a6740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627f088be90 con 0x5627f088db10 2024-04-25T10:06:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec293a6740 10 client.0 _ll_register_callbacks cb 0x7fec18055480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec24b65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f08ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec24b65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f08ba000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45752/0 (socket says 172.21.15.19:45752) 2024-04-25T10:06:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec24b65640 1 -- 172.21.15.19:0/496734063 learned_addr learned my addr 172.21.15.19:0/496734063 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec1ffff640 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088db10 0x5627f08aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec1f7fe640 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088def0 0x5627f09148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec24b65640 1 -- 172.21.15.19:0/496734063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088db10 msgr2=0x5627f08aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec24b65640 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088db10 0x5627f08aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec24b65640 1 -- 172.21.15.19:0/496734063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088def0 msgr2=0x5627f09148e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec24b65640 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088def0 0x5627f09148e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.747+0000 7fec24b65640 1 -- 172.21.15.19:0/496734063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627f08662c0 con 0x5627f08aef90 2024-04-25T10:06:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7fec24b65640 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f08ba000 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fec18004fd0 tx=0x7fec18001300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b859ada82dcc367f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7febfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:06:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7febfeffd640 1 -- 172.21.15.19:0/496734063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1807c020 con 0x5627f08aef90 2024-04-25T10:06:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7febfeffd640 1 -- 172.21.15.19:0/496734063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fec18015970 con 0x5627f08aef90 2024-04-25T10:06:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7fec293a6740 1 -- 172.21.15.19:0/496734063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec10042070 con 0x5627f08aef90 2024-04-25T10:06:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7febfeffd640 1 -- 172.21.15.19:0/496734063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec1805f4f0 con 0x5627f08aef90 2024-04-25T10:06:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7fec293a6740 10 client.11970 fetch_fsmap learned FSMap version 129 2024-04-25T10:06:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7fec293a6740 1 -- 172.21.15.19:0/496734063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x5627f08662c0 con 0x5627f08aef90 2024-04-25T10:06:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7febfeffd640 1 -- 172.21.15.19:0/496734063 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec10042070 con 0x5627f08aef90 2024-04-25T10:06:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7febfeffd640 1 -- 172.21.15.19:0/496734063 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fec18015b90 con 0x5627f08aef90 2024-04-25T10:06:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7fec293a6740 10 client.11970 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:06:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7fec293a6740 10 client.11970 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:06:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7fec293a6740 20 client.11970 populate_metadata read hostname 'smithi019' 2024-04-25T10:06:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7fec293a6740 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x5627f08ba540 0x5627f088b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7fec293a6740 4 client.11970 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7fec293a6740 1 -- 172.21.15.19:0/496734063 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5627f07d1630 con 0x5627f08ba540 2024-04-25T10:06:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.748+0000 7fec1ffff640 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x5627f08ba540 0x5627f088b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec1ffff640 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x5627f08ba540 0x5627f088b230 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c425b73a7fad4019 server_cookie=9f6963430244696d in_seq=0 out_seq=0 2024-04-25T10:06:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7febfeffd640 10 client.11970 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7febfeffd640 1 -- 172.21.15.19:0/496734063 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5627f07d1630 con 0x5627f08ba540 2024-04-25T10:06:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7febfeffd640 10 client.11970 handle_command_reply: tid=0 2024-04-25T10:06:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 1 client.11970 shutdown 2024-04-25T10:06:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 1 -- 172.21.15.19:0/496734063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 msgr2=0x5627f08ba000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f08ba000 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fec18004fd0 tx=0x7fec18001300 comp rx=0 tx=0).stop 2024-04-25T10:06:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 1 -- 172.21.15.19:0/496734063 shutdown_connections 2024-04-25T10:06:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x5627f08ba540 0x5627f088b230 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5627f08aef90 0x5627f08ba000 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627f088def0 0x5627f09148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 1 --2- 172.21.15.19:0/496734063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5627f088db10 0x5627f08aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7febfeffd640 0 client.11970 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 1 -- 172.21.15.19:0/496734063 >> 172.21.15.19:0/496734063 conn(0x5627f0860770 msgr2=0x5627f088c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 1 -- 172.21.15.19:0/496734063 shutdown_connections 2024-04-25T10:06:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 1 -- 172.21.15.19:0/496734063 wait complete. 2024-04-25T10:06:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.749+0000 7fec293a6740 20 client.11970 trim_cache size 0 max 16384 2024-04-25T10:06:58.772 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:06:58.772 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T10:06:58.773 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T10:06:58.773 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:06:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 -- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec0006b4d0 msgr2=0x7fec0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec0006b4d0 0x7fec0006d990 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fec08009a20 tx=0x7fec08008040 comp rx=0 tx=0).stop 2024-04-25T10:06:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 -- 172.21.15.19:0/368830254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec200cd0b0 msgr2=0x7fec200cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec200cd0b0 0x7fec200cd530 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fec200b6520 tx=0x7fec18007bd0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 -- 172.21.15.19:0/368830254 shutdown_connections 2024-04-25T10:06:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fec0006b4d0 0x7fec0006d990 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec200cd0b0 0x7fec200cd530 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec20152b30 0x7fec20159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 --2- 172.21.15.19:0/368830254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec0c004800 0x7fec201f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 -- 172.21.15.19:0/368830254 >> 172.21.15.19:0/368830254 conn(0x7fec20061c30 msgr2=0x7fec2015d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 -- 172.21.15.19:0/368830254 shutdown_connections 2024-04-25T10:06:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:58.750+0000 7fec265c8640 1 -- 172.21.15.19:0/368830254 wait complete. 2024-04-25T10:06:58.776 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:06:59.180 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:59.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.278+0000 7ff488e5a640 1 Processor -- start 2024-04-25T10:06:59.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.278+0000 7ff488e5a640 1 -- start start 2024-04-25T10:06:59.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff488e5a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4840b5c30 0x7ff484168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff488e5a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4840ceee0 con 0x7ff48415db70 2024-04-25T10:06:59.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff488e5a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4840d1d10 con 0x7ff4840b6cb0 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff488e5a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840d3800 con 0x7ff4840b5c30 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff48259c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff48415db70 0x7ff484162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54342/0 (socket says 172.21.15.19:54342) 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff48259c640 1 -- 172.21.15.19:0/1171158662 learned_addr learned my addr 172.21.15.19:0/1171158662 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262352196 0 0) 0x7ff4840ceee0 con 0x7ff48415db70 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff482d9d640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4840b5c30 0x7ff484168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff470003520 con 0x7ff48415db70 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 453331995 0 0) 0x7ff470003520 con 0x7ff48415db70 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4840b5c30 msgr2=0x7ff484168ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4840b5c30 0x7ff484168ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 >> v1:172.21.15.112:6789/0 conn(0x7ff4840b6cb0 legacy=0x7ff4840b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4840d1ff0 con 0x7ff48415db70 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7ff474001670 con 0x7ff48415db70 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 >> v1:172.21.15.19:6789/0 conn(0x7ff48415db70 legacy=0x7ff484162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4700039e0 0x7ff470003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4840b5c30 0x7ff470004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.279+0000 7ff481d9b640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff470004d40 0x7ff4700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.280+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840d3800 con 0x7ff4700039e0 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.280+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47000baa0 con 0x7ff470004d40 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.280+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47000bca0 con 0x7ff4840b5c30 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.280+0000 7ff48259c640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4700039e0 0x7ff470003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.280+0000 7ff47bfff640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4840b5c30 0x7ff470004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.280+0000 7ff482d9d640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff470004d40 0x7ff4700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.280+0000 7ff48259c640 1 -- 172.21.15.19:0/1171158662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4840b5c30 msgr2=0x7ff470004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.280+0000 7ff48259c640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4840b5c30 0x7ff470004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.280+0000 7ff48259c640 1 -- 172.21.15.19:0/1171158662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff470004d40 msgr2=0x7ff4700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.280+0000 7ff48259c640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff470004d40 0x7ff4700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.280+0000 7ff48259c640 1 -- 172.21.15.19:0/1171158662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff470003520 con 0x7ff4700039e0 2024-04-25T10:06:59.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.281+0000 7ff48259c640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4700039e0 0x7ff470003e00 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7ff47400c210 tx=0x7ff470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5842ad73a23c23d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.281+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474007cf0 con 0x7ff4700039e0 2024-04-25T10:06:59.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.281+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff474005af0 con 0x7ff4700039e0 2024-04-25T10:06:59.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.281+0000 7ff481d9b640 1 -- 172.21.15.19:0/1171158662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474007070 con 0x7ff4700039e0 2024-04-25T10:06:59.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.281+0000 7ff488e5a640 1 -- 172.21.15.19:0/1171158662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4700039e0 msgr2=0x7ff470003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.281+0000 7ff488e5a640 1 --2- 172.21.15.19:0/1171158662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4700039e0 0x7ff470003e00 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7ff47400c210 tx=0x7ff470005830 comp rx=0 tx=0).stop 2024-04-25T10:06:59.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.281+0000 7ff482d9d640 1 -- 172.21.15.19:0/1171158662 reap_dead start 2024-04-25T10:06:59.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.281+0000 7ff488e5a640 1 -- 172.21.15.19:0/1171158662 shutdown_connections 2024-04-25T10:06:59.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.281+0000 7ff488e5a640 1 -- 172.21.15.19:0/1171158662 >> 172.21.15.19:0/1171158662 conn(0x7ff4840d0520 msgr2=0x7ff4840d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.281+0000 7ff488e5a640 1 -- 172.21.15.19:0/1171158662 shutdown_connections 2024-04-25T10:06:59.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.281+0000 7ff488e5a640 1 -- 172.21.15.19:0/1171158662 wait complete. 2024-04-25T10:06:59.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.283+0000 7ff488e5a640 1 Processor -- start 2024-04-25T10:06:59.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.283+0000 7ff488e5a640 1 -- start start 2024-04-25T10:06:59.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.283+0000 7ff488e5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4700039e0 0x7ff48415bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.283+0000 7ff488e5a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff48415ed20 0x7ff484160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.283+0000 7ff488e5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff48415ac90 0x7ff48415b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.283+0000 7ff488e5a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840ce530 con 0x7ff48415ac90 2024-04-25T10:06:59.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.283+0000 7ff488e5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48415df50 con 0x7ff4700039e0 2024-04-25T10:06:59.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.283+0000 7ff488e5a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840cf1c0 con 0x7ff48415ed20 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff482d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff48415ac90 0x7ff48415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff482d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff48415ac90 0x7ff48415b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45778/0 (socket says 172.21.15.19:45778) 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff482d9d640 1 -- 172.21.15.19:0/3405824887 learned_addr learned my addr 172.21.15.19:0/3405824887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff48259c640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4700039e0 0x7ff48415bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff47bfff640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff48415ed20 0x7ff484160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff482d9d640 1 -- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff48415ed20 msgr2=0x7ff484160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff482d9d640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff48415ed20 0x7ff484160cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff482d9d640 1 -- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4700039e0 msgr2=0x7ff48415bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff482d9d640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4700039e0 0x7ff48415bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff482d9d640 1 -- 172.21.15.19:0/3405824887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48415f200 con 0x7ff48415ac90 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff482d9d640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff48415ac90 0x7ff48415b140 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7ff48408cce0 tx=0x7ff47c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff47affd640 1 -- 172.21.15.19:0/3405824887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c019070 con 0x7ff48415ac90 2024-04-25T10:06:59.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff47affd640 1 -- 172.21.15.19:0/3405824887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c011e90 con 0x7ff48415ac90 2024-04-25T10:06:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff488e5a640 1 -- 172.21.15.19:0/3405824887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48415f4e0 con 0x7ff48415ac90 2024-04-25T10:06:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff47affd640 1 -- 172.21.15.19:0/3405824887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c003bd0 con 0x7ff48415ac90 2024-04-25T10:06:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.284+0000 7ff488e5a640 1 -- 172.21.15.19:0/3405824887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48415faa0 con 0x7ff48415ac90 2024-04-25T10:06:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.285+0000 7ff488e5a640 1 -- 172.21.15.19:0/3405824887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48415fe80 con 0x7ff48415ac90 2024-04-25T10:06:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.286+0000 7ff47affd640 1 -- 172.21.15.19:0/3405824887 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff47c01c070 con 0x7ff48415ac90 2024-04-25T10:06:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.286+0000 7ff47affd640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff46406b4d0 0x7ff46406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.286+0000 7ff47affd640 1 -- 172.21.15.19:0/3405824887 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ff47c08acb0 con 0x7ff48415ac90 2024-04-25T10:06:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.286+0000 7ff48259c640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff46406b4d0 0x7ff46406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.287+0000 7ff48259c640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff46406b4d0 0x7ff46406d990 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff474032000 tx=0x7ff474009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.291+0000 7ff47affd640 1 -- 172.21.15.19:0/3405824887 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff48415fe80 con 0x7ff48415ac90 2024-04-25T10:06:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.506+0000 7ff488e5a640 1 -- 172.21.15.19:0/3405824887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff484073990 con 0x7ff48415ac90 2024-04-25T10:06:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.506+0000 7ff47affd640 1 -- 172.21.15.19:0/3405824887 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+4956 (secure 0 0 0) 0x7ff47c010040 con 0x7ff48415ac90 2024-04-25T10:06:59.512 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:06:59.512 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":129,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:38.424174+0000","modified":"2024-04-25T10:05:55.398362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":192,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11799":{"gid":11799,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/1758455116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1758455116},{"type":"v1","addr":"172.21.15.19:6835","nonce":1758455116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:06:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 129 2024-04-25T10:06:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 -- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff46406b4d0 msgr2=0x7ff46406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff46406b4d0 0x7ff46406d990 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff474032000 tx=0x7ff474009030 comp rx=0 tx=0).stop 2024-04-25T10:06:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 -- 172.21.15.19:0/3405824887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff48415ac90 msgr2=0x7ff48415b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff48415ac90 0x7ff48415b140 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7ff48408cce0 tx=0x7ff47c00f580 comp rx=0 tx=0).stop 2024-04-25T10:06:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 -- 172.21.15.19:0/3405824887 shutdown_connections 2024-04-25T10:06:59.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff46406b4d0 0x7ff46406d990 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff48415ac90 0x7ff48415b140 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff48415ed20 0x7ff484160cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 --2- 172.21.15.19:0/3405824887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4700039e0 0x7ff48415bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 -- 172.21.15.19:0/3405824887 >> 172.21.15.19:0/3405824887 conn(0x7ff4840d0520 msgr2=0x7ff4840b8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 -- 172.21.15.19:0/3405824887 shutdown_connections 2024-04-25T10:06:59.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.508+0000 7ff488e5a640 1 -- 172.21.15.19:0/3405824887 wait complete. 2024-04-25T10:06:59.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 129, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:38.424174+0000', 'modified': '2024-04-25T10:05:55.398362+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 192, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11799': {'gid': 11799, 'name': 'c', 'rank': 0, 'incarnation': 126, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6835/1758455116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 1758455116}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 1758455116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:06:59.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:06:59.525 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:06:59.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.621+0000 7fa9193b7640 1 Processor -- start 2024-04-25T10:06:59.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.622+0000 7fa9193b7640 1 -- start start 2024-04-25T10:06:59.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.622+0000 7fa9193b7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa9140b5620 0x7fa9140b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.622+0000 7fa9193b7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa91406c2d0 0x7fa91406c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.622+0000 7fa9193b7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91406cc80 con 0x7fa91415b7d0 2024-04-25T10:06:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.622+0000 7fa9193b7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91406cf50 con 0x7fa9140b5620 2024-04-25T10:06:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.622+0000 7fa9193b7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140ce530 con 0x7fa91406c2d0 2024-04-25T10:06:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.622+0000 7fa9127fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa9140b5620 0x7fa9140b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.622+0000 7fa912ffd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa91415b7d0 0x7fa9140cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54346/0 (socket says 172.21.15.19:54346) 2024-04-25T10:06:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.623+0000 7fa912ffd640 1 -- 172.21.15.19:0/3440728014 learned_addr learned my addr 172.21.15.19:0/3440728014 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:06:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.623+0000 7fa9137fe640 1 --2- 172.21.15.19:0/3440728014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa91406c2d0 0x7fa91406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.623+0000 7fa9127fc640 1 -- 172.21.15.19:0/3440728014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa91406c2d0 msgr2=0x7fa91406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.623+0000 7fa9127fc640 1 --2- 172.21.15.19:0/3440728014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa91406c2d0 0x7fa91406c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.623+0000 7fa9127fc640 1 -- 172.21.15.19:0/3440728014 >> v1:172.21.15.19:6789/0 conn(0x7fa91415b7d0 legacy=0x7fa9140cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.623+0000 7fa9127fc640 1 -- 172.21.15.19:0/3440728014 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140ce810 con 0x7fa9140b5620 2024-04-25T10:06:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.623+0000 7fa911ffb640 1 -- 172.21.15.19:0/3440728014 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054943774 0 0) 0x7fa91406cc80 con 0x7fa91415b7d0 2024-04-25T10:06:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.623+0000 7fa9137fe640 1 --2- 172.21.15.19:0/3440728014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa91406c2d0 0x7fa91406c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa9127fc640 1 --2- 172.21.15.19:0/3440728014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9140b5620 0x7fa9140b5ab0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7fa9080097a0 tx=0x7fa90802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34aa4e1a5f6667e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa911ffb640 1 -- 172.21.15.19:0/3440728014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908037070 con 0x7fa9140b5620 2024-04-25T10:06:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa911ffb640 1 -- 172.21.15.19:0/3440728014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908032440 con 0x7fa9140b5620 2024-04-25T10:06:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa911ffb640 1 -- 172.21.15.19:0/3440728014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908032810 con 0x7fa9140b5620 2024-04-25T10:06:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa9193b7640 1 -- 172.21.15.19:0/3440728014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9140b5620 msgr2=0x7fa9140b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa9193b7640 1 --2- 172.21.15.19:0/3440728014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9140b5620 0x7fa9140b5ab0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7fa9080097a0 tx=0x7fa90802a180 comp rx=0 tx=0).stop 2024-04-25T10:06:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa9193b7640 1 -- 172.21.15.19:0/3440728014 shutdown_connections 2024-04-25T10:06:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa9193b7640 1 --2- 172.21.15.19:0/3440728014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa91406c2d0 0x7fa91406c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa9193b7640 1 --2- 172.21.15.19:0/3440728014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9140b5620 0x7fa9140b5ab0 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa9193b7640 1 -- 172.21.15.19:0/3440728014 >> 172.21.15.19:0/3440728014 conn(0x7fa9140d0800 msgr2=0x7fa9140d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa9193b7640 1 -- 172.21.15.19:0/3440728014 shutdown_connections 2024-04-25T10:06:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.624+0000 7fa9193b7640 1 -- 172.21.15.19:0/3440728014 wait complete. 2024-04-25T10:06:59.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.626+0000 7fa9193b7640 1 Processor -- start 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.626+0000 7fa9193b7640 1 -- start start 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.626+0000 7fa9193b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa91406c2d0 0x7fa91414f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.626+0000 7fa9193b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9140b5620 0x7fa9140b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.626+0000 7fa9193b7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa91415b7d0 0x7fa914160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.626+0000 7fa9193b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91415bbb0 con 0x7fa9140b5620 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.626+0000 7fa9193b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91406d230 con 0x7fa91406c2d0 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.626+0000 7fa9193b7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91415bd30 con 0x7fa91415b7d0 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9127fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9140b5620 0x7fa9140b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9127fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9140b5620 0x7fa9140b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45796/0 (socket says 172.21.15.19:45796) 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9127fc640 1 -- 172.21.15.19:0/627684244 learned_addr learned my addr 172.21.15.19:0/627684244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9137fe640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa91415b7d0 0x7fa914160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa912ffd640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa91406c2d0 0x7fa91414f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9127fc640 1 -- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa91415b7d0 msgr2=0x7fa914160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9127fc640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa91415b7d0 0x7fa914160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9127fc640 1 -- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa91406c2d0 msgr2=0x7fa91414f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9127fc640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa91406c2d0 0x7fa91414f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9127fc640 1 -- 172.21.15.19:0/627684244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91415f7a0 con 0x7fa9140b5620 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa912ffd640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa91406c2d0 0x7fa91414f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9127fc640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9140b5620 0x7fa9140b4240 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7fa908035040 tx=0x7fa9080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa8fb7fe640 1 -- 172.21.15.19:0/627684244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908037040 con 0x7fa9140b5620 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa8fb7fe640 1 -- 172.21.15.19:0/627684244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908030770 con 0x7fa9140b5620 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9193b7640 1 -- 172.21.15.19:0/627684244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91415fa80 con 0x7fa9140b5620 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa8fb7fe640 1 -- 172.21.15.19:0/627684244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908030b80 con 0x7fa9140b5620 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.627+0000 7fa9193b7640 1 -- 172.21.15.19:0/627684244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa914160040 con 0x7fa9140b5620 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.629+0000 7fa8fb7fe640 1 -- 172.21.15.19:0/627684244 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa908044020 con 0x7fa9140b5620 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.629+0000 7fa8fb7fe640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa90006b500 0x7fa90006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.629+0000 7fa8fb7fe640 1 -- 172.21.15.19:0/627684244 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa9080aa900 con 0x7fa9140b5620 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.630+0000 7fa912ffd640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa90006b500 0x7fa90006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.630+0000 7fa912ffd640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa90006b500 0x7fa90006d9c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa90c000f60 tx=0x7fa90c007cc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.630+0000 7fa91c197740 1 Processor -- start 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.630+0000 7fa91c197740 1 -- start start 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.631+0000 7fa91c197740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 0x55c24fbf6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.631+0000 7fa91c197740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc14020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.631+0000 7fa91c197740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fbcbf10 0x55c24fc12190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.631+0000 7fa91c197740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24fb2ea10 con 0x55c24fc13ad0 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.631+0000 7fa91c197740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24fad7b30 con 0x55c24fbf6230 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.631+0000 7fa91c197740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c24fb2ac50 con 0x55c24fbcbf10 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.631+0000 7fa9127fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc14020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.631+0000 7fa9127fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc14020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45812/0 (socket says 172.21.15.19:45812) 2024-04-25T10:06:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.631+0000 7fa9127fc640 1 -- 172.21.15.19:0/3134284398 learned_addr learned my addr 172.21.15.19:0/3134284398 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.631+0000 7fa9137fe640 1 --2- 172.21.15.19:0/3134284398 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fbcbf10 0x55c24fc12190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.631+0000 7fa912ffd640 1 --2- 172.21.15.19:0/3134284398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 0x55c24fbf6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.632+0000 7fa9127fc640 1 -- 172.21.15.19:0/3134284398 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fbcbf10 msgr2=0x55c24fc12190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.632+0000 7fa9127fc640 1 --2- 172.21.15.19:0/3134284398 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fbcbf10 0x55c24fc12190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.632+0000 7fa9127fc640 1 -- 172.21.15.19:0/3134284398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 msgr2=0x55c24fbf6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.632+0000 7fa9127fc640 1 --2- 172.21.15.19:0/3134284398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 0x55c24fbf6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.632+0000 7fa9127fc640 1 -- 172.21.15.19:0/3134284398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c24faece10 con 0x55c24fc13ad0 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.632+0000 7fa9127fc640 1 --2- 172.21.15.19:0/3134284398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc14020 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7fa9080a8be0 tx=0x7fa908078a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6ed19aa34e5961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.632+0000 7fa8f8ff9640 1 -- 172.21.15.19:0/3134284398 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908078e30 con 0x55c24fc13ad0 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.632+0000 7fa8f8ff9640 1 -- 172.21.15.19:0/3134284398 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90807cf80 con 0x55c24fc13ad0 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.632+0000 7fa8f8ff9640 1 -- 172.21.15.19:0/3134284398 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908081540 con 0x55c24fc13ad0 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.632+0000 7fa91c197740 1 -- 172.21.15.19:0/3134284398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 msgr2=0x55c24fc14020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.632+0000 7fa91c197740 1 --2- 172.21.15.19:0/3134284398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc14020 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7fa9080a8be0 tx=0x7fa908078a30 comp rx=0 tx=0).stop 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.633+0000 7fa91c197740 1 -- 172.21.15.19:0/3134284398 shutdown_connections 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.633+0000 7fa91c197740 1 --2- 172.21.15.19:0/3134284398 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fbcbf10 0x55c24fc12190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.633+0000 7fa91c197740 1 --2- 172.21.15.19:0/3134284398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc14020 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.633+0000 7fa91c197740 1 --2- 172.21.15.19:0/3134284398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 0x55c24fbf6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.633+0000 7fa91c197740 1 -- 172.21.15.19:0/3134284398 >> 172.21.15.19:0/3134284398 conn(0x55c24fbc8840 msgr2=0x55c24fc11040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.633+0000 7fa91c197740 1 -- 172.21.15.19:0/3134284398 shutdown_connections 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.633+0000 7fa91c197740 1 -- 172.21.15.19:0/3134284398 wait complete. 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.633+0000 7fa91c197740 1 -- start start 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.633+0000 7fa91c197740 1 Processor -- start 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa91c197740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fbf6230 0x55c24fbdf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa91c197740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fc13ad0 0x55c24fbf4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa91c197740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc145c0 0x55c24fc14a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa91c197740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24fb2ea10 con 0x55c24fbf6230 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa91c197740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24faeb0c0 con 0x55c24fc13ad0 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa91c197740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c24fbcccb0 con 0x55c24fc145c0 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa91c197740 10 client.0 _ll_register_callbacks cb 0x55c24fadeec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa912ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fbf6230 0x55c24fbdf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa912ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fbf6230 0x55c24fbdf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45814/0 (socket says 172.21.15.19:45814) 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa912ffd640 1 -- 172.21.15.19:0/289566255 learned_addr learned my addr 172.21.15.19:0/289566255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa9137fe640 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc145c0 0x55c24fc14a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa9127fc640 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fc13ad0 0x55c24fbf4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.634+0000 7fa912ffd640 1 -- 172.21.15.19:0/289566255 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc145c0 msgr2=0x55c24fc14a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa912ffd640 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc145c0 0x55c24fc14a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa912ffd640 1 -- 172.21.15.19:0/289566255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fc13ad0 msgr2=0x55c24fbf4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa912ffd640 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fc13ad0 0x55c24fbf4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa912ffd640 1 -- 172.21.15.19:0/289566255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c24f8432d0 con 0x55c24fbf6230 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa912ffd640 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fbf6230 0x55c24fbdf470 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fa90c001db0 tx=0x7fa90c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1c0fb43a290af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa8f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/289566255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c027060 con 0x55c24fbf6230 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/289566255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa90c010040 con 0x55c24fbf6230 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa91c197740 1 -- 172.21.15.19:0/289566255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa90807cf80 con 0x55c24fbf6230 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/289566255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa90c015e50 con 0x55c24fbf6230 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa91c197740 10 client.11991 fetch_fsmap learned FSMap version 129 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa91c197740 1 -- 172.21.15.19:0/289566255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x55c24f90d6b0 con 0x55c24fbf6230 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/289566255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90807cf80 con 0x55c24fbf6230 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/289566255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa90c013420 con 0x55c24fbf6230 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa91c197740 10 client.11991 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:06:59.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa91c197740 10 client.11991 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa91c197740 20 client.11991 populate_metadata read hostname 'smithi019' 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa91c197740 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55c24fbe2380 0x55c24fbe2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa91c197740 4 client.11991 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.635+0000 7fa91c197740 1 -- 172.21.15.19:0/289566255 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c24fb8d920 con 0x55c24fbe2380 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa9127fc640 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55c24fbe2380 0x55c24fbe2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa9127fc640 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55c24fbe2380 0x55c24fbe2760 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f52e9845b83b34 server_cookie=cee1cf215a4768e2 in_seq=0 out_seq=0 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa8f1ffb640 10 client.11991 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/289566255 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c24fb8d920 con 0x55c24fbe2380 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa8f1ffb640 10 client.11991 handle_command_reply: tid=0 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa91c197740 1 client.11991 shutdown 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa91c197740 1 -- 172.21.15.19:0/289566255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fbf6230 msgr2=0x55c24fbdf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa91c197740 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fbf6230 0x55c24fbdf470 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fa90c001db0 tx=0x7fa90c011000 comp rx=0 tx=0).stop 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa91c197740 1 -- 172.21.15.19:0/289566255 shutdown_connections 2024-04-25T10:06:59.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa91c197740 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55c24fbe2380 0x55c24fbe2760 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa91c197740 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc145c0 0x55c24fc14a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa91c197740 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fc13ad0 0x55c24fbf4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa8f1ffb640 0 client.11991 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa91c197740 1 --2- 172.21.15.19:0/289566255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fbf6230 0x55c24fbdf470 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.636+0000 7fa91c197740 1 -- 172.21.15.19:0/289566255 >> 172.21.15.19:0/289566255 conn(0x55c24fbcbf10 msgr2=0x55c24fbc5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.637+0000 7fa91c197740 1 -- 172.21.15.19:0/289566255 shutdown_connections 2024-04-25T10:06:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.637+0000 7fa91c197740 1 -- 172.21.15.19:0/289566255 wait complete. 2024-04-25T10:06:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.637+0000 7fa91c197740 20 client.11991 trim_cache size 0 max 16384 2024-04-25T10:06:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.652+0000 7fa91c197740 1 Processor -- start 2024-04-25T10:06:59.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.652+0000 7fa91c197740 1 -- start start 2024-04-25T10:06:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa91c197740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 0x55c24fbcc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa91c197740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc11040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa91c197740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc145c0 0x55c24fc11580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa91c197740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa914200530 con 0x55c24fc13ad0 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa91c197740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141ff730 con 0x55c24fbf6230 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa91c197740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141f7870 con 0x55c24fc145c0 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa9127fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc11040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa9127fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc11040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45828/0 (socket says 172.21.15.19:45828) 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa9127fc640 1 -- 172.21.15.19:0/3972911174 learned_addr learned my addr 172.21.15.19:0/3972911174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa9127fc640 1 -- 172.21.15.19:0/3972911174 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc145c0 msgr2=0x55c24fc11580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa9127fc640 1 --2- 172.21.15.19:0/3972911174 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc145c0 0x55c24fc11580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa9127fc640 1 -- 172.21.15.19:0/3972911174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 msgr2=0x55c24fbcc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa9127fc640 1 --2- 172.21.15.19:0/3972911174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 0x55c24fbcc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.653+0000 7fa9127fc640 1 -- 172.21.15.19:0/3972911174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c24fbf7780 con 0x55c24fc13ad0 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa9127fc640 1 --2- 172.21.15.19:0/3972911174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc11040 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7fa9080a8b50 tx=0x7fa90803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cf3766d5412ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa8f3fff640 1 -- 172.21.15.19:0/3972911174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90807cf80 con 0x55c24fc13ad0 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa8f3fff640 1 -- 172.21.15.19:0/3972911174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908078e30 con 0x55c24fc13ad0 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa8f3fff640 1 -- 172.21.15.19:0/3972911174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908030d20 con 0x55c24fc13ad0 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa91c197740 1 -- 172.21.15.19:0/3972911174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 msgr2=0x55c24fc11040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa91c197740 1 --2- 172.21.15.19:0/3972911174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc11040 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7fa9080a8b50 tx=0x7fa90803e000 comp rx=0 tx=0).stop 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa91c197740 1 -- 172.21.15.19:0/3972911174 shutdown_connections 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa91c197740 1 --2- 172.21.15.19:0/3972911174 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc145c0 0x55c24fc11580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa91c197740 1 --2- 172.21.15.19:0/3972911174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc13ad0 0x55c24fc11040 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa91c197740 1 --2- 172.21.15.19:0/3972911174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 0x55c24fbcc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa91c197740 1 -- 172.21.15.19:0/3972911174 >> 172.21.15.19:0/3972911174 conn(0x55c24fbcbf10 msgr2=0x55c24fbdf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa91c197740 1 -- 172.21.15.19:0/3972911174 shutdown_connections 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.654+0000 7fa91c197740 1 -- 172.21.15.19:0/3972911174 wait complete. 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.655+0000 7fa91c197740 1 -- start start 2024-04-25T10:06:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.655+0000 7fa91c197740 1 Processor -- start 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa91c197740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 0x55c24fc7b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa91c197740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc13ad0 0x55c24fc11040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa91c197740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc145c0 0x55c24fc11580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa91c197740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24fbe0400 con 0x55c24fc145c0 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa91c197740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141ff2d0 con 0x55c24fbf6230 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa91c197740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c24fbe19a0 con 0x55c24fc13ad0 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa91c197740 10 client.0 _ll_register_callbacks cb 0x55c24fbc6c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa9137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc145c0 0x55c24fc11580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa9137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc145c0 0x55c24fc11580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45830/0 (socket says 172.21.15.19:45830) 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa9137fe640 1 -- 172.21.15.19:0/844563372 learned_addr learned my addr 172.21.15.19:0/844563372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa9127fc640 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc13ad0 0x55c24fc11040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa912ffd640 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 0x55c24fc7b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa9137fe640 1 -- 172.21.15.19:0/844563372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc13ad0 msgr2=0x55c24fc11040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa9137fe640 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc13ad0 0x55c24fc11040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa9137fe640 1 -- 172.21.15.19:0/844563372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 msgr2=0x55c24fc7b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa9137fe640 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 0x55c24fc7b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa9137fe640 1 -- 172.21.15.19:0/844563372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c24fbf4a40 con 0x55c24fc145c0 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa9137fe640 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc145c0 0x55c24fc11580 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fa904004660 tx=0x7fa904007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d795fb041496c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa8f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/844563372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90401b070 con 0x55c24fc145c0 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/844563372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa904017440 con 0x55c24fc145c0 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.656+0000 7fa91c197740 1 -- 172.21.15.19:0/844563372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa908078e30 con 0x55c24fc145c0 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/844563372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa904026050 con 0x55c24fc145c0 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa91c197740 10 client.11997 fetch_fsmap learned FSMap version 129 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa91c197740 1 -- 172.21.15.19:0/844563372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x55c24fbf5560 con 0x55c24fc145c0 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/844563372 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908078e30 con 0x55c24fc145c0 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/844563372 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 129) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa904029020 con 0x55c24fc145c0 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa91c197740 10 client.11997 fetch_fsmap finished waiting for FSMap version 129 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa91c197740 10 client.11997 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa91c197740 20 client.11997 populate_metadata read hostname 'smithi019' 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa91c197740 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55c24fbf8560 0x55c24fc11e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa91c197740 4 client.11997 mds_command: new command op to 11799 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa91c197740 1 -- 172.21.15.19:0/844563372 --> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c24fc099d0 con 0x55c24fbf8560 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa912ffd640 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55c24fbf8560 0x55c24fc11e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa8f1ffb640 10 client.11997 ms_handle_connect on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.657+0000 7fa912ffd640 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55c24fbf8560 0x55c24fc11e50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aff0d4591862cff server_cookie=756e2f315401b982 in_seq=0 out_seq=0 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa8f1ffb640 1 -- 172.21.15.19:0/844563372 <== mds.0 v2:172.21.15.19:6833/1758455116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55c24fc099d0 con 0x55c24fbf8560 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa8f1ffb640 10 client.11997 handle_command_reply: tid=0 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 1 client.11997 shutdown 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 1 -- 172.21.15.19:0/844563372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc145c0 msgr2=0x55c24fc11580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc145c0 0x55c24fc11580 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fa904004660 tx=0x7fa904007700 comp rx=0 tx=0).stop 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 1 -- 172.21.15.19:0/844563372 shutdown_connections 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.19:6833/1758455116,v1:172.21.15.19:6835/1758455116] conn(0x55c24fbf8560 0x55c24fc11e50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c24fc145c0 0x55c24fc11580 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x55c24fc13ad0 0x55c24fc11040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 1 --2- 172.21.15.19:0/844563372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c24fbf6230 0x55c24fc7b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 1 -- 172.21.15.19:0/844563372 >> 172.21.15.19:0/844563372 conn(0x55c24fbcbf10 msgr2=0x55c24fbf93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa8f1ffb640 0 client.11997 ms_handle_reset on v2:172.21.15.19:6833/1758455116 2024-04-25T10:06:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 1 -- 172.21.15.19:0/844563372 shutdown_connections 2024-04-25T10:06:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 1 -- 172.21.15.19:0/844563372 wait complete. 2024-04-25T10:06:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa91c197740 20 client.11997 trim_cache size 0 max 16384 2024-04-25T10:06:59.673 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:06:59.673 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "342f98bd-d5a6-4934-bb27-7f8bea67a3be", 2024-04-25T10:06:59.673 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T10:06:59.673 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 11799, 2024-04-25T10:06:59.673 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T10:06:59.674 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T10:06:59.674 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T10:06:59.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 66.269018825000003, 2024-04-25T10:06:59.674 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 129, 2024-04-25T10:06:59.674 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 192, 2024-04-25T10:06:59.674 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 192, 2024-04-25T10:06:59.674 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 67.241493016999996 2024-04-25T10:06:59.674 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 -- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa90006b500 msgr2=0x7fa90006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa90006b500 0x7fa90006d9c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa90c000f60 tx=0x7fa90c007cc0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 -- 172.21.15.19:0/627684244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9140b5620 msgr2=0x7fa9140b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9140b5620 0x7fa9140b4240 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7fa908035040 tx=0x7fa9080046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 -- 172.21.15.19:0/627684244 shutdown_connections 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa90006b500 0x7fa90006d9c0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa91415b7d0 0x7fa914160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9140b5620 0x7fa9140b4240 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 --2- 172.21.15.19:0/627684244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa91406c2d0 0x7fa91414f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 -- 172.21.15.19:0/627684244 >> 172.21.15.19:0/627684244 conn(0x7fa9140d0800 msgr2=0x7fa9141e88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 -- 172.21.15.19:0/627684244 shutdown_connections 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:06:59.658+0000 7fa9193b7640 1 -- 172.21.15.19:0/627684244 wait complete. 2024-04-25T10:06:59.677 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:59.677 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:59.715 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:59.756 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:59.757 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:06:59.757 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:59.773 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:59.773 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:06:59.773 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:59.813 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:59.814 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:06:59.814 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:59.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:59.829 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:06:59.829 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:59.871 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:59.907 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:00.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.002+0000 7fafdabe0640 1 Processor -- start 2024-04-25T10:07:00.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.002+0000 7fafdabe0640 1 -- start start 2024-04-25T10:07:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.003+0000 7fafdabe0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fafd40d1020 0x7fafd40d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.003+0000 7fafdabe0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd40b97a0 con 0x7fafd415dbe0 2024-04-25T10:07:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.003+0000 7fafdabe0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd40d1c50 con 0x7fafd40b2cb0 2024-04-25T10:07:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.003+0000 7fafdabe0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd415dfc0 con 0x7fafd40d1020 2024-04-25T10:07:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.003+0000 7fafd897c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fafd415dbe0 0x7fafd40d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54348/0 (socket says 172.21.15.19:54348) 2024-04-25T10:07:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.003+0000 7fafd897c640 1 -- 172.21.15.19:0/293922595 learned_addr learned my addr 172.21.15.19:0/293922595 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd917d640 1 --2- 172.21.15.19:0/293922595 >> v2:172.21.15.112:3300/0 conn(0x7fafd40d1020 0x7fafd40d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307684849 0 0) 0x7fafd40b97a0 con 0x7fafd415dbe0 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafbc003520 con 0x7fafd415dbe0 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330341793 0 0) 0x7fafd40d1c50 con 0x7fafd40b2cb0 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafd40b97a0 con 0x7fafd40b2cb0 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2794517689 0 0) 0x7fafbc003520 con 0x7fafd415dbe0 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 >> v2:172.21.15.112:3300/0 conn(0x7fafd40d1020 msgr2=0x7fafd40d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 --2- 172.21.15.19:0/293922595 >> v2:172.21.15.112:3300/0 conn(0x7fafd40d1020 0x7fafd40d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 >> v1:172.21.15.112:6789/0 conn(0x7fafd40b2cb0 legacy=0x7fafd40d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd40d1f30 con 0x7fafd415dbe0 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fafcc0019f0 con 0x7fafd415dbe0 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 >> v1:172.21.15.19:6789/0 conn(0x7fafd415dbe0 legacy=0x7fafd40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafbc003b90 0x7fafbc003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 0x7fafbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafbc0051c0 0x7fafbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc00c090 con 0x7fafbc004800 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc00c210 con 0x7fafbc003b90 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc00c410 con 0x7fafbc0051c0 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd3fff640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 0x7fafbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd897c640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafbc003b90 0x7fafbc003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd917d640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafbc0051c0 0x7fafbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd3fff640 1 -- 172.21.15.19:0/293922595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafbc0051c0 msgr2=0x7fafbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd3fff640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafbc0051c0 0x7fafbc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd3fff640 1 -- 172.21.15.19:0/293922595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafbc003b90 msgr2=0x7fafbc003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd3fff640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafbc003b90 0x7fafbc003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd917d640 1 -- 172.21.15.19:0/293922595 reap_dead start 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd3fff640 1 -- 172.21.15.19:0/293922595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc003520 con 0x7fafbc004800 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.004+0000 7fafd897c640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafbc003b90 0x7fafbc003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.005+0000 7fafd3fff640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 0x7fafbc004c80 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7fafc80018a0 tx=0x7fafbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b39761aa6360eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.005+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc80014f0 con 0x7fafbc004800 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.005+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc80052d0 con 0x7fafbc004800 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.005+0000 7fafd37fe640 1 -- 172.21.15.19:0/293922595 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc802ac00 con 0x7fafbc004800 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.006+0000 7fafdabe0640 1 -- 172.21.15.19:0/293922595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 msgr2=0x7fafbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.006+0000 7fafdabe0640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 0x7fafbc004c80 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7fafc80018a0 tx=0x7fafbc005830 comp rx=0 tx=0).stop 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.006+0000 7fafdabe0640 1 -- 172.21.15.19:0/293922595 shutdown_connections 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.006+0000 7fafdabe0640 1 --2- 172.21.15.19:0/293922595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 0x7fafbc004c80 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.006+0000 7fafdabe0640 1 -- 172.21.15.19:0/293922595 >> 172.21.15.19:0/293922595 conn(0x7fafd4061a20 msgr2=0x7fafd4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.006+0000 7fafdabe0640 1 -- 172.21.15.19:0/293922595 shutdown_connections 2024-04-25T10:07:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.006+0000 7fafdabe0640 1 -- 172.21.15.19:0/293922595 wait complete. 2024-04-25T10:07:00.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.007+0000 7fafdabe0640 1 Processor -- start 2024-04-25T10:07:00.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.007+0000 7fafdabe0640 1 -- start start 2024-04-25T10:07:00.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafdabe0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 0x7fafd40b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafdabe0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafd41f0a30 0x7fafd40b8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafdabe0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafd41f3450 0x7fafd41f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafdabe0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd40b92a0 con 0x7fafbc004800 2024-04-25T10:07:00.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafdabe0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd40d1a00 con 0x7fafd41f3450 2024-04-25T10:07:00.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafdabe0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd40b9420 con 0x7fafd41f0a30 2024-04-25T10:07:00.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafd897c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 0x7fafd40b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafd897c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 0x7fafd40b8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45850/0 (socket says 172.21.15.19:45850) 2024-04-25T10:07:00.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafd897c640 1 -- 172.21.15.19:0/2780835342 learned_addr learned my addr 172.21.15.19:0/2780835342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:00.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafd3fff640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafd41f0a30 0x7fafd40b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafd917d640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafd41f3450 0x7fafd41f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafd897c640 1 -- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafd41f0a30 msgr2=0x7fafd40b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafd897c640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafd41f0a30 0x7fafd40b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafd897c640 1 -- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafd41f3450 msgr2=0x7fafd41f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafd897c640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafd41f3450 0x7fafd41f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafd897c640 1 -- 172.21.15.19:0/2780835342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd41f41a0 con 0x7fafbc004800 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.008+0000 7fafd917d640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafd41f3450 0x7fafd41f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.009+0000 7fafd897c640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 0x7fafd40b8200 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7fafcc001f10 tx=0x7fafcc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.009+0000 7fafd17fa640 1 -- 172.21.15.19:0/2780835342 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0019b0 con 0x7fafbc004800 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.009+0000 7fafd17fa640 1 -- 172.21.15.19:0/2780835342 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc018070 con 0x7fafbc004800 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.009+0000 7fafdabe0640 1 -- 172.21.15.19:0/2780835342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd4144bc0 con 0x7fafbc004800 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.009+0000 7fafd17fa640 1 -- 172.21.15.19:0/2780835342 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc013420 con 0x7fafbc004800 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.009+0000 7fafdabe0640 1 -- 172.21.15.19:0/2780835342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd4145180 con 0x7fafbc004800 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.010+0000 7fafdabe0640 1 -- 172.21.15.19:0/2780835342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd4073990 con 0x7fafbc004800 2024-04-25T10:07:00.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.010+0000 7fafd17fa640 1 -- 172.21.15.19:0/2780835342 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafcc0135c0 con 0x7fafbc004800 2024-04-25T10:07:00.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.011+0000 7fafd17fa640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc006b500 0x7fafc006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.011+0000 7fafd17fa640 1 -- 172.21.15.19:0/2780835342 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fafcc08a880 con 0x7fafbc004800 2024-04-25T10:07:00.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.011+0000 7fafd3fff640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc006b500 0x7fafc006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.011+0000 7fafd3fff640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc006b500 0x7fafc006d9c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fafc8032000 tx=0x7fafc8009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.015+0000 7fafd17fa640 1 -- 172.21.15.19:0/2780835342 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fafcc059430 con 0x7fafbc004800 2024-04-25T10:07:00.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.231+0000 7fafdabe0640 1 -- 172.21.15.19:0/2780835342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafd4062140 con 0x7fafbc004800 2024-04-25T10:07:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.231+0000 7fafd17fa640 1 -- 172.21.15.19:0/2780835342 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+4956 (secure 0 0 0) 0x7fafcc058d30 con 0x7fafbc004800 2024-04-25T10:07:00.237 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:00.237 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":129,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:38.424174+0000","modified":"2024-04-25T10:05:55.398362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":192,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11799":{"gid":11799,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/1758455116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1758455116},{"type":"v1","addr":"172.21.15.19:6835","nonce":1758455116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:07:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 129 2024-04-25T10:07:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 -- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc006b500 msgr2=0x7fafc006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc006b500 0x7fafc006d9c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fafc8032000 tx=0x7fafc8009030 comp rx=0 tx=0).stop 2024-04-25T10:07:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 -- 172.21.15.19:0/2780835342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 msgr2=0x7fafd40b8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 0x7fafd40b8200 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7fafcc001f10 tx=0x7fafcc00e530 comp rx=0 tx=0).stop 2024-04-25T10:07:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 -- 172.21.15.19:0/2780835342 shutdown_connections 2024-04-25T10:07:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fafc006b500 0x7fafc006d9c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafd41f3450 0x7fafd41f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafd41f0a30 0x7fafd40b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 --2- 172.21.15.19:0/2780835342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafbc004800 0x7fafd40b8200 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 -- 172.21.15.19:0/2780835342 >> 172.21.15.19:0/2780835342 conn(0x7fafd4061a20 msgr2=0x7fafd40b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 -- 172.21.15.19:0/2780835342 shutdown_connections 2024-04-25T10:07:00.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.233+0000 7fafdabe0640 1 -- 172.21.15.19:0/2780835342 wait complete. 2024-04-25T10:07:00.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:00.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.346+0000 7f829abb0640 1 Processor -- start 2024-04-25T10:07:00.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.347+0000 7f829abb0640 1 -- start start 2024-04-25T10:07:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.347+0000 7f829abb0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f829415ba00 0x7f829409a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.347+0000 7f829abb0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f82941602e0 0x7f82940ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.347+0000 7f829abb0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82940cd710 con 0x7f829415f5f0 2024-04-25T10:07:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.347+0000 7f829abb0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8294099b70 con 0x7f82941602e0 2024-04-25T10:07:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.347+0000 7f829abb0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8294099e50 con 0x7f829415ba00 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.348+0000 7f8293fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f82941602e0 0x7f82940ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.348+0000 7f829914d640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f829415f5f0 0x7f82940c52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54358/0 (socket says 172.21.15.19:54358) 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.348+0000 7f829914d640 1 -- 172.21.15.19:0/58541846 learned_addr learned my addr 172.21.15.19:0/58541846 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.348+0000 7f829894c640 1 --2- 172.21.15.19:0/58541846 >> v2:172.21.15.112:3300/0 conn(0x7f829415ba00 0x7f829409a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.348+0000 7f8293fff640 1 -- 172.21.15.19:0/58541846 >> v2:172.21.15.112:3300/0 conn(0x7f829415ba00 msgr2=0x7f829409a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.348+0000 7f8293fff640 1 --2- 172.21.15.19:0/58541846 >> v2:172.21.15.112:3300/0 conn(0x7f829415ba00 0x7f829409a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.348+0000 7f8293fff640 1 -- 172.21.15.19:0/58541846 >> v1:172.21.15.19:6789/0 conn(0x7f829415f5f0 legacy=0x7f82940c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.348+0000 7f8293fff640 1 -- 172.21.15.19:0/58541846 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829409a130 con 0x7f82941602e0 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.348+0000 7f829894c640 1 --2- 172.21.15.19:0/58541846 >> v2:172.21.15.112:3300/0 conn(0x7f829415ba00 0x7f829409a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f8293fff640 1 --2- 172.21.15.19:0/58541846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82941602e0 0x7f82940ccda0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f82940b4370 tx=0x7f828002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ce26528935a338 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f82937fe640 1 -- 172.21.15.19:0/58541846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280037070 con 0x7f82941602e0 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f82937fe640 1 -- 172.21.15.19:0/58541846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828002add0 con 0x7f82941602e0 2024-04-25T10:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f82937fe640 1 -- 172.21.15.19:0/58541846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280032660 con 0x7f82941602e0 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f829abb0640 1 -- 172.21.15.19:0/58541846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82941602e0 msgr2=0x7f82940ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f829abb0640 1 --2- 172.21.15.19:0/58541846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82941602e0 0x7f82940ccda0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f82940b4370 tx=0x7f828002a080 comp rx=0 tx=0).stop 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f829abb0640 1 -- 172.21.15.19:0/58541846 shutdown_connections 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f829abb0640 1 --2- 172.21.15.19:0/58541846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82941602e0 0x7f82940ccda0 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f829abb0640 1 --2- 172.21.15.19:0/58541846 >> v2:172.21.15.112:3300/0 conn(0x7f829415ba00 0x7f829409a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f829abb0640 1 -- 172.21.15.19:0/58541846 >> 172.21.15.19:0/58541846 conn(0x7f82940b3270 msgr2=0x7f82940b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f829abb0640 1 -- 172.21.15.19:0/58541846 shutdown_connections 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.349+0000 7f829abb0640 1 -- 172.21.15.19:0/58541846 wait complete. 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.351+0000 7f829abb0640 1 Processor -- start 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.351+0000 7f829abb0640 1 -- start start 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.351+0000 7f829abb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829415ba00 0x7f82940cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.351+0000 7f829abb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f829415f5f0 0x7f82940b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.351+0000 7f829abb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82941602e0 0x7f82940b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.351+0000 7f829abb0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829415bde0 con 0x7f829415ba00 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.351+0000 7f829abb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82940cd2e0 con 0x7f829415f5f0 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.351+0000 7f829abb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829415dbf0 con 0x7f82941602e0 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829894c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829415ba00 0x7f82940cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829894c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829415ba00 0x7f82940cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45874/0 (socket says 172.21.15.19:45874) 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829894c640 1 -- 172.21.15.19:0/3550041071 learned_addr learned my addr 172.21.15.19:0/3550041071 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829914d640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82941602e0 0x7f82940b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f8293fff640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f829415f5f0 0x7f82940b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829894c640 1 -- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82941602e0 msgr2=0x7f82940b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829894c640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82941602e0 0x7f82940b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829894c640 1 -- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f829415f5f0 msgr2=0x7f82940b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829894c640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f829415f5f0 0x7f82940b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829894c640 1 -- 172.21.15.19:0/3550041071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8294154f70 con 0x7f829415ba00 2024-04-25T10:07:00.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829894c640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829415ba00 0x7f82940cb0a0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f828800d7d0 tx=0x7f828800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f82917fa640 1 -- 172.21.15.19:0/3550041071 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288004030 con 0x7f829415ba00 2024-04-25T10:07:00.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f82917fa640 1 -- 172.21.15.19:0/3550041071 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82880041d0 con 0x7f829415ba00 2024-04-25T10:07:00.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829abb0640 1 -- 172.21.15.19:0/3550041071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829415de90 con 0x7f829415ba00 2024-04-25T10:07:00.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f82917fa640 1 -- 172.21.15.19:0/3550041071 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82880096a0 con 0x7f829415ba00 2024-04-25T10:07:00.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.352+0000 7f829abb0640 1 -- 172.21.15.19:0/3550041071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829415e450 con 0x7f829415ba00 2024-04-25T10:07:00.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.353+0000 7f829abb0640 1 -- 172.21.15.19:0/3550041071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8294073990 con 0x7f829415ba00 2024-04-25T10:07:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.354+0000 7f82917fa640 1 -- 172.21.15.19:0/3550041071 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8288009840 con 0x7f829415ba00 2024-04-25T10:07:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.354+0000 7f82917fa640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f827806b4d0 0x7f827806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.354+0000 7f82917fa640 1 -- 172.21.15.19:0/3550041071 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f828808a6e0 con 0x7f829415ba00 2024-04-25T10:07:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.354+0000 7f8293fff640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f827806b4d0 0x7f827806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.355+0000 7f8293fff640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f827806b4d0 0x7f827806d990 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8280034000 tx=0x7f828002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.358+0000 7f82917fa640 1 -- 172.21.15.19:0/3550041071 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8288059310 con 0x7f829415ba00 2024-04-25T10:07:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.540+0000 7f829abb0640 1 -- 172.21.15.19:0/3550041071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82940b40a0 con 0x7f829415ba00 2024-04-25T10:07:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.541+0000 7f82917fa640 1 -- 172.21.15.19:0/3550041071 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+17195 (secure 0 0 0) 0x7f8288058c10 con 0x7f829415ba00 2024-04-25T10:07:00.546 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:00.547 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":192,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:52.378385+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:36.700326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:37.402558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":6,"to":0}]},{"pgid":"29.b","mappings":[{"from":6,"to":0}]},{"pgid":"29.13","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/392508344":"2024-04-26T10:05:52.378361+0000","172.21.15.19:6833/392508344":"2024-04-26T10:05:52.378361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 -- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f827806b4d0 msgr2=0x7f827806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f827806b4d0 0x7f827806d990 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8280034000 tx=0x7f828002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 -- 172.21.15.19:0/3550041071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829415ba00 msgr2=0x7f82940cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829415ba00 0x7f82940cb0a0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f828800d7d0 tx=0x7f828800dc90 comp rx=0 tx=0).stop 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 -- 172.21.15.19:0/3550041071 shutdown_connections 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f827806b4d0 0x7f827806d990 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82941602e0 0x7f82940b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f829415f5f0 0x7f82940b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 --2- 172.21.15.19:0/3550041071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829415ba00 0x7f82940cb0a0 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 -- 172.21.15.19:0/3550041071 >> 172.21.15.19:0/3550041071 conn(0x7f82940b3270 msgr2=0x7f82940c5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 -- 172.21.15.19:0/3550041071 shutdown_connections 2024-04-25T10:07:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.543+0000 7f829abb0640 1 -- 172.21.15.19:0/3550041071 wait complete. 2024-04-25T10:07:00.559 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:52:54.954281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:36.700326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:37.402558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:00.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:00.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.657+0000 7f66b2753640 1 Processor -- start 2024-04-25T10:07:00.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.657+0000 7f66b2753640 1 -- start start 2024-04-25T10:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.658+0000 7f66b2753640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66ac15da90 0x7f66ac161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.658+0000 7f66b2753640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66ac161ba0 0x7f66ac0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.658+0000 7f66b2753640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66ac162040 con 0x7f66ac0b9750 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.658+0000 7f66b2753640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ac162600 con 0x7f66ac161ba0 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.658+0000 7f66b2753640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac162320 con 0x7f66ac15da90 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.658+0000 7f66b0cf0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66ac161ba0 0x7f66ac0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.658+0000 7f66abfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66ac15da90 0x7f66ac161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.658+0000 7f66b0cf0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66ac161ba0 0x7f66ac0d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:49752/0 (socket says 172.21.15.19:49752) 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.658+0000 7f66b0cf0640 1 -- 172.21.15.19:0/1625460625 learned_addr learned my addr 172.21.15.19:0/1625460625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.659+0000 7f66aaffd640 1 -- 172.21.15.19:0/1625460625 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630657946 0 0) 0x7f66ac162040 con 0x7f66ac0b9750 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.659+0000 7f66aaffd640 1 -- 172.21.15.19:0/1625460625 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6698003520 con 0x7f66ac0b9750 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.659+0000 7f66b0cf0640 1 -- 172.21.15.19:0/1625460625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66ac15da90 msgr2=0x7f66ac161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.659+0000 7f66b0cf0640 1 --2- 172.21.15.19:0/1625460625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66ac15da90 0x7f66ac161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.659+0000 7f66b0cf0640 1 -- 172.21.15.19:0/1625460625 >> v1:172.21.15.112:6789/0 conn(0x7f66ac0b9750 legacy=0x7f66ac0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.659+0000 7f66b0cf0640 1 -- 172.21.15.19:0/1625460625 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac0d1b10 con 0x7f66ac161ba0 2024-04-25T10:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.659+0000 7f66abfff640 1 --2- 172.21.15.19:0/1625460625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66ac15da90 0x7f66ac161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:00.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66b0cf0640 1 --2- 172.21.15.19:0/1625460625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ac161ba0 0x7f66ac0d2310 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009e50 tx=0x7f66a402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52cc398f707e6d82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66aaffd640 1 -- 172.21.15.19:0/1625460625 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4036070 con 0x7f66ac161ba0 2024-04-25T10:07:00.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66aaffd640 1 -- 172.21.15.19:0/1625460625 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a4031600 con 0x7f66ac161ba0 2024-04-25T10:07:00.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66aaffd640 1 -- 172.21.15.19:0/1625460625 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a40319b0 con 0x7f66ac161ba0 2024-04-25T10:07:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66b2753640 1 -- 172.21.15.19:0/1625460625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ac161ba0 msgr2=0x7f66ac0d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66b2753640 1 --2- 172.21.15.19:0/1625460625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ac161ba0 0x7f66ac0d2310 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009e50 tx=0x7f66a402a680 comp rx=0 tx=0).stop 2024-04-25T10:07:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66b2753640 1 -- 172.21.15.19:0/1625460625 shutdown_connections 2024-04-25T10:07:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66b2753640 1 --2- 172.21.15.19:0/1625460625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ac161ba0 0x7f66ac0d2310 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66b2753640 1 --2- 172.21.15.19:0/1625460625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66ac15da90 0x7f66ac161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66b2753640 1 -- 172.21.15.19:0/1625460625 >> 172.21.15.19:0/1625460625 conn(0x7f66ac061a20 msgr2=0x7f66ac061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66b2753640 1 -- 172.21.15.19:0/1625460625 shutdown_connections 2024-04-25T10:07:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.660+0000 7f66b2753640 1 -- 172.21.15.19:0/1625460625 wait complete. 2024-04-25T10:07:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.662+0000 7f66b2753640 1 Processor -- start 2024-04-25T10:07:00.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.662+0000 7f66b2753640 1 -- start start 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.662+0000 7f66b2753640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ac15da90 0x7f66ac0cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.662+0000 7f66b2753640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66ac161ba0 0x7f66ac159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.662+0000 7f66b2753640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac15a300 0x7f66ac0c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.662+0000 7f66b2753640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0ced80 con 0x7f66ac15a300 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.662+0000 7f66b2753640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac15de70 con 0x7f66ac15da90 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.662+0000 7f66b2753640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0cef00 con 0x7f66ac161ba0 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66b0cf0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac15a300 0x7f66ac0c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66b0cf0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac15a300 0x7f66ac0c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45876/0 (socket says 172.21.15.19:45876) 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66b0cf0640 1 -- 172.21.15.19:0/1005059250 learned_addr learned my addr 172.21.15.19:0/1005059250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66ab7fe640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66ac161ba0 0x7f66ac159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66abfff640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ac15da90 0x7f66ac0cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66b0cf0640 1 -- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66ac161ba0 msgr2=0x7f66ac159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66b0cf0640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66ac161ba0 0x7f66ac159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66b0cf0640 1 -- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ac15da90 msgr2=0x7f66ac0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66b0cf0640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ac15da90 0x7f66ac0cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66b0cf0640 1 -- 172.21.15.19:0/1005059250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac15a790 con 0x7f66ac15a300 2024-04-25T10:07:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66b0cf0640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac15a300 0x7f66ac0c8720 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f66a4034040 tx=0x7f66a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66a8ff9640 1 -- 172.21.15.19:0/1005059250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4041020 con 0x7f66ac15a300 2024-04-25T10:07:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66a8ff9640 1 -- 172.21.15.19:0/1005059250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a4007820 con 0x7f66ac15a300 2024-04-25T10:07:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66b2753640 1 -- 172.21.15.19:0/1005059250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ac0c9450 con 0x7f66ac15a300 2024-04-25T10:07:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66a8ff9640 1 -- 172.21.15.19:0/1005059250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4036040 con 0x7f66ac15a300 2024-04-25T10:07:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.663+0000 7f66b2753640 1 -- 172.21.15.19:0/1005059250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ac0c9a10 con 0x7f66ac15a300 2024-04-25T10:07:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.664+0000 7f66b2753640 1 -- 172.21.15.19:0/1005059250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66ac0ca0c0 con 0x7f66ac15a300 2024-04-25T10:07:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.665+0000 7f66a8ff9640 1 -- 172.21.15.19:0/1005059250 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66a4044020 con 0x7f66ac15a300 2024-04-25T10:07:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.665+0000 7f66a8ff9640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f668c06b4d0 0x7f668c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.666+0000 7f66a8ff9640 1 -- 172.21.15.19:0/1005059250 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f66a40aaee0 con 0x7f66ac15a300 2024-04-25T10:07:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.666+0000 7f66abfff640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f668c06b4d0 0x7f668c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.666+0000 7f66abfff640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f668c06b4d0 0x7f668c06d990 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6694009560 tx=0x7f6694009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.670+0000 7f66a8ff9640 1 -- 172.21.15.19:0/1005059250 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66a404b030 con 0x7f66ac15a300 2024-04-25T10:07:00.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.849+0000 7f66b2753640 1 -- 172.21.15.19:0/1005059250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66ac0cfd90 con 0x7f66ac15a300 2024-04-25T10:07:00.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.849+0000 7f66a8ff9640 1 -- 172.21.15.19:0/1005059250 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v192) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66a4079560 con 0x7f66ac15a300 2024-04-25T10:07:00.855 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T10:07:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 -- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f668c06b4d0 msgr2=0x7f668c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f668c06b4d0 0x7f668c06d990 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6694009560 tx=0x7f6694009190 comp rx=0 tx=0).stop 2024-04-25T10:07:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 -- 172.21.15.19:0/1005059250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac15a300 msgr2=0x7f66ac0c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac15a300 0x7f66ac0c8720 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f66a4034040 tx=0x7f66a4004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 -- 172.21.15.19:0/1005059250 shutdown_connections 2024-04-25T10:07:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f668c06b4d0 0x7f668c06d990 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac15a300 0x7f66ac0c8720 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66ac161ba0 0x7f66ac159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 --2- 172.21.15.19:0/1005059250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ac15da90 0x7f66ac0cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 -- 172.21.15.19:0/1005059250 >> 172.21.15.19:0/1005059250 conn(0x7f66ac061a20 msgr2=0x7f66ac0d3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 -- 172.21.15.19:0/1005059250 shutdown_connections 2024-04-25T10:07:00.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.851+0000 7f66b2753640 1 -- 172.21.15.19:0/1005059250 wait complete. 2024-04-25T10:07:00.867 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:00.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.965+0000 7f8cb4189640 1 Processor -- start 2024-04-25T10:07:00.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.965+0000 7f8cb4189640 1 -- start start 2024-04-25T10:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.966+0000 7f8cb4189640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cac15b7d0 0x7f8cac0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.966+0000 7f8cb4189640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8cac0b5620 0x7f8cac0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.966+0000 7f8cb4189640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac06cf50 con 0x7f8cac06c2d0 2024-04-25T10:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.966+0000 7f8cb4189640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0ce530 con 0x7f8cac15b7d0 2024-04-25T10:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.966+0000 7f8cb4189640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0ce810 con 0x7f8cac0b5620 2024-04-25T10:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.966+0000 7f8cb1f25640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cac15b7d0 0x7f8cac0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.966+0000 7f8cb1f25640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cac15b7d0 0x7f8cac0cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45888/0 (socket says 172.21.15.19:45888) 2024-04-25T10:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.966+0000 7f8cb1f25640 1 -- 172.21.15.19:0/2304467044 learned_addr learned my addr 172.21.15.19:0/2304467044 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.967+0000 7f8cb1724640 1 --2- 172.21.15.19:0/2304467044 >> v2:172.21.15.112:3300/0 conn(0x7f8cac0b5620 0x7f8cac0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.967+0000 7f8cb1f25640 1 -- 172.21.15.19:0/2304467044 >> v2:172.21.15.112:3300/0 conn(0x7f8cac0b5620 msgr2=0x7f8cac0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.967+0000 7f8cb1f25640 1 --2- 172.21.15.19:0/2304467044 >> v2:172.21.15.112:3300/0 conn(0x7f8cac0b5620 0x7f8cac0b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.967+0000 7f8cb1f25640 1 -- 172.21.15.19:0/2304467044 >> v1:172.21.15.112:6789/0 conn(0x7f8cac06c2d0 legacy=0x7f8cac06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.967+0000 7f8cb1f25640 1 -- 172.21.15.19:0/2304467044 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0ceaf0 con 0x7f8cac15b7d0 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.967+0000 7f8cb1724640 1 --2- 172.21.15.19:0/2304467044 >> v2:172.21.15.112:3300/0 conn(0x7f8cac0b5620 0x7f8cac0b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.967+0000 7f8cb1f25640 1 --2- 172.21.15.19:0/2304467044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cac15b7d0 0x7f8cac0cf010 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0ce180 tx=0x7f8c9c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f85b13d06581644 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.967+0000 7f8cb0f23640 1 -- 172.21.15.19:0/2304467044 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c036070 con 0x7f8cac15b7d0 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.968+0000 7f8cb0f23640 1 -- 172.21.15.19:0/2304467044 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c031600 con 0x7f8cac15b7d0 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.968+0000 7f8cb0f23640 1 -- 172.21.15.19:0/2304467044 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0319d0 con 0x7f8cac15b7d0 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.968+0000 7f8cb4189640 1 -- 172.21.15.19:0/2304467044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cac15b7d0 msgr2=0x7f8cac0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.968+0000 7f8cb4189640 1 --2- 172.21.15.19:0/2304467044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cac15b7d0 0x7f8cac0cf010 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0ce180 tx=0x7f8c9c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.968+0000 7f8cb4189640 1 -- 172.21.15.19:0/2304467044 shutdown_connections 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.968+0000 7f8cb4189640 1 --2- 172.21.15.19:0/2304467044 >> v2:172.21.15.112:3300/0 conn(0x7f8cac0b5620 0x7f8cac0b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.968+0000 7f8cb4189640 1 --2- 172.21.15.19:0/2304467044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cac15b7d0 0x7f8cac0cf010 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.968+0000 7f8cb4189640 1 -- 172.21.15.19:0/2304467044 >> 172.21.15.19:0/2304467044 conn(0x7f8cac0d0800 msgr2=0x7f8cac0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.968+0000 7f8cb4189640 1 -- 172.21.15.19:0/2304467044 shutdown_connections 2024-04-25T10:07:00.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.968+0000 7f8cb4189640 1 -- 172.21.15.19:0/2304467044 wait complete. 2024-04-25T10:07:00.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.969+0000 7f8cb4189640 1 Processor -- start 2024-04-25T10:07:00.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.969+0000 7f8cb4189640 1 -- start start 2024-04-25T10:07:00.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb4189640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac06c2d0 0x7f8cac14b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb4189640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cac0b5620 0x7f8cac149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb4189640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cac15b7d0 0x7f8cac0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb4189640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0cf6d0 con 0x7f8cac0b5620 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb4189640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0cf550 con 0x7f8cac06c2d0 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb4189640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac15bbb0 con 0x7f8cac15b7d0 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb1724640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cac0b5620 0x7f8cac149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb1724640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cac0b5620 0x7f8cac149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45894/0 (socket says 172.21.15.19:45894) 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb1724640 1 -- 172.21.15.19:0/2749909699 learned_addr learned my addr 172.21.15.19:0/2749909699 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb1f25640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac06c2d0 0x7f8cac14b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb2726640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cac15b7d0 0x7f8cac0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.970+0000 7f8cb1724640 1 -- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cac15b7d0 msgr2=0x7f8cac0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.971+0000 7f8cb1724640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cac15b7d0 0x7f8cac0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.971+0000 7f8cb1724640 1 -- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac06c2d0 msgr2=0x7f8cac14b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.971+0000 7f8cb1724640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac06c2d0 0x7f8cac14b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.971+0000 7f8cb1724640 1 -- 172.21.15.19:0/2749909699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac14ba90 con 0x7f8cac0b5620 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.971+0000 7f8cb2726640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cac15b7d0 0x7f8cac0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.971+0000 7f8cb1724640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cac0b5620 0x7f8cac149040 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000c7f0 tx=0x7f8ca000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.971+0000 7f8caa7fc640 1 -- 172.21.15.19:0/2749909699 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0015070 con 0x7f8cac0b5620 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.971+0000 7f8caa7fc640 1 -- 172.21.15.19:0/2749909699 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca0007a30 con 0x7f8cac0b5620 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.971+0000 7f8cb4189640 1 -- 172.21.15.19:0/2749909699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac14bd70 con 0x7f8cac0b5620 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.971+0000 7f8caa7fc640 1 -- 172.21.15.19:0/2749909699 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0007e00 con 0x7f8cac0b5620 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.971+0000 7f8cb4189640 1 -- 172.21.15.19:0/2749909699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac0cd050 con 0x7f8cac0b5620 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.972+0000 7f8cb4189640 1 -- 172.21.15.19:0/2749909699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac0cd220 con 0x7f8cac0b5620 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.972+0000 7f8caa7fc640 1 -- 172.21.15.19:0/2749909699 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ca0018070 con 0x7f8cac0b5620 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.973+0000 7f8caa7fc640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c9806b4d0 0x7f8c9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.973+0000 7f8caa7fc640 1 -- 172.21.15.19:0/2749909699 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f8ca008a3f0 con 0x7f8cac0b5620 2024-04-25T10:07:00.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.973+0000 7f8cb1f25640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c9806b4d0 0x7f8c9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.973+0000 7f8cb1f25640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c9806b4d0 0x7f8c9806d990 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c033000 tx=0x7f8c9c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:00.977+0000 7f8caa7fc640 1 -- 172.21.15.19:0/2749909699 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ca0058930 con 0x7f8cac0b5620 2024-04-25T10:07:01.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.158+0000 7f8cb4189640 1 -- 172.21.15.19:0/2749909699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8cac0ccb40 con 0x7f8cac0b5620 2024-04-25T10:07:01.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.158+0000 7f8caa7fc640 1 -- 172.21.15.19:0/2749909699 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v192) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8ca005c9d0 con 0x7f8cac0b5620 2024-04-25T10:07:01.164 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T10:07:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 -- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c9806b4d0 msgr2=0x7f8c9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c9806b4d0 0x7f8c9806d990 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c033000 tx=0x7f8c9c009000 comp rx=0 tx=0).stop 2024-04-25T10:07:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 -- 172.21.15.19:0/2749909699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cac0b5620 msgr2=0x7f8cac149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cac0b5620 0x7f8cac149040 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000c7f0 tx=0x7f8ca000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 -- 172.21.15.19:0/2749909699 shutdown_connections 2024-04-25T10:07:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f8c9806b4d0 0x7f8c9806d990 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8cac15b7d0 0x7f8cac0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cac0b5620 0x7f8cac149040 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 --2- 172.21.15.19:0/2749909699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac06c2d0 0x7f8cac14b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 -- 172.21.15.19:0/2749909699 >> 172.21.15.19:0/2749909699 conn(0x7f8cac0d0800 msgr2=0x7f8cac0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 -- 172.21.15.19:0/2749909699 shutdown_connections 2024-04-25T10:07:01.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.160+0000 7f8cb4189640 1 -- 172.21.15.19:0/2749909699 wait complete. 2024-04-25T10:07:01.176 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.274+0000 7f5aeb06b640 1 Processor -- start 2024-04-25T10:07:01.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.274+0000 7f5aeb06b640 1 -- start start 2024-04-25T10:07:01.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.275+0000 7f5aeb06b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ae415a9f0 0x7f5ae40c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.275+0000 7f5aeb06b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ae40cdbf0 0x7f5ae40ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.275+0000 7f5aeb06b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ae40cd520 0x7f5ae40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.275+0000 7f5aeb06b640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae40b0020 con 0x7f5ae40cd520 2024-04-25T10:07:01.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.275+0000 7f5aeb06b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae40cb5a0 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.275+0000 7f5aeb06b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae40c6940 con 0x7f5ae415a9f0 2024-04-25T10:07:01.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.275+0000 7f5ae9608640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ae40cd520 0x7f5ae40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.275+0000 7f5ae9608640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ae40cd520 0x7f5ae40ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45910/0 (socket says 172.21.15.19:45910) 2024-04-25T10:07:01.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.275+0000 7f5ae9608640 1 -- 172.21.15.19:0/471181443 learned_addr learned my addr 172.21.15.19:0/471181443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:01.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.276+0000 7f5ae3fff640 1 --2- 172.21.15.19:0/471181443 >> v2:172.21.15.112:3300/0 conn(0x7f5ae40cdbf0 0x7f5ae40ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.276+0000 7f5ae8e07640 1 --2- 172.21.15.19:0/471181443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ae415a9f0 0x7f5ae40c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.276+0000 7f5ae9608640 1 -- 172.21.15.19:0/471181443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ae415a9f0 msgr2=0x7f5ae40c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.276+0000 7f5ae9608640 1 --2- 172.21.15.19:0/471181443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ae415a9f0 0x7f5ae40c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.276+0000 7f5ae9608640 1 -- 172.21.15.19:0/471181443 >> v2:172.21.15.112:3300/0 conn(0x7f5ae40cdbf0 msgr2=0x7f5ae40ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.276+0000 7f5ae9608640 1 --2- 172.21.15.19:0/471181443 >> v2:172.21.15.112:3300/0 conn(0x7f5ae40cdbf0 0x7f5ae40ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.276+0000 7f5ae9608640 1 -- 172.21.15.19:0/471181443 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae415add0 con 0x7f5ae40cd520 2024-04-25T10:07:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.276+0000 7f5ae9608640 1 --2- 172.21.15.19:0/471181443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae40cd520 0x7f5ae40ce8f0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40d1830 tx=0x7f5adc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bbdff0c1ea89c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.276+0000 7f5ae37fe640 1 -- 172.21.15.19:0/471181443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc037070 con 0x7f5ae40cd520 2024-04-25T10:07:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.276+0000 7f5ae37fe640 1 -- 172.21.15.19:0/471181443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc032440 con 0x7f5ae40cd520 2024-04-25T10:07:01.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.277+0000 7f5ae37fe640 1 -- 172.21.15.19:0/471181443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc032810 con 0x7f5ae40cd520 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.277+0000 7f5aeb06b640 1 -- 172.21.15.19:0/471181443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae40cd520 msgr2=0x7f5ae40ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.277+0000 7f5aeb06b640 1 --2- 172.21.15.19:0/471181443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae40cd520 0x7f5ae40ce8f0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40d1830 tx=0x7f5adc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.277+0000 7f5aeb06b640 1 -- 172.21.15.19:0/471181443 shutdown_connections 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.277+0000 7f5aeb06b640 1 --2- 172.21.15.19:0/471181443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae40cd520 0x7f5ae40ce8f0 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.277+0000 7f5aeb06b640 1 --2- 172.21.15.19:0/471181443 >> v2:172.21.15.112:3300/0 conn(0x7f5ae40cdbf0 0x7f5ae40ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.277+0000 7f5aeb06b640 1 --2- 172.21.15.19:0/471181443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ae415a9f0 0x7f5ae40c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.277+0000 7f5aeb06b640 1 -- 172.21.15.19:0/471181443 >> 172.21.15.19:0/471181443 conn(0x7f5ae40b2390 msgr2=0x7f5ae40b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.277+0000 7f5aeb06b640 1 -- 172.21.15.19:0/471181443 shutdown_connections 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.277+0000 7f5aeb06b640 1 -- 172.21.15.19:0/471181443 wait complete. 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.278+0000 7f5aeb06b640 1 Processor -- start 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.278+0000 7f5aeb06b640 1 -- start start 2024-04-25T10:07:01.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.279+0000 7f5aeb06b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ae40cd520 0x7f5ae414cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.279+0000 7f5aeb06b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae40cdbf0 0x7f5ae40ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.279+0000 7f5aeb06b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ae415a9f0 0x7f5ae40aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.279+0000 7f5aeb06b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae40cb230 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.279+0000 7f5aeb06b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae40caf50 con 0x7f5ae415a9f0 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.279+0000 7f5aeb06b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae40ce070 con 0x7f5ae40cd520 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.279+0000 7f5ae3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae40cdbf0 0x7f5ae40ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.279+0000 7f5ae3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae40cdbf0 0x7f5ae40ae560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45914/0 (socket says 172.21.15.19:45914) 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.279+0000 7f5ae3fff640 1 -- 172.21.15.19:0/1061463508 learned_addr learned my addr 172.21.15.19:0/1061463508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5ae8e07640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ae40cd520 0x7f5ae414cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5ae9608640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ae415a9f0 0x7f5ae40aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5ae3fff640 1 -- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ae40cd520 msgr2=0x7f5ae414cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5ae3fff640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ae40cd520 0x7f5ae414cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5ae3fff640 1 -- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ae415a9f0 msgr2=0x7f5ae40aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5ae3fff640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ae415a9f0 0x7f5ae40aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5ae3fff640 1 -- 172.21.15.19:0/1061463508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae414d5e0 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5ae3fff640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae40cdbf0 0x7f5ae40ae560 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4007e90 tx=0x7f5ad400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5ae17fa640 1 -- 172.21.15.19:0/1061463508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4014620 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5ae17fa640 1 -- 172.21.15.19:0/1061463508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad4019070 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5aeb06b640 1 -- 172.21.15.19:0/1061463508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae414d8c0 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5ae17fa640 1 -- 172.21.15.19:0/1061463508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4014a00 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.280+0000 7f5aeb06b640 1 -- 172.21.15.19:0/1061463508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae40af7a0 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.281+0000 7f5aeb06b640 1 -- 172.21.15.19:0/1061463508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae4073a00 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.282+0000 7f5ae17fa640 1 -- 172.21.15.19:0/1061463508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ad4014ba0 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.282+0000 7f5ae17fa640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac406b4d0 0x7f5ac406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.282+0000 7f5ae17fa640 1 -- 172.21.15.19:0/1061463508 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5ad408ac00 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.282+0000 7f5ae8e07640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac406b4d0 0x7f5ac406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.283+0000 7f5ae8e07640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac406b4d0 0x7f5ac406d990 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5acc006fd0 tx=0x7f5acc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.286+0000 7f5ae17fa640 1 -- 172.21.15.19:0/1061463508 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ad4091080 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.467+0000 7f5aeb06b640 1 -- 172.21.15.19:0/1061463508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ae406b3f0 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.467+0000 7f5ae17fa640 1 -- 172.21.15.19:0/1061463508 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v192) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ad40598c0 con 0x7f5ae40cdbf0 2024-04-25T10:07:01.473 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T10:07:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 -- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac406b4d0 msgr2=0x7f5ac406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac406b4d0 0x7f5ac406d990 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5acc006fd0 tx=0x7f5acc008040 comp rx=0 tx=0).stop 2024-04-25T10:07:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 -- 172.21.15.19:0/1061463508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae40cdbf0 msgr2=0x7f5ae40ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae40cdbf0 0x7f5ae40ae560 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4007e90 tx=0x7f5ad400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 -- 172.21.15.19:0/1061463508 shutdown_connections 2024-04-25T10:07:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5ac406b4d0 0x7f5ac406d990 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ae415a9f0 0x7f5ae40aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae40cdbf0 0x7f5ae40ae560 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 --2- 172.21.15.19:0/1061463508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ae40cd520 0x7f5ae414cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 -- 172.21.15.19:0/1061463508 >> 172.21.15.19:0/1061463508 conn(0x7f5ae40b2390 msgr2=0x7f5ae406e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 -- 172.21.15.19:0/1061463508 shutdown_connections 2024-04-25T10:07:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.469+0000 7f5aeb06b640 1 -- 172.21.15.19:0/1061463508 wait complete. 2024-04-25T10:07:01.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.581+0000 7fe5c8d90640 1 Processor -- start 2024-04-25T10:07:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.581+0000 7fe5c8d90640 1 -- start start 2024-04-25T10:07:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.582+0000 7fe5c8d90640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe5c40ce330 0x7fe5c406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.582+0000 7fe5c8d90640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c41626e0 con 0x7fe5c406abe0 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.582+0000 7fe5c8d90640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c40cd270 con 0x7fe5c40700a0 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.582+0000 7fe5c8d90640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c40cd550 con 0x7fe5c40ce330 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.582+0000 7fe5c259c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe5c406abe0 0x7fe5c406afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54360/0 (socket says 172.21.15.19:54360) 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.582+0000 7fe5c259c640 1 -- 172.21.15.19:0/3803385750 learned_addr learned my addr 172.21.15.19:0/3803385750 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131695469 0 0) 0x7fe5c41626e0 con 0x7fe5c406abe0 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5bbfff640 1 --2- 172.21.15.19:0/3803385750 >> v2:172.21.15.112:3300/0 conn(0x7fe5c40ce330 0x7fe5c406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5b0003520 con 0x7fe5c406abe0 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2591665270 0 0) 0x7fe5b0003520 con 0x7fe5c406abe0 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 >> v2:172.21.15.112:3300/0 conn(0x7fe5c40ce330 msgr2=0x7fe5c406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 --2- 172.21.15.19:0/3803385750 >> v2:172.21.15.112:3300/0 conn(0x7fe5c40ce330 0x7fe5c406fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 >> v1:172.21.15.112:6789/0 conn(0x7fe5c40700a0 legacy=0x7fe5c415d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c40cd830 con 0x7fe5c406abe0 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fe5b4001670 con 0x7fe5c406abe0 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 >> v1:172.21.15.19:6789/0 conn(0x7fe5c406abe0 legacy=0x7fe5c406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5b00039b0 0x7fe5b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b0004800 0x7fe5b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5b00051c0 0x7fe5b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40cd550 con 0x7fe5b0004800 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b000c160 con 0x7fe5b00051c0 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b000c360 con 0x7fe5b00039b0 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5bbfff640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b0004800 0x7fe5b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c259c640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5b00039b0 0x7fe5b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c2d9d640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5b00051c0 0x7fe5b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5bbfff640 1 -- 172.21.15.19:0/3803385750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5b00039b0 msgr2=0x7fe5b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5bbfff640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5b00039b0 0x7fe5b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5bbfff640 1 -- 172.21.15.19:0/3803385750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5b00051c0 msgr2=0x7fe5b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5bbfff640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5b00051c0 0x7fe5b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5bbfff640 1 -- 172.21.15.19:0/3803385750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b0003520 con 0x7fe5b0004800 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c2d9d640 1 -- 172.21.15.19:0/3803385750 reap_dead start 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.583+0000 7fe5c259c640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5b00039b0 0x7fe5b0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.584+0000 7fe5bbfff640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b0004800 0x7fe5b0004c80 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009a90 tx=0x7fe5b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc9992c85a8cf0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.584+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac036070 con 0x7fe5b0004800 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.584+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ac031400 con 0x7fe5b0004800 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.584+0000 7fe5c1d9b640 1 -- 172.21.15.19:0/3803385750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac009690 con 0x7fe5b0004800 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.585+0000 7fe5c8d90640 1 -- 172.21.15.19:0/3803385750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b0004800 msgr2=0x7fe5b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.585+0000 7fe5c8d90640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b0004800 0x7fe5b0004c80 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009a90 tx=0x7fe5b0005830 comp rx=0 tx=0).stop 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.585+0000 7fe5c8d90640 1 -- 172.21.15.19:0/3803385750 shutdown_connections 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.585+0000 7fe5c8d90640 1 --2- 172.21.15.19:0/3803385750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5b0004800 0x7fe5b0004c80 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.585+0000 7fe5c8d90640 1 -- 172.21.15.19:0/3803385750 >> 172.21.15.19:0/3803385750 conn(0x7fe5c40b2220 msgr2=0x7fe5c40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.585+0000 7fe5c8d90640 1 -- 172.21.15.19:0/3803385750 shutdown_connections 2024-04-25T10:07:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.585+0000 7fe5c8d90640 1 -- 172.21.15.19:0/3803385750 wait complete. 2024-04-25T10:07:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.586+0000 7fe5c8d90640 1 Processor -- start 2024-04-25T10:07:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.586+0000 7fe5c8d90640 1 -- start start 2024-04-25T10:07:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c8d90640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5b0004800 0x7fe5c41f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c8d90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c414e1c0 0x7fe5c41f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c8d90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5c414af00 0x7fe5c40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c8d90640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c406fb60 con 0x7fe5c414af00 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c8d90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c4161780 con 0x7fe5c414e1c0 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c8d90640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c406fce0 con 0x7fe5b0004800 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5c414af00 0x7fe5c40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5c414af00 0x7fe5c40cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45932/0 (socket says 172.21.15.19:45932) 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c2d9d640 1 -- 172.21.15.19:0/2385726843 learned_addr learned my addr 172.21.15.19:0/2385726843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c259c640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5b0004800 0x7fe5c41f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5bbfff640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c414e1c0 0x7fe5c41f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c2d9d640 1 -- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5b0004800 msgr2=0x7fe5c41f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c2d9d640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5b0004800 0x7fe5c41f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c2d9d640 1 -- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c414e1c0 msgr2=0x7fe5c41f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c2d9d640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c414e1c0 0x7fe5c41f3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5c2d9d640 1 -- 172.21.15.19:0/2385726843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c41f7ef0 con 0x7fe5c414af00 2024-04-25T10:07:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.587+0000 7fe5bbfff640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c414e1c0 0x7fe5c41f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.588+0000 7fe5c2d9d640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5c414af00 0x7fe5c40cf9f0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fe5c41616c0 tx=0x7fe5bc00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.588+0000 7fe5baffd640 1 -- 172.21.15.19:0/2385726843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc019070 con 0x7fe5c414af00 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.588+0000 7fe5baffd640 1 -- 172.21.15.19:0/2385726843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc006d60 con 0x7fe5c414af00 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.588+0000 7fe5c8d90640 1 -- 172.21.15.19:0/2385726843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c41f81c0 con 0x7fe5c414af00 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.588+0000 7fe5baffd640 1 -- 172.21.15.19:0/2385726843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc003b90 con 0x7fe5c414af00 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.588+0000 7fe5c8d90640 1 -- 172.21.15.19:0/2385726843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c41f8780 con 0x7fe5c414af00 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.589+0000 7fe5c8d90640 1 -- 172.21.15.19:0/2385726843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c4073990 con 0x7fe5c414af00 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.589+0000 7fe5baffd640 1 -- 172.21.15.19:0/2385726843 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5bc01c070 con 0x7fe5c414af00 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.590+0000 7fe5baffd640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe5a406b4d0 0x7fe5a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.590+0000 7fe5baffd640 1 -- 172.21.15.19:0/2385726843 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe5bc089960 con 0x7fe5c414af00 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.590+0000 7fe5c259c640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe5a406b4d0 0x7fe5a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.591+0000 7fe5c259c640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe5a406b4d0 0x7fe5a406d990 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40017e0 tx=0x7fe5b4007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.594+0000 7fe5baffd640 1 -- 172.21.15.19:0/2385726843 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5bc057f40 con 0x7fe5c414af00 2024-04-25T10:07:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.811+0000 7fe5c8d90640 1 -- 172.21.15.19:0/2385726843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5c40b2ff0 con 0x7fe5c414af00 2024-04-25T10:07:01.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.811+0000 7fe5baffd640 1 -- 172.21.15.19:0/2385726843 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+4956 (secure 0 0 0) 0x7fe5bc05bfe0 con 0x7fe5c414af00 2024-04-25T10:07:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:01.817 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":129,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:38.424174+0000","modified":"2024-04-25T10:05:55.398362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":192,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11799":{"gid":11799,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/1758455116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1758455116},{"type":"v1","addr":"172.21.15.19:6835","nonce":1758455116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:07:01.817 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 129 2024-04-25T10:07:01.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 -- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe5a406b4d0 msgr2=0x7fe5a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe5a406b4d0 0x7fe5a406d990 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40017e0 tx=0x7fe5b4007040 comp rx=0 tx=0).stop 2024-04-25T10:07:01.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 -- 172.21.15.19:0/2385726843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5c414af00 msgr2=0x7fe5c40cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5c414af00 0x7fe5c40cf9f0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fe5c41616c0 tx=0x7fe5bc00ea00 comp rx=0 tx=0).stop 2024-04-25T10:07:01.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 -- 172.21.15.19:0/2385726843 shutdown_connections 2024-04-25T10:07:01.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe5a406b4d0 0x7fe5a406d990 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5c414af00 0x7fe5c40cf9f0 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c414e1c0 0x7fe5c41f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 --2- 172.21.15.19:0/2385726843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5b0004800 0x7fe5c41f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 -- 172.21.15.19:0/2385726843 >> 172.21.15.19:0/2385726843 conn(0x7fe5c40b2220 msgr2=0x7fe5c40ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 -- 172.21.15.19:0/2385726843 shutdown_connections 2024-04-25T10:07:01.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.813+0000 7fe5c8d90640 1 -- 172.21.15.19:0/2385726843 wait complete. 2024-04-25T10:07:01.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.925+0000 7fb97c14c640 1 Processor -- start 2024-04-25T10:07:01.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.925+0000 7fb97c14c640 1 -- start start 2024-04-25T10:07:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.926+0000 7fb97c14c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb97415dc00 0x7fb9740c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.926+0000 7fb97c14c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9740ceb20 0x7fb9740d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.926+0000 7fb97c14c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97415dfe0 con 0x7fb9740d1c00 2024-04-25T10:07:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.926+0000 7fb97c14c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9740d2060 con 0x7fb9740ceb20 2024-04-25T10:07:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.926+0000 7fb97c14c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740d00c0 con 0x7fb97415dc00 2024-04-25T10:07:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.926+0000 7fb97a6e9640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb9740d1c00 0x7fb9740ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54364/0 (socket says 172.21.15.19:54364) 2024-04-25T10:07:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.926+0000 7fb97a6e9640 1 -- 172.21.15.19:0/548638085 learned_addr learned my addr 172.21.15.19:0/548638085 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.926+0000 7fb979ee8640 1 --2- 172.21.15.19:0/548638085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb97415dc00 0x7fb9740c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.926+0000 7fb9796e7640 1 --2- 172.21.15.19:0/548638085 >> v2:172.21.15.112:3300/0 conn(0x7fb9740ceb20 0x7fb9740d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.926+0000 7fb978ee6640 1 -- 172.21.15.19:0/548638085 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964446357 0 0) 0x7fb96c001490 con 0x7fb9740d1c00 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.926+0000 7fb978ee6640 1 -- 172.21.15.19:0/548638085 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb960003520 con 0x7fb9740d1c00 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.927+0000 7fb979ee8640 1 -- 172.21.15.19:0/548638085 >> v2:172.21.15.112:3300/0 conn(0x7fb9740ceb20 msgr2=0x7fb9740d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.927+0000 7fb979ee8640 1 --2- 172.21.15.19:0/548638085 >> v2:172.21.15.112:3300/0 conn(0x7fb9740ceb20 0x7fb9740d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.927+0000 7fb979ee8640 1 -- 172.21.15.19:0/548638085 >> v1:172.21.15.19:6789/0 conn(0x7fb9740d1c00 legacy=0x7fb9740ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.927+0000 7fb978ee6640 1 -- 172.21.15.19:0/548638085 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 223085671 0 0) 0x7fb960003520 con 0x7fb9740d1c00 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.927+0000 7fb979ee8640 1 -- 172.21.15.19:0/548638085 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9740d03a0 con 0x7fb97415dc00 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.927+0000 7fb979ee8640 1 --2- 172.21.15.19:0/548638085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb97415dc00 0x7fb9740c6df0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9700097a0 tx=0x7fb97002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf669426d17999cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.928+0000 7fb978ee6640 1 -- 172.21.15.19:0/548638085 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970037070 con 0x7fb97415dc00 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.928+0000 7fb978ee6640 1 -- 172.21.15.19:0/548638085 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97002aba0 con 0x7fb97415dc00 2024-04-25T10:07:01.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.928+0000 7fb978ee6640 1 -- 172.21.15.19:0/548638085 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c001bc0 con 0x7fb97415dc00 2024-04-25T10:07:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.928+0000 7fb97c14c640 1 -- 172.21.15.19:0/548638085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb97415dc00 msgr2=0x7fb9740c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.928+0000 7fb97c14c640 1 --2- 172.21.15.19:0/548638085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb97415dc00 0x7fb9740c6df0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9700097a0 tx=0x7fb97002a110 comp rx=0 tx=0).stop 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.928+0000 7fb97c14c640 1 -- 172.21.15.19:0/548638085 shutdown_connections 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.928+0000 7fb97c14c640 1 --2- 172.21.15.19:0/548638085 >> v2:172.21.15.112:3300/0 conn(0x7fb9740ceb20 0x7fb9740d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.928+0000 7fb97c14c640 1 --2- 172.21.15.19:0/548638085 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb97415dc00 0x7fb9740c6df0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.928+0000 7fb97c14c640 1 -- 172.21.15.19:0/548638085 >> 172.21.15.19:0/548638085 conn(0x7fb97406f5d0 msgr2=0x7fb97406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.928+0000 7fb97c14c640 1 -- 172.21.15.19:0/548638085 shutdown_connections 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.928+0000 7fb97c14c640 1 -- 172.21.15.19:0/548638085 wait complete. 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.929+0000 7fb97c14c640 1 Processor -- start 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.929+0000 7fb97c14c640 1 -- start start 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb97c14c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9740ceb20 0x7fb9740b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb97c14c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9740d1c00 0x7fb974156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb97c14c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb97415dc00 0x7fb9740ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb97c14c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740cc130 con 0x7fb9740ceb20 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb97c14c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740cc720 con 0x7fb97415dc00 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb97c14c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740cfd90 con 0x7fb9740d1c00 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb979ee8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9740ceb20 0x7fb9740b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb979ee8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9740ceb20 0x7fb9740b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45944/0 (socket says 172.21.15.19:45944) 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb979ee8640 1 -- 172.21.15.19:0/2591721602 learned_addr learned my addr 172.21.15.19:0/2591721602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb9796e7640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9740d1c00 0x7fb974156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb979ee8640 1 -- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9740d1c00 msgr2=0x7fb974156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb979ee8640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9740d1c00 0x7fb974156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.930+0000 7fb97a6e9640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb97415dc00 0x7fb9740ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.931+0000 7fb979ee8640 1 -- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb97415dc00 msgr2=0x7fb9740ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.931+0000 7fb979ee8640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb97415dc00 0x7fb9740ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.931+0000 7fb979ee8640 1 -- 172.21.15.19:0/2591721602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9740b4a80 con 0x7fb9740ceb20 2024-04-25T10:07:01.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.931+0000 7fb9796e7640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9740d1c00 0x7fb974156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.931+0000 7fb979ee8640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9740ceb20 0x7fb9740b44a0 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7fb970035040 tx=0x7fb9700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.931+0000 7fb96a7fc640 1 -- 172.21.15.19:0/2591721602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970037050 con 0x7fb9740ceb20 2024-04-25T10:07:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.931+0000 7fb96a7fc640 1 -- 172.21.15.19:0/2591721602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb970043070 con 0x7fb9740ceb20 2024-04-25T10:07:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.931+0000 7fb97c14c640 1 -- 172.21.15.19:0/2591721602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9740b4d60 con 0x7fb9740ceb20 2024-04-25T10:07:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.931+0000 7fb96a7fc640 1 -- 172.21.15.19:0/2591721602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970006e10 con 0x7fb9740ceb20 2024-04-25T10:07:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.931+0000 7fb97c14c640 1 -- 172.21.15.19:0/2591721602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9740b5600 con 0x7fb9740ceb20 2024-04-25T10:07:01.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.932+0000 7fb97c14c640 1 -- 172.21.15.19:0/2591721602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9740b4f30 con 0x7fb9740ceb20 2024-04-25T10:07:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.933+0000 7fb96a7fc640 1 -- 172.21.15.19:0/2591721602 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb97003ca10 con 0x7fb9740ceb20 2024-04-25T10:07:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.933+0000 7fb96a7fc640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb95806b4d0 0x7fb95806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.933+0000 7fb96a7fc640 1 -- 172.21.15.19:0/2591721602 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb9700ab0e0 con 0x7fb9740ceb20 2024-04-25T10:07:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.933+0000 7fb9796e7640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb95806b4d0 0x7fb95806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.934+0000 7fb9796e7640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb95806b4d0 0x7fb95806d990 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb97415a4c0 tx=0x7fb964008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:01.937+0000 7fb96a7fc640 1 -- 172.21.15.19:0/2591721602 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb970030e20 con 0x7fb9740ceb20 2024-04-25T10:07:02.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.118+0000 7fb97c14c640 1 -- 172.21.15.19:0/2591721602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb974073990 con 0x7fb9740ceb20 2024-04-25T10:07:02.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.119+0000 7fb96a7fc640 1 -- 172.21.15.19:0/2591721602 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+17195 (secure 0 0 0) 0x7fb970079d20 con 0x7fb9740ceb20 2024-04-25T10:07:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:02.125 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":192,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:52.378385+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:36.700326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:37.402558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":6,"to":0}]},{"pgid":"29.b","mappings":[{"from":6,"to":0}]},{"pgid":"29.13","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/392508344":"2024-04-26T10:05:52.378361+0000","172.21.15.19:6833/392508344":"2024-04-26T10:05:52.378361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:02.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 -- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb95806b4d0 msgr2=0x7fb95806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb95806b4d0 0x7fb95806d990 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb97415a4c0 tx=0x7fb964008040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 -- 172.21.15.19:0/2591721602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9740ceb20 msgr2=0x7fb9740b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9740ceb20 0x7fb9740b44a0 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7fb970035040 tx=0x7fb9700046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 -- 172.21.15.19:0/2591721602 shutdown_connections 2024-04-25T10:07:02.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb95806b4d0 0x7fb95806d990 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb97415dc00 0x7fb9740ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9740d1c00 0x7fb974156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 --2- 172.21.15.19:0/2591721602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9740ceb20 0x7fb9740b44a0 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 -- 172.21.15.19:0/2591721602 >> 172.21.15.19:0/2591721602 conn(0x7fb97406f5d0 msgr2=0x7fb9740b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 -- 172.21.15.19:0/2591721602 shutdown_connections 2024-04-25T10:07:02.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.121+0000 7fb97c14c640 1 -- 172.21.15.19:0/2591721602 wait complete. 2024-04-25T10:07:02.138 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:07:02.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:02.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.233+0000 7f74725b0640 1 Processor -- start 2024-04-25T10:07:02.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.233+0000 7f74725b0640 1 -- start start 2024-04-25T10:07:02.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f74725b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c15dd30 0x7f746c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f74725b0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f746c0d0ad0 0x7f746c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f74725b0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746c0d1df0 con 0x7f746c0b9770 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f74725b0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746c0d20d0 con 0x7f746c0d0ad0 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f74725b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0d23b0 con 0x7f746c15dd30 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f7470b4d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f746c0d0ad0 0x7f746c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f746b7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f746c0b9770 0x7f746c0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54370/0 (socket says 172.21.15.19:54370) 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f746b7fe640 1 -- 172.21.15.19:0/2502055905 learned_addr learned my addr 172.21.15.19:0/2502055905 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f746bfff640 1 --2- 172.21.15.19:0/2502055905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c15dd30 0x7f746c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f7470b4d640 1 -- 172.21.15.19:0/2502055905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c15dd30 msgr2=0x7f746c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f7470b4d640 1 --2- 172.21.15.19:0/2502055905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c15dd30 0x7f746c0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f7470b4d640 1 -- 172.21.15.19:0/2502055905 >> v1:172.21.15.19:6789/0 conn(0x7f746c0b9770 legacy=0x7f746c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f7470b4d640 1 -- 172.21.15.19:0/2502055905 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c0d2690 con 0x7f746c0d0ad0 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f746affd640 1 -- 172.21.15.19:0/2502055905 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441001921 0 0) 0x7f746c0d1df0 con 0x7f746c0b9770 2024-04-25T10:07:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.234+0000 7f746bfff640 1 --2- 172.21.15.19:0/2502055905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c15dd30 0x7f746c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.235+0000 7f7470b4d640 1 --2- 172.21.15.19:0/2502055905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0d0ad0 0x7f746c0b6a40 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f7464009d90 tx=0x7f746402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e43b5a4658d3df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.235+0000 7f746affd640 1 -- 172.21.15.19:0/2502055905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464031600 con 0x7f746c0d0ad0 2024-04-25T10:07:02.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.235+0000 7f746affd640 1 -- 172.21.15.19:0/2502055905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74640317a0 con 0x7f746c0d0ad0 2024-04-25T10:07:02.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.235+0000 7f746affd640 1 -- 172.21.15.19:0/2502055905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464031b70 con 0x7f746c0d0ad0 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.236+0000 7f74725b0640 1 -- 172.21.15.19:0/2502055905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0d0ad0 msgr2=0x7f746c0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.236+0000 7f74725b0640 1 --2- 172.21.15.19:0/2502055905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0d0ad0 0x7f746c0b6a40 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f7464009d90 tx=0x7f746402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.236+0000 7f74725b0640 1 -- 172.21.15.19:0/2502055905 shutdown_connections 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.236+0000 7f74725b0640 1 --2- 172.21.15.19:0/2502055905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0d0ad0 0x7f746c0b6a40 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.236+0000 7f74725b0640 1 --2- 172.21.15.19:0/2502055905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c15dd30 0x7f746c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.236+0000 7f74725b0640 1 -- 172.21.15.19:0/2502055905 >> 172.21.15.19:0/2502055905 conn(0x7f746c061c30 msgr2=0x7f746c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.236+0000 7f74725b0640 1 -- 172.21.15.19:0/2502055905 shutdown_connections 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.236+0000 7f74725b0640 1 -- 172.21.15.19:0/2502055905 wait complete. 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.237+0000 7f74725b0640 1 Processor -- start 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.237+0000 7f74725b0640 1 -- start start 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f74725b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0b9770 0x7f746c15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f74725b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c0d0ad0 0x7f746c1fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f74725b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c15dd30 0x7f746c156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f74725b0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0d1b90 con 0x7f746c0b9770 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f74725b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0d1a10 con 0x7f746c15dd30 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f74725b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0d2e90 con 0x7f746c0d0ad0 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f746bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0b9770 0x7f746c15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f746bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0b9770 0x7f746c15faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45966/0 (socket says 172.21.15.19:45966) 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f746bfff640 1 -- 172.21.15.19:0/2057876021 learned_addr learned my addr 172.21.15.19:0/2057876021 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f746b7fe640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c0d0ad0 0x7f746c1fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f7470b4d640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c15dd30 0x7f746c156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f746bfff640 1 -- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c0d0ad0 msgr2=0x7f746c1fb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f746bfff640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c0d0ad0 0x7f746c1fb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f746bfff640 1 -- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c15dd30 msgr2=0x7f746c156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f746bfff640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c15dd30 0x7f746c156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f746bfff640 1 -- 172.21.15.19:0/2057876021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c160130 con 0x7f746c0b9770 2024-04-25T10:07:02.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f7470b4d640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c15dd30 0x7f746c156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f746bfff640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0b9770 0x7f746c15faa0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f746000c820 tx=0x7f746000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f7468ff9640 1 -- 172.21.15.19:0/2057876021 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460015070 con 0x7f746c0b9770 2024-04-25T10:07:02.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.238+0000 7f7468ff9640 1 -- 172.21.15.19:0/2057876021 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7460007a30 con 0x7f746c0b9770 2024-04-25T10:07:02.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.239+0000 7f7468ff9640 1 -- 172.21.15.19:0/2057876021 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460007de0 con 0x7f746c0b9770 2024-04-25T10:07:02.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.239+0000 7f74725b0640 1 -- 172.21.15.19:0/2057876021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c160410 con 0x7f746c0b9770 2024-04-25T10:07:02.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.239+0000 7f74725b0640 1 -- 172.21.15.19:0/2057876021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c1609d0 con 0x7f746c0b9770 2024-04-25T10:07:02.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.240+0000 7f74725b0640 1 -- 172.21.15.19:0/2057876021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746c0cf840 con 0x7f746c0b9770 2024-04-25T10:07:02.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.240+0000 7f7468ff9640 1 -- 172.21.15.19:0/2057876021 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7460018070 con 0x7f746c0b9770 2024-04-25T10:07:02.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.240+0000 7f7468ff9640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745806b4d0 0x7f745806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.241+0000 7f7468ff9640 1 -- 172.21.15.19:0/2057876021 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f746008a3f0 con 0x7f746c0b9770 2024-04-25T10:07:02.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.241+0000 7f746b7fe640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745806b4d0 0x7f745806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.241+0000 7f746b7fe640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745806b4d0 0x7f745806d990 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f746c0b8c90 tx=0x7f745c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.245+0000 7f7468ff9640 1 -- 172.21.15.19:0/2057876021 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f746c0cf840 con 0x7f746c0b9770 2024-04-25T10:07:02.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.461+0000 7f74725b0640 1 -- 172.21.15.19:0/2057876021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f746c0c9e70 con 0x7f746c0b9770 2024-04-25T10:07:02.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.462+0000 7f7468ff9640 1 -- 172.21.15.19:0/2057876021 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+4956 (secure 0 0 0) 0x7f7460058fa0 con 0x7f746c0b9770 2024-04-25T10:07:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:02.468 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":129,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:38.424174+0000","modified":"2024-04-25T10:05:55.398362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":192,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11799":{"gid":11799,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/1758455116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1758455116},{"type":"v1","addr":"172.21.15.19:6835","nonce":1758455116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:07:02.468 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 129 2024-04-25T10:07:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 -- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745806b4d0 msgr2=0x7f745806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745806b4d0 0x7f745806d990 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f746c0b8c90 tx=0x7f745c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 -- 172.21.15.19:0/2057876021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0b9770 msgr2=0x7f746c15faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0b9770 0x7f746c15faa0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f746000c820 tx=0x7f746000cce0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 -- 172.21.15.19:0/2057876021 shutdown_connections 2024-04-25T10:07:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f745806b4d0 0x7f745806d990 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f746c15dd30 0x7f746c156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f746c0d0ad0 0x7f746c1fb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 --2- 172.21.15.19:0/2057876021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c0b9770 0x7f746c15faa0 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 -- 172.21.15.19:0/2057876021 >> 172.21.15.19:0/2057876021 conn(0x7f746c061c30 msgr2=0x7f746c06c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 -- 172.21.15.19:0/2057876021 shutdown_connections 2024-04-25T10:07:02.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.464+0000 7f74725b0640 1 -- 172.21.15.19:0/2057876021 wait complete. 2024-04-25T10:07:02.481 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:02.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.576+0000 7fe48b35c640 1 Processor -- start 2024-04-25T10:07:02.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.576+0000 7fe48b35c640 1 -- start start 2024-04-25T10:07:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe48b35c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4840d1dd0 0x7fe4840ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe48b35c640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe48415dfe0 con 0x7fe48415dc00 2024-04-25T10:07:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe48b35c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4840d00c0 con 0x7fe4840c6df0 2024-04-25T10:07:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe48b35c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4840d03a0 con 0x7fe4840d1dd0 2024-04-25T10:07:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe4890f8640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe48415dc00 0x7fe4840ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54378/0 (socket says 172.21.15.19:54378) 2024-04-25T10:07:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe4890f8640 1 -- 172.21.15.19:0/3229387158 learned_addr learned my addr 172.21.15.19:0/3229387158 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287701674 0 0) 0x7fe480001110 con 0x7fe48415dc00 2024-04-25T10:07:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe470003520 con 0x7fe48415dc00 2024-04-25T10:07:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe4898f9640 1 --2- 172.21.15.19:0/3229387158 >> v2:172.21.15.112:3300/0 conn(0x7fe4840d1dd0 0x7fe4840ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2494729240 0 0) 0x7fe470003520 con 0x7fe48415dc00 2024-04-25T10:07:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 >> v2:172.21.15.112:3300/0 conn(0x7fe4840d1dd0 msgr2=0x7fe4840ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe47bfff640 1 --2- 172.21.15.19:0/3229387158 >> v2:172.21.15.112:3300/0 conn(0x7fe4840d1dd0 0x7fe4840ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 >> v1:172.21.15.112:6789/0 conn(0x7fe4840c6df0 legacy=0x7fe4840d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4840cc440 con 0x7fe48415dc00 2024-04-25T10:07:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fe480001840 con 0x7fe48415dc00 2024-04-25T10:07:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.577+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 >> v1:172.21.15.19:6789/0 conn(0x7fe48415dc00 legacy=0x7fe4840ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe47bfff640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 0x7fe470003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe47bfff640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4700043b0 0x7fe470004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe47bfff640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe470005080 0x7fe47000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840d03a0 con 0x7fe4700039b0 2024-04-25T10:07:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47000bfe0 con 0x7fe470005080 2024-04-25T10:07:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47000c1e0 con 0x7fe4700043b0 2024-04-25T10:07:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe4890f8640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 0x7fe470003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe4888f7640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4700043b0 0x7fe470004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe4898f9640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe470005080 0x7fe47000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe4890f8640 1 -- 172.21.15.19:0/3229387158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4700043b0 msgr2=0x7fe470004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe4890f8640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4700043b0 0x7fe470004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe4890f8640 1 -- 172.21.15.19:0/3229387158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe470005080 msgr2=0x7fe47000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe4890f8640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe470005080 0x7fe47000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe4890f8640 1 -- 172.21.15.19:0/3229387158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe470003520 con 0x7fe4700039b0 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe4898f9640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe470005080 0x7fe47000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.578+0000 7fe4898f9640 1 -- 172.21.15.19:0/3229387158 reap_dead start 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.579+0000 7fe4890f8640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 0x7fe470003d90 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c3e0 tx=0x7fe470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd1035533233ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.579+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48000c0e0 con 0x7fe4700039b0 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.579+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe480034070 con 0x7fe4700039b0 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.579+0000 7fe47bfff640 1 -- 172.21.15.19:0/3229387158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480005d50 con 0x7fe4700039b0 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.579+0000 7fe48b35c640 1 -- 172.21.15.19:0/3229387158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 msgr2=0x7fe470003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.579+0000 7fe48b35c640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 0x7fe470003d90 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c3e0 tx=0x7fe470005830 comp rx=0 tx=0).stop 2024-04-25T10:07:02.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.579+0000 7fe48b35c640 1 -- 172.21.15.19:0/3229387158 shutdown_connections 2024-04-25T10:07:02.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.579+0000 7fe48b35c640 1 --2- 172.21.15.19:0/3229387158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 0x7fe470003d90 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.579+0000 7fe48b35c640 1 -- 172.21.15.19:0/3229387158 >> 172.21.15.19:0/3229387158 conn(0x7fe48406f5d0 msgr2=0x7fe48406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.579+0000 7fe48b35c640 1 -- 172.21.15.19:0/3229387158 shutdown_connections 2024-04-25T10:07:02.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.580+0000 7fe48b35c640 1 -- 172.21.15.19:0/3229387158 wait complete. 2024-04-25T10:07:02.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.581+0000 7fe48b35c640 1 Processor -- start 2024-04-25T10:07:02.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.581+0000 7fe48b35c640 1 -- start start 2024-04-25T10:07:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.581+0000 7fe48b35c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 0x7fe4840b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.581+0000 7fe48b35c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4840c88f0 0x7fe48414e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.581+0000 7fe48b35c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe484157640 0x7fe484158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.581+0000 7fe48b35c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840cfd90 con 0x7fe4700039b0 2024-04-25T10:07:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.581+0000 7fe48b35c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840cc130 con 0x7fe484157640 2024-04-25T10:07:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.581+0000 7fe48b35c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840cdd70 con 0x7fe4840c88f0 2024-04-25T10:07:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.581+0000 7fe4890f8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 0x7fe4840b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe4890f8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 0x7fe4840b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45980/0 (socket says 172.21.15.19:45980) 2024-04-25T10:07:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe4890f8640 1 -- 172.21.15.19:0/1458168976 learned_addr learned my addr 172.21.15.19:0/1458168976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe4888f7640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4840c88f0 0x7fe48414e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe4898f9640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe484157640 0x7fe484158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe4890f8640 1 -- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4840c88f0 msgr2=0x7fe48414e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe4890f8640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4840c88f0 0x7fe48414e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe4890f8640 1 -- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe484157640 msgr2=0x7fe484158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe4890f8640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe484157640 0x7fe484158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe4890f8640 1 -- 172.21.15.19:0/1458168976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48406d840 con 0x7fe4700039b0 2024-04-25T10:07:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe4890f8640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 0x7fe4840b4db0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c4d0 tx=0x7fe480001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe479ffb640 1 -- 172.21.15.19:0/1458168976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480007270 con 0x7fe4700039b0 2024-04-25T10:07:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe479ffb640 1 -- 172.21.15.19:0/1458168976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe480000ed0 con 0x7fe4700039b0 2024-04-25T10:07:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe48b35c640 1 -- 172.21.15.19:0/1458168976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48414fce0 con 0x7fe4700039b0 2024-04-25T10:07:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe479ffb640 1 -- 172.21.15.19:0/1458168976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480007650 con 0x7fe4700039b0 2024-04-25T10:07:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.582+0000 7fe48b35c640 1 -- 172.21.15.19:0/1458168976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4841502a0 con 0x7fe4700039b0 2024-04-25T10:07:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.583+0000 7fe48b35c640 1 -- 172.21.15.19:0/1458168976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe484073990 con 0x7fe4700039b0 2024-04-25T10:07:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.584+0000 7fe479ffb640 1 -- 172.21.15.19:0/1458168976 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe48002f480 con 0x7fe4700039b0 2024-04-25T10:07:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.584+0000 7fe479ffb640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe46806b4d0 0x7fe46806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.584+0000 7fe479ffb640 1 -- 172.21.15.19:0/1458168976 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe4800ab210 con 0x7fe4700039b0 2024-04-25T10:07:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.584+0000 7fe4888f7640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe46806b4d0 0x7fe46806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.585+0000 7fe4888f7640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe46806b4d0 0x7fe46806d990 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe4841f6c80 tx=0x7fe474009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.588+0000 7fe479ffb640 1 -- 172.21.15.19:0/1458168976 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe480049030 con 0x7fe4700039b0 2024-04-25T10:07:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.810+0000 7fe48b35c640 1 -- 172.21.15.19:0/1458168976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4840c66c0 con 0x7fe4700039b0 2024-04-25T10:07:02.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.810+0000 7fe479ffb640 1 -- 172.21.15.19:0/1458168976 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+4956 (secure 0 0 0) 0x7fe48415dfe0 con 0x7fe4700039b0 2024-04-25T10:07:02.816 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:02.816 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":129,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:38.424174+0000","modified":"2024-04-25T10:05:55.398362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":192,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11799":{"gid":11799,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/1758455116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1758455116},{"type":"v1","addr":"172.21.15.19:6835","nonce":1758455116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:07:02.816 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 129 2024-04-25T10:07:02.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 -- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe46806b4d0 msgr2=0x7fe46806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe46806b4d0 0x7fe46806d990 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe4841f6c80 tx=0x7fe474009040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 -- 172.21.15.19:0/1458168976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 msgr2=0x7fe4840b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 0x7fe4840b4db0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c4d0 tx=0x7fe480001c10 comp rx=0 tx=0).stop 2024-04-25T10:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 -- 172.21.15.19:0/1458168976 shutdown_connections 2024-04-25T10:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe46806b4d0 0x7fe46806d990 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe484157640 0x7fe484158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4840c88f0 0x7fe48414e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 --2- 172.21.15.19:0/1458168976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4700039b0 0x7fe4840b4db0 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 -- 172.21.15.19:0/1458168976 >> 172.21.15.19:0/1458168976 conn(0x7fe48406f5d0 msgr2=0x7fe4840b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 -- 172.21.15.19:0/1458168976 shutdown_connections 2024-04-25T10:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.813+0000 7fe48b35c640 1 -- 172.21.15.19:0/1458168976 wait complete. 2024-04-25T10:07:02.829 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.924+0000 7f1db3d63640 1 Processor -- start 2024-04-25T10:07:02.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.924+0000 7f1db3d63640 1 -- start start 2024-04-25T10:07:02.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.925+0000 7f1db3d63640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1dac0d1dd0 0x7f1dac0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.925+0000 7f1db3d63640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dac15dfe0 con 0x7f1dac0c6df0 2024-04-25T10:07:02.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.925+0000 7f1db3d63640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dac0d00c0 con 0x7f1dac15dc00 2024-04-25T10:07:02.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.925+0000 7f1db3d63640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dac0d03a0 con 0x7f1dac0d1dd0 2024-04-25T10:07:02.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.925+0000 7f1db2300640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1dac0d1dd0 0x7f1dac0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.925+0000 7f1db12fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1dac0c6df0 0x7f1dac0d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54384/0 (socket says 172.21.15.19:54384) 2024-04-25T10:07:02.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.925+0000 7f1db12fe640 1 -- 172.21.15.19:0/3050006701 learned_addr learned my addr 172.21.15.19:0/3050006701 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:02.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.925+0000 7f1db2300640 1 -- 172.21.15.19:0/3050006701 >> v1:172.21.15.112:6789/0 conn(0x7f1dac15dc00 legacy=0x7f1dac0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.925+0000 7f1db2300640 1 -- 172.21.15.19:0/3050006701 >> v1:172.21.15.19:6789/0 conn(0x7f1dac0c6df0 legacy=0x7f1dac0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.925+0000 7f1db2300640 1 -- 172.21.15.19:0/3050006701 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac0cc440 con 0x7f1dac0d1dd0 2024-04-25T10:07:02.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.926+0000 7f1db2300640 1 --2- 172.21.15.19:0/3050006701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac0d1dd0 0x7f1dac0ccc40 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009d90 tx=0x7f1d9c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703b1a83afa8afe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.926+0000 7f1db0afd640 1 -- 172.21.15.19:0/3050006701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c007740 con 0x7f1dac0d1dd0 2024-04-25T10:07:02.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.926+0000 7f1db0afd640 1 -- 172.21.15.19:0/3050006701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c0078e0 con 0x7f1dac0d1dd0 2024-04-25T10:07:02.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.926+0000 7f1db0afd640 1 -- 172.21.15.19:0/3050006701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c007cb0 con 0x7f1dac0d1dd0 2024-04-25T10:07:02.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.927+0000 7f1db3d63640 1 -- 172.21.15.19:0/3050006701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac0d1dd0 msgr2=0x7f1dac0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.927+0000 7f1db3d63640 1 --2- 172.21.15.19:0/3050006701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac0d1dd0 0x7f1dac0ccc40 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009d90 tx=0x7f1d9c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.927+0000 7f1db3d63640 1 -- 172.21.15.19:0/3050006701 shutdown_connections 2024-04-25T10:07:02.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.927+0000 7f1db3d63640 1 --2- 172.21.15.19:0/3050006701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac0d1dd0 0x7f1dac0ccc40 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.927+0000 7f1db3d63640 1 -- 172.21.15.19:0/3050006701 >> 172.21.15.19:0/3050006701 conn(0x7f1dac06f5d0 msgr2=0x7f1dac06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.927+0000 7f1db3d63640 1 -- 172.21.15.19:0/3050006701 shutdown_connections 2024-04-25T10:07:02.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.927+0000 7f1db3d63640 1 -- 172.21.15.19:0/3050006701 wait complete. 2024-04-25T10:07:02.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.928+0000 7f1db3d63640 1 Processor -- start 2024-04-25T10:07:02.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.928+0000 7f1db3d63640 1 -- start start 2024-04-25T10:07:02.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db3d63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac0c6df0 0x7f1dac15a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db3d63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac0d1dd0 0x7f1dac157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db3d63640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1dac15dc00 0x7f1dac149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db3d63640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0cfd90 con 0x7f1dac0c6df0 2024-04-25T10:07:02.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db3d63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0cc130 con 0x7f1dac0d1dd0 2024-04-25T10:07:02.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db3d63640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0cdd70 con 0x7f1dac15dc00 2024-04-25T10:07:02.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db1aff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac0c6df0 0x7f1dac15a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db1aff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac0c6df0 0x7f1dac15a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46004/0 (socket says 172.21.15.19:46004) 2024-04-25T10:07:02.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db1aff640 1 -- 172.21.15.19:0/3686967683 learned_addr learned my addr 172.21.15.19:0/3686967683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db2300640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1dac15dc00 0x7f1dac149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db12fe640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac0d1dd0 0x7f1dac157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db1aff640 1 -- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1dac15dc00 msgr2=0x7f1dac149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db1aff640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1dac15dc00 0x7f1dac149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db1aff640 1 -- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac0d1dd0 msgr2=0x7f1dac157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db1aff640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac0d1dd0 0x7f1dac157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db1aff640 1 -- 172.21.15.19:0/3686967683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac158370 con 0x7f1dac0c6df0 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db12fe640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac0d1dd0 0x7f1dac157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.929+0000 7f1db1aff640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac0c6df0 0x7f1dac15a6c0 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b3f0 tx=0x7f1da400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.930+0000 7f1daa7fc640 1 -- 172.21.15.19:0/3686967683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4015020 con 0x7f1dac0c6df0 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.930+0000 7f1daa7fc640 1 -- 172.21.15.19:0/3686967683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da4011830 con 0x7f1dac0c6df0 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.930+0000 7f1db3d63640 1 -- 172.21.15.19:0/3686967683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac0b4840 con 0x7f1dac0c6df0 2024-04-25T10:07:02.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.930+0000 7f1daa7fc640 1 -- 172.21.15.19:0/3686967683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4011c20 con 0x7f1dac0c6df0 2024-04-25T10:07:02.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.930+0000 7f1db3d63640 1 -- 172.21.15.19:0/3686967683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac0b4e00 con 0x7f1dac0c6df0 2024-04-25T10:07:02.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.931+0000 7f1db3d63640 1 -- 172.21.15.19:0/3686967683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac073990 con 0x7f1dac0c6df0 2024-04-25T10:07:02.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.931+0000 7f1daa7fc640 1 -- 172.21.15.19:0/3686967683 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1da4018020 con 0x7f1dac0c6df0 2024-04-25T10:07:02.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.932+0000 7f1daa7fc640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d9806b4d0 0x7f1d9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.932+0000 7f1daa7fc640 1 -- 172.21.15.19:0/3686967683 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1da408a120 con 0x7f1dac0c6df0 2024-04-25T10:07:02.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.932+0000 7f1db12fe640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d9806b4d0 0x7f1d9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.933+0000 7f1db12fe640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d9806b4d0 0x7f1d9806d990 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1dac153ac0 tx=0x7f1da0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:02.936+0000 7f1daa7fc640 1 -- 172.21.15.19:0/3686967683 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1da401f030 con 0x7f1dac0c6df0 2024-04-25T10:07:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.116+0000 7f1db3d63640 1 -- 172.21.15.19:0/3686967683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dac0c66c0 con 0x7f1dac0c6df0 2024-04-25T10:07:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.117+0000 7f1daa7fc640 1 -- 172.21.15.19:0/3686967683 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+17195 (secure 0 0 0) 0x7f1da4058c20 con 0x7f1dac0c6df0 2024-04-25T10:07:03.123 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:03.123 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":192,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:05:52.378385+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:36.700326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:37.402558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":6,"to":0}]},{"pgid":"29.b","mappings":[{"from":6,"to":0}]},{"pgid":"29.13","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/392508344":"2024-04-26T10:05:52.378361+0000","172.21.15.19:6833/392508344":"2024-04-26T10:05:52.378361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 -- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d9806b4d0 msgr2=0x7f1d9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d9806b4d0 0x7f1d9806d990 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1dac153ac0 tx=0x7f1da0009040 comp rx=0 tx=0).stop 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 -- 172.21.15.19:0/3686967683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac0c6df0 msgr2=0x7f1dac15a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac0c6df0 0x7f1dac15a6c0 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b3f0 tx=0x7f1da400b8b0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 -- 172.21.15.19:0/3686967683 shutdown_connections 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f1d9806b4d0 0x7f1d9806d990 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1dac15dc00 0x7f1dac149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac0d1dd0 0x7f1dac157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 --2- 172.21.15.19:0/3686967683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dac0c6df0 0x7f1dac15a6c0 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 -- 172.21.15.19:0/3686967683 >> 172.21.15.19:0/3686967683 conn(0x7f1dac06f5d0 msgr2=0x7f1dac0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 -- 172.21.15.19:0/3686967683 shutdown_connections 2024-04-25T10:07:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.119+0000 7f1db3d63640 1 -- 172.21.15.19:0/3686967683 wait complete. 2024-04-25T10:07:03.136 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:07:03.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.232+0000 7fe9a4be0640 1 Processor -- start 2024-04-25T10:07:03.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.232+0000 7fe9a4be0640 1 -- start start 2024-04-25T10:07:03.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.233+0000 7fe9a4be0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9a015b7d0 0x7fe9a00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.233+0000 7fe9a4be0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9a00b5620 0x7fe9a00b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.233+0000 7fe9a4be0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe9a006c2d0 0x7fe9a006c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.233+0000 7fe9a4be0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a00b5fc0 con 0x7fe9a015b7d0 2024-04-25T10:07:03.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.233+0000 7fe9a4be0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a006cca0 con 0x7fe9a006c2d0 2024-04-25T10:07:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.233+0000 7fe9a4be0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a006cf50 con 0x7fe9a00b5620 2024-04-25T10:07:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.233+0000 7fe99e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9a015b7d0 0x7fe9a00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.233+0000 7fe99e59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9a015b7d0 0x7fe9a00cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46018/0 (socket says 172.21.15.19:46018) 2024-04-25T10:07:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.233+0000 7fe99e59c640 1 -- 172.21.15.19:0/1756673858 learned_addr learned my addr 172.21.15.19:0/1756673858 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.233+0000 7fe997fff640 1 --2- 172.21.15.19:0/1756673858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9a00b5620 0x7fe9a00b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.233+0000 7fe99ed9d640 1 --2- 172.21.15.19:0/1756673858 >> v2:172.21.15.112:3300/0 conn(0x7fe9a006c2d0 0x7fe9a006c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.234+0000 7fe99e59c640 1 -- 172.21.15.19:0/1756673858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9a00b5620 msgr2=0x7fe9a00b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.234+0000 7fe99e59c640 1 --2- 172.21.15.19:0/1756673858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9a00b5620 0x7fe9a00b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.234+0000 7fe99e59c640 1 -- 172.21.15.19:0/1756673858 >> v2:172.21.15.112:3300/0 conn(0x7fe9a006c2d0 msgr2=0x7fe9a006c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.234+0000 7fe99e59c640 1 --2- 172.21.15.19:0/1756673858 >> v2:172.21.15.112:3300/0 conn(0x7fe9a006c2d0 0x7fe9a006c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.234+0000 7fe99e59c640 1 -- 172.21.15.19:0/1756673858 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00ce530 con 0x7fe9a015b7d0 2024-04-25T10:07:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.234+0000 7fe99e59c640 1 --2- 172.21.15.19:0/1756673858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a015b7d0 0x7fe9a00cf010 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fe9a015e340 tx=0x7fe99002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f943496f7c0dab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.234+0000 7fe99dd9b640 1 -- 172.21.15.19:0/1756673858 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990037070 con 0x7fe9a015b7d0 2024-04-25T10:07:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.235+0000 7fe99dd9b640 1 -- 172.21.15.19:0/1756673858 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe990032440 con 0x7fe9a015b7d0 2024-04-25T10:07:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.235+0000 7fe99dd9b640 1 -- 172.21.15.19:0/1756673858 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990032810 con 0x7fe9a015b7d0 2024-04-25T10:07:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.235+0000 7fe9a4be0640 1 -- 172.21.15.19:0/1756673858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a015b7d0 msgr2=0x7fe9a00cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.235+0000 7fe9a4be0640 1 --2- 172.21.15.19:0/1756673858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a015b7d0 0x7fe9a00cf010 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fe9a015e340 tx=0x7fe99002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.235+0000 7fe9a4be0640 1 -- 172.21.15.19:0/1756673858 shutdown_connections 2024-04-25T10:07:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.235+0000 7fe9a4be0640 1 --2- 172.21.15.19:0/1756673858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9a00b5620 0x7fe9a00b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.235+0000 7fe9a4be0640 1 --2- 172.21.15.19:0/1756673858 >> v2:172.21.15.112:3300/0 conn(0x7fe9a006c2d0 0x7fe9a006c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.235+0000 7fe9a4be0640 1 --2- 172.21.15.19:0/1756673858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a015b7d0 0x7fe9a00cf010 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.235+0000 7fe9a4be0640 1 -- 172.21.15.19:0/1756673858 >> 172.21.15.19:0/1756673858 conn(0x7fe9a00d0800 msgr2=0x7fe9a00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.235+0000 7fe9a4be0640 1 -- 172.21.15.19:0/1756673858 shutdown_connections 2024-04-25T10:07:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.235+0000 7fe9a4be0640 1 -- 172.21.15.19:0/1756673858 wait complete. 2024-04-25T10:07:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.236+0000 7fe9a4be0640 1 Processor -- start 2024-04-25T10:07:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.236+0000 7fe9a4be0640 1 -- start start 2024-04-25T10:07:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.237+0000 7fe9a4be0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9a006c2d0 0x7fe9a0159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.237+0000 7fe9a4be0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a00b5620 0x7fe9a00ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.237+0000 7fe9a4be0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a015b7d0 0x7fe9a00acf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.237+0000 7fe9a4be0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a006d230 con 0x7fe9a00b5620 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.237+0000 7fe9a4be0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00ceaf0 con 0x7fe9a015b7d0 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.237+0000 7fe9a4be0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00cf550 con 0x7fe9a006c2d0 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.237+0000 7fe997fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a00b5620 0x7fe9a00ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.237+0000 7fe997fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a00b5620 0x7fe9a00ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46032/0 (socket says 172.21.15.19:46032) 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.237+0000 7fe997fff640 1 -- 172.21.15.19:0/2478246609 learned_addr learned my addr 172.21.15.19:0/2478246609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe99e59c640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9a006c2d0 0x7fe9a0159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe99ed9d640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a015b7d0 0x7fe9a00acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe997fff640 1 -- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9a006c2d0 msgr2=0x7fe9a0159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe997fff640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9a006c2d0 0x7fe9a0159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe997fff640 1 -- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a015b7d0 msgr2=0x7fe9a00acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe997fff640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a015b7d0 0x7fe9a00acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe997fff640 1 -- 172.21.15.19:0/2478246609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00ab990 con 0x7fe9a00b5620 2024-04-25T10:07:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe997fff640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a00b5620 0x7fe9a00ab090 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7fe98800e9b0 tx=0x7fe98800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe996ffd640 1 -- 172.21.15.19:0/2478246609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988017070 con 0x7fe9a00b5620 2024-04-25T10:07:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe996ffd640 1 -- 172.21.15.19:0/2478246609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98800c9a0 con 0x7fe9a00b5620 2024-04-25T10:07:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe996ffd640 1 -- 172.21.15.19:0/2478246609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98800cd90 con 0x7fe9a00b5620 2024-04-25T10:07:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe9a4be0640 1 -- 172.21.15.19:0/2478246609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a00abc70 con 0x7fe9a00b5620 2024-04-25T10:07:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.238+0000 7fe9a4be0640 1 -- 172.21.15.19:0/2478246609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a00ab5d0 con 0x7fe9a00b5620 2024-04-25T10:07:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.239+0000 7fe9a4be0640 1 -- 172.21.15.19:0/2478246609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a0073a00 con 0x7fe9a00b5620 2024-04-25T10:07:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.240+0000 7fe996ffd640 1 -- 172.21.15.19:0/2478246609 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe98801a070 con 0x7fe9a00b5620 2024-04-25T10:07:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.240+0000 7fe996ffd640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe98006b4d0 0x7fe98006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.240+0000 7fe996ffd640 1 -- 172.21.15.19:0/2478246609 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe98808a4b0 con 0x7fe9a00b5620 2024-04-25T10:07:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.240+0000 7fe99e59c640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe98006b4d0 0x7fe98006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.241+0000 7fe99e59c640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe98006b4d0 0x7fe98006d990 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe990034000 tx=0x7fe99002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.244+0000 7fe996ffd640 1 -- 172.21.15.19:0/2478246609 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9880590c0 con 0x7fe9a00b5620 2024-04-25T10:07:03.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.460+0000 7fe9a4be0640 1 -- 172.21.15.19:0/2478246609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe9a0159c00 con 0x7fe9a00b5620 2024-04-25T10:07:03.466 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T10:07:03.461+0000 7fc4ede6f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:07:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.462+0000 7fe996ffd640 1 -- 172.21.15.19:0/2478246609 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v130) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe9880589c0 con 0x7fe9a00b5620 2024-04-25T10:07:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:07:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 -- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe98006b4d0 msgr2=0x7fe98006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe98006b4d0 0x7fe98006d990 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe990034000 tx=0x7fe99002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 -- 172.21.15.19:0/2478246609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a00b5620 msgr2=0x7fe9a00ab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a00b5620 0x7fe9a00ab090 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7fe98800e9b0 tx=0x7fe98800ee70 comp rx=0 tx=0).stop 2024-04-25T10:07:03.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 -- 172.21.15.19:0/2478246609 shutdown_connections 2024-04-25T10:07:03.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe98006b4d0 0x7fe98006d990 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9a015b7d0 0x7fe9a00acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9a00b5620 0x7fe9a00ab090 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 --2- 172.21.15.19:0/2478246609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9a006c2d0 0x7fe9a0159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 -- 172.21.15.19:0/2478246609 >> 172.21.15.19:0/2478246609 conn(0x7fe9a00d0800 msgr2=0x7fe9a0160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 -- 172.21.15.19:0/2478246609 shutdown_connections 2024-04-25T10:07:03.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.466+0000 7fe9a4be0640 1 -- 172.21.15.19:0/2478246609 wait complete. 2024-04-25T10:07:03.475 INFO:tasks.ceph.mds.c.smithi019.stderr: -2530> 2024-04-25T10:05:59.387+0000 7f99eaed3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:03.485 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:07:03.505 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T10:07:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.585+0000 7f0b4afe7640 1 Processor -- start 2024-04-25T10:07:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.586+0000 7f0b4afe7640 1 -- start start 2024-04-25T10:07:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.586+0000 7f0b4afe7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b440c6010 0x7f0b440ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.586+0000 7f0b4afe7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0b440cd520 0x7f0b440cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.586+0000 7f0b4afe7640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b440c6940 con 0x7f0b4415a9f0 2024-04-25T10:07:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.586+0000 7f0b4afe7640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b440b0020 con 0x7f0b440c6010 2024-04-25T10:07:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.586+0000 7f0b4afe7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b440cb5a0 con 0x7f0b440cd520 2024-04-25T10:07:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.587+0000 7f0b43fff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b440c6010 0x7f0b440ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.587+0000 7f0b48d83640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0b4415a9f0 0x7f0b440cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54386/0 (socket says 172.21.15.19:54386) 2024-04-25T10:07:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.587+0000 7f0b48d83640 1 -- 172.21.15.19:0/2079981999 learned_addr learned my addr 172.21.15.19:0/2079981999 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.587+0000 7f0b49584640 1 --2- 172.21.15.19:0/2079981999 >> v2:172.21.15.112:3300/0 conn(0x7f0b440cd520 0x7f0b440cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.587+0000 7f0b43fff640 1 -- 172.21.15.19:0/2079981999 >> v2:172.21.15.112:3300/0 conn(0x7f0b440cd520 msgr2=0x7f0b440cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.587+0000 7f0b43fff640 1 --2- 172.21.15.19:0/2079981999 >> v2:172.21.15.112:3300/0 conn(0x7f0b440cd520 0x7f0b440cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.587+0000 7f0b43fff640 1 -- 172.21.15.19:0/2079981999 >> v1:172.21.15.19:6789/0 conn(0x7f0b4415a9f0 legacy=0x7f0b440cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.587+0000 7f0b437fe640 1 -- 172.21.15.19:0/2079981999 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351795368 0 0) 0x7f0b3c001490 con 0x7f0b4415a9f0 2024-04-25T10:07:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.587+0000 7f0b43fff640 1 -- 172.21.15.19:0/2079981999 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4415add0 con 0x7f0b440c6010 2024-04-25T10:07:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b43fff640 1 --2- 172.21.15.19:0/2079981999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b440c6010 0x7f0b440ccfe0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f0b440afd90 tx=0x7f0b3802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2be0a320a29e0b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b437fe640 1 -- 172.21.15.19:0/2079981999 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38037070 con 0x7f0b440c6010 2024-04-25T10:07:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b437fe640 1 -- 172.21.15.19:0/2079981999 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b38032440 con 0x7f0b440c6010 2024-04-25T10:07:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b437fe640 1 -- 172.21.15.19:0/2079981999 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38032810 con 0x7f0b440c6010 2024-04-25T10:07:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b4afe7640 1 -- 172.21.15.19:0/2079981999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b440c6010 msgr2=0x7f0b440ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b4afe7640 1 --2- 172.21.15.19:0/2079981999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b440c6010 0x7f0b440ccfe0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f0b440afd90 tx=0x7f0b3802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b4afe7640 1 -- 172.21.15.19:0/2079981999 shutdown_connections 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b4afe7640 1 --2- 172.21.15.19:0/2079981999 >> v2:172.21.15.112:3300/0 conn(0x7f0b440cd520 0x7f0b440cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b4afe7640 1 --2- 172.21.15.19:0/2079981999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b440c6010 0x7f0b440ccfe0 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b4afe7640 1 -- 172.21.15.19:0/2079981999 >> 172.21.15.19:0/2079981999 conn(0x7f0b440b2390 msgr2=0x7f0b440b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b4afe7640 1 -- 172.21.15.19:0/2079981999 shutdown_connections 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.588+0000 7f0b4afe7640 1 -- 172.21.15.19:0/2079981999 wait complete. 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.589+0000 7f0b4afe7640 1 Processor -- start 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.590+0000 7f0b4afe7640 1 -- start start 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.590+0000 7f0b4afe7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b440c6010 0x7f0b440ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.590+0000 7f0b4afe7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b440cd520 0x7f0b440c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.590+0000 7f0b4afe7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b4415a9f0 0x7f0b440af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.590+0000 7f0b4afe7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b440cb230 con 0x7f0b440c6010 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.590+0000 7f0b4afe7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b440caf50 con 0x7f0b440cd520 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.590+0000 7f0b4afe7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b440cbf70 con 0x7f0b4415a9f0 2024-04-25T10:07:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b48d83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b440c6010 0x7f0b440ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b48d83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b440c6010 0x7f0b440ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46052/0 (socket says 172.21.15.19:46052) 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b48d83640 1 -- 172.21.15.19:0/4253375591 learned_addr learned my addr 172.21.15.19:0/4253375591 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b43fff640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b440cd520 0x7f0b440c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b49584640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b4415a9f0 0x7f0b440af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b48d83640 1 -- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b4415a9f0 msgr2=0x7f0b440af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b48d83640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b4415a9f0 0x7f0b440af3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b48d83640 1 -- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b440cd520 msgr2=0x7f0b440c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b48d83640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b440cd520 0x7f0b440c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b48d83640 1 -- 172.21.15.19:0/4253375591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b440adbe0 con 0x7f0b440c6010 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b49584640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b4415a9f0 0x7f0b440af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b48d83640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b440c6010 0x7f0b440ad600 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00c7c0 tx=0x7f0b3c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b417fa640 1 -- 172.21.15.19:0/4253375591 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c018070 con 0x7f0b440c6010 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b417fa640 1 -- 172.21.15.19:0/4253375591 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c00a640 con 0x7f0b440c6010 2024-04-25T10:07:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b4afe7640 1 -- 172.21.15.19:0/4253375591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b440adec0 con 0x7f0b440c6010 2024-04-25T10:07:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b417fa640 1 -- 172.21.15.19:0/4253375591 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00aa30 con 0x7f0b440c6010 2024-04-25T10:07:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.591+0000 7f0b4afe7640 1 -- 172.21.15.19:0/4253375591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b440ae480 con 0x7f0b440c6010 2024-04-25T10:07:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.592+0000 7f0b4afe7640 1 -- 172.21.15.19:0/4253375591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4406cc80 con 0x7f0b440c6010 2024-04-25T10:07:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.593+0000 7f0b417fa640 1 -- 172.21.15.19:0/4253375591 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b3c01b070 con 0x7f0b440c6010 2024-04-25T10:07:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.593+0000 7f0b417fa640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3006b500 0x7f0b3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.593+0000 7f0b417fa640 1 -- 172.21.15.19:0/4253375591 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f0b3c089ac0 con 0x7f0b440c6010 2024-04-25T10:07:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.593+0000 7f0b43fff640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3006b500 0x7f0b3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.594+0000 7f0b43fff640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3006b500 0x7f0b3006d9c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0b38034000 tx=0x7f0b3802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.598+0000 7f0b417fa640 1 -- 172.21.15.19:0/4253375591 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b440c6940 con 0x7f0b440c6010 2024-04-25T10:07:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:03.817+0000 7f0b4afe7640 1 -- 172.21.15.19:0/4253375591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0b44073a00 con 0x7f0b440c6010 2024-04-25T10:07:04.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.464+0000 7f0b417fa640 1 -- 172.21.15.19:0/4253375591 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v131) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0b3c057fb0 con 0x7f0b440c6010 2024-04-25T10:07:04.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.466+0000 7f0b4afe7640 1 -- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3006b500 msgr2=0x7f0b3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.466+0000 7f0b4afe7640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3006b500 0x7f0b3006d9c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0b38034000 tx=0x7f0b3802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.467+0000 7f0b4afe7640 1 -- 172.21.15.19:0/4253375591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b440c6010 msgr2=0x7f0b440ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.467+0000 7f0b4afe7640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b440c6010 0x7f0b440ad600 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00c7c0 tx=0x7f0b3c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:07:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.467+0000 7f0b4afe7640 1 -- 172.21.15.19:0/4253375591 shutdown_connections 2024-04-25T10:07:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.467+0000 7f0b4afe7640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0b3006b500 0x7f0b3006d9c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.467+0000 7f0b4afe7640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b4415a9f0 0x7f0b440af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.467+0000 7f0b4afe7640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b440cd520 0x7f0b440c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.467+0000 7f0b4afe7640 1 --2- 172.21.15.19:0/4253375591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b440c6010 0x7f0b440ad600 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.467+0000 7f0b4afe7640 1 -- 172.21.15.19:0/4253375591 >> 172.21.15.19:0/4253375591 conn(0x7f0b440b2390 msgr2=0x7f0b4406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.467+0000 7f0b4afe7640 1 -- 172.21.15.19:0/4253375591 shutdown_connections 2024-04-25T10:07:04.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.467+0000 7f0b4afe7640 1 -- 172.21.15.19:0/4253375591 wait complete. 2024-04-25T10:07:04.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:07:04.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.632+0000 7fe6ce2ee640 1 Processor -- start 2024-04-25T10:07:04.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.632+0000 7fe6ce2ee640 1 -- start start 2024-04-25T10:07:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6ce2ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c8161890 0x7fe6c80b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6ce2ee640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c80cf1d0 con 0x7fe6c8161240 2024-04-25T10:07:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6ce2ee640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c8161620 con 0x7fe6c80b54a0 2024-04-25T10:07:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6ce2ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c80b7120 con 0x7fe6c8161890 2024-04-25T10:07:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6c7fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe6c8161240 0x7fe6c80ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54388/0 (socket says 172.21.15.19:54388) 2024-04-25T10:07:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6c7fff640 1 -- 172.21.15.19:0/1633620446 learned_addr learned my addr 172.21.15.19:0/1633620446 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:04.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340534530 0 0) 0x7fe6c80cf1d0 con 0x7fe6c8161240 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6b4003520 con 0x7fe6c8161240 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6c77fe640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c8161890 0x7fe6c80b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3263777650 0 0) 0x7fe6b4003520 con 0x7fe6c8161240 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c8161890 msgr2=0x7fe6c80b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6c6ffd640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c8161890 0x7fe6c80b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 >> v1:172.21.15.112:6789/0 conn(0x7fe6c80b54a0 legacy=0x7fe6c80d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8161e00 con 0x7fe6c8161240 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.633+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fe6b0001670 con 0x7fe6c8161240 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 >> v1:172.21.15.19:6789/0 conn(0x7fe6c8161240 legacy=0x7fe6c80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c6ffd640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c8161890 0x7fe6b40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c6ffd640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b4003f60 0x7fe6b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c6ffd640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4004d40 0x7fe6b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b400ba70 con 0x7fe6b4003f60 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b400bc20 con 0x7fe6b4004d40 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b400be20 con 0x7fe6c8161890 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c77fe640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b4003f60 0x7fe6b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c7fff640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c8161890 0x7fe6b40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6cc88b640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4004d40 0x7fe6b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c77fe640 1 -- 172.21.15.19:0/1633620446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c8161890 msgr2=0x7fe6b40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c77fe640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c8161890 0x7fe6b40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c77fe640 1 -- 172.21.15.19:0/1633620446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4004d40 msgr2=0x7fe6b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c77fe640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4004d40 0x7fe6b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.634+0000 7fe6c77fe640 1 -- 172.21.15.19:0/1633620446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b4003520 con 0x7fe6b4003f60 2024-04-25T10:07:04.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.635+0000 7fe6c77fe640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b4003f60 0x7fe6b4004800 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009ac0 tx=0x7fe6b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de24559bce530772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.635+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8036070 con 0x7fe6b4003f60 2024-04-25T10:07:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.635+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b8031400 con 0x7fe6b4003f60 2024-04-25T10:07:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.635+0000 7fe6c6ffd640 1 -- 172.21.15.19:0/1633620446 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8009690 con 0x7fe6b4003f60 2024-04-25T10:07:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.635+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/1633620446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b4003f60 msgr2=0x7fe6b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.635+0000 7fe6ce2ee640 1 --2- 172.21.15.19:0/1633620446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6b4003f60 0x7fe6b4004800 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009ac0 tx=0x7fe6b4005830 comp rx=0 tx=0).stop 2024-04-25T10:07:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.635+0000 7fe6cc88b640 1 -- 172.21.15.19:0/1633620446 reap_dead start 2024-04-25T10:07:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.635+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/1633620446 shutdown_connections 2024-04-25T10:07:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.635+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/1633620446 >> 172.21.15.19:0/1633620446 conn(0x7fe6c80d04a0 msgr2=0x7fe6c80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.635+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/1633620446 shutdown_connections 2024-04-25T10:07:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.635+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/1633620446 wait complete. 2024-04-25T10:07:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.637+0000 7fe6ce2ee640 1 Processor -- start 2024-04-25T10:07:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.637+0000 7fe6ce2ee640 1 -- start start 2024-04-25T10:07:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.637+0000 7fe6ce2ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4003f60 0x7fe6c815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6ce2ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c80cc5a0 0x7fe6c80c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6ce2ee640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c80c7f40 0x7fe6c80c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6ce2ee640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c81626a0 con 0x7fe6c80cc5a0 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6ce2ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c81623c0 con 0x7fe6b4003f60 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6ce2ee640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c80d15f0 con 0x7fe6c80c7f40 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c80cc5a0 0x7fe6c80c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c80cc5a0 0x7fe6c80c7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46082/0 (socket says 172.21.15.19:46082) 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c77fe640 1 -- 172.21.15.19:0/4287972784 learned_addr learned my addr 172.21.15.19:0/4287972784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6cc88b640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c80c7f40 0x7fe6c80c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c7fff640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4003f60 0x7fe6c815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c77fe640 1 -- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c80c7f40 msgr2=0x7fe6c80c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c77fe640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c80c7f40 0x7fe6c80c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c77fe640 1 -- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4003f60 msgr2=0x7fe6c815f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c77fe640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4003f60 0x7fe6c815f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c77fe640 1 -- 172.21.15.19:0/4287972784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c815fd00 con 0x7fe6c80cc5a0 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c77fe640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c80cc5a0 0x7fe6c80c7630 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8034040 tx=0x7fe6b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c4ff9640 1 -- 172.21.15.19:0/4287972784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8040020 con 0x7fe6c80cc5a0 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c4ff9640 1 -- 172.21.15.19:0/4287972784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b8003a40 con 0x7fe6c80cc5a0 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/4287972784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c80c7b70 con 0x7fe6c80cc5a0 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6c4ff9640 1 -- 172.21.15.19:0/4287972784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8036040 con 0x7fe6c80cc5a0 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.638+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/4287972784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c80ccb60 con 0x7fe6c80cc5a0 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.639+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/4287972784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c8073a00 con 0x7fe6c80cc5a0 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.640+0000 7fe6c4ff9640 1 -- 172.21.15.19:0/4287972784 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6b8043020 con 0x7fe6c80cc5a0 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.640+0000 7fe6c4ff9640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6a806b4d0 0x7fe6a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.640+0000 7fe6c4ff9640 1 -- 172.21.15.19:0/4287972784 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe6b80aa5e0 con 0x7fe6c80cc5a0 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.640+0000 7fe6c7fff640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6a806b4d0 0x7fe6a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.641+0000 7fe6c7fff640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6a806b4d0 0x7fe6a806d990 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0001eb0 tx=0x7fe6b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.645+0000 7fe6c4ff9640 1 -- 172.21.15.19:0/4287972784 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe6b804a070 con 0x7fe6c80cc5a0 2024-04-25T10:07:04.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:04.828+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/4287972784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe6c806d240 con 0x7fe6c80cc5a0 2024-04-25T10:07:05.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.521+0000 7fe6c4ff9640 1 -- 172.21.15.19:0/4287972784 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v194) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe6b8079240 con 0x7fe6c80cc5a0 2024-04-25T10:07:05.527 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:07:05.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.525+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6a806b4d0 msgr2=0x7fe6a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.525+0000 7fe6ce2ee640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6a806b4d0 0x7fe6a806d990 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0001eb0 tx=0x7fe6b0009040 comp rx=0 tx=0).stop 2024-04-25T10:07:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.525+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/4287972784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c80cc5a0 msgr2=0x7fe6c80c7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.525+0000 7fe6ce2ee640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c80cc5a0 0x7fe6c80c7630 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8034040 tx=0x7fe6b8004730 comp rx=0 tx=0).stop 2024-04-25T10:07:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.525+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/4287972784 shutdown_connections 2024-04-25T10:07:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.525+0000 7fe6ce2ee640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe6a806b4d0 0x7fe6a806d990 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.525+0000 7fe6ce2ee640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6c80c7f40 0x7fe6c80c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.525+0000 7fe6ce2ee640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c80cc5a0 0x7fe6c80c7630 secure :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8034040 tx=0x7fe6b8004730 comp rx=0 tx=0).stop 2024-04-25T10:07:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.525+0000 7fe6ce2ee640 1 --2- 172.21.15.19:0/4287972784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b4003f60 0x7fe6c815f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.525+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/4287972784 >> 172.21.15.19:0/4287972784 conn(0x7fe6c80d04a0 msgr2=0x7fe6c815ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.526+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/4287972784 shutdown_connections 2024-04-25T10:07:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.526+0000 7fe6ce2ee640 1 -- 172.21.15.19:0/4287972784 wait complete. 2024-04-25T10:07:05.545 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:07:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.650+0000 7f934c7e0640 1 Processor -- start 2024-04-25T10:07:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.650+0000 7f934c7e0640 1 -- start start 2024-04-25T10:07:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.651+0000 7f934c7e0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9344145480 0x7f93440afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.651+0000 7f934c7e0640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f93440b52e0 0x7f93440b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.651+0000 7f934c7e0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93440b57e0 con 0x7f934406abe0 2024-04-25T10:07:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.651+0000 7f934c7e0640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93440ad8a0 con 0x7f93440b52e0 2024-04-25T10:07:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.651+0000 7f934c7e0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93440b3590 con 0x7f9344145480 2024-04-25T10:07:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.651+0000 7f9349d7b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f93440b52e0 0x7f93440b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.651+0000 7f9349d7b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f93440b52e0 0x7f93440b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46098/0 (socket says 172.21.15.19:46098) 2024-04-25T10:07:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.651+0000 7f9349d7b640 1 -- 172.21.15.19:0/4050106246 learned_addr learned my addr 172.21.15.19:0/4050106246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.652+0000 7f934a57c640 1 --2- 172.21.15.19:0/4050106246 >> v2:172.21.15.112:3300/0 conn(0x7f9344145480 0x7f93440afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.652+0000 7f9349d7b640 1 -- 172.21.15.19:0/4050106246 >> v2:172.21.15.112:3300/0 conn(0x7f9344145480 msgr2=0x7f93440afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.652+0000 7f9349d7b640 1 --2- 172.21.15.19:0/4050106246 >> v2:172.21.15.112:3300/0 conn(0x7f9344145480 0x7f93440afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.652+0000 7f9349d7b640 1 -- 172.21.15.19:0/4050106246 >> v1:172.21.15.112:6789/0 conn(0x7f934406abe0 legacy=0x7f934406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.652+0000 7f9349d7b640 1 -- 172.21.15.19:0/4050106246 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440b3870 con 0x7f93440b52e0 2024-04-25T10:07:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.652+0000 7f9349d7b640 1 --2- 172.21.15.19:0/4050106246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93440b52e0 0x7f93440b4350 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f934406cf70 tx=0x7f933802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa02228adc4eefc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.653+0000 7f934957a640 1 -- 172.21.15.19:0/4050106246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338037070 con 0x7f93440b52e0 2024-04-25T10:07:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.653+0000 7f934957a640 1 -- 172.21.15.19:0/4050106246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9338032440 con 0x7f93440b52e0 2024-04-25T10:07:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.653+0000 7f934957a640 1 -- 172.21.15.19:0/4050106246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338032810 con 0x7f93440b52e0 2024-04-25T10:07:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.653+0000 7f934c7e0640 1 -- 172.21.15.19:0/4050106246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93440b52e0 msgr2=0x7f93440b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.653+0000 7f934c7e0640 1 --2- 172.21.15.19:0/4050106246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93440b52e0 0x7f93440b4350 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f934406cf70 tx=0x7f933802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.653+0000 7f934c7e0640 1 -- 172.21.15.19:0/4050106246 shutdown_connections 2024-04-25T10:07:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.653+0000 7f934c7e0640 1 --2- 172.21.15.19:0/4050106246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93440b52e0 0x7f93440b4350 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.653+0000 7f934c7e0640 1 --2- 172.21.15.19:0/4050106246 >> v2:172.21.15.112:3300/0 conn(0x7f9344145480 0x7f93440afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.653+0000 7f934c7e0640 1 -- 172.21.15.19:0/4050106246 >> 172.21.15.19:0/4050106246 conn(0x7f93440b5af0 msgr2=0x7f93440b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.653+0000 7f934c7e0640 1 -- 172.21.15.19:0/4050106246 shutdown_connections 2024-04-25T10:07:05.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.653+0000 7f934c7e0640 1 -- 172.21.15.19:0/4050106246 wait complete. 2024-04-25T10:07:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.654+0000 7f934c7e0640 1 Processor -- start 2024-04-25T10:07:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.654+0000 7f934c7e0640 1 -- start start 2024-04-25T10:07:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.655+0000 7f934c7e0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f934406abe0 0x7f93441390c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.655+0000 7f934c7e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93440b52e0 0x7f93441e6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.655+0000 7f934c7e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9344145480 0x7f93441e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.655+0000 7f934c7e0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440aff90 con 0x7f9344145480 2024-04-25T10:07:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.655+0000 7f934c7e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440b3e30 con 0x7f93440b52e0 2024-04-25T10:07:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.655+0000 7f934c7e0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93440b4890 con 0x7f934406abe0 2024-04-25T10:07:05.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.655+0000 7f934ad7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9344145480 0x7f93441e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.655+0000 7f934ad7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9344145480 0x7f93441e72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46100/0 (socket says 172.21.15.19:46100) 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.655+0000 7f934ad7d640 1 -- 172.21.15.19:0/1385241925 learned_addr learned my addr 172.21.15.19:0/1385241925 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.655+0000 7f9349d7b640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93440b52e0 0x7f93441e6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f934a57c640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f934406abe0 0x7f93441390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f934ad7d640 1 -- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f934406abe0 msgr2=0x7f93441390c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f934ad7d640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f934406abe0 0x7f93441390c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f934ad7d640 1 -- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93440b52e0 msgr2=0x7f93441e6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f934ad7d640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93440b52e0 0x7f93441e6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f934ad7d640 1 -- 172.21.15.19:0/1385241925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93441e1400 con 0x7f9344145480 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f934a57c640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f934406abe0 0x7f93441390c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f934ad7d640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9344145480 0x7f93441e72c0 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f933400b970 tx=0x7f933400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f9342ffd640 1 -- 172.21.15.19:0/1385241925 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334019070 con 0x7f9344145480 2024-04-25T10:07:05.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f9342ffd640 1 -- 172.21.15.19:0/1385241925 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933400cb40 con 0x7f9344145480 2024-04-25T10:07:05.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f9342ffd640 1 -- 172.21.15.19:0/1385241925 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334014440 con 0x7f9344145480 2024-04-25T10:07:05.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f934c7e0640 1 -- 172.21.15.19:0/1385241925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93441e16e0 con 0x7f9344145480 2024-04-25T10:07:05.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.656+0000 7f934c7e0640 1 -- 172.21.15.19:0/1385241925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93441e1ca0 con 0x7f9344145480 2024-04-25T10:07:05.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.657+0000 7f934c7e0640 1 -- 172.21.15.19:0/1385241925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9344073a00 con 0x7f9344145480 2024-04-25T10:07:05.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.657+0000 7f9342ffd640 1 -- 172.21.15.19:0/1385241925 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f933401c070 con 0x7f9344145480 2024-04-25T10:07:05.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.658+0000 7f9342ffd640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933006b4d0 0x7f933006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.658+0000 7f9342ffd640 1 -- 172.21.15.19:0/1385241925 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f9334089d90 con 0x7f9344145480 2024-04-25T10:07:05.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.658+0000 7f934a57c640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933006b4d0 0x7f933006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.659+0000 7f934a57c640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933006b4d0 0x7f933006d990 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f933c006fd0 tx=0x7f933c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.662+0000 7f9342ffd640 1 -- 172.21.15.19:0/1385241925 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93340585d0 con 0x7f9344145480 2024-04-25T10:07:05.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:05.845+0000 7f934c7e0640 1 -- 172.21.15.19:0/1385241925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f93441449a0 con 0x7f9344145480 2024-04-25T10:07:06.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.523+0000 7f9342ffd640 1 -- 172.21.15.19:0/1385241925 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v195) v1 ==== 169+0+0 (secure 0 0 0) 0x7f933405c670 con 0x7f9344145480 2024-04-25T10:07:06.528 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T10:07:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 -- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933006b4d0 msgr2=0x7f933006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933006b4d0 0x7f933006d990 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f933c006fd0 tx=0x7f933c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 -- 172.21.15.19:0/1385241925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9344145480 msgr2=0x7f93441e72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9344145480 0x7f93441e72c0 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f933400b970 tx=0x7f933400be30 comp rx=0 tx=0).stop 2024-04-25T10:07:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 -- 172.21.15.19:0/1385241925 shutdown_connections 2024-04-25T10:07:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f933006b4d0 0x7f933006d990 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9344145480 0x7f93441e72c0 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93440b52e0 0x7f93441e6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 --2- 172.21.15.19:0/1385241925 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f934406abe0 0x7f93441390c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 -- 172.21.15.19:0/1385241925 >> 172.21.15.19:0/1385241925 conn(0x7f93440b5af0 msgr2=0x7f93440b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 -- 172.21.15.19:0/1385241925 shutdown_connections 2024-04-25T10:07:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.525+0000 7f934c7e0640 1 -- 172.21.15.19:0/1385241925 wait complete. 2024-04-25T10:07:06.546 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_hack_allow_loading_invalid_metadata 2024-04-25T10:07:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.654+0000 7f76283f4640 1 Processor -- start 2024-04-25T10:07:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.654+0000 7f76283f4640 1 -- start start 2024-04-25T10:07:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.655+0000 7f76283f4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f76201616f0 0x7f7620161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.655+0000 7f76283f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76200ce0a0 0x7f76200ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.655+0000 7f76283f4640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762006c8d0 con 0x7f76201610f0 2024-04-25T10:07:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.655+0000 7f76283f4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76201614d0 con 0x7f76201616f0 2024-04-25T10:07:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.655+0000 7f76283f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76200c6960 con 0x7f76200ce0a0 2024-04-25T10:07:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.655+0000 7f762598f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f76201616f0 0x7f7620161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.655+0000 7f7626190640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f76201610f0 0x7f762006c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54402/0 (socket says 172.21.15.19:54402) 2024-04-25T10:07:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.655+0000 7f7626190640 1 -- 172.21.15.19:0/3063879877 learned_addr learned my addr 172.21.15.19:0/3063879877 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.656+0000 7f7626991640 1 --2- 172.21.15.19:0/3063879877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76200ce0a0 0x7f76200ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.656+0000 7f762598f640 1 -- 172.21.15.19:0/3063879877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76200ce0a0 msgr2=0x7f76200ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.656+0000 7f762598f640 1 --2- 172.21.15.19:0/3063879877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76200ce0a0 0x7f76200ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.656+0000 7f762598f640 1 -- 172.21.15.19:0/3063879877 >> v1:172.21.15.19:6789/0 conn(0x7f76201610f0 legacy=0x7f762006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.656+0000 7f762518e640 1 -- 172.21.15.19:0/3063879877 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992022219 0 0) 0x7f762006c8d0 con 0x7f76201610f0 2024-04-25T10:07:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.656+0000 7f762598f640 1 -- 172.21.15.19:0/3063879877 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7620162260 con 0x7f76201616f0 2024-04-25T10:07:06.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f762598f640 1 --2- 172.21.15.19:0/3063879877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76201616f0 0x7f7620161b50 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f76140097a0 tx=0x7f761402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c5bd224ef41d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f762518e640 1 -- 172.21.15.19:0/3063879877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614037070 con 0x7f76201616f0 2024-04-25T10:07:06.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f762518e640 1 -- 172.21.15.19:0/3063879877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7614032440 con 0x7f76201616f0 2024-04-25T10:07:06.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f762518e640 1 -- 172.21.15.19:0/3063879877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614032810 con 0x7f76201616f0 2024-04-25T10:07:06.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f76283f4640 1 -- 172.21.15.19:0/3063879877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76201616f0 msgr2=0x7f7620161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f76283f4640 1 --2- 172.21.15.19:0/3063879877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76201616f0 0x7f7620161b50 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f76140097a0 tx=0x7f761402a180 comp rx=0 tx=0).stop 2024-04-25T10:07:06.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f76283f4640 1 -- 172.21.15.19:0/3063879877 shutdown_connections 2024-04-25T10:07:06.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f76283f4640 1 --2- 172.21.15.19:0/3063879877 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76200ce0a0 0x7f76200ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f76283f4640 1 --2- 172.21.15.19:0/3063879877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76201616f0 0x7f7620161b50 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f76283f4640 1 -- 172.21.15.19:0/3063879877 >> 172.21.15.19:0/3063879877 conn(0x7f762006f500 msgr2=0x7f762006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f76283f4640 1 -- 172.21.15.19:0/3063879877 shutdown_connections 2024-04-25T10:07:06.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.657+0000 7f76283f4640 1 -- 172.21.15.19:0/3063879877 wait complete. 2024-04-25T10:07:06.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.658+0000 7f76283f4640 1 Processor -- start 2024-04-25T10:07:06.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.659+0000 7f76283f4640 1 -- start start 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.659+0000 7f76283f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76200ce0a0 0x7f7620145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.659+0000 7f76283f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76201610f0 0x7f762015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.659+0000 7f76283f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76201616f0 0x7f76201f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.659+0000 7f76283f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76200cfce0 con 0x7f76200ce0a0 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.659+0000 7f76283f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76200cfa00 con 0x7f76201616f0 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.659+0000 7f76283f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76200cf690 con 0x7f76201610f0 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.659+0000 7f7626190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76200ce0a0 0x7f7620145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.659+0000 7f7626190640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76200ce0a0 0x7f7620145000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46120/0 (socket says 172.21.15.19:46120) 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.659+0000 7f7626190640 1 -- 172.21.15.19:0/2300836642 learned_addr learned my addr 172.21.15.19:0/2300836642 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f762598f640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76201610f0 0x7f762015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f7626991640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76201616f0 0x7f76201f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f7626190640 1 -- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76201610f0 msgr2=0x7f762015da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f7626190640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76201610f0 0x7f762015da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f7626190640 1 -- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76201616f0 msgr2=0x7f76201f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f7626190640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76201616f0 0x7f76201f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f7626190640 1 -- 172.21.15.19:0/2300836642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7620145540 con 0x7f76200ce0a0 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f7626991640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76201616f0 0x7f76201f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f7626190640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76200ce0a0 0x7f7620145000 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f761800c500 tx=0x7f761800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f761effd640 1 -- 172.21.15.19:0/2300836642 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618018070 con 0x7f76200ce0a0 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f761effd640 1 -- 172.21.15.19:0/2300836642 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618007580 con 0x7f76200ce0a0 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f76283f4640 1 -- 172.21.15.19:0/2300836642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7620158c30 con 0x7f76200ce0a0 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f761effd640 1 -- 172.21.15.19:0/2300836642 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618004a20 con 0x7f76200ce0a0 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.660+0000 7f76283f4640 1 -- 172.21.15.19:0/2300836642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7620158ed0 con 0x7f76200ce0a0 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.661+0000 7f76283f4640 1 -- 172.21.15.19:0/2300836642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7620073a00 con 0x7f76200ce0a0 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.661+0000 7f761effd640 1 -- 172.21.15.19:0/2300836642 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f761801b070 con 0x7f76200ce0a0 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.662+0000 7f761effd640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f760c06b500 0x7f760c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.662+0000 7f761effd640 1 -- 172.21.15.19:0/2300836642 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7618089310 con 0x7f76200ce0a0 2024-04-25T10:07:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.662+0000 7f762598f640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f760c06b500 0x7f760c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.663+0000 7f762598f640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f760c06b500 0x7f760c06d9c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7614034000 tx=0x7f761402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.666+0000 7f761effd640 1 -- 172.21.15.19:0/2300836642 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7618058340 con 0x7f76200ce0a0 2024-04-25T10:07:06.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.846+0000 7f76283f4640 1 -- 172.21.15.19:0/2300836642 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"} v 0) v1 -- 0x7f7620156530 con 0x7f76200ce0a0 2024-04-25T10:07:06.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.849+0000 7f761effd640 1 -- 172.21.15.19:0/2300836642 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"}]=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7618057c40 con 0x7f76200ce0a0 2024-04-25T10:07:06.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.851+0000 7f76283f4640 1 -- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f760c06b500 msgr2=0x7f760c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.851+0000 7f76283f4640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f760c06b500 0x7f760c06d9c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7614034000 tx=0x7f761402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.851+0000 7f76283f4640 1 -- 172.21.15.19:0/2300836642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76200ce0a0 msgr2=0x7f7620145000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.851+0000 7f76283f4640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76200ce0a0 0x7f7620145000 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f761800c500 tx=0x7f761800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.851+0000 7f76283f4640 1 -- 172.21.15.19:0/2300836642 shutdown_connections 2024-04-25T10:07:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.851+0000 7f76283f4640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f760c06b500 0x7f760c06d9c0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.851+0000 7f76283f4640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76201616f0 0x7f76201f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.852+0000 7f76283f4640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76201610f0 0x7f762015da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.852+0000 7f76283f4640 1 --2- 172.21.15.19:0/2300836642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76200ce0a0 0x7f7620145000 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.852+0000 7f76283f4640 1 -- 172.21.15.19:0/2300836642 >> 172.21.15.19:0/2300836642 conn(0x7f762006f500 msgr2=0x7f762006f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.852+0000 7f76283f4640 1 -- 172.21.15.19:0/2300836642 shutdown_connections 2024-04-25T10:07:06.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.852+0000 7f76283f4640 1 -- 172.21.15.19:0/2300836642 wait complete. 2024-04-25T10:07:06.868 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:07:06.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.965+0000 7fabce08f640 1 Processor -- start 2024-04-25T10:07:06.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.965+0000 7fabce08f640 1 -- start start 2024-04-25T10:07:06.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.966+0000 7fabce08f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc815dac0 0x7fabc80d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.966+0000 7fabce08f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc80d2420 con 0x7fabc80d0ab0 2024-04-25T10:07:06.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.966+0000 7fabce08f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc80d2700 con 0x7fabc80d1f30 2024-04-25T10:07:06.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.966+0000 7fabce08f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80d3030 con 0x7fabc815dac0 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.966+0000 7fabc7fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fabc80d0ab0 0x7fabc80b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54414/0 (socket says 172.21.15.19:54414) 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.966+0000 7fabc7fff640 1 -- 172.21.15.19:0/424983529 learned_addr learned my addr 172.21.15.19:0/424983529 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.966+0000 7fabc77fe640 1 --2- 172.21.15.19:0/424983529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc815dac0 0x7fabc80d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.967+0000 7fabc67fc640 1 -- 172.21.15.19:0/424983529 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991181192 0 0) 0x7fabc80d2420 con 0x7fabc80d0ab0 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.967+0000 7fabc67fc640 1 -- 172.21.15.19:0/424983529 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb0003520 con 0x7fabc80d0ab0 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.967+0000 7fabc67fc640 1 -- 172.21.15.19:0/424983529 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097403276 0 0) 0x7fabc80d2700 con 0x7fabc80d1f30 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.967+0000 7fabc67fc640 1 -- 172.21.15.19:0/424983529 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc80d2420 con 0x7fabc80d1f30 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.967+0000 7fabc67fc640 1 -- 172.21.15.19:0/424983529 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2791459064 0 0) 0x7fabb0003520 con 0x7fabc80d0ab0 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.967+0000 7fabc77fe640 1 -- 172.21.15.19:0/424983529 >> v1:172.21.15.112:6789/0 conn(0x7fabc80d1f30 legacy=0x7fabc80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.967+0000 7fabc77fe640 1 -- 172.21.15.19:0/424983529 >> v1:172.21.15.19:6789/0 conn(0x7fabc80d0ab0 legacy=0x7fabc80b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.967+0000 7fabc77fe640 1 -- 172.21.15.19:0/424983529 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc80d29e0 con 0x7fabc815dac0 2024-04-25T10:07:06.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.968+0000 7fabc77fe640 1 --2- 172.21.15.19:0/424983529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc815dac0 0x7fabc80d19f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fabc80d31b0 tx=0x7fabb802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13627af2ff96cac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.968+0000 7fabc67fc640 1 -- 172.21.15.19:0/424983529 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8037070 con 0x7fabc815dac0 2024-04-25T10:07:06.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.968+0000 7fabc67fc640 1 -- 172.21.15.19:0/424983529 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb802aca0 con 0x7fabc815dac0 2024-04-25T10:07:06.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.968+0000 7fabc67fc640 1 -- 172.21.15.19:0/424983529 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00019f0 con 0x7fabc815dac0 2024-04-25T10:07:06.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.968+0000 7fabce08f640 1 -- 172.21.15.19:0/424983529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc815dac0 msgr2=0x7fabc80d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.968+0000 7fabce08f640 1 --2- 172.21.15.19:0/424983529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc815dac0 0x7fabc80d19f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fabc80d31b0 tx=0x7fabb802a210 comp rx=0 tx=0).stop 2024-04-25T10:07:06.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.968+0000 7fabce08f640 1 -- 172.21.15.19:0/424983529 shutdown_connections 2024-04-25T10:07:06.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.968+0000 7fabce08f640 1 --2- 172.21.15.19:0/424983529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc815dac0 0x7fabc80d19f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.968+0000 7fabce08f640 1 -- 172.21.15.19:0/424983529 >> 172.21.15.19:0/424983529 conn(0x7fabc8061c30 msgr2=0x7fabc806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.968+0000 7fabce08f640 1 -- 172.21.15.19:0/424983529 shutdown_connections 2024-04-25T10:07:06.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.968+0000 7fabce08f640 1 -- 172.21.15.19:0/424983529 wait complete. 2024-04-25T10:07:06.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.970+0000 7fabce08f640 1 Processor -- start 2024-04-25T10:07:06.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.970+0000 7fabce08f640 1 -- start start 2024-04-25T10:07:06.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.970+0000 7fabce08f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc80d0ab0 0x7fabc815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.970+0000 7fabce08f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc80d1f30 0x7fabc80cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.970+0000 7fabce08f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabc815dac0 0x7fabc81527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.970+0000 7fabce08f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b97d0 con 0x7fabc815dac0 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.970+0000 7fabce08f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc815dea0 con 0x7fabc80d0ab0 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.970+0000 7fabce08f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b9950 con 0x7fabc80d1f30 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabc815dac0 0x7fabc81527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabc815dac0 0x7fabc81527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46130/0 (socket says 172.21.15.19:46130) 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc7fff640 1 -- 172.21.15.19:0/146866391 learned_addr learned my addr 172.21.15.19:0/146866391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc77fe640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc80d0ab0 0x7fabc815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc6ffd640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc80d1f30 0x7fabc80cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc7fff640 1 -- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc80d1f30 msgr2=0x7fabc80cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc7fff640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc80d1f30 0x7fabc80cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc7fff640 1 -- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc80d0ab0 msgr2=0x7fabc815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc7fff640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc80d0ab0 0x7fabc815ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc7fff640 1 -- 172.21.15.19:0/146866391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc80cb9d0 con 0x7fabc815dac0 2024-04-25T10:07:06.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc77fe640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc80d0ab0 0x7fabc815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:06.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc6ffd640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc80d1f30 0x7fabc80cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:06.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabc7fff640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabc815dac0 0x7fabc81527e0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7fabc000cc90 tx=0x7fabc000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabaffff640 1 -- 172.21.15.19:0/146866391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0017070 con 0x7fabc815dac0 2024-04-25T10:07:06.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabaffff640 1 -- 172.21.15.19:0/146866391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc0012400 con 0x7fabc815dac0 2024-04-25T10:07:06.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabce08f640 1 -- 172.21.15.19:0/146866391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc815e190 con 0x7fabc815dac0 2024-04-25T10:07:06.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabaffff640 1 -- 172.21.15.19:0/146866391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00127d0 con 0x7fabc815dac0 2024-04-25T10:07:06.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.971+0000 7fabce08f640 1 -- 172.21.15.19:0/146866391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc815e750 con 0x7fabc815dac0 2024-04-25T10:07:06.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.972+0000 7fabce08f640 1 -- 172.21.15.19:0/146866391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc8073a00 con 0x7fabc815dac0 2024-04-25T10:07:06.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.973+0000 7fabaffff640 1 -- 172.21.15.19:0/146866391 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabc001a070 con 0x7fabc815dac0 2024-04-25T10:07:06.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.973+0000 7fabaffff640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabb406b500 0x7fabb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.973+0000 7fabaffff640 1 -- 172.21.15.19:0/146866391 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fabc008a040 con 0x7fabc815dac0 2024-04-25T10:07:06.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.973+0000 7fabc77fe640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabb406b500 0x7fabb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.974+0000 7fabc77fe640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabb406b500 0x7fabb406d9c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fabb8034000 tx=0x7fabb802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:06.977+0000 7fabaffff640 1 -- 172.21.15.19:0/146866391 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fabc80cb9d0 con 0x7fabc815dac0 2024-04-25T10:07:07.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.159+0000 7fabce08f640 1 -- 172.21.15.19:0/146866391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fabc80b8b80 con 0x7fabc815dac0 2024-04-25T10:07:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.163+0000 7fabaffff640 1 -- 172.21.15.19:0/146866391 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7fabc0058fe0 con 0x7fabc815dac0 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 -- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabb406b500 msgr2=0x7fabb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabb406b500 0x7fabb406d9c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fabb8034000 tx=0x7fabb802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 -- 172.21.15.19:0/146866391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabc815dac0 msgr2=0x7fabc81527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabc815dac0 0x7fabc81527e0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7fabc000cc90 tx=0x7fabc000a430 comp rx=0 tx=0).stop 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 -- 172.21.15.19:0/146866391 shutdown_connections 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fabb406b500 0x7fabb406d9c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabc815dac0 0x7fabc81527e0 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fabc80d1f30 0x7fabc80cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 --2- 172.21.15.19:0/146866391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc80d0ab0 0x7fabc815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 -- 172.21.15.19:0/146866391 >> 172.21.15.19:0/146866391 conn(0x7fabc8061c30 msgr2=0x7fabc80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 -- 172.21.15.19:0/146866391 shutdown_connections 2024-04-25T10:07:07.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.165+0000 7fabce08f640 1 -- 172.21.15.19:0/146866391 wait complete. 2024-04-25T10:07:07.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:07:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.277+0000 7ff6b5122640 1 Processor -- start 2024-04-25T10:07:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.277+0000 7ff6b5122640 1 -- start start 2024-04-25T10:07:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.277+0000 7ff6b5122640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6b015dc70 0x7ff6b00d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.277+0000 7ff6b5122640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b006c220 con 0x7ff6b006ce10 2024-04-25T10:07:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.277+0000 7ff6b5122640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b00d0080 con 0x7ff6b009a450 2024-04-25T10:07:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.277+0000 7ff6b5122640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b00d0c90 con 0x7ff6b015dc70 2024-04-25T10:07:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.278+0000 7ff6aed9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6b015dc70 0x7ff6b00d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.278+0000 7ff6af59e640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff6b006ce10 0x7ff6b00cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54422/0 (socket says 172.21.15.19:54422) 2024-04-25T10:07:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.278+0000 7ff6af59e640 1 -- 172.21.15.19:0/430921521 learned_addr learned my addr 172.21.15.19:0/430921521 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.278+0000 7ff6aed9d640 1 -- 172.21.15.19:0/430921521 >> v1:172.21.15.112:6789/0 conn(0x7ff6b009a450 legacy=0x7ff6b006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.278+0000 7ff6aed9d640 1 -- 172.21.15.19:0/430921521 >> v1:172.21.15.19:6789/0 conn(0x7ff6b006ce10 legacy=0x7ff6b00cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.278+0000 7ff6aed9d640 1 -- 172.21.15.19:0/430921521 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b00d0360 con 0x7ff6b015dc70 2024-04-25T10:07:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.278+0000 7ff6ae59c640 1 -- 172.21.15.19:0/430921521 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716432935 0 0) 0x7ff6b006c220 con 0x7ff6b006ce10 2024-04-25T10:07:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.279+0000 7ff6aed9d640 1 --2- 172.21.15.19:0/430921521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b015dc70 0x7ff6b00d1120 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00097a0 tx=0x7ff6a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519e6cce297a37dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.279+0000 7ff6ae59c640 1 -- 172.21.15.19:0/430921521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0037070 con 0x7ff6b015dc70 2024-04-25T10:07:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.279+0000 7ff6ae59c640 1 -- 172.21.15.19:0/430921521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a0032440 con 0x7ff6b015dc70 2024-04-25T10:07:07.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.279+0000 7ff6ae59c640 1 -- 172.21.15.19:0/430921521 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0032810 con 0x7ff6b015dc70 2024-04-25T10:07:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.279+0000 7ff6b5122640 1 -- 172.21.15.19:0/430921521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b015dc70 msgr2=0x7ff6b00d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.279+0000 7ff6b5122640 1 --2- 172.21.15.19:0/430921521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b015dc70 0x7ff6b00d1120 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00097a0 tx=0x7ff6a002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.279+0000 7ff6b5122640 1 -- 172.21.15.19:0/430921521 shutdown_connections 2024-04-25T10:07:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.279+0000 7ff6b5122640 1 --2- 172.21.15.19:0/430921521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b015dc70 0x7ff6b00d1120 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.279+0000 7ff6b5122640 1 -- 172.21.15.19:0/430921521 >> 172.21.15.19:0/430921521 conn(0x7ff6b00b3270 msgr2=0x7ff6b00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.279+0000 7ff6b5122640 1 -- 172.21.15.19:0/430921521 shutdown_connections 2024-04-25T10:07:07.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.279+0000 7ff6b5122640 1 -- 172.21.15.19:0/430921521 wait complete. 2024-04-25T10:07:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.281+0000 7ff6b5122640 1 Processor -- start 2024-04-25T10:07:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.281+0000 7ff6b5122640 1 -- start start 2024-04-25T10:07:07.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.281+0000 7ff6b5122640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b006ce10 0x7ff6b015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.281+0000 7ff6b5122640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b009a450 0x7ff6b01588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.281+0000 7ff6b5122640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6b015dc70 0x7ff6b014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.281+0000 7ff6b5122640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b015e050 con 0x7ff6b009a450 2024-04-25T10:07:07.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.281+0000 7ff6b5122640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b00d0920 con 0x7ff6b006ce10 2024-04-25T10:07:07.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.281+0000 7ff6b5122640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b00cf9c0 con 0x7ff6b015dc70 2024-04-25T10:07:07.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.281+0000 7ff6a7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b009a450 0x7ff6b01588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6a7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b009a450 0x7ff6b01588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46134/0 (socket says 172.21.15.19:46134) 2024-04-25T10:07:07.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6a7fff640 1 -- 172.21.15.19:0/964569800 learned_addr learned my addr 172.21.15.19:0/964569800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6af59e640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6b015dc70 0x7ff6b014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6aed9d640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b006ce10 0x7ff6b015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6a7fff640 1 -- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6b015dc70 msgr2=0x7ff6b014b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6a7fff640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6b015dc70 0x7ff6b014b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6a7fff640 1 -- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b006ce10 msgr2=0x7ff6b015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6a7fff640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b006ce10 0x7ff6b015a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6a7fff640 1 -- 172.21.15.19:0/964569800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b015b960 con 0x7ff6b009a450 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6aed9d640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b006ce10 0x7ff6b015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6a7fff640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b009a450 0x7ff6b01588a0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c430 tx=0x7ff69800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6a77fe640 1 -- 172.21.15.19:0/964569800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698011790 con 0x7ff6b009a450 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6a77fe640 1 -- 172.21.15.19:0/964569800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff698017070 con 0x7ff6b009a450 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6b5122640 1 -- 172.21.15.19:0/964569800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b015ade0 con 0x7ff6b009a450 2024-04-25T10:07:07.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6a77fe640 1 -- 172.21.15.19:0/964569800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698011bb0 con 0x7ff6b009a450 2024-04-25T10:07:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.282+0000 7ff6b5122640 1 -- 172.21.15.19:0/964569800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b015b3a0 con 0x7ff6b009a450 2024-04-25T10:07:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.283+0000 7ff6b5122640 1 -- 172.21.15.19:0/964569800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b0073990 con 0x7ff6b009a450 2024-04-25T10:07:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.284+0000 7ff6a77fe640 1 -- 172.21.15.19:0/964569800 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff69801c460 con 0x7ff6b009a450 2024-04-25T10:07:07.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.284+0000 7ff6a77fe640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff69006b500 0x7ff69006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.284+0000 7ff6a77fe640 1 -- 172.21.15.19:0/964569800 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff698089f20 con 0x7ff6b009a450 2024-04-25T10:07:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.284+0000 7ff6aed9d640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff69006b500 0x7ff69006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.285+0000 7ff6aed9d640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff69006b500 0x7ff69006d9c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0034000 tx=0x7ff6a002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.288+0000 7ff6a77fe640 1 -- 172.21.15.19:0/964569800 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff698058ec0 con 0x7ff6b009a450 2024-04-25T10:07:07.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:07.542+0000 7ff6b5122640 1 -- 172.21.15.19:0/964569800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7ff6b00b40a0 con 0x7ff6b009a450 2024-04-25T10:07:08.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.534+0000 7ff6a77fe640 1 -- 172.21.15.19:0/964569800 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v678) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff6980587c0 con 0x7ff6b009a450 2024-04-25T10:07:08.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.537+0000 7ff6b5122640 1 -- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff69006b500 msgr2=0x7ff69006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.537+0000 7ff6b5122640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff69006b500 0x7ff69006d9c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0034000 tx=0x7ff6a002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:08.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.537+0000 7ff6b5122640 1 -- 172.21.15.19:0/964569800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b009a450 msgr2=0x7ff6b01588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.537+0000 7ff6b5122640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b009a450 0x7ff6b01588a0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c430 tx=0x7ff69800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.537+0000 7ff6b5122640 1 -- 172.21.15.19:0/964569800 shutdown_connections 2024-04-25T10:07:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.537+0000 7ff6b5122640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ff69006b500 0x7ff69006d9c0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.537+0000 7ff6b5122640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6b015dc70 0x7ff6b014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.537+0000 7ff6b5122640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b009a450 0x7ff6b01588a0 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.537+0000 7ff6b5122640 1 --2- 172.21.15.19:0/964569800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b006ce10 0x7ff6b015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.537+0000 7ff6b5122640 1 -- 172.21.15.19:0/964569800 >> 172.21.15.19:0/964569800 conn(0x7ff6b00b3270 msgr2=0x7ff6b00b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.537+0000 7ff6b5122640 1 -- 172.21.15.19:0/964569800 shutdown_connections 2024-04-25T10:07:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:08.538+0000 7ff6b5122640 1 -- 172.21.15.19:0/964569800 wait complete. 2024-04-25T10:07:08.556 INFO:tasks.cephfs_test_runner:test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:07:08.557 INFO:tasks.cephfs_test_runner: 2024-04-25T10:07:08.557 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:07:08.557 INFO:tasks.cephfs_test_runner:Ran 13 tests in 801.522s 2024-04-25T10:07:08.557 INFO:tasks.cephfs_test_runner: 2024-04-25T10:07:08.557 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:07:08.558 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:07:08.568 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T10:07:08.606 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T10:07:08.606 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:07:08.606 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:08.625 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:08.625 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T10:07:08.626 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:08.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13709 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:08.684 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:07:08.684 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:08.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:08.702 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T10:07:08.702 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:08.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14575 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497664k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:08.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:08.760 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T10:07:08.778 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:07:08.778 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:07:08.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:08.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:08.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:08.847 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:08.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:08.916 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T10:07:09.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:09.004 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T10:07:09.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:09.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T10:07:09.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:09.092 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-04-25T10:07:09.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:09.092 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T10:07:09.111 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:07:09.111 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:07:09.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:09.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:07:09.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:09.178 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:07:09.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:09.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T10:07:09.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:09.349 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T10:07:09.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:09.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T10:07:09.434 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-04-25T10:07:09.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:09.436 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:07:09.447 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:07:09.447 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.550+0000 7f5385a15640 1 Processor -- start 2024-04-25T10:07:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.550+0000 7f5385a15640 1 -- start start 2024-04-25T10:07:09.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f5385a15640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5380144750 0x7f538006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f5385a15640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f538006eb10 0x7f53800ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f5385a15640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f53800cee60 0x7f538015f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f5385a15640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5380144b30 con 0x7f538006eb10 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f5385a15640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53800712d0 con 0x7f53800cee60 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f5385a15640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53800715b0 con 0x7f5380144750 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f537e7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f538006eb10 0x7f53800ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f537e7fc640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f538006eb10 0x7f53800ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38786/0 (socket says 172.21.15.19:38786) 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f537e7fc640 1 -- 172.21.15.19:0/15481440 learned_addr learned my addr 172.21.15.19:0/15481440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f537effd640 1 --2- 172.21.15.19:0/15481440 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5380144750 0x7f538006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f537f7fe640 1 --2- 172.21.15.19:0/15481440 >> v2:172.21.15.112:3300/0 conn(0x7f53800cee60 0x7f538015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f537e7fc640 1 -- 172.21.15.19:0/15481440 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5380144750 msgr2=0x7f538006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f537e7fc640 1 --2- 172.21.15.19:0/15481440 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5380144750 0x7f538006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f537e7fc640 1 -- 172.21.15.19:0/15481440 >> v2:172.21.15.112:3300/0 conn(0x7f53800cee60 msgr2=0x7f538015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f537e7fc640 1 --2- 172.21.15.19:0/15481440 >> v2:172.21.15.112:3300/0 conn(0x7f53800cee60 0x7f538015f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f537e7fc640 1 -- 172.21.15.19:0/15481440 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5380071890 con 0x7f538006eb10 2024-04-25T10:07:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.551+0000 7f537f7fe640 1 --2- 172.21.15.19:0/15481440 >> v2:172.21.15.112:3300/0 conn(0x7f53800cee60 0x7f538015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.552+0000 7f537e7fc640 1 --2- 172.21.15.19:0/15481440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f538006eb10 0x7f53800ce920 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f53800b4aa0 tx=0x7f537402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9c57b75214aee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.552+0000 7f537dffb640 1 -- 172.21.15.19:0/15481440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374037070 con 0x7f538006eb10 2024-04-25T10:07:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.552+0000 7f537dffb640 1 -- 172.21.15.19:0/15481440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5374032440 con 0x7f538006eb10 2024-04-25T10:07:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.552+0000 7f537dffb640 1 -- 172.21.15.19:0/15481440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374032810 con 0x7f538006eb10 2024-04-25T10:07:09.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.552+0000 7f5385a15640 1 -- 172.21.15.19:0/15481440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f538006eb10 msgr2=0x7f53800ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.552+0000 7f5385a15640 1 --2- 172.21.15.19:0/15481440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f538006eb10 0x7f53800ce920 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f53800b4aa0 tx=0x7f537402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.553+0000 7f5385a15640 1 -- 172.21.15.19:0/15481440 shutdown_connections 2024-04-25T10:07:09.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.553+0000 7f5385a15640 1 --2- 172.21.15.19:0/15481440 >> v2:172.21.15.112:3300/0 conn(0x7f53800cee60 0x7f538015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.553+0000 7f5385a15640 1 --2- 172.21.15.19:0/15481440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f538006eb10 0x7f53800ce920 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.553+0000 7f5385a15640 1 --2- 172.21.15.19:0/15481440 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5380144750 0x7f538006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.553+0000 7f5385a15640 1 -- 172.21.15.19:0/15481440 >> 172.21.15.19:0/15481440 conn(0x7f538006abe0 msgr2=0x7f538006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.553+0000 7f5385a15640 1 -- 172.21.15.19:0/15481440 shutdown_connections 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.553+0000 7f5385a15640 1 -- 172.21.15.19:0/15481440 wait complete. 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.554+0000 7f5385a15640 1 Processor -- start 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.554+0000 7f5385a15640 1 -- start start 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f5385a15640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f538006eb10 0x7f5380151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f5385a15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53800cee60 0x7f5380152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f5385a15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5380144750 0x7f5380152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f5385a15640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5380145450 con 0x7f53800cee60 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f5385a15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538006f020 con 0x7f5380144750 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f5385a15640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53801455d0 con 0x7f538006eb10 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f537e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53800cee60 0x7f5380152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f537e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53800cee60 0x7f5380152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38798/0 (socket says 172.21.15.19:38798) 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f537e7fc640 1 -- 172.21.15.19:0/1891436104 learned_addr learned my addr 172.21.15.19:0/1891436104 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f537effd640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f538006eb10 0x7f5380151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f537f7fe640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5380144750 0x7f5380152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f537e7fc640 1 -- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f538006eb10 msgr2=0x7f5380151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f537e7fc640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f538006eb10 0x7f5380151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f537e7fc640 1 -- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5380144750 msgr2=0x7f5380152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f537e7fc640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5380144750 0x7f5380152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.555+0000 7f537e7fc640 1 -- 172.21.15.19:0/1891436104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5380153730 con 0x7f53800cee60 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.556+0000 7f537e7fc640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53800cee60 0x7f5380152440 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f537402a220 tx=0x7f53740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.556+0000 7f5367fff640 1 -- 172.21.15.19:0/1891436104 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374041020 con 0x7f53800cee60 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.556+0000 7f5367fff640 1 -- 172.21.15.19:0/1891436104 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53740306c0 con 0x7f53800cee60 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.556+0000 7f5385a15640 1 -- 172.21.15.19:0/1891436104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5380141a70 con 0x7f53800cee60 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.556+0000 7f5367fff640 1 -- 172.21.15.19:0/1891436104 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374037040 con 0x7f53800cee60 2024-04-25T10:07:09.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.556+0000 7f5385a15640 1 -- 172.21.15.19:0/1891436104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5380151980 con 0x7f53800cee60 2024-04-25T10:07:09.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.557+0000 7f5385a15640 1 -- 172.21.15.19:0/1891436104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5380073990 con 0x7f53800cee60 2024-04-25T10:07:09.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.557+0000 7f5367fff640 1 -- 172.21.15.19:0/1891436104 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5374044020 con 0x7f53800cee60 2024-04-25T10:07:09.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.558+0000 7f5367fff640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f536c06b4d0 0x7f536c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.558+0000 7f5367fff640 1 -- 172.21.15.19:0/1891436104 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f53740aa900 con 0x7f53800cee60 2024-04-25T10:07:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.558+0000 7f537effd640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f536c06b4d0 0x7f536c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.558+0000 7f537effd640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f536c06b4d0 0x7f536c06d990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5370006fd0 tx=0x7f5370008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.562+0000 7f5367fff640 1 -- 172.21.15.19:0/1891436104 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f537404b030 con 0x7f53800cee60 2024-04-25T10:07:09.592 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:09.594 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:09.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.744+0000 7f5385a15640 1 -- 172.21.15.19:0/1891436104 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f538006b480 con 0x7f536c06b4d0 2024-04-25T10:07:09.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.746+0000 7f5367fff640 1 -- 172.21.15.19:0/1891436104 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13523 (secure 0 0 0) 0x7f538006b480 con 0x7f536c06b4d0 2024-04-25T10:07:09.751 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:09.752 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:07:09.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 -- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f536c06b4d0 msgr2=0x7f536c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f536c06b4d0 0x7f536c06d990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5370006fd0 tx=0x7f5370008040 comp rx=0 tx=0).stop 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 -- 172.21.15.19:0/1891436104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53800cee60 msgr2=0x7f5380152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53800cee60 0x7f5380152440 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f537402a220 tx=0x7f53740046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 -- 172.21.15.19:0/1891436104 shutdown_connections 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f536c06b4d0 0x7f536c06d990 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5380144750 0x7f5380152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53800cee60 0x7f5380152440 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 --2- 172.21.15.19:0/1891436104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f538006eb10 0x7f5380151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 -- 172.21.15.19:0/1891436104 >> 172.21.15.19:0/1891436104 conn(0x7f538006abe0 msgr2=0x7f538015a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 -- 172.21.15.19:0/1891436104 shutdown_connections 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.748+0000 7f5385a15640 1 -- 172.21.15.19:0/1891436104 wait complete. 2024-04-25T10:07:09.764 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":616,"stamp":"2024-04-25T10:07:09.111148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":469760,"kb_used_data":20488,"kb_used_omap":12,"kb_used_meta":449139,"kb_avail":754504960,"statfs":{"total":773094113280,"available":772613079040,"internally_reserved":0,"allocated":20979712,"data_stored":10485456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":459918898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624362,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-95,"num_write_kb":-12361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":-34,"num_keys_recovered":-10,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001463"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":527,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:07:06.529011+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:07:06.529011+0000","last_peered":"2024-04-25T10:07:06.529011+0000","last_clean":"2024-04-25T10:07:06.529011+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:07:06.529011+0000","last_undegraded":"2024-04-25T10:07:06.529011+0000","last_fullsized":"2024-04-25T10:07:06.529011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56384,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315456,"statfs":{"total":96636764160,"available":96579026944,"internally_reserved":0,"allocated":3067904,"data_stored":1753458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51652,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320188,"statfs":{"total":96636764160,"available":96583872512,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51648,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320192,"statfs":{"total":96636764160,"available":96583876608,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59964,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82112,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94289728,"statfs":{"total":96636764160,"available":96552681472,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81590727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55808,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316032,"statfs":{"total":96636764160,"available":96579616768,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575021,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55804,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316036,"statfs":{"total":96636764160,"available":96579620864,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575022,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56388,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315452,"statfs":{"total":96636764160,"available":96579022848,"internally_reserved":0,"allocated":3067904,"data_stored":1753458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:09.765 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:09.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.860+0000 7f3eb8ea3640 1 Processor -- start 2024-04-25T10:07:09.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.860+0000 7f3eb8ea3640 1 -- start start 2024-04-25T10:07:09.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb8ea3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3eb40b6cb0 0x7f3eb40b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb8ea3640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3eb40b5a60 0x7f3eb40d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb8ea3640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb40ceee0 con 0x7f3eb415db70 2024-04-25T10:07:09.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb8ea3640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb40d1d10 con 0x7f3eb40b5a60 2024-04-25T10:07:09.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb8ea3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40d1ff0 con 0x7f3eb40b6cb0 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb2d9d640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3eb40b5a60 0x7f3eb40d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb259c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3eb415db70 0x7f3eb4162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54240/0 (socket says 172.21.15.19:54240) 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb259c640 1 -- 172.21.15.19:0/2302957938 learned_addr learned my addr 172.21.15.19:0/2302957938 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb1d9b640 1 --2- 172.21.15.19:0/2302957938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3eb40b6cb0 0x7f3eb40b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb2d9d640 1 -- 172.21.15.19:0/2302957938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3eb40b6cb0 msgr2=0x7f3eb40b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb2d9d640 1 --2- 172.21.15.19:0/2302957938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3eb40b6cb0 0x7f3eb40b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb2d9d640 1 -- 172.21.15.19:0/2302957938 >> v1:172.21.15.19:6789/0 conn(0x7f3eb415db70 legacy=0x7f3eb4162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb2d9d640 1 -- 172.21.15.19:0/2302957938 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40d22d0 con 0x7f3eb40b5a60 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb159a640 1 -- 172.21.15.19:0/2302957938 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175174156 0 0) 0x7f3eb40ceee0 con 0x7f3eb415db70 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.861+0000 7f3eb1d9b640 1 --2- 172.21.15.19:0/2302957938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3eb40b6cb0 0x7f3eb40b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.862+0000 7f3eb2d9d640 1 --2- 172.21.15.19:0/2302957938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb40b5a60 0x7f3eb40d27f0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009d90 tx=0x7f3eac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722f8c395681b27e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.862+0000 7f3eb159a640 1 -- 172.21.15.19:0/2302957938 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac007740 con 0x7f3eb40b5a60 2024-04-25T10:07:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.862+0000 7f3eb159a640 1 -- 172.21.15.19:0/2302957938 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac0078e0 con 0x7f3eb40b5a60 2024-04-25T10:07:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.862+0000 7f3eb159a640 1 -- 172.21.15.19:0/2302957938 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac007cb0 con 0x7f3eb40b5a60 2024-04-25T10:07:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.862+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2302957938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb40b5a60 msgr2=0x7f3eb40d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.862+0000 7f3eb8ea3640 1 --2- 172.21.15.19:0/2302957938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb40b5a60 0x7f3eb40d27f0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009d90 tx=0x7f3eac02a660 comp rx=0 tx=0).stop 2024-04-25T10:07:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.863+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2302957938 shutdown_connections 2024-04-25T10:07:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.863+0000 7f3eb8ea3640 1 --2- 172.21.15.19:0/2302957938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb40b5a60 0x7f3eb40d27f0 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.863+0000 7f3eb8ea3640 1 --2- 172.21.15.19:0/2302957938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3eb40b6cb0 0x7f3eb40b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.863+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2302957938 >> 172.21.15.19:0/2302957938 conn(0x7f3eb40d0520 msgr2=0x7f3eb40d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.863+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2302957938 shutdown_connections 2024-04-25T10:07:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.863+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2302957938 wait complete. 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.864+0000 7f3eb8ea3640 1 Processor -- start 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.864+0000 7f3eb8ea3640 1 -- start start 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb8ea3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb40b5a60 0x7f3eb41582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb8ea3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3eb40b6cb0 0x7f3eb40c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb8ea3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb415db70 0x7f3eb41e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb8ea3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40cf1c0 con 0x7f3eb40b5a60 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb8ea3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40ce530 con 0x7f3eb415db70 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb8ea3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40cf340 con 0x7f3eb40b6cb0 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb40b5a60 0x7f3eb41582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb40b5a60 0x7f3eb41582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38818/0 (socket says 172.21.15.19:38818) 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb259c640 1 -- 172.21.15.19:0/2551309654 learned_addr learned my addr 172.21.15.19:0/2551309654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb1d9b640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3eb40b6cb0 0x7f3eb40c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb2d9d640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb415db70 0x7f3eb41e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb259c640 1 -- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3eb40b6cb0 msgr2=0x7f3eb40c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb259c640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3eb40b6cb0 0x7f3eb40c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb259c640 1 -- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb415db70 msgr2=0x7f3eb41e8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb259c640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb415db70 0x7f3eb41e8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb259c640 1 -- 172.21.15.19:0/2551309654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40c9180 con 0x7f3eb40b5a60 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb2d9d640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb415db70 0x7f3eb41e8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb259c640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb40b5a60 0x7f3eb41582a0 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c500 tx=0x7f3ea400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/2551309654 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4018070 con 0x7f3eb40b5a60 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/2551309654 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea4007580 con 0x7f3eb40b5a60 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2551309654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb40ca230 con 0x7f3eb40b5a60 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.865+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/2551309654 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4004a20 con 0x7f3eb40b5a60 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.866+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2551309654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb40c8440 con 0x7f3eb40b5a60 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.867+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2551309654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb4073990 con 0x7f3eb40b5a60 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.867+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/2551309654 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ea401b070 con 0x7f3eb40b5a60 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.867+0000 7f3ea2ffd640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3e9406b500 0x7f3e9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.868+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/2551309654 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3ea4089290 con 0x7f3eb40b5a60 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.868+0000 7f3eb1d9b640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3e9406b500 0x7f3e9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.868+0000 7f3eb1d9b640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3e9406b500 0x7f3e9406d9c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3eb41593f0 tx=0x7f3ea8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:09.872+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/2551309654 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ea4057c70 con 0x7f3eb40b5a60 2024-04-25T10:07:10.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.048+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2551309654 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3eb40d1820 con 0x7f3e9406b500 2024-04-25T10:07:10.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.049+0000 7f3ea2ffd640 1 -- 172.21.15.19:0/2551309654 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13523 (secure 0 0 0) 0x7f3eb40d1820 con 0x7f3e9406b500 2024-04-25T10:07:10.055 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:10.055 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3e9406b500 msgr2=0x7f3e9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3e9406b500 0x7f3e9406d9c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3eb41593f0 tx=0x7f3ea8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2551309654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb40b5a60 msgr2=0x7f3eb41582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb40b5a60 0x7f3eb41582a0 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c500 tx=0x7f3ea400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2551309654 shutdown_connections 2024-04-25T10:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f3e9406b500 0x7f3e9406d9c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb415db70 0x7f3eb41e8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3eb40b6cb0 0x7f3eb40c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 --2- 172.21.15.19:0/2551309654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb40b5a60 0x7f3eb41582a0 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2551309654 >> 172.21.15.19:0/2551309654 conn(0x7f3eb40d0520 msgr2=0x7f3eb40b75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2551309654 shutdown_connections 2024-04-25T10:07:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.051+0000 7f3eb8ea3640 1 -- 172.21.15.19:0/2551309654 wait complete. 2024-04-25T10:07:10.067 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":616,"stamp":"2024-04-25T10:07:09.111148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":469760,"kb_used_data":20488,"kb_used_omap":12,"kb_used_meta":449139,"kb_avail":754504960,"statfs":{"total":773094113280,"available":772613079040,"internally_reserved":0,"allocated":20979712,"data_stored":10485456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":459918898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624362,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-95,"num_write_kb":-12361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":-34,"num_keys_recovered":-10,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001463"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":527,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:07:06.529011+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:07:06.529011+0000","last_peered":"2024-04-25T10:07:06.529011+0000","last_clean":"2024-04-25T10:07:06.529011+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:07:06.529011+0000","last_undegraded":"2024-04-25T10:07:06.529011+0000","last_fullsized":"2024-04-25T10:07:06.529011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56384,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315456,"statfs":{"total":96636764160,"available":96579026944,"internally_reserved":0,"allocated":3067904,"data_stored":1753458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51652,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320188,"statfs":{"total":96636764160,"available":96583872512,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51648,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320192,"statfs":{"total":96636764160,"available":96583876608,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59964,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82112,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94289728,"statfs":{"total":96636764160,"available":96552681472,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81590727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55808,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316032,"statfs":{"total":96636764160,"available":96579616768,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575021,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55804,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316036,"statfs":{"total":96636764160,"available":96579620864,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575022,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56388,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315452,"statfs":{"total":96636764160,"available":96579022848,"internally_reserved":0,"allocated":3067904,"data_stored":1753458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:10.068 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:07:10.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:10.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.162+0000 7f0a8429a640 1 Processor -- start 2024-04-25T10:07:10.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.163+0000 7f0a8429a640 1 -- start start 2024-04-25T10:07:10.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.163+0000 7f0a8429a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a7c15dbe0 0x7f0a7c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.163+0000 7f0a8429a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0a7c0b2cb0 0x7f0a7c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.163+0000 7f0a8429a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7c0b97a0 con 0x7f0a7c0b92a0 2024-04-25T10:07:10.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a8429a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a7c15dfc0 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a8429a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a7c0d1a00 con 0x7f0a7c0b2cb0 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a82837640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0a7c0b92a0 0x7f0a7c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54250/0 (socket says 172.21.15.19:54250) 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a82837640 1 -- 172.21.15.19:0/164334807 learned_addr learned my addr 172.21.15.19:0/164334807 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a82036640 1 --2- 172.21.15.19:0/164334807 >> v2:172.21.15.19:3300/0 conn(0x7f0a7c15dbe0 0x7f0a7c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a81835640 1 --2- 172.21.15.19:0/164334807 >> v2:172.21.15.112:3300/0 conn(0x7f0a7c0b2cb0 0x7f0a7c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a81034640 1 -- 172.21.15.19:0/164334807 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204406292 0 0) 0x7f0a7c0b97a0 con 0x7f0a7c0b92a0 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a81034640 1 -- 172.21.15.19:0/164334807 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a64003520 con 0x7f0a7c0b92a0 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a82036640 1 -- 172.21.15.19:0/164334807 >> v2:172.21.15.112:3300/0 conn(0x7f0a7c0b2cb0 msgr2=0x7f0a7c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a82036640 1 --2- 172.21.15.19:0/164334807 >> v2:172.21.15.112:3300/0 conn(0x7f0a7c0b2cb0 0x7f0a7c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a82036640 1 -- 172.21.15.19:0/164334807 >> v1:172.21.15.19:6789/0 conn(0x7f0a7c0b92a0 legacy=0x7f0a7c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.164+0000 7f0a82036640 1 -- 172.21.15.19:0/164334807 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c0d1c50 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.165+0000 7f0a82036640 1 --2- 172.21.15.19:0/164334807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a7c15dbe0 0x7f0a7c0d0910 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c06cff0 tx=0x7f0a6c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51f305b10c5f667 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.165+0000 7f0a81034640 1 -- 172.21.15.19:0/164334807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c007740 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.165+0000 7f0a81034640 1 -- 172.21.15.19:0/164334807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6c0078e0 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.165+0000 7f0a81034640 1 -- 172.21.15.19:0/164334807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c007cb0 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.165+0000 7f0a8429a640 1 -- 172.21.15.19:0/164334807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a7c15dbe0 msgr2=0x7f0a7c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.165+0000 7f0a8429a640 1 --2- 172.21.15.19:0/164334807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a7c15dbe0 0x7f0a7c0d0910 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c06cff0 tx=0x7f0a6c02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.165+0000 7f0a8429a640 1 -- 172.21.15.19:0/164334807 shutdown_connections 2024-04-25T10:07:10.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.165+0000 7f0a8429a640 1 --2- 172.21.15.19:0/164334807 >> v2:172.21.15.112:3300/0 conn(0x7f0a7c0b2cb0 0x7f0a7c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.165+0000 7f0a8429a640 1 --2- 172.21.15.19:0/164334807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a7c15dbe0 0x7f0a7c0d0910 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.165+0000 7f0a8429a640 1 -- 172.21.15.19:0/164334807 >> 172.21.15.19:0/164334807 conn(0x7f0a7c061a20 msgr2=0x7f0a7c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.166+0000 7f0a8429a640 1 -- 172.21.15.19:0/164334807 shutdown_connections 2024-04-25T10:07:10.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.166+0000 7f0a8429a640 1 -- 172.21.15.19:0/164334807 wait complete. 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.167+0000 7f0a8429a640 1 Processor -- start 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.167+0000 7f0a8429a640 1 -- start start 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.167+0000 7f0a8429a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7c0b2cb0 0x7f0a7c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.167+0000 7f0a8429a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a7c0b92a0 0x7f0a7c1f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.167+0000 7f0a8429a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a7c15dbe0 0x7f0a7c0cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.167+0000 7f0a8429a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c0b6ab0 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.167+0000 7f0a8429a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c0d2210 con 0x7f0a7c0b2cb0 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.167+0000 7f0a8429a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c0b6c30 con 0x7f0a7c0b92a0 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.167+0000 7f0a82837640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a7c15dbe0 0x7f0a7c0cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a82837640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a7c15dbe0 0x7f0a7c0cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38830/0 (socket says 172.21.15.19:38830) 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a82837640 1 -- 172.21.15.19:0/2127395448 learned_addr learned my addr 172.21.15.19:0/2127395448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:10.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a81835640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a7c0b92a0 0x7f0a7c1f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a82036640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7c0b2cb0 0x7f0a7c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a82837640 1 -- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a7c0b92a0 msgr2=0x7f0a7c1f7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a82837640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a7c0b92a0 0x7f0a7c1f7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a82837640 1 -- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7c0b2cb0 msgr2=0x7f0a7c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a82837640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7c0b2cb0 0x7f0a7c15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a82837640 1 -- 172.21.15.19:0/2127395448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c1f7bb0 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a82837640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a7c15dbe0 0x7f0a7c0cca30 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400c430 tx=0x7f0a7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a7affd640 1 -- 172.21.15.19:0/2127395448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74017070 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a7affd640 1 -- 172.21.15.19:0/2127395448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a74005ce0 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a7affd640 1 -- 172.21.15.19:0/2127395448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a740072d0 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a8429a640 1 -- 172.21.15.19:0/2127395448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c0ccf70 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.168+0000 7f0a8429a640 1 -- 172.21.15.19:0/2127395448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c0cb990 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.169+0000 7f0a8429a640 1 -- 172.21.15.19:0/2127395448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7c073990 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.170+0000 7f0a7affd640 1 -- 172.21.15.19:0/2127395448 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a7401a070 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.170+0000 7f0a7affd640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a6806b4d0 0x7f0a6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.170+0000 7f0a7affd640 1 -- 172.21.15.19:0/2127395448 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f0a74089330 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.170+0000 7f0a82036640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a6806b4d0 0x7f0a6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.171+0000 7f0a82036640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a6806b4d0 0x7f0a6806d990 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c034000 tx=0x7f0a6c02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.174+0000 7f0a7affd640 1 -- 172.21.15.19:0/2127395448 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a7c1f7bb0 con 0x7f0a7c15dbe0 2024-04-25T10:07:10.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.351+0000 7f0a8429a640 1 -- 172.21.15.19:0/2127395448 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a7c1605b0 con 0x7f0a6806b4d0 2024-04-25T10:07:10.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.352+0000 7f0a7affd640 1 -- 172.21.15.19:0/2127395448 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13523 (secure 0 0 0) 0x7f0a7c1605b0 con 0x7f0a6806b4d0 2024-04-25T10:07:10.357 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:10.358 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:07:10.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.353+0000 7f0a8429a640 1 -- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a6806b4d0 msgr2=0x7f0a6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.353+0000 7f0a8429a640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a6806b4d0 0x7f0a6806d990 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c034000 tx=0x7f0a6c02f000 comp rx=0 tx=0).stop 2024-04-25T10:07:10.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.353+0000 7f0a8429a640 1 -- 172.21.15.19:0/2127395448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a7c15dbe0 msgr2=0x7f0a7c0cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.353+0000 7f0a8429a640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a7c15dbe0 0x7f0a7c0cca30 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400c430 tx=0x7f0a7400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.353+0000 7f0a8429a640 1 -- 172.21.15.19:0/2127395448 shutdown_connections 2024-04-25T10:07:10.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.353+0000 7f0a8429a640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f0a6806b4d0 0x7f0a6806d990 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.353+0000 7f0a8429a640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a7c15dbe0 0x7f0a7c0cca30 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.353+0000 7f0a8429a640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a7c0b92a0 0x7f0a7c1f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.353+0000 7f0a8429a640 1 --2- 172.21.15.19:0/2127395448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a7c0b2cb0 0x7f0a7c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.354+0000 7f0a8429a640 1 -- 172.21.15.19:0/2127395448 >> 172.21.15.19:0/2127395448 conn(0x7f0a7c061a20 msgr2=0x7f0a7c0d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.354+0000 7f0a8429a640 1 -- 172.21.15.19:0/2127395448 shutdown_connections 2024-04-25T10:07:10.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.354+0000 7f0a8429a640 1 -- 172.21.15.19:0/2127395448 wait complete. 2024-04-25T10:07:10.370 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":616,"stamp":"2024-04-25T10:07:09.111148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":469760,"kb_used_data":20488,"kb_used_omap":12,"kb_used_meta":449139,"kb_avail":754504960,"statfs":{"total":773094113280,"available":772613079040,"internally_reserved":0,"allocated":20979712,"data_stored":10485456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":459918898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624362,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-95,"num_write_kb":-12361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":-34,"num_keys_recovered":-10,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001463"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":527,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:07:06.529011+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:07:06.529011+0000","last_peered":"2024-04-25T10:07:06.529011+0000","last_clean":"2024-04-25T10:07:06.529011+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:07:06.529011+0000","last_undegraded":"2024-04-25T10:07:06.529011+0000","last_fullsized":"2024-04-25T10:07:06.529011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56384,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315456,"statfs":{"total":96636764160,"available":96579026944,"internally_reserved":0,"allocated":3067904,"data_stored":1753458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51652,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320188,"statfs":{"total":96636764160,"available":96583872512,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51648,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320192,"statfs":{"total":96636764160,"available":96583876608,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59964,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82112,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94289728,"statfs":{"total":96636764160,"available":96552681472,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81590727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55808,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316032,"statfs":{"total":96636764160,"available":96579616768,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575021,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55804,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316036,"statfs":{"total":96636764160,"available":96579620864,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575022,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56388,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315452,"statfs":{"total":96636764160,"available":96579022848,"internally_reserved":0,"allocated":3067904,"data_stored":1753458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:10.371 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:10.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.466+0000 7fde9ff42640 1 Processor -- start 2024-04-25T10:07:10.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.466+0000 7fde9ff42640 1 -- start start 2024-04-25T10:07:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.467+0000 7fde9ff42640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde980ce590 0x7fde98165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.467+0000 7fde9ff42640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde98071480 con 0x7fde98154ee0 2024-04-25T10:07:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.467+0000 7fde9ff42640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde98071760 con 0x7fde980cfec0 2024-04-25T10:07:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.467+0000 7fde9ff42640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde98072090 con 0x7fde980ce590 2024-04-25T10:07:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.467+0000 7fde9e4df640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde980ce590 0x7fde98165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.467+0000 7fde9dcde640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fde98154ee0 0x7fde980cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54258/0 (socket says 172.21.15.19:54258) 2024-04-25T10:07:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.467+0000 7fde9dcde640 1 -- 172.21.15.19:0/4016226172 learned_addr learned my addr 172.21.15.19:0/4016226172 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.467+0000 7fde9e4df640 1 -- 172.21.15.19:0/4016226172 >> v1:172.21.15.112:6789/0 conn(0x7fde980cfec0 legacy=0x7fde980cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.467+0000 7fde9e4df640 1 -- 172.21.15.19:0/4016226172 >> v1:172.21.15.19:6789/0 conn(0x7fde98154ee0 legacy=0x7fde980cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.467+0000 7fde9e4df640 1 -- 172.21.15.19:0/4016226172 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98071a40 con 0x7fde980ce590 2024-04-25T10:07:10.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.468+0000 7fde9e4df640 1 --2- 172.21.15.19:0/4016226172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde980ce590 0x7fde98165900 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7fde90009d90 tx=0x7fde9002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b2c2eaeb6c57da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.468+0000 7fde9ccdc640 1 -- 172.21.15.19:0/4016226172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90007740 con 0x7fde980ce590 2024-04-25T10:07:10.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.468+0000 7fde9ccdc640 1 -- 172.21.15.19:0/4016226172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde900078e0 con 0x7fde980ce590 2024-04-25T10:07:10.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.468+0000 7fde9ccdc640 1 -- 172.21.15.19:0/4016226172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90007cb0 con 0x7fde980ce590 2024-04-25T10:07:10.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.468+0000 7fde9ff42640 1 -- 172.21.15.19:0/4016226172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde980ce590 msgr2=0x7fde98165900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.468+0000 7fde9ff42640 1 --2- 172.21.15.19:0/4016226172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde980ce590 0x7fde98165900 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7fde90009d90 tx=0x7fde9002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.469+0000 7fde9ff42640 1 -- 172.21.15.19:0/4016226172 shutdown_connections 2024-04-25T10:07:10.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.469+0000 7fde9ff42640 1 --2- 172.21.15.19:0/4016226172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde980ce590 0x7fde98165900 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.469+0000 7fde9ff42640 1 -- 172.21.15.19:0/4016226172 >> 172.21.15.19:0/4016226172 conn(0x7fde9806abe0 msgr2=0x7fde9806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.469+0000 7fde9ff42640 1 -- 172.21.15.19:0/4016226172 shutdown_connections 2024-04-25T10:07:10.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.469+0000 7fde9ff42640 1 -- 172.21.15.19:0/4016226172 wait complete. 2024-04-25T10:07:10.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.470+0000 7fde9ff42640 1 Processor -- start 2024-04-25T10:07:10.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.470+0000 7fde9ff42640 1 -- start start 2024-04-25T10:07:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.470+0000 7fde9ff42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde980ce590 0x7fde98156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9ff42640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde980cfec0 0x7fde98143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9ff42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde98154ee0 0x7fde980b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9ff42640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde981552c0 con 0x7fde98154ee0 2024-04-25T10:07:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9ff42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde98071230 con 0x7fde980ce590 2024-04-25T10:07:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9ff42640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde98155440 con 0x7fde980cfec0 2024-04-25T10:07:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9e4df640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde98154ee0 0x7fde980b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9e4df640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde98154ee0 0x7fde980b0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38848/0 (socket says 172.21.15.19:38848) 2024-04-25T10:07:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9e4df640 1 -- 172.21.15.19:0/3475599136 learned_addr learned my addr 172.21.15.19:0/3475599136 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:10.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9e4df640 1 -- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde980cfec0 msgr2=0x7fde98143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9e4df640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde980cfec0 0x7fde98143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9e4df640 1 -- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde980ce590 msgr2=0x7fde98156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9e4df640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde980ce590 0x7fde98156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9e4df640 1 -- 172.21.15.19:0/3475599136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98155f80 con 0x7fde98154ee0 2024-04-25T10:07:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9e4df640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde98154ee0 0x7fde980b0c80 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7fde9002ace0 tx=0x7fde90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde8e7fc640 1 -- 172.21.15.19:0/3475599136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90040070 con 0x7fde98154ee0 2024-04-25T10:07:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde8e7fc640 1 -- 172.21.15.19:0/3475599136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde90031c90 con 0x7fde98154ee0 2024-04-25T10:07:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9ff42640 1 -- 172.21.15.19:0/3475599136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde98156220 con 0x7fde98154ee0 2024-04-25T10:07:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde8e7fc640 1 -- 172.21.15.19:0/3475599136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90045660 con 0x7fde98154ee0 2024-04-25T10:07:10.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.471+0000 7fde9ff42640 1 -- 172.21.15.19:0/3475599136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde981567e0 con 0x7fde98154ee0 2024-04-25T10:07:10.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.472+0000 7fde9ff42640 1 -- 172.21.15.19:0/3475599136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde98073990 con 0x7fde98154ee0 2024-04-25T10:07:10.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.473+0000 7fde8e7fc640 1 -- 172.21.15.19:0/3475599136 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde90039040 con 0x7fde98154ee0 2024-04-25T10:07:10.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.473+0000 7fde8e7fc640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fde7c06b500 0x7fde7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.473+0000 7fde8e7fc640 1 -- 172.21.15.19:0/3475599136 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fde900ab9d0 con 0x7fde98154ee0 2024-04-25T10:07:10.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.474+0000 7fde9dcde640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fde7c06b500 0x7fde7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.474+0000 7fde9dcde640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fde7c06b500 0x7fde7c06d9c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fde94001550 tx=0x7fde94007a40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.477+0000 7fde8e7fc640 1 -- 172.21.15.19:0/3475599136 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde90043360 con 0x7fde98154ee0 2024-04-25T10:07:10.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.658+0000 7fde9ff42640 1 -- 172.21.15.19:0/3475599136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde9806d360 con 0x7fde98154ee0 2024-04-25T10:07:10.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.658+0000 7fde8e7fc640 1 -- 172.21.15.19:0/3475599136 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+13419 (secure 0 0 0) 0x7fde9007aac0 con 0x7fde98154ee0 2024-04-25T10:07:10.664 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:10.664 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":195,"fsid":"342f98bd-d5a6-4934-bb27-7f8bea67a3be","created":"2024-04-25T09:52:42.741093+0000","modified":"2024-04-25T10:07:06.522287+0000","last_up_change":"2024-04-25T09:52:53.898534+0000","last_in_change":"2024-04-25T09:52:44.945978+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:52:54.954281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"131c238a-d117-470a-a62f-2589738cf304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6801","nonce":3769136101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6803","nonce":3769136101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6807","nonce":3769136101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3769136101},{"type":"v1","addr":"172.21.15.19:6805","nonce":3769136101}]},"public_addr":"172.21.15.19:6801/3769136101","cluster_addr":"172.21.15.19:6803/3769136101","heartbeat_back_addr":"172.21.15.19:6807/3769136101","heartbeat_front_addr":"172.21.15.19:6805/3769136101","state":["exists","up"]},{"osd":1,"uuid":"3a714f50-4bcb-4995-b6b4-788924493ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6809","nonce":495652536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6811","nonce":495652536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6815","nonce":495652536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":495652536},{"type":"v1","addr":"172.21.15.19:6813","nonce":495652536}]},"public_addr":"172.21.15.19:6809/495652536","cluster_addr":"172.21.15.19:6811/495652536","heartbeat_back_addr":"172.21.15.19:6815/495652536","heartbeat_front_addr":"172.21.15.19:6813/495652536","state":["exists","up"]},{"osd":2,"uuid":"d6c10f87-21de-44ad-bbf7-9a886f78550f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6817","nonce":3205329770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6819","nonce":3205329770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6823","nonce":3205329770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3205329770},{"type":"v1","addr":"172.21.15.19:6821","nonce":3205329770}]},"public_addr":"172.21.15.19:6817/3205329770","cluster_addr":"172.21.15.19:6819/3205329770","heartbeat_back_addr":"172.21.15.19:6823/3205329770","heartbeat_front_addr":"172.21.15.19:6821/3205329770","state":["exists","up"]},{"osd":3,"uuid":"0726e0ce-2ffe-41e8-a69f-7a8274edaf6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6825","nonce":577725691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6827","nonce":577725691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6831","nonce":577725691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":577725691},{"type":"v1","addr":"172.21.15.19:6829","nonce":577725691}]},"public_addr":"172.21.15.19:6825/577725691","cluster_addr":"172.21.15.19:6827/577725691","heartbeat_back_addr":"172.21.15.19:6831/577725691","heartbeat_front_addr":"172.21.15.19:6829/577725691","state":["exists","up"]},{"osd":4,"uuid":"33e334e0-01c4-4451-a5ed-47ccec5978c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6801","nonce":1718655451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6803","nonce":1718655451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6807","nonce":1718655451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1718655451},{"type":"v1","addr":"172.21.15.112:6805","nonce":1718655451}]},"public_addr":"172.21.15.112:6801/1718655451","cluster_addr":"172.21.15.112:6803/1718655451","heartbeat_back_addr":"172.21.15.112:6807/1718655451","heartbeat_front_addr":"172.21.15.112:6805/1718655451","state":["exists","up"]},{"osd":5,"uuid":"ed9503ba-06a9-4784-9879-6a20100a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6809","nonce":2948328480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6811","nonce":2948328480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6815","nonce":2948328480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2948328480},{"type":"v1","addr":"172.21.15.112:6813","nonce":2948328480}]},"public_addr":"172.21.15.112:6809/2948328480","cluster_addr":"172.21.15.112:6811/2948328480","heartbeat_back_addr":"172.21.15.112:6815/2948328480","heartbeat_front_addr":"172.21.15.112:6813/2948328480","state":["exists","up"]},{"osd":6,"uuid":"1a2be840-63b7-4280-bfe5-b908a9351c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6818","nonce":3693987094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6822","nonce":3693987094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6830","nonce":3693987094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3693987094},{"type":"v1","addr":"172.21.15.112:6826","nonce":3693987094}]},"public_addr":"172.21.15.112:6818/3693987094","cluster_addr":"172.21.15.112:6822/3693987094","heartbeat_back_addr":"172.21.15.112:6830/3693987094","heartbeat_front_addr":"172.21.15.112:6826/3693987094","state":["exists","up"]},{"osd":7,"uuid":"6efd9150-bb02-4e46-9205-3a713f594e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6817","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6819","nonce":3405244921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6821","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6823","nonce":3405244921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6829","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6831","nonce":3405244921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3405244921},{"type":"v1","addr":"172.21.15.112:6827","nonce":3405244921}]},"public_addr":"172.21.15.112:6819/3405244921","cluster_addr":"172.21.15.112:6823/3405244921","heartbeat_back_addr":"172.21.15.112:6831/3405244921","heartbeat_front_addr":"172.21.15.112:6827/3405244921","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.068234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.108054+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137702+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.137455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.067371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.171842+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.132124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:50.185582+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/1758455116":"2024-04-26T10:07:03.461999+0000","172.21.15.19:6833/1758455116":"2024-04-26T10:07:03.461999+0000","172.21.15.19:6835/392508344":"2024-04-26T10:05:52.378361+0000","172.21.15.19:6833/392508344":"2024-04-26T10:05:52.378361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:10.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 -- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fde7c06b500 msgr2=0x7fde7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fde7c06b500 0x7fde7c06d9c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fde94001550 tx=0x7fde94007a40 comp rx=0 tx=0).stop 2024-04-25T10:07:10.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 -- 172.21.15.19:0/3475599136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde98154ee0 msgr2=0x7fde980b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde98154ee0 0x7fde980b0c80 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7fde9002ace0 tx=0x7fde90004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:10.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 -- 172.21.15.19:0/3475599136 shutdown_connections 2024-04-25T10:07:10.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fde7c06b500 0x7fde7c06d9c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde98154ee0 0x7fde980b0c80 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde980cfec0 0x7fde98143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 --2- 172.21.15.19:0/3475599136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde980ce590 0x7fde98156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 -- 172.21.15.19:0/3475599136 >> 172.21.15.19:0/3475599136 conn(0x7fde9806abe0 msgr2=0x7fde980ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 -- 172.21.15.19:0/3475599136 shutdown_connections 2024-04-25T10:07:10.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:10.660+0000 7fde9ff42640 1 -- 172.21.15.19:0/3475599136 wait complete. 2024-04-25T10:07:11.678 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:07:11.678 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:07:11.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.790+0000 7fd4f0641640 1 Processor -- start 2024-04-25T10:07:11.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.790+0000 7fd4f0641640 1 -- start start 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.791+0000 7fd4f0641640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4e80b5c40 0x7fd4e80ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.791+0000 7fd4f0641640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e80cf320 con 0x7fd4e815b5a0 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.791+0000 7fd4f0641640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e80cf040 con 0x7fd4e815b980 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.791+0000 7fd4f0641640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8161970 con 0x7fd4e80b5c40 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.791+0000 7fd4ee3dd640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd4e815b5a0 0x7fd4e80b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54272/0 (socket says 172.21.15.19:54272) 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.791+0000 7fd4ee3dd640 1 -- 172.21.15.19:0/5989632 learned_addr learned my addr 172.21.15.19:0/5989632 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.791+0000 7fd4edbdc640 1 --2- 172.21.15.19:0/5989632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4e80b5c40 0x7fd4e80ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.791+0000 7fd4ed3db640 1 -- 172.21.15.19:0/5989632 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949100240 0 0) 0x7fd4e4001110 con 0x7fd4e815b5a0 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.792+0000 7fd4ed3db640 1 -- 172.21.15.19:0/5989632 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4d4003520 con 0x7fd4e815b5a0 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.792+0000 7fd4ed3db640 1 -- 172.21.15.19:0/5989632 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599525899 0 0) 0x7fd4e80cf040 con 0x7fd4e815b980 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.792+0000 7fd4ed3db640 1 -- 172.21.15.19:0/5989632 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e4001110 con 0x7fd4e815b980 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.792+0000 7fd4edbdc640 1 -- 172.21.15.19:0/5989632 >> v1:172.21.15.112:6789/0 conn(0x7fd4e815b980 legacy=0x7fd4e80ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.792+0000 7fd4edbdc640 1 -- 172.21.15.19:0/5989632 >> v1:172.21.15.19:6789/0 conn(0x7fd4e815b5a0 legacy=0x7fd4e80b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.792+0000 7fd4edbdc640 1 -- 172.21.15.19:0/5989632 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e8161c50 con 0x7fd4e80b5c40 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.792+0000 7fd4edbdc640 1 --2- 172.21.15.19:0/5989632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4e80b5c40 0x7fd4e80ce310 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80097c0 tx=0x7fd4d802a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d311a65a0b3d8d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.793+0000 7fd4ed3db640 1 -- 172.21.15.19:0/5989632 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8037070 con 0x7fd4e80b5c40 2024-04-25T10:07:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.793+0000 7fd4ed3db640 1 -- 172.21.15.19:0/5989632 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d8032440 con 0x7fd4e80b5c40 2024-04-25T10:07:11.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.793+0000 7fd4ed3db640 1 -- 172.21.15.19:0/5989632 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8032810 con 0x7fd4e80b5c40 2024-04-25T10:07:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.793+0000 7fd4f0641640 1 -- 172.21.15.19:0/5989632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4e80b5c40 msgr2=0x7fd4e80ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.793+0000 7fd4f0641640 1 --2- 172.21.15.19:0/5989632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4e80b5c40 0x7fd4e80ce310 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80097c0 tx=0x7fd4d802a200 comp rx=0 tx=0).stop 2024-04-25T10:07:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.793+0000 7fd4f0641640 1 -- 172.21.15.19:0/5989632 shutdown_connections 2024-04-25T10:07:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.793+0000 7fd4f0641640 1 --2- 172.21.15.19:0/5989632 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4e80b5c40 0x7fd4e80ce310 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.793+0000 7fd4f0641640 1 -- 172.21.15.19:0/5989632 >> 172.21.15.19:0/5989632 conn(0x7fd4e80d0530 msgr2=0x7fd4e80d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.793+0000 7fd4f0641640 1 -- 172.21.15.19:0/5989632 shutdown_connections 2024-04-25T10:07:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.793+0000 7fd4f0641640 1 -- 172.21.15.19:0/5989632 wait complete. 2024-04-25T10:07:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.795+0000 7fd4f0641640 1 Processor -- start 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.795+0000 7fd4f0641640 1 -- start start 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.795+0000 7fd4f0641640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e80b5c40 0x7fd4e80c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.795+0000 7fd4f0641640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4e815b5a0 0x7fd4e80cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.795+0000 7fd4f0641640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4e815b980 0x7fd4e81560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.795+0000 7fd4f0641640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e81625d0 con 0x7fd4e80b5c40 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.795+0000 7fd4f0641640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8162450 con 0x7fd4e815b5a0 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.795+0000 7fd4f0641640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8162750 con 0x7fd4e815b980 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4ee3dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e80b5c40 0x7fd4e80c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4ee3dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e80b5c40 0x7fd4e80c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38854/0 (socket says 172.21.15.19:38854) 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4ee3dd640 1 -- 172.21.15.19:0/2756419276 learned_addr learned my addr 172.21.15.19:0/2756419276 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4edbdc640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4e815b5a0 0x7fd4e80cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4eebde640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4e815b980 0x7fd4e81560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4ee3dd640 1 -- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4e815b980 msgr2=0x7fd4e81560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4ee3dd640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4e815b980 0x7fd4e81560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4ee3dd640 1 -- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4e815b5a0 msgr2=0x7fd4e80cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4ee3dd640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4e815b5a0 0x7fd4e80cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4ee3dd640 1 -- 172.21.15.19:0/2756419276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e80cab80 con 0x7fd4e80b5c40 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4edbdc640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4e815b5a0 0x7fd4e80cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4eebde640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4e815b980 0x7fd4e81560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4ee3dd640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e80b5c40 0x7fd4e80c8fe0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c6b0 tx=0x7fd4e400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4deffd640 1 -- 172.21.15.19:0/2756419276 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4017070 con 0x7fd4e80b5c40 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4deffd640 1 -- 172.21.15.19:0/2756419276 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e4005ab0 con 0x7fd4e80b5c40 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4f0641640 1 -- 172.21.15.19:0/2756419276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e80cae60 con 0x7fd4e80b5c40 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4deffd640 1 -- 172.21.15.19:0/2756419276 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4007070 con 0x7fd4e80b5c40 2024-04-25T10:07:11.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.796+0000 7fd4f0641640 1 -- 172.21.15.19:0/2756419276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e80cb420 con 0x7fd4e80b5c40 2024-04-25T10:07:11.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.797+0000 7fd4f0641640 1 -- 172.21.15.19:0/2756419276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd4e8073a00 con 0x7fd4e80b5c40 2024-04-25T10:07:11.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.798+0000 7fd4deffd640 1 -- 172.21.15.19:0/2756419276 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4e401a070 con 0x7fd4e80b5c40 2024-04-25T10:07:11.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.798+0000 7fd4deffd640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.799+0000 7fd4deffd640 1 -- 172.21.15.19:0/2756419276 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd4e4089ce0 con 0x7fd4e80b5c40 2024-04-25T10:07:11.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.799+0000 7fd4edbdc640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.799+0000 7fd4deffd640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] conn(0x7fd4cc071a10 0x7fd4cc073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.799+0000 7fd4deffd640 1 -- 172.21.15.19:0/2756419276 --> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4cc074520 con 0x7fd4cc071a10 2024-04-25T10:07:11.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.799+0000 7fd4deffd640 1 -- 172.21.15.19:0/2756419276 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4e408a090 con 0x7fd4e80b5c40 2024-04-25T10:07:11.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.799+0000 7fd4eebde640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] conn(0x7fd4cc071a10 0x7fd4cc073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.799+0000 7fd4edbdc640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8034000 tx=0x7fd4d802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.799+0000 7fd4eebde640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] conn(0x7fd4cc071a10 0x7fd4cc073e50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.800+0000 7fd4deffd640 1 -- 172.21.15.19:0/2756419276 <== osd.0 v2:172.21.15.19:6800/3769136101 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd4cc074520 con 0x7fd4cc071a10 2024-04-25T10:07:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.813+0000 7fd4f0641640 1 -- 172.21.15.19:0/2756419276 --> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd4e81555d0 con 0x7fd4cc071a10 2024-04-25T10:07:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.813+0000 7fd4deffd640 1 -- 172.21.15.19:0/2756419276 <== osd.0 v2:172.21.15.19:6800/3769136101 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd4e81555d0 con 0x7fd4cc071a10 2024-04-25T10:07:11.819 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:07:11.819 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:07:11.819 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:07:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.814+0000 7fd4f0641640 1 -- 172.21.15.19:0/2756419276 >> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] conn(0x7fd4cc071a10 msgr2=0x7fd4cc073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.814+0000 7fd4f0641640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.19:6800/3769136101,v1:172.21.15.19:6801/3769136101] conn(0x7fd4cc071a10 0x7fd4cc073e50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.814+0000 7fd4f0641640 1 -- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4cc06b4d0 msgr2=0x7fd4cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.814+0000 7fd4f0641640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8034000 tx=0x7fd4d802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.814+0000 7fd4f0641640 1 -- 172.21.15.19:0/2756419276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e80b5c40 msgr2=0x7fd4e80c8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.814+0000 7fd4f0641640 1 --2- 172.21.15.19:0/2756419276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4e80b5c40 0x7fd4e80c8fe0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c6b0 tx=0x7fd4e400cb70 comp rx=0 tx=0).stop 2024-04-25T10:07:11.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.814+0000 7fd4eebde640 1 -- 172.21.15.19:0/2756419276 reap_dead start 2024-04-25T10:07:11.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.814+0000 7fd4f0641640 1 -- 172.21.15.19:0/2756419276 shutdown_connections 2024-04-25T10:07:11.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.814+0000 7fd4f0641640 1 -- 172.21.15.19:0/2756419276 >> 172.21.15.19:0/2756419276 conn(0x7fd4e80d0530 msgr2=0x7fd4e815c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.814+0000 7fd4f0641640 1 -- 172.21.15.19:0/2756419276 shutdown_connections 2024-04-25T10:07:11.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.814+0000 7fd4f0641640 1 -- 172.21.15.19:0/2756419276 wait complete. 2024-04-25T10:07:11.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:07:11.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.926+0000 7fb4f10b5640 1 Processor -- start 2024-04-25T10:07:11.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.926+0000 7fb4f10b5640 1 -- start start 2024-04-25T10:07:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4f10b5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb4ec15b980 0x7fb4ec0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4f10b5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4ec072280 con 0x7fb4ec0ce230 2024-04-25T10:07:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4f10b5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4ec0714c0 con 0x7fb4ec0cc4f0 2024-04-25T10:07:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4f10b5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4ec071270 con 0x7fb4ec15b980 2024-04-25T10:07:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4e3fff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb4ec0ce230 0x7fb4ec15b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54288/0 (socket says 172.21.15.19:54288) 2024-04-25T10:07:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4e3fff640 1 -- 172.21.15.19:0/2152915378 learned_addr learned my addr 172.21.15.19:0/2152915378 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:11.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4eb59e640 1 --2- 172.21.15.19:0/2152915378 >> v2:172.21.15.112:3300/0 conn(0x7fb4ec15b980 0x7fb4ec0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4ea59c640 1 -- 172.21.15.19:0/2152915378 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990347734 0 0) 0x7fb4ec072280 con 0x7fb4ec0ce230 2024-04-25T10:07:11.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4ea59c640 1 -- 172.21.15.19:0/2152915378 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4d8003520 con 0x7fb4ec0ce230 2024-04-25T10:07:11.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4ea59c640 1 -- 172.21.15.19:0/2152915378 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099689266 0 0) 0x7fb4ec0714c0 con 0x7fb4ec0cc4f0 2024-04-25T10:07:11.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4ea59c640 1 -- 172.21.15.19:0/2152915378 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4ec072280 con 0x7fb4ec0cc4f0 2024-04-25T10:07:11.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4eb59e640 1 -- 172.21.15.19:0/2152915378 >> v1:172.21.15.112:6789/0 conn(0x7fb4ec0cc4f0 legacy=0x7fb4ec0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4eb59e640 1 -- 172.21.15.19:0/2152915378 >> v1:172.21.15.19:6789/0 conn(0x7fb4ec0ce230 legacy=0x7fb4ec15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.927+0000 7fb4eb59e640 1 -- 172.21.15.19:0/2152915378 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec0717a0 con 0x7fb4ec15b980 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.928+0000 7fb4eb59e640 1 --2- 172.21.15.19:0/2152915378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4ec15b980 0x7fb4ec0cc8d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009d90 tx=0x7fb4e402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62c7ae1cbc0a7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.928+0000 7fb4ea59c640 1 -- 172.21.15.19:0/2152915378 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4007740 con 0x7fb4ec15b980 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.928+0000 7fb4ea59c640 1 -- 172.21.15.19:0/2152915378 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e40078e0 con 0x7fb4ec15b980 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.928+0000 7fb4ea59c640 1 -- 172.21.15.19:0/2152915378 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4007cb0 con 0x7fb4ec15b980 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.929+0000 7fb4f10b5640 1 -- 172.21.15.19:0/2152915378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4ec15b980 msgr2=0x7fb4ec0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.929+0000 7fb4f10b5640 1 --2- 172.21.15.19:0/2152915378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4ec15b980 0x7fb4ec0cc8d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009d90 tx=0x7fb4e402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.929+0000 7fb4f10b5640 1 -- 172.21.15.19:0/2152915378 shutdown_connections 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.929+0000 7fb4f10b5640 1 --2- 172.21.15.19:0/2152915378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4ec15b980 0x7fb4ec0cc8d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.929+0000 7fb4f10b5640 1 -- 172.21.15.19:0/2152915378 >> 172.21.15.19:0/2152915378 conn(0x7fb4ec06abe0 msgr2=0x7fb4ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.929+0000 7fb4f10b5640 1 -- 172.21.15.19:0/2152915378 shutdown_connections 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.929+0000 7fb4f10b5640 1 -- 172.21.15.19:0/2152915378 wait complete. 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.930+0000 7fb4f10b5640 1 Processor -- start 2024-04-25T10:07:11.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.930+0000 7fb4f10b5640 1 -- start start 2024-04-25T10:07:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4f10b5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4ec0cc4f0 0x7fb4ec14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4f10b5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4ec0ce230 0x7fb4ec1599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4f10b5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ec15b980 0x7fb4ec159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4f10b5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec06e8a0 con 0x7fb4ec15b980 2024-04-25T10:07:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4f10b5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec071d60 con 0x7fb4ec0cc4f0 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4f10b5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec0ae500 con 0x7fb4ec0ce230 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4eb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ec15b980 0x7fb4ec159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4eb59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ec15b980 0x7fb4ec159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38866/0 (socket says 172.21.15.19:38866) 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4eb59e640 1 -- 172.21.15.19:0/684402651 learned_addr learned my addr 172.21.15.19:0/684402651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4ead9d640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4ec0cc4f0 0x7fb4ec14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4e3fff640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4ec0ce230 0x7fb4ec1599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4eb59e640 1 -- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4ec0ce230 msgr2=0x7fb4ec1599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4eb59e640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4ec0ce230 0x7fb4ec1599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4eb59e640 1 -- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4ec0cc4f0 msgr2=0x7fb4ec14ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4eb59e640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4ec0cc4f0 0x7fb4ec14ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.931+0000 7fb4eb59e640 1 -- 172.21.15.19:0/684402651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec0aac50 con 0x7fb4ec15b980 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.932+0000 7fb4eb59e640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ec15b980 0x7fb4ec159ee0 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009d90 tx=0x7fb4e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.932+0000 7fb4e37fe640 1 -- 172.21.15.19:0/684402651 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4040070 con 0x7fb4ec15b980 2024-04-25T10:07:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.932+0000 7fb4e37fe640 1 -- 172.21.15.19:0/684402651 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e4031bc0 con 0x7fb4ec15b980 2024-04-25T10:07:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.932+0000 7fb4f10b5640 1 -- 172.21.15.19:0/684402651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ec0a84a0 con 0x7fb4ec15b980 2024-04-25T10:07:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.932+0000 7fb4e37fe640 1 -- 172.21.15.19:0/684402651 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4009450 con 0x7fb4ec15b980 2024-04-25T10:07:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.932+0000 7fb4f10b5640 1 -- 172.21.15.19:0/684402651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ec0adbe0 con 0x7fb4ec15b980 2024-04-25T10:07:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.933+0000 7fb4f10b5640 1 -- 172.21.15.19:0/684402651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ec073a00 con 0x7fb4ec15b980 2024-04-25T10:07:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.933+0000 7fb4e37fe640 1 -- 172.21.15.19:0/684402651 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4e4039040 con 0x7fb4ec15b980 2024-04-25T10:07:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.934+0000 7fb4e37fe640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4cc06b500 0x7fb4cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.934+0000 7fb4e37fe640 1 -- 172.21.15.19:0/684402651 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb4e40ab3f0 con 0x7fb4ec15b980 2024-04-25T10:07:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.934+0000 7fb4ead9d640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4cc06b500 0x7fb4cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.934+0000 7fb4ead9d640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4cc06b500 0x7fb4cc06d9c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc007fd0 tx=0x7fb4dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:11.938+0000 7fb4e37fe640 1 -- 172.21.15.19:0/684402651 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4e4079ca0 con 0x7fb4ec15b980 2024-04-25T10:07:12.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.119+0000 7fb4f10b5640 1 -- 172.21.15.19:0/684402651 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4ec0ceee0 con 0x7fb4cc06b500 2024-04-25T10:07:12.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.119+0000 7fb4e37fe640 1 -- 172.21.15.19:0/684402651 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb4ec0ceee0 con 0x7fb4cc06b500 2024-04-25T10:07:12.125 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 -- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4cc06b500 msgr2=0x7fb4cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4cc06b500 0x7fb4cc06d9c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc007fd0 tx=0x7fb4dc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 -- 172.21.15.19:0/684402651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ec15b980 msgr2=0x7fb4ec159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ec15b980 0x7fb4ec159ee0 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009d90 tx=0x7fb4e4004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 -- 172.21.15.19:0/684402651 shutdown_connections 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fb4cc06b500 0x7fb4cc06d9c0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ec15b980 0x7fb4ec159ee0 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4ec0ce230 0x7fb4ec1599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 --2- 172.21.15.19:0/684402651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4ec0cc4f0 0x7fb4ec14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 -- 172.21.15.19:0/684402651 >> 172.21.15.19:0/684402651 conn(0x7fb4ec06abe0 msgr2=0x7fb4ec15d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 -- 172.21.15.19:0/684402651 shutdown_connections 2024-04-25T10:07:12.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.121+0000 7fb4f10b5640 1 -- 172.21.15.19:0/684402651 wait complete. 2024-04-25T10:07:12.138 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:07:12.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:07:12.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.237+0000 7f08569e0640 1 Processor -- start 2024-04-25T10:07:12.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.237+0000 7f08569e0640 1 -- start start 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.238+0000 7f08569e0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08500cf600 0x7f08500cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.238+0000 7f08569e0640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08500c75e0 con 0x7f085015bb50 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.238+0000 7f08569e0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08500c78c0 con 0x7f08500cbf20 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.238+0000 7f08569e0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08500c8680 con 0x7f08500cf600 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.238+0000 7f084ffff640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f085015bb50 0x7f08500c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54300/0 (socket says 172.21.15.19:54300) 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.238+0000 7f084ffff640 1 -- 172.21.15.19:0/3930653232 learned_addr learned my addr 172.21.15.19:0/3930653232 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.239+0000 7f084f7fe640 1 --2- 172.21.15.19:0/3930653232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08500cf600 0x7f08500cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.239+0000 7f084effd640 1 -- 172.21.15.19:0/3930653232 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402449585 0 0) 0x7f08500c75e0 con 0x7f085015bb50 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.239+0000 7f084effd640 1 -- 172.21.15.19:0/3930653232 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0838003520 con 0x7f085015bb50 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.239+0000 7f084effd640 1 -- 172.21.15.19:0/3930653232 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148582798 0 0) 0x7f08500c78c0 con 0x7f08500cbf20 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.239+0000 7f084effd640 1 -- 172.21.15.19:0/3930653232 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08500c75e0 con 0x7f08500cbf20 2024-04-25T10:07:12.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.239+0000 7f084f7fe640 1 -- 172.21.15.19:0/3930653232 >> v1:172.21.15.112:6789/0 conn(0x7f08500cbf20 legacy=0x7f08500cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.239+0000 7f084f7fe640 1 -- 172.21.15.19:0/3930653232 >> v1:172.21.15.19:6789/0 conn(0x7f085015bb50 legacy=0x7f08500c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.239+0000 7f084f7fe640 1 -- 172.21.15.19:0/3930653232 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08500c7ba0 con 0x7f08500cf600 2024-04-25T10:07:12.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.240+0000 7f084f7fe640 1 --2- 172.21.15.19:0/3930653232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08500cf600 0x7f08500cfa40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f08440097d0 tx=0x7f084402a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38f1ac3e21008347 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.240+0000 7f084effd640 1 -- 172.21.15.19:0/3930653232 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0844037070 con 0x7f08500cf600 2024-04-25T10:07:12.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.240+0000 7f084effd640 1 -- 172.21.15.19:0/3930653232 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0844032440 con 0x7f08500cf600 2024-04-25T10:07:12.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.240+0000 7f084effd640 1 -- 172.21.15.19:0/3930653232 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08440327f0 con 0x7f08500cf600 2024-04-25T10:07:12.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.240+0000 7f08569e0640 1 -- 172.21.15.19:0/3930653232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08500cf600 msgr2=0x7f08500cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.240+0000 7f08569e0640 1 --2- 172.21.15.19:0/3930653232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08500cf600 0x7f08500cfa40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f08440097d0 tx=0x7f084402a290 comp rx=0 tx=0).stop 2024-04-25T10:07:12.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.240+0000 7f08569e0640 1 -- 172.21.15.19:0/3930653232 shutdown_connections 2024-04-25T10:07:12.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.240+0000 7f08569e0640 1 --2- 172.21.15.19:0/3930653232 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08500cf600 0x7f08500cfa40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.240+0000 7f08569e0640 1 -- 172.21.15.19:0/3930653232 >> 172.21.15.19:0/3930653232 conn(0x7f085006f500 msgr2=0x7f085006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.240+0000 7f08569e0640 1 -- 172.21.15.19:0/3930653232 shutdown_connections 2024-04-25T10:07:12.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.240+0000 7f08569e0640 1 -- 172.21.15.19:0/3930653232 wait complete. 2024-04-25T10:07:12.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.241+0000 7f08569e0640 1 Processor -- start 2024-04-25T10:07:12.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.242+0000 7f08569e0640 1 -- start start 2024-04-25T10:07:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.242+0000 7f08569e0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08500cbf20 0x7f08500cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.242+0000 7f08569e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500cf600 0x7f08500b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.242+0000 7f08569e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085015bb50 0x7f08500b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.242+0000 7f08569e0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08500c8160 con 0x7f085015bb50 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.242+0000 7f08569e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08500cff80 con 0x7f08500cf600 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.242+0000 7f08569e0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08500c72a0 con 0x7f08500cbf20 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f0854f7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085015bb50 0x7f08500b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f0854f7d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085015bb50 0x7f08500b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38868/0 (socket says 172.21.15.19:38868) 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f0854f7d640 1 -- 172.21.15.19:0/3822279091 learned_addr learned my addr 172.21.15.19:0/3822279091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f084ffff640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08500cbf20 0x7f08500cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f084f7fe640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500cf600 0x7f08500b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f0854f7d640 1 -- 172.21.15.19:0/3822279091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08500cbf20 msgr2=0x7f08500cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f0854f7d640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08500cbf20 0x7f08500cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f0854f7d640 1 -- 172.21.15.19:0/3822279091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500cf600 msgr2=0x7f08500b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f0854f7d640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500cf600 0x7f08500b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f0854f7d640 1 -- 172.21.15.19:0/3822279091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08500b5670 con 0x7f085015bb50 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f0854f7d640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085015bb50 0x7f08500b5130 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f084800c7c0 tx=0x7f084800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f084cff9640 1 -- 172.21.15.19:0/3822279091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848003b70 con 0x7f085015bb50 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f084cff9640 1 -- 172.21.15.19:0/3822279091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0848003d10 con 0x7f085015bb50 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f08569e0640 1 -- 172.21.15.19:0/3822279091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08500c9b90 con 0x7f085015bb50 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f084cff9640 1 -- 172.21.15.19:0/3822279091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084800aa50 con 0x7f085015bb50 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.243+0000 7f08569e0640 1 -- 172.21.15.19:0/3822279091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08500ca150 con 0x7f085015bb50 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.244+0000 7f08569e0640 1 -- 172.21.15.19:0/3822279091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08500c9940 con 0x7f085015bb50 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.245+0000 7f084cff9640 1 -- 172.21.15.19:0/3822279091 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f084800abf0 con 0x7f085015bb50 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.245+0000 7f084cff9640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f083c06b4d0 0x7f083c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.245+0000 7f084cff9640 1 -- 172.21.15.19:0/3822279091 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f0848089f40 con 0x7f085015bb50 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.245+0000 7f084cff9640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] conn(0x7f083c071a10 0x7f083c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.245+0000 7f084cff9640 1 -- 172.21.15.19:0/3822279091 --> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f083c074520 con 0x7f083c071a10 2024-04-25T10:07:12.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.245+0000 7f084ffff640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f083c06b4d0 0x7f083c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.245+0000 7f084cff9640 1 -- 172.21.15.19:0/3822279091 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f084808c5e0 con 0x7f085015bb50 2024-04-25T10:07:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.245+0000 7f084f7fe640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] conn(0x7f083c071a10 0x7f083c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.246+0000 7f084ffff640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f083c06b4d0 0x7f083c06d990 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f08400011e0 tx=0x7f0840007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.246+0000 7f084f7fe640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] conn(0x7f083c071a10 0x7f083c073e50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.246+0000 7f084cff9640 1 -- 172.21.15.19:0/3822279091 <== osd.1 v2:172.21.15.19:6808/495652536 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f083c074520 con 0x7f083c071a10 2024-04-25T10:07:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.259+0000 7f08569e0640 1 -- 172.21.15.19:0/3822279091 --> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f08500b2dc0 con 0x7f083c071a10 2024-04-25T10:07:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f084cff9640 1 -- 172.21.15.19:0/3822279091 <== osd.1 v2:172.21.15.19:6808/495652536 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f08500b2dc0 con 0x7f083c071a10 2024-04-25T10:07:12.265 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:07:12.266 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:07:12.266 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:07:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f08569e0640 1 -- 172.21.15.19:0/3822279091 >> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] conn(0x7f083c071a10 msgr2=0x7f083c073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f08569e0640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.19:6808/495652536,v1:172.21.15.19:6809/495652536] conn(0x7f083c071a10 0x7f083c073e50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f08569e0640 1 -- 172.21.15.19:0/3822279091 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f083c06b4d0 msgr2=0x7f083c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f08569e0640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f083c06b4d0 0x7f083c06d990 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f08400011e0 tx=0x7f0840007d30 comp rx=0 tx=0).stop 2024-04-25T10:07:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f08569e0640 1 -- 172.21.15.19:0/3822279091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085015bb50 msgr2=0x7f08500b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f08569e0640 1 --2- 172.21.15.19:0/3822279091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085015bb50 0x7f08500b5130 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f084800c7c0 tx=0x7f084800cc80 comp rx=0 tx=0).stop 2024-04-25T10:07:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f0854f7d640 1 -- 172.21.15.19:0/3822279091 reap_dead start 2024-04-25T10:07:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f08569e0640 1 -- 172.21.15.19:0/3822279091 shutdown_connections 2024-04-25T10:07:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f08569e0640 1 -- 172.21.15.19:0/3822279091 >> 172.21.15.19:0/3822279091 conn(0x7f085006f500 msgr2=0x7f085006f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f08569e0640 1 -- 172.21.15.19:0/3822279091 shutdown_connections 2024-04-25T10:07:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.260+0000 7f08569e0640 1 -- 172.21.15.19:0/3822279091 wait complete. 2024-04-25T10:07:12.276 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:07:12.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.372+0000 7fa654dce640 1 Processor -- start 2024-04-25T10:07:12.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.372+0000 7fa654dce640 1 -- start start 2024-04-25T10:07:12.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa654dce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa65015da90 0x7fa650161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa654dce640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa6500b9750 0x7fa6500ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa654dce640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa650162040 con 0x7fa650161ba0 2024-04-25T10:07:12.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa654dce640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa650162600 con 0x7fa6500b9750 2024-04-25T10:07:12.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa654dce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa650162320 con 0x7fa65015da90 2024-04-25T10:07:12.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa64dd9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa6500b9750 0x7fa6500ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa64dd9b640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa6500b9750 0x7fa6500ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38870/0 (socket says 172.21.15.19:38870) 2024-04-25T10:07:12.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa64dd9b640 1 -- 172.21.15.19:0/84424452 learned_addr learned my addr 172.21.15.19:0/84424452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:12.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa64e59c640 1 --2- 172.21.15.19:0/84424452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa65015da90 0x7fa650161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa64dd9b640 1 -- 172.21.15.19:0/84424452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa65015da90 msgr2=0x7fa650161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa64dd9b640 1 --2- 172.21.15.19:0/84424452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa65015da90 0x7fa650161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa64dd9b640 1 -- 172.21.15.19:0/84424452 >> v1:172.21.15.112:6789/0 conn(0x7fa650161ba0 legacy=0x7fa65006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.373+0000 7fa64dd9b640 1 -- 172.21.15.19:0/84424452 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500d1b10 con 0x7fa6500b9750 2024-04-25T10:07:12.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.374+0000 7fa64dd9b640 1 --2- 172.21.15.19:0/84424452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6500b9750 0x7fa6500ce670 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7fa644009870 tx=0x7fa64402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d693ce52a9518996 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.374+0000 7fa64d59a640 1 -- 172.21.15.19:0/84424452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644037070 con 0x7fa6500b9750 2024-04-25T10:07:12.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.374+0000 7fa64d59a640 1 -- 172.21.15.19:0/84424452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa644032440 con 0x7fa6500b9750 2024-04-25T10:07:12.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.374+0000 7fa64d59a640 1 -- 172.21.15.19:0/84424452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644032810 con 0x7fa6500b9750 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.375+0000 7fa654dce640 1 -- 172.21.15.19:0/84424452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6500b9750 msgr2=0x7fa6500ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.375+0000 7fa654dce640 1 --2- 172.21.15.19:0/84424452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6500b9750 0x7fa6500ce670 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7fa644009870 tx=0x7fa64402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.375+0000 7fa654dce640 1 -- 172.21.15.19:0/84424452 shutdown_connections 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.375+0000 7fa654dce640 1 --2- 172.21.15.19:0/84424452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6500b9750 0x7fa6500ce670 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.375+0000 7fa654dce640 1 --2- 172.21.15.19:0/84424452 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa65015da90 0x7fa650161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.375+0000 7fa654dce640 1 -- 172.21.15.19:0/84424452 >> 172.21.15.19:0/84424452 conn(0x7fa650061a20 msgr2=0x7fa650061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.375+0000 7fa654dce640 1 -- 172.21.15.19:0/84424452 shutdown_connections 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.375+0000 7fa654dce640 1 -- 172.21.15.19:0/84424452 wait complete. 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.376+0000 7fa654dce640 1 Processor -- start 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.376+0000 7fa654dce640 1 -- start start 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa654dce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa65015da90 0x7fa6501517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa654dce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa650161ba0 0x7fa6500c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa654dce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6500c8370 0x7fa65015a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa654dce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500d3320 con 0x7fa65015da90 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa654dce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65015de70 con 0x7fa6500c8370 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa654dce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500d34a0 con 0x7fa650161ba0 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa64e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa65015da90 0x7fa6501517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa64e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa65015da90 0x7fa6501517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38872/0 (socket says 172.21.15.19:38872) 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa64e59c640 1 -- 172.21.15.19:0/1023431670 learned_addr learned my addr 172.21.15.19:0/1023431670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa64dd9b640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa650161ba0 0x7fa6500c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa64ed9d640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6500c8370 0x7fa65015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa64e59c640 1 -- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa650161ba0 msgr2=0x7fa6500c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa64e59c640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa650161ba0 0x7fa6500c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa64e59c640 1 -- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6500c8370 msgr2=0x7fa65015a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa64e59c640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6500c8370 0x7fa65015a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.377+0000 7fa64e59c640 1 -- 172.21.15.19:0/1023431670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500c8920 con 0x7fa65015da90 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.378+0000 7fa64e59c640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa65015da90 0x7fa6501517a0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fa64800e9b0 tx=0x7fa64800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.378+0000 7fa63effd640 1 -- 172.21.15.19:0/1023431670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648017070 con 0x7fa65015da90 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.378+0000 7fa654dce640 1 -- 172.21.15.19:0/1023431670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6500c8c00 con 0x7fa65015da90 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.378+0000 7fa63effd640 1 -- 172.21.15.19:0/1023431670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64800c9a0 con 0x7fa65015da90 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.378+0000 7fa63effd640 1 -- 172.21.15.19:0/1023431670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64800cd90 con 0x7fa65015da90 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.378+0000 7fa654dce640 1 -- 172.21.15.19:0/1023431670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6500c91c0 con 0x7fa65015da90 2024-04-25T10:07:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.379+0000 7fa654dce640 1 -- 172.21.15.19:0/1023431670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa650073a00 con 0x7fa65015da90 2024-04-25T10:07:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.379+0000 7fa63effd640 1 -- 172.21.15.19:0/1023431670 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa64801a070 con 0x7fa65015da90 2024-04-25T10:07:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.380+0000 7fa63effd640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa63006b4d0 0x7fa63006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.380+0000 7fa63effd640 1 -- 172.21.15.19:0/1023431670 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa64808a460 con 0x7fa65015da90 2024-04-25T10:07:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.380+0000 7fa64dd9b640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa63006b4d0 0x7fa63006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.380+0000 7fa64dd9b640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa63006b4d0 0x7fa63006d990 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa644034000 tx=0x7fa64402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.384+0000 7fa63effd640 1 -- 172.21.15.19:0/1023431670 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa648058d30 con 0x7fa65015da90 2024-04-25T10:07:12.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.566+0000 7fa654dce640 1 -- 172.21.15.19:0/1023431670 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6500d1590 con 0x7fa63006b4d0 2024-04-25T10:07:12.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.567+0000 7fa63effd640 1 -- 172.21.15.19:0/1023431670 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa6500d1590 con 0x7fa63006b4d0 2024-04-25T10:07:12.572 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 -- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa63006b4d0 msgr2=0x7fa63006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa63006b4d0 0x7fa63006d990 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa644034000 tx=0x7fa64402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 -- 172.21.15.19:0/1023431670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa65015da90 msgr2=0x7fa6501517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa65015da90 0x7fa6501517a0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fa64800e9b0 tx=0x7fa64800ee70 comp rx=0 tx=0).stop 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 -- 172.21.15.19:0/1023431670 shutdown_connections 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa63006b4d0 0x7fa63006d990 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6500c8370 0x7fa65015a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa650161ba0 0x7fa6500c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 --2- 172.21.15.19:0/1023431670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa65015da90 0x7fa6501517a0 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 -- 172.21.15.19:0/1023431670 >> 172.21.15.19:0/1023431670 conn(0x7fa650061a20 msgr2=0x7fa6500b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 -- 172.21.15.19:0/1023431670 shutdown_connections 2024-04-25T10:07:12.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.569+0000 7fa654dce640 1 -- 172.21.15.19:0/1023431670 wait complete. 2024-04-25T10:07:12.586 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:07:12.587 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:07:12.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.686+0000 7fe5dd646640 1 Processor -- start 2024-04-25T10:07:12.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.686+0000 7fe5dd646640 1 -- start start 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5dd646640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe5d806c330 0x7fe5d80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5dd646640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d80d1d50 con 0x7fe5d806c7e0 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5dd646640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d80d2020 con 0x7fe5d8158230 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5dd646640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d8158610 con 0x7fe5d806c330 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5cffff640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe5d806c330 0x7fe5d80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5d77fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe5d806c7e0 0x7fe5d806f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54304/0 (socket says 172.21.15.19:54304) 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5d77fe640 1 -- 172.21.15.19:0/3891757050 learned_addr learned my addr 172.21.15.19:0/3891757050 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5d67fc640 1 -- 172.21.15.19:0/3891757050 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112301025 0 0) 0x7fe5d80d1d50 con 0x7fe5d806c7e0 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5d67fc640 1 -- 172.21.15.19:0/3891757050 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5c4003520 con 0x7fe5d806c7e0 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5cffff640 1 -- 172.21.15.19:0/3891757050 >> v1:172.21.15.112:6789/0 conn(0x7fe5d8158230 legacy=0x7fe5d80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5cffff640 1 -- 172.21.15.19:0/3891757050 >> v1:172.21.15.19:6789/0 conn(0x7fe5d806c7e0 legacy=0x7fe5d806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5d67fc640 1 -- 172.21.15.19:0/3891757050 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532303412 0 0) 0x7fe5d80d2020 con 0x7fe5d8158230 2024-04-25T10:07:12.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.687+0000 7fe5cffff640 1 -- 172.21.15.19:0/3891757050 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d80d2300 con 0x7fe5d806c330 2024-04-25T10:07:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.688+0000 7fe5cffff640 1 --2- 172.21.15.19:0/3891757050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5d806c330 0x7fe5d80b4e60 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009790 tx=0x7fe5c002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb8f3ff2654c51f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.688+0000 7fe5d67fc640 1 -- 172.21.15.19:0/3891757050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0037070 con 0x7fe5d806c330 2024-04-25T10:07:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.688+0000 7fe5d67fc640 1 -- 172.21.15.19:0/3891757050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c0032440 con 0x7fe5d806c330 2024-04-25T10:07:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.688+0000 7fe5d67fc640 1 -- 172.21.15.19:0/3891757050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0032810 con 0x7fe5d806c330 2024-04-25T10:07:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.689+0000 7fe5dd646640 1 -- 172.21.15.19:0/3891757050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5d806c330 msgr2=0x7fe5d80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.689+0000 7fe5dd646640 1 --2- 172.21.15.19:0/3891757050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5d806c330 0x7fe5d80b4e60 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009790 tx=0x7fe5c002a210 comp rx=0 tx=0).stop 2024-04-25T10:07:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.689+0000 7fe5dd646640 1 -- 172.21.15.19:0/3891757050 shutdown_connections 2024-04-25T10:07:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.689+0000 7fe5dd646640 1 --2- 172.21.15.19:0/3891757050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5d806c330 0x7fe5d80b4e60 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.689+0000 7fe5dd646640 1 -- 172.21.15.19:0/3891757050 >> 172.21.15.19:0/3891757050 conn(0x7fe5d8061600 msgr2=0x7fe5d80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.689+0000 7fe5dd646640 1 -- 172.21.15.19:0/3891757050 shutdown_connections 2024-04-25T10:07:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.689+0000 7fe5dd646640 1 -- 172.21.15.19:0/3891757050 wait complete. 2024-04-25T10:07:12.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.690+0000 7fe5dd646640 1 Processor -- start 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.690+0000 7fe5dd646640 1 -- start start 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5dd646640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5d806c330 0x7fe5d80cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5dd646640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5d806c7e0 0x7fe5d815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5dd646640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5d8158230 0x7fe5d8145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5dd646640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80d25e0 con 0x7fe5d8158230 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5dd646640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d806bea0 con 0x7fe5d806c330 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5dd646640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d815e320 con 0x7fe5d806c7e0 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5d8158230 0x7fe5d8145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5d8158230 0x7fe5d8145b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38884/0 (socket says 172.21.15.19:38884) 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5d77fe640 1 -- 172.21.15.19:0/2488571467 learned_addr learned my addr 172.21.15.19:0/2488571467 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5d6ffd640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5d806c330 0x7fe5d80cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5cffff640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5d806c7e0 0x7fe5d815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5d77fe640 1 -- 172.21.15.19:0/2488571467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5d806c7e0 msgr2=0x7fe5d815abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5d77fe640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5d806c7e0 0x7fe5d815abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5d77fe640 1 -- 172.21.15.19:0/2488571467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5d806c330 msgr2=0x7fe5d80cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5d77fe640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5d806c330 0x7fe5d80cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5d77fe640 1 -- 172.21.15.19:0/2488571467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d8150b90 con 0x7fe5d8158230 2024-04-25T10:07:12.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.691+0000 7fe5d77fe640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5d8158230 0x7fe5d8145b60 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000caa0 tx=0x7fe5d000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.692+0000 7fe5cf7fe640 1 -- 172.21.15.19:0/2488571467 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0017070 con 0x7fe5d8158230 2024-04-25T10:07:12.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.692+0000 7fe5cf7fe640 1 -- 172.21.15.19:0/2488571467 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d000ac00 con 0x7fe5d8158230 2024-04-25T10:07:12.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.692+0000 7fe5dd646640 1 -- 172.21.15.19:0/2488571467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d8159b60 con 0x7fe5d8158230 2024-04-25T10:07:12.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.692+0000 7fe5cf7fe640 1 -- 172.21.15.19:0/2488571467 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0012400 con 0x7fe5d8158230 2024-04-25T10:07:12.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.692+0000 7fe5dd646640 1 -- 172.21.15.19:0/2488571467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d815a100 con 0x7fe5d8158230 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.693+0000 7fe5dd646640 1 -- 172.21.15.19:0/2488571467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe5d806de50 con 0x7fe5d8158230 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.693+0000 7fe5cf7fe640 1 -- 172.21.15.19:0/2488571467 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5d001a070 con 0x7fe5d8158230 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.694+0000 7fe5cf7fe640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe5b806b4d0 0x7fe5b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.694+0000 7fe5cf7fe640 1 -- 172.21.15.19:0/2488571467 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe5d0089da0 con 0x7fe5d8158230 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.694+0000 7fe5cf7fe640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x7fe5b8071a10 0x7fe5b8073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.694+0000 7fe5cf7fe640 1 -- 172.21.15.19:0/2488571467 --> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5b8074520 con 0x7fe5b8071a10 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.694+0000 7fe5cf7fe640 1 -- 172.21.15.19:0/2488571467 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5d008c4a0 con 0x7fe5d8158230 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.694+0000 7fe5d6ffd640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe5b806b4d0 0x7fe5b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.694+0000 7fe5cffff640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x7fe5b8071a10 0x7fe5b8073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.694+0000 7fe5cffff640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x7fe5b8071a10 0x7fe5b8073e50 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.694+0000 7fe5d6ffd640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe5b806b4d0 0x7fe5b806d990 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8000f60 tx=0x7fe5c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.695+0000 7fe5cf7fe640 1 -- 172.21.15.19:0/2488571467 <== osd.2 v2:172.21.15.19:6816/3205329770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe5b8074520 con 0x7fe5b8071a10 2024-04-25T10:07:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.708+0000 7fe5dd646640 1 -- 172.21.15.19:0/2488571467 --> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe5d815a8e0 con 0x7fe5b8071a10 2024-04-25T10:07:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.708+0000 7fe5cf7fe640 1 -- 172.21.15.19:0/2488571467 <== osd.2 v2:172.21.15.19:6816/3205329770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe5d815a8e0 con 0x7fe5b8071a10 2024-04-25T10:07:12.714 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:07:12.714 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:07:12.714 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:07:12.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.709+0000 7fe5dd646640 1 -- 172.21.15.19:0/2488571467 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x7fe5b8071a10 msgr2=0x7fe5b8073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.709+0000 7fe5dd646640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.19:6816/3205329770,v1:172.21.15.19:6817/3205329770] conn(0x7fe5b8071a10 0x7fe5b8073e50 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.709+0000 7fe5dd646640 1 -- 172.21.15.19:0/2488571467 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe5b806b4d0 msgr2=0x7fe5b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.709+0000 7fe5dd646640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fe5b806b4d0 0x7fe5b806d990 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8000f60 tx=0x7fe5c8009040 comp rx=0 tx=0).stop 2024-04-25T10:07:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.709+0000 7fe5dd646640 1 -- 172.21.15.19:0/2488571467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5d8158230 msgr2=0x7fe5d8145b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.709+0000 7fe5dd646640 1 --2- 172.21.15.19:0/2488571467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5d8158230 0x7fe5d8145b60 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000caa0 tx=0x7fe5d000cf60 comp rx=0 tx=0).stop 2024-04-25T10:07:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.709+0000 7fe5d77fe640 1 -- 172.21.15.19:0/2488571467 reap_dead start 2024-04-25T10:07:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.709+0000 7fe5dd646640 1 -- 172.21.15.19:0/2488571467 shutdown_connections 2024-04-25T10:07:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.709+0000 7fe5dd646640 1 -- 172.21.15.19:0/2488571467 >> 172.21.15.19:0/2488571467 conn(0x7fe5d8061600 msgr2=0x7fe5d80b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.709+0000 7fe5dd646640 1 -- 172.21.15.19:0/2488571467 shutdown_connections 2024-04-25T10:07:12.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.709+0000 7fe5dd646640 1 -- 172.21.15.19:0/2488571467 wait complete. 2024-04-25T10:07:12.725 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:07:12.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.820+0000 7fdccee5a640 1 Processor -- start 2024-04-25T10:07:12.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.820+0000 7fdccee5a640 1 -- start start 2024-04-25T10:07:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.821+0000 7fdccee5a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdcc806c9e0 0x7fdcc80b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.821+0000 7fdccee5a640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc806cf50 con 0x7fdcc815b7d0 2024-04-25T10:07:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.821+0000 7fdccee5a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc80ce530 con 0x7fdcc80b5620 2024-04-25T10:07:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.821+0000 7fdccee5a640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc80ce810 con 0x7fdcc806c9e0 2024-04-25T10:07:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.821+0000 7fdccd3f7640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdcc806c9e0 0x7fdcc80b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.821+0000 7fdcccbf6640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdcc815b7d0 0x7fdcc80cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54314/0 (socket says 172.21.15.19:54314) 2024-04-25T10:07:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.821+0000 7fdcccbf6640 1 -- 172.21.15.19:0/3860296098 learned_addr learned my addr 172.21.15.19:0/3860296098 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.822+0000 7fdccd3f7640 1 -- 172.21.15.19:0/3860296098 >> v1:172.21.15.112:6789/0 conn(0x7fdcc80b5620 legacy=0x7fdcc806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.822+0000 7fdccd3f7640 1 -- 172.21.15.19:0/3860296098 >> v1:172.21.15.19:6789/0 conn(0x7fdcc815b7d0 legacy=0x7fdcc80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.822+0000 7fdccd3f7640 1 -- 172.21.15.19:0/3860296098 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc80ceaf0 con 0x7fdcc806c9e0 2024-04-25T10:07:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.822+0000 7fdccd3f7640 1 --2- 172.21.15.19:0/3860296098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc806c9e0 0x7fdcc80b5ab0 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009d90 tx=0x7fdcc002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582229e9f5543ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.823+0000 7fdcc77fe640 1 -- 172.21.15.19:0/3860296098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0007740 con 0x7fdcc806c9e0 2024-04-25T10:07:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.823+0000 7fdcc77fe640 1 -- 172.21.15.19:0/3860296098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc00078e0 con 0x7fdcc806c9e0 2024-04-25T10:07:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.823+0000 7fdcc77fe640 1 -- 172.21.15.19:0/3860296098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0007cb0 con 0x7fdcc806c9e0 2024-04-25T10:07:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.823+0000 7fdccee5a640 1 -- 172.21.15.19:0/3860296098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc806c9e0 msgr2=0x7fdcc80b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.823+0000 7fdccee5a640 1 --2- 172.21.15.19:0/3860296098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc806c9e0 0x7fdcc80b5ab0 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009d90 tx=0x7fdcc002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.823+0000 7fdccee5a640 1 -- 172.21.15.19:0/3860296098 shutdown_connections 2024-04-25T10:07:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.823+0000 7fdccee5a640 1 --2- 172.21.15.19:0/3860296098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc806c9e0 0x7fdcc80b5ab0 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.823+0000 7fdccee5a640 1 -- 172.21.15.19:0/3860296098 >> 172.21.15.19:0/3860296098 conn(0x7fdcc80d0800 msgr2=0x7fdcc80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.823+0000 7fdccee5a640 1 -- 172.21.15.19:0/3860296098 shutdown_connections 2024-04-25T10:07:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.823+0000 7fdccee5a640 1 -- 172.21.15.19:0/3860296098 wait complete. 2024-04-25T10:07:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.824+0000 7fdccee5a640 1 Processor -- start 2024-04-25T10:07:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.824+0000 7fdccee5a640 1 -- start start 2024-04-25T10:07:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.825+0000 7fdccee5a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc806c9e0 0x7fdcc80cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.825+0000 7fdccee5a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdcc80b5620 0x7fdcc80cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.825+0000 7fdccee5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcc815b7d0 0x7fdcc80cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.825+0000 7fdccee5a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc815bd30 con 0x7fdcc806c9e0 2024-04-25T10:07:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.825+0000 7fdccee5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc815bbb0 con 0x7fdcc815b7d0 2024-04-25T10:07:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.825+0000 7fdccee5a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80d1f40 con 0x7fdcc80b5620 2024-04-25T10:07:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.825+0000 7fdcccbf6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc806c9e0 0x7fdcc80cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.825+0000 7fdcccbf6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc806c9e0 0x7fdcc80cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38902/0 (socket says 172.21.15.19:38902) 2024-04-25T10:07:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.825+0000 7fdcccbf6640 1 -- 172.21.15.19:0/2929678684 learned_addr learned my addr 172.21.15.19:0/2929678684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdccd3f7640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcc815b7d0 0x7fdcc80cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdcc7fff640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdcc80b5620 0x7fdcc80cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdcccbf6640 1 -- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdcc80b5620 msgr2=0x7fdcc80cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdcccbf6640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdcc80b5620 0x7fdcc80cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdcccbf6640 1 -- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcc815b7d0 msgr2=0x7fdcc80cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdcccbf6640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcc815b7d0 0x7fdcc80cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdcccbf6640 1 -- 172.21.15.19:0/2929678684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc80cc1f0 con 0x7fdcc806c9e0 2024-04-25T10:07:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdcccbf6640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc806c9e0 0x7fdcc80cd6a0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c4b0 tx=0x7fdcb800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdcc57fa640 1 -- 172.21.15.19:0/2929678684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8017070 con 0x7fdcc806c9e0 2024-04-25T10:07:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdcc57fa640 1 -- 172.21.15.19:0/2929678684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb8005ce0 con 0x7fdcc806c9e0 2024-04-25T10:07:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdcc57fa640 1 -- 172.21.15.19:0/2929678684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8007270 con 0x7fdcc806c9e0 2024-04-25T10:07:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdccee5a640 1 -- 172.21.15.19:0/2929678684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc80cc4d0 con 0x7fdcc806c9e0 2024-04-25T10:07:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.826+0000 7fdccee5a640 1 -- 172.21.15.19:0/2929678684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc80cca90 con 0x7fdcc806c9e0 2024-04-25T10:07:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.827+0000 7fdccee5a640 1 -- 172.21.15.19:0/2929678684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc80b60d0 con 0x7fdcc806c9e0 2024-04-25T10:07:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.827+0000 7fdcc57fa640 1 -- 172.21.15.19:0/2929678684 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdcb801a070 con 0x7fdcc806c9e0 2024-04-25T10:07:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.828+0000 7fdcc57fa640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdcb406b4d0 0x7fdcb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.828+0000 7fdcc57fa640 1 -- 172.21.15.19:0/2929678684 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fdcb8089270 con 0x7fdcc806c9e0 2024-04-25T10:07:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.828+0000 7fdcc7fff640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdcb406b4d0 0x7fdcb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.828+0000 7fdcc7fff640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdcb406b4d0 0x7fdcb406d990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fdcc815a890 tx=0x7fdcbc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:12.832+0000 7fdcc57fa640 1 -- 172.21.15.19:0/2929678684 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdcc80b60d0 con 0x7fdcc806c9e0 2024-04-25T10:07:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.010+0000 7fdccee5a640 1 -- 172.21.15.19:0/2929678684 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcc8073a00 con 0x7fdcb406b4d0 2024-04-25T10:07:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.011+0000 7fdcc57fa640 1 -- 172.21.15.19:0/2929678684 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdcc8073a00 con 0x7fdcb406b4d0 2024-04-25T10:07:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 -- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdcb406b4d0 msgr2=0x7fdcb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdcb406b4d0 0x7fdcb406d990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fdcc815a890 tx=0x7fdcbc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 -- 172.21.15.19:0/2929678684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc806c9e0 msgr2=0x7fdcc80cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc806c9e0 0x7fdcc80cd6a0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c4b0 tx=0x7fdcb800c970 comp rx=0 tx=0).stop 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 -- 172.21.15.19:0/2929678684 shutdown_connections 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdcb406b4d0 0x7fdcb406d990 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcc815b7d0 0x7fdcc80cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdcc80b5620 0x7fdcc80cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 --2- 172.21.15.19:0/2929678684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcc806c9e0 0x7fdcc80cd6a0 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 -- 172.21.15.19:0/2929678684 >> 172.21.15.19:0/2929678684 conn(0x7fdcc80d0800 msgr2=0x7fdcc80cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 -- 172.21.15.19:0/2929678684 shutdown_connections 2024-04-25T10:07:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.013+0000 7fdccee5a640 1 -- 172.21.15.19:0/2929678684 wait complete. 2024-04-25T10:07:13.029 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:07:13.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:07:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.128+0000 7f5c04b77640 1 Processor -- start 2024-04-25T10:07:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.128+0000 7f5c04b77640 1 -- start start 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.129+0000 7f5c04b77640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c0015da90 0x7f5c00161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.129+0000 7f5c04b77640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c00161ba0 0x7f5c000d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.129+0000 7f5c04b77640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c00162040 con 0x7f5c000b9750 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.129+0000 7f5c04b77640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c00162600 con 0x7f5c0015da90 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.129+0000 7f5c04b77640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c00162320 con 0x7f5c00161ba0 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.130+0000 7f5bfed9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c00161ba0 0x7f5c000d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.130+0000 7f5bfe59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c0015da90 0x7f5c00161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.130+0000 7f5bfed9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c00161ba0 0x7f5c000d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:56828/0 (socket says 172.21.15.19:56828) 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.130+0000 7f5bfed9d640 1 -- 172.21.15.19:0/403966859 learned_addr learned my addr 172.21.15.19:0/403966859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.130+0000 7f5bfed9d640 1 -- 172.21.15.19:0/403966859 >> v2:172.21.15.112:3300/0 conn(0x7f5c0015da90 msgr2=0x7f5c00161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.130+0000 7f5bfed9d640 1 --2- 172.21.15.19:0/403966859 >> v2:172.21.15.112:3300/0 conn(0x7f5c0015da90 0x7f5c00161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.130+0000 7f5bfed9d640 1 -- 172.21.15.19:0/403966859 >> v1:172.21.15.112:6789/0 conn(0x7f5c000b9750 legacy=0x7f5c000ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.130+0000 7f5bfed9d640 1 -- 172.21.15.19:0/403966859 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c000d1b10 con 0x7f5c00161ba0 2024-04-25T10:07:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.130+0000 7f5bfe59c640 1 --2- 172.21.15.19:0/403966859 >> v2:172.21.15.112:3300/0 conn(0x7f5c0015da90 0x7f5c00161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.131+0000 7f5bfed9d640 1 --2- 172.21.15.19:0/403966859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c00161ba0 0x7f5c000d2310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009e50 tx=0x7f5bf802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab869a0375c175e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.131+0000 7f5bfd59a640 1 -- 172.21.15.19:0/403966859 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8036070 con 0x7f5c00161ba0 2024-04-25T10:07:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.131+0000 7f5bfd59a640 1 -- 172.21.15.19:0/403966859 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8031600 con 0x7f5c00161ba0 2024-04-25T10:07:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.131+0000 7f5bfd59a640 1 -- 172.21.15.19:0/403966859 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80319d0 con 0x7f5c00161ba0 2024-04-25T10:07:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.132+0000 7f5c04b77640 1 -- 172.21.15.19:0/403966859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c00161ba0 msgr2=0x7f5c000d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.132+0000 7f5c04b77640 1 --2- 172.21.15.19:0/403966859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c00161ba0 0x7f5c000d2310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009e50 tx=0x7f5bf802a680 comp rx=0 tx=0).stop 2024-04-25T10:07:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.132+0000 7f5c04b77640 1 -- 172.21.15.19:0/403966859 shutdown_connections 2024-04-25T10:07:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.132+0000 7f5c04b77640 1 --2- 172.21.15.19:0/403966859 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c00161ba0 0x7f5c000d2310 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.132+0000 7f5c04b77640 1 --2- 172.21.15.19:0/403966859 >> v2:172.21.15.112:3300/0 conn(0x7f5c0015da90 0x7f5c00161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.132+0000 7f5c04b77640 1 -- 172.21.15.19:0/403966859 >> 172.21.15.19:0/403966859 conn(0x7f5c00061a20 msgr2=0x7f5c00061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.132+0000 7f5c04b77640 1 -- 172.21.15.19:0/403966859 shutdown_connections 2024-04-25T10:07:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.132+0000 7f5c04b77640 1 -- 172.21.15.19:0/403966859 wait complete. 2024-04-25T10:07:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.133+0000 7f5c04b77640 1 Processor -- start 2024-04-25T10:07:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.133+0000 7f5c04b77640 1 -- start start 2024-04-25T10:07:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5c04b77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c0015da90 0x7f5c00158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5c04b77640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c00161ba0 0x7f5c000b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5c04b77640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c000b52a0 0x7f5c000c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5c04b77640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c000ced80 con 0x7f5c000b52a0 2024-04-25T10:07:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5c04b77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0015de70 con 0x7f5c0015da90 2024-04-25T10:07:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5c04b77640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c000cef00 con 0x7f5c00161ba0 2024-04-25T10:07:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5bfed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c000b52a0 0x7f5c000c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5bfed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c000b52a0 0x7f5c000c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38912/0 (socket says 172.21.15.19:38912) 2024-04-25T10:07:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5bfed9d640 1 -- 172.21.15.19:0/1777990559 learned_addr learned my addr 172.21.15.19:0/1777990559 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5bfe59c640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c0015da90 0x7f5c00158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5bfdd9b640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c00161ba0 0x7f5c000b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5bfed9d640 1 -- 172.21.15.19:0/1777990559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c00161ba0 msgr2=0x7f5c000b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5bfed9d640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c00161ba0 0x7f5c000b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5bfed9d640 1 -- 172.21.15.19:0/1777990559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c0015da90 msgr2=0x7f5c00158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5bfed9d640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c0015da90 0x7f5c00158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5bfed9d640 1 -- 172.21.15.19:0/1777990559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c000c8370 con 0x7f5c000b52a0 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.134+0000 7f5bfed9d640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c000b52a0 0x7f5c000c75d0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8034040 tx=0x7f5bf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.135+0000 7f5beeffd640 1 -- 172.21.15.19:0/1777990559 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8036050 con 0x7f5c000b52a0 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.135+0000 7f5beeffd640 1 -- 172.21.15.19:0/1777990559 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8042070 con 0x7f5c000b52a0 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.135+0000 7f5c04b77640 1 -- 172.21.15.19:0/1777990559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c000c8650 con 0x7f5c000b52a0 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.135+0000 7f5beeffd640 1 -- 172.21.15.19:0/1777990559 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8009510 con 0x7f5c000b52a0 2024-04-25T10:07:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.135+0000 7f5c04b77640 1 -- 172.21.15.19:0/1777990559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c000c8c10 con 0x7f5c000b52a0 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.135+0000 7f5c04b77640 1 -- 172.21.15.19:0/1777990559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5c00073a00 con 0x7f5c000b52a0 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.136+0000 7f5beeffd640 1 -- 172.21.15.19:0/1777990559 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bf8007bf0 con 0x7f5c000b52a0 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.136+0000 7f5beeffd640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5be006b4d0 0x7f5be006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.137+0000 7f5beeffd640 1 -- 172.21.15.19:0/1777990559 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5bf80aae20 con 0x7f5c000b52a0 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.137+0000 7f5bfe59c640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5be006b4d0 0x7f5be006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.137+0000 7f5beeffd640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] conn(0x7f5be0071a10 0x7f5be0073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.137+0000 7f5beeffd640 1 -- 172.21.15.19:0/1777990559 --> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bf80020b0 con 0x7f5be0071a10 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.137+0000 7f5beeffd640 1 -- 172.21.15.19:0/1777990559 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bf80ad520 con 0x7f5c000b52a0 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.137+0000 7f5bfdd9b640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] conn(0x7f5be0071a10 0x7f5be0073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.137+0000 7f5bfe59c640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5be006b4d0 0x7f5be006d990 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009560 tx=0x7f5bf0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.137+0000 7f5bfdd9b640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] conn(0x7f5be0071a10 0x7f5be0073e50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.138+0000 7f5beeffd640 1 -- 172.21.15.19:0/1777990559 <== osd.3 v2:172.21.15.19:6824/577725691 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5bf80020b0 con 0x7f5be0071a10 2024-04-25T10:07:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.151+0000 7f5c04b77640 1 -- 172.21.15.19:0/1777990559 --> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5c000d1590 con 0x7f5be0071a10 2024-04-25T10:07:13.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.151+0000 7f5beeffd640 1 -- 172.21.15.19:0/1777990559 <== osd.3 v2:172.21.15.19:6824/577725691 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5c000d1590 con 0x7f5be0071a10 2024-04-25T10:07:13.157 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:07:13.158 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:07:13.158 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:07:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.151+0000 7f5c04b77640 1 -- 172.21.15.19:0/1777990559 >> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] conn(0x7f5be0071a10 msgr2=0x7f5be0073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.151+0000 7f5c04b77640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.19:6824/577725691,v1:172.21.15.19:6825/577725691] conn(0x7f5be0071a10 0x7f5be0073e50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.152+0000 7f5c04b77640 1 -- 172.21.15.19:0/1777990559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5be006b4d0 msgr2=0x7f5be006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.152+0000 7f5c04b77640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f5be006b4d0 0x7f5be006d990 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009560 tx=0x7f5bf0009190 comp rx=0 tx=0).stop 2024-04-25T10:07:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.152+0000 7f5c04b77640 1 -- 172.21.15.19:0/1777990559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c000b52a0 msgr2=0x7f5c000c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.152+0000 7f5c04b77640 1 --2- 172.21.15.19:0/1777990559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c000b52a0 0x7f5c000c75d0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8034040 tx=0x7f5bf8004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.152+0000 7f5bfed9d640 1 -- 172.21.15.19:0/1777990559 reap_dead start 2024-04-25T10:07:13.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.152+0000 7f5c04b77640 1 -- 172.21.15.19:0/1777990559 shutdown_connections 2024-04-25T10:07:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.152+0000 7f5c04b77640 1 -- 172.21.15.19:0/1777990559 >> 172.21.15.19:0/1777990559 conn(0x7f5c00061a20 msgr2=0x7f5c000d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.152+0000 7f5c04b77640 1 -- 172.21.15.19:0/1777990559 shutdown_connections 2024-04-25T10:07:13.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.152+0000 7f5c04b77640 1 -- 172.21.15.19:0/1777990559 wait complete. 2024-04-25T10:07:13.168 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:07:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.264+0000 7f201dbd5640 1 Processor -- start 2024-04-25T10:07:13.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.264+0000 7f201dbd5640 1 -- start start 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.264+0000 7f201dbd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f201815b980 0x7f20180cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.264+0000 7f201dbd5640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2018072280 con 0x7f20180cc4f0 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.264+0000 7f201dbd5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20180714c0 con 0x7f20180ce230 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.264+0000 7f201dbd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201815bdc0 con 0x7f201815b980 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.265+0000 7f20177fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f20180cc4f0 0x7f20180cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54328/0 (socket says 172.21.15.19:54328) 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.265+0000 7f20177fe640 1 -- 172.21.15.19:0/3179713849 learned_addr learned my addr 172.21.15.19:0/3179713849 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.265+0000 7f2017fff640 1 --2- 172.21.15.19:0/3179713849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f201815b980 0x7f20180cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.265+0000 7f20167fc640 1 -- 172.21.15.19:0/3179713849 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249308536 0 0) 0x7f2018072280 con 0x7f20180cc4f0 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.265+0000 7f20167fc640 1 -- 172.21.15.19:0/3179713849 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2000003520 con 0x7f20180cc4f0 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.265+0000 7f2017fff640 1 -- 172.21.15.19:0/3179713849 >> v1:172.21.15.112:6789/0 conn(0x7f20180ce230 legacy=0x7f201815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.265+0000 7f20167fc640 1 -- 172.21.15.19:0/3179713849 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973926427 0 0) 0x7f20180714c0 con 0x7f20180ce230 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.265+0000 7f2017fff640 1 -- 172.21.15.19:0/3179713849 >> v1:172.21.15.19:6789/0 conn(0x7f20180cc4f0 legacy=0x7f20180cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.265+0000 7f2017fff640 1 -- 172.21.15.19:0/3179713849 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20180717a0 con 0x7f201815b980 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.266+0000 7f2017fff640 1 --2- 172.21.15.19:0/3179713849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f201815b980 0x7f20180cc8d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2010009dc0 tx=0x7f201002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f52ebddfcab29f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.266+0000 7f20167fc640 1 -- 172.21.15.19:0/3179713849 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010007740 con 0x7f201815b980 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.266+0000 7f20167fc640 1 -- 172.21.15.19:0/3179713849 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20100078e0 con 0x7f201815b980 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.266+0000 7f20167fc640 1 -- 172.21.15.19:0/3179713849 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010007cb0 con 0x7f201815b980 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.266+0000 7f201dbd5640 1 -- 172.21.15.19:0/3179713849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f201815b980 msgr2=0x7f20180cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.266+0000 7f201dbd5640 1 --2- 172.21.15.19:0/3179713849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f201815b980 0x7f20180cc8d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2010009dc0 tx=0x7f201002a6d0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.267+0000 7f201dbd5640 1 -- 172.21.15.19:0/3179713849 shutdown_connections 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.267+0000 7f201dbd5640 1 --2- 172.21.15.19:0/3179713849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f201815b980 0x7f20180cc8d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.267+0000 7f201dbd5640 1 -- 172.21.15.19:0/3179713849 >> 172.21.15.19:0/3179713849 conn(0x7f201806abe0 msgr2=0x7f201806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.267+0000 7f201dbd5640 1 -- 172.21.15.19:0/3179713849 shutdown_connections 2024-04-25T10:07:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.267+0000 7f201dbd5640 1 -- 172.21.15.19:0/3179713849 wait complete. 2024-04-25T10:07:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.268+0000 7f201dbd5640 1 Processor -- start 2024-04-25T10:07:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.268+0000 7f201dbd5640 1 -- start start 2024-04-25T10:07:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f201dbd5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20180cc4f0 0x7f2018149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f201dbd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20180ce230 0x7f2018159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f201dbd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f201815b980 0x7f2018159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f201dbd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2018071d60 con 0x7f201815b980 2024-04-25T10:07:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f201dbd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2018071270 con 0x7f20180ce230 2024-04-25T10:07:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f201dbd5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201806e8a0 con 0x7f20180cc4f0 2024-04-25T10:07:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f2017fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f201815b980 0x7f2018159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f2017fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f201815b980 0x7f2018159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38914/0 (socket says 172.21.15.19:38914) 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f2017fff640 1 -- 172.21.15.19:0/107364922 learned_addr learned my addr 172.21.15.19:0/107364922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f20177fe640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20180cc4f0 0x7f2018149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f2016ffd640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20180ce230 0x7f2018159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f2017fff640 1 -- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20180cc4f0 msgr2=0x7f2018149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f2017fff640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20180cc4f0 0x7f2018149d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f2017fff640 1 -- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20180ce230 msgr2=0x7f2018159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f2017fff640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20180ce230 0x7f2018159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f2017fff640 1 -- 172.21.15.19:0/107364922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20181561f0 con 0x7f201815b980 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.269+0000 7f2017fff640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f201815b980 0x7f2018159ff0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f2010009ee0 tx=0x7f2010004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.270+0000 7f1ffffff640 1 -- 172.21.15.19:0/107364922 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010040070 con 0x7f201815b980 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.270+0000 7f201dbd5640 1 -- 172.21.15.19:0/107364922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20181564d0 con 0x7f201815b980 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.270+0000 7f1ffffff640 1 -- 172.21.15.19:0/107364922 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2010031bc0 con 0x7f201815b980 2024-04-25T10:07:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.270+0000 7f1ffffff640 1 -- 172.21.15.19:0/107364922 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010009450 con 0x7f201815b980 2024-04-25T10:07:13.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.270+0000 7f201dbd5640 1 -- 172.21.15.19:0/107364922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2018156a90 con 0x7f201815b980 2024-04-25T10:07:13.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.271+0000 7f201dbd5640 1 -- 172.21.15.19:0/107364922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2018073a00 con 0x7f201815b980 2024-04-25T10:07:13.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.271+0000 7f1ffffff640 1 -- 172.21.15.19:0/107364922 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2010039040 con 0x7f201815b980 2024-04-25T10:07:13.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.271+0000 7f1ffffff640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200406b4d0 0x7f200406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.272+0000 7f1ffffff640 1 -- 172.21.15.19:0/107364922 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f20100ab950 con 0x7f201815b980 2024-04-25T10:07:13.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.272+0000 7f20177fe640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200406b4d0 0x7f200406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.272+0000 7f20177fe640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200406b4d0 0x7f200406d990 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2008007fd0 tx=0x7f2008009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.276+0000 7f1ffffff640 1 -- 172.21.15.19:0/107364922 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f201007a8f0 con 0x7f201815b980 2024-04-25T10:07:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.456+0000 7f201dbd5640 1 -- 172.21.15.19:0/107364922 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f201806d3a0 con 0x7f200406b4d0 2024-04-25T10:07:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.457+0000 7f1ffffff640 1 -- 172.21.15.19:0/107364922 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f201806d3a0 con 0x7f200406b4d0 2024-04-25T10:07:13.463 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:07:13.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 -- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200406b4d0 msgr2=0x7f200406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200406b4d0 0x7f200406d990 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2008007fd0 tx=0x7f2008009040 comp rx=0 tx=0).stop 2024-04-25T10:07:13.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 -- 172.21.15.19:0/107364922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f201815b980 msgr2=0x7f2018159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f201815b980 0x7f2018159ff0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f2010009ee0 tx=0x7f2010004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:13.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 -- 172.21.15.19:0/107364922 shutdown_connections 2024-04-25T10:07:13.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f200406b4d0 0x7f200406d990 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f201815b980 0x7f2018159ff0 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20180ce230 0x7f2018159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 --2- 172.21.15.19:0/107364922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f20180cc4f0 0x7f2018149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 -- 172.21.15.19:0/107364922 >> 172.21.15.19:0/107364922 conn(0x7f201806abe0 msgr2=0x7f201815b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 -- 172.21.15.19:0/107364922 shutdown_connections 2024-04-25T10:07:13.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.459+0000 7f201dbd5640 1 -- 172.21.15.19:0/107364922 wait complete. 2024-04-25T10:07:13.475 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:07:13.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:07:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.575+0000 7f605688f640 1 Processor -- start 2024-04-25T10:07:13.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.575+0000 7f605688f640 1 -- start start 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f605688f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f605015dc20 0x7f60500ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f605688f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f60500d32f0 0x7f60500cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f605688f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605015e000 con 0x7f6050161710 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f605688f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60500d1de0 con 0x7f60500d32f0 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f605688f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6050161c60 con 0x7f605015dc20 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f604f7fe640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6050161710 0x7f605006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54342/0 (socket says 172.21.15.19:54342) 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f604f7fe640 1 -- 172.21.15.19:0/190936099 learned_addr learned my addr 172.21.15.19:0/190936099 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f604ffff640 1 --2- 172.21.15.19:0/190936099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f605015dc20 0x7f60500ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f6054e2c640 1 --2- 172.21.15.19:0/190936099 >> v2:172.21.15.112:3300/0 conn(0x7f60500d32f0 0x7f60500cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f604effd640 1 -- 172.21.15.19:0/190936099 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3235043445 0 0) 0x7f605015e000 con 0x7f6050161710 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f604effd640 1 -- 172.21.15.19:0/190936099 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6038003520 con 0x7f6050161710 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f6054e2c640 1 -- 172.21.15.19:0/190936099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f605015dc20 msgr2=0x7f60500ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f6054e2c640 1 --2- 172.21.15.19:0/190936099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f605015dc20 0x7f60500ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f6054e2c640 1 -- 172.21.15.19:0/190936099 >> v1:172.21.15.19:6789/0 conn(0x7f6050161710 legacy=0x7f605006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f6054e2c640 1 -- 172.21.15.19:0/190936099 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6050161f40 con 0x7f60500d32f0 2024-04-25T10:07:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.576+0000 7f604ffff640 1 --2- 172.21.15.19:0/190936099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f605015dc20 0x7f60500ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.577+0000 7f6054e2c640 1 --2- 172.21.15.19:0/190936099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60500d32f0 0x7f60500cebb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6048009d90 tx=0x7f604802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cce232eaf984c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.577+0000 7f604effd640 1 -- 172.21.15.19:0/190936099 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048007740 con 0x7f60500d32f0 2024-04-25T10:07:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.577+0000 7f604effd640 1 -- 172.21.15.19:0/190936099 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60480078e0 con 0x7f60500d32f0 2024-04-25T10:07:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.577+0000 7f604effd640 1 -- 172.21.15.19:0/190936099 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048007cb0 con 0x7f60500d32f0 2024-04-25T10:07:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.578+0000 7f605688f640 1 -- 172.21.15.19:0/190936099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60500d32f0 msgr2=0x7f60500cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.578+0000 7f605688f640 1 --2- 172.21.15.19:0/190936099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60500d32f0 0x7f60500cebb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6048009d90 tx=0x7f604802a660 comp rx=0 tx=0).stop 2024-04-25T10:07:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.578+0000 7f605688f640 1 -- 172.21.15.19:0/190936099 shutdown_connections 2024-04-25T10:07:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.578+0000 7f605688f640 1 --2- 172.21.15.19:0/190936099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60500d32f0 0x7f60500cebb0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.578+0000 7f605688f640 1 --2- 172.21.15.19:0/190936099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f605015dc20 0x7f60500ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.578+0000 7f605688f640 1 -- 172.21.15.19:0/190936099 >> 172.21.15.19:0/190936099 conn(0x7f60500d0420 msgr2=0x7f60500d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.578+0000 7f605688f640 1 -- 172.21.15.19:0/190936099 shutdown_connections 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.578+0000 7f605688f640 1 -- 172.21.15.19:0/190936099 wait complete. 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.579+0000 7f605688f640 1 Processor -- start 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.579+0000 7f605688f640 1 -- start start 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f605688f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60500d32f0 0x7f60500aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f605688f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f605015dc20 0x7f60500d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f605688f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6050161710 0x7f60500d2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f605688f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60500b5e50 con 0x7f605015dc20 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f605688f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6050162500 con 0x7f60500d32f0 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f605688f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60500b5fd0 con 0x7f6050161710 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f604f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f605015dc20 0x7f60500d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f604f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f605015dc20 0x7f60500d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38926/0 (socket says 172.21.15.19:38926) 2024-04-25T10:07:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f604f7fe640 1 -- 172.21.15.19:0/3968447659 learned_addr learned my addr 172.21.15.19:0/3968447659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f6054e2c640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6050161710 0x7f60500d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f604ffff640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60500d32f0 0x7f60500aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f604f7fe640 1 -- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6050161710 msgr2=0x7f60500d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f604f7fe640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6050161710 0x7f60500d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f604f7fe640 1 -- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60500d32f0 msgr2=0x7f60500aab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f604f7fe640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60500d32f0 0x7f60500aab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.580+0000 7f604f7fe640 1 -- 172.21.15.19:0/3968447659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60500cafe0 con 0x7f605015dc20 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.581+0000 7f604f7fe640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f605015dc20 0x7f60500d22e0 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f604000c390 tx=0x7f604000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.581+0000 7f604cff9640 1 -- 172.21.15.19:0/3968447659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040017070 con 0x7f605015dc20 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.581+0000 7f604cff9640 1 -- 172.21.15.19:0/3968447659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6040005ce0 con 0x7f605015dc20 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.581+0000 7f605688f640 1 -- 172.21.15.19:0/3968447659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60500cb2c0 con 0x7f605015dc20 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.581+0000 7f604cff9640 1 -- 172.21.15.19:0/3968447659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60400072f0 con 0x7f605015dc20 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.581+0000 7f605688f640 1 -- 172.21.15.19:0/3968447659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60500cb880 con 0x7f605015dc20 2024-04-25T10:07:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.582+0000 7f605688f640 1 -- 172.21.15.19:0/3968447659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60500cf380 con 0x7f605015dc20 2024-04-25T10:07:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.582+0000 7f604cff9640 1 -- 172.21.15.19:0/3968447659 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f604001a070 con 0x7f605015dc20 2024-04-25T10:07:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.583+0000 7f604cff9640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f603c06b4d0 0x7f603c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.583+0000 7f604cff9640 1 -- 172.21.15.19:0/3968447659 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f6040089460 con 0x7f605015dc20 2024-04-25T10:07:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.583+0000 7f604cff9640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x7f603c071a10 0x7f603c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.583+0000 7f604cff9640 1 -- 172.21.15.19:0/3968447659 --> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6040010e50 con 0x7f603c071a10 2024-04-25T10:07:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.583+0000 7f604cff9640 1 -- 172.21.15.19:0/3968447659 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6040090050 con 0x7f605015dc20 2024-04-25T10:07:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.583+0000 7f604ffff640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f603c06b4d0 0x7f603c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.583+0000 7f6054e2c640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x7f603c071a10 0x7f603c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.584+0000 7f604ffff640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f603c06b4d0 0x7f603c06d990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6044009a30 tx=0x7f6044008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.584+0000 7f6054e2c640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x7f603c071a10 0x7f603c073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.584+0000 7f604cff9640 1 -- 172.21.15.19:0/3968447659 <== osd.4 v2:172.21.15.112:6800/1718655451 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6040010e50 con 0x7f603c071a10 2024-04-25T10:07:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.597+0000 7f605688f640 1 -- 172.21.15.19:0/3968447659 --> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6050073a00 con 0x7f603c071a10 2024-04-25T10:07:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f604cff9640 1 -- 172.21.15.19:0/3968447659 <== osd.4 v2:172.21.15.112:6800/1718655451 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6050073a00 con 0x7f603c071a10 2024-04-25T10:07:13.604 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:07:13.604 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:07:13.604 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:07:13.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f605688f640 1 -- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x7f603c071a10 msgr2=0x7f603c073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f605688f640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:6800/1718655451,v1:172.21.15.112:6801/1718655451] conn(0x7f603c071a10 0x7f603c073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f605688f640 1 -- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f603c06b4d0 msgr2=0x7f603c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f605688f640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f603c06b4d0 0x7f603c06d990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6044009a30 tx=0x7f6044008040 comp rx=0 tx=0).stop 2024-04-25T10:07:13.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f605688f640 1 -- 172.21.15.19:0/3968447659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f605015dc20 msgr2=0x7f60500d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f605688f640 1 --2- 172.21.15.19:0/3968447659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f605015dc20 0x7f60500d22e0 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f604000c390 tx=0x7f604000c850 comp rx=0 tx=0).stop 2024-04-25T10:07:13.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f6054e2c640 1 -- 172.21.15.19:0/3968447659 reap_dead start 2024-04-25T10:07:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f605688f640 1 -- 172.21.15.19:0/3968447659 shutdown_connections 2024-04-25T10:07:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f605688f640 1 -- 172.21.15.19:0/3968447659 >> 172.21.15.19:0/3968447659 conn(0x7f60500d0420 msgr2=0x7f60500b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f605688f640 1 -- 172.21.15.19:0/3968447659 shutdown_connections 2024-04-25T10:07:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.598+0000 7f605688f640 1 -- 172.21.15.19:0/3968447659 wait complete. 2024-04-25T10:07:13.614 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:07:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.710+0000 7fa0728ac640 1 Processor -- start 2024-04-25T10:07:13.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.710+0000 7fa0728ac640 1 -- start start 2024-04-25T10:07:13.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa0728ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c1616f0 0x7fa06c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa0728ac640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa06c0ce0a0 0x7fa06c0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa0728ac640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06c06c8d0 con 0x7fa06c1610f0 2024-04-25T10:07:13.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa0728ac640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06c1614d0 con 0x7fa06c0ce0a0 2024-04-25T10:07:13.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa0728ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0c6960 con 0x7fa06c1616f0 2024-04-25T10:07:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa06b7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c1616f0 0x7fa06c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa070e49640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa06c0ce0a0 0x7fa06c0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa06bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa06c1610f0 0x7fa06c06c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:33242/0 (socket says 172.21.15.19:33242) 2024-04-25T10:07:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa06bfff640 1 -- 172.21.15.19:0/392609216 learned_addr learned my addr 172.21.15.19:0/392609216 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa070e49640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa06c0ce0a0 0x7fa06c0ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:55466/0 (socket says 172.21.15.19:55466) 2024-04-25T10:07:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa06b7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c1616f0 0x7fa06c161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:56892/0 (socket says 172.21.15.19:56892) 2024-04-25T10:07:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.711+0000 7fa06affd640 1 -- 172.21.15.19:0/392609216 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063116061 0 0) 0x7fa06c06c8d0 con 0x7fa06c1610f0 2024-04-25T10:07:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.712+0000 7fa06b7fe640 1 -- 172.21.15.19:0/392609216 >> v2:172.21.15.112:3300/0 conn(0x7fa06c0ce0a0 msgr2=0x7fa06c0ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.712+0000 7fa06b7fe640 1 --2- 172.21.15.19:0/392609216 >> v2:172.21.15.112:3300/0 conn(0x7fa06c0ce0a0 0x7fa06c0ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.712+0000 7fa06b7fe640 1 -- 172.21.15.19:0/392609216 >> v1:172.21.15.112:6789/0 conn(0x7fa06c1610f0 legacy=0x7fa06c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.712+0000 7fa06b7fe640 1 -- 172.21.15.19:0/392609216 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c162260 con 0x7fa06c1616f0 2024-04-25T10:07:13.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.712+0000 7fa070e49640 1 --2- 172.21.15.19:0/392609216 >> v2:172.21.15.112:3300/0 conn(0x7fa06c0ce0a0 0x7fa06c0ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:13.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.712+0000 7fa06b7fe640 1 --2- 172.21.15.19:0/392609216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c1616f0 0x7fa06c161b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0097a0 tx=0x7fa05c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=372b16cc7f1ff880 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.712+0000 7fa06affd640 1 -- 172.21.15.19:0/392609216 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c037070 con 0x7fa06c1616f0 2024-04-25T10:07:13.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.712+0000 7fa06affd640 1 -- 172.21.15.19:0/392609216 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c032440 con 0x7fa06c1616f0 2024-04-25T10:07:13.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.713+0000 7fa06affd640 1 -- 172.21.15.19:0/392609216 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c032810 con 0x7fa06c1616f0 2024-04-25T10:07:13.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.713+0000 7fa0728ac640 1 -- 172.21.15.19:0/392609216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c1616f0 msgr2=0x7fa06c161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.713+0000 7fa0728ac640 1 --2- 172.21.15.19:0/392609216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c1616f0 0x7fa06c161b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0097a0 tx=0x7fa05c02a180 comp rx=0 tx=0).stop 2024-04-25T10:07:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.713+0000 7fa0728ac640 1 -- 172.21.15.19:0/392609216 shutdown_connections 2024-04-25T10:07:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.713+0000 7fa0728ac640 1 --2- 172.21.15.19:0/392609216 >> v2:172.21.15.112:3300/0 conn(0x7fa06c0ce0a0 0x7fa06c0ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.713+0000 7fa0728ac640 1 --2- 172.21.15.19:0/392609216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c1616f0 0x7fa06c161b50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.713+0000 7fa0728ac640 1 -- 172.21.15.19:0/392609216 >> 172.21.15.19:0/392609216 conn(0x7fa06c06f500 msgr2=0x7fa06c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.713+0000 7fa0728ac640 1 -- 172.21.15.19:0/392609216 shutdown_connections 2024-04-25T10:07:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.713+0000 7fa0728ac640 1 -- 172.21.15.19:0/392609216 wait complete. 2024-04-25T10:07:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.714+0000 7fa0728ac640 1 Processor -- start 2024-04-25T10:07:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.714+0000 7fa0728ac640 1 -- start start 2024-04-25T10:07:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.715+0000 7fa0728ac640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c0ce0a0 0x7fa06c149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.715+0000 7fa0728ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa06c1610f0 0x7fa06c157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.715+0000 7fa0728ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c1616f0 0x7fa06c0b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.715+0000 7fa0728ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0cfce0 con 0x7fa06c1616f0 2024-04-25T10:07:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.715+0000 7fa0728ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0cfa00 con 0x7fa06c1610f0 2024-04-25T10:07:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.715+0000 7fa0728ac640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0cf690 con 0x7fa06c0ce0a0 2024-04-25T10:07:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.715+0000 7fa070e49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c1616f0 0x7fa06c0b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.715+0000 7fa070e49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c1616f0 0x7fa06c0b6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38938/0 (socket says 172.21.15.19:38938) 2024-04-25T10:07:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.715+0000 7fa070e49640 1 -- 172.21.15.19:0/1328835968 learned_addr learned my addr 172.21.15.19:0/1328835968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.715+0000 7fa06bfff640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c0ce0a0 0x7fa06c149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.715+0000 7fa06b7fe640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa06c1610f0 0x7fa06c157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.716+0000 7fa070e49640 1 -- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c0ce0a0 msgr2=0x7fa06c149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.716+0000 7fa070e49640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c0ce0a0 0x7fa06c149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.716+0000 7fa070e49640 1 -- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa06c1610f0 msgr2=0x7fa06c157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.716+0000 7fa070e49640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa06c1610f0 0x7fa06c157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.716+0000 7fa070e49640 1 -- 172.21.15.19:0/1328835968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c149890 con 0x7fa06c1616f0 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.716+0000 7fa070e49640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c1616f0 0x7fa06c0b6cf0 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7fa06400efc0 tx=0x7fa064009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.716+0000 7fa068ff9640 1 -- 172.21.15.19:0/1328835968 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064018030 con 0x7fa06c1616f0 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.716+0000 7fa068ff9640 1 -- 172.21.15.19:0/1328835968 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06400cde0 con 0x7fa06c1616f0 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.716+0000 7fa0728ac640 1 -- 172.21.15.19:0/1328835968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c0b5c50 con 0x7fa06c1616f0 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.716+0000 7fa068ff9640 1 -- 172.21.15.19:0/1328835968 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064003bb0 con 0x7fa06c1616f0 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.716+0000 7fa0728ac640 1 -- 172.21.15.19:0/1328835968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c0b6210 con 0x7fa06c1616f0 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.717+0000 7fa0728ac640 1 -- 172.21.15.19:0/1328835968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c073a00 con 0x7fa06c1616f0 2024-04-25T10:07:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.718+0000 7fa068ff9640 1 -- 172.21.15.19:0/1328835968 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa064005d10 con 0x7fa06c1616f0 2024-04-25T10:07:13.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.718+0000 7fa068ff9640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05806b4d0 0x7fa05806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.718+0000 7fa068ff9640 1 -- 172.21.15.19:0/1328835968 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa06408a920 con 0x7fa06c1616f0 2024-04-25T10:07:13.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.718+0000 7fa06bfff640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05806b4d0 0x7fa05806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.719+0000 7fa06bfff640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05806b4d0 0x7fa05806d990 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa060000f60 tx=0x7fa060007d50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.722+0000 7fa068ff9640 1 -- 172.21.15.19:0/1328835968 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa064022070 con 0x7fa06c1616f0 2024-04-25T10:07:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.904+0000 7fa0728ac640 1 -- 172.21.15.19:0/1328835968 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa06c0cf150 con 0x7fa05806b4d0 2024-04-25T10:07:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.904+0000 7fa068ff9640 1 -- 172.21.15.19:0/1328835968 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa06c0cf150 con 0x7fa05806b4d0 2024-04-25T10:07:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:07:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 -- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05806b4d0 msgr2=0x7fa05806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05806b4d0 0x7fa05806d990 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa060000f60 tx=0x7fa060007d50 comp rx=0 tx=0).stop 2024-04-25T10:07:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 -- 172.21.15.19:0/1328835968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c1616f0 msgr2=0x7fa06c0b6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c1616f0 0x7fa06c0b6cf0 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7fa06400efc0 tx=0x7fa064009f90 comp rx=0 tx=0).stop 2024-04-25T10:07:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 -- 172.21.15.19:0/1328835968 shutdown_connections 2024-04-25T10:07:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fa05806b4d0 0x7fa05806d990 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c1616f0 0x7fa06c0b6cf0 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa06c1610f0 0x7fa06c157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 --2- 172.21.15.19:0/1328835968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa06c0ce0a0 0x7fa06c149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 -- 172.21.15.19:0/1328835968 >> 172.21.15.19:0/1328835968 conn(0x7fa06c06f500 msgr2=0x7fa06c0ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 -- 172.21.15.19:0/1328835968 shutdown_connections 2024-04-25T10:07:13.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:13.907+0000 7fa0728ac640 1 -- 172.21.15.19:0/1328835968 wait complete. 2024-04-25T10:07:13.923 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:07:13.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:07:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.022+0000 7f22807f4640 1 Processor -- start 2024-04-25T10:07:14.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.022+0000 7f22807f4640 1 -- start start 2024-04-25T10:07:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.023+0000 7f22807f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780b5620 0x7f22780b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.023+0000 7f22807f4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f227806c2d0 0x7f227806c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.023+0000 7f22807f4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227806cca0 con 0x7f227815b7d0 2024-04-25T10:07:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.023+0000 7f22807f4640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22780b5fc0 con 0x7f227806c2d0 2024-04-25T10:07:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.023+0000 7f22807f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227806cf50 con 0x7f22780b5620 2024-04-25T10:07:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.023+0000 7f227ed91640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f227806c2d0 0x7f227806c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.023+0000 7f227ed91640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f227806c2d0 0x7f227806c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38946/0 (socket says 172.21.15.19:38946) 2024-04-25T10:07:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.023+0000 7f227ed91640 1 -- 172.21.15.19:0/1848297170 learned_addr learned my addr 172.21.15.19:0/1848297170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:14.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.023+0000 7f227dd8f640 1 --2- 172.21.15.19:0/1848297170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780b5620 0x7f22780b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.024+0000 7f227ed91640 1 -- 172.21.15.19:0/1848297170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780b5620 msgr2=0x7f22780b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.024+0000 7f227ed91640 1 --2- 172.21.15.19:0/1848297170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780b5620 0x7f22780b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.024+0000 7f227ed91640 1 -- 172.21.15.19:0/1848297170 >> v1:172.21.15.112:6789/0 conn(0x7f227815b7d0 legacy=0x7f22780cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.024+0000 7f227ed91640 1 -- 172.21.15.19:0/1848297170 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22780ce530 con 0x7f227806c2d0 2024-04-25T10:07:14.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.024+0000 7f227ed91640 1 --2- 172.21.15.19:0/1848297170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227806c2d0 0x7f227806c760 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f2268009e50 tx=0x7f226802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce915c4c1842f658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.024+0000 7f227d58e640 1 -- 172.21.15.19:0/1848297170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268036070 con 0x7f227806c2d0 2024-04-25T10:07:14.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.024+0000 7f227d58e640 1 -- 172.21.15.19:0/1848297170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2268031600 con 0x7f227806c2d0 2024-04-25T10:07:14.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.024+0000 7f227d58e640 1 -- 172.21.15.19:0/1848297170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22680319d0 con 0x7f227806c2d0 2024-04-25T10:07:14.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.025+0000 7f22807f4640 1 -- 172.21.15.19:0/1848297170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227806c2d0 msgr2=0x7f227806c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.025+0000 7f22807f4640 1 --2- 172.21.15.19:0/1848297170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227806c2d0 0x7f227806c760 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f2268009e50 tx=0x7f226802a710 comp rx=0 tx=0).stop 2024-04-25T10:07:14.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.025+0000 7f22807f4640 1 -- 172.21.15.19:0/1848297170 shutdown_connections 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.025+0000 7f22807f4640 1 --2- 172.21.15.19:0/1848297170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227806c2d0 0x7f227806c760 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.025+0000 7f22807f4640 1 --2- 172.21.15.19:0/1848297170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22780b5620 0x7f22780b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.025+0000 7f22807f4640 1 -- 172.21.15.19:0/1848297170 >> 172.21.15.19:0/1848297170 conn(0x7f22780d0800 msgr2=0x7f22780d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.025+0000 7f22807f4640 1 -- 172.21.15.19:0/1848297170 shutdown_connections 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.025+0000 7f22807f4640 1 -- 172.21.15.19:0/1848297170 wait complete. 2024-04-25T10:07:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.026+0000 7f22807f4640 1 Processor -- start 2024-04-25T10:07:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.026+0000 7f22807f4640 1 -- start start 2024-04-25T10:07:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f22807f4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f227806c2d0 0x7f22781549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f22807f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22780b5620 0x7f22780c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f22807f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815b7d0 0x7f2278158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f22807f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227806d230 con 0x7f227815b7d0 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f22807f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22780ceaf0 con 0x7f22780b5620 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f22807f4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227815bbb0 con 0x7f227806c2d0 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227ed91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815b7d0 0x7f2278158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227ed91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815b7d0 0x7f2278158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38948/0 (socket says 172.21.15.19:38948) 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227ed91640 1 -- 172.21.15.19:0/2970403974 learned_addr learned my addr 172.21.15.19:0/2970403974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227dd8f640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22780b5620 0x7f22780c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227e590640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f227806c2d0 0x7f22781549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227ed91640 1 -- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f227806c2d0 msgr2=0x7f22781549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227ed91640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f227806c2d0 0x7f22781549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227ed91640 1 -- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22780b5620 msgr2=0x7f22780c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227ed91640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22780b5620 0x7f22780c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227ed91640 1 -- 172.21.15.19:0/2970403974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22780b3980 con 0x7f227815b7d0 2024-04-25T10:07:14.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227e590640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f227806c2d0 0x7f22781549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:14.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.027+0000 7f227dd8f640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22780b5620 0x7f22780c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:14.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.028+0000 7f227ed91640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815b7d0 0x7f2278158270 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f2268034040 tx=0x7f2268004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.028+0000 7f2276ffd640 1 -- 172.21.15.19:0/2970403974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268036050 con 0x7f227815b7d0 2024-04-25T10:07:14.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.028+0000 7f2276ffd640 1 -- 172.21.15.19:0/2970403974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2268042070 con 0x7f227815b7d0 2024-04-25T10:07:14.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.028+0000 7f22807f4640 1 -- 172.21.15.19:0/2970403974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22780b3c60 con 0x7f227815b7d0 2024-04-25T10:07:14.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.028+0000 7f2276ffd640 1 -- 172.21.15.19:0/2970403974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268009510 con 0x7f227815b7d0 2024-04-25T10:07:14.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.028+0000 7f22807f4640 1 -- 172.21.15.19:0/2970403974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22780b4220 con 0x7f227815b7d0 2024-04-25T10:07:14.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.029+0000 7f22807f4640 1 -- 172.21.15.19:0/2970403974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2278073a00 con 0x7f227815b7d0 2024-04-25T10:07:14.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.029+0000 7f2276ffd640 1 -- 172.21.15.19:0/2970403974 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2268007bf0 con 0x7f227815b7d0 2024-04-25T10:07:14.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.030+0000 7f2276ffd640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f226406b4d0 0x7f226406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.030+0000 7f2276ffd640 1 -- 172.21.15.19:0/2970403974 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f22680aae60 con 0x7f227815b7d0 2024-04-25T10:07:14.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.030+0000 7f2276ffd640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x7f2264071a10 0x7f2264073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.030+0000 7f227e590640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f226406b4d0 0x7f226406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.030+0000 7f2276ffd640 1 -- 172.21.15.19:0/2970403974 --> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f226804ae80 con 0x7f2264071a10 2024-04-25T10:07:14.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.030+0000 7f2276ffd640 1 -- 172.21.15.19:0/2970403974 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22680ad560 con 0x7f227815b7d0 2024-04-25T10:07:14.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.030+0000 7f227dd8f640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x7f2264071a10 0x7f2264073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.030+0000 7f227e590640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f226406b4d0 0x7f226406d990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2270001700 tx=0x7f2270009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.031+0000 7f227dd8f640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x7f2264071a10 0x7f2264073e50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.031+0000 7f2276ffd640 1 -- 172.21.15.19:0/2970403974 <== osd.5 v2:172.21.15.112:6808/2948328480 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f226804ae80 con 0x7f2264071a10 2024-04-25T10:07:14.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.045+0000 7f22807f4640 1 -- 172.21.15.19:0/2970403974 --> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f22780d3040 con 0x7f2264071a10 2024-04-25T10:07:14.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.045+0000 7f2276ffd640 1 -- 172.21.15.19:0/2970403974 <== osd.5 v2:172.21.15.112:6808/2948328480 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f22780d3040 con 0x7f2264071a10 2024-04-25T10:07:14.051 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:07:14.051 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:07:14.051 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:07:14.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.046+0000 7f22807f4640 1 -- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x7f2264071a10 msgr2=0x7f2264073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.046+0000 7f22807f4640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:6808/2948328480,v1:172.21.15.112:6809/2948328480] conn(0x7f2264071a10 0x7f2264073e50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.046+0000 7f22807f4640 1 -- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f226406b4d0 msgr2=0x7f226406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.046+0000 7f22807f4640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f226406b4d0 0x7f226406d990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2270001700 tx=0x7f2270009040 comp rx=0 tx=0).stop 2024-04-25T10:07:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.046+0000 7f22807f4640 1 -- 172.21.15.19:0/2970403974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815b7d0 msgr2=0x7f2278158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.046+0000 7f22807f4640 1 --2- 172.21.15.19:0/2970403974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f227815b7d0 0x7f2278158270 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f2268034040 tx=0x7f2268004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.046+0000 7f227ed91640 1 -- 172.21.15.19:0/2970403974 reap_dead start 2024-04-25T10:07:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.046+0000 7f22807f4640 1 -- 172.21.15.19:0/2970403974 shutdown_connections 2024-04-25T10:07:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.046+0000 7f22807f4640 1 -- 172.21.15.19:0/2970403974 >> 172.21.15.19:0/2970403974 conn(0x7f22780d0800 msgr2=0x7f22780d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.046+0000 7f22807f4640 1 -- 172.21.15.19:0/2970403974 shutdown_connections 2024-04-25T10:07:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.046+0000 7f22807f4640 1 -- 172.21.15.19:0/2970403974 wait complete. 2024-04-25T10:07:14.062 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:07:14.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.158+0000 7ffab5930640 1 Processor -- start 2024-04-25T10:07:14.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.158+0000 7ffab5930640 1 -- start start 2024-04-25T10:07:14.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.159+0000 7ffab5930640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffab015dc20 0x7ffab00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.159+0000 7ffab5930640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab006cda0 con 0x7ffab0161710 2024-04-25T10:07:14.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.159+0000 7ffab5930640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab015e000 con 0x7ffab00d32f0 2024-04-25T10:07:14.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.159+0000 7ffab5930640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab00d1de0 con 0x7ffab015dc20 2024-04-25T10:07:14.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.159+0000 7ffaaeffd640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffab015dc20 0x7ffab00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.159+0000 7ffaae7fc640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ffab0161710 0x7ffab006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54350/0 (socket says 172.21.15.19:54350) 2024-04-25T10:07:14.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.159+0000 7ffaae7fc640 1 -- 172.21.15.19:0/657661409 learned_addr learned my addr 172.21.15.19:0/657661409 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:14.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.160+0000 7ffaaeffd640 1 -- 172.21.15.19:0/657661409 >> v1:172.21.15.112:6789/0 conn(0x7ffab00d32f0 legacy=0x7ffab006c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.160+0000 7ffaaeffd640 1 -- 172.21.15.19:0/657661409 >> v1:172.21.15.19:6789/0 conn(0x7ffab0161710 legacy=0x7ffab006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.160+0000 7ffaaeffd640 1 -- 172.21.15.19:0/657661409 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab0161c60 con 0x7ffab015dc20 2024-04-25T10:07:14.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.160+0000 7ffaaeffd640 1 --2- 172.21.15.19:0/657661409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab015dc20 0x7ffab00ce670 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80097a0 tx=0x7ffaa802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c131c84fd1e7ec23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.160+0000 7ffaadffb640 1 -- 172.21.15.19:0/657661409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8037070 con 0x7ffab015dc20 2024-04-25T10:07:14.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.160+0000 7ffaadffb640 1 -- 172.21.15.19:0/657661409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa8032440 con 0x7ffab015dc20 2024-04-25T10:07:14.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.160+0000 7ffaadffb640 1 -- 172.21.15.19:0/657661409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8032810 con 0x7ffab015dc20 2024-04-25T10:07:14.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.161+0000 7ffab5930640 1 -- 172.21.15.19:0/657661409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab015dc20 msgr2=0x7ffab00ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.161+0000 7ffab5930640 1 --2- 172.21.15.19:0/657661409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab015dc20 0x7ffab00ce670 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80097a0 tx=0x7ffaa802a230 comp rx=0 tx=0).stop 2024-04-25T10:07:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.161+0000 7ffab5930640 1 -- 172.21.15.19:0/657661409 shutdown_connections 2024-04-25T10:07:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.161+0000 7ffab5930640 1 --2- 172.21.15.19:0/657661409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab015dc20 0x7ffab00ce670 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.161+0000 7ffab5930640 1 -- 172.21.15.19:0/657661409 >> 172.21.15.19:0/657661409 conn(0x7ffab00d0420 msgr2=0x7ffab00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.161+0000 7ffab5930640 1 -- 172.21.15.19:0/657661409 shutdown_connections 2024-04-25T10:07:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.161+0000 7ffab5930640 1 -- 172.21.15.19:0/657661409 wait complete. 2024-04-25T10:07:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.162+0000 7ffab5930640 1 Processor -- start 2024-04-25T10:07:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.162+0000 7ffab5930640 1 -- start start 2024-04-25T10:07:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffab5930640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffab00d32f0 0x7ffab01590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffab5930640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffab015dc20 0x7ffab015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffab5930640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0161710 0x7ffab0160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffab5930640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0162500 con 0x7ffab0161710 2024-04-25T10:07:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffab5930640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0162220 con 0x7ffab00d32f0 2024-04-25T10:07:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffab5930640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00b5e50 con 0x7ffab015dc20 2024-04-25T10:07:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffaaf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0161710 0x7ffab0160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffaaf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0161710 0x7ffab0160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38972/0 (socket says 172.21.15.19:38972) 2024-04-25T10:07:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffaaf7fe640 1 -- 172.21.15.19:0/1058188653 learned_addr learned my addr 172.21.15.19:0/1058188653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffaaeffd640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffab00d32f0 0x7ffab01590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffaae7fc640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffab015dc20 0x7ffab015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffaaf7fe640 1 -- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffab015dc20 msgr2=0x7ffab015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffaaf7fe640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffab015dc20 0x7ffab015ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffaaf7fe640 1 -- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffab00d32f0 msgr2=0x7ffab01590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffaaf7fe640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffab00d32f0 0x7ffab01590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.163+0000 7ffaaf7fe640 1 -- 172.21.15.19:0/1058188653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab015f2e0 con 0x7ffab0161710 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.164+0000 7ffaaeffd640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffab00d32f0 0x7ffab01590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.164+0000 7ffaae7fc640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffab015dc20 0x7ffab015ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.164+0000 7ffaaf7fe640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0161710 0x7ffab0160b10 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000c730 tx=0x7ffaa000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.164+0000 7ffaa7fff640 1 -- 172.21.15.19:0/1058188653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0016020 con 0x7ffab0161710 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.164+0000 7ffaa7fff640 1 -- 172.21.15.19:0/1058188653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa0006d50 con 0x7ffab0161710 2024-04-25T10:07:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.164+0000 7ffab5930640 1 -- 172.21.15.19:0/1058188653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab015f5c0 con 0x7ffab0161710 2024-04-25T10:07:14.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.164+0000 7ffaa7fff640 1 -- 172.21.15.19:0/1058188653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0003bb0 con 0x7ffab0161710 2024-04-25T10:07:14.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.164+0000 7ffab5930640 1 -- 172.21.15.19:0/1058188653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab01f49b0 con 0x7ffab0161710 2024-04-25T10:07:14.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.165+0000 7ffab5930640 1 -- 172.21.15.19:0/1058188653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab00d16b0 con 0x7ffab0161710 2024-04-25T10:07:14.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.165+0000 7ffaa7fff640 1 -- 172.21.15.19:0/1058188653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffaa00050d0 con 0x7ffab0161710 2024-04-25T10:07:14.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.166+0000 7ffaa7fff640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa9006b4d0 0x7ffa9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.166+0000 7ffaa7fff640 1 -- 172.21.15.19:0/1058188653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ffaa0089ba0 con 0x7ffab0161710 2024-04-25T10:07:14.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.166+0000 7ffaaeffd640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa9006b4d0 0x7ffa9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.166+0000 7ffaaeffd640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa9006b4d0 0x7ffa9006d990 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8034000 tx=0x7ffaa802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.170+0000 7ffaa7fff640 1 -- 172.21.15.19:0/1058188653 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffaa0020020 con 0x7ffab0161710 2024-04-25T10:07:14.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.348+0000 7ffab5930640 1 -- 172.21.15.19:0/1058188653 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7ffab00739e0 con 0x7ffa9006b4d0 2024-04-25T10:07:14.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.349+0000 7ffaa7fff640 1 -- 172.21.15.19:0/1058188653 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ffab00739e0 con 0x7ffa9006b4d0 2024-04-25T10:07:14.354 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:07:14.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 -- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa9006b4d0 msgr2=0x7ffa9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa9006b4d0 0x7ffa9006d990 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8034000 tx=0x7ffaa802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:14.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 -- 172.21.15.19:0/1058188653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0161710 msgr2=0x7ffab0160b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0161710 0x7ffab0160b10 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000c730 tx=0x7ffaa000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 -- 172.21.15.19:0/1058188653 shutdown_connections 2024-04-25T10:07:14.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7ffa9006b4d0 0x7ffa9006d990 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0161710 0x7ffab0160b10 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffab015dc20 0x7ffab015ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 --2- 172.21.15.19:0/1058188653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffab00d32f0 0x7ffab01590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 -- 172.21.15.19:0/1058188653 >> 172.21.15.19:0/1058188653 conn(0x7ffab00d0420 msgr2=0x7ffab00b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 -- 172.21.15.19:0/1058188653 shutdown_connections 2024-04-25T10:07:14.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.351+0000 7ffab5930640 1 -- 172.21.15.19:0/1058188653 wait complete. 2024-04-25T10:07:14.367 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:07:14.367 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:07:14.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.466+0000 7fd61fa38640 1 Processor -- start 2024-04-25T10:07:14.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.467+0000 7fd61fa38640 1 -- start start 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.467+0000 7fd61fa38640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd61815bb50 0x7fd6180c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.467+0000 7fd61fa38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd6180cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.467+0000 7fd61fa38640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6180c75e0 con 0x7fd6180cfb40 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.467+0000 7fd61fa38640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6180c8680 con 0x7fd61815bb50 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.467+0000 7fd61fa38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180c78c0 con 0x7fd6180c70d0 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.468+0000 7fd61d7d4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd61815bb50 0x7fd6180c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.468+0000 7fd61d7d4640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd61815bb50 0x7fd6180c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38974/0 (socket says 172.21.15.19:38974) 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.468+0000 7fd61d7d4640 1 -- 172.21.15.19:0/4041880655 learned_addr learned my addr 172.21.15.19:0/4041880655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.468+0000 7fd61cfd3640 1 --2- 172.21.15.19:0/4041880655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd6180cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.468+0000 7fd61d7d4640 1 -- 172.21.15.19:0/4041880655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 msgr2=0x7fd6180cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.468+0000 7fd61d7d4640 1 --2- 172.21.15.19:0/4041880655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd6180cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.468+0000 7fd61d7d4640 1 -- 172.21.15.19:0/4041880655 >> v1:172.21.15.112:6789/0 conn(0x7fd6180cfb40 legacy=0x7fd618070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.468+0000 7fd61d7d4640 1 -- 172.21.15.19:0/4041880655 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6180c7ba0 con 0x7fd61815bb50 2024-04-25T10:07:14.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.469+0000 7fd61d7d4640 1 --2- 172.21.15.19:0/4041880655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd6180c6b90 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7fd608009e50 tx=0x7fd60802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561165904801ac27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.469+0000 7fd617fff640 1 -- 172.21.15.19:0/4041880655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608036070 con 0x7fd61815bb50 2024-04-25T10:07:14.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.469+0000 7fd617fff640 1 -- 172.21.15.19:0/4041880655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd608031600 con 0x7fd61815bb50 2024-04-25T10:07:14.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.469+0000 7fd617fff640 1 -- 172.21.15.19:0/4041880655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080319d0 con 0x7fd61815bb50 2024-04-25T10:07:14.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.469+0000 7fd61fa38640 1 -- 172.21.15.19:0/4041880655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 msgr2=0x7fd6180c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.469+0000 7fd61fa38640 1 --2- 172.21.15.19:0/4041880655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd6180c6b90 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7fd608009e50 tx=0x7fd60802a7a0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.469+0000 7fd61fa38640 1 -- 172.21.15.19:0/4041880655 shutdown_connections 2024-04-25T10:07:14.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.469+0000 7fd61fa38640 1 --2- 172.21.15.19:0/4041880655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd6180cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.469+0000 7fd61fa38640 1 --2- 172.21.15.19:0/4041880655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd6180c6b90 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.469+0000 7fd61fa38640 1 -- 172.21.15.19:0/4041880655 >> 172.21.15.19:0/4041880655 conn(0x7fd61806f500 msgr2=0x7fd61806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.470+0000 7fd61fa38640 1 -- 172.21.15.19:0/4041880655 shutdown_connections 2024-04-25T10:07:14.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.470+0000 7fd61fa38640 1 -- 172.21.15.19:0/4041880655 wait complete. 2024-04-25T10:07:14.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.471+0000 7fd61fa38640 1 Processor -- start 2024-04-25T10:07:14.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.471+0000 7fd61fa38640 1 -- start start 2024-04-25T10:07:14.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.471+0000 7fd61fa38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd6181f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.471+0000 7fd61fa38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6180cfb40 0x7fd61815e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.471+0000 7fd61fa38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd61815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.471+0000 7fd61fa38640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61815f6d0 con 0x7fd61815bb50 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.471+0000 7fd61fa38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180c8160 con 0x7fd6180cfb40 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.471+0000 7fd61fa38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61815f850 con 0x7fd6180c70d0 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61dfd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd61815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61dfd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd61815c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38980/0 (socket says 172.21.15.19:38980) 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61dfd5640 1 -- 172.21.15.19:0/822422000 learned_addr learned my addr 172.21.15.19:0/822422000 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61cfd3640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6180cfb40 0x7fd61815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61d7d4640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd6181f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61dfd5640 1 -- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 msgr2=0x7fd6181f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61dfd5640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd6181f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61dfd5640 1 -- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6180cfb40 msgr2=0x7fd61815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61dfd5640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6180cfb40 0x7fd61815e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61dfd5640 1 -- 172.21.15.19:0/822422000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61815ea50 con 0x7fd61815bb50 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61cfd3640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6180cfb40 0x7fd61815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61d7d4640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6180c70d0 0x7fd6181f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61dfd5640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd61815c230 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fd61000c3f0 tx=0x7fd61000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd615ffb640 1 -- 172.21.15.19:0/822422000 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6100052d0 con 0x7fd61815bb50 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd615ffb640 1 -- 172.21.15.19:0/822422000 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd610005490 con 0x7fd61815bb50 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61fa38640 1 -- 172.21.15.19:0/822422000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61815ed30 con 0x7fd61815bb50 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd615ffb640 1 -- 172.21.15.19:0/822422000 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610007260 con 0x7fd61815bb50 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.472+0000 7fd61fa38640 1 -- 172.21.15.19:0/822422000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6180b52f0 con 0x7fd61815bb50 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.473+0000 7fd61fa38640 1 -- 172.21.15.19:0/822422000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd61815f250 con 0x7fd61815bb50 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.474+0000 7fd615ffb640 1 -- 172.21.15.19:0/822422000 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd610007400 con 0x7fd61815bb50 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.474+0000 7fd615ffb640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd60406b4d0 0x7fd60406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.474+0000 7fd615ffb640 1 -- 172.21.15.19:0/822422000 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd6100895f0 con 0x7fd61815bb50 2024-04-25T10:07:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.474+0000 7fd615ffb640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x7fd604071a10 0x7fd604073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.474+0000 7fd615ffb640 1 -- 172.21.15.19:0/822422000 --> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd604074520 con 0x7fd604071a10 2024-04-25T10:07:14.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.474+0000 7fd615ffb640 1 -- 172.21.15.19:0/822422000 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd610090050 con 0x7fd61815bb50 2024-04-25T10:07:14.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.474+0000 7fd61d7d4640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd60406b4d0 0x7fd60406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.475+0000 7fd61cfd3640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x7fd604071a10 0x7fd604073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.475+0000 7fd61cfd3640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x7fd604071a10 0x7fd604073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.475+0000 7fd61d7d4640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd60406b4d0 0x7fd60406d990 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd608033000 tx=0x7fd608009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.476+0000 7fd615ffb640 1 -- 172.21.15.19:0/822422000 <== osd.6 v2:172.21.15.112:6816/3693987094 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd604074520 con 0x7fd604071a10 2024-04-25T10:07:14.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.489+0000 7fd61fa38640 1 -- 172.21.15.19:0/822422000 --> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd618073a00 con 0x7fd604071a10 2024-04-25T10:07:14.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.489+0000 7fd615ffb640 1 -- 172.21.15.19:0/822422000 <== osd.6 v2:172.21.15.112:6816/3693987094 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd618073a00 con 0x7fd604071a10 2024-04-25T10:07:14.495 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:07:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:07:14.496 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:07:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.490+0000 7fd61fa38640 1 -- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x7fd604071a10 msgr2=0x7fd604073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.490+0000 7fd61fa38640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:6816/3693987094,v1:172.21.15.112:6818/3693987094] conn(0x7fd604071a10 0x7fd604073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.490+0000 7fd61fa38640 1 -- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd60406b4d0 msgr2=0x7fd60406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.490+0000 7fd61fa38640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd60406b4d0 0x7fd60406d990 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd608033000 tx=0x7fd608009000 comp rx=0 tx=0).stop 2024-04-25T10:07:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.490+0000 7fd61fa38640 1 -- 172.21.15.19:0/822422000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 msgr2=0x7fd61815c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.490+0000 7fd61fa38640 1 --2- 172.21.15.19:0/822422000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd61815bb50 0x7fd61815c230 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fd61000c3f0 tx=0x7fd61000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.490+0000 7fd61dfd5640 1 -- 172.21.15.19:0/822422000 reap_dead start 2024-04-25T10:07:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.490+0000 7fd61fa38640 1 -- 172.21.15.19:0/822422000 shutdown_connections 2024-04-25T10:07:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.490+0000 7fd61fa38640 1 -- 172.21.15.19:0/822422000 >> 172.21.15.19:0/822422000 conn(0x7fd61806f500 msgr2=0x7fd6180cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.490+0000 7fd61fa38640 1 -- 172.21.15.19:0/822422000 shutdown_connections 2024-04-25T10:07:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.490+0000 7fd61fa38640 1 -- 172.21.15.19:0/822422000 wait complete. 2024-04-25T10:07:14.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:07:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.602+0000 7fddc841f640 1 Processor -- start 2024-04-25T10:07:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.602+0000 7fddc841f640 1 -- start start 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.602+0000 7fddc841f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddc006c9e0 0x7fddc00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc841f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc006cf50 con 0x7fddc00b5620 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc841f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc00ce530 con 0x7fddc015b7d0 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc841f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00ce810 con 0x7fddc006c9e0 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc59ba640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fddc00b5620 0x7fddc006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54356/0 (socket says 172.21.15.19:54356) 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc59ba640 1 -- 172.21.15.19:0/2011832775 learned_addr learned my addr 172.21.15.19:0/2011832775 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826941614 0 0) 0x7fddc006cf50 con 0x7fddc00b5620 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc69bc640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddc006c9e0 0x7fddc00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddac003520 con 0x7fddc00b5620 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1504140477 0 0) 0x7fddac003520 con 0x7fddc00b5620 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddc006c9e0 msgr2=0x7fddc00b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddc006c9e0 0x7fddc00b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 >> v1:172.21.15.112:6789/0 conn(0x7fddc015b7d0 legacy=0x7fddc00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc00ceaf0 con 0x7fddc00b5620 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fddb0001690 con 0x7fddc00b5620 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 >> v1:172.21.15.19:6789/0 conn(0x7fddc00b5620 legacy=0x7fddc006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddc006c9e0 0x7fddac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddac003f60 0x7fddac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddac004d40 0x7fddac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00ce810 con 0x7fddac003f60 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddac00baa0 con 0x7fddac004d40 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddac00bca0 con 0x7fddc006c9e0 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc59ba640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddac003f60 0x7fddac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.603+0000 7fddc61bb640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddc006c9e0 0x7fddac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.604+0000 7fddc69bc640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddac004d40 0x7fddac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.604+0000 7fddc59ba640 1 -- 172.21.15.19:0/2011832775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddc006c9e0 msgr2=0x7fddac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.604+0000 7fddc59ba640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddc006c9e0 0x7fddac0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.604+0000 7fddc59ba640 1 -- 172.21.15.19:0/2011832775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddac004d40 msgr2=0x7fddac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.604+0000 7fddc59ba640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddac004d40 0x7fddac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.604+0000 7fddc59ba640 1 -- 172.21.15.19:0/2011832775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac003520 con 0x7fddac003f60 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.604+0000 7fddc59ba640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddac003f60 0x7fddac004800 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c210 tx=0x7fddac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1469273535760c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.605+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0007d10 con 0x7fddac003f60 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.605+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb0005af0 con 0x7fddac003f60 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.605+0000 7fddc51b9640 1 -- 172.21.15.19:0/2011832775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0007070 con 0x7fddac003f60 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.605+0000 7fddc841f640 1 -- 172.21.15.19:0/2011832775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddac003f60 msgr2=0x7fddac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.605+0000 7fddc841f640 1 --2- 172.21.15.19:0/2011832775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddac003f60 0x7fddac004800 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c210 tx=0x7fddac005830 comp rx=0 tx=0).stop 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.605+0000 7fddc69bc640 1 -- 172.21.15.19:0/2011832775 reap_dead start 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.605+0000 7fddc841f640 1 -- 172.21.15.19:0/2011832775 shutdown_connections 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.605+0000 7fddc841f640 1 -- 172.21.15.19:0/2011832775 >> 172.21.15.19:0/2011832775 conn(0x7fddc00d0800 msgr2=0x7fddc00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.605+0000 7fddc841f640 1 -- 172.21.15.19:0/2011832775 shutdown_connections 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.605+0000 7fddc841f640 1 -- 172.21.15.19:0/2011832775 wait complete. 2024-04-25T10:07:14.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.606+0000 7fddc841f640 1 Processor -- start 2024-04-25T10:07:14.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.606+0000 7fddc841f640 1 -- start start 2024-04-25T10:07:14.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.607+0000 7fddc841f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddac003f60 0x7fddc01f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.607+0000 7fddc841f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc01f0c10 0x7fddc00cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.607+0000 7fddc841f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc00cc8d0 0x7fddc00b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.607+0000 7fddc841f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc015bd30 con 0x7fddc00cc8d0 2024-04-25T10:07:14.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.607+0000 7fddc841f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc015bbb0 con 0x7fddc01f0c10 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.607+0000 7fddc841f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00b5ab0 con 0x7fddac003f60 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.607+0000 7fddc69bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc00cc8d0 0x7fddc00b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.607+0000 7fddc69bc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc00cc8d0 0x7fddc00b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39006/0 (socket says 172.21.15.19:39006) 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.607+0000 7fddc69bc640 1 -- 172.21.15.19:0/1765925103 learned_addr learned my addr 172.21.15.19:0/1765925103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.607+0000 7fddc59ba640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc01f0c10 0x7fddc00cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.607+0000 7fddc69bc640 1 -- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddac003f60 msgr2=0x7fddc01f48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.608+0000 7fddc69bc640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddac003f60 0x7fddc01f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.608+0000 7fddc69bc640 1 -- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc01f0c10 msgr2=0x7fddc00cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.608+0000 7fddc69bc640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc01f0c10 0x7fddc00cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.608+0000 7fddc69bc640 1 -- 172.21.15.19:0/1765925103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc01f3a40 con 0x7fddc00cc8d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.608+0000 7fddc69bc640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc00cc8d0 0x7fddc00b8050 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7fddc00d4ca0 tx=0x7fddb800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.608+0000 7fddb6ffd640 1 -- 172.21.15.19:0/1765925103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800fce0 con 0x7fddc00cc8d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.608+0000 7fddb6ffd640 1 -- 172.21.15.19:0/1765925103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb800fe80 con 0x7fddc00cc8d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.608+0000 7fddb6ffd640 1 -- 172.21.15.19:0/1765925103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8015670 con 0x7fddc00cc8d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.608+0000 7fddc841f640 1 -- 172.21.15.19:0/1765925103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc01f3d20 con 0x7fddc00cc8d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.608+0000 7fddc841f640 1 -- 172.21.15.19:0/1765925103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc01f42e0 con 0x7fddc00cc8d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.609+0000 7fddc841f640 1 -- 172.21.15.19:0/1765925103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc0073a00 con 0x7fddc00cc8d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.609+0000 7fddb6ffd640 1 -- 172.21.15.19:0/1765925103 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddb8015810 con 0x7fddc00cc8d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.610+0000 7fddb6ffd640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdda406b4d0 0x7fdda406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.610+0000 7fddb6ffd640 1 -- 172.21.15.19:0/1765925103 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fddb808aef0 con 0x7fddc00cc8d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.610+0000 7fddc61bb640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdda406b4d0 0x7fdda406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.610+0000 7fddc61bb640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdda406b4d0 0x7fdda406d990 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fddc00c85f0 tx=0x7fddbc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.614+0000 7fddb6ffd640 1 -- 172.21.15.19:0/1765925103 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fddb8019030 con 0x7fddc00cc8d0 2024-04-25T10:07:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.796+0000 7fddc841f640 1 -- 172.21.15.19:0/1765925103 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fddc01611e0 con 0x7fdda406b4d0 2024-04-25T10:07:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.797+0000 7fddb6ffd640 1 -- 172.21.15.19:0/1765925103 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fddc01611e0 con 0x7fdda406b4d0 2024-04-25T10:07:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:07:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 -- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdda406b4d0 msgr2=0x7fdda406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdda406b4d0 0x7fdda406d990 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fddc00c85f0 tx=0x7fddbc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 -- 172.21.15.19:0/1765925103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc00cc8d0 msgr2=0x7fddc00b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc00cc8d0 0x7fddc00b8050 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7fddc00d4ca0 tx=0x7fddb800f1b0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 -- 172.21.15.19:0/1765925103 shutdown_connections 2024-04-25T10:07:14.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fdda406b4d0 0x7fdda406d990 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddc00cc8d0 0x7fddc00b8050 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc01f0c10 0x7fddc00cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 --2- 172.21.15.19:0/1765925103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddac003f60 0x7fddc01f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 -- 172.21.15.19:0/1765925103 >> 172.21.15.19:0/1765925103 conn(0x7fddc00d0800 msgr2=0x7fddc00b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 -- 172.21.15.19:0/1765925103 shutdown_connections 2024-04-25T10:07:14.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.799+0000 7fddc841f640 1 -- 172.21.15.19:0/1765925103 wait complete. 2024-04-25T10:07:14.816 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:07:14.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:07:14.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.915+0000 7f3344522640 1 Processor -- start 2024-04-25T10:07:14.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.915+0000 7f3344522640 1 -- start start 2024-04-25T10:07:14.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f3344522640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f333c15dc20 0x7f333c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f3344522640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c161710 0x7f333c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f3344522640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f333c0b9330 0x7f333c06bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f3344522640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333c15e000 con 0x7f333c15dc20 2024-04-25T10:07:14.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f3344522640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333c0d1de0 con 0x7f333c0b9330 2024-04-25T10:07:14.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f3344522640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c161c60 con 0x7f333c161710 2024-04-25T10:07:14.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f33422be640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f333c15dc20 0x7f333c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f33422be640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f333c15dc20 0x7f333c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39012/0 (socket says 172.21.15.19:39012) 2024-04-25T10:07:14.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f33422be640 1 -- 172.21.15.19:0/1175561524 learned_addr learned my addr 172.21.15.19:0/1175561524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f3341abd640 1 --2- 172.21.15.19:0/1175561524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c161710 0x7f333c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f3342abf640 1 --2- 172.21.15.19:0/1175561524 >> v2:172.21.15.112:3300/0 conn(0x7f333c0b9330 0x7f333c06bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f33422be640 1 -- 172.21.15.19:0/1175561524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c161710 msgr2=0x7f333c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f33422be640 1 --2- 172.21.15.19:0/1175561524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c161710 0x7f333c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f33422be640 1 -- 172.21.15.19:0/1175561524 >> v2:172.21.15.112:3300/0 conn(0x7f333c0b9330 msgr2=0x7f333c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f33422be640 1 --2- 172.21.15.19:0/1175561524 >> v2:172.21.15.112:3300/0 conn(0x7f333c0b9330 0x7f333c06bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f33422be640 1 -- 172.21.15.19:0/1175561524 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c161f40 con 0x7f333c15dc20 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f3342abf640 1 --2- 172.21.15.19:0/1175561524 >> v2:172.21.15.112:3300/0 conn(0x7f333c0b9330 0x7f333c06bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.916+0000 7f3341abd640 1 --2- 172.21.15.19:0/1175561524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c161710 0x7f333c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.917+0000 7f33422be640 1 --2- 172.21.15.19:0/1175561524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c15dc20 0x7f333c0ce670 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f333c0b6570 tx=0x7f333402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3f278d5b8132ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.917+0000 7f33412bc640 1 -- 172.21.15.19:0/1175561524 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334037070 con 0x7f333c15dc20 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.917+0000 7f33412bc640 1 -- 172.21.15.19:0/1175561524 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3334032440 con 0x7f333c15dc20 2024-04-25T10:07:14.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.917+0000 7f33412bc640 1 -- 172.21.15.19:0/1175561524 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334032810 con 0x7f333c15dc20 2024-04-25T10:07:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.918+0000 7f3344522640 1 -- 172.21.15.19:0/1175561524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c15dc20 msgr2=0x7f333c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.918+0000 7f3344522640 1 --2- 172.21.15.19:0/1175561524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c15dc20 0x7f333c0ce670 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f333c0b6570 tx=0x7f333402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.918+0000 7f3344522640 1 -- 172.21.15.19:0/1175561524 shutdown_connections 2024-04-25T10:07:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.918+0000 7f3344522640 1 --2- 172.21.15.19:0/1175561524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c161710 0x7f333c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.918+0000 7f3344522640 1 --2- 172.21.15.19:0/1175561524 >> v2:172.21.15.112:3300/0 conn(0x7f333c0b9330 0x7f333c06bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.918+0000 7f3344522640 1 --2- 172.21.15.19:0/1175561524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c15dc20 0x7f333c0ce670 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.918+0000 7f3344522640 1 -- 172.21.15.19:0/1175561524 >> 172.21.15.19:0/1175561524 conn(0x7f333c0d0420 msgr2=0x7f333c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.918+0000 7f3344522640 1 -- 172.21.15.19:0/1175561524 shutdown_connections 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.918+0000 7f3344522640 1 -- 172.21.15.19:0/1175561524 wait complete. 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.919+0000 7f3344522640 1 Processor -- start 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.919+0000 7f3344522640 1 -- start start 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3344522640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c0b9330 0x7f333c0aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3344522640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c15dc20 0x7f333c160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3344522640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c161710 0x7f333c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3344522640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0b5e50 con 0x7f333c15dc20 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3344522640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c162500 con 0x7f333c0b9330 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3344522640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0b5fd0 con 0x7f333c161710 2024-04-25T10:07:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3341abd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c15dc20 0x7f333c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3341abd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c15dc20 0x7f333c160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39022/0 (socket says 172.21.15.19:39022) 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3341abd640 1 -- 172.21.15.19:0/1268934437 learned_addr learned my addr 172.21.15.19:0/1268934437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3342abf640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c161710 0x7f333c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f33422be640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c0b9330 0x7f333c0aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3341abd640 1 -- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c161710 msgr2=0x7f333c0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3341abd640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f333c161710 0x7f333c0cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3341abd640 1 -- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c0b9330 msgr2=0x7f333c0aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3341abd640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c0b9330 0x7f333c0aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3341abd640 1 -- 172.21.15.19:0/1268934437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c160c50 con 0x7f333c15dc20 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.920+0000 7f3341abd640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c15dc20 0x7f333c160710 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f333000c820 tx=0x7f333000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.921+0000 7f333affd640 1 -- 172.21.15.19:0/1268934437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330015070 con 0x7f333c15dc20 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.921+0000 7f333affd640 1 -- 172.21.15.19:0/1268934437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3330007a30 con 0x7f333c15dc20 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.921+0000 7f3344522640 1 -- 172.21.15.19:0/1268934437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c149f00 con 0x7f333c15dc20 2024-04-25T10:07:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.921+0000 7f333affd640 1 -- 172.21.15.19:0/1268934437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330007de0 con 0x7f333c15dc20 2024-04-25T10:07:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.921+0000 7f3344522640 1 -- 172.21.15.19:0/1268934437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c14a4c0 con 0x7f333c15dc20 2024-04-25T10:07:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.922+0000 7f3344522640 1 -- 172.21.15.19:0/1268934437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f333c073a00 con 0x7f333c15dc20 2024-04-25T10:07:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.922+0000 7f333affd640 1 -- 172.21.15.19:0/1268934437 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3330018070 con 0x7f333c15dc20 2024-04-25T10:07:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.922+0000 7f333affd640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f332806b500 0x7f332806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.923+0000 7f333affd640 1 -- 172.21.15.19:0/1268934437 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f333008a020 con 0x7f333c15dc20 2024-04-25T10:07:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.923+0000 7f333affd640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] conn(0x7f3328071a40 0x7f3328073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.923+0000 7f333affd640 1 -- 172.21.15.19:0/1268934437 --> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f333001be50 con 0x7f3328071a40 2024-04-25T10:07:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.923+0000 7f333affd640 1 -- 172.21.15.19:0/1268934437 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3330091050 con 0x7f333c15dc20 2024-04-25T10:07:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.923+0000 7f33422be640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f332806b500 0x7f332806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.923+0000 7f3342abf640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] conn(0x7f3328071a40 0x7f3328073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.923+0000 7f33422be640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f332806b500 0x7f332806d9c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3334034000 tx=0x7f333402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.923+0000 7f3342abf640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] conn(0x7f3328071a40 0x7f3328073e80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.924+0000 7f333affd640 1 -- 172.21.15.19:0/1268934437 <== osd.7 v2:172.21.15.112:6817/3405244921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f333001be50 con 0x7f3328071a40 2024-04-25T10:07:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.937+0000 7f3344522640 1 -- 172.21.15.19:0/1268934437 --> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f333c0d16b0 con 0x7f3328071a40 2024-04-25T10:07:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f333affd640 1 -- 172.21.15.19:0/1268934437 <== osd.7 v2:172.21.15.112:6817/3405244921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f333c0d16b0 con 0x7f3328071a40 2024-04-25T10:07:14.944 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T10:07:14.944 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:07:14.944 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T10:07:14.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f3344522640 1 -- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] conn(0x7f3328071a40 msgr2=0x7f3328073e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f3344522640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:6817/3405244921,v1:172.21.15.112:6819/3405244921] conn(0x7f3328071a40 0x7f3328073e80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f3344522640 1 -- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f332806b500 msgr2=0x7f332806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f3344522640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f332806b500 0x7f332806d9c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3334034000 tx=0x7f333402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f3344522640 1 -- 172.21.15.19:0/1268934437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c15dc20 msgr2=0x7f333c160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f3344522640 1 --2- 172.21.15.19:0/1268934437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c15dc20 0x7f333c160710 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f333000c820 tx=0x7f333000cce0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f3342abf640 1 -- 172.21.15.19:0/1268934437 reap_dead start 2024-04-25T10:07:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f3344522640 1 -- 172.21.15.19:0/1268934437 shutdown_connections 2024-04-25T10:07:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f3344522640 1 -- 172.21.15.19:0/1268934437 >> 172.21.15.19:0/1268934437 conn(0x7f333c0d0420 msgr2=0x7f333c0d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f3344522640 1 -- 172.21.15.19:0/1268934437 shutdown_connections 2024-04-25T10:07:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:14.938+0000 7f3344522640 1 -- 172.21.15.19:0/1268934437 wait complete. 2024-04-25T10:07:14.954 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:07:15.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.051+0000 7f6dcfe6b640 1 Processor -- start 2024-04-25T10:07:15.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.051+0000 7f6dcfe6b640 1 -- start start 2024-04-25T10:07:15.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcfe6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 0x7f6dc80b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcfe6b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6dc8162450 0x7f6dc80cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcfe6b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc80cf320 con 0x7f6dc815b980 2024-04-25T10:07:15.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcfe6b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc80cf040 con 0x7f6dc8162450 2024-04-25T10:07:15.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcfe6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc8161970 con 0x7f6dc815b5a0 2024-04-25T10:07:15.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcdc07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 0x7f6dc80b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dce408640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6dc8162450 0x7f6dc80cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcdc07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 0x7f6dc80b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.19:56998/0 (socket says 172.21.15.19:56998) 2024-04-25T10:07:15.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcd406640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6dc815b980 0x7f6dc80ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.19:33282/0 (socket says 172.21.15.19:33282) 2024-04-25T10:07:15.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dce408640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6dc8162450 0x7f6dc80cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.19:55538/0 (socket says 172.21.15.19:55538) 2024-04-25T10:07:15.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcdc07640 1 -- 172.21.15.19:0/1500906653 learned_addr learned my addr 172.21.15.19:0/1500906653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:15.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcdc07640 1 -- 172.21.15.19:0/1500906653 >> v2:172.21.15.112:3300/0 conn(0x7f6dc8162450 msgr2=0x7f6dc80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcdc07640 1 --2- 172.21.15.19:0/1500906653 >> v2:172.21.15.112:3300/0 conn(0x7f6dc8162450 0x7f6dc80cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcdc07640 1 -- 172.21.15.19:0/1500906653 >> v1:172.21.15.112:6789/0 conn(0x7f6dc815b980 legacy=0x7f6dc80ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.052+0000 7f6dcdc07640 1 -- 172.21.15.19:0/1500906653 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc8161c50 con 0x7f6dc815b5a0 2024-04-25T10:07:15.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.053+0000 7f6dcdc07640 1 --2- 172.21.15.19:0/1500906653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 0x7f6dc80b5530 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009870 tx=0x7f6dc402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8100f5cc4810c0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.053+0000 7f6dccc05640 1 -- 172.21.15.19:0/1500906653 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4037070 con 0x7f6dc815b5a0 2024-04-25T10:07:15.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.053+0000 7f6dccc05640 1 -- 172.21.15.19:0/1500906653 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc4032440 con 0x7f6dc815b5a0 2024-04-25T10:07:15.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.053+0000 7f6dccc05640 1 -- 172.21.15.19:0/1500906653 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4032810 con 0x7f6dc815b5a0 2024-04-25T10:07:15.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.054+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1500906653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 msgr2=0x7f6dc80b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.054+0000 7f6dcfe6b640 1 --2- 172.21.15.19:0/1500906653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 0x7f6dc80b5530 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009870 tx=0x7f6dc402a270 comp rx=0 tx=0).stop 2024-04-25T10:07:15.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.054+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1500906653 shutdown_connections 2024-04-25T10:07:15.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.054+0000 7f6dcfe6b640 1 --2- 172.21.15.19:0/1500906653 >> v2:172.21.15.112:3300/0 conn(0x7f6dc8162450 0x7f6dc80cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.054+0000 7f6dcfe6b640 1 --2- 172.21.15.19:0/1500906653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 0x7f6dc80b5530 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.054+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1500906653 >> 172.21.15.19:0/1500906653 conn(0x7f6dc80d0530 msgr2=0x7f6dc80d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.054+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1500906653 shutdown_connections 2024-04-25T10:07:15.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.054+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1500906653 wait complete. 2024-04-25T10:07:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.055+0000 7f6dcfe6b640 1 Processor -- start 2024-04-25T10:07:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.055+0000 7f6dcfe6b640 1 -- start start 2024-04-25T10:07:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dcfe6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 0x7f6dc80c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dcfe6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc815b980 0x7f6dc814cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dcfe6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc8162450 0x7f6dc814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dcfe6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc80b5bf0 con 0x7f6dc8162450 2024-04-25T10:07:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dcfe6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc80b5a70 con 0x7f6dc815b980 2024-04-25T10:07:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dcfe6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc80b5d70 con 0x7f6dc815b5a0 2024-04-25T10:07:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dce408640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc8162450 0x7f6dc814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dce408640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc8162450 0x7f6dc814eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39028/0 (socket says 172.21.15.19:39028) 2024-04-25T10:07:15.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dce408640 1 -- 172.21.15.19:0/1269381888 learned_addr learned my addr 172.21.15.19:0/1269381888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dcdc07640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 0x7f6dc80c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dcd406640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc815b980 0x7f6dc814cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dce408640 1 -- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 msgr2=0x7f6dc80c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dce408640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 0x7f6dc80c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dce408640 1 -- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc815b980 msgr2=0x7f6dc814cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dce408640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc815b980 0x7f6dc814cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dce408640 1 -- 172.21.15.19:0/1269381888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc814d620 con 0x7f6dc8162450 2024-04-25T10:07:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.056+0000 7f6dce408640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc8162450 0x7f6dc814eb20 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009d30 tx=0x7f6dc000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.057+0000 7f6dbe7fc640 1 -- 172.21.15.19:0/1269381888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0018020 con 0x7f6dc8162450 2024-04-25T10:07:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.057+0000 7f6dbe7fc640 1 -- 172.21.15.19:0/1269381888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc0014480 con 0x7f6dc8162450 2024-04-25T10:07:15.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.057+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1269381888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc814d900 con 0x7f6dc8162450 2024-04-25T10:07:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.057+0000 7f6dbe7fc640 1 -- 172.21.15.19:0/1269381888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0014870 con 0x7f6dc8162450 2024-04-25T10:07:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.057+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1269381888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc80cb550 con 0x7f6dc8162450 2024-04-25T10:07:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.058+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1269381888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc814de20 con 0x7f6dc8162450 2024-04-25T10:07:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.058+0000 7f6dbe7fc640 1 -- 172.21.15.19:0/1269381888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6dc0005d10 con 0x7f6dc8162450 2024-04-25T10:07:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.058+0000 7f6dbe7fc640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6dac06b500 0x7f6dac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.059+0000 7f6dbe7fc640 1 -- 172.21.15.19:0/1269381888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f6dc008aac0 con 0x7f6dc8162450 2024-04-25T10:07:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.059+0000 7f6dcdc07640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6dac06b500 0x7f6dac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.059+0000 7f6dcdc07640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6dac06b500 0x7f6dac06d9c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4034000 tx=0x7f6dc402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.063+0000 7f6dbe7fc640 1 -- 172.21.15.19:0/1269381888 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6dc814de20 con 0x7f6dc8162450 2024-04-25T10:07:15.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.243+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1269381888 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dc8157130 con 0x7f6dac06b500 2024-04-25T10:07:15.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.244+0000 7f6dbe7fc640 1 -- 172.21.15.19:0/1269381888 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6dc8157130 con 0x7f6dac06b500 2024-04-25T10:07:15.249 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:07:15.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6dac06b500 msgr2=0x7f6dac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6dac06b500 0x7f6dac06d9c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4034000 tx=0x7f6dc402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1269381888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc8162450 msgr2=0x7f6dc814eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc8162450 0x7f6dc814eb20 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009d30 tx=0x7f6dc000c490 comp rx=0 tx=0).stop 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1269381888 shutdown_connections 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f6dac06b500 0x7f6dac06d9c0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dc8162450 0x7f6dc814eb20 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc815b980 0x7f6dc814cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 --2- 172.21.15.19:0/1269381888 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc815b5a0 0x7f6dc80c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1269381888 >> 172.21.15.19:0/1269381888 conn(0x7f6dc80d0530 msgr2=0x7f6dc8156390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1269381888 shutdown_connections 2024-04-25T10:07:15.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.246+0000 7f6dcfe6b640 1 -- 172.21.15.19:0/1269381888 wait complete. 2024-04-25T10:07:15.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.358+0000 7efef8d68640 1 Processor -- start 2024-04-25T10:07:15.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.359+0000 7efef8d68640 1 -- start start 2024-04-25T10:07:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.359+0000 7efef8d68640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efef415db60 0x7efef40ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.359+0000 7efef8d68640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efef40b5f00 0x7efef40d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.359+0000 7efef8d68640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef40cec00 con 0x7efef415df40 2024-04-25T10:07:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.359+0000 7efef8d68640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef40ce930 con 0x7efef415db60 2024-04-25T10:07:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.359+0000 7efef8d68640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef40b6450 con 0x7efef40b5f00 2024-04-25T10:07:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.360+0000 7efef259c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efef415db60 0x7efef40ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.360+0000 7efef259c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efef415db60 0x7efef40ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39038/0 (socket says 172.21.15.19:39038) 2024-04-25T10:07:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.360+0000 7efef259c640 1 -- 172.21.15.19:0/1566960481 learned_addr learned my addr 172.21.15.19:0/1566960481 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.360+0000 7efef1d9b640 1 --2- 172.21.15.19:0/1566960481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efef40b5f00 0x7efef40d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.360+0000 7efef259c640 1 -- 172.21.15.19:0/1566960481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efef40b5f00 msgr2=0x7efef40d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.360+0000 7efef259c640 1 --2- 172.21.15.19:0/1566960481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efef40b5f00 0x7efef40d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.360+0000 7efef259c640 1 -- 172.21.15.19:0/1566960481 >> v1:172.21.15.112:6789/0 conn(0x7efef415df40 legacy=0x7efef4161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.360+0000 7efef259c640 1 -- 172.21.15.19:0/1566960481 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef40ceee0 con 0x7efef415db60 2024-04-25T10:07:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef259c640 1 --2- 172.21.15.19:0/1566960481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef415db60 0x7efef40ce3f0 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7efee40098a0 tx=0x7efee402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f502ab17b0568a65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef159a640 1 -- 172.21.15.19:0/1566960481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4037070 con 0x7efef415db60 2024-04-25T10:07:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef159a640 1 -- 172.21.15.19:0/1566960481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee4032440 con 0x7efef415db60 2024-04-25T10:07:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef159a640 1 -- 172.21.15.19:0/1566960481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4032810 con 0x7efef415db60 2024-04-25T10:07:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef8d68640 1 -- 172.21.15.19:0/1566960481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef415db60 msgr2=0x7efef40ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef8d68640 1 --2- 172.21.15.19:0/1566960481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef415db60 0x7efef40ce3f0 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7efee40098a0 tx=0x7efee402a200 comp rx=0 tx=0).stop 2024-04-25T10:07:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef8d68640 1 -- 172.21.15.19:0/1566960481 shutdown_connections 2024-04-25T10:07:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef8d68640 1 --2- 172.21.15.19:0/1566960481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efef40b5f00 0x7efef40d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef8d68640 1 --2- 172.21.15.19:0/1566960481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef415db60 0x7efef40ce3f0 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef8d68640 1 -- 172.21.15.19:0/1566960481 >> 172.21.15.19:0/1566960481 conn(0x7efef4061600 msgr2=0x7efef4061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef8d68640 1 -- 172.21.15.19:0/1566960481 shutdown_connections 2024-04-25T10:07:15.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.361+0000 7efef8d68640 1 -- 172.21.15.19:0/1566960481 wait complete. 2024-04-25T10:07:15.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.363+0000 7efef8d68640 1 Processor -- start 2024-04-25T10:07:15.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.363+0000 7efef8d68640 1 -- start start 2024-04-25T10:07:15.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.363+0000 7efef8d68640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efef40b5f00 0x7efef414eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef8d68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef415db60 0x7efef41604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef8d68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef415df40 0x7efef4160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef8d68640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef40d3280 con 0x7efef415db60 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef8d68640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef40d1d50 con 0x7efef415df40 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef8d68640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef40d3400 con 0x7efef40b5f00 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef415db60 0x7efef41604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef415db60 0x7efef41604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39040/0 (socket says 172.21.15.19:39040) 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef1d9b640 1 -- 172.21.15.19:0/4290913207 learned_addr learned my addr 172.21.15.19:0/4290913207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef259c640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efef40b5f00 0x7efef414eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef2d9d640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef415df40 0x7efef4160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef1d9b640 1 -- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efef40b5f00 msgr2=0x7efef414eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef1d9b640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efef40b5f00 0x7efef414eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef1d9b640 1 -- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef415df40 msgr2=0x7efef4160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef1d9b640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef415df40 0x7efef4160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef1d9b640 1 -- 172.21.15.19:0/4290913207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef40c9270 con 0x7efef415db60 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef259c640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efef40b5f00 0x7efef414eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:15.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef1d9b640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef415db60 0x7efef41604d0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7efee800e840 tx=0x7efee800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efee2ffd640 1 -- 172.21.15.19:0/4290913207 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8017070 con 0x7efef415db60 2024-04-25T10:07:15.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efee2ffd640 1 -- 172.21.15.19:0/4290913207 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee800c940 con 0x7efef415db60 2024-04-25T10:07:15.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef8d68640 1 -- 172.21.15.19:0/4290913207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef40c9550 con 0x7efef415db60 2024-04-25T10:07:15.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efee2ffd640 1 -- 172.21.15.19:0/4290913207 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee800cd10 con 0x7efef415db60 2024-04-25T10:07:15.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.364+0000 7efef8d68640 1 -- 172.21.15.19:0/4290913207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef40c9b10 con 0x7efef415db60 2024-04-25T10:07:15.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.365+0000 7efef8d68640 1 -- 172.21.15.19:0/4290913207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef4073270 con 0x7efef415db60 2024-04-25T10:07:15.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.366+0000 7efee2ffd640 1 -- 172.21.15.19:0/4290913207 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efee801a070 con 0x7efef415db60 2024-04-25T10:07:15.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.366+0000 7efee2ffd640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efed406b500 0x7efed406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.366+0000 7efee2ffd640 1 -- 172.21.15.19:0/4290913207 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7efee808a300 con 0x7efef415db60 2024-04-25T10:07:15.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.366+0000 7efef259c640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efed406b500 0x7efed406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.367+0000 7efef259c640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efed406b500 0x7efed406d9c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7efee4034000 tx=0x7efee402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.370+0000 7efee2ffd640 1 -- 172.21.15.19:0/4290913207 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efee80592a0 con 0x7efef415db60 2024-04-25T10:07:15.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.548+0000 7efef8d68640 1 -- 172.21.15.19:0/4290913207 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efef408b570 con 0x7efed406b500 2024-04-25T10:07:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.549+0000 7efee2ffd640 1 -- 172.21.15.19:0/4290913207 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13395 (secure 0 0 0) 0x7efef408b570 con 0x7efed406b500 2024-04-25T10:07:15.554 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:15.555 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 -- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efed406b500 msgr2=0x7efed406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efed406b500 0x7efed406d9c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7efee4034000 tx=0x7efee402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 -- 172.21.15.19:0/4290913207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef415db60 msgr2=0x7efef41604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef415db60 0x7efef41604d0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7efee800e840 tx=0x7efee800ed00 comp rx=0 tx=0).stop 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 -- 172.21.15.19:0/4290913207 shutdown_connections 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7efed406b500 0x7efed406d9c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef415df40 0x7efef4160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef415db60 0x7efef41604d0 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 --2- 172.21.15.19:0/4290913207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efef40b5f00 0x7efef414eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 -- 172.21.15.19:0/4290913207 >> 172.21.15.19:0/4290913207 conn(0x7efef4061600 msgr2=0x7efef41618e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 -- 172.21.15.19:0/4290913207 shutdown_connections 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:15.551+0000 7efef8d68640 1 -- 172.21.15.19:0/4290913207 wait complete. 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":619,"stamp":"2024-04-25T10:07:15.112325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":469760,"kb_used_data":20488,"kb_used_omap":12,"kb_used_meta":449139,"kb_avail":754504960,"statfs":{"total":773094113280,"available":772613079040,"internally_reserved":0,"allocated":20979712,"data_stored":10485456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":459918898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12582912,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-8,"num_write_kb":-12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.589123"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":527,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:07:06.529011+0000","last_change":"2024-04-25T09:52:56.062584+0000","last_active":"2024-04-25T10:07:06.529011+0000","last_peered":"2024-04-25T10:07:06.529011+0000","last_clean":"2024-04-25T10:07:06.529011+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:07:06.529011+0000","last_undegraded":"2024-04-25T10:07:06.529011+0000","last_fullsized":"2024-04-25T10:07:06.529011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:52:54.954371+0000","last_clean_scrub_stamp":"2024-04-25T09:52:54.954371+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.515024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575024,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56384,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315456,"statfs":{"total":96636764160,"available":96579026944,"internally_reserved":0,"allocated":3067904,"data_stored":1753458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51652,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320188,"statfs":{"total":96636764160,"available":96583872512,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51648,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320192,"statfs":{"total":96636764160,"available":96583876608,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59964,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82112,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94289728,"statfs":{"total":96636764160,"available":96552681472,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81590727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55808,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316032,"statfs":{"total":96636764160,"available":96579616768,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55804,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316036,"statfs":{"total":96636764160,"available":96579620864,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56388,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315452,"statfs":{"total":96636764160,"available":96579022848,"internally_reserved":0,"allocated":3067904,"data_stored":1753458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:15.572 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T09:52:54.954371+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=9, tm_min=52, tm_sec=54, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=7, tm_sec=10, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:07:15.573 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T10:07:35.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:35.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.682+0000 7fd1e817e640 1 Processor -- start 2024-04-25T10:07:35.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.682+0000 7fd1e817e640 1 -- start start 2024-04-25T10:07:35.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.683+0000 7fd1e817e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd1e0144750 0x7fd1e006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.683+0000 7fd1e817e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 0x7fd1e006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.683+0000 7fd1e817e640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e0144b30 con 0x7fd1e006eb10 2024-04-25T10:07:35.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.683+0000 7fd1e817e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e00712d0 con 0x7fd1e0144750 2024-04-25T10:07:35.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.683+0000 7fd1e817e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00715b0 con 0x7fd1e00cf030 2024-04-25T10:07:35.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.683+0000 7fd1e5719640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd1e006eb10 0x7fd1e00ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59776/0 (socket says 172.21.15.19:59776) 2024-04-25T10:07:35.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.683+0000 7fd1e5719640 1 -- 172.21.15.19:0/920760860 learned_addr learned my addr 172.21.15.19:0/920760860 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.683+0000 7fd1e671b640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 0x7fd1e006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.683+0000 7fd1e5f1a640 1 --2- 172.21.15.19:0/920760860 >> v2:172.21.15.112:3300/0 conn(0x7fd1e0144750 0x7fd1e006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.683+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328714196 0 0) 0x7fd1e0144b30 con 0x7fd1e006eb10 2024-04-25T10:07:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.683+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1cc003520 con 0x7fd1e006eb10 2024-04-25T10:07:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 495189495 0 0) 0x7fd1cc003520 con 0x7fd1e006eb10 2024-04-25T10:07:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 msgr2=0x7fd1e006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 0x7fd1e006d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 >> v2:172.21.15.112:3300/0 conn(0x7fd1e0144750 msgr2=0x7fd1e006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 --2- 172.21.15.19:0/920760860 >> v2:172.21.15.112:3300/0 conn(0x7fd1e0144750 0x7fd1e006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e0071890 con 0x7fd1e006eb10 2024-04-25T10:07:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3956686395 0 0) 0x7fd1d0001670 con 0x7fd1e006eb10 2024-04-25T10:07:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 >> v1:172.21.15.19:6789/0 conn(0x7fd1e006eb10 legacy=0x7fd1e00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 0x7fd1cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1cc003f30 0x7fd1cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc004d40 0x7fd1cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00712d0 con 0x7fd1cc003f30 2024-04-25T10:07:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00715b0 con 0x7fd1cc004d40 2024-04-25T10:07:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc00ba10 con 0x7fd1e00cf030 2024-04-25T10:07:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e5f1a640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 0x7fd1cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e5719640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1cc003f30 0x7fd1cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e671b640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc004d40 0x7fd1cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e5f1a640 1 -- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc004d40 msgr2=0x7fd1cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e5f1a640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc004d40 0x7fd1cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e5f1a640 1 -- 172.21.15.19:0/920760860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1cc003f30 msgr2=0x7fd1cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e5f1a640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1cc003f30 0x7fd1cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e5f1a640 1 -- 172.21.15.19:0/920760860 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc003520 con 0x7fd1e00cf030 2024-04-25T10:07:35.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.684+0000 7fd1e5719640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1cc003f30 0x7fd1cc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:35.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.685+0000 7fd1e5f1a640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 0x7fd1cc0039b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009af0 tx=0x7fd1cc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3b4e2e66267fd98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.685+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc036070 con 0x7fd1e00cf030 2024-04-25T10:07:35.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.685+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc031400 con 0x7fd1e00cf030 2024-04-25T10:07:35.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.685+0000 7fd1e4f18640 1 -- 172.21.15.19:0/920760860 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc009690 con 0x7fd1e00cf030 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.686+0000 7fd1e817e640 1 -- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 msgr2=0x7fd1cc0039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.686+0000 7fd1e817e640 1 --2- 172.21.15.19:0/920760860 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 0x7fd1cc0039b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009af0 tx=0x7fd1cc005830 comp rx=0 tx=0).stop 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.686+0000 7fd1e671b640 1 -- 172.21.15.19:0/920760860 reap_dead start 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.686+0000 7fd1e817e640 1 -- 172.21.15.19:0/920760860 shutdown_connections 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.686+0000 7fd1e817e640 1 -- 172.21.15.19:0/920760860 >> 172.21.15.19:0/920760860 conn(0x7fd1e006abe0 msgr2=0x7fd1e006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.686+0000 7fd1e817e640 1 -- 172.21.15.19:0/920760860 shutdown_connections 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.686+0000 7fd1e817e640 1 -- 172.21.15.19:0/920760860 wait complete. 2024-04-25T10:07:35.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.687+0000 7fd1e817e640 1 Processor -- start 2024-04-25T10:07:35.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.687+0000 7fd1e817e640 1 -- start start 2024-04-25T10:07:35.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e817e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 0x7fd1e013a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e817e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e013b860 0x7fd1e00aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e817e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e013bce0 0x7fd1e00aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e817e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e01455d0 con 0x7fd1e013b860 2024-04-25T10:07:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e817e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e0145450 con 0x7fd1e013bce0 2024-04-25T10:07:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e817e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e015c000 con 0x7fd1e00cf030 2024-04-25T10:07:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e5719640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e013b860 0x7fd1e00aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e5719640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e013b860 0x7fd1e00aa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35634/0 (socket says 172.21.15.19:35634) 2024-04-25T10:07:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e5719640 1 -- 172.21.15.19:0/3760767753 learned_addr learned my addr 172.21.15.19:0/3760767753 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e5f1a640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 0x7fd1e013a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e671b640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e013bce0 0x7fd1e00aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e5719640 1 -- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 msgr2=0x7fd1e013a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e5719640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 0x7fd1e013a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e5719640 1 -- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e013bce0 msgr2=0x7fd1e00aacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e5719640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e013bce0 0x7fd1e00aacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e5719640 1 -- 172.21.15.19:0/3760767753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e013a5f0 con 0x7fd1e013b860 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.688+0000 7fd1e671b640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e013bce0 0x7fd1e00aacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.689+0000 7fd1e5719640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e013b860 0x7fd1e00aa7b0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0007fd0 tx=0x7fd1d00077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.689+0000 7fd1d67fc640 1 -- 172.21.15.19:0/3760767753 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000f9c0 con 0x7fd1e013b860 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.689+0000 7fd1d67fc640 1 -- 172.21.15.19:0/3760767753 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d0018070 con 0x7fd1e013b860 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.689+0000 7fd1e817e640 1 -- 172.21.15.19:0/3760767753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e00acb00 con 0x7fd1e013b860 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.689+0000 7fd1d67fc640 1 -- 172.21.15.19:0/3760767753 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000fd60 con 0x7fd1e013b860 2024-04-25T10:07:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.689+0000 7fd1e817e640 1 -- 172.21.15.19:0/3760767753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e00ad0c0 con 0x7fd1e013b860 2024-04-25T10:07:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.690+0000 7fd1e817e640 1 -- 172.21.15.19:0/3760767753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e006f230 con 0x7fd1e013b860 2024-04-25T10:07:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.690+0000 7fd1d67fc640 1 -- 172.21.15.19:0/3760767753 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1d001d460 con 0x7fd1e013b860 2024-04-25T10:07:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.691+0000 7fd1d67fc640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1c406b4d0 0x7fd1c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.691+0000 7fd1d67fc640 1 -- 172.21.15.19:0/3760767753 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd1d008a230 con 0x7fd1e013b860 2024-04-25T10:07:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.691+0000 7fd1e5f1a640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1c406b4d0 0x7fd1c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.692+0000 7fd1e5f1a640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1c406b4d0 0x7fd1c406d990 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc033000 tx=0x7fd1dc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.695+0000 7fd1d67fc640 1 -- 172.21.15.19:0/3760767753 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1e006f230 con 0x7fd1e013b860 2024-04-25T10:07:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.872+0000 7fd1e817e640 1 -- 172.21.15.19:0/3760767753 --> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1e00aa040 con 0x7fd1c406b4d0 2024-04-25T10:07:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.874+0000 7fd1d67fc640 1 -- 172.21.15.19:0/3760767753 <== mgr.4106 v2:172.21.15.112:6832/2701671132 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13386 (secure 0 0 0) 0x7fd1e00aa040 con 0x7fd1c406b4d0 2024-04-25T10:07:35.880 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:35.880 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T10:07:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 -- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1c406b4d0 msgr2=0x7fd1c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1c406b4d0 0x7fd1c406d990 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc033000 tx=0x7fd1dc008030 comp rx=0 tx=0).stop 2024-04-25T10:07:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 -- 172.21.15.19:0/3760767753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e013b860 msgr2=0x7fd1e00aa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e013b860 0x7fd1e00aa7b0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0007fd0 tx=0x7fd1d00077a0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 -- 172.21.15.19:0/3760767753 shutdown_connections 2024-04-25T10:07:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7fd1c406b4d0 0x7fd1c406d990 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e013bce0 0x7fd1e00aacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1e013b860 0x7fd1e00aa7b0 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 --2- 172.21.15.19:0/3760767753 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1e00cf030 0x7fd1e013a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 -- 172.21.15.19:0/3760767753 >> 172.21.15.19:0/3760767753 conn(0x7fd1e006abe0 msgr2=0x7fd1e0140ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 -- 172.21.15.19:0/3760767753 shutdown_connections 2024-04-25T10:07:35.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.875+0000 7fd1e817e640 1 -- 172.21.15.19:0/3760767753 wait complete. 2024-04-25T10:07:35.892 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":629,"stamp":"2024-04-25T10:07:35.116507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":469748,"kb_used_data":20488,"kb_used_omap":12,"kb_used_meta":449139,"kb_avail":754504972,"statfs":{"total":773094113280,"available":772613091328,"internally_reserved":0,"allocated":20979712,"data_stored":10485456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":459918898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002354"},"pg_stats":[{"pgid":"1.0","version":"195'119","reported_seq":538,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:07:15.919157+0000","last_change":"2024-04-25T10:07:15.919217+0000","last_active":"2024-04-25T10:07:15.919157+0000","last_peered":"2024-04-25T10:07:15.919157+0000","last_clean":"2024-04-25T10:07:15.919157+0000","last_became_active":"2024-04-25T09:52:56.061506+0000","last_became_peered":"2024-04-25T09:52:56.061506+0000","last_unstale":"2024-04-25T10:07:15.919157+0000","last_undegraded":"2024-04-25T10:07:15.919157+0000","last_fullsized":"2024-04-25T10:07:15.919157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'119","last_scrub_stamp":"2024-04-25T10:07:15.919049+0000","last_deep_scrub":"195'119","last_deep_scrub_stamp":"2024-04-25T10:07:15.919049+0000","last_clean_scrub_stamp":"2024-04-25T10:07:15.919049+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:04.390951+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575028,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56380,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315460,"statfs":{"total":96636764160,"available":96579031040,"internally_reserved":0,"allocated":3067904,"data_stored":1753458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51652,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320188,"statfs":{"total":96636764160,"available":96583872512,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51648,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320192,"statfs":{"total":96636764160,"available":96583876608,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59964,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82112,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94289728,"statfs":{"total":96636764160,"available":96552681472,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81590727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55808,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316032,"statfs":{"total":96636764160,"available":96579616768,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575026,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55804,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316036,"statfs":{"total":96636764160,"available":96579620864,"internally_reserved":0,"allocated":2473984,"data_stored":1163090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575027,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56380,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315460,"statfs":{"total":96636764160,"available":96579031040,"internally_reserved":0,"allocated":3067904,"data_stored":1753458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:35.894 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:07:35.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.988+0000 7f444fc8f640 1 Processor -- start 2024-04-25T10:07:35.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.989+0000 7f444fc8f640 1 -- start start 2024-04-25T10:07:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.989+0000 7f444fc8f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f444815b7d0 0x7f44480cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.989+0000 7f444fc8f640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f44480b5620 0x7f44480b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.989+0000 7f444fc8f640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444806cf50 con 0x7f444806c2d0 2024-04-25T10:07:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.989+0000 7f444fc8f640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44480ce530 con 0x7f44480b5620 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.989+0000 7f444fc8f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44480ce810 con 0x7f444815b7d0 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.990+0000 7f444d22a640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f44480b5620 0x7f44480b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.990+0000 7f444e22c640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f444806c2d0 0x7f444806c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59786/0 (socket says 172.21.15.19:59786) 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.990+0000 7f444e22c640 1 -- 172.21.15.19:0/2023134309 learned_addr learned my addr 172.21.15.19:0/2023134309 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.990+0000 7f444da2b640 1 --2- 172.21.15.19:0/2023134309 >> v2:172.21.15.112:3300/0 conn(0x7f444815b7d0 0x7f44480cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.990+0000 7f444d22a640 1 -- 172.21.15.19:0/2023134309 >> v2:172.21.15.112:3300/0 conn(0x7f444815b7d0 msgr2=0x7f44480cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.990+0000 7f444d22a640 1 --2- 172.21.15.19:0/2023134309 >> v2:172.21.15.112:3300/0 conn(0x7f444815b7d0 0x7f44480cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.990+0000 7f444d22a640 1 -- 172.21.15.19:0/2023134309 >> v1:172.21.15.19:6789/0 conn(0x7f444806c2d0 legacy=0x7f444806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.990+0000 7f444d22a640 1 -- 172.21.15.19:0/2023134309 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44480ceaf0 con 0x7f44480b5620 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.990+0000 7f444da2b640 1 --2- 172.21.15.19:0/2023134309 >> v2:172.21.15.112:3300/0 conn(0x7f444815b7d0 0x7f44480cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444d22a640 1 --2- 172.21.15.19:0/2023134309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44480b5620 0x7f44480b5ab0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f444808c0d0 tx=0x7f443802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d31ba1382c65ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444ca29640 1 -- 172.21.15.19:0/2023134309 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438037070 con 0x7f44480b5620 2024-04-25T10:07:35.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444ca29640 1 -- 172.21.15.19:0/2023134309 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4438032440 con 0x7f44480b5620 2024-04-25T10:07:35.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444ca29640 1 -- 172.21.15.19:0/2023134309 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438032810 con 0x7f44480b5620 2024-04-25T10:07:36.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444fc8f640 1 -- 172.21.15.19:0/2023134309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44480b5620 msgr2=0x7f44480b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444fc8f640 1 --2- 172.21.15.19:0/2023134309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44480b5620 0x7f44480b5ab0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f444808c0d0 tx=0x7f443802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444fc8f640 1 -- 172.21.15.19:0/2023134309 shutdown_connections 2024-04-25T10:07:36.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444fc8f640 1 --2- 172.21.15.19:0/2023134309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44480b5620 0x7f44480b5ab0 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444fc8f640 1 --2- 172.21.15.19:0/2023134309 >> v2:172.21.15.112:3300/0 conn(0x7f444815b7d0 0x7f44480cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444fc8f640 1 -- 172.21.15.19:0/2023134309 >> 172.21.15.19:0/2023134309 conn(0x7f44480d0800 msgr2=0x7f44480d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444fc8f640 1 -- 172.21.15.19:0/2023134309 shutdown_connections 2024-04-25T10:07:36.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.991+0000 7f444fc8f640 1 -- 172.21.15.19:0/2023134309 wait complete. 2024-04-25T10:07:36.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.992+0000 7f444fc8f640 1 Processor -- start 2024-04-25T10:07:36.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.993+0000 7f444fc8f640 1 -- start start 2024-04-25T10:07:36.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.993+0000 7f444fc8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f444806c2d0 0x7f44481580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.993+0000 7f444fc8f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44480b5620 0x7f44480cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.993+0000 7f444fc8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f444815b7d0 0x7f44480b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.993+0000 7f444fc8f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44480cf6d0 con 0x7f444815b7d0 2024-04-25T10:07:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.993+0000 7f444fc8f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44480cf550 con 0x7f444806c2d0 2024-04-25T10:07:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.993+0000 7f444fc8f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444815bbb0 con 0x7f44480b5620 2024-04-25T10:07:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.993+0000 7f444e22c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f444815b7d0 0x7f44480b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.993+0000 7f444e22c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f444815b7d0 0x7f44480b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35648/0 (socket says 172.21.15.19:35648) 2024-04-25T10:07:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.993+0000 7f444e22c640 1 -- 172.21.15.19:0/3950763668 learned_addr learned my addr 172.21.15.19:0/3950763668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:36.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f444d22a640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44480b5620 0x7f44480cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f444da2b640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f444806c2d0 0x7f44481580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f444e22c640 1 -- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44480b5620 msgr2=0x7f44480cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f444e22c640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44480b5620 0x7f44480cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f444e22c640 1 -- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f444806c2d0 msgr2=0x7f44481580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f444e22c640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f444806c2d0 0x7f44481580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f444e22c640 1 -- 172.21.15.19:0/3950763668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44480cd660 con 0x7f444815b7d0 2024-04-25T10:07:36.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f444d22a640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44480b5620 0x7f44480cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:36.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f444e22c640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f444815b7d0 0x7f44480b8050 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f444000caa0 tx=0x7f444000cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f443e7fc640 1 -- 172.21.15.19:0/3950763668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440017070 con 0x7f444815b7d0 2024-04-25T10:07:36.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f443e7fc640 1 -- 172.21.15.19:0/3950763668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444000ab40 con 0x7f444815b7d0 2024-04-25T10:07:36.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f444fc8f640 1 -- 172.21.15.19:0/3950763668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44480cd940 con 0x7f444815b7d0 2024-04-25T10:07:36.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f443e7fc640 1 -- 172.21.15.19:0/3950763668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440012400 con 0x7f444815b7d0 2024-04-25T10:07:36.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.994+0000 7f444fc8f640 1 -- 172.21.15.19:0/3950763668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44480cdf00 con 0x7f444815b7d0 2024-04-25T10:07:36.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.995+0000 7f444fc8f640 1 -- 172.21.15.19:0/3950763668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4448073a00 con 0x7f444815b7d0 2024-04-25T10:07:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.995+0000 7f443e7fc640 1 -- 172.21.15.19:0/3950763668 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f444001a070 con 0x7f444815b7d0 2024-04-25T10:07:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.996+0000 7f443e7fc640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f442c06b4d0 0x7f442c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.996+0000 7f443e7fc640 1 -- 172.21.15.19:0/3950763668 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4440089d10 con 0x7f444815b7d0 2024-04-25T10:07:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.996+0000 7f444da2b640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f442c06b4d0 0x7f442c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:35.997+0000 7f444da2b640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f442c06b4d0 0x7f442c06d990 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f44440096d0 tx=0x7f4444008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.000+0000 7f443e7fc640 1 -- 172.21.15.19:0/3950763668 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4440058d40 con 0x7f444815b7d0 2024-04-25T10:07:36.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.179+0000 7f444fc8f640 1 -- 172.21.15.19:0/3950763668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f44480b3a00 con 0x7f444815b7d0 2024-04-25T10:07:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.182+0000 7f443e7fc640 1 -- 172.21.15.19:0/3950763668 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v38)=0 v38) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4440058640 con 0x7f444815b7d0 2024-04-25T10:07:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.183+0000 7f443e7fc640 1 -- 172.21.15.19:0/3950763668 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f444005c790 con 0x7f444815b7d0 2024-04-25T10:07:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 -- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f442c06b4d0 msgr2=0x7f442c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f442c06b4d0 0x7f442c06d990 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f44440096d0 tx=0x7f4444008040 comp rx=0 tx=0).stop 2024-04-25T10:07:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 -- 172.21.15.19:0/3950763668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f444815b7d0 msgr2=0x7f44480b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f444815b7d0 0x7f44480b8050 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f444000caa0 tx=0x7f444000cea0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 -- 172.21.15.19:0/3950763668 shutdown_connections 2024-04-25T10:07:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7f442c06b4d0 0x7f442c06d990 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f444815b7d0 0x7f44480b8050 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44480b5620 0x7f44480cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 --2- 172.21.15.19:0/3950763668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f444806c2d0 0x7f44481580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 -- 172.21.15.19:0/3950763668 >> 172.21.15.19:0/3950763668 conn(0x7f44480d0800 msgr2=0x7f44480b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 -- 172.21.15.19:0/3950763668 shutdown_connections 2024-04-25T10:07:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.185+0000 7f444fc8f640 1 -- 172.21.15.19:0/3950763668 wait complete. 2024-04-25T10:07:36.202 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:07:36.202 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:36.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.296+0000 7faaf4fe2640 1 Processor -- start 2024-04-25T10:07:36.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.296+0000 7faaf4fe2640 1 -- start start 2024-04-25T10:07:36.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaf4fe2640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faaf015dbe0 0x7faaf00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaf4fe2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faaf00b2cb0 0x7faaf00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaf4fe2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faaf00b92a0 0x7faaf00b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaf4fe2640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf015dfc0 con 0x7faaf015dbe0 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaf4fe2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf00d1a00 con 0x7faaf00b2cb0 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaf4fe2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf00b97a0 con 0x7faaf00b92a0 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaee59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faaf015dbe0 0x7faaf00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaee59c640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faaf015dbe0 0x7faaf00d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35656/0 (socket says 172.21.15.19:35656) 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaee59c640 1 -- 172.21.15.19:0/2681673289 learned_addr learned my addr 172.21.15.19:0/2681673289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaeed9d640 1 --2- 172.21.15.19:0/2681673289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faaf00b92a0 0x7faaf00b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaedd9b640 1 --2- 172.21.15.19:0/2681673289 >> v2:172.21.15.112:3300/0 conn(0x7faaf00b2cb0 0x7faaf00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaee59c640 1 -- 172.21.15.19:0/2681673289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faaf00b92a0 msgr2=0x7faaf00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaee59c640 1 --2- 172.21.15.19:0/2681673289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faaf00b92a0 0x7faaf00b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaee59c640 1 -- 172.21.15.19:0/2681673289 >> v2:172.21.15.112:3300/0 conn(0x7faaf00b2cb0 msgr2=0x7faaf00d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaee59c640 1 --2- 172.21.15.19:0/2681673289 >> v2:172.21.15.112:3300/0 conn(0x7faaf00b2cb0 0x7faaf00d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.297+0000 7faaee59c640 1 -- 172.21.15.19:0/2681673289 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf00d1c50 con 0x7faaf015dbe0 2024-04-25T10:07:36.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.298+0000 7faaee59c640 1 --2- 172.21.15.19:0/2681673289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf015dbe0 0x7faaf00d0910 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7faaf006cff0 tx=0x7faae002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c84ea88a27200a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.298+0000 7faaed59a640 1 -- 172.21.15.19:0/2681673289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0037070 con 0x7faaf015dbe0 2024-04-25T10:07:36.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.298+0000 7faaed59a640 1 -- 172.21.15.19:0/2681673289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7faae0032440 con 0x7faaf015dbe0 2024-04-25T10:07:36.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.298+0000 7faaed59a640 1 -- 172.21.15.19:0/2681673289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0032830 con 0x7faaf015dbe0 2024-04-25T10:07:36.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.299+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2681673289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf015dbe0 msgr2=0x7faaf00d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.299+0000 7faaf4fe2640 1 --2- 172.21.15.19:0/2681673289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf015dbe0 0x7faaf00d0910 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7faaf006cff0 tx=0x7faae002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.299+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2681673289 shutdown_connections 2024-04-25T10:07:36.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.299+0000 7faaf4fe2640 1 --2- 172.21.15.19:0/2681673289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faaf00b92a0 0x7faaf00b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.299+0000 7faaf4fe2640 1 --2- 172.21.15.19:0/2681673289 >> v2:172.21.15.112:3300/0 conn(0x7faaf00b2cb0 0x7faaf00d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.299+0000 7faaf4fe2640 1 --2- 172.21.15.19:0/2681673289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf015dbe0 0x7faaf00d0910 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.299+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2681673289 >> 172.21.15.19:0/2681673289 conn(0x7faaf0061a20 msgr2=0x7faaf0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.299+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2681673289 shutdown_connections 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.299+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2681673289 wait complete. 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.300+0000 7faaf4fe2640 1 Processor -- start 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.300+0000 7faaf4fe2640 1 -- start start 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaf4fe2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf00b2cb0 0x7faaf00b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaf4fe2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf00b92a0 0x7faaf01f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaf4fe2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faaf015dbe0 0x7faaf015f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaf4fe2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf00b6ab0 con 0x7faaf00b2cb0 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaf4fe2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf00d2210 con 0x7faaf00b92a0 2024-04-25T10:07:36.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaf4fe2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf00b6c30 con 0x7faaf015dbe0 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf00b2cb0 0x7faaf00b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf00b2cb0 0x7faaf00b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35662/0 (socket says 172.21.15.19:35662) 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaee59c640 1 -- 172.21.15.19:0/2148785535 learned_addr learned my addr 172.21.15.19:0/2148785535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaeed9d640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faaf015dbe0 0x7faaf015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaedd9b640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf00b92a0 0x7faaf01f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaee59c640 1 -- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faaf015dbe0 msgr2=0x7faaf015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaee59c640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faaf015dbe0 0x7faaf015f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaee59c640 1 -- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf00b92a0 msgr2=0x7faaf01f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaee59c640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf00b92a0 0x7faaf01f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaee59c640 1 -- 172.21.15.19:0/2148785535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf00b5c00 con 0x7faaf00b2cb0 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaedd9b640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf00b92a0 0x7faaf01f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaeed9d640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faaf015dbe0 0x7faaf015f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaee59c640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf00b2cb0 0x7faaf00b4c40 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7faae002a220 tx=0x7faae00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faadeffd640 1 -- 172.21.15.19:0/2148785535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0037040 con 0x7faaf00b2cb0 2024-04-25T10:07:36.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2148785535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf00b5ee0 con 0x7faaf00b2cb0 2024-04-25T10:07:36.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.301+0000 7faadeffd640 1 -- 172.21.15.19:0/2148785535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7faae0042070 con 0x7faaf00b2cb0 2024-04-25T10:07:36.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.302+0000 7faadeffd640 1 -- 172.21.15.19:0/2148785535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0030bf0 con 0x7faaf00b2cb0 2024-04-25T10:07:36.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.302+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2148785535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf01f6d00 con 0x7faaf00b2cb0 2024-04-25T10:07:36.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.302+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2148785535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf00b62d0 con 0x7faaf00b2cb0 2024-04-25T10:07:36.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.303+0000 7faadeffd640 1 -- 172.21.15.19:0/2148785535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faae0047460 con 0x7faaf00b2cb0 2024-04-25T10:07:36.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.303+0000 7faadeffd640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faad006b4d0 0x7faad006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.303+0000 7faadeffd640 1 -- 172.21.15.19:0/2148785535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5421+0+0 (secure 0 0 0) 0x7faae00aad60 con 0x7faaf00b2cb0 2024-04-25T10:07:36.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.304+0000 7faaedd9b640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faad006b4d0 0x7faad006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.304+0000 7faaedd9b640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faad006b4d0 0x7faad006d990 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7faae4006fd0 tx=0x7faae4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.307+0000 7faadeffd640 1 -- 172.21.15.19:0/2148785535 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faae0032e20 con 0x7faaf00b2cb0 2024-04-25T10:07:36.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.526+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2148785535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faaf00b60b0 con 0x7faaf00b2cb0 2024-04-25T10:07:36.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.527+0000 7faadeffd640 1 -- 172.21.15.19:0/2148785535 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+3708 (secure 0 0 0) 0x7faae0045360 con 0x7faaf00b2cb0 2024-04-25T10:07:36.534 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T10:07:36.534 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":131,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1116601174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1116601174},{"type":"v1","addr":"172.21.15.112:6835","nonce":1116601174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1096367993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1096367993},{"type":"v1","addr":"172.21.15.19:6834","nonce":1096367993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":9874,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/801869494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":801869494},{"type":"v1","addr":"172.21.15.112:6837","nonce":801869494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":12075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3384048790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3384048790},{"type":"v1","addr":"172.21.15.19:6835","nonce":3384048790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":131}],"filesystems":[]} 2024-04-25T10:07:36.534 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 131 2024-04-25T10:07:36.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.528+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faad006b4d0 msgr2=0x7faad006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.528+0000 7faaf4fe2640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faad006b4d0 0x7faad006d990 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7faae4006fd0 tx=0x7faae4008040 comp rx=0 tx=0).stop 2024-04-25T10:07:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.528+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2148785535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf00b2cb0 msgr2=0x7faaf00b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.529+0000 7faaf4fe2640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf00b2cb0 0x7faaf00b4c40 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7faae002a220 tx=0x7faae00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.529+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2148785535 shutdown_connections 2024-04-25T10:07:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.529+0000 7faaf4fe2640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:6832/2701671132,v1:172.21.15.112:6833/2701671132] conn(0x7faad006b4d0 0x7faad006d990 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.529+0000 7faaf4fe2640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faaf015dbe0 0x7faaf015f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.529+0000 7faaf4fe2640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf00b92a0 0x7faaf01f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.529+0000 7faaf4fe2640 1 --2- 172.21.15.19:0/2148785535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf00b2cb0 0x7faaf00b4c40 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.529+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2148785535 >> 172.21.15.19:0/2148785535 conn(0x7faaf0061a20 msgr2=0x7faaf00d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.529+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2148785535 shutdown_connections 2024-04-25T10:07:36.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T10:07:36.529+0000 7faaf4fe2640 1 -- 172.21.15.19:0/2148785535 wait complete. 2024-04-25T10:07:36.546 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:07:36.546 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:07:36.546 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:07:36.547 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:36.555 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:07:36.555 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:07:36.555 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:36.605 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:07:36.605 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:07:36.605 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:36.617 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:07:36.617 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:07:36.617 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:36.668 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:07:36.669 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:07:36.669 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:07:36.669 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:36.705 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:07:36.705 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:07:36.705 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:36.774 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:07:36.775 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:07:36.775 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:36.844 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:07:36.844 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:07:36.844 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:36.911 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:07:36.912 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:07:36.912 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:36.948 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:07:36.948 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:07:36.948 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:37.016 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:07:37.016 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:07:37.016 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:37.088 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:07:37.088 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:07:37.089 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:37.171 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:07:37.171 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:07:37.172 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:07:37.172 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:37.197 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:07:37.197 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:07:37.197 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:37.226 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:07:37.226 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:07:37.227 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:07:37.227 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:37.248 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:07:37.248 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:07:37.248 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:37.279 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:07:37.280 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:07:37.280 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:37.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-25T10:07:37.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-25T10:07:37.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-25T10:07:37.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-04-25T10:07:37.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-04-25T10:07:37.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-25T10:07:37.333 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:07:37.333 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:07:37.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T10:07:37.363 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:07:37.364 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:07:37.681 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:07:37.681 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:07:37.751 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:07:37.752 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:07:37.808 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:07:37.809 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:07:37.860 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T10:07:37.860 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:07:38.033 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T10:07:38.033 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:07:38.124 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T10:07:38.125 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:07:38.190 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T10:07:38.190 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:07:38.256 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:07:38.257 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033/data/mon.a.tgz 2024-04-25T10:07:38.258 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-04-25T10:07:38.274 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.waWeSil7Me 2024-04-25T10:07:38.274 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.waWeSil7Me 2024-04-25T10:07:38.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.waWeSil7Me 2024-04-25T10:07:38.533 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.waWeSil7Me is 534KB 2024-04-25T10:07:38.632 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.waWeSil7Me 2024-04-25T10:07:38.647 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033/data/mon.b.tgz 2024-04-25T10:07:38.647 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-04-25T10:07:38.665 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.T0dEoAZBWa 2024-04-25T10:07:38.666 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.T0dEoAZBWa 2024-04-25T10:07:38.854 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.T0dEoAZBWa 2024-04-25T10:07:38.937 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.T0dEoAZBWa is 548KB 2024-04-25T10:07:39.036 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.T0dEoAZBWa 2024-04-25T10:07:39.050 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033/data/mon.c.tgz 2024-04-25T10:07:39.050 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-04-25T10:07:39.104 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.noUE1l6ur4 2024-04-25T10:07:39.104 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.noUE1l6ur4 2024-04-25T10:07:39.271 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.noUE1l6ur4 2024-04-25T10:07:39.352 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.noUE1l6ur4 is 548KB 2024-04-25T10:07:39.452 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.noUE1l6ur4 2024-04-25T10:07:39.466 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:07:39.466 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:07:39.469 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:07:39.555 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:07:39.583 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:07:39.584 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:07:39.585 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033/remote/smithi019/crash 2024-04-25T10:07:39.586 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:07:39.623 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033/remote/smithi112/crash 2024-04-25T10:07:39.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:07:39.637 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:39.647 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:07:39.668 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:39.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:39.771 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:39.795 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:07:39.795 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:07:39.814 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:07:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:07:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34921.log 2024-04-25T10:07:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34801.log 2024-04-25T10:07:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34710.log 2024-04-25T10:07:39.878 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34670.log 2024-04-25T10:07:39.878 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34921.log.gz 2024-04-25T10:07:39.878 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34801.log.gz 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34710.log: /var/log/ceph/ceph-client.admin.34670.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34710.log.gz 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.audit.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34670.log.gz 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34881.log 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34881.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.4.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34881.log.gz 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:07:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34630.log 2024-04-25T10:07:39.880 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.6.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:07:39.880 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41429.log 2024-04-25T10:07:39.880 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34630.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34630.log.gz 2024-04-25T10:07:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34841.log 2024-04-25T10:07:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.1.41429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41429.log.gz 2024-04-25T10:07:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34761.log 2024-04-25T10:07:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34841.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34841.log.gz 2024-04-25T10:07:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:07:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34761.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34761.log.gz 2024-04-25T10:07:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:07:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41819.log 2024-04-25T10:07:39.923 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.3.41819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41819.log.gz 2024-04-25T10:07:40.065 INFO:teuthology.orchestra.run.smithi112.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:07:40.066 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84721.log 2024-04-25T10:07:40.066 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42390.log 2024-04-25T10:07:40.066 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110156.log 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84902.log 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84721.log.gz 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42390.log.gz 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61219.log 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84902.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110156.log.gz 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45905.log 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.84902.log.gz 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113696.log 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95438.log 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.45905.log.gz 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113696.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.90570.log 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61219.log.gz 2024-04-25T10:07:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95438.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113696.log.gz 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102616.log 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90570.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95438.log.gz 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83426.log 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90570.log.gz 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103504.log 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102616.log.gz 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90402.log 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83426.log.gz 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57255.log 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103504.log: /var/log/ceph/ceph-client.admin.90402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88469.log 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103504.log.gz 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90402.log.gz 2024-04-25T10:07:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79828.log 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57255.log.gz 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113444.log 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85635.log 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88469.log.gz 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79828.log.gz 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47612.log 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113444.log.gz 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112419.log 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91808.log 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85635.log.gz 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112419.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.47612.log.gz 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.112774.log 2024-04-25T10:07:40.069 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112419.log.gz 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110474.log 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91808.log.gz 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86015.log 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101826.log 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112774.log.gz 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.110474.log.gz 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78983.log 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.101826.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86015.log.gz 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99631.log 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83324.log 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78983.log.gz 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99631.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.101826.log.gz 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48500.log 2024-04-25T10:07:40.070 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99631.log.gz 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62509.log 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48080.log 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83324.log.gz 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62509.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48500.log.gz 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99976.log 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62509.log.gz 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57568.log 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99976.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48080.log.gz 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91304.log 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99976.log.gz 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105863.log 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57568.log.gz 2024-04-25T10:07:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.91304.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.66998.log 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58510.log 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91304.log.gz 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66998.log: gzip -5 --verbose 87.7% -- -- replaced with /var/log/ceph/ceph-client.admin.105863.log.gz /var/log/ceph/ceph-client.admin.112587.log 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58510.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66998.log.gz 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89168.log 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112587.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58510.log.gz 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64550.log 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89168.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112587.log.gz 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89111.log 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89168.log.gz 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84816.log 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79203.log 2024-04-25T10:07:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89111.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64550.log.gz 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67656.log 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89111.log.gz 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84816.log.gz 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112858.log 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67656.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79203.log.gz 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96040.log 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67656.log.gz 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80294.log 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112858.log.gz 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.96040.log: -- /var/log/ceph/ceph-client.admin.75861.log 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85973.log 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96040.log.gz 2024-04-25T10:07:40.073 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75861.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.80294.log.gz 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.90131.log 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75861.log.gz 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109236.log 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90131.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85973.log.gz 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111808.log 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109236.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90131.log.gz 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101285.log 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109236.log.gz 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61724.log 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57381.log 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.101285.log: /var/log/ceph/ceph-client.admin.61724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76075.log 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101285.log.gz 2024-04-25T10:07:40.074 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.111808.log.gz 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61724.log.gz 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75142.log 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42988.log 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57381.log.gz 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75142.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76075.log.gz 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89753.log 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75142.log.gz 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106470.log 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110114.log 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42988.log.gz 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106470.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.80561.log -- replaced with /var/log/ceph/ceph-client.admin.89753.log.gz 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106470.log.gz 2024-04-25T10:07:40.075 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.104816.log 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80561.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110114.log.gz 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49016.log 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80561.log.gz 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.104816.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.0.104816.log.gz --verbose 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.85889.log 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40467.log 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102700.log 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49016.log.gz 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85889.log.gz 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105559.log 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40467.log.gz 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85584.log 2024-04-25T10:07:40.076 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105559.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102700.log.gz 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65578.log 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.105559.log.gz 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88217.log 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85584.log.gz 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45074.log 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44344.log 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65578.log.gz 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.45074.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.88217.log.gz --verbose 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.76524.log 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45074.log.gz 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66050.log 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43322.log 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44344.log.gz 2024-04-25T10:07:40.077 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76524.log.gz 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57515.log 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66050.log.gz 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103139.log 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61944.log 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43322.log.gz 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103139.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57515.log.gz 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89473.log 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61944.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103139.log.gz 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57087.log 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89473.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61944.log.gz 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75562.log 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89473.log.gz 2024-04-25T10:07:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45523.log 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57087.log.gz 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47150.log 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.2.45523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79455.log 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75562.log.gz 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45523.log.gz 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103223.log 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47150.log.gz 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110324.log 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103223.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79455.log.gz 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45821.log 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110324.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103223.log.gz 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79067.log 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96650.log 2024-04-25T10:07:40.079 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110324.log.gz 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105601.log 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45821.log.gz 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79067.log.gz 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90276.log 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105601.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96650.log.gz 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41393.log 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105601.log.gz 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83636.log 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90276.log.gz 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43543.log 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89252.log 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41393.log.gz 2024-04-25T10:07:40.080 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63102.log 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43543.log.gz 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47438.log 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.83636.log.gz 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63102.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89252.log.gz 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108671.log 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47438.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63102.log.gz 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79744.log 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47438.log.gz 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59687.log 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108671.log: /var/log/ceph/ceph-client.admin.79744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90486.log 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108671.log.gz 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5/var/log/ceph/ceph-client.admin.59687.log: --verbose -- /var/log/ceph/ceph-client.admin.109661.log 2024-04-25T10:07:40.081 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79744.log.gz 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61525.log 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8%/var/log/ceph/ceph-client.admin.109661.log: -- replaced with /var/log/ceph/ceph-client.admin.59687.log.gz 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.90486.log.gz -5 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.103922.log 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61525.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.46325.log 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109661.log.gz 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103922.log: 89.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.61525.log.gz 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43655.log 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103922.log.gz 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64707.log 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67843.log 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46325.log.gz 2024-04-25T10:07:40.082 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43655.log.gz 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43924.log 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.67843.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64707.log.gz 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89294.log 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67843.log.gz 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65662.log 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41435.log 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43924.log.gz 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89294.log.gz -5 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.66447.log 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65662.log: /var/log/ceph/ceph-client.admin.41435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94567.log 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65662.log.gz 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66447.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41435.log.gz 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59625.log 2024-04-25T10:07:40.083 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66447.log.gz 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83552.log 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105517.log 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94567.log.gz 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44428.log 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59625.log.gz 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83552.log.gz 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84335.log 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79287.log 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105517.log.gz 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44428.log.gz 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58202.log 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.84335.log.gz 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106554.log 2024-04-25T10:07:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58202.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79287.log.gz 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113654.log 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106554.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58202.log.gz 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112942.log 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113654.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106554.log.gz 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94124.log 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113654.log.gz 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112942.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112942.log.gz 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91556.log 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94124.log.gz 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44596.log 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75184.log 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44596.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91556.log.gz 2024-04-25T10:07:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74764.log 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49829.log 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74764.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75184.log.gz 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44596.log.gz 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67548.log 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74764.log.gz 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45863.log 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67548.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49829.log.gz 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107516.log 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44386.log 88.2% 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67548.log.gz 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45863.log.gz 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113990.log 2024-04-25T10:07:40.086 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107516.log.gz 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79245.log 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103391.log 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44386.log.gz 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113990.log.gz 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89963.log 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103391.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.79245.log.gz 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.90047.log 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89963.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103391.log.gz 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90360.log 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76482.log 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89963.log.gz 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90047.log.gz 2024-04-25T10:07:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78630.log 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90360.log.gz 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45653.log 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78630.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76482.log.gz 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106596.log 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49268.log 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78630.log.gz 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100071.log 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45653.log.gz 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106596.log.gz 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60793.log 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100071.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49268.log.gz 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57610.log 2024-04-25T10:07:40.088 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100071.log.gz 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107083.log 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62112.log 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107083.log: gzip -5 --verbose -- 89.4% /var/log/ceph/ceph-client.admin.74101.log -- replaced with /var/log/ceph/ceph-client.admin.57610.log.gz 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60793.log.gz 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57465.log 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107083.log.gz 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74101.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62112.log.gz 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59865.log 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48374.log 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.74101.log.gz 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59497.log 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57465.log.gz 2024-04-25T10:07:40.089 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59865.log.gz 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98820.log 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59497.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48374.log.gz 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58118.log 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.98820.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59497.log.gz 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85719.log 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98820.log.gz 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113486.log 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58118.log: /var/log/ceph/ceph-client.admin.85719.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58118.log.gz 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0%/var/log/ceph/ceph-client.admin.113486.log: -- replaced with /var/log/ceph/ceph-client.admin.85719.log.gz 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96124.log 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.2.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113486.log.gz 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88553.log 2024-04-25T10:07:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89378.log 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88553.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96124.log.gz 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95344.log 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88553.log.gz 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100810.log 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46241.log 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89378.log.gz 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100810.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95344.log.gz 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40907.log 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100810.log.gz 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111506.log 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46241.log.gz 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111368.log 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111506.log: gzip -5 --verbose 88.2% -- /var/log/ceph/ceph-client.admin.57736.log -- replaced with /var/log/ceph/ceph-client.admin.40907.log.gz 2024-04-25T10:07:40.091 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111368.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111506.log.gz 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90827.log 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111368.log.gz 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61097.log 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108713.log 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57736.log.gz 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61097.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90827.log.gz 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109946.log 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.61097.log.gz 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75737.log 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109946.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.108713.log.gz /var/log/ceph/ceph-client.admin.105266.log 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75737.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109946.log.gz 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93890.log 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75737.log.gz 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74544.log 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.93890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59770.log 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74544.log: 90.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.105266.log.gz 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65382.log 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74544.log.gz 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90953.log 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59770.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.93890.log.gz 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49703.log 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59770.log.gz 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79912.log 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90953.log.gz 2024-04-25T10:07:40.093 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.65382.log.gz 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88301.log 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79581.log 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79912.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49703.log.gz 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57297.log 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79912.log.gz 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88301.log.gz 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107558.log 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57297.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79581.log.gz 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101369.log 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57297.log.gz/var/log/ceph/ceph-client.admin.107558.log: 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79371.log 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107558.log.gz 2024-04-25T10:07:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.101369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43157.log 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79371.log: gzip 88.5% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.101369.log.gz 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.109068.log 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79371.log.gz 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60437.log 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43157.log.gz 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112545.log 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110240.log 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109068.log.gz 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60437.log.gz 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83894.log 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112545.log.gz 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46619.log 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105779.log 2024-04-25T10:07:40.095 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110240.log.gz 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46619.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83894.log.gz 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101892.log 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105779.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46619.log.gz 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67254.log 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105779.log.gz 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.101892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60329.log 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67254.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101892.log.gz 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101748.log 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76272.log 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67254.log.gz 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60329.log.gz 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.101748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2024-04-25T10:07:40.096 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101748.log.gz 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.76272.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.62635.log 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46703.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76272.log.gz 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61808.log 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107220.log 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62635.log.gz 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56933.log 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107220.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.103181.log -- replaced with /var/log/ceph/ceph-client.admin.61808.log.gz 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56933.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107220.log.gz 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56933.log.gz 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67040.log 2024-04-25T10:07:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109902.log 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103181.log.gz 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59991.log 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67040.log.gz 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110198.log 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.109902.log.gz 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47270.log 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59991.log: /var/log/ceph/ceph-client.admin.110198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109988.log 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59991.log.gz 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47270.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110198.log.gz 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96776.log 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47270.log.gz 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99368.log 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109988.log.gz 2024-04-25T10:07:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104437.log 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96776.log: /var/log/ceph/ceph-client.admin.99368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84545.log 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104437.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99368.log.gz 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84031.log 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.96776.log.gz 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100239.log 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5%/var/log/ceph/ceph-client.admin.84545.log: -- replaced with /var/log/ceph/ceph-client.admin.104437.log.gz 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84031.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84545.log.gz 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58037.log 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84031.log.gz 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64593.log 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100239.log: /var/log/ceph/ceph-client.admin.58037.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113276.log 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100239.log.gz 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61766.log 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58037.log.gz 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63849.log 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.64593.log.gz 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113276.log.gz 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60113.log 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61766.log.gz 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98993.log 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60113.log: gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.63849.log.gz -- 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.95050.log 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.98993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112503.log 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95050.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.60113.log.gz 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57694.log 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run.smithi019.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.98993.log.gz 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112503.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95050.log.gz 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63017.log 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112503.log.gz 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80421.log 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91682.log 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57694.log.gz 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80421.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.60533.log 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63017.log.gz 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80421.log.gz 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89879.log 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91682.log.gz 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42574.log 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60533.log.gz 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105905.log 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57652.log 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89879.log.gz 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42574.log.gz 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76314.log 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104521.log 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105905.log.gz 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76314.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57652.log.gz 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43327.log 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76314.log.gz 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102868.log 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102312.log 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104521.log.gz 2024-04-25T10:07:40.102 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102868.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43327.log.gz 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110282.log 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102868.log.gz 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48290.log 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110282.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102312.log.gz -5 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58806.log 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110282.log.gz 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110072.log 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48290.log: /var/log/ceph/ceph-client.admin.58806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44008.log 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48290.log.gz 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74167.log 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58806.log.gz 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110072.log.gz 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34639.log 2024-04-25T10:07:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74167.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44008.log.gz 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49871.log 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.34639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96842.log 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34639.log.gz -- replaced with /var/log/ceph/ceph-client.admin.74167.log.gz 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102000.log 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59101.log 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2%/var/log/ceph/ceph-client.admin.102000.log: -- replaced with /var/log/ceph/ceph-client.admin.49871.log.gz 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96842.log.gz 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46661.log 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.59101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79631.log 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59101.log.gz 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.102000.log.gz 2024-04-25T10:07:40.104 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108797.log 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79631.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46661.log.gz 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62975.log 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79631.log.gz 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103265.log 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62975.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108797.log.gzgzip 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.112671.log 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88703.log 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62975.log.gz 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112671.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103265.log.gz 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85677.log 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112671.log.gz 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67212.log 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94746.log 2024-04-25T10:07:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67212.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85677.log.gz 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.88703.log.gz 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67921.log 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67212.log.gz 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90995.log 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94746.log.gz 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40561.log 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90995.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.67921.log.gz -5 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.113360.log 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40561.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90995.log.gz 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105017.log 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40561.log.gz 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94082.log 2024-04-25T10:07:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41225.log 87.5% 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113360.log.gz 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105017.log.gz 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49449.log 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94082.log.gz 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98928.log 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49449.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41225.log.gz 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91346.log 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49449.log.gz 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.98928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103749.log 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100682.log 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103749.log: 90.9% 89.6% -- replaced with /var/log/ceph/ceph-client.admin.98928.log.gz -- replaced with /var/log/ceph/ceph-client.admin.91346.log.gz 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79954.log 2024-04-25T10:07:40.107 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103749.log.gz 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96166.log 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100682.log.gz 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67590.log 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105349.log 87.5% 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79954.log.gz 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67590.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96166.log.gz 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112900.log 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46493.log 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr: 91.4%/var/log/ceph/ceph-client.admin.112900.log: -- replaced with /var/log/ceph/ceph-client.admin.67590.log.gz 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105349.log.gz 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41267.log 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112900.log.gz 2024-04-25T10:07:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80510.log 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46493.log.gz 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41267.log: /var/log/ceph/ceph-client.admin.80510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84629.log 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41267.log.gz 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80510.log.gz 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43340.log 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80603.log 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43340.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84629.log.gz 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60047.log 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43340.log.gz 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94830.log 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80603.log: /var/log/ceph/ceph-client.admin.60047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59241.log 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80603.log.gz 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94354.log 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60047.log.gz 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59241.log: gzip -5 88.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.94830.log.gz /var/log/ceph/ceph-client.admin.75987.log 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43656.log 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59241.log.gz 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94354.log: /var/log/ceph/ceph-client.admin.75987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99883.log 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94354.log.gz 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43656.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75987.log.gz 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49100.log 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43656.log.gz 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111592.log 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99883.log: /var/log/ceph/ceph-client.admin.49100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88151.log 2024-04-25T10:07:40.110 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99883.log.gz 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111592.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49100.log.gz 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49354.log 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95218.log 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.111592.log.gz 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110661.log 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41351.log 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95218.log: 89.0% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.88151.log.gz -- replaced with /var/log/ceph/ceph-client.admin.49354.log.gz 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.110661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91039.log 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.110661.log.gz 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95218.log.gz 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41139.log 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41351.log: /var/log/ceph/ceph-client.admin.91039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42532.log 2024-04-25T10:07:40.111 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41351.log.gz 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111053.log 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91039.log.gz 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41139.log.gz 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58848.log 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42532.log.gz 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85028.log 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111053.log: /var/log/ceph/ceph-client.admin.58848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61459.log 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111053.log.gz 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85028.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85028.log.gz 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58848.log.gz 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89069.log 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113132.log 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46073.log 2024-04-25T10:07:40.112 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89069.log.gz 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61459.log.gz 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89027.log 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106771.log 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113132.log.gz 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89027.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46073.log.gz 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106157.log 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89027.log.gz 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102742.log 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106771.log.gz 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102438.log 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106157.log: /var/log/ceph/ceph-client.admin.102742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103973.log 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106157.log.gz 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102438.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102742.log.gz 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109860.log 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102438.log.gz 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106954.log 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103973.log: /var/log/ceph/ceph-client.admin.109860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66573.log 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103973.log.gz 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109860.log.gz 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108275.log 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75100.log 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66573.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106954.log.gz 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104269.log 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66573.log.gz 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108275.log.gz 2024-04-25T10:07:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112132.log 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42862.log 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75100.log.gz 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112132.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104269.log.gz 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113822.log 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102480.log 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.112132.log.gz 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106302.log 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42862.log.gz 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46745.log 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113822.log.gz 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102480.log: /var/log/ceph/ceph-client.admin.106302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109110.log 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102480.log.gz 2024-04-25T10:07:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46745.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106302.log.gz 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67338.log 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46745.log.gz 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95878.log 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109110.log: /var/log/ceph/ceph-client.admin.67338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61902.log 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109110.log.gz 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67338.log.gz 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95878.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103878.log 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61902.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95878.log.gz 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101327.log 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61902.log.gz 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83384.log 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103878.log: /var/log/ceph/ceph-client.admin.101327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94704.log 2024-04-25T10:07:40.116 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103878.log.gz 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7%/var/log/ceph/ceph-client.admin.83384.log: -- replaced with /var/log/ceph/ceph-client.admin.101327.log.gz 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85070.log 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94704.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95302.log 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83384.log.gz 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.94704.log.gz 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42616.log 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95302.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85070.log.gz 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58375.log 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42616.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95302.log.gz 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95260.log 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58375.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42616.log.gz 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42778.log 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58375.log.gz 2024-04-25T10:07:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75478.log 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88595.log 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95260.log.gz 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75478.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.42778.log.gz 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.113192.log 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88769.log 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88595.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75478.log.gz 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90911.log 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88595.log.gz 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85540.log 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113192.log.gz 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78564.log 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88769.log.gz 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95741.log 2024-04-25T10:07:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85540.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.90911.log.gz 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80983.log 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85540.log.gz 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46577.log 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48668.log 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95741.log.gz 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78564.log.gz 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84944.log 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48668.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46577.log.gz 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65034.log 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.80983.log.gz 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84944.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48668.log.gz 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104143.log 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84944.log.gz 2024-04-25T10:07:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100029.log 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111257.log 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104143.log.gz 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1%/var/log/ceph/ceph-client.admin.100029.log: -- replaced with /var/log/ceph/ceph-client.admin.65034.log.gz 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47859.log 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100029.log.gz 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67380.log 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78941.log 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.111257.log.gz 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67380.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.43326.log 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47859.log.gz 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44050.log 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78941.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67380.log.gz 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79497.log 2024-04-25T10:07:40.120 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78941.log.gz 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90173.log 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43326.log.gz 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44050.log.gz 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84073.log 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90173.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79497.log.gz 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95134.log 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90173.log.gz 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84073.log.gz 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57339.log 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95134.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95134.log.gz 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48038.log 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mgr.y.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57339.log.gz 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99410.log 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62551.log 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99410.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48038.log.gz 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50202.log 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99410.log.gz 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62551.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62551.log.gz 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48248.log 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84503.log 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48248.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50202.log.gz 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65100.log 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48248.log.gz 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94788.log 2024-04-25T10:07:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84503.log: /var/log/ceph/ceph-client.admin.65100.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.84503.log.gz 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65100.log.gz 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60197.log 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85165.log 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103349.log 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94788.log.gz 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40949.log 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60197.log.gz 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.85165.log.gz 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88259.log 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7%/var/log/ceph/ceph-client.admin.40949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76144.log 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103349.log.gz 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40949.log.gz 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88259.log: gzip -5 --verbose -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88259.log.gz 2024-04-25T10:07:40.123 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.106512.log 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85245.log 87.7% 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76144.log.gz 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90089.log 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106512.log.gz 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104479.log 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85245.log: /var/log/ceph/ceph-client.admin.90089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106870.log 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104479.log: 87.8% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90089.log.gz 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.85245.log.gz 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58331.log 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104479.log.gz 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43966.log 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106870.log.gz -5 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.79539.log 2024-04-25T10:07:40.124 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58331.log: /var/log/ceph/ceph-client.admin.43966.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58331.log.gz 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43966.log.gz 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79413.log 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58764.log 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48790.log 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79539.log.gz 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.58764.log: -5 --verbose 89.2% -- /var/log/ceph/ceph-client.admin.60876.log 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79413.log.gz 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48790.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107178.log 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58764.log.gz 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48790.log.gz 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60876.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106691.log 2024-04-25T10:07:40.125 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60876.log.gz 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107178.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47528.log 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94479.log 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107178.log.gz 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47528.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106691.log.gz 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47528.log.gz 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.75394.log 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107306.log 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94479.log.gz 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93758.log 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75394.log.gz 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96566.log 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107306.log: /var/log/ceph/ceph-client.admin.93758.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107306.log.gz 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85847.log 2024-04-25T10:07:40.126 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.93758.log.gz 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88985.log 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96566.log.gz 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95531.log 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85847.log.gz 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101243.log 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.88985.log.gz 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84671.log 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95531.log.gz 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.101243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58722.log 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101243.log.gz 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90665.log 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84671.log.gz 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64948.log 2024-04-25T10:07:40.127 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58722.log.gz 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96252.log 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90665.log.gz 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48972.log 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64948.log.gz 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96208.log 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96252.log.gz 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46031.log 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48972.log.gz/var/log/ceph/ceph-client.admin.96208.log: 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58160.log 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96208.log.gz 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110408.log 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46031.log: /var/log/ceph/ceph-client.admin.58160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48626.log 2024-04-25T10:07:40.128 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110408.log: 87.8% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58160.log.gz 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46031.log.gz 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113948.log 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48626.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.110408.log.gz 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90225.log 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48626.log.gz 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103707.log 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113948.log: /var/log/ceph/ceph-client.admin.90225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108461.log 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103707.log: 88.2% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90225.log.gz 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113948.log.gz 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103707.log.gz 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83852.log 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.108461.log: -- /var/log/ceph/ceph-client.admin.112377.log 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76356.log 2024-04-25T10:07:40.129 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108461.log.gz 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83852.log.gz 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91514.log 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112377.log: /var/log/ceph/ceph-client.admin.76356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111095.log 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91514.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112377.log.gz 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76356.log.gz 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91514.log.gz 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108545.log 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61682.log 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111095.log.gz 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95657.log 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0%/var/log/ceph/ceph-client.admin.61682.log: -- replaced with /var/log/ceph/ceph-client.admin.108545.log.gz 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42180.log 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61682.log.gz 2024-04-25T10:07:40.130 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100155.log 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95657.log: /var/log/ceph/ceph-client.admin.42180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64431.log 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100155.log: 88.1% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42180.log.gz 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95657.log.gz 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100155.log.gz 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.92073.log 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111940.log 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64431.log.gz 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.92073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98725.log 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.92073.log.gz 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107348.log 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111940.log: /var/log/ceph/ceph-client.admin.98725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66278.log 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107348.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98725.log.gz 2024-04-25T10:07:40.131 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107348.log.gz 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr: 91.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103665.log 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.111940.log.gz/var/log/ceph/ceph-client.admin.66278.log: 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114114.log 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66278.log.gz/var/log/ceph/ceph-client.admin.103665.log: 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105392.log 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103665.log.gz 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63807.log 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.114114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91134.log 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105392.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.114114.log.gz/var/log/ceph/ceph-client.admin.63807.log: 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.105392.log.gz 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63807.log.gz 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56849.log 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65966.log 2024-04-25T10:07:40.132 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56849.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91134.log.gz 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80336.log 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56849.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63891.log 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65966.log: /var/log/ceph/ceph-client.admin.80336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106386.log 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63891.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65966.log.gz 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80336.log.gz 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101033.log 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45779.log 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63891.log.gz 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106386.log: /var/log/ceph/ceph-client.admin.101033.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106386.log.gz 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101033.log.gz 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113234.log 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95977.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45779.log.gz 2024-04-25T10:07:40.133 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92115.log 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113234.log.gz 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50124.log 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95977.log.gz 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88343.log 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.92115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96692.log 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.50124.log: 87.8% 0.0% -- replaced with /var/log/ceph/ceph-client.0.50124.log.gz 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92115.log.gz 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42946.log 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9%/var/log/ceph/ceph-client.admin.96692.log: -- replaced with /var/log/ceph/ceph-client.admin.88343.log.gz 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95573.log 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96692.log.gz 2024-04-25T10:07:40.134 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108030.log 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42946.log: /var/log/ceph/ceph-client.admin.95573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63957.log 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108030.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42946.log.gz 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95573.log.gz 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58468.log 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63957.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.108030.log.gz 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59431.log 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63957.log.gz 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106259.log 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58468.log: /var/log/ceph/ceph-client.admin.59431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106031.log 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58468.log.gz 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62677.log 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59431.log.gz 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106259.log.gz 2024-04-25T10:07:40.135 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107127.log 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85369.log 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106031.log.gz 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107127.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74670.log 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62677.log.gz 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107127.log.gz 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45205.log 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85369.log: /var/log/ceph/ceph-client.admin.74670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42736.log 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85369.log.gz 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6%/var/log/ceph/ceph-client.admin.45205.log: -- replaced with /var/log/ceph/ceph-client.admin.74670.log.gz 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79786.log 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8%/var/log/ceph/ceph-client.admin.42736.log: -- replaced with /var/log/ceph/ceph-client.admin.45205.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100854.log 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.136 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42736.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84157.log 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79786.log: /var/log/ceph/ceph-client.admin.100854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104015.log 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3%/var/log/ceph/ceph-client.admin.84157.log: -- replaced with /var/log/ceph/ceph-client.admin.79786.log.gz 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100854.log.gz 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84157.log.gz 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56364.log 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58287.log 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56364.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104015.log.gz 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63301.log 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109152.log 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr: 91.4%/var/log/ceph/ceph-client.admin.63301.log: -- replaced with /var/log/ceph/ceph-client.admin.56364.log.gz/var/log/ceph/ceph-client.admin.58287.log: 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63301.log.gz 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58287.log.gz 2024-04-25T10:07:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64992.log 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83702.log 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64992.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109152.log.gz 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83768.log 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101201.log 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64992.log.gz 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64709.log 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83768.log: /var/log/ceph/ceph-client.admin.101201.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.83702.log.gz 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83768.log.gz 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101201.log.gz 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.107432.log 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64709.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.57957.log 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64709.log.gz 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57129.log 2024-04-25T10:07:40.138 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107432.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58426.log 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57957.log: /var/log/ceph/ceph-client.admin.57129.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57957.log.gz 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57129.log.gz 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103054.log 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60371.log 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103054.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58426.log.gz 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49398.log 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49226.log 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103054.log.gz 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66134.log 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49398.log: /var/log/ceph/ceph-client.admin.49226.log: 91.5% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60371.log.gz -- replaced with /var/log/ceph/ceph-client.admin.49398.log.gz 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49226.log.gz 2024-04-25T10:07:40.139 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45737.log 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110366.log 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45737.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66134.log.gz 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58245.log 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45737.log.gz 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57045.log 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110366.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58245.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110366.log.gz/var/log/ceph/ceph-client.admin.57045.log: 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58245.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57045.log.gz 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47570.log 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47024.log 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42222.log 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47570.log.gz 2024-04-25T10:07:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47024.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47024.log.gz 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76186.log 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113570.log 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42222.log: /var/log/ceph/ceph-client.admin.76186.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42222.log.gz 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76186.log.gz 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88919.log 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113570.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113570.log.gz 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58914.log 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46283.log 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88919.log: /var/log/ceph/ceph-client.admin.58914.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58914.log.gz 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.88919.log.gz 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64389.log 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84419.log 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46283.log: /var/log/ceph/ceph-client.admin.64389.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46283.log.gz 2024-04-25T10:07:40.141 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64389.log.gz 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62238.log 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48542.log 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84419.log: /var/log/ceph/ceph-client.admin.62238.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84419.log.gz 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62238.log.gz 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88019.log 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48542.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48542.log.gz 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75819.log 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100113.log 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88019.log: /var/log/ceph/ceph-client.admin.75819.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75819.log.gz 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.88019.log.gz 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64144.log 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49310.log 2024-04-25T10:07:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100113.log: /var/log/ceph/ceph-client.0.64144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64144.log.gz 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100113.log.gz 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63059.log 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83198.log 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49310.log: /var/log/ceph/ceph-client.admin.63059.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49310.log.gz 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61417.log 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63059.log.gz 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83198.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83198.log.gz 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63206.log 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91218.log 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61417.log: /var/log/ceph/ceph-client.admin.63206.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61417.log.gz 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102784.log 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63206.log.gz 2024-04-25T10:07:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91218.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91218.log.gz 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78899.log 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67128.log 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102784.log: /var/log/ceph/ceph-client.admin.78899.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102784.log.gz 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78899.log.gz 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111312.log 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67128.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67128.log.gz 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75520.log 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102354.log 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111312.log: /var/log/ceph/ceph-client.admin.75520.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.111312.log.gz 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75520.log.gz 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102994.log 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84461.log 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102354.log: /var/log/ceph/ceph-client.admin.102994.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102354.log.gz 2024-04-25T10:07:40.144 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102994.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67170.log 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84461.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84461.log.gz 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80091.log 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99589.log 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67170.log: /var/log/ceph/ceph-client.admin.80091.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67170.log.gz 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.80091.log.gz --verbose -- /var/log/ceph/ceph-client.admin.66363.log 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99589.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99589.log.gz 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80252.log 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40823.log 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66363.log: /var/log/ceph/ceph-client.admin.80252.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66363.log.gz 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80252.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80773.log 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.145 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40823.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40823.log.gz 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66176.log 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42448.log 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80773.log: /var/log/ceph/ceph-client.admin.66176.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80773.log.gz 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66176.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101495.log 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42448.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42448.log.gz 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62425.log 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80379.log 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.101495.log: /var/log/ceph/ceph-client.admin.62425.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62425.log.gz 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.101495.log.gz 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48458.log 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63144.log 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80379.log: /var/log/ceph/ceph-client.admin.48458.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80379.log.gz 2024-04-25T10:07:40.146 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48458.log.gz 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94166.log 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63144.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63144.log.gz 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49937.log 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43661.log 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94166.log: /var/log/ceph/ceph-client.admin.49937.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94166.log.gz 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49937.log.gz 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99799.log 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43661.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43661.log.gz 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112335.log 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111724.log 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99799.log: /var/log/ceph/ceph-client.admin.112335.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99799.log.gz 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112335.log.gz 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102151.log 2024-04-25T10:07:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111724.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.111724.log.gz 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83114.log 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99326.log 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102151.log: /var/log/ceph/ceph-client.admin.83114.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83114.log.gz 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.102151.log.gz 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63513.log 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76632.log 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99326.log: /var/log/ceph/ceph-client.admin.63513.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99326.log.gz 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63513.log.gz 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46982.log 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76632.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76632.log.gz 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74502.log 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95092.log 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46982.log: /var/log/ceph/ceph-client.admin.74502.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46982.log.gz 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74502.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94872.log 2024-04-25T10:07:40.148 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95092.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95092.log.gz 2024-04-25T10:07:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106199.log 2024-04-25T10:07:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43337.log 2024-04-25T10:07:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94872.log: /var/log/ceph/ceph-client.admin.106199.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94872.log.gz 2024-04-25T10:07:40.149 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106199.log.gz 2024-04-25T10:07:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88811.log 2024-04-25T10:07:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43337.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43337.log.gz 2024-04-25T10:07:40.428 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93824.log 2024-04-25T10:07:40.428 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88385.log 2024-04-25T10:07:40.428 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88811.log: /var/log/ceph/ceph-client.admin.93824.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88811.log.gz 2024-04-25T10:07:40.428 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62814.log 2024-04-25T10:07:40.428 INFO:teuthology.orchestra.run.smithi019.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.93824.log.gz/var/log/ceph/ceph-client.admin.88385.log: 2024-04-25T10:07:40.428 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88385.log.gz 2024-04-25T10:07:40.428 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59944.log 2024-04-25T10:07:40.428 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84293.log 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62814.log: /var/log/ceph/ceph-client.admin.59944.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62814.log.gz 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108942.log 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84293.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59944.log.gz 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84293.log.gz 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46787.log 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88853.log 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46787.log: /var/log/ceph/ceph-client.admin.108942.log: 89.0% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46787.log.gz 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108942.log.gz 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89837.log 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105643.log 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89837.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.88853.log.gz 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89837.log.gz 2024-04-25T10:07:40.429 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91083.log 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48888.log 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105643.log: /var/log/ceph/ceph-client.admin.91083.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105643.log.gz 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91083.log.gz 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111172.log 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95615.log 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111172.log: /var/log/ceph/ceph-client.admin.48888.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48888.log.gz 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111172.log.gz 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105821.log 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112629.log 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95615.log: /var/log/ceph/ceph-client.admin.105821.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95615.log.gz 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105821.log.gz 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62372.log 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105685.log 2024-04-25T10:07:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112629.log: /var/log/ceph/ceph-client.admin.62372.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112629.log.gz 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62372.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41585.log 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99283.log 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105685.log: /var/log/ceph/ceph-client.admin.41585.log: 88.3% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105685.log.gz 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41585.log.gz 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83468.log 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46115.log 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99283.log: /var/log/ceph/ceph-client.admin.83468.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99283.log.gz 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83468.log.gz 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41519.log 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63765.log 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41519.log: /var/log/ceph/ceph-client.admin.46115.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46115.log.gz 2024-04-25T10:07:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101411.log 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41519.log.gz 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79025.log 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.101411.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63765.log.gz 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101411.log.gz 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108419.log 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100323.log 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108419.log: /var/log/ceph/ceph-client.admin.79025.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79025.log.gz 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108419.log.gz 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48164.log 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59303.log 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100323.log: /var/log/ceph/ceph-client.admin.48164.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100323.log.gz 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48164.log.gz 2024-04-25T10:07:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114072.log 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95386.log 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59303.log: /var/log/ceph/ceph-client.admin.114072.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114072.log.gz 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59303.log.gz 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46367.log 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90528.log 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95386.log: /var/log/ceph/ceph-client.admin.46367.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95386.log.gz 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46367.log.gz 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47312.log 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44176.log 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47312.log: /var/log/ceph/ceph-client.admin.90528.log: 88.6% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90528.log.gz 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47312.log.gz 2024-04-25T10:07:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44092.log 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76819.log 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44176.log: /var/log/ceph/ceph-client.admin.44092.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44176.log.gz 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44092.log.gz 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108587.log 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49661.log 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.76819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76819.log.gz 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109749.log 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108587.log.gz 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60595.log 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109749.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49661.log.gz 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102658.log 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109749.log.gz 2024-04-25T10:07:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113402.log 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102658.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102658.log.gz 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.60595.log.gz 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44218.log 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91742.log 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113402.log.gz 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79329.log 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44218.log.gz 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84251.log 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.91742.log.gz 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80899.log 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79329.log.gz 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74628.log 2024-04-25T10:07:40.435 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84251.log.gz 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62280.log 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80899.log.gz 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90005.log 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0%/var/log/ceph/ceph-client.admin.62280.log: -- replaced with /var/log/ceph/ceph-client.admin.74628.log.gz 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86057.log 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62280.log.gz 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.90005.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.94312.log 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90005.log.gz 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99504.log 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94312.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94312.log.gz 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.86057.log.gz 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64706.log 2024-04-25T10:07:40.436 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103307.log 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99504.log: /var/log/ceph/ceph-client.admin.64706.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99504.log.gz 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64706.log.gz 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100516.log 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60263.log 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103307.log: /var/log/ceph/ceph-client.admin.100516.log: 87.6% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103307.log.gz 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100516.log.gz 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112731.log 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.60263.log: /var/log/ceph/ceph-client.admin.83156.log 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112731.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.60263.log.gz 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112731.log.gz 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84115.log 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65882.log 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83156.log.gz/var/log/ceph/ceph-client.admin.84115.log: 2024-04-25T10:07:40.437 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96608.log 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65882.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84115.log.gz 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.65882.log.gz 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56807.log 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109800.log 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56807.log: /var/log/ceph/ceph-client.admin.96608.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96608.log.gz 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56807.log.gz 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84377.log 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80689.log/var/log/ceph/ceph-client.admin.109800.log: 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109800.log.gz 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49745.log 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80689.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84377.log.gz 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80689.log.gz 2024-04-25T10:07:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47108.log 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41477.log 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49745.log: /var/log/ceph/ceph-client.admin.47108.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49745.log.gz 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47108.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95489.log 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111410.log 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41477.log: /var/log/ceph/ceph-client.admin.95489.log: 88.5% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95489.log.gz 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41477.log.gz 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47996.log 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43662.log 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111410.log.gz 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46409.log 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43662.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47996.log.gz 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43662.log.gz 2024-04-25T10:07:40.439 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108984.log 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46409.log: -- /var/log/ceph/ceph-client.admin.80941.log 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46409.log.gz 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49787.log 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80941.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108984.log.gz 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80941.log.gz 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100281.log 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96734.log 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49787.log: /var/log/ceph/ceph-client.admin.100281.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49787.log.gz 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100281.log.gz 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107666.log 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56891.log 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96734.log: /var/log/ceph/ceph-client.admin.107666.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96734.log.gz 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107666.log.gz 2024-04-25T10:07:40.440 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111234.log 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110030.log 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111234.log: /var/log/ceph/ceph-client.admin.56891.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56891.log.gz 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111234.log.gz 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99123.log 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66913.log 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110030.log: /var/log/ceph/ceph-client.admin.99123.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110030.log.gz 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105182.log 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0%/var/log/ceph/ceph-client.admin.66913.log: -- replaced with /var/log/ceph/ceph-client.admin.99123.log.gz 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80815.log 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66913.log.gz/var/log/ceph/ceph-client.admin.105182.log: 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43663.log 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.105182.log.gz 2024-04-25T10:07:40.441 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63723.log 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr: 87.2%/var/log/ceph/ceph-client.admin.43663.log: -- replaced with /var/log/ceph/ceph-client.admin.80815.log.gz 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62322.log 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43663.log.gz 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49491.log 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63723.log: /var/log/ceph/ceph-client.admin.62322.log: 88.1% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62322.log.gz 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63723.log.gz 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112293.log 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84587.log 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49491.log.gz 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85411.log 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84587.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112293.log.gz 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84587.log.gz 2024-04-25T10:07:40.442 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64305.log 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.85411.log: -- /var/log/ceph/ceph-client.admin.101159.log 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85411.log.gz 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109619.log 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.101159.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64305.log.gz 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101159.log.gz 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48416.log 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42820.log/var/log/ceph/ceph-client.admin.109619.log: 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109619.log.gz 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48584.log 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42820.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48416.log.gz 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42820.log.gz 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91262.log 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.97029.log 2024-04-25T10:07:40.443 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48584.log.gz 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111874.log 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.97029.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91262.log.gz 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.97029.log.gz 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108233.log 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43334.log 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111874.log: /var/log/ceph/ceph-client.admin.108233.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108233.log.gz 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.111874.log.gz 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62719.log 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94208.log 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43334.log.gz 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110969.log 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94208.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62719.log.gz 2024-04-25T10:07:40.444 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94208.log.gz 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85805.log 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105116.log 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110969.log: /var/log/ceph/ceph-client.admin.85805.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110969.log.gz 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85805.log.gz 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80465.log 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91640.log 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105116.log: /var/log/ceph/ceph-client.admin.80465.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80465.log.gz 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.105116.log.gz 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61050.log 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105060.log 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91640.log.gz 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48930.log 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105060.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.105060.log.gz 2024-04-25T10:07:40.445 INFO:teuthology.orchestra.run.smithi019.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.61050.log.gz 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95783.log 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49184.log 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48930.log: /var/log/ceph/ceph-client.admin.95783.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48930.log.gz 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108839.log 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95783.log.gz 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58680.log 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49184.log: /var/log/ceph/ceph-client.admin.108839.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49184.log.gz 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108839.log.gz 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75945.log 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100905.log 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75945.log: /var/log/ceph/ceph-client.admin.58680.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58680.log.gz 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75945.log.gz 2024-04-25T10:07:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67506.log 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66710.log 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100905.log: /var/log/ceph/ceph-client.admin.67506.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100905.log.gz 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67506.log.gz 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107474.log 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40865.log 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107474.log: /var/log/ceph/ceph-client.admin.66710.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66710.log.gz 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107474.log.gz 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103097.log 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40865.log: -- /var/log/ceph/ceph-client.admin.102574.log 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40865.log.gz 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57171.log 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103097.log.gz 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74932.log 2024-04-25T10:07:40.447 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3%/var/log/ceph/ceph-client.admin.57171.log: -- replaced with /var/log/ceph/ceph-client.admin.102574.log.gz 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78762.log 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57171.log.gz 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74932.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74932.log.gz 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.95176.log 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61640.log 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78762.log: /var/log/ceph/ceph-client.admin.95176.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78762.log.gz 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100724.log 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61640.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95176.log.gz 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40991.log 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61640.log.gz 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100766.log 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40991.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100724.log.gz 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40991.log.gz 2024-04-25T10:07:40.448 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60661.log 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107039.log 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60661.log: /var/log/ceph/ceph-client.admin.100766.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100766.log.gz 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60661.log.gz 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.85496.log 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83510.log 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107039.log: /var/log/ceph/ceph-client.admin.85496.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107039.log.gz 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107964.log 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83510.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85496.log.gz 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83810.log 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83510.log.gz 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107964.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.107964.log.gz 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106428.log 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64886.log 2024-04-25T10:07:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83810.log: /var/log/ceph/ceph-client.admin.106428.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83810.log.gz 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.106428.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61351.log 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102910.log 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61351.log: /var/log/ceph/ceph-client.admin.64886.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64886.log.gz 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58638.log 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61351.log.gz 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102952.log 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58638.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102910.log.gz 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89641.log 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58638.log.gz 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56580.log 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89641.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102952.log.gz 2024-04-25T10:07:40.450 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89641.log.gz 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62196.log 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75657.log 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56580.log: /var/log/ceph/ceph-client.admin.62196.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62196.log.gz 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56580.log.gz 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66092.log 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108317.log 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75657.log.gz 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43339.log 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108317.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66092.log.gz 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108317.log.gz 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66405.log 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108891.log 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43339.log: /var/log/ceph/ceph-client.admin.66405.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43339.log.gz 2024-04-25T10:07:40.451 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66405.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112816.log 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86123.log 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112816.log: /var/log/ceph/ceph-client.admin.108891.log: 88.5% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108891.log.gz 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112816.log.gz 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104311.log 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41181.log 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86123.log: /var/log/ceph/ceph-client.admin.104311.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86123.log.gz 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104311.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65798.log 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40780.log 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41181.log: /var/log/ceph/ceph-client.admin.65798.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65798.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41181.log.gz 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75016.log 2024-04-25T10:07:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99058.log 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40780.log: /var/log/ceph/ceph-client.admin.75016.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40780.log.gz 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75016.log.gz 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57862.log 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66218.log 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57862.log: /var/log/ceph/ceph-client.admin.99058.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57862.log.gz 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49142.log 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99058.log.gz/var/log/ceph/ceph-client.admin.66218.log: 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 88.8% -5 --verbose -- /var/log/ceph/ceph-client.admin.101075.log 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66218.log.gz 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47909.log 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9%/var/log/ceph/ceph-client.admin.101075.log: -- replaced with /var/log/ceph/ceph-client.admin.49142.log.gz 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108503.log 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101075.log.gz 2024-04-25T10:07:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106912.log 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47909.log.gz 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74233.log 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108503.log.gz 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47228.log 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74233.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106912.log.gz 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63639.log 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47228.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74233.log.gz 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47228.log.gz 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90318.log 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88637.log 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63639.log.gz 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46157.log 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88637.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90318.log.gz 2024-04-25T10:07:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43330.log 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.88637.log.gz/var/log/ceph/ceph-client.admin.46157.log: 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63555.log 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46157.log.gz 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42490.log 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63555.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43330.log.gz 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63555.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106344.log 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42490.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42490.log.gz 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.89515.log 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45947.log 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106344.log: /var/log/ceph/ceph-client.admin.89515.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106344.log.gz 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63429.log 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7%/var/log/ceph/ceph-client.admin.45947.log: -- replaced with /var/log/ceph/ceph-client.admin.89515.log.gz 2024-04-25T10:07:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45947.log.gz 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.85931.log 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48332.log 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63429.log: /var/log/ceph/ceph-client.admin.85931.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63429.log.gz 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96338.log 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.85931.log.gz/var/log/ceph/ceph-client.admin.48332.log: 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95699.log 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48332.log.gz 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75436.log 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96338.log: /var/log/ceph/ceph-client.admin.95699.log: 88.7% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96338.log.gz 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95699.log.gz 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93956.log 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63597.log 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75436.log: /var/log/ceph/ceph-client.admin.93956.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75436.log.gz 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93956.log.gz 2024-04-25T10:07:40.456 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61285.log 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47066.log 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63597.log.gz 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78804.log 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47066.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47066.log.gz 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61285.log.gz 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88085.log 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94524.log 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78804.log.gz 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108191.log 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94524.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94524.log.gz 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.88085.log.gz 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99241.log 2024-04-25T10:07:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75226.log 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108191.log: /var/log/ceph/ceph-client.admin.99241.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108191.log.gz 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99241.log.gz 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113780.log 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96422.log 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75226.log: /var/log/ceph/ceph-client.admin.113780.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75226.log.gz 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40738.log 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113780.log.gz 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99926.log 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96422.log: /var/log/ceph/ceph-client.admin.40738.log: 88.7% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40738.log.gz 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96422.log.gz 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93998.log 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88427.log 2024-04-25T10:07:40.458 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99926.log.gz 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.93998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57820.log 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88427.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93998.log.gz 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88427.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75352.log 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65536.log 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75352.log: /var/log/ceph/ceph-client.admin.57820.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57820.log.gz 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75352.log.gz 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56765.log 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44760.log 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65536.log: /var/log/ceph/ceph-client.admin.56765.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65536.log.gz 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56765.log.gz 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107600.log 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76566.log 2024-04-25T10:07:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44760.log: /var/log/ceph/ceph-client.admin.107600.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44760.log.gz 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74586.log 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76566.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.107600.log.gz 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46829.log 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74586.log: 90.9% 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74586.log.gz 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76566.log.gz 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57778.log 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59559.log 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46829.log.gz 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81236.log 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57778.log.gz 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65620.log 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.81236.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81236.log.gz 2024-04-25T10:07:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104185.log 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr: 90.7%/var/log/ceph/ceph-client.admin.65620.log: -- replaced with /var/log/ceph/ceph-client.admin.59559.log.gz 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65620.log.gz 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109194.log 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66531.log 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104185.log.gz/var/log/ceph/ceph-client.admin.109194.log: 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63343.log 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5%/var/log/ceph/ceph-client.admin.66531.log: -- replaced with /var/log/ceph/ceph-client.admin.109194.log.gz 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101117.log 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66531.log.gz 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106115.log 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63343.log: /var/log/ceph/ceph-client.admin.101117.log: 89.2% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63343.log.gz 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101117.log.gz 2024-04-25T10:07:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104563.log 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.106115.log: -- /var/log/ceph/ceph-client.admin.46940.log 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106115.log.gz 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45989.log 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46940.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.104563.log.gz 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46940.log.gz 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79996.log 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104353.log 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45989.log.gz 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113528.log 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104353.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79996.log.gz 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104353.log.gz 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46451.log 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98767.log 2024-04-25T10:07:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113528.log: /var/log/ceph/ceph-client.admin.46451.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113528.log.gz 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94396.log 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6%/var/log/ceph/ceph-client.admin.98767.log: -- replaced with /var/log/ceph/ceph-client.admin.46451.log.gz 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66955.log 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98767.log.gz 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62154.log 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94396.log: /var/log/ceph/ceph-client.admin.66955.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66955.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94396.log.gz 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89599.log 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62028.log 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62154.log: /var/log/ceph/ceph-client.admin.89599.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62154.log.gz 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89599.log.gz 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99841.log 2024-04-25T10:07:40.463 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66615.log 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62028.log: /var/log/ceph/ceph-client.admin.99841.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62028.log.gz 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99841.log.gz 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102396.log 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47738.log 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102396.log: /var/log/ceph/ceph-client.admin.66615.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102396.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66615.log.gz 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61986.log 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61153.log 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47738.log: /var/log/ceph/ceph-client.admin.61986.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47738.log.gz 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57213.log 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61986.log.gz 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34903.log 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57213.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57213.log.gz 2024-04-25T10:07:40.464 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61153.log.gz 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101453.log 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105947.log 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.101453.log: /var/log/ceph/ceph-client.admin.34903.log: 88.5% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101453.log.gz 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34903.log.gz 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65287.log 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89210.log 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105947.log: /var/log/ceph/ceph-client.0.65287.log: 0.0% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105947.log.gz 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.0.65287.log.gz 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91598.log 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43659.log 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89210.log.gz 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49533.log 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3%/var/log/ceph/ceph-client.admin.43659.log: -- replaced with /var/log/ceph/ceph-client.admin.91598.log.gz 2024-04-25T10:07:40.465 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43659.log.gz 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84986.log 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78696.log 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49533.log.gz 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84986.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84986.log.gz 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65840.log 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104101.log 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65840.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.78696.log.gz 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65840.log.gz 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42658.log 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104101.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104101.log.gz 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44512.log 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42658.log: -- /var/log/ceph/ceph-client.admin.109411.log 2024-04-25T10:07:40.466 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42658.log.gz 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63250.log 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109411.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44512.log.gz 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.109411.log.gz 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94966.log 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111658.log 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63250.log.gz 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112461.log 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111658.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94966.log.gz 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66008.log 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.111658.log.gz 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63471.log 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9%/var/log/ceph/ceph-client.admin.66008.log: -- replaced with /var/log/ceph/ceph-client.admin.112461.log.gz 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49619.log 2024-04-25T10:07:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63471.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66008.log.gz 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63471.log.gz 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63385.log 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66321.log 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63385.log: /var/log/ceph/ceph-client.admin.49619.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63385.log.gz 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49619.log.gz 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107390.log 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66321.log: gzip -5 --verbose 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66321.log.gz 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.60727.log 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82915.log 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107390.log: /var/log/ceph/ceph-client.admin.60727.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107390.log.gz 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105989.log 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82915.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60727.log.gz 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.82915.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109705.log 2024-04-25T10:07:40.468 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108149.log 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.105989.log: /var/log/ceph/ceph-client.admin.109705.log: 88.2% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105989.log.gz 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109705.log.gz 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76440.log 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47696.log 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108149.log: /var/log/ceph/ceph-client.admin.76440.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108149.log.gz 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76440.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67464.log 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76398.log 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67464.log: /var/log/ceph/ceph-client.admin.47696.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67464.log.gz 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47696.log.gz 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91176.log 2024-04-25T10:07:40.469 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94250.log 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76398.log.gz 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56496.log 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94250.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91176.log.gz 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94250.log.gz 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47396.log 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75903.log 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56496.log: /var/log/ceph/ceph-client.admin.47396.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47396.log.gz 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56496.log.gz 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90869.log 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89557.log 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75903.log.gz 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109026.log 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89557.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90869.log.gz 2024-04-25T10:07:40.470 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89557.log.gz 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80171.log 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108755.log 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80171.log: /var/log/ceph/ceph-client.admin.109026.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.109026.log.gz 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80171.log.gz 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74890.log 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65704.log 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108755.log: /var/log/ceph/ceph-client.admin.74890.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.108755.log.gz 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74890.log.gzgzip 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.89336.log 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67296.log 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89336.log: /var/log/ceph/ceph-client.admin.65704.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65704.log.gz 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89336.log.gz 2024-04-25T10:07:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43030.log 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42904.log 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67296.log: /var/log/ceph/ceph-client.admin.43030.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67296.log.gz 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43030.log.gz 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104227.log 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43072.log 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42904.log: /var/log/ceph/ceph-client.admin.104227.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42904.log.gz 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104227.log.gz 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62467.log 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94609.log 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43072.log: /var/log/ceph/ceph-client.admin.62467.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43072.log.gz 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81049.log 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62467.log.gz 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62070.log 2024-04-25T10:07:40.472 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94609.log: /var/log/ceph/ceph-client.admin.81049.log: 88.2% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94609.log.gz 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81049.log.gz 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109534.log 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62070.log: /var/log/ceph/ceph-mds.a.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62070.log.gz 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103584.log 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109534.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109534.log.gz 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79161.log 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103584.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103584.log.gz 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99547.log 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79161.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79161.log.gz 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46199.log 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99547.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99547.log.gz 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113612.log 2024-04-25T10:07:40.473 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46199.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46199.log.gz 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107853.log 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113612.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113612.log.gz 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88511.log 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.107853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.107853.log.gz 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96524.log 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88511.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88511.log.gz 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90444.log 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96524.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96524.log.gz 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100640.log 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90444.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90444.log.gz 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41990.log 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100640.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100640.log.gz 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94040.log 2024-04-25T10:07:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41990.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.41990.log.gz 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108629.log 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.94040.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94040.log.gz 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114032.log 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.108629.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108629.log.gz 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109331.log 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.114032.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114032.log.gz 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45695.log 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109331.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109331.log.gz 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76031.log 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45695.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45695.log.gz 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85327.log 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76031.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76031.log.gz 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113318.log 2024-04-25T10:07:40.475 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85327.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85327.log.gz 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100989.log 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113318.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113318.log.gz 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59369.log 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100989.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100989.log.gz 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44134.log 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59369.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59369.log.gz 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102085.log 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44134.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44134.log.gz 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110927.log 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102085.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.102085.log.gz 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58554.log 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.110927.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110927.log.gz 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43881.log 2024-04-25T10:07:40.476 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58554.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58554.log.gz 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47953.log 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43881.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43881.log.gz 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104395.log 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47953.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47953.log.gz 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66489.log 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104395.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.104395.log.gz 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103792.log 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66489.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66489.log.gz 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83240.log 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103792.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103792.log.gz 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83989.log 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83240.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83240.log.gz 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113906.log 2024-04-25T10:07:40.477 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83989.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83989.log.gz 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95008.log 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113906.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.113906.log.gz 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47354.log 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.95008.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95008.log.gz 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49577.log 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47354.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47354.log.gz 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113738.log 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49577.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49577.log.gz 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102217.log 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113738.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113738.log.gz 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41097.log 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102217.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102217.log.gz 2024-04-25T10:07:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84774.log 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41097.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41097.log.gz 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42348.log 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84774.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84774.log.gz 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109492.log 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42348.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42348.log.gz 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91430.log 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109492.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109492.log.gz 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67422.log 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91430.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91430.log.gz 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75268.log 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67422.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67422.log.gz 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74383.log 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75268.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75268.log.gz 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100436.log 2024-04-25T10:07:40.479 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74383.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74383.log.gz 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43114.log 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100436.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100436.log.gz 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47654.log 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43114.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43114.log.gz 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35006.log 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47654.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47654.log.gz 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106997.log 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.35006.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35006.log.gz 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44470.log 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106997.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.106997.log.gz 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111011.log 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44470.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44470.log.gz 2024-04-25T10:07:40.480 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96464.log 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111011.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111011.log.gz 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100947.log 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96464.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96464.log.gz 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62894.log 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100947.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.100947.log.gz 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60971.log 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62894.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62894.log.gz 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75058.log 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60971.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60971.log.gz 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112090.log 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75058.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75058.log.gz 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104629.log 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112090.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112090.log.gz 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91388.log 2024-04-25T10:07:40.481 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104629.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104629.log.gz 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101561.log 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91388.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91388.log.gz 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63681.log 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.101561.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101561.log.gz 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96380.log 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63681.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63681.log.gz 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48710.log 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96380.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96380.log.gz 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40657.log 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48710.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48710.log.gz 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100197.log 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40657.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40657.log.gz 2024-04-25T10:07:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91472.log 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100197.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100197.log.gz 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84859.log 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91472.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91472.log.gz 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66871.log 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84859.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84859.log.gz 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42306.log 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66871.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66871.log.gz 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74317.log 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42306.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42306.log.gz 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.91995.log 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74317.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74317.log.gz 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76228.log 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.91995.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.91995.log.gz 2024-04-25T10:07:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42264.log 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76228.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76228.log.gz 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78498.log 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42264.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42264.log.gz 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44260.log 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78498.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.78498.log.gz 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109576.log 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44260.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44260.log.gz 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83594.log 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.109576.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109576.log.gz 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99715.log 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83594.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83594.log.gz 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111464.log 2024-04-25T10:07:40.484 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99715.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99715.log.gz 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43684.log 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111464.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111464.log.gz 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67084.log 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43684.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43684.log.gz 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80857.log 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67084.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67084.log.gz 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83048.log 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80857.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80857.log.gz 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74806.log 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83048.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.83048.log.gz 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64473.log 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74806.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74806.log.gz 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99757.log 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64473.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64473.log.gz 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64347.log 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99757.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99757.log.gz 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112251.log 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64347.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64347.log.gz 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89795.log 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112251.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112251.log.gz 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64708.log 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89795.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89795.log.gz 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83282.log 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64708.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64708.log.gz 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104057.log 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83282.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83282.log.gz 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113864.log 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.104057.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104057.log.gz 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96296.log 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.113864.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113864.log.gz 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46535.log 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96296.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96296.log.gz 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79684.log 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46535.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46535.log.gz 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85763.log 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79684.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79684.log.gz 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28582.log 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85763.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85763.log.gz 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74848.log 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.tmp-client.admin.28582.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28582.log.gz 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79870.log 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74848.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74848.log.gz 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44554.log 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79870.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79870.log.gz 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48206.log 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44554.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44554.log.gz 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66790.log 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48206.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48206.log.gz 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96082.log 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66790.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66790.log.gz 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101934.log 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.96082.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.96082.log.gz 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65924.log 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.101934.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.101934.log.gz 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75310.log 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65924.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65924.log.gz 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103834.log 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75310.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75310.log.gz 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107262.log 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.103834.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103834.log.gz 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99673.log 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.107262.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107262.log.gz 2024-04-25T10:07:40.490 INFO:teuthology.orchestra.run.smithi112.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:07:40.490 INFO:teuthology.orchestra.run.smithi112.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:07:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47817.log 2024-04-25T10:07:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.99673.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99673.log.gz 2024-04-25T10:07:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57423.log 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47817.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47817.log.gz 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82982.log 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57423.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57423.log.gz 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41309.log 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82982.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.82982.log.gz 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86337.log 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41309.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41309.log.gz 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74974.log 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.86337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.86337.log.gz 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100598.log 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74974.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74974.log.gz 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49058.log 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.100598.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100598.log.gz 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48122.log 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49058.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49058.log.gz 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85454.log 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48122.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48122.log.gz 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111550.log 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85454.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85454.log.gz 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102826.log 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.111550.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111550.log.gz 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89921.log 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.102826.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102826.log.gz 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112024.log 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89921.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89921.log.gz 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98862.log 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.112024.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.112024.log.gz 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44302.log 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.98862.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.98862.log.gz 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58596.log 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44302.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44302.log.gz 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41035.log 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58596.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58596.log.gz 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56430.log 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41035.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41035.log.gz 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56646.log 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56430.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56430.log.gz 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80645.log 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56646.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56646.log.gz 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80731.log 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80645.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80645.log.gz 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90785.log 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80731.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80731.log.gz 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62593.log 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90785.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90785.log.gz 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106073.log 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62593.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62593.log.gz 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.106073.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106073.log.gz 2024-04-25T10:07:41.037 INFO:teuthology.orchestra.run.smithi112.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:07:41.351 INFO:teuthology.orchestra.run.smithi019.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:07:42.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-25T10:07:42.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-25T10:07:42.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-25T10:07:42.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-04-25T10:07:42.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-04-25T10:07:42.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-25T10:07:44.223 INFO:teuthology.orchestra.run.smithi112.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:07:44.385 INFO:teuthology.orchestra.run.smithi112.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:07:45.411 INFO:teuthology.orchestra.run.smithi112.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:07:45.465 INFO:teuthology.orchestra.run.smithi019.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:07:45.942 INFO:teuthology.orchestra.run.smithi112.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:07:45.997 INFO:teuthology.orchestra.run.smithi019.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:07:46.378 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:07:46.418 INFO:teuthology.orchestra.run.smithi019.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:07:48.167 INFO:teuthology.orchestra.run.smithi019.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:07:58.850 INFO:teuthology.orchestra.run.smithi112.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:07:58.852 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T10:07:58.852 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m19.002s 2024-04-25T10:07:58.852 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m40.577s 2024-04-25T10:07:58.852 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m1.232s 2024-04-25T10:08:02.175 INFO:teuthology.orchestra.run.smithi019.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:08:02.177 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T10:08:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m22.349s 2024-04-25T10:08:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m54.946s 2024-04-25T10:08:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m2.044s 2024-04-25T10:08:02.177 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:08:02.177 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033/remote/smithi019/log 2024-04-25T10:08:02.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:08:10.087 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033/remote/smithi112/log 2024-04-25T10:08:10.088 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:08:15.676 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:08:15.699 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:08:15.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:08:15.702 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:08:15.780 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:08:15.789 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:08:15.789 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:08:15.792 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:08:15.810 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:08:15.810 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:08:15.811 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:08:15.811 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-25T10:08:15.811 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv01.front.sepia.ceph.com 3 7 377 89 +1343us[+1354us] +/- 66ms 2024-04-25T10:08:15.811 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 -1146us[-1146us] +/- 62ms 2024-04-25T10:08:15.811 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -225us[ -225us] +/- 83ms 2024-04-25T10:08:15.811 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:08:15.813 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:08:15.813 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T10:08:15.813 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 24 +1428us[+1435us] +/- 67ms 2024-04-25T10:08:15.813 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -1101us[-1101us] +/- 62ms 2024-04-25T10:08:15.813 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -177us[ -177us] +/- 83ms 2024-04-25T10:08:15.813 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:08:15.814 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:08:15.825 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:08:15.825 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:08:15.839 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:08:15.957 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:08:16.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:08:16.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:08:16.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:08:16.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:08:16.078 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:08:16.088 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:08:16.131 INFO:teuthology.task.internal:Duration was 1277.679386 seconds 2024-04-25T10:08:16.132 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:08:16.169 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:08:16.170 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:08:16.172 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:08:16.209 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:08:16.209 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:08:16.448 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:08:16.448 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T10:08:16.449 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:08:16.469 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T10:08:16.469 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:08:16.507 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:08:16.508 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:08:16.511 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:08:16.572 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:08:16.572 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:08:16.577 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:08:16.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:08:16.837 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:08:16.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:08:16.867 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:08:16.896 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:08:16.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:08:16.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:08:16.940 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-04-25T10:08:16.959 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-04-25T10:08:16.972 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:08:17.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:08:17.013 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:08:17.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:08:17.028 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:08:17.041 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:08:17.041 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033/remote/smithi019 2024-04-25T10:08:17.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:08:17.090 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673033/remote/smithi112 2024-04-25T10:08:17.091 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:08:17.122 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:08:17.123 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:08:17.127 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:08:17.172 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:08:17.182 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:08:17.182 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:08:17.254 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:08:17.254 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:08:17.257 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:08:17.269 INFO:teuthology.orchestra.run.smithi019.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:08 /home/ubuntu/cephtest 2024-04-25T10:08:17.271 INFO:teuthology.orchestra.run.smithi112.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:08 /home/ubuntu/cephtest 2024-04-25T10:08:17.272 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:08:17.332 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:08:17.441 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} duration: 1277.6793856620789 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:08:17.441 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:08:17.521 INFO:teuthology.run:pass